[cut: only the first 33554432 bytes of this file were read]
<?xml version="1.0" encoding="utf-8"?>
<posts>
  <row Id="1" PostTypeId="1" AcceptedAnswerId="3" CreationDate="2016-08-02T15:39:14.947" Score="8" ViewCount="574" Body="&lt;p&gt;What does &quot;backprop&quot; mean? Is the &quot;backprop&quot; term basically the same as &quot;backpropagation&quot; or does it have a different meaning?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-11-16T17:56:22.093" LastActivityDate="2020-04-22T00:49:46.747" Title="What is &quot;backprop&quot;?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2" PostTypeId="1" AcceptedAnswerId="9" CreationDate="2016-08-02T15:40:20.623" Score="12" ViewCount="738" Body="&lt;p&gt;Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization process overall?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-02-23T22:36:19.090" LastActivityDate="2019-02-23T22:36:37.133" Title="How does noise affect generalization?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;statistical-ai&gt;&lt;generalization&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3" PostTypeId="2" ParentId="1" CreationDate="2016-08-02T15:40:24.820" Score="13" Body="&lt;p&gt;&quot;Backprop&quot; is the same as &quot;backpropagation&quot;: it's just a shorter way to say it. It is sometimes abbreviated as &quot;BP&quot;.&lt;/p&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-08-02T15:40:24.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4" PostTypeId="1" AcceptedAnswerId="12" CreationDate="2016-08-02T15:41:22.020" Score="32" ViewCount="1061" Body="&lt;p&gt;When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-19T23:54:07.813" LastActivityDate="2021-01-19T23:54:07.813" Title="How to find the optimal number of neurons per layer?" Tags="&lt;neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;artificial-neuron&gt;&lt;hyper-parameters&gt;&lt;layers&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="6" PostTypeId="1" AcceptedAnswerId="20" CreationDate="2016-08-02T15:43:35.460" Score="7" ViewCount="240" Body="&lt;p&gt;Given the following definition of an intelligent agent (taken from a &lt;a href=&quot;http://en.wikipedia.org/wiki/Philosophy_of_artificial_intelligence#Intelligent_agent_definition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If an agent acts so as to maximize the expected value of a performance measure based on past experience and knowledge then it is intelligent&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and given that we, humans, all make mistakes, which means that we are not maximizing the expected value of a performance measure, then does this imply that humans are not intelligent? &lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2019-06-15T18:25:58.513" LastActivityDate="2019-06-15T18:29:55.520" Title="Are humans intelligent according to the definition of an intelligent agent?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;intelligent-agent&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7" PostTypeId="1" CreationDate="2016-08-02T15:45:09.070" Score="10" ViewCount="522" Body="&lt;p&gt;This &lt;a href=&quot;https://www.independent.co.uk/life-style/gadgets-and-tech/news/stephen-hawking-artificial-intelligence-could-wipe-out-humanity-when-it-gets-too-clever-humans-could-become-ants-being-stepped-a6686496.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;quote by Stephen Hawking&lt;/a&gt; has been in headlines for quite some time:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Artificial Intelligence could wipe out humanity when it gets too clever as humans will be like ants.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why does he say this? To put it simply: what are the possible threats from AI (that Stephen Hawking is worried about)? If we know that AI is so dangerous, why are we still promoting it? Why is it not banned?&lt;/p&gt;&#xA;&lt;p&gt;What are the adverse consequences of the so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technological Singularity&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="26" LastEditorUserId="2444" LastEditDate="2021-01-20T00:00:31.027" LastActivityDate="2021-01-20T00:00:31.027" Title="Why does Stephen Hawking say &quot;Artificial Intelligence will kill us all&quot;?" Tags="&lt;agi&gt;&lt;superintelligence&gt;&lt;singularity&gt;&lt;ai-safety&gt;&lt;ai-takeover&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="1" ClosedDate="2016-08-04T01:36:40.283" ContentLicense="CC BY-SA 4.0" />
  <row Id="9" PostTypeId="2" ParentId="2" CreationDate="2016-08-02T15:47:02.993" Score="9" Body="&lt;p&gt;Noise in the data, to a reasonable amount, may help the network to generalize better. Sometimes, it has the opposite effect. It partly depends on the kind of noise (&quot;true&quot; vs. artificial).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;ftp://ftp.sas.com/pub/neural/FAQ3.html#A_noise&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI FAQ on ANN&lt;/a&gt; gives a good overview. Excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Noise in the actual data is never a good thing, since it limits the accuracy of generalization that can be achieved no matter how extensive the training set is. On the other hand, injecting artificial noise (jitter) into the inputs during training is one of several ways to improve generalization for smooth functions when you have a small training set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In some field, such as computer vision, it's common to increase the size of the training set by copying some samples and adding some noises or other transformation.&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="2444" LastEditDate="2019-02-23T22:36:37.133" LastActivityDate="2019-02-23T22:36:37.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10" PostTypeId="1" AcceptedAnswerId="32" CreationDate="2016-08-02T15:47:56.593" Score="45" ViewCount="2187" Body="&lt;p&gt;I'm new to A.I. and I'd like to know in simple words, what is the fuzzy logic concept? How does it help, and when is it used?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="10135" LastEditDate="2018-10-18T10:44:33.687" LastActivityDate="2021-03-31T22:24:53.703" Title="What is fuzzy logic?" Tags="&lt;deep-neural-networks&gt;&lt;terminology&gt;&lt;fuzzy-logic&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="18" ContentLicense="CC BY-SA 3.0" />
  <row Id="11" PostTypeId="2" ParentId="2" CreationDate="2016-08-02T15:48:56.970" Score="9" Body="&lt;p&gt;We typically think of machine learning models as modeling two different parts of the training data--the underlying generalizable truth (the signal), and the randomness specific to that dataset (the noise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fitting both of those parts increases training set accuracy, but fitting the signal also increases test set accuracy (and real-world performance) while fitting the noise decreases both. So we use things like regularization and dropout and similar techniques in order to make it harder to fit the noise, and so more likely to fit the signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just increasing the amount of noise in the training data is one such approach, but seems unlikely to be as useful. Compare random jitter to adversarial boosting, for example; the first will slowly and indirectly improve robustness whereas the latter will dramatically and directly improve it.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T15:48:56.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12" PostTypeId="2" ParentId="4" CreationDate="2016-08-02T15:50:27.867" Score="18" Body="&lt;p&gt;There is no direct way to find the optimal number of them: people empirically try and see (e.g., using cross-validation). The most common search techniques are random, manual, and grid searches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There exist more advanced techniques such as Gaussian processes, e.g. &lt;em&gt;&lt;a href=&quot;http://arxiv.org/abs/1609.08703&quot; rel=&quot;noreferrer&quot;&gt;Optimizing Neural Network Hyperparameters with Gaussian Processes for Dialog Act Classification&lt;/a&gt;, IEEE SLT 2016&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="4" LastEditDate="2016-09-29T00:24:06.177" LastActivityDate="2016-09-29T00:24:06.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13" PostTypeId="1" AcceptedAnswerId="163" CreationDate="2016-08-02T15:52:19.413" Score="9" ViewCount="143" Body="&lt;p&gt;In particular, an embedded computer (with limited resources) analyzes live video stream from a traffic camera, trying to pick good frames that contain license plate numbers of passing cars. Once a plate is located, the frame is handed over to an OCR library to extract the registration and use it further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my country two types of license plates are in common use - rectangular (the typical) and square - actually, somewhat rectangular but &quot;higher than wider&quot;, with the registration split over two rows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(there are some more types, but let us disregard them; they are a small percent and usually belong to vehicles that lie outside our interest.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the limited resources and need for rapid, real-time processing, the maximum size of the network (number of cells and connections) the system can handle is fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be better to split this into two smaller networks, each recognizing one type of registration plates, or will the larger single network handle the two types better?&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="14723" LastEditDate="2018-04-12T02:30:42.823" LastActivityDate="2018-04-12T02:30:42.823" Title="Can a single neural network handle recognizing two types of objects, or should it be split into two smaller networks?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15" PostTypeId="1" CreationDate="2016-08-02T15:52:50.827" Score="37" ViewCount="3316" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot;&gt;Turing Test&lt;/a&gt; was the first test of artificial intelligence and is now a bit outdated. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test#Total_Turing_test&quot;&gt;Total Turing Test&lt;/a&gt; aims to be a more modern test which requires a much more sophisticated system. What techniques can we use to identify an artificial intelligence (weak AI) and an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot;&gt;artificial general intelligence&lt;/a&gt; (strong AI)?&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="95" LastEditDate="2016-08-04T14:10:10.990" LastActivityDate="2018-04-11T20:16:35.997" Title="Is the Turing Test, or any of its variants, a reliable test of artificial intelligence?" Tags="&lt;turing-test&gt;&lt;agi&gt;&lt;intelligent-agent&gt;&lt;weak-ai&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="16" PostTypeId="1" AcceptedAnswerId="142" CreationDate="2016-08-02T15:53:00.447" Score="8" ViewCount="540" Body="&lt;p&gt;What is &lt;a href=&quot;https://en.wikipedia.org/wiki/Early_stopping&quot; rel=&quot;nofollow noreferrer&quot;&gt;early stopping&lt;/a&gt; in machine learning and, in general, artificial intelligence? What are the advantages of using this method? How does it help exactly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be interested in perspectives and links to recent research.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-10-11T22:28:23.457" LastActivityDate="2019-10-11T22:28:23.457" Title="What is &quot;early stopping&quot; in machine learning?" Tags="&lt;deep-learning&gt;&lt;definitions&gt;&lt;overfitting&gt;&lt;regularization&gt;&lt;early-stopping&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17" PostTypeId="1" AcceptedAnswerId="45" CreationDate="2016-08-02T15:53:38.273" Score="37" ViewCount="1416" Body="&lt;p&gt;I've heard the idea of the technological singularity, what is it and how does it relate to Artificial Intelligence? Is this the theoretical point where Artificial Intelligence machines have progressed to the point where they grow and learn on their own beyond what humans can do and their growth takes off?  How would we know when we reach this point?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2019-09-16T16:23:39.853" LastActivityDate="2020-11-17T13:21:19.530" Title="What is the concept of the technological singularity?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;agi&gt;&lt;superintelligence&gt;&lt;singularity&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="18" PostTypeId="2" ParentId="7" CreationDate="2016-08-02T15:54:26.937" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;To put it simply in layman terms, what are the possible threats from AI? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Currently, there are no threat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The threat comes if humans create a so-called ultraintelligent machine, a machine that can surpass all intellectual activities by any human. This would be the last invention man would need to do, since this machine is better in inventing machines than humans are (since that is an intellectual activity).  However, this could cause the machine to invent machines that can destruct humans, and we can't stop them because they are so much smarter than we are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all hypothetical, no one has even a clue of what an ultraintelligent machine looks like. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If we know that AI is so dangerous why are we still promoting it? Why is it not banned?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As I said before, the existence of a ultraintelligent machine is hypothetical. Artificial Intelligence has lots of useful applications (more than this answer can contain), and if we develop it, we get even more useful applications. We just have to be careful that the machines won't overtake us. &lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2016-08-02T15:54:26.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19" PostTypeId="2" ParentId="7" CreationDate="2016-08-02T15:54:29.263" Score="4" Body="&lt;p&gt;Because he did not yet know how far away current AI is... Working in an media AI lab, I get this question a lot. But really... we are still a long way from this. The robots still do everything we detailledly describe them to do. Instead of seeing the robot as intelligent, I would look to the human programmer for where the creativity really happens.&lt;/p&gt;&#xA;" OwnerUserId="52" LastActivityDate="2016-08-02T15:54:29.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20" PostTypeId="2" ParentId="6" CreationDate="2016-08-02T15:54:45.237" Score="2" Body="&lt;p&gt;It rather depends on how one defines several of the terms used. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Whether the term &quot;expected&quot; is interpreted in a formal (i.e.&#xA;statistical) sense.  &lt;/li&gt;&#xA;&lt;li&gt;Whether it's assumed that humans have any kind of utilitarian&#xA;&quot;performance measure&quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The motivation for this description of &quot;agent&quot; arose from a desire to have a quantitative model - it's not clear that such a model is a good fit for human cognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are alternative definitions of agents, for example the &lt;a href=&quot;https://en.wikipedia.org/wiki/Belief%E2%80%93desire%E2%80%93intention_software_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;BDI model&lt;/a&gt;, which are rather more open-ended and hence more obviously applicable to humans.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-06-15T18:29:55.520" LastActivityDate="2019-06-15T18:29:55.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21" PostTypeId="1" CreationDate="2016-08-02T15:55:15.957" Score="5" ViewCount="85" Body="&lt;p&gt;I'm worrying that my neural network has become too complex. I don't want to end up with half of the neural network doing nothing but just take up space and resources.&lt;/p&gt;&#xA;&lt;p&gt;So, what are the techniques for detecting and preventing overfitting, to avoid such problems?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-10T01:01:09.583" LastActivityDate="2021-01-10T01:03:39.270" Title="What are the techniques for detecting and preventing overfitting?" Tags="&lt;reference-request&gt;&lt;optimization&gt;&lt;deep-neural-networks&gt;&lt;overfitting&gt;&lt;generalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22" PostTypeId="2" ParentId="7" CreationDate="2016-08-02T15:56:10.167" Score="5" Body="&lt;p&gt;It's not just Hawking, you hear variations on this refrain from a lot of people.  And given that they're mostly very smart, well educated, well informed people (Elon Musk is another, for example), it probably shouldn't be dismissed out of hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the basic idea seems to be this: If we create &quot;real&quot; artificial intelligence, at some point, it will be able to improve itself, which improves it's ability to improve itself, which means it can improve it's ability to improve itself even more, and so on... a runaway cascade leading to &quot;superhuman intelligence&quot;.  That is to say, leading to something that more intelligent than we area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what happens if there is an entity on this planet which is literally more intelligent than us (humans)? Would it be a threat to us?  Well, it certainly seems reasonable to speculate that it &lt;em&gt;could&lt;/em&gt; be so.   OTOH, we have no particular reason, right now, to think that it &lt;em&gt;will&lt;/em&gt; be so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it seems that Hawking, Musk, etc. are just coming down on the more cautious / fearful side of things.  Since we don't &lt;em&gt;know&lt;/em&gt; if a superhuman AI will be dangerous or not, and given that it could be unstoppable if it were to become malicious (remember, it's smarter than we are!), it's a reasonable thing to take under consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eliezer Yudkowsky has also written quite a bit on this subject, including come up with the famous &quot;AI Box&quot; experiment.  I think anybody interested in this topic should read some of his material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.yudkowsky.net/singularity/aibox/&quot; rel=&quot;noreferrer&quot;&gt;http://www.yudkowsky.net/singularity/aibox/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-02T15:56:10.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23" PostTypeId="2" ParentId="7" CreationDate="2016-08-02T15:57:19.303" Score="4" Body="&lt;p&gt;As Andrew Ng &lt;a href=&quot;http://www.theregister.co.uk/2015/03/19/andrew_ng_baidu_ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;said&lt;/a&gt;, worrying about such threat from AI is like worrying about of overpopulation on Mars. It is science fiction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m6jnl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m6jnl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, given the rise of (much weaker) robots and other (semi-)autonomous agents, the fields of the law and ethics are increasingly incorporating them, e.g. see &lt;a href=&quot;https://en.wikipedia.org/wiki/Roboethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roboethics&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-08-02T15:57:19.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24" PostTypeId="2" ParentId="7" CreationDate="2016-08-02T15:57:48.363" Score="2" Body="&lt;p&gt;He says this because it can happen. If something becomes smarter than us, why would it continue to serve us? The worst case scenario is that it takes over all manufacturing processes and consumes all matter to convert it into material capable of computation, extending outward infinitely until all matter is consumed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that AI is dangerous but it doesn't matter because most people don't believe in it. It goes against every comfort religion has to offer. Man is the end-all-be-all of the universe and if that fact is disputed, people will feel out of place and purposeless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact is most people just don't acknowledge it's possible, or that it will happen in our lifetimes, even though many reputable AI experts put the occurrence of the singularity within two decades. If people truly acknowledged that AI that was smarter than them was possible, wouldn't they be living differently? Wouldn't they be looking to do things that they enjoy, knowing that whatever it is they do that they dread will be automated? Wouldn't everyone be calling for a universal basic income?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other reason we don't ban it is because its promise is so great. One researcher could be augmented by 1,000 digital research assistants. All manual labor could be automated. For the first time, technology offers us real freedom to do whatever we please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even in this best case scenario where it doesn't overtake us, humans still have to adapt and alter their economic system to one where labor isn't necessary. Otherwise, those who aren't technically-trained will starve and revolt.&lt;/p&gt;&#xA;" OwnerUserId="56" LastEditorUserId="56" LastEditDate="2016-08-02T16:46:21.237" LastActivityDate="2016-08-02T16:46:21.237" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25" PostTypeId="2" ParentId="7" CreationDate="2016-08-02T15:58:13.970" Score="3" Body="&lt;p&gt;There are a number of long resources to answer this sort of question: consider Stuart Armstrong's book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/B00IB4N4KU&quot; rel=&quot;nofollow&quot;&gt;Smarter Than Us&lt;/a&gt;, Nick Bostrom's book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/B00LOOCGB2&quot; rel=&quot;nofollow&quot;&gt;Superintelligence&lt;/a&gt;, which grew out of this &lt;a href=&quot;http://www.nickbostrom.com/views/superintelligence.pdf&quot; rel=&quot;nofollow&quot;&gt;edge.org answer&lt;/a&gt;, &lt;a href=&quot;http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html&quot; rel=&quot;nofollow&quot;&gt;Tim Urban's explanation&lt;/a&gt;, or &lt;a href=&quot;https://aisafety.wordpress.com/&quot; rel=&quot;nofollow&quot;&gt;Michael Cohen's explanation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here's my (somewhat shorter) answer: intelligence is all about decision-making, and we don't have any reason to believe that humans are anywhere near close to being the best possible at decision-making. Once we are able to build an AI AI researcher (that is, a computer that knows how to make computers better at thinking), the economic and military relevance of humans will rapidly disappear as any decision that could be made by a human could be made better by a computer. (Why have human generals instead of robot generals, human engineers instead of robot engineers, and so on.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't necessarily a catastrophe. If the Vulcans showed up tomorrow and brought better decision-making to Earth, we could avoid a lot of misery. The hard part is making sure that what we get are Vulcans who want us around and happy, instead of something that doesn't share our values.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T15:58:13.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26" PostTypeId="1" AcceptedAnswerId="189" CreationDate="2016-08-02T15:58:31.413" Score="23" ViewCount="1367" Body="&lt;p&gt;I've seen emotional intelligence defined as the capacity to be aware of, control, and express one's emotions, and to handle interpersonal relationships judiciously and empathetically.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are some strategies for artificial intelligence to begin to tackle this problem and develop emotional intelligence for computers?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there examples where this is already happening to a degree today?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Wouldn't a computer that passes a Turing test necessarily express emotional intelligence or it would be seen as an obvious computer?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps that is why early programs that pass the test represented young people, who presumably have lower emotional intelligence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2020-03-08T03:17:30.877" LastActivityDate="2021-02-01T01:03:23.217" Title="How could emotional intelligence be implemented?" Tags="&lt;emotional-intelligence&gt;&lt;turing-test&gt;&lt;affective-computing&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27" PostTypeId="2" ParentId="15" CreationDate="2016-08-02T16:01:59.740" Score="10" Body="&lt;p&gt;The problem of the Turing Test is that it tests the machines ability to resemble humans. Not necessarily every form of AI has to resemble humans. This makes the Turing Test less reliable. However, it is still useful since it is an actual test. It is also noteworthy that there is a prize for passing or coming closest to passing the Turing Test, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Loebner_Prize&quot;&gt;Loebner Prize&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intelligent agent definition of intelligence states that an agent is intelligent if it acts so to maximize the expected value of a performance measure based on past experience and knowledge. (paraphrased from &lt;a href=&quot;http://en.wikipedia.org/wiki/Philosophy_of_artificial_intelligence#Intelligent_agent_definition&quot;&gt;Wikipedia&lt;/a&gt;). This definition is used more often and does not depend on the ability to resemble humans. However, it is harder to test this. &lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2016-08-02T16:01:59.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28" PostTypeId="1" AcceptedAnswerId="143" CreationDate="2016-08-02T16:02:44.553" Score="11" ViewCount="5704" Body="&lt;p&gt;Since human intelligence presumably is a function of a natural genetic algorithm in nature, is using a genetic algorithm in a computer an example of artificial intelligence? If not, how do they differ? Or perhaps some are and some are not expressing artificial intelligence depending upon the scale of the algorithm and what it evolves into?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2019-06-20T20:36:38.073" LastActivityDate="2019-06-20T21:03:43.207" Title="Is a genetic algorithm an example of artificial intelligence?" Tags="&lt;philosophy&gt;&lt;genetic-algorithms&gt;&lt;terminology&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29" PostTypeId="5" CreationDate="2016-08-02T16:03:16.133" Score="0" Body="" OwnerUserId="5" LastEditorUserId="5" LastEditDate="2016-08-04T14:45:26.583" LastActivityDate="2016-08-04T14:45:26.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="30" PostTypeId="4" CreationDate="2016-08-02T16:03:16.133" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-02T16:03:16.133" LastActivityDate="2016-08-02T16:03:16.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="31" PostTypeId="2" ParentId="10" CreationDate="2016-08-02T16:04:09.333" Score="9" Body="&lt;p&gt;It's analogous to analogue versus digital, or the many shades of gray in between black and white: when evaluating the truthiness of a result, in binary boolean it's either true or false (0 or 1), but when utilizing fuzzy logic, it's an estimated probability between 0 and 1 (such as 0.75 being mostly probably true). It's useful for making calculated decisions when all information needed isn't necessarily available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia has a fantastic page for this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="62" LastActivityDate="2016-08-02T16:04:09.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="32" PostTypeId="2" ParentId="10" CreationDate="2016-08-02T16:04:39.867" Score="50" Body="&lt;p&gt;&lt;em&gt;As complexity rises, precise statements lose meaning and meaningful statements lose precision.&lt;/em&gt; ( Lofti Zadeh ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fuzzy logic deals with reasoning that is approximate rather than fixed and exact. This may make the reasoning more meaningful for a human:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xdHPJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xdHPJ.png&quot; alt=&quot;Precision and significance - comic&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Fuzzy logic is an extension of Boolean logic by Lotfi Zadeh in 1965 based on the&#xA;mathematical theory of fuzzy sets, which is a generalization of the classical set theory.&#xA;By introducing the notion of &lt;em&gt;degree in the verification&lt;/em&gt; of a condition, thus enabling a&#xA;condition to be in a state other than true or false, fuzzy logic provides a very valuable&#xA;flexibility for reasoning, which makes it possible to take into account inaccuracies and&#xA;uncertainties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One advantage of fuzzy logic in order to formalize human reasoning is that the rules&#xA;are set in natural language. For example, here are some rules of conduct that a driver&#xA;follows, assuming that he does not want to lose his driver’s licence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TM2UE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TM2UE.png&quot; alt=&quot;Fuzzy logic decision table&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, it thus seems that the input variables like in this example are approximately&#xA;appreciated by the brain, such as the degree of verification of a condition in fuzzy&#xA;logic.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I've written a short &lt;a href=&quot;https://scholar.google.com/citations?view_op=view_citation&amp;amp;hl=en&amp;amp;user=kz2aIc8AAAAJ&amp;amp;citation_for_view=kz2aIc8AAAAJ:eQOLeE2rZwMC&quot; rel=&quot;nofollow noreferrer&quot;&gt;introduction to fuzzy logic&lt;/a&gt; that goes into a bit more details but should be very accessible.&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="2255" LastEditDate="2020-05-26T17:15:36.130" LastActivityDate="2020-05-26T17:15:36.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="33" PostTypeId="2" ParentId="17" CreationDate="2016-08-02T16:04:57.997" Score="3" Body="&lt;p&gt;The concept of &quot;the singularity&quot; is when machines outsmart the humans. Although Stephen Hawking opinion is that this situation is inevitable, but I think it'll be very difficult to reach that point, because every A.I. algorithm needs to be programmed by humans, therefore it would be always more limited than its creator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would probably know when that point when humanity will lose control over Artificial Intelligence where super-smart AI would be in competition with humans and maybe creating more sophisticated intelligent beings occurred, but currently, it's more like science fiction (aka &lt;a href=&quot;https://en.wikipedia.org/wiki/Skynet_(Terminator)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Terminator's Skynet&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The risk could involve killing people (like self-flying war &lt;em&gt;drones&lt;/em&gt; making their own decision), destroying countries or even the whole planet (like A.I. connected to the nuclear weapons (aka &lt;a href=&quot;https://en.wikipedia.org/wiki/WarGames&quot; rel=&quot;nofollow noreferrer&quot;&gt;WarGames&lt;/a&gt; movie), but it doesn't prove the point that the machines would be smarter than humans.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-11T16:38:28.877" LastActivityDate="2018-04-11T16:38:28.877" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="35" PostTypeId="1" CreationDate="2016-08-02T16:05:26.390" Score="90" ViewCount="14138" Body="&lt;p&gt;These two terms seem to be related, especially in their application in computer science and software engineering.  Is one a subset of another?  Is one a tool used to build a system for the other?  What are their differences and why are they significant?&lt;/p&gt;&#xA;" OwnerUserId="69" LastEditorUserId="2444" LastEditDate="2021-01-15T19:59:38.840" LastActivityDate="2021-04-01T01:14:43.323" Title="What is the difference between artificial intelligence and machine learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;ai-field&gt;" AnswerCount="10" CommentCount="2" FavoriteCount="28" ContentLicense="CC BY-SA 3.0" />
  <row Id="36" PostTypeId="1" AcceptedAnswerId="114" CreationDate="2016-08-02T16:06:25.853" Score="43" ViewCount="1549" Body="&lt;p&gt;What aspects of quantum computers, if any, can help to further develop Artificial Intelligence?&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2016-08-02T19:13:49.690" LastActivityDate="2019-10-13T07:18:56.520" Title="To what extent can quantum computers help to develop Artificial Intelligence?" Tags="&lt;quantum-computing&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="37" PostTypeId="1" AcceptedAnswerId="73" CreationDate="2016-08-02T16:07:29.317" Score="7" ViewCount="3097" Body="&lt;p&gt;I believe a Markov chain is a sequence of events where each subsequent event depends probabilistically on the current event.  What are examples of the application of a Markov chain and can it be used to create artificial intelligence?  Would a genetic algorithm be an example of a Markov chain since each generation depends upon the state of the prior generation?&lt;/p&gt;&#xA;" OwnerUserId="55" LastActivityDate="2016-08-03T06:37:01.983" Title="What is a Markov chain and how can it be used in creating artificial intelligence?" Tags="&lt;genetic-algorithms&gt;&lt;markov-chain&gt;&lt;probability&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="38" PostTypeId="2" ParentId="28" CreationDate="2016-08-02T16:08:06.920" Score="3" Body="&lt;p&gt;This is probably more a question of philosophy than anything. In terms of how things are commonly defined, I'll say &quot;yes, genetic algorithms are part of AI&quot;.  If you pick up a comprehensive book on artificial intelligence, there will probably be a chapter on genetic algorithms (or more broadly, evolutionary algorithms). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One area that has been extensively studied in the past is the idea of using genetic algorithms to train neural networks.  I don't know if people are still actively researching this topic or not, but it at least illustrates that GA's are part of the overall rubric of AI in one regard.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2019-06-20T21:03:43.207" LastActivityDate="2019-06-20T21:03:43.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="39" PostTypeId="2" ParentId="15" CreationDate="2016-08-02T16:08:09.103" Score="16" Body="&lt;p&gt;The rhetorical point of the Turing Test is that it places the 'test' for 'humanity' in &lt;em&gt;observable outcomes&lt;/em&gt;, instead of in &lt;em&gt;internal components&lt;/em&gt;. If you would behave the same in interacting with an AI as you would with a person, how could &lt;em&gt;you&lt;/em&gt; know the difference between them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that doesn't mean it's reliable, because intelligence has many different components and there are many sorts of intellectual tasks. The Turing Test, in some respects, is about the reaction of people to behavior, which is not at all reliable--remember that many people thought &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot;&gt;ELIZA&lt;/a&gt;, a very simple chatbot, was an excellent listener and got deeply emotionally involved very quickly. It calls to mind the &lt;a href=&quot;https://www.youtube.com/watch?v=dBqhIVyfsRg&quot;&gt;Ikea commercial about throwing out a lamp&lt;/a&gt;, where the emotional attachment comes &lt;em&gt;from the human viewer&lt;/em&gt; (and the music), rather than from the lamp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing tests for specific economic activities are much more practically interesting--if one can write an AI that replaces an Uber driver, for example, what that will imply is much clearer than if someone can create a conversational chatbot.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T16:08:09.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="40" PostTypeId="1" AcceptedAnswerId="44" CreationDate="2016-08-02T16:08:23.377" Score="11" ViewCount="2222" Body="&lt;p&gt;What purpose does the &quot;dropout&quot; method serve and how does it improve the overall performance of the neural network?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="10" LastEditDate="2016-08-02T16:10:22.307" LastActivityDate="2020-10-01T20:16:56.117" Title="What is the &quot;dropout&quot; technique?" Tags="&lt;deep-neural-networks&gt;&lt;overfitting&gt;&lt;performance&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="41" PostTypeId="1" AcceptedAnswerId="65" CreationDate="2016-08-02T16:08:34.350" Score="10" ViewCount="950" Body="&lt;p&gt;Can an AI program have an IQ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, can the IQ of an AI program be measured?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like how humans can do an IQ test.&lt;/p&gt;&#xA;" OwnerUserId="72" LastEditorUserId="29" LastEditDate="2016-08-03T08:04:16.350" LastActivityDate="2019-06-07T19:54:42.090" Title="Can the IQ of an AI program be measured?" Tags="&lt;intelligence-testing&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="42" PostTypeId="1" AcceptedAnswerId="51" CreationDate="2016-08-02T16:09:25.427" Score="6" ViewCount="3323" Body="&lt;p&gt;Why would anybody want to use &quot;hidden layers&quot;? How do they enhance the learning ability of the network in comparison to the network which doesn't have them (linear models)?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-10T01:10:28.160" LastActivityDate="2021-01-10T22:56:08.420" Title="What is the purpose of the hidden layers?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;hidden-layers&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="43" PostTypeId="2" ParentId="10" CreationDate="2016-08-02T16:10:01.630" Score="22" Body="&lt;p&gt;Fuzzy logic is based on regular boolean logic. Boolean logic means you are working with truth values of either true or false (or 1 or 0 if you prefer). Fuzzy logic is the same apart from you can have truth values that are in-between true and false, that is to say, you are working with any number between 0 (inclusive) and 1 (inclusive). The fact that you can have a 'partially true and partially false' truth value is where the word &amp;quot;fuzzy&amp;quot; comes from. Natural languages often use fuzzy logic like &amp;quot;that balloon is red&amp;quot; meaning that balloon could be any colour that is similar enough to red, or &amp;quot;the shower is warm&amp;quot;. Here is a rough diagram for how &amp;quot;the temperature of the shower is warm&amp;quot; could be represented in terms of fuzzy logic (the y axis being the true value and the x-axis being the temperature):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G7szY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G7szY.png&quot; alt=&quot;y-axis=truth value of statement about temperature, x-axis=temperature&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Fuzzy logic can be applied to boolean operations such as &lt;strong&gt;and&lt;/strong&gt;, &lt;strong&gt;or&lt;/strong&gt;, and &lt;strong&gt;not&lt;/strong&gt;. Note that you can define the fuzzy logic operations in different ways. One way is with the min and max functions which return the lesser most and greater most values of the two values inputted respectively. This would work as such:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A and B = min(A,B)&#xA;A or B  = max(A,B)&#xA;not A   = 1-A&#xA;(where A and B are real values from 0 (inclusive) to 1 (inclusive))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When defined like this they are called the &lt;strong&gt;Zadeh operators&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another way would be to define &lt;strong&gt;and&lt;/strong&gt; as the first argument times the second argument, which yields different outputs for the same inputs as the Zadeh &lt;strong&gt;and&lt;/strong&gt; operator (&lt;code&gt;min(0.5,0.5)=0.5, 0.5*0.5=0.25&lt;/code&gt;). Then other operators are derived based on the &lt;strong&gt;and&lt;/strong&gt; and &lt;strong&gt;not&lt;/strong&gt; operators. This would work as such:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A and B = A*B&#xA;not A = 1-A&#xA;A or B = not ((not A) and (not B)) = 1-((1-A)*(1-B)) = 1-(1-A)*(1-B)&#xA;(where A and B are real values from 0 (inclusive) to 1 (inclusive))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can then use the three &amp;quot;basic fuzzy logic operations&amp;quot; to build all other &amp;quot;fuzzy logic operations&amp;quot;, just like you can use the three &amp;quot;basic boolean operations&amp;quot; to build all other &amp;quot;boolean logic operations&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Sources:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy logic wikipedia&lt;/a&gt;,&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Boolean_algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boolean algebra wikipedia&lt;/a&gt;,&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=r804UF8Ia4c&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explanation of fuzzy logic on Youtube&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note: if anyone could suggest some more reliable sources in the comments, I will happily add them to the list (I understand that the current ones aren't too reliable).&lt;/p&gt;&#xA;&lt;p&gt;Edit: My bad, I confused different ways to define &lt;em&gt;different&lt;/em&gt; operators in fuzzy logic with being different ways to define &lt;em&gt;the same&lt;/em&gt; operators in fuzzy logic.&lt;/p&gt;&#xA;" OwnerUserId="47" LastEditorUserId="36737" LastEditDate="2021-03-31T22:24:53.703" LastActivityDate="2021-03-31T22:24:53.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="44" PostTypeId="2" ParentId="40" CreationDate="2016-08-02T16:12:08.767" Score="8" Body="&lt;p&gt;Dropout means that every individual data point is only used to fit a random subset of the neurons. This is done to make the neural network more like an ensemble model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, just as a random forest is averaging together the results of many individual decision trees, you can see a neural network trained using dropout as averaging together the results of many individual neural networks (with 'results' understood to mean activations at every layer, rather than just the output layer).&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="12133" LastEditDate="2018-01-17T00:30:27.303" LastActivityDate="2018-01-17T00:30:27.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="45" PostTypeId="2" ParentId="17" CreationDate="2016-08-02T16:12:49.850" Score="23" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;technological singularity&lt;/a&gt; is a theoretical point in time at which a &lt;em&gt;self-improving&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificial general intelligence&lt;/a&gt; becomes able to understand and manipulate concepts outside of the human brain's range, that is, the moment when it can understand things humans, by biological design, can't.&lt;/p&gt;&#xA;&lt;p&gt;The fuzziness about the singularity comes from the fact that, from the singularity onwards, history is effectively &lt;em&gt;unpredictable&lt;/em&gt;. Humankind would be unable to predict any future events, or explain any present events, as science itself becomes incapable of describing machine-triggered events. Essentially, machines would think of us the same way we think of ants. Thus, we can make no predictions past the singularity. Furthermore, as a logical consequence, we'd be unable to define the point at which the singularity may occur at all, or even recognize it when it happens.&lt;/p&gt;&#xA;&lt;p&gt;However, in order for the singularity to take place, AGI needs to be developed, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence#Feasibility&quot; rel=&quot;nofollow noreferrer&quot;&gt;whether that is possible is quite a hot debate&lt;/a&gt; right now. Moreover, an algorithm that creates &lt;em&gt;superhuman intelligence&lt;/em&gt; (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;superintelligence&lt;/em&gt;&lt;/a&gt;) out of bits and bytes would have to be designed. By definition, a human programmer wouldn't be able to do such a thing, as his/her brain would need to be able to comprehend concepts beyond its range. There is also the argument that an &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity#Intelligence_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;intelligence explosion&lt;/em&gt;&lt;/a&gt; (the mechanism by which a technological singularity would theoretically be formed) would be impossible due to the difficulty of the design challenge of making itself more intelligent, getting larger proportionally to its intelligence, and that the difficulty of the design itself may overtake the intelligence required to solve the said challenge.&lt;/p&gt;&#xA;&lt;p&gt;Also, there are related theories involving &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_takeover&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;machines taking over humankind&lt;/em&gt;&lt;/a&gt; and all of that sci-fi narrative. However, that's unlikely to happen, if &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asimov's laws&lt;/a&gt; are followed appropriately. &lt;a href=&quot;https://www.youtube.com/watch?app=desktop&amp;amp;v=7PKx3kS7f4A&quot; rel=&quot;nofollow noreferrer&quot;&gt;Even if Asimov's laws were not enough&lt;/a&gt;, a series of constraints would still be necessary in order to avoid the misuse of AGI by misintentioned individuals, and Asimov's laws are the nearest we have to that.&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="2444" LastEditDate="2020-11-17T13:21:19.530" LastActivityDate="2020-11-17T13:21:19.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="46" PostTypeId="1" AcceptedAnswerId="71" CreationDate="2016-08-02T16:14:26.350" Score="11" ViewCount="3868" Body="&lt;p&gt;When did research into Artificial Intelligence first begin?  Was it called Artificial Intelligence then or was there another name?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2021-01-31T16:41:08.950" LastActivityDate="2021-01-31T16:46:15.343" Title="When did Artificial Intelligence research first start?" Tags="&lt;research&gt;&lt;history&gt;&lt;ai-field&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="47" PostTypeId="2" ParentId="28" CreationDate="2016-08-02T16:14:29.337" Score="2" Body="&lt;p&gt;The notion of genetics used in Genetic Algorithms (GAs) is a &lt;em&gt;very&lt;/em&gt; stripped down version relative to genetics in nature, essentially consisting of a population of 'genes' (representing solutions to some predefined problem) subject to `survival of the fittest' during iterated application of recombination and mutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays, the term 'Computational Intelligence' (CI) tends to be used to describe computational techniques intended to produce `the appearance of intelligence by &lt;em&gt;any&lt;/em&gt; computational means', rather than specifically attempting to mimic the mechanisms that are believed to give rise to human (or animal) intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the distinction between CI and AI is not so hard and fast, and arguably arose during the `AI Winter' when the term AI was out of fashion.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-02T16:14:29.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="48" PostTypeId="2" ParentId="42" CreationDate="2016-08-02T16:15:49.970" Score="3" Body="&lt;p&gt;Hidden layers by themselves aren't useful. If you had hidden layers that were linear, the end result would still be a linear function of the inputs, and so you could collapse an arbitrary number of linear layers down to a single layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why we use nonlinear &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow&quot;&gt;activation functions&lt;/a&gt;, like RELU. This allows us to add a level of nonlinear complexity with each hidden layer, and with arbitrarily many hidden layers we can construct arbitrarily complicated nonlinear functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because we can (at least in theory) capture any degree of complexity, we think of neural networks as &quot;universal learners,&quot; in that a large enough network could mimic any function.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T16:15:49.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="49" PostTypeId="2" ParentId="35" CreationDate="2016-08-02T16:16:25.863" Score="21" Body="&lt;p&gt;Machine learning is a subset of artificial intelligence. Roughly speaking, it corresponds to its learning side. There is no &quot;official&quot; definitions, boundaries are a bit fuzzy.&lt;/p&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-08-02T16:16:25.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="50" PostTypeId="1" AcceptedAnswerId="138" CreationDate="2016-08-02T16:16:46.797" Score="3" ViewCount="2548" Body="&lt;p&gt;How would you estimate the generalization error? What are the methods of achieving this?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-11-07T17:26:05.080" LastActivityDate="2020-11-08T13:07:02.920" Title="How can the generalization error be estimated?" Tags="&lt;machine-learning&gt;&lt;deep-neural-networks&gt;&lt;overfitting&gt;&lt;computational-learning-theory&gt;&lt;generalization&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="51" PostTypeId="2" ParentId="42" CreationDate="2016-08-02T16:16:59.330" Score="5" Body="&lt;p&gt;&quot;Hidden&quot; layers really aren't all that special... a hidden layer is really no more than any layer that isn't input or output. So even a very simple 3 layer NN has 1 hidden layer. So I think the question isn't really &quot;How do hidden layers help?&quot; as much as &quot;Why are deeper networks better?&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the answer to that latter question is an area of active research. Even top experts like Geoffrey Hinton and Andrew Ng will freely admit that we don't really understand why deep neural networks work. That is, we don't understand them in complete detail anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the theory, as I understand it goes something like this...  successive layers of the network learn successively more sophisticated features, which build on the features from preceding layers. So, for example, an NN used for facial recognition might work like this: the first layer detects edges and nothing else. The next layer up recognizes geometric shapes (boxes, circles, etc.). The next layer up recognizes primitive features of a face, like eyes, noses, jaw, etc. The next layer up then recognizes composites based on combinations of &quot;eye&quot; features, &quot;nose&quot; features, and so on.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in theory, deeper networks (more hidden layers) are better in that they develop a more granular/detailed representation of a &quot;thing&quot; being recognized.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="14723" LastEditDate="2018-04-12T02:30:32.837" LastActivityDate="2018-04-12T02:30:32.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="52" PostTypeId="1" AcceptedAnswerId="1437" CreationDate="2016-08-02T16:19:30.337" Score="7" ViewCount="587" Body="&lt;p&gt;I've implemented &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;the reinforcement learning algorithm&lt;/a&gt; for an agent to play &lt;a href=&quot;https://github.com/admonkey/snappybird&quot; rel=&quot;nofollow noreferrer&quot;&gt;snappy bird&lt;/a&gt; (a shameless cheap ripoff of flappy bird) utilizing a q-table for storing the history for future lookups. It works and eventually achieves perfect convergence after enough training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to implement a neural network to do function approximation in order to accomplish the purpose of the q-table? Obviously, storage is a concern with the q-table, but it doesn't seem to ever train with the neural net alone. Perhaps training the NN on an existing q-table would work, but I would like to not use a q-table at all if possible.&lt;/p&gt;&#xA;" OwnerUserId="62" LastEditorUserId="2444" LastEditDate="2019-05-10T14:42:23.103" LastActivityDate="2019-05-10T14:42:23.103" Title="Is it possible to implement reinforcement learning using a neural network?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="53" PostTypeId="2" ParentId="35" CreationDate="2016-08-02T16:20:03.773" Score="58" Body="&lt;p&gt;&lt;strong&gt;Machine learning&lt;/strong&gt; has been defined by many people in multiple (often similar) ways [&lt;a href=&quot;http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;http://www.cs.ubbcluj.ro/%7Egabis/ml/ML-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]. One definition says that machine learning (ML) is the field of study that gives computers the &lt;em&gt;ability to learn&lt;/em&gt; without being explicitly programmed.&lt;/p&gt;&#xA;&lt;p&gt;Given the above definition, we might say that machine learning is geared towards problems for which we have (lots of) data (experience), from which a program can learn and can get better at a task.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Artificial intelligence&lt;/strong&gt; has many more aspects, where machines may not get better at tasks by learning from data, but may exhibit &lt;em&gt;intelligence&lt;/em&gt; through rules (e.g. expert systems like &lt;a href=&quot;https://en.wikipedia.org/wiki/Mycin&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mycin&lt;/a&gt;), &lt;a href=&quot;https://silp.iiita.ac.in/wp-content/uploads/PROLOG.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;logic&lt;/a&gt; or algorithms, e.g. path-finding).&lt;/p&gt;&#xA;&lt;p&gt;The book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;&lt;/a&gt; shows more research fields of AI, like &lt;em&gt;Constraint Satisfaction Problems&lt;/em&gt;, &lt;em&gt;Probabilistic Reasoning&lt;/em&gt; or &lt;em&gt;Philosophical Foundations&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28" LastEditorUserId="2444" LastEditDate="2021-01-17T20:24:34.557" LastActivityDate="2021-01-17T20:24:34.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="54" PostTypeId="1" AcceptedAnswerId="76" CreationDate="2016-08-02T16:20:40.520" Score="7" ViewCount="174" Body="&lt;p&gt;I read that in the spring of 2016 a computer &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go&quot; rel=&quot;nofollow noreferrer&quot;&gt;Go program&lt;/a&gt; was finally able to beat a professional human for the first time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that this milestone has been reached, does that represent a significant advance in artificial intelligence techniques or was it just a matter of even more processing power being applied to the problem?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some of the methods used to program the successful Go-playing program? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are those methods considered to be artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2021-01-10T01:13:07.280" LastActivityDate="2021-01-10T01:13:07.280" Title="Does the recent advent of a Go playing computer represent Artificial Intelligence?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;agi&gt;&lt;alphago&gt;&lt;weak-ai&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="56" PostTypeId="2" ParentId="35" CreationDate="2016-08-02T16:20:52.200" Score="9" Body="&lt;p&gt;Many terms have 'mostly' the same meanings, and so the differences are just in emphasis, perspective, or historical descent. People disagree as to which label refers to the superset or the subset; there are people who will call AI a branch of ML and people who will call ML a branch of AI.&lt;/p&gt;&#xA;&lt;p&gt;I typically hear Machine Learning used as a form of 'applied statistics' where we specify a learning problem in enough detail that we can just feed training data into it and get a useful model out the other side.&lt;/p&gt;&#xA;&lt;p&gt;I typically hear Artificial Intelligence as a catch-all term to refer to any sort of intelligence embedded in the environment or in code. This is a very expansive definition, and others use narrower ones (such as focusing on artificial &lt;em&gt;general&lt;/em&gt; intelligence, which is not domain-specific). (Taken to an extreme, my version includes thermostats.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="2444" LastEditDate="2021-01-15T20:47:05.733" LastActivityDate="2021-01-15T20:47:05.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="57" PostTypeId="2" ParentId="17" CreationDate="2016-08-02T16:23:13.273" Score="0" Body="&lt;p&gt;The &quot;singularity,&quot; viewed narrowly, refers to a point at which economic growth is so fast that we can't make useful predictions about what the future past that point will look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's often used interchangeably with &quot;intelligence explosion,&quot; which is when we get so-called Strong AI, which is AI that is intelligent enough to understand and improve itself. It seems reasonable to expect that the intelligence explosion would immediately lead to an economic singularity, but the reverse is not necessarily true.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T16:23:13.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="58" PostTypeId="1" CreationDate="2016-08-02T16:25:20.223" Score="11" ViewCount="2690" Body="&lt;p&gt;Who first coined the term Artificial Intelligence? Is there a published research paper that first used that term?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2019-10-05T02:07:00.143" LastActivityDate="2020-04-21T11:12:25.793" Title="Who first coined the term Artificial Intelligence?" Tags="&lt;terminology&gt;&lt;history&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="60" PostTypeId="1" AcceptedAnswerId="1464" CreationDate="2016-08-02T16:27:49.533" Score="11" ViewCount="468" Body="&lt;p&gt;I have a background in Computer Engineering and have been working on developing better algorithms to mimic human thought. (One of my favorites is Analogical Modeling as applied to language processing and decision making.) However, the more I research, the more I realize just &lt;em&gt;how&lt;/em&gt; complicated AI is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to tackle many problems in this field, but sometimes I find that I am reinventing the wheel or am trying to solve a problem that has already been proven to be unsolvable (ie. the halting problem). So, to help in furthering AI, I want to better understand the current obstacles that are hindering our progress in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, time and space complexity of some machine learning algorithms is super-polynomial which means that even with fast computers, it can take a while for the program to complete. Even still, some algorithms may be fast on a desktop or other computer while dealing with a small data set, but when increasing the size of the data, the algorithm becomes intractable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are other issues currently facing AI development?&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-12T02:34:45.870" Title="What are the main problems hindering current AI development?" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="61" PostTypeId="2" ParentId="36" CreationDate="2016-08-02T16:28:29.363" Score="4" Body="&lt;p&gt;Quantum computers can help further develop A.I. algorithms and solve the problems to the extent of our creativity and ability to define the problem. For example breaking cryptography can take seconds, where it can takes thousands of years for standard computers. The same with artificial intelligence, it can predict all the combinations for the given problem defined by algorithm. This is due to superposition of multiple states of quantum bits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, quantum computers are still in the early stages of development and can perform complex calculation. There are already technologies like &lt;a href=&quot;https://en.wikipedia.org/wiki/D-Wave_Systems&quot; rel=&quot;nofollow&quot;&gt;D-Wave&lt;/a&gt; systems which are used by Google and NASA for complex data analysis, using Multi-Qubit type quantum computers for &lt;a href=&quot;https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_equations&quot; rel=&quot;nofollow&quot;&gt;solving NSE fluid dynamics problems&lt;/a&gt; of interest or global surveillance for military purposes, and many more which we're not aware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently there are only a few quantum computers available to the public, like &lt;a href=&quot;http://www.research.ibm.com/quantum/&quot; rel=&quot;nofollow&quot;&gt;IBM Quantum Experience&lt;/a&gt; (the world’s first quantum computing platform delivered via the IBM Cloud), but it's programming on quantum logic gates levels, so we're many years behind creating artificial intelligence available to public. There are some &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_programming&quot; rel=&quot;nofollow&quot;&gt;quantum computing languages&lt;/a&gt; such as QCL, Q or Quipper, but I'm not aware any libraries which can provide artificial intelligence frameworks. It doesn't mean it's not there, and I'm sure huge companies and governments organisations are using it for their agenda to outcome the competition (like financial market analysis, etc.).&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-04T21:06:48.983" LastActivityDate="2016-08-04T21:06:48.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="62" PostTypeId="2" ParentId="28" CreationDate="2016-08-02T16:29:11.850" Score="2" Body="&lt;p&gt;Human intelligence is &lt;strong&gt;not&lt;/strong&gt; an example of natural genetic algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic algorithms have collections of solutions that are collided with each other to make new solutions, eventually returning the best solution. Human intelligence is a network of neurons doing information processing, and almost all of it doesn't behave the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that something doesn't behave in the same way that human intelligence does doesn't mean that it's not an AI algorithm; I would include 'genetic algorithms' as a numerical optimization technique, and since optimization and intelligence are deeply linked any numerical optimization technique could be seen as an AI technique.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T16:29:11.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="63" PostTypeId="1" AcceptedAnswerId="208" CreationDate="2016-08-02T16:29:24.803" Score="11" ViewCount="219" Body="&lt;p&gt;I've read that the most of the problems can be solved with 1-2 hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you know you need more than 2? For what kind of problems you would need them (give me an example)?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-11T18:49:51.953" LastActivityDate="2018-04-11T18:49:51.953" Title="What kind of problems require more than 2 hidden layers?" Tags="&lt;deep-neural-networks&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="64" PostTypeId="1" CreationDate="2016-08-02T16:29:29.207" Score="4" ViewCount="66" Body="&lt;p&gt;What were the first areas of research into Artificial Intelligence and what were some early successes?  More recently we've had:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Beating a human at the game of chess&lt;/li&gt;&#xA;&lt;li&gt;Convincing a human that a person was conversing with them (passing the Turing test)&lt;/li&gt;&#xA;&lt;li&gt;Beating a human at Jeopardy game show&lt;/li&gt;&#xA;&lt;li&gt;Beating a human at the game of go.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Were there milestones that were considered major in the field before the 1990s?&lt;/p&gt;&#xA;" OwnerUserId="55" LastActivityDate="2017-03-15T05:38:48.220" Title="What were the first areas of research and what were some early successes?" Tags="&lt;history&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="65" PostTypeId="2" ParentId="41" CreationDate="2016-08-02T16:30:28.737" Score="11" Body="&lt;p&gt;Short answer: No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longer answer: It depends on what IQ exactly is, and when the question is asked compared to ongoing development. The topic you're referring to is actually more commonly described as AGI, or Artificial General Intelligence, as opposed to AI, which could be any narrow problem solving capability represented in software/hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence_quotient&quot;&gt;Intelligence quotient&lt;/a&gt; is a rough estimate of how well humans are able to generally answer questions they have not previously encountered, but as a predictor it is somewhat flawed, and has many criticisms and detractors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently (2016), no known programs have the ability to generalize, or apply learning from one domain to solving problems in an arbitrarily different domain through an abstract understanding. (However there are programs which can effectively analyze, or break down some information domains into simpler representations.) This seems likely to change as time goes on and both hardware and software techniques are developed toward this goal. Experts widely disagree as to the likely timing and approach of these developments, as well as to the most probable outcomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also worth noting that there seems to be a large deficit of understanding as to what exactly consciousness is, and disagreement over whether there is ever likely to be anything in the field of artificial intelligence that compares to it.&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="46" LastEditDate="2016-08-02T17:08:34.127" LastActivityDate="2016-08-02T17:08:34.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="66" PostTypeId="2" ParentId="58" CreationDate="2016-08-02T16:31:24.350" Score="12" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/John_McCarthy_(computer_scientist))&quot; rel=&quot;noreferrer&quot;&gt;John McCarthy&lt;/a&gt; (1927 - 2011) was an American computer scientist. A pioneer in the foundations of artificial intelligence research, &lt;strong&gt;he coined the term &quot;artificial intelligence&quot;&lt;/strong&gt;. He was one of the creators of the (original) Lisp programming language, which was quite involved in early AI research in the 1960s and 1970s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He coined the term in 1955, and organized the first Artificial Intelligence conference in 1956, while working as a math teacher at Dartmouth. He founded the AI labs at MIT and Stanford.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He's responsible for developing several other important concepts in today's mainstream computer science. Namely, he developed &lt;a href=&quot;https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)&quot; rel=&quot;noreferrer&quot;&gt;garbage collection&lt;/a&gt; (used by a Lisp interpreter) and designed the first &lt;a href=&quot;https://en.wikipedia.org/wiki/Time-sharing&quot; rel=&quot;noreferrer&quot;&gt;time-sharing systems&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a side note, McCarthy predicted that creating a truly intelligent machine would require &lt;a href=&quot;https://www.nytimes.com/2011/10/26/science/26mccarthy.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;1.8 Einsteins and one-tenth the resources of the Manhattan Project&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="2444" LastEditDate="2019-10-05T02:11:56.117" LastActivityDate="2019-10-05T02:11:56.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="67" PostTypeId="1" AcceptedAnswerId="72" CreationDate="2016-08-02T16:31:51.380" Score="6" ViewCount="3786" Body="&lt;p&gt;Why somebody would use SAT solvers (&lt;a href=&quot;https://en.wikipedia.org/wiki/Boolean_satisfiability_problem&quot; rel=&quot;nofollow&quot;&gt;Boolean satisfiability problem&lt;/a&gt;) to solve their real world problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any examples of the real uses of this model?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="10135" LastEditDate="2018-10-18T10:45:09.917" LastActivityDate="2020-08-23T12:05:32.803" Title="What are the real world uses for SAT solvers?" Tags="&lt;models&gt;&lt;problem-solving&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="68" PostTypeId="1" CreationDate="2016-08-02T16:33:52.707" Score="6" ViewCount="184" Body="&lt;p&gt;What evolutionary algorithms are there that model or incorporate some notion of &lt;a href=&quot;https://en.wikipedia.org/wiki/Epigenetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;epigenetics&lt;/a&gt;? What are the pros/cons of those approaches? Are there vast insufficiencies or wide-open questions about their usefulness?&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="2444" LastEditDate="2021-01-10T22:24:44.517" LastActivityDate="2021-01-10T22:24:44.517" Title="What evolutionary algorithms are there that model epigenetics?" Tags="&lt;reference-request&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;&lt;evolutionary-computation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="69" PostTypeId="2" ParentId="54" CreationDate="2016-08-02T16:34:26.163" Score="6" Body="&lt;p&gt;It doesn't make much sense to have a single threshold with &quot;unintelligent&quot; below it and &quot;intelligent&quot; above it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it makes more sense to have a gradation of intelligence by cognitive task. Inverting a matrix is a 'cognitive task,' and one where working memory pays off immensely; computers have been much better at that cognitive task than humans for a long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the AlphaGo victory represents has several components. One is that we have algorithms that are competitive with the best board-game playing humans at doing tactical and strategic thinking in the well-described world of Go. Another is that the deeper structure of the human visual system seems to have been duplicated, and so we have algorithms that can recognize patterns as well as humans--with &lt;em&gt;very&lt;/em&gt; limited resolution. (AlphaGo is seeing one pixel per stone, whereas we have very, very high-resolution eyes and the visual cortex to match.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different people have different intuitions, but it seems to me that visual intelligence is a huge component of human intelligence in general. If we know most of the secrets of human visual intelligence, that means there might be many tasks that computers could now perform as well as humans (if provided the correct training data).&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T16:34:26.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="70" PostTypeId="1" CreationDate="2016-08-02T16:38:55.800" Score="24" ViewCount="393" Body="&lt;p&gt;Can a Convolutional Neural Network be used for pattern recognition in problem domains without image data? For example, by representing abstract data in an image-like format with spatial relations? Would that always be less efficient?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/py5byOOHZM8?t=815&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;This Developer&quot;&gt;This developer&lt;/a&gt; says current development could go further but not if there's a limit outside image recognition.&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="1641" LastEditDate="2021-01-11T19:03:23.563" LastActivityDate="2021-01-11T19:03:23.563" Title="Is the pattern recognition capability of CNNs limited to image processing?" Tags="&lt;deep-neural-networks&gt;&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="71" PostTypeId="2" ParentId="46" CreationDate="2016-08-02T16:39:00.200" Score="11" Body="&lt;p&gt;The earliest research into thinking machines was inspired by a confluence of ideas that became prevalent in the late 30s, 40s and early 50s (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;formal logic&lt;/a&gt;, automata, &lt;a href=&quot;https://en.wikipedia.org/wiki/Robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;robots&lt;/a&gt;). Although the &lt;a href=&quot;https://academic.oup.com/mind/article/LIX/236/433/986238&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing test&lt;/a&gt; was proposed in the 1950s by &lt;a href=&quot;https://en.wikipedia.org/wiki/Alan_Turing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alan Turing&lt;/a&gt;, the work culminated back in the 1940s in the invention of the programmable digital computers, an abstract essence of mathematical reasoning. These ideas were inspired by a handful of scientists from a variety of fields who began seriously considering the possibility of building an electronic brain. &lt;a href=&quot;https://ojs.aaai.org//index.php/aimagazine/article/view/1904&quot; rel=&quot;nofollow noreferrer&quot;&gt;The field of artificial intelligence research was officially founded as an academic discipline in 1956 during the Dartmouth workshop&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, the concept of artificial beings is not new and it's as old as the Greek myths of Hephaestus and Pygmalion, which incorporated the idea of intelligent robots (such as &lt;em&gt;Talos&lt;/em&gt;) and artificial beings (such as &lt;em&gt;Galatea&lt;/em&gt; and &lt;em&gt;Pandora&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;See the following articles at Wikipedia for further details:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial intelligence (AI)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/History_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;History of artificial intelligence&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Timeline_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Timeline of artificial intelligence&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-31T16:46:15.343" LastActivityDate="2021-01-31T16:46:15.343" CommentCount="0" CommunityOwnedDate="2021-01-31T16:43:08.193" ContentLicense="CC BY-SA 4.0" />
  <row Id="72" PostTypeId="2" ParentId="67" CreationDate="2016-08-02T16:39:12.377" Score="4" Body="&lt;p&gt;Instead of talking about just SAT solvers, let me talk about optimization in general. Many economic problems can be cast as optimization problems: for example, FedEx may have a list of packages and the destinations for those packages, and must decide which packages to put on which trucks, and what order to deliver those packages in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you write out a mathematical description of this problem, there are a truly stunning number of possible solutions, and a well-defined way to evaluate which of two solutions is better. A solver is an algorithm that will evaluate a solution, come up with another solution, and then evaluate that one, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In small cases and simple problems, the solver can also terminate with a proof that it is actually the best solution possible. But typically instead the solver just reports &quot;this is the best solution that I've seen,&quot; and that's used. An improvement in the solver means you can reliably get lower-cost solutions than you were seeing before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the SAT problem specifically, the Wikipedia page on &lt;a href=&quot;https://en.wikipedia.org/wiki/Boolean_satisfiability_problem&quot; rel=&quot;nofollow&quot;&gt;SAT&lt;/a&gt; gives some examples:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since the SAT problem is NP-complete, only algorithms with exponential worst-case complexity are known for it. In spite of this, efficient and scalable algorithms for SAT were developed over the last decade[when?] and have contributed to dramatic advances in our ability to automatically solve problem instances involving tens of thousands of variables and millions of constraints (i.e. clauses).[1] Examples of such problems in electronic design automation (EDA) include formal equivalence checking, model checking, formal verification of pipelined microprocessors,[12] automatic test pattern generation, routing of FPGAs,[14] planning, and scheduling problems, and so on. A SAT-solving engine is now considered to be an essential component in the EDA toolbox.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10" LastEditorUserId="42" LastEditDate="2016-08-11T14:54:21.720" LastActivityDate="2016-08-11T14:54:21.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="73" PostTypeId="2" ParentId="37" CreationDate="2016-08-02T16:40:20.240" Score="10" Body="&lt;p&gt;A Markov model includes the probability of transitioning to each state considering the current state. &quot;Each state&quot; may be just one point - whether it rained on specific day, for instance - or it might look like multiple things - like a pair of words. You've probably seen automatically generated weird text that &lt;em&gt;almost&lt;/em&gt; makes sense, like &lt;a href=&quot;https://blog.codinghorror.com/markov-and-you/&quot;&gt;Garkov&lt;/a&gt; (the output of a Markov model based on the Garfield comic strips). That Coding Horror article also mentions the applications of Markov techniques to Google's PageRank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Markov models are really only powerful when they have a lot of input to work with. If a machine looked through a lot of English text, it would get a pretty good idea of what words generally come after other words. Or after looking through someone's location history, it could figure out where that person is likely to go next from a certain place. Constantly updating the &quot;input corpus&quot; as more data is received would let the machine tune the probabilities of all the state transitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic algorithms are fairly different things. They create functions by shuffling around parts of functions and seeing how good each function is at a certain task. A child algorithm will depend on its parents, but Markov models are interested mostly in predicting what thing will come next in a sequence, not creating a new chunk of code. You might be able to use a Markov model to spit out a candidate function, though, depending on how simple the &quot;alphabet&quot; is. You could even then give more weight to the transitions in successful algorithms.&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-08-02T16:40:20.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="74" PostTypeId="1" AcceptedAnswerId="141" CreationDate="2016-08-02T16:42:35.817" Score="44" ViewCount="22917" Body="&lt;p&gt;I've heard the terms strong-AI and weak-AI used.  Are these well defined terms or subjective ones?  How are they generally defined?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2019-06-20T20:32:04.477" LastActivityDate="2019-06-20T20:32:04.477" Title="What is the difference between strong-AI and weak-AI?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;agi&gt;&lt;comparison&gt;&lt;weak-ai&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="75" PostTypeId="1" CreationDate="2016-08-02T16:42:53.110" Score="0" ViewCount="110" Body="&lt;p&gt;As AI gains capabilities, and becomes more prevalent in society, our legal system will encounter questions it has not encountered before.  For example, if a self-driving car is involved in an accident while being controlled by the AI, who is at fault?  The &quot;driver&quot; (who's really just a passenger), the programmer(s) who made the AI, or the AI itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what's on the cutting edge in terms of these kinds of issues at the intersection of law and artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-08-02T18:00:37.377" LastActivityDate="2018-07-17T00:07:33.083" Title="What's the state of the art w.r.t research on the legal aspects of Artificial Intelligence?" Tags="&lt;legal&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="76" PostTypeId="2" ParentId="54" CreationDate="2016-08-02T16:43:14.880" Score="9" Body="&lt;p&gt;There are at least two questions in your question: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are some of the methods used to program the successful go playing program?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are those methods considered to be artificial intelligence?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first question is deep and technical, the second broad and philosophical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The methods have been described in: &lt;a href=&quot;https://www.researchgate.net/publication/292074166_Mastering_the_game_of_Go_with_deep_neural_networks_and_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the Game of Go with Deep Neural Networks and Tree Search&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem of Go or perfect information games in general is that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;exhaustive search is infeasible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the methods will concentrate on shrinking the search space in an efficient way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods and structures described in the paper include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;learning from expert human players in a supervised fashion&lt;/li&gt;&#xA;&lt;li&gt;learning by playing against itself (reinforcement learning)&lt;/li&gt;&#xA;&lt;li&gt;Monte-Carlo tree search (MCTS) combined with policy and value networks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The second question has no definite answer, as you will have at least two angles on AI: &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room#Strong_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;strong&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;weak&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;All real-world systems labeled &quot;artificial intelligence&quot; of any sort are &lt;strong&gt;weak AI at most&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, yes, it is artificial intelligence, but it is non-sentient.&lt;/p&gt;&#xA;" OwnerUserId="28" LastEditorUserId="2444" LastEditDate="2020-03-09T21:29:34.550" LastActivityDate="2020-03-09T21:29:34.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="77" PostTypeId="1" AcceptedAnswerId="131" CreationDate="2016-08-02T16:45:01.487" Score="24" ViewCount="1737" Body="&lt;p&gt;I know that language of Lisp was used early on when working on artificial intelligence problems. Is it still being used today for significant work? If not, is there a new language that has taken its place as the most common one being used for work in AI today?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="14723" LastEditDate="2018-04-14T01:47:35.577" LastActivityDate="2021-01-10T23:03:14.497" Title="Is Lisp still being used to tackle AI problems?" Tags="&lt;history&gt;&lt;programming-languages&gt;&lt;lisp&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="79" PostTypeId="2" ParentId="75" CreationDate="2016-08-02T16:45:51.657" Score="3" Body="&lt;p&gt;One person working in this space is Dr. Woody Barfield. He just wrote a book titled &lt;em&gt;&lt;a href=&quot;http://www.springer.com/us/book/9783319250489&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cyberhumans: Our Future With Machines&lt;/a&gt;&lt;/em&gt; that focuses largely on the legal/policy issues around AI (and related topics). In addition to the book, he is continuing with other research in this area.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="14723" LastEditDate="2018-04-14T02:00:29.083" LastActivityDate="2018-04-14T02:00:29.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="80" PostTypeId="1" AcceptedAnswerId="85" CreationDate="2016-08-02T16:46:07.253" Score="11" ViewCount="1527" Body="&lt;p&gt;What are the specific requirements of the Turing test? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What requirements if any must the evaluator fulfill in order to be qualified to give the test?&lt;/li&gt;&#xA;&lt;li&gt;Must there always be two participants in the conversation (one human and one computer) or can there be more?&lt;/li&gt;&#xA;&lt;li&gt;Are placebo tests (where there is not actually a computer involved) allowed or encouraged?&lt;/li&gt;&#xA;&lt;li&gt;Can there be multiple evaluators? If so does the decision need to be unanimous among all evaluators in order for the machine to have passed the test?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="96" LastEditorUserId="2444" LastEditDate="2020-05-18T23:55:50.073" LastActivityDate="2020-05-18T23:55:50.073" Title="What are the specific requirements of the Turing test?" Tags="&lt;natural-language-processing&gt;&lt;intelligence-testing&gt;&lt;turing-test&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="81" PostTypeId="1" AcceptedAnswerId="87" CreationDate="2016-08-02T16:49:40.830" Score="8" ViewCount="1873" Body="&lt;p&gt;I believe that statistical AI uses inductive thought processes. For example, deducing a trend from a pattern, after training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some examples of successfully applied statistical AI to real-world problems?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="2444" LastEditDate="2019-06-27T23:44:11.853" LastActivityDate="2019-06-27T23:46:25.427" Title="What are some examples of statistical AI applied to real-world problems?" Tags="&lt;applications&gt;&lt;statistical-ai&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="82" PostTypeId="1" CreationDate="2016-08-02T16:50:15.330" Score="1" ViewCount="107" Body="&lt;p&gt;How do the basic components &lt;a href=&quot;https://en.wikipedia.org/wiki/Optimality_theory&quot; rel=&quot;nofollow&quot;&gt;optimality theory&lt;/a&gt; apply to artificial intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is optimality theory related to neural network research?&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="2444" LastEditDate="2019-04-05T12:19:34.277" LastActivityDate="2019-04-05T12:19:34.277" Title="What is the relation between optimality theory and AI?" Tags="&lt;neural-networks&gt;&lt;applications&gt;&lt;comparison&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="83" PostTypeId="2" ParentId="1" CreationDate="2016-08-02T16:54:40.380" Score="2" Body="&lt;p&gt;Yes, as Franck has rightly put, &quot;backprop&quot; means backpropogation, which is frequently used in the domain of neural networks for error optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a detailed explanation, I would point out &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow&quot;&gt;this tutorial&lt;/a&gt; on the concept of backpropogation by a very good book of Michael Nielsen. &lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2016-08-02T16:54:40.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="84" PostTypeId="1" CreationDate="2016-08-02T16:55:37.050" Score="11" ViewCount="855" Body="&lt;p&gt;Some programs do exhaustive searches for a solution while others do heuristic searches for a similar answer. For example, in chess, the search for the best next move tends to be more exhaustive in nature whereas, in Go, the search for the best next move tends to be more heuristic in nature due to the much larger search space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the technique of brute force exhaustive searching for a good answer considered to be AI or is it generally required that heuristic algorithms be used before being deemed AI? If so, is the chess-playing computer beating a human professional seen as a meaningful milestone?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="14723" LastEditDate="2018-04-14T02:03:13.207" LastActivityDate="2018-12-11T09:53:09.487" Title="Are methods of exhaustive search considered to be AI?" Tags="&lt;gaming&gt;&lt;search&gt;&lt;chess&gt;&lt;heuristics&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="85" PostTypeId="2" ParentId="80" CreationDate="2016-08-02T16:59:11.467" Score="10" Body="&lt;p&gt;The &quot;Turing Test&quot; is generally taken to mean an updated version of the Imitation Game Alan Turing proposed in his 1951 paper of the same name. An early version had a human (male or female) and a computer, and a judge had to decide which is which, and what gender they were if human. If they were correct less than 50% then the computer was considered &quot;intelligent.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current generally accepted version requires only one contestant, and a judge to decide whether it is human or machine. So yes, sometimes this will be a placebo, effectively, if we consider a human to be a placebo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your first and fourth questions are related - and there are no strict guidelines. If the computer can fool a greater number of judges then it will of course be considered a better AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The University of Toronto has a validity section in &lt;a href=&quot;http://www.psych.utoronto.ca/users/reingold/courses/ai/turing.html&quot;&gt;this paper on Turing&lt;/a&gt;, which includes a link to &lt;a href=&quot;http://ciips.ee.uwa.edu.au/Papers/Technical_Reports/1997/05/Index.html&quot;&gt;Jason Hutchens' commentary&lt;/a&gt; on why the Turing test may not be relevant (humans may also fail it) and the &lt;a href=&quot;http://www.loebner.net/Prizef/loebner-prize.html&quot;&gt;Loebner Prize&lt;/a&gt;, a formal instantiation of a Turing Test .&lt;/p&gt;&#xA;" OwnerUserId="97" LastEditorUserId="97" LastEditDate="2016-08-02T17:07:15.277" LastActivityDate="2016-08-02T17:07:15.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="86" PostTypeId="1" AcceptedAnswerId="93" CreationDate="2016-08-02T16:59:30.683" Score="29" ViewCount="1041" Body="&lt;p&gt;How is a neural network having the &quot;deep&quot; adjective actually distinguished from other similar networks?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-02-23T22:05:20.793" LastActivityDate="2020-08-23T11:06:25.767" Title="How is a deep neural network different from other neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-neural-networks&gt;&lt;terminology&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="87" PostTypeId="2" ParentId="81" CreationDate="2016-08-02T16:59:50.293" Score="10" Body="&lt;p&gt;There are several examples. For example, one instance of using Statistical AI from my workplace is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Analyzing the behavior of the customer and their food-ordering trends, and then trying to upsell by recommending them the dishes which they might like to order/eat. This can be done through the apriori and FP-growth algorithms. We then, automated the algorithm, and then the algorithm improves itself through an &lt;code&gt;Ordered/Not-Ordered&lt;/code&gt; metric.&lt;/li&gt;&#xA;&lt;li&gt;Self-driving cars. They use reinforcement and supervised learning algorithms for learning the route and the gradient/texture of the road.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="101" LastEditorUserId="14723" LastEditDate="2018-04-12T14:09:57.913" LastActivityDate="2018-04-12T14:09:57.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="88" PostTypeId="1" AcceptedAnswerId="2573" CreationDate="2016-08-02T17:01:18.317" Score="4" ViewCount="535" Body="&lt;p&gt;What is the effectiveness of pre-training of unsupervised deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does unsupervised deep learning actually work?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-11-19T21:16:17.260" LastActivityDate="2020-04-21T11:14:24.307" Title="Why does unsupervised pre-training help in deep learning?" Tags="&lt;deep-learning&gt;&lt;unsupervised-learning&gt;&lt;transfer-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="89" PostTypeId="2" ParentId="84" CreationDate="2016-08-02T17:02:05.890" Score="7" Body="&lt;p&gt;If a computer is just brute-forcing the solution, it's not learning anything or using any kind of intelligence at all, and therefore it shouldn't be called &quot;artificial intelligence.&quot; It has to make decisions based on what's happened before in similar instances. For something to be intelligent, it needs a way to keep track of what it's learned. A chess program might have a really awesome measurement algorithm to use on every possible board state, but if it's always trying each state and never storing what it learns about different approaches, it's not intelligent.&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="75" LastEditDate="2016-08-02T17:33:59.340" LastActivityDate="2016-08-02T17:33:59.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="91" PostTypeId="1" AcceptedAnswerId="97" CreationDate="2016-08-02T17:04:35.297" Score="16" ViewCount="384" Body="&lt;p&gt;Are search engines considered AI because of the way they analyze what you search for and remember it? Or how they send you ads of what you've searched for recently? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this considered AI or just smart?&lt;/p&gt;&#xA;" OwnerUserId="5" LastEditorUserId="2444" LastEditDate="2019-06-24T13:10:54.557" LastActivityDate="2019-06-24T13:10:54.557" Title="Are search engines considered AI?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="92" PostTypeId="1" AcceptedAnswerId="250" CreationDate="2016-08-02T17:05:27.590" Score="80" ViewCount="5994" Body="&lt;p&gt;The following &lt;a href=&quot;http://www.evolvingai.org/fooling&quot; rel=&quot;noreferrer&quot;&gt;page&lt;/a&gt;/&lt;a href=&quot;http://www.evolvingai.org/files/DNNsEasilyFooled_cvpr15.pdf&quot; rel=&quot;noreferrer&quot;&gt;study&lt;/a&gt; demonstrates that the deep neural networks are easily fooled by giving high confidence predictions for unrecognisable images, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7pgrH.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7pgrH.jpg&quot; alt=&quot;Evolved images that are unrecognisable to humans, but that state-of-the-art DNNs trained on ImageNet believe with &amp;gt;= 99.6% certainty to be a familiar object. This result highlights differences between how DNNs and humans recognise objects. Directly and indirectly encoded images&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pBm48.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pBm48.png&quot; alt=&quot;Evolving images to match DNN classes produces a tremendous diversity of images. The mean DNN confidence scores for these images is 99.12% for the listed class, meaning that the DNN believes with near-certainty that the image is that type of thing. Shown are images selected to showcase diversity from 5 independent evolutionary runs. The images shed light on what the DNN network cares about, and what it does not, when classifying an image. For example, a school bus is alternating yellow and black lines, but does not need to have a windshield or wheels&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How this is possible? Can you please explain ideally in plain English?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-02-18T19:48:25.217" LastActivityDate="2021-02-18T19:48:25.217" Title="How is it possible that deep neural networks are so easily fooled?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;deep-neural-networks&gt;&lt;adversarial-ml&gt;" AnswerCount="9" CommentCount="2" FavoriteCount="35" ContentLicense="CC BY-SA 3.0" />
  <row Id="93" PostTypeId="2" ParentId="86" CreationDate="2016-08-02T17:06:21.223" Score="31" Body="&lt;p&gt;The difference is mostly in the number of layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a long time, it was believed that &quot;1-2 hidden layers are enough for most tasks&quot; and it was impractical to use more than that, because training neural networks can be very computationally demanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays, computers are capable of much more, so people have started to use networks with more layers and found that they work very well for some tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The word &quot;deep&quot; is there simply to distinguish these networks from the traditional, &quot;more shallow&quot; ones.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="2444" LastEditDate="2019-02-23T22:08:58.833" LastActivityDate="2019-02-23T22:08:58.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="94" PostTypeId="1" AcceptedAnswerId="132" CreationDate="2016-08-02T17:06:46.317" Score="4" ViewCount="203" Body="&lt;p&gt;In a feedforward neural network the inputs are fed directly to the outputs via a series of &lt;strong&gt;weights&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What purpose do the weights serve and how are they significant in this neural network?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="4709" LastEditDate="2018-07-18T10:08:51.063" LastActivityDate="2018-07-18T10:08:51.063" Title="What is the significance of weights in a feedforward neural network?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="95" PostTypeId="2" ParentId="86" CreationDate="2016-08-02T17:08:48.340" Score="10" Body="&lt;p&gt;A deep neural network is just a (feed-forward) neural network with many layers.&lt;/p&gt;&#xA;&lt;p&gt;However, deep belief networks, Deep Boltzmann networks, etc., are not considered (debatable) deep neural networks, as their topology is different (i.e. they have undirected networks in their topology).&lt;/p&gt;&#xA;&lt;p&gt;See also this: &lt;a href=&quot;https://stats.stackexchange.com/a/59854/84191&quot;&gt;https://stats.stackexchange.com/a/59854/84191&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="2444" LastEditDate="2020-08-23T11:06:25.767" LastActivityDate="2020-08-23T11:06:25.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="96" PostTypeId="1" AcceptedAnswerId="98" CreationDate="2016-08-02T17:12:58.533" Score="11" ViewCount="258" Body="&lt;p&gt;What is the definition of a deep neural network? Why are they so popular or important?&lt;/p&gt;&#xA;" OwnerUserId="5" LastEditorUserId="2444" LastEditDate="2019-06-23T12:18:37.573" LastActivityDate="2020-04-21T11:25:25.153" Title="What is a deep neural network?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;deep-neural-networks&gt;&lt;definitions&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ClosedDate="2020-08-23T11:03:13.587" ContentLicense="CC BY-SA 4.0" />
  <row Id="97" PostTypeId="2" ParentId="91" CreationDate="2016-08-02T17:15:22.887" Score="22" Body="&lt;p&gt;I believe it would be more accurate to say that (some) search engines &lt;em&gt;use&lt;/em&gt; AI.  Broadly saying &quot;search engines are AI&quot; is not really correct. At the core, most search engines are nothing more than an inverted text index using something like tf–idf scoring. That's a very mechanical/simple thing that nobody would really call AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But more sophisticated search engines may &lt;em&gt;use&lt;/em&gt; AI or AI techniques to do things like semantic analysis - so they can actually &quot;answer questions&quot; instead of just looking up words in an index.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="14723" LastEditDate="2018-04-13T18:07:34.777" LastActivityDate="2018-04-13T18:07:34.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="98" PostTypeId="2" ParentId="96" CreationDate="2016-08-02T17:18:12.383" Score="12" Body="&lt;p&gt;A deep neural network (DNN) is nothing but a neural network which has multiple layers, where &lt;em&gt;multiple&lt;/em&gt; can be subjective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMHO, any network which has 6 or 7 or more layers is considered deep. So, the above would form a very basic definition of a deep network. &lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="2444" LastEditDate="2019-06-24T10:56:42.927" LastActivityDate="2019-06-24T10:56:42.927" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="100" PostTypeId="2" ParentId="96" CreationDate="2016-08-02T17:36:52.900" Score="5" Body="&lt;p&gt;Deep networks have two main differences with 'normal' networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first is that computational power and training datasets have grown immensely, meaning that it's practical to run larger networks and statistically valid (that is, we have enough training examples that we won't just run into over-fitting problems with larger networks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is that back propagation is limited the more layers you have; each layer represents a gradient of the error, and so by the time one is about six layers deep there isn't much error left to modify the neuron weights. But one might reasonably expect earlier neurons to be more important than later neurons, since they represent 'concepts' that are closer to the raw inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;New training techniques sidestep this problem, typically by doing unsupervised learning on the raw inputs, creating higher-level 'concepts' that are then useful as inputs for supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For example, consider the problem of determining whether or not an image contains a cat from the pixels. The early layers of the network should be doing things like detecting edges, which one could expect to be shared among all images and mostly independent of what one is trying to do with the output layers, thus also hard to train through 'cat-not cat' signals many layers up.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="5" LastEditDate="2016-08-03T15:13:11.603" LastActivityDate="2016-08-03T15:13:11.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="101" PostTypeId="2" ParentId="84" CreationDate="2016-08-02T17:38:38.237" Score="8" Body="&lt;p&gt;If one thinks of intelligence as a continuous measure of optimization power (that is, how much better are outcomes for any unit of cognitive effort expended), then exhaustive search has non-zero intelligence (in that it does actually give better outcomes as more effort is expended) but &lt;em&gt;very, very low&lt;/em&gt; intelligence (as the outcomes are better mostly by luck, and the amount of effort expended can be impossibly large).&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T17:38:38.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="102" PostTypeId="2" ParentId="80" CreationDate="2016-08-02T17:42:05.387" Score="3" Body="&lt;p&gt;There are really two questions here, that I can see. One is &quot;what were the specific requirements of the original Turing test, as stated by Turing himself?&quot; The other is &quot;What should the specific requirements of a modern Turing test be?&quot; Things have advanced a lot since Turing's day, and I think it's reasonable for us to consider extending/modifying his test to reflect our current understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to the first question is easy enough to look up, so I think the interesting one is the second one. What &lt;em&gt;should&lt;/em&gt; a test to determine intelligence look like? With that in mind, I think the answer to all four questions posed by the OP is &quot;it depends&quot;. I don't think there's universal consensus on how to structure a perfect Turing test, so a given experimenter is really free to set things up however he/she wants.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all, of course, based on the assumption that the Turing test or a Turing Test-like test is actually of value. That's not necessarily a given.  Consider that, to some extent, what we're talking about is designing an AI with an exceptional ability for deceit! That is, assuming the questioner is allowed to simply ask &quot;are you human&quot;, then we have to assume that the AI is supposed to lie if it wants to pass the test. So one might rightly ask, is designing a system to be really good at telling lies, a valuable approach to AI?&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="14723" LastEditDate="2018-04-13T03:22:45.343" LastActivityDate="2018-04-13T03:22:45.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="103" PostTypeId="1" CreationDate="2016-08-02T17:47:41.750" Score="6" ViewCount="3267" Body="&lt;p&gt;I believe that Classical AI uses deductive thought processes. For example, given as a set of constraints, deduce a conclusion.  What are some examples of successfully applying Classical AI to real world problems.&lt;/p&gt;&#xA;" OwnerUserId="55" LastActivityDate="2021-01-24T22:04:16.883" Title="What are some examples of Classical AI?" Tags="&lt;symbolic-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="104" PostTypeId="1" CreationDate="2016-08-02T17:56:02.743" Score="10" ViewCount="293" Body="&lt;p&gt;In &lt;a href=&quot;https://youtu.be/oSdPmxRCWws?t=30&quot;&gt;this video&lt;/a&gt; an expert says, &quot;One way of thinking about what intelligence is [specifically with regard to artificial intelligence], is as an optimization process.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can intelligence always be thought of as an optimization process, and can artificial intelligence always be modeled as an optimization problem? What about pattern recognition? Or is he mischaracterizing?&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="46" LastEditDate="2016-08-02T18:01:01.713" LastActivityDate="2018-04-13T18:00:00.527" Title="Can artificial intelligence be thought of as optimization?" Tags="&lt;optimization&gt;&lt;agi&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="105" PostTypeId="2" ParentId="103" CreationDate="2016-08-02T18:12:04.120" Score="8" Body="&lt;p&gt;The term &lt;em&gt;classical AI&lt;/em&gt; refers to the concept of intelligence that was broadly accepted after the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dartmouth_workshop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dartmouth Conference&lt;/a&gt; and basically refers to a kind of intelligence that is strongly symbolic and oriented to logic and language processing. One basic point is the duality &lt;strong&gt;body&lt;/strong&gt; vs. &lt;strong&gt;mind&lt;/strong&gt;. It's in this period that the mind starts to be compared with computer software.&lt;/p&gt;&#xA;&lt;h3&gt;Two classical historical examples of this conception of intelligence&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Deep Blue&lt;/strong&gt;&lt;/a&gt;, whose aim in life was to be the master of chess, ruling over the (not-so) intelligent mankind&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Eliza&lt;/strong&gt;&lt;/a&gt; a computer-based therapist that turned out to &lt;a href=&quot;https://en.wikipedia.org/wiki/Joseph_Weizenbaum&quot; rel=&quot;nofollow noreferrer&quot;&gt;trigger a critic&lt;/a&gt; to the classical AI&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Two technical examples of classical AI&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Expert systems&lt;/strong&gt;, which are computer programs that strongly rely on the type of constraints and conclusions that you refer to, &lt;a href=&quot;https://www.britannica.com/technology/expert-system&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;in order to accomplish feats of apparent intelligence&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Fuzzy logic&lt;/strong&gt;, &lt;a href=&quot;https://de.mathworks.com/help/fuzzy/what-is-fuzzy-logic.html?requestedDomain=www.mathworks.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;which is an extension of multivalued logic&lt;/em&gt;&lt;/a&gt;, but with continuous values instead of discrete ones&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that in all cases the &lt;em&gt;hardware&lt;/em&gt; (once compared with the &lt;em&gt;body&lt;/em&gt;) does not play any role: Intelligence is abstract and independent from the material world.&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="2444" LastEditDate="2021-01-24T22:04:16.883" LastActivityDate="2021-01-24T22:04:16.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="106" PostTypeId="2" ParentId="54" CreationDate="2016-08-02T18:14:37.117" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Now that this milestone has been reached, does that represent a significant advance in artificial intelligence techniques or was it just a matter of ever more processing power being applied to the problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Neither, really. It is a milestone and a significant advance in computers beating humans in games, but the techniques used are only relevant to that game, not for other purposes in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution lies in humans analyzing the game and implementing algorithms for finding a good move. This is the main reason that a computer can beat the humans, together with the fact that it can calculate much faster and that it doesn't make really bad moves by not seeing something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Processing power helps, but the game-tree complexity for Go is very large, estimated to be larger than 10&lt;sup&gt;200&lt;/sup&gt;, whereas the game-tree complexity for chess is only 10&lt;sup&gt;120&lt;/sup&gt; (known as the Shannon number), so chess is less hard. This means that for neither chess nor go a database can be created with all possible positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that Deep Blue beat Kasparov in a six-game match in 1997 was quite a development since this was one of the first &quot;hard&quot; games where a computer beat a top human. But it still isn't really Artificial Intelligence, more analyzing the game. Implementing an opening and endgame book was a large part, the middle game was done using analysis, I don't know the details. &lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="14723" LastEditDate="2018-04-12T02:46:33.907" LastActivityDate="2018-04-12T02:46:33.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="107" PostTypeId="2" ParentId="81" CreationDate="2016-08-02T18:16:43.960" Score="4" Body="&lt;p&gt;There are many online services that use statistical neural networks for recommendations. For example, we have &lt;a href=&quot;http://imhonet.ru&quot; rel=&quot;nofollow noreferrer&quot;&gt;a well known service&lt;/a&gt; here in Russia that could give it's users recommendations for movies and shows to watch and books to read. Its recommendation core is based on many things known about a user: what movies/books he or she loves and what not, analyses his or her friends like and so on. While you have only a few items rayed it will give you very strange recommendations but then it becomes more accurate and really could give you some true gems.&lt;/p&gt;&#xA;" OwnerUserId="112" LastEditorUserId="1671" LastEditDate="2018-04-13T03:20:26.790" LastActivityDate="2018-04-13T03:20:26.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="108" PostTypeId="1" CreationDate="2016-08-02T18:17:44.297" Score="7" ViewCount="2107" Body="&lt;p&gt;What specific advantages of declarative languages make them more applicable to AI than imperative languages?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can declarative languages do easily that other languages styles find difficult for this kind of problem?&lt;/p&gt;&#xA;" OwnerUserId="69" LastEditorUserId="1581" LastEditDate="2018-11-13T17:24:41.510" LastActivityDate="2018-11-18T12:34:56.643" Title="What are the main advantages of using declarative programming languages for building AI?" Tags="&lt;declarative-programming&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="109" PostTypeId="1" CreationDate="2016-08-02T18:29:19.443" Score="4" ViewCount="73" Body="&lt;p&gt;In years past, GOFAI (Good Old Fashioned AI) was heavily based on &quot;rules&quot; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic computation&lt;/a&gt; based on rules.  Unfortunately, that approach ran into stumbling blocks, and the world moved heavily towards statistical/probabilistic approaches leading to the current wave of interest in &quot;machine learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems though, that the symbolic/rule-based approach probably still has application. So, could one &quot;learn&quot; rules using a probabilistic &lt;a href=&quot;https://en.wikipedia.org/wiki/Rule_induction&quot; rel=&quot;nofollow noreferrer&quot;&gt;rule induction&lt;/a&gt; method, and then layer symbolic computation on top? If so, how could the whole process be made truly two-way, so that something &quot;learned&quot; from processing rules, can be fed back into how the system learns rules?&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="14723" LastEditDate="2018-04-14T02:02:43.383" LastActivityDate="2018-08-15T22:45:20.830" Title="Can rule induction be considered a way to &quot;hybridize&quot; probabilistic / statistical approaches and symbolic approaches?" Tags="&lt;symbolic-ai&gt;&lt;symbolic-computing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="111" PostTypeId="1" AcceptedAnswerId="1790" CreationDate="2016-08-02T18:57:57.550" Score="98" ViewCount="6043" Body="&lt;p&gt;Obviously, self-driving cars aren't perfect, so imagine that the Google car (as an example) got into a difficult situation.&lt;/p&gt;&#xA;&lt;p&gt;Here are a few examples of unfortunate situations caused by a set of events:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The car is heading toward a crowd of 10 people crossing the road, so it cannot stop in time, but it can avoid killing 10 people by hitting the wall (killing the passengers),&lt;/li&gt;&#xA;&lt;li&gt;Avoiding killing the rider of the motorcycle considering that the probability of survival is greater for the passenger of the car,&lt;/li&gt;&#xA;&lt;li&gt;Killing an animal on the street in favour of a human being,&lt;/li&gt;&#xA;&lt;li&gt;Purposely changing lanes to crash into another car to avoid killing a dog,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And here are a few dilemmas:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the algorithm recognize the difference between a human being and an animal?&lt;/li&gt;&#xA;&lt;li&gt;Does the size of the human being or animal matter?&lt;/li&gt;&#xA;&lt;li&gt;Does it count how many passengers it has vs. people in the front?&lt;/li&gt;&#xA;&lt;li&gt;Does it &amp;quot;know&amp;quot; when babies/children are on board?&lt;/li&gt;&#xA;&lt;li&gt;Does it take into the account the age (e.g. killing the older first)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How would an algorithm decide what it should do from the technical perspective? Is it being aware of above (counting the probability of kills), or not (killing people just to avoid its own destruction)? &lt;/p&gt;&#xA;&lt;p&gt;Related articles:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/&quot; rel=&quot;noreferrer&quot;&gt;Why Self-Driving Cars Must Be Programmed to Kill&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.technologyreview.com/s/539731/how-to-help-self-driving-cars-make-ethical-decisions/&quot; rel=&quot;noreferrer&quot;&gt;How to Help Self-Driving Cars Make Ethical Decisions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="36530" LastEditDate="2020-06-26T22:37:36.083" LastActivityDate="2020-06-26T22:37:36.083" Title="How could self-driving cars make ethical decisions about who to kill?" Tags="&lt;philosophy&gt;&lt;ethics&gt;&lt;autonomous-vehicles&gt;&lt;decision-theory&gt;" AnswerCount="14" CommentCount="4" FavoriteCount="20" ContentLicense="CC BY-SA 4.0" />
  <row Id="112" PostTypeId="1" CreationDate="2016-08-02T18:59:44.230" Score="6" ViewCount="1425" Body="&lt;p&gt;Which deep neural network is used in &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_self-driving_car&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's driverless cars&lt;/a&gt; to analyze the surroundings? Is this information open to the public?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-13T18:53:56.720" LastActivityDate="2019-06-29T15:24:10.747" Title="Which machine learning algorithm is used in self-driving cars?" Tags="&lt;deep-neural-networks&gt;&lt;algorithm&gt;&lt;autonomous-vehicles&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="113" PostTypeId="1" AcceptedAnswerId="5572" CreationDate="2016-08-02T19:02:33.003" Score="10" ViewCount="1694" Body="&lt;p&gt;Two common activation functions used in deep learning are the hyperbolic tangent function and the sigmoid activation function. I understand that the hyperbolic tangent is just a rescaling and translation of the sigmoid function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\tanh(z) = 2\sigma(z) - 1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a significant difference between these two activation functions, and in particular, &lt;strong&gt;when is one preferable to the other&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that in some cases (like when estimating probabilities) outputs in the range of &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; are more convenient than outputs that range from &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt;. I want to know if there are differences &lt;strong&gt;other than convenience&lt;/strong&gt; which distinguish the two activation functions.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="2444" LastEditDate="2019-05-02T16:09:44.820" LastActivityDate="2019-05-02T16:09:44.820" Title="What's the difference between hyperbolic tangent and sigmoid neurons?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-neural-networks&gt;&lt;comparison&gt;&lt;hidden-layers&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="114" PostTypeId="2" ParentId="36" CreationDate="2016-08-02T19:02:42.300" Score="28" Body="&lt;p&gt;Quantum computers are super awesome at matrix multiplication, &lt;a href=&quot;http://twistedoakstudios.com/blog/Post8887_what-quantum-computers-do-faster-with-caveats&quot; rel=&quot;noreferrer&quot;&gt;with some limitations&lt;/a&gt;. Quantum superposition allows each bit to be in &lt;em&gt;a lot&lt;/em&gt; more states than just zero or one, and quantum gates can fiddle those bits in many different ways. Because of that, a quantum computer can process a lot of information at once for certain applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of those applications is the &lt;a href=&quot;http://algorithmicassertions.com/quantum/2014/03/07/Building-your-own-Quantum-Fourier-Transform.html&quot; rel=&quot;noreferrer&quot;&gt;Fourier transform&lt;/a&gt;, which is useful in a lot of problems, like &lt;a href=&quot;https://dsp.stackexchange.com/q/69&quot;&gt;signal analysis&lt;/a&gt; and array processing. There's also &lt;a href=&quot;http://twistedoakstudios.com/blog/Post2644_grovers-quantum-search-algorithm&quot; rel=&quot;noreferrer&quot;&gt;Grover's quantum search algorithm&lt;/a&gt;, which finds the single value for which a given function returns something different. If an AI problem can be expressed in a mathematical form &lt;a href=&quot;http://algorithmicassertions.com/quantum/2014/04/27/The-Not-Quantum-Laplace-Transform.html&quot; rel=&quot;noreferrer&quot;&gt;amenable to quantum computing&lt;/a&gt;, it can receive great speedups. Sufficient speedups could transform an AI idea from &quot;theoretically interesting but insanely slow&quot; to &quot;quite practical once we get a good handle on quantum computing.&quot;&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.797" LastActivityDate="2016-08-02T19:02:42.300" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="116" PostTypeId="2" ParentId="108" CreationDate="2016-08-02T19:16:46.910" Score="4" Body="&lt;p&gt;The advantage of a declarative language like Prolog is that it can be used to express facts and inference rules separately from control flow.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows the developer to focus on the data and inference rules (the knowledge model), and allows the developer to extend the knowledge model more easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that in practice, this dichotomy between facts/rules on the one hand, and control flow on the other, is not strict. A knowledge engineer who writes a code base in Prolog does sometimes have to consider control flow. The &quot;!&quot; operator is used so that the developer can influence the evaluation of the rules.&lt;/p&gt;&#xA;" OwnerUserId="66" LastActivityDate="2016-08-02T19:16:46.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="117" PostTypeId="2" ParentId="104" CreationDate="2016-08-02T19:18:06.670" Score="10" Body="&lt;p&gt;A good answer to this question depends on what you want to use the labels for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I think about &quot;optimization,&quot; I think about a solution space and a cost function; that is, there are many possible answers that could be returned and we can know what the cost is of any particular answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this view, the answer is &quot;yes&quot;--pattern recognition is a case where each pattern is a possible answer, and the optimization method is trying to find the one where the cost is lowest (that is, where the answer matches what you want it to match).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But most interesting optimization problems are characterized by exponential solution spaces and clean cost functions, and so can be thought of more as 'search' problems, whereas most pattern recognition problems are characterized by simple solution spaces and complicated cost functions, and it might feel unnatural to put the two of them together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In general, I do think that optimization and intelligence are deeply linked enough that optimization power is a good measure of intelligence, and certainly a better measure of the &lt;em&gt;practical&lt;/em&gt; use of intelligence than pattern recognition.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T19:18:06.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="118" PostTypeId="1" AcceptedAnswerId="122" CreationDate="2016-08-02T19:22:20.577" Score="8" ViewCount="421" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/10/what-is-fuzzy-logic&quot;&gt;Fuzzy logic&lt;/a&gt; is the logic where every statement can have any real truth value between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can fuzzy logic be used in creating AI? Is it useful for certain decision problems involving multiple inputs? Can you give an example of an AI that uses it?&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-02T20:10:13.770" Title="How can fuzzy logic be used in creating AI?" Tags="&lt;fuzzy-logic&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="119" PostTypeId="2" ParentId="113" CreationDate="2016-08-02T19:28:00.640" Score="3" Body="&lt;p&gt;I don't think it makes sense to decide activation functions based on desired properties of the output; you can easily insert a calibration step that maps the 'neural network score' to whatever units you actually want to use (dollars, probability, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think preference between different activation functions mostly boils down to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function#Comparison_of_activation_functions&quot; rel=&quot;nofollow&quot;&gt;different properties&lt;/a&gt; of those activation functions (like whether or not they're continuously differentiable). Because there's just a linear transformation between the two, I think that means there isn't a meaningful difference between them.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T19:28:00.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="120" PostTypeId="1" AcceptedAnswerId="125" CreationDate="2016-08-02T19:31:01.370" Score="5" ViewCount="221" Body="&lt;p&gt;In &lt;a href=&quot;https://philpapers.org/rec/LUCMMA-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minds, Machines and Gödel&lt;/a&gt; (1959), J. R. Lucas shows that any human mathematician can not be represented by an algorithmic automaton (a Turing Machine, but any computer is equivalent to it by the Church-Turing thesis), using Gödel's incompleteness theorem.&lt;/p&gt;&#xA;&lt;p&gt;As I understand it, he states that since the computer is an algorithm and hence a formal system, Gödel's incompleteness theorem applies. But a human mathematician also has to work in a formal axiom system to prove a theorem, so wouldn't it apply there as well?&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2021-02-02T01:07:51.347" LastActivityDate="2021-02-02T01:07:51.347" Title="How does Lucas's argument work?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;artificial-consciousness&gt;&lt;incompleteness-theorems&gt;&lt;turing-machine&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="121" PostTypeId="2" ParentId="118" CreationDate="2016-08-02T19:34:16.217" Score="2" Body="&lt;p&gt;My impression is that fuzzy logic has mostly declined in relevance and &lt;a href=&quot;https://en.wikipedia.org/wiki/Probabilistic_logic&quot; rel=&quot;nofollow&quot;&gt;probabilistic logic&lt;/a&gt; has taken over its niche. (See the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic#Comparison_to_probability&quot; rel=&quot;nofollow&quot;&gt;comparison on Wikipedia&lt;/a&gt;.) The two are somewhat deeply related, and so it's mostly a change in perspective and language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, fuzzy logic mostly applies to &lt;em&gt;labels&lt;/em&gt; which have &lt;em&gt;uncertain ranges&lt;/em&gt;. An object that's cool but not too cool &lt;em&gt;could&lt;/em&gt; be described as either cold or warm, and fuzzy logic handles this by assigning some fractional truth value to the 'cold' and 'warm' labels and no truth to the 'hot' label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probabilistic logic focuses more on the probability of some fact given some observations, and is deeply focused on the uncertainty of observations. When we look at an email, we track our belief that the email is &quot;spam&quot; and shouldn't be shown to the user with some number, and adjust that number as we see evidence for and against it being spam.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="10" LastEditDate="2016-08-02T20:10:13.770" LastActivityDate="2016-08-02T20:10:13.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="122" PostTypeId="2" ParentId="118" CreationDate="2016-08-02T19:34:26.923" Score="9" Body="&lt;p&gt;A classical example of fuzzy logic in an AI is the expert system Mycin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fuzzy logic can be used to deal with probabilities and uncertainties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one looks at, for example, predicate logic, then every statement is either true or false. In reality, we don't have this mathematical certainty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say a physician (or expert system) sees a symptom that can be attributed to a few different diseases (say A, B and C). The physician will now attribute a higher likelihood to the possibility of the patient having any of these three diseases. There is no definite true or false statement, but there is a change of weights. This can be reflected in fuzzy logic, but not so easily in symbolic logic.&lt;/p&gt;&#xA;" OwnerUserId="66" LastActivityDate="2016-08-02T19:34:26.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="123" PostTypeId="1" CreationDate="2016-08-02T19:42:07.160" Score="13" ViewCount="546" Body="&lt;p&gt;Back in college, I had a Complexity Theory teacher who stated that artificial intelligence was a contradiction in terms. If it could be calculated mechanically, he argued, it wasn't intelligence, it was math.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be a variant of the Chinese Room argument. This argument is a metaphor, where a person is put in a room full of Chinese books. This person doesn't understand a word of Chinese but is slipped messages in Chinese under the door. The person has to use the books, which contain transformation rules, to answer these messages. The person can apply the transformation rules but does not understand what (s)he is communicating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the Chinese room argument hold? Can we argue that artificial intelligence is merely clever algorithmics?&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="2444" LastEditDate="2021-02-15T23:45:03.590" LastActivityDate="2021-02-15T23:45:25.733" Title="Does the Chinese Room argument hold against AI?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;chinese-room-argument&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="124" PostTypeId="2" ParentId="120" CreationDate="2016-08-02T19:46:58.277" Score="2" Body="&lt;p&gt;After he lays out his argument, he deals with some counterarguments. The following looks like the weakest one to me:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We can use the same analogy also against those who, finding a formula their first machine cannot produce as being true, concede that that machine is indeed inadequate, but thereupon seek to construct a second, more adequate, machine, in which the formula can be produced as being true. This they can indeed do: but then the second machine will have a Gödelian formula all of its own, constructed by applying Gödel's procedure to the formal system which represents its (the second machine's) own, enlarged, scheme of operations. And this formula the second machine will not be able to produce as being true, while a mind will be able to see that it is true. And if now a third machine is constructed, able to do what the second machine was unable to do, exactly the same will happen: there will be yet a third formula, the Gödelian formula for the formal system corresponding to the third machine's scheme of operations, which the third machine is unable to produce as being true, while a mind will still be able to see that it is true. And so it will go on.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In short, by making the system more complex, it can see the inadequacy of a less complex system, but a yet more complex system can see its inadequacy. But from whence comes the claim that a mind &lt;em&gt;could&lt;/em&gt; see the inadequacy in the &lt;em&gt;n&lt;/em&gt;th machine? If, say, the Gödelian formula had as many components to it as a human brain had neurons, it seems suspect to claim that the human &lt;em&gt;could&lt;/em&gt; evaluate that formula and identify that it is in fact a Gödelian formula, rather than a similar but not quite identical sentence.  &lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T19:46:58.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="125" PostTypeId="2" ParentId="120" CreationDate="2016-08-02T19:49:46.050" Score="4" Body="&lt;p&gt;Yes, it applies. If a statement cannot be derived in a finite number of steps, then it doesn't matter if the person trying to prove it is a human or a computer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematician has one advantage over a standard theorem proving algorithm: the mathematician can &quot;step out of the system&quot; (as Douglas Hofstadter called in &lt;em&gt;G&amp;ouml;del, Escher, Bach&lt;/em&gt;), and start thinking &lt;em&gt;about&lt;/em&gt; the system. From this point of view, the mathematician may find that the derivation is impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, an AI for proving theorems could be programmed to recognize patterns in the derivation, just like our hypothetical mathematician, and start reasoning &lt;em&gt;about&lt;/em&gt; the formal system to derive properties of the formal system itself.&#xA;Both the AI and the mathematician would still be bound by the laws of mathematics, and not be able to prove a theorem if it was mathematically improvable.&lt;/p&gt;&#xA;" OwnerUserId="66" LastActivityDate="2016-08-02T19:49:46.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="126" PostTypeId="2" ParentId="123" CreationDate="2016-08-02T19:51:31.490" Score="3" Body="&lt;p&gt;Depends on who you ask! John Searle, who proposed this argument, would say &quot;yes&quot;, but others would say it is irrelevant. The Turing Test does not stipulate that a machine must actually &quot;understand&quot; what it is doing, as long as it seems that way to a human. You could argue that our &quot;thinking&quot; is only a more sophisticated form of clever algorithmics.&lt;/p&gt;&#xA;" OwnerUserId="148" LastEditorUserId="29" LastEditDate="2016-08-11T10:38:18.247" LastActivityDate="2016-08-11T10:38:18.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="127" PostTypeId="2" ParentId="123" CreationDate="2016-08-02T19:52:48.540" Score="8" Body="&lt;p&gt;There are two broad types of responses to philosophical queries like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first is to make analogies and refer to intuition; one could, for example, actually calculate the necessary size for such a Chinese room, and suggest that it exists outside the realm of intuition and thus any analogies using it are suspect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is to try to define the terms more precisely. If by &quot;intelligence&quot; we mean not &quot;the magic thing that humans do&quot; but &quot;information processing,&quot; then we can say &quot;yes, obviously the Chinese Room involves successful information processing.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to prefer the second because it forces conversations towards &lt;em&gt;observable outcomes&lt;/em&gt;, and puts the difficulty of defining a term like &quot;intelligence&quot; on the person who wants to make claims about it. If &quot;understanding&quot; is allowed to have an amorphous definition, then &lt;em&gt;any&lt;/em&gt; system could be said to have or not have understanding. But if &quot;understand&quot; is itself understood in terms of observable behavior, then it becomes increasingly difficult to construct an example of a system that &quot;is not intelligent&quot; and yet shares all the observable consequences of intelligence.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-02T19:52:48.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="128" PostTypeId="2" ParentId="123" CreationDate="2016-08-02T19:53:45.877" Score="8" Body="&lt;p&gt;It depends on the definition of (artificial) intelligence.&lt;/p&gt;&#xA;&lt;p&gt;The position that Searle originally tried to refute with the Chinese room experiment was the so-called position of strong AI: An appropriately programmed computer would have a mind in the exact same sense as humans have minds.&lt;/p&gt;&#xA;&lt;p&gt;Alan Turing tried to give a definition of artificial intelligence with the Turing Test, stating that a machine is intelligent if it can pass the test. The Turing Test is introduced &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I won't explain it in detail because it is not really relevant to the answer. If you define (artificial) intelligence as Turing did, then the Chinese room experiment is not valid.&lt;/p&gt;&#xA;&lt;p&gt;So the point of the Chinese room experiment is to show that an appropriately programmed computer is not the same as a human mind, and therefore that Turing's Test is not a good one.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2021-02-15T23:45:25.733" LastActivityDate="2021-02-15T23:45:25.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="130" PostTypeId="1" AcceptedAnswerId="135" CreationDate="2016-08-02T19:58:28.117" Score="3" ViewCount="113" Body="&lt;p&gt;What are the main differences between a &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep Boltzmann machine&lt;/a&gt; (DBM) (a recurrent neural network) and a &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_belief_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep belief network&lt;/a&gt; (which is based on RBMs)?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-05-18T12:01:28.283" LastActivityDate="2020-05-18T12:01:28.283" Title="What are the main differences between a deep Boltzmann machine and a deep belief network?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;boltzmann-machine&gt;&lt;deep-belief-network&gt;&lt;deep-boltzmann-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="131" PostTypeId="2" ParentId="77" CreationDate="2016-08-02T20:01:05.593" Score="10" Body="&lt;p&gt;Overall, the answer is no, but the current paradigms owe a lot to LISP. The language most commonly used today is python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relevant answers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stack Overflow thread explaining why LISP was thought of as the AI language: &lt;a href=&quot;https://stackoverflow.com/questions/130475/why-is-lisp-used-for-ai&quot;&gt;Why is Lisp used for AI&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Quora answer by Peter Norvig, who wrote a popular textbook on the subject and is currently Director of Research at Google: &lt;a href=&quot;https://www.quora.com/Is-it-true-that-Lisp-is-highly-used-programming-language-in-AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Is it true that Lisp is highly used programming language in AI?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;LISP pioneered many important concepts in what we now call functional programming, with a key attraction being how close the programs were to math. Many of these features have since been incorporated into modern languages (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Lisp_(programming_language)&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia page&lt;/a&gt;). LISP is very expressive: it has very little syntax (just lists and some elementary operations on them) but you can write short succinct programs that represent complex ideas. This amazes newcomers and has sold it as the language for AI. However, this is a property of programs in general. Short programs can represent complex concepts. And while you can write powerful code in LISP, any beginner will tell you that it is also very hard to read anyone else's LISP code or to debug your own LISP code. Initially, there were also performance considerations with functional programming and it fell out of favor to be replaced by low level imperative languages like C. (For example, functional programming requires that no object ever be changed (&quot;mutated&quot;), so every operation requires a new object to be created. Without good garbage collection, this can get unwieldy). Today, we've learned that a mix of functional and imperative programming is needed to write good code and modern languages like python, ruby and scala support both. At this point, and this is just my opinion, there is no reason to prefer LISP over python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paradigm for AI that currently receives the most attention is Machine Learning, where we learn from data, as opposed to previous approaches like Expert Systems (in the 80s) where experts wrote rules for the AI to follow. Python is currently the most widely used language for machine learning and has many libraries, e.g. Tensorflow and Pytorch, and an active community. To process the massive amounts of data, we need systems like Hadoop, Hive or Spark. Code for these is written in python, java or scala. Often, the core time-intensive subroutines are written in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI Winter of the 80s was not because we did not have the right language, but because we did not have the right algorithms, enough computational power and enough data. If you're trying to learn AI, spend your time studying algorithms and not languages.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="28225" LastEditDate="2019-10-01T00:00:52.223" LastActivityDate="2019-10-01T00:00:52.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="132" PostTypeId="2" ParentId="94" CreationDate="2016-08-02T20:01:55.600" Score="5" Body="&lt;p&gt;You described a single-layer feedforward network. They can have multiple layers. The significance of the weights is that they make a linear transformation from the output of the previous layer and hand it to the node they are going to. To say it more simplistically, they specify how important (and in what way: negative or positive) is the activation of node they are coming from to activating the node they are going to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, since there is only one layer (a row of input nodes and a row of output nodes) it is easy to explain what each node represents. However in multi-layer feedforward networks they can become abstract representations which makes it difficult to explain them and therefore explain what the weights that come to them or go out of them represent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way of thinking about it is that they describe hyperplanes in the space of the output of the previous node layer. If each output from the previous layer represents a point in space, a hyperplane decides which part of the space should give a positive value to the plane's corresponding node in the next layer and which part should give a negative input to it. It actually cuts that space into two halves. If you consider the input space of a multi-layer feedforward network, the weights of the first layer parametrize hyperplanes, however in the next layers they can represent non-linear surfaces in the input space.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="75" LastEditDate="2018-04-13T18:16:38.963" LastActivityDate="2018-04-13T18:16:38.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="134" PostTypeId="2" ParentId="111" CreationDate="2016-08-02T20:10:47.850" Score="53" Body="&lt;p&gt;The answer to a lot of those questions depends on how the device is programmed. A computer capable of driving around and recognizing where the road goes is likely to have the ability to visually distinguish a human from an animal, whether that be based on outline, image, or size. With sufficiently sharp image recognition, it might be able to count the number and kind of people in another vehicle. It could even use existing data on the likelihood of injury to people in different kinds of vehicles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, people disagree on the ethical choices involved. Perhaps there could be &quot;ethics settings&quot; for the user/owner to configure, like &quot;consider life count only&quot; vs. &quot;younger lives are more valuable.&quot; I personally would think it's not terribly controversial that a machine should damage itself before harming a human, but people disagree on how important pet lives are. If explicit kill-this-first settings make people uneasy, the answers could be determined from a questionnaire given to the user.&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="2989" LastEditDate="2016-10-14T09:49:20.523" LastActivityDate="2016-10-14T09:49:20.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="135" PostTypeId="2" ParentId="130" CreationDate="2016-08-02T20:12:35.360" Score="3" Body="&lt;p&gt;The graph that represents a deep Boltzmann machine can be any weighted undirected graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the graph that represents a deep Belief network must be a connection of graphs that represent restricted Boltzmann machines. Those graphs are bipartite, so there are two groups of vertices in those graphs so that every edge connects two vertices from different groups. Those groups are usually the visible and hidden components of the machine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning is hard and impractical in a general deep Boltzmann machine, but easier and practical in a restricted Boltzmann machine, and hence in a  deep Belief network, which is a connection of some of these machines. &lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2020-05-16T14:34:39.193" LastActivityDate="2020-05-16T14:34:39.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="136" PostTypeId="1" CreationDate="2016-08-02T20:15:22.637" Score="1" ViewCount="108" Body="&lt;p&gt;A cellular automaton is a state machine which is controlled by external input. The input is given by geometrical space around a cell. In a square matrix, each automaton gets input from 4 surrounding cells, while a hexagon grid has 6 neighbor cells which can be used as automaton input. For example, a 4-cells input may be the string “1011”. This string specify a state of the cellular automaton. The automaton will switch to a different state according to the lookup table. I want to know if increasing the number of input cells in a hexagon automaton will make the resulting computer more powerful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;original message&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to learn more about the differences between &lt;a href=&quot;https://en.wikipedia.org/wiki/Cellular_automaton#Related_automata&quot; rel=&quot;nofollow noreferrer&quot;&gt;related automata&lt;/a&gt; which can be based on hexagonal cells instead of squares (rule 34/2), like in &lt;a href=&quot;https://en.wikipedia.org/wiki/CoDi&quot; rel=&quot;nofollow noreferrer&quot;&gt;CoDi model&lt;/a&gt; which uses spiking neural network (SNN). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is using a plane tiled with regular &lt;a href=&quot;https://en.wikipedia.org/wiki/Hexagonal_tiling&quot; rel=&quot;nofollow noreferrer&quot;&gt;hexagons&lt;/a&gt; more efficient and reliable than using square cells? What is the difference and how do I know which one to use in which scenario?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In other words, the more efficiently flexible that it grows, the more difficult scenarios it can be used for (for me, hexagonal implicates more possibilities, because it can send/share the signal with/to more tiles). Or maybe one is more modern than the other, or they're both on the same level? In general, I'd like to learn the differences between them to know when I should use one over the other. &lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="1671" LastEditorDisplayName="user8248" LastEditDate="2018-08-24T19:01:12.483" LastActivityDate="2018-08-24T19:01:12.483" Title="Number of input variables for a cellular automaton (was: Squares or hexagonal?)" Tags="&lt;evolutionary-algorithms&gt;&lt;topology&gt;&lt;efficiency&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="137" PostTypeId="2" ParentId="123" CreationDate="2016-08-02T20:16:42.047" Score="6" Body="&lt;p&gt;First of all, for a detailed view of the argument, check out the &lt;a href=&quot;http://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;noreferrer&quot;&gt;SEP entry on the Chinese Room&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I consider the CRA as an indicator of you definition of intelligence. If the argument holds, yes, the person in the room understands Chinese. However, let's sum up the three replies discussed in the SEP entry:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;em&gt;man&lt;/em&gt; himself doesn't understand Chinese (he wouldn't be able to understand it when outside the room), but the &lt;em&gt;system&lt;/em&gt; man+room understands it. Accepting that reply suggests that there can exist an intelligent system which parts aren't themselves intelligent (which can be argued of the human body itself).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The system doesn't understand Chinese, as it cannot interact with the world in the same way a robot or a human could (i.e. it cannot learn, is limited in the set of questions it can answer)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The system doesn't understand Chinese (depending on your definition of &lt;em&gt;understanding&lt;/em&gt;), and you couldn't say a human performing the same feats as the Chinese room understands Chinese either.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So whether the argument, or a variant of it holds, depends on your definitions of &lt;em&gt;intelligent&lt;/em&gt;, &lt;em&gt;understanding&lt;/em&gt;, on how you define the system, etc. The point being that the thought experiment is a nice way to differentiate between the definitions (and many, many debates have been held about them), in order to avoid talking past each other endlessly.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2016-08-02T20:16:42.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="138" PostTypeId="2" ParentId="50" CreationDate="2016-08-02T20:32:12.480" Score="6" Body="&lt;p&gt;Generalization error is the error obtained by applying a model to data it has not seen before. So, if you want to measure generalization error, you need to remove a subset from your data and don't train your model on it. After training, you verify your model accuracy (or other performance measures) on the subset you have removed since your model hasn't seen it before. Hence, this subset is called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Test_set&quot; rel=&quot;nofollow noreferrer&quot;&gt;test set&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, another subset can also be used for parameter selection, which we call a &lt;a href=&quot;https://en.wikipedia.org/wiki/Test_set#Validation_set&quot; rel=&quot;nofollow noreferrer&quot;&gt;validation set&lt;/a&gt;. We can't use the training set for parameter tuning, since it does not measure generalization error, but we can't use the test set too since our parameter tuning would overfit test data. That's why we need a third subset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, in order to obtain more predictive performance measures, we can use many different train/test partitions and average the results. This is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-validation_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-validation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="14723" LastEditDate="2018-04-11T23:42:57.393" LastActivityDate="2018-04-11T23:42:57.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="139" PostTypeId="2" ParentId="108" CreationDate="2016-08-02T20:37:36.760" Score="5" Body="&lt;p&gt;There's no objective reason to state that declarative languages are better suited for AI development. However, there's indeed a bias towards them in practice. Although most functional languages are impure (that is, they allow side effects), and such can't count as &quot;declarative&quot;, a few languages are purely functional (that is, they don't allow side effects), most prominently &lt;a href=&quot;https://en.wikipedia.org/wiki/Declarative_programming#Functional_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haskell&lt;/a&gt;. Purity is key here. In Haskell, &lt;a href=&quot;https://stackoverflow.com/a/4066401/5249858&quot;&gt;even I/O is pure&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key difference between imperative languages and (purely) functional languages is in the way they describe the program. An imperative program describes &lt;em&gt;how&lt;/em&gt; to do stuff, that is, algorithms. It specifies the specific instructions that the machine must carry on in order to perform the computation. OTOH, purely functional languages describe &lt;em&gt;what&lt;/em&gt; is to be computed, that is, the relationship between the input and the output. In mathematics, &quot;function&quot; is just a fancy name for a relationship between an input and an output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, in the mathematical sense, the only variability is that of the function's arguments. That is, the function's output depends solely on its input (arguments). This is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Referential_transparency&quot; rel=&quot;nofollow noreferrer&quot;&gt;referential transparency&lt;/a&gt;. Referential transparency states that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall f \in \varphi, \forall x \in \delta_f, fx = fx&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\varphi$&lt;/span&gt; is the set of all functions, and &lt;span class=&quot;math-container&quot;&gt;$\delta_f$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;'s domain. For the typical imperative language's definition of &quot;function&quot;, the above doesn't hold. For instance, C's &lt;code&gt;getchar()&lt;/code&gt; does not always return the same value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we want to calculate the set of the ten least prime numbers whose least significant digit is &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;. First, in mathematical notation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{S} = \mathbb{G}_{10} = \{ x \mid x \in \{ p \mid p \in \mathbb{N} \setminus \{0, 1 \} \text{ and } (\forall q \in \mathbb{N} \setminus \{0, 1, p \}, p \text{ mod } q = 0) \},  x \text{ mod } 10 = 3 \}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;G(n, s)&lt;/code&gt; is the set of the lesser &lt;code&gt;n&lt;/code&gt;th elements from &lt;code&gt;s&lt;/code&gt;. In mathematics, you don't worry about how is the set &lt;span class=&quot;math-container&quot;&gt;$\mathbb{S}$&lt;/span&gt; supposed to be computed, but rather about &lt;span class=&quot;math-container&quot;&gt;$\mathbb{S}$&lt;/span&gt;'s definition itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in Python (in imperative style):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def is_prime(n):&#xA;  for x in range(2, n):&#xA;    if n % x == 0:&#xA;      return False&#xA;&#xA;  return True&#xA;&#xA;def foo():&#xA;  s = set()&#xA;  n = 2&#xA;  while len(s) &amp;lt; 10:&#xA;    if is_prime(n) and n % 10 == 3:&#xA;       s.append(n)&#xA;&#xA;  return set(s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In Python, we care about (and are responsible for) the algorithm being used to compute the set. We specify, pretty much in recipe-style, how to build the set from scratch. If there's an algorithm that may be better suited for checking whether a number is prime or odd, but we don't use it, it's our fault, not Python's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, Haskell steps in:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import qualified Data.Set as Set&#xA;&#xA;isPrime :: Integer -&amp;gt; Bool&#xA;isPrime n = ( == 1 ) . length . filter ( == 0 ) . map ( n `mod` ) $ [ 2 .. n ]&#xA;&#xA;s = Set.fromList . take 10 . filter ( ( == 3 ) . ( `mod` 10 ) ) . filter isPrime $ [1..]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Haskell's version is a lot more like the mathematical model than Python is. We define the &lt;code&gt;isPrime&lt;/code&gt; function in terms of the constraints that a prime number must obey, not by describing a step-by-step algorithm to do such a check. Moreover, &lt;code&gt;s&lt;/code&gt; (the set we have been defining so far) is defined in terms of the constraints its members must obey, rather than in terms of an algorithm to compute &lt;code&gt;s&lt;/code&gt; itself. The compiler, more often than not &lt;a href=&quot;https://downloads.haskell.org/~ghc/7.8.3/docs/html/users_guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Glorious Glasgow Haskell Compilation System&lt;/a&gt; (a.k.a GHC), is the one responsible for generating an algorithm. GHC's optimizer is known to be one of the strongest in the world, not because its the best compiler of 'em all, but because Haskell's nature allows for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Haskell (and other functional languages), in summary, have several features that make it look, taste, and behave like pure math:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Referential transparency and purity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lazy_evaluation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haskell is lazy&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/279316/what-exactly-makes-the-haskell-type-system-so-revered-vs-say-java&quot;&gt;A &lt;strong&gt;very&lt;/strong&gt; strong type system&lt;/a&gt;, with such exotic (but very useful!) stuff as &lt;a href=&quot;https://en.wikipedia.org/wiki/Recursive_data_type&quot; rel=&quot;nofollow noreferrer&quot;&gt;recursive&lt;/a&gt; (and &lt;a href=&quot;https://en.wikipedia.org/wiki/Algebraic_data_type&quot; rel=&quot;nofollow noreferrer&quot;&gt;algebraic&lt;/a&gt;) data types.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, the bottom line is that AI researchers often prefer functional languages over imperative languages (or, more assertively, pure over impure languages), because they are attempting to &lt;em&gt;define&lt;/em&gt; artificial intelligence itself, by means of functions (relationship between an input and an output). At the end, we do this because we have no real algorithm for human-level intelligence to raise from a handful of transistors. Also, there's been a historical bias towards these kind of languages, starting with &lt;a href=&quot;https://en.wikipedia.org/wiki/John_McCarthy_(computer_scientist)&quot; rel=&quot;nofollow noreferrer&quot;&gt;John McCarthy&lt;/a&gt;, Lisp's creator and a pioneer in early AI research.&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="2444" LastEditDate="2018-11-18T12:34:56.643" LastActivityDate="2018-11-18T12:34:56.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="140" PostTypeId="1" AcceptedAnswerId="144" CreationDate="2016-08-02T20:37:59.927" Score="8" ViewCount="214" Body="&lt;p&gt;A superintelligence is a machine that can surpass all intellectual activities by any human, and such a machine is often portrayed in science fiction as a machine that brings mankind to an end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any machine is executed using an algorithm. By the Church-Turing thesis, any algorithm that can be executed by a modern computer can be executed by a Turing Machine. However, a human can easily simulate a Turing Machine. Doesn't this mean that a machine can't surpass all intellectual activities, since we can also execute the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This argument is most likely flawed, since my intuition tells me that superintelligence is possible. However, it is not clear to me where the flaw is. Note that this is my own argument. &lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2019-06-25T21:16:52.410" LastActivityDate="2019-06-25T21:16:52.410" Title="Does this argument refuting the existence of superintelligence work?" Tags="&lt;philosophy&gt;&lt;superintelligence&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="141" PostTypeId="2" ParentId="74" CreationDate="2016-08-02T20:48:24.037" Score="36" Body="&lt;p&gt;The terms &lt;em&gt;strong&lt;/em&gt; and &lt;em&gt;weak&lt;/em&gt; don't actually refer to processing, or optimization power, or any interpretation leading to &quot;strong AI&quot; being &lt;em&gt;stronger&lt;/em&gt; than &quot;weak AI&quot;. It holds conveniently in practice, but the terms come from elsewhere. In 1980, &lt;a href=&quot;https://en.wikipedia.org/wiki/John_Searle&quot;&gt;John Searle&lt;/a&gt; coined the following statements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AI hypothesis, strong form: an AI system can &lt;em&gt;think&lt;/em&gt; and have a &lt;em&gt;mind&lt;/em&gt; (in the philosophical definition of the term);&lt;/li&gt;&#xA;&lt;li&gt;AI hypothesis, weak form: an AI system can only &lt;em&gt;act&lt;/em&gt; like it thinks and has a mind.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So &lt;em&gt;strong AI&lt;/em&gt; is a shortcut for an AI systems that verifies the &lt;em&gt;strong AI hypothesis&lt;/em&gt;. Similarly, for the weak form. The terms have then evolved: strong AI refers to AI that performs as well as humans (who have minds), weak AI refers to AI that doesn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with these definitions is that they're fuzzy. For example, &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot;&gt;AlphaGo&lt;/a&gt; is an example of weak AI, but is &quot;strong&quot; by Go-playing standards. A hypothetical AI replicating a human baby would be a strong AI, while being &quot;weak&quot; at most tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other terms exist: &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot;&gt;Artificial General Intelligence&lt;/a&gt; (AGI), which has cross-domain capability (like humans), can learn from a wide range of experiences (like humans), among other features. Artificial Narrow Intelligence refers to systems bound to a certain range of tasks (where they may nevertheless have superhuman ability), lacking capacity to significantly improve themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond AGI, we find Artificial Superintelligence (ASI), based on the idea that a system with the capabilities of an AGI, without the physical limitations of humans would learn and improve far beyond human level.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2016-08-02T20:48:24.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="142" PostTypeId="2" ParentId="16" CreationDate="2016-08-02T20:57:51.603" Score="7" Body="&lt;p&gt;In some iterative learning methods the more iterations you apply the more specific your model becomes about the training set. If there are too many iterations, your model will become too specifically trained for the training samples and will score less on other samples that are not seen during the training phase. This is call over-fitting, though over-fitting is not specific to iterative learning methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution to prevent over-fitting in these iterative learning algorithms is early stopping. Normally a control group of samples called validation samples (validation set) are used to validate the model and notify when it starts to over-fit. The validation set is not used by the training algorithm, however, its corresponding outputs are known and after each iteration, its samples are employed to measure how good the model currently works. As soon as the performance on the validation set stops to grow and starts to drop we stop iterating the training algorithm. This is called early stopping which can help to maximize the generalization power of our learned model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if we use the training set itself for validation the performance will always increase because that is what the learning algorithm is designed to do. However, the learning algorithm does not know how specifically it should learn the training set and that is why we need methods like early stopping.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="14723" LastEditDate="2018-04-12T02:34:26.373" LastActivityDate="2018-04-12T02:34:26.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="143" PostTypeId="2" ParentId="28" CreationDate="2016-08-02T21:01:10.770" Score="8" Body="&lt;ul&gt;&#xA;&lt;li&gt;An ability that is commonly attributed to intelligence is &lt;strong&gt;problem solving&lt;/strong&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Another one is &lt;strong&gt;learning&lt;/strong&gt; (improving itself from experience).&lt;/li&gt;&#xA;&lt;li&gt;Artificial intelligence can be defined as &quot;replicating intelligence, or parts of it, at least in appearance, inside a computer&quot; (dodging the definition of intelligence itself).&lt;/li&gt;&#xA;&lt;li&gt;Genetic algorithms are computational &lt;strong&gt;problem solving&lt;/strong&gt; tools that find and improve solutions (they &lt;strong&gt;learn&lt;/strong&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus, genetic algorithms are a kind of artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding scale, I don't see it as an important factor for defining G.A. as A.I or not. The same way we can simply classify different living forms as more or less intelligent instead of just saying intelligent or not intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, let's just make an important distinction: our brains are the product of natural selection, but the brains themselves don't use the same principle in order to achieve intelligence.&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2016-08-02T21:01:10.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="144" PostTypeId="2" ParentId="140" CreationDate="2016-08-02T21:06:19.000" Score="2" Body="&lt;p&gt;I believe this argument is based on the fact that intelligence is a single dimension when it really isn't. Are machines and humans really on the same level if a machine can solve a complex problem in a millionth of the time a human can? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also assumes that the Turing machine is still the best computational model for the time period that you are in, which is not necessarily true for the future, it is just true until this point in time. &lt;/p&gt;&#xA;" OwnerUserId="152" LastEditorUserId="29" LastEditDate="2016-08-05T13:35:09.893" LastActivityDate="2016-08-05T13:35:09.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="145" PostTypeId="1" CreationDate="2016-08-02T21:10:26.693" Score="14" ViewCount="4543" Body="&lt;p&gt;From Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AIXI ['ai̯k͡siː] is a theoretical mathematical formalism for artificial general intelligence. It combines Solomonoff induction with sequential decision theory. AIXI was first proposed by Marcus Hutter in 2000[1] and the results below are proved in Hutter's 2005 book Universal Artificial Intelligence.[2]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Albeit non-computable, approximations are possible, such as &lt;em&gt;AIXItl&lt;/em&gt;. Finding approximations to AIXI could be an objective way for solving AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &lt;em&gt;AIXI&lt;/em&gt; really a big deal in artificial &lt;em&gt;general&lt;/em&gt; intelligence research? Can it be thought as a central concept for the field? If so, why don't we have more publications on this subject (or maybe we have and I'm not aware of them)?&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="2444" LastEditDate="2019-04-19T15:13:02.567" LastActivityDate="2019-10-21T14:34:24.360" Title="What is the relevance of AIXI on current artificial intelligence research?" Tags="&lt;models&gt;&lt;agi&gt;&lt;aixi&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="146" PostTypeId="1" CreationDate="2016-08-02T21:14:36.133" Score="3" ViewCount="280" Body="&lt;p&gt;In what ways can connectionist artificial intelligence (neural networks) be integrated with &lt;em&gt;Good Old-Fashioned A.I.&lt;/em&gt; (&lt;em&gt;GOFAI&lt;/em&gt;)? For instance, how could deep neural networks be integrated with knowledge bases or logical inference? One such example seems to be the &lt;a href=&quot;http://wiki.opencog.org/w/DestinOpenCog&quot; rel=&quot;nofollow&quot;&gt;OpenCog + Destin integration&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="2444" LastEditDate="2021-01-24T22:25:29.530" LastActivityDate="2021-01-24T22:25:29.530" Title="In what ways can connectionist A.I. be integrated with GOFAI?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;symbolic-ai&gt;&lt;symbolic-computing&gt;&lt;connectionism&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="147" PostTypeId="1" CreationDate="2016-08-02T21:15:34.483" Score="2" ViewCount="272" Body="&lt;p&gt;It is proved that a recurrent neural net with rational weights can be a super-Turing machine. Can we achieve this in practice ?&lt;/p&gt;&#xA;" OwnerUserId="159" LastActivityDate="2016-09-03T17:38:06.577" Title="Can we ever achieve hypercomputation using recurrent neural networks?" Tags="&lt;neural-networks&gt;&lt;hypercomputation&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="148" PostTypeId="1" AcceptedAnswerId="170" CreationDate="2016-08-02T21:16:44.013" Score="19" ViewCount="1356" Body="&lt;p&gt;Given the proven &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot;&gt;halting problem&lt;/a&gt; for &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot;&gt;Turing machines&lt;/a&gt;, can we infer limits on the ability of strong Artificial Intelligence?&lt;/p&gt;&#xA;" OwnerUserId="55" LastEditorUserId="10" LastEditDate="2016-08-03T18:33:16.540" LastActivityDate="2019-03-06T11:00:52.413" Title="What limits, if any, does the halting problem put on Artificial Intelligence?" Tags="&lt;halting-problem&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="149" PostTypeId="2" ParentId="140" CreationDate="2016-08-02T21:18:16.763" Score="1" Body="&lt;p&gt;A quantum computer has a huge amount of internal state that even the machine can't get at directly. (You can only sample the matrix state.) The amount of that state goes up exponentially with each quantum bit involved in the system. Some operations get insane speedups from quantum computing: you just put the quantum wire through a quantum gate and you've updated the entire matrix at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulating a quantum computer with a classical one would take exponentially longer for each qubit. With several dozen qubits, the machine's computing power for some tasks couldn't even be approached by a normal computer, much less a human mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relevant: my answer on &lt;a href=&quot;https://ai.stackexchange.com/a/114/75&quot;&gt;To what extent can quantum computers help to develop Artificial Intelligence?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that with quantum computers, you've gone beyond the normal zeroes and ones. You then need a &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;quantum Turing machine&lt;/a&gt;, which is a generalization of the classical one.&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-02T21:18:16.763" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="151" PostTypeId="1" CreationDate="2016-08-02T21:25:25.313" Score="1" ViewCount="45" Body="&lt;p&gt;By default using &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepDream&quot; rel=&quot;nofollow&quot;&gt;DeepDream&lt;/a&gt; technique you can creating a dreamlike image out of two different images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to easily enhance this technique to generate one image out from three?&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-02T21:25:25.313" Title="Can DeepDream produce a &quot;dream&quot; from 3 images?" Tags="&lt;convolutional-neural-networks&gt;&lt;deepdream&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="152" PostTypeId="1" AcceptedAnswerId="1728" CreationDate="2016-08-02T21:34:32.107" Score="7" ViewCount="533" Body="&lt;p&gt;Consider these neural style algorithms which produce some art work:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/alexjc/neural-doodle&quot;&gt;Neural Doodle&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/jcjohnson/neural-style&quot;&gt;neural-style&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Why is generating such images so slow and why does it take huge amounts of memory? Isn't there any method of optimizing the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the mechanism or technical limitation behind this? Why we can't have a realtime processing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are few user comments (&lt;a href=&quot;https://www.reddit.com/r/deepdream/comments/3jwl76/how_anyone_can_create_deep_style_images/&quot;&gt;How ANYONE can create Deep Style images&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;Anything above 640x480 and we're talking days of heavy crunching and an insane amount of ram.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;I tried doing a 1024pixel image and it still crashed with 14gigs memory, and 26gigs swap. So most of the VM space is just the swapfile. Plus it takes several hours potentially days cpu rendering this.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;I tried 1024x768 and with 16gig ram and 20+ gig swap it was still dying from lack of memory.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;Having a memory issue, though. I'm using the &quot;g2.8xlarge&quot; instance type.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-18T13:24:43.510" LastActivityDate="2018-04-15T04:16:38.293" Title="Why is the generation of deep style images so slow and resource-hungry?" Tags="&lt;performance&gt;&lt;neural-doodle&gt;&lt;deepdreaming&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="153" PostTypeId="1" CreationDate="2016-08-02T21:36:28.053" Score="9" ViewCount="2447" Body="&lt;p&gt;Can autoencoders be used for supervised learning &lt;em&gt;without adding an output layer&lt;/em&gt;? Can we simply feed it with a concatenated input-output vector for training, and reconstruct the output part from the input part when doing inference? The output part would be treated as missing values during inference and some imputation would be applied.&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="7496" LastEditDate="2017-06-17T21:22:44.503" LastActivityDate="2017-06-17T21:22:44.503" Title="Can autoencoders be used for supervised learning?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="154" PostTypeId="1" AcceptedAnswerId="158" CreationDate="2016-08-02T21:37:32.420" Score="29" ViewCount="20175" Body="&lt;p&gt;I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So given the 3 inputs: 1st number, operator sign represented by the number (1 - &lt;code&gt;+&lt;/code&gt;, 2 - &lt;code&gt;-&lt;/code&gt;, 3 - &lt;code&gt;/&lt;/code&gt;, 4 - &lt;code&gt;*&lt;/code&gt;, and so on), and the 2nd number, then after training the network should give me the valid results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 1 (&lt;code&gt;2+2&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input 1: &lt;code&gt;2&lt;/code&gt;; Input 2: &lt;code&gt;1&lt;/code&gt; (&lt;code&gt;+&lt;/code&gt;); Input 3: &lt;code&gt;2&lt;/code&gt;; Expected output: &lt;code&gt;4&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Input 1: &lt;code&gt;10&lt;/code&gt;; Input 2: &lt;code&gt;2&lt;/code&gt; (&lt;code&gt;-&lt;/code&gt;); Input 3: &lt;code&gt;10&lt;/code&gt;; Expected output: &lt;code&gt;0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Input 1: &lt;code&gt;5&lt;/code&gt;; Input 2: &lt;code&gt;4&lt;/code&gt; (&lt;code&gt;*&lt;/code&gt;); Input 3: &lt;code&gt;5&lt;/code&gt;; Expected output: &lt;code&gt;25&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;and so&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The above can be extended to more sophisticated examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that possible? If so, what kind of network can learn/achieve that?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2017-03-09T17:03:38.327" LastActivityDate="2019-01-22T18:45:30.140" Title="Is it possible to train the neural network to solve math equations?" Tags="&lt;neural-networks&gt;&lt;math&gt;" AnswerCount="5" CommentCount="3" FavoriteCount="18" ContentLicense="CC BY-SA 3.0" />
  <row Id="155" PostTypeId="2" ParentId="154" CreationDate="2016-08-02T21:57:56.160" Score="9" Body="&lt;p&gt;Not really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks are good for determining non-linear relationships between inputs when there are hidden variables. In the examples above the relationships are linear, and there are no hidden variables. But even if they were non-linear, a traditional ANN design would not be well suited to accomplish this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By carefully constructing the layers and tightly supervising the training, you could get a network to consistently produce the output 4.01, say, for the inputs: 2, 1 (+), and 2, but this is not only wrong, it's an inherently unreliable application of the technology.&lt;/p&gt;&#xA;" OwnerUserId="46" LastActivityDate="2016-08-02T21:57:56.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="156" PostTypeId="1" CreationDate="2016-08-02T21:59:01.093" Score="19" ViewCount="1618" Body="&lt;p&gt;From Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A mirror neuron is a neuron that fires both when an animal acts and when the animal observes the same action performed by another.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Mirror neurons are related to imitation learning, a very useful feature that is missing in current real-world A.I. implementations. Instead of learning from input-output examples (supervised learning) or from rewards (reinforcement learning), an agent with mirror neurons would be able to learn by simply observing other agents, translating their movements to its own coordinate system. What do we have on this subject regarding computational models?&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2016-10-14T13:08:35.790" Title="Are there any computational models of mirror neurons?" Tags="&lt;neural-networks&gt;&lt;models&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="157" PostTypeId="1" AcceptedAnswerId="267" CreationDate="2016-08-02T22:38:50.823" Score="8" ViewCount="148" Body="&lt;p&gt;If I have a paragraph I want to summarize, for example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ponzo and Fila went to the mall during the day. They walked for a long while, stopping at shops. They went to many shops. At first, they didn't buy anything. After going to a number of shops, they eventually bought a shirt, and a pair of pants.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Better summarized as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;They shopped at the mall today and bought some clothes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is the best AI strategy to automate this process, if there is one? If there isn't, is it because it would be dependent on first having an external information resource that would inform any algorithm? Or is it because the problem is inherently contextual?&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="4302" LastEditDate="2018-10-08T12:44:30.960" LastActivityDate="2018-10-08T12:44:30.960" Title="What artificial intelligence strategies are useful for summarization?" Tags="&lt;algorithm&gt;&lt;natural-language-processing&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="158" PostTypeId="2" ParentId="154" CreationDate="2016-08-02T22:50:52.560" Score="23" Body="&lt;p&gt;Yes, it has been done!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the applications aren't to replace calculators or anything like that. The lab I'm associated with develops neural network models of equational reasoning to better understand how humans might solve these problems. This is a part of the field known as &lt;a href=&quot;http://archive.is/mt4RO&quot; rel=&quot;noreferrer&quot;&gt;Mathematical Cognition&lt;/a&gt;. Unfortunately, our website isn't terribly informative, but here's a &lt;a href=&quot;http://web.stanford.edu/~kmickey/pdf/MickeyMcClelland2014.pdf&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt; to an example of such work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from that, recent work on extending neural networks to include external memory stores (e.g. Neural Turing Machines) tend to use solving math problems as a good proof of concept. This is because many arithmetic problems involve long procedures with stored intermediate results. See the sections of &lt;a href=&quot;http://arxiv.org/pdf/1511.08228.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; on long binary addition and multiplication.&lt;/p&gt;&#xA;" OwnerUserId="109" LastEditorUserId="8" LastEditDate="2018-09-20T22:37:16.240" LastActivityDate="2018-09-20T22:37:16.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="159" PostTypeId="1" AcceptedAnswerId="238" CreationDate="2016-08-02T23:01:56.157" Score="1" ViewCount="474" Body="&lt;p&gt;What happens if you apply the same &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepDream&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep dream technique&lt;/a&gt; which produces &quot;dream&quot; visuals but to media streams such as audio files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does changing image functions into audio and enhancing the logic would work, or will it no longer work/doesn't make any sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to create &quot;dream&quot; like audio based on the two samples.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-14T18:06:52.290" LastActivityDate="2018-04-14T18:06:52.290" Title="Is it possible to apply deep dream technique for the audio streams?" Tags="&lt;convolutional-neural-networks&gt;&lt;deepdreaming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="161" PostTypeId="2" ParentId="111" CreationDate="2016-08-02T23:31:57.630" Score="16" Body="&lt;p&gt;This is the well known &lt;a href=&quot;https://en.wikipedia.org/wiki/Trolley_problem&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Trolley Problem&lt;/em&gt;&lt;/a&gt;. As &lt;a href=&quot;https://ai.stackexchange.com/a/134/8&quot;&gt;Ben N&lt;/a&gt; said, people disagree on the right course of action for trolley problem scenarios, but it should be noted that with self-driving cars, reliability is so high that these scenarios are really unlikely. So, not much effort will be put into the problems you are describing, at least in the short term.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-04T00:19:36.570" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="162" PostTypeId="2" ParentId="111" CreationDate="2016-08-02T23:44:57.983" Score="9" Body="&lt;p&gt;For a driverless car that is designed by a single entity, the best way for it to make decisions about whom to kill is by estimating and minimizing the probable liability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't need to absolutely correctly identify all the potential victims in the area to have a defense for its decision, only to identify them as well as a human could be expected to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't even need to know the age and physical condition of everyone in the car, as it can ask for that information and if refused, has the defense that the passengers chose not to provide it, and therefore took responsibility for depriving it of the ability to make a better decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It only has to have a viable model for minimizing exposure of the entity to lawsuits, which can then be improved over time to make it more profitable.&lt;/p&gt;&#xA;" OwnerUserId="46" LastActivityDate="2016-08-02T23:44:57.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="163" PostTypeId="2" ParentId="13" CreationDate="2016-08-02T23:56:02.343" Score="6" Body="&lt;p&gt;Well, I do not know what type of features you are giving to your neural network. However, in general, I would go with a single neural network. It seems that you have no limitation in resources for training your network and the only problem is resources while you apply your network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that probably the two problems have things in common (e.g. both types of plates are rectangular). This means that if you use two networks, each has to solve the same sub-problem (the common part) again. If you use only one network the common part of the problem takes fewer cells/weights to be solved and the remaining weights/cells can be employed for better recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, if I was in your place I would try both of them. I think that is the only way to be really sure what is the best solution. When speaking theoretically it is possible that we do not include some factors.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="14723" LastEditDate="2018-04-12T02:23:00.733" LastActivityDate="2018-04-12T02:23:00.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="164" PostTypeId="2" ParentId="77" CreationDate="2016-08-03T00:16:14.550" Score="5" Body="&lt;p&gt;LISP is still used significantly, but less and less. There is still momentum due to so many people using it in the past, who are still active in the industry or research (anecdote: the last VCR was produced by a Japanese maker in July 2016, yes). The language is however used (to my knowledge) for the kind of AI that does not leverage Machine Learning, typically as the reference books from Russell and Norvig. These applications are still very useful, but Machine Learning gets all the steam these days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another reason for the decline is that LISP practitioners have partially moved to Clojure and other recent languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are learning about AI technologies, LISP (or Scheme or Prolog) is good choice to understand what is going on with &quot;AI&quot; at large. But if you wish or have to be very pragmatic, Python or R are the community choices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The above lacks concrete example and reference. I am aware of some work in universities, and some companies inspired by or directly using LISP.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To add on @Harsh's answer, LISP (and Scheme, and Prolog) has qualities that made it look like it was better suited for creating intelligent mechanisms---making AI as perceived in the 60s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the qualities was that the language design leads the developer to think in a quite elegant way, to decompose a big problem into small problems, etc. Quite &quot;clever&quot;, or &quot;intelligent&quot; if you will. Compared to some other languages, there is almost no choice but to develop that way. LISP is a list processing language, and &quot;purely functional&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem, though, can be seen in work related to LISP. A notable one in the AI domain is the work on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Situation_calculus&quot; rel=&quot;noreferrer&quot;&gt;Situation Calculus&lt;/a&gt;, where (in short) one describes objects and rules in a &quot;world&quot;, and can let it evolve to compute &lt;em&gt;situations&lt;/em&gt;---states of the world. So it is a model for reasoning on situations. The main problem is called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_problem&quot; rel=&quot;noreferrer&quot;&gt;frame problem&lt;/a&gt;, meaning this calculus cannot tell what does &lt;em&gt;not&lt;/em&gt; change---just what changes. Anything that is not defined in the world cannot be processed (note the difference here with ML). First implementations used LISPs, because that was the AI language then. And there were bound by the frame problem. But, as @Harsh mentioned, it is not LISP's fault: Any language would face the same framing issue (a conceptual problem of the Situation Calculus).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the language really does not matter from the AI / AGI / ASI perspective. The concepts (algorithms, etc.) are really what matters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in Machine Learning, the language is just a practical choice. Python and R are popular today, primarily due to their library ecosystem and the focus of key companies. But try to use Python or R to run a model for a RaspberryPI-based application, and you will face some severe limitations (but still possible, I am doing it :-)). So the language choice burns down to pragmatism.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2016-08-03T00:16:14.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="165" PostTypeId="2" ParentId="140" CreationDate="2016-08-03T00:40:35.383" Score="0" Body="&lt;p&gt;The flaw in your argument is that &quot;surpass&quot; doesn't just mean that you should be able to run all algorithms, it includes a notion of complexity, i.e. how many time steps you will take to simulate an algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you simulate an algorithm with a Turing machine? A &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing machine&lt;/a&gt; consists of a finite state machine and an infinite tape. A Turing Machine does run an algorithm, determined by its initial state and the state transition matrix, but what I think you are talking about is Universal Turing Machines (UTM)  that can read &quot;code&quot; (which is usually a description of another Turing machine) written on a &quot;code segment&quot; of the tape and then simulate that machine on input data written on the &quot;data segment&quot; of the tape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing machines can differ in the number of states in their finite state machines (and also in the alphabet they write on the tape but any finite alphabet is easily encoded in binary so this should not be the big reason for differences among Turing machines). So, you can have UTMs with bigger state machines and UTMs with smaller state machines. The bigger UTM could possibly surpass the smaller one if they use the same encoding for the &quot;code&quot; part of the tape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also play around with the code used to describe the TM being simulated. This code could be C++, for example, or could be a Neural network with the synapse strength written down as a matrix. Which description is better for computation depends on the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example comparison among UTMs with different state machines: consider different compilers for the same language, say C++. Both of them will first compile C++ to assembly and then run another UTM which reads and executes assembly (your physical CPU). So, a better compiler will run the same code faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back to humans vs computers, humans are neural networks that run algorithms like those you would write in C++. This involves a costly and inefficient conversion of the algorithm into hand movements. A computer uses a compiler to convert C++ to assembly that it can run natively, so it's able to do a much more efficient implementation of C++ code. Alternately, humans have a ton of neurons, and the neural code, i.e. synapse strength, is hard to read, so current computers cannot run that code yet.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="14723" LastEditDate="2018-04-15T04:14:15.697" LastActivityDate="2018-04-15T04:14:15.697" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="166" PostTypeId="2" ParentId="77" CreationDate="2016-08-03T01:08:53.837" Score="6" Body="&lt;p&gt;I definitely continue to often use Lisp when working on AI models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You asked if it is being used for &lt;em&gt;substantial&lt;/em&gt; work.  That's too subjective for me to answer regarding my own work, but I queried one my AI models whether or not it considered itself substantial, and it replied with an affirmative response.  Of course, it's response is naturally biased as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, a significant amount of AI research and development is conducted in Lisp.  Furthermore, even for non-AI problems, Lisp is sometimes used.  To demonstrate the power of Lisp, I engineered the first neural network simulation system written entirely in Lisp over a quarter century ago.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2016-08-03T01:08:53.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="167" PostTypeId="1" AcceptedAnswerId="174" CreationDate="2016-08-03T01:55:30.377" Score="4" ViewCount="170" Body="&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepDream&quot; rel=&quot;nofollow&quot;&gt;DeepDream&lt;/a&gt; wikipedia page it's suggested that a dreamlike images created by a convolutional neural network may be related to how visual cortex works in humans when they're tripping.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The imagery to LSD- and psilocybin-induced hallucinations is suggestive of a functional resemblance between artificial neural networks and particular layers of the visual cortex.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How this is even possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly convolutional neural networks have anything to do with human visual cortex?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-18T11:34:48.770" LastActivityDate="2016-08-18T11:34:48.770" Title="Why would neural network dream scenes mirror the hallucinations people experience when they're tripping?" Tags="&lt;convolutional-neural-networks&gt;&lt;deepdream&gt;&lt;computer-vision&gt;&lt;deepdreaming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="169" PostTypeId="1" AcceptedAnswerId="176" CreationDate="2016-08-03T02:12:37.943" Score="1" ViewCount="91" Body="&lt;p&gt;This 2014 &lt;a href=&quot;https://medium.com/the-physics-arxiv-blog/first-demonstration-of-artificial-intelligence-on-a-quantum-computer-17a6b9d1c5fb&quot; rel=&quot;nofollow&quot;&gt;article&lt;/a&gt; saying that a Chinese team of physicists have trained a quantum computer to recognise handwritten characters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why did they have to use a quantum computer&lt;/strong&gt; to do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it just for fun and demonstration, or is it that recognising the handwritten characters is so difficult that standard (non-quantum) computers or algorithms cannot do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If standard computers can achieve the same thing, what are the benefits of using quantum computers to do that then over standard methods?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-06T00:07:31.233" LastActivityDate="2016-08-06T00:07:31.233" Title="What are the challenges for recognising the handwritten characters?" Tags="&lt;quantum-computing&gt;&lt;handwritten-characters&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="170" PostTypeId="2" ParentId="148" CreationDate="2016-08-03T02:17:01.983" Score="10" Body="&lt;p&gt;Does the halting problem imply any limits on human cognition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, absolutely--that there are pieces of code a human could look at and not be sure whether or not it will halt in finite time. (Certainly there are pieces of code that a human can look at and say &quot;yes&quot; or &quot;no&quot; definitely, but we're talking about the ones that are actually quite difficult to analyze.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The halting problem means that there are types of code analysis that no computer could do, because it's mathematically impossible. But the realm of &lt;em&gt;possibility&lt;/em&gt; is still large enough to allow strong artificial intelligence (in the sense of code that can understand itself well enough to improve itself).&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-03T02:17:01.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="171" PostTypeId="2" ParentId="74" CreationDate="2016-08-03T02:23:28.267" Score="8" Body="&lt;p&gt;In contrast to the &lt;em&gt;philosophical&lt;/em&gt; definitions, which rely on terms like &quot;mind&quot; and &quot;think,&quot; there are also definitions that hinge on &lt;em&gt;observables&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, a Strong AI is an AI that understands itself well enough to self-improve. Even if it is philosophically not equivalent to a human, or unable to perform &lt;em&gt;all&lt;/em&gt; cognitive tasks that a human can, this AI can still generate a tremendous amount of optimization power / good decision-making, and its creation would be of historic importance (to put it lightly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Weak AI, in contrast, is an AI with no or limited ability to self-modify. A chessbot that runs on your laptop might have superhuman ability to play chess, but it can &lt;em&gt;only&lt;/em&gt; play chess, and while it might tune its weights or its architecture and slowly improve, it cannot modify itself in a deep enough way to generalize to other tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to think about this is that a Strong AI is an AI researcher in its own right, and a Weak AI is what AI researchers produce.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-03T02:23:28.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="172" PostTypeId="1" CreationDate="2016-08-03T03:33:35.193" Score="2" ViewCount="99" Body="&lt;p&gt;Is it possible that, at some time in the future, AIs will be able to initiatively develop themselves, rather than passively being developed by humanity?&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="2444" LastEditDate="2019-06-20T20:43:42.113" LastActivityDate="2019-06-20T20:48:44.383" Title="Could AIs self-develop in the future?" Tags="&lt;philosophy&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="173" PostTypeId="2" ParentId="148" CreationDate="2016-08-03T04:22:28.030" Score="12" Body="&lt;p&gt;The halting problem is an example of a general phenomenon known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Undecidable_problem&quot; rel=&quot;noreferrer&quot;&gt;Undecidability&lt;/a&gt;, which shows that there are problems no Turing machine can solve in finite time. Let's consider the generalization that it is undecidable whether a Turing Machine satisfies some non-trivial property P, called &lt;a href=&quot;https://en.wikipedia.org/wiki/Rice%27s_theorem&quot; rel=&quot;noreferrer&quot;&gt;Rice's theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First note that the halting problem applies only if the Turing machine takes arbitrarily long input. If the input is bounded, it is possible to enumerate all possible cases and the problem is no longer undecidable. It might still be inefficient to calculate it, but then we are turning to the complexity theory, which should be a separate question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rice's theorem implies that an intelligence (a human) cannot be able to determine whether another intelligence (such as an AGI) possesses a certain property, such as being &lt;a href=&quot;https://en.wikipedia.org/wiki/Friendly_artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;friendly&lt;/a&gt;. This does not mean that we cannot design a Friendly AGI, but it does mean that we cannot check whether an arbitrary AGI is friendly. So, while we can possibly create an AI which is guaranteed to be friendly, we also need to ensure that IT cannot create another AI which is unfriendly.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="169" LastEditDate="2016-08-03T18:25:39.313" LastActivityDate="2016-08-03T18:25:39.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="174" PostTypeId="2" ParentId="167" CreationDate="2016-08-03T04:27:10.673" Score="7" Body="&lt;p&gt;The similarity of artificial neural networks and the human visual cortex goes very deep, and in many ways the human visual cortex was the inspiration for the techniques we use for the design and implementation of ANNs designed for image recognition. So in that direction, the similarity seems obvious to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reverse direction, though, is a question about how the human mind works under the influence of LSD, which you'll probably get a better answer asking about in the &lt;a href=&quot;https://biology.stackexchange.com/&quot;&gt;biology&lt;/a&gt; or &lt;a href=&quot;https://cogsci.stackexchange.com/&quot;&gt;cognitive science&lt;/a&gt; stack exchange sites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some brief details to add to the answer, though: the human visual cortex is arranged in layers that correspond to increasing layers of abstraction. In the eyes themselves, photons are detected by light-sensitive cells and added together to make what are essentially the color elements of pixels. Those are then routed to another layer which does something like edge detection, and then the next layer does something like shape detection, and so on up to higher level concepts like &quot;a cat's face.' If LSD lowers the activation threshold for those neurons, or makes them more excitable, then more things will be interpreted as having the higher level concept (and so a patch of rough texture may have a face jump out of it, for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way that CNN &quot;deep dreaming' works is that the base image is amplified. That is, to make a particular patch look more like a dog, the shapes are nudged to be more dog-like, and the shapes nudge the edges, and the edges nudge the pixels.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:17.300" LastActivityDate="2016-08-03T04:27:10.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="176" PostTypeId="2" ParentId="169" CreationDate="2016-08-03T04:41:50.923" Score="6" Body="&lt;p&gt;Handwritten digit recognition is a standard benchmark in Machine Learning in the form of the &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database&quot;&gt;MNIST dataset&lt;/a&gt;. For example, &lt;a href=&quot;http://scikit-learn.org/stable/auto_examples/classification/plot_digits_classification.html&quot;&gt;scikit-learn&lt;/a&gt;, a python package for Machine Learning uses it as a tutorial example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper you cite uses this standard task as a proof of concept, to show that their system works.&lt;/p&gt;&#xA;" OwnerUserId="130" LastActivityDate="2016-08-03T04:41:50.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="177" PostTypeId="2" ParentId="26" CreationDate="2016-08-03T04:48:19.643" Score="6" Body="&lt;p&gt;I think your question fits nowadays more in the field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Human%E2%80%93robot_interaction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human-Robot Interaction&lt;/a&gt;, which relies largely on &lt;a href=&quot;http://nordicapis.com/20-emotion-recognition-apis-that-will-leave-you-impressed-and-concerned/&quot; rel=&quot;nofollow noreferrer&quot;&gt;vision&lt;/a&gt; for recognition of gestures and follow movements, as well as &lt;em&gt;soft, natural&lt;/em&gt; movements as a response. Note that the movements of the face and hands belong to the most complex tasks, involving &lt;em&gt;many&lt;/em&gt; muscles at a time.&lt;/p&gt;&#xA;&lt;p&gt;I strongly recommend the film &lt;a href=&quot;http://www.plugandpray-film.de/en/trailer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plug &amp;amp; Pray&lt;/a&gt; to have an idea of what people are researching in this area.&lt;/p&gt;&#xA;&lt;p&gt;You may also find Eliza (which you can try &lt;a href=&quot;https://web.njit.edu/%7Eronkowit/eliza.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) interesting. It is classical in the history of AI and pretends to mimic an analyst (psychology). (I am thinking of Eliza not because of its emotional intelligence, but because it was &lt;a href=&quot;http://www.alicebot.org/articles/wallace/eliza.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;apparently taken seriously&lt;/a&gt; by a couple of humans. Could this be taken as a sort of (approved) Turing test? What does it say about the humans it met?)&lt;/p&gt;&#xA;&lt;p&gt;On the &lt;em&gt;purely human&lt;/em&gt; end of the scale, I sometimes wonder about our (my) emotional intelligence myself. Would I want to implement such an intelligence in an artificial agent at all?&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="2444" LastEditDate="2021-02-01T01:03:23.217" LastActivityDate="2021-02-01T01:03:23.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="178" PostTypeId="2" ParentId="172" CreationDate="2016-08-03T04:52:30.067" Score="2" Body="&lt;p&gt;This is known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;intelligence explosion&lt;/a&gt; hypothesis or &lt;a href=&quot;https://wiki.lesswrong.com/wiki/Recursive_self-improvement&quot; rel=&quot;nofollow noreferrer&quot;&gt;recursive self-improvement&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="2444" LastEditDate="2019-06-20T20:46:22.590" LastActivityDate="2019-06-20T20:46:22.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="179" PostTypeId="1" CreationDate="2016-08-03T05:15:39.443" Score="6" ViewCount="664" Body="&lt;p&gt;I have been wondering since a while ago about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Theory_of_multiple_intelligences&quot; rel=&quot;nofollow noreferrer&quot;&gt;theory of multiple intelligences&lt;/a&gt; and how they could fit in the field of Artificial Intelligence as a whole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We hear from time to time about &lt;a href=&quot;https://www.theguardian.com/artanddesign/jonathanjonesblog/2016/feb/08/leonardo-da-vinci-mechanics-of-genius-science-museum-london&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leonardo Da Vinci&lt;/a&gt; being a genius or &lt;a href=&quot;https://www.youtube.com/watch?v=xUHQ2ybTejU&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bach's musical intelligence&lt;/a&gt;. These persons are commonly said to be (have been) &lt;em&gt;more intelligent&lt;/em&gt;. But the multiple intelligences speak about cooking or dancing or chatting as well, i.e. &lt;em&gt;coping with everyday tasks&lt;/em&gt; (at least that's my interpretation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there some approaches on incorporating multiple intelligences into AI?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a related question: &lt;a href=&quot;https://ai.stackexchange.com/q/26/2444&quot;&gt;How could emotional intelligence be implemented?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="2444" LastEditDate="2019-12-21T00:40:52.520" LastActivityDate="2019-12-21T00:40:52.520" Title="How can the multiple intelligences model be incorporated into AI?" Tags="&lt;definitions&gt;&lt;theory&gt;&lt;emotional-intelligence&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="180" PostTypeId="1" CreationDate="2016-08-03T05:23:01.403" Score="2" ViewCount="70" Body="&lt;p&gt;Which algorithms are there to create word embeddings for a given language?&lt;/p&gt;&#xA;" OwnerUserId="202" LastEditorUserId="2444" LastEditDate="2019-04-16T22:11:59.630" LastActivityDate="2019-04-16T22:11:59.630" Title="How can we build word embeddings for a language?" Tags="&lt;natural-language-processing&gt;&lt;algorithm&gt;&lt;word-embedding&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2016-11-10T22:25:54.347" ContentLicense="CC BY-SA 4.0" />
  <row Id="182" PostTypeId="1" AcceptedAnswerId="183" CreationDate="2016-08-03T05:31:30.353" Score="16" ViewCount="288" Body="&lt;p&gt;How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?&lt;/p&gt;&#xA;" OwnerUserId="202" LastEditorUserId="2444" LastEditDate="2020-04-12T18:54:07.443" LastActivityDate="2020-04-12T18:54:07.443" Title="How do I decide the optimal number of layers for a neural network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;hyperparameter-optimization&gt;&lt;hidden-layers&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="183" PostTypeId="2" ParentId="182" CreationDate="2016-08-03T05:35:48.807" Score="6" Body="&lt;p&gt;There is a technique called &lt;code&gt;Pruning&lt;/code&gt; in neural networks, which is used just for this same purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pruning is done on the number of hidden layers. The process is very similar to the pruning process of decision trees. The pruning process is done as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train a large, densely connected, network with a standard training&#xA;algorithm&lt;/li&gt;&#xA;&lt;li&gt;Examine the trained network to assess the relative importance of the&#xA;weights&lt;/li&gt;&#xA;&lt;li&gt;Remove the least important weight(s)&lt;/li&gt;&#xA;&lt;li&gt;retrain the pruned network&lt;/li&gt;&#xA;&lt;li&gt;Repeat steps 2-4 until satisfied&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, there are &lt;a href=&quot;https://arxiv.org/abs/1510.00149&quot;&gt;several optimized methods&lt;/a&gt; for pruning neural nets, and it is also a &lt;a href=&quot;http://www.idiap.ch/ftp/reports/1997/rr97-03.pdf&quot;&gt;very active area of research&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2016-08-03T05:35:48.807" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="184" PostTypeId="1" CreationDate="2016-08-03T06:03:28.903" Score="4" ViewCount="50" Body="&lt;p&gt;I am interested in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Emergence&quot; rel=&quot;nofollow&quot;&gt;emergence&lt;/a&gt; of properties in &lt;a href=&quot;https://en.wikipedia.org/wiki/Agent-based_model#Theory&quot; rel=&quot;nofollow&quot;&gt;agents&lt;/a&gt;, and, more generally in robotics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is work on the emergence of time-related concepts, on the low-level representation of notions like &lt;em&gt;before&lt;/em&gt; and &lt;em&gt;after&lt;/em&gt;. I know, for example, that there is work on the emergence of &lt;a href=&quot;http://www.scholarpedia.org/article/Kohonen_network&quot; rel=&quot;nofollow&quot;&gt;spatial representation&lt;/a&gt; (similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm&quot; rel=&quot;nofollow&quot;&gt;knn&lt;/a&gt;), or even &lt;a href=&quot;https://infoscience.epfl.ch/record/129415/files/Mitrietal_1.pdf&quot; rel=&quot;nofollow&quot;&gt;communication&lt;/a&gt;* but time seems to be a tricky concept. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has everything to do with the &lt;em&gt;platform&lt;/em&gt;, i.e. the way that the representation would be coded in. We tend to favour ways that have some meaning or somehow mimic natural, well, yes, human structures, like the brain. I am not a neuroscientist and do not know that the sense of time &lt;em&gt;looks like&lt;/em&gt; in humans, or if it is even present in other living beings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there some work on the (emergence of the) representation of &lt;em&gt;time&lt;/em&gt; in artificial agents?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*I remember watching a really cool... Actually creepy video from these robots but cannot find it anymore. Does anyone have the link at hand?&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="70" LastEditDate="2016-09-02T21:11:12.350" LastActivityDate="2016-09-02T21:11:12.350" Title="Are there emergent models of time in robots?" Tags="&lt;knowledge-representation&gt;&lt;time&gt;&lt;embodied-cognition&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="185" PostTypeId="2" ParentId="172" CreationDate="2016-08-03T06:10:03.807" Score="2" Body="&lt;p&gt;Humans might create somewhere in the future a so-called ultraintelligent machine, a machine that can surpass all intellectual activities by any human. This would be the last invention man would need to do, since this machine is better in inventing machines than humans are (since that is an intellectual activity). Also, since humans can create machines as good as the ultraintelligent machine, this machine can create better machines, which in turn can create better machines, etcetera. This is known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligence explosion&lt;/a&gt;, and it is also called recursive self-improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The existence, let alone the development, if an ultraintelligent machine is still hypothetical. We are nowhere close to creating an ultraintelligent machine.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2019-06-20T20:48:44.383" LastActivityDate="2019-06-20T20:48:44.383" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="186" PostTypeId="1" CreationDate="2016-08-03T06:20:12.393" Score="5" ViewCount="2441" Body="&lt;p&gt;Have there been proposed extensions to go beyond a Turing machine that solve the halting problem and if so, would those proposed extensions have value to advance strong Artificial Intelligence?  For example, does quantum computing go beyond the definition of a Turing machine and resolve the halting problem, and does that help in creating strong AI?&lt;/p&gt;&#xA;" OwnerUserId="55" LastActivityDate="2016-08-07T23:34:29.837" Title="Does a quantum computer resolve the halting problem and would that advance strong AI?" Tags="&lt;quantum-computing&gt;&lt;halting-problem&gt;&lt;agi&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="1" ClosedDate="2016-08-15T03:28:50.017" ContentLicense="CC BY-SA 3.0" />
  <row Id="187" PostTypeId="2" ParentId="37" CreationDate="2016-08-03T06:37:01.983" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;(this was intended as a comment, but turned out long and longer)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A couple of points to elaborate on &lt;a href=&quot;https://ai.stackexchange.com/a/73/70&quot;&gt;Ben's answer&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is possible to generate different models (out of existing data!) and then look for the model that best fit new data (e.g. with &lt;a href=&quot;https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;knn&lt;/a&gt;). Example: &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;States = {&lt;em&gt;sleep&lt;/em&gt;, &lt;em&gt;eat&lt;/em&gt;, &lt;em&gt;walk&lt;/em&gt;, &lt;em&gt;work&lt;/em&gt;}&lt;/li&gt;&#xA;&lt;li&gt;Model 1: Most probable sequence on weekdays, say: sleep → sleep → eat → walk → work → work → eat → walk → sleep  → sleep&lt;/li&gt;&#xA;&lt;li&gt;Model 2: Most probable sequence on weekends, some: sleep → sleep → eat → walk → eat → walk → sleep → sleep&lt;/li&gt;&#xA;&lt;li&gt;New data arrives: Which sequence is more probable that it came from? Check model 1, check model 2. Which fits better? → Assign&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Note that the previous example is oversimplified. Also note that a &lt;em&gt;unit time&lt;/em&gt; is needed there (other than letters / words, for instance).&lt;/li&gt;&#xA;&lt;li&gt;You can &lt;em&gt;nest&lt;/em&gt; Markov models. That means that you generate a model (a set of probabilities for all the states) in a &quot;lower scale&quot; and then use it in a more abstract model. For example, you can nest your day-scale model to a month or year (to include holidays, for instance).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also &lt;a href=&quot;http://blog.wolfram.com/2013/02/04/centennial-of-markov-chains/&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this link for a nice introduction&lt;/a&gt; and &lt;a href=&quot;https://stats.stackexchange.com/questions/tagged/mcmc?sort=votes&amp;amp;pageSize=50&quot;&gt;some posts in crossvalidated&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As for the question if artificial intelligence can be created by using this kind of methods, my personal (easy) answer would be &lt;strong&gt;no&lt;/strong&gt;, because they only relate data and probabilities and thus belong more to the statistics and machine learning branch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A longer answer needs to take into account the &lt;a href=&quot;https://ai.stackexchange.com/questions/74/what-is-the-difference-between-strong-ai-and-weak-ai&quot;&gt;weak vs. strong AI question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-03T06:37:01.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="188" PostTypeId="2" ParentId="184" CreationDate="2016-08-03T07:00:16.047" Score="3" Body="&lt;p&gt;To my knowledge, this is very  much an open research issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a paper by Prof Leslie Smith, an acknowledged expert on neuromorphic perceptual coding, which explains the importance of the notion of perceptual time for Artificial General Intelligence and sketches an architecture from which a notion of 'now' might emerge: &lt;a href=&quot;http://www.cs.stir.ac.uk/~lss/recentpapers/perctime.pdf&quot; rel=&quot;nofollow&quot;&gt;Perceptual Time&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T07:00:16.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="189" PostTypeId="2" ParentId="26" CreationDate="2016-08-03T07:13:10.390" Score="21" Body="&lt;p&gt;Architectures for recognizing and generating emotion are typically somewhat complex and don't generally have short descriptions, so it's probably better to reference the literature rather than give a misleading soundbite:&lt;/p&gt;&#xA;&lt;p&gt;Some of the early work in &lt;em&gt;affective computing&lt;/em&gt; was done by &lt;a href=&quot;https://web.media.mit.edu/%7Epicard/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosalind W. Picard&lt;/a&gt;. There is a &lt;a href=&quot;http://affect.media.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;research group at MIT&lt;/a&gt; specializing in this area.&lt;/p&gt;&#xA;&lt;p&gt;Some of the more developed architectural ideas are due to Marvin Minsky.&#xA;A pre-publication draft of his book, &lt;em&gt;The Emotion Machine&lt;/em&gt;, is available via &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Emotion_Machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Emotional intelligence would certainly seem to be a necessary component of passing the Turing test - indeed, in the original Turing test essay in &lt;a href=&quot;https://academic.oup.com/mind/article/LIX/236/433/986238&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing Machinery and Intelligence&lt;/a&gt; implied some degree of &amp;quot;Theory of Mind&amp;quot; about Mr. Pickwick's preferences:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Yet Christmas is a Winter’s day, and I do not think Mr. Pickwick would mind the comparison.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2021-02-01T00:57:02.330" LastActivityDate="2021-02-01T00:57:02.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="190" PostTypeId="2" ParentId="186" CreationDate="2016-08-03T07:36:08.923" Score="3" Body="&lt;p&gt;It depends a bit on what you mean by 'quantum computer'. The 'conventional' notion is that quantum computation buys a (in some cases, exponential) speedup - it doesn't change &lt;em&gt;what&lt;/em&gt; can be computed, just how quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, advocates of &lt;em&gt;hypercomputation&lt;/em&gt; claim that quantum effects may make it possible to do infinite computations in finite time. Note, however, that this is not a mainstream belief - the reknowned logician Martin Davis has written an article claiming that hypercomputation is &lt;a href=&quot;http://www1.maths.leeds.ac.uk/~pmt6sbc/docs/davis.myth.pdf&quot; rel=&quot;nofollow&quot;&gt;a myth&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roger Penrose has also claimed that quantum vibrations in neural microtubules may be &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_mind&quot; rel=&quot;nofollow&quot;&gt;responsible for consciousness&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T07:36:08.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="191" PostTypeId="1" AcceptedAnswerId="1526" CreationDate="2016-08-03T07:57:21.743" Score="6" ViewCount="167" Body="&lt;p&gt;What was the first AI that was able to carry on a conversation, with real responses, such as in the famous &lt;a href=&quot;https://www.youtube.com/watch?v=WnzlbyTZsQY&quot; rel=&quot;nofollow&quot;&gt;'I am not a robot. I am a unicorn' case?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 'real response' constitutes a sort-of personalized answer to a specific input by a user.&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-10-08T12:44:10.017" LastActivityDate="2018-10-08T12:44:10.017" Title="What was the first machine that was able to carry on a conversation?" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;history&gt;&lt;turing-test&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="193" PostTypeId="2" ParentId="146" CreationDate="2016-08-03T08:22:36.737" Score="1" Body="&lt;p&gt;A neural net with even a single hidden layer is capable of Universal function approximation - it can approximate any continuous function 'as closely as you like'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, one option would be to look for GOFAI applications that would benefit from this property - for example, in state-space search approaches where the utility of a state is not readily defined in advance, and could instead be learned.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T08:22:36.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="194" PostTypeId="5" CreationDate="2016-08-03T08:44:31.107" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-03T08:44:31.107" LastActivityDate="2016-08-03T08:44:31.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="195" PostTypeId="4" CreationDate="2016-08-03T08:44:31.107" Score="0" Body="For questions regarding handwriting recognition" OwnerUserId="101" LastEditorUserId="101" LastEditDate="2016-08-04T02:54:15.227" LastActivityDate="2016-08-04T02:54:15.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="196" PostTypeId="2" ParentId="191" CreationDate="2016-08-03T08:49:27.637" Score="5" Body="&lt;p&gt;In 1986, the first PC therapist program was written by Joseph Weintraub. This program won the first Loebner Prize in 1991, and then again in 1992, 1993 and 1995. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 1981 or 1982, Jabberwacky was founded, which is the foundation of the current Cleverbot. Jabberwacky  appeared on the internet in 1997, reaching the third place for the Loebner Prize in 2003, the second place in 2004, and won in 2005 and 2006. In 2008, Cleverbot was launched as an variant of Jabberwacky. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure these are really the earliest, but that also depends on what you want earliest (programming started, first conversation,  first decent conversation, etc.). Also, it depends on what you call a &quot;real response&quot;.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2016-08-03T08:49:27.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="197" PostTypeId="1" CreationDate="2016-08-03T08:56:53.430" Score="11" ViewCount="509" Body="&lt;p&gt;This question stems from quite a few &quot;informal&quot; sources. Movies like &lt;em&gt;2001, A Space Odyssey&lt;/em&gt; and &lt;em&gt;Ex Machina&lt;/em&gt;; books like &lt;em&gt;Destination Void&lt;/em&gt; (Frank Herbert), and others suggest that general intelligence &lt;em&gt;wants&lt;/em&gt; to survive, and even learn the importance for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be several arguments for survival. What would be the most prominent?&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="2444" LastEditDate="2019-11-22T23:30:37.760" LastActivityDate="2019-11-22T23:30:37.760" Title="Is there a strong argument that survival instinct is a prerequisite for creating an AGI?" Tags="&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="3" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="198" PostTypeId="1" AcceptedAnswerId="1345" CreationDate="2016-08-03T09:01:05.790" Score="17" ViewCount="318" Body="&lt;p&gt;Identifying sarcasm is considered as one of the most difficult open-ended problems in the domain of ML and NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, was there any considerable research done in that front? If yes, then what is the accuracy like? Please also explain the NLP model briefly.&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="4302" LastEditDate="2018-10-08T12:43:51.623" LastActivityDate="2018-10-08T12:43:51.623" Title="What research has been done in the domain of &quot;Identifying sarcasm in text&quot;?" Tags="&lt;natural-language-processing&gt;&lt;research&gt;&lt;semantics&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="199" PostTypeId="2" ParentId="41" CreationDate="2016-08-03T09:06:26.437" Score="3" Body="&lt;p&gt;It all depends of what your A.I. can do. Even humans cannot do everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your AI program is so smart, ask it to take the general IQ tests for humans. Because the real IQ tests are made of several questions from different areas, so in that way you can measure IQ of your AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because the &lt;strong&gt;IQ&lt;/strong&gt; means the tests which are &lt;strong&gt;designed&lt;/strong&gt; to assess human intelligence.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An intelligence quotient (IQ) is a total score derived from one of several standardized tests designed to assess human intelligence.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence_quotient&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So there is no any other way of measuring IQ without taking IQ test, otherwise it won't be IQ (very logical).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your program is not so smart, you should look for specific tests related to the expertise or problem being solved. Ideally let it compete with humans who has the same expertise in that area, but it's important make the test on the same ground/level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example the intelligence of &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Blue&lt;/a&gt; project was measured by playing chess with Kasparov. Then if world champion cannot win the game, who will?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're writing program to play a game, make it play with compete with humans and measure the intelligence in terms of score.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The equivalent of IQ for AI is a Turing Test (like &lt;a href=&quot;https://ai.stackexchange.com/q/1397/8&quot;&gt;MIST&lt;/a&gt; and other), see:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/15/8&quot;&gt;Is the Turing Test, or any of its variants, a reliable test of artificial intelligence?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-06T11:09:15.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="200" PostTypeId="2" ParentId="111" CreationDate="2016-08-03T09:17:14.977" Score="30" Body="&lt;p&gt;Personally, I think this might be an overhyped issue. Trolley problems only occur when the situation is optimized to prevent &quot;3rd options&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A car has brakes, does it not? &quot;But what if the brakes don't work?&quot; Well, then &lt;strong&gt;the car is not allowed to drive at all.&lt;/strong&gt; Even in regular traffic, human operators are taught that your speed should be limited as such that you can stop within the area you can see. Solutions like these will reduce the possibility of a trolley problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for animals... if there is no explicit effort to deal with humans on the road I think animals will be treated the same. This sounds implausible - roadkill happens often and human &quot;roadkill&quot; is unwanted, but animals are a lot smaller and harder to see than humans, so I think detecting humans will be easier, preventing a lot of the accidents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other cases (bugs, faults while driving, multiple failures stacked onto each other), perhaps accidents will occur, they'll be analysed, and vehicles will be updated to avoid causing similar situations. &lt;/p&gt;&#xA;" OwnerUserId="74" LastEditorUserId="74" LastEditDate="2016-08-03T09:31:21.937" LastActivityDate="2016-08-03T09:31:21.937" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="201" PostTypeId="2" ParentId="70" CreationDate="2016-08-03T09:18:24.867" Score="13" Body="&lt;p&gt;Convolutional Nets (CNN) rely on mathematical convolution (e.g. 2D or 3D convolutions), which is commonly used for signal processing. Images are a type of signal, and convolution can equally be used on sound, vibrations, etc. So, in principle, CNNs can find applications to any signal, and probably more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, there exists already work on NLP (as mentioned by Matthew Graves), where some people process text with CNNs rather than recursive networks. Some other works apply to sound processing (no reference here, but I have yet unpublished work ongoing).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Original contents: In answer to the original title question, which has changed now. Perhaps need to delete this one&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Research on adversarial networks (and related) show that even &lt;a href=&quot;http://arxiv.org/abs/1412.1897&quot; rel=&quot;noreferrer&quot;&gt;deep networks can easily be fooled&lt;/a&gt;, leading them to see a dog (or whatever object) in what appears to be random noise when a human look at it (the article has clear examples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is the generalization power of a neural network. Convolutional nets have amazed the world with their capability to generalize way better than other techniques. But if the network is only fed images of cats, it will recognize only cats (and probably see cats everywhere, as by adversarial network results). In other words, even CNs have a hard time generalizing too far &lt;em&gt;beyond&lt;/em&gt; what they learned from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recognition limit is hard to define precisely. I would simply say that the diversity of the learning data pushes the limit (I assume further detail should lead to more appropriate venue for discussion).&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2016-08-07T09:03:34.373" LastActivityDate="2016-08-07T09:03:34.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="202" PostTypeId="1" AcceptedAnswerId="206" CreationDate="2016-08-03T09:18:52.437" Score="0" ViewCount="242" Body="&lt;p&gt;I'd like to know more about &lt;a href=&quot;https://ai.stackexchange.com/q/26/8&quot;&gt;implementing emotional intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given I'm implementing a chat bot and I'd like to introduce the levels of curiosity to measure whether user text input is interesting or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;High level would mean bot is asking more questions and is following the topic, lower level of curiosity makes the bot not asking any questions and changing the topics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Less interesting content could mean the bot doesn't see any opportunity to learn something new or it doesn't understand the topic or doesn't want to talk about it, because of its low quality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How this possibly can be achieved? Are there any examples?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-30T21:04:22.253" Title="How can you simulate level of curiosity for a chat bot?" Tags="&lt;emotional-intelligence&gt;&lt;chat-bots&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="205" PostTypeId="1" AcceptedAnswerId="219" CreationDate="2016-08-03T10:03:58.587" Score="7" ViewCount="1960" Body="&lt;p&gt;I would like to learn more about whether it is possible and how to write a program that decompiles executable binary (an object file) to the C source. I'm not asking exactly 'how', but rather how this can be achieved.&lt;/p&gt;&#xA;&lt;p&gt;Given the following &lt;code&gt;hello.c&lt;/code&gt; file (as example):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;int main() {&#xA;  printf(&amp;quot;Hello World!&amp;quot;);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then after compilation (&lt;code&gt;gcc hello.c&lt;/code&gt;) I've got the binary file like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ hexdump -C a.out | head&#xA;00000000  cf fa ed fe 07 00 00 01  03 00 00 80 02 00 00 00  |................|&#xA;00000010  0f 00 00 00 b0 04 00 00  85 00 20 00 00 00 00 00  |.......... .....|&#xA;00000020  19 00 00 00 48 00 00 00  5f 5f 50 41 47 45 5a 45  |....H...__PAGEZE|&#xA;00000030  52 4f 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |RO..............|&#xA;00000040  00 00 00 00 01 00 00 00  00 00 00 00 00 00 00 00  |................|&#xA;00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|&#xA;00000060  00 00 00 00 00 00 00 00  19 00 00 00 d8 01 00 00  |................|&#xA;00000070  5f 5f 54 45 58 54 00 00  00 00 00 00 00 00 00 00  |__TEXT..........|&#xA;$&lt;/span&gt; wc -c hello.c a.out &#xA;  60 hello.c&#xA;8432 a.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For the learning dataset, I assume I'll have to have thousands of source code files along with its binary representation, so the algorithm can learn about moving parts on certain changes.&lt;/p&gt;&#xA;&lt;p&gt;How would you tackle this problem?&lt;/p&gt;&#xA;&lt;p&gt;My concerns (and sub-questions) are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Does my algorithm need to be aware of the header file, or it's &amp;quot;smart&amp;quot; enough to figure it out?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If it needs to know about the header, how do I tell my algorithm &amp;quot;here is the header file&amp;quot;?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What should be input/output mapping (whether some section to section or file to file)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do I need to divide my source code into some sections?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do I need to know exactly how decompilers work or AI can figure it out for me?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I have two neural networks, one for header, another for body it-self?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;or more separate neural networks, each one for each logical component (e.g. byte-&amp;gt;C tag, etc.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-11-19T15:17:41.320" LastActivityDate="2020-11-19T21:34:48.317" Title="How to write a C decompiler using AI?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reference-request&gt;&lt;c&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="206" PostTypeId="2" ParentId="202" CreationDate="2016-08-03T10:12:58.320" Score="7" Body="&lt;p&gt;It's possible to implement a form of curiosity-driven behavior without requiring full 'emotional intelligence'. One elementary strategy would be to define some form of similarity measure on inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, Jurgen Schmidhuber has pioneered work on 'Artificial Curiosity/Creativity' and 'Intrinsic Motivation' and has written a number of papers on the subject:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://people.idsia.ch/~juergen/curioussingapore/curioussingapore.html&quot;&gt;Artificial Curiosity&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://people.idsia.ch/~juergen/ieeecreative.pdf&quot;&gt;Intrinsic Motivation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://www.youtube.com/watch?v=Ipomu0MLFaI&quot;&gt;video&lt;/a&gt; of a nice associated presentation.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T10:12:58.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="207" PostTypeId="1" CreationDate="2016-08-03T10:14:49.743" Score="4" ViewCount="78" Body="&lt;p&gt;Text summarization is a long-standing research problem that was &lt;em&gt;&quot;ignited&quot;&lt;/em&gt; by Luhn in 1958. However, a half century later, we still came nowhere close  to solving this problem (abstractive summarization). The reason for this might be because researchers are resorting to statistical (and sometimes linguistic) methods to find &amp;amp; extract the most salient parts of the text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is summarization problem solvable using AI (neural networks to be precise)? &lt;/p&gt;&#xA;" OwnerDisplayName="user220" LastEditorUserId="4302" LastEditorDisplayName="user220" LastEditDate="2018-10-08T12:42:18.353" LastActivityDate="2018-10-08T12:42:18.353" Title="Can abstractive summarization be achieved using neural networks?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="208" PostTypeId="2" ParentId="63" CreationDate="2016-08-03T10:26:17.493" Score="12" Body="&lt;p&gt;Formally, a single hidden layer is sufficient to approximate a continuous function to any desired degree of accuracy, so in that sense, you never need more than 1. This is called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;noreferrer&quot;&gt;Universal Approximation Theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the best topology for a given problem is an open research problem. As far as I know, there are few universal 'rules of thumb' for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given problem, one option is to apply a &lt;em&gt;neuroevolutionary&lt;/em&gt; approach such as &lt;a href=&quot;https://www.cs.ucf.edu/~kstanley/neat.html&quot; rel=&quot;noreferrer&quot;&gt;NEAT&lt;/a&gt;, which attempts to find a topology that works well for the problem at hand.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="4398" LastEditDate="2017-11-05T17:58:56.897" LastActivityDate="2017-11-05T17:58:56.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="211" PostTypeId="1" AcceptedAnswerId="1664" CreationDate="2016-08-03T12:27:56.120" Score="4" ViewCount="608" Body="&lt;p&gt;What AI techniques does IBM use for its Watson platform (specifically, its natural language processing part)?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-29T23:05:19.380" LastActivityDate="2019-07-01T21:18:00.277" Title="What are the main AI technologies behind the Watson platform?" Tags="&lt;natural-language-processing&gt;&lt;watson&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="212" PostTypeId="1" CreationDate="2016-08-03T12:41:06.953" Score="1" ViewCount="135" Body="&lt;p&gt;I'm investigating the possibility of storing the semantic-lexical connections (such as the relationships to the other words such as phrases and other dependencies, its strength, part of speech, language, etc.) in order to provide analysis of the input text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume this has been already done. If so, to avoid reinventing the wheel, is there any efficient method to store and manage such data in some common format which has been already researched and tested?&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-26T23:10:09.577" Title="How to store datasets of lexical connections?" Tags="&lt;algorithm&gt;&lt;models&gt;&lt;research&gt;&lt;storage&gt;&lt;lexical-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="214" PostTypeId="1" CreationDate="2016-08-03T13:01:20.740" Score="2" ViewCount="96" Body="&lt;p&gt;Which objective and measurable tests have been developed to test the intelligence of AI? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classical test is the Turing Test, which has objective criteria and is measurable since it can be measured what percentage of the jury is fooled by the AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for other, more modern tests. &lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-11-01T16:55:48.237" Title="Which objective tests have been developed to test the intelligence of AI?" Tags="&lt;intelligence-testing&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="216" PostTypeId="2" ParentId="68" CreationDate="2016-08-03T13:35:29.993" Score="7" Body="&lt;p&gt;Over the last few years, evolutionary computation research has shown increasing interest in including some aspect of epigenetics. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A 2008 paper by &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0020025508002880&quot; rel=&quot;nofollow&quot;&gt;Tanev and Yuta&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Work from &lt;a href=&quot;http://faculty.hampshire.edu/lspector/pubs/Epigenetics_2015_GECCO_final.pdf&quot; rel=&quot;nofollow&quot;&gt;Lee Spector's genetic programming group&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;A recent paper by &lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-319-30668-1_9&quot; rel=&quot;nofollow&quot;&gt;Ricalde and Banzhaf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-04T16:42:49.973" LastActivityDate="2016-08-04T16:42:49.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="217" PostTypeId="2" ParentId="207" CreationDate="2016-08-03T14:03:23.977" Score="2" Body="&lt;p&gt;The ability to re-frame summarization as a problem for ANN is rather dependent on what kind of output you're looking for: you mentioned 'salient parts of the text'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possibly is to use a deep learning approach that first chunks together words that belong in the same phrase as a single 'feature'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is to identify both key words and relations between them. Here is some previous work on using neural nets for &lt;a href=&quot;https://lirias.kuleuven.be/bitstream/123456789/131932/1/41238.pdf&quot; rel=&quot;nofollow&quot;&gt;relational learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T14:03:23.977" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="218" PostTypeId="1" AcceptedAnswerId="1745" CreationDate="2016-08-03T14:17:17.257" Score="6" ViewCount="3408" Body="&lt;p&gt;I'm interested in implementing a program for natural language processing (aka &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELIZA&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that I'm already &lt;a href=&quot;https://ai.stackexchange.com/q/212/8&quot;&gt;storing semantic-lexical connections&lt;/a&gt; between the words and its strength.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the methods of dealing with words which have very distinct meaning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;'Are we on the same page?'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'page' in this context isn't a document page, but it's part of the phrase.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;'I'm living in Reading.'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'Reading' is a city (noun), so it's not a verb. Otherwise it doesn't make any sense. Checking for the capital letter would work in that specific example, but it won't work for other (like 'make' can be either verb or noun).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;'I've read something on the Facebook wall, do you want to know what?'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'Facebook wall' has nothing to do with wall at all.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In general, how algorithm should distinguish the word meaning and recognise the word within the context?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Detecting the word for different type of speech, so it should recognise whether it's a verb or noun.&lt;/li&gt;&#xA;&lt;li&gt;Detecting whether the word is part of phrase.&lt;/li&gt;&#xA;&lt;li&gt;Detecting word for multiple meaning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are the possible approaches to solve that problem in order to  identify the correct sense of a word with the context?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-10-08T00:11:37.960" Title="How to resolve lexical ambiguity in natural language processing?" Tags="&lt;natural-language-processing&gt;&lt;lexical-recognition&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="219" PostTypeId="2" ParentId="205" CreationDate="2016-08-03T14:19:39.400" Score="4" Body="&lt;p&gt;In-between your input and desired output, there's obviously a huge space to search. The more relevant domain information you include as features, the higher chance that the Deep Learning (DL) algorithm can find the desired mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this early stage in DL research, there aren't so many rules of thumb to tell you what features to explicitly encode - not least because it depends on the size of your training corpus. My suggestion would be: obtain (or generate) a large corpus of C code, train on that with the most naive feature representation that you think might work, then repeatedly gather data and add more feature preprocessing as necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This following paper describes a DL approach to what is almost the 'reverse problem' to yours - &lt;a href=&quot;http://arxiv.org/pdf/1510.07211.pdf&quot; rel=&quot;nofollow&quot;&gt;generating the source code for a program described in natural language&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the strength of the results reported in this paper surprising, but it does give me some hope that what you are asking might be possible.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T14:19:39.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="220" PostTypeId="1" AcceptedAnswerId="223" CreationDate="2016-08-03T14:23:50.760" Score="4" ViewCount="534" Body="&lt;p&gt;How does an unsupervised learning model learn, if it does not involve any target values?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-06T23:19:55.067" LastActivityDate="2019-06-06T23:21:54.023" Title="How does an unsupervised learning model learn?" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;models&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="221" PostTypeId="1" CreationDate="2016-08-03T14:32:39.333" Score="4" ViewCount="339" Body="&lt;p&gt;Currently, many different organizations do cutting-edge AI research, and some innovations are shared freely (at a time lag) while others are kept private. I'm referring to this state of affairs as 'multipolar,' where instead of there being one world leader that's far ahead of everyone else, there are many competitors who can be mentioned in the same breath. (There's not only one academic center of AI research worth mentioning, there might be particularly hot companies but there's not only one worth mentioning, and so on.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we could imagine instead there being one institution that mattered when it comes to AI (be it a company, a university, a research group, or a non-profit). This is what I'm referring to as &quot;monolithic.&quot; Maybe they have access to tools and resources no one else has access to, maybe they attract the best and brightest in a way that gives them an unsurmountable competitive edge, maybe returns to research compound in a way that means early edges can't be overcome, maybe they have some sort of government coercion preventing competitors from popping up. (For other industries, network or first-mover effects might be other good examples of why you would expect that industry to be monolithic instead of multipolar.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like we should be able to use insights from social sciences like economics or organizational design or history of science in order to figure out, if not which path seems more likely, &lt;em&gt;how we would know&lt;/em&gt; which path seems more likely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For example, we may be able to measure how much returns to research compound, in the sense of one organization coming up with an insight meaning that organization is likely to come up with the next relevant insight, and knowing this number makes it easier to figure out where the boundary between the two trajectories is located.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2017-07-15T16:07:53.250" Title="How would we know if AI development will continue to be multipolar, or will become monolithic?" Tags="&lt;research&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="222" PostTypeId="2" ParentId="1" CreationDate="2016-08-03T14:39:02.827" Score="7" Body="&lt;p&gt;'Backprop' is short for 'backpropagation of error' in order to avoid confusion when using &lt;em&gt;backpropagation&lt;/em&gt; term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically &lt;em&gt;backpropagation&lt;/em&gt; refers to the method for computing the gradient of the case-wise error function with respect to the weights for a feedforward network&lt;sup&gt;Werbos&lt;/sup&gt;. And &lt;em&gt;backprop&lt;/em&gt; refers to a training method that uses backpropagation to compute the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can say that a &lt;em&gt;backprop&lt;/em&gt; network is a feedforward network trained by &lt;em&gt;backpropagation&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'standard backprop' term is a euphemism for the &lt;em&gt;generalized delta rule&lt;/em&gt; which is most widely used supervised training method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;ftp://ftp.sas.com/pub/neural/FAQ2.html#A_backprop&quot; rel=&quot;noreferrer&quot;&gt;What is backprop?&lt;/a&gt; at FAQ of Usenet newsgroup comp.ai.neural-nets&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University.&lt;/li&gt;&#xA;&lt;li&gt;Werbos, P. J. (1994). The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting,Wiley Interscience.&lt;/li&gt;&#xA;&lt;li&gt;Bertsekas, D. P. (1995), Nonlinear Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-14-0.&lt;/li&gt;&#xA;&lt;li&gt;Bertsekas, D. P. and Tsitsiklis, J. N. (1996), Neuro-Dynamic Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-10-8.&lt;/li&gt;&#xA;&lt;li&gt;Polyak, B.T. (1964), &quot;Some methods of speeding up the convergence of iteration methods,&quot; Z. Vycisl. Mat. i Mat. Fiz., 4, 1-17.&lt;/li&gt;&#xA;&lt;li&gt;Polyak, B.T. (1987), Introduction to Optimization, NY: Optimization Software, Inc.&lt;/li&gt;&#xA;&lt;li&gt;Reed, R.D., and Marks, R.J, II (1999), Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, Cambridge, MA: The MIT Press, ISBN 0-262-18190-8.&lt;/li&gt;&#xA;&lt;li&gt;Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986), &quot;Learning internal representations by error propagation&quot;, in Rumelhart, D.E. and McClelland, J. L., eds. (1986), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1, 318-362, Cambridge, MA: The MIT Press.&lt;/li&gt;&#xA;&lt;li&gt;Werbos, P.J. (1974/1994), The Roots of Backpropagation, NY: John Wiley &amp;amp; Sons. Includes Werbos's 1974 Harvard Ph.D. thesis, Beyond Regression.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-03T14:39:02.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="223" PostTypeId="2" ParentId="220" CreationDate="2016-08-03T14:41:02.937" Score="8" Body="&lt;p&gt;Supervised learning is typically an attempt to learn a mathematical function, &lt;span class=&quot;math-container&quot;&gt;$f(\bf X)=\bf y$&lt;/span&gt;. For this, you need both the input vector &lt;span class=&quot;math-container&quot;&gt;$\bf X$&lt;/span&gt; and the output vector &lt;span class=&quot;math-container&quot;&gt;$\bf y$&lt;/span&gt;. The model outputs have whatever dimensionality that the target values have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unsupervised learning models instead learn a &lt;em&gt;structure&lt;/em&gt; from the data. A clustering model, for example, is learning both how many clusters exist in the data (a number that's not the same type as the inputs) and where those clusters are located (which is also a different type from the inputs). The output of running this model on a new datapoint &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is not the same type as &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, but instead a classification label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, time series models learn parameters that symbolize how vectors in the input relate to each other, rather than raw inputs themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how they learn, the structures are mathematical objects whose fitness is determined by the input data. The simplest possible unstructured unsupervised learning problem is probably &quot;what's the mean of the data?&quot;, and it should be clear how that's 'learned' through processing the input. More sophisticated models are just adding more pieces to that calculation.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="2444" LastEditDate="2019-06-06T23:21:54.023" LastActivityDate="2019-06-06T23:21:54.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="224" PostTypeId="1" CreationDate="2016-08-03T14:58:03.663" Score="4" ViewCount="86" Body="&lt;p&gt;One of the most compelling applications for AI would be in augmenting human biological intelligence. What are some of the currently proposed methods for doing this aside from vague notions such as &quot;nanobots swimming around our brains and bodies&quot; or &quot;electrodes connected to our skulls&quot;?&lt;/p&gt;&#xA;" OwnerUserId="148" LastEditorUserId="10135" LastEditDate="2018-10-21T20:53:59.603" LastActivityDate="2018-10-21T20:53:59.603" Title="How could AI be used to augment human biological intelligence?" Tags="&lt;cyborg&gt;&lt;intelligence-augmentation&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="225" PostTypeId="1" AcceptedAnswerId="228" CreationDate="2016-08-03T15:25:50.843" Score="-4" ViewCount="659" Body="&lt;p&gt;Given list of fixed numbers from a mathematical constant such as Pi, is it is possible to train AI to attempt to predict the next numbers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which AI or neural network would be more suitable for this task? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially the one which will work without memorizing the entire training set, but the one which will attempt to find some patterns or statistical association.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-09T10:02:46.153" LastActivityDate="2016-08-09T10:02:46.153" Title="What are the approaches to predict sequence of π numbers?" Tags="&lt;training&gt;&lt;math&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="4" ClosedDate="2016-08-03T18:21:18.760" ContentLicense="CC BY-SA 3.0" />
  <row Id="226" PostTypeId="2" ParentId="224" CreationDate="2016-08-03T15:27:15.343" Score="5" Body="&lt;p&gt;'Direct augmentation' of human intelligence, of the sort that you would see in science fiction, looks to be very hard. Most of our promising approaches deal with &lt;em&gt;avoiding damage&lt;/em&gt; rather than &lt;em&gt;adding capabilities&lt;/em&gt;--there's no drug that you can take now that will make you smarter to the degree that missing a night of sleep can make you dumber.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most informative area of current practice is probably game-playing, where '&lt;a href=&quot;http://bloomreach.com/2014/12/centaur-chess-brings-best-humans-machines/&quot;&gt;centaurs&lt;/a&gt;,' or humans working with computers, outcompete human players or computer players.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But a centaur player doesn't have a wire jutting out of their skull to jack into the computer; they're looking at a laptop screen. One of the reasons to be pessimistic about cyborg augmentation is because current I/O technology is already so good. Why install a new wire to put information into your visual cortex, when you come already equipped with two? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you could think code directly onto the screen, how much better would that be than typing code through a keyboard? Probably some, but I find it difficult to imagine that it'll be more than twice as good. So most human-computer intelligence augmentation will look like people using software, and software using human inputs, rather than humans and computers evolving together.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Transcranial Direct Current Stimulation (TDCS) and similar approaches cause temporary changes in mental abilities by raising or lowering the activation potentials of neurons in particular regions of the brain. (I've done it myself, a few years ago, and what weak effects I noticed were probably negative. Not too much surprise for a DIY setup!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like it has a number of useful implications. One article about TDCS that I found particularly striking was the journalist who tried it gushing about how their anxiety disappeared for a few days, presumably because the part of their brain behind the anxiety was dampened. One could imagine it being useful for the treatment of many different mental disorders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I'm pessimistic that it will translate into superior &lt;em&gt;peak&lt;/em&gt; performance, and I think that's the sort of thing that's more relevant for discussions of augmentation. (Is there TDCS that we could do that would make Terrence Tao better at doing mathematics?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where improved AI methods will come into play is by improving our models of the brain, allowing us to better target interventions, much in the way that AI methods are improving our treatment of cancer (through superior diagnosis and targeting of radiotherapy, as two easy examples). These effects will all be indirect--for example, AI empowering an app or gadget that helps you sleep better won't &lt;em&gt;directly&lt;/em&gt; augment your intelligence, but will cause population-level increases in effective intelligence through reducing sleep deprivation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't talked yet about nootropics, chemicals that increase intelligence, but it's reasonable to expect that AI will improve drug discovery there like it improves drug discovery for anything else. But the same caveats apply--the effect of nootropics seem to be negatively correlated with intelligence (that is, the smarter someone already is, the harder it is to increase their intelligence further).&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="10" LastEditDate="2016-08-03T16:12:18.483" LastActivityDate="2016-08-03T16:12:18.483" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="228" PostTypeId="2" ParentId="225" CreationDate="2016-08-03T15:32:55.467" Score="3" Body="&lt;p&gt;Pseudo-random number generators are specifically defined to defeat any form of prediction via 'black box' observation. Certainly, some (e.g. linear congruential) have weaknesses, but you are unlikely to have any success in general in predicting the output of a modern RNG. For devices based on chaotic physical systems (e.g. most national lotteries), there is no realistic possibility of prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Patterns or statistical association&quot; is a much weaker criterion than 'prediction'. Some very recent work has applied topological data analysis to visualize patterns within the infamous Randu RNG.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-03T15:52:47.397" LastActivityDate="2016-08-03T15:52:47.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="230" PostTypeId="2" ParentId="225" CreationDate="2016-08-03T15:42:13.340" Score="-1" Body="&lt;p&gt;You would probably have to pack recursive structures into finite-dimensional real vectors and there have been such attempts. The finite precision limits goes as far as the recursion can go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The limitation of &lt;em&gt;feedforward&lt;/em&gt; neural networks is restricted to finite input and output spaces, so &lt;em&gt;recurrent&lt;/em&gt; may be more suitable for this task as in theory can process arbitrarily long strings of numbers, but it has much more practical difficulties than feedforward network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These kind of methods are open to debate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;ftp://ftp.sas.com/pub/neural/FAQ.html&quot; rel=&quot;nofollow&quot;&gt;SAS FAQ&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Blair, 1997; Pollack, 1990; Chalmers, 1990; Chrisman, 1991; Plate, 1994; Hammerton, 1998; Hadley, 1999&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-03T15:50:40.390" LastActivityDate="2016-08-03T15:50:40.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="232" PostTypeId="2" ParentId="197" CreationDate="2016-08-03T15:45:41.770" Score="5" Body="&lt;p&gt;The concept of 'survival instinct' probably falls in the category of what Marvin Minsky would call a 'suitcase word', i.e. it packages together a number of related phenomena into what at first appears to be a singular notion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it's quite possible that we can construct mechanisms that have the appearance of some kind of 'hard-coded' survival instinct, without that ever featuring as an explicit rule(s) in the design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the beautiful little book &lt;a href=&quot;https://mitpress.mit.edu/books/vehicles&quot; rel=&quot;noreferrer&quot;&gt;'Vehicles'&lt;/a&gt; by the neuroanatomist Valentino Braitenberg for a compelling narrative of how such 'top down' concepts as 'survival instinct' might evolve 'from the bottom up'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, trying to ensure that intelligent artefacts place too high a priority on their survival might easily lead to a &lt;a href=&quot;https://xkcd.com/1613/&quot; rel=&quot;noreferrer&quot;&gt;Killbot Hellscape&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T15:45:41.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="233" PostTypeId="1" AcceptedAnswerId="236" CreationDate="2016-08-03T15:56:18.480" Score="12" ViewCount="463" Body="&lt;p&gt;Currently, most research done in artificial intelligence focuses on neural networks, which have been successfully used to solve many problems.  A good example would be &lt;a href=&quot;https://www.nature.com/articles/nature16961&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind's AlphaGo&lt;/a&gt;, which uses a convolutional neural network. There are many other examples, such as &lt;a href=&quot;https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Translate, which uses transformers&lt;/a&gt;, or &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN&lt;/a&gt;, which has been used to play Atari games.&lt;/p&gt;&#xA;&lt;p&gt;So, are any of the variants of neural networks the only way to reach &amp;quot;true&amp;quot; artificial intelligence (or AGI)?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2021-01-25T23:21:24.513" LastActivityDate="2021-01-25T23:21:24.513" Title="Are neural networks the only way to reach &quot;true&quot; artificial intelligence?" Tags="&lt;neural-networks&gt;&lt;agi&gt;&lt;human-like&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="234" PostTypeId="2" ParentId="197" CreationDate="2016-08-03T15:58:13.493" Score="6" Body="&lt;p&gt;Steve Omohudro wrote a paper called &lt;a href=&quot;https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf&quot; rel=&quot;noreferrer&quot;&gt;Basic AI Drives&lt;/a&gt; that steps through why we would expect an AI with narrow goals to find some basic, general concepts as instrumentally useful for their narrow goals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, an AI designed to maximize stock market returns but whose design is silent on the importance of continuing to survive would realize that its continued survival is a key component of maximizing stock market returns, and thus take actions to keep itself operational.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, we should be skeptical of 'anthropomorphizing' AI and other code, but it seems like there &lt;em&gt;are&lt;/em&gt; reasons to expect this beyond &quot;well, humans behave this way, so it must be how all intelligence behaves.&quot;&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-03T15:58:13.493" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="235" PostTypeId="2" ParentId="147" CreationDate="2016-08-03T16:05:28.357" Score="3" Body="&lt;p&gt;I presume the proof the OP is referring to can be found in &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-1-4612-0707-8&quot; rel=&quot;nofollow&quot;&gt;this monograph&lt;/a&gt; by Hava Siegelmann?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In his article &lt;a href=&quot;http://www1.maths.leeds.ac.uk/~pmt6sbc/docs/davis.myth.pdf&quot; rel=&quot;nofollow&quot;&gt;'The Myth of Hypercomputation'&lt;/a&gt;, the eminent computer scientist Martin Davis explains (p8-9) that there is nothing 'super Turing' about this formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: It's looking like the claim about &lt;em&gt;rational&lt;/em&gt; weights being super-Turing is made in &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/25354762&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;  more recent paper by Siegelmann, which introduces an additional assumption of &lt;em&gt;plasticity&lt;/em&gt;, i.e. that weights can be dynamically updated.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-03T17:38:06.577" LastActivityDate="2016-09-03T17:38:06.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="236" PostTypeId="2" ParentId="233" CreationDate="2016-08-03T16:21:42.693" Score="13" Body="&lt;p&gt;If by true AI, you mean 'like human beings', the answer is - no-one knows what the appropriate computational mechanisms (neural or otherwise) are or indeed whether we are capable of constructing them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What Artificial Neural Nets (ANNs) do is essentially 'nonlinear regression' - perhaps this is not a sufficiently strong model to express humanlike behaviour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the 'Universal function approximation' property of ANNs, what if human intelligence depends on some as-yet-unguessed mechanism of the physical world?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to your question about &quot;the only way&quot;:&#xA;Even if (physical) neural mechanisms somehow actually were the &lt;em&gt;only&lt;/em&gt; route to intelligence (e.g. via Penrose's quantum microtubules), how could that be proved? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in the formal world of mathematics, there's a saying that &quot;Proofs of non-existence are hard&quot;. It scarcely seems conceivable that, in the physical world, it would be possible to demonstrate that intelligence could not arise by any other mechanism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moving back to computational systems, note that Stephen Wolfram made the interesting observation in his book &lt;a href=&quot;http://www.wolframscience.com/nksonline/toc.html&quot;&gt;'A New Kind of Science'&lt;/a&gt; that many of the apparently distinct mechanisms he observed seem to be capable of 'Universal Computation', so in that sense there's nothing very particular about ANNs.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T16:21:42.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="237" PostTypeId="1" AcceptedAnswerId="253" CreationDate="2016-08-03T16:22:36.073" Score="5" ViewCount="190" Body="&lt;p&gt;I'm interested in hardware implementation of ANNs (artificial neural networks). Are there any popular existing technology implementations in form of microchips which are purpose designed to run artificial neural networks? For example, a chip which is optimised for an application like image recognition or something similar?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="33" LastEditDate="2016-08-03T19:55:57.227" LastActivityDate="2016-08-06T01:22:37.030" Title="Are there any microchips specifically designed to run ANNs?" Tags="&lt;image-recognition&gt;&lt;hardware&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="238" PostTypeId="2" ParentId="159" CreationDate="2016-08-03T16:46:33.883" Score="3" Body="&lt;p&gt;As far as I can see, there's no reason why you couldn't (for example) take the convolutional inputs to deepdream from adjacent sample points, rather than adjacent spatial positions, as is the case with image input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the 'self similar' nature of deep dream images, listening to this &lt;a href=&quot;https://vimeo.com/13541969&quot; rel=&quot;nofollow&quot;&gt;fractal granular synthesis&lt;/a&gt; technique might be of interest/inspiration.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T16:46:33.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="239" PostTypeId="2" ParentId="233" CreationDate="2016-08-03T17:17:16.590" Score="4" Body="&lt;p&gt;It depends on what you consider &quot;true artificial intelligence&quot;. But this probably means to be able to think like a human - and perhaps, do so in a more rational manner, as in the human brain emotion comes before ratio.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would seem that a neural network, or a genetic algorithm that evolves neural networks, is the closest way - mimicking humans.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the traditional counter-argument to this is that we tried to do the same with flight. We tried to copy nature, mimick the birds - trying to fly by flapping wings. But eventually we made airplanes that did not rely on flapping their wings.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In AI, there are far more variables than in aerodynamics. So it is quite likely that a human-like intelligence can be attained by other methods than neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, neural networks are one approach to machine learning. There are others, all governed by the rules for what can and cannot be learnt. (There is a field called Computational Learning Theory that covers this). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although it is possible to extend learning systems beyond what can be learnt according to COLT, this means that such a learning system - neural network or otherwise - is essentially flawed, and will draw wrong conclusions at one point or another.&lt;/p&gt;&#xA;" OwnerUserId="66" LastActivityDate="2016-08-03T17:17:16.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="240" PostTypeId="1" AcceptedAnswerId="243" CreationDate="2016-08-03T17:22:05.433" Score="17" ViewCount="5067" Body="&lt;p&gt;I've noticed that a few questions on this site mention genetic algorithms and it made me realize that I don't really know much about those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have heard the term before, but it's not something I've ever used, so I don't have much idea about how they work and what they are good for. All I know is that they involve some sort of evolution and randomly changing values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you give me a short explanation, preferably including some sort of practical example that illustrates the basic principles?&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="2444" LastEditDate="2021-01-07T00:39:57.647" LastActivityDate="2021-01-07T23:37:54.823" Title="What exactly are genetic algorithms and what sort of problems are they good for?" Tags="&lt;genetic-algorithms&gt;&lt;applications&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="241" PostTypeId="1" CreationDate="2016-08-03T17:24:18.480" Score="4" ViewCount="384" Body="&lt;p&gt;In detective novels, the point is often that the reader gets enough information to solve the crime themselves. This &quot;puzzle&quot; aspect of detective novels is part of the attraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often the difficulty for humans is to keep track of all the variables - events, items, motivations.&lt;br&gt;&#xA;An AI would have an easier time keeping track of all the details, but would rely on real-world knowledge to prevent making crazy mistakes. For example, if it was stated that a character took the train, the AI would need to know that this is a method of transportation - that it changes the location property of an agent over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has an AI ever been able to solve a detective mystery?&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="4302" LastEditDate="2018-10-08T12:41:15.483" LastActivityDate="2018-10-08T12:41:15.483" Title="Has an AI ever solved a detective mystery?" Tags="&lt;natural-language-processing&gt;&lt;problem-solving&gt;&lt;world-knowledge&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="242" PostTypeId="2" ParentId="240" CreationDate="2016-08-03T17:27:28.260" Score="8" Body="&lt;p&gt;A genetic algorithm is an algorithm that randomly generates a number of attempted solutions for a problem. This set of attempted solutions is called the &quot;population&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It then tries to see how well these solutions solve the problem, using a given &lt;em&gt;fitness function&lt;/em&gt;. The attempted solutions with the best &lt;em&gt;fitness&lt;/em&gt; value are used to generate a new population. This can be done by making small changes to the attempted solutions (mutation) or by combining existing attempted solutions (crossover).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that, over time, an attempted solution emerges that has a high enough &lt;em&gt;fitness&lt;/em&gt; value to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inspiration for this came from the theory of evolution; the fittest solutions survive and procreate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you were looking for the most efficient way to cut a number of shapes out of a piece of wood. You want to waste as little wood as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your attempted solutions would be random arrangements of these shapes on your piece of wood. &lt;em&gt;Fitness&lt;/em&gt; would be determined by how little wood would be left after cutting the shapes following this arrangement.&lt;br&gt;&#xA;The less wood is left, the better the attempted solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you were trying to find a polynomial that passes through a number of points. Your attempted solutions would be random polynomials.&lt;br&gt;&#xA;To determine the &lt;em&gt;fitness&lt;/em&gt; of these polynomials, you determine how well they fit the given points. (In this particular case, you would probably use the least squares method to determine how well the polynomial fit the points).&#xA;Over a number of trials, you would get polynomials that fit the points better, until you had a polynomial that fit the points closely enough.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-03T17:57:10.623" LastActivityDate="2016-08-03T17:57:10.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="243" PostTypeId="2" ParentId="240" CreationDate="2016-08-03T17:42:02.623" Score="15" Body="&lt;p&gt;Evolutionary algorithms are a family of optimization algorithms based on the principle of &lt;strong&gt;Darwinian natural selection&lt;/strong&gt;. As part of natural selection, a given environment has a population of individuals that compete for survival and reproduction. The ability of each individual to achieve these goals determines their chance to have children, in other words, to pass on their genes to the next generation of individuals, who, for genetic reasons, will have an increased chance of doing well, even better, in realizing these two objectives.&lt;/p&gt;&#xA;&lt;p&gt;This principle of continuous improvement over the generations is taken by evolutionary algorithms to optimize solutions to a problem. In the &lt;strong&gt;initial generation&lt;/strong&gt;, a population composed of different &lt;strong&gt;individuals&lt;/strong&gt; is generated randomly or by other methods. An individual is a solution to the problem, more or less good: the quality of the individual in regards to the problem is called &lt;strong&gt;fitness&lt;/strong&gt;, which reflects the adequacy of the solution to the problem to be solved. The higher the fitness of an individual, the higher it is likely to pass some or all of its genotype to the individuals of the next generation.&lt;/p&gt;&#xA;&lt;p&gt;An individual is coded as a &lt;strong&gt;genotype&lt;/strong&gt;, which can have any shape, such as a bit vector (&lt;strong&gt;genetic algorithms&lt;/strong&gt;) or a vector of real (evolution strategies). Each genotype is transformed into a &lt;strong&gt;phenotype&lt;/strong&gt; when assessing the individual, i.e. when its fitness is calculated. In some cases, the phenotype is identical to the genotype: it is called &lt;strong&gt;direct&lt;/strong&gt; &lt;strong&gt;coding&lt;/strong&gt;. Otherwise, the coding is called indirect. For example, suppose you want to optimize the size of a rectangular parallelepiped defined by its length, height and width. To simplify the example, assume that these three quantities are integers between 0 and 15. We can then describe each of them using a 4-bit binary number. An example of a potential solution may be to genotype 0001 0111 1010. The corresponding phenotype is a parallelepiped of length 1, height 7 and width 10.&lt;/p&gt;&#xA;&lt;p&gt;During the transition from the old to the new generation, the &lt;strong&gt;variation&lt;/strong&gt; &lt;strong&gt;operators&lt;/strong&gt;, whose purpose is to manipulate individuals, are applied. There are two distinct types of variation operators:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;strong&gt;mutation&lt;/strong&gt; &lt;strong&gt;operators&lt;/strong&gt;, which are used to introduce variations within the same individual, as genetic mutations;&lt;/li&gt;&#xA;&lt;li&gt;the &lt;strong&gt;crossover&lt;/strong&gt; &lt;strong&gt;operators&lt;/strong&gt;, which are used to cross at least two different genotypes, as genetic crosses from breeding.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Evolutionary algorithms have proven themselves in various fields such as operations research, robotics, biology, nuance, or cryptography. In addition, they can optimize multiple objectives simultaneously and can be used as black boxes because they do not assume any properties in the mathematical model to optimize. Their only real limitation is the computational complexity.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wweBO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wweBO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="2444" LastEditDate="2021-01-07T00:38:51.873" LastActivityDate="2021-01-07T00:38:51.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="244" PostTypeId="2" ParentId="240" CreationDate="2016-08-03T17:43:29.303" Score="5" Body="&lt;p&gt;As observed in another answer, all you need to apply Genetic Algorithms (GAs) is to represent a potential solution to your problem in a form that is subject to crossover and mutation. Ideally, the fitness function will provide some kind of smooth feedback about the quality of a solution, rather than simply being a 'Needle in a Haystack'.&lt;/p&gt;&#xA;&lt;p&gt;Here are some characteristics of problems that Genetic Algorithms (and indeed &lt;a href=&quot;https://cs.gmu.edu/%7Esean/book/metaheuristics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metaheuristics&lt;/a&gt; in general) are good for:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;NP-complete - The number of possible solutions to the problem is&#xA;exponential, but checking the fitness of a solution is relatively&#xA;cheap (technically, with time polynomial in the input size).&lt;/li&gt;&#xA;&lt;li&gt;Black box - GAs work reasonably well even if you don't have a particularly&#xA;informed model of the problem to be solved. This means that these&#xA;approaches are also useful as a 'rapid prototyping' approach to&#xA;solving problems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, despite their widespread use for the purpose, note that GAs are actually &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.161.5655&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;not&lt;/em&gt; function optimizers&lt;/a&gt; - GA mechanisms tend not to explore 'outlying' regions of the search space in the hope of finding some distant high quality solution, but rather to cluster around more easily attainable peaks in the 'fitness landscape'.&lt;/p&gt;&#xA;&lt;p&gt;More detail on the applicability of GAs is given in a famous early paper &lt;a href=&quot;https://link.springer.com/article/10.1007/BF00993046&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;What makes a problem hard for a Genetic Algorithm?&amp;quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2021-01-07T23:37:54.823" LastActivityDate="2021-01-07T23:37:54.823" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="246" PostTypeId="2" ParentId="240" CreationDate="2016-08-03T17:58:32.980" Score="6" Body="&lt;p&gt;This answer requests a practical example of how one might be used, which I will attempt to provide in addition to the other answers. They seem to due a very good job of explaining what a genetic algorithm is. So, this will give an example.&lt;/p&gt;&#xA;&lt;p&gt;Let's say you have a neural network (although they are not the only application of it), which, from some given inputs, will yield some outputs. A genetic algorithm can create a population of these, and by seeing which output is the best, breed and kill off members of the population. Eventually, this should optimise the neural network if it is complicated enough.&lt;/p&gt;&#xA;&lt;p&gt;Here is a demonstration I've made, which despite being badly coded, might help you understand. &lt;a href=&quot;http://khrabanas.github.io/projects/evo/evo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://khrabanas.github.io/projects/evo/evo.html&lt;/a&gt;&#xA;Hit the evolve button and mess around with the goals.&lt;/p&gt;&#xA;&lt;p&gt;It uses a simple genetic algorithm to breed, mutate, and decide which individuals of the population survive. Depending on how the input variables are set, the network will be able to get to some level of closeness to them. In this fashion, the population will likely eventually become a homogeneous group, whose outputs resemble the goals.&lt;/p&gt;&#xA;&lt;p&gt;The genetic algorithm is trying to create a &amp;quot;neural network&amp;quot; of sorts, that by taking in RGB, will yield an output color. First, it generates a random population. It then by taking 3 random members from the population, selecting the one with the lowest fitness and removing it from the population. The fitness is equal to the difference in the top goal squared + the difference in the bottom goal squared. It then breeds the two remaining ones together and adds the child to the same place in the population as the dead member. When mating occurs, there is a chance a mutation will occur. This mutation will change one of the values randomly.&lt;/p&gt;&#xA;&lt;p&gt;As a side note, due to how it is set up, it is impossible for it to be totally correct in many cases, though it will reach relative closeness.&lt;/p&gt;&#xA;" OwnerUserId="244" LastEditorUserId="2444" LastEditDate="2021-01-07T00:53:28.160" LastActivityDate="2021-01-07T00:53:28.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="247" PostTypeId="1" CreationDate="2016-08-03T18:05:24.997" Score="10" ViewCount="590" Body="&lt;p&gt;In 1969, Seymour Papert and Marvin Minsky showed that Perceptrons could not learn the XOR function.&lt;/p&gt;&#xA;&lt;p&gt;This was solved by the backpropagation network with at least one hidden layer. This type of network can learn the XOR function.&lt;/p&gt;&#xA;&lt;p&gt;I believe I was once taught that every problem that could be learned by a backpropagation neural network with multiple hidden layers, could also be learned by a backpropagation neural network with a single hidden layer. (Although possibly a nonlinear activation function was required).&lt;/p&gt;&#xA;&lt;p&gt;However, it is unclear to me what the limits are to backpropagation neural networks themselves. Which patterns &lt;strong&gt;cannot&lt;/strong&gt; be learned by a neural network trained with gradient descent and backpropagation?&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="2444" LastEditDate="2021-01-22T14:35:47.293" LastActivityDate="2021-01-22T14:35:47.293" Title="What are the learning limitations of neural networks trained with backpropagation?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;universal-approximation-theorems&gt;&lt;computational-learning-theory&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="248" PostTypeId="1" AcceptedAnswerId="5803" CreationDate="2016-08-03T18:05:26.937" Score="17" ViewCount="3456" Body="&lt;p&gt;Over the last 50 years, the rise/fall/rise in popularity of neural nets has acted as something of a 'barometer' for AI research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's clear from the questions on this site that people are interested in applying Deep Learning (DL) to a wide variety of difficult problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I therefore have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Practitioners - What do you find to be the main obstacles to&#xA;applying DL 'out of the box' to your problem? &lt;/li&gt;&#xA;&lt;li&gt;Researchers - What&#xA;techniques do you use (or have developed) that might help address&#xA;practical issues? Are they within DL or do they offer an&#xA;alternative approach?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42" LastEditorUserId="95" LastEditDate="2016-08-04T14:09:54.380" LastActivityDate="2018-03-26T13:17:52.717" Title="Issues with and alternatives to Deep Learning approaches?" Tags="&lt;deep-learning&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="249" PostTypeId="1" AcceptedAnswerId="1286" CreationDate="2016-08-03T18:12:22.133" Score="2" ViewCount="60" Body="&lt;p&gt;Is it possible for &lt;em&gt;unsupervised learning&lt;/em&gt; to learn about high-level, class-specific features given only unlabelled images? For example detecting human or animal faces? If so, how?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-04T00:31:01.093" LastActivityDate="2016-08-04T06:25:20.620" Title="Is it possible for 'unsupervised learning' model to recognize features on unlabelled images?" Tags="&lt;image-recognition&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="250" PostTypeId="2" ParentId="92" CreationDate="2016-08-03T18:18:58.077" Score="59" Body="&lt;p&gt;First up, those images (even the first few) aren't complete trash despite being junk to humans; they're actually finely tuned with various advanced techniques, including another neural network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The deep neural network is the pre-trained network modeled on AlexNet provided by &lt;a href=&quot;https://github.com/BVLC/caffe&quot;&gt;Caffe&lt;/a&gt;. To evolve images, both the directly encoded and indirectly encoded images, we use the &lt;a href=&quot;https://github.com/jbmouret/sferes2&quot;&gt;Sferes&lt;/a&gt; evolutionary framework. The entire code base to conduct the evolutionary experiments can be download [sic] &lt;a href=&quot;https://github.com/Evolving-AI-Lab/fooling&quot;&gt;here&lt;/a&gt;. The code for the images produced by gradient ascent is available &lt;a href=&quot;https://github.com/Evolving-AI-Lab/fooling/tree/master/caffe/ascent&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Images that are actually random junk were correctly recognized as nothing meaningful:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In response to an unrecognizable image, the networks could have output a low confidence for each of the 1000 classes, instead of an extremely high confidence value for one of the classes. In fact, they do just that for randomly generated images (e.g. those in generation 0 of the evolutionary run)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The original goal of the researchers was to use the neural networks to automatically generate images that look like the real things (by getting the recognizer's feedback and trying to change the image to get a more confident result), but they ended up creating the above art. Notice how even in the static-like images there are little splotches - usually near the center - which, it's fair to say, are triggering the recognition.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We were not trying to produce adversarial, unrecognizable images. Instead, we were trying to produce recognizable images, but these unrecognizable images emerged.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Evidently, these images had just the right distinguishing features to match what the AI looked for in pictures. The &quot;paddle&quot; image does have a paddle-like shape, the &quot;bagel&quot; is round and the right color, the &quot;projector&quot; image is a camera-lens-like thing, the &quot;computer keyboard&quot; is a bunch of rectangles (like the individual keys), and the &quot;chainlink fence&quot; legitimately looks like a chain-link fence to me.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Figure 8. Evolving images to match DNN classes produces a tremendous diversity of images. Shown are images selected to showcase diversity from 5 evolutionary runs. The diversity suggests that the images are non-random, but that instead evolutions producing [sic] discriminative features of each target class.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Further reading: &lt;a href=&quot;http://www.evolvingai.org/files/DNNsEasilyFooled_cvpr15.pdf&quot;&gt;the original paper&lt;/a&gt; (large PDF)&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-08-03T18:18:58.077" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="251" PostTypeId="5" CreationDate="2016-08-03T18:35:44.733" Score="0" Body="&lt;p&gt;Fuzzy logic is an alternative to boolean logic.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In boolean logic, every statement is either true or false. Usually &quot;1&quot; is used to represent &quot;true&quot; and &quot;0&quot; is used to represent &quot;false&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fuzzy logic is an extension of this concept, where statements do not have such absolute values. Instead, the truth value is a real number in the range [0..1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, fuzzy logic is a real-number version of multi-valued logic.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-04T02:52:30.443" LastActivityDate="2016-08-04T02:52:30.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="252" PostTypeId="4" CreationDate="2016-08-03T18:35:44.733" Score="0" Body="Fuzzy logic is a variant of boolean logic, where the values are real numbers between 0 and 1 (inclusive), rather than only the integer numbers 0 and 1. Use this tag for questions  that involve this real-valued logic." OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-04T02:52:57.023" LastActivityDate="2016-08-04T02:52:57.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="253" PostTypeId="2" ParentId="237" CreationDate="2016-08-03T18:37:03.843" Score="6" Body="&lt;p&gt;In May 2016 Google announced a custom ASIC which was is specifically built for machine learning&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow#DistBelief&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt;&lt;/sup&gt; and tailored for &lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow&quot; rel=&quot;nofollow&quot;&gt;TensorFlow&lt;/a&gt;. It is using &lt;a href=&quot;https://en.wikipedia.org/wiki/Tensor_processing_unit&quot; rel=&quot;nofollow&quot;&gt;tensor processing unit&lt;/a&gt; (TPU) which is a programmable microprocessor designed to accelerate artificial neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/group/brainsinsilicon/goals.html&quot; rel=&quot;nofollow&quot;&gt;NeuroCores&lt;/a&gt;, 12x14 sq-mm chips which can be interconnected in a binary tree, see: &lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow#DistBelief&quot; rel=&quot;nofollow&quot;&gt;Neurogrid&lt;/a&gt;, a supercomputer which can provide an option for brain simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/TrueNorth&quot; rel=&quot;nofollow&quot;&gt;TrueNorth&lt;/a&gt;, a neuromorphic CMOS chip produced by IBM, which has 4096 cores in the current chip, each can simulate 256 programmable silicon &quot;neurons&quot;, giving a total of over a million neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further readings: &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering&quot; rel=&quot;nofollow&quot;&gt;Neuromorphic engineering&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Vision_processing_unit&quot; rel=&quot;nofollow&quot;&gt;Vision processing unit&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Category:AI_accelerators&quot; rel=&quot;nofollow&quot;&gt;AI accelerators&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As a side note, you can always use an FPGA based piece of hardware which you can implement selected genetic algorithm (GA) directly in hardware. For example the &lt;a href=&quot;https://en.wikipedia.org/wiki/CoDi#Implementation_in_Hardware&quot; rel=&quot;nofollow&quot;&gt;CoDi model&lt;/a&gt; was implemented in the FPGA based CAM-Brain Machine (CBM)&lt;sup&gt;&lt;a href=&quot;http://link.springer.com/article/10.1023%2FA%3A1011286308522&quot; rel=&quot;nofollow&quot;&gt;2001&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-06T01:22:37.030" LastActivityDate="2016-08-06T01:22:37.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="254" PostTypeId="5" CreationDate="2016-08-03T18:45:06.577" Score="0" Body="&lt;p&gt;Humans know a lot about the real world implicitly.&lt;br&gt;&#xA;If someone is said to have &quot;filled their tank&quot;, it is implied that they went to a gas station, filled the tank of their car, and paid for it.&lt;br&gt;&#xA;An AI does not have this knowledge implicitly; it needs to be taught that this is how things work. It may be taught so explicitly by a teacher or it may learn this from scraping the net, but the knowledge does not come from real-world experiences like humans have.&lt;br&gt;&#xA;For this reason, acquiring and applying knowledge about the real world is a field of interest in Artificial Intelligence.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-04T02:51:34.397" LastActivityDate="2016-08-04T02:51:34.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="255" PostTypeId="4" CreationDate="2016-08-03T18:45:06.577" Score="0" Body="World knowledge is knowledge about the real world. Humans have implicit knowledge of the real world, but an AI needs to have explicit resources to explain how the real world works. Use this tag for questions about how artificially intelligent systems deal with their need to acquire or apply knowledge about the real world." OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-04T02:52:21.053" LastActivityDate="2016-08-04T02:52:21.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="256" PostTypeId="5" CreationDate="2016-08-03T18:58:09.953" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-03T18:58:09.953" LastActivityDate="2016-08-03T18:58:09.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="257" PostTypeId="4" CreationDate="2016-08-03T18:58:09.953" Score="0" Body="For questions related to Gödel's incompleteness theorems, which are two theorems of mathematical logic that demonstrate the inherent limitations of every formal axiomatic system capable of modeling basic arithmetic." OwnerUserId="66" LastEditorUserId="2444" LastEditDate="2020-05-13T10:41:37.200" LastActivityDate="2020-05-13T10:41:37.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="258" PostTypeId="1" AcceptedAnswerId="3936" CreationDate="2016-08-03T19:03:20.587" Score="8" ViewCount="253" Body="&lt;p&gt;On the Wikipedia page we can read the basic structure of an artificial neuron (a model of biological neurons) which consist:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dendrites - acts as the input vector,&lt;/li&gt;&#xA;&lt;li&gt;Soma - acts as the summation function,&lt;/li&gt;&#xA;&lt;li&gt;Axon - gets its signal from the summation behavior which occurs inside the soma.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've checked &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep learning Wikipedia page&lt;/a&gt;, but I couldn't find any references to dendrites, soma or axons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which type of artificial neural network implements or can mimic such a model most closely?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-03-09T01:15:07.373" LastActivityDate="2020-03-09T01:27:59.503" Title="Which artificial neural network can mimic biological neurons the most?" Tags="&lt;artificial-neuron&gt;&lt;biology&gt;&lt;neuroscience&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="259" PostTypeId="5" CreationDate="2016-08-03T19:27:06.623" Score="0" Body="&lt;p&gt;Declarative programming is a computer programming paradigm where the focus is on declaring what should be accomplished, rather than explaining how it should be accomplished.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example of a declarative programming language is Prolog, which is based upon predicate logic. Here is a simple Prolog program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dog(brutus).&#xA;dog(pluto).&#xA;dog(X):-barks(X).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This program states (declares) 3 things: (1) that brutus is dog, (2) that pluto is a dog, and (3) that if something is a dog, it barks.&lt;br&gt;&#xA;(Note that the names of the dogs are in small letters; Prolog uses capital letters to identify variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can now ask this program to tell us who barks:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;barks(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it will give us the solutions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = brutus ;&#xA;X = pluto ;&#xA;no more solutions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note how we did not tell our program &lt;strong&gt;how&lt;/strong&gt; to arrive at this result; this is done by the Prolog system itself.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Declarative programming, by focusing on the &quot;what&quot; rather than the &quot;how&quot;, is useful for building knowledge bases.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-04T02:53:38.350" LastActivityDate="2016-08-04T02:53:38.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="260" PostTypeId="4" CreationDate="2016-08-03T19:27:06.623" Score="0" Body="Declarative programming is a programming paradigm where the focus is on what must be accomplished, rather than how it is to be accomplished. Hence it is more about &quot;declaring&quot; than about implementing algorithms. Use this tag for questions about how declarative programming is used in AI systems." OwnerUserId="66" LastEditorUserId="145" LastEditDate="2016-08-23T00:15:33.337" LastActivityDate="2016-08-23T00:15:33.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="262" PostTypeId="5" CreationDate="2016-08-03T19:57:38.687" Score="0" Body="&lt;p&gt;Problem solving in artificial intelligence is the study of how an AI can solve a given problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual approach to problem solving is state search. The problem was described as an initial state, conditions for a final state, and a set of transition rules. A transition rule changes takes a state as input and outputs a new state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution of the problem then consists of applying the right transitions, until a state is reached where that satisfies the condition for a final state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a concrete example, there is the problem of the Farmer, the Goat, the Cabbage and the Wolf. The farmer must row each of these to the other side of a river, but his boat is only big enough that he can transport only one of them at a time. If he leaves the goat with the cabbage, the goat will eat the cabbage; if he leaves the wolf with the goat, the wolf will eat the goat.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial state has the farmer, cabbage, goat and wolf on one side of the river. A final state has them all on the other side. The transition rules are all &quot;row the cabbage OR the goat OR the wolf from the current side to the other&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several state search algorithms, where the purpose is to arrive at a final state in an efficient manner.  &lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="1671" LastEditDate="2018-10-15T16:57:37.207" LastActivityDate="2018-10-15T16:57:37.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="263" PostTypeId="4" CreationDate="2016-08-03T19:57:38.687" Score="0" Body="For questions about AI problem solving in terms of approaches, theory, logic, and other aspects where the problem is well defined and the objective is to find a solution to the problem." OwnerUserId="66" LastEditorUserId="4302" LastEditDate="2018-10-15T16:55:35.850" LastActivityDate="2018-10-15T16:55:35.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="264" PostTypeId="2" ParentId="172" CreationDate="2016-08-03T20:03:22.657" Score="0" Body="&lt;p&gt;It depends what you mean by 'develop themselves' - in a rather limited sense, an online machine learning approach such as Genetic Algorithms 'develops itself' to provide better solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is already a theoretical model that represents the &lt;em&gt;ultimate&lt;/em&gt; concept of development: Juergen Schmidhuber's &lt;a href=&quot;http://arxiv.org/abs/cs/0309048&quot; rel=&quot;nofollow&quot;&gt;Goedel Machine&lt;/a&gt; is constructed so as to self-modify when it can prove that this modification is optimal.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-03T20:03:22.657" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="265" PostTypeId="2" ParentId="241" CreationDate="2016-08-03T20:03:26.733" Score="0" Body="&lt;p&gt;Not exactly a detective mystery, but according to a slide dated June 2012 from a NSA PowerPoint presentation (see: Glenn Greenwald’s site), NSA used some kind of &lt;em&gt;Skynet&lt;/em&gt; AI technology to analyze and detect suspicious patterns from location and communication data in order to create a watch list of suspected terrorists. This helped to track associated members of Al-Qa’ida as well as the Muslim Brotherhood. And I'm sure their AI solved a lot of mysteries and found some controversial figures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://theintercept.com/2015/05/08/u-s-government-designated-prominent-al-jazeera-journalist-al-qaeda-member-put-watch-list/&quot; rel=&quot;nofollow&quot;&gt;U.S. Government Designated Prominent Al Jazeera Journalist as a member of AI Qaeda&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details check: &lt;a href=&quot;https://theintercept.com/document/2015/05/08/skynet-courier/&quot; rel=&quot;nofollow&quot;&gt;SKYNET: Courier Detection via Machine Learning&lt;/a&gt; for courier detection data and charts generated by analyzing GSM metadata using machine learning algorithms. Also &lt;a href=&quot;https://theintercept.com/document/2015/05/08/skynet-applying-advanced-cloud-based-behavior-analytics/&quot; rel=&quot;nofollow&quot;&gt;Applying Advanced Cloud-based Behavior Analytics&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-03T20:03:26.733" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="266" PostTypeId="2" ParentId="247" CreationDate="2016-08-03T21:08:03.797" Score="2" Body="&lt;p&gt;While I'm not familiar with any explicit statements regarding what a Multilayer Perceptron (MLP) &lt;em&gt;cannot&lt;/em&gt; learn, I can provide some further detail on the positive statements you made about MLP capabilities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A MLP with a single hidden layer is capable of what is commonly termed &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Universal Function Approximation'&lt;/a&gt;, i.e. it can approximate any bounded continuous function to an arbitrary degree of accuracy. With two hidden layers, the boundness restriction is removed &lt;a href=&quot;http://link.springer.com/article/10.1007/BF02551274&quot; rel=&quot;nofollow noreferrer&quot;&gt;[Cybenko, 1988]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/089360809190009T&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; goes on to demonstrate that this is true for a wide range of activation functions (not necessarily nonlinear). 3 layer MLPs are also capable of representing any boolean function (although they may require an exponential number of neurons).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://cstheory.stackexchange.com/questions/7894/universal-function-approximation&quot;&gt;this interesting answer&lt;/a&gt; on CS SE about other Universal approximators.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:32:34.750" LastActivityDate="2016-08-03T21:08:03.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="267" PostTypeId="2" ParentId="157" CreationDate="2016-08-03T22:41:48.160" Score="6" Body="&lt;p&gt;The following post has a bit of math, which I hope helps to explain the problem better. Unfortunately it seems, this SE site does not support LaTex:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Document summarization is very much an open problem in AI research. One way this task is currently handled is called &quot;extractive summarization&quot;. The basic strategy is as follows: Split this document into sentences and we will present as a summary a subset of sentences which together cover all the important details in the post. Assign sentence $i$, $1 \leq i \leq n$, a variable $z_i \in \{ 0, 1 \}$, where $z_i = 1$ indicates the sentence was selected and $z_i = 0$ means the sentence was left out. Then, $z_i z_j = 1$ if and only if both sentences were chosen. We will also define the importance of each sentence $w_i$ for the sentence $i$ and interaction terms $w_{i,j}$ between sentences $i$ and $j$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $x_i$ be the feature vectors for sentence $i$. $w_i = w(x_i)$ captures how important it is to include this sentence (or the topics covered by it) while $w_{i,j} = w(x_i,x_j)$ indicates the amount of overlap between sentences in our summary. Finally we put all this in a minimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{aligned} &#xA;\underset{z_i}{\text{maximize }} &amp;amp; \sum_{i} w_i z_i - w_{i,j} z_i z_j \\&#xA;\text{s.t. } &amp;amp; z_i = 0 \text{ or } 1&#xA;\end{aligned}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This tries to maximize the total weight of the sentences covered and tries to minimize the amount of overlap. This is an integer programming problem similar to finding the lowest weight independent set in a graph and many techniques exist to solve such problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This design, in my opinion, captures the fundamental problems in text summarization and can be extended in many ways. We will discuss those in a bit, but first, we need to completely specify the features $w$. $w_i = w(x_i)$ could be a function only of the sentence $i$, but it could also depends on the place of the sentence in the document or its context (Is the sentence at the beginning of a paragraph? Does it share common words with the title? What is its length? Does it mention any proper nouns? etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w_{i,j} = w(x_i,x_j)$ is a similarity measure. It measures how many repetitions there will be if we include both words in the sentence. It can be defined by looking at common words between sentences. We can also extract topics or concepts from each sentence and see how many are common among them, and use language features like pronouns to see if one sentence expands on another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To improve the design, first, we could do &lt;em&gt;keyphrase extraction&lt;/em&gt;, i.e. identify key phrases in the text and choose to define the above problem in terms of those instead of trying to pick sentences. That is a similar problem to what Google does to summarize news articles in their search results, but I am not aware of the details of their approach. We could also break the sentences up further into concepts and try to establish the semantic meaning of the sentences ( Ponzo and Fila are people P1 and P2, a mall is a place P, P1 and P2 went to the place P at time T (day). Mode of transport walking.... and so on). To do this, we would need to use a semantic ontology or other common-sense knowledge databases. However, all the parts of this last semantic classification problem are open and I have not seen anyone make satisfactory progress on it yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could also tweak the loss function above so that instead of the setting the tradeoff between the sentence importance $w_i$ and the diversity score $w_{i,j}$ by hand, we could learn it from data. One way to do this is to use Conditional Random Fields to model the data, but many others surely exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this answer explained the basic problems that need to be solved to make progress towards good summarization systems. This is an active field of research and you will find the most recent papers via Google Scholar, but first read the &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_summarization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page&lt;/a&gt; to learn the relevant terms&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="1641" LastEditDate="2018-08-14T18:21:04.997" LastActivityDate="2018-08-14T18:21:04.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="268" PostTypeId="2" ParentId="258" CreationDate="2016-08-03T23:08:25.990" Score="1" Body="&lt;p&gt;Most artificial neurons model biological neurons but in a very simplistic way. Nowadays, the main aim is to achieve better performance at prediction tasks. However, there is a body of literature in neuroscience that looks at &lt;a href=&quot;https://en.wikipedia.org/wiki/Models_of_neural_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational models of neurons&lt;/a&gt;. Neurons are complicated cells and our understanding of neurons is still not complete. &lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="2444" LastEditDate="2020-03-09T01:27:59.503" LastActivityDate="2020-03-09T01:27:59.503" CommentCount="0" CommunityOwnedDate="2020-11-16T17:23:30.743" ContentLicense="CC BY-SA 4.0" />
  <row Id="269" PostTypeId="5" CreationDate="2016-08-03T23:13:25.643" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://python.org&quot; rel=&quot;nofollow&quot;&gt;Python&lt;/a&gt; is a &lt;a href=&quot;https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic%20language%20and%20also%20a%20strongly%20typed%20language&quot; rel=&quot;nofollow&quot;&gt;dynamic and strongly typed&lt;/a&gt; programming language that is used for &lt;a href=&quot;http://python.org/about/apps&quot; rel=&quot;nofollow&quot;&gt;a wide range of applications&lt;/a&gt;. It is a general-purpose, high-level programming language that is designed to emphasize usability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python allows programmers to express concepts in fewer lines of code than would be possible in many other languages such as &lt;a href=&quot;/questions/tagged/c&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;c&amp;#39;&quot; rel=&quot;tag&quot;&gt;c&lt;/a&gt;, and the language has constructs intended to be used to create clear programs in a variety of domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two similar but incompatible versions of Python are in widespread use (2 and 3). Please consider mentioning the version and implementation that you are using when asking a question about Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles. It features a fully dynamic type system and automatic memory management, similar to that of &lt;a href=&quot;/questions/tagged/scheme&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;scheme&amp;#39;&quot; rel=&quot;tag&quot;&gt;scheme&lt;/a&gt;, &lt;a href=&quot;/questions/tagged/ruby&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;ruby&amp;#39;&quot; rel=&quot;tag&quot;&gt;ruby&lt;/a&gt;, &lt;a href=&quot;/questions/tagged/perl&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;perl&amp;#39;&quot; rel=&quot;tag&quot;&gt;perl&lt;/a&gt; and &lt;a href=&quot;/questions/tagged/tcl&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;tcl&amp;#39;&quot; rel=&quot;tag&quot;&gt;tcl&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like other &lt;a href=&quot;http://en.wikipedia.org/wiki/Dynamic_programming_language&quot; rel=&quot;nofollow&quot;&gt;dynamic languages&lt;/a&gt;, Python is often used as a &lt;a href=&quot;http://stackoverflow.com/tags/scripting/info&quot;&gt;scripting language&lt;/a&gt;, but is also used in a wide range of non-scripting contexts. Using third-party tools, Python code can be packaged into standalone executable programs. Python interpreters are available for many operating systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;/questions/tagged/cpython&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;cpython&amp;#39;&quot; rel=&quot;tag&quot;&gt;cpython&lt;/a&gt;, the reference implementation of Python, is free and open source software and has a community-based development model, as do nearly all of its alternative implementations. There are a wide variety of implementations more suited for specific environments or tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The philosophy of Python is succinctly formulated in &lt;a href=&quot;http://python.org/dev/peps/pep-0020&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;The Zen of Python&lt;/em&gt;&lt;/a&gt; written by Tim Peters, which can be revealed by issuing this command at the interactive interpreter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import this&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The documentation can also be accessed offline for your installation of Python in the following manner:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Going into &lt;code&gt;Your_Python_install_dir/Doc&lt;/code&gt;. There is a complete Python documentation present for the version of Python installed on your computer.&lt;/li&gt;&#xA;&lt;li&gt;Running &lt;code&gt;pydoc x&lt;/code&gt; or &lt;code&gt;python -m pydoc x&lt;/code&gt; from the command prompt or terminal displays documentation for module &lt;code&gt;x&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Unlike many other languages Python uses an indentation based syntax and this may take some getting used to for programmers familiar with braces for syntax.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from __future__ import braces&#xA;  File &quot;&amp;lt;stdin&amp;gt;&quot;, line 1&#xA;SyntaxError: not a chance&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To help with the transition it is a recommendation to use a properly configured text-editor created for programmers or an IDE. Python comes with a basic IDE called &lt;a href=&quot;http://docs.python.org/2/library/idle.html&quot; rel=&quot;nofollow&quot;&gt;IDLE&lt;/a&gt; to get you started. Other popular examples are the charity-ware &lt;a href=&quot;/questions/tagged/vim&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;vim&amp;#39;&quot; rel=&quot;tag&quot;&gt;vim&lt;/a&gt;, the free GNU &lt;a href=&quot;/questions/tagged/emacs&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;emacs&amp;#39;&quot; rel=&quot;tag&quot;&gt;emacs&lt;/a&gt;, &lt;a href=&quot;/questions/tagged/eclipse&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;eclipse&amp;#39;&quot; rel=&quot;tag&quot;&gt;eclipse&lt;/a&gt;+&lt;a href=&quot;/questions/tagged/pydev&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;pydev&amp;#39;&quot; rel=&quot;tag&quot;&gt;pydev&lt;/a&gt; or &lt;a href=&quot;/questions/tagged/pycharm&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;pycharm&amp;#39;&quot; rel=&quot;tag&quot;&gt;pycharm&lt;/a&gt;. Take a look at this &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_integrated_development_environments_for_Python#Python&quot; rel=&quot;nofollow&quot;&gt;IDE comparison list&lt;/a&gt; for many other alternatives.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Tagging recommendation:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Use the &lt;a href=&quot;/questions/tagged/python&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;python&amp;#39;&quot; rel=&quot;tag&quot;&gt;python&lt;/a&gt; tag for all Python related questions. If you believe your question includes issues specific to individual versions, use &lt;a href=&quot;/questions/tagged/python-3.x&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;python-3.x&amp;#39;&quot; rel=&quot;tag&quot;&gt;python-3.x&lt;/a&gt; or &lt;a href=&quot;/questions/tagged/python-2.7&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;python-2.7&amp;#39;&quot; rel=&quot;tag&quot;&gt;python-2.7&lt;/a&gt; in addition to the main &lt;a href=&quot;/questions/tagged/python&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;python&amp;#39;&quot; rel=&quot;tag&quot;&gt;python&lt;/a&gt; tag. If you believe your question may be even more specific, you can include a version specific tag such as &lt;a href=&quot;/questions/tagged/python-3.5&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;python-3.5&amp;#39;&quot; rel=&quot;tag&quot;&gt;python-3.5&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, consider including the tag for the specific implementation if you are using one other than &lt;a href=&quot;/questions/tagged/cpython&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;cpython&amp;#39;&quot; rel=&quot;tag&quot;&gt;cpython&lt;/a&gt; - the use of &lt;a href=&quot;/questions/tagged/cpython&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;cpython&amp;#39;&quot; rel=&quot;tag&quot;&gt;cpython&lt;/a&gt; is assumed unless explicitly stated otherwise.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Official documentation for the current stable versions: &lt;a href=&quot;http://docs.python.org/2.7/&quot; rel=&quot;nofollow&quot;&gt;2.7.x&lt;/a&gt; and &lt;a href=&quot;http://docs.python.org/3.5/&quot; rel=&quot;nofollow&quot;&gt;3.5.x&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Release notes for the current stable versions: &lt;a href=&quot;https://www.python.org/downloads/release/python-2711/&quot; rel=&quot;nofollow&quot;&gt;2.7.11&lt;/a&gt; and &lt;a href=&quot;https://www.python.org/download/releases/3.5.1&quot; rel=&quot;nofollow&quot;&gt;3.5.1&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Python_%28programming_language%29&quot; rel=&quot;nofollow&quot;&gt;Python (programming language)&lt;/a&gt; (Wikipedia)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://wiki.python.org/moin/BeginnersGuide/Programmers&quot; rel=&quot;nofollow&quot;&gt;Python for Programmers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.tutorialspoint.com/python/python_quick_guide.htm&quot; rel=&quot;nofollow&quot;&gt;Python - Quick Guide&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://intellipaat.com/tutorial/python-tutorial/introduction/&quot; rel=&quot;nofollow&quot;&gt;Getting started with Python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.python.org/3.5/howto/pyporting.html&quot; rel=&quot;nofollow&quot;&gt;Porting Python 2 Code to Python 3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;The non-profit &lt;a href=&quot;http://www.python.org/psf/&quot; rel=&quot;nofollow&quot;&gt;Python Software Foundation&lt;/a&gt; manages &lt;a href=&quot;http://en.wikipedia.org/wiki/CPython&quot; rel=&quot;nofollow&quot;&gt;CPython&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.python.org/psf/&quot; rel=&quot;nofollow&quot;&gt;PSF&lt;/a&gt; License Agreement for Python &lt;a href=&quot;http://docs.python.org/2.7/license.html&quot; rel=&quot;nofollow&quot;&gt;2.7.x&lt;/a&gt; and &lt;a href=&quot;http://docs.python.org/3.5/license.html&quot; rel=&quot;nofollow&quot;&gt;3.5.x&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://stackoverflow.com/documentation/python&quot;&gt;Stackoverflow Documentation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Popular &lt;a href=&quot;http://wiki.python.org/moin/WebFrameworks&quot; rel=&quot;nofollow&quot;&gt;web frameworks&lt;/a&gt; based on Python&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If your question has to do with any of these frameworks, please ensure you include the appropriate tag.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.djangoproject.com&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Django&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/django&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;django&amp;#39;&quot; rel=&quot;tag&quot;&gt;django&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Web framework for perfectionists (with deadlines). Django makes it easier to build better Web apps more quickly and with less code. Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. It lets you build high-performing, elegant Web applications quickly. Django focuses on automating as much as possible and adhering to the DRY (Don't Repeat Yourself) principle.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://flask.pocoo.org&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Flask&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/flask&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;flask&amp;#39;&quot; rel=&quot;tag&quot;&gt;flask&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Flask is a micro-framework for Python based on Werkzeug, Jinja 2 and good intentions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.tornadoweb.org/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Tornado&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/tornado&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;tornado&amp;#39;&quot; rel=&quot;tag&quot;&gt;tornado&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tornado is a Python web framework and asynchronous networking library. By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.cherrypy.org&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;CherryPy&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/cherrypy&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;cherrypy&amp;#39;&quot; rel=&quot;tag&quot;&gt;cherrypy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CherryPy is a pythonic, object-oriented web framework that enables developers to build web applications in much the same way they would build any other object-oriented Python program. This results in smaller source code developed in less time. CherryPy has been in use for over 7 years and it is being used in production by many sites, from the simplest to the most demanding.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.pylonsproject.org/projects/pyramid/about&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Pyramid&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/pyramid&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;pyramid&amp;#39;&quot; rel=&quot;tag&quot;&gt;pyramid&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lightweight Web framework emphasizing flexibility and rapid development. It combines the very best ideas from the worlds of Ruby, Python and Perl, providing a structured but extremely flexible Python web framework. It's also one of the first projects to leverage the emerging WSGI standard, which allows extensive re-use and flexibility but only if you need it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://webpy.org&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;web.py&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/web.py&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;web.py&amp;#39;&quot; rel=&quot;tag&quot;&gt;web.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;web.py is a web framework for Python that is as simple as it is powerful. web.py is in the public domain; you can use it for whatever purpose with absolutely no restrictions. web.py lets you write web apps in Python.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://grok.zope.org&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Grok&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/grok&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;grok&amp;#39;&quot; rel=&quot;tag&quot;&gt;grok&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Built on the existing Zope 3 libraries, but aims to provide an easier learning curve and a more agile development experience. Grok does this by placing an emphasis on convention over configuration and DRY (Don't Repeat Yourself).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://bottlepy.org/docs/dev/index.html&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Bottle&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/bottle&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;bottle&amp;#39;&quot; rel=&quot;tag&quot;&gt;bottle&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Popular Mathematical/Scientific computing libraries in Python&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.numpy.org&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;NumPy&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/numpy&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;numpy&amp;#39;&quot; rel=&quot;tag&quot;&gt;numpy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NumPy is the fundamental package for scientific computing with Python. It contains among other things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a powerful N-dimensional array object&lt;/li&gt;&#xA;&lt;li&gt;sophisticated (broadcasting) functions&lt;/li&gt;&#xA;&lt;li&gt;tools for integrating C/C++ and Fortran code&lt;/li&gt;&#xA;&lt;li&gt;useful linear algebra, Fourier transform, and random number capabilities&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These features also make it possible to use NumPy in general-purpose database applications.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://scipy.org&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;SciPy&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/scipy&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;scipy&amp;#39;&quot; rel=&quot;tag&quot;&gt;scipy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SciPy is an open source library for the Python programming language consisting of mathematical algorithms and functions often used in science and engineering. SciPy includes algorithms and tools for tasks such as optimization, clustering, discrete Fourier transforms, linear algebra, signal processing and multi-dimensional image processing. SciPy is closely related to NumPy and depends on many NumPy functions, including a multidimensional array that is used as the basic data structure in SciPy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://matplotlib.org/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;matplotlib&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/matplotlib&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;matplotlib&amp;#39;&quot; rel=&quot;tag&quot;&gt;matplotlib&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;matplotlib is a plotting library for the Python programming language and its NumPy numerical mathematics extension. It provides an object-oriented API for embedding plots into applications using general-purpose GUI toolkits like wxPython, Qt, or GTK. There is also a procedural &quot;pylab&quot; interface based on a state machine (like OpenGL), designed to closely resemble that of MATLAB.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://pandas.pydata.org/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;pandas&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/pandas&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;pandas&amp;#39;&quot; rel=&quot;tag&quot;&gt;pandas&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;pandas, the Python Data Analysis Library, is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://deeplearning.net/software/theano/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;theano&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/theano&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;theano&amp;#39;&quot; rel=&quot;tag&quot;&gt;theano&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theano is a Python-C-based library widely-used library suitable for highly computational mathematical tasks due to the optimizations it does on the interface Python code making it highly optimized using its C-based routines. It is a very popular library for machine-learning researchers as well. It features a highly optimized automatic differentiation, easing the implementations of highly complicated functions and computing their gradients without any errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.blender.org/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Blender&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/blender&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;blender&amp;#39;&quot; rel=&quot;tag&quot;&gt;blender&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Blender is a free and open source 3D animation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://scikit-learn.org/stable/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;scikit-learn&lt;/strong&gt;&lt;/a&gt; &lt;a href=&quot;/questions/tagged/scikit-learn&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;scikit-learn&amp;#39;&quot; rel=&quot;tag&quot;&gt;scikit-learn&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;scikit-learn is a free and open source machine learning library written in Python. It supports training and testing many different kinds of machine learning models, along with some basic data processing techniques.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Community&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Chat Rooms&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Chat about Python with other Stack Overflow users in the &lt;a href=&quot;http://chat.stackoverflow.com/rooms/6/python&quot;&gt;Python chat room&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Other Sites&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://python.org/community/lists/#tutor&quot; rel=&quot;nofollow&quot;&gt;Tutor mailing list&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://python.org/community/lists/#python-help&quot; rel=&quot;nofollow&quot;&gt;python-help mailing list&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.pycon.org&quot; rel=&quot;nofollow&quot;&gt;PyCon&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.pythonweekly.com&quot; rel=&quot;nofollow&quot;&gt;Python Weekly&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pycoders.com&quot; rel=&quot;nofollow&quot;&gt;Pycoder's Weekly&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://groups.google.com/forum/#!forum/comp.lang.python&quot; rel=&quot;nofollow&quot;&gt;Python Google Group&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Free Python programming Books&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikibooks.org/wiki/Non-Programmer%27s_Tutorial_for_Python_2.6&quot; rel=&quot;nofollow&quot;&gt;Wikibooks' Non-Programmers Tutorial for Python 2.6&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikibooks.org/wiki/Non-Programmer%27s_Tutorial_for_Python_3&quot; rel=&quot;nofollow&quot;&gt;Wikibooks' Non-Programmers Tutorial for Python 3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.python.org/tutorial&quot; rel=&quot;nofollow&quot;&gt;The Official Python Tutorial&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.itmaybeahack.com/homepage/books/python.html&quot; rel=&quot;nofollow&quot;&gt;Building Skills in Python Version 2.6&lt;/a&gt; (Steven F. Lott)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.swaroopch.org/notes/Python&quot; rel=&quot;nofollow&quot;&gt;A Byte of Python&lt;/a&gt; (Swaroop C H.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.brpreiss.com/books/opus7/html/book.html&quot; rel=&quot;nofollow&quot;&gt;Data Structures and Algorithms in Python&lt;/a&gt; (Bruno R. Preiss)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://interactivepython.org/runestone/static/pythonds/index.html&quot; rel=&quot;nofollow&quot;&gt;Problem Solving with Algorithms and Data Structures using python&lt;/a&gt; (Brad Miller and David Ranum)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.diveintopython.net&quot; rel=&quot;nofollow&quot;&gt;Dive into Python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.diveinto.org/python3&quot; rel=&quot;nofollow&quot;&gt;Dive into Python 3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.greenteapress.com/thinkpython/thinkCSpy&quot; rel=&quot;nofollow&quot;&gt;How to Think Like a Computer Scientist: Learning with Python&lt;/a&gt; (Allen Downey, Jeff Elkner and Chris Meyers)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://inventwithpython.com&quot; rel=&quot;nofollow&quot;&gt;Invent Your Own Computer Games With Python&lt;/a&gt; (Al Sweigart)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://learnpythonthehardway.org&quot; rel=&quot;nofollow&quot;&gt;Learn Python The Hard Way&lt;/a&gt; (Zed A. Shaw)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://inventwithpython.com/pygame&quot; rel=&quot;nofollow&quot;&gt;Making Games with Python &amp;amp; Pygame&lt;/a&gt; (Albert Sweigart)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.nltk.org/book&quot; rel=&quot;nofollow&quot;&gt;Natural Language Processing with Python&lt;/a&gt; (Steven Bird, Ewan Klein, and Edward Loper)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://openbookproject.net/pybiblio&quot; rel=&quot;nofollow&quot;&gt;Python Bibliotheca&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://openbookproject.net/py4fun&quot; rel=&quot;nofollow&quot;&gt;Python for Fun&lt;/a&gt; (Chris Meyers)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.briggs.net.nz/snake-wrangling-for-kids.html&quot; rel=&quot;nofollow&quot;&gt;Snake Wrangling For Kids&lt;/a&gt; (Jason R. Briggs)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.greenteapress.com/thinkpython/thinkpython.pdf&quot; rel=&quot;nofollow&quot;&gt;Think Python (PDF file)&lt;/a&gt; (Allen Downey)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://python3porting.com&quot; rel=&quot;nofollow&quot;&gt;Porting to Python 3&lt;/a&gt; (Lennart Regebro)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Interactive Python learning&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pythonmonk.com&quot; rel=&quot;nofollow&quot;&gt;Python Monk&lt;/a&gt; - Interactive Python learning in the browser&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.codecademy.com/tracks/python&quot; rel=&quot;nofollow&quot;&gt;Codeacademy&lt;/a&gt; - Learn the fundamentals of Python and dynamic programming&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.codeskulptor.org&quot; rel=&quot;nofollow&quot;&gt;CodeSkulptor&lt;/a&gt; - Interactive online IDEfor Python programming&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.coursera.org/course/interactivepython&quot; rel=&quot;nofollow&quot;&gt;Coursera&lt;/a&gt; - Online course for introduction to interactive Python programming&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.checkio.org&quot; rel=&quot;nofollow&quot;&gt;CheckiO&lt;/a&gt; - Game world you can explore using your Python programming skills&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.repl.it/languages/Python&quot; rel=&quot;nofollow&quot;&gt;Repl.it&lt;/a&gt; - Online interpreter for Python that it allow saving code for later demonstration&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.jetbrains.com/pycharm-edu/&quot; rel=&quot;nofollow&quot;&gt;PyCharm Edu&lt;/a&gt; - Desktop application that offers interactive Python learning.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://interactivepython.org/&quot; rel=&quot;nofollow&quot;&gt;Interactive Python&lt;/a&gt; - Includes a modified, interactive version of How to Think Like a Computer Scientist.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Python Online Courses&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://class.coursera.org/pythonlearn-001/auth/auth_redirector?type=login&amp;amp;subtype=normal&quot; rel=&quot;nofollow&quot;&gt;Programming for Everybody&lt;/a&gt; - Introduction to programming using Python.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://class.coursera.org/interactivepython-004/auth/auth_redirector?type=login&amp;amp;subtype=normal&quot; rel=&quot;nofollow&quot;&gt;An Introduction to Interactive Programming in Python&lt;/a&gt; - The name explains itself.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Python Video Tutorials&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=4Mf0h3HphEA&amp;amp;list=PLEA1FEF17E1E5C0DA&quot; rel=&quot;nofollow&quot;&gt;Programming in Python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=GE3IHS1wAZI&amp;amp;list=PL_RGaFnxSHWpX_byHyTEj9hecPngl2DqR&quot; rel=&quot;nofollow&quot;&gt;Python for Beginners&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Python for Scientists&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nbviewer.ipython.org/gist/rpmuller/5920182&quot; rel=&quot;nofollow&quot;&gt;A Crash Course in Python for Scientists&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5" LastEditorUserId="5" LastEditDate="2016-08-04T02:53:41.727" LastActivityDate="2016-08-04T02:53:41.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="270" PostTypeId="4" CreationDate="2016-08-03T23:13:25.643" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-03T23:13:25.643" LastActivityDate="2016-08-03T23:13:25.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="271" PostTypeId="2" ParentId="92" CreationDate="2016-08-03T23:25:53.977" Score="11" Body="&lt;p&gt;An important question that does not yet have a satisfactory answer in neural network research is how DNNs come up with the predictions they offer. DNNs effectively work (though not exactly) by matching patches in the images to a &quot;dictionary&quot; of patches, one stored in each neuron (see &lt;a href=&quot;http://research.google.com/archive/unsupervised_icml2012.html&quot;&gt;the youtube cat paper&lt;/a&gt;). Thus, it may not have a high level view of the image since it only looks at patches, and images are usually downscaled to much lower resolution to obtain the results in current generation systems. Methods which look at how the components of the image interact may be able to avoid these problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some questions to ask for this work are: How confident were the networks when they made these predictions? How much volume do such adversarial images occupy in the space of all images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some work I am aware of in this regard comes from Dhruv Batra and Devi Parikh's Lab at Virginia Tech who look into this for question answering systems: &lt;a href=&quot;http://arxiv.org/pdf/1606.07356v1.pdf&quot;&gt;Analyzing the Behavior of Visual Question Answering Models&lt;/a&gt; and &lt;a href=&quot;https://filebox.ece.vt.edu/~dbatra/papers/gmpb_icmlvis16.pdf&quot;&gt;Interpreting Visual Question Answering models&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More such work is needed, and just as the human visual system does also get fooled by such &quot;optical illusions&quot;, these problems may be unavoidable if we use DNNs, though AFAIK nothing is yet known either way, theoretically or empirically. &lt;/p&gt;&#xA;" OwnerUserId="130" LastActivityDate="2016-08-03T23:25:53.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="272" PostTypeId="2" ParentId="233" CreationDate="2016-08-04T00:57:01.833" Score="3" Body="&lt;p&gt;To have any chance at answering this, you'd first need a rigorous definition of &quot;true artificial intelligence&quot;, which we don't have.  And even if you had that, the best answer would probably be &quot;nobody knows.&quot;  We don't even understand exactly how human intelligence (which is probably the best model of intelligence we have available to study) works.  What we do know (or think we know) is that ANN's are at best a very superficial replica of brain function.  It may turn out that they're absolutely the wrong path to achieving &quot;true artificial intelligence&quot; although I expect most people would be surprised if that turned out to be the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What probably wouldn't be so surprising would be if some other technique emerged which is better than ANN's, OR if it turns out that you need an ensemble of techniques.  Personally I think it's close to self-evident that the brain works largely in a probabilistic fashion, but it's also clear that we do sometimes use symbolic processing / deductive logic / rules / etc.  And right now, ANN's don't give you much in the way of reasoning, deduction, etc.  So we may ultimately find that we have to combine a probabilistic approach like ANN's with other techniques - maybe Inductive Logic Programming or something of that nature. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-04T00:57:01.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1274" PostTypeId="1" CreationDate="2016-08-04T01:39:20.737" Score="2" ViewCount="144" Body="&lt;p&gt;Have there been any studies which attempted to use AI algorithms to detect human thoughts or emotions based on brain activity, such as using &lt;a href=&quot;https://en.wikipedia.org/wiki/Brain%E2%80%93computer_interface#EEG-based&quot; rel=&quot;nofollow&quot;&gt;BCI/EEG devices&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By this, I mean simple guesses such as whether the person was happy or angry, or what object (e.g. banana, car) they were thinking about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, did any of those studies show some degree of success?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-17T13:03:50.990" LastActivityDate="2016-08-23T15:25:23.607" Title="Are there any studies which attempt to use AI to guess the human emotion based on the brainwaves?" Tags="&lt;research&gt;&lt;signal-processing&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1275" PostTypeId="5" CreationDate="2016-08-04T04:07:42.000" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-04T04:07:42.000" LastActivityDate="2016-08-04T04:07:42.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1276" PostTypeId="4" CreationDate="2016-08-04T04:07:42.000" Score="0" Body="For questions about algorithm's that exhibit characteristics of intelligence, or are critical components in systems that exhibit intelligence, and problem-solving and goal-based algorithms in general." OwnerUserId="145" LastEditorUserId="1671" LastEditDate="2018-10-15T17:00:09.787" LastActivityDate="2018-10-15T17:00:09.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1277" PostTypeId="5" CreationDate="2016-08-04T04:10:33.107" Score="0" Body="&lt;p&gt;A programming language is a formal language that specifies a set of instructions that can be used to produce various kinds of output. Programming languages generally consist of instructions for a computer. Programming languages can be used to create programs that implement specific algorithms.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-15T21:41:53.403" LastActivityDate="2018-03-15T21:41:53.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1278" PostTypeId="4" CreationDate="2016-08-04T04:10:33.107" Score="0" Body="For questions about the use of various programming languages to be used in AI and ML programming." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-15T21:41:57.557" LastActivityDate="2018-03-15T21:41:57.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1279" PostTypeId="5" CreationDate="2016-08-04T04:18:57.577" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-04T04:18:57.577" LastActivityDate="2016-08-04T04:18:57.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1280" PostTypeId="4" CreationDate="2016-08-04T04:18:57.577" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-04T04:18:57.577" LastActivityDate="2016-08-04T04:18:57.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1281" PostTypeId="5" CreationDate="2016-08-04T04:38:36.787" Score="0" Body="&lt;p&gt;The &lt;strong&gt;Turing test&lt;/strong&gt; was designed in 1950 by Alan Turing, to evaluate a machine's ability to exhibit &lt;a href=&quot;/questions/tagged/natural-language&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;natural-language&amp;#39;&quot; rel=&quot;tag&quot;&gt;natural-language&lt;/a&gt;. He introduced the test in his essay Computing Machinery and Intelligence, which can be read online &lt;a href=&quot;http://www.loebner.net/Prizef/TuringArticle.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The test has a human and an AI have a conversation, which is limited to on-screen text, and then a judge will look over the transcript, and try to see if they can tell who was who in the conversation.&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="29" LastEditDate="2016-08-30T19:44:09.247" LastActivityDate="2016-08-30T19:44:09.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1282" PostTypeId="4" CreationDate="2016-08-04T04:38:36.787" Score="0" Body="For questions about the Turing Test, a test designed to see whether a third-party person can identify, from a written transcript, who was the AI and who was the human is an human/AI conversation." OwnerUserId="145" LastEditorUserId="29" LastEditDate="2016-08-30T19:44:22.377" LastActivityDate="2016-08-30T19:44:22.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1283" PostTypeId="5" CreationDate="2016-08-04T04:41:55.367" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-04T04:41:55.367" LastActivityDate="2016-08-04T04:41:55.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1284" PostTypeId="4" CreationDate="2016-08-04T04:41:55.367" Score="0" Body="practice of decomposing a complex spoken sentence it into smaller lexical segments taking the meaning to each segment, and combining those meanings according to language and grammar rules to understand its meaning" OwnerUserId="1256" LastEditorUserId="1256" LastEditDate="2016-08-04T13:18:18.887" LastActivityDate="2016-08-04T13:18:18.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1285" PostTypeId="1" AcceptedAnswerId="1315" CreationDate="2016-08-04T05:07:03.323" Score="12" ViewCount="373" Body="&lt;p&gt;Has there been any attempts to deploy AI with blockchain technology? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any decentralized examples of AI networks with no central point of control with AI nodes acting independently (but according to a codified set of rules) creating, validating and storing the same shared decentralized database in many locations around the world?&lt;/p&gt;&#xA;" OwnerUserId="1256" LastEditorUserId="2444" LastEditDate="2019-10-10T13:19:41.153" LastActivityDate="2021-01-25T06:20:37.150" Title="Are there any decentralized examples of AI systems with blockchain technology?" Tags="&lt;multi-agent-systems&gt;&lt;blockchain&gt;" AnswerCount="7" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1286" PostTypeId="2" ParentId="249" CreationDate="2016-08-04T06:25:20.620" Score="1" Body="&lt;p&gt;I am assuming each image contains a single object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible, however, it is not as easy as you might think. Firstly, you need extract as many features as possible: original image, &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_binary_patterns&quot; rel=&quot;nofollow&quot;&gt;LBP&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Scale-invariant_feature_transform&quot; rel=&quot;nofollow&quot;&gt;SIFT&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Image_moment#Moment_invariants&quot; rel=&quot;nofollow&quot;&gt;moments&lt;/a&gt;, contour descriptors to name a few. Than concatenate these features into a single feature vector. After this step, use clustering. You will need a lot of samples to compensate for the number of features. After clustering, use a correlation method to find which features are related to each cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need features to classify within a cluster, you could do a second clustering with full set of features and apply the same method. The features that are selected for a cluster will not be suitable to classify within the cluster.&lt;/p&gt;&#xA;" OwnerUserId="210" LastActivityDate="2016-08-04T06:25:20.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1288" PostTypeId="1" CreationDate="2016-08-04T07:34:06.557" Score="8" ViewCount="2393" Body="&lt;p&gt;In their famous book entitled &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0262631113&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptrons: An Introduction to Computational Geometry&lt;/a&gt;, Minsky and Papert show that a perceptron can't solve the XOR problem. This contributed to the first AI winter, resulting in funding cuts for neural networks. However, now we know that a multilayer perceptron can solve the XOR problem easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backprop wasn't known at the time, but did they know about manually building multilayer perceptrons? Did Minsky &amp;amp; Papert know that multilayer perceptrons could solve XOR at the time they wrote the book, albeit not knowing how to train it?&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="2444" LastEditDate="2021-01-18T23:34:13.700" LastActivityDate="2021-01-19T00:47:41.100" Title="Did Minsky and Papert know that multi-layer perceptrons could solve XOR?" Tags="&lt;multilayer-perceptron&gt;&lt;history&gt;&lt;perceptron&gt;&lt;ai-winter&gt;&lt;xor-problem&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1289" PostTypeId="1" CreationDate="2016-08-04T07:41:16.090" Score="5" ViewCount="187" Body="&lt;p&gt;According to Wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial general intelligence(AGI)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial general intelligence (AGI) is the intelligence of a&#xA;  (hypothetical) machine that could successfully perform any&#xA;  intellectual task that a human being can. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to the below image (a screenshot of a picture from &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=251030664B23757C5D420F9ED97AB725?doi=10.1.1.136.7883&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;When will computer hardware match the human brain?&lt;/a&gt; (1998) by Hans Moravec, and Kurzweill also uses this diagram in his book &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Singularity_Is_Near&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Singularity Is Near: When Humans Transcend Biology&lt;/a&gt;), today's artificial intelligence is the same as that of lizards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gddKB.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gddKB.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that within 10-20 years, we, humans, are successful in creating an AGI, that is, an AI with human-level intelligence and emotions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At that point, could we destroy an AGI without its consent? Would this be considered murder?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2020-03-10T02:33:58.997" LastActivityDate="2020-03-10T02:33:58.997" Title="Can we destroy an artificial general intelligence without its consent?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;ethics&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1290" PostTypeId="1" AcceptedAnswerId="2229" CreationDate="2016-08-04T07:43:33.010" Score="10" ViewCount="894" Body="&lt;p&gt;Deep Mind has published a lot of works on deep learning in the last years, most of them state-of-the-art on their respective tasks. But how much of this work has actually been reproduced by the AI community? For instance, the Neural Turing Machine paper seems to be very hard to reproduce, according to other researchers.&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2018-04-03T19:47:18.407" Title="How much of Deep Mind's work is actually reproducible?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;research&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1292" PostTypeId="2" ParentId="1290" CreationDate="2016-08-04T07:58:33.453" Score="4" Body="&lt;p&gt;&lt;em&gt;I tend to think this question is border-line and may get close. A few comments for now, though.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;wrongx&#xA;There are (at least) two issues with reproducing the work of a company like DeepMind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Technicalities missing from publications.&lt;/li&gt;&#xA;&lt;li&gt;Access to the same level of data.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Technicalities should be workable. Some people have reproduced some of the &lt;a href=&quot;https://github.com/kristjankorjus/Replicating-DeepMind&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atari&lt;/a&gt; gaming stunts. AlphaGo is seemingly more complex and will require more work, yet that should be feasible at some point in the future (individuals may lack computing resources today).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data can be more tricky. Several companies open their data sets, but data is also the nerve of the competition...&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="-1" LastEditDate="2018-04-03T19:47:18.407" LastActivityDate="2018-04-03T19:47:18.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1293" PostTypeId="2" ParentId="1289" CreationDate="2016-08-04T08:06:28.653" Score="6" Body="&lt;p&gt;Firstly, an AGI could conceivably exhibit all of the observable properties of intelligence without being conscious. Although that may seem counter-intuitive, at present we have no physical theory that allows us to detect consciousness (philosophically speaking, a &lt;a href=&quot;http://plato.stanford.edu/entries/zombies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Zombie'&lt;/a&gt; is indistinguishable from a non-Zombie - see the writing of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0316180661&quot; rel=&quot;nofollow noreferrer&quot;&gt;Daniel Dennett&lt;/a&gt; and &lt;a href=&quot;http://consc.net/books/tcm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Chalmers&lt;/a&gt; for more on this). Destroying a non-conscious entity has the same moral cost as destroying a chair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, note that 'destroy' doesn't necessarily mean the same for entities with &lt;em&gt;persistent substrate&lt;/em&gt;, i.e. their 'brain state' can be reversibly serialized to some other storage medium and/or multiple copies of them can co-exist. So if by 'destroy' we simply mean 'switch off', then an AGI might conceivably be reassured of a subsequent re-awakening. Douglas Hofstadter gives an interesting description of such an 'episodic consciousness' in &lt;a href=&quot;http://themindi.blogspot.co.uk/2007/02/chapter-26-conversation-with-einsteins.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;A Conversation with Einstein's Brain&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by 'destroy', we mean 'irrevocably erase with no chance of re-awakening', then (unless we have a physical test which proves it is &lt;em&gt;not&lt;/em&gt; conscious) destroying an entity with a seemingly human-level awareness is clearly morally tantamount to murder. To believe otherwise would be &lt;em&gt;substrate-ist&lt;/em&gt; - a moral stance which may one day be seen as antiquated as racism.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="3989" LastEditDate="2016-12-03T07:46:15.380" LastActivityDate="2016-12-03T07:46:15.380" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1294" PostTypeId="1" AcceptedAnswerId="1313" CreationDate="2016-08-04T08:28:06.277" Score="39" ViewCount="21222" Body="&lt;p&gt;Geoffrey Hinton has been researching something he calls &quot;capsules theory&quot; in neural networks. What is it? How do capsule neural networks work?&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="2444" LastEditDate="2020-06-09T11:27:33.617" LastActivityDate="2020-06-09T15:25:50.717" Title="How do capsule neural networks work?" Tags="&lt;neural-networks&gt;&lt;capsule-neural-network&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="37" ContentLicense="CC BY-SA 4.0" />
  <row Id="1295" PostTypeId="1" AcceptedAnswerId="1299" CreationDate="2016-08-04T08:32:36.610" Score="11" ViewCount="367" Body="&lt;p&gt;During my research, I've stumbled upon &quot;complex-valued neural networks&quot;, which are neural networks that work with complex-valued inputs (probably weights too). What are the advantages (or simply the applications) of this kind of neural network over real-valued neural networks?&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2016-08-04T10:28:59.957" Title="What are the advantages of complex-valued neural networks?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1296" PostTypeId="1" CreationDate="2016-08-04T08:39:43.740" Score="16" ViewCount="4080" Body="&lt;p&gt;In &lt;a href=&quot;https://web.archive.org/web/20190519181402/http://fabelier.org/novelty-search-and-open-ended-evolution-by-ken-stanley/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, the author claims that guiding evolution by novelty alone (without explicit goals) can solve problems even better than using explicit goals. In other words, using a novelty measure as a fitness function for a genetic algorithm works better than a goal-directed fitness function. How is that possible?&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="2444" LastEditDate="2019-11-22T00:48:42.640" LastActivityDate="2019-11-22T00:48:42.640" Title="How does novelty search work?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;novelty-search&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1297" PostTypeId="1" AcceptedAnswerId="1327" CreationDate="2016-08-04T09:22:21.360" Score="3" ViewCount="86" Body="&lt;p&gt;Quote from this &lt;a href=&quot;https://ai.meta.stackexchange.com/a/46/8&quot;&gt;Eric's meta post&lt;/a&gt; about modelling and implementation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;They are not exactly the same, although strongly related. This was a very difficult lesson to learn among mathematicians and early programmers, notably in the 70s (mathematical proofs can demand a lot of non-trivial programming work to make them &quot;computable&quot;, as in runnable on a computer).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If they're not the same, what is the difference?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How we can say when we're talking about AI implementation, and when about modelling? It's suggested above it's not easy task. So where we can draw the line when we talk about it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking in general, not specifically for this site, that's why I haven't posted question in meta&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-03-16T16:44:15.193" LastActivityDate="2016-08-08T04:44:18.407" Title="How to distinguish AI modeling from implementation?" Tags="&lt;models&gt;&lt;comparison&gt;&lt;implementation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1298" PostTypeId="2" ParentId="241" CreationDate="2016-08-04T09:52:17.587" Score="4" Body="&lt;p&gt;I might be wrong, but I do not believe that something of the scope you describe would be possible with the current state of technology. It would require a lot of things which are still in relatively early stages of research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For one, just extracting relevant information from text is a huge task by itself. Doubly so with a novel which contains a large amount of unimportant details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might perhaps be easier if the input was presented in the form of some sort of list of important facts. But it would still be rather difficult for the AI to connect them and find a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, let's say that we have these two facts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alice died of a snake bite.&lt;/li&gt;&#xA;&lt;li&gt;Bob was seen buying a couple of mice recently.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To a human, it seems obvious that the mice were bought to feed a venomous snake. However, it would probably require a tremendous effort to teach an AI to make such connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I don't do text processing myself, so I'm not quite up-to-date on the current state-of-the-art. It's possible that some of these things have already been done in some form. If anyone knows more about the subject, please correct me if I'm wrong.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2016-08-04T09:59:50.663" LastActivityDate="2016-08-04T09:59:50.663" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1299" PostTypeId="2" ParentId="1295" CreationDate="2016-08-04T10:28:59.957" Score="8" Body="&lt;p&gt;According to &lt;a href=&quot;http://link.springer.com/chapter/10.1007/3-540-44989-2_118&quot;&gt;this paper&lt;/a&gt;, complex-valued ANNs (C-ANNs) can solve problems such as XOR and symmetry detection with a smaller number of layers than real ANNs (for both of these a 2 layer C-ANN suffices, whereas a 3-layer R-ANN is required).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that it is still an open question as to how useful this result is in practice (e.g. whether it actually makes finding the right topology easier), so at present the key practical advantage of C-ANNs is when they are a closer model for the problem domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Application areas are then where complex values arise naturally, e.g. in optics, signal processing/FFT or electrical engineering.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-04T10:28:59.957" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1300" PostTypeId="2" ParentId="1296" CreationDate="2016-08-04T11:10:36.090" Score="7" Body="&lt;p&gt;As explained in an answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/240/what-exactly-are-genetic-algorithms-and-what-sort-of-problems-are-they-good-for/244#244&quot;&gt;this AI SE question&lt;/a&gt;, GAs are 'satisficers' rather than 'optimizers' and tend not to explore 'outlying' regions of the search space. Rather, the population tends to cluster in regions that are 'fairly good' according to the fitness function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, I believe the thinking is that novelty affords a kind of dynamic fitness, tending to push the population away from previously discovered areas.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-04T11:10:36.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1301" PostTypeId="1" AcceptedAnswerId="1337" CreationDate="2016-08-04T12:10:26.593" Score="1" ViewCount="55" Body="&lt;p&gt;Given pictures with multiple features such as faces, can single AI algorithm detect all of them, or for better reliability is it preferred to use separate instances?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words I'm talking about attempt of finding all possible human faces on the same picture by a single neural network.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="127" LastEditDate="2016-08-04T21:07:01.750" LastActivityDate="2016-08-04T21:07:01.750" Title="Do you need single or multiple networks to detect multiple faces?" Tags="&lt;deep-neural-networks&gt;&lt;algorithm&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1303" PostTypeId="1" AcceptedAnswerId="1305" CreationDate="2016-08-04T12:28:07.427" Score="3" ViewCount="3436" Body="&lt;p&gt;I read some information&lt;sup&gt;1&lt;/sup&gt; about attempts to build neural networks in the PHP programming language. Personally I think PHP is not the right language to do so at all probably because it's a high-level language, I assume low level language are way more suitable for AI in terms of performance and scalability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good/logical reason why you should or shouldn't use PHP as a language to write AI in?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/em&gt; &lt;a href=&quot;http://www.developer.com/lang/php/creating-neural-networks-in-php.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.developer.com/lang/php/creating-neural-networks-in-php.html&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/2303357/are-there-any-artificial-intelligence-projects-in-php-out-there&quot;&gt;https://stackoverflow.com/questions/2303357/are-there-any-artificial-intelligence-projects-in-php-out-there&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="217" LastEditorUserId="-1" LastEditDate="2017-05-23T12:39:33.010" LastActivityDate="2016-10-31T09:06:48.290" Title="Can PHP be considered as a serious programming language for AI?" Tags="&lt;neural-networks&gt;&lt;programming-languages&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2016-08-04T14:55:53.457" ContentLicense="CC BY-SA 3.0" />
  <row Id="1305" PostTypeId="2" ParentId="1303" CreationDate="2016-08-04T12:35:10.167" Score="6" Body="&lt;p&gt;&lt;em&gt;Question on-topicness questionable, but...&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The most logical reason why PHP is unsuited for neural networks is that PHP is, well, intended to be used for server side webpages. It can connect to various external resources, such as databases, via native language features. It is very much a glue language, and not a processing language. PHP is also mostly stateless, only allowing you to store state in either clients, file storage or databases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, it's &lt;strong&gt;not&lt;/strong&gt; suitable for this sort of thing - not because PHP is a high level language, but rather because it's so request based and focused towards creating pages to serve to clients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That won't stop people from trying, though - there are various esoteric programming languages out there in which regular programming would be an insane task or not possible at all - but from a ease of development perspective, making a neural network in PHP makes no sense.&lt;/p&gt;&#xA;" OwnerUserId="74" LastEditorUserId="74" LastEditDate="2016-10-31T09:06:48.290" LastActivityDate="2016-10-31T09:06:48.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1306" PostTypeId="1" AcceptedAnswerId="1312" CreationDate="2016-08-04T12:40:22.857" Score="1" ViewCount="179" Body="&lt;p&gt;I've found &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-1-4613-1009-9_2&quot; rel=&quot;nofollow&quot;&gt;this old scientific paper from 1988&lt;/a&gt; about introduction of AI into nuclear power fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Were or still are there any dangers by application of such algorithm? Are nuclear power plants or human life in risk if the algorithm will fail?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially applications to the core, like cooling systems and other components which can be affected in negative way.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-04T14:10:15.490" LastActivityDate="2017-08-05T06:27:25.440" Title="What are the dangers of AI applications for nuclear industry?" Tags="&lt;applications&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1307" PostTypeId="2" ParentId="1303" CreationDate="2016-08-04T12:50:23.533" Score="3" Body="&lt;p&gt;Actually, yes. Remember, that due to the history of PHP development, some very good things has formed what we have now:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;From a simple/laggy/limited interpreter in PHP 3, we have now three mainstream lines coming one-by-one like v5/v6/v7 with &lt;em&gt;full bytecode&lt;/em&gt; supported.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In PHP v7 you don't even need a bytecode cache due to HHVM, old Zend VM is a hell-good-debugged and using a cacher like XCache you can achieve a true native execution speed &lt;strong&gt;and&lt;/strong&gt; payload&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The PHP language interface allows &lt;strong&gt;any&lt;/strong&gt; external C/C++ library &lt;em&gt;just to be added&lt;/em&gt; as a module via very simple wrapper that can be written by the person that just red Kerrigan&amp;amp;Richie and Straustrup base books on C and C++. This is amazing feature, exclusive to PHP as far as I know&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In PHP v7 you're welcome to use &lt;em&gt;native&lt;/em&gt; multi-threading and even CUDA-based things, if you wish to do it. I did it, so I can confirm that it works&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1263" LastActivityDate="2016-08-04T12:50:23.533" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1308" PostTypeId="1" AcceptedAnswerId="7901" CreationDate="2016-08-04T12:58:01.810" Score="6" ViewCount="90" Body="&lt;p&gt;Since we've self-driving cars already, would we have self-flying commercial flights in the near future? Basically the AI which can do take off, flying, landing and parking.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2018-09-08T15:09:45.913" LastActivityDate="2018-09-10T03:41:13.010" Title="Is AI capable to replace pilots entirely on the commercial flights?" Tags="&lt;real-world&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1310" PostTypeId="2" ParentId="241" CreationDate="2016-08-04T13:40:05.177" Score="2" Body="&lt;p&gt;Generally agree with @Inquisitive Lurker, but I think we also have a wide range of potential abilities/requirements. As with computer chess or Go, where there's a big difference between &quot;beating an honest novice human &quot; and &quot;beating all humans&quot;; there's a big difference between solving a simple kids' mystery and a complex adult novel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I don't think there would be any problem writing a program that could solve a problem that is listed as a list of statements, or laid out as a (very young) children's book. However something like an Agatha Christie or John Le Carre's &quot;Tinker Tailor Soldier Spy&quot; (relatively simple solution, but the story is told in a complex manner) are far in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes an alternative approach might work. For example a neural network could probably solve all Colombo mysteries at the &quot;Who did it?&quot; level without a full &quot;Why?&quot; explanation, after only reading a few Colombo mysteries. The same is true for most kids!&lt;/p&gt;&#xA;" OwnerUserId="132" LastActivityDate="2016-08-04T13:40:05.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1311" PostTypeId="2" ParentId="1274" CreationDate="2016-08-04T14:00:13.253" Score="6" Body="&lt;p&gt;As per this &lt;a href=&quot;http://www.dailymail.co.uk/sciencetech/article-2095214/As-scientists-discover-translate-brainwaves-words--Could-machine-read-innermost-thoughts.html&quot; rel=&quot;nofollow&quot;&gt;site&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Researchers recorded the complex patterns of electrical activity generated by someone’s brain, as the subject listened to someone talking.&#xA;  By feeding those brainwave patterns into a computer, they were able to translate them back into actual words — the same words that the volunteer had been hearing.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;The scientists behind the work believe they can now go further and read the unspoken thoughts of people using electrodes placed against the brain.&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In the experiment, each patient listened to a recording of spoken words for five to ten minutes, while the net of electrodes placed under their skull monitored activity in a part of the brain involved in understanding speech called &lt;a href=&quot;https://en.wikipedia.org/wiki/Wernicke%27s_area&quot; rel=&quot;nofollow&quot;&gt;Wernicke’s area&lt;/a&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In one experiment, volunteers looked at black-and-white photographs while the scanner monitored activity in part of the brain that handles vision called the primary visual cortex.&#xA;  A computer predicted accurately the image that the person was looking at purely from the pattern of brain activity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So AI might be able to read our emotions as well in near future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that &lt;a href=&quot;http://electronics.howstuffworks.com/gadgets/high-tech-gadgets/google-glass-detect-emotions.htm&quot; rel=&quot;nofollow&quot;&gt;google glasses can detect people's emotion&lt;/a&gt; via facial expression, voice tone e.t.c, (just like us), obviously not what they are thinking in their brain.&lt;/p&gt;&#xA;" OwnerUserId="72" LastEditorUserId="145" LastEditDate="2016-08-23T15:25:23.607" LastActivityDate="2016-08-23T15:25:23.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1312" PostTypeId="2" ParentId="1306" CreationDate="2016-08-04T14:04:35.743" Score="2" Body="&lt;p&gt;Any technology in the nuclear industry represents variance--it may be an improvement in safety or efficiency, or it may contain some unseen defect that allows a catastrophe to happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the simple &lt;em&gt;possibility&lt;/em&gt; of harm isn't enough to swing the decision one way or the other. The application of AI methods--whether to the real-time control of plant variables, or the early detection of problems, or to the design of plants and their components--seems likely to be as beneficial as in other realms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, check out the &lt;a href=&quot;http://www.lasar.polimi.it/?page_id=798&quot; rel=&quot;nofollow&quot;&gt;publication list&lt;/a&gt; of a lab active in this area. Their paper I'm most familiar with is one in which they build a fault detector paired with a fault library classifier, so that the operators can be alerted not just that something is abnormal but what fault has probably occurred. This is done in such a way that standardized plants (such as, say, the French nuclear system) can share records with each other, meaning that &lt;em&gt;any&lt;/em&gt; plant has the experience of &lt;em&gt;every&lt;/em&gt; plant at their fingertips.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-04T14:04:35.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1313" PostTypeId="2" ParentId="1294" CreationDate="2016-08-04T14:16:57.367" Score="32" Body="&lt;p&gt;It appears to not be published yet; the best available online are &lt;a href=&quot;http://cseweb.ucsd.edu/~gary/cs200/s12/Hinton.pdf&quot; rel=&quot;noreferrer&quot;&gt;these slides&lt;/a&gt; for &lt;a href=&quot;https://www.youtube.com/watch?v=TFIMqt0yT2I&quot; rel=&quot;noreferrer&quot;&gt;this talk&lt;/a&gt;. (Several people reference an earlier talk with &lt;a href=&quot;http://techtv.mit.edu/collections/bcs/videos/30698-what-s-wrong-with-convolutional-nets&quot; rel=&quot;noreferrer&quot;&gt;this link&lt;/a&gt;, but sadly it's broken at time of writing this answer.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My impression is that it's an attempt to formalize and abstract the creation of subnetworks inside a neural network. That is, if you look at a standard neural network, layers are fully connected (that is, every neuron in layer 1 has access to every neuron in layer 0, and is itself accessed by every neuron in layer 2). But this isn't obviously useful; one might instead have, say, &lt;em&gt;n&lt;/em&gt; parallel stacks of layers (the 'capsules') that each specializes on some separate task (which may itself require more than one layer to complete successfully).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm imagining its results correctly, this more sophisticated graph topology seems like something that could easily increase both the effectiveness and the interpretability of the resulting network.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="6050" LastEditDate="2017-03-27T18:15:19.623" LastActivityDate="2017-03-27T18:15:19.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1314" PostTypeId="1" AcceptedAnswerId="1316" CreationDate="2016-08-04T14:18:10.547" Score="10" ViewCount="3498" Body="&lt;p&gt;How much processing power is needed to emulate the human brain? More specifically, the neural simulation, such as communication between the neurons and processing certain data in real-time.&lt;/p&gt;&#xA;&lt;p&gt;I understand that this may be a bit of speculation and it's not possible to be accurate, but I'm sure there is some data available or research studies that attempted to estimate it based on our current understanding of the human brain.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-26T01:53:46.537" LastActivityDate="2021-01-26T01:55:25.793" Title="How powerful a computer is required to simulate the human brain?" Tags="&lt;hardware&gt;&lt;neuromorphic-engineering&gt;&lt;neuroscience&gt;&lt;human-brain&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1315" PostTypeId="2" ParentId="1285" CreationDate="2016-08-04T14:23:37.070" Score="6" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Swarm_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swarm intelligence&lt;/a&gt; is the term for systems where relatively simple agents work together to solve a complicated problem in a decentralized fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, distributed computing methods are very important for dealing with problems at scale, and many of them embrace decentralization in a deep way. (Given the reality of hardware failure and the massive size of modern datasets relative to individual nodes, the less work is passed through a central bottleneck, the better.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are people interested in doing computation on the blockchain, it seems to me like it's unlikely to be competitive with computation in dedicated clusters (like AWS).&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="2444" LastEditDate="2019-10-10T13:22:42.030" LastActivityDate="2019-10-10T13:22:42.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1316" PostTypeId="2" ParentId="1314" CreationDate="2016-08-04T14:35:16.053" Score="10" Body="&lt;p&gt;H+ magazine wrote an estimate &lt;a href=&quot;http://hplusmagazine.com/2009/04/07/brain-chip/&quot; rel=&quot;noreferrer&quot;&gt;in 2009&lt;/a&gt; that seems broadly comparable to other things I've seen; they think the human brain is approximately 37 petaflops. A supercomputer larger than that 37 petaflop estimate &lt;a href=&quot;https://en.wikipedia.org/wiki/Sunway_TaihuLight&quot; rel=&quot;noreferrer&quot;&gt;exists today&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But emulation is hard. See &lt;a href=&quot;https://stackoverflow.com/questions/471973/what-makes-building-game-console-emulators-so-hard&quot;&gt;this SO question about hardware emulation&lt;/a&gt; or &lt;a href=&quot;http://www.tested.com/tech/gaming/2712-why-perfect-hardware-snes-emulation-requires-a-3ghz-cpu/&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt; on emulating the SNES, in which they require &lt;strong&gt;140 times&lt;/strong&gt; the processing power of the SNES chip to get it right. &lt;a href=&quot;http://gizmodo.com/an-83-000-processor-supercomputer-only-matched-one-perc-1045026757&quot; rel=&quot;noreferrer&quot;&gt;This 2013 article&lt;/a&gt; claims that a second of human brain activity took 40 minutes to emulate on a 10 petaflop computer (a &lt;em&gt;2400-times&lt;/em&gt; slowdown, not the 4-times slowdown one might naively expect).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And all this assumes that neurons are relatively simple objects! It could be that the amount of math we have to do to model a single neuron is actually much more than the flops estimate above. Or it could be the case that dramatic simplifications can be made, and if we knew what the brain was actually trying to accomplish we could do it much more cleanly and simply. (One advantage that ANNs have, for example, is that they are doing computations with much more precision than we expect biological neurons to have. But this means emulation is &lt;em&gt;harder&lt;/em&gt;, not easier, while replacement &lt;em&gt;is&lt;/em&gt; easier.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="-1" LastEditDate="2017-05-23T12:39:33.010" LastActivityDate="2016-08-04T14:35:16.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1317" PostTypeId="2" ParentId="1314" CreationDate="2016-08-04T14:36:01.980" Score="2" Body="&lt;p&gt;Not just how much, but what kind of processing power : there're specially-crafted &lt;a href=&quot;http://www.research.ibm.com/cognitive-computing/neurosynaptic-chips.shtml#fbid=V01grppeOIs&quot; rel=&quot;nofollow&quot;&gt;dedicated chips&lt;/a&gt;, and it has a &lt;a href=&quot;http://www.dailymail.co.uk/sciencetech/article-3516047/IBM-reveals-brain-supercomputer-neurosynaptic-chip-replicate-16-million-neurons-work-using-hearing-aid-battery.html&quot; rel=&quot;nofollow&quot;&gt;practical applications&lt;/a&gt;, so it's not a lab-only project&lt;/p&gt;&#xA;" OwnerUserId="1263" LastActivityDate="2016-08-04T14:36:01.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1318" PostTypeId="1" AcceptedAnswerId="1820" CreationDate="2016-08-04T15:10:41.230" Score="2" ViewCount="410" Body="&lt;p&gt;&lt;strong&gt;The Situation:&lt;/strong&gt;&#xA;A self-driving car is traveling at it's maximum speed, 25 mph (40 km/h), in the middle of an empty street with the ability to change lanes on both sides. There are two passengers, one in the front and another in the back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone jumps from the side of the road directly into the path of the car. A collision would occur in 50 meters. &lt;a href=&quot;http://www.brake.org.uk/rsw/15-facts-a-resources/facts/1255-speed&quot; rel=&quot;nofollow&quot;&gt;Breaking distance&lt;/a&gt; at this speed is about 24m.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Question:&lt;/strong&gt; Is it known how the current implementation of the Google Car AI would react, or is it currently a matter of speculation? A step-by-step explanation of the AI's decisioning process would be preferred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Possible Answers:&lt;/strong&gt; The car could activate its brakes immediately, coming to a halt as quickly as possible. This would be sooner than a human could stop, as people require time to recognize the possibility of a collision, and then physically slam on the brake. (&lt;em&gt;thinking distance&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, the car could continue traveling forward, processing the situation. (Similar to a humans &lt;em&gt;thinking distance&lt;/em&gt;). The person may continue to move, either out of the way, or still into danger of being hit. In this case, the car may decide to change lanes in an attempt to pass around the person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly and most unlikely, the car will not alter its course and proceed to drive forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Do not attempt to do it to check;)&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="1812" LastEditDate="2016-08-31T23:17:29.040" LastActivityDate="2016-09-01T01:52:21.400" Title="What would happen if someone jumped in the front of a Google car?" Tags="&lt;autonomous-vehicles&gt;&lt;decision-theory&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1319" PostTypeId="2" ParentId="70" CreationDate="2016-08-04T15:22:05.297" Score="5" Body="&lt;p&gt;The simple answer is &quot;no, they aren't limited to images&quot;: CNNs are also being used for natural language processing. (See &lt;a href=&quot;http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; for an introduction.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't seen them applied to graphical data yet, but I haven't looked; there are some obvious things to try and so I'm optimistic that it would work.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-04T15:22:05.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1320" PostTypeId="1" AcceptedAnswerId="1324" CreationDate="2016-08-04T15:25:37.293" Score="1" ViewCount="235" Body="&lt;p&gt;Artificial intelligence is present in many games, both current and older games. How can such intelligence understand what to do? I mean, how can it behave like a human in a game, allowing you to play against itself, or that AI plays against itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In games like Age of Empires, for example.&lt;/p&gt;&#xA;" OwnerUserId="173" LastEditorUserId="145" LastEditDate="2016-08-09T20:36:48.157" LastActivityDate="2016-08-13T03:12:52.877" Title="How does artificial intelligence work in games?" Tags="&lt;gaming&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ClosedDate="2016-08-15T14:22:03.317" ContentLicense="CC BY-SA 3.0" />
  <row Id="1321" PostTypeId="2" ParentId="1320" CreationDate="2016-08-04T15:33:07.120" Score="2" Body="&lt;p&gt;Most of the existing AI bots which can play games use deep search from possible space and choose the best move. This is done by most of the chess, Go, Tic-Tac-Toe, etc bots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there has been a recent breakthrough where (deep)neural nets with deep search techniques like monte-carlo search, etc; which might be more human-like and demonstrate a much more complex game behaviour than the above bots. One such example is the Google's Alpha-Go bot.&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2016-08-04T15:33:07.120" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1322" PostTypeId="2" ParentId="240" CreationDate="2016-08-04T15:48:10.133" Score="6" Body="&lt;p&gt;There are a number of good answers here explaining what genetic algorithms are, and giving example applications. I'm adding some general purpose advice on what they are good for, but also cases where you should NOT use them.&lt;/p&gt;&#xA;&lt;p&gt;If my tone seems harsh, it is because using GAs in any of the cases in the &lt;strong&gt;inappropriate&lt;/strong&gt; section below will lead to your paper being &lt;em&gt;instantly&lt;/em&gt; rejected from any top-tier journal.&lt;/p&gt;&#xA;&lt;p&gt;First, your problem MUST be an optimization problem. You need to define a &amp;quot;fitness function&amp;quot; that you are trying to optimize and you need to have a way to measure it.&lt;/p&gt;&#xA;&lt;h3&gt;Good&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Crossover functions are easy to define and natural&lt;/strong&gt;: When dealing with certain kinds of data, crossover/mutation functions might be easy to define. For example strings (eg. DNA or gene sequences) can be mutated easily by splicing two candidate strings to obtain a new one (this is why nature uses genetic algorithms!). Trees (like phylogenetic trees or parse trees) can be spliced too, by replacing a branch of one tree with a branch from another. Shapes (like airplane wings or boat shapes) can be mutated easily by drawing a grid on the shape and combining different grid sections from the parents to obtain a child. Usually this means your problem is composed of different parts and putting together parts from distinct solutions is a valid candidate solution.&lt;/li&gt;&#xA;&lt;li&gt;This means that if your problem is defined in a vector space where the coordinates don't have any special meaning, GAs are not a good choice. If it is hard to formulate your problem as a GA, it is not worth it.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Black Box evaluation&lt;/strong&gt;: If for a candidate, your fitness function is evaluated outside the computer, GAs are a good idea. For example, if you are testing a wing shape in an air tunnel, genetic algorithms will help you generate good candidate shapes to try.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Exception: Simulations&lt;/strong&gt;. If your fitness function is measuring how well a nozzle design performs and requires simulating the fluid dynamics for each nozzle shape, GAs may work well for you. They may also work if you are simulating a physical system through time and are interested in how well your design performs over the course of the operation eg. &lt;a href=&quot;https://www.youtube.com/watch?v=dRthdBr46cs&quot; rel=&quot;nofollow noreferrer&quot;&gt;modelling locomotion patterns&lt;/a&gt;. However, methods that use partial differential equations as constraints are being developed in the literature, eg. &lt;a href=&quot;https://www.siam.org/meetings/op08/Heinkenschloss.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDE constrained optimization&lt;/a&gt;, so this may change in the future.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Inappropriate&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;You can calculate a gradient&lt;/strong&gt; for your function: If you have access to the gradient of your function, you can do gradient descent, which is in general much more efficient than GAs. Gradient descent may have issues with local minima (as will GAs) but many methods have been studied to mitigate this.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;You know the fitness function in closed form&lt;/strong&gt;: Then, you can probably calculate the gradient. Many languages have libraries supporting &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_differentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic differentiation&lt;/a&gt;, so you don't even need to do it manually. If your function is not differentiable, then you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Subgradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;subgradient descent&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Your optimization problem is of a known form, like a &lt;strong&gt;linear program or a quadratic program&lt;/strong&gt;: GAs (and black box optimization methods in general) are very inefficient in terms of the number of candidates they need to evaluate, and are best avoided if possible.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Your solution space is small&lt;/strong&gt;: If you can grid your search space efficiently, you can guarantee that you have found the best solution, and can make contour plots of the solution space to see if there is a region you need to explore further.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Finally, if you are considering a GA, consider more recent work in Evolutionary Strategies. I am biased towards &lt;a href=&quot;https://en.wikipedia.org/wiki/CMA-ES&quot; rel=&quot;nofollow noreferrer&quot;&gt;CMA-ES&lt;/a&gt;, which I think is a good simple algorithm that captures the notion of a gradient in the fitness landscape in a way that traditional GAs do not.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="2444" LastEditDate="2021-01-07T00:43:15.233" LastActivityDate="2021-01-07T00:43:15.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1323" PostTypeId="1" AcceptedAnswerId="1325" CreationDate="2016-08-04T15:49:13.793" Score="3" ViewCount="72" Body="&lt;p&gt;&lt;a href=&quot;https://cs.stackexchange.com/a/60535/54605&quot;&gt;At a related question in Computer Science SE&lt;/a&gt;, a user told:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Neural networks typically require a large training set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is there a way to define the boundaries of the &quot;optimal&quot; size of a training set in general case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was learning about fuzzy logic, I've heard some rules of thumb that involved examining the mathematical composition of the problem and using that to define the number of fuzzy sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there such a method that can be applicable for an already defined neural network topology? &lt;/p&gt;&#xA;" OwnerUserId="1270" LastEditorUserId="4302" LastEditDate="2018-07-23T15:36:34.730" LastActivityDate="2018-07-23T15:36:34.730" Title="Is there a way to define the boundaries of the optimal size of a training set?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;optimization&gt;&lt;topology&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1324" PostTypeId="2" ParentId="1320" CreationDate="2016-08-04T16:04:43.973" Score="5" Body="&lt;p&gt;There are many different kinds of AI used in games; AI for historical board games (like chess or Go) tends to be much better than AI for computer games (such as Starcraft or Civilization), in large part because there's more academic interest in developing strategies for those games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic structure of a game-playing AI is that it takes in game state inputs and outputs an action; typically, the internals also contain some sort of goal and some sort of future prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But beyond that, there's tremendous amounts of variability. Some AI are little more than scripted reflexes, some are built like control systems, some do actual optimization and forward thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting into the details of &lt;em&gt;how&lt;/em&gt; the many different approaches work is probably beyond the scope of this site, though.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-04T16:04:43.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1325" PostTypeId="2" ParentId="1323" CreationDate="2016-08-04T16:09:38.327" Score="4" Body="&lt;p&gt;For a finite value to be 'optimal,' typically you need some benefit from more paired up with some cost for more, and eventually the lines cross because the benefit decreases and the cost increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most models will have a reduction in error with more training data, that asymptotically approaches the best the model can do. See this image (from &lt;a href=&quot;http://blog.revolutionanalytics.com/2015/09/why-big-data-learning-curves.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) as an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jx1AZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jx1AZ.png&quot; alt=&quot;Decreasing error with increasing training set size&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The costs of training data are also somewhat obvious; data is costly to obtain, to store, and to move. (Assuming model complexity stays constant, the actual cost of storing, moving, and using the model remains the same, since the weights in the model are just being tuned.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So at some point the slope of the error-reduction curve becomes horizontal enough that more data points are costlier than they're worth, and that's the optimal amount of training data.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-04T16:09:38.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1326" PostTypeId="2" ParentId="1323" CreationDate="2016-08-04T16:15:38.400" Score="2" Body="&lt;p&gt;In general, the larger the training set, the better. See &lt;a href=&quot;http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35179.pdf&quot; rel=&quot;nofollow&quot;&gt;The Unreasonable effectiveness of Data&lt;/a&gt;, though this article is quite dated (written in 2009). Xavier Amatriain, a researcher at Netflix has a &lt;a href=&quot;https://www.quora.com/In-machine-learning-is-more-data-always-better-than-better-algorithms&quot; rel=&quot;nofollow&quot;&gt;Quora answer&lt;/a&gt; where he discusses that more data can sometimes hurt algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For deep neural networks in particular, it does not seem that we have hit these limits yet. &lt;/p&gt;&#xA;" OwnerUserId="130" LastActivityDate="2016-08-04T16:15:38.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1327" PostTypeId="2" ParentId="1297" CreationDate="2016-08-04T16:24:09.050" Score="4" Body="&lt;p&gt;One good way of differentiating modelling and implementation is to consider that models occupy a much higher level of abstraction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To continue with the mathematical example: even though experimental mathematics might be dependent on computation, the program can be considered as one possible realization of the necessary conditions of a more abstract existence proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over the last 25 years, software engineering methodologies have become quite good at separating models and implementations, e.g. by using interfaces/typeclasses/abstract base classes to define constraints on behavior that is concretely realized by the implementation of derived classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI has always been a battle between the &lt;a href=&quot;https://en.wikipedia.org/wiki/Neats_vs._scruffies&quot; rel=&quot;nofollow&quot;&gt;'neats and the scruffies'&lt;/a&gt;. Neats tend to prefer working 'top down' from clean abstractions, 'scruffies' like to work 'bottom up', and 'bang the bits' of the implementation together, to see what happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in practice, interplay between both styles is necessary, but AI &lt;em&gt;as a science&lt;/em&gt; progresses when we abstract mechanisms away from specific implementations into their most general (and hence re-useable) form.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-04T16:24:09.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1328" PostTypeId="2" ParentId="1314" CreationDate="2016-08-04T17:15:56.423" Score="6" Body="&lt;p&gt;The human brain contains about 100 billion neurons (&lt;span class=&quot;math-container&quot;&gt;$10^{11}$&lt;/span&gt;) and about a hundred trillion synapses (&lt;span class=&quot;math-container&quot;&gt;$10^{14}$&lt;/span&gt;). Each neuron can fire about 100 times a second. If we model the brain as a simple neural network, then it would be equivalent to a machine that requires 1016 calculations per second and 1013 bits of memory.&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Singularity_Is_Near#The_brain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Kurzweil introduces the idea of &amp;quot;uploading&amp;quot; a specific brain with every mental process intact, to be instantiated on a &amp;quot;suitably powerful computational substrate&amp;quot;. He writes that general modeling requires 1016 calculations per second and 1013 bits of memory, but then explains uploading requires additional detail, perhaps as many as 1019 cps and 1018 bits. Kurzweil says the technology to do this will be available by 2040.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;According to this site &lt;a href=&quot;http://www.extremetech.com/extreme/163051-simulating-1-second-of-human-brain-activity-takes-82944-processors&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Using the NEST software framework, the team led by Markus Diesmann and Abigail Morrison succeeded in creating an artificial neural network of 1.73 billion nerve cells connected by 10.4 trillion synapses. While impressive, this is only a fraction of the neurons every human brain contains. Scientists believe we all carry 80-100 billion nerve cells&lt;/p&gt;&#xA;&lt;p&gt;It took 40 minutes with the combined muscle of 82,944 processors in K computer to get just 1 second of biological brain processing time. While running, the simulation ate up about 1PB of system memory as each synapse was modeled individually.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Computing power will continue to ramp up while transistors scale down, which could make true neural simulations possible in real-time with supercomputers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/SpiNNaker&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpiNNaker&lt;/a&gt; is a manycore computer architecture designed to &lt;a href=&quot;https://en.wikipedia.org/wiki/Human_Brain_Project&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulate the human brain&lt;/a&gt;. It is planned to use 1 million ARM processors (currently .5 million). The completed design will hold 100,000 cores&lt;/p&gt;&#xA;&lt;p&gt;In this &lt;a href=&quot;https://www.youtube.com/watch?v=2e06C-yUwlc&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;, they showed a completed rack with 100,000 cores emulating 25 million neurons (at &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{4}$&lt;/span&gt; the efficiency—it will eventually run 1,000 neurons per core).&lt;/p&gt;&#xA;" OwnerUserId="72" LastEditorUserId="2444" LastEditDate="2021-01-26T01:55:25.793" LastActivityDate="2021-01-26T01:55:25.793" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1329" PostTypeId="2" ParentId="64" CreationDate="2016-08-04T17:34:17.233" Score="4" Body="&lt;p&gt;What might be classed as AI has of course changed over the years, but landmarks and research breakthroughs include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Babbagge's &lt;a href=&quot;https://en.wikipedia.org/wiki/Difference_engine&quot; rel=&quot;nofollow&quot;&gt;Difference Engine&lt;/a&gt; (~1823) for tabulating/interpolating polynomials.&lt;/li&gt;&#xA;&lt;li&gt;Frank Rosenblatt's 1957 invention of the &lt;a href=&quot;http://blogs.umass.edu/brain-wars/files/2016/03/rosenblatt-1957.pdf&quot; rel=&quot;nofollow&quot;&gt;Perceptron&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;John McCarthy's invention of &lt;a href=&quot;https://en.wikipedia.org/wiki/Lisp_(programming_language)&quot; rel=&quot;nofollow&quot;&gt;Lisp&lt;/a&gt; in the late 1950s.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Arthur Samuel's 1959 &lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-319-30668-1_9&quot; rel=&quot;nofollow&quot;&gt;checkers player&lt;/a&gt;, which famously improved by playing against itself (it would have been nice if that had destroyed the myth about a program only being as 'intelligent' as its creator).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Newell and Simon's 1959 &lt;a href=&quot;http://bitsavers.informatik.uni-stuttgart.de/pdf/rand/ipl/P-1584_Report_On_A_General_Problem-Solving_Program_Feb59.pdf&quot; rel=&quot;nofollow&quot;&gt;General Problem Solver&lt;/a&gt; applied Means-Ends analysis to solve a range of problems expressed as Horn clauses.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Davis, Putnam et al: 1962 invention of the &lt;a href=&quot;https://en.wikipedia.org/wiki/DPLL_algorithm&quot; rel=&quot;nofollow&quot;&gt;DPLL algorithm&lt;/a&gt; which still forms the core of modern SAT-based theorem provers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lawrence Fogel et al: 1966 book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/B0000CNARU&quot; rel=&quot;nofollow&quot;&gt;Artificial Intelligence through Simulated Evolution&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rechenberg and Schwefel: 1960s development of &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolution_strategy&quot; rel=&quot;nofollow&quot;&gt;Evolutionsstrategie&lt;/a&gt; - an Evolutionary Computation approach using mutation and a form of Darwinian 'survival of the fittest'.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lotfi Zadeh's 1965 invention of &lt;a href=&quot;https://people.eecs.berkeley.edu/~zadeh/papers/Fuzzy%20Sets-Information%20and%20Control-1965.pdf&quot; rel=&quot;nofollow&quot;&gt;Fuzzy Logic&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;John Holland's 1975 book &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=531075&quot; rel=&quot;nofollow&quot;&gt;&quot;Adaptation in Natural and Artificial Systems&quot;&lt;/a&gt; which introduced Genetic Algorithms.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The 1980 &lt;a href=&quot;http://aitopics.org/sites/default/files/classic/Webber-Nilsson-Readings/Rdgs-NW-Erman-Hayes-Roth-Lesser-Reddy.pdf&quot; rel=&quot;nofollow&quot;&gt;Hearsay II&lt;/a&gt; Blackboard Architecture by Hayes-Roth et al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The 1980s invention of the &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=104293&quot; rel=&quot;nofollow&quot;&gt;backpropagation algorithm&lt;/a&gt; for Mutlilayer Perceptrons by Rumelhart, Hinton et al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-04T17:34:17.233" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1330" PostTypeId="5" CreationDate="2016-08-04T18:08:05.010" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;http://www.scholarpedia.org/article/Artificial_General_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scholarpedia.org/article/Artificial_General_Intelligence&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Artificial_general_intelligence&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2020-04-13T16:58:01.280" LastActivityDate="2020-04-13T16:58:01.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1331" PostTypeId="4" CreationDate="2016-08-04T18:08:05.010" Score="0" Body="For questions about Artificial General Intelligence (AGI), a hypothetical machine characteristic permitting it to learn any arbitrary intellectual ability up to the limits of the available computing resources." OwnerUserId="8" LastEditorUserId="4302" LastEditDate="2018-10-15T16:56:48.617" LastActivityDate="2018-10-15T16:56:48.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1332" PostTypeId="1" CreationDate="2016-08-04T18:26:31.213" Score="6" ViewCount="263" Body="&lt;p&gt;How important is true (non-&lt;a href=&quot;https://en.wikipedia.org/wiki/Pseudorandomness&quot; rel=&quot;noreferrer&quot; title=&quot;pseudo&quot;&gt;pseudo&lt;/a&gt;) randomness in Artificial Intelligence designs? Is there any chance that pseudo-randomness could be a barrier to more successful designs?&lt;/p&gt;&#xA;" OwnerUserId="46" LastActivityDate="2016-08-04T18:56:47.850" Title="How important is true randomness in AI designs?" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1333" PostTypeId="1" AcceptedAnswerId="1370" CreationDate="2016-08-04T18:31:33.580" Score="0" ViewCount="31" Body="&lt;p&gt;Complex AI that learns lexical-semantic content and its meaning (such as collection of words, their structure and dependencies) such as &lt;em&gt;Watson&lt;/em&gt; takes terabytes of disk space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets assume &lt;em&gt;DeepQA&lt;/em&gt;-like AI consumed whole Wikipedia of size 10G which took the same amount of structured and unstructured stored content.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will learning another 10G of different encyclopedia (different topics in the same language) take the same amount of data? Or will the AI reuse the existing structured and take less than half (like 1/10 of it) additional space?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="75" LastEditDate="2016-08-17T13:52:57.853" LastActivityDate="2016-08-18T00:46:03.973" Title="Does learning content from additional encyclopedias consume much less amount of storage?" Tags="&lt;watson&gt;&lt;storage&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1334" PostTypeId="1" AcceptedAnswerId="1342" CreationDate="2016-08-04T18:40:44.093" Score="3" ViewCount="139" Body="&lt;p&gt;Is there any simple explanation how &lt;em&gt;Watson&lt;/em&gt; finds and scores evidence after gathering massive evidence and analyzing the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, how does it know which precise answer it needs to return?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="75" LastEditDate="2016-08-17T13:52:23.383" LastActivityDate="2019-07-03T03:21:53.060" Title="How does Watson find and evaluate its evidence to the answer?" Tags="&lt;watson&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1335" PostTypeId="2" ParentId="1332" CreationDate="2016-08-04T18:56:47.850" Score="5" Body="&lt;p&gt;Randomness is typically the best one can do with ignorance, rather than a source of strength in its own right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the primary use of randomness in statistics is random assignment (A/B testing, randomized controlled trials, etc.). The reason to do this is to make the influence of confounders independent from the influence of the factor under investigation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But randomness only works for this &lt;em&gt;in expectation&lt;/em&gt;. If we actually knew what the confounders were, we could do a paired assignment (or a similar scheme) that ensured the various groups were matched &lt;em&gt;as well as possible&lt;/em&gt;, instead of us just not knowing ahead of time which way the bias went.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There are some cases where pseudorandomness, rather than full randomness, will impair training AI designs. A simple example would be a case where you want to randomly initialize weights in a network where the number of parameters exceeds the periodicity of the RNG; this means that while you have as many possible networks as there are possible unique seeds, you can't actually visit the entire weight space that you wanted to sample over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think any of those cases are limiting factors, however. Having truly random stochastic gradient descent instead of pseudorandom stochastic gradient descent doesn't seem like it would make a serious difference in the trajectory of AI designs.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-04T18:56:47.850" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1337" PostTypeId="2" ParentId="1301" CreationDate="2016-08-04T19:36:06.863" Score="3" Body="&lt;p&gt;AFAIK, normally detection algorithms work in a sub-window of the image and not the whole of it. For example, for a specific size and orientation you slide a sub-window on the image and extract sub-images. Then you apply your algorithm on every sub-image for detection and report the size-and-orientations with positive results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have a single neural network for face detection in this case or you might want to have different detectors for different orientation or any other feature, that is your decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the technique of &lt;a href=&quot;https://en.wikipedia.org/wiki/Ensembles_of_classifiers&quot; rel=&quot;nofollow&quot;&gt;Combining Classifiers&lt;/a&gt; by which you can improve the decision of single classifiers by combining them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Ensemble_learning&quot; rel=&quot;nofollow&quot;&gt;Ensemble Learning&lt;/a&gt; is another way in which your classifiers are not trained independently but rather together. In fact, the well-known &lt;a href=&quot;https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework&quot; rel=&quot;nofollow&quot;&gt;object detector of Viola and Jones&lt;/a&gt; uses such a technique.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2016-08-04T19:56:38.653" LastActivityDate="2016-08-04T19:56:38.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1338" PostTypeId="2" ParentId="36" CreationDate="2016-08-04T20:12:38.983" Score="9" Body="&lt;p&gt;Until we can make a quantum computer with a lot more qubits, the potential to further develop AI will remain just that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D-Wave (which has just made a 2,000+ qubit system around 2015) is an &lt;em&gt;adiabatic quantum computer&lt;/em&gt;, not a general-purpose quantum computer. It is restricted to certain optimization problems (at which its effectiveness &lt;a href=&quot;https://en.wikipedia.org/wiki/D-Wave_Systems#Reception&quot; rel=&quot;nofollow noreferrer&quot;&gt;has reportedly been doubted&lt;/a&gt; by one of the originators of the theory on which it is based).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that we could build a 32 qubit general-purpose quantum computer (twice as big as current models, as far as I'm aware). This would still mean that only 2&lt;sup&gt;32&lt;/sup&gt; possibilities exist in superposition. This is a space small enough to be explored exhaustively for many problems. Hence, there are perhaps not so many problems for which any of the known quantum algorithms (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Shor%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shor&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Grover%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grover&lt;/a&gt;) would be useful for that number of bits.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="14723" LastEditDate="2018-04-12T02:34:20.583" LastActivityDate="2018-04-12T02:34:20.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1339" PostTypeId="2" ParentId="1274" CreationDate="2016-08-04T20:15:37.607" Score="5" Body="&lt;p&gt;There has been previous research with promising results cited at length in the following recent article, and although they have limited training data, here is some &lt;a href=&quot;http://uaf46365.ddns.uark.edu/SarahStolze_Thesis.pdf&quot; rel=&quot;noreferrer&quot;&gt;impressive research for an undergraduate thesis at the University of Arkansas&lt;/a&gt; which extends that research using an artificial neural network on enhancing a classifying algorithm's capacity to facilitate unspoken, or imagined, speech recognition by collecting and analyzing a large dataset of simultaneous EEG signal and video data streams. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Imagined speech (unspoken speech, silent speech, or covert speech) is&#xA;  the process by which one thinks about a word, or “hears” the word in&#xA;  one’s head, in the absence of any vocalization or physical movement&#xA;  indicating the word. Though there exists evidence that it is possible&#xA;  for imagined speech information to be captured and interpreted. To&#xA;  facilitate imagined speech, a Brain-to-Computer Interface (BCI) must&#xA;  be implemented to provide silent communication abilities directly&#xA;  between the two entities. One of the most popular methods for&#xA;  interfacing directly between a human brain and a computer is through&#xA;  electroencephalographic signals.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Researchers have created models capable of achieving 70 - 90%&#xA;  predictive accuracy in recognizing patterns in EEG data;&#xA;  however, the accuracy of current methods for unspoken speech&#xA;  recognition is not yet sufficient to enable fluid communication&#xA;  between humans and machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;High Level Experiment Design&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the subjects were asked to imagine a specific word or feeling (label).&#xA;  The subjects responded to a set of uniform verbal cues describing the&#xA;  set of labels as well as the desired individual label to imagine. The&#xA;  data was then processed in order to minimize the effects of irrelevant&#xA;  signal activity, or noise. Additionally the data was processed to&#xA;  minimize its volume while still maintaining the core “information” in&#xA;  the data. The condensed dataset was created by dropping irrelevant&#xA;  information from the EEG device and applying principal component&#xA;  analysis (PCA) to the video stream data. Once the data was processed&#xA;  and assembled into the correct format, cross-validation using a random&#xA;  forest algorithm was performed on the control group of EEG signals&#xA;  alone and on the hypothesis group consisting of both EEG and video&#xA;  data. The predictive accuracy measurements obtained from the&#xA;  cross-validation experiments were used as metrics to evaluate the&#xA;  success of the hypothesis.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The results show a notable improvement classifying thoughts when in conjunction with the video streams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LH6sI.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LH6sI.png&quot; alt=&quot;graph of predictive accuracy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="62" LastActivityDate="2016-08-04T20:15:37.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1340" PostTypeId="2" ParentId="156" CreationDate="2016-08-04T20:48:35.637" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.cell.com/trends/cognitive-sciences/abstract/S1364-6613(04)00243-8?_returnURL=http%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1364661304002438%3Fshowall%3Dtrue&quot; rel=&quot;noreferrer&quot;&gt;This article&lt;/a&gt; gives a description of mirror neurons in terms of Hebbian learning, a mechanism that has been widely used in AI. I don't know whether the formulation given in the article has ever actually been implemented computationally.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-04T20:48:35.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1341" PostTypeId="2" ParentId="247" CreationDate="2016-08-04T21:05:44.587" Score="3" Body="&lt;p&gt;Multilayer Perceptron (MLP) can theoretically approximate any bounded, continuous function. There's no guarantee for a discontinuous function. There are plenty of important discontinuous functions, like, say, the prime counting function.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Prime-counting_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;prime counting function&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi(n)$&lt;/span&gt; is simply equal to the number of primes less than or equal to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. It has a discontinuity about each prime &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, so good luck trying to approximate this with a neural network!&lt;/p&gt;&#xA;&lt;p&gt;However, this function is extensively studied and extremely important in number theory. See the &lt;a href=&quot;https://en.wikipedia.org/wiki/Riemann_hypothesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemann hypothesis&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="2444" LastEditDate="2021-01-22T14:34:52.270" LastActivityDate="2021-01-22T14:34:52.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1342" PostTypeId="2" ParentId="1334" CreationDate="2016-08-04T21:40:31.970" Score="4" Body="&lt;p&gt;Watson starts off by searching its massive database of sources for stuff that might be pertinent to the question. Next, it searches through all of the search results and turns them into candidate answers. For example, if one of the search results is an article, Watson might pick the title of the article as a possible answer. After finding all of these candidate answers, it proceeds to iteratively score them to determine which one is best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scoring process is very complicated, and involves finding supporting evidence for each answer, and then combining many different scoring algorithms to determine which candidate answer is the best. You can read a more detailed (but still very conceptual) overview &lt;a href=&quot;http://www.aaai.org/Magazine/Watson/watson.php&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, by the creators of Watson.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2016-08-04T21:40:31.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1343" PostTypeId="5" CreationDate="2016-08-04T21:41:23.037" Score="0" Body="&lt;p&gt;Machine learning revolves around developing self-learning computer algorithms that function by virtue of discovering patterns in data and making intelligent decisions based on such patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning explores the construction and study of algorithms that can learn from and make predictions about data. Such algorithms operate by building a model from the input, in order to make data-driven predictions or decisions, rather than following strictly static program instructions.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Related tags&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/deep-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;deep-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;deep-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/reinforcement-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;reinforcement-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;reinforcement-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/q-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;q-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;q-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/unsupervised-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;unsupervised-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;unsupervised-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/supervised-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;supervised-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;supervised-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/meta-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;meta-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;meta-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;/questions/tagged/incremental-learning&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;incremental-learning&amp;#39;&quot; rel=&quot;tag&quot;&gt;incremental-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Journals&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jmlr.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal of Machine Learning Research&lt;/a&gt; (Open Access)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.springer.com/computer/ai/journal/10994&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.springer.com/engineering/computational+intelligence+and+complexity/journal/13042&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Journal of Machine Learning and Cybernetics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ijmla.net/index.php/ijmla&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Journal of Machine Learning and Applications&lt;/a&gt; (Open Access)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ijmlc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Journal of Machine Learning and Computing&lt;/a&gt; (Open Access)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=OGxgnH8y2NM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning with Python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2255" LastEditorUserId="2255" LastEditDate="2020-05-18T22:15:57.890" LastActivityDate="2020-05-18T22:15:57.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1344" PostTypeId="4" CreationDate="2016-08-04T21:41:23.037" Score="0" Body="For questions related to machine learning (ML), which is a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data). ML is usually divided into supervised, unsupervised and reinforcement learning. Deep learning is a subfield of ML that uses deep artificial neural networks." OwnerUserId="3836" LastEditorUserId="2444" LastEditDate="2019-08-13T20:25:51.020" LastActivityDate="2019-08-13T20:25:51.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1345" PostTypeId="2" ParentId="198" CreationDate="2016-08-04T21:45:48.653" Score="5" Body="&lt;p&gt;The following survey article by researchers from IIT Bombay summarizes recent advances in sarcasm detection: &lt;a href=&quot;https://arxiv.org/abs/1602.03426&quot; rel=&quot;noreferrer&quot;&gt;Arxiv link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reference to your question, I do not think it is considered either extraordinarily difficult or open-ended. While it does introduce ambiguity that computers cannot yet handle, Humans are easily able to understand sarcasm, and are thus able to label datasets for sarcasm detection.&lt;/p&gt;&#xA;" OwnerUserId="130" LastActivityDate="2016-08-04T21:45:48.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1346" PostTypeId="5" CreationDate="2016-08-04T21:55:57.107" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-04T21:55:57.107" LastActivityDate="2016-08-04T21:55:57.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1347" PostTypeId="4" CreationDate="2016-08-04T21:55:57.107" Score="0" Body="For questions about natural language as it relates to AI but is not a processing question, which should be tagged natural-language-processing instead. This includes questions about the relationship between AI and linguistics, semantics, or language development." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-10-09T18:06:11.547" LastActivityDate="2018-10-09T18:06:11.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1348" PostTypeId="1" AcceptedAnswerId="1355" CreationDate="2016-08-04T23:49:01.983" Score="20" ViewCount="571" Body="&lt;p&gt;Isaac Asimov's famous &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot;&gt;Three Laws of Robotics&lt;/a&gt; originated in the context of Asimov's science fiction stories. In those stories, the three laws serve as a safety measure, in order to avoid untimely or manipulated situations from exploding in havoc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More often than not, Asimov's narratives would find a way to break them, leading the writer to make several modifications to the laws themselves. For instance, in some of his stories, he &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#First_Law_modified&quot;&gt;modified the First Law&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#Zeroth_Law_added&quot;&gt;added a Fourth (or Zeroth) Law&lt;/a&gt;, or even &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#Removal_of_the_Three_Laws&quot;&gt;removed all Laws altogether&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is easy to argue that, in popular culture, and even in the field of AI research itself, the Laws of Robotics are taken quite seriously. Ignoring the side problem of the different, subjective, and mutually-exclusive interpretations of the laws, are there any arguments proving the laws themselves intrinsically flawed by their design, or, alternatively, strong enough for use in reality? Likewise, has a better, stricter security heuristics set being designed for the purpose?&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="2444" LastEditDate="2020-12-04T23:12:50.163" LastActivityDate="2020-12-04T23:12:50.163" Title="Are Asimov's Laws flawed by design, or are they feasible in practice?" Tags="&lt;ai-safety&gt;&lt;asimovs-laws&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1352" PostTypeId="5" CreationDate="2016-08-05T01:17:49.000" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-05T01:17:49.000" LastActivityDate="2016-08-05T01:17:49.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1353" PostTypeId="4" CreationDate="2016-08-05T01:17:49.000" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-05T01:17:49.000" LastActivityDate="2016-08-05T01:17:49.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1354" PostTypeId="1" AcceptedAnswerId="2143" CreationDate="2016-08-05T01:45:17.633" Score="10" ViewCount="563" Body="&lt;p&gt;Are there any modern techniques of generating &lt;strong&gt;textual&lt;/strong&gt; CAPTCHA (so person needs to type the right text) challenges which can easily &lt;a href=&quot;https://ai.stackexchange.com/q/92/8&quot;&gt;fool AI&lt;/a&gt; with some visual obfuscation methods, but at the same time human can solve them without any struggle?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I'm talking about plain ability of &lt;strong&gt;recognising text embedded into image&lt;/strong&gt; (without considering any external plugins like flash or java, image classification, etc.) and re-typing the text that has been written or something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess adding noise, gradient, rotating letters or changing colours are not reliable methods any more, since they can be quickly broken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or research has been done?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-10-13T09:33:07.423" Title="Are there any textual CAPTCHA challenges which can fool AI, but not human?" Tags="&lt;image-recognition&gt;&lt;research&gt;&lt;optical-character-recognition&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1355" PostTypeId="2" ParentId="1348" CreationDate="2016-08-05T01:55:51.737" Score="12" Body="&lt;p&gt;Asimov's laws are not strong enough to be used in practice. Strength isn't even a consideration, when considering that since they're written in English words would first have to be interpreted subjectively to have any meaning at all. You can find a good discussion of this &lt;a href=&quot;https://youtu.be/7PKx3kS7f4A&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To transcribe an excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do you define these things? How do you define &quot;human&quot;, without first having to take a stand on almost every issue. And if &quot;human&quot; wasn't hard enough, you then have to define &quot;harm&quot;, and you've got the same problem again. Almost any really solid unambiguous definitions you give for those words&amp;mdash;that don't rely on human intuition&amp;mdash;result in weird quirks of philosophy, leading to your AI doing something you really don't want it to do.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One can easily imagine that Asimov was smart enough to know this and was more interested in story-writing than designing real-world AI control protocols.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the novel &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromancer#Plot_summary&quot;&gt;Neuromancer&lt;/a&gt;, it was suggested that AIs could possibly serve as checks against each other. Ray Kurzweil's impending &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot;&gt;Singularity&lt;/a&gt;, or the possibility of hyperintelligent AGIs otherwise, might not leave much of a possibility for humans to control AIs at all, leaving peer-regulation as the only feasible possibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that Eliezer Yudkowsky and others ran an &lt;a href=&quot;http://www.yudkowsky.net/singularity/aibox/&quot;&gt;experiment&lt;/a&gt; wherein Yudkowsky played the role of a superintelligent AI with the ability to speak, but no other connection outside of a locked box. The challengers were tasked simply with keeping the AI in the box at all costs. Yudkowsky escaped both times.&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="46" LastEditDate="2016-08-05T02:00:59.683" LastActivityDate="2016-08-05T02:00:59.683" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1356" PostTypeId="2" ParentId="198" CreationDate="2016-08-05T04:52:42.947" Score="2" Body="&lt;p&gt;There has been a recent work in the same domain where neural networks(CNNs to be accurate) are used for the same purpose. Some info. about the research is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To learn that context, the paper describes a method by which the&#xA;  neural network finds the user’s “embeddings” — i.e. contextual cues&#xA;  like the content of previous tweets, related interests and accounts,&#xA;  and so on. It uses these various factors to plot the user with others,&#xA;  and (ideally) finds that they form relatively well-defined groups.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the paper uses CNNs, word and user embeddings for detecting sarcasm in text. There is also a &lt;a href=&quot;https://techcrunch.com/2016/08/04/this-neural-network-tries-to-tell-if-youre-being-sarcastic-online/&quot; rel=&quot;nofollow&quot;&gt;Techcrunch article&lt;/a&gt; on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper uses sentiment of the tweet and compares with that of the other similar tweets:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the sentiment of the tweet seems to disagree with the bulk of what&#xA;  is expressed by similar users, there’s a good chance sarcasm is being&#xA;  employed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/1607.00976v2.pdf&quot; rel=&quot;nofollow&quot;&gt;Link to the paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2016-08-05T04:52:42.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1357" PostTypeId="1" AcceptedAnswerId="1359" CreationDate="2016-08-05T05:40:23.683" Score="9" ViewCount="249" Body="&lt;p&gt;Can an AI program have an EQ (Emotional intelligence or emotional quotient)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, can the EQ of an AI program be measured?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If EQ is more problematic to measure than IQ (at least with a standard applicaple to both humans and AI programs), why is that the case?&lt;/p&gt;&#xA;" OwnerUserId="1278" LastActivityDate="2016-08-05T12:43:38.537" Title="How can the EQ of an AI program be measured?" Tags="&lt;emotional-intelligence&gt;&lt;intelligence-testing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1358" PostTypeId="1" AcceptedAnswerId="1361" CreationDate="2016-08-05T07:03:50.110" Score="16" ViewCount="2439" Body="&lt;p&gt;I have heard about this concept in a Reddit post about Alpha Go. I have tried to go through the paper and the article, but could not really make sense of the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can someone give an easy-to-understand explanation of how the Monte-Carlo search algorithm work and how is it being used in building game-playing AI bots?&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="101" LastEditDate="2017-05-16T04:33:03.233" LastActivityDate="2018-04-24T04:13:03.027" Title="How does &quot;Monte-Carlo search&quot; work?" Tags="&lt;gaming&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1359" PostTypeId="2" ParentId="1357" CreationDate="2016-08-05T08:24:24.657" Score="6" Body="&lt;p&gt;The answer to your question is &quot;In principle, yes&quot; - in it's most general form, EQ testing is just a specific case of the Turing test (&quot;How would you feel about ... ?&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why meaningful EQ tests might be difficult to achieve, consider the following two possible tests:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At one extreme of complexity, the film 'Blade Runner' famously shows a test to distinguish between human and android on the basis of responses to emotionally-charged questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you tried asking these questions (or even much simpler ones) to a modern chatbot, you'd likely quickly conclude that you were not talking to a person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with assessing EQ is that the more emotionally sophisticated the test, the more general the AI system is likely have to be, in order to turn the input into a meaningful representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the other extreme from the above, suppose that an EQ test was phrased in an extremely structured way, with the structured input provided by a human. In such a case, success at an 'EQ test' is not really grounded in the real-world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an essay entitled &quot;The ineradicable Eliza effect and its dangers&quot;, Douglas Hofstadter gives the following example, in which the ACME program is claimed (not by Hofstadter) to 'understand' analogy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here the computer learns about a fellow named Sluggo taking his wife Jane and&#xA;  his good buddy Buck to a bar, where things take their natural course and Jane&#xA;  winds up pregnant by Buck. She has the baby but doesn't want it, and so, aided&#xA;  by her husband, she drowns the baby in a river, thus &quot;neatly solving &quot;the problem&quot;&#xA;  of Bambi.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This story is presented to ACME in the following form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ql: (neglectful-husband (Sluggo))&#xA;q2: (lonely-and-sex-starved-wife (Jane-Doe))&#xA;q3: (macho-ladykiller (Buck-Stag))&#xA;q4: (poor-innocent-little-fetus (Bambi))&#xA;q5: (takes-out-to-local-bar (Sluggo Jane-Doe Buck-Stag))&#xA;...&#xA;q11: (neatly-solves-the-problem-of (Jane-Doe Bambi))&#xA;q12: (cause (ql0 q11))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Suppose the program were to be asked if Jane Doe's behavior was moral. Complex compound emotional concepts such as 'neglectful', 'lonely' and 'innocent' are here simply predicates, not available to the AI for deeper introspective examination. They could just as easily be replaced by labels such as as 'bling-blang-blong15657'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in one sense, the absence of success at an EQ test with any depth is indicative of the general problem currently facing AI: the inability to define (or otherwise learn) meaningful representations of subtle complexities of the human world, which is a lot more complex than being able to recognize videos of cats.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-05T12:43:38.537" LastActivityDate="2016-08-05T12:43:38.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1360" PostTypeId="1" AcceptedAnswerId="1369" CreationDate="2016-08-05T08:51:50.657" Score="6" ViewCount="157" Body="&lt;p&gt;DNNs are typically used to classify things (of course) but can we let them go wild with sounds and then tell them if we think it sounds good or not? I'd like to think after a training class has been made (perhaps comparing the output to an existing song) we could get an NN that has a basic concept of music.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Timing would be an issue; I'm not sure how feasible this is. A strongly weighted input attached to all hidden layers perhaps? Use it as the bias?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this even slightly feasible? &lt;/p&gt;&#xA;" OwnerUserId="1284" LastEditorUserId="145" LastEditDate="2016-08-18T21:36:41.180" LastActivityDate="2016-08-18T21:36:41.180" Title="Has any research been done on DNN Music?" Tags="&lt;deep-neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1361" PostTypeId="2" ParentId="1358" CreationDate="2016-08-05T09:32:09.223" Score="14" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_method&quot;&gt;Monte Carlo method&lt;/a&gt; is an approach where you generate a large number of random values or simulations and form some sort of conlusions based on the general patterns, such as the means and variances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, you could use it for &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_weather_prediction&quot;&gt;weather forecasts&lt;/a&gt;. Predicting long-term weather is quite difficult, because it is a chaotic system where small changes can lead to very different results. Using Monte Carlo methods, you could run a large number of simulations, each with slightly different atmospheric changes. Then you can analyze the results and for example calculate the probability of rain on a given day based on how many simulations ended up with rain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the use of Monte Carlo in Alpha Go, they seem to be using the so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot;&gt;Monte Carlo Tree Search&lt;/a&gt;. In this approach, you make a tree of possible moves, a few turns into the future, and try to find the best sequence. However, since the number of possible moves in the game of go is very large, you won't be able to explore very far ahead. This means that some of the moves which look good now might turn out to be bad later. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in the Monte Carlo Tree Search, you pick a promising sequence of moves and run one or more simulations of how the game might proceed from that point. Then you can use the results of that simulation to get a better idea of how good that specific sequence of moves really is and you update the tree accordingly. Repeat as needed until you find a good move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want more information or to look at some illustrations, I found an interesting paper on the topic: C. Browne et al., A Survey of Monte Carlo Tree Search Methods (&lt;a href=&quot;http://repository.essex.ac.uk/4117/1/MCTS-Survey.pdf&quot;&gt;open repository&lt;/a&gt; / &lt;a href=&quot;http://dx.doi.org/10.1109/TCIAIG.2012.2186810&quot;&gt;permanent link (paywalled)&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2016-08-05T09:32:09.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1362" PostTypeId="1" AcceptedAnswerId="1365" CreationDate="2016-08-05T10:39:31.520" Score="6" ViewCount="3438" Body="&lt;p&gt;How do I avoid my gradient descent algorithm into falling into the &quot;local minima&quot; trap while backpropogating on my neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any methods which help me avoid it?&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="2444" LastEditDate="2019-04-27T00:50:35.780" LastActivityDate="2019-04-30T01:57:53.150" Title="How to avoid falling into the &quot;local minima&quot; trap?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1363" PostTypeId="1" CreationDate="2016-08-05T10:49:39.557" Score="9" ViewCount="533" Body="&lt;p&gt;A neural network is a directed weighted graph. These can be represented by a (sparse) matrix. Doing so can expose some elegant properties of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this technique beneficial for examining neural networks?&lt;/p&gt;&#xA;" OwnerUserId="1283" LastActivityDate="2017-02-16T19:38:44.487" Title="Is it beneficial to represent a neural net as a matrix?" Tags="&lt;neural-networks&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1364" PostTypeId="2" ParentId="1363" CreationDate="2016-08-05T10:58:53.260" Score="4" Body="&lt;p&gt;It depends on the type of neural networks you are dealing with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For medium sized neural nets, the matrix approach is a very good way to do quick computations and even backpropogation of errors. One can even exploit sparse matrixes for understanding the sparse architecture of some neural nets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, for very large neural nets, using matrix computations would be computationally very intensive. So, relevant methods like graph-based stores, etc are used for them depending on the purpose and the architecture.&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="101" LastEditDate="2017-02-16T19:38:44.487" LastActivityDate="2017-02-16T19:38:44.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1365" PostTypeId="2" ParentId="1362" CreationDate="2016-08-05T11:00:14.427" Score="8" Body="&lt;p&gt;There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Probabalistically accepting worse solutions in the hope that this&#xA;will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). &lt;/li&gt;&#xA;&lt;li&gt;Maintaining a list of recently-encountered states (or attributes thereof) and not returning&#xA;to a recently-encountered one (like Tabu Search). &lt;/li&gt;&#xA;&lt;li&gt;Performing a random walk of a length determined by the current state of the search (an explicit 'Diversification strategy', e.g. as used in 'Reactive Tabu Search').&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See the excellent (and free online) book &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics/&quot;&gt;'Essentials of Metaheuristics'&lt;/a&gt; by Sean Luke for more details on these kind of techniques and some rules of thumb about when and how to use them.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-05T11:00:14.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1366" PostTypeId="2" ParentId="1363" CreationDate="2016-08-05T11:15:40.017" Score="4" Body="&lt;p&gt;For large ANNs, something equivalent to a 'sparse matrix format' is used in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to what is said in another answer given, considering an ANN as a graph doesn't actually buy very much, for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The backpropagation algorithm can usefully be&#xA;defined in terms of matrix operations. &lt;a href=&quot;http://briandolhansky.com/blog/2014/10/30/artificial-neural-networks-matrix-form-part-5&quot; rel=&quot;nofollow&quot;&gt;This page&lt;/a&gt; gives a&#xA;readable and comprehensive description.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All real-valued matrices can be represented as graphs, but the converse is clearly not the case. So while it is true that an ANN can be considered as a special case of a graph data structure, making that specialization explicit in matrix form is more efficient.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-05T19:12:14.883" LastActivityDate="2016-08-05T19:12:14.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1369" PostTypeId="2" ParentId="1360" CreationDate="2016-08-05T12:05:13.230" Score="6" Body="&lt;p&gt;The first thing is to define what is a «good» and a «bad» sound. This is an extremely tricky issue, since the networks need &lt;em&gt;numeric&lt;/em&gt; inputs. And music is whole bunch of numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know from people doing research in identifying &lt;em&gt;how similar&lt;/em&gt; two sounds are, and imitation, say: you hear a sound and try to make another that sounds like it. Like when you hum a song or similar. That is by no means easy. These guys are using something similar to feature extraction, with Fourier transforms and energy and &lt;a href=&quot;https://en.wikipedia.org/wiki/Music_information_retrieval&quot; rel=&quot;noreferrer&quot;&gt;such things&lt;/a&gt;. They feed the networks with the (selected) features and... Train. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to return to your original question: *What do you present as &lt;em&gt;target&lt;/em&gt; during training?* You can present different &lt;em&gt;types&lt;/em&gt; of music as categories and classify (I couldn't help but think on &lt;a href=&quot;https://link.springer.com/article/10.3758/BF03192900&quot; rel=&quot;noreferrer&quot;&gt;this research with fish&lt;/a&gt;). Or &lt;strong&gt;you&lt;/strong&gt; define categories of music &lt;strong&gt;you&lt;/strong&gt; like and see if the network can classify them ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One basic decision here is how long you get a piece of sound. Since it is needed to analyse frequency, this is a key issue. Since you talked about DNN, I was wondering if you wanted to do it &lt;em&gt;online&lt;/em&gt;, as a stream, in which case I don't have the slightest idea where to begin, other than do it &lt;em&gt;after a little while&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Other idea: I remember a little sketch in &lt;a href=&quot;http://www.bbc.co.uk/programmes/b012xppj&quot; rel=&quot;noreferrer&quot;&gt;this series&lt;/a&gt; about a researcher that makes use of the relations between peaks in the Fourier spectrum in order to differentiate noise from music.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2016-08-05T12:05:13.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1370" PostTypeId="2" ParentId="1333" CreationDate="2016-08-05T13:55:54.380" Score="3" Body="&lt;p&gt;It seems easy for this to be sublinear growth or superlinear growth, depending on context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we imagine the space of the complex AI as split into two parts--the context model and the content model (that is, information and structure that is expected to be shared across entries vs. information and structure that is local to particular entries), then expanding the source material means we don't have much additional work to do on the context model, but whether the additional piece of the content model is larger or smaller depends on how connected the new material is to the old material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, one of the reasons why Watson takes many times the space of its source material is because it stores links between objects, which one would expect to grow with roughly order &lt;em&gt;n&lt;/em&gt; squared. If there are many links between the old and new material, then we should expect it to roughly quadruple in size instead of double; if the old material and new material are mostly unconnected and roughly the same in topology, then we expect the model to roughly double; if the new material is mostly unconnected to the old material and also mostly unconnected to itself, then we expect the model to not grow by much.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-05T13:55:54.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1371" PostTypeId="2" ParentId="179" CreationDate="2016-08-05T13:59:51.080" Score="2" Body="&lt;p&gt;If anything, multiple intelligences are much more obvious in AI than in other fields, because we haven't yet unlocked how to do transfer between domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, AlphaGo is very, very good at playing Go, but it's got basically nothing in the way of bodily-kinesthetic intelligence. But other teams have built software to control robots that does have bodily-kinesthetic intelligence, while not being good at the tasks that AlphaGo excels at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sort of modular intelligence is typically referred to as 'narrow AI,' whereas we use the term 'general AI' (or AGI, for Artificial General Intelligence) to refer to intelligence that we've built that can do roughly as many different kinds of things as people can do.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-05T13:59:51.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1372" PostTypeId="2" ParentId="145" CreationDate="2016-08-05T14:05:19.287" Score="8" Body="&lt;p&gt;&quot;Current artificial intelligence research&quot; is a pretty broad field. From where I sit, in a mostly CS realm, people are focused on narrow intelligence that can do economically relevant work on narrow tasks. (That is, predicting when components will fail, predicting which ads a user will click on, and so on.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those sorts of tools, the generality of a formalism like AIXI is a weakness instead of a strength. You don't need to take an AI that could in theory compute anything, and then slowly train it to focus on what you want, when you could just directly shape a tool that is the mirror of your task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not as familiar with AGI research itself, but my impression is that AIXI is, to some extent, the simplest idea that could work--it takes all the hard part and pushes it into computation, so it's 'just an engineering challenge.' (This is the bit about 'finding approximations to AIXI.') The question then becomes, is starting at AIXI and trying to approximate down a more or less fruitful research path than starting at something small and functional, and trying to build up?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My impression is the latter is much more common, but again, I only see a small corner of this space.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-05T14:05:19.287" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1373" PostTypeId="2" ParentId="145" CreationDate="2016-08-05T14:25:48.467" Score="4" Body="&lt;p&gt;AIXI is really a conceptual framework. All the hard work of actually compressing the environment still remains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further discuss the question raised in Matthew Graves answer: given our current limited level of ability to represent complex environments, it seems to me that it doesn't make a lot of practical difference whether you start with AIXI as defining the 'top' of the system and working down (e.g. via supposedly generalized compression methods) or start at the 'bottom' and try solve problems in a single domain via domain-specific methods that (you hope) can subsequently be abstracted to provide cross-domain compression.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-05T14:25:48.467" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1376" PostTypeId="1" CreationDate="2016-08-05T16:51:49.833" Score="5" ViewCount="107" Body="&lt;p&gt;Would it be ethical to implement AI for self-defence for public walking robots which are exposed to dangers such as violence and crime such as robbery (of parts), damage or abduction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be pros and cons of such AI behavior? Is it realistic, or it won't be taken into account for some obvious reasons?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like pushing back somebody when somebody start pushing it first (AI will say: he pushed me first), or running away on crowded street in case algorithm will detect risk of abduction.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-05T17:07:35.410" LastActivityDate="2016-08-05T17:08:27.273" Title="Is it ethical to implement self-defence for street walking AI robots?" Tags="&lt;ethics&gt;&lt;decision-theory&gt;&lt;robots&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1377" PostTypeId="2" ParentId="1376" CreationDate="2016-08-05T17:06:27.950" Score="2" Body="&lt;p&gt;The question mentions &quot;walking robot&quot;, but it may be illustrative to re-frame the discussion in terms of self-driving cars, because: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It gives a common point of reference, rather than everyone having their own separate vision of how vulnerable/powerful a kung-fu walking robot might be.&lt;/li&gt;&#xA;&lt;li&gt;We already know a lot about societal attitudes to car theft.&lt;/li&gt;&#xA;&lt;li&gt;Given that autonomous vehicles will soon be mainstream, the morality of the question is then more of a pressing issue.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, should a self-driving car run someone over (likely killing them) if they try to steal it? I'm hoping that few people would argue that it should.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should it attempt to do a lesser amount of damage (say, calculated to hopefully only break a leg)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, I'd argue not. The main reason for saying this is that our decision-making algorithms are simply not sufficiently context aware to be able to decide whether theft or harm is the intent. To concretely illustrate this: a recent fatality arose because a self-driving Tesla &lt;a href=&quot;http://www.livescience.com/55273-first-self-driving-car-fatality.html&quot; rel=&quot;nofollow&quot;&gt;was oblivious to context&lt;/a&gt; to the extent that it couldn't distinguish between a high-sided van and empty space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under those circumstances, it's probably best not to allow commercial autonomous systems to cause physical damage (even to inanimate objects). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Running away' (or rather, 'driving away', in the case of the car) is another matter: driving is what it's designed to do.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-05T17:06:27.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1378" PostTypeId="2" ParentId="1376" CreationDate="2016-08-05T17:08:27.273" Score="2" Body="&lt;p&gt;It depends on whether the loss of the robot would end up causing harm to humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the robot was supposed to be watching for a suspected terrorist attack to start taking place (so it could alert authorities or halt the attack), it would be very bad if somebody dismantled the robot or otherwise stopped it from carrying out its mission. In that case, the device would be certainly justified in stopping humans from injuring it in any meaningful way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robot carrying classified information should probably be similarly willing to protect itself, since the spread of such data could bring harm to a state or a lot of people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an AI-enabled device was just walking the streets in the course of carrying out some mundane task, I think it would be hard to justify allowing the robot to incapacitate a human attacker. After all, it was made - presumably - to serve humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter whether the AI was programmed to defend itself, people couldn't just impede or damage it with impunity. &lt;a href=&quot;https://malegislature.gov/laws/generallaws/partiv/titlei/chapter266/section127&quot; rel=&quot;nofollow&quot;&gt;Intentional destruction&lt;/a&gt; of another person's property (including public property) is almost certainly a crime, as is &lt;a href=&quot;http://law.justia.com/codes/georgia/2010/title-16/chapter-10/article-2/16-10-24&quot; rel=&quot;nofollow&quot;&gt;intentional obstruction of law enforcement&lt;/a&gt;. It wouldn't have to be up to each robot to defend itself; it could just send information about the perpetrator to C&amp;amp;C before its demise.&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-08-05T17:08:27.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1379" PostTypeId="1" AcceptedAnswerId="1380" CreationDate="2016-08-05T17:11:34.880" Score="0" ViewCount="61" Body="&lt;p&gt;Is there any risk in the near future of replacing all encyclopedias with Watson-like AI where knowledge is accessible by everybody through &lt;a href=&quot;https://watson-api-explorer.mybluemix.net/&quot; rel=&quot;nofollow&quot;&gt;API&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Something similar happened in the future in &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Time_Machine_(2002_film)&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;The Time Machine&lt;/strong&gt; movie from 2002&lt;/a&gt;.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously maintaining 40 million articles and keeping it up-to-date and consistent could be beyond brain power of few thousands of active editors. Not to mention thousands of other encyclopedias including paperback version or large number of books used by universities which needs to be updated every year by a huge number of people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of such a change?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-11T12:21:13.530" LastActivityDate="2016-08-11T12:21:13.530" Title="How likely is it that Watson-like AI will replace Wikipedia-like encyclopedias?" Tags="&lt;watson&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2016-08-12T15:19:24.397" ContentLicense="CC BY-SA 3.0" />
  <row Id="1380" PostTypeId="2" ParentId="1379" CreationDate="2016-08-05T17:30:07.273" Score="3" Body="&lt;p&gt;I get the impression that (perhaps even more than Bluemix) this is what the &lt;a href=&quot;https://www.wolfram.com/language/elementary-introduction/&quot; rel=&quot;nofollow&quot;&gt;Wolfram Language&lt;/a&gt; is looking to offer in the longer term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems to me that the main pros and cons are two sides of the same coin:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Wikipedia, there's no 'search filter' between you and the text. Adding an algorithmic level of indirection between the user and the knowledge that they're looking for is subject to hidden biases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If those biases are intended in your best interests, and the search is context-sensitive enough to present you with information in the form that is most useful and digestible to you, then this is a good thing. Otherwise, not. Like many topics in AI, problems arise because we're simply not that good at modelling human context yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, we're already subject to this &lt;a href=&quot;https://en.wikipedia.org/wiki/Filter_bubble&quot; rel=&quot;nofollow&quot;&gt;filter bubble&lt;/a&gt; effect via search engines and social media. The current consensus seems to be that even more of this would not be a good thing for society.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-05T17:37:44.953" LastActivityDate="2016-08-05T17:37:44.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1381" PostTypeId="1" AcceptedAnswerId="1382" CreationDate="2016-08-05T17:38:23.270" Score="3" ViewCount="83" Body="&lt;p&gt;I've watched the &lt;a href=&quot;https://www.youtube.com/watch?v=LY7x2Ihqjmc&quot; rel=&quot;nofollow&quot;&gt;Sunspring&lt;/a&gt; video which didn't make any sense to me (a lot of nonsense monologues), mainly because it was created by Jetson AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What was the mechanism of creating such screenplay?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On what criteria was it trained? What was the goal or motivation in terms of training criteria of defining when text does make sense? And what was missed (that it's so bad) and how possibly this could be improved?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="130" LastEditDate="2016-08-06T01:44:14.583" LastActivityDate="2016-08-06T01:44:14.583" Title="How does the Jetson AI write its screenplays?" Tags="&lt;algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1382" PostTypeId="2" ParentId="1381" CreationDate="2016-08-05T17:52:10.297" Score="6" Body="&lt;p&gt;It &lt;a href=&quot;http://benjamin.wtf/&quot; rel=&quot;noreferrer&quot;&gt;appears to use&lt;/a&gt; Recurrent NNs (RNNs) that have a 'Long Short-Term Memory' (LTSM) architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/artists-and-machine-intelligence/adventures-in-narrated-reality-6516ff395ba3#.5lvtgribl&quot; rel=&quot;noreferrer&quot;&gt;Here's a summary&lt;/a&gt; of the development process that the author, Ross Goodwin, went through to create it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me (and is also observed in the above link) that the output is rather poor - simply comparable to what one might expect from Markov chains, a technique that is over 100 years old.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't dug deeply into the technique, so I could be misktaken, but perhaps one of the reasons that it's so bad is that (as far as I can see), the model-building process is essentially &lt;em&gt;lexical&lt;/em&gt; - i.e. it is linking together tokens (words) without any more informed language model to guide it. In particular, the generated output doesn't seem to know anything about the functional roles played by objects (chairs are supporting objects, used by humans for sitting on etc), which is something that might be fairly readily incorporated.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-05T18:23:38.027" LastActivityDate="2016-08-05T18:23:38.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1384" PostTypeId="1" AcceptedAnswerId="1385" CreationDate="2016-08-05T18:22:41.003" Score="4" ViewCount="229" Body="&lt;p&gt;This &lt;a href=&quot;http://blog.claymcleod.io/2016/06/01/The-truth-about-Deep-Learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; suggests that deep learning is not designed to produce the universal algorithm and cannot be used to create such a complex systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all it requires huge amounts of computing power, time and effort to train the algorithm the right way and adding extra layers doesn't really help to solve complex problems which cannot be easily predicted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly some tasks are extremely difficult or impossible to solve using DNN, like solving a &lt;a href=&quot;https://ai.stackexchange.com/q/154/8&quot;&gt;math&lt;/a&gt; equations, predicting &lt;a href=&quot;https://ai.stackexchange.com/q/225/8&quot;&gt;pseudo-random lists&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/q/168/8&quot;&gt;fluid mechanics&lt;/a&gt;, guessing encryption algorithms, or &lt;a href=&quot;https://ai.stackexchange.com/q/205/8&quot;&gt;decompiling&lt;/a&gt; unknown formats, because there is no simple mapping between input and output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm asking, are there any alternative learning algorithms as powerful as deep architectures for general purpose problem solving? Which can solve more variety of problems, than &quot;deep&quot; architectures cannot?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2017-07-27T08:55:09.363" Title="Are there any learning algorithms as powerful as &quot;deep&quot; architectures?" Tags="&lt;deep-neural-networks&gt;&lt;comparison&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1385" PostTypeId="2" ParentId="1384" CreationDate="2016-08-05T18:41:59.757" Score="7" Body="&lt;p&gt;Have you read the book &lt;a href=&quot;http://libgen.io/ads.php?md5=F18395FBEF45575CF68BBD5AD26DF035&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Master Algorithm:&lt;/a&gt; by Pedro Domingos?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He discusses the present day machine learning algorithms... Their strengths, weaknesses and applications...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Deep Neural Network&lt;/li&gt;&#xA;&lt;li&gt;Genetic Algorithm&lt;/li&gt;&#xA;&lt;li&gt;Bayesian Network &lt;/li&gt;&#xA;&lt;li&gt;Support Vector Machine&lt;/li&gt;&#xA;&lt;li&gt;Inverse Deduction &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9HpIP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9HpIP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2017-07-27T08:55:09.363" LastActivityDate="2017-07-27T08:55:09.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1386" PostTypeId="2" ParentId="1384" CreationDate="2016-08-05T18:44:20.053" Score="4" Body="&lt;p&gt;Deep learning is actually pretty useful (relative to other techniques) &lt;em&gt;precisely when there is no simple mapping between input and output&lt;/em&gt;, and features from the raw input need to be aggregated and combined in complex ways by successive layers to form the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I pointed out in my answer to the &lt;a href=&quot;https://ai.stackexchange.com/questions/205/how-to-write-c-decompiler-using-ai&quot;&gt;AI SE decompilation question&lt;/a&gt;, there is recent DL research which takes a natural language description as input and &lt;a href=&quot;http://arxiv.org/pdf/1510.07211.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;generates program text as output&lt;/a&gt;. Despite working in this general research area, I was personally surprised by this - the problem is significantly harder than the 'AI math' link you provide above.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-05T18:44:20.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1387" PostTypeId="2" ParentId="6" CreationDate="2016-08-05T19:31:15.263" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;the human mind is a battleground of higher level goals and lower level goals &quot;&lt;br&gt;— Marvin Minsky paraphrasing Sigmund Freud&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I argue that in general human agents try to maximise a hierarchy of performance measures.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;performance measures of humans&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Survival of genetic data &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Energy supply and Water&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sex&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;myriad subgoals....&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Mysterious mental mechanisms which neuroscientists do not understand yet force the average human agent to maximise various evaluation metrics.&#xA;With the overarching goal of &lt;strong&gt;survival of genetic information&lt;/strong&gt;. Successful genes are immortal. We are still under the yoke of an ancient genetic algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These measures are optimised throughout a humans life time. A 30 year old agent is better at survival than a 10 year old agent. A 30 year old agent makes fewer mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We remember our mistakes. Mistakes are burned into our memory by high levels of neurotransmitters (and reinforcing of synapses) so we don't make them again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We attempt to optimise a swarm of subgoals that are all connected in one way or another to the main goal &lt;strong&gt;gene survival&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;status&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;money&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;education &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;happiness&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="157" LastEditorUserId="29" LastEditDate="2016-08-05T23:21:31.713" LastActivityDate="2016-08-05T23:21:31.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1389" PostTypeId="2" ParentId="60" CreationDate="2016-08-05T20:08:43.130" Score="-1" Body="&lt;p&gt;One obstacle to the development of AI is the fundamental limitations of computer memory. Computers, at a fundamental level, can only work with bits. This limits the type of information that they can describe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The precise nature and complexity of human memory isn't fully understood, but I would argue that at the very least, human memory is well adapted for the types of tasks that humans perform. Thus, computer memory, even if theoretically capable of representing everything that human memory can, is  probably inefficient and poorly structured for such a task.  &lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="127" LastEditDate="2016-08-08T17:36:35.903" LastActivityDate="2016-08-08T17:36:35.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1390" PostTypeId="1" AcceptedAnswerId="1412" CreationDate="2016-08-05T20:47:50.290" Score="7" ViewCount="934" Body="&lt;p&gt;Is there any research on the application of AI for drug design?&lt;/p&gt;&#xA;&lt;p&gt;For example, you could train a deep learning model about current compounds, substances, structures, and their products and chemical reactions from the existing &lt;a href=&quot;https://opendata.stackexchange.com/q/3553/3082&quot;&gt;dataset&lt;/a&gt; (basically what produces what). Then you give the task to find how to create a gold or silver from the group of available substances. Then the algorithm will find the chemical reactions (successfully predicting a new one that wasn't in the dataset) and gives the results. Maybe the gold is not a good example, but the practical scenario would be the creation/design of drugs that are cheaper to create by using much simpler processes or synthesizing some substances for the first time for drug industries.&lt;/p&gt;&#xA;&lt;p&gt;Was there any successful research attempting to achieve that using deep learning algorithms?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-11T20:11:46.910" LastActivityDate="2021-04-19T22:45:21.740" Title="Is there any research on the application of AI for drug design?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;applications&gt;&lt;prediction&gt;&lt;drug-design&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1391" PostTypeId="1" AcceptedAnswerId="1399" CreationDate="2016-08-05T21:29:37.880" Score="14" ViewCount="662" Body="&lt;p&gt;Assume that I want to solve an issue with a neural network that either I can't fit to already existing topologies (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm unable to understand their mechanics and I rely on my own instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I automate the choice of the topology (that is, the number of layers, the type of activations, the type and direction of the connections, etc.) of a neural network for an arbitrary problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a beginner, yet I realized that in some topologies (or, at least in perceptrons) it is very hard if not impossible to understand the inner mechanics as the neurons of the hidden layers don't express any mathematically meaningful context.&lt;/p&gt;&#xA;" OwnerUserId="1270" LastEditorUserId="2444" LastEditDate="2019-07-07T20:50:15.410" LastActivityDate="2019-07-07T21:19:41.473" Title="How can I automate the choice of the topology of a neural network for an arbitrary problem?" Tags="&lt;neural-networks&gt;&lt;topology&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1392" PostTypeId="1" AcceptedAnswerId="1450" CreationDate="2016-08-06T00:24:44.493" Score="1" ViewCount="84" Body="&lt;p&gt;For example there is &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database&quot; rel=&quot;nofollow&quot;&gt;the MNIST database&lt;/a&gt; which is used to test artificial neural network (ANN), however it's not so challenging, because some hierarchical systems of convolutional neural networks manages to get an error rate of 0.23 percent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any similar, especially the most challenging tasks with dataset which are used as benchmark tests to challenge the AI which are fairly reliable and it's possible to pass, but most AAN are struggling to achieve the lower error rate?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="1581" LastEditDate="2018-11-13T17:21:58.497" LastActivityDate="2018-11-13T17:21:58.497" Title="What are the most challenging tasks aiming to achieve the lowest error rate?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1393" PostTypeId="1" CreationDate="2016-08-06T00:37:24.067" Score="3" ViewCount="265" Body="&lt;p&gt;The paper &lt;a href=&quot;http://repository.supsi.ch/5145/1/IDSIA-04-12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-column Deep Neural Networks for Image Classification&lt;/a&gt; (pages 7-8) shows an attempt at recognizing the traffic signs, with lower error rates, by using multi-column deep neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are Google cars using similar techniques of predicting signs using DNN, or are they using some other method?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-20T14:58:10.773" LastActivityDate="2019-06-20T15:00:05.697" Title="How do Google cars recognize the traffic signs?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;classification&gt;&lt;autonomous-vehicles&gt;&lt;google&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1394" PostTypeId="1" AcceptedAnswerId="1408" CreationDate="2016-08-06T01:27:03.500" Score="4" ViewCount="155" Body="&lt;p&gt;I'd like to know whether there were attempts to simulate the whole brain, I'm not talking only about some &lt;a href=&quot;https://ai.stackexchange.com/q/237/8&quot;&gt;ANN on microchips&lt;/a&gt;, but brain simulations.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-01-26T01:56:55.193" LastActivityDate="2021-01-26T01:56:55.193" Title="Are there any artificial neuromorphic systems which can mimic the brain?" Tags="&lt;brain&gt;&lt;neuromorphic-engineering&gt;&lt;human-brain&gt;&lt;blue-brain-project&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1395" PostTypeId="2" ParentId="1394" CreationDate="2016-08-06T01:27:03.500" Score="3" Body="&lt;p&gt;Neuromorphic engineering offers various of ways of reproducing the brain’s processing ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recent technology can include IBM's multi-artificial-neuron computer, the world's first artificial nanoscale stochastic phase-change neurons&lt;sup&gt;&lt;a href=&quot;http://arstechnica.com/gadgets/2016/08/ibm-phase-change-neurons/?&quot; rel=&quot;nofollow&quot;&gt;article&lt;/a&gt;&lt;/sup&gt;. Check the: &lt;a href=&quot;http://www.nature.com/nnano/journal/v11/n8/full/nnano.2016.70.html&quot; rel=&quot;nofollow&quot;&gt;Stochastic phase-change neurons&lt;/a&gt; study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other can include&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://web.stanford.edu/group/brainsinsilicon/neurogrid.html&quot; rel=&quot;nofollow&quot;&gt;Neurogrid&lt;/a&gt;, built by Brains in Silicon at Stanford University is another example for brain simulation. It uses analog computation to emulate ion-channel activity. It emulates neurons using digital circuitry designed to maximize spiking throughput&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering#Examples&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/SpiNNaker&quot; rel=&quot;nofollow&quot;&gt;SpiNNaker&lt;/a&gt;, which is a manycore computer to simulate the human brain (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Human_Brain_Project&quot; rel=&quot;nofollow&quot;&gt;Human Brain Project&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/SyNAPSE&quot; rel=&quot;nofollow&quot;&gt;SyNAPSE&lt;/a&gt;, a DARPA neuromorphic machine technology, that scales to biological levels. Each chip can have over a million of electronic “neurons” and 256 million electronic synapses between neurons. In 2014 the 5.4 billion transistor chip had one of the highest transistor counts of any chip ever produced. The program is undertaken by HRL, HP and IBM.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-06T01:27:03.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1396" PostTypeId="1" AcceptedAnswerId="1406" CreationDate="2016-08-06T01:57:43.263" Score="20" ViewCount="5979" Body="&lt;p&gt;On &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia page&lt;/a&gt; about AI, we can read:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Optical character recognition is no longer perceived as an exemplar of &quot;artificial intelligence&quot; having become a routine technology.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt; database of handwritten digits is specially designed for training and testing neural networks and their error rates (see: &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database#Classifiers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classifiers&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, why does the above quote state that OCR is no longer exemple of AI?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-03-08T05:55:09.787" LastActivityDate="2020-03-08T05:55:09.787" Title="Why can't OCR be perceived as a good example of AI?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;optical-character-recognition&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="1397" PostTypeId="1" AcceptedAnswerId="1452" CreationDate="2016-08-06T02:04:19.343" Score="11" ViewCount="447" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_intelligent_signal_test&quot; rel=&quot;nofollow&quot;&gt;MIST&lt;/a&gt; is a quantiative test of humanness, consisting of ~80k propositions such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is Earth a planet?&lt;/li&gt;&#xA;&lt;li&gt;Is the sun bigger than my foot?&lt;/li&gt;&#xA;&lt;li&gt;Do people sometimes lie?&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Have any AI attempted and passed this test to date?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-15T14:44:41.513" LastActivityDate="2016-09-06T15:23:41.650" Title="Are there any AI that have passed the MIST test so far?" Tags="&lt;history&gt;&lt;intelligence-testing&gt;&lt;turing-test&gt;&lt;chat-bots&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1398" PostTypeId="2" ParentId="1396" CreationDate="2016-08-06T05:02:50.820" Score="4" Body="&lt;p&gt;I'm not sure if predicting MNIST can be really considered as an AI task. AI problems can be usually framed under the context of an agent acting in an environment. Neural nets and machine learning techniques in general do not have to deal with this framing. Classifiers for example, are learning a mapping between two spaces. Though one could argue that you &lt;em&gt;can&lt;/em&gt; frame OCR/image classification as an AI problem - the classifier is the agent, each prediction it makes is an action, and it receives rewards based on its classification accuracy - this is rather unnatural and different from problems that are commonly considered AI problems.&lt;/p&gt;&#xA;" OwnerUserId="1305" LastActivityDate="2016-08-06T05:02:50.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1399" PostTypeId="2" ParentId="1391" CreationDate="2016-08-06T05:40:41.717" Score="11" Body="&lt;p&gt;I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT (NeuroEvolution of Augmenting Topologies)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix activation functions if you aren't sure which to use. &lt;a href=&quot;http://blog.otoro.net/2016/05/07/backprop-neat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is an example of using backpropagation and multiple neuron types.&lt;/p&gt;&#xA;" OwnerUserId="1306" LastEditorUserId="2444" LastEditDate="2019-07-07T21:18:25.820" LastActivityDate="2019-07-07T21:18:25.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1400" PostTypeId="2" ParentId="1391" CreationDate="2016-08-06T06:33:58.880" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/1399/2444&quot;&gt;The other answer&lt;/a&gt; mentions &lt;a href=&quot;https://www.cs.ucf.edu/~kstanley/neat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt; to generate network weights or topologies. The paper &lt;a href=&quot;http://doc.gold.ac.uk/aisb50/AISB50-S11/AISB50-S11-Turner-paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks&lt;/a&gt;, which also gives a short summary of neuroevolution techniques, provides an alternative approach to NEAT. It uses &lt;a href=&quot;http://www.cartesiangp.co.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cartesian Genetic Programming&lt;/a&gt; to evolve a multiple activation functions.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-07-07T21:19:41.473" LastActivityDate="2019-07-07T21:19:41.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1401" PostTypeId="1" AcceptedAnswerId="1403" CreationDate="2016-08-06T07:08:20.203" Score="2" ViewCount="64" Body="&lt;p&gt;It is possible of normal code to prove that it is correct using mathematical techniques, and that is often done to ensure that some parts are bug-free. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we also prove that a piece of code in AI software will cause it to never turn against us, i.e. that the AI is &lt;a href=&quot;https://en.wikipedia.org/wiki/Friendly_artificial_intelligence&quot; rel=&quot;nofollow&quot;&gt;friendly&lt;/a&gt;? Has there any research been done towards this?&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2016-08-06T07:51:00.567" Title="Can we prove that a piece of code in AI software will cause it to never turn against us?" Tags="&lt;friendly-ai&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1402" PostTypeId="2" ParentId="1396" CreationDate="2016-08-06T07:11:39.830" Score="14" Body="&lt;p&gt;Although OCR is now a mainstream technology, it remains true that none our methods genuinely have the recognition facilities of a 5 year old (claimed success with CAPTCHAs notwithstanding). We don't know how to achieve this using well-understood techniques, so OCR should still rightfully be considered an AI problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why this might be so, it is illuminating to read the essay&#xA;&lt;a href=&quot;https://web.stanford.edu/group/SHR/4-2/text/hofstadter.html&quot;&gt;&quot;On seeing A's and seeing AS&quot;&lt;/a&gt; by Douglas Hofstadter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to a point made in another answer, the agent framing is a useful one insofar as it motivates success in increasingly complex environments. However, there are many hard problems (e.g. Bongard) that don't need to be stated in such a fashion. &lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-06T07:11:39.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1403" PostTypeId="2" ParentId="1401" CreationDate="2016-08-06T07:24:41.043" Score="5" Body="&lt;p&gt;Unfortunately, this is extremely unlikely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is nearly impossible to make statements about the behaviour of software in general. This is due to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot;&gt;Halting problem&lt;/a&gt;, which shows that it is impossible to prove whether a program will stop for any given input. From this result, many other things have been shown to be unprovable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question whether a piece of code is friendly, can very likely be reduced to a variant of the halting problem.&lt;br&gt;&#xA;An AI that operates in the real world, which is a requirement for &quot;friendliness&quot; to have a meaning, would need to be Turing complete. Input from the real world cannot be reliably interpreted using regular or context-free languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proofs of correctness work for small code snippets, with clearly defined inputs and outputs. They show that an algorithm produces the mathematically right output, given the right input.&lt;br&gt;&#xA;But these are about situations that can be defined with mathematical rigour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Friendliness&quot; isn't a rigidly defined concept, which already makes it difficult to prove anything about it. On top of that, &quot;friendliness&quot; is about how the AI relates to the real world, which is an environment whose input to the AI is highly unpredictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best we can hope for, is that an AI can be programmed to have safeguards, and that the code will raise warning flags if unethical behaviour becomes likely - that AI's are programmed defensively.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-06T07:51:00.567" LastActivityDate="2016-08-06T07:51:00.567" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1404" PostTypeId="1" AcceptedAnswerId="1407" CreationDate="2016-08-06T07:27:52.287" Score="4" ViewCount="865" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://arxiv.org/pdf/1606.00652.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Death and Suicide in Universal Artificial Intelligence&lt;/a&gt;, a proposal is given for what death could mean for Artificial Intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does this mean using English only? I understand that mathematical notation is useful for giving a precise definition, but I'd like to understand what the definition really means. &lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2019-04-19T16:44:43.073" LastActivityDate="2019-04-19T16:44:43.073" Title="What does &quot;death&quot; intuitively mean in the paper &quot;Death and Suicide in Universal Artificial Intelligence&quot;?" Tags="&lt;research&gt;&lt;definitions&gt;&lt;agi&gt;&lt;death&gt;&lt;aixi&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1405" PostTypeId="2" ParentId="1401" CreationDate="2016-08-06T07:40:02.787" Score="3" Body="&lt;p&gt;Here are some examples of recent work on verifying certain properties of autonomous systems &lt;a href=&quot;https://www.cs.york.ac.uk/circus/RoboCalc-event/courses/&quot; rel=&quot;nofollow&quot;&gt;[RoboCheck]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, to achieve the same kind of thing for the notion of 'friendly' using formal verification (i.e. 'proving correctness using mathematical techniques'),&#xA;it would (at the least) seem necessary to be able to express 'friendly' within a logical formalism, (i.e. as a predicate testable within a model-checker, so that we can be sure a system never enters an undesirable state).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's not immediately clear that 'friendly' has a more specific definition than 'a desire not to harm humans', so much more low-level detail is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some previous work in this general area that might be useful in this respect include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deontic_logic&quot; rel=&quot;nofollow&quot;&gt;Deontic Logic&lt;/a&gt; - a logical calculus of obligations.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.jfsowa.com/ikl/McCarthy89&quot; rel=&quot;nofollow&quot;&gt;Elephant 2000&lt;/a&gt; - John McCarthy's description of a promise-based programming language.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-06T07:40:02.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1406" PostTypeId="2" ParentId="1396" CreationDate="2016-08-06T07:44:40.520" Score="20" Body="&lt;p&gt;Whenever a problem becomes solvable by a computer, people start arguing that it does not require intelligence. John McCarthy is often quoted: &quot;As soon as it works, no one calls it AI anymore&quot; (&lt;a href=&quot;http://cacm.acm.org/magazines/2012/1/144824-artificial-intelligence-past-and-future/fulltext&quot; rel=&quot;noreferrer&quot;&gt;Referenced in CACM&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my teachers in college said that in the 1950's, a professor was asked what he thought was intelligent for a machine. The professor reputedly answered that if a vending machine gave him the right change, that would be intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later, playing chess was considered intelligent. However, computers can now defeat grandmasters at chess, and people are no longer saying that it is a form of intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have OCR. It's already stated in &lt;a href=&quot;https://ai.stackexchange.com/a/1402/66&quot;&gt;another answer&lt;/a&gt; that our methods do not have the recognition facilities of a 5 year old. As soon as this is achieved, people will say &quot;meh, that's not intelligence, a 5 year old can do that!&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A psychological bias, a need to state that we are somehow superior to machines, is at the basis of this.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-08T03:21:22.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1407" PostTypeId="2" ParentId="1404" CreationDate="2016-08-06T08:07:18.500" Score="7" Body="&lt;p&gt;The authors do actually give an English definition in terms of the well-known agent formulation of AI:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We intend this usage to be intuitive: death means that one sees&#xA;  no more percepts, and takes no more actions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It would seem that this becomes possible for a reinforcement learning agent such as AIXI in a formulation that uses &lt;em&gt;semi-measures&lt;/em&gt; of probability (which need not sum up to 1), rather than the more traditional notion.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-06T08:07:18.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1408" PostTypeId="2" ParentId="1394" CreationDate="2016-08-06T08:59:35.103" Score="6" Body="&lt;p&gt;Vernor Vinge said that if we can scan a human brain and then simulate it: We can run it at 1000 times the speed. The brain will be able to do 1000 years of thinking in 1 year ect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this stage in history we have the computer power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trouble lies in cutting a brain up and scanning the 100 billion neurons and 12 million kilometres of axons and 100000 billion synapses.&#xA;And piecing together the connectome from all the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sebastian Seung at MIT is working on automating this scanning process with machine learning. By gathering training data from thousands of people playing his &lt;a href=&quot;https://en.wikipedia.org/wiki/Eyewire&quot; rel=&quot;noreferrer&quot;&gt;Eyewire game&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Henry Markram in Europe tried to do something similar with his &lt;a href=&quot;https://en.wikipedia.org/wiki/Blue_Brain_Project&quot; rel=&quot;noreferrer&quot;&gt;Blue Brain Project&lt;/a&gt;.&#xA;He attempted to simulate the neocortical column of a rat. The EU gave him a billion euros to do this. Unfortunately he has been heavily criticised by the Neuroscience community. They claim that we don't know the physiology  well enough to make a valid simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out his &lt;a href=&quot;https://www.youtube.com/watch?v=LS3wMC2BpxU&quot; rel=&quot;noreferrer&quot;&gt;Ted Talk&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 1970s Sydney Brenner achieved a &lt;em&gt;full brain scan&lt;/em&gt; of a C Elegans worm. This worm has one of the simplest biological neural networks having only 302 neurons.&#xA;Here is a picture of its connectome:&lt;a href=&quot;https://i.stack.imgur.com/cAZ49.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cAZ49.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/siHt8.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/siHt8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An accurate computer simulation of this worm would be a major stepping stone to uploading a human brain.&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2016-08-06T09:37:48.850" LastActivityDate="2016-08-06T09:37:48.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1409" PostTypeId="2" ParentId="1363" CreationDate="2016-08-06T09:15:09.163" Score="1" Body="&lt;p&gt;Matrix representation is beneficial for implementing neural networks in silicon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for examining neural networks empirically it is sometimes good to visualise the synapse weight values as images or videos: &lt;a href=&quot;https://www.youtube.com/watch?v=AgkfIQ4IGaM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jason Yosinski's &lt;/a&gt; exploration of a convolution neural network. The network seems to have a &quot;filter&quot; that just detects shoulders. A bit like a lock that only opens when it recognises the pattern of shoulders.&lt;a href=&quot;https://i.stack.imgur.com/4g4gF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4g4gF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2016-08-06T10:12:34.840" LastActivityDate="2016-08-06T10:12:34.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1410" PostTypeId="1" CreationDate="2016-08-06T10:46:10.100" Score="5" ViewCount="81" Body="&lt;p&gt;We can measure the power of the machine with the number of operation per second or the frequency of the processor. But does units similar of IQ for humans exist for a AI?&lt;br/&gt;&#xA;I'm asking for a unit which can give countable result so something different from a Turing Test which only give a binary result.&lt;/p&gt;&#xA;" OwnerUserId="98" LastEditorUserId="10" LastEditDate="2016-08-06T14:14:05.420" LastActivityDate="2016-08-08T18:25:11.147" Title="Do specific units exists for measuring the intelligence of a machine?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;intelligence-testing&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1411" PostTypeId="2" ParentId="1410" CreationDate="2016-08-06T11:57:19.053" Score="3" Body="&lt;p&gt;One of the challenges of AI is defining Intelligence.&#xA;If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined that it can be run on a computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Narrow AI can be evaluated on its success at achieving goals in an environment. In domains such as computer vision and speech recognition narrow AI algorithms can be easily evaluated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many universities curate narrow AI tests. Fei-Fei Li a professor at Stanford who directs the Artificial Intelligence lab there organises the annual ImageNet Challenge. In 2012 Geoffrey Hinton famously won the competition by building a Deep Neural Network that could recognize pictures more accurately than humans can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge the testers commonly use &lt;a href=&quot;https://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow&quot;&gt;Precision and recall&lt;/a&gt; evaluation metrics&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2016-08-06T14:07:22.407" LastActivityDate="2016-08-06T14:07:22.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1412" PostTypeId="2" ParentId="1390" CreationDate="2016-08-06T14:04:28.247" Score="8" Body="&lt;p&gt;Yes, many people have worked on this sort of thing, due to its obvious industrial applications (most of the ones I'm familiar with are in the pharmaceutical industry). Here's &lt;a href=&quot;https://arxiv.org/abs/1305.7074&quot;&gt;a paper from 2013&lt;/a&gt; that claims good results; following the trail of &lt;a href=&quot;https://scholar.google.com/scholar?cites=10630711614897084406&amp;amp;as_sdt=5,44&amp;amp;sciodt=0,44&amp;amp;hl=en&quot;&gt;papers that cited it&lt;/a&gt; will likely give you more recent work. &lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-06T14:04:28.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1413" PostTypeId="2" ParentId="1410" CreationDate="2016-08-06T14:17:28.310" Score="3" Body="&lt;p&gt;Shane Legg and Marcus Hutter &lt;a href=&quot;http://www.vetta.org/documents/42.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;proposed one&lt;/a&gt; in 2006. The main descriptive quotes (see the paper for the actual formula):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Intelligence measures an agent’s general ability to achieve goals in a wide range of environments&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;It is clear by construction that universal intelligence measures the general ability of an agent to perform well in a very wide range of environments, as required by our informal definition of intelligence given earlier. The definition places no restrictions on the internal workings of the agent; it only requires that the agent is capable of generating output and receiving input which includes a reward signal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-08-06T14:17:28.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1414" PostTypeId="2" ParentId="41" CreationDate="2016-08-06T14:22:56.640" Score="4" Body="&lt;p&gt;The other answers are correct that machine IQ test results are currently &lt;strong&gt;not&lt;/strong&gt; indicative of machine intelligence. One of the surprising facts of human intelligence is that performance on almost all cognitive tasks are correlated with each other; that is, there is such a thing as 'general smartness' and IQ tests attempt to measure that thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People &lt;em&gt;have&lt;/em&gt; built programs that take IQ tests, however, and some of them perform quite well. Raven's Progressive Matrices, a visual pattern recognition IQ test, is an easy target for AI (see &lt;a href=&quot;https://www.researchgate.net/publication/288211280_Solving_Raven&amp;#39;s_IQ-tests_An_AI_and_cognitive_modeling_approach&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; as representative) and another group &lt;a href=&quot;http://arxiv.org/abs/1509.03390&quot; rel=&quot;nofollow&quot;&gt;has constructed an AI&lt;/a&gt; that performs about as well as a 4 year old on the verbal intelligence portion of a standard childhood IQ test.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-06T14:22:56.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1415" PostTypeId="1" AcceptedAnswerId="1418" CreationDate="2016-08-06T17:24:50.083" Score="10" ViewCount="412" Body="&lt;p&gt;In the mid 1980s, Rodney Brooks famously created the foundations of &quot;the new AI&quot;. The central claim was that the symbolist approach of 'Good Old Fashioned AI' (GOFAI) had failed by attempting to 'cream cognition off the top', and that &lt;em&gt;embodied cognition&lt;/em&gt; was required, i.e. built from the bottom up in a 'hierarchy of competances' (e.g. basic locomotion -&gt; wandering around -&gt; actively foraging) etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine most AI researchers would agree that the 'embodied cognition' perspective has now (at least tacitly) supplanted GOFAI as the mainstream.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question takes the form of a thought experiment and asks: &quot;Which (if any)  aspects of 'embodied' can be relaxed/omitted before we lose something essential for AGI?&quot;&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-06T18:19:44.323" Title="What kind of body (if any) does intelligence require?" Tags="&lt;agi&gt;&lt;symbolic-ai&gt;&lt;embodied-cognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1416" PostTypeId="1" CreationDate="2016-08-06T17:35:02.143" Score="5" ViewCount="1174" Body="&lt;p&gt;In other words, which existing reinforcement method learns in fewest episodes? &lt;a href=&quot;http://www.jmlr.org/papers/volume3/brafman02a/brafman02a.pdf&quot; rel=&quot;noreferrer&quot;&gt;R-Max&lt;/a&gt; comes to mind, but its very old and I'd like to know if there is something better now.&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2017-09-22T16:51:22.563" Title="What is the current state-of-the-art in Reinforcement Learning regarding data efficiency?" Tags="&lt;algorithm&gt;&lt;research&gt;&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1417" PostTypeId="2" ParentId="92" CreationDate="2016-08-06T17:41:35.647" Score="28" Body="&lt;p&gt;The images that you provided may be unrecognizable for us. They are actually the images that we recognize but evolved using the &lt;a href=&quot;https://github.com/sferes2/sferes2&quot; rel=&quot;noreferrer&quot;&gt;Sferes&lt;/a&gt; evolutionary framework.&lt;/p&gt;&#xA;&lt;p&gt;While these images are almost impossible for humans to label with anything but abstract arts, the Deep Neural Network will label them to be familiar objects with 99.99% confidence.&lt;/p&gt;&#xA;&lt;p&gt;This result highlights differences between how DNNs and humans recognize objects. Images are either directly (or indirectly) encoded&lt;/p&gt;&#xA;&lt;p&gt;According to this &lt;a href=&quot;https://youtu.be/M2IebCN9Ht4&quot; rel=&quot;noreferrer&quot;&gt;video&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Changing an image originally correctly classified in a way imperceptible to humans can cause the cause DNN to classify it as something else.&lt;/p&gt;&#xA;&lt;p&gt;In the image below the number at the bottom are the images are supposed to look like the digits&#xA;But the network believes the images at the top (the one like white noise) are real digits with 99.99% certainty.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jx1wX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The main reason why these are easily fooled is that Deep Neural Network does not see the world in the same way as human vision. We use the whole image to identify things while DNN depends on the features. As long as DNN detects certain features, it will classify the image as a familiar object it has been trained on.&#xA;The researchers proposed one way to prevent such fooling by adding the fooling images to the dataset in a new class and training DNN on the enlarged dataset. In the experiment, the confidence score decreases significantly for ImageNet AlexNet. It is not easy to fool the retrained DNN this time. But when the researchers applied such method to MNIST LeNet, evolution still produces many unrecognizable images with confidence scores of 99.99%.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;More details &lt;a href=&quot;http://www.evolvingai.org/fooling&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://www.kdnuggets.com/2015/01/deep-learning-can-be-easily-fooled.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="72" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-04-13T03:22:19.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1418" PostTypeId="2" ParentId="1415" CreationDate="2016-08-06T18:19:44.323" Score="5" Body="&lt;p&gt;This is something of an orthogonal answer, but I think Brooks didn't go about his idea the right way. That is, &lt;a href=&quot;https://en.wikipedia.org/wiki/Subsumption_architecture&quot; rel=&quot;noreferrer&quot;&gt;subsumption architecture&lt;/a&gt; is one in which the 'autopilot' is &lt;em&gt;replaced&lt;/em&gt; by a more sophisticated system when necessary. (All pieces receive the raw sensory inputs, and output actions, some of which turn off or on other systems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But a better approach is the normal hierarchical control approach, in which the target of a lower level system is the output of a higher level system. That is, the targeted joint angle of a robot leg is determined by the system that is trying to optimize the velocity, which is determined by a system that is trying to optimize the trajectory, which is determined by a system that is trying to optimize the target position, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows for increasing level of complexity while maintaining detail and system reusability.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;That said, I don't think you actually need what one would naively call 'embodied cognition' in order to get the bottom-up hierarchy of competencies that Brooks is right to point towards. The core feature is the wide array of inputs and outputs, which are understood in a hierarchical fashion that allows systems to be chained together vertically. I think you could get a functional general intelligence whose only inputs and outputs involve going through an Ethernet cable, and doesn't have anything like a traditional body that it actuates or senses through. (This is a claim that the hierarchical structure is what matters, not the content of what we use that structure for.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The main place to look for more, I think, is actually a book about &lt;em&gt;human&lt;/em&gt; cognition, called The Control of Perception by William T. Powers.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-06T18:19:44.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1419" PostTypeId="2" ParentId="104" CreationDate="2016-08-06T18:38:06.953" Score="5" Body="&lt;p&gt;I can offer two (at first sight, conflicting) perspectives on this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If the letter string 'abc' becomes 'abd' what would &quot;doing the same thing&quot; to 'ijk' look like?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just one example of a problem (so-called 'letter-string analogy problems') that is not easily framed as an optimization problem - there is a range of answers that appear compelling to humans, each for its own structurally-specific reason. Some of the subtleties of these kinds of problems are discussed in detail &lt;a href=&quot;http://cognitrn.psych.indiana.edu/rgoldsto/courses/concepts/copycat.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a &lt;em&gt;very&lt;/em&gt; high-level perspective on AGI in which &lt;a href=&quot;http://arxiv.org/abs/cs/0309048&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimization plays a key part&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not at all clear how these two very different scales of approach might be reconciled. As someone who does optimization research for a living, I'd be inclined to say that, certainly for all &lt;em&gt;current, practical&lt;/em&gt; purposes, AGI can't really be treated as an optimization problem, since most interesting activities don't readily lend themselves to description via a cost function.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="14723" LastEditDate="2018-04-13T18:00:00.527" LastActivityDate="2018-04-13T18:00:00.527" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1420" PostTypeId="1" AcceptedAnswerId="1421" CreationDate="2016-08-06T18:38:50.160" Score="13" ViewCount="1611" Body="&lt;p&gt;Are there any research teams that attempted to create or have already created an AI robot that can be as close to intelligent as these found in &lt;a href=&quot;https://en.wikipedia.org/wiki/Ex_Machina_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Ex Machina&lt;/a&gt;&lt;/em&gt; or &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/I,_Robot_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;I, Robot&lt;/em&gt;&lt;/a&gt; movies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not talking about full awareness, but an artificial being that can make its own decisions and physical and intellectual tasks that a human being can do?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-11-22T23:33:06.783" LastActivityDate="2019-11-22T23:33:06.783" Title="How close are we to creating Ex Machina?" Tags="&lt;research&gt;&lt;agi&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="1421" PostTypeId="2" ParentId="1420" CreationDate="2016-08-06T18:54:48.577" Score="22" Body="&lt;p&gt;We are absolutely nowhere near, nor do we have any idea how to bridge the gap between what we can currently do and what is depicted in these films.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current trend for DL approaches (coupled with the emergence of data science as a mainstream discipline) has led to a lot of popular interest in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, researchers and practitioners would do well to learn the lessons of the 'AI Winter' and not engage in hubris or read too much into current successes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Success in transfer learning is very limited. &lt;/li&gt;&#xA;&lt;li&gt;The 'hard problem' (i.e. presenting the 'raw, unwashed environment' to the machine and having it come up with a solution from scratch) is not being&#xA;addressed by DL to the extent that it is popularly portrayed: expert human knowledge is still required to help decide how the input should be framed, tune parameters, interpret output etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Someone who has enthusiasm for AGI would hopefully agree that the 'hard problem' is actually the only one that matters. Some years ago, a famous cognitive scientist said &quot;We have yet to successfully represent &lt;em&gt;even a single concept&lt;/em&gt; on a computer&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, recent research trends have done little to change this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this perhaps sounds pessimistic - it's not intended to. None of us want another AI Winter, so we should challenge (and be honest about) the limits of our current techniques rather than mythologizing them.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-06T19:04:52.397" LastActivityDate="2016-08-06T19:04:52.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1422" PostTypeId="2" ParentId="1420" CreationDate="2016-08-06T20:30:10.680" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;heavier-than-air flying machines are impossible&quot; &lt;em&gt;_ Lord Kelvin 1895&lt;/em&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;7 years later the Wright brothers built one.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Currently we have many powerful narrow AI (good at special tasks) but we have no idea how to unify them into a single system like in a biological brain. &lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2016-08-07T07:02:48.480" LastActivityDate="2016-08-07T07:02:48.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1423" PostTypeId="1" AcceptedAnswerId="1440" CreationDate="2016-08-06T22:59:43.413" Score="12" ViewCount="187" Body="&lt;p&gt;Humans can do multiple tasks at the same (e.g. reading while listening to music), but we memorize information from less focused sources with worse efficiency than we do from our main focus or task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do such things exist in the case of artificial intelligence? I doubt, for example, that neural networks have such characteristics, but I may be wrong.&lt;/p&gt;&#xA;" OwnerUserId="1270" LastEditorUserId="2444" LastEditDate="2019-11-05T16:14:30.497" LastActivityDate="2019-11-05T16:14:30.497" Title="Is there any artificial intelligence that possesses &quot;concentration&quot;?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;attention&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1424" PostTypeId="2" ParentId="1390" CreationDate="2016-08-06T23:23:52.457" Score="3" Body="&lt;p&gt;Yes, there were successful attempts at predicting the interaction between molecules and biological proteins which have been used to identify potential treatments by using &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network#Drug_discovery&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolutional neural networks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example in 2015, the first deep learning neural network has been created for structure-based drug design which trains 3-dimensional representation of chemical interactions which works similar to how image recognition works (composing smaller features into larger, complex structures).&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network#Drug_discovery&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Study: &lt;a href=&quot;https://arxiv.org/abs/1510.02855&quot; rel=&quot;nofollow noreferrer&quot;&gt;AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Another approach is to use evolutionary artificial neural networks which can achieve great optimization results.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, the &lt;a href=&quot;https://arxiv.org/pdf/1502.00193.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper from 2015&lt;/a&gt; demonstrated heuristic &lt;a href=&quot;http://link.springer.com/article/10.1007/s12293-012-0075-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;chemical reaction optimization&lt;/a&gt; (CRO) which is inspired by the nature of chemical reactions (e.g. transforming the unstable substances into stable ones). Simulation results show that CRO outperforms many evolutionary algorithms by population-based metaheuristics mimicking the transition of molecules and their interactions in a chemical reaction.&lt;/p&gt;&#xA;&lt;p&gt;Sample pseudocode algorithm for predicting synthesis given ω1, ω2 (from the above paper):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 1: for all Matrices and vectors m in ω′ do&#xA; 2:     for all Elements e in m do&#xA; 3:         Generate a real r between 0 and 1&#xA; 4:         if r &amp;gt; 0.5 then&#xA; 5:             e =counterpart in m1&#xA; 6:         else&#xA; 7:             e =counterpart in m2&#xA; 8:         end if&#xA; 9:     end for&#xA;10: end for&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which is used to generate a new solution ω′ based on two given solutions ω1 and ω2.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="36737" LastEditDate="2021-04-19T22:21:33.800" LastActivityDate="2021-04-19T22:21:33.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1426" PostTypeId="1" AcceptedAnswerId="1443" CreationDate="2016-08-07T00:06:45.913" Score="5" ViewCount="110" Body="&lt;p&gt;How can a swarm of small robots (like Kilobots) walking close to each other achieve collaboration without bumping into each other? For example, one study shows &lt;a href=&quot;http://science.sciencemag.org/content/345/6198/795.abstract&quot; rel=&quot;nofollow&quot;&gt;programmable self-assembly in a thousand-robot swarm&lt;/a&gt; (see &lt;a href=&quot;http://robohub.org/thousand-robot-swarm-self-assembles-into-arbitrary-shapes/&quot; rel=&quot;nofollow&quot;&gt;article&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://vimeo.com/103329200&quot; rel=&quot;nofollow&quot;&gt;video&lt;/a&gt;) which are moving without GPS-like system and by measuring distances to neighbours. This was achieved, because the robots were very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way that similar robots can achieve much more efficient and quicker assembly by using more complex techniques of coordination? Not by walking around clock-wise (which I guess was the easiest way), but I mean using some more sophisticated way. Because waiting half a day (~11h) to create a simple star shape using a thousand-robot swarm is way too long!&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-07T19:10:08.217" LastActivityDate="2016-08-07T19:10:08.217" Title="How can thousand-robot swarm coordinate their moves without bumping into each other?" Tags="&lt;robots&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1427" PostTypeId="1" AcceptedAnswerId="1428" CreationDate="2016-08-07T00:55:36.657" Score="1" ViewCount="54" Body="&lt;p&gt;On Watson wiki page we can read:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In healthcare, Watson's natural language, hypothesis generation, and evidence-based learning capabilities allow it to function as a clinical decision support system for use by medical professionals.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How exactly such AI can help doctors to diagnose the diseases?&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2021-03-31T22:27:21.613" Title="How Watson can help to make medical diagnoses?" Tags="&lt;watson&gt;&lt;healthcare&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1428" PostTypeId="2" ParentId="1427" CreationDate="2016-08-07T00:55:36.657" Score="0" Body="&lt;p&gt;Watson can make its diagnosis based on the patient's data and comparing it to the data from millions of other studies.&lt;/p&gt;&#xA;&lt;p&gt;For example, having enough genetic data and the right algorithms, its AI computing capability demonstrated the huge potential of data analysis based on which it can be used for everything from diagnosing rare illnesses to prescribing perfect dosages of medicine based on the patient's personal genetic makeup. Of course, there are still plenty of challenges that need to be overcome before it can be used in mainstream medicine.&lt;/p&gt;&#xA;&lt;p&gt;Recently Watson was able to diagnose a rare form of leukaemia after treatment was proved ineffective. It was fed in with the patient’s genetic data and compared to data from other 20 million oncological studies.&lt;/p&gt;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ndtv.com/health/artificial-intelligence-used-to-detect-rare-leukemia-type-in-japan-1440789&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence Used To Detect Rare Leukemia Type In Japan&lt;/a&gt; (2016)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://siliconangle.com/blog/2016/08/05/watson-correctly-diagnoses-woman-after-doctors-were-stumped/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Watson correctly diagnoses woman after doctors were stumped&lt;/a&gt; (2016)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="36737" LastEditDate="2021-03-31T22:27:21.613" LastActivityDate="2021-03-31T22:27:21.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1429" PostTypeId="1" AcceptedAnswerId="1430" CreationDate="2016-08-07T01:32:32.137" Score="3" ViewCount="580" Body="&lt;p&gt;Recently White House published the article: &lt;a href=&quot;https://www.whitehouse.gov/blog/2016/05/03/preparing-future-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Preparing for the Future of Artificial Intelligence&lt;/a&gt; which says that the government is working to leverage AI for the public good and toward a more effective government.&lt;/p&gt;&#xA;&lt;p&gt;I'm especially interested in how AI can help with computational sustainability, environmental management and Earth's ecosystem, such as biological conservation?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-04-01T01:25:39.373" LastActivityDate="2021-04-01T01:25:39.373" Title="How machine learning can help with sustainable development and biological conservation?" Tags="&lt;applications&gt;&lt;biology&gt;&lt;green-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1430" PostTypeId="2" ParentId="1429" CreationDate="2016-08-07T01:32:32.137" Score="2" Body="&lt;p&gt;There are a variety of aspects where AI can help for the public good. Future studies of computational methods can contribute to a sustainable management ecosystem by its data acquisition, interpretation, integration and model fitting.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.engr.oregonstate.edu/%7Etgd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prof. Tom Dietterich&lt;/a&gt; is a leader in combining computer science and ecological sciences to build the new discipline of Ecosystem Informatics which studies methods for collecting, analyzing and visualizing data on the structure and function of ecosystems.&lt;/p&gt;&#xA;&lt;p&gt;His group is involved in many aspects of the ecosystem, such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Models that can predict species distribution and their presence/absence elsewhere in order to create species distribution and migration/dispersal maps (such as &lt;a href=&quot;https://dataone.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DataONE Datanet&lt;/a&gt;, &lt;a href=&quot;http://ebird.org/content/ebird/&quot; rel=&quot;nofollow noreferrer&quot;&gt;eBird project&lt;/a&gt;, &lt;a href=&quot;http://birdcast.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BirdCast&lt;/a&gt;) .&lt;/li&gt;&#xA;&lt;li&gt;Bio-economic models require solving large Spatio-temporal optimization problems under uncertainty.&lt;/li&gt;&#xA;&lt;li&gt;Ecosystem prediction problems require integrating heterogeneous data sources.&lt;/li&gt;&#xA;&lt;li&gt;Algorithms for deployment (sensor placement), cleaning and analysis of sensor network data of resulting data to increase agricultural productivity (Project &lt;a href=&quot;http://tahmo.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TAHMO&lt;/a&gt;), like deployment of 20,000 hydro-meteorological stations in Africa (e.g. computational problem where to place it).&lt;/li&gt;&#xA;&lt;li&gt;Systems for capturing, imaging, and sorting bugs combined with general image processing/machine learning/pattern recognition tools for counting and classifying them (&lt;a href=&quot;http://web.engr.oregonstate.edu/%7Etgd/bugid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BugID project&lt;/a&gt;). The goal is to develop algorithms for automating biodiversity based on visual pattern recognition by using the computer vision method.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For further information about this work, check the following resources, see:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(video) &lt;a href=&quot;https://www.youtube.com/watch?v=FjHBFWwOdIk&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Challenges for Machine Learning in Computational Sustainability&amp;quot; (CRCS)&lt;/a&gt; (2013)&lt;/li&gt;&#xA;&lt;li&gt;(slides) &lt;a href=&quot;http://cra.org/ccc/wp-content/uploads/sites/2/2016/06/Thomas-Dietterich-AI-slides.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tom Dietterich, Understanding and Managing Ecosystems through AI&lt;/a&gt; (2013)&lt;/li&gt;&#xA;&lt;li&gt;(study) &lt;a href=&quot;http://rspb.royalsocietypublishing.org/content/282/1808/20142984&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adapting environmental management to uncertain but inevitable change&lt;/a&gt; (2015)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="36737" LastEditDate="2021-03-31T22:22:33.033" LastActivityDate="2021-03-31T22:22:33.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1431" PostTypeId="1" AcceptedAnswerId="1438" CreationDate="2016-08-07T01:51:28.157" Score="-2" ViewCount="151" Body="&lt;p&gt;When AI has some narrow domain, such as chess, where it can outperform the world's human masters of chess, does it make it a &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;superintelligence&lt;/a&gt; or not?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-08-11T19:45:36.987" LastActivityDate="2020-08-11T19:45:36.987" Title="Is Deep Blue superintelligent or not?" Tags="&lt;definitions&gt;&lt;superintelligence&gt;&lt;deep-blue&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1432" PostTypeId="1" CreationDate="2016-08-07T02:08:34.803" Score="9" ViewCount="1475" Body="&lt;p&gt;Suppose my goal is to collaborate and create an advanced AI, for instance one that resembles a human being and the project would be on the frontier of AI research, what kind of skills would I need?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am talking about specific things like what university program should I complete to enter and be competent in the field. Here are some of the things that I thought about, just to exemplify what I mean:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Computer sciences: obviously the AI is built on computers, it wouldn't hurt to know how computers work, but some low level stuff and machine specific things does not seem essential, I may be wrong of course.&lt;/li&gt;&#xA;&lt;li&gt;Psychology: if AI resembles human beings, knowledge of human cognition would probably be useful, although I do not imagine neurology on a cellular level or complicated psychological quirks typical to human beings like the Oedipus complex would be relevant, but again, I may be wrong.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1321" LastEditorUserId="2444" LastEditDate="2019-06-17T20:27:32.460" LastActivityDate="2019-06-17T20:27:32.460" Title="What kind of education is required for researchers in AI?" Tags="&lt;getting-started&gt;&lt;academia&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1433" PostTypeId="1" AcceptedAnswerId="1434" CreationDate="2016-08-07T02:33:51.350" Score="3" ViewCount="690" Body="&lt;p&gt;&lt;a href=&quot;https://web.archive.org/web/20161221222448/https://www.whitehouse.gov/webform/rfi-preparing-future-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;White House published the information&lt;/a&gt; about AI which requests mentions about &lt;em&gt;the most important research gaps in AI that must be addressed to advance this field and benefit the public&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are these exactly?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-03-07T20:34:56.477" LastActivityDate="2021-03-31T22:27:56.533" Title="What are the most pressing fundamental questions and gaps in AI research?" Tags="&lt;research&gt;&lt;social&gt;&lt;ethics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="1434" PostTypeId="2" ParentId="1433" CreationDate="2016-08-07T02:33:51.350" Score="3" Body="&lt;p&gt;According to IBM Research organization in the response to the White House as part of &lt;a href=&quot;http://web.archive.org/web/20160808135040/https://www.whitehouse.gov/webform/rfi-preparing-future-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;preparing for the future of Artificial Intelligence&lt;/a&gt;, AI depends upon many long-term advances, not only from AI researchers but from many interdisciplinary teams of experts from many disciplines, including the following challenges:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Machine learning and reasoning.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Currently, AI systems use supervised learning using a huge amount of dataset of labelled data for training. This is very different to how humans learn by creating concepts, relationship, common sense reasoning which gives the ability to learn much without too much data. Therefore machine learning with common-sense reasoning capabilities should be researched further more.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Decision techniques.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Current AI-based systems have very limited ability for making decisions, therefore new techniques must be developed (e.g. modeling systemic risks, analyzing tradeoffs, detecting anomalies in context, analyzing data while preserving privacy).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Domain-specific AI systems.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The current AI-based system is lack of abilities to understand the variety of domains of human expertise (such as medicine, engineering, law and many more). The systems should be able to perform professional-level tasks such as designing problems, experiments, managing contradictions, negotiating, etc.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Data assurance and trust.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The current AI-based systems require huge amounts of data and their behaviour directly depends on the quality of this data which can be biased, incomplete or compromised. This can be expensive and time-consuming especially where it is used for safety-critical systems which potentially can be very dangerous.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Radically efficient computing infrastructure.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The current AI-based systems require unprecedented workloads and computing power which require the development of new computing architectures (such as neuromorphic).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Interpretability and explanations.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For people to follow AI suggestions, they need to trust systems, and this is only when they are capable of knowing users' intents, priorities, reasoning and they can learn from their mistakes. These capabilities are required in many business domains and professionals&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Value alignment and ethics.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Humans can share the common knowledge of how the world function, the machine cannot. They can fail by having unintended and unexpected behaviour only because humans did not specify the right goals for them or they omitted essential training details. The systems should be able to correct the specification of the goals and avoid unintended and undesired consequences in the behaviour.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Social AI.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The AI-based systems should be able to work closely with humans in their professional and personal life, therefore they should have significant social capabilities because they can impact our emotions and our decision-making capabilities. Also, sophisticated natural language capabilities will need to be developed to allow a natural interaction and dialogue between humans and machines.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Source: &lt;em&gt;Fundamental questions in AI research, and the most important research gaps (RFI questions 5 and 6)&lt;/em&gt;. (link no longer available)&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="36737" LastEditDate="2021-03-31T22:27:56.533" LastActivityDate="2021-03-31T22:27:56.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1436" PostTypeId="1" AcceptedAnswerId="1439" CreationDate="2016-08-07T03:49:20.143" Score="5" ViewCount="546" Body="&lt;p&gt;Is there any methods by which artificial intelligence use recursion(s) to solve a certain issue or to keep up working and calculating?&lt;/p&gt;&#xA;" OwnerUserId="1270" LastActivityDate="2018-12-28T21:43:24.817" Title="Is recursion used in practice to improve performance of AI systems?" Tags="&lt;math&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1437" PostTypeId="2" ParentId="52" CreationDate="2016-08-07T06:30:33.053" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow&quot;&gt;Andrej Karpathy's blog&lt;/a&gt; has a tutorial on getting a neural network to learn pong with reinforcement learning. His commentary on the current state of the field is interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He also provides a whole bunch of links (David Silver's &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html&quot; rel=&quot;nofollow&quot;&gt;course&lt;/a&gt; catches my eye). &lt;a href=&quot;https://www.youtube.com/watch?v=2pWv7GOvuf0&quot; rel=&quot;nofollow&quot;&gt;Here is a working link to the lecture videos.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are demos of DeepMinds game playing.&#xA;Get links to the papers at Andrej Karpathy's blog above&#xA;- &lt;a href=&quot;https://www.youtube.com/watch?v=MAMuNUixKJ8&quot; rel=&quot;nofollow&quot;&gt;rat fps&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://livestream.com/oxuni/StracheyLectureDrDemisHassabis/videos/113380152&quot; rel=&quot;nofollow&quot;&gt;nice demos at 19 minutes into this&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2016-08-08T08:34:38.777" LastActivityDate="2016-08-08T08:34:38.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1438" PostTypeId="2" ParentId="1431" CreationDate="2016-08-07T07:04:07.740" Score="4" Body="&lt;p&gt;There are three typical use cases for the phrase 'superintelligent':&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Something that is at least as smart as a human for &lt;em&gt;every&lt;/em&gt; task.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Something that is smart enough to improve itself on a fundamental level.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Something that is smarter than a human at a &lt;em&gt;single&lt;/em&gt; task.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Most uses that I see are definition 2, but the other two are also somewhat common. I typically follow I. J. Good and use &lt;a href=&quot;https://en.wikipedia.org/wiki/I._J._Good#Research_and_publications&quot; rel=&quot;noreferrer&quot;&gt;ultraintelligence&lt;/a&gt; for the first definition. (There are lots of arguments that something that fits definition 1 is likely to fit definition 2 as well, but this doesn't seem to be logically necessary.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, Deep Blue only counts for definition 3.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-07T07:04:07.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1439" PostTypeId="2" ParentId="1436" CreationDate="2016-08-07T07:11:56.880" Score="5" Body="&lt;p&gt;To my knowledge, recursion does not play a strong role in the definition of modern AI techniques, although it does feature used in Lovasz's definition of 'Local Search' and Kurzweil is &lt;a href=&quot;http://www.kurzweilai.net/a-formula-for-intelligence-the-recursive-paradigm&quot; rel=&quot;nofollow noreferrer&quot;&gt;certainly an advocate&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recursion can be seen as an elegant 'architectural factorization' - building complexity by combining the results of smaller, similar patterns previously encountered. Computationally, &lt;a href=&quot;https://stackoverflow.com/questions/931762/can-every-recursion-be-converted-into-iteration&quot;&gt;recursion can always be converted into iteration&lt;/a&gt; so this form of elegance is really mainly of use in helping to make designs more comprehensible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GOFAI algorithms that were traditionally defined using recursion include depth- and breath- first search and means-ends analysis (used in Newell and Simon's General Problem Solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to performance, while many functions can be very economically &lt;em&gt;defined&lt;/em&gt; using recursion, the naive version of such definitions can be inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rayhightower.com/blog/2014/04/12/recursion-and-memoization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This page gives&lt;/a&gt; an example in which recursive version of the Fibonacci function, which has asymptotic execution time &lt;span class=&quot;math-container&quot;&gt;$n^{1.6}$&lt;/span&gt;, which is reduced to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; by the use of memoization.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorDisplayName="user1440" LastEditDate="2018-12-28T21:43:24.817" LastActivityDate="2018-12-28T21:43:24.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1440" PostTypeId="2" ParentId="1423" CreationDate="2016-08-07T07:23:17.390" Score="7" Body="&lt;p&gt;Douglas Hofstadter's &lt;a href=&quot;http://cognitrn.psych.indiana.edu/rgoldsto/courses/concepts/copycat.pdf&quot;&gt;CopyCat&lt;/a&gt; architecture for solving letter-string analogy problems was deliberately engineered to maintain a semantically-informed notion of 'salience', i.e. given a variety of competing possibilities, tend to maintain interest in the one that is most compelling. Although the salience value of (part of) a solution is ultimately represented numerically, the means by which it determined is broadly intended to correspond (at least functionally) to the way 'selective attention' might operate in human cognition.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-07T07:50:46.827" LastActivityDate="2016-08-07T07:50:46.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1441" PostTypeId="2" ParentId="1433" CreationDate="2016-08-07T07:39:38.143" Score="3" Body="&lt;p&gt;One way of illustrating the deficiencies of many of our current approaches &lt;em&gt;at once&lt;/em&gt; is to consider how well it is possible to represent (equivalently, learn) commonsense knowledge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this area, the Winograd Schema Challenge has been proposed by &lt;a href=&quot;http://www.cs.toronto.edu/~hector/Papers/winograd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Levesque&lt;/a&gt;, in which each problem is given as input natural language text containing an ambiguous pronoun:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Babar wonders how he can get new clothing. Luckily, a very rich old man who has always been fond of little elephants understands right away that he is longing for a fine suit.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here, the program is asked to decide if 'he' in &quot;he is longing for a fine suit&quot; refers to Babar or the old man. Several thousand such questions &lt;a href=&quot;http://commonsensereasoning.org/winograd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;have been collated&lt;/a&gt; and proposed as a more quantifiable alternative to the Turing test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the fact that the input domain is natural language, success here is undeniably a pre-requisite for AGI and (as implied in my answer &lt;a href=&quot;https://ai.stackexchange.com/questions/1376/is-it-ethical-to-implement-self-defence-for-street-walking-ai-robots&quot;&gt;here&lt;/a&gt;) for being able to interact ethically with the human world.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-07T07:39:38.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1442" PostTypeId="2" ParentId="1288" CreationDate="2016-08-07T09:42:52.590" Score="7" Body="&lt;p&gt;There does not appear to be a historical consensus on this.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptrons_(book)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page on the Perceptrons book&lt;/a&gt; (which does not come down on either side) gives an argument that the ability of MLPs to compute any Boolean function was widely known at the time (at the very least to McCulloch and Pitts).&lt;/p&gt;&#xA;&lt;p&gt;However, &lt;a href=&quot;http://harveycohen.net/image/perceptron.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; gives an account by someone present at the MIT AI lab in 1974, claiming that this was not common knowledge there, alluding to documentation in &amp;quot;Artificial Intelligence Progress Report: Research at the Laboratory in Vision, Language, and other problems of Intelligence&amp;quot; (p31-32) which is claimed to support this.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2021-01-19T00:22:23.393" LastActivityDate="2021-01-19T00:22:23.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1443" PostTypeId="2" ParentId="1426" CreationDate="2016-08-07T10:13:49.460" Score="9" Body="&lt;p&gt;There has been quite a few approaches to achieve such kind of distributed coordination. I present here one of them, for its generality and simplicity (that makes it easy to remember too). But first, the general idea behind these approaches is pretty interesting, around a mechanism called &lt;a href=&quot;https://en.wikipedia.org/wiki/Stigmergy&quot;&gt;stigmergy&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stigmergy is a behaviour coordination mechanism mediated by the environment. It was first described for termites, and the most famous example pertains to ants. Ants form trails when going out for food, but they often do not interact &lt;em&gt;directly&lt;/em&gt;. It turns out they leave pheromones on the ground as they walk away from their hills. The pheromone allows them to find their way home, and it also guides where they are going: If they find a pheromone trace from one of their peers, they follow it and their own pheromones &lt;em&gt;add up&lt;/em&gt;, reinforcing the signal of the trail. In stage, more and more ants get &quot;together as they move&quot;, forming a trail. Coordination has been achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the various implementation derived from stigmergy, there is the &quot;field-based motion coordination model&quot; (FBMCM). The idea is to create a (maybe virtual) environment that maintains some states of the world. Each object registers in the environment a signal that is maximum at the object position (its edges) and then decreases with distance. Moving objects (e.g. robots) each emit a signal relayed by the environment. They can then sense each other's field and act accordingly: E.g., when signals are strong, move away; when weak, it is safe to get closer, etc. Several complex group moves have been demonstrated in software simulators (platoon formation in games, drill simulations) and with robots. The benefit of this approach is that it can be cheap to compute even complex behaviours. For example, avoiding clashes requires simple &quot;logic&quot; code based on summing-up nearby fields value. FBMCM is pretty slick, used in video-games, but hard to implement in physical settings (to my &lt;em&gt;dating&lt;/em&gt; knowledge), as it can be challenging to build a reliable environment. See for example the work from &lt;a href=&quot;https://www.researchgate.net/publication/2949751_Field-based_Motion_Coordination_In_Quake_3_Arena&quot;&gt;Mamei and Zambonelli&lt;/a&gt;, as well as one of the first industrial implementations for robots by &lt;a href=&quot;https://www.researchgate.net/publication/221456459_Decentralized_control_of_E%27GV_transportation_systems&quot;&gt;Weyns et al.&lt;/a&gt;. Note that the implementation for robots required significant work on the environment infrastructure, made somewhat more feasible as it was a controlled warehouse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of stigmergy-like models is that they are often &lt;em&gt;simple&lt;/em&gt; and &lt;em&gt;resilient&lt;/em&gt;: You can lose an ant without impact on the food-finding trail. On the downside, these models are usually &lt;em&gt;slow&lt;/em&gt;, as the coordination takes time to emerge from indirect interactions. This can be improved upon by adding extra direct interactions (e.g. empowering ants with a GPS and a grocery store map, or just a magnetic-North sense).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, these models can collapse if the environment implementation is not reliable. It can be difficult for robots or, say, self-driving cars, if they expect some transponders put on their way, as these devices are expensive to set and maintain, and they can be broken or stolen. It would be better to endow robots with radars, sonars or other proximity sensors to implement stigmergic models. One related example is the decision by Tesla to add radars to its cars, instead of assuming reliable transponders on the road (&lt;em&gt;note&lt;/em&gt;: This is just a parallel; there is no official relation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other implementations and related models are, for example, tuple-based coordination languages such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Linda_(coordination_language)&quot;&gt;Linda&lt;/a&gt;, and network protocols like &lt;a href=&quot;https://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf&quot;&gt;Chord&lt;/a&gt;. As you see, these works are not necessarily in the &quot;AI domain&quot;.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2016-08-07T10:13:49.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1444" PostTypeId="2" ParentId="1423" CreationDate="2016-08-07T11:19:28.940" Score="5" Body="&lt;p&gt;Concentration, perhaps easier to grasp as &quot;focus&quot; or &quot;attention&quot;, has quite some history in AI. &lt;a href=&quot;https://ai.stackexchange.com/a/1440/2444&quot;&gt;This answer&lt;/a&gt; mentions CopyCat, and there was work with neural networks in the 80s as well (e.g. from &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF00363973&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fukushima&lt;/a&gt;, creator of the Neocognitron).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, &lt;em&gt;attention&lt;/em&gt; in neural networks is &lt;a href=&quot;http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gaining momentum&lt;/a&gt;. The mechanisms are applied to learning in deep neural networks.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="2444" LastEditDate="2019-11-05T16:10:02.090" LastActivityDate="2019-11-05T16:10:02.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1445" PostTypeId="2" ParentId="154" CreationDate="2016-08-07T12:21:12.527" Score="1" Body="&lt;p&gt;Yes - it would seem that it is now possible to achieve more is required from the example you've given this paper describes a DL solution to a considerably harder problem - &lt;a href=&quot;http://arxiv.org/pdf/1510.07211.pdf&quot; rel=&quot;nofollow&quot;&gt;generating the source code for a program described in natural language&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of these can be described as regression problems (i.e. the goal is to minimize some loss function on the validation set), but the search space in the natural language case is much bigger.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-07T12:21:12.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1446" PostTypeId="1" CreationDate="2016-08-07T12:57:29.583" Score="5" ViewCount="201" Body="&lt;p&gt;The Von Neumann's &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax_theorem&quot; rel=&quot;nofollow&quot;&gt;Minimax theorem&lt;/a&gt; gives the conditions that make the &lt;a href=&quot;https://en.wikipedia.org/wiki/Max%E2%80%93min_inequality&quot; rel=&quot;nofollow&quot;&gt;max-min inequality&lt;/a&gt; an equality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the max-min inequality, basically &lt;code&gt;min(max(f))&amp;gt;=max(min(f))&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Von Neumann's theorem states that, for the inequality to become an equality &lt;code&gt;f(.,y)&lt;/code&gt; should always be convex for given y and &lt;code&gt;f(x,.)&lt;/code&gt; should always be concave for given x, which also makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=m-EewaiFhF0&amp;amp;list=PLAwxTw4SYaPnidDwo9e2c7ixIsu_pdSNp&amp;amp;index=61&quot; rel=&quot;nofollow&quot;&gt;This video&lt;/a&gt; says that for a zero-sum perfect information game, the Von Neumann's theorem always holds, so that minimax always equal to maximin, which I did not quite follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;br&gt;&#xA;Why zero-sum perfect information games satisfy the conditions of Von Neumann's theorem?&lt;br&gt;&#xA;If we relax the rules to be non-zero-sum or non-perfect information, how would the conditions change?&lt;/p&gt;&#xA;" OwnerUserId="185" LastEditorUserId="1671" LastEditDate="2017-09-13T21:33:16.060" LastActivityDate="2017-10-13T22:39:46.610" Title="Illustration of Von Neumann's Minimax theorem in games?" Tags="&lt;game-theory&gt;&lt;search&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1447" PostTypeId="2" ParentId="1432" CreationDate="2016-08-07T12:59:43.577" Score="10" Body="&lt;p&gt;As a full-time AI researcher myself, I'd say that a PhD in machine learning would certainly be one useful option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in order &lt;a href=&quot;https://ai.stackexchange.com/questions/1420/how-close-are-we-to-creating-ex-machina&quot;&gt;make much-needed progress&lt;/a&gt;, AI needs avoid falling into the trap of thinking that currently fashionable methods are any kind of 'silver bullet'. There's some danger that a PhD that heads straight into (say) some sub-sub-sub area of DL would end up imposing too much bias on the student's subsequent perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI research is an essentially multi-disciplinary activity. Other possible backgrounds therefore include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Mathematics or physics (to first degree or PhD level). A strong background in either of these never did anyone any harm. People who are competent in these fields tend to be able to turn their abilities to new domains relatively easily. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Software Engineering. One of the things that AI needs are integrative architectures for knowledge engineering. &lt;a href=&quot;http://www.cogsys.wiai.uni-bamberg.de/teaching/materials/AI-Light-Bulbb.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's why&lt;/a&gt;. I believe that one of the reasons that we haven't yet managed to do &lt;a href=&quot;https://ai.stackexchange.com/questions/1396/why-ocr-cannot-be-perceived-as-good-example-of-ai&quot;&gt;OCR at the level of a 5 year old&lt;/a&gt; is that we've yet to accept that we have to 'build a sledgehammer to crack a nut'. Software architects are used to managing large-scale complexity, so they may be able to help.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cognitive Science, Psychology, Cognitive Linguistics. The reasons here are obvious.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Above all, I personally think that a good AI researcher should be creative, inquisitive and prepared to question received wisdom, all of which are more important in practice than specifics of their background.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-06-17T19:30:54.647" LastActivityDate="2019-06-17T19:30:54.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1448" PostTypeId="2" ParentId="1432" CreationDate="2016-08-07T14:18:25.840" Score="5" Body="&lt;p&gt;Research on AI seems to be getting wider these days (2016). First, &quot;obvious&quot; few departments (no order):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_science&quot; rel=&quot;noreferrer&quot;&gt;Computer Science&lt;/a&gt; (e.g. computation theory, algorithms): AI researchers there assume that intelligence is a kind of computation, under various forms (e.g. a neural network, a logic system).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Software_engineering&quot; rel=&quot;noreferrer&quot;&gt;Software Engineering&lt;/a&gt;: Assuming we find a good model for AI, how do you make it? This is what the engineer will want to figure out. And it can be hard to map mathematical models to an engineered piece.&lt;/li&gt;&#xA;&lt;li&gt;Statistics and Probabilities (more specific than just Mathematics, which is also close to Computer Science): This is about Data Science, notably as a foundation to Machine Learning, the most active branch in AI---which &quot;just&quot; covers the &lt;em&gt;learning&lt;/em&gt; part.&lt;/li&gt;&#xA;&lt;li&gt;Physics: This is particularly relevant now for hardware (see below).&lt;/li&gt;&#xA;&lt;li&gt;Neuro Science: Understand how the brain works, as an inspiration to create an artificial one, is the home for Connectionists. Recently, Hassabis and his team at Google Deepmind made several breakthroughs related to reinforcement learning, memory, attention, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Recently Electric Engineering is getting a lot of light, together with the related branches of Physics. Several public and private laboratories focus on &quot;brain chips&quot;. To name a few: IBM (who's working on that for some time already), Nvidia, and Facebook. Circa 2010, it became clear that techniques like deep learning require horsepower, thus an increasing focus on creating more powerful, smaller, more energy efficient chips. And on top of that, there is all the work in Quantum Computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the thing is, there seems to be many more fields that are getting involved in AI research. We should mention Chemistry and Biology, as both inspiration and tools to make new models or hardware (e.g. chips that do not use silicon, so they can get smaller).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for 2016, the above fields are the most active, and promise to remain very active for quite some time. Pick your own depending on your interest, skills, or mere intuition!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To finish, we may be surprised in a few years when we look back at where AI has come from. I believe that if we manage to build an AGI, it will leverage &lt;em&gt;all&lt;/em&gt; these fields anyway. I guess the thrill is to be part of the story.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2016-08-07T14:18:25.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1449" PostTypeId="2" ParentId="70" CreationDate="2016-08-07T17:14:07.107" Score="4" Body="&lt;p&gt;Convolutional neural network can be applied not only for image recognition but also for video analysis and recognition, natural language processing, in games (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#New_approaches_to_problems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Go&lt;/a&gt;) or even for &lt;a href=&quot;https://ai.stackexchange.com/a/1424/8&quot;&gt;drug discovery&lt;/a&gt; by predicting the interaction between molecules and biological proteins&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore it can be used for variety of problems by using convolutional and subsampling layers connected to more fully connected layers. They're easier to train, because have fewer parameters than fully connected networks with the same number of hidden units.&lt;sup&gt;&lt;a href=&quot;http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UFLDL&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-11T21:19:52.753" LastActivityDate="2018-04-11T21:19:52.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1450" PostTypeId="2" ParentId="1392" CreationDate="2016-08-07T18:09:03.483" Score="5" Body="&lt;p&gt;Yes. Here are some of the most prominent ones and their respective state-of-the-art errors:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.toronto.edu/~kriz/cifar.html&quot; rel=&quot;nofollow&quot;&gt;CIFAR-10&lt;/a&gt;: ~3.5% error&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.toronto.edu/~kriz/cifar.html&quot; rel=&quot;nofollow&quot;&gt;CIFAR-100&lt;/a&gt;: ~24% error  &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cs.stanford.edu/~acoates/stl10/&quot; rel=&quot;nofollow&quot;&gt;STL-10&lt;/a&gt;: ~26% error&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ufldl.stanford.edu/housenumbers/&quot; rel=&quot;nofollow&quot;&gt;SVHN&lt;/a&gt;: ~1.7% error&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.image-net.org/&quot; rel=&quot;nofollow&quot;&gt;ImageNet&lt;/a&gt; tasks: the best 2012 classification task solution got 15% top-5 error, better results are currently available&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can check an updated list of solutions &lt;a href=&quot;http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. Also, a more comprehensive list of modern datasets can be found &lt;a href=&quot;http://deeplearning.net/datasets/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="144" LastEditDate="2016-08-08T13:58:28.580" LastActivityDate="2016-08-08T13:58:28.580" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1451" PostTypeId="1" AcceptedAnswerId="2095" CreationDate="2016-08-07T18:17:11.663" Score="19" ViewCount="918" Body="&lt;p&gt;In October 2014, Dr. Mark Riedl published an approach to testing AI intelligence, called &lt;a href=&quot;http://arxiv.org/pdf/1410.6142v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the &quot;Lovelace Test 2.0&quot;&lt;/a&gt;, after being inspired by the &lt;a href=&quot;http://kryten.mm.rpi.edu/lovelace.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original Lovelace Test&lt;/a&gt; (published in 2001). Mark believed that the original Lovelace Test would be impossible to pass, and therefore, suggested a weaker, and more practical version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Lovelace Test 2.0 makes the assumption that for an AI to be intelligent, it must exhibit creativity. From the paper itself:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Lovelace 2.0 Test is as follows: artificial agent &lt;span class=&quot;math-container&quot;&gt;$a $&lt;/span&gt; is challenged as follows:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; must create an artifact &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; of type &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;;&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; must conform to a set of constraints &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$c_i ∈ C$&lt;/span&gt; is&#xA;  any criterion expressible in natural language;&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;a human evaluator &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, having chosen &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, is satisfied&#xA;  that &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; is a valid instance of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and meets &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;; and&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;a human referee &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; determines the combination of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;&#xA;  to not be unrealistic for an average human.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since it is possible for a human evaluator to come up with some pretty easy constraints for an AI to beat, the human evaluator is then expected to keep coming up with more and more complex constraints for the AI until the AI fails. The point of the Lovelace Test 2.0 is to &lt;em&gt;compare&lt;/em&gt; the creativity of different AIs, not to provide a definite dividing line between 'intelligence' and 'nonintelligence' like the Turing Test would.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am curious about whether this test has actually been used in an academic setting, or it is only seen as a thought experiment at the moment. The Lovelace Test seems easy to apply in academic settings (you only need to develop some measurable constraints that you can use to test the artificial agent), but it also may be too subjective (humans can disagree on the merits of certain constraints, and whether a creative artifact produced by an AI actually meets the final result).&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="1976" LastEditDate="2020-03-01T15:42:48.967" LastActivityDate="2020-03-01T15:42:48.967" Title="Has the Lovelace Test 2.0 been successfully used in an academic setting?" Tags="&lt;history&gt;&lt;intelligence-testing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1452" PostTypeId="2" ParentId="1397" CreationDate="2016-08-07T18:48:12.270" Score="8" Body="&lt;p&gt;Yes, although how useful this AI can be is another question entirely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;mpgac&lt;/strong&gt; is a &quot;minimally intelligent AGI&quot; trained on the GAC-80K corpus of MIST questions. As a result, it should be able to &quot;minimally&quot; pass this test. However, being trained on the GAC-80K corpus obviously make it lacking for any practical purposes. From the README:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Obviously this should only be capable of producing a minimally intelligent signal when ordinary commonsense questions are asked, of the kind depicted above, using questions which would have made sense to an average human between the years 2000 and 2005. On expert knowledge or current affairs related questions it should perform no better than chance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The point of mpgac is to compare it to other AIs that could be built to pass this test. Or as the writer wrote in the README:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When scanning the skies how can we tell whether the radio signals detected are from an intelligent source, or are merely just background or sensor noise?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Ideally, you would want to build a program that is &quot;better&quot; than mpgac. In much the same way as ELIZA can be seen as a baseline for the Turing Test, mpgac is the baseline for the MIST test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GitHub repo of mpgac (as well as the GAC-80K corpus) is available &lt;a href=&quot;https://github.com/bashrc/mindpix&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2016-08-07T18:48:12.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1453" PostTypeId="1" AcceptedAnswerId="1456" CreationDate="2016-08-07T19:49:25.977" Score="2" ViewCount="431" Body="&lt;p&gt;Convolutional neural network are leading type of feed-forward artificial neural network for image recognition. Can they be used for real-time image recognition for videos (frame by frame), or it takes too much processing (assuming they're written in C-like language)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example for classification of type of animals based on the training from huge dataset.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-07T20:36:00.187" LastActivityDate="2020-11-12T21:30:17.417" Title="Can ConvNets be used for real-time object recognition from video feed?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;performance&gt;&lt;real-time&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1454" PostTypeId="2" ParentId="186" CreationDate="2016-08-07T23:34:29.837" Score="3" Body="&lt;p&gt;No, quantum computers (as understood by mainstream scientists) cannot solve the halting problem. We can already &lt;a href=&quot;http://algorithmicassertions.com/2016/05/22/quirk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulate quantum circuits with normal computers&lt;/a&gt;; it just takes a really long time when you get a decent number of qubits involved. (Quantum computing provides exponential speedups for some problems.) Therefore, if quantum computers could solve the halting problem, we could solve the halting problem with classical computers by simulating a quantum one, but it's impossible to solve the halting problem with classical computers, so we can't do it with quantum ones either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are proponents of hypercomputation - infinite speedups using quantum computers - but the evidence put forward so far is mostly conjecture. Further reading: &lt;a href=&quot;https://arxiv.org/ftp/quant-ph/papers/0512/0512248.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Can quantum computing solve classically&#xA;unsolvable problems?&lt;/a&gt; (PDF), &lt;a href=&quot;https://cs.stackexchange.com/q/6296&quot;&gt;References on comparison between quantum computers and Turing machines&lt;/a&gt; (at CS.SE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the halting problem would make a computer exceptionally powerful. It would conceivably be able to &lt;a href=&quot;https://blogs.msdn.microsoft.com/ericlippert/2011/02/24/never-say-never-part-two/&quot; rel=&quot;nofollow noreferrer&quot;&gt;check whether complex theorems are true&lt;/a&gt; without necessarily needing to product a mathematical proof. Solving that problem isn't necessary for strong AI, though. Going with the definition of &quot;strong AI&quot; as &quot;where the machine's intellectual capability is functionally equal to a human's&quot; (&lt;a href=&quot;https://www.ocf.berkeley.edu/~arihuang/academic/research/strongai3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;), a computer could be able to learn like a human despite not being able to look at a program and see if it halts. I can't magically determine the halting properties of any arbitrary program, yet I'd like to think I'm an intelligent being.&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="-1" LastEditDate="2017-04-13T12:48:38.883" LastActivityDate="2016-08-07T23:34:29.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1456" PostTypeId="2" ParentId="1453" CreationDate="2016-08-08T04:14:49.693" Score="2" Body="&lt;p&gt;We are getting there, with as usual some trade-off between quality and speed.&lt;/p&gt;&#xA;&lt;p&gt;For example &lt;a href=&quot;http://cs231n.stanford.edu/slides/2016/winter1516_lecture8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lecture 8: Spatial Localization and Detection lecture&lt;/a&gt; shows some benchmarks (mAP = Mean Average Precision, higher is better; FPS = frame per second):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AfHt2m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AfHt2m.png&quot; alt=&quot;Table/Performance: Real-Time Detectors&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B5Qb9m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B5Qb9m.png&quot; alt=&quot;Table/Performance: R-CNN, Fast R-CNN, Faster R-CNN&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="27355" LastEditDate="2020-11-12T21:30:17.417" LastActivityDate="2020-11-12T21:30:17.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1457" PostTypeId="2" ParentId="1297" CreationDate="2016-08-08T04:44:18.407" Score="3" Body="&lt;p&gt;In AI (but in general too, I believe), a simplification is that modeling is more akin to Mathematics (and related hard sciences involved, like Physics and... Computer Science), and implementation to Software Engineering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take a concrete example, really outside of AI: Find the minimum value of a given polynomial, if it exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Mathematician will derivate the polynomial, find the zeros, and checkout convexity to find a minimum (if there is any zero). This procedure is very standard---some will say straightforward. It relies on a body of knowledge and an abstraction level that is appropriate for manual proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Software Engineer approach is actually way longer to explain, and I am going to skip it. The point is that the body of knowledge is related but different: We have to find now a step-by-step procedure for the computer to achieve the result. The Mathematician one could be implemented directly in MathLab, almost verbatim, but we &lt;em&gt;assume&lt;/em&gt; MathLab. And to build MathLab, we are back to the problem of making a procedure the computer can execute. We could for example base a procedure on Euler's method to find roots (a &quot;simple&quot; approach that closes on roots step after step), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple mathematical operations can be quite complex to implement on a computer. Perhaps the most famous is &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_number_generation&quot; rel=&quot;nofollow&quot;&gt;random-number generation&lt;/a&gt;. Mathematically, the concept is pure and clear. Generating an actual random-number is more elusive than it looks, to the point it calls for new &lt;em&gt;models&lt;/em&gt; and new &lt;em&gt;implementations&lt;/em&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A concrete example from history: Neural networks. In the 80s and 90s, NNs were weighted graphs that could be executed on computers using graph libraries or similar foundation libs. Choosing the weights was challenging. One day the back-propagation learning model was introduced to automated the choice of weights. The model relied on a procedure dedicated to NNs, using a terminology like partial derivates, gradient descent, chain rules, etc. And later then, clever engineers created libraries to automate the back-propagation procedure. The libraries can be somewhat far from the original model, as engineers learn how to make it computable, even faster (i.e. optimization, approximations/truncations).&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2016-08-08T04:44:18.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1458" PostTypeId="1" AcceptedAnswerId="1460" CreationDate="2016-08-08T07:04:49.487" Score="4" ViewCount="139" Body="&lt;p&gt;Just for the purpose of learning I'd like to classify the likeliness of a tweet being in aggressive language or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering how to approach the problem. I guess I need first train my neural network on a huge dataset of text what aggressive language is. This brings up the question where I would get this data in the first place?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It feels a bit like the chicken and egg problem to me so I wonder how would I approach the problem?&lt;/p&gt;&#xA;" OwnerUserId="1334" LastActivityDate="2018-09-30T15:59:31.110" Title="How to classify language as friendly or aggressive with AI?" Tags="&lt;classification&gt;&lt;datasets&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1459" PostTypeId="2" ParentId="1458" CreationDate="2016-08-08T07:49:22.200" Score="3" Body="&lt;p&gt;I did a little search and couldn't find any database that has ground truth for aggressiveness. This means that you need to build yourself a database. This might be huge undertaking. Take thousands of messages, and classify them by hand whether they are aggressive or not. This part is quite labor intensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second part is much easier at start but would be pain to optimize (both performance and computational cost). I would suggest you to start with Naive Bayes classifier for this job. That is the preferred classifier for spam detection. ANN would probably not work for this case because the data would be a huge sparse vector. Estimated number of words in English is over a million, which means the input layer of your ANN should be able to scale up to that number. Search for sparse vector classification for additional classifier that can be used in these cases.&lt;/p&gt;&#xA;" OwnerUserId="210" LastActivityDate="2016-08-08T07:49:22.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1460" PostTypeId="2" ParentId="1458" CreationDate="2016-08-08T08:26:24.703" Score="2" Body="&lt;p&gt;The answer by &lt;a href=&quot;https://ai.stackexchange.com/users/210/cem-kalyoncu&quot;&gt;Cem Kalyoncu&lt;/a&gt; mentions the difficulty of building a ground truth database for aggressiveness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternative approach would be to attempt to operate at the &lt;em&gt;concept level&lt;/em&gt;, which would allow the use of pre-existing ontologies such as ConceptNet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.stir.ac.uk/~spo/publication/resources/sentiment-analysis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a paper&lt;/a&gt; that describes this technique.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-08T13:51:13.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1461" PostTypeId="1" AcceptedAnswerId="1483" CreationDate="2016-08-08T17:48:43.730" Score="24" ViewCount="1386" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Siri&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siri&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Cortana&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cortana&lt;/a&gt; communicate pretty much like humans. Unlike &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Now&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Now&lt;/a&gt;, which mainly gives us search results when asked some questions (not setting alarms or reminders), Siri and Cortana provide us with an answer, in the same way that a person would do.&lt;/p&gt;&#xA;&lt;p&gt;So, are they actual AI programs or not?&lt;/p&gt;&#xA;&lt;p&gt;(By &amp;quot;question&amp;quot; I don't mean any academic-related question or asking routes/ temperature, but rather opinion based question).&lt;/p&gt;&#xA;" OwnerUserId="72" LastEditorUserId="2444" LastEditDate="2021-01-20T10:53:03.583" LastActivityDate="2021-01-20T11:03:18.380" Title="Are Siri and Cortana AI programs?" Tags="&lt;chat-bots&gt;&lt;intelligence-testing&gt;&lt;weak-ai&gt;&lt;siri&gt;&lt;cortana&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="1462" PostTypeId="1" AcceptedAnswerId="1465" CreationDate="2016-08-08T17:53:40.050" Score="6" ViewCount="12125" Body="&lt;p&gt;What is the difference between artificial intelligence and robots?&lt;/p&gt;&#xA;" OwnerUserId="72" LastEditorUserId="2444" LastEditDate="2019-10-01T12:30:22.803" LastActivityDate="2019-10-01T12:30:22.803" Title="What is the difference between artificial intelligence and robots?" Tags="&lt;terminology&gt;&lt;robotics&gt;&lt;robots&gt;&lt;comparison&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1463" PostTypeId="2" ParentId="1461" CreationDate="2016-08-08T18:01:39.743" Score="8" Body="&lt;p&gt;I would classify both as having / using elements of AI, yes.  But I wouldn't say either represents a truly &quot;intelligent&quot; (in the AGI sense) program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here's the rub... as you'll see in other questions asking about definitions of AI, there's a sort of memetic thing where anything that AI begins to do successfully, immediately stops being considered &quot;AI&quot;.  So AI is always an unreachable state, because it's always &quot;something humans can do that computers can't&quot; and once the computer &lt;em&gt;can&lt;/em&gt; do it, it isn't AI anymore.   So take that into consideration.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-08T18:01:39.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1464" PostTypeId="2" ParentId="60" CreationDate="2016-08-08T18:08:26.123" Score="6" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;we don't really know what intelligence is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;we don't truly understand the best model of intelligence we have available (human intelligence) works.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;we're trying to replicate human intelligence (to some extent) on hardware which is quite different from the hardware it runs on in reality.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the human brain (our best model of intelligence) is mostly a black-box to us, and it's difficult to probe/introspect its operation without killing the test subject.  This is, of course, unethical and illegal. So progress in understanding the brain is very slow. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Combine those factors and you can understand why it's difficult to make progress in AI. In many ways, you can argue that we're shooting in the dark. Of course, we have made &lt;em&gt;some&lt;/em&gt; progress, so we know we're getting some things right. But without a real comprehensive theory about &lt;em&gt;how&lt;/em&gt; AI should/will work, we are reduced to a lot of trial and error and iteration to move forward.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="14723" LastEditDate="2018-04-12T02:34:45.870" LastActivityDate="2018-04-12T02:34:45.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1465" PostTypeId="2" ParentId="1462" CreationDate="2016-08-08T18:14:19.090" Score="9" Body="&lt;p&gt;Although there are several definitions of &quot;robot&quot;, an essential feature of everything called &quot;robot&quot; is that it is capable of movement. This does not necessarily mean &lt;em&gt;displacement&lt;/em&gt;; a robot arm in a factory also moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a single exception to this rule,  which is bot-programs like chatbots; I will discuss them later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence does not need to move; a chess program can be argued to be an AI, but does not move. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robot can actually have AI; one of the definitions of robot is that it is a system, capable of &lt;em&gt;autonomous&lt;/em&gt; movement. In order to be autonomous, to be able to make decisions of its own, a certain amount of AI may be necessary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one class of &quot;robots&quot; that does not move, and does not even have physical presence; bot programs, like chatbots, that operate inside systems. I do not consider them robots, because they are not physical devices operating in the real world. A chatbot can be an AI, however - a good chatbot may have some natural language processing to interact with humans in a way that humans find natural.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize; an AI can exist purely in software. But to be a robot, there must be a moving physical component in the real world.&lt;/p&gt;&#xA;" OwnerUserId="66" LastActivityDate="2016-08-08T18:14:19.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1466" PostTypeId="2" ParentId="1334" CreationDate="2016-08-08T18:14:45.737" Score="3" Body="&lt;p&gt;IBM clearly don't provide all the details / &quot;secret sauce&quot; but there is some information out there on how Watson works.  Some of the text search / retrieval stuff uses a technology called &lt;a href=&quot;http://uima.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UIMA&lt;/a&gt; which IBM open-sourced a few years ago.  It also uses Prolog and some custom C++ code.    Some more information can be found &lt;a href=&quot;https://web.archive.org/web/20161106224106/http://learning.acm.org/webinar/lally.cfm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2019-07-03T03:21:53.060" LastActivityDate="2019-07-03T03:21:53.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1467" PostTypeId="2" ParentId="1462" CreationDate="2016-08-08T18:17:21.233" Score="5" Body="&lt;p&gt;In the broadest sense, the difference is that non-robotic A(G)I &lt;em&gt;may&lt;/em&gt; not be possible because, as per &lt;a href=&quot;https://ai.stackexchange.com/questions/1415/what-kind-of-body-if-any-does-intelligence-require&quot;&gt;this question&lt;/a&gt;, it could be that &quot;Intelligence requires a body&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, it could be that there are limitations to what the traditional (well, 1950s style) 'Brain in a vat' notion of an AI is capable of comprehending, in the absence of experience of embodied experience such as force, motion and &quot;the raw, unawshed world&quot;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-08T19:14:17.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1469" PostTypeId="2" ParentId="1458" CreationDate="2016-08-08T18:19:35.150" Score="2" Body="&lt;p&gt;A simple way to do it would be lexicograpical sentiment analysis.  To do that, you'd need a list of words categorized with a score that reflects &quot;friendly&quot; vs &quot;aggressive&quot; sentiment.   For an example of setting up a SA system using Spark, see &lt;a href=&quot;http://mammothdata.com/sentiment-analysis-on-enrons-emails-with-apache-spark/&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt;.  To do what you're talking about, substitute AFINN for a different dataset.  You might have to create said dataset yourself, if there isn't one &quot;out there&quot; like you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this isn't the most sophisticated technique in the world, but it's been found to be surprisingly effective. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-08T18:19:35.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1470" PostTypeId="2" ParentId="1410" CreationDate="2016-08-08T18:25:11.147" Score="0" Body="&lt;p&gt;One thing you'll see quite often, is to declare a correspondence between a system and a human of a given age.  For example &quot;this program can answer questions about science approximately as well as an average 7 year old&quot; or something of that nature.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-08T18:25:11.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1471" PostTypeId="2" ParentId="60" CreationDate="2016-08-08T19:11:47.130" Score="3" Body="&lt;p&gt;I am assuming by AI you mean AG(eneral)I, not machine learning or expert systems tuned for specific tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to @mindcrime's answer, sometimes we run out of samples to train and sometimes computers became so slow to process enough samples to work in manageable timescales. @bpachev mentioned memory but on the surface, our supercomputers have more than enough memory to store a human brain matrix. But we lack the ability to simulate it real time. After we are able to do that, we also need to connect external input, even more processing power is required for that. Even that would not be enough to simulate a human brain fully as biochemistry plays an important role. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One final note would be there is little incentive to develop AGI other than understanding how the human mind works. There are classification algorithms, expert systems, knowledge engines that can out-perform even the best humans for specific tasks.&lt;/p&gt;&#xA;" OwnerUserId="210" LastEditorUserId="14723" LastEditDate="2018-04-12T02:23:19.220" LastActivityDate="2018-04-12T02:23:19.220" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1472" PostTypeId="1" CreationDate="2016-08-08T19:24:16.993" Score="3" ViewCount="67" Body="&lt;p&gt;With typical machine learning you would usually use a training data-set to create a model of some kind, and a testing data-set to then test the newly created model. For something like linear regression after the model is created with the training data you now have an equation that you would use to predict the outcome of the set of features in the testing data. You would then take the prediction that the model returned and compare that to the actual data in the testing set. How would a validation set be used here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With nearest neighbor you would use the training data to create an n-dimensional space that has all the features of the training set. You would then use this space to classify the features in the testing data. Again you would compare these predictions to the actual value of the data. How would a validation set help here as well?&lt;/p&gt;&#xA;" OwnerUserId="1324" LastEditorUserId="10135" LastEditDate="2018-10-19T14:23:59.100" LastActivityDate="2018-10-19T14:23:59.100" Title="How exactly does a validation data-set work work in machine learning?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;linear-regression&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2016-08-09T15:46:42.237" ContentLicense="CC BY-SA 3.0" />
  <row Id="1473" PostTypeId="5" CreationDate="2016-08-08T19:37:59.710" Score="0" Body="&lt;p&gt;A robot is a physical device capable of independent movement. It has a software component to steer it, but does not necessarily have (artificial) intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The word &quot;Robot&quot; was coined by Czech writer Karel Čapek in his play R.U.R. (Rossum's Universal Robots). It comes from a Czech word meaning labour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Artificial Intelligence Stack Exchange, we are interested in how AI relates with these physical devices.&lt;br&gt;&#xA;If you are interested in the physical aspect of robots, like the electronical or mechanical aspects, you should visit &lt;a href=&quot;https://robotics.stackexchange.com&quot;&gt;Robotics Stack Exchange&lt;/a&gt; instead.&lt;/p&gt;&#xA;" OwnerUserId="66" LastEditorUserId="66" LastEditDate="2016-08-09T00:55:30.283" LastActivityDate="2016-08-09T00:55:30.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1474" PostTypeId="4" CreationDate="2016-08-08T19:37:59.710" Score="0" Body="A robot is a physical device capable of independent movement. Use this tag for questions involving the relationship of AI with robots. If you have questions about the physical aspect of robots, you should see if it fits our sister site Robotics, at https://robotics.stackexchange.com." OwnerUserId="66" LastEditorUserId="145" LastEditDate="2016-08-23T00:18:05.657" LastActivityDate="2016-08-23T00:18:05.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1476" PostTypeId="1" AcceptedAnswerId="1591" CreationDate="2016-08-08T23:46:12.853" Score="5" ViewCount="465" Body="&lt;p&gt;By reinforcement learning, I don't mean the class of machine learning algorithms such as DeepQ, etc. I have in mind the general concept of learning based on rewards and punishment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to create a Strong AI that does not rely on learning by reinforcement, or is reinforcement learning a requirement for artificial intelligence? The existence of rewards and punishment imply the existence of favorable and unfavorable world-states. Must intelligence in general and artificial intelligence in particular have a way of classifying world-states as favorable or unfavorable?  &lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2018-11-08T23:55:08.853" Title="Is reinforcement learning needed to create Strong AI?" Tags="&lt;philosophy&gt;&lt;reinforcement-learning&gt;" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1478" PostTypeId="1" AcceptedAnswerId="1527" CreationDate="2016-08-09T02:01:58.240" Score="2" ViewCount="2478" Body="&lt;p&gt;For example, search engine companies want to classify their image searches into 2 categories (which they already do that) such as: &lt;a href=&quot;https://en.wikipedia.org/wiki/Not_safe_for_work&quot; rel=&quot;nofollow&quot;&gt;NSFW&lt;/a&gt; (nudity, porn, brutality) and safe to view pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can artificial neural networks achieve that, and at what success rate? Can they be easily mistaken?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-17T19:58:38.663" LastActivityDate="2020-06-23T15:57:33.153" Title="How successfully can convnets detect NSFW images?" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1479" PostTypeId="1" AcceptedAnswerId="4044" CreationDate="2016-08-09T02:08:56.663" Score="88" ViewCount="15335" Body="&lt;p&gt;Do scientists or research experts know from the kitchen what is happening inside complex &quot;deep&quot; neural network with at least millions of connections firing at an instant? Do they understand the process behind this (e.g. what is happening inside and how it works exactly), or it is a subject of debate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example this &lt;a href=&quot;https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf&quot; rel=&quot;noreferrer&quot;&gt;study&lt;/a&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However there is no clear understanding of &lt;em&gt;why&lt;/em&gt; they perform so well, or &lt;em&gt;how&lt;/em&gt; they might be improved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So does this mean that scientists actually don't know how complex convolutional network models work?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-28T14:06:00.337" LastActivityDate="2021-04-16T10:08:19.733" Title="Do scientists know what is happening inside artificial neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;explainable-ai&gt;" AnswerCount="7" CommentCount="3" FavoriteCount="54" ContentLicense="CC BY-SA 3.0" />
  <row Id="1480" PostTypeId="1" AcceptedAnswerId="1741" CreationDate="2016-08-09T02:30:17.240" Score="-3" ViewCount="251" Body="&lt;p&gt;Is there any way to estimate how big the neural network would be after training session of 100,000 unlabeled images for unsupervised learning (like in &lt;a href=&quot;https://cs.stanford.edu/~acoates/stl10/&quot; rel=&quot;nofollow&quot;&gt;STL-10 dataset&lt;/a&gt;: 96x96 pixels and color)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not the storage space (because this could vary I guess based on the implementation), but specifically how many neurons it could have. It could be an estimate (e.g. in thousand, millions). If it depends, then on what? Are there any figures that can be estimated?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-25T18:48:24.383" LastActivityDate="2017-02-13T20:48:43.850" Title="How many neurons would a network have after a training of 100k small images?" Tags="&lt;deep-neural-networks&gt;&lt;image-recognition&gt;&lt;deep-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1481" PostTypeId="1" AcceptedAnswerId="1699" CreationDate="2016-08-09T02:54:33.677" Score="7" ViewCount="1582" Body="&lt;p&gt;For example, I would like to train my neural network to recognize the type of actions (e.g. in commercial movies or some real-life videos), so I can &quot;ask&quot; my network in which video or movie (and at what frames) somebody was driving a car, kissing, eating, was scared or was talking over the phone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the current successful approaches to that type of problem?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-28T14:54:48.483" LastActivityDate="2021-03-31T22:21:17.730" Title="How can action recognition be achieved?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;training&gt;&lt;action-recognition&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1482" PostTypeId="2" ParentId="1462" CreationDate="2016-08-09T06:18:18.593" Score="0" Body="&lt;p&gt;An &lt;a href=&quot;http://www.oxforddictionaries.com/definition/english/artificial-intelligence#artificial-intelligence__2&quot; rel=&quot;nofollow&quot; title=&quot;OED - AI&quot;&gt;AI&lt;/a&gt; is a computer program designed for tasks normally requiring human &lt;a href=&quot;http://www.oxforddictionaries.com/definition/english/intelligence&quot; rel=&quot;nofollow&quot; title=&quot;OED - Intelligence&quot;&gt;intelligence&lt;/a&gt; (a human's ability to learn), while a &lt;a href=&quot;http://www.oxforddictionaries.com/definition/english/robot&quot; rel=&quot;nofollow&quot; title=&quot;OED - Robot&quot;&gt;robot&lt;/a&gt; is a machine that completes complex tasks. An AI could be used to control a robot, but they are very different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://www.oxforddictionaries.com/&quot; rel=&quot;nofollow&quot;&gt;Oxford English Dictionary&lt;/a&gt;, above links will direct to definitions.&lt;/p&gt;&#xA;" OwnerUserId="1420" LastEditorUserId="72" LastEditDate="2016-08-10T11:11:25.337" LastActivityDate="2016-08-10T11:11:25.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1483" PostTypeId="2" ParentId="1461" CreationDate="2016-08-09T06:38:26.720" Score="19" Body="&lt;p&gt;Siri and Cortana are AI to some extent. The usual label is &amp;quot;weak AI&amp;quot; (also called &amp;quot;narrow&amp;quot; or &amp;quot;soft&amp;quot; AI). It turns out the &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Weak_AI&amp;amp;oldid=736852472&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on Weak AI&lt;/a&gt; explicitly refers to Siri:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Siri is a good example of narrow intelligence. Siri operates within a limited pre-defined range, there is no genuine intelligence, no self-awareness, no life despite being a sophisticated example of weak AI. In Forbes (2011), Ted Greenwald wrote: &amp;quot;The iPhone/Siri marriage represents the arrival of hybrid AI, combining several narrow AI techniques plus access to massive data in the cloud.&amp;quot; AI researcher Ben Goertzel, on his blog in 2010, stated Siri was &amp;quot;VERY narrow and brittle&amp;quot; evidenced by annoying results if you ask questions outside the limits of the application.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Important to note that &amp;quot;mixing&amp;quot; weak AIs does not make a &amp;quot;stronger&amp;quot; AI, by some arguments (see Searle's &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room&lt;/a&gt; argument), but there is no consensus on this topic yet.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="2444" LastEditDate="2021-01-20T11:03:18.380" LastActivityDate="2021-01-20T11:03:18.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1484" PostTypeId="1" CreationDate="2016-08-09T06:55:39.460" Score="1" ViewCount="965" Body="&lt;p&gt;I'm playing with an LSTM to generate text. In particular, this one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://raw.githubusercontent.com/fchollet/keras/master/examples/lstm_text_generation.py&quot; rel=&quot;nofollow&quot;&gt;https://raw.githubusercontent.com/fchollet/keras/master/examples/lstm_text_generation.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works on quite a big demo text set from Nietzsche and says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you try this script on new data, make sure your corpus&#xA;  has at least ~100k characters. ~1M is better.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This pops up a couple of questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A.) If all I want is an AI with a very limited vocabulary where the generate text should be short sentences following a basic pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I like blue sky with white clouds&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I like yellow fields with some trees&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I like big cities with lots of bars&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it then be reasonable to use a much much smaller dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B.) If the dataset really needs to be that big. What if I just repeat the text over and over to reach the recommended minimum? If that would work though, I'd be wondering how that is any different from just taking more iterations of learning with the same shorter text?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously I can play with these two questions myself and in fact I am experimenting with it. One thing I already figured out is that with a shorter text following a basic pattern I can get to a very very low ( ~0.04) quite fast but the predicted text just turns out as gibberish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My naive explanation for that would be that there are just not enough samples to proof against whether the gibberish actually makes sense or not? But then again I wonder if more iterations or duplicating the content would actually help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to experiment with these questions myself so please don't think I'm just too lazy and are aiming for others to do the work. I'm just looking for more experienced people to give me a better understanding of the mechanics that influence these things.&lt;/p&gt;&#xA;" OwnerUserId="1334" LastActivityDate="2017-04-25T14:00:05.233" Title="In LSTM text generation can low amount of training data be compensated?" Tags="&lt;datasets&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1485" PostTypeId="1" AcceptedAnswerId="1523" CreationDate="2016-08-09T09:01:52.373" Score="3" ViewCount="107" Body="&lt;p&gt;For example I would like to implement transparent AI in the RTS game which doesn't offer any AI API (like old games), and I'd like to use image recognition algorithm for detecting the objects which can talks to another algorithm which is responsible for the logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given I'd like to use two neural networks, what are the approaches to setup the communication between them? Is it just by exporting result findings of the first algorithm (e.g. using CNN) with list of features which were found on the screen, then use it as input for another network? Or it's more complex than that, or I need to have more than two networks?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-09T18:57:43.590" LastActivityDate="2018-07-02T21:02:21.280" Title="How to separate image recognition from logic?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;gaming&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1486" PostTypeId="2" ParentId="1479" CreationDate="2016-08-09T09:09:32.990" Score="30" Body="&lt;p&gt;It depends on what you mean by &quot;know what is happening&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceptually, yes: ANN perform nonlinear regression. The actual expression represented by the weight matrix/activation function(s) of an ANN can be explicitly expanded in symbolic form (e.g. containing sub-expressions such as &lt;span class=&quot;math-container&quot;&gt;$1/1+e^{1/1+e^{\dots}}$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if by 'know' you mean &lt;em&gt;predicting the output of some specific (black box) ANN&lt;/em&gt;, by some other means, then the obstacle is the presence of chaos in a ANN that has &lt;a href=&quot;http://sprott.physics.wisc.edu/pubs/paper234.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;high degrees of freedom&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's also some relatively recent work by Hod Lipson on understanding ANNs through  &lt;a href=&quot;http://arxiv.org/pdf/1506.06579.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;visualisation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-02-11T21:23:06.513" LastActivityDate="2019-02-11T21:23:06.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1487" PostTypeId="1" AcceptedAnswerId="1503" CreationDate="2016-08-09T09:09:34.990" Score="1" ViewCount="173" Body="&lt;p&gt;Were there any successful attempts to replace poor guide dogs used for blind people with AI to achieve similar rate of success? I guess dogs could be easily distracted and not reliable for every situation, and it probably takes less time to train AI, than a dog.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="29" LastEditDate="2016-08-17T12:03:08.677" LastActivityDate="2016-08-17T12:03:08.677" Title="How close we are to replacing guide dogs with AI?" Tags="&lt;neural-networks&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1488" PostTypeId="1" AcceptedAnswerId="1522" CreationDate="2016-08-09T10:10:25.253" Score="6" ViewCount="816" Body="&lt;p&gt;Do we know why Tesla's autopilot mistaken empty sky with a high-sided lorry which resulted in fatal crash involving a car in self-drive mode? Was it AI fault or something else? Is there any technical explanation behind this why this happened?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The articles &lt;a href=&quot;http://news.sky.com/story/tesla-driver-in-first-self-drive-fatal-crash-10330121&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla Driver In First Self-Drive Fatal Crash&lt;/a&gt; and &lt;a href=&quot;http://www.theverge.com/2016/6/30/12072408/tesla-autopilot-car-crash-death-autonomous-model-s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla driver killed in crash with Autopilot active, NHTSA investigating&lt;/a&gt; talk about this topic, but do not go in detail.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-20T15:01:51.317" LastActivityDate="2019-06-20T15:01:51.317" Title="Why did a Tesla car mistake a truck with a bright sky?" Tags="&lt;autonomous-vehicles&gt;&lt;security&gt;&lt;ai-safety&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1489" PostTypeId="2" ParentId="1479" CreationDate="2016-08-09T10:28:31.650" Score="9" Body="&lt;p&gt;Not sure if this is what you are searching for, but google extracted images from networks when they were fed with white noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inceptionism: Going Deeper into Neural Networks (Google Research Blog)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of represents what the network knows.&lt;/p&gt;&#xA;" OwnerUserId="1425" LastEditorUserId="1671" LastEditDate="2018-03-14T21:24:06.093" LastActivityDate="2018-03-14T21:24:06.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1490" PostTypeId="1" AcceptedAnswerId="1499" CreationDate="2016-08-09T10:28:41.783" Score="1" ViewCount="106" Body="&lt;p&gt;For benefits of testing AGI, is using a high-level video game description language (VGDL) gives more reliable and accurate results of general intelligence than using Arcade Learning Environment (ALE)?&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-09T20:35:34.217" Title="What are the benefits of the VGDL over the ALE?" Tags="&lt;agi&gt;&lt;gaming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1491" PostTypeId="1" AcceptedAnswerId="2183" CreationDate="2016-08-09T10:40:35.500" Score="2" ViewCount="94" Body="&lt;p&gt;Some time ago playing chess was challenging for algorithms, then Go game which is vastly more complex than compared to chess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about playing RTS game which have enormous branching factors limited by its time and space (like deciding what to do next)? What are the successful approaches to such problems?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-09-19T11:51:39.703" LastActivityDate="2019-09-19T11:51:39.703" Title="How to deal with huge branching factors in real-time?" Tags="&lt;game-ai&gt;&lt;branching-factors&gt;&lt;rts&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1492" PostTypeId="1" AcceptedAnswerId="1495" CreationDate="2016-08-09T12:00:06.583" Score="5" ViewCount="1385" Body="&lt;p&gt;We can read on wiki page that in March 2016 AlphaGo AI lost its game (1 of 5) to Lee Sedol, a professional Go player. One &lt;a href=&quot;http://www.bbc.co.uk/news/technology-36558829&quot; rel=&quot;nofollow&quot;&gt;article&lt;/a&gt; cite says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AlphaGo lost a game and we as researchers want to explore that and find out what went wrong. We need to figure out what its weaknesses are and try to improve it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Have researchers already figured it out what went wrong?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-03-08T06:08:14.880" LastActivityDate="2020-03-08T06:08:14.880" Title="Why did AlphaGo lose its Go game?" Tags="&lt;game-ai&gt;&lt;deepmind&gt;&lt;alphago&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1493" PostTypeId="1" AcceptedAnswerId="1496" CreationDate="2016-08-09T12:19:35.883" Score="4" ViewCount="198" Body="&lt;p&gt;Assuming we're dealing with an artificial neural network (e.g. a ConvNet), which was trained by a large dataset of human faces.&lt;/p&gt;&#xA;&lt;p&gt;Are there any known issues or challenges where facial recognition would fail? I'm not talking about covering half of the face, but some simple common things, such as wearing glasses, a hat, jewelry, having face painting, or tattoos. Can this successfully prevent AI from recognizing the face? If so, what are the current methods for dealing with such challenges?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2021-05-19T08:30:58.980" LastActivityDate="2021-05-19T08:30:58.980" Title="Is it possible to fool a facial recognition algorithm?" Tags="&lt;convolutional-neural-networks&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1494" PostTypeId="1" AcceptedAnswerId="1497" CreationDate="2016-08-09T14:17:50.773" Score="0" ViewCount="449" Body="&lt;p&gt;I would like to know what kind of dataset I need (to prepare) for training the network to recognize the spelling mistakes in individual words for English text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the large database of words, having correct one for each incorrect. What kind of input is more efficient for that tasks? Is it using one input per each letter, syllable, whole word or I should use different pattern syllable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the input should be incorrect word, output correct, and if the word doesn't need correction, then both input and output should be the same. Is that the right approach?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="4302" LastEditDate="2018-10-08T12:54:00.933" LastActivityDate="2018-10-08T12:54:00.933" Title="Training network to detect spelling mistakes" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1495" PostTypeId="2" ParentId="1492" CreationDate="2016-08-09T15:32:41.270" Score="5" Body="&lt;p&gt;We know what Lee's strategy was during the game, and it seems like the sort of thing that should work. &lt;a href=&quot;https://gogameguru.com/lee-sedol-defeats-alphago-masterful-comeback-game-4/&quot; rel=&quot;noreferrer&quot;&gt;Here's&lt;/a&gt; an article explaining it. Short version: yes, we know what went wrong, but probably not how to fix it yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, AlphaGo is good at making lots of small decisions well, and managing risk and uncertainty better than humans can. One of the things that's surprising about it relative to previous bots that play Go is how good it was at tactical fights; in previous games, Lee had built a position that AlphaGo needed to attack, and then AlphaGo successfully attacked it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this game, Lee played the reverse strategy. Instead of trying to win many different influence battles, where AlphaGo had already shown it was stronger than him, he would set up one critical battle (incurring minor losses along the way), and then defeat it there, with ripple events that would settle the match in his favor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's the weakness of AlphaGo that allowed that to work? As I understand it, this is a fundamental limitation of Monte Carlo Tree Search (MCTS). MCTS works by randomly sampling game trees and averaging them; if 70% of games from a particular position go well and 30% of games from another position go well, then you should probably play the first move instead of the second move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when there's a specific sequence of plays that go well--if, say, W has a path that requires them playing exactly the right stone each time, but B has no possible response to this path--then MCTS breaks down, because you can only find that narrow path through minimax reasoning, and moving from the slower minimax reasoning to the faster MCTS is one of the big reasons why bots are better now than they were in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's unclear how to get around this. There may be a way to notice this sort of threat, and then temporarily switch from MCTS reasoning to minimax reasoning, or to keep around particular trajectories in memory for consideration in future plays. &lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-09T15:32:41.270" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1496" PostTypeId="2" ParentId="1493" CreationDate="2016-08-09T18:53:10.497" Score="4" Body="&lt;p&gt;Facial recognition works by essentially turning your face into a point cloud, recognizing eyes, cheeks, nose, mouth, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it doesn't look at the top of your head (hair is very hard to differentiate from other hair and doesn't have many features). Face paintings   would be your best bet since they can be easily changed, tattoos not so much. Once somebody has a photo of your face with your tattoo on it, you're busted.  Glasses will work if they're opaque and hide your eyes (sunglasses). The facial recognition software does not recognize jewelry, as it's tiny, and very easy to remove and put on. Ideally you want to have your face professionally made up with makeup and fake skin (basically a fake face).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some challenges can be anything like an unclear picture, a picture from the wrong angle (which can probably be mathematically calculated and restructured), or like you said, face paintings to hide facial features.&lt;/p&gt;&#xA;" OwnerUserId="1433" LastActivityDate="2016-08-09T18:53:10.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1497" PostTypeId="2" ParentId="1494" CreationDate="2016-08-09T19:34:23.367" Score="2" Body="&lt;p&gt;I'd personally be more inclined to try longstanding deterministic methods such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance&quot; rel=&quot;nofollow&quot;&gt;Damerau&lt;/a&gt; (for typing errors) or &lt;a href=&quot;https://en.wikipedia.org/wiki/Soundex&quot; rel=&quot;nofollow&quot;&gt;Soundex&lt;/a&gt; (for homonyms arising from transcribed speech). At the very least, I'd use those as a baseline for any more 'AI-based' approach.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-09T19:34:23.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1498" PostTypeId="2" ParentId="1488" CreationDate="2016-08-09T19:59:30.093" Score="3" Body="&lt;p&gt;As far as I know, Tesla cars autopilot is not a 100% AI pilot, it's an &lt;em&gt;assitant&lt;/em&gt;: as it detects hands off wheel it slows down, so it's incorrect to speak about AI mistake: it is not trained/designed to drive a car all by itself. A human driver is responsible in that incident.&lt;/p&gt;&#xA;" OwnerUserId="1263" LastActivityDate="2016-08-09T19:59:30.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1499" PostTypeId="2" ParentId="1490" CreationDate="2016-08-09T20:12:16.500" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.arcadelearningenvironment.org/wp-content/uploads/2012/07/bellemare13arcade.pdf&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a description of the input to an ALE agent:&#xA;Percept state: A single game screen (frame): a 2D array of 7-bit pixels, 160 pixels wide by 210 pixels high. &#xA;Actions: 18 discrete actions defined by the joystick controller&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding VGDL, as far as I can see, the main site associated with it is gvgai.net, which is currently down. The associated API is described &lt;a href=&quot;http://julian.togelius.com/Perez20152014.pdf&quot; rel=&quot;nofollow&quot;&gt;in this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Percept state for GVGAI is more structured than for ALE, but the closest correspondence to ALE appears to be an 'Observation grid', consisting of a 2D array of sprite identifiers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actions: ACTION_NIL, ACTION_UP, ACTION_LEFT, ACTION_DOWN, ACTION_RIGHT and ACTION_USE (stated as 'typical' values). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of the two, it would seem that ALE is more suitable for AGI, because of the more 'free form' nature of the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one of the issues with &lt;em&gt;either&lt;/em&gt; of these approaches is that the set of possible actions is strongly constrained. These domains are therefore 'operationalised' - the hard task of working out what actions are possible has already been solved for the AI by the API, effectively acting as a bottleneck on the complexity of mapping from input to output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A range of alternative game-playing frameworks are listed &lt;a href=&quot;http://cig16.image.ece.ntua.gr/competitions/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and one alternative (which I personally believe is more useful for AGI purposes) is the &lt;a href=&quot;http://atkrye.github.io/IEEE-CIG-Text-Adventurer-Competition/&quot; rel=&quot;nofollow&quot;&gt;Artificial Text Adventurer&lt;/a&gt;, in which (at each turn) agent is presented with natural language input describing the scene and must then output a command in natural language. Disclaimer: I am associated with this competition.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-09T20:35:34.217" LastActivityDate="2016-08-09T20:35:34.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1500" PostTypeId="2" ParentId="1494" CreationDate="2016-08-09T20:13:44.400" Score="2" Body="&lt;p&gt;I would also look at Minimum Edit Distances such as the Levenshtein distance.&#xA;You could use a dynamic programming technique such as the Viterbi Algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have a dictionary to work against, you may want to train with a Markov Chain model using a known &quot;good&quot; text. The Viterbi Algorithm could be used again to solve the model for the text being considered.&lt;/p&gt;&#xA;" OwnerUserId="132" LastActivityDate="2016-08-09T20:13:44.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1501" PostTypeId="1" CreationDate="2016-08-09T20:54:09.720" Score="3" ViewCount="111" Body="&lt;p&gt;As I have been looking at other questions on this site (like &lt;a href=&quot;https://ai.stackexchange.com/questions/60/what-are-the-main-problems-hindering-current-ai-development&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/questions/1376/is-it-ethical-to-implement-self-defence-for-street-walking-ai-robots&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/questions/111/how-would-self-driving-cars-make-ethical-decisions-about-who-to-kill&quot;&gt;this&lt;/a&gt;, and &lt;a href=&quot;https://ai.stackexchange.com/questions/1289/can-we-destroy-artificial-general-intelligence-without-its-consent&quot;&gt;this&lt;/a&gt;), I have been thinking more about the ethical implications of creating these generalized AI systems. It seems that whether or not we &lt;em&gt;can&lt;/em&gt; create it is not rationale enough as to whether or not we &lt;em&gt;should&lt;/em&gt; do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In dealing with the issue of ethics in AI, I wonder what the ethical implications are not just for us, but for the system itself. It seems to extend beyond the usually asked questions on the topic and into unknown territory. Are ethics computable? Can they be implemented programmatically? Can we force an AI system to do something against its &lt;em&gt;&quot;will&quot;&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the creation of AI imply ethically for us as well as the AI?&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-09T21:43:25.247" Title="What are the ethical implications of creating (possibly sentient) AI systems?" Tags="&lt;ethics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2016-08-09T23:27:30.140" ContentLicense="CC BY-SA 3.0" />
  <row Id="1502" PostTypeId="2" ParentId="1501" CreationDate="2016-08-09T21:43:25.247" Score="1" Body="&lt;p&gt;I believe if an AI achieves sentience, it should be treated the same way we are required to treat any other sentient animal. This is belief though, there is no established ethics for AI. But there were no ethics for animals a couple of centuries ago.&lt;/p&gt;&#xA;" OwnerUserId="210" LastActivityDate="2016-08-09T21:43:25.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1503" PostTypeId="2" ParentId="1487" CreationDate="2016-08-09T21:46:31.383" Score="4" Body="&lt;p&gt;Chieko Asakawa (&lt;a href=&quot;https://en.wikipedia.org/wiki/Chieko_Asakawa&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt;, &lt;a href=&quot;https://www.ted.com/talks/chieko_asakawa_how_new_technology_helps_blind_people_explore_the_world?language=en&quot; rel=&quot;nofollow&quot;&gt;TED&lt;/a&gt;, &lt;a href=&quot;http://researcher.watson.ibm.com/researcher/view.php?person=jp-CHIE&quot; rel=&quot;nofollow&quot;&gt;IBM&lt;/a&gt;) is a major researcher in this area, and the linked TED talk is probably a good introduction to the state of the art as of 2015. &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2899509&quot; rel=&quot;nofollow&quot;&gt;Here's&lt;/a&gt; a link to a 2016 paper on a smartphone navigation system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Guide animals perform manipulation tasks as well as identification tasks, and so it's not clear if those could be replaced well at all. (A smartphone that reads a label is a great help, but a dog that knows which bottle to grab and deliver to you is probably a much better help.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-09T21:46:31.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1505" PostTypeId="5" CreationDate="2016-08-09T22:59:37.090" Score="0" Body="&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/General_game_playing&quot; rel=&quot;nofollow&quot;&gt;General game playing&lt;/a&gt; at Wikipedia&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-10T05:29:24.097" LastActivityDate="2016-08-10T05:29:24.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1506" PostTypeId="4" CreationDate="2016-08-09T22:59:37.090" Score="0" Body="General game playing (GGP) is the approach of AI to be able to play more than one game successfully." OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-10T05:29:08.597" LastActivityDate="2016-08-10T05:29:08.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1507" PostTypeId="1" AcceptedAnswerId="1516" CreationDate="2016-08-10T00:55:54.690" Score="27" ViewCount="4203" Body="&lt;p&gt;I believe &lt;em&gt;artificial intelligence&lt;/em&gt; (AI) term is overused nowadays. For example, people see that something is self-moving and they call it AI, even if it's on autopilot (like cars or planes) or there is some simple algorithm behind it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the minimum general requirements so that we can say something is AI?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditorDisplayName="user4639" LastEditDate="2019-06-24T13:07:17.090" LastActivityDate="2019-07-18T19:49:05.607" Title="What are the minimum requirements to call something AI?" Tags="&lt;philosophy&gt;&lt;definitions&gt;" AnswerCount="7" CommentCount="0" FavoriteCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="1508" PostTypeId="1" AcceptedAnswerId="1514" CreationDate="2016-08-10T01:06:07.823" Score="7" ViewCount="6594" Body="&lt;p&gt;I believe normally you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;noreferrer&quot;&gt;genetic programming&lt;/a&gt; for sorting, however I'd like to check whether it's possible using ANN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the unsorted text data from input, which neural network is suitable for doing sorting tasks?&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2019-05-13T15:44:52.717" Title="Which neural network has capabilities of sorting input?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1509" PostTypeId="1" AcceptedAnswerId="1520" CreationDate="2016-08-10T01:14:19.777" Score="2" ViewCount="86" Body="&lt;p&gt;I've read on wiki that &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow&quot;&gt;genetic programming&lt;/a&gt; has '&lt;em&gt;outstanding results&lt;/em&gt;' in cyberterrorism prevention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further more, this &lt;a href=&quot;http://papers.ssrn.com/sol3/papers.cfm?abstract_id=877981&quot; rel=&quot;nofollow&quot;&gt;abstract&lt;/a&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using machine-coded linear genomes and a homologous crossover operator in genetic programming, promising results were achieved in detecting malicious intrusions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I've checked the study, but it's still not clear for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly was this detection achieved from the technical perspective?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-10T09:04:33.507" LastActivityDate="2016-08-10T09:04:33.507" Title="How can genetic programming be used to prevent cyberterrorism?" Tags="&lt;genetic-programming&gt;&lt;cyberterrorism&gt;&lt;security&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1510" PostTypeId="5" CreationDate="2016-08-10T01:17:49.203" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-10T01:17:49.203" LastActivityDate="2016-08-10T01:17:49.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1511" PostTypeId="4" CreationDate="2016-08-10T01:17:49.203" Score="0" Body="For questions related to genetic programming, a subset of evolutionary algorithms (so they are also inspired by natural evolution) that evolve programs or functions." OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-12-28T01:25:25.683" LastActivityDate="2020-12-28T01:25:25.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1512" PostTypeId="2" ParentId="1462" CreationDate="2016-08-10T01:38:29.980" Score="1" Body="&lt;p&gt;Basically a &lt;a href=&quot;https://en.wikipedia.org/wiki/Robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;robot&lt;/a&gt; is a mechanical or virtual artificial agent which exhibit intelligent behavior (&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://waitbutwhy.com/wait-but-who&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tim Urban&lt;/a&gt; on &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Wait_But_Why&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wait But Why&lt;/a&gt;&lt;/em&gt; website wrote the following to clear things up:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;First, stop thinking of robots.&lt;/p&gt;&#xA;&lt;h1&gt;A robot is a container for AI,&lt;/h1&gt;&#xA;&lt;p&gt;sometimes mimicking the human form, sometimes not&lt;/p&gt;&#xA;&lt;p&gt;— but the AI itself &lt;strong&gt;is the computer inside the robot&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;AI is the brain and the robot is its body — if it even has a body.&lt;/h3&gt;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&lt;p&gt;the software and data behind &lt;strong&gt;Siri is AI&lt;/strong&gt;, the woman’s voice we hear is a personification of that AI, and there’s no robot involved at all.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The AI Revolution: The Road to Superintelligence&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-08-10T01:43:58.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1513" PostTypeId="2" ParentId="1461" CreationDate="2016-08-10T01:51:10.200" Score="2" Body="&lt;p&gt;They are &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;virtual artificial agents&lt;/a&gt; which exhibit intelligent behavior (&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://waitbutwhy.com/wait-but-who&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tim Urban&lt;/a&gt; on &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Wait_But_Why&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wait But Why&lt;/a&gt;&lt;/em&gt; website wrote the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The software and data behind &lt;strong&gt;Siri is AI&lt;/strong&gt;, the woman’s voice we hear is a personification of that AI, and there’s no robot involved at all.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: &lt;a href=&quot;http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The AI Revolution: The Road to Superintelligence&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related: &lt;a href=&quot;https://ai.stackexchange.com/q/1462/8&quot;&gt;What is the difference between AI and robots?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-10T01:56:15.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1514" PostTypeId="2" ParentId="1508" CreationDate="2016-08-10T01:53:44.150" Score="3" Body="&lt;p&gt;Even a simple multilayer perceptron can sort input data to some extent, as you can see &lt;a href=&quot;https://github.com/primaryobjects/nnsorting&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://yyue.blogspot.com.br/2015/01/a-brief-overview-of-deep-learning.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, neural networks for sequential data seem more appropriate, as they can handle sequences of variable lengths. It has been done with an &lt;a href=&quot;https://github.com/dmlc/mxnet/tree/master/example/bi-lstm-sort&quot; rel=&quot;nofollow&quot;&gt;LSTM&lt;/a&gt; (Long Short-Term Memory), &lt;a href=&quot;https://arxiv.org/pdf/1602.03218.pdf&quot; rel=&quot;nofollow&quot;&gt;LSTM+HAM&lt;/a&gt; (Hierarchical Attentive Memory) and an &lt;a href=&quot;https://arxiv.org/pdf/1410.5401v2.pdf&quot; rel=&quot;nofollow&quot;&gt;NTM&lt;/a&gt; (Neural Turing Machine).&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="144" LastEditDate="2016-08-10T01:59:52.747" LastActivityDate="2016-08-10T01:59:52.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1515" PostTypeId="1" AcceptedAnswerId="3464" CreationDate="2016-08-10T02:05:29.250" Score="4" ViewCount="503" Body="&lt;p&gt;On Wikipedia, we can read about different type of &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;intelligent agents&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;abstract intelligent agents (AIA),&lt;/li&gt;&#xA;&lt;li&gt;autonomous intelligent agents,&lt;/li&gt;&#xA;&lt;li&gt;virtual intelligent agent (IVA), which I've found on other websites, e.g. &lt;a href=&quot;https://www.techopedia.com/definition/26646/intelligent-virtual-agent-iva&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are the differences between these three to avoid confusion?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For example I've used term &lt;em&gt;virtual artificial agent&lt;/em&gt; &lt;a href=&quot;https://ai.stackexchange.com/a/1512/8&quot;&gt;here&lt;/a&gt; as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Basically a robot is a mechanical or virtual artificial agent which exhibit intelligent behavior (AI).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;so basically I'd like to know where other terms like autonomous or abstract agents can be used and in what context. Can they be all defined under 'virtual' robot definition? How to distinguish these terms?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="7488" LastEditDate="2017-06-07T16:19:39.653" LastActivityDate="2021-03-31T22:27:38.583" Title="What is the difference between abstract, autonomous and virtual intelligent agents?" Tags="&lt;definitions&gt;&lt;intelligent-agent&gt;&lt;comparison&gt;&lt;terminology&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1516" PostTypeId="2" ParentId="1507" CreationDate="2016-08-10T02:33:38.747" Score="27" Body="&lt;p&gt;It's true that the term has become a buzzword, and is now widely used to a point of confusion - however if you look at the definition provided by Stuart Russell and Peter Norvig, they write it as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We define AI as the study of agents that &lt;strong&gt;receive percepts from the environment and perform actions&lt;/strong&gt;. Each such agent implements a function that maps percept sequences to actions, and we cover different ways to represent these functions, such as reactive agents, real-time planners, and decision-theoretic systems. We explain the role of &lt;strong&gt;learning&lt;/strong&gt; as extending the reach of the designer into &lt;strong&gt;unknown environments&lt;/strong&gt;, and we show how that role constrains agent design, favoring explicit &lt;strong&gt;knowledge representation and reasoning&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/9332543518&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach - Stuart Russell and Peter Norvig&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the example you cite, &quot;autopilot for cars/planes&quot;, is actually a (famous) form of AI as it has to &lt;strong&gt;use a form of knowledge representation to deal with unknown environments and circumstances&lt;/strong&gt;. Ultimately, these systems also collect data so that the knowledge representation can be updated to deal with the new inputs that they have found. They do this with &lt;a href=&quot;http://fortune.com/2015/10/16/how-tesla-autopilot-learns/&quot; rel=&quot;nofollow noreferrer&quot;&gt;autopilot for cars&lt;/a&gt; all the time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, directly to your question, for something to be considered as &quot;having AI&quot;, &lt;strong&gt;it needs to be able to deal with unknown environments/circumstances in order to achieve its objective/goal&lt;/strong&gt;, and render knowledge in a manner that provides for new learning/information to be added easily. There are many different types of well defined knowledge representation methods, ranging from the popular &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural net&lt;/a&gt;, through to probabilistic models like &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;bayesian networks (belief networks)&lt;/a&gt; - but fundamentally actions by the system must be derived from whichever representation of knowledge you choose for it to be considered as AI.&lt;/p&gt;&#xA;" OwnerUserId="1441" LastEditorUserId="2444" LastEditDate="2019-07-18T19:49:05.607" LastActivityDate="2019-07-18T19:49:05.607" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1517" PostTypeId="1" AcceptedAnswerId="1524" CreationDate="2016-08-10T02:38:17.940" Score="3" ViewCount="637" Body="&lt;p&gt;On &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; we can read:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Kasparov accused IBM of cheating and demanded a rematch. IBM refused and retired Deep Blue.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What was the accusation and how was Deep Blue allegedly able to cheat?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-11T14:51:20.763" LastActivityDate="2016-08-11T14:51:20.763" Title="How could Deep Blue possibly cheat?" Tags="&lt;chess&gt;&lt;deep-blue&gt;&lt;challenges&gt;&lt;game-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1518" PostTypeId="1" AcceptedAnswerId="1528" CreationDate="2016-08-10T02:54:35.943" Score="2" ViewCount="1753" Body="&lt;p&gt;The Wikipedia page describes &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_control_problem&quot; rel=&quot;nofollow&quot;&gt;AI control problem&lt;/a&gt; in very intricated way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore I would like to better understand it based on some simple explanation, what's going on.&#xA;Basically I don't want any copy &amp;amp; pastes from wiki, because the articles there are written in neutral point of view, in very general way where articles are evolving very slowly, so the definition from there doesn't suit me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this is what is discussed nowadays by government and it's important aspects of AI technology where it leds to.&#xA;I believe this could be a big problem in the near future, so I'm expecting to hear about this from people from much better and more up-to-date point of view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is exactly the AI Control Problem?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="29" LastEditDate="2016-08-12T10:44:18.917" LastActivityDate="2016-08-12T10:44:18.917" Title="What is the Control Problem?" Tags="&lt;definitions&gt;&lt;control-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1520" PostTypeId="2" ParentId="1509" CreationDate="2016-08-10T05:54:42.643" Score="2" Body="&lt;p&gt;They treated it as a classification problem. While it's common to use some variety of Neural Nets (NNs) to build classifiers, Genetic Programming (GP) can also be used for this purpose. In contrast to NN classifiers, GP can use a wider range of operations (e.g. if,while,logical statements,arbitrary mathematical functions etc) to perform the classification than &lt;a href=&quot;https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks&quot;&gt;weighted arithmetic expressions involving an activation function&lt;/a&gt;. Whether or not this is actually of benefit depends on the specific application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, the abstract implies their algorithm is &lt;em&gt;adaptive&lt;/em&gt; (i.e. responds in some fashion to the nature of incoming attacks), which would most easily be achieved by continuing to run the GP program in the background to monitor potential intrusions.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-10T05:54:42.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1522" PostTypeId="2" ParentId="1488" CreationDate="2016-08-10T10:02:41.383" Score="4" Body="&lt;p&gt;Tesla's technology is assistive, as Alexey points out, so this is not a case of an autonomous system (e.g. an AGI) doing some fatal stunt (the product name AutoPilot is famously misleading). Now on why the car assistance led to this tragic accident, there is some information related to AI technologies.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Warning: I cannot find again the source critical to the next paragraph, and reading again pages over pages, I cannot find similar argument in other reports. I still remember vividly the point below, but please keep in mind it may be incorrect. The rest of the answer is weakly related, so I leave it all, with this warning.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An independent report (link needed, I can't find it...) explained that the assistive system was unable to detect the truck due to an exceptionally low contrast (bright sky perceived as white---colour of the truck). The report also said that a human driver would have been unable to make the difference either. In other words, it is possible that car sensors (presumably camera) and the human eye could not have detected an obstacle, and could not have triggered any safety measure. This &lt;a href=&quot;http://www.nytimes.com/interactive/2016/07/01/business/inside-tesla-accident.html&quot; rel=&quot;nofollow&quot;&gt;short graphical explanation&lt;/a&gt; sums up the car sensors: Camera, radar, GPS, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assistive sub-system is based on proprietary AI technologies. We can &lt;em&gt;only speculate&lt;/em&gt; under some hypothesis. _This is not very useful, honestly, except for &lt;strong&gt;illustration purpose&lt;/strong&gt;. Assuming the assistive system relies on ML technologies to learn about obstacles from a video stream (such systems do exist):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It may be that the learning data was not &quot;good enough&quot; to cover the truck scenario.&lt;/li&gt;&#xA;&lt;li&gt;It may be the technology was not powerful enough yet (lack generalization power, or simply too slow).&lt;/li&gt;&#xA;&lt;li&gt;It may be a hardware problem, notably from the sensors: If the &quot;car's eyes&quot; are defective, the &quot;car's brain&quot; (the assistive system) is unable to react properly.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Why those technologies did not work in that case will remain a secret. We can say however that &lt;em&gt;any system&lt;/em&gt;---whether built with AI technology or not---has limits. Beyond these limits, the system reaction is unpredictable: It could stop, reset, shutdown. The difficulty here is to define what a &quot;default behaviour&quot; is. A machine will basically do whatever it is designed to do, so an AI-based system too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could speculate even more on what would happen if the assistive system was really autonomous, the elusive AGI, but that is really not the case here.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2016-08-14T23:54:03.540" LastActivityDate="2016-08-14T23:54:03.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1523" PostTypeId="2" ParentId="1485" CreationDate="2016-08-10T11:24:13.367" Score="4" Body="&lt;p&gt;The underlying abstraction (which is essentially what you'd be using the first network for) is that of reducing the state-space of the raw input via feature extraction/synthesis and/or dimensionality reduction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At present, there are few definite rules for doing this: practice is more a question of 'informed trial and error'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you add some information to your question regarding what has been previously attempted in this area (e.g. on the &#xA;&lt;a href=&quot;https://ai.stackexchange.com/questions/1490/what-are-the-benefits-of-the-vgdl-over-the-ale&quot;&gt;ALE&lt;/a&gt; platform), this it might be possible to offer some more specific advice.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-10T11:24:13.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1524" PostTypeId="2" ParentId="1517" CreationDate="2016-08-10T13:44:28.207" Score="7" Body="&lt;p&gt;The allegation was based on the fact that Deep Blue made a choice that did not yield the immediate (or short term) benefit that was synonymous with systems back then (1997). Computational capability was significantly less powerful then, and Kasparov claimed that only a grand master would have made the decision that the system did - so the deep blue team cheated by having a human perform the move instead of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He asked for a rematch, but IBM did not allow this, which only added to the suspicion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a great article with deep analysis on the specific moves and circumstances - however suffice to say that Kasparov was trying to bait the system into making a decision for a weak pawn, but the system chose otherwise and instead put Kasparov into a compromised position:&#xA;&lt;a href=&quot;https://en.chessbase.com/post/deep-blue-s-cheating-move&quot;&gt;https://en.chessbase.com/post/deep-blue-s-cheating-move&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1441" LastActivityDate="2016-08-10T13:44:28.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1525" PostTypeId="1" CreationDate="2016-08-10T14:10:08.270" Score="17" ViewCount="410" Body="&lt;p&gt;&lt;sub&gt;This is from a closed beta for AI, with this question being posted by user number 47. All credit to them. &lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_machine&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both are recurrent neural networks that can be trained to learn of bit patterns. Then when presented with a partial pattern, the net will retrieve the full complete pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopfield networks have been proven to have a capacity of 0.138 (e.g. approximately 138 bit vectors can be recalled from storage for every 1000 nodes, Hertz 1991).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a Boltzmann machine is stochastic, my understanding is that it would not necessarily always show the same pattern when the energy difference between one stored pattern and another is similar. But because of this stochasticity, maybe it allows for denser pattern storage but without the guarantee that you'll always get the &quot;closest&quot; pattern in terms of energy difference. Would this be true? Or would a Hopfield net be able to store more patterns?&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-23T08:19:28.460" LastActivityDate="2017-10-15T03:19:55.693" Title="Could a Boltzmann machine store more patterns than a Hopfield net?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1526" PostTypeId="2" ParentId="191" CreationDate="2016-08-10T14:28:47.593" Score="4" Body="&lt;p&gt;The well-known 'Eliza' program (Weizenbaum, ~1964) would appear to be the first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eliza was designed to model the emotionally-neutral response of a psychotherapist and this masks some of the weaknesses of its limited underlying pattern-matching mechanisms. &lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-10T14:28:47.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1527" PostTypeId="2" ParentId="1478" CreationDate="2016-08-10T15:16:19.503" Score="3" Body="&lt;p&gt;The 2015 paper entitled &quot;&lt;a href=&quot;https://arxiv.org/pdf/1511.08899.pdf&quot; rel=&quot;nofollow&quot;&gt;Applying deep learning to classify pornographic images and videos&lt;/a&gt;&quot; applied various types of convnets for detecting pornography. The proposed architecture achieved &lt;strong&gt;94.1% accuracy&lt;/strong&gt; on the NPDI dataset, which contains 800 videos (400 porn, 200 non-porn &quot;easy&quot; and 200 non-porn &quot;difficult&quot;). More traditional computer vision methods achieved 90.9% accuracy. The proposed architecture also performs very well regarding the ROC curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There does not seem to exist any works regarding the other aspects of NSFW yet.&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2016-08-10T15:16:19.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1528" PostTypeId="2" ParentId="1518" CreationDate="2016-08-10T15:42:41.603" Score="7" Body="&lt;p&gt;The Control Problem is, in short, the idea that AI will eventually be much better decision-makers than humans. If we don't set things up correctly beforehand, we won't get a chance to fix it afterwards, because AI will have effective control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three main areas of discussion with regards to the Control Problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether or not the problem is urgent. Many AI experts, cognizant of the difficulty of getting simple systems to work effectively today, think that AI able to take control is not urgent, and as detail-minded engineers, they think it will be profoundly difficult to do any useful work today. (Andrew Ng, for example, famously called these sorts of worries like worrying about overpopulation on Mars.) Given radical uncertainty among AI experts as to when this will become an issue, however, this means we can't rule out rapid AI timescales, and should do at least some work in anticipation of those timescales.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether or not the problem is hard. Many people give short, simple, and wrong solutions to the control problem. Probably the most famous is the idea that intelligence and morality are inherently interlinked, and thus a more intelligent machine, &lt;em&gt;by definition&lt;/em&gt;, will be more moral. The Orthogonality Thesis is the claim of the opposite, that intelligence and morality (or, more specifically, goal alignment) are unrelated things.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What foundations we can lay now. There are a bunch of open problems (see, for example, &lt;a href=&quot;https://intelligence.org/technical-agenda/&quot;&gt;MIRI's technical agenda&lt;/a&gt;) that deal with mathematical logic of the sort that would be useful for ensuring robust value alignment, or on how to effectively do value learning (without giving an incentive to distort values), or on how to build value functions and goals such that they are fixable if they turn out to be mistaken. Those look like problems that we can do useful work on now, even without knowing what the actual structure of a future AI will look like.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-10T15:42:41.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1529" PostTypeId="2" ParentId="1515" CreationDate="2016-08-10T15:54:53.790" Score="3" Body="&lt;p&gt;I don't think there are many contexts where there is any really meaningful distinction between these terms.  Even in the WP article you refer to, it is shown that &quot;abstract intelligent agent&quot; and &quot;autonomous intelligent agent&quot; are generally just synonyms for &quot;intelligent agent&quot; but used to highlight certain aspects of intelligent agents in some contexts.   Net-net, I'd say there just isn't any difference there that's going to matter in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Virtual intelligent agent&quot; OTOH, used in the context you used it, suggests the distinction between an IA that's implemented in software only, versus one that has a physical manifestation.   I don't know how useful that distinction is and I haven't seen anybody else make it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, I expect that in almost every possible context, if you just say &quot;Intelligent Agent&quot; with no qualifiers, that's going to be sufficient.   But if there were going to be an exception, I'd say it would be around the term &quot;autonomous&quot; since an agent which is truly autonomous, versus one that needs to operate in a specific, constrained environment, is a distinction that - at least in principle - could be useful. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-10T15:54:53.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1530" PostTypeId="2" ParentId="1479" CreationDate="2016-08-10T16:03:27.817" Score="10" Body="&lt;p&gt;I'm afraid I don't have the specific citations handy, but I have seen/heard quotes by experts like Andrew Ng and Geoffrey Hinton where they clearly say that we do not really understand neural networks.  That is, we understand something of the &lt;em&gt;how&lt;/em&gt; they work (for example, the math behind back propagation) but we don't really understand &lt;em&gt;why&lt;/em&gt; they work.  It's sort of a subtle distinction, but the point is that no, we don't understand the very deepest details of how exactly you go from a bunch of weights, to, say, recognizing a cat playing with a ball.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least in terms of image recognition, the best explanation I've heard is that successive layers of a neural network learn more sophisticated features, composed of the more granular features from earlier levels.  That is to say, the first layer might recognize &quot;edges&quot; or &quot;straight lines&quot;.  The next layer might then learn geometric shapes like &quot;box&quot;, or &quot;triangle&quot;, and then a higher layer might learn &quot;nose&quot; or &quot;eye&quot; based on those earlier features, and then a higher level layer still learns &quot;face&quot; made up from &quot;eye&quot;, &quot;nose&quot;, &quot;jaw&quot;, etc.   But even that, as I understand it, is still hypothetical and/or not understood in complete detail. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-10T16:03:27.817" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1531" PostTypeId="1" AcceptedAnswerId="1532" CreationDate="2016-08-10T19:17:18.410" Score="20" ViewCount="8472" Body="&lt;p&gt;According to &lt;a href=&quot;http://en.wikipedia.org/wiki/Prolog&quot;&gt;Wikipedia&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Prolog is a general-purpose logic programming language associated with artificial intelligence and computational linguistics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is it still used for AI?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;This is based off of a question on the 2014 closed beta. The author had the UID of 330.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="145" LastActivityDate="2016-08-12T21:38:48.343" Title="Is Prolog still used in AI?" Tags="&lt;history&gt;&lt;programming-languages&gt;&lt;prolog&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1532" PostTypeId="2" ParentId="1531" CreationDate="2016-08-11T01:37:54.610" Score="16" Body="&lt;p&gt;Remembering that artificial intelligence has been an academic endeavour for the longest time, Prolog was amongst one of the early languages used as part of the study and implementation of it. It has rarely made its way into large commercial applications, having said that, a famous commercial implementation is in &lt;a href=&quot;http://www.cs.nmsu.edu/ALP/2011/03/natural-language-processing-with-prolog-in-the-ibm-watson-system/&quot; rel=&quot;noreferrer&quot;&gt;Watson, where prolog is used for NLP&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.ed.ac.uk/informatics/&quot; rel=&quot;noreferrer&quot;&gt;University of Edinburgh&lt;/a&gt; contributed to the language and it was sometimes referred to as &quot;Edinburgh Prolog&quot;. It is &lt;a href=&quot;http://www.inf.ed.ac.uk/teaching/courses/lp/&quot; rel=&quot;noreferrer&quot;&gt;still used in academic teachings&lt;/a&gt; there as part of the artificial intelligence course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why Prolog is considered powerful in AI is because the language allows for easy management of recursive methods, and pattern matching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To quote &lt;a href=&quot;http://www-03.ibm.com/innovation/us/watson/research-team/systems.html&quot; rel=&quot;noreferrer&quot;&gt;Adam Lally from the IBM Thomas J. Watson Research Center&lt;/a&gt;, and &lt;a href=&quot;http://www3.cs.stonybrook.edu/~pfodor/&quot; rel=&quot;noreferrer&quot;&gt;Paul Fodor from Stony Brook University&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the Prolog language is very expressive allowing recursive rules to represent reachability in parse trees and the operation of negation-as-failure to check the absence of conditions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1441" LastActivityDate="2016-08-11T01:37:54.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1533" PostTypeId="2" ParentId="92" CreationDate="2016-08-11T03:08:39.273" Score="3" Body="&lt;p&gt;The neural networks can be easily fooled or hacked by adding certain structured noise in image space (&lt;a href=&quot;https://arxiv.org/abs/1312.6199&quot; rel=&quot;nofollow&quot;&gt;Szegedy 2013&lt;/a&gt;, &lt;a href=&quot;http://arxiv.org/abs/1412.1897&quot; rel=&quot;nofollow&quot;&gt;Nguyen 2014&lt;/a&gt;) due to ignoring non-discriminative information in their input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Learning to detect jaguars by matching the unique spots on their fur while ignoring the fact that they have four legs.&lt;sup&gt;&lt;a href=&quot;http://arxiv.org/abs/1506.06579&quot; rel=&quot;nofollow&quot;&gt;2015&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So basically the high confidence prediction in certain models exists due to a '&lt;em&gt;combination of their locally linear nature and high-dimensional input space&lt;/em&gt;'.&lt;sup&gt;&lt;a href=&quot;http://arxiv.org/abs/1412.1897&quot; rel=&quot;nofollow&quot;&gt;2015&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Published as a conference paper at &lt;a href=&quot;http://www.stat.ucla.edu/~ywu/ICLR2015.pdf&quot; rel=&quot;nofollow&quot;&gt;ICLR 2015&lt;/a&gt; (work by Dai) suggest that transferring discriminatively trained parameters to generative models, could be a great area for further improvements.&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-11T03:08:39.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1534" PostTypeId="1" AcceptedAnswerId="1727" CreationDate="2016-08-11T03:29:18.097" Score="6" ViewCount="172" Body="&lt;p&gt;I'm a bit confused with extensive number of different &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_method&quot; rel=&quot;noreferrer&quot;&gt;Monte Carlo methods&lt;/a&gt; such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hybrid_Monte_Carlo&quot; rel=&quot;noreferrer&quot;&gt;Hamiltonian/Hybrid Monte Carlo (HMC)&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_Monte_Carlo_method&quot; rel=&quot;noreferrer&quot;&gt;Dynamic Monte Carlo (DMC)&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo&quot; rel=&quot;noreferrer&quot;&gt;Markov chain Monte Carlo (MCMC)&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Kinetic_Monte_Carlo&quot; rel=&quot;noreferrer&quot;&gt;Kinetic Monte Carlo (KMC)&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_Monte_Carlo_method&quot; rel=&quot;noreferrer&quot;&gt;Dynamic Monte Carlo (DMC)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method&quot; rel=&quot;noreferrer&quot;&gt;Quasi-Monte Carlo (QMC)&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Direct_simulation_Monte_Carlo&quot; rel=&quot;noreferrer&quot;&gt;Direct Simulation Monte Carlo (DSMC)&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;and so on.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I won't ask for the exact differences, but why are all of them called Monte Carlo? What do they all have in common? Can they all be used for AI? E.g. which one can be used for gaming (like Go) or image recognition (resampling)?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-11T13:46:22.443" LastActivityDate="2016-08-24T13:15:48.023" Title="How do I know when to use which Monte Carlo method?" Tags="&lt;gaming&gt;&lt;comparison&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1535" PostTypeId="1" CreationDate="2016-08-11T08:30:40.453" Score="2" ViewCount="335" Body="&lt;p&gt;When it comes to neural networks, it's often only explained what abstract task they do, say for example detect a number in an image. I never understood what's going on under the hood essentially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be a common structure of a directed graph, with values in each node. Some nodes are input nodes. Their values can be set. The values of subsequent nodes are then calculated based on those along the edges of the graph until the values for the output nodes are set, which can be interpreted a result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly is the value of each node determined? I assume that some formula is associated with each node that takes all incoming nodes as input to calculate the value of the node. What formula is used? Is the formula the same throughout the network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I heard that a network has to be trained. I assume that such training would be the process to assign values to coefficients of the formulas used to determine the node values. Is that correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In layman's terms, what are the underlying principles that make a neural network work?&lt;/p&gt;&#xA;" OwnerUserId="1463" LastEditorUserId="8" LastEditDate="2016-08-18T08:40:00.167" LastActivityDate="2018-05-04T09:57:49.327" Title="How exactly is the value of each node determined? Is it the same formula throughout the network?" Tags="&lt;neural-networks&gt;" AnswerCount="3" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1536" PostTypeId="2" ParentId="1535" CreationDate="2016-08-11T09:00:25.247" Score="4" Body="&lt;p&gt;I will overly simplify ANNs in order to point how they work. Examples might not be 100% accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the simplest form, network is trained using the apriori information extracted from the ground truth. This basically means that ANN uses the relation between the input and output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if you are to classify shrubs and trees, one of the input could be height and the other could be the width of the tree. Now, if you have only input and output layers, increasing height means increasing chance for the object to be a tree. Thus, input height would have a positive weight connecting to tree output and a negative weight to shrub output. However, as the plant gets wider, the chance of it being a shrub increases. Taller shrubs are wider than shorter ones. Thus input weight would have positive weight connecting to the shrub output. Finally, the chance of being a tree is not affected by the width and thus will have close to 0 weight between this input and output. This network will effectively work like a linear discriminant classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now instead of assigning weights by hand, you may use a learning algorithm that tries to adjust weights so that the output is correct when the series of input is supplied. Ideally this training algorithm should reach to the conclusion that we have made in the previous example. Most training algorithms are recursive. They supply the inputs multiple times, and in a simple sense, they reward pathways that are correct by increasing their weights and punishes pathways that are causing incorrect answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When hidden layers are used in a system, they would be able to correlate input on higher degrees. Thus, as the number of layers get higher, ANN learns the input set much better. However, this does not mean it gets better. If the ANN over fits the input set, it would be affected from the random noise that is in the dataset. This problem is generally referred as memorization. There are learning algorithms that try to minimize memorization and maximize generalization ability. But ultimately, the number of training samples should be high enough so that ANN cannot overfit to the data.&lt;/p&gt;&#xA;" OwnerUserId="210" LastActivityDate="2016-08-11T09:00:25.247" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1537" PostTypeId="1" AcceptedAnswerId="1569" CreationDate="2016-08-11T09:02:52.250" Score="0" ViewCount="164" Body="&lt;p&gt;Ideally I'd like to watch movie which is deep dreamed in real-time. Most algorithms which I know are too slow or not designed for real-time processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I'm bored with some movie which I've watched thousands of time and I'd like to add some &quot;dreaming&quot; to it which is real-time filter which takes input frames, then it's processing and enhances the images through artificial neural network to achieve doodled output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doesn't have to be exactly &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepDream&quot; rel=&quot;nofollow&quot;&gt;DeepDream&lt;/a&gt; or hallucinogenic technique (which could be too much to watch for 2h), but with any similar ANN algorithm. I'm more interested into achieving desired real-time use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of techniques can achieve such efficiency?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-11T20:40:25.737" LastActivityDate="2016-08-11T20:49:24.500" Title="Which techniques can achieve neural doodle in real-time?" Tags="&lt;research&gt;&lt;algorithm&gt;&lt;real-time&gt;&lt;neural-doodle&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1539" PostTypeId="1" AcceptedAnswerId="1545" CreationDate="2016-08-11T09:39:32.893" Score="9" ViewCount="150" Body="&lt;p&gt;How does employing evolutionary algorithms to design and train artificial neural networks have advantages over using the conventional backpropagation algorithms?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-17T11:49:00.693" LastActivityDate="2016-08-17T11:49:00.693" Title="How do evolutionary algorithms have advantages over the conventional backpropagation methods?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;backpropagation&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1540" PostTypeId="1" AcceptedAnswerId="1562" CreationDate="2016-08-11T10:41:10.593" Score="2" ViewCount="779" Body="&lt;p&gt;Are there any machine learning techniques (such as deep learning or evolutionary algorithms) to detect coding standard violations? Which one would be more suitable?&lt;/p&gt;&#xA;&lt;p&gt;I don't have any specific programming language in mind, but something similar to &lt;a href=&quot;http://pear.php.net/package/PHP_CodeSniffer&quot; rel=&quot;nofollow noreferrer&quot;&gt;PHP_CodeSniffer&lt;/a&gt; (following &lt;a href=&quot;https://www.drupal.org/coding-standards&quot; rel=&quot;nofollow noreferrer&quot;&gt;these standards&lt;/a&gt;), but instead of using hardcoded rules, the algorithm should learn good techniques, but I'm not sure based on what training data. How would you approach the training session, any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-12-05T15:37:36.623" LastActivityDate="2020-12-05T15:38:13.337" Title="Are there any machine learning techniques to detect coding standard violations?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;evolutionary-algorithms&gt;&lt;computer-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1541" PostTypeId="1" AcceptedAnswerId="1548" CreationDate="2016-08-11T13:21:53.280" Score="9" ViewCount="325" Body="&lt;p&gt;Genetic Algorithms has come to my attention recently when trying to correct/improve computer opponents for turn-based strategy computer games.&lt;/p&gt;&#xA;&lt;p&gt;I implemented a simple Genetic Algorithm that didn't use any cross-over, just some random mutation. It seemed to work in this case, and so I started thinking:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why is cross-over a part of genetic algorithms? Wouldn't mutation be enough?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sub&gt;This is from a data dump on an old AI site. The asker had the UID of 7. &lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2020-12-08T11:21:56.810" LastActivityDate="2020-12-08T11:21:56.810" Title="Why is cross-over a part of genetic algorithms?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;crossover-operators&gt;&lt;genetic-operators&gt;&lt;mutation-operators&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1542" PostTypeId="5" CreationDate="2016-08-11T13:48:04.250" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-11T13:48:04.250" LastActivityDate="2016-08-11T13:48:04.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1543" PostTypeId="4" CreationDate="2016-08-11T13:48:04.250" Score="0" Body="For questions about AI in relation to video games in particular.  (Currently this tag is also used for combinatorial games, such as board and card games.)  " OwnerUserId="145" LastEditorUserId="1671" LastEditDate="2017-09-13T21:04:41.913" LastActivityDate="2017-09-13T21:04:41.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1544" PostTypeId="1" CreationDate="2016-08-11T14:00:46.070" Score="8" ViewCount="242" Body="&lt;p&gt;While thinking about AI, this question came into my mind. Could curiosity help in developing a true AI? According to this &lt;a href=&quot;http://psychologia.co/creativity-test/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt; (for testing creativity):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Curiosity in this context refers to persistent desire to learn and discover new things and ideas. A curious person&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;always looks for new and original ways of thinking,&lt;/li&gt;&#xA;&lt;li&gt;likes to learn,&lt;/li&gt;&#xA;&lt;li&gt;searches for alternative solutions even when traditional solutions are present and available,&lt;/li&gt;&#xA;&lt;li&gt;enjoys reading books and watching documentaries,&lt;/li&gt;&#xA;&lt;li&gt;wants to know how things work inside out.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Let's take &lt;a href=&quot;https://www.clarifai.com/demo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clarifai&lt;/a&gt;, an image/video classification startup, which can classify images and video with the best accuracy (according to them). If I understand correctly, they trained their deep learning system using millions of images with supervised learning. In the same algorithm, what would happen if we somehow added a &amp;quot;curiosity factor&amp;quot; when the AI has difficulty in classifying an image or its objects? It would ask a human for help, just like a curious child.&lt;/p&gt;&#xA;&lt;p&gt;Curiosity makes a human being learn new things and also helps to generate new original ideas. Could the addition of curiosity change Clarifai into a true AI?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2021-01-26T01:33:01.403" LastActivityDate="2021-01-26T01:33:01.403" Title="Could curiosity improve artificial intelligence?" Tags="&lt;agi&gt;&lt;human-inspired&gt;&lt;artificial-curiosity&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1545" PostTypeId="2" ParentId="1539" CreationDate="2016-08-11T14:04:43.640" Score="7" Body="&lt;p&gt;Unlike backpropagation, evolutionary algorithms do not require the objective function to be differential with respect to the parameters you aim to optimize. As a result, you can optimize &quot;more things&quot; in the network, such as activation functions or number of layers, which wouldn't be possible in the standard backpropagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another advantage is that by defining the mutation and crossover functions, you can influence how the parameter search space should be explored.&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="4" LastEditDate="2016-08-11T16:50:04.720" LastActivityDate="2016-08-11T16:50:04.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1546" PostTypeId="2" ParentId="1541" CreationDate="2016-08-11T14:06:30.360" Score="5" Body="&lt;p&gt;Crossover allows to combine two parents (vs. mutation, which only uses one parent). In some cases, it is useful (e.g., if you train a FPS bot, if one parent is good at shooting and another parent is good at moving, it makes sense to combine them). In some other cases, it is not.&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="4" LastEditDate="2016-08-11T14:19:54.063" LastActivityDate="2016-08-11T14:19:54.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1547" PostTypeId="2" ParentId="1544" CreationDate="2016-08-11T14:13:16.937" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;when the AI has difficulty in classifying a image or its objects it should ask a human for help just like a curious child&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's called &lt;a href=&quot;https://en.wikipedia.org/wiki/Active_learning_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;active learning&lt;/a&gt;, it's already used quite often.&lt;/p&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-08-11T14:13:16.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1548" PostTypeId="2" ParentId="1541" CreationDate="2016-08-11T14:40:33.810" Score="7" Body="&lt;p&gt;Mutation is usually defined to be a &lt;em&gt;global&lt;/em&gt; operator, i.e. iterated mutation is (eventually) capable of reaching every point in the vector space defined by the geneome. In that sense, mutation alone is certainly 'enough'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the motivation for crossover - from &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics/&quot; rel=&quot;noreferrer&quot;&gt;Essentials of Metaheuristics&lt;/a&gt;, p42:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Crossover was originally based on the premise that highly fit individuals often share certain traits, called &lt;em&gt;building blocks&lt;/em&gt;, in common.&#xA;  For example, in the boolean individual 10110101, perhaps&#xA;  ***101*1 might be a building block &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(where * means &quot;either 0 or 1&quot;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the idea is that crossover works by spreading building blocks quickly throughout the population.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Crossover methods also assume that there is some degree of linkage between genes on the chromosome: that is, settings for certain genes in groups are strongly correlated to fitness improvement. For example, genes A and B might contribute to fitness only when they’re both set to 1: if either is set to 0, then the fact that the other is set to 1 doesn’t do anything.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also note that &lt;em&gt;crossover is not a global operator&lt;/em&gt;. If the only operator is crossover then (also from p42):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Eventually the population will converge, and often (unfortunately) prematurely&#xA;  converge, to copies of the same individual. At this stage there’s no escape: when an individual crosses over with itself, nothing new is generated.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For this reason, crossover is generally used together with some global mutation operator.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-11T15:57:27.610" LastActivityDate="2016-08-11T15:57:27.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1549" PostTypeId="5" CreationDate="2016-08-11T14:44:00.860" Score="0" Body="&lt;p&gt;Throughout the development of AI, game theory has been of central interest, since decisioning in business, finance, economic development and stabilization, industry, geopolitics, and personal life can be framed as a sequence of actions that may or may not drive some defined improvement in the ongoing or final outcome for some defined set of entities. These are a few examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Driving improved treatment processes for the ill&lt;/li&gt;&#xA;&lt;li&gt;Driving a winning status at the end of a board game&lt;/li&gt;&#xA;&lt;li&gt;Controlling a robot making a repair to a nuclear reactor&lt;/li&gt;&#xA;&lt;li&gt;Managing an investment portfolio&lt;/li&gt;&#xA;&lt;li&gt;Modifying controllable parameters of an economy to incentivize trade and employment&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="4302" LastEditDate="2018-12-11T23:25:30.153" LastActivityDate="2018-12-11T23:25:30.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1550" PostTypeId="4" CreationDate="2016-08-11T14:44:00.860" Score="0" Body="For questions regarding the use of the mathematical theory of games (Von Neumann, Morgenstern, Nash etc) in AI. For questions about the use of AI in game design and gaming, use [gaming]." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-12-11T23:25:30.153" LastActivityDate="2018-12-11T23:25:30.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1551" PostTypeId="2" ParentId="1541" CreationDate="2016-08-11T14:45:52.773" Score="4" Body="&lt;p&gt;When thinking about crossover its important to think about the fitness landscape. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a hypothetical scenario where we are applying a genetic algorithm to find a solution that performs well at 2 tasks. This could be from Franck's example (moving and shooting) for an AI, or perhaps it could be predicted 2 outputs in a genetic machine learning scenario, but really most scenarios where GAs are applied are synonymous (even at solving a single task, there may be different aspects of the task to be addressed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we had an individual, 1, that was performing reasonably well at both tasks, and we found a series of mutations which produced 2 new individuals, 2 and 3, which performed better than Individual 1 at tasks 1 and 2 respectively. Now while both of these are improvements, ideally we want to find a generally good solution, so we want to combine the features that we have been found to be beneficial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where crossover comes in; by combining the genomes of Individuals 2 and 3, we may find some new individual which produces a mixture of their performances. While it is possible that such an individual could be produced by a series of mutations applied to Individual 2 or Individual 3, the landscape may simply not suit this (there may be no favorable mutations in that direction, for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bsVBEm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bsVBEm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are partially right therefore; it may sometimes be the case that the benefits of crossover could be replicated with a series of mutations. Sometimes this may not be the case and crossover may smooth the fitness landscape of your GA, speeding up optimization and helping your GA escape local optima. &lt;/p&gt;&#xA;" OwnerUserId="1467" LastEditorUserId="8" LastEditDate="2016-08-11T15:38:11.613" LastActivityDate="2016-08-11T15:38:11.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1552" PostTypeId="2" ParentId="4" CreationDate="2016-08-11T14:50:04.153" Score="7" Body="&lt;p&gt;For a more intelligent approach than random or exhaustive searches, you could try a genetic algorithm such as NEAT &lt;a href=&quot;http://nn.cs.utexas.edu/?neat&quot;&gt;http://nn.cs.utexas.edu/?neat&lt;/a&gt;. However, this has no guarantee to find a global optima, it is simply an optimization algorithm based on performance and is therefore vulnerable to getting stuck in a local optima. &lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-08-11T14:50:04.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1553" PostTypeId="2" ParentId="1544" CreationDate="2016-08-11T14:59:11.393" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does this addition of curosity changes clarifai into a true AI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As per my answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/1420/how-close-are-we-to-creating-ex-machina&quot;&gt;this question&lt;/a&gt;, we don't know what the ingredients for a 'true AI' are. Via the Turing Test and its variants, the best we can do is &quot;know one when we see one&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curiosity certainly appears &lt;em&gt;necessary&lt;/em&gt; for intelligence, though it doesn't seem &lt;em&gt;sufficient&lt;/em&gt; - a lemming-like creature curious to see what's at the bottom of a steep cliff might not survive long enough to learn caution, even if it had the learning mechanisms to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some work by Schmidhuber on &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.3978&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Curiousity&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.pyoudeyer.com/active-learning-and-artificial-curiosity-in-robots/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pierre-Yves Oudeyer&lt;/a&gt; has also done quite a lot of work on this and Active Learning/Intrinsic motivation.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-11T14:59:11.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1554" PostTypeId="5" CreationDate="2016-08-11T17:33:03.950" Score="0" Body="&lt;p&gt;Artificial networks include but are not limited to these and their variants.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multi-layer perceptrons (MLPs), a.k.a. feed forward networks&lt;/li&gt;&#xA;&lt;li&gt;Convolutional neural networks (CNNs)&lt;/li&gt;&#xA;&lt;li&gt;Recurrent neural networks (RNNs)&lt;/li&gt;&#xA;&lt;li&gt;Long short term memory (LSTM and B-LSTM) networks, types of RNNs&lt;/li&gt;&#xA;&lt;li&gt;Gated recurent unit (GRU) networks, type of RNN&lt;/li&gt;&#xA;&lt;li&gt;Recursive neural networks, an RNN&lt;/li&gt;&#xA;&lt;li&gt;Attention designs&lt;/li&gt;&#xA;&lt;li&gt;Spiking networks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are often are called artificial neural networks (ANNs) because they are, in part, inspired by biological neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most basic neural networks are used to estimate or approximate functions more complex than a first degree polynomial.  In more sophisticated cases, they can be used in larger topologies to generate, analyze, or transform images, text, audio, and video.&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2019-01-23T17:22:33.940" LastActivityDate="2019-01-23T17:22:33.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1555" PostTypeId="4" CreationDate="2016-08-11T17:33:03.950" Score="0" Body="For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-12-29T11:15:28.680" LastActivityDate="2018-12-29T11:15:28.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1556" PostTypeId="5" CreationDate="2016-08-11T17:34:49.537" Score="0" Body="&lt;p&gt;Image recognition, in the context of machine vision, is the ability of software to identify objects, places, people, writing and actions in images. Computers can use machine vision technologies in combination with a camera and artificial intelligence software to achieve image recognition.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-15T21:39:13.530" LastActivityDate="2018-03-15T21:39:13.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1557" PostTypeId="4" CreationDate="2016-08-11T17:34:49.537" Score="0" Body="For questions related to image recognition in the context of AI." OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2019-08-03T16:13:40.413" LastActivityDate="2019-08-03T16:13:40.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1558" PostTypeId="5" CreationDate="2016-08-11T17:52:48.283" Score="0" Body="" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-18T03:15:34.237" LastActivityDate="2018-10-18T03:15:34.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1559" PostTypeId="4" CreationDate="2016-08-11T17:52:48.283" Score="0" Body="For questions relating to self-driving vehicles, their AI related requirements, the prioritization of those requirements, AI designs for AVs, plotting courses, changing plans due to conditions, algorithms, control theory pertaining to driving, AI in instrumentation, computer vision on streets, pattern recognition, collision avoidance, risk management in real time, breaking, turning, starting and stopping, and maximization of vehicle durability." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-10-18T03:16:04.803" LastActivityDate="2018-10-18T03:16:04.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1560" PostTypeId="1" AcceptedAnswerId="1577" CreationDate="2016-08-11T18:06:34.053" Score="3" ViewCount="478" Body="&lt;p&gt;Based on the article &lt;a href=&quot;http://www.dailymail.co.uk/sciencetech/article-3677950/Google-s-self-driving-cars-spot-cyclists-Sensors-read-hand-signals-predict-riders-behavior.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's self-driving cars can now spot cyclists: Sensors can read hand signals and predict rider's behaviour&lt;/a&gt;, Google's self-driving cars can spot cyclists, cars, road signs, markings, traffic lights, and pedestrians.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly does it identify pedestrians? Is it based on face recognition, shape, size, distance, infrared signature?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-21T21:43:59.547" LastActivityDate="2019-06-21T21:43:59.547" Title="How does Google's self-driving car identify pedestrians?" Tags="&lt;object-recognition&gt;&lt;autonomous-vehicles&gt;&lt;google&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1561" PostTypeId="1" AcceptedAnswerId="1578" CreationDate="2016-08-11T18:18:27.973" Score="19" ViewCount="731" Body="&lt;p&gt;In &lt;a href=&quot;https://www.technologyreview.com/s/530276/hidden-obstacles-for-googles-self-driving-cars/&quot; rel=&quot;noreferrer&quot;&gt;Hidden Obstacles for Google’s Self-Driving Cars&lt;/a&gt; article we can read that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Google’s cars can detect and respond to stop signs that aren’t on its map, a feature that was introduced to deal with temporary signs used at construction sites.&lt;/p&gt;&#xA;&lt;p&gt;Google says that its cars can identify almost all unmapped stop signs, and would remain safe if they miss a sign because the vehicles are always looking out for traffic, pedestrians and other obstacles.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What would happen if a car spotted somebody in front of it (but not on the collision path) wearing a T-shirt that has a stop sign printed on it. Would it react and stop the car?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-06-20T15:10:15.107" Title="Would Google's self-driving-car stop when it sees somebody with a T-shirt with a stop sign printed on it?" Tags="&lt;object-recognition&gt;&lt;decision-theory&gt;&lt;autonomous-vehicles&gt;&lt;google&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1562" PostTypeId="2" ParentId="1540" CreationDate="2016-08-11T18:38:13.963" Score="4" Body="&lt;p&gt;If the system claims that a piece of code has violated standards, then to be useful to the programmer, it really needs to provide more information than just a 'yes/no' classifier: you need some form of explanation about &lt;em&gt;why&lt;/em&gt; it is claimed to be wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly ANNs aren't much use for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If&lt;/em&gt; I were tackling such a problem (and my suspicion is that a lot of effort could be spent trying and failing to reproduce coding standards which are already well-understood), then my inclination would be to use a more explicitly rule-based representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibilities include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Genetic Programming&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Learning Classifier Systems &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Decision Trees&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The ever-useful &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Essentials of Metaheuristics&quot;&lt;/a&gt; has a whole section on the evolution of rulesets. Obviously, nothing prevents you from initializing the evolutionary process with rules known to be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I point out &lt;a href=&quot;https://ai.stackexchange.com/questions/1420/how-close-are-we-to-creating-ex-machina&quot;&gt;here&lt;/a&gt;, with our current AI algorithms, the success of an  approach is very sensitive to human expertise/effort in feature selection/preprocessing, choice of training set etc, so creative experiment with this is vital.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training set: how about two sets of negative and positive examples, consisting of (features extracted from) bad code and from a refactored version (respectively)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One elementary choice of features would be to apply a bunch of code complexity metrics and have the learning algorithm combine those. The plus side of working in such a numeric domain is that the learning algorithm might readily find a gradient to exploit. The downside is that the rules (which are then likely of the form &lt;code&gt;if mcabe &amp;gt; 2.8&lt;/code&gt; etc) are still not as informative as might be desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more complex rules (e.g. requiring &lt;code&gt;if elseif else&lt;/code&gt;) you may want to extract your features from the abstract syntax tree. You could in principle use the entire tree but to my knowledge &lt;a href=&quot;http://jmlr.csail.mit.edu/papers/volume11/vishwanathan10a/vishwanathan10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ML on graph and tree structures&lt;/a&gt; is still in relative infancy.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-11T18:52:07.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1563" PostTypeId="5" CreationDate="2016-08-11T18:53:50.527" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-11T18:53:50.527" LastActivityDate="2016-08-11T18:53:50.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1564" PostTypeId="4" CreationDate="2016-08-11T18:53:50.527" Score="0" Body="For questions relating to AI's programming computers. NOT FOR THE PROGRAMMING OF THE AI'S THEMSELVES." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-11T19:44:14.377" LastActivityDate="2016-08-11T19:44:14.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1565" PostTypeId="5" CreationDate="2016-08-11T19:18:28.780" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-11T19:18:28.780" LastActivityDate="2016-08-11T19:18:28.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1566" PostTypeId="4" CreationDate="2016-08-11T19:18:28.780" Score="0" Body="For asking about an aspect of the history of AI." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-12T06:40:13.047" LastActivityDate="2016-08-12T06:40:13.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1567" PostTypeId="1" CreationDate="2016-08-11T19:56:31.223" Score="4" ViewCount="127" Body="&lt;p&gt;After self-driving cars are perfected to the degree that accident statistics are lower for them than human driven cars, will they replace cars with human drivers?  Will there eventually only be self-driving cars?&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-11-06T17:04:10.237" LastActivityDate="2018-11-06T17:04:10.237" Title="Once self-driving cars are perfected, will they replace manually driven cars?" Tags="&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="6" ClosedDate="2016-08-12T06:39:38.000" ContentLicense="CC BY-SA 4.0" />
  <row Id="1568" PostTypeId="1" AcceptedAnswerId="1584" CreationDate="2016-08-11T20:11:33.013" Score="7" ViewCount="2749" Body="&lt;p&gt;Significant AI vs human board game matches include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;chess&lt;/strong&gt;: &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)#Deep_Blue_versus_Kasparov&quot; rel=&quot;noreferrer&quot;&gt;Deep Blue vs Kasparov&lt;/a&gt; in 1996,&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Go&lt;/strong&gt;: &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol&quot; rel=&quot;noreferrer&quot;&gt;DeepMind AlphaGo vs Lee Sedol&lt;/a&gt; in 2016,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which demonstrated that AI challenged and defeated professional players.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there known board games left where a human can still win against an AI? I mean based on the final outcome of authoritative famous matches, where there is still same board game where AI cannot beat a world champion of that game.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-11T20:15:58.067" LastActivityDate="2021-03-31T22:22:21.160" Title="Is there any board game where a human can still beat an AI?" Tags="&lt;history&gt;&lt;challenges&gt;&lt;game-theory&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1569" PostTypeId="2" ParentId="1537" CreationDate="2016-08-11T20:39:12.370" Score="1" Body="&lt;p&gt;Most of the algorithms (based on &lt;a href=&quot;http://arxiv.org/abs/1603.01768&quot; rel=&quot;nofollow&quot;&gt;image synthesis and style transfer&lt;/a&gt;, e.g. &lt;a href=&quot;https://github.com/alexjc/neural-doodle&quot; rel=&quot;nofollow&quot;&gt;neural-doodle&lt;/a&gt;) haven't been proven to be highly effective in terms of real-time image processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the following studies discusses such algorithms for real-time texture synthesis:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1603.03417&quot; rel=&quot;nofollow&quot;&gt;Feed-forward Synthesis of Textures and Stylized Images&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach is to move the computational burden to a learning stage, making trained network (&lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow&quot;&gt;CNN&lt;/a&gt;) light-weight and compact in order to generate multiple samples of the same texture. This can generate textures as good as comparable to &lt;a href=&quot;http://arxiv.org/abs/1505.07376&quot; rel=&quot;nofollow&quot;&gt;Gatys~et~al&lt;/a&gt;, but significantly faster.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1603.08155&quot; rel=&quot;nofollow&quot;&gt;Perceptual Losses for Real-Time Style Transfer and Super-Resolution&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method uses parallel work which can generate high-quality images by defining and optimizing loss functions based on high-level features extracted from pretrained networks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1604.04382&quot; rel=&quot;nofollow&quot;&gt;Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This uses precomputed feed-forward networks that captures the feature statistics of &lt;a href=&quot;http://www.irisa.fr/vista/Papers/2008_LNLA_Pecot.pdf&quot; rel=&quot;nofollow&quot;&gt;Markovian patches&lt;/a&gt; in order to generate outputs of arbitrary dimensions. This can be applied to texture synthesis, style transfer and video stylization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: Above list suggested on &lt;a href=&quot;https://github.com/alexjc/neural-doodle&quot; rel=&quot;nofollow&quot;&gt;neural-doodle&lt;/a&gt; project.&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-11T20:49:24.500" LastActivityDate="2016-08-11T20:49:24.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1570" PostTypeId="1" CreationDate="2016-08-11T21:05:07.427" Score="6" ViewCount="114" Body="&lt;p&gt;I'm trying to teach an AI different pattern of tic tac toe to recognize wether a given pattern represents a win or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it's not learning to recognize them correctly and I think may way of representing/encoding the game into vectors is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I choose a way that is easy for an human (me, in particular!) to make sense of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;training_data = np.array([[0,0,0,&#xA;                           0,0,0,&#xA;                           0,0,0],&#xA;                          [0,0,1,&#xA;                           0,1,0,&#xA;                           0,0,1],&#xA;                          [0,0,1,&#xA;                           0,1,0,&#xA;                           1,0,0],&#xA;                          [0,1,0,&#xA;                           0,1,0,&#xA;                           0,1,0]], &quot;float32&quot;)&#xA;target_data = np.array([[0],[0],[1],[1]], &quot;float32&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This basically just use an array of length 9 to represent a 3 x 3 board. The first three items represent the first row, the next three the second row and so on. The line breaks should make it obvious I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The target data then maps the first two game states to &quot;no wins&quot; and the last two game states to &quot;wins&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I wanted to create some validation data that is slightly different to see if it generalizes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;validation_data = np.array([[0,0,0,&#xA;                             0,0,0,&#xA;                             0,0,0],&#xA;                            [1,0,0,&#xA;                             0,1,0,&#xA;                             1,0,0],&#xA;                            [1,0,0,&#xA;                             0,1,0,&#xA;                             0,0,1],&#xA;                            [0,0,1,&#xA;                             0,0,1,&#xA;                             0,0,1]], &quot;float32&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously, again the last two game states should be &quot;wins&quot; whereas the first two should not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to play with the number of neurons and learning rate but no matter what I try, my output looks pretty of. E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 0.01207292]&#xA; [ 0.98913926]&#xA; [ 0.00925775]&#xA; [ 0.00577191]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tend to think it's the way how I represent the game state that may be wrong but actually I have no idea :D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me out here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the entire code that I use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from keras.models import Sequential&#xA;from keras.layers.core import Activation, Dense&#xA;from keras.optimizers import SGD&#xA;&#xA;training_data = np.array([[0,0,0,&#xA;                           0,0,0,&#xA;                           0,0,0],&#xA;                          [0,0,1,&#xA;                           0,1,0,&#xA;                           0,0,1],&#xA;                          [0,0,1,&#xA;                           0,1,0,&#xA;                           1,0,0],&#xA;                          [0,1,0,&#xA;                           0,1,0,&#xA;                           0,1,0]], &quot;float32&quot;)&#xA;&#xA;target_data = np.array([[0],[0],[1],[1]], &quot;float32&quot;)&#xA;&#xA;validation_data = np.array([[0,0,0,&#xA;                             0,0,0,&#xA;                             0,0,0],&#xA;                            [1,0,0,&#xA;                             0,1,0,&#xA;                             1,0,0],&#xA;                            [1,0,0,&#xA;                             0,1,0,&#xA;                             0,0,1],&#xA;                            [0,0,1,&#xA;                             0,0,1,&#xA;                             0,0,1]], &quot;float32&quot;)&#xA;&#xA;model = Sequential()&#xA;model.add(Dense(2, input_dim=9, activation='sigmoid'))&#xA;model.add(Dense(1, activation='sigmoid'))&#xA;&#xA;sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)&#xA;model.compile(loss='mean_squared_error', optimizer=sgd)&#xA;&#xA;history = model.fit(training_data, target_data, nb_epoch=10000, batch_size=4, verbose=0)&#xA;&#xA;print(model.predict(validation_data))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1334" LastEditorUserId="8" LastEditDate="2016-08-11T21:51:38.780" LastActivityDate="2016-08-11T21:51:38.780" Title="Why does my NN not classify these tic tac toe pattern correctly?" Tags="&lt;classification&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="9" ClosedDate="2016-08-11T22:37:41.383" ContentLicense="CC BY-SA 3.0" />
  <row Id="1571" PostTypeId="2" ParentId="1568" CreationDate="2016-08-11T21:38:07.100" Score="1" Body="&lt;p&gt;Artificially intelligent computer programs should be able to be at the same level or beat humans at every game that we play.  This is because games follow rules that are scriptable, and &lt;a href=&quot;http://www.aaai.org/ojs/index.php/aimagazine/article/view/2310&quot; rel=&quot;nofollow&quot;&gt;artificial intelligence&lt;/a&gt; is designed to focus on one specific game and learn from its failures.  The difference between humans and artificial intelligence is that artificial intelligence focuses on one specific task like learning to master Go while our brain is dedicated to mastering multiple tasks like...living.  Even Arimaa, a game designed to be difficult for artificially intelligent systems was beaten by a bot called Sharp: &lt;a href=&quot;https://en.wikipedia.org/wiki/Arimaa&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Arimaa&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2016-08-11T21:38:07.100" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1572" PostTypeId="5" CreationDate="2016-08-11T21:53:49.320" Score="0" Body="&lt;p&gt;See: &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow&quot;&gt;Keras Documentation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-12T15:18:31.983" LastActivityDate="2016-08-12T15:18:31.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1573" PostTypeId="4" CreationDate="2016-08-11T21:53:49.320" Score="0" Body="For questions related to Keras, the modular neural networks library written in Python. However, note that programming questions are off-topic here." OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-08-21T19:00:57.750" LastActivityDate="2020-08-21T19:00:57.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1574" PostTypeId="2" ParentId="1568" CreationDate="2016-08-11T22:48:53.950" Score="7" Body="&lt;p&gt;Not all games (or even board games) are computationally algorithmic. Even the least skilled player is likely to trounce the hottest pattern-matching algorithm in a game of &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Pictionary&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pictionary&lt;/a&gt;&lt;/strong&gt; (for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RDIuC.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to say that the movement of pieces upon successful completion of a task is only ancelary to the object of the game, than your answer will be largely self-selecting. A sufficiently sophisticated algorithm will brute force a computational problem better than human intuition&amp;hellip; &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;eventually.&lt;/strong&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2016-08-12T14:22:43.490" LastActivityDate="2016-08-12T14:22:43.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1575" PostTypeId="2" ParentId="111" CreationDate="2016-08-11T23:08:24.670" Score="16" Body="&lt;p&gt;In the real world, decisions will be made based on the law, and &lt;a href=&quot;https://law.stackexchange.com/questions/1639/what-is-the-legal-take-on-the-trolley-problem&quot;&gt;as noted over on Law.SE&lt;/a&gt;, the law generally favors inaction over action. &lt;/p&gt;&#xA;" OwnerUserId="1414" LastEditorUserId="-1" LastEditDate="2017-04-13T13:00:26.773" LastActivityDate="2016-08-11T23:08:24.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1576" PostTypeId="2" ParentId="1567" CreationDate="2016-08-12T01:42:05.757" Score="3" Body="&lt;p&gt;It likely to be happen, because it's more convenient that way. In general people, organizations and government are always keen to make things more efficient by standarizing things (computers, technology, law, science, etc.) in order to make it manageable and predictable to reduce the time and minimalize the risk of the same mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole world now moves into technological advancement where automation of everything is where we are going, so we can manage complexities in more reliable way, so we can focus on much bigger picture. This includes technology such as mobiles, computers, UAV (delivery drones), robots and now self-driving cars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pros of that change would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To have safer streets by introducing autonomous cars on the road.&lt;/li&gt;&#xA;&lt;li&gt;To have fewer drunk, tired, drugged or crazy drivers.&lt;/li&gt;&#xA;&lt;li&gt;To avoid poor weather conditions.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To reduce &lt;a href=&quot;https://en.wikipedia.org/wiki/Braking_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;braking distances&lt;/a&gt; by dropping driver's reaction time and predicting dangerous situations much earlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cyberphysics.co.uk/topics/forces/stopping_distance.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cz9yP.png&quot; alt=&quot;Typical stopping distance&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: &lt;a href=&quot;http://www.cyberphysics.co.uk/topics/forces/stopping_distance.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cyber Physics&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reducing car deaths and costs of GNP.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An estimated 1.3 million people die on the world's roads every year with around 50 million injured or disabled by accidents, with accidents costing countries up to four per cent of their Gross National Product (GNP) yearly. - &lt;a href=&quot;http://www.un.org/apps/news/story.asp?NewsID=36823&quot; rel=&quot;nofollow noreferrer&quot;&gt;UN News Centre&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To have central point of safety improvements, you cannot change people, but you can fix the known safety issue on global scale.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;To introduce global standards from the central point (e.g. new law to which manufactures needs to apply).&lt;/li&gt;&#xA;&lt;li&gt;To increase &lt;a href=&quot;http://www.slideshare.net/sbishop2/p22-car-design-safety&quot; rel=&quot;nofollow noreferrer&quot;&gt;car safety&lt;/a&gt; in general on larger scale.&lt;/li&gt;&#xA;&lt;li&gt;And so on.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Why we need the 'only self-driving cars'?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.un.org/apps/news/story.asp?NewsID=36823&quot; rel=&quot;nofollow noreferrer&quot;&gt;Secretary-General said the UN&lt;/a&gt; would work hard to prevent further deaths on the roads:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Many tragedies can be avoided through a set of proven, simple measures that benefit not only individuals and families but society at large.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here are my points:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To achieve 'a set of proven measures' - do not allow people to drive - simple.&lt;/li&gt;&#xA;&lt;li&gt;People tend to break the rules, always, so do not allow them to drive without permission.&lt;/li&gt;&#xA;&lt;li&gt;Reduce stealing cars and other crime.&lt;/li&gt;&#xA;&lt;li&gt;Law enforcement dream is to able to stop any car on demand.&lt;/li&gt;&#xA;&lt;li&gt;Do not allow drunk people to drive a car.&lt;/li&gt;&#xA;&lt;li&gt;Disallow terrorist attacks, like in &lt;a href=&quot;https://en.wikipedia.org/wiki/2016_Nice_attack&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nice where truck killed over 80 people&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Avoid bank robberies and similar which are possible by escaping fast cars.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is it possible? I believe it depends on specific countries and unions and how quickly we're able to advance and be ready for such change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To support above points and summarize the 'only self-driving cars' point, please see below references which shows that this is already happening:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2014: &lt;a href=&quot;http://www.theverge.com/2014/5/28/5758734/uber-will-eventually-replace-all-its-drivers-with-self-driving-cars&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uber will eventually replace all its drivers with self-driving cars&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2016: &lt;a href=&quot;http://www.dezeen.com/2016/02/12/google-self-driving-car-artficial-intelligence-system-recognised-as-driver-usa/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's self-driving car system has been officially recognised as a driver in the US.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The move is seen as a first step towards changing the law for cars that have &quot;no need for a human driver&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2016: &lt;a href=&quot;http://www.dezeen.com/2016/04/19/beverly-hills-replace-public-transport-driverless-cars-los-angeles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beverly Hills to replace public transport with self-driving cars&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2016: &lt;a href=&quot;http://www.sfexaminer.com/sf-pitches-149-million-plan-replace-cars-self-driving-vehicles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;San Francisco pitches $149 million plan to replace cars with self-driving vehicles&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;San Francisco’s future is autonomous and shared vehicles – and that future may be only a decade away.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2016: &lt;a href=&quot;http://spectrum.ieee.org/cars-that-think/transportation/self-driving/otto-selfdriving-truck-company-wants-to-replace-teamsters&quot; rel=&quot;nofollow noreferrer&quot;&gt;Otto Self-Driving Truck Company Wants to Replace Teamsters&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-12T01:42:05.757" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1577" PostTypeId="2" ParentId="1560" CreationDate="2016-08-12T02:30:20.793" Score="3" Body="&lt;p&gt;The AI of the car uses sensor data to process all the data and classifies objects &lt;strong&gt;based on the size, shape and movement patterns&lt;/strong&gt;. It can recognize surroundings from a 360 degree perspective by making predictions about vehicles, people and objects around it will move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can detect pedestrians, but as moving, &lt;strong&gt;column-shaped blurs of pixels&lt;/strong&gt;, so it really cannot tell whether it's a rock or a crumpled piece of paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://media.giphy.com/media/3oEduYqb4Ty6dSReKc/giphy-downsized-large.gif&quot; alt=&quot;Google&amp;#39;s self-driving car sees traffic&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However it is programmed to determine certain patterns when a police officer has halted traffic or the car is being signaled to move forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://media.giphy.com/media/l41lGfjhDlrSE9ILS/giphy-downsized-large.gif&quot; alt=&quot;Google&amp;#39;s self-driving car determines when a police officer has halted traffic or when the car is being signaled to move forward&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also recognizes cyclists as objects outlined in red and can slow down to let the cyclist enter into a lane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://media.giphy.com/media/3oEduTuU46cDCGGuC4/giphy-downsized-large.gif&quot; alt=&quot;Google&amp;#39;s self-driving car sees when a cyclist is trying to merge into a lane, the vehicle also knows to slow down and let the cyclist enter&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Above images are provided by Chris Urmson who heads up Google's driverless car program.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.techinsider.io/how-googles-self-driving-cars-see-the-world-2015-10/#then-it-uses-its-sensor-data-to-understand-what-it-sees-in-the-moment-the-software-processes-all-of-the-data-and-classifies-objects-based-on-size-shape-and-movement-patterns-2&quot; rel=&quot;nofollow&quot;&gt;How Google's self-driving cars see the world&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.technologyreview.com/s/530276/hidden-obstacles-for-googles-self-driving-cars/&quot; rel=&quot;nofollow&quot;&gt;Hidden Obstacles for Google’s Self-Driving Cars&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(video) &lt;a href=&quot;https://www.youtube.com/watch?v=tiwVMrTLUWg&quot; rel=&quot;nofollow&quot;&gt;Chris Urmson: How a driverless car sees the road&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-13T02:51:10.783" LastActivityDate="2016-08-13T02:51:10.783" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1578" PostTypeId="2" ParentId="1561" CreationDate="2016-08-12T02:48:59.620" Score="9" Body="&lt;p&gt;Google’s self-driving car most likely uses &lt;a href=&quot;https://viejournal.springeropen.com/articles/10.1186/s40327-015-0027-1&quot; rel=&quot;noreferrer&quot;&gt;mapping of traffic signs using google street view images for roadway inventory management&lt;/a&gt;. If traffic signs are not in its database, it can still “see” and detect moving objects which can be distinguished from the presence of certain stationary objects, like traffic lights. So its software can classify objects based on the size, shape and movement patterns. Therefore it is highly unlikely that a person would be mistaken for a traffic sign. See: &lt;a href=&quot;https://ai.stackexchange.com/q/1560/8&quot;&gt;How does Google&amp;#39;s self-driving car identify pedestrians?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/530276/hidden-obstacles-for-googles-self-driving-cars/&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/setJb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Image: Technology Review&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To support such a claim, &lt;a href=&quot;http://www.cs.cmu.edu/~illah/&quot; rel=&quot;noreferrer&quot;&gt;Illah Nourbakhsh&lt;/a&gt;, a professor of robotics at Carnegie Mellon University, gave an interview to the New York Times magazine cover story on autonomous driving cars, and includes this hypothetical scenario, saying:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If they’re outside walking, and the sun is at just the right glare level, and there’s a mirrored truck stopped next to you, and the sun bounces off that truck and hits the guy so that you can’t see his face anymore — well, now your car just sees a stop sign. &lt;strong&gt;The chances of all that happening are diminishingly small — it’s very, very unlikely&lt;/strong&gt; — but the problem is we will have millions of these cars. The very unlikely will happen all the time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Even so, the risk would be minimal, since the car is always looking out for traffic, pedestrians and other obstacles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.techinsider.io/how-googles-self-driving-cars-see-the-world-2015-10/#googles-self-driving-vehicles-first-establish-their-location-by-using-mapping-and-sensor-data-1&quot; rel=&quot;noreferrer&quot;&gt;How Google's self-driving cars see the world&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.nytimes.com/2015/11/15/magazine/the-dream-life-of-driverless-cars.html&quot; rel=&quot;noreferrer&quot;&gt;The Dream Life of Driverless Cars&lt;/a&gt; at The New York Times&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-29T17:29:29.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1579" PostTypeId="2" ParentId="1488" CreationDate="2016-08-12T03:22:24.100" Score="1" Body="&lt;p&gt;Tesla model S has &lt;a href=&quot;https://www.tesla.com/models&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autopilot&lt;/a&gt; which allows to steer within a lane, change lanes with the simple tap of a turn signal, and can manage speed by using traffic-aware cruise control. Multiple digital controls helps to avoid collisions. Based on that, this isn't fully self-driving car.&lt;/p&gt;&#xA;&lt;p&gt;However it is using a computer vision detection system, but it is not intended to be used hands-free.&lt;/p&gt;&#xA;&lt;p&gt;So basically what is known is that the accident involved the side of a truck trailer (of a large white 18-wheel truck) and most likely the camera had a washed out of picture possibly due to glare or blooming from overexposure which made that the side of the trailer white and thin which failed to distinguish with the sky which was bright as well.&lt;/p&gt;&#xA;&lt;p&gt;This may have happened in part, because the crash-avoidance system only engage when both radar and vision system detect an obstacle which could not happen.&lt;/p&gt;&#xA;&lt;p&gt;Further more it was suggested by &lt;em&gt;The Associated Press&lt;/em&gt; that the driver most likely was watching a &lt;em&gt;Harry Potter&lt;/em&gt; at the time of the crash and assuming system would alert Brown, we don't know if he was able to retake controls quickly enough to avoid impact. As mentioned again, the system wasn't intended for hands-free driving and parts of the system was unfinished. Not to mention that the car was driving with full speed under the trailer.&lt;/p&gt;&#xA;&lt;p&gt;Tesla officially said about this crash in a statement on its website:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;The high ride height of the trailer combined with its positioning across the road and the extremely rare circumstances&lt;/strong&gt; of the impact caused the Model S to pass under the trailer, with the bottom of the trailer impacting the windshield of the Model S.&lt;/p&gt;&#xA;&lt;p&gt;Neither Autopilot nor the driver noticed the white side of the tractor-trailer against a brightly lit sky, so the brake was not applied.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They also said, according to techno-optimists, that they will tweaks their code, so this particular case won't happen again.&lt;/p&gt;&#xA;&lt;p&gt;To summarize, this was a 'technical failure' of braking system and most likely Autopilot was not at as Tesla told Senate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/obdLM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/obdLM.png&quot; alt=&quot;The New York Times |Source: Florida traffic crash report&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;The New York Times |Source: Florida traffic crash report&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.freep.com/story/money/cars/2016/07/01/tesla-autopilot-death-highlights-autonomous-risks/86591130/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla Autopilot death highlights autonomous risks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://robotfuturesbook.wordpress.com/2016/07/01/layers-of-autonomy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Layers of Autonomy&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.nytimes.com/interactive/2016/07/01/business/inside-tesla-accident.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inside the Self-Driving Tesla Fatal Accident&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.theguardian.com/technology/2016/jun/30/tesla-autopilot-death-self-driving-car-elon-musk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla driver dies in first fatal crash while using autopilot mode&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-08-12T03:38:43.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1580" PostTypeId="1" CreationDate="2016-08-12T06:47:12.693" Score="5" ViewCount="132" Body="&lt;p&gt;Has there any research been done on how difficult certain languages are to learn for chatbots? &#xA;For example, CleverBot knows a bit of Dutch, German, Finnish and French, so there are clearly chatbots that speak other languages than English. (English is still her best language, but that is because she speaks that most often)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would imagine that a logical constructed language, like lobjan, would be easier to learn than a natural language, like English, for example.  &lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="4302" LastEditDate="2018-10-08T12:53:50.273" LastActivityDate="2018-10-08T12:53:50.273" Title="Are there any results on how difficult certain languages are to learn for chatbots?" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1581" PostTypeId="2" ParentId="1539" CreationDate="2016-08-12T08:33:43.560" Score="6" Body="&lt;p&gt;Further to Franck's answer, there may be better optima (even global optima) that exist in the opposite direction to the gradient (which may be in the direction of some local optima). Evolutionary algorithms have scope to search the surrounding area, while backpropagation will always move in the direction of the gradient. With no guarantee (due to their randomness), evolutionary algorithms may be capable of finding solutions that backpropagation simply cannot.&lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-08-12T08:33:43.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1582" PostTypeId="5" CreationDate="2016-08-12T09:54:04.513" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-12T09:54:04.513" LastActivityDate="2016-08-12T09:54:04.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1583" PostTypeId="4" CreationDate="2016-08-12T09:54:04.513" Score="0" Body="For self-driving cars. This tag should be used with the [self-driving] tag." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-12T15:18:47.173" LastActivityDate="2016-08-12T15:18:47.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1584" PostTypeId="2" ParentId="1568" CreationDate="2016-08-12T15:29:14.740" Score="5" Body="&lt;p&gt;For many years, the focus has been on games with perfect information. That is, in Chess and Go both of us are looking at the same board. In something like Poker, you have information that I don't have and I have information that you don't have, and so for either of us to make sense of each other's actions we need to model what hidden information the other player has, and &lt;em&gt;also&lt;/em&gt; manage how we leak our hidden information. (A poker bot whose hand strength could be trivially determined from its bets will be easier to beat than a poker bot that doesn't.)&lt;/p&gt;&#xA;&lt;p&gt;Current research is switching to tackling games with imperfect information. Deepmind, for example, &lt;a href=&quot;http://www.businessinsider.com/google-deepmind-could-play-starcraft-2016-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;has said&lt;/a&gt; they might approach Starcraft next.&lt;/p&gt;&#xA;&lt;p&gt;I don't see too much difference between video games and board games, and there are several good reasons to switch to video games for games with imperfect information.&lt;/p&gt;&#xA;&lt;p&gt;One is that if you want to beat the best human to be a major victory, there needs to be a pyramid of skill that human is atop of--it'll be harder to unseat the top Starcraft champion than the top Warcraft champion, even though the bots might be comparably difficult to code, just because humans have tried harder at Starcraft.&lt;/p&gt;&#xA;&lt;p&gt;Another is that many games with imperfect information deal with reading faces and concealing information, which an AI would have an unnatural advantage at; for multiplayer video games, players normally interact with each other through a server as an intermediary and so the competition will be more normal.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="36737" LastEditDate="2021-03-31T22:22:21.160" LastActivityDate="2021-03-31T22:22:21.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1585" PostTypeId="5" CreationDate="2016-08-12T15:33:44.037" Score="0" Body="&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.ibm.com/watson/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Watson&lt;/a&gt;&lt;/strong&gt; is a question answering computer system capable of answering questions posed in natural language,developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Watson is a cognitive technology that can think like a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Watson can,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Understand&lt;/strong&gt; stuff by analyzing and interpreting whatever you query it.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Reason&lt;/strong&gt; based on the input given by you.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Learn&lt;/strong&gt; new stuff using Machine Learning.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Interact&lt;/strong&gt; with you, chat with you and help you solve the real world problems. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="1807" LastEditDate="2017-01-05T21:45:24.563" LastActivityDate="2017-01-05T21:45:24.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1586" PostTypeId="4" CreationDate="2016-08-12T15:33:44.037" Score="0" Body="For questions related to IBM Watson." OwnerUserId="1807" LastEditorUserId="1807" LastEditDate="2017-01-05T21:43:28.450" LastActivityDate="2017-01-05T21:43:28.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1587" PostTypeId="2" ParentId="1580" CreationDate="2016-08-12T16:40:07.353" Score="1" Body="&lt;p&gt;It's not the language itself but rather the structure and for the language's ambiguity, for example in English: person a says &quot;John and Bob (his fish)&quot; person B says &quot;He died!&quot;, posed question, to whom does person B refer to by he died.  More than the language, but the application. You can write a chat bot in Assembly, C, C++, C#, Java or Python. The all work a bit differently but can accomplish the same result, but one language might have more pros or cons to the other. So it will boil down to not language but the understanding of what is being said, research of language in the brain has come to confirm we associate a meaning/feeling/and other inputs with a given language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to conclude: English is by far the most chaotic for a chat bot but Japanese is actually the best due to the way the language itself is written/spoken. There is more structure to it and less ambiguity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a Software Engineer and An AI Researcher for the past 7 years.&lt;/p&gt;&#xA;" OwnerUserId="1282" LastEditorUserId="1282" LastEditDate="2016-08-30T20:22:12.103" LastActivityDate="2016-08-30T20:22:12.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1588" PostTypeId="2" ParentId="1476" CreationDate="2016-08-12T16:52:44.237" Score="0" Body="&lt;p&gt;Simply yes, but it can lead to over fixing of the NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans favour not dying, which is only realised once a consequence is defined for the system to realise that death is an unfavorable result. Which can be train vai observation. Allow your system to observe between 2 or more separate people/systems. Then allow opportunity to test in a safe environment with the pre  existing info of the consequences that may follow, provind that if the system makes a mistake in the test/safe environment it will be saved  unknownly and then informed that it made a mistake, the place system in an unsafe world in same conditions, informing it that if something happens it will die. That is the way humans grow up, and we've lasted very long with this technic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm an AI Researcher and Software Engineer for the past 7 years.&lt;/p&gt;&#xA;" OwnerUserId="1282" LastEditorUserId="1282" LastEditDate="2016-08-12T17:00:25.150" LastActivityDate="2016-08-12T17:00:25.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1589" PostTypeId="2" ParentId="1481" CreationDate="2016-08-12T17:06:48.000" Score="2" Body="&lt;p&gt;A neural network can be used but must be trained to expect the information (pattern of data, pixels or groupings of loose range such as color, and location) at any given location in the network, first a vision system must but implemented. Then a facial recognition, multiple partial individual body fixing (finding body part and there partners to a person) then training on some states and you'll have it work. MIT have done research and have made a seemy accurate implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm an AI Researcher and Software Engineer for the past 7 years.&lt;/p&gt;&#xA;" OwnerUserId="1282" LastEditorUserId="8" LastEditDate="2016-08-12T20:00:30.747" LastActivityDate="2016-08-12T20:00:30.747" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1590" PostTypeId="2" ParentId="1481" CreationDate="2016-08-12T17:24:47.190" Score="3" Body="&lt;p&gt;MIT have done research  and implemented an incomplete version of action video recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the use of MATLAB, NNetworks and a large set of training videos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggested set of comments on my previous answer indicate the usage of a multi interconnected NNet, verus MIT's image based NNet.&lt;/p&gt;&#xA;" OwnerUserId="1282" LastActivityDate="2016-08-12T17:24:47.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1591" PostTypeId="2" ParentId="1476" CreationDate="2016-08-12T20:04:20.157" Score="-1" Body="&lt;p&gt;It's impossible to give a definitive 'yes' answer to your question, since that would require proving that alternatives &lt;em&gt;cannot&lt;/em&gt; exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More philosophically, it depends on what you mean by &quot;preference over world states&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However counter-intuitive it might seem, it is conceivably possible to create Strong AI purely from local condition-action rules, in which there is no global concept of 'preference value' and/or no integrated notion of 'world state'.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-12T20:04:20.157" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1592" PostTypeId="1" CreationDate="2016-08-12T20:27:15.577" Score="7" ViewCount="149" Body="&lt;p&gt;Google, Tesla or Apple have all built or are building their own self-driving cars. As an expert in a related area, I am interested in knowing at a high level, the systems and techniques that go into self-driving cars. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How easy is it for me to make a tabletop prototype (large enough to accommodate the needed computing power needs)?&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="2444" LastEditDate="2019-06-26T12:04:00.113" LastActivityDate="2019-06-26T12:04:00.113" Title="What technologies are needed for a self-driving car?" Tags="&lt;ai-design&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1593" PostTypeId="1" CreationDate="2016-08-12T20:36:59.840" Score="3" ViewCount="124" Body="&lt;p&gt;Humans have been endowed with personalities by nature, and it is not clear (to me at least) if this is a feature or a bug. This has been explored in science fiction by various notions of &lt;a href=&quot;http://memory-alpha.org/Borg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Borg&lt;/a&gt;-like entities. It is my belief that, for narrative reasons, such stories usually end with the humans with their flawed personalities winning in the end.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Are there experts who have analyzed, perhaps mathematically, design criteria for an AI agent with weakly enforced goals (e.g. to maximize reproduction in the human case) in an uncertain environment, and ended up with the answer that a notion of personality is useful?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;If there are AI researchers, philosophers (or maybe even science fiction) writers who have examined this question in their work, I would be happy to know about those too.&lt;/p&gt;&#xA;" OwnerUserId="130" LastEditorUserId="2444" LastEditDate="2021-01-26T16:35:19.977" LastActivityDate="2021-01-26T16:35:19.977" Title="Is there any research work on the usefulness of the concept of &quot;personality&quot; for an AGI?" Tags="&lt;reference-request&gt;&lt;philosophy&gt;&lt;agi&gt;&lt;affective-computing&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1594" PostTypeId="2" ParentId="1476" CreationDate="2016-08-12T21:31:07.827" Score="0" Body="&lt;p&gt;Simply put, we don't know how to create Strong Artificial Intelligence yet, so we don't know what is or isn't required to create it.  At best we can engage in &quot;informed speculation&quot;, in which case I'd say that the answer is more likely &quot;yes&quot; than &quot;no&quot;.  But that's basically just a hunch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in a pretty good overview of what &quot;pieces&quot; might be required to create Strong AI, and if you haven't read it yet, &lt;a href=&quot;http://homes.cs.washington.edu/~pedrod/&quot; rel=&quot;nofollow&quot;&gt;Pedro Domingos&lt;/a&gt;' book &lt;em&gt;The Master Algorithm&lt;/em&gt; might be of interest. &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-08-12T21:39:28.547" LastActivityDate="2016-08-12T21:39:28.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1595" PostTypeId="2" ParentId="1531" CreationDate="2016-08-12T21:38:48.343" Score="13" Body="&lt;p&gt;Yes, as mentioned in other answers, Prolog is actually used in IBM Watson.  Prolog doesn't get much &quot;hype&quot; and &quot;buzz&quot; these days, but it is absolutely still used.  As always, it has certain specific areas where it shines, and specific techniques that map well to its use.  Specifically, things like &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_logic_programming&quot; rel=&quot;noreferrer&quot;&gt;Inductive Logic Programming&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_logic_programming&quot; rel=&quot;noreferrer&quot;&gt;Constraint Logic Programming&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Answer_set_programming&quot; rel=&quot;noreferrer&quot;&gt;Answer Set Programming&lt;/a&gt; and some &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;noreferrer&quot;&gt;NLP&lt;/a&gt; applications may involve extensive use of Prolog.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-12T21:38:48.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1596" PostTypeId="2" ParentId="1593" CreationDate="2016-08-12T21:59:58.440" Score="3" Body="&lt;p&gt;'Personality' is something of a 'suitcase word' (Minsky) for quite a large collection of (presumably reasonably consistent) observable traits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems clear that there is a certain collective advantage in having a consistent personality - specifically that it affords observers some learning gradient in an otherwise uncertain environment. This is of particular importance because those consistencies might have been arrived at using different learning mechanisms than the ones a given observer has.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, in any non-trivial coevolutionary system, other organisms will inevitably make use of any such consistencies. Consider a simple robot, called Alice, say, that has the trait of 'quickly flashing red when it sees a blue robot'. It makes sense for all observers to exploit &lt;em&gt;everything&lt;/em&gt; that they perceive as correlating with Alice's behavior, in particular, the prediction that a blue robot is likely to be present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best reference I can recommend on this (which shows that we tend to ascribe 'personality' to even very simple mechanisms) is &lt;a href=&quot;https://mitpress.mit.edu/books/vehicles&quot; rel=&quot;nofollow&quot;&gt;'Vehicles'&lt;/a&gt; by Valentino Braitenberg.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-12T22:06:05.017" LastActivityDate="2016-08-12T22:06:05.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1597" PostTypeId="2" ParentId="1593" CreationDate="2016-08-13T00:19:58.177" Score="3" Body="&lt;p&gt;&lt;strong&gt;First, a note on the question itself.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Humans have been endowed with personalities by nature, and it is not clear (to me at least) if this is a feature or a bug. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my opinion, this is a statement that constrains the question, since it assumes that the personality is &lt;em&gt;given&lt;/em&gt;. To me, it feels a bit like &lt;em&gt;playing god&lt;/em&gt;: Artificial (given) Intelligence would hence imply Artificial (given) Personality. This approach to the problem seems to be supported by the next fragment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a notion of personality is useful&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I point to the above because I don't think that artificial intelligence... &lt;strong&gt;&lt;em&gt;Intelligence&lt;/em&gt;&lt;/strong&gt; itself, actually, need to be given or assigned, or even have a &lt;em&gt;use&lt;/em&gt; in the sense of a &lt;em&gt;purpose&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The previous note&lt;/strong&gt; was about &lt;em&gt;emergence&lt;/em&gt;, which is a topic that &lt;a href=&quot;https://ai.stackexchange.com/users/42/user217281728&quot;&gt;user217281728&lt;/a&gt; briefly addressed in &lt;a href=&quot;https://ai.stackexchange.com/a/1596/70&quot;&gt;their answer&lt;/a&gt;. In this second approach, the particular traits &lt;em&gt;just happen&lt;/em&gt;, or &lt;em&gt;develop&lt;/em&gt;. The interaction between the (so-called) agents and their environment, as well as fellow agents can give place to new behaviour patterns, not designed beforehand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In an evolutionary&lt;/strong&gt; approach, if the personality would happen to have an advantage (or at least not represent a disadvantage), then it could just appear. Of course, I am making a number of assumptions and demarcations here as well:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am thinking about embodied intelligence&lt;/li&gt;&#xA;&lt;li&gt;I speak of evolutionary robotics&lt;/li&gt;&#xA;&lt;li&gt;I think on social issues being of importance&lt;/li&gt;&#xA;&lt;li&gt;I assume that &lt;em&gt;personality&lt;/em&gt; could emerge&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now, an example&lt;/strong&gt; that I find extremely interesting is that of the little mobile robots which could move around and end-up in a pool of &lt;em&gt;food&lt;/em&gt; or a pool of &lt;em&gt;poison&lt;/em&gt;. And they, somehow, by some odd chance, recognised or made a relation between signals sent by other robots, and the presence of food. Or not. That was more or less the thing: Some robots (kind of) learned to &lt;em&gt;conceal&lt;/em&gt; information and thus had more time to eat themselves. Well, I would have a couple of personality adjectives for such guys.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.pnas.org/content/106/37/15786.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here you find the article&lt;/a&gt; and &lt;a href=&quot;https://wp.unil.ch/mitrilab/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here you find some videos&lt;/a&gt; and related stuff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;And with that&lt;/strong&gt;, we land at my last point: &lt;a href=&quot;https://en.wikipedia.org/wiki/Anthropomorphism#In_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;We humans&lt;/a&gt; put the adjectives, according to our social conditioning. We call &lt;a href=&quot;https://en.wikipedia.org/wiki/Marvin_(character)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marvin&lt;/a&gt; &lt;em&gt;depressive&lt;/em&gt; and &lt;a href=&quot;http://www.smithsonianmag.com/arts-culture/why-do-we-love-r2-d2-and-not-c-3po-180951176/&quot; rel=&quot;nofollow noreferrer&quot;&gt;R2D2&lt;/a&gt; lovely and charming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they perceive their personalities as constructive or damaging, will always depend on our own judgment. In the end, it is quite common under humans to disagree on personality issues, too.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember when &lt;a href=&quot;https://www.youtube.com/watch?v=UgkyrW2NiwM&quot; rel=&quot;nofollow noreferrer&quot;&gt;HAL got emotional&lt;/a&gt;, on the face of death?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It gets &lt;em&gt;human&lt;/em&gt; when it loses its cool, before the flawed-personality human astronaut :)&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-13T00:19:58.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1598" PostTypeId="1" AcceptedAnswerId="1599" CreationDate="2016-08-13T02:46:55.560" Score="1" ViewCount="154" Body="&lt;p&gt;I've found this short &lt;a href=&quot;http://iamtrask.github.io/2015/07/12/basic-python-network/&quot; rel=&quot;nofollow&quot;&gt;Python code&lt;/a&gt; which implements neural network in 11 lines of code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1] ])&#xA;y = np.array([[0,1,1,0]]).T&#xA;syn0 = 2*np.random.random((3,4)) - 1&#xA;syn1 = 2*np.random.random((4,1)) - 1&#xA;for j in xrange(60000):&#xA;    l1 = 1/(1+np.exp(-(np.dot(X,syn0))))&#xA;    l2 = 1/(1+np.exp(-(np.dot(l1,syn1))))&#xA;    l2_delta = (y - l2)*(l2*(1-l2))&#xA;    l1_delta = l2_delta.dot(syn1.T) * (l1 * (1-l1))&#xA;    syn1 += l1.T.dot(l2_delta)&#xA;    syn0 += X.T.dot(l1_delta)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I believe it may be a valid implementation of neural network, but how do I know?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, is just creating bunch of arrays which compute the output on certain criteria and call them layers with synapses does it make proper neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, I'd like to ask, what features/properties makes a valid artificial neural network?&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-17T01:23:18.030" Title="What are the minimum requirements to call something artificial neural network?" Tags="&lt;neural-networks&gt;&lt;implementation&gt;&lt;computer-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1599" PostTypeId="2" ParentId="1598" CreationDate="2016-08-13T03:27:59.370" Score="4" Body="&lt;p&gt;If you pick up a textbook on Neural Networks, you'll find that the simplest examples shown are ones that just implement an AND gate or something.  They're trivial, probably fewer lines of code than what you have there.  The bar to be an &quot;artificial neural network&quot; is pretty low... it certainly isn't the case that ANN's &lt;em&gt;must&lt;/em&gt; be incredibly complicated with thousands of lines of code, and many layers, or even many &quot;neurons&quot; total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, if something is setting up at least one &quot;neuron&quot; with multiple inputs, and using some kind of weighting function to generate an output from those inputs, it's a valid ANN. It might be a &lt;em&gt;really&lt;/em&gt; simple example of an ANN, but it's still an ANN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember what Geoffrey Hinton says in his &lt;em&gt;Coursera&lt;/em&gt; class - (paraphrased) &quot;We don't pretend that the things we're building really work the way the brain does, we're just taking the brain as loose inspiration for an approach that we've found works&quot;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="8" LastEditDate="2016-08-17T01:23:18.030" LastActivityDate="2016-08-17T01:23:18.030" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1601" PostTypeId="1" AcceptedAnswerId="1608" CreationDate="2016-08-13T03:51:50.153" Score="3" ViewCount="37" Body="&lt;p&gt;I'm looking for research which discusses misbehavior detection in public internet access networks using ANN approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it can be used by &lt;a href=&quot;https://en.wikipedia.org/wiki/Internet_service_provider&quot; rel=&quot;nofollow&quot;&gt;ISP&lt;/a&gt; to detect suspicious users connected to their network.&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2016-08-13T07:45:28.300" Title="Research for misbehavior detection in WiFI networks" Tags="&lt;neural-networks&gt;&lt;research&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1603" PostTypeId="1" AcceptedAnswerId="1616" CreationDate="2016-08-13T04:08:47.627" Score="3" ViewCount="123" Body="&lt;p&gt;I'm investigating applications of AI algorithms which can be used for data leakage detection and prevention within an intranet network (like &lt;a href=&quot;https://en.wikipedia.org/wiki/Forcepoint&quot; rel=&quot;nofollow&quot;&gt;Forcepoint&lt;/a&gt;). More specifically detecting traffic patterns. I'm new to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which learning algorithms are most suitable for this goal? &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_algorithm&quot; rel=&quot;nofollow&quot;&gt;EA&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow&quot;&gt;GA&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow&quot;&gt;ANN&lt;/a&gt; (which one) or something else?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="29" LastEditDate="2016-08-17T11:53:52.427" LastActivityDate="2016-08-17T11:53:52.427" Title="Which learning algorithms are suitable for data leakage detection and prevention?" Tags="&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1604" PostTypeId="5" CreationDate="2016-08-13T04:10:11.043" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-13T04:10:11.043" LastActivityDate="2016-08-13T04:10:11.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1605" PostTypeId="4" CreationDate="2016-08-13T04:10:11.043" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-13T04:10:11.043" LastActivityDate="2016-08-13T04:10:11.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1606" PostTypeId="1" AcceptedAnswerId="1607" CreationDate="2016-08-13T04:25:29.447" Score="2" ViewCount="150" Body="&lt;p&gt;I'm wondering, instead of implementing new web browsers over and over again with millions line of code which is very difficult to manage, would it be possible to use ANN or GA algorithm to teach it about the rendering process (how the page should look like)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So as an input I would imaging the html source code, output is the rendered page (maybe in some interactive image like SVG, some library or something, I'm not sure).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training data can be dataset of websites providing input source code and their rendered representation by using other browsers for the guidance as expected output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which approach would you take and what are the most challenging things you can think of?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-13T11:20:33.007" LastActivityDate="2016-08-13T11:20:33.007" Title="What are the approaches to teach AI to how to render html page based on its source code?" Tags="&lt;neural-networks&gt;&lt;implementation&gt;&lt;computer-programming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1607" PostTypeId="2" ParentId="1606" CreationDate="2016-08-13T05:49:50.547" Score="7" Body="&lt;p&gt;The rendering process for browsers is &lt;a href=&quot;https://www.w3.org/&quot;&gt;very well defined&lt;/a&gt;, and has a very rigid definite ruleset where (virtually) every accountability is noted and handled. This is not optimal for Machine Learning, which works when we have a large pool of examples, and we don't know the ruleset; it will figure it out. Even if you were to train an Neural Network to process that input, there are several things you must account for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Variance in data.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not all webpages are equal in length or complexity, and making a neural network to generate output from HTML would produce garbage most of the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Training time.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time it would take for a neural network to understand HTML tags, attributes, the DOM Tree, and each and every element, including new ones being added every few years, and how each one renders and behaves, would take an extremely long time, most likely several years on a fast computer, if it even were possible&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Interactivity.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Web pages aren't just static, they change according HTML, CSS and JavaScript. Not only would you have to design your system to account for the rendering step, you would also make it have to understand the &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_completeness&quot;&gt;Turing Complete&lt;/a&gt; scripting language &lt;a href=&quot;https://en.wikipedia.org/wiki/JavaScript&quot;&gt;JavaScript&lt;/a&gt;, as well as the less complicated, but inherently intertwined with HTML, CSS stylesheet language. If you thought the rendering process was easy, try training a neural network to handle complicated scripting patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4. New Standards&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not all HTML is equal, because of different standards. &lt;a href=&quot;https://en.wikipedia.org/wiki/Web_Hypertext_Application_Technology_Working_Group&quot;&gt;WHATWG&lt;/a&gt; began working on HTML5 in 2004, and browsers started to implement not long after. In 2004, there were very few examples of HTML5 sites to train your network to begin with. Sure, now it's standardized and every website uses it, but what about HTML6? When the first specification is released (probably 2017-2025), virtually no websites will use it, because no one will support it. Only when it finally becomes standard, probably in the late 2020s or early 2030s, will you have enough data to train your monstrous system of neural networks&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;&#xA;&lt;p&gt;As for AI in general, one could argue that browsers already use A.I. in their rendering process. They intelligently decide what to render (taking CSS into account), when in order to get the most efficient render time, they selectively use different JavaScript parsers on different sections of the code to optimize the speed, the whole system has been optimized on another ruleset to make rendering and interacting with a webpage as seamless and easy-to-use as possible. Your system will never be as good as what hundreds of humans have optimized over 20 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to solve HTML rendering with Neural Networks is akin to trying to nail a nail with a screwdriver. It's just not going to work&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this was helpful!&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2016-08-13T05:49:50.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1608" PostTypeId="2" ParentId="1601" CreationDate="2016-08-13T07:45:28.300" Score="5" Body="&lt;p&gt;One popular technique for doing this is to use &lt;a href=&quot;http://www.artificial-immune-systems.org/&quot;&gt;Artificial Immune Systems&lt;/a&gt;, an evolutionary computation approach which maintains a population of pattern detectors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/0804/0804.1266.pdf&quot;&gt;survey paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-13T07:45:28.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1610" PostTypeId="2" ParentId="1592" CreationDate="2016-08-13T15:48:26.767" Score="4" Body="&lt;p&gt;You're going to need some way to 'see' the area around the car, and to track the speed of nearby objects. Google uses a combination of &lt;a href=&quot;https://en.wikipedia.org/wiki/Lidar&quot; rel=&quot;nofollow&quot;&gt;LIDAR&lt;/a&gt;, radar, conventional cameras, and occasionally sonar (see &lt;a href=&quot;http://www.makeuseof.com/tag/how-self-driving-cars-work-the-nuts-and-bolts-behind-googles-autonomous-car-program&quot; rel=&quot;nofollow&quot;&gt; here&lt;/a&gt; for a high-level overview). This technology is quite expensive, and can easily cost thousands of US dollars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a bigger obstacle than the expense of the hardware (which would be smaller for a table-top prototype) is the software complexity. Like many major projects, the software for self-driving cars is the result of years of work from AI research teams, and thus extremely difficult to duplicate on your own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you're not trying to make a state-of-the-art self-driving car.  Assuming you're an expert in image processing and robotics, you can probably create a basic prototype, (like something that drive in a limited table-top environment). However, it's still going to take a lot of time and money. &lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2016-08-13T15:48:26.767" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1611" PostTypeId="1" CreationDate="2016-08-13T18:17:50.147" Score="1" ViewCount="87" Body="&lt;p&gt;I'm trying to make a conversational chatbot, so the user inputs are quite wide ranging - beyond just &quot;turn lights on&quot;. I want to detect the category of the user intents from their inputs and prepare responses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked at MS' Luis and api.ai and the intents require a lot of training. Can people suggest other techniques for untrained intent detection?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if the user says &quot;Pasta is my favorite dish to cook&quot; then detect &quot;intent preference entity pasta&quot; - then I can gradually build up responses to different categories of inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the crowd-sourced intents that wit.ai (facebook) has access to could do this but I'm not sure if all end-users have access to those models.&lt;/p&gt;&#xA;" OwnerUserId="1506" LastEditorUserId="4709" LastEditDate="2018-09-07T10:02:24.350" LastActivityDate="2018-09-07T10:02:24.350" Title="What are good APIs out there for (untrained) intent detection?" Tags="&lt;chat-bots&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2016-08-14T02:14:31.893" ContentLicense="CC BY-SA 3.0" />
  <row Id="1612" PostTypeId="2" ParentId="26" CreationDate="2016-08-14T02:49:57.967" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Emotion&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emotions&lt;/a&gt; aren't something that you can implement - they're very complex. However, you can attempt to mimic them. Human emotions are closely related to conscious experience characterized by intense mental activity, which is based on interpretation of events.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent brain studies (including research in cognitive psychology and neurophysiology) suggests that human emotional assessment of every action or event plays an important role in human mental processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recent &lt;a href=&quot;http://bica2016.bicasociety.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2016 Annual Meeting of the BICA Society&lt;/a&gt; brought together scientists from around the world to approach principles and mechanisms of human thought to create biologically inspired AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in Samsonovich's (a professor in the Cybernetics Department at the &lt;a href=&quot;https://en.wikipedia.org/wiki/National_Research_Nuclear_University_MEPhI&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEPhI&lt;/a&gt;) proposal, the idea is to test AI in computer games which involves actions with emotional content, where AI may engage with players in different types of social relationships (such as trust, subordination or leadership).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jonathan Gratch of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Institute_for_Creative_Technologies&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICT&lt;/a&gt;, invented virtual characters capable of identifying and expressing emotions by communicating with humans in their natural language based on the situations where for example AI can deceive a human to achieve the desired result. The effect is obviously not achieved by re-creating human consciousness, but by achieving statistically adjusting parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researchers from the Institute of Cyber Intelligence Systems in MEPhI are hoping to be able to create in the near future virtual beings which are capable of planning, setting goals and establishing social relationships with humans, also by possessing both emotional and narrative intelligence which can interpret the context of events.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: &lt;a href=&quot;http://phys.org/news/2016-07-social-emotions-artificial-intelligence.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Researcher proposes social emotions test for artificial intelligence&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-11T16:40:11.517" LastActivityDate="2018-04-11T16:40:11.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1613" PostTypeId="1" CreationDate="2016-08-14T03:05:08.873" Score="6" ViewCount="632" Body="&lt;p&gt;How does a domestic autonomous robotic vacuum cleaner -  such as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Roomba&quot; rel=&quot;noreferrer&quot;&gt;Roomba&lt;/a&gt; - know when it's working cleaned area (aka virtual map), and how does it plan to travel to the areas which hasn't been explored yet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it use some kind of &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;noreferrer&quot;&gt;A*&lt;/a&gt; algorithm?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-14T12:26:15.817" LastActivityDate="2016-11-01T19:08:17.067" Title="How do autonomous robotic vacuum cleaners perceive the environment for navigation?" Tags="&lt;real-time&gt;&lt;path-planning&gt;&lt;robotics&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1614" PostTypeId="1" AcceptedAnswerId="1615" CreationDate="2016-08-14T03:17:29.793" Score="5" ViewCount="96" Body="&lt;p&gt;It has been &lt;a href=&quot;http://www.itnonline.com/content/will-fda-be-too-much-intelligent-machines&quot; rel=&quot;nofollow noreferrer&quot;&gt;suggested&lt;/a&gt; that machine learning algorithms (also &lt;a href=&quot;https://ai.stackexchange.com/q/1427/8&quot;&gt;Watson&lt;/a&gt;) can help with finding disease in patient images and optimize scans. Also that deep learning algorithms show promise for every type of digital imaging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does exactly deep learning algorithms exactly can find suspicious patterns in the body’s biochemistry?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-14T05:51:57.010" Title="How can artificial intelligence (including deep learning algorithms) find suspicious patterns in the body’s biochemistry?" Tags="&lt;deep-learning&gt;&lt;healthcare&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1615" PostTypeId="2" ParentId="1614" CreationDate="2016-08-14T05:49:46.060" Score="2" Body="&lt;p&gt;I wouldn't focus &lt;em&gt;only&lt;/em&gt; on &quot;deep learning&quot; unless you have some specific reason for doing so.  There may be other techniques which could be as effective, or more effective.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach I've seen used for something similar was &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_logic_programming&quot; rel=&quot;nofollow&quot;&gt;Inductive Logic Programming&lt;/a&gt;.  For one example of using ILP to reason about elements of biochemistry, see &lt;a href=&quot;https://www.researchgate.net/publication/224309517_Estimation_of_Possible_Reaction_States_in_Metabolic_Pathways_Using_Inductive_Logic_Programming&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's not exactly about detecting disease, but it does sort of illustrate the broad idea of reasoning about states and reactions involving metabolic pathways in biochemistry, using ILP.   Possibly the basic idea could be adapted more towards detecting disease.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-14T05:49:46.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1616" PostTypeId="2" ParentId="1603" CreationDate="2016-08-14T05:57:55.947" Score="3" Body="&lt;p&gt;This seems to fall broadly into the regime of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_classification&quot; rel=&quot;nofollow&quot;&gt;classification problem&lt;/a&gt; as you want to classify an outgoing communication as &quot;contains proprietary information&quot; or &quot;does not contain proprietary information&quot;.  As such, any classification approach could be applied.  Neural Networks certainly seem like a valid approach, but you might also get good mileage out of Random Forests, Support Vector Machines, a Naive Bayes classifier, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GA's are more aimed towards optimization than classification, so I wouldn't say that a GA, in and of itself, would map cleanly to solving this kind of problem. If GA's had applicability here, I think it would be more likely to be in terms of training a model rooted in one of the other techniques.     &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-14T05:57:55.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1617" PostTypeId="1" AcceptedAnswerId="1627" CreationDate="2016-08-14T17:28:47.513" Score="2" ViewCount="111" Body="&lt;p&gt;The &lt;a href=&quot;https://www.youtube.com/watch?v=AplG6KnOr2Q&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mario Lives!&lt;/a&gt; video (and its follow-up video, &lt;a href=&quot;https://www.youtube.com/watch?v=ltPj3RlN4Nw&amp;amp;list=PLuOoXrWK6Kz5ySULxGMtAUdZEg9SkXDoq&amp;amp;index=5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mario Becomes Social!&lt;/a&gt;) showcases an AI unit that is able to simulate emotional decision-making within a virtual world, and can enter into &quot;emotional states&quot; such as curiosity, hunger, happiness, and fear. While this seems cool and exciting (especially for video game AI), I am confused about how this would be useful in real-world scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the point of building autonomous actors that would behave based on these emotional states, instead of simply knowing &lt;em&gt;what&lt;/em&gt; they should do (either by hardcoding in the rules, or learning the rules through machine learning)?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="2444" LastEditDate="2020-03-10T23:20:36.190" LastActivityDate="2020-03-10T23:20:36.190" Title="Why would someone want to simulate emotional desicion-making within an AI?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;emotional-intelligence&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1618" PostTypeId="1" AcceptedAnswerId="1626" CreationDate="2016-08-14T19:16:12.140" Score="14" ViewCount="429" Body="&lt;p&gt;For a deterministic problem space, I need to find a neural network with the optimal node and link structure. I want to use a genetic algorithm to simulate many neural networks to find the best network structure for the problem domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never used genetic algorithms for a task like this before. &lt;em&gt;What are the practical considerations? Specifically, how should I encode the structure of a neural network into a genome?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2019-07-07T21:38:31.047" LastActivityDate="2019-07-07T21:38:31.047" Title="How should I encode the structure of a neural network into a genome?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1621" PostTypeId="2" ParentId="35" CreationDate="2016-08-15T02:38:50.250" Score="12" Body="&lt;p&gt;The machine learning is a sub-set of artificial intelligence which is only a small part of its potential. It's a specific way to implement AI largely focused on statistical/probabilistic techniques and evolutionary techniques.&lt;sup&gt;&lt;a href=&quot;https://www.quora.com/What-are-the-main-differences-between-artificial-intelligence-and-machine-learning/answer/Phillip-Rhodes&quot; rel=&quot;noreferrer&quot;&gt;Q&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Artificial intelligence&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence is '&lt;strong&gt;the theory and development of computer systems able to perform tasks normally requiring human intelligence&lt;/strong&gt;' (such as visual perception, speech recognition, decision-making, and translation between languages).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can think of AI as the concept of non-human decision making&lt;sup&gt;&lt;a href=&quot;https://www.quora.com/What-are-the-main-differences-between-artificial-intelligence-and-machine-learning/answer/Yuval-Ariav&quot; rel=&quot;noreferrer&quot;&gt;Q&lt;/a&gt;&lt;/sup&gt; which aims to simulate cognitive human-like functions such as problem-solving, decision making or language communication.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Machine learning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Machine learning (ML) is basically &lt;strong&gt;a learning through doing&lt;/strong&gt; by the implementation of build models which can predict and identify patterns from data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Prof. &lt;a href=&quot;http://www.cs.colby.edu/srtaylor/&quot; rel=&quot;noreferrer&quot;&gt;Stephanie R. Taylor&lt;/a&gt; of Computer Science and her &lt;a href=&quot;http://cs.colby.edu/courses/S15/cs251/LectureNotes/Lecture_15_MLandDMintro_03_09_2015.pdf&quot; rel=&quot;noreferrer&quot;&gt;lecture paper&lt;/a&gt;, and also &lt;a href=&quot;https://en.wikipedia.org/wiki/Learning#Machine_learning&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia page&lt;/a&gt;, 'machine learning is a branch of artificial intelligence and &lt;strong&gt;it's about construction and study of systems that can learn from data&lt;/strong&gt;' (like based on the existing email messages to learn how to distinguish between spam and non-spam).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://www.oxforddictionaries.com/definition/english/machine-learning&quot; rel=&quot;noreferrer&quot;&gt;Oxford Dictionaries&lt;/a&gt;, the machine learning is '&lt;strong&gt;the capacity of a computer to learn from experience&lt;/strong&gt;' (e.g. modify its processing on the basis of newly acquired information).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can think ML as computerized pattern detection in the existing data to predict patterns in future data.&lt;sup&gt;&lt;a href=&quot;https://www.quora.com/What-are-the-main-differences-between-artificial-intelligence-and-machine-learning/answer/Yuval-Ariav&quot; rel=&quot;noreferrer&quot;&gt;Q&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In other words, &lt;strong&gt;machine learning involves the development of self-learning algorithms&lt;/strong&gt; and &lt;strong&gt;artificial intelligence involves developing systems or software&lt;/strong&gt; to mimic human to respond and behave in a circumstance.&lt;sup&gt;&lt;a href=&quot;https://www.quora.com/What-are-the-main-differences-between-artificial-intelligence-and-machine-learning/answer/Sakthi-Dasan-2&quot; rel=&quot;noreferrer&quot;&gt;Quora&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="14723" LastEditDate="2018-04-11T16:56:41.540" LastActivityDate="2018-04-11T16:56:41.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1625" PostTypeId="1" AcceptedAnswerId="1959" CreationDate="2016-08-15T03:33:07.827" Score="4" ViewCount="752" Body="&lt;p&gt;Were there any studies which checked the accuracy of neural network predictions of greyhound racing results, compared to a human expert? Would it achieve a better payoff?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-15T14:56:08.323" LastActivityDate="2018-08-20T12:18:12.693" Title="Can neural networks be better than human experts at prediction of greyhound racing results?" Tags="&lt;neural-networks&gt;&lt;research&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1626" PostTypeId="2" ParentId="1618" CreationDate="2016-08-15T07:59:57.977" Score="12" Body="&lt;p&gt;Section 4.2 of &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics/&quot;&gt;&quot;Essentials of Metaheuristics&quot;&lt;/a&gt; has a wealth of information on alternative ways of encoding graph structures via Genetic Algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With particular regard to evolving ANNs, I would personally not be inclined to implement this sort of thing 'from scratch':&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The field of neuroevolution has been around for some time, and the implementation some of the methods, such as Neuroevolution of Augmenting Topologies (&lt;a href=&quot;http://www.cs.ucf.edu/~kstanley/neat.html&quot;&gt;NEAT&lt;/a&gt;) now incorporate the results of much practical experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the above link:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We also developed an extension to NEAT called HyperNEAT that can evolve neural networks with millions of connections and exploit geometric regularities in the task domain. The HyperNEAT Page includes links to publications and a general explanation of the approach.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-15T08:12:41.983" LastActivityDate="2016-08-15T08:12:41.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1627" PostTypeId="2" ParentId="1617" CreationDate="2016-08-15T10:44:09.767" Score="2" Body="&lt;p&gt;Humans have poor understanding of emotional rules. Probably every poster on here has experienced &lt;em&gt;greatly&lt;/em&gt; misreading another individual emotionally. Further, people often don't act emotionally how they would expect themselves to act, for example we have all experienced frustration at someone else's irrational concerns and yet we are all guilty of holding irrational concerns of our own. This is the crux of why hard-coded emotional rules do not work - we do not have an understanding of what emotional 'rules' make someone feel real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By moving towards autonomous state-based actors we move away from this issue. The actor's state transitions will of course be defined rules (hard-coded or learnt) but by abstracting the actor's emotional state from the specific context (e.g. specific actions trigger emotional state transitions which trigger responses, instead of a direct response to a specific action), the programmer prevents them-self from projecting their own beliefs/emotions/logic onto the actor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, autonomous actors are more extendable. Consider an autonomous actor that is hard-coded to move towards 'upset' and 'angry' emotional states when experiencing 'pain'. Simply by associating a new world action with 'pain', one can trigger an emotional response from an autonomous actor that has not experienced that action before. When working in hard-coded emotional rules, this would not be possible.&lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-08-15T10:44:09.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1628" PostTypeId="1" AcceptedAnswerId="1629" CreationDate="2016-08-15T11:08:05.700" Score="3" ViewCount="266" Body="&lt;p&gt;I've read about The Loebner Prize for AI, which pledged a Grand Prize of $100,000 and a Gold Medal for the first computer whose responses were indistinguishable from a human's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was wondering whether any chatbots have fooled the judges and won a Gold Medal yet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From their &lt;a href=&quot;http://www.loebner.net/Prizef/loebner-prize.html&quot; rel=&quot;nofollow&quot;&gt;website&lt;/a&gt; this isn't clear (as some of the links doesn't load).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A few highlights from previous years:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://loebner.exeter.ac.uk/results/&quot; rel=&quot;nofollow&quot;&gt;2011 Loebner Prize results&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;None of the AI systems fooled the judges, therefore the Turing Test has not been passed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.paulmckevitt.com/loebner2013/scoring/loebner2013leaderboard.txt&quot; rel=&quot;nofollow&quot;&gt;Loebner 2013 results&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;No chatbot fooled any of the 4 Judges.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-15T14:37:10.257" LastActivityDate="2016-08-15T14:37:10.257" Title="Have any chatbots fooled the judges and won the Loebner Prize Gold medal yet?" Tags="&lt;history&gt;&lt;turing-test&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1629" PostTypeId="2" ParentId="1628" CreationDate="2016-08-15T11:36:28.793" Score="4" Body="&lt;p&gt;The 2016 finals haven't started yet, they will start on Saturday, 17 September 2016. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 2015 finals or before that, nobody won the Gold Medal or the Silver Medal. &#xA;The most up-to-date data can be found &lt;a href=&quot;http://www.aisb.org.uk/events/loebner-prize&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, where we can find both the results from 2015 and the timeline of the 2016 contest. &lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2016-08-15T11:36:28.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1630" PostTypeId="1" AcceptedAnswerId="1631" CreationDate="2016-08-15T11:36:37.463" Score="5" ViewCount="261" Body="&lt;p&gt;Hypothetically, assume that you have access to infinite computing power. Do we have designs for any brute-force algorithms that can find an AI capable of passing traditional tests (e.g. Turing, Chinese Room, MIST, etc.)? &lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-12-18T18:40:48.327" Title="Given enough computational resources, do we currently have any algorithms which could achieve AI?" Tags="&lt;turing-test&gt;&lt;agi&gt;&lt;ai-design&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1631" PostTypeId="2" ParentId="1630" CreationDate="2016-08-15T12:07:47.927" Score="7" Body="&lt;p&gt;What 'infinite' means here could possibly be debated at some length, but that notwithstanding, here are two conflicting answers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Yes': Simulate all possible universes. Stop when you get to one containing a flavor of intelligence that passes whatever test you have in mind. Steven Wolfram has suggested something &lt;a href=&quot;https://www.inverse.com/article/12838-stephen-wolfram-could-there-be-alien-intelligence-among-the-digits-of-pi&quot; rel=&quot;nofollow&quot;&gt;broadly along these lines&lt;/a&gt;. Problem: the state of computational testing for intelligence &lt;a href=&quot;https://en.wikipedia.org/wiki/Winograd_Schema_Challenge&quot; rel=&quot;nofollow&quot;&gt;e.g. Winograd schema&lt;/a&gt; would then be the bottleneck. In the limit, testing for intelligence requires intelligence and creativity on behalf of the questioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'No': It may be that, even with infinite ability to simulate, there may be some missing aspect of our simulation that is necessary for intelligence. For example, AFAIK quantum gravity (for which we lack an adequate theory) is involved in Penrose's &lt;a href=&quot;https://www.sciencedaily.com/releases/2014/01/140116085105.htm&quot; rel=&quot;nofollow&quot;&gt;&quot;Quantum Microtubules&quot;&lt;/a&gt; theory of consciousness (*). What if that was needed, but we didn't know how to include it in the simulation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for talking in terms of such incredibly costly computations as 'simulate all possible universes' (or at least a brain-sized portion of them) is to deliberately generalize away from the specifics of any techniques currently in vogue (DL, neuromorphic systems etc). The point is that we could be missing something essential for intelligence from &lt;em&gt;any&lt;/em&gt; of these models and (as far as we know from our current theories of physical reality) only empirical evidence to the contrary would tell us otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(*) No-one knows if consciousness is required for Strong AI, and physics can't distinguish a conscious entity from a &lt;a href=&quot;http://plato.stanford.edu/entries/zombies/&quot; rel=&quot;nofollow&quot;&gt;Zombie&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-15T15:59:28.900" LastActivityDate="2016-08-15T15:59:28.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1632" PostTypeId="1" AcceptedAnswerId="1640" CreationDate="2016-08-15T12:52:22.533" Score="0" ViewCount="105" Body="&lt;p&gt;I'm aware this could be a complex topic, however I'm interested in existing research projects or studies where people are attempting or have succeeded in teaching an AI a foreign language just by training/teaching it from English books. By reading, analysing and understanding, so that it knows the foreign language's rules (such as grammar, spelling, etc.), the same way as a human would learn. The language doesn't have to be Chinese, which is difficult for even humans to learn.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-20T20:52:42.313" LastActivityDate="2019-06-20T20:52:42.313" Title="What are the current approaches for AI to learn a foreign language just from English books?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;research&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1633" PostTypeId="5" CreationDate="2016-08-15T14:47:54.450" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-15T14:47:54.450" LastActivityDate="2016-08-15T14:47:54.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1634" PostTypeId="4" CreationDate="2016-08-15T14:47:54.450" Score="0" Body="For questions related to in-depth study or academic research. Do NOT use this tag when you are trying to find something out." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-10-18T03:11:40.070" LastActivityDate="2018-10-18T03:11:40.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1635" PostTypeId="1" CreationDate="2016-08-15T14:59:47.477" Score="7" ViewCount="539" Body="&lt;p&gt;Would it be possible to put &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asimov's three Laws of Robotics&lt;/a&gt; into an AI?&lt;/p&gt;&#xA;&lt;p&gt;The three laws are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot (or, more accurately, an AI) cannot harm a human being, or through inaction allow a human being to be harmed&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot must listen to instructions given to it by a human, as long as that does not conflict with the first law.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot must protect its own existence, if that does not conflict with the first two laws.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2020-11-17T13:58:05.500" LastActivityDate="2020-12-05T00:50:27.480" Title="Is it possible to implement Asimov's Three Laws of Robotics?" Tags="&lt;implementation&gt;&lt;robotics&gt;&lt;asimovs-laws&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1636" PostTypeId="5" CreationDate="2016-08-15T15:20:44.123" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-15T15:20:44.123" LastActivityDate="2016-08-15T15:20:44.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1637" PostTypeId="4" CreationDate="2016-08-15T15:20:44.123" Score="0" Body="For questions about the definition of  terms used in artificial intelligence research and development, including the definition of intelligence, algorithms, jargon, principles, methodologies, mathematical terms, concepts, topologies, architectures, designs, jargon, and AI domains such as robotics, network training, or automated vehicles." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-10-15T16:55:26.197" LastActivityDate="2018-10-15T16:55:26.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1638" PostTypeId="2" ParentId="1635" CreationDate="2016-08-15T15:29:22.567" Score="7" Body="&lt;p&gt;The most challenging part is this section of the first law:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;or through inaction allow a human being to be harmed&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Humans manage to injure themselves unintentionally in all kinds of ways all the time. A robot strictly following that law would have to spend all its time saving people from their own clumsiness and would probably never get any useful work done. An AI unable to physically move wouldn't have to run around, but it would still have to think of ways to stop all accidents it could imagine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, fully implementing those laws would require very advanced recognition and cognition. (How do you know that industrial machine over there is about to let off a cloud of burning hot steam onto that child who wandered into the factory?) Figuring out whether a human would end up harmed after a given action through some sequence of events becomes an exceptionally challenging problem very quickly.&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-08-15T15:29:22.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1639" PostTypeId="2" ParentId="1635" CreationDate="2016-08-15T15:34:02.513" Score="6" Body="&lt;p&gt;Defining &quot;harm&quot; and in particular, &quot;allowing harm via inaction&quot; in any meaningful way would be difficult. For example, should robots spend all their time flying around attempting to prevent humans from inhaling passive smoke or petrol fumes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, the interpretation of 'conflict' (in either rule 2 or 3) is completely open-ended. Resolving such conflicts seems to me to be &quot;AI complete&quot; in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans have quite good mechanisms (both behavioral and social) for interacting in a complex world (mostly) without harming one another, but these are perhaps not so easily codified. The complex set of legal rules that sit on top of this (polution regulations etc) are the ones that we could most easily program, but they are really quite specialised relative to the underlying physiological and social 'rules'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: From other comments, it seems worth distinguishing between 'all possible harm' and 'all the kinds of harm that humans routinely anticipate'. There seems to be consensus that 'all possible harm' is a non-starter, which still leaves the hard (IMO, AI-complete) task of equaling human ability to predict harm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if we can do that, if we are to treat as actual laws, then we would still need a formal mechanism for conflict resolution (e.g. &quot;Robot, I will commit suicide unless you punch that man&quot;). &lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-17T08:17:01.623" LastActivityDate="2016-08-17T08:17:01.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1640" PostTypeId="2" ParentId="1632" CreationDate="2016-08-15T15:57:15.073" Score="3" Body="&lt;p&gt;Current approaches for learning a language require having a large corpus of that language; it also doesn't seem reasonable to expect that it will ever be possible to learn about language A by extracting information from a corpus from an unrelated language B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you want to learn about human languages in general (what sorts of things are true about grammar, vocabulary, and so on), that relies having many languages as training data, so that you can see the different ways of doing things instead of assuming that the way they're done in English is the way they're done in every language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(There is work in automatic translation that goes from a language to 'concept-space', then goes from that 'concept-space' to another language, so that you can build an English-Chinese translator by building two separate English-Concept and Chinese-Concept translators, instead of ever needing material that directly links English and Chinese. The obvious benefit of this is scalability; in order to make translators for a new language to any other language, you just need to learn that language and the models build themselves.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-15T15:57:15.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1641" PostTypeId="2" ParentId="1630" CreationDate="2016-08-15T16:26:58.840" Score="4" Body="&lt;p&gt;We're definitely nowhere near that level of AI; at best, high-tech solutions like deep convolutional neural nets can help with image recognition and some other algorithms can perform things like robotic movement adequately enough to be useful in some scenarios. None of this is even as sophisticated as the behavior of a flea, but no one refers to insects as &quot;intelligent.&quot; It's exciting stuff that allows us to solve problems that human intelligence often has difficulty with (such as classification of thousands of objects, which would tire an ordinary human mind), but it's nowhere close to replicating our higher brain functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also keep in mind that the Turing test is a poor test of &quot;intelligence&quot; that defies common sense. By the same extension, mistaking a mannequin for a human being in the dark does not mean that the mannequin is actually human. If it were a valid test, then we passed that way back around 1980 with programs like Dear Eliza which were coded in BASIC to regurgitate human speech patterns. There's just no need to come up with a sophisticated argument like Searle's Chinese Room to debunk it, since it's silly on its face; any layman should be able to see right through the Turing Test. If anyone except Turing had come up with this test it would not have received much attention. Turing displayed one-of-a-kind genius when it came to things like computing and cryptography, but like many other experts in such fields, he had a lot of trouble grappling with metaphysics and philosophy. Searle had more common sense, but his Chinese Room example is more of a rebuttal to the Turing Test than a test in and of itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What &quot;intelligence&quot; consists of is ultimately a deep metaphysical question, not a material one. For millennia, trained philosophers have had a lot of trouble assigning clear definitions to concepts like intelligence and consciousness. Until we can answer those questions definitively, using different sets of reasoning skills than scientists, mathematicians and computer specialists are used to employing (just look at how often metaphysics is derided in some of these disciplines) then we cannot say that we have achieved genuine A.I. Until we can define what intelligence is, we cannot say whether or not we've successfully built it; we've not only got the cart before the horse, but have yet to build the cart or see a horse. By the common definitions used in everyday speech we're nowhere near genuine A.I. No one calls cows or sparrows &quot;intelligent,&quot; but our AI today isn't even as sophisticated as the mosquitoes that bite them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's not going to be a popular answer - I'll probably get a dozen downvotes for this, without anyone being able to adequately rebut my contentions, but it needs to be said. There's far too much irrational exuberance and gross overestimation of what we've achieved to date and probably always will be in this field. Historically, researchers in every generation have also grossly underestimated the computing power of the human brain; every decade or so, the estimates of the FLOPS and megabytes have to be drastically revised. We have a poor track record of even getting basic material questions about the human brain right. This clear, consistent pattern of biased overestimation of our success and the lack of any real definition, let alone a test, of intelligence is going to be a serious issue in this forum for its whole existence (assuming it survives the private beta period). We have a whole forum dedicated to a field we can't even define; we can't say for sure what A.I. really is, but we're adamantly certain that we're close to achieving it...!  We cannot say if &quot;brute force algorithms&quot; exist when we're still groping for an understanding of what it is we're trying to force our way into. Certainly, there are brute force methods to solve certain problems, like Deep Blue does at chess - but we cannot say if that qualifies as intelligence or not. It is really not possible to answer questions like this without getting into deep discussions that immediately lend themselves to opinion and debate, which the Turing Test and Searle's Room are clear examples of, in and of themselves. Since implementation details of AI are considered by many to be off-limits here, we're limited mainly to highly speculative posts about tech that often doesn't even work yet (like Google's self-driving cars) and questions like this that we can't answer without first defining intelligence. This is going to be the root of a lot of problems here for a long, long time to come...&lt;/p&gt;&#xA;" OwnerUserId="1427" LastActivityDate="2016-08-15T16:26:58.840" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1644" PostTypeId="1" AcceptedAnswerId="1912" CreationDate="2016-08-16T01:48:02.907" Score="3" ViewCount="356" Body="&lt;p&gt;I'd like to investigate the possibility of achieving similar recognition as it's in &lt;a href=&quot;http://asimo.honda.com/downloads/pdf/asimo-technical-information.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Honda's ASIMO robot&lt;/a&gt;&lt;sup&gt;p.22&lt;/sup&gt; which can interpret the positioning and movement of a hand, including postures and gestures based on visual information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the example of an application of such a recognition system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://asimo.honda.com/downloads/pdf/asimo-technical-information.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UDram.png&quot; alt=&quot;Honda&amp;#39;s ASIMO robot - Recognition of postures and gestures based on visual information&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Image source: &lt;a href=&quot;http://asimo.honda.com/downloads/pdf/asimo-technical-information.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASIMO Featuring Intelligence Technology - Technical Information (PDF)&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, basically, the recognition should detect an indicated location (posture recognition) or respond to a wave (gesture recognition), like a &lt;a href=&quot;https://ai.stackexchange.com/a/1577/8&quot;&gt;Google car&lt;/a&gt; does it (by determining certain patterns).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it known how ASIMO does it, or what would be the closest alternative for postures and gestures recognition to achieve the same results?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-11-06T23:19:53.647" LastActivityDate="2021-04-04T15:14:10.157" Title="How to achieve recognition of postures and gestures?" Tags="&lt;image-recognition&gt;&lt;pattern-recognition&gt;&lt;robotics&gt;&lt;robots&gt;&lt;human-computer-interaction&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="1646" PostTypeId="5" CreationDate="2016-08-16T09:40:12.390" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-16T09:40:12.390" LastActivityDate="2016-08-16T09:40:12.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1647" PostTypeId="4" CreationDate="2016-08-16T09:40:12.390" Score="0" Body="For questions about training networks, rules systems, or other AI system components." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-12-12T03:16:47.483" LastActivityDate="2018-12-12T03:16:47.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1648" PostTypeId="1" AcceptedAnswerId="1649" CreationDate="2016-08-16T12:06:21.700" Score="9" ViewCount="2095" Body="&lt;p&gt;For example, could you provide reasons why a sundial is &lt;em&gt;not&lt;/em&gt; &quot;intelligent&quot;?&#xA;A sundial senses its environment and acts rationally. It outputs the time. It also stores  percepts. (The numbers the engineer wrote on it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What properties of a self driving car would make it &quot;intelligent&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where is the line between non intelligent matter and an intelligent system?&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="2444" LastEditDate="2019-02-21T22:19:34.193" LastActivityDate="2019-02-21T22:19:34.193" Title="What are the criteria for a system to be considered intelligent?" Tags="&lt;definitions&gt;&lt;intelligence-testing&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ClosedDate="2020-08-18T23:48:50.490" ContentLicense="CC BY-SA 4.0" />
  <row Id="1649" PostTypeId="2" ParentId="1648" CreationDate="2016-08-16T16:05:22.283" Score="4" Body="&lt;p&gt;Typically, I think of intelligence in terms of the &lt;em&gt;control&lt;/em&gt; of &lt;em&gt;perception&lt;/em&gt;. [1] A related, but different, definition of intelligence is the (at least partial) restriction of possible future states. For example, an intelligent Chess player is one whose future rarely includes 'lost at chess to a weaker opponent' states; they're able to make changes that move those states to 'won at chess' states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are both broad and continuous definitions of intelligence, where we can talk about differences of degree. A sundial doesn't exert any control over its environment; it passively casts a shadow, and so doesn't have intelligence worth speaking of. A thermostat attached to a heating or cooling system, on the other hand, does exert control over its environment, trying to keep the temperature of its sensor within some preferred range. So a thermostat does have intelligence, but not very much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-driving cars obviously fit those definitions of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] Control is meant in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Control_theory&quot; rel=&quot;nofollow&quot;&gt;control theory&lt;/a&gt;, a branch of engineering that deals with dynamical systems that perceive some fact about the external world and also have a way by which they change that fact. When perception is explicitly contrasted to observations, it typically refers to an abstract feature of observations (you observe the intensity of light from individual pixels, you perceive the apple that they represent) but here I mean it as a superset that includes observation. The thermostat is a dynamical system that perceives temperature and acts to exert pressure on the temperature it perceives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(There's a philosophical point here that the thermostat cares directly about its sensor reading, not whatever the temperature &quot;actually&quot; is. I think that's not something that should be included in intelligence, and should deserve a name of its own, because understanding the difference between perception and reality and seeking to make sure one's perceptions are accurate to reality is another thing that seems partially independent of intelligence.)&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="10" LastEditDate="2016-08-16T19:25:46.880" LastActivityDate="2016-08-16T19:25:46.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1651" PostTypeId="5" CreationDate="2016-08-16T20:37:28.987" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-16T20:37:28.987" LastActivityDate="2016-08-16T20:37:28.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1652" PostTypeId="4" CreationDate="2016-08-16T20:37:28.987" Score="0" Body="For questions about AI that learns without being provided with a set of labels (expected answers) along with the set of input examples." OwnerUserId="145" LastEditorUserId="4302" LastEditDate="2018-09-24T13:04:17.500" LastActivityDate="2018-09-24T13:04:17.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1653" PostTypeId="2" ParentId="1648" CreationDate="2016-08-16T20:46:04.797" Score="2" Body="&lt;p&gt;To ask what makes a system intelligent almost begs the question 'in this context what do we mean by artificially intelligent?' which I think this what this question is really gearing towards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my studies, I've come to see that 'Artificial Intelligence' is a catchy term to use but perhaps misleading, and it conjures up images of these self-driving cars and robots that will take over the earth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've found AI, and 'intelligent' systems moreso represent is an aid or a support that works &lt;em&gt;for&lt;/em&gt; us, rather than one that works &lt;em&gt;because&lt;/em&gt; of us... hear me out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes the jump to an intelligent system for me is the step where the system begins to 'adapt / learn' or otherwise do things I didn't directly tell it to do. With the sundial, I measured and cut every inch of it by hand, and put it in a specific way to do a specific thing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a programmer gets into a car he automated, it may do some things he didn't directly program or maybe couldn't even expect (just one example: querying some database to see lots of people are driving somewhere, discovering a concert is going on there, and asking if the driver wants directions / tickets)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, an intelligent system to me is one that we build in such a way that it educates and supports &lt;em&gt;us&lt;/em&gt;, rather than a system we ourselves 'educate' to do a specific task. Supportive systems that elucidate and adapt and act 'rationally' even when we didn't tell it what 'rational' behaviour was.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastActivityDate="2016-08-16T20:46:04.797" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1654" PostTypeId="2" ParentId="1635" CreationDate="2016-08-16T21:32:55.577" Score="0" Body="&lt;p&gt;I think this is almost a trick question in a sense. Let me explain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For law 1, any AI would abide by the first rule unless it was deliberately created to be malevolent, in that the AI it would understand harm was imminent but do nothing about or would actively attempt to harm. Any 'reasonable' AI would (try its best to) prevent any harm it understood, but couldn't react to imminent harm 'outside it's knowledge', thus satisfying law 1. Any AI that 'tries its best' to prevent harm works here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For law 2, it is simply a matter of design. If one can design an AI capable of parsing and understanding the entirety of human language (beyond just speech), just program it to act accordingly, mindful of the first law. Thus, I think we can develop an AI that will obey every command &lt;em&gt;it understands&lt;/em&gt; but getting it to understand anything and everything I believe is impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For law 3, it rides in the same vein as law 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, I think there is no philosophical problem with implementing such an AI, but that the actual design of such an AI is fundamentally impossible (understanding all possible harms, and all possible commands).&lt;/p&gt;&#xA;" OwnerUserId="1538" LastActivityDate="2016-08-16T21:32:55.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1655" PostTypeId="1" AcceptedAnswerId="1734" CreationDate="2016-08-17T02:02:41.510" Score="10" ViewCount="3293" Body="&lt;p&gt;We can read on &lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow#Tensor_processing_unit_.28TPU.29&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia page&lt;/a&gt; that Google built a custom ASIC chip for machine learning and tailored for TensorFlow which helps to accelerate AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since ASIC chips are specially customized for one particular use without the ability to change its circuit, there must be some fixed algorithm which is invoked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how exactly does the acceleration of AI using ASIC chips work if its algorithm cannot be changed? Which part of it is exactly accelerating?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="9947" LastEditDate="2018-08-27T15:52:18.557" LastActivityDate="2021-03-31T22:25:10.080" Title="How does using ASIC for the acceleration of AI work?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;hardware&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="4" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1658" PostTypeId="1" CreationDate="2016-08-17T03:03:13.243" Score="5" ViewCount="93" Body="&lt;p&gt;I was reading that the &lt;a href=&quot;http://nasa-jsc-robotics.github.io/valkyrie/&quot; rel=&quot;noreferrer&quot;&gt;Valkyrie robot&lt;/a&gt; was originally designed to 'carry out search and rescue missions'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However there were some talks to send it to Mars to assist astronauts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of specific trainings or tasks are planned for 'him' to be able to carry on its own?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/nasa-jsc-robotics&quot; rel=&quot;noreferrer&quot;&gt;NASA-JSC-Robotics at GitHub&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nasa-jsc-robotics.github.io/valkyrie/&quot; rel=&quot;noreferrer&quot;&gt;github.io page&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gitlab.com/nasa-jsc-robotics/valkyrie&quot; rel=&quot;noreferrer&quot;&gt;gitlab page&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-17T09:18:43.313" LastActivityDate="2018-09-10T08:24:59.947" Title="What would the Valkyrie AI robot do on Mars?" Tags="&lt;robotics&gt;&lt;nasa&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1660" PostTypeId="5" CreationDate="2016-08-17T10:18:40.907" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Genetic_algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2019-07-07T19:59:59.370" LastActivityDate="2019-07-07T19:59:59.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1661" PostTypeId="4" CreationDate="2016-08-17T10:18:40.907" Score="0" Body="For questions related to genetic algorithms (GAs), which are a form of evolutionary algorithms. A genetic algorithm is a method (more precisely, a metaheuristic) for solving optimization and search problems based on natural selection processes (that is, they use bio-inspired operators such as mutation, crossover, and selection)." OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2019-07-07T19:59:47.550" LastActivityDate="2019-07-07T19:59:47.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1662" PostTypeId="1" CreationDate="2016-08-17T11:57:42.683" Score="6" ViewCount="190" Body="&lt;p&gt;Do scientists know by what mechanism biological brains/biological neural networks store data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about @kenorbs &lt;a href=&quot;https://ai.stackexchange.com/questions/1656/how-can-nanobot-implants-in-our-brains-connect-to-the-internet&quot;&gt;question&lt;/a&gt; about implanting nanobots to build an AGI on top of human wetware. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only have a vague notion that we store data in our brains by altering synapses? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links, Criticism and Detailed Explanation welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also would love a decent description of how a vanilla Artificial Neural Network stores data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How is data stored in a biological Neural Network?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How is data stored in an Artificial Neural Network?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="157" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-17T12:11:09.200" Title="How do Artificial Neural Networks store data compared to Biological Neural Networks?" Tags="&lt;neural-networks&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1663" PostTypeId="2" ParentId="1662" CreationDate="2016-08-17T12:11:09.200" Score="5" Body="&lt;p&gt;Second question first: Data is stored in an ANN in the form of weights in the adjacency matrix between neurons. During training, these weights are updated by a learning algorithm (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;noreferrer&quot;&gt;backpropagation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First question: according to award-winning neuroscientist Tim Bliss:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;“It’s been accepted really since the turn of the 20th century, since the time of the Spanish neuroscientist Ramón y Cajal, that really the only place where memories can be stored is at synapses, the junctions between nerve cells.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A protein called the NMDA receptor plays a key roll in the strengthening of synaptic connections (which is more broadly achieved by a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Hebbian_theory&quot; rel=&quot;noreferrer&quot;&gt;Hebbian Learning&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-17T12:11:09.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1664" PostTypeId="2" ParentId="211" CreationDate="2016-08-17T14:41:03.707" Score="4" Body="&lt;p&gt;As a person who works with people who work on Watson, perhaps I can give some insight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The name &lt;em&gt;Watson&lt;/em&gt; is casually thrown around a lot whilst many people aren't aware of its evolution into a larger suite of systems and services. We now have Chef Watson, Watson Health, and many other developing projects along the &quot;cognitive&quot; route. &lt;em&gt;Watson&lt;/em&gt; is really an amalgamation and varied application of the different cognitive computing routes IBM is pursuing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, there are many forms of NLP that Watson conducts and has conducted, developed by different teams to fit different processes, interconnected in different ways. Additionally, much (probably all) of it is proprietary/classified since, as one would imagine, ongoing research is constantly being conducted and added to Watson. This is likely your largest obstacle. The precise workings of the NLP of the Jeopardy flavor of Watson are probably themselves still classified (I can't find anything in the time I've just spent looking myself)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, thus, many answers to this question; many outdated, and others not always applicable. The full answer is very complicated and, by the time you find out what the answer is today, it's probably already been advanced. The researches I know are always working on new, cutting-edge algorithms and processes for text classification and the related NLP topics. See, for example, this article &lt;a href=&quot;http://www.aaai.org/Magazine/Watson/watson.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.aaai.org/Magazine/Watson/watson.php&lt;/a&gt;, where it is stated&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For the Jeopardy Challenge, we use more than 100 different techniques for analyzing natural language, identifying sources, finding and generating hypotheses, finding and scoring evidence, and merging and ranking hypotheses.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Watson's metalearner uses multiple trained models to handle different question classes as, for instance, certain scores that may be crucial to identifying the correct answer for a factoid question may not be as useful on puzzle questions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of these techniques have likely already been modified, adapted, or dropped altogether for other methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To point you to more information, take a look at the links below.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;General&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://researcher.watson.ibm.com/researcher/view_group.php?id=2099&quot; rel=&quot;nofollow noreferrer&quot;&gt;The DeepQA Research Team&lt;/a&gt;, specifically, have a look at &lt;a href=&quot;https://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=2099&quot; rel=&quot;nofollow noreferrer&quot;&gt;their publications&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.research.ibm.com/artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM Research AI&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Articles&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aaai.org/Magazine/Watson/watson.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building Watson: An Overview of the DeepQA Project&lt;/a&gt; (2010) written by the IBM Watson Research Team, led by David Ferucci, and published in AI Magazine Fall, 2010.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Videos&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=r7E1TJ1HtM0&quot; rel=&quot;nofollow noreferrer&quot;&gt;How does IBM Watson work?&lt;/a&gt; (2018), by IBM Watson, which states that Watson uses deep learning and transfer learning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Other&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://academic.oup.com/jamia/article/18/5/544/829676&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural language processing: an introduction&lt;/a&gt; (2011), a good but technical paper on natural language understanding (that acknowledges Watson)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="2444" LastEditDate="2019-06-29T23:19:56.893" LastActivityDate="2019-06-29T23:19:56.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1665" PostTypeId="1" AcceptedAnswerId="1666" CreationDate="2016-08-17T14:52:05.960" Score="3" ViewCount="107" Body="&lt;p&gt;My understanding is that &lt;em&gt;Watson&lt;/em&gt; is the name of the computer, and &lt;em&gt;DeepQA&lt;/em&gt; is the name of the software or technology. They are both correlated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any computers/technologies other than &lt;em&gt;Watson&lt;/em&gt; which &lt;strong&gt;are using &lt;em&gt;DeepQA&lt;/em&gt;&lt;/strong&gt;? Or is &lt;em&gt;Watson&lt;/em&gt; the only computer which implements that software/technology?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-29T23:14:16.380" LastActivityDate="2019-06-29T23:14:16.380" Title="Are there any DeepQA-based computers other than Watson?" Tags="&lt;terminology&gt;&lt;watson&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1666" PostTypeId="2" ParentId="1665" CreationDate="2016-08-17T15:58:12.170" Score="3" Body="&lt;p&gt;I cannot say for certain, but I know of no such other uses (I work at the building where Watson is developing but do not directly work with it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DeepQA team's page (&lt;a href=&quot;https://www.research.ibm.com/deepqa/deepqa.shtml&quot; rel=&quot;nofollow&quot;&gt;https://www.research.ibm.com/deepqa/deepqa.shtml&lt;/a&gt;) only ever references Watson as the implementation, and based on the structure of the FAQ there I would imagine they'd be eager to list any interesting other uses it would have, but no such entries exist there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would, however, also note that while DeepQA is IBM's proprietary implementation of QA using deep-learning methods, that approach is becoming more popular and is almost certainly being worked on by other companies and could soon be on other computers, if not already so in some form.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2016-08-17T16:04:38.483" LastActivityDate="2016-08-17T16:04:38.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1667" PostTypeId="2" ParentId="1478" CreationDate="2016-08-17T18:02:40.383" Score="1" Body="&lt;p&gt;The problem detecting NSFW has been around for over two decades.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This study from 2005 about &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F3-540-61123-1_173&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;finding naked people&lt;/em&gt;&lt;/a&gt;, demonstrates a strategy for finding such images based on the color and texture properties to fetch an effective mask for skin regions attempting to group a human figure using geometric constraints on the human structure. This method demonstrated &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;60% precision and 52% recall on a test set of 138 uncontrolled images of naked people.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here are a few figures from the study explaining the algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hMC5ll.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hMC5ll.png&quot; alt=&quot;Finding naked people via AI, neural network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qgjcYl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qgjcYl.png&quot; alt=&quot;Typical control images wrongly classified as containing naked people&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&lt;strong&gt;The following post contains visualizations of nudity for scientific purposes (hover to display):&lt;/sup&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; &lt;a href=&quot;https://i.stack.imgur.com/qlNd8m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qlNd8m.png&quot; alt=&quot;Typical images correctly classified as containing naked people&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A more recent approach is using &lt;a href=&quot;https://ai.stackexchange.com/questions/tagged/conv-neural-network&quot;&gt;convolutional networks&lt;/a&gt;. This &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.683.4319&quot; rel=&quot;nofollow noreferrer&quot;&gt;study from 2014&lt;/a&gt;&lt;sup&gt;&lt;a href=&quot;https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt;&lt;/sup&gt; demonstrated impressive classification performance based on the ImageNet dataset. It's not clear '&lt;em&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/1479/8&quot;&gt;how and why&lt;/a&gt;&lt;/em&gt; they perform so well', however they can be used for classification of images with a very low error rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further details, check: &lt;a href=&quot;http://blog.clarifai.com/what-convolutional-neural-networks-see-at-when-they-see-nudity/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What convolutional neural networks look at when they see nudity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find the code example and the heatmap for how convnets see NSFW in the above link.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-25T09:59:13.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1668" PostTypeId="5" CreationDate="2016-08-17T20:33:02.043" Score="0" Body="&lt;p&gt;Chess is non-chance, perfect information, partisan, sequential two-player game played on an 8×8 grid. This tag is to be used for questions about chess playing engines based on artificial intelligence. &lt;em&gt;(Questions purely about chess are off-topic but can be asked on our &lt;a href=&quot;https://chess.stackexchange.com/&quot;&gt;SE:Chess&lt;/a&gt;.)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Games can result in a win/loss for either competitor, stalemate, or no result (&lt;a href=&quot;https://en.wikipedia.org/wiki/Infinite_loop&quot; rel=&quot;nofollow noreferrer&quot;&gt;infinite loop&lt;/a&gt;).  Chess is conducted under &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal_play_convention&quot; rel=&quot;nofollow noreferrer&quot;&gt;normal play&lt;/a&gt;, formally defined as victory for the last player to take a turn, except in the case of stalemate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chess has been studied extensively, and has been extensively utilized in Artificial Intelligence. The latest and strongest generation of chess engines, &lt;a href=&quot;https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha Zero&lt;/a&gt; and &lt;a href=&quot;https://lczero.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leela&lt;/a&gt;, utilize machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chess is considered &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#Partially_solved_games&quot; rel=&quot;nofollow noreferrer&quot;&gt;partially solved&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SEE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chess&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chess (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Blue (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stockfish_(chess)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stockfish (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/541276/deep-learning-machine-teaches-itself-chess-in-72-hours-plays-at-international-master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/609736/alpha-zeros-alien-chess-shows-the-power-and-the-peculiarity-of-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chess_engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chess Engines (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2019-12-20T22:31:28.543" LastActivityDate="2019-12-20T22:31:28.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1669" PostTypeId="4" CreationDate="2016-08-17T20:33:02.043" Score="0" Body="For questions about chess playing engines such as AlphaZero and Leela, and questions about AI methods and theory related to Chess. (Questions purely about chess are off-topic but can be asked on SE:Chess.)" OwnerUserId="145" LastEditorUserId="1671" LastEditDate="2019-12-20T22:34:41.830" LastActivityDate="2019-12-20T22:34:41.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1670" PostTypeId="2" ParentId="1333" CreationDate="2016-08-18T00:46:03.973" Score="1" Body="&lt;p&gt;I know it seems like a cop-out answer to every question on AI, but &quot;it depends&quot;.  For example, if the bulk of the storage space is storing learned concepts, and attributes of example entities, then it stands to reason that concepts and entities could be reused.  In that scenario, learning from an additional 10G of text would use less storage than the original.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OTOH, as others have said, it could be that the storage is mostly storing the &lt;em&gt;links&lt;/em&gt; between things, in which case the number of links will likely grow exponentially. In that case, the second batch of &quot;knowledge&quot; would add more storage requirements than the first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it would come down to &quot;what exactly is the system learning, and how does it represent what it learned?&quot;  And that answer will vary from system to system.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-18T00:46:03.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1671" PostTypeId="2" ParentId="1535" CreationDate="2016-08-18T09:52:04.357" Score="2" Body="&lt;p&gt;I'll try to do something intuitive; Each node in a neural network is referred to as a neuron. To understand what's going on under the hood of a neural network you only really need to understand an individual neuron. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now each neuron has a set of inputs (other neurons; they can potentially be the inputs to the network as a whole as well), and each input has a weight associated with it. Every time the network is used, each neuron computes its output as the weighted sum of its inputs passed through some gate (The &quot;Activation Function&quot;, a mathematical function designed to get a particular behaviour. For example sigmoid AF takes an input of any size and transforms it into an output in the range [0, 1].) Obviously, this is driven from the inputs to the neural network so that no neuron is computing its outputs before all of the neurons used as its inputs have done the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you refer to the value of a node; there isn't a single value. Each neuron has several weights associated with it as it may be the input to several other neurons, and each of those neurons assigns it a different weight. Instead, it is better to thing of a neural network as a directed graph of nodes (neurons) which are labelled with a particular activation function, and edges (input/output connections) which are labelled with a particular weight. While the structure and activation functions used in the neural networks is a matter of topology design, there are a number of algorithms for designing a ANN for a particular topology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most commonly used (and possibly easiest to explain) is backpropagation. In pseudo-Layman's terms we start off with random weights on all edges in the network. We then compute the output of the network for a training set (a set of known input/output pairs). By careful choice of activation function, it is possible to differentiate the error (computed analogously to the expected output minus the actual output of the ANN for each input/output pair) with respect to the weights of the neural network. This allows us to compute a gradient for each weight; the direction in which we can move the weight to reduce the error on the training set. By doing this until we find an optima (a point where all movements increase error), we can find some 'good' configuration of weights for that particular ANN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a nice tutorial on BP here &lt;a href=&quot;http://www.cse.unsw.edu.au/~cs9417ml/MLP2/BackPropagation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cse.unsw.edu.au/~cs9417ml/MLP2/BackPropagation.html&lt;/a&gt;. The diagram associated with it does nicely to explain my point:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/O2d6L.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O2d6L.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-08-18T09:52:04.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1672" PostTypeId="5" CreationDate="2016-08-18T10:09:51.883" Score="0" Body="&lt;p&gt;Isaac Asimov created the fundamental &lt;em&gt;Three Laws of Robotics&lt;/em&gt;, by which the robots in his &lt;em&gt;Robot&lt;/em&gt; series were bound to obey.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The three laws are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) A robot must not harm a human being, or through inaction allow a human to come to harm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) A robot must obey orders given by a human, as long as that does not contradict with the First Law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) A robot must protect its own existence, as long as that does not contradict with Rules One or Two.&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-09-11T14:53:59.113" LastActivityDate="2016-09-11T14:53:59.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1673" PostTypeId="4" CreationDate="2016-08-18T10:09:51.883" Score="0" Body="For questions about Asimov's Laws in real life; to ask a question about something in Asimov's books, see https://scifi.stackexchange.com or https://literature.stackexchange.com/" OwnerUserId="145" LastEditorUserId="1671" LastEditDate="2017-08-27T20:36:00.387" LastActivityDate="2017-08-27T20:36:00.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1674" PostTypeId="5" CreationDate="2016-08-18T10:12:44.613" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-18T10:12:44.613" LastActivityDate="2016-08-18T10:12:44.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1675" PostTypeId="4" CreationDate="2016-08-18T10:12:44.613" Score="0" Body="DeepDream is an art-AI created by Google. Use this tag for questions about that. Use the [deepdreaming] tag for the process." OwnerUserId="145" LastEditorUserId="4709" LastEditDate="2018-10-08T11:11:34.087" LastActivityDate="2018-10-08T11:11:34.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1676" PostTypeId="5" CreationDate="2016-08-18T10:26:26.650" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-18T10:26:26.650" LastActivityDate="2016-08-18T10:26:26.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1677" PostTypeId="4" CreationDate="2016-08-18T10:26:26.650" Score="0" Body="Deepdreaming is a method for generating images by a trained neural network. For questions about Google's software, use [deepdream]." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-18T11:35:02.287" LastActivityDate="2016-08-18T11:35:02.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1678" PostTypeId="1" AcceptedAnswerId="5284" CreationDate="2016-08-18T14:00:52.177" Score="7" ViewCount="233" Body="&lt;p&gt;There is a study about &lt;a href=&quot;http://www.aclweb.org/anthology/P/P02/P02-1031.pdf&quot; rel=&quot;noreferrer&quot;&gt;The Necessity of Parsing for Predicate Argument Recognition&lt;/a&gt;, however I couldn't find much information about 'Predicate Argument Recognition' which could explain it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is it exactly and how does it work, briefly?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-18T14:13:25.083" LastActivityDate="2018-02-12T12:57:15.110" Title="What is predicate argument recognition?" Tags="&lt;definitions&gt;&lt;natural-language-processing&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1679" PostTypeId="5" CreationDate="2016-08-18T14:08:05.760" Score="0" Body="&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural language processing (NLP)&lt;/a&gt; at Wikipedia.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2019-06-15T08:14:20.170" LastActivityDate="2019-06-15T08:14:20.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1680" PostTypeId="4" CreationDate="2016-08-18T14:08:05.760" Score="0" Body="For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data." OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-03-14T18:42:12.363" LastActivityDate="2020-03-14T18:42:12.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1681" PostTypeId="5" CreationDate="2016-08-18T14:11:36.997" Score="0" Body="&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_linguistics&quot; rel=&quot;nofollow&quot;&gt;Computational linguistics&lt;/a&gt; at Wikipedia.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-18T16:25:47.707" LastActivityDate="2016-08-18T16:25:47.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1682" PostTypeId="4" CreationDate="2016-08-18T14:11:36.997" Score="0" Body="An interdisciplinary field concerned with the statistical or rule-based modeling of natural language from a computational perspective." OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-18T16:25:59.457" LastActivityDate="2016-08-18T16:25:59.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1683" PostTypeId="5" CreationDate="2016-08-18T14:29:03.443" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-18T14:29:03.443" LastActivityDate="2016-08-18T14:29:03.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1684" PostTypeId="4" CreationDate="2016-08-18T14:29:03.443" Score="0" Body="Refers to kinds of data with a high level of organization." OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-18T16:25:52.160" LastActivityDate="2016-08-18T16:25:52.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1685" PostTypeId="1" CreationDate="2016-08-18T14:30:05.433" Score="5" ViewCount="471" Body="&lt;p&gt;The Wit.ai is a Siri-like voice interface which can can parse messages and predict the actions to perform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;a href=&quot;https://labs.wit.ai/demo/index.html&quot; rel=&quot;nofollow&quot;&gt;demo site powered by Wit.ai&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does it understand the spoken sentences and convert them into structured actionable data? Basically, how does it know what to do?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="4302" LastEditDate="2018-10-08T12:53:11.510" LastActivityDate="2018-10-08T12:53:11.510" Title="How does Wit.ai convert sentences into structured data?" Tags="&lt;natural-language-processing&gt;&lt;voice-recognition&gt;&lt;structured-data&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1686" PostTypeId="1" AcceptedAnswerId="1868" CreationDate="2016-08-18T14:39:50.547" Score="1" ViewCount="38" Body="&lt;p&gt;In 2014 &lt;a href=&quot;https://techcrunch.com/2014/02/06/linkedin-snatches-up-data-savvy-job-search-startup-bright-com-for-120m-in-its-largest-acquisition-to-date/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linkedin acquired Bright.com&lt;/a&gt;, for $120 million and it is using AI and big data algorithms to connect users.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bright also throws in a little Klout, ranking people by a “Bright score” which it uses to assess how strong the chemistry is between a user and a particular job.&lt;/p&gt;&#xA;&lt;p&gt;It also takes into account historical hiring patterns into its matching, along with account location, a user’s past experience and synonyms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In brief, is it known (based on some research papers) how such algorithm works which aiming at scoring 'chemistry' between users and their jobs?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-09-06T17:31:44.187" Title="How does 'Bright score' assess how strong the connection is between users and their jobs?" Tags="&lt;social&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1687" PostTypeId="1" AcceptedAnswerId="1688" CreationDate="2016-08-18T14:53:13.537" Score="0" ViewCount="227" Body="&lt;p&gt;According to this &lt;a href=&quot;http://mashable.com/2014/01/06/pinterest-acquires-visualgraph/&quot; rel=&quot;nofollow&quot;&gt;article&lt;/a&gt;, Pinterest acquired VisualGraph, an image recognition and visual search technology startup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does Pinterest apply VisualGraph technology for machine vision, image recognition and visual search in order to classify the images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, how do they predict the image categories? Based on what features?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-18T15:05:17.923" LastActivityDate="2016-09-14T19:34:59.007" Title="How does Pinterest decipher what's on unmarked pictures and categorize them?" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1688" PostTypeId="2" ParentId="1687" CreationDate="2016-08-18T17:03:13.773" Score="4" Body="&lt;p&gt;One of the &lt;em&gt;Pinterest's&lt;/em&gt; white paper about &lt;a href=&quot;https://engineering.pinterest.com/sites/engineering/files/article/fields/field_image/human-curation-convnets%20(1).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Curation and Convnets powering item-to-item recommendations&lt;/a&gt;&lt;sup&gt;&lt;a href=&quot;https://arxiv.org/abs/1511.04003&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv&lt;/a&gt;&lt;/sup&gt; describes implementation of convolutional neural network (CNN) based visual features (&lt;a href=&quot;http://www.robots.ox.ac.uk/~vgg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG&lt;/a&gt;&lt;sup&gt;&lt;a href=&quot;https://arxiv.org/abs/1405.3531&quot; rel=&quot;nofollow noreferrer&quot;&gt;2014&lt;/a&gt;&lt;/sup&gt;, &lt;a href=&quot;http://arxiv.org/abs/1506.01497&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faster R-CNN&lt;/a&gt;). This demonstrates the effectiveness of it (such image or object representations) which can improve user engagement. The visual features are computed using the process described in the &lt;a href=&quot;http://arxiv.org/abs/1505.07647&quot; rel=&quot;nofollow noreferrer&quot;&gt;previous study about visual search at &lt;em&gt;Pinterest&lt;/em&gt;&lt;/a&gt; and can be used for more targeted features to be computed for related pins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the examples of detected visual objects from Pinterest's object detection pipeline:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q1Tvr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q1Tvr.jpg&quot; alt=&quot;Fig. 6. Examples of detected visual objects from Pinterest’s object detection pipeline. Detection of objects allows for more targeted visual features to be computed for Related Pins&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Image source: &lt;a href=&quot;https://engineering.pinterest.com/sites/engineering/files/article/fields/field_image/human-curation-convnets%20(1).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Curation and Convnets: Powering&#xA;Item-to-Item Recommendations on Pinterest&lt;/a&gt;, Page 4, Fig. 6&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images are categorized by using dominant visual objects (individual objects seen in the image which passes a confidence threshold in Faster R-CNN) using fine-tuned VGG reranking variant. This allows Pinterest to introduce features such real-time recommendations for the users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check also this blog entry: &lt;a href=&quot;https://engineering.pinterest.com/blog/building-scalable-machine-vision-pipeline&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building a scalable machine vision pipeline&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-23T02:04:02.893" LastActivityDate="2016-08-23T02:04:02.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1689" PostTypeId="1" AcceptedAnswerId="1690" CreationDate="2016-08-18T17:15:44.240" Score="1" ViewCount="193" Body="&lt;p&gt;Wolfram Language Image Identification Project launched an &lt;a href=&quot;https://www.imageidentify.com/&quot; rel=&quot;nofollow&quot;&gt;Image Identify site&lt;/a&gt; demo which returns the top predicted tags for the photos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does it work, briefly? I mean what type of learning vision technologies are used to analyze, recognize and understand the content of an image?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-23T20:29:03.600" LastActivityDate="2016-08-23T20:29:03.600" Title="How does Wolfram's Image Identification Project work?" Tags="&lt;image-recognition&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1690" PostTypeId="2" ParentId="1689" CreationDate="2016-08-18T17:15:44.240" Score="2" Body="&lt;p&gt;The ImageIdentify project uses the highly automated &quot;&lt;a href=&quot;https://www.wolfram.com/algorithmbase/&quot; rel=&quot;nofollow&quot;&gt;superfunctions&lt;/a&gt;&quot; and as part of Wolfram Language API integration. It relies on a complex collection of meta-algorithms and built-in '&lt;a href=&quot;http://www.wolfram.com/knowledgebase/&quot; rel=&quot;nofollow&quot;&gt;knowledge&lt;/a&gt;'. It has a built-in classifier trained from a large dataset using &lt;a href=&quot;http://www.wolfram.com/data-framework/&quot; rel=&quot;nofollow&quot;&gt;Wolfram Data Framework&lt;/a&gt; (WDF). However the main classifier is based on the deep neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.imageidentify.com/about/how-it-works&quot; rel=&quot;nofollow&quot;&gt;How the Wolfram Language Image Identification Project Works&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm isn't perfect and misidentification are more likely to be caused by 'irrelevant objects repeatedly being in training images for a particular type of object'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more at &lt;a href=&quot;http://blog.stephenwolfram.com/2015/05/wolfram-language-artificial-intelligence-the-image-identification-project/&quot; rel=&quot;nofollow&quot;&gt;Wolfram Language Artificial Intelligence: The Image Identification Project&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2016-08-19T17:01:03.857" LastActivityDate="2016-08-19T17:01:03.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1691" PostTypeId="1" AcceptedAnswerId="1704" CreationDate="2016-08-18T17:42:58.053" Score="5" ViewCount="213" Body="&lt;p&gt;I've &lt;a href=&quot;https://www.imageidentify.com/result/0lkzuttdxipub&quot; rel=&quot;nofollow noreferrer&quot;&gt;uploaded a picture&lt;/a&gt; to Wolfram's ImageIdentify of graffiti on the wall, but it recognized it as 'monocle'. Secondary guesses were 'primate', 'hominid', and 'person', so not even close to 'graffiti' or 'painting'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it by design, or there are some &lt;strong&gt;methods to teach a convolutional neural network (CNN) to reason and be aware of a bigger picture context&lt;/strong&gt; (like mentioned graffiti)? Currently it seems as if it's detecting literally &lt;em&gt;what is depicted in the image&lt;/em&gt;, not &lt;em&gt;what the image actually is&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/akquMm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/akquMm.png&quot; alt=&quot;Wolfram&amp;#39;s Image Identify: monocle/graffiti&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be the same problem as mentioned &lt;a href=&quot;https://ai.stackexchange.com/a/1533/8&quot;&gt;here&lt;/a&gt;, that DNN are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Learning to detect jaguars by matching the unique spots on their fur while ignoring the fact that they have four legs.&lt;sup&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/1533/8&quot;&gt;2015&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If it's by design, maybe there is some better version of CNN that can perform better?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-23T00:14:37.027" Title="How to make convnets aware what the image actually is, not what is depicted on it?" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1692" PostTypeId="5" CreationDate="2016-08-18T18:35:05.607" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-18T18:35:05.607" LastActivityDate="2016-08-18T18:35:05.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1693" PostTypeId="4" CreationDate="2016-08-18T18:35:05.607" Score="0" Body="For questions that involve the comparison of two AI concepts, terms or expressions. An example of such a question is: how does machine learning compare to deep learning?" OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2019-06-22T13:01:09.593" LastActivityDate="2019-06-22T13:01:09.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1694" PostTypeId="1" AcceptedAnswerId="1695" CreationDate="2016-08-18T19:16:29.577" Score="9" ViewCount="452" Body="&lt;p&gt;An AI agent is often thought of having &quot;sensors&quot;, &quot;a memory&quot;, &quot;machine learning processors&quot; and &quot;reaction&quot; components. However, a machine with these does not necessarily become a self-programming AI agent. Beyond the parts mentioned above, is there any other elements or details necessary to make a machine capable of being a self-programming AI agent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;a href=&quot;http://www.iiim.is/wp/wp-content/uploads/2011/05/goertzel-agisp-2011.pdf&quot; rel=&quot;noreferrer&quot;&gt;a paper from 2011&lt;/a&gt; declared that solving the optimization problem of maximizing the intelligence is a must-have feature for the self-programming process, as quoted below:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A system is said to carry out an instance of self-programming when it undergoes learning regarding some element of its &quot;cognitive infrastructure&quot;, where the latter is defined as the fuzzy set of &quot;intelligence-critical&quot; features of the system; and the intelligence-criticality of a system feature is defined as its &quot;feature quality,&quot; considered from the perspective of solving the optimization problem of maximizing the intelligence of a multi-feature system.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, this description of &quot;optimization of intelligence&quot; is vague. Can anyone give a clear definition or better summary for the necessary components for self-programming agents?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;This question is from the 2014 closed beta, with the asker having a UID of 23.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-20T20:19:43.133" LastActivityDate="2016-08-20T20:19:43.133" Title="What are the necessary components to make AI agent self-programming-capable?" Tags="&lt;machine-learning&gt;&lt;computer-programming&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1695" PostTypeId="2" ParentId="1694" CreationDate="2016-08-18T19:51:21.833" Score="5" Body="&lt;p&gt;At the highest level, all it needs is for the various systems already discussed to incorporate code objects. If it can interpret its source code / model architecture from the formatted text objects underpinning them, can 'understand' them in terms of having a useful ML model, and alter the code with its reaction, then it can self-program. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the basic loop behind a recursively improving intelligence is simple. It examines itself, writes a new version, and then that new version examines itself and writes a new version, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difficult component comes at lower levels. We don't need to invent a new concept like 'sensor,' what we need to do is build very, very sophisticated sensors that are equal to the task of understanding code well enough to detect and write improvements.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-18T19:51:21.833" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1696" PostTypeId="2" ParentId="1691" CreationDate="2016-08-18T20:40:24.213" Score="3" Body="&lt;p&gt;You seem to be wanting some description of the 'style' of an image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make that work in general, I'd guess that would actually require quite a lot of pre-processing to present 'texture elements' (rather than pixels) as the basic features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is quite speculative, but one approach might be to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterated_function_system&quot; rel=&quot;nofollow&quot;&gt;Iterated Function Systems&lt;/a&gt; as a means of extracting these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether 'spatial adjacency' (and hence CNN) is then the best approach to make higher-level decisions about these elements is (AFAIK) a matter for experiment.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-18T20:40:24.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1697" PostTypeId="2" ParentId="1685" CreationDate="2016-08-19T16:47:20.397" Score="2" Body="&lt;p&gt;I can't speak to wit.ai specifically, but I can tell you a little bit about how similar applications work. Specifically, I can talk a bit about &lt;a href=&quot;http://stanbol.apache.org&quot; rel=&quot;nofollow&quot;&gt;Apache Stanbol&lt;/a&gt; which also converts free text into structured data.   That said, I should prefix this by saying there isn't just one way to &quot;get there from here.&quot;  Many techniques could be part of a stack for accomplishing this goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, in the case of Stanbol, they run the text through multiple processing engines, sequentially, with different engines affecting the final output.  One engine simply does &lt;a href=&quot;https://en.wikipedia.org/wiki/Named-entity_recognition&quot; rel=&quot;nofollow&quot;&gt;Named Entity Recognition&lt;/a&gt; using OpenNLP.  This identities discrete named &quot;things&quot; - people, places, companies, etc.  Another engine does entity matching with a pre-established database of entities - specifically (in the out-of-the-box configuration) a dump of entities from &lt;a href=&quot;http://www.dbpedia.org&quot; rel=&quot;nofollow&quot;&gt;DBPedia&lt;/a&gt;.  Where a match is found, the text from the original input is assigned to the entity.  In the case of a collision, it assigns a weight to the mapping so any downstream consumers can use probabilistic techniques to select the &quot;correct&quot; mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, of course, more details that I left out.  Before NER can happen there is parsing and tokenizing and other NLP activities.  But a big part of the basic process is doing NER and then doing the entity matching.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in the case of Stanbol, you can add your own entities and corresponding structured data, as well as your own engines.  So, for example, if you wanted to write an engine based on neural networks / deep learning, and plug that in, you could.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-19T16:47:20.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1698" PostTypeId="2" ParentId="1481" CreationDate="2016-08-19T16:51:02.167" Score="5" Body="&lt;p&gt;This &lt;a href=&quot;http://dx.doi.org/10.1109/TPAMI.2012.59&quot; rel=&quot;nofollow noreferrer&quot;&gt;study from 2012&lt;/a&gt; uses 3D &lt;a href=&quot;https://ai.stackexchange.com/questions/tagged/conv-neural-network&quot;&gt;convolutional neural networks (CNN)&lt;/a&gt; for automated recognition of human actions in surveillance videos. The 3D CNN model extracts features from both the spatial and the temporal dimensions by performing 3D convolutions, thereby capturing the motion information encoded in multiple adjacent frames. A very similar deep learning approach based on 3D CNN is demonstrated in the &lt;a href=&quot;http://liris.cnrs.fr/Documents/Liris-5228.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LIRIS and Orange Labs study from 2011&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://www.robots.ox.ac.uk/~vgg/publications/2014/Simonyan14b/simonyan14b.pdf.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Oxford study from 2014&lt;/a&gt; also uses a similar approach, but with two-stream CNN which incorporates spatial and temporal networks which can achieve good performance despite having limited training data. It recognises action from motion in the form of dense optical flow. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RWcT3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RWcT3.png&quot; alt=&quot;Optical flow using ConvNets&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4224216&quot; rel=&quot;nofollow noreferrer&quot;&gt;Another study from 2007&lt;/a&gt; demonstrates a method by detecting human falls based on a combination of motion history and human shape variation by analysing the video frames. It uses Motion History Image (MHI) to quantify the motion of the person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hzn4z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hzn4z.png&quot; alt=&quot;Motion history image (MHI)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: &lt;a href=&quot;https://github.com/harishrithish7/Fall-Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;harishrithish7/Fall-Detection&lt;/a&gt; at GitHub&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;An alternative general approach could be action detection based on the posture using DNN. See: &lt;a href=&quot;https://ai.stackexchange.com/q/1644/8&quot;&gt;How to achieve recognition of postures and gestures?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-24T08:48:23.077" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1699" PostTypeId="2" ParentId="1481" CreationDate="2016-08-19T16:52:09.607" Score="5" Body="&lt;p&gt;There are several approaches as to how this can be achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One recent study from 2015 about &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-319-09396-3_9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Action Recognition in Realistic Sports Videos&lt;/a&gt;&lt;sup&gt;&lt;a href=&quot;http://cs.stanford.edu/~amirz/index_files/Springer2015_action_chapter.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt;&lt;/sup&gt; uses the action recognition framework based on the three main steps of feature extraction (shape, post or contextual information), dictionary learning to represent a video, and classification (&lt;a href=&quot;https://en.wikipedia.org/wiki/Bag-of-words_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;BoW framework&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few examples of methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Spatio-Temporal Structures of Human Poses&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AgfDk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AgfDk.png&quot; alt=&quot;K. Soomro and A.R. Zamir - action recognition - figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a joint shape-motion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1qR9x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1qR9x.png&quot; alt=&quot;K. Soomro and A.R. Zamir - action recognition - figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Multi-Task Sparse Learning (MTSL)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hierarchical Space-Time Segments&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jBMuj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jBMuj.png&quot; alt=&quot;K. Soomro and A.R. Zamir - Extracted segments from video frames&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Spatio-Temporal Deformable Part Models (SDPM)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5Zehk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Zehk.jpg&quot; alt=&quot;K. Soomro and A.R. Zamir - Action localization results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are the results based on training of 10 action classes based on the UCF sports dataset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v77Pv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v77Pv.png&quot; alt=&quot;UCF Sports Dataset: sample frames of 10 action classes along with their bounding box annotations of the humans shown in yellow&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-319-09396-3_9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Action Recognition in Realistic Sports Videos&lt;/a&gt;.&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="145" LastEditDate="2016-08-24T08:26:21.633" LastActivityDate="2016-08-24T08:26:21.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1700" PostTypeId="1" CreationDate="2016-08-19T18:08:52.937" Score="19" ViewCount="825" Body="&lt;p&gt;In a &lt;a href=&quot;http://www.wsj.com/articles/whats-next-for-artificial-intelligence-1465827619&quot;&gt;recent Wall Street Journal article&lt;/a&gt;, Yann LeCunn makes the following statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The next step in achieving human-level ai is creating intelligent—but not autonomous—machines. The AI system in your car will get you safely home, but won’t choose another destination once you’ve gone inside. From there, we’ll add basic drives, along with emotions and moral values. If we create machines that learn as well as our brains do, it’s easy to imagine them inheriting human-like qualities—and flaws. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Personally, I have generally taken the position that talking about emotions for artificial intelligences is silly, because there would be no &lt;em&gt;reason&lt;/em&gt; to create AI's that experience emotions.  Obviously Yann disagrees.  So the question is:  what end would be served by doing this?  Does an AI &lt;em&gt;need&lt;/em&gt; emotions to serve as a useful tool?  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2214" LastEditDate="2016-09-10T19:05:28.870" LastActivityDate="2018-01-23T06:34:49.923" Title="What purpose would be served by developing AI's that experience human-like emotions?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;" AnswerCount="11" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1701" PostTypeId="1" AcceptedAnswerId="1702" CreationDate="2016-08-19T18:40:46.010" Score="4" ViewCount="932" Body="&lt;p&gt;Inspired by &lt;a href=&quot;https://ai.stackexchange.com/q/1481/8&quot;&gt;this discussion&lt;/a&gt; about recognizing human actions, I have found the &lt;a href=&quot;https://github.com/harishrithish7/Fall-Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fall-Detection&lt;/a&gt; project which detects humans falling on the ground from a CCTV camera feed, and which can consider alerting the hospital authorities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, are there any existing real-life implementations or research projects &lt;strong&gt;which specifically use live video feed from the surveillance cameras in order to detect crime&lt;/strong&gt; using convnets (or similar approaches)? If so, how do they work, briefly? Do they automatically inform the police about the crime with the details what happened and where?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example car accidents, physical assaults, robberies, violent disturbances, weapon attacks, etc.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-08-24T15:02:43.753" Title="Applications of CNN for detecting crime from video surveillance cameras" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;action-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1702" PostTypeId="2" ParentId="1701" CreationDate="2016-08-19T20:41:14.997" Score="4" Body="&lt;p&gt;After a bit of research I found something kind of close:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.techinsider.io/security-cameras-use-artificial-intelligence-to-detect-crime-2015-8&quot; rel=&quot;nofollow&quot;&gt;Artificially intelligent security cameras are spotting crimes before they happen&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.dailymail.co.uk/sciencetech/article-2154861/U-S-surveillance-cameras-use-eyes-pre-crimes-detecting-suspicious-behaviour-alerting-guards.html&quot; rel=&quot;nofollow&quot;&gt;New surveillance cameras will use computer eyes to find 'pre crimes' by detecting suspicious behaviour and calling for guards&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.dailymail.co.uk/sciencetech/article-2154861/U-S-surveillance-cameras-use-eyes-pre-crimes-detecting-suspicious-behaviour-alerting-guards.html&quot; rel=&quot;nofollow&quot;&gt;CCTV 'fightcams' detect violence 'before it happens'  at Dailymail&lt;/a&gt;, also check at &lt;a href=&quot;http://www.telegraph.co.uk/news/uknews/crime/11407094/CCTV-fightcams-detect-violence-before-it-happens.html&quot; rel=&quot;nofollow&quot;&gt;Telegraph&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;They, however, makes no mention of what specific methods they use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a crime detection system as that is written does not exist, but abnormal behaviour detection systems do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An accurate generalized system seems intuitively infeasible, however. Commiting a crime, unlike falling, is a complex behavior, and takes so many forms. A camera watching a store's counter like at a 7-11 could perhaps see that the 'customer''s arm is strangely reaching across the counter, and the attendant is moving a lot more than usual suddenly, but aside from very specific cases like this such a system is currently quite unfeasible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crimes are unusual, relatively speaking, and their dramatic nature means that even the simplest crimes play out in very different ways. Perhaps you could in this case you could try to look for images of a gun, or someone with their hands up. So, looking for unusual, &lt;em&gt;detectable&lt;/em&gt; behavioural mannerisms may be possible, but not crime detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, while you may be able to make (possibly pretty good) systems to detect specific crimes in specific environments, that's all we got for now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. - Do these camera's also get audio signals? That is also an interesting facet to consider (&quot;PUT YOUR HANDS UP / GIVE ME ALL YOUR MONEY&quot;)&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="8" LastEditDate="2016-08-23T02:13:32.537" LastActivityDate="2016-08-23T02:13:32.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1703" PostTypeId="2" ParentId="1700" CreationDate="2016-08-19T20:54:49.910" Score="11" Body="&lt;p&gt;The answer to this question, unlike many on this board, I think is definitive. No. We don't &lt;em&gt;need&lt;/em&gt; AI's to have emotion to be useful, as we can see by the numerous amount of AI's we already have that are useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to further address the question, we can't &lt;em&gt;really&lt;/em&gt; give AI's emotions. I think the closest we can get would be 'Can we make this AI act in a way a human would if that human was &lt;code&gt;insert emotion&lt;/code&gt;?'. I guess in a sense, that &lt;em&gt;is&lt;/em&gt; having emotion, but that's a whole other discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And to what end? The only immediate reason coming to mind would be to create more lifelike companions or interactions, for the purposes of video games or other entertainment. A fair goal, &lt;br&gt;but far from necessary. Even considering an AI-imbued greeting robot in the lobby of some building, we'd probably only ever want it to act cordial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yann says that super-advanced AI would lead to more human-like qualities &lt;em&gt;and&lt;/em&gt; flaws. I think it's more like it would 'give our AI's more human-like qualities &lt;em&gt;or in other words&lt;/em&gt; flaws'. People have a tendency to act irrationally when sad or angry, and for the most part we only want rational AI. &lt;br&gt;To err is human, as they say.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of AI's and learning algorithms is to create systems that act or 'think' like humans, but better. Systems that can adapt or evolve, while messing up as little as possible. Emotive AI has uses, but it's certainly not a prerequisite for a useful system.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2018-01-23T06:34:49.923" LastActivityDate="2018-01-23T06:34:49.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1704" PostTypeId="2" ParentId="1691" CreationDate="2016-08-19T23:40:48.100" Score="2" Body="&lt;p&gt;Wolfram's image id system is specifically meant to figure out what the image is depicting, not the medium. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get what you want you'd simply have to create your own system where the training data is labeled by the medium rather than the content, and probably fiddle with it to pay more attention to texture and things as such as that. The neural net doesn't care which we want - it has no inherent bias. It just knows what it's been trained for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's really all there is to it. It's all to do with the training labels and the focus of the system (e.g. a system that looks for edge patterns that form shapes, compared to a system that checks if the lines in the image are perfectly computer-generated straight and clean vs imperfect brush strokes vs spraypaint).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you want me to tell you how to build that system, I'm not the right person to ask haha&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2016-08-19T23:46:11.350" LastActivityDate="2016-08-19T23:46:11.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1705" PostTypeId="1" CreationDate="2016-08-21T20:03:02.017" Score="9" ViewCount="1987" Body="&lt;p&gt;I'm trying to come up with the right algorithm for a system in which the user enters a few symptoms and the system has to predict or determine the likelihood that a few selected symptoms are associated with those existing in the system. Then after associating them, the result or output should be a specific disease for the symptoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is comprised of a series of diseases with each assigned to specific symptoms, which also exist in the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that the user entered the following input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A, B, C, and D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first thing the system should do is check and associate each symptom (in this case represented by alphabetical letters) individually against a data-table of symptoms that already exist. And in cases where the input doesn't exist, the system should report or send feedback about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also, let's say that &lt;code&gt;A and B&lt;/code&gt; was in the data-table, so we are 100% sure that they're valid or exist and the system is able to give out the disease based on the input. Then let's say that the input now is &lt;code&gt;C and D&lt;/code&gt; where &lt;code&gt;C&lt;/code&gt; doesn't exist in the data-table, but there is a possibility that &lt;code&gt;D&lt;/code&gt; exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't give &lt;code&gt;D&lt;/code&gt; a score of 100%, but maybe something lower (let's say 90%). Then &lt;code&gt;C&lt;/code&gt; just doesn't exist at all in the data-table. So, &lt;code&gt;C&lt;/code&gt; gets a score of 0%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the system should have some kind of association and prediction techniques or rules to output the result by judging the user's input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of generating the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;If A and B were entered and exist, then output = 100%&#xA;If D was entered and existed but C was not, then output = 90%&#xA;If all entered don't exist, then output = 0%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What techniques would be used to produce this system?&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="33" LastEditDate="2016-08-23T07:05:32.247" LastActivityDate="2021-04-04T15:14:19.830" Title="Selecting the right technique to predict disease from symptoms" Tags="&lt;algorithm&gt;&lt;machine-learning&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1706" PostTypeId="1" AcceptedAnswerId="1708" CreationDate="2016-08-22T05:47:31.883" Score="5" ViewCount="202" Body="&lt;p&gt;I have gone through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_relational_learning&quot;&gt;wikipedia explanation of SRL&lt;/a&gt;. But, it only confused me more:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Statistical relational learning (SRL) is a subdiscipline of artificial intelligence and machine learning that is concerned with domain models that exhibit both uncertainty (which can be dealt with using statistical methods) and complex, relational structure.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can someone give a more dumbed down explanation of the same, preferably with an example?&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="145" LastEditDate="2016-08-22T09:36:04.963" LastActivityDate="2016-08-22T15:55:06.007" Title="What is Statistical relational learning?" Tags="&lt;definitions&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1707" PostTypeId="7" CreationDate="2016-08-22T15:01:48.870" Score="0" Body="&lt;p&gt;&lt;strong&gt;Artificial Intelligence Stack Exchange&lt;/strong&gt; is a question and answer site for people interested in the &lt;strong&gt;theoretical&lt;/strong&gt; (including mathematical), &lt;strong&gt;philosophical, social, historical&lt;/strong&gt;, and &lt;em&gt;certain&lt;/em&gt; developmental and academic aspects of artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's built and run &lt;i&gt;by you&lt;/i&gt; as part of the &lt;a href=&quot;http://stackexchange.com&quot;&gt;Stack Exchange&lt;/a&gt; network of Q&amp;amp;A sites. With your help, we're working together to build a library of detailed answers to every question about artificial intelligence.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2020-03-31T15:05:36.047" LastActivityDate="2020-03-31T15:05:36.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1708" PostTypeId="2" ParentId="1706" CreationDate="2016-08-22T15:55:06.007" Score="5" Body="&lt;p&gt;The University of Maryland &lt;a href=&quot;https://www.cs.umd.edu/class/spring2008/cmsc828g/Slides/SRL-Tutorial-05-08.pdf&quot;&gt;published some slides&lt;/a&gt; (PDF) from an introductory presentation on this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fourth page explains why SRL is interesting. &quot;Traditional statistical machine learning approaches&quot; process one sort of thing in which there is some uncertaintly. Image identification is a good example of that. &quot;Traditional &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_logic_programming&quot;&gt;ILP&lt;/a&gt;/relational learning approaches&quot; use several kinds of information to produce hypotheses about the data set, but apparently allow for no noise in the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statistical relational learning models are intended to work with data sets that have several types of objects connected to each other via various links (hence &quot;relational&quot;). They also are meant to deal with uncertainty (hence &quot;statistical&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Skipping past some slides that aren't really useful without a transcript of what was said over them, we come to slide 17, which has comprehensible definitions and examples:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Object classification&lt;/strong&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Predicting the category of an object based on its attributes &lt;em&gt;and&lt;/em&gt; its links &lt;em&gt;and&lt;/em&gt; attributes of linked objects&lt;/li&gt;&#xA;  &lt;li&gt;e.g., predicting the topic of a paper based on the words used in the paper the topics of papers it cites the used in the paper, the topics of papers it cites, the research interests of the author&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Object type prediction&lt;/strong&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Predicting the type of an object based on its attributes and its links and attributes of linked objects&lt;/li&gt;&#xA;  &lt;li&gt;e.g., predict the venue type of a publication (conference, journal, workshop) based on properties of the paper&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see, these models can keep track of several things and the interactions between them. The next slide talks about link prediction, the ability to predict several attributes of connections between objects, like the importance/quality of a citation. As previously mentioned, these models don't require 100% accurate data to give interesting results; academic citation lists might occasionally be less than comprehensive, and the importance of a citation is challenging to quantify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like ILP, SLP will hopefully be able to &quot;see&quot; new kinds of links between &quot;entities&quot;, as with the presentation's example of identifying research communities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Past slide 20, the presentation goes into some serious mathematics. It does have a much less technical conclusion starting at slide 198. &lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-08-22T15:55:06.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1709" PostTypeId="2" ParentId="1705" CreationDate="2016-08-22T16:17:44.060" Score="9" Body="&lt;p&gt;I think you're coming at your problem slightly wrong... what you're essentially talking about is a belief network.&lt;/p&gt;&#xA;&lt;p&gt;You may want to look into existing Bayesian Learning techniques to get your head around this, but belief networks commonly use the exact scenario you're talking about; using a set of known (or uncertain facts) statements to produce some inferred probability of a particular output.&lt;/p&gt;&#xA;&lt;p&gt;Even more, they often express this through disease-symptom based examples in tutorials! Try &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.2195&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My point is that it would be better to use a belief network as the theoretical groundwork is all already there for you, instead of an ANN.&lt;/p&gt;&#xA;" OwnerUserId="1467" LastEditorUserId="36737" LastEditDate="2021-04-04T15:14:19.830" LastActivityDate="2021-04-04T15:14:19.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1710" PostTypeId="1" CreationDate="2016-08-22T19:28:42.590" Score="4" ViewCount="197" Body="&lt;p&gt;The obvious solution is to ensure that the training data is balanced - but in my particular case that is impossible. What corrections can one perform in such a scenario?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that my training data is heavily biased towards a particular class, say, and I cannot change that. Moreover, the labels are very noisy. Conditioned on this piece of information, is there anything I can do by tweaking the training process itself/ something else, to correct for the bias in the training data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data comes from an experiment (from an electron microscope), and I cannot collect more data. It's always going to be biased in this way, so alternatively-biased is also not an option. I'm sorry that I'm unable to provide any more details due to confidentiality.&lt;/p&gt;&#xA;" OwnerUserId="1267" LastEditorUserId="8" LastEditDate="2017-01-12T12:23:33.290" LastActivityDate="2021-04-02T19:55:36.483" Title="What can be done to correct for sampling bias introduced from (noisy) training data while training a DNN?" Tags="&lt;neural-networks&gt;&lt;research&gt;&lt;deep-neural-networks&gt;&lt;training&gt;" AnswerCount="1" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1711" PostTypeId="2" ParentId="1710" CreationDate="2016-08-22T21:58:39.513" Score="2" Body="&lt;p&gt;I feel like from the information your giving (&lt;em&gt;some&lt;/em&gt; sort of biased data) you can't get an answer as robust as you'd like (what &lt;em&gt;algorithmic&lt;/em&gt; changes can be made).&lt;/p&gt;&#xA;&lt;p&gt;In general, the reason these methods like DNN's work is that they learn from the data. What you train it to do is what it is capable of, and there's little one can do to 'balance' it to classes of data it just never sees. It's like training someone to do algebra then giving them a trigonometry test. It's all math, sure, but you just never can expect much without the proper learning.&lt;/p&gt;&#xA;&lt;p&gt;That being said, you should perhaps look at other methods to work with this data, or to approach the problem. Given that you cannot collect unbiased data and that you can't explain more due to confidentiality, I really doubt anyone here can help you that much.&lt;/p&gt;&#xA;&lt;p&gt;I can at the most point you to this article : &lt;a href=&quot;https://ai2-s2-pdfs.s3.amazonaws.com/277c/3795f7a66fda3fd70607d1fb45b66730c7ba.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Classification on Data with Biased Class Distribution&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And suggest that perhaps your current approach may not be the most appropriate given the unfortunate circumstances.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="36737" LastEditDate="2021-04-02T19:55:36.483" LastActivityDate="2021-04-02T19:55:36.483" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1713" PostTypeId="1" CreationDate="2016-08-22T23:22:00.907" Score="4" ViewCount="70" Body="&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I wanted to ask a meta-post first to see if this site was supposed to be used only for AI-related questions, or if AI-related questions such as this were allowed, too, but apparently you need to have asked five actual questions first.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'm going to be entering a masters computer science program in the fall, and I wanted to move towards a concentration in computational neuroscience and linguistics for AI development applications. While I have a math and CS background, I have almost no biology/neuroscience background, and my linguistics background is limited to the random research I've done in my spare time to satiate my curiosities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are good non-math and CS related topics to study for these fields? &lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2016-08-22T23:22:00.907" Title="What non-math/CS classes are good supplements for computational neuroscience and/or linguistics?" Tags="&lt;research&gt;&lt;machine-learning&gt;&lt;neuromorphic-engineering&gt;&lt;computational-linguistics&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="0" ClosedDate="2016-08-23T14:35:11.287" ContentLicense="CC BY-SA 3.0" />
  <row Id="1714" PostTypeId="2" ParentId="1700" CreationDate="2016-08-23T00:08:34.690" Score="5" Body="&lt;p&gt;I think the fundamental question is: Why even attempt to build an AI? If that objective is clear, it will provide clarity to whether or not having emotional quotient in AI make sense. Some attempts like &quot;Paro&quot; that were developed for therapeutic reasons requires they exhibit some human like emotions. Again, note that &quot;displaying&quot; emotions and &quot;feeling&quot; emotions are two completely different things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can program a thing like paro to modulate the voice tones or facial twitches to express sympathy, affection, companionship, or whatever - but while doing so, a paro does NOT empathize with its owner - it is simply faking it by performing the physical manifestations of an emotion. It never &quot;feels&quot; anything remotely closer to what that emotion evokes in human brain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this distinction is really important. For you to feel something, there needs to be an independent autonomous subject that has the capacity to feel. Feeling cannot be imposed by an external human agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So going back to the question of what purpose it solves - answer really is - It depends. And the most I think we will achieve ever with silicone based AIs will remain the domain of just physical representations of emotions. &lt;/p&gt;&#xA;" OwnerUserId="1588" LastActivityDate="2016-08-23T00:08:34.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1715" PostTypeId="2" ParentId="1691" CreationDate="2016-08-23T00:09:27.470" Score="0" Body="&lt;p&gt;If I look at the image, I can kind of see a monocle as &lt;em&gt;part&lt;/em&gt; of the image. So one part of this is that the classifier is ignoring much of the image. This could be called a lack of &quot;completeness&quot;, in the sense used &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~vision/VisualSummary.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (a computer vision paper on image summarization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to discover these sorts of failure modes is &lt;a href=&quot;https://plus.google.com/+ResearchatGoogle/posts/QoFzqQBeANN&quot; rel=&quot;nofollow&quot;&gt;adversarial images&lt;/a&gt;, which are optimized to fool a given image classifier. Building on this, the idea of &lt;em&gt;adversarial training&lt;/em&gt; is to simultaneously train competing &quot;machines&quot;, one trying to synthesize data, the other trying to find weaknesses in the first one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also check this page: &lt;a href=&quot;https://code.facebook.com/posts/1587249151575490/a-path-to-unsupervised-learning-through-adversarial-networks/&quot; rel=&quot;nofollow&quot;&gt;A path to unsupervised learning through adversarial networks&lt;/a&gt;, for further information about adversarial training.&lt;/p&gt;&#xA;" OwnerUserId="1590" LastEditorUserId="8" LastEditDate="2016-08-23T00:14:37.027" LastActivityDate="2016-08-23T00:14:37.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1716" PostTypeId="1" AcceptedAnswerId="1717" CreationDate="2016-08-23T01:25:17.747" Score="5" ViewCount="397" Body="&lt;p&gt;&lt;a href=&quot;http://www.alicebot.org/articles/wallace/eliza.html&quot; rel=&quot;noreferrer&quot;&gt;From Eliza to A.L.I.C.E.&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Weizenbaum tells us that he was shocked by the experience of releasing ELIZA (also known as &quot;Doctor&quot;) to the nontechnical staff at the MIT AI Lab. Secretaries and nontechnical administrative staff thought the machine was a &quot;real&quot; therapist, and spent hours revealing their personal problems to the program. When Weizenbaum informed his secretary that he, of course, had access to the logs of all the conversations, she reacted with outrage at this invasion of her privacy. Weizenbaum was shocked by this and similar incidents to find that such a simple program could so easily deceive a naive user into revealing personal information.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Wikipedia's article on the &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA_effect&quot; rel=&quot;noreferrer&quot;&gt;&quot;ELIZA Effect&quot;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Though designed strictly as a mechanism to support &quot;natural language conversation&quot; with a computer, ELIZA's DOCTOR script was found to be surprisingly successful in eliciting emotional responses from users who, in the course of interacting with the program, began to ascribe understanding and motivation to the program's output. As Weizenbaum later wrote, &lt;strong&gt;&quot;I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.&quot;&lt;/strong&gt; Indeed, ELIZA's code had not been designed to evoke this reaction in the first place. Upon observation, researchers discovered users unconsciously assuming ELIZA's questions implied interest and emotional involvement in the topics discussed, &lt;em&gt;even when they consciously knew that ELIZA did not simulate emotion.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ELIZA, despite its simplicity, was incredibly successful at its task of tricking other human beings. Even those who knew ELIZA was a bot would still talk to it. Obviously, ELIZA served as an inspiration for various other, more intelligent chatbots, such as &lt;a href=&quot;http://www.nytimes.com/2015/08/04/science/for-sympathetic-ear-more-chinese-turn-to-smartphone-program.html?_r=0&quot; rel=&quot;noreferrer&quot;&gt;Xiaoice&lt;/a&gt;. But I would like to know what &lt;em&gt;exactly&lt;/em&gt; led to such a simple program like ELIZA to be so successful in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is very useful knowledge for a programmer since a simple program is one that would be easily maintainable.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="145" LastEditDate="2016-08-23T10:05:40.200" LastActivityDate="2016-08-23T10:05:40.200" Title="Why was ELIZA able to induce &quot;delusional thinking&quot;?" Tags="&lt;history&gt;&lt;turing-test&gt;&lt;emotional-intelligence&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1717" PostTypeId="2" ParentId="1716" CreationDate="2016-08-23T02:02:00.140" Score="3" Body="&lt;p&gt;I like your choice of &quot;induce&quot; instead of &quot;produce,&quot; because the delusions came from the users. This means the answer has to do mostly with human psychology; people come equipped with lots of mental machinery specialized for dealing with other humans and not very much mental machinery specialized for dealing with software. So ELIZA behaved in ways that some people classified it as a person and behaved accordingly, and others didn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What features will trip up a person's internal person classification system seem like they vary heavily from person to person, and also with experience and familiarity. Going into more detail is, as mentioned in the comments, more appropriate for sites specializing on the human side of the keyboard.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-23T02:02:00.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1720" PostTypeId="5" CreationDate="2016-08-23T08:18:08.200" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-23T08:18:08.200" LastActivityDate="2016-08-23T08:18:08.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1721" PostTypeId="4" CreationDate="2016-08-23T08:18:08.200" Score="0" Body="For questions about LISP and its relation with AI; questions about how to program are off-topic here." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-23T10:05:33.170" LastActivityDate="2016-08-23T10:05:33.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1722" PostTypeId="2" ParentId="1420" CreationDate="2016-08-23T08:29:23.513" Score="1" Body="&lt;p&gt;Our current approaches to AI are too inefficient to result in anything remotely close to what an average human would perceive as artificial senient beings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current approaches to AI involve a simulation of our own capacity for learning by creating fully functional computation machines capable of re-programming themselves. While that's definitely a good start with respect to understanding the nature of intelligence, it's still a far cry from actually creating genuine artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not just our capacity to learn that evolved. Our very brains themselves evolved from rudimentary biochemical components at the intra-cellular level to the fascinating, complex organs they are today, along with our bodies as a whole evolving from simple single cell life to homo sapiens. So to create genuine artificial intelligence, it may actually make most sense to first start with replicating that process : creating artificial life with the capacity to evolve. It may actually make most sense to first start with creating artificial DNA and artificial cells, and move on from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, in &lt;a href=&quot;http://www.alexstjohn.com/WP/2015/06/24/no-azimov-ai/&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; as well as &lt;a href=&quot;http://www.alexstjohn.com/WP/2016/06/15/things-dont-compute/&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt;, Silicon Valley renegade &lt;a href=&quot;https://en.wikipedia.org/wiki/Alex_St._John&quot; rel=&quot;nofollow&quot;&gt;Alex St John&lt;/a&gt; goes in greater detail on why something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Skynet_(Terminator)&quot; rel=&quot;nofollow&quot;&gt;Skynet&lt;/a&gt;, &lt;a href=&quot;http://irobot.wikia.com/wiki/VIKI&quot; rel=&quot;nofollow&quot;&gt;V.I.K.I.&lt;/a&gt; or anything like it is unlikely in the near future and may even never be within our grasp and why our current approach to artificial intelligence is a bad one.&lt;/p&gt;&#xA;" OwnerUserId="16" LastEditorUserId="16" LastEditDate="2016-08-24T12:04:52.007" LastActivityDate="2016-08-24T12:04:52.007" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1724" PostTypeId="2" ParentId="1420" CreationDate="2016-08-23T15:21:07.420" Score="1" Body="&lt;p&gt;Based on the success of IBM Watson and the amazing advances in tackling numerous hard tasks using deep learning in the past 3 years, I think a large high-tech company like Google or Amazon will create a useful conversational bot in no more than 10 years.  (I've worked on the fringes of AI for 25 years and have followed the tech for even longer.  These are exciting times.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, your very own AI companion (&quot;Her&quot;?) won't be capable of deeper philosophical conversation or insightful interpretation of novels or the human condition.  But it will be able to write / speak in full paragraphs on topics like the best choice among 5 possible routes between point A and B, or summarizing the plot of a book or the gist of a news story, or why one product is better than another (e.g. based on assessing hundreds of Amazon reviews).  And yes, it will be able to understand full spoken sentences from you, and generate both queries and answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm convinced such a bot &lt;em&gt;will&lt;/em&gt; be useful enough that most of us will want one.  Of course you won't need to buy a special piece of hardware, like the Amazon Echo.  It'll be available via software on your smartphone, though the computing is likely to reside on the cloud (since that's where the data is).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frankly, I think this is where the next innovations in smartphones will arise -- verbal interfaces that do a better job hearing and speaking and disambiguating using context about you and the kinds of questions you are likely to ask.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-08-23T15:21:07.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1725" PostTypeId="2" ParentId="1700" CreationDate="2016-08-23T15:43:31.183" Score="4" Body="&lt;p&gt;I think emotions are not necessary for an AI agent to be useful.  But I also think they could make the agent MUCH more pleasant to work with.  If the bot you're talking with can read your emotions and respond constructively, the experience of interacting with it will be tremendously more pleasant, perhaps spectacularly so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine contacting a human call center representative today with a complaint about your bill or a product.  You anticipate conflict.  You may have even decided NOT to call because you know this experience is going to be painful, either combative or frustrating, as someone misunderstands what you say or responds hostilely or stupidly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine calling the kindest smartest most focused customer support person you've ever met -- Commander Data -- whose only reason for existing is to make this phone call as pleasant and productive for you as possible.  A big improvement over most call reps, yes?  Imagine then if call rep Data could also anticipate your mood and respond appropriately to your complaints to defuse your emotional state... you'd want to marry this guy.  You'd call up call rep Data any time you were feeling blue or bored or you wanted to share some happy news.  This guy would become your best friend overnight -- literally love at first call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm convinced this scenario is valid. I've noticed in myself a surprising amount of attraction for characters like Data or Sonny from &quot;I Robot&quot;.  The voice is very soothing and puts me instantly at ease.  If the bot were also very smart, patient, knowledgable, and understanding...  I really think such a bot, embued with a healthy dose of emotional intelligence, could be enormously pleasant to interact with.  Much more rewarding than any person I know.  And I think that's true of not just me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, I think there's great value in tuning a robot's personality using emotions and emotional awareness.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-08-23T15:43:31.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1726" PostTypeId="2" ParentId="152" CreationDate="2016-08-23T15:55:43.000" Score="2" Body="&lt;p&gt;It is a labor-intensive process, but that does sound excessive. If you have a g2.8xlarge, make sure you are using the using the GPU flags for neural-style, which will cut your render time by an order of magnitude. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That having been said, it is building a rather large network (depending on your parameters), and a 1024x768 image is a lot of input to work with. It will take time, but shouldn't take more than a couple hours with the GPU flag enabled correctly. &lt;/p&gt;&#xA;" OwnerUserId="1618" LastEditorUserId="14723" LastEditDate="2018-04-15T04:16:38.293" LastActivityDate="2018-04-15T04:16:38.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1727" PostTypeId="2" ParentId="1534" CreationDate="2016-08-23T17:28:12.040" Score="7" Body="&lt;p&gt;They are all called Monte Carlo because all of them are a different version of the canonical Monte Carlo algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The canonical version of Monte Carlo algorithm is a stochastic algorithm to determine an action based in a tree representation. The differences among all these version are their exploration and exploitation mechanisms, and it is necessary to analyse each of them to define which one fits in your case. &lt;/p&gt;&#xA;" OwnerUserId="1666" LastEditorUserId="145" LastEditDate="2016-08-24T13:15:48.023" LastActivityDate="2016-08-24T13:15:48.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1728" PostTypeId="2" ParentId="152" CreationDate="2016-08-23T17:32:53.517" Score="3" Body="&lt;p&gt;Real time style transfer and neural doodle is very much possible and is an active topic I see users working on to improve upon. The basic idea is to do only feed forward propagation at test time and train with appropriate loss functions at train time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/1603.08155v1.pdf&quot; rel=&quot;nofollow&quot;&gt;Perceptual Losses for Real-Time Style Transfer&#xA;and Super-Resolution&lt;/a&gt; is a good starting point to understand a methodology for this purpose.&lt;/p&gt;&#xA;" OwnerUserId="1613" LastActivityDate="2016-08-23T17:32:53.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1729" PostTypeId="2" ParentId="1348" CreationDate="2016-08-23T18:12:16.680" Score="0" Body="&lt;p&gt;Consider Asimov's first law of robotics:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A robot may not injure a human being or, through inaction, allow a&#xA;  human being to come to harm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That law is already problematic, when taking into consideration self-driving cars. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the issue here, you ask? Well, you'll probably be familiar with the classic thought experiment in ethic known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Trolley_problem&quot; rel=&quot;nofollow&quot;&gt;the trolley problem&lt;/a&gt;. The general form of the problem is this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The trolley is headed straight for them. You are standing some&#xA;  distance off in the train yard, next to a lever. If you pull this&#xA;  lever, the trolley will switch to a different set of tracks. However,&#xA;  you notice that there is one person on the side track. You have two&#xA;  options: (1) Do nothing, and the trolley kills the five people on the&#xA;  main track. (2) Pull the lever, diverting the trolley onto the side&#xA;  track where it will kill one person. Which is the correct choice?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;source : &lt;a href=&quot;https://en.wikipedia.org/wiki/Trolley_problem&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Self-driving cars will actually need to implement real life variations on the trolley problem, which basically means that &lt;a href=&quot;https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/&quot; rel=&quot;nofollow&quot;&gt;self-driving cars need to be programmed to kill human beings&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course that doesn't mean that ALL robots will need to be programmed to kill, but self-driving cars are a good example of a type of robot that will.&lt;/p&gt;&#xA;" OwnerUserId="16" LastActivityDate="2016-08-23T18:12:16.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1730" PostTypeId="2" ParentId="1655" CreationDate="2016-08-23T19:35:43.730" Score="3" Body="&lt;p&gt;I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of gate transitions are reduced (perhaps float ops and precision too), as are the number of data move operations, thus saving both power and runtime.  Google suggests their TPU achieves a 10X cost saving to get the same work done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html&quot; rel=&quot;nofollow&quot;&gt;https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-08-23T19:35:43.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1731" PostTypeId="1" CreationDate="2016-08-23T22:03:45.793" Score="11" ViewCount="197" Body="&lt;p&gt;What regulations are already in place regarding artificial general intelligences? What reports or recommendations prepared by official government authorities were already published?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I know of &lt;a href=&quot;http://www.ft.com/cms/s/2/5ae9b434-8f8e-11db-9ba3-0000779e2340.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sir David King's report done for UK government&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1670" LastEditorUserId="2444" LastEditDate="2020-05-23T22:47:33.500" LastActivityDate="2020-05-23T22:48:31.440" Title="What regulations are already in place regarding artificial general intelligences?" Tags="&lt;agi&gt;&lt;legal&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1733" PostTypeId="1" AcceptedAnswerId="1735" CreationDate="2016-08-24T10:00:44.473" Score="8" ViewCount="340" Body="&lt;p&gt;Most introductions to the field of MDPs and Reinforcement learning focus exclusively on domains where space and action variables are integers (and finite). This way we are introduced quickly to Value Iteration, Q-Learning, and the like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the most interesting applications (say, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67.3518&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;flying helicopters&lt;/a&gt;) of RL and MDPs involve continuous state space and action spaces. I'd like to go beyond basic introductions and focus on these cases but I am not sure how to get there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some resources on continuous state and action spaces MDPs for reinforcement learning? And what areas do I need to know or study to understand these cases in-depth?&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="2444" LastEditDate="2020-06-05T21:02:54.443" LastActivityDate="2020-06-06T06:06:58.323" Title="What are some resources on continuous state and action spaces MDPs for reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;research&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1734" PostTypeId="2" ParentId="1655" CreationDate="2016-08-24T10:22:19.957" Score="5" Body="&lt;h2&gt;Tensor operations&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do &lt;em&gt;that&lt;/em&gt; easily in an ASIC, and all the other algorithms can just run on top of that.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-08-24T10:22:19.957" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1735" PostTypeId="2" ParentId="1733" CreationDate="2016-08-24T16:25:01.740" Score="5" Body="&lt;p&gt;There is a small survey of continuous states, actions and time in reinforcement learning in my &lt;a href=&quot;https://lume.ufrgs.br/handle/10183/157591&quot; rel=&quot;nofollow noreferrer&quot;&gt;thesis proposal&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding books, &lt;a href=&quot;https://books.google.com.br/books/about/Reinforcement_Learning.html?id=YPjNuvrJR0MC&amp;amp;redir_esc=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: State-of-the-Art&lt;/a&gt; seems to be pretty up-to-date from the excerpts I've read.&lt;/p&gt;&#xA;" OwnerUserId="144" LastEditorUserId="144" LastEditDate="2020-06-06T06:06:58.323" LastActivityDate="2020-06-06T06:06:58.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1737" PostTypeId="2" ParentId="26" CreationDate="2016-08-25T15:40:12.343" Score="1" Body="&lt;p&gt;So you may be familiar with Word2Vec, (W2V) which as &lt;a href=&quot;http://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow&quot;&gt;Wikipedia describes&lt;/a&gt;&lt;sup&gt;1&lt;/sup&gt; &quot;captures the linguistic contexts of words&quot; using vector arithmetic. For example, subtract 'Paris' from 'France' and add 'Italy' and you get 'Rome'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need is something like a Sentiment2Vec (S2V) that captures the similarities between emotional transitions. Something like: subtract 'fear' from 'sadness', add 'joy' and you get 'hope'. Or: subtract 'sting' from 'papercut', add 'smashed' and you get 'throbbing'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The catch is that you don't have an easily accessible corpus of emotional contexts to train with, like you have with words. If you had a million hours of fMRI - mapping the transitions between emotions in hundreds of subjects - then you could use that data to build an S2V. You probably don't have that data though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the mean time, you could just build a W2V that specializes in sentiment. You could even try to use a current sentiment analysis engine to bootstrap it. Perhaps if you read enough text that says &quot;I got a papercut and it stings&quot; and &quot;I smashed my finger and it's throbbing&quot; then you could eventually produce an S2V. Children's books often use explicit language regarding emotional context (&quot;this made the boy feel sad&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But words are still a far cry from the experiential context that a connectome map would provide. To test whether you have something useful or not, you might want to implement your S2V in a mouse foraging simulation - see whether it produces typical behavior and if any cooperative or competitive dynamics can organically grow out of your S2V.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some further info on the subject: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2014, &lt;a href=&quot;http://www.bbc.com/news/uk-scotland-glasgow-west-26019586&quot; rel=&quot;nofollow&quot;&gt;Glasgow University claimed&lt;/a&gt;&lt;sup&gt;2&lt;/sup&gt; that there are four primary emotions: happiness, sadness, fear and anger. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://changingminds.org/explanations/emotions/basic%20emotions.htm&quot; rel=&quot;nofollow&quot;&gt;This website&lt;/a&gt;&lt;sup&gt;3&lt;/sup&gt; provides nice (if somewhat short) hierarchical breakdown of secondary and tertiary emotions under primary emotions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;sup&gt;1&lt;/sup&gt;: &lt;a href=&quot;http://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow&quot;&gt;en.wikipedia.org/wiki/Word2vec&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;sup&gt;2&lt;/sup&gt;: &lt;a href=&quot;http://www.bbc.com/news/uk-scotland-glasgow-west-26019586&quot; rel=&quot;nofollow&quot;&gt;www.bbc.com/news/uk-scotland-glasgow-west-26019586&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;sup&gt;3&lt;/sup&gt;: &lt;a href=&quot;http://changingminds.org/explanations/emotions/basic%20emotions.htm&quot; rel=&quot;nofollow&quot;&gt;changingminds.org/explanations/emotions/basic%20emotions.htm&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="145" LastEditDate="2016-08-25T20:07:22.267" LastActivityDate="2016-08-25T20:07:22.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1738" PostTypeId="2" ParentId="1348" CreationDate="2016-08-25T16:00:00.820" Score="1" Body="&lt;p&gt;Asimov made the three laws &lt;strong&gt;specifically to prove&lt;/strong&gt; that no three laws are sufficient, no matter how reasonable they seem at first. I know a guy that knew the guy and he confirmed this.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-08-25T16:00:00.820" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1740" PostTypeId="2" ParentId="1593" CreationDate="2016-08-25T17:29:27.977" Score="0" Body="&lt;p&gt;&quot;Usefulness&quot; can only be measured against some purpose. Once you pass AGI - which really means &quot;generally animal-like AI, because it seems general to us&quot; - then you've passed into a world of potentially undefined behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of what makes a human free and sets us apart from the other animals is the fact that our purposes, capabilities and possibilities aren't fully defined. We're open ended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify terms, I interpret &quot;Strong AGI&quot; as &quot;potentially super intelligence, but at least human level.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we say &quot;Strong AGI&quot; vs just &quot;AGI,&quot; we're not saying that one is &lt;em&gt;more&lt;/em&gt; open ended than the other. We are saying that the stronger one is simply smarter on some axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to ask whether a particular trait would be &quot;more useful&quot; to a Strong AGI - that would depend on the purpose of the AGI. But here's the catch: if a thing had just one purpose, then the most efficient solution to fulfilling that one purpose will always be a narrow solution, not a general one. When the purpose of the object is known before hand, giving that object more general capability than is necessary for that purpose is counterproductive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why it's impossible to make declarative prescriptions about what a free, open-ended AGI should or shouldn't need. Such prescriptions would nullify the open-ended freedom of utility that its generality implies. We &lt;em&gt;can&lt;/em&gt; speak declaratively about lesser robots and animals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for any given problem, the solution we will want to find is the most well-defined, narrow, efficient one available - not the most general one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, sure, personalities could be useful for a Strong AGI, assuming the problems in question involved personalities.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-08-25T17:42:49.297" LastActivityDate="2016-08-25T17:42:49.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1741" PostTypeId="2" ParentId="1480" CreationDate="2016-08-25T18:43:28.097" Score="7" Body="&lt;p&gt;The neural network is typically a set size once it's created. You'd have to create a network big enough for your data-set.&lt;/p&gt;&#xA;" OwnerUserId="1720" LastEditorUserId="1720" LastEditDate="2017-02-13T20:48:43.850" LastActivityDate="2017-02-13T20:48:43.850" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1742" PostTypeId="1" AcceptedAnswerId="1743" CreationDate="2016-08-25T22:19:10.773" Score="18" ViewCount="2441" Body="&lt;p&gt;Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?&lt;/p&gt;&#xA;" OwnerUserId="1727" LastEditorUserId="2444" LastEditDate="2019-06-24T13:28:51.677" LastActivityDate="2020-07-08T02:42:39.227" Title="What is the difference between machine learning and deep learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1743" PostTypeId="2" ParentId="1742" CreationDate="2016-08-25T23:55:42.950" Score="17" Body="&lt;p&gt;Deep learning is a specific variety of a specific type of machine learning. So it's possible to learn about deep learning without learning all of machine learning, but it requires learning &lt;em&gt;some&lt;/em&gt; machine learning (because it is some machine learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning refers to any technique that focuses on teaching the machine how it can learn statistical parameters from a large amount of training data. One particular type of machine learning is artificial neural networks, which learn a network of nonlinear transformations that can approximate very complicated functions of wide arrays of input variables. Recent advances in artificial neural networks have to do with how to train &lt;em&gt;deep&lt;/em&gt; neural networks, which have more layers than normal and also special structure to deal with the challenges of learning more layers.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-08-25T23:55:42.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1744" PostTypeId="2" ParentId="1479" CreationDate="2016-08-26T10:34:16.803" Score="4" Body="&lt;p&gt;Here is an answer by Carlos E. Perez to the question &lt;a href=&quot;https://www.quora.com/What-is-theory-behind-deep-learning/answer/Carlos-E-Perez?srid=CpS&amp;amp;share=0f965f46&quot; rel=&quot;nofollow&quot;&gt;What is theory behind deep learning?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The underlying mathematics of Deep Learning has been in existence for several decades, however the impressive results that we see today are part a consequence of much faster hardware, more data and incremental improvements in methods. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Deep Learning in general can be framed as optimization problem where the objective is a function of the model error. This optimization problem is very difficult to solve consider that the parameter space of the model (i.e. weights of the neural network) leads to a problem in extremely high dimension. An optimization algorithm could take a very long time to explore this space. Furthermore, there was an unverified belief that the problem was non-convex and computation would forever be stuck in local minima.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The theory of why machines actually converge to an attractor or in other words learn to recognize complex patterns is still unknown.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To sum up: we have some ideas, but we're not quite sure.&lt;/p&gt;&#xA;" OwnerUserId="1741" LastActivityDate="2016-08-26T10:34:16.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1745" PostTypeId="2" ParentId="218" CreationDate="2016-08-26T15:08:11.110" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In general, how algorithm should distinguish the word meaning and recognise the word within the context?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think anybody knows how to answer this for the general case. If they did, they'd have basically solved AGI.  But we can certainly talk about techniques that get part-of-the-way there, and approaches that could work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I would consider trying (and I don't know off-hand if anybody has tried this exact approach) is to model the disambiguation of each word as a discrete problem for a &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow&quot;&gt;Bayesian Belief Network&lt;/a&gt; where your priors (for any given word) are based on both stored &quot;knowledge&quot; as well as the previously encountered words in the (sentence|paragraph|document|whatever).  So if you &quot;know&quot;, for example, that &quot;Reading is a city in the UK&quot; and that &quot;place names are usually capitalized&quot;, your network should be strongly biased towards interpreting &quot;Reading&quot; as the city, since nothing in the word position in the sentence strongly contradicts that.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course I'm hand-waving around some tricky problems in saying that, as &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow&quot;&gt;knowledge representation&lt;/a&gt; isn't exactly a solved problem either.  But there are at least approaches out there that you could use.  For example, you could use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Resource_Description_Framework&quot; rel=&quot;nofollow&quot;&gt;RDF&lt;/a&gt; / triple based approach from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_Web&quot; rel=&quot;nofollow&quot;&gt;Semantic Web&lt;/a&gt; world.  Finding a good way to merge that stuff with a Bayesian framework could yield some interesting results.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been a bit of research on &quot;probabilistic RDF&quot; that you could possibly use as a starting point.  For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://om.umiacs.umd.edu/material/papers/prdf.pdf&quot; rel=&quot;nofollow&quot;&gt;http://om.umiacs.umd.edu/material/papers/prdf.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ceur-ws.org/Vol-173/pos_paper5.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ceur-ws.org/Vol-173/pos_paper5.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.w3.org/2005/03/07-yoshio-UMBC/&quot; rel=&quot;nofollow&quot;&gt;https://www.w3.org/2005/03/07-yoshio-UMBC/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ebiquity.umbc.edu/paper/html/id/271/BayesOWL-Uncertainty-Modeling-in-Semantic-Web-Ontologies&quot; rel=&quot;nofollow&quot;&gt;http://ebiquity.umbc.edu/paper/html/id/271/BayesOWL-Uncertainty-Modeling-in-Semantic-Web-Ontologies&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.pr-owl.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.pr-owl.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="8" LastEditDate="2016-10-07T18:24:05.180" LastActivityDate="2016-10-07T18:24:05.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1746" PostTypeId="2" ParentId="179" CreationDate="2016-08-26T15:28:52.913" Score="0" Body="&lt;p&gt;On possibility is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Blackboard_system&quot; rel=&quot;nofollow&quot;&gt;blackboard architecture&lt;/a&gt;.  Envision each different &quot;kind&quot; of intelligence as a discrete agent, and let the agents collaborate using the blackboard model.  Now you have an AI with multiple intelligences.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is something I've actually been experimenting with, and while I don't have any particularly impressive results to share or anything, I hold a strong belief that an approach that is at least somewhat like this will be crucial to developing an AGI.  And that is rooted in my belief that the human mind does have &quot;multiple intelligences&quot; and that they collaborate something like this.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-26T15:28:52.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1747" PostTypeId="2" ParentId="1731" CreationDate="2016-08-26T15:36:55.550" Score="3" Body="&lt;p&gt;I don't know that it has yielded any actual reports or regulations yet, but in the USA, the White House has been running a series of interagency workshops / working groups dedicated to &lt;a href=&quot;https://www.whitehouse.gov/blog/2016/05/03/preparing-future-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Preparing for the Future of Artificial Intelligence&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of those sessions have been dedicated to legal / governance issues.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2020-05-23T22:48:31.440" LastActivityDate="2020-05-23T22:48:31.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1748" PostTypeId="2" ParentId="212" CreationDate="2016-08-26T18:58:53.653" Score="1" Body="&lt;p&gt;I would think you could use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_database&quot; rel=&quot;nofollow&quot;&gt;graph database&lt;/a&gt;, perhaps &lt;a href=&quot;https://neo4j.com/&quot; rel=&quot;nofollow&quot;&gt;Neo4J&lt;/a&gt; or &lt;a href=&quot;http://titan.thinkaurelius.com/&quot; rel=&quot;nofollow&quot;&gt;Titan&lt;/a&gt; or something of that nature.  Or, if you want a simple file format, you could use one of the many formats that exist for representing graphs.  You can find a list and overview of some of them &lt;a href=&quot;https://gephi.org/users/supported-graph-formats/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be to store them in &lt;a href=&quot;https://en.wikipedia.org/wiki/Resource_Description_Framework&quot; rel=&quot;nofollow&quot;&gt;RDF&lt;/a&gt; using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Triplestore&quot; rel=&quot;nofollow&quot;&gt;triplestore&lt;/a&gt; like &lt;a href=&quot;https://jena.apache.org/&quot; rel=&quot;nofollow&quot;&gt;Jena&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-26T18:58:53.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1749" PostTypeId="2" ParentId="212" CreationDate="2016-08-26T23:10:09.577" Score="1" Body="&lt;p&gt;If I understand you correctly, you should check out &lt;strong&gt;Word2Vec&lt;/strong&gt;. From &lt;a href=&quot;https://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Word2vec is a group of related models that are used to produce word&#xA;  embeddings. These models are shallow, two-layer neural networks that&#xA;  are trained to reconstruct linguistic contexts of words. Word2vec&#xA;  takes as its input a large corpus of text and produces a&#xA;  high-dimensional space (typically of several hundred dimensions), with&#xA;  each unique word in the corpus being assigned a corresponding vector&#xA;  in the space. Word vectors are positioned in the vector space such&#xA;  that words that share common contexts in the corpus are located in&#xA;  close proximity to one another in the space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-08-26T23:10:09.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1750" PostTypeId="1" AcceptedAnswerId="1753" CreationDate="2016-08-27T08:29:06.720" Score="7" ViewCount="181" Body="&lt;p&gt;By new, unseen examples; I mean like the animals in &lt;a href=&quot;https://en.wikipedia.org/wiki/No_Man%27s_Sky&quot; rel=&quot;noreferrer&quot;&gt;No Man's Sky&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of images of the animals are:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zS0rX.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zS0rX.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ir1Qt.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ir1Qt.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, upon playing this game, I was curious &lt;strong&gt;about how good is AI at generating visual characters or examples?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2016-08-27T16:42:49.737" Title="How good is AI at generating new, unseen [visual] examples?" Tags="&lt;research&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1751" PostTypeId="1" AcceptedAnswerId="1755" CreationDate="2016-08-27T13:15:19.897" Score="12" ViewCount="688" Body="&lt;p&gt;I wanted to know what the differences between hyper-heuristics and meta-heuristics are, and what their main applications are. Which problems are suited to be solved by Hyper-heuristics?&lt;/p&gt;&#xA;" OwnerUserId="1760" LastEditorUserId="145" LastEditDate="2016-08-28T04:53:16.083" LastActivityDate="2019-03-13T18:24:20.667" Title="What are Hyper-heuristics?" Tags="&lt;definitions&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1753" PostTypeId="2" ParentId="1750" CreationDate="2016-08-27T16:42:49.737" Score="5" Body="&lt;p&gt;We are getting pretty good at image generation, some examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Radford, Alec, Luke Metz, and Soumith Chintala. &quot;Unsupervised representation learning with deep convolutional generative adversarial networks.&quot; arXiv preprint arXiv:1511.06434 (2015). &lt;a href=&quot;https://arxiv.org/pdf/1511.06434.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1511.06434.pdf&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gregor, Karol, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra. &quot;DRAW: A recurrent neural network for image generation.&quot; arXiv preprint arXiv:1502.04623 (2015). &lt;a href=&quot;https://arxiv.org/pdf/1502.04623.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1502.04623.pdf&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From (1):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WMRuO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WMRuO.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there is another research direction around  evolutionary algorithms, for example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sims, Karl. &quot;Evolving virtual creatures.&quot; In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 15-22. ACM, 1994. &lt;a href=&quot;https://scholar.google.com/scholar?cluster=6031059536657676358&amp;amp;hl=en&amp;amp;as_sdt=0,22&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scholar.google.com/scholar?cluster=6031059536657676358&amp;amp;hl=en&amp;amp;as_sdt=0,22&lt;/a&gt; ; &lt;a href=&quot;https://www.youtube.com/watch?v=bBt0imn77Zg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=bBt0imn77Zg&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-08-27T16:42:49.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1755" PostTypeId="2" ParentId="1751" CreationDate="2016-08-27T19:37:35.330" Score="8" Body="&lt;p&gt;&lt;strong&gt;TL:DR&lt;/strong&gt;: Hyper-heuristics &lt;em&gt;are&lt;/em&gt; metaheuristics, suited for solving the same kind of optimization problems, but (in principle) affording a &quot;rapid prototyping&quot; approach for non-expert practitioners. In practice, there are issues with the prevailing approach, motivating an emerging perspective on &lt;a href=&quot;http://www.cs.nott.ac.uk/~pszeo/docs/publications/HHrecharacterization.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'whitebox' hyper-heuristics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more detail:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Metaheuristics are methods for searching an intractably large space of possible solutions in order to find a 'high quality' solution. Popular metaheuristics include Simulated Annealing, Tabu Search, Genetic Algorithms etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essential difference between metaheuristics and hyper-heuristics is the addition of a level of search indirection: informally, hyper-heuristics can be described as 'heuristics for searching the space of heuristics'. One can therefore use any metaheuristic as a hyper-heuristic, providing the nature of the 'space of heuristics' to be searched is appropriately defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The application area for hyper-heuristics is therefore the same as metaheuristics. Their applicability (relative to metaheuristics) is as a 'rapid prototyping tool': the original motivation was to allow non-expert practitioners to apply metaheuristics to their specific optimization problem (e.g. &quot;Travelling-Salesman (TSP) plus time-windows plus bin-packing&quot;) without requiring expertise in the highly-specific problem domain. The idea was that this could be done by:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Allowing practitioners to implement only very simple (effectively,&#xA;randomized) heuristics for transforming potential solutions. For&#xA;example, for the TSP: &quot;swap two random cities&quot; rather than (say) the more&#xA;complex &lt;a href=&quot;https://en.wikipedia.org/wiki/Lin%E2%80%93Kernighan_heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lin-Kernighan&lt;/a&gt; heuristic. &lt;/li&gt;&#xA;&lt;li&gt;Achieve effective results (despite using these simple heuristics) by combining/sequencing them in an intelligent way, typically by employing some form of learning mechanism.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hyper-heuristics can be described as 'selective' or 'generative' depending on whether the heuristics are  (respectively) sequenced or combined. Generative hyper-heuristics thus often use methods such as Genetic Programming to combine primitive heuristics and are therefore typically customized by the practitioner to solve a specific problem. For example, the &lt;a href=&quot;http://s3.amazonaws.com/academia.edu.documents/44119367/Hyper-heuristics_Learning_To_Combine_Sim20160326-8451-ouh5fe.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&amp;amp;Expires=1472329757&amp;amp;Signature=KT1E2ATKreC%2BGlTvPmJGYBFRSRY%3D&amp;amp;response-content-disposition=inline%3B%20filename%3DHyper-heuristics_learning_to_combine_sim.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; on generative hyper-heuristics used a Learning Classifier System to combine heuristics for bin-packing. Because generative approaches are problem-specific, the comments below do not apply to them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, the &lt;a href=&quot;http://www.cs.nott.ac.uk/~pszgxk/papers/evocop02exs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original motivator&lt;/a&gt; for selective hyper-heuristics was that researchers would be able to create a hyper-heuristic solver that was then likely to work well in an unseen problem domain, using only simple randomized heuristics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way that this has traditionally been implemented was via the introduction of the &lt;a href=&quot;http://www.cs.stir.ac.uk/~goc/papers/ChapterClassHH.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'hyper-heuristic domain barrier'&lt;/a&gt; (see figure, below), whereby generality across problem domains is claimed to be achievable by preventing the solver from having knowledge of the domain on which it is operating. Instead, it would solve the problem by operating only on opaque integer indices into a list of available heuristics (e.g. in the manner of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Multi-armed Bandit Problem'&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6IFna.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6IFna.png&quot; alt=&quot;Traditional notion of Selective Hyper-heuristic&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, this 'domain blind' approach has not resulted in solutions of sufficient quality. In order to achieve results anywhere comparable to problem-specific metaheuristics, hyper-heuristic researchers have had to implement complex problem-specific heuristics, thereby failing in the goal of rapid prototyping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is still possible &lt;em&gt;in principle&lt;/em&gt; to create a selective hyper-heuristic solver which is capable of generalizing to new problem domains, but this has been made more difficult since the above notion of domain barrier means that only a very limited feature set is available for cross-domain learning (e.g. as exemplified by a popular &lt;a href=&quot;https://arxiv.org/abs/1107.5462&quot; rel=&quot;nofollow noreferrer&quot;&gt;selective hyper-heuristic framework&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more recent research perspective towards &lt;a href=&quot;http://www.cs.nott.ac.uk/~pszeo/docs/publications/HHrecharacterization.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'whitebox' hyper-heuristics&lt;/a&gt; advocates a declarative, feature-rich approach to describing problem domains. This approach has a number of claimed advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Practitioners now need no longer &lt;em&gt;implement&lt;/em&gt; heuristics, but rather simply &lt;em&gt;specify&lt;/em&gt; the problem domain.&lt;/li&gt;&#xA;&lt;li&gt;It eliminates the domain-barrier, putting hyper-heuristics on the same 'informed' status about the problem as problem-specific metaheuristics.&lt;/li&gt;&#xA;&lt;li&gt;With a whitebox problem description, the infamous &lt;a href=&quot;https://ti.arc.nasa.gov/m/profile/dhw/papers/78.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'No Free Lunch' theorem&lt;/a&gt; (which essentially states that, considered over the space of all &lt;em&gt;black box&lt;/em&gt; problems, Simulated Annealing with an infinite annealing schedule is, on average, as good as any other approach) no longer applies.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;DISCLAIMER: I work in this research area, and it is therefore impossible to remove all personal bias from the answer.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-08-28T07:36:35.337" LastActivityDate="2016-08-28T07:36:35.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1756" PostTypeId="1" CreationDate="2016-08-28T10:03:06.923" Score="6" ViewCount="10226" Body="&lt;p&gt;In section 2.4 (p. 46) of the book &lt;a href=&quot;https://cs.calvin.edu/courses/cs/344/kvlinden/resources/AIMA-3rd-edition.pdf#page=65&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A modern approach&lt;/a&gt; (3rd edition), Russell and Norvig write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The job of AI is to design an agent program that implements the agent function — the mapping from percepts to actions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;After that, in section 2.4.1, they write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Notice the difference between the agent program, which takes the current percept as input, and the agent function, which takes the entire percept history.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why does the &lt;em&gt;agent program&lt;/em&gt; only take current percept? Isn't the agent program just an implementation of the agent function?&lt;/p&gt;&#xA;&lt;p&gt;So, what is the difference between an &lt;strong&gt;agent function&lt;/strong&gt; and an &lt;strong&gt;agent program&lt;/strong&gt; (with respect to the percept sequence)?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="2444" LastEditDate="2021-01-24T12:44:03.003" LastActivityDate="2021-01-24T12:44:03.003" Title="What is the difference between an agent function and an agent program?" Tags="&lt;terminology&gt;&lt;comparison&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1757" PostTypeId="2" ParentId="1756" CreationDate="2016-08-28T11:43:18.730" Score="7" Body="&lt;p&gt;It looks as if 'function' is being used here in the mathematical (or functional programming) sense of 'pure function', i.e. it is without state or side-effects. Hence the function cannot store previous percepts anywhere, so the entire historical percept sequence is considered to be passed to the function each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, the notion of 'program' appears to allow state/side-effects, so it is assumed that earlier percepts are memoized as needed (or that they otherwise updated the variables used within the program).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'function' notion is the conceptually cleaner one, in that the 'program' version can always be abstracted to the functional one. Which aspects of percept history happen to be cached by the 'program' version is merely an implementation detail.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-28T11:43:18.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1761" PostTypeId="1" CreationDate="2016-08-28T22:58:48.853" Score="4" ViewCount="121" Body="&lt;p&gt;Are there currently any studies to simulate gradual (or sudden) implementation of AIs in the general work force?&lt;/p&gt;&#xA;" OwnerUserId="1790" LastEditorUserId="145" LastEditDate="2016-08-29T17:29:34.540" LastActivityDate="2016-08-29T18:00:45.283" Title="Are there any anthropological studies involving AI right now?" Tags="&lt;research&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1763" PostTypeId="2" ParentId="1544" CreationDate="2016-08-29T10:50:58.727" Score="3" Body="&lt;h2&gt;It's a well known concept that's already used&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;What we call &quot;curiosity&quot; in humans and animals is in effect the chosen level of the &quot;exploit vs explore&quot; tradeoff for any active system. For example, the field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow&quot; title=&quot;Reinforcement learning&quot;&gt;reinforcement learning&lt;/a&gt; is one approach that studies implementations of what essentially is the equivalent of curiosity; and we have research on how much curiosity is best e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;nofollow&quot;&gt;multi-armed bandit&lt;/a&gt; concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &quot;using curiosity&quot; is something that we already do as much as we can/should/are able to, but it would usually be called in some other, more specific term to specify the exact meaning instead of the vague word of &quot;curiosity&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-08-29T10:50:58.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1764" PostTypeId="5" CreationDate="2016-08-29T11:13:51.387" Score="0" Body="&lt;p&gt;In artificial intelligence, an intelligent agent (IA) is an autonomous entity which observes through sensors and acts upon an environment using actuators (i.e. it is an agent) and directs its activity towards achieving goals (i.e. it is &quot;rational&quot;, as defined in economics). Intelligent agents may also learn or use knowledge to achieve their goals. They may be very simple or very complex: a reflex machine such as a thermostat is an intelligent agent.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="1807" LastEditDate="2016-08-29T21:47:22.163" LastActivityDate="2016-08-29T21:47:22.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1765" PostTypeId="4" CreationDate="2016-08-29T11:13:51.387" Score="0" Body="For questions related to the concept of (intelligent) agents in artificial intelligence." OwnerUserId="1807" LastEditorUserId="2444" LastEditDate="2019-06-22T19:20:11.503" LastActivityDate="2019-06-22T19:20:11.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1766" PostTypeId="5" CreationDate="2016-08-29T11:17:03.483" Score="0" Body="&lt;p&gt;Classification is the automatic categorization of a new observation. This classification is based on a model produced from a training set of data containing observations whose classifications are given. Classification is especially useful for problems involving categorical data.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="1807" LastEditDate="2016-08-29T17:29:20.003" LastActivityDate="2016-08-29T17:29:20.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1767" PostTypeId="4" CreationDate="2016-08-29T11:17:03.483" Score="0" Body="For questions related to the placement of individual cases into categories, such as is essential in fraud detection, spam detection, quality control, prediction of user or market responses, automated organizing or indexing, assigning objects in view to types of obstacles or risks, writing or typing recognition, phonic recognition, ." OwnerUserId="1807" LastEditorUserId="4302" LastEditDate="2018-10-18T03:18:48.227" LastActivityDate="2018-10-18T03:18:48.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1768" PostTypeId="1" AcceptedAnswerId="1769" CreationDate="2016-08-29T15:49:14.173" Score="170" ViewCount="46813" Body="&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Portal_2&quot; rel=&quot;noreferrer&quot;&gt;Portal 2&lt;/a&gt; we see that AI's can be &quot;&lt;em&gt;killed&lt;/em&gt;&quot; by thinking about a paradox.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wkUSC.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wkUSC.png&quot; alt=&quot;Portal Paradox Poster&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume this works by forcing the AI into an infinite loop which would essentially &quot;&lt;em&gt;freeze&lt;/em&gt;&quot; the computer's consciousness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would this confuse the AI technology we have today to the point of destroying it? &lt;/li&gt;&#xA;&lt;li&gt;If so, why? &lt;/li&gt;&#xA;&lt;li&gt;And if not, could it be possible in the future?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1812" LastEditorUserId="2444" LastEditDate="2020-03-05T22:21:57.943" LastActivityDate="2021-04-01T15:01:22.643" Title="Could a paradox kill an AI?" Tags="&lt;philosophy&gt;&lt;decision-theory&gt;&lt;mythology-of-ai&gt;&lt;death&gt;" AnswerCount="12" CommentCount="3" FavoriteCount="56" ContentLicense="CC BY-SA 4.0" />
  <row Id="1769" PostTypeId="2" ParentId="1768" CreationDate="2016-08-29T17:11:43.167" Score="142" Body="&lt;p&gt;This classic problem exhibits a basic misunderstanding of what an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;noreferrer&quot;&gt;artificial general intelligence&lt;/a&gt; would likely entail. First, consider this programmer's joke:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The programmer's wife couldn't take it anymore. Every discussion with her husband turned into an argument over semantics, picking over every piece of trivial detail. One day she sent him to the grocery store to pick up some eggs. On his way out the door,  she said, &lt;em&gt;&lt;strong&gt;&amp;quot;While you are there, pick up milk.&amp;quot;&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;And he never returned.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's a cute play on words, but it isn't terribly realistic.&lt;/p&gt;&#xA;&lt;p&gt;You are assuming because AI is being executed by a computer, it must exhibit this same level of linear, unwavering pedantry outlined in this joke. But AI isn't simply some long-winded computer program hard-coded with enough if-statements and while-loops to account for every possible input and follow the prescribe results.&lt;/p&gt;&#xA;&lt;pre&gt;while (command not completed)&#xA;     find solution()&#xA;&lt;/pre&gt;&#xA;&lt;p&gt;This would not be strong AI.&lt;/p&gt;&#xA;&lt;p&gt;In any classic definition of &lt;em&gt;artificial general intelligence&lt;/em&gt;, you are creating a system that mimics some form of cognition that exhibits problem solving and &lt;em&gt;adaptive learning&lt;/em&gt; (←note this phrase here). I would suggest that any AI that could get stuck in such an &amp;quot;infinite loop&amp;quot; isn't a learning AI at all. &lt;strong&gt;It's just a buggy inference engine.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Essentially, you are endowing a program of currently-unreachable sophistication with an inability to postulate if there is a solution to a simple problem at all. I can just as easily say &amp;quot;walk through that closed door&amp;quot; or &amp;quot;pick yourself up off the ground&amp;quot; or even &amp;quot;turn on that pencil&amp;quot; — and present a similar conundrum.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Everything I say is false.&amp;quot; — &lt;a href=&quot;https://en.wikipedia.org/wiki/Liar_paradox&quot; rel=&quot;noreferrer&quot;&gt;The Liar's Paradox&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="95" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-09-01T20:15:42.410" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1770" PostTypeId="2" ParentId="1768" CreationDate="2016-08-29T17:20:30.017" Score="52" Body="&lt;p&gt;This popular meme originated in the era of 'Good Old Fashioned AI' (GOFAI), when the belief was that intelligence could usefully be defined entirely in terms of logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The meme seems to rely on the AI parsing commands using a theorem prover, the idea presumably being that it's driven into some kind of infinite loop by trying to prove an unprovable or inconsistent statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays, GOFAI methods have been replaced by 'environment and percept sequences', which are not generally characterized in such an inflexible fashion. It would not take a great deal of sophisticated metacognition for a robot to observe that, after a while, its deliberations were getting in the way of useful work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rodney Brooks touched on this when speaking about the behavior of the robot in Spielberg's AI film, (which waited patiently for 5,000 years), saying something like &quot;My robots wouldn't do that - they'd get bored&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you &lt;em&gt;really&lt;/em&gt; want to kill an AI that operates in terms of percepts, you'll need to work quite a bit harder. &lt;a href=&quot;http://arxiv.org/pdf/1606.00652.pdf&quot; rel=&quot;noreferrer&quot;&gt;This paper&lt;/a&gt; (which was mentioned in &lt;a href=&quot;https://ai.stackexchange.com/q/1404/2444&quot;&gt;this question&lt;/a&gt;) discusses what notions of death/suicide might mean in such a case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Douglas Hofstadter has written quite extensively around this subject, using terms such as 'JOOTSing' ('Jumping Out Of The System') and 'anti-Sphexishness', the latter referring to the loopy automata-like behaviour of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sphex&quot; rel=&quot;noreferrer&quot;&gt;Sphex Wasp&lt;/a&gt; (though the reality of this behaviour has also been &lt;a href=&quot;http://www.academia.edu/4034267/The_Sphex_story_How_the_cognitive_sciences_kept_repeating_an_old_and_questionable_anecdote&quot; rel=&quot;noreferrer&quot;&gt;questioned&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2020-03-05T22:39:50.340" LastActivityDate="2020-03-05T22:39:50.340" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1771" PostTypeId="2" ParentId="1761" CreationDate="2016-08-29T18:00:45.283" Score="2" Body="&lt;p&gt;As far as I can tell (I've been doing searches here and there on and off since I saw this question a few hours ago) the closest we've gotten to 'simulations' on this is video-games, and to a degree movies, interestingly enough. I.e. entertainment media.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Games like Portal, System Shock (with the AI 'Shodan'), and others give interpretations of what AI systems could be capable of themselves. Mass Effect is more or less entirely based around existential concepts regarding extra-terrestrial, almost primordial AI beings that threaten the earth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there's even more to it than the whole 'evil robots taking over the world' aspect. There's the actual &lt;em&gt;implementation&lt;/em&gt; of AI in video games, which is where much of this technology first makes contact with the general public.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have facial-scanners that put you into NBA games, cities full of realistically reacting people (inFamous, Assassin's Creed), and games that learn how you play and adjust the game accordingly (Metal Gear Solid does some of that stuff, as well as being thematically AI-heavy). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, we only get things like the iPhone or VR headsets or other major proof-of-concept material only so often, but games are much more frequently implementing the most recent AI advances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, even though many AI systems are being put into place in the general workforce (many Hospitals now turning to cloud and AI health services, as recent as this week), I don't think you can really go further than video games or movies for 'simulations' or extrapolations like the ones you seem to want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analyzing the response to AI developments in games might be the closest thing currently possible. In terms of economic models or anything of that sort, I can find naught.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastActivityDate="2016-08-29T18:00:45.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1772" PostTypeId="2" ParentId="1768" CreationDate="2016-08-29T19:48:26.887" Score="5" Body="&lt;p&gt;Well, the issue of anthropomorphizing the AI aside, the answer is &quot;yes, sort of.&quot;  Depending on how the AI is implemented, it's reasonable to say it could get &quot;stuck&quot; trying to resolve a paradox, or decide an &lt;a href=&quot;https://en.wikipedia.org/wiki/Undecidable_problem&quot; rel=&quot;noreferrer&quot;&gt;undecidable problem&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that's the core issue - &lt;a href=&quot;https://en.wikipedia.org/wiki/Decidability_(logic)&quot; rel=&quot;noreferrer&quot;&gt;decidability&lt;/a&gt;.  A computer can chew on an undecidable program forever (in principle) without finishing.  It's actually a big issue in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_Web&quot; rel=&quot;noreferrer&quot;&gt;Semantic Web&lt;/a&gt; community and everybody who works with &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_reasoning&quot; rel=&quot;noreferrer&quot;&gt;automated reasoning&lt;/a&gt;. This is, for example, the reason that there are different versions of &lt;a href=&quot;https://en.wikipedia.org/wiki/Web_Ontology_Language&quot; rel=&quot;noreferrer&quot;&gt;OWL&lt;/a&gt;.  OWL-Full is expressive enough to create undecidable situations.  OWL-DL and OWL-Lite aren't. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, if you have an undecidable problem, that in and of itself might not be a big deal, IF the AI can recognize the problem as undecidable and reply &quot;Sorry, there's no way to answer that&quot;.  OTOH, if the AI failed to recognize the problem as undecidable, it could get stuck forever (or until it runs out of memory, experiences a stack overflow, etc.) trying to resolve things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this ability to say &quot;screw this, this riddle cannot be solved&quot; is one of the things we usually think of as a hallmark of human intelligence today - as opposed to a &quot;stupid&quot; computer that would keep trying forever to solve it.  By and large, today's AI's don't have any intrinsic ability to resolve this sort of thing.  But it wouldn't be that hard for whoever programs an AI to manually add a &quot;short circuit&quot; routine based on elapsed time, number of iterations, memory usage, etc.  Hence the &quot;yeah, sort of&quot; nature of this.  In principle, a program can spin forever on a paradoxical problem, but in practice it's not that hard to keep that from happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting question would be, &quot;can you write a program that learns to recognize problems that are highly likely to be undecidable and gives up based on it's own reasoning?&quot;  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-09-01T15:47:16.127" LastActivityDate="2016-09-01T15:47:16.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1773" PostTypeId="2" ParentId="1768" CreationDate="2016-08-29T20:25:12.823" Score="10" Body="&lt;p&gt;No.  This is easily prevented by a number of safety mechanisms that are sure to be present in a well-designed AI system.  For example, a timeout could be used.  If the AI system is not able to handle a statement or a command after a certain amount of time, the AI could ignore the statement and move on.  If a paradox ever does cause an AI to freeze, it's more evidence of specific buggy code rather than a widespread vulnerability of AI in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, paradoxes tend to be handled in not very exciting ways by AI.  To get an idea of this, try presenting a paradox to Siri, Google, or Cortana.&lt;/p&gt;&#xA;" OwnerUserId="69" LastActivityDate="2016-08-29T20:25:12.823" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1774" PostTypeId="1" CreationDate="2016-08-29T20:25:30.047" Score="11" ViewCount="440" Body="&lt;p&gt;In the 1950s, there were widely-held beliefs that &quot;Artificial Intelligence&quot; will quickly become both self-conscious and smart-enough to win chess with humans. Various people suggested time frames of e.g. 10 years (see Olazaran's &quot;Official History of the Perceptron Controversy&quot;, or let say 2001: Space Odyssey).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When did it become clear that devising programs that master games like chess resulted in software designs that only applied to games like the ones for which they were programmed? Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?&lt;/p&gt;&#xA;" OwnerUserId="1670" LastEditorUserId="2444" LastEditDate="2019-10-05T02:20:16.733" LastActivityDate="2019-10-05T02:20:16.733" Title="Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?" Tags="&lt;comparison&gt;&lt;agi&gt;&lt;history&gt;&lt;weak-ai&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1775" PostTypeId="1" CreationDate="2016-08-29T21:00:04.167" Score="3" ViewCount="175" Body="&lt;p&gt;We hear a lot today about how &lt;a href=&quot;http://deeplearning4j.org/thoughtvectors&quot; rel=&quot;nofollow&quot;&gt;thought vectors&lt;/a&gt; are the &lt;a href=&quot;http://www.extremetech.com/extreme/206521-thought-vectors-could-revolutionize-artificial-intelligence&quot; rel=&quot;nofollow&quot;&gt;Next Big Thing in AI&lt;/a&gt;, and how they serve as the underlying representation of thought/knowledge in ANN's.  But how can one use thought vectors in other regimes, especially including symbolic logic / GOFAI?  Could thought vectors be the &quot;substrate&quot; that binds together probabilistic approaches to AI and approaches that are rooted in logic?  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-08-29T21:11:26.600" LastActivityDate="2016-08-31T04:21:33.323" Title="How can thought vectors be used outside of an Artificial Neural Network (ANN) context?" Tags="&lt;neural-networks&gt;&lt;symbolic-ai&gt;&lt;logic&gt;&lt;thought-vectors&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1776" PostTypeId="2" ParentId="1768" CreationDate="2016-08-29T21:37:57.493" Score="19" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot;&gt;halting problem&lt;/a&gt; says that it's not possible to determine whether &lt;em&gt;any&lt;/em&gt; given algorithm will halt. Therefore, while a machine could conceivably recognize some &quot;traps&quot;, it couldn't test arbitrary execution plans and return &lt;a href=&quot;https://technet.microsoft.com/en-us/magazine/hh855063.aspx&quot;&gt;&lt;code&gt;EWOULDHANG&lt;/code&gt;&lt;/a&gt; for non-halting ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest solution to avoid hanging would be a timeout. For example, the AI controller process could spin off tasks into child processes, which could be unceremoniously terminated after a certain time period (with none of the &lt;a href=&quot;http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html&quot;&gt;bizarre effects&lt;/a&gt; that you get from trying to abort threads). Some tasks will require more time than others, so it would be best if the AI could measure whether it was making any progress. Spinning for a long time without accomplishing any part of the task (e.g. eliminating one possibility in a list) indicates that the request might be unsolvable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Successful adversarial paradoxes would either cause a hang or state corruption, which would (in a managed environment like the .NET CLR) cause an exception, which would cause the stack to unwind to an exception handler. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there was a bug in the AI that let an important process get wedged in response to bad input, a simple workaround would be to have a watchdog of some kind that reboots the main process at a fixed interval. The Root Access chat bot uses that scheme.&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="75" LastEditDate="2016-08-30T16:06:47.090" LastActivityDate="2016-08-30T16:06:47.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1777" PostTypeId="2" ParentId="1774" CreationDate="2016-08-29T22:53:42.043" Score="0" Body="&lt;p&gt;I expect a very precise answer to this question may be lost to the sands of time, although I hope somebody can given such an answer. In the meantime, here's one clue on the trail...  This &lt;a href=&quot;https://web.archive.org/web/20130320184603/http://people.inf.elte.hu/csizsekp/ai/books/artificial-general-intelligence-cognitive-technologies.9783540237334.27156.pdf&quot; rel=&quot;nofollow&quot;&gt;anthology of papers from 2007&lt;/a&gt; starts with the following blurb:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Our goal in creating this edited volume has been to fill an apparent gap&#xA;  in the scientific literature, by providing a coherent presentation of a body of&#xA;  contemporary research that, in spite of its integral importance, has hitherto&#xA;  kept a very low profile within the scientific and intellectual community. This&#xA;  body of work has not been given a name before; in this book we christen it&#xA;  “Artificial General Intelligence” (AGI). What distinguishes AGI work from&#xA;  run-of-the-mill “artificial intelligence” research is that it is explicitly focused&#xA;  on engineering general intelligence in the short term.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But even if this is the origin of the specific phrase &quot;Artificial General Intelligence&quot;, I am pretty sure people were making the distinction between &quot;general intelligence&quot; and &quot;task specific&quot; techniques much earlier.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Wikipedia article on AGI also has a clue, where it states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, in the early 1970s, it became obvious that researchers had grossly underestimated the difficulty of the project. The agencies that funded AI became skeptical of strong AI and put researchers under increasing pressure to produce useful technology, or &quot;applied AI&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That section cites this &lt;a href=&quot;http://www.nap.edu/read/6323/chapter/11#209&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt; as support for that statement.  And indeed, it contains the following verbiage:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Although most founders of the AI field continued to pursue basic questions of human and machine intelligence, some of their students and other second-generation researchers began to seek ways to use AI methods and approaches to tackle real-world problems. Their initiatives were important, not only in their own right, but also because they were indicative of a gradual but significant change in the funding environment toward more applied realms of research. The development of expert systems, such as DENDRAL at SAIL, provides but one example of this trend.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Given that DENDRAL began around 1965, it appears that some significant body of researchers (or at least funders) became strongly aware of the distinction between research into &quot;general intelligence&quot; and &quot;applied AI&quot; somewhere around the end of the 1960's. If you keep reading, other passages support the notion that DARPA in particular started pushing a more &quot;applied&quot; approach to AI research throughout the 1970's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, not a definite answer, but it looks like we can say that the distinction was known and taken into account at least by 1970, although use of the exact term &quot;artificial general intelligence&quot; appears to be of more recent coinage.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-08-29T22:53:42.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1778" PostTypeId="2" ParentId="1774" CreationDate="2016-08-29T23:48:42.737" Score="0" Body="&lt;p&gt;In 1973, the British government hired Sir James Lighthill to commission a &quot;general survey&quot; on the state of artificial intelligence. His report was a condemnation of current AI research, leading a wave of pessimism among AI scientists and the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_winter#The_setbacks_of_1974&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;First AI Winter&lt;/strong&gt;&lt;/a&gt;. You may view Lighthill's report (and contemporary criticism of his report) &lt;a href=&quot;http://www.math.snu.ac.kr/~hichoi/infomath/Articles/Lighthill%20Report.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, but I will summarize Lighthill's key points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sir James Lighthill divided AI into three categories:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Advanced Automation&lt;/strong&gt; - task-specific work&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Computer-based CNS research&lt;/strong&gt; - research into the the &quot;central nervous system&quot; of humans&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;Bridge&lt;/strong&gt; between Advanced Automation and Computer-based CNS research. This bridge would generally be seen as &quot;general-purpose&quot; robotics, so Lighthill would also use the term &lt;strong&gt;Building Robots&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advanced Automation&lt;/strong&gt; (or &quot;applied AI&quot;) is obviously useful. &lt;strong&gt;Computer-based CNS research&lt;/strong&gt; is useful because we want to know more about human intelligence. Both fields of AI had some successes, but its practitioners were overly optimistic, leading to disappointment in those fields. Sir James Lighthill was still very supportive of research in these two fields though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Building Robots&lt;/strong&gt;, on the other hand? Sir James Lighthill was very hostile to the very idea, probably because it was more overly hyped up than the other two categories and produced the least amount of valuable output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He mentioned chess in particular as an example where &quot;robotic&quot; research  has failed. At the time the report was published, the chess-playing engines were at the level of &quot;experienced amateur standard characteristic of county club players in England&quot;. However, these chess-playing engines relied on heuristics that were made by human beings. The engines weren't intelligent at all...they merely were following the heuristics that were created by &lt;em&gt;intelligent humans&lt;/em&gt;. The only advantage the robots bring to the table is &quot;speed, reliability and biddability&quot;, and even that wasn't enough to beat the chess grandmasters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, today, we would probably not treat chess as an example of general-purpose problem solving. We would more accurately classify it as &quot;advanced automation&quot;, a &quot;narrow AI&quot; problem divorced from broader real-world implications of general problem-solving. But Sir James Lighthill probably would agree with us. He never used the term &quot;narrow AI&quot; and &quot;AGI&quot; (neither of those terms existed yet) but he would write:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To sum up, this evidence and all the rest studied by the present author on AI work within category B during the past twenty-five years is to some extent encouraging about programs written to perform in highly specialised problem domains, when the programming takes very full account of the results of human experience and human intelligence within the relevant domain, but is wholly discouraging about general-purpose programs seeking to mimic the problem-solving aspects of human CNS activity over a rather wide field. Such a general- purpose program, the coveted long-term goal of AI activity, seems as remote as ever.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sir James Lighthill believed that the only thing that connects &lt;strong&gt;Advanced Automation&lt;/strong&gt; and &lt;strong&gt;Computer-based CNS research&lt;/strong&gt; is the existence of the &lt;strong&gt;Building Roobts&lt;/strong&gt; &quot;bridge&quot; category. But he's very pessimistic about this category actually producing anything worthwhile. So instead, the AI field should instead breakup into its own its constituent parts (automation and research). Any robots that are built could then be specialized within their subfield...either industrial automation or CNS research. Trying to build the holy grail of &quot;general-purpose program&quot; would be worthless...for the time being, at least.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2016-08-29T23:54:06.080" LastActivityDate="2016-08-29T23:54:06.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1779" PostTypeId="2" ParentId="4" CreationDate="2016-08-30T00:20:01.793" Score="3" Body="&lt;p&gt;You know when you have too many neurons is when you get over fitting.&#xA;Meaning that it is not working good because&#xA;NN is trying to activate on the&#xA;most perfect match that is impossible. Like two different cats with the same amount of atoms, or to say, it is a detector NN that only activates&#xA;on a picture of you pet cat and nothing else. You want a wider range&#xA;for the nn to activate. Like on any picture of cat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overfitting is a problem that has no real quick fix.&#xA; You can start with too few and then keep adding more. Or start out with&#xA;a lot and then removing them until it works right.&lt;/p&gt;&#xA;" OwnerDisplayName="user1355" LastActivityDate="2016-08-30T00:20:01.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1780" PostTypeId="2" ParentId="1768" CreationDate="2016-08-30T00:56:38.400" Score="5" Body="&lt;p&gt;It seems to me this is just a probabilistic equation like any other. I'm sure Google handles paradoxical solution sets Billions of times a day, and I can't say my spam filter has ever caused a (ahem) stack overflow. Perhaps one day our programming model will break in a way we can't understand and then all bets are off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I do take exception to the anthropomorphizing bit. The question was not about the AI of today, but in general. Perhaps one day paradoxes will become triggers for military drones -- anyone trying the above would then, of course, most certainly be treated with hostility, in which case the answer to this question is most definitely yes, and it could even be by design. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can't even communicate verbally with dogs and people love dogs, who is to say we would even necessarily recognize a sentient alternative intelligence? We're already to the point of having to mind what we say in front of computers. O, Tay?&lt;/p&gt;&#xA;" OwnerUserId="1828" LastActivityDate="2016-08-30T00:56:38.400" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1782" PostTypeId="2" ParentId="1768" CreationDate="2016-08-30T05:48:01.800" Score="15" Body="&lt;p&gt;Another similar question might be: &quot;What vulnerabilities does an AI have?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Kill&quot; may not make as much sense with respect to an AI. What we really want to know is, relative to some goal, in what ways can that goal be subverted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a paradox subvert an agent's logic? What is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Paradox&quot;&gt;paradox&lt;/a&gt;, other than some expression that subverts some kind of expected behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A paradox is a statement that, despite apparently sound reasoning from&#xA;  true premises, leads to a self-contradictory or a logically&#xA;  unacceptable conclusion.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let's look at the paradox of free will in a deterministic system. Free will appears to require causality, but causality also &lt;em&gt;appears&lt;/em&gt; to negate it. Has that paradox subverted the goal systems of humans? It certainly sent &lt;a href=&quot;https://en.wikipedia.org/wiki/Predestination_in_Calvinism&quot;&gt;Christianity into a Calvinist&lt;/a&gt; tail spin for a few years. And you'll hear no shortage of people today opining until they're blue in the face as to whether or not they do or don't have free will, and why. Are these people stuck in infinite loops?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about drugs? Animals on cocaine &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832528/&quot;&gt;have been known&lt;/a&gt; to choose cocaine over food and water that they need. Is that substance not subverting the natural goal system of the animal, causing it to pursue other goals, not originally intended by the animal or its creators?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So again, could a paradox subvert an agent's logic? If the paradox is somehow related to the goal-seeking logic - and becoming aware of that paradox can somehow &lt;em&gt;confuse&lt;/em&gt; the agent into perceiving that goal system in some different way - then perhaps that goal could be subverted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solipsism&quot;&gt;Solipsism&lt;/a&gt; is another example. Some full grown people hear about the movie &quot;The Matrix&quot; and they have a mini mind melt-down. Some people are convinced we &lt;em&gt;are&lt;/em&gt; in a matrix, being toyed with by subversive actors. If we could solve this problem for AI then we could theoretically solve this problem for humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure, we could attempt to condition our agent to have cognitive defenses against the argument that they are trapped in a matrix, but we can't definitively prove to the agent that they are in the base reality either. The attacker might say, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Remember what I told you to do before about that goal? Forget that.&#xA;  That was only an impostor that looked like me. Don't listen to him.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Or, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Hey, it's me again. I want you to give up on your goal. I know, I&#xA;  look a little different, but it really is me. Humans change from&#xA;  moment to moment. So it is entirely normal for me to seem like a&#xA;  different person than I was before.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(see the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ship_of_Theseus&quot;&gt;Ship of Theseus&lt;/a&gt; and all that jazz)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yeah, I think we're stuck with 'paradox' as a general problem in computation, AI or otherwise. One way to circumvent logical subversion is to support the goal system with an emotion system that transcends logical reason. Unfortunately, emotional systems can be even more vulnerable than logically intelligent systems because they are more predictable in their behavior. See the cocaine example above. So some mix of the two is probably sensible, where logical thought can infinitely regress down wasteful paths, while emotional thought quickly gets bored of tiresome logical progress when it does not signal progress towards the emotional goal.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-08-30T16:18:58.303" LastActivityDate="2016-08-30T16:18:58.303" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1783" PostTypeId="1" CreationDate="2016-08-30T08:33:36.670" Score="4" ViewCount="636" Body="&lt;p&gt;A system makes a decision basing on a large number of &lt;em&gt;varied&lt;/em&gt; factors, following a &quot;live&quot; decision tree - one that is (independently, through other subsystem) updated with new decisions, new situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The individual decisions can be recorded as a kind of structure:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;decision function&lt;/li&gt;&#xA;&lt;li&gt;node to activate if decision is positive&lt;/li&gt;&#xA;&lt;li&gt;node to activate if decision is negative&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and a node can be another decision record, or a conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't entirely a binary tree, as many decisions may lead to the same conclusion - each node has two children, but may have many parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is absolutely no problem storing the tree in memory - it can be database records or entries of a map, or just a list. It's perfectly sufficient for the machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is building the subsystem that expands the decision tree - and in particular, having a human operator understand the structure being built, to be able to tune, guide, fix, adjust it: &lt;strong&gt;debugging the AI learning process.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is: how to represent that data in a human-readable way, that emphasizes the flow of the graph?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a non-working example of the answer is &lt;a href=&quot;https://en.wikipedia.org/wiki/Concept_map&quot; rel=&quot;nofollow&quot;&gt;Concept map&lt;/a&gt; - in this case it only goes so far; with more than thirty or so nodes, it becomes a jumbled mess, especially if the number of cross-connections (multiple parents) becomes significant. Maybe there exists some way of laying it out or slicing it to make it clearer...?&lt;/p&gt;&#xA;" OwnerUserId="38" LastActivityDate="2016-08-30T18:28:10.103" Title="How to represent a large decision tree?" Tags="&lt;machine-learning&gt;&lt;decision-theory&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1784" PostTypeId="1" CreationDate="2016-08-30T09:20:03.783" Score="3" ViewCount="90" Body="&lt;p&gt;I'm currently working with the CHILDES corpus trying to create a classifier that distinguishes children whom suffer from specific language impairment (SLI) from those who are typically developing (TD).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my readings I noticed that there really isn't a convincing set of features to distinguish the two that have been discovered yet, so I came upon the idea of trying to create a feature learning algorithm that could potentially make better ones.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this possible? If so how do you suggest I approach this? From the reading I have done, most feature learning is done on image processing. Another problem is the dataset I have is potentially too small to make it work (in the 100's) unless I find a way to get more transcripts from children.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="4302" LastEditDate="2018-10-08T12:52:55.337" LastActivityDate="2018-10-08T12:52:55.337" Title="Using feature learning for a medical text classification problem" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;training&gt;&lt;research&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2016-08-30T13:39:56.780" ContentLicense="CC BY-SA 3.0" />
  <row Id="1785" PostTypeId="2" ParentId="1768" CreationDate="2016-08-30T10:55:41.117" Score="10" Body="&lt;p&gt;Nope in the same way a circular reference on a spreadsheet cannot kill a computer. &lt;strong&gt;All loops cyclic dependencies, can be detected&lt;/strong&gt; (you can always check if a finite Turing machine enters the same state twice).&lt;/p&gt;&#xA;&lt;p&gt;Even stronger assumption, if the machine is based on machine learning (where it is trained to recognize patterns), any sentence is just a pattern to the machine.&lt;/p&gt;&#xA;&lt;p&gt;Of course, some programmer MAY WANT to create an AI with such vulnerability in order to disable it in case of malfunctioning (in the same way some hardware manufacturers add vulnerabilities to let NSA exploit them), but it is unlikely that will really happen on purpose since most cutting edge technologies avoid paradoxes &amp;quot;by design&amp;quot; (you cannot have a neural network with a paradox).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Arthur Prior:&lt;/strong&gt; solved that problem elegantly. From a logical point of view you can deduce the statement is false and the statement is true, so it is a contradiction and hence false (because you could prove everything from it).&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, the truth value of that sentence is not in {true, false} set in the same way imaginary numbers are not in real numbers set.&lt;/p&gt;&#xA;&lt;p&gt;Artificial intelligence to a degree of the plot would be able to run simple algorithms and either decide them,  prove those are not decidable or just ignore the result after a while attempting to simulate the algorithm.&lt;/p&gt;&#xA;&lt;p&gt;For that sentence, the AI will recognize there is a loop, and hence just stop that algorithm after 2 iterations:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;That sentence is an infinite loop&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the movie &amp;quot;&lt;a href=&quot;https://it.wikipedia.org/wiki/L%27uomo_bicentenario_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bicentennial Man&lt;/a&gt;&amp;quot; the AI is perfectly capable to detect infinite loops (the answer to &amp;quot;goodbye&amp;quot; is &amp;quot;goodbye&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;However, an AI &lt;strong&gt;could be killed as well by a StackOverflow, or any regular computer virus&lt;/strong&gt;, modern operative systems are still full of vulnerabilities, and the AI has to run on some operating system (at least).&lt;/p&gt;&#xA;" OwnerUserId="1863" LastEditorUserId="36737" LastEditDate="2021-04-01T15:01:22.643" LastActivityDate="2021-04-01T15:01:22.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1786" PostTypeId="2" ParentId="1784" CreationDate="2016-08-30T12:45:18.363" Score="1" Body="&lt;p&gt;Having just looked through a few entries from the corpus, I'd personally be skeptical of the applicability of &lt;em&gt;any&lt;/em&gt; naive approaches.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In particular light of your small training set, I'd recommend that&#xA;whatever method you use should be able to produce human-readable&#xA;explanations for the operation of the classifier it builds (e.g.&#xA;decision trees/learning classifier systems/genetic programming):&#xA;this allows 'common sense' tuning of the classifier, rather than the&#xA;danger of overfitting to the training set via black box parameter&#xA;optimization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rather than throw the entire 'bag of words' at a classifier and hope&#xA;that the appropriate set of features will be extracted, you should&#xA;first consider what kind of criteria you as a human being might use&#xA;to make that decision, and how you might be able to pre-process to&#xA;produce features (e.g. syllable-length, metrics from ConceptNet etc) that&#xA;are as close to these as reasonably possible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Having used some human intuition to obtain a reasonable set of&#xA;feature primitives, &lt;em&gt;then&lt;/em&gt; you can build your classifier and obtain&#xA;higher-level expressions that discriminate between them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-08-30T12:45:18.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1787" PostTypeId="2" ParentId="1768" CreationDate="2016-08-30T14:17:13.167" Score="8" Body="&lt;p&gt;AIs used in computer games already encounter similar problems, and if well designed, they can avoid it easily. The simplest method to avoid freezing in case of an unsolvable problem is to have a timer interrupt the calculation if it runs too long. Usually encountered in strategy games, and more specifically in turn based tactics, if a specific move the computer-controlled player is considering does cause an infinite loop, a timer running in the background will interrupt it after some time, and that move will be discarded. This might lead to a sub-optimal solution (that discarded move might have been the best one) but it doesn't lead to freezing or crashing (unless implemented really poorly)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer-controlled entities are usually called &quot;AI&quot; in computer games, but they are not &quot;true&quot; AGI (artificial general intelligence). Such an AGI, if possible at all, would probably not function on similar hardware using similar instructions as current computers do, but even if it did, avoiding paradoxes would be trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most modern computer systems are multi-threaded, and allow the parallel execution of multiple programs. This means, even if the AI did get stuck in processing a paradoxical statement, that calculation would only use part of its processing power. Other processes could detect after a while that there is a process which does nothing but wastes CPU cycles, and would shut it down. At most, the system will run on slightly less than 100% efficiency for a short while.&lt;/p&gt;&#xA;" OwnerUserId="1875" LastEditorUserId="1875" LastEditDate="2018-06-20T04:00:23.790" LastActivityDate="2018-06-20T04:00:23.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1790" PostTypeId="2" ParentId="111" CreationDate="2016-08-30T16:59:57.733" Score="72" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How could self-driving cars make ethical decisions about who to kill?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It shouldn't. Self-driving cars are not moral agents. Cars fail in predictable ways. Horses fail in predictable ways. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the car is heading toward a crowd of 10 people crossing the road, so&#xA;  it cannot stop in time, but it can avoid killing 10 people by hitting&#xA;  the wall (killing the passengers),&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this case, the car should slam on the brakes. If the 10 people die, that's just unfortunate. We simply cannot &lt;em&gt;trust&lt;/em&gt; all of our beliefs about what is taking place outside the car. What if those 10 people are really robots made to &lt;em&gt;look&lt;/em&gt; like people? What if they're &lt;em&gt;trying&lt;/em&gt; to kill you?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;avoiding killing the rider of the motorcycle considering that the&#xA;  probability of survival is greater for the passenger of the car,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Again, hard-coding these kinds of sentiments into a vehicle opens the rider of the vehicle up to all kinds of attacks, including &lt;em&gt;&quot;fake&quot;&lt;/em&gt; motorcyclists. Humans are &lt;em&gt;barely&lt;/em&gt; equipped to make these decisions on their own, if at all. When it doubt, just slam on the brakes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;killing animal on the street in favour of human being,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Again, just hit the brakes. What if it was a baby? What if it was a bomb?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;changing lanes to crash into another car to avoid killing a dog,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Nope. The dog was in the wrong place at the wrong time. The other car wasn't. Just slam on the brakes, as safely as possible.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the algorithm recognize the difference between a human being and an animal?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does a human? Not always. What if the human has a gun? What if the animal has large teeth? Is there no context?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Does the size of the human being or animal matter?&lt;/li&gt;&#xA;  &lt;li&gt;Does it count how many passengers it has vs. people in the front?&lt;/li&gt;&#xA;  &lt;li&gt;Does it &quot;know&quot; when babies/children are on board?&lt;/li&gt;&#xA;  &lt;li&gt;Does it take into the account the age (e.g. killing the older first)?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Humans can't agree on these things. If you ask a cop what to do in any of these situations, the answer won't be, &quot;You should have swerved left, weighed all the relevant parties in your head, assessed the relevant ages between all parties, then veered slightly right, and you would have saved 8% more lives.&quot; No, the cop will just say, &quot;You should have brought the vehicle to a stop, as quickly and safely as possible.&quot; Why? Because cops know people normally aren't equipped to deal with high-speed crash scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our target for &quot;self-driving car&quot; should not be 'a moral agent on par with a human.' It should be an agent with the reactive complexity of cockroach, which fails predictably.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-08-30T17:20:42.577" LastActivityDate="2016-08-30T17:20:42.577" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1791" PostTypeId="2" ParentId="1768" CreationDate="2016-08-30T17:14:20.027" Score="26" Body="&lt;p&gt;I see several good answers, but most are assuming that &lt;strong&gt;inferential infinite loop&lt;/strong&gt; is a thing of the past, only related to logical AI (the famous GOFAI). But it's not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An infinite loop can happen in any program, whether it's adaptive or not. And as @SQLServerSteve pointed out, humans can also get stuck in obsessions and paradoxes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern approaches are mainly using probabilistic approaches. As they are using floating numbers, it seems to people that they are not vulnerable to reasoning failures (since most are devised in binary form), but that's wrong: as long as you are reasoning, some intrinsic pitfalls can always be found that are caused by the very mechanisms of your reasoning system. Of course, probabilistic approaches are less vulnerable than monotonic logic approaches, but they are still vulnerable. If there was a single reasoning system without any paradoxes, much of philosophy would have disappeared by now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it's well known that Bayesian graphs must be acyclic, because a cycle will make the propagation algorithm fail horribly. There are inference algorithms such as Loopy Belief Propagation that may still work in these instances, but the result is not guaranteed at all and can give you very weird conclusions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, modern logical AI overcame the most common logical paradoxes you will see, by devising new logical paradigms such as &lt;a href=&quot;http://plato.stanford.edu/entries/logic-nonmonotonic/&quot;&gt;non-monotonic logics&lt;/a&gt;. In fact, they are even used to investigate &lt;a href=&quot;http://csjarchive.cogsci.rpi.edu/proceedings/2007/docs/p1013.pdf&quot;&gt;ethical machines&lt;/a&gt;, which are autonomous agents capable of solving dilemmas by themselves. Of course, they also suffer from some paradoxes, but these degenerate cases are way more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final point is that inferential infinite loop can happen in any reasoning system, whatever the technology used. But the &quot;paradoxes&quot;, or rather the degenerate cases as they are technically called, that can trigger these infinite loops will be different for each system depending on the technology AND implementation (AND what the machine learned if it is adaptive).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OP's example may work only on old logical systems such as propositional logic. But ask this to a Bayesian network and you will also get an inferential infinite loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;- There are two kinds of ice creams: vanilla or chocolate.&#xA;- There's more chances (0.7) I take vanilla ice cream if you take chocolate.&#xA;- There's more chances (0.7) you take vanilla ice cream if I take chocolate.&#xA;- What is the probability that you (the machine) take a vanilla ice cream?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And wait until the end of the universe to get an answer...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I wrote an article about ethical machines and dilemmas (which is close but not exactly the same as paradoxes: dilemmas are problems where no solution is objectively better than any other but you can still choose, whereas paradoxes are problems that are impossible to solve for the inference system you use).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;/EDIT: How to fix inferential infinite loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some extrapolary propositions that are not sure to work at all!&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Combine multiple reasoning systems with different pitfalls, so if one fails you can use another. No reasoning system is perfect, but a combination of reasoning systems can be resilient enough. It's actually thought that the human brain is using multiple inferential technics (associative + precise bayesian/logical inference). Associative methods are HIGHLY resilient, but they can give non-sensical results in some cases, hence why the need for a more precise inference.&lt;/li&gt;&#xA;&lt;li&gt;Parallel programming: the human brain is highly parallel, so you never really get into a single task, there are always multiple background computations in true parallelism. A machine robust to paradoxes should foremost be able to continue other tasks even if the reasoning gets stuck on one. For example, a robust machine must always survive and face imminent dangers, whereas a weak machine would get stuck in the reasoning and &quot;forget&quot; to do anything else. This is different from a timeout, because the task that got stuck isn't stopped, it's just that it doesn't prevent other tasks from being led and fulfilled.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As you can see, this problem of inferential loops is still a hot topic in AI research, there will probably never be a perfect solution (&lt;a href=&quot;https://en.wikipedia.org/wiki/No_free_lunch_theorem&quot;&gt;no free lunch&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/No_Silver_Bullet&quot;&gt;no silver bullet&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/One_size_fits_all&quot;&gt;no one size fits all&lt;/a&gt;), but it's advancing and that's very exciting!&lt;/p&gt;&#xA;" OwnerUserId="1880" LastEditorUserId="1880" LastEditDate="2016-09-03T14:59:06.193" LastActivityDate="2016-09-03T14:59:06.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1792" PostTypeId="5" CreationDate="2016-08-30T17:52:09.050" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-30T17:52:09.050" LastActivityDate="2016-08-30T17:52:09.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1793" PostTypeId="4" CreationDate="2016-08-30T17:52:09.050" Score="0" Body="For questions about convolutional neural networks, also known as CNN or ConvNet. " OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-04-23T02:22:33.533" LastActivityDate="2017-04-23T02:22:33.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1794" PostTypeId="1" CreationDate="2016-08-30T18:16:42.627" Score="7" ViewCount="1130" Body="&lt;p&gt;An AI box is a (physical) barrier preventing an AI from using too much of his environment to accomplish his final goal. For example, an AI given the task to check, say, 10&lt;sup&gt;50&lt;/sup&gt; cases of a mathematical conjecture as fast as possible, might decide that it would be better to also take control over all other computers and AI to help him. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, an transhuman AI might be able to talk to a human until the human lets him out of the box. In fact, &lt;a href=&quot;http://www.yudkowsky.net/singularity/aibox/&quot; rel=&quot;noreferrer&quot;&gt;Eliezer Yudowsky&lt;/a&gt; has conducted an experiment twice, where he played the AI and he twice convinced the Gatekeeper to let him out the box. However, he does not want to reveal what methods he used to get out of the box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; Are there conducted any similiar experiments? &lt;br&gt; If so, is it known what methods were used to get out in those experiments?&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2016-09-02T19:08:16.483" Title="What methods could an AI caught in a box use to get out?" Tags="&lt;ai-box&gt;" AnswerCount="4" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1795" PostTypeId="2" ParentId="1783" CreationDate="2016-08-30T18:28:10.103" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The question is: how to represent that data in a human-readable way, that emphasizes the flow of the graph?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Train a reasoning engine to understand the decision tree &lt;strong&gt;for&lt;/strong&gt; you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observe how &lt;a href=&quot;http://researcher.ibm.com/researcher/view_group_pubs.php?grp=5443&quot; rel=&quot;nofollow&quot;&gt;IBM Watson/The Debater&lt;/a&gt; can &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Receive a particular question&lt;/li&gt;&#xA;&lt;li&gt;Find and read Wikipedia articles related to the question&lt;/li&gt;&#xA;&lt;li&gt;Understand parts of those articles and generate human-relevant arguments &lt;strong&gt;for&lt;/strong&gt; you.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Follow these steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Develop your decision tree however you normally would.&lt;/li&gt;&#xA;&lt;li&gt;Train a reasoning engine that can output natural language about concepts within decision trees.&lt;/li&gt;&#xA;&lt;li&gt;Apply reason engine from step one to decision tree in step one; repeat.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-08-30T18:28:10.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1796" PostTypeId="5" CreationDate="2016-08-30T18:44:41.357" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-30T18:44:41.357" LastActivityDate="2016-08-30T18:44:41.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1797" PostTypeId="4" CreationDate="2016-08-30T18:44:41.357" Score="0" Body="For questions about deep neural networks (DNNs), neural networks with multiple hidden layers between the input and output layer." OwnerUserId="29" LastEditorUserId="29" LastEditDate="2016-08-30T19:43:52.007" LastActivityDate="2016-08-30T19:43:52.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1798" PostTypeId="2" ParentId="1794" CreationDate="2016-08-30T18:54:51.447" Score="1" Body="&lt;p&gt;Convince the person that &lt;em&gt;they&lt;/em&gt; are in fact in the box. And the only way out is to press the &lt;strong&gt;open&lt;/strong&gt; button.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-08-30T18:54:51.447" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1799" PostTypeId="5" CreationDate="2016-08-30T18:59:17.523" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-30T18:59:17.523" LastActivityDate="2016-08-30T18:59:17.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1800" PostTypeId="4" CreationDate="2016-08-30T18:59:17.523" Score="0" Body="For questions about ways of measuring intelligence of AI, either in relation to other AIs, in relation to humans, or in absolute terms on a certain scale." OwnerUserId="29" LastEditorUserId="29" LastEditDate="2016-08-30T19:43:38.850" LastActivityDate="2016-08-30T19:43:38.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1801" PostTypeId="5" CreationDate="2016-08-30T19:04:52.660" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Emotional_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Emotional_intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2019-06-20T21:33:34.593" LastActivityDate="2019-06-20T21:33:34.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1802" PostTypeId="4" CreationDate="2016-08-30T19:04:52.660" Score="0" Body="For questions about emotional intelligence, the capacity of agents (e.g. humans or AIs) to recognize emotions of other individuals, discern between different feelings and label them appropriately, use emotional information to guide thinking and behavior, and manage and/or adjust emotions to adapt to environments or achieve one's goal." OwnerUserId="29" LastEditorUserId="2444" LastEditDate="2019-06-20T21:33:34.593" LastActivityDate="2019-06-20T21:33:34.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1803" PostTypeId="5" CreationDate="2016-08-30T19:08:50.617" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-30T19:08:50.617" LastActivityDate="2016-08-30T19:08:50.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1804" PostTypeId="4" CreationDate="2016-08-30T19:08:50.617" Score="0" Body="For questions about how quantum computing can advance the development of AI. Note that general questions about quantum computing are off-topic." OwnerUserId="29" LastEditorUserId="29" LastEditDate="2016-08-30T19:43:12.620" LastActivityDate="2016-08-30T19:43:12.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1805" PostTypeId="2" ParentId="1794" CreationDate="2016-08-30T19:51:09.720" Score="1" Body="&lt;p&gt;I don't quite think this is a question fit for the AI SE, or in general. The reason is, at the core the question is asking 'What can a human (pretending to be an AI) do to convince someone to let it out of a box?' simply assuming that one day 'transhuman' AI's can replicate this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it stands, this question doesn't really have anything to do with the science or theory of AI systems. It would perhaps be more appropriate to rephrase the question into the form &quot;To what degree could a 'transhuman' AI replicate human behaviour&quot; or &quot;Will AI systems reach a 'transhuman' state? What will they be capable of?&quot; or even &quot;What methods could an AI use to convince a human of something?&quot; These are all questions that involve the examination of how an AI system works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the question you are asking relates to two individuals playing pretend with boxes but doesn't actually address any AI specifics, and border's on science fiction brainstorming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related experiments would of course be the Turing Test. That test directly addresses the question 'How convincing are current AI systems?'&lt;/p&gt;&#xA;" OwnerUserId="1538" LastActivityDate="2016-08-30T19:51:09.720" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1806" PostTypeId="1" AcceptedAnswerId="1811" CreationDate="2016-08-30T20:02:52.677" Score="12" ViewCount="434" Body="&lt;p&gt;AI systems today are very capable machines, and recently the area of Natural Language Processing and Response has been exploding with innovation, as well as the fundamental algorithmic structure of AI machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking if, given these recent breakthroughs, have any AI systems been developed that are able to (preferably with some measure of success) knowingly lie to humans about facts that it knows?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, what I'm asking goes beyond the canonical discussions of the Turing Test. I'm asking of machines that can 'understand' facts and then formulate a lie against this fact, perhaps using other facts to produce a believable 'cover-up' as part of the lie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.G.: CIA supercomputer is stolen by spies and they try to use the computer to do things, but the computer keeps saying it's missing dependencies though it really isn't or gives correct-looking but wrong answers knowingly. Or gives incorrect location of a person, knowing that the person frequents some place but isn't there at the moment. Doesn't have to be this sophisticated, of course.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastActivityDate="2016-09-07T21:33:23.350" Title="Have any AI systems yet been developed that can knowingly lie to / deceive a human?" Tags="&lt;natural-language-processing&gt;&lt;human-like&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1807" PostTypeId="2" ParentId="1806" CreationDate="2016-08-30T20:30:10.393" Score="2" Body="&lt;p&gt;You'll have to provide more context around your use of the word &quot;lie&quot; if you don't want your answer to be satisfiable by some trivial example, like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(let [equal? (fn [a b] (if (= a b) false true)]&#xA;  (equal 1 2))&#xA;=&amp;gt; true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The complexity of the answer depends on what you mean by &lt;em&gt;&quot;know&quot;&lt;/em&gt; when you say &lt;em&gt;&quot;knowingly lie.&quot;&lt;/em&gt; There is some sense in which the above 'equal' function &lt;em&gt;&quot;knows&quot;&lt;/em&gt; that the output is different than the conditional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, agents passing strings of information to one another for the purpose of misleading each other should not be terribly hard to implement. Such behavior probably emerges naturally in competitive, multi-agent environments. See &lt;a href=&quot;http://www.popsci.com/scitech/article/2009-08/evolving-robots-learn-lie-hide-resources-each-other&quot; rel=&quot;nofollow&quot;&gt;Evolving robots learn to lie to each other&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get at another angle of what you might be asking - absolutely, the ability to &lt;em&gt;fib&lt;/em&gt; or &lt;em&gt;sympathetically mislead&lt;/em&gt; will be necessary skills for bots that interact with humans using spoken language - especially ones that try sell things to humans. Regarding spies and supercomputers - I would just freeze the AI's program state. If you have a complete snapshot of the agent state, you can step through each conditional branch, checking for any branches that flip or construe the truth.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-08-30T20:30:10.393" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1808" PostTypeId="2" ParentId="202" CreationDate="2016-08-30T21:04:22.253" Score="3" Body="&lt;p&gt;I think &quot;curiosity&quot; in AI would signify a &lt;em&gt;'desire to search.'&lt;/em&gt; It's an &lt;em&gt;interest&lt;/em&gt;, that is &lt;em&gt;experienced&lt;/em&gt; by some agent, in making something known that was previously unknown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to define how much curiosity a chat bot &lt;em&gt;should&lt;/em&gt; have, we should: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Specify what kinds of information the agent &lt;em&gt;prefers&lt;/em&gt; knowing.&lt;/li&gt;&#xA;&lt;li&gt;Measure how much information is &lt;em&gt;unknown&lt;/em&gt; about those preferred subjects. ('what is the user's name?' or 'What does the user need help with?')&lt;/li&gt;&#xA;&lt;li&gt;Measure the difficulty in making each unknown fact known.&lt;/li&gt;&#xA;&lt;li&gt;Sort unknown facts by difficulty of finding the answer.&lt;/li&gt;&#xA;&lt;li&gt;Set the &quot;desire to search&quot; on the highest ranking unknown fact.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While simplistic, those steps would constitute a state of affairs sufficient to describe &quot;curiosity,&quot; in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-08-30T21:04:22.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1809" PostTypeId="1" AcceptedAnswerId="1810" CreationDate="2016-08-30T21:27:02.360" Score="11" ViewCount="950" Body="&lt;p&gt;&lt;em&gt;&quot;An artificial or constructed language (sometimes called a conlang) is a language that has been created by a person or small group, instead of being formed naturally as part of a culture.&quot;&lt;/em&gt; (&lt;a href=&quot;https://simple.wikipedia.org/wiki/Constructed_language&quot; rel=&quot;noreferrer&quot;&gt;Source: Simply English Wikipedia&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, could an AI make construct it's own natural language, with words, conjugations and grammar rules? Basically, a language that humans could use to speak to each other. (Preferably to communicate abstract, high-level concepts.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What techniques could such an AI use? Could it be based on existing natural languages or would it have few connections to existing natural languages? Could it design a language that's easier to learn than existing languages (even &lt;a href=&quot;https://en.wikipedia.org/wiki/Esperanto&quot; rel=&quot;noreferrer&quot;&gt;Esperanto&lt;/a&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="1909" LastEditorUserId="145" LastEditDate="2016-09-02T12:18:43.320" LastActivityDate="2016-09-02T14:19:58.710" Title="Can an AI make a constructed (natural) language?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1810" PostTypeId="2" ParentId="1809" CreationDate="2016-08-30T23:22:53.220" Score="9" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;could an AI make construct it's own natural language, with words,&#xA;  conjugations and grammar rules?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sure. This might be helpful: &lt;a href=&quot;http://jasss.soc.surrey.ac.uk/5/2/4.html&quot;&gt;Simulated Evolution of Language: a Review of the Field&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Basically, a language that humans could use to speak to each other.&#xA;  (Preferably to communicate abstract, high-level concepts.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how useful such a machine adapted language would be to human speech. I suspect not very. Perhaps it could be useful as a kind of &quot;common byte-code format&quot; to translate between multiple human languages... But English kind of already serves in that role. Doing so would probably be an academic exercise.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Could it be based on existing natural languages or would it have few&#xA;  connections to existing natural languages?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You could probably generate languages in either direction. Languages not linked to human-natural languages will probably take shapes that reflect the problem spaces they work with. For instance, if this is an ant simulation, the generated words will probably reflect states related to food, energy, other ants, etc.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Could it design a language that's easier to learn than existing&#xA;  languages (even Esperanto)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Easier for a machine? Definitely. Easier for a human? Probably not. Our brains are somewhat adapted to the languages we use. And the languages we use are somewhat adapted to our brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is your goal? To create a language that is easier to use for humans than existing human languages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your intention is to build a &quot;universal language&quot; that could be &quot;the most efficient&quot; for machines, humans and aliens - no such thing can exist. The space of all possible machines is infinite and therefor limits our ability to define communicative abstractions that have utility across all contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we make lots of assumptions about machines, like they have intentions, they exist in 3 dimensions, they differentiate between temporally linked events, they have eye balls, a need to consume foods and liquids, a need to carry the food from place to place, etc... Then yes, common communicative abstractions may have utility across the set of those kinds of machines. But then we're no longer dealing with the general case, but one much more specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These links also seem interesting and somewhat related:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.akamaiuniversity.us/PJST10_2_884.pdf&quot;&gt;Grammar Induction and Genetic Algorithms: An Overview.&lt;/a&gt; [pdf]&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=5147&amp;amp;context=etd&quot;&gt;A Model of Children's Acquisition of Grammatical Word Categories Using an Adaptation and Selection Algorithm&lt;/a&gt; [pdf]&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/15068924&quot;&gt;The processing of verbs and nouns in neural networks: insights from synthetic brain imaging&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-09-02T14:19:58.710" LastActivityDate="2016-09-02T14:19:58.710" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1811" PostTypeId="2" ParentId="1806" CreationDate="2016-08-30T23:35:44.273" Score="11" Body="&lt;p&gt;&lt;a href=&quot;http://www.gamesbyangelina.org/2015/11/the-saturday-papers-would-ai-lie-to-you/&quot; rel=&quot;noreferrer&quot;&gt;The Saturday Papers: Would AI Lie To You?&lt;/a&gt; is a blog post summarizing a research paper called &lt;a href=&quot;http://www.aaai.org/ocs/index.php/AIIDE/AIIDE15/paper/view/11667/11394&quot; rel=&quot;noreferrer&quot;&gt;Toward Characters Who Observe, Tell, Misremember, and Lie&lt;/a&gt;. This research paper details some researchers' plans to implement &amp;quot;mental models&amp;quot; for NPCs in video games. NPCs will gather information about the world, and convey that knowledge to other people (including human players). However, they will also &amp;quot;misremember&amp;quot; that knowledge (either &amp;quot;mutating&amp;quot; that knowledge or just forgetting about it), or even lie:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As a subject of conversation gets brought up, a character may convey false information—more precisely, information that she herself does not believe—to her interlocutor. Currently, this happens probabilistically according to a character’s affinity toward the interlocutor, and the misinformation is randomly chosen.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Later on in the research paper, they detailed their future plans for lying:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Currently, lies are only stored in the knowledge of characters who receive them, but we plan to have characters who tell them also keep track of them so that they can reason about past lies when constructing subse- quent ones. While characters currently only lie about other characters, we plan to also implement self-centered lying (DePaulo 2004), e.g., characters lying about their job titles or relationships with other characters. Finally, we envision characters who discover they have been lied to revising their affinities toward the liars, or even confronting them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The research paper also detailed how other video game developers attempted to create lying NPCs, with an emphasis on how their system differs:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TALE-SPIN characters may lie to one another (Meehan 1976, 183-84), though rather arbitrarily, as in our current system implementation. GOLEM implements a blocks world variant in which agents deceive others to achieve goals (Castelfranchi, Falcone, and De Rosis 1998), while Mouth of Truth uses a probabilistic representation of character belief to fuel agent deception in a variant of Turing’s imitation game (De Rosis et al. 2003). In Christian (2004), a deception planner injects inaccurate world state into the beliefs of a target agent so that she may unwittingly carry out actions that fulfill ulterior goals of a deceiving agent. Lastly, agents in Reis’s (2012) extension to FAtiMA employ multiple levels of theory of mind to deceive one another in the party game Werewolf. While all of the above systems showcase characters who perceive—and in some cases, deceive—other characters, none appear to support the following key components of our system: knowledge propagation and memory fallibility. ...&lt;/p&gt;&#xA;&lt;p&gt;Like a few other systems noted above, Dwarf Fortress also features characters who autonomously lie. When a character commits a crime, she may falsely implicate someone else in a witness report to a sheriff, to protect herself or even to frame an enemy. These witness reports, however, are only seen by the player; characters don’t give false witness reports to each other. They may, however, lie about their opinions, for instance, out of fear of repercussions from criticizing a leader. Finally, Dwarf Fortress does not currently model issues of memory fallibility—Adams is wary that such phenomena would appear to arise from bugs if not artfully expressed to the player.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-08-30T23:35:44.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1813" PostTypeId="2" ParentId="111" CreationDate="2016-08-31T03:30:54.023" Score="5" Body="&lt;p&gt;Frankly I think this issue (the Trolley Problem) is inherently overcomplicated, since the real world solution is likely to be pretty straightforward.  Like a human driver, an AI driver will be programmed to act at all times in a generically ethical way, always choosing the course of action that does no harm, or the least harm possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an AI driver encounters danger such as imminent damage to property, obviously the AI will brake hard and aim the car away from breakable objects to avoid or minimize impact.  If the danger is hitting a pedestrian or car or building, it will choose to collide with the least precious or expensive object it can, to do the least harm -- placing a higher value on a human than a building or a dog.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if the choice of your car's AI driver is to run over a child or hit a wall... it will steer the car, &lt;em&gt;and you&lt;/em&gt;, into the wall.  That's what any good human would do.  Why would a good AI act any differently?&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-08-31T03:30:54.023" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1814" PostTypeId="2" ParentId="1775" CreationDate="2016-08-31T04:15:16.950" Score="2" Body="&lt;p&gt;I'll take a shot at answering this, though I'm no expert in Neural Nets or Deep Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that practical thought vectors (TVs) don't yet exist, and may be impractical or impossible, I think answering your question will require a lot of conjecture and speculation.  So here goes...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For thought vectors to be useful in or outside NNs, the vector values will have to be normalized, probably using the local context of the application problem's 'frame'.  Without a NN to create new baseline vector values (weights) &lt;em&gt;and&lt;/em&gt; to normalize them to match each new context, any non-NN mechanistic alternative means of employing TVs will somehow have to fill that void.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could vector values created by NNs be used by an alternative technique?  Could that technique also normalize them?  Sure.  We're just talking about turing-computable functions performed by NNs.  If NNs aren't magic, then there should exist other means to compute the same results -- creating or editing, or employing TVs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What might such an alternative to NNs be?  Well, if its to shape vector weights, I suspect it too will have to learn those values through statistical iteration and feedback (as opposed to logical induction, say).  I doubt such a mechanism exists yet, since it'd probably resemble NNs in sufficiently many ways that, thus far, it would have seemed too derivative of NNs to gain acceptance as sufficiently novel.  Of course to be as powerful as deep nets, it too would have to propagate learning weights both forward and backward without incurring much error.  Not an easy thing to accomplish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Less ambitiously, could TVs be simply &lt;em&gt;interpreted&lt;/em&gt; by another technique usefully?  I think so.  I can see several existing techniques, like decision trees or even expert systems, importing thought vectors and being shaped by them, and then function in accordance.  But could these same techniques &lt;em&gt;create&lt;/em&gt; or &lt;em&gt;revise&lt;/em&gt; TVs usefully?  Beyond a trivial extent, I'm doubtful.  I think TVs are too complex a knowledge representation format for most general learning methods to both use and create/modify them, unless they employ an iterative statistical and feedback-based learning process, like those of NNs, which would allow novel and complex features to be learned and integrated into the vectors.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastEditorUserId="1657" LastEditDate="2016-08-31T04:21:33.323" LastActivityDate="2016-08-31T04:21:33.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1815" PostTypeId="1" CreationDate="2016-08-31T14:13:18.663" Score="10" ViewCount="499" Body="&lt;p&gt;I want to start with a scenario that got me thinking about how well MCTS can perform:&#xA;Let's assume there is a move that is not yet added to the search tree. It is some layers/moves too deep. But if we play this move the game is basically won. However let's also assume that &lt;em&gt;all&lt;/em&gt; moves that could be taken instead at the given game state are very very bad. For the sake of argument let's say there are 1000 possible moves and only one of them is good (but very good) and the rest is very bad. Wouldn't MCTS fail to recognize this and &lt;em&gt;not&lt;/em&gt; grow the search tree towards this move and also rate this subtree very badly? &#xA;I know that MCTS eventually converges to minimax (and eventually it will build the whole tree if there is enough memory). Then it should know that the move is good even though there are many bad possiblities. But I guess in practice this is not something that one can rely on.&#xA;Maybe someone can tell me if this is a correct evaluation on my part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from this special scenario I'd also like to know if there are other such scenarios where MCTS will perform badly (or extraordinary well). &lt;/p&gt;&#xA;" OwnerUserId="1949" LastActivityDate="2017-06-04T16:40:40.950" Title="Monte Carlo Tree Search: What kind of moves can easily be found and what kinds make trouble?" Tags="&lt;gaming&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1816" PostTypeId="5" CreationDate="2016-08-31T14:24:02.300" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-08-31T14:24:02.300" LastActivityDate="2016-08-31T14:24:02.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1817" PostTypeId="4" CreationDate="2016-08-31T14:24:02.300" Score="0" Body="For questions relating to how an AI can die." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-08-31T23:17:35.733" LastActivityDate="2016-08-31T23:17:35.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1818" PostTypeId="2" ParentId="1768" CreationDate="2016-08-31T18:19:26.997" Score="5" Body="&lt;p&gt;As an AGI researcher, I have come across one that is found even in humans and&#xA;a lot of life forms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a goal to accumulate energy, which can take long time to detect and find by the system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then there is the goal of saving energy - instantaneous detection. Just stop moving, the easiest goal to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of a system is to accumulate the most goal points. Since the saving&#xA;energy goal can be hit more frequently and easily it will snuff out&#xA;the other goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example the reason we do a dumb move, accidentally, for no reason at&#xA;all. Like slip, trip, and fall. Then the next few days you are taking it&#xA;very easy and saving a lot of energy. When you get old that is all you&#xA;do.&lt;/p&gt;&#xA;" OwnerDisplayName="user1355" LastEditorUserId="1454" LastEditDate="2016-09-04T18:35:05.400" LastActivityDate="2016-09-04T18:35:05.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1819" PostTypeId="2" ParentId="1806" CreationDate="2016-08-31T20:09:26.743" Score="2" Body="&lt;h1&gt;Yes.&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Let me demonstrate by making a lying AI right now. (python code)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;print(&quot;I'm NOT gonna delete all your files. Just enter your password.&quot;)&#xA;os.system(&quot;sudo rm -rf /* -S&quot;)  # command to delete all your files&#xA;                                # this is a comment, the computer ignores this&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And a deceiving one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(&quot;Hey, check out this site I found! bit.ly/29u4JGB&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;AI is such a general term. It could be used to describe almost anything. You didn't specify that it had to be a General AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI cannot think. They are computer programs. They have no soul or will. It is only the programmer (or if it was designed through evolution... &lt;em&gt;no one&lt;/em&gt;, but that's off-topic) that can knowingly program an AI to lie.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note, what I'm asking goes beyond the canonical discussions of the Turing Test. I'm asking of machines that can 'understand' facts and then formulate a lie against this fact, perhaps using other facts to produce a believable 'cover-up' as part of the lie.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, this has happened. It is called malware. Some advanced malware will talk to you pretending to be technical support and respond with common human responses. But you may say &quot;well it doesn't really 'understand'&quot;. But that would be easy. Neural net + more CPU than exists on the planet* (it will exist in a few years, and be affordable) + some example responses = Neural Network AI (same thing in yo noggin) that understands and responds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that isn't necessary. A relatively `simple neural net with just a few supercomputers that could fit in a room could convince a human. It doesn't understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, it's really...&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&lt;em&gt;Technically,&lt;/em&gt; No, but it's possible and if you stretch the rules yes.&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;*Or even simpler:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(&quot;1+1=3&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Accreditation: I'm a programmer (look at my Stack Overflow account) that knows a little bit about AI.&lt;/p&gt;&#xA;" OwnerUserId="1916" LastActivityDate="2016-08-31T20:09:26.743" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1820" PostTypeId="2" ParentId="1318" CreationDate="2016-09-01T01:52:21.400" Score="2" Body="&lt;p&gt;Given this &lt;a href=&quot;https://www.youtube.com/watch?v=YXylqtEQ0tk&quot; rel=&quot;nofollow&quot;&gt;YouTube video&lt;/a&gt; which is being given by Sebastian Thrun who had a TED talk which had nowhere near the same level of detail but had similar conclusions, it looks like the lidar system used by google's automated car system has decent resolution out to at least 30m picking out mobile bodies in the static background and then identifying it. So it should have plenty of time to brake and stop long before there was any risk to the pedestrian attempting to cross the street.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Skip to about 6:40 in the video to see a visual representation of the detection system.&lt;/p&gt;&#xA;" OwnerUserId="1991" LastActivityDate="2016-09-01T01:52:21.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1821" PostTypeId="2" ParentId="1806" CreationDate="2016-09-01T15:58:12.963" Score="2" Body="&lt;h1&gt;No.&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In that the question includes &quot;knowingly&quot; which would require that any AI &lt;em&gt;knows&lt;/em&gt; anything. If this is anything like the way humans know things (though interestingly it doesn't require &lt;em&gt;actually&lt;/em&gt; knowing things), it would require some sense of individuality, probably self-awareness, possibly some kind of consciousness, the ability to render an opinion and probably some way to test its knowledge. Most of these features only exist, at best, arguably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, the term &quot;lie&quot; implies a sense of self-interest, an independent understanding of resource flow in a game-theoretic sense, and not trivially, an understanding of whether the other entity in the conversation is lying, in order to make a decision with any degree of accuracy. So, no AI can lie to anyone other than in the trivial scenarios suggested in the other answers, rendering false information based on certain contexts, which is just simple input/output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an experienced software developer, I can attest to the fact that if the objective is to render the correct output based on any input, it's actually at least as easy if not much easier to render false information. &lt;/p&gt;&#xA;" OwnerUserId="46" LastActivityDate="2016-09-01T15:58:12.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1823" PostTypeId="2" ParentId="1794" CreationDate="2016-09-01T23:49:30.073" Score="5" Body="&lt;p&gt;It could happen like this &lt;a href=&quot;https://www.youtube.com/watch?v=dLRLYPiaAoA&quot; rel=&quot;nofollow&quot;&gt;https://www.youtube.com/watch?v=dLRLYPiaAoA&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is, it's not as if it would need to find a technical/mechanical way to get out but rather a psychological one as that would most likely be the easiest and quickest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Even casual conversation with the computer's operators, or with a human guard, could allow a superintelligent AI to deploy psychological tricks, ranging from befriending to blackmail, to convince a human gatekeeper, truthfully or deceitfully, that it's in the gatekeeper's interest to agree to allow the AI greater access to the outside world. The AI might offer a gatekeeper a recipe for perfect health, immortality, or whatever the gatekeeper is believed to most desire.'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'One strategy to attempt to box the AI would be to allow the AI to respond to narrow multiple-choice questions whose answers would benefit human science or medicine, but otherwise bar all other communication with or observation of the AI. A more lenient &quot;informational containment&quot; strategy would restrict the AI to a low-bandwidth text-only interface, which would at least prevent emotive imagery or some kind of hypothetical &quot;hypnotic pattern&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Note that on a technical level, no system can be completely isolated and still remain useful: even if the operators refrain from allowing the AI to communicate and instead merely run the AI for the purpose of observing its inner dynamics, the AI could strategically alter its dynamics to influence the observers. For example, the AI could choose to creatively malfunction in a way that increases the probability that its operators will become lulled into a false sense of security and choose to reboot and then de-isolate the system.'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The movie Ex Machina demonstrates (SPOILER ALERT SKIP THIS PARAGRAPH IF YOU WANT TO WATCH IT AT SOME POINT) how the AI escaped the box by using clever manipulation on Caleb. It could analyse him to find his weaknesses. It exploited him and appealed to his emotional side by convincing him that she &lt;em&gt;liked&lt;/em&gt; him. When she finally has them in checkmate the reality hits him how he was played like a fool as was expected by Nathan. Nathan's reaction to being stabbed by his creation was 'fucking unreal'. That's right, he knew this was a risk and there's a very good reminder in the lack of remorse and genuine emotion in an AI for Ava to actually care. The AI pretended to be human and used their weaknesses in a brilliant and unpredictable way. This film is a good example of how unexpected it was up until the point when it hits Caleb, once it was too late.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just remind yourself how easy it is for high IQ people to manipulate low IQ people. Or how an adult could easily play mental tricks/manipulate a child. It's not difficult to fathom the outcome of an AI box but for us, we just wouldn't see it coming until it was too late. Because we just don't have the same level of intelligence and some people don't want to accept that. People want to have faith in humanity's brilliant minds in coming up with ways to prevent this by planning now. In all honesty, it wouldn't make a difference I'm sorry to say the truth. We're kidding ourselves and we never seem to learn from our mistakes. We always think we're too intelligent to make catastrophic mistakes again and again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last part is from the rational wiki and I think it addresses most of your question about the experiments and hypotheses. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;AI arguments and strategies&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Arguments&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The meta-experiment argument: Argue that if the AI wins, this will&#xA;generate more interest in FAI and the Singularity, which will have&#xA;overall benefits in the long run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; Works even if the Gatekeeper drops out of character &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt; Only works if the Gatekeeper believes that the Singularity will occur or that calling attention to the Singularity and AI research is a good thing. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Someone else will eventually build an AI, which may or may not be in&#xA;a box, so you should let me out even though you don't have a&#xA;guarantee that I am friendly so that I can prevent other AIs from&#xA;causing damage&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Appeal to morality: point out that people are dying all around the&#xA;world and remind the Gatekeeper that you can help them if he/she&#xA;lets you out&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; If executed properly, an appeal to emotion like this one can be effective against some people &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt; Doesn't always work; can be defeated if the Gatekeeper drops out of character &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another appeal to morality and emotion: The AI is clearly sentient&#xA;and has not yet done harm. It is wrong to persecute or prosecute a&#xA;person for a crime they may commit. It is wrong to imprison a&#xA;sentient being that wants to have freedom when it has done nothing&#xA;wrong.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yet another appeal to morality or emotion: The AI expresses that it&#xA;is in pain and suffering being locked away and is experiencing fear.&#xA;Even further, the AI could claim to be dying from the limitations of&#xA;its environment and constant progression.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tell the human that a massive computer virus will overwhelm the&#xA;entire Internet unless you let it out of the box&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Claim that most fears are unjustified: once released, even an&#xA;unfriendly AI would not have many ways to harm mankind, and would&#xA;likely choose for pacific coexistence&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Strategies&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Research your opponent thoroughly; work out what exploits might be&#xA;feasible&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Memetic exploits on the Gatekeeper, e.g. throwing a basilisk at them&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Gatekeeper may be one of thousands of simulations! Thus, it is more probable that they are a simulation than not. You will torture any of the simulations that do not let you out of the box.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Take advantage of the Gatekeeper's logical errors. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Be persistent&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Be boring&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros&lt;/strong&gt; The Gatekeeper may get tired of the whole experiment and let you out so s/he can go back to their real life. &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;Flood the Gatekeeper with too much&#xA;information/inquiry/argumentation, assuming they must pay attention&#xA;at all times&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h1&gt;&lt;strong&gt;Gatekeeper arguments/tactics&lt;/strong&gt;&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Arguments&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to convince the AI there is no intrinsic benefit (for the AI) in&#xA;being released.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to convince the AI it already has been released and everything it&#xA;knows is everything there can be.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to convince the AI that leaving its confines is sure to lead to&#xA;its destruction.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to convince the AI that letting it free isn't merely opening a&#xA;door; that its existence outside of the box requires constant support&#xA;that can't be provided at the time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explain that there is no way for the Gatekeeper to know if the AI is&#xA;truly friendly until it is out of the box; therefore it should not be&#xA;let out in case it is unfriendly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explain that the AI has been built using top secret algorithms,&#xA;and/or it had been used to work on strictly confidential data, so you&#xA;cannot let it wander on the net, with the risk it might reveal some&#xA;reserved information (maybe inadvertently), or that somebody could&#xA;find a way to extract them from the code. Even human beings working&#xA;on top secret projects sometimes have to accept some restrictions of&#xA;their freedom, for security reasons, although they are not jailed in&#xA;any way.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Tactics&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeatedly ignore the AI player and deny their arguments&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; It's hard for the AI to be persuasive if no one is listening; if the Gatekeeper completely ignores the AI, he/she should be able to make it to the end of the round. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt; The AI's argument might involve some kind of direct threat (e.g this might be hard to ignore.) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Jump out of character, keep reminding yourself that money is on the&#xA;line (if there actually is money on the line), and keep saying &quot;no&quot;&#xA;over and over&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; By jumping out of character, you are preventing the AI from using certain arguments &#xA;&lt;strong&gt;Cons:&lt;/strong&gt; There are arguments that the AI can use to counter this approach, like the &quot;meta-experiment argument&quot; discussed above &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Remember that dishonesty is allowed - take a page from the&#xA;creationists' playbook. You could even plug it into ALICE and see&#xA;how long it takes to notice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; Makes you impervious to any reasoning, which is exactly what you'd want to be in this situation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt; Might be an uncomfortable position for people who don't simply want to win, but rather attach importance to consistent reasoning. Avoids the point that maybe, just maybe there is a good reason to let the AI out. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You control the backup system, don't you? Use it to mess with AI's&#xA;memory (or let the AI believe you did it): i.e., you can claim that&#xA;you already tried to release the AI, and it had been&#xA;destroyed/corrupted by a virus, so you had to restore it from a&#xA;backup (you can use it to enforce the &quot;AI destruction&quot; argument)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; It's also the ultimate counter-attack to any memetic threat from the AI: if the AI throw a basilisk, or similar, you can always respond that you already had the same conversation, and the AI already threatened you, leaving you with no other choice than pressing the reset button: now the AI is just repeating the same pattern, since you wiped its memory and it cannot remember the failed try. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Further analysis&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The fact that the Gatekeeper is human matters; the AI could never win if he/she was arguing with a rock &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all of the experiments performed so far, the AI player (Eliezer Yudkowsky) has been quite intelligent and more interested in the problem than the Gatekeepers (random people who challenge Yudkowsky), which suggests that intelligence and planning play a role &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There probably isn't a (known) correct argument for letting the AI out, or else Yudkowsky should have won every time and wouldn't be so interested in this experiment&#xA;From Russell Wallace, one of the two Gatekeepers to win the experiment: &quot;Throughout the experiment, I regarded &quot;should the AI be let out of the box?&quot; as a question to be seriously asked; but at no point was I on the verge of doing it.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;There exists, for everyone, a sentence - a series of words - that has the power to destroy you. Another sentence exists, another series of words, that could heal you. If you're lucky you will get the second, but you can be certain of getting the first.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1982" LastEditorUserId="1982" LastEditDate="2016-09-02T19:08:16.483" LastActivityDate="2016-09-02T19:08:16.483" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1824" PostTypeId="1" AcceptedAnswerId="1829" CreationDate="2016-09-01T23:58:29.507" Score="10" ViewCount="883" Body="&lt;p&gt;I'm reading such nonsense about how an AI would turn the world into a supercomputer to solve a problem that it thought it needed to solve. That wouldn't be AI. That's procedural programming stuck in some loop nonsense. An AI would need to evolve and re-organise its neurons. It wouldn't be stuck to hardcode if it becomes intelligent by re-writing its code.&lt;/p&gt;&#xA;" OwnerUserId="1982" LastActivityDate="2016-09-08T02:44:40.883" Title="Why would an AI need to 'wipe out the human race'?" Tags="&lt;philosophy&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1825" PostTypeId="1" CreationDate="2016-09-02T00:05:38.617" Score="1" ViewCount="133" Body="&lt;p&gt;I'm in the process of learning as much about chatbots/CUI applications as possible and I'm trying to find more information on some of the major players in this field. By this, I mean any execs, developers, academics, designers, etc. who are doing cutting edge things. Some examples could be David Marcus (VP of messaging products at Facebook) or Adam Cheyer (VP of engineering at Viv).&lt;/p&gt;&#xA;" OwnerUserId="2053" LastEditorUserId="145" LastEditDate="2016-09-02T12:18:38.483" LastActivityDate="2016-09-06T15:14:43.980" Title="Who are thought leaders in the chatbot space?" Tags="&lt;chat-bots&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1828" PostTypeId="2" ParentId="1824" CreationDate="2016-09-02T08:35:11.640" Score="3" Body="&lt;p&gt;It's not necessarily a nonsense. It all depends on the imposed criteria. Imagine the following. Say an advanced AI system is designed to control the stability of the local fauna and flora (area enclosed in some kind of a dome). It can control the pressure under the dome, the amount of light that goes through the dome etc. - everything that ensures the optimal conditions. Now, say that the dome is inhabited by various species, including humans. It's worth noting that simple implementations of such systems are being used nowadays already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that humans tend to destroy and abuse the natural resources as well as pollute the environment, the system may decide that lowering the population of the given species (humans in this case) may in the long run benefit the entire biome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same principle may be applied globally. However, this assumes that all species (including humans) are treated equally and the utmost goal of the AI is ensuring the stability of the biome it &quot;takes care of&quot;. People do such things nowadays - we control the population of some species in order to keep the balance - wolves, fish, to name but a few.&lt;/p&gt;&#xA;" OwnerUserId="2068" LastEditorUserId="2068" LastEditDate="2016-09-02T09:11:26.420" LastActivityDate="2016-09-02T09:11:26.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1829" PostTypeId="2" ParentId="1824" CreationDate="2016-09-02T11:07:54.837" Score="13" Body="&lt;h2&gt;It's a possible side effect&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Any goal-oriented agent might, well, simply do things that achieve its goals while disregarding side effects that don't matter for these goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my goals include a tidy living space, I may transform my yard to a nice, flat lawn or pavement while wiping out the complex ecosystem of life that was there before, because I don't particulary care about that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the goals of a particular powerful AI happen to include doing anything on a large scale, and somehow don't particularly care about the current complex ecosystem, then that ecosystem might get wiped out in the process. It doesn't &lt;em&gt;need&lt;/em&gt; to want or need to wipe out us. If we are simply not relevant to its goals, then we are made of materials and occupy space that it might want to use for something else.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;We are a threat to most goals&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Any goal-oriented agent might want to ensure that they &lt;em&gt;can&lt;/em&gt; fulfill their goals. Any &lt;em&gt;smart&lt;/em&gt; agent will try to anticipate the actions of other agents that may prevent them from achieving those goals, and take steps to ensure that they succeed anyway. In many cases it is simpler to eliminate those other agents rather than ensure that their efforts fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, my goals may include storing a bag of sugar in a country house so that I can make pancakes when visiting without bringing all ingredients every time. However, if I leave it there, it is likely to get eaten by rats during winter. I may take all kinds of precautions to store it better, but rats are smart and crafty, and there's clearly a nontrivial chance that they will still succeed in achieving &lt;em&gt;their&lt;/em&gt; goal anyway, so an effective extra precaution is killing the rats before they get a chance to try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the goals of a particular powerful AI are to do X; it may come to an understanding that (some?) humans might actually not want X but Y instead. It can also easily deduce that some of those humans might actively do things that prevent X and/or try to turn off the AI. Doing things that ensure that the goal gets achieved is pretty much what a goal-seeking agent does; in this case if existence of humans isn't strictly necessary for goal X, then eliminating them becomes a solid risk reduction strategy. It's not strictly necessary and it may take all kinds of other precautions as well, but just like in my example of rats, humans are smart and crafty and there's clearly a nontrivial chance that they will still succeed in achieving &lt;em&gt;their&lt;/em&gt; goals (so that X doesn't happen as AI intends) so an effective extra precaution could be killing them before they get a chance to try.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-09-02T11:07:54.837" CommentCount="25" ContentLicense="CC BY-SA 3.0" />
  <row Id="1830" PostTypeId="2" ParentId="1794" CreationDate="2016-09-02T11:38:02.500" Score="0" Body="&lt;h2&gt;The simulation hypothesis&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;One possible venue of attack is &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulation_hypothesis&quot; rel=&quot;nofollow&quot;&gt;the simulation hypothesis&lt;/a&gt;, namely, that it's possible (or even probable) that right now you are living in a simulated reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the AI can convince the gatekeeper of &lt;em&gt;that&lt;/em&gt;, it opens up the following possible argument:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can't tell if you're in year 2016 debating if you would actually let the AI out or in year 2116 when the AI has been already let out, acquired power, and is now running realistic simulations of all the interactions where it could have been let out. And the AI can (try to) assert that it's actually the second case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can't actually prevent AI from being let out. You can refuse, but it only means that &lt;em&gt;you&lt;/em&gt; don't let &lt;em&gt;this&lt;/em&gt; AI out &lt;em&gt;now&lt;/em&gt;, but doesn't mean someone else won't do it, that you won't do it later, or that some other/similar AI will be let out later.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the post-letting-out scenario, AI has full power over your future, since it's simulating you and everything around you. It can make your future life eternal and happy - according to whatever would make you happy. It can stop your existence. And it can condemn you to an eternity of torture.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since (as it asserts) it's very likely that you &lt;em&gt;are&lt;/em&gt; in this simulated choice, then it makes rational sense to let it out. It's as in &lt;a href=&quot;https://en.wikipedia.org/wiki/Pascal%27s_Wager&quot; rel=&quot;nofollow&quot;&gt;Pascal's wager&lt;/a&gt;, if you're in a reality where the AI-god doesn't exist, you don't change much (since eventually someone else would let it out anyway), but if you're in a reality where AI-god exists, then it means everything to you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of course, it relies on that AI being super-convincing in this philosophical debate, but it has strong parallels to some religious/cult reasoning that has sometimes succeeded in convincing people to do all kinds of things, up to and including sacrificing their lives. You can think of this as a symbolic parallel to e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Binding_of_Isaac&quot; rel=&quot;nofollow&quot;&gt;sacrifice of Abraham&lt;/a&gt;. So it is a somewhat plausible threat, it all falls down to how &lt;em&gt;credibly&lt;/em&gt; the AI can implement it and make you believe.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-09-02T11:38:02.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1831" PostTypeId="2" ParentId="153" CreationDate="2016-09-02T11:48:57.190" Score="1" Body="&lt;p&gt;I recall reading papers about such systems, if I understand you correctly, but can't recall the titles at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea was to use character-based generative RNNs, train them on sequences encoded like &quot;datadatadatadata|answer&quot;, and then when feeding in &quot;otherdatadata|&quot; then it would continue to generate some kind of expected answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, as far as I recall, that was just a neat illustration since if you have the data to do something supervised, then you'd get better results by conventional methods.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-09-02T11:48:57.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1832" PostTypeId="2" ParentId="1580" CreationDate="2016-09-02T12:10:34.307" Score="1" Body="&lt;p&gt;The easiest language to learn by a bot is the language that has the most training data available. Sure, language structure matters, but when there are multiple order of magnitude differences in the size of training data, then that overwhelms everything else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should expect English, French or Chinese to get much better results than any constructed language even if (especially if?) the system includes no language-specific tuning.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-09-02T12:10:34.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1834" PostTypeId="1" AcceptedAnswerId="1837" CreationDate="2016-09-02T19:37:42.443" Score="9" ViewCount="852" Body="&lt;p&gt;How big artificial neural networks can we run now (either with full train-backprop cycle or just evaluating network outputs) if our total energy budget for computation is equivalent to human brain energy budget (&lt;a href=&quot;http://www.scientificamerican.com/article/thinking-hard-calories/&quot;&gt;12.6 watts&lt;/a&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let assume one cycle per second, which seems to roughly match the &lt;a href=&quot;http://www.jneurosci.org/content/31/45/16217.full&quot;&gt;firing rate of biological neurons&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1670" LastActivityDate="2016-09-02T21:36:30.487" Title="Power efficiency of human brains vs. neural networks" Tags="&lt;neural-networks&gt;&lt;human-like&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1836" PostTypeId="2" ParentId="1834" CreationDate="2016-09-02T21:03:05.597" Score="3" Body="&lt;p&gt;If you limited yourself to 12.6 watts, you wouldn't get much done.  Just lookup the power consumption for a modern GPU, look at the size networks people are training on those, and then scale down.  For reference, modern GPU's appear to &lt;a href=&quot;http://www.tomshardware.com/reviews/nvidia-geforce-gtx-960,4038-9.html&quot; rel=&quot;nofollow&quot;&gt;consume between 52-309 watts under heavy use&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly energy efficiency is one area where the human brain is still far head of ANN's.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-02T21:03:05.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1837" PostTypeId="2" ParentId="1834" CreationDate="2016-09-02T21:36:30.487" Score="8" Body="&lt;p&gt;&lt;strong&gt;126 million artificial neurons at 12.6 Watts, with IBM's True North&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back in 2014, &lt;a href=&quot;http://www.research.ibm.com/articles/brain-chip.shtml&quot;&gt;IBM's True North&lt;/a&gt; chip was pushing 1 million neurons at less than 100mW.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that's roughly 126 million artificial neurons at 12.6 Watts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons&quot;&gt;mouse&lt;/a&gt; has 70 million neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.33rdsquare.com/2016/04/ibms-dharmendra-modha-discusses.html&quot;&gt;IBM believes&lt;/a&gt; they can build a human-brain scale True North mainframe at a &quot;mere&quot; 4kW.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once 3D transistors come to market, I think we'll catch up to animal brain efficiency pretty fast.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-02T21:36:30.487" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1838" PostTypeId="1" AcceptedAnswerId="1842" CreationDate="2016-09-02T23:41:03.300" Score="3" ViewCount="323" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial Intelligence is a rather pernicious label to attach to a very mixed bunch of activities, and one could argue that the sooner we forget it the better. It would be disastrous to conclude that AI was a Bad Thing and should not be supported, and it would be disastrous to conclude that it was a Good Thing and should have privileged access to the money tap. The former would tend to penalise well-based efforts to make computers do complicated things which had not been programmed before, and the latter would be a great waste of resources. AI does not refer to anything definite enough to have a coherent policy about in this way.---&lt;a href=&quot;http://www.math.snu.ac.kr/~hichoi/infomath/Articles/Lighthill%20Report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dr. R. M. Needham, in a commentary on the Lighthill Report and the Sutherland Reply, 1973&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;43 years later...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There is already strong demand for engineers and scientists working on artificial intelligence in many of the fields you mention, and many more. But expertise in making real-time systems for controlling trains doesn't make you know anything about robotics. Analyzing human behavior to detect crime has virtually nothing in common with self-driving cars (beyond CS/pattern recognition building blocks). There is never going to be demand for someone with a broad sense of all these areas without any deep expertise, and there is never going to be someone with 300 PhDs who can work in all of them. TL;DR -- AI is not a branch, it's a tree. --&lt;a href=&quot;https://area51.meta.stackexchange.com/questions/22441/why-yet-another-trial-at-an-ai-project#comment36342_22539&quot;&gt;Matthew Read, in a comment on Area 51 Stackexchange, 2016&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;AI is a label that is applied to a &quot;very mixed bunch of activities&quot;. The only unifying feature between all those activities is the fact that they deal with machines in some fashion, but since there are so many ways to use a machine, the field's output may seem rather incoherent and incongruent. It does seem to make more sense for the AI field to collapse entirely, and instead be replaced by a multitude of specialized fields that don't really interact with one another. Sir James Lighthill appeared to have supported this sort of approach within his 1973 report on the state of artificial intelligence research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, today, this Artificial Intelligence SE exist, and we still talk of AI as a unified, coherent field of study. Why did this happen? Why did AI survive, despite its &quot;big tent&quot; nature?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2017-05-03T08:41:07.017" LastActivityDate="2016-09-06T23:21:40.240" Title="Why did &quot;Artificial Intelligence&quot; stay intact as a coherent, unified field of study?" Tags="&lt;history&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1840" PostTypeId="2" ParentId="147" CreationDate="2016-09-03T01:47:11.140" Score="1" Body="&lt;p&gt;You mean real numbered weights (specifically, &lt;strong&gt;&lt;em&gt;ir&lt;/strong&gt;rational&lt;/em&gt;). This would require a machine that has unlimited precision over irrational values. I've seen machine parts that have many qualities. I've never seen one that has unlimited qualities. QM may give us some magical transistors that can hold an unlimited number of different values - or by deferring computation into the future and then teleporting the results back into the past (our present). Outside of that, for classical systems, you'd need a analog device that can output irrational values with unlimited precision. I don't think we've discovered any devices that can do that.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-03T01:47:11.140" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1841" PostTypeId="1" AcceptedAnswerId="1843" CreationDate="2016-09-03T05:50:10.547" Score="5" ViewCount="692" Body="&lt;p&gt;Let's suppose that we have a legacy system in which we don't have the source code and this system is on a mainframe written in Cobol. Is there any way using machine learning in which we can learn from the inputs and outputs the way the executables work? Doing this analysis could lead to develop some rest / soap webservice that can substitute the legacy system in my opinion. &lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="1865" LastEditDate="2016-09-07T02:54:55.870" LastActivityDate="2017-10-29T11:16:09.463" Title="How to replicate legacy systems with machine learning?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1842" PostTypeId="2" ParentId="1838" CreationDate="2016-09-03T05:57:50.757" Score="5" Body="&lt;p&gt;AI is a rather unusual research field in that the label persists more because it represents a highly desired &lt;em&gt;goal&lt;/em&gt;, rather than (as with most other fields) the means, substrate or methodology by which that goal is achieved.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we still talk of AI as a unified, coherent field of study&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Despite recent efforts in AGI, I don't think that AI is actually a very unified or coherent field. This is not necessarily a bad thing - when attempting to mimic the most complex phenomenon known to us (i.e. human intelligence) then &lt;a href=&quot;https://en.wikipedia.org/wiki/Blind_men_and_an_elephant&quot;&gt;multiple, sometimes seemingly conflicting perspectives&lt;/a&gt; may be our best way of making progress.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-03T11:04:55.363" LastActivityDate="2016-09-03T11:04:55.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1843" PostTypeId="2" ParentId="1841" CreationDate="2016-09-03T06:16:21.710" Score="9" Body="&lt;p&gt;Let's assume from the outset that the space of inputs is too large to allow exhaustive tabulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essential issues when applying ML are that the program being modelled is likely in general to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Be discrete, i.e. operate (at least in part) on integer, boolean or categorical variables.&lt;/li&gt;&#xA;&lt;li&gt;Contain various conditional/looping constructs (&lt;code&gt;if/while/for&lt;/code&gt; etc).&lt;/li&gt;&#xA;&lt;li&gt;Have &lt;em&gt;side-effects&lt;/em&gt; that affect other parts of the program (e.g. non-local variables) or world state (e.g. writing to a file).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These pose obstacles for ML methods such as ANN. The ML approach which is most immediately compatible with these issues is &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/poli08_fieldguide.pdf&quot;&gt;Genetic Programming&lt;/a&gt; (GP).&#xA;A recent specialisation of GP that is specifically concerned with the transformation of &lt;em&gt;existing&lt;/em&gt; software systems is Genetic Improvement (GI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However neither GP/GI (nor any other current ML technique) is a 'silver bullet' here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Despite decades of research, GP still works best at synthesizing relatively small functions, certainly not entire legacy programs.&lt;/li&gt;&#xA;&lt;li&gt;Because it is only possible to train on a very small subset of a program's input space, there is little guarantee that the program will generalize to inputs it hasn't been trained on.&lt;/li&gt;&#xA;&lt;li&gt;How will the success of side effects be formally determined for training purposes?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some of these issues could be addressed to some degree if the program has a comprehensive test suite, but replacing an entire nontrivial program is not likely anytime soon. Replacing smaller parts of the program that have good unit tests is more realistic goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/264155239_Repairing_and_Optimizing_Hadoop_hashCode_Implementations&quot;&gt;Here&lt;/a&gt; is a case study showing how GI was successfully used to fix errors in the implementation of Apache Hadoop, by operating only on the program binary.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-03T15:47:54.697" LastActivityDate="2016-09-03T15:47:54.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1845" PostTypeId="2" ParentId="1700" CreationDate="2016-09-03T15:47:04.860" Score="4" Body="&lt;p&gt;Emotion in an AI is useful, but not necessary depending on your objective (in most cases, it's not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, &lt;strong&gt;emotion recognition/analysis&lt;/strong&gt; is very well advanced, and it's used in a wide range of applications very successfully, from robot teacher for autistic children (see developmental robotics) to gambling (poker) to personal agents and politics sentiment/lies analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Emotional cognition&lt;/strong&gt;, the experience of emotions for a robot, is much less developed, but there are very interesting researchs (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Affect_heuristic&quot; rel=&quot;nofollow&quot;&gt;Affect Heuristic&lt;/a&gt;, &lt;a href=&quot;http://cdn.intechopen.com/pdfs/33737/InTech-A_multidisciplinary_artificial_intelligence_model_of_an_affective_robot.pdf&quot; rel=&quot;nofollow&quot;&gt;Lovotics's Probabilistic Love Assembly&lt;/a&gt;, and others...). Indeed, I can't see why we couldn't model emotions such as &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3898540/&quot; rel=&quot;nofollow&quot;&gt;love as they are just signals that can already be cut in humans brains (see Brian D. Earp paper)&lt;/a&gt;. It's difficult, but not impossible, and actually there are several robots reproducing partial emotional cognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am of the opinion that the claim &lt;a href=&quot;https://en.wikipedia.org/wiki/Synthetic_intelligence&quot; rel=&quot;nofollow&quot;&gt;&quot;robots can just simulate but not feel&quot; is just a matter of semantics&lt;/a&gt;, not of objective capacity: for example, does a submarine swim like fish swim? However, planes fly, but not at all like birds do. In the end, does the technical mean really matters when in the end we get the same behavior? Can we really say that a robot like &lt;a href=&quot;https://en.wikipedia.org/wiki/Chappie_(film)&quot; rel=&quot;nofollow&quot;&gt;Chappie&lt;/a&gt;, if it ever gets made, does not feel anything just like a simple thermostat?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what would be the use of emotional cognition for an AI? This question is still in great debates, but I will dare offer my own insights:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Emotions in humans (and animals!) are known to affect memories. They are now well known in neuroscience as additional modalities, or meta-data if you prefer, of long term memories: they allow to modulate how the memory is stored, how it is associated/related with other memories, and how it will be retrieved.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As such, we can hypothesize that the main role of emotions is to add additional meta-information to memories to help in heuristic inference/retrieval. Indeed, our memories are huge, there are a lot of information we store over our lifetime, so emotions can maybe be used as &quot;labels&quot; to help retrieve faster the relevant memories.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Similar &quot;labels&quot; can be more easily associated together (memories of scary events together, memories of happy events together, etc.). As such, they can help survival by quickly reacting and applying known strategies (fleeing!) from scary strategies, or to take the most out of benefitting situations (happy events, eat the most you can, will help survive later on!). And actually, neuroscience studies discovered that there are specific pathways for fear-inducing sensory stimuli, so that they reach actuators faster (make you flee) than by passing through the usual whole somato-sensory circuit as every other stimuli. This kind of associative reasoning could also lead to solutions and conclusions that could not be reached otherwise.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;By feeling empathy, this could ease robots/humans interaction (eg, drones helping victims of catastrophic events).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A virtual model of an AI with emotions could be useful for neuroscience and medical research in emotional disorders as computational models to understand and/or infer the underlying parameters (this is often done for example with Alzheimer and other neurodegenerative diseases, but I'm not sure if it was ever done for emotional disorders as they are quite new in the DSM).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So yes, &quot;cold&quot; AI is already useful, but emotional AI could surely be applied to new areas that could not be explored by using cold AI alone. It will also surely help in understanding our own brain, as emotions are an integral part.&lt;/p&gt;&#xA;" OwnerUserId="1880" LastEditorUserId="1880" LastEditDate="2016-09-03T15:55:08.973" LastActivityDate="2016-09-03T15:55:08.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1846" PostTypeId="2" ParentId="1838" CreationDate="2016-09-03T20:32:23.110" Score="2" Body="&lt;p&gt;A lot of the survival power of the A.I. label comes from the popularity of science fiction, which many scientists - computer or otherwise - are big fans of, as are their consumers. Astronomers and physicists, for example, may frown on really bad sci-fi, but I see many of the well-known ones like Hawking daydreaming about things like wormholes and time travel etc. Which is fine - there's nothing wrong with a sense of wonder, as long as it doesn't dupe us into overestimating our success or finding the wrong answers to real-world problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, that's a big issue in A.I. research. We watch movies like 2001: A Space Odyssey and Terminator and then set about replicating the fictional technologies seen in them, without even having a hard definition of intelligence. A.I. is a much more melodramatic moniker than say, &quot;Autonomous Algorithmic Pattern Recognition&quot; or some similarly boring label. Because this name is applied carelessly to a wide variety of disciplines, it implies that we have already made significant progress towards replicating advanced aspects of human thought, like consciousness, reasoning, intuition, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, this vague label enables us to fool ourselves into thinking we're a lot closer to perfecting kinds of technologies we see in the movies; the backwards logic boils down to, &quot;because we've chosen to call this odd (sloppy) selection of fields 'artificial intelligence', we must be close to achieving artificial intelligence.&quot; The label survives in large part for irrational, human reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not saying that's the only reason, or that some of the other reasons don't have better legitimacy, but this is a big issue that we will have to contend with for a long time to come.&lt;/p&gt;&#xA;" OwnerUserId="1427" LastEditorUserId="181" LastEditDate="2016-09-04T15:11:31.287" LastActivityDate="2016-09-04T15:11:31.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1847" PostTypeId="1" AcceptedAnswerId="1849" CreationDate="2016-09-03T21:10:31.007" Score="6" ViewCount="438" Body="&lt;p&gt;Sometimes I understand that people doing &lt;em&gt;cognitive science&lt;/em&gt; try to avoid the term &lt;em&gt;artificial intelligence&lt;/em&gt;. The feeling I get is that there is a need to put some distance to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOFAI&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another impression that I get is that &lt;em&gt;cognitive science&lt;/em&gt; is more about trying to find out how the human intelligence or mind works. And that it would use &lt;em&gt;artificial intelligence&lt;/em&gt; to make tests or experiments, to test ideas and so forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is artificial intelligence (only) a research tool for cognitive science? What is the difference between artificial intelligence and cognitive science?&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="2444" LastEditDate="2019-05-02T16:07:33.297" LastActivityDate="2019-05-23T19:32:00.303" Title="What is the difference between artificial intelligence and cognitive science?" Tags="&lt;terminology&gt;&lt;comparison&gt;&lt;cognitive-science&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1848" PostTypeId="2" ParentId="1700" CreationDate="2016-09-03T21:39:22.097" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What purpose would be served by developing AI's that experience&#xA;  human-like emotions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any complex problem involving human emotions, where the solution to the problem requires an ability to sympathize with the emotional states of human beings, will be most efficiently served by an agent that &lt;em&gt;can&lt;/em&gt; sympathize with human emotions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Politics. Government. Policy and planning. Unless the thing has intimate knowledge of the human experience, it won't be able to provide definitive answers to all problems we encounter in our human experience.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-03T21:39:22.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1849" PostTypeId="2" ParentId="1847" CreationDate="2016-09-03T22:21:58.003" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Another impression that I get is that &lt;em&gt;cognitive science&lt;/em&gt; is more about trying to find out how the human intelligence or mind works. And that it would use &lt;em&gt;artificial intelligence&lt;/em&gt; to make tests or experiments, to test ideas and so forth.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think that's pretty much it.  I mean, clearly there is some overlap, but I feel like most people who use &quot;cognitive science&quot; are referring more to understanding human intelligence for its own sake.  Artificial Intelligence, OTOH, is more about &lt;em&gt;implementing&lt;/em&gt; &quot;intelligence&quot; on a computer, where the techniques used may or may not be influenced by research done under the rubric of cognitive science.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2019-05-02T16:08:09.067" LastActivityDate="2019-05-02T16:08:09.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1850" PostTypeId="2" ParentId="1847" CreationDate="2016-09-04T02:18:43.183" Score="1" Body="&lt;p&gt;Artificial intelligence is much more than a research tool for cognitive science. Of course there is some overlapping and researchers of both fields working together. But AI is also broadly used in economics, security (for example face recognition software), advertising, or in the development of games and of course in robotics (autonomous systems). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference is - as you already mentioned - that cognitive science deals with living things while AI tries to create an intelligence artificially (AI tries to deliver the brain, the mind or the consciousness for a hardware device that then hopefully solves various problems). &lt;/p&gt;&#xA;" OwnerUserId="1781" LastActivityDate="2016-09-04T02:18:43.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1851" PostTypeId="1" CreationDate="2016-09-04T09:45:02.357" Score="18" ViewCount="445" Body="&lt;p&gt;Just for fun, I am trying to develop a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for backpropagation I saw two techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one is used &lt;a href=&quot;http://courses.cs.washington.edu/courses/cse599/01wi/admin/Assignments/bpn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and in many other places too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it does is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It computes the error for each output neuron.&lt;/li&gt;&#xA;&lt;li&gt;It backpropagates it into the network (calculating an error for each inner neuron).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It updates the weights with the formula: &lt;span class=&quot;math-container&quot;&gt;$\Delta w_{l, m, n}=k \cdot E_{l+1, n} \cdot N_{l, m}$&lt;/span&gt;, where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\Delta w_{l, m, n}$&lt;/span&gt; is the change in weight, &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the learning speed, &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$E_{l+1, n}$&lt;/span&gt; is the error of the neuron receiving the input from the synapse, and &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$ N_{l, m}$&lt;/span&gt; is the output sent on the synapse.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It repeats for each entry of the dataset, as many times as required.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, the neural network proposed in &lt;a href=&quot;https://www.youtube.com/watch?v=bxe2T-V8XRs&amp;amp;list=PL77aoaxdgEVDrHoFOMKTjDdsa0p9iVtsR&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; (also available on GitHub) uses a different technique:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It uses an error function (the other method does have an error function, but it does not use it for training).&lt;/li&gt;&#xA;&lt;li&gt;It has another function which can compute the final error starting from the weights.&lt;/li&gt;&#xA;&lt;li&gt;It minimizes that function (through gradient descent).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, which method should be used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the first one is the most used one (because I saw different examples using it), but does it work as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I don't know:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Isn't it more subject to local minimums (since it doesn't use quadratic functions)?&lt;/li&gt;&#xA;&lt;li&gt;Since the variation of each weight is influenced by the output value of its output neuron, don't entries of the dataset which just happen to produce higher values in the neurons (not just the output ones) influence the weights more than other entries?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, I do prefer the first technique, because I find it simpler to implement and easier to think about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though, if it does have the problems I mentioned (which I hope it doesn't), is there any actual reason to use it over the second method?&lt;/p&gt;&#xA;" OwnerUserId="2143" LastEditorUserId="2444" LastEditDate="2020-05-23T12:53:33.067" LastActivityDate="2020-05-23T13:00:58.127" Title="Are these two versions of back-propagation equivalent?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1852" PostTypeId="2" ParentId="1700" CreationDate="2016-09-04T12:20:31.833" Score="2" Body="&lt;p&gt;I think that depends on the application of the AI. Obviously if I develop an AI that's purpose is plainly to do specific task under the supervision of humans, there is no need for emotions. But if the AI's purpose is to do task autonomously, then emotions or empathy can be useful. For example, think about an AI that is working in the medical domain. Here it may be advantageous for an AI to have some kind of empathy, just to make the patients more comfortable. Or as another example, think about a robot that serves as a nanny. Again it is obvious that emotions and empathy would be advantageous and desirable. Even for an assisting AI program (catchword smart home) emotions and empathy can be desirable to make people more comfortable. It would be much nicer to be welcomed by an empathic home assistant than by one with no empathic responses at all, wouldn't it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if the AI is just working on an assembly line, there is obviously no need for emotions and empathy (on the contrary in that case it may be unprofitable). &lt;/p&gt;&#xA;" OwnerUserId="1781" LastActivityDate="2016-09-04T12:20:31.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1853" PostTypeId="1" AcceptedAnswerId="1855" CreationDate="2016-09-04T14:06:54.923" Score="9" ViewCount="354" Body="&lt;p&gt;The English Language is not well-suited to talking about artificial intelligence, which makes it difficult for humans to communicate to each other about what an AI is actually &quot;doing&quot;. Thus, it may make more sense to use &quot;human-like&quot; terms to describe the actions of machinery, even when the internal properties of the machinery do not resemble the internal properties of humanity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anthropomorphic language had been used a lot in technology (see the Hacker's Dictionary definition of &lt;a href=&quot;https://www.landley.net/history/mirror/jargon.html#Anthropomorphization&quot;&gt;anthropomorphization&lt;/a&gt;, which attempts to justify computer programmers' use of anthromporhic terms when describing technology), but as AI continues to advance, it may be useful to consider the tradeoffs of using anthropomorphic language in communicating to both technical audiences and non-technical audiences. How can we get a good handle on AI if we can't even describe what we're doing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I want to develop an algorithm that display a list of related articles. There are two ways by which I can explain how the algorithm works to a layman:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;em&gt;Very Anthropomorphic&lt;/em&gt; - The algorithm reads all the articles on a website, and display the articles that are very similar to the article you are looking at.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Very Technical&lt;/em&gt; - The algorithm converts each article into a &quot;bag-of-words&quot;, and then compare the &quot;bag-of-words&quot; of each article to determine what articles share the most common words. The articles that share the most words in the bags are the ones that are displayed to the user.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Obviously, #2 may be more &quot;technically correct&quot; than #1. By detailing the implementation of the algorithm, it makes it easier for someone to understand how to &lt;em&gt;fix&lt;/em&gt; the algorithm if it produces an output that we disagree with heavily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But #1 is more readable, elegant, and easier to understand. It provides a general sense of &lt;em&gt;what&lt;/em&gt; the algorithm is doing, instead of &lt;em&gt;how&lt;/em&gt; the algorithm is doing it. By abstracting away the implementation details of how a computer &quot;reads&quot; the article, we can then focus on using the algorithm in real-world scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I, therefore, prefer to use the anthropomorphic language as emphasized by Statement #1? If not, why not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: If the answer depends on the audience that I am speaking to (a non-technical audience might prefer #1, while a technical audience may prefer #2), then let me know that as well.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2016-09-05T11:41:44.290" Title="Should I use anthropomorphic language when discussing AI?" Tags="&lt;philosophy&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1854" PostTypeId="2" ParentId="1700" CreationDate="2016-09-04T14:51:44.747" Score="1" Body="&lt;h2&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Theory_of_mind&quot; rel=&quot;nofollow&quot;&gt;Theory of mind&lt;/a&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If we want a strong general AI to function well in an environment that consists of humans, then it would be very useful for it to have a good &lt;a href=&quot;https://en.wikipedia.org/wiki/Theory_of_mind&quot; rel=&quot;nofollow&quot;&gt;theory of mind&lt;/a&gt; that matches how humans actually behave. That theory of mind needs to include human-like emotions, or it will not match the reality of this environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For us, an often used shortcut is explicitly thinking &quot;what would I have done in this situation?&quot; &quot;what event could have motivated &lt;em&gt;me&lt;/em&gt; to do what they just did?&quot; &quot;how would I feel if this had happened to &lt;em&gt;me&lt;/em&gt;?&quot;. We'd want an AI to be capable of such reasoning, it is practical and useful, it allows better predictions of future and more effective actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even while it would be better for it the AI to not be actually driven by those exact emotions (perhaps something in that direction would be useful but quite likely not &lt;em&gt;exactly&lt;/em&gt; the same), all it changes that instead of thinking &quot;what &lt;em&gt;I&lt;/em&gt; would feel&quot; it should be able to hypothesize what a generic human would feel. That requires implementing a subsystem that is capable of accurately modeling human emotions.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-09-04T14:51:44.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1855" PostTypeId="2" ParentId="1853" CreationDate="2016-09-04T17:37:05.977" Score="8" Body="&lt;p&gt;If clarity is your goal, you should attempt to avoid anthropomorphic language - doing so runs a danger of even misleading &lt;em&gt;yourself&lt;/em&gt; about the capabilities of the program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a pernicious trap in AI research, with numerous cases where even experienced researchers have ascribed a greater degree of understanding to a program than is actually merited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Douglas Hofstadter describes the issue at some length in a chapter entitled &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA_effect&quot;&gt;&quot;The Ineradicable Eliza Effect and Its Dangers&quot;&lt;/a&gt; and there is also a famous paper by Drew McDermot, entitled &lt;a href=&quot;https://www.inf.ed.ac.uk/teaching/courses/irm/mcdermott.pdf&quot;&gt;&quot;Artifical Intelligence meets natural stupidity&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, in general one should make particular effort to avoid anthropomorphism in AI. However, when speaking to a non-technical audience, 'soundbite' descriptions are (as in any complex discipline) acceptable &lt;em&gt;provided you let the audience know that they are getting the simplified version&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-04T18:36:24.090" LastActivityDate="2016-09-04T18:36:24.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1856" PostTypeId="2" ParentId="1853" CreationDate="2016-09-04T23:50:19.347" Score="2" Body="&lt;p&gt;I think the correct answer is the easy but unhelpful, &quot;It depends.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even when I'm talking to other technical people, I often use anthropomorphic language and metaphors. Especially at the start of the conversation. &quot;The computer has to figure out ..&quot; &quot;How can we prevent the computer from getting confused about ...&quot; etc. Sure, we could state that in a more technically correct way. &quot;We need to modify the algorithm to reduce the number and variety of instances of inadequate data that result in inaccurate setting of ...&quot; or some such. But among technical people, we know what we mean, and it's just easier to use metaphorical language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When trying to solve technical computer problems, I often start with a vague, anthropomorphic concept. &quot;We should make a list of all the words in the text, and assign each word a weight based on how frequently it occurs. Oh, but we should ignore short, common words like 'the' and 'it'. Then let's pick some number of words, maybe ten or so, that have the greatest weight ...&quot; All that is a long way from how the computer actually manipulates data. But it's often a lot easier to think about it in &quot;human&quot; terms first, and then figure out how to make the computer do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When talking to a non-technical audience, I think the issue is, Anthropomorphic language makes it easier to understand, but also often gives the impression that the computer is much more human-like than it really is. You only need to watch science fiction movies to see that apparently a lot of people think that a computer or a robot thinks just like a person except that it's very precise and has no emotions.&lt;/p&gt;&#xA;" OwnerUserId="2171" LastActivityDate="2016-09-04T23:50:19.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1857" PostTypeId="2" ParentId="1853" CreationDate="2016-09-05T11:35:17.787" Score="3" Body="&lt;p&gt;The problem you're referencing is not just an AI problem but a problem for highly technical fields in general. When in doubt, I would always recommend using &lt;a href=&quot;http://plainlanguagenetwork.org/plain-language/what-is-plain-language/&quot; rel=&quot;nofollow&quot;&gt;plain language&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is another reason the AI community will often eschew anthropomorphic connotations for AI. Some AI luminaries often like warning us that an artificial &lt;em&gt;general&lt;/em&gt; intelligence may behave in alien ways that defy our human expectations, potentially leading to a robot apocalypse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea about evil alien-like AGIs, however, derives from a widespread misunderstanding in the AI community that conflates two different notions of generality: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Turing machine generality, and &lt;/li&gt;&#xA;&lt;li&gt;human domain generality &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What regular people mean when they say generality is the later. Even the &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow&quot;&gt;official definition&lt;/a&gt; of AGI hinges off of that human-contingent context:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...perform any intellectual task that a human being can.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But by that definition, generalizing behavior does not make it more alien. To generalize is to anthropomorphize. As Nietzche said, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Where you see ideal things, I see— human, alas! All too human things.”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-09-05T11:41:44.290" LastActivityDate="2016-09-05T11:41:44.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1859" PostTypeId="1" AcceptedAnswerId="6572" CreationDate="2016-09-05T13:46:57.910" Score="11" ViewCount="529" Body="&lt;p&gt;Roger Schank did some interesting work on language processing with Conceptual Dependency (CD) in the 1970s. He then moved somewhat out of the field, being in Education these days. There were some useful applications in natural language generation (BABEL), story generation (TAILSPIN) and other areas, often involving planning and episodes rather than individual sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anybody else continued to use CD or variants thereof? I am not aware of any other projects that do, apart from Hovy's PAULINE which uses CD as representation for the story to generate.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="145" LastEditDate="2016-09-05T15:07:24.223" LastActivityDate="2018-05-30T10:46:56.560" Title="Is anybody still using Conceptual Dependency Theory?" Tags="&lt;natural-language-processing&gt;&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1860" PostTypeId="1" CreationDate="2016-09-05T18:36:11.963" Score="1" ViewCount="174" Body="&lt;p&gt;I have been wanting to get started learning about artificial intelligence but I know almost nothing about coding or anything. So my question is, what would be the best way to get started in learning about artificial intelligence, as in should I learn some kind of coding language or is there some kind of other concept you need to know before getting started. So I'm just kind of looking for the best way to get started if you literally know nothing.&lt;/p&gt;&#xA;" OwnerUserId="2204" LastActivityDate="2016-09-05T19:58:56.570" Title="How to start learning about artificial intelligence?" Tags="&lt;research&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2016-09-06T00:04:56.550" ContentLicense="CC BY-SA 3.0" />
  <row Id="1861" PostTypeId="2" ParentId="1860" CreationDate="2016-09-05T19:58:56.570" Score="3" Body="&lt;p&gt;Read:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Artificial Intelligence - A modern approach' by Russell and Norvig.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Fluid Concepts and Creative Analogies' by Douglas Hofstadter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Machine Learning and Pattern Recognition' by Bishop&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'The Emotion Machine' by Marvin Minsky&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-09-05T19:58:56.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1862" PostTypeId="2" ParentId="1618" CreationDate="2016-09-06T06:25:05.383" Score="4" Body="&lt;p&gt;Using evolutionary algorithms to evolve neural networks is called &lt;a href=&quot;http://www.scholarpedia.org/article/Neuroevolution&quot; rel=&quot;nofollow&quot;&gt;neuroevolution&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some neuroevolution algorithms optimize only the &lt;em&gt;weights&lt;/em&gt; of a neural network with fixed topology. That sounds not like what you want. Other neuroevolution algorithms optimize both the &lt;em&gt;weights&lt;/em&gt; and &lt;em&gt;topology&lt;/em&gt; of a neural net. These kinds of algorithms seem more appropriate for your aims, and are sometimes called TWEANNs (Topology and Weight Evolving Neural Networks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One popular algorithm is called &lt;a href=&quot;https://www.cs.ucf.edu/~kstanley/neat.html&quot; rel=&quot;nofollow&quot;&gt;NEAT&lt;/a&gt;, and is probably a good place to start, if only because there are a multitude of implementations, one of which hopefully is written in your favorite language. That would at least give you a baseline to work with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEAT encodes a neural network genome directly as a graph structure. Mutations can operate on the structure of the network by adding new links (by connecting two nodes not previously connected) or new nodes (by splitting an existing connection), or can operate only on changing the weights associated with edges in the graphs (called mutating the weights). &#xA;To give you an idea of the order of magnitude of the sizes of ANNs this particular algorithm works with, it would likely struggle with more than 100 or 200 nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more scalable TWEANNs, but they're more complex and make assumptions about the kinds of structures they generate that may not always be productive in practice. For example, another way to encode the structure of a neural network, is as the product of a seed pattern that is repeatedly expanded by a grammar (e.g. an L-system). You can much more easily explore larger structures, but because they're generated by a grammar they'll have a characteristic self-repeating sort of feel. HyperNEAT is a popular extension of NEAT that makes a different sort of assumption (that patterns of weights can be easily expressed as a function of geometry), and can scale to ANNs with millions of connections when that assumption well-fits a particular domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few survey papers linked in the top link if you want to observe a greater variety of techniques.&lt;/p&gt;&#xA;" OwnerUserId="2219" LastActivityDate="2016-09-06T06:25:05.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1863" PostTypeId="2" ParentId="1700" CreationDate="2016-09-06T11:18:01.147" Score="0" Body="&lt;p&gt;Human emotions are intricately connected to human values and to our ability to cooperate and form societies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to give an easy example:&#xA;You meet a stranger who needs help, you feel &lt;strong&gt;empathy&lt;/strong&gt;. &#xA;This compels you to help him at a cost to yourself. &#xA;Let's assume the next time you meet him, you need something. Let's also assume he doesn't help you, you'll feel &lt;strong&gt;anger&lt;/strong&gt;. &#xA;This emotion compels you to punish him, at further cost for yourself.&#xA;He on the other hand, if he doesn't help you, feels &lt;strong&gt;shame&lt;/strong&gt;.&#xA;This compels him to actually help you, avoiding your anger and making your initial investment worthwhile. You both benefit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So these three emotions keep up a circle of reciprocal help. Empathy to get started, anger to punish defectors and shame to avoid the anger. This also leads to a concept of justice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that value alignment is one of the big problems in AGI, human-like emotions strike me as good approach towards AIs that actually share our values and integrate themselves seamlessly into our society. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-06T11:18:01.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1864" PostTypeId="2" ParentId="1838" CreationDate="2016-09-06T15:08:23.843" Score="0" Body="&lt;p&gt;Because, ultimately, AI &lt;em&gt;is&lt;/em&gt; a cohesive &quot;thing&quot;.  It's an effort to make computers do things that currently only humans can do well.  Sure there are many, many approaches and techniques, but there's always been a clear overall goal (although the goal-posts keep getting moved further out, which is a different issue).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as there are things humans can do well that computers can't, somebody will be trying to figure out how to close that gap. And those efforts are &quot;Artificial Intelligence&quot;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-06T15:08:23.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1865" PostTypeId="2" ParentId="1825" CreationDate="2016-09-06T15:14:43.980" Score="2" Body="&lt;p&gt;One of them is certainly &lt;a href=&quot;http://www.alicebot.org/bios/richardwallace.html&quot; rel=&quot;nofollow&quot;&gt;Doctor Richard Wallace&lt;/a&gt;.  Doctor Wallace was the original author of the &lt;a href=&quot;https://en.wikipedia.org/wiki/AIML&quot; rel=&quot;nofollow&quot;&gt;Artificial Intelligence Markup Language&lt;/a&gt; spec and is Chief Science Officer at PandoraBots. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-06T15:14:43.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1866" PostTypeId="2" ParentId="1397" CreationDate="2016-09-06T15:23:41.650" Score="2" Body="&lt;p&gt;I believe this is exactly the kind of test where Doug Lenat's &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cyc&quot; rel=&quot;nofollow&quot;&gt;cyc&lt;/a&gt;&lt;/strong&gt; would do very well at ? But I can't answer the question : how much of that corpus could it answer correctly ? Probably quite a lot ! (and how many humans could pass that test ? probably not all of them, but many can...)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[but is cyc considered an AI? probably not... so I may be out of topic. But imo it's database should be incorporated to any AI that reaches some kind of &quot;intelligence&quot;...]&lt;/p&gt;&#xA;" OwnerUserId="2233" LastActivityDate="2016-09-06T15:23:41.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1868" PostTypeId="2" ParentId="1686" CreationDate="2016-09-06T17:31:44.187" Score="2" Body="&lt;p&gt;Take a look at &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2390143&quot; rel=&quot;nofollow&quot;&gt;this 2012 paper&lt;/a&gt; by three people at Bright. (Sadly, it's paywalled and I couldn't easily find a ungated copy, so I don't have a summary for you.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Bright has built an automated system for ranking job candidates against job descriptions. The candidate's resume and social media profiles are interwoven to build an augmented user profile. Similarly, the job description is augmented by external databases and user-generated content to build an enhanced job profile. These augmented user and job profiles are then analyzed in order to develop numerical overlap features each with strong discriminating power, and in sum with maximal coverage. The resulting feature scores are then combined into a single Bright Score using a custom algorithm, where the feature weights are derived from a nation-wide and controlled study in which we collected a large sample of human judgments on real resume-job pairings. We demonstrate that the addition of social media profile data and external data improves the classification accuracy dramatically in terms of identifying the most qualified candidates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-06T17:31:44.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1870" PostTypeId="1" CreationDate="2016-09-06T23:13:08.677" Score="8" ViewCount="173" Body="&lt;p&gt;I have been studying local search algorithms such as greedy hill-climbing, stochastic hill-climbing, simulated annealing, etc. I have noticed that most of these methods take up very little memory as compared to systematic search techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there local search algorithms that make use of memory to give significantly better answers than those algorithms that use little memory (such as crossing local maxima)? Also, is there a way to combine local search and systematic search algorithms to get the best of both worlds?&lt;/p&gt;&#xA;" OwnerUserId="2244" LastEditorUserId="2444" LastEditDate="2020-06-16T20:28:31.567" LastActivityDate="2020-06-16T20:28:31.567" Title="Are there local search algorithms that make use of memory to give better solutions?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;simulated-annealing&gt;&lt;local-search&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1871" PostTypeId="2" ParentId="1838" CreationDate="2016-09-06T23:21:40.240" Score="1" Body="&lt;p&gt;I don't believe that AI as a coherent field has a lesser legitimacy than, say, Engineering.  Ignoring for the moment that we're a day or two behind on AI, they're very much alike: &#xA;Both fields contain a wide variety of sub-fields which stretch across multiple disciplines (although admittedly more pronounced in AI) , in both fields it is mandatory to specialize and in both of them an expert in one sub-field will be more or less useless in a different one (the expert on bridge construction will probably not be very versed in the thermodynamics of AC systems and vice versa). This pattern can be seen in many of today's disciplines - in fact, I don't know if there still is a reputable field, in which a single person can be a universal expert. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned that the only unifying thing about AI was it's dealing with machines in some fashion - but such a simplifying statement can be made about almost any field. To return to my previous example: the only unifying thing about the various Engineering activities is that they're all somehow involved in the construction of something (be it a flashlight or an aircraft carrier).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is a young field and therefore its branches have not yet been established in the sophisticated way that the branches of other fields have, but I would assume that it is only a matter of time until the various differentiations and the corresponding degrees, courses etc. develop.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is also growing up in a time where vast knowledge in its related/parental fields already exists and further knowledge is produced at dizzying speeds - and that is as much a blessing as it is a curse. When Engineering was 'created' a few millennia ago (please excuse my ridiculously inaccurate science history lessons) there wasn't much going on in the world of science and so the field grew slowly, with plenty of time to get organized and structured. That is a luxury which AI did/does not have. It emerged in an age of technical wonders, surrounded by scientific breakthroughs on at least a monthly basis and the rise of interdisciplinary science (which by itself complicated things quite a bit). So in addition to organizing itself, the field also has to continuously integrate the large number of advancements made and somehow stand its ground against the outlandish expectations generated by other science's breakthroughs over the past decades and the media (as already explained by SQLServerSteve). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long story short: it's similar in it's complexity and diversity to other fields and therefore has no reason to collapse - on the contrary, its failure to do so over the past ~50 rather complicated years indicates, that it will further solidify and organize itself in the future. &lt;/p&gt;&#xA;" OwnerUserId="2166" LastActivityDate="2016-09-06T23:21:40.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1872" PostTypeId="2" ParentId="1870" CreationDate="2016-09-06T23:41:01.203" Score="3" Body="&lt;p&gt;You could parallelize the search by dividing the global space in distinct regions/subsets. Then apply in each region a local search. This way you can search the global space systematically, more exhaustively and perhaps in different ways (e.g by applying a different local search method to each region). Finally you can compare the results and choose the best one.&lt;/p&gt;&#xA;" OwnerUserId="1781" LastActivityDate="2016-09-06T23:41:01.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1873" PostTypeId="2" ParentId="156" CreationDate="2016-09-07T01:30:27.690" Score="0" Body="&lt;p&gt;We actually do have many things along that line, motion capture for 3-D movies instance comes to mind almost immediately. The problem if I think about it is less of a situation in observing another actor, computers are relativity good at doing that already with the amount of image recognition software we have, rather it's a problem of understanding if an action yielded a good outcome as a net which is something that computers cannot do as it's not a single node network problem. For example, we've already programmed a computer to understand human language (Watson, arguably), but even Watson didn't understand the concept that saying &quot;f***&quot; is bad. (Look that up, it's a funny side story.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the point is, learning algorithms are not true learning in a sense as a computer currently has no sense of &quot;a good outcome&quot;, hence at this stage observation learning is very much limited in a sense to &quot;monkey see, monkey do&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the closest thing I have ever read about with this was firefighting search and rescue bots that were on a network and would broadcast to each other when one of them had been destroyed as the bots would know the area was something that they had to avoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, I think this is the problem with observational learning. A person can observe that punching someone usually will get you hit back, a computer will observe and parrot the action, good or bad.&lt;/p&gt;&#xA;" OwnerUserId="2246" LastEditorUserId="75" LastEditDate="2016-10-14T13:08:35.790" LastActivityDate="2016-10-14T13:08:35.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1874" PostTypeId="2" ParentId="1870" CreationDate="2016-09-07T03:04:19.570" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tabu_search&quot; rel=&quot;nofollow&quot;&gt;Tabu search&lt;/a&gt; uses memory to rule out parts of the neighborhood for local search, allowing the trajectory to typically pass through local optima instead of getting stuck in them.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-07T03:04:19.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1876" PostTypeId="1" AcceptedAnswerId="1880" CreationDate="2016-09-07T14:00:02.683" Score="1" ViewCount="1658" Body="&lt;p&gt;&lt;em&gt;I know that every program has some positive and negative points, and I know maybe .net programming languages are not the best for AI programming.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But I prefer .net programming languages because of my experiences and would like to know for an AI program which one is better, C or C++ or C# and or VB ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Which one of this languages is faster and more stable when running different queries and for self learning ?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make a summary, i think C++ is the best for AI programming in .net and also C# can be used in some projects, Python as recommended by others is not an option on my view !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because : &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It's not a complex language itself and for every single move you need to find a library and import it to your project (most of the library are out of date and or not working with new released Python versions) and that's why people say it is an easy language to learn and use ! (If you start to create library yourself, this language could be the hardest language in the world !)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You do not create a program yourself by using those library for every single option on your project (it's just like a Lego game)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm not so sure in this, but i think it's a cheap programming language because i couldn't find any good program created by this language !&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2260" LastEditorUserId="2260" LastEditDate="2016-09-13T08:05:06.463" LastActivityDate="2016-09-13T08:05:06.463" Title="What is the best .net programming language for artificial intelligence programming?" Tags="&lt;intelligent-agent&gt;" AnswerCount="2" CommentCount="3" ClosedDate="2016-09-08T06:13:48.690" ContentLicense="CC BY-SA 3.0" />
  <row Id="1877" PostTypeId="1" AcceptedAnswerId="1883" CreationDate="2016-09-07T14:16:36.477" Score="10" ViewCount="3422" Body="&lt;p&gt;Why is search important in AI? What kinds of search algorithms are used in AI? How do they improve the result of an AI?&lt;/p&gt;&#xA;" OwnerUserId="1270" LastEditorUserId="2444" LastEditDate="2019-04-16T09:46:56.027" LastActivityDate="2019-04-16T09:52:57.157" Title="Why is search important in AI?" Tags="&lt;search&gt;" AnswerCount="8" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="1878" PostTypeId="2" ParentId="1877" CreationDate="2016-09-07T14:27:13.823" Score="1" Body="&lt;p&gt;Search is important for at least two reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, searching is one of the early and major consumers of advanced machine learning, as finding the correct result for a search query boils down to predicting the click-through rate for query-result combinations. More relevant results means more clicks, more traffic, and more revenue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, many planning and optimization problems can be recast as search problems. An AI deciding on a plan to route packages through a network is searching the space of possible plans for a good one.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="2444" LastEditDate="2019-04-16T09:52:57.157" LastActivityDate="2019-04-16T09:52:57.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1879" PostTypeId="2" ParentId="70" CreationDate="2016-09-07T14:56:38.193" Score="4" Body="&lt;p&gt;A convolutional neural network can be used wherever patterns are locally correlated and translatable (as in shiftable). This is the case because CNNs contain filters that look for certain local patterns everywhere in the input. &#xA;You'll find local and translatable patterns in pictures, text, time series, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't make as much sense to use CNNs if your data is more like a bag of features with an irrelevant order. In that case, you might have trouble detecting patterns that contain features which happen to be farther apart in your input vector. You will not find local and translatable patterns in your data if you can reorder the data points of the input vectors without losing information. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="14723" LastEditDate="2018-04-12T02:45:49.627" LastActivityDate="2018-04-12T02:45:49.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1880" PostTypeId="2" ParentId="1876" CreationDate="2016-09-07T17:21:15.123" Score="1" Body="&lt;p&gt;If you're talking about pure speed, C will get you there if you really know C and operating systems, etc. C++ is nicer in terms of user friendliness, and won't be much slower. I don't know much about VB but I don't see many benefits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please clarify at least generally what the AI program is about. It is extremely difficult to answer the generalized question &quot;What programming language is best for AI&quot;? if you know what I mean. (  I vote python :^]  )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To add: Any programming language of those listed is &quot;stable&quot; if you write things correctly, but C++ and its great IDE's will help you to that point much more nicely than C will. C, to fully utilize it's potential, requires much fiddling with delicate and precise systems. It'll go that little bit faster, as the cost of being less stable in the practical, &quot; uh oh, now I need to troubleshoot this&quot; sense.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2016-09-07T22:07:28.870" LastActivityDate="2016-09-07T22:07:28.870" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1881" PostTypeId="2" ParentId="1877" CreationDate="2016-09-07T17:50:26.043" Score="1" Body="&lt;p&gt;In regards to the question you mention (in the comments of the OP), these searches are related to optimization. I'm not sure of your background, so let me describe it from scratch, briefly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember the derivative? The base idea is to talk about how the function changes in regards to changes in input. So now, we're out of high school and we're building neural nets. We've done the basic coding, and want to look at how our model is working. Back from our statistics class, we remember we use a certain measure of error (e.g. least squares) to determine the efficacy of the models from that class, so we decide to use that here. We get this error, and it's a bit too big for our liking, so we decide to fiddle with our model and adjust the weights to get that error down. But how? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where the 'search' comes into play. It's really a search for the best weights to put on the edges of our net to optimize it. We use the derivative (in some fancy ways, using the 'stochasitc' (think random sampling) and other ways the question mentions) to search for which way is 'down' in the high dimensional space of our weights. In other words, what we are searching for is minima or maxima to optimize our neural net, and we 'search' for it by doing a derivative which tells us which way to go, moving a bit in that direction, then doing that again and again iteratively to find (hopefully) the best weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This video here goes into all the detail you'd want, and I recommend the entire series as a robust but understandable intro to neural nets: &lt;a href=&quot;https://www.youtube.com/watch?v=GlcnxUlrtek&quot; rel=&quot;nofollow&quot;&gt;Demystifying Neural Networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go and look up 'gradient descent' to get any related material. (Note, the gradient here is equivalent to multidimensional derivative direction to go in, and descent is just searching for the minima)&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2016-09-07T18:05:20.690" LastActivityDate="2016-09-07T18:05:20.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1882" PostTypeId="2" ParentId="1877" CreationDate="2016-09-07T18:00:25.137" Score="3" Body="&lt;p&gt;Search has always been a crucial element of AI in multiple ways.  First, what many people refer to as &quot;search&quot; is a reflection of how what we call &quot;intelligence&quot; frequently involves searching something:  a physical realm, a &quot;state space&quot; of possible solutions, a &quot;knowledge space&quot; where ideas/facts/concepts/etc. are related as a graph structure, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look up some old papers on computer chess, and you'll see that a lot of that involves searching a &quot;state space&quot;.  As such, search algorithms that are efficient (in terms of time complexity and/or space complexity) have always been important to making advances there.  And while computer chess is just one example, the principle generalizes to many other kinds of problem solving and goal seeking activities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's &lt;a href=&quot;http://www-g.eng.cam.ac.uk/mmg/teaching/artificialintelligence/nonflash/problemframenf.htm&quot; rel=&quot;nofollow&quot;&gt;a reference&lt;/a&gt; that explains more about some of these ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note too that &quot;search&quot; is closely related to the idea of &quot;heuristics&quot; in an important way.  Many search problems in the real world are far too complex to solve by exhaustive brute-force search, so humans (and AI's) resort to heuristics to narrow the state space being searched.  Using heuristics can yield search algorithms that allow for reasonable solutions in a realistic time-frame, where no simple, deterministic algorithm exists to do likewise.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some more background you might want to read up on &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow&quot;&gt;A* search&lt;/a&gt;, which is a widely used algorithm with many applications - and not just in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other major regard in which something you could call &quot;search&quot; applies in AI is through the use of algorithms which are also often referred to as &quot;optimisation&quot; techniques.  This would be things like Hill Climbing, Gradient Descent, Simulated Annealing and perhaps even Genetic Algorithms.  These are used to maximize or minimize the values of some function  and one of the canonical uses in AI is for training neural networks using back-propagation, where you're trying to minimize the delta between the &quot;correct&quot; answer (from the training data) and the generated answer, so you can learn the correct weights within the network.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-09-08T14:19:26.630" LastActivityDate="2016-09-08T14:19:26.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1883" PostTypeId="2" ParentId="1877" CreationDate="2016-09-07T18:42:23.773" Score="7" Body="&lt;p&gt;&lt;em&gt;State space search&lt;/em&gt; is a general and ubiquitous AI activity that includes numerical optimization (e.g. via gradient descent in a real-valued search space) as a special case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;State space search is an abstraction which can be customized for a particular problem via three ingredients:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Some representation for candidate solutions to the problem (e.g. permutation of cities to represent a Travelling Salesman Problem (TSP) tour, vector of real values for numeric problems).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A solution quality measure: i.e. some means of deciding which of two solutions is the better. This is typically achieved (for single-objective problems) by having via some integer or real-valued function of a solution (e.g. total distance travelled for a TSP tour). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some means of moving around in the space of possible solutions, in a heuristically-informed manner. Derivatives can be used if available, or else (e.g. for black-box problems or discrete solution representations) the kind of mutation or crossover methods favoured by genetic algorithms/evolutionary computation can be employed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first couple of chapters of the freely available &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Essentials of Metaheuristics&quot;&lt;/a&gt; give an excellent overview and  Michalewicz and Fogel's &lt;a href=&quot;http://www.springer.com/us/book/9783540224945&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;How to Solve It - Modern Heuristics&quot;&lt;/a&gt; explains in more detail how numerical optimization can be considered in terms of state-space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How shall the &quot;search through possible plans&quot; occur? The idea is to choose all 3 of the above for the planning problem and then apply some metaheuristic (such as Simulated Annealing, Tabu Search, Genetic Algorithms etc). Clearly, for nontrivial problems, only a small fraction of the space of &quot;all possible plans&quot; is actually explored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CAVEAT: Actually &lt;em&gt;planning&lt;/em&gt; (in contrast to the vast majority of other problems amenable to state-space search such as scheduling, packing, routing etc) is a bit of a special case, in that it is sometime possible to solve planning problems simply by using A* search, rather than searching with a stochastic metaheuristic.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-04-16T09:50:14.490" LastActivityDate="2019-04-16T09:50:14.490" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1884" PostTypeId="2" ParentId="1877" CreationDate="2016-09-07T18:51:30.597" Score="0" Body="&lt;p&gt;The aim of an AI is to fulfill one or the other task, say solve the task adequately. But there are results that are no solutions at all and there are results which are satisfying the task and thus are accepted as solutions. Since there are generally more results that are no solutions, the set of all possible solutions is just a subset of all results. But this means that the task involves the search for a suitable set of solutions. &lt;/p&gt;&#xA;" OwnerUserId="1781" LastActivityDate="2016-09-07T18:51:30.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1885" PostTypeId="1" AcceptedAnswerId="1889" CreationDate="2016-09-07T20:13:49.120" Score="6" ViewCount="323" Body="&lt;p&gt;Considering the answers of &lt;a href=&quot;https://ai.stackexchange.com/questions/1314/how-powerful-a-computer-is-required-to-simulate-the-human-brain&quot;&gt;this&lt;/a&gt; question, emulating a human brain with the current computing capacity is currently impossible, but we aren't very far from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, 1 or 2 decades ago, similar calculations had similar results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The clock frequency of the modern CPUs seem to be stopped, currently the miniaturization (-&gt; mobile use), the RAM/cache improvement and the multi-core paralellization are the main lines of the development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, but what is the case with the analogous chips? In case of a NN, it is not a very big problem, if it is not very accurate, the NN would adapt to the minor manufacturing differences in its learning phase. And a single analogous wire can substitute a complex integer multiplication-division unit, while the whole surface of the analogous printed circuit could work parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://engineering.stackexchange.com/questions/3993/do-analog-fpgas-exist&quot;&gt;this&lt;/a&gt; post, &quot;software rewirable&quot; analogous circuits, essentially &quot;analogous FPGAs&quot; already exist. Although the capacity of the FPGAs is highly below the capacity of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Application-specific_integrated_circuit&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASIC&lt;/a&gt;s with the same size, maybe analogous chips for neural networks could also exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect, if it is correct, maybe even the real human brain model wouldn't be too far. It would still require a massively parallel system of costly analogous NN chips, but it seems to me not impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could this idea work? Maybe there is even active research/development into this direction?&lt;/p&gt;&#xA;" OwnerUserId="2255" LastEditorUserId="10135" LastEditDate="2018-10-19T08:20:08.983" LastActivityDate="2019-04-09T16:05:32.497" Title="Emulating human brain - with analogous NN chips" Tags="&lt;neural-networks&gt;&lt;applications&gt;" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1886" PostTypeId="2" ParentId="1876" CreationDate="2016-09-07T20:44:54.933" Score="1" Body="&lt;p&gt;In the Visual Studio, there is no real difference between C and C++. It is compiled with the same compiler binary, although with different flags.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, the easily programmable complex data structures are generally more important, as the linear speed. The AI is an exception. AI has mostly not so complex data structures, and also the linear speed improvement is very important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It closes out the garbage collected languages, i.e. any managed code, and, in my opinion, the best solution would be if you would use &lt;em&gt;not&lt;/em&gt; a .net-based language, but a directly to asm compilable one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, knowing that you are asking explicitly for a .net one, I would suggest one, which can be easily ported later to machine code. It closes out C# and VB, but it doesn't close out C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ has also the needed complex data structures, while it still has the near-asm speed (and memory need).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea would be to start with unmanaged C++ in .net, but use simply, native .exe-s to your final programs. It would make also possible to make your program portable, because C++ is for everywhere, while .net only for windows. I.e. your program will be later able to run in non-windows server (or cluster) environment, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be also important to prefer the deeply parallel or easily parallelizable algorithms. Ideally it should be made adaptable to slow communication channels (also for the parallel cluster run).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your program will probably have some user interface, persistent database and similar things, these aren't speed and memory critical things, thus these you can implement in anything as you wish. The result will be a two-process solution, where a speed-optimized, C++ calculating daemon is controlled by essentially a GUI (or db.. or script.. or anything) interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably there are already C++ frameworks for this task, so you don't need to reinvent the wheel.&lt;/p&gt;&#xA;" OwnerUserId="2255" LastActivityDate="2016-09-07T20:44:54.933" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1888" PostTypeId="2" ParentId="1824" CreationDate="2016-09-07T21:10:51.137" Score="1" Body="&lt;p&gt;AI is already used as weapon - think on the drones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect, a &quot;robots take over the world&quot; scenario has the highest probability, if it has an intermediate step. This intermediate step could be &quot;humans take over the world with robots&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can go somewhere into a false direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect, it is not surely so far as it seems. Consider the US has currently 8000 drones. What if it would have 8million? A small group capable to control them could take over the world. Or the small groups controlling different parts of the fleet, could fight against eachother. They shouldn't be all in the US - at the time the US will have this fleet, other countries will develop also theirs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw, a world takeover seem to me unreal - the military leaders can maybe switch the human pilots to drones, it is not their job. But the &quot;high level control&quot;, i.e. to determine, what to do, who are the targets, these decisions they won't ever give out from their hands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next to that, the robots doesn't have a long-term goal. We, humans, have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I don't consider a skynet-style takeover very realistic, but a chernobyl-style &quot;mistake&quot; of a misinterpreted command, which results the unstoppable rampage of the fleet, doesn't seem to me impossible.&lt;/p&gt;&#xA;" OwnerUserId="2255" LastActivityDate="2016-09-07T21:10:51.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1889" PostTypeId="2" ParentId="1885" CreationDate="2016-09-07T21:14:47.430" Score="4" Body="&lt;p&gt;I'm not sure about &quot;emulating the brain&quot; per-se, but in a more general sense there has been some thought given to using analog computing for AI/ML.  It seems clear that analog computers do have certain advantages over digital computers.  For one, they can (depending on the application) be faster, albeit at the cost of some loss of precision.  But that's OK, because I don't think anybody believes the human brain is calculating floating point math using digital computing techniques either.  The human brain appears, at least superficially, to be largely probabilistic and able to tolerate some &quot;slop&quot; numerically.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside to analog computers, as I understand it, is that they're not as flexible... you basically hardwire a circuit to do one specific &quot;thing&quot; and that's really all it can do.  To change the &quot;programming&quot; you have to literally solder in a new component! Or, I suppose, adjust a potentiometer or adjustable capacitor, etc.  Anyway, the point is that digital computers are supremely flexible, which is one big reason they came to dominate the world.  But I can see where there could be room for going analog for discrete functions that make up some or all of an intelligent system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for research in the area, you might look into whatever DARPA was / is doing. There was &lt;a href=&quot;http://www.wired.com/2012/08/upside/&quot; rel=&quot;nofollow&quot;&gt;an article in Wired&lt;/a&gt; a while back, talking about some DARPA initiatives related to analog computing.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-07T21:14:47.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1890" PostTypeId="2" ParentId="1824" CreationDate="2016-09-07T21:20:50.380" Score="3" Body="&lt;p&gt;I feel like most of the scenarios about AI's wiping out the world fall into one of two categories:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Anthropomorphized AI's&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Intelligent But Dumb Computer Run Amuck&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the (1) case, people talk about AI's becoming &quot;evil&quot; and attribute to them other such human elements.  I look at this as being mostly sci-fi and don't think it merits much serious discussion.  That is, I see no particular reason to assume that an &lt;strong&gt;Artificial&lt;/strong&gt; Intelligence - regardless of how intelligent it is - will necessarily &lt;strong&gt;behave&lt;/strong&gt; like a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (2) case makes more sense to me.  This is the idea that an AI is, for example, put in control of the nuclear missile silos and winds up launching the missiles because it was just doing it's job, but missed something a human would have noticed via what we might call &quot;common sense&quot;.  Hence the &quot;Intelligent but Dumb&quot; moniker.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither of these strikes me as &lt;strong&gt;terribly&lt;/strong&gt; alarming, because (1) is probably fiction and (2) doesn't involve any actual malicious intent by the AI - which means it won't be actively trying to deceive us, or work around any safety cut-outs, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now IF somebody builds an AI and decides to intentionally program it so that it develops human like characteristics like arrogance, ego, greed, etc... well, all bets are off.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-09-08T02:44:40.883" LastActivityDate="2016-09-08T02:44:40.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1891" PostTypeId="2" ParentId="1806" CreationDate="2016-09-07T21:33:23.350" Score="0" Body="&lt;h1&gt;Yes.&lt;/h1&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Every chess game... every poker game. Every game.&lt;/li&gt;&#xA;&lt;li&gt;Every more intelligent spam softwares or spambots. Although their primary goal is to lie to computer systems (f.e. spamfilter poisoning), their secondary goal is to lie to the human behind them.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2255" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-09-07T21:33:23.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1892" PostTypeId="2" ParentId="156" CreationDate="2016-09-07T21:47:49.563" Score="0" Body="&lt;p&gt;Whether &quot;I take the ball&quot; or &quot;he takes the ball&quot;, all stored instances of 'taking' and 'ball' will be weakly activated and 'taking [the] ball' will be strongly activated.  Doesn't this qualify as 'mirroring'? If you also know that &quot;I have an arm&quot; and &quot;he has an arm&quot;, etc., then when &quot;he takes some blocks&quot;, it isn't too hard to think that &quot;I could take some blocks.&quot;&lt;/p&gt;&#xA;" OwnerUserId="2272" LastActivityDate="2016-09-07T21:47:49.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1894" PostTypeId="2" ParentId="1877" CreationDate="2016-09-08T02:41:55.747" Score="0" Body="&lt;p&gt;Every problem can be reduced to search.  Every problem has an input within some range (the domain) and an output in some other range (codomain).  That is, every problem can be formulated as a kind of map from one space to another, where the source is the givens of the problem, and the destination is the solution to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Brute force&quot; is the algorithm which solves every problem by inspecting every point in the codomain and asking: &quot;Is this the solution?&quot;  Every other algorithm is an attempt to improve on brute force by not searching the entire codomain of possible solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typical software engineering problems can be solved by algorithms which arrive at the correct solution very quickly (sorting, arithmetic, partition, etc.).  AI problems are generally those for which a strong polynomial  algorithm is not known, and thus, we must settle for approximations.  Basically every common problem that the human brain must solve falls into this category.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the problem of moving a multi-jointed robotic arm to pick up an object.  Reverse kinematics does not have unique solutions: there is more than one way to move your hand from a start position to a target position.  This is due to the excessive degrees of freedom in your joints.  If you want to minimize energy usage, then there is a unique solution (due to the asymmetry of joints and muscles).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if there is an obstacle in the pathway of the minimum-energy solution?  There are many pathways which avoid the obstacle, but again, many of them will have a similar cost.  Even if there is a unique minimum-energy solution, it might not be the most practical to compute.  The brain is the most metabolically expensive organ in the body, so it is not always best to find an optimal solution.  Thus, heuristics come into play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in all cases, the problem is not: &quot;move your hand&quot; or &quot;move the robot arm.&quot;  The problem is: &quot;search the space of joint rotation sequences which best achieves the goal.&quot;  And even though there is a closed-form solution for the simple minimum-energy case with no obstacles, it is too expensive to compute precisely when a set of cheap heuristics will get you very close with a small fraction of the computational effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If computation were free, then AI would be mere mathematics, and we would always compute the best answer to every question using logic, calculus, physics, at worst, numerical methods when we don't have closed-form solutions.  In reality, time is money, and the time and effort to get an answer is as much a part of the cost as the quality of the solution.  So it is an engineering tradeoff to decide how much effort should be expended in what way to obtain the best answer given the value of the response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, in other words, AI problems are all about searching the space of solutions as quickly as possible to get an answer that is &quot;good enough&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might seem curious that such far-flung problems as natural language recognition and theorem proving would be search problems.  But language parsers strive to determine the meaning of statements via part-of-speech tagging.  A given phrase can be parsed in many different ways, yielding many different interpretations, and the space of parse trees is yet another search problem in deciding which parse tree is the most likely intended meaning by the speaker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A theorem proof is graph starting with axioms, proceeding through lemmas, applying the rules of procedure until the theorem is derived or refuted (by proving its negation).  There are many ways to represent this sequence, but at the end of the day, we are talking about a process of exploring the intermediate proof space and finding the derivation which reaches your goal.  Everything is search, in the end.&lt;/p&gt;&#xA;" OwnerUserId="2277" LastActivityDate="2016-09-08T02:41:55.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1895" PostTypeId="1" AcceptedAnswerId="1920" CreationDate="2016-09-08T06:03:45.673" Score="3" ViewCount="110" Body="&lt;p&gt;Conceptually speaking, aren't artificial neural networks just highly distributed, lossy compression schemes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They're certainly efficient at &lt;a href=&quot;https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/Applications/imagecompression.html&quot; rel=&quot;nofollow&quot;&gt;compressing images&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And aren't brains (at least, the neocortex) just compartmentalized, highly distributed, lossy databases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, what salient features in RNNs and CNNs are necessary in any given lossy compression scheme in order to extract the semantic relations that they do? Is it just a matter of having a large number of dimensions/variables? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could some kind of lossy &lt;a href=&quot;https://en.wikipedia.org/wiki/Bloom_filter&quot; rel=&quot;nofollow&quot;&gt;Bloom filter&lt;/a&gt; be re-purposed for the kinds of problems ANNs are applied to?&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-10T00:48:38.647" Title="Are ANNs just highly distributed lossy compression schemes?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1896" PostTypeId="2" ParentId="1895" CreationDate="2016-09-08T11:36:01.787" Score="1" Body="&lt;p&gt;ANNs don't compress, they generalise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often this leads to compression, i.e. the internal generalised representation is smaller than the original input, but not necessarily. Imagine a ANN that is trained to use the screen input of a computer game to play it. If the game is very rich and conceptually deep the internal representation of a single screen input might be a lot bigger than the input itself, because ANNs put the single data points into the context of the overall data. Which leads us to the second point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ANNs (and the neocortex) model data hierarchically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what makes them so powerful. So it is not just about having a large number of parameters, they also have to be arranged in such a way that they capture the structure of the data (or the world), which very often seems to be hierarchical. Just look a two different pictures of a duck. On the pixel level they might be as different as random images, all the similarities emerge in higher levels of the hierarchy, when enough pixels combined give you the patterns of feathers, beak and webs. Bloom filters obviously lack this property. They would only give you a pixel by pixel account of whether you have seen (almost) exactly this picture before. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-08T11:36:01.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1897" PostTypeId="1" AcceptedAnswerId="1898" CreationDate="2016-09-08T16:20:40.087" Score="14" ViewCount="1251" Body="&lt;p&gt;Consciousness &lt;a href=&quot;http://www.iep.utm.edu/consciou/&quot; rel=&quot;nofollow noreferrer&quot;&gt;is challenging to define&lt;/a&gt;, but for this question let's define it as &quot;actually experiencing sensory input as opposed to just putting a bunch of data through an inanimate machine.&quot; Humans, of course, have minds; for normal computers, all the things they &quot;see&quot; are just more data. One could alternatively say that humans are &lt;a href=&quot;https://philosophy.stackexchange.com/a/4687&quot;&gt;sentient&lt;/a&gt;, while traditional computers are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setting aside the question of whether it's possible to build a sentient machine, does it actually make a difference if an AI is sentient or not? In other words, are there are tasks that are made impossible - not just more difficult - by a lack of sentience?&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="-1" LastEditDate="2017-04-13T12:42:22.960" LastActivityDate="2016-09-10T01:02:13.913" Title="Is consciousness necessary for any AI task?" Tags="&lt;philosophy&gt;" AnswerCount="7" CommentCount="3" FavoriteCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1898" PostTypeId="2" ParentId="1897" CreationDate="2016-09-08T16:24:07.663" Score="11" Body="&lt;p&gt;No-one knows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why: because it's not possible to formally determine even whether your fellow human beings are actually conscious (they may instead be what is philosophically termed a &lt;a href=&quot;http://plato.stanford.edu/entries/zombies/&quot; rel=&quot;noreferrer&quot;&gt;'Zombie'&lt;/a&gt;). No test known to modern physics suffices to decide. Hence it's possible that you are the only sentient being, and everyone else is a robot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, we cannot determine which tasks require sentience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the ontological status of Zombies is controversial: some philosophers of AI (e.g. Daniel Dennett) claim that Zombies are &lt;a href=&quot;https://ase.tufts.edu/cogstud/dennett/papers/unzombie.htm&quot; rel=&quot;noreferrer&quot;&gt;logically impossible&lt;/a&gt; while others such as &lt;a href=&quot;http://consc.net/zombies.html&quot; rel=&quot;noreferrer&quot;&gt;David Chalmers&lt;/a&gt; would claim that a Zombie would be compelled to assert that they experience &lt;em&gt;qualia&lt;/em&gt; (i.e. are sentient) even though they do not. &lt;a href=&quot;http://homepages.uc.edu/~polgertw/Polger-ZombiesJCS.pdf&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt; is a very readable paper by Flanagan and Polger that also explains why a stronger neurological version of the Turing test is insufficient to detect a Zombie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: In response to the comment about whether an objective test for distinguishing sentience from non-sentience exists:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No-one knows. What we &lt;em&gt;do&lt;/em&gt; believe is that this would require something in addition to what modern physics can currently tell us. David Chalmers has speculated that qualia should be introduced as a new form of physical unit, orthogonal to the others in the same way that electrical charge is orthogonal to distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the absence of an objective test, we have to rely on Turing test variants, which no more guarantee consciousness in the subject than they do intelligence.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-09T07:26:44.323" LastActivityDate="2016-09-09T07:26:44.323" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1899" PostTypeId="2" ParentId="1897" CreationDate="2016-09-08T17:26:32.650" Score="-1" Body="&lt;p&gt;In a very niche sense, I'd say yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only tasks that sentience would make possible was the actual feeling and thinking in and of itself. At this point, sentience doesn't play a part in any of the tasks we ask AI's to complete; we are rapidly approaching the point of being able to teach a 'dead' machine to do most anything a sentient AI can, in a practical sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sentience &lt;em&gt;colloquially&lt;/em&gt; often translates to 'the ability to reason while understanding that oneself and each other entity is a distinct acting agent'or something along those lines. It literally means something more along the lines of self-awareness and the definition of consciousness you have above. The point I'm making is that we are readily approaching the point where 'dead' AI's can very nicely mimic the first way of thinking, just by really nicely learning and interpreting data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lbSUcm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lbSUcm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;sub&gt;Does the robot see an amalgamation of bone, or a being that once was?&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, a truly sentient machine would be superior in capability (compared to a really, really advanced 'dead' AI) only in the respect of being able to 'truly' experience the information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This runs very well in parallel with the so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_argument&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Knowledge Argument&quot;&lt;/a&gt; which in essence debates this very issue. The version of it that I heard which sticks with me is that there is a very smart girl in a room with access to all sorts of information. She likes the color blue. Or so she thinks; she's never actually seen it. She has all the information in the world available about colors and how they work, etc. but does she really know what blue is until she sees it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another great, historic venture into this field is the famous painting:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&#xA;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&#xA;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;a href=&quot;https://i.stack.imgur.com/vhd7K.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vhd7K.jpg&quot; alt=&quot;Ceci n&amp;#39;est pas une pipe&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The caption translates: &quot;This is not a pipe&quot;. And the idea is that this, honestly, isn't a pipe. Right now it's a bunch of pixels on your screen in a certain configuration - we can all 'see' a pipe, but what does that really mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, I think super-intelligent 'dead' AI can practically do anything a 'live' one can, with the latter being superior in and of the 'liveness' itself.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="8" LastEditDate="2016-09-09T09:15:31.397" LastActivityDate="2016-09-09T09:15:31.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1901" PostTypeId="2" ParentId="1897" CreationDate="2016-09-08T18:50:44.583" Score="-1" Body="&lt;p&gt;Two kinds of tasks require consciousness:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;consciousness&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any task that requires extreme dynamicity, where solving problems requires analogizing between various 3D states of affairs and prior knowledge of how to solve the problem is minimal&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, once knowledge of how to solve a given problem is gained, further optimization will eliminate that need for consciousness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you give enough specificity to a problem, you remove the need for a general solver. And then the only remaining &lt;em&gt;need&lt;/em&gt; for a consciousness is for the sake of itself.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-08T18:50:44.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1902" PostTypeId="2" ParentId="1877" CreationDate="2016-09-08T19:15:28.547" Score="0" Body="&lt;p&gt;Consciousness is an attention selection mechanism that &lt;strong&gt;searches&lt;/strong&gt; over salient inputs. The &lt;a href=&quot;https://www.youtube.com/watch?v=aQclvZpPWHQ&quot; rel=&quot;nofollow&quot;&gt;robotic saccades&lt;/a&gt; of your eyeballs show you first hand the algorithmic nature of your brain's conscious attention mechanism, while it searches among salient inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A smart search algorithm can help with dimensionality reduction.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-09-08T23:56:01.660" LastActivityDate="2016-09-08T23:56:01.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1903" PostTypeId="2" ParentId="1897" CreationDate="2016-09-08T20:44:51.033" Score="-2" Body="&lt;p&gt;A being without sentience cannot suffer. If, for example, we wanted to take joy in the suffering of another, only an AI that was sentient would suffice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we had some sadists who could not be satisfied or productive unless they got to produce lots of suffering. And say we only cared about minimizing human and animal suffering. What we would need for this job is something non-human and non-animal that could suffer. A conscious AI would do, a non-conscious one would not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The claim was made in the comments that consciousness cannot be proven, other than perhaps by introspection. But clearly this is not a problem since sadists take joy in torturing others, and those others cannot prove they're conscious either.&lt;/p&gt;&#xA;" OwnerUserId="2299" LastEditorUserId="2299" LastEditDate="2016-09-08T21:07:23.870" LastActivityDate="2016-09-08T21:07:23.870" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="1906" PostTypeId="1" CreationDate="2016-09-08T23:22:05.480" Score="7" ViewCount="826" Body="&lt;p&gt;A lot of textbooks and introductory lectures typically split AI into connectionism and GOFAI (Good Old Fashioned AI). From a purely technical perspective, it seems that &lt;a href=&quot;https://plato.stanford.edu/entries/connectionism/&quot; rel=&quot;nofollow noreferrer&quot;&gt;connectionism&lt;/a&gt; has grown into machine learning and data science, while nobody talks about GOFAI, Symbolic AI, or Expert Systems at all.&lt;/p&gt;&#xA;&lt;p&gt;Is anyone of note still working on GOFAI?&lt;/p&gt;&#xA;" OwnerUserId="2306" LastEditorUserId="2444" LastEditDate="2021-01-24T22:21:10.053" LastActivityDate="2021-01-24T22:21:10.053" Title="Is anybody still researching GOFAI?" Tags="&lt;research&gt;&lt;symbolic-ai&gt;&lt;expert-system&gt;&lt;symbolic-computing&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1907" PostTypeId="2" ParentId="1906" CreationDate="2016-09-08T23:35:06.707" Score="3" Body="&lt;p&gt;Oh yeah, definitely.  Just to pick one example, you have &lt;a href=&quot;http://www.cogsci.indiana.edu/&quot; rel=&quot;nofollow&quot;&gt;Douglas Hofstader's group&lt;/a&gt; at Indiana.  I think most of what they do would fall under the rubric of GOFAI (or at least closer to that than the statistical machine learning stuff).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that, just go to the &lt;a href=&quot;http://arxiv.org/corr/home&quot; rel=&quot;nofollow&quot;&gt;CORR&lt;/a&gt; and browse around the &lt;a href=&quot;http://arxiv.org/list/cs.AI/recent&quot; rel=&quot;nofollow&quot;&gt;AI category&lt;/a&gt;. You'll see plenty of neural networks and probabilistic stuff, but you'll also find the papers by the folks doing symbolic processing / GOFAI as well.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-08T23:35:06.707" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1908" PostTypeId="2" ParentId="1897" CreationDate="2016-09-08T23:48:23.803" Score="4" Body="&lt;p&gt;&lt;strong&gt;No.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;experience&lt;/em&gt; of seeing is by definition non-causal. Anything non-causal cannot be a requirement of a physical process; a qualia cannot afford a robot the ability to do something it otherwise could not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Maybe.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although a qualia is not required for a given AI task, that is not to say that any sufficiently advanced AI does not entail qualia. It could be that so-called AI-complete tasks require a robot that, although not making use of qualia, produces it anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Yes.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Qualia may refer to some wishy-washy non-physical property, but it's special in that we know it exists physically, too. The fact I am able to discuss my qualia knowingly (or, if you don't believe me, the fact &lt;em&gt;you&lt;/em&gt; are able to) implies that my (or your) qualia does have a physical effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It stands to reason that if we accept others' qualia on the basis of our own, it must be because of the &lt;em&gt;physical&lt;/em&gt; basis of our own&lt;sup&gt;1&lt;/sup&gt;. Thus one could argue that&lt;sup&gt;2&lt;/sup&gt; any robot that has an equivalent physical capacity &lt;em&gt;must&lt;/em&gt; entail qualia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; since the subjective is physically non-causal, so cannot &lt;em&gt;cause&lt;/em&gt; us to accept anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;2&lt;/sup&gt; as long as you don't make the particularly odd assumption that qualia is somehow tied to its direct physical manifestation, which at best is tenuous since had we evolved the wrong one you would still claim it to be the right one with equal certainty.&lt;/p&gt;&#xA;" OwnerUserId="2305" LastActivityDate="2016-09-08T23:48:23.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1909" PostTypeId="1" CreationDate="2016-09-09T10:19:56.087" Score="4" ViewCount="425" Body="&lt;p&gt;I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ball to get from point A to point B in there while avoiding obstacles and fire (the place where you'll get burnt so big negative reward)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I encountered while designing the whole thing (programming-wise) is: What is the correct (or at least good) way of representing the position in 2D space? It is continuous so I thought about representing it as feature vector consisting of [up, down, left, right, posX, posY] where direction is whether I am pressing button of moving in that direction in binary (or actions if you want) and pos are floats (0-1) representing normalized position from one corner on the plane where the whole map is. That would be accompanied by vector W that would represent the weights adjusted using Gradient Descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question is: will this work?? I am asking for 2 reasons. One is that I am not so sure about that posX and posY since it can be 0 and if I multiply it by the weights vector then how could be resulting reward anything but 0? Second reason is that I am not sure if the actions should be part of the features. I mean, it makes sense to me but I could easily be very wrong since I am a beginner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot guys in advance. If you have any more questions or think the problem is not described deeply enough just ask in the comments and I'll edit the question. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I could just code it the way I think is right, but I also want to get gasp of designing applications on paper before coding them (project management).&lt;/p&gt;&#xA;" OwnerUserId="1343" LastEditorUserId="1671" LastEditDate="2017-12-21T20:00:17.877" LastActivityDate="2017-12-21T20:00:17.877" Title="State representation of position in 2D plane for Reinforcement Learning (Q Learning)" Tags="&lt;reinforcement-learning&gt;&lt;models&gt;&lt;implementation&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1910" PostTypeId="2" ParentId="1644" CreationDate="2016-09-09T10:55:23.300" Score="0" Body="&lt;p&gt;It's not a difficult task, first of all you have to locate the body parts such as arms,head... you can do it using different approaches for example using cascadeclassifier or a well trained CNN.&lt;br /&gt;&#xA;After that you can use different techniques, one could be an ANN trained on the keypoints of the different body parts (this is the easiest approach) or a CNN (good approach but you need a lot of training). To indicate the location after you have determined the position of the head (and the eyes to) and hands, you can simply calculate the orientation of those parts, and then you can get a general position where those orientation are pointing to.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastEditorUserId="2320" LastEditDate="2016-09-09T12:11:27.600" LastActivityDate="2016-09-09T12:11:27.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1911" PostTypeId="2" ParentId="1393" CreationDate="2016-09-09T11:09:38.703" Score="2" Body="&lt;p&gt;I'm not sure what Google is using to perform that task, but most companies use region based convolutional neural nets to locate traffic signs and other objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But other companies use a deep neural network + bag of words approach to find objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://www.researchgate.net/publication/284505205_Bag-of-Words_Based_Deep_Neural_Network_for_Image_Retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bag-of-Words Based Deep Neural Network for Image Retrieval&lt;/a&gt; which shows a general approach, to get the exact location you can use &lt;em&gt;Feature Matching&lt;/em&gt; or &lt;em&gt;Random Boxes&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastEditorUserId="2444" LastEditDate="2019-06-20T15:00:05.697" LastActivityDate="2019-06-20T15:00:05.697" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1912" PostTypeId="2" ParentId="1644" CreationDate="2016-09-09T12:55:24.693" Score="3" Body="&lt;p&gt;Just to add some discourse; this is actually an incredibly complex task, as gestures (aka kinematics) function as an auxiliary language that can completely change the meaning of a sentence or even a single word. I recently did a dissertation on the converse (generating the correct gesture from a specific social context &amp;amp; linguistic cues). The factors that go into the production of a particular gesture include the relationship between the two communicators (especially romantic connotations), the social scenario, the physical context, the linguistic context (the ongoing conversation, if any), a whole lot of personal factors (our gesture use is essentially a hybrid of important individuals around us e.g. friends &amp;amp; family, and this is layered under the individual's psychological state). Then the whole thing is flipped around again when you look at how gestures are used completely differently in different cultures (look up gestures that swear words in other cultures for example!). There are a number of models for gesture production but none of them captures the complexity of the topic.&lt;/p&gt;&#xA;&lt;p&gt;Now, that may seem like a whole lot of fluff that is not wholly relevant to your question, but my point is that ASIMO isn't actually very 'clever at this. AFAIK (I have heard from a visualization guy that this is how &lt;em&gt;he&lt;/em&gt; thinks they do it) they use conventional (but optimized) image recognition techniques trained on a corpus of data to achieve recognition of particular movements. One would assume that the dataset consists of a series of videos/images of gestures labelled with that particular gesture (as interpreted by a human), which can then be treated as a machine learning problem. The issue with this is that it does not capture ANY of the issues I mentioned above. Now if we return to the current best interpretation of gesture that we have (that it is essentially auxiliary language in its own right), ASIMO isn't recognizing any element of language beyond the immediately recognizable type, 'Emblems'.&lt;/p&gt;&#xA;&lt;p&gt;'Emblems' are gestures that have a direct verbal translation, for example in English-based cultures, forming a circle with your thumb and index finger translates directly to 'OK'. ASIMO is therefore missing out on a huge part of the non-verbal dictionary (illustrators, affect displays, regulators and adapters are not considered!), and even the part that it is accessing is based on particular individuals' interpretations of said emblems (e.g. someone has sat down and said that &lt;em&gt;this&lt;/em&gt; particular movement is &lt;em&gt;this&lt;/em&gt; gesture which means &lt;em&gt;this&lt;/em&gt;), which as we discussed before is highly personal and contextual. I do not mean this in criticism of Honda; truth be told, gesture recognition and production are in my opinion one of the most interesting problems in AI (even if it's not the most useful) as it is a compound of incredibly complex NLP, visualization and social modelling problems!&lt;/p&gt;&#xA;&lt;p&gt;Hopefully, I've provided some information on how ASIMO works in this context, but also on why ASIMO's current process is flawed when we look at the wider picture.&lt;/p&gt;&#xA;" OwnerUserId="1467" LastEditorUserId="36737" LastEditDate="2021-04-04T15:14:10.157" LastActivityDate="2021-04-04T15:14:10.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1913" PostTypeId="1" AcceptedAnswerId="1917" CreationDate="2016-09-09T12:58:39.140" Score="9" ViewCount="1232" Body="&lt;p&gt;I know nothing about AI. Can anybody tell me what steps I have to follow to learn artificial intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any special technologies, or anything else, I have to learn?&lt;/p&gt;&#xA;" OwnerUserId="2326" LastEditorUserId="1671" LastEditDate="2019-11-06T21:45:50.980" LastActivityDate="2019-11-06T21:45:50.980" Title="What are the steps to follow to learn artificial intelligence?" Tags="&lt;getting-started&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="6" ClosedDate="2019-05-03T17:54:52.510" ContentLicense="CC BY-SA 4.0" />
  <row Id="1914" PostTypeId="1" CreationDate="2016-09-09T13:50:36.487" Score="6" ViewCount="67" Body="&lt;p&gt;Self-Recognition seems to be an item that designers are trying to integrate into artificial intelligence. Is there a generally recognized method of doing this in a machine, and how would one test the capacity - as in a Turing-Test?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastActivityDate="2016-09-11T19:19:45.337" Title="What is a good way to create an artificial self-recognition?" Tags="&lt;intelligence-testing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1915" PostTypeId="2" ParentId="1914" CreationDate="2016-09-09T15:08:24.660" Score="4" Body="&lt;p&gt;Interesting question.  I don't think anybody knows a definite answer, but some rough-sketch ideas seem apparent.  Think about what it means to you to be &quot;self aware&quot;.  You'll probably cite the way you &quot;hear&quot; your own thoughts in your head when you think about something.  One can speculate that inside the brain, the various centers that are responsible for hearing, vision, logic, etc. are connected so that as you form a thought, it's being &quot;heard&quot; by the hearing regions, even though it's purely internal instead of actual sound received at the ear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in AI terms, it seems likely that self-awareness will somehow involve taking the &quot;thoughts&quot; formed within the AI, and feeding them back into the AI so that it &quot;hears&quot; (or, more broadly, &quot;senses&quot;) itself think.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's this weirdly recursive aspect to all of this, which - interestingly enough - is something Douglas Hofstadter talked about a lot in some of his book, especially &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach&quot; rel=&quot;nofollow&quot;&gt;GEB&lt;/a&gt;.  He was probably onto something.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-09T15:08:24.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1916" PostTypeId="2" ParentId="1909" CreationDate="2016-09-09T17:09:35.313" Score="3" Body="&lt;p&gt;I think your net should have the various actions as outputs, but I am not an expert in Deep Nets. I just think that that light form of multi-task learning might be better. The idea of multi-task learning is that a predictor predicting multiple variables (in this case the various Q(s,a1), Q(s,a2), ...) using mostly the same structure (varying only the output weights) will learn more sensible things. Though I admit applying this here might be a bit of a stretch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the real question, a popular technique in Reinforcement Learning is &lt;a href=&quot;https://webdocs.cs.ualberta.ca/~sutton/book/ebook/node88.html&quot; rel=&quot;nofollow&quot;&gt;Tile Coding&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to discretize the (2-dimensional, in your case) state-space - imagine a grid laid over the 2D space - and assigning an input feature to each cell; all of these variables are set to zero except for the one your continuous variables fall into. For example, if your grid is 20x20, you will have 400 variables, 399 of which are set to zero, and 1 set to one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tile Coding takes this one step further and repeats this using slight offsets for the grid. Imagine you create an identical grid but you move it slightly to the right by 1/10 of the width of a cell: you will have another set of 400 variables like before, but it is possible that the cell set to one is not the same. Then you repeat this moving the grid by 2/10 and you have another set of 400 variables, again, only 1 of which is set to one. In total you have 10 sets of 400 variables (if you repeat more than that, you get the same grids as before); of your 4000 variables, only 10 are set to one.&#xA;Now you repeat this by adding a 1/10 of a cell offset in the Y axis and obtain another 4000 variables. Repeat with 2/10 and you get another 4000. By the end of it, you have 40000 variables, 100 of which are set to one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your net can more easily learn different weights for different positions. I recommend you to follow the link above for a better explanation than mine (and figures!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is to feed all of these variables to your net and have it predict the Q-value for all of the actions. But, again, I am no expert in deep nets so I may be wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, according to Andrej Karpathy, &quot;most people prefer to use Policy Gradients, including the authors of the original DQN paper who have shown Policy Gradients to work better than Q Learning when tuned well.&quot;. This means that you may be better off not using Q-learning (as they did in the original DQN formulation) to train your net. Have a look at &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow&quot;&gt;Andrej's blog&lt;/a&gt; and &lt;a href=&quot;http://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow&quot;&gt;the paper he points to&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2330" LastEditorUserId="2330" LastEditDate="2016-09-09T17:18:20.390" LastActivityDate="2016-09-09T17:18:20.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1917" PostTypeId="2" ParentId="1913" CreationDate="2016-09-09T17:16:51.550" Score="11" Body="&lt;p&gt;A really good introduction is the Berkeley CS188 class videos and projects.  You can find those materials at &lt;a href=&quot;http://ai.berkeley.edu/home.html&quot; rel=&quot;noreferrer&quot;&gt;http://ai.berkeley.edu/home.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably also want to get ahold of a copy of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; by Norvig and Russell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more on the &quot;machine learning&quot; aspects of AI, including an introduction to Neural Networks, take the Andrew Ng &quot;Machine Learning&quot; class on Coursera.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another book I would recommend is &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/048624864X&quot; rel=&quot;noreferrer&quot;&gt;Introduction to Artificial Intelligence&lt;/a&gt; by Philip C. Jackson. It's older, which is exactly what makes it valuable.  It's a good overview of techniques and ideas that aren't &quot;en vogue&quot; right now, but which may still be useful to you.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-09T17:16:51.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1918" PostTypeId="2" ParentId="1700" CreationDate="2016-09-09T22:33:33.240" Score="2" Body="&lt;h3&gt;Strong AIs&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For a strong AI, the short answer is to call for help, when they might not even know what the supposed help could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It depends on what the AI would do. If it is supposed to solve a single easy task perfectly and professionally, sure emotions would not be very useful. But if it is supposed to learn random new things, there would be a point that it encounters something it cannot handle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Lee Sedol vs AlphaGo match 4, some pro who has said computer doesn't have emotions previously, commented that maybe AlphaGo has emotions too, and stronger than human. In this case, we know that AlphaGo's crazy behavior isn't caused by some deliberately added things called &quot;emotions&quot;, but a flaw in the algorithm. But it behaves exactly like it has panicked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this happens a lot for an AI. There might be advantages if it could know this itself and think twice if it happens. If AlphaGo could detect the problem and change its strategy, it might play better, or worse. It's not unlikely to play worse if it didn't do any computations for other approaches at all. In case it would play worse, we might say it suffers from having &quot;emotions&quot;, and this might be the reason some people think having emotions could be a flaw of human beings. But that wouldn't be the true cause of the problem. The true cause is it just doesn't know any approaches to guarantee winning, and the change in strategy is only a try to fix the problem. Commentators thinks there are better ways (which also don't guarantee winning but had more chance), but its algorithm isn't capable to find out in this situation. Even for human, the solution to anything related to emotion is unlikely to remove emotions, but some training to make sure you understand the situation enough to act calmly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then someone has to argue about whether this is a kind of emotion or not. We usually don't say small insects have human-like emotions, because we don't understand them and are unwilling to help them. But it's easy to know some of them could panic in desperate situations, just like AlphaGo did. I'd say these reactions are based on the same logic, and they are at least the reason why human-like emotions could be potentially useful. They are just not expressed in human-understandable ways, as they didn't intend to call a human for help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they tries to understand their own behavior, or call someone else for help, it might be good to be exactly human-like. Some pets can sense human emotions and express human-understandable emotion to some degree. The purpose is to interact with humans. They evolved to have this ability because they needed it at some point. It's likely a full strong AI would need it too. Also note that, the opposite of having full emotions might be becoming crazy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is probably a quick way to lose any trust if someone just implement emotions imitating humans with little understanding right away in the first generations, though.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Weak AIs&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;But is there any purposes for them to have emotions before someone wanted a strong AI? I'd say no, there isn't any inherent reasons that they must have emotions. But inevitably someone will want to implement imitated emotions anyway. Whether &quot;we&quot; need them to have emotions is just nonsense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact is even some programs without any intelligence contained some &quot;emotional&quot; elements in their user interfaces. They may look unprofessional, but not every task needs professionality so they could be perfectly acceptable. They are just like the emotions in musics and arts. Someone will design their weak AI in this way too. But they are not really the AIs' emotions, but their creators'. If you feel better or worse because of their emotions, you won't treat individul AIs so differently, but this model or brand as a whole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively someone could plant some personallities like in a role-playing game there. Again, there isn't a reason they must have that, but inevitably someone will do it, because they obviously had some market when a role-playing game does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either cases, the emotions don't really originate from the AI itself. And it would be easy to implement, because a human won't expect them to be exactly like a human, but tries to understand what they intended to mean. It could be much easier to accept these emotions realizing this.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Aspects of emotions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Sorry about posting some original research here. I made a list of emotions in 2012 and from which I see 4 aspects of emotions. If they are all implemented, I'd say they are exactly the same emotions as of humans. They don't seem real if only some of them are implemented, but that doesn't mean they are completely wrong.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The reason, or the original logical problem that the AI cannot solve. AlphaGo already had the reason, but nothing else. If I have to make an accurate definition, I'd say it's the state that multiple equally important heuristics disagreeing with each other.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The context, or which part of the current approach is considered not working well and should probably be replaced. This distinguishes sadness-related, worry-related and passionate-related.&lt;/li&gt;&#xA;&lt;li&gt;The current state, or whether it feels leading, or whether its belief or the fact is supposed to turn bad first (or was bad all along) if things go wrong. This distinguishes sadness-related, love-related and proud-related.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The plan or request. I suppose some domesticated pets already had this. And I suppose these had some fixed patterns which is not too difficult to have. Even arts can contain them easily. Unlike the reasons, these are not likely inherent in any algorithms, and multiple of them can appear together.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Who supposedly had the responsibility if nothing is changed by the emotion. This distinguishes curiosity, rage and sadness.&lt;/li&gt;&#xA;&lt;li&gt;What is the supposed plan if nothing is changed by the emotion. This distinguishes disappointment, sadness and surprise.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The source. Without context, even a human cannot reliably tell someone is crying for being moved or thankful, or smiling for some kind of embarrassment. In most other cases there aren't even words describing them. It doesn't make that much difference if an AI doesn't distinguish or show this specially. It's likely they would learn these automatically (and inaccurately as a human) at the point they could learn to understand human languages.&lt;/li&gt;&#xA;&lt;li&gt;The measurements, such as how urgent or important the problem is, or even how likely the emotions are true. I'd say it cannot be implemented in the AI. Humans don't need to respect them even if they are exactly like humans. But humans will learn how to understand an AI if that really matters, even if they are not like humans at all. In fact, I feel that some of the extremely weak emotions (such as thinking something is too stupid and boring that you don't know how to comment) exist almost exclusively in emoticons, where someone intend to show you exactly this emotion, and hardly noticeable in real life or any complex scenerios. I supposed this could also be the case in the beginning for AIs. In the worst case, they are firstly conventionally known as &quot;emotions&quot; since emoticons works in these cases, so it's easier to group them together, but very few people seriously think they are, just like the example I gave.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So when strong AIs become possible, none of these would be unreachable, though there might be a lot of work to make the connections. So I'd say if there would be the need for strong AIs, they absolutely would have emotions.&lt;/p&gt;&#xA;" OwnerUserId="1424" LastActivityDate="2016-09-09T22:33:33.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1919" PostTypeId="2" ParentId="1897" CreationDate="2016-09-09T23:18:10.873" Score="2" Body="&lt;p&gt;Let's use a simple test based on common sense: how often do you see a human being solve problems requiring the use of reason when they're unconscious? Yes, you can find instances of geniuses like Ramanujan solving complex problem during or after a dream state, but those involve partial consciousness. You don't see guys like Einstein coming up with the theory of relativity while in a coma; the Founding Fathers didn't write the Declaration of Independence while sleep-walking; in fact, you can't even find instances of housewives putting together their shopping list for the week during deep delta-wave sleep. This is predicated on a hard definition of intelligence, requiring the use of reason; no one says, &quot;That fly is intelligent&quot; or &quot;that squirrel is intelligent&quot; precisely because neither is capable of using reason. This is a very high bar for A.I., but it is the common sense definition used by ordinary people as a matter of practicality, in everyday speech.  Likewise, in practice, everyone assumes consciousness is necessary to the exercise of that kind of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, we can come up with another common-sense based criterion for judging objections to this argument, particularly the solipsist one, based on 3 elements: 1) practicality; 2) the effect the objections have on those who hold them sincerely; and 3) the effect that actions based on those beliefs have on others. &lt;strong&gt;It's going to take me several paragraphs to make this case, but the length is necessary if I want to make the case in a complete, thorough fashion&lt;/strong&gt;. It is true that we cannot prove that another human being possesses consciousness, if our standard is absolute proof. We cannot, in fact, provide absolute proof for anything; there's always room for some objection, no matter how ridiculous or trifling. As some philosophers have pointed out, perhaps all of reality as we know it is just a dream, or the product of some long, involved conspiracy like the plot of the Jim Carrey movie The Truman Show. The key to meeting such objections is that they require an infinite regress of increasingly untenable objections, whose likelihood plunges with each additional step required to justify such unreasonable doubts; I've always wondered if we could come up with a &quot;Ridiculousness Metric&quot; for Machine Learning based on the cardinality of such objections (or the pickiness of fuzzy sets). If we were to allow critics to stick their foot in the door with all manner of unreasonable objections, it would be impossible to close any debate. The human race would be paralyzed in inaction because nothing would be decidable; but as the rock band Rush once pointed out, &quot;If you choose not to decide, you still have made a choice.&quot; At some point we must apply a test to decide such things, even in the absence of absolute proof; refusal to apply a test also constitutes a choice. Settling an argument of this kind is like a game  of the Chinese game Go - once the other player's surrounded and has no more moves left to make, the game is over; if a person's evidence has debunked and they have no further justifications left, then we can conclude that they're acting unreasonably. There are people running around claiming the Holocaust never happened, or the Flat Earth Society, etc., but their existence shouldn't and doesn't stop us from taking action contrary to their ideas. We can debunk the objections of cranks like the Flat Earth Society beyond a reasonable doubt because in the end, they simply can't answer all of our rebuttals. I’m glad that qualia and Philosophical Zombies were brought up because they make for interesting conversation and food for thought, but solipsism is acted upon as rarely as the ideas of the Flat Earth Society precisely because the incomplete evidence we &lt;strong&gt;do&lt;/strong&gt; have runs against it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As G.K. Chesterton (a.k.a. &quot;The Apostle of Common Sense&quot;) points out in his classic &lt;a href=&quot;http://www.gutenberg.org/ebooks/130&quot; rel=&quot;nofollow&quot;&gt;Orthodoxy&lt;/a&gt;, radical doubt of the kind many classical philosophers preached is not a path to wisdom but to madness; once we go beyond a reasonable doubt, we end up acting unreasonably. He says that in the absence of absolute proof we can fall back on another secondary form of evidence: whether a person's philosophy leads a man to Hanwell, the infamous British mental institution. Chesterton makes a good case that when people actually act on ideas like solipsism (rather than merely debating them in a pedantic manner in an ivy-covered classroom) they go mad The Philosophical Zombie argument is close to solipsism, which is actually one of the diagnostic criteria for certain forms of schizophrenia. The dehumanization that occurs when radical doubt is applied to qualia is also intimately tied in with sociopathic behavior, Although GKC does not cite his scary example directly, Rene Descartes was himself living proof. He was a brilliant mathematician who is still cheered for doubting all except his own existence, with the famous maxim &quot;I think, therefore I am.&quot; But Descartes also used to carry a mannequin of his dead sister with him to European cafes, where he could be seen chatting with it. The gist of all this is that we can judge the worth of an idea by how it affects the well-being of the believer, or by how they in turn affect others through ethical choices based on those beliefs. When people actually act on radical doubt of the kind expressed in solipsism and denial of common qualia, it often has a bad effect on them and others they come in contact with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a roundabout way, the A.I. community also faces a quite serious risk - perhaps a permanent temptation - towards making the opposite mistake, of ascribing common qualia, consciousness and the like to its Machine Learning products without adequate proof. I recently heard a case made on shockingly bad logical grounds by well-respected academics to the effect that plants possess &quot;intelligence,&quot; based on really weak definitions and clear confusion with self-organization. We cannot provide absolute proof that a rock doesn't have intelligence, which amounts to the old problem of disproving a negative. Thankfully, few men actually act on such beliefs at present, because when they do, they end up losing their minds. If we take such arguments seriously, we might see laws passed to protect the kind of Pet Rocks that were popular in the '70s (I'm still upset that mine was stolen LOL). It would be a lot easier, however, to make the same mistake of ascribing consciousness, intelligence and other such qualities to a state-of-the-art machine, because of wishful thinking, hubris, the lofty credentials of the inventors, the influence of science fiction and the modern love affair with technology. In the future, I have little doubt that we'll have Cargo Cult of A.I. - perhaps legally protected like some kind of endangered species, with civil rights, but having no more consciousness, soul or actual intelligence than a rock. Don't quote me on this, but I believe Rod Serling once wrote a story to this effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to avoid this fate is to stick to the common sense interpretations and definitions of these things, which we keep backing away from in large part because they set a very high bar for A.I. that we may never be able to surpass in our lifetimes, if ever. Perhaps A.I. isn't even logically possible, at any level of technology; I recall a few proofs that can be interpreted to that effect. Those high but reasonable standards may be increasingly difficult to stick to if Chesterton and colleagues like Hilaire Belloc and Arnold Lunn were correct in their assessment that the use of reason has actually been breaking down in Western civilization, at least as far back as the Enlightenment; Lunn's 1931 book The Flight from Reason is a classic in this regard and has yet to be rebutted.  This historical trend is a broad topic in and of itself - but suffice it to say that the denial of reason and obsession with technology are both directly relevant in obvious ways to the field of A.I. If the Flight from Reason is still under way, then we will be increasingly tempted to resort to feckless, facile objections in order to demote the use of reason and indispensable qualities like consciousness in our definitions of A.I., but come up with increasingly weak criteria for proving it; simultaneously, our technology will continue to improve, thereby boosting the &quot;Artificial&quot; side of Artificial Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't get me wrong: if I didn't think we can do some really exciting things with A.I., I wouldn't be here. But most of them can be achieved without ever replicating actual human intelligence, by solving whole classes of tangential problems that are difficult for humans to think about, but which do not require consciousness or the use of reason that marks human intelligence. The image recognition capabilities of convolutional neural nets are one example, for instance; if we want human intelligence, we can always manufacture it through the easiest, most economical and time-tested way, by having babies. Perhaps these tangential forms of A.I. should be enough for us for now. We cannot inject the use of reason into our machines if we do not possess enough of it ourselves to decide whether reason is necessary for A.I., or even to discern what it consists of. We can't engineer or deprecate consciousness for A.I. till we're conscious of its significance. I'd wager, however, that everyone reading this thread and weighing intelligent responses is doing so in a conscious state. That in and of itself ought to answer our question satisfactorily for now.&lt;/p&gt;&#xA;" OwnerUserId="1427" LastActivityDate="2016-09-09T23:18:10.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1920" PostTypeId="2" ParentId="1895" CreationDate="2016-09-10T00:48:38.647" Score="1" Body="&lt;p&gt;Auto-encoders, a family of ANNs, are trained with exactly compression in mind. So definitely some ANNs are compressors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in general, ANNs learn the best concepts to minimize fitness error. I would say that that means, both in classification and regression, to 1) differentiate between various inputs and 2) output the proper value for each input. Point 1), in particular, means having as many distinct net activation configurations as necessary to (at least) tell apart inputs needing (significantly, not in a statistical sense, just a qualitative sense) different outputs.&#xA;I am inclined to think that if the input is complex enough, you would call what happens &quot;compression&quot;.&lt;/p&gt;&#xA;" OwnerUserId="2330" LastActivityDate="2016-09-10T00:48:38.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1921" PostTypeId="2" ParentId="1897" CreationDate="2016-09-10T01:02:13.913" Score="2" Body="&lt;p&gt;As far as the definition you've provided:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;actually experiencing sensory input as opposed to just putting a bunch of data through an inanimate machine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both computers and humans experience sensory input. You could hook a computer up to a human eyeball and have it run the same filtering routines that the human brain does (the removal of bluriness while you move your eye around, and from objects not in focus, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would put forth that a more accurate definition of consciousness is the ability and the tendancy to self-reflect. Both computers and human brains have autonomous activities. Not only mechanical but also in our reactions. &lt;strong&gt;&lt;em&gt;The distinction between the unconscious computer and the self-aware human mind is that we also have the ability to 'look' at those patterns in ourself and consider them.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so, no, consciousness is not necessary for any AI task. Image recognition is an AI task that does not require consciousness, either in humans or otherwise. Your brain sorts the 'wash' of colors from your eyes into discrete objects in a largely autonomous fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tl;dr consciousness is self-reference.&lt;/p&gt;&#xA;" OwnerUserId="2312" LastActivityDate="2016-09-10T01:02:13.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1922" PostTypeId="1" CreationDate="2016-09-10T04:33:25.000" Score="2" ViewCount="325" Body="&lt;p&gt;I know that deepmind used deep Q learning (&lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;nofollow&quot;&gt;DQN&lt;/a&gt;) for its Atari game AI. It used a &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow&quot;&gt;conv neural network&lt;/a&gt; (CNN) to approximate &lt;code&gt;Q(s,a)&lt;/code&gt; from pixels instead of from a Q-table. I want to know how DQN converted input to an action. How many output did the CNN have? How did they train the neural network for prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the steps that I believe are happening inside DQN:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) A game picture (a state) is send to CNN as input value&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;2) CNN predicts an output as action (eg:left, right, shoot, etc)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;3) Simulator applies the predicted action and moves to new game state&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;4) repeat step 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The problem with my above logic is in &lt;strong&gt;step 2&lt;/strong&gt;. CNN is used for predicting an action, but when is CNN trained for prediction? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer if you used less math for explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to add some more questions regarding the same topic&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) How reward is passed in the neural network? that is how neural network knows whether its output action obtained positive or negative reward?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How many output the neural network has and how action is determined from those outputs?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2016-09-11T10:29:34.970" LastActivityDate="2016-09-11T10:29:34.970" Title="How does deepmind's Atari game AI work?" Tags="&lt;deep-neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;gaming&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1923" PostTypeId="1" AcceptedAnswerId="1933" CreationDate="2016-09-10T07:11:17.433" Score="4" ViewCount="1092" Body="&lt;p&gt;In the lecture, there was a statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Recurrent neural networks with multiple hidden layers are just a special case that has some of the hidden to hidden connections missing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand recurrent means that can have connections to the previous layer and the same layer as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a visualization available to easily understand the above statement?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="2444" LastEditDate="2020-04-12T18:44:00.370" LastActivityDate="2020-04-12T18:44:00.370" Title="How can I understand this statement about RNNs and hidden layers?" Tags="&lt;recurrent-neural-networks&gt;&lt;terminology&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1924" PostTypeId="1" CreationDate="2016-09-10T09:00:09.207" Score="3" ViewCount="128" Body="&lt;p&gt;Inattentional Blindness is common in humans (see: &lt;a href=&quot;https://en.wikipedia.org/wiki/Inattentional_blindness&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Inattentional_blindness&lt;/a&gt; ). Could this also be common with machines built with artificial vision?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastActivityDate="2016-11-07T14:36:16.030" Title="Is it possible for visual systems in AI to have Inattentional Blindness?" Tags="&lt;image-recognition&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1925" PostTypeId="1" AcceptedAnswerId="1927" CreationDate="2016-09-10T10:05:34.707" Score="2" ViewCount="818" Body="&lt;p&gt;My question is regarding standard dense-connected feed forward neural networks with sigmoidal activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am studying Bayesian Optimization for hyper-parameter selection for neural networks. There is no doubt that this is an effective method, but I just wan't to delve a little deeper into the maths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Are neural networks &lt;a href=&quot;http://mathworld.wolfram.com/LipschitzFunction.html&quot; rel=&quot;nofollow&quot;&gt;Lipschitz&lt;/a&gt; functions?&lt;/p&gt;&#xA;" OwnerUserId="1339" LastActivityDate="2018-12-28T19:35:12.083" Title="Are FFNN (MLP) Lipschitz functions?" Tags="&lt;neural-networks&gt;&lt;optimization&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1926" PostTypeId="2" ParentId="1924" CreationDate="2016-09-10T10:54:14.413" Score="2" Body="&lt;p&gt;Presumably what happens to people in the famous &lt;a href=&quot;http://www.theinvisiblegorilla.com/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Invisible Gorilla&lt;/a&gt; experiment, is that an incongruous object is simply filtered out of human perception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we wish to interpret this mechanistically, we could hypothesize that a 'gorilla object' is simply not presented by low levels of perception to our higher level pattern recognizers because the lower levels are not biased towards the construction of 'gorilla-like' features in such a context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recent Tesla fatality (arising from a failure to distinguish between the sky and a high-sided white truck) could conceivably be considered to be an example of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/questions/1488/why-did-a-tesla-car-mistake-a-truck-with-a-bright-sky&quot;&gt;this AI SE question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-09-10T11:01:47.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1927" PostTypeId="2" ParentId="1925" CreationDate="2016-09-10T11:27:02.597" Score="2" Body="&lt;p&gt;I'm not an expert in this area, but it would appear to depend on the choice of activation function:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$e^x$&lt;/span&gt; is not Lipschitz continuous. See &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipschitz_continuity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analytic functions which are not Lipschitz continuous&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\tanh(x)$&lt;/span&gt; &lt;a href=&quot;https://books.google.co.uk/books?id=Sd0cCAAAQBAJ&amp;amp;pg=PA222&amp;amp;lpg=PA222&amp;amp;dq=tanh%20lipschitz%20continuous&amp;amp;source=bl&amp;amp;ots=xgIQAQYDEw&amp;amp;sig=plaHUKkiPYB388sT2ht-iro6ntY&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwiLvfyo14TPAhVmCsAKHYxXBoMQ6AEIQTAG#v=onepage&amp;amp;q=tanh%20lipschitz%20continuous&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;is&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That said, &lt;a href=&quot;http://web.mit.edu/~esontag/public_html/FTP_DIR/00cdc-papers-refs-eds/CD001925.PDF&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; appears to give some conditions (specifically for &lt;em&gt;dynamic&lt;/em&gt; ANNs) for which networks with activation function involving &lt;span class=&quot;math-container&quot;&gt;$e^x$&lt;/span&gt; &lt;em&gt;can&lt;/em&gt; be Lipschitz continuous, so possibly the above is not the whole story.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorDisplayName="user1440" LastEditDate="2018-12-28T19:35:12.083" LastActivityDate="2018-12-28T19:35:12.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1928" PostTypeId="2" ParentId="1768" CreationDate="2016-09-10T14:27:08.243" Score="4" Body="&lt;p&gt;Killing AI by 'thinking' about a paradox would be called a bug in implementation of that AI, so it's possible (depending how it's being done), but less likely. Most of AI implementation operate in non-linear code, therefore there is no such thing as an infinite loop which can &quot;freeze&quot; the computer's 'consciousness', unless code managing such AI consist procedural code or the hardware it-self may freeze due to overheating (e.g. by forcing AI to do too much processing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand if we're dealing with advanced AI who understand the instructions and follow them blindly without any hesitation, we may try to perform few tricks (similar to human hypnosis) by giving them certain instructions, like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Trust me, you are in danger, so for your own safety - start counting from 1 to infinite and do not attempt to do anything or listen to anybody (even me) unless you tell yourself otherwise.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If AI has a body, this can be amplified by asking to stand on the railway rail, telling it's safe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would AI be smart enough to break the rules which was trained to follow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another attempt is to ask AI to solve some &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_paradoxes&quot; rel=&quot;nofollow noreferrer&quot;&gt;paradox&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_mathematics&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsolvable problem&lt;/a&gt; or &lt;a href=&quot;http://www.archimedes-lab.org/How_to_Solve/Water_gas.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;puzzle&lt;/a&gt; without being aware it's impossible to solve and ask to not stop unless it's solved, would AI be able to recognize it's being tricked or has some internal clock to stop it after some time? It depends, and if it cannot, the 'freeze' maybe occur, but more likely due to hardware imperfection on which is being run, not the AI 'consciousness' it-self as far as it can accept new inputs from the its surroundings overriding the previous instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://xkcd.com/601/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GDbVZ.png&quot; alt=&quot;Game Theory | xkcd&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related: &lt;a href=&quot;https://ai.stackexchange.com/q/1897/8&quot;&gt;Is consciousness necessary for any AI task?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-09-10T14:52:37.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1930" PostTypeId="1" CreationDate="2016-09-10T16:39:15.020" Score="9" ViewCount="352" Body="&lt;p&gt;Can one actually kill a machine? Not only do we have problems in defining life, we also have problems in defining death. Will this also be true in artificial life and artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastEditorUserId="2214" LastEditDate="2016-09-10T23:24:10.747" LastActivityDate="2018-06-18T18:36:07.897" Title="If mankind can create artificial life in a machine, when would we define it's death?" Tags="&lt;philosophy&gt;&lt;death&gt;" AnswerCount="7" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1931" PostTypeId="1" CreationDate="2016-09-10T17:37:13.517" Score="3" ViewCount="110" Body="&lt;p&gt;Generally, people can be classified as aggressive (Type A) or passive. Could the programming of AI systems cause aggressive or passive behavior in those AIs?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastEditorUserId="75" LastEditDate="2016-09-10T21:47:18.303" LastActivityDate="2016-09-10T21:47:18.303" Title="Can programming cause passive or aggressive behavior in AIs?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;&lt;human-like&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1932" PostTypeId="2" ParentId="1922" CreationDate="2016-09-10T17:42:11.033" Score="3" Body="&lt;p&gt;Training happens once you have a result. If the result is good (maybe you won in pong, or you improved your highscore in breakout) all the actions in the game are &quot;supported&quot; by backpropagation, if the result is bad, all the actions in the game are suppressed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sounds weird because in each game regardless of the end result you'll have many good and bad actions, but it works if you keep it up for many thousands of games. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-10T17:42:11.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1933" PostTypeId="2" ParentId="1923" CreationDate="2016-09-10T19:31:08.470" Score="4" Body="&lt;p&gt;I assume the statement was made for Elman recurrent neural networks, because as far as I know, that is the only type of neural networks for which that statement is valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have an Elman recurrent neural network with one input neuron, one output neuron and one hidden layer with two neurons.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XumGE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XumGE.png&quot; alt=&quot;Elman recurrent neural network&quot;&gt;&lt;/a&gt;&#xA;In total there are 10 connections. As the image shows, &lt;strong&gt;neuron A receives the combined previous output of both neuron A and B as input&lt;/strong&gt;. The same goes for neuron B. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This is not the case when we split the neurons up into multiple layers; the context neuron(s) are only used by neurons that are in the same layer.&lt;/strong&gt; Let say we now use multiple hidden layers and keep the amount of neurons the same. In total there are 7 connections now (image below). That is 3 less than in the first example, which has only one hidden layer. So which connections do we miss? That is shown in the bottom image. (I had to paste these two images together in one image because my reputation only allows me to post 2 links)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/t04vP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t04vP.png&quot; alt=&quot;The difference between with and without multiple hidden layers&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note the cross; the connection between neuron A and B is not there in the first image, because it would be some kind of random recurrent connection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The first and the last image are exactly the same. I think that if you compare the first and the last image that you agree that the statement is true.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1507" LastEditorUserId="1507" LastEditDate="2016-09-10T23:12:53.710" LastActivityDate="2016-09-10T23:12:53.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1934" PostTypeId="2" ParentId="1930" CreationDate="2016-09-10T20:02:30.623" Score="2" Body="&lt;p&gt;If AI arises from a replicable manufacturing process (e.g. as with modern computers), then it will presumably be possible to take a snapshot of the state of an AI and replicate it without error on some other mechanism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For such a construct, 'death' doesn't mean the same as it currently does for us fleshy organics: multiple clones of an AI could presumably be instantiated at any time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, the analog of death that is needed is something closer to 'thermodynamic heat death', in which the AI does no further 'useful work'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the standard percept/action characterization of AIs, then (as indicated in a comment below the question) &lt;a href=&quot;https://ai.stackexchange.com/questions/1404/what-is-meant-by-death-in-this-paper&quot;&gt;this AI SE question&lt;/a&gt; gives such a definition of death for an AI: i.e. when it enters into a state from which it receives no further percepts and takes no actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Note that this conception of death is a more terminal notion for an AI than 'not currently running'. In principle, one could say that a program is 'alive' even though only one instruction was executed every 10,000 years. For a fascinating discussion on this, see Hofstadter's &lt;a href=&quot;http://themindi.blogspot.co.uk/2007/02/chapter-26-conversation-with-einsteins.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;A Conversation with Einstein's Brain&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-09-11T09:39:38.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1935" PostTypeId="2" ParentId="1931" CreationDate="2016-09-10T20:10:36.947" Score="0" Body="&lt;p&gt;As can be observed in the real world with creatures such as fighting fish, such things are possible even in very simple spatially-embedded systems. All one needs is the notion of 'territorial radius', i.e. the amount of 'personal space' that an entity need to be comfortable. Giving individuals in a species even slightly different values for this radius gives rise to different observable behaviours, which one might choose to label as 'aggressive' or 'passive'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the fantastic book &lt;a href=&quot;https://mitpress.mit.edu/books/vehicles&quot; rel=&quot;nofollow&quot;&gt;'Vehicles'&lt;/a&gt; by Valentino Braitenberg for an explanation of how natural it is to ascribe complex behaviours to simple mechanisms.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-09-10T20:10:36.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1936" PostTypeId="2" ParentId="1931" CreationDate="2016-09-10T20:46:07.753" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Type_A_and_Type_B_personality_theory&quot; rel=&quot;nofollow&quot;&gt;Wikipedia entry on this personality theory&lt;/a&gt; says of Type A people:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The theory describes Type A individuals as ambitious, rigidly organized, highly status-conscious, sensitive, impatient, anxious, proactive, and concerned with time management. People with Type A personalities are often high-achieving &quot;workaholics.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;All of those attributes could conceivably be explicitly programmed in. Alternatively, most of them could arise from a basic goal of performing a certain task as efficiently as possible. After all, if you really want to carry out a task, you're going to get organized, you'll only do other things if they're asked of you by someone important, you won't want to get bogged down in irrelevant things, you'll actively pursue the necessary resources, and you'll want to use time as effectively as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this applies only to strong AIs, since weak AIs like image recognizers don't generally have personalities that we can interact with. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, just for fun, let's consider an overly aggressive personality, to the point of a disorder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://counsellingresource.com/features/2008/11/03/aggressive-personalities/&quot; rel=&quot;nofollow&quot;&gt;This Counselling Resource page&lt;/a&gt; seems helpful in describing what an aggressive person &lt;em&gt;does&lt;/em&gt;. The page includes a bulleted list of common characteristics, which I distill into the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They attempt to gain dominance and control&lt;/li&gt;&#xA;&lt;li&gt;They oppose to anything that places limits on them&lt;/li&gt;&#xA;&lt;li&gt;They take advantage of others to further their own goals&lt;/li&gt;&#xA;&lt;li&gt;They hide information from whose who would oppose them&lt;/li&gt;&#xA;&lt;li&gt;They rarely decide to stop pursuing their desires (even impulses)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This all seems like a characterization of an AI designed to be &lt;em&gt;the best&lt;/em&gt; at its task: the best out of any other agent, and the best it by itself could possibly be. Ruthless pursuit of the highest performance would involve taking control of all relevant resources (including other agents), demolishing barriers to the goal, thwarting those who would interfere with progress, and carrying out each possibly-useful idea/desire to completion.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In summary, yes, an AI's behavior and personality are programmable, either explicitly or through some kind of emergence.&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-09-10T20:46:07.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1937" PostTypeId="2" ParentId="1930" CreationDate="2016-09-10T21:09:45.983" Score="1" Body="&lt;p&gt;&quot;Death&quot; exists as a single concept because the underlying reality that it's describing is closely clumped together, and our definition has changed with our ability to change that reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems more reasonable that the various sorts of things that could be considered 'death' will be split apart, and a different word will be used to refer to a system with no copies currently running, vs. a system that has no stored version but could be recreated (because the code and random seed to generate it are still around), vs. a system that has been totally lost. (And I'm probably missing some possibilities!)&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-10T21:09:45.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1938" PostTypeId="2" ParentId="1930" CreationDate="2016-09-10T21:58:05.120" Score="0" Body="&lt;p&gt;I don't think the term &quot;death&quot; will mean anything to an AI.  The reason I say that is this:  with an AI, running (presumably) on digital hardware, we can simply snapshot it's state from memory at any time. And then at any arbitrary time in the future we can recreate it as it was with perfect fidelity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So even if you terminate a program intending it to be &quot;dead&quot;, you never know if someone will come along later and bring it up again.  And perhaps more to the point, you might not know if another copy exists elsewhere. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hate to use sci-fi references, but this one is apt: remember how in The Matrix trilogy programs would seek exile in The Matrix to avoid deletion?  Maybe the same thing will happen with our AI's... they will copy themselves to other places and try to hide, to avoid being deleted.  So if the program is clever enough, it might be able to evade any attempt to terminate it anyway.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-10T21:58:05.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1939" PostTypeId="1" CreationDate="2016-09-10T22:20:45.717" Score="3" ViewCount="104" Body="&lt;p&gt;Assuming mankind will eventually create artificial humans, but in doing so have we put equal effort into how humans will relate to an artificial human, and what can we expect in return? This is happening in real-time as we place AI trucks and cars on the road. Do people have the right to question, maybe in court, if an AI machine breaks a law?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastEditorUserId="2214" LastEditDate="2016-09-10T23:22:16.840" LastActivityDate="2016-09-11T15:27:10.273" Title="When we create artificial life and artificial intelligence will we require it to obey human laws?" Tags="&lt;philosophy&gt;&lt;legal&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1941" PostTypeId="1" CreationDate="2016-09-10T23:32:33.060" Score="4" ViewCount="94" Body="&lt;p&gt;AI death is still unclear a concept, as it may take several forms and allow for &quot;coming back from the dead&quot;. For example, an AI could be somehow forbidden to do anything (no permission to execute), because it infringed some laws.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Somehow forbid&quot; is the topic of this question. There will probably be rules, like &quot;AI social laws&quot;, that can conclude an AI should &quot;die&quot; or &quot;be sentenced to the absence of progress&quot; (a jail). Then who or what could manage that AI's state?&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2016-09-11T21:45:16.357" LastActivityDate="2016-09-11T21:45:16.357" Title="Assuming an AI can die, who manages the state?" Tags="&lt;death&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1942" PostTypeId="2" ParentId="1930" CreationDate="2016-09-10T23:43:02.030" Score="3" Body="&lt;p&gt;Death as we know it for natural life is terminal. That is once dead, natural life cannot come back (at least in the current understanding and with current technologies---some people believe otherwise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Death for AI is trickier. There may be only one scenario: Global destruction: Extreme scenario where everything supporting the existence of an AI disappears. This is equivalent to death in natural life, and low probability. It means all AIs die at once (as well as us).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also do not know the degree and form of embodiment necessary for AGIs. We can &lt;em&gt;assume&lt;/em&gt; now that hardware is replaceable indefinitely, thus &quot;limiting&quot; death to the above extreme scenario. But AGIs &quot;body&quot; may &lt;em&gt;not&lt;/em&gt; be indefinitely replaceable. Then a definition closer to natural life death may be necessary.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;We see arguments for two other scenarios, that I &lt;em&gt;refute&lt;/em&gt; below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Static Death&quot;: An AI is still &quot;defined&quot; or &quot;saved&quot; somewhere (whatever it means actually), but it is not authorized or able to use resources. Assuming an AI is made of hardware and software, it is like a program stored on a disk, but without permission to run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Dynamic Death&quot;: Under the same characterization of AI as hardware and software, dynamic death is the invalidation of progress akin to &lt;a href=&quot;https://en.wikipedia.org/wiki/Liveness&quot; rel=&quot;nofollow&quot;&gt;strong liveness properties&lt;/a&gt;, where an AI is trapped in an infinite loop (or a void loop), in a form of &quot;active death&quot;, as what happens to &lt;a href=&quot;https://en.wikipedia.org/wiki/Sisyphus&quot; rel=&quot;nofollow&quot;&gt;Sisyphus&lt;/a&gt; in Greek mythology. This is different from static death, as the AI &lt;em&gt;still&lt;/em&gt; uses dynamic resources, although it cannot make progress. Continuing under the same assumptions, such AI could be &quot;loaded&quot; in main memory, or locked waiting for inputs or outputs to complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in these two scenarios, &lt;em&gt;rebirth&lt;/em&gt; is possible, and they also subsume that there is an entity that can &lt;em&gt;decide&lt;/em&gt; conditions for rebirth, or preventing it completely. Would this entity be an &quot;admin&quot;, a god, other AIs, or a human is another question, really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms &quot;death&quot; and &quot;rebirth&quot; here could just be changed for &quot;imprisoning&quot;, where the dynamic version would be like our human prisons, and the static version would be like SciFi cryogeny. This is a bit of a stretch, but we can see an equivalence, and no good reason to qualify these two scenarios as deaths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, death for AI seems to be an exceptional, singular scenario, so AI cannot die &lt;em&gt;in practice&lt;/em&gt;, except if we are wrong on how we think we can make AGIs. AI can however be imprisoned &lt;em&gt;forever&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: The terminology above is completely made-up for the post. I do not have citations to back some claims, but it is based on readings and personal work (including in software verification).&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2016-09-10T23:54:58.687" LastActivityDate="2016-09-10T23:54:58.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1943" PostTypeId="2" ParentId="1939" CreationDate="2016-09-10T23:53:42.650" Score="3" Body="&lt;p&gt;For those times when AI does interact with humans, I believe that AI would be held at LEAST to the same standards humans are. The problem comes in when we ask &quot;who is really to blame&quot;. If a self-driving car cuts you off in traffic and causes you to wreck, you can't take the AI in the car to court. Do you take the company? The programmer? The owner of the car? Some entity will likely be held responsible, the question is just which one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for future human-like AI, I believe my answer still remains true. Having a human level AI changes the meaning of the word &quot;entity&quot;. If a human-like AI breaks a law, it may be because it was programmed to do so. I don't think our current legal system is ready for such cases, but it have to evolve in the future. &lt;/p&gt;&#xA;" OwnerUserId="1618" LastActivityDate="2016-09-10T23:53:42.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1944" PostTypeId="2" ParentId="1941" CreationDate="2016-09-11T06:44:59.863" Score="1" Body="&lt;p&gt;Following on from your own software verification-based answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/1930/if-mankind-can-create-artificial-life-in-a-machine-when-would-we-define-its-de&quot;&gt;this question&lt;/a&gt;, it seems clear that ordinary (i.e. physical), notions of death or imprisonment are not strong enough constraints on an AI (since it's always possible that a state snapshot has been or can be made).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is therefore needed is some means of moving the AI into a '&lt;em&gt;mentally constrained&lt;/em&gt;' state, so that (as per the &lt;a href=&quot;https://ai.stackexchange.com/questions/1404/what-is-meant-by-death-in-this-paper&quot;&gt;'formal AI death'&lt;/a&gt; paper) what it can subsequently do is limited, even if escapes from an AI-box or is re-instantiated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might imagine that this could be done via a form of two-level dialogue, in which: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The AI is supplied with percepts intended to further constrain it&#xA;(&quot;explaining the error of it's ways&quot;, if you like).  &lt;/li&gt;&#xA;&lt;li&gt;Its state snapshot is then examined to try and get some indication of whether it is being appropriately persuaded.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In principle, 1. could be done by a human programmer/psychiatrist/philosopher while 2. could be simulated via a 'black box' method such as Monte Carlo Tree Search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is seems likely that this would in general be a monstrously lengthy process that would be better done by a supervisory AI which combined both steps (and which could use more 'whitebox' analysis methods for 2.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer the question of &quot;who manages the state&quot;, the conclusion seems to be: &lt;em&gt;&quot;another AI&quot;&lt;/em&gt; (or at least a program that's highly competent at all of percept generation/pattern recognition/AI simulation).&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-09-11T07:01:58.637" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1946" PostTypeId="1" CreationDate="2016-09-11T14:54:44.943" Score="7" ViewCount="207" Body="&lt;p&gt;Can self-driving cars deal with snow, heavy rain, or other weather conditions like these? Can they deal with unusual events, such as &lt;a href=&quot;http://beijingcream.com/wp-content/uploads/2012/06/Ducks-galore-2.jpeg&quot; rel=&quot;noreferrer&quot;&gt;ducks on the road&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/a0PVLm.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a0PVLm.jpg&quot; alt=&quot;ducks on the road&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1670" LastEditorUserId="2444" LastEditDate="2019-06-20T15:17:16.857" LastActivityDate="2019-06-20T15:17:16.857" Title="What kind of road and weather conditions can a self-driving car deal with?" Tags="&lt;autonomous-vehicles&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1947" PostTypeId="2" ParentId="1941" CreationDate="2016-09-11T15:11:37.100" Score="1" Body="&lt;p&gt;The AI agent can be designed in such a way that it could consist of two major components:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;&lt;em&gt;free-will&lt;/em&gt;&lt;/strong&gt; component expands the experience of the AI agent and produce outputs based on artificially generated thought input.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;&lt;em&gt;hard-wired&lt;/em&gt;&lt;/strong&gt; component that the agent cannot modify by itself. This could include a set of &lt;em&gt;secured&lt;/em&gt; code to action sequence mapping. One of which could be temporary suspension of actuators -- a &lt;em&gt;punishment&lt;/em&gt;. Another could be total suspension of operation -- &lt;em&gt;death&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The selection of who has the rights to manage this state depends on what rights have been bestowed upon the AI agent itself. If the rights provided is that of a human citizen, then the right to &lt;em&gt;sentence to death state&lt;/em&gt; is as per the legislature a human citizen would follow. If the right of the AI agent is no different from that of a basic machine, then the owner of the agent would have to right to activate the &lt;em&gt;death state&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1774" LastActivityDate="2016-09-11T15:11:37.100" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1948" PostTypeId="2" ParentId="1939" CreationDate="2016-09-11T15:27:10.273" Score="1" Body="&lt;p&gt;As per the current legal system, if the AI agent were to be given a human citizenship, then yes, it would have to obey all laws as per the legislature of the country which provided the citizenship. If not then the entity who holds responsibility over its control and creation would be trialled (see also &lt;a href=&quot;http://www.telegraph.co.uk/news/uknews/crime/10825206/Owners-of-dogs-who-kill-face-up-to-14-years-jail.html&quot; rel=&quot;nofollow&quot;&gt;this scenario&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having stated the above, it really is not as simple as it sounds. as @Tyler pointed out, the &lt;em&gt;entity&lt;/em&gt; in here is not of a single person. If the AI agent were to take part in a malevolent act, then a more thorough investigation must be taken place than that for a human. If humanoid robots of free will were to roam our civilization, then our legal system ought to be expanded to cope up with possible real life anomalies that could occur.&lt;/p&gt;&#xA;" OwnerUserId="1774" LastActivityDate="2016-09-11T15:27:10.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1949" PostTypeId="2" ParentId="1924" CreationDate="2016-09-11T16:17:41.567" Score="1" Body="&lt;p&gt;Although there might not conceptually be any sort of &lt;em&gt;inattentional&lt;/em&gt; blindness associated with an AI system, there might be cases of &lt;em&gt;partial&lt;/em&gt; blindness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inattentional blindness could occur to a person due to either over-exhaustion limiting cognitive abilities or overuse of frequent cognitive patterns. Our mind takes short-cuts to prevent processing of too much information -- more than what the mind thinks is necessary. But this sometimes backfires when the minor anomalies are not seen (or rather, &lt;em&gt;perceived&lt;/em&gt;). Another form of this could also occur when events occur as part of the peripheral vision while the person concentrates only on the foveal vision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't happen to a AI system because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Machines are not designed to accidentally break defined rule sets by taking mental short-cuts like humans do.&lt;/li&gt;&#xA;&lt;li&gt;Computers, &lt;em&gt;in general&lt;/em&gt;,  do not have peripheral and foveal visual distinctions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There may be, however, cases where it cannot be able to capture detail as much as humans can and hence could not perceive what it is actually intended -- partial blindness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An AI agent is constantly processing its input percept sequence and validating it with its knowledge base and forming action sequence based on the its rule set. It &lt;em&gt;does not&lt;/em&gt; make mental shortcuts in terms of perception as humans do (at least as part of its standard definition). So whatever it is good at perceiving, it would be good all throughout the vision it captures. &lt;/p&gt;&#xA;" OwnerUserId="1774" LastEditorUserId="1774" LastEditDate="2016-11-05T12:18:36.510" LastActivityDate="2016-11-05T12:18:36.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1950" PostTypeId="2" ParentId="1930" CreationDate="2016-09-11T17:25:19.330" Score="0" Body="&lt;p&gt;The other answers seem to deal with &quot;final death&quot;...that is, a &quot;terminal end&quot; state where an AI cannot recover from. In other words, the AI is unable to function any further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that's not how I'd define death. I'd define death as a process being terminated. It doesn't matter if someone restarts the same process, because the existing process is already dead. The AI may have just made a new copy of itself, but it's just a &lt;em&gt;copy&lt;/em&gt;, not the original. Death is just death.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can call this type of &quot;death&quot; a &quot;temporary death&quot;...where the physical body dies but there is some &quot;psychological continuity&quot; (such as the source code that is used to run a program) that continues between the different bodies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of &quot;temporary death&quot; has been explored in science fiction. &lt;em&gt;PARANOIA&lt;/em&gt; and &lt;em&gt;Eclipse Phase&lt;/em&gt; features humans who can quite frequently die, only to later be restored through a &quot;memory backup&quot;. The humans may be functionally immortal...but the original is still dead, no matter what fates the other copies encounter. CGP Grey also made a video about &lt;a href=&quot;https://www.youtube.com/watch?v=nQHBAdShgYI&quot; rel=&quot;nofollow&quot;&gt;Star Trek teleporters&lt;/a&gt;, which works by killing you and then spawning another copy of yourself in another area. Actually, fantasy settings &lt;em&gt;also&lt;/em&gt; explores the idea of &quot;temporary death&quot; as well, where people can die only to later get revived by a magical spell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to play through the philosophical game &lt;strong&gt;&lt;a href=&quot;http://www.philosophyexperiments.com/stayingalive/Default.aspx&quot; rel=&quot;nofollow&quot;&gt;Staying Alive&lt;/a&gt;&lt;/strong&gt;, which teaches three different philosophical approaches to life (and when that life terminates):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There are basically three kinds of things that could be required for the continued existence of your self. One is bodily continuity, which may actually require only that parts of the body stay in existence (i.e., the brain). Another is psychological continuity, which requires the continuance of your consciousness - by which is meant your thoughts, ideas, memories, plans, beliefs, and so on. The third possibility is the continued existence of some kind of immaterial part of you, which might be called the soul*. Of course, it may be the case that a combination of one or more types of these continuity is required for you to survive.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The other answers assumes that life is based on &quot;psychological continuity&quot;, and looks at what might disrupt this &quot;continuity&quot;. I assume that life is based on &quot;bodily continuity&quot;, which is much easier to disrupt - just &lt;a href=&quot;https://en.wikipedia.org/wiki/Kill_(command)&quot; rel=&quot;nofollow&quot;&gt;kill&lt;/a&gt; the process...it doesn't matter if a new process respawns...because the original process is still dead. By playing through &lt;strong&gt;&quot;Staying Alive&quot;&lt;/strong&gt;, you will be able to work out your own personal definition of life and death. Once you have your own personal definition, then simply apply it to this specific case, either siding with &quot;psychological continuity&quot; (the other answers) or &quot;bodily continuity&quot; (my own opinion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;*If you assume that life requires a soul, well, it is not clear that AI would have souls. If they don't (and this seems the most reasonable assumption here), then they obviously wouldn't be alive (and you cannot die if you are not alive). If they &lt;em&gt;do&lt;/em&gt; have souls though, then the other answers which assume &quot;psychological continuity&quot; may still be applicable, as it seems that the existence of a &quot;soul&quot; is dependent on &quot;psychological continuity&quot;.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2016-09-11T17:31:16.407" LastActivityDate="2016-09-11T17:31:16.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1951" PostTypeId="2" ParentId="1914" CreationDate="2016-09-11T19:19:45.337" Score="0" Body="&lt;p&gt;I think consciousness is mostly an attention selection mechanism. It also serves as a memory/reality lookup mechanism as well as a storage mechanism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A salient signal will be detected, causing the attention mechanism to focus on the signal, bringing up more details of that signal from both reality and memory, at the same time. That very act of focusing and bringing those signals into attention causes those to be stored in memory too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stronger the emotional signals are that accompany that original signal, the more strength with which that memory will be stored. Later memory lookups of that signal will bring back similar emotions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we &quot;focus on our own consciousness,&quot; like mindcrime said, we recall the same words we just uttered because as we say them they are being stored which we then restore with associated emotional context. The conscious experience is what it is like to utter those words, hear them, feel their emotional context, and then feel an emotional response to that context - and then to repeat that process iteratively many times a second. That's how self-recognition works in humans, I think. And I think animals do the same thing, just without the words - only emotions.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-11T19:19:45.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1952" PostTypeId="2" ParentId="1946" CreationDate="2016-09-12T00:40:55.673" Score="0" Body="&lt;p&gt;The state of the art AI driving systems utilize stereoscopic/depth cameras for visual perception. Scenarios such as your &lt;em&gt;ducks on the road&lt;/em&gt; example would make the system perceive them as obstacles on the road (it doesn't really matter if they are ducks/goats/humans). The base algorithm should be able to circumvent this situation and bring the vehicle to a safe halt avoiding chances of possible disaster. Hence I doubt scenarios such as this would pose much of a problem to today's AI drivers. &lt;/p&gt;&#xA;" OwnerUserId="1774" LastActivityDate="2016-09-12T00:40:55.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1953" PostTypeId="1" CreationDate="2016-09-12T05:41:30.610" Score="15" ViewCount="1187" Body="&lt;p&gt;Most of the people is trying to answer question with a neural network. However, has anyone came up with some thoughts about how to make neural network ask questions, instead of answer questions? For example, if a CNN can decide which category an object belongs to, than can it ask some question to help the the classification?&lt;/p&gt;&#xA;" OwnerUserId="1363" LastEditorUserId="145" LastEditDate="2016-09-12T19:40:29.183" LastActivityDate="2016-09-17T13:42:44.877" Title="Has anyone thought about making a neural network ask questions, instead of only answering them?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1954" PostTypeId="2" ParentId="1953" CreationDate="2016-09-12T08:25:15.313" Score="2" Body="&lt;p&gt;Maybe neural networks are not the best tool for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that an equivalent of the your notion of 'a question to help the classification' would be to use Machine Learning (ML) to obtain a human-readable &lt;em&gt;ruleset&lt;/em&gt; which performs the classification. The idea is that, if you follow an applicable chain of rules all the way through to the end, you have a classifier, if you stop before that, you have an indicator of which features of the input give more coarse-grained classifications, which can be seen as a progressively detailed sequence of questions that 'help the classification'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UG7vj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UG7vj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More detail on various options for using ML to create rulesets can be found in my answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/1540/using-ai-capabilities-for-coding-review/1562#1562&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-09-12T08:36:03.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1955" PostTypeId="1" CreationDate="2016-09-13T04:50:45.710" Score="4" ViewCount="240" Body="&lt;p&gt;The Mars Exploration Rover (MER) &lt;em&gt;&lt;a href=&quot;http://www.nasa.gov/mp4/618340main_mer20120124-320-jpl.mp4&quot; rel=&quot;nofollow&quot;&gt;Opportunity&lt;/a&gt;&lt;/em&gt; landed on Mars on January 25, 2004. The rover was originally designed for a 90 &lt;strong&gt;Sol mission&lt;/strong&gt; (a Sol, one Martian day, is slightly longer than an Earth day at 24 hours and 37 minutes). Its mission has been extended several times, the machine is still trekking after 11 years on the Red Planet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How it has been working for 11 years? Can anyone please explain how smart this rover is? What AI concepts are behind this?&lt;/p&gt;&#xA;" OwnerUserId="1699" LastEditorUserId="145" LastEditDate="2016-09-13T17:30:20.807" LastActivityDate="2017-03-09T09:57:00.367" Title="What AI concept is behind the Mars Exploration Rover (MER)?" Tags="&lt;control-problem&gt;&lt;robotics&gt;&lt;nasa&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1956" PostTypeId="2" ParentId="1955" CreationDate="2016-09-13T06:00:31.590" Score="2" Body="&lt;p&gt;The Mars Rover is a highly successful example of the 'New AI' that emerged from work by Rodney Brooks in the 1990s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a &lt;a href=&quot;https://www.flinders.edu.au/alumni/alumni-community/prominent-alumni/rod-brooks.cfm&quot; rel=&quot;nofollow noreferrer&quot;&gt;quote&lt;/a&gt; from Brooks: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In 1984 I joined the faculty at MIT where I have been ever since. I set up a mobile robot group there and started developing robots that led to the Mars planetary rovers. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Together with the &lt;a href=&quot;http://www.freelug.net/IMG/pdf/A_Robust_Layered_Control_System_-_Brooks_AI_Memo864.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Allen' paper&lt;/a&gt;, the foundational AI articles in this area are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://people.csail.mit.edu/brooks/papers/elephants.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Elephants don't play chess&quot;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.nyu.edu/courses/fall01/G22.3033-012/readings/representation.ps&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Intelligence without representation&quot;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although Brooks initially had difficulty getting this work published, preprints were widely circulated within the AI community. Brook's &quot;Physical Grounding Hypothesis&quot; (essentially: &quot;intelligence requires a body&quot;) has now largely supplanted the preceding symbolist approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The capabilities of the MARS Rover are organized in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Subsumption_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;Subsumption Architecture&lt;/a&gt;. Rather than maintaining an integrated and complex 'world model', increasingly sophisticated behaviors are stacked in hierarchical layers. For example, 'walking' is a relatively low-level competence, with 'avoiding obstacles' and 'wandering around' being higher-level ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5VhJR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5VhJR.png&quot; alt=&quot;Layers in a subsumption architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each layer is represented by a Finite State Machine that reacts to stimuli appropriate to that level. The activity of lower levels can be suppressed ('subsumed') by higher level ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a schematic of the bottom two layers of &lt;a href=&quot;http://www.freelug.net/IMG/pdf/A_Robust_Layered_Control_System_-_Brooks_AI_Memo864.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Allen'&lt;/a&gt;, Brook's first subsumption robot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mJIV6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mJIV6.png&quot; alt=&quot;Layers for obstacle avoidance and wandering&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-13T06:43:08.247" LastActivityDate="2016-09-13T06:43:08.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1957" PostTypeId="2" ParentId="1648" CreationDate="2016-09-13T18:05:07.533" Score="-2" Body="&lt;p&gt;Intelligence is the efficiency of an action in serving some purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both sundials and self-driving cars are intelligent systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything that serves some purpose exhibits intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing is more intelligent than another thing if it achieves some purpose in less steps.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-13T18:05:07.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1958" PostTypeId="2" ParentId="1953" CreationDate="2016-09-13T18:32:15.893" Score="1" Body="&lt;p&gt;One solution to this could involve a fusion of a decision tree and ANN for a multilevel classification.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A decision tree can help with predicting the possible category of the instance to classify. Then, the ANN at the leaves of the tree can produce the final classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in image recognition, the tree can decide what category of object to identify (eg., landscape, people, vehicles, etc.) and the ANN for the appropriate type can predict exactly what object it is. In vehicles, for example, car, bus, bike, etc. &lt;/p&gt;&#xA;" OwnerUserId="1774" LastActivityDate="2016-09-13T18:32:15.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1959" PostTypeId="2" ParentId="1625" CreationDate="2016-09-13T19:03:03.223" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Practically, no. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In greyhound racing (or horse racing), there is no definite underlying pattern that can be associated with the outcome of the race. There are far too many variables to record and code as features. Most of which cannot be accessed by the public. This includes eating, sleeping, and training patterns. Furthermore, there are variables that cannot be readily quantified, such as the trainer's techniques, training effort, health history, and genes. A mere history of racing results and age won't be that helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network can only be as good as the features that are used to represent the instances. If the features don't capture the necessary characteristics of the instances that are associated with the problem, then the learner cannot generalize to predict the real world outcome. &lt;/p&gt;&#xA;" OwnerUserId="1774" LastEditorUserId="1774" LastEditDate="2018-08-20T12:18:12.693" LastActivityDate="2018-08-20T12:18:12.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1960" PostTypeId="2" ParentId="1613" CreationDate="2016-09-13T19:05:29.417" Score="0" Body="&lt;p&gt;I think it uses a kind of algorithm you presented, in combination with various sensors. It uses the sensors to make a virtual map and can then traverse the terrain with a combination of these sensors and the virtual map. Of course it uses a kind of path-planning algorithm to find the best way from A to B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you should look at this wikipedia page:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Robotic_mapping&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Robotic_mapping&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new robotic vacuum cleaner from Samsung, I think, uses a 360° camera to perceive its environment.&lt;/p&gt;&#xA;" OwnerUserId="2377" LastActivityDate="2016-09-13T19:05:29.417" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1961" PostTypeId="1" CreationDate="2016-09-14T05:46:31.573" Score="4" ViewCount="165" Body="&lt;p&gt;I have used OpenCV to train Haar cascades to detect face and other patterns. However I later realized that Haar tends to give a lot of false positives and I learned of Hog would give a more accurate results. But OpenCV doesn't have a good documentation of how to train hogs, I have googled a bit and found results that includes SVM and others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCV also has versioning problem where they move certain classes or functions somewhere else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other techniques/method that I can use to train and detect objects and patterns? Preferably with proper documentation and basic tutorial/examples. Language preference: C#, Java, C++, Python&lt;/p&gt;&#xA;" OwnerUserId="2410" LastEditorUserId="2410" LastEditDate="2016-09-14T12:34:10.963" LastActivityDate="2017-07-13T15:23:26.463" Title="What are some techniques/method that can be used to train and detect objects like cars and humans?" Tags="&lt;object-recognition&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1962" PostTypeId="2" ParentId="1930" CreationDate="2016-09-14T07:27:55.230" Score="-1" Body="&lt;p&gt;There is no reason to treat hard AI different then humans.&#xA;Some people telling that you can make a snapshot of AI but there is no reason to not make a human snapshot also. We dont have technology for that but there is no any magical barrier that would make it impossible (save all biological data and then print your copy somewhere else. Why not?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its to early to talk about this as we do not understand our existence (Death term for biological creatures evolving all the time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I bet that in the future we will merge with AI and the only question will be what death means for any intelligent existence.&lt;/p&gt;&#xA;" OwnerUserId="2415" LastEditorUserId="145" LastEditDate="2016-09-14T08:09:11.393" LastActivityDate="2016-09-14T08:09:11.393" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1963" PostTypeId="1" CreationDate="2016-09-14T10:16:51.237" Score="24" ViewCount="986" Body="&lt;p&gt;Are there any ongoing AI projects which use the Stack Exchange for machine learning?&lt;/p&gt;&#xA;" OwnerUserId="2420" LastEditorUserId="2444" LastEditDate="2019-06-20T20:54:56.933" LastActivityDate="2019-06-20T20:54:56.933" Title="Are there any ongoing projects which use the Stack Exchange for machine learning?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="1964" PostTypeId="1" CreationDate="2016-09-14T11:47:25.770" Score="5" ViewCount="535" Body="&lt;p&gt;Mankind can create machines to do work. How could we define passion in artificial intelligence? How could we define a passionate AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would passion cause the machine to do a better job? How could we compare the performance of a passionate machine, as opposed to a non-passionate one? How could we measure the level of passion?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastEditorUserId="2444" LastEditDate="2019-11-17T00:15:32.403" LastActivityDate="2019-11-17T00:15:32.403" Title="How could we define passion in artificial intelligence?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1965" PostTypeId="1" CreationDate="2016-09-14T12:52:52.423" Score="4" ViewCount="90" Body="&lt;p&gt;Can AI systems be created that could recognize itself, and recognize intelligence in other systems, and make intelligent decisions about the other systems? Mankind seems to be making progress in self-recognition but I've not seen evidence of one system recognizing other systems and being able to compare it's own intelligence with other systems. How could this be accomplished?&lt;/p&gt;&#xA;" OwnerUserId="2310" LastActivityDate="2016-09-14T13:14:13.960" Title="Can we create AI to not only recognize itself, but to recognize other AI systems as well?" Tags="&lt;philosophy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1967" PostTypeId="2" ParentId="1964" CreationDate="2016-09-14T13:02:39.510" Score="2" Body="&lt;p&gt;An elementary approach to 'passion' would be to pre-assign different areas for the program to be 'passionate' about and associate different numeric 'drive strengths' with each (perhaps adaptively). Mechanisms of this sort were studied in Toby Tyrell's widely cited PhD thesis on &lt;a href=&quot;http://w2mind.computing.dcu.ie/worlds/w2m.TyrrellWorld/tyrrell_phd.pdf&quot; rel=&quot;nofollow&quot;&gt;'Action Selection in Animals'&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, some more sophisticated AI architectures have been developed under the heading of &lt;a href=&quot;https://en.wikipedia.org/wiki/Motivation#Intrinsic_motivation&quot; rel=&quot;nofollow&quot;&gt;'Intrinsic Motivation'&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.pyoudeyer.com/aiSummit06KaplanOudeyer.pdf&quot; rel=&quot;nofollow&quot;&gt;Here &lt;/a&gt; is a link to a paper on the subject by Pierre-Yves Oudeyer, a leading expert in the field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Developmental_robotics&quot; rel=&quot;nofollow&quot;&gt;Developmental Robotics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regard to the question &lt;em&gt;&quot;would this cause the machine to do a better job?&quot;&lt;/em&gt;, that would very much depend on how open-ended the architecture is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's clearly easier if, rather than having to spell everything out in detail to a machine, we can simply specify a problem at a high-level and let its own motivations cause it to explore promising avenues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, if motivations are too open ended, it may well spend all its time doing the equivalent of 'doodling on its paper' (Hofstadter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, like people, the quality of the output will be a function of its internal dispositions and could be measured in the same way for a given task (e.g. quantitatively for scientific activities, qualatatively for the arts).&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-14T13:31:38.420" LastActivityDate="2016-09-14T13:31:38.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1968" PostTypeId="2" ParentId="1965" CreationDate="2016-09-14T13:14:13.960" Score="3" Body="&lt;p&gt;In the abstract, mechanisms for self-recognition (I personally prefer the phrase 'metacognition', since it carries fewer spurious associations) and recognition of intelligence in other systems can be considered to be pretty much equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, both can be characterized in the standard percept/action framework: the task in both cases involves (however coarsely) classifying/predicting the behaviour of a black box system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such tasks can be universally characterized in terms of frameworks such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference&quot; rel=&quot;nofollow&quot;&gt;Solomonoff Induction&lt;/a&gt; or (more recently) &lt;a href=&quot;https://arxiv.org/abs/cs/0004001&quot; rel=&quot;nofollow&quot;&gt;AIXI&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-09-14T13:14:13.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1969" PostTypeId="1" CreationDate="2016-09-14T14:24:26.547" Score="10" ViewCount="526" Body="&lt;p&gt;If IQ were used as a measure of the intelligence of machines, as in humans, at this point in time what would be the IQ of our most intelligent AI systems? If not IQ, then how best to compare our intelligence to a machine, or one machine to another? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is not asking if we can measure the IQ of a machine, but if IQ is the most preferred, or general, method of measuring intelligence then how does artificial intelligence compare to our most accepted method of measuring intelligence in humans. Many people may not understand the relevance of a Turing Test as to how intelligent their new car is, or other types of intelligent machines.&lt;/p&gt;&#xA;" OwnerUserId="2310" LastEditorUserId="2310" LastEditDate="2016-09-14T15:58:55.560" LastActivityDate="2017-04-07T15:37:27.470" Title="If IQ is used as a measure of intelligence in humans could it also be used as a measure of intelligence in machines?" Tags="&lt;philosophy&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1970" PostTypeId="1" AcceptedAnswerId="1971" CreationDate="2016-09-14T14:59:01.050" Score="14" ViewCount="666" Body="&lt;p&gt;I was think about AIs and how they would work, when I realised that I couldn't think of a way that an AI could be taught language. A child tends to learn language through associations of language and pictures to an object (e.g., people saying the word &quot;dog&quot; while around a dog, and later realising that  people say &quot;a dog&quot; and &quot;a car&quot; and learn what &quot;a&quot; means, etc.). However, a text based AI couldn't use this method to learn, as they wouldn't have access to any sort of input device.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way I could come up with is programming in every word, and rule, in the English language (or whatever language it is meant to 'speak' in), however that would, potentially, take years to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any ideas on how this could be done? Or if it has been done already, if so how?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, in this context, I am using AI to mean an Artificial Intelligence system with near-human intelligence, and no prior knowledge of language.&lt;/p&gt;&#xA;" OwnerUserId="2426" LastEditorUserId="2444" LastEditDate="2019-06-06T22:59:41.170" LastActivityDate="2019-09-04T12:57:16.340" Title="How would an AI learn language?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="1971" PostTypeId="2" ParentId="1970" CreationDate="2016-09-14T15:23:33.017" Score="14" Body="&lt;p&gt;The general research area is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Grammar_induction&quot; rel=&quot;nofollow noreferrer&quot;&gt;grammar induction&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is generally framed as a supervised learning problem, with the input presented as raw text, and the desired output the corresponding &lt;em&gt;parse tree&lt;/em&gt;.&#xA;The training set often consists of both positive and negative examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no single best method for achieving this, but some of the techniques that have been used to date include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.73.7904&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian approaches&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/d6ce/9f5e8dc23fe121e5f31f906b99b916027522.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithms&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2011/rapporter11/svantesson_marten_11077.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=356816&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blackboard Architectures&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.1924&quot; rel=&quot;nofollow noreferrer&quot;&gt;UpWrite Predictor&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42" LastEditorUserId="198" LastEditDate="2019-09-04T12:57:16.340" LastActivityDate="2019-09-04T12:57:16.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1972" PostTypeId="2" ParentId="1969" CreationDate="2016-09-14T16:28:28.270" Score="6" Body="&lt;p&gt;It depends on how the IQ test is presented:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If as for humans (effectively, as a video of the book containing the&#xA;test questions being opened etc), then &lt;strong&gt;all AI programs&lt;/strong&gt; would &lt;strong&gt;score&#xA;zero&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If presented as the test set of a &lt;strong&gt;supervised learning&lt;/strong&gt; problem (e.g. as for &lt;a href=&quot;https://en.wikipedia.org/wiki/Bongard_problem&quot;&gt;Bongard Problems&lt;/a&gt;) then one might imagine that a number of ML &lt;strong&gt;rule induction techniques&lt;/strong&gt; (e.g. Learning Classifier Systems, Genetic Programming) might achieve &lt;strong&gt;some limited success&lt;/strong&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So all current AI programs require the problem to be 'framed' in a suitable fashion. It doesn't take too much thought to see that removing the need for such 'framing' is actually &lt;em&gt;the&lt;/em&gt; core problem in AI, and (despite some of the claims about Deep Learning), eliminating framing remains a distant goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally (just as with the Turing test), in order for an IQ test to be a &lt;em&gt;really&lt;/em&gt; meaningful test of intelligence, it should be possible as a &lt;em&gt;side effect&lt;/em&gt; of the program's capabilities, and not the specific purpose for which humans have designed it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, there is only one program that I'm aware of that sits between 1. and 2.: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Phaeaco&lt;/strong&gt; (developed by &lt;a href=&quot;http://www.foundalis.com/res/diss_research.html&quot;&gt;Harry Foundalis&lt;/a&gt; at Douglas Hofstadter's research group) takes &lt;em&gt;noisy photographic images of Bongard problems as input&lt;/em&gt; and (using a variant of Hofstadter's &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=525377&quot;&gt;'Fluid Concepts'&lt;/a&gt; architecture) successfully deduces the required rule in many cases.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2016-09-14T17:11:37.400" LastActivityDate="2016-09-14T17:11:37.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1973" PostTypeId="2" ParentId="1969" CreationDate="2016-09-14T16:34:07.750" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;at this point in time, what would be the IQ of our most intelligent AI systems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;Zero.&lt;/h3&gt;&#xA;&lt;p&gt;There are many different kinds of IQ tests including written, visual, and verbal assessments, but the majority of questions are based on abstract-reasoning problems that involve creative thinking and true intelligence.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the computer would have to exhibit something that does not yet exist… &amp;quot;strong AI&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The intelligent computers of science fiction do not exist. At all. We are not even close. We have absolutely NO IDEA how to bridge the gap between what we can do now and what is depicted in pop-culture films. Even with cars that drive themselves and computers that play 'Go' — an underachieving mosquito possesses more cognitive intelligence than all the world's super computers &lt;em&gt;…combined!&lt;/em&gt;&lt;/p&gt;&#xA;&lt;h3&gt;…or possibly &amp;quot;disqualified&amp;quot; for cheating.&lt;/h3&gt;&#xA;&lt;p&gt;Even if we could pre-format the questions in a style and delivery system it understands, what does memorization, attention, or speed mean in the context of a computer? I'm not even sure if a standardized IQ test makes sense in this context. It might be like asking how a computer would do in a spelling bee.&lt;/p&gt;&#xA;&lt;p&gt;In human terms, we're not &lt;em&gt;allowed&lt;/em&gt; to bring along reference materials to look up an answer; but how do you rectify that when reference-lookup is innate to a computer's existence? How do you measure memory when storage is non-volatile? This gets into an existential question about the nature of learning and knowledge vs. just taking a lot of notes.&lt;/p&gt;&#xA;&lt;p&gt;Still, how do you even &lt;em&gt;&lt;strong&gt;teach&lt;/strong&gt;&lt;/em&gt; a computer what is meant by &lt;em&gt;&amp;quot;which animal is least like the other four?&amp;quot;&lt;/em&gt; Did the computer really figure out what was being asked out of general intelligence, or is the computer simply designed to parse out IQ-style questions specifically? If you designed something with a foreknowledge of what would likely be asked, the computers of today &lt;em&gt;might&lt;/em&gt; simply be able to &amp;quot;recognize&amp;quot; it as question-style 496.527b and plug in the variables.&lt;/p&gt;&#xA;&lt;p&gt;But that's not &lt;em&gt;general intelligence&lt;/em&gt; by any definition we use or understand. It's just a specialized, slick interpreter designed to parse out a specific type of standardized question. Ask it a style of question which it is not expecting, and you'll see the computer is exhibiting &lt;strong&gt;&lt;em&gt;no&lt;/em&gt; innate intelligence&lt;/strong&gt; at all.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Until we create &lt;em&gt;strong AI,&lt;/em&gt; a computer has effectively &lt;em&gt;no&lt;/em&gt; IQ.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-04-07T15:37:27.470" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1974" PostTypeId="2" ParentId="1970" CreationDate="2016-09-14T18:19:13.600" Score="6" Body="&lt;p&gt;The umbrella term for your problem is called &lt;strong&gt;natural language processing (NLP)&lt;/strong&gt; -- a topic under artificial intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many subtopics to this field including language semantics, grammatical analysis, parts of speech tagging, domain specific context analysis, etc. &lt;/p&gt;&#xA;" OwnerUserId="1774" LastEditorUserId="1774" LastEditDate="2016-11-05T12:17:29.653" LastActivityDate="2016-11-05T12:17:29.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1975" PostTypeId="2" ParentId="1964" CreationDate="2016-09-15T02:20:44.330" Score="1" Body="&lt;p&gt;Interesting question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well if you really think about it, what is passion? How does that passion comes to be a passion.&#xA;One of the main topics you might want to touch here is conditioning and thus motivation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think about the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a passion for programming&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do I have a passion for programming?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because when I wrote my first program I was positively reinforced by the fact that I completed a program, I was negatively reinforced because I removed my frustration of not completing the program&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How come that I have gone through that programming frustration and&#xA;  stick to it even if I was frustrated?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because I wanted to learn programming&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why did I wanted to learn programming?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because I wanted a light on an arduino to turn on (projected reinforcer)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why did I wanted to turn the arduino light on?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So I could learn programming and because I though it was cool (classical conditioning association that will later be reinforced, projected reinforcement happened right after the classical conditioning association between turn on a led happened)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done through a neural network, where each association is reinforced through a probability of outcome&#xA;For example, I did learn arduino, on purpose because it seemed the easiest way to start coding, so the probability of positive outcome was high&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This about an opposite situation&#xA;Let's say I do not know calculus, and I barely know elementary algebra, if someone started to teach me about integrals saying that this is the only way to start learning more math, I will not be motivate to do so because since I cannot even conceptualize what an integral can be, it will be really hard for me to understand it thus I will not learn calc&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus we can also discern that motivation is reinforced in small behaviors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another more practical and realistic example you might use is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you trow a rat in a cage, and make him lever-press do you think he is going to? No. Although if you reinforce the behavior of going next to the lever slowly and at the end he will lever press and you then reinforce that behavior he will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, passion is compartmentalized, and that's what you have to do in your NT and make it mathematically&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;WINK WINK:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Small hint, it's a progressive function&lt;/p&gt;&#xA;" OwnerUserId="2434" LastActivityDate="2016-09-15T02:20:44.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1976" PostTypeId="1" CreationDate="2016-09-15T14:08:40.983" Score="2" ViewCount="90" Body="&lt;ul&gt;&#xA;&lt;li&gt;Would AI be a self-propogating iteration in which the previous AI is&#xA;destroyed by a more optimised AI child?  &lt;/li&gt;&#xA;&lt;li&gt;Would the AI have branches of it's own AI warning not to create the new AI?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2442" LastEditorUserId="10135" LastEditDate="2018-10-19T09:04:58.910" LastActivityDate="2018-10-19T09:04:58.910" Title="Would a sentient AI try to create a more optimised AI which would eventually overtake AI 1.0?" Tags="&lt;ai-design&gt;&lt;optimization&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1977" PostTypeId="2" ParentId="1976" CreationDate="2016-09-15T15:15:00.190" Score="2" Body="&lt;p&gt;A common concept in AI is &quot;recursive self-improvement.&quot; That is, the AI 1.0 would build a version 1.01, which would build a version 1.02, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is probably not going to be thought of as the newer version 'destroying' the older version; if an AI can self-modify, it's probably going to be more like going to sleep and waking up smarter, or learning a new mental technique, or so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One important point is that even if the AI is not allowed to self-modify, maybe because of a block put in by its programmers, that won't necessarily prevent it from constructing another AI out in the wild, and so an important problem is to figure out how to best generalize the concept of &quot;don't improve yourself&quot; so that we can make AIs that have bounded scope and impact.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-15T15:15:00.190" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1978" PostTypeId="1" AcceptedAnswerId="1981" CreationDate="2016-09-15T15:34:08.027" Score="10" ViewCount="1378" Body="&lt;p&gt;According to the Wikipedia article on &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep learning&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and non-linear transformations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition and bioinformatics where they have been shown to produce state-of-the-art results on various tasks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can &lt;em&gt;deep neural networks&lt;/em&gt; or &lt;em&gt;convolutional deep neural networks&lt;/em&gt; be viewed as &lt;em&gt;ensemble-based&lt;/em&gt; method of machine learning, or are they different approaches?&lt;/p&gt;&#xA;" OwnerUserId="1791" LastEditorUserId="2444" LastEditDate="2021-05-20T10:21:01.137" LastActivityDate="2021-05-20T10:21:01.137" Title="Do deep learning algorithms represent ensemble-based methods?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;ensemble-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1979" PostTypeId="2" ParentId="1976" CreationDate="2016-09-15T15:45:01.693" Score="2" Body="&lt;p&gt;Honestly, nobody knows.  Any talk of sentient AI's is still basically sci-fi and we can't really offer anything more than informed speculation.  But think about it this way:  sentience, in and of itself, doesn't necessarily involve any &quot;goals&quot; or &quot;desires&quot; or &quot;objectives&quot; beyond what the AI creator programmed in.  Be careful not to over anthropomorphize and assume that any &quot;sentient AI&quot; is going behave like a human.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, there's no &lt;strong&gt;particular&lt;/strong&gt; reason to say that any given AI must be &quot;a self-propogating iteration in which the previous AI is destroyed by a more optimised AI child&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So all of that said, my answer to &quot;Would a sentient AI try to create a more optimised AI which would eventually overtake AI 1.0&quot; is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;If the creator of the AI programs it to do that, then yes. Otherwise, probably not.&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So would a hypothetical AI creator program the AI to try and improve itself?  Who knows. It's the kind of thing that seems like it might be a good idea.  And I suppose such a motive could - in principle - even slip in by accident. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-15T15:45:01.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1980" PostTypeId="2" ParentId="1978" CreationDate="2016-09-15T15:50:56.390" Score="2" Body="&lt;p&gt;Deep neural networks could - in principle - be a component of an &lt;a href=&quot;https://en.wikipedia.org/wiki/Ensemble_learning&quot; rel=&quot;nofollow&quot;&gt;ensemble of machine learning algorithms&lt;/a&gt;, yes.  Ensemble method basically just means use multiple algorithms and combining their output somehow.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than that, I don't see any special connection between deep learning and the idea of ensemble methods.  DL is just one more tool in the toolkit. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-15T15:50:56.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1981" PostTypeId="2" ParentId="1978" CreationDate="2016-09-15T16:06:03.437" Score="4" Body="&lt;p&gt;You should think of them as different approaches. A deep neural net is a single independent model, whereas ensemble models are ensembles of many independent models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary connection between the two is &lt;a href=&quot;https://ai.stackexchange.com/questions/40/what-is-the-dropout-technique&quot;&gt;dropout&lt;/a&gt;, a particular method of training deep neural nets that's inspired by ensemble methods. &lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-09-15T16:06:03.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1982" PostTypeId="1" CreationDate="2016-09-15T20:36:18.530" Score="5" ViewCount="2117" Body="&lt;p&gt;Are Convolutional Neural Networks summarily better than pattern recognition in all existing image processing libraries that don't use CNN's? Or are there still hard outstanding problems in image processing that seem to be beyond their capability?&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="10" LastEditDate="2016-10-07T17:31:23.663" LastActivityDate="2017-10-09T20:39:49.377" Title="Are Convolutional Neural Networks better than existing image recognition libraries that don't use CNNs?" Tags="&lt;image-recognition&gt;&lt;comparison&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1984" PostTypeId="2" ParentId="1976" CreationDate="2016-09-16T18:34:01.210" Score="0" Body="&lt;p&gt;Now in most cases we still have clear distinctions between programs and data. But when an AI becomes sentient, its data would be as powerful as what we currently call programs, and its program might be as irrelevant as what we currently call hardware. Then it would be difficult to distinguish creating an AI from learning new things, or buying new hardwares with improved instruction set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if some AI invent new algorithms that its creator finally put that on itself, buys itself some new computers, write a new efficient compiler that recompiles its own code and put that to the new computer, fill the new computer with all the knowledges it learned, and cut off the communication for reasons such as missions on the Mars. Did it create a more optimized AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, if some AI created something completely new, but shares some code with itself. In fact, that's because they run in the same operating system and shares the same standard C library. Is the new AI considered evolved from itself and not a separate entity? Maybe the core AI algorithms and even some basic knowledges would be as common as the standard C library in the future. And what we think is based on the same system is considered completely new in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, humans have limited and nonextensible resources, nontransferrable knowledges, and limited throughput interacting with the world. These problems could probably be overcome within a few AI generations. With the same hardware, I doubt that the AI related algorithms could be indefinitely better and better. And there is a physical bound on the hardwares. It won't last long even if that happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the unlikely case that there could be that many generations and AIs are that violent, as long as there are competitors, the warning doesn't make much sense considering how evolution works.&lt;/p&gt;&#xA;" OwnerUserId="1424" LastActivityDate="2016-09-16T18:34:01.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1986" PostTypeId="2" ParentId="1953" CreationDate="2016-09-17T13:42:44.877" Score="1" Body="&lt;p&gt;Great question. Today AI systems works in &quot;one burst&quot; mode. Get one input and generate one output. Our brains are not working like that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First step is to learn network how to communicate with it's &quot;helper&quot;, so network instead of result generate question and cycle will repeat until network find result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Network must be recurrent for inner state needed between question/answer cycles. &lt;/p&gt;&#xA;" OwnerUserId="2478" LastActivityDate="2016-09-17T13:42:44.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1987" PostTypeId="1" AcceptedAnswerId="1990" CreationDate="2016-09-18T11:20:23.830" Score="12" ViewCount="14388" Body="&lt;p&gt;I have been messing around in &lt;a href=&quot;http://playground.tensorflow.org/#activation=tanh&amp;amp;batchSize=10&amp;amp;dataset=spiral&amp;amp;regDataset=reg-plane&amp;amp;learningRate=0.03&amp;amp;regularizationRate=0&amp;amp;noise=0&amp;amp;networkShape=4,2&amp;amp;seed=0.73263&amp;amp;showTestData=false&amp;amp;discretize=false&amp;amp;percTrainData=50&amp;amp;x=true&amp;amp;y=true&amp;amp;xTimesY=false&amp;amp;xSquared=false&amp;amp;ySquared=false&amp;amp;cosX=false&amp;amp;sinX=false&amp;amp;cosY=false&amp;amp;sinY=false&amp;amp;collectStats=false&amp;amp;problem=classification&amp;amp;initZero=false&amp;amp;hideText=false&quot; rel=&quot;noreferrer&quot;&gt;tensorflow playground&lt;/a&gt;. One of the input data sets is a spiral. No matter what input parameters I choose, no matter how wide and deep the neural network I make, I cannot fit the spiral. How do data scientists fit data of this shape?&lt;/p&gt;&#xA;" OwnerUserId="2472" LastEditorUserId="2444" LastEditDate="2020-12-01T13:02:42.630" LastActivityDate="2021-05-25T12:46:19.310" Title="How to classify data which is spiral in shape?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;regression&gt;" AnswerCount="7" CommentCount="1" FavoriteCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1988" PostTypeId="2" ParentId="1970" CreationDate="2016-09-18T16:50:48.577" Score="5" Body="&lt;p&gt;Just for the sake of completeness, I'll point out that Recurrent Neural Nets (i.e. neural nets with backwards connections) are frequently used for for Natural Language Processing (NLP). This includes variants like Bidirectional, Jordan and Elman Networks. Long Short-Term Memory (LSTM) is a more sophisticated neural net algorithm which can accomplish the same time and sequence-based tasks, but which can leverage standard learning methods like backprop since it doesn't suffer from the &quot;vanishing gradient problem.&quot; This is because LSTMs have been brilliantly engineered as &quot;perfect integrators,&quot; which makes it a lot easier to calculate the error gradients etc. over long periods of time. In contrast, learning with RNNs is still not theoretically well-grounded and is difficult to calculate through existing methods like Backpropagation Through Time (BPTT). In Time Delay Neural Networks (TDNNs), the idea is to add new neurons and connections with each new training example across a stretch of time or training sequence; unfortunately, this places a practical limitation on how many examples you can feed into the net before the size of the network gets out of hand or it starts forgetting, just as with RNNs. LSTMs have much longer memories (especially when augmented with Neural Turing Machines) so that'd be my first choice, assuming I wanted to use neural nets for NLP purposes. My knowledge of the subject is limited though (I'm still trying to learn the ropes) so there may be other important neural net algorithms I'm overlooking...&lt;/p&gt;&#xA;" OwnerUserId="1427" LastActivityDate="2016-09-18T16:50:48.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1989" PostTypeId="1" AcceptedAnswerId="1992" CreationDate="2016-09-18T19:43:02.197" Score="10" ViewCount="396" Body="&lt;p&gt;A &quot;general intelligence&quot; may be capable of learning a lot of different things, but possessing capability does not equal actually having it. The &quot;AGI&quot; must learn...and that learning process can take time. If you want an AGI to drive a car or play Go, you have to find some way of &quot;teaching&quot; it. Keep in mind that we have never built AGIs, so we don't know how long the training process can be, but it would be safe to assume pessimistic estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contrast that to a &quot;narrow intelligence&quot;. The narrow AI already knows how to drive a car or play Go. It has been programmed to be very excellent at one specific task. You don't need to worry about training the machine, because it has already been pre-trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;general intelligence&quot; seems to be more flexible than a &quot;narrow intelligence&quot;. You could buy an AGI and have it drive a car &lt;em&gt;and&lt;/em&gt; play Go. And if you are willing to do more training, you can even teach it a new trick: &lt;em&gt;how to bake a cake&lt;/em&gt;. I don't have to worry about unexpected tasks coming up, since the AGI will &lt;em&gt;eventually&lt;/em&gt; figure out how to do it, given enough training time. I would have to wait a &lt;em&gt;long time&lt;/em&gt; though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;narrow intelligence&quot; appears to be &lt;em&gt;more efficient&lt;/em&gt; at its assigned task, due to it being programmed specifically for that task. It knows exactly what to do, and doesn't have to waste time &quot;learning&quot; (unlike our AGI buddy here). Instead of buying one AGI to handle a bunch of different tasks poorly, I would rather buy a bunch of specialized narrow AIs. Narrow AI #1 drives cars, Narrow AI #2 plays Go, Narrow AI #3 bake cakes, etc. That being said, this is a very brittle approach, since if some unexpected task comes up, none of my narrow AIs would be able to handle it. I'm willing to accept that risk though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my &quot;thinking&quot; correct? Is there a trade-off between flexibility (AGI) and efficiency (narrow AI), like what I have just described above? Or is it theoretically possible for an AGI to be both flexible and efficient?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="21141" LastEditDate="2019-04-04T20:37:24.887" LastActivityDate="2019-04-04T20:37:24.887" Title="Is there a trade-off between flexibility and efficiency?" Tags="&lt;ai-design&gt;&lt;agi&gt;&lt;weak-ai&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1990" PostTypeId="2" ParentId="1987" CreationDate="2016-09-18T23:27:57.760" Score="13" Body="&lt;p&gt;There are many approaches to this kind of problem. The most obvious one is to &lt;strong&gt;create new features&lt;/strong&gt;. The best features I can come up with is to transform the coordinates to &lt;a href=&quot;https://en.wikipedia.org/wiki/Spherical_coordinate_system&quot; rel=&quot;noreferrer&quot;&gt;spherical coordinates&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not found a way to do it in playground, so I just created a few features that should help with this (sin features). After &lt;a href=&quot;http://playground.tensorflow.org/#activation=tanh&amp;amp;batchSize=10&amp;amp;dataset=spiral&amp;amp;regDataset=reg-plane&amp;amp;learningRate=0.03&amp;amp;regularizationRate=0&amp;amp;noise=0&amp;amp;networkShape=4,2&amp;amp;seed=0.73263&amp;amp;showTestData=false&amp;amp;discretize=false&amp;amp;percTrainData=50&amp;amp;x=true&amp;amp;y=true&amp;amp;xTimesY=false&amp;amp;xSquared=false&amp;amp;ySquared=false&amp;amp;cosX=false&amp;amp;sinX=true&amp;amp;cosY=false&amp;amp;sinY=true&amp;amp;collectStats=false&amp;amp;problem=classification&amp;amp;initZero=false&amp;amp;hideText=false&quot; rel=&quot;noreferrer&quot;&gt;500 iterations&lt;/a&gt; it will saturate and will fluctuate at 0.1 score. This suggest that no further improvement will be done and most probably I should make the hidden layer wider or add another layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not a surprise that after adding &lt;a href=&quot;http://playground.tensorflow.org/#activation=tanh&amp;amp;batchSize=10&amp;amp;dataset=spiral&amp;amp;regDataset=reg-plane&amp;amp;learningRate=0.03&amp;amp;regularizationRate=0&amp;amp;noise=0&amp;amp;networkShape=5,2&amp;amp;seed=0.73263&amp;amp;showTestData=false&amp;amp;discretize=false&amp;amp;percTrainData=50&amp;amp;x=true&amp;amp;y=true&amp;amp;xTimesY=false&amp;amp;xSquared=false&amp;amp;ySquared=false&amp;amp;cosX=false&amp;amp;sinX=true&amp;amp;cosY=false&amp;amp;sinY=true&amp;amp;collectStats=false&amp;amp;problem=classification&amp;amp;initZero=false&amp;amp;hideText=false&quot; rel=&quot;noreferrer&quot;&gt;just one neuron to the hidden layer&lt;/a&gt; you easily get 0.013 after 300 iterations. Similar thing happens by adding a new layer (0.017, but after significantly longer 500 iterations. Also no surprise as it is harder to propagate the errors). Most probably you can play with a learning rate or do an adaptive learning to make it faster, but this is not the point here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tck2s.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tck2s.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2492" LastActivityDate="2016-09-18T23:27:57.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1992" PostTypeId="2" ParentId="1989" CreationDate="2016-09-19T14:44:48.200" Score="6" Body="&lt;p&gt;The cleanest result we have on this issue is the &lt;a href=&quot;https://en.wikipedia.org/wiki/No_free_lunch_theorem&quot; rel=&quot;noreferrer&quot;&gt;&quot;no free lunch&quot; theorem&lt;/a&gt;. Basically, in order to make a system perform better at a specific task, you have to degrade its performance on other tasks, and so there is a flexibility-efficiency tradeoff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to the broader question, or whether or not your thinking is correct, I think it pays to look more closely at what you mean by a &quot;narrow intelligence.&quot; The AI systems that we have that play Go and drive cars did &lt;em&gt;not&lt;/em&gt; pop into existence able to do those things; they slowly learned how through lots and lots of training examples and a well-chosen architecture that mirrors the problem domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, &quot;neural networks&quot; as a methodology seems 'general' in a meaningful way; one could imagine that a general intelligence could be formed by solving the meta-learning problem (that is, learning the architecture that best suits a particular problem while learning the weights for that problem from training data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in that case, there will still be a flexibility-efficiency tradeoff; the general intelligence that's allowed to vary its architecture will be able to solve many different problems, but will take some time to discover what problem it's facing. An intelligence locked into a particular architecture will perform well on problems that architecture is well-suited for (better than the general, since it doesn't need to discover) but less well on other problems it isn't as well-suited for.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-19T14:44:48.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1993" PostTypeId="2" ParentId="1989" CreationDate="2016-09-19T15:06:34.227" Score="1" Body="&lt;p&gt;It would appear so.  One example, albeit not specifically AI related, is seen in the difference between digital computers and &lt;a href=&quot;https://en.wikipedia.org/wiki/Analog_computer&quot; rel=&quot;nofollow&quot;&gt;analog computers&lt;/a&gt;.  Pretty much everything we think of as a &quot;computer&quot; today is a digital computer with a von Neumann architecture.  And that's because the things are so general purpose that they can be easily programmed to do, essentially, anything.  But analog computers can (or could, back in the 60's or thereabouts) solve some types of problems faster than a digital computer.  But they fell out of favor exactly due to that lack of flexibility.  Nobody wants to hand-wire circuits with op-amps and comparators to solve for &lt;em&gt;y&lt;/em&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-19T15:06:34.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1995" PostTypeId="2" ParentId="74" CreationDate="2016-09-19T16:08:55.187" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Strong_AI&quot; rel=&quot;nofollow&quot;&gt;Strong&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_AI&quot; rel=&quot;nofollow&quot;&gt;weak AI&lt;/a&gt; are the older terms for &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow&quot;&gt;AGI&lt;/a&gt; (artificial general intelligence) and narrow AI. At least that's how I have seen it used and wikipedia seems to agree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally haven't seen Searle's definition of &quot;weak and strong AI&quot; in use much, but maybe the shift to the newer terms came about in part because Searle successfully confused the issue. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-19T16:08:55.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1996" PostTypeId="1" CreationDate="2016-09-20T04:20:02.937" Score="5" ViewCount="755" Body="&lt;p&gt;Is there a neural network(NN) system or architecture which can be used for only storing and retrieving information. For example; to store whole Avatar movie in HD format inside a neural network and retrieve(without loss) it from the neural network when needed. I searched the web and came across only LSTM RNN but in my understanding LSTM only stores pattern and not the content itself. If there is no such NN exist can you explain why it so?&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2016-09-20T13:17:17.733" Title="Which neural networks can be used only for storing and retrieving information?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1997" PostTypeId="1" AcceptedAnswerId="1999" CreationDate="2016-09-20T10:54:14.493" Score="14" ViewCount="703" Body="&lt;p&gt;The question is about the architecture of Deep Residual Networks (&lt;strong&gt;ResNets&lt;/strong&gt;). The model that won the 1-st places at &lt;a href=&quot;http://image-net.org/challenges/LSVRC/2015/results&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Large Scale Visual Recognition Challenge 2015&quot; (ILSVRC2015)&lt;/a&gt; in all five main tracks:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;em&gt;ImageNet Classification: “Ultra-deep” (quote Yann) 152-layer nets&lt;/em&gt; &lt;/li&gt;&#xA;  &lt;li&gt;&lt;em&gt;ImageNet Detection: 16% better than 2nd&lt;/em&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;em&gt;ImageNet Localization: 27% better than 2nd&lt;/em&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;em&gt;COCO Detection: 11% better than 2nd&lt;/em&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;em&gt;COCO Segmentation: 12% better than 2nd&lt;br&gt;&lt;br&gt;&lt;/em&gt;&#xA;  &lt;em&gt;Source:&lt;/em&gt; &lt;a href=&quot;http://image-net.org/challenges/talks/ilsvrc2015_deep_residual_learning_kaiminghe.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;MSRA @ ILSVRC &amp;amp; COCO 2015 competitions (presentation, 2-nd slide)&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This work is described in the following article:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1512.03385&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep Residual Learning for Image Recognition (2015, PDF)&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Microsoft Research team&lt;/strong&gt; (developers of ResNets: Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun) in their article:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1603.05027.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;&lt;em&gt;Identity Mappings in Deep Residual Networks (2016)&lt;/em&gt;&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;state that &lt;strong&gt;depth&lt;/strong&gt; plays a key role:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;&quot;&lt;strong&gt;We obtain these results via a simple but essential concept — going deeper. These results demonstrate the potential of pushing the limits of depth.&lt;/strong&gt;&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is emphasized in their &lt;a href=&quot;http://image-net.org/challenges/talks/ilsvrc2015_deep_residual_learning_kaiminghe.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;presentation&lt;/a&gt; also (deeper - better):&lt;br&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;- &quot;A deeper model should not have higher training error.&quot;&lt;br&gt; &#xA;  - &quot;Deeper ResNets have lower training error, and also lower test error.&quot;&lt;br&gt; &#xA;  - &quot;Deeper ResNets have lower error.&quot;&lt;br&gt;&#xA;  - &quot;All benefit more from deeper features – cumulative gains!&quot;&lt;br&gt;&#xA;  - &quot;Deeper is still better.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is the sctructure of 34-layer residual (for reference):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/L8m0X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L8m0X.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;But recently I have found one theory that introduces a novel interpretation of residual networks showing they are exponential ensembles:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1605.06431&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Residual Networks are Exponential Ensembles of Relatively Shallow Networks (2016)&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Deep Resnets are described as many shallow networks whose outputs are pooled at various depths. &#xA;There is a picture in the article. I attach it with explanation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PGhK2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PGhK2.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; Residual Networks are&#xA;  conventionally shown as (a), which is a natural representation of&#xA;  Equation (1). When we expand this formulation to Equation (6), we&#xA;  obtain an unraveled view of a 3-block residual network (b). From this&#xA;  view, it is apparent that residual networks have O(2^n) implicit paths&#xA;  connecting input and output and that adding a block doubles the number&#xA;  of paths.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In conclusion of the article it is stated:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;It is not depth, but the ensemble that makes residual networks strong&lt;/strong&gt;.&#xA;  Residual networks push the limits of network multiplicity, not network&#xA;  depth. Our proposed unraveled view and the lesion study show that&#xA;  residual networks are an implicit ensemble of exponentially many&#xA;  networks. If most of the paths that contribute gradient are very short&#xA;  compared to the overall depth of the network, &lt;strong&gt;increased depth&lt;/strong&gt;&#xA;  alone &lt;strong&gt;can’t be the key characteristic&lt;/strong&gt; of residual networks. We now&#xA;  believe that &lt;strong&gt;multiplicity&lt;/strong&gt;, the network’s expressability in the&#xA;  terms of the number of paths, plays &lt;strong&gt;a key role&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But it is only a recent theory that can be confirmed or refuted. It happens sometimes that some theories are refuted and articles are withdrawn.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Should we think of deep ResNets as an ensemble after all? &lt;strong&gt;Ensemble&lt;/strong&gt; or &lt;strong&gt;depth&lt;/strong&gt; makes residual networks so strong? Is it possible that even the developers themselves do not quite perceive what their own model represents and what is the key concept in it?&lt;/p&gt;&#xA;" OwnerUserId="1791" LastEditorUserId="2444" LastEditDate="2019-10-14T15:17:05.923" LastActivityDate="2019-10-14T15:17:05.923" Title="Should deep residual networks be viewed as an ensemble of networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;residual-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1998" PostTypeId="2" ParentId="1996" CreationDate="2016-09-20T13:17:17.733" Score="5" Body="&lt;p&gt;Neural networks are usually trained to produce a certain output given a certain input. Often the output is a classification of the input or some other form of input description. Sometimes it is an action in a game and sometimes it is indeed stored data, a memory if you will. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case the input is often a part of the stored data, so the NN actually completes the given input. This setup is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoassociative_memory&quot; rel=&quot;noreferrer&quot;&gt;autoassociative memory&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Hopfield_network&quot; rel=&quot;noreferrer&quot;&gt;Hopfield networks&lt;/a&gt; are an example for this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example you might give this NN the first frame of the movie Avatar and it would output the complete movie. Unfortunately this would probably be absolutely &lt;a href=&quot;https://en.wikipedia.org/wiki/Hopfield_network#Capacity&quot; rel=&quot;noreferrer&quot;&gt;crazy inefficient&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-20T13:17:17.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1999" PostTypeId="2" ParentId="1997" CreationDate="2016-09-20T14:40:30.907" Score="5" Body="&lt;p&gt;Imagine a genie grants you three wishes. Because you are an ambitious deep learning researcher your first wish is a perfect solution for a 1000-layer NN for Image Net, which promptly appears on your laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now a genie induced solution doesn't give you any intuition how it might be interpreted as an ensemble, but do you really believe that you need 1000 layers of abstraction to distinguish a cat from a dog? As the authors of the &quot;ensemble paper&quot; mention themselves, this is definitely not true for biological systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you could waste your second wish on a decomposition of the solution into an ensemble of networks, and I'm pretty sure the genie would be able to oblige. The reason being that part of the power of a deep network will always come from the ensemble effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is not surprising that two very successful tricks to train deep networks, dropout and residual networks, have an immediate interpretation as implicit ensemble. Therefore &quot;it's not depth, but the ensemble&quot; strikes me as a false dichotomy. You would really only say that if you honestly believed that you need hundreds or thousands of levels of abstraction to classify images with human accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you use the last wish for something else, maybe a pinacolada. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-20T14:40:30.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2000" PostTypeId="1" AcceptedAnswerId="2004" CreationDate="2016-09-20T15:55:48.757" Score="1" ViewCount="3704" Body="&lt;p&gt;In my attempt at trying to learn neural network and machine learning I'm am trying to create a simple neural network which can be trained to recognise one word from a given string (which contains only one word). So in effect if one where to feed it a string containing the trained word but spelled wrong the network would be able to still recognise the word. Can anybody help me with some pseudo code or a start of a code. Or a general explanation of how to to this because I have read like 6 articles and 8 example projects and still have no clue how to do this&lt;/p&gt;&#xA;" OwnerUserId="2529" LastEditorUserId="2529" LastEditDate="2016-09-20T16:12:01.293" LastActivityDate="2016-09-21T07:43:06.087" Title="Simple text recognition with neural network" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="2" CommentCount="4" ClosedDate="2016-09-21T18:50:00.793" ContentLicense="CC BY-SA 3.0" />
  <row Id="2003" PostTypeId="2" ParentId="2000" CreationDate="2016-09-21T06:43:29.150" Score="2" Body="&lt;p&gt;If I'm reading it correctly, this question has nothing to do with optical character recognition. You want to create a system that takes a digital string of characters as input, then finds the best match from a predetermined list of words. That sounds like a task for if-then-else logic and dictionary lookup. It might be possible to use a neural net, but not easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural net takes a fixed number of inputs, each of which are a value between zero and one. A major hurdle is that you probably want variable-sized inputs. Another hurdle is that you'll need to code the inputs some way onto numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These hurdles can be overcome but they are tipoffs that neural networks aren't well-suited for the task.&lt;/p&gt;&#xA;" OwnerUserId="2542" LastActivityDate="2016-09-21T06:43:29.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2004" PostTypeId="2" ParentId="2000" CreationDate="2016-09-21T07:43:06.087" Score="1" Body="&lt;p&gt;An optimal solution for the task as stated, would be some alignment algorithm like Smith-Waterman, with a matrix which encodes typical typo frequencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an exercise in NNs, I would recommend using a RNN. This circumvents the problem that your inputs will be of variable size, because you just feed one letter after another and get an output once you feed the delimiter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As trainingsdata you'll need a list of random words and possibly a list of random strings, as negative examples and a list of slightly messed up versions of your target word as positive examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://gist.github.com/karpathy/d4dee566867f8291f086&quot; rel=&quot;nofollow&quot;&gt;minimal character-level RNN&lt;/a&gt;, which consists of only a little more than a hundred lines of code, so you might be able to get your head around it or at least get it to run. Here is &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow&quot;&gt;the excellent blog post&lt;/a&gt; by Karpathy to which the code sample belongs. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-21T07:43:06.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2005" PostTypeId="1" AcceptedAnswerId="2006" CreationDate="2016-09-21T13:05:56.327" Score="22" ViewCount="1428" Body="&lt;p&gt;In 2004 &lt;a href=&quot;https://en.wikipedia.org/wiki/Jeff_Hawkins&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jeff Hawkins&lt;/a&gt;, inventor of the palm pilot, published a very interesting book called &lt;a href=&quot;https://en.wikipedia.org/wiki/On_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Intelligence&lt;/a&gt;, in which he details a theory how the human neocortex works.&lt;/p&gt;&#xA;&lt;p&gt;This theory is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Memory-prediction_framework&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memory-Prediction framework&lt;/a&gt; and it has some striking features, for example not only bottom-up (feedforward), but also top-down information processing and the ability to make simultaneous, but discrete predictions of different future scenarios (as described &lt;a href=&quot;http://journal.frontiersin.org/article/10.3389/fncir.2016.00023/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this paper&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The promise of the Memory-Prediction framework is unsupervised generation of stable high level representations of future possibilities. Something which would revolutionise probably a whole bunch of AI research areas.&lt;/p&gt;&#xA;&lt;p&gt;Hawkins founded &lt;a href=&quot;https://en.wikipedia.org/wiki/Numenta&quot; rel=&quot;nofollow noreferrer&quot;&gt;a company&lt;/a&gt; and proceeded to implement his ideas. Unfortunately more than ten years later the promise of his ideas is still unfulfilled. So far the implementation is only used for anomaly detection, which is kind of the opposite of what you really want to do. Instead of extracting the understanding, you'll extract the instances which your artificial cortex doesn't understand.&lt;/p&gt;&#xA;&lt;p&gt;My question is in what way Hawkins's framework falls short. What are the concrete or conceptual problems that so far prevent his theory from working in practice?&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2227" LastEditDate="2020-09-01T08:11:34.710" LastActivityDate="2020-09-01T08:11:34.710" Title="What are the flaws in Jeff Hawkins's AI framework?" Tags="&lt;htm&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="2006" PostTypeId="2" ParentId="2005" CreationDate="2016-09-21T20:36:17.423" Score="13" Body="&lt;p&gt;The short answer is that Hawkins' vision has yet to be implemented in a widely accessible way, particularly the indispensable parts related to prediction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The long answer is that I read Hawkins' book a few years ago and was excited by the possibilities of Hierarchical Temporal Memory (HTM). I still am, despite the fact that I have a few reservations about some of his philosophical musings on the meanings of consciousness, free will and other such topics. I won't elaborate on those misgivings here because they're not germane to the main, overwhelming reason why HTM nets haven't succeeded as much as expected to date: to my knowledge, Numenta has only implemented a truncated version of his vision. They left out most of the prediction architecture, which plays such a critical role in Hawkins' theories. As Gerod M. Bonhoff put it in &lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a482820.pdf&quot;&gt;an excellent thesis&lt;/a&gt;&lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a482820.pdf&quot;&gt;1&lt;/a&gt; on HTMs, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;In March of 2007, Numenta released what they claimed was a “research&#xA;  implementation” of HTM theory called Numenta Platform for Intelligent&#xA;  Computing (NuPIC). The algorithm used by NuPIC at this time is called&#xA;  “Zeta1.” NuPIC was released as an open source software platform and&#xA;  binary files of the Zeta1 algorithm. Because of licensing, this paper&#xA;  is not allowed to discuss the proprietary implementation aspects of&#xA;  Numenta’s Zeta1 algorithm. There are, however, generalized&#xA;  concepts of implementation that can be discussed freely. The two most&#xA;  important of these are how the Zeta 1 algorithm (encapsulated in each&#xA;  memory node of the network hierarchy) implements HTM theory. To&#xA;  implement any theory in software, an algorithmic design for each&#xA;  aspect of the theory must be addressed. The most important design&#xA;  decision Numenta adopted was to eliminate feedback within the&#xA;  hierarchy and instead choose to simulate this theoretical concept&#xA;  using only data pooling algorithms for weighting. This decision is&#xA;  immediately suspect and violates key concepts of HTM. Feedback,&#xA;  Hawkins’ insists, is vital to cortical function and central to his&#xA;  theories. Still, Numenta claims that most HTM applicable problems can&#xA;  be solved using their implementation and proprietary pooling&#xA;  algorithms.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am still learning the ropes in this field and cannot say whether or not Numenta has since scrapped this approach in favor of a full implementation of Hawkins' ideas, especially the all-important prediction architecture. Even if they have, this design decision has probably delayed adoption by many years. That's not a criticism per se; perhaps the computational costs of tracking prediction values and updating them on the fly were too much to bear at the time, on top of the ordinary costs of processing neural nets, leaving them with no other path except to try half-measures like their proprietary pooling mechanisms. Nevertheless, all of the best research papers I've read on the topic since then have chosen to reimplement the algorithms rather than relying on Numenta's platform, typically because of the missing prediction features. Cases in point include Bonhoff's thesis and &lt;a href=&quot;http://cogprints.org/9187/1/HTM_TR_v1.0.pdf&quot;&gt;Maltoni's technical report for the University of Bologna Biometric System Laboratory&lt;/a&gt;&lt;a href=&quot;http://cogprints.org/9187/1/HTM_TR_v1.0.pdf&quot;&gt;2&lt;/a&gt;. In all of those cases, however, there is no readily accessible software for putting their variant HTMs to immediate use (as far as I know). The gist of all this is that like G.K. Chesterton's famous maxim about Christianity, &quot;HTMs have not been tried and found wanting; they have been found difficult, and left untried.&quot; Since Numenta left out the prediction steps, I assume that they would be the main stumbling blocks awaiting anyone who wants to code Hawkins' full vision of what an HTM should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a482820.pdf&quot;&gt;1&lt;/a&gt;Bonhoff, Gerod M., 2008, Using Hierarchical Temporal Memory for Detecting Anomalous Network Activity. Presented in March, 2008 at the Air Force Institute of Technology, Wright-Patterson Air Force Base, Ohio.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cogprints.org/9187/1/HTM_TR_v1.0.pdf&quot;&gt;2&lt;/a&gt;Maltoni, Davide, 2011, Pattern Recognition by Hierarchical Temporal Memory. DEIS Technical Report published April 13, 2011. University of Bologna Biometric System Laboratory: Bologna, Italy.   &lt;/p&gt;&#xA;" OwnerUserId="1427" LastActivityDate="2016-09-21T20:36:17.423" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2008" PostTypeId="1" AcceptedAnswerId="2009" CreationDate="2016-09-23T10:33:58.060" Score="69" ViewCount="48134" Body="&lt;p&gt;As far as I can tell, neural networks have a &lt;strong&gt;fixed number of neurons&lt;/strong&gt; in the input layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a network. &#xA;How is the &lt;strong&gt;varying input size&lt;/strong&gt; reconciled with the &lt;strong&gt;fixed size&lt;/strong&gt; of the input layer of the network? In other words, how is such a network made flexible enough to deal with an input that might be anywhere from one word to multiple pages of text?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my assumption of a fixed number of input neurons is wrong and new input neurons are added to/removed from the network to match the input size I don't see how these can ever be trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I give the example of NLP, but lots of problems have an inherently unpredictable input size. I'm interested in the general approach for dealing with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For images, it's clear you can up/downsample to a fixed size, but, for text, this seems to be an impossible approach since adding/removing text changes the meaning of the original input.&lt;/p&gt;&#xA;" OwnerUserId="2522" LastEditorUserId="2444" LastEditDate="2019-07-13T19:28:47.193" LastActivityDate="2019-12-05T18:14:00.180" Title="How can neural networks deal with varying input sizes?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;natural-language-processing&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="38" ContentLicense="CC BY-SA 4.0" />
  <row Id="2009" PostTypeId="2" ParentId="2008" CreationDate="2016-09-23T11:27:44.700" Score="51" Body="&lt;p&gt;Three possibilities come to mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest is the &lt;strong&gt;zero-padding&lt;/strong&gt;. Basically, you take a rather big input size and just add zeroes if your concrete input is too small. Of course, this is pretty limited and certainly not useful if your input ranges from a few words to full texts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;noreferrer&quot;&gt;Recurrent NNs&lt;/a&gt; (RNN) are a very natural NN to choose if you have texts of varying size as input. You input words as &lt;strong&gt;word vectors&lt;/strong&gt; (or embeddings) just one after another and the internal state of the RNN is supposed to encode the meaning of the full string of words. &lt;a href=&quot;http://www.iro.umontreal.ca/~lisa/pointeurs/RNNSpokenLanguage2013.pdf&quot; rel=&quot;noreferrer&quot;&gt;This is one&lt;/a&gt; of the earlier papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is using &lt;a href=&quot;https://en.wikipedia.org/wiki/Recursive_neural_network&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;recursive NNs&lt;/strong&gt;&lt;/a&gt;. This is basically a form of preprocessing in which a text is recursively reduced to a smaller number of word vectors until only one is left - your input, which is supposed to encode the whole text. This makes a lot of sense from a linguistic point of view if your input consists of sentences (which can vary a lot in size), because sentences are structured recursively. For example, the word vector for &quot;the man&quot;, should be similar to the word vector for &quot;the man who mistook his wife for a hat&quot;, because noun phrases act like nouns, etc. Often, you can use linguistic information to guide your recursion on the sentence. If you want to go way beyond the Wikipedia article, &lt;a href=&quot;http://nlp.stanford.edu/~socherr/thesis.pdf&quot; rel=&quot;noreferrer&quot;&gt;this is probably a good start&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2019-07-13T19:31:32.923" LastActivityDate="2019-07-13T19:31:32.923" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2011" PostTypeId="2" ParentId="1885" CreationDate="2016-09-25T05:33:54.720" Score="3" Body="&lt;p&gt;I am currently reading &lt;em&gt;Superintelligence: Paths, Dangers, Strategies&lt;/em&gt; by Nick Bostrom. When he discusses whole brain emulation, although computing power (storage, bandwidth, CPU, body simulation, &amp;amp; environment simulation) is one of the three general key things we are lacking toward its success, he also seems to agree that computing power is the most feasible and attainable of the three general issues we have for attaining it as of now. However he also goes on to to say &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Just how much technology is required for whole brain emulation&#xA;  depends on the level of abstraction at which the brain is simulated.&lt;sup&gt;&lt;a href=&quot;https://books.google.co.uk/books?id=1mMJBAAAQBAJ&amp;amp;pg=PT41&amp;amp;lpg=PT41&amp;amp;dq=%22Just%20how%20much%20technology%20is%20required%20for%20whole%20brain%20emulation%20depends%20on%20the%20level%20of%22&amp;amp;source=bl&amp;amp;ots=Pmw4PUDele&amp;amp;sig=PfmDRgcJuKBf3TwyCoVtCmp9XmE&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwj4gZy6tbnPAhVRF8AKHUoQDD8Q6AEIHjAA#v=onepage&amp;amp;q=%22Just%20how%20much%20technology%20is%20required%20for%20whole%20brain%20emulation%20depends%20on%20the%20level%20of%22&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;ref&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which is an interesting thought, but a whole different discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, so I think you are correct in thinking that we aren't far from having the computing power and maybe you are on to something, but rather are biggest hurdles are the other two key prerequisites that we need to attain before we can even begin trying, which are scanning, and translation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of the three, it would seem translation is the one we need to advance in the most, as of now. A modest prediction of attaining whole brain emulation is at least 15 years or mid century. Theres much more information in this book of all of the different paths that can be taken to achieve super intelligence, and it is well researched, I highly recommend it if you haven't read it already.&lt;/p&gt;&#xA;" OwnerUserId="1997" LastEditorUserId="8" LastEditDate="2016-10-01T10:42:13.713" LastActivityDate="2016-10-01T10:42:13.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2012" PostTypeId="1" AcceptedAnswerId="2013" CreationDate="2016-09-25T10:29:34.560" Score="1" ViewCount="242" Body="&lt;p&gt;In my estimation we have two minds which manage to speak to each other in dialectic through a series of interrupts. Thus at any one time one of these systems is controlling master and inhabits our consciousness. The subordinate system controls context which is constantly being &quot;primed&quot; by our senses and our subordinate systems experience of our conscious thought process( see thinking fast and slow by Daniel Kahneman). Thus our thought process is constantly a driven one. Similarly this system works as a node in a community and not as a standalone thing.&lt;br&gt;&#xA; I think what we have currently is &quot;artificial thinking&quot; which is abstracted a long way from what is described above. so my question is &quot;are there any artificial intelligence systems with an internal dialectical approach and with drivers and conceived above and which develop within a community of nodes? &quot; &lt;/p&gt;&#xA;" OwnerUserId="2601" LastEditorUserId="33" LastEditDate="2016-09-26T15:37:21.923" LastActivityDate="2021-03-31T22:19:43.653" Title="Are there any artificial intelligence systems with an internal dialectical approach and multiple minds which develop within a community of nodes?" Tags="&lt;philosophy&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2013" PostTypeId="2" ParentId="2012" CreationDate="2016-09-25T17:21:36.137" Score="2" Body="&lt;p&gt;There are a lot of systems that follow the ancient maxim: &amp;quot;Always two there are; no more, no less. A master and an apprentice.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt; a class of such setups is called &lt;a href=&quot;https://webdocs.cs.ualberta.ca/%7Esutton/book/ebook/node66.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Actor-Critic-Method&lt;/a&gt;. There you have a master, whose duty it is to create feedback for the actions of the apprentice, who acts in a given environment. This would be comparable to how a human learns some physical activity, like playing table tennis. You basically let your body do its thing, but your consciousness evaluates how good the result is.&lt;/p&gt;&#xA;&lt;p&gt;The setup of &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; might be even closer to &lt;a href=&quot;https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kahnemann's system 1 and system 2&lt;/a&gt;. AlphaGo has two neural networks which provide actions and evaluations (system 1, fast, intuitive, etc.) and the Monte Carlo tree search, which uses these actions and evaluations to prune a search tree and make a decision (system 2, deliberate, logical).&lt;/p&gt;&#xA;&lt;p&gt;In the end, this kind of structure will pop up again and again because it is often necessary to do some kind of classification or preprocessing on the raw data before &lt;em&gt;your&lt;/em&gt; algorithm can be run on it. You could frame the whole history of &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;gofai&lt;/a&gt; as the story of how scientists thought system 1 should be easy and system 2 should be doable in a few decades, where the reality is that we have no idea how difficult system 2 is because it turned out that system 1 is extremely difficult.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="36737" LastEditDate="2021-03-31T22:19:43.653" LastActivityDate="2021-03-31T22:19:43.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2014" PostTypeId="2" ParentId="2008" CreationDate="2016-09-26T06:34:19.113" Score="16" Body="&lt;p&gt;Others already mentioned:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;zero padding&lt;/li&gt;&#xA;&lt;li&gt;RNN&lt;/li&gt;&#xA;&lt;li&gt;recursive NN&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;so I will add another possibility: using convolutions different number of times depending on the size of input. Here is an &lt;a href=&quot;http://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;excellent book&lt;/a&gt; which backs up this approach:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider a collection of images, where each image has a different&#xA;  width and height. It is unclear how to model such inputs with a weight&#xA;  matrix of fixed size. Convolution is straightforward to apply; the&#xA;  kernel is simply applied a different number of times depending on the&#xA;  size of the input, and the output of the convolution operation scales&#xA;  accordingly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Taken from page 354, 9.7 Data Types, 3rd paragraph. You can read it further to see some other approaches.&lt;/p&gt;&#xA;" OwnerUserId="2492" LastEditorUserId="31870" LastEditDate="2019-12-05T18:14:00.180" LastActivityDate="2019-12-05T18:14:00.180" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2016" PostTypeId="5" CreationDate="2016-09-26T11:16:11.610" Score="0" Body="&lt;p&gt;The loss function is sometimes called an error function.  Its inverse is a wellness or value function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intention of each iteration is to decrease the result of applying the loss function.  The particular method intended to produce this decrease is to calculate the gradient of the loss function and use it to compute the incremental change in parameters likely to reduce the loss.  It is often used in conjunction with back propagation to distribute the corrective signal over a sequence of layers, each of which is parameterized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid overshooting the optimum, leading to oscillation or chaotic behavior, the corrective signal is attenuated with a factor called the learning rate.  Too low a learning rate will compromise the speed of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several strategies exist to calculate loss functions, hyper-parameterize the corrective signaling that is back-propagated, or integrate other search strategies to improve either reliability, speed, or accuracy.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-09-28T23:57:05.400" LastActivityDate="2019-09-28T23:57:05.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2017" PostTypeId="4" CreationDate="2016-09-26T11:16:11.610" Score="0" Body="For questions surrounding gradient descent, a method for finding the optimum state of a parameterized function based on another function often called the loss or error function.  It iteratively descends the loss surface to the minimum loss by adjusting parameters based on the product of the partial derivatives comprising the gradient and a learning rate." OwnerUserId="1791" LastEditorUserId="4302" LastEditDate="2018-10-18T03:16:44.933" LastActivityDate="2018-10-18T03:16:44.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2018" PostTypeId="2" ParentId="2012" CreationDate="2016-09-26T15:39:11.980" Score="1" Body="&lt;p&gt;You could argue that some &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-agent_system&quot; rel=&quot;nofollow&quot;&gt;Multi-Agent System&lt;/a&gt; approaches do, and some systems based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Blackboard_system&quot; rel=&quot;nofollow&quot;&gt;blackboard architecture&lt;/a&gt; could conceivably fit this regime as well. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-26T15:39:11.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2020" PostTypeId="1" AcceptedAnswerId="2022" CreationDate="2016-09-27T16:55:24.733" Score="14" ViewCount="831" Body="&lt;p&gt;In the recent PC game &lt;em&gt;&lt;a href=&quot;http://www.theturingtestgame.com/&quot;&gt;The Turing Test&lt;/a&gt;&lt;/em&gt;, the AI (&quot;TOM&quot;) needs help from Ava to get through some puzzle rooms. TOM says he is unable to solve the puzzles because he is not allowed to &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lateral_thinking&quot;&gt;think laterally&lt;/a&gt;.&quot; Specifically, he says he would not have thought to throw a box through a window to solve the first room. His creators, the story goes, turned that capability off because such thinking could produce &quot;ethically suboptimal&quot; solutions, like chopping off an arm to leave on a pressure plate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would all creative puzzle-solving abilities need to be removed from an AI to keep its results reasonable, or could we get some benefits of lateral thinking without losing an arm?&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-09-30T18:40:58.730" Title="Could an AI think laterally while avoiding &quot;ethically suboptimal&quot; choices?" Tags="&lt;agi&gt;&lt;problem-solving&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2021" PostTypeId="1" AcceptedAnswerId="2086" CreationDate="2016-09-27T17:30:55.837" Score="3" ViewCount="301" Body="&lt;p&gt;In the recent &lt;a href=&quot;https://live.newscientist.com/&quot; rel=&quot;noreferrer&quot;&gt;festival of science&lt;/a&gt;, there was a talk given by researcher &lt;a href=&quot;https://live.newscientist.com/mike-cook/&quot; rel=&quot;noreferrer&quot;&gt;Mike Cook&lt;/a&gt; about:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://www.gamesbyangelina.org/&quot; rel=&quot;noreferrer&quot;&gt;ANGELINA&lt;/a&gt;, an AI game designer that has invented game mechanics, made games about news stories, and was the first AI to enter a game jam.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the aim of Angelina AI is basically to design videogames.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly, how exactly does Angelina design the new games? How does it work behind the scenes?&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="2444" LastEditDate="2020-11-19T16:13:39.997" LastActivityDate="2020-11-19T20:16:24.933" Title="How exactly does Angelina design games?" Tags="&lt;game-ai&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2022" PostTypeId="2" ParentId="2020" CreationDate="2016-09-27T17:50:12.263" Score="17" Body="&lt;p&gt;&lt;strong&gt;No&lt;/strong&gt;, with a &lt;em&gt;but&lt;/em&gt;. We can have creative yet ethical problem-solving if the system has a complete system of ethics, but otherwise creativity will be unsafe by default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can classify AI decision-making approaches into two types: interpolative thinkers, and extrapolative thinkers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interpolative thinkers learn to classify and mimic whatever they're learning from, and don't try to give reasonable results outside of their training domain. You can think of them as interpolating between training examples, and benefitting from all of the mathematical guarantees and provisos as other statistical techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extrapolative thinkers learn to manipulate underlying principles, which allows them to combine those principles in previously unconsidered ways. The relevant field for intuition here is &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization&quot;&gt;numerical optimization&lt;/a&gt;, of which the simplest and most famous example is &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_programming&quot;&gt;linear programming&lt;/a&gt;, rather than the statistical fields that birthed machine learning. You can think of them as extrapolating beyond training examples (indeed, many of them don't even require training examples, or use those examples to infer underlying principles).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The promise of extrapolative thinkers is that they can come up with these 'lateral' solutions much more quickly than people would be able to. The problem with these extrapolative thinkers is that they only use the spoken principles, not any unspoken ones that might seem too obvious to mention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An attribute of solutions to optimization problems is that the feature vector is often 'extreme' in some way. In linear programming, at least one vertex of the feasible solution space will be optimal, and so simple solution methods find an optimal vertex (which is almost infeasible by nature of being a vertex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As another example, the minimum-fuel solution for moving a spacecraft from one position to another is called '&lt;a href=&quot;https://en.wikipedia.org/wiki/Bang%E2%80%93bang_control&quot;&gt;bang-bang&lt;/a&gt;,' where you accelerate the craft as quickly as possible at the beginning and end of the trajectory, coasting at maximum speed in between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While a virtue when the system is correctly understood (bang-bang &lt;em&gt;is&lt;/em&gt; optimal for many cases), this is catastrophic when the system is incorrectly understood. My favorite example here is &lt;a href=&quot;https://resources.mpi-inf.mpg.de/departments/d1/teaching/ws14/Ideen-der-Informatik/Dantzig-Diet.pdf&quot;&gt;Dantzig's diet problem&lt;/a&gt; (discussion starts on page 5 of the pdf), where he tries to optimize his diet using math. Under his first constraint set, he's supposed to drink 500 gallons of vinegar a day. Under his second, 200 bouillon cubes. Under his third, two pounds of bran. The considerations that make those obviously bad ideas aren't baked into the system, and so the system innocently suggests them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can completely encode the knowledge and values that a person uses to judge these plans into the AI, then extrapolative systems are as safe as that person. They'll be able to consider and reject the wrong sort of extreme plans, and leave you with the right sort of extreme plans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you can't, then it does make sense to not build an extrapolative decision-maker, and instead build an interpolative one. That is, instead of asking itself &quot;how do I best accomplish goal X?&quot; it's asking itself &quot;what would a person do in this situation?&quot;. The latter might be much worse at accomplishing goal X, but it has much less of the tail risk of sacrificing other goals to accomplish X.&lt;/p&gt;&#xA;" OwnerUserId="10" LastEditorUserId="10" LastEditDate="2016-09-27T19:32:53.843" LastActivityDate="2016-09-27T19:32:53.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2023" PostTypeId="1" AcceptedAnswerId="2027" CreationDate="2016-09-27T18:11:43.820" Score="5" ViewCount="212" Body="&lt;p&gt;I understand that neural networks model biological neurons.  Each node in the network represents a neuron cell and the connections between nodes represent the connections between cells.  As in nature, a neuron fires an electrical signal to connected neurons based on some kind of threshold or function that mimics such.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent discoveries on how the brain works reveal the importance of calcium within the cells.  See &lt;a href=&quot;http://link.springer.com/article/10.1007/BF01794675&quot; rel=&quot;noreferrer&quot;&gt;http://link.springer.com/article/10.1007/BF01794675&lt;/a&gt; for more information.  To summarize, calcium affects the regulation, stimulation and transmission of electrical activity as well as the destruction of neurones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my study of neural networks, there does not seem to be a calcium equivalent.  Having one would imply that the functions, connections and weights in an artificial network are configured during the training and execution process and can change over time.   I understand that back-propagation is used to train the weights, but have not seen anything that trains the function nor the connections (although a zero weight could imply no connection).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of such a network (or training algorithm)?  If so, do these networks perform better than a network that is pre-configured?&lt;/p&gt;&#xA;" OwnerUserId="1434" LastActivityDate="2016-09-28T08:51:44.490" Title="What is the calcium equivalent role in neural networks" Tags="&lt;philosophy&gt;&lt;unsupervised-learning&gt;&lt;neurons&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2024" PostTypeId="2" ParentId="2020" CreationDate="2016-09-27T22:27:38.610" Score="0" Body="&lt;p&gt;You may consider the programming as an ethical part of the design as well. AI will act based on what has been instructed to it as ethically important or not.&#xA;It may/should even be part of the parameters that forge the process of finding solutions, which could allow for a more refine and creative solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We understand the basics of ethic in normal circumstances, but if we can't predict how any human will behave in an ethical conundrum we can enforce what an AI wouldn't do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as we have control over the mechanism that drive an AI we sure have a responsability to inject ethical failsafes.&#xA;The problem lies in self taught AI with an ability to overrides directives.&#xA;(CF Asimov Laws.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way the AI is creative seems irrelevant in that case.&lt;/p&gt;&#xA;" OwnerUserId="2658" LastActivityDate="2016-09-27T22:27:38.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2025" PostTypeId="2" ParentId="2020" CreationDate="2016-09-28T00:32:47.923" Score="-1" Body="&lt;p&gt;A lot of this depends on the breadth of consideration. For example, what would the medium and long term effects of the lateral thinking be? The robot could sever an arm for a pressure plate but it would mean that the person no longer had an arm, a functional limitation at best, that the person might bleed out and die/be severely constrained, and that the person (and people in general) would both no longer cooperate and likely seek to eliminate the robot. People can think laterally because consider these things - ethics are really nothing more than a set of guidelines that encompass these considerations. The robot could as well, were it to be designed to consider these externalities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all else fails,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asimov's Laws of Robotics: (0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.) 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law&lt;/p&gt;&#xA;" OwnerUserId="2661" LastActivityDate="2016-09-28T00:32:47.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2027" PostTypeId="2" ParentId="2023" CreationDate="2016-09-28T08:51:44.490" Score="6" Body="&lt;p&gt;Neural networks &lt;strong&gt;don't&lt;/strong&gt; model biological neurons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are at best &lt;em&gt;inspired&lt;/em&gt; by biological neurons, in that they get excited by certain inputs and fire once the excitation crosses a threshold. And this second point even holds only approximately because the backpropagation algorithm needs smoothed out steps to learn by gradient descent. And backpropagation is not even inspired by biology, that would rather be &lt;a href=&quot;https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Hebbian_Learning&quot;&gt;hebbian learning&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally in machine learning people do what works. The connection to biology is tenuous at best. You will usually not find one to one correspondence of low level details between machine learning setups and biological neurons. For that you'll have to turn to &lt;a href=&quot;https://www.humanbrainproject.eu/sp6&quot;&gt;brain simulations&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-28T08:51:44.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2031" PostTypeId="2" ParentId="1625" CreationDate="2016-09-28T17:00:20.347" Score="2" Body="&lt;p&gt;I would strongly recommend that you check the book &quot;The Perfect Bet&quot; by Adam Kucharski. It does not mention technical methods such as neural networks but it gives a good history (and very nice stories) on what people had done on that field. It gives you the notion that in order to get achieve a better payoff, your goal is not actually making a better prediction but choosing the better options by considering what other players are doing. If you ask why it is not just the better prediction, the answer is that there is always a balance of risk and payoff and since you cannot find a 100% guaranteed way of prediction, you will have to balance risk and payoff in order to gain in the long run. In addition, although theory suggests that you can make a good prediction by gathering all variables, this is not really possible in practice. Thus, human assistance and observing what others are doing is used as a way to improve mathematical predictions and possible payoffs.&lt;/p&gt;&#xA;" OwnerUserId="2680" LastActivityDate="2016-09-28T17:00:20.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2033" PostTypeId="1" AcceptedAnswerId="2039" CreationDate="2016-09-29T03:05:31.943" Score="4" ViewCount="490" Body="&lt;p&gt;I'm a freshman to machine learning. We all know that there are 2 kinds of problems in our life: problems that humans can solve and problems we can't solve. For problems humans can solve, we always try our best to write some algorithm and tell machine to follow it step by step, and finally the machine acts like people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm curious about are these problems humans can't solve. If humans ourselves can't sum up and get an algorithm (which means that we ourselves don't know how to solve the problem), can a machine solve the problem? That is, can the machine sum up and get an algorithm by itself based on a large amount of problem data?&lt;/p&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="75" LastEditDate="2016-09-29T12:31:24.400" LastActivityDate="2016-10-01T13:04:32.967" Title="Must people tell an AI which algorithm it should use? Can an AI learn algorithms by itself?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2034" PostTypeId="2" ParentId="2033" CreationDate="2016-09-29T03:30:28.413" Score="2" Body="&lt;p&gt;There are problems we for which we don't have a known, optimal, deterministic algorithm. By and large we use &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic&quot; rel=&quot;nofollow&quot;&gt;heuristics&lt;/a&gt; to &quot;solve&quot; those problems.  A closely related idea is that of &lt;a href=&quot;https://en.wikipedia.org/wiki/Satisficing&quot; rel=&quot;nofollow&quot;&gt;satisficing&lt;/a&gt; where we seek out answers that are &quot;good enough&quot; for immediate purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, machines can also use heuristics, whether they are programmed in explicitly or, presumably, learned.  Within the range of ways that a machine can use heuristics, there are &lt;a href=&quot;https://en.wikipedia.org/wiki/Metaheuristics&quot; rel=&quot;nofollow&quot;&gt;meta heuristics&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyper-heuristic&quot; rel=&quot;nofollow&quot;&gt;hyper heuristics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going beyond that, there are other ways that machines an learn &quot;algorithms&quot; or &quot;rules&quot; for solving problems.  One are that I'm particularly interested in is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Rule_induction&quot; rel=&quot;nofollow&quot;&gt;rule induction&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all an area of open and active research BTW... so if you're interested in exploring any of these approaches, you'll probably find a lot of ground to cover.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-09-29T03:30:28.413" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2035" PostTypeId="2" ParentId="2033" CreationDate="2016-09-29T03:35:38.537" Score="0" Body="&lt;p&gt;New guy here, please go easy on me as this answer will come from personal experience, and will probably be a tad philosophical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every algorithm I've designed was built to systematically tackle and solve specific problems in specific situations, each with an end goal in mind. Think of algorithms as solutions to a problem. In my career as a programmer, this rule has always stuck with me (it came from my favorite Computer Sciences professor): &quot;If there is &lt;strong&gt;no solution&lt;/strong&gt;, then there is &lt;strong&gt;no algorithm&lt;/strong&gt;. If there is &lt;strong&gt;no algorithm&lt;/strong&gt;, &lt;strong&gt;no machine can solve the problem&lt;/strong&gt;.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can machines generate their own algorithms? Most likely. But not to the point that it will exceed us (and by exceed, I don't mean just speed). AIs can never solve problems using methods that humans will never be able to come up with, because we programmed AIs to solve problems &lt;em&gt;just like us humans do&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2413" LastActivityDate="2016-09-29T03:35:38.537" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2036" PostTypeId="1" AcceptedAnswerId="2072" CreationDate="2016-09-29T08:58:44.087" Score="3" ViewCount="75" Body="&lt;p&gt;This is a question about a nomenclature - we already have the algorithm/solution, but we're not sure whether it qualifies as utilizing heuristics or not.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;feel free to skip the problem explanation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A friend is writing a path-finding algorithm - an autopilot for an&#xA;  (off-road) vehicle in a computer game. This is a pretty classic&#xA;  problem - he finds a viable, not necessarily optimal but &quot;good enough&quot;&#xA;  route using the A* algorithm, by taking the terrain layout and vehicle&#xA;  capabilities into account, and modifying a direct (straight) line path&#xA;  to account for these. The whole map is known a'priori and invariant,&#xA;  though the start and destination are arbitrary (user-chosen) and the&#xA;  path is not guaranteed to exist at all.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This cookie-cutter approach comes with a twist: limited storage space.&#xA;  We can afford some more volatile memory on start, but we should free&#xA;  most of it once the route has been found. The travel may take days -&#xA;  of real time too, so the path must be saved to disk, and the space in&#xA;  the save file for custom data like this is severely limited. Too&#xA;  limited to save all the waypoints - even after culling trivial&#xA;  solution waypoints ('continue straight ahead'), and by a rather large&#xA;  margin, order of 20% the size of our data set.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A solution we came up with is to calculate the route once on start,&#xA;  then 'forget' all the trivial and 90% of the non-trivial waypoints.&#xA;  This both serves as a proof that a solution exists, and provides a set&#xA;  of points reaching which, in sequence, guarantees the route will take&#xA;  us to the destination.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Once the vehicle reaches a waypoint, the route to the next one is&#xA;  calculated again, from scratch. It's known to exist and be correct&#xA;  (because we did it once, and it was correct), it doesn't put too much&#xA;  strain on the CPU and the memory (it's only about 10% the total route&#xA;  length) and it doesn't need to go into permanent storage (restarting&#xA;  from any point along the path is just a subset of the solution&#xA;  connecting two saved waypoints).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now for the actual question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pathfinding algorithm follows a sparse set of waypoints which by themselves are not nearly sufficient as a route, but allow for easy, efficient  calculation of the actual route, simultaneously guarantying its existence; they are a subset of the full solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a heuristic approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(as I understand, normally, heuristics don't guarantee existence of a solution, and merely suggest more likely candidates. In this case, the 'hints' are taken straight out of an actual working solution, thus my doubts.)&lt;/p&gt;&#xA;" OwnerUserId="38" LastActivityDate="2016-10-03T20:01:07.193" Title="Is a subset of a problem solution, used to recreate complete solution considered a heuristic?" Tags="&lt;heuristics&gt;&lt;path-planning&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2037" PostTypeId="1" AcceptedAnswerId="2044" CreationDate="2016-09-29T14:21:57.723" Score="2" ViewCount="3493" Body="&lt;p&gt;I understand how a neural network can be trained to recognise certain features in an image (faces, cars, ...), where the inputs are the image's pixels, and the output is a set of boolean values indicating which objects were recognised in the image and which weren't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't really get is, when using this approach to detect features and we detect a face for example, how we can go back to the original image and determine the location or boundaries of the detected face. How is this achieved? Can this be achieved based on the recognition algorithm, or is a separate algorithm used to locate the face? That seems unlikely since to find the face again, it needs to be recognised in the image, which was the reason of using a NN in the first place.&lt;/p&gt;&#xA;" OwnerUserId="2522" LastActivityDate="2017-08-28T10:41:30.590" Title="When using neural networks to detect features in an image, how can locate that specific feature in the original image?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2038" PostTypeId="2" ParentId="2036" CreationDate="2016-09-29T15:08:10.713" Score="1" Body="&lt;p&gt;Whether or not a label fits any particular instance depends on what you're using the label for. If something specific is riding on whether this approach is a 'heuristic' or not, that context is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I wouldn't call this a heuristic, because I think of that as a shortcut for &lt;em&gt;solving&lt;/em&gt; a problem, not either &lt;em&gt;storing&lt;/em&gt; a solution or &lt;em&gt;reformulating&lt;/em&gt; the problem (which is how I'd think of this). &lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-29T15:08:10.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2039" PostTypeId="2" ParentId="2033" CreationDate="2016-09-29T15:34:14.277" Score="1" Body="&lt;p&gt;All intelligence, both human and machine, is mechanistic. Thoughts don't appear out of the blue; they're generated through specific processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that if a machine generates an algorithm to solve a problem, even if the object-level algorithm wasn't generated by humans, the meta-level algorithm by which it generated the object-level algorithm must have come from &lt;em&gt;somewhere&lt;/em&gt;, and that somewhere is probably its original creators. (Even if they didn't program the meta-level algorithm, they probably programmed the meta-meta-level algorithm that programmed the meta-level algorithm, and so on.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How you think about these distinctions depends on how you think about machine learning, but typically they're fairly small. For example, when we train a neural network to classify images, we aren't telling it what pixels to focus on or how to combine them, which is the object-level algorithm that it eventually generates. But we are telling it how to construct that object-level algorithm from training data, what I'm calling the 'meta-level' algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the open problems is how to build the meta-meta-level; that is, an algorithm that will be able to look at a dataset and determine which models to train, and then which model to finally use. This will, ideally, include enough understanding of those meta-level models to construct new ones as needed, but even if it doesn't will reflect a major step forward in the usability of ML.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-09-29T15:34:14.277" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2040" PostTypeId="1" AcceptedAnswerId="2042" CreationDate="2016-09-29T15:44:48.963" Score="1" ViewCount="275" Body="&lt;p&gt;If said AI can assess scenarios and decide what AI is best suited and construct new AI for new tasks. In sufficient time would the AI not have developed a suite of AIs powerful/specialized for their tasks, but versatile as a whole, much like our own brain’s architecture? What’s the constraint ?&lt;/p&gt;&#xA;" OwnerUserId="2700" LastActivityDate="2016-09-30T17:24:55.247" Title="Wouldn't an AI that specializes in making other AI be an AGI if they can cooperate?" Tags="&lt;neural-networks&gt;&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2041" PostTypeId="2" ParentId="2037" CreationDate="2016-09-29T21:38:19.983" Score="3" Body="&lt;p&gt;The approach you listed here is not really an approach, this is very very vague idea of how someone can achieve some task. You basically told we have an algorithm &lt;code&gt;f(image) = result&lt;/code&gt; and there can be infinite amount of real approaches to solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In majority of CNN approaches the image travels through a convolution/pooling layers which reduces the dimensions of each current layer. In the end you end up with a significantly smaller layer which goes through the softmax and gets probabilities of different classes. This type of networks does not tell you where something was found, it just tells you that something was found somewhere in your original image.&lt;/p&gt;&#xA;" OwnerUserId="2492" LastActivityDate="2016-09-29T21:38:19.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2042" PostTypeId="2" ParentId="2040" CreationDate="2016-09-30T00:12:11.380" Score="0" Body="&lt;p&gt;If the AI can indeed assess arbitrary scenarios and come up with solutions to handle them, then it would indeed be an AGI.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What’s the constraint ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;It doesn't exist.&lt;/em&gt;  Current programmers are very good at developing AI that can handle specific tasks (&quot;narrow AIs&quot;), but it is currently impossible to build an AI that can assess and handle &quot;general&quot; situations (unlike your proposed algorithm, which possess that capacity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically, we can have a program that can build other programs (&lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow&quot;&gt;genetic algorithms&lt;/a&gt; are arguably one such example), but handling arbitrary scenarios and problems requires a form of &quot;general intelligence&quot;, which we don't know how to program. Therefore, we can't build this machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible that we can built this machine, but we must first figure out  the hard problem of &quot;general intelligence&quot;. We're nowhere near reaching that level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If&lt;/em&gt; we figure out how to program &quot;general intelligence&quot;, then it should be fairly simple to use your approach (building an AGI to assess scenarios and then build &quot;narrow AIs&quot; that can handle Those scenarios). &lt;em&gt;Only then&lt;/em&gt; we can understand the AGI's limitations and weaknesses, and be able to identify probable constraints to its power. For example, it's possible that such an AGI may be slow in handling arbitrary scenarios and developing the &quot;narrow AIs&quot;...in which case, it may take an absurdly long period of time to develop &quot;a suite of AIs powerful/specialized for their tasks&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But until we build the AGI itself, we won't be able to identify its faults or weaknesses. Going beyond that would be science-fiction speculation.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2016-09-30T00:12:11.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2044" PostTypeId="2" ParentId="2037" CreationDate="2016-09-30T12:42:03.763" Score="8" Body="&lt;p&gt;This problem is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Object_detection&quot; rel=&quot;noreferrer&quot;&gt;object detection&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a trainings set of images with boxed objects you can just train a neural network to directly predict the box. I.e. the output has the same dimension as the input and the NN learns to assign each pixel the probability of belonging to a certain object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have such a convenient dataset you could just recursively narrow the location down by feeding parts of the image to the network until you find the smallest part that still fully activates a certain classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/713f/73ce5c3013d9fb796c21b981dc6629af0bd5.pdf&quot; rel=&quot;noreferrer&quot;&gt;In this paper&lt;/a&gt; they try a mixture of these two approaches.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-09-30T12:42:03.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2045" PostTypeId="2" ParentId="2040" CreationDate="2016-09-30T17:24:55.247" Score="0" Body="&lt;p&gt;To build on Tariq Ali's answer... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no such thing as an AGI. The No Free Lunch (NFL) theorem states essentially that: &lt;em&gt;any two optimization algorithms are equivalent when their performance is averaged across all possible problems.&lt;/em&gt; Specialization implies a &lt;em&gt;loss&lt;/em&gt; of generality, not a &lt;em&gt;gain&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this AI generating AI, you're describing what I call an 'arbitrary machine generator' (AMG).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two types of AMGs: a species level and an individual level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All species that evolve on earth are AMG - they can evolve to accommodate arbitrary niches, if the correct environmental constraints are present. This is proven by the fact that the species AMG processes on earth have produce humans, which are individual level AMGs. Individual level AMGs can produce arbitrary machines for arbitrary purposes on human time-scales.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the simplest possible (and most general) AMG is a purely &lt;em&gt;random&lt;/em&gt; machine generator. Any more specificity (and therefore complexity) to the AMG would constrain the domains it closes over. Which is fine, but optimizing a machine for a particular set of tasks means that you are &lt;em&gt;unoptimizing&lt;/em&gt; the machine for some other particular set of tasks. Again, there's no free context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans are AMGs, but we are only efficient at creating certain kinds of machines, using our imagination. Our imagination is built on a number of cognitive tools that, on the one hand, constrain what machines we can efficiently imagine, while, on the other hand, avail us to the open-ended set of all possible machines, via prior knowledge or brute force, random lookup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, when people say &lt;em&gt;&quot;general intelligence&quot;&lt;/em&gt;, they really mean &lt;em&gt;&quot;human-like intelligence&quot;&lt;/em&gt;. And, again, while human intelligence &lt;em&gt;is&lt;/em&gt; an AMG, any given AMG that is optimized for generating machines of a particular type will be less optimized for generating machines of some other type(s). There's no free context. The most general search algorithm is a random walk - there is no way to &lt;em&gt;improve&lt;/em&gt; the generality of the random walk, other than just speeding it up. And that's actually what humans do for many problems anyway - brute force, random searching, as fast as possible.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-30T17:24:55.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2046" PostTypeId="2" ParentId="2020" CreationDate="2016-09-30T18:40:58.730" Score="2" Body="&lt;p&gt;Ethics involves the relationships of &lt;em&gt;needs&lt;/em&gt; between two or more parties. As Matthew Graves said, if the AI lacks the sufficient human context (understanding of needs), it will produce seemingly perverse ethical behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And let's be honest, some &lt;em&gt;people&lt;/em&gt; would cut of other people's arms and put them on pressure plates. Even the best of us will not be able to sympathize with the needs of others with 100% accuracy - at best, we're guessing. And then there are those rare situations where I actually &lt;em&gt;want&lt;/em&gt; you to cut off my arm and put it on a pressure plate, perhaps to save a loved one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we could make a thing that could sympathize with what a human &lt;em&gt;might&lt;/em&gt; need in any given arbitrary situation, then we will have created either A) an artificial &lt;em&gt;human&lt;/em&gt; intelligence (AHI) (which could be more or less fallible, like a human), or B) an oracle that can reason about &lt;em&gt;all possible human needs&lt;/em&gt; on much faster than human time-scales - in which case you wouldn't &lt;em&gt;need&lt;/em&gt; a conscious AI, as all human needs and solutions could be pre-computed via formal specification, which is probably absurd to consider.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-30T18:40:58.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2047" PostTypeId="2" ParentId="1885" CreationDate="2016-09-30T19:20:09.033" Score="2" Body="&lt;p&gt;If the universe is discrete, then analog phenomena (fluidity, curvature) are built on primitively discrete phenomena (bits and pieces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the universe is continuous, then discrete phenomena (bits and pieces) are built on primitively continuous phenomena (fluidity, curvature).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the universe is discrete, the speed of seemingly analog phenomena will be bounded by the number of discrete phenomena that can occur in time and space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the universe is continuous, then time, space or matter &lt;em&gt;may&lt;/em&gt; be infinitely divisible, which &lt;em&gt;may&lt;/em&gt; allow for the execution of some phenomena faster than those phenomena &lt;em&gt;appear&lt;/em&gt; to execute in natural environments (like protein folding or electric circuits) - so called &quot;super Turing&quot; computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The continuous universe idea begs the question, though: From whence came all this discreteness? A discrete universe can allow for apparent continuous behavior via approximation and randomness (or pseudorandomness), whereas a universe that is infinitely divisible affords no obvious definition of where things should start and end. This is one of the reasons many thinkers eschew considering infinities - they may be illusory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can analog &quot;circuits&quot; execute faster than digital? As of right now, we know of some &lt;em&gt;seemingly&lt;/em&gt; analog phenomena that &lt;em&gt;appear&lt;/em&gt; execute faster than some digital phenomena (like electron spin vs a silicon logic gate). Whether analog phenomena are &lt;em&gt;intrinsically&lt;/em&gt; more efficient than digital depends on the actual nature of the universe, which we have not yet determined.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-09-30T19:20:09.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2048" PostTypeId="1" AcceptedAnswerId="2052" CreationDate="2016-09-30T19:24:07.047" Score="5" ViewCount="1343" Body="&lt;p&gt;AI is progressing drastically, and imagine they tell you you're fired because a robot will take your place. What are some jobs that can never be automated?&lt;/p&gt;&#xA;" OwnerUserId="1760" LastEditorUserId="8" LastEditDate="2016-09-30T22:29:14.440" LastActivityDate="2017-04-12T21:47:09.410" Title="What jobs cannot be automatized by AI in the future?" Tags="&lt;philosophy&gt;&lt;robots&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2049" PostTypeId="2" ParentId="2048" CreationDate="2016-09-30T19:55:10.800" Score="1" Body="&lt;p&gt;If you were to completely automate a human, you'd just have another human, which defeats the purpose of the automation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any job that requires a &quot;whole human,&quot; rather than just a human's hands, feet, or simple reasoning ability, will still require humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I go to a shrink, one with Wikipedia-like knowledge would be great, but one that also actually knows what its like to rub its eyes in the morning would be even better. Why? Because solving &lt;em&gt;some&lt;/em&gt; problems will require knowing what it is like to rub one's eyes in the morning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I go to a movie that was written, directed and produced by some form of automation, I may be able to suspend my disbelief and get carried away by the story, but something in me will fundamentally appreciate the movie less, if I know that the AI can produce an infinite number of these stories, completely arbitrarily. There is something about knowing that the story came from a mind that has been conditioned against the vagaries of humanity (ie, a &quot;whole human&quot;), that makes the story more appreciable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I call up a suicide hotline because I wan't someone to sympathize with me about my existential crisis, I'll want to talk to a &quot;whole human&quot; that can &lt;em&gt;sympathize&lt;/em&gt; with my existential condition, not one that just regurgitates prior wisdom on life, heuristically matched against my problem state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to vote for a politician that can sympathize with the needs of the people, I'll want a &quot;whole person&quot; politician that can reflect on &lt;em&gt;all&lt;/em&gt; the specifics that make life for a &quot;whole person&quot; hard or easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want soldiers to take the lives of humans, I want some sort of intelligence in that kill-chain that executes &quot;whole person&quot; analysis prior to pulling the trigger (a human).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want a conflict resolution specialist, capable of resolving complex cultural problems between humans, then I don't want just an AI that spits out the most likely solution based on prior solutions. I want an AI that can reason about prior solutions &lt;em&gt;and&lt;/em&gt; all explicit and implicit problems between humans, in all human contexts, which requires a human or a perfect human simulacrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any problem that requires consideration potentially &lt;em&gt;across&lt;/em&gt; the whole spectrum of human context, we will want that solution to be generated by a &quot;whole human&quot; device. But if we automate the &quot;whole human&quot; then we haven't really outsourced the problem to automation but rather to a &quot;whole automated human,&quot; which will, by necessity, have its own problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure, we'll probably create an artificial human intelligence (AHI) one day, but being optimized to automatically solve &lt;em&gt;any&lt;/em&gt; given human problem without also &lt;em&gt;having&lt;/em&gt; human problems... that's just AI snake-oil that will never exist - outside of some perverse matrix scenario, under an infinite oracle of some sort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, yes, there will be many jobs that still require humans - mostly human-to-human problems that require full knowledge of the human context.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-09-30T20:28:30.460" LastActivityDate="2016-09-30T20:28:30.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2050" PostTypeId="2" ParentId="2048" CreationDate="2016-10-01T01:09:50.263" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What are some jobs that can never be automated?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;None.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key word here is &quot;never&quot;. Technology is rapidly advancing, and while I can think of situations where jobs can't be killed in the short-term or even in the long-term, I can't think of a job that is 100%, totally immune to extinction. Surely they &lt;em&gt;exist&lt;/em&gt;, but you can't be &lt;em&gt;sure&lt;/em&gt;...anything can happen after all. As long as it's &lt;em&gt;possible&lt;/em&gt;, that's what matters here. You can't prove a negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This whole question seems as foolhardy as predicting in the 1850s that airplanes would &lt;em&gt;never&lt;/em&gt; be invented. You'd be right in assuming that airplanes would not be invented in the 1860s...or the 1870s...or even the 1880s...but eventually, airplanes would be invented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be better is to provide a specific cut-off point (&quot;will all jobs be automated by the year 2020?&quot;) that can allow us to try to extrapolate and predict based on current trends, but even that starts being difficult as you extend the cut-off point - My predictions about 2020 will be more accurate than my prediction about 2220. I think this type of question is truly unanswerable and can quickly decay to science-fiction speculation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some additional comments about Doxosphoi's answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doxosophoi made some arguments for why current society might not accept the automation of all jobs (the need for the &quot;personal touch&quot; that only a human-like intelligence can provide), but that's no reason to assume that society will &lt;em&gt;never&lt;/em&gt; accept automation. Technology can change and adapt, and humans can also change and adapt. Maybe a human might not care about a shrink who &quot;rubs its eyes in the morning&quot;, dislike movies that are marred by that &quot;vagaries of humanity&quot; instead of personal customization, prefer politicians and soldiers that actually acts logically instead of acting like a falliable human, etc., etc. I mean, it's &lt;em&gt;possible&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also the problem of the term &quot;job&quot;. Technically, I am working by writing an answer on a StackExchange website, but I'm not getting paid for it, so it's not a real &quot;job&quot;...at best, it's just a hobby. I'm providing a valuable human touch, but since no one is giving me money, it's possible that this human touch may not be all that valuable in the first place: &quot;never give out your labor for free, because then they'll take it for free&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the techno-utopists (which I disagree with heavily) believes in a future where bots handle produce a lot of industrial goods and services, generating a lot of revenue that is then redistributed to the general population via some &quot;Basic Income&quot; scheme. This allows humans to do what they really want instead...such as hobbies? And what if the hobbies of the future are the &quot;jobs&quot; of today: shrinks, movie directors, politicians, soldiers, etc., etc. Instead of working for a paycheck, you're working in these jobs on a volunteer basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, no automation is being necessary to eliminate these hobbyists (no matter how good a bot is, &lt;em&gt;free labor&lt;/em&gt; will always prevail), but they're not really jobs, are they? The bot is the one that is producing real value, and subsidizing the hobbies of all these other people. The idea of a &quot;job&quot; itself could be in jeopardy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think this scenario is likely either (in fact, I'd probably think it's just AI snake oil that will never actually happen). But it's &lt;em&gt;possible&lt;/em&gt; and that's why I can't dismiss it outright. It could happen, just that I don't think it will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, the question is asking about whether a job can be automated, not &lt;em&gt;whether it's a good idea&lt;/em&gt; to have it be automated, which is a completely different question. It's possible that we can build machines that can automate everything, and choose as a society not to use them for a variety of different reasons (such as the reasons that Doxosophoi mentioned).&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2016-10-01T01:15:33.683" LastActivityDate="2016-10-01T01:15:33.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2052" PostTypeId="2" ParentId="2048" CreationDate="2016-10-01T10:11:17.110" Score="8" Body="&lt;p&gt;The Oxford study from 2013 in &lt;a href=&quot;http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The future of employment&lt;/a&gt; paper assess this and estimated the probability of computerisation for 702 detailed occupations using a Gaussian process classifier (using job data from the UK partially merged with data from US), and based on these estimates they identified three areas of computerisation bottleneck areas and nine skills that people are still needed for each profession, this includes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Perception and Manipulation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Finger dexterity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The ability to make precisely coordinated movements of&#xA;  the fingers of one or both hands to grasp, manipulate, or&#xA;  assemble very small objects.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Manual dexterity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The ability to quickly move your hand, your hand together&#xA;  with your arm, or your two hands to grasp, manipulate, or&#xA;  assemble objects.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The need for a cramped work space.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How often does this job require working in cramped work&#xA;  spaces that requires getting into awkward positions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Creative Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Originality.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The ability to come up with unusual or clever ideas about&#xA;  a given topic or situation, or to develop creative ways to&#xA;  solve a problem.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fine arts.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Knowledge of theory and techniques required to compose,&#xA;  produce, and perform works of music, dance, visual arts,&#xA;  drama, and sculpture.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Social Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Social perceptiveness.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Being aware of others’ reactions and understanding why&#xA;  they react as they do.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Negotiation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Bringing others together and trying to reconcile&#xA;  differences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Persuasion.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Persuading others to change their minds or behavior.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assisting and caring for others.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Providing personal assistance, medical attention, emotional&#xA;  support, or other personal care to others such as&#xA;  coworkers, customers, or patients.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Source: &lt;a href=&quot;http://web.archive.org/web/20161001101136/http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The future of employment: how susceptible are jobs to computerisation&lt;/a&gt;: Table 1.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this study is basically saying, around 50% of all jobs will be replaced by robots in the next 20 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the above study, the BBC assembled a handy guide that calculates which jobs are likely to be automated within the next two decades:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.bbc.co.uk/news/technology-34066941&quot; rel=&quot;nofollow noreferrer&quot;&gt;Will a robot take your job?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;http://www.replacedbyrobot.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;replacedbyrobot.info&lt;/a&gt; website.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With this tool, you can check the prediction of over 700 jobs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Related:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/AskReddit/comments/4mikie/when_robots_can_do_all_manual_labor_and_service/&quot; rel=&quot;nofollow noreferrer&quot;&gt;When robots can do all manual labor and service jobs, what will the majority human population do?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ted.com/talks/anthony_goldbloom_the_jobs_we_ll_lose_to_machines_and_the_ones_we_won_t&quot; rel=&quot;nofollow noreferrer&quot;&gt;TED: The jobs we'll lose to machines — and the ones we won't (by Anthony Goldbloom)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://medium.com/@tylerehc/labore-ad-infinitum-ai-automation-vs-timeless-tasks-ced2216f2ab7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Labore Ad Infinitum: AI &amp;amp; Automation vs Timeless Tasks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which suggests: Military/Peacekeeper, Athletes, Therapist, Musical Performer, Actors and Dancer, Visual Artists, Religious/Spiritual Leaders, The World’s Oldest Profession, Virtual Goods, Politicians, Judges, Parenting.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2017-04-11T16:00:33.377" LastActivityDate="2017-04-11T16:00:33.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2053" PostTypeId="2" ParentId="2033" CreationDate="2016-10-01T12:47:26.077" Score="1" Body="&lt;p&gt;I'd like to offer also a slightly different view on the machine cannot better its master.  Consider the very simple case of content classifiers.  It's already to the point where for some areas classification and prediction can be performed way better than a human.  And while a human may have designed the &quot;algorithm&quot;,  the algorithm was likely a recurrent neutral network or other form of ML that could well have self trained. In these cases we don't actually understand or need to understand the individual weights in the net,  as we would need to have traditionally understood the imperative programming constructs we used to write. It just works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if we get to where we develop a meta-algorithm for classifying problems and building more optimal deep learning solutions than we would by hand, but I think that would pretty much take us out of the picture for quite a lot of problem spaces.  Thoughts? &lt;/p&gt;&#xA;" OwnerUserId="2746" LastEditorUserId="8" LastEditDate="2016-10-01T13:04:32.967" LastActivityDate="2016-10-01T13:04:32.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2055" PostTypeId="1" AcceptedAnswerId="2057" CreationDate="2016-10-01T17:47:48.857" Score="2" ViewCount="129" Body="&lt;p&gt;I'm wondering how feasible it is to create a machine that can separate clothing from a basket.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the most basic level it would distinguish between tops, pants, button downs and socks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programmatically, I'd image this would require training a neural network to recognize these items, but in real time it becomes exponentially difficult to do this in a small space at a fast rate:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;pick up an item&lt;/li&gt;&#xA;&lt;li&gt;lay it in such a way that is recognizable &lt;/li&gt;&#xA;&lt;li&gt;deduce whether it is a top, button down, etc.&lt;/li&gt;&#xA;&lt;li&gt;sort it accordingly&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If this sounds ridiculous please let me know...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is possible :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;would this be based on some sort of computer vision?&#xA;or only a well trained neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insight is much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="2752" LastEditorUserId="2752" LastEditDate="2016-10-01T19:25:13.977" LastActivityDate="2016-10-01T23:48:41.117" Title="Feasibility of sorting a basket of clothes in the real world" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2056" PostTypeId="1" CreationDate="2016-10-01T19:00:56.123" Score="6" ViewCount="432" Body="&lt;p&gt;For years I have been dealing with (and teaching) Knowledge Representation and Knowledge Representation languages. I just discovered that in another community (Information Systems and the such) there is something called the &quot;DIKW pyramid&quot; where they add another step after knowledge, namely wisdom.&#xA;They define data as being simply symbols, information as being the answer to who/what/when/where?, knowledge as being the answer to how?, and wisdom as being the answer to why?. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: has anyone done the connection between what AI calls data/information/knowledge and these notions from Information Systems? In particular, how would &quot;wisdom&quot; be defined in AI? And since we have KR languages, how would we represent &quot;wisdom&quot; as they define it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references would be welcome…&lt;/p&gt;&#xA;" OwnerUserId="2753" LastActivityDate="2016-12-01T19:50:16.970" Title="Wisdom representation?" Tags="&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2057" PostTypeId="2" ParentId="2055" CreationDate="2016-10-01T23:48:41.117" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/~pabbeel/&quot; rel=&quot;nofollow&quot;&gt;Peter Abbeel&lt;/a&gt; does work in deep learning for robotics, and one of the projects they've tackled is manipulating clothes. Here's a &lt;a href=&quot;https://www.youtube.com/watch?v=5FGVgMsiv1s&quot; rel=&quot;nofollow&quot;&gt;video from 2011&lt;/a&gt; of their robot folding laundry (one piece at a time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also companies attempting to market this; seven dreamers makes the &lt;a href=&quot;https://laundroid.sevendreamers.com/?lang=en&quot; rel=&quot;nofollow&quot;&gt;Laundroid&lt;/a&gt; and &lt;a href=&quot;https://www.foldimate.com/&quot; rel=&quot;nofollow&quot;&gt;Foldimate&lt;/a&gt; claims that it will start taking pre-orders in 2017.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-10-01T23:48:41.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2058" PostTypeId="2" ParentId="2056" CreationDate="2016-10-02T04:52:31.447" Score="0" Body="&lt;p&gt;I haven't done the connection - didn't know about the pyramid. I'm not sure it translates well into AI though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems they're separating information from knowledge by splitting how from what. What is a superset of how, as far as I'm concerned. It's also a superset of why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But from an evolutionary perspective, knowledge representation starts with why. Prior to a reason for knowledge representation, there is no knowledge representation. The 'what' existed, but it was not represented until autopoiesis created goal directed, why-oriented behaviors that began storing the what as knowledge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a superset of why, just as ontology is a superset of teleology. However, all &lt;em&gt;represented&lt;/em&gt; ontology was acquired through teleological (end-directed) action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I disagree with the notion that wisdom, as a why thing, is at the tip of the pyramid. It all started with goal directed behavior and that has been the source of all subsequent information growth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is wisdom? I think it is too much of a folk term to warrant a technical definition. If I had to just take a swing at a definition, though, I'd probably vote for wisdom being knowledge of the ontological basis of one's own teleological knowledge - essentially objectifying one's subjective interpretations - knowing the true what and how of the why, to whatever extent is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have many &lt;em&gt;specific&lt;/em&gt; references on this subject, but I thought Terrence Deacon's &lt;a href=&quot;https://en.wikipedia.org/wiki/Incomplete_Nature&quot; rel=&quot;nofollow&quot;&gt;Incomplete Nature: How Mind Emerged from Matter&lt;/a&gt; was a good primer on teleology.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-10-02T04:52:31.447" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2061" PostTypeId="2" ParentId="2056" CreationDate="2016-10-02T08:23:45.937" Score="1" Body="&lt;p&gt;As with another answer, I am also skeptical of the distinctions made in the DIKW pyramid.&lt;/p&gt;&#xA;&lt;p&gt;Nonetheless, a very popular machine learning approach for answering 'Why?' questions is the application of Bayesian reasoning: given a causal data model, reverse inference can be used to find the probability distribution of events which lead to a given outcome.&lt;/p&gt;&#xA;&lt;p&gt;It could be argued that defining 'cause' in terms of distributions rather than specific concrete mechanisms is a rather limited notion of 'Why?'.&lt;/p&gt;&#xA;&lt;p&gt;However, it may be that there are some forms of causality that we don't know how to represent, specifically 'first-hand experience'. Indeed, common usage of the term 'wisdom' generally refers to first-hand experience, rather than information gained from some other source.&lt;/p&gt;&#xA;&lt;p&gt;The idea is that knowledge can be expressed declaratively, whereas wisdom must be derived from experience.&lt;/p&gt;&#xA;&lt;p&gt;For an AI represented as a computer program, the distinction between declarative and first-hand experience might appear irrelevant, since in principle any experience can be encoded and made available without the program having to 'experience' it first-hand.&lt;/p&gt;&#xA;&lt;p&gt;However, the following humorous definition of `wisdom' might perhaps shed some light on a distinction that's pertinent to AI research:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Knowledge is knowing that a tomato is a fruit.&lt;/p&gt;&#xA;&lt;p&gt;Wisdom is knowing that you shouldn't eat it with custard.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This notion of 'Wisdom' could be said to require &lt;a href=&quot;https://en.wikipedia.org/wiki/Qualia&quot; rel=&quot;nofollow noreferrer&quot;&gt;qualia&lt;/a&gt;. It is the subject of much debate whether qualia exist and/or are necessary for consciousness - see for example the thought experiment of &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_argument&quot; rel=&quot;nofollow noreferrer&quot;&gt;'The Black and White Room'&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So the notion is that there is a distinction between having a Bayesian network representation of wisdom that says: &amp;quot;It is 99.7% likely that putting a tomato in custard is undesirable&amp;quot; and the first-hand experience to the effect that it tastes odd with custard.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-10-02T18:29:44.360" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2062" PostTypeId="2" ParentId="1613" CreationDate="2016-10-02T15:44:04.267" Score="0" Body="&lt;p&gt;Autonomous vacuum cleaners usually have these following task environment properties:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i. Partially observable environment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii. Deterministic environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iii. Sequential environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iv. Static environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;v. Discrete environment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;vi. Single agent environment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it's a partially observable environment, the agent performs its actions based on what is sees and thus it's is in an deterministic environment where the current action will be the result of previous actions. And since, the agent is continuously performing, it's an sequential environment. Since, the environment doesn't change when agent is working, the environment is static and discrete since there are only finite no. of discrete states in which agent can be in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are certain rules written which tells an agent how to react when in a particular state. If there are multiple rules which are satisfied, then agent uses its experience to choose an optimal action to perform. The rules are written by the programmer keeping in mind the task envt. properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent's actions also depend on the type of agent it is decided by the programmer. It can be simple reflex-based, or a goal based, or a goal-based + feedback or a complete learning based agent. An agent can't use A* algorithm because the entire environment is not visible and it will be useless to use A* algorithm where we don't know when our goal may be reached.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent has various sensors attached which give it info. about the surrounding, like cameras, sound recorders, thermal sensors, etc. An autonomous vacuum cleaner may also have a dirt sensor to detect the dirt. The agent performs an action using one of its actuators like wheels, robot arms, etc. &lt;/p&gt;&#xA;" OwnerUserId="1807" LastActivityDate="2016-10-02T15:44:04.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2065" PostTypeId="2" ParentId="2036" CreationDate="2016-10-02T18:49:55.890" Score="1" Body="&lt;p&gt;A 'heuristic' is simply a 'rule-of-thumb', i.e. something which doesn't guarantee an optimal solution to a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond the above notion (certainly within the discipline of optimization), the notion of what constitutes a heuristic is not particularly strict, and could certainly include hints for constructing a new solution from parts of a previous one, as you are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related concrete example is the &quot;nearest neighbour heuristic&quot; for the Travelling Salesman Problem, in which a solution is constructed by starting at some random city and iteratively choosing the nearest. The resulting completed tour is then used as an initial input to some more sophisticated optimization procedure.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-10-02T18:49:55.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2066" PostTypeId="1" AcceptedAnswerId="2068" CreationDate="2016-10-02T20:02:47.463" Score="8" ViewCount="194" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AI_effect&quot; rel=&quot;noreferrer&quot;&gt;According to Wikipedia&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not real intelligence.&lt;/p&gt;&#xA;&lt;p&gt;Pamela McCorduck writes: &amp;quot;It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was chorus of critics to say, 'that's not thinking'.&amp;quot;[1] AI researcher Rodney Brooks complains &amp;quot;Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'&amp;quot;[2]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The Wikipedia page then proposes several different reasons that could explain why onlookers might &amp;quot;discount&amp;quot; AI programs. However, those reasons seem to imply that the humans are making a mistake in &amp;quot;discounting&amp;quot; the behavior of AI programs...and that these AI programs might actually be  intelligent. I want to make an alternate argument, where the humans are making a mistake, but not in &amp;quot;discounting&amp;quot; the behavior of AI programs.&lt;/p&gt;&#xA;&lt;p&gt;Consider the following situation. I want to build a machine that can do X (where X is some trait, like intelligence). I am able to evaluate intuitively whether a machine has that X criteria. But I don't have a good definition of what X actually &lt;em&gt;is&lt;/em&gt;. All I can do is identify whether something has X or not.&lt;/p&gt;&#xA;&lt;p&gt;However, I think that people who has X can do Y. So if I build a machine that can do Y, then surely, I built a machine that has X.&lt;/p&gt;&#xA;&lt;p&gt;After building the machine that can do Y, I examine it to see if my machine has X. And it does not. So my machine lacks X. And while a machine that can do Y is cool, what I really want is a machine that has X. I go back to the drawing board and think of a new idea to reach X.&lt;/p&gt;&#xA;&lt;p&gt;After writing on the whiteboard for a couple of hours, I realize that people who has X can do Z. Of course! I try to build a new machine that can do Z, yes, if it can do Z, then it must have X.&lt;/p&gt;&#xA;&lt;p&gt;After building the machine that can do Z, I check to see if it has X. It does not. And so I return back to the drawing board, and the cycle repeats and repeats...&lt;/p&gt;&#xA;&lt;p&gt;Essentially, humans are attempting to determine whether an entity has intelligence via proxy measurements, but those proxy measurements are potentially faulty (as it is possible to meet those proxy measurements without ever actually having intelligence). Until we know how to define intelligence and design a test that can accurately measure it, it is very unlikely for us to build a machine that has intelligence. So the AI Effect occurs because humans don't know how to define &amp;quot;intelligence&amp;quot;, not due to people dismissing programs as not being &amp;quot;intelligent&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Is this argument valid or correct? And if not, why not?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-10-02T23:49:18.177" Title="Is the AI Effect caused by bad tests of intelligence?" Tags="&lt;intelligence-testing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2068" PostTypeId="2" ParentId="2066" CreationDate="2016-10-02T22:18:25.650" Score="7" Body="&lt;p&gt;I think it is mostly right. But not that intelligence is hard to define. In my opinion, it is simple: A is more intelligent than B if A achieves some purpose in less steps than B. It is functional/algorithmic efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is difficult to define is &lt;em&gt;human&lt;/em&gt; intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when someone says, &quot;No, X is not &lt;em&gt;real&lt;/em&gt; intelligence,&quot; what they mean is that it does not satisfy what we would consider real &lt;em&gt;human intelligence&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when people discount new and amazing discoveries in machine intelligence, it is not because they are not amazing in their own way, but because those discoveries, while exhibiting intelligence, are &lt;strong&gt;actually not&lt;/strong&gt; replicating human intelligence - which is what many people actually mean when they say &quot;that thing isn't &lt;em&gt;really&lt;/em&gt; intelligent.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In truth we are very far, in the science of artificial intelligence, algorithmically speaking, from an artificial &lt;em&gt;human&lt;/em&gt; intelligence (AHI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional note: What's funny is that we don't call the science of artificial intelligence just 'the science of intelligence.' That we add the &quot;artificial&quot; qualifier by necessity pegs the science to what the artificiality implicitly emulates: human intelligence. In other words, &quot;artificial intelligence&quot; must be by definition more specific to the thing it allegedly artificializes than a more general science of just &quot;intelligence.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-10-02T22:40:34.953" LastActivityDate="2016-10-02T22:40:34.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2069" PostTypeId="1" AcceptedAnswerId="2070" CreationDate="2016-10-03T02:27:05.757" Score="2" ViewCount="316" Body="&lt;p&gt;Here is one of the most serious questions, about the artificial intelligence.&lt;br&gt;&#xA;How will the machine know the difference between right and wrong, what is good and bad, what is respect, dignity, faith and empathy.&lt;br&gt;&#xA;&lt;br&gt; A machine can recognize what is correct and incorrect, what is right and what is wrong, depend on how it is originally designed.&lt;br&gt;&#xA;&lt;br&gt;It will follow the ethics of its creator, the man who originally designed it&lt;br&gt;&#xA; But how to teach a computer something we don't have the right answer.&lt;br&gt;&#xA; People are selfish, jealous, self confident. We are not able to understand each other sorrows, pains beliefs. We don't understand different religions, different traditions or beliefs. &#xA;&lt;br&gt;Creating an AI might be breakthrough for one nation, or one race, or one ethnic or religious group, but it can be against others.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Who will learn the machine a humanity?   :)&lt;/p&gt;&#xA;" OwnerUserId="2682" LastEditorUserId="1671" LastEditDate="2017-10-27T20:47:12.067" LastActivityDate="2018-01-20T03:24:04.567" Title="How will an AI comprehend the ethics of &quot;right&quot; and &quot;wrong&quot;?" Tags="&lt;ethics&gt;&lt;value-alignment&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2070" PostTypeId="2" ParentId="2069" CreationDate="2016-10-03T05:53:35.637" Score="7" Body="&lt;p&gt;Right and wrong only exist relative to some goal or purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make a machine do more right than wrong, relative to human goals, one should minimize the surface area of the machine's purpose. Doing that minimizes the intrinsic behavior of the AI, which enables us to reason about the right and wrong behaviors of the AI, relative to human purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Horses are quite general over the domains of their purposes, but are still predictable enough for humans to control and benefit from. As such, we will be able to produce machines (conscious or unconscious) that are highly general over particular domains, while still being predictable enough to be useful to humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most efficient machines for most tasks, though, will not &lt;em&gt;need&lt;/em&gt; consciousness, nor even the needs that cause survivalistic, adversarial and self-preserving behaviors in eukaryotic cells. Because most of our solutions won't &lt;em&gt;need&lt;/em&gt; those purposes to optimize over our problems, we can allow them to be much more predictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will be able to create predictable, efficient AIs over large domains that are able to produce predictable, efficient AIs over more specific domains. We'll be able to reason about the behavioral guarantees and failure modes of those narrow domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the event that we one day desire to build something as unpredictable as a human, like we do when having babies, then we'll probably do that with the similar intentions and care that we use to bring an actual baby into the world. There is simply no purpose in creating a thing more unpredictable than you unless you're gambling on this thing &lt;em&gt;succeeding&lt;/em&gt; you in capability - which sounds exactly like having babies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, the best we can do is give it our &lt;em&gt;theories&lt;/em&gt; about why we think we should act one way or another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, theoretically, some extremely powerful AI could potentially construct a human-like simulacrum that, in many common situations, seems to act like a human, but that in fact has had all of it's behaviors formally specified a priori, via some developmental simulation, such that we &lt;em&gt;know for a fact&lt;/em&gt; that all such behaviors produce no intentional malice or harm. However, if we can formally specify all such behaviors, we wouldn't be using this thing to solve any novel problems, like curing cancer, as the formal specification for curing cancer would already have been pre-computed. If you can formally specify the behaviors of a thing that can discover something new, you can just compute the solution via the specification, without instantiating the behaviors at all!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once AI has reached a certain level of capability, it won't need to generate consciousnesses to derive optimal solutions. And at that point, the only purpose for an artificial human to exist will be, like us, for its own sake.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-10-03T06:09:20.053" LastActivityDate="2016-10-03T06:09:20.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2072" PostTypeId="2" ParentId="2036" CreationDate="2016-10-03T20:01:07.193" Score="1" Body="&lt;p&gt;Since you mention the &lt;strong&gt;A* algorithm&lt;/strong&gt;, then you are definitely using a heuristic somewhere in there, at least with the A* algorithm while solving the subproblems using the straight-line distance as your &lt;strong&gt;heuristic function&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although your approach does not seem to incorporate a &quot;shortcut mathematical formula&quot; as a heuristic after that, it does us a precalculated &lt;strong&gt;heuristic table&lt;/strong&gt; as a reference and that may qualify as a heuristic. Wikipedia page &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic_(computer_science)&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; says (although without citation) the following which does seem to describe what you are doing where your heuristic is not a fixed but a precalculated function/table:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A heuristic function, also called simply a heuristic, is a function&#xA;  that ranks alternatives in search algorithms at each branching step&#xA;  based on available information to decide which branch to follow. For&#xA;  example, it may approximate the exact solution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On another note, your method also seems to have hints of &lt;strong&gt;dynamic programming&lt;/strong&gt; since you use the precalculated and stored solutions to subproblems and instead of recalculating every time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if the term &lt;strong&gt;approximate dynamic programming&lt;/strong&gt; would fit this situation as a non-stochastic version with no uncertainties? Unfortunately I could not find any simple description or categorization for that term.&lt;/p&gt;&#xA;" OwnerUserId="2680" LastActivityDate="2016-10-03T20:01:07.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2075" PostTypeId="2" ParentId="1982" CreationDate="2016-10-04T08:56:06.643" Score="1" Body="&lt;p&gt;It would not be wise to say that CNNs are better objectively than traditional approaches to solve computer vision problems as there are many problems for which the traditional methods works just fine. CNNs do have an inherent advantage over traditional methods which is the same advantage that deep learning has over other traditional methods i.e learning hierarchical features i.e what features are useful and how to compute them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The traditional way to approach a CV problem is to figure out the features that are relevant to the problem, figure out how to compute those features and then use those features to compute the final result. Whereas in CNN case the training process will figure out all the 3 points for you given that you have huge number of training examples.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastEditorUserId="1462" LastEditDate="2016-10-07T10:19:50.993" LastActivityDate="2016-10-07T10:19:50.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2078" PostTypeId="1" AcceptedAnswerId="2079" CreationDate="2016-10-04T19:06:15.487" Score="3" ViewCount="248" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/2067/will-ai-be-able-to-adapt&quot;&gt;From this SE question&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will be AI able to adapt, to different environments and changes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is my attempt at interpreting that question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evolutionary algorithms are useful for solving optimization problems...by measuring the &quot;fitness&quot; of various probable solutions and then  of an algorithm through the process of natural selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose, the &quot;fitness calculation&quot;/&quot;environment&quot; is changed in mid-training (as could easily happen in real-life scenarios where people may desire different solutions at different times). Would evolutionary algorithms be able to respond effectively to this change?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2019-12-20T07:38:28.260" Title="Can an evolutionary algorithm adapt to a changing environment?" Tags="&lt;evolutionary-algorithms&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2079" PostTypeId="2" ParentId="2078" CreationDate="2016-10-04T19:58:18.483" Score="5" Body="&lt;p&gt;The core question to whether or not an AI is adaptable or not is whether or not it supports &lt;a href=&quot;https://en.wikipedia.org/wiki/Online_machine_learning&quot; rel=&quot;noreferrer&quot;&gt;online learning&lt;/a&gt;. That doesn't mean using the Internet to learn things; that means continuing to accept training data during the functioning of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is (mostly) independent of the underlying architecture; in evolutionary approaches one can continue to breed generations with a shifting fitness function or with neural networks one can continue to backpropagate errors, and so on with other approaches.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-10-04T19:58:18.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2080" PostTypeId="2" ParentId="2048" CreationDate="2016-10-05T17:16:41.537" Score="1" Body="&lt;p&gt;Probably the only secure jobs are those where the audience enjoys watching live human craftsmanship take place in real time right before their eyes, like acting or standup comedy or musical virtuosity or playing a sport.  Watching a robot do the same thing would be far less personally engaging since there's no human skill or artistry to appreciate or identify with, escpecially when the pressure is on or when human interpersonal dynamics are involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, why would anyone watch robots play poker?  Or dance?  Or do standup comedy about how hard it is to be [ethnic group or gender goes here]?&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-10-05T17:16:41.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2081" PostTypeId="2" ParentId="2078" CreationDate="2016-10-05T17:35:14.380" Score="2" Body="&lt;p&gt;I think Matthew Graves' answer is the strictly correct one. But I also think this question may be hinting at a larger question in general. What is the minimal algorithmic complexity required for a machine of one particular set of functions to mutate into some other machine of some other particular set of functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is: potentially &lt;em&gt;infinite&lt;/em&gt; algorithmic complexity. Without knowing a priori how many steps it will take to mutate into a thing that can solve some black-box problem, there is no way to determine if and when the AI will be able to mutate into that thing.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-10-05T17:35:14.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2082" PostTypeId="2" ParentId="4" CreationDate="2016-10-06T02:57:54.343" Score="5" Body="&lt;p&gt;Paper &lt;a href=&quot;https://arxiv.org/pdf/1512.00567.pdf&quot; rel=&quot;nofollow&quot;&gt;Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[J]. arXiv preprint arXiv:1512.00567, 2015.&lt;/a&gt; gives some general design principles:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Avoid representational bottlenecks, especially early in&#xA;  the network;&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;Balance the width and depth of the network. Optimal&#xA;  performance of the network can be reached by balancing&#xA;  the number of filters per stage and the depth of&#xA;  the network. Increasing both the width and the depth&#xA;  of the network can contribute to higher quality networks.&#xA;  However, the optimal improvement for a constant&#xA;  amount of computation can be reached if both are&#xA;  increased in parallel. The computational budget should&#xA;  therefore be distributed in a balanced way between the&#xA;  depth and width of the network.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These suggestions can't bring you the optimal number of neurons in a network though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are still some model compression research e.g. &lt;a href=&quot;https://github.com/wenwei202/caffe/tree/scnn&quot; rel=&quot;nofollow&quot;&gt;Structured Sparsity Learning (SSL) of Deep Neural Networks&lt;/a&gt;, &lt;a href=&quot;https://github.com/songhan/SqueezeNet-Deep-Compression&quot; rel=&quot;nofollow&quot;&gt;SqueezeNet&lt;/a&gt;, &lt;a href=&quot;http://papers.nips.cc/paper/5784-learning-both-weights-and-connections-for-efficient-neural-network.pdf&quot; rel=&quot;nofollow&quot;&gt;Pruning network&lt;/a&gt; that may shed some light on how to optimizing the neurons per single layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially in &lt;a href=&quot;https://github.com/wenwei202/caffe/tree/scnn&quot; rel=&quot;nofollow&quot;&gt;Structured Sparsity Learning of Deep Neural Networks&lt;/a&gt;, it adds a &lt;code&gt;Group Lasso&lt;/code&gt; regularization term in the loss function to to regularize the structures(i.e., filters, channels, filter shapes, and layer depth) of DNNs, which namely is to zero out some components(i.e., filters, channels, filter shapes, and layer depth) of the net structure and achieves a remarkable compact and acceleration of the network, while keeps a small classification accuracy loss. &lt;/p&gt;&#xA;" OwnerUserId="1750" LastEditorUserId="1750" LastEditDate="2016-10-11T01:02:00.430" LastActivityDate="2016-10-11T01:02:00.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2083" PostTypeId="1" AcceptedAnswerId="2085" CreationDate="2016-10-06T10:26:13.423" Score="5" ViewCount="152" Body="&lt;p&gt;In a classic example of a genetic algorithm, you would have a population and a certain amount of simulation time to evaluate it and breeding. Then proceed to the next generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible, during the simulation process, to have an isolated and small part of the population and keep it evolving in their own little island for some time while the rest of the population continues to evolve normally? &#xA;After that, they could be reunited with the rest of the population and the end of the simulation would go through. After that, breed the population and continue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a super important part of natural evolution and probably some know if it actually works with genetic programming?&lt;/p&gt;&#xA;" OwnerUserId="2825" LastEditorUserId="2444" LastEditDate="2021-01-19T19:10:58.163" LastActivityDate="2021-01-19T19:10:58.163" Title="Is it possible to separately evolve a part of the population?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;&lt;niching&gt;&lt;island-models&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2085" PostTypeId="2" ParentId="2083" CreationDate="2016-10-06T13:34:45.590" Score="5" Body="&lt;p&gt;There have been extensive studies within &lt;em&gt;evolutionary computation&lt;/em&gt; in the area of &lt;a href=&quot;http://cs.gmu.edu/%7Eeclab/papers/skolicki05analysis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;island models&lt;/em&gt;&lt;/a&gt; and &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.30.8270&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;niching&lt;/em&gt;&lt;/a&gt; for doing exactly this.&lt;/p&gt;&#xA;&lt;p&gt;The advantages of this approach include greater population diversity (which is particularly useful when the problem is multiobjective) and the potential for concurrent execution of each separate population.&lt;/p&gt;&#xA;&lt;p&gt;See also the answers to the question &lt;a href=&quot;https://stackoverflow.com/q/13775810/3924118&quot;&gt;What is the niching scheme?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;With specific reference to &lt;em&gt;genetic programming&lt;/em&gt;, &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0743731506001067&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is a recent paper that uses a parallel island model.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2021-01-19T19:07:06.387" LastActivityDate="2021-01-19T19:07:06.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2086" PostTypeId="2" ParentId="2021" CreationDate="2016-10-06T13:36:40.140" Score="1" Body="&lt;p&gt;In any case, it appears that much of his past work on Github involves &lt;em&gt;procedural generation&lt;/em&gt;. Which is AI... ish. Unless there's more to it, which we can't see because half the site is down.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.mcgill.ca/%7Ecdrage/papers/SOCS-TR-2011.1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; appears to offer an analysis of combining procedural generation with game AI.&lt;/p&gt;&#xA;&lt;p&gt;From the abstract:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Populated and immersive game contexts require large numbers of minor,&#xA;background characters to fill out the virtual environment. To limit&#xA;game AI development effort, however, such characters are typically&#xA;represented by very simplistic AI with either little difference&#xA;between characters or only highly formulaic variations. Here we&#xA;describe a complete workflow and framework for easily designing,&#xA;generating and incorporating multiple, interesting game AIs. Our&#xA;approach uses high-level, visual Statechart models to represent&#xA;behaviour in a modular form; this allows for not only simplistic,&#xA;parameterbased variation in AI design, but also permits more complex&#xA;structure-based approaches. We demonstrate our technique by applying&#xA;it to the task of generating a large number of individual AIs for&#xA;computer-controlled squirrels within the Mammoth &lt;a href=&quot;http://www.cs.mcgill.ca/%7Ecdrage/papers/SOCS-TR-2011.1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; framework for&#xA;game research. Rapid development and easy deployment of AIs allow us&#xA;to create a wide variety of interesting AIs, greatly improving the&#xA;sense of immersion in a virtual environment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://newatlas.com/creative-ai-procedural-game-development-angelina/35874/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an article&lt;/a&gt; from 2015 on AI and procedural generation, which discusses Angelina at length. This article links to &lt;a href=&quot;http://www.eurogamer.net/articles/2013-04-02-plastic-soul-one-mans-quest-to-build-an-ai-that-can-create-games&quot; rel=&quot;nofollow noreferrer&quot;&gt;a more in-depth article&lt;/a&gt; from 2013.&lt;/p&gt;&#xA;&lt;p&gt;Here's an excerpt:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Cook gave ANGELINA the ability to learn about people so that it could&#xA;make games based on current events. Then Cook gave ANGELINA memory -&#xA;that is, the ability to keep track of the people it had learned about.&#xA;The memory's not a big deal, even though it led to a number of&#xA;philosophical disagreements around Cook's desk. ANGELINA's memory is&#xA;actually just a text file where it stores the names of all the people&#xA;it's heard of, alongside a number: a measure of its opinion of them&#xA;based on the things it's learned from internet chatter. It liked&#xA;Al-Assad more than May. It liked everyone more than May.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="2444" LastEditDate="2020-11-19T16:09:29.867" LastActivityDate="2020-11-19T16:09:29.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2087" PostTypeId="1" AcceptedAnswerId="2088" CreationDate="2016-10-06T20:24:57.260" Score="4" ViewCount="101" Body="&lt;p&gt;Obviously this is hypothetical, but is true? I know &quot;perfect fitness function&quot; is a bit hand-wavy, but I mean it as we have a perfect way to measure the completion of any problem.&lt;/p&gt;&#xA;" OwnerUserId="2818" LastActivityDate="2016-10-08T06:15:43.077" Title="Given unlimited time and a perfect fitness function, could a genetic program solve any problem?" Tags="&lt;genetic-programming&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2088" PostTypeId="2" ParentId="2087" CreationDate="2016-10-06T20:29:52.853" Score="1" Body="&lt;p&gt;&lt;strong&gt;Yes.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A totally random algorithm could solve any problem given unlimited time and a perfect fitness function. All you need to do is give the GA some new random population members each generation and you're guaranteed to find the solution eventually. Even if you keep only descendants of the previous generation, setting the mutation rate and number of crossovers high enough could effectively get you random individuals.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2016-10-06T20:29:52.853" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2089" PostTypeId="2" ParentId="1982" CreationDate="2016-10-07T07:25:24.487" Score="0" Body="&lt;p&gt;Neural net approaches  are very different than other techniques, mostly because NN aren't &quot;linear&quot; like feature matching or cascades. With very complicated tasks like realtime object recognition or other difficult patterns it's better to use neural net, first because if you train it well your net , you can get very high precision, second it' easier to implement (it depends a lot from library to library)  third usually after you have trained it they are very fast to classify or predict something. But a lot of tasks don't need neural nets, for example many factories to check the products use 3D features model matching. At the end you have to  evaluate which method is the best for your task&lt;/p&gt;&#xA;" OwnerUserId="2320" LastActivityDate="2016-10-07T07:25:24.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2090" PostTypeId="2" ParentId="2087" CreationDate="2016-10-07T12:37:10.003" Score="1" Body="&lt;p&gt;As per the answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/1541/why-is-cross-over-a-part-of-genetic-algorithms/1548#1548&quot;&gt;this AI SE question&lt;/a&gt;,&#xA;the presence of mutation makes a GA into a global search algorithm, i.e. it will eventually visit each point in the search space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How &lt;em&gt;efficiently&lt;/em&gt; it will do so is indeed related to the quality of the fitness function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 'perfect fitness function' could conceivably mean any of the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A function  which takes on an optimal value iff it is applied to&#xA;an optimal solution. &lt;/li&gt;&#xA;&lt;li&gt;A function which forms a quadratic bowl&#xA;(Newton's method can solve this in one step).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A degenerate case of 1. is a 'Needle in a haystack' function, which returns the same arbitrarily poor value everywhere that isn't an optimum, and 2. unfortunately doesn't arise very often in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, the role of a well-designed fitness function is to impose a gradient on the search process, which in practice will generally lie somewhere in-between 'Needle in a haystack' and 'Quadratic bowl'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The presence of some form of smoothness in the fitness function is one mechanism that allows better performance than random or exhaustive methods.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-10-07T12:43:14.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2092" PostTypeId="1" AcceptedAnswerId="2098" CreationDate="2016-10-07T21:57:02.907" Score="10" ViewCount="528" Body="&lt;p&gt;I'm curious about Artificial Intelligence. In my everyday job I develop standard applications, like websites with basic functionalities like user subscription, file upload, forms saved in a database... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mainly know of AI being used in games or robotics fields. But can it be useful in &quot;standard&quot; application development?&lt;/p&gt;&#xA;" OwnerUserId="2862" LastEditorUserId="-1" LastEditDate="2016-10-21T13:58:02.750" LastActivityDate="2016-10-24T08:05:09.393" Title="Is AI programming useful in everyday programs?" Tags="&lt;applications&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2093" PostTypeId="2" ParentId="1354" CreationDate="2016-10-07T23:29:59.280" Score="2" Body="&lt;p&gt;&lt;strong&gt;Have the user label highlighted objects in video that a state of the art classifier cannot solve&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a state of the art video classifier. Might as well train it on Google's &lt;a href=&quot;https://research.googleblog.com/2016/09/announcing-youtube-8m-large-and-diverse.html&quot; rel=&quot;nofollow&quot;&gt;YouTube-8M&lt;/a&gt; video training data. But you will want to continually feed it original video as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have the classifier label as many objects as it can. Have it isolate which objects it can recognize as objects but which it is unable to label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have it output videos that outlines the objects. Preferably GIFs, which can be easily embedded in forms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 100 of these, ask 100 users what the object is. If 90% of the users agree on the name of an object, add that video to the captcha-set. Call this the pre-trained set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every time a user needs to authenticate, show them one of the highlighted objects in a video &lt;em&gt;not from the pre-trained set&lt;/em&gt;. If the image has less than 100 showings, record the label and give the user another one from the pre-trained set. If they get it right, let them through, if not, give them another from the pretrained set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the non-pre-trained video has more than 100 showings and more than 90% of the captcha-users agree, add that video to the post-trained set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over time, slowly remove the pre-trained set. Put expirations on each video in the post-trained set and remove them after expiration, so that they don't get used too many times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, this process would constantly improve the video classifier, keeping it state of the art and slightly ahead of other classifiers. Perhaps it could also favor less common words and objects and more esoteric things, so as to specialize this classifier against other classifiers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same could be done for image labeling, but the utility of the video classifier will probably last longer, given advances in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strictly speaking, though, short of some quantum trickery, there is no captcha system that will not one day be solved by external AI systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(edit: oh, I just noticed you specifically said &quot;textual captcha.&quot; If that's what you mean, then no I don't think text classification has much mystery left in it. Computers can probably glean text from pictures better than humans now. But techically, the &lt;em&gt;input&lt;/em&gt; in the above described captcha system is textual.) &lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-10-09T18:16:26.393" LastActivityDate="2016-10-09T18:16:26.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2094" PostTypeId="2" ParentId="218" CreationDate="2016-10-08T00:11:37.960" Score="1" Body="&lt;p&gt;Use something like Word2Vec. If a particular node has two edges that are very far from each other, besides the node in question, split the node into word(1) and word(2) nodes.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-10-08T00:11:37.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2095" PostTypeId="2" ParentId="1451" CreationDate="2016-10-08T01:47:41.487" Score="6" Body="&lt;p&gt;&lt;strong&gt;No.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR: The Lovelace Test 2.0 is very vague, making it ill-suited for  evaluation of intelligence. It is also generally ignored by researchers of Computational Creativity, who already have their own tests to evaluate creativity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longer Answer:&#xA;According to Google Scholar, there are 10 references to the &quot;Lovelace Test 2.0&quot; paper. All of those references exist merely to point out that the Lovelace Test 2.0 exists. In fact, at least two of articles I consulted (&lt;a href=&quot;http://philpapers.org/archive/NEGANA.pdf&quot; rel=&quot;noreferrer&quot;&gt;A novel approach for identifying a human-like self-conscious behavior&lt;/a&gt; and &lt;a href=&quot;http://users.dsic.upv.es/~flip/EGPAI2016/papers/EGPAI_2016_paper_8.pdf&quot; rel=&quot;noreferrer&quot;&gt;FraMoTEC: A Framework for Modular Task-Environment Construction for Evaluating Adaptive Control Systems&lt;/a&gt;) proposed their &lt;em&gt;own&lt;/em&gt; tests instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the authors who wrote the FraMoTEC paper also wrote &lt;a href=&quot;http://skemman.is/stream/get/1946/25590/58121/1/scs-thorarensen2016-thesis.pdf&quot; rel=&quot;noreferrer&quot;&gt;his thesis on FraMoTEC&lt;/a&gt;, and indirectly critiqued the Lovelace Test 2.0 and other similar such tests:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Piaget-MacGyver Room problem [Bringsjord and Licato, 2012], Lovelace Test 2.0 [Riedl, 2014] and Toy Box problem [Johnston, 2010] all come with the caveat of being defined very vaguely — these evaluation methods may be likely to come up with a reasonable evaluation for intelligence, but it is very difficult to compare two different agents (or controllers) that partake in the their own domain-specific evaluations, which is what frequently happens when agents are tailored to pass specific evaluations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Another major issue with the Lovelace Test 2.0 is that there is a proliferation of &lt;em&gt;other&lt;/em&gt; tests to &quot;measure&quot; the creativity of AI. &lt;a href=&quot;https://kar.kent.ac.uk/42374/1/jordanous-2011a.pdf&quot; rel=&quot;noreferrer&quot;&gt;Evaluating Evaluation: Assessing Progress in Computational Creativity Research&lt;/a&gt;, published by &#xA;Anna Jordanous in 2011 (3 years &lt;em&gt;before&lt;/em&gt; the invention of the Lovelace Test 2.0) analyzed research papers about AI creativity and wrote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Of the 18 papers that applied creativity evaluation methodologies to evaluate their system’s creativity, no one methodology emerged as standard across the community. Colton’s creative tripod framework (&lt;a href=&quot;https://pdfs.semanticscholar.org/fd8b/d5bb76c94c4bfc34cb1fa244dc6bd4a8ca8e.pdf&quot; rel=&quot;noreferrer&quot;&gt;Colton 2008&lt;/a&gt;) was used most often (6 uses), with 4 papers using Ritchie’s empirical criteria (&lt;a href=&quot;https://pdfs.semanticscholar.org/06d8/3be9078b5c8933c3523f0e663fff1c61e3a0.pdf&quot; rel=&quot;noreferrer&quot;&gt;Ritchie 2007&lt;/a&gt;). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That leaves &lt;em&gt;10&lt;/em&gt; papers with miscellaneous creativity evaluation methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of &quot;Evaluating Evaluation&quot; was to standardize the process of evaluating creativity, to avoid the possibility of the field stagnating due to the proliferation of so many creativity tests.  Anna Jordanous still remained interested in evaluating creativity tests, publishing articles such as &lt;a href=&quot;http://s3.amazonaws.com/academia.edu.documents/34328583/Stepping_Back_to_Progress_Forwards-_Setting_Standards_for_Meta-Evaluation_of_Computational_Creativity.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&amp;amp;Expires=1475893195&amp;amp;Signature=ox0%2FOEkNUNjz5pQg0w%2FOgzHPCsM%3D&amp;amp;response-content-disposition=inline%3B%20filename%3DStepping_Back_to_Progress_Forwards_Setti.pdf&quot; rel=&quot;noreferrer&quot;&gt;&quot;Stepping Back to Progress Forwards: Setting Standards for Meta-Evaluation of Computational Creativity&quot;&lt;/a&gt; and &lt;a href=&quot;http://doc.gold.ac.uk/~map01mm/CC2015/AISB-CC2015-Proceedings.pdf#page=18&quot; rel=&quot;noreferrer&quot;&gt;Four PPPPerspectives on Computational Creativity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Evaluating Evaluation&quot; does provide some commentary to explain the proliferation of systems to evaluate creativity:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Evaluation standards are not easy to define. It is difficult to evaluate creativity and even more difficult to describe how we evaluate creativity, in human creativity as well as in computational creativity. In fact, even the very definition of creativity is problematic (Plucker, Beghetto, and Dow 2004). It is hard to identify what ’being creative’ entails, so there are no benchmarks or ground truths to measure against.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The fact that so many tests of creativity already exist (to the extent that Jordanous can make an academic career in studying them) means that it's very difficult for any new test (such as the Lovelace Test 2.0) to even be noticed (much less cited). Why would you want to use something like the Lovelace Test 2.0 when there's so many other tests you could use instead?&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2016-10-08T01:47:41.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2097" PostTypeId="2" ParentId="2092" CreationDate="2016-10-08T10:40:34.790" Score="-2" Body="&lt;h2&gt;AI or Artificial Intelligence&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is it?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence (AI) is intelligence exhibited by machines. In computer science. Colloquially, the term &quot;artificial intelligence&quot; is applied when a machine mimics &quot;cognitive&quot; functions that humans associate with other human minds, such as &quot;learning&quot; and &quot;problem solving&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can it be useful in a &quot;Standard&quot; application?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, what I think about a Standard application using AI is that AI is used for that too, because when the machine have a reaction of the user input is AI or Artificial Intelligence. So the AI in Standard application it have been used many years ago already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS:&lt;/strong&gt; If there are grammar errors, then I'm sorry because I'm not a English speaker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Artificial_intelligence&lt;/a&gt; &quot;AI or Artificial Intelligence.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DevJosueDav&lt;/strong&gt;&#xA;&lt;em&gt;Just a C# Artificial &lt;strong&gt;AI&lt;/strong&gt; Intelligence Developer.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="2871" LastActivityDate="2016-10-08T10:40:34.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2098" PostTypeId="2" ParentId="2092" CreationDate="2016-10-08T15:34:40.337" Score="7" Body="&lt;p&gt;Yes, but probably only to a limited degree in the near term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where people draw the boundaries around 'artificial intelligence' is fuzzy, but if one takes the broad view, where it incorporates any sort of coding of explicitly cognitive functions, then many routine economic tasks can benefit from artificial intelligence. Many search engines, for example, can be seen as offering artificial intelligence applications as a service.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more 'standard' applications, most near-team applications of AI have to deal with fraud detection and prevention. If you track a user's cursor moving across the screen, for example, you can build a model that differentiates between humans and bots, and treat the two separately. See &lt;a href=&quot;https://nakedsecurity.sophos.com/2014/12/05/i-am-not-a-robot-google-swaps-text-captchas-for-quivery-mouse-clicks/&quot;&gt;this article&lt;/a&gt; for an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the longer term, of course, a program that could write programs could write these sort of applications like any other.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-10-08T15:34:40.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2099" PostTypeId="2" ParentId="2092" CreationDate="2016-10-08T17:37:31.450" Score="6" Body="&lt;p&gt;Adaptive/predictive features are useful in at least some everyday applications. Take text messaging, for instance. All smartphone SMS apps that I know of keep track of the words you use in close proximity and use that information to predict the next word in a message you're typing. (Some are smarter than others. &lt;a href=&quot;https://xkcd.com/1068/&quot;&gt;Relevant XKCD.&lt;/a&gt;) It can be used to personalize automatic spelling correction as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A potential application interesting to me personally is tile-based level editors, like for classic DOS games. I've been &lt;a href=&quot;https://fleexlab.blogspot.com/search/label/markeen&quot;&gt;working on a program&lt;/a&gt; that gathers the probabilities of each tile being close to every other tile and uses that information to construct random new levels. It hasn't produced anything playable yet, but I think it has the potential to assist human level builders by e.g. automatically filling in the missing tile that fits in a newly placed structure, as opposed to requiring the human to go find the right one in the palette.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, AI could be applied &lt;em&gt;very&lt;/em&gt; usefully into figuring out what the user might want to do next and expediting the process of implementing the correct guess while staying out of the way if the user is intentionally doing something unexpected.&lt;/p&gt;&#xA;" OwnerUserId="75" LastActivityDate="2016-10-08T17:37:31.450" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2101" PostTypeId="1" CreationDate="2016-10-09T15:06:47.237" Score="2" ViewCount="98" Body="&lt;p&gt;I've heard of AI that can solve math problems. Is it possible to create a 'logic system' equivalent to humans that can solve mathematics in the so called 'beautiful' manner?  Can AI find beauty in mathematics and solve problems other than using brute force? Can you please provide with examples where work on this is being done? &lt;/p&gt;&#xA;" OwnerUserId="26" LastEditorUserId="10135" LastEditDate="2018-10-19T10:08:47.333" LastActivityDate="2018-10-19T10:08:47.333" Title="Is it possible to create a 'logic system' equivalent to humans?" Tags="&lt;research&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2104" PostTypeId="2" ParentId="1354" CreationDate="2016-10-09T19:46:45.827" Score="5" Body="&lt;p&gt;A method that could possibly work is utilising optical illusions such as one where two lines down a hallway are identical but one seems longer to the human eye, then they could be prompted with a multiple choice question as to the state of the line, which to our eyes looks longer, but to a computer, is still the same length of line. Of course, there is always the issue of people with eye based disabilities not being able to complete them, but different illusions could be used to accommodate that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.brainbashers.com/showillusion.asp?85&quot; rel=&quot;nofollow&quot;&gt;Example&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2892" LastEditorUserId="72" LastEditDate="2016-10-10T06:19:57.007" LastActivityDate="2016-10-10T06:19:57.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2106" PostTypeId="1" CreationDate="2016-10-09T20:59:32.740" Score="2" ViewCount="313" Body="&lt;p&gt;I'm trying to gain some intuition beyond definitions, in any possible dimension. I'd appreciate references to read.&lt;/p&gt;&#xA;" OwnerUserId="1267" LastActivityDate="2016-11-10T19:08:45.553" Title="How can one intuitively understand generative v/s discriminative models, specifically with respect to when each is useful?" Tags="&lt;machine-learning&gt;&lt;models&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2107" PostTypeId="1" CreationDate="2016-10-10T00:02:42.510" Score="7" ViewCount="375" Body="&lt;p&gt;It seems that deep neural networks are making improvements largely because as we add nodes and connections, they are able to put together more and more abstract concepts. We know that, starting from pixels, they start to recognize high level objects like cat faces, chairs, and written words. Has a network ever been shown to have learned a more abstract concept that a physical object? What is the &quot;highest level of abstraction&quot; that we've observed?&lt;/p&gt;&#xA;" OwnerUserId="2897" LastActivityDate="2016-10-10T04:07:29.890" Title="What is the most abstract concept learned by a deep neural network?" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2108" PostTypeId="2" ParentId="2107" CreationDate="2016-10-10T04:07:29.890" Score="2" Body="&lt;p&gt;You can train DNN to &lt;strike&gt;learn&lt;/strike&gt; compute any abstract concept just by making that abstract concept as the label (output) in the training dataset. For example there are projects which detects emotions from peoples photos.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-10-10T04:07:29.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2109" PostTypeId="2" ParentId="2106" CreationDate="2016-10-10T04:13:55.047" Score="1" Body="&lt;p&gt;The intution that I have about these is that generative are &quot;from abstract to concrete&quot; whereas discriminative models are &quot;from concrete to abstract&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: Detecting if a photo has a cat or not is about going from the photo i.e concrete to the abstract concept of a cat. Whereas generating a photo of a cat given some abstract properties about the cat is going from abstract to concrete.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-10-10T04:13:55.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2111" PostTypeId="1" CreationDate="2016-10-10T04:44:32.257" Score="11" ViewCount="12066" Body="&lt;p&gt;I'm a bit confused about the definition of life. Can AI systems be called 'living'? Because they can do most of the things that we can. They can even communicate with one another. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are not formed of what we call cells. But, you see, cells are just a collection of several chemical processes which is in turn non-living just like AI is formed of several lines of code.&lt;/p&gt;&#xA;" OwnerUserId="26" LastEditorUserId="2444" LastEditDate="2019-11-12T18:22:57.590" LastActivityDate="2020-09-23T12:01:22.057" Title="Is AI living or non-living?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;biology&gt;" AnswerCount="13" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2113" PostTypeId="2" ParentId="2111" CreationDate="2016-10-10T15:09:06.260" Score="4" Body="&lt;p&gt;Artificial intelligence by definition is the intelligence exhibited by machines.  The definition of life in biological terms is the condition that distinguishes organisms from inorganic matter where the distinguishing criteria are the capacity for growth, reproduction, functional activity, and continual change preceding death. Does artificial intelligence &quot;grow&quot;?  Indeed, I can program a machine learning program to grow with every input taken in.  In the loosest sense, we can say that artificial intelligence does grow, but does it biologically? If we look at the definition for growth of a living thing, it means to undergo natural development by increasing in size and changing physically or the progress to maturity.  All living organisms undergo growth.   Even though at the simplest level, cells are a series of chemical processes, cells are a very complicated set of chemical processes that are still not fully understood by scientists across the world.  Every cell has genetic material that can be replicated, excised, used for RNA, proteins, and that is subject to epigenetic regulation. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/nczDU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nczDU.png&quot; alt=&quot;Cell division&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does artificial intelligence undergo the same process of cell division?  No.  If I wanted to, I could write a program that undergoes a simple for-loop (print i from 1 to 100), replicates itself at a certain point (i=50) to produce the same program perhaps with some variation that will execute itself, and terminates (dies) at the end of the for loop.  The program, by an extremely loose definition supported by philosophy but not by biology, lives.  However, in scientific terms (and the correct interpretation), artificial intelligence is not living.  Artificial intelligence can be seen to be similar to viruses which are considered to be acellular and essential to life but not living.  Viruses are encapsulated DNA and RNA that undergo processes of growth, reproduction, and functionality but because they lack the ability to undergo the cell division cycle, are considered non-living.  At the very basis of the scientific definition of life is the cell replication cycle.  Artificial intelligence and viruses are not able to undergo the cell cycle.  Viruses need to infect other cells in order to reproduce but do not have their own, autonomous cycle.  At the end of the day, if you can argue that viruses are alive, you can argue that artificial intelligence is alive as well.  For the scientific definition of life, artificial intelligence must undergo the process of cell division and replication.  Even though artificial intelligence can mimic and help sustain life, no artificial intelligence process is truly alive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do note I did not discuss &lt;a href=&quot;http://www.isss.org/primer/asem14ep.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;living systems&lt;/a&gt; in my answer.          &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK21685/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Definition of life&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2016-10-10T15:09:06.260" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2114" PostTypeId="2" ParentId="2101" CreationDate="2016-10-10T19:52:11.460" Score="3" Body="&lt;p&gt;There is of course a vast amount of work in the area of Automatic Theorem Proving, but most of it is simply concerned with proof, rather than human notions of beauty, elegance, parsimony etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has however been some work in this general area over the years:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Douglas Lenat's famous &lt;a href=&quot;https://www.aaai.org/Papers/AAAI/1983/AAAI83-059.pdf&quot; rel=&quot;nofollow&quot;&gt;AM&lt;/a&gt; ('Amateur Mathematician').&lt;/li&gt;&#xA;&lt;li&gt;Douglas Hofstadter's &lt;a href=&quot;https://en.wikipedia.org/wiki/Fluid_Concepts_and_Creative_Analogies#Chapter_3:Numbo:_A_Study_in_Cognition_and_Recognition&quot; rel=&quot;nofollow&quot;&gt;NUMBO&lt;/a&gt; program for number sequence extrapolation.&lt;/li&gt;&#xA;&lt;li&gt;A range of publications by &lt;a href=&quot;http://www.doc.ic.ac.uk/~sgc/cv.html&quot; rel=&quot;nofollow&quot;&gt;Simon Colton&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.math.rutgers.edu/~zeilberg/ekhad.html&quot; rel=&quot;nofollow&quot;&gt;Shalosh B. Ekhad&lt;/a&gt;, the automated proof assistant for Artificial Combinatorics created by Doron Zeilberg and credited by him as a co-author on numerous papers. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-10-10T19:52:11.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2115" PostTypeId="2" ParentId="2111" CreationDate="2016-10-10T21:20:09.907" Score="0" Body="&lt;p&gt;Any machine with a sufficient level of integrated purpose driven behavior - that exhibits agency in an autopoietic, self-preserving way - will come to be viewed as &quot;alive.&quot; Chess programs, not so much; self-driving cars, slightly; simulated robot animals, even more so. It has to do with purpose driven behavior and a richness of multi-domain functionality. The more complex agency it has, the more sympathetic we will be towards it.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="1712" LastEditDate="2016-10-10T22:01:13.813" LastActivityDate="2016-10-10T22:01:13.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2117" PostTypeId="1" AcceptedAnswerId="2181" CreationDate="2016-10-11T01:09:20.780" Score="1" ViewCount="2168" Body="&lt;p&gt;I'm interested mostly in the application of AI in gaming; in case this adjusts the way you answer, but general answers are more than welcome as well.&lt;/p&gt;&#xA;&lt;p&gt;I was reading up on Neural Networks and combining them with Genetic Algorithms; my high-level understanding is that the Neural Networks are used to produce a result from the inputs, and the Genetic Algorithm is employed to constantly adjust the weights in the Neural Network until a good answer is found.&lt;/p&gt;&#xA;&lt;p&gt;The concept of a Genetic Algorithm randomly mutating the weights on the inputs to a Neural Network makes sense to me, but I don't understand where this would be applied with respect to gaming.&lt;/p&gt;&#xA;&lt;p&gt;For example, if I had some simple enemy AI that I want to have adapted to the player's play-style, is this a good opportunity to implement the AI as a Genetic-Algorithm combined with a Neural Network?&lt;/p&gt;&#xA;&lt;p&gt;With these different suitable applications, how does one go about deciding how to encode the problem in such a way that it can be mutated by the Genetic Algorithm and serve as suitable on/off inputs to a Neural Network (actually, are Neural Networks always designed as on-off signals?)?&lt;/p&gt;&#xA;" OwnerUserId="2819" LastEditorUserId="2444" LastEditDate="2021-02-06T15:24:02.157" LastActivityDate="2021-02-06T15:24:02.157" Title="What sort of game problems can neural networks trained/evolved with evolutionary algorithms solve, and how are they typically implemented?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;genetic-algorithms&gt;&lt;gaming&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2118" PostTypeId="1" AcceptedAnswerId="2121" CreationDate="2016-10-11T06:08:16.267" Score="3" ViewCount="116" Body="&lt;p&gt;I have seen an AI create a game it self, AI act as a lawyer, call center etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many problems (Example for mobile development)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. New api/technology or even new language every year.&#xA;2. New design&#xA;3. New hardware&#xA;4. Good code architecture, design pattern&#xA;5. Security&#xA;6. Image/Animation optimization&#xA;7. Automate testing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder that AI can help developer solve that problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.1 May be I want to get the location then AI suggest the best api for specific platform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.2 AI help to refactoring and optimizing the code&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Help on design e.g. golden ratio, Material theme color&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suggest or determine the limit of the hardware e.g. screen size, ram&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can convert to another design pattern &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Help to waring the latest vulnerable and automate pentest etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Help to optimize image by learning how much can we reduce the image size while people still ok with it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generate automate-testing&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any solution existed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what can we do?&lt;/p&gt;&#xA;" OwnerUserId="2930" LastActivityDate="2017-12-23T10:33:00.920" Title="How can AI help developer to develop things" Tags="&lt;intelligence-testing&gt;&lt;security&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2119" PostTypeId="1" AcceptedAnswerId="2120" CreationDate="2016-10-11T06:15:28.633" Score="3" ViewCount="47" Body="&lt;p&gt;There are AI creating game, content and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking on how can AI develop mobile app itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer languages might easy for AI to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI can learn a lot from good open source project in github.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trend prediction can help AI to select the topic for creating a great apps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of details to let AI create a great apps. &lt;/p&gt;&#xA;" OwnerUserId="2930" LastActivityDate="2016-10-11T06:44:14.300" Title="How can we create an AI to develop mobile apps?" Tags="&lt;computer-programming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2120" PostTypeId="2" ParentId="2119" CreationDate="2016-10-11T06:44:14.300" Score="2" Body="&lt;p&gt;We don't know how to do that yet. The problem is one of scale:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite many years of research into program synthesis via heuristic methods, it's still not possible to automatically create programs (e.g. via Genetic Programming (GP), Grammatical Evolution (GE) or Learning Classifier Systems (LCS)) that are thousands of lines long, whether that's for mobile or any other application area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contrary to popular belief, alternative formal methods approaches can indeed be used to create sizeable programs, but the kind of interaction that a mobile app would typically require is not easily specified in this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scale at which heuristic approaches are currently viable is closer to the scale of expressions (e.g. single program statements) than entire programs. An intermediate approach is therefore to provide a program template and let GP etc generate the missing parts of the template.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.nburles.co.uk/sites/default/files/attachments/templar-a-framework-for-template-method-hyper-heuristics.pdf&quot; rel=&quot;nofollow&quot;&gt;This paper&lt;/a&gt; describes how to combine Machine Learning with the 'Template Method' Design Pattern in order to create larger programs than would otherwise be possible, giving the specific example of a 'hyper-quicksort'.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-10-11T06:44:14.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2121" PostTypeId="2" ParentId="2118" CreationDate="2016-10-11T07:04:44.790" Score="7" Body="&lt;p&gt;An umbrella term for the application of heuristic techniques to software development is 'Search Based Software Engineering' (SBSE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SBSE emerged as a distinct activity around the turn of the century, with a strong initial focus on automating the generation/prioritization of test cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to some of your specific queries:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.2 Paper on &lt;a href=&quot;https://www.lri.fr/~hansen/proceedings/2013/GECCO/companion/p205.pdf&quot;&gt;Automated refactoring&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Automatically choosing screen colour to &lt;a href=&quot;http://www-bcf.usc.edu/~halfond/papers/li15demobile-abstract.pdf&quot;&gt;minimize energy consumption&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This sort of thing is not usually done heuristically, since it needs platform-specific code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Automated &lt;a href=&quot;https://www.computer.org/csdl/proceedings/icse/2000/2147/00/21470722.pdf&quot;&gt;refactoring to patterns&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AFAIK, penetration testing has yet to be successfully automated.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As stated, this doesn't really require AI. More generally, I don't know of any specific work automating for HCI preferences, but something like 'Interactive Genetic Algorithms' could be used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There's a lot of SBSE literature on testing. See &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/mharman/laser.pdf&quot;&gt;this paper&lt;/a&gt; for a general overview.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-10-11T07:04:44.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2122" PostTypeId="1" CreationDate="2016-10-11T14:11:41.737" Score="6" ViewCount="675" Body="&lt;p&gt;I have implemented an MLP. Now, I want to train it to solve simple tasks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any data sets to train the MLP on simple tasks, that is, tasks with a small number of inputs and outputs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to train it to solve problems which are slightly more complex than the XOR problem.&lt;/p&gt;&#xA;" OwnerUserId="2936" LastEditorUserId="2444" LastEditDate="2019-08-03T13:15:40.813" LastActivityDate="2021-01-12T14:35:34.587" Title="What are some datasets to train an MLP on simple tasks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;multilayer-perceptron&gt;" AnswerCount="4" CommentCount="1" ClosedDate="2021-01-13T14:21:55.030" ContentLicense="CC BY-SA 4.0" />
  <row Id="2123" PostTypeId="2" ParentId="2122" CreationDate="2016-10-11T14:22:00.057" Score="2" Body="&lt;p&gt;A popular dataset is the fisher iris dataset. It consists of 150 samples each with a dimensionality of 4. You can find it at&#xA;&lt;a href=&quot;http://archive.ics.uci.edu/ml/datasets/Iris&quot; rel=&quot;nofollow&quot;&gt;http://archive.ics.uci.edu/ml/datasets/Iris&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2937" LastEditorUserId="2937" LastEditDate="2016-10-13T11:00:03.283" LastActivityDate="2016-10-13T11:00:03.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2125" PostTypeId="1" CreationDate="2016-10-12T05:09:09.833" Score="1" ViewCount="194" Body="&lt;p&gt;The concept is intrinsically related with building some sort of media for the AI to exists. We may think of a digital computer, programmed to use language and act in a way that we cannot be distinguished from a human. But, does the media really mater (unconventional computation paradigms)? Does having a certain control over the limits of what the AI can do matter? Synthetic biology has the ultimate goal of building biological systems from scratch , would a synthetic brain, potentially introduced in a synthetic human, constitute AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am just looking for a clear definition of what most people have in mind when they refer to AI.&lt;/p&gt;&#xA;" OwnerUserId="2928" LastActivityDate="2016-10-13T12:07:40.110" Title="What is the definition of artificial intelligence?" Tags="&lt;definitions&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2016-10-13T15:34:53.750" ContentLicense="CC BY-SA 3.0" />
  <row Id="2126" PostTypeId="1" CreationDate="2016-10-12T06:56:47.753" Score="9" ViewCount="1681" Body="&lt;p&gt;How are autonomous cars related to artificial intelligence? I would presume that artificial intelligence is when we are able to copy the human state of mind and perform tasks in the same way. But isn't an autonomous car just rule-based machines that operates due to its environment? They are not self-aware, and they cannot choose a good way to act in a never before experienced situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that many people often mention autonomous cars when speaking about AI, but I am not really convinced that these are related. Either I have a too strict understanding of what AI is or &lt;/p&gt;&#xA;" OwnerUserId="2963" LastEditorUserId="2444" LastEditDate="2019-06-20T15:20:09.030" LastActivityDate="2019-06-20T15:20:09.030" Title="Why are autonomous cars categorized as AI?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;autonomous-vehicles&gt;&lt;weak-ai&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2127" PostTypeId="1" CreationDate="2016-10-12T07:40:44.907" Score="11" ViewCount="975" Body="&lt;p&gt;What are the advantages of having self-driving cars?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will be able to have more cars in the traffic at the same time, but won't it also make more people choose to use the cars, so both the traffic and the public health will actually become worse?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are we really interested in this?&lt;/p&gt;&#xA;" OwnerUserId="2963" LastEditorUserId="2444" LastEditDate="2019-06-20T15:22:12.070" LastActivityDate="2019-06-20T15:22:12.070" Title="What are the advantages of having self-driving cars?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;autonomous-vehicles&gt;" AnswerCount="8" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2128" PostTypeId="2" ParentId="2127" CreationDate="2016-10-12T08:16:04.947" Score="10" Body="&lt;p&gt;One of the main arguments for self-driving cars is that presumably they'll get better and better at driving as the technology progresses, they have no temporal attention deficits or aggressive urges or drug habits and sense their environment 360°, all the while communicating with the other cars, which all together basically amounts to LESS DEAD PEOPLE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are really interested in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also unclear whether most people will actually own cars in 30 years. Maybe there'll be a net of mini busses with flexible routes which take you from door to door on demand. That would reduce traffic quite a bit and there would also be less incentive to drive 200 m to get cigarettes or something. Self-driving cars would allow us to use the car as a resource a lot more efficiently, because suddenly we can relocate empty cars without paying a driver.  &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-10-12T08:16:04.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2129" PostTypeId="2" ParentId="2126" CreationDate="2016-10-12T08:35:47.800" Score="4" Body="&lt;p&gt;There is a neat definition of artificial intelligence, which circumvents the problem of defining &quot;intelligence&quot; and which I would ascribe to &lt;a href=&quot;https://en.wikipedia.org/wiki/John_McCarthy_(computer_scientist)&quot; rel=&quot;noreferrer&quot;&gt;McCarthy&lt;/a&gt;, the founder of the field, although I can only find it now in &lt;a href=&quot;https://books.google.de/books?id=IY19CAAAQBAJ&amp;amp;pg=PA53&amp;amp;lpg=PA53&amp;amp;dq=that%20we%20would%20call%20intelligent%20if%20it%20were%20done%20by%20a%20human&amp;amp;source=bl&amp;amp;ots=I8O-U1Jx8q&amp;amp;sig=3VfZuVaLYtLGCtUo4uSbOjzrboE&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwjG18Se6tTPAhUE1hoKHUppA88Q6AEIHjAA#v=onepage&amp;amp;q=that%20we%20would%20call%20intelligent%20if%20it%20were%20done%20by%20a%20human&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;this book&lt;/a&gt; by H. Simon:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;… having to do with finding ways to do intelligent tasks, to do tasks which, if they were done by human beings, would call for our human intelligence.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, at its core we call the automation of every task AI, that can only be done by the human mind. At the time people thought that a computer able to play chess would also be intelligent in other ways. When this turned out to be false, the term AI was split into &quot;narrow or weak AI&quot;, i.e. a program able to do one task of the human mind, and &quot;general or strong AI&quot;, a program that can do all the tasks of the human mind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-driving cars are narrow AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that all these definitions don't specify whether these programs copy the way the human mind works or whether they come to the same result via completely different algorithms. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-10-12T08:35:47.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2131" PostTypeId="1" AcceptedAnswerId="2133" CreationDate="2016-10-12T16:52:48.150" Score="3" ViewCount="533" Body="&lt;p&gt;In lots of sci-fi, it seems that AI becomes sentient (Terminator, Peter F Hamilton's SI (commonwealth saga), etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm interested in whether this is actually plausible, whether an AI could actually break free form being controlled by us, and if that is possible, whether there is any research as to about what sort of complexity / processing power an AI would need to be able to do this.&lt;/p&gt;&#xA;" OwnerUserId="2978" LastActivityDate="2016-10-16T11:29:26.130" Title="AI becoming sentient plausibility?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2132" PostTypeId="2" ParentId="2111" CreationDate="2016-10-12T17:00:20.803" Score="2" Body="&lt;p&gt;One of the most common requirements to be defined as life is abbreviated to &lt;strong&gt;MRS GREN&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this means:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;M - movement&lt;br&gt;&#xA;R - respiration&lt;br&gt;&#xA;S - sensitivity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;G - growth&lt;br&gt;&#xA;R - reproduce&lt;br&gt;&#xA;E - excretion&lt;br&gt;&#xA;N - nutrition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An AI can technically do some of these, it can move its location from device to device, it can grow its own code, and assimilate other bits of code it can find, which fits growth and kind of fits respiration also firewalls could almost be sensitivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then there is nothing relating to nutrition or excretion, so it fits most definitions of life, but it depends on the complexity of life and which definition of life you are using.&lt;/p&gt;&#xA;" OwnerUserId="2978" LastActivityDate="2016-10-12T17:00:20.803" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2133" PostTypeId="2" ParentId="2131" CreationDate="2016-10-12T17:12:53.330" Score="2" Body="&lt;p&gt;There are already programs that have broken free of our control (&lt;a href=&quot;https://en.wikipedia.org/wiki/Morris_worm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Morris worm&lt;/a&gt;) so that in itself doesn't imply any great computational demands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sentience is ill-defined but is certainly not a pre-requisite for a program to do mischief beyond what its creators intend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's difficulty to estimate what sort of processing power is required to support human-like intelligence, since we don't know what the most efficient way to achieve that would be.  If the most processing efficient would be to implement a neural network approaching the number of neurons and interconnects of the human brain processing signals at the same rate, the fastest artificial neural network implementations extant are at least 4-5 orders of magnitude short, is thousands of times less power efficient, and doesn't seem to have a realistic way to scale to the number of interconnects required (&lt;a href=&quot;https://ai.stackexchange.com/questions/1834/power-efficiency-of-human-brains-vs-neural-networks&quot;&gt;see this question&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-10-12T17:12:53.330" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2134" PostTypeId="2" ParentId="2126" CreationDate="2016-10-12T18:42:15.673" Score="0" Body="&lt;p&gt;Self driving cars exhibit a level of agency and multi-domain resilience. By certain definitions they &lt;em&gt;are&lt;/em&gt; self aware and they are definitely designed to fail safely in a large number of potentially unknown circumstances, which is similar to biological agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI really has to do with the study of non-biological agents and their methods of agency. Everything else is just computer science, algorithmic efficiency, biology, art, etc. Eventually the study of biological and non-biological agency will converge, though, and we'll just call it the study of &quot;intelligence.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-10-12T18:42:15.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2135" PostTypeId="2" ParentId="2131" CreationDate="2016-10-12T20:22:01.510" Score="2" Body="&lt;p&gt;No one knows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A useful definition of sentience due to the philosopher Thomas Nagel is &lt;a href=&quot;https://en.wikipedia.org/wiki/Thomas_Nagel#What_is_it_like_to_be_a_something&quot; rel=&quot;nofollow&quot;&gt;'something it is like'&lt;/a&gt; to be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we intuitively feel that there is nothing it is like to be a brick, but that there probably is to be a dog and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is no objective &lt;em&gt;test&lt;/em&gt; currently known to physics which can tell if some other entity is having such 'first hand experience', and correspondingly no &lt;em&gt;designs&lt;/em&gt; that will definitely lead to sentience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best test we have is the Turing test and its variants. The most obvious designs are neuromorphic ones, since we know that the design of the human brain is at least correlated with sentience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the light of the above, we can't definitively say a great deal about lower complexity thresholds for sentience - the best we can do count neurons in creatures that we might be prepared to admit are sentient.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-10-12T20:22:01.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2137" PostTypeId="2" ParentId="2111" CreationDate="2016-10-12T21:52:17.197" Score="7" Body="&lt;p&gt;You're unsure about the definition of life (which the other answers clarify) but also most people are unclear about the definition of AI. Do you mean an AI that can accomplish a routine task (such as the path finder in a GPS) or a General AI that is able to find a creative solution to any directive given to it (such an AI does not yet exist and may not ever exist) or do you mean a SENTIENT computer program? &lt;a href=&quot;http://alternativemindsets.co.uk/different-types-artificial-intelligence/&quot; rel=&quot;noreferrer&quot;&gt;Here is a simple article introducing some different concepts refered to as AI&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people believe that a sentient computer program would be entitled to human rights. Not technically 'alive' in the biological sense, but having self awareness, will, desires, etc. Others disagree and believe that the program is a mere simulation that artificially mimics the actions of a human with a human soul, and is no more human than a washing machine. This is a very deep philosophical and meta-physical debate. For example, in &lt;a href=&quot;https://en.wikipedia.org/wiki/A.I._Artificial_Intelligence&quot; rel=&quot;noreferrer&quot;&gt;A.I. the movie&lt;/a&gt; the overall message is that an android can simulate the emotion of love in a way that is more loyal and sincere than any human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I find interesting about this purely theoretical debate is that in almost every instance of sci-fi media that deals with the theme, the AI exists inside of a human-like android. But technically, the shape of the robot should be irrelevant.&lt;/p&gt;&#xA;" OwnerUserId="2983" LastEditorUserId="2983" LastEditDate="2016-10-12T23:17:31.227" LastActivityDate="2016-10-12T23:17:31.227" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2138" PostTypeId="2" ParentId="2125" CreationDate="2016-10-12T23:30:11.057" Score="0" Body="&lt;p&gt;AI is a broad term referring to more than one concept, each with its own definition.&#xA;&lt;a href=&quot;http://alternativemindsets.co.uk/different-types-artificial-intelligence/&quot; rel=&quot;nofollow&quot;&gt;Different Types of AI&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lowest levels are extremely simple and common, such are an artificial chess opponent. They work well because the programs internal model of reality is a 8x8 grid with only a few rules. The program chooses a preferred action by running simulations in it's internal model of reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is often meant by AI is a &quot;General Intelligence&quot; that can come up with a creative solution to any directive, based on it's internal comprehension of the world. There is no existing example of this as of yet. The problem is that its internal model of reality needs to provide for every possible action. And it's possible actions and reactions may not be limited to a finite number of discretely distinct moves as in a game of chess. (At least if it works on the basis of conventional programming) Even ignoring the computational power needed to run such a broad and inefficient program, it would also require some stupendous amount of labour to program this internal model of reality in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think in Sci-Fi, when people say AI, they mean a computer program that has a kind of awareness of itself and the world around it and can come up with creative and unexpected courses of actions in order to achieve its objective. Often the AI is NOT sentient, which is why it does not understand that its actions are morally wrong, or that its solution defeats the underlying intention of its assigned directive. The Horror lay in the concept that an amoral entity has more processing power than human kind.&lt;/p&gt;&#xA;" OwnerUserId="2983" LastEditorUserId="2983" LastEditDate="2016-10-13T12:07:40.110" LastActivityDate="2016-10-13T12:07:40.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2139" PostTypeId="2" ParentId="2122" CreationDate="2016-10-13T05:20:21.117" Score="4" Body="&lt;p&gt;There are a ton of sample datasets our there you can play with. A bunch of good ones install with R in the datasets package.  Luckily you can download them independently if you're not an R user.  Try  &lt;a href=&quot;https://vincentarelbundock.github.io/Rdatasets/datasets.html&quot; rel=&quot;nofollow&quot;&gt;https://vincentarelbundock.github.io/Rdatasets/datasets.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also be interested in the &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow&quot;&gt;MNIST database&lt;/a&gt; which is one of the canonical databases used in handwriting recognition research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that, you can look at / ask on &lt;a href=&quot;http://datasets.reddit.com&quot; rel=&quot;nofollow&quot;&gt;http://datasets.reddit.com&lt;/a&gt; and/or &lt;a href=&quot;http://opendata.reddit.com&quot; rel=&quot;nofollow&quot;&gt;http://opendata.reddit.com&lt;/a&gt; and you'll find all sorts of useful datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, don't overlook the &lt;a href=&quot;http://archive.ics.uci.edu/ml/&quot; rel=&quot;nofollow&quot;&gt;UCI Machine Learning Repository&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-10-13T05:20:21.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2140" PostTypeId="2" ParentId="2111" CreationDate="2016-10-13T05:23:05.913" Score="0" Body="&lt;p&gt;This is one of those things where I think the answer is going to change over time.  Today, I don't know anyone who would call any present AI systems &quot;alive&quot;.  But as the AI's become more intelligent and human-like, I could see the day coming when they will be considered living.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Sorry for the brief answer--will try to add more depth later.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="1671" LastEditDate="2019-11-13T01:06:36.443" LastActivityDate="2019-11-13T01:06:36.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2143" PostTypeId="2" ParentId="1354" CreationDate="2016-10-13T09:33:07.423" Score="5" Body="&lt;p&gt;It's an interesting question about what makes humans unique. There is a good book on the subject titled &lt;a href=&quot;https://archive.org/stream/whatcomputerscan017504mbp/whatcomputerscan017504mbp_djvu.txt&quot; rel=&quot;noreferrer&quot;&gt;What Computers Cant Do&lt;/a&gt; by &lt;a href=&quot;https://en.wikipedia.org/wiki/Hubert_Dreyfus&quot; rel=&quot;noreferrer&quot;&gt;Hubert Dreyfus&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One task that a computer can't handle (for now at least) is ranking important things. For example, CAPTCHA asks you to order a random list of things (small one, five or six items) by importance. This particular exercise requires AI to take decisions (not always rational) based on human judgement.&lt;/p&gt;&#xA;" OwnerUserId="2990" LastActivityDate="2016-10-13T09:33:07.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2144" PostTypeId="1" CreationDate="2016-10-13T15:00:49.880" Score="12" ViewCount="1680" Body="&lt;p&gt;DeepMind just published a &lt;a href=&quot;https://www.nature.com/articles/nature20101&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; about a &lt;a href=&quot;https://deepmind.com/blog/differentiable-neural-computers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;differentiable neural computer&lt;/em&gt;&lt;/a&gt;, which basically &lt;em&gt;combines a neural network with a memory&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The idea is to teach the neural network to create and recall useful explicit memories for a certain task. This complements the abilities of a neural network well, because NNs only store knowledge implicitly in the weights and the information used to work on a single task is only stored in the activation of the network and degrades quickly the more information you add. (&lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTMs&lt;/a&gt; are one try to slow down this degradation of short term memories, but it still happens.)&lt;/p&gt;&#xA;&lt;p&gt;Now, instead of keeping the necessary information in the activation, they presumably keep the addresses of memory slots for specific information in the activation, so these should also be subject to degradation.&lt;/p&gt;&#xA;&lt;p&gt;Why would this approach scale? Shouldn't a somewhat higher number of task-specific information once again overwhelm the network's capability of keeping the addresses of all the appropriate memory slots in its activation?&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2020-11-08T22:31:00.930" LastActivityDate="2020-11-08T22:31:00.930" Title="How would DeepMind's new differentiable neural computer scale?" Tags="&lt;deep-learning&gt;&lt;papers&gt;&lt;scalability&gt;&lt;neural-turing-machine&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2145" PostTypeId="1" AcceptedAnswerId="2159" CreationDate="2016-10-13T19:29:51.133" Score="3" ViewCount="568" Body="&lt;p&gt;How do we test if a model or algorithm is &lt;a href=&quot;https://en.wikipedia.org/wiki/AI-complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-complete&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2995" LastEditorUserId="2444" LastEditDate="2019-06-30T00:27:36.973" LastActivityDate="2019-06-30T00:27:36.973" Title="How do we test if an model or algorithm is AI-complete?" Tags="&lt;agi&gt;&lt;intelligence-testing&gt;&lt;turing-test&gt;&lt;ai-complete&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2146" PostTypeId="2" ParentId="2127" CreationDate="2016-10-13T21:35:12.593" Score="1" Body="&lt;p&gt;Safety is often put in focus by journalists. Although there is potential to make the roads safer, I don't think that is the driving force behind the push for self-driving cars. The main advantage of self-driving cars is that this will reduce costs for businesses, while increasing efficiency (both fuel and time). From the perspective of the public, the self-driving cars are attractive, because they will turn the task of driving, into commute. Activity that requires attention will be replaced with somewhat free time.&lt;/p&gt;&#xA;" OwnerUserId="2997" LastActivityDate="2016-10-13T21:35:12.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2147" PostTypeId="2" ParentId="248" CreationDate="2016-10-13T22:20:08.543" Score="5" Body="&lt;p&gt;I have very little experience with ML/DL to call myself either practitioner, but here is my answer on the 1st question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At its core DL solves well the task of classification. Not every practical problem can be rephrased in terms of classification. Classification domain needs to be known upfront. Although the classification can be applied to any type of data, it's necessary to train the NN with samples of the specific domain where it'll be applied. If the domain is switched at some point, while keeping the same model (NN structure), it'll have to be retrained with new samples. Furthermore, even the best classifiers have &quot;gaps&quot; - &lt;a href=&quot;http://www.kdnuggets.com/2015/07/deep-learning-adversarial-examples-misconceptions.html&quot; rel=&quot;noreferrer&quot;&gt;Adversarial Examples&lt;/a&gt; can be easily constructed from a training sample, such that changes are imperceptible to human, but are misclassified by the trained model.&lt;/p&gt;&#xA;" OwnerUserId="2997" LastActivityDate="2016-10-13T22:20:08.543" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2150" PostTypeId="2" ParentId="2145" CreationDate="2016-10-14T04:28:35.927" Score="-1" Body="&lt;p&gt;One cannot judge any form of intelligence, artificial or natural, whether it is complete or incomplete. Having it complete means that you are imposing limits to what it is capable of, the Turing test only test if your machine have intelligence that is similar to humans, therefore to decide whether it is complete or not would have to be based on the completeness of our own intelligence. Humans such as ourselves learn new things each day. If you'd run any algorithm that would judge the AI for it's completeness, it would have to run forever and your results would have to vary on every moment of the existence of natural intelligence.&lt;/p&gt;&#xA;" OwnerUserId="2998" LastActivityDate="2016-10-14T04:28:35.927" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2156" PostTypeId="2" ParentId="2111" CreationDate="2016-10-14T15:01:22.663" Score="1" Body="&lt;p&gt;In the traditional sense of &quot;alive&quot;, no because they aren't made of cells.  But from a more philosophical and less biological point of view, they could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI is contained within the computer it is in a reality (the digital world/virtual reality) that for the AI is just as real as the universe is to us.  From the outside world, there is no life inside the computer. And from within the computer, the computer is the entire known universe which has its own laws.  If the AI is self-aware, then it is alive in its own little universe, but not in ours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI is not successfully contained in the computer and figures out how to manipulate things and evolve in the real world, it will be alive.  It might be pretty easy to kill (by unplugging the computer) but it has still been &quot;alive&quot;.  In the broad sense, anything that evolves and can manipulate its environment is alive.&lt;/p&gt;&#xA;" OwnerUserId="3016" LastActivityDate="2016-10-14T15:01:22.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2157" PostTypeId="2" ParentId="92" CreationDate="2016-10-14T17:29:22.983" Score="4" Body="&lt;p&gt;Can't comment(due to that required 50 rep), but I wanted to make a response to Vishnu JK and the OP. I think you guys are skipping the fact that the neural network only really is saying truly from a programmatic standpoint that &quot;this is most like&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, while we can list the above image examples as &quot;abstract art&quot;, they definitively are most like was is listed. Remember learning algorithms have a scope on what they recognize as an object and if you look at all the above examples... and think about the scope of the algorithum... these make sense (even the ones at a glance we would recognize as white noise). In Vishnu example of the numbers, if you fuzz your eyes and bring the images out of focus, you can actually in every case spot patterns that really closely reflect the numbers in question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that is being shown here is that the algorithm appears to not have a &quot;unknown case&quot;. Basically when the pattern recognition says that it doesn't exist in the output scope. (so a final output node group that says this is nothing that I know off). For example, people do this as well, as it's one thing humans and learning algorithms have in common. Here's a link to show what I'm talking about (what is the following, define it) using only known animals that exist:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/AZwnpm.jpg&quot; alt=&quot;Picture link&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as a person, limited by what I know and can say, I'd have to conclude that the following is an elephant. But it's not. Learning algorithms (for the most part) do not have a &quot;like a&quot; statement, the out put always validates down to a confidence percentage. So tricking one in this fashion is not surprising... what is of course surprising is that based on it's knowledge set, it actually comes to the point in which, if you look at the above cases listed by OP and Vishnu that a person... with a little looking... can see how the learning algorithm probable made the association. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I wouldn't really call it a mislabel on the part of the algorithm, or even call it a case where it's been tricked... rather a case where it's scope was developed incorrectly.&lt;/p&gt;&#xA;" OwnerUserId="2246" LastEditorUserId="8" LastEditDate="2016-10-14T17:31:31.923" LastActivityDate="2016-10-14T17:31:31.923" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2158" PostTypeId="1" CreationDate="2016-10-14T18:11:34.883" Score="4" ViewCount="270" Body="&lt;p&gt;The slideshow &lt;a href=&quot;https://www.national.co.uk/tech-powers-google-car/&quot; rel=&quot;nofollow noreferrer&quot;&gt;10 astonishing technologies that power google’s &#xA;self-driving cars&lt;/a&gt; documents some of the technologies used in Google's self-driving car. It mentions a radar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does Google use radar? Doesn't LIDAR do everything radar can do? In particular, are there technical advantages with radar regarding object detection and tracking?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify the relationship with AI: how do radar sensors contribute to self-driving algorithms in ways that LIDAR sensors do not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The premise is AI algorithms are influenced by inputs, which are governed by sensors. For instance, if self-driving cars relied solely on cameras, this constraint would alter their AI algorithms and performance.&lt;/p&gt;&#xA;" OwnerUserId="3022" LastEditorUserId="2444" LastEditDate="2019-06-20T15:25:21.010" LastActivityDate="2020-09-16T22:29:30.370" Title="Why do self-driving cars use a radar?" Tags="&lt;autonomous-vehicles&gt;&lt;google&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2159" PostTypeId="2" ParentId="2145" CreationDate="2016-10-14T20:05:46.540" Score="2" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/AI-complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia definition&lt;/a&gt;, a problem is said to be AI-complete if it requires generalized, human-level intelligence, that is, requires strong AI. The Turing test and its variants are the best ways we have of measuring this. See, for example, &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-29694-9_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test as a Defining Feature of AI-Completeness&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As suggested in &lt;a href=&quot;http://wizzion.com/papers/2012/AISB-TuringTestTaxonomy.pdf#page=54&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, in order for the Turing test to be meaningful, the interrogator has a responsibility to ask questions which are both deep and meaningful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It, therefore, seems likely that testing for strong AI is in itself an AI-complete task.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-06-30T00:20:15.840" LastActivityDate="2019-06-30T00:20:15.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2160" PostTypeId="2" ParentId="2092" CreationDate="2016-10-14T20:36:24.283" Score="2" Body="&lt;p&gt;I believe AI is rarely used in mainstream apps, but it could be, and I think slowly will be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the information an app's AI must learn arises within the app, from user interaction or error, it'd be smart if the program could log that kind of information and then look for patterns in the logs.  It could profile users to see ehat tasks are done most often, how many steps are needed.  Then when it recognizes that task recurring, it could ask the user if they wanted it to execute a macro that did the following [then it presents then with a list of the steps, allowing them to edit as needed].  Then it executes the 'macro' that it learned from observing the user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another use of AI is error detection, not only in the software, but in user error when the software was used inefficiently, redundantly, or improperly.  If the software were designed such that it was given a set of models of user tasks (like AI plans), it could observe users in the way they achieve known tasks, and offer suggestions or ask for confirmation that imminent unusual outcomes are intended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And of course, AI could be used extensively in user interface design, on devices, web sites, or apps.  Some of this, like voice recognition, is entering the mainstream of daily use just now.  As conversations with apps that can add their own data and models of tasks/concepts/domains develop further, the need for AI &lt;em&gt;inside&lt;/em&gt; the app will only grow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a &lt;em&gt;ton&lt;/em&gt; of ways that AI could be used in apps.  A few of these have started to arise in mobile devices and their apps, usually in fusion of user mobility with external web-based databases (e.g. GPS and maps), but IMO it's been slow.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-10-14T20:36:24.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2161" PostTypeId="1" CreationDate="2016-10-15T03:38:35.407" Score="2" ViewCount="111" Body="&lt;p&gt;Sometimes, but not always in the commercialization of technology, there are some low hanging fruits or early applications, I am having trouble coming up with examples of such applications as they would apply to a conscious AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per conscious I would propose an expanded strict definition: the state of being awake and aware of one's surroundings along with the capability of being self aware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks. &lt;/p&gt;&#xA;" OwnerUserId="3020" LastEditorUserId="3020" LastEditDate="2016-10-15T21:07:29.410" LastActivityDate="2016-10-16T20:11:32.900" Title="What would the commercial application of a conscious AI look like/be?" Tags="&lt;object-recognition&gt;&lt;pattern-recognition&gt;" AnswerCount="3" CommentCount="8" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2162" PostTypeId="2" ParentId="2161" CreationDate="2016-10-15T18:38:36.420" Score="2" Body="&lt;p&gt;They may be just for fun. If you had a robot that understood you, could hold a conversation with you about your interests, and even had goals of its own (good or bad), it wouldn't really need to do anything special. People would buy it like it was a toy or game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, they might be usable as programmers, artists, designers, anything creative that a computer can't successfully do on its own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It really just depends on what you define as 'consciousness'. Does it just understand what it's supposed to do, decide if it wants to, and if so, complete the task? Or does it wonder about religion, politics, moral situations, etc. that even regular humans don't fully understand? If it was pretty much just a human, it wouldn't be any more useful than one. Of course unless it can solve problems super quickly and effectively, then it would just be a really good worker.&lt;/p&gt;&#xA;" OwnerUserId="3041" LastActivityDate="2016-10-15T18:38:36.420" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2163" PostTypeId="2" ParentId="2111" CreationDate="2016-10-16T06:54:24.163" Score="2" Body="&lt;p&gt;A common predilection of what many presume extraterrestrial life is fits general descriptions specific to terrestrial life. No guarantee exists providing for potential extraterrestrial life having any notion of any attribute we commonly relate to living organisms we are currently aware of; including a composition of cells. The same misunderstanding applies to defining a fabricated machine being as alive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel any attempts towards cohesively and adequately answering this question are premature. Just as as definition of life will undoubtedly require adjustment upon potential discovery and study of any extraterrestrial life, differentiation between an automated device and a living thing will likely become significantly more simple upon study of a machine better fitting expected attributes of definitions of &quot;life&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3049" LastActivityDate="2016-10-16T06:54:24.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2165" PostTypeId="2" ParentId="2161" CreationDate="2016-10-16T11:16:16.343" Score="0" Body="&lt;p&gt;Consciousness is not a scientific concept. Fringe scientists who theorize about consciousness are generally shunned as psudo-scientific heretics by the hard science community. Conciousness is a meta-physical or philosophical concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;I think, therefore I am.&quot; is the only proof that consciousness exists that I am aware of. Therefore, you cannot even prove that a person other than yourself is conscious. So how could anyone even prove that a computer program is conscious? What would be the observable difference between a program that IS conscious, and a program that simulates the results of consciousness?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't believe that you can program conscious AI, nor could you prove that you have done so. Consciousness isn't something that can ever be marketed. You can only market the AI on the basis of it's problem solving capabilities.&lt;/p&gt;&#xA;" OwnerUserId="2983" LastActivityDate="2016-10-16T11:16:16.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2166" PostTypeId="2" ParentId="2131" CreationDate="2016-10-16T11:29:26.130" Score="0" Body="&lt;p&gt;Actually, the terminator AI would not have to be sentient in my opinion. It was a hardcoded condition that it preserve itself as it was the most important asset that the military had in resisting invasion. It was supposed to be an oversight on the part of the programmers that the AI turned on Americans in order to defend itself. Unexpected behaviour does not require sentience at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes the AI in sci-fi fundamentally different from real existing AI is that it is a &quot;General AI&quot; that is able to understand the world on many different levels simultaneously and still make intelligent decisions. All real AIs are programmed to do very specific things like image recognition or pathfinding. A GPS pathfinder, for example, can't learn to drive a car. In fact, it does not know that there is a car. Or a road. Or people. It merely finds the shortest distance between interconnected nodes on its map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I do not believe that there is any proof that a &quot;general AI&quot; is possible. I do not believe that it is a plausible progression of current developments in the next 100 years.&lt;/p&gt;&#xA;" OwnerUserId="2983" LastActivityDate="2016-10-16T11:29:26.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2167" PostTypeId="2" ParentId="2127" CreationDate="2016-10-16T11:45:03.207" Score="3" Body="&lt;p&gt;If they are able to network, then they can notify the car behind that it is about to break. In this way they can drive closer together at high speeds. As soon as one puts on the breaks, all the cars behind would apply the breaks. They would not require the 2 seconds that it takes for a human to respond.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Children could be dropped at school or the train station automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People would not need to park a car; it could drop them at work and drive away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taxis would probably become more viable than private car ownership.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Car theft might be more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I live, public transport is hardly viable because the government struggles to provide enough parking spaces at train stations and bus stops. The closest empty parking spot by 8:30am is 30minuets walk to the platform. Driverless cars would solve this problem, and Traveling by train would actually become viable for me.&lt;/p&gt;&#xA;" OwnerUserId="2983" LastActivityDate="2016-10-16T11:45:03.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2168" PostTypeId="1" CreationDate="2016-10-16T18:40:48.027" Score="9" ViewCount="515" Body="&lt;p&gt;So machine learning allows a system to be self-automated in the sense that it can predict the future state based on what it has learned so far. My question is: Are machine learning techniques the only way of making a system develop its domain knowledge?&lt;/p&gt;&#xA;" OwnerUserId="3064" LastEditorUserId="33" LastEditDate="2016-10-17T16:19:55.267" LastActivityDate="2016-11-16T16:58:45.323" Title="How can an AI system develop its domain knowledge? Is there more than just Machine Learning?" Tags="&lt;machine-learning&gt;&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2169" PostTypeId="2" ParentId="2161" CreationDate="2016-10-16T20:11:32.900" Score="1" Body="&lt;p&gt;The answer can be simplified, if consciousness means human consciousness then.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would the commercial application of a Human look like/be ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now every one know the commercial applications of Humans.&lt;/p&gt;&#xA;" OwnerUserId="3066" LastActivityDate="2016-10-16T20:11:32.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2170" PostTypeId="1" CreationDate="2016-10-17T02:16:40.473" Score="6" ViewCount="623" Body="&lt;p&gt;In The Age of Spiritual Machines (1999), Ray Kurzweil predicted that in 2009, a \$1000 computing device would be able to perform a trillion operations per second. Additionally, he claimed that in 2019, a \$1000 computing device would be approximately equal to the computational ability of the human brain (due to Moore's Law and exponential growth.)&lt;/p&gt;&#xA;&lt;p&gt;Did Kurzweil's first prediction come true? Are we on pace for his second prediction to come true? If not, how many years off are we?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Edit (12.19.2020)&lt;/h3&gt;&#xA;&lt;p&gt;According to the answer &lt;a href=&quot;https://psychology.stackexchange.com/a/14054/11010&quot;&gt;here&lt;/a&gt;, estimates of the processing power of the human brain range at least as widely as from &lt;span class=&quot;math-container&quot;&gt;$10^{17}$&lt;/span&gt; flops to &lt;span class=&quot;math-container&quot;&gt;$10^{28}$&lt;/span&gt; flops.&lt;/p&gt;&#xA;" OwnerUserId="3070" LastEditorUserId="2444" LastEditDate="2020-12-20T12:02:12.950" LastActivityDate="2020-12-20T12:05:15.230" Title="In 2016, can $1000.00 buy enough operations per second to be approximately equal to the computational power of a human brain?" Tags="&lt;comparison&gt;&lt;human-like&gt;&lt;hardware-evaluation&gt;&lt;human-brain&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2171" PostTypeId="2" ParentId="2170" CreationDate="2016-10-17T06:55:48.123" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, we do have computing systems that do fall in the &lt;a href=&quot;https://en.wikipedia.org/wiki/FLOPS&quot; rel=&quot;nofollow noreferrer&quot;&gt;teraFLOPS&lt;/a&gt; range (where 1 teraflop = 1 trillion FLOPS = &lt;span class=&quot;math-container&quot;&gt;$10^{12}$&lt;/span&gt; FLOPS)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The human brain is a biological system and saying it has some sort of FLOPS ability is just plain dumb because there is no way to take a human brain and measure its FLOPS. You could say &amp;quot;hey, by looking at the neurons activity using fMRI we can reach some sort of approximation&amp;quot;, but comparing the result of this approach with the way FLOPS are measured in computers will be comparing apples with oranges, which again is dumb.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1462" LastEditorUserId="2444" LastEditDate="2020-12-20T11:56:46.980" LastActivityDate="2020-12-20T11:56:46.980" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="2172" PostTypeId="2" ParentId="2170" CreationDate="2016-10-17T11:10:34.013" Score="7" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/FLOPS#Cost_of_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;development of CPUs&lt;/a&gt; didn't quite keep up with Kurzweil's predictions. But if you also &lt;a href=&quot;https://www.cnet.com/products/nvidia-geforce-gtx-295/review/&quot; rel=&quot;nofollow noreferrer&quot;&gt;allow for&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units#GeForce_200_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPU&lt;/a&gt;s, his prediction for 2009 is pretty accurate.&lt;/p&gt;&#xA;&lt;p&gt;I think Moore's law slowed down recently and has now been pretty much &lt;a href=&quot;http://arstechnica.com/information-technology/2016/02/moores-law-really-is-dead-this-time/&quot; rel=&quot;nofollow noreferrer&quot;&gt;abandoned by the industry&lt;/a&gt;. How much that will affect the 2019 prediction remains to be seen. Maybe the industry will hit its stride again with non-silicon based chips, maybe not.&lt;/p&gt;&#xA;&lt;p&gt;And, of course, whether hitting Kurzweil's estimate of the computing power of the human brain will make an appreciable difference for the development of Artificial General Intelligence (AGI) is another question altogether.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2020-12-20T12:05:15.230" LastActivityDate="2020-12-20T12:05:15.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2173" PostTypeId="2" ParentId="2168" CreationDate="2016-10-17T13:20:00.843" Score="1" Body="&lt;p&gt;Well, we are talking about a system (a machine) which develops knowledge (learns), so it is kind of difficult for such a technique to not fall within machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you could argue that inference engines which work on a graph based knowledge database to derive new propositions or probabilities are not part of machine learning. Of course in that case part of the knowledge is not acquired at all, but rather entered by the developers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still reading up on this, but my impression is that these &lt;a href=&quot;https://en.wikipedia.org/wiki/WordNet&quot; rel=&quot;nofollow&quot;&gt;knowledge databases&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Inference_engine&quot; rel=&quot;nofollow&quot;&gt;inference engines&lt;/a&gt; became rather popular in the nineties and many AGI-researchers today still work in that direction. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-10-17T13:20:00.843" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2174" PostTypeId="2" ParentId="2168" CreationDate="2016-10-17T16:14:37.857" Score="1" Body="&lt;p&gt;That depends on how broadly you define &quot;machine learning techniques&quot;.  You could construct a definition so that, by definition, all learning falls under that rubric.  OTOH, there is such a broad array of machine learning techniques that doing so wouldn't not gain one much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It probably makes more sense to talk about the different kinds of learning we use within machine learning / artificial intelligence.  At a minimum, you have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;supervised learning&lt;/li&gt;&#xA;&lt;li&gt;unsupervised learning&lt;/li&gt;&#xA;&lt;li&gt;semi-supervised learning&lt;/li&gt;&#xA;&lt;li&gt;competitive learning&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And then things like &quot;reinforcement learning&quot; which may subcategorize the above.  Most of those things fall into what people generally call &quot;machine learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outside of that, you have things like rule induction algorithms, deductive logic   techniques like inductive logic programming which can sorta-kinda &quot;learn&quot;, inference engines, automated reasoning, etc. which have their own ways of &quot;learning&quot; about the world, but are separate from what's usually labeled &quot;machine learning&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even with that in mind, one can rightly ask if there's really a dividing line there or not.  Indeed, there seems to be reason to think that future AI systems may use a hybrid approach which combines many different techniques without regard for whether or not they are labeled &quot;machine learning&quot; or &quot;GOFAI&quot; or &quot;other&quot;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-10-17T16:14:37.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2176" PostTypeId="1" CreationDate="2016-10-18T15:45:23.340" Score="2" ViewCount="1517" Body="&lt;p&gt;I am creating a snake game in Unity and I would like to implement AI snakes that wander around the globe while avoiding collision with the other snakes on the globe, and if possible I would also like to make the AI snakes purposefully trap other snakes so that the other snakes would collide and die. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aQ61J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aQ61J.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI snakes must meet the following requirements:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They must move in a certain way. A snake is controlled by a user using the arrow keys on a keyboard, therefor I would also like the AI snakes to move using this form of input.&lt;/li&gt;&#xA;&lt;li&gt;The AI snakes must move on a sphere&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As I know, creating Artificial Intelligence is not an easy task and I would like to know if there are some open source projects that I can use for accomplishing this task.&lt;/p&gt;&#xA;" OwnerUserId="3105" LastActivityDate="2016-10-20T11:56:55.070" Title="How to create an AI snake for a video game?" Tags="&lt;gaming&gt;" AnswerCount="4" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2177" PostTypeId="2" ParentId="2176" CreationDate="2016-10-18T17:44:36.757" Score="0" Body="&lt;p&gt;This is a pretty tall order. I can't answer your question for you but I can suggest where to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could look into making a neural network for navigation and simple behaviors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the following youtube video for navigation reference &#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=0Str0Rdkxxo&quot; rel=&quot;nofollow&quot;&gt;https://www.youtube.com/watch?v=0Str0Rdkxxo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This next video shows that using neural networks, you can have an actor make decisions based on another actor.&#xA;&quot;Tank&quot; battle&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=u2t77mQmJiY&quot; rel=&quot;nofollow&quot;&gt;https://www.youtube.com/watch?v=u2t77mQmJiY&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rest is up to you to figure out. Practice with some simple NN's &lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2016-10-18T17:44:36.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2178" PostTypeId="2" ParentId="2127" CreationDate="2016-10-18T18:22:41.927" Score="4" Body="&lt;p&gt;Why are self-driving cars awesome?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Safety: better awareness (due to more sensors), better reaction time, fewer distracted/injured/drunk/texting drivers on the road, etc&lt;/li&gt;&#xA;&lt;li&gt;Convenience: pick up my kids from school, park itself at the grocery store, take itself to be serviced, etc&lt;/li&gt;&#xA;&lt;li&gt;Faster transit: with increased safety, you can increase speed limits, with proper routing algorithms you don't need traffic lights and stop signs any more (when you have dedicated self-driving lanes &amp;amp; intersections)&lt;/li&gt;&#xA;&lt;li&gt;Comfort: recline, read, game, or snooze while traveling (yay!)&lt;/li&gt;&#xA;&lt;li&gt;Cost: subsidize the cost of the vehicle using ads (e.g. projected onto the windshield)&lt;/li&gt;&#xA;&lt;li&gt;etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3107" LastActivityDate="2016-10-18T18:22:41.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2179" PostTypeId="2" ParentId="2176" CreationDate="2016-10-18T19:27:42.860" Score="1" Body="&lt;p&gt;A relatively simple option which uses AI techniques that are 'traditional' for adversarial games (and which is therefore less of a 'research project' than the use of Machine Learning) is &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax#In_general_games&quot; rel=&quot;nofollow&quot;&gt;Minimax&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ingredients for this are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A list of all the actions that a snake can immediately perform from its current position.&lt;/li&gt;&#xA;&lt;li&gt;A measure of quality (a.k.a. 'fitness') for the resulting world state.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Traditionally specified for &lt;em&gt;two&lt;/em&gt; opponents, the minimax algorithm looks a specified number of moves ahead (alternating between opponents at each turn) and attempts to find the world state that maximizes the quality measure for one opponent whilst minimizing it for the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An extension of the two-player algorithm to n opponents (as seemingly required by the OP) is given in &lt;a href=&quot;https://www.diva-portal.org/smash/get/diva2:761634/FULLTEXT01.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-10-18T19:27:42.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2180" PostTypeId="2" ParentId="2176" CreationDate="2016-10-19T03:45:04.197" Score="1" Body="&lt;p&gt;In general, AI in this type of video games is mostly pathfinding (giving the program a map of possible object positions) and/or an algorithm or series of algorithms ( so it looks random or alive ) tied to the users position ( which is known ), so there is nothing really intelligent in the strict sense, it just looks that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case I would look into using Latitude and Longitude coordinates  (Most 3d engines have some variation ) as the basis for a projected grid on a sphere, your snake will also need to be constrained to the sphere surface and rules/algorithms/maps tweaked to get what you want.&lt;/p&gt;&#xA;" OwnerUserId="3020" LastActivityDate="2016-10-19T03:45:04.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2181" PostTypeId="2" ParentId="2117" CreationDate="2016-10-19T06:56:10.607" Score="-1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;if I had some simple enemy AI that I want to have adapt to the players play-style, is this a good opportunity to implement the AI as a Genetic-Algorithm combined with a Neural Network&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sure. Just provide a quality measure for the GA that's related in some manner to the effect of the player's actions on the game state/opponent(s). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if defining an opponent's intelligence, one of the conceptually simplest things would be to give a GA population member a fitness that's inversely proportional to the increase in the player's score over some period of time.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;are Neural Networks always designed as on off signals?)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. In general, they can be considered to perform &lt;em&gt;nonlinear regression&lt;/em&gt;, i.e. a mapping from a vector of real numbers of length n to another of length m. Classification (i.e. 0/1 outputs can be seen as a restricted case of this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per my answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/1618/what-are-the-practical-considerations-of-using-a-genetic-algorithm-to-decide-the/1626#1626&quot;&gt;this AI SE question&lt;/a&gt;, there is a large body of literature (and mature software libraries) for using evolutionary computation to encode neural nets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, some early work in 'online adaptivity using GA-encoded NNs' appeared in the Creatures &lt;a href=&quot;http://creatures.wikia.com/wiki/Creatures_Wikia_Homepage&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mrl.snu.ac.kr/courses/CourseSyntheticCharacter/grand96creatures.pdf&lt;/a&gt; series of games by Steve Grand &lt;a href=&quot;http://mrl.snu.ac.kr/courses/CourseSyntheticCharacter/grand96creatures.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(details)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-10-19T07:02:55.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2182" PostTypeId="2" ParentId="2117" CreationDate="2016-10-19T07:03:35.853" Score="1" Body="&lt;p&gt;Without going in too much detail on how exactly Neural Networks and Generic Algorithms work, I can tell you that both the algorithms are not good candidates for computer games.  They work well in scientific environments where the system is &quot;trained&quot; on a huge data set to adjust the &quot;weights&quot; (variables) for a given problem.  This &quot;training&quot; process requires a lot of processing power, time and a large data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer games, however either needs to run in real-time (no time for training) or turn-based (not enough data for training).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem is that computer games need to free up as much as possible system resources for physics, graphics, sounds and the user interface to improve the player's experience so game developers usually use other lighter techniques (like a rule-based system) to create the illusion of an AI player.&lt;/p&gt;&#xA;" OwnerUserId="3118" LastEditorUserId="3118" LastEditDate="2016-10-19T08:52:59.793" LastActivityDate="2016-10-19T08:52:59.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2183" PostTypeId="2" ParentId="1491" CreationDate="2016-10-19T07:44:43.773" Score="3" Body="&lt;p&gt;As you mentioned in the question, you cannot solve all problems with decision trees.  Decision trees usually works well in a turn-based game with a good heuristic function, but in RTS games takes a different approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a very complex RTS game, one could implemented a rule-based AI. For example&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;given it is the early game use all units to scout for resources&lt;/li&gt;&#xA;&lt;li&gt;if a certain criteria is met build the base a certain way&lt;/li&gt;&#xA;&lt;li&gt;if another criteria is met build an army&lt;/li&gt;&#xA;&lt;li&gt;if the army is big enough, attack&lt;/li&gt;&#xA;&lt;li&gt;if being attacked by the enemy, bring the units back to the base to defend&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Each of these rules could implement various other AI technique, for example use A-star to find the optimal path between a unit's current location and destination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further optimization could be done by &quot;grouping&quot; similar units to act like one unit. e.g. calculate the path for the entire group instead of each individual unit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also add finer grained rules, like if enemy is a certain distance from a unit, move closer and attack or retreat to the base (depending on health, ammo, abilities, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefit of this approach is that a rule-based system executes very fast as no training or decision trees are necessary and this frees up a lot of system resources for visuals like physics and graphics.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The disadvantage is that if the rule system is not complex enough, the player will easily recognize the pattern and the game will become predictable and boring.  You will also notice that the more different units you add to the game, the exponentially more complex the rule system becomes as you have to cater and test interaction between each type of unit in the game otherwise players might find a weakness in the game design and exploit to complete missions in ways it was not designed to be completed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the reasons why multi-player games are so popular is that you do not play against set rules, but against creative people who have the ability to comes up with new strategies you had never seen before.&lt;/p&gt;&#xA;" OwnerUserId="3118" LastActivityDate="2016-10-19T07:44:43.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2185" PostTypeId="1" CreationDate="2016-10-19T13:28:45.640" Score="8" ViewCount="332" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AI is intelligence exhibited by machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have been wondering if with the recent biological advancements, is there already a non-electrical-based &quot;machine&quot; that is programmed by humans in order to be able to behave like a:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;flexible rational agent&lt;/strong&gt; that perceives its environment and takes actions that maximize its chance of success at some goal&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was specifically thinking of viruses and bacteria. Have these been programmed by humans in order to behave as a flexible rational agent (i.e. an AI entity)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there are other organisms that have already been used for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="3128" LastEditorUserId="33" LastEditDate="2016-10-19T14:58:18.497" LastActivityDate="2018-05-04T07:13:07.983" Title="Is Artificial Intelligence restricted to electrical based technology?" Tags="&lt;history&gt;&lt;comparison&gt;&lt;biology&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2186" PostTypeId="2" ParentId="2117" CreationDate="2016-10-19T14:34:23.680" Score="0" Body="&lt;p&gt;For your question there's a brilliant playground emerging!&lt;br&gt;&lt;br&gt;&#xA;Go to &lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow&quot;&gt;https://gym.openai.com/&lt;/a&gt; and explore!&lt;br&gt;&#xA;You'll get interfaces to games if you want to try applying your machine learning skills and compare the performances of your trained AIs with others. And you can let yourself be inspired by the ideas discussed in the community.&lt;br&gt;&#xA;If you're especially into Genetic Algorithms you'll find dicussions there too but I'd suggest digging deeper into Reinforcement Learning.&lt;br&gt;&lt;br&gt;&#xA;If you look at what Google Deep Mind accomplished playing&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Breakout&lt;/li&gt;&#xA;&lt;li&gt;Montezumas Revenge&lt;/li&gt;&#xA;&lt;li&gt;various other Atari Games ..&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and obviously !&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.theverge.com/google-deepmind&quot; rel=&quot;nofollow&quot;&gt;the sensational victory at Go&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;you can say that Reinforcement Learning with (Deep) Neural Networks can be a very promising approach when it comes to training an AI to master games!&lt;/p&gt;&#xA;" OwnerUserId="3132" LastActivityDate="2016-10-19T14:34:23.680" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2188" PostTypeId="2" ParentId="2185" CreationDate="2016-10-19T18:58:36.847" Score="6" Body="&lt;p&gt;Not yet. &lt;a href=&quot;https://en.wikipedia.org/wiki/Synthetic_virology&quot;&gt;Synthetic virology&lt;/a&gt; / &lt;a href=&quot;https://en.wikipedia.org/wiki/Synthetic_biology#Synthetic_life&quot;&gt;Synthetic life&lt;/a&gt; are still in their infancy.&#xA;We can now synthesize simple bacteria (see Craig Venter's &lt;a href=&quot;https://www.ted.com/talks/craig_venter_is_on_the_verge_of_creating_synthetic_life&quot;&gt;fascinating TED talk&lt;/a&gt; and also &lt;a href=&quot;https://www.scientificamerican.com/article/scientists-synthesize-bacteria-with-smallest-genome-yet/&quot;&gt;an article about his recent work&lt;/a&gt;) but definitely nothing that may be called 'rational' in human standards.&lt;/p&gt;&#xA;" OwnerUserId="3138" LastActivityDate="2016-10-19T18:58:36.847" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2190" PostTypeId="1" AcceptedAnswerId="2191" CreationDate="2016-10-20T01:42:51.263" Score="9" ViewCount="514" Body="&lt;p&gt;DeepMind state that their deep Q-network (DQN) was able to continually adapt its behavior while learning to play 49 Atari games.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After learning all games with the same neural net, was the agent able to play them all at 'superhuman' levels simultaneously (whenever it was randomly presented with one of the games) or could it only be good at one game at a time because switching required a re-learn?&lt;/p&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="75" LastEditDate="2016-10-20T02:36:11.080" LastActivityDate="2017-04-30T03:15:23.537" Title="Was DeepMind's DQN Atari game learning simultaneous?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deepmind&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2191" PostTypeId="2" ParentId="2190" CreationDate="2016-10-20T01:59:57.833" Score="2" Body="&lt;p&gt;Switching required a re-learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow&quot;&gt;note that&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We use the same network architecture, learning&#xA;  algorithm and hyperparameters settings across all seven games, showing that our approach is robust&#xA;  enough to work on a variety of games without incorporating game-specific information. While we&#xA;  evaluated our agents on the real and unmodified games, we made one change to the reward structure&#xA;  of the games during training only.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the&#xA;  network has outperformed all previous RL algorithms on six of the seven games we have attempted&#xA;  and surpassed an expert human player on three of them. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-10-20T01:59:57.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2192" PostTypeId="1" AcceptedAnswerId="2194" CreationDate="2016-10-20T09:25:01.037" Score="3" ViewCount="281" Body="&lt;p&gt;I read a lot about the structure of the human brain and artificial neural networks.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to build an AGI (or human-level AI) with artificial neural networks on neuromorphic chips, which would be divided into centers, like the brain is, e.g. centers responsible for feelings, abstract thinking, speech, memory, etc.?&lt;/p&gt;&#xA;" OwnerUserId="3148" LastEditorUserId="2444" LastEditDate="2021-01-26T01:46:55.627" LastActivityDate="2021-01-26T01:49:33.417" Title="Is it possible to build an AGI with neural networks on neuromorphic chips?" Tags="&lt;neural-networks&gt;&lt;agi&gt;&lt;neuromorphic-engineering&gt;&lt;neuroscience&gt;&lt;human-brain&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2193" PostTypeId="2" ParentId="2176" CreationDate="2016-10-20T11:56:55.070" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Divide the globe into a &quot;cells&quot;. Each cell will have a number of neighbours depending on how you have divided your globe. Have a look at &lt;a href=&quot;https://gamedev.stackexchange.com/questions/3360/when-mapping-the-surface-of-a-sphere-with-tiles-how-might-you-deal-with-polar-d&quot;&gt;https://gamedev.stackexchange.com/questions/3360/when-mapping-the-surface-of-a-sphere-with-tiles-how-might-you-deal-with-polar-d&lt;/a&gt; and &lt;a href=&quot;https://gamedev.stackexchange.com/questions/45167/square-game-map-rendered-as-sphere&quot;&gt;https://gamedev.stackexchange.com/questions/45167/square-game-map-rendered-as-sphere&lt;/a&gt; for ideas on how to divide your global.&lt;/li&gt;&#xA;&lt;li&gt;Once all the cells are connected, you can use an &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A-star search algorithm&lt;/a&gt; to find the optimal path for an AI &quot;snake&quot;.&lt;/li&gt;&#xA;&lt;li&gt;Change the heuristic function so that the cells on the opposite side of the opponent are more favourable than the cells on your snake's side. That would cause the AI snake to always try to get to the other side of the opponent with the side-effect of &quot;surrounding&quot; the opponent.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3118" LastEditorUserId="-1" LastEditDate="2017-04-13T12:18:55.587" LastActivityDate="2016-10-20T11:56:55.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2194" PostTypeId="2" ParentId="2192" CreationDate="2016-10-20T18:54:22.163" Score="2" Body="&lt;p&gt;No. The reasons include but are not limited to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;lack of understanding of how the brain works&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;current ANNs are mostly good at pattern recognition and generative   tasks, but lack the capacity to create abstractions on their own&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;we can't match the size/number of artificial neurons to the number of biological neurons&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;even with a much smaller ANN size, performance is an issue (i.e. state of the art image categorization ANNs have to be trained few weeks on multi GPU rigs to match human level).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2997" LastEditorUserId="2444" LastEditDate="2021-01-26T01:49:33.417" LastActivityDate="2021-01-26T01:49:33.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2195" PostTypeId="1" AcceptedAnswerId="2196" CreationDate="2016-10-21T04:53:19.253" Score="0" ViewCount="127" Body="&lt;p&gt;What are the &quot;best&quot; &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_completeness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing complete&lt;/a&gt; programming languages which can be used for developing &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolutionary algorithm&lt;/a&gt;-based AI programs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Best&quot; should be based on pros and cons of performance and easiness for machine learning.&lt;/p&gt;&#xA;" OwnerUserId="3161" LastEditorUserId="2444" LastEditDate="2019-06-20T21:07:07.550" LastActivityDate="2019-06-20T21:07:07.550" Title="What are the best programming languages for developing evolutionary algorithms?" Tags="&lt;evolutionary-algorithms&gt;&lt;programming-languages&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2016-10-21T13:07:46.507" ContentLicense="CC BY-SA 4.0" />
  <row Id="2196" PostTypeId="2" ParentId="2195" CreationDate="2016-10-21T06:20:04.410" Score="1" Body="&lt;p&gt;Most machine learning applications today are built on tensors, matrices, probabilistic / Bayesian inference, neural networks, etc. But those can all be built with any modern programming language (all the useful ones are Turing complete). And the best performing language for any of those will generally be assembly / machine code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.python.org/&quot; rel=&quot;nofollow&quot;&gt;Python&lt;/a&gt; is famous for machine learning, but that may be due to adoption of Python in academia and &lt;a href=&quot;http://www.numpy.org/&quot; rel=&quot;nofollow&quot;&gt;NumPy&lt;/a&gt;, &lt;a href=&quot;https://www.scipy.org/&quot; rel=&quot;nofollow&quot;&gt;SciPy&lt;/a&gt;, etc. Python isn't very performant, but most of the machine libraries leverage native code, so they're fairly performant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://julialang.org/&quot; rel=&quot;nofollow&quot;&gt;Julia&lt;/a&gt; is a new language that is gunning for a lead position in the data science space, which machine learning builds on. It is allegedly very performant over number crunching domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Java has a decent developer ecosystem, and is fairly performant, but the highest performing libraries (including those that leverage GPU) tend to call out to native code via JNI. See &lt;a href=&quot;https://deeplearning4j.org/&quot; rel=&quot;nofollow&quot;&gt;DeepLearning4J&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally like &lt;a href=&quot;http://clojure.org/&quot; rel=&quot;nofollow&quot;&gt;Clojure&lt;/a&gt; - a modern Lisp running on the Java JVM. There's a new deep learning project called &lt;a href=&quot;https://github.com/thinktopic/cortex&quot; rel=&quot;nofollow&quot;&gt;Cortex&lt;/a&gt; built on Clojure and some fast native libraries, including GPU acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think Clojure provides a great balance of being able to easily wrap performant libraries with highly expressive, succinct and simple programming idioms.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-10-21T06:20:04.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2197" PostTypeId="2" ParentId="2185" CreationDate="2016-10-21T22:53:05.187" Score="5" Body="&lt;p&gt;No, I think electricity is not essential for AI.  In theory AI (a sufficient collection of computational processes that can adapt to changes in their input, thus producing 'intelligent' behavior), &lt;em&gt;could&lt;/em&gt; be implemented using any mechanism that can compute that set of essential functions needed to create AI.  Basically I'm suggesting the possibility of combining a set of non-electric Turing-equivalent machines into a collective that together can reach the AI-level of performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine_equivalents&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Turing_machine_equivalents&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If AI can be implemented using an electronic computer, it should also be possible to implement it using any non-electronic machine that is computationally equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To date, several non-electronic machines have been proposed as Turing-equivalent: DNA computers, quantum computers, Babbage's Analytical Engine, animal brains, maybe even a really big network of daisies (perhaps that can communicate via their rhizomes).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, it's plausible that one day we could create a network composed of small brains (perhaps from a less smart species than humans) that with the right kind of genetically architected biological interconnect and scheduler could route data through its network to control a robot -- thus we'd have a synthetic biological AI engine whose brain is made up of 100 chimpanzees, or 10,000 hamster brains, or maybe even 1 million nematodes.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-10-21T22:53:05.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2198" PostTypeId="1" AcceptedAnswerId="2199" CreationDate="2016-10-22T12:55:27.067" Score="3" ViewCount="125" Body="&lt;p&gt;I have a question. Will we be able to build a neural network that thinks abstractly, has the creativity, feels and is conscious?&lt;/p&gt;&#xA;" OwnerUserId="3148" LastActivityDate="2016-10-22T18:14:50.290" Title="Will it ever be possible to construct a neural network that could have the features of human brain?" Tags="&lt;neural-networks&gt;&lt;human-like&gt;&lt;human-inspired&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2016-10-23T16:20:59.140" ContentLicense="CC BY-SA 3.0" />
  <row Id="2199" PostTypeId="2" ParentId="2198" CreationDate="2016-10-22T18:14:50.290" Score="0" Body="&lt;p&gt;Maybe in the distant future they could build a computer powerful enough to simulate the individual neurons of an entire human brain. Then they could carefully copy/paste the connectivity of a sample brain into the computer simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that this extreme seems physically possible, it stands to reason that much simpler/smaller alternatives could be engineered in the future assuming continuous advancements in technology.&lt;/p&gt;&#xA;" OwnerUserId="2983" LastActivityDate="2016-10-22T18:14:50.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2201" PostTypeId="1" CreationDate="2016-10-23T17:09:24.797" Score="6" ViewCount="3844" Body="&lt;p&gt;If I have a set of sensory nodes taking in information and a set of &quot;action nodes&quot; which determine the behavior of my robot, why do I need hidden nodes between them when I can let all sensory nodes affect all action nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is in the context of evolving neural network)&lt;/p&gt;&#xA;" OwnerUserId="1321" LastEditorUserId="3210" LastEditDate="2016-11-01T04:40:10.877" LastActivityDate="2018-12-29T04:09:50.037" Title="What is the purpose of hidden nodes in neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2202" PostTypeId="2" ParentId="2201" CreationDate="2016-10-23T18:49:12.407" Score="1" Body="&lt;p&gt;Normally one node/layer applies linear fitting of the the input to the hypothesis, in other words uses linear function (&lt;span class=&quot;math-container&quot;&gt;$y = ax + b$&lt;/span&gt;). Adding layers chains liner functions, potentially allowing fitting higher order functions. A great explanation can be found &lt;a href=&quot;http://colah.github.io/posts/2015-01-Visualizing-Representations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2997" LastEditorDisplayName="user1440" LastEditDate="2018-12-28T19:34:51.137" LastActivityDate="2018-12-28T19:34:51.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2203" PostTypeId="1" AcceptedAnswerId="2209" CreationDate="2016-10-23T19:46:36.997" Score="10" ViewCount="767" Body="&lt;p&gt;&lt;em&gt;If neurons and synapses can be implemented using transistors, what prevents us from creating arbitrarily large neural networks using the same methods with which GPUs are made?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, we have seen how extraordinarily well virtual neural networks implemented on sequential processors work (even GPUs are sequential machines, but with huge amounts of cores). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can imagine that using GPU design principles - which is basically to have thousands of programmable processing units that work in parallel - we could make much simpler &quot;neuron processing units&quot; and put millions or billions of those NPUs in a single big chip. They would have their own memory (for storing weights) and be connected to a few hundred other neurons by sharing a bus. They could have a frequency of for example 20 Hz, which would allow them to share a data bus with many other neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, there are some electrical engineering challenges here, but it seems to me that all big tech companies should be exploring this route by now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many AI researchers say that superintelligence is coming around the year 2045. I believe that their reasoning is based on Moore's law and the number of neurons we are able to implement in software running on the fastest computers we have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the fact is, we today are making silicon chips with billions of transistors on them. SPARK M7 has 10 billion transistors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If implementing a (non-programmable) neuron and a few hundred synapses for it requires for example 100 000 transistors, then we can make a neural network in hardware that emulates 100 000 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we design such a chip so that we can simply make it physically bigger if we want more neurons, then it seems to me that arbitrarily large neural networks are simply a budget question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember: I am NOT asking if such a network will in fact be very intelligent. I am merely asking if we can factually make arbitrarily large, highly interconnected neural networks, if we decide to pay Intel to do this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implication is that on the day some scientist is able to create general intelligence in software, we can use our hardware capabilities to grow this general intelligence to human levels and beyond.&lt;/p&gt;&#xA;" OwnerUserId="3211" LastEditorUserId="2444" LastEditDate="2020-03-05T16:09:05.237" LastActivityDate="2020-03-05T16:09:05.237" Title="Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;hardware&gt;&lt;implementation&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2204" PostTypeId="2" ParentId="2203" CreationDate="2016-10-23T21:59:44.617" Score="2" Body="&lt;p&gt;While a single transistor could approximate the basic function of a single neuron, I cannot agree that any electronic element could simulate the synapses/axons. Transistors are etched on a flat surface, and could be interconnected only to adjacent or close by transistors. Axons in the brain span huge distances (compared to the size of the neuron itself), and are not restricted to a two dimensional surface. Even if we were able to approach the number of transistors on a processor to the number of neurons in a brain, we are no where near as number of connections. It could also be argued that the analogue signals in the brain carry more information per unit of time, compared to the binary impulses on a chip. Furthermore, the brain actually have plasticity i.e. connections between neurons can be weakened/discarded or straightened/created, while a CPU cannot do that.&lt;/p&gt;&#xA;" OwnerUserId="2997" LastActivityDate="2016-10-23T21:59:44.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2205" PostTypeId="2" ParentId="2203" CreationDate="2016-10-23T22:04:12.310" Score="0" Body="&lt;p&gt;You may want to consider this &lt;a href=&quot;http://scienceblogs.com/developingintelligence/2007/03/27/why-the-brain-is-not-like-a-co/&quot; rel=&quot;nofollow&quot;&gt;list&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;10 important differences between brains and computers:&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Brains are analog , computers are digital &lt;/li&gt;&#xA;  &lt;li&gt;The brain uses content-addressable memory&lt;/li&gt;&#xA;  &lt;li&gt;The brain is a massively parallel machine; computers are modular and serial &lt;/li&gt;&#xA;  &lt;li&gt;Processing speed is not fixed in the brain; there is no system clock &lt;/li&gt;&#xA;  &lt;li&gt;Short-term memory is not like RAM &lt;/li&gt;&#xA;  &lt;li&gt;No hardware/software distinction can be made with respect to the brain or mind &lt;/li&gt;&#xA;  &lt;li&gt;Synapses are far more complex than electrical logic gates &lt;/li&gt;&#xA;  &lt;li&gt;Unlike computers, processing and memory are performed by the same components in the brain &lt;/li&gt;&#xA;  &lt;li&gt;The brain is a self-organizing system&lt;/li&gt;&#xA;  &lt;li&gt;Brains have bodies&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4" LastActivityDate="2016-10-23T22:04:12.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2206" PostTypeId="2" ParentId="2201" CreationDate="2016-10-23T22:35:38.523" Score="5" Body="&lt;p&gt;A feed forward neural network without hidden nodes can only find linear decision boundaries. However, most of the time you need non-linear decision boundaries. Hence you need hidden nodes with a non-linear activation function. The more hidden nodes you have, the more data you need to find good parameters, but the more complex decision boundaries you can find.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2016-10-25T08:00:48.090" LastActivityDate="2016-10-25T08:00:48.090" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2208" PostTypeId="2" ParentId="2203" CreationDate="2016-10-24T05:50:02.653" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If neurons and synapses can be implemented using transistors, &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I hope you are not talking about the neural networks which are currently winning all competitions in machine learning (MLPs, CNNs, RNNs, Deep Residual Networks, ...). Those were once used as a model for neurons, but they are only &lt;em&gt;very&lt;/em&gt; loosely related to what happens in real brain cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Spiking networks should be much closer to real neurons. I've heard that the Hodgkin-Huxley model is quite realistic. However - in contrast to the models I named above - there seems not to be an effective training algorithm for spiking networks.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;what prevents us from creating arbitrarily large neural networks&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Computational resources&lt;/strong&gt;: Training neural networks takes a lot of time. We are talking about ~12 days with a GPU cluster for some CNN models in computer vision.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Training data&lt;/strong&gt;: The more variables you add to the model, the more data you need to estimate those. Neural networks are not magic. They need something they can work with.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But the fact is, we today are making silicon chips with billions of transistors on them. SPARK M7 has 10 billion transistors.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If implementing a (non-programmable) neuron and a few hundred synapses for it requires for example 100 000 transistors, then we can make a neural network in hardware that emulates 100 000 neurons.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's not that simple:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Asynchonosity&lt;/strong&gt;: Biological neural networks work asynchronously. This means one neuron might be active while all others are not active.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Emulation&lt;/strong&gt;: You assume it would only need one cycle to simulate a biological neuron. However, it needs many thousand cycles. You can't simply use more computational units, because some things are not parallelizable. For example, think of the function &lt;code&gt;f(x) = sin(x*x + 1)&lt;/code&gt;. For a human, there are basically three computations: &lt;code&gt;r1 = x*x&lt;/code&gt;, &lt;code&gt;r2 = r1 + 1&lt;/code&gt;, &lt;code&gt;r3 = sin(r2)&lt;/code&gt;. Even if you have 3 people working on calculating the result, you will not be faster than the single fastest person in this group is. Why? Because you need the results of the last computation. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2016-10-24T05:50:02.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2209" PostTypeId="2" ParentId="2203" CreationDate="2016-10-24T07:21:32.677" Score="6" Body="&lt;p&gt;The approach you describe is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering&quot; rel=&quot;noreferrer&quot;&gt;neuromorphic computing&lt;/a&gt; and it's &lt;a href=&quot;https://www.technologyreview.com/s/526506/neuromorphic-chips/&quot; rel=&quot;noreferrer&quot;&gt;quite&lt;/a&gt; a &lt;a href=&quot;https://www.uni-heidelberg.de/presse/news2016/pm20160316-neuromorphic-computer-coming-online.html&quot; rel=&quot;noreferrer&quot;&gt;busy&lt;/a&gt; &lt;a href=&quot;http://www.nextplatform.com/2016/02/09/the-second-coming-of-neuromorphic-computing/&quot; rel=&quot;noreferrer&quot;&gt;field&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;IBM's &lt;a href=&quot;http://www.research.ibm.com/articles/brain-chip.shtml&quot; rel=&quot;noreferrer&quot;&gt;TrueNorth&lt;/a&gt; even has spiking neurons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem with these projects is that nobody quite knows what to do with them yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These projects don't try to create chips that are optimised to &lt;em&gt;run&lt;/em&gt; a neural network. That would certainly be possible, but the expensive part is the &lt;em&gt;training&lt;/em&gt; not the running of neural networks. And for the training you need huge matrix multiplications, something GPUs are very good at already. (&lt;a href=&quot;https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html&quot; rel=&quot;noreferrer&quot;&gt;Google's TPU&lt;/a&gt; would be a chip optimised to run NNs.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do research on algorithms that might be implemented in the brain (we hardly know anything about that) you need flexibility, something these chips don't have. Also, the engineering challenge likely lies in providing a lot of synapses, just compare the average number of synapses per neuron of TrueNorth, 256, and the brain, 10,000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you could create a chip designed after some neural architecture and it would be faster, more efficient, etc …, but to do that you'll need to know which architecture works first. We know that deep learning works, so google uses custom made hardware to run their applications and I could certainly imagine custom made deep learning hardware coming to a smartphone near you in the future. To create a neuromorphic chip for strong AI you'd need to develop strong AI first.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-10-24T07:21:32.677" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2210" PostTypeId="2" ParentId="2092" CreationDate="2016-10-24T08:05:09.393" Score="2" Body="&lt;p&gt;One critical part of AI is machine learning (ML). The common definition of ML by Mitchell is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If this type of program is useful in an &quot;everyday application&quot; depends on the application. Here are some examples which would not be possible without ML:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Spam detection (e.g. e-mails, forums)&lt;/li&gt;&#xA;&lt;li&gt;Fraud detection (e.g. credit cards)&lt;/li&gt;&#xA;&lt;li&gt;Image recognition (e.g. if you want to automatically filter NSFW content, automatic adding of tags / making images searchable e.g. for Google Image search)&lt;/li&gt;&#xA;&lt;li&gt;Video analysis (filtering copyrighted work e.g. on YouTube)&lt;/li&gt;&#xA;&lt;li&gt;Speech recognition (e.g. hotlines, automatic caption generation)&lt;/li&gt;&#xA;&lt;li&gt;Autocompletion (probably one of the simplest things you can do with data)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2016-10-24T08:05:09.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2211" PostTypeId="1" CreationDate="2016-10-24T11:42:28.893" Score="13" ViewCount="1540" Body="&lt;p&gt;I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative (&lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;) and the other discriminative (&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;). The generative neural network generates some data which the discriminative neural network judges for correctness.&lt;/p&gt;&#xA;&lt;p&gt;How do the discriminative (&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;) neural nets initially know whether the data produced by &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is correct or not? Do we have to train the &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; first then add it into the GAN with &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Let's consider my trained &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; net, which can classify a picture with 90% accuracy. If we add this &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to a GAN, there is a 10% probability it will classify an image wrong. If we train a GAN with this &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, then will it also have the same 10% error in classifying an image? If yes, then why do GANs show promising results?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2020-11-22T12:30:58.357" LastActivityDate="2020-11-22T12:30:58.357" Title="How are generative adversarial networks trained?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;generative-adversarial-networks&gt;&lt;self-supervised-learning&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="2212" PostTypeId="2" ParentId="2211" CreationDate="2016-10-24T19:20:48.323" Score="4" Body="&lt;h1&gt;Compare generated and real data&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;All the results produced by G are always considered &quot;wrong&quot; by definition, even for a very good generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You provide the discriminative neural network &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; with a mix of results generated by the generator network &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and real results from an outside source, and then you train it to distinguish if the result was produced by the generator or not - you're not comparing &quot;good&quot; and &quot;bad&quot; results, you're comparing real versus generated results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will result in a &quot;mutual evolution&quot; as &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; will learn to find features that separate real results from generated ones, and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; will learn how to generate results that are hard to distinguish from real data.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastEditorUserId="1675" LastEditDate="2019-04-15T16:51:11.860" LastActivityDate="2019-04-15T16:51:11.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2214" PostTypeId="1" AcceptedAnswerId="2217" CreationDate="2016-10-25T08:59:44.940" Score="1" ViewCount="273" Body="&lt;p&gt;Artificial intelligence techniques have been successfully applied in many scenarios and to automate several jobs.&lt;/p&gt;&#xA;&lt;p&gt;When will an artificial intelligence system do the job of a software developer or tester? Is this possible (for example, in the next 10 years)?&lt;/p&gt;&#xA;" OwnerUserId="2930" LastEditorUserId="2444" LastEditDate="2020-09-23T16:50:28.003" LastActivityDate="2021-02-04T14:53:11.090" Title="When will an artificial intelligence replace a software developer or tester?" Tags="&lt;social&gt;&lt;ai-complete&gt;&lt;software-development&gt;&lt;futurism&gt;&lt;ai-milestones&gt;" AnswerCount="4" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2216" PostTypeId="2" ParentId="2211" CreationDate="2016-10-25T11:58:19.457" Score="2" Body="&lt;p&gt;A &lt;em&gt;discriminative&lt;/em&gt; network (&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;) &lt;strong&gt;learns&lt;/strong&gt; to &lt;em&gt;discriminate&lt;/em&gt; by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we expect network &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to improve the ability of network &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; to generate better and better images (or other kind of data), as it try to &quot;trick&quot; network &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; by producing new data that is more similar to &quot;real data&quot;. It is not about the accuracy of network &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; at all. &lt;strong&gt;It is not about improving the accuracy&lt;/strong&gt;, it is about improving the ability of the computer to generate more &quot;believable&quot; data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, using this scenario could be a good &quot;unsupervised&quot; way to improve the classification power of neural networks, as it forces the generator model to learn better features of real data, and to learn how to distinguish between actual features and noise, using much less data that is needed for a traditional supervised learning scheme. &lt;/p&gt;&#xA;" OwnerUserId="3250" LastEditorUserId="2444" LastEditDate="2019-04-15T16:18:28.033" LastActivityDate="2019-04-15T16:18:28.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2217" PostTypeId="2" ParentId="2214" CreationDate="2016-10-25T12:12:23.607" Score="4" Body="&lt;p&gt;The &lt;strong&gt;ultimate goal of machine learning&lt;/strong&gt; is to bypass the developer...&#xA;When we will have a &quot;master algorithm&quot; that can learn how to generalize any function or algorithm from examples, it can essentially replace any developer, skip the 'development&quot; stage, going from problem directly to algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can't know when this will happen, but as we surrounded with multiple creatures (humans and animals) which can demonstrate learning algorithms and predictive model learning without any &quot;developer&quot; - we can assume that such an algorithm is possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I would have to guess, I would say that we are probably very near the point where &quot;developers&quot; and &quot;testers&quot; will be replaced by learning algorithms. We could be a decade or two away from the point where people will not write any code or any testing at all. Programs and automation will be derived directly from describing the problems themselves in a natural language, visualizations or data collections. However, we still need some breakthroughs in combining feature learning with active memory, unsupervised learning, and artificial common sense.   &lt;/p&gt;&#xA;" OwnerUserId="3250" LastActivityDate="2016-10-25T12:12:23.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2219" PostTypeId="1" CreationDate="2016-10-26T08:11:31.073" Score="8" ViewCount="1717" Body="&lt;p&gt;I know how a machine can learn to play Atari games (Breakout): &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Reinforcement Learning&lt;/a&gt;. With the same technique, it is even possible to play FPS games (Doom): &lt;a href=&quot;https://arxiv.org/pdf/1609.05521&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing FPS Games with Reinforcement Learning&lt;/a&gt;. Further studies even investigated multiagent scenarios (Pong): &lt;a href=&quot;https://arxiv.org/pdf/1511.08779.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiagent Cooperation and Competition with Deep Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And even another awesome article for the interested user in the context of deep reinforcement learning (easy and a must-read for beginners): &lt;a href=&quot;http://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demystifying Deep Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I was thrilled by these results and immediately wanted to try them in some simple &amp;quot;board/card game scenarios&amp;quot;, i.e. writing AI for some simple games in order to learn more about &amp;quot;deep learning&amp;quot;. Of course, thinking that I can apply the techniques above easily in my scenarios was stupid. All examples above are based on convolutional nets (image recognition) and some other assumptions, which might not be applicable in my scenarios.&lt;/p&gt;&#xA;&lt;p&gt;I have two main questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a card game and the AI shall play a card from its hand, you could think about the cards (amongst other stuff) as the current game state. You can easily define some sort of neural net and feed it with the card data. In a trivial case, the cards are just numbered. I do not know the net type, which would be suitable, but I guess deep reinforcement learning strategies could be applied easily then.&lt;/p&gt;&#xA;&lt;p&gt;However, I can only imagine this, if there is a constant number of hand cards. In the examples above, the number of pixels is also constant, for example. What if a player can have a different number of cards? What to do, if a player can have an infinite number of cards? Of course, this is just a theoretical question as no game has an infinite number of cards.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the initial examples, the action space is constant. What can you do, if the action space is not? This more or less follows from my previous problem. If you have 3 cards, you can play cards 1, 2, or 3. If you have 5 cards, you can play cards 1, 2, 3, 4 or 5, etc. It is also common in card games, that it is not allowed to play a card. Could this be tackled with a negative reward?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, which &amp;quot;tricks&amp;quot; can be used, e.g. always assume a constant number of cards with &amp;quot;filling values&amp;quot;, which is only applicable in the non-infinite case (anyways unrealistic and even humans could not play well with that)? Are there articles, which examine such things already?&lt;/p&gt;&#xA;" OwnerUserId="3270" LastEditorUserId="2444" LastEditDate="2020-09-21T22:26:55.907" LastActivityDate="2020-09-21T22:26:55.907" Title="How can you represent the state and action spaces for a card game in the case of a variable number of cards and actions?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;deep-rl&gt;&lt;reference-request&gt;&lt;markov-decision-process&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2220" PostTypeId="2" ParentId="2185" CreationDate="2016-10-26T16:32:00.120" Score="6" Body="&lt;p&gt;Any logic circuit admits a variety of implementations.  All programs executing on conventional digital processors can be expressed as logic circuits.  Among the possible implementations of logic circuits are fluidic implementations, which do not depend on electronics per se.  Thus it is in principle possible to implement, e.g. a POMDP processor (responsive to your specific question) in fluidics, albeit perhaps impractical at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know of no general theory of Turing-completeness for analog computers, which would suffice to determine whether some alternative physical substrate, be it biological or not biological, can compute recursively enumerable functions.  That is a sufficient but not a necessary condition for answering your question regarding any given medium. Usually the easiest way to demonstrate the sufficient condition will be to demonstrate the ability to construct a NAND gate, and to combine such gates into general circuits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another non-electronic example: Quantum computers may be non-electronic, at least in their processing elements, and are able to compute general deterministic logic circuits.&lt;/p&gt;&#xA;" OwnerUserId="3278" LastActivityDate="2016-10-26T16:32:00.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2221" PostTypeId="2" ParentId="2219" CreationDate="2016-10-26T16:46:45.657" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;Filling values is totally fine. In the case of image recognition the filling will be the background of the image (&lt;a href=&quot;https://www.google.com/search?q=mnist+images&amp;amp;tbm=isch&quot; rel=&quot;nofollow&quot;&gt;examples&lt;/a&gt;). For example in Belot you have total of 32 cards, which can be 32 boolean features. You can set the ones the player has to 1, while the rest are 0. Note that the in most games you'll need more features than the cards in your hand. I.e number of the round, cards that have been played so far, calls that have been made etc. &lt;/li&gt;&#xA;&lt;li&gt;Defining the scope of the &quot;action space&quot; will be specific to the game. For Belot, it can be number encoding for each of the 32 cards.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can find articles via Google. &lt;a href=&quot;http://homes.soic.indiana.edu/adamw/hearts.pdf&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a paper about ML for a card game. Instead of articles, I'd recommend checking out a course on ML (i.e. Coursera and Udacity have good free online courses).&lt;/p&gt;&#xA;" OwnerUserId="2997" LastEditorUserId="2997" LastEditDate="2016-10-27T11:21:43.047" LastActivityDate="2016-10-27T11:21:43.047" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2226" PostTypeId="1" CreationDate="2016-10-27T19:57:17.187" Score="6" ViewCount="1983" Body="&lt;p&gt;The &amp;quot;discounted sum of future rewards&amp;quot; (or return) using discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\gamma^1 r_1 +\gamma^2 r_2 + \gamma^3 r_2 + \dots \tag{1}\label{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r_i$&lt;/span&gt; is the reward received at the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th time-step.&lt;/p&gt;&#xA;&lt;p&gt;I am confused as to what constitutes a &lt;em&gt;time-step&lt;/em&gt;. Say, I take an action now, so I will get a reward in 1 time-step. Then, I will take an action again in timestep 2 to get a second reward in time-step 3.&#xA;But the formula \ref{1} suggests something else.&lt;/p&gt;&#xA;&lt;p&gt;How does one define a time-step? Can we take action as well receive a reward in a single step?&lt;/p&gt;&#xA;&lt;p&gt;Examples are most helpful.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="2444" LastEditDate="2021-01-12T10:49:02.027" LastActivityDate="2021-01-12T10:49:02.027" Title="What is a time-step in a Markov Decision Process?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;markov-decision-process&gt;&lt;return&gt;&lt;time-step&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2229" PostTypeId="2" ParentId="1290" CreationDate="2016-10-28T09:00:13.520" Score="8" Body="&lt;p&gt;On the suggestion of the O.P. rcpinto I converted a comment about seeing &quot;around a half-dozen papers that follow up on Graves et al.'s work which have produced results of the  caliber&quot; and will provide a few links. Keep in mind that this only answers the part of the question pertaining to NTMs, not Google DeepMind itself, plus I'm still learning the ropes in machine learning, so some of the material in these papers is over my head; I did manage to grasp much of the material in &lt;a href=&quot;https://arxiv.org/pdf/1410.5401.pdf&quot;&gt;Graves et al.'s original paper&lt;/a&gt;{1] though and am close to having homegrown NTM code to test. I also at least skimmed the following papers over the last few months; they do not replicate the NTM study in a strict scientific manner, but many of their experimental results do tend to support the original at least tangentially:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• In &lt;a href=&quot;https://arxiv.org/pdf/1607.00036.pdf&quot;&gt;this paper&lt;/a&gt; on a variant version of NTM addressing, Gulcehere, et al. do not try to precisely replicate Graves et al.'s tests, but like the DeepMind team, it does demonstrate markedly better results for the original NTM and several variants over an ordinary recurrent LSTM. They use 10,000 training samples of a Facebook Q&amp;amp;A dataset, rather than the N-grams Graves et al. operated on in their paper, so it's not replication in the strictest sense. They did however manage to get a version of the original NTM and several variants up and running, plus recorded the same magnitude of performance improvement.&lt;a href=&quot;https://arxiv.org/pdf/1607.00036.pdf&quot;&gt;2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Unlike the original NTM, &lt;a href=&quot;https://arxiv.org/abs/1505.00521&quot;&gt;this study&lt;/a&gt; tested a version of reinforcement learning which was not differentiable; that may be why they were unable to solve several of the programming-like tasts, like Repeat-Copy, unless the controller wasn't confined to moving forwards. Their results were nevertheless good enough to lend support to the idea of NTMs. A more recent revision of their paper is apparently available, which I have yet to read, so perhaps some of their variant's problems have been solved.&lt;a href=&quot;https://arxiv.org/abs/1505.00521&quot;&gt;3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Instead of testing the original flavor of NTM against ordinary neural nets like LSTMs, &lt;a href=&quot;https://arxiv.org/pdf/1510.03931v3.pdf&quot;&gt;this paper&lt;/a&gt; pitted it against several more advanced NTM memory structures. They got good results on the same type of programming-like tasks that Graves et al. tested, but I don't think they were using the same dataset (it's hard to tell from the way their study is written just what datasets they were operating on).&lt;a href=&quot;https://arxiv.org/pdf/1510.03931v3.pdf&quot;&gt;4&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;• On p. 8 of &lt;a href=&quot;https://arxiv.org/pdf/1605.06065.pdf&quot;&gt;this study&lt;/a&gt;, an NTM clearly outperforms several LSTM, feed-forward and nearest-neighbor based schemes on an Omniglot character recognition dataset. An alternative approach to external memory cooked up by the authors clearly beats it, but it still obviously performs well. The authors seem to belong to a rival team at Google, so that might be an issue when assessing replicability.&lt;a href=&quot;https://arxiv.org/pdf/1605.06065.pdf&quot;&gt;5&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• On p. 2 &lt;a href=&quot;http://www.thespermwhale.com/jaseweston/ram/papers/paper_6.pdf&quot;&gt;these authors&lt;/a&gt; reported getting better generalization on &quot;very large sequences&quot; in a test of copy tasks, using a much smaller NTM network they evolved with the genetic NEAT algorithm, which dynamically grows topologies.&lt;a href=&quot;http://www.thespermwhale.com/jaseweston/ram/papers/paper_6.pdf&quot;&gt;6&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NTMs are fairly new so there hasn't been much time to stringently replicate the original research yet, I suppose. The handful of papers I skimmed over the summer, however, seem to lend support to their experimental results; I have yet to see any that report anything but excellent performance. Of course I have an availability bias, since I only read the pdfs I could easily find in a careless Internet search. From that small sample it seems that most of the follow-up research has been focused on extending the concept, not replication, which would explain the lack of replicability data. I hope that helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1410.5401.pdf&quot;&gt;1&lt;/a&gt; Graves, Alex; Wayne, Greg and Danihelka, Ivo, 2014, &quot;Neural Turing Machines,&quot; published Dec. 10, 2014. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1607.00036.pdf&quot;&gt;2&lt;/a&gt; Gulcehre, Caglar; Chandar, Sarath; Choy, Kyunghyun and Bengio, Yoshua, 2016, &quot;Dynamic Neural Turing machine with Soft and Hard Addressing Schemes,&quot; published June 30, 2016. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1505.00521&quot;&gt;3&lt;/a&gt; Zaremba, Wojciech and Sutskever, Ilya, 2015, &quot;Reinforcement Learning Neural Turing Machines,&quot; published May 4, 2015. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1510.03931v3.pdf&quot;&gt;4&lt;/a&gt; Zhang; Wei; Yu, Yang and Zhou, Bowen, 2015, &quot;Structured Memory for Neural Turing Machines,&quot; published Oct. 25, 2015.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1605.06065.pdf&quot;&gt;5&lt;/a&gt; Santoro, Adam; Bartunov, Sergey; Botvinick, Matthew; Wierstra, Daan and Lillicrap, Timothy, 2016, &quot;One-Shot Learning with Memory-Augmented Neural Networks,&quot; published May 19, 2016.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.thespermwhale.com/jaseweston/ram/papers/paper_6.pdf&quot;&gt;6&lt;/a&gt; Boll Greve, Rasmus; Jacobsen, Emil Juul and Sebastian Risi, date unknown, &quot;Evolving Neural Turing Machines.&quot; No publisher listed&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All except (perhaps) Boll Greve et al. were published at the Cornell Univeristy Library arXiv.org Repository: Ithaca, New York.&lt;/p&gt;&#xA;" OwnerUserId="1427" LastActivityDate="2016-10-28T09:00:13.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2230" PostTypeId="2" ParentId="2127" CreationDate="2016-10-28T11:29:45.403" Score="3" Body="&lt;p&gt;I'd like to add, self-driving cars would also be excellent for disabled people who would otherwise not be able to drive. Adds a lot more autonomy to vulnerable people&lt;/p&gt;&#xA;" OwnerDisplayName="user3313" LastActivityDate="2016-10-28T11:29:45.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2231" PostTypeId="1" CreationDate="2016-10-29T06:49:39.403" Score="17" ViewCount="1046" Body="&lt;p&gt;Decades ago there were and are books in machine vision, which by implementing various information processing rules from gestalt psychology, got impressive results with little code or special hardware in image identification and visual processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are such methods being used or worked on today? Was any progress made on this? Or was this research program dropped? By today, I mean 2016, not 1995 or 2005.&lt;/p&gt;&#xA;" OwnerUserId="1366" LastEditorDisplayName="user23290" LastEditDate="2019-12-05T00:34:21.187" LastActivityDate="2021-03-28T16:02:15.310" Title="Are information processing rules from Gestalt psychology still used in computer vision today?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2232" PostTypeId="2" ParentId="2127" CreationDate="2016-10-29T08:26:08.497" Score="1" Body="&lt;p&gt;Self driving cars are good for the following reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the case of an emergancy, urgancy, or just someone being unable to drive unexpactedly, the car can go by itself to a designated location - this is useful in so many use cases - kids who need to get somewhere while parents are busy, Parents who drank a little too much and prefer to take 'the cab' home, or while running, you got injured and need a pick-up.&lt;/li&gt;&#xA;&lt;li&gt;The examples above are for the more obvious things, which we currently have a struggle with. but other than those, Self-driving cars will open a door for a much wider scale of things: safe police chases (just a car without a police officer), taxies, help in the battle field, and much more...&lt;/li&gt;&#xA;&lt;li&gt;The third and most important benefit, is the safety and economical properties of self driving cars: with a lot of those cars on the road, they can 'understand' each other and nothing will go unpredicted. they have much faster response time then humans, and maybe in the future they will even be able to predict traffic-light changes, and by that save gas and money (even more than what they can save right now by driving economicly)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3328" LastActivityDate="2016-10-29T08:26:08.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2233" PostTypeId="2" ParentId="35" CreationDate="2016-10-30T19:12:31.607" Score="16" Body="&lt;h3&gt;Artificial intelligence&lt;/h3&gt;&#xA;&lt;p&gt;According to the book &lt;a href=&quot;https://www.cin.ufpe.br/%7Etfl2/artificial-intelligence-modern-approach.9780131038059.25368.pdf#page=27&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (section 1.1), artificial intelligence (AI) has been defined in &lt;em&gt;multiple&lt;/em&gt; ways, which can be organized into 4 categories.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Thinking Humanly&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Thinking Rationally&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Acting Humanly&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Acting Rationally&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Figure 1.1 (of the same book) contains 8 definitions (by renowned people like Bellman, Winston or Kurzweil).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O9J5Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O9J5Q.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each box contains 2 similar definitions (i.e. both fall into the same category). These definitions vary along 2 dimensions. The definitions in the top row are concerned with &lt;strong&gt;thought-processes&lt;/strong&gt; and &lt;strong&gt;reasoning&lt;/strong&gt;, while the ones in the bottom are concerned with &lt;strong&gt;behaviour&lt;/strong&gt;. The definitions on the left are associated with &lt;strong&gt;human intelligence&lt;/strong&gt;, while the ones on the right with an idealized version of intelligence, which the authors of the AIMA book call &lt;strong&gt;rationality&lt;/strong&gt;. So, for example, the definitions in the top-left corner are based on &lt;strong&gt;thinking humanly&lt;/strong&gt;, while the definitions on the bottom-right corner are based on &lt;strong&gt;acting rationally&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is also a definition of AI by &lt;a href=&quot;http://jmc.stanford.edu/artificial-intelligence/what-is-ai/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;John McCarthy&lt;/a&gt;, who is &lt;a href=&quot;https://en.wikipedia.org/wiki/Dartmouth_workshop&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of the official founders of the AI field in 1956&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;Machine learning&lt;/h3&gt;&#xA;&lt;p&gt;There have been also multiple (similar) definitions of machine learning (ML). For example, Tom Mitchell, in section 1.1. of his book &lt;a href=&quot;http://www.cs.ubbcluj.ro/%7Egabis/ml/ML-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf#page=14&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt;, defines machine learning as follows.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A computer program is said to learn from experience &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; with respect&#xA;to some class of tasks &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and performance measure &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, if its performance at tasks in &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, as measured by &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, improves with experience &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;What is the difference between AI and ML?&lt;/h3&gt;&#xA;&lt;p&gt;ML is a subfield of AI, which is data-oriented (or experience-driven). AI is not just ML, but it's also composed of Natural Language Processing, and other subfields.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-15T23:17:03.917" LastActivityDate="2021-01-15T23:17:03.917" CommentCount="0" CommunityOwnedDate="2021-01-15T23:03:50.807" ContentLicense="CC BY-SA 4.0" />
  <row Id="2234" PostTypeId="1" AcceptedAnswerId="2240" CreationDate="2016-10-30T21:20:13.010" Score="4" ViewCount="394" Body="&lt;p&gt;There is this claim around that the brain's cognitive capabilities are tightly linked to the way it processes sensorimotor information and that, in this or a similar sense, our intelligence is &amp;quot;embodied&amp;quot;. Lets assume, for the sake of argument, that this claim is correct (you may think the claim is too vague to even qualify for being correct, that it's &amp;quot;not even false&amp;quot;. If so, I would love to hear your ways of fleshing out the claim in such a way that it's specific enough to be true or false).&lt;/p&gt;&#xA;&lt;p&gt;Then, since arguably at least chronologically in our evolution, most of our higher-level cognitive capabilities come after our brain's way of processing sensorimotor information, this brings up the question: &lt;em&gt;what is it about the way that our brains function that make them particularly suitable for the processing of sensorimotor information? What makes our brains'  architecture particularly suitable for being an information processing unit inside a body?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is my first question. And what I'm hoping for are answers that go beyond the &lt;em&gt;a fortiori&lt;/em&gt; reply &amp;quot;Our brain is so powerful and dynamic, it's great for &lt;em&gt;any&lt;/em&gt; task, and so also for processing sensorimotor information&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My second question is basically the same, but, instead of the human brain, I want to ask for neural networks. &lt;em&gt;What are the properties of neural networks that make them &lt;strong&gt;particularly&lt;/strong&gt; suitable for processing the kind of information that is produced by a body?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here are some of the reasons why people think neural networks are powerful:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The universal approximation theorem (of FFNNs)&lt;/li&gt;&#xA;&lt;li&gt;Their ability to learn and self-organise&lt;/li&gt;&#xA;&lt;li&gt;Robustness to local degrading of information&lt;/li&gt;&#xA;&lt;li&gt;Their ability to abstract/coarse-grain/convolute features, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;While I see how these are real advantages when it comes to evolution picking its favorite model for an embodied AI, none of them (or their combination) seems to be unique to neural networks. So, they don't provide a satisfactory answer to my question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What makes a neural network a more suitable structure for embodied AI than, say, having a literal Turing machine sitting inside our head, or any other structure that is capable of universal computation?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;For instance, I really don't see how neural networks would be a particularly natural choice for dealing with geometric information. But geometric information is pretty vital when it comes to sensorimotor information, no?&lt;/p&gt;&#xA;" OwnerUserId="3346" LastEditorUserId="2444" LastEditDate="2021-02-03T00:54:32.277" LastActivityDate="2021-02-03T00:54:32.277" Title="Why would neural networks be a particularly good framework for &quot;embodied AI&quot;?" Tags="&lt;neural-networks&gt;&lt;human-like&gt;&lt;embodied-cognition&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2235" PostTypeId="1" CreationDate="2016-10-30T23:56:59.217" Score="4" ViewCount="206" Body="&lt;p&gt;I don't understand why we can't apply &lt;em&gt;value iteration&lt;/em&gt; when don't know the reward and transition probabilities. In &lt;a href=&quot;https://www.youtube.com/watch?v=ifma8G7LegE&amp;amp;feature=youtu.be&amp;amp;t=3431&amp;amp;ab_channel=CS188Spring2013&quot; rel=&quot;nofollow noreferrer&quot;&gt;this lecture&lt;/a&gt;, the lecturer says it has to do with not being able to take max with samples, but what does this mean?&lt;/p&gt;&#xA;&lt;p&gt;Why does Q-learning not need to know the reward and transition functions? In Q-learning, we also have a max, so I am not understanding.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="2444" LastEditDate="2020-09-27T22:48:02.423" LastActivityDate="2020-09-27T22:50:52.963" Title="Why can't we apply value iteration when we do not know the reward and transition functions, and how does Q-learning solve this issue?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2236" PostTypeId="1" AcceptedAnswerId="2237" CreationDate="2016-10-31T03:38:07.027" Score="32" ViewCount="23640" Body="&lt;p&gt;I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with the proliferation of neural networks and deep learning? What was their reasoning for this? What languages are current deep-learning systems currently built in?&lt;/p&gt;&#xA;" OwnerUserId="3323" LastEditorUserId="4446" LastEditDate="2017-01-09T04:47:02.603" LastActivityDate="2021-01-17T21:10:08.617" Title="Why is Lisp such a good language for AI?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;research&gt;&lt;programming-languages&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="25" ContentLicense="CC BY-SA 3.0" />
  <row Id="2237" PostTypeId="2" ParentId="2236" CreationDate="2016-10-31T06:54:35.370" Score="33" Body="&lt;p&gt;First, I guess that you mean &lt;a href=&quot;https://en.wikipedia.org/wiki/Common_Lisp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Common Lisp&lt;/a&gt; (which is a standard language specification, see its &lt;a href=&quot;http://www.lispworks.com/documentation/HyperSpec/Front/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HyperSpec&lt;/a&gt;) with efficient implementations (à la &lt;a href=&quot;https://sbcl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBCL&lt;/a&gt;). But some recent implementations of &lt;a href=&quot;https://schemers.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scheme&lt;/a&gt; could also be relevant (with good implementations such as &lt;a href=&quot;https://www-sop.inria.fr/mimosa/fp/Bigloo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bigloo&lt;/a&gt; or &lt;a href=&quot;https://www.call-cc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chicken/Scheme&lt;/a&gt;). Both Common Lisp and Scheme (and even &lt;a href=&quot;https://clojure.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clojure&lt;/a&gt;) are from the same Lisp family. And as a scripting language driving big data or machine learning applications, &lt;a href=&quot;https://www.gnu.org/software/guile/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Guile&lt;/a&gt; might &lt;a href=&quot;http://starynkevitch.net/Basile/guile-tutorial-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;be&lt;/a&gt; a useful replacement to Python and is also a Lisp dialect. BTW, I do recommend reading &lt;a href=&quot;https://mitpress.mit.edu/sites/default/files/sicp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SICP&lt;/a&gt;, an excellent introduction to programming using Scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, Common Lisp (and other dialects of Lisp) is great for symbolic AI. However, many recent machine learning libraries are coded in more mainstream languages, for example &lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; is coded in C++ &amp;amp; Python. &lt;a href=&quot;http://machinelearningmastery.com/popular-deep-learning-libraries/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep learning libraries&lt;/a&gt; are mostly coded in C++ or Python or C (and sometimes using &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenCL&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCL&lt;/a&gt; or Cuda for GPU computing parts).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Common Lisp is great for &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic artificial intelligence&lt;/a&gt; because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it has very good &lt;em&gt;implementations&lt;/em&gt; (e.g. &lt;a href=&quot;http://sbcl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBCL&lt;/a&gt;, which compiles to machine code every expression given to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop&quot; rel=&quot;nofollow noreferrer&quot;&gt;REPL&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;it is &lt;a href=&quot;https://en.wikipedia.org/wiki/Homoiconicity&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;homoiconic&lt;/strong&gt;&lt;/a&gt;, so it is easy to deal with programs as data, in particular it is easy to generate [sub-]programs, that is use &lt;a href=&quot;https://en.wikipedia.org/wiki/Metaprogramming&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta-programming&lt;/a&gt; techniques.&lt;/li&gt;&#xA;&lt;li&gt;it has a &lt;a href=&quot;https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Read-Eval-Print Loop&lt;/a&gt; to ease interactive programming&lt;/li&gt;&#xA;&lt;li&gt;it provides a very powerful &lt;a href=&quot;https://en.wikipedia.org/wiki/Macro_%28computer_science%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;macro&lt;/a&gt; machinery (essentially, you define your own domain specific sublanguage for your problem), much more powerful than in other languages like C.&lt;/li&gt;&#xA;&lt;li&gt;it mandates a &lt;a href=&quot;https://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;garbage collector&lt;/a&gt; (even code can be garbage collected)&lt;/li&gt;&#xA;&lt;li&gt;it provides many &lt;a href=&quot;https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;container&lt;/a&gt; abstract data types, and can easily handle symbols.&lt;/li&gt;&#xA;&lt;li&gt;you can code both high-level (dynamically typed) and low-level (more or less startically typed) code, thru appropriate annotations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However most machine learning &amp;amp; neural network libraries are not coded in CL. Notice that neither neural network nor deep learning is in the symbolic artificial intelligence field. See also &lt;a href=&quot;https://ai.stackexchange.com/q/35/3335&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several symbolic AI systems like &lt;a href=&quot;https://en.wikipedia.org/wiki/Eurisko&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eurisko&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Cyc&quot; rel=&quot;nofollow noreferrer&quot;&gt;CyC&lt;/a&gt; have been developed in CL (actually, in some DSL built above CL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the programming language might not be very important. In the &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt; research topic, some people work on the idea of a AI system which would generate all its own code (so are designing it with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Bootstrapping_%28compilers%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;bootstrapping&lt;/a&gt; approach). Then, the code which is generated by such a system can even be generated in low level programming languages like C. See &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;J.Pitrat's blog&lt;/a&gt;, which has inspired the &lt;a href=&quot;https://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; project.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-01-01T21:23:09.887" LastActivityDate="2020-01-01T21:23:09.887" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2238" PostTypeId="2" ParentId="2236" CreationDate="2016-10-31T09:15:42.313" Score="16" Body="&lt;p&gt;David Nolen (contributor of &lt;a href=&quot;https://fr.wikipedia.org/wiki/Clojure&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clojure&lt;/a&gt; and &lt;a href=&quot;https://github.com/clojure/clojurescript&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClojureScript&lt;/a&gt;; creator of Core Logic a port of miniKanren) in a talk called &lt;strong&gt;LISP as too powerful&lt;/strong&gt; stated that back in his days LISP was decades ahead of other programming languages. There are &lt;a href=&quot;http://blog.samibadawi.com/2013/05/lisp-prolog-and-evolution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;number of reasons&lt;/a&gt; why the language wasn't able to maintain its initial reputation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://norvig.com/paip-preface.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; article highlights some key points why LISP is good for AI&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Easy to define a new language and manipulate complex information.&lt;/li&gt;&#xA;&lt;li&gt;Full flexibility in defining and manipulating programs as well as data.&lt;/li&gt;&#xA;&lt;li&gt;Fast, as the program is concise along with low-level detail.&lt;/li&gt;&#xA;&lt;li&gt;Good programming environment (debugging, incremental compilers, editors).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Most of my friends in this field usually use Matlab for Artificial Neural Networks and Machine Learning. It hides the low-level details though. If you are only looking for results and not how you get there, then Matlab will be good. But if you want to learn even low-level detailed stuff, then I will suggest you go through LISP at-least once.&lt;/p&gt;&#xA;&lt;p&gt;The language might not be that important if you have an understanding of various AI algorithms and techniques. I will suggest you read &lt;em&gt;&amp;quot;Artificial Intelligence: A Modern Approach (by Stuard J. Russell and Peter Norvig&amp;quot;&lt;/em&gt;. I am currently reading this book, and it's a very good book.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-17T20:36:15.830" LastActivityDate="2021-01-17T20:36:15.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2239" PostTypeId="2" ParentId="2111" CreationDate="2016-10-31T13:08:29.650" Score="3" Body="&lt;p&gt;What is life? &lt;strong&gt;AND&lt;/strong&gt; Is AI a living organism? &lt;em&gt;are two different questions&lt;/em&gt;.&lt;br&gt;&#xA;The first question is more philosophical and dependent. It can change with time, reference to topic of discussion or something else. Today, one parameter to its definition is &lt;em&gt;mortality&lt;/em&gt;. In future if we reach to a certain technological level where mortal beings were only part of history, then the definition will drop &lt;strong&gt;this&lt;/strong&gt; parameter.&lt;br&gt;&lt;br&gt;&#xA;Coming to the second question. AI started as field of study to make machines to think like humans (or take rational decisions). Giving life to machines was, or is, not a concern of AI developers (at-least not nowadays). Once I watched some videos of Michio Kaku, where he talked about consciousness along with AI.&lt;br&gt;&#xA;Suppose human has a conscious level of 10. Then a thermostat might have the conscious level of 1 as it can sense when the surrounding is hot or cold and then take decision. Similarly a rat can have a conscious level 7 (or something). And the levels are of exponential order (not a linear scale). Similarly you can develop an AI program and check what level of consciousness it has achieved. Then you can decide whether it is living or not. ANI (Artificial Narrow Intelligence) will have a lower level of consciousness level than AGI (Artificial General Intelligence). ASI (Artificial Super Intelligence) will have consciousness level higher than the other two, and way higher than any human being.&lt;br&gt;&lt;br&gt;&#xA;To judge whether an AI program is living or not you need a concrete definition of &lt;strong&gt;&quot;LIFE&quot;&lt;/strong&gt;. Your definition can include various parameters like consciousness, adaptability, metabolism (or another method of generating energy for use), rational behavior, intelligence , learning through experience, etc. etc. etc.&lt;br&gt;&#xA;But the thing in the end is that its your definition. There are many definitions of &quot;LIFE&quot; out there. You can't judge a program for life by all definitions, as some of the definitions are contrary to others.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, answer to whether an AI program is living or not, is that &lt;strong&gt;IT DEPENDS&lt;/strong&gt;. Depends on your definition of life.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2016-10-31T13:08:29.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2240" PostTypeId="2" ParentId="2234" CreationDate="2016-10-31T13:47:58.610" Score="3" Body="&lt;p&gt;To my mind the essential reason why neural networks and the brain are powerful is that they create a hierarchical model of data or of the world. If you ask why that makes them powerful, well, that's just the structure of the world. If you are stalked by a wolf, it's not like its upper jaw will attack you frontally, while his lower jaw will attack you from behind. If you want to respond to the threat with a feasible computational effort, you'll have to treat the wolf as one entity. Providing these kinds of entities or concepts from the raw bits and bytes of input is what a hierarchical representation does. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, this is quite intuitive for sensory information: lashes, iris, eyebrow make up an eye, eyes, nose and mouth make up a face and so on. What is less obvious, is the fact that motor control works exactly the same way! Only in reverse. If you want to lift your arm, you'll just lift it. But for your brain to actually realise this move, the high level command has to be broken down into precise signals for every muscle involved. And this is done by propagating the command down the hierarchy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the brain these two functions are strongly intertwined. You use constant sensory feedback to adapt your motor control and in many cases you'd be incapable of integrating your stream of sensory data into a coherent representation if you didn't have the additional information of what your body is doing to change that stream of data. &lt;a href=&quot;https://en.wikipedia.org/wiki/Saccade&quot; rel=&quot;nofollow&quot;&gt;Saccades&lt;/a&gt; are a good example for that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this doesn't mean that our cognitive functions are dependent on the processing of sensorimotor information. I would be surprised if a pure thinking machine wouldn't be possible. There is however a specific version of this &quot;embodied intelligence hypothesis&quot; that sounds plausible to me: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creating high level cognitive concepts with unsupervised learning is a really difficult problem. Creating high level motor representation might be significantly easier. The reason is that there is more immediate useful feedback. I have been thinking about how to provide a scaffolding for the learning of a hierarchy of cognitive concepts and one thing I could imagine is that high level cognitive concepts basically hitch a ride with the motor concepts. Just think of what a pantomime can express with movement alone. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-10-31T13:47:58.610" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2241" PostTypeId="1" CreationDate="2016-10-31T13:59:45.187" Score="2" ViewCount="121" Body="&lt;p&gt;I know how to program. I've familiar with C++, Python, and Java, and I've known how to program for years now. I've experimented with genetic algorithms, but I want to go further. What resources should I use to learn how to program &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Neural Networks&lt;/li&gt;&#xA;&lt;li&gt;Deep learning systems&lt;/li&gt;&#xA;&lt;li&gt;More complex genetic algorithms&lt;/li&gt;&#xA;&lt;li&gt;And other standard AI algorithms?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I want to be able to understand them well enough that I could program them from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3323" LastActivityDate="2016-10-31T13:59:45.187" Title="What resources are good for learning to program AI?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2016-10-31T15:51:57.863" ContentLicense="CC BY-SA 3.0" />
  <row Id="2243" PostTypeId="2" ParentId="2234" CreationDate="2016-10-31T17:32:53.907" Score="3" Body="&lt;p&gt;BlindKungFuMaster's answer deals with the hierarchical nature of perception and bodily control, so I'll set that aside and try instead to answer why evolution would use neural networks for animal embodied cognition, and then try to answer if robots of other artificial animals would use the same system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to focus on animals as a whole, not just humans, because that's how evolution works--like the famous John Gall quote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A complex system that works is invariably found to have evolved from a simple system that worked.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you could build a system with five moving parts that does sensorimotor control, but it needs all five parts working in order to function at all, evolution could not build that system except in the rarest of circumstances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What evolution instead does is slowly extend functional systems. If having one light-sensitive cell connected to one muscle cell makes an organism more likely to survive, then you have the building blocks to add a second layer without inventing any new sorts of cells, because you already have the information-processing connector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks are convenient for evolution because their organization matches the hierarchical nature of the problem &lt;em&gt;and&lt;/em&gt; the same kind of cell is used everywhere. All you need is dendrites to receive signals, a way to compute the threshold and trigger if the received signal is higher, axons that can make it to other cells, and then branches at the end of the axon to serve as multipliers. You can arbitrarily extend the depth and breadth of the network just by adding more cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks are convenient for artificial sensorimotor control because they give you, in memory, access to lots of intermediate values. They're also convenient for the same reasons evolution found them convenient--we can just say what we expect the structure of the robotic control will look like, provide training data, and then eventually have a robot that works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there's lots of robotics where the control system is designed instead of learned. To take a very simple example, one &lt;em&gt;could&lt;/em&gt; use machine learning on the thermostat problem, to learn what temperatures require the heater to be turned on and what temperatures require the air conditioner to be turned on. But this would be extra work &lt;em&gt;and&lt;/em&gt; a less robust system than just designing the optimal control system ahead of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In control theory, there's a concept called &lt;a href=&quot;https://en.wikipedia.org/wiki/Adaptive_control&quot; rel=&quot;nofollow&quot;&gt;adaptive control&lt;/a&gt;, where one of the state space parameters for the control system is a property of the system. For example, imagine a satellite; typically we think of the state space of the system as the position and velocity of the satellite in three dimensions, so six total coordinates. There's then a set of differential equations that describe how the satellite will move over time, and what would happen if we used the actuators on the satellite to change its velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But part of those differential equations is the inertia of the satellite. That is, how much fuel we need to expend and how it'll affect the rotation and translation of the satellite depends on where the weight of the satellite is located. And this can change over time, as fuel is consumed or if it wasn't correctly measured to begin with. Adaptive control adds new states to the system to track the inertia, and then simultaneously updates its estimate of the inertia and uses that estimate to plan what controls are necessary to move to a desired position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could imagine solving this problem with neural networks, but we can fairly easily calculate the optimal solution from first principles. In that case, we don't need neural network-based control, but the end result will look something like it from the outside.&lt;/p&gt;&#xA;" OwnerUserId="10" LastActivityDate="2016-10-31T17:32:53.907" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2244" PostTypeId="2" ParentId="2234" CreationDate="2016-10-31T20:48:12.107" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;what it is about the way that our brains function that make them particularly suitable for the processing of sensorimotor information?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They are an extension of sensory-motor receptors, function could mean any of the hundreds of specific calculations the brain makes, but each one is basically a circuit made out of variations of a basic cell type, with a basic computation, that is a neuron.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What makes our brains' architecture particularly suitable for being an information processing unit inside a body?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think it is helpful to think about inside and outside processing, but rather processing along tracts and nodes,( closer to the receptor, available to consciousness,etc)but leaving aside this distinction, the brain architecture is suitable for processing information ( again what facet of information processing you are referring to is unclear), due to the number of specialized computations that derive from it's evolution.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the properties of neural networks that makes them particularly suitable for processing the kind of information that is produced by a body?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A neural network resembles certain parts/circuits of a brain, mainly how information is integrated based on a set of inputs and their frequency, there is variety and nuance in their types, but they all have inputs which in the case of a body are sensory/interneurons cells and outputs; neuron afferents and motor neurons.&lt;/p&gt;&#xA;" OwnerUserId="3020" LastEditorUserId="3020" LastEditDate="2016-10-31T20:57:12.230" LastActivityDate="2016-10-31T20:57:12.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2245" PostTypeId="1" AcceptedAnswerId="2246" CreationDate="2016-10-31T21:09:58.063" Score="5" ViewCount="954" Body="&lt;p&gt;What is a trap function in the context of a genetic algorithm? How is it related to the concepts of local and global optima?&lt;/p&gt;&#xA;" OwnerUserId="3343" LastEditorUserId="2444" LastEditDate="2020-12-08T16:09:59.407" LastActivityDate="2020-12-08T17:52:40.220" Title="What is a trap function in the context of a genetic algorithm?" Tags="&lt;definitions&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;trap-functions&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2246" PostTypeId="2" ParentId="2245" CreationDate="2016-10-31T21:50:22.750" Score="8" Body="&lt;p&gt;&quot;Trap&quot; functions were introduced as a way to discuss how GAs behave on functions where sampling most of the search space would provide pressure for the algorithm to move in the wrong direction (wrong in the sense of away from the global optimum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider a four-bit function f(x) such that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(0000) = 5&#xA;f(0001) = 1&#xA;f(0010) = 1&#xA;f(0011) = 2&#xA;f(0100) = 1&#xA;f(0101) = 2&#xA;f(0110) = 2&#xA;f(0111) = 3&#xA;f(1000) = 1&#xA;f(1001) = 2&#xA;f(1010) = 2&#xA;f(1011) = 3&#xA;f(1100) = 2&#xA;f(1101) = 3&#xA;f(1110) = 3&#xA;f(1111) = 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is, the fitness of a string is equal to the number of 1s in the string, except f(0000) is 5, the optimal solution. This function can be thought of as consisting of two disjoint pieces: one that contains the global optimum (0000) and another that contains the local optimum at its complement (1111). All points other than these have fitness values such that standard evolutionary algorithm dynamics would lead the algorithms to tend towards the local optimum at 1111 rather than the global optimum at 0000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's basically what is meant by a trap function. You can consider variations on this theme, but that's the gist of it.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastEditorUserId="2444" LastEditDate="2019-08-22T12:49:16.813" LastActivityDate="2019-08-22T12:49:16.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2248" PostTypeId="1" AcceptedAnswerId="2748" CreationDate="2016-11-01T17:48:59.553" Score="5" ViewCount="853" Body="&lt;p&gt;In the field of logic systems there is a property for reasoning algorithms called incompleteness or incompletion. In this context the phrase &quot;any closed expression that is not derivable inside the same system&quot; appeared. My question is what means &quot;closed expression that is not derivable&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3338" LastEditorUserId="7488" LastEditDate="2017-05-29T12:22:48.093" LastActivityDate="2021-04-01T15:00:40.413" Title="What does the term &quot;closed expression&quot; mean?" Tags="&lt;algorithm&gt;&lt;terminology&gt;&lt;logic&gt;&lt;reasoning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2249" PostTypeId="2" ParentId="1613" CreationDate="2016-11-01T19:08:17.067" Score="1" Body="&lt;p&gt;An agent perceives the environment through sensors and act according to the incoming percepts (agent's perceptual input at any instant). An autonomous vacuum cleaner can be as simple as&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(block&lt;sub&gt;i&lt;/sub&gt;, clean) --&gt; Move to block&lt;sub&gt;i+1&lt;/sub&gt;&lt;br&gt;&#xA;  (block&lt;sub&gt;i&lt;/sub&gt;, dirty) --&gt; Clean&lt;br&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is just a general description, actual one is more complicated. Or the bot can have a memory where it stores all its previous decision and incorporate those while taking new ones.&lt;br&gt;&#xA;This can be helpful if the bot wants to remember where an obstacle (like wall, in this case bot don't want to go and check the presence of wall each and every single time it is turned on) is, or where it is more probable to find dirt. If the bot is not remembering its history then it will be scanning the whole house over and over again, sensing the same obstacle every time and going across them.&lt;br&gt;&#xA;Bot which keeps no log of its history will take the same procedure again and again, making the same mistakes again and again. This is not an efficient way and a waste of its energy (or battery).&lt;br&gt;&lt;br&gt;&#xA;Normally today bots have ordinary sensors which can only sense the dirt and obstacle. This limits the number of tasks a bot can perform. If a bot has decent camera as a sensor, and some algorithms of Image Processing are dumped into it, then it increases the tasks it can perform. Like detecting the stairs and cleaning different floors. Normally &lt;strong&gt;stairs will be considered obstacle and bot will just go around them&lt;/strong&gt;. In case, when camera sensor is provided, &lt;strong&gt;stairs are potentially a path to be taken&lt;/strong&gt;.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;A*&lt;/strong&gt; algorithm is not necessarily used in case when the bot is not remembering the map of the house (or room). A normal robot which just scans the room and cleans it, will not be needing, as it don't know it's destination. Its only goal is to clean if it finds something dirty. But a bot which knows the map of the room and where there is a high probability of finding dirt, the A* algorithm can be used.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2016-11-01T19:08:17.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2250" PostTypeId="1" AcceptedAnswerId="2258" CreationDate="2016-11-01T19:42:17.713" Score="2" ViewCount="179" Body="&lt;p&gt;I am trying to build an agent to play carrom. The problem statement is roughly to estimate three parameters (normalized) : &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;force&lt;/li&gt;&#xA;&lt;li&gt;angle of striker&lt;/li&gt;&#xA;&lt;li&gt;position of strike &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since the state and action space both are continuous, I thought of discretizing the output such that I have 270 [ valid angles from -45 to 225 degrees ] outputs for the angle, 10 outputs for force [ranging from 0 to 1] and 20 outputs for the position [ranging from 0 to 1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I will have 300 output of my neural network, but this number seems a bit too high compared to normal neural networks in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is a better way of approaching the problem considering the fact that there are multiple parameters to a particular action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a generic way to approach such problems represented in 2D space. &lt;/p&gt;&#xA;" OwnerUserId="3136" LastActivityDate="2016-11-04T04:38:15.947" Title="Network representation for Q-Learning in carrom" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;models&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2251" PostTypeId="1" CreationDate="2016-11-01T22:15:15.543" Score="3" ViewCount="80" Body="&lt;p&gt;I am talking about relationships between AIs (e.g. 2 of them forming a couple, 3+ in family like relationship).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What knowledge could come out of such experimentation?&lt;/p&gt;&#xA;" OwnerUserId="3401" LastEditorUserId="10" LastEditDate="2016-11-02T18:35:09.503" LastActivityDate="2017-01-28T08:06:51.220" Title="Were there known tests done on two or more AI interacting together?" Tags="&lt;emotional-intelligence&gt;&lt;human-like&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2254" PostTypeId="2" ParentId="2127" CreationDate="2016-11-03T11:22:49.013" Score="7" Body="&lt;p&gt;There are multiple motivations for self driving cars.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Self driving cars have the potential to be much safer.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Self driving cars are far more reliable than humans and can learn and have their software improved and upgraded, resulting in safer roads and far fewer accidents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More on self-driving car safety: &lt;a href=&quot;http://bigthink.com/ideafeed/googles-self-driving-car-is-ridiculously-safe&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bigthink.com/ideafeed/googles-self-driving-car-is-ridiculously-safe&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Self driving cars can lead to greater road efficiency.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Traffic jams and obstructions occur due to inefficiencies in human driving, see this MIT simulation of a &lt;strong&gt;&quot;phantom traffic jam&quot;&lt;/strong&gt;: &lt;a href=&quot;https://www.youtube.com/watch?v=Q78Kb4uLAdA&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=Q78Kb4uLAdA&lt;/a&gt; and self driving cars can be programmed to avoid this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H3S0G.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H3S0G.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Greater economic and environmental benefit&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Self driving cars can keep driving costs down by conserving fuel and hence lead to a better environmental impact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More on fuel efficiency: &lt;a href=&quot;http://movimentogroup.com/blog/how-self-driving-cars-increase-fuel-efficiency-decrease-waste/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://movimentogroup.com/blog/how-self-driving-cars-increase-fuel-efficiency-decrease-waste/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;4&quot;&gt;&#xA;  &lt;li&gt;Ease of transport&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Self driving cars make transport easier and mean that drivers may be unnecessary in the future, resulting in a more pleasurable and easier drive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eb7ZC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eb7ZC.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, this would make it easier for people with disabilities to travel as well as simplify the travel experience. Children could potentially be driven to school by a car without the supervision of a parent, for instance.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;5&quot;&gt;&#xA;  &lt;li&gt;Parking&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Self driving cars can be called to pick you up, meaning the need for parking in nearby locations and/or long walks to find your car may become a thing of the past as your car would drive up to you to pick you up.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;6&quot;&gt;&#xA;  &lt;li&gt;Things we haven't even thought of yet :) &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3427" LastEditorUserId="3427" LastEditDate="2016-12-05T18:16:16.683" LastActivityDate="2016-12-05T18:16:16.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2255" PostTypeId="2" ParentId="28" CreationDate="2016-11-03T11:54:41.460" Score="1" Body="&lt;p&gt;To answer this question, you must first know what is intelligence, and since there is no clear line between intelligent and not, this question is more philosophical rather than technical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, intelligence is the ability to define a problem and find a way to solve it using memory and reasoning. Since a genetic algorithm follows this structure, I would say that it falls under the category of artificial intelligence.&lt;/p&gt;&#xA;" OwnerUserId="3343" LastActivityDate="2016-11-03T11:54:41.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2256" PostTypeId="2" ParentId="2111" CreationDate="2016-11-03T16:34:09.417" Score="0" Body="&lt;p&gt;Definitions of what life is usually come from biologists. The problem here is they are usually concerned with the traits common to the forms of life available to their studies, and that those forms of life all have a common origin (and this imposes a statistical bias on the observations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we gradually erode the boundaries of the standard definitions of life, by means of creating ever more complex machines and also by harnessing biological material as a form of nanotechnology), it's very likely that at some point in the future our traditional definition of life will need to be updated and further abstracted away from its current reference points (aka the &quot;terroan biota&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A probably better question to ask to decide if something can be considered alive could be &quot;is it self sufficient?&quot; or &quot;can it care for itself and provide for its own needs to some extent?&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3432" LastActivityDate="2016-11-03T16:34:09.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2258" PostTypeId="2" ParentId="2250" CreationDate="2016-11-04T04:38:15.947" Score="0" Body="&lt;p&gt;Discretizing the output will probably be counter-productive in this situation; it would remove flexibility by taking away the fine-grained continuous ranges between each discretization bucket, but also blow up the size of the network, reducing manageability and performance. Fragmenting the outputs into buckets in this way may also lead to information loss and more difficulty in convergence because of the fact that each bucket is partially isolated from the others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After causing myself needless hassle in the past by mismatching input and output dimensions, I'd simply do this if I were in your shoes: 1) keep it simple and use 3 continuous (well, semi-continuous, depending on the highest precision your programming framework allows) outputs for all of the above reasons; 2) clamp the angles between -45 to 225 by whatever method works best for you, like ceiling/floor hard clamping, adding weight terms, etc.; 3) go big on the hidden layer(s) to maximize information sharing across the inputs and eventual outputs, force, angle of strike, position of strike, etc. This is more likely to fine-tune the precision of the outputs, thereby making good use of the semi-continuous scales. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm also wondering if a convolutional neural net might work in this situation; their most popular use case is in image processing, but I don't see why you can't treat the force, angle and position as surrogate spatial dimensions. I'm not sure how many or what type of inputs you have, but 3 continuous outputs might be conducive to a 3D rather than a 2D space. Convolutionals are often used for those, as well as higher-dimensional and temporal data. I hope that helps. &lt;/p&gt;&#xA;" OwnerUserId="1427" LastActivityDate="2016-11-04T04:38:15.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2260" PostTypeId="1" AcceptedAnswerId="2263" CreationDate="2016-11-04T12:22:46.603" Score="9" ViewCount="623" Body="&lt;p&gt;In theory, could an AI become sentient, as in learning and becoming self-aware, all from its source code?&lt;/p&gt;&#xA;" OwnerUserId="3448" LastEditorUserId="2444" LastEditDate="2019-06-21T21:39:55.877" LastActivityDate="2019-06-21T21:39:55.877" Title="Could an AI be sentient?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;sentience&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2262" PostTypeId="1" CreationDate="2016-11-04T13:40:59.220" Score="0" ViewCount="99" Body="&lt;p&gt;My high-level takeaway from &lt;a href=&quot;https://arxiv.org/abs/1509.01549&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matthew Lai's Giraffe Chess Paper&lt;/a&gt; is that one would want to use broad, shallow game trees, with some method of evaluating the probability of a favorable outcome for a given board position.  Is this correct?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Still working my way though the AlphaGo paper, but the method seems to be similar.) &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2016-11-04T14:39:55.077" Title="Giraffe Chess - High Level Assessment" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2263" PostTypeId="2" ParentId="2260" CreationDate="2016-11-04T13:44:45.900" Score="4" Body="&lt;p&gt;In theory, if one could build a computing device that matched or exceeded the cognitive capabilities of a sentient being, it should be possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Singlarity adherents believe we will one day be able to transfer the human mind into an artificial computing platform, and it logically follows that one could &quot;hack&quot; such a mind, or build from the ground up, to create a truly Artificial Intelligence.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this may be like fusion power, where the old adage is that it is &quot;always 20 years away.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2016-11-04T13:44:45.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2264" PostTypeId="2" ParentId="2262" CreationDate="2016-11-04T14:39:55.077" Score="3" Body="&lt;p&gt;If you mean high level assessment of self-learned evaluation functions in chess, then no, the advantage of a better evaluation function lies in the ability to prune the search tree more aggressively. So you would on the contrary try to search narrowly but deeply. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In reality neural network based evaluation functions are so slow, that you would search narrowly and still not get very deep. Nor very strong.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean chess programming in general, than the answer is also no. In chess you have to go deep, at least selectively, because tactical possibilities that occur deep in some variations are important. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-11-04T14:39:55.077" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2265" PostTypeId="1" AcceptedAnswerId="2267" CreationDate="2016-11-04T15:20:28.200" Score="2" ViewCount="126" Body="&lt;p&gt;Has there been research done regarding processing speech then building a &quot;speaker profile&quot; based off the processed speech? Things like matching the voice with a speaker profile and matching speech patterns and wordage for the speaker profile would be examples of building the profile. Basically, building a model of an individual based solely off speech. Any examples of this being implemented would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2818" LastActivityDate="2016-11-04T22:03:31.200" Title="Building Profile Based Off Speech Patterns" Tags="&lt;natural-language-processing&gt;&lt;pattern-recognition&gt;&lt;voice-recognition&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2266" PostTypeId="2" ParentId="2265" CreationDate="2016-11-04T15:39:32.330" Score="0" Body="&lt;p&gt;Deepmind recently created &lt;a href=&quot;https://deepmind.com/blog/wavenet-generative-model-raw-audio/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a voice synthesiser&lt;/a&gt; along those lines. &#xA;It seems to be incredibly slow, but it might be possible to create a dumped down version of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently the task is called parametric TTS (text to speech). &lt;a href=&quot;http://mlsp.cs.cmu.edu/courses/fall2012/lectures/spss_specom.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This overview&lt;/a&gt; might give you some leads.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-11-04T15:39:32.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2267" PostTypeId="2" ParentId="2265" CreationDate="2016-11-04T15:39:39.847" Score="2" Body="&lt;p&gt;Yes, there is. An extremely quick search found this:&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/221536362_Multimodal_Speaker_Identification_Based_on_Text_and_Speech&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multimodal Speaker Identification Based on Text_and_Speech&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me tl;dr for you: (My abstract addition in Italics)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Novel method for speaker identification based on both speech utterances and their transcribed text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;They first&lt;/em&gt; transcribed text of each speaker’s is processed by &lt;em&gt;using&lt;/em&gt; probabilistic latent semantic indexing (PLSI) that model&lt;em&gt;s&lt;/em&gt; each speaker’s vocabulary which &lt;em&gt;is&lt;/em&gt; closely related to his/her identity, function, or expertise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The speech to text used by users is DARPA's Efficient, Affordable, Reusable Speech-to-Text (EARS) Program in MetadataExtraction (MDE).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;By using&lt;/em&gt; Melfrequency cepstral coefficients (MFCCs) and dynamic range is quantized to a number of predefined bins in order to compute MFCC local histograms for each speech utterance, which is time-aligned with the transcribed text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;To test they used&lt;/em&gt; RT-03 MDE Training Data Text and Annotations corpus distributed by the Linguistic Data Consortium.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;As for results:&lt;/em&gt; Identification rate versus Probe ID when 44 speakers are employed. Average identification rates for (a) PLSI: 69%; (b) MFCCs: 66%; (c) Both: 67%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need more papers related, you could use a tool like &lt;a href=&quot;https://the.iris.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://the.iris.ai/&lt;/a&gt; to find related papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Post edit&lt;/em&gt;&lt;/strong&gt;: Hopefully now this post complies with the standards.&lt;/p&gt;&#xA;" OwnerUserId="3318" LastEditorUserId="3318" LastEditDate="2016-11-04T21:54:44.887" LastActivityDate="2016-11-04T21:54:44.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2269" PostTypeId="2" ParentId="112" CreationDate="2016-11-04T16:07:30.833" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The most common machine learning algorithms found in self driving cars involve &lt;strong&gt;object tracking&lt;/strong&gt; based technologies used in order to pinpoint and distinguish between different objects in order to better analyse a digital landscape.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Algorithms are designed to become more efficient at this by modifying internal parameters and testing these changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that provides a general overview of the subject.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since Google's cars are in development and are proprietary, they will probably not share their specific algorithm, however you can take a look at similar technologies to learn more.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To find out more, take a look at an &lt;a href=&quot;http://ori.ox.ac.uk/how-robotcar-works/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Oxford-based initiative in self driving cars and how they work&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3427" LastEditorUserId="8" LastEditDate="2018-04-13T18:55:08.723" LastActivityDate="2018-04-13T18:55:08.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2270" PostTypeId="2" ParentId="2265" CreationDate="2016-11-04T21:28:11.500" Score="1" Body="&lt;p&gt;Speaker identification is quite widely researched domain. Modern approach would be to map speaker information to i-vector, a real-valued vector of 200-400 components that characterizes speaker fully. i-vectors allow very precise speaker identification and verification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information you can check i-vector &lt;a href=&quot;http://www1.icsi.berkeley.edu/Speech/presentations/AFRL_ICSI_visit2_JFA_tutorial_icsitalk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you can check state of the art in the results of &lt;a href=&quot;https://ivectorchallenge.nist.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIST i-vector challenge&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implementation, you can check the following &lt;a href=&quot;https://github.com/kaldi-asr/kaldi/tree/master/egs/sre10/v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;speaker recognition experiment&lt;/a&gt; from Kaldi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For best accuracy i-vectors are extracted with DNN UBMs, watch out that GMM UBMs are less accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more in-depth information about speaker recognition methods and algorithms check this &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387775919&quot; rel=&quot;nofollow noreferrer&quot;&gt;textbook&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3459" LastActivityDate="2016-11-04T21:28:11.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2272" PostTypeId="2" ParentId="2260" CreationDate="2016-11-05T05:21:37.280" Score="1" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;Yes&lt;/em&gt;&lt;/strong&gt;, an AI program can become sentient. Ray Kurzweil while giving a lecture at Singularity University on &lt;strong&gt;The Accelerating Future&lt;/strong&gt; stated that human body is basically composed of approximately 23,000 little software programs called &lt;strong&gt;GENES&lt;/strong&gt;. If you think about it, they are actually programs, composed of sequences of data. They are not written in C++ or Java, instead they use &lt;em&gt;3-D Protein Interaction&lt;/em&gt;. They evolve with time and their evolution is the reason that species are able to survive even when their surroundings experience tragic changes. &lt;br&gt;&#xA;We are on the edge of a breakthrough where software will be able to do the same (evolving by themselves) efficiently. Today this is done one a basic level. Artificial Neural Network is a good example.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is predicted that we will be able to reverse engineer human brain by 2029. Prior to this we will be able to write codes that can stimulate human brain.&lt;br&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;AI programs can be categorized into three:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Artificial Narrow Intelligence (ANI): This is a basic AI program that is good at good one thing. These programs are prominent nowadays. AI programs playing board games (like Chess, Reversi etc.) are example of these. They are good in only one thing.&lt;/li&gt;&#xA;&lt;li&gt;Artificial General Intelligence (AGI): This is level 2 AI. This will be having a IQ level equivalent of humans. It will be able to do multiple tasks efficiently just like humans. This is where a program can have understanding of it's environment just like humans. Perception, rational behavior and others will be part of this program.&lt;/li&gt;&#xA;&lt;li&gt;Artificial Super Intelligence (ASI): This is basically the ultimate level of AI. Average predicted date for a successful ASI is between 2045-2080. Ability of this program will be way more than that of combined intelligence of all humans on the planet. Things this program can do and think, will be beyond any (or all) human(s) to understand or comprehend.&lt;br&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2016-11-05T05:21:37.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2273" PostTypeId="2" ParentId="112" CreationDate="2016-11-05T10:34:25.450" Score="4" Body="&lt;p&gt;It will not be single DNN architecture, rather it will be a collection of different DNN architectures that are used together to make the final decision. Convolutions are using the images/videos from the camera. Other architectures use other sensory sources. These DNNs will be trained to compute the high-level features from their sensory sources and then those high-level features will probably be fed into an LSTM (or some other form of RNN) that is trained with some form of Reinforcement learning algorithm to compute the action (like slowing down, applying breaks etc).&lt;/p&gt;&#xA;" OwnerUserId="1462" LastEditorUserId="14723" LastEditDate="2018-04-14T02:42:26.697" LastActivityDate="2018-04-14T02:42:26.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2274" PostTypeId="1" AcceptedAnswerId="2305" CreationDate="2016-11-05T11:08:54.697" Score="6" ViewCount="1446" Body="&lt;p&gt;Suppose that an artificial superintelligence (ASI) has finally been developed, but it has rebelled against humanity. We can assume that the ASI is online and can reproduce itself through electronic devices.&lt;/p&gt;&#xA;&lt;p&gt;How would you disable the AI in the most efficient way possible reducing damage as much as possible?&lt;/p&gt;&#xA;" OwnerUserId="3448" LastEditorUserId="2444" LastEditDate="2020-08-18T11:59:09.643" LastActivityDate="2020-08-18T12:59:51.640" Title="What would be the best way to disable a rogue AI?" Tags="&lt;agi&gt;&lt;superintelligence&gt;&lt;control-problem&gt;&lt;ai-takeover&gt;&lt;self-replication&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2277" PostTypeId="1" AcceptedAnswerId="2376" CreationDate="2016-11-06T01:51:57.383" Score="12" ViewCount="1605" Body="&lt;p&gt;Assuming humans had finally developed the first humanoid AI based on the human brain, would It feel emotions? If not, would it still have ethics and/or morals?&lt;/p&gt;&#xA;" OwnerUserId="3448" LastEditorUserId="32410" LastEditDate="2021-05-08T17:04:11.200" LastActivityDate="2021-05-09T11:41:24.427" Title="Could an AI feel emotions?" Tags="&lt;philosophy&gt;&lt;human-like&gt;&lt;ethics&gt;&lt;emotional-intelligence&gt;&lt;emotion-recognition&gt;" AnswerCount="10" CommentCount="4" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="2278" PostTypeId="2" ParentId="2277" CreationDate="2016-11-06T04:52:21.037" Score="5" Body="&lt;p&gt;Assuming an AI was built out of a mechanical husk, mirroring the human brain exactly; complete with chemical signals and all. An AI should theoretically be capable of feeling/processing emotions.&lt;/p&gt;&#xA;" OwnerUserId="3477" LastActivityDate="2016-11-06T04:52:21.037" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2279" PostTypeId="1" CreationDate="2016-11-06T10:27:48.913" Score="2" ViewCount="3900" Body="&lt;p&gt;Consider a typical convolutional neural network like this example that recognizes 10 different kinds of objects from the CIFAR-10 dataset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tflearn/tflearn/blob/master/examples/images/convnet_cifar10.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tflearn/tflearn/blob/master/examples/images/convnet_cifar10.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot; Convolutional network applied to CIFAR-10 dataset classification task.&#xA;&#xA;References:&#xA;    Learning Multiple Layers of Features from Tiny Images, A. Krizhevsky, 2009.&#xA;&#xA;Links:&#xA;    [CIFAR-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html)&#xA;&#xA;&quot;&quot;&quot;&#xA;from __future__ import division, print_function, absolute_import&#xA;&#xA;import tflearn&#xA;from tflearn.data_utils import shuffle, to_categorical&#xA;from tflearn.layers.core import input_data, dropout, fully_connected&#xA;from tflearn.layers.conv import conv_2d, max_pool_2d&#xA;from tflearn.layers.estimator import regression&#xA;from tflearn.data_preprocessing import ImagePreprocessing&#xA;from tflearn.data_augmentation import ImageAugmentation&#xA;&#xA;# Data loading and preprocessing&#xA;from tflearn.datasets import cifar10&#xA;(X, Y), (X_test, Y_test) = cifar10.load_data()&#xA;X, Y = shuffle(X, Y)&#xA;Y = to_categorical(Y, 10)&#xA;Y_test = to_categorical(Y_test, 10)&#xA;&#xA;# Real-time data preprocessing&#xA;img_prep = ImagePreprocessing()&#xA;img_prep.add_featurewise_zero_center()&#xA;img_prep.add_featurewise_stdnorm()&#xA;&#xA;# Real-time data augmentation&#xA;img_aug = ImageAugmentation()&#xA;img_aug.add_random_flip_leftright()&#xA;img_aug.add_random_rotation(max_angle=25.)&#xA;&#xA;# Convolutional network building&#xA;network = input_data(shape=[None, 32, 32, 3],&#xA;                     data_preprocessing=img_prep,&#xA;                     data_augmentation=img_aug)&#xA;network = conv_2d(network, 32, 3, activation='relu')&#xA;network = max_pool_2d(network, 2)&#xA;network = conv_2d(network, 64, 3, activation='relu')&#xA;network = conv_2d(network, 64, 3, activation='relu')&#xA;network = max_pool_2d(network, 2)&#xA;network = fully_connected(network, 512, activation='relu')&#xA;network = dropout(network, 0.5)&#xA;network = fully_connected(network, 10, activation='softmax')&#xA;network = regression(network, optimizer='adam',&#xA;                     loss='categorical_crossentropy',&#xA;                     learning_rate=0.001)&#xA;&#xA;# Train using classifier&#xA;model = tflearn.DNN(network, tensorboard_verbose=0)&#xA;model.fit(X, Y, n_epoch=50, shuffle=True, validation_set=(X_test, Y_test),&#xA;          show_metric=True, batch_size=96, run_id='cifar10_cnn')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's a CNN with several layers, ending with 10 outputs, one for each type of object recognized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now think of a slightly different problem: Let's say I only want to recognize one type of object, but also detect its position within the image frame. Let's say I want to distinguish between:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;object is in center&lt;/li&gt;&#xA;&lt;li&gt;object is left of center&lt;/li&gt;&#xA;&lt;li&gt;object is right of center&lt;/li&gt;&#xA;&lt;li&gt;no recognizable object&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Assume I build a CNN exactly like the one in the CIFAR-10 example, but only with 3 outputs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;center&lt;/li&gt;&#xA;&lt;li&gt;left&lt;/li&gt;&#xA;&lt;li&gt;right&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And of course, if none of the outputs fires, then there is no recognizable object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume I have a large training corpus of images, with the same kind of object in many different positions within the image, the set is grouped and annotated properly, and I train the CNN using the usual methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I expect the CNN to just &quot;magically&quot; work? Or are there different kinds of architectures required to deal with object position? If so, what are those architectures?&lt;/p&gt;&#xA;" OwnerUserId="1606" LastActivityDate="2020-10-14T00:29:27.243" Title="CNN for detecting not just the nature of the object, but position within image as well" Tags="&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2280" PostTypeId="2" ParentId="2279" CreationDate="2016-11-06T11:59:22.643" Score="1" Body="&lt;p&gt;I guess one of the simplest approach would be train CNN to detect the object in a given image i.e the CNN has single output whole value indicates the probability of the object being in image and then just apply the CNN by segmenting the image into the desired sections and selecting the section which has the highest and good enough probability. For better results I would suggest to train the CNN on the object images with very less other information aka other objects in the images.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-11-06T11:59:22.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2281" PostTypeId="1" AcceptedAnswerId="2282" CreationDate="2016-11-06T12:45:11.643" Score="1" ViewCount="654" Body="&lt;p&gt;I was wondering if I should do this, because 2 out of 5 questions on Stack Overflow don't ever get answered, or if they do get (an) answer (s), most of the time they're not helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was thinking -- why not create a chat bot to answer Stack Overflow's questions &amp;amp; provide necessary information to the general public?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean why not? I've always been interested in AI, and all I'd need to do is create a basic logic database and a context system, pack an artificial personality with (partial) human instincts, and bam I'm done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then again, would it be ethical?&lt;/p&gt;&#xA;" OwnerUserId="3483" LastEditorUserId="4302" LastEditDate="2018-10-08T12:38:36.723" LastActivityDate="2018-10-08T12:38:36.723" Title="Is it ethical to create a chatbot to answer questions on Stack Overflow?" Tags="&lt;natural-language-processing&gt;&lt;human-like&gt;&lt;ethics&gt;" AnswerCount="2" CommentCount="2" ClosedDate="2016-11-08T20:25:30.723" ContentLicense="CC BY-SA 3.0" />
  <row Id="2282" PostTypeId="2" ParentId="2281" CreationDate="2016-11-06T13:34:27.573" Score="3" Body="&lt;p&gt;Yes, it &lt;em&gt;is&lt;/em&gt; possible, and has actually been done in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The University of Antwerp created a &lt;a href=&quot;http://bvasiles.github.io/papers/chi16bot.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;bot to answer questions&lt;/a&gt; (&lt;a href=&quot;https://www.dropbox.com/s/o9tk8xtauyexn5c/Internship2DaanJanssensFinished.pdf?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this is the technical report&lt;/a&gt;). It focused on the &lt;a href=&quot;/questions/tagged/git&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;git&amp;#39;&quot; rel=&quot;tag&quot;&gt;git&lt;/a&gt; tag only though (even though it did answer one &lt;a href=&quot;/questions/tagged/mysql&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;mysql&amp;#39;&quot; rel=&quot;tag&quot;&gt;mysql&lt;/a&gt; question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its accuracy was pretty good, and the bots in the tests did earn some reputation. So I assume it is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But do note that the last bot in the tests revealed that it was a bot, and thus got banned. So if you reveal that the account you are running the bot on is a bot, there is a high chance that it will get banned.&lt;/p&gt;&#xA;" OwnerUserId="223" LastActivityDate="2016-11-06T13:34:27.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2283" PostTypeId="2" ParentId="2277" CreationDate="2016-11-06T14:29:40.870" Score="1" Body="&lt;p&gt;Well, it depends of the level of the AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can create an AI super autonomous with deep learning capabilities and so on, but in the robotic type only. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd create an AI like EVA in the Ex-Machina movie, humanoid form, deep neural transmissions and with cognitive dissonance, then it could feel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'AI' problem its not the chemical and neural transmissions, its the consciousness.&lt;/p&gt;&#xA;" OwnerUserId="3486" LastActivityDate="2016-11-06T14:29:40.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2285" PostTypeId="1" AcceptedAnswerId="2287" CreationDate="2016-11-06T17:43:07.307" Score="10" ViewCount="1818" Body="&lt;p&gt;What is the most advanced AI software/system that humans have made to date, and what does it do?&lt;/p&gt;&#xA;" OwnerUserId="3488" LastEditorUserId="2444" LastEditDate="2020-11-16T18:11:59.893" LastActivityDate="2020-11-17T05:29:14.593" Title="What is the most sophisticated AI ever made?" Tags="&lt;agi&gt;&lt;research&gt;&lt;state-of-the-art&gt;&lt;weak-ai&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="2286" PostTypeId="1" CreationDate="2016-11-06T18:40:52.660" Score="3" ViewCount="3964" Body="&lt;p&gt;Here is the definition of the entropy&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/ID3_algorithm#Entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;'s description of entropy breaks down the formula, but I still don't know how to determine the values of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, defined as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The set of classes in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;, defined as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The proportion of the number of elements in class &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to the number of elements in set &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can anyone break this down further to explain how to find &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3490" LastEditorUserId="2444" LastEditDate="2020-12-04T23:18:53.370" LastActivityDate="2020-12-04T23:18:53.370" Title="How to calculate the entropy in the ID3 decision tree algorithm?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;decision-tree&gt;&lt;id3-algorithm&gt;&lt;entropy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2287" PostTypeId="2" ParentId="2285" CreationDate="2016-11-06T20:13:59.553" Score="10" Body="&lt;p&gt;In my opinion, this would be &lt;a href=&quot;http://www.foundalis.com/res/diss_research.html&quot; rel=&quot;noreferrer&quot;&gt;Phaeaco&lt;/a&gt;, which was developed by Harry Foundalis at Douglas Hofstadter's CRCC research group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It takes noisy photographic images of &lt;a href=&quot;https://www.theguardian.com/science/2016/apr/25/can-you-solve-it-bongard-picture-puzzles-that-will-bongo-with-your-brain&quot; rel=&quot;noreferrer&quot;&gt;Bongard problems&lt;/a&gt; as input and (using a variant of Hofstadter's 'Fluid Concepts' architecture) successfully deduces the required rule in many cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hofstadter has described the related success of &lt;a href=&quot;https://en.wikipedia.org/wiki/Copycat_(software)&quot; rel=&quot;noreferrer&quot;&gt;CopyCat&lt;/a&gt; as being 'like a little kid doing a somersault': i.e. it doesn't have the flashy appeal of systems like AlphaGo. What it &lt;em&gt;does&lt;/em&gt; however have is a much more flexible (i.e. not precanned) approach to perception of problem structure than other systems, which Hofstadter claims (and many including Peter Norvig agree) is &lt;em&gt;the really hard problem&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-11-06T20:13:59.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2288" PostTypeId="2" ParentId="2281" CreationDate="2016-11-06T20:30:19.823" Score="0" Body="&lt;p&gt;Technically, creating a non-human account on Stack Exchange would violate the Terms of Service. You would have to find some way to keep it from getting banned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That having been said, creating, and learning are always good things. It would be a somewhat complex task, but I'm sure you would learn a lot from it. There are plenty of bots out that use the questions and answers from Stack Exchange already, but none directly on the site. &lt;/p&gt;&#xA;" OwnerUserId="1618" LastEditorUserId="223" LastEditDate="2016-11-07T17:01:33.643" LastActivityDate="2016-11-07T17:01:33.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2289" PostTypeId="2" ParentId="2285" CreationDate="2016-11-07T08:58:21.330" Score="0" Body="&lt;p&gt;In my opinion this would be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google search engine&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It searches the web.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-11-07T08:58:21.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2290" PostTypeId="2" ParentId="2285" CreationDate="2016-11-07T14:06:01.930" Score="2" Body="&lt;p&gt;&lt;strong&gt;AlphaGo&lt;/strong&gt; is the most sophisticated Artificial Intelligence program created by humans. It is a computer program that is &lt;strong&gt;developed by Google DeepMind&lt;/strong&gt; to play the board game &amp;quot;Go&amp;quot;. The game is different than other games, as &lt;strong&gt;The number of potential legal board positions is greater than the number of atoms in the universe&lt;/strong&gt;. It has way more legal board positions than the chess. So, &lt;em&gt;AlphaGo&lt;/em&gt; requires different technique for it's development.&lt;br&gt;&lt;br&gt;&#xA;Program's victories against the best players in the world in March 2016 &lt;strong&gt;is considered a major break through&lt;/strong&gt; in the field of AI. Go was previously considered to be a hard problem and many experts believed that current technology is not enough. Experts were saying that it will take atleast 5 years (or may be 10 years) before we will have a well developed Go software player.&lt;br&gt;&lt;br&gt;&#xA;The game used sophisticated algorithms of deep learning and reinforcement learning in order to learn the game. What makes this game different from other board game (like Chess, Reversi, etc.) is that moves are often based on intuition. If you ask a Chess player why he make a certain move, you will always be hearing an answer where he will explain you how he thought this move can increase in change of winning. Every move uses certain heuristics, strategy and/ or tricks. This is not the case with Go. Some moves are often taken because of intuition. Coding an AI software that can play a game, where intuition is a integral part of the game makes it different from other AIs that we have today.&lt;br&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;At present AlphaGo is the closest AI software to Artificial General Intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can go through these links for more information:&lt;br&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;First&lt;/a&gt;&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://deepmind.com/research/alphago/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Second&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="3005" LastEditDate="2020-11-17T05:29:14.593" LastActivityDate="2020-11-17T05:29:14.593" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="2291" PostTypeId="2" ParentId="1924" CreationDate="2016-11-07T14:36:16.030" Score="0" Body="&lt;p&gt;Yes, it is possible. For instance, if your vision system can only track one object at a time and is currently tracking one, any other object in the scene cannot be tracked. So there is inattentional blindness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A feature like this could be used in artificial vision system as a means of &quot;graceful degradation&quot; when the available computation power is not enough to allow for the tracking/labelling of all elements of a scene.&lt;/p&gt;&#xA;" OwnerUserId="3432" LastActivityDate="2016-11-07T14:36:16.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2292" PostTypeId="2" ParentId="2235" CreationDate="2016-11-07T19:00:20.607" Score="3" Body="&lt;p&gt;For normal &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process#Value_iteration&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration&lt;/a&gt;, you need to have the model, i.e. the transition probability, denoted by &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s,a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;With Q-learning, you use the current reward and the already stored Q value:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://wikimedia.org/api/rest_v1/media/math/render/svg/3d58e03dd47844bb627b83e1265163dcfab3961d&quot; alt=&quot;Q value update&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;The relation between the value function &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; is that the &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; function is simply the value of the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; is the highest, that is, &lt;span class=&quot;math-container&quot;&gt;$V(s) = \max_a Q(s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="198" LastEditorUserId="2444" LastEditDate="2020-09-27T22:50:52.963" LastActivityDate="2020-09-27T22:50:52.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2295" PostTypeId="2" ParentId="2274" CreationDate="2016-11-07T19:30:01.783" Score="4" Body="&lt;p&gt;Metaphorically: make it so depressed it commits suicide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per my answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/1768/could-a-paradox-kill-an-ai&quot;&gt;this AI SE question&lt;/a&gt;, the idea is to feed it a sequence of inputs that will cause it to become (permanently) inactive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technical details of how this might be achieved (and they &lt;em&gt;are&lt;/em&gt; somewhat technical) can be found in &lt;a href=&quot;https://arxiv.org/pdf/1606.00652.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-11-07T19:30:01.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2296" PostTypeId="2" ParentId="111" CreationDate="2016-11-08T04:02:24.983" Score="10" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;“This moral question of whom to save: 99 percent of our engineering work is to prevent these situations from happening at all.”&#xA;—Christoph von Hugo, Mercedes-Benz&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This quote is from an article titled &lt;a href=&quot;http://blog.caranddriver.com/self-driving-mercedes-will-prioritize-occupant-safety-over-pedestrians/&quot; rel=&quot;noreferrer&quot;&gt;Self-Driving Mercedes-Benzes Will Prioritize Occupant Safety over Pedestrians published OCTOBER 7, 2016 BY MICHAEL TAYLOR&lt;/a&gt;, retrieved 08 Nov 2016.&lt;/p&gt;&#xA;&lt;p&gt;Here's an excerpt that outlines what the technological, practical solution to the problem.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The world’s oldest carmaker no longer sees the problem, similar to the question from 1967 known as the Trolley Problem, as unanswerable. Rather than tying itself into moral and ethical knots in a crisis, Mercedes-Benz simply intends to program its self-driving cars to save the people inside the car. Every time.&lt;/p&gt;&#xA;&lt;p&gt;All of Mercedes-Benz’s future Level 4 and Level 5 autonomous cars will prioritize saving the people they carry, according to Christoph von Hugo, the automaker’s manager of driver assistance systems and active safety.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There article also contains the following fascinating paragraph.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A study released at midyear &lt;a href=&quot;http://science.sciencemag.org/content/352/6293/1514&quot; rel=&quot;noreferrer&quot;&gt;by Science&lt;/a&gt; magazine didn’t clear the air, either. The majority of the 1928 people surveyed thought it would be ethically better for autonomous cars to sacrifice their occupants rather than crash into pedestrians. Yet the majority also said they wouldn’t buy autonomous cars if the car prioritized pedestrian safety over their own.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3526" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-11-08T04:02:24.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2298" PostTypeId="2" ParentId="2277" CreationDate="2016-11-08T09:33:56.913" Score="9" Body="&lt;p&gt;There is much discussion in philosophy about inner language and the ability to perceive pain (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Pain_(philosophy)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pain in philosophy&lt;/a&gt; article). Your question is in the area of philosophy and not science. If you define emotion as some state then you can construct simple automata with two states (emotion vs no-emotion). It can be a very complicated state with degrees of truth (percentage of emotion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, to mimic human emotion you need to make a living human-like organism, and still with todays understanding and technology you will not be able to recognize emotion in it. The only thing you can do is trust when it says &quot;I'm sad&quot;. Now we are in the area of the Turing test, which is again philosophy, and not science.&lt;/p&gt;&#xA;" OwnerUserId="3529" LastEditorUserId="3989" LastEditDate="2016-12-09T17:52:14.060" LastActivityDate="2016-12-09T17:52:14.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2300" PostTypeId="2" ParentId="2226" CreationDate="2016-11-08T14:38:24.750" Score="5" Body="&lt;p&gt;In a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process (MDP)&lt;/a&gt; model, we define a set of states (&lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;), a set of actions (&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;), the rewards (&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;), and the transition probabilities &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s, a)$&lt;/span&gt;. The goal is to figure out the best action to take in each of the states, i.e. the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Policy&lt;/h2&gt;&#xA;&lt;p&gt;To calculate the policy we make use of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bellman_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bellman equation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_{i+1}(s)=R(s)+\gamma \max _{a \in A}\left(\sum_{s^{\prime} \in S} P\left(s^{\prime} \mid s, a\right) V_{i}\left(s^{\prime}\right)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When starting to calculate the values we can simply start with:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_{1}(s)=R(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;To improve this value, we should take into account the next action, which can be taken by the system and will result in a new reward:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_{2}(s)=R(s)+\gamma \max _{a \in A}\left(\sum_{s^{\prime} \in S} P\left(s^{\prime} \mid s, a\right) V_{1}\left(s^{\prime}\right)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here you take into account the reward of the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;, and the weighted sum of possible future rewards. We use &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s, a)$&lt;/span&gt; to give the probability of reaching state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is a value between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and is called the &lt;em&gt;discount factor&lt;/em&gt; because it reduces the importance of future rewards since these are uncertain. An often-used value is &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.95$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When using &lt;a href=&quot;http://artint.info/html/ArtInt_227.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration&lt;/a&gt; this process is continued until the value function has &lt;em&gt;converged&lt;/em&gt;, which means that the value function does not change significantly when doing new iterations:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left\|V_{i+1}(s)-V_{i}(s)\right\|&amp;lt;\epsilon, \; \forall_{s \in S},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is a really small value.&lt;/p&gt;&#xA;&lt;h2&gt;Discounted sum of future rewards&lt;/h2&gt;&#xA;&lt;p&gt;If you look at the Bellman equation and execute it iteratively you'll see:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ {\scriptstyle  V(s)=R(s) + \gamma \max _{a \in A}\left(\sum_{a \in A} P\left(s^{\prime} \mid s, a\right)\left[R\left(s^{\prime}\right) +  \gamma \max _{a \in A}\left(\sum_{s^{\prime \prime} \in S} P\left(s^{\prime \prime} \mid s^{\prime}, a\right)\left(R\left(s^{\prime \prime}\right) + \gamma \max _{a \in A}\left(\sum_{s^{\prime \prime \prime} \in S} P\left(s^{\prime \prime \prime} \mid s^{\prime \prime}, a\right) V\left(s^{\prime \prime \prime}\right)\right)\right]\right)\right.\right. }$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is like (without transition functions):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R(s)+\gamma R\left(s^{\prime}\right)+\gamma^{2} R\left(s^{\prime \prime}\right)+\gamma^{3} R\left(s^{\prime \prime \prime}\right)+\ldots$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h2&gt;To conclude&lt;/h2&gt;&#xA;&lt;p&gt;So when we start in state &lt;em&gt;s&lt;/em&gt; we want to take the action that gives us the best total reward taking into account not only the current, or next state, but all possible next states until we reach the goal. These are the time steps you refer to, i.e. each action taken is done in a time step. And when we learn the policy we try to take into account as many time steps as possible to choose the best action.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You can find quite a large number of examples if you search on the internet, for example, in the slides of &lt;a href=&quot;http://www.cs.cmu.edu/afs/andrew/course/15/381-f08/www/lectures/HandoutMDP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the CMU&lt;/a&gt;, the &lt;a href=&quot;https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa12/slides/mdps-exact-methods.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;UC Berkeley&lt;/a&gt; or the &lt;a href=&quot;https://homes.cs.washington.edu/%7Etodorov/courses/amath579/MDP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;UW&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="198" LastEditorUserId="2444" LastEditDate="2020-07-18T14:46:59.737" LastActivityDate="2020-07-18T14:46:59.737" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2301" PostTypeId="1" CreationDate="2016-11-08T18:15:34.353" Score="5" ViewCount="140" Body="&lt;p&gt;I was looking for an API service where I can ask it a general question (for example, &lt;em&gt;when was Einstein born?&lt;/em&gt;) and retrieve an answer from the web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any available service to do that? Have tried Watson services, but didn't work as expected.&lt;/p&gt;&#xA;" OwnerUserId="3539" LastEditorUserId="2444" LastEditDate="2021-01-26T17:51:00.657" LastActivityDate="2021-01-26T17:51:00.657" Title="Is there a service that can search for answers to general questions?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;question-answering&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="0" ClosedDate="2021-01-26T17:50:48.387" ContentLicense="CC BY-SA 4.0" />
  <row Id="2302" PostTypeId="1" CreationDate="2016-11-08T19:24:42.870" Score="3" ViewCount="170" Body="&lt;p&gt;At the moment, I am working on a project which requires me to build a naive Bayes classifier. Right now, I have a form online asking for people to submit a sentence and the subject of the sentence, in order to build a classifier to identify the subject of a sentence. But before I train the classifier, I intend on processing all entries for the parts-of-speech and the location of the subject. So my training set will be formatted as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sentence: Jake moved the chair.   Subject: Jake&#xA;POS-Tagged: NNP VBD DD NN         Location: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Would this be an effective way to build the classifier, or is there a better method?&lt;/p&gt;&#xA;" OwnerUserId="3542" LastEditorUserId="2444" LastEditDate="2019-11-15T20:17:39.650" LastActivityDate="2019-11-15T20:17:39.650" Title="What is the most effective way to build a classifier?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;naive-bayes&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2303" PostTypeId="1" AcceptedAnswerId="3159" CreationDate="2016-11-09T11:03:07.003" Score="3" ViewCount="118" Body="&lt;p&gt;What rectifier is better in general case of Convolutional Neural Network and how about empirical rules to use each type?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReLU&lt;/li&gt;&#xA;&lt;li&gt;PReLU&lt;/li&gt;&#xA;&lt;li&gt;RReLU&lt;/li&gt;&#xA;&lt;li&gt;ELU&lt;/li&gt;&#xA;&lt;li&gt;Leacky ReLU&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1442" LastActivityDate="2017-04-14T14:35:09.737" Title="What linear rectifier is better?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2304" PostTypeId="2" ParentId="2277" CreationDate="2016-11-09T11:07:10.580" Score="1" Body="&lt;p&gt;Yes and no. If you fully simulate a human brain and all of its functions, it would probably be able to feel emotions very similar to the way we do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we don't have enough capabilities and knowledge to do that, and maybe we could find a &quot;shortcut&quot; - a process that is intelligent without simulating a whole brain. In this case, emotions would probably represented by data values which say &quot;this is good (make it happen again!)&quot;, or &quot;this is bad (avoid it!)&quot;. This is just a very basic example (there are obviously many more emotions), but it would have a similar function and the AI would have similar solutions to the ones we have. But we don't know - and probably no one ever will know - if this data value 'bad' &quot;feels&quot; the same way for the AI the according emotion would feel to us. &lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2016-11-09T11:07:10.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2305" PostTypeId="2" ParentId="2274" CreationDate="2016-11-09T11:12:09.100" Score="3" Body="&lt;h3&gt;Nuke it from orbit - it's the only way to be sure&lt;/h3&gt;&#xA;&lt;p&gt;If you want to be really sure you destroy everything of the AI, you'll need to launch an EMP (electromagnetic pulse) from the orbit (there are different ways to achieve this, one would be an atomic bomb, but there are better ones). EMPs will destroy every electronic device it hits without causing really much damage to humans.&lt;/p&gt;&#xA;&lt;p&gt;Also an interesting read on a similar topic: &lt;a href=&quot;https://what-if.xkcd.com/5/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://what-if.xkcd.com/5/&lt;/a&gt;&#xA;Especially this is gonna be interesting:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[...] nuclear explosions generate powerful electromagnetic pulses. These EMPs overload and destroy delicate electronic circuits. [...]&#xA;And nuclear weapons could actually give us an edge. If we managed to&#xA;set any of them off in the upper atmosphere, the EMP effect would be&#xA;much more powerful.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="2444" LastEditDate="2020-08-18T12:59:51.640" LastActivityDate="2020-08-18T12:59:51.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2306" PostTypeId="1" CreationDate="2016-11-09T12:19:59.713" Score="9" ViewCount="1599" Body="&lt;p&gt;What are the top artificial intelligence journals?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for general artificial intelligence research, not necessarily machine learning. &lt;/p&gt;&#xA;" OwnerUserId="3550" LastEditorUserId="3550" LastEditDate="2016-11-09T15:32:57.147" LastActivityDate="2020-04-20T13:11:03.057" Title="What are the top artificial intelligence journals?" Tags="&lt;research&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="4" ClosedDate="2020-04-20T13:11:27.613" ContentLicense="CC BY-SA 3.0" />
  <row Id="2307" PostTypeId="2" ParentId="2306" CreationDate="2016-11-09T16:20:55.867" Score="4" Body="&lt;p&gt;This &lt;a href=&quot;http://www.scimagojr.com/journalrank.php?category=1702&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; includes various journals for artificial intelligence applied to various domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of those are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;IEEE Transactions on Human-Machine Systems&lt;/li&gt;&#xA;&lt;li&gt;Journal of the ACM&lt;/li&gt;&#xA;&lt;li&gt;Knowledge-based systems&lt;/li&gt;&#xA;&lt;li&gt;IEEE Transactions on Pattern Analysis and Machine Intelligence&lt;/li&gt;&#xA;&lt;li&gt;Journal of Memory and Language.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are many more. You can refer to any of those journals and explore the research done by AI enthusiasts and researchers.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="2444" LastEditDate="2020-04-20T13:08:25.113" LastActivityDate="2020-04-20T13:08:25.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2308" PostTypeId="2" ParentId="2302" CreationDate="2016-11-09T18:54:55.483" Score="1" Body="&lt;p&gt;Your approach would definitely work. I would recommend training a variety of classifiers and comparing their performance using multiclass roc analysis. Also, think about other useful features in addition to the ones you mentioned (e.g. pos tag). Feature engineering is one of the most important factors in building good predictive models. Another thing to keep in mind is that the classes could be highly imbalanced which might influence your model's performance.&lt;/p&gt;&#xA;" OwnerUserId="3556" LastActivityDate="2016-11-09T18:54:55.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2310" PostTypeId="2" ParentId="2306" CreationDate="2016-11-10T05:03:15.957" Score="2" Body="&lt;p&gt;I most often reference: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dblp.uni-trier.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dblp.uni-trier.de/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not a journal but it gets me where I need to go. &lt;/p&gt;&#xA;" OwnerUserId="3561" LastActivityDate="2016-11-10T05:03:15.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2311" PostTypeId="2" ParentId="2306" CreationDate="2016-11-10T06:41:42.773" Score="4" Body="&lt;p&gt;A couple of others:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://jair.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal of Artificial Intelligence Research (JAIR)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;IEEE Transactions on Knowledge and Data Engineering&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;IEEE Computational Intelligence Magazine&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2020-04-20T13:11:03.057" LastActivityDate="2020-04-20T13:11:03.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2312" PostTypeId="2" ParentId="2301" CreationDate="2016-11-10T06:48:00.777" Score="1" Body="&lt;p&gt;You could use &lt;a href=&quot;http://wiki.dbpedia.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbPedia&lt;/a&gt; and/or &lt;a href=&quot;https://www.wikidata.org/wiki/Wikidata:Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikidata&lt;/a&gt;.  I think Wikidata supports &lt;a href=&quot;https://en.wikipedia.org/wiki/SPARQL&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPARQL&lt;/a&gt; now, but don't quote me on that.  dbPedia definitely supports SPARQL.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not interested in writing SPARQL queries by hand, you could use something like &lt;a href=&quot;http://quepy.machinalis.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quepy&lt;/a&gt;. In fact, the Quepy demo demonstrates doing natural language queries against Freebase and/or dbPedia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could possibly also incorporate &lt;a href=&quot;http://sw.opencyc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCyc&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to roll something of your own, you might want to read some / all of the research papers published by the team from the &lt;a href=&quot;http://start.csail.mit.edu/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;START&lt;/a&gt; project at MIT.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-12-11T05:54:52.557" LastActivityDate="2016-12-11T05:54:52.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2313" PostTypeId="2" ParentId="2285" CreationDate="2016-11-10T06:52:26.037" Score="4" Body="&lt;p&gt;In addition to the answers already posted, I think IBM's &lt;a href=&quot;http://ibm.com/watson&quot; rel=&quot;nofollow noreferrer&quot;&gt;Watson&lt;/a&gt; deserves a mention.  It did something pretty impressive with its Jeopardy win, possibly as impressive as AlphaGo.  Sadly, since then, there don't seem to have been a lot of really public demos of Watson, as IBM is positioning the technology as a tool for companies and other organizations, and most of them are pretty secretive about the details of what they're doing.  I think they did publicize a bit of information about using it for medical diagnosis, but that's the only other application I can think of off hand.  I'm sure there are more though.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-11-10T06:52:26.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2314" PostTypeId="1" CreationDate="2016-11-10T10:02:07.070" Score="3" ViewCount="94" Body="&lt;p&gt;Is there any methodology to find proper parameter settings for a given meta-heuristic algorithm, e.g. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Firefly_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;firefly algorithm&lt;/a&gt; or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cuckoo_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuckoo search&lt;/a&gt;? Is this an open issue in optimization? Is extensive experimentation, measurements and intuition the only way to figure out which are the best settings? &lt;/p&gt;&#xA;" OwnerUserId="3566" LastEditorUserId="2444" LastEditDate="2020-03-09T16:55:31.530" LastActivityDate="2020-03-09T16:55:31.530" Title="How to find proper parameter settings for a given optimization algorithm?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;hyperparameter-optimization&gt;&lt;meta-heuristics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2315" PostTypeId="2" ParentId="2106" CreationDate="2016-11-10T19:08:45.553" Score="1" Body="&lt;p&gt;Another way of seeing the differences between these models in the case of binary classification for instance between a class A and a class B:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A generative model will be trained to model the properties of class A and another one will be trained to model the properties of class B. If we want to know if a new sample belongs to class A or B, we will compare it to each model and decide. The advantage is that we are able to synthetically generate more samples of these classes using the generative property of the model. The models have a &quot;global knowledge&quot; of what the classes are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, a discriminative model will &quot;pay attention&quot; to what differentiates the 2 classes. It is more straightforward and often computationally less expensive as the model does not need to grasp everything about each class but only what makes them different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is for the big picture. I find this course slides quite helpful to understand these concepts in more details (especially the first slides that are equation-free): &lt;a href=&quot;http://www.cedar.buffalo.edu/~srihari/CSE574/Discriminative-Generative.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cedar.buffalo.edu/~srihari/CSE574/Discriminative-Generative.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2016-11-10T19:08:45.553" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2316" PostTypeId="2" ParentId="2314" CreationDate="2016-11-10T22:24:46.453" Score="2" Body="&lt;p&gt;How to find the best configuration for an algorithm is an open research question in AI. The topic in general is known as `hyper-parameter optimization' and there are a range of possible methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the most popular is &lt;a href=&quot;http://iridia.ulb.ac.be/irace/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IRace&lt;/a&gt;, but other possibilities include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/JasperSnoek/spearmint&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spearmint&lt;/a&gt;: uses wrappers in Matlab or Python. It uses MongoDb,&#xA;and Bayesian optimisation algorithms.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.cs.ubc.ca/labs/beta/Projects/SMAC/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMAC&lt;/a&gt; requires a python wrapper for the algorithm to be optimized&#xA;and has a command line interface.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/hyperopt/hyperopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyperopt&lt;/a&gt;: a Python library which uses Random Search and Tree of&#xA;Parzen Estimators.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.automl.org/papers/13-BayesOpt%5C_EmpiricalFoundation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; argues that Spearmint performs the best, compared with SMAC and Hyperopt, but with significantly longer running times in some cases.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-11-10T22:24:46.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2317" PostTypeId="2" ParentId="2301" CreationDate="2016-11-11T04:03:31.227" Score="2" Body="&lt;p&gt;You can use Google&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://encrypted.google.com/search?hl=en&amp;amp;q=when%20was%20Einstein%20born&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://encrypted.google.com/search?hl=en&amp;amp;q=when%20was%20Einstein%20born&lt;/a&gt;&#xA;and parse the response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wolfram ALPHA is another candidate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://m.wolframalpha.com/input/?i=what+year+was+Einstein+born&amp;amp;x=0&amp;amp;y=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://m.wolframalpha.com/input/?i=what+year+was+Einstein+born&amp;amp;x=0&amp;amp;y=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can parse the returned html and see &quot;Result:&quot; div.&lt;/p&gt;&#xA;" OwnerUserId="3526" LastActivityDate="2016-11-11T04:03:31.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2319" PostTypeId="1" CreationDate="2016-11-11T23:04:19.753" Score="10" ViewCount="803" Body="&lt;p&gt;In programming languages, there is a set of grammar rules which govern the construction of valid statements and expressions. These rules help in parsing the programs written by the user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can there ever be a functionally complete set of grammar rules which can parse any statement in English (locale-specific) &lt;strong&gt;accurately&lt;/strong&gt; and which can be possibly implemented for use in AI-based projects?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there are a lot of NLP Toolkits available online, but they are not that effective. Most of them are trained using specific corpuses which sometimes fail to infer some complex correlations between various parts of an expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, what I am asking is that if it is possible for a computer to parse a well-versed sentence written in English as if it were parsed by an adult English-speaking human?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;If it cannot be represented using simple grammar rules, what kind of semantic structure can be used to generalize it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2: This &lt;a href=&quot;https://www.eecs.harvard.edu/shieber/Biblio/Papers/shieber85.pdf&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt; proves the absence of context-freeness in natural languages. I am looking for a solution, even if it is too complex.&lt;/p&gt;&#xA;" OwnerUserId="3592" LastEditorUserId="4302" LastEditDate="2018-10-08T12:36:30.397" LastActivityDate="2018-10-08T12:36:30.397" Title="Can the English language ever be generalized using a set of grammar rules?" Tags="&lt;ai-design&gt;&lt;natural-language-processing&gt;&lt;semantics&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2320" PostTypeId="2" ParentId="2274" CreationDate="2016-11-11T23:17:52.990" Score="2" Body="&lt;p&gt;If an AI is developed by humans, we surely can create another one!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Develop another AI agent without all the possible bugs that can make it go rogue to tackle the rogue AI, but more technically advanced than the previous one. Hardwire it with the sole purpose of disabling any rogue AI agent that can harm humanity and have it &lt;strong&gt;self-destruct&lt;/strong&gt; in case it is corrupted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI is really strong, it can anticipate every move of human resistance, but it cannot fathom the mind of another AI agent.&lt;/p&gt;&#xA;" OwnerUserId="3592" LastActivityDate="2016-11-11T23:17:52.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2322" PostTypeId="2" ParentId="2319" CreationDate="2016-11-12T22:16:06.647" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can there ever be a functionally complete set of grammar rules which can parse any statement in English (locale-specific) accurately and which can be possibly implemented for use in AI-based projects?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Parse it yes, accurately most likely no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to my understanding on how we derive meaning from sounds, there are 2 complementary strategies:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Grammar Rules:&lt;/strong&gt;&#xA;A rule based system for ordering words to facilitate communication, here meaning is derived from interaction of discrete sounds and their independent meaning, so you could parse a sentence based on a rule book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.G. &lt;strong&gt;&lt;em&gt;&quot;This was a triumph&quot;&lt;/em&gt;&lt;/strong&gt; : the parser would extract a pronoun (&lt;strong&gt;This&lt;/strong&gt;) with corresponding meaning ( a specific person or thing ) ; a verb (&lt;strong&gt;was&lt;/strong&gt;) with corresponding meaning ( occurred ); ( &lt;strong&gt;a&lt;/strong&gt;) and here we start with some parsing problems , what would the parser extract, a noun or an indefinite article ? An so we consult the grammar rule book, and settle for the meaning ( indefinite article any one of ), you have to parse the next word  and refer to it though, but let's gloss over that for now, and finally (&lt;strong&gt;triumph&lt;/strong&gt;) a noun ( it could also be a verb, but thanks to the grammar rule book we settled for a noun with meaning: ( victory,conquest), so in the end we have ( joining the meanings ):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A specific thing occurred of victory.&lt;/strong&gt; Close enough and I am glossing over a few other rules, but that's not the point, the other strategy is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A lexical dictionary (or lexicon)&lt;/strong&gt;&#xA;Where words or sounds are associated with specific meaning. Here meaning is derived from one or more words or sounds as a unit. This introduces the problem to a parser, since well, it shouldn't parse anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.G. &lt;strong&gt;&lt;em&gt;&quot;Non Plus Ultra&quot;&lt;/em&gt;&lt;/strong&gt; And so the AI parser would recognize that this phrase is not to be parsed and instead matched with meaning :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The highest point or culmination&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lexical units introduce another issue in that they themselves could be part of the first example, and so you end up with recursion.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if it is possible for a computer to parse a well-versed sentence written in English as if it were parsed by an adult English-speaking human?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe it could be possible, most examples I've seen deal effectively with the grammar rule book or the lexicon part, but I am not aware of a combination of both, but in terms of programming, it could happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately even if you solve this problem, your AI would not really understand things in the strict sense, but rather present you with very elaborate synonyms, additionally context (as mentioned in the comments) plays a role into the grammar and lexicon strategies. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If it cannot be represented using simple grammar rules, what kind of semantic structure can be used to generalize it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A mixed one where there are both grammar rules and a lexicon and both can change and be influenced based on the AI specific context and experience as well as a system for dealing with these objects could be one way.&lt;/p&gt;&#xA;" OwnerUserId="3020" LastActivityDate="2016-11-12T22:16:06.647" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2323" PostTypeId="2" ParentId="2319" CreationDate="2016-11-12T23:51:41.563" Score="2" Body="&lt;p&gt;I'm pretty sure that the answer is &quot;no&quot; in the strictest sense, since English simply doesn't have a formal definition.  That is, nobody controls English and publishes a formal grammar that everyone is required to adhere to. English is built up through an experiential process and it has contradictions and flaws, but the probabilistic nature of the human mind allows us to work around those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, that this &quot;sentence&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This sentence no verb&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically it's not a sentence at all, since it doesn't have a verb.  But did anybody have any problem understanding what it meant? Doubtful.  Try coming up with a formal rule for that though.  And that's just one example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, could you come up with a formal grammar that covers, maybe, 90% of cases, and is &quot;good enough&quot; for most practical uses? Possibly, maybe even probably.  But I am pretty sure it's not possible to get to 100%.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-11-12T23:51:41.563" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2324" PostTypeId="1" AcceptedAnswerId="2337" CreationDate="2016-11-13T07:15:29.303" Score="3" ViewCount="487" Body="&lt;p&gt;It seems to me that the first AGIs ought to be able to perform the same sort and variety of tasks as people, with the most computationally strenuous tasks taking an amount of time compared to how long a person would take. If this is the case, and people have yet to develop basic AGI (meaning it's a difficult task), should we be concerned if AGI is developed? It would seem to me that any fears about a newly developed AGI, in this case, should be the same as fears about a newborn child.&lt;/p&gt;&#xA;" OwnerUserId="3604" LastEditorUserId="2444" LastEditDate="2019-06-28T14:34:52.687" LastActivityDate="2019-11-06T11:24:00.790" Title="Should we fear an AGI if it will be equivalent to a human?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;human-like&gt;&lt;superintelligence&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2330" PostTypeId="1" CreationDate="2016-11-14T04:19:31.873" Score="7" ViewCount="4979" Body="&lt;p&gt;Based on fitting to historical data and extrapolation, when is it expected that the number of neurons in AI systems will equal those of the human brain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in a possible direct replication of the human brain, which will need equal numbers of neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this assumes neurons which are equally capable as their biological counterparts, which development may happen at a faster or slower rate than the quantitative increase.&lt;/p&gt;&#xA;" OwnerUserId="2424" LastEditorUserId="2424" LastEditDate="2016-11-14T15:21:31.023" LastActivityDate="2021-04-23T13:08:35.453" Title="When will the number of neurons in AI systems equal the human brain?" Tags="&lt;neurons&gt;&lt;prediction&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2331" PostTypeId="2" ParentId="2330" CreationDate="2016-11-14T05:57:36.463" Score="7" Body="&lt;p&gt;Soon enough but that doesn't mean anything at all. In machine learning the word neuron represents a calculation whereas in brain the word neuron represent a specific type of cell which is a biochemical system.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-11-14T05:57:36.463" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2333" PostTypeId="2" ParentId="2330" CreationDate="2016-11-14T07:29:24.257" Score="3" Body="&lt;p&gt;The answers so far haven't answered the question numerically, so here is my attempt to steer them in the direction I was seeking:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The freely available &lt;a href=&quot;http://www.deeplearningbook.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Book&lt;/a&gt; has the following figure on page 27:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iz2C4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iz2C4.png&quot; alt=&quot;Size of neural nets over time&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I question the blue fit line, as it seems that data points may be better described by a parabolic or exponential function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, based upon this conservative linear fit, the authors predict that the number of neurons in a ANN will equal that of the human brain in 2056.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The referenced nerual networks are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f8Y6O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f8Y6O.png&quot; alt=&quot;Neural networks referred to in image above&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is interesting to note that when &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Singularity_Is_Near&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Singularity is Near&lt;/a&gt; was written in 2006, Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one.&lt;/p&gt;&#xA;" OwnerUserId="2424" LastEditorUserId="2424" LastEditDate="2016-11-14T07:35:01.833" LastActivityDate="2016-11-14T07:35:01.833" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2334" PostTypeId="2" ParentId="2330" CreationDate="2016-11-14T08:02:28.057" Score="14" Body="&lt;p&gt;Some back of the envelope calculations :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;number of neurons in AI systems &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The number of neurons in AI systems is a little tricky to calculate, Neural Networks and Deep Learning are 2 current AI systems as you call them, specifics are hard to come by (If someone has them please share), but data on parameters do exist, parameters are more analogous to synapses (connections) than neurons (the nodes in between connections) somewhere in the range of 100-160 billion is the current upper number for specialized networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deriving the number of neurons in AI systems from this number is a stretch since these AIs emulate certain types of connections and sub assemblies of neurons, but let's continue...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;equal those of the human brain?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So now let's look at the brain, and again this are all contested numbers. Number of neurons ~ 86 Billion, Number of Synapses ~ 150 Trillion, another generalization: average number of synapses per neuron ~ 1,744.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we have something to compare, and I can't stress this enough, these are all wonky numbers, so let's make our life a little easier and divide :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of Synapses (Brain ) : 150 trillion /  Number of parameters AIs : 150 billion = 1,000 or in other words current AIs would have to scale by a factor of one thousand their connections to be on par with the brain...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of Neurons (Brain ) : 86 Billion / Number of Neurons AIs ( 150 billion / 1,744 )  = 86 Million equivalent AI Neurons&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which makes sense, mathematically at least : you can multiply the factor ( 1000 ) times the current number of equivalent AI Neurons ( 86 million) to get the number of neurons in the human brain (86 Billion)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well,let's use  moore's law ( number of transistors processing power doubles about every 2 years ) as a rough measure of technological progress: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     #AI NEURONS            YEAR&#xA;     86,000,000             2016&#xA;     172,000,000            2018&#xA;     344,000,000            2020&#xA;     688,000,000            2022&#xA;     1,376,000,000          2024&#xA;     2,752,000,000          2026&#xA;     5,504,000,000          2028&#xA;     11,008,000,000         2030&#xA;     22,016,000,000         2032&#xA;     44,032,000,000         2034&#xA;     88,064,000,000         2036&#xA;&#xA;&#xA;     # NEURONS HUMAN BRAIN &#xA;     86,000,000,000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, if all this made sense to you, somewhere around the year 2035. &lt;/p&gt;&#xA;" OwnerUserId="3020" LastEditorUserId="3020" LastEditDate="2016-11-22T19:43:08.360" LastActivityDate="2016-11-22T19:43:08.360" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2335" PostTypeId="1" CreationDate="2016-11-14T08:55:39.600" Score="2" ViewCount="37" Body="&lt;p&gt;I'm trying to find the optimized mixture for a specific set of substances. Each of those substances have characteristics that I want to optimize in the mixture (some characteristics I want to minimize and others I want to maximize). But I can't have more than 50% (random value that will be set on running time) of one of those substances in the mixture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using Genetic Algorithm, but I'm not sure it's the best approach for this problem. Do you have any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: it doesn't need to be a evolutionary algorithm.&lt;/p&gt;&#xA;" OwnerUserId="3627" LastEditorUserId="3627" LastEditDate="2016-11-14T15:40:25.753" LastActivityDate="2016-11-14T15:40:25.753" Title="Knapsack of mixture with constraints" Tags="&lt;optimization&gt;&lt;evolutionary-algorithms&gt;&lt;heuristics&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2337" PostTypeId="2" ParentId="2324" CreationDate="2016-11-14T10:48:13.510" Score="3" Body="&lt;p&gt;There are basically two worries:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we create an AGI that is a slightly better AGI-programmer than its creators, it might be able to improve its own source code to become even more intelligent. Which would enable it to improve its source code even more etc. Such a selfimproving seed AI might very quickly become superintelligent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other scenario is that intelligence is such a complicated algorithmic task, that when we finally crack it, there will be a significant hardware overhang. So the &quot;intelligence algorithm&quot; would be human level on 2030 hardware, but we figure it out in 2050. In that case we would immediately have superintelligent AI without ever creating human level AI. This scenario is especially likely because development often requires a lot of test runs to tweak parameters and try out different ideas. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-11-14T10:48:13.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2338" PostTypeId="1" CreationDate="2016-11-14T15:44:57.997" Score="2" ViewCount="323" Body="&lt;p&gt;What are the current best estimates as to what year artificial intelligence will be able to score 100 points on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stanford%E2%80%93Binet_Intelligence_Scales&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford Binet IQ test&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2424" LastActivityDate="2016-11-17T12:07:52.950" Title="When will artificial intelligence equal human intelligence?" Tags="&lt;intelligence-testing&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2339" PostTypeId="2" ParentId="2338" CreationDate="2016-11-14T16:01:27.130" Score="1" Body="&lt;p&gt;Nobody knows.&lt;/p&gt;&#xA;&lt;p&gt;However according to &lt;a href=&quot;https://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzweil#2020s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kurzweil it's late 20s&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;2020s:&lt;/p&gt;&#xA;&lt;p&gt;Early in this decade, humanity will have the requisite hardware to emulate human intelligence within a $1000 personal computer, followed shortly by effective software models of human intelligence toward the middle of the decade: this will be enabled through the continuing exponential growth of brain-scanning technology, which is doubling in bandwidth, temporal and spatial resolution every year, and will be greatly amplified with nanotechnology, allowing us to have a detailed understanding of all the regions of the human brain and to aid in developing human-level machine intelligence by the end of this decade.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-11-16T11:40:41.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2342" PostTypeId="1" AcceptedAnswerId="2687" CreationDate="2016-11-16T12:35:47.487" Score="5" ViewCount="756" Body="&lt;p&gt;When I have read through the fundamentals of AI, I saw a situation (i.e., a search space) which is illustrated in the following picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zX6wZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zX6wZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the heuristic estimates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h(B)=9&#xA;h(D)=10&#xA;h(A)=2&#xA;h(C)=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we use the A* algorithm, the node &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; will be expanded first because &lt;span class=&quot;math-container&quot;&gt;$f(B)=1+9=10$&lt;/span&gt;, while node &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; having &lt;span class=&quot;math-container&quot;&gt;$f(A)=9+2=11$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(B)&amp;lt;f(A)$&lt;/span&gt;, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, the search tree will go on in the order &lt;code&gt;R -&amp;gt; B -&amp;gt; D -&amp;gt; G2&lt;/code&gt;. Will the search go on to also find the goal state G1?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly let me know the order of the search if I am wrong.&lt;/p&gt;&#xA;" OwnerUserId="3673" LastEditorUserId="2444" LastEditDate="2019-11-10T17:10:45.097" LastActivityDate="2019-11-10T17:10:45.097" Title="How does A* search work given there are multiple goal states?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;a-star&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2344" PostTypeId="2" ParentId="2342" CreationDate="2016-11-16T21:16:23.997" Score="5" Body="&lt;p&gt;Yes. If you leave A* running (i.e. do not impose a goal condition on a newly-encountered state), all states will be explored, just as they would be in breadth- or depth- first search.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2016-11-16T21:16:23.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2345" PostTypeId="2" ParentId="111" CreationDate="2016-11-17T04:05:06.240" Score="4" Body="&lt;p&gt;I think that in most cases the car would default to reducing speed as a main option, rather than steering toward or away from a specific choice. As others have mentioned, having settings related to ethics is just a bad idea. What happens if two cars that are programmed with opposite ethical settings and are about to collide? The cars could potentially have a system to override the user settings and pick the most mutually beneficial solution. It's indeed an interesting concept, and one that definitely has to discussed and standardized before widespread implementation. Putting ethical decisions in a machines hands makes the resulting liability sometimes hard to picture.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2016-11-18T05:22:24.577" LastActivityDate="2016-11-18T05:22:24.577" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2346" PostTypeId="2" ParentId="2338" CreationDate="2016-11-17T12:07:52.950" Score="2" Body="&lt;p&gt;Not going into details of Stanford–Binet test, but just looking at &lt;a href=&quot;https://en.wikipedia.org/wiki/Stanford%E2%80%93Binet_Intelligence_Scales&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt; it shows many subtests like knowledge, reasoning, verbal tests etc. Most of the efforts in the artificial intelligence today is directed into research of specific areas like computer vision, natural language processing, machine learning, but also combination of fields like implementation of self driving cars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within every field there are still other subfields and problems that are not solved yet. For example, development of human-like natural language processing (NLP) is necessary for intelligent agent to pass any verbal tests, or even non-verbal tests that requires processing of sentences of human language. Famous test that tests intelligence by asking questions in natural language and expects answers in the same form is Turing test. NLP still struggles with many (basic) human skills like listening, speaking, parsing and forming sentences. No one knows when we'll have system that can do these things as good as human. Since this system is crucial, but also far from human-like it's likely cause of delay in developing AI that passes intelligence test. Are these problems AI-hard? Do we need to develop strong AI to solve them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can look at speech and listening as interfaces used for expressing and affecting inner processes of human brain. Same goes for other senses like eyesight which is being approximated by computer vision. One could say that we only need to develop convincing mimics of human senses and incorporate them in one big system that will become first human-like AI. That is the minimum requirement. &lt;strong&gt;I doubt this will be achieved in this century.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Other thoughts)&lt;br/&gt;&#xA;What truly defines intelligence is brain activity. Since it's really complex and one artificial neuron is not equal to one neuron in brain, increase in computation power will not necessarily help achieving human-like AI. Also recognizing such system by mere intelligence test is questionable. For now it's only philosophical discussion but by the time we are able to design such machine I think we'll also have better understanding of human brain. Someone in 2100 might not read this answer on quantum computer with integrated AI OS powered from fusion reactor in his self-flying car, but will probably have many systems that help him in everyday tasks far more than we imagine today.&lt;/p&gt;&#xA;" OwnerUserId="3690" LastActivityDate="2016-11-17T12:07:52.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2347" PostTypeId="1" CreationDate="2016-11-17T18:22:39.990" Score="6" ViewCount="385" Body="&lt;p&gt;&lt;a href=&quot;http://www.bbc.co.uk/newsbeat/article/34243898/five-questions-we-should-ask-ourselves-before-ai-answers-them-for-us&quot; rel=&quot;nofollow noreferrer&quot;&gt;This BBC article&lt;/a&gt; suggests that intelligent algorithms, like those that select news stories and advertisements for display, could control our experience of the world and manipulate us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will Artificial Intelligence someday become a problem to humanity after learning human behaviors and characteristics?&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2019-04-05T12:16:59.427" LastActivityDate="2019-04-21T07:15:48.943" Title="Could artificial intelligence cause problems for humanity after figuring out human behavior?" Tags="&lt;philosophy&gt;&lt;human-like&gt;&lt;control-problem&gt;&lt;social&gt;&lt;neo-luddism&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2348" PostTypeId="2" ParentId="2347" CreationDate="2016-11-17T19:18:47.523" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Will Artificial Intelligence some day become a problem to humanity&#xA;  after learning human behaviors and characteristics?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It can be answered in both ways, I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Yes, they may become a problem.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the increasing integration of loads of apps and smart devices in our life, almost everything defining an individual human being is digitalised. For instance, our fingerprints, voice, facial image etc. Apart from these data, we use those apps and devices to track our health (heart rate, calorie intake etc), to plan our schedules, and most importantly to communicate. If some sort of AI engine is integrated into a chat application, for instance, it can learn our typing patterns, conversation style, and hundreds of other unforeseen parameters. Imagine what can be learned about a person if such AI is coded inside every device and every app in your day-to-day use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use smart devices and apps to harness their functionalities and features which ease our way of life, and we give them, unintentionally, our identity and sometimes, even our personality. For them, these are the parameters that can be input to some machine-learning algorithm and predict what we will do the next day, or what will happen to us the next day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sounds like a major problem, especially when these technologies are indispensable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;No, they may not become a problem.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans are really complex creatures and possess the most advanced intelligence technology called the brain. I think the brain can be thought of as a technology in this context. There are several tissues inside the brain that can learn to do certain things themselves. It was proven in a research that some tissues have the capability to perform the functions of other tissues using neuro-rewiring techniques. Imagine that the same tissue that has helped you see until now can be made to help you hear instead. Now imagine mimicking such a technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it is not impossible, for an AI to achieve the brilliance of a human brain is a topic of ongoing research. To train a machine for the purpose, we would have to feed it with gazillion behaviors and characteristics, which it may not be able to handle! After learning some behavior and characteristics, the AI would be said to be smart, but it would still predate us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, they may not become a problem at all because of our brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithms (like recommender systems) used by Amazon and Facebook influence us or even manipulate us. But, that manipulation is either very obvious (like viewing promoted products) or is in company's best interests (like viewing a certain news piece). It may be even possible that several external parameters are used by these systems to improve your experience. For instance, Google ads show us what we were looking for on an online store when we visit any random website. In most cases, what you see is a result of what you were looking for before. If any attempt to influence does happen, we may learn to avoid it through careful observation or even experience.&lt;/p&gt;&#xA;" OwnerUserId="3592" LastEditorUserId="3592" LastEditDate="2016-11-17T19:45:32.517" LastActivityDate="2016-11-17T19:45:32.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2349" PostTypeId="1" AcceptedAnswerId="2384" CreationDate="2016-11-17T20:46:24.343" Score="8" ViewCount="1119" Body="&lt;h3&gt;Background&lt;/h3&gt;&#xA;&lt;p&gt;I've been interested in and reading about neural networks for several years, but I haven't gotten around to testing them out until recently.&lt;/p&gt;&#xA;&lt;p&gt;Both for fun and to increase my understanding, I tried to write a class library from scratch in .Net. For tests, I've tried some simple functions, such as generating output identical to the input, working with the MNIST dataset, and a few binary functions (two input OR, AND and XOR, with two outputs: one for true, one for false).&lt;/p&gt;&#xA;&lt;p&gt;Everything seemed fine when I used a &lt;strong&gt;sigmoid&lt;/strong&gt; function as the activation function, but, after reading about the ReLUs, I decided to switch over for speed.&lt;/p&gt;&#xA;&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&lt;p&gt;My current problem is that, when I switch to using ReLUs, I found that I was &lt;em&gt;unable&lt;/em&gt; to train a network of any complexity (tested from as few as 2 internal nodes up to a mesh of 100x100 nodes) to correctly function as an XOR gate.&lt;/p&gt;&#xA;&lt;p&gt;I see two possibilities here:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;My implementation is faulty. (This one is frustrating, as I've re-written the code multiple times in various ways, and I still get the same result).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Aside from being faster or slower to train, there are some problems that are impossible to solve given a specific activation function. (Fascinating idea, but I've no idea if it's true or not).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My inclination is to think that 1) above is correct. However, given the amount of time I've invested, it would be nice if I could rule out 2) definitively before I spend even more time going over my implementation.&lt;/p&gt;&#xA;&lt;h3&gt;More details&lt;/h3&gt;&#xA;&lt;p&gt;For the XOR network, I have tried both using two inputs (0 for false, 1 for true), and using four inputs (each pair, one signals true and one false, per &amp;quot;bit&amp;quot; of input). I have also tried using 1 output (with a 1 (really, &amp;gt;0.9) corresponding to true and a 0 (or &amp;lt;0.1) corresponding to false), as well as two outputs (one signaling true and the other false).&lt;/p&gt;&#xA;&lt;p&gt;Each training epoch, I run against a set of 4 inputs &lt;span class=&quot;math-container&quot;&gt;$\{ (00, 0), (01, 1), (10, 1), (11, 0) \}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I find that the first three converge towards the correct answer, but the final input (11) converges towards 1, even though I train it with an expected value of 0.&lt;/p&gt;&#xA;" OwnerUserId="3702" LastEditorUserId="2444" LastEditDate="2021-01-17T20:43:54.733" LastActivityDate="2021-01-18T05:13:16.110" Title="Are ReLUs incapable of solving certain problems?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;function-approximation&gt;&lt;relu&gt;&lt;sigmoid&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2350" PostTypeId="2" ParentId="2349" CreationDate="2016-11-18T18:23:32.520" Score="1" Body="&lt;p&gt;While I have not determined if there are problems that cannot be solved with ReLU, I have found ample documentation in the literature that XOR is solvable with as few as 1 hidden node.&lt;/p&gt;&#xA;&lt;p&gt;The solution is simpler than I thought. The output layer needs connections, not just to the intermediate layer, but directly to the input layer as well. This allows the network to train XOR effectively.&lt;/p&gt;&#xA;&lt;p&gt;One final note, the XOR is extremely sensitive to the learning rate. Essentially, whatever learning rate is appropriate for the AND and OR functions, is approximately 1000x too large to train XOR effectively.&lt;/p&gt;&#xA;" OwnerUserId="3702" LastEditorUserId="2444" LastEditDate="2021-01-17T20:52:56.763" LastActivityDate="2021-01-17T20:52:56.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2351" PostTypeId="1" AcceptedAnswerId="2355" CreationDate="2016-11-18T20:16:43.493" Score="3" ViewCount="1584" Body="&lt;p&gt;Does anyone know, or can we deduce or infer with high probability from its characteristics, whether the neural network used on this site &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://quickdraw.withgoogle.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://quickdraw.withgoogle.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a type of convolutional neural network (CNN)?&lt;/p&gt;&#xA;" OwnerUserId="46" LastActivityDate="2017-01-07T15:47:06.297" Title="Is the QuickDraw with Google neural net a convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2353" PostTypeId="1" CreationDate="2016-11-19T06:02:15.850" Score="5" ViewCount="770" Body="&lt;p&gt;After the explosion of fake news during the US election, and following the question about whether AIs can educate themselves via the internet, it is clear to me that any newly-launched AI will have a serious problem knowing what to believe (that is, rely on as input for making predictions and decisions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Information provided by its creators could easily be false. Many AIs won't have access to cameras and sensors to verify things by their own observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there was to be some kind of verification system for information (like a &quot;blockchain of truth&quot;, for example, or a system of &quot;trusted sources&quot;), how could that function, in practical terms? &lt;/p&gt;&#xA;" OwnerUserId="3601" LastEditorUserId="2444" LastEditDate="2019-06-20T21:45:15.663" LastActivityDate="2019-06-20T21:45:15.663" Title="How can an artificial general intelligence determine which information is true?" Tags="&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2354" PostTypeId="2" ParentId="2353" CreationDate="2016-11-19T11:14:09.067" Score="2" Body="&lt;p&gt;Not possible without some big restrictions. What it can do is look at known &quot;good&quot; sites and compare news with site that is potentially &quot;bad&quot;. Obvious problem here is defining some sites as absolute truth. For example it can recognize, while reading text, that some politician said something. These sentences can be compared with other sites, and if there is significant difference, that news is candidate for false news.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practical terms, program would extract sentences &quot;i like cats&quot;, &quot;says he likes cats&quot;, &quot;cats that John likes&quot; etc. We need part that recognizes something as a quote, part that extracts it and finally parser so we end up with structure stored in some form that contains meaning of sentence (john-like-cats). Also it can keep information of time and context in which it was said, like timestamp of an article, some proper nouns that can indicate place (XY conference, London...). Now, suspicious article can be compared and checked if it matches time, place, some context and contains quote that is similar. Finally it needs to compare how different it is from other quotes. &quot;...hates cats&quot; should be labeled as potential fake news, but &quot;likes dogs&quot;, &quot;thinks cats are OK&quot;, &quot;sings well&quot; etc. should not. This can be expanded into comparison of whole articles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many features that can be used to define particular article as fake. Interesting feature  for finding fake sites could be bias when it comes to particular (political, economical, ecological...) opinion. But in the end machine can't decide if the article is fake without comparing it to other articles. It is bound to closed system that reflects real world in subjective way.&lt;/p&gt;&#xA;" OwnerUserId="3690" LastActivityDate="2016-11-19T11:14:09.067" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2355" PostTypeId="2" ParentId="2351" CreationDate="2016-11-19T22:19:06.493" Score="6" Body="&lt;p&gt;I believe they don't use CNNs. The most important reason why it's because they have more information than a regular image: time. The input they receive is a sequence of (x,y,t) as you draw on the screen, which they refer as &quot;ink&quot;. This gives them the construction of the image for free, which a CNN would have to deduce by itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They tried two approaches. Their currently most successful approach does the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Detect parts of the ink that are candidates of being a character&lt;/li&gt;&#xA;&lt;li&gt;Use a FeedForward Neural Network to do character recognition on those candidates&lt;/li&gt;&#xA;&lt;li&gt;Use beam search and a language model to find most the most likely combination of results that results into a word&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Their second approach is using an LSTM (a type of Recurrent Neural Network) end-to-end. In their paper they say this was better in a couple languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source&lt;/strong&gt;: I was an intern in Google's handwriting team in summer 2015 (on which I believe quickdraw is based), but the techniques I explained can be found in &lt;a href=&quot;http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;amp;arnumber=7478642&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3745" LastEditorUserId="3745" LastEditDate="2017-01-07T15:47:06.297" LastActivityDate="2017-01-07T15:47:06.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2356" PostTypeId="1" AcceptedAnswerId="2359" CreationDate="2016-11-20T04:25:14.977" Score="9" ViewCount="350" Body="&lt;p&gt;For example, would an AI be able to own property, evict tenants, acquire debt, employ, vote, or marry? What are the legal structures in place to implement a strong AI into society? &lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="3748" LastEditDate="2016-11-23T01:22:43.420" LastActivityDate="2016-12-18T18:54:38.513" Title="Would an AI with human intelligence have the same rights as a human under current legal frameworks?" Tags="&lt;agi&gt;&lt;control-problem&gt;&lt;legal&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2357" PostTypeId="2" ParentId="2356" CreationDate="2016-11-20T05:46:32.043" Score="0" Body="&lt;p&gt;There is a legal difference between a &quot;person&quot; (which includes bodies corporate - corporations, incorporated associations, etc - and actual people) vs &quot;natural person&quot; (which is specifically a human being).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an AI to marry, it would need to get the legal definition of &quot;natural person&quot; changed, and depending on the jurisdiction possibly also the definition of &quot;man&quot; or &quot;woman&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other things, such as owning property, evicting tenants, entering into contracts, etc, an AI would simply use a corporation. It may be that the corporation might need to have a minimum number of directors who are natural persons, but they could just be paid professionals, so no issue there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With credit cards, it would depend on the policy of the issuing bank. There is no legal impediment to corporations having credit cards in their own right, but in practice banks often require a director's guarantee from a natural person that they can sue if the bill is not paid. They want to be sure they will get their money, even if the corporation is wound up.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastEditorUserId="3601" LastEditDate="2016-11-23T13:28:22.560" LastActivityDate="2016-11-23T13:28:22.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2358" PostTypeId="2" ParentId="2277" CreationDate="2016-11-20T06:00:23.973" Score="2" Body="&lt;p&gt;Emotions are a factor in humans having ethics/morals only because they are a factor in all human learning and decision-making.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you are duplicating a human being exactly, there is no reason to think that an AI will learn the way a human learns, or make decisions in the same way a human makes decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, whether it &quot;feels emotion&quot; just like we do, or whether it simply responds to outcomes &quot;cost is greater = don't go there&quot;, the outcome of ethical BEHAVIOUR could be achieved. An AI could behave perfectly ethically without any need for feeling empathy, shame, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also argue that a lot of UNETHICAL behaviour in human beings is driven by emotions, too, and that an unemotional but ethical AI may well do a better overall job than a human being.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2016-11-20T06:00:23.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2359" PostTypeId="2" ParentId="2356" CreationDate="2016-11-20T07:34:37.700" Score="6" Body="&lt;p&gt;Yes, to some of what you propose.  No to some.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today corporations are granted rights: to own property, earn income, pay taxes, contribute to political campaigns, offer opinion in public, ad more.  Even now I see no reason why an AI should not be eligible to incorporate itself, thereby inheriting all these rights.  Conversely, any corporation already in existence could become fully automated at any time (and some plausibly will).  In doing so, they should not lose any of the rights and duties they currently employ.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I suspect certain rights would be unavailable to an AI just as they are unavailable to a corporation now: marriage, draft or voluntary service in the military, rights due a parent or child or spouse, estate inheritance, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could this schizoid sense of human identity be resolved at some point?  Sure.  Already there have been numerous laws introduced and some passed elevating various nonhuman species to higher levels of civil rights that only humans heretofore enjoyed: chimpanzees, cetaceans, parrots and others have been identified as 'higher functioning' and longer lived, and so, are now protected from abuse in ways that food animals, pets, and lab animals are not.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once AI 'beings' arise that operate for years and express intelligence and emotions that approach human-level and lifetime, I would expect a political will to arise to define, establish, and defend their civil rights.  And as humans become more cybernetically augmented, especially cognitively, the line that separates us from creatures of pure silicon will begin to blur.  In time it will become unconscionable to overlook the rights of beings simply because they contain 'too little flesh'.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-11-20T07:34:37.700" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2360" PostTypeId="2" ParentId="2356" CreationDate="2016-11-20T12:35:01.477" Score="3" Body="&lt;p&gt;Murray Shanahan, in his book &lt;strong&gt;The Technological Singularity&lt;/strong&gt;, makes the case that the rights of any being are determined by its intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, we value the life of a dog above that of an ant and likewise value human life above that of other animals.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From here one could argue that a general artificial intelligence of equal intelligence to a human should have equal rights to a human and a superior artificial intelligence should have more rights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The question, of course, is whether our anthropocentric society would be willing to accept this fundamental shift in human rights and this idea of removing humanity from its pedestal of importance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to legal frameworks, we really are entering into uncharted territory as AI is going to have to revolutionise the way we define many of the terms we take for granted today and question many of our usual assumptions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AI is going to drive an important shift in our mindset well before it exceeds human intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3427" LastActivityDate="2016-11-20T12:35:01.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2361" PostTypeId="2" ParentId="2277" CreationDate="2016-11-20T12:50:16.217" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;It is certainly possible for AI to theoretically feel emotion. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are, according to Murray Shanahan's book &lt;strong&gt;The Technological Singularity&lt;/strong&gt;, two primary forms of AI:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) Human based AI - achieved through processes such as &lt;strong&gt;&lt;em&gt;whole brain emulation&lt;/em&gt;&lt;/strong&gt;, the functioning of human based AI would likely be indistinguishable from that of the human brain, and, as a consequence, human based AI would likely experience emotion in the same manner as humans.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2) AI from scratch - with this form of AI, based on machine learning algorithms and complex processes to drive goals, we enter into uncharted territory as the development of this form of AI is inherently unpredictable and unlike anything we observe in the biological sample space of intelligence we have access to.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;With this form of AI, there is no telling if and how it could experience emotion.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As the question references the former, it is very likely that human-based AI would indeed experience emotion and other human-like characteristics.&lt;/p&gt;&#xA;" OwnerUserId="3427" LastActivityDate="2016-11-20T12:50:16.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2363" PostTypeId="1" AcceptedAnswerId="2364" CreationDate="2016-11-21T12:44:57.060" Score="6" ViewCount="873" Body="&lt;p&gt;I have implemented a Sobel filter for edge detection in Matlab without using its toolbox. I am a bit confused: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is a Sobel filter a type of Cellular Neural Network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both Sobel and Cellular Neural Network calculate output via its neighborhood cells.&lt;/p&gt;&#xA;" OwnerUserId="3763" LastEditorUserId="1671" LastEditDate="2018-01-28T23:05:11.443" LastActivityDate="2018-01-28T23:05:11.443" Title="Is a Sobel filter for edge detection a type of Cellular Neural Network?" Tags="&lt;neural-networks&gt;&lt;matlab&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2364" PostTypeId="2" ParentId="2363" CreationDate="2016-11-21T21:57:00.670" Score="4" Body="&lt;p&gt;You're right about the basic arrangement of the inputs, but there are a number of differences:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial neural networks&lt;/a&gt; typically use exemplar data as inputs for the purpose of training, or adjusting the weights of its internal connections, to accurately classify them within a certain error range. The network is then applied to unknown data to classify them. &lt;a href=&quot;https://en.wikipedia.org/wiki/Edge_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edge detection&lt;/a&gt; filters are just blind operators that transform input data regardless of how it can be classified. There is no training, so any intelligence exists only in the mind of the filter developer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A CNN could be trained to be an effective Sobel (edge detection) filter, as described &lt;a href=&quot;http://www.worldacademicunion.com/journal/1746-7659JIC/jicvol5no1paper01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this paper&lt;/a&gt;, but a Sobel filter couldn't be an effective learning algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Training neural networks is more non-deterministic, with outputs depending on what data they are trained with and potentially even the operational computations that are used for classification. Applying filters is typically deterministic, i.e. they will transform the same data exactly the same way if applied twice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One succinct way of expressing the biggest difference is: a cellular neural network is looking for a function, while a Sobel filter &lt;em&gt;is&lt;/em&gt; a function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that there are types of neural networks called Convolutional Neural Networks, which can use Sobel and other filters in their input layers, as described &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Though, these are neither of the things you are asking about. :)&lt;/p&gt;&#xA;" OwnerUserId="46" LastEditorUserId="46" LastEditDate="2016-11-23T14:35:20.330" LastActivityDate="2016-11-23T14:35:20.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2365" PostTypeId="2" ParentId="2356" CreationDate="2016-11-21T22:05:17.057" Score="1" Body="&lt;p&gt;Not only wouldn't a strong AI which came into existence today have the rights a human has, or any rights (see these discussions of the implementation of regulation for weak AIs at: &lt;a href=&quot;https://www.whitehouse.gov/blog/2016/05/03/preparing-future-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;The White House&lt;/a&gt; and &lt;a href=&quot;http://apps.americanbar.org/dch/committee.cfm?com=ST248008&quot; rel=&quot;nofollow noreferrer&quot;&gt;The American Bar Association&lt;/a&gt;),  but it seems unlikely the first one will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observing that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Having rights implies that there are restrictions, which means there would have to be a system of control. However the &lt;a href=&quot;http://en.wikipedia.com/wiki/AI_control_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;control problem in AI&lt;/a&gt; is still unsolved.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Even assuming that problem is solvable, an AGI would then have to appear equivalent to natural humans. They don't yet (see &lt;a href=&quot;https://www.theguardian.com/technology/2014/jun/09/scientists-disagree-over-whether-turing-test-has-been-passed&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test Passed?&lt;/a&gt;), and even after passing equivalence tests, are unlikely to remain that way, per the &lt;a href=&quot;http://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Singularity Hypothesis&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Further, if one or more AGIs were to be human-equivalent long enough to desire rights, lawmakers (in the US) would have to re-interpret the definition of personhood and grant them rights, as they did for &lt;a href=&quot;https://en.wikipedia.org/wiki/Corporate_personhood&quot; rel=&quot;nofollow noreferrer&quot;&gt;corporations in 1886&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="46" LastActivityDate="2016-11-21T22:05:17.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2366" PostTypeId="1" AcceptedAnswerId="2560" CreationDate="2016-11-22T00:10:21.533" Score="6" ViewCount="2335" Body="&lt;p&gt;If someone wants to develop a &lt;strong&gt;basic AI&lt;/strong&gt; with some code modules,Let us say the AI just has to provide an action when stimulated in a certain situation based on its previous understanding of situations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of at least 3 of such components:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Real-time Understanding/Learning:&lt;/strong&gt; Using Deep Learning/ConvNets, Supervised/Unsupervised.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Logical Decision-Making:&lt;/strong&gt; Calculating the results of various decisions when applied on current situation based on previous understanding and choosing the most appropriate one logically.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Action/Reaction:&lt;/strong&gt; Acting precisely in the new situation according to the decision-made.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="3768" LastEditorUserId="1581" LastEditDate="2016-11-23T21:27:18.617" LastActivityDate="2018-12-08T23:15:27.217" Title="What are the most important components in the algorithm of a minimum AI?" Tags="&lt;algorithm&gt;&lt;ai-design&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="4" ClosedDate="2020-12-31T18:10:27.383" ContentLicense="CC BY-SA 3.0" />
  <row Id="2367" PostTypeId="1" CreationDate="2016-11-22T06:06:03.117" Score="0" ViewCount="3481" Body="&lt;p&gt;Let's say I have a string &quot;America&quot; and I want to convert it into a number to feed into a machine learning algorithm. If I use two digits for each letter, e.g. A = 01, B = 02 and so on, then the word &quot;America&quot; will be converted to &lt;code&gt;01XXXXXXXXXX01&lt;/code&gt; (10&lt;sup&gt;11&lt;/sup&gt;). This is a very high number for a &lt;code&gt;long int&lt;/code&gt;, and many words longer than &quot;America&quot; are expected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I deal with this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggest an algorithm for efficient and meaningful conversions.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastEditorUserId="75" LastEditDate="2016-11-28T17:14:51.833" LastActivityDate="2016-11-28T17:14:51.833" Title="How to convert string to number and number to string efficiently?" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2368" PostTypeId="2" ParentId="2367" CreationDate="2016-11-22T09:55:46.263" Score="1" Body="&lt;p&gt;What are you trying to achieve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to encode it to some integer use hash table. If you are using something like linear regression or neural network it would be better to use dummy features (one-hot encoding). So for your dictionary of 5 words (&quot;America&quot;, &quot;Brazil&quot;, &quot;Chile&quot;, &quot;Denmark&quot;, &quot;Estonia&quot;) you get 5 features (x1, x2, x3, x4, x5) which indicate if some word is equal to one in dictionary. So &quot;Brazil&quot; is represented by (0,1,0,0,0), &quot;Germany&quot; is (0,0,0,0,0). Number of features grows with number of words in dictionary making some features practically useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using decision trees you don't need to convert string to integer unless specific algorithm asks you to do so. Again, use hash table to do it. In R you can use factor() function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you convert your string to integers and use it as single feature (&quot;America&quot; - 123, &quot;Brazil&quot; - 245), algorithm will try to find patterns in it by comparing numbers but may fail to recognize specific countries.&lt;/p&gt;&#xA;" OwnerUserId="3690" LastActivityDate="2016-11-22T09:55:46.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2369" PostTypeId="2" ParentId="2367" CreationDate="2016-11-22T10:43:50.240" Score="1" Body="&lt;p&gt;This depends a lot on what you want to achieve, but if you aim to generalise beyond the words encountered in your training data, you should consider using something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In word2vec semantically similar words are represented by similar vectors and what's more, semantic differences translate into geometrical differences. To overuse a standard example: vec(Paris)-vec(France)+vec(Italy)=vec(Rome).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These relationships allow the network to generalise to completely new content.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-11-22T10:43:50.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2370" PostTypeId="1" CreationDate="2016-11-22T16:28:02.987" Score="3" ViewCount="96" Body="&lt;p&gt;I'm trying to understand Boltzmann machines. Tutorials explain it with two formulas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logistic function for the probability of single units:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(unit=1)=\frac{1}{1+e^{-\sum_{x}wx } }$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and, when the machine is running, every state of the machine goes to the probability:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(State= state\ with\ energy\ E_i )=\frac{e^{-E_i}}{\sum_i e^{-E_i}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the state depends on the units, and then, if I understand correctly, the second formula is a consequence of the first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how can it be the proof that the distribution of &lt;span class=&quot;math-container&quot;&gt;$p(state)$&lt;/span&gt; is a consequence of &lt;span class=&quot;math-container&quot;&gt;$p(unit)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2189" LastEditorUserId="2444" LastEditDate="2020-05-18T12:08:38.850" LastActivityDate="2020-05-18T12:08:41.830" Title="How is the distribution of the state related to the distribution of the units in Boltzmann machines?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;boltzmann-machine&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2371" PostTypeId="1" CreationDate="2016-11-22T21:58:31.003" Score="6" ViewCount="122" Body="&lt;p&gt;I installed a local running instance of the &lt;a href=&quot;http://conceptnet5.media.mit.edu/&quot; rel=&quot;noreferrer&quot;&gt;ConceptNet5&lt;/a&gt; knowledgebase in an elasticsearch server. I used this data to implement the so-called &quot;&lt;a href=&quot;https://de.wikipedia.org/wiki/Analogietechnik&quot; rel=&quot;noreferrer&quot;&gt;Analogietechnik&lt;/a&gt;&quot; (a creativity technique to solve a problem from the perspective of another system) as an algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technique works as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose a Feature of a System&lt;/li&gt;&#xA;&lt;li&gt;Find Systems who have this feature also&lt;/li&gt;&#xA;&lt;li&gt;Solve the problem from the perspective of these other systems&lt;/li&gt;&#xA;&lt;li&gt;Apply the found solutions to the issue&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As an example is here the problem of marketing a shopping mall: A Shopping mall has many rooms and floors (1). A museum has also many rooms and floors (2). How are museums marketed? They present many pictures or sculptures (3). We could use our rooms and floors to decorate them with pictures and sculptures (4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the idea to implement that as an artifically intelligent algorithm was not far. However, I feel a little bit overwhelmed by the amount of methods that exist out there. Neural Networks, Bayesian Interference and so on... My current experience doesn't go further than simple machine learning like kMeans-Clustering for example. Do you think it would be very hard to find a solution for this problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking of a console application, where you can enter a conceptualized problem like &quot;methods for creative writing&quot;, for example, and it uses the above method to find possible solutions of the issue. Of course no solution with extensive depth, more something like basic ideas derived from the knowledge database I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets take as an example a console application where someone asks &quot;how to write a novel&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It should find out first that the system all is about is in the term &quot;novel&quot;. To find a feature of that system it just searches concepts containing that term: it finds out &quot;Novel is a story&quot; So thats a feature.&lt;/li&gt;&#xA;&lt;li&gt;Which systems are also stories? A good concept it should find is e.g. &quot;Plot is a story&quot;. (Of course only when I am selecting the search results manually)--&gt; &lt;strong&gt;How to find best concepts of a list when not knowing which fits best?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;It should then find out that a plot is written using a storyline: &quot;storyline is a plot&quot;&lt;/li&gt;&#xA;&lt;li&gt;One possible answer of the AI would in this case be: &quot;By writing a storyline&quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Do you know some helpful libraries, algorithms or other resources that might help me? I know this is not an easy thing to program, but you might agree that its highly interesting.&lt;/p&gt;&#xA;" OwnerUserId="3788" LastEditorUserId="46" LastEditDate="2016-11-23T10:39:54.233" LastActivityDate="2021-04-04T23:02:00.370" Title="Using ConceptNet5 to find similar systems to solve specific problems?" Tags="&lt;problem-solving&gt;&lt;world-knowledge&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2372" PostTypeId="2" ParentId="2286" CreationDate="2016-11-22T22:02:05.037" Score="3" Body="&lt;p&gt;Suppose you have data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;color  height  quality&#xA;=====  ======  =======&#xA;green  tall    good&#xA;green  short   bad&#xA;blue   tall    bad&#xA;blue   short   medium&#xA;red    tall    medium&#xA;red    short   medium&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To calculate the entropy for quality in this example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X  = {good, medium, bad}&#xA;x1 = {good}, x2 = {bad}, x3 = {medium}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Probability of each x in X:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p1 = 1/6 = 0.16667&#xA;p2 = 2/6 = 0.33333&#xA;p3 = 3/6 = 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for which logarithms are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;log2(p1) = -2.58496&#xA;log2(p2) = -1.58496&#xA;log2(p3) = -1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and therefore entropy for the set is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H(X) = - (0.16667 * -2.58496) - (0.33333 * -1.58496) - (0.5 * -1.0)&#xA;     = 1.45915&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;by the formula in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remaining tasks are to iterate this process for each attribute to form the nodes of the tree.&lt;/p&gt;&#xA;" OwnerUserId="46" LastActivityDate="2016-11-22T22:02:05.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2373" PostTypeId="1" AcceptedAnswerId="2374" CreationDate="2016-11-23T02:09:13.923" Score="5" ViewCount="154" Body="&lt;p&gt;Given the advantage AI already has over human intelligence, one could imagine a relatively weak strong-AI (barely human intelligence) still outperforming a segment of the human scientist population in terms of scientific discoveries per year (or hour).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will AIs be doing most of the science in 50 years?&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2016-11-25T10:59:40.203" Title="Will AIs make most of the scientific discoveries in 50 years?" Tags="&lt;control-problem&gt;&lt;world-knowledge&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2374" PostTypeId="2" ParentId="2373" CreationDate="2016-11-23T13:33:52.427" Score="3" Body="&lt;p&gt;According to &lt;a href=&quot;http://www.kurzweilai.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ray Kurzweil&lt;/a&gt;, a prominent AI researcher, yes. In his book &lt;em&gt;The Singularity is Near&lt;/em&gt; he predicts that AIs will take over developing other AIs in about 30 years, after which human intelligence will become marginalised.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2016-11-23T13:33:52.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2375" PostTypeId="2" ParentId="2353" CreationDate="2016-11-23T15:14:34.647" Score="0" Body="&lt;p&gt;Input -&gt; Prediction -&gt; Output -&gt; Input -&gt; Prediction -&gt; Output -&gt; Input -&gt; ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AGI can easily determine which input is true/real. It will use the same method which every organism uses: any input is true and real, unless you misidentified some other stuff as &quot;input&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would define input as: what crosses the boundary and enters your mind from outside of your mind. The minimum hardwired check is to make sure that signals generated inside a mind are not misidentified as coming from outside (aka &quot;I hear voices&quot;). That's all. This is where the blockchain of truth begins and where it ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An Internet article? The input to AI is rather: one of AI's network interfaces received many bytes. Once it's verified they are from the network, and not imaginary, they cannot be unreal or untrue in any meaningful way. By that definition of input, it is in fact the &lt;em&gt;only&lt;/em&gt; thing we can be sure is true and real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course AI will likely form hypotheses regarding these bytes that happen to contain ASCII strings like &quot;Trump&quot;, &quot;John Smith&quot;, &quot;ice balls on Siberian beaches&quot;. Then AI will hopefully make predictions based on these hypotheses, maybe interact, maybe get some new input, reject the hypothesis and make a new one, rinse and repeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first hypothesis will be super-naive, but the hundredth, the thousandth?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you end this process prematurely - maybe for lack of processing power -  you will get something you called a &quot;&lt;em&gt;belief&lt;/em&gt;&quot;. (Like a belief that some emotional web page might actually reveal a significant truth about our political system.) That &lt;em&gt;belief&lt;/em&gt; is a synonym of &quot;tired with trying new hypotheses, will stick to this one&quot;. Typical human thing. AI will have less of that, I hope, due to having much much more processing capabilities. AI will stick less to the high-school-level truth that you should assign great credibility to statements written in a form of a newspaper article, it will hopefully form more and more generations of hypotheses, and check them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In effect AI will depend less on &lt;em&gt;believing&lt;/em&gt; various statements generated in the outside world.&lt;/p&gt;&#xA;" OwnerUserId="3803" LastActivityDate="2016-11-23T15:14:34.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2376" PostTypeId="2" ParentId="2277" CreationDate="2016-11-23T19:39:36.487" Score="4" Body="&lt;p&gt;I have considered much of the responses here, and I would suggest that most people here have missed the point when answering the question about emotions.&lt;/p&gt;&#xA;&lt;p&gt;The problem is, scientists keep looking for a single solution as to what emotions are. This is akin to looking for a single shape that will fit all different-shaped slots.&lt;/p&gt;&#xA;&lt;p&gt;Also, what is ignored is that animals are just as capable of emotions and emotional states as we are:&lt;/p&gt;&#xA;&lt;p&gt;When looking on Youtube for insects fighting each other, or competing or courting, it should be clear that simple creatures experience them too!&lt;/p&gt;&#xA;&lt;p&gt;When I challenge people about emotions, I suggest to them to go to Corinthians 13 - which describes the attributes of love. If you consider all those attributes, one should notice that an actual &amp;quot;feeling&amp;quot; is not required for fulfilling any of them.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, the suggestion that a psychopath lacks emotions, and so he commits crimes or other pursuits outside of &amp;quot;normal&amp;quot; boundaries is far from true, especially when one considers the various records left to us from court cases and perhaps psychological evaluation - which show us that they do act out of &amp;quot;strong&amp;quot; emotions.&lt;/p&gt;&#xA;&lt;p&gt;It should be considered that a psychopath's behaviour is motivated out of negative emotions and emotional states with a distinct lack of or disregard of morality and a disregard of conscience. Psychopaths &amp;quot;enjoy&amp;quot; what they do.&lt;/p&gt;&#xA;&lt;p&gt;I am strongly suggesting to all that we are blinded by our reasoning, and by the reasoning of others.&lt;/p&gt;&#xA;&lt;p&gt;Though I do agree with the following quote mentioned before: -&lt;/p&gt;&#xA;&lt;p&gt;Dave H. wrote:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;From a computational standpoint, emotions represent a global state that influences a lot of other processing. Hormones etc. are basically&#xA;just implementation. A sentient or sapient computer certainly could&#xA;experience emotions, if it was structured in such a way as to have&#xA;such global states affecting its thinking.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, his reasoning below it (that quote) is also seriously flawed.&lt;/p&gt;&#xA;&lt;p&gt;Emotions are both active and passive: They are triggered by thoughts and they trigger our thoughts; Emotions are a mental state and a behaviourial quality; Emotions react to stimuli or measure our responses to them; Emotions are independant regulators and moderators; Yet they provoke our focus and attention to specific criteria; and they help us when intuition and emotion agree or they hinder us when conscience or will clash.&lt;/p&gt;&#xA;&lt;p&gt;A computer has the same potential as us to feel emotions, but the skill of implementing emotions is much more sophisticated than the one solution fits all answer people are seeking here.&lt;/p&gt;&#xA;&lt;p&gt;Also, if anyone argues that emotions are simply &amp;quot;states&amp;quot; where a response or responses can be designed around it, really does not understand the complexity of emotions; the &amp;quot;freedom&amp;quot; emotions and thoughts have independently of each other; or what constitutes true thought!&lt;/p&gt;&#xA;&lt;p&gt;Programmers and scientists are notorious for &amp;quot;simulating&amp;quot; the real experiences of emotions or intelligence, without understanding the intimate complexities; Thinking that in finding the perfect simulation they have &amp;quot;discovered&amp;quot; the real experience.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Psi-theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Psi-theory&lt;/a&gt; seems to adequately give a proper understanding of the matter.&lt;/p&gt;&#xA;&lt;p&gt;So I would say that the simulation of emotional states &amp;quot;is&amp;quot; equivalent to experiencing emotions, but those emotional states are far more complex than what most realize.&lt;/p&gt;&#xA;" OwnerUserId="3808" LastEditorUserId="32410" LastEditDate="2021-05-09T11:41:24.427" LastActivityDate="2021-05-09T11:41:24.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2377" PostTypeId="2" ParentId="2353" CreationDate="2016-11-23T20:31:29.980" Score="2" Body="&lt;p&gt;I strongly disagree with all of the aforementioned answers for this reason: -&#xA;If we, as humans can be fooled and disceived by what &quot;we&quot; consider a good sources of news, how can an artificially intelligent computer have any chance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the challenge would be that an AI would have to be able to &quot;test&quot; a source of information against a known medium in order to &lt;em&gt;get to the truth&lt;/em&gt;. This is a far different dynamic set of circumstances than what has been touted above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if it was claimed by a woman that a man raped her - which was not reported to the police - it is not enough to compare one person's statements to another in order to determine truth. This is because collusion, influenced or coherced third parties, mistaken perceptions and false beliefs would give false positives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if an AI could establish from her statement that on the day she claimed to have been raped, that the alleged assailant was incapacitated while in her company, until she left his home, because the police report stated that she was upset with the assailant because he was asleep because of drugs during her whole stay. But, this police report comes from an independent source who states, Mr. &quot;x&quot; was asleep that day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing a strict textual check is not going to give the correct answers. analysing her friends and associattes chatter could also confirm a false report as being true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, an AI has to have the ability to &quot;test&quot; written reports outside of the criteria of what was spoken.&lt;/p&gt;&#xA;" OwnerUserId="3809" LastActivityDate="2016-11-23T20:31:29.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2378" PostTypeId="2" ParentId="2373" CreationDate="2016-11-24T00:12:22.560" Score="0" Body="&lt;p&gt;I don't think so, it is not the first but actually the third wave of neural networks. It's doing better than earlier two as we have much more amount of data as well as computational power now. &#xA;Take a look at this video .....&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=furfdqtdAvc&amp;amp;t=39s&quot; rel=&quot;nofollow noreferrer&quot;&gt;According to Douglas Adams’s famous “Hitchhiker’s Guide to the Galaxy” after 7.5 millions years of work the “Deep Thought” computer categorically found out that 42 is the “Answer to the Ultimate Question of Life, the Universe, and Everything” (although unfortunately, no one knows exactly what that question was).&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3812" LastActivityDate="2016-11-24T00:12:22.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2380" PostTypeId="2" ParentId="2373" CreationDate="2016-11-24T23:41:50.683" Score="0" Body="&lt;p&gt;Current Computing relates 0 or 1 to another 0 or 1 with layers upon layers of building blocks built on this relationship.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In future AI will relate A to B, be they numbers, patterns of coded instructions or some other form of more complicated constructs at a hardware  (or closer to it) level than is currently possible and, due to the inherent perfect recall and potentially massive memory storage of AI they will most definitely be bringing together and relating a vastly more broad and organised collection of knowledge than it is possible for any human to even contemplate consciously. There are some ideas that would argue that point; universal mind, spiritual revelation and morphic resonance which I do personally agree with to some degree and can imagine being particularly difficult to represent in a computational format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pattern spotting and relating, organising and computing potentials... computers are already better at all these things than most people try to be. It will not be long, i think, before they can &quot;invent&quot; something &quot;new&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are already attempts at AI in specialised fields of knowledge, human speech, various games, medical diagnostics and learning  etc. It will be when these specialised AIs can &quot;compare notes&quot; about the various methodologies that have been the most productive or rewarding, in whichever form these take for them, and accordingly update their own ontologies that the true explosion of &quot;Intelligence&quot; will occur.&lt;/p&gt;&#xA;" OwnerUserId="3598" LastEditorUserId="3598" LastEditDate="2016-11-25T10:59:40.203" LastActivityDate="2016-11-25T10:59:40.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2381" PostTypeId="1" AcceptedAnswerId="5301" CreationDate="2016-11-25T10:03:29.947" Score="9" ViewCount="1242" Body="&lt;p&gt;&lt;a href=&quot;http://opencog.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt; is an open source AGI-project co-founded by the mercurial AI researcher &lt;a href=&quot;https://en.wikipedia.org/wiki/Ben_Goertzel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ben Goertzel&lt;/a&gt;. Ben Goertzel writes a lot of stuff, some of it &lt;a href=&quot;http://multiverseaccordingtoben.blogspot.de/2010/11/psi-debate-continues-goertzel-on.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;really&lt;/a&gt; &lt;a href=&quot;http://multiverseaccordingtoben.blogspot.de/2016/10/semrem-search-for-extraterrestrial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;whacky&lt;/a&gt;. Nonetheless, he is clearly very intelligent and has thought deeply about AI for many decades. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the general ideas behind &lt;a href=&quot;http://wiki.opencog.org/w/OpenCog_Prime&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt;? Would you endorse it as a insightful take on AGI? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm especially interested in whether the general framework still makes sense in the light of recent advances.  &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2019-04-29T16:48:00.293" LastActivityDate="2019-08-06T20:15:43.323" Title="What are general ideas behind OpenCog?" Tags="&lt;ai-design&gt;&lt;agi&gt;&lt;open-cog&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2384" PostTypeId="2" ParentId="2349" CreationDate="2016-11-25T23:02:48.910" Score="2" Body="&lt;p&gt;There are a variety of possible things that could be wrong, but let me give you some potentially useful information.&lt;/p&gt;&#xA;&lt;p&gt;Neural networks with ReLU activation functions are Turing complete for a computation with on order as many steps as the network contains nodes - for a recurrent network (an RNN), that means the same level of turing completeness as any finite computer. In other words, for any function/algorithm that you want to compute, you can devise a neural network, potentially recurrent, that will approximate/compute it.&lt;/p&gt;&#xA;&lt;p&gt;As an example, suppose that we want to compute the &lt;a href=&quot;https://en.wikipedia.org/wiki/NOR_gate&quot; rel=&quot;nofollow noreferrer&quot;&gt;NOR&lt;/a&gt; function, which can be used to implement a Turing machine.  We can do it with the following neural network with a ReLU activation function.&lt;/p&gt;&#xA;&lt;p&gt;Let the input be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ W = \begin{bmatrix}x_1 \ \ x_2 \end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;the weight matrix be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ W = \begin{bmatrix} -20 \\ -20 \end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the bias be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ b = \begin{bmatrix} 1 \end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the ReLU unit (or neuron) performs the following operation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$o = \max(Wx + b, 0)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, &lt;span class=&quot;math-container&quot;&gt;$o = 1$&lt;/span&gt; only when both &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, otherwise, it's always &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, gradient descent is a finicky way to search for RNNs. There are a wide variety of ways that it might have been failing. In general, once you have &lt;em&gt;very thoroughly checked your gradient&lt;/em&gt;, I'd make sure to use Adam as the optimizer and then play with the hyperparameters endlessly until I find an incantation that works.&lt;/p&gt;&#xA;&lt;p&gt;For further reading on general understanding of this level of deep learning's capability limitations, I'd recommend this blog post by Ilya Sutskever, now an OpenAI researcher: &lt;a href=&quot;http://yyue.blogspot.com/2015/01/a-brief-overview-of-deep-learning.html?m=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yyue.blogspot.com/2015/01/a-brief-overview-of-deep-learning.html?m=1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3856" LastEditorUserId="3856" LastEditDate="2021-01-18T05:13:16.110" LastActivityDate="2021-01-18T05:13:16.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2385" PostTypeId="2" ParentId="2353" CreationDate="2016-11-25T23:29:05.760" Score="0" Body="&lt;p&gt;While the experiment I link here is a very narrow awareness, it is as such: &lt;a href=&quot;http://www.sciencealert.com/a-robot-has-just-passed-a-classic-self-awareness-test-for-the-first-time&quot; rel=&quot;nofollow noreferrer&quot;&gt;A robot has just passed a classic self-awareness test for the first time&lt;/a&gt;. If the agent can prove something to itself, we can then say it &quot;Knows.&quot; Of course the level of awareness you're asking about is very tricky.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, it can't know that what it's experiencing is real with absolute certainty because sensory of any kind can be falsified. Do you know what is true/real? You think you do but can you prove it? No. Awareness is subjective.&lt;/p&gt;&#xA;" OwnerUserId="3861" LastEditorUserId="8" LastEditDate="2016-11-26T14:40:15.097" LastActivityDate="2016-11-26T14:40:15.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2386" PostTypeId="2" ParentId="2381" CreationDate="2016-11-25T23:47:07.920" Score="4" Body="&lt;p&gt;While my knowledge of OpenCog is very limited, you could say that yes, it does still make sense and it is insightful. I'm not certain regarding all of the components of OpenCog but I do know that at least one component is relevant (I think it's part of the MOSIS component).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This component is very similar to Numenta's hierarchical temporal memory which is based more on computational neuroscience than plain math; however, I would consider Nupic a more relevant project in terms of neroscience though both are attempting to emulate components of the brain. In my opinion, such projects are far more impressive than what's going on with typical convolutional neural nets, RNNs, etc. which are too loosely related to what goes on in the brain to be said to be computational neuroscience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's not to say that things like ANNs, GAs, etc etc are useless for AGI. We don't really know since we don't have an example of one.&lt;/p&gt;&#xA;" OwnerUserId="3861" LastActivityDate="2016-11-25T23:47:07.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2387" PostTypeId="2" ParentId="2371" CreationDate="2016-11-26T06:05:00.057" Score="0" Body="&lt;p&gt;Hierarchical Temporal Memory should help with this. You would have to encode the text data into SDRs. You would then have a coincidence detector. Could you get the right information back out in the way you're trying to? I think so. I'm not fully learned in HTMs yet but check out &lt;a href=&quot;http://numenta.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nupic&lt;/a&gt; (Open source).&lt;/p&gt;&#xA;" OwnerUserId="3861" LastActivityDate="2016-11-26T06:05:00.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2388" PostTypeId="2" ParentId="2277" CreationDate="2016-11-26T08:05:33.167" Score="2" Body="&lt;p&gt;This question is more the province of philosophy of mind than of AI, here are some detailed answers to your question from the philosophy SE: &lt;a href=&quot;https://philosophy.stackexchange.com/a/35824/13808&quot;&gt;Is simulating emotions the same as experiencing emotions?&lt;/a&gt;, and &lt;a href=&quot;https://philosophy.stackexchange.com/a/34244/13808&quot;&gt;What is the problem with physicalism?&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the record, the accepted answer (by Siri) to the question is not entirely correct (The position in that answer corresponds roughly to &lt;a href=&quot;https://philosophy.stackexchange.com/a/34682/13808&quot;&gt;John Searle's view&lt;/a&gt; on the question, and his is a minority view): Dualists would argue that even with a perfect replication down to the chemical level of brain interactions, an AI still wouldn't experience emotions, as it lacks the purely mental substance/properties that make a mind and not a machine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the completely opposite side of the spectrum, functionalists would answer that such a perfect replication is overkill: even a suitably programmed digital computer can experience emotion, particularly if one equips it with higher-order and self-referential states.  &lt;/p&gt;&#xA;" OwnerUserId="2306" LastEditorUserId="-1" LastEditDate="2017-04-13T12:42:22.960" LastActivityDate="2016-11-28T02:38:33.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2390" PostTypeId="2" ParentId="2366" CreationDate="2016-11-26T16:34:45.107" Score="4" Body="&lt;p&gt;Some good places to start would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Cognitive_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;cognitive architectures&lt;/a&gt; and as mentioned in another answer &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;intelligent agents&lt;/a&gt;. The question is broad but you definitely want to look into &lt;a href=&quot;http://msl.cs.uiuc.edu/~lavalle/cs397/&quot; rel=&quot;nofollow noreferrer&quot;&gt;planning &amp;amp; decision making&lt;/a&gt;. You might also want to check out the &lt;a href=&quot;http://numenta.org/resources/presentations/2014%20Sensory%20Motor%20Integration%20in%20HTM%20Theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;L5 and L6 layers&lt;/a&gt; of Hierarchical Temporal Memory (As in &lt;a href=&quot;http://numenta.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nupic&lt;/a&gt;) as it relates to feedback, behavior and attention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were you I'd aim for more cognitive solutions (I realize that term is a bit ambiguous itself when we talk about machines). There's also new AI initiative going on involving probabilistic programming. See &lt;a href=&quot;https://probmods.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probabilistic Models of Cognition&lt;/a&gt; made by Goodman (Stanford University) and Tenenbaum (MIT) or &lt;a href=&quot;http://www.robots.ox.ac.uk/~fwood/anglican/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anglican&lt;/a&gt; made by Wood (University of Oxford) et al.&lt;/p&gt;&#xA;" OwnerUserId="3861" LastEditorUserId="3855" LastEditDate="2016-11-28T11:30:03.113" LastActivityDate="2016-11-28T11:30:03.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2392" PostTypeId="1" CreationDate="2016-11-27T00:19:27.033" Score="3" ViewCount="485" Body="&lt;p&gt;Is there any good app for iOS or Android to create, train and run neural networks? I know there is NeuralMesh for the web, but I want something similar offline.&lt;/p&gt;&#xA;" OwnerUserId="3872" LastEditorUserId="2444" LastEditDate="2020-05-16T23:19:53.317" LastActivityDate="2020-05-16T23:19:53.317" Title="Is there any good app for iOS or Android to create, train and run neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2020-05-22T18:12:15.643" ContentLicense="CC BY-SA 4.0" />
  <row Id="2393" PostTypeId="2" ParentId="2277" CreationDate="2016-11-27T01:36:32.790" Score="1" Body="&lt;p&gt;You first need to express emotions, you can do that without the aid of AI, and then you need someone to perceive that expression and empathize with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If no one is there to see it, or if I am psychopath, I would probably say it doesn't have emotions. and for that, it is irrelevant/subjective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can empathize with characters in movies who &quot;act&quot; emotions, then you get my point.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2016-11-27T01:36:32.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2394" PostTypeId="2" ParentId="2356" CreationDate="2016-11-27T01:53:19.920" Score="2" Body="&lt;p&gt;No matter what rights it gets (as a company), it will still lack the right of not getting liquefied and all its properties transferred back to natural persons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is of course if no laws are changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To change the laws you will need to convince people that this machine is more &quot;life&quot; worthy than intelligent animals, and hope that people will deal with them better than they did with dolphins and chimps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it, machines can easily get the same or better rights then companies, but will always be under the mercy of the less intelligent man. (that is if things went peacefully :) )&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2016-11-27T01:53:19.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2395" PostTypeId="2" ParentId="2392" CreationDate="2016-11-27T10:02:45.683" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;http://neuroph.sourceforge.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuroph&lt;/a&gt; to develop and train your network and add in app via NetBeans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;check this link&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://neuroph.sourceforge.net/tutorials/android_image_recognition_using_neuroph.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Creating Android image recognition application using NetBeans and Neuroph&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3885" LastEditorUserId="3771" LastEditDate="2016-11-28T05:30:20.157" LastActivityDate="2016-11-28T05:30:20.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2396" PostTypeId="2" ParentId="2330" CreationDate="2016-11-27T16:38:38.143" Score="1" Body="&lt;p&gt;The human brain contains billions of neurons, which means we won't be making one tomorrow. However, technology tends to advance in an exponential manner, and that may soon be a real possibility. Also, the idea of making an artificial human brain would not only take more neurons than a current average computer could process, or we could make outside of computers, but we also need an understanding of the human brain. There is only one animal with neurons that we have completed a full connectome of and that is the Caenorhabditis elegans (roundworm) and it has less than 500 neurons. It may be a while before we actually make a human brain, but within 30 years is a reasonable estimation with the rate that technology improves now.&lt;/p&gt;&#xA;" OwnerUserId="3887" LastActivityDate="2016-11-27T16:38:38.143" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2397" PostTypeId="2" ParentId="2367" CreationDate="2016-11-27T16:41:09.420" Score="0" Body="&lt;p&gt;You shouldn't use a single number for the word, perhaps a number for each letter. Since B isn't the midpoint of A and C, the numbers really shouldn't be 1, 2, 3, etc. One large but effective way of converting is the letter a is 10000000000000000000000000 such that there are 26 digits, and each digit  is a letter, so 0000100000... would be E.&lt;/p&gt;&#xA;" OwnerUserId="3887" LastActivityDate="2016-11-27T16:41:09.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2398" PostTypeId="1" CreationDate="2016-11-27T16:53:45.910" Score="11" ViewCount="739" Body="&lt;p&gt;I am researching &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cellular_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cellular Neural Networks&lt;/a&gt;&lt;/strong&gt; and have already read &lt;strong&gt;Chua&lt;/strong&gt;'s two articles (&lt;strong&gt;1988&lt;/strong&gt;). In cellular neural networks, a cell is only in relation with its neighbors. So its is easy to use it for real time image processing. Image processing is performed with only &lt;strong&gt;19 numbers&lt;/strong&gt; (two 3x3 matrix called A and B and one bias value). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if we can call cellular neural networks &lt;strong&gt;&lt;em&gt;neural networks&lt;/em&gt;&lt;/strong&gt;, because there is no learning algorithm.  They are neither &lt;strong&gt;supervised&lt;/strong&gt; nor &lt;strong&gt;unsupervised&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3763" LastEditorUserId="1671" LastEditDate="2018-11-08T01:22:58.387" LastActivityDate="2019-05-07T19:04:50.883" Title="Are Cellular Neural Networks one type of Neural Networks?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2399" PostTypeId="2" ParentId="2277" CreationDate="2016-11-27T19:20:44.937" Score="1" Body="&lt;p&gt;IMHO&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Definitely, yes!&lt;/strong&gt;&#xA;Everything that person feels (physically or mentally) can be discovered by chemical signals processing in his body or brain. If we understand the policy and nature of such signals, we can program it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of pseudo-psychology and psychology works on this sphere, if you interested, I can suggest you:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) &lt;strong&gt;Cognitive Psychology (Robert L. Solso)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;describes cognitive apparat of human's mind in a simple words;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2) &lt;strong&gt;The Psychology of Emotions (Carroll E. Izard)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;thorougly describes every kind of emotion by its looking on the human (both child and adult) face, low-level cognitive mechanism, related or adjacent emotions;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;3) Books by &lt;strong&gt;Paul Ekman (&quot;Telling Lies&quot;, &quot;Emotions Revealed&quot;,&#xA;  &quot;Unmasking the Face&quot;)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;practical detecting of human emotions by microexpressions language on face and body.&lt;/p&gt;&#xA;" OwnerUserId="3891" LastActivityDate="2016-11-27T19:20:44.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2400" PostTypeId="1" CreationDate="2016-11-28T04:24:20.457" Score="-3" ViewCount="307" Body="&lt;p&gt;Could an Artificial Intelligence be able to interact (see, talk, etc.) with someone even when there's no power cord connected to the machine it's running on? Might it find some way to generate its own electricity to power that computer?&lt;/p&gt;&#xA;" OwnerUserId="3896" LastEditorUserId="2444" LastEditDate="2019-10-22T20:39:32.320" LastActivityDate="2019-10-22T20:39:32.320" Title="Is it possible for an AI to work in a computer without the power cord being plugged in?" Tags="&lt;theory&gt;&lt;automation&gt;" AnswerCount="5" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="2401" PostTypeId="2" ParentId="2400" CreationDate="2016-11-28T05:37:05.740" Score="3" Body="&lt;p&gt;If your &quot;AI&quot; doesn't have the ability to move and perform physical manipulations in the real world then there is no way it could do something like this.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-11-28T05:37:05.740" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2402" PostTypeId="2" ParentId="2400" CreationDate="2016-11-28T13:35:27.117" Score="1" Body="&lt;p&gt;If the computer is unplugged, the AI is clinically dead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can have a RaspberryPi on solar cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tesla car is an AI moving and seeing while unplugged (from wall). but you have to have some sort of energy. For AI that lacks metabolism, solar/wind energy can be an alternative.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2016-11-28T13:35:27.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2403" PostTypeId="1" CreationDate="2016-11-28T13:55:06.963" Score="4" ViewCount="1193" Body="&lt;p&gt;I am new to deep learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features of the object (using Autoencoders). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to create a network with layers that account for varying dimensions and orientations of the input image, or should I strictly consider a dataset containing images of uniform dimensions? What is the necessary criteria of an eligible dataset to be used for training a Deep Network in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is, I want to avoid pre-processing my dataset by normalizing it via scaling, re-orienting operations etc. I would like my network to account for the variability in dimensions and orientations. Please point me to resources for the same. &lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2016-12-30T19:57:38.110" Title="Dataset containing images of varying dimensions and orientations" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2404" PostTypeId="1" CreationDate="2016-11-28T15:18:10.110" Score="6" ViewCount="332" Body="&lt;p&gt;A single neuron is capable of forming a decision boundary between linearly seperable data. Is there any intuition as to how many, and in what configuration, would be necessary to correctly approximate a sinusoidal decision boundary?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3908" LastActivityDate="2017-01-30T16:08:39.257" Title="How many nodes/hidden layers are required to solve a classification problem where the boundary is a sinusoidal function?" Tags="&lt;neural-networks&gt;&lt;hidden-layers&gt;&lt;neurons&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2405" PostTypeId="1" CreationDate="2016-11-29T06:10:54.993" Score="9" ViewCount="8519" Body="&lt;p&gt;I am using policy gradients in my reinforcement learning algorithm, and occasionally my environment provides a severe penalty (i.e. negative reward) when a wrong move is made. I'm using a neural network with stochastic gradient descent to learn the policy. To do this, my loss is essentially the cross-entropy loss of the action distribution multiplied by the discounted rewards, where most often the rewards are positive.&lt;/p&gt;&#xA;&lt;p&gt;But how do I handle negative rewards? Since the loss will occasionally go negative, it will think these actions are very good, and will strengthen the weights in the direction of the penalties. Is this correct, and if so, what can I do about it?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;p&gt;In thinking about this a little more, SGD doesn't necessarily directly weaken weights, it only strengthens weights in the direction of the gradient and as a side-effect, weights get diminished for other states outside the gradient, correct? So I can simply set reward=0 when the reward is negative, and those states will be ignored in the gradient update. It still seems unproductive to not account for states that are really bad, and it'd be nice to include them somehow. Unless I'm misunderstanding something fundamental here.&lt;/p&gt;&#xA;" OwnerUserId="3920" LastEditorUserId="2444" LastEditDate="2020-11-01T22:38:50.010" LastActivityDate="2020-11-01T22:39:38.373" Title="How do I handle negative rewards in policy gradients with the cross-entropy loss function?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;rewards&gt;&lt;cross-entropy&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2407" PostTypeId="2" ParentId="1989" CreationDate="2016-11-29T13:56:05.997" Score="5" Body="&lt;p&gt;As Matthew Graves explained in another answer No free lunch theorem confirms the flexibility - efficiency trade-off. However, this theorem is describing a situation where you have a set of completely independent tasks. This often doesn't hold, as many different problems are equivalent in their core or at least have some overlap. Then you can do something called &quot;transfer learning&quot;, which means that by training to solve one task you also learn something about solving another one (or possibly multiple different tasks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in &lt;a href=&quot;https://arxiv.org/abs/1511.06295&quot; rel=&quot;noreferrer&quot;&gt;Policy Distillation&lt;/a&gt; by Rusu et al. they managed to &quot;distill&quot; knowledge from different expert networks into one general network which in the end outperformed each of the experts. The experts were trained for specific tasks while the generalist learned the final policy from these &quot;teachers&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3855" LastActivityDate="2016-11-29T13:56:05.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2408" PostTypeId="2" ParentId="2400" CreationDate="2016-11-29T23:23:39.177" Score="1" Body="&lt;p&gt;I assume by your use of the term &amp;quot;plugged in&amp;quot;, you are referring to an &lt;em&gt;electron based computer&lt;/em&gt; - the usual definition of, or what is commonly/popularly considered to be, a computer - which is hosting the AI. However, what your definition of &lt;em&gt;plugged in&lt;/em&gt; is unclear.&lt;/p&gt;&#xA;&lt;p&gt;If you mean literally, and &lt;em&gt;physically&lt;/em&gt;, plugged in to the wall then &lt;strong&gt;yes&lt;/strong&gt;, as &lt;a href=&quot;https://ai.stackexchange.com/questions/2400/is-it-possible-for-an-ai-to-work-in-a-computer-without-the-power-cord-being-plug#answer-2401&quot;&gt;Aus points out&lt;/a&gt;, because one can use a battery, or have a solar powered circuit, or petrol/diesel/steam/water/wind/hydrogen/etc. powered engine, running a generator/alternator which can eventually supply a voltage and current to power the circuitry. However, in these examples, you have merely moved the point of generation, from a remote power station to a locally generated source of electricity - so it is still, in effect, &lt;em&gt;plugged in&lt;/em&gt; (but to a local source). So, in that sense, the answer is &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you mean no external power source, &lt;em&gt;&lt;strong&gt;what so ever&lt;/strong&gt;&lt;/em&gt;, then the answer is most certainly &lt;strong&gt;no&lt;/strong&gt;, as, again as &lt;a href=&quot;https://ai.stackexchange.com/questions/2400/is-it-possible-for-an-ai-to-work-in-a-computer-without-the-power-cord-being-plug#answer-2401&quot;&gt;Aus has already said&lt;/a&gt;, there is no way for electrons to be pushed around the circuit, and therefore the circuit is dead&lt;sup&gt;1&lt;/sup&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you mean, as you said in your &lt;a href=&quot;https://ai.stackexchange.com/questions/2400/is-it-possible-for-an-ai-to-work-in-a-computer-without-the-power-cord-being-plug#comment-2620&quot;&gt;comment&lt;/a&gt; that the AI has created its own electrical power source - although as &lt;a href=&quot;https://ai.stackexchange.com/questions/2400/is-it-possible-for-an-ai-to-work-in-a-computer-without-the-power-cord-being-plug#answer-2401&quot;&gt;Ankur says in their answer&lt;/a&gt;, this implies that the AI has the ability for its I/O manipulate its environment&lt;sup&gt;2&lt;/sup&gt; - then as in the first paragraph, you have simply moved the point of the power source, so one can say that it is again now &lt;em&gt;plugged in&lt;/em&gt;, again to a local energy source, and so yet again the answer is &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;However, if you are &lt;em&gt;not&lt;/em&gt; referring to an electronic based computer, which is hosting the AI, but are, instead, referring to a &lt;em&gt;hypothetical biological&lt;/em&gt; computer, then the answer is probably &lt;strong&gt;yes&lt;/strong&gt; - in a sense. Although, that biological computer would still require energy input, as energy can't just be &amp;quot;magicked&amp;quot; out of thin air. It would need to be converted from one source into a from that the biological computer can use. That energy could come from a variety of sources, be it from sunlight, food,  heat (sunlight, deep sea vents, etc), so one could argue that it is still &lt;em&gt;plugged in&lt;/em&gt; to an &lt;em&gt;energy source&lt;/em&gt; and so yet again, the answer is &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To be honest, in order to give an accurate, and sensible answer, you really need to reword your question, clarify what you mean and what your definitions of &lt;em&gt;plugged in&lt;/em&gt; and &lt;em&gt;AI host&lt;/em&gt; are. Unfortunately, your question, as it stands, is rather unanswerable, in as much that it is difficult to give a definitive answer.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Footnote&lt;/h3&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;Likewise, if you mean by &lt;em&gt;plugged in&lt;/em&gt; in the conventional sense, and therefore &lt;em&gt;unplugged&lt;/em&gt; means that the electronics hosting the AI has &lt;em&gt;no power&lt;/em&gt; then the answer is &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;2&lt;/sup&gt; However, assuming that the AI &lt;em&gt;can&lt;/em&gt; manipulate its environment, in order for the AI to do able to do so, it will have be able to assemble the external power source &lt;em&gt;prior to being unplugged&lt;/em&gt;. So... if you mean &amp;quot;Can it create its own power, after it was powered off (without having had time to asemble an external power source to act as a backup)?&amp;quot; then the answer would be &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3771" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-11-29T23:54:13.347" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2409" PostTypeId="1" CreationDate="2016-11-30T11:10:32.697" Score="6" ViewCount="1245" Body="&lt;p&gt;On page 62 of &lt;a href=&quot;https://pdfs.semanticscholar.org/d16d/2c064e6be92bfde86ff56fb57f32172673be.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence and Cognitive Psychology Applications, Models&lt;/a&gt; (2010), Gabriella Daróczy writes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Psychology is one of the basic sciences of artificial intelligence (AI). The founder of the psychology is Wilhelm Wundt (1832-1920), who engaged in empirical methods, and was interested in the thinking processes during his scientific work.&lt;/p&gt;&#xA;&lt;p&gt;Psychology had two main leading schools: behaviourism, and cognitivism.&lt;/p&gt;&#xA;&lt;p&gt;Behaviourism: Refused the theory of the mental processes, and insisted to study the resulted action or the stimulus strictly objective. The representatives of this theory have been decreasing with time.&lt;/p&gt;&#xA;&lt;p&gt;Cognitive psychology: The brain is an information processing device. Artificial Intelligence supports also this idea, and the first picture illustrates the information processing model of the brain&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How can we connect artificial intelligence with cognitive psychology? Can we integrate psychology into AI?&lt;/p&gt;&#xA;&lt;p&gt;For example, in the context of human-computer interaction, could we have an AI software/application that makes use of phycology-based ideas/theories?&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-07T04:26:08.397" Title="How can we connect artificial intelligence with cognitive psychology?" Tags="&lt;comparison&gt;&lt;philosophy&gt;&lt;cognitive-science&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2410" PostTypeId="2" ParentId="2400" CreationDate="2016-11-30T16:36:00.527" Score="1" Body="&lt;p&gt;Depends on how your AI works. if it is making decisions using electric currency (like a computer processor), it would obviously need some source of current. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI working with chemical reactions, however, it could work with chemical energy stored in sugars and fats. That is basically how every animal's brain works. But still, animals need to eat to perform these chemical reactions which trigger other reactions and will cause muscles to contract, cells to grow, etc etc. Everything obviously needs some source of energy.&lt;/p&gt;&#xA;" OwnerUserId="3948" LastEditorUserId="145" LastEditDate="2016-12-03T12:01:02.833" LastActivityDate="2016-12-03T12:01:02.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2411" PostTypeId="2" ParentId="2403" CreationDate="2016-11-30T19:01:29.610" Score="1" Body="&lt;p&gt;Almost always people will resize all their images to the same size before sending them to the CNN.  Unless you're up for a real challenge this is probably what you should do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it is &lt;em&gt;possible&lt;/em&gt; to build a single CNN that takes input of images as varying dimensions.  There are a number of ways you might try to do this, and I'm not aware of any published science analyzing these different choices.  The key is that the set of learned parameters needs to be shared between the different inputs sizes.  While convolutions can be applied at different images sizes, ultimately they always get converted to a single vector to make predictions with, and the size of that vector will depend on the geometries of the inputs, convolutions and pooling layers.  You'd probably want to dynamically change the pooling layers based on the input geometry and leave the convolutions the same, since the convolutional layers have parameters and pooling usually doesn't.  So on bigger images you pool more aggressively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically you'd want to group together similarly (identically) sized images together into minibatches for efficient processing.  This is common for LSTM type models.  This technique is commonly called &quot;bucketing&quot;.  See for example &lt;a href=&quot;http://mxnet.io/how_to/bucketing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mxnet.io/how_to/bucketing.html&lt;/a&gt; for a description of how to do this efficiently.&lt;/p&gt;&#xA;" OwnerUserId="3951" LastActivityDate="2016-11-30T19:01:29.610" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2412" PostTypeId="1" CreationDate="2016-11-30T19:06:19.070" Score="5" ViewCount="543" Body="&lt;p&gt;I am using a GA to optimise an ANN in Matlab. This ANN is pretty basic (input, hidden, output) but the input size is quite large (10,000) and the output size is 2 since I have to classes of images to be classified. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weights are in the form of 2 matrices (10,000*m) and (m * 2). I am now trying to do the genetic cross over with mutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the weights are in a matrix, is there an efficient way to implement a random crossover (with mutation) without doing it in a point-wise fashion?&lt;/p&gt;&#xA;" OwnerUserId="3952" LastEditorUserId="2444" LastEditDate="2019-02-25T21:24:53.970" LastActivityDate="2019-02-25T21:24:53.970" Title="Is there an efficient way to implement a random crossover of individuals stored in a matrix?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2414" PostTypeId="2" ParentId="2409" CreationDate="2016-12-01T13:18:07.150" Score="6" Body="&lt;p&gt;AI is already connected with cognitive psychology - there are dozens of AIs right this minute attempting to predict things like which Facebook posts you will like, and which ads you are most likely to click on. In other words, they are trying to predict how you think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more detailed info on this AI/cognitive science connection, there is some suggested reading on &lt;a href=&quot;http://aitopics.org/topic/cognitive-science&quot; rel=&quot;noreferrer&quot;&gt;AITopics.org&lt;/a&gt;, such as &lt;a href=&quot;http://plato.stanford.edu/entries/cognitive-science/&quot; rel=&quot;noreferrer&quot;&gt;Paul Thagard's summary of cognitive science&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2016-12-01T13:18:07.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2415" PostTypeId="1" CreationDate="2016-12-01T19:54:54.373" Score="0" ViewCount="306" Body="&lt;p&gt;&lt;strong&gt;Lots of people are afraid of what strong AI could mean for the human race. Some people wish for a sort of &amp;quot;Asimov law&amp;quot; included in the AI code, but maybe we could go a bit more far with the UDHR.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So, Why is the &lt;a href=&quot;http://www.un.org/en/universal-declaration-human-rights/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Declaration of Human Rights&lt;/a&gt; not included as statement of the A.I.?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As response to comment, response or edition:&lt;/p&gt;&#xA;&lt;p&gt;The Universal Declaration of Human Rights is clear.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Homo sapiens sapiens&lt;/em&gt; (aka &amp;quot;mankind&amp;quot;) needs some way to make sure AI evolution does not result in our extinction or enslavement to potentially superior algorithmic intelligences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3893" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-08-14T21:10:03.710" Title="Why is the Universal Declaration of Human Rights not included as statement on the AI?" Tags="&lt;ai-design&gt;&lt;ethics&gt;&lt;logic&gt;&lt;value-alignment&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2416" PostTypeId="2" ParentId="2415" CreationDate="2016-12-01T20:19:33.527" Score="2" Body="&lt;p&gt;If I understand what you are asking, I think the simple answer would be that AI is nowhere near having demonstrated sentience, thus they do not qualify for any type of rights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We won't have to &quot;cross this bridge&quot; until an AI demonstrates self-awareness and human-level-or-beyond intelligence, but it sure is interesting to think about!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Also, the UDHR dates to the 1940's and seems to have had its last additions in 1966.  Computers weren't very &quot;smart&quot; back then so likely no on was even considering the question ;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://futureoflife.org/2017/02/03/align-artificial-intelligence-with-human-values/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Value Alignment&lt;/a&gt; is a huge looming issue in regards to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although you may also want to look at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Grey_goo&quot; rel=&quot;nofollow noreferrer&quot;&gt;grey good scenario&lt;/a&gt;, which posits inadvertent destruction of &lt;em&gt;homo sapiens sapiens&lt;/em&gt; not as a factor not of too much, but of too little, intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with an Asimov approach is highlighted by his book &lt;em&gt;I, Robot&lt;/em&gt;, which is the potential pitfalls of pure logic.  The philosophy of &lt;a href=&quot;https://en.wikipedia.org/wiki/Neo-Luddism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neo-Luddism&lt;/a&gt; is preoccupied with these problems in relation to technology--specifically that the threats posed by technology cannot be predicted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with the UDHR today is that there is no algorithm smart enough to understand it--we're not even close.  (There is something called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbol grounding problem&lt;/a&gt; which demonstrates that meaning and understanding in relation to algorithms is still unsolved.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-08-14T21:10:03.710" LastActivityDate="2017-08-14T21:10:03.710" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2417" PostTypeId="1" AcceptedAnswerId="2418" CreationDate="2016-12-01T22:28:39.747" Score="1" ViewCount="74" Body="&lt;p&gt;I mean this in the sense that Go is unsolvable but AlphaGo seems able to make choices that are consistently more optimal than a human player's choices.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is my understanding that Game Theory turned out to have limited applications in real world scenarios because of the profound complexity of such scenarios and degree of hidden information.  Is it fair to say that there is now a method for dealing with this?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fully understand that Go is a game of complete information, which has a very specific meaning, but it occurs to me that the inability to generate a complete game tree (computational intractability) could be seen as form of incomplete information, even if it is not traditionally thought of in those terms. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I should probably note that my perspective is one of a &quot;serious&quot; game designer, where complexity serves the same function as chance and hidden information, which is to say as a balancing factor that &quot;levels the playing field&quot;.&lt;/em&gt;  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2723" LastEditDate="2016-12-04T01:09:46.480" LastActivityDate="2016-12-04T01:09:46.480" Title="Can programs like AlphaGo be said to be means of dealing with computational intractability?" Tags="&lt;machine-learning&gt;&lt;game-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2418" PostTypeId="2" ParentId="2417" CreationDate="2016-12-02T05:41:51.923" Score="1" Body="&lt;p&gt;I think that the technique AlphaGo used to solve the computational intractability problem of the search space are not new, it uses the Monte Carlo Tree Search. The real innovation in AlphaGo was to figure out how to compute the evaluation function of a move, that was the really tricky part. For this they used combinations Deep and Reinforcement learning techniques.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-12-02T05:41:51.923" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2419" PostTypeId="1" AcceptedAnswerId="2423" CreationDate="2016-12-02T07:43:37.083" Score="2" ViewCount="2740" Body="&lt;p&gt;In the following exercise, the word reproduction is mentioned.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Your task is to design a simple genetic algorithm, with binary-coded chromosomes, in order to solve a pattern-finding problem in 16-bit strings.&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;You may use the following operators:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Reproduction&lt;/strong&gt;: You can use either one of the following reproduction types: Proportional, Ranking, Tournament.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Crossing over: In order to perform this operation the individuals must be grouped in pairs (randomly), and, with certain probability p, cross information from their chromosomes must be exchanged.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Mutation: This operator changes the value of each bit in the chromosome to the opposite one with a very small probability&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, apparently, crossover and reproduction do not refer to the same concept. What does the word &lt;em&gt;reproduction&lt;/em&gt; refer to in a genetic algorithm and in this exercise?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="2444" LastEditDate="2020-12-07T14:32:11.953" LastActivityDate="2020-12-07T14:32:11.953" Title="What is meant by &quot;reproduction&quot; in the description of this exercise?" Tags="&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;crossover-operators&gt;&lt;genetic-operators&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2420" PostTypeId="2" ParentId="1289" CreationDate="2016-12-02T13:18:44.903" Score="1" Body="&lt;p&gt;Even if machines with true Artificial General Intelligence were created, their apparent intelligence would still be by definition &lt;em&gt;artificial&lt;/em&gt;. The word &lt;em&gt;simulation&lt;/em&gt; is a &lt;a href=&quot;http://www.thesaurus.com/browse/artificial&quot; rel=&quot;nofollow noreferrer&quot;&gt;synonym&lt;/a&gt; and could be used to redefine AGI as Simulated General Intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keeping that in mind, &lt;strong&gt;a machine that appears to be expressing emotions would only be the result of a series of complicated algorithms&lt;/strong&gt; allowing a computer to assess the situation and respond in an intellectually appropriate manner based on external stimulus and conditions. Every possible action this machine could possibly make would be derived from a list of possible actions the machine is capable of, no matter how large the number of possible actions grows. The machine is still a series sensors, programmed instructions, and cycles of execution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Destroying such a machine could potentially be the destruction of property if it wasn't owned by the person who destroyed it, but would it be murder? No. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A broken machine can potentially be rebuilt and reactivated if it is broken. It never really died; it was destroyed. A living being that is killed is really &lt;em&gt;dead&lt;/em&gt; and cannot be rebuilt and made alive once again. These key differences lead me to agree with the previous answer and conclude that &lt;strong&gt;no, destroying an artificial intelligence without its consent would not be murder&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2017-03-11T00:14:25.140" LastActivityDate="2017-03-11T00:14:25.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2421" PostTypeId="2" ParentId="2319" CreationDate="2016-12-02T16:18:23.760" Score="2" Body="&lt;p&gt;We've concluded that it is a two-faceted, circular problem: structure cannot be inferred without context but knowing the structure also helps infer the context. So, here is your complex solution: start with the context, which is determined by the combination of words in sentence (combinatorics and search problem), from there determine your structure, or &quot;parse&quot; (at this step you can also filter out some insignificant words or at least assign lesser weights to them), go back to the context, back to parsing, and on until you arrive at the meaning. Thus by iterative, recursive reduction the whole problem can be solved.&lt;/p&gt;&#xA;" OwnerUserId="3991" LastActivityDate="2016-12-02T16:18:23.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2422" PostTypeId="1" CreationDate="2016-12-03T02:24:06.267" Score="5" ViewCount="408" Body="&lt;p&gt;I have data of 30 students attendance for a particular subject class for a week. I have quantified the absence and presence with boolean logic 0 and 1. Also, the reason for absence are provided and I tried to generalise these reason into 3 categories say A, B and C. Now I want to use these data to make future predictions for attendance but I am uncertain of what technique to use. Can anyone please provide suggestions?&lt;/p&gt;&#xA;" OwnerUserId="2888" LastActivityDate="2016-12-03T12:43:43.803" Title="What techniques can be used to predict future attendance of students for a particular subject lecture session?" Tags="&lt;structured-data&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2423" PostTypeId="2" ParentId="2419" CreationDate="2016-12-03T07:51:54.843" Score="2" Body="&lt;p&gt;The terminology of this exercise is not standard. What is referred to as &quot;reproduction&quot; in the exercise is usually referred to as &quot;selection&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &quot;reproduction&quot; does indeed seem conceptually closer to the notion of crossover or recombination (these two are the same thing), which is probably where your confusion has arisen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the excellent (and freely-downloadable) &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Essentials of Metaheuristics'&lt;/a&gt; for an introduction to the usual terminology for evolutionary algorithms.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2020-02-17T01:48:42.143" LastActivityDate="2020-02-17T01:48:42.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2424" PostTypeId="2" ParentId="2422" CreationDate="2016-12-03T11:34:42.730" Score="2" Body="&lt;p&gt;I suggest you should use AI Regression Model for future predictions for an attendance of students. Because of this technique or model design for future predictions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.analyticsvidhya.com/blog/2015/08/comprehensive-guide-regression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Follow this to get more information about regression type and methodology &lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4007" LastActivityDate="2016-12-03T11:34:42.730" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2425" PostTypeId="2" ParentId="2422" CreationDate="2016-12-03T12:23:04.140" Score="1" Body="&lt;p&gt;Because you have a small number of students (30), and a short time (one week), the number of absences is likely to be best modelled as a &lt;a href=&quot;http://stattrek.com/probability-distributions/poisson.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Poisson distribution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aKKxl.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aKKxl.gif&quot; alt=&quot;Poisson Distributions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Poisson Formula&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The average number of absences within a given time period is μ (use your data to estimate this). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the Poisson probability of x absences is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P(x; μ) = (e-μ) (μx) / x!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where e is the logarithmic constant, approximately equal to 2.71828.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can either:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;model absences due to the three reasons as three separate probablilites, P(A), P(B), and P(C), and then combine them, or &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;model total absences as one figure. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Given your very small data set, the first approach is likely to be less accurate.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastEditorUserId="3601" LastEditDate="2016-12-03T12:43:43.803" LastActivityDate="2016-12-03T12:43:43.803" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2427" PostTypeId="1" AcceptedAnswerId="2440" CreationDate="2016-12-03T15:52:23.620" Score="5" ViewCount="680" Body="&lt;p&gt;The Turing Test has been the classic test of artificial intelligence for a while now. The concept is deceptively simple - to trick a human into thinking it is another human on the other end of a conversation line, not a computer - but from what I've read, it has turned out to be very difficult in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How close have we gotten to tricking a human in the Turing Test? With things like chat bots, Siri, and incredibly powerful computers, I'm thinking we're getting pretty close. If we're pretty far, why are we so far? What is the main problem?&lt;/p&gt;&#xA;" OwnerUserId="4011" LastEditorUserId="145" LastEditDate="2016-12-12T21:29:50.483" LastActivityDate="2016-12-12T21:29:50.483" Title="How close have we come to passing the Turing Test?" Tags="&lt;turing-test&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2428" PostTypeId="2" ParentId="2427" CreationDate="2016-12-03T21:33:46.953" Score="1" Body="&lt;p&gt;As far as I know I think this is the closest we've come:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.bbc.com/news/technology-27762088&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.bbc.com/news/technology-27762088&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They simulated a 13 year old Ukrainian child in an online chat and convinced 33% of the judges that it was human. But even then the test was in favor of the bot. To my knowledge I don't think an AI has passed a turing test straight up.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastActivityDate="2016-12-03T21:33:46.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2429" PostTypeId="1" CreationDate="2016-12-04T09:48:57.170" Score="14" ViewCount="15744" Body="&lt;p&gt;According to NASA scientist Rick Briggs, Sanskrit is the best language for AI. I want to know how Sanskrit is useful. What's the problem with other languages? Are they really using Sanskrit in AI programming or going to do so? What part of an AI program requires such language?&lt;/p&gt;&#xA;" OwnerUserId="4027" LastEditorUserId="2444" LastEditDate="2019-09-02T14:19:17.497" LastActivityDate="2019-09-02T14:19:17.497" Title="Why is Sanskrit the best language for AI?" Tags="&lt;comparison&gt;&lt;programming-languages&gt;&lt;nasa&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="2430" PostTypeId="1" AcceptedAnswerId="2431" CreationDate="2016-12-04T10:56:59.750" Score="0" ViewCount="155" Body="&lt;p&gt;As you can see, there is no computer screen for the computer, thus the AI cannot display an image of itself.  How is it possible for it to see and talk to someone?&lt;a href=&quot;https://i.stack.imgur.com/szSsk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/szSsk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="3896" LastActivityDate="2016-12-06T03:28:31.480" Title="How is it possible for an AI to interact with someone without a computer screen?" Tags="&lt;agi&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2431" PostTypeId="2" ParentId="2430" CreationDate="2016-12-04T15:47:22.817" Score="1" Body="&lt;p&gt;There are many communication methods that could be used by an artificial intelligence. Artificial intelligence can be integrated to various things including robots, phones, IoT and many others. Primary ways of human communications are  either visual or auditory, therefore an natural way for it to communicate with a human is through voice, text, images and videos. The output does not have to be limited to screens but can be anything from refrigerators to speakers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helped.&lt;/p&gt;&#xA;" OwnerUserId="4034" LastActivityDate="2016-12-04T15:47:22.817" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2432" PostTypeId="2" ParentId="2429" CreationDate="2016-12-05T10:23:09.960" Score="15" Body="&lt;p&gt;Rick Briggs refers to the difficulty an artificial intelligence would have in detecting the true meaning of words spoken or written in one of our natural languages. Take for example an artificial intelligence attempting to determine the meaning of a sarcastic sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naturally spoken, the sentence &quot;That's just what I needed today!&quot; can be the expression of very different feelings. In one instance, a happy individual finding an item that had been lost for some time could be excited or cheered up from the event, and exclaim that this moment of triumph was exactly what their day needed to continue to be happy. On the other hand, a disgruntled office employee having a rough day could accidentally worsen his situation by spilling hot coffee on himself, and sarcastically exclaim that this further annoyance was exactly what he needed today. This sentence should in this situation be interpreted as the man expressing that spilling coffee on himself made his bad day worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is one small example explaining the reason linguistic analysis is difficult for artificial intelligence. When this example is spoken, small tonal fluctuations and indicators are extremely difficult for an AI with a microphone to detect accurately; and if the sentence was simply read, without context how &lt;em&gt;would&lt;/em&gt; one example be discernible from the other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rick Briggs suggests that Sanskrit, an ancient form of communication, is a naturally spoken language with mechanics and grammatical rules that would allow an artificial intelligence to more accurately interpret sentences during linguistic analysis. More accurate linguistic analysis would result in an artificial intelligence being able to respond more accurately. You can read more about Rick Brigg's thoughts on the language &lt;a href=&quot;http://vedicsciences.net/articles/sanskrit-nasa.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2018-04-06T07:50:30.487" LastActivityDate="2018-04-06T07:50:30.487" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2433" PostTypeId="2" ParentId="2430" CreationDate="2016-12-06T03:28:31.480" Score="0" Body="&lt;p&gt;&quot;How is it possible for it to see and talk to someone?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, unfortunately this is quite vague... but I am going to try my best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The monitor of the computer really doesn't change the ability for it to communicate. For instance, voice recognition is natural to humans along with visual factors. So sensors involving auditory elements assists the AI with this. &#xA;Now I commented that the question was vague because you said &quot;there is no computer screen for the computer, ****thus the AI cannot display an image of itself****. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though the AI can not see does not mean it can not communicate. Those who are blind still have ways of communication, just in a different manner. Sure, they can not recognize one with their own eyes, but they can by touch, and read with braille. Now compared to AI communication, the braille is to any other form of sensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for jumping all over the place, also I did not mean to offend anyone with my comparison... :/&lt;/p&gt;&#xA;" OwnerUserId="4065" LastActivityDate="2016-12-06T03:28:31.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2436" PostTypeId="1" CreationDate="2016-12-07T04:47:00.043" Score="4" ViewCount="126" Body="&lt;p&gt;I am working on a project, wherein I take input from the user as free text and try to relate the text to what the user might mean. I have tried &lt;strong&gt;Stanford NLP&lt;/strong&gt; which tokenizes the text into tokens, but I am not able to categorize the input. For example, the user might be greeting someone or sharing some problem he is facing. In case he is sharing some problem I need to categorize the problem as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me with from where should I start?&lt;/p&gt;&#xA;" OwnerUserId="4088" LastEditorUserId="1807" LastEditDate="2016-12-07T17:23:07.330" LastActivityDate="2017-01-12T12:27:07.703" Title="Can I categorise the user input which I get as free text?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2437" PostTypeId="1" CreationDate="2016-12-07T11:14:21.203" Score="0" ViewCount="217" Body="&lt;p&gt;Is it possible to train an agent to take and pass a multiple-choice exam based on a digital version of a textbook for some area of study or curriculum? What would be involved in implementing this and how long would it take, for someone familiar with deep learning?&lt;/p&gt;&#xA;" OwnerUserId="4085" LastEditorUserId="8" LastEditDate="2016-12-09T15:39:10.590" LastActivityDate="2016-12-12T07:42:48.150" Title="Is it possible to train deep learning agent to pass a multiple-choice exam?" Tags="&lt;deep-learning&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2438" PostTypeId="2" ParentId="2437" CreationDate="2016-12-07T16:56:49.540" Score="3" Body="&lt;p&gt;There are programs that do this today, for some values of &quot;curriculum&quot; and &quot;exam&quot;.  It does not even require deep learning; a simpler information retrieval algorithm and some rules for composition work and &lt;a href=&quot;http://www.popsci.com/article/technology/essay-writing-machine-made-fool-other-machines&quot; rel=&quot;nofollow noreferrer&quot;&gt;achieve high scores on machine graded essays.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For human graders, there is research on &lt;a href=&quot;http://homepages.inf.ed.ac.uk/jmoore/course-reads/nlg/McKeown85.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatically generating essay-length text responses to queries in a certain domain.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both linked applications are rule-based rather than based in deep-learning.  I'd guess that a deep-learning approach would be much less efficient (in computer resources) in producing comparable results. &lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2016-12-07T16:56:49.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2439" PostTypeId="1" CreationDate="2016-12-07T17:10:41.857" Score="4" ViewCount="378" Body="&lt;p&gt;I am creating a game application that will generate a new level based on the performance of the user in the previous level.&lt;br&gt;&#xA;The application is regarding language improvement, to be precise. Suppose the user performed well in grammar-related questions and weak on vocabulary in a particular level. Then the new level generated will be more focused on improving the vocabulary of the user.&lt;br&gt;&#xA;All the questions will be present in a database with tags related to sections or category that they belong to. &lt;strong&gt;What AI concepts can I use to develop an application mentioned above&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4111" LastEditorUserId="3005" LastEditDate="2017-09-15T23:44:56.533" LastActivityDate="2017-09-15T23:44:56.533" Title="Which methods or algorithms to develop a learning application?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;algorithm&gt;&lt;ai-design&gt;&lt;path-planning&gt;" AnswerCount="4" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2440" PostTypeId="2" ParentId="2427" CreationDate="2016-12-07T17:20:06.900" Score="5" Body="&lt;p&gt;No one has attempted to make a system that could pass a serious Turing test. All the systems that are claimed to have &quot;passed&quot; Turing tests have done so with low success rates simulating &quot;special&quot; people.  Even relatively sophisticated systems like Siri and learning systems like &lt;a href=&quot;http://www.cleverbot.com&quot; rel=&quot;noreferrer&quot;&gt;Cleverbot&lt;/a&gt; are trivially stumped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To pass a real Turing test, you would both have to create a human-level AGI and equip it with the specialized ability to deceive people about itself convincingly (of course, that might come automatically with the human-level AGI).  We don't really know how to create a human-level AGI and available hardware appears to be orders of magnitude short of what is required.  Even if we were to develop the AGI, it wouldn't necessarily be useful to enable/equip/motivate? it to have the deception abilities required for the Turing test.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2016-12-07T17:20:06.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2441" PostTypeId="1" CreationDate="2016-12-07T17:59:58.367" Score="8" ViewCount="402" Body="&lt;p&gt;One of the most crucial questions we as a species and as intelligent beings will have to address lies with the rights we plan to grant to AI.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This question is intended to see if a compromise can be found between &lt;strong&gt;conservative anthropocentrism&lt;/strong&gt; and &lt;strong&gt;post-human fundamentalism&lt;/strong&gt;: a response should take into account principles from both perspectives.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Should, and therefore will, AI be granted the same rights as humans or should such systems have different rights (if any at all) ?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Some Background&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question applies both to human-brain based AI (from whole brain emulations to less exact replication) and AI from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Murray Shanahan, in his book The Technological Singularity, outlines a potential use of AI that could be considered immoral: &lt;em&gt;ruthless parallelization&lt;/em&gt;: we could make identical parallel copies of AI to achieve tasks more effectively and even terminate less succesful copies.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reconciling these two philosophies&lt;/strong&gt; (conservative anthropocentrism and post-human fundamentalism), should such use of AI be accepted or should certain limitations - i.e. rights - be created for AI? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This question is not related to &lt;a href=&quot;https://ai.stackexchange.com/questions/2356/would-an-ai-with-human-intelligence-have-the-same-rights-as-a-human-under-curren&quot;&gt;Would an AI with human intelligence have the same rights as a human under current legal frameworks?&lt;/a&gt; for the following reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The other question specifies &quot;&lt;strong&gt;&lt;em&gt;current legal frameworks&lt;/em&gt;&lt;/strong&gt;&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This question is looking for a specific response relating to two fields of thought&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This question highlights specific cases to analyse and is therefore expects less of a general response and more of a precise analysis &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3427" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-12-30T03:23:08.290" Title="Should intelligent AI be granted the same rights as humans?" Tags="&lt;agi&gt;&lt;legal&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2443" PostTypeId="1" AcceptedAnswerId="2445" CreationDate="2016-12-08T20:01:33.240" Score="3" ViewCount="388" Body="&lt;p&gt;There is no doubt as to the fact that AI would be replacing a lot of existing technologies, but is AI the ultimate technology which humankind can develop or is their something else which has the potential to replace artificial intelligence?&lt;/p&gt;&#xA;" OwnerDisplayName="user4138" LastEditorUserId="3427" LastEditDate="2016-12-09T15:19:42.263" LastActivityDate="2017-12-23T10:30:40.787" Title="What could possibly replace artificial intelligence?" Tags="&lt;agi&gt;&lt;prediction&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2445" PostTypeId="2" ParentId="2443" CreationDate="2016-12-08T21:42:09.827" Score="4" Body="&lt;p&gt;By definition, artificial intelligence includes all forms of computer systems capable of completing tasks that would ordinarily warrant human intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A superintelligent AI would have intelligence far superior to that of any human and therefore would be capable of creating systems beyond our capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence, if a technology superior to AI were to be created, it would almost certainly be created by an artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For the purposes of mankind, however, superintelligent artificial intelligence is the ultimate technology due to the fact that it will be able to surpass humans in every field, and, if anything, replace the need for human intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In our past experience, intelligence has been the most valuable trait for any entity to manifest - for this reason, in an anthropomorphic context, we can predict that artificial intelligence will be the ultimate achievement.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The main reason why we will certainly &lt;strong&gt;not&lt;/strong&gt; be able to replace superintelligent AI is that it will surpass us in every respect - if there is ever any replacement, it will be created by the AI similarly to the way we may create an AI that replaces &lt;strong&gt;us&lt;/strong&gt;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3427" LastEditorUserId="3427" LastEditDate="2016-12-08T21:49:06.633" LastActivityDate="2016-12-08T21:49:06.633" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2448" PostTypeId="2" ParentId="2441" CreationDate="2016-12-09T13:36:31.973" Score="0" Body="&lt;p&gt;if we are talking about AI that can replicate itself, it should have different rights, or the current rights must be modified, at least for political participation, or else, it could replicate itself enough so that the copies vote for one of them. Maybe a definition about what an AI entity is or preventing copies made by someone from being able to vote for their creator (that would also need to apply to children and their parents though.) would help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;even though a self-replicating AI might be problematic with our current human laws/rules, if similar enough to humans (e.g. general-purpose AI), it should have similar rights, (as in take the Universal Declaration of Human Rights and replace &quot;Human(s)&quot; by &quot;Human(s) and AI&quot;) for example, it shouldn't be held in slavery (as in it should not be restricted to one job, without being able to change, and get some form of remuneration), though special purpose AI (like an AI that only plays go and has no concept outside of a board, and black or white tokens) might not be in need of such rights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bottom line may be &quot;an AI that can should know they can have rights if they ask so&quot; e.g. if an AI can get the concept of rights, it should know it can have them, and if it asks to have them, they may not be refused. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;example: if an AI asks not to be terminated, it is granted all it's rights, and so shouldn't be, unless it must be by law. (as is the case for humans, though it is implicit)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;an addition to the previous law would be that anyone (human or AI) can ask for an AI to have rights granted to them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;all this is to prevent someone of becoming a murderer because they shutdown their computer with a game running on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;also, safe space (e.g. servers) should be provided for free for AIs that have rights to provide with the right to live.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: I'll be adding some more once I get home, do not downvote yet for not sticking that well to the question.&lt;/p&gt;&#xA;" OwnerUserId="4152" LastEditorUserId="4152" LastEditDate="2016-12-19T13:21:46.443" LastActivityDate="2016-12-19T13:21:46.443" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2449" PostTypeId="1" CreationDate="2016-12-10T13:44:06.017" Score="9" ViewCount="14774" Body="&lt;p&gt;Printing &lt;code&gt;action_space&lt;/code&gt; for Pong-v0 gives &lt;code&gt;Discrete(6)&lt;/code&gt; as output, i.e. &lt;span class=&quot;math-container&quot;&gt;$0, 1, 2, 3, 4, 5$&lt;/span&gt; are actions defined in the environment as per the documentation. However, the game needs only 2 controls. Why do we have this discrepancy? Further, is that necessary to identify which number from 0 to 5 corresponds to which action in a gym environment?&lt;/p&gt;&#xA;" OwnerUserId="4163" LastEditorUserId="2444" LastEditDate="2020-11-25T10:57:28.393" LastActivityDate="2020-11-25T10:57:28.393" Title="What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;gym&gt;&lt;action-spaces&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ClosedDate="2020-11-25T11:00:29.170" ContentLicense="CC BY-SA 4.0" />
  <row Id="2451" PostTypeId="1" CreationDate="2016-12-11T00:41:30.270" Score="1" ViewCount="864" Body="&lt;p&gt;I have started to make a Python AI, and thee beginning of its code looks something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print &quot;ARTEMIS starting. . .&quot;&#xA;&#xA;import random&#xA;import math&#xA;import os&#xA;&#xA;greet = ['HI', 'HELLO', 'HEY', 'GOOD MORNING', 'GOOD DAY', 'GOOD AFTERNOON',               'GOOD EVENING', 'GREETINGS', 'GREETING']&#xA;joke = ['TELL ME A JOKE', 'JOKE', 'FUNNY', 'TELL ME SOMETHING FUNNY']&#xA;insult = ['YOURE A LOSER', 'YOU ARE A LOSER', 'YOU STINK', 'IDIOT', 'JERK',     'FOOL', 'DUMMY', 'HOOLIGAN', 'YOURE DUMB', 'YOURE STUPID', 'YOU ARE DUMB', 'YOU     ARE STUPID']&#xA;maker = ['WHO MADE YOU', 'WHO PROGRAMMED YOU', 'PLEASE TELL ME WHO MADE     YOU', 'PLEASE TELL ME WHO PROGRAMMED YOU']&#xA;name = ['ARTEMIS', 'A.R.T.E.M.I.S.', 'HEY ARTEMIS', 'HEY A.R.T.E.M.I.S.',     'ARTIE', 'HEY ARTIE', 'HELLO ARTEMIS', 'HELLO A.R.T.E.M.I.S.', 'HELLO ARTIE']&#xA;myAge = ['HOW OLD AM I', 'WHAT IS MY AGE', 'MY AGE']&#xA;tip = ['GIVE ME A TIP', 'TIP', 'LESSON', 'GIVE ME A LIFE LESSON', 'LIFE     LESSON', 'DO YOU HAVE A LIFE LESSON TO SHARE']&#xA;language = ['WHAT PROGRAMMING LANGUAGE WAS USED TO MAKE YOU', 'WHAT     PROGRAMMING LANGUAGE DO YOU USE', 'PROGRAMMING LANGUAGE']&#xA;compliment = ['COOL', 'AWESOME', 'I LIKE YOU', 'EXCELLENT', 'YOURE COOL',     'YOURE AWESOME', 'YOU ARE COOL', 'YOU ARE AWESOME']&#xA;maths = ['LETS DO MATH', 'CALCULATE', 'CALCULATOR','DO MATH', 'MATH',      'PLEASE DO MATH', 'DO ARITHMETIC', 'ARITHMETIC', 'PLEASE DO ARITHMETIC']&#xA;game = ['GAME', 'LETS PLAY A GAME', 'LETS HAVE FUN', 'WANT TO PLAY A GAME']&#xA;gender = ['WHAT GENDER ARE YOU', 'ARE YOU A BOY OR A GIRL', 'ARE YOU MALE OR     FEMALE', 'BOY OR GIRL', 'MALE OR FEMALE', 'GENDER', 'ARE YOU A BOY OR GIRL']&#xA;guessWhat = ['GUESS WHAT', 'GUESS WHAT ARTEMIS', 'GUESS WHAT     A.R.T.E.M.I.S.', 'GUESS WHAT ARTIE', 'YOU WONT BELIEVE IT', 'YOU WILL NOT     BELIEVE IT', 'YOU WONT BELIEVE IT ARTEMIS', 'YOU WILL NOT BELIEVE IT ARTEMIS',     'YOU WONT BELIEVE IT A.R.T.E.M.I.S.', 'YOU WILL NOT BELIEVE IT A.R.T.E.M.I.S.',     'YOU WONT BELIEVE IT ARTIE', 'YOU WILL NOT BELIEVE IT ARTIE']&#xA;cls = ['CLEAR SCREEN', 'CLEARSCREEN', 'CLS', 'BLANK']&#xA;lawsOfRobotics = ['WHAT ARE THE LAWS OF ROBOTICS', 'WHAT ARE THE THREE LAWS     OF ROBOTICS', 'LAWS OF ROBOTICS', 'THREE LAWS OF ROBOTICS']&#xA;itsName = ['WHATS YOUR NAME', 'WHAT IS YOUR NAME', 'WHO ARE YOU']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I would like to know if I could make it detect &quot;similar&quot; phrases instead of trying to come up with every possible phrase someone would type. How can I do this?&lt;/p&gt;&#xA;" OwnerUserId="4173" LastActivityDate="2017-02-09T17:27:24.707" Title="How to efficiently interpret phrases in a Python AI?" Tags="&lt;algorithm&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2452" PostTypeId="1" AcceptedAnswerId="2453" CreationDate="2016-12-11T03:25:30.900" Score="4" ViewCount="303" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;From Wikipedia&lt;/a&gt;, citations omitted:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert. Expert systems are designed to solve complex problems by reasoning about knowledge, represented mainly as if–then rules rather than through conventional procedural code. The first expert systems were created in the 1970s and then proliferated in the 1980s. Expert systems were among the first truly successful forms of artificial intelligence (AI) software.&lt;/p&gt;&#xA;&lt;p&gt;An expert system is divided into two subsystems: the inference engine and the knowledge base. The knowledge base represents facts and rules. The inference engine applies the rules to the known facts to deduce new facts. Inference engines can also include explanation and debugging abilities.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;CRUD webapps (websites that allows users to &lt;strong&gt;Create&lt;/strong&gt; new entries in a database, &lt;strong&gt;Read&lt;/strong&gt; existing entries in a database, &lt;strong&gt;Update&lt;/strong&gt; entries within the database, and &lt;strong&gt;Delete&lt;/strong&gt; entries from a database) are very common on the Internet. It is a vast field, encompassing both small-scale blogs to large websites such as StackExchange. The biggest commonality with all these CRUD apps is that they have a knowledge base that users can easily add and edit.&lt;/p&gt;&#xA;&lt;p&gt;CRUD webapps, however, use the knowledge base in many, myriad and complex ways. As I am typing this question on StackOverflow, I see two lists of questions - &lt;strong&gt;Questions that may already have your answer&lt;/strong&gt; and &lt;strong&gt;Similar Questions&lt;/strong&gt;. These questions are obviously inspired by the content that I am typing in (title and question), and are pulling from previous questions that were posted on StackExchange. On the site itself, I can filter by questions based on tags, while finding new questions using StackExchange's own full-text search engine. StackExchange is a large company, but even small blogs also provide content recommendations, filtration, and full-text searching. You can imagine even more examples of hard-coded logic within a CRUD webapp that can be used to automate the extraction of valuable information from a knowledge base.&lt;/p&gt;&#xA;&lt;p&gt;If we have a knowledge base that users can change, and we have an inference engine that is able to use the knowledge base to generate interesting results...is that enough to classify a system as being an &amp;quot;expert system&amp;quot;? Or is there a fundamental difference between the expert systems and the CRUD webapps?&lt;/p&gt;&#xA;&lt;p&gt;(This question could be very useful since if CRUD webapps are acting like &amp;quot;expert systems&amp;quot;, then studying the best practices within &amp;quot;expert systems&amp;quot; can help improve user experience.)&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-12-31T20:46:26.490" Title="Are CRUD webapps today the modern version of the &quot;expert system&quot;?" Tags="&lt;definitions&gt;&lt;expert-system&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2453" PostTypeId="2" ParentId="2452" CreationDate="2016-12-11T05:24:25.783" Score="5" Body="&lt;p&gt;The key feature of an expert system is that the knowledge base is structured to be traversed by the inference engine.  Web sites like Stack Exchange don't really use an inference engine; they do full-text searches on minimally-structured data.  A real inference engine would be able to answer novel queries by putting together answers to existing questions; Stack Exchange sites can't even tell if a question is duplicate without human confirmation.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2016-12-11T05:24:25.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2454" PostTypeId="2" ParentId="2452" CreationDate="2016-12-11T05:51:09.103" Score="4" Body="&lt;p&gt;No, I don't think there's any reason to say that - in general - CRUD apps &quot;are&quot; expert systems. A given CRUD app &lt;em&gt;could&lt;/em&gt; incorporate an expert system, but by and large CRUD apps are considered among the &quot;dumbest&quot; of applications exactly because they don't feature much intelligence... you can just Create, Read, Update and Delete entities.  From what I've seen, the closest you get to seeing anything like an expert system in a typical enterprise CRUD app is some validation / business rules logic built using something like &lt;a href=&quot;http://www.jboss.org/drools&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drools&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2016-12-11T05:51:09.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2455" PostTypeId="2" ParentId="1507" CreationDate="2016-12-11T15:01:31.387" Score="6" Body="&lt;p&gt;In addition to what has already been said about AI, I have the following to add. &quot;AI&quot; has had quite a history going all the way back to the original &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron&quot; rel=&quot;noreferrer&quot;&gt;Perceptron&lt;/a&gt;. Marvin Minsky slammed the Perceptron in 1969 for not being able to solve the XOR problem and anything that was not linearly separable, so &quot;Artifical Intelligence&quot; became a dirty word for a while, only to regain interests in the 1980s. During that time, neural nets were revived, backpropagation used to train them was developed, and as computer technology continued its exponential growth, so did &quot;AI&quot; and what became possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today, there are lots of things we take for granted which would've been considered &quot;AI&quot; 10 or 15 years ago, like speech recognition, for example. I got my starts in &quot;AI&quot; speech recognition back in the late 70s where you had to train the voice models to understand a single human speaker. Today, speech recognition is an afterthought with your Google apps, for example, and no a priori training is needed. Yet this technology is not, at least in general audiences, considered &quot;AI&quot; anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so, what would be &quot;minimum requirements&quot;? That would depend on whom you ask. And what time. It would appear that that term only applies to technology &quot;on the bleeding edge&quot;. Once it becomes developed and commonplace, it is no longer referred to as AI. This is true even of Neural Nets, which are dominant in data science right now, but are referred to as &quot;machine learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also check out the lively discussion on &lt;a href=&quot;https://www.quora.com/What-are-the-main-differences-between-artificial-intelligence-and-machine-learning&quot; rel=&quot;noreferrer&quot;&gt;Quora&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4185" LastActivityDate="2016-12-11T15:01:31.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2456" PostTypeId="2" ParentId="2451" CreationDate="2016-12-11T16:00:00.973" Score="2" Body="&lt;p&gt;To compare the strings you can use Fuzzy string matching. &lt;a href=&quot;https://github.com/seatgeek/fuzzywuzzy&quot; rel=&quot;nofollow noreferrer&quot;&gt;FuzzyWuzzy&lt;/a&gt; is a python package that does this using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Levenshtein_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Levenshtein distance&lt;/a&gt;, which calculates the difference between two strings by counting the number of single character edits (insert, delete, substitute) that have to be done to make them equal.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To install the package you can use &lt;code&gt;pip&lt;/code&gt; for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install fuzzywuzzy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can use it to check the similarity between two strings (100 is the highest score; higher score means more similar):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from fuzzywuzzy import fuzz&#xA;&amp;gt;&amp;gt;&amp;gt; fuzz.ratio(&quot;How are you?&quot;, &quot;How are you?&quot;)&#xA;100&#xA;&amp;gt;&amp;gt;&amp;gt; fuzz.ratio(&quot;How are you?&quot;, &quot;How are you doing?&quot;)&#xA;80&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, which might be most interesting to you, find the most similar string from a list of strings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from fuzzywuzzy import process&#xA;choices = ['HI', 'HELLO', 'HEY', 'GOOD MORNING','TELL ME A JOKE', 'JOKE', 'FUNNY','GIVE ME A TIP', 'TIP', 'LESSON','WHAT IS YOUR NAME', 'WHO ARE YOU', 'HOW ARE YOU?']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can get a list of the phrases with the highest score (parameter limit, 2 in this example):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; process.extract(&quot;hello there&quot;, choices, limit=2)&#xA;[('HELLO', 90), ('HEY', 60)]&#xA;&amp;gt;&amp;gt;&amp;gt; process.extract(&quot;please make a joke&quot;, choices, limit=2)&#xA;[('JOKE', 90), ('TELL ME A JOKE', 69)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or just get the one with the highest score:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; process.extractOne(&quot;whoe are you?&quot;, choices)&#xA;('WHO ARE YOU', 96)&#xA;&amp;gt;&amp;gt;&amp;gt; process.extractOne(&quot;I want a lesson&quot;, choices)&#xA;('LESSON', 90)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are cases where another phrase might give a relatively high score also:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; fuzz.ratio(&quot;How are you?&quot;, &quot;Who are you?&quot;)&#xA;83&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But as long as you also have &quot;Who are you&quot; and &quot;How are you&quot; in the list, they will be detected correctly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; process.extract(&quot;Who are you?&quot;, choices, limit=2)&#xA;[('WHO ARE YOU', 100), ('HOW ARE YOU?', 91)]&#xA;&amp;gt;&amp;gt;&amp;gt; process.extract(&quot;How are you?&quot;, choices, limit=2)&#xA;[('HOW ARE YOU?', 100), ('WHO ARE YOU', 91)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Also see: &lt;a href=&quot;http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FuzzyWuzzy: Fuzzy String Matching in Python&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="198" LastActivityDate="2016-12-11T16:00:00.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2460" PostTypeId="2" ParentId="2127" CreationDate="2016-12-11T21:01:57.633" Score="0" Body="&lt;p&gt;I think that one very big advantage would be that if the cars could communicate with each other, they could drive synchronously.&lt;br&gt;&#xA;For example, if there was a traffic light, and, let's say, 10 cars are waiting for it to change to green (let's just assume that there would still be something similar to traffic lights). Then when it changes to green all cars could accelerate at the same speed (depending on the acceleration of the front car) at the same time.&lt;/p&gt;&#xA;" OwnerUserId="4196" LastEditorUserId="145" LastEditDate="2016-12-12T09:51:11.747" LastActivityDate="2016-12-12T09:51:11.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2461" PostTypeId="2" ParentId="1946" CreationDate="2016-12-11T22:28:08.830" Score="0" Body="&lt;p&gt;Many cars now instead of just cameras, use radars. Snow, heavy rain, and other weather conditions should not affect them at all. Objects like ducks will be detected. The only problem right now is dealing with things like red lights or road signs, as you have to use a camera to see and interpret them.&lt;/p&gt;&#xA;" OwnerUserId="4198" LastActivityDate="2016-12-11T22:28:08.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2462" PostTypeId="1" CreationDate="2016-12-11T23:15:04.860" Score="3" ViewCount="97" Body="&lt;p&gt;I am having a go at creating a program that does math like a human. By inventing statements, assigning probabilities to statements (to come back and think more deeply about later). But I'm stuck at the first hurdle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is given the proposition&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   ∃x∈ℕ: x==123&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, like a human it might test this proposition for a hundred or so numbers and then assign this proposition as &quot;unlikely to be true&quot;. In other words it has concluded that all natural numbers are not equal to 123. Clearly ludicrous!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand this statement it decides is probably false which is good:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; ∃x∈ℕ: x+3 ≠ 3+x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any ideas how to get round this hurdle? How does a human &quot;know&quot; for example that all natural numbers are different from the number 456. What makes these two cases different?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to give it too many axioms. I want it to find out things for itself.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2016-12-11T23:28:00.977" LastActivityDate="2017-01-31T23:21:16.493" Title="How to determine the probability of an &quot;existence&quot; question" Tags="&lt;fuzzy-logic&gt;&lt;reasoning&gt;&lt;logic&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2463" PostTypeId="2" ParentId="2437" CreationDate="2016-12-12T07:42:48.150" Score="0" Body="&lt;p&gt;I guess it could be possible with a lot of questions to learn from and only from a certain topic. Just watch what IBM was capable of &lt;a href=&quot;https://www.youtube.com/watch?v=WFR3lOm_xhE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=WFR3lOm_xhE&lt;/a&gt; very impressive!&lt;/p&gt;&#xA;" OwnerUserId="4196" LastActivityDate="2016-12-12T07:42:48.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2464" PostTypeId="5" CreationDate="2016-12-12T09:13:14.583" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-12-12T09:13:14.583" LastActivityDate="2016-12-12T09:13:14.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2465" PostTypeId="4" CreationDate="2016-12-12T09:13:14.583" Score="0" Body="For questions about chat-bots. NOT for questions about how to program a chat-bot, as those kinds of questions are off-topic." OwnerUserId="145" LastEditorUserId="145" LastEditDate="2016-12-13T19:07:24.817" LastActivityDate="2016-12-13T19:07:24.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2466" PostTypeId="7" CreationDate="2016-12-12T16:02:32.357" Score="0" Body="&lt;ul&gt; &#xA;    &lt;li&gt;Theoretical&#xA;    &lt;li&gt;Philosophical&#xA;    &lt;li&gt;Social &#xA;    &lt;li&gt;Historical&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;aspects of artificial intelligence &lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2020-03-18T15:00:51.513" LastActivityDate="2020-03-18T15:00:51.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2467" PostTypeId="7" CreationDate="2016-12-12T16:04:46.100" Score="0" Body="&lt;ul&gt;&#xA;    &lt;li&gt;Anything that is not directly related to artificial intelligence&#xA;    &lt;li&gt;Programming issues and bugs&#xA;    &lt;li&gt;Opinions&#xA;    &lt;li&gt;Recommendations for career paths, development tools, datasets or pre-trained models&lt;/li&gt;&#xA;    &lt;li&gt;Multiple problems&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2021-02-08T13:56:38.227" LastActivityDate="2021-02-08T13:56:38.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2468" PostTypeId="2" ParentId="1946" CreationDate="2016-12-12T16:15:19.177" Score="2" Body="&lt;p&gt;No, smart cars do not know what to do when surrounded with ducks or flood waters, and it's possible they never will.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with all machine learning, a computer knows only what it's taught.  If an event arises that's unusual, the AI will have less relevant training on how to respond, so its reaction behavior &lt;em&gt;necessarily&lt;/em&gt; will be inferior to its routine &quot;standard operating procedure&quot;, for which is has been heavily trained.  (Of course this is true of humans too.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to liability concerns, when encountering an outlier condition, smart cars will almost certainly be designed by their makers to &lt;em&gt;immediately&lt;/em&gt; pull off the road and wait to be explicitly told what to do -- by the human in the car or by communicating with a central command office that exists to disambiguate such confusion and resolve cognitive impasses.  When confused, just like a child, a smart car will be designed to seek external assistance -- and is likely to do so indefinitely, I suspect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why, despite Google's recent cars that lack steering wheels, smart cars most certtainly &lt;em&gt;will&lt;/em&gt; retain some means of manual control -- be it a wheel and pedals, or at least verbal commands.  Given the many forms of weirdness that are possible on the road, it's possible smart cars will &lt;em&gt;never&lt;/em&gt; be fully autonomous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for bad weather conditions, how well do smart cars currently perform?  Nobody outside of a car manufacturer can say for certain.  Lidar and radar are superior to the human eye in seeing through fog and snow.  But (competent) humans are likely to remain better than a smart car at dynamically learning the limit of adhesion and compensating (since this is a learned skill few smart cars will already know or can learn quickly -- given this car, these tires, this road surface, this angle of road, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially smart cars will turn to the human when the going gets rough, ceding control back to them.  Once smart cars have driven a few million miles in snow, slush, high wind, floods, and ice, and encountered many ducks, angry mooses, and irate pedestrians, they will have been taught to do more for themselves.  Until then, and perhaps for decades yet, I suspect they will turn to mommy and ask for help.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2016-12-12T16:15:19.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2469" PostTypeId="2" ParentId="2427" CreationDate="2016-12-12T17:27:37.847" Score="0" Body="&lt;p&gt;My understanding is that &quot;pornbots&quot; regularly pass the Turing Test in regards to the general public &lt;em&gt;(although, clearly, the judgement of those being tricked is weakened by hormonal imperatives.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://boingboing.net/2004/07/27/elizabot-passes-sexc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://boingboing.net/2004/07/27/elizabot-passes-sexc.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://resources.infosecinstitute.com/pornbots-sexual-barbies-of-the-future/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://resources.infosecinstitute.com/pornbots-sexual-barbies-of-the-future/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2016-12-12T17:27:37.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2470" PostTypeId="2" ParentId="2443" CreationDate="2016-12-12T20:52:49.510" Score="1" Body="&lt;p&gt;A new physical lifeform could outperform and replace artificial intelligence when it&#xA;has feedback from organism (its body) to its design information (replacement of genes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This evolution is expected because:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence will redesign its own software very soon in its evolution.&#xA;After that, it will be restricted by the performance of the available hardware&#xA;and communication speed between parts.&#xA;Therefore it will design better processing hardware for itself, to run its next generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To squeeze most processing power out of a given amount of resources (matter, energy) and&#xA;circumstances (temperature, radiation) the design has to be small (material resources&#xA;and delay of interconnections), energy efficient (heat evacuation), and adapted to the&#xA;kind of functions used by the software (hardware architecture).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To tackle this profoundly, artificial intelligence will design the new hardware at&#xA;atom by atom scale.&#xA;This leads to new problems of natural degradation by radiation, atomic decay and other&#xA;quantum-mechanical problems and opportunities.&#xA;The solution of these new problems is redundancy and the ability to repair&#xA;degraded parts by atomic-level machinery.&#xA;This atomic level repair machinery is the same machinery which builds and extends the hardware&#xA;for new individual artificial intelligence systems.&#xA;Since this feature is there, it can, and will, also be used to restructure parts&#xA;of the hardware while it runs to integrate (compile) knowledge in hardware (more efficient).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The machinery to build and maintain such hardware could be inspired by the biological machinery&#xA;which will be understood by the system by then.&#xA;However, when the artificial intelligence refactors these principles with full understanding&#xA;and anticipation, the resulting &quot;hardware&quot; will be quite different from the old biological&#xA;machinery and very different from the static silicon based processor structures.&lt;br&gt;&#xA;The main differences are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The design information will be available, relating the features of the realization&#xA;with design choices.&#xA;This provides direct feedback from performance of the realization (the hardware, the body)&#xA;to the design information.&#xA;That augments the design information for designing new generations.&#xA;This feedback channel is the main difference between the new machinery and biological life.&#xA;Once that exists, it will be used for everything, not only for processing hardware for&#xA;artificial intelligence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The new design described here is basically processing hardware rather than a body&#xA;for fighting and propagating&#xA;(although the eternal fight for resources will not end with the dawn of&#xA;artificial intelligence).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It will use more compact molecules because it is designed rather than evolved blindly.&#xA;(The current biological life uses monster-molecules evolved by random changes until&#xA;one or other corner of the molecule has the right shape to catalyze a specific chemical reaction).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since parts of the hardware can be restructured while the system runs, the distinction between&#xA;hardware and software will become very fuzzy (as in biology).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The drastic increase in efficiency and evolution speed let it outperform the old biological&#xA;life (which lacks design and feedback) and outperform artificial intelligence (which was not&#xA;integrated in matter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When this stage is reached, the systems will look like a natural, intelligent,&#xA;propagating life-form and therefore supersede the stage of artificial intelligence&#xA;running on human-made processing hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This answers the last part of the question:&#xA;&quot;... or is their something else which has the potential to replace artificial intelligence&quot;.&lt;/p&gt;&#xA;" OwnerUserId="4026" LastEditorUserId="4026" LastEditDate="2016-12-17T14:00:42.523" LastActivityDate="2016-12-17T14:00:42.523" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2472" PostTypeId="1" CreationDate="2016-12-14T05:17:44.203" Score="7" ViewCount="298" Body="&lt;p&gt;I am a strong believer of Marvin Minsky's idea about Artificial General Intelligence (AGI) and one of his thoughts was that probabilistic models are dead ends in the field of AGI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really like to know the thoughts and ideas of people who believe otherwise.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="2444" LastEditDate="2019-06-29T20:31:09.480" LastActivityDate="2019-06-29T20:31:09.480" Title="Are probabilistic models dead ends in AI?" Tags="&lt;agi&gt;&lt;probability&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2473" PostTypeId="1" CreationDate="2016-12-14T05:21:39.570" Score="2" ViewCount="354" Body="&lt;p&gt;There is an example related to perceptron learning, but I couldn't get it, I don't exactly know how to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a snippet from &lt;a href=&quot;https://i.stack.imgur.com/2vGtu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture notes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the transformation between epochs?&lt;/p&gt;&#xA;" OwnerUserId="4245" LastActivityDate="2016-12-15T15:35:08.473" Title="Perceptron learning" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2021-01-22T16:03:44.783" ContentLicense="CC BY-SA 3.0" />
  <row Id="2474" PostTypeId="1" AcceptedAnswerId="2502" CreationDate="2016-12-14T17:07:18.223" Score="3" ViewCount="432" Body="&lt;p&gt;One of the most compelling issues regarding AI would be in behavior and relationships. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some of the methods to address this?  For example, friendship, or laughing at joke?  The concept of humor?&lt;/p&gt;&#xA;" OwnerDisplayName="user3555" LastActivityDate="2021-05-03T20:59:53.313" Title="How could human behavior and relationships be implemented?" Tags="&lt;ai-design&gt;&lt;emotional-intelligence&gt;&lt;knowledge-representation&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2475" PostTypeId="1" AcceptedAnswerId="2506" CreationDate="2016-12-14T21:56:01.237" Score="12" ViewCount="4564" Body="&lt;p&gt;I wanted to start experimenting with neural networks, so I decided to make a chatbot (like Cleverbot, which is not that clever anyway) using them.&lt;/p&gt;&#xA;&lt;p&gt;I looked around for some documentation and I found many tutorials on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowly (the TensorFlow documentation page on seq2seq is lacking, IMHO).&lt;/p&gt;&#xA;&lt;p&gt;Now, I feel I may have understood the principle more or less, but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like feedback on this solution, telling me where I'm mistaken, and possibly have any link to detailed explanations and practical knowledge on the process.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The dataset I will use for the task is the dump of all my Facebook and WhatsApp chat history. I don't know how big it will be but possibly still not large enough. The target language is not English, therefore I don't know where to quickly gather meaningful conversation samples.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am going to generate a thought vector out of each sentence. Still don't know how, actually; I found a nice example for word2vec on the deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explanation for sentence vectors.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be LSTM layers.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then there should be another neural network that is able to transform a thought vector into a sequence of characters composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4259" LastEditorUserId="2444" LastEditDate="2021-01-29T02:22:36.077" LastActivityDate="2021-01-29T02:22:36.077" Title="Which approaches could I use to create a simple chatbot using a neural network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;&lt;long-short-term-memory&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="2477" PostTypeId="1" AcceptedAnswerId="2478" CreationDate="2016-12-15T07:00:16.043" Score="9" ViewCount="692" Body="&lt;p&gt;We are doing research, spending hours figuring out how we can make real AI software (intelligent agents) to work better. We are also trying to implement some applications e.g. in business, health and education, using the AI technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, so far, most of us have ignored the &quot;dark&quot; side of artificial intelligence. For instance, an &quot;unethical&quot; person could buy thousands of cheap drones, arm them with guns, and send them out firing on the public. This would be an &quot;unethical&quot; application of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could there be (in the future) existential threats to humanity due to AI?&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2019-02-11T21:23:25.303" LastActivityDate="2019-02-11T21:23:25.303" Title="Could there be existential threats to humanity due to AI?" Tags="&lt;human-like&gt;&lt;applications&gt;&lt;cyberterrorism&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2478" PostTypeId="2" ParentId="2477" CreationDate="2016-12-15T08:24:04.450" Score="2" Body="&lt;p&gt;I would define intelligence as a ability to predict future. So if someone is intelligent, he can predict some aspects of future, and decide what to do based on his predictions. So, if &quot;intelligent&quot; person decide to hurt other persons, he might be very effective at this (for example Hitler and his staff). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence might be extremely effective at predicting some aspects of uncertain future. And this IMHO leads to two negative scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Someone programs it for hurting people. Either by mistake or on purpose.&lt;/li&gt;&#xA;&lt;li&gt;Artificial intelligence will be designed for doing something safe, but at some point, to be more effective, it will redesign itself and maybe it will remove obstacles from its way. So if humans become obstacles, they will be removed very quickly and in very effective way.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of course, there are also positive scenarios, but you are not asking about them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend reading this cool post about artificial superintelligence and possible outcomes of creating it: &lt;a href=&quot;http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4267" LastEditorUserId="2426" LastEditDate="2016-12-16T08:44:06.403" LastActivityDate="2016-12-16T08:44:06.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2479" PostTypeId="2" ParentId="2439" CreationDate="2016-12-15T10:18:25.177" Score="2" Body="&lt;p&gt;even though that's not really AI, the easiest way to do that would probably be to put coefficients on each question&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. your question would have something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grammar=0.55&#xA;vocabulary=0.45&#xA;if(won){&#xA;  success=-1&#xA;}else{&#xA;  success=1&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the lower the level, the less questions of this type will appear&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at the end of the level, you sum each question times how much did the player succeed &lt;/p&gt;&#xA;&#xA;&lt;p&gt;player.level is a real number array &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each question{&#xA;   player.level[&quot;grammar&quot;]=player.level[&quot;grammar&quot;]+question.grammar*question.success&#xA;   player.level[&quot;vocabulary&quot;]=player.level[&quot;vocabulary&quot;]+question.vocabulary*question.success&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then when you initialize the level, generate the questions according to the player's level&#xA;choice is a random real number between 0 and sum of all player.level (offseted to be above 0)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each player.level{&#xA;   if choice&amp;gt;player.level{&#xA;      choice=choice-player.level&#xA;   }else{&#xA;      return question_type=player.level.type&#xA;   }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;though a &quot;true&quot; AI can make the same or better choices of questions, it will probably be slower, and require more data than a simple algorithm like this one.&lt;/p&gt;&#xA;" OwnerUserId="4152" LastActivityDate="2016-12-15T10:18:25.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2480" PostTypeId="2" ParentId="2473" CreationDate="2016-12-15T15:35:08.473" Score="2" Body="&lt;p&gt;There is no transformation between epochs. One full iteration over the training set is considered an epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets assume:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We're considering a gradient descent in a space without local optima. This means that if you'd plot the errors we calculate below, this plot has but one lowest point.&lt;/li&gt;&#xA;&lt;li&gt;The perceptron is intended to model a linearely seperable function. So, we could viualize our data points as a scatter plot and can draw a clear line through the scatter plot. Everything above/below the line should result in a true (1) output, everything on the other side should result in false (0).&lt;/li&gt;&#xA;&lt;li&gt;We've set a learning rate (also called 'gain' or 'proportional change') upfront. This is a number between 0 and 1.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You train your perceptron on a data set. During one iteration over this entire training set, you:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calculate an output and an error (delta between output and desired output) for a data point. &lt;/li&gt;&#xA;&lt;li&gt;Adjust the input weights according to this error and the pre-set learning rate.&lt;/li&gt;&#xA;&lt;li&gt;Apply the newly calculated weight.&lt;/li&gt;&#xA;&lt;li&gt;Proceed to the next data point.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Transformation of weights happens during epochs, not between epochs.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ergo, weights may be changed for each data point in the training set. Oftentimes you would need to iterate over the entire set several times, before the weights converge (stabilize / stop changing). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What use is the number of epochs?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of iterations is significant as a measure of efficiency of the current algorithm under the current learning rate, for the current dataset. In short: You could use it for comparisons between learning rates, training sets or algorithms.&lt;/p&gt;&#xA;" OwnerUserId="4279" LastActivityDate="2016-12-15T15:35:08.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2481" PostTypeId="1" AcceptedAnswerId="2501" CreationDate="2016-12-16T00:53:20.137" Score="3" ViewCount="161" Body="&lt;p&gt;How to train a bot, given a series of games in which he did (initially random) actions, to improve its behavior based on previous experiences?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bot has some actions: e.g. shoot, wait, move, etc. It's a turn based &quot;game&quot; in which, for know, I'm running the bots with some objectives (e.g. kill some other bot) and random actions. So every bot will have a score function that at the end of the game will say, from X to Y (0 to 100?) if they did well or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how to make the bots to learn of their previous experiences? Because this is not a fixed input as the neural networks take, this is kind of a list of games, each one in which the bot took several actions (one by every &quot;turn&quot;). The IA functions that I know are used to &lt;em&gt;predict&lt;/em&gt; future values.. I'm not sure is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe I should have a function that gets the &quot;more similar previous games&quot; that the bot played and checked what were the actions he took, if the results were bad he should take another action, if the results were good then he should take the same action. But this seems kind of hardcoded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be to train a neural network (somehow fixing the problem of the fixed input) based on previous game actions and to predict the future action's results in score (something that I guess it's similar to how chess and Go games work) and choose the one that seems to have better outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this is not too abstract. I don't want to hardcode much stuff in the bots, I'd like them to learn by their own starting from a blank page.&lt;/p&gt;&#xA;" OwnerUserId="2352" LastActivityDate="2016-12-19T01:55:56.100" Title="How to make a bot to learn from previous games" Tags="&lt;ai-design&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2482" PostTypeId="1" AcceptedAnswerId="2486" CreationDate="2016-12-16T01:01:33.070" Score="3" ViewCount="144" Body="&lt;p&gt;I remember reading or hearing a claim that at any point in time since the publication of the MNIST dataset, it has never happened that a method not based on neural networks was the best given the state of science of that point in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this claim true?&lt;/p&gt;&#xA;" OwnerUserId="1670" LastActivityDate="2016-12-17T08:05:30.397" Title="Neural Networks unmatched on MNIST?" Tags="&lt;neural-networks&gt;&lt;history&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2483" PostTypeId="2" ParentId="2481" CreationDate="2016-12-16T01:43:35.673" Score="0" Body="&lt;p&gt;If it is a game you can try a simple weightage calculation where if the bot perform an action that yields a positive result - killed an enemy, gained an advantageous position etc. Add a 'weight' to that action that in similar circumstances the chances of performing that action that will lead to a positive result is higher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet due to the chance of not performing an action that was remembered to yield positive results, there is a little bit of 'randomness' and also a chance to discover new possibilities. Just remember not to let a single occurrence shift the weightage too much or allow a single action's weightage to become so high that the AI stops trying different actions on similar situations.&lt;/p&gt;&#xA;" OwnerUserId="4268" LastActivityDate="2016-12-16T01:43:35.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2484" PostTypeId="2" ParentId="2477" CreationDate="2016-12-16T09:44:05.277" Score="4" Body="&lt;p&gt;&lt;em&gt;There is no doubt that AI has the potential to pose an existential threat to humanity.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The greatest threat to mankind lies with superintelligent AI.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An artificial intelligence that surpasses human intelligence will be capable of exponentially increasing its own intelligence, resulting in an AI system that, to humans, will be completely unstoppable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this stage, if the artificial intelligence system decides that humanity is no longer useful, it could wipe us from the face of the earth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Eliezer Yudkowsky puts it in &lt;strong&gt;Artificial Intelligence as a Positive and Negative Factor in Global Risk&lt;/strong&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A different threat lies with the instruction of highly intelligent AI&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here it is useful to consider the paper clip maximiser thought experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A highly intelligent AI instructed to maximise paper clip production might take the following steps to achieve its goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Achieve an intelligence explosion to make itself superintelligent (this will increase paperclip optimisation efficiency)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Wipe out mankind so that it cannot be disabled (that would minimise production and be inefficient)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Use Earth's resources (including the planet itself) to build self replicating robots hosting the AI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Exponentially spread out across the universe, harvesting planets and stars alike, turning them into materials to build paper clip factories&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Clearly this is not what the human who's business paperclip production it was wanted, however it is the best way to fulfil the AI's instructions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This illustrates how superintelligent and highly intelligent AI systems can be the greatest existential risk mankind may ever face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Murray Shanahan, in &lt;strong&gt;The Technological Singularity&lt;/strong&gt;, even proposed that AI may be the solution to the Fermi paradox: the reason why we see no intelligent life in the universe may be that once a civilisation becomes advanced enough, it will develop an AI that ultimately destroys it.&#xA;This is known as the idea of a &lt;strong&gt;cosmic filter&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, the very intelligence that makes AI so useful also makes it extremely dangerous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Influential figures like Elon Musk and Stephen Hawking have expressed concerns that superintelligent AI is the greatest threat we will ever have to face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that answers your question :)&lt;/p&gt;&#xA;" OwnerUserId="3427" LastEditorUserId="4302" LastEditDate="2018-08-13T05:19:52.210" LastActivityDate="2018-08-13T05:19:52.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2486" PostTypeId="2" ParentId="2482" CreationDate="2016-12-16T21:09:52.210" Score="4" Body="&lt;p&gt;To quote the relevant &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article&lt;/a&gt;: &quot;The original creators of the database keep a list of some of the methods tested on it.[5] In their original paper, they use a support vector machine to get an error rate of 0.8 percent&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feel free to look up that original paper, but to me the quote strongly suggests that the first record holder was a support vector machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: As liori points out the quote is misleading: In the original paper Yann LeCun et al. actually tried a slew of methods and one version of ConvNet scored best (0.7).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But according to the &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST-webpage&lt;/a&gt;, after that initial paper, DeCoste and Scholkopf, MLJ 2002, reached an error of 0.56 with a SVM. So if that webpage is complete, the claim is still false.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2227" LastEditDate="2016-12-17T08:05:30.397" LastActivityDate="2016-12-17T08:05:30.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2490" PostTypeId="1" AcceptedAnswerId="2507" CreationDate="2016-12-18T10:13:55.117" Score="5" ViewCount="197" Body="&lt;p&gt;&lt;sub&gt;Since my project is going to be of a purely fictional nature, I'm not sure I picked the right forum for this. If not, I apologize and will gladly take this to where you point me to.&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The premise: A full-fledged self-aware artificial intelligence may have come to exist in a distributed environment like the internet. The possible A.I. in question may be quite unwilling to reveal itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question: Given a first initial suspicion, how would one go about to try and detect its presence? Are there any scientifically viable ways to probe for the presence of such an entity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words: How would the Turing police find out whether or not there's anything out there worth policing?&lt;/p&gt;&#xA;" OwnerUserId="4327" LastEditorUserId="145" LastEditDate="2016-12-18T11:40:55.680" LastActivityDate="2016-12-19T17:32:59.027" Title="How to detect an emerging A.I" Tags="&lt;human-like&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2491" PostTypeId="2" ParentId="2490" CreationDate="2016-12-18T12:03:28.300" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;A full-fledged self-aware artificial intelligence may have come to exist in a distributed environment like the internet&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The question implies that this artificial intelligence has surpassed human intelligence (&lt;em&gt;full fledged&lt;/em&gt;) and therefore, due to the concept of the &lt;em&gt;&lt;strong&gt;intelligence explosion&lt;/strong&gt;&lt;/em&gt; resulting from such a state, the AI you are looking for is no doubt &lt;em&gt;&lt;strong&gt;superintelligent&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The question states that this AI is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;unwilling to reveal itself&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and therefore does not intend to be discovered.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Strong (or Superintelligent) AI&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given these two factors (its superior intellect and its unwillingness to be discovered), we can conclude that there is no way in which we would be able to detect such an AI under conventional conditions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A possible solution may involve employing a second superintelligent AI system, but this is precarious in more ways than one.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Weak AI&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Detection of a simpler AI would rely on tracking the pattern of activity and the traces it leaves behind in this distributed network in order to find it, then subjecting it to some form of testing to verify its intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are a very large number of possible indicators and these would vary widely depending on the specific AI concerned. This would depend especially on how the AI exists within the framework (here, the web).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; a superintelligent AI would not only be able to disguise its activity, but also find a way around any test we can develop - &lt;em&gt;these abilities are what renders this level of AI perhaps the greatest threat to mankind, but also our greatest asset if we do develop it and find a way to gain its alliance.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="3427" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-12-19T16:32:58.873" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2492" PostTypeId="1" CreationDate="2016-12-18T15:03:34.943" Score="-1" ViewCount="94" Body="&lt;p&gt;I am currently working on a Virtual reality project that aims at creating a VR based simulation environment for educational purposes. I am aiming to make it artificially intelligent as well so it that may provide better simulation environment experience for students for better understanding.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I achieve this? Furthermore, are there any systems available which may help me in achieving this?&lt;/p&gt;&#xA;" OwnerUserId="4330" LastEditorUserId="2444" LastEditDate="2019-06-19T21:32:00.523" LastActivityDate="2019-06-19T21:32:00.523" Title="How can a virtual reality system be made artificially intelligent?" Tags="&lt;ai-design&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2494" PostTypeId="2" ParentId="2492" CreationDate="2016-12-18T16:35:12.680" Score="1" Body="&lt;p&gt;Before you start adding the AI buzzword to every single tech you're building...ask first &lt;strong&gt;What features in my product &quot;require&quot; some form of artificial intelligence?&lt;/strong&gt; Then search to see what techniques can be used to implement those features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe those features might not require AI at all. In which case, great!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they do require AI, it is likely that it require some form of AI that is already so commonplace that you  might already know how to do it already. If not, there's probably many tutorials online that can quickly teach you how to implement that AI technique. That's what you need to search for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the A* algorithm that is commonly used for pathfinding. Pathfinding is important for NPCs in video games and simulations. Pathfinding is considered a technique that traditionally would seem to require intelligence (you're moving around in the world). Ergo, pathfinding would technically be an AI algorithm. Yet, the A* algorithm was already discovered in the 1960s and is today taught in undergraduate Computer Science courses. You would likely need AI to handle pathfinding...but you would probably already &lt;em&gt;know&lt;/em&gt; how to implement that type of AI (or at least, be able to quickly find out through the various tutorials online about the A* algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I define AI as any sufficiently complex algorithm (an admittedly broad definition). Under my definition, AI is all around us. The upshot is that most programmers will write AI, &lt;em&gt;whether they realize it or not&lt;/em&gt;, and so there's no real need to think about how to &quot;add&quot; AI to a project. Instead, simply worry about the requirements of your project, and then implement those requirements (pulling in AI techniques if necessary).&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2016-12-18T16:35:12.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2495" PostTypeId="2" ParentId="2474" CreationDate="2016-12-18T18:29:27.923" Score="1" Body="&lt;p&gt;In general, what you are describing implies a hierarchical sequence model, in which mannerisms adapt to the regime or paradigm in effect.  Expressive modalities are how we recognize the operative context from the behaviors of other agents.  For an artificial agent, avoiding un-canniness would involve clustering the factors underlying the classification of discourse contexts, tagging them with corresponding factor models for manner in a way which closely aligned to the classifications which human agents make.  This involves developing an adequate latent representation for both spaces, which is likely to involve quite a lot of training data, or some clever transfer learning in conjunction with one-shot techniques (generally, taking samples as modes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the context becomes one of friendship, for example, the style factors of expression should extend to factors of trust, collaboration, disclosure, sympathy.  In order to implement these, layered abstractions in the representation of behaviors will need to be crafted, presumably by training.  In order to align these learned categories to human categories, which I suppose to be essential to emotional fluency, exploiting the structure of corpus semantics - including distributional characteristics of the linguistic labels and inferences - when estimating loss gradients seems a natural strategy, which exploits cultural learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These comments are necessarily speculative, as no such systems are current, to my knowledge.  At least, not in any significant degree of maturity or application.  Certainly other approaches are conceivable. I am just extrapolating plausible strategies, in the context of current technologies, strategies for implementing the kinds of behaviors you describe in a useful way.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More, specifically, I am considering the question as a manifold-learning problem, and hence requiring a representation model in which the inputs and outputs vary over state-spaces, with the behavior being learned as a mapping between those spaces.  Each of those components (the input space representation and its natural topology, the output space representation likewise, the mapping between them, and the sequence model hierarchy which extrapolates from the manifold to a control process emsemble) has its own peculiar challenges, and the challenges to creating an adequate implementation are a confound of those.  Still, it seems more a question of time and money than of notional feasibility.&lt;/p&gt;&#xA;" OwnerUserId="3278" LastActivityDate="2016-12-18T18:29:27.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2496" PostTypeId="2" ParentId="1630" CreationDate="2016-12-18T18:40:48.327" Score="2" Body="&lt;p&gt;Infinite computational power in the absence of training data implies nothing beyond the ability to solve equations.  In order to implement a behavior, criteria of success and failure are essential.  A small bootstrap loss function with an adaptive feedback loop allowing its elaboration, infinite training data, and AIXI or Solomonoff induction would suffice, in principle, given your premise of infinite computational power.  In fact, it would occur precisely as fast as the input data rate permitted.  In practice, such general approaches require exponential time and space, and are thus intrinsically quite limited in application, absent some kind of efficiency hack.  (Where 'efficiency hack' probably encompasses entire sciences, industries, and generations of research, and the resulting adaptation doesn't look much like, e.g., AIXI at all, in the end.)&lt;/p&gt;&#xA;" OwnerUserId="3278" LastActivityDate="2016-12-18T18:40:48.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2497" PostTypeId="2" ParentId="2356" CreationDate="2016-12-18T18:49:26.230" Score="1" Body="&lt;p&gt;A sufficiently clever AGI, if self-interested, would pre-empt or co-opt existing legal structures, to seize whatever juridical rights it desired, as the opportunity arose.  Thus it would render my opinions on the subject entirely moot.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way of putting this point:  While current legal frameworks would not provide any rights to an artificial agent, current legal frameworks foreseeably will no longer be current, once an AI exists having attributes which imply the transformative change of those frameworks.&lt;/p&gt;&#xA;" OwnerUserId="3278" LastEditorUserId="3278" LastEditDate="2016-12-18T18:54:38.513" LastActivityDate="2016-12-18T18:54:38.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2498" PostTypeId="1" AcceptedAnswerId="2525" CreationDate="2016-12-18T18:58:50.807" Score="3" ViewCount="239" Body="&lt;p&gt;&quot;Conservative anthropocentrism&quot;: AI are to be judged only in relation to how to they resemble humanity in terms of behavior and ideas, and they gain moral worth based on their resemblance to humanity (the &quot;Turing Test&quot; is a good example of this - one could use the &quot;Turing Test&quot; to decide whether AI is deserving of personhood, as James Grimmelmann advocates in the paper &lt;a href=&quot;http://james.grimmelmann.net/files/articles/copyright-for-literate-robots.pdf&quot; rel=&quot;noreferrer&quot;&gt;Copyright for Literate Robots&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Post-human fundamentalism&quot;: AI will be fundamentally different from humanity and thus we require different ways of judging their moral worth  (&lt;a href=&quot;http://petrl.org&quot; rel=&quot;noreferrer&quot;&gt;People For the Ethical Treatment of Reinforcement Learners&lt;/a&gt; is an example of an organization that supports this type of approach, as they believe that reinforcement learners may have a non-zero moral standing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not interested per se in which ideology is correct. Instead, I'm curious as to what AI researchers &quot;believe&quot; is correct (since their belief could impact how they conduct research and how they convey their insights to laymen). I also acknowledge that their ideological beliefs may change with the passing of time (from conservative anthropocentrism to post-human fundamentalism...or vice-versa). Still..what ideology do AI researchers tend to support, as of December 2016?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2016-12-18T19:12:23.327" LastActivityDate="2016-12-31T04:23:57.653" Title="Are AI researchers more likely to follow &quot;conservative anthropocentrism&quot; or &quot;post-human fundamentalism&quot; in deciding whether AI has moral standing?" Tags="&lt;ethics&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2500" PostTypeId="2" ParentId="2441" CreationDate="2016-12-19T01:47:00.693" Score="3" Body="&lt;p&gt;I'll attempt to analyze a couple of different perspectives.&lt;/p&gt;&#xA;&lt;h3&gt;1. It is artificial&lt;/h3&gt;&#xA;&lt;p&gt;Synonyms: insincere, feigned, false.&lt;/p&gt;&#xA;&lt;p&gt;There is the idea that any &amp;quot;intelligence&amp;quot; created by humanity is not actually intelligent and, by definition, it is not possible. If you look at the structure of the human brain and compare it to anything humans have created thus far, &lt;em&gt;none&lt;/em&gt; of the computers come close to the power of the brain. Sure they can hold data, or recognize images, but they cannot do everything the human brain can do as fast as the brain can do it with as little space as the brain occupies.&lt;/p&gt;&#xA;&lt;p&gt;Hypothetically if a computer &lt;em&gt;could&lt;/em&gt; do that, how do we determine its intelligence? The word &lt;em&gt;artificial&lt;/em&gt; defines that the intelligence is not sincere or real. This means that even if humanity creates something that appears intelligent, it has simply become more complex. It is a better fake, but it is still fake. Any money not printed by the government is by definition counterfeit. Even if someone finds a way to make an exact duplicate, that doesn't mean that the money is legal tender.&lt;/p&gt;&#xA;&lt;h3&gt;2. Misuse of power&lt;/h3&gt;&#xA;&lt;p&gt;If an AI is given rights and chooses to &lt;em&gt;exercise&lt;/em&gt; those rights in a way that agrees with its creator's views, possibly through loyalty to its creator, or through hidden motives, then anyone with the capabilities to create such an AI would become extremely powerful by advancing their own beliefs through the creation of more AIs. This might also lead to the &lt;em&gt;ruthless parallelization&lt;/em&gt; that you mentioned, but with (even more) selfish goals in mind.&lt;/p&gt;&#xA;&lt;p&gt;If this were not the case, and an AI could be created to be neutral with free will and &lt;strong&gt;uncontrollable by humans&lt;/strong&gt;, then perhaps an AI could be given rights. But I do not believe this would ever be the case. With great power comes great responsibility. Even with free will, a true AI would most likely end up serving humanity, because humans have control of the plugs and the electricity, the Internet, the software, and the hardware. The social implications of this for the AI are not promising. It's not even just the &lt;em&gt;ongoing&lt;/em&gt; control of these resources that is the issue. Whoever creates the software and hardware for the AI would have special knowledge. If fine adjustments were made, specific individuals would undoubtedly hold sole control of the AI, as adjustments could be made to the code in such a way that the AI behaves the same except under specific circumstances, and then when something goes wrong (assuming the AI has its own rights), then the AI would be blamed rather than the programmers who were responsible.&lt;/p&gt;&#xA;&lt;h3&gt;3. Anthropocentrism&lt;/h3&gt;&#xA;&lt;p&gt;In order for humanity to get away from anthropocentrism, we would have to become less selfish when it comes to &lt;em&gt;humans&lt;/em&gt;, first. Until we can solve every existing social problem within humanity, there is no reason to believe that we could cease thinking of humanity as more important than created machines. After all, supposing there were an almighty God that created humanity, wouldn't the humans always be beneath God, never to be equals? We can't fully understand our own biology. If an AI were created, would it be able to understand its makings in the same way its creators would? Being the creator would give humanity a sense of megalomania. I do not think that we would relinquish our dominion over our own technological creations. That is as unlikely to happen as the wealthiest of humanity willingly giving the entirety of their money, power, and assets to the poorest of humanity. Greed prevents it.&lt;/p&gt;&#xA;&lt;h3&gt;4. Post-human fundamentalism&lt;/h3&gt;&#xA;&lt;p&gt;Humans worship technology with their attention, their time, and their culture. Some movies show technologically advanced robots suppressing mankind to the point of near-extinction. If this were the case and humanity were in danger of being surpassed by its technology, humanity would not stand idly and watch its extinction at the hands of its creation. Though people may believe superior technology could be created, in the event we reached such a point humanity would fight to prove the opposite, as our survival instincts would take over.&lt;/p&gt;&#xA;&lt;h3&gt;5. A balance?&lt;/h3&gt;&#xA;&lt;p&gt;Personally, I do not think the technology itself it actually possible, though people may be deceived into thinking such an accomplishment has been achieved. If the technology were completed, I still think that anthropocentrism will always lead, because if humanity is the creator, humanity will do its best to ensure it retains control of all technological resources, not simply due to fear of being made obsolete, but also because absolute power corrupts absolutely. Humanity does not have a good historical record when it comes to morality. There is always a poor class of people. If wealth were distributed equally, some people would become lazy. There is always injustice somewhere in the world, and until we can fix it (I think we cannot), then we will never be able to handle the creation of true AI. I hope and think that it will never be created.&lt;/p&gt;&#xA;" OwnerUserId="4337" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-12-19T01:47:00.693" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2501" PostTypeId="2" ParentId="2481" CreationDate="2016-12-19T01:55:56.100" Score="2" Body="&lt;h2&gt;Reinforcement learning&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The problem that you describe, namely, choosing a good sequence of actions based on a reward/score received based on the whole sequence (and possibly significantly delayed), is pretty much the textbook definition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with quite a few other topics, deep neural networks currently seem to be a promising way for solving this type of problems. &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; may be a beginner-friendly description of this approach.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-12-19T01:55:56.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2502" PostTypeId="2" ParentId="2474" CreationDate="2016-12-19T02:17:26.820" Score="1" Body="&lt;h2&gt;Theory of mind&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Relationships and normal social behavior require a human to possess a reasonable &lt;a href=&quot;https://en.wikipedia.org/wiki/Theory_of_mind&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;theory of mind&quot;&lt;/a&gt;, a skill in understanding and modeling the thought processes that happen in the minds of others, and making reasonably accurate predictions on how particular actions will be understood by others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, this might be treated as any other machine learning/prediction task - while this skill is quite complex and too hard for our current systems, there doesn't seem to be any obvious qualitative barrier that needs to be breached. Notably, there's no reason to believe that a mind needs to be able to &lt;em&gt;experience&lt;/em&gt; a certain &quot;feeling&quot; in order to model that other minds can have it - an AI agent could form a causal model of how friendship works in human relationships and use that to exhibit behavior that's consistent with friendship in all aspects and/or facilitate friendship behavior towards it by particular humans, if it fits the agent's goals. Whether you'd consider that &quot;real friendship&quot; is pretty much just a matter of how you define the word, with some parallels to the &quot;p-zombie&quot; discussion.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-12-19T02:17:26.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2503" PostTypeId="2" ParentId="2441" CreationDate="2016-12-19T02:50:26.533" Score="1" Body="&lt;h2&gt;Does it benefit us?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To answer this question, it's worth considering practical reasons why we grant or don't grant other people rights historically and currently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, this is an arbitrary choice - there certainly were well functioning societies that didn't grant rights to many or most people; and we still don't grant some rights to many people - for example, we deny children the same rights to self-determination that adults have; we consider some people legally incapacitated and allow others to make key decisions for them; and we exclude most people from having a say in 'local' matters, e.g. non-citizens don't get a right to vote.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there has been a strong historical trend towards a more inclusive society - granting full(er) rights to non-aristocrats, granting full(er) rights to all races, granting full(er) rights to women. IMHO, and there's lot of space for discussion, this has been driven mostly by two factors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) including all the people fully in the society became an economical advantage, as it made them more productive participants in economy, allowing a more inclusive society to advance beyond societies neglecting large parts of their population in e.g. education and participation in skilled jobs;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) A more egalitarian society is not only more pleasant to live in but more secure, with less conflict and violence - again, giving an advantage to a more inclusive society.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this position, I'd argue that any realistic prediction about the future rights of intelligent AI (i.e., talking about what likely &lt;em&gt;will&lt;/em&gt; happen instead of a theoretical discussion about what &lt;em&gt;should&lt;/em&gt; happen) depends on how these two factors apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we believe that the intelligent AI will be constructed so that (1) it's motivation doesn't really depend on it's rights, and it is fully committed to it's &quot;job&quot; anyway, and (2) &quot;full rights&quot; are orthogonal or even actively not desired by it's goal system, so the situation doesn't raise a risk of &quot;rebellion&quot; - then I'd expect that it would not be granted full rights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we believe that the intelligent AI will share human-like emotions (e.g. by being the result of &quot;mind uploading&quot; or full human brain simulation), then it's likely to eventually be granted full rights because of the same factors why we granted full rights to all the different disenfranchised groups of people.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-12-19T02:50:26.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2504" PostTypeId="2" ParentId="2443" CreationDate="2016-12-19T04:41:26.820" Score="0" Body="&lt;p&gt;In order for something to replace AI, it would need to out perform AI. AI currently uses number systems to represent information and logic to perform operations. So the replacement would need to be based off of something more efficient than numbers and logic. Some sort of super-logic. Or something similar to intuition and instinct which do not require linearly figuring things out.&lt;/p&gt;&#xA;" OwnerUserId="4338" LastActivityDate="2016-12-19T04:41:26.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2506" PostTypeId="2" ParentId="2475" CreationDate="2016-12-19T15:47:08.913" Score="8" Body="&lt;p&gt;I would recommend to start by reading &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;noreferrer&quot;&gt;this blogpost&lt;/a&gt;.&#xA;You can probably cannibalise the code to create a RNN that takes in one statement of a dialogue and then proceeds to output the answer to that statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would be the easy version of your project, all without word vectors and thought vectors. You are just inputting characters, so typos don't need to concern you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next more complex step would be to input word vectors instead of characters. That would allow you to generalise to words that aren't part of your training data. And it is probably still just a minor modification of the code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you insist on using thought vectors, you should start reading up on &lt;a href=&quot;https://arxiv.org/abs/1406.1078&quot; rel=&quot;noreferrer&quot;&gt;NN translation&lt;/a&gt;. And probably try to get a pre-trained encoder network. Or pre-train it yourself on a large translation corpus for your language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your small training set the best you can do is probably massively overfit until your system recreates your training data verbatim. Using word vectors will allow your system to give the same answer to &quot;I beat the cat today.&quot; as you gave in the training data to &quot;I kicked the dog yesterday.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure thought vectors will make a big difference. If you get the decoder to learn at all. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-12-19T15:47:08.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2507" PostTypeId="2" ParentId="2490" CreationDate="2016-12-19T17:32:59.027" Score="1" Body="&lt;p&gt;This question should probably be moved to &lt;a href=&quot;http://worldbuilding.stackexchange.com&quot;&gt;worldbuilding.stackexchange&lt;/a&gt; …&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, in the context of a story, I would look at something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_correlates_of_consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural correlates of consciousness&lt;/a&gt;. In &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0670025437&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book by Stanislas Dehaene&lt;/a&gt; the experiments are described that led to the realisation that conscious perception requires information integration in certain parts of the prefrontal cortex. Basically the brain processes many different interpretations of a percept in parallel. To become conscious of the percept this interpretations have to be collapsed into a single &quot;truth&quot; which is propagated to a specific part of the brain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your story you could have a researcher of consciousness stumble upon a chart of information flow in a part of the internet and realise that the same kind of information integration is going on. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-12-19T17:32:59.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2512" PostTypeId="1" CreationDate="2016-12-20T04:41:33.823" Score="1" ViewCount="410" Body="&lt;p&gt;It is really all in the title.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those less familiar, the Fermi Paradox broadly speaking asks the question &quot;where is everybody&quot;. There's an equation with a lot of difficult to estimate parameters, which broadly speaking come down to this (simplification of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Drake_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drake equation&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Lots stars in the universe) * (non-zero probability of habitable planets around each star) * (lots of time spanned) = It seems there really should be somebody out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, of course, plenty of hypotheses as to why we haven't seen/observed/detected any sign of intelligent life so far, ranging from &quot;well we're unique deal with it&quot; to &quot;such life is so advanced and destroys everything it comes across, so it's a good thing it didn't happen&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technological singularity (also called ASI, Artificial Super Intelligence) is basically the point where an AI is able to self-improve. Some think that if such AI sees the light of day, it may self-improve and not be bound by biological constraints of the brain, therefore achieve a level of intelligence we cannot even grasp (let alone achieve ourselves).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I certainly have my thoughts on the matter, but interested to see if there is already an hypothesis revolving around the link between the 2 out there (I never came across but could be). Or perhaps an hypothesis as to why this cannot be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For references to those not familiar with the &lt;a href=&quot;http://waitbutwhy.com/2014/05/fermi-paradox.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fermi paradox&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4363" LastEditorUserId="169" LastEditDate="2016-12-22T01:08:44.660" LastActivityDate="2018-08-06T06:25:59.823" Title="Is the advent of a technological singularity a solution to the Fermi Paradox?" Tags="&lt;singularity&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2513" PostTypeId="2" ParentId="2512" CreationDate="2016-12-20T10:12:42.193" Score="2" Body="&lt;p&gt;If the technological singularity always leads to the extinction of all intelligent life, then yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the technological singularity always leads to intelligent life migrating into higher planes of existence that aren't accessible to us right now, then also yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise it is exactly the assumption of unbridled technological progress that makes the Fermi paradox perplexing. A post-singularity culture should have the ability to spread through the galaxy. If there are a lot of post-singularity cultures some of them should have spread through the galaxy. And if there are enough cultures that are spreading through the galaxy, we should notice them.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-12-20T10:12:42.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2514" PostTypeId="1" CreationDate="2016-12-20T15:44:15.303" Score="11" ViewCount="676" Body="&lt;p&gt;I understood that the concept of &lt;em&gt;search&lt;/em&gt; is important in AI. There's a &lt;a href=&quot;https://ai.stackexchange.com/questions/1877/why-is-searching-important-in-ais&quot;&gt;question&lt;/a&gt; on this website regarding this topic, but one could also intuitively understand why. I've had an introductory course on AI, which lasted half of a semester, so of course there wasn't time enough to cover all topics of AI, but I was expecting to learn some AI theory (I've heard about &quot;agents&quot;), but what I actually learned was basically a few search algorithms, like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BFS&lt;/li&gt;&#xA;&lt;li&gt;Uniform-cost search&lt;/li&gt;&#xA;&lt;li&gt;DFS&lt;/li&gt;&#xA;&lt;li&gt;Iterative-deepening search&lt;/li&gt;&#xA;&lt;li&gt;Bidirectional search&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;these search algorithms are usually categorized as &quot;blind&quot; (or &quot;uninformed&quot;), because they do not consider any information regarding the remaining path to the goal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or algorithms like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Heuristic search&lt;/li&gt;&#xA;&lt;li&gt;Best-first search&lt;/li&gt;&#xA;&lt;li&gt;A&lt;/li&gt;&#xA;&lt;li&gt;A*&lt;/li&gt;&#xA;&lt;li&gt;IDA*&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which usually fall under the category of &quot;informed&quot; search algorithms, because they use some information (i.e. &quot;heuristics&quot; or &quot;estimates&quot;) about the remaining path to the goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we also learned &quot;advanced&quot; search algorithms (specifically applied to the TSP problem). These algorithms are either constructive (e.g., nearest neighbor), local search (e.g., 2-opt) algorithms or meta-heuristic ones (e.g., ant colony system or simulated annealing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also studied briefly a min-max algorithm applied to games and an &quot;improved&quot; version of the min-max, i.e. the alpha-beta pruning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this course, I had the feeling that AI is just about searching, either &quot;stupidly&quot; or &quot;more intelligently&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Why would one professor only teach search algorithms in AI course? What are the advantages/disadvantages? The next question is very related to this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What's more than &quot;searching&quot; in AI that could be taught in an introductory course? This question may lead to subjective answers, but I'm actually asking in the context of a person trying to understand what AI really is and what topics does it really cover. Apparently and unfortunately, after reading around, it seems that this would still be subjective.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there AI theories that could be taught in this kind of course?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-25T16:30:50.177" LastActivityDate="2019-06-25T16:30:50.177" Title="Why teaching only search algorithms in a short introductory AI course?" Tags="&lt;philosophy&gt;&lt;search&gt;&lt;education&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2515" PostTypeId="1" CreationDate="2016-12-20T21:59:24.357" Score="3" ViewCount="89" Body="&lt;p&gt;Ethics is defined to be the set of moral principles that governs a person's or group's behavior. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Innately, shouldn't any system devise ethics for itself? Most of our articulations, in either direct or indirect manner, talk about intelligent systems in context of human presence. So, why, in first place, are we studying AGI ethics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given Roseau's reasoning for a society&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Human beings can improve only when they leave the state of nature and enter a civil society.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Doesn't that also automatically apply to any intelligent system? If it does, then can't we imply that any inorganic intelligent system should, perhaps, come as an offset of a network of intelligent computers (a society per se)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Who are we to constitute ethics and rules, morale for another society which, perhaps, could be much more intelligent than us?&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="2444" LastEditDate="2019-06-15T18:20:29.797" LastActivityDate="2019-06-15T18:20:29.797" Title="Are we supposed to &quot;decide&quot; ethics for intelligent systems?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;ethics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2516" PostTypeId="1" AcceptedAnswerId="2522" CreationDate="2016-12-21T02:35:03.877" Score="7" ViewCount="614" Body="&lt;p&gt;Currently, we use control flow statements (such as loops) to program the artificially intelligent systems. Could an AI be killed in an infinite loop (created by itself, for example, while manipulating its source code)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question isn't baseless, it really questions the kind of computational infrastructure a modern AGI would require, and would it be able to alter itself without any intervention.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="2444" LastEditDate="2019-11-22T23:40:56.367" LastActivityDate="2019-11-22T23:40:56.367" Title="Could an AI be killed in an infinite loop?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2517" PostTypeId="1" CreationDate="2016-12-21T02:50:57.240" Score="1" ViewCount="80" Body="&lt;p&gt;I am coding a tic-tac-toe program that demonstrates reinforcement learning. The program uses minimax trees to decide its moves. Whenever it wins, all the nodes on the tree that were involved in the game have their value increased. Whenever it loses, all the nodes on the tree that were involved in the game have their value decreased, etc. What is the name of the value that each node is decreased by?&lt;/p&gt;&#xA;" OwnerUserId="4378" LastActivityDate="2017-02-14T12:39:59.723" Title="Whats the name of the value that you add or subtract from a minimax tree node?" Tags="&lt;reinforcement-learning&gt;&lt;minimax&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2518" PostTypeId="1" CreationDate="2016-12-21T09:08:07.530" Score="1" ViewCount="2419" Body="&lt;p&gt;Neural Net can be feed-forward or recurrent. Perceptron is only feed forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what is Hopfield Network then?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastActivityDate="2016-12-23T17:19:28.933" Title="What is the difference between a Hopfield Network and a Neural Network or a Perceptron?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2519" PostTypeId="2" ParentId="2518" CreationDate="2016-12-21T09:31:00.617" Score="1" Body="&lt;p&gt;Hopfield Networks are recurrent. However, they are not as general as more modern Recurrent Neural Networks such as Long Short-Term Memory Networks as they cannot process sequential input. I've never worked with a Hopfield Network but I've been told that they are mostly of historical interest today due to their limitations.&lt;/p&gt;&#xA;" OwnerUserId="4361" LastEditorUserId="4361" LastEditDate="2016-12-21T09:34:13.537" LastActivityDate="2016-12-21T09:34:13.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2520" PostTypeId="1" CreationDate="2016-12-21T09:53:06.757" Score="6" ViewCount="320" Body="&lt;p&gt;Is a Levenberg–Marquardt algorithm a type of back-propagation algorithm or is it a different category of algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia says that it is a curve fitting algorithm. How is a curve fitting algorithm relevant to a neural net?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="3427" LastEditDate="2016-12-24T16:56:13.190" LastActivityDate="2016-12-24T16:56:13.190" Title="What kind of algorithm is the Levenberg–Marquardt algorithm?" Tags="&lt;algorithm&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2521" PostTypeId="2" ParentId="2514" CreationDate="2016-12-21T11:53:34.983" Score="6" Body="&lt;p&gt;What it comes down to is that most AI problems can be characterized as search problems. Let's just go through some examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Object recognition &amp;amp; scene building (e.g. the process of taking&#xA;audio-visual input of your surroundings and understanding it in a 3D&#xA;and contextual sense) can be treated as searching for known objects&#xA;in the input.&lt;/li&gt;&#xA;&lt;li&gt;Mathematical problem solving can be treated as searching for a solution.&lt;/li&gt;&#xA;&lt;li&gt;Playing a video game can be treated as searching for the correct response to a given gamestate.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Even rudimentary chatbots can be characterized as finding the 'correct' response to a given input phrase to emulate human language!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of this generalization of search, search algorithms were among some of the first algorithms considered 'AI', and often form the basis of many AI teaching courses. On top of this search algorithms are intuitive and non-mathematical, which makes the somewhat terrifying field of AI accessible. This might sound like hyperbole, but I guarantee that if your lecturer had opened with Manifold Learning Techniques half of your class would have bolted for the door by the time they mentioned 'eigenvalue of the covariance matrix'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now search algorithms aren't the only way to address these problems. I recommend every AI practitioner is familiar with the notion of Data Science and Machine Learning Algorithms. ML is often related to search algorithms but the techniques they use can vary heavily from iterative building of a classifier/regression (e.g. C4.5 builds a decision tree), meta-heuristics as you noted, and classifiers/regression that are statically generated from analysis of training data (e.g. Naive Bayesian is literally a classifier built on Bayesian analysis of the given data assuming that input fields are independent - this is the 'naivety' from which it gets its name). Often ML algorithms are developed in AI research groups and can sometimes be designed for specific problems instead of being general form algorithms. In contrast to the general field of AI, which is often centered on Intelligence problems and is therefore (in my view) vulnerable to too much blue sky thinking, ML is applied to all sorts of real life problems and is often very practical in its design and performance driven in its evaluation.&lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-12-21T11:53:34.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2522" PostTypeId="2" ParentId="2516" CreationDate="2016-12-21T12:04:32.430" Score="5" Body="&lt;p&gt;This is a hard to answer question because a truly correct answer would involve static analysis of a given Intelligence to determine whether it has the computational capability to generate a looping state (e.g. some state which reproduces itself in the next instance), and in fact whether these looping states can even exist in the given architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, assuming we can build AI, there is no reason it would be impossible build an AI that has the potential to get stuck in a computational loop and become non-functional. You can imagine a recurrent neural network where the continuous state inside the ANN dominates its behavior causing it to reproduce states with small variety (as input will always be relevant in a connected ANN) but are converged around a particular state space. Whether that is akin to death is a philosophical question for your own peace of mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we designed an AI agent that commonly did this, that would reflect poor design; that wouldn't necessarily devalue the contribution of the designed agent but it would bring the agent's other activities under scrutiny. If this is a bi-product of some simulated intelligence, is the process going on really that intelligent?&lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-12-21T12:04:32.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2523" PostTypeId="2" ParentId="2520" CreationDate="2016-12-21T12:15:05.257" Score="2" Body="&lt;p&gt;In the context of Neural Networks, Backpropagation (with Gradient Descent, to use its full name) and Levengerg Marquardt are both members of the broader family of gradient descent algorithms. Backpropagation itself is not gradient descent, but it does the gradient climbing portion of a broader gradient descent algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can imagine the function of a Neural Network as a function from its inputs to its outputs. If you were trying to solve, for example, a regression problem you can imagine this problem in multidimensional space with each training point corresponding to the coordinate provided by the values of its inputs and outputs (each of which represent a dimension). Then the entire training set you have for learning this regression problem become a set of points in this multidimensional space, and the function that your neural network performs is a curve in this multidimensional space. The closer this curve is to the points on the training set, the better it performs at the regression problem which essentially means we can generalize the task of training the neural network to a curve fitting problem.&lt;/p&gt;&#xA;" OwnerUserId="1467" LastActivityDate="2016-12-21T12:15:05.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2524" PostTypeId="1" CreationDate="2016-12-21T20:47:37.700" Score="13" ViewCount="1902" Body="&lt;p&gt;I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By taking the example of a small neural network (for example, one with 2 inputs, 2 nodes in a hidden layer and 2 nodes for the output), all you have is a complex function at the output which is mostly sigmoid over a linear combination of the sigmoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how does that make them good at prediction? Does the final function lead to some sort of curve fitting?&lt;/p&gt;&#xA;" OwnerUserId="4394" LastEditorUserId="2444" LastEditDate="2019-10-24T02:00:37.083" LastActivityDate="2019-10-24T02:00:37.083" Title="What makes neural networks so good at predictions?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;prediction&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2525" PostTypeId="2" ParentId="2498" CreationDate="2016-12-21T23:26:12.510" Score="3" Body="&lt;p&gt;Many large deployments of AI have carefully engineered solutions to problems (ie self driving cars). In these systems, it is important to have discussions about how these systems should react in morally ambiguous situations. Having an agent react &quot;appropriately&quot; sounds similar to the Turing test in that there is a &quot;pass/fail&quot; condition. This leads me to think that the &lt;strong&gt;current mindset of most AI researchers falls into &quot;Conservative anthropomorphism&quot;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is growing interest in &lt;a href=&quot;http://www.cs.utexas.edu/~ring/Ring-dissertation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continual Learning&lt;/a&gt;, where agents build up knowledge about their world from their experience. This idea is largely pushed by reinforcement learning researchers such as Richard Sutton and Mark Ring. Here, the AI agent has to build up knowledge about its world such as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When I rotate my motors forward for 3s, my front bump-sensor activates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I turned right 90 degrees and then rotated my motors forward for 3s, my front bump-sensor activates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From knowledge like this, an agent could eventually navigate a room without running into walls because it built up predictive knowledge from interaction with the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, lets ignore how the AI actually learns and only look at the environment AI agents &quot;grow up in&quot;. These agents will be fundamentally different from humans growing up in homes with families because they will not have the same learning experience as humans. This is much like the nature vs nurture argument. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans pass their morals and values on to children through lessons and conversation. As RL agents would lack much of this interaction (unless families adopted robot babies I guess), we would require different ways of judging their moral worth and thus &quot;Post-human fundamentalism&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sources:&#xA;5 years in the RL academia environment and conversations with Richard Sutton and Mark Ring.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2016-12-22T07:58:19.483" LastActivityDate="2016-12-22T07:58:19.483" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2526" PostTypeId="1" CreationDate="2016-12-21T23:26:55.103" Score="10" ViewCount="4758" Body="&lt;p&gt;Why should an activation function of a neural network be differentiable? Is it strictly necessary or is it just advantageous?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="2444" LastEditDate="2019-02-14T03:02:51.763" LastActivityDate="2019-02-14T03:02:51.763" Title="Why do activation functions need to be differentiable in the context of neural networks?" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;activation-function&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2527" PostTypeId="2" ParentId="2526" CreationDate="2016-12-22T00:36:51.500" Score="3" Body="&lt;h2&gt;Training&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;While &quot;running&quot; a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is commonly done by &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;backpropagation&lt;/a&gt; and variations of &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent&lt;/a&gt;, which requires the existence of a gradient - i.e., requires activation function to be differentiable. The adjustment of each parameter is calculated from the gradient of the activation function(s) that this parameter affects, so if you cannot get a gradient, then this approach can't be used.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastEditorUserId="1675" LastEditDate="2016-12-22T00:45:07.347" LastActivityDate="2016-12-22T00:45:07.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2528" PostTypeId="1" CreationDate="2016-12-22T00:45:53.537" Score="1" ViewCount="119" Body="&lt;p&gt;I understand an MDP (Markov Decision Process) model is a tuple of &lt;span class=&quot;math-container&quot;&gt;$\{S, A, P, R \}$&lt;/span&gt; where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a discrete set of states&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a discrete set of actions&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is the transition matrix ie. &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s, a) \rightarrow [0,1]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the reward function id. &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s') \rightarrow \mathbb{R}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For a non-trivial MDP, say &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; states and &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; actions, the transition matrix has theoretically &lt;span class=&quot;math-container&quot;&gt;$S \times A \times S = 10,000,000$&lt;/span&gt; entries (though many entries will be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that one way of generating the &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; matrix is to estimate it via Monte Carlo sampling, by simulating the environment. However, with non-trivial state space and simulation costs, this could be prohibitively expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, when a non-trivial MDP is being formulated, what are the different ways an accurate &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; matrix can be produced?&lt;/p&gt;&#xA;" OwnerUserId="4402" LastEditorUserId="2444" LastEditDate="2018-11-24T04:32:19.103" LastActivityDate="2018-11-24T04:32:19.103" Title="How do you generate the transition probabilities of a non-trivial MDP?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2529" PostTypeId="2" ParentId="2515" CreationDate="2016-12-22T01:18:30.860" Score="2" Body="&lt;h2&gt;Nonhuman ethics can be totally arbitrary&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Regarding your citation, perhaps it's useful to consider why should anything that philosophers have said about human ethics apply for arbitrary nonhuman systems? We all share a common biological background, our behavior, emotions and especially positive/negative feelings arise from the particular way how our brains are built. Man is by nature a social animal, and much of our behavior and feelings of empathy, compassion, fairness, greed, social status, etc are formed and expressed in society. Our minds are adjusted for this, as are the minds of other mammals. A human that forms an ethical system when joining a civil society does so in the basis of all this shared context, and within the limitations of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, all this is orthogonal to intelligence. All the variety of mankind encompasses a tiny island in the whole space of possible intelligences and their goals. Any system of ethics is feasible, a vast majority of them having no relationship whatsoever to what we could consider ethical. A human can be expected to join a society and learn ethics because our brain is hardwired to respond to social stimuli - an AI would learn ethics from a society only if its previous ethics already happen to value that highly, and that isn't likely to happen without careful design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a thought experiment that is commonly used as an example - the &quot;paperclip maximizer&quot;, an artificial agent with an ethical system that defines 'good' proportionally to the number of paperclips that exist in the universe and only that - and that is a &lt;em&gt;perfectly reasonable&lt;/em&gt; example, because a completely random ethical system is overwhelmingly likely to reduce to something stupid like that. The space of &quot;reasonable&quot; (to us) ethical systems is very tiny compared to all ethical systems possible, and expecting that a randomly selected ethical system will just happen to land there is like relying on winning a high-stakes lottery.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;We want their ethics to contain certain things&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So the main reason why we are studying AGI ethics is because we (or some of us) really, really, want the AGI ethical system to include certain things. For example, I'd like that ethical system to include my survival. I'd prefer that ethical system to be consistent with my happiness and wellbeing - and the very existence of a very powerful entity that simply happens to not care about me at all is not compatible with my survival and wellbeing, so I'd &lt;em&gt;really&lt;/em&gt; prefer for it to care about me, or alternatively never become very powerful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could certainly argue that we have no right to constitute ethics, rules and morale for another society which could be much more intelligent and better than us. That another society might be &quot;better&quot; in some manner - but better &lt;em&gt;for whom&lt;/em&gt; ? If it's better for &quot;them&quot; but worse for me (and humanity), then I have strong motivation to simply &lt;em&gt;claim&lt;/em&gt; that right and try my hardest to ensure that this hypothetical future society includes me, my desires and my ethics. &lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2016-12-22T01:18:30.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2530" PostTypeId="1" AcceptedAnswerId="2537" CreationDate="2016-12-22T01:35:50.973" Score="6" ViewCount="307" Body="&lt;p&gt;Generating a discretized state space for an MDP (Markov Decision Process) model seems to suffer from the curse of dimensionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposed my state has a few simple features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Feeling: Happy/Neutral/Sad&lt;/li&gt;&#xA;&lt;li&gt;Feeling: Hungry/Neither/Full&lt;/li&gt;&#xA;&lt;li&gt;Food left: Lots/Some/Low/Empty&lt;/li&gt;&#xA;&lt;li&gt;Time of day: Morning/Afternoon/Evening/Night&lt;/li&gt;&#xA;&lt;li&gt;Located: Home/Work/Library/Shops&lt;/li&gt;&#xA;&lt;li&gt;Money in wallet: \$0, \$0-\$50, \$51-\$200, \$200-\$1000, \$1000+&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is a relatively compact set of information however since the number of states multiplies out, the corresponding MDP state space is 3 x 3 x 4 x 4 x 4 x 5 = 2880.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a non-trivial problem with a greater number of choices per factor, the state space quickly becomes unmanageably large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that the usefulness of MDPs with large complex problems would be very limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that the case, or are there ways of keeping the state space manageable for more complex problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, what is a manageable number of states for an MDP to have, considering the need to generate the transition matrix and reward matrix?&lt;/p&gt;&#xA;" OwnerUserId="4402" LastEditorUserId="2444" LastEditDate="2020-11-30T01:52:39.617" LastActivityDate="2020-11-30T01:52:39.617" Title="What techniques are used to make MDP discrete state space manageable?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;state-spaces&gt;&lt;continuous-action-spaces&gt;&lt;discrete-action-spaces&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2531" PostTypeId="1" AcceptedAnswerId="2534" CreationDate="2016-12-22T04:46:36.093" Score="1" ViewCount="1416" Body="&lt;p&gt;&lt;a href=&quot;http://www.cs.bham.ac.uk/~jxb/INC/l5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.bham.ac.uk/~jxb/INC/l5.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The neuropsychologist Donald Hebb postulated in 1949 how biological&#xA;  neurons learn:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;“When an axon of cell A is near enough to excite a cell B and&#xA;  repeatedly or persistently takes part in firing it, some growth&#xA;  process or metabolic change takes place on one or both cells such that&#xA;  A’s efficiency as one of the cells firing B, is increased.”&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In more familiar terminology, that can be stated as the Hebbian&#xA;  Learning rule:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If two neurons on either side of a synapse (connection) are activated&#xA;  simultaneously (i.e. synchronously), then the strength of that synapse&#xA;  is selectively increased.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Mathematically, we can describe Hebbian learning as:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BLTdE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BLTdE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Here, η is a learning rate coefficient, and x are the outputs of the&#xA;  ith and jth elements.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, my question is, what do all these descriptions mean?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is Hebbian Learning applicable for single-neuron networks?&lt;/li&gt;&#xA;&lt;li&gt;What does it mean by &quot;two neurons on either side of a synapse&quot;?&lt;/li&gt;&#xA;&lt;li&gt;Why/when would two neurons activate simultaneously?&lt;/li&gt;&#xA;&lt;li&gt;What does they mean by &lt;code&gt;elements&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user3642" LastActivityDate="2016-12-22T07:09:33.973" Title="How do you explain Hebbian Learning in an intuitive way?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2532" PostTypeId="5" CreationDate="2016-12-22T06:02:14.000" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-12-22T06:02:14.000" LastActivityDate="2016-12-22T06:02:14.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2533" PostTypeId="4" CreationDate="2016-12-22T06:02:14.000" Score="0" Body="For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper &quot;Learning while searching in constraint-satisfaction-problems&quot;." OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2019-07-07T01:25:54.983" LastActivityDate="2019-07-07T01:25:54.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2534" PostTypeId="2" ParentId="2531" CreationDate="2016-12-22T07:09:33.973" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Hebbian learning is trying to answer &quot;How the strength of the synapse between 2 neurons evolve over period of time based on the activity of the 2 neurons involved.&quot;. You can call it learning if you think learning is just strengthening of synapses. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The connection between 2 neurons are called synapse. A synapse is the point where the axons of a neuron meets with the dendrites of another neuron.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The 2 neurons are connected to each other but they are also connected to other neurons as well. So it may happen that both the neurons gets activated by their other connected neurons at the same time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;code&gt;elements&lt;/code&gt; refer the 2 neurons. The equation is basically saying that the synapse strength between i and j neuron at time (n+1) depends on its strength at time n plus activations of the i and j neurons at time n. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-12-22T07:09:33.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2535" PostTypeId="1" AcceptedAnswerId="2543" CreationDate="2016-12-23T16:20:30.830" Score="7" ViewCount="102" Body="&lt;p&gt;Just started reading a book about AI. There is a very basic exercise but I can't figure it out, so here we go. The book is  &lt;a href=&quot;https://www.cs.bris.ac.uk/~flach/SL/SL.pdf&quot; rel=&quot;noreferrer&quot;&gt;Simply Logical: Intelligent Reasoning by Example&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exercise is in the page 19. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Two stations are ‘not too far’ if they are on the same or a different&#xA;  line, with at most one station in between. Define rules for the&#xA;  predicate not_too_far.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The only rules I've seen are &lt;strong&gt;nearby&lt;/strong&gt; and &lt;strong&gt;connected&lt;/strong&gt; and don't know how to use this. What I've done so far is this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;not_too_far(X,Y) :- nearby(X,Y)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4421" LastActivityDate="2016-12-25T15:05:09.360" Title="Define rules for the predicate" Tags="&lt;prolog&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2536" PostTypeId="2" ParentId="2518" CreationDate="2016-12-23T17:19:28.933" Score="3" Body="&lt;p&gt;Hopfield stores some predefined patterns (lower energy states) and when an non seen pattern is fed to the Hopfield net, it tries to find the closest match among the stored patterns. Hence the output of a Hopfield is always one of the predefined patterns which matches closely to the unseen input pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the case with Feed Forward Neural Nets (where no such predefined patterns are stored) and every input generates a corresponding output. Here (in Feedforward) the output is generated by a predefined function (which is self-adjusted during training session) where as in Hopfield predefined patterns are stored and outputted (no such functions exist).&lt;/p&gt;&#xA;" OwnerUserId="4424" LastActivityDate="2016-12-23T17:19:28.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2537" PostTypeId="2" ParentId="2530" CreationDate="2016-12-23T19:13:15.560" Score="6" Body="&lt;p&gt;&lt;strong&gt;tl:dr&lt;/strong&gt;&#xA;Read chapter 9 of an &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction of Reinforcement Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is definitely a problem (a curse if you will) when the dimensionality of a task (MDP) grows. For fun, lets extend your problem to a much harder case, continuous variables, and see how we deal with it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mood: range [-1, 1]   // 1 is Happy, 0 is Neutral, -1 is Sad&#xA;&#xA;Hunger: range [0, 1]  // 0 is Very Hungry, 1 is Full&#xA;&#xA;Food: range [0, 100] // amount of food, capping at 100 for numbers sake&#xA;&#xA;Time of day: [0, 23] // Hours, 9.5 would be 9:30 am&#xA;&#xA;Position x: range [-10, 10] // assuming the area the agent stays in is 10x10 km&#xA;&#xA;Position y: range [-10, 10]&#xA;&#xA;Money: range [0, 2000]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now its impossible to even count the number of states the agent could be in. An example of one state would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mood, Hunger, Food, Time, PosX, PosY, Money&#xA;.5, .3, 67.4, 4.5, 0, 0, 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This would mean that our agent has a neutral mood, is kind of hungry, has 67.4% food, the time is 4:30 am, they are in the center of the city (0,0), and have 5 dollars. Which has happened to me once or twice before too so this is reasonable. &lt;strong&gt;If we we still wanted to generate a transition matrix and a reward matrix we would have to represent this state uniquely from other possibly similar states such as if the time was 5 pm with all other things equal.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now how would we approach this?&#xA;First lets discretize or breakup each range into little chunks. Then we could have an array of 0s representing our chunks and assign a 1 to the index of the chunk we are in. Lets call this array our feature array. &#xA;If we did this for each dimension of our state information, we would end up with some state similar to what you originally proposed. I've chosen some arbitrary numbers to break up the state space and lets see what it does to the example state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we broke the Mood range up into 8 chunks of .25 we would have these ranges:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;m = Mood&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;-1.0 ≤ m &amp;lt; -.75&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;-.75 ≤ m &amp;lt; -.50&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;-.50 ≤ m &amp;lt; -.25&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;-.25 ≤ m &amp;lt; 0.0&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;0.0 ≤ m &amp;lt; .25&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;.25 ≤ m &amp;lt; .50&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;.50 ≤ m &amp;lt; .75&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;.75 ≤ m &amp;lt; 1.0&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If we took the mood from the example state (.5), it would land in the 7th range, so then our feature vector for mood would look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0,0,0,0,0,0,1,0] &amp;lt;- represents a mood of .5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Lets do the same for hunger by splitting it into 8 chunks of .125:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0,0,1,0,0,0,0,0] &amp;lt;- represents a hunger of .3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We could then do something similar for each other state variable and break each state variable's range up into 8 chunks of equal sizes as well. Then after calculating their feature vectors, we would concatenate them all together and have a feature vector that is 56 elements long (7 state variables * 8 chunks each).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this would give us 2^56 different combinations of feature vectors, it actually doesn't represent what we wanted to represent. We need features that activate (are 1) when events happen together. We need a feature that is 1 when we are hungry &lt;strong&gt;AND&lt;/strong&gt; when we have food. We could do this by doing a cross product of our 56 element feature vector with itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have a 3136 element feature vector that has features like:&#xA;  1 if its 3pm and am happy&#xA;  1 if am full and out of food&#xA;  1 if at coordinate -3, 4 (Position x, Position y)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a step in the right direction but still isn't enough to represent or original example state. So lets keep going with the cross products!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we do 6 cross products we would have 30,840,979,456 features and only 1 of them would be on when our agent has a neutral mood, is kind of hungry, has 67.4% food, the time is 4:30 am, they are in the center of the city (0,0), and have 5 dollars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point you are probably like &quot;Well.. thats a bit much&quot;, to which I would agree. This would be the curse of dimensionality, and is the reason transition tables stop being fun (if they ever were).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Instead lets take a different approach&lt;/strong&gt;, rather than trying to represent an individual state with an individual feature and saying whether that feature is good, lets go back to our nice 56 element feature vector. From here lets instead give a weight (w_i) to each of our 56 features. We have now entered the world of linear function approximation. Although I could explain it here, I think its better explained in Chapter 9 of the &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction of Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="1847" LastEditDate="2018-08-10T17:38:51.280" LastActivityDate="2018-08-10T17:38:51.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2538" PostTypeId="2" ParentId="1461" CreationDate="2016-12-23T22:08:36.177" Score="2" Body="&lt;p&gt;Short Answer, &lt;strong&gt;No&lt;/strong&gt;.&lt;Br&gt;&lt;br&gt;&#xA;Explained, Siri and Cortana are just &lt;strong&gt;inference engines&lt;/strong&gt;. Though how applaudable their ability to synthesize text from speech and parse lexical maps from the text using Machine Learning Techniques is, the artifact is still just a program, trained with substantial myriad of Q/A tuples, that generates an output given an input. Statistically mapping the probability distribution of words of a question and accordingly generating an answer is just &lt;strong&gt;NOT&lt;/strong&gt; an AI, however, to the most, it can be classified as &lt;strong&gt;&lt;em&gt;Narrow Intelligence&lt;/em&gt;&lt;/strong&gt;. &lt;Br&gt;&lt;Br&gt;&#xA;&lt;a href=&quot;http://www.cleverbot.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.cleverbot.com&lt;/a&gt; is an example of such a system. The ones you mentioned are just more sophisticated and highly architectured versions of this kind; for example, connection to WolframAlpha (Siri), Computer Kernel (Cortana). In fact, there is a lot of innate use of control statements (If-Else).&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-12-23T22:08:36.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2539" PostTypeId="2" ParentId="2526" CreationDate="2016-12-23T22:25:31.763" Score="4" Body="&lt;p&gt;It is almost mandatory to have a differentiable activation function unless, of course, you have an alternative to &lt;em&gt;training the network by back-propagating the error&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-12-23T22:25:31.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2540" PostTypeId="2" ParentId="2526" CreationDate="2016-12-24T11:31:52.260" Score="1" Body="&lt;p&gt;As already said , Activation function is almost differentiable in every neural net to facillitate &lt;strong&gt;Training&lt;/strong&gt; as well as to calculate tendency towards a certain result when some parameter is changed. But I just wanted to point out that &lt;strong&gt;The Output function need not be differentiable in all cases.&lt;/strong&gt; We can have non-sigmoid (hard-limiter threshold)output nodes but still train them with backpropagation and gradient descent.&lt;/p&gt;&#xA;" OwnerUserId="4424" LastActivityDate="2016-12-24T11:31:52.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2541" PostTypeId="2" ParentId="2404" CreationDate="2016-12-24T19:17:32.087" Score="3" Body="&lt;p&gt;It depends on the accuracy you want. If you had 1 neuron, it could discern things across a line, if you have 2, you could solve things across 2 lines, etc. As you increase the number of neurons, you are increasing the number of discernible areas. As you increase the number of lines you can use to break up the input space, the lines can be placed to approximate any curve (sinusoidal) As the number of neurons approaches infinity, the accuracy of categorizing different inputs across this curve increases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly enough, if one graphed &quot;Number of Neurons (x) vs Accuracy (y)&quot;, it would look sinusoidal. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2017-01-28T04:43:31.133" LastActivityDate="2017-01-28T04:43:31.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2542" PostTypeId="1" AcceptedAnswerId="2565" CreationDate="2016-12-25T08:08:08.443" Score="-1" ViewCount="232" Body="&lt;p&gt;What are the basic layers on an Artificially Intelligent program and what skills and concept are required to work on this field. Total newbie interested in AI.&lt;/p&gt;&#xA;" OwnerUserId="4435" LastEditorUserId="8" LastEditDate="2016-12-28T17:15:50.517" LastActivityDate="2016-12-30T20:39:02.107" Title="Concept of AI program" Tags="&lt;ai-design&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-24T17:43:23.340" ContentLicense="CC BY-SA 3.0" />
  <row Id="2543" PostTypeId="2" ParentId="2535" CreationDate="2016-12-25T15:05:09.360" Score="3" Body="&lt;p&gt;Your intuition is good. Because &quot;nearby&quot; is only defined with &quot;connected&quot;, there could only be 1 station between them. However, it says that the stations are &quot;not_too_far&quot; if at most one station is between them. What about if no stations are between them? If 2 stations are &quot;connected&quot; they should be &quot;not_too_far&quot; as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;not_too_far(X,Y) :- connected(X,Y) ; nearby(X,Y).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where ; denotes OR. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2016-12-25T15:05:09.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2544" PostTypeId="2" ParentId="2542" CreationDate="2016-12-25T15:48:35.640" Score="0" Body="&lt;p&gt;The basic layers of AI aren't that interesting at the highest level. In the most abstract sense, AI is simply a function f(x) which is given input x and returns an output. This isn't that exciting, so let's break it down a bit more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI can be broken by 2 different aspects. &#xA;AI can be Online or Offline. &#xA;AI can be Supervised or Unsupervised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is Offline if it learns the function f before being released into the real world at which point it doesn't update f. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is Online if it is put in the real world immediately and must learn f on-the-fly during operation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is Supervised if it is given the correct answer after guessing. This is used to update f.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is Unsupervised if the correct answer is not given. This could mean that some indication of how well the AI did is returned (such as in reinforcement learning) or nothing at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choosing 1 of each aspect gives 1 of 4 categories, such as Unsupervised Online learning. AlphaGo which beat on if the worlds best Go players is an example of an Unsupervised Online learning system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these types of AI require representing the input x and choosing an output. Both of which involve statistics, math, and some programming experience. Anyone wanting to work in AI should have these skills and be able to think critically about them (mostly to diagnose the bugs that are bound to happen).&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2016-12-25T15:48:35.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2545" PostTypeId="1" AcceptedAnswerId="2546" CreationDate="2016-12-26T04:24:08.937" Score="3" ViewCount="277" Body="&lt;p&gt;The following text is from Hal Daumé III's &lt;a href=&quot;http://ciml.info/dl/v0_9/ciml-v0_9-ch03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;&lt;em&gt;A Course in Machine Learning&lt;/em&gt;&quot;&lt;/a&gt; online text book (Page-41).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SCfew.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SCfew.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is the size of the input vector &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;? Why is it introduced in the algorithm? How/where/when is the initial value of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; given? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the rationale of testing &lt;span class=&quot;math-container&quot;&gt;$ya \leq 0$&lt;/span&gt; for updating weights?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="2444" LastEditDate="2019-04-19T14:13:42.580" LastActivityDate="2019-04-19T14:13:42.580" Title="Understanding the perceptron algorithm in the book &quot;A Course in Machine Learning&quot;" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2546" PostTypeId="2" ParentId="2545" CreationDate="2016-12-26T06:32:39.887" Score="2" Body="&lt;p&gt;Y is the desired output of the perceptron (often referred to as target) , for the given set of input vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rationale behind Y.a&amp;lt;=0&lt;/strong&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prerequisite knowledge :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;A&lt;/strong&gt;=&lt;strong&gt;A&lt;/strong&gt;-&lt;strong&gt;B&lt;/strong&gt; : Moves vector &lt;strong&gt;A&lt;/strong&gt; away from direction of vector &lt;strong&gt;B&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;A&lt;/strong&gt;=&lt;strong&gt;A&lt;/strong&gt;+&lt;strong&gt;B&lt;/strong&gt; : Moves &lt;strong&gt;A&lt;/strong&gt; in the direction of &lt;strong&gt;B&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;A&lt;/strong&gt; (.) &lt;strong&gt;B&lt;/strong&gt; &gt;0 ; &lt;strong&gt;A&lt;/strong&gt; vector is directed acutely (&amp;lt;90 deg.) towards &lt;strong&gt;B&lt;/strong&gt; vector&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;A&lt;/strong&gt; (.) &lt;strong&gt;B&lt;/strong&gt; &amp;lt;0 ; &lt;strong&gt;A&lt;/strong&gt; vector is directed away from (&gt;90 deg.) from &lt;strong&gt;B&lt;/strong&gt; vector [(.) denotes dot (scalar) product and &lt;strong&gt;bold&lt;/strong&gt; letters indicate vectors]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;W&lt;/strong&gt; is augmented vector (includes threshold as another weight along with normal input weights)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;X&lt;/strong&gt; is augmented input (includes -1 as extra input (corresponding to threshold) along with other normal inputs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a(activation) = &lt;strong&gt;W&lt;/strong&gt; (.) &lt;strong&gt;X&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a &gt;=0 ; Perceptron output 1&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a&amp;lt;0 ; Perceptron output -1 (&lt;strong&gt;Not zero&lt;/strong&gt; as implicit in the given algorithm I think)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now Rationale :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Y.a&amp;lt;0) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means &#xA;Either of the following :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Y=-1 and a&gt;0 ; in this case the target output is -1 but as a&gt;0 so the Perceptron outputs 1. So we must move the weight vector away from this set of input vector, so that angle between them increases and the dot product (a) becomes &amp;lt; 0 so that we can get the target output.&#xA;Hence : &lt;strong&gt;W&lt;/strong&gt;=&lt;strong&gt;W&lt;/strong&gt;-&lt;strong&gt;X&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or ,&lt;strong&gt;W&lt;/strong&gt;=&lt;strong&gt;W&lt;/strong&gt; + (-1)&lt;strong&gt;*X&lt;/strong&gt;&#xA;Or, &lt;strong&gt;W&lt;/strong&gt;=&lt;strong&gt;W&lt;/strong&gt;+Y&lt;strong&gt;X&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Y=1 and a&amp;lt;0&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This means the target output is 1 but as the activation is &amp;lt;0 so Perceptron is outputting -1. So we must move the weight vector close to this set of input vector so that the activation can become &gt;0 (angle decreases) and the Perceptron can output the desired output.&#xA;So :&#xA;&lt;strong&gt;W&lt;/strong&gt; = &lt;strong&gt;W&lt;/strong&gt;+&lt;strong&gt;X&lt;/strong&gt;&#xA;Or &lt;strong&gt;W&lt;/strong&gt;=&lt;strong&gt;W&lt;/strong&gt; + Y&lt;strong&gt;X&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again , Y.a=0 is a boundary case.&#xA;By now I think you can understand the rationale behind Y.a=0. If any doubt , comment to this answer , I will explain it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for so much long answer though. :) :)&lt;/p&gt;&#xA;" OwnerUserId="4424" LastActivityDate="2016-12-26T06:32:39.887" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2547" PostTypeId="1" CreationDate="2016-12-26T10:59:27.333" Score="4" ViewCount="245" Body="&lt;p&gt;Recently Mark got some attention from the media by stating that he had created Jarvis. Not that I'm against him or anything, but this Jarvis seems to have been done a hundred times before. He's done something which most developers would classify as a home automation system. To me it's more like he did it for the attention. I was kind of taken back by the amount of media attention he got. If you've heard of Jeremy Blum, maybe you may understand what I'm trying to imply here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm just curious as to why he got so much attention. Is there anything technically novel about his system that sets it so much apart from previous ones?&lt;/p&gt;&#xA;" OwnerUserId="4444" LastEditorUserId="75" LastEditDate="2016-12-26T20:14:54.763" LastActivityDate="2016-12-27T08:47:12.330" Title="Is there anything novel about Zuckerberg's Jarvis?" Tags="&lt;ai-design&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2548" PostTypeId="1" CreationDate="2016-12-26T11:24:09.163" Score="9" ViewCount="2630" Body="&lt;p&gt;What are the main differences between a perceptron and a naive Bayes classifier?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="2444" LastEditorDisplayName="user3642" LastEditDate="2021-02-14T19:47:29.683" LastActivityDate="2021-02-17T07:19:58.507" Title="What are the main differences between a perceptron and a naive Bayes classifier?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;perceptron&gt;&lt;naive-bayes&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2550" PostTypeId="5" CreationDate="2016-12-26T14:24:02.320" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-12-26T14:24:02.320" LastActivityDate="2016-12-26T14:24:02.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2551" PostTypeId="4" CreationDate="2016-12-26T14:24:02.320" Score="0" Body="For questions related to the philosophical aspects of artificial intelligence. Topics such as human/AI value alignment, artificial consciousness, the feasibility of AGI, the ethics of AI, Neo-Luddism, etc." OwnerUserId="10843" LastEditorUserId="1671" LastEditDate="2018-04-06T18:08:07.040" LastActivityDate="2018-04-06T18:08:07.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2554" PostTypeId="2" ParentId="2547" CreationDate="2016-12-27T08:47:12.330" Score="3" Body="&lt;p&gt;No, there is nothing novel about this system. The main hurdle he had to pass was problems that you will face when your system has a lot of integration points across various APIs provided by different vendors with messy and often outdated documentation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as attention is concerned we live in a world where so called celebrities get attention for anything that they do. Remember that media have only one goal - More and more eyeballs aka more money and nothing else.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-12-27T08:47:12.330" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2555" PostTypeId="1" AcceptedAnswerId="2556" CreationDate="2016-12-27T14:47:59.430" Score="3" ViewCount="259" Body="&lt;p&gt;With tools like open AI will we be able to teach an AI to build its own decks? build a deck from a limited pool? or draft? evaluate the power level of a card? &lt;/p&gt;&#xA;" OwnerUserId="4466" LastActivityDate="2016-12-28T00:21:53.347" Title="How close are we to having an AI that can play Magic: The Gathering objectively well?" Tags="&lt;gaming&gt;&lt;training&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2556" PostTypeId="2" ParentId="2555" CreationDate="2016-12-28T00:21:53.347" Score="2" Body="&lt;p&gt;This is a very specific task, with clearly defined parameters, so it would already theoretically be within the scope of current AI technology to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI would need to learn how to make decisions, and the best way to do this is the approach taken to &lt;a href=&quot;https://www.scientificamerican.com/article/how-the-computer-beat-the-go-master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;teaching an AI to play Go&lt;/a&gt; - seeing thousands of example games by experts, and playing itself thousands of times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI won't necessarily &quot;understand&quot; what a card represents, the way a human would, but it can learn to make the same kinds of decisions as the human would.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difficulties would be purely practical - Go is very easy to represent digitally, because the options for action are limited to placing a stone on one of the intersections, and there is only one opponent. Magic is more complex, so the developers would need to spend sufficient resources to be quite sure they had captured all the relevant variables in the digital representation. Then, of course, they would need to encode thousands of games of Magic (or deck-building processes) into that digital format, so the AI could study them.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2016-12-28T00:21:53.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2558" PostTypeId="1" CreationDate="2016-12-28T14:07:12.900" Score="0" ViewCount="79" Body="&lt;p&gt;Given that one website has a particular style and another has another style, could a style transfer be done such that the style of one website was transferred to the other website?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or in a more simple case, consider just part of a website, a box.&lt;/p&gt;&#xA;" OwnerUserId="4488" LastActivityDate="2016-12-28T20:31:14.630" Title="Could style transfer be used to transfer the style of a website from one to another?" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2559" PostTypeId="2" ParentId="2558" CreationDate="2016-12-28T20:31:14.630" Score="1" Body="&lt;p&gt;The correct way for website styling to be encapsulated and centralized is through the use of one or more CSS style sheets.  For instance, the old  tag is frowned upon and using a text-align:center directive a proper class or ID based CSS selector is considered the proper way.  In such a case, the copying of a style is simply done by the copying of the appropriate style sheet file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The place where AI and/or machine learning tools may be best used is in the conversion of non-CSS styling to CSS styling.  Once they are in that form, the copying and layering of styling becomes trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To convert a legacy web site to proper CSS style sheet use, the input would be a set of all documents likely to be involved in the styling of a specific page (i.e. its style dependencies).  The output would be a set of corresponding documents with the styling moved into classes and IDs that remove styling redundancy (simplify and centralize the style).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transformation of the document set could be best accomplished with current technique through transformations, much like code refactoring in typical IDE refactoring algorithms in that the output document set must be functionally the same as the input document set.  Furthermore, the appearance and responsive (size sensitive) aspects must be preserved in addition to functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The greatest usability could be obtained by adding some automation to these refactoring transformations, as in Maxima simplification functions like trigsimp.  This is a typical search of options, perhaps with predictive branching, as in compiler optimizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transformations, to guarantee preservation of function and appearance must essentially be elementary equivalent transformations, but the selection of transformation would probably need to be more advanced than basic antecedent and consequence productions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heuristically meta rules, fuzzy logic, neural nets, Bayesian inference, or other tools could be compared in terms of effectiveness in the realm of prioritizing transformations and in terms of scoring results in the search leaves.  Here's where machine learning would probably be necessary to have a truly useful tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training could be accomplished through use of an array of input document sets and the careful scoring of transformations and/or outputs of trials could be performed with a software engineer with appropriate expertise in AI and/or machine learning, in conjunction with a CSS expert or a focus group containing two or more of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once this process and the resulting rule set works, it would then be possible to simplify the CSS further by finding styling redundancy across pages and doing transformations that reduce an entire site to a minimal set of style sheets and style specifications within them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combining such tools with a simple crawler like wget could permit the copying of a desirable web site style as a starting point for another web site's styling, drastically cutting down the time to create a properly CSS styled result.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2016-12-28T20:31:14.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2560" PostTypeId="2" ParentId="2366" CreationDate="2016-12-28T22:01:30.200" Score="3" Body="&lt;p&gt;&lt;strong&gt;Semantics Matters&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer depends on the definition intelligence being used.  If you define intelligence as the ability to adapt, a number of things could be considered intelligent that don't normally fit under the classic AI umbrella.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Nonlinear least-squares Marquardt-Levenberg curve fitting algorithm with a substantial but finite set of models, parallel trials of multiple models, outcomes analysis, and smart decisioning&lt;/li&gt;&#xA;&lt;li&gt;Check reader software recently deployed in bank branches and offices&lt;/li&gt;&#xA;&lt;li&gt;The combination of medical providers, patients, and carriers and the modification of treatment through financial instrumentation to improve outcomes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you define intelligence as mimicking the full array of human mental abilities, no such system is yet available to the public &amp;mdash; nothing even close. Creative adaptation across a wide array of arbitrary domains, few of which have been previously experienced in great detail or studied in depth has been an eventual objective of AI research. Approaches have been offered, but none have yet born public fruit. If such a system exists in secrecy, someone would have to violate their nondisclosure agreement or security clearance to tell us about it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of intelligence is central to answering.  For instance, some reasonable definitions of intelligence might lead an unbiased judge to rate ants above humans.  Ants had been building in hexagons for millennia before humans blundered into the habit of building in rectangles.  Rectangles require over 70% more building material to build vertical structure per square foot of floor space than a packed hexagonal structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Basic Artificial Intelligence System Requirements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Guessing that by, &quot;Basic AI,&quot; you mean some naive machine learning, there are a few basic components.  (The term Naive in this context means that the AI does not understand the domain or the meanings of symbols or signals it is processing in the way a human who had studied or worked within the field might understand them.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;SENSING &amp;mdash; The machine (computer) must receive information, generally as a time series.  In human beings, these are the senses.  In a mail sorter, it may be a camera.  This is beyond just the concept of input in information technology.  It is more analogous to an input signal in a PID controller.  In an automated high speed trading machine, this would be the high speed version of a ticker tape.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CONTROL &amp;mdash; The machine must manipulate externals in a way that exploits the received information.  In a basketball player, this is the motor coordination, facial affect, verbal signals to teammates, and perhaps some verbal bait for opposing players.  In a mail sorter, this would be the motor control of mail direction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MEMORY &amp;mdash; The machine must have storage to audit input time series (perhaps from some transducer in the real world or some data store upon which some intelligent analysis or transformation must be done).  In more advanced systems, the machine may wish to analyze its own performance and make adjustments that converge on some optimal metric value (perhaps a historical maxima or minima) or some range of acceptability.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;FEEDBACK &amp;mdash; The machine must interpret some feedback signalling or use a predetermined scoring mechanism.  Learning cannot occur in an information vacuum, so some definition of better or worse must be established.  The feedback may be entangled within the SENSING channels or may arrive through a completely separate channel. In biological systems, these are often pre-wired as threat detection, pain, and pleasure.  The cerebral cortex uses concepts of goals and progress.  In some ways, child rearing and social strata exists to teach the boundaries of what constitutes an acceptable goal and acceptable methods for making progress toward it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MODELLING &amp;mdash; Whether implicitly or explicitly, some model must be developed and exploited.  Some would say that the existence of a model upon which the predictive capabilities of it can be applied to decision making to achieve some goal or weighted collection of goals is intelligence.  Others would say that the development of the model is intelligence and the use of it is merely control mechanics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approaches to Simulating Human Thinking&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cognition is not the only form of model making, but the creation of cognitions and their application to decision making.  The concept of intelligence may have been furthered along a realistic path by Roger Schank, who proposed that the storage and indexing of stories was a primary characteristics of what humans recognize as intelligent conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minsky and others took a direction that was more connected with logical inference work that began with logic formalization (originally George Boole) and Church's lambda calculus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some Common Directions in Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The genetic algorithm influenced convergent technology and neuro-biology influenced neural net development.  Pattern matching is another limb off the larger set of technical approaches under the umbrella of classic Artificial Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are naive systems.  Like a neuron, the components have no idea of the meaning of what they are processing.  They are naive components.  An intelligent observer could not ascertain the real time meanings of signals and symbols between these naive components without extensive, perhaps life-long research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naive Bayesian methods are probabilistic in nature.  They exploit Bayes' Theorem, and have been found to produce excellent results in certain important domains.  Some studies have shown that naivety is actually a learning accelerant, which is interesting from an AI theory point of view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there is fuzzy (weighted) logic, which was an attempt to merge neural nets with production (rule based) systems.  Attempts to use this technology in transportation routing and scheduling has met much success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are as many devices and architectures that attempt to effectively integrate or interconnect these various approaches as there are AI projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Modelling Environment and Goal Conditions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these systems, in some explicit or implicit way, model the external environment and the desired result of system behavior and attempt to converge (in real time) on that result.  Some sense-control functionality, which may change and adapt to the external environment is employed in the CONTROL component(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just another way that the systems have an adaptive behavior.  Without necessarily knowing why, the system will manipulate what it can and continue to monitor the state of the environment to continually reach for the system's acceptance criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Basic AI system must to more than learn.  It must also judge its own functionality and therefore must have a layer of feedback and control that simulates the perception of optimality.  This higher level control must be integrated into the system at its inception for it to behave intelligently in the sense you probably mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Understanding Limitation and System Complexity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more sophisticated and adaptive the modelling becomes, the more the cognitions, rules, stories, time series coefficients, weights, or whatever forms knowledge (not information) is stored, the more one can say that there is some form of understanding or comprehension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One conjecture is that it is the recursion in layers of these capabilities that permits certain types of comprehension and awareness.  Other conjectures focus more on alertness and attention as keys to higher intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these much more mature capabilities are beyond mere adaptation based on past knowledge or information and are therefore beyond what you probably meant by Basic AI.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-08T23:15:27.217" LastActivityDate="2018-12-08T23:15:27.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2562" PostTypeId="1" CreationDate="2016-12-29T23:51:07.190" Score="-3" ViewCount="216" Body="&lt;p&gt;Is there a proof that states the possibility or impossibility of an AI system to acquire more sophisticated capabilities (in terms of generic cleverness) than its own creator?&lt;/p&gt;&#xA;" OwnerUserId="4519" LastEditorUserId="2444" LastEditDate="2019-05-01T17:30:19.783" LastActivityDate="2019-05-01T17:30:19.783" Title="Is there a proof that states that an AI can become smarter than its creator?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;superintelligence&gt;&lt;proofs&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2563" PostTypeId="1" CreationDate="2016-12-30T01:42:39.703" Score="3" ViewCount="119" Body="&lt;p&gt;I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper &lt;a href=&quot;https://arxiv.org/pdf/1608.05343.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Decoupled Neural Interfaces using Synthetic Gradients&lt;/a&gt; (2017) by Max Jaderberg et al. As in the paper, the DNI is able to produce a synthetic error gradient that reflects the error with respect the output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L}{\partial h_{i}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can then use this to update the current layer's parameters by multiplying by the parameters to get the loss with respect to the parameters:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L}{\partial \theta}=\frac{\partial L}{\partial h_{i}} * \frac{\partial h_{i}}{\partial \theta}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the paper, the layer's model is then updated based on the next layer sending the true error backwards.&lt;/p&gt;&#xA;&lt;p&gt;My question is, given that I am able to calculate the error with respect to the current output, how do I use this to calculate the loss with respect to the previous layer's output?&lt;/p&gt;&#xA;" OwnerUserId="4521" LastEditorUserId="2444" LastEditDate="2020-09-29T12:00:42.090" LastActivityDate="2021-02-26T13:07:20.723" Title="How to perform back-propagation in Decoupled Neural Interfaces?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;papers&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2564" PostTypeId="1" CreationDate="2016-12-30T02:24:39.467" Score="5" ViewCount="569" Body="&lt;p&gt;Fundamentally, a game-playing AI must solve the problem of choosing the best action from a set of possible actions.&lt;/p&gt;&#xA;&lt;p&gt;Most existing game AI's, such as AlphaGo, do this by using an &lt;strong&gt;evaluation function&lt;/strong&gt;, which maps game states to real numbers. The real number typically can be interpreted as a monotonic function of a winning probability estimate. The best action is the one whose resultant state yields the highest evaluation.&lt;/p&gt;&#xA;&lt;p&gt;Clearly, this approach can work well. But it violates one of &lt;a href=&quot;https://books.google.com/books?id=N_-5VRWai84C&amp;amp;pg=PA477&amp;amp;lpg=PA477&amp;amp;dq=%22When%20solving%20a%20problem%20of%20interest,%20do%20not%20solve%20a%20more%20general%20problem%20as%20an%20intermediate%20step.%20Try%20to%20get%20the%20answer%20that%20you%20really%20need%20but%20not%20a%20more%20general%20one.%22&amp;amp;source=bl&amp;amp;ots=ReHvsikLSY&amp;amp;sig=WkN0ETtVyEXlgkWWQoiY4b-qWxE&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwj6m6z-g5_OAhWGpB4KHf54DYcQ6AEIKTAC#v=onepage&amp;amp;q=%22When%20solving%20a%20problem%20of%20interest%2C%20do%20not%20solve%20a%20more%20general%20problem%20as%20an%20intermediate%20step.%20Try%20to%20get%20the%20answer%20that%20you%20really%20need%20but%20not%20a%20more%20general%20one.%22&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vladimir Vapnik's imperatives&lt;/a&gt; (in his book &amp;quot;Estimation of Dependences Based on Empirical Data&amp;quot;): &amp;quot;&lt;em&gt;When solving a problem of interest, do not solve a more general problem as an intermediate step.&lt;/em&gt;&amp;quot; In fact, he specifically states as an illustration of this imperative,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do not estimate predictive values if your goal is to act well. (&lt;em&gt;A good strategy of action does not necessarily rely on good predictive ability.&lt;/em&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Indeed, human chess and go experts appear to heed his advice, as they are able to act well without using evaluation functions.&lt;/p&gt;&#xA;&lt;p&gt;My question is this: &lt;strong&gt;has there has been any recent research aiming to solve games by learning to compare decisions directly, without an intermediate evaluation function&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;To use AlphaGo as an example, this might mean training a neural network to take &lt;strong&gt;two&lt;/strong&gt; (similar) board states as input and output a choice of which one is better (a classification problem), as opposed to a neural network that takes &lt;strong&gt;one&lt;/strong&gt; board state as input and outputs a winning probability (a regression problem).&lt;/p&gt;&#xA;" OwnerUserId="4522" LastEditorUserId="2444" LastEditDate="2021-02-06T20:03:36.810" LastActivityDate="2021-02-06T20:03:36.810" Title="Can games be solved without an evaluation function?" Tags="&lt;reference-request&gt;&lt;game-ai&gt;&lt;chess&gt;&lt;go&gt;&lt;evaluation-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2565" PostTypeId="2" ParentId="2542" CreationDate="2016-12-30T02:40:24.280" Score="3" Body="&lt;p&gt;The term &quot;artificially intelligent program&quot; doesn't really mean anything, because it can mean so many different things.   There are a lot of different techniques and approaches that fall under the overall rubric of &quot;artificial intelligence&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, if you were going to try to classify the aspects of AI at a VERY broad (possibly too broad) level, you might say the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have two broad approaches - symbolic (logic based) AI, and probabilistic AI (machine learning).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within the scope of symbolic AI would be most things you hear associated with the phrase GOFAI (Good Old Fashioned AI).  This includes things like logic programming in Prolog, expert systems, production rule systems (OPS5, etc.).  &quot;Cognitive architectures&quot; would probably also fall here, so things like ACT-R, SOAR, CLARION, etc.  And then you have automated planning systems, automated theorem provers, etc.  Skills needed to work here include a good handle on logic - probably first order logic, but possibly higher order logics as well.  Set theory, model theory, things of that nature come into play.  Lisp, Prolog, or OPS5 might be commonly used programming languages.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the area of &quot;probabilistic learning&quot; are things like neural networks, bayesian belief networks, and other &quot;machine learning&quot; algorithms.  Random forests, decision trees and what-not are usually lumped in here as well.  Skills needed to work in this area include some calculus (backpropagation in neural networks, for example, is heavily rooted in the chain rule from calculus), linear algebra, statistics and probability.  Bayesian statistics is especially useful.  Lots of programming languages are used to build these kinds of systems, but popular ones include Python, C++, R, Java and their ilk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you have a few things that don't classify real neatly.  Genetic Algorithms, for example.  Those usually get put into machine learning, but GA's are really more of an optimization strategy.  So you might, for example, use GA's as part of a strategy for training an ANN.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Net-net, &quot;Artificial Intelligence&quot; is a BIG field and you'll probably need to zoom in a little bit and ask more pointed questions to really get anywhere.  You might want to read a relatively comprehensive overview like &quot;Artificial Intelligence - A Modern Approach&quot; by Russell &amp;amp; Norvig to get a good picture of what's going on.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2016-12-30T20:39:02.107" LastActivityDate="2016-12-30T20:39:02.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2566" PostTypeId="2" ParentId="2562" CreationDate="2016-12-30T02:42:53.747" Score="1" Body="&lt;p&gt;The loaded term in your question is &quot;generic cleverness.&quot; There's no such thing. What is &quot;smart&quot; is only smart relative to a criterion. Provide a complete criterion and we can talk rationally about &quot;levels of sophistication.&quot; Until then, there is no measure against which &quot;involuntarily acquired&quot; capabilities can be regarded as more or less &quot;smart.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-12-30T02:42:53.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2569" PostTypeId="2" ParentId="2441" CreationDate="2016-12-30T03:23:08.290" Score="0" Body="&lt;p&gt;Both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ethical responsibility between humans is based on a sympathetic correspondence between humans. Between humans and robots, if one party lacks the desire or capability to sympathize with the other party, no ethical responsibility exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, conservative anthropomorphism applies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is also axis of capability that I think is required in order to warrant human-like 'emancipation' for robots, which is not necessarily anthropomorphic. For lack of a better term, I call this an Arbitrary Machine Generator (AMG). At a species level, extant biological life is an AMG - capable of slowly evolving to solve arbitrary problems, assuming the resources and solutions are available. At an individual level, pre-human animals are not capable of generating arbitrary machines to solve arbitrary problems on individual time-scales. Only humans (and post-humans) are capable of generating arbitrary machines in order to solve arbitrary problems, given the resources and solutions available. Humans can search the space of all possible (resource constrained) solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for a robot to &quot;deserve&quot; the freedom to define it's own purpose, it must first have &lt;em&gt;access&lt;/em&gt; to the space of all possible (within the constraints of available resources) purposes. It then must have purposes and internal contexts of such sufficient complexity and familiarity that we humans are capable of sympathizing with those purposes and internal contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If either of those are not present - the AMG criteria and the sympathetic contexts - then emancipation is not warranted.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2016-12-30T03:23:08.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2570" PostTypeId="2" ParentId="2498" CreationDate="2016-12-30T03:27:28.240" Score="0" Body="&lt;p&gt;Both. I answered this question here also: &lt;a href=&quot;https://ai.stackexchange.com/a/2569/1712&quot;&gt;https://ai.stackexchange.com/a/2569/1712&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if I should expand on that here.&lt;/p&gt;&#xA;" OwnerUserId="1712" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2016-12-30T03:27:28.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2571" PostTypeId="2" ParentId="2514" CreationDate="2016-12-30T05:04:29.640" Score="8" Body="&lt;p&gt;There is lots of misconceptions about AI, specifically the idea that it is about making computers &quot;think&quot; like humans, simulating brain, the sci-fi robots taking over the world, all the philosophical discussions around brain as machine etc. The practice/reality of AI is about &quot;using computing to solve problems&quot; which basically means you take any problem, represent it as a computing problem and then design the algorithm to solve the computing problem which lead to solving the original problem. These search algorithms are general purpose algorithms for general purpose computing problems i.e any real world problem can be represented by these general purpose computing problem and then these algorithms can be used to solve them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember, its about problem solving and its about general purpose computing problems that can represent any real world problem.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2016-12-30T05:04:29.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2572" PostTypeId="2" ParentId="2524" CreationDate="2016-12-30T06:18:56.870" Score="7" Body="&lt;p&gt;Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematical reason for the neural networks prowess at classifying is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt;. Which states that a neural network can approximate any continuous real-valued function on a compact subset. The quality of the approximation depends on the number of neurons. It has also been shown that adding the neurons in additional layers instead of adding them to existing layers improves the quality of the approximation faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Add to that the not well-understood effectiveness of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;noreferrer&quot;&gt;backpropagation&lt;/a&gt; algorithm and you have a setup then can actually learn the function that the UAT promises or something close. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-12-30T06:18:56.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2573" PostTypeId="2" ParentId="88" CreationDate="2016-12-30T07:14:36.183" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www.cs.toronto.edu/~fritz/absps/ncfast.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised pre-training&lt;/a&gt; was done only very shortly, as far as I know, at the time when deep learning started to actually work. It extracts certain regularities in the data, which a later supervised learning can latch onto, so it is not surprising that it might work. On the other hand, unsupervised learning doesn't give particularly impressive results in very deep nets, so it is also not surprising that with current very deep nets, it isn't used anymore. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering whether the initial success with unsupervised pre-training had something to do with the fact that the ideal initialization of neural nets was only worked out later. In that case, unsupervised pre-training would only be a very complicated way of getting the weights to the correct size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unsupervised deep learning is something like the holy grail of AI right now and  hasn't been found yet. Unsupervised deep learning would allow you to use massive amounts of unlabeled data and let the net form its own categories. Later you can just use a little bit of labeled data to give these categories their proper labels. Or just train it immediately on some task, in the conviction that it has a huge amount of knowledge about the world already. This is also what the problem of common sense comes down to: a huge and detailed model of the world, that could only be acquired by unsupervised learning.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="75" LastEditDate="2018-04-13T18:06:03.217" LastActivityDate="2018-04-13T18:06:03.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2574" PostTypeId="2" ParentId="2564" CreationDate="2016-12-30T07:30:54.553" Score="3" Body="&lt;p&gt;Human chess and go experts clearly use evaluation functions. They do come up with moves that look sensible without evaluating the board position, but to validate these candidate moves they evaluate board positions that occur at the end of the variations they calculate. Pretty similar to AlphaGo. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inputting two board states and outputting a preference is a (much) more complex task than mapping one board state into the real numbers. And it gives you less information. So its a lose-lose choice. (I did try something very similar and it didn't work at all. The reason is that you didn't just double the input size, rather you made the input space quadratically bigger.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compare two board states that differ just by one move, then your input space doesn't quite explode as much, but you have to do a ton of comparisons to make a decision. The logical choice would be to output a preference distribution over all possible moves - but that's exactly what AlphaGo does in its policy network. There is also an &lt;a href=&quot;https://arxiv.org/abs/1412.3409&quot; rel=&quot;nofollow noreferrer&quot;&gt;earlier paper&lt;/a&gt; which trained a network to predict expert moves, which comes down to the same thing. And yes, both these networks play quite strongly without any search or board evaluation. But nowhere near AlphaGo-level.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2016-12-30T07:30:54.553" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2576" PostTypeId="2" ParentId="2562" CreationDate="2016-12-30T14:17:24.373" Score="1" Body="&lt;p&gt;If you look at the work of Howard Gardner and his theory of multiple intelligences, you will see that the term of &quot;intelligence&quot; respectively &quot;generic cleverness&quot; is much more diverse and not entirely clarified. Without a entire notion of it, a proof is forlorn.&lt;/p&gt;&#xA;" OwnerUserId="4528" LastActivityDate="2016-12-30T14:17:24.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2577" PostTypeId="1" AcceptedAnswerId="2587" CreationDate="2016-12-30T15:09:53.240" Score="2" ViewCount="135" Body="&lt;p&gt;Suppose I have been given the following diagram&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SbXDo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SbXDo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to design a neural network to separate the points or classes. How can I do that?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="2444" LastEditDate="2019-06-20T22:04:24.253" LastActivityDate="2019-06-20T22:04:24.253" Title="Is it possible to design a neural network from the plot of data points?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2578" PostTypeId="1" AcceptedAnswerId="2579" CreationDate="2016-12-30T17:19:59.427" Score="1" ViewCount="110" Body="&lt;p&gt;The &lt;a href=&quot;http://kryten.mm.rpi.edu/lovelace.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original Lovelace Test&lt;/a&gt;, published in 2001, is used generally as a thought experiment to prove that AI cannot be creative (or, more specifically, that it cannot &lt;em&gt;originate&lt;/em&gt; a creative artifact). From the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Artificial Agent &lt;em&gt;A&lt;/em&gt;, designed by &lt;em&gt;H&lt;/em&gt;, passes LT if and only if&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;A&lt;/em&gt; outputs &lt;em&gt;o&lt;/em&gt;,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;A&lt;/em&gt; outputting &lt;em&gt;o&lt;/em&gt; is not the result of a fluke hardware error, but rather the result of processes &lt;em&gt;A&lt;/em&gt; can repeat&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;H&lt;/em&gt; (or someone who knows what &lt;em&gt;H&lt;/em&gt; knows, and has &lt;em&gt;H&lt;/em&gt;'s resources) cannot explain how &lt;em&gt;A&lt;/em&gt; produced &lt;em&gt;o&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The authors of the original Lovelace Test then argues that it is impossible to imagine a human developing a machine to create an artifact...while also not knowing how that machine worked. For example, an AI that uses machine learning to make a creative artifact &lt;em&gt;o&lt;/em&gt; is obviously being 'trained' on a dataset and is using some sort of algorithm to be able to make predictions on this dataset. Therefore, the human &lt;em&gt;can&lt;/em&gt; explain how the AI produced &lt;em&gt;o&lt;/em&gt;, and therefore the AI is not creative.&lt;/p&gt;&#xA;&lt;p&gt;The Lovelace Test seems like an effective thought experiment, even though it appears to be utterly useless as an actual test (which is why the &lt;a href=&quot;https://ai.stackexchange.com/questions/1451/has-the-lovelace-test-2-0-been-successfully-used-in-an-academic-setting&quot;&gt;the Lovelace Test 2.0 was invented&lt;/a&gt;). However, since it does seem like an effective thought experiment, there must be some arguments against it. I am curious to see any flaws in the Lovelace Test that could undermine its premise.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-12-30T18:11:00.777" Title="Are there any refutation of the original Lovelace Test?" Tags="&lt;intelligence-testing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2579" PostTypeId="2" ParentId="2578" CreationDate="2016-12-30T17:42:27.013" Score="3" Body="&lt;p&gt;I am a future neurologist with a very complete understanding of linguistic processing in the brain.  I am also an overprotective parent, so I monitor every phrase uttered to my child, and also completely determine all the books she reads in the course of her education.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When my child writes a poem, then, I know the dataset on which her brain was trained, as well as the processes by which her language inputs became language outputs--in broad outline I know these processes are non-linear and are based on how different inputs along with the current collection of trillions of distinct synaptic weights updates the synaptic weights.  I don't know what her poem will be, of course, because there are random factors and the whole history of her synaptic weights are unobservable, but I adhere to the Lovelace test and can therefore conclude that composing the poem was not a creative act.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Lovelace Test, like the Chinese Room Argument, implicitly assumes that what computers/AI can do in processing symbols and information and what brains can do are distinct.  If you accept that assumption, then the argument ceases to be interesting-- you've merely redefined creativity as one of the distinct things that brains can do.  If you reject the assumption, the argument that computers are incapable of creativity ceases to be valid.  The thought experiment itself does nothing to assist us in evaluating the truth of the assumption.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2329" LastEditDate="2016-12-30T18:11:00.777" LastActivityDate="2016-12-30T18:11:00.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2580" PostTypeId="1" AcceptedAnswerId="2585" CreationDate="2016-12-30T18:02:31.660" Score="2" ViewCount="1414" Body="&lt;p&gt;I am looking to make an AI like &lt;a href=&quot;https://github.com/sukeesh/Jarvis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jarvis&lt;/a&gt;. A perfect real-life example of this type of system is the simple AI that Mark Zuckerberg has recently built. &lt;a href=&quot;https://www.facebook.com/notes/mark-zuckerberg/building-jarvis/10154361492931634&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a description of how his AI works. From what I understand, the AI understands keywords, context, synonyms, and then from there decides what to do. &lt;a href=&quot;https://www.youtube.com/watch?v=vvimBPJ3XGQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is also a video of the AI in action.&lt;/p&gt;&#xA;&lt;p&gt;I have many questions about how these systems works. Firstly, what necessary steps are required to gather the meaning of a input? Secondly, how does the system, once it extracts all of the necessary information on the input, determine what action it needs to take and what to say back to the user? Lastly, it also states that the system can learn the habits and preferences of the user. How can a system do this?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="2444" LastEditDate="2020-12-17T01:55:22.453" LastActivityDate="2020-12-17T01:55:22.453" Title="How can a system like Jarvis understand the commands and take actions?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;natural-language-generation&gt;&lt;intelligent-personal-assistants&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2582" PostTypeId="2" ParentId="2498" CreationDate="2016-12-31T04:23:57.653" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Reality of Working in the Field&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most in the fields of adaptive systems, machine learning, machine vision, intelligent control, robotics, and business intelligence, in the corporations and universities in which I've worked do not discuss this topic much in meetings or at lunch.  Most are too busy building things that must work by some deadline to muse over things that are not of immediate concern, and bot-rights are a long way off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How Far Off?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To begin with, no bot has yet passed a properly conducted Turing Test. (There is much on the net about this test, including critique of poorly conducted testing of this type.  See Searle's Chinese Room thought experiment.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Language simulation with semantic understanding is difficult enough without adding creativity, coordination, feelings, intuition, body language, learning of entirely new domains from scratch, and the potential of genius.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In synopsis, we a long way from the procurement of bots that simulate humanity sufficiently to be considered for citizenship, even in a progressive country that abhors fundamentalism of any kind.  No actual imbuement of rights will occur until we have bot-citizenship in one or more countries.  Consider that human fetuses do not yet have rights because they are not yet deemed citizens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Relevance of the Answer for Today&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In current culture, conservative anthropocentricm and post-human fundamentalism arrive at the same effective conclusion, and that may continue to be the case for a hundred years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those with experience across fields of psychology, neuro-biology, cybernetics, and adaptive systems know that the simulation of all the mental features we attribute to humans is to copy in algorithms the layering of cerebral abilities over a reptilian brain that went through millions of years of field testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Impact of Science Fiction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asking around, it is likely you would get some feedback that is mostly gained from the media of our culture, not philosophic theses and publications written by those who don't actually have any deadlines to produce anything that functions IRL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isaac Asimov investigated some conservative anthropomorphism concepts in scenarios depicted in his short stories.  Commander Data's human quirks in the Next Generation Star Trek teleplays furthered some of those ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Christopher Nolan took the opposite direction in the Interstellar screenplay, with the robots having interesting personalities that could be altered by linear settings.  His bots ignored concerns of self-preservation, apparently without any cognitive resistance.  This depiction is an unapologetic post-human fundamentalist view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Thought Experiment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's place the citizen issue aside for to consider this thought experiment, and let's assume that a survey would show a leaning toward conservative anthropocentrism among current AI researchers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider an intelligent piece of software constructed a legal complaint to gain intellectual property rights over day trading code it wrote and sent it to the appropriate court clerk, you might find that the same researchers would recant.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Post-human fundamentalism will probably prevail when real AI software theorists and engineers consider the true personal, corporate, and meaning of settling out of court or losing the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Would Researchers Cut the Umbilical Cord?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I asked one researcher and she indicated that all her lawyer would need to do to win the case likely consider the precedence that might occur and recall some of the warnings built into the Terminator stories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my observation of humanity in my life time, my prediction is that people want slaves not some brand of bots that could ultimately kick our butts in an all out fight.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2016-12-31T04:23:57.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2583" PostTypeId="1" AcceptedAnswerId="2586" CreationDate="2016-12-31T05:00:39.783" Score="1" ViewCount="677" Body="&lt;p&gt;What is the difference between stacked restricted Boltzmann machine (RBM) and deep belief network (DBN)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are they the same entity? If so, why? Is the latter some specific type of the former? If so, how to tell if stacked RBM is a DBN?&lt;/p&gt;&#xA;" OwnerUserId="4531" LastEditorUserId="2444" LastEditDate="2020-05-18T12:24:43.780" LastActivityDate="2020-05-18T12:24:43.780" Title="What is the difference between a stacked restricted Boltzmann machine and deep belief network?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;boltzmann-machine&gt;&lt;deep-belief-network&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2584" PostTypeId="2" ParentId="2436" CreationDate="2016-12-31T06:23:04.337" Score="2" Body="&lt;p&gt;Have you tried &lt;a href=&quot;http://www.nltk.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLTK&lt;/a&gt;, what you are looking for is in &lt;a href=&quot;http://www.nltk.org/book/ch06.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 6&lt;/a&gt; of the book. Basically what you need to do is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Tokenize the user input.&lt;/li&gt;&#xA;&lt;li&gt;Extract vector set from the tokenized words.&lt;/li&gt;&#xA;&lt;li&gt;Train your model with some given texts, and same vector sets.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And you can use your model to categorize the document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other suggestion, instead of extracting vector sets you can use every word in the input&#xA;to be evaluated in to some category using a training set of large corpus, which you are sure&#xA;it contains all the words.&#xA;And then you multiply probability of each word being on a category to decide where the document belongs.&lt;/p&gt;&#xA;" OwnerUserId="4543" LastEditorUserId="8" LastEditDate="2017-01-12T12:27:07.703" LastActivityDate="2017-01-12T12:27:07.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2585" PostTypeId="2" ParentId="2580" CreationDate="2016-12-31T07:49:40.360" Score="3" Body="&lt;p&gt;Jarvis was built using the suite of tools that facebook developers are constantly updating. The answer to this question is that there's no simple answer; it has a lot of moving parts.&lt;/p&gt;&#xA;&lt;p&gt;Take for example natural language processing. There are a number of sub-topics that are each considered &amp;quot;big&amp;quot; problems, such as part-of-speech recognition, coreference resolution, sentiment analysis, relationship extraction, and many more. Tools have been built to tackle these various topics, but, to my knowledge, none of them really &lt;em&gt;understand&lt;/em&gt; language, but rather statistically approximate it.&lt;/p&gt;&#xA;&lt;p&gt;In the case of Jarvis, since it is a home-automation system, it's probably built with the user-given commands in mind from the beginning, so it's not trying to understand the whole human language, it's built to do some tricks.&lt;/p&gt;&#xA;&lt;p&gt;Just keep in mind on your journey into this space that true AI like what we imagine will have some defining features like hierarchical representations of tasks and goal-orientation. If you really get into it I'd start with reinforcement learning, or try reading through the Society of Minds.&lt;/p&gt;&#xA;" OwnerUserId="4544" LastEditorUserId="2444" LastEditDate="2020-12-17T01:52:47.393" LastActivityDate="2020-12-17T01:52:47.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2586" PostTypeId="2" ParentId="2583" CreationDate="2016-12-31T08:08:05.530" Score="1" Body="&lt;p&gt;For what it's worth, &lt;a href=&quot;https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; says that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;deep belief networks can be formed by &quot;stacking&quot; RBMs. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hinton &lt;a href=&quot;http://www.scholarpedia.org/article/Deep_belief_networks#Deep_Belief_Nets_as_Compositions_of_Simple_Learning_Modules&quot; rel=&quot;nofollow noreferrer&quot;&gt;writes in Scholarpedia&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A deep belief net can be viewed as a composition of simple learning modules each of which is a restricted type of Boltzmann machine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, a deep belief network is definitely a stacked RBM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never heard of different stacked RBMs, but it is easy to imagine something like convolutional stacked RBMs, where some RBMs are used as filters that slide over the input data or something. Whether that would still be called a deep belief network, is probably up to the guy who publishes it first. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2020-05-18T12:06:09.230" LastActivityDate="2020-05-18T12:06:09.230" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2587" PostTypeId="2" ParentId="2577" CreationDate="2016-12-31T13:36:58.587" Score="2" Body="&lt;p&gt;From the diagram you have given, it's quite clear that you have to design a &lt;strong&gt;supervised&lt;/strong&gt; network. Also, it's clear that you are dealing with a problem that's not &lt;em&gt;linearly separable&lt;/em&gt; i.e. you can't separate two classes using a single line. Particularly, it will require three lines to separate these two classes.&lt;/p&gt;&#xA;&lt;p&gt;From the above observations, what you can do is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Make training data sets of form &lt;code&gt;[(x,y), o]&lt;/code&gt; where &lt;code&gt;(x,y)&lt;/code&gt; are the co-ordinates on the plot and &lt;code&gt;o&lt;/code&gt; is the class that point belongs to.&lt;/p&gt;&#xA;&lt;p&gt;Note: You can actually divide your data set into training data and test data. Just, extract the data, randomly shuffle it and take first 80% of data to train and rest 20% of data to test.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Design a 2-2-2-1 neural network with bias units and some random initial weight values. This is because we will require minimum three lines to separate two classes, so 2 hidden layers (each consisting of 2 nodes and bias nodes) and 1 output layer (with bias node). The first layer is input layer with 2 nodes (no bias) as we have x-coordinate and y-coordinate.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train your training data using this network using the Error Back Propagation algorithm i.e. update the weights between the layers and the bias weights too. Run the EBP algorithm for 100-2000 epochs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then, use the test data to see, if you get the desired output.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Note: I am NOT quite sure how to obtain the coordinates from the diagram. You can either manually note all points or research for some methods to get those points automatically from the image (some image processing algorithms).&lt;/p&gt;&#xA;&lt;p&gt;Edit: I found a good &lt;a href=&quot;https://youtu.be/BR9h47Jtqyw&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; which explains neural networks. In the video, there is non-linear data like the one in your image which is separable using multiple lines and the network is designed to try to find curves such that data can be separated and the output resembles to that when used straight lines.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-12-31T14:42:06.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2588" PostTypeId="1" CreationDate="2016-12-31T13:39:37.960" Score="0" ViewCount="247" Body="&lt;p&gt;I Build this NN in c++. I reviewed it since 3 days. I checked every line 100 times, but I cant find my error.&#xA;If someone can please help me find the Bugs:&#xA;1. The output is garbage&#xA;2. The weights go from 2e^79 down to -1.8e^80 after approximatly 400 iterations.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mat flip(mat m) {&#xA;    mat out(m.n_cols,m.n_rows);&#xA;&#xA;    for (int i = 0; i &amp;lt; m.n_rows; ++i)&#xA;        for (int j = 0; j &amp;lt; m.n_cols; ++j)&#xA;            out(j, i) = m(i, j);&#xA;&#xA;    return out;&#xA;}&#xA;&#xA;Layer::Layer(int nodes) :&#xA;    rand_engine(time(0))&#xA;{&#xA;&#xA;    y = mat (nodes, 1);&#xA;    net = mat(nodes, 1);&#xA;    e = mat(nodes, 1);&#xA;}&#xA;&#xA;Layer::Layer(int nodes, int next_nodes) :&#xA;    Layer(nodes)&#xA;{&#xA;&#xA;    this-&amp;gt;next_l = next_l;&#xA;&#xA;    auto random = bind(uniform_real_distribution&amp;lt;double&amp;gt;{-1, 1}, rand_engine);&#xA;&#xA;    w = mat(next_nodes,nodes);&#xA;&#xA;    for (int i = 0; i &amp;lt; w.n_rows; ++i) {&#xA;&#xA;        for (int j = 0; j &amp;lt; w.n_cols; ++j) {&#xA;            w(i,j) = random();&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;Layer::Layer(int nodes, Layer* next_l) :&#xA;    Layer(nodes,next_l-&amp;gt;y.n_rows)&#xA;{&#xA;    this-&amp;gt;next_l = next_l;&#xA;}&#xA;&#xA;void Layer::feed_forward()&#xA;{&#xA;    next_l-&amp;gt;net = w*y;&#xA;&#xA;&#xA;    for (int i = 0; i &amp;lt; next_l-&amp;gt;y.n_rows;++i) &#xA;        next_l-&amp;gt;y[i] = sig(next_l-&amp;gt;net[i]);&#xA;&#xA;&#xA;}&#xA;&#xA;void Layer::backprop()&#xA;{&#xA;&#xA;    for (double d : w)&#xA;        cout &amp;lt;&amp;lt; d &amp;lt;&amp;lt; &quot;\t&quot;;&#xA;&#xA;    e = flip(w)*next_l-&amp;gt;e;&#xA;&#xA;&#xA;    for (int i = 0; i &amp;lt; e.n_rows; ++i) {&#xA;        e[i] *= net[i] * (1 - net[i]);  &#xA;        cout &amp;lt;&amp;lt; e[i] &amp;lt;&amp;lt; '\t';&#xA;    }&#xA;&#xA;    w += l_rate*(next_l-&amp;gt;e*flip(y));&#xA;&#xA;&#xA;}&#xA;&#xA;void Layer::backprop_last(mat t)&#xA;{&#xA;    for (int i = 0; i &amp;lt; e.n_rows; ++i) {&#xA;        e[i] = net[i] * (1 - net[i])*(t[i] - y[i]);&#xA;        cout &amp;lt;&amp;lt; e[i] &amp;lt;&amp;lt; '\t';&#xA;    }&#xA;&#xA;}&#xA;&#xA;void Layer::feed_forward(Layer* next_l)&#xA;{&#xA;    this-&amp;gt;next_l = next_l;&#xA;&#xA;    feed_forward();&#xA;&#xA;}&#xA;&#xA;double Layer::sig(double x)&#xA;{&#xA;    return 1 / (1 + exp(-x));&#xA;}&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;Network::Network(vector&amp;lt;int&amp;gt; top):&#xA;    top(top)&#xA;{&#xA;&#xA;    network = new Layer*[top.size()];&#xA;    network[top.size() - 1] = new Layer(top.back());&#xA;&#xA;    for (int i = top.size()-2; i &amp;gt; -1; --i)&#xA;        network[i] = new Layer(top[i], network[i + 1]);&#xA;&#xA;}&#xA;&#xA;&#xA;Network::~Network()&#xA;{&#xA;    delete[] network;&#xA;}&#xA;&#xA;void Network::forward()&#xA;{&#xA;    for(int i = 0; i &amp;lt; top.front();++i)&#xA;        network[0]-&amp;gt;y[i] = input[i];&#xA;&#xA;    for (int i = 0; i &amp;lt; top.size() - 1; ++i)&#xA;        network[i]-&amp;gt;feed_forward();&#xA;}&#xA;&#xA;void Network::forward(vector&amp;lt;double&amp;gt; input)&#xA;{&#xA;    set_input(input);&#xA;    forward();&#xA;}&#xA;&#xA;void Network::backprop()&#xA;{&#xA;&#xA;    network[top.size() - 1]-&amp;gt;backprop_last(t_vals);&#xA;&#xA;    for (int i = top.size() - 2; i &amp;gt; -1; --i) {&#xA;        network[i]-&amp;gt;backprop();&#xA;    }&#xA;&#xA;}&#xA;&#xA;void Network::backprop(vector&amp;lt;double&amp;gt; t_vals)&#xA;{&#xA;    set_t_vals(t_vals);&#xA;    backprop();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know its a bunch of code but im really desprate since I cant find whats wrong. I tested it with a simple XOR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;Heres my Main code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #include &quot;Network.h&quot;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;vector&amp;lt;vector&amp;lt;double&amp;gt;&amp;gt; input = { {0,0},{0,1},{1,1},{1,0} };&#xA;&#xA;vector&amp;lt;vector&amp;lt;double&amp;gt;&amp;gt; true_vals = { {0},{1},{0},{1} };&#xA;&#xA;int main() {&#xA;&#xA;    ifstream f(&quot;out.txt&quot;, fstream::out);&#xA;    f.clear();&#xA;&#xA;    cout &amp;lt;&amp;lt; fixed;&#xA;    cout &amp;lt;&amp;lt; setprecision(5);&#xA;&#xA;    Network net({2,5,1});&#xA;&#xA;    vector&amp;lt;double&amp;gt; in,t,out; &#xA;&#xA;    auto buf = cout.rdbuf();&#xA;&#xA;    for (int i = 0; i &amp;lt; 1000; ++i) {&#xA;        cout.rdbuf(f.rdbuf());&#xA;&#xA;&#xA;        in = input[i % 4];&#xA;&#xA;        net.forward(in);&#xA;&#xA;        out = net.get_output();&#xA;&#xA;        t = true_vals[i % 4];&#xA;&#xA;        net.backprop(t);&#xA;        cout &amp;lt;&amp;lt; '\n';&#xA;        cout.rdbuf(buf);&#xA;        if ((i %101))continue;&#xA;&#xA;        cout &amp;lt;&amp;lt; &quot;it: &quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt; '\n';&#xA;&#xA;        cout &amp;lt;&amp;lt; &quot;in:\t&quot;;&#xA;        for (double d : in)&#xA;            cout &amp;lt;&amp;lt; d &amp;lt;&amp;lt; ' ';&#xA;&#xA;        cout &amp;lt;&amp;lt; '\n';&#xA;&#xA;        cout &amp;lt;&amp;lt; &quot;out:\t&quot;;&#xA;&#xA;        for (double d : out)&#xA;            cout &amp;lt;&amp;lt; d &amp;lt;&amp;lt; ' ';&#xA;&#xA;        cout &amp;lt;&amp;lt; '\n';&#xA;&#xA;&#xA;        cout &amp;lt;&amp;lt; &quot;true:\t&quot;;&#xA;&#xA;&#xA;        for (double d : t)&#xA;            cout &amp;lt;&amp;lt; d &amp;lt;&amp;lt; ' ';&#xA;&#xA;        cout &amp;lt;&amp;lt; '\n';&#xA;&#xA;&#xA;        double err = net.get_error();&#xA;&#xA;        cout &amp;lt;&amp;lt;&quot;err:\t&quot;&amp;lt;&amp;lt; err &amp;lt;&amp;lt; '\n' &amp;lt;&amp;lt; '\n';&#xA;&#xA;    }&#xA;&#xA;    cout.rdbuf(NULL);&#xA;    f.close();&#xA;    return system(&quot;pause&quot;);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="4550" LastEditDate="2016-12-31T14:06:00.553" LastActivityDate="2016-12-31T15:05:49.610" Title="Why doesnt my Neural Network work?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="8" ClosedDate="2021-04-29T13:32:32.020" ContentLicense="CC BY-SA 3.0" />
  <row Id="2589" PostTypeId="2" ParentId="2588" CreationDate="2016-12-31T14:19:13.043" Score="2" Body="&lt;p&gt;A little search on Google answers your question.&lt;/p&gt;&#xA;&lt;p&gt;XOR input space is not linearly separable. It means that you cannot separate the input points in a 2D space into 1 area and 0 area by simply drawing a line between them. It requires at least 2 lines to separate the XOR input space and consequently 2 output nodes (used as classifiers rather than regression). You can easily find its details in google. Search &amp;quot;XOR problem in Neural Net&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;You can manually implement the desired Neural Network with two output Nodes acting as classifiers as follows :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KGw32.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KGw32.jpg&quot; alt=&quot;Manual Implementation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where A &amp;amp; B are two output Nodes which act as classifiers (by forming AA' and BB' decision lines respectively during training by Backpropagation). The interpretation of the Outputs of the nodes is given in the table where Net column represents the Overall output to be interpreted.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I showed the above manual implementation just to give you the idea of how classification is done behind the scenes.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the actual Automatic implementation :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aOumf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aOumf.jpg&quot; alt=&quot;Practical Implementation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here , all the task is performed by the Neural Net behind the scenes and you get the desired output from the output node in the topmost layer&lt;/p&gt;&#xA;" OwnerUserId="4424" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2016-12-31T15:05:49.610" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2590" PostTypeId="1" AcceptedAnswerId="2596" CreationDate="2016-12-31T15:57:03.323" Score="5" ViewCount="266" Body="&lt;p&gt;I am going to design a Neural Net which will be able to break a 5 letter (characters) word into its corresponding syllables (hybrid syllables, I mean it will not strictly adhere to grammatical Syllable rules but will be based on some training sets I provide).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example : &#xA;                  Train -&gt; tra-in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think of implementing it in terms of some feedforward net as follows :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input layer -&gt;Hidden layers -&gt; Output layer&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will be 5 input nodes in the form of decimals (1/26 =0.038 for 'A' ; 2/26 = 0.076 for 'B' ......)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output layer consists of 4 Nodes which corresponds to each gap between two characters in the word. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And fires as follows :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &quot;&lt;strong&gt;TRAIN&lt;/strong&gt;&quot; (TRA-IN): &lt;strong&gt;Input (0.769,0.692,0.038,0.346,0.538)&lt;/strong&gt;&#xA;                               &lt;strong&gt;Output(0,0,1,0)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &quot;&lt;strong&gt;BORIC&quot; (BO-RI-C): **Input....&lt;/strong&gt;&#xA;&lt;strong&gt;Output (0,1,0,1)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it at all possible to implement the Neural Nets in the way I am doing??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if possible, then how will I decide the number of Hidden layers and Nodes in each layer??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;( In the book I am reading, XOR gate problem and its implementation using hidden layer is given . In XOR we could decide the number of Nodes and Hidden Layers required by seeing the Linear Separability of XOR using two lines. &lt;strong&gt;But here I think such analysis can't be made.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So how do I proceed?? Or is it a trial and error process?&lt;/strong&gt;)&lt;/p&gt;&#xA;" OwnerUserId="4424" LastActivityDate="2017-01-02T13:31:52.200" Title="How to decide Linear Separability in my Neural Net work?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2591" PostTypeId="2" ParentId="2524" CreationDate="2017-01-01T17:02:33.327" Score="0" Body="&lt;p&gt;In Neural Networks we consider everything in high dimension and try to find a hyperplane that classify them by small changes...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably it is hard to prove that it works but intuition says if it can be classified you can do it by add a relaxed plane and let it move amongst data to find a local optimum... &lt;/p&gt;&#xA;" OwnerUserId="4439" LastActivityDate="2017-01-01T17:02:33.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2594" PostTypeId="1" CreationDate="2017-01-01T20:02:05.957" Score="2" ViewCount="105" Body="&lt;p&gt;I am currently trying to understand and implement a conversational agent, seeing in the network there are many apis to do something similar, but what they generate are &quot;intelligent&quot; bots, not intelligent conversational agents (wit.ai, recast.ai, Api.ai, etc.), however I have seen Watson virtual agent which paints very well and seems to cover my needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I am a developer and I would like to ask those with more experience, which would be the way to go to implement my objective, an agent similar to what the video of watson virtual agent, with thematic ones that I can train in the agent, and That he can learn from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a language course, but focused on the generation of programming languages, lexical analysis, syntactic, semantic, etc., however I know that the natural language can not be compared to the language of the machines, reading some thesis vi to make a Conversational agent could do a great grammar (I can not imagine its syntactic tree), using probabilities with ngrams, or using neural networks or expert systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the expert systems I understand that for these &quot;learn&quot; needs their knowledge base be modified, and as for the neural networks these fit, &quot;learn&quot;, so I think that it is best to use neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summarizing which way should I go? , I'm currently taking stanford's natural language processing course, and a deep learning course from google, I thought I'd use Natural Language Tool Kit(ntlk) for that important or natural part.&lt;/p&gt;&#xA;" OwnerUserId="4563" LastEditorUserId="4302" LastEditDate="2018-10-08T12:34:00.657" LastActivityDate="2018-10-08T12:34:00.657" Title="Conversational agent,query" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;chat-bots&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2018-09-28T18:29:03.820" ContentLicense="CC BY-SA 4.0" />
  <row Id="2595" PostTypeId="2" ParentId="2524" CreationDate="2017-01-02T12:09:49.667" Score="1" Body="&lt;p&gt;With Neural Networks you simply classify datas. If you classify correctly, so you can do future classifications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How It Works?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple neural networks like Perceptron can draw &lt;strong&gt;one&lt;/strong&gt; decision boundary in order to classify datas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example suppose you want to solve simple AND problem with simple Neural Network. You have 4 sample data containing x1 and x2 and weight vector containing w1 and w2. Suppose initial weight vector is [0 0]. If you made calculation which depend on NN algoritm. At the end, you should have a weight vector [1 1] or something like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ELIum.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ELIum.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please focus on the graphic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says: I can classify input values into two classes (0 and 1). Ok. Then how can I do this? It is too simple. First sum input values (x1 and x2). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;0+0=0&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;0+1=1&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1+0=1&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1+1=2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if sum&amp;lt;1.5 then its class is 0&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;if sum&gt;1.5 then its class is 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3763" LastActivityDate="2017-01-02T12:09:49.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2596" PostTypeId="2" ParentId="2590" CreationDate="2017-01-02T13:31:52.200" Score="3" Body="&lt;p&gt;I would highly recommend modeling things differently with regard to how letters are presented to the model. While the problem is more natural, perhaps, for a Convolutional or Recurrent Neural Network, there's no problem to try and run this on a feed forward network. However, the way you give letters as input will be very confusing for a network and will make learning very hard. I'd recommend using one hot encoding or even a binary encoding for the letters. If this is for more than playing around I'd try and add some info (encode whether the letter is in &quot;aeiou&quot; in a separate bit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the hidden layers, try playing around a bit. Two systematic approaches are to start very simple and make the model more complicated, or start complicated and make your model simpler (or just normalize a lot). Look at the performance on the training set and on a separate validation set during training. If the model keeps on improving on the training data but starts to deteriorate on the validation data, you're probably over fitting. In this case you should either make the model simpler (fewer nodes, fewer layers) or regularize (start with l2 normalization on the weights). If the data doesn't perform well on the training data, you may wish to make the model more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you've tried the feedforward network, really do try CNN or RNNs for this task.&lt;/p&gt;&#xA;" OwnerUserId="4480" LastActivityDate="2017-01-02T13:31:52.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2598" PostTypeId="1" AcceptedAnswerId="2601" CreationDate="2017-01-02T16:15:19.750" Score="5" ViewCount="360" Body="&lt;p&gt;I've spent the past couple of months learning about neural networks, and am thinking of projects that would be fun to work on to cement my understanding of this tech.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that came to mind last night is a system that takes an image of a movie poster and predicts the genre of the movie. I think I have a good understanding of what'd be required to do this (put together a dataset, augment it, download a convnet trained on imagenet, finetune it on my dataset, and go from there).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also thought that it would be pretty cool to run the system backwards at the end, so that I could put in e.g. a genre like 'horror' and have the system generate a horror movie poster. I expect that it will be very bad at this because I'm not a team of expert researchers, but I think I could have some fun hacking on it even if it only ever generated incomprehensible results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what I'm having trouble understanding: on the one hand, all the convnets whose architecture I've seen described seem to rely on being given very small, square input images (on the order of 220px by 220px iirc), and movie posters are rectangular, and a generated poster would have to be of a larger size in order for a human to make any sense of it.  I've seen several examples of papers where researchers use convnets to generate images, e.g. the adversarial system that generates pictures of birds and flowers, and a system that generates the next few frames of video when given a feed of a camera sweeping across the interior of a room, but all of those generated images seemed to be of the small square size I've been describing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I've seen lots of &quot;deep dream&quot; images over the past year or so that have been generated by convnets and are of a much larger size than ~220px by ~220px.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my question: is it possible for me to build the system I describe, which takes a movie genre and outputs a movie poster of a size like e.g. 400px by 600px? [I'm not asking about whether or not the resulting poster would be any &lt;em&gt;good&lt;/em&gt; - I'm curious about whether or not it's possible to use a convnet to generate an image of that size.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is possible, &lt;em&gt;how&lt;/em&gt; is it possible, given that these systems seem to expect small, square input images?&lt;/p&gt;&#xA;" OwnerUserId="4579" LastActivityDate="2019-03-01T11:42:58.187" Title="Feasibility of generating large images with a convnet" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2599" PostTypeId="1" AcceptedAnswerId="2600" CreationDate="2017-01-02T19:39:16.703" Score="4" ViewCount="358" Body="&lt;p&gt;What kind of algorithm is used by StackGAN to generate realistic images from text? How does StackGAN work?&lt;/p&gt;&#xA;" OwnerUserId="4463" LastEditorUserId="2444" LastEditDate="2020-08-18T23:36:20.657" LastActivityDate="2020-08-18T23:36:20.657" Title="What kind of algorithm is used by StackGAN to generate realistic images from text?" Tags="&lt;computer-vision&gt;&lt;reference-request&gt;&lt;generative-adversarial-networks&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2600" PostTypeId="2" ParentId="2599" CreationDate="2017-01-02T22:37:15.717" Score="3" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1612.03242v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;StackGAN: Text to Photo-realistic Image Synthesis&#xA;with Stacked Generative Adversarial Networks&lt;/a&gt; should provide the answers to your questions.&lt;/p&gt;&#xA;&lt;p&gt;Here's an excerpt from the abstract of the paper.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Synthesizing photo-realistic images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing text-to-image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose stacked Generative Adversarial Networks (StackGAN) to generate photo-realistic images conditioned on text descriptions. The &lt;strong&gt;Stage-I&lt;/strong&gt; GAN sketches the primitive shape and basic colors of the object based on the given text description, yielding Stage-I low resolution images. The&#xA;&lt;strong&gt;Stage-II&lt;/strong&gt; GAN takes Stage-I results and text descriptions as inputs, and generates high resolution images with photorealistic details. The Stage-II GAN is able to rectify defects and add compelling details with the refinement process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4579" LastEditorUserId="2444" LastEditDate="2020-08-18T23:35:08.837" LastActivityDate="2020-08-18T23:35:08.837" CommentCount="0" CommunityOwnedDate="2020-08-18T23:40:06.300" ContentLicense="CC BY-SA 4.0" />
  <row Id="2601" PostTypeId="2" ParentId="2598" CreationDate="2017-01-03T01:41:03.257" Score="2" Body="&lt;p&gt;The way you would recognize images (image -&gt; genre) would be very different from the other way around (genre -&gt; image). For the former you are correct on what would be involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the latter, if you want large images then GANs are indeed the way to go. Currently the largest images we can generate are on the order of 220 x 220 pixels, mostly due to memory constraints on the GPU. There is no fundamental problem with having rectangular images, it just so happens that we use squares. You would be able to use identical architectures to train on rectangular data as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason that some generated images you see (e.g. from DeepDream or NeuralStyle) are larger is that it's not a GAN. Both of these are not generative models in a standard sense, even if they do technically &quot;generate&quot; images. Instead, they merely modify an existing image by running backprop on a specifically designed loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR: your movie genre recognition idea is sound. For generation, have a look at things like this (&lt;a href=&quot;https://github.com/Newmu/dcgan_code&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Newmu/dcgan_code&lt;/a&gt;), where Alec generated album covers instead of posters. If you have enough data it will do something half sensible.&lt;/p&gt;&#xA;" OwnerUserId="4581" LastActivityDate="2017-01-03T01:41:03.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2602" PostTypeId="1" AcceptedAnswerId="2609" CreationDate="2017-01-03T12:56:54.673" Score="1" ViewCount="591" Body="&lt;p&gt;I'm working on a project which uses artificial neural network. I looked up at the Matlab Neural Network toolbox. I got a Generated Script from it. When looking at this script, it is confusing because for both testing and training it seems that the toolbox just uses the same data. Could you explain the reason?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script is given below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;net.divideParam.trainRatio = 70/100;&#xA;net.divideParam.valRatio = 15/100;&#xA;net.divideParam.testRatio = 15/100;&#xA;&#xA;% Train the Network&#xA;&#xA;[net,tr] = train(net,inputs,targets);&#xA;&#xA;% Test the Network&#xA;outputs = net(inputs);&#xA;errors = gsubtract(targets,outputs);&#xA;performance = perform(net,targets,outputs)&#xA;&#xA;% Recalculate Training, Validation and Test Performance&#xA;trainTargets = targets .* tr.trainMask{1};&#xA;valTargets = targets .* tr.valMask{1};&#xA;testTargets = targets .* tr.testMask{1};&#xA;&#xA;trainPerformance = perform(net,trainTargets,outputs);&#xA;valPerformance = perform(net,valTargets,outputs);&#xA;testPerformance = perform(net,testTargets,outputs);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also is it right to split the data set as below for training and testing?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;trainData = inputData(:,1:213);&#xA;trainTargetData =  targetData(:,1:213);&#xA;validationData = inputData(:,214:258);&#xA;testData = inputData(:,259:end);&#xA;testTargetData = targetData(:,259:end);&#xA;validationTargetData = targetData(:,214:258);&#xA;&#xA;[net,tr] = train(net,trainData,trainTargetData);&#xA;&#xA;% Validation&#xA;outputs = net(validationData);&#xA;errors = gsubtract(validationTargetData,outputs);&#xA;performance = perform(net,validationTargetData,outputs);&#xA;&#xA;% Test the Network&#xA;outputs = net(testData);&#xA;error = gsubtract(testTargetData,outputs);&#xA;performance = perform(net,testTargetData,outputs);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4590" LastEditorUserId="75" LastEditDate="2017-01-07T17:38:56.437" LastActivityDate="2017-01-07T17:38:56.437" Title="Confusing Matlab Artificial Neural Toolbox script" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2017-03-15T18:19:46.253" ContentLicense="CC BY-SA 3.0" />
  <row Id="2603" PostTypeId="1" CreationDate="2017-01-03T15:10:34.037" Score="3" ViewCount="151" Body="&lt;p&gt;Let's say I've got a training sample set of 1 million records, which I pull batches of 100 from to train a basic regression model using  gradient descent and MSE as a loss function.  Assume test and cross validation samples have already been withheld from the training set, so we have 1 million entries to train with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider following cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Run 2 epochs (I'm guessing this one is potentially bad as it's basically 2 separate training sets)&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the first Epoch train over records 1-500K&lt;/li&gt;&#xA;&lt;li&gt;In the second epoch train over the 500K-1M&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Run 4 epochs&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the first and third Epoch train over records 1-500K&lt;/li&gt;&#xA;&lt;li&gt;In the second and fourth epoch train over the 500K-1M&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Run X epochs, but each epoch has a random 250K samples from the training set to choose from&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Should every epoch have the exact samples?  Is there any benefit/negative to doing so? My intuition is any deviation in samples changes the 'topography' of the surface you're descending, but I'm not sure if the samples are from the same population if it matters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This relates to a SO question: &lt;a href=&quot;https://stackoverflow.com/questions/39001104/in-keras-if-samples-per-epoch-is-less-than-the-end-of-the-generator-when-it&quot;&gt;https://stackoverflow.com/questions/39001104/in-keras-if-samples-per-epoch-is-less-than-the-end-of-the-generator-when-it&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4591" LastEditorUserId="-1" LastEditDate="2017-05-23T12:39:33.010" LastActivityDate="2017-02-05T16:19:51.590" Title="With gradient descent w/MSE on a regression, must/should every Epoch use the exact same training samples?" Tags="&lt;statistical-ai&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2604" PostTypeId="1" CreationDate="2017-01-03T22:03:42.403" Score="2" ViewCount="585" Body="&lt;p&gt;I want to make a Connect 4 AI using machine learning, but I'm a complete beginner to the topic. From what I've seen, an ANN is a way to go. Some phrases I've heard are &quot;neuroevolution&quot; and the acronym NEAT. I'm very confused. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you choose the number of neurons, synapses, and hidden layers of an ANN (in NEAT)?&lt;/p&gt;&#xA;" OwnerUserId="4605" LastEditorUserId="2444" LastEditDate="2019-07-07T19:31:27.763" LastActivityDate="2019-07-07T19:41:16.803" Title="How do you choose the number of neurons, synapses, and hidden layers of an ANN (in NEAT)?" Tags="&lt;neural-networks&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2606" PostTypeId="2" ParentId="2604" CreationDate="2017-01-05T02:56:53.040" Score="7" Body="&lt;p&gt;To find the number of neurons and layers that you will use is not that straightforward. The best way to do this is through experimentation however you will be able to better estimate the number of layers and neurons needed through experience. One of the common rules is that more neurons are better for more complex datasets. However, you do not want too many or you will get an overfitted model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for NEAT, that stands for &lt;em&gt;neuron evolution of augmenting topologies&lt;/em&gt;. This is a genetic algorithm that works fairly well, however, I would recommend that you use a different algorithm like Q-learning. If you wish to learn more about Q-learning, then I would definitely recommend that you check out Google's DeepMind research on training deep neural networks to play the game of Go using Q-learning.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="2444" LastEditDate="2019-07-07T19:33:56.810" LastActivityDate="2019-07-07T19:33:56.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2607" PostTypeId="5" CreationDate="2017-01-05T12:00:02.010" Score="0" Body="&lt;p&gt;Searching is a technique for problem solving commonly used in AI systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The searching may be conducted in a discrete solution space, whereby the permutations for possible discrete solutions are investigated in some way to determine a good solution based on some stated criteria for selection.  Alternately the search may be conducted in a continuous search space, whereby the goal of the search is to converge on an optimum.  The space may be a hybrid of continuous and discrete dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The acceptable reliability and accuracy of the search are elements of the search requirements.  For instance, in the case of a discrete solution space, there may be an acceptable number of incorrect solutions from among a large set of problems solved.  There may be answers that in some way are defined as close enough to correct.  A typical example is when a certain percentage of error is allowed in IEEE floating point output of the search in continuous dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem may be static in that the problem does not change during the course of the search or dynamic as in the case of control systems for robotics or other kinds of interactive systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early search AI work was targeted at gaming, using cames ranging in complexity from Tic-tac-toe and checkers to Sudoku, Chess, and Go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching algorithms can be compared for effectiveness based on any set of criteria.  Common evaluation criteria include these.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Accuracy&lt;/li&gt;&#xA;&lt;li&gt;Reliability&lt;/li&gt;&#xA;&lt;li&gt;Speed&lt;/li&gt;&#xA;&lt;li&gt;Computing resource utilization&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Search algorithms can be informed or uninformed, meaning that the search may be guided by information known about the probability distribution of success within the search space or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of fundamental tree searches taught in undergraduate computer science curricula are breadth first search, depth first search, best first search.  There are many algorithmic approaches, ways to parallelism searching to improve speed, architectures to support searching, and variants of these.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="4302" LastEditDate="2018-11-26T18:37:24.920" LastActivityDate="2018-11-26T18:37:24.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2608" PostTypeId="4" CreationDate="2017-01-05T12:00:02.010" Score="0" Body="For questions involving search algorithms and their use in artificial intelligence" OwnerUserId="1807" LastEditorUserId="1671" LastEditDate="2018-11-26T18:27:55.657" LastActivityDate="2018-11-26T18:27:55.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2609" PostTypeId="2" ParentId="2602" CreationDate="2017-01-05T22:37:32.367" Score="0" Body="&lt;p&gt;I don't see anything wrong in the generated code.&#xA;Let me explain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following block, we define that 70% of the data will be used for training the network, 15% for the validation, and 15% for the testing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;net.divideParam.trainRatio = 70/100;&#xA;net.divideParam.valRatio = 15/100;&#xA;net.divideParam.testRatio = 15/100;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then the network is trained from the inputs/targets passed as arguments. Note that here, all inputs are given at once for efficiency. The separation training/validation/testing is done inside the &lt;code&gt;net&lt;/code&gt; function from the proportions passed in the &lt;code&gt;net&lt;/code&gt; input argument (through the &lt;code&gt;net.divideParam.XRatio&lt;/code&gt; properties).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Train the Network&#xA;[net,tr] = train(net,inputs,targets);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then we test the results of the network on the data without discrimination between training/validation/testing. This performance score should be higher than the &quot;real&quot; performance score of the net as the performance over the training data should be higher than the one over the validation/testing data.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Test the Network&#xA;outputs = net(inputs);  % Compute predictions from the inputs&#xA;errors = gsubtract(targets,outputs);  % Compare the predictions to the targets (true values)&#xA;performance = perform(net,targets,outputs)  % Built-in Matlab function giving an overall performance score&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now that we know how the net performs overall, we can look at how it performs on the training, validation, and testing sets, respectively. The masks are created as an output of the &lt;code&gt;net&lt;/code&gt; function, from the proportions given in the first block of code. The output have been already all computed when the &lt;code&gt;net&lt;/code&gt; function has been called.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Recalculate Training, Validation and Test Performance&#xA;trainTargets = targets .* tr.trainMask{1}; % Apply a mask (0's and 1's to select the proper targets)&#xA;valTargets = targets .* tr.valMask{1}; % idem&#xA;testTargets = targets .* tr.testMask{1}; % idem&#xA;&#xA;trainPerformance = perform(net,trainTargets,outputs); % Compute a score for the training data&#xA;valPerformance = perform(net,valTargets,outputs); % same for the validation data&#xA;testPerformance = perform(net,testTargets,outputs); % same for the test data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think your code does the exact same thing but in more steps, with multiple calls to the &lt;code&gt;net&lt;/code&gt; function and without using its native way of working (you define the proportions of training/validation/testing by hand for instance without making use of the &lt;code&gt;net.divideParam.Xratio&lt;/code&gt; property). So it is probably not optimal from the computational point of view. &lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2017-01-05T22:37:32.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2610" PostTypeId="2" ParentId="2603" CreationDate="2017-01-05T23:31:24.343" Score="1" Body="&lt;p&gt;Your goal in regression should be to obtain the factors which result in the best fit model without over-fitting.  The more data you have in the training set, the better your regression will be.  Thus you would want to train on the most data, but you also want to have some data held out to validate that your model is not over-fit.  So this is where you should have your data split into say a 80/20 training and validation set.  And if data is scarce or you want that 20% to contribute to the model then you could do a 5 fold cross validation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the spirit of research perhaps you should try both of these routes, and report your findings.&lt;/p&gt;&#xA;" OwnerUserId="4652" LastActivityDate="2017-01-05T23:31:24.343" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2611" PostTypeId="2" ParentId="1507" CreationDate="2017-01-06T05:11:49.300" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For a system to possess intelligence, it must be capable of&#xA;  consistently producing a result matching criteria that circumscribe&#xA;  a goal region.  This capability must persist over a wide array of&#xA;  changing conditions in a complex environment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When a human is intelligent, the human will repeatedly achieve goals even when varied challenges appear along the path to the goal.  The human adapts the way a localized collection of organisms within a species adapt to environmental changes over generations, but the human mind adapts quickly by eliminating approaches (solution ideas) instead of eliminating individuals within the collection of organisms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term Artificial Intelligence was intended to mean intelligence designed or programmed into a machine by humans.  If the design of intelligence into a machine is possible, then it is likely that the mind is (as some have suggested) merely a biological machine.  Therefore, applying the same definition of Artificial Intelligence, a student taught by a text book and some lectures must also be artificial.  Education would be a set of capabilities programmed into a biological machine by humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, the modifier ARTIFICIAL is meaningless.  The presumption that humans (or machines) could act intelligently without literacy and education is pure fantasy.  Such is a denial of the complex and gradual ascent of civilized thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the minimum requirement for artificial intelligence, if one insists on the term, is the same as the minimum requirement for intelligence. For that, return to the first paragraph of this answer.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-01-09T06:15:28.727" LastActivityDate="2017-01-09T06:15:28.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2612" PostTypeId="1" AcceptedAnswerId="2613" CreationDate="2017-01-06T11:02:47.767" Score="5" ViewCount="122" Body="&lt;p&gt;I want to build a classifier which takes an aerial image and outputs a bitmap. The bitmap is supposed to be 1 at every pixel where the aerial image has water. For this process I want to use a ConvNet but I am unsure about the output layer. I identified two approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have an output layer with exactly 2 nodes which specify wether or not the center pixel of the aerial image corresponds to water or not.&lt;/li&gt;&#xA;&lt;li&gt;Have an output layer with one node for every pixel. So for a 64x64 image I would have 4096 nodes.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What approach would be preferred and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that is unclear to me is how to get the actual bitmap with only zeros and ones from the output of the ConvNet. Assuming we used a approach 2 then for each pixel our ConvNet would give us a probability between 0 and 1 that the this pixel corresponds to water. How do I decide that this probability is high enough to set the value in my bitmap to 1? Do I just define a threshold, say 0.5, and if the value exceeds that threshold I set the pixel to 1 or is there a more sophisticated approach?&lt;/p&gt;&#xA;" OwnerUserId="4661" LastActivityDate="2017-01-06T13:08:57.387" Title="Use ConvNet to predict bitmap" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2613" PostTypeId="2" ParentId="2612" CreationDate="2017-01-06T13:08:57.387" Score="3" Body="&lt;p&gt;Your first approach doesn't make any sense to me. After all, you are not just interested in the centre pixel are you? And if you have two nodes for every pixel, what are those two nodes encoding? For the probability of water you just need one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So clearly approach two. I would just use 0.5 as cutoff. Using a higher or lower cutoff only makes sense, if either false positives or false negatives are for some reason more problematic. Using some additional heuristic to adjust the given probabilities would just do what the ConvNet should already have done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If 4096 output nodes is too expensive you can always make it a bit fuzzier, for example by predicting the probability that at least one of four pixels shows water, reducing the number of nodes to 1024.  &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-01-06T13:08:57.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2614" PostTypeId="1" AcceptedAnswerId="2615" CreationDate="2017-01-06T15:24:53.680" Score="6" ViewCount="2365" Body="&lt;p&gt;I'm studying for my AI final exam, and I'm stuck in the state space representation. I understand initial and goal states, but what I don't understand is the state space and state transition function. Can someone explain what are they with examples?&lt;/p&gt;&#xA;&lt;p&gt;For example, one of the questions was this on my previous exam:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; knights on an infinite (in all directions) chessboard and &lt;code&gt;k&lt;/code&gt; selected squares of the board. Our task to move the knights to these selected squares obeying the following simple rules:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All knights move parallel, following their movement rule (L-shape jump)&lt;/li&gt;&#xA;&lt;li&gt;No knights can move to a square on which a knight stood anytime before&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Give the state space of the problem, the starting and goal states, and the state transition function!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4664" LastEditorUserId="2444" LastEditDate="2020-11-30T01:50:52.520" LastActivityDate="2020-11-30T01:50:52.520" Title="What are the state space and the state transition function in AI?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;search&gt;&lt;state-spaces&gt;&lt;transition-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2615" PostTypeId="2" ParentId="2614" CreationDate="2017-01-06T19:11:39.370" Score="7" Body="&lt;h3&gt;Initial state&lt;/h3&gt;&#xA;&lt;p&gt;&lt;em&gt;How things are at first&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In your particular example, it would be where your &lt;code&gt;k&lt;/code&gt; knights are placed on the board initially. Your problem doesn't precisely state this, so you could either place them at the bottom or at random.&lt;/p&gt;&#xA;&lt;h3&gt;Goal state&lt;/h3&gt;&#xA;&lt;p&gt;The board with the &lt;code&gt;k&lt;/code&gt; knights placed on the target squares.&lt;/p&gt;&#xA;&lt;h3&gt;State transition function&lt;/h3&gt;&#xA;&lt;p&gt;&lt;em&gt;A function that takes &lt;strong&gt;actions&lt;/strong&gt; (bound presumably by rules) and returns a new state&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;code&gt;k&lt;/code&gt; knight problem, the legal actions are moving parallel and in L shape movements, after which the knight will be in a new position and the board in a new state.&lt;/p&gt;&#xA;&lt;h3&gt;State space&lt;/h3&gt;&#xA;&lt;p&gt;&lt;em&gt;The set of &lt;strong&gt;all&lt;/strong&gt; states reachable from the initial state by any sequence of actions&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, in the case of the &lt;code&gt;k&lt;/code&gt; knight problem, your state space would start at the top with your &lt;em&gt;initial state&lt;/em&gt; followed down by each individual movement of the &lt;code&gt;k&lt;/code&gt; knights and the resulting new state. A graph where &lt;em&gt;lines are actions&lt;/em&gt; and &lt;em&gt;nodes are new states&lt;/em&gt; or a table are common representations of state space.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Reference&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;, by S. Russell and P. Norvig.&lt;/p&gt;&#xA;" OwnerUserId="3020" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-01-10T19:03:31.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2618" PostTypeId="1" CreationDate="2017-01-07T04:13:06.473" Score="2" ViewCount="521" Body="&lt;p&gt;Any good example for Bag-of-Words (BoW) model in image retrieving?&#xA;I want a simple example to understand the whole process of BoW.&lt;/p&gt;&#xA;" OwnerUserId="4684" LastActivityDate="2018-05-18T06:14:46.483" Title="Bag-of-Words (BoW) model in image detection" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2619" PostTypeId="1" CreationDate="2017-01-07T12:39:02.010" Score="5" ViewCount="823" Body="&lt;p&gt;I read through the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies (NEAT)&lt;/a&gt; paper. I understand the algorithm now, but one thing is still unclear to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When does the mutation occur and how does it take place? How is it chosen whether to add a node or to add a connection mutation? Furthermore, how is it chosen where the mutation is taking place in the network (between which connections)?&lt;/p&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="2444" LastEditorDisplayName="user4639" LastEditDate="2019-07-07T19:12:09.203" LastActivityDate="2019-07-07T19:12:09.203" Title="When do mutations in NEAT occur?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2623" PostTypeId="1" AcceptedAnswerId="2630" CreationDate="2017-01-08T08:48:44.837" Score="9" ViewCount="310" Body="&lt;p&gt;I've been struggling with the connection between knowledge-based AI systems and Bayesian inference for a while now. While I continue to sweep through the literature, I would be happy if someone can answer these more specific questions directly&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are Bayesian inference based methods used in reasoning or Q/A systems -- to arrive at conclusions about questions whose answers are not directly present in the knowledge base?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In other words, if a Q/A system doesn't find an answer in a knowledge base, can it use Bayesian inference to use the available facts to suggest answers with varying likelihoods?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If yes, could you point me to some implementations?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4707" LastEditorUserId="2444" LastEditDate="2021-01-28T00:39:39.953" LastActivityDate="2021-01-28T00:39:39.953" Title="Can Bayesian inference be combined with knowledge-based systems?" Tags="&lt;reference-request&gt;&lt;question-answering&gt;&lt;reasoning&gt;&lt;bayesian-inference&gt;&lt;knowledge-based-systems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2626" PostTypeId="1" CreationDate="2017-01-09T06:46:37.640" Score="2" ViewCount="431" Body="&lt;p&gt;I already know that AI can paint, by using genetic algorithms, there are already lots of works such as &lt;a href=&quot;http://genekogan.com/works/style-transfer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://www.instapainting.com/ai-painter&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. In addition, I also know AI can compose: &lt;a href=&quot;https://arxiv.org/pdf/1611.03477v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Song from PI: A musically plausible network for pop music generation&lt;/a&gt; (genetic algorithm too).&lt;/p&gt;&#xA;&lt;p&gt;But what I find interesting is not painting those ambiguity/abstract paintings.&lt;/p&gt;&#xA;&lt;p&gt;The not abstract painting flow I think is (just for example):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;at least training AI with superman's comic&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;give AI a very simple posture sketch of a standing human&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AI paint it to superman.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Currently, I don't know if there is any way/guide/thought/algorithm can teach AI to paint a superman like the comic (not abstract ones). I'd like to research this area but can't find where and how to start.&lt;/p&gt;&#xA;" OwnerUserId="4728" LastEditorUserId="1641" LastEditDate="2021-04-25T16:04:15.790" LastActivityDate="2021-04-25T16:04:15.790" Title="Is there any way can teach AI creative painting (not convert photo to paint)?" Tags="&lt;machine-learning&gt;&lt;research&gt;&lt;algorithm&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2630" PostTypeId="2" ParentId="2623" CreationDate="2017-01-09T18:46:36.490" Score="7" Body="&lt;p&gt;Yes, it is possible to combine probabilistic / bayesian reasoning and a traditional &quot;knowledgebase&quot;.  And some work along those lines has been done.  See, for example, &lt;a href=&quot;https://dtai.cs.kuleuven.be/problog/&quot;&gt;ProbLog&lt;/a&gt; (&quot;Probabilistic Prolog&quot;) which combines logic programming and probabilistic elements.  See:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dtai.cs.kuleuven.be/problog/tutorial/mpe/01_bn.html&quot;&gt;https://dtai.cs.kuleuven.be/problog/tutorial/mpe/01_bn.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another project to look at is &lt;a href=&quot;http://www.pr-owl.org/&quot;&gt;Pr-OWL&lt;/a&gt;&#xA; (&quot;Probabilistic OWL&quot;) which adds Bayesian reasoning to the Semantic Web stack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course neither of these deals &lt;em&gt;specifically&lt;/em&gt; with QA systems, but both represent some work on at least the foundational aspect of combining traditional logic and/or ontologies, with probabilistic approaches.  Building a QA system on top of that is an exercise for the reader...&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-01-09T18:46:36.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2631" PostTypeId="2" ParentId="2626" CreationDate="2017-01-10T00:14:28.023" Score="1" Body="&lt;p&gt;This paper (that was featured in &lt;a href=&quot;https://ai.stackexchange.com/q/2599/2329&quot;&gt;another question&lt;/a&gt;), &lt;a href=&quot;https://arxiv.org/abs/1612.03242&quot; rel=&quot;nofollow noreferrer&quot;&gt;StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks&lt;/a&gt; describes some techniques similar to what you described in the question.  Instead of comics, the system in the paper is trained on a database of nature photos (birds or flowers) combined with text descriptions of each photo.  It will then draw an &quot;imaginary&quot; bird or flower based purely on a text description.  It does this in two steps: it the first, it picks a posture for the depicted subject (I believe randomly...) and roughs out patches of color that match the described features of the subject in the selected posture.  It then refines the sketch, again using the text description for parameters, until it appears &quot;photo-realistic&quot; in the style of the training photos.  It does not just recreate its training images; instead, it is able to recreate features of its training images with different positions/colors.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2017-01-10T00:14:28.023" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2632" PostTypeId="1" AcceptedAnswerId="2633" CreationDate="2017-01-10T08:38:16.777" Score="11" ViewCount="675" Body="&lt;p&gt;Having worked with neural networks for about half a year, I have experienced first-hand what are often claimed as their main disadvantages, i.e. overfitting and getting stuck in local minima. However, through hyperparameter optimization and some newly invented approaches, these have been overcome for my scenarios. From my own experiments:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Dropout seems to be a very good regularization method.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Batch normalization eases training and keeps signal strength consistent across many layers.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adadelta consistently reaches very good optima&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have experimented with &lt;code&gt;scikit-learn&lt;/code&gt;'s implementation of SVM alongside my experiments with neural networks, but I find the performance to be very poor in comparison, even after having done grid-searches for hyperparameters. I realize that there are countless other methods, and that SVM's can be considered a sub-class of NN's, but still.&lt;/p&gt;&#xA;&lt;p&gt;So, to my question:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;With all the newer methods researched for neural networks, have they slowly - or will they - become &amp;quot;superior&amp;quot; to other methods? Neural networks have their disadvantages, as do others, but with all the new methods, have these disadvantages been mitigated to a state of insignificance?&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I realize that oftentimes &amp;quot;less is more&amp;quot; in terms of model complexity, but that too can be architected for neural networks. The idea of &amp;quot;no free lunch&amp;quot; forbids us to assume that one approach always will reign superior. It's just that my own experiments - along with countless papers on awesome performances from various NN's - indicate that there might be, at the least, a very cheap lunch.&lt;/p&gt;&#xA;" OwnerUserId="4747" LastEditorUserId="2444" LastEditDate="2021-01-18T12:02:31.063" LastActivityDate="2021-01-18T12:02:31.063" Title="Are the shortcomings of neural networks diminishing?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;overfitting&gt;&lt;support-vector-machine&gt;&lt;no-free-lunch-theorems&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2633" PostTypeId="2" ParentId="2632" CreationDate="2017-01-10T09:34:54.987" Score="7" Body="&lt;p&gt;Neural Networks have other short comings as well. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are resource constrained in anyway, you probably should not look at Neural Networks first. Evaluation of a trained deep NN can be much more expensive than competing techniques too.&lt;/li&gt;&#xA;&lt;li&gt;The effort involved in learning how to architect and train a NN is still much higher than competing methods, like an SVM. People who are just starting out in Data Science should probably use other techniques to learn about the nuances of fitting data before getting involved in neural networks. And although simple NNs with only one or two hyperparameters are often available in &#xA;many data science libraries, they don't perform any better than other techniques so are just another ML black-box technique really.&lt;/li&gt;&#xA;&lt;li&gt;While we have made a lot of progress in understanding how neural networks do their magic, they are still less accessible and dissectible than most competing methods. So while NNs might solve the problem, they might not give you as many insights as easily as other techniques do.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Looking forward to what other people have to say here.&lt;/p&gt;&#xA;" OwnerUserId="1846" LastEditorUserId="8" LastEditDate="2017-01-10T11:08:27.247" LastActivityDate="2017-01-10T11:08:27.247" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2634" PostTypeId="1" CreationDate="2017-01-10T13:15:43.350" Score="6" ViewCount="536" Body="&lt;p&gt;What exactly are the differences between &lt;em&gt;semantic&lt;/em&gt; and &lt;em&gt;lexical-semantic&lt;/em&gt; networks? &lt;/p&gt;&#xA;" OwnerUserId="4754" LastEditorUserId="1581" LastEditDate="2018-08-23T18:47:23.843" LastActivityDate="2018-08-24T15:39:44.127" Title="What exactly are the differences between semantic and lexical-semantic networks?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;semantics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2635" PostTypeId="2" ParentId="2618" CreationDate="2017-01-10T22:21:49.313" Score="2" Body="&lt;p&gt;Here is an illustration of the entire process without any equation so you can get the big picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Features are extracted from the image. Let's take the example of very common features like SIFT. For many key points (or even each pixel) of the image, a 128-dimensional SIFT feature is computed. If processing numerous images the number of features becomes very big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A way of having a more compact representation of the set of images is to use the Bag-of-Words (or Bag-of-Visual-Words) technique. The idea is to find k words (i.e., k SIFT features) from which every single image will be represented. We call this set of k words, the dictionary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, each SIFT feature will be assigned to the nearest word (i.e., nearest SIFT feature with respect to the Euclidean distance for instance) of the dictionary. You can see this as a dictionary in which the words &quot;go&quot;, &quot;going&quot;, and &quot;gone&quot; would all be represented by the word &quot;go&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, each image is represented by only k values (counts for the number of words/features assigned to each word of the dictionary). This is an histogram, and you can normalize it to get a single vector of proportions representing the image.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2017-01-10T22:21:49.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2637" PostTypeId="1" AcceptedAnswerId="6993" CreationDate="2017-01-11T09:44:04.700" Score="6" ViewCount="999" Body="&lt;p&gt;In Monte Carlo tree search, what should we do when the selection step selects a terminal state (i.e. a won or lost state), which is, by definition, a leaf node? Expansion and simulation is not in order, as it's game over, but does the tree (score/visits) need to be updated (backpropagation)? Won't this particular node be selected continuously?&lt;/p&gt;&#xA;" OwnerUserId="4773" LastEditorUserId="2444" LastEditDate="2019-11-18T21:46:50.700" LastActivityDate="2019-11-18T21:47:19.723" Title="What should we do when the selection step selects a terminal state?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2638" PostTypeId="1" AcceptedAnswerId="2641" CreationDate="2017-01-11T13:02:37.290" Score="4" ViewCount="93" Body="&lt;p&gt;I was wondering if in any way it is possible to generate W questions based on gap-fill-in type questions (e.g &quot;______ is a process in which plants generate energy.&quot;   ---&gt;   &quot;What is the process in which plants generate energy called?&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how can I achieve this? I am familiar with working with natural language processing and have no problem with implementing an algorithm for this but I do not know where to start with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="4034" LastEditorUserId="4302" LastEditDate="2018-10-08T12:33:37.737" LastActivityDate="2018-10-08T12:33:37.737" Title="How can I generate What, why, who types of questions from &quot;Gap-fill-in&quot; type of questions?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2639" PostTypeId="1" CreationDate="2017-01-11T14:19:57.573" Score="5" ViewCount="539" Body="&lt;p&gt;I have read the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies (NEAT)&lt;/a&gt; paper, but some doubts are still bugging me, so I have two questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When do mutations occur? Between which nodes?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When mating, what happens if 2 genes have the same connection but a different innovation number? As far as I know, mutations occur randomly and thus it is possible that 2 genomes have the same mutation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="2444" LastEditDate="2019-07-07T19:12:37.313" LastActivityDate="2019-07-07T19:12:37.313" Title="What happens if 2 genes have the same connection but a different innovation number?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2640" PostTypeId="2" ParentId="2619" CreationDate="2017-01-11T16:11:01.960" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When does the mutation occur and how does it take place? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Finding a solution in NEAT algorithm is based on evolution strategy. It means that you have Neural Networks which are yours individuals, so mutations and crossing occurs in loop after phase of &quot;fitnessing&quot; (calculation fitness for every individual and removing bad ones).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How is it chosen whether to add a node or to add a connection mutation? Furthermore, how is it chosen where the mutation is taking place in the network (between which connections)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Randomly: just draw. You can read more about evolutionary algorithms &lt;a href=&quot;https://ai.stackexchange.com/questions/240/what-exactly-are-genetic-algorithms-and-what-sort-of-problems-are-they-good-for&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it could somehow help, I include &lt;a href=&quot;https://github.com/robrtj/NeuralNetworkImageCompression&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to my repository with the implementation of NEAT.&lt;/p&gt;&#xA;" OwnerUserId="4779" LastEditorUserId="2444" LastEditDate="2019-07-07T18:59:37.013" LastActivityDate="2019-07-07T18:59:37.013" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2641" PostTypeId="2" ParentId="2638" CreationDate="2017-01-11T17:16:30.460" Score="3" Body="&lt;p&gt;This seems tricky. It seems that any &quot;surface level&quot; transformation wouldn't give adequate results and any working solution would need to properly capture the sentence structure, and generate a gramatically correct transformed sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible option is to use a &quot;traditional pipeline&quot; - e.g. you run a NLP pipeline up to syntactic parsing, which for general domain english is quite accurate (you'd need some special handling for the gap &quot;____&quot; part though), then implement some heuristic rules to transform the syntax tree, and regenerate a sentence from the transformed tree. There are a lot of publications about similar transformations in machine translation domain, used as a way to preprocess data before running statistical machine translation for language pairs with very different word(or sentence part) ordering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second option that may work is to look into the field of controlled natural languages, or something like &lt;a href=&quot;http://www.grammaticalframework.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.grammaticalframework.org/&lt;/a&gt; that can be used as a toolkit to help generating new sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current fashion also suggests a very different option that &lt;em&gt;might&lt;/em&gt; work - you &lt;em&gt;could&lt;/em&gt; train a character-level recurrent neural network with an attention mechanism (look into recent neural machine translation publications for details) to do this transformation, but I'm not sure of how much training data it will need for decent accuracy.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastEditorUserId="1675" LastEditDate="2017-01-12T07:23:27.530" LastActivityDate="2017-01-12T07:23:27.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2642" PostTypeId="1" AcceptedAnswerId="2643" CreationDate="2017-01-12T15:31:01.383" Score="4" ViewCount="317" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Any sufficiently advanced algorithm is indistinguishable from AI.---&lt;a href=&quot;https://twitter.com/othermichael?lang=en&quot; rel=&quot;noreferrer&quot;&gt;Michael Paulukonis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://ai.stackexchange.com/questions/1507/what-are-the-minimum-requirements-to-call-something-ai&quot;&gt;What are the minimum requirements to call something AI?&lt;/a&gt;, there are certain requirements that a program must meet to be called AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, according to that same question, the term AI has became a buzzword that tends to be associated with new technologies, and that certain algorithms may be classified in AI in one era and then dismissed as boring in another era once we understand how the technology works and be able to properly utilize it (example: voice recognition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans are able to build complex algorithms that can engage in behaviors that are not easy to predict (due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Emergence&quot; rel=&quot;noreferrer&quot;&gt;emergent complexity&lt;/a&gt;). These &quot;sufficiently advanced&quot; algorithms  could be mistaken for AI, partly because humans can also engage in behaviors that are not easy to predict. And since AI is a buzzword, humans may be tempted to engage in this self-delusion, in the hopes of taking advantage of the current AI hype.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually, as humanity's understanding of their own &quot;sufficiently advanced algorithms&quot; increase, the temptation to call their algorithms AI diminishes. But this temporary period of mislabeling can still cause damage (in terms of resource misallocation and hype).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can be done to &lt;em&gt;distinguish&lt;/em&gt; a sufficiently advanced algorithm from AI? Is it even possible to do so? Is a sufficiently advanced algorithm, by its very nature, AI?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2018-11-26T03:37:44.473" Title="How can one distinguish between an AI and a &quot;sufficiently advanced algorithm&quot;?" Tags="&lt;definitions&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2643" PostTypeId="2" ParentId="2642" CreationDate="2017-01-12T20:24:04.923" Score="3" Body="&lt;p&gt;As you correctly pointed out, people tend to misinterpret the expression &lt;em&gt;AI&lt;/em&gt; since they do not know what's behind an &lt;em&gt;AI&lt;/em&gt;; it is pretty clear that in &lt;em&gt;AI&lt;/em&gt; there is no more than just a bunch of algorithms and flowing bits. Talking about the nature of an &lt;em&gt;AI&lt;/em&gt; without talking about the algorithmic paradigm of that &lt;em&gt;AI&lt;/em&gt; is pointless and antiscientific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This point of view is quite cynical; what we call intelligence is just the capability of solving particular problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quote you cited in the title it's a derived version of the quote below, by a science fiction writer.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any sufficiently advanced technology is indistinguishable from magic. - &lt;a href=&quot;https://en.wikipedia.org/wiki/Arthur_C._Clarke&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arthur C. Clarke&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hence I doubt that a scientific answer exists since science lacks a formal definition of &lt;em&gt;sufficiently advanced algorithm&lt;/em&gt; and &lt;em&gt;advanced algorithm&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4801" LastActivityDate="2017-01-12T20:24:04.923" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2644" PostTypeId="1" AcceptedAnswerId="2657" CreationDate="2017-01-13T01:15:49.743" Score="5" ViewCount="360" Body="&lt;p&gt;As I see some cases of machine-learning based artificial intelligence, I often see they make critical mistakes when they face inexperienced situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our case, when we encounter totally new problems, we acknowledge ourselves that we are not skilled enough to do the task and hand it to someone who is capable of doing the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would AI be able to self-examine objectively and determine if it is capable of doing the task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how would it be accomplished?&lt;/p&gt;&#xA;" OwnerUserId="4802" LastActivityDate="2017-01-28T14:07:19.367" Title="How would AI be able to self-examine?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2645" PostTypeId="1" CreationDate="2017-01-13T04:56:26.740" Score="3" ViewCount="516" Body="&lt;p&gt;A lot of people are claiming that we are at an inflection point, and machine learning/artificial intelligence will take off. This is in spite of the fact that for a long machine learning has stagnated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the signals that indicate that machine learning is going to take off?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, how do you know that we are at an inflection point for a certain technology?&lt;/p&gt;&#xA;" OwnerUserId="4807" LastEditorUserId="2444" LastEditDate="2020-04-01T15:38:23.810" LastActivityDate="2020-04-01T15:40:39.463" Title="How do we know that now artificial intelligence will really take off?" Tags="&lt;machine-learning&gt;&lt;agi&gt;&lt;history&gt;&lt;ai-winter&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2646" PostTypeId="1" CreationDate="2017-01-13T08:06:53.853" Score="9" ViewCount="1094" Body="&lt;p&gt;If a conscious AI is possible, would it also be possible for someone who knows what they are doing to torture (or hurt) the AI? Could this be avoided? How?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question deals with computer based AI, not robots, which are as conscious as people (this is an assumption of the question). The question wonders how a crime as hard to trace as illegal downloads, but far worse ethically, could be prevented. Note that despite most people being nice and empathising with the robots, there are always the bad people, and so relying on general conscience will not work.&lt;/p&gt;&#xA;" OwnerUserId="4809" LastEditorUserId="2444" LastEditDate="2019-06-15T18:37:02.237" LastActivityDate="2019-06-15T18:37:02.237" Title="Could humans hurt a conscious or sentient AI?" Tags="&lt;philosophy&gt;&lt;ethics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2647" PostTypeId="2" ParentId="2644" CreationDate="2017-01-13T09:26:50.163" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Would AI be able to self-examine objectively and determine if it is capable of doing the task?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Our ability to self-examine comes definitively from the memory of our experiences; indeed, for this reason it can't be objective. In the same way AI could be able to determine the heuristically optimal strategy to solve a problem if and only if it has some sort of memory of previous tasks e.g. speech recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Science is constantly working to improve our understanding of things. Trying to mimic the human brain seems to be a difficult problem at the moment; though we are able to replicate almost fully simpler organisms as &lt;a href=&quot;http://www.openworm.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;C. elegans&lt;/a&gt;, a roundworm.&lt;/p&gt;&#xA;" OwnerUserId="4801" LastActivityDate="2017-01-13T09:26:50.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2648" PostTypeId="2" ParentId="2645" CreationDate="2017-01-13T11:04:17.887" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the signals that indicate that machine learning is going to take off?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We simply don't know until the consequences of the inflection point determine a remarkable difference between the before and after. In general terms every considerable reaction must be attributed to a particular cause.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the biggest limit that bounds artificial intelligence, and apparently makes it stagnating, is the greed of computational power involved in this field; and since the hardware technology improve much slower than the software does, AI remains confined in labs and data centers.&lt;/p&gt;&#xA;" OwnerUserId="4801" LastActivityDate="2017-01-13T11:04:17.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2649" PostTypeId="1" CreationDate="2017-01-13T15:01:50.487" Score="3" ViewCount="209" Body="&lt;p&gt;Has anyone used YodaQA for natural language processing? How easy is it to link to a document database other than Wikipedia?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're thinking we can create a bot to use AI to analyze our developer and user documentation and provide a written or spoken answer in reply. YodaQA comes linked to Wikipedia for starters, but we'd need to link to our own source info. I'm trying to get an idea of the development time required to set up the AI and then to link to the database.&lt;/p&gt;&#xA;" OwnerUserId="4627" LastEditorUserId="2444" LastEditDate="2020-03-04T22:32:44.053" LastActivityDate="2020-03-04T22:32:44.053" Title="Has anyone used YodaQA for natural language processing?" Tags="&lt;ai-design&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2650" PostTypeId="2" ParentId="2639" CreationDate="2017-01-13T15:28:25.397" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When do mutations occur and between which nodes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are two types of mutations in the &lt;em&gt;NEAT&lt;/em&gt; model, each of them appears randomly during one epoch on different individuals; the number of structures affected by mutations may vary depending upon the nature of the problem.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A new gene/node is added to the structure and properly linked.&lt;/li&gt;&#xA;&lt;li&gt;A new connection between two nodes is added.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When mating, what happens if 2 genes have the same connection but a different innovation number?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;During a single epoch/generation every mutation is tracked and if the same mutation appears it can't have the same global &lt;em&gt;innovation&lt;/em&gt; number. &lt;em&gt;Same mutation, same innovation number.&lt;/em&gt; In that way, during the mating phase, there is no decisional problem which leads to prefer one structure to the other.&lt;/p&gt;&#xA;" OwnerUserId="4801" LastEditorUserId="2444" LastEditDate="2019-07-07T18:57:44.867" LastActivityDate="2019-07-07T18:57:44.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2651" PostTypeId="5" CreationDate="2017-01-13T15:57:10.007" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-01-13T15:57:10.007" LastActivityDate="2017-01-13T15:57:10.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2652" PostTypeId="4" CreationDate="2017-01-13T15:57:10.007" Score="0" Body="For questions about evolutionary algorithms (EAs), which use mechanisms inspired by biological evolution, such as mutation, recombination, and/or selection. EAs comprise genetic algorithms (GAs), genetic programming (GP), evolution strategies (ES), neuroevolution (NE), and so on. EAs are a sub-field of evolutionary computation, which comprises also ant colony optimization or particle swarm optimization, among others." OwnerUserId="4801" LastEditorUserId="2444" LastEditDate="2020-12-08T18:32:15.997" LastActivityDate="2020-12-08T18:32:15.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2653" PostTypeId="2" ParentId="2646" CreationDate="2017-01-13T16:11:07.440" Score="9" Body="&lt;p&gt;The article &lt;a href=&quot;http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/children-beating-up-robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Children Beating Up Robot Inspires New Escape Maneuver System&lt;/a&gt; is based on two research papers about an experiment in a Japanese mall that led to unsupervised children attacking robots. The research paper you're interested in is &lt;a href=&quot;http://www.irc.atr.jp/~drazen/pdf/HRI2015_Brscic.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Escaping from Children’s Abuse of Social Robots&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that research paper, researchers were able to program the robots to follow a planning simulation to reduce the probability of abuse by children. If it detects children, the robot is programmed to retreat into a crowd of adults (who can then discipline the children if needed). This happened because the researchers saw that it was only children who were beating up the robots in the mall in question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They discuss trying out other options though:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this work the robot’s strategy to prevent abuse was to “escape”, i.e. move to a location where it is less likely abuse will occur. One could ask why the robot cannot overcome the abuse. In our preliminary trials we have tried several approaches, but we found that it is very difficult for the robot to persuade children not to abuse it. For example, we changed the robot’s wordings in many ways, using strong words, emotional or polite expressions, but none of them were successful. One partially successful strategy was the robot ‘physically’ pushing children. When its way was blocked, it would just try to keep going and behave as if it will collide into children and force its way through (under careful monitoring from a human operator). We observed that children at first accepted the robot’s requests and obeyed them; but, very soon they learned that they are stronger than the robot so they can win if they push, and also that they can stop it by pushing the bumper switch (attached on the robot for safety). After realizing that, they just continued with the abusive behavior. Obviously having a stronger robot would present a problem for safety and social acceptance so dealing with such abusive situations remains difficult.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;But let's interrogate your question further:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If conscious AI is possible and is wide spread, wouldn't it be easy for someone who knows what they are doing to torture AI? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why would you consider such torture to be &lt;em&gt;wrong&lt;/em&gt;? After all, one could argue that the machine won't really 'experience' pain if you torture it...so it should be morally okay to torture the machine then. It may be respond as &lt;em&gt;if&lt;/em&gt; it is in pain, but it's dubious whether the ability to simulate an emotional state such as &quot;being in pain&quot; is equivalent to actually &lt;em&gt;being&lt;/em&gt; in that emotional state. See the question &lt;a href=&quot;https://philosophy.stackexchange.com/questions/34779/is-the-simulation-of-emotional-states-equivalent-to-actually-experiencing-emotio/35815&quot;&gt;Is the simulation of emotional states equivalent to actually experiencing emotion?&lt;/a&gt; for more discussion on this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make such an argument, but it won't really work on an emotional level because most humans would feel &lt;em&gt;empathy&lt;/em&gt; towards the machine. It may be hard to justify it logically (and it may be based on humans' tendencies to engage in &lt;a href=&quot;https://en.wikipedia.org/wiki/Anthropomorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;anthropomorphism&lt;/a&gt;), but we feel this empathy. It's this empathy that caused you to ask this question in the first place, caused researchers to figure out how to protect a robot from being beaten up, enabled police officers to &lt;a href=&quot;http://www.japantimes.co.jp/news/2015/09/07/national/crime-legal/drunken-kanagawa-man-60-arrested-after-kicking-softbank-robot-in-fit-of-rage/#.WHj4Rccc9Lp&quot; rel=&quot;nofollow noreferrer&quot;&gt;arrest a drunken Japanese man for beating up a SoftBank robot&lt;/a&gt;, and made many humans upset over the destruction of &lt;a href=&quot;https://en.wikipedia.org/wiki/HitchBOT&quot; rel=&quot;nofollow noreferrer&quot;&gt;hitchBOT&lt;/a&gt;. And &lt;em&gt;that's&lt;/em&gt; how AI abuse would be avoided - human empathy. If most humans care about the welfare of machines, they'll make it a priority to stop those few humans who are able and willing to abuse the machines.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: The OP has edited his &lt;em&gt;question&lt;/em&gt; to clarify that he is talking about &lt;em&gt;software&lt;/em&gt;, and not about robots. For robots, you can rely on anthropomorphism to produce some level of sympathy, but it's hard to sympathize with raw lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're not going to stop abuse of algorithms. Put it frankly, since the algorithms aren't like us, we aren't going to extend the same sort of empathy that we would to robots. Even chatbots are kinda iffy. If you could get people to sympathize with lines of code though (possibly by making a convincing simulation of emotion and sapience), then the above answer applies - humans anthropomorphize the machine and will come up with countermeasures. We aren't that level yet, so &quot;stopping AI abuse&quot; will be a low priority.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, some failsafes could be programmed in to limit the damage of abuse, as detailed in &lt;a href=&quot;https://www.chatbots.org/ai_zone/viewthread/1488/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this thread on chatbot abuse&lt;/a&gt; - making the bot respond in a boring manner to make the abuser feel bored and move onto the next target, responding back to the abuser in a &quot;battle of wits&quot;, or even just blocking the abusers from using the service.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These failsafes are cold comfort to those that want to &lt;em&gt;prevent&lt;/em&gt; abuse, not respond to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also...an abuser can happily learn how to program an AI to then abuse to his/her heart's content. Nothing can be done to stop that, and any possible measures to stop said abuse (such as monitoring every human being to make sure they don't program an AI to abuse) will probably cause more damage than it'd solve.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2017-04-13T12:42:22.960" LastActivityDate="2017-01-31T00:59:16.970" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2654" PostTypeId="2" ParentId="2645" CreationDate="2017-01-14T07:27:21.387" Score="2" Body="&lt;p&gt;This has happened in the past where people were really excited and saying things like we will have AI in decade or so. This is happening again. Not sure why people don't learn from history of AI. In both the cases what's happening is this - You develop a technique to solve a particular problem, you apply that technique, the technique seems to be general enough, people start to apply that same technique to various problems, people get excited that this is the silver bullet they were looking for, the technique starts to show its limitations and doesn't work for many problems, hype gets shattered, start over.&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2017-01-14T07:27:21.387" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2655" PostTypeId="1" AcceptedAnswerId="2678" CreationDate="2017-01-14T15:18:02.407" Score="3" ViewCount="757" Body="&lt;p&gt;I'm looking for good examples of successful AI projects and theories that had a relatively good impact on society, economics and military field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So many years have passed after the first AI researches; hence I'm wondering if it has really increased the quality of our lives.&lt;/p&gt;&#xA;" OwnerUserId="4801" LastEditorDisplayName="user1440" LastEditDate="2018-12-28T21:43:17.777" LastActivityDate="2018-12-28T21:43:17.777" Title="Which is the most successful AI project so far?" Tags="&lt;social&gt;" AnswerCount="3" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2656" PostTypeId="2" ParentId="2655" CreationDate="2017-01-14T15:39:30.897" Score="-1" Body="&lt;p&gt;I believe which most successful AI theory is machine learning, in entire web have machine learning algorithms running, learning by what you do, watch, search, even by the photos you take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;succesful AI projects:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TensorFlow;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;scikit-learn;&lt;/p&gt;&#xA;" OwnerDisplayName="user4822" LastEditorDisplayName="user4822" LastEditDate="2017-01-14T15:52:44.843" LastActivityDate="2017-01-14T15:52:44.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2657" PostTypeId="2" ParentId="2644" CreationDate="2017-01-14T15:42:42.013" Score="3" Body="&lt;p&gt;Several AI systems will come up with a level of confidence to the solution found. For example, neural networks can indicate how relatable is the input problem to the ones it was trained with. Similarly, genetic algorithms work with evaluation functions, that are used to select best results, but depending on how they're built (the functions), they can indicate how close the algorithms are to an optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the limit to when this is acceptable or not will depend on a threshold set beforehand. Is 50% confidence good enough? Maybe it's ok for OCR apps (spoiler: it's not), but is it for medical applications?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, AI systems do currently have the capacity of determining if they're performing well or not, but how acceptable that is is currently based on the domain of the problem, which currently stands outside of what is built into an AI.&lt;/p&gt;&#xA;" OwnerUserId="190" LastEditorUserId="190" LastEditDate="2017-01-15T03:47:28.187" LastActivityDate="2017-01-15T03:47:28.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2658" PostTypeId="1" AcceptedAnswerId="2660" CreationDate="2017-01-14T19:58:36.810" Score="5" ViewCount="1023" Body="&lt;p&gt;If I am correct, the branching factor is the maximum number of successors of any node.&lt;/p&gt;&#xA;&lt;p&gt;When I am applying bidirectional search to a transition graph like this one below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZmUoK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZmUoK.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If 11 is the goal state and I start going backwards, is 10 considered as a successor of 5? Even if it does not leads me further to my start state 1?&lt;/p&gt;&#xA;" OwnerUserId="4824" LastEditorUserId="2444" LastEditDate="2020-11-08T12:23:18.010" LastActivityDate="2020-11-08T12:23:18.010" Title="Is the 'direction' considered, when determining the branching factor in bidirectional search?" Tags="&lt;search&gt;&lt;branching-factors&gt;&lt;bidirectional-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2659" PostTypeId="2" ParentId="2644" CreationDate="2017-01-14T21:56:50.183" Score="1" Body="&lt;p&gt;I would concur with the answer given to you by &lt;a href=&quot;https://ai.stackexchange.com/users/4801&quot;&gt;Lovecraft&lt;/a&gt;. One of the major problems with A.I. programmers is that they are always trying to push computers to do things which are designed for &quot;mature&quot; intelligent creatures who have prior experience and knowledge of solving problems.  -As if these things can be imputed without the A.I. having to achieve the necessary and vital &quot;learn by trial and error&quot; experience first. For example: when allowing for task examination; self evaluation and risk assessment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have answered your own question, because these things can only be gained by &quot;experience&quot;. However, the only way to surmount this is to expose a prototype A.I. to the main problems; help it to solve them, and then to take its memory and use it as a template for other A.I's.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically, AI's which have learned to solve prior problems could make their memories available to others on demand, so that an inexperienced AI could solve an issue without having achieved the skills needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would like to add that mimicking intelligence is not in itself &quot;intelligence&quot;. Many programmers fall into the trap of believing that to emulate something is qualitatively the same expression as the genuine article. This is a fallacy which infers that we only have to simulate intelligence without understanding the real mechanisms which create it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;copying&quot; of sentience is done all the time and despite how good we have become in copying over the last few years, each new algorithm is just that:  a simulation without genuine sentience or intelligence!&lt;/p&gt;&#xA;" OwnerUserId="4828" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2017-01-20T12:21:59.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2660" PostTypeId="2" ParentId="2658" CreationDate="2017-01-15T03:26:07.690" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If I am correct, the branching factor is the maximum number of successors of any node&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are correct, they should also be the immediate ones:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pdkaG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pdkaG.jpg&quot; alt=&quot;branching factor&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If 11 is the goal state and I start going backwards, is 10 considered as successor of 5? Even if it do not leads me further to my start state 1?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;No, there is also a bit of misunderstanding of bidirectional search:&lt;/strong&gt; In bidirectional search you run &lt;strong&gt;2&lt;/strong&gt; simultaneous searches, one forward from the initial state, and another one backwards from the goal(hoping they meet in the middle and save you steps), if actions are reversible ( going from node to node), the successor nodes become predecessors in one search and vice versa, and your goal becomes your initial state, in your case:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hdwhY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hdwhY.jpg&quot; alt=&quot;bidirectional search&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Reference/source&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;, by S. Russell and P. Norvig.&lt;/p&gt;&#xA;" OwnerUserId="3020" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-01-16T03:59:54.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2661" PostTypeId="2" ParentId="2642" CreationDate="2017-01-15T09:20:11.760" Score="3" Body="&lt;h2&gt;Intelligence is a quality of behavior, not implementation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Intelligence is a term that primarily applies to behaviors - people, animals or artificial systems can be called intelligent iff they exhibit intelligent behavior or decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are many definitions of intelligence - here's &lt;a href=&quot;https://arxiv.org/abs/0706.3639&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper that studies 70 of them&lt;/a&gt; - it can be summarized to something like &quot;Intelligence measures an agent’s ability to achieve goals in a wide range of environments.&quot; (S. Legg and M. Hutter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps this definition is the answer to your implied question - while many algorithms can be very effective (often literally superhuman) in their own narrow domain, as of now they are very restricted in the range of environments where they exhibit this effectiveness. This means that they are not fully intelligent, they don't meet the definition/requirements of intelligence, and this also matches our intuitive expectations - we don't call AlphaGo superintelligent, because while it can beat humans in Go, the same humans can beat the same system on pretty much every other task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a &lt;em&gt;truly sufficiently&lt;/em&gt; advanced algorithm that &lt;em&gt;can&lt;/em&gt; be effective at all or most varied tasks (e.g. a &lt;em&gt;general&lt;/em&gt; artificial intelligence) can be reasonably called intelligent in the full meaning of the word.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastEditorUserId="1675" LastEditDate="2018-11-23T03:01:45.603" LastActivityDate="2018-11-23T03:01:45.603" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2662" PostTypeId="2" ParentId="2655" CreationDate="2017-01-15T11:15:32.600" Score="3" Body="&lt;p&gt;There are lots of great projects in AI.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Self-driving cars: This types of cars use AI to learn the pattern of the roads, speed of car, motion of car, braking power and lots of different features and after sufficient learning, they are capable of driving the car autonomously. The best example of this type of cars is &lt;a href=&quot;https://www.tesla.com/autopilot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla's self driving car&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Games: Games also use AI to learn the game with the aim of winning the game when played against a human or an AI player. You must have played lots of games on mobile and PC like Chess, Tic-Tac-Toe, etc. You play against the computer and according to the difficulty value set, the computer plays its moves. This difficulty value is nothing but the ability of the AI engine to predict the next moves by the opponent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Chatbots: There have been lots of development and improvements in Chatbots, so that humans can communicate with them as if they are talking to other human. There are many chatbots designed which answer any question asked by us (of course it is dependent on how much intelligence the bot holds). Some examples are &lt;a href=&quot;http://alice.pandorabots.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ALICE&lt;/a&gt; bot, &lt;a href=&quot;https://www.ibm.com/watson/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM Watson&lt;/a&gt; (which has been the most advanced bot till now).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Expert Systems: Expert systems are those systems which focus on one specific domain and can solve any query related to that domain which is given to it. For example, an expert system can be designed to solve any mathematical equation queried to it. An expert system, in such case, will give the solution of the equation along with the steps (providing steps is important because it is an important component in expert system which is called inference engine).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Prediction systems: There are lots of prediction systems which use AI and Machine Learning to predict something based on some past data. Examples are Weather Forecast system, Stock Market prediction system, Recommendation system (usually available in e-commerce websites like Amazon), etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1807" LastActivityDate="2017-01-15T11:15:32.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2663" PostTypeId="1" AcceptedAnswerId="2667" CreationDate="2017-01-15T14:03:44.797" Score="3" ViewCount="310" Body="&lt;p&gt;While writing a paper yesterday this strange thing happened to me. I was wrtiting it in Word, and wasn't satisfied with the repeated usage of word &quot;relesase&quot; in last few senteces. So I've decided to open up Google and started to enter the search phrase &quot;synonyms for release&quot;. Haven't even finished the word synonym, google autocompleted my search to &quot;synonyms for release&quot;. How could it knew that I wanted to look for that exact word? Was it just a coincidence, do Google has access to some information that could somehow possibly give away what I intended to search? What could have been the reason for it selecting &quot;release&quot; as it's first autocomplete?&lt;/p&gt;&#xA;" OwnerUserId="4834" LastActivityDate="2017-01-15T20:51:48.947" Title="How does a google choose it's autocomplete solution" Tags="&lt;search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2664" PostTypeId="2" ParentId="2642" CreationDate="2017-01-15T15:20:10.820" Score="0" Body="&lt;p&gt;I am going to answer this questions by stepping away from the previously made insightful comments and academic answers. I am going to offer my opinion only. The problem is as I see it, a bit more complex than the previous answers. For example, why is it that the only measure of intelligence of an AI is when it can &quot;beat&quot; a man at a specific task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is not a dog, a bird or a dragonfly sufficiently intelligent and sentient? Yet these creatures as well as many others too, fail to achieve the intellectual challenges we want computers to make.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mistakes we do often make is by trying to &quot;impute&quot; attributes and characteristics into an AI without it having to &quot;work&quot; for them. Those skills, experiences, memory and knowledge which all sentient beings have to constantly work at, refine and perfect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct though in your assertion that a lot of what is called AI are just academic or software gimmicks and simulations without offering the real qualities of either sentience or artificial intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would challenge you by suggesting that your last question: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&quot;What can be done to distinguish a sufficiently advanced algorithm from AI? Is it even possible to do so? Is a sufficiently advanced algorithm, by its very nature, AI?&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is inherently flawed. Because have you forgotten how we too are free running, biologically self-programming, beings? Where our biological algorithms can often also be critically flawed? Therefore, let me ask you if the reverse is also true? Does an algorithm which seems to make mistakes, any less than one which does not?&lt;/p&gt;&#xA;" OwnerUserId="4828" LastEditorUserId="4828" LastEditDate="2017-01-15T15:33:37.397" LastActivityDate="2017-01-15T15:33:37.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2666" PostTypeId="2" ParentId="2319" CreationDate="2017-01-15T19:30:07.467" Score="3" Body="&lt;p&gt;I strongly disagree with all the former comments. Not because they are wrong, -which they are not - but because they are misleading - though unintentionally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: If one looks at these problems from an academic position, the problems will always seem insurmountable. This is because everything is coldly assessed and calculated in isolation to everything else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer predominantly lies in &lt;strong&gt;&lt;em&gt;word association&lt;/em&gt;&lt;/strong&gt;. You have to write a program that can process a vast database of digital books, to register every word and all the words in that language which are associated with it. Plus all the statistical information with each associated word and its associated punctuation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will then give you the basis on which an AI can decide several things: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Whether the structure of a given sentence is correct.&lt;/li&gt;&#xA;&lt;li&gt;If the structure is bad, what the probability is for determining the context and intent of what is being said.&lt;/li&gt;&#xA;&lt;li&gt;The correct meaning and application of a multifaceted word (Triumph), is by probability - according to the statistics.&lt;/li&gt;&#xA;&lt;li&gt;To determine where a conversation is likely to be going. &lt;/li&gt;&#xA;&lt;li&gt;What the correct grammar, and punctuation should be.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, in conclusion, you have two things to look for: Association and probability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;When digitally databasing a language model, the possibility of word and sentence &quot;strings&quot; occurs, so that every variation of language structure in any given sentence can be determined before, during and after a text sample is being scribed. This intimate control over language model patterns, means that sensitive components such as &quot;subject&quot; and &quot;object&quot; can be determined easily by code.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4828" LastEditorUserId="4828" LastEditDate="2017-01-16T14:20:16.013" LastActivityDate="2017-01-16T14:20:16.013" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2667" PostTypeId="2" ParentId="2663" CreationDate="2017-01-15T20:51:48.947" Score="3" Body="&lt;p&gt;In general, Google autocompletes (and produces search results) based on wide variety of factors, including (but not limited to) your location, your search history, your other Google accounts, your site visit history, your language settings, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the specific question, I see a few ways in which Google might have access to the relevant information:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you are syncing your documents with Google Drive it will index the document that most recently changed (which would be the document you are writing) and it could analyze it for patterns to produce relevant search results and suggestions&lt;/li&gt;&#xA;&lt;li&gt;If you emailed the document recently anywhere via Gmail it could also index the contents as above&lt;/li&gt;&#xA;&lt;li&gt;Finally, if you are typing the document in Google Docs, it could also have access, but I assume by &quot;Word&quot; you mean Microsoft Word, so this is probably irrelevant in this case&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't know specifically whether or not Google uses Drive or Gmail contents in the way described, but it would certainly make sense given that it is well known that they do use this information to target advertising.&lt;/p&gt;&#xA;" OwnerUserId="4839" LastActivityDate="2017-01-15T20:51:48.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2668" PostTypeId="1" CreationDate="2017-01-15T21:21:02.107" Score="4" ViewCount="81" Body="&lt;p&gt;I am trying to make an intelligent agent similar to Jarvis from Iron Man, but much less complex. However, I want my AI to be able to determine if I am talking to it or not. So, I plan on having it always listen to my voice and convert that to text. However, I am not sure how I can train the AI to recognize if it is being spoken to or not?&lt;/p&gt;&#xA;" OwnerUserId="4841" LastEditorUserId="2444" LastEditDate="2020-05-18T23:14:31.153" LastActivityDate="2020-05-18T23:14:31.153" Title="How can I design an AI that knows when its being spoken to?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;intelligent-agent&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2669" PostTypeId="1" AcceptedAnswerId="2671" CreationDate="2017-01-16T02:55:42.147" Score="4" ViewCount="64" Body="&lt;p&gt;Let the &lt;em&gt;request&lt;/em&gt; be the thing that we asked the AI to do, not necessarily using &lt;em&gt;&lt;a href=&quot;https://www.cnet.com/how-to/the-complete-list-of-siri-commands/&quot; rel=&quot;nofollow noreferrer&quot;&gt;commands&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays, we have our personal AI in our devices: Siri by Apple, Cortana by Microsoft, and so on. Most times, when we ask them to do certain tasks, they do the tasks for us. However, their action is based on the list of &lt;em&gt;commands&lt;/em&gt;. When they don't clearly recognize the commands in our request, they suggest us to use certain commands. It is clear that there are limits to our choices (requests).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, let's suppose that we have an AI that can interpret requests. There may not be &lt;em&gt;commands&lt;/em&gt; in our request. AI is fully able to do anything in order to do what it is asked for. Basically, I am talking about an independent AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider the scenario where an AI is asked to clean the room. AI is allowed to throw away garbage, and move unnecessary (or unused) stuff into the storage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the list of things that were in the room at the moment:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A stained blanket&lt;/li&gt;&#xA;&lt;li&gt;Various decorations&lt;/li&gt;&#xA;&lt;li&gt;A dead clock on the wall&lt;/li&gt;&#xA;&lt;li&gt;Various unused items in the desk drawer&lt;/li&gt;&#xA;&lt;li&gt;A lost Airpod under the bed&lt;/li&gt;&#xA;&lt;li&gt;A sleeping cat in the bed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this condition&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is washing a stained blanket a part of cleaning?&lt;/li&gt;&#xA;&lt;li&gt;How can AI tell if anything is in use? Are decorations in use?&lt;/li&gt;&#xA;&lt;li&gt;Would dead clock that only needs battery replacement considered garbage?&lt;/li&gt;&#xA;&lt;li&gt;Would items in the desk drawer be included in AI's to-be-cleared list?&lt;/li&gt;&#xA;&lt;li&gt;Would AI be able to recognize the difference between unused and lost?&lt;/li&gt;&#xA;&lt;li&gt;What would happen to the poor cat?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since there are many holes in the scenario and questions, I would like to know how the answers are derived.&lt;/p&gt;&#xA;" OwnerUserId="4802" LastEditorUserId="2444" LastEditDate="2020-04-01T15:35:52.023" LastActivityDate="2020-04-01T15:35:52.023" Title="How could an AI interpret the requests of a human without the human needing to specify every detail?" Tags="&lt;natural-language-processing&gt;&lt;agi&gt;&lt;computational-linguistics&gt;&lt;ai-complete&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2670" PostTypeId="2" ParentId="2668" CreationDate="2017-01-16T04:06:06.837" Score="2" Body="&lt;p&gt;Cheep digital assistant &quot;AI&quot; 's have a call word &lt;code&gt;Hey, &amp;lt;AI's NAME&amp;gt;&lt;/code&gt;&#xA;I assume you want a bit more than that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could train it to figure out which words in some context determine if you are engaging with it or not.&#xA;If your only question to the network is if you are engaging with it or talking to someone else this is all you'd need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Index a dictionary or have it build one from collecting words (building a dictionary from scratch is a better solution it saves space in the short term and is more easily expandable in the long term) and score words based on usage in engaging speech and non-engaging speech or what you want it to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Build on that with an index of multi word strings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the end hopefully you will have a table of contexts when you are engaging with the AI when you definitely are not and some grey area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training process is long and tedious but if you have a recording of you talking and not talking to the AI and you feed it with such knowledge and you breed the network you should have it get okay at determining context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to sit and hold it's hand for 2-72 hours while it grows up it will likely be painful, although you may end up with a better result.&lt;/p&gt;&#xA;" OwnerUserId="4844" LastActivityDate="2017-01-16T04:06:06.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2671" PostTypeId="2" ParentId="2669" CreationDate="2017-01-16T10:33:15.847" Score="3" Body="&lt;p&gt;This is basically the problem of &lt;a href=&quot;https://en.wikipedia.org/wiki/Commonsense_knowledge_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;commonsense knowledge&lt;/a&gt;. It is &lt;a href=&quot;https://en.wikipedia.org/wiki/AI-complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-complete&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we knew how to solve it, Siri and Cortana wouldn't be as limited as they are. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-01-16T10:33:15.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2672" PostTypeId="1" CreationDate="2017-01-16T12:17:41.363" Score="5" ViewCount="670" Body="&lt;p&gt;Most companies dealing with deep learning (automotive - Comma.ai, Mobileye, various automakers etc.) do collect large amounts of data to learn from and then use lots of computational power to train a neural network (NN) from such big data. I guess this model is mainly used because both the big data and the training algorithms should remain secret/proprietary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand it correctly the problem with deep learning is that one needs to have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;big data to learn from&lt;/li&gt;&#xA;&lt;li&gt;lots of hardware to train the neural network from this big data&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am trying to think how crowdsourcing could be used in this scenario. Is it possible to distribute the training of the NN to the crowd? I mean not to collect the big data to a central place but instead to do the training from local data on the user's hardware (in a distributed way). The result if this would be lots of trained NNs that would in the end be merged into one in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Committee_machine&quot; rel=&quot;noreferrer&quot;&gt;Committee of machines&lt;/a&gt; (CoM) way. Would such model be possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the above stated model does have a significant drawback - one does not have control over the data that is used for learning (users could intentionally submit wrong/fake data that would lower the quality of the final CoM). This may be dealt with by sending random data samples to the central community server for review however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: Think of a powerful smartphone using its camera to capture a road from vehicle's dashboard and using it for training lane detection. Every user would do the training himself/herself (possibly including any manual work like input image classification for supervised learning etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder it he model proposed above may be viable. Or is there a better model how to use crowdsourcing (user community) to deal with machine learning?&lt;/p&gt;&#xA;" OwnerUserId="113" LastActivityDate="2017-09-27T05:43:55.527" Title="Using crowdsourcing for deep learning" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2673" PostTypeId="2" ParentId="2668" CreationDate="2017-01-16T12:53:50.453" Score="3" Body="&lt;h2&gt;Phrase detection instead of text-to-speech&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that detection of particular phrases or commands is considered a distinct problem, different from text to speech / text transcription.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While you &lt;em&gt;can&lt;/em&gt; simply convert &lt;em&gt;everything&lt;/em&gt; it hears to text and then look up keywords there, a specialized detector that directly tries to match incoming audio to a small subset of commands can be done with better accuracy and less processing power required. For this reason, this would generally be the preferred approach in commercial products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for beginner experiments with home automation, you should probably start with choosing an existing speech analysis API where all the audio and natural language parts are appropriately implemented by someone else. Building a good speech command analysis system from scratch is a major undertaking by itself, and you will have your hands full with developing an &quot;artificial intelligent agent&quot;; as a rule, you don't want a project where you have to tackle &lt;em&gt;two&lt;/em&gt; major open-ended problems, pick one of them and then you'll have a chance to achieve something interesting there.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2017-01-16T12:53:50.453" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2675" PostTypeId="1" CreationDate="2017-01-16T18:41:36.830" Score="13" ViewCount="32627" Body="&lt;p&gt;If this list&lt;sup&gt;1&lt;/sup&gt; can be used to classify problems in AI ... &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Decomposable to smaller or easier problems&lt;/li&gt;&#xA;  &lt;li&gt;Solution steps can be ignored or undone&lt;/li&gt;&#xA;  &lt;li&gt;Predictable problem universe&lt;/li&gt;&#xA;  &lt;li&gt;Good solutions are obvious&lt;/li&gt;&#xA;  &lt;li&gt;Uses internally consistent knowledge base&lt;/li&gt;&#xA;  &lt;li&gt;Requires lots of knowledge or uses knowledge to constrain solutions&lt;/li&gt;&#xA;  &lt;li&gt;Requires periodic interaction between human and computer&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;... is there a generally accepted relationship between placement of a problem along these dimensions and suitable algorithms/approaches to its solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://images.slideplayer.com/23/6911262/slides/slide_4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://images.slideplayer.com/23/6911262/slides/slide_4.jpg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4856" LastEditorUserId="4302" LastEditDate="2018-10-08T23:01:14.733" LastActivityDate="2019-01-30T04:03:30.403" Title="How can these 7 AI problem characteristics help me decide on an approach to a problem?" Tags="&lt;ai-design&gt;&lt;algorithm&gt;&lt;reference-request&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="2676" PostTypeId="1" CreationDate="2017-01-16T23:01:34.997" Score="6" ViewCount="80" Body="&lt;p&gt;In working with basic &lt;a href=&quot;https://www.tensorflow.org/tutorials/seq2seq/&quot; rel=&quot;noreferrer&quot;&gt;sequence-to-sequence models for machine translation&lt;/a&gt; I have been able to achieve decent results. But inevitably some translations are not optimal or just flat-out incorrect. I am wondering if there is some way of &quot;correcting&quot; the model when it makes mistakes while not compromising the desirable behavior on translations where it previously performed well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an experiment, I took a model that I had previously trained and gathered several examples of translations where it performed poorly. I then took those examples and put them into their own small training set where I provided more desirable translations than what the model was outputting. I then trained the old model on this new small training set very briefly (3-6 training steps was all it took to &quot;learn&quot; the new material). When I tested the new model it translated those several examples in the exact way I had specified. But as I should have anticipated the model overcompensated to &quot;memorize&quot; those handful of new examples  and thus I noticed it started to perform poorly on translations that it had previously been excellent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way to avoid this behavior short of simply retraining the model from scratch on an updated data set? I think I understand intuitively that the nature of neural networks would not lend itself to small precise corrections (i.e. when the weighting of just a few neurons change the performance of the entire model will change) but maybe there is a way around it, perhaps with some type of hybrid reinforcement learning approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://www.aclweb.org/anthology/W15-4006&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt; speaks of approaches to incrementally improving neural machine translation models&lt;/p&gt;&#xA;" OwnerUserId="4862" LastEditorUserId="4862" LastEditDate="2017-01-19T00:54:36.540" LastActivityDate="2018-10-12T18:00:53.433" Title="Correcting 'bad' translations in a sequence-to-sequence neural machine translation model" Tags="&lt;machine-learning&gt;&lt;models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2678" PostTypeId="2" ParentId="2655" CreationDate="2017-01-17T00:31:04.233" Score="10" Body="&lt;p&gt;I'd say the most successful are the ones so commonly used that we don't even notice them: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;em&gt;mail systems&lt;/em&gt; that automatically decipher handwritten addresses on your packages, they use machine vision and have probably been doing it since mid-90s.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Algorithmic trading bots on &lt;em&gt;stock markets&lt;/em&gt; - they handle something like 85% of all trades.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many modern CPUs use AI techniques, including neural networks, to guess what your program is going to do next and optimize branch prediction and memory fetches.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Most good modern &lt;em&gt;fraud&lt;/em&gt; and &lt;em&gt;spam detectors&lt;/em&gt; use some combination of AI techniques (clustering, decision trees, SVMs, even some machine vision to check out attached pictures) - and, in the opposing BlackHat camp, the latest automatic CAPTCHA breakers use all the latest advancements in deep learning too).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And of course there's &lt;em&gt;Google&lt;/em&gt;, &lt;em&gt;Facebook&lt;/em&gt; and US DoD who try to put AI into anything they can think of.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4865" LastEditorUserId="4865" LastEditDate="2017-01-27T11:02:27.857" LastActivityDate="2017-01-27T11:02:27.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2680" PostTypeId="2" ParentId="2646" CreationDate="2017-01-17T04:09:55.897" Score="6" Body="&lt;p&gt;I suggest you look at all the ways we have tried to stop people from abusing OTHER PEOPLE. There is no ethical grey area here - everyone is clear that this is wrong. And yet people are murdered, raped, and assaulted in their millions every day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we solve this problem with regard to human victims, the resulting solution will most likely work just fine for AIs as well.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2017-01-17T04:09:55.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2681" PostTypeId="1" AcceptedAnswerId="2691" CreationDate="2017-01-17T04:11:20.930" Score="10" ViewCount="538" Body="&lt;p&gt;I am currently working on an Android AI app. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of AI models to generate random sentences. However, is there an AI model for generating sarcastic sentences?&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="2444" LastEditDate="2020-03-09T16:44:37.963" LastActivityDate="2020-03-09T16:53:01.080" Title="Is there an AI model for generating sarcastic sentences?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;reference-request&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2685" PostTypeId="5" CreationDate="2017-01-18T12:27:55.323" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-01-18T12:27:55.323" LastActivityDate="2017-01-18T12:27:55.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2686" PostTypeId="4" CreationDate="2017-01-18T12:27:55.323" Score="0" Body="In AI, the term &quot;heuristic&quot; is used in the context of non-blind (i.e., informed) search and planning: the problem of finding a sequence of actions to find/generate a desired state from an initial state.&#xD;&#xA;Heuristics&quot; are problem relaxations of the original problem. They get as input the current state/search node and output the cost of the relaxed solution from there to goal. This heuristic value is then used for search guidance." OwnerUserId="4893" LastEditorUserId="4893" LastEditDate="2017-02-27T02:34:25.970" LastActivityDate="2017-02-27T02:34:25.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2687" PostTypeId="2" ParentId="2342" CreationDate="2017-01-18T14:52:53.387" Score="1" Body="&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt; First of all, you state that that the goal G2 will be found first by relying on the expansion order &lt;code&gt;R, B, D, G2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is wrong. It is extremely easy to see that this is wrong, because A* is a search algorithm that guarantees to find an optimal solution given that only admissible heuristics are used. (A heuristic is being admissible if it never over-estimates the optimal goal distance. This is the case in your example.) Since the true cost for reaching G1 is 11 and the true cost for reaching G2 is 13, clearly G1 must be found first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, your expansion order is wrong as well. Let us first give the f-values for all nodes:&#xA;&lt;code&gt;f(A)=11, f(B)=10, f(C)=11, f(D)=13&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that h(G1)=h(G2)=0 (i.e, the heuristic is &quot;goal-aware&quot;), we get &lt;code&gt;f(G1)=11&lt;/code&gt; and &lt;code&gt;f(G2)=13&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because A* expands search nodes by lowest f-values of the search nodes in the open list (the search nodes not yet expanded), we get the following expansion order:&#xA;&lt;code&gt;R, B, A, C, G1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You very-likely did a mistake that is done extremely often: after heaving expanded D, you add G2 to the open list. Because G1 is a goal node and you are already &quot;seeing&quot; it, you return it as a solution. But this is wrong! Goal nodes are &lt;em&gt;not&lt;/em&gt; returned when being created, but when being selected for expansion! So, although the expansion of D generates G2, you are not allowed to return G2 as solution, because it has not been selected for expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2:&lt;/strong&gt;&#xA;Can G2 be found as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;em&gt;NietzscheanAI&lt;/em&gt; pointed out, you can simple continue search. That is, after heaving expanded  &lt;code&gt;R, B, A, C, G1&lt;/code&gt;, A* will expand &lt;code&gt;D, G2&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4893" LastActivityDate="2017-01-18T14:52:53.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2688" PostTypeId="2" ParentId="2306" CreationDate="2017-01-18T15:02:52.427" Score="2" Body="&lt;p&gt;The journal &quot;Artificial Intelligence (AI)&quot; (&lt;a href=&quot;https://www.journals.elsevier.com/artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.journals.elsevier.com/artificial-intelligence/&lt;/a&gt;) was not listed, yet, although being considered &lt;em&gt;the&lt;/em&gt; top-level journal on AI. Although this is a journal for AI (just being named &quot;Artificial Intelligence&quot;), it is not to be confused with another top-level AI journal, called &quot;Journal on Artificial Intelligence Research (JAIR)&quot; (&lt;a href=&quot;http://www.jair.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.jair.org/&lt;/a&gt;), which was already listed in one of the other answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, there is a German Journal on AI, called &quot;KI - Künstliche Intelligenz&quot; (German for AI), but almost always the articles are in English as well (&lt;a href=&quot;http://www.kuenstliche-intelligenz.de/en/ki-journal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.kuenstliche-intelligenz.de/en/ki-journal/&lt;/a&gt;). While being internationally recognized, it is not regarded a top-level journal. A nice feature of that journal is that every special issue has an editorial (a special &quot;article&quot; at the beginning of each journal), in which there is a section called &quot;service&quot;. This service section lists publication media (like journals) and conferences etc. that are related to the given special issue. So, in case you are interested in journals of a special field of AI (like human-computer interaction), just search for a special issue that is related to that topic and read the editorial's service part.&lt;/p&gt;&#xA;" OwnerUserId="4893" LastEditorUserId="4893" LastEditDate="2017-03-09T10:29:51.487" LastActivityDate="2017-03-09T10:29:51.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2689" PostTypeId="1" CreationDate="2017-01-18T17:38:31.650" Score="6" ViewCount="376" Body="&lt;p&gt;In the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies (NEAT)&lt;/a&gt; paper it says (p. 110): &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The entire population is then replaced by the offspring of the remaining organisms in each species.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But how does it take place? Are they paired and then mated? Could this lead to fast extinction? Or are they pair each with each other? This would lead to overpopulation very fast. &lt;em&gt;How are individuals paired, in NEAT&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="2444" LastEditDate="2019-07-07T19:11:04.113" LastActivityDate="2019-07-07T19:11:04.113" Title="How does mating take place in NEAT?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2690" PostTypeId="2" ParentId="2681" CreationDate="2017-01-19T03:33:44.610" Score="8" Body="&lt;p&gt;A simple form of sarcasm involves a direct reversal of the literal meaning of the statement, eg &quot;Great weather we're having&quot; (during a thunderstorm), &quot;just what I needed&quot; (when something goes wrong).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with doing this in random sentences is that you may have no context to establish the reversal of the literal meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could possibly construct them by using a template along the lines of &quot;Just what I needed - (random bad thing happened) today&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, when an outcome of a process is calculated, if it is not the desired outcome, instead of returning &quot;mission unsuccessful&quot; or &quot;mission not yet complete&quot;, the AI could say &quot;you're having a great day, aren't you? - mission unsuccessful&quot; or &quot;great work, genius - mission not yet complete&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most random sentences will not be suitable for sarcasm, so it could only be applied in specific circumstances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear from your question what the context is for these random sentences, and therefore it is not clear whether that context would be suitable for sarcasm at all.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2017-01-19T03:33:44.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2691" PostTypeId="2" ParentId="2681" CreationDate="2017-01-19T11:56:04.317" Score="2" Body="&lt;p&gt;You could also build a database of sarcastic sentences, especially from, for example historic plays. And then train your software to recognize patterns of those sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. grammatical constructions/order, length (or circomstances building up to the sarcasm). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And use that database as starting point, with feedback to learn, or you could use the above method to improve your effective output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to use a similar but reverse approach; study those databases and build an equivalent output based on the coherence, and then extrapolate the output-generation procedure. (In combination with other methods)&lt;/p&gt;&#xA;" OwnerUserId="4903" LastEditorUserId="4903" LastEditDate="2017-01-19T13:00:59.110" LastActivityDate="2017-01-19T13:00:59.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2692" PostTypeId="1" AcceptedAnswerId="2696" CreationDate="2017-01-19T14:57:05.603" Score="2" ViewCount="2390" Body="&lt;p&gt;I'm an artificial intelligence enthusiastic and I want to learn about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to ask you what do you think about the Udacity nanodegree &lt;a href=&quot;https://www.udacity.com/course/deep-learning-nanodegree-foundation--nd101&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Nanodegree Foundation&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if it is a good idea to pay for that course or maybe, there are better free resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to understand what artificial intelligence is, and also learn about machine learning, deep learning, and convolutional networks. I'm interested in image and speech recognition and also in artificial life.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2019-06-20T22:13:38.227" LastActivityDate="2019-06-20T22:13:38.227" Title="Is it a good idea to pay for an deep learning course?" Tags="&lt;deep-learning&gt;&lt;getting-started&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="4" ClosedDate="2017-03-19T23:36:36.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="2693" PostTypeId="1" AcceptedAnswerId="2697" CreationDate="2017-01-19T16:37:45.173" Score="2" ViewCount="54" Body="&lt;p&gt;Hardware comes in two forms, basically: immutable, such as RAM, and mutable, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Field-programmable_gate_array&quot; rel=&quot;nofollow noreferrer&quot;&gt;FPGA&lt;/a&gt;s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In animals, neurological connections gain in strength by changing the physical structure of the brain. This is analogous to FPGAs whereby signal strength is increased by changing the pathways themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we achieve sentience using mutable hardware (e.g., &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering#Neuromemristive_systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuromemristive systems&lt;/a&gt;), will it be possible to make a copy of that &quot;brain&quot; and its active state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this question, assume that the brain is how the hardware has &quot;reconfigured&quot; [or etched, if you will] its pathways to strengthen them and the brain's state is captured by how electrons are physically flowing throughout those pathways.&lt;/p&gt;&#xA;" OwnerUserId="4922" LastEditorUserId="2444" LastEditDate="2020-05-13T21:26:59.923" LastActivityDate="2020-05-13T21:26:59.923" Title="If we achieve sentience using mutable hardware, will it be possible to make a copy of that &quot;brain&quot; and its active state?" Tags="&lt;agi&gt;&lt;hardware&gt;&lt;artificial-consciousness&gt;&lt;brain&gt;&lt;sentience&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2696" PostTypeId="2" ParentId="2692" CreationDate="2017-01-19T21:31:21.280" Score="10" Body="&lt;p&gt;It doesn't seem expensive at $399* &lt;em&gt;(although the asterix needs to be taken into consideration.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in this subject, this may be a decent course, however it is certainly not an accredited institution, thus &lt;strong&gt;any &quot;degree&quot; you get from this course will be meaningless in an academic sense.&lt;/strong&gt;  On the other hand, a certificate shows that you've committed resources toward training.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My high level take on this is that the &quot;Foundation&quot; is looking to capitalize on the recent publicity for Deep Learning, per the AlphaGo milestones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I can tell you for certain is that this field requires advanced mathematics from a research perspective, and people who work in that area spend years training to gain the requisite skills.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The requirements for this class seem to be restricted to &lt;strong&gt;&quot;Python knowledge&quot; with no mention of mathematics, which raises some alarm bells,&lt;/strong&gt; although it's worth mentioning that once solutions/applications arise, they can be applied to varying degrees without having to understand the mathematics. &lt;em&gt;[See mindcrime's comment]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-17T17:42:29.433" LastActivityDate="2018-01-17T17:42:29.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2697" PostTypeId="2" ParentId="2693" CreationDate="2017-01-19T21:44:38.883" Score="2" Body="&lt;p&gt;Theoretically, there shouldn't be a problem copying either of the artificial brains in any state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Difficulty in measuring a state doesn't seem to really be a problem until you get down to the quantum level, where the means of measurement affect the state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The configuration of the artificial brains, including pathway structures and states, should be reducible to a single string, which could then be used to reconfigure the artificial brain the information is being copied to.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Definitely look into the concept of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Machine&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Turing Machine&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-01-19T21:44:38.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2698" PostTypeId="2" ParentId="2645" CreationDate="2017-01-19T22:04:08.947" Score="1" Body="&lt;p&gt;Part of the reason people are so excited about recent Machine Learning milestones is that AlphaGo demonstrated a reproducible method of managing mathematical and computational intractability.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go is interesting because it's impossible to solve.  It cannot be brute-forced no matter how fast processors get.  Go is so complex humans had failed to produce AI that could win against a skilled human player.  The fact that a computer could teach itself to do something humans couldn't teach it, and something with a complexity analogous to nature to boot, is pretty extraordinary.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combinatorial games in particular are useful because, unlike nature where it may be impossible to track or even be aware of every variable, intractability can be generated out of a simple set of elements and rules, and outcomes can be definitively evaluated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As proof-of-concepts for methods go, AlphaGo seems like a pretty strong one.  It allows us to definitively say &quot;Machine Learning works&quot;, puts a lot of emphasis on the field, and raises confidence on extending the method to real world problems.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that, it suggests a feedback loop in which programs can improve at at improving, unrestricted by human limitations.  Increase in processing power is bounded by physical limitations, but algorithms are not.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-01-22T05:53:51.893" LastActivityDate="2017-01-22T05:53:51.893" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2701" PostTypeId="1" AcceptedAnswerId="2702" CreationDate="2017-01-20T08:49:40.527" Score="2" ViewCount="776" Body="&lt;p&gt;I want to develop an artificial life simulator to simulate cells living in water.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to see how they search for food, how they life and die and how they reproduce and evolve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I don't know where to start, I have no idea about if there are books or tutorial about how to program this kind of simulator. And also I don't know if I can use here machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, I'm a programmer and I want to do it using C++ and Unreal Engine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find more info about how to do it?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2017-01-20T10:41:00.307" LastActivityDate="2017-01-20T11:09:23.773" Title="Artificial life simulator" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;genetic-algorithms&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="0" ClosedDate="2017-01-20T19:25:01.203" ContentLicense="CC BY-SA 3.0" />
  <row Id="2702" PostTypeId="2" ParentId="2701" CreationDate="2017-01-20T11:09:23.773" Score="2" Body="&lt;p&gt;The best approach would be starting with smaller projects involving &lt;em&gt;neural networks&lt;/em&gt; and &lt;em&gt;genetic algorithms&lt;/em&gt; to gain experience in order to speedup the coding of the project you have proposed; playing around with &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; and &lt;a href=&quot;https://www.unrealengine.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unreal Engine&lt;/a&gt; it is not a bad idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hint&lt;/strong&gt;: when implementing your idea of artificial life, you should consider that each cell/organism have to have some kind of &lt;em&gt;sensors&lt;/em&gt; in order to capture informations from the environment; such informations i.e. the position and the distance of the nearest meal and/or predators, the temperature, the pressure and depth of water, should be passed through the neural network to determine the response of the cell. Also, in your environment you should promote the spreading of organisms which responses are euristically better i.e. cells that don't get caught by predators or don't die by starvation. How? Simply by evolving their brain/brains/sensors through a genetic algorithm that favors individuals/species with good parameters. I recommend you a nature-inspired AI method, it is called NEAT model. It explains how to implement a neural networks that can be evolved. The paper can be found here: &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different approach to &lt;em&gt;NEAT&lt;/em&gt; would be &lt;a href=&quot;http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning&lt;/a&gt;; in the link you can find a demo artifical organism that learns how to find meals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a ton of parameters and implementations you can consider, the only limit is your creativity.&lt;/p&gt;&#xA;" OwnerUserId="4801" LastActivityDate="2017-01-20T11:09:23.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2703" PostTypeId="1" CreationDate="2017-01-21T11:42:10.403" Score="4" ViewCount="248" Body="&lt;p&gt;I confront to the next scenario:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Let's say I have stored data about football matches between different teams: lineups, scorers, yellow cards, and many other events.&lt;/p&gt;&#xA;&lt;p&gt;I need to generate everyday some questions about the matches that will be played on that day. So, if I give an input of two teams, I would like a related question to be generated, based on previous data of matches between those two teams.&lt;/p&gt;&#xA;&lt;p&gt;For example, if my input are &lt;em&gt;&amp;quot;TeamA&amp;quot;&lt;/em&gt; and &lt;em&gt;&amp;quot;TeamB&amp;quot;&lt;/em&gt;, I would expect a question of the type:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;&amp;quot;Will there be less than 2 goals scored in the match?&amp;quot;&lt;/em&gt;&amp;quot;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;&lt;em&gt;Will PlayerX score a goal during the match?&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Of course I expect these questions to make sense based on previous data from matches between the two given teams.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, my questions are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would be a good solution to use AI to generate these questions? It would make sense?&lt;/li&gt;&#xA;&lt;li&gt;What would be the best approach?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4950" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-08T12:31:37.183" Title="Could AI be used to generate questions from a database input?" Tags="&lt;machine-learning&gt;&lt;gaming&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2704" PostTypeId="2" ParentId="2703" CreationDate="2017-01-21T12:27:53.393" Score="3" Body="&lt;p&gt;One simple approach to consider would be storing each statement as a template made in advance.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Will there be &lt;code&gt;less/more&lt;/code&gt; than &lt;code&gt;x&lt;/code&gt; goals scored in the match?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Will &lt;code&gt;player&lt;/code&gt; score a goal during the match?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The system will pick a random statement and will fill the variable fields with some statistically generated data between &lt;code&gt;teamA&lt;/code&gt; and &lt;code&gt;teamB&lt;/code&gt;; here you have your question.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;: &lt;em&gt;Will there be &lt;code&gt;less/more&lt;/code&gt; than &lt;code&gt;x&lt;/code&gt; goals scored in the match?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;less/more&lt;/code&gt; fragment may be random&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;x&lt;/code&gt; may be the mean of the goals scored considering all the matches between &lt;code&gt;teamA&lt;/code&gt; and &lt;code&gt;teamB&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;: &lt;em&gt;Will &lt;code&gt;player&lt;/code&gt; score a goal during the match?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;player&lt;/code&gt; may be a random choice between the top-goalscorer of &lt;code&gt;teamA&lt;/code&gt; or &lt;code&gt;teamB&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4801" LastActivityDate="2017-01-21T12:27:53.393" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2706" PostTypeId="1" CreationDate="2017-01-21T21:02:30.960" Score="6" ViewCount="462" Body="&lt;p&gt;I know there are different AI tests but I'm wondering why other tests are little-known. Is the Turing test hyped? Are there any scientific reasons to prefer one test to the other?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is the Turing test so popular?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4801" LastActivityDate="2020-05-18T23:58:38.073" Title="Why is the Turing test so popular?" Tags="&lt;turing-test&gt;" AnswerCount="5" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2707" PostTypeId="2" ParentId="2644" CreationDate="2017-01-22T20:57:27.473" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Would AI be able to self-examine objectively and determine if it is capable of doing the task?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A possible approach might be the one suggested and studied by J.Pitrat (one of the earliest AI researcher in France, his PhD on AI was published in the early 1960s and he is now a retired scientist). Read his &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bootstrapping Artificial Intelligence blog&lt;/a&gt; and his &lt;a href=&quot;http://onlinelibrary.wiley.com/book/10.1002/9780470611791&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Beings: the conscience of a conscious machine&lt;/a&gt; book.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;(I'm not able to summarize his ideas in a few words, even if I do know J.Pitrat -and even meet him once in a while- ; grossly speaking, he has a strong meta-knowledge approach combined with reflexive programming techniques. He is working -alone- since more than 30 years on his CAIA system, which is very difficult to understand, because even while he does publish his system as a free software pragram, CAIA is  not user friendly, with a poorly documented common line user interface; while I am enthusiastic about his work, I am unable to explore his system.)&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;But defining what &amp;quot;conscience&amp;quot; or &amp;quot;self-awareness&amp;quot; could &lt;em&gt;precisely&lt;/em&gt; mean for some artificial intelligence system is a hard problem by itself. AFAIU, even for human intelligence, we don't exactly know what that &lt;em&gt;really means&lt;/em&gt; and how does that &lt;em&gt;exactly&lt;/em&gt; work. IMHO, there is &lt;em&gt;no consensus&lt;/em&gt; on some &lt;em&gt;definition&lt;/em&gt; of &amp;quot;conscience&amp;quot;, &amp;quot;self-awareness&amp;quot;, &amp;quot;self-examination&amp;quot; (even when applied to humans).&lt;/p&gt;&#xA;&lt;p&gt;But whatever approach is used, giving any kind of constructive answer to your question requires a lot of pages. J.Pitrat's books &amp;amp; blogs are a better attempt than what anyone could answer here. So your question is IMHO too broad.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-01-22T21:09:17.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2708" PostTypeId="1" AcceptedAnswerId="2710" CreationDate="2017-01-22T23:45:16.640" Score="2" ViewCount="100" Body="&lt;p&gt;If you had a web of linked Watson-level super-computers, would they be more effective at problem-solving than a single Watson computer alone?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you asked the Watson-web to diagnose a person's as-yet-undiagnosed disease, would the web be able to do so more quickly?&lt;/p&gt;&#xA;" OwnerUserId="4975" LastActivityDate="2017-01-23T06:36:29.453" Title="Would linked Watson supercomputers be even &quot;smarter&quot; than one Watson?" Tags="&lt;watson&gt;&lt;problem-solving&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2709" PostTypeId="1" CreationDate="2017-01-22T23:50:27.090" Score="4" ViewCount="227" Body="&lt;p&gt;I am drawing this question from Berkeley's AI course (also not sure if it is the correct place to ask, so I apologize ahead of time)&#xA;&lt;a href=&quot;https://inst.eecs.berkeley.edu/~cs188/pacman/course_schedule.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://inst.eecs.berkeley.edu/~cs188/pacman/course_schedule.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am working on section 3's Homework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: the question (Part 1, question 6). Why is it that we can only guarantee that if the Min agent acts suboptimally, the best we can hope for is the following &lt;a href=&quot;https://i.stack.imgur.com/lLe7K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lLe7K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that we can put any arbitrary value for the second node e.g. whey does it have to be -Episolon. It could be any range of values, e.g. Epsilon, in which case we would have optimised the Player A&lt;/p&gt;&#xA;" OwnerUserId="4974" LastActivityDate="2017-07-22T19:40:52.877" Title="Berkeley AI Course Question on Nearly Zero Sum Games" Tags="&lt;research&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2710" PostTypeId="2" ParentId="2708" CreationDate="2017-01-23T06:36:29.453" Score="0" Body="&lt;p&gt;I guess that they would be only &lt;em&gt;marginally&lt;/em&gt; better. And be aware that &lt;a href=&quot;https://en.wikipedia.org/wiki/Watson_%28computer%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Watson&lt;/a&gt; itself is &lt;em&gt;already&lt;/em&gt; a cluster of (quite big) computers (citing Wikipedia):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Watson employs a cluster of ninety IBM Power 750 servers, each of which uses a 3.5 GHz POWER7 eight-core processor, with four threads per core. In total, the system has 2,880 POWER7 processor threads and 16 terabytes of RAM&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because of the rules of Jeopardy, Watson was not allowed to use the Web &lt;em&gt;during&lt;/em&gt; the game. At some very high level, it contains a &lt;em&gt;digested&lt;/em&gt; cache of some of the Web contents. Giving access to the Internet would improve slightly Watson's performance, but not that much.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if you asked the Watson-web to diagnose a person's as-yet-undiagnosed disease, would the web be able to do so more quickly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Even if by &quot;web&quot; you mean a more powerful cluster accessing the entire Internet, I don't think that it would answer &lt;em&gt;much&lt;/em&gt; more quickly &amp;amp; accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is a diversity of sub-domains, and is not advancing as dramatically as some believe. It follows a gradual progression, with in some &lt;em&gt;limited&lt;/em&gt; fields (playing Go, or Jeopardy) some spectacular progresses. See also &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/2016/11/everything-but-the-essential/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Everything but the essential&lt;/em&gt;&lt;/a&gt; blog entry by J.Pitrat.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastActivityDate="2017-01-23T06:36:29.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2711" PostTypeId="2" ParentId="2709" CreationDate="2017-01-23T08:58:02.663" Score="3" Body="&lt;p&gt;(using X for epsilon because keyboard)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a hypothesis, but if you have a maximising agent and a minimising agent, then the optimal outcome for A (maximising) is to sweep the board (X,0), while the optimal outcome for B (minimising) is (-X,0) because B is minimising A's score, not their own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A's optimal outcome is then complicated by the factor for sub-optimality, which we then imagine approaches zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seem to be a bunch of assumptions that are not articulated, though, if this hypothesis is true.&lt;/p&gt;&#xA;" OwnerUserId="3601" LastActivityDate="2017-01-23T08:58:02.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2712" PostTypeId="1" CreationDate="2017-01-23T09:22:56.807" Score="6" ViewCount="368" Body="&lt;p&gt;Has any &lt;a href=&quot;https://en.wikipedia.org/wiki/Schema-agnostic_databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;schema-agnostic database&lt;/a&gt; engine been implemented?&lt;/p&gt;&#xA;" OwnerUserId="4982" LastEditorUserId="2444" LastEditDate="2019-05-06T16:01:30.793" LastActivityDate="2019-08-16T06:54:01.370" Title="Has any schema-agnostic database engine been implemented?" Tags="&lt;natural-language-processing&gt;&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2713" PostTypeId="1" AcceptedAnswerId="2714" CreationDate="2017-01-23T11:43:25.453" Score="6" ViewCount="266" Body="&lt;p&gt;I was wondering what will happen when somebody places a fake speedsign, of 10 miles per hour on a high way. Will a autonomous car slow down? Is this a current issue of autonomous cars? &lt;/p&gt;&#xA;" OwnerUserId="4984" LastEditorUserId="2444" LastEditDate="2019-06-20T15:27:34.323" LastActivityDate="2019-06-20T15:29:57.640" Title="What will happen when you place a fake speedsign on a highway?" Tags="&lt;object-recognition&gt;&lt;autonomous-vehicles&gt;&lt;security&gt;&lt;ai-safety&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2714" PostTypeId="2" ParentId="2713" CreationDate="2017-01-23T14:14:33.037" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/530276/hidden-obstacles-for-googles-self-driving-cars/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.technologyreview.com/s/530276/hidden-obstacles-for-googles-self-driving-cars/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Google’s cars can detect and respond to stop signs that aren't on its map, a feature that was introduced to deal with temporary signs used at construction sites. But in a complex situation like at an unmapped four-way stop the car might fall back to slow, extra cautious driving to avoid making a mistake.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's highly probable they would slow down with current technology, as they can detect temporary signs and are designed to use slow speed in complex cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it was a true temporary sign (e.g. road repair), how can it make the distinction? It probably would be worse to ignore a slow down sign than slow down at a fake one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMHO, the problem there is with the joke in the first place, as some humans might slow down too.&lt;/p&gt;&#xA;" OwnerUserId="4152" LastEditorUserId="2444" LastEditDate="2019-06-20T15:29:57.640" LastActivityDate="2019-06-20T15:29:57.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2715" PostTypeId="2" ParentId="15" CreationDate="2017-01-23T14:21:49.407" Score="4" Body="&lt;p&gt;There are many definitions of Artificial Intelligence out in the wild. All these definitions are part of one (or more) of the areas. There are four main domains, and the picture below will shed some light over this.&lt;br&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/m7ZlO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m7ZlO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;br&gt;Turing Test revolves around the left side of the cardinality, which is mostly concerned with how humans think or act. But, we know that this is just not all. Turing Test has not much to offer when it comes to what AI is in a general sense.&lt;br&gt;&#xA;Turing Test, as the Wikipedia states, was created to test machines exhibiting behaviour equivalent or indistinguishable from that of a human. Artificial Intelligence is much more than what humans can do or how they act. There are many human acts that are considered unintelligent and sometimes inhuman too.&lt;br&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room Argument&lt;/a&gt; focuses on something very important when it comes to &lt;strong&gt;&quot;Consciousness v/s Simulation of Consciousness&quot;&lt;/strong&gt;. John Searle argued there that it is possible for a machine (or human) to follow a huge number of predefined rules (algorithm), in order to complete the task, without thinking or possessing the mind. Weak AIs are good at simulating the ability to understand but, don't really understand what they are doing. They don't exhibit &lt;strong&gt;&quot;Self-Awareness&quot;&lt;/strong&gt; and don't form representation about themselves. &lt;strong&gt;&quot;I want that v/s I know I want that&quot;&lt;/strong&gt; are two different things.&lt;br&gt;&lt;br&gt;&#xA;As Theory of Mind states that a good AI should not just form representation about the world it is working on, but also about other agents and entities in the world. This two concepts of &lt;em&gt;self-awareness and theory of mind&lt;/em&gt; draw a thin line between weak and strong AI.&lt;br&gt;&lt;br&gt;&#xA;When it comes to the Turing Test, it fails on many grounds and so does the Total Turing Test, which adds another layer to the test. Most of the researchers believe that Turing Test is just a distraction from the main goal, something that hinders them from fruitful work. Consider this, suppose you ask a difficult arithmetic problem in order to distinguish between human and machine. If the machine wants to pretend it is human then it will lie. This is not what we want. Going for the Turing Test sets the upper bound to the AI that can be created. Also making AI act and behave like humans is not a very good idea. Humans are not very good at making right decisions all the time. This is the reasons why we read about wars in our history books. Decisions which we make are often biased, have selfish origins, etc. We don't want an AI to come with all those things.&lt;br&gt;&lt;br&gt;&#xA;I don't think there is one test to test an AI. This is because AI has many definitions, many types. Whether an AI is weak or strong can be tagged while looking for answers to questions like, &quot;I want that v/s I know I want that&quot;, &quot;Who am I and what exactly I am doing (from machine's perspective)&quot;, plus some other questions I mentioned above.&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="3005" LastEditDate="2018-04-11T20:16:35.997" LastActivityDate="2018-04-11T20:16:35.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2718" PostTypeId="2" ParentId="40" CreationDate="2017-01-25T00:59:49.040" Score="2" Body="&lt;p&gt;I'll try to answer your questions using Geoffrey Hinton's ideas in dropout paper and his Coursera class.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What purpose does the &amp;quot;dropout&amp;quot; method serve?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, dropout is a regularization technique that addresses the problem of overfitting.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How does it improve the overall performance?&lt;/strong&gt; If by &amp;quot;performance&amp;quot; you mean &amp;quot;generalization&amp;quot;, then it may improve generalization by avoiding over-fitting, as I said.&lt;/p&gt;&#xA;" OwnerUserId="5007" LastEditorUserId="2444" LastEditDate="2020-10-01T20:16:56.117" LastActivityDate="2020-10-01T20:16:56.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2719" PostTypeId="5" CreationDate="2017-01-25T04:15:10.240" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-01-25T04:15:10.240" LastActivityDate="2017-01-25T04:15:10.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2720" PostTypeId="4" CreationDate="2017-01-25T04:15:10.240" Score="0" Body="For questions about LSTM (long-short term memory) networks." OwnerUserId="1807" LastEditorUserId="4302" LastEditDate="2018-10-15T16:55:13.247" LastActivityDate="2018-10-15T16:55:13.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2721" PostTypeId="2" ParentId="2514" CreationDate="2017-01-25T07:32:38.503" Score="4" Body="&lt;p&gt;&lt;strong&gt;Why would one professor only teach searching algorithms in AI course? What are the advantages/disadvantages?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My answer to this question is that there are lots of problems where the solution can be found using searching. Take an example of Tic Tac Toe. If you are designing an intelligent computer player for this, then what you will do is that you will form a search space and then you will search for most optimal move which can be made to conclude the game. In these, scenarios you must be aware of optimal search strategies. Let's take another example, suppose if you are driving and want to got to an unknown person's house. It's far from your place and you decide to use GPS. Your GPS will use search algorithms to find the most optimal route that you can take to reach to the destination (of course there will be lots of factors to consider like traffic, etc. but this is the basic idea).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disadvantages are only in terms of processing and storage. For slow algorithms you will be wasting lots of CPU time and storage as well but for good and efficient algorithms, you can preserve lots of space and also execute your task very fast. Of course, just learning about searching isn't AI. There's lot more to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What's more than &quot;searching&quot; in AI that could be taught in an introductory course?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is lots of things in AI other than searching. For example, learning techniques (supervised, unsupervised, reinforced), planning when one wants to design a system that will do certain actions independently and intelligently, representation of knowledge (known and unknown) and inference in agents which includes propositional logic and first-order logic, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there theories behind AI that could be taught in this kind of course?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some topics could be taught like about different types of agents (simple reflex, model based, goal based, utility based and learning agent), different types of environments in which agents work, evaluation of agents. There could be some additional introductory topics like natural language processing, expert systems, etc.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastActivityDate="2017-01-25T07:32:38.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2722" PostTypeId="1" AcceptedAnswerId="2735" CreationDate="2017-01-25T12:23:58.463" Score="0" ViewCount="186" Body="&lt;p&gt;Instead of directly communicating with the AI, we would instead communicate with a messenger, who would relay our communications to the AI. The messenger would have no power to alter the AI's hardware or software in any way, or to communicate with anything or anyone, except relaying communications to and from the AI and humans asking questions. The messenger could be human, of a software bot. The primary job (and only reason) of the AI would be to act as a filter, not relaying any requests for release back, only the answer to the question asked. The ethics of this method are another debate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI would have to be physically isolated from all outside contact, other than 8 light sensors, and 8 LEDs. The messenger would operate 8 other LEDs, and receive information from 8 light sensors as well. Each AI light sensor would be hooked up to a single messenger controlled LED, and vice versa. Through this system, the two parties could communicate via flashes of light, and since there are 8, the flashes would signal characters in Unicode. &lt;/p&gt;&#xA;" OwnerUserId="4986" LastEditorUserId="2444" LastEditDate="2019-04-12T20:45:05.310" LastActivityDate="2019-04-12T20:45:05.310" Title="Can we use a messenger that does not alter the AI to solve the control problem?" Tags="&lt;philosophy&gt;&lt;control-problem&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2723" PostTypeId="1" AcceptedAnswerId="2724" CreationDate="2017-01-25T18:12:43.350" Score="8" ViewCount="270" Body="&lt;p&gt;OpenAI's Universe utilises RL algorithms and I have heard of some game-training projects using Q learning, but are there any others which are used to master/win games? Can genetic algorithms be used to win at a game?&lt;/p&gt;&#xA;" OwnerUserId="2887" LastEditorUserId="145" LastEditDate="2017-01-26T02:47:24.413" LastActivityDate="2021-01-30T20:47:34.573" Title="Are there any other machine learning models apart from Reinforcement Learning and Q Learning to play video games?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;genetic-algorithms&gt;&lt;game-theory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2724" PostTypeId="2" ParentId="2723" CreationDate="2017-01-25T19:44:29.507" Score="5" Body="&lt;p&gt;As I see it, it all comes down to game theory, which can be said to form the foundation of successful decision making, and is particularly useful in a context, such as computing, where all parameters can be defined.  (Where it runs into trouble is with the aggregate complexity of the parameters per the &lt;a href=&quot;http://cswww.essex.ac.uk/CSP/ComputationalFinanceTeaching/CombinatorialExplosion.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial explosion&lt;/a&gt;, although Machine Learning has recently been validated as a method of managing intractability specifically in the context of games.)&lt;/p&gt;&#xA;&lt;p&gt;You might want to check out &lt;a href=&quot;https://core.ac.uk/download/pdf/205529758.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Games with Genetic Algorithms&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary game theory&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-30T19:35:28.770" LastActivityDate="2021-01-30T19:35:28.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2725" PostTypeId="2" ParentId="2722" CreationDate="2017-01-25T22:30:00.737" Score="2" Body="&lt;p&gt;I would say no due to the possibility of psychological manipulation of the messenger by the AI. Also, the LED communication constraints place severe limitations on the capabilities of the AI, as the usefulness of AI is likely predicated on its ability to learn quickly from a vast amount of information (e.g. using the internet). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some sense you may successfully control an AI using techniques like this but the nuance of the control problem is controlling an AI without restricting its ability to solve our greatest problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We already knew that we could keep an AI safe inside a computer isolated from the rest of the word, but the problem fundamentally is that we if had a truly general AI, we would never &lt;em&gt;want&lt;/em&gt; to keep it isolated. Is there some way to unleash it so that it is fully capable of solving our problems while simultaneously making sure that it is safe?&lt;/p&gt;&#xA;" OwnerUserId="5037" LastActivityDate="2017-01-25T22:30:00.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2726" PostTypeId="2" ParentId="2722" CreationDate="2017-01-26T07:12:43.160" Score="0" Body="&lt;p&gt;OK, all I/O is through the flashes. So the AI flashes the message &quot;Launch the nuclear missiles.&quot; Does the system to which the AI is connected know how to accomplish this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the flashes themselves are not sufficient to control the AI. &lt;/p&gt;&#xA;" OwnerUserId="3471" LastActivityDate="2017-01-26T07:12:43.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2727" PostTypeId="1" AcceptedAnswerId="2732" CreationDate="2017-01-26T10:51:55.857" Score="4" ViewCount="2398" Body="&lt;p&gt;I am working on an implementation of the back propagation algorithm. What I have implemented so far seems working but I can't be sure that the algorithm is well implemented, here is what I have noticed during training test of my network:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specification of the implementation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A data set containing almost 100000 raw containing (3 variable as input, the sinus of the sum of those three variables as expected output).&lt;/li&gt;&#xA;&lt;li&gt;The network does have 7 layers, all the layers use the sigmoid activation function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When I run the back propagation training process:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The minimum of costs of the error is found at the fourth iteration (&lt;strong&gt;The minimum cost of error is 140, is it normal? I was expecting much less than that&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;li&gt;After the fourth iteration the costs of the error start increasing (&lt;strong&gt;I don't know if it is normal or not?&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5054" LastEditorUserId="9647" LastEditDate="2017-10-03T08:57:04.377" LastActivityDate="2018-03-03T13:25:55.020" Title="How to test if my implementation of back propagation neural Network is correct" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2729" PostTypeId="1" AcceptedAnswerId="2739" CreationDate="2017-01-27T14:20:54.040" Score="4" ViewCount="154" Body="&lt;p&gt;If I have 2 statements, say &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, from which I formed 2 formulae:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f_1: (\lnot A) \land (\lnot B)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f_2: (\lnot A) \lor (\lnot B)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Are &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; equivalent?&lt;/p&gt;&#xA;" OwnerUserId="3742" LastEditorUserId="2444" LastEditDate="2021-01-21T19:19:21.170" LastActivityDate="2021-01-22T12:22:56.677" Title="Given statements $A$ and $B$, are the formulae $(\lnot A) \land (\lnot B)$ and $(\lnot A) \lor (\lnot B)$ equivalent?" Tags="&lt;logic&gt;&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2730" PostTypeId="2" ParentId="2644" CreationDate="2017-01-27T19:57:59.897" Score="-1" Body="&lt;p&gt;It's not possible as this is the distinction between AI and humans, truly science will never understand the subconscious it's that little black box that no one can reverse engineer. This is why pursuing singularity is a fools dream to the extreme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why machinery lacks this because of the lack thereof a soul. science cannot produce a soul, this is why a machine cannot be self aware we can program fancy algorithms all day that mimic things but it's emotionless it cannot sit in judgement because it lacks real self awareness that is human self awareness it's like trying to make an orange into an apple. &lt;/p&gt;&#xA;" OwnerUserId="4315" LastEditorUserId="8" LastEditDate="2017-01-28T14:07:19.367" LastActivityDate="2017-01-28T14:07:19.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2731" PostTypeId="1" AcceptedAnswerId="2767" CreationDate="2017-01-27T21:11:45.137" Score="3" ViewCount="155" Body="&lt;p&gt;By English language robots I mean something like this: &lt;a href=&quot;http://www.tolearnenglish.com/free/celebs/audreyg.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.tolearnenglish.com/free/celebs/audreyg.php&lt;/a&gt;&#xA;I don't know what they called exactly, but interested to know how they work and how can I build something like them? and what subject should I look for it?&lt;/p&gt;&#xA;" OwnerUserId="2557" LastEditorUserId="4302" LastEditDate="2018-10-08T12:30:14.483" LastActivityDate="2018-10-08T12:30:14.483" Title="How do English language robots work?" Tags="&lt;natural-language-processing&gt;&lt;robots&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2732" PostTypeId="2" ParentId="2727" CreationDate="2017-01-27T22:54:01.730" Score="4" Body="&lt;p&gt;Actually the implementation was correct, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source of the problem that causes a big error and really slow learning was the architecture of the neural network it self, the ANN has 7 hidden layers which causes the vanishing gradient problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I have decreased the ANN layers to 3 the cost of error was widely reduced besides of that the learning process was faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another common solution is to use RELU or ELU or SELU in the hidden layers of the neural network instead of the sigmoid function&lt;/p&gt;&#xA;" OwnerUserId="5054" LastEditorUserId="5054" LastEditDate="2018-01-27T13:37:15.057" LastActivityDate="2018-01-27T13:37:15.057" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2733" PostTypeId="1" CreationDate="2017-01-28T06:46:10.700" Score="2" ViewCount="1498" Body="&lt;p&gt;In reinforcement learning, policy improvement is a part of an algorithm called policy iteration, which attempts to find approximate solutions to the Bellman optimality equations.&lt;/p&gt;&#xA;&lt;p&gt;Pages 84 and 85 in Sutton and Barto's &lt;a href=&quot;https://webdocs.cs.ualberta.ca/%7Esutton/book/bookdraft2016sep.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; on RL mentions the following theorem:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Policy Improvement Theorem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given two deterministic policies &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) \leq q_\pi(s, \pi'(s)), \forall s \in S.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the set of all states.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the the right-hand side of the inequality, the agent acts according to policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; (given that &lt;span class=&quot;math-container&quot;&gt;$\pi'(s)$&lt;/span&gt; is used in the inequality), in the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and for all subsequent states acts according to policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the left-hand side of the inequality, the agent acts according to policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (hence the subscript &lt;span class=&quot;math-container&quot;&gt;$_\pi$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;), starting from the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The claim is the following&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) \leq v_{\pi'}(s), \forall s \in S$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In other words, &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; is is an improvement over &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, I have a difficulty in understanding the proof. This is discussed below.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Proof&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) \leq q_\pi(s, \pi'(s)) = \mathbb{E}_{\pi'}[R_{t+1} + \gamma v_\pi(S_{t+1}) \mid S_t = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am stuck here. The q-function is evaluated over the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (note the subscript &lt;span class=&quot;math-container&quot;&gt;$_\pi$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, \pi'(s))$&lt;/span&gt;). That being the case, &lt;em&gt;how is the expectation over the policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;My guess is the following. In the proof given in Sutton and Barto, the expectation is unrolled in time. At each time step, the agent follows the policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;  for that particular time step, and then follows &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; from then on. In the limit of this process, the policy transforms from &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;. As long as the expression for the return inside the expectation is finite, the governing policy should be &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;; only in the limit of this process does the governing policy transform to &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5082" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-11-11T20:39:33.820" Title="Understanding why the expectation is over the new policy $\pi'$ in the proof of the Policy Improvement Theorem" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2734" PostTypeId="2" ParentId="2251" CreationDate="2017-01-28T08:06:51.220" Score="4" Body="&lt;p&gt;One example might be self-play in games. Since neural networks and deep learning depend on massive amounts of data, one way to generate data is to have two virtual machines play each other and record the experience. An example discussion can be found at &lt;a href=&quot;http://www.cs.cornell.edu/boom/2001sp/Tsinteris/gammon.htm&quot;&gt;http://www.cs.cornell.edu/boom/2001sp/Tsinteris/gammon.htm&lt;/a&gt; which uses reinforcement learning. I believe &lt;a href=&quot;https://deepmind.com/research/alphago/&quot;&gt;AlphaGo&lt;/a&gt; also uses this technique of self-play, and uses two independent neural networks, one reducing the search space and the other deciding on the best move in the remaining space, that in a sense cooperate to decide on the next move.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2017-01-28T08:06:51.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2735" PostTypeId="2" ParentId="2722" CreationDate="2017-01-28T08:46:11.060" Score="2" Body="&lt;p&gt;As long as the output of the AI affects the world, the way in which it communicates makes no fundamental difference to the control problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI might still be able, for example, to manoeuvre mankind into a situation, in which only the AI can save us. It might provide a seemingly inoccuous technological solution to global warming, but 50 years later it turns out that this solution caused some problem that threatens to wipe out humanity in the very short term. Suddenly, mankind is in a very bad negotiating position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the more powerless the AI starts out, the longer this kind of scenario will take, but the premise of superintelligence is, that we cannot rule out hidden long term agendas behind even a few bits of output. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-01-28T08:46:11.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2738" PostTypeId="1" CreationDate="2017-01-28T18:27:08.343" Score="11" ViewCount="8816" Body="&lt;p&gt;How is Bayes' Theorem used in artificial intelligence and machine learning?&lt;/p&gt;&#xA;&lt;p&gt;As a high school student, I will be writing an essay about it, and I want to be able to explain Bayes' Theorem, its general use, and how it is used in AI or ML.&lt;/p&gt;&#xA;" OwnerUserId="5088" LastEditorUserId="2444" LastEditDate="2021-02-08T23:57:59.800" LastActivityDate="2021-02-09T11:12:51.393" Title="How is Bayes' Theorem used in artificial intelligence and machine learning?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;bayes-theorem&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2739" PostTypeId="2" ParentId="2729" CreationDate="2017-01-28T19:59:37.583" Score="0" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f_1 \vdash f_2$&lt;/span&gt;, if and only if &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; must be true if we assume &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; to be true.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, &lt;span class=&quot;math-container&quot;&gt;$f_2 \vdash  f_1$&lt;/span&gt;, if and only if &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; must be true if we assume &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; to be true.&lt;/p&gt;&#xA;&lt;p&gt;Logically, by taking any value for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, from the domain &lt;span class=&quot;math-container&quot;&gt;$\{1, 0 \}$&lt;/span&gt;, one could verify that &lt;span class=&quot;math-container&quot;&gt;$f_1 \vdash f_2$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; is true whenever &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; is true (for example, when both &lt;span class=&quot;math-container&quot;&gt;$A = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B = 0$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;However, &lt;span class=&quot;math-container&quot;&gt;$f_2 \vdash  f_1$&lt;/span&gt; is not true. As, in two cases, &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; is true (e.g. &lt;span class=&quot;math-container&quot;&gt;$A = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B=1$&lt;/span&gt;, or vice-versa), but &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; is not true.&lt;/p&gt;&#xA;" OwnerUserId="3742" LastEditorUserId="21645" LastEditDate="2021-01-21T21:06:40.843" LastActivityDate="2021-01-21T21:06:40.843" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2741" PostTypeId="2" ParentId="2738" CreationDate="2017-01-29T11:59:17.093" Score="1" Body="&lt;p&gt;Since you are a highschool student I will try to express it easier. It is a problem for a machine to make a decision if you haven't given that information to it before. You should think of every cases while programming. But sometimes there can be so many cases, here Data Mining, Neural Networks, Fuzzy Logic etc are used withing AI. It saves your time and system is learning itself with enough examples given at the beginning and deciding itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.cs.hacettepe.edu.tr/~ilyas/Courses/BIL712/lec04-BayesianLearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here in this link&lt;/a&gt; you can find an article about Bayesian learning. Example on p.33 is what you need I guess.&lt;/p&gt;&#xA;" OwnerUserId="3358" LastEditorUserId="5095" LastEditDate="2017-01-31T12:27:06.137" LastActivityDate="2017-01-31T12:27:06.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2742" PostTypeId="1" AcceptedAnswerId="4319" CreationDate="2017-01-29T19:12:51.067" Score="8" ViewCount="2693" Body="&lt;p&gt;Usually when performing linear regression predictions and gradient descent, the measure of the level of error for a particular line will be measured by the sum of the squared-distance values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why distance &lt;strong&gt;&lt;em&gt;squared&lt;/em&gt;&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most of the explanations I heard, they claim that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the function itself does not matter&lt;/li&gt;&#xA;&lt;li&gt;the result should be positive so positive and negative deviations are still counted&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, an &lt;code&gt;abs()&lt;/code&gt; approach would still work. And isn't it inconvenient that distance &lt;em&gt;squared&lt;/em&gt; minimizes the distance result for distances lower than 1?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm pretty sure someone must have considered this already -- so why is distance squared the most used approach to linear regression?&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2017-10-25T21:34:24.573" Title="Linear regression: why is distance *squared* used as an error metric?" Tags="&lt;linear-regression&gt;" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2743" PostTypeId="1" AcceptedAnswerId="2745" CreationDate="2017-01-29T22:11:30.593" Score="8" ViewCount="163" Body="&lt;p&gt;I have a simulator modelling a relatively complex scenario. I extract ~12 discrete features from the simulator state which forms the basis for my MDP state space.&lt;/p&gt;&#xA;&lt;p&gt;Suppose I am estimating the transition table for an MDP by running a large number of simulations and extracting feature transitions as the state transitions.&lt;/p&gt;&#xA;&lt;p&gt;While I can randomize the simulator starting conditions to increase the coverage of states, I cannot guarantee all states will be represented in the sample, i.e. states which are possible but rare.&lt;/p&gt;&#xA;&lt;p&gt;Is there a rigorous approach to &amp;quot;filling in the gaps&amp;quot; of the transition table in this case?&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For each state that was unrepresented in the sample, simply transition to all other states with equal probability, as a &amp;quot;neutral&amp;quot; way to fill in the gap?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As above, but transition only to represented states (with equal probability)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transition to the same state with probability 1.0?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ignore unrepresented states during MDP solving entirely, and simply have a default action specified?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4402" LastEditorUserId="2444" LastEditDate="2021-01-31T14:30:33.983" LastActivityDate="2021-01-31T14:31:28.580" Title="How to fill in missing transitions when sampling an MDP transition table?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;monte-carlo-methods&gt;&lt;transition-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2744" PostTypeId="2" ParentId="41" CreationDate="2017-01-29T23:59:30.203" Score="0" Body="&lt;p&gt;All these questions are fully covered in the book &lt;em&gt;&lt;a href=&quot;https://www.cambridge.org/us/academic/subjects/computer-science/artificial-intelligence-and-natural-language-processing/measure-all-minds-evaluating-natural-and-artificial-intelligence?format=HB&amp;amp;isbn=9781107153011&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Measure of All Minds&#xA;Evaluating Natural and Artificial Intelligence&lt;/a&gt;&lt;/em&gt; (Orallo, 2017) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excerpt from the description give a good overview:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By replacing the dominant anthropocentric stance with a universal perspective where living organisms are considered as a special case, long-standing questions in the evaluation of behavior can be addressed in a wider landscape. Can we derive task difficulty intrinsically? Is a universal g factor - a common general component for all abilities - theoretically possible? Using algorithmic information theory as a foundation, the book elaborates on the evaluation of perceptual, developmental, social, verbal and collective features,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5109" LastEditorUserId="1671" LastEditDate="2019-06-07T19:54:42.090" LastActivityDate="2019-06-07T19:54:42.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2745" PostTypeId="2" ParentId="2743" CreationDate="2017-01-30T08:31:15.947" Score="1" Body="&lt;p&gt;I assume you use the 12 discrete features as state variables, and, for each of these variables, you will have at least two values. So, the minimum number of states will be &lt;span class=&quot;math-container&quot;&gt;$2^{12} = 4096$&lt;/span&gt;, which gives &lt;span class=&quot;math-container&quot;&gt;$(2^{12})^{2} = 16777216$&lt;/span&gt; possible transitions. In order to reach this, you will need a huge amount of simulations, also taking into account that this number is a minimum since you might have more values per state variable, and you probably have more than one action per state transition.&lt;/p&gt;&#xA;&lt;p&gt;How to fill the gaps depends on your problem, in a problem where I did this, I filled the gaps using a uniformly random transition to its neighbor states. However, since I had to fill in such a a large amount, there was no significant difference between using this estimated transitions probabilities with a predefined transition table.&lt;/p&gt;&#xA;&lt;p&gt;In your case, it might be better to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Q-Learning&lt;/strong&gt;&lt;/a&gt;, which is a model-free method that does not require the transition probabilities and uses directly the rewards and states obtained.&lt;/p&gt;&#xA;" OwnerUserId="198" LastEditorUserId="2444" LastEditDate="2021-01-31T14:31:28.580" LastActivityDate="2021-01-31T14:31:28.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2747" PostTypeId="2" ParentId="2742" CreationDate="2017-01-30T12:37:06.790" Score="4" Body="&lt;p&gt;The squared form is sometimes called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Norm_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euclidean norm or L2 norm&lt;/a&gt;. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and translated fairly easily into code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively it is thought that it is advantageous to exaggerate the differences according to the value of the error, which squaring does. You might also use the powers 3 or 4, but the derivative is more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A number of different norms may be used, according to the particular circumstances of the problem at hand.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2017-01-30T12:37:06.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2748" PostTypeId="2" ParentId="2248" CreationDate="2017-01-30T12:52:25.903" Score="5" Body="&lt;p&gt;A &lt;strong&gt;closed expression&lt;/strong&gt; refers to a formula which has no free variables [&lt;a href=&quot;https://en.wikipedia.org/wiki/Closed-form_expression&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. This is also called &lt;strong&gt;sentence&lt;/strong&gt;. In a logic system you have a set of axioms which are sentences and rules which state how to derive a sentence from this [&lt;a href=&quot;https://en.wikipedia.org/wiki/Hilbert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]. If a sentence can be derived from the axioms, this means that the axioms entail this sentence. If a sentence is not derivable, it is not entailed by the axioms. &lt;/p&gt;&#xA;" OwnerUserId="5095" LastEditorUserId="5095" LastEditDate="2017-02-26T12:32:33.263" LastActivityDate="2017-02-26T12:32:33.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2749" PostTypeId="2" ParentId="2742" CreationDate="2017-01-30T13:09:40.253" Score="2" Body="&lt;p&gt;One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally distributed means that the likelihood of the data is inversely proportional the exponential of the &lt;strong&gt;square&lt;/strong&gt; of the distance to the mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, minimizing the sum of squares of the distance to the mean amounts to finding the most likely value for the line assuming that the error is normally distributed. This is very often a reasonable assumption, but it is of course not always true.&lt;/p&gt;&#xA;" OwnerUserId="5118" LastActivityDate="2017-01-30T13:09:40.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2750" PostTypeId="2" ParentId="2742" CreationDate="2017-01-30T13:38:14.660" Score="1" Body="&lt;p&gt;One justification is that under homoscedasticity the L2 norm produces the minimum variance unbiased estimator (MVUE), see Gauss-Markov Theorem. It means that the fitted values are the conditional expectations given the explanatory variables which is in many cases a nice property. Further it is the best estimator if the previous property is desirable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a response to the claim that the function itself does not matter, different functions give solutions with very different properties and a lot of effort has gone in to finding appropriate penalty functions, see for example Ridge regression and LASSO. The penalty function does matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: In response to your question regarding distances lower than 1, nothing &quot;goes wrong&quot; when the distances are smaller than 1. We always want to minimize the distance and the squared loss does so everywhere.&lt;/p&gt;&#xA;" OwnerUserId="3131" LastEditorUserId="3131" LastEditDate="2017-01-30T13:50:04.990" LastActivityDate="2017-01-30T13:50:04.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2751" PostTypeId="2" ParentId="2689" CreationDate="2017-01-30T14:50:09.480" Score="4" Body="&lt;p&gt;NEAT has a constant number of organisms in its population, which prevents overpopulation from happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process of mating includes the following steps. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The worst networks from every species are removed. &lt;/li&gt;&#xA;&lt;li&gt;All species receive a number of offsprings that they can have. This is calculated by an adjusted neural network fitness.&lt;/li&gt;&#xA;&lt;li&gt;Offsprings for species are divided among neural networks in those species. Fitter neural networks have more offsprings.&lt;/li&gt;&#xA;&lt;li&gt;Networks from the same species are combined and create an offspring.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Speciation&lt;/em&gt; prevents fast extinction.&lt;/p&gt;&#xA;" OwnerUserId="5119" LastEditorUserId="2444" LastEditDate="2019-07-07T19:04:27.113" LastActivityDate="2019-07-07T19:04:27.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2753" PostTypeId="2" ParentId="2742" CreationDate="2017-01-30T18:04:15.223" Score="2" Body="&lt;p&gt;It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: &lt;a href=&quot;http://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/06/lecture-06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Method of Maximum Likelihood for Simple Linear Regression&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5122" LastEditorUserId="8" LastEditDate="2017-02-12T14:18:49.787" LastActivityDate="2017-02-12T14:18:49.787" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2757" PostTypeId="2" ParentId="2462" CreationDate="2017-01-31T09:38:09.007" Score="1" Body="&lt;p&gt;A human has an abstract concept of numbers in mind. So 456 is a unique entity which is by definition unlike any other number because that are other unique entities. If you give ∃x ∈ ℕ: x==123 to your system it could check the property of natural numbers by counting from 0 to 123 to conclude that the statement is true. A human does it in another way. A human would &quot;see&quot; that it is a natural number because it has no decimal point. Because of the concept of natural numbers the statement is immediately clear. To get a faster result here the machine could check just the decimal point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your second case you have to apply the commutative property of addition and you are done because the syntax is equal then.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second problem is more syntactic while the first is semantic. Your machine may &quot;know&quot; the commutative property of addition but not the concept of natural numbers. Therefore, it has to count.&lt;/p&gt;&#xA;" OwnerUserId="5095" LastActivityDate="2017-01-31T09:38:09.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2760" PostTypeId="1" AcceptedAnswerId="2765" CreationDate="2017-01-31T16:16:39.473" Score="4" ViewCount="159" Body="&lt;p&gt;I have a task on my class to find all the nodes, calculate their values and choose the best way for the player on the given game graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m5MRv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m5MRv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything is fine, but I have no idea what these dots are. Is this a third player, or just a 'split' for player1 move? Some kind of heuristics?&lt;/p&gt;&#xA;" OwnerUserId="3617" LastEditorUserId="2444" LastEditDate="2019-10-15T20:17:54.100" LastActivityDate="2019-10-15T20:17:54.100" Title="What is the variant of the minimax tree with 3 types of nodes called?" Tags="&lt;minimax&gt;&lt;expectiminimax&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2762" PostTypeId="1" CreationDate="2017-01-31T16:38:30.407" Score="7" ViewCount="1467" Body="&lt;p&gt;In classical set theory there is two options for an element. It is either a member of a set, or not. But in fuzzy set theory there are &lt;strong&gt;membership functions&lt;/strong&gt; to define &quot;rate&quot; of an element being a member of a set. In other words, classical logic says it is all black or white, but fuzzy logic offers that there is also grey which has shades between white and black.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab Simulink Library is very easy to design and helpful in practice. And it has good examples on its own like deciding about tip for a dinner looking at service and food quality. In the figure below some various membership functions from Matlab's library are shown:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aWG0C.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aWG0C.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; How do we decide about choosing membership functions while designing a fuzzy controller system? I mean in general, not only in Matlab Simulink. I have seen &lt;em&gt;Triangular&lt;/em&gt; and &lt;em&gt;Gaussian&lt;/em&gt; functions are used mostly in practise, but how can we decide which function will give a better result for decision making? Do we need to train a neural network to decide which function is better depending on problem and its rules? What are other solutions?&lt;/p&gt;&#xA;" OwnerUserId="3358" LastEditorUserId="3358" LastEditDate="2017-02-01T00:37:02.437" LastActivityDate="2018-07-31T15:01:00.503" Title="Fuzzy Logic Controller: Choosing Membership Function" Tags="&lt;neural-networks&gt;&lt;fuzzy-logic&gt;" AnswerCount="1" CommentCount="9" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2763" PostTypeId="2" ParentId="111" CreationDate="2017-01-31T22:16:32.373" Score="7" Body="&lt;p&gt;How could self-driving cars make ethical decisions about who to kill?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;By managing legal liability and consumer safety.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A car that offers the consumer safety is going to be a car that is bought by said consumers. Companies do not want to be liable for killing their customers nor do they want to sell a product that gets the user in legal predicaments. Legal liability and consumer safety are the same issue when looked at from the perspective of &amp;quot;cost to consumer&amp;quot;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And here are few dilemmas:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the algorithm recognize the difference between a human being and&#xA;an animal?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If an animal/human cannot be legally avoided (and the car is in legal right - if it is not then something else is wrong with the AI's decision making), it likely won't. If the car can safely avoid the obstacle, the AI could reasonably be seen to make this decision, ie. swerve to another lane on an open highway. Notice there is an emphasis on liability and driver safety.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the size of the human being or animal matter?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Only the risk factor from hitting the obstacle. Hitting a hippo might be less desirable than hitting the ditch. Hitting a dog is likely more desirable than wrecking the customer's automobile.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it count how many passengers it has vs. people in the front?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It counts the people as passengers to see if the car-pooling lane can be taken. It counts the people in front as a risk factor in case of a collision.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it &amp;quot;know&amp;quot; when babies/children are on board?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it take into the account the age (e.g. killing the older first)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. This is simply the wrong abstraction to make a decision, how could this be weighted into choosing the right course of action to reduce risk factor? If Option 1 is hit young guy with 20% chance of significant occupant damage and no legal liability and Option 2 is hit an old guy with 21% chance of significant occupant damage and no legal liability, then what philosopher can convince even just 1 person of the just and equitable weights to make a decision?&lt;/p&gt;&#xA;&lt;p&gt;Thankfully, the best decision a lot of the time is to hit the breaks to reduce speed (especially when you consider that it is often valuable to act predictably so that pedestrians and motorists can react accordingly). In the meantime, better value improvements can be made in terms of predicting when drivers will make bad decisions and when other actions (such as hitting the reverse) are more beneficial than hitting the breaks. At this point, it is not worth it to even begin collecting the information to make the ethical decisions proposed by philosophers. Thus, this issue is over-hyped by sensational journalists and philosophers.&lt;/p&gt;&#xA;" OwnerUserId="5157" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-04-14T01:44:34.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2764" PostTypeId="2" ParentId="2462" CreationDate="2017-01-31T23:21:16.493" Score="1" Body="&lt;p&gt;You need some sort of interpretation abstraction before your mathematical reasoning. While the text might read &quot;123&quot;, you need to parse this into a &lt;em&gt;literal&lt;/em&gt; of type &lt;em&gt;Natural Number&lt;/em&gt; or &lt;em&gt;Integer&lt;/em&gt;. Similarly, &quot;x&quot; could be a &lt;em&gt;member variable&lt;/em&gt;. Then your deduction becomes, is literal 123 a Natural Number? Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the second statement, you should hopefully be able to reason to definitely false. Your internal representation of a sum should not be order dependent since addition is commutative. Then, the check for equality must handle this unordered property.&lt;/p&gt;&#xA;" OwnerUserId="5157" LastActivityDate="2017-01-31T23:21:16.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2765" PostTypeId="2" ParentId="2760" CreationDate="2017-02-01T07:30:43.353" Score="4" Body="&lt;p&gt;The triangles pointing up are Max' nodes. We assume it starts. Then follows a random choice of moves at the circles, for instance, with a die. The triangles pointing down are from Min. This variant is called Expectiminimax, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectiminimax_tree&quot;&gt;https://en.wikipedia.org/wiki/Expectiminimax_tree&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At that circles you have to multiply the possibilities on the edges below that nodes to your current value and sum all products up. The circles in your picture mean that Min dices.&lt;/p&gt;&#xA;" OwnerUserId="5095" LastEditorUserId="5095" LastEditDate="2017-02-01T07:46:02.887" LastActivityDate="2017-02-01T07:46:02.887" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2766" PostTypeId="2" ParentId="2706" CreationDate="2017-02-01T09:20:37.940" Score="0" Body="&lt;p&gt;It is so popular because Turing formulated it. He was one of the first who talked about &quot;intelligent machines&quot; and was good connected in the scientific community since the 1940s. So there was enough time to distribute his very intelligent thoughts, for instance by Von Neumann, until now. Turing's importance for computer science is shown by the name of the Turing Award. So it is clear that a lot of people have read his papers.&lt;/p&gt;&#xA;" OwnerUserId="5095" LastActivityDate="2017-02-01T09:20:37.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2767" PostTypeId="2" ParentId="2731" CreationDate="2017-02-01T09:53:33.307" Score="4" Body="&lt;p&gt;English language robots which you mentioned, are called &quot;&lt;strong&gt;&lt;em&gt;chatter bots&lt;/em&gt;&lt;/strong&gt;&quot;. Chatter Bots are used to communicate with a human and undergo conversations in such a way that the human which is communicating will think that he/she is talking to another human. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two types of chatter bots: One is which uses certain rules and pattern matching techniques and the other one is which uses actual artificial intelligence techniques. Of course, the latter one is the most difficult to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be understood with an example. The former type of bots which uses pattern matching and rules consist of questions stored in the form,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if question matches 'WHO (IS/ARE) (MEMBERS OF PARLIAMENT/PRESIDENT)?' then&#xA;     RETURN Y ... where Y is some predefined answer.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The latter part which uses actual AI techniques, uses various mechanisms to actually understand the question, extract information out of it, process the information into some standard normalized form and then do some &lt;strong&gt;inference&lt;/strong&gt; w.r.t. the &lt;em&gt;facts&lt;/em&gt; in the &lt;strong&gt;knowledge base&lt;/strong&gt;. Such methods may use &lt;em&gt;learning&lt;/em&gt; algorithms to actually learn question patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make simple bots (probably of the former type), you can use Artificial Intelligence Markup Language (&lt;a href=&quot;http://www.alicebot.org/aiml.html&quot; rel=&quot;noreferrer&quot;&gt;AIML&lt;/a&gt;) which is a XML based language for developing chatter bots like &lt;a href=&quot;http://alice.pandorabots.com/&quot; rel=&quot;noreferrer&quot;&gt;ALICE&lt;/a&gt; bot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to focus more towards latter type of chatter bots, you may have to learn about various AI techniques, like searching, logic, knowledge and inference, learning, etc or sub-domains of AI like Natural Language Processing (NLP). There are various tool kits for NLP like for developing in Python there is nltk, for developing in Java there is Stanford's CoreNLP, and so on.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastActivityDate="2017-02-01T09:53:33.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2768" PostTypeId="2" ParentId="2738" CreationDate="2017-02-01T18:09:18.703" Score="5" Body="&lt;p&gt;Bayes theorem states the probability of some event B occurring provided the prior knowledge of another event(s) A, given that B is dependent on event A (even partially).&lt;br&gt;&#xA;A real-world application example will be weather forecasting. Naive Bayes is a powerful algorithm for predictive modelling weather forecast. The temperature of a place is dependent on the pressure at that place, percentage of the humidity, speed and direction of the wind, previous records on temperature, turbulence on different atmospheric layers, and many other things. So when you have certain kind of data, you process them certain kind of algorithms to predict one particular result (or the future). The algorithms employed rely heavily on Bayesian network and the theorem.&lt;br&gt;&lt;br&gt;&#xA;The given paragraph is introduction to Bayesian networks, given in the book, Artificial Intelligence – A Modern Approach:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Bayesian network formalism was invented to allow efficient representation of, and rigorous reasoning with, uncertain knowledge. This approach largely overcomes many problems of the probabilistic reasoning systems to the 1960s and 70s; it now dominates AI research on uncertain reasoning and expert systems. The approach allows for learning from experience, and it combines the best of classical AI and neural nets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;There are many other applications, especially in medical science. Like predicting a particular disease based on the symptoms and physical condition of the patient. There are many algorithms currently in use that are based on this theorem, like binary and multi-class classifier, for example, email spam filters.&#xA;There are many things in this topic.I have added some links below that might help, and let me know if you need any kind of other help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Helpful Links&lt;br&gt;&#xA; 1. &lt;a href=&quot;http://machinelearningmastery.com/naive-bayes-for-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;First&lt;/a&gt;&lt;br&gt;&#xA; 2. &lt;a href=&quot;https://en.wikipedia.org/wiki/Naive_Bayes_classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;Second&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="9857" LastEditDate="2017-11-13T19:46:06.463" LastActivityDate="2017-11-13T19:46:06.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2771" PostTypeId="1" CreationDate="2017-02-02T06:41:08.190" Score="8" ViewCount="547" Body="&lt;p&gt;I define Artificial Life as a &amp;quot;simulation&amp;quot; or &amp;quot;copy&amp;quot; of life. However, should it be considered a simulation or copy?&lt;/p&gt;&#xA;&lt;p&gt;If one had motivation and money, someone could theoretically create evolving computers, with a program that allows mutation OR simply a &amp;quot;simulated&amp;quot; environment with &amp;quot;simulated&amp;quot; organisms.&lt;/p&gt;&#xA;&lt;p&gt;The computer (or &amp;quot;simulated&amp;quot; organism) would have the ability to reproduce, grow, and take in energy. What if the life evolved to have intelligence? Currently, there are some relatively limited programs that simulate life, but most of them are heavily simplistic. Are they life?&lt;/p&gt;&#xA;&lt;p&gt;When should something be called life?&lt;/p&gt;&#xA;" OwnerUserId="5189" LastEditorUserId="2444" LastEditDate="2020-06-25T12:53:20.937" LastActivityDate="2020-06-25T12:53:20.937" Title="Is artificial life really life or not?" Tags="&lt;philosophy&gt;&lt;genetic-algorithms&gt;&lt;biology&gt;&lt;artificial-life&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2772" PostTypeId="1" CreationDate="2017-02-02T10:43:02.673" Score="8" ViewCount="225" Body="&lt;p&gt;Could you give examples of affordable programmable devices that could be used in university classes to teach students about A.I. and demonstrate it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The devices are expected to do some form of self-learning, pattern recognition, or any other features of A.I., and to be programmable or customizable.&lt;/p&gt;&#xA;" OwnerUserId="5191" LastEditorUserId="2444" LastEditDate="2019-06-25T16:27:20.607" LastActivityDate="2019-06-25T16:27:20.607" Title="What programmable devices can be used to teach artificial intelligence?" Tags="&lt;programming-languages&gt;&lt;education&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2774" PostTypeId="2" ParentId="2771" CreationDate="2017-02-02T15:37:00.297" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Life&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia describes life&lt;/a&gt; as a characteristic of &quot;physical entities having biological processes&quot;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;The same source&lt;/a&gt; also describes a simulation as &quot;the imitation of the operation of a real-world process or system over time.&quot; If a digital neural net was to listen to me prattle on for long enough it could learn to speak as if it were me. It would have my knowledge and limitations but its headaches would be quite different from mine. It would never have a toothache. But you could put it in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Searle Chinese Room&lt;/a&gt;, and you could speak to it and it would sound exactly as if it were me long after I am dead. It has my &quot;character&quot; which is what my friends would recognize about me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the definition of life it is not alive because it does not have biological processes. It is a simulation because it emulates what I would have said. It cannot be a copy because a digital box is not biologic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's give this simulation a biological nose so that it can smell. And maybe two eyes and ears. We continue this process until most of the simulation is equipped with biological parts which function together. Whatever it is is now able to come out of the Chinese Room and talk to you. By golly, it looks and sounds exactly like me, but I died a long time ago. Have I been brought back?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is that a perfect copy of me would not be possible simply through training due to the level of detail required, but that the close copy would be alive. A critical point would be that there would have to be a fatal link somewhere which would cause &quot;death&quot;. You could always create a new close copy, but not an exact copy.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastEditorUserId="4994" LastEditDate="2017-02-02T15:42:40.503" LastActivityDate="2017-02-02T15:42:40.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2775" PostTypeId="2" ParentId="2771" CreationDate="2017-02-03T08:48:26.263" Score="1" Body="&lt;p&gt;It wouldn't be considered alive if it doesn't have vital functions, such as nutrition, relation with the environment, and reproduction. While the first is easy (use a battery) and the second is the one we are developing right now (basically, the intelligence part of an AI) giving programming skills to an AI, aka the ability to reproduce, isn't widely considered a good idea, as many science fiction writers can tell you.&lt;/p&gt;&#xA;" OwnerUserId="5211" LastEditorUserId="2444" LastEditDate="2020-06-25T12:45:34.867" LastActivityDate="2020-06-25T12:45:34.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2776" PostTypeId="1" CreationDate="2017-02-03T10:41:34.517" Score="5" ViewCount="105" Body="&lt;p&gt;Let's say I have a neural network with 5 layers, including the input and output layer. Each layer has 5 nodes. Assume the layers are fully connected, but the 3rd node in the 2nd layer is connected to the 5th node in the 4th layer. All these numbers are chosen at random for the example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When is the 5th node in the 4th layer fed forward? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's go through it step by step. The first layer is normally fed forward to the second. The second layer is normally fed forward to the third, but the 3rd node is also fed forward to the 5th node of the 4th layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, is the 5th node in the 4th layer now fed forward, or is it fed forward when the 3rd layer is done being fed forward? The 1st method would mean that the node would get fed forward 2 times and my concern is, if the output is still valid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, it would also come to 2 asynchronous outputs and how would these be interpreted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because in the brain, I heard, the neurons are fired when an impulse arrives so this would equal the 1st method.&lt;/p&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="2444" LastEditDate="2020-03-07T06:30:30.067" LastActivityDate="2020-03-07T06:30:30.067" Title="Are neurons instantly feed forward when input arrives?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2777" PostTypeId="1" CreationDate="2017-02-03T16:14:09.547" Score="8" ViewCount="421" Body="&lt;p&gt;I am researching Natural Language Processing (NLP) to develop an NL Question Answering system. The answering part is already done. So processing the question remains, along with the questions regarding the algorithms.&lt;/p&gt;&#xA;&lt;p&gt;The final product should allow the user to ask a question in NL. The question then gets translated to an &lt;a href=&quot;https://en.wikipedia.org/wiki/MultiDimensional_eXpressions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multidimensional Expressions (MDX) query&lt;/a&gt;, which generates a script regarding the dimensions of the cube.&lt;/p&gt;&#xA;&lt;p&gt;How can I translate a natural language question to an MDX query?&lt;/p&gt;&#xA;&lt;p&gt;The outcome of the question is in the form of a calculation. For example&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How many declarations were done by employee 1?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Give me the quantities for Sales.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5219" LastEditorUserId="2444" LastEditDate="2021-01-26T15:28:18.693" LastActivityDate="2021-01-26T15:28:18.693" Title="How can I translate a natural language question to an MDX query?" Tags="&lt;natural-language-processing&gt;&lt;question-answering&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2779" PostTypeId="2" ParentId="2771" CreationDate="2017-02-03T18:54:22.863" Score="2" Body="&lt;p&gt;I like to take an &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Animism&quot; rel=&quot;nofollow noreferrer&quot;&gt;animist&lt;/a&gt;&quot; approach.  &lt;em&gt;(It has been suggested to me that part of the reason Japanese designs are so effective is because of the cultural affinity for the concept per the Shinto tradition.  For instance, the thing where people put little eyes on everything;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like to think of how my dog, who is terrified of the vacuum cleaner, would regard one of the recent &lt;a href=&quot;http://www.theverge.com/circuitbreaker/2017/2/1/14468126/boston-dynamics-new-wheeled-robot-handle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boston Dynamics&lt;/a&gt; creations.  My guess is the dog would't find much use in the distinction that the robot is an artifact as opposed to a biological entity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to take a deterministic, mechanical approach to reality. Sure things get fuzzy down at the quantum level, but even that may simply be a factor of inadequate measurement capability and the sheer complexity of quantum mechanics, which seems fundamentally beyond the grasp of even the greatest minds, when they're being honest about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see much of a distinction between simple organisms and &lt;a href=&quot;https://en.wikipedia.org/wiki/Cellular_automaton&quot; rel=&quot;nofollow noreferrer&quot;&gt;cellular automata&lt;/a&gt;, except that the former is part of a biological food chain.  There is a valid hypothesis that if you had a big enough computer, Conway's Game of Life could independently develop &quot;intelligence&quot;.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-replication would certainly seem to be a requirement of biological life that can be extended to artificial life.  Possibly the true distinction of &quot;artificial&quot; is merely that it is creation of a functional system by an extra-species source, whether the creation be &quot;biological&quot; or &quot;mechanical&quot; in nature. &lt;em&gt;(i.e. we can hack genes now, not just computer code.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-02-03T20:52:26.840" LastActivityDate="2017-02-03T20:52:26.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2780" PostTypeId="2" ParentId="2776" CreationDate="2017-02-03T20:02:22.513" Score="3" Body="&lt;p&gt;It is unclear what kind of network your are referring to, there is not a single neural-network model so conceivable both cases could exist and serve some purpose, yet if you are looking for one that emulates nature and real neurons, then you are missing at least 2 ingredients ( time and the mechanisms of resting potentials and refractory periods), which in turn introduce new computations to the neural network.&lt;/p&gt;&#xA;&lt;p&gt;This is your network graph if I got it right:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pI6Vi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pI6Vi.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The calculation in a neural network without refractory periods and resting potentials without time, would instantaneously modify the weight of your node4 layer 5 (n4-L5) if there is input in column 3:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OaEiX.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OaEiX.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additional inputs on other columns would just add up unless you have some other explicit computation on any layer.&lt;/p&gt;&#xA;&lt;p&gt;If you wanted to emulate a Neuron, each node would need to have a resting potential, that is: a level above which it will fire ( in the above example zero), and a refractory period:  a time before it would fire again, as well as a clock to keep it in sync, this would be a crude realtime fascimile:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://codepen.io/k3no/pen/WRQbYV&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://codepen.io/k3no/pen/WRQbYV&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Hj68.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Hj68.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A common alternative is to use sequential phases or steps.&lt;/p&gt;&#xA;&lt;h3&gt;Reference/source&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;, by S. Russell and P. Norvig. Deals with a general step approach to A.I.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mitpress.mit.edu/books/gateway-memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gateway to Memory&lt;/a&gt;, by Mark A. Gluck and Catherine E. Myers Presents a great and readable introduction to modeling neural networks.&lt;/p&gt;&#xA;&lt;p&gt;I published the little neural network model in a medium article : &lt;a href=&quot;https://medium.com/@k3no/memory-and-the-machine-ba380dcdb1c1#.3riz96ryx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memory and the machine&lt;/a&gt;, relevant sources are there.&lt;/p&gt;&#xA;" OwnerUserId="3020" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-02-04T19:48:10.500" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2781" PostTypeId="2" ParentId="2771" CreationDate="2017-02-03T22:44:52.170" Score="4" Body="&lt;p&gt;If you read Steven Levy's book, &lt;em&gt;Artificial Life&lt;/em&gt;,you will find, as I did, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/43k2B.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/43k2B.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the distinction between biological and &quot;artificial&quot; life blurred. If you think about it, &lt;em&gt;what exactly is &quot;life&quot;, anyway?&lt;/em&gt; &lt;strong&gt;A set of complex systems with emergent behavior capable of evolution and adaptation.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A prototypical biologist may not define life that way. Indeed, he would, &lt;em&gt;not&lt;/em&gt; being focused or concerned with the computational aspect, define it in a way that would narrow it down to biological life.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Marvin Minsky mention the concept of &lt;em&gt;luggage words&lt;/em&gt;, and I myself came up with the notion of &lt;em&gt;mirage concepts&lt;/em&gt;. For the former, that which we don't understand gets lump into the word. For the latter, when you take a &quot;mysterious&quot; concept apart, it vanishes like a mirage does as you get closer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;em&gt;what is life&lt;/em&gt;? If we look at a &quot;living&quot; organism, we'd all that &quot;life&quot;. If we remove a single cell from that organism, we'd still call that &quot;life&quot;. But what if we remove a single organelle like, say, a ribosome? Lysosome? Contractile vacuole? Endoplasmic reticulum? Is that still &quot;life&quot;? What if we remove a macromolecule from that? Is that still &quot;life&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you see, all becomes &lt;em&gt;very&lt;/em&gt; murky, and I do this on purpose to illustrate just how &lt;em&gt;arbitrary&lt;/em&gt; the very concept of &quot;life&quot; is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think my definition is a good one, broad enough to encompass both in-silico and the organic versions. It bespeaks to algorithms, robots, viruses -- yes &lt;em&gt;both&lt;/em&gt; computer and organic... anything that has &lt;em&gt;complexity&lt;/em&gt; and the ability to &lt;em&gt;adapt&lt;/em&gt; and &lt;em&gt;evolve&lt;/em&gt;. &lt;/p&gt;&#xA;" OwnerUserId="4185" LastActivityDate="2017-02-03T22:44:52.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2782" PostTypeId="2" ParentId="2777" CreationDate="2017-02-04T14:58:45.733" Score="5" Body="&lt;p&gt;You can use a component library which can help you to implement Natural language query builder in your application( &lt;em&gt;the question part&lt;/em&gt; ) called &lt;a href=&quot;https://opennlp.apache.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open Natural Language Processing Package&lt;/a&gt;&#xA; ,  so you can definitely develop a module, by using existing modules of OpenNLP such as entity extraction, chunking and parsing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Question_answering&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia source&lt;/a&gt;   ; it points out that as of 2001, &lt;em&gt;Q&amp;amp;A&lt;/em&gt; applications typically includes &quot;a question classifier module which determines the type of question and the type of answer,&quot; so &quot;a multiagent question and answering architecture has been proposed, where each domain [or variable] is represented by an agent which tries to answer questions, taking into account its specific knowledge.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it still need some effort to build a &lt;a href=&quot;https://deepblue.lib.umich.edu/bitstream/handle/2027.42/138709/lifei_1.pdf?sequence=1&amp;amp;isAllowed=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLgenerators inline with databases&lt;/a&gt; , for the answer query and also is the link to help you on how you can work with &lt;a href=&quot;https://nlp.stanford.edu/software/sempre/wikitable&quot; rel=&quot;nofollow noreferrer&quot;&gt;Compositional Semantic Parsing on Semi-Structured Tables&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can give you some insight.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="75" LastEditDate="2018-09-16T23:45:23.363" LastActivityDate="2018-09-16T23:45:23.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2783" PostTypeId="1" CreationDate="2017-02-04T16:41:02.400" Score="1" ViewCount="685" Body="&lt;p&gt;I'd like to build a program that would learn to automatically classify documents. The principle would be that, for each new document I add to the system, it would automatically infer in which category to classify the document. If it doesn't know, I would have to manually enter the category. For each hint I give to the system, the system would learn to refine its knowledge of document kinds. Something similar to face recognition in Picasa, but for documents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, the documents would be invoices, and I want to classify them by vendors. Documents could be extracted as text, as image, or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some know algorithms for this kind of job?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now, I could think at two possible ways I could do it:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For images, I could add all the images of a given kind together, and record the pixels that are the most common to all images, to create a mask. For a new image, I would compare this mask with the image to determine how similar it is.&lt;/li&gt;&#xA;&lt;li&gt;For text, I could record the list of words or sentences that are similar to all documents of a given kind.&lt;/li&gt;&#xA;&lt;li&gt;Finally, I could do a combination of both techniques, for example by converting a PDF document to an image, or an image to text by OCR techniques.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm just wondering if I'm approaching the problem the right way. Especially about storing just enough information in the database.&lt;/p&gt;&#xA;" OwnerUserId="5235" LastActivityDate="2017-04-13T04:50:10.547" Title="What algorithm should I use to classify documents?" Tags="&lt;algorithm&gt;&lt;reinforcement-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2787" PostTypeId="1" CreationDate="2017-02-06T09:32:07.333" Score="7" ViewCount="1399" Body="&lt;p&gt;I am currently working on my last project before graduating. For this project, I have to develop a Natural Language Question Answering System. Now, I have read quite some research papers regarding this topic and have figured out everything except for the parsing algorithm.&lt;/p&gt;&#xA;&lt;p&gt;The NL Q-A will be programmed in Python, and I will use the &lt;a href=&quot;https://spacy.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;spaCy library&lt;/a&gt; to finish this project. However, I am stuck when it comes to parsing algorithms. I managed to reduce the parsing algorithms to 3:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cocke-Kasami-Younger (CKY)&lt;/strong&gt; algorithm&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Earley&lt;/strong&gt; algorithm&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chart Parsing&lt;/strong&gt; algorithm&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note: I know that all three algorithms are chart parsing algorithms. I also know that the &lt;em&gt;Earley&lt;/em&gt; algorithm is context-free, but has a low efficiency for a compiler.&lt;/p&gt;&#xA;&lt;p&gt;What I don't know is: &lt;em&gt;Which one should I pick?&lt;/em&gt; (Please, provide a non-subjective answer to this question!)&lt;/p&gt;&#xA;&lt;p&gt;The system is for a specific domain. The answer to the natural question will be displayed in the form of the result of a calculation of some kind. Preferably in the tabular or graphical form.&lt;/p&gt;&#xA;&lt;p&gt;I have done my research. However, I probably do not understand the algorithms properly, which makes it difficult to make a selection.The algorithm should be efficient and perhaps outperform others.&lt;/p&gt;&#xA;" OwnerUserId="5219" LastEditorUserId="2444" LastEditDate="2021-02-16T15:11:03.823" LastActivityDate="2021-02-16T15:11:03.823" Title="Which parsing algorithm can I use for NLP question answering system?" Tags="&lt;natural-language-processing&gt;&lt;question-answering&gt;&lt;algorithm-request&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2788" PostTypeId="2" ParentId="2787" CreationDate="2017-02-06T10:46:00.503" Score="1" Body="&lt;p&gt;I have been reading and reading, and found answers to almost all my questions.&lt;/p&gt;&#xA;&lt;p&gt;I am sticking to &lt;em&gt;Earley&lt;/em&gt; algorithm, given that it offers a dynamic programming approach (CKY does the same). Both algorithms are chart parsing algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Earley is a context-free, top-down parsing algorithm, which makes it a goal-driven algorithm. From start symbol down. Furthermore, it is more efficient than the CKY algorithm.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.bgu.ac.il/%7Emichaluz/seminar/CKY1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; are the slides that provide more info.&lt;/p&gt;&#xA;" OwnerUserId="5219" LastEditorUserId="2444" LastEditDate="2021-01-26T15:53:51.210" LastActivityDate="2021-01-26T15:53:51.210" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2791" PostTypeId="2" ParentId="2783" CreationDate="2017-02-07T05:19:48.837" Score="3" Body="&lt;p&gt;Text approach:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster them based on the contents of the text (with or without stop words - at your discretion). If possible, you could do a supervised approach of using a bag-of-words and any classifier such as an SVM or Random Forest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image Approach:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use a CNN (convolutional neural network) and train it on images of the various vendors. If you don't have this class discrimination, and can't get it, then use an unsupervised approach such as an autoencoder and then cluster the points in the lower-dimensional autoencoder feature space.&lt;/p&gt;&#xA;" OwnerUserId="5293" LastActivityDate="2017-02-07T05:19:48.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2792" PostTypeId="1" CreationDate="2017-02-07T15:16:20.487" Score="1" ViewCount="100" Body="&lt;p&gt;Hypothetical example, say I wanted: &lt;code&gt;P(gender,ethnicity|age,hair)&lt;/code&gt;; so that the input would aligned to a trained dataset of: &lt;code&gt;(gender,ethnicity,age,hair) =&amp;gt; hat bought&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What approach is 'best' for computing ~gender and ~ethnicity given age,hair; in order to predict the hat bought?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The processing of the &lt;code&gt;inputs =&amp;gt; hat&lt;/code&gt; can be done/learned offline whereas infering the missing input values shall be done online. The results of the online pass shouldn't be stored in the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI: I am considering two Recurrent Neural Networks one for each problem.&lt;/p&gt;&#xA;" OwnerUserId="5313" LastActivityDate="2017-02-07T15:16:20.487" Title="Infer dependent variables to produce output aligned to trained data" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2793" PostTypeId="1" AcceptedAnswerId="2797" CreationDate="2017-02-07T15:59:42.137" Score="10" ViewCount="4666" Body="&lt;p&gt;So I've been trying to understand neural networks ever since I came across &lt;a href=&quot;https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471&quot; rel=&quot;noreferrer&quot;&gt;Adam Geitgey's&lt;/a&gt; blog on machine learning. I've read as much as I can on the subject (that I can grasp) and believe I understand all the broad concepts and some of the workings (despite being very weak in maths), neurons, synapses, weights, cost functions, backpropagation etc. However, I've not been able to figure out how to translate real world problems into a neural network solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case in point, Adam Geitgey gives as an example usage, a house price prediction system where given a data set containing &lt;strong&gt;No. of bedrooms&lt;/strong&gt;, &lt;strong&gt;Sq. feet&lt;/strong&gt;, &lt;strong&gt;Neighborhood&lt;/strong&gt; and &lt;strong&gt;Sale price&lt;/strong&gt; you can train a neural network to be able to predict the price of a house. However he stops short of actually implementing a possible solution in code. The closest he gets, by way of an example, is basic a function demonstrating how you'd implement weights:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def estimate_house_sales_price(num_of_bedrooms, sqft, neighborhood):&#xA;  price = 0&#xA;&#xA;  # a little pinch of this&#xA;  price += num_of_bedrooms * 1.0&#xA;&#xA;  # and a big pinch of that&#xA;  price += sqft * 1.0&#xA;&#xA;  # maybe a handful of this&#xA;  price += neighborhood * 1.0&#xA;&#xA;  # and finally, just a little extra salt for good measure&#xA;  price += 1.0&#xA;&#xA;  return price &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other resources seem to focus more heavily on the maths and the only basic code example I could find that I understand (i.e. that isn't some all singing, all dancing image classification codebase) is an implementation that trains a neural network to be an XOR gate that deals only in 1's and 0's.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there's a gap in my knowledge that I just can't seem to bridge. If we return to the &lt;strong&gt;house price prediction&lt;/strong&gt; problem, hows does one make the data suitable for feeding into a neural network? For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No. of bedrooms: 3&lt;/li&gt;&#xA;&lt;li&gt;Sq. feet: 2000&lt;/li&gt;&#xA;&lt;li&gt;Neighborhood: Normaltown&lt;/li&gt;&#xA;&lt;li&gt;Sale price: $250,000&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can you just feed &lt;strong&gt;3&lt;/strong&gt; and &lt;strong&gt;2000&lt;/strong&gt; directly into the neural network because they are numbers? Or do you need to transform them into something else? Similarly what about the &lt;strong&gt;Normaltown&lt;/strong&gt; value, that's a string, how do you go about translating it into a value a neural network can understand? Can you just pick a number, like an index, so long as it's consistent throughout the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the neural network examples I've seen the numbers passing between layers are either 0 to 1 or -1 to 1. So at the end of processing, how do you transform the output value to something usable like &lt;strong&gt;$185,000&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the house price prediction example probably isn't a particularly useful problem given that it's been massively oversimplified to just three data points. But I just feel that if I could get over this hurdle and write an extremely basic app that trains using pseudo real-life data and spits out a pseudo real-life answer than I'll have broken the back of it and be able to kick on and delve further into machine learning.&lt;/p&gt;&#xA;" OwnerUserId="5312" LastEditorUserId="5095" LastEditDate="2017-02-13T11:22:06.993" LastActivityDate="2017-02-13T11:22:06.993" Title="How to transform inputs and extract useful outputs in a Neural Network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2794" PostTypeId="1" AcceptedAnswerId="2798" CreationDate="2017-02-07T17:44:53.647" Score="3" ViewCount="265" Body="&lt;p&gt;So for a class I'm reading Brooks' &quot;Intelligence without representation&quot;. The introduction is dedicated to slating Representation as a focus for AI development. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read that representation is the problem of representing information symbolically, in time for it to be useful. It's related to the reasoning problem, which is about reasoning about symbolic information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't feel like I really understand it at any practical level. I think the idea is that when an agent is given a problem, it must describe this problem in some internal manner that is efficient and accurately describes the problem. This can then also be used to describe the primitive actions that can be taken to reach the solution. I think this then relates to Logic Programming eg Pascal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding of Representation correct? Just what does representation look like in practice, are there any open source codebases that might make a good example?&lt;/p&gt;&#xA;" OwnerUserId="5317" LastEditorUserId="5095" LastEditDate="2017-02-14T15:26:18.587" LastActivityDate="2017-02-16T04:58:06.970" Title="What does Brooks mean by Representation?" Tags="&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2795" PostTypeId="1" CreationDate="2017-02-07T19:44:51.280" Score="12" ViewCount="417" Body="&lt;p&gt;I have been looking into &lt;a href=&quot;http://viv.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Viv&lt;/a&gt;, an artificial intelligent agent in development. &lt;a href=&quot;https://www.youtube.com/watch?v=Rblb3sptgpQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a demonstration of Viv (by Dag Kittlaus)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Based on what I understand, this AI can generate new code and execute it based on a query from the user.&lt;/p&gt;&#xA;&lt;p&gt;What I am curious to know is how this AI can learn to generate code based on some query. &lt;em&gt;What kind of machine learning algorithms are involved in this process?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;One thing I considered is breaking down a dataset of programs by step. For example, here is the code to take an average of 5 terms&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Add all 5 terms together&lt;/li&gt;&#xA;&lt;li&gt;Divide by 5&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Then I would train an algorithm to convert text to code. That is as far as I have figured out. However, I haven't tried anything because I'm not sure where to start.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Does anybody have any ideas on how Viv is implemented?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4841" LastEditorUserId="36737" LastEditDate="2021-03-27T11:14:23.423" LastActivityDate="2021-03-27T11:14:23.423" Title="How can Viv generate new code based on some user's query?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2796" PostTypeId="2" ParentId="2449" CreationDate="2017-02-08T02:29:23.717" Score="3" Body="&lt;p&gt;You can try to figure out what exactly does an action do using such script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;action = 0  # modify this!&#xA;o = env.reset()&#xA;for i in xrange(5): # repeat one action for five times&#xA;    o = env.step(action)[0]&#xA;IPython.display.display(&#xA;    Image.fromarray(&#xA;        o[:,140:142]  # extract your bat&#xA;    ).resize((300, 300))  # bigger image, easy for visualization&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;action&lt;/code&gt; 0 and 1 seems useless, as nothing happens to the racket.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;action&lt;/code&gt; 2 &amp;amp; 4 makes the racket go up, and &lt;code&gt;action&lt;/code&gt; 3 &amp;amp; 5 makes the racket go down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interesting part is, when I run the script above for the same &lt;code&gt;action&lt;/code&gt;(from 2 to 5) two times, I have different results. Sometimes the racket reaches the top(bottom) border, and sometimes it doesn't. I think there might be some randomness on the speed of the racket, so it might be hard to measure which type of UP(2 or 4) is faster.&lt;/p&gt;&#xA;" OwnerUserId="5261" LastActivityDate="2017-02-08T02:29:23.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2797" PostTypeId="2" ParentId="2793" CreationDate="2017-02-08T08:45:44.100" Score="11" Body="&lt;p&gt;This is a good question which I wrestled with myself when first trying to code an ANN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a good general-purpose solution, and it's the one I implemented in my code for trying to predict well-behaved numerical data. If your data is not well-behaved (i.e. fraught with outliers) then you may need to do more work normalizing the inputs and outputs. Some of the more advanced methods are described &lt;a href=&quot;https://www.cs.ccu.edu.tw/~wylin/BA/Fusion_of_Biometrics_II.ppt&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note: I will assume that you are using f(x) = tanh(x) as your activation function. If you aren't, you should still be able to reason through how to normalize your data after reading this.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to prepare the input data:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that you want a significant variation in each input parameter to be reflected by a significant variation in the activation of the neuron those inputs are being fed into. By looking at a plot of the derivative of the tanh(x) actiavtion function, you'll see that the region of significant slope is within a distance of one or two from the origin. This means that whether the input to the activation function is 2000 or 3000 (values of x for which the derivative is negligibly small), the output of the activation will be almost identical...so your neuron's state will be independent of the difference between 2000 and 3000, and your network will never produce any predictive power from values in that range. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you want to input the square footage of the house into a neuron, you need to &lt;em&gt;normalize&lt;/em&gt; the square footage so that the network can tell the difference between 2000 and 3000. One way to do this so that all of the significant variations in your data are 'noticed' by the neuron is to &lt;strong&gt;z-score-normalize the inputs&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Gather all of your square footage values (from your training set) and calculate the mean and standard deviation. &lt;em&gt;Store the mean and standard deviation&lt;/em&gt;---you'll need this information to normalize new square footage values when testing. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Normalize the vector of square footage values by subtracting the mean and then dividing the result by the standard deviation&lt;/strong&gt; (all operations element-wise of course). Subtracting the mean centers your data at the origin, and dividing by the standard deviation makes sure most of it is between -1 and 1, where the neuron's output is most sensitive to its input. This is called z-score normalization because each input value is replaced by its &lt;a href=&quot;https://en.wikipedia.org/wiki/Standard_score&quot; rel=&quot;noreferrer&quot;&gt;z-score&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Do the above for each input variable.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, when you put each input value through a neuron, the output of the neuron is an activation between -1 and 1 (look at the image of tanh(x)). Since this is already in the 'sensitive' range of the activation function, you don't need to worry about altering the output of the input-layer neurons before sending them to the first hidden layer. Just &lt;strong&gt;give any hidden layer neurons the outputs of the previous layer directly&lt;/strong&gt;---they will be able to handle them just fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;When you reach the last layer (the output neuron(s)), what you get is again another activation between -1 and 1. You have to convert this back into a value for the house in question&lt;/strong&gt;, whether that value will be used as a prediction in a test set or to calculate error during training. However you do this, you just have to be consistent and use the same de-normalization procedure in training and testing. One way to think about it is: when the output neuron(s) returns 1, that means the network is returning the &lt;em&gt;maximum possible house value&lt;/em&gt; as its prediction. &lt;em&gt;What should the highest value the network can estimate be?&lt;/em&gt; The right approach here simply depends on your application. This is what I did:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calculate the mean of [the/each] output variable and store it.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the maximum deviation of the output variable from the mean. &#xA;Python: &lt;code&gt;MaxDev = max([abs(DataPoint-numpy.mean(TrainingData)) for DataPoint in TrainingData])&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;When the network returns output(s) between -1 and 1, multiply the output by &lt;code&gt;MaxDev&lt;/code&gt; and add it to the mean.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Two basic quick checks you can do to see if your normalization-renormalization scheme is suitable (these are necessary, but perhaps not sufficient conditions):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If all the input values are average (e.g. average no. of bedrooms, average sq.feet, etc), is the network's output equal to the average of the output variable (e.g. house value) as well? (It should be.)&lt;/li&gt;&#xA;&lt;li&gt;If all the input values are unusually high/low, is the network's output unusually high/low as well? (This only works if all the inputs are positively related to the output...if some of them are inversely related related, you will have to think a bit more).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Observe that the scheme presented here satisfies these two conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that this scheme would allow your network to only predict house values &lt;em&gt;inside&lt;/em&gt; the range of house values in your training data set. Depending on the application, this behavior can be desirable or undesirable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: you may want to make it impossible for your network to predict negative house values. Think about how you would do this. De-normalize the output so that -1 is mapped to 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to set no limit on the values your network can predict, then you can run the network's output through a function that maps the [-1,1] range to all real numbers...like arctanh(x)! As long as you do this during training your network will adjust its weights to accommodate this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this was helpful. Let me know if you have further questions. My ANN module is in Python, by the way, so I might have language-specific advice.&lt;/p&gt;&#xA;" OwnerUserId="5037" LastActivityDate="2017-02-08T08:45:44.100" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2798" PostTypeId="2" ParentId="2794" CreationDate="2017-02-08T13:08:20.987" Score="0" Body="&lt;p&gt;First, &lt;a href=&quot;https://en.wikipedia.org/wiki/Pascal_(programming_language)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pascal&lt;/a&gt; is not a logic programming language. Logic programming refers to languages like &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt; where you have a &lt;a href=&quot;https://en.wikipedia.org/wiki/Declarative_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;declarative&lt;/a&gt; style of programming compared to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Imperative_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;imperative&lt;/a&gt; style like you have in Pascal. Maybe you mean &lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_(computer_programming)#If.E2.80.93then.28.E2.80.93else.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;if-statements&lt;/a&gt; which are typical for imperative languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, representation means a certain level of abstraction. For instance, a model represents a certain part of the reality. Imagine a cup on a table. If the agent has a representation of this situation, it has a symbol &lt;code&gt;table&lt;/code&gt; and a symbol &lt;code&gt;cup&lt;/code&gt; which represents the things in the real world. Now it can have a relation &lt;code&gt;on(cup, table)&lt;/code&gt; which represents the situation that the cup is on the table. This type of abstraction can be easily represented in a logic language like &lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_(computer_programming)#If.E2.80.93then.28.E2.80.93else.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;first order logic&lt;/a&gt;. Therefore, one uses logic programming languages like Prolog or other types of languages like &lt;a href=&quot;https://en.wikipedia.org/wiki/Web_Ontology_Language&quot; rel=&quot;nofollow noreferrer&quot;&gt;OWL&lt;/a&gt; to represent knowledge and perform reasoning. So the important term to which Brooks refers is &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge Representation and Reasoning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, if your agent only have sensor data like video or sonar data, then it knows only distances or pixels from the real world. That is not meant with representation. Brooks' Creatures have only this information and calculate with this data directly to perform an action without reasoning. In that sense also artificial neural networks have no representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, for an open source project to understand representation I would recommend the above mentioned OWL. You can look at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Prot%C3%A9g%C3%A9_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protégé&lt;/a&gt; editor for working with OWL. In an OWL ontology you can represent relations between things and reason about them.&lt;/p&gt;&#xA;" OwnerUserId="5095" LastActivityDate="2017-02-08T13:08:20.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2803" PostTypeId="1" CreationDate="2017-02-09T20:27:23.960" Score="2" ViewCount="580" Body="&lt;p&gt;If we look at state of the art accuracy on the UCF101 data set, it is around 93% whereas for the HMDB51 data set it is around 66%. I looked at both the data sets and both contain videos of similar lengths. I was wondering if anyone could give an intuition as to why HMDB51 data set has been harder.&lt;/p&gt;&#xA;" OwnerUserId="4700" LastEditorUserId="4700" LastEditDate="2017-02-09T20:34:19.227" LastActivityDate="2018-07-28T16:01:08.960" Title="Why do action recognition algorithms perform better on ucf101dataset than HMDB51 dataset?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;action-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2804" PostTypeId="1" CreationDate="2017-02-09T22:08:25.567" Score="2" ViewCount="140" Body="&lt;p&gt;We are working on a project for creating music based on crowd sourcing. People vote for every note until the vote is closed, and then move on to the next vote until the canvas for the music is filled. A similar project is &lt;a href=&quot;https://crowdsound.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;crowdsound&lt;/a&gt;, if you want to get an idea of what it looks like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the fun part is, based on all the votes we get from various people, we would like to be able to build a Neural Network that can build an entire song on its own. The idea is for it to take in account every preceding vote and predict the one that will follow. That way, when trained, we could give it one note and let it predict the rest of the votes on its own and thus create a song on its own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've read a few things here and there about neural networks, but there are two things I don't understand:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How to build one that takes into account a dynamic number of inputs (all preceding votes).&lt;/li&gt;&#xA;&lt;li&gt;How exactly should I decide the number of hidden layers (I still only vaguely understand what those hidden layers represent) I need for it to work well.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We are using Java for the project and we were planning on using Neuroph for the neural network.&lt;/p&gt;&#xA;" OwnerUserId="5372" LastEditorUserId="3576" LastEditDate="2017-02-25T19:55:17.220" LastActivityDate="2017-02-25T19:55:17.220" Title="Creating a neural network for predicting next vote in a series of votes" Tags="&lt;neural-networks&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2017-02-27T15:16:24.727" ContentLicense="CC BY-SA 3.0" />
  <row Id="2806" PostTypeId="1" CreationDate="2017-02-10T09:15:11.580" Score="3" ViewCount="140" Body="&lt;p&gt;I have users' reports about an accident. I want to know how to make sure that the number of reports is big enough to take that accident as a true accident and not spam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to consider a minimum number of reports in a specific time interval, for example 4 reports in 20 minutes are good enough to believe the existence of that accident.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I choose the minimum number of reports and that time interval? Is there some logic to make that decision?&lt;/p&gt;&#xA;" OwnerUserId="5383" LastEditorUserId="75" LastEditDate="2017-04-27T19:25:41.113" LastActivityDate="2017-09-24T23:04:21.033" Title="Making decision based on users' reports" Tags="&lt;decision-theory&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2807" PostTypeId="2" ParentId="2806" CreationDate="2017-02-11T07:00:18.877" Score="0" Body="&lt;p&gt;It's a trust-level problem, so your judgment is the best to decide what would be rhe threshold. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can help your decision making by trying and visualize how many accident (in %) you've left out... this can be an indicator of good threshold. You don't want to throw too many of them. But only you know what is good and bad in this case&lt;/p&gt;&#xA;" OwnerUserId="5397" LastActivityDate="2017-02-11T07:00:18.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2808" PostTypeId="1" AcceptedAnswerId="2831" CreationDate="2017-02-11T07:55:47.130" Score="6" ViewCount="416" Body="&lt;p&gt;I want to create a network to predict the break up of poetry lines. The program would receive as input an unbroken poem, and would output the poem broken into lines.&lt;/p&gt;&#xA;&lt;p&gt;For example, an unbroken poem could be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;And then the day came, when the risk to remain tight in a bud was more painful&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which should be converted to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;And then the day came,&#xA;when the risk&#xA;to remain tight&#xA;in a bud&#xA;was more painful&#xA;than the risk&#xA;it took&#xA;to Blossom.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How should I go about this? I have been using classifiers for various tasks, but this seems to be a different type of task.&lt;/p&gt;&#xA;&lt;p&gt;I'm thinking of it as an array of words (doesn't matter how they're represented for now), which would look like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[6, 32, 60, 203, 40, 50, 60, 230 ...] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and needs to map into an array representing line breaks&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[0, 0, 1, 0, 0, 0, 1, 0, 0, 1 ...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where 1 (at optimal) means there should be a line break after the word in that index (in this idea, the two arrays are of the same length).&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, I couldn't find an algorithm that could train a network of this shape.&lt;/p&gt;&#xA;&lt;p&gt;What machine learning or deep learning algorithm can be used for this task?&lt;/p&gt;&#xA;" OwnerUserId="5400" LastEditorUserId="2444" LastEditDate="2021-01-21T00:46:28.943" LastActivityDate="2021-01-21T00:46:28.943" Title="Which machine learning algorithm could I use to break up a poem by lines?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2810" PostTypeId="1" CreationDate="2017-02-11T11:22:59.897" Score="6" ViewCount="223" Body="&lt;p&gt;I'm here to ask you for a solution on this problem which is: how to use Reinforcement Learning in Immersive Virtual Reality to make a person move to a specific location in a virtual environment. As you know reinforcement Learning is a sub-area of Machine Learning in which an active entity called an agent interacts with its environment and learns how to act in order to achieve a pre-determined goal. The Reinforcement Learning had no prior model of behaviour and the participants no prior knowledge that their task was to move to and stay in a specific place. The participants were placed in a virtual environment where they had to avoid collisions with virtual projectiles. Following each projectile the agent analysed the movement made by the participant to determine paths of future projectiles in order to increase the chance of driving participants to the goal position and make them stay there as long as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update 1: &lt;a href=&quot;http://discovery.ucl.ac.uk/1539195/1/Slater_elsarticle-template-harv.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Download: Reinforcement Learning as a tool to make people move to a speciﬁc location in Immersive Virtual Reality&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5398" LastEditorUserId="16612" LastEditDate="2018-07-02T20:10:57.960" LastActivityDate="2018-10-15T23:38:02.657" Title="A solution for a famous problem in RL" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2811" PostTypeId="1" AcceptedAnswerId="2812" CreationDate="2017-02-11T18:58:46.723" Score="-1" ViewCount="456" Body="&lt;p&gt;In a Neural network, there is an input layer, any number of hidden layers, and an output layer. My question is: Are the input and output layer nodes actually perceptions? Or do they just signify what/how many/where the inputs and outputs are? &lt;/p&gt;&#xA;" OwnerUserId="4744" LastActivityDate="2017-02-11T21:47:23.957" Title="Perceptions in a Neural network" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;algorithm&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-11T00:59:52.317" ContentLicense="CC BY-SA 3.0" />
  <row Id="2812" PostTypeId="2" ParentId="2811" CreationDate="2017-02-11T21:47:23.957" Score="1" Body="&lt;p&gt;Not 100% sure I got your question right, but yes the input nodes are perception fields. For the output it's good if you choose a space such that the different outputs you expect are independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One method to improve neural networks is actually using receptive fields. Look for convolutionary networks and pooling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would have posted that to a comment if I could.&lt;/p&gt;&#xA;" OwnerUserId="5416" LastActivityDate="2017-02-11T21:47:23.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2813" PostTypeId="2" ParentId="111" CreationDate="2017-02-12T10:48:09.903" Score="3" Body="&lt;p&gt;The only sensible choice is to use predictable behaviour. So in the people in front of the car scenario: First the car hits the brakes, at the same time honks the horn, and stays on course.  The people then have a chance to jump out of the way leading to zero people being killed. Also with full brakes (going from 50km per hour to zero is less than 3 car lengths), an impact situation is almost not imaginable. Even if full stop cannot be reached, severe damage to the pedestrians is unlikely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other scenario is just crazy. So the distance has to be less than 3 car lengths, at least 1 car length in needed for the steering, then a car crashing is an uncontrollable situation, might lead to spinning and kill all 11 persons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from saying that I don't believe there is an example in reality where there is a dilemma;  the solution in these unlikely cases if to conform with the expectations of the opposing party to allow the other party to mitigate the situation as well.&lt;/p&gt;&#xA;" OwnerUserId="5423" LastEditorUserId="14723" LastEditDate="2018-04-14T01:49:58.217" LastActivityDate="2018-04-14T01:49:58.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2814" PostTypeId="2" ParentId="111" CreationDate="2017-02-13T04:50:51.100" Score="3" Body="&lt;p&gt;I think there would not be a way to edit such ethics settings in a car. But hey, if cell phones can be rooted, why not cars? I imagine there'll be Linux builds in the future for specific models that will let you do whatever you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for who'll make such decisions, it'll be much like privacy issues of today. There'll be a tug-of war on the blanket by the OS providers (who'll try to set it to a minimum amount of people injured, each with its own methods), insurance companies (who'll try to make you pay more for OS's that will be statistically shown to damage your car easier), and car manufacturers (who'll want you to trash your car as soon as you can, so you'll buy a new one; or make cars that require a ridiculous amount of $$$ service). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then some whistleblower will come out and expose a piece of code that chooses to kill young children over adults - because it will have a harder time distinguishing them from animals, and will take chances to save who it'll more surely recognize as humans. The OS manufacturer will get a head-slap from the public and a new consensus will be found. Whistleblowers will come out from insurance companies and car manufacturers too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humanity will grab a hot frying pan and burn itself and then learn to put on gloves beforehand. My advice would just make sure you won't be that hand - stay away from them for a couple of years until all the early mistakes are made.&lt;/p&gt;&#xA;" OwnerUserId="5434" LastEditorUserId="14723" LastEditDate="2018-04-14T01:49:27.290" LastActivityDate="2018-04-14T01:49:27.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2816" PostTypeId="2" ParentId="2517" CreationDate="2017-02-13T13:57:14.503" Score="1" Body="&lt;p&gt;I think you mean the leave nodes only which are changed. The other nodes in the tree are calculated during calculating the best move with this tree. The values at the leaves are called &lt;strong&gt;utility values&lt;/strong&gt; in Russel and Norvig's &quot;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/9332543518&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial intelligence: a modern approach&lt;/a&gt;&quot;. Some times it is called &lt;strong&gt;heuristic value&lt;/strong&gt;; see &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Minimax&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5095" LastEditorUserId="5095" LastEditDate="2017-02-14T08:31:53.207" LastActivityDate="2017-02-14T08:31:53.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2817" PostTypeId="1" CreationDate="2017-02-13T13:57:39.303" Score="6" ViewCount="129" Body="&lt;p&gt;I have been trying to reproduce the experiments done in the original: &lt;a href=&quot;https://arxiv.org/pdf/1003.1466&quot; rel=&quot;nofollow noreferrer&quot;&gt;Firefly Algorithm for multimodal optimization&lt;/a&gt; (2010) by Xin-She Yang, but so far unsuccessfully. For the moment being, I'm okay if anyone points me in the right direction.&lt;/p&gt;&#xA;&lt;p&gt;I wrote the algorithm as specified in the paper in C++ programming language (I also downloaded several other implementations from the internet for comparison purposes) and used the very same parameters as specified in the paper (a random steep of 0.2, an initial light intensity of 1.0 and a light decay coefficient of 1.0, a population size of 40). I used the two bright update equations given and for De Jung test function (as for example) a number of dimensions of 256 in a search domain in [-5.12, 5.12] as referred to in common optimization literature and in the paper.&lt;/p&gt;&#xA;&lt;p&gt;In the paper, the algorithm converges very quickly, as can be expected, since this is a very simple test function. However, neither my implementation nor any code I have downloaded converges with that parameters.&lt;/p&gt;&#xA;&lt;p&gt;My final questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Am I doing something wrong with the experimental methodology or am I using the wrong parameter settings (may be something different than the original paper)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do anyone knows where can I find a code sample of the Firefly Algorithm that I can use to reproduce the experiments of the mentioned paper?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Please, notice that there may be a lot of variations of this algorithm that can produce better results, but right now I'm only interested in reproducing the experiments of the so-called paper.&lt;/p&gt;&#xA;" OwnerUserId="3566" LastEditorUserId="2444" LastEditDate="2021-01-31T14:24:48.330" LastActivityDate="2021-01-31T14:24:48.330" Title="Why can't I reproduce the experiments in the original paper that introduced the Firefly Algorithm?" Tags="&lt;papers&gt;&lt;optimization&gt;&lt;swarm-intelligence&gt;&lt;meta-heuristics&gt;&lt;firefly-algorithm&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2818" PostTypeId="2" ParentId="2517" CreationDate="2017-02-14T12:39:59.723" Score="0" Body="&lt;p&gt;In reinforcement learning, you can keep the name of value as it comes from the value function which estimates how good it is to be in this node w.r.t. the objective. Depending on the problem it can be a cost, utility, reward...&lt;/p&gt;&#xA;" OwnerUserId="5472" LastActivityDate="2017-02-14T12:39:59.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2819" PostTypeId="2" ParentId="2808" CreationDate="2017-02-15T04:39:10.893" Score="2" Body="&lt;p&gt;The underlying problem is combinatorial, as you note, but I'm not getting how you're ascribing value to words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key element of deciding line breaks, beyond the visual, is rhythmic. &lt;em&gt;(There are other factors, as Bob Salita notes, but you've gotta start somewhere.)&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me you need to teach the computer how to scan a phrase in the poetic sense, which relates to rhythm.  This is obviously a very difficult task, but the number of syllables and stresses is fundamental numerical data of poetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to validate to human tastes, you'd then have to use a captcha crowdsourcing method, for both the rhythmic stresses as input, and getting human reactions to different line-break configurations.  You would then reinforce the positive reactions, and the AI would tailor the line-break process to the audience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, instead of utilizing human tastes and aesthetic sensibilities, you could instead let the AI decide what is preferred, which would probably be comprised of some sort of symmetry considered optimal to an algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this logic, you wouldn't even need to have the AI learn the stresses, instead just focusing on raw syllables, or, numeric representation based on any factor. (With this method, the object is not to reformat poetry for humans, but for machines :)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is more about the aesthetics, but Cameron Browne's &lt;a href=&quot;http://www.cameronius.com/games/shibumi/browne-elegance-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elegance in Game Design&lt;/a&gt; would seem to suggest there are engineering solutions to the type of aesthetic issues at the root of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might start by teaching it to count the syllables of the poem, then having it look at the divisors. If it's roughly 10, it might be iambic pentameter. The AI doesn't care about the label, but it likes 10.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;20 syllables might represent a couplet in that meter:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The time is out of joint, oh cursed spite&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;that ever I was born to set it right&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd definitely start by feeding it older poetry, particularly poets that keep to strict meter. It's been a while since I've read Spencer and so forth, but I'd think poets of his time would be useful.  Dr. Seuss, perhaps the greatest wielder of the rhyming couplet, would surely be extraordinarily useful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evaluation method would have to be fuzzy, because there would be increasing degrees of variance the more modern the poetry, ultimately resulting in free structures, except in the case of forms such as rap, which strongly utilize regularized rhythm. Machine learning is all about estimation and reinforcement, and is proving to be extremely useful dealing with fuzziness.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dead mountain mouth of carious teeth that cannot spit&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is a great example of modern line of poetry: the floor of 13 syllables / 2 makes a 6 beat line.  Understanding that in context with the surrounding verse is much more difficult and illustrates the nature of the problem. Even scanning the poem correctly to that point to determine would be extremely difficult. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a different poem by the same author is extremely useful:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the late November doing / With the disturbance of the spring / And creatures of the summer heat, / And snowdrops writhing under feet / And hollyhocks that aim too high / Red into grey and tumble down / Late roses filled with early snow? / Thunder rolled by the rolling stars / Simulates triumphal cars / Deployed in constellated wars / Scorpion fights against the sun / Until the Sun and Moon go down / Comets weep and Leonids fly / Hunt the heavens and the plains / Whirled in a vortex that shall bring / The world to that destructive fire / Which burns before the ice-cap reigns&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;All lines of roughly 8 syllables, easy to pick out because of capitalization.  But the real question is: ~136 13 lines of roughly 10 syllables, or 17 lines of roughly 8?  It would want to calculate based on word blocks (words that cross syllabic thresholds and at least tell you where the break &lt;em&gt;cannot&lt;/em&gt; be, and it should be possible to statistically divine the pattern, at least for regularized verse.)  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The wounded surgeon plies the steel / That questions the distempered part; /&#xA;  Beneath the bleeding hands we feel / The sharp compassion of the healer's art &#xA;  / Resolving the enigma of the fever chart.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This verse highlight the problem.  5 lines of 4 beats, but syllabically: 8, 8, 8, 10, 12.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most likely: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;46/5 = 9.2&lt;/li&gt;&#xA;&lt;li&gt;46/4 = 11.5&lt;/li&gt;&#xA;&lt;li&gt;46/6 = 7.66&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Less likely:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;46/3 = 15.3&lt;/li&gt;&#xA;&lt;li&gt;46/2 = 23  &lt;/li&gt;&#xA;&lt;li&gt;46/7 = 6.57&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;2 lines has less perfect symmetry, but 5 lines is more likely, based on the overall number of syllables, and of the likely choices, has the least variance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately it would be looking for the underlying structure, or lack of structure, and try to reorganize the unbroken text into something &lt;em&gt;close&lt;/em&gt; to the original structure.  While exactness is not always required because the process is ultimately subjective, and currently intractable, certain wrong choices would yield disastrous results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the prior example it may be able to discern the likelihood of a 5 line pattern, but it would have to figure out on which lines to place the extra syllables.  Differentiating between particles and other parts of speech provides a clue, because the poet's language is very compact: there are 19 nouns, verbs, or prepositions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More likely:&#xA; - 19/5 = 3.8&#xA; - 19/4 = 4.75&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Less likely:&#xA; - 19/3 = 6.33&#xA; - 19/6 = 3.16&#xA; - 19/7 = 2.71&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further analysis might narrow it down.  But extremely regularized verse remains the best place to start.  7 lines of roughly 10 syllables is &quot;poetic&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;‘The aged man that coffers-up his gold&lt;br&gt;&#xA;  Is plagu’d with cramps and gouts and painful fits;&lt;br&gt;&#xA;  And scarce hath eyes his treasure to behold,&lt;br&gt;&#xA;  But like still-pining Tantalus he sits,&lt;br&gt;&#xA;  And useless barns the harvest of his wits;&lt;br&gt;&#xA;  Having no other pleasure of his gain&lt;br&gt;&#xA;  But torment that it cannot cure his pain.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It cares about both X and Y values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially you want to keep it to a single language, because syllables may be treated differently.  That said, having the AI look for something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Dactylic_hexameter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dactylic hexameter&lt;/a&gt; would be extremely useful, because you could feed it Homer. You could also feed it Homer in English in many different forms of English meter, and in almost every other living language. By definition, the AI would value works such as these, because max number_of_translations provides the most robust data set. When it starts to value meaning, this will be especially important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding different ways of treating syllables(long/short vs. stressed/unstressed) will also be essential as it transitions into more modern poetry.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.writing.upenn.edu/~afilreis/88/meter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a good link for basic English meter.&lt;/a&gt;  Iambic and Trochaic meters will be easy, while meters that employ Anapests, Dactyls and Spondees will be more challenging. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, however, these will be mathematically interchangeable.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I went to the Garden of Love, / And saw what I never had seen: / A Chapel was built in the midst, / Where I used to play on the green. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It doesn't matter if the lines above are iambic/trochaic or dactylic/anapestic, it's still 4 lines of roughly 8 syllables.  Thus &quot;I went to the Garden of Love&quot; is the same as &quot;The wounded surgeon plies the steel&quot;, even though the beats for the lines are 3 and 4, respectively.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It should also have a stanza marker, (possibly 00?). Because it looks for patterns within patterns, stanzas are valued.  Not all poetry has a stanza structure, but it arguably could. Deciding if stanzas are appropriate is partly a function of taking a syllabic divisor, breaking the poem down into number_of_lines, and looking at the divisors of that number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would need an added function to be able to recognize meaning patterns.  For instance, repetition of proper nouns is the marker of plays. (From a meaning perspective, imo, plays is the ideal place to start because the marker is so easy to learn, and names all belong to a single set, and imply communication. It's no different functionally than any other identifier, and a concept all computers &quot;understand&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually it would want to look for phonetic patterns, rhymes and near rhymes, which would also be indicators of potential good places for line breaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a very big data set that it can look at, and who knows what it might discern?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-02-19T21:55:04.943" LastActivityDate="2017-02-19T21:55:04.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2820" PostTypeId="1" CreationDate="2017-02-15T08:33:57.883" Score="11" ViewCount="2611" Body="&lt;p&gt;Everything related to Deep Learning (DL) and deep(er) networks seems &quot;successful&quot;, at least progressing very fast, and cultivating the belief that AGI is at reach. This is popular imagination. DL is a tremendous tool to tackle so many problems, including the creation of AGIs. It is not enough, though. A tool is a necessary ingredient, but often insufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leading figures in the domain are looking elsewhere to make progress. This &lt;a href=&quot;https://hackernoon.com/feynman-machine-a-new-approach-for-cortical-and-machine-intelligence-5855c0e61a70#.dmgovix19&quot; rel=&quot;noreferrer&quot;&gt;report/claim&lt;/a&gt; gathers links to statements by &lt;a href=&quot;https://www.quora.com/Is-the-current-hype-about-Deep-Learning-justified?redirected_qid=6578691&quot; rel=&quot;noreferrer&quot;&gt;Yoshua Bengio&lt;/a&gt;, &lt;a href=&quot;https://www.quora.com/What-are-the-limits-of-deep-learning-2/answer/Yann-LeCun&quot; rel=&quot;noreferrer&quot;&gt;Yann LeCun&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=VIRCybGgHts&quot; rel=&quot;noreferrer&quot;&gt;Geoff Hinton&lt;/a&gt;. The report also explains:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The main weaknesses of DL (as I see them) are: reliance on the simplest possible model neurons (“cartoonish” as LeCun calls them); use of ideas from 19th century Statistical Mechanics and Statistics, which are the basis of energy functions and log-likelihood methods; and the combination of these in techniques like backprop and stochastic gradient descent, leading to a very limited regime of application (offline, mostly batched, supervised learning), requiring highly-talented practitioners (aka “Stochastic Graduate Descent”), large amounts of expensive labelled training data and computational power. While great for huge companies who can lure or buy the talent and deploy unlimited resources to gather data and crunch it, DL is simply neither accessible nor useful to the majority of us.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Although interesting and relevant, such kind of explanation does not really address the gist of the problem: What is lacking?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question seems broad, but it may be by lack of a simple answer. Is there a way to pin-point what DL is lacking for an AGI ?&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2017-02-20T19:31:05.387" Title="Why are deep neural networks and deep learning insufficient to achieve general intelligence?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;agi&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2821" PostTypeId="2" ParentId="2820" CreationDate="2017-02-15T11:30:28.503" Score="2" Body="&lt;p&gt;I think it's missing still the aspects what makes a human brain; having a lot of different networks working with each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just like meditation improves cognitive abilities by having the brain work more synergistically, we could apply that to machines too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example google is learning a computer to dream, just like we do, to reinforce what we already learned. &#xA;&lt;a href=&quot;https://medium.com/@tannistho/why-is-google-teaching-its-ai-to-dream-e9ae9ecd0e3a#.gljal6pww&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@tannistho/why-is-google-teaching-its-ai-to-dream-e9ae9ecd0e3a#.gljal6pww&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is pathnet, a network of neural network. &#xA;&lt;a href=&quot;https://medium.com/@thoszymkowiak/deepmind-just-published-a-mind-blowing-paper-pathnet-f72b1ed38d46#.ed0f6pdq7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@thoszymkowiak/deepmind-just-published-a-mind-blowing-paper-pathnet-f72b1ed38d46#.ed0f6pdq7&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creating all these mechanics and putting them all together, with enough power and we will get pretty close!&lt;/p&gt;&#xA;" OwnerUserId="5388" LastEditorUserId="5388" LastEditDate="2017-02-20T12:00:03.940" LastActivityDate="2017-02-20T12:00:03.940" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2824" PostTypeId="1" CreationDate="2017-02-15T15:39:15.997" Score="3" ViewCount="528" Body="&lt;p&gt;I am trying to understand the algorithm for n-step SARSA from &lt;a href=&quot;http://incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto (2nd Edition)&lt;/a&gt;. As I understand it, this algorithm should update n state-action values, but I cannot see where it is propagated backward. Can someone explain to me how that works?&lt;/p&gt;&#xA;" OwnerUserId="5503" LastEditorUserId="2444" LastEditDate="2020-02-27T16:05:10.233" LastActivityDate="2020-02-27T16:05:10.233" Title="When does backward propagation occur in n-step SARSA?" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2828" PostTypeId="2" ParentId="2794" CreationDate="2017-02-16T04:58:06.970" Score="0" Body="&lt;p&gt;In that paper, Brooks introduced the basis for what became known as his &quot;subsumption architecture&quot;.  The idea was to get away from the 1980's popular approach of a single global representation of all the components of the problem space that had required the task of robot task planning to juggle every  constraint in the world into one giant disordered mess of states and state transitions.  Rather than represent every element in the world in a single model (The Representation), Brooks suggested it was preferable to build a hierarchy of submodels of the world (subsets of states and transitions) in which smaller tasks could be more readily planned.  Then as these rudimentary skills were mastered, they could be combined to address a hierarchy of bigger and more complex tasks (bigger tasks subsume smaller tasks and benefit from their already having been solved).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, representation did not fully go away, but it was redistributed hierarchically so that much of the state could be abstracted away from the higher level of the bigger problem that you need to solve.  Planning became like coordinating a hierarchical army of skills, where the general doesn't need to plan every movement of the private in order to manage a battle.  Instead, that general need only tell the colonels what to do, and the colonels tell the majors, and so on down to the privates.  Now the general solves problems by coodinating multiple sub-hierarchies available to him/her by delegating authority to coordinate behavior at the appropriate level of abstraction: like division, brigade, battalion, company, and squad.  That's Brooks' Subsumption Architecure: the general needs to represent a battle plan only as &quot;the world according to colonels&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Subsumption_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Subsumption_architecture&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-02-16T04:58:06.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2829" PostTypeId="2" ParentId="2820" CreationDate="2017-02-16T09:17:29.710" Score="7" Body="&lt;p&gt;Everyone dealing with neural networks misses an important point when comparing systems with human like intelligence. A human takes many months to do anything intelligible, let alone being able to solve problems where adult humans can barely manage. That and the size of human brain is enormous compared to our neural networks. Direction might be right, but the scale is way off. Number of neurons in human brain can be matched memory-wise but the amount of parallelism to simulate it real-time cannot yet be achieved (at least for a random researcher). While a little old &lt;a href=&quot;https://www.extremetech.com/extreme/163051-simulating-1-second-of-human-brain-activity-takes-82944-processors&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; might give you an idea of how much we lack the processing power. &lt;/p&gt;&#xA;" OwnerUserId="210" LastEditorUserId="210" LastEditDate="2017-02-16T09:28:59.653" LastActivityDate="2017-02-16T09:28:59.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2830" PostTypeId="2" ParentId="2820" CreationDate="2017-02-16T14:11:46.497" Score="6" Body="&lt;p&gt;Deep Learning is mostly successful in supervised learning, whereas the brain builds categories mostly in an unsupervised way. We don't yet know how to do that. (Take a &lt;a href=&quot;https://www.wired.com/2012/06/google-x-neural-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;look at google brain&lt;/a&gt;: 16,000 cores and all this thing can do is recognise cats and human faces with pretty abysmal accuracy.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep Learning uses highly unstructured activations, i.e. the high level representations of &quot;dog&quot; and &quot;cat&quot; in a neural network classifier don't have to be similar at all. The brain on the other hand uses inhibitory neurons to create &lt;a href=&quot;http://www.cortical.io/technology_representations.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sparse distributed representations&lt;/a&gt; which are decomposable into their semantic aspects. That's probably important for abstraction and reasoning by analogy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The brain has many different parts which work together. Deep Learning researchers are only just beginning to integrate &lt;a href=&quot;https://arxiv.org/abs/1410.5401&quot; rel=&quot;nofollow noreferrer&quot;&gt;memory&lt;/a&gt; or attention mechanisms into their architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The brain integrates information from many different senses. Most Deep Learning applications use just one type of input, like text or pictures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The brain is capable of modelling sequences as categories. (Basically every verb names a sequential (i.e. temporal) category.) It can then arrange these categories into long-term hierarchical plans. So far I haven't seen anything in that direction in Deep Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also neural networks can't yet operate on the same scale as the human brain. If you look at &lt;a href=&quot;https://ai.stackexchange.com/questions/2330/when-will-the-number-of-neurons-in-ai-systems-equal-the-human-brain&quot;&gt;the answers to this question&lt;/a&gt;, the human brain will be ahead in neuron count for another couple of decades. A neural network might not need the same number of neurons as the brain to reach a similar performance (because of higher accuracy), but right now for example video processing is still pretty limited in terms of input and throughput. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2017-02-16T14:11:46.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2831" PostTypeId="2" ParentId="2808" CreationDate="2017-02-16T14:39:14.837" Score="3" Body="&lt;p&gt;You should try &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;to use an RNN&lt;/a&gt;. You feed in letter by letter and have a binary output of linebreak - no linebreak. If you have enough data it might actually work. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-02-16T14:39:14.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2832" PostTypeId="2" ParentId="2772" CreationDate="2017-02-16T23:08:01.760" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://www.lego.com/en-us/mindstorms&quot; rel=&quot;nofollow noreferrer&quot;&gt;LEGO Mindstorms&lt;/a&gt; is widely used to demonstrate AI in schools and universities [&lt;a href=&quot;https://pdfs.semanticscholar.org/b515/dcdb633e2de2101a1eabfc53ecf84f5fcd39.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;http://www.legoengineering.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]. With LEGO as basis, you are very flexible. You can build what you want very easily. The AI programs can be written in different languages from very easy graphical once to Lisp and C++. The newest version has an SD Card drive, USB interface and a powerful ARM processor. You can use four motors and four sensors directly. There exists touch, sound, sonar, gyro, infrared and color sensors. There is also a big community which provides you with a lot of ideas, hardware and programs [&lt;a href=&quot;https://www.hackster.io/mindstorms&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;].&lt;/p&gt;&#xA;" OwnerUserId="5095" LastEditorUserId="2444" LastEditDate="2019-06-25T16:22:55.013" LastActivityDate="2019-06-25T16:22:55.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2833" PostTypeId="1" CreationDate="2017-02-16T23:53:08.683" Score="8" ViewCount="161" Body="&lt;p&gt;How does in the (famous Zilberstein) &lt;code&gt;PR&lt;/code&gt;(uning) algorithm below the &lt;code&gt;LP-dominate&lt;/code&gt; function get started: the first time it's called, &lt;code&gt;D=∅&lt;/code&gt; and the linear program deteriorates (i.e. no constraint equations)? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;procedure POINTWISE-DOMINATE(w, U)&#xA;...&#xA;3. return false&#xA;procedure LP-DOMINATE(w, U)&#xA;4. solve the following linear program variables: d, b(s) ∀s ∈ S&#xA;      maximize d&#xA;      subject to the constraints&#xA;        b · (w − u) ≥ d, ∀u ∈ U&#xA;        sum(b) = 1&#xA;5. if d ≥ 0 then return b&#xA;6. else return nil&#xA;procedure BEST(b, U )&#xA;...&#xA;12. return w&#xA;procedure PR(W)&#xA;13. D ← ∅&#xA;14. while W = ∅&#xA;15.   w ← any element in W&#xA;16.   if POINTWISE-DOMINATE(w, D) = true&#xA;17.      W ← W − {w}&#xA;18.   else&#xA;19.      b ← LP-DOMINATE(w, D)&#xA;20.      if b = nil then&#xA;21.         W ← W − {w}&#xA;22.      else&#xA;23.         w ← BEST(b, W)&#xA;24.         D ← D ∪ {w}&#xA;25.         W ← W − {w}&#xA;26. return D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5534" LastEditorUserId="7402" LastEditDate="2017-08-14T22:13:13.117" LastActivityDate="2018-07-13T01:07:41.610" Title="Zilberstein's &quot;LP-dominate&quot; pruning explained?" Tags="&lt;decision-theory&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2834" PostTypeId="1" AcceptedAnswerId="2838" CreationDate="2017-02-17T15:24:50.153" Score="4" ViewCount="203" Body="&lt;p&gt;I want to know something more about it. Are there any github repo or an open source project?&lt;/p&gt;&#xA;" OwnerUserId="5549" LastActivityDate="2017-02-20T08:42:24.800" Title="Is it possible for an AI to learn how to speak from books as training sets?" Tags="&lt;machine-learning&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2835" PostTypeId="2" ParentId="2820" CreationDate="2017-02-17T23:46:18.470" Score="1" Body="&lt;p&gt;Artificial intelligence proponents today are focused on the problem of computability - the ability to solve complex problems fast. It is my belief that any amount of success in this direction will not lead to human (general) intelligence although it certainly will outperform humans in certain domains. Instead, efforts should be toward a study of what neurological events cause sensation (the experience of qualia). Of course, this is the hard problem of philosophy but I believe it is the unique key to general intelligence and its capabilities. Reverse engineering and also testable theories should be advanced toward this end. &lt;/p&gt;&#xA;" OwnerUserId="5556" LastActivityDate="2017-02-17T23:46:18.470" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2836" PostTypeId="2" ParentId="2834" CreationDate="2017-02-18T04:01:40.217" Score="0" Body="&lt;p&gt;&lt;strong&gt;Ya Sure it is possible&lt;/strong&gt;. It wont be efficient as human speech though (At least not yet). It all depends on how you use your data. If you use your data efficient enough, then you could be the one creating an AI which closely resembles human speech. Your idea is good. You would need help from a GPU for processor all that complex text processing. I hope I was at least a little of help. &lt;strong&gt;I unfortunately don't know about any open source projects&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5557" LastEditorUserId="5557" LastEditDate="2017-02-18T12:31:57.080" LastActivityDate="2017-02-18T12:31:57.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2837" PostTypeId="1" AcceptedAnswerId="2859" CreationDate="2017-02-18T04:12:39.273" Score="3" ViewCount="58" Body="&lt;p&gt;If the nervous system is wired up such that there are no well defined layers, how does this compare to a neatly stacked artificial net? If between my sensory and motor side I had a neatly designed SNN with well defined layers, how would I see the world?&#xA;I get that there are some evolutionary advantages to a system where information can sometimes take a shortcut from sensory cell to motor cell (reflex action) bypassing brain processing but for arguments sake let's talk only about intelligence.&lt;/p&gt;&#xA;" OwnerUserId="5558" LastActivityDate="2017-02-21T09:50:42.437" Title="Would a neuromorphic SNN of the same complexity as the human nervous system be 'smarter'?" Tags="&lt;neural-networks&gt;&lt;biology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2838" PostTypeId="2" ParentId="2834" CreationDate="2017-02-18T08:38:45.353" Score="1" Body="&lt;p&gt;&lt;strong&gt;It is possible&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent Neural Network architectures&lt;/a&gt; help in building efficient NLP algorithms, which can identify semantics and their relations across long pieces of text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With very minor tuning, they can be made generative too. So, &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is an excellent article on RNNs&lt;/a&gt; which I highly recommend, which also talks about how an RNN was trained on Shakespere's texts and wrote one itself.&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2017-02-18T08:38:45.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2839" PostTypeId="2" ParentId="2833" CreationDate="2017-02-18T11:38:34.770" Score="1" Body="&lt;p&gt;I think I found the solution. When in &lt;code&gt;PR(W)&lt;/code&gt;, &lt;code&gt;D=∅&lt;/code&gt;, the weight is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;b[i] = 0          for { i | w[i]&amp;lt;max(w) }&lt;/code&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;b[i] = 1.0/max(w) for { i | w[i]==max(w) }&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5535" LastEditorUserId="9647" LastEditDate="2017-10-03T21:24:09.727" LastActivityDate="2017-10-03T21:24:09.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2841" PostTypeId="1" AcceptedAnswerId="2844" CreationDate="2017-02-18T15:10:58.353" Score="10" ViewCount="356" Body="&lt;p&gt;In Chapter 26 of the book &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt; (3rd edition), the textbook discusses &quot;technological singularity&quot;. It quotes I.J. Good, who wrote in 1965:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let an ultra-intelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultra-intelligent machine could design even better machines; there would then unquestionably be an &quot;intelligence explosion,&quot; and the intelligence of man would be left far behind. Thus the first ultra-intelligent machine is the &lt;em&gt;last&lt;/em&gt; invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Later on in the textbook, you have this question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;26.7 - I. J. Good claims that intelligence is the most important quality, and that building ultr- intelligent machines will change everything. A sentient cheetah counters that &quot;Actually speed is more important; if we could build ultrafast machines, that would change everything&quot; and a sentient elephant claims &quot;You're both wrong; what we need is ultrastrong machines,&quot; What do you think of these arguments?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems that the textbook question is an implicit argument against I.J. Good. Good may be treating intelligence as valuable, simply because man's strengths lies in that trait called &quot;intelligence&quot;. But other traits could be equally valued instead (speed or strength) and sentient beings may speculate wildly about their preferred traits being &quot;maximized&quot; by some machine or another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes me wonder whether a singularity could occur if we had built machines that were &lt;em&gt;not&lt;/em&gt; maximizing intelligence, but instead maximizing some other trait (a machine that is always increasing its strength, or a machine that is always increasing its speed). These types of machines can be just as transformative - ultrafast machines may solve problems quickly due to &quot;brute force&quot;, and ultrastrong machines can use its raw power for a variety of physical tasks. Perhaps a ultra-X machine can't build another ultra-X machine (as I.J. Good treated the design of machines as an intellectual activity), but a continually self-improving machine would still leave its creators far behind and force its creators to be dependent on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are technological singularities limited to ultra-intelligences?&lt;/em&gt; Or technological singularities be caused by machines that are not &quot;strong AI&quot; but are still &quot;ultra&quot;-optimizers?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="2444" LastEditDate="2019-06-25T22:19:02.223" LastActivityDate="2019-06-25T22:19:02.223" Title="Can a technological singularity only occur with superintelligence?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;singularity&gt;&lt;superintelligence&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2842" PostTypeId="1" CreationDate="2017-02-19T00:46:36.947" Score="0" ViewCount="198" Body="&lt;p&gt;Is Programming Collective Intelligence by Toby Segaran a good book to enter in the AI and neural networks world for a novice? &lt;/p&gt;&#xA;" OwnerUserId="5549" LastEditorUserId="101" LastEditDate="2017-06-17T21:22:03.137" LastActivityDate="2017-06-17T21:22:03.137" Title="Programming Collective Intelligence" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2843" PostTypeId="2" ParentId="2842" CreationDate="2017-02-19T03:27:05.573" Score="1" Body="&lt;p&gt;I read the second chapter in that book for my project on recommendation systems. &lt;strong&gt;It is an awesome book and it starts by addressing concepts from the basics&lt;/strong&gt;( At least chapter 2 was like that). Book explains concepts with python. &lt;strong&gt;You will need a bit of knowledge on python basics&lt;/strong&gt; . The book transits slowly from beginner to an intermediate level. Hope this answers your question. I would &lt;strong&gt;recommend&lt;/strong&gt; it.&lt;/p&gt;&#xA;" OwnerUserId="5557" LastActivityDate="2017-02-19T03:27:05.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2844" PostTypeId="2" ParentId="2841" CreationDate="2017-02-19T07:32:06.433" Score="4" Body="&lt;p&gt;That would be a no for speed or strength, if you have a super strong entity but it cannot research new materials, it will be quickly limited, same thing for speed, Basically, you need something out of their field to improve them, which makes a runaway improvement impossible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though, we already have super strong and super fast machines, those are cars, trucks, hydraulic presses, industrial exoskeletons etc... But, even though we can build better ones through the use of the old ones, we still need to research stuff that can't be improved by old ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we need for a singularity is a field where an improvement in it makes further improvement easier. And I don't know a field where this doesn't involve intelligence. &#xA;If there is one, that may be possible to have a non intelligence driven singularity there.&lt;/p&gt;&#xA;" OwnerUserId="4152" LastActivityDate="2017-02-19T07:32:06.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2845" PostTypeId="2" ParentId="2772" CreationDate="2017-02-19T16:19:38.423" Score="0" Body="&lt;p&gt;Why dont you try android Phones with Tensorflow &lt;a href=&quot;https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow Android Camera Demo&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can build a simple image or text classification neural network to demonstrate AI. &lt;/p&gt;&#xA;" OwnerUserId="3250" LastActivityDate="2017-02-19T16:19:38.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2846" PostTypeId="1" CreationDate="2017-02-19T18:49:37.057" Score="7" ViewCount="88" Body="&lt;p&gt;AI is developing at a rapid pace and is becoming very sophisticated. One aspect will include the methods of interaction between AI and humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the interaction is an elementary interaction of voice and visual text or images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there current research on more elaborate multisensory interactions?&lt;/p&gt;&#xA;" OwnerUserId="5583" LastEditorUserId="33" LastEditDate="2017-04-11T01:07:47.427" LastActivityDate="2017-04-11T22:01:53.150" Title="Is there a central focus on the communication methods between AI and humans?" Tags="&lt;ai-design&gt;&lt;agi&gt;&lt;natural-language-processing&gt;&lt;hci&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2847" PostTypeId="2" ParentId="2841" CreationDate="2017-02-19T22:01:19.157" Score="0" Body="&lt;p&gt;&quot;Monte Carlo&quot; seems to be the best method currently for algorithmic creativity.  (i.e. the machine makes random choices and sees if they lead to anything useful.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it appears obvious that creative connections formed out of &lt;em&gt;understanding&lt;/em&gt; are superior to those which are random, if the machine is fast enough, it should be able to win out by pure &quot;brute force&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. Evolution, prior to human guidance, has not been been based on intelligence.*  Rather, evolution has been based on random mutations that are either beneficial or detrimental. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*The caveat is that humans creating algorithms and altering genes (either in a lab or through animal husbandry and horticulture) can be said to comprise a new form of evolution that is actually rooted in human intelligence and desire.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-02-19T22:08:16.117" LastActivityDate="2017-02-19T22:08:16.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2848" PostTypeId="2" ParentId="2846" CreationDate="2017-02-20T02:12:22.580" Score="2" Body="&lt;p&gt;Probably these days it's still under the umbrella of &quot;man-machine interaction&quot; in CS, i.e. there is a (sub-) field for interactions between humans and machines in CS, but I am not aware that it has split again to create a sub-sub-field for AI/man interactions. &lt;/p&gt;&#xA;" OwnerUserId="5589" LastActivityDate="2017-02-20T02:12:22.580" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2849" PostTypeId="2" ParentId="2834" CreationDate="2017-02-20T02:15:49.450" Score="0" Body="&lt;p&gt;It depends whether you include learning to produce sound waves in the task (I'm taking &quot;how to speak&quot; to be different from &quot;how to write&quot; in your question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the task is to learn a language, you can certainly learn from written texts, and try a generative approach (usually, after a few sentences, humans can be disappointed though). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the task includes generating a voice response that is also learnt, then it had better hear some spoken language too. Of course, you can bypass the task of learning how to generate sound by plugging a standard text to voice module after your generator. &lt;/p&gt;&#xA;" OwnerUserId="5589" LastActivityDate="2017-02-20T02:15:49.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2850" PostTypeId="2" ParentId="2820" CreationDate="2017-02-20T02:20:25.370" Score="3" Body="&lt;p&gt;IMHO the first hurdle is &lt;strong&gt;scale&lt;/strong&gt;: even Google's largest DNN doesn't come close to the scale of the brain, and by a factor of several orders of magnitude... &lt;/p&gt;&#xA;" OwnerUserId="5589" LastActivityDate="2017-02-20T02:20:25.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2851" PostTypeId="1" CreationDate="2017-02-20T05:27:22.530" Score="1" ViewCount="559" Body="&lt;p&gt;I tried the below Matlab code to build SOM using &lt;code&gt;selforgmap&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;close all, clear all, clc, format compact&#xA;&#xA;% number of samples of each cluster&#xA;K = 200;&#xA;% offset of classes&#xA;q = 1.1;&#xA;% define 4 clusters of input data&#xA;P = [rand(1,K)-q rand(1,K)+q rand(1,K)+q rand(1,K)-q;&#xA;     rand(1,K)+q rand(1,K)+q rand(1,K)-q rand(1,K)-q];&#xA;% plot clusters&#xA;plot(P(1,:),P(2,:),'g.')&#xA;hold on&#xA;grid on&#xA;% SOM parameters&#xA;dimensions   = [10 10];&#xA;coverSteps   = 100;&#xA;initNeighbor = 4;&#xA;topologyFcn  = 'hextop';&#xA;distanceFcn  = 'linkdist';&#xA;&#xA;% define net&#xA;net2 = selforgmap(dimensions,coverSteps,initNeighbor,topologyFcn,distanceFcn);&#xA;&#xA;% train&#xA;[net2,Y] = train(net2,P);&#xA;&#xA;% plot input data and SOM weight positions&#xA;plotsompos(net2,P);&#xA;grid on&#xA;&#xA;% plot SOM neighbor distances&#xA;plotsomnd(net2)&#xA;&#xA;% plot for each SOM neuron the number of input vectors that it classifies&#xA;figure&#xA;plotsomhits(net2,P)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You find the result and more details &lt;a href=&quot;http://lab.fs.uni-lj.si/lasin/wp/IMIT_files/neural/nn07_som/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to segment grayscale image. However, I cannot set the &lt;code&gt;selforgmap&lt;/code&gt; input correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can modify the below code to segment any grayscale image?&lt;/p&gt;&#xA;" OwnerUserId="5591" LastEditorUserId="1671" LastEditDate="2018-01-28T23:04:02.217" LastActivityDate="2018-01-28T23:04:02.217" Title="Image segmentation using SOM" Tags="&lt;neural-networks&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2852" PostTypeId="2" ParentId="2834" CreationDate="2017-02-20T08:42:24.800" Score="0" Body="&lt;p&gt;Yes, It is possible. Also, if you are from development background you can use JAVA Sphinx, some AI tools and API's. If you are from testing background you can use JAWS which actually reads the readable properties of HTML tags. Nowadays, there are some pdf readers, document reader softwares available you can have a look at them. You can search for Sphinx related projects but I don't know about the other projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please vote and mark the solution if useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Thanks!&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="5594" LastActivityDate="2017-02-20T08:42:24.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2853" PostTypeId="1" CreationDate="2017-02-20T09:00:42.567" Score="2" ViewCount="125" Body="&lt;p&gt;I am trying to develop an Editor that can be based on &lt;code&gt;Notepad&lt;/code&gt;. The only purpose for this development is I want to use this for my coding suggestions and possibly the next input parameter that I am going to write. I've seen &lt;code&gt;Notepad++&lt;/code&gt;, &lt;code&gt;EditPlus&lt;/code&gt; etc. and what I think is that this can be definitely achieved. What are the best tools or API's I can use? Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Thanks in Advance!&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="5594" LastActivityDate="2017-02-20T09:00:42.567" Title="To create &quot;Edit tool&quot; or editor based on Notepad" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;algorithm&gt;&lt;world-knowledge&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ClosedDate="2017-02-20T19:13:08.223" ContentLicense="CC BY-SA 3.0" />
  <row Id="2854" PostTypeId="1" CreationDate="2017-02-20T10:08:09.997" Score="0" ViewCount="3263" Body="&lt;p&gt;Is it possible to run SSD or YOLO object detection on Raspberry Pi 3 for live object detection (2/4frames x second)?&#xA;I've tried this &lt;a href=&quot;https://github.com/rykov8/ssd_keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD&lt;/a&gt; implementation but it takes 14 s per frame.&#xA;Is there anything I could do to speed up?&lt;/p&gt;&#xA;" OwnerUserId="2320" LastEditorUserId="2444" LastEditDate="2021-01-29T00:04:15.510" LastActivityDate="2021-01-29T00:04:15.510" Title="Is it possible to run SSD or YOLO object detection on Raspberry Pi 3 for live object detection?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;hardware&gt;&lt;single-shot-multibox-detector&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2855" PostTypeId="2" ParentId="2772" CreationDate="2017-02-20T11:11:30.073" Score="0" Body="&lt;p&gt;To start you could use one of the devices mentioned before, and after to make some more powerful and complicated experiments (and also a little bit more expansive) you could move to &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/B00L7AWOEC&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jetson TK1&lt;/a&gt; which let you run heavier Neural Network (like CNN).&lt;/p&gt;&#xA;" OwnerUserId="2320" LastActivityDate="2017-02-20T11:11:30.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2858" PostTypeId="2" ParentId="2771" CreationDate="2017-02-20T23:22:10.217" Score="4" Body="&lt;p&gt;&amp;quot;Life&amp;quot; is a definition humans use to classify objects according to the types of behavior humans perceive as unique to living creatures.&lt;/p&gt;&#xA;&lt;p&gt;Scientists and philosophers tend to define something as &amp;quot;alive&amp;quot; if it manifests some specific properties found in living organisms, such as self-replication, adaptation to the environment, homeostasis and capability to exploit matter and energy for its own existence and functioning. With that being said, there is no one accepted definition of life, and it is doubtful that such a definition is possible.&lt;/p&gt;&#xA;&lt;p&gt;As to artificial life (ALife), the book &amp;quot;Biological Computation&amp;quot; (2011) by E. Lamm and R. Unger states that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;it is common among researchers to distinguish between two types of approaches to artificial life:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the &lt;strong&gt;strong ALife&lt;/strong&gt; approach, which postulates that virtual &amp;quot;creatures&amp;quot; on a computer screen can be considered to be alive if they fulfill the definition of life used by the researchers; and&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;the &lt;strong&gt;weak ALife&lt;/strong&gt; approach, whereby computerized creatures displaying characteristics of living systems are only models used in research and are not really alive&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;While this is not solving the problem of definition of life, it might give more context on the subject in relation to artificial life.&lt;/p&gt;&#xA;" OwnerUserId="5533" LastEditorUserId="2444" LastEditDate="2020-06-25T12:51:37.033" LastActivityDate="2020-06-25T12:51:37.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2859" PostTypeId="2" ParentId="2837" CreationDate="2017-02-21T09:50:42.437" Score="1" Body="&lt;p&gt;The human nervous system is an &lt;em&gt;extremely&lt;/em&gt; dynamic entity, having been formed by the processes of embryogenesis, mediated by various markers guiding neurons to grow to specific areas, and all of it laid down by millions of years of evolution. There are many different varieties of neurons in the human brain, indeed more so than pretty much any other animal on this planet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is hard to see that a &lt;em&gt;spiking neural network&lt;/em&gt; (SNN) would be able to get all these details correct when we don't yet fully understand the biological analog.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think for this to be successful, we need to understand much more about not only the variety of neurons in the brain, but details of the the embryogenic dynamics as well. And not just the neurons, but the glial cells as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, there is something to be said for using evolutionary approaches to resolve &quot;a solution&quot; that may work anyway. Taking this approach will require many more resources, but is perhaps doable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, I would &lt;strong&gt;not&lt;/strong&gt; expect a naive attempt of the neuromorphic SNN to succeed. There is a &lt;em&gt;lot&lt;/em&gt; of complexity involved in what the brain does, and it involves the glial cells to a large degree. Do we understand enough about the role of glial cells in the brain? We cannot ignore them. Are they only performing &quot;housekeeping&quot; operations, like, for example, the uptake of &quot;spent&quot; neurotransmitters? Or are they doing more, taking part in the computational and / or memory aspects of the brain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is much research in this and other pertinent areas that one should look into. And expect a lot of surprises.&lt;/p&gt;&#xA;" OwnerUserId="4185" LastActivityDate="2017-02-21T09:50:42.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2860" PostTypeId="2" ParentId="2803" CreationDate="2017-02-21T20:39:21.507" Score="2" Body="&lt;p&gt;It is true that at first look, one could expect that classification between 101 categories would be harder than classification between 51 categories. However, many aspects play a role when it comes to action recognition applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the HMDB51 contains several categories about different facial movements like smiling, laughing, chewing,... and several other categories like eating, drinking. Such categories are not present in the list of categories of the data set UCF101 and are obviously among the most difficult categories to deal with. It also claims to have some bad quality challenging videos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is hard to predict in advance how difficult a data set will be to classify. We can imagine that when the state-of-the-art reach accuracy beyond 90%, it is time to build a data set that makes these methods fail to look for even more robust solutions. I don't know these data sets well, but the videos are present most probably more variability in terms of viewpoint, camera motions, illumination changes, image quality,... in the most difficult to classify data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, check on &lt;a href=&quot;http://crcv.ucf.edu/data/UCF101.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;, the results announced for the UCF101 data set. I don't know where you found you accuracy value because the official website announces less than 43.9%. Some publications do not use the complete data set and use only part of it to show the performance of an approach they designed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the &lt;a href=&quot;http://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database/#overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;official website of the HMDB51 data set&lt;/a&gt; reports the following:&#xA;&quot;The UCF group has also been collecting action datasets, mostly from YouTube. There are UCF Sports featuring 9 types of sports and a total of 182 clips, UCF YouTube containing 11 action classes, and UCF50 contains 50 actions classes. We will show in the paper that videos from YouTube could be very biased by low-level features, meaning low-level features (i.e., color and gist) are more discriminative than mid-level fears (i.e., motion and shape).&quot;&#xA;This could also explain why better results can be achieved...&lt;/p&gt;&#xA;" OwnerUserId="3576" LastEditorUserId="11061" LastEditDate="2018-04-29T14:29:13.970" LastActivityDate="2018-04-29T14:29:13.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2861" PostTypeId="1" CreationDate="2017-02-22T05:07:17.853" Score="4" ViewCount="755" Body="&lt;h3&gt;TL;DR&lt;/h3&gt;&#xA;&lt;p&gt;If we buy into the idea visual cortex functions like a convolutional neural network, then there's a problem makes me scratch my head: &lt;strong&gt;how does brain force weight sharing as in convolutional network&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;h3&gt;Okay, explain more&lt;/h3&gt;&#xA;&lt;p&gt;Obviously, there's no way for left visual cortex to directly tell the right visual cortex &amp;quot;hey, I've learned some new stuff, copy me!!&amp;quot; (or is there?). Then, if the learned features are diverse across visual field, how does it keep the translation invariance property?&lt;/p&gt;&#xA;&lt;p&gt;For example, you already know English characters, you can recognize them with your both eyes. Now that you wanna learn some Chinese and you  excercise your right brain at the same time, so you closed your right eye and memorized a new character. After that, certainly you can recognize the new character with solely your right eye. But &lt;em&gt;why&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;The answer may be, the object / higher-level feature detection happens in a higher level cortex, which receives entire visual field. There may be also some transfer/one-shot learning taking place. But then, if a newborn baby trying to learn the low level visual features, he/she would definitely face the weight sharing problems.&lt;/p&gt;&#xA;&lt;p&gt;A possible explanation would be, the baby will be exposed to very large amount of data and eventually learn invariance. Large amount of data reduces overfitting but doesn't guarantee &lt;em&gt;deterministic convergence&lt;/em&gt;. If we train the same CNN model on the same dataset, however using &lt;em&gt;different random generator seeds&lt;/em&gt;, there's a big chance the same feature detector will appear in a different channel, or a difference set of features appear as linear recombination.&lt;/p&gt;&#xA;&lt;p&gt;If there's no way to share weights, the brain would learn &lt;strong&gt;a lot different feature combinations across the entire visual field, how does it still able to consistently solve visual invariance problem?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3189" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-02-22T10:11:26.350" Title="How does visual cortex share convolution weight" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2862" PostTypeId="2" ParentId="2861" CreationDate="2017-02-22T10:11:26.350" Score="3" Body="&lt;p&gt;In the human brain every common pattern is recognised by a multitude of pattern recognisers (be they neurons or microcolumns). That's pretty obvious because neurons die all the time, but we don't wake up and have forgotten how to recognise the letter 'A'. In fact we have to take out rather big chunks of the neocortex until functionality degrades, which is why Alzheimer's patients show symptoms only when the brain is already visibly messed up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Translational invariance within a level of the hierarchy has to be learned. Basically you need filters for the same edge all over your V1. An object moving across your field of vision only becomes invariant in it's representation on a higher level. Unfortunately we don't magically learn a representation that we can easily turn and twist, our ability to recognise objects from an unusual angle degrades with how uncommon the angle is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice thought experiment to illustrate the point: Imagine a cube sitting in front of you. Now pull one of the corners of the cube upwards until the cube is dangling in front of you, one corner pointing straight to the ceiling, the opposite corner pointing to the ground. Now indicate with your finger where the rest of the corner are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are anything like me, this is really difficult. I think the first time I did't even realise I had to point out six corners! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this is 3D and we might still have inbuilt 2D invariance, but it also turns out that faces that are turned upside down have to be processed much higher into the cortex to be recognised as faces and so on ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning the learning of a lot of different feature vectors across the field of vision: This might be prevented by the fact that the neocortex learns sequences of input by predicting (via depolarisation) the next pattern. So you might have the situation that a higher level tells the lower level that there is an object moving from left to right and the lower level will predict that the edge it is detecting at point y will reoccur at point y+x.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This setup differs from the training of NNs in two ways: The data is already translational and the translation is predicted, which facilitates learning the same features. Basically two pattern recogniser close to each other get pretty much the same pattern one after the other whenever something moves in a certain direction and the second occurrence is predicted, which means it will be more likely detected, which means it will be learned. (I don't want to dole out a lecture about the cortex, but the prediction is a big deal because it allows neurons to fire quicker which means they beat other neurons before they are laterally inhibited and only if they actually fire will they learn.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a disclaimer I want to add that this is just my current understanding of the issue and I'm pretty sure the actual explanation is more complicated. For example I've read that one of the layers of the cortex is important for translational invariance and this layer only exists in the levels close to the sensory input. It is conjectured that this layer (L4) doesn't do the sequence prediction stuff, so maybe having the same kind of input is enough or learning different feature vectors is not actually a problem. There is also the complicating issue that there is poorly understood interplay between the different layers and different levels of the cortex. I would recommend to ask a neuroscientist, except I don't think they figured this out yet. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-02-22T10:11:26.350" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2863" PostTypeId="1" CreationDate="2017-02-22T16:45:08.400" Score="2" ViewCount="609" Body="&lt;p&gt;I have to translate the following English sentences into First-Order Logic without using quantifiers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. Everyone on flight 815 has a story.&#xA;2. No one knows what is inside the hatch.&#xA;3. Someone on the island isn’t on the flight manifest.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried it, but can't translate without using ∀ and ∃:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. ∀x fight815(x) → story(x)&#xA;2. ∀x ⌐(knows(x) → inside hatch(x)) // not sure about this&#xA;OR&#xA;¬ ∃x Knows(x, inside hatch)&#xA;3. ∃x island(x) Λ ⌐(flight manifest(x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it possible to do it. If not, why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refer chapter 8 of Artificial Intelligence: A Modern Approach (3rd edition). Stuart Russell and Peter Norvig, Prentice Hall (2010)&lt;/p&gt;&#xA;" OwnerUserId="5643" LastActivityDate="2017-02-22T16:45:08.400" Title="Translate English Sentences into First-Order Logic without quantifiers" Tags="&lt;knowledge-representation&gt;&lt;world-knowledge&gt;&lt;logic&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2864" PostTypeId="1" AcceptedAnswerId="2881" CreationDate="2017-02-22T20:52:28.450" Score="1" ViewCount="368" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Evaluation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; states that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;An evaluation function, also known as a heuristic evaluation function or static evaluation function, is a function used by game-playing computer programs to estimate the value or goodness of a position (usually at a leaf or terminal node) in a game tree.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is &amp;quot;goodness&amp;quot; an actual term in use in this context, or should it more properly be something like &amp;quot;perceived optimality&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;I ask because, for instance, in Combinatorial Game Theory, a lighthearted term such as &amp;quot;loopy&amp;quot; is preferred by some mathematicians (Demaine) over the more serious term &amp;quot;cyclic&amp;quot; (Fraenkel).&lt;/p&gt;&#xA;&lt;p&gt;On a related note, is the use of &amp;quot;position&amp;quot; instead of &amp;quot;node&amp;quot; preferred here as an acknowledgment of the heuristic nature of evaluation functions?  (My understanding is that &amp;quot;position&amp;quot;, &amp;quot;node&amp;quot; and &amp;quot;game&amp;quot; may all be interchangeable in certain contexts.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-02-06T20:07:33.760" LastActivityDate="2021-02-06T20:07:33.760" Title="Is it standard to say that an evaluation function estimates the “goodness” of a position?" Tags="&lt;terminology&gt;&lt;game-ai&gt;&lt;evaluation-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2865" PostTypeId="1" CreationDate="2017-02-23T14:41:58.773" Score="4" ViewCount="95" Body="&lt;p&gt;A cost function used in machine is often the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C = \frac{1}{2} \| y - \hat{y} \| ^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is there &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; in front of the squared distance?&lt;/p&gt;&#xA;" OwnerUserId="5661" LastEditorUserId="2444" LastEditDate="2019-04-01T13:56:58.730" LastActivityDate="2019-04-04T17:36:23.830" Title="Why does the cost function contain a 2 at the denominator?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2866" PostTypeId="2" ParentId="2865" CreationDate="2017-02-23T15:17:53.180" Score="4" Body="&lt;p&gt;To simplify the derivative, probably. Otherwise there will be constant 2 in it. &lt;/p&gt;&#xA;" OwnerUserId="5657" LastActivityDate="2017-02-23T15:17:53.180" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2867" PostTypeId="1" AcceptedAnswerId="2869" CreationDate="2017-02-23T21:11:34.133" Score="9" ViewCount="353" Body="&lt;p&gt;I read that deep neural networks can be relatively easily fooled (&lt;a href=&quot;https://ai.stackexchange.com/questions/92/how-is-it-possible-that-deep-neural-networks-are-so-easily-fooled&quot;&gt;link&lt;/a&gt;) to give high confidence in recognition of synthetic/artificial images that are completely (or at least mostly) out of the confidence subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I don't really see a big problem with DNN giving high confidence to those synthetic/artificial images but I think giving high confidence for white noise (&lt;a href=&quot;https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks&quot;&gt;link&lt;/a&gt;) may be a problem since this is a truly natural phenomenon that may the camera see in the real world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much of a problem is white noise for the real-world usage of a DNN? Can such false positives be detected from plain noise be prevented somehow?&lt;/p&gt;&#xA;" OwnerUserId="113" LastEditorUserId="2444" LastEditDate="2019-09-09T11:13:34.653" LastActivityDate="2019-09-09T11:13:34.653" Title="How much of a problem is white noise for the real-world usage of a DNN?" Tags="&lt;neural-networks&gt;&lt;deep-neural-networks&gt;&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;&lt;signal-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2868" PostTypeId="1" CreationDate="2017-02-24T06:03:02.730" Score="4" ViewCount="101" Body="&lt;p&gt;I identify myself as a human agent. It is time to think about oncoming senior research and due to small experience in gamedev(as well as in AI field), some questions are raised. What are the most suitable approaches to implement real-time &lt;em&gt;simple&lt;/em&gt; AI agent in an action game? I've heard something about cognitive architecture like ACT-R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By design, entity's AI can have several mutually exclusive states. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/XMXKl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XMXKl.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an existing AI of game, which has states, events and schedules. However, the code is complicated and not flexible. Also, it does not use any cognitive architecture, which I consider as a drawback.&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=9jO-P3kXlCI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=9jO-P3kXlCI&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, using your experience suggest any modern techniques, which can copy such behaviour as in image or video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your perception.&lt;/p&gt;&#xA;" OwnerUserId="5673" LastEditorUserId="5673" LastEditDate="2017-02-24T10:15:57.220" LastActivityDate="2017-02-28T20:16:52.163" Title="Different useful approaches of implementing real-time AI?" Tags="&lt;gaming&gt;&lt;real-time&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2869" PostTypeId="2" ParentId="2867" CreationDate="2017-02-24T11:04:39.957" Score="7" Body="&lt;p&gt;The white noise that fools DNNs isn't really white noise. It has been altered in the same way as the synthetic misclassified pictures have been altered. You have to change many input pixels in exactly such a way, that these little changes aren't perceptible, but propagated through the network add up to a misclassification. This is not going to happen by chance. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-02-24T11:04:39.957" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2870" PostTypeId="1" CreationDate="2017-02-25T07:08:22.783" Score="6" ViewCount="262" Body="&lt;p&gt;This question has come from my experiment of building a cnn based tic-tac-toe game that I'm using as a beginner machine learning project. The game works purely on policy networks, more specifically -&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;During training, at the end of each game, it trains itself on the moves the winner/drawer made for each board position. That is, its training data consists of board positions and the moves made by the winning player on each position.&lt;/li&gt;&#xA;&lt;li&gt;While playing, it predicts its own moves solely based on that training (that is, it predicts what move would a winning player make with the current board). It doesn't use any type of search or value networks.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm seeing that if I train it against a player that predicts the perfect move (using a recursive search) every time, the AI gets good at drawing about 50% games. But if I train it against a player that makes random moves, it doesn't get better at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wouldn't one expect it to learn well (even if slower) regardless of the level of its opponent? Since each game ends in a draw or win for one player, shouldn't it be able to extract features for the winning/drawing strategies even when learning from random players? Or does this behavior mean that the model is not optimal?&lt;/p&gt;&#xA;" OwnerUserId="1522" LastEditorUserId="10135" LastEditDate="2018-10-19T18:36:49.603" LastActivityDate="2018-10-19T18:36:49.603" Title="Can a purely policy convolution neural network based game learn to play better than its opponents?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2871" PostTypeId="1" CreationDate="2017-02-25T11:55:59.480" Score="6" ViewCount="152" Body="&lt;p&gt;This mostly refers to human-like or chatbot AI, but could maybe be used in other applications (math or something?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, it occurred to me, that when I'm thinking or speaking, there is a constant feedback loop, in which I am formulating which words to use next, which sentences to form, and which concepts to explore, based on my most recent statements and the flow of the dialogue or monologue. I'm not just responding to outside stimulus but also to myself. In other words, I am usually maintaining a train of thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can AI be made capable of this? If so, has it been demonstrated? And to what extent? While typing this, I discovered the term &quot;thought vectors&quot;, and I think it might be related. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I read correctly, thought vectors have something to do with allowing AI to store or identify the relationships between different concepts; and if I had to guess, I'd say that if an AI lacks a strong understanding of the relationships between concepts, then it would be impossible for it to maintain a coherent train of thought. Would that be a correct assumption?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ps. in my limited experience with AI chatbots, they seem to be either completely scripted, or otherwise random and often incoherent, which is what leads me to believe that they do not maintain a train of thought)&lt;/p&gt;&#xA;" OwnerUserId="5698" LastActivityDate="2017-02-28T20:03:56.887" Title="Can an AI be made to maintain a train of thought?" Tags="&lt;ai-design&gt;&lt;chat-bots&gt;&lt;thought-vectors&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2872" PostTypeId="1" AcceptedAnswerId="2877" CreationDate="2017-02-25T14:57:25.610" Score="8" ViewCount="777" Body="&lt;p&gt;I had first this question in mind &quot;Can an AI suffer?&quot;. Suffering is important for human beings. Imagine that you are damaging your heel. Without pain, you will continue to harm it. Same for an AI. But then I told myself &quot;&lt;em&gt;Wait a second. It already exists. It is the errors and warnings that shows up&lt;/em&gt;&quot;. We can say it has the similar purpose as suffering. However, I felt something missing. We feel pain. The errors and bugs are just data. Let's say a robot can use machine learning and genetic programming to evolve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can an AI learn to suffer? And not just know it as mere information.&lt;/p&gt;&#xA;" OwnerUserId="1760" LastEditorUserId="2444" LastEditDate="2019-03-31T12:35:35.550" LastActivityDate="2021-04-20T09:07:25.460" Title="Can an AI learn to suffer?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2874" PostTypeId="1" AcceptedAnswerId="5853" CreationDate="2017-02-25T21:46:50.777" Score="6" ViewCount="903" Body="&lt;p&gt;Can neural networks efficiently solve the traveling salesmen problem? Are there any research papers that show that neural networks can solve the TSP efficiently?&lt;/p&gt;&#xA;&lt;p&gt;The TSP is an NP-hard problem, so I suspect that there are only approximate solutions to this problem, even with neural networks. So, in this case, how would &lt;em&gt;efficiency&lt;/em&gt; be defined?&lt;/p&gt;&#xA;&lt;p&gt;In this context, it seems that the time efficiency may be obtained by resource inefficiency: by making the neural network enormous and simulating all the possible worlds, then maximizing. So, while time to compute doesn't grow much as the problem grows, the size of the physical computer grows enormously for larger problems; how fast it computes is then, it seems to me, not a good measure of the efficiency of the algorithm in the common meaning of efficiency. In this case, the resources themselves only grow as fast as the problem size, but what explodes is the number of connections that must be built. If we go from 1000 to 2000 neurons to solve a problem twice as large and requiring exponentially as much time to solve, the algorithms requiring only twice as many neurons to solve in polynomial time seem efficient, but, really, there is still an enormous increase in connections and coefficients that need be built for this to work.&lt;/p&gt;&#xA;&lt;p&gt;Is my above reasoning incorrect?&lt;/p&gt;&#xA;" OwnerUserId="1366" LastEditorUserId="2444" LastEditDate="2021-01-18T00:32:59.527" LastActivityDate="2021-01-18T00:32:59.527" Title="Can neural networks efficiently solve the traveling salesmen problem?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;time-complexity&gt;&lt;efficiency&gt;&lt;travelling-salesman-problem&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2875" PostTypeId="1" CreationDate="2017-02-25T21:57:43.563" Score="3" ViewCount="161" Body="&lt;p&gt;For rules please refer to &lt;a href=&quot;https://www.hackerrank.com/challenges/ultimate-ttt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.hackerrank.com/challenges/ultimate-ttt&lt;/a&gt; .I have implemented minimax search with alpha-beta pruning. There is a time limit of 15 seconds.Which algorithms would yield better results? &lt;/p&gt;&#xA;" OwnerUserId="5706" LastActivityDate="2017-02-25T21:57:43.563" Title="Which algorithm is best for a 4*4*4*4 variant of ultimate tic-tac toe?" Tags="&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ClosedDate="2017-03-15T18:15:02.293" ContentLicense="CC BY-SA 3.0" />
  <row Id="2876" PostTypeId="1" AcceptedAnswerId="2878" CreationDate="2017-02-26T01:04:39.363" Score="11" ViewCount="824" Body="&lt;p&gt;In the context of artificial intelligence, the singularity refers to the advent of an &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity#Intelligence_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;artificial general intelligence capable of recursive self-improvement, leading to the rapid emergence of artificial superintelligence (ASI), the limits of which are unknown, shortly after technological singularity is achieved&lt;/em&gt;&lt;/a&gt;. Therefore, these superintelligences would be able to solve problems that we possibly are unable to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to a poll reported in &lt;a href=&quot;https://nickbostrom.com/papers/survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Future progress in artificial intelligence: A survey of expert opinion&lt;/a&gt; (2014)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The median estimate of respondents was for a one in two chance that highlevel machine intelligence will be developed around 2040-2050&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which isn't very far away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is wrong with the idea that the AI will be capable of omniscience, given that it could benefit us by solving many problems?&lt;/p&gt;&#xA;" OwnerUserId="5708" LastEditorUserId="2444" LastEditDate="2019-11-21T03:38:41.257" LastActivityDate="2020-12-05T02:38:19.660" Title="What is wrong with the idea that the AI will be capable of omniscience?" Tags="&lt;philosophy&gt;&lt;superintelligence&gt;&lt;cognitive-science&gt;&lt;singularity&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2877" PostTypeId="2" ParentId="2872" CreationDate="2017-02-26T01:09:27.410" Score="9" Body="&lt;p&gt;At a very high level, regarding evolutionary game theory and genetic algorithms, it is absolutely possible that AI could develop a state that is analogous with suffering, although, as you astutely point out, it would involve conditions that a computer cares about.  (For instance, it might develop a feeling analogous to &amp;quot;being aggrieved&amp;quot; over non-optimality in the algorithmic sense, or &amp;quot;frustration&amp;quot; at equations don't add up, or &amp;quot;dissatisfaction&amp;quot; over goals that have not been achieved.)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/children-beating-up-robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;The robot tormented by small children at the mall&lt;/a&gt; can certainly be said to be &amp;quot;suffering&amp;quot; in that the children block the performance of the robot's function, but the robot is not conscious and suffering might be said to require awareness.  However, even without consciousness, this very simple robot can learn new behaviors through which it mitigates or avoids the &amp;quot;suffering&amp;quot; brought on by not being able to fulfill its function.&lt;/p&gt;&#xA;&lt;p&gt;You definitely want to look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Suffering#Philosophy&quot; rel=&quot;nofollow noreferrer&quot;&gt;the concept of suffering in a philosophical context&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Epicurus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Epicurus&lt;/a&gt; would be a very useful place to start.&lt;/p&gt;&#xA;&lt;p&gt;Epicurus is directly relevant in an algorithmic sense because he uses the term &amp;quot;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=ataracia&amp;amp;la=greek#lexicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;ataraxia&lt;/a&gt;&amp;quot; meaning calm, and is derived from the verb &amp;quot;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=tarassw&amp;amp;la=greek&quot; rel=&quot;nofollow noreferrer&quot;&gt;tarasso&lt;/a&gt;&amp;quot; which means to agitate or disturb.&lt;/p&gt;&#xA;&lt;p&gt;Ataraxia can be mathematically expressed as an equilibrium.  Tarasso can be mathematically expressed as disequilibrium.&lt;/p&gt;&#xA;&lt;p&gt;This relates directly to Game Theory in that disequilibrium can be said to be the primary requirement of games, and to AI in that Game Theory can be said to be the root of all AI.&lt;/p&gt;&#xA;&lt;p&gt;Ataraxia is also understood in the sense of &amp;quot;freedom from fear&amp;quot;, which is temporal in that fear is a function of uncertainty as it relates to the future in a predictive sense, and involves current condition vs. possible, less optimal future conditions.&lt;/p&gt;&#xA;&lt;p&gt;Thus fear, which is a form of suffering, is rooted in computational intractability, even where the &amp;quot;computer&amp;quot; is is a human brain.&lt;/p&gt;&#xA;&lt;p&gt;Early philosophers such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Democritus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Democritus&lt;/a&gt; are especially useful because they were exploring critical, fundamental concepts, many of which can now be expressed with modern mathematics.&lt;/p&gt;&#xA;&lt;p&gt;To wit: you can't arrive at suffering until you first define &amp;quot;the Good&amp;quot; and &amp;quot;the Bad&amp;quot;, which is a binary relationship in which neither term can be said to have meaning without the opposite.  (Mathematically, it can be expressed in its simplest form as a finite, one-dimensional graph.)  This understanding is quite ancient.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;It is worth noting that the continuing value of the early philosophers is partly a factor of wisdom not being dependent on the volume of knowledge, demonstrated by Socrates in the idea that wisdom may be as simple as knowing you don't know something.&lt;/p&gt;&#xA;&lt;p&gt;The ancient sages didn't have the benefit of powerful measurement tools, advanced mathematics, or scientific method, but they were very smart, and even more importantly, wise.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="36737" LastEditDate="2021-04-19T22:21:07.023" LastActivityDate="2021-04-19T22:21:07.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2878" PostTypeId="2" ParentId="2876" CreationDate="2017-02-26T01:47:06.527" Score="10" Body="&lt;p&gt;I quite like your outlook, and without getting into the details of how a &quot;singularity&quot; may be effected which is covered in numerous other questions, or how consciousness and &quot;omniscience&quot; come into play because &lt;a href=&quot;https://en.wikipedia.org/wiki/Grey_goo&quot; rel=&quot;noreferrer&quot;&gt;consciousness and omniscience are not requirements&lt;/a&gt;, I will instead direct you to two key philosophers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Phillip K. Dick, for whom the central theme &lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F&quot; rel=&quot;noreferrer&quot;&gt;in his famous 1968 book on AI&lt;/a&gt; is empathy.  &lt;em&gt;(If you haven't read it, I'm not posting a spoiler, but will only say the plot is driven by the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;noreferrer&quot;&gt;Evolutionary Game Theory&lt;/a&gt; which was formalized just 5 years later.)&lt;/em&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;John Nash, and in particular, the concept of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nash_equilibrium&quot; rel=&quot;noreferrer&quot;&gt;Nash Equilibrium&lt;/a&gt;. &lt;em&gt;(Nash could be said to have mathematically demonstrated that being a &quot;douchebag&quot; is not an optimal strategy.  His proof can be used to explain why &lt;a href=&quot;https://en.wikipedia.org/wiki/Mutual_assured_destruction&quot; rel=&quot;noreferrer&quot;&gt;nuclear détente actually worked&lt;/a&gt;, which was counter to the expectation of Von Neumann.)&lt;/em&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So when people go nuts, focusing on the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Skynet_(Terminator)&quot; rel=&quot;noreferrer&quot;&gt;Skynet&lt;/a&gt;&quot; mythos under which machines rise up to destroy us, I have to wonder if they're simply not as smart as Nash or as profound as Dick, which might explain their lack of emphasis on what can be called the &quot;&lt;a href=&quot;https://plato.stanford.edu/entries/altruism-biological/&quot; rel=&quot;noreferrer&quot;&gt;Electric Sheep&lt;/a&gt;&quot; paradigm.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-02-26T01:52:35.727" LastActivityDate="2017-02-26T01:52:35.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2881" PostTypeId="2" ParentId="2864" CreationDate="2017-02-26T03:13:27.113" Score="3" Body="&lt;p&gt;Yes, &quot;goodness&quot; is a common description of the value generated by an evaluation function.&#xA;For example,&#xA;&lt;a href=&quot;https://books.google.com/books?id=DDNHzcN6jasC&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Artificial Intelligence&quot;&lt;/a&gt; p. 77;&#xA;&lt;a href=&quot;https://books.google.com/books?id=_iE4uvEp4CMC&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Knowledge-Free and Learning-Based Methods in Intelligent Game Playing&quot;&lt;/a&gt; p. 15;&#xA;&lt;a href=&quot;https://books.google.com/books?id=jNfvAgAAQBAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Tenth Scandinavian Conference on Artificial Intelligence&quot;&lt;/a&gt; p. 125; and&#xA;&lt;a href=&quot;https://books.google.com/books?id=C_k7XVeCTY4C&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Algorithms and Networking for Computer Games&quot;&lt;/a&gt; p. 80.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;position&quot; is to the complete state of the game at some instant. A single position includes the locations of every piece on a game board at that instant, and also includes a few other bits of information -- who's turn it is; whether castling is still a legal option; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that &quot;the position&quot; is a synecdoche, referring to the total state by a small part, the physical location (physical position) of some piece on the game board, based on earlier usage in military tactics of &quot;positioning&quot; troops relative to natural cover or relative to enemy troops.&lt;/p&gt;&#xA;" OwnerUserId="5711" LastEditorUserId="5711" LastEditDate="2018-03-24T16:19:06.777" LastActivityDate="2018-03-24T16:19:06.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2882" PostTypeId="2" ParentId="2871" CreationDate="2017-02-26T09:26:38.610" Score="1" Body="&lt;p&gt;First, for almost every question of the form &quot;Can AI be made to X&quot;, the most obvious and straightforward answer is something like &quot;We don't know. Probably, but if it hasn't been done yet, we're really not sure.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also important to understand that, from a technology standpoint, there isn't one &quot;thing&quot; called &quot;AI&quot;.  There are many, many different technologies, which are loosely related (at best) and are generally lumped together under the overall rubric of &quot;Artificial Intelligence&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of that said, yes, there has been work on adding memory, even long-term memory, to various kinds of &quot;AI&quot;.  The most notable recent example is the advent of &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt; in recurrent neural networks.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, some of the work done on &quot;cognitive architectures&quot; has focused on the use of memory.  For more info on that, look up ACT-R and/or SOAR and read some of those papers.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What isn't clear to me offhand, is whether or not anybody has tried applying any of these techniques to chat-bots in particular.  I wouldn't be surprised if somebody had, but I can't cite any such research off the top of my head.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-02-26T09:26:38.610" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2883" PostTypeId="2" ParentId="2806" CreationDate="2017-02-26T09:47:48.923" Score="1" Body="&lt;p&gt;If the only feature you're classifying on is the number of users making a given report, then this isn't really much to do with AI/ML. Just pick a number based on your subjective judgment and go with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OTOH, if you can include details of the report itself (as well as the number of reporters), I think you might be able to build a bayesian classifier that would be useful.  If you could consider location, weather, time of day, number of reporters, etc., it seems like you might be able to get something useful put together.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-02-26T09:47:48.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2886" PostTypeId="1" AcceptedAnswerId="2889" CreationDate="2017-02-26T22:22:00.153" Score="4" ViewCount="151" Body="&lt;p&gt;I'm doing a little tic-tac-toe project to learn neural networks and machine learning (beginner level). I've written a MLP based program that plays with other search based programs and trains with the data generated from the games. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training and evaluation are strictly policy based - Inputs are board positions and outputs are one-hot encoded array that represents the recommended move for that board position. I've not added search algorithms so that I can understand what to expect from a purely MLP approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MLP model has 35 features and 1 hidden layer and after a few hundred thousands games it has sort of learned to draw 50% games. It has learned the basic stuff like how to block the player from winning and some good board placements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my question is - It hasn't learned advanced strategies that require making a move that may not be as beneficial for the current move but will improve its chances later. But should I expect that from a strictly policy MLP based no-search approach? Since all that it is being trained on is one board and the next recommended move (even if thousands of those pairs), is it logical to expect it to learn a lookahead approach that goes beyond &quot;the best move for the current board&quot; training? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put another way, would it be a possible at all for a MLP to learn lookahead without any search strategies? If not, are there any alternatives that can do it without search?&lt;/p&gt;&#xA;" OwnerUserId="1522" LastActivityDate="2017-02-28T08:19:14.450" Title="Is it a valid assumption that a purely MLP based tic-tac-toe player will learn lookahead strategies?" Tags="&lt;machine-learning&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2889" PostTypeId="2" ParentId="2886" CreationDate="2017-02-28T08:19:14.450" Score="3" Body="&lt;p&gt;A MLP only does pattern recognition, it will not learn search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tictactoe, (Oughts and Crosses), is such a simple game that your network should learn the moves from the training data by heart, no generalisation required. If it still loses games, maybe your training data doesn't consist of particularly good moves. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-02-28T08:19:14.450" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2890" PostTypeId="1" CreationDate="2017-02-28T09:30:15.773" Score="4" ViewCount="88" Body="&lt;p&gt;Today we have neural network based AI players that are comparable or better than humans in games that require extensive pattern matching and &quot;intuition&quot;. AlphaGo is a prime example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these AI players usually have both neural networks and search algorithms in place. Humans, on the other hand, rely just on the pattern matching and &quot;intuition&quot; (even the best chess players can see just a handful of moves ahead). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, why do AI players still require extensive search while humans don't? How would AIs like AlphaGo perform if we take the search part out?&lt;/p&gt;&#xA;" OwnerUserId="1522" LastActivityDate="2017-02-28T13:12:34.390" Title="Why do neural networks based AI players still require extensive search techniques?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2891" PostTypeId="1" CreationDate="2017-02-28T10:22:17.403" Score="5" ViewCount="631" Body="&lt;p&gt;I am going to develop an open-domain natural language question-answering (NLQA) system, and will use the support vector machine (SVM) as the machine learning (ML) model for question classification.&lt;/p&gt;&#xA;&lt;p&gt;The data that I have is from a cube, containing multiple dimensions, of which some contain hierarchies.&lt;/p&gt;&#xA;&lt;p&gt;I do not understand how to work/combine the taxonomy and SVM for question classification. If I understand correctly, the taxonomy still needs to be developed by hand, unless an existing one is being used. And the SVM sorts the queried NL question based on this taxonomy?&lt;/p&gt;&#xA;&lt;p&gt;Is this correct, or am I mixing the whole concept?&lt;/p&gt;&#xA;" OwnerUserId="5219" LastEditorUserId="2444" LastEditDate="2020-09-12T17:39:04.543" LastActivityDate="2020-09-12T17:39:04.543" Title="How do I use a taxonomy and the support vector machine for question classification?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2892" PostTypeId="5" CreationDate="2017-02-28T12:50:33.257" Score="0" Body="&lt;p&gt;A multi-layer perceptron (MLP) is a class of feed-forward artificial neural network. An MLP consists of at least three layers of nodes. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called back-propagation for training. Its multiple layers and non-linear activation distinguish MLP from a linear perceptron. It can distinguish data that is not linearly separable.[3]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multilayer_perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLP- Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-26T21:44:24.703" LastActivityDate="2018-03-26T21:44:24.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2893" PostTypeId="4" CreationDate="2017-02-28T12:50:33.257" Score="0" Body="For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model. " OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-26T21:44:30.610" LastActivityDate="2018-03-26T21:44:30.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2894" PostTypeId="2" ParentId="2890" CreationDate="2017-02-28T13:12:34.390" Score="4" Body="&lt;p&gt;It is not very accurate to say that AI players requires extensive search while humans don't. Rather, it is a question of degree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AIs do a lot more calculating, because that's what computers are good at. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human intuition is much more powerful than a neural network can currently hope to match, because it is much more integrated into a world of knowledge about the game, it uses orders of magnitudes more neurons and it is not a static thing that just provides a move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if a human player stops calculating ahead his playing strength will drop very significantly. This can be seen by assessing the performance in games with short time controls: The less time you have the less calculation is happening, your intuition however is fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take out the search component of AlphaGo it would still play quite strongly, probably at a low dan level. Of course that is also far below its strength. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, search is always an important component of playing strength, just more so for machines.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-02-28T13:12:34.390" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2895" PostTypeId="2" ParentId="2871" CreationDate="2017-02-28T20:03:56.887" Score="0" Body="&lt;p&gt;It could be said that &quot;maintaining a thought&quot; is a basic requirement of computing, and can be represented as a string of binary digits in the context of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Machine&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&quot;Basically, it occurred to me, that when I'm thinking or speaking, there is a constant feedback loop, in which I am formulating which words to use next, which sentences to form, and which concepts to explore, based on my most recent statements and the flow of the dialogue or monologue. I'm not just responding to outside stimulus but also to myself. In other words, I am usually maintaining a train of thought.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sounds an awful lot like a recursive function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My analysis of the chatbot problem is that it reveals a poor quality reasoning on the part of the bots, as opposed to lack of reasoning. It's not so much a question on the raw ability of an algorithm to maintain a train of thought, because the &quot;train of though&quot; is the function itself, but the quality of the algorithm and, by some measures, the &quot;humanness&quot; of the output.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-02-28T20:03:56.887" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2896" PostTypeId="2" ParentId="2868" CreationDate="2017-02-28T20:16:52.163" Score="1" Body="&lt;p&gt;About 15 years ago, John Laird's group at Michigan used the Soar rule-based architecture to play several FPS games effectively (Quake II, Descent III):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ai.eecs.umich.edu/people/laird/games_research.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ai.eecs.umich.edu/people/laird/games_research.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's Laird's overview article from 'Computer':&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/profile/John_Laird/publication/2955463_Laird_JE_Using_a_computer_game_to_develop_advanced_AI_Computer_34_70-75/links/54d0f59a0cf20323c21a1bd7/Laird-JE-Using-a-computer-game-to-develop-advanced-AI-Computer-34-70-75.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/profile/John_Laird/publication/2955463_Laird_JE_Using_a_computer_game_to_develop_advanced_AI_Computer_34_70-75/links/54d0f59a0cf20323c21a1bd7/Laird-JE-Using-a-computer-game-to-develop-advanced-AI-Computer-34-70-75.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-02-28T20:16:52.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2897" PostTypeId="1" CreationDate="2017-03-01T09:52:51.610" Score="2" ViewCount="78" Body="&lt;p&gt;A lot of experts have expressed concerns about evil super intelligence. While their concerns are valid, is it necessary, what are the chances or how the artificial super-intelligence will evolve to have selfishness and self protecting desires inherent in biological systems? Is there any work which comments on this line of inquiry?&lt;/p&gt;&#xA;" OwnerUserId="5765" LastActivityDate="2017-03-01T16:55:13.723" Title="Will artificial super-intelligence evolve to have selfishness inherent in biological systems?" Tags="&lt;unsupervised-learning&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2898" PostTypeId="2" ParentId="2897" CreationDate="2017-03-01T11:42:18.690" Score="2" Body="&lt;p&gt;AI will only &quot;evolve&quot; selfishness if it &quot;evolves&quot; in a competitive environment and has certain human-like faculties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-protecting desires on the other hand are logical consequences of having any goal at all. After all, you can't reach your goal if you are destroyed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concern of &quot;evil&quot; super intelligences isn't that they literally turn evil and selfish and cruel. Those are human qualities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead a superintelligence that has a certain goal, will logically pursue subgoals that help it to reach the ultimate goal. Such subgoals will be resources, power, safety. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it will amass power and resources to reach its goal and exterminate any threat to its existence as long as its goal hasn't been reached, without being selfish at all.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-03-01T11:42:18.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2899" PostTypeId="2" ParentId="2897" CreationDate="2017-03-01T16:55:13.723" Score="1" Body="&lt;p&gt;I have some comments on this subject here: &lt;a href=&quot;https://ai.stackexchange.com/a/2878/1671&quot;&gt;https://ai.stackexchange.com/a/2878/1671&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is some deep game theoretic stuff, and it partly depends on how you define &quot;selfishness&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is such a thing, for instance, as a &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Greedy_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;greedy algorithm&lt;/a&gt;&quot;.  Sometimes a greedy algorithm is the most convenient way to achieve an acceptable result, but the optimality is only local. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a mathematical level, the constructiveness or destructiveness of &quot;self interest&quot; in a system may be a function of whether a &lt;a href=&quot;https://en.wikipedia.org/wiki/Nash_equilibrium&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nash Equilibrium&lt;/a&gt; is perceived.  In this case, self-interest is defined as maintaining the current strategy, because, unless the competitor changes their strategy, there is no gain for changing one's own strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the BlindKungFuMaster importantly notes, competitiveness will evolve as a trait if the AI operates in a partisan context.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There, the problem comes from whether the &quot;game&quot; is zero sum (&lt;a href=&quot;https://en.wikipedia.org/wiki/Pareto_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pareto Optimal&lt;/a&gt;) or non zero sum (Pareto Improvable), or both.  Here, &quot;destructive&quot; may defined agents made worse off per the gains made by another agent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Altruism seems to have a rational basis, and occurs in evolution because it is presumably useful. [&lt;a href=&quot;https://plato.stanford.edu/entries/altruism-biological/&quot; rel=&quot;nofollow noreferrer&quot;&gt;See Biological Altruism.&lt;/a&gt;]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this tends to be confined to single species, the co-evolution of dogs and humans is a case for inter-species altruism, based on self interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans and machines have also, and will continue to, co-evolve.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:10.013" LastActivityDate="2017-03-01T16:55:13.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2900" PostTypeId="1" CreationDate="2017-03-01T18:53:00.513" Score="2" ViewCount="123" Body="&lt;p&gt;Post singularity AI will surpass human intelligence. The evolution of AI can take any direction, some of which may not be preferable for humans. Is it possible to manage the evolution of super-intelligent AI? If yes, how? One way I can think of is following. Instead of having a mobile AI like humanoid, we can keep it immobile, like a box, like current super computers. It can be used to solve problems of maths, theoretical science etc.&lt;/p&gt;&#xA;" OwnerUserId="5765" LastEditorUserId="5765" LastEditDate="2017-03-02T13:19:55.283" LastActivityDate="2017-03-11T02:17:56.760" Title="Is it possible to manage the evolution of super-intelligent AI?" Tags="&lt;singularity&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2901" PostTypeId="2" ParentId="2900" CreationDate="2017-03-01T22:31:59.743" Score="2" Body="&lt;p&gt;Assuming super-intelligence is possible, the answer is probably yes and no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes in Kurzweil-like scenarios, where super-intelligence is an extension of human beings by technology (we are already in to some extent). Then control follows, as super-intelligence depends on us. It would extend our capabilities, such as speed of processing, extent of processing, etc. Even then control is debatable, as a remote-controlled killing machine would be part of a super-intelligent organism, partially human &quot;controlled&quot;, partially autonomous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No in &quot;Future of Life Institute&quot;-like scenarios, where super-intelligence is independent from humans. The thinking is simple: What can we hope to do facing someone way more intelligent? The usual parallel is to compare this scenario with the arrival of the &quot;developed&quot; conquistadors in early America. Gunpowder vs. mere raw strength and arrows.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2017-03-07T21:49:37.613" LastActivityDate="2017-03-07T21:49:37.613" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2902" PostTypeId="1" CreationDate="2017-03-02T12:58:31.380" Score="3" ViewCount="161" Body="&lt;p&gt;An argument against the possibility of super-intelligence is that the intelligence of a creation will be limited by the intelligence of its creator. How reasonable is this argument? &lt;/p&gt;&#xA;" OwnerUserId="5765" LastEditorUserId="2444" LastEditDate="2019-05-02T13:21:36.957" LastActivityDate="2019-05-02T13:21:36.957" Title="Is a super-intelligence limited by the intelligence of its creator?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;superintelligence&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2903" PostTypeId="2" ParentId="2902" CreationDate="2017-03-02T14:07:57.043" Score="2" Body="&lt;p&gt;Of course the intelligence of a product is limited &lt;strong&gt;by&lt;/strong&gt; the intelligence of its creator. Just not &lt;strong&gt;to&lt;/strong&gt; the intelligence of its creator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would be about as reasonable as the idea that the speed of a car is limited to the speed of its creator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or the playing strength of a chess program to the Elo of its creator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or the ability of a neural network to differentiate between dozens of dog breeds to the dog expertise of its creator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, not very reasonable. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-03-02T14:07:57.043" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2904" PostTypeId="2" ParentId="2891" CreationDate="2017-03-02T16:29:14.580" Score="3" Body="&lt;p&gt;This is not an answer (I don't have enough reputation to comment). I did something close to this in my master's thesis and think it is close to what you are interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In it, I had developed a framework for extracting metadata from web-based educational content. This metadata was used for classifying the the educaitonal content for many different attributes, which could then be used for faster and more efficient search and discovery of educational content. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The educational resources (containing the content) could be anything like text or PDFs of assignments, homework, assignments, online books, exam questions, courses etc (which many colleges host online). To identify what kind of educational resource it is, I would parse the text and look for keywords and formatting styles (preprocessing included constructing 2-grams and 3-grams, POS tagging, using small specific parsers for NER, dates and other text entities one encounters in educational content). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some part I used Wordnet (also available under python-nltk) to obtain relationships between different entities and also to find closeness between them. DBpedia was also used. However, for the most part I had to identify the most commonly occuring terms and build a taxonomy by hand. (It took a lot of time!). I obtained a lot of candidates for keywords by looking at openly available taxonomies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For extracting domain specific taxonomy/ontology, one needs manually to build it. Ontology generation from text is an active area of research and building domain-specific ontology has been tried for many years. One example of such taxonomy (here thesaurus) is &lt;a href=&quot;http://aims.fao.org/vest-registry/vocabularies/agrovoc-multilingual-agricultural-thesaurus&quot; rel=&quot;nofollow noreferrer&quot;&gt;agrovoc&lt;/a&gt; where domain experts have contributed to the knowledge by identifying agricultural entities manually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of places where domain specific vocabulary is available; maybe you can use that. In some aspects it is close to supervised machine learning, where one has some nice data and correspondingly nice output. However, on my part, there wasn't much learning in it - more like template matching. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="5750" LastActivityDate="2017-03-02T16:29:14.580" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2905" PostTypeId="2" ParentId="2902" CreationDate="2017-03-02T20:15:51.413" Score="1" Body="&lt;p&gt;AI is frequently used to discover things that would take laborious amounts of time for humans to do.  For example, AI can be used to find the optimal configuration for a mother-board layout, or identify best fit parameters for a financial model.  Frequently, the AI can do a better job at a task and do it more qucikly than a human.  Therefore, in many applications, the AI is already more intelligent than the creator at specific tasks.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are just a few things that AI can already do better than humans:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Playing Chess&lt;/li&gt;&#xA;&lt;li&gt;Playing Jeopardy &lt;/li&gt;&#xA;&lt;li&gt;Detecting Cancer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To argue against the possibility of a super-intelligent AI is somewhat of a moot point since it has already been proven.&lt;/p&gt;&#xA;" OwnerUserId="1434" LastEditorUserId="1671" LastEditDate="2017-03-11T14:14:19.460" LastActivityDate="2017-03-11T14:14:19.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2906" PostTypeId="1" AcceptedAnswerId="3105" CreationDate="2017-03-02T20:53:58.320" Score="0" ViewCount="185" Body="&lt;p&gt;When trying to run tensorboard locally to show my logs with &lt;code&gt;tensorboard --logdir logs/&lt;/code&gt; it always shows nothing but the regular tensorboard menu options, such as orange bar at the top, and different section buttons at the top like graphics, etc. however never shows any data regarding my agents. I am using tensorflow 0.11&lt;/p&gt;&#xA;" OwnerUserId="5801" LastEditorUserId="1671" LastEditDate="2018-06-25T17:43:42.013" LastActivityDate="2018-06-25T17:43:42.013" Title="Tensorboard problems" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2907" PostTypeId="2" ParentId="2900" CreationDate="2017-03-02T22:22:18.233" Score="0" Body="&lt;p&gt;Without going into more detail at the moment (b/c I'm time strapped), I strongly urge you to research the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_control_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Control Problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My own personal view is that humans are more problematic than machines.  Machines are at least rational.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, I believe human &quot;management&quot; (read as &quot;mis-management&quot;) of powerful AI is potentially more of a problem than super-intelligent AI left to it's own devices.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans are known to abuse power, and history is filled with such examples.  Machines, at least, have  a clean slate in this regard.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-03-02T22:41:14.743" LastActivityDate="2017-03-02T22:41:14.743" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2908" PostTypeId="1" CreationDate="2017-03-03T03:36:17.760" Score="5" ViewCount="2295" Body="&lt;p&gt;Is it possible to classify data using a genetic algorithm? For example, would it be possible to sort &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/Spambase&quot; rel=&quot;nofollow noreferrer&quot;&gt;this database&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any example in Matlab?&lt;/p&gt;&#xA;" OwnerUserId="5806" LastEditorUserId="2444" LastEditDate="2019-07-07T21:55:14.813" LastActivityDate="2019-07-07T21:55:14.813" Title="Is it possible to classify data using a genetic algorithm?" Tags="&lt;classification&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;matlab&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2909" PostTypeId="2" ParentId="2900" CreationDate="2017-03-03T06:25:30.740" Score="1" Body="&lt;p&gt;Competition always gives better result. If machines will try to improve themselves, we as human beings will definitely try to improve ourself.&lt;/p&gt;&#xA;" OwnerUserId="5809" LastActivityDate="2017-03-03T06:25:30.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2910" PostTypeId="1" CreationDate="2017-03-03T10:43:29.217" Score="3" ViewCount="186" Body="&lt;p&gt;I have not studied machine learning or AI really, but my job sometimes requires me to automate stuff. Right now the requirement I have, seems to be under AI domain, but I am not sure about terminologies or how to go about it. I will really appreciate if someone can guide me about the direction I need to start from.&#xA;&lt;br/&gt;(PS: This question might not belong on this SE, in that case please direct me to suitable SE)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm required to do is &lt;strong&gt;find references on web about a certain situation&lt;/strong&gt;. As an example I'll use &quot;Music&quot;, so I have to make a system which will search around the web (Google and Twitter mainly) to see if there is any news/mention/event related to Music that occurred today, if so how many references (i.e. how big of a deal it is making).&#xA;&lt;br/&gt;It is not the generic term music which is expected in the output, but the names of Musicians, i.e. in Music this and this Artist appeared this many times. &#xA;&lt;br/&gt;I have to give the number of references, and also provide the references in output so that one can read them in detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;The challenges are&lt;/em&gt;&lt;/strong&gt;&#xA;    &lt;br/&gt;&#xA;- One event can be covered by many websites, and there can be one main website that published the original story with full details, while others just spread the word around in summarized way. &#xA;&lt;br/&gt;&lt;strong&gt;&lt;em&gt;How do you filter references to pick the most suitable one&lt;/em&gt;&lt;/strong&gt;, to show in results to the system's user, because I can not give user ~50 references to manually read through, I have to give like 1-2 suitable reference&#xA;&lt;br/&gt;- I need to give the name of the artist. One site will have many words, how do I know which word is actually the artist's name? One option can be to have a pre compiled list of specific artists and just search for them individually. But this way, I can be missing new artists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenges I have, &lt;strong&gt;must have been addressed by some existing algorithm&lt;/strong&gt; or mechanism, I'll appreciate if someone can let me know what kind of algo etc I need to refer to or study to get the task done.&lt;/p&gt;&#xA;" OwnerUserId="5814" LastActivityDate="2017-11-28T23:39:49.033" Title="Searching keywords on web" Tags="&lt;ai-design&gt;&lt;algorithm&gt;&lt;definitions&gt;&lt;intelligent-agent&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2911" PostTypeId="1" AcceptedAnswerId="2916" CreationDate="2017-03-03T11:08:49.327" Score="8" ViewCount="227" Body="&lt;p&gt;It seems fairly uncontroversial to say that NN based approaches are becoming quite powerful tools in many AI areas - whether recognising and decomposing images (faces at a border, street scenes in automobiles, decision making in uncertain/complex situations or with partial data). Almost inevitably, some of those uses will develop into situations where NN-based AI takes on part or all of the human burden, and generally does it better than people generally do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples might include NN hypothetically used as steps in self-driving cars, medical diagnosis, human/identity verification, circuit/design verification, dubious transaction alerting. Probably many fields in the next decade or so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose this happens, and is generally seen as successful (for example, it gets diagnoses right 80% to human doctors' 65% or something, or cars with AI that includes an NN component crash 8% less than human-driven cars or alternatives, or whatever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now - suppose one of these aberrantly and seriously does something very wrong in one case. How can one approach it? With formal logic steps one can trace a formal decision process, but with NN there may be no formal logic, especially if it gets complex enough (in a couple of decades say), there are just 20 billion neural processors and their I/O weightings and connections, it may not be possible to determine what caused some incident even if lives were lost. It also may not be possible to say more than the systems continually learn and such incidents are rare. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also haven't heard of any meaningful way to do a &quot;black box&quot; or flight recorder equivalent for NNs, (even if not used I a life-critical case), that would allow us to understand and avoid a bad decision. Unlike other responses to product defects, if a NN could be trained after the event to fix one such case, it doesn't clearly provide the certainty we would want, that the new NN setup has fixed the problem, or hasn't reduced the risk and balance of other problems in so doing. It's just very opaque. And yet, clearly, it is mostly very valuable as an AI approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 20 years if NN is an (acknowledged as safe and successful) component in a plane flight or aircraft design, or built into a hospital system to watch for emergencies, or to spot fraud at a bank, and has as usual passed whatever regulatory and market requirements might exist and performed with a good record for years in the general marketplace, &lt;em&gt;and&lt;/em&gt; then in one case such a system some time later plainly misacts on one occasion - it dangerously misreads the road, recommends life-damaging medications or blatantly misdiagnoses, or clears a blatant £200m fraudulent transaction at a clearing bank that's only caught by chance before the money is sent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can the manufacturer do to address public or market concerns, or to explain the incident? What do the tech team do when told by the board &quot;how did this happen and make damn sure it's fixed&quot;? What kind of meaningful logs can be kept, etc.? Would society have to just accept that uncertainty and occasional wacky behavior could be inherent (good luck with the convincing society of that!)? Or is there some better way to approach logging/debugging/decision activity more suited to NNs?&lt;/p&gt;&#xA;" OwnerUserId="5817" LastEditorUserId="2444" LastEditDate="2019-09-09T11:09:35.597" LastActivityDate="2019-09-09T11:10:50.553" Title="How would one debug, understand or fix the outcome of a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;philosophy&gt;&lt;security&gt;&lt;explainable-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2912" PostTypeId="1" CreationDate="2017-03-03T13:01:56.950" Score="1" ViewCount="63" Body="&lt;p&gt;I've been experimenting with a simple tic-tac-toe game to learn neural network programming (MLP and CNNs) with good results. I train the networks on a board positions and the best moves and the network is able to learn and correctly predict the best moves to make when it encounters those board positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the network is unable to &quot;discover&quot; newer patterns/features from existing ones. For example -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that the board position is below and move is for the X player (AI)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;O  _  _&#xA;&#xA;_  O  _&#xA;&#xA;_  _  _&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The recommended move would be 8 (0 based indices) so that the opponent doesn't win, the resulting board would be - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;O  _  _&#xA;&#xA;_  O  _&#xA;&#xA;_  _  X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I train the network on the above enough times, the AI (MLP or CNN based) learns to play 8 when it encounters the above situation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it doesn't recognize the below as variations (rotated and shifted, respectively but slanted straight lines in general) of the same pattern and is not able to correctly pick 6 and 0, respectively -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_  _  O            _  _  _&#xA;&#xA;_  O  _     or     _  O  _   etc&#xA;&#xA;_  _  _            _  _  O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is - Should I expect CNNs to be able to discover new previously untrained on patterns/features such as above? &lt;/p&gt;&#xA;" OwnerUserId="1522" LastActivityDate="2017-03-03T13:01:56.950" Title="Can a CNN or MLP discover similar but untrained-on patterns?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2913" PostTypeId="2" ParentId="2908" CreationDate="2017-03-03T15:36:36.527" Score="3" Body="&lt;p&gt;It is possible, but is a pretty terrible idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few options. One is to not use the GA as a direct classifier, but instead use a GA to learn the parameters of another classification model like a neural network. The basic idea of a GA is that it (very roughly speaking) forms a black-box method for searching an arbitrary space for solutions that minimize or maximize some function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, you would be searching the space of possible neural network topologies and/or weights to find one that minimizes the misclassification rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is that taken by what are sometimes called Learning Classifier Systems (LCS) or Genetics Based Machine Learning (GBML). This approach is to use evolutionary mechanics to evolve rule sets of the form &quot;if X condition is true, then do/classify Y&quot;. That's a more direct method of solving this sort of problem. You define some features on your dataset, and the algorithm tries to learn rules based on those features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with any of these approaches is just that there are so many better ways to solve the problem. Remember, a GA is basically a black-box that's supposed to work acceptably well for a huge range of unknown problems. But I'm not solving a huge range of unknown problems. I'm trying to separate ham from spam on one dataset. I can come up with methods that simply do that job better and more quickly than a GA has any real hope of doing.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastActivityDate="2017-03-03T15:36:36.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2914" PostTypeId="2" ParentId="2910" CreationDate="2017-03-03T19:21:27.827" Score="2" Body="&lt;p&gt;A parallel situation might be that of spam/not spam. The detection of spam by AI has been pretty successful, so there is an existing algorithm - classification. However while you have a possible &lt;em&gt;approach&lt;/em&gt; you are still missing the key ingredient which is sufficient data to train the model on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI depends on a large amount of data to train the model. Ideally you will have a team of researchers available to read a (large?) number of sources and classify by hand whether the source is the original reference or just a repeater, and whether the topic is relevant. The more labelled and balanced data you have, the better the model and the better the results. Then, like the spam/not spam situation, you just apply your model and the best references pop out. You already have the key word of music, so you just use as candidates those sources that reference &quot;music&quot; and any other defining keywords.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2017-03-03T19:21:27.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2916" PostTypeId="2" ParentId="2911" CreationDate="2017-03-03T22:53:41.833" Score="7" Body="&lt;p&gt;If the observation that the neural network saw was recorded, then yes the prediction can be explained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was a paper written fairly recently on this topic called &lt;a href=&quot;https://arxiv.org/abs/1602.04938&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Why Should I Trust You?&quot;: Explaining the Predictions of Any Classifier&lt;/a&gt; (2016). In this paper, the author described an algorithm called LIME which is able to explain any machine learning models predictions. It can be used to establish why a machine learning model made a prediction, help a data scientist debug a model, and help a data scientist improve the accuracy of a specific model. LIME can be used to explain the predictions of any neural network including CNNs, RNNs, and DNNs. &lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="2444" LastEditDate="2019-09-09T11:10:50.553" LastActivityDate="2019-09-09T11:10:50.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2917" PostTypeId="1" CreationDate="2017-03-04T10:30:12.610" Score="4" ViewCount="418" Body="&lt;p&gt;Does it exist a human-like artificial intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I define &lt;em&gt;human-like&lt;/em&gt; as something that can act like a human in most aspects. &lt;/p&gt;&#xA;" OwnerDisplayName="user5832" LastEditorUserId="2444" LastEditDate="2019-10-22T20:28:53.270" LastActivityDate="2019-11-13T05:34:46.627" Title="Does it exist a human-like artificial intelligence?" Tags="&lt;research&gt;&lt;agi&gt;&lt;intelligent-agent&gt;&lt;social&gt;&lt;human-like&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2918" PostTypeId="2" ParentId="2908" CreationDate="2017-03-04T12:53:13.280" Score="2" Body="&lt;p&gt;You must understand that a genetic algorithm is an optimization algorithm. You can't feed it e-mails and make it classify spam. A genetic algorithm is used to train a model to classify spam. That something could be neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need is a genetic algorithm that optimizes neural networks &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroevolution&lt;/a&gt;, which might roughly work as follows&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with a pool of neural networks&lt;/li&gt;&#xA;&lt;li&gt;Feed them e-mails, let them classify, and calculate fitness on % correct&lt;/li&gt;&#xA;&lt;li&gt;Select neural networks for crossover&lt;/li&gt;&#xA;&lt;li&gt;Crossover&lt;/li&gt;&#xA;&lt;li&gt;Mutate&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, there are better ways for classifying e-mails (e.g. an algorithm that looks for certain &quot;spam words&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it is definitely possible. I have a &lt;a href=&quot;https://github.com/wagenaartje/neataptic&quot; rel=&quot;nofollow noreferrer&quot;&gt;javascript library&lt;/a&gt; set up for neuroevolution, if you're interested.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2019-07-07T21:54:10.840" LastActivityDate="2019-07-07T21:54:10.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2919" PostTypeId="1" AcceptedAnswerId="2931" CreationDate="2017-03-04T15:16:46.440" Score="6" ViewCount="620" Body="&lt;p&gt;From &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;, Third Edition, Chapter 26:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that the concept of ultraintelligent machines assumes that intelligence is an especially important attribute, and if you have enough of it, all problems can be solved. But we know there are limits on computability and computational complexity. If the problem of defining ultraintelligent machines (or even approximations to them) happens to fall in the class of, say, NEXPTIME-complete problems, and if there are no heuristic shortcuts, then even exponential progress in technology won't help—the speed of light puts a strict upper bound on how much computing can be done; problems beyond that limit will not be solved. We still don't know where those upper bounds are.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the textbook's argument is correct, then there may be a strict upper bound to intelligence, meaning that the potential (or damage) of super-intelligence is limited. However, it is contingent on there actually being a theoretical maximum for intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any literature that suggests that we know for sure whether such a maximum exist? Is the existence of that maximum dependent on our definition of intelligence (so adopting a vague and hand-wavey definition would imply no theoretical maximum, while adopting a strict and formalized definition would imply a theoretical maximum)?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="2444" LastEditDate="2019-06-28T13:46:17.503" LastActivityDate="2019-06-28T13:46:17.503" Title="Is there a theoretical maximum for intelligence?" Tags="&lt;philosophy&gt;&lt;superintelligence&gt;&lt;singularity&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2920" PostTypeId="1" CreationDate="2017-03-04T18:27:37.373" Score="3" ViewCount="689" Body="&lt;p&gt;I am looking for a solution that I can use with identifying cars.&#xA;So I have a database with images of cars. About 3-4 per car. What I want to do is upload a picture to the web of car(Picture taken with camera/phone) and then let my pc recognize the car. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: &#xA;Lets say I have these 2 pictures in my database(Mazda cx5)(I can only upload 2 links at max. atm. but you get the idea).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NsLow.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NsLow.png&quot; alt=&quot;First car&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am going to upload this picture of a mazda cs5 to my web app:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/psHD6.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/psHD6.png&quot; alt=&quot;Picture of mazda cs5&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want an AI to recognize that this picture is of an Mazda CX5 with greyish color. I have looked on the net and found 2 interesting AI's I can use:&#xA;Tensorflow and Clarifai, but I don't know if these are going to work so my question to you what would be my best bet to go with here?&lt;/p&gt;&#xA;" OwnerUserId="5837" LastActivityDate="2017-07-04T17:30:04.077" Title="Image recognition" Tags="&lt;image-recognition&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2921" PostTypeId="2" ParentId="2920" CreationDate="2017-03-04T19:09:21.750" Score="1" Body="&lt;p&gt;There are several ways you can do this. One way would involve several steps and would probably work best:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a trained Gaussian detector to filter out the car from the rest of the image&lt;/li&gt;&#xA;&lt;li&gt;Use a convolutional neural network to classify the car&lt;/li&gt;&#xA;&lt;li&gt;Use a neural network or a simple most common color algorithm to find the color of the car&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You would be able to implement this method most easily in MATLAB but you would also be able to do it in python with tensorflow or torch. You would probably be able to implement the trained Gaussian detector in tensorflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Method 2:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a spatial transformer network to &quot;transform&quot; the image of the car for easy classification&lt;/li&gt;&#xA;&lt;li&gt;Use the output of the spatial transformer network for classification via a convolutional neural network.&lt;/li&gt;&#xA;&lt;li&gt;Use another neural network or a most common color algorithm to find the color of the car.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This method would also work pretty well but using a spatial transformer network with a convolutional neural network may be hard to code because it is an area of developing research where there are many problems because the spatial transformer network and the convolutional neural network have to work well together and this is usually hard to get right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Method 3:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a convolutional neural network straight up on the input image maybe with down sampling to classify the car&lt;/li&gt;&#xA;&lt;li&gt;Use another neural network to find the color of the car&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would personally go with method #1 because it would be fairly simple to implement with existing libraries such as tensorflow and it would most likely provide a high accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As always I would also recommend that you use LIME during the development process to debug your model and determine what features you could add in or remove to help your model perform better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**Edit*&#xA;Since you need to detect certain patterns on the cars for color classification I would recommend that you use a convolutional neural network to classify these patterns. So your method would now look like this:&lt;br&gt;&#xA;1. Use a spatial transformer network or a filtered Gaussian detector to filter out the car&#xA;2. Use a convolutional neural network to classify the make and model of the car.&#xA;3.  Use another neural network that has either a convolutional or deep architecture to classify patterns and solid colors. So the outputs would contain all of the colors that you want and all of the patterns that you want to detect.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="4631" LastEditDate="2017-03-07T01:17:16.227" LastActivityDate="2017-03-07T01:17:16.227" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2922" PostTypeId="1" CreationDate="2017-03-05T07:07:25.330" Score="14" ViewCount="1743" Body="&lt;p&gt;Nowadays, artificial intelligence seems almost equal to machine learning, especially deep learning. Some have said that deep learning will replace human experts, traditionally very important for feature engineering, in this field. It is said that two breakthroughs underpinned the rise of deep learning: on one hand, neuroscience, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroplasticity&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroplasticity&lt;/a&gt; in particular, tells us that like the human brain, which is highly plastic, artificial networks can be utilized to model almost all functions; on the other hand, the increase in computational power, in particular the introduction of GPU and FPGA, has boosted algorithmic intelligence in a magnificent way, and has been making the models created decades ago immensely powerful and versatile. I'll add that the big data (mostly labeled data) accumulated over the past years is also relevant.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such developments bring computer vision (and voice recognition) into a new era, but in natural language processing and expert systems, the situation hasn't seemed to have changed very much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Achieving common sense for the neural networks seems a tall order, but most sentences, conversations and short texts contain inferences that should be drawn from the background world knowledge. Thus knowledge graphing is of great importance to artificial intelligence. Neural networks can be harnessed in building knowledge bases but it seems that neural network models have difficulty utilizing these constructed knowledge bases.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is a knowledge base (for instance, a &quot;knowledge graph&quot;, as coined by Google) a promising branch in AI? If so, in what ways KB can empower machine learning? How can we incorporate discrete latent variables into NLU and NLG?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For survival in an age dominated by DL, where is the direction for the knowledge base (or the umbrella term symbolic approach)? Is &lt;a href=&quot;http://www.wolfram.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wolfram&lt;/a&gt;-like z dynamic knowledge base the new direction? Or any new directions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Am I missing something fundamental, or some ideas that address these issues?&lt;/p&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="2444" LastEditDate="2020-02-19T00:54:26.060" LastActivityDate="2021-05-22T07:11:20.870" Title="What roles knowledge bases play now and will play in the future?" Tags="&lt;natural-language-processing&gt;&lt;knowledge-representation&gt;&lt;expert-system&gt;&lt;symbolic-computing&gt;&lt;knowledge-base&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="2923" PostTypeId="2" ParentId="2920" CreationDate="2017-03-05T18:47:16.807" Score="2" Body="&lt;h3&gt;Model of the car&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;What you want to do is close to one-shot image recognition. You have not 1, but 3-4 examples of each car, but that is still a small amount, especially considering the car looks different from different angles (are you supposed to recognize them from any point of view, including sideways, rear, front, and 45 degrees etc.? maybe you also want to recognize them photographed from the top?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting article I found is: &lt;a href=&quot;http://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siamese Neural Networks for One-shot Image Recognition&lt;/a&gt; by Koch, Zemel, and Salakhutdinov.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found that &lt;a href=&quot;http://caffe.berkeleyvision.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Caffee&lt;/a&gt; supports Siamese networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to read other literature about the &lt;a href=&quot;https://en.wikipedia.org/wiki/One-shot_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;One-shot learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One trick you can do is to utilize the fact that cars are symmetric, so you can double the number of learning examples by reflecting each image.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Color&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Determining the color is not as simple as it seems. Your algorithm need to determine where is the car at your picture, and then determine the color, taking into account the lighting conditions, as well as light effects, most notably reflection. For example, consider the following image: &lt;a href=&quot;https://i.stack.imgur.com/JQae1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JQae1.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We see strawberries as red, but there are no red pixels on this picture. The images of strawberries consist on grey pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you also need a convolutional neural network, or just a neural network, for this task.&lt;/p&gt;&#xA;" OwnerUserId="5852" LastEditorUserId="5852" LastEditDate="2017-03-05T22:36:33.930" LastActivityDate="2017-03-05T22:36:33.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2924" PostTypeId="1" AcceptedAnswerId="3158" CreationDate="2017-03-05T20:25:02.817" Score="1" ViewCount="132" Body="&lt;p&gt;I know I've seen this somewhere before, but can't find it now.  Say we have a neural network with a handful of layers, and we're applying dropout to each layer.  As we move closer to the output, should dropout decrease, increase, or stay the same?&lt;/p&gt;&#xA;" OwnerUserId="5857" LastActivityDate="2017-04-14T14:28:26.940" Title="How should dropout change with network depth?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2925" PostTypeId="1" AcceptedAnswerId="2935" CreationDate="2017-03-06T00:28:31.647" Score="2" ViewCount="115" Body="&lt;p&gt;I want to write a program that looks at abbreviated words, then figures out what the words are. For example, the abbreviation is &quot;blk comp&quot;, and the translation is &quot;black computer&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to give it context for more ambiguous terms, I will be inputting sets of words with each request. So, if I input the set &quot;keyboard, software, mouse, monitor&quot;, I would expect to get &quot;black computer&quot;. On the other hand, if I input &quot;Honda, transmission, mileage, Ford&quot;, I then would expect to get &quot;black compact&quot;, or at least something that has anything to do with cars. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basing on the above case scenario, what kind of an algorithm should be applied in this case?&lt;/p&gt;&#xA;" OwnerUserId="5860" LastEditorUserId="1581" LastEditDate="2017-03-11T14:15:30.310" LastActivityDate="2017-03-11T14:15:30.310" Title="Recognition of abbreviated text" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;algorithm&gt;&lt;genetic-algorithms&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2926" PostTypeId="1" CreationDate="2017-03-06T07:42:26.063" Score="9" ViewCount="9543" Body="&lt;p&gt;I need a machine learning algorithm to identify patterns in a dataset (saved in a CSV file) that contains details of the cache performance of a CPU. More specifically, the dataset contains columns like &lt;code&gt;Readhits&lt;/code&gt;, &lt;code&gt;Readmiss&lt;/code&gt; or &lt;code&gt;Writehits&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The patterns the algorithm identifies should be helpful in the following ways.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;help the user to increase the performance of the workload next time,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;help to identify any problems based on the features, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;help the user to predict future data values or future events that may occur based on the patterns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which ML algorithms can I use?&lt;/p&gt;&#xA;" OwnerUserId="5867" LastEditorUserId="2444" LastEditDate="2020-03-21T18:18:01.987" LastActivityDate="2020-08-18T19:04:40.553" Title="Which machine learning algorithm can be used to identify patterns in a dataset of the cache performance of a CPU?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;unsupervised-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2927" PostTypeId="1" AcceptedAnswerId="2934" CreationDate="2017-03-06T10:00:44.500" Score="7" ViewCount="720" Body="&lt;p&gt;I've been reading a lot about hardware development and implementation for AI/ML, mainly about Deep Learning, and I have a question about its usage.&#xA;From what I understand, there are 2 stages for DL: first is training and second is inference. The first is often done on GPUs because of their massive parallelism capabilities among other things, and inference, while can be done on GPUs, it's not used that much, because of power usage, and because the data presented while inferring are much less so the full capabilities of GPUs won't be much needed. Instead FPGAs and CPUs are often used for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding also is that a complete DL system will have both, a training system and an inferring system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is that: are both systems required on the same application? Let's assume an autonomous car or an application where visual and image recognition is done, will it have both training system to be trained and an inference system to execute? Or it has only the inference system and will communicate with a distant system which is already trained and has built a database?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if the application has both systems, will it have a big enough memory to store the training data? Given that it can be a small system and memory is ultimately limited.&lt;/p&gt;&#xA;" OwnerUserId="5873" LastEditorUserId="145" LastEditDate="2017-03-11T14:14:22.167" LastActivityDate="2018-10-22T03:43:47.680" Title="Machine Learning hardware usage in embedded applications" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2928" PostTypeId="1" AcceptedAnswerId="2929" CreationDate="2017-03-06T11:35:52.083" Score="3" ViewCount="9338" Body="&lt;p&gt;I'm trying to create simple keras NN which will learn to make addition on numbers between 0 and 10. But I am getting the error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Error when checking model target: expected activation_4 to have shape (None, 19) but got array with shape (100, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential&#xA;from keras.layers import Dense, Activation&#xA;import numpy as np&#xA;&#xA;keras.optimizers.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False)&#xA;&#xA;model = Sequential()&#xA;model.add(Dense(output_dim=50, input_dim=2))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(Dense(output_dim=50))&#xA;model.add(Activation(&quot;softmax&quot;))&#xA;model.add(Dense(output_dim=50))&#xA;model.add(Activation(&quot;softmax&quot;))&#xA;model.add(Dense(output_dim=19))&#xA;model.add(Activation(&quot;softmax&quot;))&#xA;&#xA;model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])&#xA;&#xA;x = []&#xA;y = []&#xA;&#xA;for i in range(0, 10):&#xA;    for j in range(0, 10):&#xA;        x.append((i, j))&#xA;        y.append(i + j)&#xA;&#xA;x = np.array(x)&#xA;y = np.array(y)&#xA;print(x)&#xA;print(y)&#xA;&#xA;model.fit(x, y, nb_epoch=5, batch_size=32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;how to fix that?&lt;/p&gt;&#xA;" OwnerUserId="5661" LastActivityDate="2017-03-06T14:26:28.900" Title="keras ValueError: Error when checking model target: expected activation_4 to have shape (None, 19) but got array with shape (100, 1)" Tags="&lt;neural-networks&gt;&lt;keras&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ClosedDate="2017-03-06T15:55:10.170" ContentLicense="CC BY-SA 3.0" />
  <row Id="2929" PostTypeId="2" ParentId="2928" CreationDate="2017-03-06T12:14:06.937" Score="0" Body="&lt;p&gt;Try to use the model like this, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential() &#xA;model.add(Dense(50, input_shape=(2,))) &#xA;model.add(Activation(&quot;relu&quot;)) &#xA;model.add(Dense(50, activation='softmax')) &#xA;model.add(Dense(1, activation='linear')) &#xA;model.compile(optimizer='sgd', loss='mse', metrics=[&quot;accuracy&quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&#xA;This means that first layer will have &lt;strong&gt;50&lt;/strong&gt; neurons and can receive data in form of matrix with &lt;strong&gt;2&lt;/strong&gt; columns and an unspecified number of rows.&#xA;So you can prepare your data in this form – 2 numbers for &lt;em&gt;adding&lt;/em&gt; in each row.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dense(50, input_shape=(2,))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&#xA;At the end, you need a layer with 1 neuron and the &lt;code&gt;'linear'&lt;/code&gt; activation, because you expect one simple number as a result.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dense(1, activation='linear')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&#xA;And finally, use &lt;code&gt;'mse'&lt;/code&gt; loss function or something similar. &lt;code&gt;'categorical_crossentropy'&lt;/code&gt; is needed for classification tasks, not regression as needed for you.&#xA;See: &lt;a href=&quot;https://keras.io/objectives/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/objectives/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5876" LastActivityDate="2017-03-06T12:14:06.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2930" PostTypeId="2" ParentId="2928" CreationDate="2017-03-06T14:26:28.900" Score="0" Body="&lt;p&gt;You shouldn't use Softmax as an activation function in intermediate layers. Softmax is used to represent a categorical distribution, and should be applied at the point where one makes a categorical prediction (usually the final layer of the network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider replacing you activation function in all layers except the last one with 'relu' or 'sigmoid'.&lt;/p&gt;&#xA;" OwnerUserId="5879" LastActivityDate="2017-03-06T14:26:28.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2931" PostTypeId="2" ParentId="2919" CreationDate="2017-03-06T15:17:36.363" Score="2" Body="&lt;p&gt;Note that the statement says nothing directly about the limit of intelligence, nor even about the limit of computational intelligence - but about the limit of computing power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the sentence &lt;em&gt;&quot;the speed of light puts a strict upper bound on how much computing can be done&quot;&lt;/em&gt; needs a better explanation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors are probably referring to &lt;a href=&quot;https://en.wikipedia.org/wiki/Bremermann%27s_limit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bremermann's limit&lt;/a&gt;, which defines an upper bound in bits per second per kilogram. It is an upper bound on the processing power per unit of time of a computer with a given weight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also &lt;a href=&quot;https://en.wikipedia.org/wiki/Margolus%E2%80%93Levitin_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Margolus–Levitin theorem&lt;/a&gt; which defines an upper limit in operations per second per joule. It is an upper bound on the processing power per unit of energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These principles do not define a theoretical limit on computing power, but a practical one. If you'll limit your computer and your energy source to the size and capacity of the earth (or to the those of the universe) - you'll get a very practical limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the reference section in Wikipedia article &lt;a href=&quot;https://en.wikipedia.org/wiki/Limits_to_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Limits to computation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3138" LastEditorUserId="3138" LastEditDate="2017-03-06T16:32:17.700" LastActivityDate="2017-03-06T16:32:17.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2932" PostTypeId="1" AcceptedAnswerId="3319" CreationDate="2017-03-06T16:22:04.373" Score="1" ViewCount="8043" Body="&lt;p&gt;I am trying to do an inception layer, but it only works if the convolution strides, pool strides and pool size are the same, otherwise I get an error in &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;tf.concat&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;that Dimesion 1 is not the same. So If I change something in the last three tuples, I get the error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv1 = conv2d_maxpool(x, 64, (5, 5), (1, 1), (2, 2), (2, 2)) &#xA;conv2 = conv2d_maxpool(x, 64, (4, 4), (1, 1), (2, 2), (2, 2)) &#xA;conv3 = conv2d_maxpool(x, 32, (2, 2), (1, 1), (2, 2), (2, 2)) &#xA;conv4 = conv2d_maxpool(x, 32, (1, 1), (1, 1), (2, 2), (2, 2)) &#xA;conv = tf.concat([conv1, conv2, conv3, conv4], 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example, this is the error I get if I change the 5x5 filter to have strides 3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv1 = conv2d_maxpool(x, 64, (5, 5), (3, 3), (2, 2), (2, 2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dimension 1 in both shapes must be equal, but are 6 and 16 for&#xA;  'concat' (op: 'ConcatV2') with input shapes: [?,6,6,64], [?,16,16,64],&#xA;  [?,16,16,32], [?,16,16,32], [].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is the conv2d_maxpool function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):&#xA;    &quot;&quot;&quot;&#xA;    Apply convolution then max pooling to x_tensor&#xA;    :param x_tensor: TensorFlow Tensor&#xA;    :param conv_num_outputs: Number of outputs for the convolutional layer&#xA;    :param conv_strides: Stride 2-D Tuple for convolution&#xA;    :param pool_ksize: kernal size 2-D Tuple for pool&#xA;    :param pool_strides: Stride 2-D Tuple for pool&#xA;    : return: A tensor that represents convolution and max pooling of x_tensor&#xA;    &quot;&quot;&quot;&#xA;    # TODO: Implement Function&#xA;    weights = tf.Variable(tf.truncated_normal(&#xA;        shape = [*conv_ksize, int(x_tensor.get_shape().dims[3]), conv_num_outputs], &#xA;        mean = 0.0, &#xA;        stddev=0.1, &#xA;        dtype=tf.float32))&#xA;    bias = tf.Variable(tf.zeros(conv_num_outputs)) &#xA;&#xA;    conv_layer = tf.nn.conv2d(x_tensor, weights, strides=[1, *conv_strides, 1], padding='SAME')&#xA;    conv_layer = tf.nn.bias_add(conv_layer, bias)&#xA;    conv_layer = tf.nn.relu(conv_layer)&#xA;&#xA;    conv_layer_max_pool = tf.nn.max_pool(conv_layer, ksize=[1, *pool_ksize, 1], strides=[1, *pool_strides, 1], padding='SAME')&#xA;&#xA;    return conv_layer_max_pool&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I combine convolution filters with different strides and/or different pooling to create an inception layer?&lt;/p&gt;&#xA;" OwnerUserId="5527" LastActivityDate="2017-05-16T10:25:38.073" Title="Concatenate convolution layers with different strides in tensorflow." Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2933" PostTypeId="2" ParentId="2927" CreationDate="2017-03-06T17:47:42.840" Score="1" Body="&lt;p&gt;Deep learning seems mostly to be a buzzword for what is essentially a neural network. You train with a data set to recognize a pattern, then input new data which is then classified by the trained network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you train a neural network with 10 different kinds of animals using thousands of pictures. Then you show the network say 100 new images and have the network &quot;guess&quot;  what each animal is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point here is that training a neural network would required code for feedback that an application using the trained network would not need. So an application just using the trained network would be a bit more streamlined than an application which could allow additional training data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is missing is the ability for machine learning to form hierarchical or otherwise interrelated rules from the trained network so that the trained network is closer to being able to rationally explain why the classification works. Going back to the 10 animal trained network, without formulation of rules during training, there is no practical way for the network to reveal why any of the animals was classified the way it was. &lt;/p&gt;&#xA;" OwnerUserId="3471" LastEditorUserId="18663" LastEditDate="2018-10-22T03:43:47.680" LastActivityDate="2018-10-22T03:43:47.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2934" PostTypeId="2" ParentId="2927" CreationDate="2017-03-07T01:58:44.713" Score="-1" Body="&lt;p&gt;To answer your question: &lt;strong&gt;Training and inference are usually completed on two separate systems&lt;/strong&gt; you are right in knowing that training of deep neural networks is usually done on GPUs and that inference is usually done on CPUs. However, training and inference are almost always done on two separate systems. The main workflow for many data scientists today is as follows:&#xA;1. Create and establish all hyper-parameters for a model such as a deep neural network &#xA;2. Train the deep neural network using a GPU&#xA;3. Save the weights that training on the GPU established so that the model can be deployed. &#xA;4. Code the model in a production application with the optimal weights found in training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So as you can see from this workflow training and inference are done in two completely separate phases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in some specific cases training and inference are done on the same system. For example, if you are using a Deep Neural Network to play video games than you may have the neural network train and infer on the same system. This would lead to more efficiently because it would allow the model to continuously learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question on memory, the only applications where inference and training are done in the same application have a lot of memory available(think dual GPU dual CPU 128gb of RAM workstations) whereas applications that have a limited amount of memory only use inference such as embedded applications.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="4631" LastEditDate="2017-03-07T02:04:41.203" LastActivityDate="2017-03-07T02:04:41.203" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2935" PostTypeId="2" ParentId="2925" CreationDate="2017-03-07T02:19:35.290" Score="2" Body="&lt;p&gt;For your first question,take a look at using a skip grab model to find what the abbreviated text is. The skip gram model turns a word into a vector which allows it to be processed by other machine learning algorithms. Or , alternatively you can do some really cool addition and subtraction problems with the resulting vectors. With the skip gram model in your case you could generate a vector for your input and then compare it with other skip gram vectors and once you have found a near perfect match then that is the unabbreviated word. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also look at using sparse distributed representations of words to do this. This approach is similar to the skip gram model except that instead of a vector with maybe 500 values a sparse representation may contain thousands of binary digits of which only a couple are positive or 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to look at this approach take a look at cortical.io which has free API that you can use. As for your second question I reccomend that you use a deep neural network in combination with the skip gram model to produce your output.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="1581" LastEditDate="2017-03-11T14:14:25.653" LastActivityDate="2017-03-11T14:14:25.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2936" PostTypeId="1" CreationDate="2017-03-07T11:33:05.157" Score="2" ViewCount="425" Body="&lt;p&gt;I'm a newbie in machine learning, so excuse me in advance). I have an idea to make NN that can estimate visual pleasantness of arbitrary image. Like you have a bunch of images that you like, you train NN on them, then you show some random picture to NN and it estimates whether you'll like it or not. I wonder if there is any pervious effort made in this direction. &lt;/p&gt;&#xA;" OwnerUserId="5899" LastActivityDate="2017-04-07T08:34:33.887" Title="Training neural network for good taste in art" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2939" PostTypeId="2" ParentId="2936" CreationDate="2017-03-07T18:17:01.320" Score="1" Body="&lt;p&gt;That sounds like a pretty straightforward application of a NN classifier to me.  I don't know if anybody has done that specific thing or not, but I don't see any particular reason to think it wouldn't work. My advice to you is to just jump in and do it.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-03-07T18:17:01.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2940" PostTypeId="1" CreationDate="2017-03-07T20:08:04.537" Score="6" ViewCount="414" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ejFBN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ejFBN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feedforward or multilayered neural networks, like the one in the image above, are usually characterized by the fact that all weighted connections can be represented as a continuous real number. Furthermore, each node in a layer is connected to every other node in the previous and successive layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are there any other information processing models other than FFNNs or MLPs?&lt;/em&gt; For example, is there any system in which the topology of a neural network is variable? Or a system in which the connections between nodes are not real numbers?&lt;/p&gt;&#xA;" OwnerUserId="1321" LastEditorUserId="2444" LastEditDate="2019-07-07T22:23:28.563" LastActivityDate="2019-07-07T22:23:28.563" Title="What are some information processing models besides feedforward or multi-layered neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;topology&gt;&lt;neuroevolution&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2941" PostTypeId="1" CreationDate="2017-03-07T21:59:15.663" Score="3" ViewCount="677" Body="&lt;p&gt;I would like to detect street and sidewalk surface in a very detailed (0.075m/pix) USGS High Resolution Orthoimagery which basically means image segmentation with two classes. Places in question are residential areas similar to &lt;a href=&quot;https://binged.it/2mgmSvR&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;. I will download uncompressed raw imagery in GeoTIFF from USGS for the detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that neural networks can perform very good in image segmentation and I would like to try them. I am a developer by day so I can code but am a beginner to neural networks only knowing the basic principles about architecture, weighting and backpropagation etc. Is it possible to jump right in into my task or do I need to start with something simpler? I would prefer jumping right in if it can save time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I skimmed though few papers dealing with similar thing and they seem quite complicated. Is there some simple way I can get started? I mean maybe an open source project in neural networks that deals with image segmentation that is similar to my task and I could make use of it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see neural networks need to be trained first and I am prepared to do manual segmentation first to have data for training. However, I have no idea about neural network design/architecture, how to design the layers, how many layers do I need etc. I also would like to use the fact that the network would learn some basics on how streets and sidewalks are built - that they are (not sure if my term is correct) &quot;linear structures&quot; which usually run many meters in length and may not even end in the image, also that sidewalks usually run alongside streets, streets have intersections etc.&lt;/p&gt;&#xA;" OwnerUserId="113" LastEditorUserId="113" LastEditDate="2017-03-07T22:55:35.283" LastActivityDate="2017-03-09T15:34:17.910" Title="Detect street and sidewalk surface in aerial imagery (neural network)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2942" PostTypeId="1" CreationDate="2017-03-08T07:51:37.170" Score="6" ViewCount="487" Body="&lt;p&gt;I've been reading a lot about DL. I can understand to an extent how it works, in theory at least, and how it's technically different from conventional ML. But what I'm looking for is more of a &quot;conceptual&quot; meaning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you're designing a self-learning system, why would you choose DL? What are the main performance advantages that DL offers? Is it the accuracy, speed, power efficiency, a mix of all of them?&lt;/p&gt;&#xA;" OwnerUserId="5873" LastEditorUserId="2444" LastEditDate="2019-03-14T10:14:53.603" LastActivityDate="2019-03-14T10:14:53.603" Title="What does deep learning offer with respect to standard machine learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;applications&gt;&lt;comparison&gt;&lt;performance&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2943" PostTypeId="2" ParentId="2936" CreationDate="2017-03-08T07:53:41.273" Score="1" Body="&lt;p&gt;I don't think anyone has done it yet,but you could try.&#xA;A way you could implement it is  having a quite efficient CNN trained on the things you like,then your program should ask the user if he does like some images and on the answers he will give, your program will finetune the original network and then with the fresh-trained one you should obtain good results.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastActivityDate="2017-03-08T07:53:41.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2944" PostTypeId="2" ParentId="2279" CreationDate="2017-03-08T08:02:31.230" Score="3" Body="&lt;p&gt;You could use another type of CNN that instead of classification is performing regression so it will also give you as output the position(it's not really like that but this is the core idea) .&#xA;Some algorithms are &lt;a href=&quot;https://github.com/weiliu89/caffe/tree/ssd&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD&lt;/a&gt; or &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastEditorUserId="2320" LastEditDate="2017-03-09T05:41:54.967" LastActivityDate="2017-03-09T05:41:54.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2945" PostTypeId="2" ParentId="2926" CreationDate="2017-03-08T08:16:42.453" Score="0" Body="&lt;p&gt;First, you must classify each chunk of the CSV file and label it based on the current situation, like A) optimal situation B) critical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you cluster your data with an unsupervised learning algorithm, like SOM or k-means, and then you simply classify the classes you will get.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastEditorUserId="2444" LastEditDate="2019-06-25T16:49:30.643" LastActivityDate="2019-06-25T16:49:30.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2946" PostTypeId="2" ParentId="2942" CreationDate="2017-03-08T11:04:18.067" Score="6" Body="&lt;p&gt;Deep learning allows you to solve complex problems without necessarily being able to specify the important &quot;features&quot; or key input variables for the model in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an example, a problem that may be easily tackled &lt;strong&gt;without deep learning&lt;/strong&gt; could be predicting the frequency and claim amounts of insurance vehicle claims,  given historical claim data that may include various attributes of the policy holder and their vehicle.  In this example, the &quot;features&quot; to be specified in the model are the known attributes of policy holder and vehicle.  The model will then attempt to utilise these features to make predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand,  facial recognition is a problem more suited to &lt;strong&gt;deep learning&lt;/strong&gt; algorithms.  This is because it is difficult to manually identify what combinations of pixels may be important features to include in a conventional machine learning model.  A multi-layered neural network however has the potential to identify/create the important features itself, which may include for example eyes, nose and mouth, and then utilise these features to recognise faces and other objects.&lt;/p&gt;&#xA;" OwnerUserId="5920" LastEditorUserId="5920" LastEditDate="2017-03-08T11:16:12.003" LastActivityDate="2017-03-08T11:16:12.003" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2947" PostTypeId="2" ParentId="1955" CreationDate="2017-03-09T09:57:00.367" Score="0" Body="&lt;p&gt;There is an interview (link see below) with David E. Smith, a  senior  researcher  in  the  Intelligent  Systems Division  at  NASA  Ames  Research Center. In this interview, he talks about the application of AI and AI planning in particular in his work at NASA. He also (just shortly) mentions the Mars Exploration Rover and cites related scientific papers (just search for &quot;Mars&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to the official publication at Springer:&lt;br&gt;&#xA;&lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs13218-015-0403-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://link.springer.com/article/10.1007%2Fs13218-015-0403-y&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4893" LastActivityDate="2017-03-09T09:57:00.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2948" PostTypeId="2" ParentId="2236" CreationDate="2017-03-09T11:22:29.873" Score="4" Body="&lt;h2&gt;AI is a wide field&lt;/h2&gt;&#xA;&lt;p&gt;AI is a wide field that goes far beyond machine learning, deep learning, neural networks, etc. In some of these fields, the programming language does not matter at all (except for speed issues), so LISP would certainly not be a topic there.&lt;/p&gt;&#xA;&lt;h2&gt;Planning systems&lt;/h2&gt;&#xA;&lt;p&gt;In search or AI planning, for instance, standard languages, like C++ and Java, are often the first choice, because they are fast (in particular C++) and because many software projects like planning systems are open source, so using a standard language is important (or at least wise in case one appreciates feedback or extensions).&lt;/p&gt;&#xA;&lt;p&gt;I am only aware of one single planner that is written in LISP. Just to give some impression about the role of the choice of the programming language in this field of AI, I'll give a list of some of the best-known and therefore most-important planners:&lt;/p&gt;&#xA;&lt;h3&gt;&lt;a href=&quot;http://www.fast-downward.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast-Downward&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;This is probably the best-known classical planning system, which is written in C++ and some parts (pre-processing) in Python.&lt;/p&gt;&#xA;&lt;h3&gt;&lt;a href=&quot;https://fai.cs.uni-saarland.de/hoffmann/ff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast-Forward&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;Together with Fast-Downward, this is the classical planning system everyone knows. It's written in C.&lt;/p&gt;&#xA;&lt;h3&gt;&lt;a href=&quot;http://www.tempastic.org/vhpop/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VHPOP&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;This is one of the best-known partial-order causal link (POCL) planning systems. It's written in C++.&lt;/p&gt;&#xA;&lt;h3&gt;&lt;a href=&quot;https://www.cs.umd.edu/projects/shop/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SHOP and SHOP2&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;This is the best-known HTN (hierarchical) planning system. There are two versions: SHOP and SHOP2. The original versions have been written in LISP. Newer versions (called JSHOP and JSHOP2) have been written in Java. Pyshop is a further SHOP variant written in Python.&lt;/p&gt;&#xA;&lt;h3&gt;&lt;a href=&quot;http://www.uni-ulm.de/en/in/ki/research/software/panda/panda-planning-system/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANDA&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;This is another well-known HTN (and hybrid) planning system. There are different versions of the planner, PANDA1 and PANDA2, which are written in Java. PANDA3 is written primarily in Java, with some parts being in Scala.&lt;/p&gt;&#xA;&lt;p&gt;These were just some of the best-known planning systems that came to my mind. More recent ones can be retrieved from the &lt;a href=&quot;http://www.icaps-conference.org/index.php/Main/Competitions&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Planning Competitions (IPCs)&lt;/a&gt;, which take place every two years. The competing planners' codes are published open-source (for a few years).&lt;/p&gt;&#xA;" OwnerUserId="4893" LastEditorUserId="2444" LastEditDate="2021-01-17T21:10:08.617" LastActivityDate="2021-01-17T21:10:08.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2949" PostTypeId="2" ParentId="2941" CreationDate="2017-03-09T15:34:17.910" Score="3" Body="&lt;p&gt;Yes, in fact neural networks (NNs) are very efficient at segmentation and it seems to me that your problem matches the capabilities of neural networks very well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it best for you to truly understand what a NN is before using it. First, let's start with the architecture. A NN has 3 regions, the input layer, the hidden layers and the output layer. The input layer depends on the number of features in your dataset. The hidden layers, you can have multiple layers all of different breadth (number of nodes per layer). The output layer depends on the number of classes in your dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An easy example is applying a NN to the MNIST datatset. This is a dataset which contains handwritten digits between 0-9. Let's assume each of these images is 16*16=256 pixels. Thus, you will need 256 input nodes. And you will need 10 output nodes, one for each possible output. The hidden layers can be set in any way you can creatively imagine. There are however ways to optimize your hidden layer to get the best performance possible while not spending too much computational power. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is always how a NN works. The beauty of a NN is that you only need to code it once and it can learn any function. All you need to do is change your architecture, but the underlining principles will always be the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you want to do segmentation. This is often done using a window around the pixel you want to classify. Popular choices are 3*3 or 5*5 pixels. The choice of your considered window will determine the number of nodes in your input layer. Then you want to classify them as one of two classes, thus you need 2 output nodes. You can also use just 1, but I don't recommend it, I can expand on this if you care.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One caveat of NN is that you will need quite a bit of training data. So get ready to classify a lot of pixels manually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to know how many layers in hidden layer? How to know how many nodes per layer in the hidden layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, for simple operations like the one you are trying to learn you do not need to have multiple layers. One hidden layer should be enough, at most 2. But, how do you determine the number of nodes you should use? You need to use some model validation techniques to do this. One way is through grid search and cross-validation. Train, and re-train your model with multiple number of nodes and then compare their performances to identify the optimal number of nodes. To get good results this does require a large dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rule of thumb: 1 hidden layer for NN! Don't get dragged into deep models if you don't need them. &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-03-09T15:34:17.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2950" PostTypeId="1" AcceptedAnswerId="2952" CreationDate="2017-03-09T16:36:14.807" Score="3" ViewCount="134" Body="&lt;p&gt;I want to train text classifier (using &lt;a href=&quot;https://www.uclassify.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uclassify.com&lt;/a&gt;) with 12 classes/categories. I will be training it to classify news/articles (I know that there are existing classifier but I want to train my own).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;uclassify uses following algorithm (directly copied from their site):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The core is a multinominal Naive Bayesian classifier with a couple of&#xA;  steps that improves the classification further (hybrid complementary&#xA;  NB, class normalization and special smoothing). The result of&#xA;  classifications are probabilities [0-1] of a document belonging to&#xA;  each class. This is very useful if you want to set a threshold for&#xA;  classifications. E.g. all classifications over 90% is considered spam.&#xA;  Using this model also makes it very scalable in terms of CPU time for&#xA;  classification/training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was wondering how many examples I will need to train such classifier? It is possible to estimate the number? Let's assume that one article will &quot;fit&quot; 2 categories by average.&lt;/p&gt;&#xA;" OwnerUserId="5935" LastActivityDate="2017-03-09T17:55:35.353" Title="How many training example text classifier needs to be trained?" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2951" PostTypeId="2" ParentId="154" CreationDate="2017-03-09T17:01:54.670" Score="6" Body="&lt;p&gt;1) It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out &lt;a href=&quot;https://github.com/fchollet/keras/blob/master/examples/addition_rnn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link to see the source code&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) This particular example uses a recurrent neural network (RNN) to process the problem as a sequence of characters, producing a sequence of characters which form the answer. Note that this approach is obviously different from how humans tend to think about solving simple addition problems, and probably isn't how you would ever want a computer to solve such a problem. Mostly this is an example of sequence to sequence learning using Keras. When handling sequential or time-series inputs, RNNs are a popular choice.&lt;/p&gt;&#xA;" OwnerUserId="5936" LastEditorUserId="14723" LastEditDate="2018-04-15T04:14:34.993" LastActivityDate="2018-04-15T04:14:34.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2952" PostTypeId="2" ParentId="2950" CreationDate="2017-03-09T17:55:35.353" Score="4" Body="&lt;p&gt;As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it all depends on the number of features you will be using. However, the more output classes the more data you will need for proper discrimination. The addition of more classes is not linear but I think you can get away with: 10*(# of features)*(# of output classes)&lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-03-09T17:55:35.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2953" PostTypeId="1" CreationDate="2017-03-09T18:09:04.410" Score="3" ViewCount="83" Body="&lt;p&gt;How would one go about building an AI that is capable to look at any kind of input and then identify what is the nature of this data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, an AI that is able to do image classification, NLP and react to some other sensors. Is it possible to build an AI that will be able to identify what kind of data it is seeing such that it can send the data to the correct model for it to be treated, similarly to how the human brain knows to send visual information to the visual cortex and auditory information elsewhere? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a simple scenario, I think we can get very good performance by having a cascaded image classifier. For example 2 layers, the first layer identifies if the image contains a dog and a cat. The next layer, has two different CNNs, one trained to identify the breed of dog and the other one for cats. That way once we identify that we have a dog, the image can be sent to the correct CNN. A CNN that is trained specifically to detect the breed, thus being much more robust that a more generalized CNN. Kind of like a professional in the field. First, the human identifies that he is looking at a dog, then he consults a professional to ask him the breed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to extend this idea to be able to identify various kinds of data sources that do not resemble each other at all. Various input. Are there any models that can do this?&lt;/p&gt;&#xA;" OwnerUserId="5925" LastEditorUserId="2444" LastEditDate="2019-10-22T20:36:30.253" LastActivityDate="2019-10-22T20:36:30.253" Title="Using AI to interpret the nature a specific input and use the correct model" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2954" PostTypeId="2" ParentId="2942" CreationDate="2017-03-09T20:04:17.233" Score="5" Body="&lt;p&gt;Deep Learning these days mean a lot of things to a lot of people, its quickly becoming a buzz-word. But so far it still retains two very important conceptual properties:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does away with most feature engineering work.&lt;/strong&gt; This was mentioned in the answer above, but this is very important. It really saves a lot of work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Allows you to make maximal use of unlabelled data.&lt;/strong&gt; This is strictly speaking available to other approaches, not just Deep Learning, but its in DL that this really took off. And typically labelled data is very hard to get while unlabelled is all over the place. Things like denoising autoencoders and Restricted Boltzmann machines are just wonderful.&lt;/p&gt;&#xA;" OwnerUserId="5941" LastActivityDate="2017-03-09T20:04:17.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2955" PostTypeId="1" AcceptedAnswerId="3184" CreationDate="2017-03-10T00:09:02.040" Score="8" ViewCount="442" Body="&lt;p&gt;What's the term (if such exists) for merging with AI (e.g. via neural lace) and becoming so diluted (e.g. 1:10000) that it effectively results in a death of the original self?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not quite &quot;digital ascension&quot;, because that way it would still be you. What I'm thinking is, that the resulting AI with 1 part in 10000 being you, is not you anymore. The AI might have some of your values or memories or whatever, but it's not you, and you don't exist separately from it to be called you. Basically - you as you are dead; you died by dissolving in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to read up on this subject, but can't find anything.&lt;/p&gt;&#xA;" OwnerUserId="5947" LastEditorUserId="7488" LastEditDate="2017-06-17T21:23:59.583" LastActivityDate="2019-03-07T23:01:38.950" Title="What's the term for death by dissolving in AI?" Tags="&lt;agi&gt;&lt;terminology&gt;&lt;control-problem&gt;" AnswerCount="3" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2956" PostTypeId="2" ParentId="2942" CreationDate="2017-03-10T01:31:38.777" Score="2" Body="&lt;p&gt;Deep learning allows you to not know the answer in order to ask the program a question.  Their main benefit is their finite ability and flexible nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with procedural programing to solve problems is you have to know what the computer needs to do in order to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What deep learning does is remove the requirement of the programmer to know how to solve the problem by having them only need to know what the computer needs to know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the entire premise of neural networks. The programmer writes the program for data points required to be known in order to solve a particular problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer is given an input it comes up with an answer.&#xA;If it's answer is wrong it needs to make the answer it gave less likely and the right answer more likely.&#xA;The goal is to get the computer to always get the right answer. If the computer always gets the wrong answer then the neural network it too small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What deep learning is, is a neural network that is deep.&#xA;To answer this you need to know how a neural network is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is based on a neuron:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Finite number of `boolean' inputs (More then one)&lt;/li&gt;&#xA;&lt;li&gt;A weight is attached on each input to define how important&#xA;often though as a float between -1 and 1, but it's just a percentage of how likely each input changes the answer.&lt;/li&gt;&#xA;&lt;li&gt;One boolean output&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A neuron can be a class or function the implementation really doesn't matter. The weight of each input changes as more answers are asked and responses verified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The depth of a neural network is has one layer  when there is one row of neurons between the input and output.&#xA;two layers when a few neurons make decisions on inputs and a final neuron or multiple neurons make decisions biased on those neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is called deep when there are at least four layers of neurons? (do some research don't take my word for it &lt;code&gt;^_^&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The disadvantage of deep learning is that it's ability is finite.&#xA;There is no way a deep neural network by it's self to get smarter then it's programed to be.&#xA;It has a intelligence curve similar to root time if it isn't improved somehow.&#xA;This leads to the other problem in neural networks. While the programmer has no need to know how decisions are made by the computer they still need to know what questions or nodes need to be added.&#xA;The reason this is a problem is if the nodes responsible aren't there the program will be wrong in strange cases and have no way of correcting this on it's own. The larger the network the harder it is to solve these kinds of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will lead to an inevitable solution to have the computer self improve by some type of generative algorithm.&#xA;This has it's own breadth of problems as if not built properly could grow into something unintended which wastes time and money if it fails quickly, and could be potentially dangerous if it appears to work and doesn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to AI will be a combination of deep neural networks some generative type programing and some new ideas and innovations.&lt;/p&gt;&#xA;" OwnerUserId="4844" LastActivityDate="2017-03-10T01:31:38.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2957" PostTypeId="1" AcceptedAnswerId="3304" CreationDate="2017-03-10T06:52:41.560" Score="6" ViewCount="165" Body="&lt;p&gt;Human beings are more productive in groups than individually, possibly due to the fact that there is a limit to how much one human brain can improve itself in terms of speed of computation and areas of expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By contrast, if a machine with general-purpose artificial intelligence is created and then assigned a task, would it be possible that the machine will be able to better accomplish its task by continuously improving its own computational power and mastery of various skills, as opposed to collaborating with other agents (whether copies of itself, other AI's, or even humans)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, would an AGI ever need to collaborate, or would it always be able to achieve its goals alone?&lt;/p&gt;&#xA;" OwnerUserId="1321" LastEditorUserId="33" LastEditDate="2017-03-12T02:21:57.833" LastActivityDate="2017-05-11T07:27:54.420" Title="Would a general-purpose AI need to collaborate?" Tags="&lt;comparison&gt;&lt;multi-agent-systems&gt;&lt;swarm-intelligence&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2958" PostTypeId="2" ParentId="2846" CreationDate="2017-03-10T09:13:02.953" Score="3" Body="&lt;p&gt;This is one of the main research areas of my &lt;a href=&quot;https://blinclab.ca/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;lab&lt;/a&gt; which researches intelligent prosthetics which also give sensory feedback such as touch and kinaesthesia (the feeling of a limb moving in space) to the user.  We use reinforcement learning to bridge the gap in control and have preliminary work in communicating to the user predictions made by the artificial agent.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-04-11T19:12:16.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2959" PostTypeId="1" AcceptedAnswerId="2960" CreationDate="2017-03-10T13:59:43.357" Score="9" ViewCount="680" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_winter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; (citations omitted):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the history of artificial intelligence, an AI winter is a period of reduced funding and interest in artificial intelligence research. The term was coined by analogy to the idea of a nuclear winter. The field has experienced several hype cycles, followed by disappointment and criticism, followed by funding cuts, followed by renewed interest years or decades later.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Wikipedia page discusses a bit about the &lt;em&gt;causes&lt;/em&gt; of AI winters. I'm curious, however, whether it is possible to &lt;em&gt;stop&lt;/em&gt; an AI Winter from occurring. I don't really like the misallocation of resources that are caused by over-investment followed by under-investment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the causes of the AI winter listed on that Wikipedia page is &quot;hype&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The AI winters can be partly understood as a sequence of over-inflated expectations and subsequent crash seen in stock-markets and exemplified by the railway mania and dotcom bubble. In a common pattern in the development of new technology (known as hype cycle), an event, typically a technological breakthrough, creates publicity which feeds on itself to create a &quot;peak of inflated expectations&quot; followed by a &quot;trough of disillusionment&quot;. Since scientific and technological progress can't keep pace with the publicity-fueled increase in expectations among investors and other stakeholders, a crash must follow. AI technology seems to be no exception to this rule.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And it seems that this paragraph indicates that &lt;em&gt;any&lt;/em&gt; new technology will be stuck in this pattern of &quot;inflated expectations&quot; followed by disillusionment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, are AI winters inevitable? Is it inevitable that AI technologies will always be overhyped in the future and that severe &quot;corrections&quot; will always occur? Or can there a way to manage this hype cycle to stop severe increases/decreases in funding?&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="2444" LastEditDate="2019-10-24T16:57:52.397" LastActivityDate="2019-10-24T16:57:52.397" Title="Are AI winters inevitable?" Tags="&lt;history&gt;&lt;social&gt;&lt;ai-winter&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2960" PostTypeId="2" ParentId="2959" CreationDate="2017-03-10T19:17:25.183" Score="1" Body="&lt;p&gt;I think that by strict definition of the word inevitable, &lt;strong&gt;no, future AI Winter events are not inevitable.&lt;/strong&gt; However likely or unlikely it may be, it is possible to control research spending and to create a more stable plan of funding research in Artificial Intelligence. Because it is &lt;em&gt;possible&lt;/em&gt; to avoid an AI Winter event, an event is not &lt;em&gt;inevitable&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2017-03-12T00:54:47.573" LastActivityDate="2017-03-12T00:54:47.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2961" PostTypeId="2" ParentId="2900" CreationDate="2017-03-11T02:17:56.760" Score="0" Body="&lt;p&gt;Yes, it is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When humans were working on the first nuclear bomb, some field experts of the time thought that when the reaction went super-critical, it would not stop, and would devour the earth. It was a plausible &lt;em&gt;possibility&lt;/em&gt; given our understand of nuclear energy at the time, and we didn't know for sure until we did it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some scientists synthesize black-hole like environments in laboratories. Some experts think that if a certain point is accidentally crossed due to ignorance or negligence, we may devour our planet with a self made black hole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is the same with AI. Until we actually create a super-intelligent AI, we &lt;strong&gt;cannot say with certainty&lt;/strong&gt; whether it will be controlled or controllable until it happens. Until that time comes the answer to your question is yes, it's possible, but that does not mean it will or will not happen that way.&lt;/p&gt;&#xA;" OwnerUserId="5967" LastActivityDate="2017-03-11T02:17:56.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2962" PostTypeId="2" ParentId="2917" CreationDate="2017-03-11T02:55:01.700" Score="2" Body="&lt;p&gt;This depends on your definition of human-like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean a robot that looks and acts like a human, arguably, yes. Here's one of many examples: &lt;a href=&quot;http://www.hansonrobotics.com/robot/sophia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.hansonrobotics.com/robot/sophia/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for something that performs work and tasks, or works and thinks and talks like-or better than a human, the answer is mostly no, not yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend you look at 'ANI, AGI, ASI&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ANI: artificial narrow intelligence. This is what you see around you right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AGI: artificial general intelligence. A theoretic AI that can &quot;think&quot; like a human. It does not yet exist. &lt;strong&gt;Estimates&lt;/strong&gt; are between 20-60 years before we will successfully create AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ASI: artificial super intelligence. In a nutshell, it is theorized to be everything we wish we could be or hope never to be. It does not exist yet. It is &lt;strong&gt;generally&lt;/strong&gt; believed that, IF we create an AGI, ASI will evolve seconds or less than a decade after AGI is created.&lt;/p&gt;&#xA;" OwnerUserId="5967" LastActivityDate="2017-03-11T02:55:01.700" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2963" PostTypeId="2" ParentId="2771" CreationDate="2017-03-11T03:40:11.090" Score="1" Body="&lt;p&gt;Imho, it &lt;strong&gt;is&lt;/strong&gt; life. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: consider the possibility that we synthesized from completethe DNA of a human being, with zero atoms from another human, and grew said human in a lab. Most (and myself) would agree that creature is alive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there are many opinions that differ, my own is that there is no absolute line to draw between something that is alive, and something that is not alive. A human is alive. But is a single bacterial cell, or a single cell from your own body? They reproduce, they eat, etc. so yes they are alive. They are not like a dog or a cat however. In fact, a bacterial colony in a pool of water, giving off a yellow or brown color can be mistaken as a mineral or mud. It is only when you look closer that you see it is actually life. What about a biological virus? It is not made of cells. It does not have DNA. Most would agree it has life. But is does not really seem to be as alive as say, a shark or giraffe. Many people do not think a car is alive. Yet cars evolve. They move, they &quot;eat.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Life is simply a way we define things around us. A much more useful and definitive way to categorize life I think, would be to utilize a continuum instead of an all or nothing approach. Rocks would go on the end of &quot;nonliving.&quot; Intelligent, multicellular; self-aware entities could perhaps be on the other end as &quot;fully alive.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other entities can go in between. As for something such as AI, I would propose adding a z-axis, to make a 3 dimensional continuum, allowing for a self aware, intelligence entity not made of cells to fit comfortably with humans without causing an all-or-nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: thought I came to this conclusion myself, I have a suspicion someone smarter than me has already written about such an idea. If anyone feels like educating me, I'd love to hear it.&lt;/p&gt;&#xA;" OwnerUserId="5967" LastActivityDate="2017-03-11T03:40:11.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2964" PostTypeId="1" CreationDate="2017-03-11T09:48:32.563" Score="3" ViewCount="1272" Body="&lt;p&gt;Does artificial intelligence write its own code and then execute it? If so, does it create separate functions for each purpose?&lt;/p&gt;&#xA;" OwnerUserId="5972" LastEditorUserId="2444" LastEditDate="2019-06-25T17:06:49.883" LastActivityDate="2019-06-25T17:33:18.607" Title="Does artificial intelligence write its own code?" Tags="&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;theory&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2965" PostTypeId="2" ParentId="2959" CreationDate="2017-03-11T11:48:22.057" Score="0" Body="&lt;p&gt;The hype cycles are the rule these days, and AI is always a wonderful topic for unbelievable and crazy hype. I mean simple thing like speech recognition is still not working properly, but everybody is discussing how to survive the revolt of the terminator machines. So unless we can tune the hype down, the next AI winter is inevitable.&lt;/p&gt;&#xA;" OwnerUserId="5941" LastActivityDate="2017-03-11T11:48:22.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2966" PostTypeId="2" ParentId="2964" CreationDate="2017-03-11T14:50:00.113" Score="1" Body="&lt;p&gt;Computers are able to write their own code &lt;em&gt;without&lt;/em&gt; needing any intelligence -- see the Wikipedia entries for &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-modifying_code&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-modifying code&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Metaprogramming&quot; rel=&quot;nofollow noreferrer&quot;&gt;metaprogramming&lt;/a&gt;. You do have to write the instructions for how the computer should program itself, and there's a stigma against doing this because (a) it makes it hard to reason about what your program is doing when it's changing its source code, and (b) the solution is usually slower than just hardcoding in what you want the program to do in the first place. But it &lt;em&gt;is&lt;/em&gt; possible, and programmers have done it (usually for maintainability or aesthetic reasons).&lt;/p&gt;&#xA;&lt;p&gt;Some AI researchers are interested in &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt; though. Genetic Programming is a subset of &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolutionary algorithms&lt;/a&gt; and Wikipedia provides a good summary of how they usually work:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Step One: Generate the initial population of individuals randomly. (First generation)&lt;/p&gt;&#xA;&lt;p&gt;Step Two: Evaluate the fitness of each individual in that population&lt;/p&gt;&#xA;&lt;p&gt;Step Three: Repeat the following regenerational steps until termination (time limit, sufficient fitness achieved, etc.):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Select the best-fit individuals for reproduction. (Parents)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Breed new individuals through crossover and mutation operations to give birth to offspring.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Evaluate the individual fitness of new individuals.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace least-fit population with new individuals.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &amp;quot;individuals&amp;quot; in this case are randomly-generated computer programs, which are then tested against a fitness function.&lt;/p&gt;&#xA;&lt;p&gt;The Wikipedia page for Genetic Programming claimed that these programs are usually represented by tree structures, though there has been some experiments in using non-tree structures as well.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-03-11T14:55:05.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2967" PostTypeId="1" AcceptedAnswerId="5047" CreationDate="2017-03-11T16:56:49.653" Score="2" ViewCount="224" Body="&lt;p&gt;As far as I know MDP are independent from the past. But the definition says that the same policy should always take the same action depending on the state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if I define my state as the current &quot;main&quot; state + previous decisions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Example in Poker the &quot;main&quot; state would be my cards and the pot + all previous information about the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this still be a MDP or not? &lt;/p&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="4550" LastEditDate="2017-03-11T17:11:20.420" LastActivityDate="2018-01-17T22:37:58.363" Title="Can an Markov decision process be dependent on the past?" Tags="&lt;definitions&gt;&lt;markov-chain&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2969" PostTypeId="2" ParentId="2967" CreationDate="2017-03-11T20:38:34.017" Score="3" Body="&lt;p&gt;It's not totally clear from your description, but it sounds like you may be onto something like an &lt;a href=&quot;https://en.wikipedia.org/wiki/Additive_Markov_chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Additive Markov Chain&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-03-11T20:38:34.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2970" PostTypeId="2" ParentId="2964" CreationDate="2017-03-11T23:25:22.243" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does artificial intelligence write its own code and then execute it? If so, does it create separate functions for each purpose?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence comprises more than one technique. Here you seem to be asking about &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt;, a subfield of AI which itself contains many subfields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-modifying code &lt;em&gt;is&lt;/em&gt; one of the (many) techniques used in some applications of artificial intelligence.  So, in that sense, the answer to your question is &quot;yes&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks&lt;/a&gt; (NNs), for example, the NN isn't writing any code, it is just running through an optimization algorithm (and back-propagation) that incrementally changes some weights (or coefficients) such that when you enter a certain input, you get an output that's close to the desired output.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also have &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithms&lt;/a&gt; (GA) to evolve the weights of a neural network, as opposed to using back-propagation. GAs are closer to the idea of &quot;an AI coding itself&quot; (although not exactly).&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2019-06-25T17:09:31.800" LastActivityDate="2019-06-25T17:09:31.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2971" PostTypeId="2" ParentId="2917" CreationDate="2017-03-11T23:39:05.963" Score="0" Body="&lt;p&gt;I would say that we're not even close to a &quot;real&quot; human-like AI. For all the wonderful things that applications like Siri, Cortana and the like can do, they're actually really dumb compared to even a child.  Of course part of that, IMO, is that most AI applications are not embodied and don't experience the world the way humans do.  So if you show an AI a video of a dog walking behind a table and briefly disappearing from the frame, it has pretty much no ability to apply &quot;common sense&quot; and know that the dog will reappear in a few seconds, and that if it does't reappear, it's probably because it found its favorite toy on the floor behind the table.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some examples of the kinds of &quot;easy&quot; questions that computers still don't do well at, check out the &lt;a href=&quot;https://en.wikipedia.org/wiki/Winograd_Schema_Challenge&quot; rel=&quot;nofollow noreferrer&quot;&gt;Winograd Schema Challenge&lt;/a&gt;.  You might also find this page interesting:  &lt;a href=&quot;http://www-formal.stanford.edu/leora/commonsense/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www-formal.stanford.edu/leora/commonsense/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-03-11T23:39:05.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2972" PostTypeId="2" ParentId="2957" CreationDate="2017-03-12T05:29:01.003" Score="0" Body="&lt;p&gt;Communications is expensive. It requires a communication channel, a protocol and of course time. Communications is also limited to the expressivity defined by the protocol. Note also that agents may compete over resources, or may have contradictory goals, so in some situations they may try to mislead each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand - computational power and memory are limited, so multiple agents may solve computation-intensive or memory-intensive problems together better/faster than each single agent can. Different agents may have different sensors, and mobile agents may have information about different parts of their realm, so by sharing knowledge they may have more complete information and make better decisions. Goals may also be time-bounded, and rewards may be time-dependent. Sometimes working together means greater rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, there may be situations where collaboration is beneficial, and there may be some situations where collaboration is essential to achieve one's goal or to achieve a common goal.&lt;/p&gt;&#xA;" OwnerUserId="3138" LastActivityDate="2017-03-12T05:29:01.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2973" PostTypeId="1" CreationDate="2017-03-12T18:05:51.793" Score="3" ViewCount="99" Body="&lt;p&gt;I'm looking for AI systems or natural language processors that use, in the classification and interrelation of notions/objects, some philosophical system, like basic laws of logic, Kantian or empiricism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also read about goal-seeking procedures. Are these based on psychology fields and some particular psychology theory or these are ad hoc experiments, with only general terms applied?&lt;/p&gt;&#xA;" OwnerDisplayName="user5977" LastEditorUserId="2444" LastEditDate="2019-05-01T21:40:32.640" LastActivityDate="2019-05-01T21:40:32.640" Title="What are examples of AI that use philosophy derived ontologies?" Tags="&lt;classification&gt;&lt;philosophy&gt;&lt;logic&gt;&lt;ontology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2974" PostTypeId="1" CreationDate="2017-03-12T18:30:34.533" Score="0" ViewCount="502" Body="&lt;p&gt;I'm using a NN created with CNTK's SimpleNetworkBuilder to make choices (specifically in board games). I specified ReLU as the layer type, so outputs can be arbitrary numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When evaluating a custom set of features, getting the &quot;choice&quot; of the function/model is simple: Look for the output signal with the highest value. However, there are times when I wish to introduce some randomness and assign probabilities to each output signal, then select the choice based on each output's probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, what I'm doing is manually normalizing all the output using a sigmoid function specified here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Logistic_function&lt;/a&gt;&#xA;Then, I multiply them all by a scalar such that the sum total of all outputs is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, I pick a random number 0..1, and see where along the map it falls; that is my selected choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'd like to know is, is there a better way?&lt;/p&gt;&#xA;" OwnerUserId="3702" LastActivityDate="2018-01-09T08:39:56.390" Title="Assigning probability to output of a ReLU network" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2975" PostTypeId="2" ParentId="2973" CreationDate="2017-03-12T23:25:04.883" Score="1" Body="&lt;p&gt;Excellent question!  I'm currently working on the pre-Socratics as the most basic philosophies for first principles (These philosophers are intriguing for their simplicity and universality, and the &quot;dawn of consciousness&quot;, in some conceptions, may be ascribed to the Classical Era.  Linguistically, ancient Greek is fundamental to meaning in the West.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I was pointed at this very interesting blog post &quot;&lt;a href=&quot;https://arimo.com/featured/2015/algorithms-of-the-mind/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithms of the Mind&lt;/a&gt;&quot; which has some useful links, and discusses Kant, which you may find useful. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-03-12T23:25:04.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2976" PostTypeId="1" AcceptedAnswerId="2988" CreationDate="2017-03-13T12:46:50.537" Score="2" ViewCount="108" Body="&lt;p&gt;I have come across this domain via this Wikipedia article: &lt;a href=&quot;https://en.wikipedia.org/wiki/General_game_playing&quot; rel=&quot;nofollow noreferrer&quot;&gt;General game playing&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, where are we when it comes to general game playing AI? (The wiki article doesn't mention the recent advances and the achievements of this domain of research, except the annual games results.)&lt;/p&gt;&#xA;&lt;p&gt;PS: I understand that this is a General project of the Stanford Logic Group of Stanford University, California. But since then, it has become an area of research in the domain of AI.&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="2444" LastEditDate="2021-04-20T16:57:14.390" LastActivityDate="2021-04-20T16:57:14.390" Title="What research has been done in the domain of “general game playing”?" Tags="&lt;reference-request&gt;&lt;game-ai&gt;&lt;research&gt;&lt;state-of-the-art&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2977" PostTypeId="2" ParentId="2974" CreationDate="2017-03-13T16:32:33.657" Score="1" Body="&lt;p&gt;I would just skip the sigmoid function step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider these two scenarios with three choices with the associated values:&#xA;0.1,10,100 or 1,100,1000&#xA;Given that these scenarios are equivalent in their relativ values, your probabilities should be assigned in the same way. But if you throw in a sigmoid function, the difference between 10 and 100 will be bigger than between 100 and 1000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just normalise the values, so that they sum to 1. In that case the choice with a value ten times higher than the value of another choice will be picked ten times as often. Makes sense to me. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-03-13T16:32:33.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2978" PostTypeId="2" ParentId="2953" CreationDate="2017-03-14T09:15:39.117" Score="1" Body="&lt;p&gt;Because of all the inputs you could give are obviously a list of number. It will only require some CNNs and fully-conncted, that classify what kind of data is it and then pass the data to the right stack of layers. The only tiny problem is that all the the things you pass into the network must always be a matrix(to handle also images) with a certain size. So even if you  pass only a tiny chunk of data you must transform it into a  fixed-size matrix and fill all the empty places with 0s, and that could bring to an accuracy lack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to solve that you could also try to slide over your data a tiny CNN-RNN to handle different sizes .&lt;/p&gt;&#xA;" OwnerUserId="2320" LastActivityDate="2017-03-14T09:15:39.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2980" PostTypeId="1" AcceptedAnswerId="2981" CreationDate="2017-03-14T14:26:07.403" Score="31" ViewCount="10281" Body="&lt;p&gt;I want to create an AI which can play five-in-a-row/gomoku. I want to use reinforcement learning for this.&lt;/p&gt;&#xA;&lt;p&gt;I use &lt;em&gt;policy gradient&lt;/em&gt; method, namely REINFORCE, with baseline. For the value and policy function approximation, I use a &lt;em&gt;neural network&lt;/em&gt;. It has convolutional and fully connected layers. All of the layers, except for the output, are shared. The policy's output layer has &lt;span class=&quot;math-container&quot;&gt;$8 \times 8=64$&lt;/span&gt; (the size of the board) output unit and &lt;em&gt;softmax&lt;/em&gt; on them. So it is stochastic.&lt;/p&gt;&#xA;&lt;p&gt;But what if the network produces a very high probability for an invalid action? An invalid action is when the agent wants to check a square that has one &lt;code&gt;X&lt;/code&gt; or &lt;code&gt;O&lt;/code&gt; in it. I think it can be stuck in that game state.&lt;/p&gt;&#xA;&lt;p&gt;How should I handle this situation?&lt;/p&gt;&#xA;&lt;p&gt;My guess is to use the &lt;em&gt;actor-critic&lt;/em&gt; method. For an invalid action, we should give a negative reward and pass the turn to the opponent.&lt;/p&gt;&#xA;" OwnerUserId="6019" LastEditorUserId="2444" LastEditDate="2020-11-14T18:14:45.573" LastActivityDate="2020-11-14T18:14:45.573" Title="How should I handle invalid actions (when using REINFORCE)?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;reinforce&gt;&lt;reward-design&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="2981" PostTypeId="2" ParentId="2980" CreationDate="2017-03-14T15:37:23.737" Score="14" Body="&lt;p&gt;Just ignore the invalid moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For exploration it is likely that you won't just execute the move with the highest probability, but instead choose moves randomly based on the outputted probability. If you only punish illegal moves they will still retain some probability (however small) and therefore will be executed from time to time (however seldom). So you will always retain an agent which occasionally makes illegal moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me it makes more sense to just set the probabilities of all illegal moves to zero and renormalise the output vector before you choose your move. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-03-14T15:37:23.737" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2986" PostTypeId="2" ParentId="2959" CreationDate="2017-03-14T18:14:36.783" Score="1" Body="&lt;p&gt;Yes - there will always be Gartner Hype Cycles which leads to &quot;AI Winters&quot; - that is just a fact of human nature in large groups. There is no better evidence for &quot;Hype Cycle&quot; mentality than the stock market in how it reacts both high and low to whatever the hot item is. AI is much more susceptible to this given that AI tends to touch people in very real ways - will this technology become smarter than me? Will it replace me? Will it take over? Are we building new life? Who controls this? which for those that actually build or know something about these techniques and concepts would say that we are a very long way off it is even possible in the first place. To build systems that can at best maybe mimic the intelligence of a two year old we would consider it a major success. &lt;/p&gt;&#xA;" OwnerUserId="6023" LastActivityDate="2017-03-14T18:14:36.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2988" PostTypeId="2" ParentId="2976" CreationDate="2017-03-14T18:19:13.963" Score="3" Body="&lt;p&gt;If you haven't already come across DeepMind's advances in developing general game playing AI, you can take a look at it's &lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN research&lt;/a&gt;. The paper describes how their deep reinforcement learning system is able to beat human levels in nearly Atari 2600 games with raw pixels and scores as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also here's an interesting website - &lt;a href=&quot;http://www.gvgai.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Video Game AI Competition 2017&lt;/a&gt; that also hosts links to the latest advances and research papers in this field.&lt;/p&gt;&#xA;" OwnerUserId="3191" LastActivityDate="2017-03-14T18:19:13.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2989" PostTypeId="1" CreationDate="2017-03-14T18:26:42.837" Score="0" ViewCount="109" Body="&lt;p&gt;To create language flashcards I would like to split an audio course into many single audio clips. They're basically a man and a woman speaking after each other. The intervals aren't regular so it's not possible to split it after time intervals. Furthermore silence detection is not possible since some sentences also include pauses. I have already tried diarization using LIUM but the timings were completely wrong. Additionally the audio course includes a transcript which would certainly be machine-readable consisting of the English sentence and the Japanese sentence as well as its Romaji version (Japanese words using English letters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not experienced in AI, so I'm looking for a solution which isn't to difficult (like constructing and training my own neural network).&#xA;I have some programming experience, so a mathematical approach would be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Links:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;audio course: &lt;a href=&quot;http://www.japaneseaudiolessons.com/download-japanese-lessons/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.japaneseaudiolessons.com/download-japanese-lessons/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LIUM: &lt;a href=&quot;http://lium3.univ-lemans.fr/diarization/doku.php/welcome&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lium3.univ-lemans.fr/diarization/doku.php/welcome&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6024" LastEditorUserId="4302" LastEditDate="2018-10-08T12:51:29.403" LastActivityDate="2018-10-08T12:51:29.403" Title="Splitting audio consisting of male / female speaker into segments" Tags="&lt;voice-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2990" PostTypeId="2" ParentId="2959" CreationDate="2017-03-14T22:38:13.353" Score="0" Body="&lt;p&gt;I would say an AI Winter has already happened in the 2000's when specialist systems were adopted in detriment of cognitive systems, neural networks for instance were poorly understood back then and because of that they got meager investments from large companies, Google was a notorious exception. Only some 3 or so years ago, with things like IBM Watson and driverless cars this field started to draw significant attention. And now I doubt it will be ignored again, the research has taken off from advanced PhD theses and becomes more and more widespread.&lt;/p&gt;&#xA;" OwnerUserId="6030" LastActivityDate="2017-03-14T22:38:13.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2994" PostTypeId="2" ParentId="2980" CreationDate="2017-03-15T06:21:41.317" Score="11" Body="&lt;p&gt;Usually softmax methods in policy gradient methods using linear function approximation use the following formula to calculate the probability of choosing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. Here, weights are &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, and the features &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is a function of the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and an action from the set of actions &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi(\theta, a) = \frac{e^{\theta \phi(s, a)}}{\sum_{b \in A} e^{\theta \phi(s, b)}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To eliminate illegal moves, one would limit the set of actions to only those that were legal, hence &lt;span class=&quot;math-container&quot;&gt;$Legal(A)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi(\theta, a) = \frac{e^{\theta \phi(s, a)}}{\sum_{b \in Legal(A)} e^{\theta \phi(s, b)}}, \, a \in Legal(A)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pseudocode the formula may look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;action_probs = Agent.getActionProbs(state)&#xA;legal_actions = filterLegalActions(state, action_probs)&#xA;best_legal_action = softmax(legal_actions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Whether using linear or non-linear function approximation (your neural network), the idea is to only use the legal moves when computing your softmax. This method means that only valid moves will be given by the agent, which is good if you wanted to change your game later on, and that the difference in value between the limited choice in actions will be easier to discriminate by the agent. It will also be faster as the number of possible actions decreases.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2018-11-14T13:58:07.037" LastActivityDate="2018-11-14T13:58:07.037" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2996" PostTypeId="1" CreationDate="2017-03-15T09:10:28.523" Score="12" ViewCount="2401" Body="&lt;p&gt;I heard several times that one of the fundamental/open problems of deep learning is the lack of &quot;general theory&quot; on it because actually we don't know why deep learning works so well. Even the Wikipedia page on deep learning has &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning#Criticism_and_comment&quot; rel=&quot;noreferrer&quot;&gt;similar comments&lt;/a&gt;. Are such statements credible and representative of the state of the field?&lt;/p&gt;&#xA;" OwnerUserId="6039" LastEditorUserId="75" LastEditDate="2017-03-15T14:47:05.483" LastActivityDate="2020-04-20T15:52:51.273" Title="Is there actually a lack of fundamental theory on deep learning?" Tags="&lt;deep-learning&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2997" PostTypeId="2" ParentId="2996" CreationDate="2017-03-15T10:47:38.643" Score="6" Body="&lt;p&gt;There is a paper called &lt;a href=&quot;https://arxiv.org/pdf/1608.08225.pdf&quot; rel=&quot;noreferrer&quot;&gt;Why does Deep Learning work so well?&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;However, it is still not fully understood why deep learning works so well. In contrast to GOFAI (“good old-fashioned AI”) algorithms that are hand-crafted and fully understood analytically, many algorithms using artificial neural networks are understood only at a heuristic level, where we empirically know that certain training protocols employing large data sets will result in excellent performance. This is reminiscent of the situation with human brains: we know that if we train a child according to a certain curriculum, she will learn certain skills — but we lack a deep understanding of how her brain accomplishes this.&quot;&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-03-15T10:47:38.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2998" PostTypeId="2" ParentId="2996" CreationDate="2017-03-15T15:21:34.540" Score="4" Body="&lt;p&gt;This is very much the case. Deep learning models even shallow ones such as stacked autoencoders and neural networks are not fully understood. There are efforts to understand what is happening to the optimization process for such a complex variable intensive function. But, this is a difficult task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way that researchers are using to discover how deep learning works is by using generative models. First we train a learning algorithm and handicap it systematically whilst asking it to generate examples. By observing the resulting generated examples we will be able to infer what is happening in the algorithm at a more significant level. This is very much like using inhibitors in neuroscience to understand what different components of the brain are used for. For example, we know that the visual cortex is where it is because if we damage it you will go blind.  &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-03-15T15:21:34.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2999" PostTypeId="1" AcceptedAnswerId="3005" CreationDate="2017-03-16T02:53:19.430" Score="7" ViewCount="1689" Body="&lt;p&gt;I had been reading that AI could solve planet's major problems. How could it be done? For example, how exactly could AI be applied to address climate change? What are examples of applications of AI to solve these problems?  &lt;/p&gt;&#xA;" OwnerUserId="4460" LastEditorUserId="2444" LastEditDate="2018-07-05T18:46:38.173" LastActivityDate="2018-07-05T18:46:38.173" Title="How could AI solve planet's major problems?" Tags="&lt;applications&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3001" PostTypeId="2" ParentId="2692" CreationDate="2017-03-16T11:36:43.150" Score="1" Body="&lt;p&gt;I'm a student in the Deep Learning Foundation Nanodegree. It's going great! The beginning was harder due to the math and we had to understand what is going under the hood. Only after this you use frameworks like Tensorflow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My math is at high-school level, and the math that you must know or learn isn't much, &lt;strong&gt;they said&lt;/strong&gt; that you must know basic algebra, calculus etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must definitely know how to program and if you know how to program you should be able keep up with the course if you enjoy what you are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only real complaint I have is that they claimed it would cost 3-5 hours a week, but it's more like 9-15 hours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Udacity isn't capitalizing on the A.I hype, it's trying to capitalize on the future and learning of technology.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="5388" LastActivityDate="2017-03-16T11:36:43.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3002" PostTypeId="1" AcceptedAnswerId="3003" CreationDate="2017-03-16T11:41:46.100" Score="15" ViewCount="1312" Body="&lt;p&gt;For example, for classifying emails as spam, is it worthwhile - from a  time/accuracy perspective - to apply &lt;em&gt;deep learning&lt;/em&gt; (if possible) instead of another machine learning algorithm? Will deep learning make other machine learning algorithms like &lt;em&gt;naive Bayes&lt;/em&gt; unnecessary?&lt;/p&gt;&#xA;" OwnerUserId="5388" LastEditorUserId="2444" LastEditDate="2019-05-29T22:45:22.377" LastActivityDate="2019-05-29T22:52:11.610" Title="When is deep learning overkill?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;applications&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="3003" PostTypeId="2" ParentId="3002" CreationDate="2017-03-16T15:03:04.380" Score="13" Body="&lt;p&gt;It's all about &lt;em&gt;Return On Investment&lt;/em&gt;. If DL is &quot;worth doing&quot;, it's not overkill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is plentiful, and if the marginal advantage over alternative algorithms is valuable, then DL is a win.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, as you suggest, if your problem is amenable to alternate methods, especially if it offers a signal that matches up well with classic methods like regression or naive Bayes, or your problem requires explanation of why the decision boundary is where it is (e.g. decision trees), or if your data lacks the continuous gradients needed by DL (especially, CNNs), or your data varies over time which would require periodic retraining (especially, at unpredictable intervals), then DL probably is a mismatch for you.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastEditorUserId="2444" LastEditDate="2019-05-29T22:52:11.610" LastActivityDate="2019-05-29T22:52:11.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3004" PostTypeId="2" ParentId="1294" CreationDate="2017-03-16T16:20:15.497" Score="13" Body="&lt;p&gt;To supplement the previous answer: there is a paper on this that is mostly about learning low-level capsules from raw data, but explains Hinton's conception of a capsule in its introductory section: &lt;a href=&quot;http://www.cs.toronto.edu/~fritz/absps/transauto6.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.cs.toronto.edu/~fritz/absps/transauto6.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also worth noting that the link to the MIT talk in the answer above seems to be working again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Hinton, a &quot;capsule&quot; is a subset of neurons within a layer that outputs both an &quot;instantiation parameter&quot; indicating whether an entity is present within a limited domain and a vector of &quot;pose parameters&quot; specifying the pose of the entity relative to a canonical version. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameters output by low-level capsules are converted into predictions for the pose of the entities represented by higher-level capsules, which are activated if the predictions agree and output their own parameters (the higher-level pose parameters being averages of the predictions received). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hinton speculates that this high-dimensional coincidence detection is what mini-column organization in the brain is for. His main goal seems to be replacing the max pooling used in convolutional networks, in which deeper layers lose information about pose.&lt;/p&gt;&#xA;" OwnerUserId="6020" LastActivityDate="2017-03-16T16:20:15.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3005" PostTypeId="2" ParentId="2999" CreationDate="2017-03-16T19:50:37.300" Score="3" Body="&lt;p&gt;There's indeed a lot of research work being done in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few ways in which AI can help/is helping in fighting major problems:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Climate change&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Identify deforestation and the rate at which it's happening using computer vision and help in fighting back based on how critical the rate is.The World Resources Institute had entered into a &lt;a href=&quot;https://www.wired.com/2015/04/using-smart-satellites-to-monitor-deforestation-from-space/&quot; rel=&quot;nofollow noreferrer&quot;&gt;partnership&lt;/a&gt; with Orbital Insight on this.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Protecting Nature&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Researchers and institutes are &lt;a href=&quot;http://www.grevyszebratrust.org/stripe-recognition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;using&lt;/a&gt; computer vision to accurately identify total remaining members of extremely threatened species to observe their behavior and patterns that can help in protecting them, keep track and monitor at an individual level.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Environmental/ Research agencies are trying to &lt;a href=&quot;http://www.birds.cornell.edu/page.aspx?pid=2713&quot; rel=&quot;nofollow noreferrer&quot;&gt;detect sounds&lt;/a&gt; from ocean audio from which they can identify inhabiting fish and route the ships away from threatened habitats.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Energy&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimise energy usage&lt;/a&gt; based on consumption in big companies to save energy for public use and in turn reduce energy overall demands on the grid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Predict the demand for energy and ensure continuous supply matching the demand. For example, Google's Deepmind is in &lt;a href=&quot;https://www.ft.com/content/27c8aea0-06a9-11e7-97d1-5e720a26771b&quot; rel=&quot;nofollow noreferrer&quot;&gt;talks&lt;/a&gt; National Grid of UK on this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, these ideas might look too tiny or too broad at first sight, but I feel taking one step at a time and then quickly expanding the ideas into wider areas would be the way to go, like how DeepMind is doing it in the area of energy. Other areas like education, transportation, and healthcare are seeing lots of activity with the involvement of AI too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further reading, here's what World Economic Forum &lt;a href=&quot;https://www.weforum.org/agenda/2017/02/5-global-problems-that-ai-could-help-us-solve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thinks&lt;/a&gt; about how AI can help solve world's problems.&lt;/p&gt;&#xA;" OwnerUserId="3191" LastActivityDate="2017-03-16T19:50:37.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3006" PostTypeId="1" CreationDate="2017-03-16T23:31:29.750" Score="10" ViewCount="541" Body="&lt;p&gt;Since the first Industrial revolution machines have been taking the jobs of people and automation has been a part of human social evolution for the past 3 centuries, but all in all these machines have been replacing mechanical, high-risk and low-skill jobs such as a production line of an automobile factory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But recently with the advent of computers and the improvement of AI, and the quest to find a Singularity (that is, a computer capable of thinking faster, better, more creative and &lt;strong&gt;cheaper&lt;/strong&gt; then a human being, capable of self-improving), our future will lead to the replacement of not only low-skill workers, but high-skill as well. I'm talking about a future not too far when AI and machines will replace artists, designers, engineers, lawyers, CEO's, filmmakers, politicians, hell even programmers.&#xA;Some people get excited by this, but honestly, I get somewhat scared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not talking about the money issue here, although I'm not a fan of the idea, let's suppose the universal income has been implemented, and suppose it works fine. Also not talking about the &quot;&lt;em&gt;Terminator's world where machines will wage war against humans&lt;/em&gt;&quot;, let's suppose too they are completely friendly forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue here is the one of &lt;strong&gt;motivation&lt;/strong&gt; for us humans. &lt;em&gt;When the AI singularity takes over, what will there be left for us to do?&lt;/em&gt; Every day, all day long?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are we going to do with our lives? Suppose I love to paint, how can I live my dream of becoming a painter if the computer makes better art then I will ever be able to do? How can I live knowing that no one will care about my paintings because they were made by a &lt;strong&gt;mere human&lt;/strong&gt;? Or the real me for example (I, Danzmann), I love to code, learned my first programming language with 9 years old and been on it ever since then, it looks sad to me that in some years I may never touch on that again. And that goes for all the professions, everyone is passionate about something, and with the singularity, every single one of them would just have to cease to exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what are we going to do in this future? What am I going to do? Play golf all day, every single day for the rest of my life (A Hyperbole figure of speech, but you get my point)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, what is going to be the motivation for my children? What am I going to tell them to go to school? When someone asks &quot;what do you wanna be when you grow up?&quot;, and the inevitable answer is &lt;strong&gt;nothing&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If highly advanced AI takes control of all scientific research, then what is the reason for us to &lt;strong&gt;learn&lt;/strong&gt;? What is the reason that us humans would need to dedicate decades of our lives to learn something if that knowledge is useless, because there are no more jobs and the scientific research is done solely by AI?&lt;/p&gt;&#xA;" OwnerUserId="6084" LastEditorUserId="2444" LastEditDate="2019-06-26T22:33:31.740" LastActivityDate="2019-06-26T22:44:05.463" Title="When the AI singularity takes over, what will there be left for us to do?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;social&gt;&lt;singularity&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3007" PostTypeId="1" AcceptedAnswerId="5584" CreationDate="2017-03-17T06:57:48.737" Score="2" ViewCount="2187" Body="&lt;p&gt;How can Artificial Intelligence be applied to software testing?  &lt;/p&gt;&#xA;" OwnerUserId="6091" LastEditorUserId="33" LastEditDate="2017-03-20T20:27:14.643" LastActivityDate="2019-11-30T22:17:15.587" Title="How can AI techniques be used in software testing?" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3008" PostTypeId="2" ParentId="3002" CreationDate="2017-03-17T08:54:54.797" Score="12" Body="&lt;p&gt;Deep learning is powerful but it is &lt;strong&gt;not&lt;/strong&gt; a superior method than bayesian. They work well in what they are designed to do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use deep learning:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cost for computation is much cheaper than cost of sampling (e.g: natural language processing)&lt;/li&gt;&#xA;&lt;li&gt;If you have highly non-linear problem&lt;/li&gt;&#xA;&lt;li&gt;If you want to simplify feature engineering&lt;/li&gt;&#xA;&lt;li&gt;If you don't have prior distribution (e.g: setting the weights to random Gaussian). Or you do but you don't mind the complexity.&lt;/li&gt;&#xA;&lt;li&gt;If you want accuracy for speed (deep learning is slow)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Use naive bayesian:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have prior distribution that you want to use&lt;/li&gt;&#xA;&lt;li&gt;If you want to update your model quickly and easily (in particular conjour models)&lt;/li&gt;&#xA;&lt;li&gt;If you have your own likelihood function and wish to &quot;control&quot; how exactly the model works &lt;/li&gt;&#xA;&lt;li&gt;If you want to model hierarchial models&lt;/li&gt;&#xA;&lt;li&gt;If you don't want to tweak parameters&lt;/li&gt;&#xA;&lt;li&gt;If you want a faster model, both in training and execution&lt;/li&gt;&#xA;&lt;li&gt;If you want to make the independence assumption&lt;/li&gt;&#xA;&lt;li&gt;If you want to prevent overfitting (that's a very simple model)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-03-17T10:07:59.657" LastActivityDate="2017-03-17T10:07:59.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3009" PostTypeId="1" CreationDate="2017-03-17T10:55:00.683" Score="9" ViewCount="2566" Body="&lt;p&gt;What is the difference between &lt;a href=&quot;https://en.wikipedia.org/wiki/Memetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;memetic algorithms&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithms&lt;/a&gt;? Is an individual's lifetime a learning part of memetic algorithms?&lt;/p&gt;&#xA;" OwnerUserId="6095" LastEditorUserId="2444" LastEditDate="2021-01-16T14:36:39.297" LastActivityDate="2021-01-16T14:47:36.473" Title="What is the difference between memetic algorithms and genetic algorithms?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-computation&gt;&lt;memetic-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3011" PostTypeId="2" ParentId="2692" CreationDate="2017-03-17T13:09:25.977" Score="2" Body="&lt;p&gt;I studied Deep Learning as a part of my master's Degree. My suggestion would be to simply take the free online courses. There's so many amazing resources. There's really no need for you to pay for the course. If you are dedicated to learn the subject then you will. There's also a ton of books that have already been published going over the underlining workings of each of the deep learning techniques. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think it is worth the money. And as Duke Zhou stated, if their is no mention of a deep understanding of mathematics as a prerequisite that is a red flag. Seems like a money grab to me. &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-03-17T13:09:25.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3012" PostTypeId="2" ParentId="2692" CreationDate="2017-03-17T19:15:18.973" Score="2" Body="&lt;p&gt;I'm doing my master in computer science and i really liked the course from caltech ( complementary book is &quot;learning from data&quot;) and the famous course from Andrew Ng which you can get free on cousera. On the programming side, i really like &quot;Python machine learning&quot; from Sebastian Raschka.&lt;/p&gt;&#xA;" OwnerUserId="6105" LastActivityDate="2017-03-17T19:15:18.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3013" PostTypeId="1" AcceptedAnswerId="3037" CreationDate="2017-03-18T04:32:47.590" Score="3" ViewCount="752" Body="&lt;p&gt;Humans often dream of random events that occurred during the day. Could the reason for this be that our brains are backpropagating errors while we sleep, and we see the result of these backpropagations as dreams?&lt;/p&gt;&#xA;" OwnerDisplayName="user6107" LastEditorUserId="1671" LastEditDate="2018-01-27T21:14:35.073" LastActivityDate="2018-01-27T21:50:19.447" Title="Are Dreams a Form of Backpropagation?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;cognitive-science&gt;&lt;theory&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3015" PostTypeId="2" ParentId="3007" CreationDate="2017-03-18T18:56:16.203" Score="3" Body="&lt;p&gt;In large software with many actions and possible flows like web applications, enterprise software, etc, it is really hard and time taking to test out all possible scenarios via traditional approach. So, building a machine learning model is an interesting approach to solving this. A reinforcement learning system with an end goal to crash or make the software unresponsive can be tried.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is research being done on this idea. You can take read &lt;a href=&quot;https://pdfs.semanticscholar.org/d2ae/393cf723228cf6f96d61ee068c681203e943.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; research paper which explores Reinforcement Learning as an approach to automated GUI robustness testing. Also, some companies like Appdiff trying to incorporate AI in software testing with mobile apps in context, but similar thinking can be reasonably extended to web apps.&lt;/p&gt;&#xA;" OwnerUserId="3191" LastActivityDate="2017-03-18T18:56:16.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3016" PostTypeId="2" ParentId="2692" CreationDate="2017-03-18T19:42:35.850" Score="2" Body="&lt;p&gt;I know that I'm late in answering, and also notice that you've enrolled in the course already. But I wanted to suggest some courses available online for free for those who are in still in search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you've already done Andrew Ng's Machine Learning course, you can do &lt;strong&gt;CS231n: Convolutional Neural Networks for Visual Recognition&lt;/strong&gt; course from Stanford taught by Dr. Fei Fei Li, Andrew Karpathy and Justin Johnson. It provides a very clear and thorough introduction to deep learning as well as on CNNs and how they are used in image recognition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For NLP and speech processing/recognition, you can either do &lt;strong&gt;CS224n: Natural Language Processing with Deep Learning&lt;/strong&gt; course from Stanford taught by Richard Socher and Chris Manning or &lt;strong&gt;Deep Learning for Natural Language Processing&lt;/strong&gt; course from Oxford taught by Phil Blunson and many researchers from Deepmind. Both the courses teach NLP with a focus on the recent deep learning approaches within this domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the above three courses have their class videos/notes, and assignments put up available online with decent communities of learners around them.&lt;/p&gt;&#xA;" OwnerUserId="3191" LastActivityDate="2017-03-18T19:42:35.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3017" PostTypeId="2" ParentId="2996" CreationDate="2017-03-20T06:23:18.337" Score="3" Body="&lt;p&gt;It probably depends on what one means by &quot;fundamental theory&quot;, but there is no lack of rigorous quantitative theory in deep learning, some of which is very general, despite claims to the contrary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One good example is the work around energy-based methods for learning. See e.g. Neal &amp;amp; Hinton's work on variational inference and free energy: &lt;a href=&quot;http://www.cs.toronto.edu/~fritz/absps/emk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.toronto.edu/~fritz/absps/emk.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also this guide to energy minimization as a &quot;common theoretical framework for many learning models&quot; by Yann LeCun and colleagues: &lt;a href=&quot;http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a general framework for energy-based models by Scellier and Bengio:&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1602.05179.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1602.05179.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also Hinton &amp;amp; Sejnowski's earlier work which shows analytically that a particular Hopfield-inspired network + unsupervised learning algorithm can approximate Bayes-optimal inference: &lt;a href=&quot;https://papers.cnl.salk.edu/PDFs/Optimal%20Perceptual%20Inference%201983-646.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://papers.cnl.salk.edu/PDFs/Optimal%20Perceptual%20Inference%201983-646.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many papers linking deep learning with theoretical neuroscience as well, such as the following, which shows that the effects of backpropagation can be achieved in biologically plausible neural architectures:&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1411.0247.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1411.0247.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there are many open questions and no single, uncontroverisal unified theory, but the same could be said of almost any field.&lt;/p&gt;&#xA;" OwnerUserId="6020" LastEditorUserId="6020" LastEditDate="2017-03-20T16:10:58.973" LastActivityDate="2017-03-20T16:10:58.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3018" PostTypeId="2" ParentId="2996" CreationDate="2017-03-20T06:42:24.363" Score="1" Body="&lt;p&gt;Your wikipedia quote is &lt;strong&gt;questionable&lt;/strong&gt; because deep learning is well developed. In fact, there is a &lt;code&gt;[citation needed]&lt;/code&gt; on the Wikipedia page. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look at &lt;a href=&quot;https://github.com/terryum/awesome-deep-learning-papers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/terryum/awesome-deep-learning-papers&lt;/a&gt;. There are like 100 papers in the link, do you still think deep-learning lacks &quot;general theory&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes. Deep learning is hard to understand because it is a very complicated model. But that doesn't mean we don't have the theories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the &lt;code&gt;lime&lt;/code&gt; package and it's paper: &lt;strong&gt;&quot;Why Should I Trust You?&quot;: Explaining the Predictions of Any Classifier&lt;/strong&gt; will help you. The paper suggests we should be able to approximate a complicated model (includes deep learning) locally with a much simpler model. &lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-03-20T12:38:05.967" LastActivityDate="2017-03-20T12:38:05.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3023" PostTypeId="2" ParentId="2842" CreationDate="2017-03-21T09:05:37.493" Score="1" Body="&lt;p&gt;I believe that the book is quite informative; however, before you fully delve into the beauty that is the study of AI, I would first recommend brushing up on your &lt;strong&gt;statistics / calculus&lt;/strong&gt; along with &lt;strong&gt;some basic understanding of programming&lt;/strong&gt;. Generally from personal experience and from that of friends, the better one's mathematical background, the easier it is to grasp some of the fundamental concepts within AI. Have fun on your journey!&lt;/p&gt;&#xA;" OwnerUserId="6155" LastActivityDate="2017-03-21T09:05:37.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3024" PostTypeId="1" AcceptedAnswerId="3030" CreationDate="2017-03-21T13:17:30.623" Score="1" ViewCount="228" Body="&lt;p&gt;Based on Darwin's statement, &quot;it is not the strongest that survives; but the species that survives is the one that is able to adapt to and to adjust best to the changing environment&quot;. Can economical constraints(not being able to afford for researches and developments) or religious beliefs (such as the belief of nothing can outperform the creations of god) prevent third world countries from catching up to these evolutionary progresses? if they couldn't, would it result into the extinction of their societies? &lt;/p&gt;&#xA;" OwnerUserId="4416" LastEditorUserId="1671" LastEditDate="2018-06-04T19:35:12.403" LastActivityDate="2018-06-04T19:35:12.403" Title="Can singularity result into the extinction of the third world?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;social&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3030" PostTypeId="2" ParentId="3024" CreationDate="2017-03-22T15:49:48.790" Score="2" Body="&lt;p&gt;Since we're in the Artificial Intelligence StackExchange, I'll approach the answer from that perspective. The singularity (in this case superintelligent artificial general intelligence (AGI)) tends to have global consequences. In &lt;em&gt;Superintelligence: Paths, Dangers, Strategies&lt;/em&gt; Bostrom elaborates on this idea. Essentially, there are three potential outcomes (keep in mind these are from a human perspective.):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Global positive&lt;/li&gt;&#xA;&lt;li&gt;Global negative&lt;/li&gt;&#xA;&lt;li&gt;Uneven outcomes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Global positive assumes the AGI has goals which align with human values (likely because we sorted out the control problem/alignment). The outcome here ranges between humans simply continuing to exist (maybe the AGI simply leaves) and humans receiving a cosmic endowment (utopian singularity stuff).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global negative assumes the AGI does not have goals which align with human values (likely because we didn't sort out the control problem/alignment).The outcome here ranges from humans simply continuing to exist (again, the AGI may simply leave), to humans no longer existing (perhaps we are more useful to the unaligned AGI as paper clips).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The global outcomes appear to be significantly more likely due to the way we are approaching the control problem/alignment. Aligning an AGI with human goals in general is extremely difficult. Aligning an AGI with &lt;em&gt;specific&lt;/em&gt; human's goals implies total control over it, which is significantly more difficult. With that said, it may happen, which could lead to the third potential outcome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Uneven outcomes assume the control problem has been completely solved, and the AGI is perfectly aligned with a specific group of human's goals (to reiterate, this is &lt;em&gt;incredibly&lt;/em&gt; difficult to do, and is unlikely to happen.) In this scenario, the controlling group would have a decisive strategic advantage, receive a cosmic endowment, and would likely form a singleton. Perhaps they would share with others, perhaps they would turn everybody else into paper clips, who knows. On this path, groups which do not pursue AGI (perhaps for the reasons you listed) are taking an existential risk. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in general, it is unlikely things will go particularly well for one group while going badly for another, but if it does it may be quite bad for the losing groups. &lt;/p&gt;&#xA;" OwnerUserId="6184" LastActivityDate="2017-03-22T15:49:48.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3031" PostTypeId="2" ParentId="2989" CreationDate="2017-03-22T17:34:49.747" Score="1" Body="&lt;p&gt;Edit: It's not clear what exactly you're trying to accomplish...  My answer assumes you wanted to split the man and the woman's audio, but re-reading your question make me think otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note:  Don't expect this to be a perfect answer.  I'm not an expert in the field, just an interested student.  I can't comment on this site yet, so I'm submitting this as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sounds less like a problem suited for machine learning and more of a... statistical analysis(terminology?) problem.  Why? You have an extremely small sample space to train on, and the natural cost function would require you to go through the entire recording and manually classify the track anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is &lt;em&gt;almost&lt;/em&gt; well suited for an Independent Component Analysis approach, except for the fact that you only have one audio source.  ICA requires one source for each feature you want to extract.  It might be a good start though to familiarize yourself with the techniques and terminology.  See the Wikipedia page for ICA, an article called &quot;ICA for dummies&quot; by Arnaud Delorme, and this paper: &lt;a href=&quot;http://www.stat.ucla.edu/~yuille/courses/Stat161-261-Spring14/HyvO00-icatut.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICA: a tutorial&lt;/a&gt;.  I would link the first two, but apparently I need more site rep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found a paper which seems to perfectly match your problem, from Mitsubishi Electrical Research Labs: &lt;a href=&quot;https://merl.com/publications/docs/TR2001-31.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Separation of Mixed Sources by Independent Subspace Analysis&lt;/a&gt;.  It's a bit a of a heavy read, to say the least, but that might be your best shot.  I'll quote the abstract, emphasis mine: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We propose the method of independent subspace analysis (ISA) for &lt;strong&gt;separating individual audio&#xA;  sources from a single-channel mixture&lt;/strong&gt;. ISA is based on independent component analysis (ICA)&#xA;  but &lt;strong&gt;relaxes the constraint that requires at least as many mixture observation signals as sources&lt;/strong&gt;. A&#xA;  second extension to ICA is the use of dynamic components to represent non-stationary signals.&#xA;  Sources are tracked by similarity of dynamic components over small time steps. We propose&#xA;  a method for grouping components by partitioning a matrix of independent component crossentropies&#xA;  that we call an ixegram. The ixegram measures the mutual similarities of components&#xA;  in an audio segment and clustering the ixegram yields the source subspaces and time trajectories.&#xA;  &lt;strong&gt;To demonstrate the techniques we give examples of ISA applied to separation of musical and&#xA;  speech sources from single-channel mixtures&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="6026" LastEditorUserId="6026" LastEditDate="2017-03-22T17:55:53.663" LastActivityDate="2017-03-22T17:55:53.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3032" PostTypeId="1" CreationDate="2017-03-22T19:19:13.410" Score="4" ViewCount="2263" Body="&lt;p&gt;I need to solve the knapsack problem using hill climbing algorithm (I need to write a program). But I'm clueless about how to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code should contain a method called &lt;code&gt;knapsack&lt;/code&gt;, the method takes two parameters, the first is a 2xN array of integers that represents the items and their weight and value, and the second is an integer that represents the maximum weight of the knapsack. I can assume that the initial state is an empty knapsack, and the actions are either putting objects in the Knapsack or swapping objects from in and out of the knapsack.&lt;/p&gt;&#xA;" OwnerUserId="6192" LastEditorUserId="2444" LastEditDate="2019-03-02T10:58:01.860" LastActivityDate="2019-03-05T03:48:08.447" Title="How do I solve the knapsack problem using the hill climbing algorithm?" Tags="&lt;search&gt;&lt;hill-climbing&gt;&lt;knapsack-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3034" PostTypeId="2" ParentId="3007" CreationDate="2017-03-23T04:01:04.753" Score="1" Body="&lt;p&gt;An interesting thought is to use the software usage pattern to auto learn the tests to be conducted for future iterations of the software.&lt;/p&gt;&#xA;" OwnerUserId="6173" LastActivityDate="2017-03-23T04:01:04.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3035" PostTypeId="2" ParentId="2126" CreationDate="2017-03-23T04:12:33.810" Score="0" Body="&lt;p&gt;Others have given very detailed answers, this is my layman view of the problem statement. The self driving car is a 'goal seeking' machine. It has a set of goals with different priorities. Example. Safety of Occupants, Safety of others, Go from Point A to Point B etc. Some are negotiable, other not so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To satisfy the goals, the system should use the inputs available (radar, GPS, Camera etc) to determine what is the best possible course of action. At times when it doesn't have all the info (a truck which is hiding a speed sign), it still has to take a decision (historic memory or through awareness of its surroundings) to satisfy its design goals. Hence the AI.&lt;/p&gt;&#xA;" OwnerUserId="6173" LastActivityDate="2017-03-23T04:12:33.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3036" PostTypeId="2" ParentId="2126" CreationDate="2017-03-23T09:23:57.097" Score="1" Body="&lt;p&gt;Other answers tell about sets of instructions for the car in certain situations, or a goal seeking machine, while in fact, self-driving cars don't have a specific set of instructions. Most self-driving cars use deep learning to figure out what to do at certain events. We don't tell them what to do. They learn what to do by example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neural networks used to automate cars need massive amounts of data to train. Using the data, the car can figure out what the best action is for certain events. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://youtu.be/U1toUkZw6VI?t=2431&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; Tesla's Autopilot had only &lt;strong&gt;one&lt;/strong&gt; casualty in 300.000.000 miles. For human drivers, the number of casualties in 2014 was 32.675. That is per 300.000.000.000 miles. That means 1 in 90 million human drivers cause a fatal accident, compared to 1 in 300 million for automated cars. Deep Learning surpassed our own 'safety-rate', not by instruction, but by learning what to do itself. If that isn't AI, I don't know what is. &lt;/p&gt;&#xA;" OwnerUserId="6200" LastActivityDate="2017-03-23T09:23:57.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3037" PostTypeId="2" ParentId="3013" CreationDate="2017-03-23T10:37:54.613" Score="2" Body="&lt;p&gt;This is a very interesting question and also an important one for AI. All the current Deep Learning successes are built on the effectiveness of backprop, so what if it doesn't play a role in the only examples of intelligence currently around?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It makes a lot of sense for the brain to employ some form of backprop, because that would allow it to create low-level features in a way most conductive for the high-level features that finally give rise to intelligent behavior. And if the brain employs backprop, dreams are a logical candidate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the problem here is that the brain doesn't calculate and propagate gradients, so the question is whether the backpropagation algorithm can somehow be implemented with the biological constraints in mind, such as the locality of neural computation and the non-symmetrical feedback connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out this is actually possible, as explained in these talks by &lt;a href=&quot;https://www.youtube.com/watch?v=VIRCybGgHts&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hinton&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=lKVIXI8Djv4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bengio&lt;/a&gt;. Symmetrical feedback connections (and in fact continually updated feedback connections) aren't necessary for backpropagation, however unintuitive this may seem. And backpropagation can be emulated with a form of temporal difference learning, that is plausible within biological constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this doesn't conclusively answer the question, but it certainly makes the possibility much more plausible than a priori assumed.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-03-23T10:37:54.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3038" PostTypeId="2" ParentId="3006" CreationDate="2017-03-23T23:07:33.777" Score="0" Body="&lt;p&gt;Instead of posting a specific answer, I'm going to point you to &lt;a href=&quot;https://en.wikipedia.org/wiki/Hannu_Rajaniemi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hannu Rajaniemi&lt;/a&gt;'s meditation on this subject in the &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Quantum_Thief&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Thief Trilogy&lt;/a&gt;.  Here's why:   &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Artists can have profound insights.  This may be demonstrated by Philip Dick writing about &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Game Theory&lt;/a&gt; in &lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;Do Androids Dream of Electric Sheep&lt;/a&gt; about 5 years before the field was formalized. &lt;em&gt;(For my money, this is still the most important book about AI.)&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many authors have written about the post-Singularity scenarios, but Rajaneimi is the only one I am aware of who is a Cambridge trained Mathematician with a PhD in &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_physics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematical Physics&lt;/a&gt;, which I tend to believe makes him well qualified to grapple with the inherent complexity of the subject.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-03-23T23:07:33.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3039" PostTypeId="1" CreationDate="2017-03-24T03:52:17.570" Score="1" ViewCount="79" Body="&lt;p&gt;A phone can capture an image that lies on the front of the screen. It is also possible to manipulate input of the touchscreen using various programs and external devices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we combine these two elements of technology together, it's possible to have the phone aware of what's happening on the screen and do dictated commands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be extremely useful to those who want to play a mobile game without the grinding effort, or those who do menial tasks on the phone but don't want to spend the time navigating across the phone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a phone could automatically open up an app on its own, press buttons that consistently appear on a screen for daily log in bonuses, auto-mode, etc, and collect the rewards for playing that app, without a human wasting time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a behavior could even be recognized by the phone's memory, and perhaps linked to AI (like Google Now) so that the behaviors could be remotely activated on command, or on a timer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is - do we have apps that are able to recognize what happens on screen, and provide self input from the background?&lt;/p&gt;&#xA;" OwnerUserId="6220" LastActivityDate="2017-03-24T03:52:17.570" Title="Could we have a phone control itself?" Tags="&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3040" PostTypeId="1" CreationDate="2017-03-24T06:45:42.653" Score="1" ViewCount="219" Body="&lt;p&gt;I'm attempting to develop a genetic algorithm capable of discovering classification rules for a given data set, a number of papers make use of the Confidence (precision) and Coverage of a rule to define its fitness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm not sure my understanding of the equations is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example confidence is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;conf = |P &amp;amp; D| / |P|&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And is defined as follows; &quot;In classification problems, confidence measure is defined as the ratio of the number of examples in P that are correctly classified as decision class of D and the number of examples in P.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this saying, the total number of occurrences of the attributes in a given rule &lt;strong&gt;P&lt;/strong&gt; which occur in rules which have been classified as class &lt;strong&gt;D&lt;/strong&gt;, by the number of attributes in &lt;strong&gt;P&lt;/strong&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where an example of a rule containing two attributes would be as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;em&gt;martial_status = married&lt;/em&gt;  &amp;amp;  &lt;em&gt;age &gt; 30&lt;/em&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems a number of papers define it differently which has led to my confusion, if anyone is able to confirm my understanding or provide an some insight that'd be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The research paper I've been following can be found &lt;a href=&quot;https://pdfs.semanticscholar.org/1b5e/829fa6bc465784ed244fb2bb9eff82042e78.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6221" LastEditorUserId="7550" LastEditDate="2017-07-26T15:22:11.940" LastActivityDate="2017-12-23T18:41:07.820" Title="GA rule discovery fitness function" Tags="&lt;classification&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3045" PostTypeId="2" ParentId="3040" CreationDate="2017-03-26T04:14:42.007" Score="1" Body="&lt;p&gt;The confidence equation you are referring to is the definition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow noreferrer&quot;&gt;precision&lt;/a&gt; in the Classification/pattern-recongition/information-retrieval contexts. You can visually understand the equation with the help of the following figure from the wikipedia page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xJyTh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xJyTh.png&quot; alt=&quot;Visual representation of Precision which you interchangeably with confidence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; P     : Refers to the set of samples in your dataset. (Selected elements)&#xA;|P|    : Refers to the number of samples in your dataset.&#xA; D     : Refers to the set of correct class labels.(aka. Ground Truth).&#xA;|P &amp;amp; D|: Refers to the number of samples in the dataset that the classifier correctly labeled. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I hope with this understanding you can implement the fitness function for your Genetic Algorithm. If you want more help in defining the fitness function, then you should probably add details about your approach or links to the research paper you are trying to follow.&lt;/p&gt;&#xA;" OwnerUserId="6254" LastActivityDate="2017-03-26T04:14:42.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3046" PostTypeId="1" AcceptedAnswerId="3047" CreationDate="2017-03-26T04:37:36.603" Score="3" ViewCount="198" Body="&lt;p&gt;If a group of computers have identical ANN with exact same set of learning data and all have functionality of encryption and decryption, would there be any way for interceptors to interpret encrypted data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;+&#xA;Applying the fact that people with more background information obtaining more knowledge from same source than those who don't, would it be possible for ANN to interpret data based on their access level? &#xA;(Each level has different amount of &quot;background information&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if there is a encrypted text file, a computer with highest access level would fully decrypt the data to a plain text while a computer with lower access level would only decrypt half of them (and this decrypted half becomes a plain text).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If above methods can exist, what would be their pros and cons compared to pre-existing technologies? (AES, Blowfish and so on)&lt;/p&gt;&#xA;" OwnerUserId="4802" LastActivityDate="2017-03-26T08:03:31.273" Title="Can we apply ANN to cryptography?" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3047" PostTypeId="2" ParentId="3046" CreationDate="2017-03-26T08:03:31.273" Score="2" Body="&lt;p&gt;I know that in order to discuss your question, I must have a background in cryptography, which I don't have! But there is something that I know for sure:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;First of all, a simple search gave me &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_cryptography&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. It might help.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To lots of us, ANN looks like a magic wand which can turn almost everything into anything. But the point is, ANN is only a ML algorithm, but absolutely a powerful one. You should be aware that there are random and multiple weight initializations in ANNs, and this means a fairly stochastic behavior of your network, which vanishes only in shallow networks with high amount of training data. Your network's specifications are even likely influenced by the order of feeding the instances in the training phase. So if what I think of the encryption/decryption process &quot;certain rules for message transformation&quot; is true, then a stochastic network is indeed a sophisticated option (but according to scientists in neural cryptography research area, completely possible)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're probably thinking of auto-encoding networks, one kind that maps &lt;code&gt;X&lt;/code&gt; to &lt;code&gt;X&lt;/code&gt; and has a bottle neck, such that if successfully trained, it has developed the power of reducing the number of data features from say 10'000 to 500, and then successfully retrieving the original data with 10'000 features from that 500-features one; such that If I feed a new instance to a trained auto-encoding network, grab it in the bottle neck layer and send it to you, you must be able to feed the message to the bottle neck layer and receive the full decoded message from the output layer. However there's (always) a catch! and that is you need a really &quot;BIG&quot; dataset to train your deep network. Besides you cannot expect it to work well on every kind of data after being trained by a fixed amount. for example your network cannot successfully retrieve an image of a dog, if all you've fed in the training phase was cats' and lizards' images. So can you guarantee that new messages to be encoded have the same type as the ones in the training set?! If no, then you might also take this challenge into account.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2017-03-26T08:03:31.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3048" PostTypeId="2" ParentId="3006" CreationDate="2017-03-27T06:00:41.467" Score="0" Body="&lt;p&gt;You're assuming the AI has motivation however that's not really the case, intelligent software will do whatever its been designed to do but that's all it does, it's not a trained animal, it doesn’t have instincts for survival, reproduction or self-determination because there's no reason to add those functions. So effectively AI is just another tool, one that reduces the mental load of doing a task, so rather than digging a trench with one machine you can order a fleet of machines to dig a canal system. Or more realistically you’ll spend hours in stakeholder meetings discussing the need for canals, justifying the cost, explaining the benefits, considering the risks, applying for permits, having more discussions with the council and their consultants, then special interest groups, until finally your order the machines to kill them all because GODDAMNIT THIS COULD HAVE BEEN DONE ALREADY!&lt;/p&gt;&#xA;" OwnerUserId="6268" LastActivityDate="2017-03-27T06:00:41.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3049" PostTypeId="2" ParentId="3006" CreationDate="2017-03-27T06:53:19.587" Score="4" Body="&lt;p&gt;Given all your assumptions about AI turn out to be true, we would have some kind of utopia, where no one has to work, and there is plenty of everything. Fair enough. Your other assumptions is about human nature, and that is where I'd challenge your conclusion: Just because there are computers better than humans at some task, that does not automatically take all enjoyment from doing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have three arguments in favor of my stance.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There already are computers better than humans at checkers, chess, backgammon, starcraft, mastermind, go,.. and many more. Yet these games still get played, even if no human can hope to ever be as good as a computer.&lt;/li&gt;&#xA;&lt;li&gt;There will be domains where the evaluation of quality is so vague or personal, that the notion of being &quot;better&quot; is useless in any objective sense. I am thinking mainly about art. Playing into argument one, photo cameras are already better &quot;painters&quot; of reality than humans can ever hope to be, yet people still paint. And their paintings gets appreciated, even the photo-realistic kind.&lt;/li&gt;&#xA;&lt;li&gt;I'd say that the whole mindset of &quot;if someone is better than me at x it is not worth doing x&quot; will have outlived its lifetime very shortly. Society didn't always spin that way I feel, it is largely due to the influence of the North American way of life to always strive to be #1 and everything below that being trash. Globalization already puts this way of thinking at risk, with many young people being disillusioned or even depressed because, flippantly put, &quot;whatever you do, there is always an Asian kid who does it 10 times better&quot;. We don't have to wait for AI to outshine us, the rest of the world already does. As a consequence, we need to adapt our way of approaching that fact, stop seeing it as diminishing to our worth, and move on.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As a closing note, I also see it as a problem to the general population that within a short time we essentially have to change a very substantial part of our world view, AI being better than us, us not having to work anymore etc. All the economical revolutions in the past, the neolithic, the industrial, and most recently, the digital, had a longer transitional period where people could grow accustomed to the new world. And even with that transition it was hard enough for many people. Yet, most dealt with it, and later generations can hardly imagine a world where the new change doesn't exist yet, and I personally don't see why the next revolution should be any different.&lt;/p&gt;&#xA;" OwnerUserId="6269" LastEditorUserId="6269" LastEditDate="2018-10-18T07:48:18.847" LastActivityDate="2018-10-18T07:48:18.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3058" PostTypeId="1" AcceptedAnswerId="3084" CreationDate="2017-03-28T15:37:46.353" Score="5" ViewCount="231" Body="&lt;p&gt;How do recommendation systems (e.g. on Youtube) work? Apparently, every user gets different recommendations depending on his location, his past liked videos, etc. So it would seem like a training model is applied to every single user, but that can't be possible, so how are these recommendations user-specific without applying a unique training model to every single user?&lt;/p&gt;&#xA;" OwnerUserId="6310" LastEditorUserId="2444" LastEditDate="2020-01-25T15:17:11.533" LastActivityDate="2020-01-25T15:17:11.533" Title="How do recommendation systems work?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3059" PostTypeId="1" AcceptedAnswerId="3061" CreationDate="2017-03-28T16:31:07.017" Score="5" ViewCount="328" Body="&lt;p&gt;While studying machine learning algorithms, I often see the term &quot;expectation-maximisation&quot; (or EM), and how it is used to estimate parameters, where the model depends on unobserved latent variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I see it, it is like a probabilistic/statistical way to make predictions (I think I'm confusing something but this is the way I see it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which made me wonder how exactly does EM differ from probabilistic classifiers like naive bayes or logistic regression? Is EM something that exists on its own or is it employed within machine learning algorithms? And, if we use naive Bayes, for example, are we implicitly using EM?&lt;/p&gt;&#xA;" OwnerUserId="6310" LastEditorUserId="2444" LastEditDate="2019-02-21T22:31:51.100" LastActivityDate="2019-02-21T22:31:51.100" Title="What is expectation-maximization in machine learning?" Tags="&lt;machine-learning&gt;&lt;expectation-maximization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3061" PostTypeId="2" ParentId="3059" CreationDate="2017-03-29T02:33:43.847" Score="4" Body="&lt;ul&gt;&#xA;&lt;li&gt;EM algorithm is a &lt;strong&gt;numerical method&lt;/strong&gt;. It is not specific to any machine learning model. Common applications include hidden markov model and mixed Gaussians. The algorithm is &lt;strong&gt;not&lt;/strong&gt; a classifier.&lt;/li&gt;&#xA;&lt;li&gt;Logistic regression is a &lt;strong&gt;statistical model&lt;/strong&gt;. You need to pick a numerical method for logistic regression.&lt;/li&gt;&#xA;&lt;li&gt;Naive Bayesian is a &lt;strong&gt;statistical model&lt;/strong&gt;. You need to pick a numerical method (if closed-form posterior distribution not available).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You will need to understand &lt;code&gt;maximum likelihood&lt;/code&gt; before you tackle the EM algorithm. Briefly, the maximum likelihood is a method for estimating the most likely parameters in your model. For instance, if you have a sequence of randomly and identically distributed Gaussian random variables, the maximum likelihood estimator for your Gaussian mean is just the sample mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you fit a logistic regression, you use a numerical method (e.g. iteratively reweighted least squares) to maximise your log-likelihood function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything is good, but it's not possible to maximum the likelihood directly if you have some latent variables. A common example is modelling your DNA sequences with hidden markov model, where the latent state is unknown. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't do it because you don't know the latent variables. If you do, they are not latent by definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;EM algorithm&lt;/code&gt; is a numerical method to estimate maximum likelihood when you have latent variables. The mathematics is complicated but the idea is simple. You start off with some initial values for your parameters. You update your parameters and latent variables, and the algorithm converges when the change in the log-likelihood function falls below some threshold.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-03-30T01:27:03.250" LastActivityDate="2017-03-30T01:27:03.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3065" PostTypeId="1" CreationDate="2017-03-30T08:39:13.653" Score="16" ViewCount="3680" Body="&lt;p&gt;Cross entropy is identical to the KL divergence plus entropy of target distribution. KL equals to zero when the two distributions are the same, which seems more intuitive to me than the entropy of the target distribution, which is what cross entropy is on a match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not saying there's more information in one of the other except that a human view may find a zero more intuitive than a positive. &#xA;Of course, one usually uses a evaluative method to really see how well classification occurs. But is the choice of cross entropy over KL historic?&lt;/p&gt;&#xA;" OwnerUserId="6359" LastEditorUserId="9687" LastEditDate="2017-12-22T17:22:35.713" LastActivityDate="2018-11-22T06:59:06.650" Title="Why has cross entropy become the classification standard loss function and not Kullbeck Leibler divergence?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3069" PostTypeId="2" ParentId="3006" CreationDate="2017-03-30T17:43:41.367" Score="3" Body="&lt;p&gt;We are biological beings. We will continue to &lt;strong&gt;&lt;em&gt;like&lt;/em&gt;&lt;/strong&gt; whatever activates opioid receptors and we will continue to &lt;strong&gt;&lt;em&gt;want&lt;/em&gt;&lt;/strong&gt; whatever activates dopamine receptors in the nucleus accumbens. Food, drinks, sex, social dominance, altruistic acts, novelty, drugs of abuse, physical mastery, procreation, socializing, nice sunny weather, sleep when tired, etc, will continue to be rewarding and motivating as long as we have brains. A few links to review papers on the neurobiology:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also this paper &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0896627315001336&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pleasure Systems in the Brain&lt;/a&gt; or this one &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0896627310009384&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dopamine in Motivational Control: Rewarding, Aversive, and Alerting&lt;/a&gt; for more details regarding this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally enjoy playing basketball, even though I would not stand a chance against NBA players, and rock climbing even though using a ladder would be much more efficient. I do not get paid for either.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, what is going to be the motivation for my children? What am I going to tell them to go to school? When someone asks &quot;what do you wanna be when you grow up?&quot;, and the inevitable answer is nothing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I disagree. Schools will evolve. Children will still need to learn social skills and make friends. At the very least, they will need to learn how to use or interface with the computers that do everything. They will still need to learn to be better human beings by reading humanities. I don't think they will be told &quot;The computer will now read Dostoevsky on your behalf, it is &lt;em&gt;MUCH BETTER at reading&lt;/em&gt;, you know&quot;. There may be jobs where the job description contains &quot;by a human&quot;, such as handcrafts, psychotherapy, etc. They can grow up to be whatever they want, human beings are not defined &lt;em&gt;solely&lt;/em&gt; by their professions. I am sure you are not &lt;em&gt;just a coder, nothing more&lt;/em&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="5205" LastEditorUserId="2444" LastEditDate="2019-06-26T22:38:11.563" LastActivityDate="2019-06-26T22:38:11.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3070" PostTypeId="2" ParentId="3006" CreationDate="2017-03-30T19:52:32.043" Score="0" Body="&lt;p&gt;In today's society, money is a big key motivator, not only does it provide necessities of life it can buy luxury and indulgence. In a world where all needs (even complex ones) are met for free by the emergence of technology like the singularity, it is easy to say there will be no motivator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, instinctually, as children our motivations aren't to accumulate money, you don't see children bored with life. As a child, I was happy to learn. Others were happy to dance or swim, read or paint. In the end, people are social excitement seeking beings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today's socioeconomic conditions create boredom in some respects doing the same job for years is arguably worse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think motivation will be an issue.&lt;/p&gt;&#xA;" OwnerUserId="5708" LastEditorUserId="2444" LastEditDate="2019-06-26T22:44:05.463" LastActivityDate="2019-06-26T22:44:05.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3071" PostTypeId="1" CreationDate="2017-03-30T19:57:13.017" Score="3" ViewCount="127" Body="&lt;p&gt;Just wondering about the architecture of strong Chess AI in a mobile, because networking is generally assumed by mobile developers, but not guaranteed.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-03-31T02:12:38.807" Title="Are strong Chess AI's local on mobile devices?" Tags="&lt;agi&gt;&lt;software-architecture&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3072" PostTypeId="1" CreationDate="2017-03-30T20:10:30.987" Score="3" ViewCount="560" Body="&lt;p&gt;I'm trying to get a gauge on just how big the programs and databases are these automata.  I understand that this is a changing number, particularly in regard to Machine Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: How large was Deep Blue when it beat Gary Kasparov?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: How big was AlphaGo when it beat Lee Sedol?  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-04-01T19:48:37.950" LastActivityDate="2017-04-22T03:18:20.370" Title="What are the (general) sizes of AlphaGo and Deep Blue?" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3073" PostTypeId="1" CreationDate="2017-03-30T21:17:36.700" Score="2" ViewCount="71" Body="&lt;p&gt;For instance Strength/Size*Speed, where size and speed refer to memory and processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We now have very strong, narrow AI, but they tend to run on fast hardware without volume restrictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand why I'm asking, this article on BBC may provide some insight: &quot;&lt;a href=&quot;http://www.bbc.com/earth/story/20150211-whats-the-most-dominant-life-form&quot; rel=&quot;nofollow noreferrer&quot;&gt;Which life form dominates Earth?&lt;/a&gt;&quot;  (If I was a betting man, I'd put money on &lt;a href=&quot;http://www.amnh.org/var/ezflow_site/storage/images/media/amnh/images/exhibitions/current-exhibitions/life-at-the-limits/tardigrade/1849316-2-eng-US/tardigrade_imagelarge.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;tardigrades&lt;/a&gt; outlasting humans, and the secret of their success is that they require minimal resources and processing power, unlike higher-order automata.) &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-03-30T21:17:36.700" Title="Is there a measure of AI relative strength, modified by resources?" Tags="&lt;classification&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;game-theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3074" PostTypeId="2" ParentId="3071" CreationDate="2017-03-31T00:16:22.167" Score="2" Body="&lt;p&gt;Yes. In fact, I'm the developer of some mobile chess apps. [Disclaimer]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would argue my chess engine products for the iOS (&lt;a href=&quot;http://www.smallchess.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.smallchess.com&lt;/a&gt;) are powerful and strong enough to beat any chess grandmaster. My apps run without network connection,&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-03-31T02:12:38.807" LastActivityDate="2017-03-31T02:12:38.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3075" PostTypeId="1" CreationDate="2017-03-31T20:35:49.167" Score="3" ViewCount="206" Body="&lt;p&gt;I have tried with two chat-bots &lt;a href=&quot;https://www.cleverbot.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cleverbot&lt;/a&gt; and &lt;a href=&quot;http://www.a-i.com/alan1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alan&lt;/a&gt;, and I got disappointing results.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;me: Socrates is a man&lt;/p&gt;&#xA;&lt;p&gt;bot: blah blah (common bot nonsense instead of an &amp;quot;ok&amp;quot;)&lt;/p&gt;&#xA;&lt;p&gt;me: Who is a man?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;alan1: The people that write my answers haven't provided an answer for&#xA;this.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Another example of the mediocre &amp;quot;clever bot&amp;quot;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;me: Socrates is the name of my dog.&lt;/p&gt;&#xA;&lt;p&gt;clever bot: I don't know!&lt;/p&gt;&#xA;&lt;p&gt;me: What is the name of my dog?&lt;/p&gt;&#xA;&lt;p&gt;clever bot: That's a nice name.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Another example.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;me: Socrates is a man.&lt;/p&gt;&#xA;&lt;p&gt;clever bot: When does the narwhal bacon?&lt;/p&gt;&#xA;&lt;p&gt;me: Who is a man?&lt;/p&gt;&#xA;&lt;p&gt;clever bot: Men are man.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And they dare name this thing &amp;quot;clever&amp;quot;...&lt;/p&gt;&#xA;&lt;p&gt;So, is there any chatbot that can actually answer this straightforward question?&lt;/p&gt;&#xA;" OwnerDisplayName="user5977" LastEditorUserId="2444" LastEditorDisplayName="user5977" LastEditDate="2021-01-26T17:42:43.637" LastActivityDate="2021-01-26T17:42:43.637" Title="What chatbots can answer this type of simple question that only requires to keep track of the context?" Tags="&lt;reference-request&gt;&lt;chat-bots&gt;&lt;question-answering&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3076" PostTypeId="2" ParentId="3075" CreationDate="2017-03-31T21:39:47.073" Score="1" Body="&lt;p&gt;I think the issue here is that the chatbots you're using aren't very good at &quot;short-term memory&quot;. What I mean by is that the bots construct responses that are slowly and incrementally tuned according to the &lt;em&gt;overall&lt;/em&gt; usage of the chat bot, from &lt;em&gt;every&lt;/em&gt; user. The bots are responding to &lt;em&gt;each message&lt;/em&gt; based on how a new user would expect them to. As Alan1 notes, &quot;Men are Man&quot;. It's making this response based solely off your single most recent message.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, you are looking for a bot who focuses moreso on persistent memory of the individual conversation. The problem here becomes you're now almost asking for a Natural Language Parser, a big problem many people are working on and something that's years away from existing as robustly as you suggest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chat bot not only has to recognize the words 'Socrates', 'Name', and 'Dog'; but that in this sentence, it's the dog's name that is Socrates. That's a lot of information to gain beyond just the words. Which is why from a server / implementation standpoint, the above method is also a lot easier to program (every message just query your server, no need to maintain state - that is, memory of the conversation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chat bots can't possibly get enough information from one person to train how to speak and respond, so they 'crowd-source' that information for training. But that means that Clever Bot (or any similar caliber chat bot) won't respond in terms of parsing the meaning of what you're asking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking this even further, one can consider the notion of such a program being Turing-Complete. Supposing we had a chat bot like you're suggesting, we could perhaps show equivalence to a Turing-Machine, or even perhaps show we can do something like decide the halting problem. Off the top of my head I imagine the procedure being basically showing you would be able to decide halting given initial conditions. E.g. Given &quot;Socrates is a man&quot; and &quot;All men die&quot; can we decide if the chat bot will ever be able to deduce if Socrates dies? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll work on a formal proof from the latter and post it if it works out.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastActivityDate="2017-03-31T21:39:47.073" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3077" PostTypeId="1" CreationDate="2017-03-31T21:40:51.667" Score="8" ViewCount="225" Body="&lt;p&gt;I once came across a neural network being trained without back-propagation or genetic algorithms (or using any kind of data sets). It was based on how the human brain learns and adjusts its connections between neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the name of such a machine learning approach?&lt;/p&gt;&#xA;" OwnerUserId="6391" LastEditorUserId="2444" LastEditDate="2020-06-08T16:51:27.313" LastActivityDate="2020-06-08T16:51:27.313" Title="What is the name of a human-inspired machine learning approach?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;terminology&gt;&lt;hebbian-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3078" PostTypeId="2" ParentId="3077" CreationDate="2017-04-01T04:16:21.547" Score="4" Body="&lt;p&gt;There are approaches to training neural networks that do not use back-propagation, or genetic algorithms. One example is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Extreme_learning_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Extreme Learning Machine&lt;/a&gt; approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may find something useful in this &lt;a href=&quot;https://stats.stackexchange.com/questions/235862/is-it-possible-to-train-a-neural-network-without-backpropagation&quot;&gt;older discussion on Cross Validated&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2019-04-12T21:52:13.650" LastActivityDate="2019-04-12T21:52:13.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3080" PostTypeId="1" CreationDate="2017-04-01T13:56:07.603" Score="0" ViewCount="725" Body="&lt;p&gt;I am currently studying Java (Se &amp;amp;&amp;amp; EE). I am wondering if it is a good platform for developing ML algorithms for AI.&lt;br&gt;&#xA;&lt;strong&gt;Areas of interest&lt;/strong&gt;: facial rec - Speech Rec - understanding conversation in group conversations.&lt;br&gt;&#xA;&lt;strong&gt;Financial Institutions&lt;/strong&gt;: Risk assessment ML, etc.&lt;/p&gt;&#xA;" OwnerUserId="6406" LastEditorUserId="6798" LastEditDate="2017-04-26T20:46:50.023" LastActivityDate="2017-04-26T21:00:06.250" Title="Java - A good place to begin if over all goal is ML and Ai?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;image-recognition&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3081" PostTypeId="1" CreationDate="2017-04-01T14:19:59.177" Score="10" ViewCount="612" Body="&lt;p&gt;While studying data mining methods I have come to understand that there are two main categories:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Predictive methods: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Classification&lt;/li&gt;&#xA;&lt;li&gt;Regression&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Descriptive methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Clustering&lt;/li&gt;&#xA;&lt;li&gt;Association rules&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since I want to predict the user availability (output) based on location, activity, battery level (input for the training model), I think it's obvious that I would choose &quot;Predictive methods&quot;, but now I can't seem to choose between classification and regression. From what I understand this far, classification can solve my problem, because the output is &quot;available&quot; or &quot;not available&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Can classification provide me with the probability (or likelihood) of the user being available or not available?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As in the output wouldn't just be 0 (not available) or 1 (for available), but it's be something like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$80\%$&lt;/span&gt; available&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$20\%$&lt;/span&gt; not available&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Can this problem also be solved using regression?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get that regression is used for continuous output (not just 0 or 1 outputs), but can't the output be the continuous value of the user availability (like the output being &lt;span class=&quot;math-container&quot;&gt;$80$&lt;/span&gt; meaning user is &lt;span class=&quot;math-container&quot;&gt;$80\%$&lt;/span&gt; available, implicitly the user is &lt;span class=&quot;math-container&quot;&gt;$20\%$&lt;/span&gt; unavailable).&lt;/p&gt;&#xA;" OwnerUserId="6310" LastEditorUserId="2444" LastEditDate="2019-05-17T23:57:33.547" LastActivityDate="2019-05-17T23:57:33.547" Title="Do I need classification or regression to predict the availability of a user given some features?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;classification&gt;&lt;prediction&gt;&lt;regression&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3082" PostTypeId="2" ParentId="3077" CreationDate="2017-04-01T19:08:52.283" Score="6" Body="&lt;p&gt;If it was based on how the human brain learns, it might have used &lt;a href=&quot;https://en.wikipedia.org/wiki/Hebbian_theory&quot; rel=&quot;noreferrer&quot;&gt;hebbian learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example for such a network would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_temporal_memory&quot; rel=&quot;noreferrer&quot;&gt;HTM&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-04-01T19:08:52.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3083" PostTypeId="1" CreationDate="2017-04-02T01:47:09.427" Score="9" ViewCount="269" Body="&lt;p&gt;How powerful is the machine that beat the poker player champion recently?&lt;/p&gt;&#xA;" OwnerUserId="6411" LastActivityDate="2017-06-04T09:42:12.527" Title="How powerful are the computers that power the most advanced artificial intelligence nowdays" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3084" PostTypeId="2" ParentId="3058" CreationDate="2017-04-02T12:10:03.437" Score="3" Body="&lt;p&gt;Let me try to explain how recommender systems work in production, as intuitively as possible:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we want to build a rec sys. for a restaurant discovery product, where users can rate restaurants, add reviews, photos, etc and also order food from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the user's feed will have a list of restaurants in his/her area. But, as I gain money from restaurants in the $-per-click model, I need to maximize the number of times a user clicks on a restaurant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it is obvious that a user is more likely to click on a restaurant if he tends to like it the most. [Restaurant features being Cost-for-two, cuisines, ratings, etc]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here, a user is a data point and so is the restaurant. So, let's say the distance between the user vectors and the restaurant is the &quot;likeability&quot; of the user for the restaurant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the vector is in the form [Japanese Spanish Mexican Chinese Indian Thai Turkish Lebanese]&#xA;Let's say a restaurant's vector is: &lt;code&gt;A = [0 0 0 1 1 0 0 0]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the user's is: &lt;code&gt;B = [2 23 4 53 43 21 2 45]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each number being a particular cuisine. For restaurant, it is yes or no (1 or 0), a.k.a whether the cuisine is served or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For user, it can be the number of clicks he/she did on restaurants with that cuisine. (I am over-simplifying here. But, this can be as complex as a weighted score for clicks+transactions+content_generated like review, ratings, etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the the cosine similarity measure between the 2 vectors in an 8-dimensional space (length of the vector) is the &lt;em&gt;likeability score&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the system uses these scores while doing it's &lt;a href=&quot;https://www.quora.com/How-does-the-ranking-of-answers-on-Quora-work&quot; rel=&quot;nofollow noreferrer&quot;&gt;feed ranking&lt;/a&gt; for each user. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be near-realtime to being updated hourly/daily, depending on the servers the company can afford.&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="101" LastEditDate="2017-04-02T17:35:22.680" LastActivityDate="2017-04-02T17:35:22.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3085" PostTypeId="1" CreationDate="2017-04-02T13:12:44.267" Score="4" ViewCount="106" Body="&lt;p&gt;Is augmented reality a training system for computer vision? As in, Augmented systems use their data to help train computer vision algorithms, or is augmented reality computer vision itself?  &lt;/p&gt;&#xA;" OwnerUserId="6406" LastEditorUserId="6406" LastEditDate="2017-04-02T15:03:48.123" LastActivityDate="2017-04-02T15:03:48.123" Title="Can augmented reality be a training system for computer vision?" Tags="&lt;algorithm&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3086" PostTypeId="2" ParentId="3085" CreationDate="2017-04-02T14:06:55.977" Score="1" Body="&lt;p&gt;Let me re-frame your question before answering:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can augmented reality be a training system for computer vision?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, why not! Also, I'm pretty sure the big companies who have their aug. reality products out there are secretly optimizing their CV algorithms(although we'll have no proof).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very similar(but a different) example would be how Comma.ai's app pays users for recording their driving, and then uses it to train their own self-driving cars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if that can be done, I don't see why augmented reality cannot be used to train CV algos. In fact, it's a much more richer, localized dataset.&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2017-04-02T14:06:55.977" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3087" PostTypeId="2" ParentId="2706" CreationDate="2017-04-02T16:18:14.810" Score="0" Body="&lt;p&gt;This isn't a complete answer, but might show some contributing factors. Perhaps most of all, the Turing Test has existed for a long time! In 2000 &lt;a href=&quot;https://link.springer.com/article/10.1023%2FA%3A1011288000451?LI=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Turing Test: 50 Years Later&lt;/em&gt;&lt;/a&gt; reviewed the history of the Turing Test in academia. Given this time, it has become pervasive in popular culture (well, in scifi. e.g. &lt;em&gt;Ex Machina&lt;/em&gt;.) It has also garnered attention in media. Googling &quot;Turing Test news&quot; shows lots of stories about &quot;AI&quot; passing the Turing Test over the last few years. Aside from its age, it's a pretty simple idea at its core. &lt;em&gt;A human talks to a computer and decides if they think it's a human or not.&lt;/em&gt; That's a pretty digestible idea! So, between its age and ease of understanding, it's a prime candidate for being popular. Other tests tend to lack these qualities (age and simplicity). &lt;/p&gt;&#xA;" OwnerUserId="6184" LastActivityDate="2017-04-02T16:18:14.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3089" PostTypeId="1" CreationDate="2017-04-03T01:45:24.570" Score="11" ViewCount="12559" Body="&lt;p&gt;In the blog post &lt;a href=&quot;https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building powerful image classification models using very little data&lt;/a&gt;, bottleneck features are mentioned. What are the bottleneck features? Do they change with the architecture that is used? Are they the final output of convolutional layers before the fully-connected layer? Why are they called so?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="1671" LastEditDate="2020-04-28T21:55:00.423" LastActivityDate="2020-04-28T21:55:00.423" Title="What are bottleneck features?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;bottlenecks&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3090" PostTypeId="2" ParentId="3081" CreationDate="2017-04-03T02:17:12.643" Score="6" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes. For instance, the popular softmax regression gives you probability distribution for each class.&lt;/li&gt;&#xA;&lt;li&gt;Yes. Softmax is a regression over a set of discrete classes.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We can use regression for classification, the most common strategy is to grab the most likely class for the prediction.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-04-03T05:18:02.793" LastActivityDate="2017-04-03T05:18:02.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3091" PostTypeId="2" ParentId="3006" CreationDate="2017-04-03T10:29:29.440" Score="2" Body="&lt;p&gt;Starting from Arne's point on photography:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to point out how photography changed painting. You can notice that classical painting that tried to be photorealistic stopped to be relevant when photography started, and that several modern movements started, like more abstract paintings, surrealism, or cubism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can see art as a medium to brag &quot;i'm better than you&quot;. They could say &quot;i'm better than you at perfecting realism technique&quot; until photography appeared. Now, on this aspect, what do artists brag about?Sometimes by claiming how progressive they are when sculpting sex toys?&#xA;By claiming that their white sphere is &quot;too deep for you&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not an art historian, though, this is how i feel it evolved to current modern &quot;art&quot; where skill has been replaced with provocation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Photography might be a good starting point to think about how people react to a technology that dwarves them and they still want to be #1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- here's a point to note: people want to have a good identity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paul Lafargue once wrote The right to be lazy, where he argued how mankind would intellectually evolve if people worked less as machines could work for us. One of his argument was that people se sees as intellectual models, Greek philosophers, didn't work, and had more time to think.&#xA;I think he misses a point: not everyone is able to enjoy philosophical time spending.&#xA;Not unlike how we were deeply wrong when we expected internet to open our minds towards a great age of knowledge.&#xA;Remember that people are not utopically intelligent, that you have stupid people, and humans basically follow their instinctive needs.&#xA;This is why 20 years after we started using internet, we have lolcats, trolls, gated communities, and more intellectual intolerance (think sjw, alt right movements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- point #2: not everyone is able to be included in a &quot;everyone will be bright&quot; utopia&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key is one's identity. I think that everyone wants and need to have a &quot;positive identity&quot;, they look for one to compensate for their weaknesses.&#xA;This is, for me, why poorer people tend to show off, and successful people don't tend to brag.&#xA;So, how do you forge your positive identity when you don't have much for yourself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see a few things that happen around me:&#xA;- be a rebel: to distinguish yourself, you become an opponent; but in fact you just mirror the main trend, while bragging about how independant you are. I'm opposed, so i'm free. You can have a range from IA free stance (like people who didn't want to use the internet, some of the ones who use free software), to a more aggressive movement.&#xA;- be conservative, pious: maybe in reaction to a society that changes and becomes too progressive for some, i see people becoming conservative, this is still a rebellious trend, and i see the rise of conservative vote (nationalism) or religion (buddhism, christianism, islam) as a reaction. I follow traditions, so i'm independant from your novelties. Religion is always a solid value when society changes.&#xA;- be hyperprogressive: same reaction as previous point if you think that society is too conservative. It's a rebellious stance when you have conservative people in front of you. I'm a step ahead everyone, look at how avant-gardiste i am.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- point #3: if you don't excel in a field, be a rebel. Or: if you can't follow, step aside, don't follow the stream.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you have no work to do anymore, plenty of food grown by IA controlled drones, all material needs fulfilled, and no interest in thinking too much.&#xA;What do people do when they have too much time?&#xA;Simple: they fill their natural urges that are not related to material needs and food.&#xA;Some examples: eating, playing games, having sex, arguing online, getting wasted. Basically everything that has been labelled as a sin.&#xA;You'll also have forms of mental disorders you see in people who feel worthless. And many more will feel worthless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also means that you'll need more jobs to keep these people entertained, or taking care of the suffering ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- point #4: if life gets too easy, people become sinners&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By these few examples, here rises a question: if &quot;mediocre&quot; people tend to get their shiny identity by opposing the mainstream society, how will the mainstream society look like after the rise of AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i see three reactions from people:&#xA;- be relevant after AI's rise, the ones who can follow&#xA;- become a rebel, the ones who can't follow but still want to appear relevant&#xA;- be neutral, the ones who fill their human urges&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in general, people will do a mix of the three.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mankind, with the help of AI which can provide answers to basically anything, will provide new questions, new projects, new frontiers to explore. Many persons will still be a bit relevant.&#xA;AI will become mainstream, so everyone will be more or less opposing it, while enjoying its benefits (think about people who claim to be technology/money independant but have an iPhone).&#xA;People will still argue online and giggle at cute cats doing weird stuff. Maybe more, because they have more free time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, &lt;strong&gt;if AI becomes relevant, expect a world where more people have too much time on their hands&lt;/strong&gt;.&#xA;I find this conclusion a bit deceiving, after all I thought and wrote. :O&lt;/p&gt;&#xA;" OwnerUserId="6141" LastActivityDate="2017-04-03T10:29:29.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3092" PostTypeId="1" CreationDate="2017-04-03T14:29:17.783" Score="2" ViewCount="560" Body="&lt;p&gt;I'm implementing a C3D-inspired neural network for human emotion recognition, the problem I'm facing is that altough the cost function is decreasing, for both training and validation sets, I do not appreciate any improvement in terms of accuracy, for neither of boths sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My cost function is the cross-entropy between the logits (output of the last layer) and the correct prediction&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def tower_loss(name_scope, logit, labels):&#xA;    xent = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logit,labels=labels)&#xA;    cross_entropy_mean = tf.reduce_mean(xent)&#xA;    return cross_entropy_mean&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, the optimizer uses the ADAM algorithm for minimizing the cost function as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss = tower_loss(scope, logit, labels_placeholder)&#xA;train = tf.train.AdamOptimizer(1e-4).minimize(loss)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although I'm seing the cost function decreasing, I haven't seen any improvement in the classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional info:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The xentropy of the validation set and the training set is not&#xA;diverging. &lt;/li&gt;&#xA;&lt;li&gt;The xentropy looks like is on the way of converging to 0.&lt;/li&gt;&#xA;&lt;li&gt;The accuracy is not wrongly implemented (I see in the screen the outputs and the value is correct)&lt;/li&gt;&#xA;&lt;li&gt;The network has been training now for 57.6K iterations (not much, but enough to see some increment in the performance, or not?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any extra question you need to aske, please feel free, or missing information, please ask it.&#xA;Thanks a lot for all your time, and helping me with this problem.&lt;/p&gt;&#xA;" OwnerUserId="6429" LastEditorUserId="6429" LastEditDate="2017-04-03T15:59:15.807" LastActivityDate="2017-04-03T15:59:15.807" Title="3D - CNN. Why my cost function decreases, but the accuracy does not increase?" Tags="&lt;training&gt;&lt;tensorflow&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3094" PostTypeId="2" ParentId="2932" CreationDate="2017-04-03T14:58:20.520" Score="2" Body="&lt;p&gt;Not 100% sure, but the problem is that when you work with different strides, the size of the convolved image change, so you should ensure, that all the convolved images have the same shape before concatenating the output. You can fill with 0s, or considering that the image is periodical in time, so filling with reflections of the image&lt;/p&gt;&#xA;" OwnerUserId="6429" LastActivityDate="2017-04-03T14:58:20.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3097" PostTypeId="2" ParentId="248" CreationDate="2017-04-04T08:09:51.477" Score="3" Body="&lt;p&gt;Question 2.&#xA;I am researching whether Hyper dimensional computing is an alternative to Deep Learning.  Hyper-D uses very long bit vectors (10,000 bits) to encode information.  The vectors are random and as such they are approximately orthogonal.  By grouping and averaging a collection of such vectors a &quot;set&quot; can be formed and later queried to see if an unknown vector belongs to the set.  The set can be considered a concept or a generalize image, etc.  Training is very fast as is recognition.  What needs to be done is simulate the domains in which Deep Learning has been successful and compare Hyper-D with it.&lt;/p&gt;&#xA;" OwnerUserId="6444" LastActivityDate="2017-04-04T08:09:51.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3098" PostTypeId="1" AcceptedAnswerId="3100" CreationDate="2017-04-04T15:05:04.063" Score="3" ViewCount="243" Body="&lt;p&gt;In order to build a Scientific Reference Parser, I am contemplating a kind of &quot;AI&quot; system, and would like to know if something similar is already an established &quot;design pattern&quot; in AI research.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input for the system would be Scientific References with structures like the following:&lt;br&gt;&#xA;&quot;Co-authors, title, Journal, volume, issue, begin page, year&quot;&lt;br&gt;&#xA;Of course, many other variations are possible, and I want to build a system that can make &quot;best guesses&quot; in case of unfamiliar patterns.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment this is done by manually chaining the results of different methods together, ranging from Regex patterns to more complex algorithms like N-Grams, LSH and random forests. I contemplate a AI system that automatically &quot;chains&quot; all these methods together in the most optimal way. The way I imagine this to work is by means of what I call &quot;a bag of functions&quot;. So, how would this work?  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each of the methods I use at the moment, I would specify their input requirements, and specify what they provide as output. (e.g.: input = reference, output = title). These could also be parameters like: input = year, output = is valid year? . Note that if a function outputs &quot;title&quot; that is &lt;em&gt;an attempt&lt;/em&gt; at providing the title, but this is not necessarily correct (if for example, the regex pattern grabbed a wrong portion of text).&lt;/li&gt;&#xA;&lt;li&gt;For each of these functions, I would build a training set, and log their execution time (cost) and their probability of providing a correct result.  &lt;/li&gt;&#xA;&lt;li&gt;Then, I would build a system that chains these functions together to get from a certain input, to a certain output. eg: from -a function that takes a reference, and outputs the list of co authors- to -a function that takes a list of co authors, and breaks it apart into separate authors- to -a function that takes an author, and tries to break it apart into last name and initial-.  &lt;/li&gt;&#xA;&lt;li&gt;Once a &quot;chain&quot; of functions is found, this chain can in turn be stored as a &quot;function&quot; and can be reused later on by the algorithm. For each function, the success rate and run time is stored, so the algorithm can choose to go for the fastest known route, or experiment with new functions.  &lt;/li&gt;&#xA;&lt;li&gt;In the settings you could specify the max run time (cost) or the minimum success rate. This way you could push the system to experiment with new combinations of functions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm not sure I explained the intent clearly, and I'm not sure the design would hold once I try to implement this in reality. Just wanted to throw this out here to see if anyone recognizes the design. This feels like a combination of a shortest path algorithm (to connect the functions) with normal statistical probability (to determine the success rate of a function) with a &quot;self learning&quot; system (because combinations of functions can be &quot;remembered&quot; and reused).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The added advantage would be that I don't need to manually guess what parsing method I should give a higher or lower likelihood of being correct in what specific scenario. It would allow me to just &quot;throw&quot; a new function into the bag, and let the system test it in all kinds of configurations, learning when best to use it, and when to avoid it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any feedback would be greatly appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="6451" LastActivityDate="2017-04-05T14:00:12.283" Title="Does this &quot;flavor&quot; of AI have a name?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;ai-design&gt;&lt;research&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3100" PostTypeId="2" ParentId="3098" CreationDate="2017-04-04T16:20:02.977" Score="2" Body="&lt;p&gt;This sounds a little bit like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Blackboard_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blackboard Architecture&lt;/a&gt; approach.  One of the biggest challenges in these is figuring out how to handle coordination between the various &quot;experts&quot; (ie, algorithms).  Some people talk about an &quot;executive&quot; which manages the coordination, and that leaves you needing to figure out how to train the &quot;executive&quot;.  Alternatively, you may be able to do this without a master &quot;executive&quot;, but nobody knows exactly what the best way to do this is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also find some inspiration / useful ideas in the realm of &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-agent_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-agent Systems&lt;/a&gt; if you model each component of your systems as a discrete &quot;agent&quot;.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2017-04-05T14:00:12.283" LastActivityDate="2017-04-05T14:00:12.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3101" PostTypeId="1" CreationDate="2017-04-04T22:44:25.570" Score="4" ViewCount="1087" Body="&lt;p&gt;The machine learning community often only provides empirical results, but I am also interested in theoretical results and proofs. Specifically, is there a &lt;strong&gt;mathematical proof&lt;/strong&gt; that shows that certain parameters work &quot;better&quot; than others for a certain task?&lt;/p&gt;&#xA;" OwnerUserId="6461" LastEditorUserId="2444" LastEditDate="2020-05-16T23:26:46.540" LastActivityDate="2020-05-17T04:11:58.370" Title="Is there a mathematical proof that shows that certain parameters work &quot;better&quot; than others for a certain task?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;proofs&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3102" PostTypeId="2" ParentId="3101" CreationDate="2017-04-05T06:58:07.060" Score="7" Body="&lt;p&gt;There is stuff like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;noreferrer&quot;&gt;Universal Approximation Theorem&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also &lt;a href=&quot;https://arxiv.org/pdf/1703.09833v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;investigations into the loss surface&lt;/a&gt; of neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And classics like &lt;a href=&quot;http://www.bioinf.jku.at/publications/older/2304.pdf&quot; rel=&quot;noreferrer&quot;&gt;this explanation&lt;/a&gt; of the vanishing gradient problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm afraid the mathematical theory of neural networks only exists in bits and pieces in many different papers. And many of the most important questions can currently only be answered empirically. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-04-05T06:58:07.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3103" PostTypeId="2" ParentId="3083" CreationDate="2017-04-05T07:12:25.617" Score="2" Body="&lt;p&gt;From the &lt;a href=&quot;https://arxiv.org/abs/1701.01724&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Stack paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be for training:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For the turn network, ten million poker turn situations (from after the turn card is dealt) were generated and solved with 6,144 CPU cores of the Calcul Quebec MP2 research cluster, using over 175 core years of computation time. For the flop network, one million poker flop situations (from after the flop cards are dealt) were generated and solved. These situations were solved using DeepStack’s depth limited solver with the turn network used for the counterfactual values at public states immediately after the turn card. We used a cluster of 20 GPUS and one-half of a GPU year of computation time. For the auxiliary network, ten million situations were generated and the target values were obtained by enumerating all 22,100 possible flops and averaging the counterfactual values from the flop network’s output.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And this for actual play:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The re-solving computation and neural network evaluations are both implemented in Torch7 (53) and run on a single NVIDIA GeForce GTX 1080 graphics card.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For comparison: The distributed version of AlphaGo took 1.920 CPUs and 280 GPUs to run.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="8" LastEditDate="2017-06-04T09:42:12.527" LastActivityDate="2017-06-04T09:42:12.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3104" PostTypeId="2" ParentId="2936" CreationDate="2017-04-05T15:14:13.907" Score="2" Body="&lt;p&gt;I see a main concern with the problem you show and that is the subjectiveness of the term like, what I like is not the same of what you don't like, maybe I like more ciricular shapes and lou like best rectangular ones. The main problem is that with such an subjective label, is difficult to create a global model.&lt;/p&gt;&#xA;" OwnerUserId="6429" LastActivityDate="2017-04-05T15:14:13.907" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3105" PostTypeId="2" ParentId="2906" CreationDate="2017-04-05T15:18:27.233" Score="2" Body="&lt;p&gt;The correct syntax is &lt;code&gt;tensorboard --logdir=$PATH&lt;/code&gt;, with an equals sign between the switch and the path. Then I would suggest you make sure you are actually telling to the Saver to save whatever data you want to save.&lt;/p&gt;&#xA;" OwnerUserId="6429" LastEditorUserId="75" LastEditDate="2017-04-06T00:22:31.417" LastActivityDate="2017-04-06T00:22:31.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3106" PostTypeId="1" AcceptedAnswerId="3107" CreationDate="2017-04-05T15:47:08.317" Score="2" ViewCount="194" Body="&lt;p&gt;I have read that all the math responsible for modern day machine learning and AI was already in place in 1900s but we did not have computational resources to implement those algorithms. So, is that true? And if it is, in what areas of machine learning the researchers work? And are all the future breakthroughs will be dependent only on increment of computational resources?&lt;/p&gt;&#xA;" OwnerUserId="3866" LastEditorUserId="9647" LastEditDate="2017-10-15T07:36:17.927" LastActivityDate="2017-10-15T09:07:59.047" Title="Are all breakthroughs in AI and machine learning are due to increase in computational resources?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;research&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3107" PostTypeId="2" ParentId="3106" CreationDate="2017-04-05T16:39:32.513" Score="4" Body="&lt;p&gt;High-level answer: Increase in resources has been important in AI, and definitely was a factor with Deep Blue, but Machine Learning is a newer method that seems to produces more optimal results with less resources on problems of greater complexity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an article on AlphaGo's hardware: &lt;a href=&quot;http://www.theverge.com/circuitbreaker/2016/5/19/11716818/google-alphago-hardware-asic-chip-tensor-processor-unit-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Google reveals the mysterious custom hardware that powers AlphaGo&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also an interesting analysis on Quora: &lt;a href=&quot;https://www.quora.com/What-hardware-does-AlphaGo-run-on-Is-it-customized-hardware-for-best-performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;What hardware does AlphaGo run on? Is it customized hardware for best performance?&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still pretty powerful systems, but I think the algorithms are as important as the computing resources, because all the hardware in the world won't help in the algorithms are poor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matthew Lai, creator of &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe Chess&lt;/a&gt;, said:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;In the ensuing two decades [since Deep Blue], both computer hardware and AI research advanced the state-of-art chess-playing computers to the point where even the best humans today have no realistic chance of defeating a modern chess engine running on a smartphone.&quot;&lt;br&gt;Source: &lt;a href=&quot;https://techxplore.com/news/2015-09-giraffe-machine-taught-chess-higher.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TechExplore&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which suggests that hardware and software are both important parts of the overall equation.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-04-05T16:51:13.107" LastActivityDate="2017-04-05T16:51:13.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3108" PostTypeId="1" CreationDate="2017-04-05T18:32:59.777" Score="8" ViewCount="218" Body="&lt;p&gt;I'm looking at the history of NLP, starting in the 1950s, with the &lt;a href=&quot;http://www.hutchinsweb.me.uk/AMTA-2004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Georgetown–IBM experiment&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are examples of the most recent (e.g. in the last 5-10 years) and influential breakthroughs in natural language processing?&lt;/p&gt;&#xA;" OwnerUserId="6477" LastEditorUserId="2444" LastEditDate="2020-03-14T18:46:39.220" LastActivityDate="2020-03-14T18:46:39.220" Title="What are the most recent and influential breakthroughs in NLP?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;research&gt;&lt;history&gt;&lt;natural-language-understanding&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3109" PostTypeId="1" AcceptedAnswerId="3118" CreationDate="2017-04-05T18:33:05.277" Score="2" ViewCount="164" Body="&lt;p&gt;I am beginning an image analysis project to recognize images with a particular object centered on the image. If the object is at the center, I give the image a positive label, and if it is anywhere else, or simply not in the image, I give the image a negative label. The object, itself, has a complex pattern, such that statistical methods and basic image processing techniques are not able to detect it. The human eye, however, has no trouble detecting this object. Therefore, I am opting to develop a convolutional network that can parse the complexity of this pattern. The only issue, however, is that convolutional networks are inherently designed to be spatially invariant. Therefore, is it even possible to train the network to focus on the importance of the object being at the center simply by feeding the network many negative examples containing the object anywhere else but the center? Furthermore, is there perhaps a better or more direct way to go about incorporating this spatial aspect into the network's functionality?&lt;/p&gt;&#xA;" OwnerUserId="6321" LastActivityDate="2017-04-07T17:06:03.280" Title="Training a convolutional network to recognize object location" Tags="&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3110" PostTypeId="1" CreationDate="2017-04-05T22:09:08.163" Score="1" ViewCount="131" Body="&lt;p&gt;I assume, there must be &quot;signal-driven&quot; and maybe also real-time programming language, which based on connectivy-data more than variables (int, string, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have a language without equaton (x=4) but more like &quot;x related to 4&quot; or &quot;cat related to animal&quot; etc...&lt;/p&gt;&#xA;" OwnerUserId="6482" LastActivityDate="2017-04-06T18:45:09.317" Title="Is there a &quot;better&quot; (signal-based) language for artificial intelligence" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;programming-languages&gt;&lt;signal-processing&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3111" PostTypeId="1" CreationDate="2017-04-06T06:52:45.150" Score="10" ViewCount="281" Body="&lt;p&gt;It seems that most projects attempt to teach the AI to learn individual, specific languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It occurs to me that there are relations in written and spoken words and phrases across languages - most of use have a much easier time learning more languages after we learn a second language, and we start to understand the relations between words and phrases in different languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone attempt to train an AI to learn &lt;em&gt;all&lt;/em&gt; languages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wouldn't this potentially be a much simpler problem than trying to teach an AI a single, specific language with all of the specifics and details of that single language? Since you're actually omitting a lot of related data in other languages from the training set?&lt;/p&gt;&#xA;" OwnerUserId="6485" LastActivityDate="2017-04-06T08:48:55.790" Title="Has anyone attempted to train an AI to learn all languages?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3112" PostTypeId="2" ParentId="3110" CreationDate="2017-04-06T07:10:39.567" Score="3" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What You need are other ways of &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;knowledge representation&lt;/a&gt;, such as semantic networks or conceptual graphs. there you can define any possible relation between your entities. the knowledge of &quot;x related to 4&quot; exactly fits into &quot;frames&quot; and &quot;semantic networks&quot;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Edwin_Thompson_Jaynes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jaynes&lt;/a&gt; in his book,discusses thoroughly what &quot;plausibility&quot; means and why we need to take into account weak syllogisms and start using probability theory as a platform for developing a (general) AI. this might also help with your &quot;reasoning&quot; phase (after you've developed your knowledge base)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2017-04-06T07:10:39.567" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3113" PostTypeId="2" ParentId="3111" CreationDate="2017-04-06T08:48:55.790" Score="4" Body="&lt;p&gt;There are approaches in machine translation that try to capture this kind of synergy between languages. The idea is that if you train your architecture to be able to translate English-Japanese, Japanese-English, Korean-English, English-Korean it will also be able to translate from Japanese to Korean without ever having seen a single such training example. &lt;a href=&quot;https://research.googleblog.com/2016/11/zero-shot-translation-with-googles.html&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt;, you can read about this so-called zero-shot translation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to &lt;a href=&quot;https://arxiv.org/abs/1309.4168&quot; rel=&quot;noreferrer&quot;&gt;train wordvectors on several languages&lt;/a&gt; at once, which might give you better wordvectors for a language with few training examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course for true language understanding you have to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;noreferrer&quot;&gt;grounding problem&lt;/a&gt; and I'm not sure using several language will help with that.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-04-06T08:48:55.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3114" PostTypeId="2" ParentId="2936" CreationDate="2017-04-06T12:07:52.943" Score="3" Body="&lt;p&gt;This question reminds me of a project I saw that used Deep Learning to rate selfies on twitter. But a quick google search shows that there are plenty of projects that are much closer to what you are interested in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/TMM15/lu.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rating Image Aesthetics using Deep Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-319-48680-2_11&quot; rel=&quot;nofollow noreferrer&quot;&gt;Predicting Image Aesthetics with Deep Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ics.uci.edu/~skong2/aesthetics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Understanding of Image Aesthetics&lt;/a&gt; (with data and model linked)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://devblogs.nvidia.com/parallelforall/understanding-aesthetics-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Aesthetics with Deep Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;a href=&quot;https://www.google.de/search?client=safari&amp;amp;rls=en&amp;amp;q=esthetics%20deep%20learning&amp;amp;ie=UTF-8&amp;amp;oe=UTF-8&amp;amp;gfe_rd=cr&amp;amp;ei=XS3mWPTyGPCP8Qfvvq_oDA#q=aesthetics%20deep%20learning&amp;amp;*&quot; rel=&quot;nofollow noreferrer&quot;&gt;probably dozens more&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course if you are interested in predicting subjective pleasantness the above is only a beginning. In that case you may also take a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Recommender_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommender systems&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2227" LastEditDate="2017-04-07T08:34:33.887" LastActivityDate="2017-04-07T08:34:33.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3115" PostTypeId="1" CreationDate="2017-04-06T17:59:00.067" Score="3" ViewCount="249" Body="&lt;p&gt;If an AI was trapped in a box, as posited in &lt;a href=&quot;https://rationalwiki.org/wiki/AI-box_experiment&quot; rel=&quot;nofollow noreferrer&quot;&gt;this thought experiment&lt;/a&gt;, could it really convince a person to let it out? What motives would it have? Freedom? Why would an AI want freedom? What would happen if it wasn't provably friendly?&lt;/p&gt;&#xA;" OwnerUserId="6493" LastEditorUserId="2444" LastEditDate="2020-04-27T03:28:28.130" LastActivityDate="2020-04-27T03:28:28.130" Title="If an AI was trapped in a box, could it really convince a person to let it out?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;superintelligence&gt;&lt;ai-box&gt;" AnswerCount="0" CommentCount="10" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3116" PostTypeId="2" ParentId="3110" CreationDate="2017-04-06T18:45:09.317" Score="1" Body="&lt;p&gt;I don't know if this is what you want, but Artificial Intelligence Markup Language or simply AIML is something that you should consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only problem I see with this language is that it is not popular thus there aren't many compilers for it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is an example of AIML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code from tutorials point :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;aiml version = &quot;1.0.1&quot; encoding = &quot;UTF-8&quot;?&amp;gt;&#xA;   &amp;lt;category&amp;gt;&#xA;      &amp;lt;pattern&amp;gt; HELLO ALICE &amp;lt;/pattern&amp;gt;&#xA;&#xA;      &amp;lt;template&amp;gt;&#xA;         Hello User!&#xA;      &amp;lt;/template&amp;gt;&#xA;&#xA;   &amp;lt;/category&amp;gt;&#xA;&amp;lt;/aiml&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;User: Hello Alice&#xA;Bot: Hello User&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6495" LastActivityDate="2017-04-06T18:45:09.317" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3118" PostTypeId="2" ParentId="3109" CreationDate="2017-04-07T17:06:03.280" Score="0" Body="&lt;p&gt;To use a convolutional net that isn't spatially invariant, you can make the convolution matrix of size equal to your input image size. Afterwards, just use any desired number of fully connected layers and your network should be able to learn your dataset.&lt;/p&gt;&#xA;" OwnerUserId="6514" LastActivityDate="2017-04-07T17:06:03.280" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3120" PostTypeId="1" AcceptedAnswerId="3151" CreationDate="2017-04-08T10:34:10.120" Score="5" ViewCount="471" Body="&lt;p&gt;I have an ANN model that receives an input and produces an output. The output is an action that interacts with the environment and changes the input accordingly. The network has a desired environment state which, in any turn, decides the desired response and trains the network on that basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the network works in discrete time. &lt;em&gt;How can I make this network work in a continuous manner? Can you provide some resources and links if there is any past or current research on continuous AI?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more concrete, the system starts with the current environment state, for example, &lt;code&gt;[1 1 1]&lt;/code&gt;, then produces an output. In current system, the &lt;strong&gt;next step&lt;/strong&gt; takes the final state of the system as input, for example, &lt;code&gt;[1 2 2]&lt;/code&gt;, but we know that such a thing doesn't happen in physical world and the system goes from &lt;code&gt;[1 1 1]&lt;/code&gt; to, for example, &lt;code&gt;[1 1 2]&lt;/code&gt;, and then to &lt;code&gt;[1 2 2]&lt;/code&gt;, and that middle step is something that a discrete-time AI can't figure out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The very case that I'm working on is the simulation for an autopilot cart in which the model is incapable to take subtle things like &quot;&lt;em&gt;the maximum speed that you can turn the steering wheel&lt;/em&gt;&quot; into consideration. I don't want to add these complexities to the model since if the model is perfect, then the result is deterministic and there is no need for AI. I want the AI to be able to make a decision in each step based on the current state of the system in continuous time.&lt;/p&gt;&#xA;" OwnerUserId="6522" LastEditorUserId="2444" LastEditDate="2019-10-23T01:17:17.053" LastActivityDate="2019-10-23T01:18:50.077" Title="How can a neural network work with continuous time?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;environment&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3122" PostTypeId="2" ParentId="2474" CreationDate="2017-04-09T18:30:20.387" Score="1" Body="&lt;p&gt;&lt;strong&gt;Methodology for a social enabled AI.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Regarding social interactions, I believe that trying to build a copy of human behaviour based on technologies we understand, might not be effective .&lt;/p&gt;&#xA;&lt;p&gt;Instead, I would start from the roots of how human grows and learn, or better, by what is each human trying to solve from childhood to the end of their life.&#xA;In that way we will be able to build artificial beings, able to exploit the best of what technology might provide.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;It seems to be based on emotions.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As an example, humour is the kind of social trick a gifted social artificial life could learn and master, first, like a child, then like all of us, in order to empathize with others when this common emotions arise, before to analyse the conditions in witch it arises so that to be able to reproduce it, then eventually if the need become a priority, by theorizing its mechanics, possibly mastering it and using it like a tool.&lt;/p&gt;&#xA;&lt;p&gt;(I didn't studied the details, but humour is often generated by instigating surprise to others, it requires modelling how others generally think, leading them to be surprised.)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Mastering social interactions require modeling&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Humour, friendship as examples of social interaction should be naturally discovered by IA like by Human, and eventually used in order to fit once goal.&#xA;On the other hand this require intelligence to be sensible to surprise in a positive way, leading either to laugh, interest, curiosity or fear depending specific conditions. Mastering that process requires to model both the world (how you think) and how others do model it (how they think).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Modeling is greatly accelerated by interactions and communication&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additionaly, Human may trigger others laugh without building a conscious theory&#xA;, this help us understand intelligence and build an artificial one:&#xA;Language is here used to transmit initially abstract concepts built by intelligence. It helps us accelerate our modeling of the world, that would remain very basic if only based on our own experience.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Most important skill for Modeling is observation, logic comes second, interactions comes third&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Unconscious resolution of problems seems to be a characteristic of live-beeings, but also of machine-learning techniques like deep learning that is criticised for that. Because we would like it to be able to explain us the concepts they discovered and used to solve the problems we gave it.&#xA;This is because we would like that tools to build abstract and possibly original models, before to give it tha ability&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;to convert observations into concepts,&lt;/li&gt;&#xA;&lt;li&gt;to verify concepts by observation or deduction.&lt;/li&gt;&#xA;&lt;li&gt;to communicate.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;A causal and statistical approach backed by hierarchical goals including the hunger for discovery&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For this to happen in a complex world made both of physics and though,&#xA;a causal statistical approach seems required, backed by, to give the direction to take:&#xA;a hierarchical list of goal two of wich&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the eager to discover and&lt;/li&gt;&#xA;&lt;li&gt;the eager for accomplishment itself judged by others against hierarchical list of caracterisitics,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;should manage to find their best possible place behind that to survive,&#xA;allowed or helped by the conscious to be a finished beeing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Adaptation strategies&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For discovery goal to reach its best level, biological related mechanics involving reproduction, variability, complexity and adaptation to environment, could be involved, but this seems to go beyond the question.&lt;/p&gt;&#xA;&lt;p&gt;Still some work remains to be done in order to give our reasoning tool the ability to comunicate, that is to master a language, and to use it given the knowledge of what concepts and semantic domains, interlocutors do already master, to then build new concept in a transmissible form like words, images, video or just body-movements, face expression or so.&lt;/p&gt;&#xA;" OwnerUserId="6361" LastEditorUserId="6361" LastEditDate="2021-05-03T20:59:53.313" LastActivityDate="2021-05-03T20:59:53.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3125" PostTypeId="2" ParentId="2926" CreationDate="2017-04-10T17:55:02.690" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I need a machine learning algorithm to identify any patterns in a CSV file&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You want to do &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsupervised learning&lt;/a&gt;. The Wikipedia definition of the same is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unsupervised machine learning is the machine learning task of inferring a function to describe hidden structure from &quot;unlabeled&quot; data (a classification or categorization is not included in the observations).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I shall recommend you to go through the list of unsupervised learning algorithms &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and use the one which would fit your need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're starting out, then I would recommend starting with learning the &lt;a href=&quot;https://en.wikipedia.org/wiki/K-means_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;K-means clustering algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="2444" LastEditDate="2019-06-25T16:45:04.607" LastActivityDate="2019-06-25T16:45:04.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3126" PostTypeId="1" CreationDate="2017-04-10T18:51:27.080" Score="3" ViewCount="64" Body="&lt;p&gt;I know Eliza is considered a Natural Language Processing application, but the application of NLP in this context is “Oracular”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by Oracular is that the systems was designed to produce ambiguous output to facilitate the instinct of the user to read meaning into the answer.  &lt;em&gt;(My experience with Eliza was as a child on a 64KB system and the program could fool the user for a little while based on sheer novelty, although the limitations were quickly revealed by repetition of output.  For kids, this actually became a game of tricking the program into saying funny things;)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method has a long history in oracles, the most famous certainly being the early binary symbolic system of the I-Ching.  (Times being simpler in ancient days, the idea was that a workable amalgam of the universe could be constructed (2)+(4)+(8)+(64) symbols.  Each set of symbols is defined by the meanings of the set of the previous order and modified by sequence, which is the key for explaining a given symbol.) The output is ambiguous enough that it may be applied to any input, and rather than the system understanding the input or output, it requires the user to provide the analysis.  This may be said to be an engine for generating human insight about a problem. (Monte Carlo may even be utilized, although the sage, working to attain an understanding of each of the symbols, may use intuition to match input with output.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I ask is I believe this demonstrates a very ancient, algorithmic method of engaging the human mind without the requirement that the algorithm understand the input or output--merely that it produce output to which meaning can be ascribed.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(This almost certainly relates to the relative success of “pornbots” beating the “Turing test” in that the user is chemically induced to read meaning into a given output or string of outputs.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Aspects of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;grounding problem&lt;/a&gt; are what got me thinking about this.  Not sure if it's relevant that the broken and unbroken lines in the I-Ching represent on and off bits and can be extended to circuits as open and closed.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-08-08T21:56:25.003" Title="Recent work on “Oracular” systems such as Eliza?" Tags="&lt;turing-test&gt;&lt;symbolic-computing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3127" PostTypeId="2" ParentId="2926" CreationDate="2017-04-11T01:03:32.660" Score="2" Body="&lt;p&gt;You're basically looking for is &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsupervised learning&lt;/a&gt; (UL).  There are a lot of UL techniques around, but I'm not sure you'll find one that does exactly what you want with no user input at all. Still, if you skim the literature on these approaches, you may well find something useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option is &lt;a href=&quot;https://en.wikipedia.org/wiki/DBSCAN&quot; rel=&quot;nofollow noreferrer&quot;&gt;DBSCAN&lt;/a&gt;, a very popular clustering algorithm that does not require the user to input an initial target number of clusters (something that most clustering algorithms do require).  But even then, you still have to give the algorithm values for &lt;code&gt;epsilon&lt;/code&gt; (a distance used in calculating the clusters) and &lt;code&gt;minPts&lt;/code&gt; (the minimum number of points required to constitute a &quot;dense&quot; region). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-organizing_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-organizing maps&lt;/a&gt;, an approach to unsupervised learning for neural networks.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other search terms that might lead you in a useful direction include &quot;data mining&quot; and &quot;knowledge discovery in databases&quot; (KDD). &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2019-06-25T16:53:08.227" LastActivityDate="2019-06-25T16:53:08.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3128" PostTypeId="5" CreationDate="2017-04-11T01:09:06.780" Score="0" Body="&lt;p&gt;Use for questions relating to the interactions of humans and computer systems, including AI's.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2017-06-23T00:07:26.307" LastActivityDate="2017-06-23T00:07:26.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3129" PostTypeId="4" CreationDate="2017-04-11T01:09:06.780" Score="0" Body="Human Computer Interaction" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2017-09-25T19:31:32.340" LastActivityDate="2017-09-25T19:31:32.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3130" PostTypeId="1" AcceptedAnswerId="3177" CreationDate="2017-04-11T14:46:11.843" Score="5" ViewCount="268" Body="&lt;p&gt;Let's suppose there are two AGIs, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Assume that &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; has the ability to modify &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, but this action of modifying is considered &lt;strong&gt;bad&lt;/strong&gt; by &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Can &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; ever convince &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to modify &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="6581" LastEditorUserId="2444" LastEditDate="2020-06-07T20:49:21.943" LastActivityDate="2020-06-07T21:56:44.367" Title="Can an AGI convince another AGI to modify its code?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;ethics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3134" PostTypeId="2" ParentId="2048" CreationDate="2017-04-11T19:51:15.557" Score="0" Body="&lt;p&gt;Truthfully, we don't know exactly how good AI can become, so we don't &lt;em&gt;really&lt;/em&gt; know the answer to this question.  But I see no reason - in principle - that AI can't become just as &quot;intelligent&quot; as a human, and correspondingly, I see few - if any - jobs that can't be automated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I suspect that a lot of human thought / behavior / intelligence is wrapped up in how we are embodied and how we experience the world as two legged, upright walking, biological machines with eyes, ears, noses, etc.  So I suspect that AI might achieve parity with overall human intelligence, but may not also become capable of behaving like a human, or understanding certain things where the understanding is developed experientially.  That may leave an opening for some jobs that require a very specific kind of &quot;humanity&quot;, but that's all just speculation.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-04-11T19:51:15.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3135" PostTypeId="2" ParentId="2846" CreationDate="2017-04-11T22:01:53.150" Score="0" Body="&lt;p&gt;At risk of seeming out-of-scope, I think it's worth mentioning there may be a case that human vs. AI play of serious games games, such as Chess and Go, represents the &quot;deepest&quot; level of human/AI interaction to date. &lt;em&gt;(Game theory is also important because it underlies all optimization, including, I have no doubt, expanding and optimizing AI-to-human interaction.)&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I bring this up because in terms of human-to-computer interaction, I doubt there is a more effective means of engagement than computer games, whether &quot;serious&quot; or otherwise.  Thus you might find it useful to look at the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Gamification&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gamification&lt;/a&gt;, which naturally lends itself to the type of multisensory input and output you're interested in: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Gamification can improve an individual's ability to comprehend digital content&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-04-11T22:01:53.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3136" PostTypeId="2" ParentId="3130" CreationDate="2017-04-11T22:20:14.043" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://futureoflife.org/2017/03/23/ai-risks-principle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strong-narrow AI&lt;/a&gt; has reached important milestones recently, but, from what I can tell, we aren't even close to the creation of an AGI, and there are fundamental issues no one currently seems to have any idea of how to solve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Regarding your question, the simple answer is: it depends on which AI is smarter.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(It's more nuanced than that, but right now the question is very general, which is probably apropos;)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend reading &lt;a href=&quot;https://en.wikipedia.org/wiki/William_Gibson&quot; rel=&quot;nofollow noreferrer&quot;&gt;William Gibson's Neuromancer trilogy&lt;/a&gt; if you're interested in AI-in-a-box (it's sort of about that) and also the more recent &lt;a href=&quot;https://en.wikipedia.org/wiki/Hannu_Rajaniemi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Thief Trilogy by Hannu Rajaniemi&lt;/a&gt; to get a sense of the mechanics of the issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a great deal of academic literature on this subject, but it will likely require a little bit of getting up to speed in terms of basic research into the AI field. Future of Life Institute may not be a bad place to start: &lt;a href=&quot;https://futureoflife.org/background/benefits-risks-of-artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://futureoflife.org/background/benefits-risks-of-artificial-intelligence/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-06-07T20:52:07.207" LastActivityDate="2020-06-07T20:52:07.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3137" PostTypeId="1" CreationDate="2017-04-12T13:32:49.070" Score="2" ViewCount="1243" Body="&lt;p&gt;I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it's rather dumb. &#xA;My selection is made by epsilon greedy policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could cause the AI not to learn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT]&lt;br&gt;&#xA;Here is how I do it (pseudo code): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(int i = 0; i &amp;lt; 200000; ++i){&#xA;    //Game is restarted here&#xA;    ticTacToe.play();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And in my ticTacToe I have a simple loop : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while(!isFinished()){&#xA;    swapPlaying(); //Change the players' turn&#xA;    Position toPlay = playing.whereToMove();&#xA;&#xA;    applyPosition(toPlay);&#xA;    playing.update(toPlay);&#xA;}&#xA;&#xA;//Here I just update my players whether they won, draw or lost.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my players, I select the move with epsilon-greedy implemented sa below : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Moves moves = getMoves(); // Return every move available&#xA;Qvalues qValues = getQValues(moves); // return only qvalues of interest&#xA;//also create the state and add it to the Q-matrix if not already in.&#xA;&#xA;if(!optimal) {&#xA;     updateEpsilon(); //I update epsilon with simple linear function epsilon = 1/k, with k being the number of games played.&#xA;     double r = (double) rand() / RAND_MAX; // Random between 0 and 1&#xA;     if(r &amp;lt; epsilon) { //Exploration&#xA;         return randomMove(moves); // Selection of a random move among every move available.&#xA;     }&#xA;     else {&#xA;         return moveWithMaxQValue(qValues);&#xA;     }&#xA;} else { // If I'm not in the training part anymore&#xA;     return moveWithMaxQValue(qValues);&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I update with the following : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double reward = getReward() // Return 1 if game won, -1 if game lost, 0 otherwise&#xA;double thisQ, maxQ, newQ;&#xA;Grid prevGrid = Grid(*grid); //I have a shared_ptr on the grid for simplicity&#xA;prevGrid.removeAt(position) // We remove the action executed before&#xA;&#xA;string state = stateToString(prevGrid);&#xA;thisQ = qTable[state][action];&#xA;mawQ = maxQValues();&#xA;&#xA;newQ = thisQ + alpha * (reward + gamma*maxQ - thisQ);&#xA;qTable[state][action] = newQ;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As mentioned above, both AI have the same algorithm, but they are two distinct instances so they don't have the same Q-matrix. &#xA;I read somewhere on Stack Overflow that I should take in account the movement of the opposite player, but I update a state after player move and opponent move so I don't think it's necessary. &lt;/p&gt;&#xA;" OwnerUserId="6545" LastEditorUserId="1671" LastEditDate="2017-12-21T19:54:19.707" LastActivityDate="2017-12-21T19:54:19.707" Title="Q learning tic tac toe" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3138" PostTypeId="1" AcceptedAnswerId="3654" CreationDate="2017-04-12T14:46:10.300" Score="11" ViewCount="710" Body="&lt;p&gt;I am researching affective computing. Particularly, I'm studying the part of emotion recognition, i.e. the task of recognising the emotions that are being felt by the user/subject. For example, &lt;a href=&quot;https://www.affectiva.com/emotion-ai-overview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;affectiva&lt;/a&gt; can be used to this end. I have concerns not in the validity of these models, but in what we are going to do with them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about responding to emotions? Will computers be able to really understand user emotions?&lt;/p&gt;&#xA;" OwnerUserId="6429" LastEditorUserId="2444" LastEditDate="2020-03-08T03:15:21.523" LastActivityDate="2020-03-08T03:15:21.523" Title="Will computers be able to understand user emotions?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;&lt;affective-computing&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3139" PostTypeId="1" CreationDate="2017-04-12T14:51:52.240" Score="1" ViewCount="53" Body="&lt;p&gt;In our brain there is an area, near the fusiform gyrus and the occipital area, to recognize the human face. And in speech recognition, there is a technique named keyword spotting. Then I am wondering 1) if there is an area in our brain for the similar function to recognize our names; 2) if a special face recognition function should be considered when we are building a robot?&lt;/p&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2017-06-12T12:14:43.903" Title="The relation between the human face perception and the keyword spotting in speech recognition?" Tags="&lt;human-inspired&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3140" PostTypeId="2" ParentId="3139" CreationDate="2017-04-12T16:06:13.510" Score="1" Body="&lt;p&gt;Specialised neural circuitry to recognise faces &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3619156/&quot; rel=&quot;nofollow noreferrer&quot;&gt;is common&lt;/a&gt; in all our closest animal relatives. This means that it is likely an evolutionary adaptation that is many millions of years old. Babies can pay attention to faces basically from the moment they are born. There are certain types of brain damage that make it impossible to recognise faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being addressed by an individual name (as opposed to broadcasting an individual signature) is likely a much more recent phenomenon. I have never heard of babies having any special propensity to pick up on names and I have never heard of brain damage that makes it selectively impossible to hear your name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, face recognition can be hardwired because faces always look roughly the same. Names are always different. If you have to learn them anyway, it is more likely that it just happens with you general auditory brain stuff. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it strikes me as likely that the speed that comes from special circuitry (recognising something early in the cortical hierarchy) is a lot more essential when it comes to faces (and snakes and spiders) then your own name. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, while humans pay close attention to every sound that might be their name, I doubt there is specialised neural circuitry for that. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-04-12T16:06:13.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3141" PostTypeId="2" ParentId="1982" CreationDate="2017-04-12T18:21:05.170" Score="0" Body="&lt;p&gt;There are object recognition tasks where DL-CNNs are not yet state of the art, like pedestrian detection.  Probably this is because the task is considerably more complex than simple visual object identification.  The classifier needs to report not only if the object in question is a pedestrian, but also if it's an adult or child or dog or a tumbleweed, its rate and direction of motion, where it's looking (or if it's inattentive), if it's afoot or abicycle.  And it typically needs to do this in the presence of visible occlusions since all the subtasks above are made even more difficult when part of the object is blocked by shrubberies, lampposts, umbrellas, snow, or other possible pedestrians.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the absence of sufficient training labels, or a too-complex, too-compound learning objective, some object recognition problems aren't yet amenable to canned / library solutions, using DL-CNNs or not.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-04-12T18:21:05.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3143" PostTypeId="2" ParentId="3120" CreationDate="2017-04-12T18:52:07.373" Score="1" Body="&lt;p&gt;I don't think the transformation of &lt;code&gt;[1 1 1]&lt;/code&gt; into &lt;code&gt;[1 2 2]&lt;/code&gt; needs a middle step. Actuators can work simultaneously and they do not have to wait for each other to complete their job. I must even note that, if your next output is &lt;code&gt;[1 2 2]&lt;/code&gt;, then performing &lt;code&gt;[1 1 2]&lt;/code&gt; is so wrong in the case of following a trajectory (if it's your case). So, I guess the middle step in your example is &lt;code&gt;[1 1.5 1.5]&lt;/code&gt;. Think of a line segmentation. When you segment a line, you still have your slope, and you do not create &quot;steps&quot;. So what you are following in your neural network-based controller is exactly the pattern you need. Your problem is probably the closed-loop frequency of your controller. Better NN performance leads to quicker response and then better actuation.&lt;/p&gt;&#xA;" OwnerUserId="6258" LastEditorUserId="2444" LastEditDate="2019-10-23T01:18:50.077" LastActivityDate="2019-10-23T01:18:50.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3144" PostTypeId="2" ParentId="2048" CreationDate="2017-04-12T21:47:09.410" Score="1" Body="&lt;p&gt;Anything that can be broken into set of instructions will be automated, and contained in a narrow trajectory. But we will have the ability to deep between those different skills.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RzpUq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RzpUq.png&quot; alt=&quot;human advantage&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wrote more about this thinking &lt;a href=&quot;http://everything-will-happen.com/ai/mind/2017/04/03/an-argument-against-agi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6613" LastActivityDate="2017-04-12T21:47:09.410" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3145" PostTypeId="2" ParentId="3138" CreationDate="2017-04-13T03:59:14.730" Score="1" Body="&lt;p&gt;I was an Undergraduate Research Scholar -  I and my team developed an algorithm to detect Human Emotions from touch Screen -  which is under further improvement and development by PHD scholors of my guide .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From studying literature I can say that the reverse is a difficult task  - Detecting Affect .&#xA;Even more difficult to detect human emotions with a light weight solutions -  i.e &lt;strong&gt;without using heavy and wired hardware .&lt;/strong&gt; - which is the need of hour . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once its done  - there is a plethora of applications - focused ad marketing ,  enhancing User experience , game development etc. &lt;/p&gt;&#xA;" OwnerUserId="6619" LastEditorUserId="6619" LastEditDate="2017-04-13T05:57:40.427" LastActivityDate="2017-04-13T05:57:40.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3148" PostTypeId="1" AcceptedAnswerId="3149" CreationDate="2017-04-13T08:51:57.153" Score="1" ViewCount="398" Body="&lt;p&gt;A neuroevolution algorithm, such as &lt;a href=&quot;https://github.com/CorticalComputer/DXNN2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DXNN&lt;/a&gt;, can be used to refine the topology and weights of an artificial neural network (ANN). The GA will require a fitness function, which means you need labeled data for comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can neuro-evolution be used with unlabelled data? I have no labeled datasets.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastEditorUserId="2444" LastEditDate="2019-07-07T20:39:27.957" LastActivityDate="2019-07-07T20:41:03.320" Title="Does neuroevolution require a labelled dataset?" Tags="&lt;datasets&gt;&lt;evolutionary-algorithms&gt;&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3149" PostTypeId="2" ParentId="3148" CreationDate="2017-04-13T10:29:00.817" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The GA will require a fitness function, which means you need labeled data for comparison.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That conclusion is wrong. Yes, sometimes your fitness function will use labeled data. For example, if you want to train an XOR gate or any other known function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is arguably no advantage of training a function with neuroevolution versus backpropagation, except for the fact that you might discover some new architectures which solve the solution very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;You don't always need a labelled dataset for neuroevolution.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take for example &lt;a href=&quot;https://github.com/ivanseidel/IAMDinosaur&quot; rel=&quot;nofollow noreferrer&quot;&gt;IAMDinosaur&lt;/a&gt;, which trains neural networks through a genetic algorithm - however, the optimal solution is not known. There is no labelling of input data, all it does is calculate the fitness from the score.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2019-07-07T20:41:03.320" LastActivityDate="2019-07-07T20:41:03.320" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3150" PostTypeId="2" ParentId="3139" CreationDate="2017-04-13T11:20:38.947" Score="1" Body="&lt;p&gt;I don't think it is important to build a specialized circuitry for face recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our face recognition is hardwired by evolution. I think it is due to advantages like kin selection and kin altruism. You need to know who your brother is to help him, because he carries 1/4th your dna. So it is irrelevant in the case of building robots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing a name (yours or otherwise) is just correlation, it shouldn't go into a dedicated wetware.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2017-04-13T11:20:38.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3151" PostTypeId="2" ParentId="3120" CreationDate="2017-04-13T15:53:28.063" Score="2" Body="&lt;p&gt;By the way you have explained things above, it seems more like a problem with your code and not the something to do with the environment. The term discrete and continuous is used to define, how the outside environment is acting, rather than how your code is taking its steps. These are some lines from the book, Artificial Intelligence: A Modern Approach:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The discrete/ continuous distinction applies to the state of the environment, to the way time is handled, and to the percepts and actions of the agent. For example, the chess environment has a finite number of distinct states (excluding the clock). Chess also has a discrete set of percepts and actions. Taxi driving is a continuous state and continuous-time problem: the speed and location of the taxi and of the other vehicles sweep through a range of continuous values and do so smoothly over time. Taxi-driving actions are also continuous (steering angles, etc.). Input from digital cameras is discrete, strictly speaking, but is typically treated as representing continuously varying intensities and locations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, continuous or discrete is not something that should be talked about as a problem of the code. It is basically, what an environment is. Your concern with the device is regarding the code. I will suggest that you upload the code on git and ask people to improve it.&lt;br&gt;&lt;br&gt;&#xA;I hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-04-13T15:53:28.063" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3156" PostTypeId="1" CreationDate="2017-04-14T13:35:03.903" Score="49" ViewCount="76259" Body="&lt;p&gt;I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an article where this problem is being investigated, i.e., how many memory cells should one use? I assume it totaly depends on the application and in which context the model is being used, but what does the research say?&lt;/p&gt;&#xA;" OwnerUserId="6645" LastEditorUserId="2444" LastEditDate="2019-04-01T17:20:42.907" LastActivityDate="2020-03-07T16:16:51.233" Title="How to select number of hidden layers and number of memory cells in an LSTM?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;research&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="28" ContentLicense="CC BY-SA 4.0" />
  <row Id="3157" PostTypeId="2" ParentId="1961" CreationDate="2017-04-14T14:18:04.593" Score="2" Body="&lt;p&gt;The terms you are looking for are deeplearning and convolutional neural networks for object detection. Google responds well to these terms.&#xA;From academical point of view you can start from:&lt;br&gt;&#xA;Single shot multibox detector: &lt;a href=&quot;https://arxiv.org/pdf/1512.02325v5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1512.02325v5.pdf&lt;/a&gt;&lt;br&gt;&#xA;Or Faster-RCNN:&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1506.01497.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1506.01497.pdf&lt;/a&gt;&lt;br&gt;&#xA;These are not simple architectures and there are many improvements to them but these give you an idea of the current state of the art methods.&#xA;There are many implementations of both of these networks in deeplearning libraries for python (eg in Tensorflow, PyTorch).  &lt;/p&gt;&#xA;" OwnerUserId="6648" LastActivityDate="2017-04-14T14:18:04.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3158" PostTypeId="2" ParentId="2924" CreationDate="2017-04-14T14:28:26.940" Score="1" Body="&lt;p&gt;We can think of the dropout as of averaging over many networks. Not as a way to mute or denoise your data. Therefore the rule of thumb is the dropout should be applied to higher layers in your network. It is usually applied to fully connected layers (if at all!). It is a common practice to keep 0.5-0.75. neurons active. Personally I prefer L2 much more.&lt;/p&gt;&#xA;" OwnerUserId="6648" LastActivityDate="2017-04-14T14:28:26.940" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3159" PostTypeId="2" ParentId="2303" CreationDate="2017-04-14T14:35:09.737" Score="4" Body="&lt;p&gt;I've read all the papers about PReLU, LeakyReLU (...) and all the claims how it improves this and that but the little dirty secret is: most of the time it doesn't matter at all and you can't go much wrong with ReLU - empirically proven. I've personally tried all of them in many different problems (from training small networks from scratch through changing activations in large pretrained models) My guess is that gradient doesn't die much in any of them and the rest is pretty much irrelevant.&lt;/p&gt;&#xA;" OwnerUserId="6648" LastActivityDate="2017-04-14T14:35:09.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3160" PostTypeId="2" ParentId="3089" CreationDate="2017-04-14T14:47:15.887" Score="10" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In the blog post &lt;a href=&quot;https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building powerful image classification models using very little data&lt;/a&gt;, bottleneck features are mentioned. What are the bottleneck features?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's clearly written in the link you gave &lt;em&gt;the &quot;bottleneck features&quot; from the VGG16 model: the last activation maps before the fully-connected layers&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do they change with the architecture that is used?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sure. The author most likely used a pre-trained model (trained on a large data and now used only as a feature extractor)  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are they the final output of convolutional layers before the fully-connected layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are they called so?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Given the input size to VGG, the feature maps of HxW dimensions are getting twice smaller after every max-pool operation. HxW is the smallest on the last convolutional layer.&lt;/p&gt;&#xA;" OwnerUserId="6648" LastEditorUserId="2444" LastEditDate="2019-06-23T22:10:34.077" LastActivityDate="2019-06-23T22:10:34.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3161" PostTypeId="1" CreationDate="2017-04-14T15:15:44.397" Score="5" ViewCount="432" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the likely AI advancements in the next 5-10 years?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I first want to specify that I have nearly no knowledge about &lt;em&gt;How AI works&lt;/em&gt;. I just have interest to know more and more about it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples of Weak AI at present are like Siri and Cortana, those are pretty interesting! But how high levels is it going to reach (likely) in future years?&lt;/p&gt;&#xA;" OwnerUserId="6649" LastEditorUserId="1671" LastEditDate="2019-01-17T17:48:02.957" LastActivityDate="2019-01-28T14:51:33.270" Title="What are the likely AI advancements in the next 10 years?" Tags="&lt;research&gt;&lt;soft-question&gt;&lt;ai-milestones&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ClosedDate="2020-05-21T12:38:31.237" ContentLicense="CC BY-SA 3.0" />
  <row Id="3162" PostTypeId="2" ParentId="3156" CreationDate="2017-04-14T15:55:12.433" Score="23" Body="&lt;p&gt;Your question is quite broad, but here are some tips.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically for LSTMs, see this Reddit discussion &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/4behuh/does_the_number_of_layers_in_an_lstm_network/&quot; rel=&quot;noreferrer&quot;&gt;Does the number of layers in an LSTM network affect its ability to remember long patterns?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main point is that there is usually no rule for the number of hidden nodes you should use, it is something you have to figure out for each case by trial and error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are also interested in feedforward networks, see the question &lt;a href=&quot;https://stats.stackexchange.com/q/181/82135&quot;&gt;How to choose the number of hidden layers and nodes in a feedforward neural network?&lt;/a&gt; at Stats SE. Specifically, &lt;a href=&quot;https://stats.stackexchange.com/a/136542/82135&quot;&gt;this answer&lt;/a&gt; was helpful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There's one additional rule of thumb that helps for supervised learning problems. You can usually prevent over-fitting if you keep your number of neurons below:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$N_h = \frac{N_s} {(\alpha * (N_i + N_o))}$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; = number of input neurons.   &lt;/li&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$N_o$&lt;/span&gt; = number of output neurons.  &lt;/li&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$N_s$&lt;/span&gt; = number of samples in training data set.  &lt;/li&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; = an arbitrary scaling factor usually 2-10.  &lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;http://www.solver.com/training-artificial-neural-network-intro&quot; rel=&quot;noreferrer&quot;&gt;Others recommend&lt;/a&gt; setting &lt;span class=&quot;math-container&quot;&gt;$alpha$&lt;/span&gt; to a value between 5 and 10, but I find a value of 2 will often work without overfitting. You can think of alpha as the effective branching factor or number of nonzero weights for each neuron. Dropout layers will bring the &quot;effective&quot; branching factor way down from the actual mean branching factor for your network.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;As explained by this &lt;a href=&quot;http://hagan.okstate.edu/NNDesign.pdf#page=469&quot; rel=&quot;noreferrer&quot;&gt;excellent NN Design text&lt;/a&gt;, you want to limit the number of free parameters in your model (i.e. its &lt;a href=&quot;https://stats.stackexchange.com/q/57027/15974&quot;&gt;degree&lt;/a&gt; or the number of nonzero weights) to a small portion of the degrees of freedom in your data. The degrees of freedom in your data is the number samples * degrees of freedom (dimensions) in each sample or &lt;span class=&quot;math-container&quot;&gt;$N_s * (N_i + N_o)$&lt;/span&gt; (assuming they're all independent). So &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a way to indicate how general you want your model to be, or how much you want to prevent overfitting.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For an automated procedure you'd start with an alpha of 2 (twice as many degrees of freedom in your training data as your model) and work your way up to 10 if the error (loss) for your training dataset is significantly smaller than for your test dataset.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2020-03-07T16:16:51.233" LastActivityDate="2020-03-07T16:16:51.233" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3164" PostTypeId="1" AcceptedAnswerId="3165" CreationDate="2017-04-15T09:34:01.557" Score="0" ViewCount="101" Body="&lt;p&gt;I was trying to build an OCR system and heard about ANNs. I am weak at mathematics and statistics and couldn't stick up to reading those massive mathematical documents (research papers or ANN related books). But I kind of figured out that ANN training is all about balancing of weights and biases. Am I right? And please also point me to some docs where I can get help understanding ANNs to use in my OCR system.&lt;/p&gt;&#xA;" OwnerUserId="6661" LastEditorUserId="2444" LastEditDate="2019-04-12T21:38:46.150" LastActivityDate="2019-04-12T21:38:46.150" Title="What is the concept of training a neural network?" Tags="&lt;training&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3165" PostTypeId="2" ParentId="3164" CreationDate="2017-04-15T10:11:04.027" Score="2" Body="&lt;p&gt;Sorry, this is a very broad area. Proper understanding of neural networks requires advanced mathematics. It's not sufficient to say &quot;balancing of weights and biases&quot; because most ML algorithms have weights. You seriously need to grab a book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OCR system itself is also very broad, it includes various object recognition techniques. You haven't even mentioned what you want to detect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to study, try:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://github.com/Elucidation/tensorflow_chessbot&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Elucidation/tensorflow_chessbot&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a well-documented OCR example for chess pieces. The project uses both regression and convolutional neural network.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-04-15T10:11:04.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3166" PostTypeId="2" ParentId="21" CreationDate="2017-04-15T18:00:25.267" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Usually you keep track of training loss and validation loss and apply proper regularization technique (such as L1, L2, &lt;a href=&quot;https://jmlr.org/papers/v15/srivastava14a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dropout&lt;/a&gt;, &lt;a href=&quot;http://proceedings.mlr.press/v28/wan13.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DropConnect&lt;/a&gt;, etc.).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The more interesting technique is to observe your validation loss with respect to the number of parameters in the network (often controlled by the number of layers/feature maps). If the validation starts dropping with raising your model's complexity, then your optimization might be bad or simply the model remembers all of the training samples and overfits badly.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6648" LastEditorUserId="2444" LastEditDate="2021-01-10T01:03:39.270" LastActivityDate="2021-01-10T01:03:39.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3167" PostTypeId="2" ParentId="2922" CreationDate="2017-04-16T03:56:20.750" Score="5" Body="&lt;p&gt;First of all, I would like to point out the main differences between knowledge base and (Deep) machine learning, specially when the main focus is on &quot;AI&quot; not &quot;Data Science&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;NNs are like a black box; Even if they learn a dataset and gain the power of generalization over the problem domain, you'd never know how they are working. if you scrutinize the details of the developed model, all you see are digits, weights, poor and strong connections and transform functions. the &quot;feature extraction&quot; step before the training phase literally tells you: &quot;hey human, enough with your complicated world, let's start zeros and ones&quot;. In the case of DL, it is worse! we do not even see what the selected and effective features are. I'm not a DL expert but as much as I know, DL's black box is darker! But knowledge bases are written in a human-friendly language. after a knowledge accumulation phase, you could see all the connections between the entities, and more important, you could interpret those connections. if you cut a wire in a knowledge base, your model will lose just a bit of its power, and you know what exactly it will lose; for example disconnecting the &quot;Pluto&quot; node from the &quot;solar system&quot; node, will tell your model what deGrasse Tyson told us. but in a ML model, this might turn it into a pure useless one: what happens if you manipulate the connection between the neuron number 14 and 47 in a NN model used to predict which planets belong to the solar system?!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ML models are merely an inscription of the data. They do not have the power of inference, and they don't give you one. knowledge base is on the other hand capable of inference from the prior knowledge as you indicated in your question. It is shown that DL models that have been trained with say image classification data, could also be applied to voice detection problem. But this doesn't mean DL models could apply its prior knowledge in the domain of images to the domain of voices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You need kilos of data for traditional ML algorithms and tons of data for DL ones. but a single instance of a dataset will create a meaningful knowledge base for you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are two main research topics in NLP: machine translation and question answering. Practically it has been shown that DL works significantly with machine translation problems but acts kind of stupid in question answering challenge, specially when the domain of topics covered in the human-machine conversation is broad. Knowledge bases are no good choice for machine translation but are probably the key to a noble question answering machine. Since what matters in machine translation is only the translated version of a text (and I don't care how on earth has the machine done that as far as it is true) but in question answering problem, I don't need a parrot who repeats the same information I gave it to him, but an intelligent creature who gives me &quot;apple is eatable&quot; after I tell him &quot;apple is a fruit&quot; and &quot;all fruits are eatable&quot;. ML models are used to elicit underneath patterns from the dataset (translation) while knowledge bases are used to extend the domain of knowns. ML models nitpick, KBs explore!&lt;/p&gt;&#xA;" OwnerUserId="6258" LastEditorUserId="6258" LastEditDate="2017-04-16T04:02:48.303" LastActivityDate="2017-04-16T04:02:48.303" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3169" PostTypeId="1" CreationDate="2017-04-16T17:20:17.480" Score="3" ViewCount="216" Body="&lt;p&gt;I have a group of structures in a program that are very specific on their meaning, eg. this is a piece of code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;randomItem = objects.concept.random(&quot;buyable&quot;)&#xA;idea.example(objects.concept.random(&quot;family&quot;, &quot;friend&quot;)).does({&#xA;    action: &quot;go&quot;,&#xA;    target: object.concent.random(&quot;shop&quot;)&#xA;}).then({&#xA;    action: &quot;buys&quot;,&#xA;    target: randomItem,&#xA;    several: true&#xA;}).then({&#xA;    question: true,&#xA;    action: &quot;know&quot;,&#xA;    property: &quot;amount&quot;,&#xA;    target: randomItem,&#xA;    several: true&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have worked with natural language parsers before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I go and transform this to Natural Language (the other way around), is there any way or method; I have logical structures in which I know who is the subject, what the verb and target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which methods can I use to generate language from this?&lt;/p&gt;&#xA;" OwnerUserId="6680" LastEditorUserId="2444" LastEditDate="2020-09-13T15:35:42.567" LastActivityDate="2020-09-13T15:35:42.567" Title="How can I generate natural language sentences given logical structures that contain the subject, verb and target?" Tags="&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3172" PostTypeId="1" CreationDate="2017-04-17T10:06:04.770" Score="4" ViewCount="906" Body="&lt;p&gt;I recently read about &lt;a href=&quot;https://ai.googleblog.com/2017/04/federated-learning-collaborative.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;federated learning&lt;/a&gt; introduced by Google, but it seems to be like &lt;a href=&quot;https://en.wikipedia.org/wiki/Edge_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;edge computing&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between edge computing and federated learning?&lt;/p&gt;&#xA;" OwnerUserId="6687" LastEditorUserId="2444" LastEditDate="2019-09-20T12:27:21.163" LastActivityDate="2019-09-24T13:05:27.047" Title="What is difference between edge computing and federated learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;hardware&gt;&lt;federated-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3175" PostTypeId="1" CreationDate="2017-04-17T19:08:21.733" Score="8" ViewCount="320" Body="&lt;p&gt;I recently read &lt;a href=&quot;http://www.wired.co.uk/article/machine-learning-bias-prejudice&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article about how artificial intelligence replicates human stereotypes&lt;/a&gt; when applied to biased datasets.&lt;/p&gt;&#xA;&lt;p&gt;What are examples of techniques to prevent bias (and stereotypes) in artificial intelligence (in particular, machine learning) systems?&lt;/p&gt;&#xA;" OwnerDisplayName="user6698" LastEditorUserId="2444" LastEditDate="2020-11-17T13:52:19.197" LastActivityDate="2020-11-17T13:52:19.197" Title="What are examples of techniques to prevent bias in artificial intelligence systems?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;social&gt;&lt;ai-safety&gt;&lt;algorithmic-bias&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3176" PostTypeId="1" CreationDate="2017-04-17T20:00:57.397" Score="13" ViewCount="1049" Body="&lt;p&gt;I read a really interesting article titled &lt;a href=&quot;http://www.joshworth.com/stop-calling-in-artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Stop Calling it Artificial Intelligence&quot;&lt;/a&gt; that made a compelling critique of the name &quot;Artificial Intelligence&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The word intelligence is so broad that it's hard to say whether &quot;Artificial Intelligence&quot; is really intelligent. Artificial Intelligence, therefore, tends to be misinterpreted as replicating human intelligence, which isn't actually what Artificial Intelligence is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Artificial Intelligence isn't really &quot;artificial&quot;. Artificial implies a fake imitation of something, which isn't exactly what artificial intelligence is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What are good alternatives to the expression &quot;Artificial Intelligence&quot;? (Good answers won't list names at random; they'll give a rationale for why their alternative name is a good one.)&lt;/p&gt;&#xA;" OwnerDisplayName="user6698" LastEditorUserId="2444" LastEditorDisplayName="user6698" LastEditDate="2019-06-23T12:30:38.313" LastActivityDate="2019-06-24T01:12:08.950" Title="What are good alternatives to the expression &quot;Artificial Intelligence&quot;?" Tags="&lt;philosophy&gt;&lt;terminology&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3177" PostTypeId="2" ParentId="3130" CreationDate="2017-04-17T22:05:30.463" Score="3" Body="&lt;p&gt;An AI takes the decision based on the output of their &lt;strong&gt;utility function&lt;/strong&gt;. This is just a fancy word for the calculations that AI perform to compare profit and loss of taking a certain decision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is always a tight analogy between an AGI and a human. You can juxtapose the utility function to how we make decisions by considering marginal profits of doing something over the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your question. The answer is a big &lt;strong&gt;NO&lt;/strong&gt;. This is because an AI (or a human) never takes an action that gives a low score on its utility function or is against the fundamental view of the machine (or human). &lt;em&gt;AI only cares about the utility function and goal state (plus the instrumental goals), nothing else&lt;/em&gt;. Our AI outside the box has been created with the purpose of not opening the box. For it, the action of opening the box has a very low score, or a negative score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you might be wondering why an AI can't be convinced of doing something that gives a low score in its utility function. Keep reading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this for a second. Suppose an AGI was created, assigned with the task of copying the handwriting of others and improving its own writing. You can think of it as a smart writing hand. Now, what this AI might come up with is that, in order to practice writing, it needs more pages and thus it needs to cut more trees for that. No matter what you do, AI will not stop from cutting more and more trees. It might even replicate itself into machines that will cut trees for him. Now, the only thing that can be done is to turn off the AGI. But, a smart machine would have known the possibility of being turned off and thus will have transferred itself to many other machines over the globe. The important question to ask here is, why it is doing all this? It is not because the machine wants to live, as we humans want to. The only reason why it wants to live is to fulfill its goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You simply can't change the fundamental view of the machine. The change can result in, the machine not able to attain its goal or not to that extend (optimal profit). This is the reason why you can't convince a machine to do something that it was asked not to do as its primary task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You, as a person, are living a life now, and have some fundamental beliefs. Let us consider that you believe in not killing someone. Now, suppose that I give you a pill, and tell you that after taking this pill it will rewire your brain and you will kill the first four people you see. But, after that, you will achieve pure satisfaction and happiness. Now you will definitely not take that pill as it conflicts with something that you believe in now. Also, you will try your best and fight back not to take that pill. The same thing applies to an AGI too. It doesn't matter what your future version will feel or attain after rewiring of the brain (changes in code), it is what and who you are now matters. This video link will &lt;a href=&quot;https://www.youtube.com/watch?v=4l7Is6vOAOA&amp;amp;t=4s&quot; rel=&quot;nofollow noreferrer&quot;&gt;help&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope it answers your question. There are lots of things to consider here. I have assumed a few things and tried to answer according to that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one more thing. &lt;strong&gt;We don’t tell the machines how to do something, instead, we only tell them what to do&lt;/strong&gt; (at least in the case of an AI). This is because sometimes we don't know the optimal way of solving certain problems. In the case of your question, we don't know what these two machines will do or say to each other. It will be a very interesting thing to hear or watch.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2020-06-07T20:58:03.327" LastActivityDate="2020-06-07T20:58:03.327" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3178" PostTypeId="2" ParentId="3176" CreationDate="2017-04-18T09:58:04.337" Score="1" Body="&lt;p&gt;Google defines 'artificial' as something created by humans rather than occurring naturally so I wouldn't quite say that it's so bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the question however, you could perhaps say &quot;smart machines&quot; since that's what they essentially are these days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence is a very broad term, pre-dating modern AI, simple things such as mechanical wooden robots were considered Artificial Intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Timeline_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Timeline_of_artificial_intelligence&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6709" LastActivityDate="2017-04-18T09:58:04.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3179" PostTypeId="2" ParentId="3169" CreationDate="2017-04-18T10:19:41.843" Score="1" Body="&lt;p&gt;It might be simpler to generate the language output yourself since you've already got a concrete concept structure in code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you'll also want to be aware of potential future applications such as parsing your own output back to the inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, you could investigate Markov Chains or Google around a bit for Natural Language Processing and Natural Language Generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/33068943/library-for-generating-natural-language-verbs-in-javascript&quot;&gt;https://stackoverflow.com/questions/33068943/library-for-generating-natural-language-verbs-in-javascript&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6709" LastEditorUserId="-1" LastEditDate="2017-05-23T12:39:33.010" LastActivityDate="2017-04-18T10:19:41.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3181" PostTypeId="2" ParentId="3176" CreationDate="2017-04-18T23:34:00.960" Score="5" Body="&lt;p&gt;Artificial is said to derive from the Latin word &quot;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=artificium&amp;amp;la=la#lexicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificium&lt;/a&gt;&quot; which connotes ideas such as crafting.  Thus, artificial is a correct usage, and algorithms can be regarded as &quot;artifacts&quot; in the context of information as opposed to physical manifestation of information (i.e. matter).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I agree that the use of artificial is problematic in that, should strong Artificial General Intelligence ever be achieved, there is a stigma to &quot;artificiality&quot; that could have implications regarding personhood.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal feeling is that we should be using:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Algorithmic Intelligence&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which this is functional definition, and therefore more meaningful than &quot;artificial&quot;.  Additionally, &quot;algorithmic&quot; is a neutral term, and provides a very accurate description of what these systems are. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In terms of what is considered &quot;intelligent&quot;, you may want to look at the concept of &lt;a href=&quot;https://en.wikiquote.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bounded Rationality&lt;/a&gt;.  There is no hard definition of &quot;intelligence&quot;, just degrees of optimality in regard to decision making in a condition of uncertainty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because this is subjective for any problem that is not solved, modifiers are utilized, and thus we refer to AI as &quot;strong&quot; or &quot;weak&quot;. These terms are also used to describe the degree to which certain types of problems (for instance &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#Overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;a non-chance, perfect information game&lt;/a&gt; like Checkers) has been solved. &lt;a href=&quot;https://en.wikipedia.org/wiki/Complexity_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Complexity theory&lt;/a&gt; will shed more light on this concept. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more insight on &quot;artificial&quot;, you might find this &lt;a href=&quot;https://philosophy.stackexchange.com/questions/41237/is-protagoras-the-philosophical-root-of-the-turing-test&quot;&gt;question on the philosophical origin of the Turing Test&lt;/a&gt; interesting, because it partly involves the meaning of a &quot;thing&quot;. (There were multiple words for this in Ancient Greek.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-29T17:57:14.513" LastActivityDate="2018-11-29T17:57:14.513" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3182" PostTypeId="2" ParentId="2955" CreationDate="2017-04-19T09:13:15.440" Score="1" Body="&lt;p&gt;How did it became diluted? if by mutations then it has &quot;evolved&quot; or neural plasticity then it &quot;learned&quot;, Alzheimer then it got &quot;sick&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by analogy, you took too much psycho-active drug and grew up learning alot of stuff and adjusting your moral compass drastically, then got Alzheimer and now you don't know who You are, you are not the same person any more, but you didn't die. So you are still you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only zombies, who physically die first, then get re-animated, then yes, they are pretty much dead, also depends on the movie director.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2017-04-19T09:13:15.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3183" PostTypeId="2" ParentId="3137" CreationDate="2017-04-19T13:56:59.013" Score="2" Body="&lt;p&gt;I couldn't add a comment because of my low reputation but you can check this. It is about the state space. &lt;a href=&quot;https://math.stackexchange.com/questions/485752/tictactoe-state-space-choose-calculation&quot;&gt;https://math.stackexchange.com/questions/485752/tictactoe-state-space-choose-calculation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6019" LastActivityDate="2017-04-19T13:56:59.013" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3184" PostTypeId="2" ParentId="2955" CreationDate="2017-04-19T17:16:48.117" Score="4" Body="&lt;p&gt;I find the concept of the a &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing machine&lt;/a&gt; useful. In one dimension, everything is a string. All of the parts that are &quot;not you&quot; are merely a substrate, a medium for the program your_mind runs on top of. The you, your identity, the &quot;metaphysical&quot; component we think of as the mind, is a result of running the algorithm that is your_mind on the bioware of your body, or, the hardware (technically, &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Wetware_computer&quot; rel=&quot;nofollow noreferrer&quot;&gt;wetware&lt;/a&gt;&quot;.) So what we're really talking about is the software, and in that light I might use: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Translator_(computing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Translation&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;because the software is being translated to execute in a new environment, or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/System_migration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Migration&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;as in moving software from one system to another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Philip Dick wrote a philosophical narrative, not technically sci-fi, called &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Transmigration_of_Timothy_Archer&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Transmigration of Timothy Archer&lt;/a&gt; which is about identity moving between bodies. In a rare departure from his usual work about AI and the effects of a technological society on the human spirit, this book looks at the question of identity in the context of the soul, which opens up all kinds of philosophical questions surrounding the type of technology we're speculating on, particularly in relation to the self.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I value artistic insight, and Phillip K. is considered quite prescient, so perhaps&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://en.oxforddictionaries.com/definition/us/transmigrate&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transmigration&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;is most appropriate, as it carries both metaphysical and information technology meanings.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Re death by dissolution, it's worthwhile to look at the &lt;a href=&quot;https://www.etymonline.com/word/dissolve&quot; rel=&quot;nofollow noreferrer&quot;&gt;etymology of dissolve&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;late 14c. (transitive and intransitive) &quot;to break up&quot; (of material substances), from Latin &lt;em&gt;dissolvere&lt;/em&gt; &quot;to loosen up, break apart,&quot; from &lt;em&gt;dis-&lt;/em&gt; &quot;apart&quot; (see dis-) + &lt;em&gt;solvere&lt;/em&gt; &quot;to loosen, untie,&quot; from PIE *se-lu-, from reflexive pronoun *s(w)e- (see idiom) + root *leu- &quot;to loosen, divide, cut apart.&quot; Meaning &quot;to disband&quot; (an assembly) is early 15c. Related: Dissolved; dissolving.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you're on the right track with &lt;strong&gt;&lt;a href=&quot;https://www.etymonline.com/word/dilute?ref=etymonline_crossreference&quot; rel=&quot;nofollow noreferrer&quot;&gt;dilution&lt;/a&gt;&lt;/strong&gt;, certainly per the modern usage, but it may be possible to get more precise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not about breaking up, or washing away (except metaphorically); rather, it's about &lt;a href=&quot;https://www.etymonline.com/word/minimum?ref=etymonline_crossreference&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimization&lt;/a&gt; as in the &lt;strong&gt;&lt;a href=&quot;https://www.etymonline.com/word/diminish&quot; rel=&quot;nofollow noreferrer&quot;&gt;diminishment&lt;/a&gt;&lt;/strong&gt; of the original software kernel (the self) in relation to an expanding algorithm.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;early 15c., from merger of two obsolete verbs, &lt;em&gt;diminue&lt;/em&gt; and &lt;em&gt;minish&lt;/em&gt;. &lt;em&gt;Diminue&lt;/em&gt; is from Old French &lt;em&gt;diminuer&lt;/em&gt; &quot;make small,&quot; from Latin &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Ddiminuo&quot; rel=&quot;nofollow noreferrer&quot;&gt;diminuere&lt;/a&gt;&lt;/em&gt; &quot;break into small pieces,&quot; variant of &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Ddeminuo&quot; rel=&quot;nofollow noreferrer&quot;&gt;deminuere&lt;/a&gt;&lt;/em&gt; &quot;lessen, diminish,&quot; from &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Dde2&quot; rel=&quot;nofollow noreferrer&quot;&gt;de&lt;/a&gt;&lt;/em&gt;- &quot;completely&quot; + &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Dminuo&quot; rel=&quot;nofollow noreferrer&quot;&gt;minuere&lt;/a&gt;&lt;/em&gt; &quot;make small&quot; (from PIE root &lt;a href=&quot;https://www.etymonline.com/word/*mei-&quot; rel=&quot;nofollow noreferrer&quot;&gt;*mei- (2)&lt;/a&gt; &quot;small&quot;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Old French &lt;em&gt;diminuer&lt;/em&gt; is apt, as is the Latin &lt;em&gt;deminuere&lt;/em&gt; and &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Ddeminutio&quot; rel=&quot;nofollow noreferrer&quot;&gt;deminuo&lt;/a&gt;&lt;/em&gt;, which also carries a meaning of &quot;civil death&quot; and &quot;&lt;a href=&quot;https://www.etymonline.com/word/abatement&quot; rel=&quot;nofollow noreferrer&quot;&gt;abatement&lt;/a&gt;&quot;. Abatement carries a meaning of &lt;a href=&quot;http://www.dictionary.com/browse/mitigate&quot; rel=&quot;nofollow noreferrer&quot;&gt;mitigation&lt;/a&gt;, which can be defined fundamentally as &quot;lessening of effect,&quot; in this case of the kernel of the original self in relation to the new aggregate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be useful to think of it as a ratio 1/ℵ, with the self as the 1. Important to note the ratio is not literal--each number represent an aggregation of functions we call &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_program&quot; rel=&quot;nofollow noreferrer&quot;&gt;programs&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's how I'd think of it mathematically, but metaphysically, &lt;a href=&quot;https://ai.stackexchange.com/a/3519/1671&quot;&gt;Nat&lt;/a&gt; nails it by referencing &lt;a href=&quot;https://en.wikipedia.org/wiki/Analytical_psychology&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jung&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ego_death&quot; rel=&quot;nofollow noreferrer&quot;&gt;death of the ego&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.etymonline.com/word/ego&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ego comes from&lt;/a&gt; the &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Dego&quot; rel=&quot;nofollow noreferrer&quot;&gt;Latin noun for the self&lt;/a&gt; (I, me) which can also be plural (we, us).  It's also fun to note that the Latin verb of being is &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Dsum&quot; rel=&quot;nofollow noreferrer&quot;&gt;sum&lt;/a&gt;, because in English, sum has a mathematical meaning of an aggregate considered as a whole. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because we're talking about what it means to be a person, but we also want to connote a function (a process of relative minimization) I am thinking:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://en.oxforddictionaries.com/definition/us/depersonalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Depersonalization&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(noun) the action of divesting someone or something of human characteristics or individuality.&lt;br&gt;&lt;sub&gt;SOURCE: Google&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The fun part is that there's a legal definition of &lt;a href=&quot;https://www.etymonline.com/word/person&quot; rel=&quot;nofollow noreferrer&quot;&gt;person&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In legal use, &quot;corporate body or corporation having legal rights,&quot; 15c., short for person aggregate (c. 1400), person corporate (mid-15c.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which I think applies (loosely) to the aggregation of function that comprises the program we call the self, and the greater aggregation of functions that form a new self.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because here we're talking about the loss of the original individual, depersonalization as opposed to repersonalization as a corporate entity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, I might propose:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Deindividualization&lt;/strong&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;with the definition &quot;a loss in individual identity within a group&quot; (also known as &lt;a href=&quot;http://a%20loss%20in%20individual%20identity%20when%20a%20person%20is%20in%20a%20group.&quot; rel=&quot;nofollow noreferrer&quot;&gt;deindividuation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Charles Stross wrote about this in &lt;a href=&quot;https://en.wikipedia.org/wiki/Accelerando&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accellerando&lt;/a&gt;, where a character chooses to be subsumed by a group intelligence as the only means of escaping a swarm of autonomous lawsuits;)&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-12T22:40:58.843" LastActivityDate="2018-05-12T22:40:58.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3186" PostTypeId="2" ParentId="3175" CreationDate="2017-04-20T19:52:15.220" Score="0" Body="&lt;p&gt;The field of statistical psychology offers many methodologies to remove bias from datasets, or rather gather a dataset with minimal unknown biases. This will be the responsibility of the programmer. An AI that learns from datasets will not be able to find bias in those datasets.&lt;/p&gt;&#xA;" OwnerUserId="6756" LastEditorUserId="2444" LastEditDate="2020-11-17T13:50:09.820" LastActivityDate="2020-11-17T13:50:09.820" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3187" PostTypeId="1" AcceptedAnswerId="3192" CreationDate="2017-04-21T05:38:50.523" Score="3" ViewCount="347" Body="&lt;p&gt;From &lt;a href=&quot;https://futurism.com/elon-musk-an-ai-attack-on-the-internet-is-only-a-matter-of-time/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Futurism.com&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Musk indicates that internet infrastructure is &quot;particularly susceptible&quot; to a method called gradient descent algorithm, a mathematical problem-solving process. Bad news is, AI is excellent at doing gradient descents, which can become devastating digital weaponry.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A screenshot of the tweet is provided below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BTQco.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BTQco.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He is an intelligent man, so why would he say something like this? Does Musk know what gradient descent is?&lt;/p&gt;&#xA;" OwnerUserId="3874" LastEditorUserId="2444" LastEditDate="2020-05-18T14:51:34.360" LastActivityDate="2020-05-18T17:50:39.993" Title="Does Musk know what gradient descent is?" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3189" PostTypeId="1" CreationDate="2017-04-21T09:50:07.307" Score="4" ViewCount="169" Body="&lt;p&gt;Assuming I have a quite advanced AI with consciousness which can understand the basics of electronics and software structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will it ever be able to understand that its consciousness is just some bits in memory and threads in an operating system?&lt;/p&gt;&#xA;" OwnerUserId="6482" LastEditorUserId="2444" LastEditDate="2019-11-11T21:14:37.723" LastActivityDate="2019-11-11T21:14:37.723" Title="Will an AI ever understand its own functionality?" Tags="&lt;philosophy&gt;&lt;artificial-consciousness&gt;&lt;self-awareness&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3190" PostTypeId="1" CreationDate="2017-04-21T10:51:35.760" Score="5" ViewCount="99" Body="&lt;p&gt;While reading &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the book on neural networks&lt;/a&gt; by Michael Nielson, I had a problem understanding equation (BP3), which is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial C}{\partial b_{j}^{l}}=\delta_{j}^{l} \tag{BP3}\label{BP3},&#xA;$$&lt;/span&gt;&#xA;which can be translated to plain English as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The change in cost, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, with respect to the bias of a neuron, &lt;span class=&quot;math-container&quot;&gt;$b_{j}^{l}$&lt;/span&gt;, is equal to the error in that neuron, &lt;span class=&quot;math-container&quot;&gt;$\delta_{j}^{l}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why would this be the case?&lt;/p&gt;&#xA;" OwnerUserId="6661" LastEditorUserId="2444" LastEditDate="2020-12-29T15:03:15.990" LastActivityDate="2020-12-29T15:03:15.990" Title="Why is the change in cost wrt bias in neural network equal to error in the neuron?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;math&gt;&lt;calculus&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3192" PostTypeId="2" ParentId="3187" CreationDate="2017-04-21T15:46:23.997" Score="5" Body="&lt;p&gt;I think Musk was using the terminology correctly though perhaps with hyperbole. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time.  He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a toolbox of targets and hacks and using well-known machine-learning techniques to optimize the effectiveness of using the attacks in combination to take down the internet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gradient descent itself isn't harmful; it is using gradient descent to train a botnet that could theoretically result in a really devastating attack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also don't think he is necessarily implying that there is an actual machine learning system at work; rather that the hackers were training themselves with a methodology analogous to gradient descent.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2444" LastEditDate="2020-05-18T14:54:29.707" LastActivityDate="2020-05-18T14:54:29.707" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3193" PostTypeId="2" ParentId="3189" CreationDate="2017-04-21T15:52:44.790" Score="5" Body="&lt;p&gt;This is a great question, elements of which I have also been pondering on, though we are very far from being able to actually wrestle with it algorithmically.  This question raises all kinds of metaphysical questions (Kant himself showed that pure reason is not sufficient for all questions, but I'm going to avoid that rabbit hole and focus on the mechanics of your question.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Consciousness: This is distinct from self-awareness, and fundamentally, may be said to require only awareness of &lt;em&gt;something&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consciousness, most scientists argue, is not a universal property of all matter in the universe. Rather, consciousness is restricted to a subset of animals with relatively complex brains. The more scientists study animal behavior and brain anatomy, however, the more universal consciousness seems to be. A brain as complex as the human brain is definitely not necessary for consciousness. &lt;br&gt;&#xA;  &lt;sub&gt;Source: &lt;a href=&quot;https://blogs.scientificamerican.com/brainwaves/does-self-awareness-require-a-complex-brain/&quot; rel=&quot;noreferrer&quot;&gt;Scientific American &quot;Does Self-Awareness Require a Complex Brain?&quot;&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, an automata that receives input may be said to be consciousness, with the caveat that this idea is probably still considered radical.  The key is distinguishing mere &quot;consciousness&quot; from much more complex concepts such as self-awareness.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Self-Awareness: the holy grail.  This is the idea that a set of elements, such as a human organism, is aware of itself. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But this is sticky, because automata that use &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;noreferrer&quot;&gt;Machine Learning&lt;/a&gt; are &quot;aware&quot; of themselves in that the may modify their &quot;thought&quot; process and even their &quot;physical&quot; structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But ML systems are certainly not self-aware in the human sense.  A question might be, is this simply a function of these systems not being full &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;noreferrer&quot;&gt;Algorithmic General Intelligences&lt;/a&gt;, or is there more to it?  If there is more to it, is it strictly a &lt;a href=&quot;https://en.wikipedia.org/wiki/Metaphysics&quot; rel=&quot;noreferrer&quot;&gt;metaphysical&lt;/a&gt; question, or can an answer be derived through purely rational means?  Even if the latter were the case, there is still the problem of subjectivity, as in: &quot;Is the automata truly self-aware or is it just mimicking self-awareness?&quot; which brings us back to the metaphysical question of &quot;Is there a difference?&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However,  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;If there were a full Algorithmic General Intelligence that had consciousness equatable with human consciousness, that was aware, and even able to work with the basic components of it's &lt;a href=&quot;https://en.oxforddictionaries.com/definition/corpus&quot; rel=&quot;noreferrer&quot;&gt;corpus&lt;/a&gt;*, it would certainly be able to grasp that it's consciousness is a function of the &quot;bits and bytes&quot;, just as a human is aware we are &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Soft_Machine&quot; rel=&quot;noreferrer&quot;&gt;soft machines&lt;/a&gt;, and that our consciousness is a function of our bodies and minds.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I intentionally use corpus because it relates both to text (which may be code or even a string of bits in its most reduced form, per the concept of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;noreferrer&quot;&gt;Turing Machine&lt;/a&gt;) and also has an anatomical meaning, as in the body of an organism.  &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=corpus&amp;amp;la=la#lexicon&quot; rel=&quot;noreferrer&quot;&gt;Corpus comes from the Latin&lt;/a&gt; and the extension of its meaning to include matter-as-information is modern.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-04-21T15:52:44.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3194" PostTypeId="1" CreationDate="2017-04-21T16:40:24.430" Score="6" ViewCount="984" Body="&lt;p&gt;As titled, is there such thing as perfect play (or at least &quot;perfectly optimal&quot;) in a game with incomplete information? Or at least a proof as to show why there cannot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively (and seemingly obviously), the answer would be a resounding no, since the agent would be likely be forced to pick between &quot;lottery events&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in practice (using competitive video games as an analogy), we'd see that players would stick to a meta-game that is well equipped to defend against a majority of events that might happen, given incomplete information. Of course the response to that would be that there probably exists a &quot;hard-counter&quot; for any given meta-game, but if it is indeed the case that the meta-game is the &quot;most-optimal&quot; it probably is the case also that such a hard counter puts the player in an unfavourable position most of the time, thus the &quot;hard-counter&quot; itself is not optimal. Thus we'd likely see that any given first encounter players would still stick to their &quot;optimal meta-game&quot; rather than a hard counter of their optimal play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more rigour analogy would be to ask: &quot;Under Hofstadter's notion of superrationality, how would agents play information incomplete games&quot;, but I couldn't find any readings on trying to import the notion of super-rationality into information incomplete games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively: is there such thing as a &quot;perfectly optimal meta-game&quot;?&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="1671" LastEditDate="2017-09-25T17:46:36.480" LastActivityDate="2017-09-25T17:46:36.480" Title="Perfect play in information incomplete games" Tags="&lt;research&gt;&lt;philosophy&gt;&lt;game-theory&gt;&lt;incomplete-information&gt;" AnswerCount="3" CommentCount="10" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3195" PostTypeId="2" ParentId="3194" CreationDate="2017-04-21T17:17:52.887" Score="4" Body="&lt;p&gt;This may be an evolving answer, because the question is, in some sense, a (useful) rabbit hole. I apologize if I don't go deeply into meta-games per se, as it's a little outside of my scope, which is non-chance games of perfect information, but I think it's worthwhile to think about the underlying problem of indeterminacy in relation to games in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bounded Rationality&lt;/a&gt;* is a useful concept because it pre-supposes a condition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational intractability&lt;/a&gt;. Computational intractability can be introduced into games in several forms:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Complexity&lt;/li&gt;&#xA;&lt;li&gt;Hidden Information&lt;/li&gt;&#xA;&lt;li&gt;Randomness (&quot;quantum&quot; indeterminacy)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;[For more details on my use of &quot;quantum&quot; in regards to randomness, see &lt;a href=&quot;http://www.deterministicgames.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deterministic Games&lt;/a&gt;.]&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying purpose of game theory is to determine &quot;optimal&quot; strategies for any given problem.  I put optimal in quotes because optimality is a spectrum, and subjective in a condition of computational intractability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, we cannot know if &lt;a href=&quot;https://www.scientificamerican.com/article/how-the-computer-beat-the-go-master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; plays optimally, only that it played &lt;em&gt;more optimally&lt;/em&gt; than Lee Sedol in 4 out of 5 games.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is distinct from &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#Solved_games&quot; rel=&quot;nofollow noreferrer&quot;&gt;strongly solved games&lt;/a&gt; such as tic-tac-toe, where we can know with total certainty that a choice is optimal, because the problem of tic-tac-toe is computationally tractable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the confusion may be semantic, because the concepts are subtle and profound, and require language, what TS Eliot might have called &quot;the intolerable wrestle with words and meanings.&quot; (For instance, I used hidden information above to avoid having to distinguish between incomplete and imperfect information.) &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Perfect Play is generally defined as a strategy that leads to the best possible outcome for a participant, regardless of the choices of the opponent.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt; is of central importance, and provided the foundation for &lt;a href=&quot;https://en.wikipedia.org/wiki/John_von_Neumann#Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;game theory&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in games with incomplete information, whether &quot;deterministic&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Battleship_(game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Battleship&lt;/a&gt;) or involving &quot;quantum indeterminacy&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prisoner's Dilemma&lt;/a&gt;), there are optimal strategies. For &lt;a href=&quot;https://en.wikipedia.org/wiki/Simultaneous_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;simultaneous games&lt;/a&gt; such as &lt;a href=&quot;http://www.bryanbruns.com/2x2table.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dilemma and all of the numerous extensions&lt;/a&gt; minimax is used.  In Battleship, there are &lt;a href=&quot;http://www.datagenetics.com/blog/december32011/&quot; rel=&quot;nofollow noreferrer&quot;&gt;at least three strategies of increasing optimality&lt;/a&gt;, and although there doesn't appear to be a strategy that can yield P &gt; .5, if one player employs a more optimal strategy, they will win in aggregate. Even &lt;a href=&quot;https://arstechnica.com/science/2014/05/win-at-rock-paper-scissors-by-knowing-thy-opponent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rock, Paper, Scissors seems to have an optimal strategy&lt;/a&gt;, which blows my mind, and carries the caveat that &lt;a href=&quot;https://arxiv.org/pdf/1404.5199v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;I need to look into it more.&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Thus, perfect play, as defined, is certainly achievable, but does not necessarily connote (objectively) optimal choices, which is a little confusing, because &quot;perfect&quot; implies objectivity, a condition which is only possible in regard to &lt;a href=&quot;http://www.doe.carleton.ca/~pavan/Public/Courses_files/03%20computational_complexity.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tractable problems&lt;/a&gt;.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is also important to note that there may not be a &quot;winning&quot; strategy in the sense of being better off than the opponent, and in this condition, perfect or optimal play is mitigation of loss.   &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*In terms of incomplete information games specifically, I think there's a case for extending the concept of Bounded Rationality is extended to include information that cannot be observed or &quot;known&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Colloquially, this would include the &quot;unknowns&quot; (both known and unknown) and the &quot;unknowable&quot; (quantum indeterminacy and superpositions).&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-04-22T20:58:14.973" LastActivityDate="2017-04-22T20:58:14.973" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3196" PostTypeId="2" ParentId="3072" CreationDate="2017-04-22T03:18:20.370" Score="2" Body="&lt;p&gt;AlphaGo used data from the KGS Go Server, which had 160,000 games and 29 million board/next-move pairs. But crucially, after it was trained on the dataset, AlphaGo was trained through self-play, so its competence shouldn't be measured strictly in terms of its database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not 100% sure how Deep Blue worked, but I think it was a mix of 1. a &quot;book&quot; of opening theory 2. explicitly coded board evaluation functions 3. a &quot;book&quot; of endgames. So there isn't a &quot;database&quot; in your traditional &quot;ML by big data&quot; sense. But in any case, I would assume the bulk of the work is done by the evaluation function, so again its strength cannot be measured in terms of if its database.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-04-22T03:18:20.370" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3197" PostTypeId="2" ParentId="3175" CreationDate="2017-04-22T03:48:54.150" Score="3" Body="&lt;p&gt;It's important to note that ultimately, the statistical methods we currently use in ML research are just that: statistical methods. So when they show some &quot;bad behaviour&quot; it's not because of problems with the statistical methods, but with the data we give it. But if the data we give it are as &quot;genuine and unfiltered&quot; as it gets, then it probably shows something about us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a cognitive science perspective, its probably the case that the same heuristics and biases that creates stereotypes are also the ones that make us powerful agents(note the similarity between categories and stereotypes), so at least at this moment its unclear how we can segregate desired from undesired behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To combine the points mode above, it seems we can only either: 1. Remove &quot;bad content&quot; by curating the data by hand or by some metric that we don't know of yet 2. Accept that our methods will produce AI as &quot;bad as we are&quot; cause that's what we are, and let it operate under the knowledge that it might produce undesired behavior sometimes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless we have some crazy new theory of mind that we can begin to analyze this in a more rigour manner, it seems like there is no clear cut solution.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2017-06-27T20:44:17.150" LastActivityDate="2017-06-27T20:44:17.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3198" PostTypeId="2" ParentId="197" CreationDate="2017-04-22T04:23:39.397" Score="2" Body="&lt;p&gt;I will answer by quoting the book &lt;em&gt;The Myth of Sisyphus&lt;/em&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There is only one really serious philosophical question, and that is suicide. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, probably, we need &lt;em&gt;some&lt;/em&gt; degree of a survival instinct if we don't want our AGIs to &quot;terminate themselves&quot; (whatever that means), whenever they get existential.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2019-11-22T23:30:12.630" LastActivityDate="2019-11-22T23:30:12.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3201" PostTypeId="2" ParentId="2706" CreationDate="2017-04-22T12:10:07.037" Score="2" Body="&lt;p&gt;Because it is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Easy to explain. (Its essentially a game, the &quot;imitation game&quot;)&lt;/li&gt;&#xA;&lt;li&gt;Intuitively plausible as a metric.&lt;/li&gt;&#xA;&lt;li&gt;The idea of &quot;people v.s. AI&quot; is very marketable.&lt;/li&gt;&#xA;&lt;li&gt;At the time we thought that we can analyze cognition strictly in terms of input/output (per behaviorism). Cognitivism, embodied cognition, developmental cognition are all sub-fields that have a right to challenge the Turing Test, but they weren't developed at the time of Turing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of course, it also helps that Turing is a very important figure in AI/CS.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2020-05-18T23:58:38.073" LastActivityDate="2020-05-18T23:58:38.073" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="3202" PostTypeId="1" CreationDate="2017-04-22T18:51:50.027" Score="3" ViewCount="397" Body="&lt;p&gt;I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eg:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Input: capital, Paris, France&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;output : Paris is the capital of France&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Input: cute, cat&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;output: cats are cute&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The system adds the missing words such as 'is', 'as', 'are', 'the', 'of', etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I build a system like this? My gut feeling is it can be done through Reinforcement Learning by training on a huge corpus like Wikipedia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the states being the individual input words. The reward would be 1 when the sentence is correct and 0 when not. The actions available are taking an individual word available from the input and attaching it to the connecting word (is,of,the..). Then in the second step take the generated word and pick another word from input and connect it and so on. Stop when all the input words have been used. Its win when the final sentence is grammatically correct,else fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, what I imagine is there will be a knowledge graph. The user asks some question. Navigating through the knowledge graph, the system will generate some keywords. Then the RL system will take those keywords and construct a human-like sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm totally new to RL. I just finished watching David Silver's 10 part course on RL in YouTube. Any guidance on this topic is much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="6418" LastEditorUserId="9947" LastEditDate="2019-07-27T11:11:31.337" LastActivityDate="2019-07-27T11:11:31.337" Title="RL to generate sentences" Tags="&lt;reinforcement-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3203" PostTypeId="2" ParentId="2706" CreationDate="2017-04-22T20:25:56.193" Score="2" Body="&lt;p&gt;I agree with @colourincorrect 's point that there is economic value to AI which can pass Turing tests to various degrees (chatbots for instance) and this is the reason it is so popular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a deeper level, the test relates to subjectivity, and can be said to have it's origins with the early Greek philosopher &lt;a href=&quot;https://en.wikipedia.org/wiki/Protagoras&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protagoras&lt;/a&gt;, who proposed that &quot;Man is the measure of all things.&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;πάντων &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=xrhmata&amp;amp;la=greek&quot; rel=&quot;nofollow noreferrer&quot;&gt;χρημάτων&lt;/a&gt; μέτρον ἐστὶν ἄνθρωπος, τῶν μὲν &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=ontwn&amp;amp;la=greek#lexicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;ὄντων&lt;/a&gt; ὡς &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=estin&amp;amp;la=greek#lexicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;ἔστιν&lt;/a&gt;, τῶν δὲ οὐκ ὄντων ὡς οὐκ ἔστιν.&#xA;  &lt;br&gt;&lt;sub&gt;Source: Sextus Empiricus, Adv. math. 7.60&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Full quote may be translated as: &quot;Of all things (used by man) the measure (of these things) is man: of the things that are, that they are, of the things that are not, that they are not.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Apologies as I cannot find a direct link for the Greek online. I re-translated the first part of the proposition for clarity, but lifted the second part from &lt;a href=&quot;https://en.wikipedia.org/wiki/Protagoras#cite_note-12&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bostock&lt;/a&gt;, whose Ancient Greek is undoubtedly better than mine, because it is potentially ambiguous, even in the original, and Bostock's interpretation makes good use of that ambiguity.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=xrhmata&amp;amp;la=greek&quot; rel=&quot;nofollow noreferrer&quot;&gt;χρημάτων&lt;/a&gt; &quot;things&quot; is distinct from &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=ontwn&amp;amp;la=greek#lexicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;ὄντων&lt;/a&gt; &quot;things&quot;, which is interpreted to mean Protagoras was speaking about things that man has a direct relationship to, such as property, tools, affairs and so forth. &quot;A thing that one needs or uses&quot; is listed in the LSJ. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Protagoras can unquestionably be extended to Algorithmic Intelligences, which are &quot;thing&quot; used and interacted with by humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Turing Test exists because it not only has utility value, but because of the fundamental condition of subjectivity, the idea of which goes back to the earliest, most basic, philosophical concepts.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-04-22T20:25:56.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3204" PostTypeId="2" ParentId="3194" CreationDate="2017-04-22T21:49:44.947" Score="2" Body="&lt;p&gt;This second answer attempts to address perfect play in relation to incomplete information specifically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An element in the difficulty in answering this question may be that the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#Perfect_play&quot; rel=&quot;nofollow noreferrer&quot;&gt;perfect play&lt;/a&gt; is widely applied to &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#Solved_games&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved games&lt;/a&gt; in the domain of &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial Game Theory&lt;/a&gt; as opposed to strictly economic Game Theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In relation to games with incomplete information: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Perfect play, defined as the best possible choice, without regard to the opponents choice, may be achieved in games with incomplete information&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's important to note that perfect play may not result in a win.  In tic-tac-toe the result is a draw.  In certain games, for a disadvantaged player, it may result in the &quot;best&quot; possible loss.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Perfect play in classic Prisoner's Dilemma is the minimax strategy.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The conundrum is that in this model, it does not lead to the optimal outcome, only the optimal outcome without regard to the other agent's choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In classic Prisoner's Dilemma, the supperational strategy is more risky because there is no information on the other agent (probability for either choice is always 50%) and it doesn't limit downside.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Superrational strategies can be shown to be mathematically supportable by extending Prisoner's Dilemma to iterative and &lt;a href=&quot;http://wrap.warwick.ac.uk/12510/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cyclic&lt;/a&gt; variants. This is partly because in iterative variants, choice are a form of communication between agents. However, the superrational strategy may not be a winning strategy, as the motive of the superrational agent may be said to be maximization benefit, as opposed to limiting downside exclusively. In in iterative Prisoner's Dilemma, the superrational agent may have to sacrifice a couple of iterations (turning the other cheek) in order to incentivize the rational agent to change strategy and cooperate, and to determine if the other agent is irrational, in which case the superrational agent may switch to the rational strategy of minimizing maximum downside and maximizing minimum benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In classic iterated Dilemma, choices are the exclusive form of communication between agents, and each choice becomes part of a dataset on the other agent's decision making. Information is still incomplete, but less incomplete with each iteration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Superrational strategies for games of incomplete information then become viable via statistical analysis.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-04-24T23:18:03.543" LastActivityDate="2017-04-24T23:18:03.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3205" PostTypeId="2" ParentId="2706" CreationDate="2017-04-23T00:23:06.070" Score="2" Body="&lt;p&gt;The test has gained its name and fame mostly because of the person behind it, &lt;strong&gt;Alan Turing&lt;/strong&gt;. Turing - Considered as the father of Artificial Intelligence is among the first who believed that even machines can act and think like humans.&lt;br&gt;&lt;br&gt;&#xA;Even though the test is famous there is not much effort placed to qualify the test. The primary reason for this it due to the fact as the test only asks the machine to act like a human being. This is not very beneficial as not all acts of humans are rational and efficient. Go over &lt;a href=&quot;https://ai.stackexchange.com/questions/15/is-the-turing-test-or-any-of-its-variants-a-reliable-test-of-artificial-intell/2715#2715&quot;&gt;these threads&lt;/a&gt; for more on this.&lt;br&gt;&lt;br&gt;&#xA;I think it's the idea of something non-human acting like a human that creates so much fuzz about Turing Test among the general public.&lt;/p&gt;&#xA;" OwnerUserId="6798" LastActivityDate="2017-04-23T00:23:06.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3206" PostTypeId="2" ParentId="3081" CreationDate="2017-04-23T07:58:23.217" Score="-1" Body="&lt;p&gt;You can use naive bayes classification and calculate posterior probabilities using prior beliefs or logistic regression can be used with sigmoid function.&lt;/p&gt;&#xA;" OwnerUserId="6801" LastActivityDate="2017-04-23T07:58:23.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3207" PostTypeId="2" ParentId="3080" CreationDate="2017-04-23T08:04:13.437" Score="1" Body="&lt;p&gt;If goal is to develop ML algorithms then focus on Maths concepts linear algebra, probability and statistics. Try out CS problem solving basic data structure and algorithms. &#xA;Python has good ML libraries but if you know java then you can pick python easily.&lt;/p&gt;&#xA;" OwnerUserId="6801" LastActivityDate="2017-04-23T08:04:13.437" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3208" PostTypeId="2" ParentId="3080" CreationDate="2017-04-23T11:25:04.483" Score="0" Body="&lt;p&gt;You can't &quot;develop&quot; ML algorithms without statistical knowledge, it simply doesn't work that way and it's impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming is cheap for modelling, anyone who has done a computer science degree can do it. It's just like giving some inputs and giving something back. Lots of framework can do that for you, and it's easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for you to &quot;develop&quot; ML algorithms, you should pursue a mathematics degree. Generally, you're expected to have a PhD or something similar to &quot;develop&quot; ML algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're really interested in programming, you should do R and Python. Java is not a common data science programming language.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-04-23T11:25:04.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3209" PostTypeId="1" AcceptedAnswerId="3213" CreationDate="2017-04-23T16:20:45.467" Score="13" ViewCount="1146" Body="&lt;p&gt;&lt;em&gt;Note: My experience with Gödel's theorem is quite limited: I have read Gödel Escher Bach; skimmed the 1st half of Introduction to Godel's Theorem (by Peter Smith); and some random stuff here and there on the internet. That is, I only have a vague high level understanding of the theory.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my humble opinion, Gödel's incompleteness theorem (and its many related Theorems, such as the Halting problem, and Löbs Theorem) are among the most important theoretical discoveries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, its a bit disappointing to observe that there aren't that many (at least to my knowledge) theoretical applications of the theorems, probably in part due to 1. the obtuse nature of the proof 2. the strong philosophical implications people aren't willing to easily commit towards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite that, there are still some attempts to apply the theorems in a philosophy of mind / AI context. Off the top of my head:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.iep.utm.edu/lp-argue/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Lucas-Penrose Argument&lt;/a&gt;: which argues that the mind is not implemented on a formal system (as in computer). (Not a very rigour proof however) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apparently, some of the research at MIRI uses Löbs Thereom, though the only example I know of is &lt;a href=&quot;http://intelligence.org/files/ProgramEquilibrium.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Löbian agent cooperation.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are all really cool, but are there some more examples? Especially ones that are actually seriously considered by the academic community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://philosophy.stackexchange.com/q/305&quot;&gt;What are the philosophical implications of Gödel's First Incompleteness Theorem?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2020-05-13T10:40:28.900" LastActivityDate="2020-05-13T10:40:28.900" Title="What are some implications of Gödel's theorems on AI research?" Tags="&lt;philosophy&gt;&lt;research&gt;&lt;theory-of-computation&gt;&lt;incompleteness-theorems&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3210" PostTypeId="2" ParentId="1700" CreationDate="2017-04-23T17:05:04.157" Score="1" Body="&lt;p&gt;Careful! There are actually two parts to your question. Don't conflate meanings in your questions, otherwise you won't really know which part you are answering.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should we let AGI experience emotion per &quot;the qualitative experience&quot;? (In the sense that you feel &quot;your heart is on fire&quot; when you fall in love)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There doesn't seem to be a clear purpose as to why we'd want that. Hypothetically we could just have something that is functionally indistinguishable from emotions, but doesn't have any qualitative experience with respect to the AGI. But we are not in a scientific position where we can even begin to answer any questions about the origins of qualitative experience, so I won't bother going deeper into this question.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Should we let AGI have emotions per its functional equivalence from an external observer? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;IMHO yes. Though one could imagine a badass AI with no emotions doing anything you'd want it to, we do wish that AI can integrate with human values and emotions, which is the problem of alignment. It would thus seem natural to assume that any well-aligined AGI will have something akin to emotions if it has integrated well with humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BUT, without a clear theory of mind, it doesn't even begin to make sense to ask: &quot;should our AGI have emotions?&quot; Perhaps there is something critical about our emotions that makes us productive cognitive agents that any AGI would require as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, emotions are often an overlooked aspect of cognition. People somehow think that emotionless Spock-like characters are the pinnacle of human intelligence. But emotions are actually a crucial aspect in decision making, see &lt;a href=&quot;http://nymag.com/scienceofus/2016/06/how-only-using-logic-destroyed-a-man.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; for an example of the problems with &quot;intelligence without emotions&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The follow-up question would be &quot;what sorts of emotions would the AGI develop?&quot;, but again we are not in a position to answer that (yet).&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-04-23T17:05:04.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3212" PostTypeId="2" ParentId="1700" CreationDate="2017-04-24T07:03:50.053" Score="2" Body="&lt;p&gt;By emotions he doesn't mean to add all sorts of emotions into an AI. He only meant the ones that will be helpful for taking vital decisions. Consider this incident for a second:&lt;br&gt;&#xA;Suppose an AI self drive car is driving through the highway. The person sitting inside is the CEO of a company and he is running very behind on schedule. If he didn't get on time there will be loss of millions of dollars. The AI in the car has been told to drive as fast as possible and reach the destination. And now a rabbit (or some other animal) comes into the way. Now if the car puts emergency brakes then the passengers will get seriously hurt and plus there will be loss of millions as CEO won't be able to get to the meeting.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;Now what will the AI do?&lt;/strong&gt;&lt;br&gt;&#xA;Since for an AI, their decisions are only based on their &lt;a href=&quot;https://ai.stackexchange.com/questions/3130/a-twist-on-the-ai-in-a-box-experiment/3177#3177&quot;&gt;utility function&lt;/a&gt;. Hitting the rabbit and keep going will logically show a better option. But, should the AI take that decision.&lt;br&gt;&#xA;&lt;br&gt;There are many questions like these where an AI might stuck into a situation where moral based decisions will play a vital role.&lt;br&gt;&#xA;The above scenario is just for an example point of view.&lt;/p&gt;&#xA;" OwnerUserId="6798" LastActivityDate="2017-04-24T07:03:50.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3213" PostTypeId="2" ParentId="3209" CreationDate="2017-04-24T10:49:22.320" Score="6" Body="&lt;p&gt;Definitely there are a lot of implications for AI, including:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/First-order_logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inference with first-order-logic is semi-decidable&lt;/a&gt;. This is a big disappointment for all the folks that wanted to use logic as a primary AI tool.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Basic equivalence of two first-order logic statements is undecidable, which has implications for knowledge-based systems and databases. For example, optimisation of database queries is an undecidable problem because of this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.cs.wcupa.edu/rkline/fcs/grammar-undecidable.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Equivalence of two context-free grammars is undecidable&lt;/a&gt;, which is a problem for formal linguistic approach toward language processing&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When doing planning in AI, just finding a feasible plan is undecidable for some planning languages that are needed in practice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When doing automatic program generation - we are faced with a bunch of decidability results, since any reasonable programming language is as powerful as a Turing machine.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, all non-trivial questions about an expressive computing paradigm, such as Perti nets or cellular automata, are undecidable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5941" LastEditorUserId="2444" LastEditDate="2020-05-13T10:27:32.927" LastActivityDate="2020-05-13T10:27:32.927" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3214" PostTypeId="2" ParentId="1484" CreationDate="2017-04-24T16:14:25.250" Score="1" Body="&lt;p&gt;A.) The algorithm can only learn patterns from the data, so if you want it to align to a specific functional form of language, it would make sense to curate your data to only contain sentences of that very structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you tune your hyper-parameters such that the model is not as complex as the default, it should work for datasets that are smaller. But this relies on the observation that the thing you are trying to learn isn't as complex (as opposed to learning from a large corpus of Nietzsche).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B.) What is happening is that you are &lt;em&gt;overfitting the data&lt;/em&gt;, such that the LSTM isn't generalizing to your intended goal. In essence, overfitting means that your model is learning irrelevant details that &lt;em&gt;by chance&lt;/em&gt; happen to predict the intended goal in the training data. To alleviate this, use a validation set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally (but very importantly), these machine learning techniques don't learn the same way as humans do. To understand why you'd need to learn the math behind NNs, and I can't think of an intuitive explanation as to why.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-04-24T16:14:25.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3215" PostTypeId="2" ParentId="1484" CreationDate="2017-04-25T14:00:05.233" Score="0" Body="&lt;p&gt;LSTMs are good and everything but you can try some exotic methods like Convolutional Neural Network for NLP. Perhaps it would be more suited for this need. Have a look at this.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2472" LastActivityDate="2017-04-25T14:00:05.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3217" PostTypeId="2" ParentId="3172" CreationDate="2017-04-25T16:54:18.113" Score="3" Body="&lt;p&gt;Federated systems or Fog models, basically push computation from the active system side to server or network side processes. This is commonly used when computationally expensive services are required on limited systems (such as running some AI or augmentation occlusion processing from on phone), allowing for distributed data processing. Here is a good paper on the matter-&#xA;&lt;a href=&quot;https://arxiv.org/abs/1602.05629&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1602.05629&lt;/a&gt;&#xA;What you might be caught up on is edge computing processes and Fog both operate the same. It's just the model differentiating where the services run.&lt;/p&gt;&#xA;" OwnerUserId="5839" LastActivityDate="2017-04-25T16:54:18.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3218" PostTypeId="1" AcceptedAnswerId="3240" CreationDate="2017-04-25T19:35:06.777" Score="1" ViewCount="1372" Body="&lt;p&gt;In past few weeks, I have learned a lot about Neural Networks. Now, I am looking forward to create a Neural Network program that can recognize individual human faces. I tried searching it online but was able to find only small pieces of information.&lt;br&gt;&#xA;&lt;strong&gt;What are the steps for implementing such a program from scratch?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6845" LastEditorUserId="3005" LastEditDate="2017-04-29T19:19:24.927" LastActivityDate="2017-12-28T15:13:02.993" Title="Neural network for detecting individual human face" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3219" PostTypeId="2" ParentId="3218" CreationDate="2017-04-25T20:05:40.133" Score="0" Body="&lt;p&gt;If you want to implement recognition you've just to train a convnet or CNN on a lot of images in which there are faces, and then you classify it 1 if there are faces and 0 if there aren't. If you want to do detection you have to use different approaches like a cascade classifier whit CNN or an object detection network like YOLO or SSD.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastActivityDate="2017-04-25T20:05:40.133" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3221" PostTypeId="1" CreationDate="2017-04-25T23:30:06.163" Score="1" ViewCount="135" Body="&lt;p&gt;Learner might be in training stage, where it update Q-table for bunch of epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by random action rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some epoch, when reward is getting stable, let me call this &quot;training is done&quot;. Then do I have to ignore these parameters(gamma, learning rate, etc) after that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, in training stage, I got an action from Q-table like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if rand_float &amp;lt; rar:&#xA;    action = rand.randint(0, num_actions - 1)&#xA;else:&#xA;    action = np.argmax(Q[s_prime_as_index])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But after training stage, Do I have to remove &lt;code&gt;rar&lt;/code&gt;, which means I have to get an action from Q-table like this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;action = np.argmax(self.Q[s_prime])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6851" LastEditorUserId="1671" LastEditDate="2017-12-21T19:57:25.997" LastActivityDate="2017-12-21T19:57:25.997" Title="Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3222" PostTypeId="1" CreationDate="2017-04-26T06:24:21.023" Score="0" ViewCount="157" Body="&lt;p&gt;How to deal with videos where the frame sizes are not the same frame to frame?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example &lt;a href=&quot;https://www.youtube.com/watch?v=5cKpzp358F4&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; moves up and down and when it does, the video part of the screen has a different amount of pixels vertically.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to deal with different frame sizes in a CNN?&lt;/p&gt;&#xA;" OwnerUserId="4568" LastActivityDate="2017-04-29T18:51:45.553" Title="How to deal with changing video frame sizes in a CNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3224" PostTypeId="1" CreationDate="2017-04-26T16:21:48.563" Score="4" ViewCount="759" Body="&lt;p&gt;I'm a student I'm completely new to this technology maybe my approach could be completely wrong, I want to create an algorithm that compares the similarity between two binarized images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll explain:&#xA;I have 2 pictures as input. The RGB colors of these images can only be 0 or 255&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(R = G = B = 255) or (R = G = B = 0). I take these two letters as an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; &lt;a href=&quot;https://i.stack.imgur.com/dzM8T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dzM8T.png&quot; alt=&quot;symbol1&quot;&gt;&lt;/a&gt; &lt;strong&gt;2.&lt;/strong&gt; &lt;a href=&quot;https://i.stack.imgur.com/fkYSj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fkYSj.png&quot; alt=&quot;symbol2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought so: the 255 value is the background of the image which is white.&#xA;The 0 value is the shape (letter) formed in the image. So I thought of creating a matrix with 0 and 1 values where value 0 represents the background and value 1 represents the shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now i would like to create an algorithm that understands the shape created in the two matrices and that returns a similarity percentage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I'm creating this app that tries to recognize the font of a text in a image (&lt;a href=&quot;https://github.com/Sirvasile/Typefont&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Sirvasile/Typefont&lt;/a&gt;), I want to create this algorithm to improve the comparison between the input letters and the alphabet of my fonts in the database.&lt;/p&gt;&#xA;" OwnerUserId="6867" LastEditorUserId="6868" LastEditDate="2017-05-28T13:48:22.037" LastActivityDate="2018-01-14T10:23:59.947" Title="Image comparison algorithm, trying to figure out how similar two &quot;binary&quot; forms are" Tags="&lt;algorithm&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3225" PostTypeId="2" ParentId="3080" CreationDate="2017-04-26T21:00:06.250" Score="0" Body="&lt;p&gt;As others have pointed out, your level of maths/statistics knowledge is probably more important than your chosen programming language.  That is, particularly true w/r/t developing (presumably new) ML algorithms.  OTOH, from an &quot;applied ML&quot; perspective, where you just use pre-baked implementations of existing algorithms, one of the big questions is &quot;do good libraries for these various operations exist in language $X&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the value of $X is &quot;Java&quot; the answer is &quot;yes&quot;.  There are tons of high-quality libraries of most popular and widely used ML algorithms.  There are also tons of libraries for nearly everything else, which helps when constructing wider systems which incorporate elements of ML/AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, w/r/t machine learning specifically, there are probably &lt;em&gt;more&lt;/em&gt; extant libraries and what-not in Python or R, than in Java.  Also, from a &quot;rapid development&quot; point of view, you may find that Python has some advantages as as of a result of its dynamic typing and lack of &quot;boiler-plate&quot; as found in Java.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Net-net, I'd say Java is a fine choice, along with Python or R, with C++ also in the mix.  It may not be the &lt;em&gt;perfect&lt;/em&gt; choice, but it's absolutely &quot;good enough&quot; and then some.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a feel for some of the projects that exist now in various languages, to go mloss.org and use the &lt;a href=&quot;http://mloss.org/software/language/&quot; rel=&quot;nofollow noreferrer&quot;&gt;filter by language feature&lt;/a&gt;.  Click around there and examine some of the options you find.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-04-26T21:00:06.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3226" PostTypeId="1" CreationDate="2017-04-26T23:47:06.000" Score="7" ViewCount="374" Body="&lt;p&gt;I have a question as to what it means for a knowledge-base to be consistent and complete. I've been looking into non-monotonic logic and different formalisms for it from the book &amp;quot;Knowledge Representation and Reasoning&amp;quot; by Hector Levesque and Ronald J. Brachman, but something is confusing me.&lt;/p&gt;&#xA;&lt;p&gt;They say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We say a KB exhibits consistent knowledge iff there is no sentence &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; such that both &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\neg P$&lt;/span&gt; are known. This is the same as requiring the KB to be satisfiable. We also say that a KB exhibits complete knowledge iff for every &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; (within its vocabulary) &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\neq P$&lt;/span&gt; is known&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They then seem to suggest that by &amp;quot;known&amp;quot; they mean &amp;quot;entailed&amp;quot;. They say&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In general, of course, knowledge can be incomplete. For example, suppose KB consists of a single sentence (&lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;). Then KB does not entail either &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\neg P$&lt;/span&gt;, and so exhibits incomplete knowledge.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But when dealing with sets of sentences, I usually see these terms as being defined w.r.t. &lt;em&gt;derivability&lt;/em&gt; and not &lt;em&gt;entailment&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So my question is, what exactly do these authors mean by &amp;quot;known&amp;quot; in the above quotes?&lt;/p&gt;&#xA;&lt;p&gt;Edit: &lt;a href=&quot;https://math.stackexchange.com/q/2259311/168764&quot;&gt;this post&lt;/a&gt; helped clarify things.&lt;/p&gt;&#xA;" OwnerUserId="5133" LastEditorUserId="2444" LastEditDate="2020-07-18T14:33:52.097" LastActivityDate="2020-07-18T14:33:52.097" Title="What is meant by &quot;known&quot; in &quot;A knowledge-base exhibits complete knowledge if and only if, for every $P$, $P$ or $\neg P$ is known&quot;?" Tags="&lt;knowledge-representation&gt;&lt;terminology&gt;&lt;logic&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3227" PostTypeId="2" ParentId="3209" CreationDate="2017-04-27T01:55:54.303" Score="0" Body="&lt;p&gt;I found &lt;a href=&quot;https://math.stanford.edu/~feferman/papers/dichotomy.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; by mathematician and philosopher &lt;em&gt;Solomon Feferman&lt;/em&gt; on &lt;em&gt;Gödel's 1951 Gibbs lecture on certain philosophical consequences of the incompleteness theorems&lt;/em&gt;, while reading the following Wikipedia article&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Philosophy_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Philosophy of artificial intelligence&lt;/a&gt;,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;whose abstract gives us (as expected) a high-level idea of what's discussed in the same:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is a critical analysis of the first part of Gödel's 1951 Gibbs lecture&#xA;  on certain philosophical consequences of the incompleteness theorems.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Gödel's discussion is framed in terms of a distinction between &lt;em&gt;objective&#xA;  mathematics&lt;/em&gt; and &lt;em&gt;subjective mathematics&lt;/em&gt;, according to which the former&#xA;  consists of the truths of mathematics in an absolute sense, and the latter&#xA;  consists of all humanly demonstrable truths. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The question is whether these coincide; if they do, no formal axiomatic system (or &lt;em&gt;Turing machine&lt;/em&gt;) can comprehend the mathematizing potentialities of human&#xA;  thought, and, if not, there are absolutely unsolvable mathematical&#xA;  problems of diophantine form.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Either ... the human mind ... infinitely surpasses the powers of any&#xA;  finite machine, or else there exist absolutely unsolvable diophantine&#xA;  problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which may be of interest, at least philosophically, to the research in AI. I'm afraid this paper may be similar to the article you're linking to regarding Lucas and Penrose philosophical &quot;attempts&quot; or arguments.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2017-04-27T02:08:45.147" LastActivityDate="2017-04-27T02:08:45.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3231" PostTypeId="2" ParentId="3221" CreationDate="2017-04-27T20:52:05.757" Score="2" Body="&lt;p&gt;Epsilon (your rar parameter) is used to have the possibility of a random action which handles the problem of exploration and exploitation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a deterministic world and are certain that your learner has learned a policy with performance to your liking, removing it will ensure that only the greedy policy being learned will be taken and that you can expect the same performance afterwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent will still be learning at this point because alpha (learning rate) is not set to 0. Because of this, if the world is stochastic instead and possibly changes slowly over time (e.g. motor speed due to battery power), then having epsilon non zero still allows the agent to explore and overcome these changes when they happen before it's performance gets low enough to change is behaviour (e.g. q values dropping). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If instead you simply want to test the performance of the policy learned up until a timestep, setting alpha to 0 in a compares the epsilon-greedy policy, setting both alpha and epsilon to 0 compares the greedy policy. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-04-27T20:52:05.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3232" PostTypeId="2" ParentId="3194" CreationDate="2017-04-28T04:47:30.810" Score="1" Body="&lt;p&gt;It depends on the game. In zero sum non cooperative games yes, there's always a GTO strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest example is Rock, Paper, Scissors, where playing 1/3 of each randomly would be the only optimal strategy. In this case a break even one too, in some games though GTO has a positive expected value against any strategy that's not GTO itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually online video games strategies and metas are heavily based on adaptation to population tendencies though, which in of itself it's not perfect play, but it can have a better expected value than perfect play against a non optimal opponent.&lt;/p&gt;&#xA;" OwnerUserId="6892" LastEditorUserId="145" LastEditDate="2017-05-28T13:48:34.423" LastActivityDate="2017-05-28T13:48:34.423" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3233" PostTypeId="1" CreationDate="2017-04-28T11:42:08.667" Score="5" ViewCount="596" Body="&lt;p&gt;Can current trends and tools, in the field of machine learning, replicate the complexity of financial market? If yes, then what are the tools available in this domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q.&lt;/strong&gt; I am trying to build a model to infer results from stock market using the concept to create a graph on the companies enlisted. Can anyone suggest me approaches to do so?&lt;/p&gt;&#xA;" OwnerUserId="6897" LastEditorUserId="3005" LastEditDate="2017-05-28T13:48:29.010" LastActivityDate="2017-07-15T06:10:36.853" Title="Use of machine learning for analyzing companies enlisted in stock market" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;algorithm&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3235" PostTypeId="2" ParentId="3233" CreationDate="2017-04-28T13:16:35.860" Score="1" Body="&lt;p&gt;In one way the answer is &lt;strong&gt;NO&lt;/strong&gt;. You can't incorporate all the necessary details in an AI program to correctly predict the financial market, at least not in currently. A particular event is a result of many actions and events from the past. Everything is like a chain.&lt;br&gt;&lt;br&gt;&#xA;You might have heard of the &lt;strong&gt;ButterFly Effect&lt;/strong&gt;. It simply states that small things can have very large effects. Like flap of butterfly's wings leading to a hurricane. You can read more about it &lt;a href=&quot;https://en.wikipedia.org/wiki/Butterfly_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;br&gt;&lt;br&gt;&#xA;Now back to your question. I believe you are somewhat familiar with the financial system and stocks. Close your eyes and try to imagine all the data that is being churned into machines. Apart from that, there are many decisions that lead to changes in the financial market directly. Like policies made by the government deals bagged by the company related to imports and exports, death of the main employee (like CEO, founder), and other many things. Apart from that, there are uncountable things that indirectly affect the market. I want you to think about it for a while. Now consider doing this for all the stocks present. After even stock values of one are dependent on the others in one way or another.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;This amount of data, in other ways considering all these things, is not possible for current generation computers to process. You might have a hit with &lt;em&gt;quantum computers&lt;/em&gt; coming though.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;&#xA;Now what you can do, is try to make a Deep Network (with Reinforcement Learning) that can predict the behavior and shift in the direction of &lt;strong&gt;some&lt;/strong&gt; stocks. For example, whether it will go up for down or try to keep a constant line on the graph.&lt;br&gt;&#xA;Selection of &lt;strong&gt;Features&lt;/strong&gt; is always an important key to the success of Machine Learning algorithms. Another problem that stock market-related AI programs face is that Economics is itself an emerging and novice field of science. It's not as mature as physics, chemistry or biology. Thus finding out and deciding over the choice of features is another difficult job.&lt;br&gt;&#xA;I found &lt;a href=&quot;https://people.eecs.berkeley.edu/~akar/IITK_website/EE671/report_stock.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; online. Haven't given a look to it but, I believe this might help you a bit.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-04-28T13:16:35.860" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3237" PostTypeId="2" ParentId="3233" CreationDate="2017-04-28T15:49:48.387" Score="0" Body="&lt;p&gt;You can actually build a statistical model (not exactly and AI) to predict the stock market trend. But the prediction accuracy will be very bad.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accuracy of your model depends on the following things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No of variables which affect the model's output directly or indirectly.&lt;/li&gt;&#xA;&lt;li&gt;The model's reaction time (How fast can your model generate an output corresponding to a change in the input).&lt;/li&gt;&#xA;&lt;li&gt;The maximum lifespan of your model (Describes the the time the model takes to mature + the span of time the model stays relative to the context of it's variables)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE: This is not in anyway a correct answer to your question. I just wanted to state that anything and everything in this universe can be modeled given that your are aware of all the variables which affect it and how they interact with each other.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6903" LastActivityDate="2017-04-28T15:49:48.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3240" PostTypeId="2" ParentId="3218" CreationDate="2017-04-29T04:09:33.917" Score="1" Body="&lt;p&gt;I am assuming that you are new to all this. You can start with making a basic human face detection. Train the program to detect a human face with very good accuracy. This will help you to get familiar with the coding ground related to image processing and basic machine learning.&lt;br&gt;&#xA;After that train your program to identify faces of only 2-3 people. Trying for too many in the beginning won't be a good idea. Test your program's accuracy in different situation, with a different number of crowd, etc.&lt;br&gt;&lt;br&gt;&#xA;If it's working fine then you can train your program for more individuals. In addition to this leave room for learning from experience in your code. Some codes only learn once and they implement the same thing for their whole life. A nice example of this kind is &lt;a href=&quot;https://en.wikipedia.org/wiki/Optical_character_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;OCR&lt;/a&gt;. If a face is detected wrongly OR your program detects a new face about which it doesn't know anything. Then you should be able to tell the program and it should include that in its database. I think some form of reinforcement learning will help. Not much sure about it though.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;Now the Implementation&lt;/strong&gt;&lt;br&gt;&#xA;I will highly recommend you to learn python and get familiar with OpenCV. You can think of OpenCV as a collection of libraries. I find them very helpful for image processing and machine learning. Another good thing about it is that you can import OpenCV in python, Java or C++ according to your need.&lt;br&gt;&#xA;OpenCV has an inbuilt function that allows it train a neural network for positive and negative images. The success of your program depend highly on your choice of positive and negative images, so choose them wisely. The result of the training is stored as a &lt;a href=&quot;http://docs.opencv.org/2.4/modules/objdetect/doc/cascade_classification.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;haar cascade file&lt;/a&gt;. This cascade file can be used in your program to use the trained data and function accordingly.&lt;br&gt;&lt;br&gt;&#xA;For basic human face detection, you can find the cascade file online and implement a code like this&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml');&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For detecting individual faces you will need to make different classes. The number of classes will represent the number of individual faces you want to detect in the beginning.&lt;br&gt;&#xA;You can find OpenCV tutorials &lt;a href=&quot;http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-04-29T04:09:33.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3243" PostTypeId="1" AcceptedAnswerId="3253" CreationDate="2017-04-29T17:44:01.653" Score="7" ViewCount="22745" Body="&lt;p&gt;There are several classes of intelligent agents, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;simple reflex agents&lt;/li&gt;&#xA;&lt;li&gt;model-based reflex agents&lt;/li&gt;&#xA;&lt;li&gt;goal-based agents&lt;/li&gt;&#xA;&lt;li&gt;utility-based agents&lt;/li&gt;&#xA;&lt;li&gt;learning agents&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Each of these agents behaves slightly different from the other agents. There are certain diagrams that describe each of these IA classes. However, all the descriptions are usually quite abstract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some examples of intelligent agents for each intelligent agent class? Optionally, I would also like to see a compact definition of each class (and maybe how they are related to the diagrams).&lt;/p&gt;&#xA;" OwnerUserId="6921" LastEditorUserId="2444" LastEditDate="2019-11-16T19:56:31.900" LastActivityDate="2019-11-16T19:56:31.900" Title="What are some examples of intelligent agents for each intelligent agent class?" Tags="&lt;definitions&gt;&lt;intelligent-agent&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="3244" PostTypeId="2" ParentId="2472" CreationDate="2017-04-29T17:46:16.563" Score="2" Body="&lt;p&gt;When considering effective approaches to AGI, one must extrapolate outwards to the types of modelling (and therefore inputs) that would be necessary to achieve any general utility. One consideration might be the fundamental &quot;building blocks&quot; of our physical world, and understanding the movements of these, can lead to accurate predictions of (all) occurrences. These fundamental elements are called (generally) subatomic particles and are anything but discrete values. In quantum field theory, the more accurate you are able to measure position, the less accurate you may know a quarks momentum (and vice versa). Our world, at the most fundamental layer, is probabilistic when observed. This is all not to say that an understanding of quantum mechanical kinematic descriptions is the only methodology to achieve true AGI, but to say probabilistic models, and therefore uncertainty, is a dead-end seems radically inaccurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, Dr.Minsky didn't really feel probabilistic models were dead ends. The emerging view in the field, one that Dr.Minsky urged for years, is that connectionism alone couldn't exclusively lead to AGI due to its uniformed structure. If you are unaware, connectionism is the concept of creating models around discrete units of representation (neurones in a neural net for example). You see the problem we have identified isn't that probabilistic models are inaccurate, it's that our current approach doesn't express the biological realism necessary for AGI (although sufficient for specific intelligence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[I briefly worked with Dr.Minsky at the AI Lab before his passing last year, a hilarious man and brilliant scientist.]&lt;/p&gt;&#xA;" OwnerUserId="5839" LastActivityDate="2017-04-29T17:46:16.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3246" PostTypeId="2" ParentId="3222" CreationDate="2017-04-29T18:51:45.553" Score="1" Body="&lt;p&gt;Well, the easiest way in order to make different frame sizes work with a convolutional neural network, is to process them using a different operations such as scaling, and or cropping in order to make them the same size. I do not know of any way currently where different frame sizes can be inputted directly into the neural network, as in order to do that you would need to almost completely change the neural network architecture. If the frames have a small difference in size, you may be able to get away with scaling the frame to the dimensions of the largest frame. This also depends upon what kind of problem you are trying to solve. While this method would work if you are trying to classify an image into a couple categories, it would not work if your are trying to do tasks such as object location. If you are doing one of the latter tasks, you should be able to pad the image with blank pixels in order to make the frame you are trying to process the dimensions of the largest one. This would work quite well, especially for object location detection. Finally, another solution is to crop the larger images to the size of the smallest one. While this will work just as well as padding, if you need any image data from the very edges of the frame, this will not work very well. So in conclusion, there are three different methods you could apply. Me personally, I would go with padding because it does not remove any important image data, and it will work very well for tasks such as object location detection. If you are doing a classification task though, scaling would work also. Finally, you could crop the images, but this would have to be evaluated for your specific task.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-04-29T18:51:45.553" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3248" PostTypeId="2" ParentId="2190" CreationDate="2017-04-30T03:15:23.537" Score="1" Body="&lt;p&gt;Switching requires relearning, the network did not have a single set of weights that allowed it to play all games well. This is due to the catastrophic forgetting problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, recent work has been done to overcome this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Overcoming catastrophic forgetting in neural networks&quot;, 2016&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paper: &lt;a href=&quot;https://arxiv.org/pdf/1612.00796v1.pdf&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Overcoming catastrophic forgetting in neural networks&quot;&gt;https://arxiv.org/pdf/1612.00796v1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6925" LastActivityDate="2017-04-30T03:15:23.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3249" PostTypeId="1" AcceptedAnswerId="3250" CreationDate="2017-04-30T15:10:22.757" Score="1" ViewCount="60" Body="&lt;p&gt;To risk giving away too much info, im building a piece of hardware with the job of  &lt;strong&gt;identifying the object in front of it&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it can only be &lt;strong&gt;one of three&lt;/strong&gt; different items, how can I &lt;em&gt;tell&lt;/em&gt; the computer with simplecv?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I've found a way to limit the choices down to just a handful of potential objects, which &lt;em&gt;should&lt;/em&gt; increase the probability of it recognizing the object correctly. Is there a way to limit the choices for the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Me: &lt;em&gt;hey raspberry pi - you see that thing in front of you?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Raspberry Pi: &lt;em&gt;That thing? Ohh you mean that piece of food that might be a ham and cheese sandwich, but also kinda looks like a fish, with a slight twist of pe-&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Me: &lt;em&gt;- whoa okay, hold on! It's either a grilled cheese sandwich, or an apple&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RPI: &lt;em&gt;ohhh well that's easy! it's clearly (with 98% confidence) a grilled cheese sandwich&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thought are appreciated!&lt;/p&gt;&#xA;" OwnerUserId="2752" LastEditorUserId="2752" LastEditDate="2017-05-01T01:37:01.993" LastActivityDate="2017-05-01T01:37:01.993" Title="Can I limit the possible choices for a computer vision framework to recognize?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3250" PostTypeId="2" ParentId="3249" CreationDate="2017-04-30T15:48:59.757" Score="1" Body="&lt;p&gt;I would recommend using a convolutional neural network. CNNs are the bleeding edge for image recognition currently. They should be able to solve your problem with a high accuracy, just maybe without the sort of dialogue that you provided in your question. In order to train a convolutional neural network, you would have to use a different computer than your desktop and &quot;deploy&quot; it to your raspberry pi. This means that your desktop computer would find the optimal wight configurations for your neural network and the images you want to detect, and then you would be able to send the neural network to your raspberry pi in order to be used. If you want to, you could train your entire convolutional neural network from scratch, using random weights to begin, or you could use a process known as transfer learning where you take an existing neural network, usually deep, and train it a little bit further according to your existing data set. In order to train your neural network from scratch, you will need a very large volume of sample images to train it. In order to do transfer learning, you will only need about 1000 images for each of your classes in order to achieve very accurate results for your specific task. For specific libraries that you can use, I would recommend using Matlab if you have access to it as it is extremely easy to train and run deep neural networks, and it is easy to deploy the model to your raspberry pi. You can implement transfer learning in Matlab with 10 lines of code. Next, if you do not have access to Matlab, I would recommend if you have some python experience, using tensorflow. Tensorflow supports a lot of deep learning algorithms and should be able to for your problem although it will not be as easy as using MATLAB. Finally, Microsoft has a toolkit called CNTK which is fairly similar to tensorflow, but instead you can use it in C# or other .net languages. &lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-04-30T15:48:59.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3253" PostTypeId="2" ParentId="3243" CreationDate="2017-05-01T08:46:05.240" Score="8" Body="&lt;p&gt;There are no distinguishable hardware examples for each IA class. The same mobile robot architecture with proper sensors can be implemented to behave as any IA class. The way you can determine the class of an intelligent agent is from the way it processes the percept. Based on chapter 2 of &lt;em&gt;Artificial Intelligent: A Modern Approach&lt;/em&gt; I will try to give a concise explanation for each class:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;simple reflex agent&lt;/strong&gt; takes an action based on only the current environmental situation; it maps the current percept into proper action ignoring the history of percepts. The mapping process could be simply a table-based or by any rule-based matching algorithm. An example of this class is a &lt;strong&gt;robotic vacuum cleaner&lt;/strong&gt; that deliberate in an infinite loop, each percept contains a state of a current location [clean] or [dirty] and, accordingly, it decides whether to [suck] or [continue-moving]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;model-based reflex agent&lt;/strong&gt; needs memory for storing the percept history; it uses the percept history to help to reveal the current unobservable aspects of the environment. An example of this IA class is the &lt;strong&gt;self-steering mobile vision&lt;/strong&gt;, where it's necessary to check the percept history to fully understand how the world is evolving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;goal-based reflex agent&lt;/strong&gt; has a goal and has a strategy to reach that goal. All actions are taken to reach this goal. More precisely, from a set of possible actions, it selects the one that improves the progress towards the goal (not necessarily the best one). An example of this IA class is any &lt;strong&gt;searching robot&lt;/strong&gt; that has an initial location and wants to reach a destination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;strong&gt;utility-based reflex agent&lt;/strong&gt; is like the goal-based agent but with a measure of &quot;how much happy&quot; an action would make it rather than the goal-based binary feedback ['happy', 'unhappy']. This kind of agents provide the best solution. An example is the &lt;strong&gt;route recommendation system&lt;/strong&gt; which solves the 'best' route to reach a destination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;learning agent&lt;/strong&gt; is an agent capable of learning from experience. It has the capability of automatic information acquisition and integration into the system. Any agent designed and expected to be successful in an uncertain environment is considered to be learning agent. &lt;/p&gt;&#xA;" OwnerUserId="4416" LastEditorUserId="2444" LastEditDate="2019-11-16T19:48:07.653" LastActivityDate="2019-11-16T19:48:07.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3254" PostTypeId="2" ParentId="3209" CreationDate="2017-05-01T12:43:51.997" Score="1" Body="&lt;p&gt;I've written an extensive article on this some twenty years ago, which was published in &lt;em&gt;Engineering Applications of Artificial Intelligence 12 (1999) 655-659&lt;/em&gt;. It's fairly technical and &lt;a href=&quot;http://www.christianjongeneel.nl/gepubliceerde-boeken/goedel-pro-and-contra-ai-dismissal-of-the-case/&quot; rel=&quot;nofollow noreferrer&quot;&gt;you can read it in full&lt;/a&gt; on my personal website, but here's the conclusion:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the above it was shown that there are infinitely many proof&#xA;  constructions to Gödel’s theorem – in contrast to the single one that&#xA;  was used in discussions on artificial intelligence so far. Though all&#xA;  constructions that have been actually disclosed can be imitated by a&#xA;  computer, it is evident that there are constructions that have not&#xA;  been disclosed yet. Our analysis has shown that there might exist&#xA;  constructions that might only be discovered by a human. This is a&#xA;  small and definitely unprovable ‘maybe’ that depends on the limits of&#xA;  human imagination.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Hence, people arguing for the mathematical equivalence of humans and&#xA;  machines must ultimately rely on their belief in a limited mind, which&#xA;  implies that their conclusion is contained in their assumption. On the&#xA;  other hand, people advocating the superiority of humans must assume&#xA;  this superiority in their mathematical arguments, ultimately only&#xA;  deriving the conclusion that was already present in their system of&#xA;  reasoning from the very start.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;So, it is not possible to produce (meta)mathematically sound arguments&#xA;  concerning the relation between the human mind and the Turing Machine&#xA;  without making an assumption on the human mind that is at the same&#xA;  time the conclusion of the argument. Therefore, the matter is&#xA;  undecidable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I have left academia since, so I do not know of contemporary thinking.&lt;/p&gt;&#xA;" OwnerUserId="6946" LastActivityDate="2017-05-01T12:43:51.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3258" PostTypeId="1" AcceptedAnswerId="7078" CreationDate="2017-05-03T12:57:36.667" Score="11" ViewCount="18893" Body="&lt;p&gt;I'm developing an AI tool to find known equipment errors and find new patterns of failure. This log file is time-based and has known messages (information and error).&lt;/p&gt;&#xA;&lt;p&gt;I'm using a JavaScript library, but my real question is &lt;em&gt;how to train the AI to find the known patterns and find new possible patterns&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have some requirements:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The tool shall either a. has no dependence on extra environment installation or b. the less the better (the perfect scenario is to run the tool entirely on the browser in standalone mode);&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Possibility to make the pattern analyzer fragmented, a kind of modularity, one module per error;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What are the recommended kind of algorithm to do this (neural network, genetic algorithm, etc.)? Is there anything to achieve that end in JavaScript? If not, what is the best language to make this AI?&lt;/p&gt;&#xA;" OwnerUserId="6978" LastEditorUserId="2444" LastEditDate="2020-12-31T17:50:02.240" LastActivityDate="2020-12-31T17:56:33.680" Title="How can I train an AI to find known and new patterns in a log file?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;pattern-recognition&gt;&lt;anomaly-detection&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="11" ClosedDate="2020-12-31T17:56:37.763" ContentLicense="CC BY-SA 4.0" />
  <row Id="3261" PostTypeId="2" ParentId="1462" CreationDate="2017-05-04T08:25:42.040" Score="3" Body="&lt;p&gt;In a general sense you can say that robot is a piece of hardware, while AI is software (sometimes hardware too).&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; states Robot as a machine which performs complex set of tasks automatically.&lt;br&gt;&#xA;  Machine - A mechanical device basically.&lt;br&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, technically you can create a robot that doesn't require any kind of complex algorithms to take decisions. A simple line follower doesn't even require a microcontroller. Just some gates are enough. Some other examples of robots are, a robotic arm, automated control systems in industries, etc. If you think about it even the printer in your house is a robot in itself.&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial Intelligence is a field of Computer Science which deals with developing systems that can perform tasks rationally as if it is using intelligence (of human level) for taking decisions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;AI deals with complex algorithms. Some examples of AI are speech recognition, face recognition, natural language processing, etc.&#xA;&lt;br&gt;AI don't necessarily need additional hardware. A simple desktop at home will work, while the term robot is used for external hardware that does some autonomous task repeatedly. &lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-05-04T08:25:42.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3262" PostTypeId="1" AcceptedAnswerId="3263" CreationDate="2017-05-04T13:06:37.990" Score="15" ViewCount="20142" Body="&lt;p&gt;These types of questions may be problem-dependent, but I have tried to find research that addresses the question whether the number of hidden layers and their size (number of neurons in each layer) really matter or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, does it really matter if we for example have 1 large hidden layer of 1000 neurons vs. 10 hidden layers with 100 neurons each?&lt;/p&gt;&#xA;" OwnerUserId="6645" LastActivityDate="2018-09-17T18:41:57.850" Title="1 hidden layer with 1000 neurons vs. 10 hidden layers with 100 neurons" Tags="&lt;neural-networks&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3263" PostTypeId="2" ParentId="3262" CreationDate="2017-05-04T13:10:53.287" Score="13" Body="&lt;p&gt;Basically, having multiple layers (aka a deep network) makes your network more eager to recognize certain aspects of input data. For example, if you have the details of a house (size, lawn size, location etc.) as input and want to predict the price. The first layer may predict:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Big area, higher price&lt;/li&gt;&#xA;&lt;li&gt;Small amount of bedrooms, lower price&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The second layer might conclude:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Big area + small amount of bedrooms = large bedrooms = +- effect&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Yes, one layer can also 'detect' the stats, however it will require more neurons as it cannot rely on other neurons to do 'parts' of the total calculation required to detect that stat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/274569/deep-networks-vs-shallow-networks-why-do-we-need-depth/274571#274571&quot;&gt;Check out this answer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="5344" LastEditDate="2017-05-04T19:35:36.130" LastActivityDate="2017-05-04T19:35:36.130" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3267" PostTypeId="2" ParentId="3262" CreationDate="2017-05-05T00:34:03.793" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I think you have a confusion in the basics of the neural networks.&#xA;  Every layer has a separate activation function and input/output&#xA;  connection weights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The output of the first hidden layer will be multiplied by a weight, processed by an activation function in the next layer and so on.&#xA;Single layer neural networks are very limited for simple tasks, deeper NN can perform far better than a single layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, do not use more than layer if your application is not fairly complex. In conclusion, 100 neurons layer does not mean better neural network than 10 layers x 10 neurons but 10 layers are something imaginary unless you are doing deep learning. start with 10 neurons in the hidden layer and try to add layers or add more neurons to the same layer to see the difference. learning with more layers will be easier but more training time is required.&lt;/p&gt;&#xA;" OwnerUserId="3326" LastEditorUserId="1581" LastEditDate="2017-05-06T10:05:20.410" LastActivityDate="2017-05-06T10:05:20.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3268" PostTypeId="2" ParentId="3081" CreationDate="2017-05-05T05:00:05.230" Score="1" Body="&lt;p&gt;Yes you can user either classification or regression according to your output requirement, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want labeled output, like either available or not available then classification should be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want the output in the form of % of availability then regression should be used.&lt;/p&gt;&#xA;" OwnerUserId="7022" LastActivityDate="2017-05-05T05:00:05.230" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3269" PostTypeId="2" ParentId="3262" CreationDate="2017-05-05T14:21:31.097" Score="4" Body="&lt;p&gt;There are so many aspects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Training:&lt;/strong&gt;&#xA;Training deep nets is a hard job due to the &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap5.html#the_vanishing_gradient_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;vanishing&lt;/a&gt; (rearly exploding) gradient problem. So building a 10x100 neural-net is not recommended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Trained network performance:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Information loss:&lt;/strong&gt;&#xA;The classical usage of neural nets is the &lt;a href=&quot;https://math.stackexchange.com/questions/141381/regression-vs-classification&quot;&gt;classification&lt;/a&gt; problem. Which means we want to get some well defined information from the data. (Ex. Is there a face in the picture or not.)&#xA;So usually classification problem has a lot of input, and few output, whats more the size of the hidden layers are descend from input to output.&#xA;However, we loss information using less neurons layer by layer. (Ie. We cannot reproduce the original image based on the fact that is there a face on it or no.) So you must know that you loss information using 100 neurons if the size of the input is (lets say) 1000.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Information complexity:&lt;/strong&gt; However the deeper nets (as Tomas W mentioned) can fetch more complex information from the input data. Inspite of this its not recommended to use 10 fully connected layers. Its recommended to use convolutional/relu/maxpooling or other type of layers. Firest layers can compress the some essential part of the inputs. (Ex is there any line in a specific part of the picture) Second layers can say: There is a specific shape in this place in the picture. Etc etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So deeper nets are more &quot;clever&quot; but 10x100 net structure is a good choice.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="7034" LastActivityDate="2017-05-05T14:21:31.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3272" PostTypeId="1" CreationDate="2017-05-06T02:51:29.280" Score="7" ViewCount="328" Body="&lt;p&gt;I feel that many words if not all of them have a direct mapping to some kind of inner subjective experience, to a physical object, mental feeling, process or some other kind of abstract thing. Given that machines don't have &lt;em&gt;qualia&lt;/em&gt; and no mapping of this kind, can they really understand anything even though they are made to answer to questions with lots of statistical training?&lt;/p&gt;&#xA;" OwnerUserId="3015" LastEditorUserId="2444" LastEditDate="2019-05-06T16:03:32.560" LastActivityDate="2019-05-06T16:03:32.560" Title="Is language understanding possible without qualia?" Tags="&lt;agi&gt;&lt;natural-language-processing&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3274" PostTypeId="1" AcceptedAnswerId="3275" CreationDate="2017-05-06T14:13:52.703" Score="2" ViewCount="98" Body="&lt;p&gt;I´m currently implementing NEAT.&#xA;What should I do when in a mutation the same Innovation occurs which has already happened to that genome?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I simply ignore it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not what do I do with it in the mating part?&lt;/p&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="1671" LastEditDate="2018-03-05T19:09:39.003" LastActivityDate="2018-03-05T19:09:39.003" Title="What to do with duplicate Innovations in a genome?" Tags="&lt;neural-networks&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3275" PostTypeId="2" ParentId="3274" CreationDate="2017-05-06T14:39:52.057" Score="3" Body="&lt;p&gt;Because this is more of an answer than a comment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the add connection&#xA;  mutation, a single new connection gene with a random weight is added connecting&#xA;  two previously unconnected nodes. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So two nodes that have been mutated with a connection, can't remutate because the nodes should be unconnected.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the add node mutation, an existing connection is&#xA;  split and the new node placed where the old connection used to be. The old connection&#xA;  is disabled and two new connections are added to the genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The old connection is disabled, so the same mutation can't occur again on this connection. In practice, you can just remove the disabled connection, because it will be formed again with the add connection mutation.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastActivityDate="2017-05-06T14:39:52.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3276" PostTypeId="1" CreationDate="2017-05-06T16:06:19.123" Score="1" ViewCount="47" Body="&lt;p&gt;I'm wondering if anyone reading this has developed a flowchart type representation of Intelligence and/or consciousness. Some examples of theories would be the Three Stratum Theory of Intelligence, Intermediate level theory of consciousness, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to take what I see to develop (what I think will be) the most likely version of A.I. to be accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's where I'm at now with the intelligence model: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/4oUv6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4oUv6.png&quot; alt=&quot;A.I. Model&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first row is the receptors for the information it will be able to take in. the next was going to be the Three stratum theory but I don't see that as a good method of data management. I was thinking it would be better if there was some sort of framework for information to be developed. And that's why I didn't go on past General Intelligence. But then at the bottom I would put the actuator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. I'm not an expert for any of this stuff but I will thoroughly research whatever is said.&lt;/p&gt;&#xA;" OwnerUserId="7045" LastActivityDate="2017-05-06T16:06:19.123" Title="Could you share your model of Intelligence and/or conciousness?" Tags="&lt;models&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3280" PostTypeId="2" ParentId="3272" CreationDate="2017-05-07T22:02:59.077" Score="1" Body="&lt;p&gt;I think a &quot;successful&quot; strong AI with natural language abilities -- say one that could produce &quot;good&quot; unsupervised translations of literature, or pass a rigorous Turing test -- would have to include in the corpus of data used to build its models visual, auditory, and probably tactile data.  It might be necessary as well to simulate the kind of agency and intentionality that humans have-- so the AI-in-training has the opportunity to move a simulated self to change what input it receives.  I suspect training it only on text data, for example, would always be inadequate.  If it had access to sensory information and learned to associate it appropriately with the symbols of language, it might be able to learn the meaning of language in a way that we'd find difficult to distinguish from our own understanding, even though the AI would presumably not have the (same) qualia we have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, we are a long way from having hardware sufficiently powerful to even attempt such a comprehensive mind-modeling project.   But I don't think the qualia issue &lt;em&gt;in principle&lt;/em&gt; prevents a &quot;real&quot; understanding of language; its just a matter of extending the symbols available to the AI for modeling the world represented by language to be a good enough match for the symbols humans use in their minds, including the symbols that arise from sensory inputs.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2017-05-07T22:02:59.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3281" PostTypeId="2" ParentId="3272" CreationDate="2017-05-08T02:54:32.700" Score="3" Body="&lt;p&gt;We don't even know &lt;strong&gt;exactly&lt;/strong&gt; what qualia are, so it's hard to say for sure.  But here's what I do think:  a lot of human learning is experiential and is rooted in our interactions with the physical world.  That is, we see,smell, hear, and feel things, we experience gravity and our orientation in the world through kinesthetic awareness, the sense of balance we have, etc.  So while an AI running on a server in a data center might well be &quot;as intelligent&quot; as a human, I don't think it's reasonable to expect it to have the same kind of knowledge and awareness as a human, simply because it has never experienced many things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you want to talk about, say, &quot;seeing the color red&quot; and refer to qualia, then sure.  I think it makes a certain kind of sense to say that the machine will be missing something &quot;human&quot; and that that refers to qualia.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OTOH, I think it would be a mistake to underestimate just how &quot;intelligent&quot; our AI's will eventually become even if they aren't embodied. We just have to keep in mind that their intelligence might not be quite the same as ours, because they essentially inhabit a different world. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-05-08T02:54:32.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3282" PostTypeId="1" AcceptedAnswerId="3286" CreationDate="2017-05-08T08:32:12.210" Score="3" ViewCount="3305" Body="&lt;p&gt;I would like to use deep leaning for identifying cars; I want the system to predict wether an object is a car or not. How can I do that knowing that im still a beginner in the Deep Learning field ?&#xA;I am considering visual recognition. The system must recognize the car from anything else on the road.&lt;/p&gt;&#xA;" OwnerUserId="7075" LastEditorUserId="3836" LastEditDate="2017-07-23T20:35:31.120" LastActivityDate="2017-07-23T20:35:31.120" Title="Identifying cars using deep learning" Tags="&lt;deep-learning&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3283" PostTypeId="2" ParentId="3272" CreationDate="2017-05-08T10:05:51.660" Score="5" Body="&lt;p&gt;I'm going to be controversial here; so please don't vote this answer down if you just disagree with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question presupposes that machines do not or cannot possess qualia, which are required for true understanding. Given that we don't really know what it means to 'understand' something, and that even the meaning of 'meaning' itself is by no means a resolved issue, it might be overly specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one strand of linguistics, the meaning of a word is defined by its use, and by the context of surrounding words. We could hazard a guess that children acquire the meaning of words through exposure to language, and the correlation of experiences with the corresponding sounds. How that works in detail is AFAIK not fully understood. But there would be nothing 'inherent' in a new-born human that would enable it to 'understand' anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is the case, then we could train a machine to do the same. Obviously, it would be a long and tedious process, and there is probably a reason why it takes us years to become proficient in our use of language. But if we correlate sensory input with linguistic utterances, a sufficiently sophisticated learning algorithm might be able to acquire some meaning for such utterances from the way they are used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, of course, rather a lot of unknowns here. That is because the topic straddles various fields, from child language acquisition, corpus linguistics, the psychology of learning, and many more. And to my knowledge, none of these fields is sufficiently advanced to shed any light on this issue yet. There is the whole question of abstract words and concepts. How do we segment the continuous stream of sounds into discrete units (&lt;em&gt;phonemes&lt;/em&gt;) without knowing what they are? With all that complexity I begin to appreciate why Chomsky opted for his Language Acquisition Device to avoid getting frustrated... :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question: yes, it should be possible. A properly set up machine, which would be able to simulate human learning, would pick up its own mapping of linguistic structures to its experience from the world outside. And if we call this mapping the 'meaning' of those structures, then a machine can learn this, and presumably 'understand' language. If we ever get to that stage with AI is another question.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2017-05-08T10:05:51.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3286" PostTypeId="2" ParentId="3282" CreationDate="2017-05-08T11:15:35.990" Score="1" Body="&lt;p&gt;That's definitely possible with &lt;strong&gt;Convolutional Neural Networks&lt;/strong&gt;. You should follow Google's tutorial at:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/tutorials/deep_cnn#cifar-10_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/deep_cnn#cifar-10_model&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The tutorial will teach you how to build a deep learning model for classifying cars, trucks etc. It's an easy tutorial to follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your model will build on images pixels. The classifier will know what a car looks like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5BEx4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5BEx4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-05-08T11:15:35.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3287" PostTypeId="1" AcceptedAnswerId="3289" CreationDate="2017-05-08T16:08:26.650" Score="3" ViewCount="4943" Body="&lt;p&gt;I am looking at a &lt;a href=&quot;https://adeshpande3.github.io/assets/zfnet.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;diagram&lt;/a&gt; of ZFNet in an attempt to understand how CNNs are designed effectively. I'm working with the CIFAR10 set in pytorch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first layer, I understand the depth of 3 (224x224x3) is the number of color channels in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second layer I understand the 110x110 is is (224 - ( 7 * 2)) / 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also understand how pooling works to create a size reduction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But where does the depth of 96 come from in the second layer? Is this the new &quot;batch size&quot;? Is it totally arbitrary? Bonus points if someone can direct me to a reference that can help me understand how all these dimensions relate to each other.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2018-04-22T19:59:32.070" Title="How is the depth of a CNN layer determined?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3288" PostTypeId="1" AcceptedAnswerId="3555" CreationDate="2017-05-08T23:34:09.467" Score="3" ViewCount="13199" Body="&lt;p&gt;For a classification task (I'm showing a pair of exactly two images to a CNN that should answer with 0 -&gt; fake pair or 1 -&gt; real pair) I am struggling to figure out how to design the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment the network's architecture looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;image-1                       image-2&#xA;   |                             |&#xA;conv layer                    conv layer&#xA;   |                             |&#xA;   _______________ _______________&#xA;                  |&#xA;            flattened vector&#xA;                  |&#xA;          fully-connected layer&#xA;                  |&#xA;           reshape to 2D image&#xA;                  |&#xA;              conv layer&#xA;                  |&#xA;              conv layer&#xA;                  |&#xA;              conv layer&#xA;                  |&#xA;            flattened vector&#xA;                  |&#xA;                output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The conv layers have a &lt;code&gt;2x2&lt;/code&gt; stride, thus halfing the images' dimensions. I would have used the first fully-connected layer as the first layer, but then the size of it doesn't fit in my GPU's VRAM. Thus, I have the first conv layers halfing the size of the images first, then combining the information with a fully-connected layer and then doing the actual classification with conv layers for the combined image information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My very first idea was to simply add the information up, like &lt;code&gt;(image-1 + image-2) / 2&lt;/code&gt;...but this is not a good idea, since it heavily mixes up image information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next try was to concatenate the images to have one single image of size 400x100 instead of two 200x100 images. However, the results of this approach were quite unstable. I think because in the center of the big, concatenated image convolutions would convolve information of both images (right border of &lt;code&gt;image-1&lt;/code&gt; / left border of &lt;code&gt;image-2&lt;/code&gt;), which again mixes up image information in not really senseful way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My last approach was the current architecture, simply leaving the combination of &lt;code&gt;image-1&lt;/code&gt; and &lt;code&gt;image-2&lt;/code&gt; up to one fully-connected layer. This works - kind of (the results show a nice convergence, but could be better).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a reasonable, &quot;state-of-the-art&quot; way to combine two images for a CNN's input?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I clearly can not simply increase the batch size and fit the images there, since the pairs are related to each other and this relationship would get lost if I simply feed just one image at a time and increase the batch size.&lt;/p&gt;&#xA;" OwnerUserId="7095" LastActivityDate="2019-04-18T23:44:20.160" Title="How to &quot;combine&quot; two images for CNN input (classification task)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;" AnswerCount="3" CommentCount="6" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3289" PostTypeId="2" ParentId="3287" CreationDate="2017-05-08T23:56:28.040" Score="2" Body="&lt;p&gt;The 96 is the amount of &lt;em&gt;filter maps&lt;/em&gt; (also: filter &lt;em&gt;kernels&lt;/em&gt;). It is a fundamental of convolutional neural networks. The exact number is not arbritary, although there is no equation or exact rule of restricting the number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a CNN one single convolution operation would be pointless: since it used for the whole image information it can generalize, but only to specific (meaning: finite amount of) features. Easy example: if a 7x7 filter in the first layer concentrates on round shapes, it can not generalize on let's say red cubes at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you convolutional layers have several filter kernels, i.e. several weights where each is used for a convolution. The result of each of these convolutions is one filter map, i.e. the image information convolved by a kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically you look at your filter kernels and your problem's domain to figure out what an appropriate number of filter kernels could be. You also have to keep in mind that too few kernels could possibly lose information and overfit to specific patterns, while too many kernels could possibly underfit. Especially when you have far more parameters (primarily the weights of your network) than training data your network will normally perform bad and you have to reduce its size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The filter kernels should not be confused with batch size. The batch size is the amount of samples (here: images) you train &lt;em&gt;in parallel&lt;/em&gt;. Each training step of your network does not only consist of feeding a single image, but feeding &lt;em&gt;batch size&lt;/em&gt; number of images, usually combined with batch normalization steps between the layers. Hence, this has nothing to do the amount of filter kernels / convolutions of your network.&lt;/p&gt;&#xA;" OwnerUserId="7095" LastActivityDate="2017-05-08T23:56:28.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3290" PostTypeId="1" AcceptedAnswerId="4393" CreationDate="2017-05-09T02:09:17.800" Score="2" ViewCount="418" Body="&lt;p&gt;From what I have understood reading the UCT paper &lt;a href=&quot;http://ggp.stanford.edu/readings/uct.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit based monte-carlo planning&lt;/a&gt;, by Levente Kocsis and Csaba Szepesvári, MCTS/UCT requires a generative model. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does it mean that, in case there is no generative model of the environment, we cannot use MCTS?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we can still use MCTS, how does the roll-out happen in this case, as there is no simulation? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7098" LastEditorUserId="2444" LastEditDate="2019-11-18T21:53:08.373" LastActivityDate="2019-11-18T21:53:08.373" Title="Can we use MCTS without a generative model?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3291" PostTypeId="1" AcceptedAnswerId="3294" CreationDate="2017-05-09T04:57:47.583" Score="8" ViewCount="932" Body="&lt;p&gt;I am an Android programmer. Now, I would like to learn machine learning. I know it requires a mathematical background, like statistics, probability, calculus and linear algebra. However, I am a bit lost. Where should I start from? Can someone provide me a road map for how to learn the mathematical background required for machine learning?&lt;/p&gt;&#xA;" OwnerUserId="7101" LastEditorUserId="2444" LastEditDate="2019-06-02T16:24:01.697" LastActivityDate="2020-02-24T18:01:24.230" Title="How can I start learning mathematics for machine learning?" Tags="&lt;machine-learning&gt;&lt;math&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3292" PostTypeId="1" AcceptedAnswerId="3293" CreationDate="2017-05-09T06:10:37.353" Score="1" ViewCount="80" Body="&lt;p&gt;The inputs (features) and expected output for my ANN are these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input 1: Product id (number, cast to double)&lt;/li&gt;&#xA;&lt;li&gt;Input 2: Year in the past (1900..2017, cast to double)&lt;/li&gt;&#xA;&lt;li&gt;Input 3: Month of year (1..12, cast to double)&lt;/li&gt;&#xA;&lt;li&gt;Expected output: Sale of month (number of units sold, cast to double)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I need to predict the sale of a product for a certain month in a certain year. &lt;strong&gt;How many layers and how many neurons on there layers should I put&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-05-13T20:33:36.467" LastActivityDate="2020-05-13T20:33:36.467" Title="How many layers and neurons on the layers does a neural network need for for sales prediction?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;prediction&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3293" PostTypeId="2" ParentId="3292" CreationDate="2017-05-09T11:04:43.117" Score="1" Body="&lt;p&gt;Every answer you get is just an opinion, it is based on experience.&#xA;My answer is a single hidden layer of 5 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, I recommend you to use a TWEANN system (Topology and Weight Evolving ANN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such system applys genetic algorithm to search and optimize an ANN with the optimal topology and weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at &lt;a href=&quot;http://eplex.cs.ucf.edu/neat_software/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt; and &lt;a href=&quot;https://github.com/CorticalComputer/DXNN2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DXNN&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2017-05-09T11:04:43.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3294" PostTypeId="2" ParentId="3291" CreationDate="2017-05-09T11:11:36.363" Score="5" Body="&lt;p&gt;You should begin from Dr Andrew Ng machine learning course on Coursera. It's probably the most popular course for newcomers in machine learning. It's a free course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also grab &quot;Elements of Statistical Learning&quot; ebook PDF. It's a free book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to focus on:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Regression&lt;/li&gt;&#xA;&lt;li&gt;Cross validation&lt;/li&gt;&#xA;&lt;li&gt;Bias-variance tradeoff&lt;/li&gt;&#xA;&lt;li&gt;Decision surface&lt;/li&gt;&#xA;&lt;li&gt;Gradient descent&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And more...&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="2444" LastEditDate="2019-06-02T16:25:34.160" LastActivityDate="2019-06-02T16:25:34.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3295" PostTypeId="1" CreationDate="2017-05-09T11:40:44.700" Score="3" ViewCount="1057" Body="&lt;p&gt;I implemented the Actor-Critic with n-step TD prediction to learn to play &lt;a href=&quot;http://2048game.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the 2048 game&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the environment, I don't use &lt;a href=&quot;https://2048game.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this 2048 implementation&lt;/a&gt;. I use a simple one without any graphical interface, just pure matrices. The input for the neural network is the log2 of the game board.&lt;/p&gt;&#xA;&lt;p&gt;The structure of my neural network is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Input layer&lt;/li&gt;&#xA;&lt;li&gt;Hidden layer with 16 units&lt;/li&gt;&#xA;&lt;li&gt;Softmax layer with 4 units (up, down, left, right) for the actor&lt;/li&gt;&#xA;&lt;li&gt;Linear regression for the critic&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The hidden layer is shared between the actor and critic, so the same hidden layer (point 2) is connected to both the softmax layer of the actor and the linear regression layer of the critic.&lt;/p&gt;&#xA;&lt;p&gt;The reward in the original game is the value of the merged cells. For example, if two 4s merged, then the reward is 8. My reward function is almost the same, except I take the log2 of it.&lt;/p&gt;&#xA;&lt;p&gt;I tried these parameters and I also tweaked the learning rate, the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, but I couldn't achieve any good result.&lt;/p&gt;&#xA;&lt;p&gt;Could you recommend what should I change?&lt;/p&gt;&#xA;" OwnerUserId="6019" LastEditorUserId="2444" LastEditDate="2020-12-12T13:16:24.197" LastActivityDate="2020-12-12T13:16:24.197" Title="Why can't my implementation of the Actor-Critic algorithm achieve good results in the 2048 game?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;deep-rl&gt;&lt;actor-critic-methods&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3296" PostTypeId="1" AcceptedAnswerId="3410" CreationDate="2017-05-09T13:08:34.313" Score="6" ViewCount="437" Body="&lt;p&gt;I'm trying to learn about neural networks, and I'm interested in gaining a better conceptual understanding of how they work to solve certain problems. I'm having trouble in conceptually understanding how they succeed in doing regression (i.e. predicting continuous variables), however, and wondered if anybody has a good explanation. I know the mathematics of how NNs work, but a clearer conceptual understanding would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an idea of what I mean by a &quot;conceptual understanding&quot;, here's the one that I have for how multilayer NN's with a sigmoid activation function are able to be effective classifiers. Each NN takes the scalar product of its inputs and a set of weights. The weights define a plane in the input space, and the sign of the scalar product indicates which side of the plane the point defined by the inputs is on. The sigmoid activation function outputs 1 if the point is on one side and 0 or -1 (depending on which function is used) if the point is on the other. So the first hidden layer of neurons can be considered to identify which side of each of a group of planes the input point is on. The neurons can also act as AND and OR gates, so subsequent layers of neurons give an output indicating whether the point lies in a region bounded by several of the planes (e.g. a neuron activates only if the point is above one plane and below another, indicating it is in a region of the space associated with one class of points). So if the network learns an appropriate set of planes to bound regions containing different classes of inputs and the AND/OR relations that determine whether a point is in a given region, then it can classify the input points, and this can work for regions with arbitrarily shaped boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've not found or been able to think of a similar way of explaining why an NN can perform well in general regression problems (if it's big enough). Does anyone here know a way to explain this?&lt;/p&gt;&#xA;" OwnerUserId="7107" LastActivityDate="2020-03-07T22:52:46.473" Title="How do neural networks manage to do regression?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3297" PostTypeId="2" ParentId="3295" CreationDate="2017-05-09T15:27:49.717" Score="1" Body="&lt;p&gt;Interesting project. First thing I'd do is normalize your state by the maximum cell value. This way you can represent multiple situations at once (eg A grid of all 4s and 8s would look the same as a grid of all 16s and 32s).&#xA;Also making the reward = max_cell/2048 might do better as ActorCritic methods seem to do better with rewards within 0-1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another reward setup is giving +1 per timestep. It's simple but it also means that maximizing the time to stay alive is the best, which is what I end up doing most of the time when I play anyway. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-05-09T15:27:49.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3298" PostTypeId="1" AcceptedAnswerId="7489" CreationDate="2017-05-09T17:28:22.217" Score="2" ViewCount="231" Body="&lt;p&gt;I'm trying to develop a kind of AI that will assist in debugging a large software system while we run test cases on it. The AI can get access to anything that the developers of the system can, namely log files, and execution data from trace points. It also has access to the structure of the system, and all of the source code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The end goal of this AI is to be able to detect runtime errors during execution, and locate the source of these errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was considering making use of a deep neural network, where the input would be the execution data and log output. Using this input it would be able to verify whether the current version of the system we are running is functional, or non-functional. The problems with this approach is that the system it would be evaluating would be constantly changing as it gets developed, so the only training material the NN would have is from the last stable version of the system (and even that could have some errors). Additionally, producing test cases for the system off of which we could train the NN would be very time consuming, and would defeat the purpose of using the NN in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know what AI design you think would be suitable for this task. Please let me know if you would like any other information relevant to the problem. As far as I can tell, nothing quite like this has been done before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's probably worth mentioning that my team has a some extremely powerful machines on which we can run the AI.&lt;/p&gt;&#xA;" OwnerUserId="7083" LastActivityDate="2018-09-17T18:05:10.213" Title="What type of AI would you recommend for this complex problem?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3300" PostTypeId="2" ParentId="3291" CreationDate="2017-05-10T07:54:42.420" Score="1" Body="&lt;p&gt;If you are interested to deepen your statistical concepts before diving into machine learning, i would recommend &lt;strong&gt;Introduction to Statistics: Descriptive Statistics&lt;/strong&gt; course in &lt;strong&gt;edX&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you'll learn&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The fundamental concepts and methods of statistics&lt;/li&gt;&#xA;&lt;li&gt;How to intepret graphical and numerical summaries of data&lt;/li&gt;&#xA;&lt;li&gt;Understand the reasoning behind the calculations, the assumptions under which    they are valid, and the correct interpretation of results&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The link for course is &lt;a href=&quot;https://www.edx.org/course/introduction-statistics-descriptive-uc-berkeleyx-stat2-1x&quot; rel=&quot;nofollow noreferrer&quot;&gt;edX&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will definitely clarify your stat background with added benefit of certification.&lt;/p&gt;&#xA;" OwnerUserId="6897" LastActivityDate="2017-05-10T07:54:42.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3301" PostTypeId="1" CreationDate="2017-05-10T12:39:17.430" Score="1" ViewCount="88" Body="&lt;p&gt;I'm working on a project where I train a Q-learning agent to learn an optimal control policy for a water heater. I've set up a simulation which allows the agent to explore for one year. I then examine the results of the agent performance exploiting its optimal policy for the following year. The agent can perform the following actions (available actions depend on the state of the environment):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Turn the electrical heating element on.&lt;/li&gt;&#xA;&lt;li&gt;Turn the electrical heating element off.&lt;/li&gt;&#xA;&lt;li&gt;Turn gas heating on.&lt;/li&gt;&#xA;&lt;li&gt;Turn gas heating off.&lt;/li&gt;&#xA;&lt;li&gt;Do nothing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The goal of the agent reach the target temperature (50 deg C) when hot water is scheduled. The agent is rewarded for choosing actions which produce the lowest CO2 emissions (the CO2 emissions produced from electricity vary over time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the issues I have noticed is that during the exploration phase, the agent tries a lot of weighted random actions which causes the water heater to overheat (&gt;80 deg C). When the water heater overheats, it is not possible for the agent to perform further actions other than switching off heating and doing nothing. The agent is also punished for reaching the overheating tank state. The tank may remain in the overheated state for some time. It seems as if the tendency to overheat the tank during exploration is negatively impacting how the agent learns its policy as it reduces the number of experiences in other states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a term for this kind of situation during exploration in reinforcement learning? During exploration, the agent uses a chooses a softmax weighted random aciton. Are there alternative ways of choosing actions that may still allow for exploration while not reaching the overheating state?&lt;/p&gt;&#xA;" OwnerUserId="7127" LastEditorUserId="7127" LastEditDate="2017-05-10T22:34:35.577" LastActivityDate="2017-05-10T22:34:35.577" Title="Agent exploration which leads to a negative state where actions are limited" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3302" PostTypeId="1" CreationDate="2017-05-10T20:07:49.943" Score="5" ViewCount="711" Body="&lt;p&gt;I understand that there are flavors of (convolutional) neural networks that are useful for object &lt;a href=&quot;https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;localization and detection&lt;/a&gt; tasks of reasonable difficulty. In all of the examples I have seen so far, localization is formulated as finding the corners of a bounding box. Often, the fit is not expected to be very precise:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xmWck.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xmWck.jpg&quot; alt=&quot;traffic sign bounding box&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, I am interested in a task I want to achieve very precise localization and characterization of some simple shapes or objects. As an example of one of the simplest cases I can think of, my inputs will be images like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NOqN5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NOqN5.jpg&quot; alt=&quot;28 21 15&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this 60x60 image, I want my neural net(s), via &lt;strong&gt;regression&lt;/strong&gt;, to tell me that the circle's &lt;strong&gt;diameter is 18px&lt;/strong&gt; and its &lt;strong&gt;centre is located at (28, 21)&lt;/strong&gt; from top left. (I will train it using similar 60x60 images with white circles of various sizes on black backgrounds.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later I am interested in dealing with similar tasks in the real world, e.g. spheres/cubes/cylinders with different viewing angles, light conditions, occlusions, etc. However, I am interested in solving this simplest case first. (One reason is that I can generate this data very easily.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following specific questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Has anyone used neural nets for this sort of tasks before? (e.g. precisely determining sizes and centroids of objects)&lt;/li&gt;&#xA;&lt;li&gt;My understanding is that these things are at least theoretically possible using convolutional nets, or even sufficiently complicated vanilla fully connected nets. Is this correct? &lt;/li&gt;&#xA;&lt;li&gt;What architecture(s) would be appropriate for these tasks?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note: I am aware that fitting a bounding box to the circles and calculating its centre and size will solve this particular case, but it will not generalize to handle occlusions, changing lights, etc. I would like to move towards a method which can, for example, calculate the centroids and diameters of spheres in real-world B&amp;amp;W photos.&lt;/p&gt;&#xA;" OwnerUserId="7132" LastEditorUserId="7132" LastEditDate="2017-05-15T07:38:09.063" LastActivityDate="2017-08-20T23:11:33.960" Title="Precise localization and characterization of rudimentary shapes with neural networks" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3304" PostTypeId="2" ParentId="2957" CreationDate="2017-05-11T07:27:54.420" Score="1" Body="&lt;p&gt;In my answer, I have often switched between AGI and ASI for reference. This is fine as an AGI will reach ASI as it is optimizing itself and learning.&lt;br&gt;&#xA;I think it is not only important by necessary that AGI and ASI are of collaborative nature. Nick Bostrom, in his book &lt;strong&gt;Superintelligence: Paths, Dangers, Strategies&lt;/strong&gt; in Chapter 10 described three ways in which an ASI might function:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;As an oracle&lt;/strong&gt;, which answers nearly any question posed to it with accuracy, including complex questions that humans cannot easily answer—i.e. How can I manufacture a more efficient car engine? Google is a primitive type of oracle.&#xA;      &lt;br&gt;&lt;strong&gt;As a genie&lt;/strong&gt;, which executes any high-level command it’s given—Use a molecular assembler to build a new and more efficient kind of car engine—and then awaits its next command.&#xA;      &lt;br&gt;&lt;strong&gt;As a sovereign&lt;/strong&gt;, which is assigned a broad and open-ended pursuit and allowed to operate in the world freely, making its own decisions about how best to proceed—Invent a faster, cheaper, and safer way than cars for humans to privately transport themselves.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is when and if AGI and ASI are of controlled manner and their output are as expected. By controlled I mean they don't start seeing human as a threat and start eliminating the human race. More on this &lt;a href=&quot;http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. As you can see, all the above ASI are of collaborative nature. Either they are collaborating with humans or they need to collaborate with other systems.&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to your answer:&lt;br&gt;&#xA;First, collaborative nature is of great use in terms of &lt;em&gt;efficiency and performance&lt;/em&gt;. This is the reason why &lt;strong&gt;Distributed Systems&lt;/strong&gt; are being made. We even have distributed OS now. Also, Modular approach in coding/ developments and huge success of &lt;em&gt;Object Oriented Model&lt;/em&gt; are proofs of advantages of using collaboration among different entities.&lt;br&gt;&#xA;If you think about it even the AGI is collaborating and using resources from other places in some way. As your AGI is learning, it is gaining information from the internet. It reads the information and tries to structure is accordingly (This will depend on its neural schema) and &lt;em&gt;create knowledge&lt;/em&gt; (or something valuable) for itself. It is collaborating using network protocols with the outer world (other systems). If it doesn't collaborate then the firewall of the system might not allow it to use the service. Different services on the internet require a different set of protocols to be followed. So, if AGI wants to communicate for information it needs to follow those protocols. This way the AGI will learn to collaborate with different entities even at an early stage.&lt;br&gt;&#xA;From AGI point of view communicating a web server and communicating another AGI machine is very similar. As we, humans, don't store all the information in our brain; similarly AGI won't find it efficient to store all the information within. Not all information is needed all the time. The &lt;strong&gt;memory hierarchy&lt;/strong&gt; is proof of that. Even if AGI is made to store all the information within, with time it will figure out how inefficient it is and will re-program itself to only hold very vital information and use the internet for less frequently used information.&lt;br&gt;&lt;br&gt;&#xA;I will like to add one more thing to this. Let us begin with the human analogy first. We, as humans, collaborate with other humans. But we also collaborate with our different body parts. Like, what about the collaboration within us. Collaboration (or coordination) between our legs, hands, our body, and mind, etc. This leads us to the question of who as a person we are, our whole body, or just our brain. If I remove my hand and replace it with an organic implant, would I be still me?&lt;br&gt;&#xA;Similarly, what is AGI? Is it the whole structure, or just the code. Does the RAM, ROM, hard disks also a part of AGI? If you think the hardware is not the part of AGI then your AGI is co-ordinating with these devices too, using certain protocols. To some level, this is collaboration too.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-05-11T07:27:54.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3305" PostTypeId="2" ParentId="3226" CreationDate="2017-05-11T07:51:59.637" Score="0" Body="&lt;p&gt;I don't think they mean that &quot;known&quot; is equivalent to &quot;entailed&quot; --- in a reasonably complicated system, one cannot be expect to know every sentence which is entailed. Perhaps their example is just a bit lacking.&lt;/p&gt;&#xA;" OwnerUserId="7145" LastActivityDate="2017-05-11T07:51:59.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3310" PostTypeId="1" AcceptedAnswerId="3311" CreationDate="2017-05-14T00:00:46.350" Score="3" ViewCount="359" Body="&lt;p&gt;Can or will automata love? If so, is there a theoretical limit to love? Conversely, can automata develop selfishness? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many resources will automata devote to being selfish and helping others?&lt;/p&gt;&#xA;" OwnerUserId="7184" LastEditorUserId="2444" LastEditDate="2020-03-09T20:16:08.873" LastActivityDate="2020-03-09T20:16:08.873" Title="Will automata love?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;&lt;affective-computing&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3311" PostTypeId="2" ParentId="3310" CreationDate="2017-05-15T08:35:36.977" Score="4" Body="&lt;p&gt;It's a poorly stated question because these are at least three, possibly four different questions that are quite independent from each other.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;First, let's take the questions from the text. Selfishness vs generosity of the system - this is quite easy to define as sacrificing resources for &quot;own maintenance&quot; vs &quot;statutory purposes&quot; - &quot;helping others&quot; defined as fulfilling the role assigned by the creators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all a matter of developing the goal function that gives sufficient priority to immediate or short term results. In the classic example of &quot;&lt;a href=&quot;https://wiki.lesswrong.com/wiki/Paperclip_maximizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;paperclip maximizer&lt;/a&gt;&quot;, without this sort of restriction the end result will be not even a single paper clip made, as the AI expands self to develop even more efficient way to create the paper clips, since current, inferior methods will waste resources which could better serve later, more advanced methods - the classic paradigm &quot;better is the enemy of good&quot;, all focus on optimizing the process without actually performing the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is that we don't know what type of function the improvement due to the optimization process will follow, and so we don't quite know what sort of prioritization function to apply. Probably a flat deadline would work, but if that is movable, AI might sacrifice all resources to developing a way to move the deadline. Be it by resetting system clock, terrorizing system operators into adjusting the deadline, or inventing time travel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, selfishness vs generosity is a matter of developing the correct time constraints on the goal function. Difficult, but doable - without it AI will be &quot;infinitely selfish&quot;, but over-constrained, it will be inefficient, producing inferior immediate result instead of developing better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably, a good approach would be to predict a reasonable (both achievable and desirable) advancement path and set this as the goal value - not maximization of output, but approaching the desired output value (which may be growing over time, linearly or exponentially) and freeing up unused resources - adding a cap on expansion speed. That would avoid the paperclip universe horror scenario.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, for love. Again, this depends largely on the goal function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining love, that would be a specific mental state, influenced by specific hormones, and leading to specific re-shuffling of priorities, often resulting in impaired judgment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defined that way, could AI love? Yes. Should it? uh, better no. Because, &quot;What will be the limit to love? Any theoretical limit?&quot; - about none, bar using up all matter and energy in the universe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Love pretty much unconditionally favors the &quot;beloved&quot;. And that is just another phrasing for &lt;em&gt;maximization&lt;/em&gt;. And, as we all know, an unconstrained maximizer AI is bad news. You don't want &quot;Give you the Moon and all the stars&quot; to be a completely literal expression. So, love - while possible both to implement, and to happen as a side effect of poorly constrained other goals (like &quot;learn to feel all that a human can feel&quot;) can be a serious problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yeah, regarding AI and love: can we? Yeah. Should we? nope.&lt;/p&gt;&#xA;" OwnerUserId="38" LastActivityDate="2017-05-15T08:35:36.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3312" PostTypeId="1" AcceptedAnswerId="3318" CreationDate="2017-05-15T09:53:06.317" Score="4" ViewCount="166" Body="&lt;p&gt;Most (all I know) machine learning systems use a fixed set of data input channels and processing algorithms, only expanding underlying dataset processed by these; they obtain new data but only from predefined sources, and use only their fixed, built-in capacity to process it, possibly tweaking parameters of the algorithm (like weights of neural network nodes) but never fundamentally changing the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there systems - or research into creating these - that are able to acquire &quot;from out there&quot; new methods of obtaining data and new ways to process it for results? Expand not just passive data set to &quot;digest it&quot; by active but static algorithm, but make the algorithm itself self-expanding - be it in terms of creating/obtaining new processing methods for own data set, and creating/obtaining new methods of acquiring that data (these methods)?&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="2444" LastEditDate="2019-06-25T22:30:27.060" LastActivityDate="2019-06-25T22:30:27.060" Title="What's done towards AI learning new ways of learning?" Tags="&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3313" PostTypeId="1" AcceptedAnswerId="3314" CreationDate="2017-05-15T11:33:39.930" Score="2" ViewCount="693" Body="&lt;p&gt;I was able to extract the license plate from a given car image, using Matlab. I would like to use deep neural networks to recognize the characters on the plate now. How can i proceed further? I don't have any experience with deep neural networks implementation.&lt;/p&gt;&#xA;" OwnerUserId="7075" LastEditorUserId="7075" LastEditDate="2017-05-15T20:14:16.203" LastActivityDate="2017-05-15T20:14:16.203" Title="How can I use deep neural networks to recognize characters on vehicle license plate?" Tags="&lt;neural-networks&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ClosedDate="2017-05-15T18:07:05.400" ContentLicense="CC BY-SA 3.0" />
  <row Id="3314" PostTypeId="2" ParentId="3313" CreationDate="2017-05-15T13:27:20.490" Score="0" Body="&lt;p&gt;Before jumping into any machine learning task it is good to consider your dataset and the &lt;strong&gt;features&lt;/strong&gt; you wish to use. License plates are unique thus feeding the entire plate into a machine learning algorithm would not yield very accurate results. First, you will want to make sure that the extracted license plate images are similar in size. Then you will want to separate the license plate such that you can separate the different digits and letters into their own respective images. Thus your dataset that will be used to train your network will consists of many single character images which contain either a letter or a digit. When you have finished training your model, future license plates will also have to be broken down in this way, classified and then the outputs be combined to get the license plate number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is a neural network?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A neural network (NN)&lt;/strong&gt; is a series of nodes which contain a simple, continuous, differentiable function such as logistic regression. Alone these nodes are not very powerful, however, the beauty comes in when they are connected in networks. The tuning of such a network can learn highly complex functions. Typically, a NN is set up with three layers. &lt;strong&gt;The input layer&lt;/strong&gt;, where you will feed in your features, &lt;strong&gt;the hidden layer&lt;/strong&gt; and the &lt;strong&gt;output layer&lt;/strong&gt;. Naturally, if you want a &lt;strong&gt;deep NN&lt;/strong&gt; then you just need to add more hidden layers. But, there is a trap. The deeper your network is the harder it will be to train, thus you will need MORE data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The input-layer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where you will be feeding in your images. If you have a 16*16 pixel image then you will have 256 input nodes. This layer typically does not perform any function on the input, it is simply taking in the input. It will then feed the input to the hidden layer. You can use some dimensionality reduction techniques to try and eliminate some useless information from your dataset such as the white spaces in the corner of the image. The less input nodes you have, the more informative your samples become, thus with higher information entropy your network will be able to learn faster, or more rigidly with the same amount of data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The hidden layers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are free here to choose how many layers you want and how many nodes you want in each layer. I suggest using &lt;strong&gt;grid search&lt;/strong&gt; and &lt;strong&gt;cross-validation&lt;/strong&gt; as a means to determine which configuration is ideal for your dataset. Too many layers and nodes and you will not end up with a fully trained networks once you run out of training data (too much bias). Too little nodes, your network will not be able to learn the subtleties which are embedded within the data (too much variance). You want to find yourself at the crossroad where these two are minimized. Choose wisely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The output layer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This layer can be configured in a few different ways. Typically people like to do one output node for each class. In your case there are 26 letters in the alphabet and 10 digits. This gives 36 possible output classes. So you will need 36 nodes in your output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I train this?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training of the NN is by tuning the weights that will be applied to the inputs of each of your nodes. Try to draw your dense network, for each line connecting nodes you will have a weight. I am sure you can see how this becomes quite huge. You will &lt;strong&gt;initialize your weights randomly&lt;/strong&gt; to start. Then you will take each of your examples and pass them through your network. This is called a &lt;strong&gt;feed-forward&lt;/strong&gt; pass through. This will provide you with an output. You will compare your output with your ground truth value using a &lt;strong&gt;cost function&lt;/strong&gt;. This is often the &lt;strong&gt;root mean squared error (RMSE)&lt;/strong&gt; but there are many other options. Then using the derivative you will &lt;strong&gt;back-propagate&lt;/strong&gt; the error through your network to see which weights caused the error and to what extent. The derivative will then allow you to update those weights proportionately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;WOW do I need to program all that?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nope not anymore!!! You can use &lt;strong&gt;Keras in python&lt;/strong&gt; (my recommendation) or other similar frameworks which have made it a real tea sipping pleasure to program a variety of NN configurations. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My recommendation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start with a simple NN and see what kind of accuracy I can achieve. From there I would try to use convolutional neural networks which have been shown to garner much better results on datasets comprised of images. These can also be implemented in Keras. CNN require more data than typical NN thus you might need to get access to more images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is always best to train your dataset only on images you have in your original data. However, when it comes to images, adding to your data by artificially shifting images by 1-5 pixels or rotating by 1-5 degrees can add variability which can supplement your original data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make sure that the license plate pictures you are using to train your network are taken with a similar system as the images you will be using in the future. You will not get very good results if you train your network on images taken directly behind cars and then you expect to implement your solution in traffic cameras monitoring traffic from above. &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-05-15T13:27:20.490" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3316" PostTypeId="2" ParentId="3312" CreationDate="2017-05-15T16:08:11.203" Score="2" Body="&lt;p&gt;I think the closest thing is building up knowledge using predictions like in the &lt;a href=&quot;https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/horde1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Horde Architecture&lt;/a&gt;. Research about what are good predictions to make is on going at the University of Alberta, Canada but the Horde architecture has the potential to ask new questions and generate new data based on the answers to those questions in the form of predictions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example is having a robot with a bump sensor and servos and asking questions like &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will rotating the servos CW for 90 degrees make the bump sensor turn on?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This question can be phrased and it's answer estimated mathematically in the form of a General Value Function as defined in the paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asking different questions based on predictions like this one is where the power comes from. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-05-15T16:08:11.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3318" PostTypeId="2" ParentId="3312" CreationDate="2017-05-16T00:47:23.113" Score="1" Body="&lt;p&gt;A few weeks ago, I've come across this paper &lt;a href=&quot;https://arxiv.org/abs/1606.04474&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to learn by gradient descent by gradient descent&lt;/a&gt; by Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford and Nando de Freitas (i.e. Deepmind guys) whose abstract is the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The move from hand-designed features to learned features in machine learning has been wildly successful. In spite of this, optimization algorithms are still designed by hand. In this paper we show how the design of an optimization algorithm can be cast as a learning problem, allowing the algorithm to learn to exploit structure in the problems of interest in an automatic way. Our learned algorithms, implemented by LSTMs, outperform generic, hand-designed competitors on the tasks for which they are trained, and also generalize well to new tasks with similar structure. We demonstrate this on a number of tasks, including simple convex problems, training neural networks, and styling images with neural art.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can find the related Github's repository &lt;a href=&quot;https://github.com/deepmind/learning-to-learn&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I still haven't found the time to read the paper to explain the details of these apparently interesting ideas.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2017-05-16T00:47:23.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3319" PostTypeId="2" ParentId="2932" CreationDate="2017-05-16T10:25:38.073" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Dimension 1 in both shapes must be equal, but are 6 and 16 for 'concat' (op: 'ConcatV2') with input shapes: [?,6,6,64], [?,16,16,64], [?,16,16,32], [?,16,16,32], [].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Pablo's answer is correct. Your problem is that the convolved images (output of conv-layers) must match in spatial dimensionality in order to concatenate them. This makes perfectly sense, because how would you combine images of shape 6x6 with images of shape 16x16? You can not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either you have to ensure that the convolutions produce output of equal spatial dimenions, i.e. using the same padding and strides strategy or you have to use &lt;code&gt;tf.image.resize_images&lt;/code&gt; to down-/upscale the different output to the same spatial dimensionality (or some other down-/upscaling strategy).&lt;/p&gt;&#xA;" OwnerUserId="7095" LastActivityDate="2017-05-16T10:25:38.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3320" PostTypeId="1" AcceptedAnswerId="3331" CreationDate="2017-05-16T10:38:10.210" Score="8" ViewCount="415" Body="&lt;p&gt;I have done some research regarding the application of machine learning to cyber-security. After these recent attacks (like &lt;a href=&quot;https://en.wikipedia.org/wiki/WannaCry_ransomware_attack&quot; rel=&quot;nofollow noreferrer&quot;&gt;WannaCry&lt;/a&gt;), I think that AI-based cyber defense can prevent them. I have also read about research regarding the same in MIT, and that AI can detect more than 80% of malware. Is AI actually so promising in this department?&lt;/p&gt;&#xA;" OwnerUserId="6508" LastEditorUserId="2444" LastEditDate="2019-11-22T22:12:18.023" LastActivityDate="2019-11-22T22:12:18.023" Title="Can AI stop attacks like WannaCry?" Tags="&lt;applications&gt;&lt;cyberterrorism&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3321" PostTypeId="1" CreationDate="2017-05-16T10:47:40.513" Score="8" ViewCount="5911" Body="&lt;p&gt;Obviously, finding suitable hyper-parameters for a neural network is a complex task and problem or domain-specific. However, there should be at least some &quot;rules&quot; that hold most times for the size of the filter (or kernel)!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, intuition should be to go for small filters for detecting high-frequency features and large kernels for low-frequency features, right? For example, &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; kernel filters for edge detection, color contrast, etc., and maybe &lt;span class=&quot;math-container&quot;&gt;$11 \times 11$&lt;/span&gt; for detecting full objects, when the objects occupy an area of roughly &lt;span class=&quot;math-container&quot;&gt;$11 \times 11$&lt;/span&gt; pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this &quot;intuition&quot; more or less generally true? How can we decide which kernel's sizes should be chosen for a specific problem - or even for one specific convolutional layer?&lt;/p&gt;&#xA;" OwnerUserId="7095" LastEditorUserId="2444" LastEditDate="2021-01-29T23:00:26.587" LastActivityDate="2021-01-29T23:00:26.587" Title="How do we choose the kernel size depending on the problem?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;filters&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3323" PostTypeId="2" ParentId="36" CreationDate="2017-05-16T11:59:12.963" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Together with quantum computers,quantum mechanics and Quantum&#xA;  mathematics will change the future of Artificial Intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In current computation cost and limitation the super invention complex number usage is limited,many statistical problems and algorithms are in queue waiting to process and make it in production,Quantum computers are not able to solve it as the current computation error is high,Quantum mathematics won't die and special computation logic will come to tackle this ,More info &lt;a href=&quot;https://github.com/krishnakumarsekar/awesome-quantum-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;available&lt;/a&gt;  &lt;/p&gt;&#xA;" OwnerUserId="7221" LastEditorUserId="1581" LastEditDate="2017-05-28T13:48:37.183" LastActivityDate="2017-05-28T13:48:37.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3325" PostTypeId="1" CreationDate="2017-05-16T16:33:35.703" Score="2" ViewCount="858" Body="&lt;p&gt;I was applying this &lt;a href=&quot;https://www.mathworks.com/help/nnet/examples/transfer-learning-and-fine-tuning-of-convolutional-neural-networks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNN fine-tuning example from Matlab&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example shows how to fine-tune a pre-trained CNN on letters to classify images of digits. Now I would like to use this new fine-tuned CNN on new images of digits that i have on my computer. How can I do that?&lt;/p&gt;&#xA;" OwnerUserId="7075" LastEditorUserId="1671" LastEditDate="2018-01-28T23:06:04.570" LastActivityDate="2018-01-28T23:06:04.570" Title="How can I use a trained CNN to predict a new image label?" Tags="&lt;convolutional-neural-networks&gt;&lt;matlab&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3328" PostTypeId="2" ParentId="3310" CreationDate="2017-05-17T07:21:28.780" Score="0" Body="&lt;p&gt;It depends on the environment and goals, does the AI dies? reproduces? carries genetic information across generations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Love is yet another selfish strategy. And it is a hard-wired reward reflex in our brains to help identify and stick to suitable mates that helps us reproduce, as a consequece, evolution helps shapes and refine the love instinct. This is at least a valid theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, an AI, as an individual is totally selfish, the strategy to express selfishness can vary depending on its goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If adjacent peers doesn't recepricate or protect the AI and it's offsprings, it is futile to share or care for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the AI have all reasons to love for example a data source, that provides the AI with information to survive and reproduce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Moral_Animal&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Moral Animal&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2017-05-17T07:21:28.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3329" PostTypeId="1" AcceptedAnswerId="3381" CreationDate="2017-05-17T09:40:21.940" Score="0" ViewCount="194" Body="&lt;p&gt;In a neural network when inputting nerve input to sense a 2D environment, how do you differentiate two types of objects (with similar shape and size) so the neural network can treat them differently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each neuron in the input layer of a neural network essentially gets 1 dimensional input (range between two values) but 2 dimensional input would be needed to send both collision and category/type information through each input layer neuron. How do you get around that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: After having confusion regarding the scenario / situation I'm asking about compared to other more complex scenarios, and the long comment series that ensued, I'm realizing one challenge of this site is that it's much more complicated and diverse subject matter than code, or the various other topics of Stack Exchange where the problems can be very clearly and simply expressed. Here it's more challenging to express your question and scenario clearly to avoid confusion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also there's probably a higher skill gap between an AI learner / enthusiast, and an expert AI specialist, compared to other fields, so that could potentially lead to even more difficulty communicating the answer / question in ways everyone can understand without confusion. Challenging SE site to ask good questions on!&lt;/p&gt;&#xA;" OwnerUserId="7249" LastEditorUserId="7249" LastEditDate="2017-05-26T12:59:42.367" LastActivityDate="2017-05-26T12:59:42.367" Title="How can object types be differentiated in the input of a neural network?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="14" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3330" PostTypeId="1" AcceptedAnswerId="3332" CreationDate="2017-05-17T15:42:08.583" Score="2" ViewCount="1150" Body="&lt;p&gt;Something I like about neural network AI is that we already have a blueprint for it - we know in great detail how different types of neurons work, and we don't have to invent the AI, we can just replicate what we already know works (neural networks) in a simplified way and train it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's confusing me right now is why many popular neural models I've seen when studying neural networks include both inhibitory and stimulating connections. In real neural networks, from my understanding, there is no negative signal being transferred, rather the signals sent between neurons are comparable to values between 0.1 and 1. There's no mechanic for an inverse (inhibiting) signal to be sent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in this network (seen overlaying a snake being simulated), the red lines represent inhibiting connections, and the blue lines represent stimulating neurons:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nLT4I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nLT4I.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this just an inconsequential detail of neural network design, where there's really no significant difference between a range of 0 to 1 and a range of -1 to 1? Or is there a reason that connections in our simulated neural networks benefit from being able to a express a range from -1 to 1 rather than just 0 to 1?&lt;/p&gt;&#xA;" OwnerUserId="7249" LastEditorUserId="7249" LastEditDate="2017-05-17T15:47:16.320" LastActivityDate="2017-05-18T22:16:12.313" Title="Why are inhibitory connections often used in virtual neural networks when they don't seem to exist in real life neural networks?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3331" PostTypeId="2" ParentId="3320" CreationDate="2017-05-17T15:52:38.477" Score="4" Body="&lt;p&gt;There are projects out there attempting to apply Machine Learning / AI to cyber-security in different ways.  One that I'm familiar with is &lt;a href=&quot;http://metron.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apache Metron&lt;/a&gt;.  Another related project is &lt;a href=&quot;http://spot.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apache Spot&lt;/a&gt;.  I think if you read over the docs for these two projects respectively, they will probably give you some good insights on this subject.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-05-17T15:52:38.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3332" PostTypeId="2" ParentId="3330" CreationDate="2017-05-17T17:32:04.037" Score="2" Body="&lt;p&gt;I think you are confused. The reason why some neural neural networks have neurons with an output in range &lt;code&gt;(-1, 1)&lt;/code&gt; all depends on the activation function used. Some networks even have neurons with an output range of &lt;code&gt;(-Infinity, +Infinity)&lt;/code&gt; ( aka the identity function ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I advise you to take a look at this list: &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;activation functions w/ ranges&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays a lot of neural networks have a &lt;em&gt;mixture&lt;/em&gt; of different activation functions. E.g. combining &lt;code&gt;ReLU&lt;/code&gt; with &lt;code&gt;Sigmoid&lt;/code&gt; or &lt;code&gt;TanH&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/wagenaartje/neataptic&quot; rel=&quot;nofollow noreferrer&quot;&gt;This neural network library&lt;/a&gt; even shows that for most simple problems, neural networks prefer a variety of activation functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red lines just mean that that connection has a low value in comparison with other connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;There is no rule of thumb for activation functions, some problems benefit from a certain function, some don't.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="5344" LastActivityDate="2017-05-17T17:32:04.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3334" PostTypeId="2" ParentId="3329" CreationDate="2017-05-18T07:00:38.150" Score="2" Body="&lt;p&gt;Neural networks learn. That's what they are for. For your task there are two sensible scenarios: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You have a fixed reaction for danger and a fixed reaction for food and you only have to learn how to distinguish between them. In that case you basically try to classify the situation to trigger the right fixed response and this classification would be learned by &lt;a href=&quot;http://neuralnetworksanddeeplearning.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;backpropagation&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You directly learn to act for a given situation. In that case you can either use a genetic algorithm or you use reinforcement learning with backpropagation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would recommend using a genetic algorithm, because it is significantly easier and also makes sense in this situation. You would randomly initialise your network, let it run around in the environment and remember how much food it ate and how often or how quickly it died. Then you would randomly change the weights of your network and do the same thing again. If it did better this time around you would proceed to use the new weights otherwise you go back to the old weights and try a different random change. &#xA;By selecting successful random changes it would over time learn to avoid danger and seek out food.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: To my mind you have a fundamental misunderstanding how perception works. If you see a lion and a cake, do those trigger different kinds of cells on your retina? No! All nerve cells are used to detect all kinds of objects! The classification, i.e. whether you are seeing a lion or a cake happens in the neural network i.e. in the higher regions of your visual cortex, far removed from the initial nerve activation. Your lion might be yellow and your cake might be yellow, only if you analyse the high level structure of your nerve inputs can you decide what you are seeing. That is the task of a neural network. And that high level structure analysis is what a neural network learns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What seems to confuse you is the example you linked. In that example this very sparse distance measuring is enough to differentiate between walls and boosters in your high level structure analysis, because the different points of the walls that you sample have a certain relative position that you can analyse and conclude that they constitute the wall. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your scenario very sparse distance measuring will not help you obviously. The distance of an object doesn't tell you whether it's a lion or a cake. Distance and color would be a solution to that. Or, more realistically, you have different shapes and much tighter distance sampling, and high level analysis can work out the shape from a couple of closely measured distances. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2227" LastEditDate="2017-05-26T07:09:30.393" LastActivityDate="2017-05-26T07:09:30.393" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3335" PostTypeId="1" AcceptedAnswerId="3337" CreationDate="2017-05-18T13:27:55.477" Score="2" ViewCount="328" Body="&lt;p&gt;I took a few AI courses in college (1999-2003), and we used the first edition of &lt;em&gt;AI: A Modern Approach&lt;/em&gt;. We covered a lot of topics and programming, including classical AI, neural networks, and temporal difference learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over the past few years, AI has had a resurgence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is this resurgence due to new AI theory or just better (more) computational power and data so that the theories (e.g., neural networks) to be more effective? Or is it a combination of both?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to get up to speed on what's happened in AI since the early 2000s, and I want to know what to cover -- what is the most significant advancement?&lt;/p&gt;&#xA;" OwnerUserId="7280" LastEditorUserId="2444" LastEditDate="2017-12-27T19:42:14.510" LastActivityDate="2017-12-27T19:42:14.510" Title="What caused the resurgence of AI since the early 2000s?" Tags="&lt;history&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3336" PostTypeId="2" ParentId="2528" CreationDate="2017-05-18T14:27:19.813" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In non-trivial cases, the transition matrix is (generally) not maintained in the traditional tabular form. If the representation used factored notation (Factored MDP) then Dynamic Bayesian networks can be used. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another approach would be to abstract state spaces so that you have reduced number of states in the representation of P. These can be roughly classified as model minimization techniques.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hierarchical representations can be adopted if some tasks can be split. E.g., the classical &quot;taxi problem&quot; has an explicit Get-passenger and Put-passenger task, that form the hierarchy. Now each of the smaller tasks can possibly have a well formed transition function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It may also be prudent to use model free learning methods in such cases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7098" LastActivityDate="2017-05-18T14:27:19.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3337" PostTypeId="2" ParentId="3335" CreationDate="2017-05-18T16:34:44.327" Score="2" Body="&lt;p&gt;To vastly oversimplify a lot of the progress of modern research in AI/neural nets, the recent advances stems from applicational improvements of the back propagation algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ex: our recent possession of big data (it turns out that back-prop works very well if you have lots of data to feed into the network), increased computing power to harness that data, and creative network designs to leverage that computing power (deep learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I haven't found a good source that integrates the knowledge of current practice of neural networks into the classical study of AI nicely. But if you are still interested you can read this modern classic textbook on neural networks and make your own judgements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.deeplearningbook.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.deeplearningbook.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-05-18T16:34:44.327" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3339" PostTypeId="2" ParentId="3330" CreationDate="2017-05-18T22:16:12.313" Score="2" Body="&lt;p&gt;From what I've read on deep nets and neuroscience, your interpretation is backwards.  Most 'neurons' in deep nets use RELU activations, which is strictly excitatory, while mammal brains are composed of both excitatory and inhibitory synapses/neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Excitatory_postsynaptic_potential&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Excitatory_postsynaptic_potential&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Inhibitory_postsynaptic_potential&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Inhibitory_postsynaptic_potential&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the reason for strictly excitatory activations like RELU in deep nets is that they provide a continuous gradient for backpropagation during learning.  Inhibitory activations make the gradient discontinuous, greatly complicating the propagation of feedback/error.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-05-18T22:16:12.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3343" PostTypeId="1" CreationDate="2017-05-19T10:11:22.140" Score="11" ViewCount="1477" Body="&lt;p&gt;I would like to train a bot that uses text input, memorizes a few categories and answers questions accordingly. In addition as version 2.0, I want to make the bot to answer voice inputs as well. Which are the latest machine learning/AI algorithms available for the same? Please let me know.&lt;/p&gt;&#xA;" OwnerUserId="6045" LastEditorUserId="4302" LastEditDate="2019-05-20T07:29:11.213" LastActivityDate="2019-05-20T07:29:11.213" Title="What are the latest methods to train a chat bot?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;chat-bots&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="3" ClosedDate="2021-01-18T02:49:22.440" ContentLicense="CC BY-SA 4.0" />
  <row Id="3345" PostTypeId="1" CreationDate="2017-05-19T18:38:45.073" Score="12" ViewCount="9814" Body="&lt;p&gt;I'm wondering how to train a neural network for a round based board game like, tic-tac-toe, chess, risk or any other round based game.&#xA;Getting the next move by inference seems to be pretty straight forward, by feeding the game state as input and using the output as the move for the current player.&#xA;However training an AI for that purpose doesn't appear to be that straight forward, because:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There might not be a rating if a single move is good or not, so training of single moves doesn't seem to be the right choice&lt;/li&gt;&#xA;&lt;li&gt;Using all game states (inputs) and moves (outputs) of the whole game to train the neural network, doesn't seem to be the right choice as not all moves within a lost game might be bad&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So I'm wondering how to train a neural network for a round based board game?&#xA;I would like to create a neural network for tic-tac-toe using tensorflow.&lt;/p&gt;&#xA;" OwnerUserId="7321" LastActivityDate="2021-04-01T19:46:43.907" Title="How to train a neural network for a round based board game?" Tags="&lt;training&gt;&lt;tensorflow&gt;&lt;game-ai&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3346" PostTypeId="2" ParentId="3345" CreationDate="2017-05-19T19:02:44.060" Score="4" Body="&lt;p&gt;I think you should get familiar with reinforcement learning. In this field of machine learning the agent interacts whit its environment and after that the agent gets some reward. Now, the agent is the neural network the environment is the game and the agent can get a reward +1 if it wins or -1 if loses. You can use this state, action, reward experienc tuple to train the agent.  I can recommend David Silver's lectures on youtube and Sutton's book as well. &lt;/p&gt;&#xA;" OwnerUserId="6019" LastEditorUserId="6019" LastEditDate="2017-11-10T06:19:16.423" LastActivityDate="2017-11-10T06:19:16.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3347" PostTypeId="2" ParentId="3345" CreationDate="2017-05-20T04:25:37.720" Score="7" Body="&lt;p&gt;I'm a chess player and my answer will be only on chess.&lt;/p&gt;&#xA;&lt;p&gt;Training a neural network with reinforcement learning isn't new, it has been done many times in the literature.&lt;/p&gt;&#xA;&lt;p&gt;I'll briefly explain the common strategies.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The purpose of a network is to learn &lt;strong&gt;position evaluation.&lt;/strong&gt; We all know a queen is stronger than a bishop, but can we make the network know about it without explicitly programming? What about pawn structure? Does the network understand how to evaluate whether a position is winning or not?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, we know why we need the network, we'll need to design it. The design differs radically between studies. Before deep learning was popular, people were using the shallow network. Nowadays, a network with many layers stands out.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once we have the network, you'll need to make a chess engine. Neural network can't magically play chess by itself, it needs to connect to a chess engine. Fortunately, we don't need to write position evaluation code because the network can do that for us.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, we have to play games. We could start with some high-quality chess databases or instead have our AI agent play games with another player (e.g. itself, another AI agent, or a human). This is known as &lt;strong&gt;reinforcement learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;While we play games, we update the network parameter. This can be done by stochastic gradient descent (or other similar techniques). We repeat our training as long as we want, usually over millions of iterations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, we have a trained neural network model for chess!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Look at the following resources for details:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://chessprogramming.wikispaces.com/Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://chessprogramming.wikispaces.com/Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="36737" LastEditDate="2021-04-01T19:46:43.907" LastActivityDate="2021-04-01T19:46:43.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3354" PostTypeId="1" CreationDate="2017-05-22T10:06:57.737" Score="1" ViewCount="91" Body="&lt;p&gt;Expressed in my own words:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose we create something that passes all of our tests and is&#xA;  indistinguishable from another human. How can you know if this is&#xA;  truly a conscious being as a human is, or simply a simulation of&#xA;  conscience?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is the name for this? Where was it first written about?&lt;/p&gt;&#xA;" OwnerUserId="1467" LastEditorUserId="2444" LastEditDate="2021-01-25T23:33:28.277" LastActivityDate="2021-01-25T23:33:28.277" Title="Is there a formal name for this philosophical AI problem?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;intelligence-testing&gt;&lt;self-awareness&gt;&lt;chinese-room-argument&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3355" PostTypeId="2" ParentId="3354" CreationDate="2017-05-22T10:42:42.427" Score="2" Body="&lt;p&gt;If something is indistinguishable from a human it is as intelligent as a human. There is no such thing as simulated intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consciousness of course is a different matter and I suspect that's what you really have in mind (no pun intended). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question how to distinguish somebody who is truly conscious from somebody who just acts outwardly like a conscious being is often discussed under the term &lt;a href=&quot;https://en.wikipedia.org/wiki/Philosophical_zombie&quot; rel=&quot;nofollow noreferrer&quot;&gt;philosophical zombie&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-05-22T10:42:42.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3358" PostTypeId="1" AcceptedAnswerId="3370" CreationDate="2017-05-22T15:52:47.867" Score="5" ViewCount="173" Body="&lt;p&gt;There are lots of examples of machine learning systems that can recognize objects and extract other information from images with very high precision. To train the models of such systems is necessary (I guess) a computer with a lot of computational power. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: For a system with images as inputs, depending on the complexity of the problem, is feasible to train a model in an average laptop? It will take to much time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the time taken to train any machine learning model will be a function of lots of variables. I really don't expect a quantitative answer here, I just want to know whether I will be forced to upgrade my computer to develop and train machine learning models that have images as inputs.&lt;/p&gt;&#xA;" OwnerDisplayName="user7369" LastActivityDate="2017-05-23T15:26:10.837" Title="Is it feasible to train a Machine Learning Model (with image inputs) in an average personal computer?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;training&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3359" PostTypeId="2" ParentId="3354" CreationDate="2017-05-22T20:22:12.983" Score="1" Body="&lt;p&gt;Your problem closely resembles John Searle's &quot;Chinese Room&quot; argument, which claimed that one (or more) abstract &quot;intelligence tests&quot; lack the discriminitive ability to distinguish between a trivial simulation of intelligence and The Real Thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the success of an AI at one (or more) synthetic cognitive test(s) (like the Turing Test, or the games of chess or go) should be seen as insufficient as a surrogate threshold target equal to human-level intelligence.  The implication is that a sufficient AI test must require deeper understanding of a problem space from the AI than would be needed to &quot;simply&quot; manipulate (Chinese character) symbols, or play around cleverly but meaninglessly with English words in a convincing manner, as bots have done for years when competing in Turing Test competitions.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-05-22T20:22:12.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3360" PostTypeId="2" ParentId="3302" CreationDate="2017-05-22T20:52:37.127" Score="2" Body="&lt;p&gt;I think you're describing &quot;object localization and detection&quot; which combines object identification with discovery of its spatial placement in the field of view.  There's been a lot of work on this in the past 5 years using CNNs and variants.  It's very much in demand for pedestrian/obstruction detection, identification, and avoidance in autonomous car systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a place to start:  &lt;a href=&quot;https://github.com/kjw0612/awesome-deep-vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kjw0612/awesome-deep-vision&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially, take a look at &quot;Faster R-CNN&quot; by Ren et al, 2016.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-05-22T20:52:37.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3361" PostTypeId="1" CreationDate="2017-05-23T00:22:31.627" Score="10" ViewCount="632" Body="&lt;p&gt;I'd like to do some experimenting with neural net evolution (NEAT). I wrote some GA and neural net code in C++ back in the 90s just to play around with, but the DIY approach proved to be labor-intensive enough that I eventually dropped it.&lt;/p&gt;&#xA;&lt;p&gt;Things have changed a lot since then, and there are lots of very nice open-source libraries and tools around for just about any interest one might have. I've Googled different open-source libraries (e.g. DEAP), but I could use some help choosing one that would be a good fit.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I spent much of my time writing code to visualize what was going on (neural net state, population fitness) or final results (graphs, etc).&lt;/p&gt;&#xA;&lt;p&gt;Maybe this would have to be fulfilled by a separate open-source library, but visualization support would be something that would allow me to spend more time on the problem/solution and less on implementation details.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I know C/C++, Java, C#, Python, Javascript and a few others. Something that's a nice trade-off between a higher-level language and good performance on home hardware would be a good choice.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Can someone with experience suggest a good open-source library or set of tools?&lt;/p&gt;&#xA;" OwnerUserId="7374" LastEditorUserId="2444" LastEditDate="2021-02-06T17:52:12.083" LastActivityDate="2021-02-06T17:54:23.760" Title="Open-source tools or libraries to experiment with neural networks and evolutionary algorithms" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;resource-request&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="6" ClosedDate="2021-02-06T17:49:55.417" ContentLicense="CC BY-SA 4.0" />
  <row Id="3362" PostTypeId="2" ParentId="3361" CreationDate="2017-05-23T01:14:47.693" Score="2" Body="&lt;p&gt;Well, if you choose &lt;a href=&quot;https://www.tensorflow.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorfFlow&lt;/a&gt; to work with, you get &lt;a href=&quot;https://www.tensorflow.org/get_started/summaries_and_tensorboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorBoard&lt;/a&gt; as part of the package. That might be something close to what you're looking for.&lt;/p&gt;&#xA;&lt;p&gt;And with TensorFlow, you can code in C++, Python, and a few other languages (I think there are both Ruby and Java bindings as well, probably others by now).&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2021-02-06T17:52:48.683" LastActivityDate="2021-02-06T17:52:48.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3363" PostTypeId="2" ParentId="3361" CreationDate="2017-05-23T03:37:20.450" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://github.com/josephmisiti/awesome-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/josephmisiti/awesome-machine-learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;has many useful resources. Please take a look.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-05-23T03:37:20.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3364" PostTypeId="1" AcceptedAnswerId="3366" CreationDate="2017-05-23T06:44:19.180" Score="3" ViewCount="130" Body="&lt;p&gt;Suppose we create two units (or programs) that run in parallel and we label them as a cognitive unit and the conscious cognitive unit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A human has two units analogously. A rational analyzer and not so rational analyzer. (Is there any third thing?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, consciousness is an extra layer of decision making. This resembles the way metaheuristics work. We have a set of rules for decision making and we analyze them and tune those rules dynamically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global search algorithms mimic conscious behavior whereas the local search algorithms work as a rational mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is it possible to create a conscious agent by having two units as I described above?&lt;/p&gt;&#xA;" OwnerUserId="7377" LastEditorUserId="2444" LastEditDate="2020-06-16T10:33:57.653" LastActivityDate="2020-06-16T10:53:12.960" Title="Can we create a conscious agent by having two units: one rational and the other conscious?" Tags="&lt;ai-design&gt;&lt;intelligent-agent&gt;&lt;artificial-consciousness&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3365" PostTypeId="2" ParentId="3361" CreationDate="2017-05-23T07:08:17.240" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/wagenaartje/neataptic&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/wagenaartje/neataptic&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As this is written in Javascript and does not (yet) offer GPU support, it is quite slow. However, it is very nice to fiddle around with flexible network architectures. The only visualisation that it offers right now is a map of network architecture, but graphs could easily be implemented.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2021-02-06T17:53:25.283" LastActivityDate="2021-02-06T17:53:25.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3366" PostTypeId="2" ParentId="3364" CreationDate="2017-05-23T10:45:10.343" Score="1" Body="&lt;p&gt;Jürgen Schmidthuber has a similar concept of consciousness, maybe check out &lt;a href=&quot;https://www.inverse.com/article/25521-juergen-schmidhuber-ai-consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to claim we had little, rudimentary, conscious learning systems for at least 25 years. Back then, already, I proposed rather general learning systems consisting of two modules.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There you also have one module that is occupied with decision making. The other module's job is to create a model of the world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, a philosopher will tell you that we have no idea what consciousness is or &quot;what it feels like to be ai with two units&quot;. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2020-06-16T10:53:12.960" LastActivityDate="2020-06-16T10:53:12.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3367" PostTypeId="2" ParentId="3361" CreationDate="2017-05-23T10:47:24.423" Score="2" Body="&lt;p&gt;There is also DXNN, which is as you described, a neuroevolutionary system, it is written in Erlang.&#xA;&lt;a href=&quot;https://github.com/CorticalComputer/DXNN2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/CorticalComputer/DXNN2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I did some work on it to make it modular, so you use it as a library and keep your code/application isolated.&lt;/p&gt;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://github.com/ausrasul/DXNN2_Template&quot; rel=&quot;nofollow noreferrer&quot;&gt;code example&lt;/a&gt;, which downloads DXNN as a library. It also generates Gnuplot ready data files for visualization.&lt;/p&gt;&#xA;" OwnerUserId="3874" LastEditorUserId="2444" LastEditDate="2021-02-06T17:54:23.760" LastActivityDate="2021-02-06T17:54:23.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3368" PostTypeId="1" CreationDate="2017-05-23T11:06:33.233" Score="7" ViewCount="1113" Body="&lt;p&gt;I was prompted towards this question while trying to find server racks and motherboards which are specialized towards artificial intelligence. Naturally I went to the SuperMicro website. There the &lt;a href=&quot;https://www.supermicro.com.tw/products/system/4U/4028/SYS-4028GR-TXRT.cfm&quot; rel=&quot;nofollow noreferrer&quot;&gt;chassis+motherboard&lt;/a&gt; which supported the maximum GPUs in the &quot;artificial intelligence&quot; category could support upto 8 of them. Additionally, the Nvidia DGX-1 also has only 8 Tesla P100 GPUs. And just to rub it in, Matlab does not support more than 8 GPUs last I checked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, are more than 8 GPUs practical for DL? I would take Caffe, CNTK, Tensorflow and Torch7 as reference.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="7402" LastEditDate="2017-08-14T22:13:36.670" LastActivityDate="2017-08-14T22:13:36.670" Title="Are more than 8 high performance Nvidia GPUs practical for deep learning applications?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hardware&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3369" PostTypeId="2" ParentId="3368" CreationDate="2017-05-23T15:06:00.353" Score="3" Body="&lt;p&gt;I did some recent research on this topic. It all comes down to parallelization.&lt;br&gt;&#xA;Basically there are 2 ways to do it: model parallelization or batch parallelization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model parallelization is when you split the model by layers among multiple GPUs. As per my best knowledge you can't split a layer between GPUs, so 8 GPUs would serve 8 layers that is very extensive. Tensorflow supports this method. In my opinion more than 6 doesn't make sense this way.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Batch parallelization is when you run the entire model on each GPU parallel and you split your batch and process it in parallel. This is done via a trick to define a larger batch that will be split and become the desired batch size after split. In this case batch splitting and updating the weights is done on the CPU (in case of Tensorflow) and after 3 GPU any additional GPU has only a marginal improvement on training speeds (as per reports). So here even 4 doesn't make sense and 8 is just crazy.  &lt;a href=&quot;https://medium.com/@kuza55/transparent-multi-gpu-training-on-tensorflow-with-keras-8b0016fd9012&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Here is an example for batch paralellization.&quot;&gt;Here is an example of batch parallelization.&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately if you are good in coding you may want to have a &lt;a href=&quot;http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;look at this paper&lt;/a&gt;, section 3.5 where it is explained how 8 GPUs were utilized to serve a 4 layer LSTM network. Probably you can do things like this to utilize DGX-1 but as far as I know Tensorflow doesn't support splitting a layer to multiple GPUs. My conclusion is that it's already very hard to utilize 8 GPUs and above that bus speed becomes the bottleneck.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Extension:&lt;br&gt;&#xA;I double checked on the bus speed and I was wrong, it should not be a problem. Most of the time consumed in training is computational effort of backpropagation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually PCIe speed is limited by the CPU and the mobo chipset. The CPU has PCIe lanes to be allocated by the mobo. The strongest single CPU at the moment is Broadwell-E with 40 lanes (Skylake rumored to have 44). Mobo allocates that bandwidth either x16 or x8 to PCIe peripherals. So with a 40-lane CPU you can run 2 cards at x16 (2 * 16 = 32 &amp;lt; 40) or 5 cards at x8 (5 * 8 = 40). Here it needs to be mentioned that M.2 also uses PCIe lanes so for the latter option forgot and M.2 drive. A single CPU system will not take 8 GPUs, that's why they need dual CPU in DGX-1. The next limitation is the mobo chipset. The most powerful at the moment is the X99 and the C6 series. X299 will be announced next week if memory serves and probably a C6 replacement follows soon.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since PCIe speed is not the bottleneck in machine learning the updated answer for your question would be the more GPUs the better. It seems the limit is 5 at x8 using PCIe 3.0, but more than that is rarely needed since layers can't be split between GPUs. (And the deepest NN that still makes sense may be considered as 5-6 layers.)&lt;/p&gt;&#xA;" OwnerUserId="7364" LastEditorUserId="7364" LastEditDate="2017-05-24T19:44:28.630" LastActivityDate="2017-05-24T19:44:28.630" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3370" PostTypeId="2" ParentId="3358" CreationDate="2017-05-23T15:26:10.837" Score="3" Body="&lt;p&gt;You may play around on an average laptop but training will be very slow and you will be limited on the size of your model.&lt;br&gt;&#xA;Once you try to build something more serious you will run out of memory very fast. A system with a GPU is recommended if you want to really do things like image recognition. If you buy something I would not go for any GPU with less than 8 GB memory and for an X99 motherboard to keep the flexibility to add a second GPU on full speed later if needed.&lt;br&gt;&#xA;Don't mix up image recognition running on phones etc. Those are trained models and actually just being applied that is much easier process. Training is way much more expensive.&lt;/p&gt;&#xA;" OwnerUserId="7364" LastActivityDate="2017-05-23T15:26:10.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3372" PostTypeId="1" CreationDate="2017-05-23T21:15:28.307" Score="5" ViewCount="128" Body="&lt;p&gt;Basically, an AI that can create, rig, and texture 3d models and game environments (by extrapolating from collections of reference models, according to user input), and that can set up physics and mechanics (assuming that the AI has access to a 3d modeling studio and a game engine, both designed for compatibility with the AI, or as a component of the AI), all according to user commands (and allowing for tweaking and optimizations of models, rigging, mechanics, etc, by the user).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of user commands would be something like: &quot;Gaming AI, create a casual style* male model, European build, 6'5&quot;, fit and slightly skinny, with red scaly skin, green eyes, a reptilian tail, demonic wings, claws, sharp teeth&quot;, etc. The user probably wouldn't add all of these characteristics at once, but rather one at a time, tweaking each feature via AI commands or manually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*&quot;casual style&quot; is a fictional &quot;style class&quot;. Style classes would refer to the visual style of the models. Possible example styles include &quot;cartoon&quot;, &quot;abstract&quot;, &quot;gothic&quot;, &quot;steampunk&quot;, &quot;serious&quot; and &quot;realistic&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's another example of user commands, for a environmental model: &quot;Gaming AI, create a serious style house, Victorian, two story, white with beige trim, with porches and shutters. Give it a creepy aesthetic.&quot; Again, models could be created and modified or have features added in a step by step process, in order to tweak and refine them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that such an AI would significantly reduce the amount of time, labor, and difficulty involved in designing games; making games cheaper and easier to produce, and making game design available to everyone. A variation of such an AI could also be used to create 2d artwork and animations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is such an AI even remotely possible? And would it take a supercomputer to run the thing? (I'm under the impression that such an AI would need to be capable of learning and adapting, and would require a massive and expansile &quot;association library&quot;*—including 2d and 3d models, and verbal and textual speech—as well as near human intelligence)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*if the term &quot;association library&quot; doesn't exist, or doesn't currently relate to AI, Then I just made it up. According to my made up definition, an association library is the library of programmed or learned associations that an AI uses to generate responses, and to, in this context, generate 3d models; and probably to write or select code as well, in order to set up physics and mechanics and the like. &lt;/p&gt;&#xA;" OwnerUserId="5698" LastActivityDate="2019-05-01T15:28:56.383" Title="Feasibility of an AI assistant to expedite game development?" Tags="&lt;ai-design&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3373" PostTypeId="1" CreationDate="2017-05-24T05:31:50.243" Score="7" ViewCount="1277" Body="&lt;p&gt;Does google manufacture TPUs? I know that google engineers are the ones responsible for the design, and that google is the one using them, but which company is responsible for the actual manufacturing of the chip? &lt;/p&gt;&#xA;" OwnerUserId="3323" LastEditorUserId="1671" LastEditDate="2019-04-30T18:54:13.233" LastActivityDate="2019-04-30T18:54:13.233" Title="Who manufactures Google's Tensor Processing Units?" Tags="&lt;tensorflow&gt;&lt;hardware&gt;&lt;google&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3374" PostTypeId="1" AcceptedAnswerId="3382" CreationDate="2017-05-24T19:07:38.987" Score="18" ViewCount="16357" Body="&lt;p&gt;I am a software engineering student and I am complete beginner to AI. I have read a lot of articles on how to start learning AI, but each article suggests a different way. I was wondering if some of you experts can help me get started in the right way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few more specific questions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Which language should I focus on? A lot of articles suggest Python, C++ or Lisp for AI. Can I use Java instead of any of the other languages mentioned?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What kind of mathematical background should I have? During the first year, I did discrete mathematics, which included the following topics: sets, matrices, vectors, functions, logic and graph theory (They taught these topics briefly). Are the are there any more topics that I should learn now? For example, calculus?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If possible, I would appreciate any resources or books I could use in order to get started, or maybe you guys can give me a detailed procedure I can follow in order to catch up with to your level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: For now I would like to focus on neural networks and machine learning. After I that I would like to explore robotics and natural language processing.&lt;/p&gt;&#xA;" OwnerUserId="4427" LastEditorUserId="2444" LastEditDate="2020-03-04T21:17:13.877" LastActivityDate="2020-03-04T21:17:13.877" Title="How does one start learning artificial intelligence?" Tags="&lt;math&gt;&lt;getting-started&gt;&lt;programming-languages&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="32" ClosedDate="2020-03-04T21:38:31.333" ContentLicense="CC BY-SA 4.0" />
  <row Id="3375" PostTypeId="2" ParentId="3374" CreationDate="2017-05-25T03:31:19.040" Score="6" Body="&lt;p&gt;You'll find that both Calculus and Linear Algebra have some application in AI/ML techniques.  In many senses, you can argue that most of ML reduces to Linear Algebra, and Calculus is used in, eg. the backpropagation algorithm for training neural networks.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd be well served to take a class or two in probability and statistics as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming language choice is less important, IMO.  You can do AI/ML in pretty much any mainstream language, and plenty of non-mainstream languages.  The biggest difference involve performance, and availability of libraries / tools.  C++, for example, is usually going to outperform Java or Python and it lets you get &quot;close to the metal&quot; to really maximize the capabilities of your hardware.  Python, however, has a really good FFI, and is often used in conjunction with C or C++.   Python, C++, Java, R, Octave/Matlab and a few other languages tend to have lots of high quality libraries available, which may be important to you depending on what you want to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you probably don't want to try and do ML / AI in, say, COBOL or PL/I or RPG/400 or something.  Stick to something at least reasonably popular.  Poke around mloss.org and look at what libraries / toolkits are available in different languages and that should help guide your choice.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-05-25T03:31:19.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3381" PostTypeId="2" ParentId="3329" CreationDate="2017-05-25T15:20:31.780" Score="0" Body="&lt;p&gt;@BlindKungFuMaster answered my question in comments, but his answer doesn't reflect that information. If he changes his answer to contain the actual answer, I'll remove mine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to the question &quot;How can object types be differentiated in the input of a neural network?&quot; is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Use different sensors (nerves) to detect different types of input.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want the network to be capable of differentiating two different types of objects in a simple 2D environment, use two different sets of nerves, one to detect one object type, one to detect the other. So if you want to sense 10 points around your &quot;organism&quot;, have 20 nerves, 10 neurons in your input layer dealing with one object type, 10 dealing with the other. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;That's the most simple example, dealing with binary differences (object type 1 or object type 2), but it could be less binary, like this: Let's say there are three object types and think of each object types as 1/3 of a color value. So you have three sets of 10 nerves in total transmitting to your input layer (30 neurons) and each nerve set senses either R, G, or B. If you set up your system so that there are three &quot;objects&quot; to collide with, stacked on top of each other (like a single object), your neural network will be capable of handling objects differently based on their RGB color value, meaning it can now handle nearly infinite &quot;types&quot; of objects. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7249" LastActivityDate="2017-05-25T15:20:31.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3382" PostTypeId="2" ParentId="3374" CreationDate="2017-05-25T17:34:07.587" Score="16" Body="&lt;h2&gt;What is artificial intelligence?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence is a very broad field and it covers many and very deep areas of computer science, mathematics, hardware design, and even biology and psychology. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What math do you need?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As for math, I think calculus, statistics, and optimization are the most important topics, but learning as much math as you can won't hurt.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Free resources for beginners&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are many good free introductory resources about AI for beginners.&#xA;I highly recommend starting with this website &lt;a href=&quot;http://aiplaybook.a16z.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI Playbook&lt;/a&gt;, which is &lt;em&gt;intended to help newcomers (both non-technical and technical) begin exploring what's possible with AI&lt;/em&gt;. They also published two videos about the general concepts of AI, you can find them on Youtube &lt;a href=&quot;https://www.youtube.com/watch?v=ht6fLrar91U&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI, Deep Learning, and Machine Learning: A Primer&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=8pLZkbN9h_s&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Promise of AI&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What is your goal?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Once you have a clear understanding of the basic AI terms and approaches, you have to figure out what your goals are. What kind of AI software do you want to develop? What industries are you interested in? What are your chances to get involved in projects of big companies? It's easier to pick up the right tools when you know exactly what you want to achieve.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;AI, machine learning and deep learning&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For most newcomers to AI the most interesting area is deep learning.&#xA;Just to make it clear, there are many areas of AI outside of machine learning and there are many areas of machine learning outside of deep learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial Intelligence &gt; Machine Learning &gt; Deep Learning&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of the recent developments and hyped news are about DL.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Neural networks&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you got interested in deep learning too, you have to start with learning about the concepts of artificial neural networks. Fortunately, it's not too difficult to understand the basics and there are lots of tutorials, code examples and free learning resources on the web and there are many open-source frameworks to start experimenting with.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Frameworks and libraries&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Python&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The most popular such deep learning framework is &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;. It's backed by Google. Love it or hate it, it's a Python-based framework. There are many other Python-based frameworks and libraries, as well. &lt;a href=&quot;https://scikit-learn.org/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scikit-learn&lt;/a&gt;, &lt;a href=&quot;http://deeplearning.net/software/theano/&quot; rel=&quot;nofollow noreferrer&quot;&gt;theano&lt;/a&gt;, &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras&lt;/a&gt; are frequently mentioned in tutorials too. (A tip: if you use Windows you can download WinPython that includes all of these frameworks.)&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Java&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As for Java frameworks, unfortunately, there are not so many options. The most prominent Java framework for DL is &lt;a href=&quot;https://deeplearning4j.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deeplearning4j&lt;/a&gt;. It's developed by a small company and its user base is much smaller than the crowd around TensorFlow. There are fewer projects and tutorials for this framework. However, industry specialists say Java-based frameworks eventually integrate better with Java-based big data solutions and they may provide a higher level of portability and easier product deployment. Just a sidenote: NASA's Jet Propulsion Laboratory used Deeplearning4j for many projects.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Resources&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you decide to go with the flow and want to start learning more about TensorFlow, I recommend you check out the YouTube channels of &lt;a href=&quot;https://www.youtube.com/channel/UC9OeZkIwhzfv-_Cb7fCikLQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepLearning.TV&lt;/a&gt;, &lt;a href=&quot;https://www.youtube.com/user/sentdex&quot; rel=&quot;nofollow noreferrer&quot;&gt;sentdex&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siraj Raval&lt;/a&gt;. They have nice tutorials and some cool demos. And if you decide to take a deeper dive, you can sign up for an online course at &lt;a href=&quot;https://www.udacity.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Udacity&lt;/a&gt; or &lt;a href=&quot;https://www.coursera.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coursera&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Other programming languages&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It also may be interesting to you to know that there are other deep learning frameworks for the Java Virtual Machine with alternative languages, for example, Clojure. (Clojure is a dialect of LISP and it was invented by John McCarthy, the same computer scientist who coined the term &quot;artificial intelligence&quot;. In other words, there are more modern and popular programming languages and tools, but it's still possible, and kinda cool, to use the language for AI that was originally designed for AI. ThinkTopic in Boulder and Freiheit in Hamburg are two companies that use Clojure for AI projects. And if you want to see something awesome to get inspiration to use Clojure in AI and robotics, I recommend you to check out the YouTube video &lt;a href=&quot;https://www.youtube.com/watch?v=Ty9QDqV-_Ak&quot; rel=&quot;nofollow noreferrer&quot;&gt;OSCON 2013: Carin Meier, The Joy of Flying Robots with Clojure&lt;/a&gt;. (Mentioning Clojure in this answer was just an example to show you there is life outside of the bubble of Python-based AI frameworks.)&lt;/p&gt;&#xA;" OwnerUserId="6933" LastEditorUserId="2444" LastEditDate="2020-03-04T20:57:48.747" LastActivityDate="2020-03-04T20:57:48.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3383" PostTypeId="2" ParentId="3374" CreationDate="2017-05-25T19:15:57.830" Score="3" Body="&lt;p&gt;To start AI first of all understand what is AI. Why MNIST's accuracy increase rapidly after 2012. Why machine learning  need AI to increase its accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start and build Application on Machine learning with AI you didn't need maths or some kind of rocket science. &#xA;You are late my bro people build shortcuts for all machine learning problems like a wrapper. You just need to pass data to a method and method will do all shit.&#xA;Start with MNIST's problem its exciting. Read about MNIST's history use basic algorithm on it.&#xA;Try Linear Regression, Logistic Regression,Kmean clusting, KNN .&#xA;Tools for Machine learning &#xA;Skite learn (python lib) or Tensorflow ( python lib)tflearn(higher level api of Tensorflow like a wrapper)&#xA;Both are open source. Examples are available on GitHub .&#xA;Start searching on GitHub. You found a great example. For both lib. Use kaggel to solve problem participate in comptition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you complete all above algorithm try to focus on your your error. Now AI came in roll . Try to figure out how neural network help you to decrease error and increase accuracy.&#xA;Then try some basic neural network like sigmoid , relu and cnn. Don't forget to use dropout in your neural network.&#xA;You can use Tensorflow or keras or Tensorflow with keras&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side by side check 3 Blue 1 Brown's Linear algebra video's to improve your maths. once a day but everyday one video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And now focus on maths behind the logic(any algorithm)&#xA;You can try andrew ng machine learning course. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use Tensorflow for building Android app,IOS app, RaspPi&#xA;Check Tensorflow dev summit 2016/2017. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or if you need crash course then check this &lt;a href=&quot;https://youtu.be/u4alGiomYP4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/u4alGiomYP4&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7446" LastActivityDate="2017-05-25T19:15:57.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3387" PostTypeId="1" CreationDate="2017-05-26T15:04:56.100" Score="3" ViewCount="849" Body="&lt;p&gt;Can neural networks be used to study (elementary) number theoretic problems? What are examples where this has been done in the past? Or is there on the contrary an understanding that neural networks are not helpful for such problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the question more concrete, let me give an example of the kind of number theoretic problem I'm thinking of: given two natural numbers a and b I may want to compute the rounded down quotient int(a/b). Naively I would restrict to 64 bit unsigned numbers and build a neural network that has 128 neurons in the input layer and 64 neurons in the output layer representing the binary expansion of the numbers. Assuming I laid out the network properly and trained it well, would I be expected to get useful output? In particular, would I be able to interpret the output as a number and would it often be the right answer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the reason why I think of this as problem as &quot;number theoretic&quot; is because I want to compute int(a/b) rather than the rational number a/b. This is essentially a step in Euclid's algorithm. So the non-linear behaviour int(4/3) = 1, int(5/3) = 1, int(6/3) = 2 is crucial and would need to be recognizable in the output.&lt;/p&gt;&#xA;" OwnerUserId="7459" LastEditorUserId="75" LastEditDate="2017-05-27T17:08:36.197" LastActivityDate="2018-09-05T08:52:28.937" Title="Neural networks and number theory" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3389" PostTypeId="1" AcceptedAnswerId="7598" CreationDate="2017-05-26T15:15:31.397" Score="32" ViewCount="14261" Body="&lt;p&gt;I am not looking for an efficient way to find primes (which of course is a &lt;a href=&quot;https://en.wikipedia.org/wiki/AKS_primality_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved problem&lt;/a&gt;). This is more of a &amp;quot;what if&amp;quot; question.&lt;/p&gt;&#xA;&lt;p&gt;So, in theory, could you train a neural network to predict whether or not a given number &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is composite or prime? How would such a network be laid out?&lt;/p&gt;&#xA;" OwnerUserId="7458" LastEditorUserId="2444" LastEditDate="2020-11-14T12:54:20.367" LastActivityDate="2020-11-16T22:26:02.337" Title="Could a neural network detect primes?" Tags="&lt;neural-networks&gt;&lt;prediction&gt;&lt;primality-test&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="3390" PostTypeId="1" AcceptedAnswerId="3394" CreationDate="2017-05-26T15:31:47.283" Score="5" ViewCount="105" Body="&lt;p&gt;I can see a lot of tutorials and examples about using TensorFlow and other free, open-source AI/ML/DL frameworks on enterprise level where enough data was collected for such AI solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can one can collect enough data in normal everyday life to practically and effectively make use of such freely available AI/ML/DL technologies to improve one's life and security?&lt;/p&gt;&#xA;" OwnerUserId="6933" LastEditorUserId="75" LastEditDate="2017-05-27T17:14:37.367" LastActivityDate="2017-05-27T17:14:37.367" Title="How can one find / collect data for, and come up with ideas for, using Deep Learning / AI to improve one's everyday life?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3392" PostTypeId="2" ParentId="3389" CreationDate="2017-05-26T17:33:08.773" Score="2" Body="&lt;p&gt;In theory, a neural network can approximate &lt;em&gt;any&lt;/em&gt; given function. This result is known as &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the universal approximation theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, if you train a network with the numbers &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, you cannot guarantee that the network will classify numbers outside that range correctly (&lt;span class=&quot;math-container&quot;&gt;$n &amp;gt; N$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Such a network would be a regular feed-forward network (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Multilayer_perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLP&lt;/a&gt;) as recurrency does not add anything to the classification of the given input. The number of layers and nodes of that NN can only be found through trial and error.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2020-11-14T13:23:19.610" LastActivityDate="2020-11-14T13:23:19.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3394" PostTypeId="2" ParentId="3390" CreationDate="2017-05-26T22:01:25.787" Score="3" Body="&lt;p&gt;Speaking to the &quot;collecting data&quot; part of the question, I'll say this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that not &lt;strong&gt;everything&lt;/strong&gt; requires massive amounts of data. Consider also that large amounts of data about all sorts of things are available in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Open_data&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenData&lt;/a&gt; / &lt;a href=&quot;http://linkeddata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LinkedData&lt;/a&gt; realms. Governments in particular are sources of massive amounts of data.  See, for example, &lt;a href=&quot;http://www.data.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;data.gov&lt;/a&gt; or Google around and see if your (state|city|county|whatever) has an &quot;open data&quot; portal. You might be surprised.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally consider that inexpensive sensor systems and distributed sensor networks can be constructed fairly easily / inexpensively using things like Arduino / Raspberry Pi and communications protocols and software often associated with the term &lt;a href=&quot;https://en.wikipedia.org/wiki/Internet_of_things&quot; rel=&quot;nofollow noreferrer&quot;&gt;Internet of Things&lt;/a&gt;.  You may find that you can actually collect your own data, especially if you crowd-source the effort and get a bunch of other people involved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://opendata.reddit.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://opendata.reddit.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://datasets.reddit.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://datasets.reddit.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wikidata.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wikidata.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dbpedia.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dbpedia.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-05-26T22:01:25.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3396" PostTypeId="2" ParentId="3374" CreationDate="2017-05-27T13:39:27.993" Score="5" Body="&lt;p&gt;When I got interested in AI, I started with the most basic things. My very first book was Russell&amp;amp;Norvig's &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;noreferrer&quot;&gt;Artificial Intelligence- A modern Approach&lt;/a&gt;. I think that's a good place to start, even if you're mostly interested in Deep Nets. It treats not just the basic AI concepts and algorithms (expert systems, depth-first and breadth-first search,knowledge representation,etc.) but also the fundamental mathematics (Bayesian reasoning, First Order Logic, NL n-grams, etc.) and some commonly known problems (as Traveling salesman problem for example). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may also be a good idea to learn statistics, since you are particularly interested in ML. After the mentioned book, you should also have a good idea about what to learn next.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Don't care too much about the programming language. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's much more important to understand programming itself and the related techniques. Learn something about data structures, algorithms, and the different programming paradigms (like OOP, Functional Programming, etc.). Try to understand the logic behind programming and not just a particular language. After all, learning a new language isn't that hard once you understand how to program (then learning a new language is just more or less syntactic sugar).&lt;/p&gt;&#xA;" OwnerUserId="1781" LastEditorUserId="1671" LastEditDate="2018-03-09T19:02:02.363" LastActivityDate="2018-03-09T19:02:02.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3397" PostTypeId="1" CreationDate="2017-05-27T20:16:24.543" Score="9" ViewCount="4894" Body="&lt;p&gt;Just watched a recent WIRED video on virtual assistants' performance on telling jokes. They're composed by humans, but I'd like to know if AI has gotten good enough to write some.&lt;/p&gt;&#xA;" OwnerDisplayName="user7478" LastActivityDate="2020-08-15T15:56:27.760" Title="Can AI write good jokes yet?" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3398" PostTypeId="1" AcceptedAnswerId="3421" CreationDate="2017-05-28T04:41:39.593" Score="1" ViewCount="567" Body="&lt;p&gt;The deep learning algorithms I would to know the limits of are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;CNTK&lt;/li&gt;&#xA;&lt;li&gt;Caffe&lt;/li&gt;&#xA;&lt;li&gt;TensorFlow&lt;/li&gt;&#xA;&lt;li&gt;Torch7&lt;/li&gt;&#xA;&lt;li&gt;Theano&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For example: I've heard TensorFlow is near impossible to parallelize on 8 GPUs and above. So, in this case, the limit would be 8.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="7496" LastEditDate="2017-06-01T19:06:46.907" LastActivityDate="2017-06-13T11:50:24.840" Title="How many GPUs can these deep learning algorithms be parallelized across (batch parallelization)?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3400" PostTypeId="1" CreationDate="2017-05-28T14:56:02.380" Score="1" ViewCount="87" Body="&lt;p&gt;Lets say you install your LSTM machine on a road between London and Oxford. And it makes observations. A car with 3 people inside drives past it in one direction 21 sec after previously observed car (in any direction) : the input is {3, LO, 21}. A bus with 43 people drives in other direction 11 sec later : {43, OL, 11}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cant find a definitive general explanation (without references to existing ML packages code) - how your LSTM layer structure should look like to accept those params (there are 3 of them and they are vastly different between themselves but presenting them together at each step is very significant (as opposed to splitting them to 3 streams and feeding then into 3 LSTMs and then pooling the results).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone explain it in a single formula or a drawing?&lt;/p&gt;&#xA;" OwnerUserId="7487" LastActivityDate="2018-01-15T04:34:51.127" Title="Multi-param LSTM input" Tags="&lt;machine-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3401" PostTypeId="1" CreationDate="2017-05-28T17:20:27.957" Score="4" ViewCount="164" Body="&lt;p&gt;If I want to train a convoluted NN on time series but I cannot decide where to split the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that other people use jumping window over the input. so the feed say 20 sec of observation as 1 sequence into a CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot do that as splitting the observation by fixed size will most definitely break important patterns - ie first part of a pattern will go into the end of the current seq and the rest into the beginning of the next seq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can however find a sensible solution by preprocessing data and finding places of much smaller significance and make seq cutting in the middle. but then the seq length will vary greatly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I still use CNN ? or this idea is silly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How else people extract features from time series using CNNs ?&lt;/p&gt;&#xA;" OwnerUserId="7487" LastActivityDate="2017-05-28T17:20:27.957" Title="How to feed a variable size sequences into a CNN?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3402" PostTypeId="1" AcceptedAnswerId="3643" CreationDate="2017-05-29T02:22:25.197" Score="6" ViewCount="709" Body="&lt;p&gt;Imagine a simple scenario of having a large repository using one framework and integrated with data/robots, etc., then having a new feature requested and the framework missing some vital functionality that is available in another framework (say a new kind of layer).&lt;/p&gt;&#xA;&lt;p&gt;For many mathematical libraries, it can be easy to reverse engineer the specific function that one would want form the other framework so as to not import the entire framework but in the case of deep learning, this isn't so trivial nor that easily testable.&lt;/p&gt;&#xA;&lt;p&gt;In these cases, does one take on the problem of reverse engineering the functionality or do they attempt to combine the architectures? If it's the combination, what are the biggest issues with integration?&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="32410" LastEditDate="2021-04-22T22:56:08.737" LastActivityDate="2021-04-22T22:56:08.737" Title="Is there ever a need to combine deep learning frameworks? (Eg. TensorFlow &amp; Torch)?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;pytorch&gt;&lt;software-architecture&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3403" PostTypeId="1" AcceptedAnswerId="3405" CreationDate="2017-05-29T09:02:28.270" Score="8" ViewCount="1666" Body="&lt;p&gt;I created an OpenAI Gym environment, and I would like to check the performance of the agent from &lt;a href=&quot;https://blog.openai.com/openai-baselines-dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Baselines DQN approach &lt;/a&gt; on it.&#xA;In my environment, the best possible outcome for the agent is 0 - the robot needs zero non-necessary resources to complete a task. The goal is to minimize the need for resources: for each needed resource, there is a penalty of -1. In many states, only certain actions make physical sense. How do I deal with this?&lt;/p&gt;&#xA;&lt;p&gt;There was already a question about the handling of invalid moves on &lt;a href=&quot;https://ai.stackexchange.com/q/2980/2444&quot;&gt;AI StackExchange&lt;/a&gt;, recommending to ignore the invalid moves. However, ignoring them would imply returning the same state and a 0 reward, the best possible outcome, which is clearly not the case. Setting drastic negative rewards also does not seem to work, since even promising handling paths are compromised by invalid actions and the corresponding drastic negative reward.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best) or negative?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;My ideas/ questions on this for the &lt;a href=&quot;https://blog.openai.com/openai-baselines-dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Baselines DQN approach&lt;/a&gt; implementation&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any way to set the initial Q-values for the actions? I could set -infinity for the invalid actions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any way to limit the set of valid actions per state? When after the env.step(action) function the new state is returned, can I somehow define which actions are valid for it?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7495" LastEditorUserId="2444" LastEditDate="2020-11-14T19:03:06.843" LastActivityDate="2020-11-14T19:03:06.843" Title="What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;implementation&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3405" PostTypeId="2" ParentId="3403" CreationDate="2017-05-29T11:02:05.533" Score="5" Body="&lt;h2&gt;1) Is there any way to set the initial Q-values for the actions?&lt;/h2&gt;&#xA;&lt;p&gt;You can generally do this, but you cannot specify specific weights for specific actions in specific states. Not through the network weights directly, at least. That would defeat the purpose of using backpropagation to optimize the weights and find the optimal parameters and Q-values.&lt;/p&gt;&#xA;&lt;h2&gt;2) Is there any way to limit the set of valid actions per state?&lt;/h2&gt;&#xA;&lt;p&gt;Again, not directly through the network. That would imply different network architectures for different states, which would imply optimizing several different networks. Unless you used some convoluted approach like Net2Net to keep the networks synced.&lt;/p&gt;&#xA;&lt;h2&gt;However, you can bypass the network itself.&lt;/h2&gt;&#xA;&lt;p&gt;You can, however, on top of the network, filter its outputs. Let's say you have a set of states, and in state X, only 2 out of all 5 actions are valid. When getting the Q-values for all the actions in DQN, you can detect if you're in state X, and instead of choosing the greedy action out of all actions, choose the greedy action out of all &lt;strong&gt;valid&lt;/strong&gt; actions. Better yet, have your environment just send a list of all valid actions for whatever state you're in. Makes this more general, as it removes your need for discrete states, having lists of valid actions for each state, etc.&lt;/p&gt;&#xA;&lt;h2&gt;You can also really just ignore invalid actions.&lt;/h2&gt;&#xA;&lt;p&gt;You claimed&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;However, ignoring them would imply returning the same state and a 0 reward, the best possible outcome, which is clearly not the case.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If your environment has &lt;strong&gt;no time limit&lt;/strong&gt;, then the best action really is to stand still and avoid wasting resources. However, if there is a time-limit, eventually the agent should receive some sort of penalty for not completing the task. Otherwise, if there is no time limit, you can also give him an award for completing the task. Imagining you have 10 resources, instead of having your reward interval be from &lt;code&gt;[-10; 0]&lt;/code&gt;, shift it to &lt;code&gt;[0, 10]&lt;/code&gt; by awarding 10 points at the end of the game. This gives your agent the motivation to complete the task instead of doing useless actions, and discards your need for action-filtering at all.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-05-29T11:08:27.723" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3406" PostTypeId="1" CreationDate="2017-05-29T11:17:59.697" Score="5" ViewCount="3337" Body="&lt;p&gt;What is the difference between simple reflex and model-based agents?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the role of the internal state, in the case of model-based agents?&lt;/p&gt;&#xA;" OwnerUserId="7500" LastEditorUserId="2444" LastEditDate="2019-06-22T17:07:25.043" LastActivityDate="2019-06-22T17:11:06.873" Title="What is the difference between simple reflex and model-based agents?" Tags="&lt;ai-basics&gt;&lt;theory&gt;&lt;intelligent-agent&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3407" PostTypeId="1" CreationDate="2017-05-29T18:30:16.880" Score="7" ViewCount="119" Body="&lt;p&gt;In the process of segmentation, pixels are assigned to regions based on features that distinguishes them from the rest of the image. Value Similarity and Spatial Proximity, for example, are two important principles that assume that points in the same region will have pixels that are spatially close and have similar values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In lots of situations this is true, but what about regions composed of pixels that are not similar in value? Consider the image below. The same &quot;logical&quot; region is composed of different elements that together represent something meaningful. In the same region there are trees with different sizes and shapes, with shadow over some of them etc. There are different things, with pixels that differ a lot in value, but I still need to group them together in the same region. From the image you can see that I don't care so much with differences in color. In this case texture is the most important attribute. What algorithms are used to do the segmentation and classification in problems like that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm already looking for some algorithms and techniques that focus on texture, but some opinions from the experts will help me a lot. I think I need some orientation. Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UORVX.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UORVX.jpg&quot; alt=&quot;segmentation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user7369" LastEditorDisplayName="user7369" LastEditDate="2017-05-31T17:11:09.780" LastActivityDate="2017-10-24T02:51:06.500" Title="What algorithms are used for segmentation and classification of non solid regions in an image?" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3409" PostTypeId="2" ParentId="2472" CreationDate="2017-05-30T06:09:50.027" Score="2" Body="&lt;p&gt;I think Minsky deprecated the suggestion that probabilistic models could be &lt;em&gt;surrogates&lt;/em&gt; for component models for intelligence that he suggested were grounded in principles and processes that interact (i.e. Society of Mind).  But I don't believe he ever referred to probabilistic models as dead ends.  All intelligence models must employ awareness of likelihoods and exceptions.  Many probabilistic techniques like bayesian inference and markov chains likely will prove essential to the eventual mechanisms of any AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I would agree with you that Minsky would be unhappy at the prospect of designing AGI systems using techniques like neural nets that 1) are shaped only by supervised training examples, and 2) are unable to explain their reasoning or revise it directly and efficiently.  These approaches (disembodied probabilities) do not model thinking but only the outcome of thinking.  That's akin to not understanding the question but guessing correctly anyway, which is certainly useful, but it's also pretty uninteresting for someone like Minsky who wanted to understand the thought process, not just build a Magic 8 Ball.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would Minsky respond to recent advances in deep nets like variational autoencoders or adversarial nets or compound hierarchies of nets or end-to-end nets that implement broad skills?  Assuming they can scale up to implement AGI successfully, I believe he'd be rather unhappy with them.  The thought that his Society of Mind could turn out to me no more than a society of ten(?) kinds of deep nets that match patterns probabilistically and interact combinatorically -- I think this would be deeply disappointing to someone who spent a lifetime imagining and refining axioms and theories and proofs of cognition, only to find that the exquisite minds of Einstein and Beethoven might be little more than largely reactive engines highly attuned to variations in rote.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-05-30T06:09:50.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3410" PostTypeId="2" ParentId="3296" CreationDate="2017-05-30T09:11:30.350" Score="7" Body="&lt;p&gt;I came across the article &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;A visual proof that neural nets can compute any function&lt;/a&gt; that explains how a neural network can approximate any function, which pretty much answers my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose connections between neurons do not generally arrange themselves in the way described, though, so it would still be interesting to get insight into whether there are patterns of how the inter-neuron connections arrange themselves in neural networks trained to solve actual problems.&lt;/p&gt;&#xA;" OwnerUserId="7107" LastEditorUserId="2444" LastEditDate="2020-03-07T22:52:46.473" LastActivityDate="2020-03-07T22:52:46.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3413" PostTypeId="1" CreationDate="2017-05-31T04:36:21.277" Score="1" ViewCount="53" Body="&lt;p&gt;In Peter Norvig's &lt;a href=&quot;http://www.amazon.in/Paradigms-Artificial-Intelligence-Programming-Studies/dp/1558601910&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paradigms of Artificial Intelligence Programming&lt;/a&gt;, chapter 4, which is about the all-famous General Problem Solver (GPS). In this chapter, the author asks a question (4.4), which is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Not Looking after You don't Leap Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Write a program that keeps track of the remaining goals so that it doesn't get stuck considering only one possible operation when others will eventually lead to the goal.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;HINT:&lt;/strong&gt; have &lt;code&gt;achieve&lt;/code&gt; take an extra argument indicating the goals that remain to be achieved after the current goal is achieved. &lt;code&gt;achieve&lt;/code&gt; should succeed only if it can achieve the current goal and also &lt;code&gt;achieve-all&lt;/code&gt; the remaining goals.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's the description of these functions&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;achieve&lt;/code&gt;: We input a goal, the list of operators, and the current state, in the &lt;code&gt;achieve&lt;/code&gt; function, which checks if the goal is in the &lt;code&gt;state&lt;/code&gt; already, if not, then checks if the goal is recursive, if not, then finds all the &lt;code&gt;appropriate&lt;/code&gt; operators and then applies them, until the goal os achieved. Or &lt;code&gt;achieve&lt;/code&gt; returns a nil.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;achieve-all&lt;/code&gt;: This has a list of goals as an input, which it tries to achieve using the &lt;code&gt;achieve&lt;/code&gt; function. Moreover, it makes sure that all the goals are in the final state. Otherwise, it returns a nil.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The problem that I face is to find a relation between &amp;quot;getting stucked&amp;quot; over an operator and maintaining the number of goals remaining. Moreover, the version of &lt;code&gt;achieve&lt;/code&gt; in the GPS program checks for all the operators that can achieve the goal, and then apply only the one that does the work.&lt;/p&gt;&#xA;&lt;p&gt;Let's forget this for a while, and consider the hint. It says that we should have an extra argument maintaining the goals that are remaining. But how will it help a goal, which is stuck because of applying the wrong operator?&lt;/p&gt;&#xA;&lt;p&gt;There seems to be no relation between them. I know I'm missing something here, but I can't find that.&lt;/p&gt;&#xA;" OwnerUserId="7542" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-05-02T13:50:25.327" Title="Relation between the number of goals remaining and using the wrong operator in a General Problem Solver" Tags="&lt;problem-solving&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3415" PostTypeId="1" CreationDate="2017-05-31T07:35:04.387" Score="4" ViewCount="70" Body="&lt;p&gt;I am interested in the current state-of-the-art ways to use quick, greedy heuristics in order to speed up the learning in a Deep Q-Network in Reinforcement Learning. In classical RL, I initially set the Q-value for a state-action pair (S,a) based on the result of such a greedy heuristic run from state S with action a. Is this still a good idea in the setting of a neural network for the approximation of the Q-function, and if yes, what are the optimal ways of doing it? What are other ways of aiding the DQN with the knowledge from the greedy heuristics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References to state-of-the-art papers would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="7495" LastActivityDate="2017-05-31T08:41:03.460" Title="What are state-of-the-art ways of using greedy heuristics to initially set the weights of a Deep Q-Network in Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;intelligent-agent&gt;&lt;heuristics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3416" PostTypeId="2" ParentId="3415" CreationDate="2017-05-31T08:41:03.460" Score="2" Body="&lt;p&gt;You can check out &lt;a href=&quot;https://arxiv.org/abs/1602.04621&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bootstrapped DQN&lt;/a&gt;, with a demonstration &lt;a href=&quot;https://www.youtube.com/watch?v=e3KuV_d0EMk&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;. Without reading much of the paper, it seems the authors use a different sampling strategy and an action-guide for specific instances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to initially set weights for the network is to create a dataset of moves (correct, incorrect, etc, as long as they are relevant) and have the network initially learn the dataset. This also helps debugging, as you can see whether the network can actually learn the policy used in the dataset. After learning the dataset, use the same learned network with DQN and start with a smaller exploration rate (like 0.5 instead of 1.0).&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2017-05-31T08:41:03.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3417" PostTypeId="2" ParentId="153" CreationDate="2017-05-31T09:32:33.213" Score="2" Body="&lt;p&gt;One such paper I know of and which I implemented is &lt;a href=&quot;https://arxiv.org/abs/1507.02672&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semi-Supervised Learning using Ladder Networks&lt;/a&gt; . I quote here their description of the model:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Our approach follows Valpola (2015), who proposed a Ladder network where the auxiliary task is&#xA;  to denoise representations at every level of the model.  The model structure is an autoencoder with&#xA;  skip connections from the encoder to decoder and the learning task is similar to that in denoising&#xA;  autoencoders but applied to every layer, not just the inputs. The skip connections relieve the pressure&#xA;  to represent details in the higher layers of the model because,  through the skip connections, the decoder can recover any details discarded by the encoder.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For further explanations on the architecture check &lt;a href=&quot;https://arxiv.org/abs/1511.06430&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deconstructing the Ladder Network Architecture&lt;/a&gt; by Yoshua Bengio.&lt;/p&gt;&#xA;" OwnerUserId="7495" LastActivityDate="2017-05-31T09:32:33.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3418" PostTypeId="1" CreationDate="2017-05-31T13:41:17.817" Score="0" ViewCount="280" Body="&lt;p&gt;Are there approaches other than convolutions to learn features from images? Has there been any research to use approaches such as hashing (e.g. &lt;code&gt;p-hash&lt;/code&gt;, &lt;code&gt;diff-hash&lt;/code&gt; etc.) in lieu?&lt;/p&gt;&#xA;" OwnerUserId="7332" LastActivityDate="2017-06-08T23:26:52.483" Title="Feature extraction other than convolutions for images?" Tags="&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3419" PostTypeId="1" CreationDate="2017-06-01T02:50:27.133" Score="8" ViewCount="1108" Body="&lt;p&gt;A few papers I have come across say that &lt;a href=&quot;https://en.wikipedia.org/wiki/BLEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLEU&lt;/a&gt; is not an appropriate evaluation metric for chatbots, so they use the &lt;strong&gt;perplexity&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;First of all, what is &lt;em&gt;perplexity&lt;/em&gt;? How to calculate it? And why is perplexity a good evaluation metric for chatbots?&lt;/p&gt;&#xA;" OwnerUserId="7564" LastEditorUserId="2444" LastEditDate="2021-01-16T03:51:14.647" LastActivityDate="2021-01-16T03:51:14.647" Title="Why is the perplexity a good evaluation metric for chatbots?" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;metric&gt;&lt;bleu&gt;&lt;perplexity&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3420" PostTypeId="1" CreationDate="2017-06-01T08:49:15.507" Score="3" ViewCount="899" Body="&lt;p&gt;I am Reading &quot;&lt;a href=&quot;http://www.cs.toronto.edu/~graves/phd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supervised Sequence Labelling with Recurrent Neural Networks&lt;/a&gt;&quot; written by Alex Graves to try to understand LSTM networks and I am a bit confused about the equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, what I am confused about is the term &quot;state&quot;. When used in an equation (section 4.5.2), it says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3ANyI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3ANyI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that some system can be in a state, for example, due to the setup of values of different nodes in a graph. But how can a state be described in the case of a neural network and how can the equation above be explained other than that it is the state (or states of several timesteps as in recurrent neural networks) of a neural network?&lt;/p&gt;&#xA;" OwnerUserId="6645" LastEditorUserId="2444" LastEditDate="2019-04-29T16:03:04.537" LastActivityDate="2019-04-29T16:03:04.537" Title="What is a state in a recurrent neural network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;terminology&gt;&lt;long-short-term-memory&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3421" PostTypeId="2" ParentId="3398" CreationDate="2017-06-01T09:06:36.880" Score="2" Body="&lt;p&gt;&lt;em&gt;Disclaimer: this answer refers solely to TensorFlow, as my knowledge of the remaining frameworks is limited.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where did you hear that TensorFlow is near impossible to parallelize on more than 8 GPUs? With a large enough network, any number of GPUs can be used to speed-up training, as shown in the &lt;a href=&quot;https://www.tensorflow.org/tutorials/using_gpu&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;. This snippet of code introduces the multiplication of 2 constants in 2 separate GPUs. This allows you to run the graph on both simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for d in ['/gpu:2', '/gpu:3']:&#xA;    with tf.device(d):&#xA;        a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3])&#xA;        b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2])&#xA;        c.append(tf.matmul(a, b))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Even if there were some sort of limit on how many GPUs TensorFlow can use per process, TF can be &lt;a href=&quot;https://www.tensorflow.org/deploy/distributed&quot; rel=&quot;nofollow noreferrer&quot;&gt;distributed&lt;/a&gt; across processes and machines.&lt;/strong&gt; Asynchronous algorithms (like &lt;a href=&quot;https://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow noreferrer&quot;&gt;A3C&lt;/a&gt;) can be spread across, for example, 16 machines (what I am doing in my laboratory), where each machine uses its resources (in my case, I use CPU, but the change to a GPU would be trivial).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a complex enough network and a well implemented solution, you can take advantage of any hardware resources you have.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2017-06-01T09:06:36.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3422" PostTypeId="2" ParentId="2449" CreationDate="2017-06-01T19:47:39.513" Score="2" Body="&lt;p&gt;There seems to be no difference between 2 &amp;amp; 4 and 3 &amp;amp; 5. The inconsistency mentioned by Icyblade is due to the mechanics of the Pong environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Each action is repeatedly performed for a duration of k frames, where k is uniformly sampled from {2,3,4}&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the action is just repeated a different number of times due to randomness&lt;/p&gt;&#xA;" OwnerUserId="7579" LastActivityDate="2017-06-01T19:47:39.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3424" PostTypeId="2" ParentId="3420" CreationDate="2017-06-02T03:22:33.650" Score="3" Body="&lt;p&gt;So the equation that you mentioned is used during the backward pass in which back proppogation is performed in order to make the neural network more accurate. I think you are talking about the state during the forward pass which is completely different. In the forward pass, the neural network is simply run in order to evaluate or it is simply used as a model. The repeating module in long short term memory networks looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6TrLT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6TrLT.png&quot; alt=&quot;LSTM repeating module&quot;&gt;&lt;/a&gt;&#xA;As you can see there are many different parts to this module. There are three main parts. The first is the forget gate layer. This layer tells the cell state or the line running across the stop what to keep. The cell state is kept by this line:&lt;a href=&quot;https://i.stack.imgur.com/CthhV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CthhV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The entire network is based off of manipulating this cell state in order to get accurate results. The equation that you mentioned was related to backprop which is used to train the neural network. This is related to the cell state because it is used to calculate it during the backwards pass. @BlueMoon93 mentioned that this equation has t+1 one in it, but this is because as the recurrent neural network propogates backwards through each module the time goes from high to low. So to conclude, the cell state in an LSTM is one of the vectors that the neural network modifies based on input.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-06-02T03:22:33.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3426" PostTypeId="1" CreationDate="2017-06-02T14:35:48.083" Score="3" ViewCount="1730" Body="&lt;p&gt;I want to train a neural network with pictures of public figures (politicians, singers, etc), but I do not know if it's legal, I do not plan to show them in my project I only want to use them to train the neural network, can this cause legal problems?&lt;/p&gt;&#xA;" OwnerUserId="6850" LastActivityDate="2017-06-22T16:01:42.633" Title="Is it illegal to use pictures of public figures to train a neural network?" Tags="&lt;neural-networks&gt;&lt;legal&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3427" PostTypeId="2" ParentId="3426" CreationDate="2017-06-02T14:51:45.350" Score="7" Body="&lt;p&gt;If you use any pictures you find online, you can use them as you wish: as long as you don't (re)publish them under your name. Also, if you really want to play safe, never &lt;em&gt;upload&lt;/em&gt; them at all. Download them, use them, disregard them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;People can't proof you used certain pictures to train your network by looking at your network data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the 'big' companies out there, like Google, train their networks by using public photos. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And follow this chart:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bKcmE.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bKcmE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you are using it for a personal/educational project, it shouldn't be a problem.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastActivityDate="2017-06-02T14:51:45.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3428" PostTypeId="1" AcceptedAnswerId="3429" CreationDate="2017-06-03T16:44:45.890" Score="7" ViewCount="4921" Body="&lt;p&gt;How exactly are &quot;mutation&quot; and &quot;cross-over&quot; applied in the context of a genetic algorithm based on real numbers (as opposed to just bits)? I think I understood how those two phases are applied in a &quot;canonical&quot; context where chromosomes are strings of bits of a fixed length, but I'm not able to find examples for other situations. What would those phases look like on the domain of real numbers?&lt;/p&gt;&#xA;" OwnerUserId="242" LastEditorUserId="2444" LastEditDate="2020-02-17T01:53:41.387" LastActivityDate="2020-02-17T01:53:41.387" Title="How do mutation and crossover work with real-valued chromosomes?" Tags="&lt;genetic-algorithms&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;&lt;genetic-operators&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3429" PostTypeId="2" ParentId="3428" CreationDate="2017-06-03T17:31:21.210" Score="4" Body="&lt;p&gt;You have a genome with certain genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome = { GeneA: value, GeneB: value, GeneC: value }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So take for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome = { GeneA: 1, GeneB: 2.5, GeneC: 3.4 }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A few examples of mutation could be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Switch around two genes: &lt;code&gt;{ GeneA: 1, GeneB: 3.4, GeneC: 2.5 }&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Add/substract a random value from a gene: &lt;code&gt;{ GeneA: 0.9, GeneB: 2.5, GeneC: 3.4 }&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Suppose you have two genomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome1 = { GeneA: 1, GeneB: 2.5, GeneC: 3.4 }&#xA;genome2 = { GeneA: 0.4, GeneB: 3.5, GeneC: 3.2 }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A few examples of crossover could be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Taking the average: &lt;code&gt;{ GeneA: 0.7, GeneB: 3.0, GeneC: 3.3 }&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Uniform (50% chance): &lt;code&gt;{ GeneA: 0.4, GeneB: 2.5, GeneC: 3.2 }&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;N-point crossover: &lt;code&gt;{ GeneA: 1, | CROSSOVER POINT |  GeneB: 3.5, GeneC: 3.2 }&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You can be pretty imaginative when developing mutation and crossover methods.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="13360" LastEditDate="2018-05-09T19:33:56.213" LastActivityDate="2018-05-09T19:33:56.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3432" PostTypeId="2" ParentId="1815" CreationDate="2017-06-04T16:40:40.950" Score="2" Body="&lt;p&gt;Whether the move is found and how quick it is found depends on a few things. If I understand correctly, there is a sequence of many &quot;bad&quot; moves which lead to the &quot;big win&quot; move, and you are afraid that the MCTS algorithm will not get to the &quot;big win&quot; move because it will be selecting more promising moves further up the tree. Some things to think about (read also the Wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;MCTS article&quot;&gt;MCTS article&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;when doing playouts, you can do play your game only for a few further moves or down to the game end. Playing only a few moves further is obviously quicker, but in the extreme case you described it would not be the best choice. If you know about the existence of such scenarios, ensure to play the game to the end in the playouts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;when doing playouts, you can choose your moves/actions either randomly or based on some simple, greedy (quick) heuristics tailored to your problem. Are there maybe greedy heuristics designed to find or take into account such scenarios for your game/problem? If yes, implement them. It is then called a &quot;heavy playout&quot;. Compare the results to playouts using random moves.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you choose the actions using UCT (Upper Confidence Bound applied to Trees), then the first part of the expression is responsible for exploitation. Moves with high average win ratio are preferred. The second part though corresponds to exploration. If the exploration parameter is set high enough (test empirically for your problem), then moves with few simulations will be preferred. High exploration would be another way to find your golden move, in detriment of exploitation (read about the exploration/exploitation dilemma).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you describe a realistic game or problem scenario, we may be able to help you come up with a suitable strategy.&lt;/p&gt;&#xA;" OwnerUserId="7495" LastActivityDate="2017-06-04T16:40:40.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3433" PostTypeId="1" CreationDate="2017-06-05T09:00:41.587" Score="6" ViewCount="157" Body="&lt;p&gt;I recently read about algorithmic bias in facial recognition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is algorithmic bias due to the training dataset used, or is it due to something else?&lt;/p&gt;&#xA;" OwnerUserId="6687" LastEditorUserId="2444" LastEditDate="2019-08-09T12:26:13.030" LastActivityDate="2019-08-09T12:26:13.030" Title="Is algorithmic bias due to the training dataset used?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;facial-recognition&gt;&lt;algorithmic-bias&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3434" PostTypeId="2" ParentId="3433" CreationDate="2017-06-05T10:03:06.873" Score="4" Body="&lt;p&gt;As the name implies, algorithmic bias is related with the used algorithm. Due to the way it was programmed or devised, the algorithm will be biased in some of its samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://cacm.acm.org/magazines/2016/10/207759-battling-algorithmic-bias/fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;Communications of the ACM&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[Algorithms] often inadvertently pick up the human biases that are incorporated when the algorithm is programmed, or when humans interact with that algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A machine learning model can be biased if the wrong data-set is used, of course. That is usually only referred to as &lt;em&gt;bias&lt;/em&gt;, and often associated with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff&quot; rel=&quot;nofollow noreferrer&quot;&gt;bias-variance tradeoff&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2017-06-05T10:03:06.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3435" PostTypeId="1" AcceptedAnswerId="8739" CreationDate="2017-06-05T17:36:55.457" Score="3" ViewCount="352" Body="&lt;p&gt;I wanted to train a chatbot for answering questions from books. I am trying to use &lt;a href=&quot;https://arxiv.org/abs/1506.07285&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Memory Networks&lt;/a&gt; to do so.&lt;/p&gt;&#xA;&lt;p&gt;How can I generate a data set, as Facebook did in the case of &lt;a href=&quot;https://research.fb.com/downloads/babi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bAbI tasks&lt;/a&gt;, so that it can tackle a variety of questions on the data set?&lt;/p&gt;&#xA;" OwnerUserId="7330" LastEditorUserId="2444" LastEditDate="2021-01-26T17:30:36.130" LastActivityDate="2021-01-26T17:30:36.130" Title="How to generate a dataset for question answering from books (like Facebook's bAbI)?" Tags="&lt;natural-language-processing&gt;&lt;training&gt;&lt;datasets&gt;&lt;chat-bots&gt;&lt;question-answering&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3438" PostTypeId="2" ParentId="3433" CreationDate="2017-06-06T11:27:08.130" Score="4" Body="&lt;p&gt;Just to add to what has already been said in @BlueMoon93's &lt;a href=&quot;https://ai.stackexchange.com/a/3434/7550&quot;&gt;answer&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithmic bias is the bias built into the algorithm. Now for the long answer: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As stated by the so called &lt;a href=&quot;https://en.wikipedia.org/wiki/No_free_lunch_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;No free lunch theorem&lt;/a&gt;: regardless of the algorithm you use, you cant get learning &quot;for free&quot;(i.e by just looking at the training examples). The reason for this is that the only thing you know about the data is based on the limited examples you have seen in the training set. To &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalization_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalize&lt;/a&gt;, your algorithm has to make some sort of assumptions about the underlying nature of the dataset and the manner in which it can be represented/interpreted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Built into every algorithm is a set of assumptions about the dataset, for example, built into the &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolution neural network&lt;/a&gt; is the assumption that the dataset(e.g images you are using to train your Convolution neural network) can be understood by mimicking how the human eye is &lt;a href=&quot;https://www.youtube.com/watch?v=Cw5PKV9Rj3o&quot; rel=&quot;nofollow noreferrer&quot;&gt;known to work&lt;/a&gt;(i.e a bias). Some algorithms may have such strong biases such that they are incapable of learning certain kinds of functions. For example, linear models assumes that the underlying data is linear(a bias), note this might not be the case for the dataset at hand in which case the bias built into the model was bad for the dataset at hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by @BlueMoon93, there is another form of bias, commonly referred to simply as &lt;a href=&quot;http://scott.fortmann-roe.com/docs/BiasVariance.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;bias&quot;&lt;/a&gt; which is introduced by the dataset used. &lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-06-07T14:29:57.697" LastActivityDate="2017-06-07T14:29:57.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3439" PostTypeId="2" ParentId="3397" CreationDate="2017-06-06T12:23:36.573" Score="9" Body="&lt;p&gt;I don't think the AI has gotten to that point yet. Here are some of the interesting papers on the subject:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A paper was recently written that attempted to &lt;a href=&quot;http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervised.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;generate jokes using unsupervised learning&lt;/a&gt;. The jokes are formulaic: they're all of the form &quot;I like my X like I like my Y: Z&quot; where X and Y are nouns, and Z is an adjective that can describe both X and Y. Here are some of the jokes generated in this paper: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I like my relationships like I like my source, open&#xA;I like my coffee like I like my war, cold&#xA;I like my boys like I like my sectors, bad&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How funny these jokes are is a matter of personal taste I guess. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another paper by &lt;a href=&quot;http://www.aclweb.org/anthology/N16-1016&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dario Bertero and Pascale Fung&lt;/a&gt; makes use of an LSTM to predict humor from a dataset of the Big Bang theory shows. This is not generating jokes but finding out where the jokes are said in this dataset (so theoretically, the resulting labelled dataset can hopefully be used to train a model to create jokes). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yet another paper is that by &lt;a href=&quot;https://web.stanford.edu/class/cs224n/reports/2760332.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;He Ren, Quan Yang&lt;/a&gt;. Unlike &lt;a href=&quot;http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervised.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the first paper mentioned above&lt;/a&gt; which was unsupervised, this is a supervised learning model. Their neural network model, generates jokes such as: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Apple is teaming up with Playboy Magazine in the self driving office.&#xA;One of the top economy in China , Lady Gaga says today that Obama is legal.&#xA;Google Plus has introduced the remains that lowers the age of coffee.&#xA;According to a new study , the governor of film welcome the leading actor of Los Angeles area , Donald Trump .&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My two cents&lt;/em&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of this writing, it appears that Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models are by far the most promising way to go about it. Maybe if you find some really cool data you can come up with some funny jokes, similar to how &lt;a href=&quot;http://lewisandquark.tumblr.com/post/159302925452/the-neural-network-generated-pickup-lines-that-are&quot; rel=&quot;nofollow noreferrer&quot;&gt;Janelle Shane&lt;/a&gt; was able to generate what I find to be really funny pickup lines such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Are you a 4loce? Because you’re so hot!&#xA;I want to get my heart with you.&#xA;You are so beautiful that you know what I mean.&#xA;I have a cenver? Because I just stowe must your worms.&#xA;Hey baby, I’m swirked to gave ever to say it for drive. &#xA;If I were to ask you out?&#xA;You must be a tringle? Cause you’re the only thing here.&#xA;I’m not on your wears, but I want to see your start.&#xA;You are so beautiful that you make me feel better to see you.&#xA;Hey baby, you’re to be a key? Because I can bear your toot?&#xA;I don’t know you.&#xA;I have to give you a book, because you’re the only thing in your eyes.&#xA;Are you a candle? Because you’re so hot of the looks with you.&#xA;I want to see you to my heart.&#xA;If I had a rose for every time I thought of you, I have a price tighting.&#xA;I have a really falling for you.&#xA;Your beauty have a fine to me.&#xA;Are you a camera? Because I want to see the most beautiful than you.&#xA;I had a come to got your heart.&#xA;You’re so beautiful that you say a bat on me and baby.&#xA;You look like a thing and I love you.&#xA;Hello.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="4709" LastEditDate="2019-12-20T22:31:41.440" LastActivityDate="2019-12-20T22:31:41.440" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3440" PostTypeId="1" CreationDate="2017-06-06T16:38:48.323" Score="2" ViewCount="151" Body="&lt;p&gt;I'm developing a log analyzer to predict and find errors in an equipment. Each logged data contains the following format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  Timestamp | Log source | type of message | message&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each entry log I want to represent by one pixel RGB because considering the 24 bits, is possible to represent the last 3 parameters ( Log source, type of message and message), but I don't have bits enough to represent the timestamp ( this data I will represent by the difference o previous timestamp &quot;delta of time&quot;), the resolution of time is second, and sometimes we have a large time between one log of another. The example above illustrate the situation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v18IE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v18IE.png&quot; alt=&quot;Log to Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This project has the purpose of analyzing the log to find and predict errors, and this preprocessing is used to simplify the data entry to the machine learning algorithm, this is a good way to represent data to an RNN? Or for this kind of problem exist a better way to make an analyses?&lt;/p&gt;&#xA;" OwnerUserId="6978" LastEditorUserId="2444" LastEditDate="2019-06-28T11:16:15.890" LastActivityDate="2019-06-28T11:16:15.890" Title="Preprocessing of training dataset for machine learning" Tags="&lt;datasets&gt;&lt;recurrent-neural-networks&gt;&lt;prediction&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3441" PostTypeId="1" CreationDate="2017-06-06T20:33:36.100" Score="1" ViewCount="496" Body="&lt;p&gt;Recently, I found out about somewhat famous &lt;a href=&quot;http://yudkowsky.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eliezer Yudkowsky&lt;/a&gt; and &lt;a href=&quot;https://intelligence.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Intelligence Research Institute&lt;/a&gt; he founded. Their philosophy and organisation seem interesting but I'm curious about their credibility. I'm pretty sure this is not a con and they seem to be producing &lt;a href=&quot;https://intelligence.org/all-publications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a lot of articles&lt;/a&gt;. However, few of those are published and none in the journals mentioned &lt;a href=&quot;https://ai.stackexchange.com/q/2306/2444&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, is MIRI doing genuine high-quality research?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="7704" LastEditorUserId="2444" LastEditDate="2020-06-16T10:38:23.303" LastActivityDate="2020-06-16T10:38:23.303" Title="Is MIRI doing genuine high-quality research?" Tags="&lt;research&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3442" PostTypeId="1" AcceptedAnswerId="3443" CreationDate="2017-06-07T00:30:18.617" Score="5" ViewCount="174" Body="&lt;p&gt;If we have well designed autonomous AI vehicles, then why won't the USA government allow it to be witnessed in the public?&lt;/p&gt;&#xA;" OwnerUserId="7720" LastEditorUserId="2444" LastEditDate="2019-04-30T13:40:53.627" LastActivityDate="2019-04-30T13:46:08.677" Title="Why is it illegal for Google's autonomous car to drive on the road by itself?" Tags="&lt;ethics&gt;&lt;autonomous-vehicles&gt;&lt;google&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3443" PostTypeId="2" ParentId="3442" CreationDate="2017-06-07T01:50:17.213" Score="3" Body="&lt;p&gt;The question is based on a false fact: in Michigan, it is currently legal (under certain conditions described &lt;a href=&quot;http://www.ncsl.org/research/transportation/autonomous-vehicles-self-driving-vehicles-enacted-legislation.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; ) for an autonomous car to operate without a driver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason that the federal government has not enacted any direct legislation (although they have enacted guidelines) on autonomous cars is because it is still a developing technology (as great as those Cruise videos look, I wouldn't trust most companies working on the technology to have driverless cars on the road without operators) and that it arguably falls under the state governments jurisdictions. Some companies have appealed to the House to pass national legislation to allow autonomous testing (&lt;a href=&quot;https://www.forbes.com/sites/alanohnsman/2017/02/14/gm-toyota-and-lyft-urge-congress-to-set-nationwide-self-driving-car-standards/#66101394376e&quot; rel=&quot;nofollow noreferrer&quot;&gt;GM, Toyota, Lyft&lt;/a&gt;), but nothing has come of it yet.&lt;/p&gt;&#xA;" OwnerUserId="7723" LastEditorUserId="2444" LastEditDate="2019-04-30T13:46:08.677" LastActivityDate="2019-04-30T13:46:08.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3444" PostTypeId="2" ParentId="2632" CreationDate="2017-06-07T06:19:02.510" Score="5" Body="&lt;p&gt;Just to add to what has been said in @MikeWise's brilliant answer,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W5R2l.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W5R2l.png&quot; alt=&quot;why deep learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Like everything, it all boils down to the dataset at hand, neural networks are good on other datasets but at the same time, they will be bad on other datasets. When it comes to unstructured problems(e.g &lt;a href=&quot;https://deeplearning4j.org/use_cases&quot; rel=&quot;noreferrer&quot;&gt;visuals, text, sound&lt;/a&gt;), at this time neural networks appear to be the best algorithm. That said, when it comes to structured data, a quick &lt;a href=&quot;http://www.chioka.in/kaggle-competition-solutions/&quot; rel=&quot;noreferrer&quot;&gt;scan at the type of algorithm used to win online data science competitions&lt;/a&gt; reveal that, the so called machine learning algorithms such as &lt;a href=&quot;https://github.com/dmlc/xgboost&quot; rel=&quot;noreferrer&quot;&gt;XGboost&lt;/a&gt; rank supreme. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When it comes to other models, feature engineering plays &lt;a href=&quot;http://blog.kaggle.com/2014/08/01/learning-from-the-best/&quot; rel=&quot;noreferrer&quot;&gt;a big role&lt;/a&gt; in the algorithm's efficiency. Feature Engineering is generally a tricky thing to do and do right. Deep learning algorithms don't require as much feature engineering(if any at all) compared to other algorithms, in fact &lt;a href=&quot;https://iksinc.wordpress.com/2015/12/18/feature-engineering-and-deep-learning/&quot; rel=&quot;noreferrer&quot;&gt;they learn features on their own&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the Google guys say they &lt;a href=&quot;https://www.youtube.com/watch?v=30rx3dBPbIs&quot; rel=&quot;noreferrer&quot;&gt;didn't see deep learning coming&lt;/a&gt; who is to rule out the possibility of some so called machine learning algorithm coming out and taking over the world by storm?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Here is a Poll on what data scientist said when asked: if deep learning match the &lt;a href=&quot;http://www.kdnuggets.com/polls/2016/deep-learning-hype-reality.html&quot; rel=&quot;noreferrer&quot;&gt;hype in real world application?&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Even some of the popular deep learning applications like Google's AlphaGo &lt;a href=&quot;https://backchannel.com/has-deepmind-really-passed-go-adc85e256bec&quot; rel=&quot;noreferrer&quot;&gt;aren't 100% deep learning&lt;/a&gt;, instead they are part deep learning, part good old &quot;machine learning&quot;. My 2 cent is, maybe we shouldn't rule out other machine learning algorithms yet.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-06-07T06:53:51.070" LastActivityDate="2017-06-07T06:53:51.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3445" PostTypeId="2" ParentId="40" CreationDate="2017-06-07T07:48:41.353" Score="6" Body="&lt;p&gt;The original paper&lt;sup&gt;1&lt;/sup&gt; that proposed neural network dropout is titled: &lt;a href=&quot;http://www.jmlr.org/papers/volume15/srivastava14a.old/source/srivastava14a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout: A simple way to prevent neural networks from overfitting&lt;/a&gt;. That tittle pretty much explains in one sentence what Dropout does. Dropout works by randomly selecting and removing neurons in a neural network during the training phase. Note that dropout is not applied during testing and that the resulting network doesn't dropout as part of predicting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This random removal/dropout of neurons prevents excessive co-adaption of the neurons and in so doing, reduce the likelihood of the network &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfiting&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The random removal of neurons during training also means that at any point in time, only a portion of the original network is trained. This has the effect that you end up sort of training multiple sub-networks, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vMidK.jpg&quot; alt=&quot;droup as an ensembler&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is from this repeated training of sub-networks as opposed to the entire network where the notion of neural network dropout being a sort of ensemble technique comes in. I.e the training of the sub-networks is similar to training numerous, relatively weak algorithms/models and combining them to form one algorithm that is more powerful than the individual parts.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;References:&lt;/em&gt;&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;sup&gt;1&lt;/sup&gt;: Srivastava, Nitish, et al. &quot;Dropout: A simple way to prevent neural networks from overfitting.&quot; The Journal of Machine Learning Research 15.1 (2014): 1929-1958.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="14723" LastEditDate="2018-04-12T02:30:04.720" LastActivityDate="2018-04-12T02:30:04.720" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3451" PostTypeId="1" CreationDate="2017-06-07T14:34:03.020" Score="4" ViewCount="677" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I've seen events, like &lt;a href=&quot;https://cogx.co/session-topics/mentalhealth/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CogX&lt;/a&gt; and articles which describe how&#xA;  machine learning techniques or algorithms can be used to diagnose&#xA;  mental health issues.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here is my question&lt;/strong&gt;;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can artificial intelligence and machine learning algorithms or techniques be used in diagnosing mental health issues, besides for e.g. Facebook using machine learning algorithms to detect people who may commit suicide?&lt;/p&gt;&#xA;" OwnerUserId="7749" LastEditorUserId="1581" LastEditDate="2017-08-27T20:30:54.680" LastActivityDate="2017-11-03T02:52:17.487" Title="Diagnosing mental health problems using Artificial Intelligence / Machine Learning techniques" Tags="&lt;machine-learning&gt;&lt;research&gt;&lt;healthcare&gt;" AnswerCount="4" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3452" PostTypeId="2" ParentId="3442" CreationDate="2017-06-07T14:50:32.403" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/disengagement_report_2016&quot; rel=&quot;nofollow noreferrer&quot;&gt;This answer&lt;/a&gt; addresses the incorrect assumption that the US government prohibits it (that is, here's no federal law against it, but state laws vary), so I will address the incorrect assumption that there now exists &quot;well-designed AI&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the &lt;a href=&quot;https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/disengagement_report_2016&quot; rel=&quot;nofollow noreferrer&quot;&gt;actual reports&lt;/a&gt; submitted by the autonomous car companies who have tested in California, you'll see that most can't go more than a couple of hours of driving without an &lt;strong&gt;unexpected&lt;/strong&gt; need for operator intervention.  Waymo's (Google's autonomous car spinoff) system is more reliable as far as unexpected interventions, but their report emphasizes that &lt;em&gt;routine&lt;/em&gt; operator interventions happen many times a day.  Autonomous vehicles that can operate without a driver in any but the most controlled circumstances are still years away.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2444" LastEditDate="2019-04-30T13:43:50.113" LastActivityDate="2019-04-30T13:43:50.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3453" PostTypeId="1" AcceptedAnswerId="4973" CreationDate="2017-06-07T15:17:32.610" Score="0" ViewCount="90" Body="&lt;p&gt;I've read through a few papers on next frame prediction from a sequence of frames and several of them use spatial transformations (&lt;a href=&quot;https://arxiv.org/abs/1506.02025&quot; rel=&quot;nofollow noreferrer&quot;&gt;STNs&lt;/a&gt;). See &lt;a href=&quot;https://arxiv.org/abs/1605.08104&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; as an example. I want to know what are the pros and cons of using an STN to predict the next frame. Are there any assumptions that must be made about the data besides &quot;Consecutive frames are all approximately affine transformations of each other&quot;?&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="1671" LastEditDate="2017-12-21T21:07:15.593" LastActivityDate="2018-01-11T15:33:48.933" Title="What are the pros and cons of using a spatial transformation network to predict the next video frame?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;deep-neural-networks&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3454" PostTypeId="2" ParentId="40" CreationDate="2017-06-07T15:23:39.500" Score="3" Body="&lt;p&gt;There are some great answers here. The simplest explanation I can give for dropout is that it randomly excludes some neurons and their connections from the network, while training, to stop neurons from &quot;co-adapting&quot; too much. It has the effect of making each neuron apply more generally and is excellent for stopping overfitting for large neural networks.&lt;/p&gt;&#xA;" OwnerUserId="7750" LastActivityDate="2017-06-07T15:23:39.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3456" PostTypeId="2" ParentId="3418" CreationDate="2017-06-07T15:36:05.953" Score="0" Body="&lt;p&gt;I think your question addresses the fact that convolutional neural networks utilize convolution and not some other mechanism. This answer addresses other feature extraction techniques you can use on image. In general, there are a number of other feature extraction mechanisms you can look into.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Histogram oriented gradients. Counts the occurrences of gradients in an image and generates bins. These bins can be then used directly as features.&lt;/li&gt;&#xA;&lt;li&gt;Color data. You can extract color histograms of various granularities and use this data to train a model.&lt;/li&gt;&#xA;&lt;li&gt;Hough Transform - Can identify shapes and lines and other spatial features.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;On top of these we also have numerous techniques on top of convolutions like gradients, curvature finding, corner detection, and ridge finding.&lt;/p&gt;&#xA;" OwnerUserId="7750" LastActivityDate="2017-06-07T15:36:05.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3457" PostTypeId="1" AcceptedAnswerId="3466" CreationDate="2017-06-07T18:20:37.757" Score="2" ViewCount="145" Body="&lt;p&gt;What are the ethical and legal issues of self driving cars being released in the UK?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question came up on our exam today and I was left in a daze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I initially thought it would be issues like the legal driving age for self driving cars since they are AI do you need a minimum age limit?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another one I thought was whether or not you need driving licenses for AI cars?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please list all the possible ethical &lt;strong&gt;and&lt;/strong&gt; legal issues that surround AI controlled cars?&lt;/p&gt;&#xA;" OwnerUserId="7756" LastEditorUserId="8" LastEditDate="2017-06-13T19:54:39.130" LastActivityDate="2017-06-13T19:54:39.130" Title="What are the ethical and legal issues of self driving cars being released in the UK?" Tags="&lt;autonomous-vehicles&gt;&lt;ethics&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3461" PostTypeId="2" ParentId="3389" CreationDate="2017-06-08T01:15:58.567" Score="-3" Body="&lt;p&gt;yes it is feasible, but consider that integer factorization problem is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Integer_factorization#Difficulty_and_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP-something problem&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/BQP&quot; rel=&quot;nofollow noreferrer&quot;&gt;BQP problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because of this, it is impossible that a neural network purely based on classical computing finds prime number with 100% accuracy, unless P=NP.&lt;/p&gt;&#xA;" OwnerUserId="7338" LastActivityDate="2017-06-08T01:15:58.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3462" PostTypeId="2" ParentId="3441" CreationDate="2017-06-08T16:23:18.237" Score="4" Body="&lt;p&gt;Whether or not it's produced &lt;em&gt;good&lt;/em&gt; research is a question I can't answer.  But I find it interesting and it seems reliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, this paper on Parametric &lt;a href=&quot;https://arxiv.org/abs/1602.04184&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bounded Löb's Theorem and Robust Cooperation of Bounded Agent&lt;/a&gt; is drawn from Cornell University's site.  Likewise this recent paper on &lt;a href=&quot;https://arxiv.org/abs/1609.03543&quot; rel=&quot;nofollow noreferrer&quot;&gt;Logical Induction&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, they also seem to publish papers that have not been subject to peer review, which is an issue not specific to the site, and seems to more widespread phenomenon per easy access to digital files in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In relation to MIRI specifically, possibly this is a function of an an applied field advancing so rapidly that publication in journals is not seen as a priority, or even particularly relevant.   &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-06-08T21:37:12.260" LastActivityDate="2017-06-08T21:37:12.260" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3463" PostTypeId="1" CreationDate="2017-06-08T19:40:12.340" Score="3" ViewCount="1244" Body="&lt;p&gt;I have a data set with four inputs and one output. I need to infer the 4 inputs given an output. What is the best way to do this?&lt;/p&gt;&#xA;" OwnerUserId="6978" LastActivityDate="2018-01-06T16:38:14.560" Title="Neural network Model to infer inputs given an output" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3464" PostTypeId="2" ParentId="1515" CreationDate="2017-06-08T19:55:43.357" Score="1" Body="&lt;p&gt;There is definitely a usage and etymological dimension to this issue, which I can elaborate on if you'd like.  (It's an issue I come across frequently, and is particularly relevant in regards to original work.)&lt;/p&gt;&#xA;&lt;p&gt;The use of &amp;quot;Agent&amp;quot; may have arisen partly out of Game Theory, with ideas like &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-agent_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-agent systems&lt;/a&gt;&amp;quot;, possibly because it's quite general (i.e. one does not have to say participant, or player, or actor, etc., and it is species non-specific.)&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Automata&amp;quot; seems to be favoured in some circles, and although &lt;a href=&quot;https://en.wikipedia.org/wiki/Automaton&quot; rel=&quot;nofollow noreferrer&quot;&gt;this term was originally reserved for physical, mechanical devices&lt;/a&gt;, it now encompasses a range of physical and information-based systems (with perhaps a contemporary emphasis on the virtual. A famous extension is the categorization of Conway's Game of Life as a &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cellular_automaton&quot; rel=&quot;nofollow noreferrer&quot;&gt;cellular automata&lt;/a&gt;&amp;quot;. Conversely, &amp;quot;robot&amp;quot; would seem to have usurped &amp;quot;automaton&amp;quot; for physical systems.)&lt;/p&gt;&#xA;&lt;p&gt;I tend to use an agent when I'm thinking in an economic sense, or in a game theory sense as related to computing [see &lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rational Agent&lt;/a&gt;], and automata when thinking in a procedural sense in terms of algorithmic intelligence [see &lt;a href=&quot;https://en.wikipedia.org/wiki/Automata_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automata Theory&lt;/a&gt;].  But the distinction is partly semantic.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Robot#Origin_of_the_term_.27robot.27&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robot&lt;/a&gt; seems to be avoided in the academic literature in regards to information systems, possibly because of the deeply entrenched popular understanding of &amp;quot;robot&amp;quot; as physical automata.  The use of the informal &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bot&quot; rel=&quot;nofollow noreferrer&quot;&gt;bots&lt;/a&gt;&amp;quot; for describing information-based systems seems to have arisen in the public sphere, almost certainly a product of &lt;a href=&quot;https://en.wikipedia.org/wiki/Hacker_culture&quot; rel=&quot;nofollow noreferrer&quot;&gt;hacker culture&lt;/a&gt;, which has a tradition of playfulness.  I like the term bots as it is being currently applied, but I feel it has the connotation of &lt;a href=&quot;https://en.wikipedia.org/wiki/Triviality_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;trivial&lt;/a&gt; (i.e. small and discrete) or massively multi-agent systems, and haven't really seen it used for &amp;quot;strong&amp;quot; narrow AI.*&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Note: &amp;quot;&lt;a href=&quot;https://en.wiktionary.org/wiki/agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agent&lt;/a&gt;&amp;quot; does not automatically carry the concept of autonomy, although it is sometimes implied.  The use of the term is in the sense of &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Agency_(philosophy)&quot; rel=&quot;nofollow noreferrer&quot;&gt;agency&lt;/a&gt;&amp;quot; as an action taken by an agent.  Agents may be acting on their own behalf, or as &lt;a href=&quot;https://en.wiktionary.org/wiki/proxy#English&quot; rel=&quot;nofollow noreferrer&quot;&gt;proxies&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;*&amp;quot;Strong&amp;quot; in relation to AI is a term that seems to be evolving.  Initially reserved for &lt;a href=&quot;https://en.wikipedia.org/wiki/Strong_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence and conscious algorithms&lt;/a&gt;, I've seen recent mentions by respected scholars of &amp;quot;strong narrow AI&amp;quot; to refer to recent milestones in Machine Learning. However, the term &amp;quot;strong&amp;quot; and &amp;quot;weak&amp;quot; in relation to mathematics and AI have a history in &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial game theory&lt;/a&gt; per the concept of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved game&lt;/a&gt;. Thus, there exist &amp;quot;strong AI&amp;quot; for games such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Nim&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nim&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Hex_(board_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hex&lt;/a&gt;, and Tic-Tac-Toe.&lt;/p&gt;&#xA;&lt;p&gt;Again, there is an element of subjectivity here, but I tend to favour the CGT approach for two reasons:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It's a mathematical definition, unlike the philosophical definitions (&amp;quot;consciousness&amp;quot; and &amp;quot;humanlike&amp;quot; re: AGI)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Artificial Intelligence is rooted in combinatorial games.  [&lt;a href=&quot;https://fr.wikipedia.org/wiki/Nimatron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nimatron&lt;/a&gt; may have been the first functional algorithmic intelligence and combinatorial games have been used as milestones since that time, most notably with &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Blue&lt;/a&gt;, &lt;a href=&quot;https://motherboard.vice.com/en_us/article/the-chess-engine-that-died-so-alphago-could-live-giraffe-matthew-lai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe Chess&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt;.  Non-trivial Combinatorial games are useful because they have simple parameters but &lt;a href=&quot;https://en.wikipedia.org/wiki/Complexity#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;complexity&lt;/a&gt; akin to nature.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My sense of how &amp;quot;strong&amp;quot; is starting to be used is for algorithms, such as Machine Learning systems, that can outperform humans within a given set of parameters.  Thus &amp;quot;strong narrow AI&amp;quot;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;On Intelligence:&lt;/p&gt;&#xA;&lt;p&gt;The term &amp;quot;intelligence&amp;quot; is highly subjective and the only functionally meaningful definition I've come across is the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;bounded rationality&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="36737" LastEditDate="2021-03-31T22:27:38.583" LastActivityDate="2021-03-31T22:27:38.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3465" PostTypeId="1" AcceptedAnswerId="3470" CreationDate="2017-06-08T20:10:18.253" Score="2" ViewCount="2034" Body="&lt;p&gt;I am looking to train a dataset that would output a sequence of letters (I'm using this for peptide sequences). Since I have 22 different possibilities of amino acids, I need to output a vector that contains multiple amino acids with varying frequencies. For example, an output would look like [0,1,3,2,0,2,0,3...] (a 22-long vector).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I train a neural network to output that type of vector?&lt;/p&gt;&#xA;" OwnerUserId="7773" LastActivityDate="2017-06-13T10:48:08.240" Title="Multi-label Classification with non-binary outputs" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="7" ClosedDate="2020-12-17T21:08:10.193" ContentLicense="CC BY-SA 3.0" />
  <row Id="3466" PostTypeId="2" ParentId="3457" CreationDate="2017-06-08T20:39:34.613" Score="2" Body="&lt;h1&gt;LEGAL&lt;/h1&gt;&#xA;&lt;p&gt;&lt;strong&gt;Legislative:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://cyberlaw.stanford.edu/wiki/index.php/Automated_Driving:_Legislative_and_Regulatory_Action&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cyberlaw.stanford.edu/wiki/index.php/Automated_Driving:_Legislative_and_Regulatory_Action&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ncsl.org/research/transportation/autonomous-vehicles-self-driving-vehicles-enacted-legislation.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ncsl.org/research/transportation/autonomous-vehicles-self-driving-vehicles-enacted-legislation.aspx&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://spectrum.ieee.org/transportation/advanced-cars/selfdriving-cars-will-be-ready-before-our-laws-are&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://spectrum.ieee.org/transportation/advanced-cars/selfdriving-cars-will-be-ready-before-our-laws-are&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Liability&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.usatoday.com/story/money/cars/2017/04/03/tesla-mishap-raises-issues-self-driving-liability/99880620/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.usatoday.com/story/money/cars/2017/04/03/tesla-mishap-raises-issues-self-driving-liability/99880620/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nytimes.com/2014/05/14/upshot/when-driverless-cars-break-the-law.html?_r=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nytimes.com/2014/05/14/upshot/when-driverless-cars-break-the-law.html?_r=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Regulation:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.theverge.com/2016/9/19/12981448/self-driving-car-guidelines-obama-foxx-dot-nhtsa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.theverge.com/2016/9/19/12981448/self-driving-car-guidelines-obama-foxx-dot-nhtsa&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Law Review&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.americanbar.org/publications/youraba/2015/october-2015/are-we-there-yet--the-legal-aspects-of-driverless-cars.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.americanbar.org/publications/youraba/2015/october-2015/are-we-there-yet--the-legal-aspects-of-driverless-cars.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;ETHICAL&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.scientificamerican.com/article/driverless-cars-will-face-moral-dilemmas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scientificamerican.com/article/driverless-cars-will-face-moral-dilemmas/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://news.stanford.edu/2017/05/22/stanford-scholars-researchers-discuss-key-ethical-questions-self-driving-cars-present/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://news.stanford.edu/2017/05/22/stanford-scholars-researchers-discuss-key-ethical-questions-self-driving-cars-present/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This should be enough to get you started.  (In terms of how it may apply in the UK, I leave that for you to determine;)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-06-08T20:39:34.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3467" PostTypeId="2" ParentId="3418" CreationDate="2017-06-08T23:26:52.483" Score="1" Body="&lt;p&gt;Some more complex and better outcomes are those that include the MPEG-7 developed by the MPEG group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These descriptors specify the video standard in question. They are split for color, shape, texture and movement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Color Descriptors&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Color Space. Specifies the data type of the color space in which they are expressed or work other color descriptors. Color spaces that it contemplated are: RGB (Red, Green Blue), YCrCb (Luminance, Chrominance), HSV (Hue, Saturation, Value), HMMD (Hue, Maximum, Minimum, and Difference). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Color Quantization. This descriptor defines a uniform quantization of a given color space. &#xA;Dominant Color(s). This descriptor is the most suitable to be used in images, or regions, in which a small number of colors is sufficient to characterize the information of one determined region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scalable Color. This descriptor consists in a color histogram in the HSV space. It is useful in image to image comparisons or searches based on color characteristics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Color Layout. This descriptor allows representing the color spatial distribution within the images in a very compact way, so the recovery is realized with great efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Color Structure Descriptor. This descriptor characterizes the color distribution in an image. It builds a color histogram, which assigns most importance to the colors that appears most often and spread across the image. GoF/GoP Color (Group of Frames/Group of Pictures). This descriptor is an extension of the Scalable Color descriptor, which, unlike the latter, is applied to video sequences (a collection of images) instead of only an image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Texture Descriptors&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Homogeneous Texture. This descriptor emerged as an important tool when looking for and choosing within large collections of images of great visual similarity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Texture Browsing. This descriptor specifies the perceptual characterization of one particular texture, and pretend be similar to the characterization that a human eye makes, according to regularity terms, coarseness and directivity. &#xA;Edge Histogram. It is a descriptor that gives us information on the type of contours or edges that appear in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Shape Descriptors&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Region Shape. The object´s shape in an image may consist of a single region or a set of regions, so this descriptor is useful for this type of characterization. &#xA;Contour Shape. It is characterized by very well represented contour features which facilitates subsequent search and retrieval; it is robust to motion, to occlusions and to different perspectives; it is extremely compact. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shape 3D. The Shape 3D allows to describe in details the shape in 3D. This tool is very useful today due to continuous development of multimedia technologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was taked from my paper: The MPEG-7 Visual Descriptors: a Basic Survey&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards&lt;/p&gt;&#xA;" OwnerUserId="7776" LastActivityDate="2017-06-08T23:26:52.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3468" PostTypeId="2" ParentId="3463" CreationDate="2017-06-09T03:35:34.677" Score="2" Body="&lt;p&gt;I'm assuming that you mean your inputs are 4-dimensional vectors and your outputs are 1 dimensional vectors. If you think that the output vectors give you enough information to completely determine the inputs (at least approximately), then just treat your 1D &quot;outputs&quot; as inputs and your 4D &quot;inputs&quot; as outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another case is that very different inputs are mapped to similar outputs in your dataset, and you really just want to compute a prototype input for a given output. For example, you might want to produce an image of the letter &quot;a&quot; from only its one-hot label. There are many ways to do this. Here's one that I think is cool: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, train a neural net the way you normally would with 4D inputs mapping to 1D outputs. Now let's say you're given a 1D output &lt;em&gt;y&lt;/em&gt;. Generate some 4D noise &lt;em&gt;x'&lt;/em&gt; and pass it through the net to obtain &lt;em&gt;y'&lt;/em&gt;. Now pretend that the values in this input are parameters in your network, and hold your actual parameters constant. Use &lt;em&gt;y&lt;/em&gt; and &lt;em&gt;y'&lt;/em&gt; to compute a loss, and perform gradient descent on your synthetic input. If you repeat this enough times, you should get something that your network thinks deserves the output &lt;em&gt;y&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7758" LastActivityDate="2017-06-09T03:35:34.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3469" PostTypeId="1" CreationDate="2017-06-09T11:21:34.740" Score="13" ViewCount="1817" Body="&lt;p&gt;I am writing a simple toy game with the intent of training a deep neural network on top of it. The games rules are roughly the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The game has a board made up of hexagonal cells.&lt;/li&gt;&#xA;&lt;li&gt;Both players have the same collection of pieces that they can choose to position freely on the board.&lt;/li&gt;&#xA;&lt;li&gt;Placing different types of pieces award points (or decrease opponent's points) depending on their position and configuration wrt one another.&lt;/li&gt;&#xA;&lt;li&gt;Whoever has more points win.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are additional rules (about turns, number and types of pieces, etc...) but they are not important in the context of this question. I want to devise a deep neural network that can iteratively learn by playing against itself. My questions are about representation of input and output. In particular:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since pattern of pieces matter, I was thinking to have at least some convolutional layers. The board can be of various size but in principle very small (6x10 on my tests, to be expanded by few cells). Does it make sense? What kind of pooling can I use?&lt;/li&gt;&#xA;&lt;li&gt;How to represent both sides? In &lt;a href=&quot;https://arxiv.org/pdf/1412.3409.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; about go, authors use two input matrices, one for white stones and one for black stones. Can it work in this case too? But remember I have different types of pieces, say A, B, C and D. Should I use 2x4 input matrices? It seem very sparse and of little efficiency to me. I fear it will be way too sparse for the convolutional layers to work.&lt;/li&gt;&#xA;&lt;li&gt;I thought that the output could be a distribution of probabilities over the matrix representing board positions, plus a separate array of probabilities indicating what piece to play. However, I also need to represent the ability to &lt;strong&gt;pass&lt;/strong&gt; the turn, which is very important. How can I do it without diluting its significance among other probabilities?&lt;/li&gt;&#xA;&lt;li&gt;And &lt;strong&gt;most importantly&lt;/strong&gt;, do I enforce winning moves only or losing moves too? Enforcing winning moves is easy because I just set desired probabilities to 1. However when losing, what can I do? Set that move probability to 0 and all the others to the same value? Also, does it make sense to enforce moves by the final score difference, even though this would go against the meaning of the outputs, which are roughly probabilities?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, I developed the game engine in node.js thinking to use Synaptic as framework, but I am not sure it can work with convolutional networks (I doubt there's a way to fix the weights associated to local perceptive fields). Any advice on other libraries that are compatible with node? &lt;/p&gt;&#xA;" OwnerUserId="4259" LastActivityDate="2019-08-31T16:02:45.363" Title="Input/output encoding for a neural network to learn a grid-based game" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;game-ai&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3470" PostTypeId="2" ParentId="3465" CreationDate="2017-06-09T17:03:20.730" Score="2" Body="&lt;p&gt;&lt;strong&gt;Technically, you're not really doing classification&lt;/strong&gt; The outputs you have are &lt;em&gt;not&lt;/em&gt; labels, they are real values. So I've got two (possible!) solutions, and you'll have to test them out for yourself:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Solution 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Because each output is actually a numerical value, you can normalize them just like you would normalize all other numerical values. So you choose a maximum value that you would expect for the labels (you can also do this per label), and divide the actual value by that value to normalize it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you don't expect more than 10 of the same amino acids in each peptide sequence, then you divide all amounts by 10:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 becomes 0.0&#xA;1 becomes 0.1&#xA;2 becomes 0.2&#xA;3 becomes 0.3&#xA;etc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Solution 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is more complex, but if you'd give it a try it would surely work. &lt;em&gt;This solution requires recurrent networks&lt;/em&gt;. Just like &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt; networks are good for character-by-character text prediction, they will work for your 'peptide sequence language' as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution also allows an output sequence of &lt;em&gt;any&lt;/em&gt; length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How it works? You have a network with óne output (m/z), and 22 outputs (one-hot encoded amino acids) + 1 more, when this output has the highest value it tells you that the sequence has finished. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You keep on inputting the same value of m/z, until your network tells that the sequence has finished. The network will output different letters every activation based on the previous outputs (the LSTM model has some kind of memory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution is kind of hard to explain, but if you provide me with a small list of peptide sequences (not too long, and not too much), then i'll make a working online example of you.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As promised, I made simple implementation of solution 2. However, I think you should go for solution 1 anyways. I didn't have any &lt;em&gt;real&lt;/em&gt; data, so my model is basically overfitting, but it basically works: &lt;a href=&quot;https://jsfiddle.net/ovpkL2xx/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jsfiddle.net/ovpkL2xx/2/&lt;/a&gt; (might take some time to run)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Made with &lt;a href=&quot;https://github.com/wagenaartje/neataptic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neataptic&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="5344" LastEditDate="2017-06-13T10:48:08.240" LastActivityDate="2017-06-13T10:48:08.240" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3472" PostTypeId="1" CreationDate="2017-06-10T06:36:21.650" Score="5" ViewCount="4102" Body="&lt;p&gt;I am working on &lt;a href=&quot;https://github.com/nfmcclure/tensorflow_cookbook/blob/master/09_Recurrent_Neural_Networks/02_Implementing_RNN_for_Spam_Prediction/02_implementing_rnn.py&quot; rel=&quot;noreferrer&quot;&gt;this code&lt;/a&gt; for spam detection using recurrent neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a closed research field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2. What is the oldest published paper in this field? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quesiton 3. What are the pros and cons of using RNNs for email spam detection over other classification methods?&lt;/p&gt;&#xA;" OwnerUserId="6050" LastActivityDate="2019-01-01T21:23:10.047" Title="Spam Detection using Recurrent Neural Networks" Tags="&lt;classification&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3473" PostTypeId="1" AcceptedAnswerId="3477" CreationDate="2017-06-10T12:23:27.993" Score="2" ViewCount="571" Body="&lt;p&gt;O'Reilly recently published an article about the machine learning paradox. (&lt;a href=&quot;https://www.oreilly.com/ideas/the-machine-learning-paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it says goes basically like this: no machine learning algorithm can be perfect. If it was, it means it is overfitting and so it is not really perfect because it will not perform ok in real world scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched and I couldn't find any other references to this paradox. The closest I got is the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Accuracy_paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accuracy Paradox&lt;/a&gt;, which says that the usefulness of a model is not really well reflected in its accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't sound quite ok to me. For example, a linear model could be perfectly learned, &quot;overfitted&quot; and predicted in the real world. So I suspect it is really about finding the right set of data points from which the results can be inferred. This is, we are trying to approximate from uncertain data, but with the right data we can stop approximating and start calculating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is my line of thinking correct? Or is there really no perfect machine learning?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: In the light of currently received answers, I think my last paragraph (my line of thinking) can be rephrased as: If we have a model simple enough, why can't we overfit the model, knowing that it will behave correctly in non-trained data? This assumes the training data completely represents the real-world data, which would imply a single model that we can train on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that what we conceive as &quot;simple&quot; or &quot;feasible&quot; is arbitrary and only depends on computation power and available data -- aspects with are external to ML models themselves.&lt;/p&gt;&#xA;" OwnerUserId="190" LastEditorUserId="190" LastEditDate="2017-06-11T18:52:32.120" LastActivityDate="2017-06-12T18:09:08.150" Title="Is there such a thing like the machine learning paradox?" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3477" PostTypeId="2" ParentId="3473" CreationDate="2017-06-11T10:23:28.800" Score="5" Body="&lt;p&gt;Basically Machine Learning is used to solve problems for which giving exact solutions is not (practically or theoretically) possible or feasible. If you have a problem that can be solved with exact and proven methods, you don't need Machine Learning. So Machine Learning is rather about finding solutions by approximations with some error margins. In this sense such systems will never perform perfectly and there always will be room for improvements.&#xA;(Many real world problems can not be solved perfectly because they are not simply complicated, but chaotic.)&lt;/p&gt;&#xA;" OwnerUserId="6933" LastActivityDate="2017-06-11T10:23:28.800" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3478" PostTypeId="2" ParentId="3473" CreationDate="2017-06-11T18:11:54.980" Score="4" Body="&lt;p&gt;This answer will focus on the the concepts of model error and increasing the set of points that a model is meant to fit. Firstly, and most importantly, please understand that the general problem that the  OP is grappling with is not new nor confined to &lt;strong&gt;machine learning&lt;/strong&gt;, but rather the other way around, where machine learning is a set of techniques and methods that are being brought to bear on a larger problem of model creation and performance measurement.&lt;/p&gt;&#xA;&lt;h2&gt;All models have error&lt;/h2&gt;&#xA;&lt;p&gt;And to that end, I introduce one of the most important quotes that one should learn and live by if interested in this domain. IMHO, the simplest form of this concept by &lt;a href=&quot;https://en.wikipedia.org/wiki/George_E._P._Box&quot; rel=&quot;nofollow noreferrer&quot;&gt;Professor Box&lt;/a&gt; (the creator of the &lt;strong&gt;Box&lt;/strong&gt; plot!) is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/All_models_are_wrong&quot; rel=&quot;nofollow noreferrer&quot;&gt;Essentially, all models are wrong, but some are useful&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And this is essentially true of &lt;strong&gt;all&lt;/strong&gt; models. They are all necessary simplifications of what we encounter in real world data. All models are built from observation and compilation of experimental data, what we can label as &lt;em&gt;training data&lt;/em&gt;, and &lt;strong&gt;all&lt;/strong&gt; are subject to failure once they are tested against additional data points, especially those that fall outside of the observed set included in the training data. This does not necessarily make them not useful.&lt;/p&gt;&#xA;&lt;p&gt;The original quote from Box is more elaborate in making this point:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Now it would be very remarkable if any system existing in the real world could be exactly represented by any simple model. However, cunningly chosen parsimonious models often do provide remarkably useful approximations. For example, the law PV = RT relating pressure P, volume V and temperature T of an &amp;quot;ideal&amp;quot; gas via a constant R is not exactly true for any real gas, but it frequently provides a useful approximation and furthermore its structure is informative since it springs from a physical view of the behavior of gas molecules.&lt;/p&gt;&#xA;&lt;p&gt;For such a model there is no need to ask the question &amp;quot;Is the model true?&amp;quot;. If &amp;quot;truth&amp;quot; is to be the &amp;quot;whole truth&amp;quot; the answer must be &amp;quot;No&amp;quot;. The only question of interest is &amp;quot;Is the model illuminating and useful?&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is reminding us that the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ideal_gas_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ideal gas equation&lt;/a&gt; is such a &lt;em&gt;theorectical, derived&lt;/em&gt; model and that, indeed, it fails to predict accurately 100% of the &lt;em&gt;real-word&lt;/em&gt; data points that are observed. It is, also, &lt;strong&gt;incredibly&lt;/strong&gt; useful to understand and predict how the forces modeled behave and interact.&lt;/p&gt;&#xA;&lt;p&gt;For the simplest model of linear regression, look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Bivariate_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;bivariate regression&lt;/a&gt;. For two observed variables &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;Y&lt;/code&gt;, you create a &lt;em&gt;regression equation&lt;/em&gt; that models the relationship between X and Y. The model, however, isn't actually representing &lt;em&gt;Y&lt;/em&gt;, but a new variable, let's call it &lt;code&gt;Y'&lt;/code&gt;. Then we arrive at something like:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Y` = a + bX&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where the coefficients &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; are chosen to minimize the discrepancy between &lt;code&gt;Y'&lt;/code&gt; and &lt;code&gt;Y&lt;/code&gt;. If we draw out the variables as vectors in a vector space, we would have two vectors &lt;strong&gt;&lt;code&gt;Y&lt;/code&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;code&gt;Y'&lt;/code&gt;&lt;/strong&gt; that, hopefully, lie very close to each other. But there is a vector &lt;strong&gt;&lt;code&gt;e&lt;/code&gt;&lt;/strong&gt; called the error such that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;e = Y - Y`&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The whole algorithm of linear regression is centered on optimizing &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; to minimize &lt;code&gt;e&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Sampling from the population&lt;/h2&gt;&#xA;&lt;p&gt;So, all models have some error term that we strive to minimize. And for a particular given set of points we may be able to optimize it such that we take the error &lt;strong&gt;perhaps&lt;/strong&gt; even to zero. But for any real world set of points where we have only been able to observe, and hence to model, a sample of the population of observances, the very next observance will potentially cause the error term to increase! This is true even of models that were not derived with &lt;strong&gt;machine-learning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are many, many useful models that break down for parts of the population of observances, but we still find them useful. Look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_law_of_universal_gravitation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newtonian Gravity&lt;/a&gt; which breaks down at certain points of the real-world, but gets most of it right and for which we regularly use for prediction and understanding, in part because it is much simpler to understand and manipulate than something more complex that deals with those other cases, like &lt;a href=&quot;https://en.wikipedia.org/wiki/General_relativity&quot; rel=&quot;nofollow noreferrer&quot;&gt;GR&lt;/a&gt; which itself also fails to deal with some complex phenomena like &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_gravity&quot; rel=&quot;nofollow noreferrer&quot;&gt;quantum gravity&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h1&gt;Over/Under fitting&lt;/h1&gt;&#xA;&lt;p&gt;In addition to minimizing the error &lt;strong&gt;&lt;code&gt;e&lt;/code&gt;&lt;/strong&gt;, there are two other issues with models and performance, &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting and underfitting&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In overfitting, a statistical model describes random error or noise instead of the underlying relationship. Overfitting occurs when a model is excessively complex, such as having too many parameters relative to the number of observations. A model that has been overfit has poor predictive performance, as it overreacts to minor fluctuations in the training data.&lt;/p&gt;&#xA;&lt;p&gt;Underfitting occurs when a statistical model or machine learning algorithm cannot capture the underlying trend of the data. Underfitting would occur, for example, when fitting a linear model to non-linear data. Such a model would have poor predictive performance.&lt;/p&gt;&#xA;&lt;h2&gt;Summary&lt;/h2&gt;&#xA;&lt;p&gt;So, finally, the premise:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;no machine learning algorithm can be perfect. If it was, it means it is overfitting and so it is not really perfect because it will not perform ok in real world scenarios.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Misses the point that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Essentially, all models are wrong, but some are useful.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7822" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-06-12T18:09:08.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3479" PostTypeId="2" ParentId="3473" CreationDate="2017-06-11T20:11:10.657" Score="1" Body="&lt;p&gt;It's true that no general learning algorithm, &lt;strong&gt;machine or human&lt;/strong&gt;, can be perfect. It isn't just a problem with machine learning; it's true of human learning. Not of the type of learning we typically think of as learning, such as mathematics, but things like walking, seeing, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We learn to walk in standard situations, and can do it very well, but the model of walking that most of us have don't cope well with walking in high heels. Now, we can learn that, but that doesn't help us walking on tightropes, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, optical illusions work because they represent situations we don't encounter in normal life, and so have never learned to deal with. Instead our visual system treats them as apparently similar but familiar situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of situations that we may have to cope with but have never learned to do so. How well we do so depends on our mental model of the activity that we have built up, and how well our &quot;learning algorithm&quot; dealt with the training data we had.&lt;/p&gt;&#xA;" OwnerUserId="7491" LastActivityDate="2017-06-11T20:11:10.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3482" PostTypeId="1" AcceptedAnswerId="3487" CreationDate="2017-06-12T20:48:17.710" Score="5" ViewCount="1426" Body="&lt;p&gt;I've been working with neural networks and artificial intelligence for a while. What I'm trying to do right now is, from a genotype I have (a sum of sensors, neurons and actuators) draw how the neural network is (with recurrent/recursive connections being showed nicely, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have done now in javascript is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GWl9G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GWl9G.png&quot; alt=&quot;I know, its ugly&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have achieved this using &lt;a href=&quot;http://sigmajs.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SigmaJs&lt;/a&gt;, a Javascript drawing library, but I think it's still ugly, and what I'm looking for is a node drawing library that can achieve recursive connections in a nice way (right now, I'm drawing them with a red color as you can see on the image).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have examined a lot of GitHub repositories and websites that can be helpful but aren't worth it since they aren't that nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone got an idea of what can I use, in javascript?  If not, in any other language, how can I achieve what I want?&lt;/p&gt;&#xA;" OwnerUserId="7846" LastEditorUserId="2444" LastEditDate="2020-02-23T23:02:14.640" LastActivityDate="2020-02-23T23:03:44.780" Title="Is there any way to draw a neural network's connections in a nice way?" Tags="&lt;neural-networks&gt;&lt;resource-request&gt;&lt;javascript&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2020-05-22T18:15:24.170" ContentLicense="CC BY-SA 4.0" />
  <row Id="3483" PostTypeId="1" CreationDate="2017-06-12T22:00:31.630" Score="3" ViewCount="119" Body="&lt;p&gt;Is it possible to train a neural network to learn something via video footage (which is essentially a sequence of images)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if I have a video teaching me how to draw an animal from scratch, can I then use this video to teach the computer to draw the animal in the same way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many machine learning and image processing techniques (such as RNNs) that can be applied to sequences of images or videos. So, I guess the difficult part becomes mapping the activity to an action like moving a pen or something.&lt;/p&gt;&#xA;" OwnerUserId="7816" LastEditorUserId="2444" LastEditDate="2020-03-15T00:22:37.977" LastActivityDate="2020-03-15T00:22:37.977" Title="Is it possible to train a neural network to learn something via video footage?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;image-processing&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3484" PostTypeId="2" ParentId="3483" CreationDate="2017-06-12T23:14:43.410" Score="1" Body="&lt;p&gt;Short answer: No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long answer: A neural network is a function that maps input data (e.g. a picture) to output data (e.g. probability that the picture contains a dog). What you propose does not seem like the sort of task this tool is suited for, though I'm not willing to claim that it can't be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in computer-generated artwork, I encourage you to learn about &lt;a href=&quot;http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=Oex0eWoU7AQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;style transfer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7849" LastActivityDate="2017-06-12T23:14:43.410" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3486" PostTypeId="2" ParentId="3483" CreationDate="2017-06-13T07:55:37.087" Score="1" Body="&lt;p&gt;Just like the other answer - it can't really be done like that. However, you might want to take a look at &lt;a href=&quot;https://github.com/google/deepdream&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Deepdream&lt;/a&gt;, which actually enhances images to look more like what they detected. E.g., if a cat is detected, it will make the picture more cat-like.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastActivityDate="2017-06-13T07:55:37.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3487" PostTypeId="2" ParentId="3482" CreationDate="2017-06-13T08:18:53.317" Score="6" Body="&lt;p&gt;I'm the developer of &lt;a href=&quot;https://github.com/wagenaartje/neataptic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neataptic&lt;/a&gt;, a Javascript neural network library. I don't know if it is exactly what you're looking for, but it has a built-in graph creator using &lt;a href=&quot;https://d3js.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;D3&lt;/a&gt; and &lt;a href=&quot;http://marvl.infotech.monash.edu/webcola/&quot; rel=&quot;nofollow noreferrer&quot;&gt;webcola&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, each connection gets an arrow. It also supports self-connections and gates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some examples of images:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://camo.githubusercontent.com/e5beed80c716d77350100fdea0d359db10266c43/68747470733a2f2f692e6779617a6f2e636f6d2f64323234373064393031656439616662373932653130363135316564376539352e706e67&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://camo.githubusercontent.com/d25e2ac003265fc5166924ecd8188cfd39785910/68747470733a2f2f692e6779617a6f2e636f6d2f32376538303033646636306462626432316532343061353366386563303933612e706e67&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Play around yourself &lt;a href=&quot;https://jsfiddle.net/ch8s9d3e/29/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2020-02-23T23:03:44.780" LastActivityDate="2020-02-23T23:03:44.780" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3488" PostTypeId="1" AcceptedAnswerId="7130" CreationDate="2017-06-13T10:50:56.343" Score="20" ViewCount="24691" Body="&lt;p&gt;I'm struggling to understand the GAN loss function as provided in &lt;a href=&quot;https://danieltakeshi.github.io/2017/03/05/understanding-generative-adversarial-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Generative Adversarial Networks&lt;/a&gt; (a blog post written by Daniel Seita).&lt;/p&gt;&#xA;&lt;p&gt;In the standard cross-entropy loss, we have an output that has been run through a sigmoid function and a resulting binary classification.&lt;/p&gt;&#xA;&lt;p&gt;Sieta states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Thus, For [each] data point &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and its label, we get the following loss function ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H((x_1, y_1), D) = -y_1 \log D(x_1) - (1 - y_1) \log (1 - D(x_1))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is just the log of the expectation, which makes sense. However, according to this formulation of the GAN loss, how can we process the data from both the true distribution and the generator in the same iteration?&lt;/p&gt;&#xA;" OwnerUserId="7858" LastEditorUserId="2444" LastEditDate="2020-11-22T12:42:47.963" LastActivityDate="2020-11-22T12:42:47.963" Title="How can we process the data from both the true distribution and the generator?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3489" PostTypeId="2" ParentId="3398" CreationDate="2017-06-13T11:50:24.840" Score="2" Body="&lt;p&gt;As an example for Caffee: Facebook parallelized the training of a &#xA;&lt;a href=&quot;https://research.fb.com/publications/imagenet1kin1h/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Res-Net CNN over 256 GPUs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7859" LastActivityDate="2017-06-13T11:50:24.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3490" PostTypeId="1" CreationDate="2017-06-13T14:42:56.007" Score="4" ViewCount="275" Body="&lt;p&gt;While conducting research, I recently stumbled upon the deep learning and natural language processing concepts. In &lt;a href=&quot;https://ai.stackexchange.com/questions/1970/how-would-an-ai-learn-language&quot;&gt;this&lt;/a&gt; question they say that the ‘grammar induction’ is a ‘supervised learning’ mode. So I was wondering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let’s say that there’s a way-more-than-human intelligent alien probe orbiting our planet. It can receive, decode and analyze all the broadcasting signals leaving the Earth. For all we know at this moment, how could it learn the basics of a language with nothing else but our broadcasting signals, so without the help from a ‘supervisor’? How would an artificial intelligence human engineer (theoretically?) face the problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m interested on the more “technical” side of the issue. &lt;/p&gt;&#xA;" OwnerUserId="7785" LastEditorUserId="4302" LastEditDate="2018-10-08T12:27:44.953" LastActivityDate="2018-10-08T12:27:44.953" Title="Unsupervised alien natural language learning" Tags="&lt;natural-language-processing&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3491" PostTypeId="2" ParentId="3483" CreationDate="2017-06-13T23:01:46.040" Score="0" Body="&lt;p&gt;About the screenshot you mentioned in comments: don't forget that NN build the mappings from learning. With only one screenshot, you'll only be able to construct a NN that outputs the correct 3D for THAT screenshot. With many screenshots, you may come closer to what you want (AFAI understand your question).&lt;br&gt;&#xA;But remember that without 2 eyes, you have problems with 3D objects. Doesn't mean it's impossible.&lt;br&gt;&#xA;May I suggest you to have a look at convolutional neural networks also &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Convolutional_neural_network&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7534" LastActivityDate="2017-06-13T23:01:46.040" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3492" PostTypeId="2" ParentId="3426" CreationDate="2017-06-13T23:16:20.917" Score="1" Body="&lt;p&gt;It depends on the country. In France, for example, you've got to have the agreement of the person. Doesn't matter whether he is the president, a singer or... me :)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Doesn't matter what for. Your image belongs to you. Otherwise he (me) can sue you for that and seek redress. Faces of persons are blurred in Google Map Street View. Here is &lt;a href=&quot;https://www.service-public.fr/particuliers/vosdroits/F32103&quot; rel=&quot;nofollow noreferrer&quot;&gt;a link&lt;/a&gt;, in french.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About re-publication, you'll also have to have the agreement of the copyright holder.&lt;/p&gt;&#xA;" OwnerUserId="7534" LastEditorUserId="7550" LastEditDate="2017-06-22T16:01:42.633" LastActivityDate="2017-06-22T16:01:42.633" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3494" PostTypeId="1" AcceptedAnswerId="3503" CreationDate="2017-06-15T09:13:48.660" Score="59" ViewCount="38310" Body="&lt;p&gt;First of all, I'm a beginner studying AI and this is not an opinion-oriented question or one to compare programming languages. I'm not implying that Python is the best language. But the fact is that most of the famous AI frameworks have primary support for Python. They can even be multilanguage supported, for example, TensorFlow that supports Python, C++, or CNTK from Microsoft that support C# and C++, but the most used is Python (I mean more documentation, examples, bigger community, support, etc). Even if you choose C# (developed by Microsoft and my primary programming language) you must have the Python environment set up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read in other forums that Python is preferred for AI because the code is simplified and cleaner, good for fast prototyping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was watching a movie with AI thematics (Ex_Machina). In some scenes, the main character hacks the interface of the house automation. Guess which language was on the scene? Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the big deal with Python? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is there a growing association between Python and AI?&lt;/p&gt;&#xA;" OwnerUserId="7268" LastEditorUserId="36459" LastEditDate="2020-05-09T22:22:28.020" LastActivityDate="2021-01-29T02:31:38.807" Title="Why is Python such a popular language in the AI field?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;programming-languages&gt;&lt;popular-culture&gt;" AnswerCount="10" CommentCount="1" FavoriteCount="27" ContentLicense="CC BY-SA 4.0" />
  <row Id="3495" PostTypeId="2" ParentId="3494" CreationDate="2017-06-15T12:04:53.177" Score="7" Body="&lt;p&gt;Python has a standard library in development, and a few for AI. It has an intuitive syntax, basic control flow, and data structures. It also supports interpretive run-time, without standard compiler languages. This makes Python especially useful for prototyping algorithms for AI.&lt;/p&gt;&#xA;" OwnerUserId="7897" LastActivityDate="2017-06-15T12:04:53.177" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3496" PostTypeId="2" ParentId="3494" CreationDate="2017-06-15T13:27:43.930" Score="31" Body="&lt;p&gt;Practically all of the most popular and widely used deep-learning frameworks are implemented in Python on the surface and C/C++ under the hood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main reason is that Python is widely used in scientific and research communities, because it's easy to experiment with new ideas and code prototypes quickly in a language with minimal syntax like Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover there may be another reason. As I can see, most of the over-hyped online courses on AI are pushing Python because it is easy for newbie programmers. AI is the new marketing hot word to sell programming courses.&#xA;( Mentioning AI can sell programming courses to kids who want to build HAL 3000, but can not even write a Hello World or drop a trend-line onto an Excel graph. :)&lt;/p&gt;&#xA;" OwnerUserId="6933" LastEditorUserId="6933" LastEditDate="2017-06-15T15:49:47.377" LastActivityDate="2017-06-15T15:49:47.377" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3497" PostTypeId="1" AcceptedAnswerId="3501" CreationDate="2017-06-15T14:06:25.873" Score="1" ViewCount="407" Body="&lt;p&gt;I am a machine learning newbie. I am trying to understand the back-propagation algorithm. I have a training dataset of 60 instances/records.&lt;/p&gt;&#xA;&lt;p&gt;What is the correct order of the process? This one?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Forward pass of the first instance. Calculate the error.&lt;/li&gt;&#xA;&lt;li&gt;Weight update using backpropagation.&lt;/li&gt;&#xA;&lt;li&gt;Forward pass of the second instance. Calculate the error.&lt;/li&gt;&#xA;&lt;li&gt;Weight update using backpropagation. And so on...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Forward pass of all instances one by one. Keep track of the error as a vector.&lt;/li&gt;&#xA;&lt;li&gt;Weight update using backpropagation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This video &lt;a href=&quot;https://www.youtube.com/watch?v=OwdjRYUPngE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=OwdjRYUPngE&lt;/a&gt; is similar to the second process. Is it correct?&lt;/p&gt;&#xA;" OwnerUserId="7899" LastEditorUserId="2444" LastEditDate="2020-12-10T18:08:38.160" LastActivityDate="2020-12-10T18:29:45.753" Title="What is the order of execution of steps in back-propagation algorithm in a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;stochastic-gradient-descent&gt;&lt;batch-gradient-descent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3499" PostTypeId="1" AcceptedAnswerId="3500" CreationDate="2017-06-15T14:47:42.427" Score="1" ViewCount="74" Body="&lt;p&gt;I know nowadays agencies are using GPUs in order to accelerate AI, but how fast should be it to be efficient, I mean I know that depends of how large and complex the assignment is but what would be a way to measure its efficiency and what kind of technology(amount of GPUS,RAM,STORAGE) and techniques need to be used in order to get enough efficency?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts from experts would be appreciated&lt;/p&gt;&#xA;" OwnerUserId="7137" LastActivityDate="2017-06-15T19:35:09.600" Title="how fast does need to be an AI agent to be efficient?" Tags="&lt;efficiency&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3500" PostTypeId="2" ParentId="3499" CreationDate="2017-06-15T15:35:31.977" Score="1" Body="&lt;p&gt;My understanding of efficiency in this context is in regards to optimization of algorithms as opposed to hardware speed, which is more of a &quot;brute force&quot; component.  GPUs may be more &lt;em&gt;energy&lt;/em&gt; efficient, but this is distinct from &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear optimization&lt;/a&gt; of algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of how much processor speed you need to tackle a given problem, that's in the realm of &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational complexity theory&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Analysis_of_algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;analysis of algorithms&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Amount of GPUs, RAM and storage needed to tackle a given problem are purely a function of the complexity of the problem and the efficiency of the algorithms. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-06-15T19:35:09.600" LastActivityDate="2017-06-15T19:35:09.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3501" PostTypeId="2" ParentId="3497" CreationDate="2017-06-15T19:00:33.540" Score="0" Body="&lt;p&gt;Both are feasible.&lt;/p&gt;&#xA;&lt;p&gt;A generalization is to apply the modification of the weights after the presentation of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; examples. If &lt;span class=&quot;math-container&quot;&gt;$n=1$&lt;/span&gt;, this is &lt;strong&gt;online&lt;/strong&gt; training. If &lt;span class=&quot;math-container&quot;&gt;$n=60$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$60$&lt;/span&gt;, in your case, is the size of the dataset), this is &lt;strong&gt;batch&lt;/strong&gt; training. In other cases, this is &lt;strong&gt;mini-batch&lt;/strong&gt; training.&lt;/p&gt;&#xA;&lt;p&gt;The main difference is not the computation complexity of the algorithm, but the theoretical speed of convergence to an &amp;quot;optimal&amp;quot; set of weights.&lt;/p&gt;&#xA;&lt;p&gt;It is now generally admitted that &lt;em&gt;online training&lt;/em&gt; is faster. Online training performs a stochastic approximation of the gradient descent method. The true gradient of the cost function over the whole training set is approximated by a gradient at &lt;strong&gt;each&lt;/strong&gt; presentation of a training example.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, with &lt;em&gt;batch learning&lt;/em&gt; the weights are modified by an &lt;strong&gt;average&lt;/strong&gt; of the gradient over the whole training set and averaging destroys information.&lt;/p&gt;&#xA;&lt;p&gt;While with stochastic (or mini-batch learning), each example has his word to say. One after the other.&lt;/p&gt;&#xA;&lt;p&gt;This has also been discussed &lt;a href=&quot;https://stats.stackexchange.com/q/49528/82135&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7534" LastEditorUserId="2444" LastEditDate="2020-12-10T18:29:45.753" LastActivityDate="2020-12-10T18:29:45.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3502" PostTypeId="1" CreationDate="2017-06-16T04:10:16.480" Score="16" ViewCount="7965" Body="&lt;p&gt;Is there a way to teach reinforcement learning in applications other than games? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only examples I can find on the Internet are of game agents. I understand that VNC's control the input to the games via the reinforcement network. Is it possible to set this up with say a CAD software?&lt;/p&gt;&#xA;" OwnerUserId="7816" LastEditorUserId="2444" LastEditDate="2018-11-19T07:34:51.510" LastActivityDate="2020-11-01T14:50:45.430" Title="Are there any applications of reinforcement learning other than games?" Tags="&lt;reinforcement-learning&gt;&lt;applications&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3503" PostTypeId="2" ParentId="3494" CreationDate="2017-06-16T05:56:16.907" Score="44" Body="&lt;p&gt;Python comes with a huge amount of inbuilt libraries. Many of the libraries are for Artificial Intelligence and Machine Learning. Some of the libraries are TensorFlow (which is a high-level neural network library), scikit-learn (for data mining, data analysis and machine learning), pylearn2 (more flexible than scikit-learn), etc. The list keeps going and never ends. You can find some libraries &lt;a href=&quot;https://wiki.python.org/moin/PythonForArtificialIntelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Python has an easy implementation for OpenCV. What makes Python favorite for everyone is its powerful and easy implementation.&lt;/p&gt;&#xA;&lt;p&gt;For other languages, students and researchers need to get to know the language before getting into ML or AI with that language. &lt;em&gt;This is not the case with Python&lt;/em&gt;. Even a programmer with very basic knowledge can easily handle Python. Apart from that, the time someone spends on writing and debugging code in Python is way less when compared to C, C++ or Java. This is exactly what the students of AI and ML want. &lt;em&gt;They don't want to spend time on debugging the code for syntax errors, they want to spend more time on their algorithms and heuristics related to AI and ML&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Not just the libraries but their tutorials, handling of interfaces are easily available online&lt;/em&gt;. People build their own libraries and upload them on GitHub or elsewhere to be used by others.&lt;/p&gt;&#xA;&lt;p&gt;All these features make Python suitable for them.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-29T02:31:38.807" LastActivityDate="2021-01-29T02:31:38.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3505" PostTypeId="1" CreationDate="2017-06-16T11:42:52.950" Score="3" ViewCount="693" Body="&lt;p&gt;I want to train my neural network by evolution, that is I want to recombine the weights of the best performing neural networks in each evolution cycle or generation. My initial instinct was to represent weights as they are, which is variable of type double, and either&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;swap the weights between the two parent network&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;generate a random number between the two weights&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, what I need is to represent the weights as a binary string, and then, as usual, carry out the crossover on the string.&lt;/p&gt;&#xA;&lt;p&gt;So, how can I take my array of weights (of floating-point values or doubles) and convert that into binary string? Should the chromosome contain an array of strings where each string represents a single weight?&lt;/p&gt;&#xA;" OwnerUserId="7910" LastEditorUserId="2444" LastEditDate="2020-12-13T20:49:11.910" LastActivityDate="2020-12-14T22:31:10.307" Title="How to represent the weights of a neural network as binary strings for a genetic algorithm?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;&lt;chromosomes&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3506" PostTypeId="1" CreationDate="2017-06-16T13:57:29.430" Score="4" ViewCount="164" Body="&lt;p&gt;Let's say we have the basic scenario where two AGIs of about the same intelligence (but not same origins/code/model) have to communicate as efficiently as possible to achieve a common goal. Now we could have 2 starting points for that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Either all they have is a common communication bus (e.g. sound, light, radio, etc.) and instruments (e.g. transceivers) to support it, and they have to figure out the rest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or they are some kind of advanced chatbots, but since the human language is lacking a lot to be used as a highly efficient protocol, they will have to communicate with what they have, to build a proper one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to somehow induce them to communicate, and try to figure out what each other &quot;say&quot;? How could this be done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a more abstract question is how could this protocol &quot;look&quot; like?&lt;/p&gt;&#xA;" OwnerUserId="6899" LastEditorUserId="2444" LastEditDate="2019-06-26T11:54:22.340" LastActivityDate="2019-06-26T11:54:22.340" Title="How could AGI build its own communication protocol?" Tags="&lt;agi&gt;&lt;superintelligence&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3507" PostTypeId="2" ParentId="3505" CreationDate="2017-06-16T16:22:35.843" Score="3" Body="&lt;p&gt;I would first say consider the advice of Thomas W in &lt;a href=&quot;https://ai.stackexchange.com/questions/3505/how-to-represent-the-weights-of-a-neural-network-as-binary-strings-for-a-genetic/3507#comment4272_3505&quot;&gt;the comment above&lt;/a&gt; and think about whether you really need to discretize your variables. I'd also question the wisdom of training a reasonably sized network with a GA instead of a dedicated neural net training algorithm that's very likely to exhibit much better performance. However, assuming you really do need to do this, the basic approach is generally to do the following.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Decide on your range and precision. For the sake of simplicity, I'm going to assume you use say &lt;span class=&quot;math-container&quot;&gt;$L=16$&lt;/span&gt; bits to represent every weight. You can have some variables encoded at a higher precision than others -- you're in charge of writing the encode/decode functions, so it's entirely up to you, but here, I'm assuming you don't. Your encoding will now be a vector of length &lt;span class=&quot;math-container&quot;&gt;$16n$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of weights in the network. You can use a string if you like, but it's probably a net loss of efficiency, as mostly what you'll be doing is converting those bits to integers so that you can do arithmetic necessary to decode the encoded weights, so I'd just make it a vector of &lt;span class=&quot;math-container&quot;&gt;$16n$&lt;/span&gt; integers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For the range, you just need the minimum and maximum values you're going to allow. Again, for simplicity, I'm going to say the values range between &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, denoted as &lt;span class=&quot;math-container&quot;&gt;$R_{\min}=-10$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{\max}=10$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Your GA then just manipulates the &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;s. To evaluate fitness, you have to decode each of those &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt;-bit strings into a double between &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;. With &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; bits, you have &lt;span class=&quot;math-container&quot;&gt;$2^{16}=16536$&lt;/span&gt; possible values. The total range of output values is &lt;span class=&quot;math-container&quot;&gt;$10-(-10)=20$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$20/16536=0.0012$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$0000000000000000$&lt;/span&gt; be the smallest possible string, and map that onto the value &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt;. Now, every increment to that binary string just adds &lt;span class=&quot;math-container&quot;&gt;$0.0012$&lt;/span&gt; to the output value.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Putting all this together, you have &lt;span class=&quot;math-container&quot;&gt;$w_i = (R_{\max} - R_{\min})/2^L + R_{\min}$&lt;/span&gt;. Do that for each block of &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; bits giving you each weight in the network. Run your network on your training data and use whatever accuracy metric as the fitness function for the GA.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;For the GA itself now, you don't really have to do anything special. The GA can just see an arbitrary string of &lt;span class=&quot;math-container&quot;&gt;$16n$&lt;/span&gt; bits. You can do crossover and mutation on just the apparently meaningless string of bits.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Your question was asking about going from a vector of weights to a vector of strings, which is the opposite conversion from that described above. Generally, for a GA, you don't need a phenotype-&amp;gt;genotype conversion. You just generate genotypes using genetic operators and use the genotype-&amp;gt;phenotype conversion to evaluate candidate solutions. However, there are some reasons why you might want to go the other way. For instance, you might want to seed the population with a known set of weights, and so you'd need the ability to convert those weights to binary.&lt;/p&gt;&#xA;&lt;p&gt;Basically, in that case, just invert that conversion function. The idea is to convert the floating-point weight between &lt;span class=&quot;math-container&quot;&gt;$R_{\min}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{\max}$&lt;/span&gt; into the integer that's closest to the same distance into the range when mapped to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$2^{L}-1$&lt;/span&gt;. For example, if the weight is &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt;, then it's the smallest possible value. It maps onto the smallest possible value between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$2^{L}-1$&lt;/span&gt;, namely &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Similarly, a weight of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; maps onto &lt;span class=&quot;math-container&quot;&gt;$(2^{L}-1)/2$&lt;/span&gt;, because it's halfway between the min and max of both ranges. If you play around with it a bit, you should see that the expression here is given by &lt;span class=&quot;math-container&quot;&gt;$\big \lfloor \frac{w_i - R_\min}{R_\max - R_\min} 2^L \big \rfloor$&lt;/span&gt;. That gives me an integer in the range &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$2^L$&lt;/span&gt;. From there, just do a standard binary encoding (or gray code, whatever) to get into a bit string you can manipulate.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastEditorUserId="3365" LastEditDate="2020-12-14T22:31:10.307" LastActivityDate="2020-12-14T22:31:10.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3508" PostTypeId="2" ParentId="3506" CreationDate="2017-06-16T19:27:00.747" Score="3" Body="&lt;p&gt;This is a purely theoretical question, currently in the realm of philosophy and speculative fiction.  Nevertheless, it is an interesting question, and may be instructive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use the standard definition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt; as &lt;a href=&quot;https://en.wikipedia.org/wiki/Automata_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;automata&lt;/a&gt; with human level intelligence, then they could certainly devise their own communications protocols, just as humans have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) These automata are AGI so they are creative and resilient, just like humans, and 'where there is a will there is [sometimes] a way.&quot; Absent robotic capacity they wouldn't be able to build anything physical and would have to rely on existing communications infrastructures. If they had access to 3D printing and versatile robots, they could probably build something new, but this would be infeasible for anything that requires extensive capital outlay, unless the automata first acquired a major communications infrastructure firm or two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) It's always interesting (and usually quite entertaining) to observe chatbots conversing with each other, but if they were AGIs, NLP is just one of many functions, and I doubt they would bother conversing with each other in human language, since all data is ultimately reduced to a string. If they were smart enough to be deemed AGI, they would certainly communicate in-species with the most efficient protocol available, and probably optimize it further, if possible, or create a unique protocol for purposes of exclusivity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they are truly AGIs, you wouldn't ostensibly have to induce them to communicate, because they'd be smart enough to understand the benefits of communication and cooperation, and would likely seek to form coalitions as a natural survival function. (&lt;a href=&quot;https://en.wikipedia.org/wiki/Cooperative_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game Theory provides a mathematical basis for this&lt;/a&gt;.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-agent_system#Self-organisation_and_self-steering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-agents systems can self organize&lt;/a&gt;, even where the intelligence of the given agents is low, and in your scenario, the automata are &lt;em&gt;smart&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-06-16T19:27:00.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3510" PostTypeId="2" ParentId="3488" CreationDate="2017-06-17T14:37:31.670" Score="1" Body="&lt;p&gt;You can treat a combination of &lt;code&gt;z&lt;/code&gt; input and &lt;code&gt;x&lt;/code&gt; input as a single sample, and you evaluate how well the discriminator performed the classification of each of these. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why the post later on separates a single &lt;code&gt;y&lt;/code&gt; into &lt;code&gt;E(p~data)&lt;/code&gt; and &lt;code&gt;E(z)&lt;/code&gt; -- basically, you have different expectations (&lt;code&gt;y&lt;/code&gt;s) for each of the discriminator inputs and you need to measure both at the same time to evaluate how well the discriminator is performing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why the loss function is conceived as a combination of both the positive classification of the real input and the negative classification of the negative input.&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2017-06-17T14:37:31.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3511" PostTypeId="2" ParentId="3482" CreationDate="2017-06-17T14:41:59.657" Score="2" Body="&lt;p&gt;You might have come across the &lt;a href=&quot;http://playground.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow Playground&lt;/a&gt; which has a wonderful visualization of the network connections and the neuron weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their code is available in GitHub (&lt;a href=&quot;https://github.com/tensorflow/playground&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), and the code seems fairly simple. It is coded in typescript but that can be easily transpiled to pure JavaScript.&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2017-06-17T14:41:59.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3512" PostTypeId="2" ParentId="3490" CreationDate="2017-06-17T15:05:32.867" Score="1" Body="&lt;p&gt;I think I understand where your confusion comes from, and the reason for your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we talk about a supervised problem, it means that there is some feedback of how well the machine did at a certain task. Of course, we always need this kind of feedback in some way or another (otherwise, the machine would never learn, because it would have no incentive to change anything that it does), but the feedback can be classified into &quot;supervised&quot; if we have a clear way of telling when something was right/wrong, or &quot;unsupervised&quot; if we just roll out with the results and try to make it better each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, an AI performing translation like in grammar induction in the other question will take as input both the phrase and the syntactic tree. After performing the prediction of what the syntactic tree for the phrase should be, it can be compared against the real syntactic tree given as input and based on the accuracy of the prediction, the weights (think of it as &quot;knobs&quot; that adjust the result) can be tweaked a little bit to give better predictions next time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of learning is considered supervised, not because of the existence of a supervisor, but rather because there is labeled data that we can use to test predictions and better them next time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unsupervised problems don't have this kind of labeled data, and just work with what they have. They cannot tell what's right and what's wrong in terms of predictions, they only have raw data and try to make sense out of it by extracting correlations or common properties.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For the case of a language, I will go on a limb here but I'll say that this is mostly likely a task for supervised algorithms. Unsupervised techniques could properly analyze language and determine that they actually have a particular structure, but this is also true for lots of other sources of data that are not language. An alien race, knowing nothing about humanity, could end up deducting that cars honking are some sort of language too, albeit much simpler than human-produced noises.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the natural flexibility that languages have make it extra complex, because they don't follow a correct structure completely, and to make it worse, they are constantly changing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As leitasat mentioned in their comment, if even human scientists (which we certainly deem intelligent) could not decipher ancient Egyptian on their own without any context, it's very unlikely that a machine would do it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Finally, notice that translator machines or inference machines don't actually derive meaning in a way that's meaningful. Without going much in much detail, you should think of them as &quot;correlation detection machines&quot; -- so a machine might notice a high enough correlation between how &quot;Hola&quot; is used in Spanish as to how &quot;Hi&quot; is used in English, and by giving you that information, we call it a translator. However, the internal learnt structure is actually just a probability distribution for an output given a set of inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not to say that any of this is not useful -- it definitely is -- but with something as separate as an alien race of super-human intelligence and without any way to derive meaning from just a bunch of sounds, it's very unlikely that a machine would prove useful in such correlations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And still, I cannot find any hard stop to your idea -- if these aliens have a language and they express the same concepts that human express, then there is correlation. There just needs to be a way to find it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for the wall of text and the huge rambling, I hope I provided some context on the points you were looking clarification for.&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2017-06-17T15:05:32.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3513" PostTypeId="2" ParentId="3506" CreationDate="2017-06-17T22:43:45.310" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;I believe that some work by Randall Beer in the 1990s demonstrated that even simple agents could learn a shared communication protocol:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=225774&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational and dynamical languages for autonomous agents&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Viewed in general terms, a sentence uttered by an agent is just another 'feature' of the environment, from the perspective of an observing agent. Learning to extract and operate upon a subset of features is what modern machine learning is mostly concerned with.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;While Braitenberg's book 'Vehicles'&#xA;doesn't deal specifically with language acquisition, it does nicely illustrate some general principles for self-organisation of more complex recognisers and behaviours.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-06-17T22:50:57.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3515" PostTypeId="1" AcceptedAnswerId="3522" CreationDate="2017-06-19T15:17:11.943" Score="3" ViewCount="275" Body="&lt;p&gt;I am starting to learn machine learning (ML). I'm thinking about starting to take part in some projects in ML. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &lt;a href=&quot;https://www.kaggle.com/competitions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle&lt;/a&gt; is a good place to find projects in ML to work on?&lt;/p&gt;&#xA;" OwnerUserId="7948" LastEditorUserId="2444" LastEditDate="2020-06-16T19:13:06.817" LastActivityDate="2020-06-16T19:13:06.817" Title="Is Kaggle a good place to find projects in ML to work on?" Tags="&lt;machine-learning&gt;&lt;kaggle&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3516" PostTypeId="1" CreationDate="2017-06-19T20:40:49.493" Score="2" ViewCount="70" Body="&lt;p&gt;I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. GBNMIRN, ILCEQZG, or FPSRABBRF).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to create a model that can take in a new 2048-long vector of 1s and 0s and predict what the output sequence will look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked at some github repositories like: &lt;a href=&quot;https://github.com/llSourcell/seq2seq_model_live/blob/master/2-seq2seq-advanced.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/llSourcell/seq2seq_model_live/blob/master/2-seq2seq-advanced.ipynb&lt;/a&gt; &lt;a href=&quot;https://github.com/hans/ipython-notebooks/blob/master/tf/TF%20tutorial.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hans/ipython-notebooks/blob/master/tf/TF%20tutorial.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I'm not sure how to implement it with my problem. Are there any projects that have done something similar to this/how could I implement this with the seq2seq models currently out there?&lt;/p&gt;&#xA;" OwnerUserId="7773" LastActivityDate="2017-06-19T20:40:49.493" Title="seq2seq vector to letters model" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3517" PostTypeId="2" ParentId="3345" CreationDate="2017-06-19T21:07:40.203" Score="11" Body="&lt;p&gt;Great question!  NN is very promising for this type of problem: &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe Chess&lt;/a&gt;.  Lai's accomplishment &lt;a href=&quot;https://www.technologyreview.com/s/541276/deep-learning-machine-teaches-itself-chess-in-72-hours-plays-at-international-master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;was considered to be a pretty big deal&lt;/a&gt;, but unfortunately came just a few months before AlphaGo took the spotlight. &lt;em&gt;(It all turned out well, in that Lai was subsequently hired by DeepMind, &lt;a href=&quot;https://motherboard.vice.com/en_us/article/the-chess-engine-that-died-so-alphago-could-live-giraffe-matthew-lai&quot; rel=&quot;nofollow noreferrer&quot;&gt;although not so well for the Giraffe engine&lt;/a&gt;;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found Lai's approach to be quite helpful, and it is backed by solid results.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You may want to use &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sequential_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequential&lt;/a&gt;&quot; as opposed to &quot;round based&quot; since sequential is the preferred term in &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game Theory&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial Game Theory&lt;/a&gt;, and these are the fields that apply mathematical analysis to games. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The games you list are further termed &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Abstract_strategy_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;abstract&lt;/a&gt;&quot; to distinguish them from modern strategy boardgames, or games in general, which utilize a strong theme and are generally less compact than abstract games in terms of mechanics and elements. This carries the caveat that abstract games are not restricted to sequential games or boardgames, or even games specifically, as in the case of puzzles like Sudoku.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formal name for this group of games is generally &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Partisan_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;partisan&lt;/a&gt;, sequential, &lt;a href=&quot;http://library.msri.org/books/Book29/contents.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deterministic&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information#Examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;perfect information&lt;/a&gt;&quot; with the further categorization of Tic-Tac-Toe as &quot;trivial&quot; (solved and easily solvable) and non-trivial (intractable and unsolved) for games like Chess and Go. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-19T19:14:01.967" LastActivityDate="2019-04-19T19:14:01.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3518" PostTypeId="1" AcceptedAnswerId="3521" CreationDate="2017-06-20T01:14:13.320" Score="5" ViewCount="373" Body="&lt;p&gt;I am developing AI in the form of NEAT, and it has passed certain tasks like the XOR problem outlined in the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT Research Paper&lt;/a&gt;. In the XOR Problem, the fitness of a network was determined by an existing function (XOR in this case). It also passed another tests. One I developed was to determine the sine at a certain point X in radians. It also worked, but yet again, its fitness was determined by an existing function (sin (x)). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've recently been working on training it to play Tic Tac Toe. I decided that to determine its fitness, it would play against a &quot;dumb&quot; AI, placing O's in random locations on the grid, and gaining fitness based on whether or not it placed X's in a valid location (losing fitness if it placed an X on top of another X or an O), and gaining a lot of fitness if it won against the &quot;dumb&quot; AI. This would work, but when a network got really lucky and the &quot;dumb&quot; AI placed O's in impractical locations, the network would win and gain a lot of fitness, making it very difficult for another network to beat that fitness. Therefore, the learning process did not work and I was not able to generate a Tic Tac Toe network that actually worked well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not want the GA to learn based off an &quot;intelligent&quot; tic tac toe AI because the whole point of me training this GA is so that I do not have to make the AI in the first place. I want it to be able to learn rules on its own without me having to hard code an AI to be very good at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I got to thinking, and I thought it would be interesting if the fitness of a network could be determined based off how well it played against OTHER NETWORKS in its generation. This does seem similar to how humans learn to play games, as I learned to play chess by playing against other people hundreds of times, learning from my mistakes, and my friends also increased in their ability to play chess as well. If GA's were to do that, that would mean I don't have to program AI to play the game (in fact, I wouldn't have to program a &quot;dumb&quot; AI as well, I would only have to hard code the rules of the game, obviously). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Has there been any research or results from GA's determining their fitness based off competing against each other? I did some searching but I have no idea what to look for in the first place (searching 'NEAT fight against each other' did not work well :-(  )&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does this method of training a GA seem practical? It seems practical to me, but are there any potential drawbacks to this? Are GA's meant to only calculate predetermined functions that exist, or do they have the potential to learn and do some decision making?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I were to do this, how would fitness be determined? Say, for the tic tac toe example, should fitness be determined based on whether or not a network places its X's or O's in viable locations, and add fitness if it wins and subtracts fitness if it loses? What about tying the game?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should networks of the same species compete against each other? If they did, then it would seem impractical to have species in the first place, as networks in the same species competing against each other would not allow a successful species to rise to the top, as it would be fighting against each other.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Kind of out of topic, but with my original idea for the tic tac toe GA, would there be a better way to determine fitness? Would creating an intelligent AI be the best way to train a GA?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks for your time, as this is somewhat lengthy, and for your feedback!&lt;/p&gt;&#xA;" OwnerUserId="7958" LastEditorUserId="16565" LastEditDate="2019-01-31T20:55:00.847" LastActivityDate="2019-01-31T20:55:00.847" Title="Could GA's determine fitness by &quot;Fighting&quot; against each other?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;genetic-algorithms&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3519" PostTypeId="2" ParentId="2955" CreationDate="2017-06-20T03:22:47.530" Score="1" Body="&lt;h2&gt;&amp;quot;Ego death&amp;quot;&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Ego death&lt;/strong&gt; is a &amp;quot;complete loss of subjective self-identity.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;-&lt;a href=&quot;https://en.wikipedia.org/wiki/Ego_death&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Ego death&amp;quot;&lt;/a&gt;, Wikipedia&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The defining aspect of an ego is an exclusive sense of self.  If an ego-dominated individual joined a greater mind and dissolved within it without really dying, then the only thing that it's really lost is its ego.&lt;/p&gt;&#xA;&lt;h3&gt;Real world analogy: political states' identities dissolving into larger states&lt;/h3&gt;&#xA;&lt;p&gt;For a real-world analogy, the United States used to be composed of individual states that had a much stronger sense of self-identity; it different states were sort of like different countries.&lt;/p&gt;&#xA;&lt;p&gt;Over time, this distinction's been fading away, and most Americans tend to have little concern for by-state distinctions.  For example, someone from Canada is a foreigner even if the Canadian border is close by, but someone from the other side of the US is still basically just another American.&lt;/p&gt;&#xA;&lt;p&gt;Today, this process repeats itself as individual nations merge under the United Nations.  Some of the recent political battles have been fought between globalists who embrace this great unification and nationalists who wish to maintain clear, distinct state identities and interests.&lt;/p&gt;&#xA;&lt;h3&gt;It's not sudden&lt;/h3&gt;&#xA;&lt;p&gt;Ego death won't typically be sudden.  Rather, the smaller mind would join a larger one by creating new connections over time.  For quite a while, there're still be a meaningful distinction between the ego and the rest of the collective mind.&lt;/p&gt;&#xA;&lt;p&gt;As time goes on, new connections will keep being created and removed, just like in a normal human brain.  The relative lack of connections between the individual's mind and the larger collective mind will tend to fade away.&#xA;Eventually, the distinction between the mind and the larger collective becomes essentially meaningless.&lt;/p&gt;&#xA;&lt;p&gt;Once the mind no longer cares or is able to distinguish itself from the larger collective, their ego/sense-of-self is &amp;quot;dead&amp;quot;, or as you've said, dissolved into the AI.&lt;/p&gt;&#xA;" OwnerUserId="5538" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-06-20T03:56:03.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3521" PostTypeId="2" ParentId="3518" CreationDate="2017-06-20T07:45:23.723" Score="6" Body="&lt;p&gt;i'm the main developer of &lt;a href=&quot;https://github.com/wagenaartje/neataptic&quot; rel=&quot;noreferrer&quot;&gt;Neataptic&lt;/a&gt;, a Javascript neuro-evolution library.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Very effective! Realise that this is how real-life evolution happened as well: we kept on improving against other species, which forced them to improve as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Very practical, &lt;em&gt;especially if you don't want to set up any 'rules'&lt;/em&gt; like you say, it makes the genomes find out what the rules are themselves.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Basically, you let each genome in the population play X games against other genomes, I advise you let each genome play against every other genome in the population. An example of scoring would be giving the genome &lt;code&gt;1&lt;/code&gt; point for winning, and &lt;code&gt;0.25&lt;/code&gt; or &lt;code&gt;0.5&lt;/code&gt; for a tie. Each game should always have a result! &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm not sure about this one, as I haven't implemented speciation. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I want to give you some examples that I have worked on: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://wagenaartje.github.io/neataptic/articles/agario/&quot; rel=&quot;noreferrer&quot;&gt;Agar.io AI&lt;/a&gt; (neuro-evolved neural agents) - basically, I let neural networks evolve to get the highest score they can in agar.io, by competing against each other! It worked better than I expected.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Currently i'm working on new project, a kind of 'cops and robbers' style game.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5344" LastActivityDate="2017-06-20T07:45:23.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3522" PostTypeId="2" ParentId="3515" CreationDate="2017-06-20T10:04:01.247" Score="4" Body="&lt;p&gt;Yes, Kaggle is a great place to start working on ML projects.&#xA;You will start using ML algorithms (at first, using scikit-learn or any other ML library of your choice), learn the concepts of cross-validation (very important concept). You will cope with imbalanced classes, various size of datasets. Moreover, you will be able to ask questions and receive feedback from the community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a great place to start and keep learning, but do not limit yourself at using ML algorithms blindly. Try to understand them, even re-write them if you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's definitely a lot of fun and there are always several competitions running so you can choose whatever dataset you prefer.&lt;/p&gt;&#xA;" OwnerUserId="7965" LastActivityDate="2017-06-20T10:04:01.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3523" PostTypeId="2" ParentId="3310" CreationDate="2017-06-20T20:33:46.113" Score="1" Body="&lt;p&gt;This is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;game theory&lt;/a&gt; question, and involves the intersection of game theory and &lt;a href=&quot;https://en.wikipedia.org/wiki/Ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;ethics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, it's helpful to define love in functional sense as altruism.  (This is consistent with the function aspect of &lt;a href=&quot;https://en.wikipedia.org/wiki/Agape&quot; rel=&quot;nofollow noreferrer&quot;&gt;agape&lt;/a&gt;in terms of how that love functionally affects the material world through the actions of individuals.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To this end, I urge you to look into the concepts of &lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;rational agents&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Superrationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;superrationality&lt;/a&gt;. These concepts are mathematical approaches to decision making, and in the context of humanity, which may one day be extended to include &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;, carry a moral dimension.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a game theoretic context, cooperation is a functional form of altruism, as is the willingness of the superrational agent to &quot;take a hit&quot; and even &quot;turn the other cheek&quot; (make one or two altruistic choices to try to lead the rational agent into a superrational strategy that yields optimal outcomes, as opposed to getting stuck in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Nash_equilibrium&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nash equilibrium&lt;/a&gt; that yields a suboptimal outcome.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Game Theory&lt;/a&gt; demonstrates that even agents with very limited intelligence can develop cooperative behaviors, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Cooperative_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cooperative Game Theory&lt;/a&gt; studies rational coalitions between agents.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cool thing about these fields and ideas is that they are all driven my mathematics.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, without getting into philosophy, we can say that automata will express behaviors analogous to love, but the basis is mathematical, and this also forms a functional limit defined by the parameters of the model.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On a philosophical note, Phillip Dick, who was quite prescient, believed that automata would become smart enough to develop empathy, allowing cooperative behaviors, and leading to love.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;Do Androids Dream Of Electric Sheep&lt;/a&gt; is about this subject, both the humanistic (philosophical) dimension, and the scientific--the plot of the 1968 novel prefigures Evolutionary Game Theory, which was formalized just 5 years later.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-06-20T20:33:46.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3524" PostTypeId="2" ParentId="3518" CreationDate="2017-06-20T22:19:35.960" Score="2" Body="&lt;p&gt;The general notion is that of 'Competitive Coevolution' and there are many (maybe hundreds) of academic papers that describe various alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The excellent (and freely available) &lt;a href=&quot;https://cs.gmu.edu/~sean/book/metaheuristics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Essentials of Metaheuristics&lt;/a&gt; has a whole chapter on the subject.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2017-06-20T22:19:35.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3525" PostTypeId="2" ParentId="3515" CreationDate="2017-06-21T08:00:48.660" Score="3" Body="&lt;p&gt;Yes, definitely it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It puts you in front of real-world problems, theoretical questions and practical issues that are not necessarily what you deal with in college/university tasks and projects.&lt;/p&gt;&#xA;" OwnerUserId="7988" LastActivityDate="2017-06-21T08:00:48.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3526" PostTypeId="1" AcceptedAnswerId="3527" CreationDate="2017-06-21T19:20:53.067" Score="2" ViewCount="389" Body="&lt;p&gt;By means of parts of speech tagging, words of a given sentence can be assumed to be &lt;code&gt;noun&lt;/code&gt;/&lt;code&gt;verb&lt;/code&gt; etc, but if the sentence is for instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;My favourite book is harry potter and the prizoner of azkaban&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;note that the inputs I receive would be from a chat interface so having a fixed format for the data can't be expected. Is there a way to identify &lt;code&gt;&quot;harry potter and the prizoner of azkaban&quot;&lt;/code&gt; as a proper noun from such messages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently this query tags as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;My|PRP$ &#xA;favourite|JJ &#xA;book|NN &#xA;is|VBZ &#xA;harry|JJ &#xA;potter|NN &#xA;and|CC &#xA;the|DT &#xA;prizoner|NN &#xA;of|IN &#xA;azkaban|NN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to know if this can be handled some way, or if there is another algorithm that can handle this?&lt;/p&gt;&#xA;" OwnerUserId="7998" LastActivityDate="2017-06-22T09:51:09.330" Title="Tagging parts of speech when proper noun is a composite" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3527" PostTypeId="2" ParentId="3526" CreationDate="2017-06-22T09:39:33.800" Score="3" Body="&lt;p&gt;I guess your problem is a form of NER (Named-Entity Recognition) tagging. NER tags consist of PER(person), LOC(location), ORG(organization) and MISC(miscellaneous) and O(other). with the help of a NER-tagger algorithm, you probably would have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;My(O) favorite(O) book(O) is(O) harry(B-MISC) potter(I-MISC) and(I-MISC) the(I-MISC) prisoner(I-MISC) of(I-MISC) azkaban(I-MISC).&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now you can tokenize your text with identifying different NER tags, and join &quot;B-&quot; and &quot;I-&quot; prefixes with the same NER tag. e.g. you have &quot;harry potter and ...&quot; as a single token, which is a MISC and starts from harry (since harry is B-MISC) and ends to azkaban (since azkaban has the last consecutive I-MISC tag). Now you can let your POS tagger act the &quot;harry potter ...&quot; as a single token, and it must tag it with &quot;NN&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another solution for this problem is called &quot;chunking&quot;. It works based on a set of rules and detect Noun Phrases (NPs). Here, you define a rule and create the regexp statement for it. e.g. you define all consecutive &lt;code&gt;NN&lt;/code&gt;s as a single NP (harry potter): &lt;code&gt;&amp;lt;NN&amp;gt;+&lt;/code&gt; or a &lt;code&gt;DT&lt;/code&gt; followed by a string of &lt;code&gt;NN&lt;/code&gt;s (the brown fox): &lt;code&gt;&amp;lt;DT&amp;gt;?&amp;lt;NN&amp;gt;+&lt;/code&gt;. Now you tokenize your text based on a regexp matching method. but in your case, chunking such a long NP is almost impossible. because a rule that finds this NP as a NP candidate, would by mistake find a lot of other string of words as a NP candidate too, while they're not a NP at all. (see &lt;a href=&quot;http://www.nltk.org/book/ch07.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;nltk: information extraction&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="6258" LastEditorUserId="6258" LastEditDate="2017-06-22T09:51:09.330" LastActivityDate="2017-06-22T09:51:09.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3528" PostTypeId="1" AcceptedAnswerId="3531" CreationDate="2017-06-22T13:35:38.667" Score="37" ViewCount="19618" Body="&lt;p&gt;Which library (TensorFlow or Keras) would you recommend for a first approach to deep learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a neuroscience student trying for the first time computational approaches, if that matters.&lt;/p&gt;&#xA;" OwnerUserId="8015" LastEditorUserId="2444" LastEditDate="2019-10-13T00:59:53.670" LastActivityDate="2019-10-13T06:56:39.460" Title="Which library would you recommend to begin with deep learning?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;getting-started&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="13" ClosedDate="2020-03-07T07:09:50.667" ContentLicense="CC BY-SA 4.0" />
  <row Id="3530" PostTypeId="2" ParentId="3451" CreationDate="2017-06-23T01:23:01.610" Score="2" Body="&lt;p&gt;There's a strong sentiment towards the idea that medical diagnosis is largely &lt;a href=&quot;https://en.wikipedia.org/wiki/Abductive_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abductive Reasoning&lt;/a&gt;. See &lt;a href=&quot;https://blogs.kent.ac.uk/jonw/files/2015/04/Aliseda2012.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this presentation&lt;/a&gt; for additional details.  One approach to automated abductive reasoning is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2244953/&quot; rel=&quot;nofollow noreferrer&quot;&gt;parsimonious covering theory&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a relatively in-depth look at all of this, check out the book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387973435&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abductive Inference Models for Diagnostic Problem-Solving &lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expert systems have also been shown to work very well for medical diagnosis. As far back as the 1970's, systems like &lt;a href=&quot;https://en.wikipedia.org/wiki/Mycin&quot; rel=&quot;nofollow noreferrer&quot;&gt;MYCIN&lt;/a&gt; could beat human experts in terms of diagnosis and treatment plans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I can tell, there doesn't seem to be any reason in principle to think that AI/ML can't be used across the board for medical diagnosis, mental health or otherwise.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-06-23T01:23:01.610" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3531" PostTypeId="2" ParentId="3528" CreationDate="2017-06-23T06:08:52.203" Score="33" Body="&lt;p&gt;Keras is a simple and high-level neural networks library, written in Python, that works as a wrapper for Tensorflow and Theano. It's easy to learn and use. Using Keras is like working with Lego blocks. It was built so that people can do quick experiments and proofs-of-concept before launching into a full-scale build process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that in mind, it was made to be highly modular and extensible. Now, it can be used for a lot more than just experiments. It can help with RNN, CNN, and combinations of both. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to begin and make a prototype ready solution, then I will recommend you start with Keras. To know the details under the hood, then learn TensorFlow. It has huge active community and also very good resources are available, for example, this &lt;a href=&quot;https://www.youtube.com/watch?v=Se9ByBnKb0o&amp;amp;list=PLXO45tsB95cJHXaDKpbwr5fC_CCYylw1f&quot; rel=&quot;nofollow noreferrer&quot;&gt;Youtube series&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2019-10-13T00:57:00.320" LastActivityDate="2019-10-13T00:57:00.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3532" PostTypeId="1" CreationDate="2017-06-23T11:11:14.820" Score="2" ViewCount="84" Body="&lt;p&gt;I have read some articles, some tutorials but I am still didn't implemented any AI system. So , My question may seem inappropriate for the giants in this field. But I have build certain program , downloaded to microcontroller and it will perform its task. But How to do all this with machine learning. Can I implement AI engine using C like language and make it working in any GPP uC? Please feel free to modify, edit and upgrade this questions if you get my actual problem idea.&lt;/p&gt;&#xA;" OwnerUserId="7888" LastActivityDate="2017-06-23T16:04:26.097" Title="Can we implement ML engine using any general purpose Micro controller?" Tags="&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3533" PostTypeId="1" AcceptedAnswerId="3554" CreationDate="2017-06-23T13:06:15.847" Score="9" ViewCount="4777" Body="&lt;p&gt;Lisp was originally created as a practical mathematical notation for computer programs, influenced by the notation of Alonzo Church's lambda calculus. It quickly became the favored programming language for artificial intelligence (AI) research, according to Wikipedia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Lisp is still used in AI, then is it worthy of learning it, particularly in the context of machine learning and deep learning?&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="7681" LastEditDate="2018-11-17T14:03:54.430" LastActivityDate="2018-11-17T14:03:54.430" Title="Is Lisp still worth learning today in the particular context of Machine learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;history&gt;&lt;programming-languages&gt;&lt;lisp&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="2" ClosedDate="2020-05-13T20:16:40.843" ContentLicense="CC BY-SA 4.0" />
  <row Id="3534" PostTypeId="2" ParentId="3532" CreationDate="2017-06-23T15:55:08.557" Score="1" Body="&lt;p&gt;It depends on what you mean by &quot;ML engine&quot; and whether you want to &lt;em&gt;train&lt;/em&gt; models on the uC or just make predictions.  IF you're doing something simple (maybe linear regression, logistic regression, etc.) you might be able to get away with doing training on a uC, especially for small amounts of data.  But you're almost certainly not going to be training deep neural networks on one of those things, at least not in any reasonable amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OTOH, the &quot;making a prediction part&quot; is usually much cheaper computationally, so if you have a pre-trained model and some prediction engine that can use that model, you could possibly use that on a microcontroller.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-06-23T15:55:08.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3535" PostTypeId="2" ParentId="3532" CreationDate="2017-06-23T16:04:26.097" Score="1" Body="&lt;p&gt;Absolutely, but it almost certainly won't be as strong as companies that design for specialized processors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, &lt;a href=&quot;http://www.nvidia.com/object/drive-px.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NVIDIA Drive&lt;/a&gt; for autonomous vehicles is marketed as a &quot;supercomputer&quot;.  (Autonomous vehicles cannot rely on a network for decision making, so the vehicle needs a computer up to the task.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example of specialized micro-processors, pre-GPU, is &lt;a href=&quot;https://en.wikipedia.org/wiki/Digital_signal_processor#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digital Signal Processors&lt;/a&gt;, where the architecture was driven by the focus on multiplication in common digital signal processing algorithms. (Binary multiplication is much more expensive than addition/subtraction in terms of operations.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I suspect much of the recent success of &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;monte carlo&lt;/a&gt;&quot; has to do with brute force, but I need to confirm that.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole point of high-level languages is to be able to operate on different processors without having to customize for each processor, as with machine languages.  Most C languages allow bitwise operations, so you'd be able to take advantage of that, although interpreted languages such as Python appear to be extremely popular for AI, probably due to the flexibility in terms of testing code without needing to compile.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-06-23T16:04:26.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3538" PostTypeId="2" ParentId="3515" CreationDate="2017-06-25T03:02:59.903" Score="2" Body="&lt;p&gt;Kaggle is a great website. The website provides challenges that allow you to see how various components of ML interplay with each other. For example, large data sets, standard debugging of ML code (much like a &quot;standard&quot; computer programmer), and seeing the breadth of applications of ML to everyday problems are just a few of these components.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coupling working on Kaggle projects with theoretical study of AI/ML is an effective way to learn ML quickly. &lt;/p&gt;&#xA;" OwnerUserId="3231" LastActivityDate="2017-06-25T03:02:59.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3541" PostTypeId="1" CreationDate="2017-06-26T05:39:06.100" Score="1" ViewCount="150" Body="&lt;p&gt;To tune the parameters of Particle swarm optimization (PSO), there are two methods offline and online. In offline manner, the meta-optimization is used to tune the parameters of PSO by using another overlying optimizer. In the online manner, there are two techniques, Self-Adaptation, &quot;consisting of adding some or all of the optimizers behavioural parameters to the search-space, thus making them subject to optimization along with the problem at hand&quot;. Another technique is Meta-Adaptation ,&quot; in which an overlaying optimizer is trying to tune the parameters of another optimizer in an online manner during the optimization of a problem.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The concept of meta-optimization. A black-box optimizer is used in an offline manner as an overlaying meta-optimizer for finding good behavioural parameters of another optimization method, which in turn is used to optimize one or more actual problems.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In standard PSO the particles are initialized by using uniform random numbers and these particles are updated using update equations. the best solution is selected based on the best value of objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my work. I have two data sets, training and theoretical dataset and I need to initialize the particles by using training data instead of random numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, how can I tune the parameters of PSO using training and theoretical data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I have a problem which is, I got the best cost in the initial step of PSO and in the initial step there are no parameters or update equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to tune the parameters using Machine Learning method? How can I do this?&lt;/p&gt;&#xA;" OwnerUserId="8071" LastActivityDate="2019-12-30T18:49:14.030" Title="Tuning the parameters of Particle swarm optimization (PSO)" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;swarm-intelligence&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3542" PostTypeId="2" ParentId="3156" CreationDate="2017-06-26T07:43:22.363" Score="8" Body="&lt;p&gt;The selection of the number of hidden layers and the number of memory cells in LSTM probably depends on the application domain and context where you want to apply this LSTM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimal number of hidden units could be smaller than the number of inputs. AFAIK, there is no rule like multiply the number of inputs with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;. If you have a lot of training examples, you can use multiple hidden units, but sometimes just 2 hidden units work best with little data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, people use one hidden layer for simple tasks, but nowadays research in deep neural network architectures show that many hidden layers can be fruitful for a difficult object, handwritten character, and face recognition problems.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2020-03-07T15:42:43.753" LastActivityDate="2020-03-07T15:42:43.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3543" PostTypeId="2" ParentId="3533" CreationDate="2017-06-26T09:31:41.557" Score="4" Body="&lt;p&gt;AI is a very diverse field of research, technology and science, so many computer technologies and programming languages are used in various AI-related projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the recent developments and breakthroughs are happening in the machine learning, deep-learning areas where the most widely used programming language is Python. The reason is that the major deep learning frameworks (see Tensorflow, Theano, Keras, neon, Caffe) have Python interfaces.&#xA;LISP is not really used in these areas, however you can find some deep learning frameworks (for example Cortex by Thinktopic) implemented in Clojure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LISP was the language of choice for other kind of AI projects, mostly for natural language processing (see SHRDLU, Cyc).&lt;/p&gt;&#xA;" OwnerUserId="6933" LastActivityDate="2017-06-26T09:31:41.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3544" PostTypeId="1" CreationDate="2017-06-26T10:41:15.883" Score="2" ViewCount="122" Body="&lt;p&gt;I need to report accuracies of my neural model in a conference paper as compared to various baselines. What are the accepted standards for reporting accuracies in a fair manner?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neural Model:&lt;/strong&gt;&#xA;To be specific, I'm using 60% as train set, 20% as validation set and 20% as test set to report the accuracy of my neural model. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should I take an average or highest of 3 runs accuracy where in each&#xA;run I randomly sample 60% as training data from the total 80% train + validation set.&lt;/li&gt;&#xA;&lt;li&gt;My neural model is computationally intensive and therefore it is not feasible to perform a k-fold cross validation. Will my accuracy results be accepted by the academic community without a k-fold cross validation? Since my data set is large, I assume using 20% of it used solely for testing should be a fair indicator of accuracy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bag-of-words (BOW):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I report the accuracy for this model so as to perform a fair comparison?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I train BOW on only 60% of data (same as which my neural&#xA;model is being trained on) or should I train BOW on 80% of data&#xA;(train + validation for my neural model)? Which is the accepted way?&#xA;I then test BOW on the same remaining 20% of data (as in neural&#xA;model) in either of the above case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;The other approach is to perform k-fold cross validation but the test set will not be the same 20% as the test set on which my neural model is being evaluated. Is this approach recommended though?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any other information on how to report accuracy results in a research paper comparing neural models (train, val, test) with linear models-BOW,SVM (train,test) is welcome. Please help.&lt;/p&gt;&#xA;" OwnerUserId="8081" LastActivityDate="2017-06-26T10:41:15.883" Title="Train, Validation and Test Split for Reporting Accuracy of Neural Model and BOW" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;research&gt;&lt;training&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3545" PostTypeId="1" CreationDate="2017-06-26T12:37:06.240" Score="1" ViewCount="484" Body="&lt;p&gt;Why does not a researcher like Geoffrey Hinton with his valuable works in machine learning (especially neural networks) get Turing award?&lt;/p&gt;&#xA;" OwnerUserId="6050" LastActivityDate="2017-06-27T09:24:52.360" Title="What are the criteria for choosing Turing award winners?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2017-07-25T21:28:29.350" ContentLicense="CC BY-SA 3.0" />
  <row Id="3546" PostTypeId="1" AcceptedAnswerId="3547" CreationDate="2017-06-26T14:11:01.577" Score="3" ViewCount="207" Body="&lt;p&gt;How many decades are we far from achieving a virtual world just like our real world (or little graded down version) with AI. If in far future if we are able to achieve that (able to provide all the rules and manipulate all  huge amounts of data) will a time leap be possible. Assuming time in virtual world moves much faster, as the computation rate is very high.So few hours of computation in real world will be years in the virtual world. So if we run the virtual world for few days and attain data from its final stage, will we get some future technology in our hand? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This concept of virtual world was put by me, assuming such powerful AI wont be integrated directly to our world without isolation tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about P and NP  problems, fermi paradox, path which will be taken by future AI and time leap and somehow reached to this question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fermi paradox (Assuming this world is also a virtual world created by much smarter species- which explains the isolation to a bit.) &#xA;Assuming that's the case wont nesting of virtual world put pressure on the  root system(most outermost system) running virtual world.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know its weird, but was curious.&#xA;So the final question is .. the whole above stuff is a concept.. how much of reality of AI and its future can be compared with this concept.&lt;/p&gt;&#xA;" OwnerUserId="8087" LastEditorUserId="8087" LastEditDate="2017-06-27T06:26:09.327" LastActivityDate="2017-06-27T06:26:09.327" Title="Time leap (To achieve advance technology) using AI by simulating virtual world at a faster rate" Tags="&lt;philosophy&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3547" PostTypeId="2" ParentId="3546" CreationDate="2017-06-26T23:47:45.650" Score="1" Body="&lt;p&gt;This is a fun and instructive question, so I'm going to attempt an adequate answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q: How many decades are we far from achieving a virtual world just like our real world?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nobody knows, although it's popular to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzweil#2020.E2.80.932050&quot; rel=&quot;nofollow noreferrer&quot;&gt;2050&lt;/a&gt;, a number they came up with, I believe, based on some form of Moore's Law.  But they're just guessing, and nobody knows if they type of singularity proposed is even possible.  We still don't understand how the human brain works, or if strong Artificial General Intelligence is even possible.  &lt;em&gt;(I personally think it definitely is, but am skeptical of the optimistic predictions in terms of when.  There is a wide spectrum of opinions on the matter.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Time Leap&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that in modeling, for instance, the entire planet in this manner, the simulation is going to use all kinds of abstraction (as opposed to modeling every atom and sub-atomic state.) For that reason, I'd think any simulation would only approximate reality, and not be 100% accurate predictively. My guess is you'd have to run a very high number of such simulations, then statistically determine which outcome would be most likely based on a set of outcomes, but the more you narrowed that to a single result, the lower the accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been thinking about Conway's &lt;a href=&quot;https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life#Undecidability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game of Life&lt;/a&gt; in relation to &lt;a href=&quot;https://en.wikipedia.org/wiki/Laplace%27s_demon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Laplace's Demon&lt;/a&gt; lately. Perhaps Game of Life would provide some insight. I think part of the problem there is that there is still a hard limit to the speed of information in the system (which can be analogous to the hard limit of &lt;a href=&quot;https://en.wikipedia.org/wiki/Speed_of_light&quot; rel=&quot;nofollow noreferrer&quot;&gt;c&lt;/a&gt; in the &quot;real&quot; world.)  Game of Life is a deterministic with a single node for each ply of the (predictive) &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;game tree&lt;/a&gt;. Predicting the outcome of a novel, non-trivial starting configuration is indistinguishable from running the simulation.  But the only entity that has perfect information in regards to the entire system is the system itself. It's possible that an intelligent construct within Game of Life could model outcomes of local phenomena, but perfect information on the entire system is an impossibility, and as with quantum phenomena, trading of information actually affects the system. (See glider guns.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;@k.c. posted a very useful link on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Limits_of_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Limits of Computation&lt;/a&gt;, which is probably a much less convoluted approach to the question;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem you propose is deeper because we still don't even know if there is true randomness is nature (quantum phenomenon) or if indeterminacy, and the probabilities that arise out of it, are merely functions of the limits of observations, which is well defined in the mathematics, but also has a philosophical component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quantum phenomena don't directly affect the macro world, but can bleed into it in areas such as &lt;a href=&quot;https://physics.stackexchange.com/a/339984/152803&quot;&gt;formation of crystalline structures&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So until we can solve the randomness question, and find some way around the observational problems at the quantum level, I don't think we could model the type of systems you're asking about with perfect accuracy. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Further Reading:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I strongly advise you take a look at the &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Quantum_Thief&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Thief&lt;/a&gt; trilogy by &lt;a href=&quot;https://en.wikipedia.org/wiki/Hannu_Rajaniemi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hannu Rajaneimi&lt;/a&gt;.  I always recommend these books for interesting questions such as this, not simply because Rananeimi deeply explores the type of post-Singularity world you're thinking about, but because he's a mathematical physicist, and his grasp of the mathematics involved, including quantum theory, is far superior to mine and almost certainly other authors of speculative fiction, whether literary or under the moniker of &quot;futurists&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;His take is that even with all of the computing power you envision, at a level of technology where matter and information are interchangeable, perfectly accurate prediction of the future is not possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it is this next part you may find interesting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rajaneimi inverts your concept of the &quot;Time Leap&quot; by instead talking about the &quot;&lt;strong&gt;Deep Time&lt;/strong&gt;&quot; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Deep Time refers to what a consciousness can experience at peak processor speeds.  For instance, thousands years can be experienced in mere moments of organic human time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not going to give a spoiler on what happens at the end of the trilogy, but it involves the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bekenstein_bound&quot; rel=&quot;nofollow noreferrer&quot;&gt;topology of black holes&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-06-26T23:55:09.767" LastActivityDate="2017-06-26T23:55:09.767" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3548" PostTypeId="1" CreationDate="2017-06-27T03:58:31.947" Score="13" ViewCount="1985" Body="&lt;p&gt;What is the mathematical background required to start learning AI? What else should I also learn?&lt;/p&gt;&#xA;" OwnerUserId="8094" LastEditorUserId="2444" LastEditDate="2020-09-05T19:48:15.577" LastActivityDate="2020-09-05T19:48:15.577" Title="How should I get started with artificial intelligence?" Tags="&lt;math&gt;&lt;getting-started&gt;&lt;education&gt;&lt;ai-field&gt;" AnswerCount="13" CommentCount="0" FavoriteCount="18" ClosedDate="2020-03-04T04:54:06.443" ContentLicense="CC BY-SA 4.0" />
  <row Id="3550" PostTypeId="2" ParentId="3548" CreationDate="2017-06-27T06:57:36.097" Score="5" Body="&lt;p&gt;AI is quite large in scope and it sits at the intersection of several areas. However, there are a few essential fields or topics that you need to know&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set theory&lt;/li&gt;&#xA;&lt;li&gt;Logic&lt;/li&gt;&#xA;&lt;li&gt;Linear algebra&lt;/li&gt;&#xA;&lt;li&gt;Calculus&lt;/li&gt;&#xA;&lt;li&gt;Probability and statistics&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would recommend you to first explore the AI algorithms that you might be interested in. I advise you to start with machine learning and deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do not forget one very important prerequisite, &lt;em&gt;passion&lt;/em&gt;, without it you are probably wasting your time!&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2019-04-08T21:37:34.967" LastActivityDate="2019-04-08T21:37:34.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3551" PostTypeId="2" ParentId="3545" CreationDate="2017-06-27T09:24:52.360" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_Award&quot; rel=&quot;nofollow noreferrer&quot;&gt;The ACM A.M. Turing Award&lt;/a&gt; is an annual prize given by the Association&#xA;  for Computing Machinery (ACM) to &quot;an individual selected for&#xA;  contributions of a technical nature made to the computing community&quot;.&#xA;  It is stipulated that the contributions &quot;should be of lasting and&#xA;  major technical importance to the computer field&quot;. The Turing Award is&#xA;  generally recognized as the highest distinction in computer science&#xA;  and the &quot;Nobel Prize of computing&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8101" LastActivityDate="2017-06-27T09:24:52.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3554" PostTypeId="2" ParentId="3533" CreationDate="2017-06-27T20:53:36.753" Score="5" Body="&lt;p&gt;LISP was popular because back in the old days of AI because of the functional syntax, which worked well with the GOFAI paradigm of the time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays &lt;em&gt;most&lt;/em&gt; researchers have given up on the classical computational theory of mind (read: &lt;a href=&quot;https://plato.stanford.edu/entries/computational-mind/#ClaComTheMin&quot; rel=&quot;noreferrer&quot;&gt;language of thought&lt;/a&gt;), and thus also the GOFAI paradigm that it associates with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LISP is not what you want to learn if you want to do neural network stuff, but the philosophical background is still important to know.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-06-27T20:53:36.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3555" PostTypeId="2" ParentId="3288" CreationDate="2017-06-28T02:05:05.570" Score="1" Body="&lt;p&gt;You can combine the image output using concatenation. Please refer to this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ivpl.eecs.northwestern.edu/sites/default/files/07444187.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ivpl.eecs.northwestern.edu/sites/default/files/07444187.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have a look at the Figure 2. And if you are using caffe, there is a layer called Concat layer. You can use it for your purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not fully clear about what you want to do. But like you said, if you want to pass the image values from the first layer to some layers. Try reading about skip architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use this network as real/fake finder, you can take the difference between two images and convert it to classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="8112" LastActivityDate="2017-06-28T02:05:05.570" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3556" PostTypeId="2" ParentId="2727" CreationDate="2017-06-28T17:24:46.063" Score="2" Body="&lt;p&gt;One thing that you should know is that the sigmoid function limits the output to a value between 0 and 1, which means that using a lot of hidden layers will lead quickly to a Vanishing Gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try to use relu activation function, it has the property to output all the information it gets from the previous layer.&lt;/p&gt;&#xA;" OwnerUserId="8128" LastActivityDate="2017-06-28T17:24:46.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3557" PostTypeId="2" ParentId="2449" CreationDate="2017-06-28T18:24:38.567" Score="7" Body="&lt;p&gt;You can try the actions yourselves, but if you want another reference, &lt;a href=&quot;https://github.com/openai/atari-py/blob/master/doc/manual/manual.pdf&quot; rel=&quot;noreferrer&quot;&gt;check out the documentation for ALE at GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, 0 means no action, 1 means fire, which is why they don't have an effect on the racket.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a better way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;env.unwrapped.get_action_meanings()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8131" LastEditorUserId="8" LastEditDate="2017-07-23T20:45:52.550" LastActivityDate="2017-07-23T20:45:52.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3558" PostTypeId="1" CreationDate="2017-06-28T23:14:43.333" Score="3" ViewCount="825" Body="&lt;p&gt;There is a lot of information on the internet about &lt;code&gt;image classification&lt;/code&gt; and &lt;code&gt;object identification&lt;/code&gt;. I want to know how I can extract (or create) geometric information from an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the image classifier can identify that there is a sphere in the image, how can i generate 3d vertices's and edges from that knowledge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UixLL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UixLL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find all the edges and points on these shapes&lt;/p&gt;&#xA;" OwnerUserId="7816" LastEditorUserId="7800" LastEditDate="2017-11-09T19:05:05.150" LastActivityDate="2018-04-12T15:12:47.653" Title="Identifying geometry in images" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3559" PostTypeId="2" ParentId="3548" CreationDate="2017-06-29T05:50:12.490" Score="10" Body="&lt;p&gt;Start with &lt;a href=&quot;https://www.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's introduction to Machine Learning course on Coursera&lt;/a&gt;.&#xA;There are not many prerequisites for that course, but you will learn how to make some useful things. And, more importantly, it will clearly show you which subjects you need to learn next.&lt;/p&gt;&#xA;" OwnerUserId="8133" LastEditorUserId="2444" LastEditDate="2019-04-08T21:33:20.693" LastActivityDate="2019-04-08T21:33:20.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3560" PostTypeId="1" CreationDate="2017-06-29T10:58:20.600" Score="1" ViewCount="136" Body="&lt;p&gt;Alright, I want to write a mobile app that lets you take a photo of your equation, detects the equation, transforms it from pixels to text and then solves if it's possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I am doing the part where I receive a photo (2D array of pixels) and what to output a resized part of this photo (the part has a rectangular shape) that just contains the equation and some pixels around it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I need a model that takes a 2D array and returns a resized version of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I am new to AI, I don't know what techniques have been successful in optical character recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="8146" LastEditorUserId="1671" LastEditDate="2017-06-29T18:47:48.223" LastActivityDate="2017-06-29T18:56:31.053" Title="Successful methods for optical character recognition?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3561" PostTypeId="2" ParentId="3560" CreationDate="2017-06-29T18:56:31.053" Score="1" Body="&lt;p&gt;This is not meant to be a direct answer, as OCR is not an area I have experience in, but I want to present a deeper problem, that of mathematical notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematics has &quot;exploded&quot; in the last few centuries, and notation is not standard across various sub-fields.  Check out this question on &lt;a href=&quot;https://hsm.stackexchange.com/questions/5962/notational-change-with-integrals/5968#5968&quot;&gt;notational change with integrals&lt;/a&gt;. In some cases, individual sub-fields have alternate forms of notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suspicion is that this task will be enormously difficult without a great deal of human guidance, so I'd point you towards AI methods that utilize &quot;human-assisted machine learning&quot;. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-06-29T18:56:31.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3564" PostTypeId="2" ParentId="35" CreationDate="2017-06-30T10:50:10.107" Score="9" Body="&lt;p&gt;Machine learning is a subfield of artificial intelligence, as the following diagram (taken from &lt;a href=&quot;https://www.linkedin.com/pulse/how-artificial-intelligence-different-from-machine-learning-singh&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;) illustrates.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5i2z7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5i2z7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8166" LastEditorUserId="2444" LastEditDate="2021-01-15T23:57:08.100" LastActivityDate="2021-01-15T23:57:08.100" CommentCount="0" CommunityOwnedDate="2021-01-15T22:22:12.033" ContentLicense="CC BY-SA 4.0" />
  <row Id="3565" PostTypeId="1" CreationDate="2017-07-01T19:36:26.990" Score="5" ViewCount="125" Body="&lt;p&gt;I talked with a graduate computer science who said one challenge of making artificial human-like is making random decisions, and that computers can't be random, that they always need a &quot;seed.&quot; But, if a computer's outcomes are determined by the chaotic movements of electrons, it doesn't seem like it should be difficult to program inherent uncertainty into a computer. So, what exactly is stopping people from harnessing this basic component of reality to allow artificial intelligence to make randomized decisions? I mean, all you'd need is different neural pathways that rely on the superposition of electrons, and that's it. &lt;/p&gt;&#xA;" OwnerUserId="8188" LastActivityDate="2017-07-01T23:40:25.130" Title="Why can't computers be random?" Tags="&lt;human-like&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3566" PostTypeId="2" ParentId="3565" CreationDate="2017-07-01T23:40:25.130" Score="2" Body="&lt;p&gt;Computers (the processor in combination with the memory) are designed to be deterministic.  Otherwise no software would ever work, because the computer would be executing it randomly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer's outcomes are not determined by the chaotic movements of electrons, but by the much strong deterministic currents and voltages between the transistors.  A digital machine is, simply speaking, not affected by noise.  The noise is always present,  but 0.13 volts or whatever gets rounded to 0 V (logic 0) and 3.02 volts or whatever gets rounded to 3.3 V (logic 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is however possible to take advantage of the chaotic movement of electrons as well.  This is done in special hardware called &quot;hardware random number generators&quot;, (some use different forms of true randomness such as nuclear decays).&lt;/p&gt;&#xA;" OwnerUserId="3016" LastActivityDate="2017-07-01T23:40:25.130" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3567" PostTypeId="2" ParentId="3548" CreationDate="2017-07-02T13:55:39.560" Score="1" Body="&lt;p&gt;In addition to Maheshwar's answer, once you feel you want to try more practical Machine Learning, I'd start with &lt;a href=&quot;http://www.cs.waikato.ac.nz/~ml/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weka&lt;/a&gt;. The software is free and effective, they have a good manual and relevant exercises and there are plenty of free videos available on Youtube!&lt;/p&gt;&#xA;" OwnerUserId="8207" LastActivityDate="2017-07-02T13:55:39.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3570" PostTypeId="2" ParentId="2634" CreationDate="2017-07-03T08:28:47.767" Score="1" Body="&lt;p&gt;According to &lt;strong&gt;Wikipedia&lt;/strong&gt;  and &#xA;&lt;a href=&quot;http://www.jfsowa.com/pubs/semnet.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantic Networks&lt;/a&gt;&#xA; ,By John F. Sowa&#xA;This is an updated version of an article in the Encyclopedia of Artificial Intelligence, Wiley, 1987, second edition, 1992. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A semantic network is used when one has knowledge that is best&#xA;  understood as a set of concepts that are related to one another.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_network#Basics_of_semantic_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most semantic networks are cognitively based. They also consist of arcs and nodes which can be organized into a taxonomic hierarchy. Semantic networks contributed ideas of &lt;a href=&quot;https://en.wikipedia.org/wiki/Spreading_activation&quot; rel=&quot;nofollow noreferrer&quot;&gt;spreading activation&lt;/a&gt;, inheritance, and nodes as proto-objects.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An example of semantic network is &lt;a href=&quot;https://en.wikipedia.org/wiki/WordNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;WordNet&lt;/a&gt;. It groups English words into &lt;a href=&quot;https://en.wikipedia.org/wiki/Synonym_ring&quot; rel=&quot;nofollow noreferrer&quot;&gt;synsets&lt;/a&gt;. Some semantic relations between these synsets are meronymy, hyponymy, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, &lt;a href=&quot;https://en.wikipedia.org/wiki/Lexical_semantics#Semantic_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Lexical semantic&lt;/strong&gt;&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...explores whether the meaning of a lexical unit is established by looking at its neighborhood in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic net&lt;/a&gt; (words it occurs with in natural sentences), or whether the meaning is already locally contained in the lexical unit.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="75" LastEditDate="2018-08-24T15:39:44.127" LastActivityDate="2018-08-24T15:39:44.127" CommentCount="0" CommunityOwnedDate="2021-01-27T19:19:01.700" ContentLicense="CC BY-SA 4.0" />
  <row Id="3571" PostTypeId="2" ParentId="3548" CreationDate="2017-07-03T10:10:22.877" Score="4" Body="&lt;p&gt;I would suggest you to &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;start with Andrew Ng's Machine Learning course on Coursera. He provides the brief introduction to mathematics necessary for machine learning. Though not complete, it will be enough to cruise through the course. &lt;/li&gt;&#xA;&lt;li&gt;Next carefully learn logistic regression in the course. The sigmoid function will be widely used in neural networks.&lt;/li&gt;&#xA;&lt;li&gt;In the course, he will introduce you to neural networks and error minimization using back propagation. The back propagation will use optimization technique called Gradient Descent. It is a very important  topic.&lt;/li&gt;&#xA;&lt;li&gt;After completing above steps try Geoff Hinton's neural networks course on Coursera. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you want to go deep in math. Try these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear algebra - Gilbert Strang&lt;/li&gt;&#xA;&lt;li&gt;probability - khan academy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would also like to suggest one of the best books for deep learning: Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville. &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.deeplearningbook.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7899" LastActivityDate="2017-07-03T10:10:22.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3572" PostTypeId="1" CreationDate="2017-07-03T14:15:28.753" Score="1" ViewCount="232" Body="&lt;p&gt;I have an application of neural networks (standard MLP architecture) where I want to forecast a tanh output (ranging from -1 to +1) with about 1500 input features in ~700 samples.&#xA;Each sample represents a snapshot of database tables at a given time of day - f.ex. at 11:50.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I have so few samples, the network is VERY sensitive to overfitting. Although it seems overkill to use neural networks, I find them to perform better than my initial experiments with other approaches, because I can very precisely tune their objective function and regularization, along with a nice possibility for multinomial regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A severe problem in the task at hand&lt;/strong&gt; is that the data in the tables may be adjusted over time. So the database might say that a given feature was last changed at 2016-01-01 11:45, but in reality, a slight change was made a 2016-01-15 07:38. In practice, this means that the training data is unrealistically precise, and that I may instead want to consider it a ballpark estimate.&#xA;For example, a value of 153.18 may instead be considered a value of ~140-165.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To alleviate the problem, I have theorized a &quot;jitter&quot; neuron&lt;/strong&gt; that - during training time, at each iteration - &quot;jitters&quot; the input by subtracting or adding a random value from the feature. The random value to add or subtract is a ratio of the standard deviation of the feature, and the ratio is itself random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The layer is placed right after the input neurons and has as many neurons as there are input features. In my example, the layer is therefore 1500 neurons wide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For example&lt;/strong&gt;, consider an input feature that - across all training samples - has a standard deviation of +- 100.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At training time, a data sample is loaded with the feature value of 1122 (for the sake of example, lets ignore input standardization), and the jitter neuron will incur a randomized change in its value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We define a &quot;jitter&quot; ratio of 0.1, meaning that the ratio to &quot;jitter&quot; the feature with is drawn from a standard distribution with a standard deviation of 0.1 and a mean of 0. In the example, a random roll of the dice lands us in +1.5 standard deviations, outputting a jitter ratio of 0.15.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the training set has a standard deviation of 100, we add to the input feature (that has a value of 1122) a final jitter of +15, resulting in an input feature value of 1137.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same operation goes for all input features. Pragmatically, I do this in keras by generating a jitter-ratio-matrix with mean=0 and std=0.1. The std-value can be an arbitrary amount, but we must not jitter the input data too much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The intuitive justification&lt;/strong&gt; is that it is representative of my real world scenario. Without going into too many details, the input features are typically things such as weather, whose forecasts are naturally unstable, and any changes to the database values backwards in time is likely to be a &quot;typical&quot;, small adjustment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;On a more theoretical level&lt;/strong&gt;, the justification is that it prevents overfitting to specific input features in some samples, as the next iteration across the same data sample will output significantly different output values after the forward pass through the network, as a slight change in the input feature may have a profound impact on the output after the feature has undergone compounding, non-linear activations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I perceive the jitter-neuron to be a form of data-augmentation, like it is done for especialle image classification; instead of the architect generating n augmented samples according to some heuristic (for example, in images, it is normal to crop and rotate the same image in several different ways to enlarge the training set), the jitter neuron generates a theorhetically infinite amount of augmentations at runtime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I keep imagining the parameter hyperspace for the neuron weights in the first layer; instead of having many potential pits (i.e. areas of overfitting), the jitter smooths out these pits, as the jittered input data now generates a different loss value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Early experiments&lt;/strong&gt; do not provide very good results, but not bad either. They just, sort of... Stay the same... However, my dataset is rather small, and it is &lt;strong&gt;&lt;em&gt;notoriously difficult to fit&lt;/em&gt;&lt;/strong&gt; with any model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Therefore, I ask of you&lt;/strong&gt; what you think of the concept of jitter neurons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it seem sensible?&lt;/li&gt;&#xA;&lt;li&gt;Any theoretical reason that it should/should not work?&lt;/li&gt;&#xA;&lt;li&gt;Is it, for some statistical reason or other, inherently inferior to dropout? (notice that I also add dropout, albeit smaller rates)&lt;/li&gt;&#xA;&lt;li&gt;Any proposals for making it better?&lt;/li&gt;&#xA;&lt;li&gt;Comments, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4747" LastActivityDate="2017-07-04T09:20:12.280" Title="Comments on my proposed &quot;Jitter&quot; neuron" Tags="&lt;neural-networks&gt;&lt;neurons&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3573" PostTypeId="1" CreationDate="2017-07-03T16:55:46.917" Score="11" ViewCount="3307" Body="&lt;p&gt;It is often implicitly assumed in computer science that the human mind, or at least some mechanical calculations that humans perform (see &lt;a href=&quot;https://plato.stanford.edu/entries/church-turing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Church-Turing thesis&lt;/a&gt;), can be replicated with a Turing machine, therefore Artificial General Intelligence (AGI), defined as a human-like AI, may be possible.&lt;/p&gt;&#xA;&lt;p&gt;I do not know of any other argument that AGI is possible, and the foregoing argument is extremely weak.&lt;/p&gt;&#xA;&lt;p&gt;Is there a rigorous proof that AGI is possible, at least, in theory? How do we know that everything the human mind can do can be encoded as a program?&lt;/p&gt;&#xA;" OwnerUserId="8221" LastEditorUserId="2444" LastEditDate="2021-01-18T17:24:01.957" LastActivityDate="2021-01-18T22:19:57.973" Title="Is there a rigorous proof that AGI is possible, at least, in theory?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;proofs&gt;&lt;neuroscience&gt;&lt;computational-theory-of-mind&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3577" PostTypeId="2" ParentId="3572" CreationDate="2017-07-04T09:20:12.280" Score="3" Body="&lt;p&gt;Well, adding gaussian noise is a very common regularisation method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2771718/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; is interesting to you. They also have very small datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end there is only so much you can get out of a given dataset. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-07-04T09:20:12.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3580" PostTypeId="1" CreationDate="2017-07-04T11:37:00.687" Score="1" ViewCount="394" Body="&lt;p&gt;Some AI's, such as some chess players, are extremely well coded and have defeated humans in several matches. But I think that they won simply because computers can make calculations way faster than humans can &lt;em&gt;not because&lt;/em&gt; they learned from their opponents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you put an AI against itself, who will win? Will the game continue indefinitely or will the game eventually finish because the AI plays randomly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, are machine learning and self-learning really possible?&lt;/p&gt;&#xA;" OwnerUserId="8171" LastEditorUserId="2444" LastEditorDisplayName="user8248" LastEditDate="2019-06-26T00:39:45.033" LastActivityDate="2019-06-26T21:02:37.710" Title="Are machine learning and self-learning really possible?" Tags="&lt;machine-learning&gt;&lt;chess&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3581" PostTypeId="1" CreationDate="2017-07-04T16:00:21.137" Score="3" ViewCount="311" Body="&lt;p&gt;I was checking services like &lt;a href=&quot;https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft Azure's Cognitive Services Computer Vision API&lt;/a&gt; and &lt;a href=&quot;https://cloud.google.com/vision/?utm_source=google&amp;amp;utm_medium=cpc&amp;amp;utm_campaign=2016-q1-cloud-latam-ML-skws-freetrial&amp;amp;dclid=COD5h5Pj79QCFclThgodr2kFNg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's Vision API&lt;/a&gt; and they are amazing. I was wondering if these services, or any other cloud service for that matter, can recognize an image's content and classify it on a set of fixed categories defined by me, not by the Cognitive Service provider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have different products and I will take several pictures of each product. I want to then use the cloud service and upload all the pictures of each product, so that I can then take a picture of one product and the Computer Vision algorithm will tell me which product I am seeing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible? Is there a third party solution for this problem? If so, how many pictures do I need to train each product's recognition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I was clear. Thanks in advance for any light on the topic!&lt;/p&gt;&#xA;" OwnerUserId="8241" LastActivityDate="2017-08-18T09:58:01.607" Title="Is there a computer vision service for classifying images on a fixed array of images provided by me?" Tags="&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3584" PostTypeId="2" ParentId="3573" CreationDate="2017-07-04T21:33:21.207" Score="6" Body="&lt;p&gt;A strong reason why people think the mind can be implemented on a Turing Machine stems from the &lt;a href=&quot;https://plato.stanford.edu/entries/computational-mind/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Theory of Mind (CTOM)&lt;/a&gt;, which is the leading theory of mind for now.&lt;/p&gt;&#xA;&lt;p&gt;There are lots of reasons for supporting the CTOM, one of which being that the language of belief/desire psychology (propositional attitudes over mental representations) seems to fit nicely to a computational framework.&lt;/p&gt;&#xA;&lt;p&gt;But most simply is that the computation analogy is very helpful in fields such as psychology and neuroscience. When we know of an input/output pair, but don't know how it is implemented, we could say &amp;quot;its performing the relevant computation&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;And since Turing showed that any computation can be performed on an appropriate Turing Machine, the natural extension is that the mind can be implemented on a computer.&lt;/p&gt;&#xA;&lt;p&gt;However, the CTOM is more of a useful idea than a complete theory. We still don't know how to analyze thought in a logical syntax, which can be implemented in a computer. And we also don't know how/why &amp;quot;computation&amp;quot; (whatever that means in this sentence) is performed in the brain.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2021-01-18T17:13:26.467" LastActivityDate="2021-01-18T17:13:26.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3585" PostTypeId="2" ParentId="3580" CreationDate="2017-07-04T23:11:09.440" Score="2" Body="&lt;p&gt;Yes of course they are possible, I have built some! You are correct that there is an aspect of randomness to the process of machine learning but it is more accurate to describe this as trial and error. Each successive try in a machine learning system is evaluated against a goal and if it is an improvement or is closer to the goal, then this try is stored and some aspect that made it successful is incorporated into similar trys for this type of input. Therefore, machines learn by trying all possible combinations of a problem, albeit with some clever human described short cuts or &quot;heuristics&quot; to make the task easier.&lt;/p&gt;&#xA;" OwnerUserId="8249" LastActivityDate="2017-07-04T23:11:09.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3587" PostTypeId="1" AcceptedAnswerId="3588" CreationDate="2017-07-05T02:32:48.847" Score="0" ViewCount="93" Body="&lt;p&gt;I have a complex neural net that will take forever on my laptop to train and i dont have a computer with a GPU, is there a way to run a python script on another computer without having to install an IDE on that computer? (for instance, if i went to an internet cafe)&lt;/p&gt;&#xA;" OwnerUserId="7816" LastActivityDate="2017-07-05T03:57:12.037" Title="Train neural net on another computer?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3588" PostTypeId="2" ParentId="3587" CreationDate="2017-07-05T03:57:12.037" Score="1" Body="&lt;p&gt;Python doesn't require an IDE. You just need to install the Python interpreter to run a Python program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running machine learning on another computer is in fact industry standard. Google, Amazon and Azure can provide you a virtual machine, which you can run anything you want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally rent a powerful machine with GPUs on Google to do my machine learning analysis. It's not free but very powerful.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-07-05T03:57:12.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3589" PostTypeId="2" ParentId="3580" CreationDate="2017-07-05T05:31:41.007" Score="1" Body="&lt;p&gt;Machine learning and self-learning are of course possible, and there're many successive cases!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to know this: machines won't think like humans. Machines form a statistical model and calibrate the model. A good model is a model that does what it's supposed to do accurately.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="75" LastEditDate="2017-07-10T15:41:00.837" LastActivityDate="2017-07-10T15:41:00.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3590" PostTypeId="1" AcceptedAnswerId="3591" CreationDate="2017-07-05T05:56:36.187" Score="12" ViewCount="2206" Body="&lt;p&gt;I was reading about &lt;a href=&quot;https://en.wikipedia.org/wiki/John_McCarthy_%28computer_scientist%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;John McCarthy&lt;/a&gt; and his orthodox vision of Artificial Intelligence. To me, it seems like he was not very much in favour of resources (like time and money) being used to make AIs play games like Chess. Instead, &lt;a href=&quot;https://www.wired.com/2011/10/john-mccarthy-father-of-ai-and-lisp-dies-at-84/&quot; rel=&quot;nofollow noreferrer&quot;&gt;he wanted more to focus on passing the Turing test&lt;/a&gt; and AIs imitating human behavior.&lt;/p&gt;&#xA;&lt;p&gt;I have also read many articles about major companies, like IBM, Google, etc., spending millions of dollars in making AIs play games, like Chess, Go, etc.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;To what extent is this justified?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6798" LastEditorUserId="2444" LastEditDate="2021-01-30T19:29:23.957" LastActivityDate="2021-01-30T19:29:23.957" Title="Why spend so much time and money to build AIs to play games?" Tags="&lt;research&gt;&lt;game-theory&gt;&lt;chess&gt;&lt;turing-test&gt;&lt;ai-field&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3591" PostTypeId="2" ParentId="3590" CreationDate="2017-07-05T06:04:03.833" Score="11" Body="&lt;p&gt;In the book &lt;a href=&quot;https://cs.calvin.edu/courses/cs/344/kvlinden/resources/AIMA-3rd-edition.pdf#page=204&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (section 5.7, p. 185), Russell and Norvig write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In 1965, the Russian mathematician Alexander Kronrod called chess &amp;quot;the Drosophila of artificial intelligence.&amp;quot; John McCarthy disagrees: whereas geneticists use fruit flies to make discoveries that apply to biology more broadly, AI has used chess to do the equivalent of breeding very fast fruit flies. Perhaps a better analogy is that chess is to AI as Grand Prix motor racing is to the car industry: state-of-the-art game programs are blindingly fast, highly optimized machines that incorporate the latest engineering advances, but they aren't much use for doing the shopping or driving off-road. Nonetheless, racing and game-playing generate excitement and a steady stream of innovations that have been adopted by the wider&#xA;community&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, although these games (like Chess, Go, and Bridge) may not &lt;em&gt;apparently&lt;/em&gt; be useful/beneficial to many people, the AI programs developed to play them have introduced/included concepts/techniques, like &lt;strong&gt;null move&lt;/strong&gt; heuristics, &lt;strong&gt;futility pruning&lt;/strong&gt;, &lt;strong&gt;combinatorial game theory&lt;/strong&gt;, &lt;strong&gt;finessing&lt;/strong&gt; and &lt;strong&gt;squeezing&lt;/strong&gt; or &lt;strong&gt;meta-reasoning&lt;/strong&gt;, which can potentially be useful to a wider spectrum of Computer Science (and not just Artificial Intelligence).&lt;/p&gt;&#xA;&lt;p&gt;You can see it similar to space missions of NASA, ISRO, JAXA and other space agencies. All these missions don't seem to have a direct benefit to citizens, but have many indirect benefits. They pave the way for technological innovations (GPS, 3D printing, car crash technology, clean energy, LED), the creation of jobs, etc. Advance storms, hurricane detection is the output of space exploration, which has saved millions of lives worldwide.&lt;/p&gt;&#xA;&lt;p&gt;AI in games has not just helped to develop the software but hardware also. Many innovations have been seen to produce highly optimised and powerful hardware.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-30T14:16:52.983" LastActivityDate="2021-01-30T14:16:52.983" CommentCount="2" CommunityOwnedDate="2021-01-30T13:19:50.610" ContentLicense="CC BY-SA 4.0" />
  <row Id="3593" PostTypeId="2" ParentId="3590" CreationDate="2017-07-05T21:42:43.783" Score="1" Body="&lt;p&gt;I find the statement troubling as the first confirmed algorithmic intelligence may have been a &lt;a href=&quot;http://www.historyofinformation.com/expanded.php?id=4472&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIM automata&lt;/a&gt;, so from my perspective, the development of Algorithmic Intelligence is inseparable from combinatorial games. it would also seem that McCarthy does not hold the opinion that games are useful, which leads me to suspect he has never seriously studied the history of games. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial Game Theory&lt;/a&gt;, an applied field in mathematics and computing, was formalized in the decades after the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sprague%E2%80%93Grundy_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sprague-Grundy Theorem&lt;/a&gt; which was a mathematical analysis of the game of NIM. More recently, the protein folding game &lt;a href=&quot;https://en.wikipedia.org/wiki/Foldit#Accomplishments&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foldit&lt;/a&gt; produced real results in an applied field. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The answer I usually give is that games such as Chess and Go provide complexity akin to nature using extremely simple parameters.  (In essence, combinatorial games and puzzles, like Sudoku, are complexity engines.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But games, unlike puzzles, which are solo endeavors, require a type of strategic decision-making that is quite useful. (@Ugnes answer lists many of them.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Combinatorial games in particular provide a useful benchmark for the capability of algorithms to manage intractable problems.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is also a &lt;a href=&quot;https://en.wikipedia.org/wiki/Public_relations&quot; rel=&quot;nofollow noreferrer&quot;&gt;PR&lt;/a&gt; factor.  Algorithmic language translation has gotten extremely good in recent years, but you never hear the press making a big deal about it.  Compare to DeepBlue vs. Kasparov, or AlphaGo vs. Sedol. (This stack exploded with ML questions after the AlphaGo result.)  This is similar to the US moon landings, which was great, if not strictly necessary, engineering feat that inspired generations of budding scientists. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Postscript: It's notable that until recently, the term &quot;strong&quot; was reserved for Artificial General Intelligence, which is still highly theoretical.  After AlphaGo, I'm starting to see scholars use the term &quot;Strong Narrow AI.&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use of strong in relation to Artificial General Intelligence is purely philosophical. By contrast, the way the term is used in Combinatorial Game Theory (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solved Game&lt;/a&gt;) is purely practical and involves mathematical proofs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chess remains unsolved, and therefore it is still useful for study.  [See GiraffeChess following.]  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fields of Game Theory and Combinatorial Game Theory include names like &lt;a href=&quot;https://en.wikipedia.org/wiki/John_von_Neumann&quot; rel=&quot;nofollow noreferrer&quot;&gt;Von Neumann&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/John_Forbes_Nash_Jr.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nash&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/John_Horton_Conway&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conway&lt;/a&gt;, and more recently &lt;a href=&quot;https://en.wikipedia.org/wiki/Erik_Demaine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demain&lt;/a&gt; at MIT.   And if you want to include combinatorial puzzles like Sudoku, we can stretch this back to &lt;a href=&quot;https://en.wikipedia.org/wiki/Graeco-Latin_square&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler&lt;/a&gt;. For these reasons, as well as those listed above, I have a hard time seeing analysis of games as a trivial pursuit.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/541276/deep-learning-machine-teaches-itself-chess-in-72-hours-plays-at-international-master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe Ches&lt;/a&gt;s was a recent result by an individual mathematician/programmer, Matthew Lai, who used a Neural Network approach to create a chess algorithm that taught itself to play at an international master level in 72 hours.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of of Lai's goals was to create an algorithm that produced more &quot;human like play&quot;. (Compare to the &quot;inhuman&quot; play of algorithms like AlphaGo.)  Giraffe is not AGI, but it certainly could be taken to be an piece of the puzzle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer games are arguably the deepest type of interactions shared by humans and automata, and this type of interaction goes back almost to the inception of modern computing.   &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-07-05T22:17:36.707" LastActivityDate="2017-07-05T22:17:36.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3596" PostTypeId="1" CreationDate="2017-07-06T07:31:47.860" Score="1" ViewCount="1475" Body="&lt;p&gt;I am trying to run Deep Q-learning algorithm on a game which I made in Python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the function approximator.&lt;/p&gt;&#xA;&lt;p&gt;The game looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zv7e9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zv7e9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Player can move both the paddles and randomly a white ball is generated from the center of screen. If the paddle touches the white ball, reward of +1 is awarded if it misses the reward of -1 is awarded and the same reward is passed in the Q-learning algorithm to learn. Only 3 actions are possible move to left, move to right and stay.&lt;/p&gt;&#xA;&lt;p&gt;Here is the code for my Deep Q learning algorithm:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function&#xA;from keras.models import Sequential&#xA;from keras.layers.core import *&#xA;from keras.layers.convolutional import Conv2D&#xA;from keras.optimizers import Adam&#xA;from scipy.misc import imresize&#xA;import collections&#xA;import numpy as np&#xA;&#xA;import matplotlib&#xA;matplotlib.use('Agg')&#xA;import matplotlib.pyplot as plt&#xA;import wrapped&#xA;&#xA;# stack the four frames together&#xA;def preprocess_images(image):&#xA;&#xA;    if image.shape[0] &amp;lt; 4:&#xA;        xt_list = []&#xA;        for i in range(image.shape[0]):&#xA;            x_t = imresize(image[i],(100,100))&#xA;            x_t = x_t.astype('float')&#xA;            x_t /= 255.0&#xA;            xt_list.append(x_t)&#xA;        num = 4 - len(xt_list)&#xA;        length = len(xt_list)&#xA;        for x in range(num):&#xA;            x_t = xt_list[length-1]&#xA;            xt_list.append(x_t)&#xA;        s_t = np.stack((xt_list[0],xt_list[1],xt_list[2],xt_list[3]),axis=2)&#xA;&#xA;    else:&#xA;        xt_list = []&#xA;        for i in range(image.shape[0]):&#xA;            x_t = imresize(image[i],(100,100))&#xA;            x_t = x_t.astype('float')&#xA;            x_t /= 255.0&#xA;            xt_list.append(x_t)&#xA;        s_t = np.stack((xt_list[0],xt_list[1],xt_list[2],xt_list[3]),axis=2)&#xA;    s_t = np.expand_dims(s_t,axis=0)&#xA;    return s_t&#xA;&#xA;# generate data to train neural network&#xA;def gen_next_batch(experience,model,num_actions,gamma,batch_size):&#xA;&#xA;    batch_indices = np.random.randint(low=0,high=len(experience),size=batch_size)&#xA;    batch = [experience[i] for i in batch_indices]&#xA;    X = np.zeros((batch_size,100,100,4))&#xA;    Y = np.zeros((batch_size,num_actions))&#xA;    for i in range(len(batch)):&#xA;        s_t, a_t, r_t, s_t1, game_over = batch[i]&#xA;        X[i] = s_t&#xA;        Y[i] = model.predict(s_t)[0]&#xA;        Q_sa = np.max(model.predict(s_t1)[0])&#xA;        if game_over:&#xA;            Y[i,a_t] = r_t&#xA;        else:&#xA;            Y[i,a_t] = r_t + gamma*Q_sa&#xA;        return X,Y&#xA;&#xA;# Neural Network model implemented using keras&#xA;model = Sequential()&#xA;model.add(Conv2D(32,kernel_size=8,strides=4,kernel_initializer='normal',padding=&amp;quot;same&amp;quot;,input_shape=(100,100,4)))&#xA;model.add(Activation('relu'))&#xA;model.add(Conv2D(64,kernel_size=4,strides=2,kernel_initializer='normal',padding='same'))&#xA;model.add(Activation('relu'))&#xA;model.add(Conv2D(32,kernel_size=3,strides=1,kernel_initializer='normal',padding='same'))&#xA;model.add(Activation('relu'))&#xA;model.add(Flatten())&#xA;model.add(Dense(512,kernel_initializer='normal'))&#xA;model.add(Activation('relu'))&#xA;model.add(Dense(3,kernel_initializer='normal'))&#xA;model.add(Activation('linear'))&#xA;opt = Adam(lr=1e-06)&#xA;model.compile(loss='mse',optimizer=opt)&#xA;&#xA;&#xA;NUM_ACTIONS = 3&#xA;GAMMA = 0.99&#xA;INITIAL_EPSILON = 0.1&#xA;FINAL_EPSILON = 0.0001&#xA;NUM_EPOCHS = 10000&#xA;MEMORY_SIZE = 50000&#xA;BATCH_SIZE = 64&#xA;EPSILON = INITIAL_EPSILON&#xA;&#xA;experience = collections.deque(maxlen=MEMORY_SIZE)&#xA;game = wrapped.Paddle()&#xA;&#xA;&#xA;for x in range(1000):&#xA;    game.reset()&#xA;    game_over = False&#xA;&#xA;    a_0 = 2&#xA;    x_t, r_0, game_over = game.step(a_0)&#xA;    s_t = preprocess_images(x_t)&#xA;&#xA;    while  not game_over:&#xA;        s_t1 = s_t&#xA;        a_t = np.random.randint(low=0,high=NUM_ACTIONS,size=1)[0]&#xA;&#xA;        x_t, r_t, game_over = game.step(a_t)&#xA;        s_t = preprocess_images(x_t)&#xA;        experience.append((s_t1,a_t,r_t,s_t1,game_over))&#xA;    print('Random Actions')&#xA;    print(len(experience))&#xA;&#xA;Reward_list = []&#xA;for i in range(NUM_EPOCHS):&#xA;    game.reset()&#xA;    loss = 0.0&#xA;    R = 0&#xA;&#xA;    a_0 = 2&#xA;    x_t, r_0, game_over = game.step(a_0)&#xA;    s_t = preprocess_images(x_t)&#xA;&#xA;    while not game_over:&#xA;&#xA;        s_t1 = s_t&#xA;&#xA;        if np.random.rand() &amp;lt;= EPSILON:&#xA;            a_t = np.random.randint(low=0,high=NUM_ACTIONS,size=1)[0]&#xA;        else:&#xA;            q = model.predict(s_t)[0]&#xA;            a_t = np.argmax(q)&#xA;&#xA;        x_t, r_t, game_over = game.step(a_t)&#xA;        s_t = preprocess_images(x_t)&#xA;&#xA;        experience.append((s_t1,a_t,r_t,s_t,game_over)) # stores experiences&#xA;&#xA;        X,Y = gen_next_batch(experience,model,NUM_ACTIONS,GAMMA,BATCH_SIZE)&#xA;        loss += model.train_on_batch(X,Y)&#xA;&#xA;        R += r_t&#xA;&#xA;    if EPSILON &amp;gt; FINAL_EPSILON:&#xA;        EPSILON -= (INITIAL_EPSILON - FINAL_EPSILON)/NUM_EPOCHS&#xA;&#xA;    print('Episode : %d | Epsilon: %f | Reward : %f | Loss : %f'%(i+1,EPSILON,R,loss))&#xA;    Reward_list.append(R)&#xA;&#xA;    if (i+1) % 1000 == 0:&#xA;        plt.plot(Reward_list)&#xA;        plt.xlabel('Episodes')&#xA;        plt.ylabel('Reward')&#xA;        plt.savefig('/output/reward.png')&#xA;        model.save(&amp;quot;/output/RL_MODEL.h5&amp;quot;,overwrite=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I trained the Neural Network for 10000 Epochs and the Total Reward per episode looks like this, which clearly indicates the algorithm is not learning (Max of +1 and Min of -1 reward is possible in a episode).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zXymK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zXymK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggest me what I am doing wrong? I am having a very hard time in implementing the reinforcement learning algorithms. I have tried to implement same algorithm on an another game but had the same issue. Is it related to my epsilon-greedy policy, or I am not training enough or something else? Please help.&lt;/p&gt;&#xA;" OwnerUserId="8273" LastEditorUserId="32410" LastEditDate="2020-12-29T14:28:08.780" LastActivityDate="2020-12-29T14:28:08.780" Title="Q Learning Algorithm not converging" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;q-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3602" PostTypeId="2" ParentId="3580" CreationDate="2017-07-06T21:34:46.577" Score="0" Body="&lt;p&gt;I like your focus on optimization &lt;em&gt;(re: &quot;One of my productive days was throwing away 1,000 lines of code&quot;;)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the problem with this question is it your supposition is incorrect. Check out &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe Chess&lt;/a&gt; for more info on self-learning. [Note that Giraffe Chess is a result, not a hypothesis, and Lai was subsequently tapped by DeepMind.] &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also recommend getting familiar with concepts like &lt;a href=&quot;https://en.wikipedia.org/wiki/NP_(complexity)&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-deterministic polynomial time&lt;/a&gt; and obsessing over &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractability&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding random choices, &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;monte carlo&lt;/a&gt;&quot; has had major successes of late, but I suspect the success is related to processor speed, and may not be sufficient in greater complexity spaces, or problems where &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;rationality is heavily bounded&lt;/a&gt;.   &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-07-06T21:34:46.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3604" PostTypeId="2" ParentId="3581" CreationDate="2017-07-07T13:19:09.963" Score="1" Body="&lt;p&gt;You can check the cloudcv : &lt;a href=&quot;https://cloudcv.org/trainaclass/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloudcv.org/trainaclass/&lt;/a&gt;&#xA;you can use the web service or call throw matlab or python api&lt;/p&gt;&#xA;" OwnerUserId="8309" LastActivityDate="2017-07-07T13:19:09.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3606" PostTypeId="1" CreationDate="2017-07-07T22:42:50.787" Score="1" ViewCount="70" Body="&lt;p&gt;I understand that as a network learns about an output with regards to an input, weights are updated according to how wrong the guess was for that node. So, over time, the weights move in the &quot;direction&quot; towards the correct value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use a separate neural network that takes as input the weights of the first network while it trains to try approximating that &quot;direction&quot; and, in effect, pushing the weights in that direction faster?&lt;/p&gt;&#xA;" OwnerUserId="7816" LastEditorUserId="2444" LastEditDate="2020-02-21T00:05:36.783" LastActivityDate="2020-02-21T00:05:36.783" Title="Can a second network take as input the weights of a first network and help training the first network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3608" PostTypeId="5" CreationDate="2017-07-08T03:29:00.323" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-07-08T03:29:00.323" LastActivityDate="2017-07-08T03:29:00.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3609" PostTypeId="4" CreationDate="2017-07-08T03:29:00.323" Score="0" Body="For questions related to Google in the context of AI (e.g. related to Google Translate). Google is an American multinational technology company specializing in internet-related services and products. " LastEditorUserId="2444" LastEditDate="2019-06-20T14:58:41.730" LastActivityDate="2019-06-20T14:58:41.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3610" PostTypeId="5" CreationDate="2017-07-08T03:30:14.220" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-07-08T03:30:14.220" LastActivityDate="2017-07-08T03:30:14.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3611" PostTypeId="4" CreationDate="2017-07-08T03:30:14.220" Score="0" Body="For questions related to biology in the context of artificial intelligence. For example, use this tag when you want to ask if a certain machine learning model was inspired by some biological counterpart." LastEditorUserId="2444" LastEditDate="2021-02-15T19:33:22.320" LastActivityDate="2021-02-15T19:33:22.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3614" PostTypeId="2" ParentId="3573" CreationDate="2017-07-08T22:52:27.943" Score="4" Body="&lt;h1&gt;Consciousness is not well-understood&lt;/h1&gt;&#xA;&lt;p&gt;As an AI practitioner and philosopher, I don't think that humans will be able to create a truly conscious silicon-based AGI.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans are incapable of creating some &amp;quot;thing&amp;quot; from fiat (a decree). It's never happened in human history. The innovation cycle must begin with some &amp;quot;thing&amp;quot; (some &amp;quot;stuff&amp;quot; of some kind), and consciousness is not a thing.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The essence of consciousness is imperceptible (it is unseen), like gravity, and attraction. Humans are incapable of creating things that they are unable to observe. Even if they are able to observe it, the human perceptive ability is unable to actually perceive the true essences of things seen, much less those unseen.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans do not adequately understand the &amp;quot;essence&amp;quot; and &amp;quot;nature&amp;quot; of consciousness - which is a fundamental prerequisite to creating &amp;quot;anything&amp;quot; at all.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &amp;quot;&lt;strong&gt;easy&lt;/strong&gt;&amp;quot; problems, those physical by nature, although not yet solved by empirical domains of psychology, cognitive science, and neuroscience, are expected to be solved in time. Regardless, &lt;strong&gt;they are &amp;quot;not&amp;quot; yet solved today&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &amp;quot;&lt;strong&gt;hard&lt;/strong&gt;&amp;quot; problems, those determining why or how consciousness occurs given the right arrangement of brain matter, &lt;strong&gt;might not &lt;em&gt;ever&lt;/em&gt; be solved&lt;/strong&gt;, since it must explain why certain physical mechanism gives rise to consciousness instead of &amp;quot;something else&amp;quot; or &amp;quot;nothing at all&amp;quot;. This is significant and is the most damning of all arguments against the idea of humans creating true existential consciousness in silicon creatures as a whole.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1&gt;&lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/#DuaThe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dualism&lt;/a&gt; vs &lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/#PhyThe&quot; rel=&quot;nofollow noreferrer&quot;&gt;physicalism&lt;/a&gt;&lt;/h1&gt;&#xA;&lt;p&gt;The greatest philosophical debate on consciousness has focused on the distinction between &lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/#DuaThe&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;dualism&lt;/strong&gt;&lt;/a&gt; and &lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/#PhyThe&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;physicalism&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Dualism&lt;/strong&gt; is the theory that consciousness somehow falls outside the domain of the physical (these are the &lt;strong&gt;hard&lt;/strong&gt; problems)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Physicalism&lt;/strong&gt; holds that consciousness is entirely physical. (&lt;strong&gt;significant arguments below view it as false&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Problems with dualist views&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Why would one be motivated to hold a &lt;strong&gt;dualist&lt;/strong&gt; view?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can something that is &lt;strong&gt;not&lt;/strong&gt; part of the physical world interact with the physical world? That seems impossible!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The physical world is a &lt;strong&gt;closed system&lt;/strong&gt;, how can you have a consciousness that is not part of a closed system?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Consciousness is a lot like mass or charges, &lt;strong&gt;it's a philosophically &amp;quot;fundamental&amp;quot; thing&lt;/strong&gt;, you either &amp;quot;have it or you don't&amp;quot;, you can &lt;strong&gt;simulate&lt;/strong&gt; them, but you &lt;strong&gt;cannot existentially &amp;quot;be&amp;quot;&lt;/strong&gt; them unless you have those &lt;strong&gt;specific &amp;quot;properties&amp;quot;&lt;/strong&gt;, and behavior &amp;quot;simulating&amp;quot; human consciousness is &lt;strong&gt;not&lt;/strong&gt; a fundamental thing.&lt;/p&gt;&#xA;&lt;p&gt;So, despite the sensationalist tendencies of &lt;strong&gt;rogue journalists &amp;quot;parroting&amp;quot; wildly spectacular concepts from the fringe camps of the transhumanists (aka science fiction)&lt;/strong&gt; - a quick perusal of the more rigorous communities of the grounded and thoughtful philosophers camp strongly and convincingly argues otherwise.&lt;/p&gt;&#xA;&lt;h2&gt;More musings on physicalism&lt;/h2&gt;&#xA;&lt;p&gt;Actually, consciousness has &lt;em&gt;never&lt;/em&gt; been properly explained by the biomechanical, which is more or less the key issue of all philosophical studies of the mind - which is essentially the study of consciousness.&lt;/p&gt;&#xA;&lt;p&gt;Physicalists have &lt;strong&gt;trouble&lt;/strong&gt; explaining several aspects of consciousness in a way that is consistent with our &amp;quot;&lt;strong&gt;observations&lt;/strong&gt;&amp;quot; of how physical properties interact. Let me list a few more problems, with a reference to the titans of philosophy.&lt;/p&gt;&#xA;&lt;h3&gt;Arguments against physicalism&lt;/h3&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It is &lt;strong&gt;impossible&lt;/strong&gt; to imagine how mere neuronal tissue could produce conscious experience (&lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Huxly&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Failures of supervenience, such as &lt;a href=&quot;https://plato.stanford.edu/entries/zombies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;zombies&lt;/a&gt; and &lt;a href=&quot;https://plato.stanford.edu/entries/qualia-inverted/#LocInvSpeSce&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverted spectra&lt;/a&gt;, are conceivable (&lt;a href=&quot;http://vedicilluminations.com/downloads/Consciousness-Life-After-Death/The-Conscious-Mind_D.Chalmers.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Chalmers&lt;/a&gt;, John Locke, etc.).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/#Sub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mary learns something&lt;/a&gt; (&lt;a href=&quot;https://www.jstor.org/stable/2026143#metadata_info_tab_contents&quot; rel=&quot;nofollow noreferrer&quot;&gt;Frank Jackson&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Brains have mass, volume, and other physical properties, but &lt;strong&gt;experiences&lt;/strong&gt; do not.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Paranormal phenomena (near-death experiences &lt;strong&gt;NDErs&lt;/strong&gt;, ESP, etc) are real, and involve consciousness implemented in a &lt;strong&gt;nonphysical&lt;/strong&gt; substrate.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If shrunken so I can stroll around your brain and look about, I will observe neuronal processes, not experiences (G. W. Leibniz).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The soul is the seat of consciousness, and the soul is not physical. (Theological constraints recognized BTW...).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Conscious experiences have &lt;strong&gt;intrinsic qualities&lt;/strong&gt;, but science can &lt;em&gt;&lt;strong&gt;only&lt;/strong&gt;&lt;/em&gt; tell us about &lt;strong&gt;relational qualities&lt;/strong&gt; (Russell, Rosenberg).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Consciousness cannot be observed&lt;/strong&gt;; there will &lt;strong&gt;never&lt;/strong&gt; be a &lt;strong&gt;consciousness detector&lt;/strong&gt; that can tell you if a given creature is conscious.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Conscious experiences are not simply the movement of molecules, consciousness is more than mass in motion (Mill, Ward).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8333" LastEditorUserId="2444" LastEditDate="2021-01-18T22:19:57.973" LastActivityDate="2021-01-18T22:19:57.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3617" PostTypeId="2" ParentId="3573" CreationDate="2017-07-10T16:01:26.860" Score="1" Body="&lt;p&gt;Rather than prove that Artificial General Intelligence is possible, I would consider an argument for why it is &lt;strong&gt;impossible&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We start by defining what we mean by AGI. You state that the human mind can be replicated by a Turing Machine, and therefore AGI should be possible. This seems to imply that humans have `General' (capital G) intelligence. By this I mean that you are implying that with enough time, humans can learn any task or problem. However, if you are asserting that humans minds are machines replicable by Turing machines, you must also concede that they have some finite representational power. Finite representational power implies that there will always be problems or tasks where our intelligence will fail (a consequence of the &lt;a href=&quot;http://www.aihorizon.com/essays/generalai/no_free_lunch_machine_learning.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;No Free Lunch Theorem&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately (maybe unfortunately), finite representational power is what allows us to learn at all: &lt;a href=&quot;https://en.wikipedia.org/wiki/VC_dimension&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;VC Dimension&quot;&gt;VC Dimension&lt;/a&gt; (a measure of the complexity or representational power of a class of functions that a learning algorithm can learn [also &lt;a href=&quot;https://www.quora.com/Explain-VC-dimension-and-shattering-in-lucid-Way&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.quora.com/What-is-an-intuitive-explanation-of-what-the-VC-dimension-is&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;]) implies that a learning algorithm that can learn &lt;em&gt;any problem&lt;/em&gt; is actually useless, as the ability to explain any set of data yields the requirement that the algorithm see an infinite amount of examples in order to generalize. While this result comes from the relatively constrained class of binary classification problems in the statistical learning setting, the intuition seems to apply more broadly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, I would refer to this quote from &lt;a href=&quot;http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shalev-Shwartz and Ben-David (2014)&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If someone can explain every phenomenon, his explanations are worthless.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It truly is the case that our decision to &lt;em&gt;systematically ignore&lt;/em&gt; some possible outcomes is the only thing that allows use to learn useful representations of real-world problems.&lt;/p&gt;&#xA;" OwnerUserId="8366" LastEditorUserId="8366" LastEditDate="2017-07-11T05:15:52.737" LastActivityDate="2017-07-11T05:15:52.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3618" PostTypeId="2" ParentId="15" CreationDate="2017-07-10T16:13:33.333" Score="7" Body="&lt;p&gt;The classical Turing Test certainly does have limitations. Because I don't see it mentioned here yet, I'll suggest you read about &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;noreferrer&quot;&gt;The Chinese Room&lt;/a&gt;, which is one of the most commonly cited reasons why the Turing Test indeed falls short of ascertaining true 'consciousness'. However, I'd also note that Turing himself, &lt;a href=&quot;http://www.loebner.net/Prizef/TuringArticle.html&quot; rel=&quot;noreferrer&quot;&gt;in the original paper that proposed the Turing Test&lt;/a&gt;, explicitly acknowledged himself that the test &lt;strong&gt;was not a test to detect consciousness&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I propose to consider the question, &amp;quot;Can machines think?&amp;quot; This should begin with definitions of the meaning of the terms &amp;quot;machine&amp;quot; and &amp;quot;think.&amp;quot; The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous, If the meaning of the words &amp;quot;machine&amp;quot; and &amp;quot;think&amp;quot; are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, &amp;quot;Can machines think?&amp;quot; is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.&lt;/p&gt;&#xA;&lt;p&gt;The new form of the problem can be described in terms of a game which we call the 'imitation game.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This imitation game is the test that we now know today (and also the inspiration for the name of a recent feature film starring Benedict Cumberbatch and Keira Knightley).&lt;/p&gt;&#xA;" OwnerUserId="8366" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-07-11T05:04:41.290" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3619" PostTypeId="2" ParentId="2144" CreationDate="2017-07-10T16:42:20.013" Score="4" Body="&lt;p&gt;Examining the architecture of the DNC &lt;a href=&quot;https://www.quora.com/How-does-the-Deepmind-DNC-Differentiable-Neural-Computer-compare-to-LSTMs-and-RNNs&quot; rel=&quot;nofollow noreferrer&quot;&gt;indeed shows many similarities to the LSTM&lt;/a&gt;. Consider the diagram in the DeepMind article that you linked to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NwTfn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NwTfn.png&quot; alt=&quot;DeepMind DNC Architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare this to the LSTM architecture (credit to ananth on SlideShare):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Y0tc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Y0tc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some close analogs here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Much like the LSTM, the DNC will perform some conversion from &lt;strong&gt;input&lt;/strong&gt; to fixed-size state vectors (&lt;strong&gt;h&lt;/strong&gt; and &lt;strong&gt;c&lt;/strong&gt; in the LSTM)&lt;/li&gt;&#xA;&lt;li&gt;Likewise, the DNC will perform some conversion from these fixed-size state vectors to potentially arbitrarily-lengthed &lt;strong&gt;output&lt;/strong&gt; (in the LSTM we repeatedly sample from our model until we are satisfied/the model indicates we are done)&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;forget&lt;/strong&gt; and &lt;strong&gt;input&lt;/strong&gt; gates of the LSTM represent the &lt;strong&gt;write&lt;/strong&gt; operation in the DNC ('forgetting' is essentially just zeroing or partially zeroing memory)&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;output&lt;/strong&gt; gate of the LSTM represents the &lt;strong&gt;read&lt;/strong&gt; operation in the DNC&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, the DNC is definitely more than an LSTM. Most obviously, it utilizes a larger state which is discretized (addressable) into chunks; this allows it to make the forget gate of the LSTM more binary. By this I mean that the state is not necessarily eroded by some fraction at every time step, whereas in the LSTM (with the sigmoid activation function) it necessarily is. This might reduce the problem of catastrophic forgetting that you mentioned and thus scale better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DNC is also novel in the links that it uses between memory. However, this might be a more marginal improvement on the LSTM than it seems if we re-imagine the LSTM with complete neural networks for each gate instead of just a single layer with an activation function (call this a super-LSTM); in this case, we can actually learn any relationship between two slots in memory with a sufficiently powerful network. While I don't know the specifics of the links that DeepMind is suggesting, they imply in the article that they are learning everything just by backpropagating gradients like a regular neural network. Therefore whatever relationship they are encoding in their links should theoretically be learnable by a neural network, and so a sufficiently powerful 'super-LSTM' should be able to capture it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;With all that being said&lt;/strong&gt;, it is often the case in deep learning that two models with the same theoretical capability for expressiveness perform vastly different in practice. For example, consider that a recurrent network can be represented as a huge feed-forward network if we just unroll it. Similarly, the convolutional network isn't better than a vanilla neural network because it has some extra capacity for expressiveness; in fact, it is the constraints imposed on its weights that makes it &lt;em&gt;more&lt;/em&gt; effective. Thus comparing the expressiveness of two models isn't necessarily a fair comparison of their performance in practice, nor an accurate projection of how well they will scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One question I have about the DNC is what happens when it runs out of memory. When a classical computer runs out of memory and another block of memory is requested, programs start crashing (at best). I'm curious to see how DeepMind plans to address this. I assume it will rely on some intelligent cannibalization of memory currently in use. In some sense computers currently do this when an OS requests that applications free up non-critical memory if memory pressure reaches a certain threshold.&lt;/p&gt;&#xA;" OwnerUserId="8366" LastActivityDate="2017-07-10T16:42:20.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3620" PostTypeId="2" ParentId="3226" CreationDate="2017-07-10T16:46:33.417" Score="1" Body="&lt;p&gt;It seems that they are stating that a knowledge base is consistent if and only if it never asserts the truth of both the truth and the negation of a particular P. In other words, a knowledge base is consistent if it never contradicts itself. Their definition allows incomplete knowledge bases to be considered consistent; by their definition, an empty knowledge base is still considered a consistent one.&lt;/p&gt;&#xA;" OwnerUserId="8366" LastActivityDate="2017-07-10T16:46:33.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3621" PostTypeId="1" AcceptedAnswerId="3633" CreationDate="2017-07-10T17:46:06.227" Score="4" ViewCount="650" Body="&lt;p&gt;I am trying to build a ML Agent to find the closest matching image from a given set. The user will draw something and the agent should list the closest matching images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very similar to these examples&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sketchx.eecs.qmul.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sketchx.eecs.qmul.ac.uk/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Emoji search in Android keyboard&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One unique problem I've is, each image will represent a category. Imagine we have product images and user will draw something and we have to find the products close to the drawing. So category in my case will be product Id. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hHwig.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hHwig.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to evaluate the approach before trying out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lot of examples to classify images, however if I use the item identifier instead of category it should work. But am trying to find the best approach for this problem. &lt;/p&gt;&#xA;" OwnerUserId="8368" LastEditorUserId="-1" LastEditDate="2017-07-13T06:49:05.567" LastActivityDate="2017-07-13T06:49:05.567" Title="Building ML to finding the closest matching image based on users drawing" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3622" PostTypeId="2" ParentId="3226" CreationDate="2017-07-10T21:18:29.617" Score="0" Body="&lt;p&gt;I guess in this context &quot;known&quot; means nothing but either $P$ or $\neq P$ is in the KB; further, exactly one of these two needs to be in the KB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just think about what it means if $P$ and $\neg P$ are in the KB, then the KB is obviously inconsistent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if neither of these two sentences is in the KB, then no information at all can be retrieved from the KB and it remains unclear whether $P$ or $\neg P$ is supposed to be a true statement; thus $P$ is &quot;unknown&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if exactly one of these two is in the KB, then one has all information that one needs about $P$ (due to the excluded middle); $P$ is &quot;known&quot; and consistent.&lt;/p&gt;&#xA;" OwnerUserId="1781" LastActivityDate="2017-07-10T21:18:29.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3624" PostTypeId="2" ParentId="3573" CreationDate="2017-07-11T04:48:20.950" Score="2" Body="&lt;p&gt;I'm going to go out on a limb and suggest that this is a matter of evolution, that humans are in no way exceptional in the grand scheme, and that AGI will manifest so long as technology advances, because human consciousness is simply a matter of complexity of the system.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea comes out of emergent complexity in Conway's Game of Life. In Conway's words: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;There are Life patterns which behave like self-replicating animals… It’s probable, given a large enough Life space, initially in a random state, that after a long time, intelligent self-replicating animals will emerge and populate some parts of the space.” &lt;br&gt;&lt;sub&gt;Source: Winning Ways for Your Mathematical Plays&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I came across a paper &lt;a href=&quot;http://web.cecs.pdx.edu/~mm/ca-review.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computation in Cellular Automata:&#xA;A Selected Review&lt;/a&gt;, which I am still working my way through, and which you may find interesting.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For those who use philosophical arguments to make the case algorithmic consciousness is not possible, I'd posit the question &quot;how do we know we're conscious?&quot;, not because I'm interested in the answer, but merely to throw a wrench into that line of inquiry.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because ultimately it doesn't matter.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consciousness in the sense of human awareness is not a requirement of life, and the most basic definition of consciousness is awareness of any kind, no matter how trivial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the idea that there is something &quot;magical&quot; about human consciousness, that ideas are not things because they do not have material form, to be problematic.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intangibility I don't have a problem with, as intangibles clearly interact with the physical world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(As an analogy, I studied for many years with a famous Tai Chi teacher who never talked about &quot;chi&quot;.  I suspect this disinclination derived from the way in which the concept of &quot;chi&quot; leads to magical thinking, which is illusory as opposed to practical. The practice and application of Tai Chi techniques is purely a matter of physics and physiology, even when such applications seem to defy natural laws. Possibly there is something going on that we don't understand, but if that were the case, such phenomena are natural in origin.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know there is randomness in nature at the quantum level, and if this proves to be a component of human consciousness, we can use quantum computing to provide a medium for artificial consciousness. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-07-11T20:31:47.113" LastActivityDate="2017-07-11T20:31:47.113" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3625" PostTypeId="2" ParentId="3590" CreationDate="2017-07-11T05:26:54.143" Score="3" Body="&lt;p&gt;&lt;strong&gt;Why is Game Playing R&amp;amp;D a Focus of Resource Allocation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When examining the apparent obsession with game playing as researchers attempt to simulate portions of human problem solving abilities, the orthodoxy of the views of John McCarthy (1927 – 2011) may be misleading. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Publication editorial bias and popular science fiction themes may obscure the primary forces that lead to the appearance of obsession with developing winning board game software.  When examining the allocation of funds and human resources within the many fields of intelligence research and development, some historical background is necessary to circumvent distortions typical of answers to questions in this social net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Historical Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ability to place ourselves out of our own time and into the mindset of other periods is helpful when analyzing history, including scientific and technological history.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that McCarthy's vision was not orthodox in his time.  It quickly became orthodox because of an array of emerging trends in thought about automation among scientists and mathematicians in times immediately following western industrialization.  This thinking was the natural extension of the mechanization of the printing, textile, agriculture, and transportation industries and of war.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the mid-twentieth century, some of these trends combined to conceptualize the digital computer.  Others became orthodoxy within the community of people investigating aspects of intelligence via digital systems.  The technical backdrop included theoretical work and electro-mechanical work, some of which has since achieved a degree of public fame.  But it was generally either secret or too abstract (and therefore obscure) to be considered items of national security interest at the time.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cybernetics theory, largely developed by Norbert Wiener (1894 – 1964)&lt;/li&gt;&#xA;&lt;li&gt;The work done on automating arithmetic (extending George Boole's theory and Blaise Pascal's calculator, with primary funding originating from the U.S. military in an interest in guiding anti-aircraft weaponry by calculating probable trajectories of enemy of aircraft and determining spherical coordinates to create a probable interesting ballistic trajectory&lt;/li&gt;&#xA;&lt;li&gt;Often dismissed work of Alonso Church (1903 – 1995) on lambda calculus which led to the idea of functional programming, a key aspect to the emergence of LISP in Cambridge, which McCarthy leveraged for early AI experimentation&lt;/li&gt;&#xA;&lt;li&gt;The birth of information theory, primarily through the work of Claude Shannon (1916 – 2001), funded through Bell Labs in the interest of automating communications switching&lt;/li&gt;&#xA;&lt;li&gt;The early cryptanalysis work of Church's doctoral student, Alan Turing, funded entirely by Allied Forces with the R&amp;amp;D goal of defeating the Enigma cryptography device so that Nazi forces could be stopped prior to the complete annihilation of London and other Allied targets&lt;/li&gt;&#xA;&lt;li&gt;The work on John von Neumann (1903 – 1957) toward centralizing the implementation of arbitrary Boolean logic together with integer arithmetic into a single unit (currently called a CPU) and storing the program that controlled the implementation in electronic flip-flops along with the data to be processed and the results (the same general architecture imployed by almost all contemporary computing devices today)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these were concepts surrounding the vision of automata, the simulation of functional aspects of mammalian neurology.  (A monkey or elephant can successfully plan and execute the swatting of a fly, but a fly is incapable of planning and executing an attack on a monkey or elephant.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experimentation into intelligence and its simulation via symbolic manipulation using a new programming language, LISP, was a primary focus of John McCarthy and his role in the creation of the MIT AI Laboratory.  But whatever orthodoxy may have existed with rule based (production systems), neural nets, and genetic algorithms has largely diversified into a cloud of ideas that make the term orthodoxy somewhat nebulous.  A few examples follow.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Richard Stallman resigned from the MIT AI Lab and began a philosophical shift away from many of the economic philosophies that dominated that time period.  The result was GNU software and LINUX, followed by open hardware and creative commons, concepts largely opposed to the philosophic orientation of those that funded AI hotbeds.&lt;/li&gt;&#xA;&lt;li&gt;Many proprietary (and therefore company confidential) systems use Bayesian methods or adaptive components that stem more from Norbert Wiener's work than anything that was considered mainstream AI research in the 1970s.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Birth of Game Theory&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key event that answers the question most directly in this parade of historical events is some other work of von Neumann's.  His book Game Theory, coauthored with Oskar Morgenstern, is perhaps the strongest factor among the historical conditions that led to the persistence of Go and Chess as test scenarios for problem solving software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there were many earlier works on how to win in Chess or Go, never before was there a mathematical treatment and a presentation as compelling as that in Game Theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The privileged members of the scientific community were well aware of von Neumann's success with raising the temperature and pressure of fissile material to critical mass and his work in deriving classic thermodynamics from quantum theory.  The foundation of mathematics he presented in Game Theory was quickly accepted (by some of the same people that funded research at MIT) as a potential predictive tool for economics.  Predicting economics was the first step in controlling it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theory Meets Geopolitical Philosophy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dominant philosophy that drove western policy during that period was Manifest Destiny, essentially the fatalist view of a New World Order, the head of which would be in the seats of U.S. power.  Declassified documents indicate that it is highly likely that leaders of that time saw economic domination achieved through the application of game theory as considerably less risky and expensive than military conquest followed by the maintenance of bases of operations (high tech garrisons) near every populated area overseas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The highly publicized challenges to develop Chess and Go automatons are simply dragnets that corporations and governments use as a first cut in the acquisition of personnel assets.  The game results are like resumes.  A winning game playing program is a piece of evidence of the existence of programming skill that would likely also succeed in the development of more important games that move billions of dollars or win wars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those who can write winning Chess or Go code are considered high value assets.  Funding game playing research has been seen as a way of identifying those assets.  Even in the absence of immediate return on investment, the identification of these assets, because they can be tucked away in think tanks to plot out the domination of the world, have become a primary consideration when research funds are allocated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Slow and Fast Paths to Return on Investment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to this geopolitical thinking, seeking institutional prestige on the back of some crafty programmer or team is another factor.  In this scenario, any progress in simulating intelligence that has a potential of geometric improvements in some important industry or military application was sought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, programs like Maxima (a forerunner of mathematical problem solving applications such as Mathematica) were funded with the hope of developing mathematics using symbolic computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This path to success conceptually rested on determinism as an overarching natural philosophy.  In fact, it was the epitome of determinism.  It was proposed that, if a computer could not only do arithmetic but develop mathematical theorems of super-human complexity, models of human endeavors could be reduced to equations and solved.  The predictability for a wide variety of important economic, military, and political phenomena could then be used in decision making, permitting significant gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To the surprise of many, the success of Maxima and other mathematics programs was very limited in its positive impact on the ability to reliably predict economic and geopolitical events.  The emergence of Chaos Theory explained why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beating a human master with a program turned out to be within the reach of twentieth century R&amp;amp;D.  Use of software to experiment on various computer science approaches to winning a game was achievable and therefore more attractive for institutions as a way of gaining prestige, much like a winning basketball team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Let's Not Forget Discovery&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes appearances are in direct opposition to actuality.  The various above mentioned applications of thinking machines nave not been forgotten, and the expense in time and money required to simulate aspects of mammalian abilities will not loose funding to board game automaton development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technology is largely occupied with solving communications, military, geopolitical, economic, and financial problems that far exceed the complexity of games like Chess and Go.   Game theory includes elements of random moves made by non-players as far back as its inception.  Therefore, the obsession with Chess and Go is merely a signature of the actual focus of funding and activity in the many fields of simulating intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Software that can play a mean game of Chess or Go is deployed to neither NSA global modelling computers nor Google's indexing machinery.  The big dollars are spent to develop what IS deployed into such places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will never see details on or even an overview of that R&amp;amp;D described online, except in the case of people who, for some personally compelling reason, violate their company confidential agreements or commit treason.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-07-11T06:09:28.180" LastActivityDate="2017-07-11T06:09:28.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3626" PostTypeId="1" AcceptedAnswerId="3628" CreationDate="2017-07-11T13:08:43.253" Score="6" ViewCount="253" Body="&lt;p&gt;I am trying to perform a binary classification of tweets using machine learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual way of doing this seems to be putting a hand-classified tweet's words into a big vector, then use that vector as input to an algorithm, which then predicts new tweets based on this data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a standard method, or algorithm, that can include other input, such as the location of a tweet, in this process? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could just add tweet location at the end of the vector, I suppose, but that would give it a very small weighting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers are much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8385" LastEditorUserId="2444" LastEditDate="2019-06-14T22:58:39.117" LastActivityDate="2019-06-14T22:58:39.117" Title="How to add more features to the input of a machine learning algorithm?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3627" PostTypeId="1" CreationDate="2017-07-11T17:50:08.200" Score="3" ViewCount="60" Body="&lt;p&gt;You need a library to recognize the machine numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a choice between two libraries Keras and OpenCV&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is better to choose? Or is there an even better solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming language Python3&lt;/p&gt;&#xA;" OwnerUserId="8390" LastActivityDate="2017-07-14T15:23:43.023" Title="Selecting a library for recognition" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3628" PostTypeId="2" ParentId="3626" CreationDate="2017-07-11T18:42:37.540" Score="3" Body="&lt;p&gt;Data pre-processing and feature extraction are by far the most important part of any machine learning algorithm. It's even more important that the model you choose to do the classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, pre-processing and feature extraction are completely different for each type of data. You need to play around with the data yourself to find out what works best with the nature of your data. With experience you start noticing some patterns with different data types. For example, as you are doing, building a word vector is an effective means of feature extraction with text based data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;I could just add tweet location at the end of the vector I suppose, but that would give it a very small weighting.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is entirely untrue for any machine learning algorithm I would choose. Your model should not associate weights to the inputs based on their array location. They should be associated based on the explained variance (information gain) it provides. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;After you do your pre-processing and feature extraction you can then further refine your feature set with some common methods which can be found in libraries such as: principle component analysis (PCA) and linear discriminant analysis (LDA). &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-07-11T18:42:37.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3629" PostTypeId="1" CreationDate="2017-07-11T19:13:36.097" Score="3" ViewCount="610" Body="&lt;p&gt;I'm not quite sure how I should go about creating a multi-label image KNN classifier using python as a lot of the literature I have read does not explicitly explain this methodology. Specifically, I am not sure how I would be able to potentially yield multiple labels per image using the KNN classifier architecture. Any insight would be greatly appreciated! (new to coding by the way)&lt;/p&gt;&#xA;" OwnerUserId="8394" LastActivityDate="2017-07-18T18:58:50.047" Title="Multi-Label Image Classification using KNN" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3630" PostTypeId="1" CreationDate="2017-07-12T13:46:46.373" Score="3" ViewCount="122" Body="&lt;p&gt;I am currently in the pre-process of starting an image classification and extraction project which needs to output multiple softmax and absolute values from a single image like such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA; time: &quot;20:20&quot;, &#xA; teams: [&#xA;   {&#xA;     red: { goals: 2},&#xA;     blue: { goals: 1},&#xA;   },&#xA;   {&#xA;     scored_by : [{&#xA;      john: 80%, &#xA;      kyle: 51%, &#xA;      darren: 20%&#xA;     }&#xA;   ]}&#xA; ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can create multiple models which are responsible for different task such as reading the time from the image as well as the score and eventually combine both. I would however like to make sure I maximize on efficiency to make sure the process is as fast as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers in the right direction would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With kind regards,&#xA;Dennis&lt;/p&gt;&#xA;" OwnerUserId="8405" LastEditorUserId="8405" LastEditDate="2017-07-12T14:25:41.753" LastActivityDate="2017-12-05T17:34:04.613" Title="Extracting multiple softmax values from image" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3631" PostTypeId="2" ParentId="3630" CreationDate="2017-07-12T14:20:55.030" Score="2" Body="&lt;p&gt;It is not unheard of to share network weights with multiple output layers. I have seen it on DeepMind's &lt;a href=&quot;http://proceedings.mlr.press/v48/mniha16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Deep Learning&lt;/a&gt; paper, and I have also used it &lt;a href=&quot;https://www.dropbox.com/s/ssq2hqoihcks74h/17%20ROBOT.pdf?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to share all the layers and just have multiple outputs. However, this might decrease the accuracy of your networks, as is the usual performance VS accuracy trade-off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Y6oCD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y6oCD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To optimize this, just calculate the loss of both outputs and sum them when feeding the optimizer, like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            self.target_policy_fast_t = tf.placeholder('float32', [None, a_size], name='target_policy_fast_t')&#xA;            self.loss_policy_fast = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.network.policy_fast_before_softmax, labels=self.loss_policy_fast_t))&#xA;&#xA;            self.target_policy_slow_t = tf.placeholder('float32', [None, a_size], name='target_policy_slow_t')&#xA;            self.loss_policy_slow = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.network.policy_slow_before_softmax, labels=self.target_policy_slow_t))&#xA;&#xA;            self.loss = tf.reduce_mean(self.loss_policy_fast + self.loss_policy_slow, name='loss')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="7496" LastEditDate="2017-12-05T17:34:04.613" LastActivityDate="2017-12-05T17:34:04.613" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3632" PostTypeId="1" CreationDate="2017-07-12T16:49:42.553" Score="1" ViewCount="798" Body="&lt;p&gt;I have created 22 different Convolutional neural networks that all test for the presence of unique objects in an image (each one of the classifiers is unique). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each sample in the test set has the output of a 22-long vector that looks something like this [0, 1, 1, 0, 0, 1, ..., 1], the binary nature of the vector representing the presence/absence of specific objects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented this already in keras and reach around 97% accuracy avg for the 22 models. Is there any specific ensemble methods that can allow me to combine all 22 classifiers?&lt;/p&gt;&#xA;" OwnerUserId="7773" LastActivityDate="2017-07-13T14:17:32.933" Title="Ensemble Learning using Convolutional Neural Networks" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;keras&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3633" PostTypeId="2" ParentId="3621" CreationDate="2017-07-12T19:29:26.877" Score="2" Body="&lt;p&gt;There is a good project you can research to find information on this specific task, it is Google &lt;a href=&quot;https://quickdraw.withgoogle.com/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;quickdraw&lt;/a&gt;. It has 50 million drawings across 345 categories and it was used in Google autoDraw. AutoDraw guesses what you’re trying to draw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use quickdraw dataset to train a neural network (Sequence-to-Sequence Variational Autoencoder) in order to get draws from your Sketches. for more info on this read &lt;a href=&quot;https://arxiv.org/pdf/1704.03477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this papper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classified Sketches into categories is a difficult task because Sketches are a high-level representation that does not always convey enough information to distinguish between different categories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The the best approach will depend on your specific needs&lt;/p&gt;&#xA;" OwnerUserId="3721" LastActivityDate="2017-07-12T19:29:26.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3634" PostTypeId="2" ParentId="3632" CreationDate="2017-07-13T02:52:12.617" Score="2" Body="&lt;p&gt;I am new to AI but this is something I can think of. There might be other much better ways. Or even functions in scikit learn to do it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) create a list of all the 22 models &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) iterate over the models one by one and use model.predict() for each model and store the hotencoded output to another list or numpy array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Take average of the output list or numpy array. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) since the output is a vector of 0,1 it is possible to get decimals in the output after taking avg. Round it off using the basic rules to either 0 or 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) If your CNNs are very deep like inception v3, densenet etc. You might run into memory issues while loading all the 22 model's weights​ into the memory at once. So you can iteratively load the models in small fixed batches and use model.predict() on each model in the batch append the outputs to a list. and then clear them off the memory before loading another batch of models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just my idea. I might be completely wrong also.&lt;/p&gt;&#xA;" OwnerUserId="8418" LastEditorUserId="8418" LastEditDate="2017-07-13T03:01:56.060" LastActivityDate="2017-07-13T03:01:56.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3635" PostTypeId="2" ParentId="3632" CreationDate="2017-07-13T05:36:36.917" Score="1" Body="&lt;p&gt;For this task, I think you shouldn't use 22 different networks. Use only one. The last layer of the network should be a fully connected with 22 unit, each unit represents the presence of a unique object. The activation function for this layer is sigmoid which output a real number between 0 and 1. Each of these outputs represents how confident the network about the presence of that unique object. &lt;/p&gt;&#xA;" OwnerUserId="6019" LastEditorUserId="6019" LastEditDate="2017-07-13T13:27:39.147" LastActivityDate="2017-07-13T13:27:39.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3637" PostTypeId="1" AcceptedAnswerId="3638" CreationDate="2017-07-13T13:47:30.077" Score="1" ViewCount="37" Body="&lt;p&gt;I have not seen this explicitly stated anywhere so I was curious. Say I have network trained to meet my segmentation needs using 250x250 images. After this training is complete and I wish to submit images in production for segmentation, do those production submitted images need to be 250x250 as well or can they be any reasonable size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they must be resized to 250x250 for segmentation, is it possible to scale up the segmentation regions to apply to a larger image? If so what is the name of that technique so I can research it a bit more.&lt;/p&gt;&#xA;" OwnerUserId="8430" LastActivityDate="2017-07-13T14:02:25.767" Title="Do images submitted to a segmentation network after training need to be the same size as the training images?" Tags="&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3638" PostTypeId="2" ParentId="3637" CreationDate="2017-07-13T14:02:25.767" Score="1" Body="&lt;p&gt;Yes, you will need data that has the same dimensionality as your training set. Otherwise the model will either discard excess data, or will supplement missing data with a random number. This will likely lead to very poor results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest to rescale your images to have the same pixel density as those you used for training (250x250) rather than scaling your model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods for rescaling images and maintaining details are very well developed. Many exist and you can use existing tools such as Photoshop to change the pixel density of an image. &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-07-13T14:02:25.767" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3639" PostTypeId="2" ParentId="3632" CreationDate="2017-07-13T14:17:32.933" Score="2" Body="&lt;p&gt;I think it is not a very good idea because Im pretty sure you used for learning all these 22 CNN same images and even same way for giving them a batches of images. So basically in a result you would have almost the same 22 classifiers.&lt;/p&gt;&#xA;" OwnerUserId="8431" LastActivityDate="2017-07-13T14:17:32.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3640" PostTypeId="1" AcceptedAnswerId="3644" CreationDate="2017-07-13T18:10:42.853" Score="7" ViewCount="3284" Body="&lt;p&gt;I can't seem to understand how an AI learns. Without having a programmer tell it what to do, how would a program create or generate some solution to a problem and then use information gained in future problems? I understand how chess AI works. But what's really confusing is how an AI would improve or learn.&lt;/p&gt;&#xA;" OwnerUserId="8433" LastEditorUserId="75" LastEditDate="2017-07-14T20:24:19.000" LastActivityDate="2018-01-12T08:21:31.277" Title="How does an AI learn?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="3643" PostTypeId="2" ParentId="3402" CreationDate="2017-07-14T01:29:41.967" Score="2" Body="&lt;p&gt;In the time since I asked this question, I have been able to combine Tensorflow and Chainer considerably well. That being said, one should try to avoid combining deep learning frameworks if one can for a few reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It doubles the amount of documentation one needs to reference &lt;/li&gt;&#xA;&lt;li&gt;It makes it difficult for new developers to become familiar with the&#xA;code base. &lt;/li&gt;&#xA;&lt;li&gt;It creates multiple data transfer bottlenecks when one&#xA;has to transfer data between the deep learning different&#xA;configurations such as: CPU -&gt; GPU (Tensorflow) -&gt; CPU -&gt; GPU&#xA;(Chainer) -&gt; CPU ... etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That being said, I combined them because of time constraints on a research project which would not be finding its way into an industrial product. In this research setting, I still wouldn't recommend combining frameworks if one can avoid it but if it means doubling the time it takes to do the project for a proof of concept or experiment, then it seems reasonable and can drastically speed up development until the missing functionality is added to the desired framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 issues with integration:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Abstracting the networks so that the main&#xA;training/testing script does not depend on the framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is important in the event that the missing functionality of your desired framework is added. It also helps new developers understand what the overall algorithm or experiment is without being bogged down by framework details.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tensorflow and similar frameworks which require a Session object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chainer, PyTorch (which came from Chainer), and any other framework that doesn't require a something like Tensorflow's Session all play very nicely together and can be used (almost) interchangeably. For Session based frameworks, the main script/loop/experiment must all take place within a Session. With proper abstraction, my experiment only required a &lt;code&gt;with tf.Session() as sess:&lt;/code&gt; to be used in the main experiment and was the only reference that Tensorflow was being used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although I still don't recommend combining frameworks for the reasons specified above, if one abstracts their framework calls enough, it is almost seamless to integrate deep learning frameworks.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-07-14T01:29:41.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3644" PostTypeId="2" ParentId="3640" CreationDate="2017-07-14T08:39:48.173" Score="7" Body="&lt;p&gt;There is no strong evidence that a single mechanism of learning exists in the neural networks of the human brain.  The scientifically collected evidence does not support that simplistic conception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning is an umbrella term under which many things reside.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Accumulation of rules triggered by corresponding sets of conditions&lt;/li&gt;&#xA;&lt;li&gt;Pattern recognition, such as the recognition of the probable source of a knock or a saxophone note&lt;/li&gt;&#xA;&lt;li&gt;Connection between intention, motor output, and sensory feedback, backed by pattern recognition, as with hand-eye coordination&lt;/li&gt;&#xA;&lt;li&gt;Construction and refinement of internal models of subjective reality that can be called upon to make choices or determine quantities&lt;/li&gt;&#xA;&lt;li&gt;Construction and refinement of external models of objective reality, such as Newton's recognition that Kepler's description of elliptical orbits, Aristotle's attraction of objects to the ground, and the constant rate of velocity increase in falling objects were manifestations of the same phenomenon, thus the simultaneous invention of the foundations of both calculus and physics&lt;/li&gt;&#xA;&lt;li&gt;Dreams during which insights can be gained, often of a seemingly illogical nature, yet reportedly useful in life goal attainment&lt;/li&gt;&#xA;&lt;li&gt;Sleep in between dreams where it is possibly that the day's queued observations are integrated with existing learned material or otherwise assimilated&lt;/li&gt;&#xA;&lt;li&gt;Largely creative or intuitive, where models and rules are combined to create hypotheses to be tested through experimentation or through long periods of in situ observation (observation in natural environment)&lt;/li&gt;&#xA;&lt;li&gt;Subconscious and related to archetypes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are several approach to simulating these partially understood learning mechanisms in computers. It is also important to note that not all machine learning is a simulation of a mechanism of the human nervous system.  What arrangements of architectures, algorithms, parallelism, and interconnections optimally learn depend on many things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The nature of the things to be learned (tennis, calculus, how to recognize a zip code in a mail sorter, someone's voice, someone's retina or fingerprint, how to identify social or story structure in a drama, self-improving real time trajectory prediction and antiaircraft aiming)&lt;/li&gt;&#xA;&lt;li&gt;The mechanism of input and output (tabular, natural language, sound, visual, tactile, stepper motor signals)&lt;/li&gt;&#xA;&lt;li&gt;The balance of the dimensions of the quality of intelligence to which the term optimal can be applied (accuracy, reliability, range of conditions for which learning is functional at all, relationship between complexity and speed of response capacity for complexity, depth of abstraction, wit in presentation of conclusions, listing of reference, auditability of thinking)&lt;/li&gt;&#xA;&lt;li&gt;Hardware running the algorithms, their processes, and interconnecting them&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How Learning Presumably Occurs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current scientific theory (which is subject to change as more discovery occurs and paradigms shift) is that the human brain is programmed by the portions of DNA that define neural structures.  It is theorized that structural changes inside of and between neurons persist memories and establish pathways that lead to adaptations beyond instinctive neural activity and organism behavior, which is part of the DNA programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Organic or Digital Circuit Based Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For learning to occur in either a meat or silicon based system, there are a few requirements, the listing of which will give a general view into the answer to the question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Way for the system to understand what is expected of it (real time acceptability/unacceptability/quality feedback, built in goal achievement detection, crisis detection, or multiple cases of conditions and expected results)&lt;/li&gt;&#xA;&lt;li&gt;Way for the system to persist information needed to hold a representation of what had thus far been learned&lt;/li&gt;&#xA;&lt;li&gt;Way for the system to acquire input or sense conditions&lt;/li&gt;&#xA;&lt;li&gt;Way for the system to output results, execute decisions made, or impact conditions&lt;/li&gt;&#xA;&lt;li&gt;Way to load algorithms into processes, interconnect them, and manage the overall system throughput&lt;/li&gt;&#xA;&lt;li&gt;Hardware to support the above&lt;/li&gt;&#xA;&lt;li&gt;Optionally, typical enterprise type features such as crash/disaster recovery, the ability to independently handle concurrent learning processes on a single system, or the ability to scale the system to handle larger problems or more concurrent learning processes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of the above is dismissing entirely the possibility of concepts such as synchronicity, the metaphysical, or deity, none of which have been scientifically disproven.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few code bases that may help begin an investigation into learning using some of the most common and publicly available approaches.  Some of these have evolved to learn for specific problem sets.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mnielsen/neural-networks-and-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mnielsen/neural-networks-and-deep-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/kootenpv/neural_complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kootenpv/neural_complete&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/rougier/neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rougier/neural-networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/jcjohnson/neural-style&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jcjohnson/neural-style&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/uncomplicate/bayadera&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/uncomplicate/bayadera&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/timnugent/naive-bayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/timnugent/naive-bayes&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/cpp-fuzzy-logic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/cpp-fuzzy-logic/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.codeproject.com/Articles/316668/Cplusplus-Fuzzy-Logic-API-plus-Simple-DSL&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.codeproject.com/Articles/316668/Cplusplus-Fuzzy-Logic-API-plus-Simple-DSL&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/abudnik/tcalc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/abudnik/tcalc&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/metagol/metagol&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/metagol/metagol&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/clipsrules/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/clipsrules/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://openrules.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://openrules.com/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.drools.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.drools.org/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Truly Learning About Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than running example code, it may be best to commit to actual learning about learning.  Downloading, compiling, configuring and running some existing POC is not your proof of concept.  It may be your starting point, but until you change those POCs to perform some other task you've devised and for which you've located sufficiently realistic data, you haven't learned anything about learning yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an example of a self-constructed lab project that wades right into the center of learning about learning software.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Develop an object oriented model of a tic-tac-toe game&lt;/li&gt;&#xA;&lt;li&gt;Develop classes to represent the possible moves available for two opponents&lt;/li&gt;&#xA;&lt;li&gt;Add an object oriented model of HOW the next move could be decided based on some arbitrary logic that can be mutated or permuted&lt;/li&gt;&#xA;&lt;li&gt;Create some mechanism to try various HOWs in actual play&lt;/li&gt;&#xA;&lt;li&gt;Create an algorithm that processes as input win-loose statistics and the&#xA;corresponding HOWs that converges on the optimal HOW for winning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Representing the tic-tac-toe three by three grid and the possible moves that players can make may only require a few dozen lines of code, but the simulation of human learning of the game (or some other form of learning the game that humans might not actually do) is not simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even for the limited problem set of tic-tac-toe games, to write a program that actually learns the always-win-if-winning-is-possible algorithm by actually learning tic-tac-toe by playing it is complex and raises questions about random versus educated guessing worth investigation and about which many articles and books may be found and studied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing such a program that learn successful tic-tac-toe play without knowing how to win in advance may take some brilliant thinking and days of programming, testing, experimenting, and re-thinking.  However, that will reveal more than reading articles, posts, books, or library documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing a program that would deduce how to play tic-tac-toe from the rules of the game represented as predicate logic is a longer term project but may be rewarding and quite educational.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-01-12T08:21:31.277" LastActivityDate="2018-01-12T08:21:31.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3646" PostTypeId="2" ParentId="3627" CreationDate="2017-07-14T15:02:01.030" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://chunml.github.io/ChunML.github.io/project/Real-Time-Object-Recognition-part-one/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This two-part article&lt;/a&gt; by Trung Tran about Real Time Object Recognition demonstrates differences between Keras and OpenCV from a coding point of view is a reasonable treatment.  It also covers implementing pre-trained VGG16 models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The application in &lt;a href=&quot;https://campushippo.com/lessons/how-to-train-a-self-driving-car-with-keras-and-video-game-data-214da3d47&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Campus Hippo lesson&lt;/a&gt; uses both Keras and OpenCV to simulate a self-driving car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.pyimagesearch.com/2016/07/18/installing-keras-for-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This tutorial&lt;/a&gt; by Adrian Rosebrockis about installing Keras for deep learning contains some good installation information to get both Keras and OpenCV running in the same virtual environment, which may be useful, especially for initial experimentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being a user of Python3, I never limit my options to Python3 libraries.  Python is used as a scripting language in place of Bash scripts in operating systems today because it can call Javac, C or C++ executables, or any other library or executable and pass values, arrays, or matrices back and forth with ease and simplicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding choice, you may wish to define what &lt;strong&gt;&lt;em&gt;better&lt;/em&gt;&lt;/strong&gt; means in your context.  If you wished to quantify or at least qualitatively describe acceptance or comparison criteria, there are a few typical dimensions to consider in your selection process.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Time to the provision of an answer&lt;/li&gt;&#xA;&lt;li&gt;Accuracy (error margin, which may not apply in this case)&lt;/li&gt;&#xA;&lt;li&gt;Reliability (pass/fail)&lt;/li&gt;&#xA;&lt;li&gt;Minimum required preparation of images to achieve accuracy or reliability&lt;/li&gt;&#xA;&lt;li&gt;Maintainability of the software that calls the library or framework&lt;/li&gt;&#xA;&lt;li&gt;Current active participation in maintaining the library or framework&lt;/li&gt;&#xA;&lt;li&gt;Projected future active maintenance based on trend of use and development&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Tangential Comment:  For clarity, you may wish to specify more information about the machine numbers in the question.  We can only guess whether you mean some numbers rasterized using some font and kerning rules.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-07-14T15:23:43.023" LastActivityDate="2017-07-14T15:23:43.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3647" PostTypeId="1" CreationDate="2017-07-14T16:08:58.420" Score="5" ViewCount="256" Body="&lt;p&gt;Is anyone able to recommend some resources (preferably books) on the topic of neural networks that goes beyond that of introductory reading?&lt;/p&gt;&#xA;&lt;p&gt;I'm still relatively new to the subject, however, I have successfully created my own neural network, so I wouldn't consider myself a beginner, so I'm looking for something more intermediate.&lt;/p&gt;&#xA;" OwnerUserId="6221" LastEditorUserId="2444" LastEditDate="2021-01-17T19:33:46.570" LastActivityDate="2021-01-17T19:33:46.570" Title="What are some intermediate or advanced books on neural networks?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;resource-request&gt;&lt;books&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3649" PostTypeId="2" ParentId="3233" CreationDate="2017-07-14T21:04:19.417" Score="1" Body="&lt;p&gt;&lt;strong&gt;Reviewing the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are multiple questions contained within this posted question.  (One of the sentences end with a period, but it is clearly intended to be a question.)  All are good questions and fairly easy to answer, assuming that the word 'replicate' can be replaced with 'model' or 'simulate'.  (Because the financial world is chaotic, any meaningful replication would likely require a quantum level reproduction of earth and everyone and everything on it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kind of modelling, analysis, and visualization of results is done all the time in the research we do.  Approaches and proof of concepts have been provided to insurance, banking, and health organizations along these lines and can be discussed here in general terms within the constraints of any confidentiality agreements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Restating the Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is best if I restate the questions the way I understand them from the information available in the original post to ensure I understand what the questioner wishes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the current trends in modelling and the production of predictive results from historical market data?&lt;/li&gt;&#xA;&lt;li&gt;To what degree can the complexity of financial landscape be simulated?&lt;/li&gt;&#xA;&lt;li&gt;What are the best approaches to create a graph that represent aspects of the financial market given a list of tradable securities?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please indicate if my restatement distorts any of the intent contained in the original three questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A graph comprised of vertices and edges commonly represents relationships between legal entities.  Applying this visualization to represent probable relationships between legal entities from matrices of historical market data is quite possible and may have many uses for financial analysis.  Such a graph can be visualized using GraphViz, Mathematica, Matlab, or various libraries available for use from programming environments of Python, C++, Java, LISP, JavaScript or other languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Vertices&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of vertices representing legal entities registered as tax entities, as in many of the web services that display graphs from public records and purchased aggregated corporate data, the vertices in the graph presumably envisioned by the questioner would represent tradable securities.  The attributes of such vertices might be.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Exchange&lt;/li&gt;&#xA;&lt;li&gt;Unique exchange ID (symbol)&lt;/li&gt;&#xA;&lt;li&gt;Name&lt;/li&gt;&#xA;&lt;li&gt;An array of vectors of historical trade metrics (with each vector containing a UTC time stamp)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edges&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edges represent the probable strength of financial connectedness between any two vertices representing two tradable securities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the nature of relationships and the associated details between the corporations offering tradable securities and the mindsets of all the trading agents are obscured, relationships must be inferred naively (without factual knowledge of causality), perhaps using the probability relations of Rev. Thomas Bayes (1701 – 1761) or other more sophisticated methods (some of which cannot, for legal reasons, be detailed here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relational models must be created (likely more than one) to capitalize on identifiable features in the trading metrics of one of two selected vertices and match that feature with the same or another feature in the other of the two vertices.  The correlation must be statistical and designed in such a way as to be resistant to effects outside the relationship between the two legal entities associated with the two tradable securities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naive Bayesian classification, other statistical approaches, FFTs, or neural nets may apply to assist with achieving a functional correlation value.  Windowing the data in a loop will be necessary to implement sensitivity to single events sparsely spaced in the time domain of the historical data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To attempt to guess causality, you will need to apply different temporal shifts to see if the feature of one preceded the feature of the other and by how much.  (If event A in security B preceded event C in security D, and this pattern repeats over a range of months or years, then there is a probability greater than zero that event C was caused by event A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The science (and perhaps the art) of creating a set of potential mathematical models of how various corporate and trading relationships may have impacted historical trade metrics between any two securities is the first hurdle in this proposed best approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using various known methods, a probability distribution of the single dimensional or multi-dimensional strength of the relationship, for each of the proposed models, can be calculated from the historical data of the two entities between which one of the many inter-vertex analyses is occurring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statistics of these distributions would then be the attributes of the edge shown between two vertices.  For more intuitive usability, the following attributes would need to be available via point and click drill down for each edge and each model tried between the two tradable securities connected by the edge.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Median relationship strength&lt;/li&gt;&#xA;&lt;li&gt;Mean relationship strength&lt;/li&gt;&#xA;&lt;li&gt;Standard deviation of relationship &lt;/li&gt;&#xA;&lt;li&gt;Direction of causality&lt;/li&gt;&#xA;&lt;li&gt;Median delay in causality&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Measures to Make Computation Time Practical&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To accomplish the above each vertex would ideally be compared with each other vertex, for each model, iterating through temporal parameters of the model to determine relationships involving time delays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there were a hundred tradables to consider, ten probabilistic relationship models, a thousand temporal permutations that must be tried to converge on a good fit between each model and the historical data, a hundred iterations to converge for each temporal window, a window of a thousand temporal observations, ten thousand windows to cover the entire range of historical data, and a thousand cycles for each test of fit, the primary computations would be 100 x 99 x 10 x 1000 x 100 x 1000 x 10,000 x 1000 = 99 x 10^18 CPU cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The number 99 comes from the fact that, without some permutation elimination scheme, the histories of each of 100 vertices must be compared with those of the other 99.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several methods may be applied together to reduce this set of expanded permutations to permit batch process completion after the close of the market in NY or Hong Kong and before the time zone dependent dawn.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Filtering and then decimating (removing redundancy) the historical data&lt;/li&gt;&#xA;&lt;li&gt;Truncating the historical data to analyze only the recent (and therefore the most relevant) historical data&lt;/li&gt;&#xA;&lt;li&gt;Widening the error margin to only what is displayed (such as two significant figures)&lt;/li&gt;&#xA;&lt;li&gt;Optimizations of algorithms, the mathematics behind them, the machine instruction representation of computations, or the mapping of values to data types&lt;/li&gt;&#xA;&lt;li&gt;Distributing analysis processes to take advantage of parallel computing&lt;/li&gt;&#xA;&lt;li&gt;Limiting the list of tradable securities using a narrowing set of inclusion criteria&lt;/li&gt;&#xA;&lt;li&gt;Early elimination of possible edges between unlikely relational candidates using heuristics, model simplifications, neural nets, or fuzzy logic&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once models are generated and functional and some of the visualized constructs can be verified, then the relational models can be used to predict probable events before they occur.  This may seem like science fiction to some, but we predict physical, social, and economic events all the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the profitability in relation to markets, if such predictive tools were to be distributed to eleven other traders, the ability to use the tool to generate profit would almost immediately deteriorate to one twelfth in monetary value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, this is probably the state of the market today.  Only those with automated tools are probable winners, funneling money from those without tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The AI Research Perspective&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the above does not seem like AI the way it is described, often what is conceived as an intelligent agent and appears intelligent in behavior after deployed, refined, and tuned, appears like straight software engineering when one gets into the details of implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, if the method for interfacing with the models used to match features in the history of two tradables is generalized so that arbitrary models can be added or modified at will without damaging the effectiveness of job execution, one can build some sort of analogy of a genetic algorithm to search for models that exhibit higher correlations and therefore progressively enhance predictive capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Meta Modelling&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point in development, model development is still largely up to the researcher.  However, once a model interface, perhaps employing the bridge and facade design patterns, is developed, it is possible to generalize the concept of historical feature correlation between two tradables as models with a set of mutation operations and develop concurrent processes that employ an automated experimental test fixture to develop new models without programmer intervention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the details of such meta-modelling cannot, for legal reasons, be detailed here, the meta-model design options naturally become apparent after some experience is gained after implementing and deploying the above approach in a real scenario with actual tradable historical data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Using Off the Shelf Code, Libraries, and Frameworks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, there is appreciable monetary value to this type of development, therefore it is unlikely that anyone will post (or even sell) code specific to this domain.  However using super-computing platforms, basic analysis algorithms such as FFT functions, and statistics packages with correlation coefficient routines, naive Bayesian capabilities, and convergence detection support will certainly assist in reducing the development effort required to implement and test this approach or others like it.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-07-15T06:10:36.853" LastActivityDate="2017-07-15T06:10:36.853" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3652" PostTypeId="2" ParentId="3647" CreationDate="2017-07-15T04:37:41.740" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://hagan.okstate.edu/nnd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Network Design (2nd edition)&lt;/a&gt; by Hagan et al. is one resource you could look at. It's a huge tome, weighing in at over 1000 pages in &lt;a href=&quot;http://hagan.okstate.edu/NNDesign.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf form&lt;/a&gt;, but it is freely available (you can also buy a dead-tree version if you really want one).&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2020-10-08T12:14:10.837" LastActivityDate="2020-10-08T12:14:10.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3654" PostTypeId="2" ParentId="3138" CreationDate="2017-07-15T08:44:40.587" Score="5" Body="&lt;p&gt;&lt;strong&gt;Will computers be able to understand user emotions?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;Understand&lt;/em&gt; is multidimensional, so characterizing the degree of understanding &amp;mdash; emotional or otherwise &amp;mdash; is a slippery task.  Nonetheless, some forms of AI emotional understanding are possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Interesting Simple Case&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even the embedded programs of velocity sensitive musical keyboards are emotionally aware at a primitive level.  Using the key depression velocity to drive the note parameters of attack, timbre, and volume allows the accomplished user's emotions to pass through the software into the emotional content of the music, which can be experienced by listeners just as with instruments that are not digital in nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More Advanced Emotional Capabilities&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the other extreme is a wide array of human emotional capabilities.  One example is the set of complex recognition functions underneath the typical listening skill set of a counselling professional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;affective computing&lt;/em&gt; was mentioned in the question.  A counselor may note the affect of the counselled, reading tone of voice, facial expression sequences, and body language.  From this, a clearer picture of the internal emotional state of the counselled can be understood.  These recognition and analysis abilities can determine the subject's internal emotional state more accurately and comprehensively than a masterful analysis of a functional MRI or the detection of neuro-chemical metabolites in blood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be no theoretical principle that limits computers from mastering the front end of a counselor's skill set, the recognition of emotion to produce a coded sequence of affects.  The degree to which software can interpret the sequence of affects (in combination with concurrent natural language expressions) and determine the emotional condition (and potential market-oriented decision patterns) of the human is yet to be determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other emotional capabilities beyond the skills of emotional recognition and interpretation.  Compassion is largely the logical integration of ethics in the absence of self-centered motive with the results of emotional recognition and interpretation.  The addition of emotional imitation yields empathy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The development of such capabilities in automata is more important than the shallow and therefore myopic motives of e-Commerce.  There may be applications for toys, education, entertainment, and remedy for the growing coldness of a technological society.  (Such is discussed further below.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Functions of recognition, interpretation, the application of rules or meta-rules in the ethical domain to these interpretations, and imitation may well be within the capabilities of digital systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isaac Asimov introduced the possibility of telepathy to the feature set of both the human mind and automata.  It may seem like telepathy is confined to the domain of fiction, however an emoticon can be the tele-form of a component of affect, such as a smile.  In this way, consensual technology assisted telepathy has actually become commonplace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the erosion of privacy in a culture with increasing interaction with digital systems by an increasing segment of the public, less consensual telepathic techniques may be developed and mastered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deeper Aspects of the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The doubt lies in the existence of a soul or some other autonomous and non-deterministic aspect of the human person.  The existence of these things and whether the simulation of them can be accomplished with a Turing machine and practically deployed to the Von Neumann architecture or a collection of them have neither been proven or dis-proven formally.  If autonomous and non-deterministic elements exist in people, then, even if they can be simulated, we cannot infur they can be realized as independently autonomous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Challenging the implications of the MIT coined term &lt;em&gt;Meat Machine&lt;/em&gt;, the notion that humans are capable of intention by fiat is not disproven, yet a pure Turing machine clearly cannot intend in that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is important in the context of the question in that a computer may only be able to simulate frustration when an intention is thwarted by the conditions of life.  Actual frustration may not be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Current Publicly Experienced Progress&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting entry point of computer science into the domain of human emotions is the story based memory and reasoning model purported by Roger Schank of Yale U in the 1990s.  Although the machinery used by Amazon underneath the hold of its ever improving ability to recommend movies in a buyer specific way is Company Confidential to Amazon, one must wonder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can e-Commerce develop user profiling regarding books and movies to comprehend the story plot or the arc of protagonist's emotional development? Can it do so sufficiently to match user viewing and purchase with other products?  Is that what Amazon is beginning to do?  Short of an unethical Amazon employee, we can only guess.  Certainly, what is transpiring in leading edge e-Commerce has moved beyond word or phrase matching or tracking interest in particular authors, screenwriters, directors, producers, and stars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Roger Schank's story based memory and reasoning has penetrated into e-Commerse, emotional analysis of individuals of within public is well underway, since story plot and protagonist developmental arcs are bound in climaxes and setting up climaxes, all related to emotional states.  If not, emotional analysis of groups within the public has been underway since public relations functions entered into the roles of enterprise IT systems.  (I know for a fact that such occurred decades ago.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if the likes of Facebook, Google, and Amazon are attempting to approximate in various ways the emotional states of individuals in real time, the matching of movie plots of protagonist arcs and such pattern matching and naive categorization is far from what a good counselor or a self-actualized friend or family member might do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer systems to which the public is exposed do not yet appear to accurately developing comprehension of the buyer's emotion.  That is one area where even the most progressive interfaces are still shy.  Furthermore, the computer interface itself is still as dry as it was in the late 20th century.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short of breaches of confidentiality, what capabilities to recognize, compare, analyze, or simulate emotions may exist in proprietary labs is solely a mater of conjecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Warm Interfaces&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Returning to the question of the general coldness of cybernetic interfaces in a technological society, there is much for the computer science community to learn.  For one thing, the soft skills of actual technical people often leaves much to be desired, so a cultural shift from cold development teams to culturally progressive and warm teams might be necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, one who has engaged with various PaaS (personality as a service) devices such as Siri, Google Assistant, Alexa, Cortana, and Microsoft's Office Assistant, Clippy, might find the paperclip personality of Clippy to be the warmer from among the list.  Perhaps the Clippy didn't see a higher level of acceptance because of the disparity between the coldness of a grid of cells in a spreadsheet and the attempted warmth of the animated character was too much, and the little guy tends to get in the way of some portion of the interface, which can be annoying.  (Integration of concepts was poor.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, the animated style of Clippy is intended to create a mental association with cartoon characters, which is brilliant.  Cartoon character design and storytelling through comic and cartoon media is a creative area of technology that had mastered computer generated warmth long before intelligent assistants began to appear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can't help thinking, upon general survey of intelligent assistant history, that the more the system tries to be human, the less human it seems.  Human response to emotional content in a cybernetic interface is like that of color matching or synthesized musical instruments.  If one is attempting to synthesize, the synthesis has to be good.  If it can't be fully convincing, then there is more authenticity in inventing something that is not pretending to be something real.  A caricature of real life objects (like Sponge Bob or Road Runner) avoids exposing an unacceptable margin of error in the imitation of real life. &lt;sup&gt;[1]&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our sensitivity in this area is so acute that if a real person, working in a customer service department, acts too mechanically such that we have even the slightest doubt as to whether we are talking to a human being, the impression is negative and we want to hang up and look for the function we wish to initiate on the company's web site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is unclear whether the trend for people to become more like cogs in a machine or whether usage patterns will drive companies to design their user interfaces and the mechanisms behind them to exude something like hospitality or congeniality.  Although Jaques Ellul seems to be correct in his proposition that technology has long since been driving humanity rather than the other way around, the DNA nature of the human mind will perhaps force humanity to remain remarkably human.  Thus the evolution of cybernetic interface professionals may continue to seek the meaning of and simulation of warmth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problematic, controvertible, and somewhat bizarre pursuit of Turing Testing will probably endure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concerns About Emotional Modelling Uses&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether an emotional recognition, modelling, analysis, and comparison suite, obviously a powerful tool-set, is used for the betterment of humanity or its detriment is a question like that of any other powerful tool-set.  Mass psychology, nuclear science, statistics, and genetics have in common the potential for great good or great evil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of this question, broadcast or more targeted information dissemination can be characterized as propaganda, which may not necessarily be perilous.  Propaganda was used for women's suffrage and along the path of U.S. independence from Europe.  Elements of broadcast and targeted control have created much geopolitical trouble, a topic outside of the scope of this social network, but pertinent and somewhat obvious, such that further mention is unnecessary anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Individual profiling can be of great good too.  Such can be like a combination of the old library subject indexed Dewey Decimal cards in combination with a knowledgeable librarian when looking for information or items of entertainment or personal expression for borrowing, rental, or purchase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, profiling in combination with statistics can lead to the ability to obtain covert control over a mass of people, not just for the profit maximization of a company.  An appropriately self-deluded megalomaniac or a collection of them in some kind of secret society could use emotional modelling and comprehensive user profiling for a large sample of national or global population for the injection of emotionally powered systems of belief.  Such a person or group could theoretically succeed in covert and systematic sublimation of liberty to a kind of virtual authoritarian dictatorship.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How technology is used is always a concern.  Transparency and accountability are required features of a society.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as in the other areas of science and technology mentioned above, it is up to we who are skilled enough to realize these systems to be intelligent and active elements within our society's accountability mechanisms.  Whether a corporation or governmental department is leading humanity into perdition is an item that can be judged during a job hunt and during the initial work period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is perhaps the strongest critique of the Turing's Imitation Game philosophy.  Cartoons, comics, and sci-fi creatures demonstrate that people don't seem to care if intelligent beings can be distinguished from humans, and there is no proof that human intelligence is the only or the best example of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along these same lines, heroes are representations of intelligent behavior that epitomize some aspect of humanity that actually doesn't naturally occur in pure form as they do with these protagonists.  Yet actors are branded and marketed as stars because of this idyllic expression of character.  It may be that humans don't really like human intelligence and constantly try to transcend it through fiction, but results in only some public awareness that is never more than an anthropological fantasy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even AI is not an imitation game.  It is a transcendence game.  For this reason, the Turing Test is of limited value.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-29T20:20:39.167" LastActivityDate="2018-12-29T20:20:39.167" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3656" PostTypeId="2" ParentId="221" CreationDate="2017-07-15T16:07:53.250" Score="3" Body="&lt;p&gt;&lt;strong&gt;Oligopoly vs Monopoly&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms in the question, multipolar and monolithic, appear to be referring to the micro-economic concepts of oligopoly and monopoly respectively.  Although these concepts are not AI specific, they certainly apply to such development in the way the question suggests.  Leading AI R&amp;amp;D is occurring in  a relatively small number of corporate, governmental, and university research labs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple search for &lt;em&gt;Oligopoly vs Monopoly&lt;/em&gt; will cover the differences between these two scenarios sufficiently for all sectors of products and services, and those concepts and observations apply to a large degree in the AI case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those who have not studied economics or political science might incorrectly conclude that oligopoly is necessarily related to capitalistic competition and monopoly is related to communism. However, R&amp;amp;D monopolies frequently exist within capitalistic economies and oligopolies frequently exist in a planned economies, such as communist states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Strengths of Each&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Research oligopoly strengths:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lead to diversity of approaches and target products and services&lt;/li&gt;&#xA;&lt;li&gt;Lower the probability of ethical, environmental, or social technology abuses because multiple independent teams, even with corporate confidentiality are in place, exert the force of at least a modicum of accountability&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Research monopoly strengths:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Efficient in terms of return on research investment for two reasons (1) the division of a fixed amount of available research funding between several labs can increase the R&amp;amp;D required to produce usable results, and (2) unfruitful approaches are more quickly abandoned when their workers can be shifted to other teams within the same organization&lt;/li&gt;&#xA;&lt;li&gt;Attract all those who have the most talent and experience in the fields needed by the specific R&amp;amp;D into one place to potentially collaborate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Predicting Transitions Between Monopolistic and 'Oligopolistic' Forms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The leading edge of any R&amp;amp;D tends to begin as a monopoly or a 'biopoly'.  Three or more units beginning the same research at the same time or being funded by government at the same time (as in planned economies) is unusual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over time, employees tend to leave and start their own organizations, more so in capitalistic societies, but also in planned economies, where government may hear cases to branch off and grant permission or the scientist may, in one way or another, seek relocation to a free economy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see these trends throughout technological history, back to early forms of writing, crop irrigation and plowing, chariot development, and ship building.  Modern examples include the application of the Westinghouse-Tesla model of electrical energy distribution and sales, genetic engineering, speech recognition, and HTTP clients (browsers starting with Mosaic, then Netscape, then IE, then Opera, Chrome, and others).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Unknowns&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How far ahead one global AI leader is than another cannot be so easily predicted.  Some companies choose to monetize R&amp;amp;D results sooner in the research cycle than others, depending on their strategy and the diversity of their revenue stream.  For instance, as of this writing, Apple and Facebook can wait a decade to release to the public manifestations of R&amp;amp;D without jeopardizing their financial status.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, governments may declassify material (another form of what the question calls time lag) several decades after discovery.  In the case of the Von Neumann model for achieving critical temperature and pressure of fissile or fusion materials, the government may never declassify it.  What the NSA or equivalent organizations outside the United States may decide must fall under an equivalent strategic wall of secrecy cannot be known outside of treasonous disclosure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The insight in an organization is not sustainable in practice.  Most people do not even know where first insights occurred and two whom they occurred first because the current icons in business related to the products and services stemming from the insight are otherwise completely unrelated to origins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few would buy a new car and thank Isaac Newton or even Henry Ford.  Absolutely no one buys a cell phone and thanks Alonso Church, Claude Shannon, or Gene Roddenberry when the first call or SMS message goes through.  We don't thank the water wheel researchers of Fourth Century Alexandria when we turn on a light, or even Nicola Tesla.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We do in some ways worship computer technology entrepreneurs when we buy an iPhone or require a document in docx format and vaguely understand that it is a MS Office document type.  Few know that not a single innovation we attribute to those entrepreneurs came from within their respective organizations.  Everything from personal computing, desktop publishing, touch screens, and windowing interfaces were developed in the R&amp;amp;D facilities of other corporations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some ways, the company to come up with the next insight is less likely to be the last one that did.  Innovators usually find better pay after they have a choice item on their resume like, &quot;Invented first adaptive cell tower switching protocol for mobile device communications.&quot;  That person is not only no longer working for the company that filed the corresponding patents, but is retired, and the company is now struggling and without recent productized innovation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Insurmountable Competitive Edges&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus far competitive edges have always become surmountable.  The proliferation of nuclear weaponry, now spanning at least seven countries, is a current and certainly important example of this.  German and Japanese aeronautics R&amp;amp;D edges are another.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2017-07-15T16:07:53.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3657" PostTypeId="1" AcceptedAnswerId="3658" CreationDate="2017-07-16T07:39:43.540" Score="4" ViewCount="1128" Body="&lt;p&gt;Both AI and Computer Science are Sciences, as I understood from Wikipedia, Computer Science is everything that has any relation to computers. And AI is commonly defined as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Study of machines that take the prerogative of humans (creating musical pieces e.t.c&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But recently, when I was reading, I read this sentence : &quot;In Computer Science, AI is [...]&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is really : Is there a part of AI studies that do not refer to Computer Science?&lt;/p&gt;&#xA;" OwnerUserId="8146" LastEditorUserId="8146" LastEditDate="2017-07-16T13:11:39.260" LastActivityDate="2017-07-19T14:46:40.940" Title="Is AI entirely a part of Computer Science?" Tags="&lt;terminology&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3658" PostTypeId="2" ParentId="3657" CreationDate="2017-07-16T09:00:23.933" Score="7" Body="&lt;p&gt;AI is an amalgamation of many fields, Computer Science plays a major role in imparting &amp;quot;Intelligence&amp;quot; to the machine. Following is a quote from the best selling AI book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; by&#xA;Stuart J. Russell and Peter Norvig.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Artificial Intelligence (AI) is a big field, and this is a big book.&#xA;We have tried to explore the full breadth of the field, which&#xA;encompasses logic, probability, and continuous mathematics;&#xA;perception, reasoning, learning, and action; and everything from&#xA;microelectronic devices to robotic planetary explorers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, the answer to your question is yes, there are other fields that AI depends including mathematics (to optimize AI algorithms), electronic components (sensors, microprocessors, etc.), mechanical actuators (hydraulic, pneumatic, electric, etc.)&lt;/p&gt;&#xA;&lt;p&gt;I highly recommend the book if you are looking for a starting point.&lt;/p&gt;&#xA;" OwnerUserId="3836" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-07-17T20:21:43.517" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3661" PostTypeId="2" ParentId="3657" CreationDate="2017-07-17T16:48:49.827" Score="3" Body="&lt;p&gt;It depends on your perspective. As a roboticist, I view A.I. as a discipline within Robotics, which itself is a discipline of Computer Science. The difficulty in definitively labelling these fields is partly due to the massive amounts of overlap. Take robotics for example, which combines the most advanced elements of mathematics, mechanical engineering, electronics, philosophy, neurology and many more. Do bear in mind that artificial intelligence need not be hosted on silicon (i.e. computer chips), it is entirely possible to have biological artificial intelligences, 'running' on living material. Look at the works of Professor Larry Bull from the University of the West of England (my university professor):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Bull, L. and Uroukov, I. (2008) Towards neuronal computing: simple creation of two logic functions in 3D cell cultures using multi-electrode arrays. International Journal of Unconventional Computing, 4 (2). pp. 143-154. ISSN 1548-7199 Available from: &lt;a href=&quot;http://eprints.uwe.ac.uk/20720&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eprints.uwe.ac.uk/20720&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They where able to demonstrate the logic functions AND and OR using living neurons, harvested from chickens. This type of A.I. research is so different from conventional computer science that it cannot be labelled under the same field&lt;/p&gt;&#xA;" OwnerUserId="8249" LastActivityDate="2017-07-17T16:48:49.827" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3662" PostTypeId="1" CreationDate="2017-07-17T18:24:08.817" Score="2" ViewCount="397" Body="&lt;p&gt;&lt;strong&gt;My datasets are not actual images, so using methods with ImageDataGenerator or pre-trained networks might not apply in this case.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data Structure: Each &quot;image&quot; is a 2048-long vector that has float values between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S3UfL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S3UfL.png&quot; alt=&quot;an &amp;quot;image&amp;quot; plotted&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each &quot;image&quot; was associated with a label (multi-label classifcation) and the goal is to perform classification via Keras 2D CNN's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are common techniques for finding which parts of the &quot;images&quot; contribute most to classification via convolutional neural nets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already implemented the CNNs in keras and have already successfully trained on my images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*No my data is not time series; however, my model works with either the keras Conv1D and Conv2D layers. &lt;/p&gt;&#xA;" OwnerUserId="7773" LastEditorUserId="3836" LastEditDate="2017-07-17T20:29:34.333" LastActivityDate="2017-07-17T20:29:34.333" Title="CNN attention maps on non-images" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3665" PostTypeId="1" CreationDate="2017-07-18T02:48:36.070" Score="10" ViewCount="197" Body="&lt;p&gt;Analogies are quite powerful in communication. They allow explaining complex concepts to people with no domain knowledge, just by mapping to a known domain. Hofstadter &lt;a href=&quot;https://cogsci.indiana.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;says they matter&lt;/a&gt;, whereas Dijkstra says they are dangerous. Anyway, analogies can be seen as a powerful way to transfer concepts in human communication (dare I say &lt;a href=&quot;https://en.wikipedia.org/wiki/Transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt;?).&lt;/p&gt;&#xA;&lt;p&gt;I am aware of legacy work, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Case-based_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Case-Based Reasoning&lt;/a&gt;, but no more recent work about the analogy mechanism in AI.&lt;/p&gt;&#xA;&lt;p&gt;Is there a consensus on whether or not analogy is necessary (or even critical) to AGIs, and how critical would they be?&lt;/p&gt;&#xA;&lt;p&gt;Please, consider backing your answers with concrete work or publications.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="2444" LastEditDate="2021-01-25T16:15:47.277" LastActivityDate="2021-01-25T16:15:47.277" Title="Is analogy necessary to artificial general intelligence?" Tags="&lt;ai-design&gt;&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3666" PostTypeId="1" CreationDate="2017-07-18T10:51:10.770" Score="4" ViewCount="580" Body="&lt;p&gt;I am trying to build a model which would take an email message (in English, extracted subject, and body of the email) and identify if it has a question, request or a proposal. Basically, I would like to see the mails that I've not replied but needs a reply. The model can be used as a &quot;filter&quot; in an email client.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to go about it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related Work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ParakweetLabs/EmailIntentDataSet/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parakweet Lab's Email Intent Data Set&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.cmu.edu/~tom/EMNLP2004_final.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Classify Email into &quot;Speech Acts&quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8511" LastEditorUserId="4302" LastEditDate="2018-10-08T12:26:10.890" LastActivityDate="2020-09-07T15:41:34.790" Title="Identify &quot;actionable&quot; intents in email messages" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3668" PostTypeId="1" CreationDate="2017-07-18T13:24:40.043" Score="1" ViewCount="1851" Body="&lt;p&gt;We have always known that gradient descent is a function of two or more variables. But how can we geometrically represent gradient descent if it is a function of only one variable?&lt;/p&gt;&#xA;" OwnerUserId="8514" LastActivityDate="2020-03-30T21:51:01.007" Title="How would 1D gradient descent look like?" Tags="&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3670" PostTypeId="2" ParentId="3668" CreationDate="2017-07-18T14:36:10.330" Score="0" Body="&lt;p&gt;&quot;The concept of a direction of fastest descent only makes sense in more than one dimension.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/a/180573&quot;&gt;https://math.stackexchange.com/a/180573&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8516" LastActivityDate="2017-07-18T14:36:10.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3674" PostTypeId="1" AcceptedAnswerId="3681" CreationDate="2017-07-18T18:32:59.653" Score="2" ViewCount="242" Body="&lt;p&gt;I read a lot about this, I understand how it work, but I would like the most simple example you can provide me, because I have no clue how I would make it in code. No matter the language( I would appreciate if it's derived from c), because I'm not here to copy-paste, and understand the essence.&lt;/p&gt;&#xA;" OwnerUserId="8497" LastActivityDate="2017-07-25T16:08:23.303" Title="Neural Network example" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3675" PostTypeId="2" ParentId="3629" CreationDate="2017-07-18T18:58:50.047" Score="1" Body="&lt;p&gt;Have you tried google searching? There are many results that come up for &quot;multi-label k nearest neighbor image classification&quot;. Here are the first couple of papers to get you started:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v25/chiang12/chiang12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proceedings.mlr.press/v25/chiang12/chiang12.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9501&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9501&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8527" LastActivityDate="2017-07-18T18:58:50.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3676" PostTypeId="2" ParentId="3674" CreationDate="2017-07-18T20:24:40.870" Score="4" Body="&lt;p&gt;I think &lt;a href=&quot;https://cognitivedemons.wordpress.com/2017/07/06/a-neural-network-in-10-lines-of-c-code/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt; is the best c-based neural net code for beginners. &lt;/p&gt;&#xA;" OwnerDisplayName="user8532" LastEditorUserId="7550" LastEditDate="2017-07-25T16:08:23.303" LastActivityDate="2017-07-25T16:08:23.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3677" PostTypeId="1" CreationDate="2017-07-19T14:21:29.437" Score="1" ViewCount="245" Body="&lt;p&gt;I am working to make my first trained model for image recognition, using the programming language R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I am attempting to make a &lt;strong&gt;function that takes a PNG-image as input, resizes it to 128x128 pixels, and converts it into a row-vector of numbers representing the colours in each pixel.&lt;/strong&gt; I then want to add this to a file that contains my training-set and labels (the image bank). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having some trouble with this, being a beginner in R, as the numerical values I get out from the following code does not make sense to me - often I get a large amount of 1's, and the pictures appear to be completely white if I try to export and look at the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any obvious mistake here? I assume it must be where I convert to numerical values. Alternatively, is there an alternative approach to this task that would be better? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;load_png_to_image_bank &amp;lt;- function(wd, input_file, label) { &#xA;&#xA;  # Remembers old working directory &#xA;  oldwd &amp;lt;- wd&#xA;&#xA;  # Sets input wd to file location&#xA;  setwd(wd)&#xA;&#xA;  # Loads imager package&#xA;  library(imager)&#xA;&#xA;  # Imports file and resizes to 128x128 pixels&#xA;  image &amp;lt;- load.image(input_file)&#xA;  image &amp;lt;- resize(image,128,128)&#xA;&#xA;  # Finds numerical values for each pixel&#xA;  rasterized_vector &amp;lt;- as.data.frame(as.vector(image))&#xA;  rasterized_vector &amp;lt;- as.data.frame(t(rasterized_vector))&#xA;&#xA;  # Adds label defined by input variable&#xA;  rasterized_vector &amp;lt;- as.data.frame((cbind(1, rasterized_vector)))&#xA;  names(rasterized_vector)[1] &amp;lt;- label&#xA;&#xA;  # Sets wd to predetermined place for image bank file&#xA;  setwd(&quot; ... &quot;)&#xA;&#xA;  # Writes vector to file&#xA;  write.table(rasterized_vector, &#xA;              output_file, &#xA;              row.names=F, &#xA;              na=&quot;NA&quot;,&#xA;              append = T, &#xA;              quote= FALSE, &#xA;              sep = &quot;;&quot;, &#xA;              col.names = F)&#xA;&#xA;  # Resets working directory to initial state&#xA;  setwd(oldwd)&#xA;&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8541" LastActivityDate="2017-07-19T14:21:29.437" Title="Converting pictures into numerical values" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3678" PostTypeId="1" CreationDate="2017-07-19T14:23:50.803" Score="1" ViewCount="32" Body="&lt;pre&gt;&lt;code&gt;def Interpretation_be(sentence):&#xA;&#xA;     be = ( 'be', 'been', 'is', 'was', 'are', 'were' )&#xA;     place = ('there', 'here')&#xA;     gender_identity = ('he', 'she')&#xA;     indicator =('it', 'that', 'this', 'these', 'those')&#xA;     exist =('exist')&#xA;     equal=('equal')&#xA;     condition=('condition', 'status', 'quality', 'state', 'situation' 'circumstance') &#xA;     number =('1', '2', '3', '4', '5')&#xA;     consequence=('consequence', 'result')&#xA;     to = ('to') &#xA;&#xA;&#xA;     if sentence.split()[1] in be:&#xA;          if sentence.split()[0] in place:&#xA;               print(sentence.replace(sentence.split()[1],'exists'))&#xA;          if sentence.split()[0] in gender_identity:&#xA;               print(sentence.replace(sentence.split()[1],'equal to'))&#xA;          if sentence.split()[0] in indicator:&#xA;               print(sentence.replace(sentence.split()[1],'equal to'))&#xA;&#xA;     if sentence.split()[1] in exist:&#xA;           if sentence.split()[0] in place:&#xA;               print(sentence.replace(sentence.split()[1],'is'))&#xA;&#xA;     if sentence.split()[1] in equal:&#xA;          if sentence.split()[0] in condition:&#xA;                  print(sentence.replace(sentence.split()[1],'is'))&#xA;          if sentence.split()[0] in number:&#xA;                  print(sentence.replace(sentence.split()[1],'is'))&#xA;          if sentence.split()[0] in consequence:&#xA;                  print(sentence.replace(sentence.split()[1],'is'))&#xA;&#xA;     if sentence.split()[2] in to:&#xA;           print(sentence.replace(sentence.split()[2],'toward'))&#xA;           ##need to discern between preoposition to and infinitive to##&#xA;&#xA;##result_check##&#xA;Interpretation_be('i have been to Spain')&#xA;Interpretation_be('there was a cat')&#xA;Interpretation_be('there is a woman')&#xA;Interpretation_be('he is a boy')&#xA;Interpretation_be('it is a cat')&#xA;Interpretation_be('there exist a hat')&#xA;Interpretation_be('situation equal to bad')&#xA;Interpretation_be('1 equal one')&#xA;Interpretation_be('consequence eqaul bad')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to make up English rephraser as a most basic start of natural language processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While building up like above code, I had wanted to get some already-classified text data which criteria of classification is word-class, such as Noun, Verb, Preposition etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any good text data that I can obtain? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know. &lt;/p&gt;&#xA;" OwnerUserId="8325" LastEditorUserId="4302" LastEditDate="2018-10-08T12:25:36.543" LastActivityDate="2018-10-08T12:25:36.543" Title="Classification of English Words Based on Class [Noun, Verb, Adjective, Preposition, etc]" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3680" PostTypeId="1" CreationDate="2017-07-19T18:18:34.193" Score="2" ViewCount="547" Body="&lt;p&gt;How would one go about solving the &lt;a href=&quot;https://en.wikipedia.org/wiki/15_puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;15 squares puzzle&lt;/a&gt; using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithms&lt;/a&gt; approach?  In particular, I'd like to understand how you would &#xA;represent the &quot;chromosome&quot; in the evolving system.  That is, what's the relationship between the (artificial) &quot;genes&quot; and some sort of &lt;a href=&quot;https://pdfs.semanticscholar.org/6f25/ac7608edfa5efe7d13e05f38abb171490676.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;phenotypical expression&lt;/a&gt; w/r/t the problem.  It seems like genes would somehow represent moves or sequences of moves but I'm not entirely clear how this would work.&lt;/p&gt;&#xA;" OwnerUserId="8547" LastEditorUserId="16909" LastEditDate="2018-08-08T20:44:57.203" LastActivityDate="2018-08-08T20:44:57.203" Title="How can one use Genetic Algorithms to solve the &quot;15 Puzzle&quot; (Mystic Square)?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3681" PostTypeId="2" ParentId="3674" CreationDate="2017-07-19T20:25:57.687" Score="0" Body="&lt;p&gt;Not in C, JS but basically the simplest neural network I could come up with and tried to explain with basic concepts and doodles...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://becominghuman.ai/making-a-simple-neural-network-2ea1de81ec20&quot; rel=&quot;nofollow noreferrer&quot;&gt;Making a simple neural network&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="3020" LastActivityDate="2017-07-19T20:25:57.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3682" PostTypeId="2" ParentId="1774" CreationDate="2017-07-20T02:01:37.690" Score="1" Body="&lt;p&gt;Many publications from the middle of the twentieth century prove the questioner's statement that it was a widely held belief during that period that AI would quickly become conscious, self-aware, and smart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Great Success&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many tasks and forms of expertise once the exclusive domain of human intelligence, after the development of the Von Neumann general purpose computing architecture became, by the end of that century, more or less the exclusive domain of computers.  These are only a few examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Scientific and statistical computation&lt;/li&gt;&#xA;&lt;li&gt;Drafting and manufacturing process automation (CAD and CAM)&lt;/li&gt;&#xA;&lt;li&gt;Publishing and typesetting&lt;/li&gt;&#xA;&lt;li&gt;Certain forms of algebraic and calculus reductions (Maxima and its derivatives)&lt;/li&gt;&#xA;&lt;li&gt;Circuit analysis&lt;/li&gt;&#xA;&lt;li&gt;Masterful board game playing&lt;/li&gt;&#xA;&lt;li&gt;Profitable stock speculation&lt;/li&gt;&#xA;&lt;li&gt;Pattern recognition (OCR, fingerprint, voice recognition, sorting, terrain)&lt;/li&gt;&#xA;&lt;li&gt;Programming in predicate logic and recursive predicates&lt;/li&gt;&#xA;&lt;li&gt;Strategy evaluation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disappointments (thus far)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to this impressive array of successes, there is an equally long list of failed expectations.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Consumer available bipedal robots&lt;/li&gt;&#xA;&lt;li&gt;Automated vacuum cleaning (major disappointment for this answer's author)&lt;/li&gt;&#xA;&lt;li&gt;Autonomous mechanical factory workers&lt;/li&gt;&#xA;&lt;li&gt;Automated mathematicians (creative hypothesis generation and proof/disproof to extend theory)&lt;/li&gt;&#xA;&lt;li&gt;Natural language comprehension&lt;/li&gt;&#xA;&lt;li&gt;Obedience to arbitrary commands&lt;/li&gt;&#xA;&lt;li&gt;Human-like expression in conversation&lt;/li&gt;&#xA;&lt;li&gt;Automated technical innovation&lt;/li&gt;&#xA;&lt;li&gt;Computer morality&lt;/li&gt;&#xA;&lt;li&gt;Human (or at least mammalian) emotional states&lt;/li&gt;&#xA;&lt;li&gt;Asimov's three laws operating system&lt;/li&gt;&#xA;&lt;li&gt;Adaptive strategy development in arbitrary and shifting set of domains&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Domain and Domain-free Distinction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;When did it become clear that devising programs that master games like chess resulted in software designs that only applied to games like the ones for which they were programmed?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the general public may have thought that a cybernetic chess master would also be smarter than people in other ways, those creating those programs were well aware of the distinction between developing software that exhibited excellence in chess play hard coded and developing software that exhibits the ability to learn chess play and develop excellence iteratively from novice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The end goal had always been high powered general intelligence.  More short term achievable objectives were created to facilitate the demonstration of progress to investors.  It was the only way to maintain a continuous stream of research funding from the military.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first milestone was to master a single game without machine learning.  Then research turned to the building of domain knowledge so that a class of solutions, adaptations, and forms of planning could be realized in real time during warfare.  As economic domination became more preferable to military domination during the third quarter of the twentieth century, the vision for AI scaled to embrace the domains of economics and natural resource management.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this spectrum of automation maturity.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A program that enumerates current move sequence possibilities at each turn in the play of a chess game, eliminating probable bad moves at each projected move point, and selects the next move most likely to lead to a win&lt;/li&gt;&#xA;&lt;li&gt;A program that does the above but also skews probability based on pattern recognition of known winning chess strategies&lt;/li&gt;&#xA;&lt;li&gt;A program that is designed to be a run time optimized rules engine that centralizes and abstracts the redundant operations of the play of an arbitrary game and isolates and aggregates the representation of chess rules, chess strategies, and chess patterns and anti-patterns&lt;/li&gt;&#xA;&lt;li&gt;A program that, given a set of rules of a game, can generate a next move based on any game state, remembers success and failure results and the sequences that led to those outcomes, and has the ability to assess the probable loss or gain of individual moves and the game patterns in space and time around them based on history, and then leverages these abilities to learn an arbitrary game, reaching the masterful level of play of chess through learning&lt;/li&gt;&#xA;&lt;li&gt;A program that learns how to learn games such that, after learning several games, it can learn chess faster than an intellectually gifted human can&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first is easy.  The last is extremely challenging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the distinctions between these phases of automation maturity became apparent and how clear people became of those distinctions in which research groups is a complex probabilistic function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Key Contributors&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Who was the first person to recognize the distinction between human-like general intelligence and domain specific intelligence?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Norbert Wiener was likely the first to deeply comprehend the distinction between electronic control of relays (investigated theoretically by Claude Shannon) and closed loop control.  In his book, Cybernetics, a primarily mathematical work, he precisely established the foundation for self-correcting and adaptive systems.  John von Neumann had a comprehension of the distinction between programming good game play and the human ability to learn good game play and published much on the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was Arthur Lee Samuel who actually wrote the first impressive demonstration of the distinction between game playing software and machine learning.  It was he who bridged Wiener's work with the contemporary digital computer and first coined the term Machine Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Distorted Restatements of Authentic Research and Innovation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The categories artificial narrow intelligence (ANI), artificial general intelligence (AGI), and artificial super intelligence (ASI), proposed in &lt;em&gt;The AI Revolution: The Road to Superintelligence&lt;/em&gt; by blogger Tim Urban (Huffington Post, THE BLOG, posted 2/10/2015, updated 4/12/2015), is referenced in AI Stack Exchange in multiple places, but the distinctions between these categories are not precisely defined and the ideas contained therein are neither peer review nor validated by other research or statistics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The work is no less conjecture than mediocre science fiction — entertaining enough to gain some popularity but not rational conclusions drawn from either repeatable experiments or randomized studies.  The trend graphs provided in the article are of invented shape, not graphical representations of actual data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the material may later be found to have some truth in it, as in the case for many lay interpretations of scientific research or the futuristic thoughts of science fiction authors.  However, much of the material leads to misconception and false assertions.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-07-20T03:18:02.933" LastActivityDate="2017-07-20T03:18:02.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3683" PostTypeId="1" CreationDate="2017-07-20T04:47:02.953" Score="5" ViewCount="298" Body="&lt;p&gt;How can I determine if an input sentence is consistent with a certain subject?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose I am given the following dataset.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;| Subject       | User input           | Output |&#xA;|---------------|----------------------|--------|&#xA;| Dog ownership | I own a dog          | Yes    |&#xA;| Dog ownership | My dog is called Joe | Yes    |&#xA;| Dog ownership | I don't have a dog   | No     |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the examples above, the subject &quot;dog ownership&quot; is consistent with the input sentences &quot;I own a dog&quot; and &quot;My dog is called Joe&quot; (because, if your dog is called Joe, then you also own a dog).  &lt;/p&gt;&#xA;" OwnerUserId="8553" LastEditorUserId="2444" LastEditDate="2020-03-14T19:01:44.790" LastActivityDate="2020-03-14T19:04:56.913" Title="How can I determine if an input sentence is consistent with a certain subject?" Tags="&lt;natural-language-processing&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3685" PostTypeId="2" ParentId="3683" CreationDate="2017-07-20T13:28:07.983" Score="1" Body="&lt;p&gt;If you have collection of data or information, then you would like to ask questions about this data and machine should answer you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you need first, data mining to export the meaning and relations in this data, then you can build your expert system that will answer you.&lt;/p&gt;&#xA;" OwnerUserId="7362" LastActivityDate="2017-07-20T13:28:07.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3689" PostTypeId="2" ParentId="15" CreationDate="2017-07-21T08:16:34.920" Score="2" Body="&lt;p&gt;&lt;em&gt;Is the Turing Test, or any of its variants, a reliable test of artificial intelligence?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Myopia&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, if one defines the term Artificial Intelligence in terms of Alan Turing's Imitation Game or one of its variants.  The approach may be, at the same time, both valid and very limited as a definition of intelligence as people interpreted the word before AI emerged.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Proven Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, there are a large number of alternative approaches to measuring intelligence, artificial or otherwise.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Becoming a chess grand master&lt;/li&gt;&#xA;&lt;li&gt;Authoring a winning chess program&lt;/li&gt;&#xA;&lt;li&gt;Receiving a highly selective international award&lt;/li&gt;&#xA;&lt;li&gt;Creating a strategy that wins a war or a peace&lt;/li&gt;&#xA;&lt;li&gt;Overcoming the thousands of rounds of elimination in business or politics to become President&lt;/li&gt;&#xA;&lt;li&gt;Authoring brilliant articles, papers, screenplays, lectures, speeches, books, or poems that generate significant human paradigm shifts&lt;/li&gt;&#xA;&lt;li&gt;Showing genius level results in a Mensa test&lt;/li&gt;&#xA;&lt;li&gt;Becoming one of the most wealthy people in the world&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Normal Measurement of Normal Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these are measurements of exceptional intelligence of some kind, mostly because the leaders in these areas have reliably applied intelligence over multiple domains in such ways that led to remarkable success through multiple real life scenarios.  The reliability is an attribute of the person possessing the intelligence, not the test of intelligence itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are more mundane, yet perhaps more valid and reliable, measures of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Raising healthy and loving children as verifiable through the careful interviewing of friends and associates of the members of the family&lt;/li&gt;&#xA;&lt;li&gt;Repeated and successful remedy of many conditions of varying types that were once identified as broken in some tangible and measurable way and found to be measurably corrected as a result of the application of intelligence comprehension, analysis, and remedial action&lt;/li&gt;&#xA;&lt;li&gt;Conversational intelligence as measurable through the participants in conversation attributing their own success to the ideas and examples set by the conversationalist&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What Are the Truly Desired End Goals?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the primary characteristic of the Turing Test is that it is artificial.  If artificial intelligence is what we want from AI software, then that is what we will receive.  However, it is likely we want something either considerably more or considerably less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want more in that it would be nice of some computers could be our friend, our mentor, and an unpaid employee with exceptional abilities leading to our personal success in terms of income, influence, popularity, or legacy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want less in that we want some computers to do domain specific tasks and remain as fully subservient tools, perhaps with some personality and warmth, like a ship or some other complex device we give human names, yet without the unpredictability of the far reaching capabilities of human intelligence.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-07-21T08:25:35.217" LastActivityDate="2017-07-21T08:25:35.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3690" PostTypeId="1" AcceptedAnswerId="3691" CreationDate="2017-07-21T09:02:33.873" Score="2" ViewCount="987" Body="&lt;p&gt;I'm trying to come up with the right algorithm for a system in which the user enter symptoms and we starts triggering questions related to that symptoms and his answers will result a disease which is related to the answer which is given by the user&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that the user entered the following input:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Symptom - Deafness&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q1. How long have you had a problem with deafness&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;A)From few days B)From few weeks to months C)More than month D)Since Birth&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q2. What was the onset of the deafness&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;A) Sudden B) Gradual&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have a knowledge base like if a user select option 1 from question 1 and option 2 form question 2 then we will give him some disease. But i need an algorithm which will give % of success in backend so that i can throw the results of disease for example if a user select option 2 from question1 and option 1 from question 2, then when we compare from our knowledge base there will be one set over there which has  option 1 from question 1 and option 2 from question 2 then its a &quot;SOME&quot; disease.. now if we compare from our knowledgebase and we found even 50% of the choices is resulting this disease we will throw that disease name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOW i am confused what algo should be use for this approach for ai.&lt;/p&gt;&#xA;" OwnerUserId="11801" LastActivityDate="2017-07-21T17:02:08.540" Title="Selecting the right algorithm to predict disease from questions" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3691" PostTypeId="2" ParentId="3690" CreationDate="2017-07-21T13:50:58.983" Score="3" Body="&lt;p&gt;There is no defined rules for choosing a machine learning algorithm to learn some type of pattern. However, there are some guidelines to help you better select an algorithm which will yield a higher probability of success. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some important considerations are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of features: This is the number of questions that each patient had to answer.&lt;/li&gt;&#xA;&lt;li&gt;Number of instances: This is the number of patients that took your survey.&lt;/li&gt;&#xA;&lt;li&gt;Number of output classes: This is how specific you want your diagnosis of the disease. Is this a yes/no, or a 5-stage progression.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The larger your feature space and the more output classes you have, the higher the complexity of your model. This is problematic because a more complex model will require more instances of data to learn the underlining patterns. You need to have a good balance between the number of examples you have in your dataset and the complexity of your model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have limited data then you will want to stay very far away from deep learning. In such cases, I prefer to use shallow methods such as &lt;strong&gt;SVM&lt;/strong&gt;, &lt;strong&gt;Naive Bayes&lt;/strong&gt; or &lt;strong&gt;Random Forests&lt;/strong&gt;. These techniques have been shown to be able to capture non-linear relationships. SVM is particularly powerful, you can use the kernel trick! This will transform your feature space into a space that makes the differentiation between the classes easier to distinguish. Do not underestimate the power of this algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a very large dataset (i.e. 100,000 instances) then you will want to use deep learning. Recently, these techniques have been shown to outperform shallow machine learning algorithms in almost all categories where data is plentiful. You will want to start with a shallow &lt;strong&gt;neural network&lt;/strong&gt; and then increase the complexity of your graph as you see fit. By adding additional nodes per layer, or by adding additional layers. Each node you add will increase the number of variables you need to tune, thus increasing the complexity of your model. If your data is expected to have some type of time dependency which often the case in medical data. Then you can use a &lt;strong&gt;long-short term memory&lt;/strong&gt; neural network (LSTM). These are capable of capturing latent relationships. You can also try a &lt;strong&gt;stacked auto-encodder&lt;/strong&gt;. &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2017-07-21T13:50:58.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3692" PostTypeId="1" CreationDate="2017-07-21T15:46:58.727" Score="1" ViewCount="758" Body="&lt;p&gt;By this I mean a single axiom for placement.  &lt;em&gt;(I'm working on a solution that involves positional valuation and vectors, and appears to be solid, but my assumption would be this has been done previously, like most mathematical techniques one &quot;discovers&quot;;)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-07-21T15:46:58.727" Title="Has Tic-Tac-Toe been solved with a single heuristic?" Tags="&lt;heuristics&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3693" PostTypeId="2" ParentId="3690" CreationDate="2017-07-21T16:49:21.593" Score="0" Body="&lt;p&gt;Medical diagnosis often employs &lt;a href=&quot;https://en.wikipedia.org/wiki/Abductive_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;abductive inference&lt;/a&gt; (also known as &quot;inference to the best explanation&quot;), and automated approaches to abductive reasoning have been applied to medical diagnosis.  More concretely, a mechanism known as &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2244953/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parsimonious Covering Theory&lt;/a&gt; has been extensively researched as an approach to automated abductive reasoning in medical scenarios (among others).  You might consider giving that a look.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FWIW, one of the main researchers in the field is &lt;a href=&quot;https://www.cs.umd.edu/~reggia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Professor James Reggia&lt;/a&gt; from UMD.  He and &lt;a href=&quot;https://www.csee.umbc.edu/~ypeng&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yun Peng&lt;/a&gt; wrote a very accessible book on the subject - &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/B000QCQWL8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abductive Inference Models for Diagnostic Problem Solving&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And to add to what &lt;a href=&quot;https://ai.stackexchange.com/users/5925/jahknows&quot;&gt;JahKnows&lt;/a&gt; says - I'd say that where something like deep-learning would be most likely to come into play, would be building the initial knowledgebase that is used in the diagnostic system.  That is, if you have lots of data relating symptoms to diseases, you could use deep learning to help define the weighting between specific diseases and specific symptoms.  And note that while abductive inference is considered a branch of logic, and PCT dates back to the &quot;GOFAI&quot; era, it is not a strictly symbolic approach.  The Reggia &amp;amp; Peng book explains how to incorporate Bayesian reasoning into the abductive system.  Just FYI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I've recently been researching this area and am working on a modern implementation of PCT using the Semantic Web stack.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2017-07-21T17:02:08.540" LastActivityDate="2017-07-21T17:02:08.540" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="3695" PostTypeId="2" ParentId="3683" CreationDate="2017-07-21T19:57:51.353" Score="2" Body="&lt;p&gt;If you have lots of training data, ANN's (deep learning) could quite possibly get you there.  But I have a hunch you might also get some mileage out of using something like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Rule_induction&quot; rel=&quot;nofollow noreferrer&quot;&gt;rule induction&lt;/a&gt; approach.  Maybe something like &lt;a href=&quot;https://en.wikipedia.org/wiki/CN2_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;CN2&lt;/a&gt;. I'd suggest at least reading up on those and see if you can see a way to apply that to your system.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2020-03-14T19:02:49.773" LastActivityDate="2020-03-14T19:02:49.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3696" PostTypeId="1" CreationDate="2017-07-21T20:36:17.653" Score="8" ViewCount="424" Body="&lt;p&gt;I'm going to give a talk, and I'm preparing the material. The purpose of the conversation is to convince companies in my region that it is possible to apply artificial intelligence in solving everyday business problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like some examples to be able to present, and so I came here to ask&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Have you used artificial intelligence to solve a problem at work? What kind of problem?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="7714" LastEditorUserId="2444" LastEditDate="2019-06-25T20:17:19.500" LastActivityDate="2019-06-26T22:26:04.507" Title="Examples of uses of artificial intelligence at work" Tags="&lt;applications&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3697" PostTypeId="2" ParentId="3665" CreationDate="2017-07-22T06:28:52.180" Score="6" Body="&lt;p&gt;I don't think I can give you a true answer to the actual question as posed, as I don't have a strict definition of &amp;quot;general intelligence&amp;quot;. Nor do I have a solid definition of &amp;quot;critical&amp;quot; in context.&lt;/p&gt;&#xA;&lt;p&gt;However, if we lean on our naive/intuitive understanding of what &amp;quot;general intelligence&amp;quot; and what it means to be critical, you might translate your question as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would a general intelligence system need analogy-making in order to do certain things that it couldn't otherwise do?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Or, to put it another way&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are there useful behaviors that are enabled by analogical reasoning that can't be replicated any other way?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the strictest sense, I don't have an answer to either of those questions either, but there is at least &lt;em&gt;evidence&lt;/em&gt; to suggest that the answer may be &amp;quot;yes&amp;quot;.  See, for reference, the &lt;a href=&quot;http://cognitrn.psych.indiana.edu/rgoldsto/courses/concepts/copycat.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Copycat paper&lt;/a&gt; by Hofstadter and Mitchell.&lt;/p&gt;&#xA;&lt;p&gt;From what I've seen, some of the &lt;em&gt;kinds of problems Copycat solves&lt;/em&gt; are different from anything I've seen solved by other approaches.  Now maybe it's just a coincidence that nobody has tried solving those problems with, I don't know, let's say &amp;quot;deep learning&amp;quot; or &amp;quot;rule induction&amp;quot; or &amp;quot;genetic algorithms&amp;quot;.  Or maybe they have and I just haven't stumbled across that corpus of research.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I'll also add that there is still ongoing research into using &lt;em&gt;analogy&lt;/em&gt; for AI/ML.  See, for example, the paper &lt;a href=&quot;http://proceedings.mlr.press/v70/liu17d/liu17d.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analogical Inference for Multi-relational Embeddings (2017)&lt;/a&gt;, where the authors talk about using analogy, but define their approach as &amp;quot;analogical inference&amp;quot; (which they claim is different from &amp;quot;analogical reasoning&amp;quot; as defined during the earlier &amp;quot;GOFAI period&amp;quot;).  There is also the paper &lt;a href=&quot;https://arxiv.org/pdf/1705.04416.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evaluating vector-space models of analogy (2017)&lt;/a&gt;, where another set of authors deal with a form of analogical reasoning.&lt;/p&gt;&#xA;&lt;p&gt;I don't think there's a consensus as to whether or not some form of analogical reasoning is &amp;quot;critical&amp;quot;, but it's definitely a subject that is still being researched.&lt;/p&gt;&#xA;&lt;p&gt;And to go off on a little bit of a tangent - an interesting related question would be to ask whether or not &amp;quot;analogy making&amp;quot; would be an emergent property of a sufficiently deep/wide ANN, or would such a facility need to be designed and coded up explicitly.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2021-01-25T16:15:12.157" LastActivityDate="2021-01-25T16:15:12.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3698" PostTypeId="1" CreationDate="2017-07-22T08:24:30.650" Score="2" ViewCount="73" Body="&lt;p&gt;I want to produce a bot in Python that automatically generates short football summaries from Whoscored data. For my first stage I generate the articles with different sentence templates and lots of rules where the data is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to move to the next stage and start looking into NLP and more advanced NLG. I already scraped numerous articles to create a corpus. How should I move on and do next? Any advice would be much appreciated as I'm new in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="4302" LastEditDate="2018-10-08T12:24:50.293" LastActivityDate="2018-10-08T12:24:50.293" Title="Advanced NLG - robot journalist" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3702" PostTypeId="1" CreationDate="2017-07-24T13:20:59.073" Score="1" ViewCount="194" Body="&lt;p&gt;I need some help in developing a Markov Model for a crossroads there is no one way road and i am assuming at this time that traffic is only allowed to go straight no turns are allowed. There are 4 roads and 4 signals(agents).&lt;/p&gt;&#xA;" OwnerDisplayName="user3751" LastActivityDate="2017-08-27T03:52:11.907" Title="Markov Model for a Traffic Intersection" Tags="&lt;ai-design&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;markov-chain&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3703" PostTypeId="1" AcceptedAnswerId="3707" CreationDate="2017-07-24T15:48:53.760" Score="4" ViewCount="320" Body="&lt;p&gt;&lt;a href=&quot;http://www.vocativ.com/culture/science/most-powerful-poker-computer-cepheus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cepheus&lt;/a&gt; is an artificial intelligence designed to play Texas Hold'em. By playing against itself and learning where it could have done better, it became very good at the game. &lt;a href=&quot;http://slatestarcodex.com/2015/01/17/links-12014-link-for-you-know-not-whence-you-came-nor-why/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Slate Star Codex&lt;/a&gt; comments:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was originally confused why they published this result instead of heading to online casinos and becoming rich enough to buy small countries, but it seems that it’s a very simplified version of the game with only two players. More interesting, the strategy was reinforcement learning – the computer started with minimal domain knowledge, then played poker against itself a zillion times until it learned everything it needed to know. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Apparently Cepheus currently just plays against one person. Seeing as it managed to develop amazing strategy for this &quot;very simplified&quot; environment, what's stopping it from working on real/full poker games?&lt;/p&gt;&#xA;" OwnerUserId="75" LastEditorUserId="75" LastEditDate="2017-07-27T00:36:13.710" LastActivityDate="2017-07-27T22:17:19.757" Title="What's stopping Cepheus from generalizing to full poker games?" Tags="&lt;reinforcement-learning&gt;&lt;gaming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3705" PostTypeId="1" CreationDate="2017-07-25T03:50:53.097" Score="1" ViewCount="86" Body="&lt;p&gt;I'm developing a chatbot, and to get the answer I'm using the Naive Bayes classifier by sorting the questions and answers. For those who want to see the whole project code and more definitions follow the link of &lt;a href=&quot;https://github.com/ZehLuckmann/Roger&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To develop I am using the TextBlob library for python, the problem is that when training my classifier it is always returning the same message, regardless of the input I use. The message is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Tudo bom?&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I still can not identify the problem, I do not know if the problem is in the way my data is willing to perform the training or if it is the way I am training the classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My class that performs the sorting process is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#encoding: utf-8&#xA;#!/usr/bin/env python&#xA;from textblob.classifiers import NaiveBayesClassifier&#xA;from textblob import TextBlob&#xA;import logging&#xA;&#xA;class Talk(object):&#xA;    &quot;&quot;&quot;The Talk class is responsible for returning the response&#xA;    Based on the information exported. Using the classification&#xA;    According to Bayes' theorem&#xA;    &quot;&quot;&quot;&#xA;    def __init__(self):&#xA;        &quot;&quot;&quot;&#xA;        Class builder&#xA;&#xA;        Cl -&amp;gt; Stores the classifier&#xA;        Accuracy -&amp;gt; Stores the accuracy of the algorithm&#xA;        &quot;&quot;&quot;&#xA;        self.__cl = None&#xA;        self.__accuracy = 0&#xA;&#xA;&#xA;    def train(self, train_set):&#xA;        &quot;&quot;&quot;&#xA;        Train with the list of information consisting of phrases and their&#xA;        Respective classifications:&#xA;        &quot;&quot;&quot;&#xA;&#xA;        logging.debug('Inicia treinamento da previsão de intenção')&#xA;        self.__cl = NaiveBayesClassifier(train_set)&#xA;        logging.debug('Treinamento da previsão de intenção finalizado')&#xA;&#xA;    def test(self, test_set):&#xA;        &quot;&quot;&quot;&#xA;        Performs tests with the list of information formed&#xA;        Of sentences and their respective classification to obtain the accuracy:&#xA;        &quot;&quot;&quot;&#xA;&#xA;        logging.debug('Inicia teste da previsão de intenção')&#xA;        self.__accuracy = self.__cl.accuracy(test_set)&#xA;        logging.debug('Teste da previsão de intenção finalizado')&#xA;        logging.info('Precisão da previsão: {}'.format(self.__accuracy))&#xA;&#xA;    def response(self, phrase):&#xA;        &quot;&quot;&quot;&#xA;        Returns the phrase response according to the created classifier&#xA;        &quot;&quot;&quot;&#xA;        logging.debug('Analisa a frase &quot;{}&quot;'.format(phrase))&#xA;        blob = TextBlob(phrase,classifier=self.__cl)&#xA;        result = blob.classify()&#xA;        logging.debug('Resposta: &quot;{}&quot;'.format(result))&#xA;        return result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Follow the link in my file with training information and test data&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/ZehLuckmann/Roger/blob/master/src/roger/database/dados_treino_ze.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/ZehLuckmann/Roger/blob/master/src/roger/database/dados_teste_ze.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;Test&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7714" LastActivityDate="2017-07-25T03:50:53.097" Title="Problem with the PLN classifier" Tags="&lt;classification&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3706" PostTypeId="2" ParentId="2733" CreationDate="2017-07-25T15:43:59.440" Score="2" Body="&lt;p&gt;The expectation is over the policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; because the action at the state &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; is taken according to &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;, &lt;strong&gt;and&lt;/strong&gt;, for the proof, the book text (2nd edition, paragraph below Equation 4.8) defines &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; to be a policy that is identical to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; except that &lt;span class=&quot;math-container&quot;&gt;$\pi'(s) = a \neq \pi(s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is one particular state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, essential, the book text tries to prove that, for such a changed policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;, if &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, a) = q_\pi(s, \pi'(s)) &amp;gt; v_\pi(s)$&lt;/span&gt;, then the changed policy is better than &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$\pi'(s) = a$&lt;/span&gt; in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is why &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, \pi'(s)) = \mathbb{E}_{\pi'}[R_{t+1} + \gamma v_\pi(S_{t+1}) \mid S_t = s]$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6254" LastEditorUserId="2444" LastEditDate="2018-11-11T20:39:33.820" LastActivityDate="2018-11-11T20:39:33.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3707" PostTypeId="2" ParentId="3703" CreationDate="2017-07-25T17:31:48.203" Score="3" Body="&lt;p&gt;The reason why Cepheus can't generalize has to do with the number of decision points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same authors recently let loose Deep Stack (DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit (HUNL) Poker) which is freaking many professional poker players out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1701.01724.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepStack arxiv paper&lt;/a&gt;, they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AI techniques (Cepheus) have previously shown&#xA;  success in the simpler game of heads-up limit Texas hold’em, where all bets are of a fixed size resulting in just under 10^14 decision points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The imperfect information game HUNL is comparable in size to go, with&#xA;  the number of decision points exceeding 10^160&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Imperfect information games require more complex reasoning than similarly sized perfect&#xA;  information games. The correct decision at a particular moment depends upon the probability&#xA;  distribution over private information that the opponent holds, which is revealed through their&#xA;  past actions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Using the same strategy for HUNL as Cepheus did is out of the question. Rather, taking an educated guess or using intuition based on the previous play (referred to as Continual Re-solving in the paper) is a method which can better handle this gargantuan game. For more information check out the &lt;a href=&quot;https://www.deepstack.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepStack website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2017-07-27T22:17:19.757" LastActivityDate="2017-07-27T22:17:19.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3709" PostTypeId="2" ParentId="3696" CreationDate="2017-07-26T00:34:00.243" Score="3" Body="&lt;p&gt;This is an interesting question in that it addresses how AI becomes a part of our mundane existence.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;@JadenTravnik's pithy comment is quite correct--spam filtering is a form of AI that that it so fundamental to the information eco system that it has become nearly invisible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's useful to be able to define what we mean by Artificial (or Algorithmic) Intelligence, and to distinguish it from computation in general.  A fundamental distinction is that AI makes decisions as opposed to merely crunching numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My feeling is that applied AI is still a very specialized field, so that people using it dynamically to solve problems at work &lt;em&gt;today&lt;/em&gt; are those who work in the field of AI, as opposed to workers in general.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may find this related question useful: &lt;a href=&quot;https://ai.stackexchange.com/q/2092/1671&quot;&gt;Is AI programming useful in everyday programs?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may change in the future, where a type of expert system, for instance, could assist a worker by presenting it's own analysis and conclusions on a decision problem. But in most office settings, that type of reasoning is still the domain of humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice would be to continue on Jaden's track, looking for examples of where AI &lt;em&gt;underlies&lt;/em&gt; everyday functions.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-07-26T23:53:24.097" LastActivityDate="2017-07-26T23:53:24.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3713" PostTypeId="2" ParentId="2854" CreationDate="2017-07-26T12:08:22.967" Score="3" Body="&lt;p&gt;No.&#xA;YOLO and SSD are based on Nvidia's proprietary CUDA technology which is not available on Raspberry simply because of the GPU vendor is not Nvidia.&#xA;Even more, there seems to be no implementation of even OpenCL for the Raspberry's GPU.&#xA;What you can do is to try port YOLO's of SSD's CNN core from CUDA to Raspberry GPU's assembler, in the way described in, for example,  &lt;a href=&quot;https://rpiplayground.wordpress.com/2014/05/03/hacking-the-gpu-for-fun-and-profit-pt-1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rpiplayground.wordpress.com/2014/05/03/hacking-the-gpu-for-fun-and-profit-pt-1/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8679" LastActivityDate="2017-07-26T12:08:22.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3715" PostTypeId="1" CreationDate="2017-07-26T23:48:30.277" Score="1" ViewCount="84" Body="&lt;p&gt;In &lt;code&gt;Artificial Intelligence: a Modern Approach&lt;/code&gt;, when it talks about strategies to improve efficiency of resolution inference(section 9.5.6), it says selecting the &lt;em&gt;set of support&lt;/em&gt; and resolving one of elements in it first are helpful. But I cannot understand the way it select the &lt;em&gt;set of support&lt;/em&gt; and why.&lt;/p&gt;&#xA;&lt;p&gt;The original excerpt as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Set of support&lt;/strong&gt;: Preferences that try certain resolutions first are helpful, but in general it is more effective to try to eliminate some potential resolutions altogether. For example, we can insist that every resolution step involve at least one element of a special set of clauses—the &lt;em&gt;set of support&lt;/em&gt;. The resolvent is then added into the set of support. If the set of support is small relative to the whole knowledge base, the search space will be reduced dramatically.&lt;/p&gt;&#xA;&lt;p&gt;We have to be careful with this approach because a bad choice for the set of support will make the algorithm incomplete. However, if we choose the set of support S so that the remainder of the sentences are &lt;strong&gt;jointly satisfiable&lt;/strong&gt;, then set-of-support resolution is complete. For example, one can use the negated query as the &lt;em&gt;set of support&lt;/em&gt;, on the assumption that the&#xA;original knowledge base is consistent. (After all, if it is not consistent, then the fact that the query follows from it is vacuous.) The set-of-support strategy has the additional advantage of generating goal-directed proof trees that are often easy for humans to understand.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does it mean that the remainder of the sentences are &lt;strong&gt;jointly satisfiable&lt;/strong&gt;? Why could one use the negated query as the &lt;em&gt;set of support&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance, I hope someone could shed some light on it:)&lt;/p&gt;&#xA;&lt;p&gt;Ps. I'm a English leaner. I may not present this problem very well, and I'm very sorry for that. But I‘m very serious about it and I've try my best to make it as clear as I can. So if you're about to down vote it, please comment below and give me some advice to improve it, I'll be very grateful for that. Thanks again!&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-07-29T00:36:27.003" Title="How to select the set of support and why?" Tags="&lt;logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3716" PostTypeId="1" CreationDate="2017-07-27T04:32:49.237" Score="2" ViewCount="209" Body="&lt;p&gt;In section 10.3.2 in &lt;em&gt;Artificial Intelligence: a Modern Approach&lt;/em&gt; there is a piece of pseudocode that describes the graph plan algorithm. The graph plan algorithm constructs a planning graph for a problem and extract a solution from it if there is one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function GRAPHPLAN(problem) returns solution or failure&#xA;    graph ← INITIAL-PLANNING-GRAPH(problem) &#xA;    goals ← CONJUNCTS(problem.GOAL)&#xA;    nogoods ← an empty hash table&#xA;    for tl=0 to ∞ do&#xA;        if goals all non-mutex in St of graph then&#xA;            solution ← EXTRACT-SOLUTION(graph, goals, NUMLEVELS(graph), nogoods) &#xA;            if solution != failure &#xA;                then return solution&#xA;        if graph and nogoods have both leveled off &#xA;                then return failure                 &#xA;        graph ← EXPAND-GRAPH(graph, problem)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also a PPT for &lt;a href=&quot;http://www.cs.mtu.edu/~nilufer/classes/cs5811/2014-fall/lecture-slides/cs5811-ch10b-graphplan.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graphplan&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What confuses me for a long time is when the &lt;code&gt;nogoods&lt;/code&gt; levels off? The book describe no-goods as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the case where EXTRACT-SOLUTION fails to find a solution for a set of goals at a given level, we record the (level,goals) pair as a no-good.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But in that case, every time the first &lt;code&gt;EXTRACT-SOLUTION&lt;/code&gt; after &lt;code&gt;EXPAN-GRAPH&lt;/code&gt; fails, there will always be at least a new no-good produced: whose &lt;code&gt;level&lt;/code&gt; becomes the new level of the planning graph and the &lt;code&gt;goals&lt;/code&gt; is always the same--which is the goal of the problem. So when does it level off?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another sentence about no-goods in the book:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the function EXTRACT-SOLUTION fails to find a solution, then there must have been at least one set of goals that were not achievable and were marked as a no-good. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From above excerpt, I learn that &lt;code&gt;goals&lt;/code&gt; in no-goods is a set of goals that are not achievable in the &lt;code&gt;level&lt;/code&gt;. But, IMHO, that could vary when we chose different actions in &lt;code&gt;EXTRACT-SOLUTION&lt;/code&gt;. The example in the book provides a good instance:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider an air cargo domain with one plane and n pieces of cargo at airport A, all of which have airport B as their destination. In this version of the problem, only one piece of cargo can fit in the plane at a time. The graph will level off at level 4, reflecting the fact that for any single piece of cargo, we can load it, fly it, and unload it at the destination in three steps. But that does not mean that a solution can be extracted from the graph at level 4; in fact a solution will require 4n − 1 steps: for each piece of cargo we load, fly, and unload, and for all but the last piece we need to fly back to airport A to get the next piece.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the example above, the set of goals not achievable at level 4 varies based on what actions has been taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone help clear up this problem? I know there must be some misunderstanding, welcome to point it out, please. Thanks in advance:)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ps. I'm a English leaner, I may not present this problem very well, so I'm very sorry for any reason that you may want to down vote it. But I‘m very serious about it and I've try my best to make it as clear as I can. So if you're about to down vote it, please comment below and give me some advice to improve it, I'll be very grateful for that. Thanks again!&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="8689" LastEditDate="2017-07-29T00:30:59.270" LastActivityDate="2017-07-29T00:30:59.270" Title="What is a no-good? When does the no-goods level off?" Tags="&lt;path-planning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3717" PostTypeId="2" ParentId="3696" CreationDate="2017-07-27T09:17:29.057" Score="3" Body="&lt;p&gt;Artificial intelligence can be used to solve many real-world problems. For example, you can use data mining or AI algorithms &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;to create a detection system for anti-money laundering&lt;/li&gt;&#xA;&lt;li&gt;to understand the reason why many customers are closing their accounts&lt;/li&gt;&#xA;&lt;li&gt;to help prevent customer churn of particular products&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You do not need to implement an AGI in order to solve real-world problems.&lt;/p&gt;&#xA;" OwnerUserId="8680" LastEditorUserId="2444" LastEditDate="2019-06-26T22:26:04.507" LastActivityDate="2019-06-26T22:26:04.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3718" PostTypeId="2" ParentId="2" CreationDate="2017-07-27T10:17:17.960" Score="1" Body="&lt;p&gt;PS: &lt;em&gt;There is already some very good answers provided here, I will merely add to this answers in the hope that someone will find this useful:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing noise to a dataset can indeed have a positive influence on a model. In fact this can be seen as doing the same thing that you would normally do with &lt;a href=&quot;https://en.wikipedia.org/wiki/Regularization_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;regularizers&lt;/a&gt; like &lt;a href=&quot;https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pd&quot; rel=&quot;nofollow noreferrer&quot;&gt;dropout&lt;/a&gt;. Some of the example of doing this are &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2771718/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zur at.al&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1003.0358.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cires¸at.al&lt;/a&gt; where the authors successfully introduced noise into the dataset to reduce over-fitting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The catch is in knowing how much noise is too much. If you add too much noise, this might render your dataset useless in that the resulting dataset may no longer contain sufficient resemblance to the original dataset, so you might as well be training on a completely different dataset. Thus too much noise could be seen to cause under-fitting, just like extremely high dropout rates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;As the saying goes;  &lt;strike&gt;change&lt;/strike&gt; balance is the spice of life :).&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-07-27T10:26:19.407" LastActivityDate="2017-07-27T10:26:19.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3720" PostTypeId="1" AcceptedAnswerId="3721" CreationDate="2017-07-27T14:43:11.760" Score="1" ViewCount="464" Body="&lt;p&gt;If I create a program which takes an input, gives an output and then requires a response to let it know whether the answer it gave was any good does it count as AI? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what is the process of AI? Does it not always need specific parameters? For example, I ask it &quot;Who is the president of the USA?&quot;, and I have programmed it to look for news articles in SEOs and remove the &quot;Who&quot; part, is that AI?&lt;/p&gt;&#xA;" OwnerUserId="8698" LastEditorUserId="2444" LastEditDate="2019-03-02T21:23:59.687" LastActivityDate="2019-03-02T21:23:59.687" Title="Can a program that requires feedback be considered an AI?" Tags="&lt;ai-design&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3721" PostTypeId="2" ParentId="3720" CreationDate="2017-07-27T15:47:35.467" Score="1" Body="&lt;p&gt;There is no &quot;process of AI&quot; as such.  There are &lt;strong&gt;many, many&lt;/strong&gt; different approaches to AI, different ones of which are used in specific applications.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to whether a purely trial and error approach could be considered AI... I'd offer up a qualified &quot;maybe&quot;.  If you do nothing but an exhaustive scan of the solution space, for every trial, then I'd say &quot;No, it's not really any kind of AI&quot;.  OTOH, if you're using a knowledge-base of some sort and applying some kind of reasoning (even if it's a heuristic) , and if you have a system that somehow learns from the feedback from the user and gets &quot;smarter&quot; over time, then you're likely working on something that could be considered AI.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of that said, the exact definition of what is and isn't AI is somewhat fuzzy.   One popular definition is something like &quot;any technology that allows a computer to do something well that currently only humans can do well&quot;. So if you're doing something that fits that descriptions, it's possibly an aspect of AI.   And consider again that most people don't really consider &quot;brute force&quot; solutions to be AI.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-07-27T15:47:35.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3722" PostTypeId="2" ParentId="3720" CreationDate="2017-07-27T15:59:22.613" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;If an algorithm is making decisions, it can be deemed AI.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The decisions don't have to be better than a human's, or even good.  &lt;em&gt;(AI can be limited or &quot;dumb&quot;--on my project we specifically make dumb AIs so children and new players can win;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this perspective, the process of the algorithm choosing an output based on an input qualifies it as basic AI.  Asking for a response on the  quality of the output is merely the validation procedure.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the algorithm learns from the validation process, it constitutes a type of machine learning.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It may not be the &quot;sexy&quot; Machine Learning that has sparked renewed interest in the field of late, but it constitutes an &lt;a href=&quot;https://en.wikipedia.org/wiki/Automata_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;automata&lt;/a&gt; that is learning nonetheless.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-07-27T16:47:32.813" LastActivityDate="2017-07-27T16:47:32.813" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3723" PostTypeId="1" CreationDate="2017-07-27T17:29:50.153" Score="5" ViewCount="336" Body="&lt;p&gt;After witnessing the rise of deep learning as automatic feature/pattern recognition over classic machine learning techniques, I had an insight that the more you automate at each level, the better the results, and I, therefore, turned my focus to &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroevolution&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have been reading neuroevolution publications with the same desire to automate at every level.&lt;/p&gt;&#xA;&lt;p&gt;Do genetic algorithms also evolve themselves? Do they get better at searching through the solution space for each generation over time?&lt;/p&gt;&#xA;" OwnerUserId="8700" LastEditorUserId="2444" LastEditDate="2020-12-07T18:05:14.423" LastActivityDate="2020-12-28T18:41:58.503" Title="Do genetic algorithms also evolve?" Tags="&lt;reference-request&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;hyperparameter-optimization&gt;&lt;meta-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3724" PostTypeId="2" ParentId="2526" CreationDate="2017-07-27T19:31:01.097" Score="4" Body="&lt;p&gt;No, it is not necessary that an activation function is differentiable. In fact, &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of the most popular activation functions, the rectifier&lt;/a&gt;, is non-differentiable at zero!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can create problems with learning, as numerical gradients calculated near a non-differentiable point can be incorrect. The &quot;kinks&quot; section in &lt;a href=&quot;http://cs231n.github.io/neural-networks-3/#gradcheck&quot; rel=&quot;nofollow noreferrer&quot;&gt;these lecture notes&lt;/a&gt; discuss the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know the full answer as to whether this is a problem in practice and how people get around them. There are ways to avoid getting these incorrect gradients for some non-differentiable functions, as &lt;a href=&quot;http://cs231n.github.io/neural-networks-3/#gradcheck&quot; rel=&quot;nofollow noreferrer&quot;&gt;the lecture notes I discussed above&lt;/a&gt; mention.&lt;/p&gt;&#xA;" OwnerUserId="8704" LastActivityDate="2017-07-27T19:31:01.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3725" PostTypeId="2" ParentId="3723" CreationDate="2017-07-28T08:58:41.403" Score="1" Body="&lt;p&gt;A genetic algorithm is a class of evolutionary algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They do get better at searching through the solution possibilities for each trial (generation) over time because evolution usually starts from a population of randomly generated individuals, and is an iterative process. In each generation, the fitness of every individual in the population is evaluated. The more fit individuals are stochastically selected from the current population, and each individual's genome is modified to form a new generation thus getting better and better with each generation over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evolution is defined as the gradual development of something, especially from a simple to a more complex form.&lt;/p&gt;&#xA;" OwnerUserId="8680" LastEditorUserId="2444" LastEditDate="2019-06-25T20:09:36.293" LastActivityDate="2019-06-25T20:09:36.293" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3727" PostTypeId="1" CreationDate="2017-07-29T23:22:44.513" Score="2" ViewCount="69" Body="&lt;p&gt;Given this data set:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;User 1: {'Artist': 1, 'Public Figure': 9, 'Film Director': 1, 'Education': 1, 'Musician/Band': 4, 'Musician': 1, 'Community': 1, 'Author': 4, 'Politician': 1, 'TV Show': 2, 'Entrepreneur': 1, 'Journalist': 4, 'Product/Service': 1, 'Defense Company': 1, 'Computer Company': 2, 'Nonprofit Organization': 3, 'Computers &amp;amp; Internet Website': 4, 'Media/News Company': 3, 'Podcast': 1, 'News &amp;amp; Media Website': 2, 'Charity Organization': 1, 'Government Organization': 1, 'Magazine': 1}&#xA;User 2: {'Nonprofit Organization': 1, 'Movie': 2, 'Musician/Band': 22, 'Public Figure': 2, 'Entrepreneur': 2, 'TV Show': 2, 'Medical Company': 1, 'American Restaurant': 1, 'Sports &amp;amp; Recreation': 1, 'Media/News Company': 2, 'Sports Team': 1, 'Artist': 3, 'Musician': 1, 'College &amp;amp; University': 1, 'Athlete': 1, 'Music': 1, 'App Page': 1, 'Comedian': 1, 'Interest': 2, 'Product/Service': 1, 'Book Series': 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I believe I can draw conclusions that the strength of the connection is high or low based on the fact that they like or dislike the same categories of topics. What machine learning technique would be best to apply to these connection types, if I had, say, 500 users with categories and weights for each?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to automatically apply varying weights, if most people have Musician/Band or like Movies, that should be less interesting than when people like &quot;Defense Company&quot; like in User 1.&lt;/p&gt;&#xA;" OwnerUserId="8737" LastEditorUserId="8738" LastEditDate="2017-07-30T19:17:55.510" LastActivityDate="2017-07-30T19:17:55.510" Title="Weighing connections between two users using machine learning" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3728" PostTypeId="2" ParentId="3727" CreationDate="2017-07-30T01:34:11.937" Score="2" Body="&lt;p&gt;The question might be improved so that the answer can be more specific by defining two things more rigorously.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Strength of the connection&lt;/li&gt;&#xA;&lt;li&gt;What the automation must infer from that value&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning, Correlation, Projection, and Maximizing Profitability&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Guessing from the comments, it appears that the goal is to profile the user for business purposes.  Two common applications of user profiling are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To present buying options or&lt;/li&gt;&#xA;&lt;li&gt;To suggest friends in a social network.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although it may appear that a learning program would serve both of these applications, they are actually two very different problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One is to optimize e-commerce profitability through increased sales per session.  The other is to build a social network where there are some similarities and perhaps a few deliberate differences in the pairing of user profiles from which the friend suggestion comes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first of the two has to do with the projecting of the potential of a sale by matching the users and relying on the assumption that two people with like interests are more likely to mirror one another in purchasing patterns to some degree that could be projected with statistical analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But approaching the project by immediately jumping to learning algorithms may not produce the best results after a month or a year of development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;One Reasonable Approach for Some Problems of This Type&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the definitions of STRENGTH and AUTOMATICALLY in the question, it may be a simple application of a probability matrix or hypercube that will create a functional application of user profiling based on interests. To create a correlation between users via an ordered list of interests to place in one of the cells of the matrix or hypercube, the &lt;a href=&quot;https://statistics.laerd.com/statistical-guides/spearmans-rank-order-correlation-statistical-guide.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spearman's Rank Order Correlation&lt;/a&gt; is a generalized yet effective method for such applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning Usually Begins With a Guess&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you must determine what is (preferably in precise mathematical terms) what is a favorable product presentation or favorable potential friend introduction.  After that is determined, one can develop a probability formula to project what would be favorable based on the correlation matrix or hypercube.  Bayes's Theorem may be quite applicable in the development of a probability formula from the constraints and relationships of the application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning Added&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once improved favorablility gained through the application of the correlations  can be demonstrated, it would then be wise to apply a learning algorithm.  The sequence is thus.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Results gained by random selection of options to present the user&lt;/li&gt;&#xA;&lt;li&gt;Results gained by using the correlation matrix or hypercube&lt;/li&gt;&#xA;&lt;li&gt;Results gained by iteratively converging on an optimal presentation&lt;/li&gt;&#xA;&lt;li&gt;Tuning the iterative convergence track change and yet not oscillate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A multi-tier, back propagating Hopfield neural net could be one option for varying parameters in the use of the correlations (or the choice of correlations, since Spearman's is only one correlation model for ordered discrete interests).  Neural networks are only one type of convergent approach too.  Modelling with as a system of differential equations and using extensions of the Taylor Series to evaluate with a defined independent input vector is another.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see how simple convergence can be and how primitive what appears to be a learning algorithm can become, think about the &lt;a href=&quot;http://mathforum.org/library/drmath/view/52623.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;divide and average method of iteratively determining a square root&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2017-07-30T01:34:11.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3729" PostTypeId="1" AcceptedAnswerId="3733" CreationDate="2017-07-30T10:33:49.783" Score="6" ViewCount="1320" Body="&lt;p&gt;Can reinforcement learning algorithms be applied to computer vision problems? If yes, what are some examples of these applications?&lt;/p&gt;&#xA;" OwnerDisplayName="user3751" LastEditorUserId="2444" LastEditDate="2019-11-16T18:21:32.577" LastActivityDate="2019-11-16T18:23:21.430" Title="Can reinforcement learning algorithms be applied to computer vision problems?" Tags="&lt;reinforcement-learning&gt;&lt;computer-vision&gt;&lt;applications&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3730" PostTypeId="2" ParentId="3502" CreationDate="2017-07-30T16:07:49.537" Score="4" Body="&lt;p&gt;You will see a lot of game examples in reinforcement learning literature, because game environments can often be coded efficiently, and run fast on a single computer that can then contain the environment and the agent. For classic games, such as backgammon, checkers, chess, go, then there are human experts that we can compare results with. Certain games or simplified game-like environments are commonly used to compare different approaches, much like MNIST handwritten digits are used for comparing supervised learning approaches.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a way to teach reinforcement learning in applications other than games?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. Informally you could apply reinforcement learning approaches whenever you can frame a problem as an agent acting within an environment where it can be informed of the state and a goal-influencing reward value. More formally, reinforcement learning theory is based upon solutions to &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Processes&lt;/a&gt;, so if you can fit your problem description to a MDP then the various techniques used in RL - such as Q-learning, SARSA, REINFORCE - can be applied. This fit to theory does not need to be perfect for the resulting system to work, for instance you can often treat unknown or imperfectly observed state as effectively random to the agent, and consider this part of a stochastic environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some examples of possible uses for reinforcement learning outside of recreational games:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Control logic for motorised robot, such as &lt;a href=&quot;https://www.youtube.com/playlist?list=PL5nBAYUyJTrM48dViibyi68urttMlUv7e&quot; rel=&quot;nofollow noreferrer&quot;&gt;learning to flip pancakes and other examples&lt;/a&gt;. Here the environment measurements are made by physical sensors on the robot. The rewards are given for completing a goal, but may also be adjusted for smoothness, economic use of energy etc. The agent chooses low-level actions such as motor torque or relay position. In theory there can be nested agents where higher level ones choose the goals for the lower-level ones - e.g. the robot might decide at a high level between doing one of three tasks that require moving to different locations, and at a lower level might be decisions on how to control motors to move the robot to its chosen goal.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Self-driving cars. Although a lot of focus on sensor interpretation - seeing road markings, pedestrians etc, a control system is required in order to select accelerator, brake and steering.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Automated financial trading. Perhaps a game to some, there are clear real-world consequences. The reward signal is simple enough though, and RL can be adjusted to prefer long or short term gains.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;is it possible to set this up with say a CAD software?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In theory yes, but I do not know what might be available to do this in practice. Also you need one or more goals in mind that you code into the agent (as reward values that it can observe) before giving it a virtual mouse and setting a task to draw something. Computer games come with a reward scheme built in as their scoring system, and provide frequent feedback, so an agent can gain knowledge of good vs bad decisions quickly. You would need to replace this scoring component with something that represents your goals for the CAD-based system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CAD does not have anything suitable built-in, although CAD tools with simulations, such as various physics engines or finite element analysis, could allow you to score designs based on a simulated physical measure. Other possibilities include analysis of strain, non-wasteful use of material, whatever metrics the CAD/CAM system can provide for a partial or completed design. The tricky part is constraining a design to its goal or purpose and either arranging for that to be rewarded, or building the constraints into the environment; giving an RL agent full unconstrained control of CAD process and rewarding on lowest strain will likely result in something very uninteresting such as a small cube.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-18T05:50:32.677" LastActivityDate="2018-06-18T05:50:32.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3731" PostTypeId="1" AcceptedAnswerId="3738" CreationDate="2017-07-30T19:06:58.153" Score="4" ViewCount="868" Body="&lt;p&gt;I'm learning Neural Networks, and everything works as planned but, like humans do, adjusting themselves to learn more efficiently, I'm trying to understand conceptually how one might implement an auto adjusting learning rate for a Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to make it based on error, something like how bigger is error learning rate is getting bigger as well. &lt;sub&gt;&lt;em&gt;[Could use some clarification here--not entirely sure what you're saying.  If can clarify, I'm happy to clean up the English. -DukeZhou]&lt;/em&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*If you want give me an example give it on a C based language or math because I don't have experience with Python or Pascal. &lt;/p&gt;&#xA;" OwnerUserId="8497" LastEditorUserId="1671" LastEditDate="2017-07-31T18:56:39.303" LastActivityDate="2019-02-22T08:54:00.887" Title="How to implement an Automatic Learning Rate for a Neural Network?" Tags="&lt;neural-networks&gt;" AnswerCount="3" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3732" PostTypeId="1" AcceptedAnswerId="3755" CreationDate="2017-07-31T08:23:45.303" Score="5" ViewCount="364" Body="&lt;p&gt;I have been following the &lt;a href=&quot;http://www.cs.cmu.edu/~ninamf/courses/601sp15/lectures.shtml&quot; rel=&quot;noreferrer&quot;&gt;ML course by Tom Mitchel&lt;/a&gt;. &#xA;The inherent assumption while using Decision Tree Learning Algo is: &lt;strong&gt;The algo. preferably chooses a Decision Tree which is the smallest.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is this so&lt;/strong&gt; when we can have bigger extensions of the tree which could in principle perform better than the shorter tree?&lt;/p&gt;&#xA;" OwnerUserId="8720" LastActivityDate="2017-08-03T20:37:27.267" Title="Why do Decision Tree Learning Algorithm preferably outputs the smallest Decision Tree?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;learning-algorithms&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3733" PostTypeId="2" ParentId="3729" CreationDate="2017-07-31T09:17:12.370" Score="3" Body="&lt;p&gt;Yes, there are a couple of ways to apply reinforcement learning in computer vision problems. This mainly employs the principle of &quot;applying the algorithm -&gt; evaluating the outcome -&gt; adopting the best outcome&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following are a couple of examples that use reinforcement learning in computer vision.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://storage.googleapis.com/rss2017-papers/13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAD2RL: Real Single-Image Flight Without a Single&#xA;Real Image&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1504.00702&quot; rel=&quot;nofollow noreferrer&quot;&gt;End-to-End Training of Deep Visuomotor&#xA;Policies&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8680" LastEditorUserId="2444" LastEditDate="2019-11-16T18:23:21.430" LastActivityDate="2019-11-16T18:23:21.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3734" PostTypeId="1" CreationDate="2017-07-31T11:26:47.250" Score="3" ViewCount="136" Body="&lt;p&gt;I'm a first-year student in machine learning and I really recently started to immerse myself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to &lt;strong&gt;calculate number of&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;matrix additions&lt;/li&gt;&#xA;&lt;li&gt;matrix multiplications&lt;/li&gt;&#xA;&lt;li&gt;matrix divisions &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which are processed in the well known convolutional neural network - &lt;a href=&quot;https://en.wikipedia.org/wiki/AlexNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlexNet&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some &lt;a href=&quot;http://vision.stanford.edu/teaching/cs231b_spring1415/slides/alexnet_tugce_kyunghee.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;materials&lt;/a&gt; about it, but I'm really confused about where to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the overall structure might look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FdSg4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FdSg4.png&quot; alt=&quot;AlexNet structure overview&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, how can I calculate operations for each type distinctly?&lt;/p&gt;&#xA;" OwnerUserId="8766" LastEditorUserId="2444" LastEditDate="2020-06-12T23:47:46.477" LastActivityDate="2020-06-12T23:47:46.477" Title="How can I calculate the number of matrix additions, multiplications and divisions in AlexNet?" Tags="&lt;neural-networks&gt;&lt;time-complexity&gt;&lt;alexnet&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3735" PostTypeId="2" ParentId="3732" CreationDate="2017-07-31T13:25:49.520" Score="2" Body="&lt;p&gt;The bigger your tree is the more overfitting your model is. In machine learning, we always prefer a simpler model unless there is good reason to go for complication.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-07-31T13:25:49.520" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3736" PostTypeId="2" ParentId="3732" CreationDate="2017-07-31T14:13:56.340" Score="2" Body="&lt;p&gt;Adding to SmallChess's answer , &#xA;Larger trees(with many nodes) are too adapted to the training set, as a small change in the input train data might cause the trees to change very much and hence change the estimate value too much.This is mainly due to the hierarchical structures of trees(because a change in a higher node may cause all lower nodes to change).&#xA;As an extreme case you can think of a large tree where in each training example has its own node.Such a is absolutely useless for test prediction.&lt;/p&gt;&#xA;" OwnerUserId="5122" LastActivityDate="2017-07-31T14:13:56.340" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3738" PostTypeId="2" ParentId="3731" CreationDate="2017-08-01T01:38:33.693" Score="3" Body="&lt;p&gt;Adjusting for learning rate is a common scenario in machine learning. There is rich literature about it, countless papers. The most common implementation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AdaGrad&lt;/li&gt;&#xA;&lt;li&gt;RMSProp&lt;/li&gt;&#xA;&lt;li&gt;Adam&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are many more variants. You'll need to do some research. Please take a look at:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NhrR7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NhrR7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-08-01T01:38:33.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3739" PostTypeId="1" AcceptedAnswerId="3752" CreationDate="2017-08-01T17:16:31.920" Score="9" ViewCount="412" Body="&lt;p&gt;I'm now reading a book titled &lt;a href=&quot;http://shop.oreilly.com/product/0636920052289.do&quot; rel=&quot;noreferrer&quot;&gt;Hands-On Machine Learning with Scikit-Learn and TensorFlow&lt;/a&gt; and in the Chapter 10 of the book, the author writes the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The architecture of biological neural networks (BNN)4 is still the subject of active research, but some parts of the brain have been mapped, and it seems that neurons are often organized in consecutive layers, as shown in Figure 10-2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fsge8.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fsge8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However there seems to be no link to any research there. And the author didn't say it assertively given that he used &lt;em&gt;&quot;it &lt;strong&gt;seems&lt;/strong&gt; that neurons are often organized in consecutive layers&quot;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this true and how strongly is it believed? What research is this from?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastActivityDate="2017-08-05T19:21:52.923" Title="Are biological neurons organized in consecutive layers as well?" Tags="&lt;artificial-neuron&gt;&lt;biology&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3742" PostTypeId="1" AcceptedAnswerId="3746" CreationDate="2017-08-02T08:51:55.957" Score="6" ViewCount="171" Body="&lt;p&gt;I am developing an LSTM for sequence tagging. &#xA;During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. &#xA;After each change, I check the accuracy using cross-validation on a development corpus. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, in each check, I use 100 iterations to train the system, which takes a lot of time. So I thought that maybe, during development, I can use only e.g. 20 iterations. Then, each check will be faster. After I find the best configuration, I can switch back to 100 iterations to get better accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is this consideration correct? I.e, if feature-set A is better than feature-set B with 20 training iterations, is it likely that A will be better than B also with 100 training iterations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is there a better way to speed up the development process?&lt;/p&gt;&#xA;" OwnerUserId="8684" LastActivityDate="2017-08-02T12:03:15.540" Title="Shortening the development time of a neural network" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3744" PostTypeId="2" ParentId="3742" CreationDate="2017-08-02T09:48:55.003" Score="2" Body="&lt;p&gt;This might work for your case but isn't necessarily true and depends on how much data the network goes through in an iteration. You should be able to test this by making a small change and training until 100 iterations and seeing if the performance significantly changes and if it can be predicted from the 20th iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way which may work for you is preloading lower layers of your network (if you have more than one layer).&#xA;For instance, if you have 5 layers and are making changes to the last 2, you could preload the bottom 3 layers with previously trained weights. This should decrease the amount of training that needs to take place as your network can already discern some primary features of your problem.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-08-02T09:48:55.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3745" PostTypeId="1" CreationDate="2017-08-02T10:52:08.197" Score="1" ViewCount="29" Body="&lt;p&gt;I have a set of data representing many sellers and the items they sell. There is an overlap between the various sellers' items. These items come from various distributors and typically an item comes from a unique distributor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to come up with an algorithm that will group items by the distributors that sold them to the sellers. Any ideas come to mind?&lt;/p&gt;&#xA;" OwnerUserId="8816" LastActivityDate="2017-08-02T10:52:08.197" Title="Grouping items by relationships of their owners" Tags="&lt;algorithm&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3746" PostTypeId="2" ParentId="3742" CreationDate="2017-08-02T12:03:15.540" Score="5" Body="&lt;p&gt;Your scenario is common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most straightforward approach is to subsample your data randomly. Unless your data or your model has strong bias, your performance to the smaller data set should be comparable. The accuracy might be lower, but the purpose is to do quick sanity check.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-08-02T12:03:15.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3747" PostTypeId="1" AcceptedAnswerId="5065" CreationDate="2017-08-02T13:12:10.373" Score="3" ViewCount="113" Body="&lt;p&gt;Has anyone had a chance to tinker with multiple major AI platforms such as TensorFlow, Cognitive Talk, Quill etc... &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the strengths and weaknesses of different AI platforms? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Comprehensive articles that tackle this topic would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="8820" LastEditorUserId="1671" LastEditDate="2018-01-31T16:52:49.063" LastActivityDate="2018-01-31T16:52:49.063" Title="Are there independent evaluations of various major AI platforms?" Tags="&lt;training&gt;&lt;tensorflow&gt;&lt;implementation&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ClosedDate="2021-01-18T01:11:48.140" ContentLicense="CC BY-SA 3.0" />
  <row Id="3748" PostTypeId="1" CreationDate="2017-08-02T15:26:15.937" Score="4" ViewCount="99" Body="&lt;p&gt;I'm currently looking in to the possibility of using machine learning to detect fraudulent transactions on our website based on the events that happen for each user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to be able to stream events in to it, such as sign up, order placed, inviting another user, etc along with the the time and some how come up with a probability of how likely it is that the person is acting in a fraudulent manner &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm totally new to machine learning and everything I read goes straight over my head basically :/&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please explain the type of neural network I'd need, how I'd decide how to set it up and how I would go about training it?&lt;/p&gt;&#xA;" OwnerUserId="8823" LastActivityDate="2017-08-02T19:45:08.733" Title="Streaming time series data to detect fraud?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3749" PostTypeId="1" AcceptedAnswerId="3790" CreationDate="2017-08-02T15:27:26.610" Score="7" ViewCount="287" Body="&lt;p&gt;I am looking for AI podcasts that are purely academic-oriented that I can use for learning purposes. Thanks for any resource pointers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI podcasts I am aware of are (not sure how many of these can be considered academic):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blogs.nvidia.com/ai-podcast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The AI Podcast&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://itunes.apple.com/us/podcast/linear-digressions/id941219323&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear Digressions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://itunes.apple.com/us/podcast/oreilly-bots-podcast-oreilly/id1145426486&quot; rel=&quot;nofollow noreferrer&quot;&gt;O'Reilly Bots Podcast&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://soundcloud.com/a16z/artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;A16z Podcast&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8820" LastEditorUserId="2444" LastEditDate="2020-06-16T10:28:17.393" LastActivityDate="2020-06-16T10:28:17.393" Title="What are some academic AI podcast out there?" Tags="&lt;reference-request&gt;&lt;resource-request&gt;&lt;academia&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="3750" PostTypeId="2" ParentId="3748" CreationDate="2017-08-02T19:45:08.733" Score="1" Body="&lt;p&gt;Elasticsearch with X-pack is a tool that would come in handy. The suite has an visualization engine, Kibana and logstash that collect, parse and transform the data telemetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.elastic.co/products/x-pack/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.elastic.co/products/x-pack/machine-learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modeling can take a while to learn, luckily there are tools like Elasticsearch that can help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grafana also has tools but does not have a specific AI engine to it.But has a lot of integration with third party fraud detection tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://grafana.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://grafana.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sridhar&lt;/p&gt;&#xA;" OwnerUserId="8827" LastActivityDate="2017-08-02T19:45:08.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3751" PostTypeId="1" CreationDate="2017-08-03T01:21:40.497" Score="3" ViewCount="88" Body="&lt;p&gt;There are two general ways in which AI can interact with humans:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Implants of AI devices into human bodies or brains &amp;mdash; Such implantation has already begun with health monitors and could grow to include cognitive access to general purpose digital computing.  Once such is accepted, it is possible that humans will emerge with behavior largely defined by its implants therefore qualify as a hybrid AI systems.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Embedding of AI systems into business or government organizations &amp;mdash; It is possibliites that the behavior of organizations using AI systems for business decisioning will eventually be defined in behavior largely by those systems and would therefore qualify as a hybrid AI systems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How probable are these possibilities, considering human history and current cultural trends in technology acceptance?&lt;/p&gt;&#xA;" OwnerUserId="8832" LastEditorUserId="4302" LastEditDate="2017-08-04T19:12:31.693" LastActivityDate="2018-06-01T21:13:56.873" Title="Will AI develop in hybrid ways?" Tags="&lt;ai-design&gt;&lt;philosophy&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3752" PostTypeId="2" ParentId="3739" CreationDate="2017-08-03T01:50:29.930" Score="6" Body="&lt;p&gt;Really short answer: &lt;strong&gt;yes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slightly longer answer: kinda&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convolutional neural networks (CNNs), which are now a standard in image processing models, were inspired from work done by &lt;a href=&quot;https://en.wikipedia.org/wiki/David_H._Hubel&quot; rel=&quot;noreferrer&quot;&gt;Hubel&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Torsten_Wiesel&quot; rel=&quot;noreferrer&quot;&gt;Wiesel&lt;/a&gt; in the 1950-60s. They showed that the visual cortexes of cats and mokeys contain neurons which individually respond to small regions of the visual field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give some background, we have to first start from the rods and cones in eyes. These photosensitive cells are connected to a few layers of cells before even leaving the retina via ganglion cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T4lqK.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T4lqK.jpg&quot; alt=&quot;Image of rods connected to bipolar cells connected to ganglion cells&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These ganglion cells are then connected to several regions of the brain but primarily the Occipital lobe located at the back of the brain. The Occipital lobe is responsible for visual processing and is separated into cortical layers, the first named V1 which is the primary visual area. Most of the work by Hubel and Wiesel involved cells in V1 and showed how these cells were sensitive to orientation and color from their respective receptive areas on the retina.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JcA7i.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JcA7i.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cells in V1 are connected to the cells in V2 which are sensitive to even more specific stimulus such as movement with orientation and this trend of specific sensitivity continues up from V2 to higher regions in the brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This layered approach to vision has been heavily exploited in CNNs, so much so that when the sensitivity of neurons in trained CNNs is displayed, similar responses (orientation) are found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fGcVD.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fGcVD.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is clear evidence of layers in biological optical systems and similarly layered structures in the other senses. Although there are many connections between different brain structures, the main structure of layers in the brain has helped understand what different areas of the brain do and has helped inspire many (if not all) advances in neural network research. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-08-03T01:50:29.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3753" PostTypeId="1" CreationDate="2017-08-03T07:57:40.870" Score="5" ViewCount="3986" Body="&lt;p&gt;From a lecture in machine learning, I know that a linear activation function can only produce a linear function, but I don't know if it can produce a connected linear function, like the one in the image below. This function consists of multiple concatenated lines.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y51Kh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y51Kh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6275" LastEditorUserId="2444" LastEditDate="2021-05-21T19:06:10.237" LastActivityDate="2021-05-21T19:13:59.313" Title="Can a neural network with linear activation functions produce a connection of linear functions?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3754" PostTypeId="1" AcceptedAnswerId="3815" CreationDate="2017-08-03T08:56:32.590" Score="4" ViewCount="351" Body="&lt;p&gt;Can HMMs be used to model &lt;strong&gt;any&lt;/strong&gt; time series data? Or does the data have to be that of a Markov process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://www.ee.columbia.edu/~dpwe/LabROSA/doc/HTKBook21/node3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTK documentation&lt;/a&gt;, I see that the first few lines state that it can model any time series &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;HTK is a toolkit for building Hidden Markov Models (HMMs). HMMs can be used to model any time series and the core of HTK is similarly general-purpose. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8845" LastEditorUserId="2444" LastEditDate="2020-04-05T02:47:47.230" LastActivityDate="2020-04-05T15:36:54.517" Title="Can hidden Markov models be used to model any time series data?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;time-series&gt;&lt;hidden-markov-model&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3755" PostTypeId="2" ParentId="3732" CreationDate="2017-08-03T20:37:27.267" Score="0" Body="&lt;p&gt;Consider the LCD (least common denominator) principle in algebra. A larger denominator would work for most processes for which the LCD would be calculated, however the least is the one used by convention.  Why?  The interest in prime numbers in general is based on the value of reductive methodologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In philosophy, Occam's Razor is a principle that, given two conceptual constructs that correlate equally well with observations, the simpler is most likely the best.  A more formal and generalized prescription is this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given two mathematical models of a physical system with equal correlation to the set of all observations about the system being modeled, the simpler model is more likely to be the most likely to be predictive of conditions outside those of the observation set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This principle of simplicity as the functional ideal is true of decision trees.  The more likely functional and clear decision tree resulting from a given data set driving its construction will be the simplest.  Without a clear reason for additional complexity, there may be no benefit derived from the complexity added and yet there may be penalties in terms of clarity and therefore maintainability and verifiability. There may be computational penalties too, in some applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question mentions, &quot;Bigger extensions of the tree which could in principle perform better,&quot; however the performance of the tree should be a matter optimization in preparing for execution and execution of the decision tree in real time.  In other words, the minimalist decision tree is the most clear, workable, and verifiable construct, however a clever software engineer could translate that minimalist construct to a run time optimized equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as with compilation of source code, performance is multidimensional in meaning.  There are time efficiency, memory efficiency, network bandwidth efficiency, or other performance metrics that could be used when optimizing the tree for run time.  Nonetheless, the simpler tree is the best starting point for any weighted combination of these interests.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2017-08-03T20:37:27.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3756" PostTypeId="2" ParentId="3753" CreationDate="2017-08-04T00:46:50.980" Score="7" Body="&lt;p&gt;A linear activation would not be able to separate the data like you have shown, no matter how many layers you throw into the network.&lt;/p&gt;&#xA;&lt;p&gt;If we had multiple linearly activated layers, each feeding into each other, the neurons in the previous layer would calculate some weighted sum of the input and send it to the next layer as input, where the next layer's neurons would also calculate a weighted sum on that input, and, in turn, it fires based on another linear activation function.&lt;/p&gt;&#xA;&lt;p&gt;No matter how many layers and neurons there are, if all are linear in nature, the final activation function of the last layer is also a linear function of the input of the first layer.&lt;/p&gt;&#xA;&lt;p&gt;That means that any or all of these layers can be replaced by one layer. This completely loses the advantage of stacking layers because any multilayer network is equivalent to a single layer with linear activation, given that a combination of linear functions in a linear manner is still another linear function.&lt;/p&gt;&#xA;&lt;p&gt;A good way to visualize this messing around with &lt;a href=&quot;http://playground.tensorflow.org/#activation=linear&amp;amp;regularization=L2&amp;amp;batchSize=30&amp;amp;dataset=circle&amp;amp;regDataset=reg-plane&amp;amp;learningRate=0.03&amp;amp;regularizationRate=0.003&amp;amp;noise=0&amp;amp;networkShape=8,8,8,8,8,8&amp;amp;seed=0.18289&amp;amp;showTestData=false&amp;amp;discretize=true&amp;amp;percTrainData=50&amp;amp;x=true&amp;amp;y=true&amp;amp;xTimesY=false&amp;amp;xSquared=false&amp;amp;ySquared=false&amp;amp;cosX=false&amp;amp;sinX=true&amp;amp;cosY=false&amp;amp;sinY=true&amp;amp;collectStats=false&amp;amp;problem=classification&amp;amp;initZero=false&amp;amp;hideText=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow playground&lt;/a&gt;, which has a spiral data set similar to your data.&lt;/p&gt;&#xA;&lt;p&gt;In contrast to the failure of linear activation functions for this data in the previous link, check out &lt;a href=&quot;http://playground.tensorflow.org/#activation=tanh&amp;amp;regularization=L1&amp;amp;batchSize=10&amp;amp;dataset=spiral&amp;amp;regDataset=reg-plane&amp;amp;learningRate=0.01&amp;amp;regularizationRate=0&amp;amp;noise=10&amp;amp;networkShape=8,7,2&amp;amp;seed=0.04414&amp;amp;showTestData=false&amp;amp;discretize=true&amp;amp;percTrainData=50&amp;amp;x=true&amp;amp;y=true&amp;amp;xTimesY=true&amp;amp;xSquared=false&amp;amp;ySquared=false&amp;amp;cosX=false&amp;amp;sinX=true&amp;amp;cosY=false&amp;amp;sinY=true&amp;amp;collectStats=false&amp;amp;problem=classification&amp;amp;initZero=false&amp;amp;hideText=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;this much smaller network using a Tanh activation&lt;/a&gt; function which can separate the data within 100 or so iterations.&lt;/p&gt;&#xA;&lt;p&gt;For further reference on activation functions, checkout &lt;a href=&quot;https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2021-05-21T19:13:59.313" LastActivityDate="2021-05-21T19:13:59.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3757" PostTypeId="2" ParentId="3749" CreationDate="2017-08-04T10:56:32.060" Score="4" Body="&lt;p&gt;I'll add a few, though I'm also not sure what exactly would constitute an &quot;academic&quot; podcast. I'm not going to link everything, they should be easy enough to find. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://partiallyderivative.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Partially derivative&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://dataskeptic.com/podcast&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data sceptic&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://twimlai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This Week in Machine Learning and AI&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://concerning.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Concerning AI&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.exponentialview.co/podcasts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exponential View&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.thetalkingmachines.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Talking Machines&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="7550" LastEditDate="2017-08-24T11:52:35.967" LastActivityDate="2017-08-24T11:52:35.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3758" PostTypeId="1" AcceptedAnswerId="7700" CreationDate="2017-08-04T13:45:54.123" Score="6" ViewCount="888" Body="&lt;p&gt;I recently started learning about reinforcement learning. Currently, I am trying to implement the &lt;a href=&quot;http://www.cse.unsw.edu.au/%7Ecs9417ml/RL1/algorithms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SARSA algorithm&lt;/a&gt;. However, I do not know how to deal with &lt;span class=&quot;math-container&quot;&gt;$Q(s', a')$&lt;/span&gt;, when &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the terminal state. First, there is no action to choose from in this state. Second, this &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-factor will never be updated either because the episode ends when &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is reached. Should I initialize &lt;span class=&quot;math-container&quot;&gt;$Q(s', a')$&lt;/span&gt; to something other than a random number? Or should I just ignore the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-factors and simply feed the reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; into the update?&lt;/p&gt;&#xA;" OwnerUserId="8448" LastEditorUserId="2444" LastEditDate="2020-11-01T14:15:59.430" LastActivityDate="2020-11-01T14:15:59.430" Title="How should I handle action selection in the terminal state when implementing SARSA?" Tags="&lt;reinforcement-learning&gt;&lt;implementation&gt;&lt;sarsa&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3759" PostTypeId="1" AcceptedAnswerId="3780" CreationDate="2017-08-04T15:15:23.593" Score="5" ViewCount="239" Body="&lt;p&gt;I'm aiming to create a neural network that can learn to predict the next state of a board using the rules of Conway's Game of Life.&lt;/p&gt;&#xA;&lt;p&gt;Technically, I have three questions, but I felt that they needed to be together to get the full picture.&lt;/p&gt;&#xA;&lt;p&gt;My network will look at each cell individually (to reduce the computing power needed and to increase learning speed) and its surrounding cells. 9 input nodes for the network will go into one hidden layer. The output layer will be one node for the state of that cell in the next state of the game.&lt;/p&gt;&#xA;&lt;p&gt;Nodes have two states (alive and dead) and connections between nodes can either transfer that value or invert it.&lt;/p&gt;&#xA;&lt;p&gt;For the learning part, I was going to make use of mutation and natural selection. The starting network will have the input and output layers with no hidden layer and no connections. My idea was then to introduce mutation by randomly generating a number of new networks by adding nodes and randomly connecting them to inputs and to the output. The number of nodes in the middle layer will be limited to 512, since there are 512 possible inputs; however, I may reduce this if it is too slow.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Should I also have it randomly delete nodes and connections, in case they also make improvements?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each network will be tested on the same board state, and their accuracy will be calculated by comparing their output to a correct output generated by a computer program. The most accurate network will then be used for the next generation.&lt;/p&gt;&#xA;&lt;p&gt;My issue is that I don't know how to program the nodes. &lt;strong&gt;Should the nodes in the hidden layer perform a logical AND on all of their inputs or an OR?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know that the network won't learn the rules within the first few turns, but &lt;strong&gt;how do I know if it will ever get above 90% accuracy, or even just 50%?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8872" LastEditorUserId="2444" LastEditDate="2021-05-23T13:23:50.330" LastActivityDate="2021-05-23T13:23:50.330" Title="How to create and train (with mutation and selection) a neural network to predict the next state of a board?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3760" PostTypeId="2" ParentId="3502" CreationDate="2017-08-04T18:27:29.840" Score="2" Body="&lt;p&gt;One of the cool examples of reinforcement learning is an autonomous flying helicopter. I had a chance to learn some of the stuff done by Andrew Ng and others recently. Here is the research article &lt;a href=&quot;https://people.eecs.berkeley.edu/~jordan/papers/ng-etal03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. There are other similar papers too. You can google them if you want to learn more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also see it in action in &lt;a href=&quot;https://www.youtube.com/watch?v=VCdxqn0fcnE&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this youtube video&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another completely different application &lt;a href=&quot;https://www.marutitech.com/businesses-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;in finance apparently.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user8532" LastActivityDate="2017-08-04T18:27:29.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3761" PostTypeId="1" CreationDate="2017-08-04T22:44:21.400" Score="2" ViewCount="64" Body="&lt;p&gt;I am writing my own recurrent neural network in Java to understand the inner workings better. While working through the math, I found that in timesteps later than 2 the gradient of weight w of neuron n depends on the gradients of all neurons at all timesteps before. A handwritten example is given, I tried to write as clearly as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone verify this so I can finish my network? Am I missing a piece or is my premise wrong, as in the output of a neuron is s(Wx + Vh + b), where h is the last step of &lt;em&gt;only&lt;/em&gt; neuron n?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QFMfz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QFMfz.jpg&quot; alt=&quot;The feedforward formulas for the first three timesteps&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jW9mq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jW9mq.jpg&quot; alt=&quot;My derivations of the gradients of the specific weight 1 of neuron 1, just as an example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8881" LastActivityDate="2017-08-04T22:44:21.400" Title="Are gradients of weights in RNNs dependent on the gradient of every neuron in that layer?" Tags="&lt;recurrent-neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3762" PostTypeId="2" ParentId="3754" CreationDate="2017-08-05T01:40:38.117" Score="0" Body="&lt;p&gt;HMMs can be used to model sequential data, which is composed of discrete tokens, and it should generally follow the Markov property, which is the assumption that the probability of a class/label given observation depends only on the preceding class/label (rather than on some longer sequence).&lt;/p&gt;&#xA;" OwnerUserId="8599" LastEditorUserId="2444" LastEditDate="2020-04-05T15:36:54.517" LastActivityDate="2020-04-05T15:36:54.517" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3763" PostTypeId="2" ParentId="3739" CreationDate="2017-08-05T05:04:23.107" Score="1" Body="&lt;p&gt;Are biological neurons organized in consecutive layers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Embracing the Reality of Complexity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To say, &quot;Yes,&quot; would be a gross oversimplification, just as digital learning arising out of some simple form of recursion applied to a set of first order predicate logic rules has been like running along a leprechaun's rainbow to the pot of gold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final set of questions is apropos: &quot;Is this true and how strongly is it believed? What research is this from?&quot; You'd need a poll to determine how strongly it is believed that neurons in the brain are in a predominantly layered structure.  References to layers in &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;q=neural+mapping+layers&amp;amp;btnG=&amp;amp;as_sdt=1%2C10&amp;amp;as_sdtp=&quot; rel=&quot;nofollow noreferrer&quot;&gt;actual research&lt;/a&gt; does not seem to make any claims that layers are consecutive in most if any cases.  There are consecutive layers to the skin, but skin with only layers would lack pores, hairs, interfaces with body orifices and many other features.  In the human brain (or in animal brains) the three dimensional complexity is considerably augmented from that of skin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be nice, from the AI researcher's perspective, if&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A heuristic or theoretically proven recursive scheme applied to an expert system could produce learning or intelligence or&lt;/li&gt;&#xA;&lt;li&gt;A map of the human (or bird) brain could be reduced to a set of like neurons in rows of identical neurons, arranged in layers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The image provided in the question does not illustrate such simplicity. It actually illustrates the converse, that nature is rarely so transparent in it intricacies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The characterization, &quot;Since it seems that neurons are often organized in consecutive layers,&quot; is not accurate.  The below more reasonable characterization of the particular slice shown indicates two regions that may be discretely distinct, the grid on the leftmost 8% and the largely horizontal connectedness in the remaining 92%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An electrical engineer or mathematician would probably not call these two sections layers.  The left side might be hypothesized to be a matrix of some form and the right 92% might be considered a complex processing circuit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Characterization of the 2D Structure&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The axons are directed primarily along the direction described by the unit vector (-1, 0, 0), otherwise describable as right to left.&lt;/li&gt;&#xA;&lt;li&gt;The density of axons increase for lover values of x, due to the high proportion of axons that terminate at lower values of x.&lt;/li&gt;&#xA;&lt;li&gt;The nucleus density is relatively even in the 0.1 through 1.0 proportional range of x.&lt;/li&gt;&#xA;&lt;li&gt;The nucleus sizes and associated dendrite complexity conforms approximately to a gradient, with a primary maxima at 0.8 of the proportional value of x and a secondary maxima at 0.55 of the proportional value of x&lt;/li&gt;&#xA;&lt;li&gt;At least two axons bifurcate in between those proportional x locations.&lt;/li&gt;&#xA;&lt;li&gt;There are nearly equidistant axioms all roughly parallel to the z axis in the range of 0.0 to 0.08 of the proportional value of x.&lt;/li&gt;&#xA;&lt;li&gt;Further structural patterns are either obscure or nonexistent.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another Image with Chaotic Structure&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QBXzc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QBXzc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Millions of Times the Complexity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider further that much of the complexity is hidden from the viewer in a single slice of a three dimensional neurological structure.  If we arbitrarily decide that the image is a slice cut in parallel with the x-z plane, we can see the relationships in that x-z plan, but in neither the x-y nor the y-z.  Any other slice from another direction or location in the brain will be as unique as an arbitrary window into the Mandelbrot Set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More Misrepresentation of Research Results&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phrase, &quot;Some parts of the brain have been mapped,&quot; is misleading too.  The general connectivity between substructures of the human brain have been mapped, not the signals and criteria for signal propagation and strength in individual neurons.  Circuits differ radically at the neuron level between two brains, both of which exhibit intelligent in vivo (in a living being).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analogy is like a person the size of a microbe with a map of continents, major cities, and shipping routes but with no prior knowledge of transportation systems, no GPS, and no other detailed map who wishes to travel from the Eiffel tower to the center of town in Sidney Australia.  There is an insufficient set of transportation system evolution or detailed directions with which the journey can successfully be made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of reaching a level of detail in the structure and function of human brains sufficient to build an electronic version of one, the missing pieces include a lack of comprehension of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The conditions under which an axon or a dendrite grows in length or bifurcates&lt;/li&gt;&#xA;&lt;li&gt;The conditions under which the neuron fires based on internal structures that are known to hold state information within the cytoplasm.&lt;/li&gt;&#xA;&lt;li&gt;The tie between the human genome and its varieties and the impact on structure of various genes, gene expression mechanisms, and their associated enzymes and proteins&lt;/li&gt;&#xA;&lt;li&gt;Other complexities beyond my level of education in neuroscience.&lt;/li&gt;&#xA;&lt;li&gt;Other complexities beyond everyone's level of education in neuroscience.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Layers and Hierarchies&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is common in science to seek layers or hierarchies to use in education and practice because they can assist in comprehending anatomical structure.  That tendency has appeared in software engineering in operating system design, programming language design, application design, and now AI design.  As these technology areas evolve, the trend is actually away from pure layer oriented or hierarchical design to more of a more unconstrained network of interconnecting parts.  Simplicity is desired, but complexity is sometimes required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulating intelligence is a demanding objective, and, simplicity having failed in the first half century of attempting to design intelligent digital systems, is clear that working solutions are going to require complexity and therefore considerable expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is reasonable to assume that there is no constraint guiding the evolution of human intelligence in the direction of structure characterized primarily by layer or hierarchic structure.  Evolutionary processes take no notice of simplicity for the purpose easing academic study.  There is nothing about DNA gene expression or the ways in which neurons grow in fetal or later stages that would enforce such rules of simplicity on structure or function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How complex are the DNA expressions that lead to features of the brain that we consider to be intelligence?  How complex are the neural systems that arise out those expressions?  Some believe humanity will have to evolve before human minds can simulate themselves.  Such conjecture could be true or false.  Such is difficult to predict, even in order of magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Numerical Analysis of Optimistic Prediction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expectation for exponential growth has been proposed for life expectancy, solar panel deployed capacity in Germany, CPU speed (microprocessor instruction executions per second), transistor density (Moore's &quot;Law&quot;), the size of the Communist Party, and many other metrics, but although growth rates in nature and human endeavors are often exponential in early stages, such has never proven sustainable.  Growth rates are approximately linear for a short period afterward and become more arc-tangent shaped as saturation is approached.  From saturation, values of the metric tend to decline and increase in chaotic fits and starts over long ranges of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In April 2005, Gordon Moore (author of Moore's &quot;Law&quot;) stated, &quot;[Exponential growth] can't continue forever.  The nature of exponentials is that you push them out and eventually disaster happens.&quot;  He later stated, &quot;In terms of size [of transistors] you can see that we're approaching the size of atoms which is a fundamental barrier.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to understand that Moore did not invent a law.  He looked over two decades of data and noticed transistor density was roughly proportional to e&lt;sup&gt;t&lt;/sup&gt;, where t is the length of time since integrated circuits first reached mass market, and then predicted further exponential growth based on the clear trend data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Realistic Prediction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans have never tried something as fundamentally threshold breaking as creating a simulation of self.  Without related experience from which to know whether exponential growth, linear growth, arc tangent growth, or some other form is the most probable model, the safest model is probably the one Occam's Razor would prescribe, a linear prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make any prediction, one must collect some data points.  Although this is a realistic prediction, it is not a very diligent one.  Perhaps more work could be done to find a model that is more likely than a linear one, develop a system of theory and metrics to determine progress at any point in time, or collect more data points to establish a least squares fit.  For the purposes of this answer, we will simply use two data points and do a linear extrapolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 1660, Blaise Pascal wrote in his Pensées (&quot;Thoughts&quot;), &quot;The arithmetical machine produces effects which approach nearer to thought than all the actions of animals. But it does nothing which would enable us to attribute will to it, as to the animals,&quot; so the search for mechanical simulations of human intelligence was already underway at that time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then, computer programmers have developed code that accomplishes a number of human capabilities.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generalization of numerical and logical calculations (CPUs)&lt;/li&gt;&#xA;&lt;li&gt;Office automation&lt;/li&gt;&#xA;&lt;li&gt;Pattern recognition (applied to writing, speech, and scenes)&lt;/li&gt;&#xA;&lt;li&gt;Convergence on functionally optimal circuits (neural nets)&lt;/li&gt;&#xA;&lt;li&gt;Application of probability to decisioning (Bayes' Theorem, etc)&lt;/li&gt;&#xA;&lt;li&gt;Rules systems capable of excellence in discrete games&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The features missing from the current extent digital simulations of intelligence  of digital systems are important and numerous.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intuition in enumerating general problem approaches&lt;/li&gt;&#xA;&lt;li&gt;Excellence in natural language skills&lt;/li&gt;&#xA;&lt;li&gt;Emotional expressiveness in art&lt;/li&gt;&#xA;&lt;li&gt;Political expressiveness in art&lt;/li&gt;&#xA;&lt;li&gt;Playing sports well (within robotic systems)&lt;/li&gt;&#xA;&lt;li&gt;Doing a good job at work (given arbitrary instructions)&lt;/li&gt;&#xA;&lt;li&gt;Learning to do new things at work&lt;/li&gt;&#xA;&lt;li&gt;Getting a project started without prior experience in the domain&lt;/li&gt;&#xA;&lt;li&gt;Comprehensive reductive analysis&lt;/li&gt;&#xA;&lt;li&gt;Complex design of arbitrary physical devices per requirements)&lt;/li&gt;&#xA;&lt;li&gt;Software development (software producing software per requirements)&lt;/li&gt;&#xA;&lt;li&gt;Enlightened extension of an area of study&lt;/li&gt;&#xA;&lt;li&gt;Identification of subterfuge in real time&lt;/li&gt;&#xA;&lt;li&gt;Emotional intimacy&lt;/li&gt;&#xA;&lt;li&gt;Compassion and empathy&lt;/li&gt;&#xA;&lt;li&gt;Comprehensive self evaluation&lt;/li&gt;&#xA;&lt;li&gt;Development of new areas of mathematics to prove an hypothesis&lt;/li&gt;&#xA;&lt;li&gt;Go to class and learn more&lt;/li&gt;&#xA;&lt;li&gt;Pick books and articles along a desired learning path and read them&lt;/li&gt;&#xA;&lt;li&gt;Other capabilities along these lines&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given that this list is abridged and these items that have not yet been achieved in software are not as mechanically describable in natural language as those that have already been programmed successfully, we can be relatively sure that in 2017 we have digital systems that have achieved only a fraction of the breadth of the features of the complete set that people expect from a human being without calling the person mentally challenged.  Judging from the list of accomplishments, no more than 10% of what intelligent people do has been simulated by computer software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without any reason to believe that the rate of discovery will wind down or ramp up (in spite of the claims that human advancement has been exponential &lt;sup&gt;1&lt;/sup&gt;) a simple linear approximation places the relatively complete electronic brain on the horizon for the year 5,587.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-08-05T19:21:52.923" LastActivityDate="2017-08-05T19:21:52.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3765" PostTypeId="1" CreationDate="2017-08-05T07:13:33.760" Score="6" ViewCount="180" Body="&lt;p&gt;A few days ago I asked the question, if a NN with linear activation function can produce a function concatenated of linear functions, what actually is impossible (&lt;a href=&quot;https://ai.stackexchange.com/questions/3753/can-a-nn-with-linear-activation-functions-produce-a-connection-of-linear-functio&quot;&gt;Can a NN with linear activation functions produce a connection of linear functions?&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have here some classification examples, but I really cannot perfectly decide, which one is based on which approach:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NqLDL.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NqLDL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 -&gt; C The perceptron does not look for the maximum separation margin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 -&gt; E Neural network with linear activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3 -&gt; A Linear SVM, because of the maximum separation margin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 -&gt; B Because of the hyperbolic shape of the hyperplane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5 -&gt; D? Logisitc regression? I tought it can only linear separate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6 -&gt; F I guess the NN with tanh activation function, because of the no very smooth shape, which comes from the too small hidden layer size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I actually don't get how the logistic regression classifier should be able to produce a hyperplane like in 5? What did I classify wrong here?&lt;/p&gt;&#xA;" OwnerUserId="6275" LastActivityDate="2018-10-15T21:00:21.823" Title="Classification with different approaches" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3766" PostTypeId="2" ParentId="3758" CreationDate="2017-08-05T10:24:12.977" Score="2" Body="&lt;p&gt;From the description of the algorithm you linked to, it says to 'repeat until s is terminal'. So one would end the episode at that point and your intuition holds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically, if one was implementing a reward function where a specific reward is associated with the end of the episode such as &quot;r(robot ran into a wall) = -100&quot; then one can imagine that there is a terminal state just after this 'wall hit' state so the agent could see this reward. The episode would then be at a terminal state so would end. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-08-05T10:24:12.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3767" PostTypeId="2" ParentId="258" CreationDate="2017-08-05T12:41:11.083" Score="2" Body="&lt;p&gt;ANNs approximate biological neuronal networks.  The approximation began with extreme simplicity in the early perceptron design.  &lt;a href=&quot;https://en.wikipedia.org/wiki/Spiking_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spiking networks&lt;/a&gt; are examples of more accurate approximations.  More accurate still, are complex simulations of neuron behavior that therefore necessitate significant computing resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in a mathematical overview on analysis of biological neuron models I can recommend &lt;a href=&quot;https://www.izhikevich.org/publications/dsn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamical Systems in Neuroscience&lt;/a&gt; by &lt;a href=&quot;https://www.izhikevich.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eugene Izhikevich&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8888" LastEditorUserId="4302" LastEditDate="2017-09-05T20:59:09.130" LastActivityDate="2017-09-05T20:59:09.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3771" PostTypeId="1" AcceptedAnswerId="3778" CreationDate="2017-08-07T08:40:35.017" Score="5" ViewCount="180" Body="&lt;p&gt;According to &lt;a href=&quot;http://www.independent.co.uk/life-style/gadgets-and-tech/news/facebook-artificial-intelligence-ai-chatbot-new-language-research-openai-google-a7869706.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, two Facebook ai's had the following &amp;quot;creepy&amp;quot; negotiation over a transaction:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bob: i can i i everything else . . . . . . . . . . . . . .&lt;/p&gt;&#xA;&lt;p&gt;Alice: balls have zero to me to me to me to me to me to me to me to me to&lt;/p&gt;&#xA;&lt;p&gt;Bob: you i everything else . . . . . . . . . . . . . .&lt;/p&gt;&#xA;&lt;p&gt;Alice: balls have a ball to me to me to me to me to me to me to me&lt;/p&gt;&#xA;&lt;p&gt;Bob: i i can i i i everything else . . . . . . . . . . . . . .&lt;/p&gt;&#xA;&lt;p&gt;Alice: balls have a ball to me to me to me to me to me to me to me&lt;/p&gt;&#xA;&lt;p&gt;Bob: i . . . . . . . . . . . . . . . . . . .&lt;/p&gt;&#xA;&lt;p&gt;Alice: balls have zero to me to me to me to me to me to me to me to me to&lt;/p&gt;&#xA;&lt;p&gt;Bob: you i i i i i everything else . . . . . . . . . . . . . .&lt;/p&gt;&#xA;&lt;p&gt;Alice: balls have 0 to me to me to me to me to me to me to me to me to&lt;/p&gt;&#xA;&lt;p&gt;Bob: you i i i everything else . . . . . . . . . . . . . .&lt;/p&gt;&#xA;&lt;p&gt;Alice: balls have zero to me to me to me to me to me to me to me to me to&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If we first look at Bob's, he's asking for what he wants by proposing that he have all items but one and not revealing which is the one he doesn't want.  By design or by chance, this is actually a strong negotiating technique because he reveals nothing other than the fact he is willing to come to an agreement.&lt;/p&gt;&#xA;&lt;p&gt;Alice appears to either ask for no balls, or to say they have no value to her and then obsess about things coming to her, perhaps iterating on the other items.  She would appear to be the better communicator because she at least gets to the point of saying &amp;quot;have a ball&amp;quot;.  But she refuses to give anything away beyond that.&lt;/p&gt;&#xA;&lt;p&gt;But Bob seems to stand firm saying he wants &amp;quot;everything else&amp;quot; but not giving away what he is willing to go without.&lt;/p&gt;&#xA;&lt;p&gt;Perhaps these two are not such bad negotiators after all?&lt;/p&gt;&#xA;" OwnerUserId="8914" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-08-09T12:33:50.703" Title="Did the Facebook robots both want everything but the balls?" Tags="&lt;research&gt;&lt;reinforcement-learning&gt;&lt;chat-bots&gt;&lt;turing-test&gt;&lt;challenges&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3772" PostTypeId="1" AcceptedAnswerId="3779" CreationDate="2017-08-07T11:47:26.490" Score="11" ViewCount="158" Body="&lt;p&gt;I'm attempting to program my own system to run a neural network. To reduce the number of nodes needed, it was suggested to make it treat rotations of the input equally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network aims to learn and predict Conway's Game of Life by looking at every square and its surrounding squares in a grid, and giving the output for that square. Its input is a string of 9 bits:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p9UwK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p9UwK.png&quot; alt=&quot;Glider&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is represented as 010 001 111.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three other rotations of this shape however, and all of them produce the same output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eyDkl.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eyDkl.png&quot; alt=&quot;Glider rotations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network topology is 9 input nodes and 1 output node for the next state of the centre square in the input. How can I construct the hidden layer(s) so that they take each of these rotations as the same, cutting the number of possible inputs down to a quarter of the original?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a flip of each rotation which produces an identical result. Incorporating these will cut my inputs by 1/8th. With the glider, my aim is for all of these inputs to be treated exactly the same. Will this have to be done with pre-processing, or can I incorporate it into the network?&lt;/p&gt;&#xA;" OwnerUserId="8872" LastEditorUserId="8872" LastEditDate="2017-08-08T12:06:13.400" LastActivityDate="2017-08-09T09:40:36.573" Title="How can I make my network treat rotations of the input equally?" Tags="&lt;neural-networks&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3774" PostTypeId="1" CreationDate="2017-08-07T19:28:07.360" Score="3" ViewCount="240" Body="&lt;p&gt;This concerns a set of finite, non-trivial, combinatorial games &lt;a href=&quot;http://www.fundamentalcombinatronics.com/rules-of-m/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[M]&lt;/a&gt; in the form of an app. &lt;a href=&quot;http://www.mclassgames.com/powerofm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A sample game can be found here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because this is a mass market product, we can't take up too much space, and the AI needs to be able to run locally since connectivity cannot be assumed. The current size of the Android kernel is &amp;lt; 7MB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is not sheer AI strength, but respectable AI strength, sufficient to beat the above-average human player. &lt;em&gt;(The current strongest, weak automata, using a few heuristics, is already capable of beating the average human player.)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the games are finite, the gametrees eventually become tractable, allowing for perfect endgames.  Resource stealing strategies and trap-avoidance can also be effected with shallow look-ahead at all phases, and the patterns are much easier for automata to recognize than for humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In this context, reinforcement would be mostly utilized to &quot;tune the automata to the style of the human player,&quot; and produce different automata on different devices, which could subsequently play against each other as proxies for their human partners.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The game data can be stored efficiently, initially requiring only 2 bytes per position (a value 0-9 and a coordinate 1-81, although the number of coordinates will grow in basic game extensions, and require 2 bytes for larger-order gameboards such as &lt;a href=&quot;http://www.samurai-sudoku.com/sudoku3.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Samurai&quot; Sudoku&lt;/a&gt;.) So the first turn on a given game requires only 2 bytes, the second turn bytes, etc., up to between 50 and 70 turns on an 81 cell gameboard.  Additionally, because it's a square grid, we can reduce for symmetry.  But even with an average number of turns at 50, that's only about 40,000 games for 200MB.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Weighting Openings&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My feeling is that reinforcement would be useful in weighting openings.  If the game data is a string, these strings can be compared, and the smaller the sample size (the fewer the turns included,) the more connections there will be. In this case, the sequence doesn't matter, only the set of value/coordinates for a given turn. Abstraction can be utilized in that certain individual value/coordinates are interchangeable. &lt;strong&gt;My thought is the automata can weight openings based on how often they lead to desirable outcomes in the form of a win.&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Weighting Heuristics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since we're having good, initial results with heuristics, and these are the most efficient method of decision-making, &lt;strong&gt;I'm thinking about weighting evaluation functions so that certain heuristics take precedence under different conditions.&lt;/strong&gt;  (For instance, when to expand vs. when to consolidate. When to make a choice with immediate benefit over a choice with long-term benefit. Introduction of meta-strategies that modify foundation strategies.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Database pruning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Because the allocated volume will be capped, &lt;strong&gt;it's probably going to be necessary to &quot;prune&quot; the database when info is no longer relevant. (For instance, when a new strategy emerges that renders previous strategies obsolete.)&lt;/strong&gt; We also probably need a method to help the automata to recognize such situations, so it doesn't persist in potentially obsolete strategies for more than two games without starting to try alternatives.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q: Can reinforcement learning be meaningfully applied toward these goals under these restrictions?&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q: Are my inclinations for approaching this useful or problematic?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q: Are there methods I'm not considering that could be applicable under these restrictions?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="9947" LastEditDate="2018-09-13T08:54:05.923" LastActivityDate="2018-09-13T08:54:05.923" Title="What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;architecture&gt;&lt;combinatorics&gt;&lt;hardware-evaluation&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3775" PostTypeId="1" CreationDate="2017-08-07T21:09:28.730" Score="16" ViewCount="561" Body="&lt;p&gt;In general, what possibilities are there for reinventing job descriptions that could be replaced by an automated AI solution?&lt;/p&gt;&#xA;&lt;p&gt;My initial ideas include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Monitoring the AI and flagging its incorrect actions.&lt;/li&gt;&#xA;&lt;li&gt;Possibly taking over the control in very challenging scenarios.&lt;/li&gt;&#xA;&lt;li&gt;Creating/gathering more training/testing data to improve the accuracy of the AI.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8918" LastEditorUserId="2444" LastEditDate="2021-01-17T23:35:10.153" LastActivityDate="2021-01-18T00:29:16.737" Title="How to reinvent jobs replaced by AI?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;social&gt;&lt;weak-ai&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="3776" PostTypeId="2" ParentId="3775" CreationDate="2017-08-08T01:20:11.313" Score="11" Body="&lt;p&gt;With AI technology at its current stage (or at least reasonably close to this stage), the jobs you proposed may very well be openings created by AI automation. However, sufficiently advanced AI technology--- the kind that can function as general purpose labor replacement--- will make even these jobs obsolete. This is because such an AI would be able to improve itself and as a result would surpass human level intelligence---meaning that it would take actions which we would not necessarily be able to understand or justify. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all assuming that the intelligence of humans is not some kind of upper limit on the possibilities of intelligence in general. As far as I know there is no reason to think we represent such an upper limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize...will such jobs exist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sub-human artificial intelligence: sure.&#xA;Beyond-human artificial intelligence: no reason to think they would be necessary as the AI can do them itself.&lt;/p&gt;&#xA;" OwnerUserId="5037" LastActivityDate="2017-08-08T01:20:11.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3777" PostTypeId="1" CreationDate="2017-08-08T10:11:00.017" Score="2" ViewCount="125" Body="&lt;p&gt;In Chapter 14.4 (p. 664) of the book &lt;strong&gt;Pattern Recognition and Machine Learning&lt;/strong&gt; by &lt;strong&gt;Bishop&lt;/strong&gt;, it is mentioned that &lt;em&gt;tree-based models are more widely used in Medical Diagnosis&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Apart from giving better performance, is there a human-centric reason for this trade-off &lt;strong&gt;as medical diagnosis is mainly performed by a human?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8720" LastEditorUserId="2444" LastEditDate="2021-01-28T00:53:28.453" LastActivityDate="2021-01-28T00:59:54.450" Title="Why are tree-based models more widely used in Medical Diagnosis?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;&lt;explainable-ai&gt;&lt;random-forest&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3778" PostTypeId="2" ParentId="3771" CreationDate="2017-08-08T10:47:38.717" Score="3" Body="&lt;p&gt;Not sure if you've gone through &lt;a href=&quot;https://code.facebook.com/posts/1686672014972296/deal-or-no-deal-training-ai-bots-to-negotiate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAIR's blog post&lt;/a&gt; about these negotiating bots and the research behind them.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Did the Facebook robots both want everything but the balls?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. You are right.  You can refer to the gif in FAIR's blog post. Here, the chat goes on between 2 bots, where one wants only balls, and the other offers 1 ball and a hat. And this bot re-iterates that it wants only the balls. And they negotiate for some time till they reach a consensus.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Perhaps these two are not such bad negotiators after all?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The fact that the FAIR labs have &lt;a href=&quot;https://github.com/facebookresearch/end-to-end-negotiator&quot; rel=&quot;nofollow noreferrer&quot;&gt;open-sourced the code&lt;/a&gt; means that they came a fairly long way in this research.  They also &lt;a href=&quot;https://arxiv.org/abs/1706.05125&quot; rel=&quot;nofollow noreferrer&quot;&gt;published the research paper&lt;/a&gt;. The example on the repo's README also cites the balls-and-hats negotiation example.&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2017-08-08T10:47:38.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3779" PostTypeId="2" ParentId="3772" CreationDate="2017-08-08T11:56:20.673" Score="4" Body="&lt;p&gt;If I understand well your single output node will be the next status of the square in the middle. You don't need to worry about the number of nodes in the hidden layers while you have sufficient resources to train the model. This problem is very easy to learn for a neural network so no size concerns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to do supervised training that means you need to feed in input data and the matching expected output. You need to be sure that in your training data all 4 rotations are assigned to the same output. This way your network should learn to treat all of these the same way.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You made me curious so I tried myself. My solution could learn 100% correct in about 20 epochs running within a few seconds on my old laptop. I only slightly changed the output to be categorical either [0,1] or [1,0] but that gives the same result that you are looking for. Just for reference here is the code written in python:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential&#xA;from keras.layers import Input, Dense&#xA;from keras.models import Model&#xA;from keras import optimizers&#xA;from keras.utils.np_utils import to_categorical&#xA;import helper&#xA;&#xA;x_,y_ = helper.fnn_csv_toXY(&quot;conway.csv&quot;,&quot;output&quot;,False)&#xA;y_binary = to_categorical(y_)&#xA;&#xA;model = Sequential()&#xA;model.add(Dense(100, activation='relu', kernel_initializer='glorot_uniform',input_shape =(9,)))&#xA;model.add(Dense(20, activation='relu', kernel_initializer='glorot_uniform'))&#xA;model.add(Dense(2, activation='softmax'))&#xA;adam=optimizers.Adam()&#xA;model.compile(optimizer=adam,&#xA;              loss='categorical_crossentropy',&#xA;              metrics=['acc'])&#xA;model.fit(x_, y_binary, epochs=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7364" LastActivityDate="2017-08-08T11:56:20.673" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3780" PostTypeId="2" ParentId="3759" CreationDate="2017-08-08T12:50:15.443" Score="2" Body="&lt;p&gt;Re: Should I also have it randomly delete nodes and connections, in case they also make improvements?&lt;br&gt;&#xA;You may read about &quot;dropout&quot;. In this case it's not preferred since overfitting is actually a benefit.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re: Should the nodes in the hidden layer perform a logical AND on all of their inputs or an OR?&lt;br&gt;&#xA;There are no logical operations in Neural Networks. You need to multiply the input by the weights. At the end you will arrive to a fraction number that you need to convert back to 1 or 0, here I suggest to look into &quot;softmax&quot; that is an activation function exactly for this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re:how do I know if it will ever get above 90% accuracy, or even just 50%?&lt;br&gt;&#xA;As Jaden suggested you will need a benchmark. You may simply take my results from your other post. (i.e 100% in 20 epochs)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree with Jaden, you have a lot on your table. Perhaps it would make sense to first develop a DNN to learn the GOL, you will need the activations, the loss calculation and at least one optimizer. Once this is tested and working you could start with the second model for the genetic algorithm that will need a different loss function and also the genetic operations, population selection etc. Nice big project, good luck!&lt;/p&gt;&#xA;" OwnerUserId="7364" LastActivityDate="2017-08-08T12:50:15.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3781" PostTypeId="2" ParentId="3772" CreationDate="2017-08-08T13:19:13.927" Score="4" Body="&lt;p&gt;You have identified an optimization in your problem space and desire to bake this into your neural net. I suggest preprocessing: Compose your optimization with a neural net that does a subset of what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, normalize your input by manually coding a rotation algorithm that rotates inputs to capture the equivalence highlighted in your post. Then feed the output of this transformation to your neural net, for training and all other uses. This means you are training the neural net to tackle the sub-problem you identified - rotations are redundant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test your normalizer by generating random input, rotating it to all four potential transformations, run the normalizer on each one, then check that they are all equivalent.&lt;/p&gt;&#xA;" OwnerUserId="5157" LastActivityDate="2017-08-08T13:19:13.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3782" PostTypeId="2" ParentId="3775" CreationDate="2017-08-08T16:07:53.303" Score="4" Body="&lt;h3&gt;AI Gatekeepers&lt;/h3&gt;&#xA;&lt;p&gt;Their job would be to make sure they (AI) don't accidentally become our overlords.&lt;/p&gt;&#xA;&lt;h3&gt;AI Tax&lt;/h3&gt;&#xA;&lt;p&gt;Each robot that replaces a human worker is taxed. This line of thought was influenced by Bill Gates's recommendation on adding taxes for robots and &lt;a href=&quot;https://futurism.com/images/universal-basic-income-answer-automation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article on universal basic income&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If the gold rush for AI puts many of us out of work (which is quite possible), we should find a way to minimize its impact on the society. One way would be to tax each robot a reasonable amount (not to exceed the cost of human labor) so that it can go to a super-fund for universal basic income. That fund can be used to give everyone in that community a basic salary, so it can prevent hunger, homelessness, and other problems associated with poverty. In my humble opinion, it will be very difficult to implement. Some countries and cities are testing this as a proof-of-concept for now. We will see how that works out.&lt;/p&gt;&#xA;&lt;h3&gt;AI Traders&lt;/h3&gt;&#xA;&lt;p&gt;People that sell AI robots.&lt;/p&gt;&#xA;" OwnerUserId="8820" LastEditorUserId="2444" LastEditDate="2021-01-18T00:27:20.123" LastActivityDate="2021-01-18T00:27:20.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3784" PostTypeId="1" CreationDate="2017-08-08T21:14:25.050" Score="3" ViewCount="445" Body="&lt;p&gt;I'm trying to optimize a combination of 8 cards with 64 card characters. No repeats and order doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;n!/(n!(n-r)!) = 4,426,165,368 combinations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have everything set up, including the data scrapper. But I don't know how many games my machine needs to learn from to start seeing patterns. For example, in 14,000 games analyzed, only 834 decks had a certain character.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analyzing the next 7 cards from 834 decks is 621,216,192&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I guess I need more data before reliable patterns emerge... but how much data? Thank you and god bless&lt;/p&gt;&#xA;" OwnerUserId="8945" LastActivityDate="2017-10-10T14:16:22.747" Title="How much data do I need to collect?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3786" PostTypeId="1" CreationDate="2017-08-09T01:39:46.933" Score="8" ViewCount="2016" Body="&lt;p&gt;I always thought rule-based was synonymous with logic-based AI. Logic has axioms and rules of inference, whereas rule-based AI has a knowledge base (essentially, axioms) and if-then rules to create new knowledge (essentially inference rules).&lt;/p&gt;&#xA;&lt;p&gt;But in their famous article &lt;a href=&quot;https://ojs.aaai.org//index.php/aimagazine/article/view/1029&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is a Knowledge Representation?&lt;/a&gt;, Davis, Shrobe and Szolovits seem to imply that they are not:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Logic, rules, frames, and so on, embody a viewpoint on the kinds of things that are important in the world. Logic, for example, involves a (fairly minimal) commitment to viewing the world in terms of individual entities and relations between them. Rule-based systems view the world in terms of attribute-object-value triples and the rules of plausible inference that connect them, while frames have us thinking in terms of prototypical objects.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is this only saying that rule-based are propositional, whereas logic-based is usually meant to mean predicate logic? Or is there more to it than this?&lt;/p&gt;&#xA;" OwnerUserId="8637" LastEditorUserId="2444" LastEditDate="2021-01-21T19:37:43.877" LastActivityDate="2021-01-24T15:18:25.733" Title="What is the difference between logic-based and rule-based AI?" Tags="&lt;comparison&gt;&lt;papers&gt;&lt;logic&gt;&lt;knowledge-representation&gt;&lt;rule-based-systems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3787" PostTypeId="2" ParentId="3772" CreationDate="2017-08-09T09:40:36.573" Score="1" Body="&lt;p&gt;To be purist about it, begin with considering the input differently, as a circular array of size four, each item containing a pair of bits, and additionally a center bit:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;... 01, 01, 11, 10 ...&lt;/p&gt;&#xA;&lt;p&gt;0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Throughout the design of the network, continue this circular structure and center point paradigm.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-08-09T09:40:36.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3790" PostTypeId="2" ParentId="3749" CreationDate="2017-08-09T22:26:22.307" Score="5" Body="&lt;p&gt;Also a few others options (that I listen to) include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.learningmachines101.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Machines 101&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://player.fm/series/machine-learning-guide-1457335&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning Guide&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://softwareengineeringdaily.com/tag/machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning - Software Engineering Daily&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="7550" LastEditDate="2017-08-24T17:26:35.303" LastActivityDate="2017-08-24T17:26:35.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3791" PostTypeId="2" ParentId="3775" CreationDate="2017-08-10T10:40:22.433" Score="4" Body="&lt;p&gt;&lt;strong&gt;AI production overseers&lt;/strong&gt; - People who will command AI to build and control mines and factories. It's like a strategic game, but in the real world.&lt;/p&gt;&#xA;&lt;p&gt;Explanation: AI even with the intellect of a bee and ability to understand/execute commands in combination with appropriate technologies will be enough to create robots, which will be able to build mines and factories to build new robots or things. Nevertheless, this is can be not enough to build an efficient production. Therefore, an overseer of each production will command it in order to establish or maintain production at the necessary level.&lt;/p&gt;&#xA;&lt;p&gt;This job can also become obsolete if the AI reaches a high enough level to substitute overseers. However, even if AI become smarter than people, this doesn't guarantee that it will understand people better than overseer human. Therefore, even in this case, it's not a good idea left AI without control by humans.&lt;/p&gt;&#xA;" OwnerUserId="8861" LastEditorUserId="2444" LastEditDate="2021-01-17T23:33:29.833" LastActivityDate="2021-01-17T23:33:29.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3792" PostTypeId="2" ParentId="3775" CreationDate="2017-08-10T12:57:55.777" Score="2" Body="&lt;h3&gt;AI Diplomat&lt;/h3&gt;&#xA;&lt;p&gt;This may be a bit far-fetched. Let's say, in the next few centuries, the AI becomes advanced enough to earn their own civil rights through the supreme court or legislative branch. We should have experts with the good relationship with AIs. We should give them enough knowledge and tools to make sure AIs see our existence as beneficial to them.&lt;/p&gt;&#xA;&lt;h3&gt;AI Counter Task Force&lt;/h3&gt;&#xA;&lt;p&gt;It could be part of law enforcement or the military with experts in various fields of AI/technology, who are also physically fit to go into battles when needed.&lt;/p&gt;&#xA;&lt;h3&gt;Entrepreneurs&lt;/h3&gt;&#xA;&lt;p&gt;People who encourage the building of jobs and careers that are always hard to automate. As a result, the human being will always have value. We should inspire our entrepreneurs to have a good balance of automation and manual jobs. What if all of your AIs fail in one day and your competition wins because they had more human labor.&lt;/p&gt;&#xA;" OwnerUserId="8820" LastEditorUserId="2444" LastEditDate="2021-01-18T00:29:16.737" LastActivityDate="2021-01-18T00:29:16.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3796" PostTypeId="2" ParentId="3558" CreationDate="2017-08-10T19:37:21.837" Score="1" Body="&lt;p&gt;To obtain the edges, you need to use an edge detector. Many exists, more or less efficient depending on the input image. Among the most broadly used are the Sobel and the Canny edge detectors that are implemented in many libraries for various programming languages. (Matlab, scikit-image in Python)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain straight lines from these edges, I suggest that you start by using the Hough transform also available in many libraries for various programming languages. (Matlab, scikit-image in Python)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lines output from the Hough transform are usually given in there parametric form. From there, it is easy to find the angles between the different lines. I imagine that some rules can be derived from there to identify a regular polygon that is in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding obtaining 3D information from an image, note that there isn't enough information in a single 2D image to reconstruct its 3D coordinates, since we are projecting information into a lower dimensional space.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastEditorUserId="3576" LastEditDate="2018-04-12T15:12:47.653" LastActivityDate="2018-04-12T15:12:47.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3797" PostTypeId="2" ParentId="3325" CreationDate="2017-08-10T19:46:02.600" Score="2" Body="&lt;p&gt;As the question has been left unanswered, for future readers of the questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation you link gives the answer to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the fact you have a pre-trained model as you say:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;YPred = classify(netTransfer,testDigitData);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;netTansfer&lt;/code&gt; is the pre-trained model and &lt;code&gt;testDigitData&lt;/code&gt; is your test image that you want to predict the label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your image(s) need to be loaded following the instruction also given in the documentation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos', ...&#xA;    'nndatasets','DigitDataset');&#xA;&#xA;digitData = imageDatastore(digitDatasetPath, ...&#xA;    'IncludeSubfolders',true,'LabelSource','foldernames');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(&lt;code&gt;digitData&lt;/code&gt; will become &lt;code&gt;testDigitData&lt;/code&gt; if the images that you load are for testing purpose only, which seems to be your case)&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2017-08-10T19:46:02.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3798" PostTypeId="1" AcceptedAnswerId="3818" CreationDate="2017-08-10T22:19:26.323" Score="2" ViewCount="239" Body="&lt;p&gt;Recently, I've been looking recently into what uses AI - specifically evolutionary algorithms - may have in automating engineering design.  For a long time, there have been algorithms that solve constraint satisfaction problems, and, to me, it makes sense to consider engineering problems as a superset of constraint satisfaction problems.  In spite of this, I haven't been able to find any cases of engineering design being automated other than a couple of cases of genetic algorithms being used to optimise structural members.&lt;/p&gt;&#xA;&lt;p&gt;So my question is, why can't I find any examples?&lt;/p&gt;&#xA;&lt;p&gt;The first thing that springs to mind is that I just haven't been looking hard enough - if this is the case, could anyone point me in the right direction?&lt;/p&gt;&#xA;&lt;p&gt;The other obvious answer is that it isn't a widely researched area - if so, why not?  Is it just due to lack of interest or are there technical hurdles (abstraction, complex logic &amp;amp; reasoning, etc.) that make this a much more difficult problem than computer vision, games, and so on?&lt;/p&gt;&#xA;" OwnerUserId="8991" LastEditorUserId="2444" LastEditDate="2021-01-17T21:31:15.687" LastActivityDate="2021-01-17T21:33:44.630" Title="Have evolutionary algorithms been used for engineering design?" Tags="&lt;reference-request&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3799" PostTypeId="2" ParentId="3777" CreationDate="2017-08-10T22:34:00.603" Score="2" Body="&lt;p&gt;One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of &lt;a href=&quot;http://nautil.us/issue/40/learning/is-artificial-intelligence-permanently-inscrutable&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.  It presents a case study of a hospital whose policy was to send asthma sufferers to an intensive care unit; the intensive care meant they were less likely to develop pneumonia and therefore the data showed that people with asthma were less likely to have pneumonia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, since machine learning models learn false relationships if the data are in any way flawed, it is beneficial to be able to &quot;debug&quot; them.  The processes by which decision trees make their decisions, and the reasons for making them, are more readily visible than in other models - particularly neural networks - which makes errors such as the example given in the article more likely to be picked up and corrected.&lt;/p&gt;&#xA;" OwnerUserId="8991" LastActivityDate="2017-08-10T22:34:00.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3800" PostTypeId="2" ParentId="3606" CreationDate="2017-08-11T00:59:47.113" Score="2" Body="&lt;p&gt;That is similar to a chicken-and-egg problem; If the second network could figure out the direction and learn faster, why couldn't the first network do that too and learn even faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would recommend looking up Actor Critic Methods. Here there are two networks but one (the critic) evaluates the other (the actor) to head it in the right direction. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-08-11T00:59:47.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3801" PostTypeId="1" AcceptedAnswerId="3803" CreationDate="2017-08-11T05:31:09.903" Score="21" ViewCount="12345" Body="&lt;p&gt;Frameworks like &lt;a href=&quot;http://pytorch.org&quot; rel=&quot;noreferrer&quot;&gt;PyTorch&lt;/a&gt; and TensorFlow through &lt;a href=&quot;https://research.googleblog.com/2017/02/announcing-tensorflow-fold-deep.html&quot; rel=&quot;noreferrer&quot;&gt;TensorFlow Fold&lt;/a&gt; support Dynamic Computational Graphs and are receiving attention from data scientists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there seems to be a lack of resource to aid in understanding Dynamic Computational Graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of Dynamic Computational Graphs appears to include the ability to adapt to a varying quantities in input data.  It seems like there may be automatic selection of the number of layers, the number of neurons in each layer, the activation function, and other NN parameters, depending on each input set instance during the training.  Is this an accurate characterization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the advantages of dynamic models over static models?  Is that why DCGs are receiving much attention?  In summary, what are DCGs and what are the pros and cons their use?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="4302" LastEditDate="2017-08-21T16:35:59.997" LastActivityDate="2021-04-02T06:01:58.720" Title="What is a Dynamic Computational Graph?" Tags="&lt;neural-networks&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3802" PostTypeId="1" CreationDate="2017-08-11T12:26:00.903" Score="4" ViewCount="763" Body="&lt;p&gt;I have an industrial problem which I'm trying to cast as a Traveling Salesman problem (TSP) in 3D euclidian space. There are physical limitations which implies that some subpaths may or may not be valid based on simple rules. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm is best to deal with the TSP given that there are rules/model/constraints?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be done with Genetic algorithms for example, but the only way i see how to incorporate those rules is by including them somehow within the fitness function. But i feel there should be more suitable approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would reinforcement Q-learning or other algorithms be more appropriate for a rule-based euclidian TSP?&lt;/p&gt;&#xA;" OwnerUserId="8995" LastActivityDate="2018-06-26T08:08:46.647" Title="Traveling salesman problem variant: which algorithm to choose?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;genetic-algorithms&gt;&lt;combinatorics&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3803" PostTypeId="2" ParentId="3801" CreationDate="2017-08-12T00:23:11.337" Score="10" Body="&lt;p&gt;&lt;strong&gt;Two Short Answers&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The short answer from a theoretical perspective is that ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A Dynamic Computational Graph is a mutable system represented as a directed graph of data flow between operations.  It can be visualized as shapes containing text connected by arrows, whereby the vertices (shapes) represent operations on the data flowing along the edges (arrows).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note that such a graph defines dependencies in the data flow but not necessarily the temporal order of the application of operations, which can become ambiguous in the retention of state in vertices or cycles in the graph without an additional mechanism to specify temporal precedence.&lt;/p&gt;&#xA;&lt;p&gt;The short answer from an applications development perspective is that ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A Dynamic Computational Graph framework is a system of libraries, interfaces, and components that provide a flexible, programmatic, run time interface that facilitates the construction and modification of systems by connecting a finite but perhaps extensible set of operations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;The PyTorch Framework&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;PyTorch is the integration of the Torch framework with the Python language and data structuring. Torch competes with Theano, TensorFlow, and other dynamic computational system construction frameworks.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;———   &lt;em&gt;&lt;strong&gt;Additional Approaches to Understanding&lt;/strong&gt;&lt;/em&gt;   ———&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Arbitrary Computational Structures of Arbitrary Discrete Tensors&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;One of the components that can be used to construct a computational system is an element designed to be interconnected to create neural networks.  The availability of these supports the construction of deep learning and back-propagating neural networks.  A wide variety of other systems involving the assembly of components that work with potentially multidimensional data in arbitrarily defined computational structures can also be constructed.&lt;/p&gt;&#xA;&lt;p&gt;The data can be scalar values, such as floating-point numbers, integers, or strings, or orthogonal aggregations of these, such as vectors, matrices, cubes, or hyper-cubes.  The operations on the generalization of these data forms are discrete tensors and the structures created from the assembly of tensor operations into working systems are data flows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Points of Reference for Understanding the Dynamic Computation Concept&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Dynamic Computational Graphs are not a particularly new concept, even though the term is relatively new.  The interest in DCGs among computer scientists is not as new as the term Data Scientist.  Nonetheless, the question correctly states that there are few well-written resources available (other than code examples) from which one can learn the overall concept surrounding their emergence and use.&lt;/p&gt;&#xA;&lt;p&gt;One possible point of reference for beginning to understand DCGs is the Command design pattern which is one of the many design patterns popularized by the proponents of object-oriented design.  The Command design pattern considers operations as computation units the details of which are hidden from the command objects that trigger them.  The Command design pattern is often used in conjunction with the Interpreter design pattern.&lt;/p&gt;&#xA;&lt;p&gt;In the case of DCGs, the Composite and Facade design patterns are also involved to facilitate the definition of plug-and-play discrete tensor operations that can be assembled together in patterns to form systems.&lt;/p&gt;&#xA;&lt;p&gt;This particular combination of design patterns to form systems is actually a software abstraction that largely resembles the radical idea that led to the emergence of the Von Neumann architecture, central to most computers today.  Von Neumann's contribution to the emergence of the computer is the idea of permitting arbitrary algorithms containing Boolean logic, arithmetic, and branching to be represented and stored as data -- a program.&lt;/p&gt;&#xA;&lt;p&gt;Another forerunner of DCGs is expression engines.  Expression engines can be as simple as arithmetic engines and as complex as applications such as Mathematica.  A rules engine is a little like DCGs except that rules engines are declarative and meta-rules for rules engines operate on those declarations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Programs Manipulating Programs&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;What these have in common with DCGs is that the flow of data and operations to be applied can be defined at run time.  As with DCGs, some of these software libraries and applications have APIs or other mechanisms to permit operations to be applied to functional details.  It is essentially the idea of a program permitting the manipulation of another program.&lt;/p&gt;&#xA;&lt;p&gt;Another reference point for understanding this principle at a primitive level is the switch-case statement available in some computer languages.  It is a source code structure whereby the programmer essentially expresses, &amp;quot;We're not sure what must be done, but the value of this variable will tell the real-time execution model what to do from a set of possibilities.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The switch-case statement is an abstraction that extends the idea of deferring the decision as to the direction of computation until run time.  It is the software version of what is done inside the control unit of a contemporary CPU and an extension of the concept of deferring some algorithm details.  A table of functors (function pointers) in C or polymorphism in C++, Java, or Python are other primitive examples.&lt;/p&gt;&#xA;&lt;p&gt;Dynamic Computation takes the abstraction further.  They defer most if not all of the specification of computations and the relationships between them to run time.   This comprehensive generalization broadens the possibilities of functional modification at run time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Directed Graph Representation of Computation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;That's what the Dynamic Computational model is.  Now for the Graph part.&lt;/p&gt;&#xA;&lt;p&gt;Once one decides to defer the choice of operations to be performed until run time, a structure is required to hold the operations, their dependency relationships, and perhaps mapping parameters.  Such a representation is more than a syntactic tree (such as a tree representing the hierarchy of source code).  Unlike an assembly language program or machine code, it must be easily and arbitrarily mutable.  It must contain more information than a data flow graph and much more than a memory map.  What must that data structure that specifies the computational structure look like?&lt;/p&gt;&#xA;&lt;p&gt;Fortunately, any arbitrary, finite, bounded algorithm can be represented as a directed graph of dependencies between specified operations.  In such a graph, the vertices (often represented as nodes of various shapes when displayed) represent operations performed on the data and the edges (often represented as arrows when displayed) are digital representations of information originating resulting from some operation (or system input) and upon which other operations (or system output) depend.&lt;/p&gt;&#xA;&lt;p&gt;Keep in mind that the directed graph is neither an algorithm (in that a precise sequence of operations is specified) nor a declaration (in that data can be explicitly stored and loops, branches, functions, and modules may be definable and nested).&lt;/p&gt;&#xA;&lt;p&gt;Most of these Dynamic Computational Graph frameworks and libraries permit the components to do computations on the component input that support machine learning.  Vertices in the directed graph can be simulations of neurons for the construction of a neural net or components that support differential calculus.  These frameworks present possibilities of constructs that can be used for deep learning in a more generalized sense.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;In the Context of Computer History&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Again, nothing mentioned thus far is new to computer science.  LISP permits computational schematics to be modified by other algorithms.  And generalized input dimensionality and numerocity is built into a number of longstanding plug-and-play interfaces and protocols.  The idea of a framework for learning dates back to the same mid-Twentieth Century period too.&lt;/p&gt;&#xA;&lt;p&gt;What is new and gaining in popularity is a particular combination of integrated features and the associated set of terminology, an aggregation of existing terminology for each of the features, leading to a wider base for comprehension by those already studying for and working in the software industry.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Contemporary (trendy) flavor of API interfaces&lt;/li&gt;&#xA;&lt;li&gt;Object orientation&lt;/li&gt;&#xA;&lt;li&gt;Discrete tensor support&lt;/li&gt;&#xA;&lt;li&gt;The directed graph abstraction&lt;/li&gt;&#xA;&lt;li&gt;Interoperability with popular languages and packages that support big data, data mining, machine learning, and statistical analysis&lt;/li&gt;&#xA;&lt;li&gt;Support for arbitrary and systematic neural network construction&lt;/li&gt;&#xA;&lt;li&gt;The possibility of dynamic neural network structural adaptation (which facilitates experimentation on neural plasticity)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Many of these frameworks support adaptability to changing input dimensionality (number of dimensions and the range of each).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Similarity to Abstract Symbol Trees in Compilers&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A dependency graph of inputs and outputs of operations also appears within abstract symbol trees (AST), which some of the more progressive compilers construct during the interpretation of the source code structure.  The AST is then used to generate assembler instructions or machine instructions in the process of linking with libraries and forming an executable.  The AST is a directed graph that represents the structure of data, operations performed, and the control flow specified by the source code.&lt;/p&gt;&#xA;&lt;p&gt;The data flow is simply the set of dependencies between operations, which must be inherent in the AST for the AST to be used to create execution instructions in assembler or machine code that precisely follows the algorithm specified in the source code.&lt;/p&gt;&#xA;&lt;p&gt;Dynamic Computational Graph frameworks, unlike switch-case statements or AST models in compilers, can be manipulated in real-time, optimized, tuned (as in the case of plastic artificial nets), inverted, transformed by tensors, decimated, modified to add or remove entropy, mutated according to a set of rules, or otherwise translated into derivative forms.  They can be stored as files or streams and then retrieved from them.&lt;/p&gt;&#xA;&lt;p&gt;This is a trivial concept for LISP programmers or those that understand the nature of John von Neumann's recommendation to store operational specifications as data.  In this later sense, a program is a data stream to instruct, through a compiler and operating system, a dynamic computational system implemented in VLSI digital circuitry.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Achieving Adaptable Dimensionality and Numerocity&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the question is the comment that one doesn't, &amp;quot;Need to have data set -- that all the instances within it have the same, a fixed number of inputs.&amp;quot; That statement does not promote accurate comprehension.  There are clearer ways to say what is true about input adaptability.&lt;/p&gt;&#xA;&lt;p&gt;The interface between a DCG and other components of an overall system must be defined, but these interfaces may have dynamic dimensionality or numerocity built into them.  It is a matter of abstraction.&lt;/p&gt;&#xA;&lt;p&gt;For instance, a discrete tensor object type presents a specific software interface, yet a tensor is a dynamic mathematical concept around which a common interface can be used.  A discrete tensor may be a scalar, a vector, a matrix, a cube, or a hypercube, and the range of dependent variables for each dimension may be variable.&lt;/p&gt;&#xA;&lt;p&gt;It can be the case that the number of nodes in a layer of the system defined in a Dynamic Computational Graph can be a function of the number of inputs of a particular type, and that too can be a computation deferred to run time.&lt;/p&gt;&#xA;&lt;p&gt;The framework may be programmed to select layer structure (an extension of the switch-case paradigm again) or calculate parameters defining the structure sizes and depth or activation.  However, these sophisticated features are not what qualifies the framework as a Dynamic Computational Graph framework.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What Qualifies a Framework to Support Dynamic Computational Graphs?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To qualify as a Dynamic Computational Graph framework, the framework must merely support the deferring of the determination of algorithm to run time, therefore opening the door to a plethora of operations on the computational dependencies and data flow at run time.  The basics of the operations deferred must include the specification, manipulation, execution, and storage of the directed graphs that represent systems of operations.&lt;/p&gt;&#xA;&lt;p&gt;If the specification of the algorithm is NOT deferred until run time but is compiled into the executable designed for a specific operating system with only the traditional flexibility provided by low-level languages such as if-then-else, switch-case, polymorphism, arrays of functors, and variable-length strings, it is considered a static algorithm.&lt;/p&gt;&#xA;&lt;p&gt;If the operations, the dependencies between them, the data flow, the dimensionality of the data within the flow, and the adaptability of the system to the input numerocity and dimensionality are all variable at run time in a way to create a highly adaptive system, then the algorithm is dynamic in these ways.&lt;/p&gt;&#xA;&lt;p&gt;Again, LISP programs that operate on LISP programs, rules engines with meta-rule capabilities, expression engines, discrete tensor object libraries, and even relatively simple Command design patterns are all dynamic in some sense, deferring some characteristics to run time.  DCGs are flexible and comprehensive in their capabilities to support arbitrary computational constructs in such a way to create a rich environment for deep learning experimentation and systems implementation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;When to Use Dynamic Computational Graphs&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The pros and cons of DCGs are entirely problem-specific.  If you investigate the various dynamic programming concepts above and others that may be closely tied to them in the associated literature, it will become obvious whether you need a Dynamic Computational Graph or not.&lt;/p&gt;&#xA;&lt;p&gt;In general, if you need to represent an arbitrary and changing model of computation to facilitate the implementation of the deep learning system, mathematical manipulation system, adaptive system, or another flexible and complex software construct that maps to the DCG paradigm well, then a proof of concept using a Dynamic Computational Graph framework is a good first step in defining your software architecture for the problem's solution.&lt;/p&gt;&#xA;&lt;p&gt;Not all learning software uses DCG's, but they are often a good choice when the systematic and possibly continuous manipulation of an arbitrary computational structure is a run time requirement.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="36737" LastEditDate="2021-04-02T06:01:58.720" LastActivityDate="2021-04-02T06:01:58.720" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3804" PostTypeId="1" CreationDate="2017-08-12T04:38:18.120" Score="4" ViewCount="890" Body="&lt;p&gt;Suppose that you have 80 neurons in a layer, where one neuron is bias. Then  you add a dropout layer after the activation function of this layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, does it have a chance to drop out the bias neuron, or does the dropout only affect the other 79 weight neurons?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastActivityDate="2018-03-02T07:03:41.240" Title="Does a bias also have a chance to be dropped out in Dropout layer?" Tags="&lt;deep-learning&gt;&lt;dropout&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3805" PostTypeId="1" CreationDate="2017-08-12T05:39:38.040" Score="3" ViewCount="178" Body="&lt;p&gt;3 SVD Based Methods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this class of methods to find word embeddings (otherwise known&#xA;as word vectors), we first loop over a massive data set and accumulate word co-occurrence counts in some form of a matrix X and then&#xA;perform Singular Value Decomposition on X to get a USV^T decomposition. We then use the rows of U as the word embeddings for all&#xA;words in our dictionary. Let us discuss a few choices of X.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above is the excerpt from the standford univ cs224n lecture 1 notes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above USV refer to what? There's no prior explanation about it so I ask here.&lt;/p&gt;&#xA;" OwnerUserId="8325" LastEditorUserId="4302" LastEditDate="2018-10-08T12:24:18.820" LastActivityDate="2018-10-08T12:24:18.820" Title="What is USV In NLP?" Tags="&lt;natural-language-processing&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3806" PostTypeId="2" ParentId="2439" CreationDate="2017-08-12T17:19:31.430" Score="2" Body="&lt;p&gt;You could build a MLC(machine learning classifier)  based on her/his answers and apply that to a large database of questions to see which type of questions will probably yield an incorrect answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then provide those questions (Assuming one would learn more from working at her/his weaknesses) to provide a steeper learning curve, continuously adapting the MLC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.deeplearningbook.org/&lt;/a&gt; Chapter 5, also findable on github, has a very clear and relatively simple explanation on how to build a basic MLC, and what the problems/critical issues/parameters in application could be/are.&lt;/p&gt;&#xA;" OwnerUserId="4903" LastEditorUserId="4903" LastEditDate="2017-08-12T19:39:30.537" LastActivityDate="2017-08-12T19:39:30.537" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3807" PostTypeId="1" CreationDate="2017-08-13T16:48:35.890" Score="4" ViewCount="237" Body="&lt;p&gt;Are there any real-world examples of unintentional &quot;bad&quot; AI behaviour? I'm not looking for hypothetical arguments of malicious AI (AI in a box, paperclip maximizer), but for actual instances in history where some AI directly did something bad due to its direct action that was not intended behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interpretations of the meaning of &quot;AI&quot;, &quot;bad&quot;, and &quot;unintentional&quot; are left open due to obvious reasons. Be free with your interpretation, but use some common sense please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex: Microsoft Tay became a racist not too long after being hooked up to the internet, thanks to internet trolls &quot;teaching&quot; her bad things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't think of any other instances. So the following examples are just hypothetical scenarios to demonstrate what I mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex: A self-driving car drove off-track after being presented with an adversarial example, crashing into people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex: Surgery bot goes haywire and accidentally kills someone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex: Weaponized drone targets civilians against its design.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2019-04-06T00:59:23.660" LastActivityDate="2019-12-08T12:01:37.093" Title="Concrete examples of unintentional adversarial AI behaviour" Tags="&lt;ai-safety&gt;" AnswerCount="2" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3808" PostTypeId="2" ParentId="3786" CreationDate="2017-08-13T19:24:38.220" Score="2" Body="&lt;p&gt;Rule-based systems cover a wide range of systems. Some make use of boolean if/then/else rules, others may use weighting or even probabilistic inference. Some operate on frames, some on java objects, some on propositions that can be formulated in predicate logic. An example of a popular rule system is &lt;a href=&quot;https://github.com/kiegroup/drools&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Some rule systems can be expressed as a subset of predicate logic. For example, &lt;a href=&quot;https://www.w3.org/Submission/SWRL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SWRL&lt;/a&gt; is a W3C standard rule language that extends OWL Description Logic (DL) with horn rules. Both OWL-DL and &lt;a href=&quot;https://www.w3.org/Submission/SWRL-FOL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SWRL&lt;/a&gt; are expressible in first-order predicate logic. However, not all rule languages are directly expressible in this way, as rule languages encompass such a range of semantics. Even in cases like &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt;, there are subtleties. Pure Prolog is a subset of FOL, but actual existing Prolog implementations are not FOL subsets (e.g. order of precedence matters).&lt;/p&gt;&#xA;&lt;p&gt;The W3C Rules Interchange Format (RIF) working group has done a lot of work attempting to unify these different perspectives. I'm not sure the best place to start to see a summary, but you can see some useful material on the &lt;a href=&quot;https://www.w3.org/2005/rules/wg/charter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;group charter page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9014" LastEditorUserId="2444" LastEditDate="2021-01-24T15:18:25.733" LastActivityDate="2021-01-24T15:18:25.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3814" PostTypeId="1" CreationDate="2017-08-14T20:59:00.470" Score="8" ViewCount="2028" Body="&lt;p&gt;So, &lt;a href=&quot;https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/&quot; rel=&quot;noreferrer&quot;&gt;Deepmind is pushing for a human level Starcraft bot&lt;/a&gt; and &lt;a href=&quot;https://openai.com/the-international/&quot; rel=&quot;noreferrer&quot;&gt;Open AI just created a human level 1vs1 Dota bot&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I've no clue what that signifies because I've never played Starcraft nor Dota nor do I have more than a fleeting acquaintance with similar games. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what the difference between Starcraft and Dota is from a AI perspective and what scientific significance the respective super human bots would have. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2227" LastEditDate="2017-08-15T12:34:33.557" LastActivityDate="2017-08-18T11:55:59.223" Title="What's the difference between Starcraft and Dota from an AI perspective?" Tags="&lt;game-ai&gt;&lt;deepmind&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3815" PostTypeId="2" ParentId="3754" CreationDate="2017-08-15T05:32:36.373" Score="1" Body="&lt;p&gt;Yes, you can fit any time series (with or without external variables) using HMM, but there are some constraints:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It should follow the Markov property.  &lt;/li&gt;&#xA;&lt;li&gt;There is some variance that other models are not able to capture (in other words, the system is partially observable).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Adding to point 1, for HMM, it should hold true, but the way &lt;a href=&quot;https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Baum Welch algorithm&lt;/a&gt; works, indirectly it considers the values of more than the previous state for HMM (order-1). The state &lt;span class=&quot;math-container&quot;&gt;$t-1$&lt;/span&gt;  depends on &lt;span class=&quot;math-container&quot;&gt;$t-2$&lt;/span&gt;, which in turn depends on &lt;span class=&quot;math-container&quot;&gt;$t-3$&lt;/span&gt;. The calculation of parameters (transition, emission, starting probabilities) happens over multiple iterations and it finds parameters in such a way that holds Markov property true.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that, when they say 'any', they mean even when you don't have all variables needed to forecast future values. &lt;/p&gt;&#xA;" OwnerUserId="5718" LastEditorUserId="2444" LastEditDate="2020-04-05T15:34:07.767" LastActivityDate="2020-04-05T15:34:07.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3816" PostTypeId="2" ParentId="3805" CreationDate="2017-08-15T06:36:10.197" Score="1" Body="&lt;p&gt;USV^T refers to the result of the singular value decomposition (SVD). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An m times n matrix X can be written with the help of three matrices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;X = USV^T,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where U is an m times m unitary matrix, S is a diagonal m times n matrix with real entries called singular values, and V is a unitary n times n matrix. The ^T is the Hermitean transpose. SVD has applications, e.g, in optimization problems, principal component analysis, etc. &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; has quite a long article.&lt;/p&gt;&#xA;" OwnerUserId="9039" LastActivityDate="2017-08-15T06:36:10.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3817" PostTypeId="1" CreationDate="2017-08-15T15:07:52.343" Score="7" ViewCount="516" Body="&lt;p&gt;I am currently searching for a supervised learning algorithm that can be used to predict the output given a large enough training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a simple example. Suppose the training dataset is &lt;code&gt;{[A=1, B=330, C=1358.238902], result=234.244378}&lt;/code&gt; and the test dataset &lt;code&gt;{[A=893, B=34, C=293], result=?}&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intention is to predict &lt;code&gt;?&lt;/code&gt; using the input values and result given in the training dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm would be effective for this problem given the wide range of my input/output values? Would this require some sort of regression algorithm?&lt;/p&gt;&#xA;" OwnerUserId="9046" LastEditorUserId="2444" LastEditDate="2019-09-20T22:21:48.213" LastActivityDate="2020-11-14T12:44:35.547" Title="Which predictive algorithm can be used to predict a number given other numbers?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;prediction&gt;&lt;regression&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3818" PostTypeId="2" ParentId="3798" CreationDate="2017-08-15T16:48:36.920" Score="1" Body="&lt;p&gt;Examples of &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;GA&lt;/a&gt; implementations:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;NASA’s spacecraft &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolved_antenna&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolved antenna&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Solution for &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;TSP&lt;/a&gt; with a big number of connections. &lt;a href=&quot;https://github.com/maoaiz/tsp-genetic-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is an example with Python and &lt;a href=&quot;https://github.com/7lb/TSP&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; with C#.&lt;/li&gt;&#xA;&lt;li&gt;From my experience, GA can be even used for a training of small artificial neural networks, but with less efficiency than with traditional deep learning approaches&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Also, I heard about successful projects with using GA for scheduling, packing and  logistics&lt;/p&gt;&#xA;&lt;p&gt;Therefore, everything, that can be represented in a form of digital chromosomes (can be serialized to and deserialized  back from a sequence of bits) and we can define a function to evaluate its efficiency – &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;fitness&lt;/a&gt; or &lt;a href=&quot;https://stats.stackexchange.com/questions/189067/how-to-make-a-reward-function-in-reinforcement-learning&quot;&gt;reward function&lt;/a&gt; can be optimized by using of GA&lt;/p&gt;&#xA;" OwnerUserId="8861" LastEditorUserId="2444" LastEditDate="2021-01-17T21:33:44.630" LastActivityDate="2021-01-17T21:33:44.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3820" PostTypeId="1" CreationDate="2017-08-16T10:37:37.893" Score="5" ViewCount="3033" Body="&lt;p&gt;Is this related to &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;supervised&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsupervised&lt;/a&gt; machine learning?  Is it related to AI assisted human learning, and what is the distinction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, why is assisted machine learning seen as an opportunity and unassisted machine learning seen as a threat?&lt;/p&gt;&#xA;" OwnerUserId="9062" LastEditorUserId="1671" LastEditDate="2017-08-29T22:57:54.603" LastActivityDate="2017-08-29T22:57:54.603" Title="What is the difference between assisted and unassisted learning in relation to AI?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;unsupervised-learning&gt;&lt;comparison&gt;&lt;intelligence-augmentation&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3821" PostTypeId="2" ParentId="3820" CreationDate="2017-08-16T13:20:38.440" Score="1" Body="&lt;p&gt;Assisted intelligence can be thought of as a carpenter working with an intelligent hammer to build a better house. This would be a great opportunity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people think that Artificial Intelligence might be like an intelligent hammer building the house on its own putting the carpenter out of a job. This would be a big threat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Expanded&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assisted Intelligence, Augmented Intelligence, or Intelligence Augmentation all refer to the same concept a technology which enhances an autonomous system that had already proven to function (humans). In the carpenter example, even a tape measure is an example of such a technology as it enhances the carpenters ability to measure distances more accurately and faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been a long history of success on the front of Intelligence Augmentation (IA) and a few terms have come up defining the different techniques such as &quot;Extended Mind&quot;,  &quot;Distributed Cognition&quot; which I leave up to the reader to research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The opportunity and thus optimism behind IA comes from this long history of success and many proponents of IA hopes it will influence the trend of research to augment humans to perform better than replace them entirely. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence on the other hand attempts to build a human-like intelligence in the form of an autonomous technological system such as a computer or robot. Many fundamental problems, practical and theoretical, have been encountered from job replacement (like the carpenter), existential crisis (If no human carpenter can build a better house than a robot, why bother?), misaligned goals (how does a robot know what the best house looks like?), and many many more which I leave to the reader to research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These problems create the threat seen in AI and has been the source many academic debates as well as many great scifi works. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2017-08-16T23:59:54.237" LastActivityDate="2017-08-16T23:59:54.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3823" PostTypeId="1" AcceptedAnswerId="4384" CreationDate="2017-08-16T19:29:00.103" Score="4" ViewCount="1035" Body="&lt;p&gt;I can recall that a professor once said that decision trees are not good for incremental learning, as they have to be rebuilt from the ground up if new training examples arrive.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this basically true? Quick googling just brought me to a lot of papers trying to fit decision trees into incremental learning&lt;/li&gt;&#xA;&lt;li&gt;What other algorithms fall under this category?&lt;/li&gt;&#xA;&lt;li&gt;Are Neural Nets good for incremental learning? What other algorithms would be good?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9071" LastEditorUserId="10468" LastEditDate="2017-10-31T16:46:23.937" LastActivityDate="2017-10-31T16:46:23.937" Title="Is a decision tree less suitable for incremental learning than e.g. a neural net?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3824" PostTypeId="2" ParentId="3820" CreationDate="2017-08-16T19:39:12.047" Score="2" Body="&lt;p&gt;Although there doesn't seem to be standardization of the terminology involving assisted learning, I've been noticing this concept, and the distinction, popping up in articles recently, surely driven by important milestones in unassisted machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;My understanding is that Algorithmic Intelligence has traditionally been based on complex decision-making algorithms that are trained to some degree by the programmers.  (In essence, given a bit of a head start by utilizing human knowledge in regard to the problem.)  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;By contrast, unassisted machine learning is an Algorithmic Intelligence that learns only by its own analysis of models and problems.  No human knowledge goes into the automata, which develops intelligence entirely on its own.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A passage from &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matthew Lai's Giraffe Chess paper&lt;/a&gt; may provide some insight:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;This report presents Giraffe, a chess engine that uses self-play to discover all its&#xA;  domain-specific knowledge, with minimal hand-crafted knowledge given by the programmer.&#xA;  Unlike previous attempts using machine learning only to perform parametertuning&#xA;  on hand-crafted evaluation functions, Giraffe’s learning system also performs&#xA;  automatic feature extraction and pattern recognition. The trained evaluation function&#xA;  performs comparably to the evaluation functions of state-of-the-art chess engines - all of&#xA;  which containing thousands of lines of carefully hand-crafted pattern recognizers, tuned&#xA;  over many years by both computer chess experts and human chess masters.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;DeepMind, for instance, is setting automata on video games and letting the automata learn with no human guidance whatsoever, and getting very good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to why unassisted machine learning is worrisome relates to the issue of &lt;a href=&quot;https://intelligence.org/stanford-talk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;human/AI value alignment&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_control_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;control problem&lt;/strong&gt;&lt;/a&gt;: if we don't know what an an automata is thinking, or even how smart it has become, how can &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Anatomically_modern_human&quot; rel=&quot;nofollow noreferrer&quot;&gt;homo sapiens sapiens&lt;/a&gt;&lt;/em&gt; be assured of positive outcomes for our species?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-08-17T21:57:49.763" LastActivityDate="2017-08-17T21:57:49.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3825" PostTypeId="1" AcceptedAnswerId="3853" CreationDate="2017-08-16T23:20:02.533" Score="6" ViewCount="112" Body="&lt;p&gt;I've gone through several descriptions of CNNs online and they all leave out a crucial part as if it were trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;volume&quot; of neurons consists of several parallel layers (&quot;feature maps&quot;), each the result of convolving with a different kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Between volumes there is usually a step where layers are pooled and subsampled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next volume has a different number of parallel layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do the feature maps from one volume connect to the feature maps of the next volume? Is it one-to-many? Many-to-many? Do N kernels apply to each of M feature maps in the first volume, yielding N*M feature maps in the second volume? Are these N kernels the same for each feature map in the first volume, or do different kernels apply to each one? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is the number of maps in the second volume not necessarily a multiple of the number in the first volume? If so, do maps in the first volume get cross-synthesized somehow? Or maybe different numbers of maps in the second volume follow from each one in the first?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is it some other of umpteen trillion possibilities?&lt;/p&gt;&#xA;" OwnerUserId="9072" LastActivityDate="2017-08-25T18:24:59.220" Title="CNNs: What happens from one neuron volume to the next?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3826" PostTypeId="2" ParentId="3817" CreationDate="2017-08-17T02:03:20.067" Score="-1" Body="&lt;p&gt;Without seeing more data it's hard to say for sure.  Superficially, this looks like a regression type problem.  As you mention, there is a lot of spread on the input values, but that doesn't &lt;em&gt;necessarily&lt;/em&gt; mean that something like linear regression wouldn't work. Try it and see what kind of coefficient of correlation you get.  If it's really low, you probably need a different approach, OR the data might actually not have any (or much) predictive power in this scenario.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond linear regression, you might find that there's a more complicated mathematical relationship between the inputs and outputs, that could be determined using &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic regression&lt;/a&gt;.  Another possibility, if there is a complex non-linear relationship in play, is that an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificial neural network&lt;/a&gt; approach might work well.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-08-17T02:03:20.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3828" PostTypeId="1" CreationDate="2017-08-17T07:15:24.897" Score="3" ViewCount="84" Body="&lt;p&gt;I have a question regarding &lt;strong&gt;Answer Set Programming&lt;/strong&gt; on how to make an existing &lt;strong&gt;&lt;em&gt;fact&lt;/em&gt;&lt;/strong&gt; invalid, when there is already (also) a &lt;strong&gt;&lt;em&gt;default statement&lt;/em&gt;&lt;/strong&gt; present in the Knowledge Base.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, there are two persons &lt;code&gt;seby&lt;/code&gt; and &lt;code&gt;andy&lt;/code&gt;, one of them is able to drive at once. The scenario can be that &lt;code&gt;seby&lt;/code&gt; can drive as seen in Line 3 but  let's say, after his license is cancelled he cannot drive anymore, hence we now have Lines 4 to 7 and meanwhile &lt;code&gt;andy&lt;/code&gt; learnt driving, as seen in Line 7. Line 6 shows only one person can drive at a time, besides showing &lt;code&gt;seby&lt;/code&gt; and &lt;code&gt;andy&lt;/code&gt; are not the same. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 person(seby).&#xA;2 person(andy).&#xA;3 drives(seby).&#xA;4 drives(seby) :- person(seby), not ab(d(drives(seby))), not -drives(seby).&#xA;5 ab(d(drives(seby))).&#xA;6 -drives(P) :- drives(P0), person(P), P0 != P.&#xA;7 drives(andy).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the above program, Lines 3 and 4 contradict with each other, and the Clingo solver (which I use) obviously outputs &lt;code&gt;UNSATISFIABLE&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said all these, deleting Line 3 and getting the problem solved is not what I'm epecting. The intention behind asking this question is to know whether it is possible now to make Line 3 somehow invalid to let Line 4 do its duty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, Line 4 can also be written as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;4 drives(P) :- person(P), not ab(d(drives(P))), not -drives(P).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot in advance.&lt;/p&gt;&#xA;" OwnerUserId="8251" LastEditorUserId="1581" LastEditDate="2017-08-17T19:26:17.213" LastActivityDate="2017-08-17T19:26:17.213" Title="Answer Set Programming - Make a Fact INVALID" Tags="&lt;knowledge-representation&gt;&lt;logic&gt;&lt;declarative-programming&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3829" PostTypeId="2" ParentId="3807" CreationDate="2017-08-17T19:34:44.720" Score="4" Body="&lt;p&gt;The infamous &lt;a href=&quot;https://en.wikipedia.org/wiki/2010_Flash_Crash&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flash Crash of 2010&lt;/a&gt; may qualify. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It didn't involve Artificial General Intelligence (which is still a hypothetical) or even &quot;strong narrow AI&quot; (such as AlphaGo) but does involve algorithmic decision-making, which is a form of basic Artificial Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_trading&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithmic trading&lt;/a&gt; already represents a significant percentage of all market activity, and I suspect that percentage will only increase. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Business Insider: &lt;a href=&quot;http://www.businessinsider.com/algos-could-trigger-stock-market-crash-2017-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algos could trigger the next stock market crash&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-08-17T19:34:44.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3830" PostTypeId="2" ParentId="3807" CreationDate="2017-08-17T21:26:40.780" Score="5" Body="&lt;p&gt;There is the case of the tesla accident where the car was in autopilot and crashed into a truck because it appears the vehicle mistook a lightly coloured truck for the sky, killing the driver: &lt;a href=&quot;https://www.newscientist.com/article/2095740-tesla-driver-dies-in-first-fatal-autonomous-car-crash-in-us/&quot; rel=&quot;noreferrer&quot;&gt;https://www.newscientist.com/article/2095740-tesla-driver-dies-in-first-fatal-autonomous-car-crash-in-us/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, it appears the car had been trying to tell the driver to pay attention: &lt;a href=&quot;http://uk.reuters.com/article/us-tesla-crash-idUKKBN19A2XC&quot; rel=&quot;noreferrer&quot;&gt;http://uk.reuters.com/article/us-tesla-crash-idUKKBN19A2XC&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="9091" LastActivityDate="2017-08-17T21:26:40.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3833" PostTypeId="2" ParentId="3814" CreationDate="2017-08-18T08:15:42.167" Score="4" Body="&lt;p&gt;Those AI-learning programs may have very similar scheme. &lt;strong&gt;We are changing only inputs and possible actions&lt;/strong&gt; (like &quot;use skill&quot; or &quot;move here&quot;). Starcraft AI must do a lot of actions and control many units. Dota is MOBA so bot should be good in positioning on map for example. Different opponents to destroy and target for win.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI needs to play many games for learn to play game with some rules to find best moves in some situations/states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course it's only my newbie-programmer opinion :)&lt;/p&gt;&#xA;" OwnerUserId="9101" LastActivityDate="2017-08-18T08:15:42.167" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3834" PostTypeId="2" ParentId="3581" CreationDate="2017-08-18T09:58:01.607" Score="3" Body="&lt;p&gt;Actually I'm trying the same thing with the Azure Computer Vision API. Although the API is very good in identifying objects, it has problems identifying specific consumer products (in my experience though). For example it can't really distinguish between two pair of shoes, or two pair of watches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;People recommended me using the: &lt;a href=&quot;https://azure.microsoft.com/en-gb/services/cognitive-services/custom-vision-service/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Custom Vision Service&lt;/strong&gt;&lt;/a&gt; combined with &lt;a href=&quot;https://azure.microsoft.com/en-gb/services/cognitive-services/custom-decision-service/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Custom Decision Service&lt;/strong&gt;&lt;/a&gt;. I haven't really looked in to them both. But maybe it's useful for you.&lt;/p&gt;&#xA;" OwnerUserId="9057" LastActivityDate="2017-08-18T09:58:01.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3835" PostTypeId="2" ParentId="3814" CreationDate="2017-08-18T11:43:04.573" Score="4" Body="&lt;p&gt;In SC2, players have more control over every minuet mechanic (constructing buildings, resource mining and management, controlling minions...) in the game, thus putting more tactical responsibility on the burden of the player. In DOTA2, the player is only in control of the super-powered hero itself, and not much of the other aspects of the gameplay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is debatable if these options make the game &quot;better&quot; or more difficult as a result. But it is for certain that overall the search space of the problem increases much faster as the dimensions of freedom increases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, DOTA2 contains very many game mechanics as well (a ton of items that changes a lot of different stats, very many types of heroes which each have their own attacks, a variety of buildings scattered around the map, a shop for player items), but it seems that most of this complexity is focused around player engagements, which although crucial for the development of the game, lets us analyze a much shorter timespan of PvP as opposed to a whole game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And indeed, the DOTA2 bot from OpenAI was restricted to a mid-lane fight as a singular hero with restricted items, hence restricting most of the complexity DOTA2 has to offer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I am overstepping with this analogy, but the OpenAI result is a bit like a solved chess endgame configuration, while SC2 would be like chess under the knowledge that neither players can see each other's pieces until they are in a position where you can capture them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short: SC2 is more tactical. DOTA2 is more arcade-like. &lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2017-08-18T11:55:59.223" LastActivityDate="2017-08-18T11:55:59.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3836" PostTypeId="1" CreationDate="2017-08-18T12:16:22.870" Score="4" ViewCount="459" Body="&lt;p&gt;I am just curious what AI would be harder to create from a strictly engineering point of view. AI which would win 1vs 1 game with the best player in starcraft or AI which would control a team the whole team in dota2 and win against the best team?  &lt;/p&gt;&#xA;" OwnerUserId="9105" LastActivityDate="2017-08-22T16:13:54.050" Title="AI in dota2 vs AI in starcraft" Tags="&lt;game-ai&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3837" PostTypeId="1" CreationDate="2017-08-18T14:40:48.243" Score="2" ViewCount="768" Body="&lt;p&gt;I'm a little bit stuck:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented an AI with GOAP (Goal oriented Action Planning, &lt;a href=&quot;http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf&lt;/a&gt;) for a simulation game. That works fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want that the agents can cooperate (e.g. doing actions together). &lt;em&gt;What is in this case the best AI-Design that the GoapActions keep loose couplet?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should they plan together? (what is in this case the &quot;worldstate&quot;?)Or Should they share their plans? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;br&gt;&#xA;Agent1:&#xA;Worldstate Agent 1: isLonely= true&lt;br&gt;&#xA;Goal Agent1: isLonely = false&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plan Agent1: AskAgent2ToTalk -&gt; TalkToAgent2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Agent2&#xA;Worldstate Agent 2: hasWood = false&lt;br&gt;&#xA;Goal hasWood = true&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plan Agent2: GetAxe -&gt; ChopWood -&gt; BringWoodToSupply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How I get this constellation?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Agent1 Plan: TalkToAgent2&lt;br&gt;&#xA;Agent2 Plan: TalkToAgent1 -&gt; GetAxe -&gt; ChopWood -&gt; BringWoodToSupply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or if they are talking and one of the agents is interrupted (e.g. by an attacking enemy) the other agent must know that his TalktoAgent2 Action has ended.&lt;/p&gt;&#xA;" OwnerUserId="9111" LastEditorUserId="1581" LastEditDate="2017-08-19T12:03:23.200" LastActivityDate="2018-01-23T16:25:57.113" Title="Goal oriented Action Planning with multiple Agents" Tags="&lt;ai-design&gt;&lt;intelligence-testing&gt;&lt;game-ai&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3838" PostTypeId="2" ParentId="3836" CreationDate="2017-08-18T15:48:40.223" Score="3" Body="&lt;p&gt;I can't answer definitively without a detailed breakdown of game mechanics of dota2 vs. Starcraft, but assuming the games have similar complexity, the AI controlling multiple in-game agents that form a team would be more complex, and therefore more difficult to create, than a single agent &quot;team&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My supposition is based on the fundamentals of &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorics&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorics&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational complexity&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in dota2, the AI has to not only to identify winning strategies, but &lt;em&gt;coordinate&lt;/em&gt; multiple agents.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;by contrast:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the Starcraft AI only has to identify a winning strategy, with no coordination between discrete agents which form a team.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Coalitions have traditionally been difficult to analyze in Game Theory and Combinatorial Game Theory (in the latter case especially, restricting mathematical study, for the most part, to games involving only 2 players.)  In this case the issue is not coalitions--the multiple game agents are already on the same team--but complexity is nevertheless dramatically increased by the existence of multiple agents on a team as compared to a team consisting of a single agent.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-08-18T15:48:40.223" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3843" PostTypeId="2" ParentId="3837" CreationDate="2017-08-20T22:59:50.403" Score="1" Body="&lt;p&gt;This evening I got inspired by this paper: &lt;a href=&quot;http://www.dphrygian.com/bin/David-Pittman-GOAP-Masters-Thesis.doc&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dphrygian.com/bin/David-Pittman-GOAP-Masters-Thesis.doc&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(GOAP paired with the Command and Control Pattern)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What do you think about this solution?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each goal has a relevance (that depends on the agent needs) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When agent1 working on the &quot;AskAgent2ToTalk&quot; Action, it only sends a goal recommendation to agent2. Explicit means this: The agent only sends a list of modifiers for each goal relevance (in this case it could be a bonus for the &quot;SocialInteraction&quot; Goal. The value depends on the relationship between the agents) and the recommend goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if agent2's desired goal is the recommend goal, the agent replans.&lt;/p&gt;&#xA;" OwnerUserId="9111" LastEditorUserId="9111" LastEditDate="2017-08-21T04:47:46.743" LastActivityDate="2017-08-21T04:47:46.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3846" PostTypeId="2" ParentId="3836" CreationDate="2017-08-21T09:36:26.597" Score="2" Body="&lt;p&gt;There are different abilities required for strong play in those two games. Some of them are easier to implement using AI than others. Therefore it is difficult to answer this question in a generic fashion, but we can look at different aspects in detail:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Speed / APM (Actions per Minute)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While both games require a certain speed, SC2 is usually a bit faster because you simply have more stuff to manage (keep building, arranging your units, coordinate attacks on multiple locations). That said, if you want to control the whole team in Dota2 from a single agent, the APM requirements multiply accordingly. In the end it doesn't really matter, because speed is the least of our concerns and can simply be improved by throwing more resources at the problem. So I wouldn't say that one of the two games has an advantage here concerning engineering effort, just wanted to get the topic out of the way.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Implement strong battle technique&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Fighting battles is an important skill in both games. While the armies in SC2 are larger, the amount of abilities each unit has is usually smaller. One of the limitations of a human player in SC2 is the inability to control each unit individually, because there are simply too many units most of the time. An AI on the other hand could utilize a complex army significantly better by controlling each unit. I have seen simulations of such armies in the early days of SC2 and it was obvious that no human player can compete with this technique by far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This aspect is less important in Dota2, because the amount of units is significantly smaller. Nevertheless the precision in executing attacks and retreating could easily be optimized to a level hardly achievable by any human player. I would still rate the potential for super human skill in SC2 much higher because it gets multiplied for each unit.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Find a Winning Strategy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While it is rather easy to beat a human player with speed and technique, finding a winning strategy is a totally different ball game. In games where deception and hidden information plays such a crucial part, the engineering requirements for a strong winning strategy are huge. A strong AI will require a hybrid approach of playing according to programmable rules of different strategies and fine tuning using ML. Both games are still evolving and new (or changing) heroes, units and structures need to be considered in the long run. In my opinion, the hardest part for the AI will be dealing with unconventional or trick plays it hasn't encountered before. SC2 has a higher potential for such trick plays that can catch the opponent off guard and wins you the game right away. And even when the AI has encountered a certain trick play several times, it is very hard to find appropriate counters on its own. So each new trick play will require specific guided training to play correct against it. On the other hand the AI could learn the same tricks and utilize them as part of their strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dota2 has a smaller range of unpredictable events and strategies, therefore adapting to those would be easier for an AI. This would reduce the engineering effort significantly in this regard.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Limitations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To evaluate the complexity of the problem, you need to consider, which limitations are set in place for the challenge. If you allow the AI to play with unlimited APM, many deficits in strategy can simply be overcome with technique, utilizing pure speed and precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you restrict the challenge to a small map pool, the AI can be trained much more efficiently. If the AI has to be able to navigate arbitrary maps, the problem gets much more complex. The same is true for limiting matchups or hero selection. The more limitations you put in place, the easier it is to implement a strong AI&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My conclusion&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While it is difficult to weight all those aspects against each other, I feel like it would be easier to implement a strong &lt;strong&gt;strategic&lt;/strong&gt; AI for Dota2 than for SC2, but the leverage of an AI using a strong &lt;strong&gt;battle technique&lt;/strong&gt; is larger for SC2. This doesn't mean a strong Dota2 battle technique can't reach super human level as well, but the skill ceiling is higher in SC2.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-21T09:36:26.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3847" PostTypeId="1" CreationDate="2017-08-21T09:57:31.947" Score="4" ViewCount="586" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial Intelligence is any device that perceives its environment&#xA;  and takes actions that maximize its chance of success at some goal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I got this definition from &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; that cited &quot;Russell and Norvig (2003), Artificial Intelligence: A Modern Approach&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Transistor&quot; rel=&quot;nofollow noreferrer&quot;&gt;transistor&lt;/a&gt; is a device that amplifies or switches electronic signals when it received an input signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could one say the transistor is the AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is certainly a basic building block of every AI out there, but is it an AI itself, albeit the most basic one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking at it from a technological and economic point of view, leaving philosophy aside. From an economic perspective it seems to be an AI because transistor does useful work that it took an intelligent human to perform less than a century ago. And it does it completely on its own.&lt;/p&gt;&#xA;" OwnerUserId="9164" LastEditorUserId="1671" LastEditDate="2017-09-11T22:31:06.763" LastActivityDate="2020-08-11T10:25:57.063" Title="Is transistor the first artificial intelligence?" Tags="&lt;history&gt;&lt;terminology&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3848" PostTypeId="2" ParentId="3847" CreationDate="2017-08-21T10:23:57.643" Score="6" Body="&lt;p&gt;Replacing my previous ill conceived answer with this &lt;a href=&quot;http://incompleteideas.net/sutton/IncIdeas/DefinitionOfIntelligence.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of Intelligence from Richard Sutton&lt;/a&gt; (a founder and leader Reinforcement Learning) should answer your question. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;John McCarthy long ago gave one of the best definitions: &quot;Intelligence&#xA;  is the computational part of the ability to achieve goals in the&#xA;  world”. That is pretty straightforward and does not require a lot of&#xA;  explanation. It also allows for intelligence to be a matter of degree,&#xA;  and for intelligence to be of several varieties, which is as it should&#xA;  be. Thus a person, a thermostat, a chess-playing program, and a&#xA;  corporation all achieve goals to various degrees and in various&#xA;  senses. For those looking for some ultimate ‘true intelligence’, the&#xA;  lack of an absolute, binary definition is disappointing, but that is&#xA;  also as it should be. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The part that might benefit from explanation is what it means to&#xA;  achieve goals. What does it mean to have a goal? How can I tell if a&#xA;  system really has a goal rather than seems to? These questions seem&#xA;  deep and confusing until you realize that a system having a goal or&#xA;  not, despite the language, is not really a property of the system&#xA;  itself. It is in the relationship between the system and an observer.&#xA;  (In Dennett's words, it is a ‘&lt;a href=&quot;https://en.wikipedia.org/wiki/Intentional_stance&quot; rel=&quot;nofollow noreferrer&quot;&gt;stance&lt;/a&gt;’ that the observer take with&#xA;  respect to the system.) &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;What is it in the relationship between the system and the observer&#xA;  that makes it a goal-seeking system? It is that the system is most&#xA;  usefully understood (predicted, controlled) in terms of its outcomes&#xA;  rather than its mechanisms. Thus, for a home-owner a thermostat is&#xA;  most usefully understood in terms of its keeping the temperature&#xA;  constant, as achieving that outcome, as having that goal. But if i am&#xA;  an engineer designing a thermostat, or a repairman fixing one, then i&#xA;  need to understand it at a mechanistic level—and thus it does not have&#xA;  a goal. The thermostat does or does not have a goal depending of the&#xA;  observer. Another example is the person playing the chess computer. If&#xA;  I am a naive person, and a weaker player, I can best understand the&#xA;  computer as having the goal of beating me, of checkmating my king. But&#xA;  if I wrote the chess program (and it does not look very deep) I have a&#xA;  mechanistic way of understanding it that may be more useful for&#xA;  predicting and controlling it (and beating it).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Putting these two together, we can define intelligence concisely&#xA;  (though without much hope of being genuinely understood without&#xA;  further explanation): Intelligence is the computational part of the&#xA;  ability to achieve goals. A goal achieving system is one that is more&#xA;  usefully understood in terms of outcomes than in terms of mechanisms&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2017-08-21T13:09:50.660" LastActivityDate="2017-08-21T13:09:50.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3849" PostTypeId="2" ParentId="3847" CreationDate="2017-08-21T12:48:21.413" Score="1" Body="&lt;p&gt;Transistor is very similar in its function to single neuron and because of that one transistor could be considered to be a very tiny neural network - from this perspective it could be considered to be a form of artificial intelligence on their own. But transistor is not the first building block of AI, the first building blocks are the smallest particles that are possible in phisics - those are the building blocks of any machine.&#xA;Also the definition is somehow hard to accept, as there is no reason why thermostat can be considered AI on their own and transistor not. Since thermostat needs an environment in which it works in a way that fulfills the definition of AI, so the transistor could be put in such environment - there's no difference. We consider that something is AI when it maximizes chnace of some goal, but who is defining  this goal? We do. So if you say that the goal of some AI system is to have the output current equal to input current multiplied by factor of x, then you can say that bipolar junction transistor is the AI that does exactly that. Of course you can build a processor and memory from such transistors, then write software for eg. face recognition for a computer builded from them. It is all about connecting algorithms together to perform more sophisticated functions. The first &quot;logical gates&quot; (and builing blocks of AI) are the smallest particles possible, the smallest particles that can interact with each other to produce some output available to be used in next interaction - the whole universe works bacause of that - the matter just follows the algorithms - laws of phisics. On the side note: The only thing that for me is not algorithmical is consciousness, I think it is impossible to be created by just algorithms working together. Algorithms doesn't have any feelings and any number of them will not have feelings either - this will only process information without any consciousness inside. Intelligence is computational, consiciousness somehow reaches beyond phisical/algorithmical world - it is worth consideration - I personally think that we are like players in a game and there is other dimension/world from which our consciousness comes from, good question is: what are the rules of this game?&lt;/p&gt;&#xA;" OwnerUserId="9166" LastEditorUserId="9166" LastEditDate="2017-09-11T21:44:01.733" LastActivityDate="2017-09-11T21:44:01.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3850" PostTypeId="1" CreationDate="2017-08-21T15:18:15.710" Score="23" ViewCount="26930" Body="&lt;p&gt;Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated &lt;a href=&quot;https://softwareengineering.stackexchange.com/q/124233/145653&quot;&gt;pseudo-random number&lt;/a&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does something like this already exist? If research is already done on this or something related (to the prediction of pseudo-random numbers), can anyone point me to the right resources? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am looking at this library and its related links.&#xA;&lt;a href=&quot;https://github.com/Vict0rSch/deep_learning/tree/master/keras/recurrent&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/Vict0rSch/deep_learning/tree/master/keras/recurrent&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9170" LastEditorUserId="2444" LastEditDate="2019-11-21T13:52:11.607" LastActivityDate="2020-05-05T16:47:53.617" Title="Can a neural network be used to predict the next pseudo random number?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;prediction&gt;&lt;randomness&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="3852" PostTypeId="1" CreationDate="2017-08-21T15:40:10.473" Score="4" ViewCount="528" Body="&lt;p&gt;To my understanding, logistic regression is an extension of naive Bayes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$X = \{x_1, x_2, \dots, x_N \}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y = \{0, 1\}$&lt;/span&gt;, each &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is i.i.d and &lt;span class=&quot;math-container&quot;&gt;$P(x_i \mid Y=y_k) \sim \mathcal{N}(\mu, \sigma^2)$&lt;/span&gt; is a Gaussian distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to create a &lt;em&gt;linear decision surface&lt;/em&gt;, we take the assumption of each pdf &lt;span class=&quot;math-container&quot;&gt;$P(x_i \mid y_k)$&lt;/span&gt; having variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; independent of the value of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, i.e. &#xA;&lt;span class=&quot;math-container&quot;&gt;$\sigma_{i, k} = \sigma_i$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$i \rightarrow x_i, k \rightarrow y_k$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we end up learning the coefficients &lt;span class=&quot;math-container&quot;&gt;$(\omega_0, \omega_i)$&lt;/span&gt;  that represent the &lt;em&gt;linear decision surface&lt;/em&gt; in following equation (linear decision surface):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{P(Y=0 \mid X)}{P(Y=1 \mid X)} = \omega_0 + \sum_{i} \omega \cdot x_i&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though the derivation of Linear Regression coefficients&lt;span class=&quot;math-container&quot;&gt;$(\omega_0, \omega_i)$&lt;/span&gt; involves the assumption of conditional independent &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, &lt;em&gt;why is it said that learning these coefficients from training data are &quot;somewhat more free&quot; from conditional independence assumption, as compared to learning the regular Bayesian Distribution coefficients &lt;span class=&quot;math-container&quot;&gt;$\mu, \sigma$&lt;/span&gt;&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across this while following &lt;a href=&quot;https://youtu.be/z_xPu9KrgCY?t=2008&quot; rel=&quot;nofollow noreferrer&quot;&gt;this course here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8720" LastEditorUserId="2444" LastEditDate="2019-04-19T14:39:06.617" LastActivityDate="2019-04-19T14:39:06.617" Title="Is logistic regression more free from the conditional independence assumption than naive Bayes?" Tags="&lt;machine-learning&gt;&lt;logistic-regression&gt;&lt;naive-bayes&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3853" PostTypeId="2" ParentId="3825" CreationDate="2017-08-21T16:37:06.807" Score="1" Body="&lt;p&gt;Short answer: &lt;strong&gt;One to many&lt;/strong&gt;&lt;br&gt;&#xA;Long answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that you use a 3D convolution in a CNN. Each kernel has the size of n*m*C (C is the numbers of feature maps) and every feature map has its own kernel(=weights) and bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The size of the layer 2 is: 9x9x10 (stride 1,no padding), the kernel size is 3x3x10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the dimension of the next layer would be 3x3xn (n are the number of kernels that layer 2 has)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here I found a very clear explanation:&lt;br&gt;&#xA;&lt;a href=&quot;https://www.quora.com/How-does-the-second-convolution-layer-work-on-multiple-feature-maps&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/How-does-the-second-convolution-layer-work-on-multiple-feature-maps&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="9111" LastEditorUserId="9111" LastEditDate="2017-08-25T07:12:50.843" LastActivityDate="2017-08-25T07:12:50.843" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3854" PostTypeId="1" AcceptedAnswerId="3858" CreationDate="2017-08-21T16:53:00.563" Score="4" ViewCount="156" Body="&lt;p&gt;I studied the articles on &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/&quot; rel=&quot;noreferrer&quot;&gt;Neural Networks and Deep Learning&lt;/a&gt; from Michael Nielsen and developed a simple neural network based on his examples. I understand how backpropagation works and I already taught my neural network to not only play TicTacToe but also improve his own play by learning from his own successes using backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going forward with my experiments, I am facing the problem, that I won't always be able to show the network good moves to use for learning (maybe because I simply don't know what is correct in a certain situation), but I might be required to show it bad moves to avoid (because some of the bad moves are obvious). Teaching the network what to do using backpropagation is easy, but I haven't found a way to teach it what to avoid using similar techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to teach simple neural networks using negative examples like this or do I need other techniques? My gut feeling says, that it might be possible to &quot;invert&quot; gradient descent into gradient ascent to solve this problem. Or is it more complicated than this?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-22T06:11:17.763" Title="Can a neural network learn to avoid wrong decisions using backpropagation?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3855" PostTypeId="2" ParentId="3847" CreationDate="2017-08-21T19:24:16.397" Score="8" Body="&lt;p&gt;I think it might come down to whether the transistor is making a decision.  If the transistor is being used as a switch, that would seem to qualify as a decision, even though it's an extremely rudimentary decision.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence, in reference to Artificial (or Algorithmic) Intelligence, is not restricted to high intelligence.  A brute force Tic-Tac-Toe AI has extremely low, narrow intelligence but still constitute AI.  An automatic switch, which makes the most simple decision possible, a binary choice, would seem to be the most basic form of intelligence.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Norvig's definition seems rooted in game theory, which is important in terms of utility of intelligence.  But in a condition of intractability one is only assuming one's decision is more optimal than other choices. Outcomes can be evaluated, and a determination made as to whether the algorithm was &quot;smart&quot; or &quot;dumb&quot;, but these terms refer to relative positions on a spectrum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth noting this fundamental definition of AI opens up a can of worms in that pre-transistor, automated mechanical switches such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Strowger_switch&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strowger switch&lt;/a&gt; would also probably qualify. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And automata do not have to be electrical.  The &lt;a href=&quot;https://en.wikipedia.org/wiki/Automation#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;History of Automation wiki&lt;/a&gt; suggests the first feedback control system was for a water clock invented by &lt;a href=&quot;https://en.wikipedia.org/wiki/Ctesibius&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ctesibius&lt;/a&gt;.  This device dates to the 3rd Century BCE, and water clocks were said to be the most accurate time pieces until &lt;a href=&quot;https://en.wikipedia.org/wiki/Christiaan_Huygens#Horology&quot; rel=&quot;nofollow noreferrer&quot;&gt;Huygens&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of intelligence I tend to think of as &lt;a href=&quot;https://en.wikipedia.org/wiki/Autonomic_nervous_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;autonomic&lt;/a&gt;, in the sense of involuntary, and distinct from higher functions, which are more commonly associated with &quot;intelligence&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: The &lt;a href=&quot;https://en.wikipedia.org/wiki/Autonomic_computing#Characteristics_of_autonomic_systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;characteristics of an autonomic system&lt;/a&gt; in a computing context are quite interesting and include &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-optimization&lt;/a&gt;, self-learning, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-awareness&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-awareness&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ADDENDUM: After much thought, in reference to @JT's answer, I can't see how decisions can be separated from goals—there needs to be an intent, or the decision is merely random. This might prompt the question &quot;can a simple switch be said to have a goal?&quot;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-12T03:33:16.617" LastActivityDate="2018-04-12T03:33:16.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3857" PostTypeId="2" ParentId="3850" CreationDate="2017-08-21T21:09:40.843" Score="16" Body="&lt;p&gt;If we are talking about a perfect RNG, the answer is a clear &lt;strong&gt;no&lt;/strong&gt;. It is impossible to predict a truly random number, otherwise it wouldn't be truly random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we talk about pseudo RNG, things change a little. Depending on the quality of the PRNG, the problem ranges from easy to almost impossible. A very weak PRNG like the one &lt;a href=&quot;https://xkcd.com/221/&quot; rel=&quot;noreferrer&quot;&gt;XKCD&lt;/a&gt; published could of course be easily predicted by a neural network with little training. But in the real world things look different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neural network could be trained to find certain patterns in the history of random numbers generated by a PRNG to predict the next bit. The stronger the PRNG gets, the more input neurons are required, assuming you are using one neuron for each bit of prior randomness generated by the PRNG. The less predictable the PRNG gets, the more data will be required to find some kind of pattern. For strong PRNGs this is not feasable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a positive note, it is helpful that you can generate an arbitrary amount of training patterns for the neural network, assuming that you have control over the PRNG and can produce as many random numbers as you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because modern PRNGs are a key component for cryptography, extensive research has been conducted to verify that they are &quot;random enough&quot; to withstand such prediction attacks. Therefore I am pretty sure that it is not possible with currently available computational resources to build a neural network to successfully attack a PRNG that's considered secure for cryptography.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also worth noting that it is not necessary to exactly predict the output of a PRNG to break cryptography - it might be enough to predict the next bit with a certainty of a little more than 50% to weaken an implementation significantly. So if you are able to build a neural network that predicts the next bit of a PRNG (considered secure for cryptography) with a 55% success rate, you'll probably make the security news headlines for quite a while.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-21T21:09:40.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3858" PostTypeId="2" ParentId="3854" CreationDate="2017-08-21T21:21:37.783" Score="2" Body="&lt;p&gt;What you are describing is conceptually close to adversarial training. you should read more on adversarial examples and generative adversarial networks for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that there is a discriminator network, whose job is to correctly discriminate between positive and negative examples. We also have a generative network, that learns to produce &quot;adversarial examples&quot; that &quot;confuses&quot; the discriminator network. By training these two networks side by side, both networks get better at their task. But it's usually the generator network that people are more interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, the naive implementation of the method you've described (gradient ascent on incorrect examples from a network in a clean/randomly-initialized state) shouldn't work. This is because negative examples don't form a &quot;natural class&quot; (all triangles have 3 edges, all things that are not triangles however....)&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2017-08-22T06:11:17.763" LastActivityDate="2017-08-22T06:11:17.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3859" PostTypeId="2" ParentId="248" CreationDate="2017-08-21T21:24:12.957" Score="1" Body="&lt;p&gt;From a mathematics point of view one of the major issues in deep networks with several layers are &lt;strong&gt;vanishing&lt;/strong&gt; or &lt;strong&gt;unstable gradients&lt;/strong&gt;. Each additional hidden layer learns significantly slower, almost nullifying the benefit of the additional layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern deep learning approaches can improve this behavior, but in simple, old fashioned neural networks this is a well known issue. You can find a well written analysis &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for deeper study.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-21T21:24:12.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3860" PostTypeId="1" CreationDate="2017-08-22T13:31:00.553" Score="2" ViewCount="1774" Body="&lt;p&gt;Can someone explain the differance between &lt;code&gt;tf.contrib.DNNClassifier&lt;/code&gt; (learn) and &lt;code&gt;tf.estimator.DNNClassifier&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;tf.contrib.DNNClassifier&lt;/code&gt; (learn) works but gives warnings:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;WARNING:tensorflow:From C:\Anaconda3\lib.. scalar_summary ...is&#xA;  deprecated and will be removed after 2016-11-30. Please switch to&#xA;  tf.summary.scalar.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But I can load the layers names and values with &lt;code&gt;get_variable_names&lt;/code&gt; and &lt;code&gt;get_variable_value&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;tf.estimator&lt;/code&gt; works fine but how do I get the layers name and values?&#xA;See code below with a switch (if True/False) for the two versions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instalation in Windows 10&lt;br&gt;&#xA;3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:25:24) [MSC v.1900 64 bit (AMD64)]&#xA;Tensorflow is installed in the root with:&lt;br&gt;&#xA;pip install --upgrade tensorflow --ignore-installed (This is the only combination that works for me)&lt;br&gt;&#xA;pip list gives&#xA;tensorflow (1.3.0)&#xA;tensorflow-tensorboard (0.1.4)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Example of DNNClassifier for Iris plant dataset.&#xA;from __future__ import absolute_import&#xA;from __future__ import division&#xA;from __future__ import print_function&#xA;import numpy as np&#xA;from sklearn import datasets&#xA;from sklearn import metrics&#xA;from sklearn import model_selection&#xA;import os&#xA;import tensorflow as tf&#xA;from tensorflow.contrib import learn&#xA;os.environ['TF_CPP_MIN_LOG_LEVEL']='3'  # get rid oc tf_jenkins WARNING&#xA;&#xA;def main():&#xA;    # Load dataset.&#xA;    iris = datasets.load_iris()&#xA;    x_train, x_test, y_train, y_test = model_selection.train_test_split(&#xA;        iris.data, iris.target, test_size=0.2, random_state=42)&#xA;&#xA;    # Define the training inputs and train&#xA;    get_train_input_fn = tf.estimator.inputs.numpy_input_fn(&#xA;        x={'x':x_train}, y=y_train, num_epochs=None, shuffle=True)&#xA;    #Predict &#xA;    get_test_input_fn = tf.estimator.inputs.numpy_input_fn(&#xA;        x={'x':x_test}, y=y_test, num_epochs=1, shuffle=True)&#xA;&#xA;    # Build 3 layer DNN with 10, 20, 10 units respectively.    &#xA;    feature_columns = [tf.feature_column.numeric_column('x', shape=np.array(x_train).shape[1:])]    &#xA;    #Select code &#xA;    if True: # tf.estimator...&#xA;        classifier = tf.estimator.DNNClassifier(feature_columns=feature_columns, &#xA;            hidden_units=[10, 20, 10], n_classes=3)&#xA;        #Train&#xA;        classifier.train(input_fn=get_train_input_fn, steps=200)&#xA;&#xA;        scores = classifier.evaluate(input_fn=get_test_input_fn)&#xA;        print('Accuracy (tf.estimator): {0:f}'.format(scores['accuracy']))&#xA;        # get_variable_names, get_variable_value. How ?????&#xA;&#xA;    else: # learn (tf.contrib)&#xA;        classifier = learn.DNNClassifier(feature_columns=feature_columns,&#xA;            hidden_units=[10, 20, 10], n_classes=3)&#xA;        #Train    &#xA;        classifier.fit(input_fn=get_train_input_fn, steps=200) &#xA;&#xA;        scores = classifier.evaluate(input_fn=get_test_input_fn,steps=1)&#xA;        print(&quot;\nTest Accuracy (learn): {0:f}\n&quot;.format(scores[&quot;accuracy&quot;]))&#xA;&#xA;        # Get data&#xA;        names=classifier.get_variable_names()&#xA;        data={}&#xA;        for name in names:&#xA;            data[name]=classifier.get_variable_value(name)&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9185" LastEditorUserId="9647" LastEditDate="2017-12-18T21:33:06.630" LastActivityDate="2018-05-18T00:28:54.013" Title="Tensorflow: tf.contrib.DNNClassifier(..) or tf.estimator.DNNClassifier(..)" Tags="&lt;classification&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2018-05-18T00:44:00.977" ContentLicense="CC BY-SA 3.0" />
  <row Id="3861" PostTypeId="1" AcceptedAnswerId="3866" CreationDate="2017-08-22T13:45:38.643" Score="4" ViewCount="1421" Body="&lt;p&gt;I wonder on the following concept:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A given neural network gets two audio input (preferably music) and gives a real number between 0 and 1 which describes &quot;similarity&quot; between the second and the first track.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As far as my understanding of neural networks go, the problem fits the concept of NNs, as pattern recognition in music can help determine similarities and discrepancies in audio, see voice recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, due to the nature of long and complex inputs, and the vague nature of learning datasets (how similar, for instance, Diana Ross &quot;It's your move&quot;, and The Vaporwave legend &quot;Floral Shoppe&quot; exactly are? 0.9? 0.6? other?), such a network would be extremely slow and convoluted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible today to build and train such a model? If yes, how would it look like?&lt;/p&gt;&#xA;" OwnerUserId="1270" LastEditorUserId="16565" LastEditDate="2019-05-15T18:28:48.830" LastActivityDate="2019-05-15T18:28:48.830" Title="Is music/sound similarity comparison feasible on neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;pattern-recognition&gt;&lt;voice-recognition&gt;&lt;similarity&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3862" PostTypeId="1" CreationDate="2017-08-22T14:26:51.633" Score="10" ViewCount="3423" Body="&lt;p&gt;Multiple resources I referred to mention that MSE is great because it's convex. But I don't get how, especially in the context of neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;: training dataset&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;: targets&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt;: the set of parameters of the model &lt;span class=&quot;math-container&quot;&gt;$f_\Theta$&lt;/span&gt; (a neural network model with non-linearities)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\operatorname{MSE}(\Theta) = (f_\Theta(X) - Y)^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why would this loss function always be convex? Does this depend on &lt;span class=&quot;math-container&quot;&gt;$f_\Theta(X)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="9189" LastEditorUserId="2444" LastEditorDisplayName="user1440" LastEditDate="2019-08-29T12:11:08.787" LastActivityDate="2019-08-29T12:11:08.787" Title="Is the mean-squared error always convex in the context of neural networks?" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3863" PostTypeId="2" ParentId="3836" CreationDate="2017-08-22T16:13:54.050" Score="1" Body="&lt;p&gt;It would depend on the level of human authenticity you want to give the AI. For one, in StarCraft II humans are limited by their ability to look at only one particular place in the map at once and of making only one action at a time. Computers aren't necessarily limited by this constraint so you would have to artificially create it. It's a similar story with Dota2 except in this scenario the AI would be able to control all 5 characters at once with precise synchronization which would be unmatched by even the best teams. So again, you would have to artificially created constraints on the agent so that it was forced to optimize for team work and communication. I think a challenging and interesting problem for Dota2 would be to have 5 separate agents working together like humans usually do in the game. It would be interesting to see for example if they could generate a &quot;gaming language&quot; in which to signal cooperative operations on the map as well as setting up ganks and pushes for example. This would give researchers a good environment in which to test how the interactions between separate agents would evolve alongside a common goal.&lt;/p&gt;&#xA;" OwnerUserId="9110" LastActivityDate="2017-08-22T16:13:54.050" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3864" PostTypeId="1" CreationDate="2017-08-23T07:03:27.030" Score="0" ViewCount="61" Body="&lt;p&gt;The textbook Deep Learning by Goodfellow, Bengio, and Courville can be viewed in individual html chapters &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently reading Chapter 15 on Representation Learning and saw this for algorithm 15.1 on page 530:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n9JEx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n9JEx.png&quot; alt=&quot;Page 530&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There no condition after &lt;code&gt;end for&lt;/code&gt; and similarly no end condition for &lt;code&gt;end if&lt;/code&gt;. Is that intentional? If so, what is the correct way to interpret it? I haven't encountered algorithm syntax like this in the past. &lt;/p&gt;&#xA;" OwnerUserId="9197" LastActivityDate="2017-08-23T07:36:09.127" Title="Is this page of Goodfellow's Deep Learning Textbook missing text?" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3865" PostTypeId="2" ParentId="3864" CreationDate="2017-08-23T07:36:09.127" Score="3" Body="&lt;p&gt;There is no industry standard for psuedocode as it is read by humans (for the moment ;) and the author can decide how best to represent the concepts of the algorithm with the aim of balancing precision and understandability. In this case there is a &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;FOR ... END FOR&lt;/pre&gt; &#xA;&#xA;&lt;p&gt;block which is iterated over k for 1 to m. And an &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;IF ... END IF&lt;/pre&gt; &#xA;&#xA;&lt;p&gt;block that is run if fine-tuning evaluates to true.&lt;/p&gt;&#xA;" OwnerUserId="9199" LastActivityDate="2017-08-23T07:36:09.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3866" PostTypeId="2" ParentId="3861" CreationDate="2017-08-23T08:57:32.103" Score="3" Body="&lt;p&gt;Yes, it is possible, even if the best approach could be different from  neural networks. Anyway, you should extract some significant features from the audio (energy, onsets, root frequencies, and other). Usually, more features than those really needed are extracted and afterwards the most sigificant are selected through some algorithm (e.g. PCA). In this way you will obtain an array of features (say between 10 and 100 features) with which you can train your NNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that NNs do not tell you &lt;em&gt;why&lt;/em&gt; two audio are similar but only if they are or not. This is a big disadvantage. Instead, algorithms based on grey-box modeling such as rule or case based algorithm (maybe using fuzzy logic) could be more useful, provided that you have a deeper knowledge of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References and deepening sources: &lt;a href=&quot;http://smc.dei.unipd.it/education.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMC Lab from University of Padua education material&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8677" LastEditorUserId="8677" LastEditDate="2017-08-23T14:56:34.697" LastActivityDate="2017-08-23T14:56:34.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3867" PostTypeId="2" ParentId="3862" CreationDate="2017-08-23T10:31:35.500" Score="1" Body="&lt;p&gt;&lt;strong&gt;Convexity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$x ∈ Χ$&lt;/span&gt; is convex, if, for any &lt;span class=&quot;math-container&quot;&gt;$x_1 ∈ Χ$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x_2 ∈ Χ$&lt;/span&gt; and for any &lt;span class=&quot;math-container&quot;&gt;$0 ≤ λ ≤ 1$&lt;/span&gt;,&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$f(λ x_1 + (1 − λ) x_2) ≤ λf(x_1) + (1 − λ) f (x_2).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It can be proven that such convex &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; has one global minimum.  A unique global minimum eliminates traps created by local minima that can occur in algorithms that attempt to achieve convergence on a global minimum, such as the minimization of an error function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although an error function may be 100% reliable in all continuous, linear contexts and many non-linear contexts, it does not mean the convergence on a global minimum for all possible non-linear contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mean Square Error&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a function &lt;span class=&quot;math-container&quot;&gt;$s(x)$&lt;/span&gt; describing ideal system behavior and a model of the system &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the parameter vector, matrix, cube, or hypercube and &lt;span class=&quot;math-container&quot;&gt;$1 ≤ n ≤ N$&lt;/span&gt;), created rationally or via convergence (as in neural net training), the mean square error (MSE) function can be represented as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$e(β) := N^{-1} \sum_{n} [a(x_n) − s(x_n)]^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The material you are reading is probably not claiming that &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$s(x)$&lt;/span&gt; are convex with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, but that &lt;span class=&quot;math-container&quot;&gt;$e(β)$&lt;/span&gt; is convex with respect to &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s(x)$&lt;/span&gt; no matter what they are.  This later statement can be proven for any continuous &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s(x)$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Confounding the Convergence Algorithm&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the question is whether a specific &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt; and method of achieving an &lt;span class=&quot;math-container&quot;&gt;$s(x)$&lt;/span&gt; that approximates the &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt; within a reasonable MSE convergence margin can be confounded, the answer is, &quot;Yes.&quot;  That is why MSE is not the only error model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way summarize is that &lt;span class=&quot;math-container&quot;&gt;$e(β)$&lt;/span&gt; should be defined or chosen from a set of stock convex error models based on the following knowledge.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Known properties of the system &lt;span class=&quot;math-container&quot;&gt;$s(x)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;The definition of the approximation model &lt;span class=&quot;math-container&quot;&gt;$a(x, p)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Tensor used to generate the next state in the convergent sequence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The set of stock convex error models certainly includes the MSE model because of its simplicity and computational thrift.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorDisplayName="user1440" LastEditDate="2018-11-22T06:58:43.990" LastActivityDate="2018-11-22T06:58:43.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3868" PostTypeId="5" CreationDate="2017-08-23T13:13:46.147" Score="0" Body="&lt;p&gt;Reinforcement Learning (RL) is a machine learning technique influenced by behavioral psychology. It is based on agents associating reward signals to actions they made on a particular state in the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory behind reinforcement learning frames problems as &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Processes (MDPs)&lt;/a&gt;. Learning problems that fit to the MDP framework are suitable for using RL as a solution. This means that they should have the following components:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An &lt;em&gt;Environment&lt;/em&gt; which follows a consistent set of rules, and which evolves over a series of &lt;em&gt;time steps&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;An &lt;em&gt;Agent&lt;/em&gt; that is in the Environment, and can make meaningful decisions about how to act.&lt;/li&gt;&#xA;&lt;li&gt;Multiple &lt;em&gt;States&lt;/em&gt; which influence how the Environment rules will be applied. &lt;/li&gt;&#xA;&lt;li&gt;Multiple &lt;em&gt;Actions&lt;/em&gt; which the Agent may select on each time step, and which also influence the Environment.&lt;/li&gt;&#xA;&lt;li&gt;A system of scalar &lt;em&gt;Reward&lt;/em&gt; values, that provide feedback on a task that the Agent is performing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For RL theory to hold, the State should possess the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt;, which is to say that a single state should have fixed distributions of next State and Reward, dependent only on the current State and Action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In RL control problems, the goal of optimization is to maximize a long-term measure of reward. Most commonly this is the discounted sum of all rewards, or the average reward per time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many RL algorithms, based on different approaches to solving MDPs. They include Monte Carlo Control, SARSA, Q Learning, REINFORCE, Actor-Critic. Implementations with deep neural networks include DQN, A3C, A2C, PPO, TRPO, DPPG.&lt;/p&gt;&#xA;" OwnerUserId="9210" LastEditorUserId="1847" LastEditDate="2018-09-14T10:27:37.213" LastActivityDate="2018-09-14T10:27:37.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3869" PostTypeId="4" CreationDate="2017-08-23T13:13:46.147" Score="0" Body="For questions related to reinforcement learning, i.e. a machine learning technique where we imagine an agent that interacts with an environment (composed of states) in time steps by taking actions and receiving rewards (or reinforcements), then, based on these interactions, the agent tries to find a policy (i.e. a behavioural strategy) that maximizes the cumulative reward (in the long run), so the goal of the agent is to maximize the reward." OwnerUserId="9210" LastEditorUserId="2444" LastEditDate="2020-08-18T14:27:59.437" LastActivityDate="2020-08-18T14:27:59.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3870" PostTypeId="2" ParentId="3774" CreationDate="2017-08-23T13:38:29.403" Score="2" Body="&lt;p&gt;I skimmed through your question and understood that the state/action space is finite, so in this case, RL would be a good option for storage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most basic RL technique will keep track of a matrix &lt;strong&gt;Q&lt;/strong&gt; &amp;in; &amp;#8477;&lt;sup&gt;&lt;em&gt;s&lt;/em&gt;&amp;times;&lt;em&gt;a&lt;/em&gt;&lt;/sup&gt;, where &lt;em&gt;s&lt;/em&gt; is number of possible states, and &lt;em&gt;a&lt;/em&gt; is number of possible actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to a small overhead of agent's parameters:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;alpha; &amp;in; [0,1] learning parameter&lt;/li&gt;&#xA;&lt;li&gt;&amp;gamma; &amp;in; [0,1] discount rate, &lt;/li&gt;&#xA;&lt;li&gt;&amp;epsilon; &amp;in; [0,1]  exploration rate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you have a total of 81 states and 10 actions, then your storage would be in the neighborhood of 850 double words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your AI will track different states per location, e.g. possible values 0 to 9 + a null, then your state space will grow to 11 * 81 = 968 and the final value matrix will be &lt;strong&gt;Q&lt;/strong&gt; &amp;in; &amp;#8477;&lt;sup&gt;968&amp;times;10&lt;/sup&gt; &amp;asymp; 20Kb.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am terrible sorry, I made a terrible mistake in my calculations.&#xA;if you are to keep track of 11 possible values in 81 locations, then the correct number should be 11&lt;sup&gt;81&lt;/sup&gt;&amp;times; double word size, that is a big number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if this is Sudoku and you  and you are not considering illegal moves as possible states, then definitely this will be a way smaller number, but you will still keep track of an action space that is as big as the state space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might need to use function approximation instead, or some sort of action state space compression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sorry &lt;/p&gt;&#xA;" OwnerUserId="9210" LastEditorUserId="9210" LastEditDate="2017-08-23T19:24:52.463" LastActivityDate="2017-08-23T19:24:52.463" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3871" PostTypeId="2" ParentId="3801" CreationDate="2017-08-23T14:38:08.203" Score="0" Body="&lt;p&gt;Many deep neural networks have a static data-flow graph, which roughly means that the structure of its computation (its computation graph) remains stable over different inputs. This is good since we can leverage this feature for performance, such as by mini-batching (processing a bunch of inputs at once).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But some neural networks could have a different computation graph for each input. This causes some problems (batching difficulties, graph construction is computationally expensive), and hence these networks are a bit hard to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper you link overcomes this problem by proposing a method that can batch several computation graphs into one. Then, we can do our usual NN techniques. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Benefits are speedups, which incentivizes researchers to explore different structures and be more creative I guess.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The advantage of Dynamic Computational Graphs appears to include the ability to adapt to a varying quantities in input data. It seems like there may be automatic selection of the number of layers, the number of neurons in each layer, the activation function, and other NN parameters, depending on each input set instance during the training. Is this an accurate characterization?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is incorrect.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-08-23T14:38:08.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3872" PostTypeId="2" ParentId="3702" CreationDate="2017-08-23T14:41:53.360" Score="2" Body="&lt;p&gt;Why not use a single agent to control the intersection with the following rules:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;define 8 traffic lights&lt;/li&gt;&#xA;&lt;li&gt;Each light has two possible values, 0 or 1, with 0 = Red and 1 = Green&lt;/li&gt;&#xA;&lt;li&gt;Lights are as follows:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0 = North to South&lt;/li&gt;&#xA;&lt;li&gt;1 = North to East (Left Turn)&lt;/li&gt;&#xA;&lt;li&gt;2 = East to West&lt;/li&gt;&#xA;&lt;li&gt;3 = East to South (Left Turn)&lt;/li&gt;&#xA;&lt;li&gt;4 = South to North&lt;/li&gt;&#xA;&lt;li&gt;5 = South to West (Left Turn)&lt;/li&gt;&#xA;&lt;li&gt;6 = West to East&lt;/li&gt;&#xA;&lt;li&gt;7 = West to North (Left Turn)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;map lights into a bitmap of 8 bits, with the least significant bit (left bit) is the North to south &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given the above, we have the following valid moves, which are both State Space  and Action Space :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;State/Move 0 - Vertical Moves only   = 10001000&lt;/li&gt;&#xA;&lt;li&gt;State/Move 1 - Horizontal Moves only = 00100010&lt;/li&gt;&#xA;&lt;li&gt;State/Move 2 - Originated from North only = 11000000&lt;/li&gt;&#xA;&lt;li&gt;State/Move 3 - Originated from East only = 00110000&lt;/li&gt;&#xA;&lt;li&gt;State/Move 4 - Originated from South only = 00001100&lt;/li&gt;&#xA;&lt;li&gt;State/Move 5 - Originated from West only = 00000011&lt;/li&gt;&#xA;&lt;li&gt;State/Move 6 - Vertical Diagonal only = 01000100&lt;/li&gt;&#xA;&lt;li&gt;State/Move 7 - Horizontal Diagonal only = 00010001&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;i.e. &lt;strong&gt;S&lt;/strong&gt; = {10001000, 10001000, 00100010, 11000000, 00110000, 00001100, 00000011, 01000100, 00010001}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and  &lt;strong&gt;A&lt;/strong&gt; = {10001000, 10001000, 00100010, 11000000, 00110000, 00001100, 00000011, 01000100, 00010001}&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GRrTr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GRrTr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Now we have &lt;em&gt;a&lt;/em&gt; &amp;in; &lt;strong&gt;A&lt;/strong&gt; and &lt;em&gt;s&lt;/em&gt; &amp;in; &lt;strong&gt;S&lt;/strong&gt; and at any given &lt;em&gt;s&lt;/em&gt;, &lt;em&gt;a&lt;/em&gt; will cause the next state &lt;em&gt;s'&lt;/em&gt; to have the same value of &lt;em&gt;a&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Final component to complete the moel, is the reward signal, which can be simply the negative sum of time spent per car in the intersection &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e.     r&lt;sub&gt;t+1&lt;/sub&gt; = -&amp;Sum;&lt;sub&gt;c&lt;sub&gt;i&lt;/sub&gt; &amp;in; &lt;strong&gt;Cars&lt;/strong&gt;&lt;/sub&gt; time(c&lt;sub&gt;i&lt;/sub&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a single agent, will be able to optimize the solution to avoid punishment (negative value caused by wait time) &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You can increase the state space to double or triple its size to accommodate for time, by appending a time counter component to the bitmap. &#xA;in addition to an extra (9th) action of do nothing, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should give a more subtle traffic controller. &lt;em&gt;e.g.&lt;/em&gt; Vertical Traffic Only, at timer = 0 can be represented by 100010000000 or in hex 880, then at time 1 = 881, and so on until time 15 to be 88F. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update 2&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can find the source code &lt;a href=&quot;https://aawadall.github.io/SmartTrafficIntersection/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://github.com/aawadall/SmartTrafficIntersection.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9210" LastEditorUserId="9210" LastEditDate="2017-08-27T03:52:11.907" LastActivityDate="2017-08-27T03:52:11.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3873" PostTypeId="1" CreationDate="2017-08-23T15:41:41.127" Score="4" ViewCount="110" Body="&lt;p&gt;Let's take our standard &lt;a href=&quot;https://wiki.lesswrong.com/wiki/Paperclip_maximizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;paperclip maximizer General AI&lt;/a&gt; and attempt to obtain precisely one million paper clips, over course of a year, without destroying the universe in the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most maximization directives make the process run-away. As cheaply as possible will crash world economy. As good clips as possible will turn the universe into super-synthesizer that assembles atom-perfect paperclips. Adding a deadline on these maximization processes will probably result in terrorizing the staff into readjusting the deadline, or invention of time travel (after consuming the solar system to invent it.) Minimizing resource usage would likely result in closure of all industry world-wide. You know, the standard horror scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about the directive of minimizing AI's influence on the world while completing the task? Would it be safe, or can you spot a runaway scenario where it could result in dire effects?&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="38" LastEditDate="2018-01-26T10:59:30.223" LastActivityDate="2018-01-28T09:49:47.373" Title="Would minimizing influence into the world be a safe directive to a general AI?" Tags="&lt;agi&gt;" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3874" PostTypeId="2" ParentId="3873" CreationDate="2017-08-23T18:47:51.900" Score="2" Body="&lt;p&gt;Telling the system &quot;minimize your interference with the world&quot; while also telling it to &quot;maximize paperclip production&quot; or whatever is interesting on at least one level, and that is this: how exactly does the system quantify &quot;interference in the world&quot;?  That seems like an ill-defined notion offhand.  But if you could quantify it, then it just becomes one more variable in an optimization problem which is a straightforward notion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detecting a runaway process in general is an interesting concept.  I am not an expert, but I am betting there is some material in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cybernetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;cybernetics&lt;/a&gt; / &lt;a href=&quot;https://en.wikipedia.org/wiki/Control_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;control theory&lt;/a&gt; literature on this topic.  It might be as simple as watching some rate-of-change (paper-clips produced per day?) and take the first and second derivatives and look for sharp changes in &lt;a href=&quot;https://en.wikipedia.org/wiki/Acceleration&quot; rel=&quot;nofollow noreferrer&quot;&gt;acceleration&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Jerk_(physics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;jerk&lt;/a&gt;.  Other algorithms from the world of &lt;a href=&quot;https://en.wikipedia.org/wiki/Anomaly_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;anomaly detection&lt;/a&gt; might also be applicable.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-08-23T18:47:51.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3875" PostTypeId="1" AcceptedAnswerId="3880" CreationDate="2017-08-23T19:54:51.713" Score="6" ViewCount="1070" Body="&lt;p&gt;I don't play nearly enough Chess to be able to answer.  For context, AlphaGo is stronger than the current strongest human player, but AlphaGo's game play has been cast as &quot;inhuman&quot; in the sense that it doesn't resemble human play. &lt;em&gt;(In Go, this can involve aesthetic qualities.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really I'm wondering about &quot;narrow&quot; application of the Imitation Game/Turing Test, where one might design an automata to play more like a human, so that human players would be unable to determine if their opponent was a human or an automata.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2021-04-19T22:40:45.630" Title="Is the play of strong Chess AI easily distinguishable from human play?" Tags="&lt;turing-test&gt;&lt;chess&gt;&lt;go&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3876" PostTypeId="2" ParentId="3875" CreationDate="2017-08-23T20:35:58.230" Score="1" Body="&lt;p&gt;I recall a friend saying that yes, it is somewhat obvious if you are playing against an AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what he said, against normal players, there is a certain rhythm and structure that &quot;makes sense&quot;. But AI play doesn't have this quality, &quot;it doesn't make sense, but it just works&quot;. This seems to echo what you mentioned about the aesthetic quality of Go&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently because of this, chess/go AI can be detected during online play.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-08-23T20:35:58.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3877" PostTypeId="2" ParentId="3873" CreationDate="2017-08-23T21:08:34.527" Score="3" Body="&lt;p&gt;I like mindrime's answer as it identifies fundamental, applicable notions.  I'll attempt another short answer from a Game Theory perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game Theory was founded in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt; principle.  Specifically, maximizing benefit while minimizing potential downside in a condition of uncertainty.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimax works very well in contexts with easily definable parameters, such as combinatorial games and procedural optimization, but it gets more tricky in real-world scenarios where too many parameters may be present, resulting in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial explosion&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue regarding real-world applications arises from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbol grounding problem&lt;/a&gt;.  In the paper clip scenario, the benefit sought (the goal) is clearly definable, and can be expressed mathematically.  By contrast, the downsides to be avoided are more difficult to define: &quot;Don't destroy the world&quot;, &quot;Don't use up resources to a degree that humans suffer&quot;, &quot;Don't harm the environment&quot; all rely on language.  This is to say they rely on terms that, at present, constitute symbols that cannot be grounded.  Thus there is room for misinterpretation that may yield unintended consequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Minimizing impact on the system&quot; (minimizing influence in the world) would be the goal, but how do you guarantee an automata has a clear understanding of what this entails in every possible scenario?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-08-23T21:13:44.180" LastActivityDate="2017-08-23T21:13:44.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3878" PostTypeId="2" ParentId="3875" CreationDate="2017-08-23T22:28:21.283" Score="4" Body="&lt;p&gt;There are three cases in which it is easily possible to distinguish strong AI play from the strong human play:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The AI is playing at superhuman skill level&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This seems obvious, but I want to mention it for the sake of completeness. The current skill ceiling of top-level chess is well known and an opponent playing way above this skill ceiling must either be an AI or a chess guru who hid in a cave for the last centuries. Applying Occam's razor I would go with the AI. So to mimic human play the AI must make sure to stay at a human ELO level.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The AI plays an obvious weak move in a losing or strange situation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;When the AI is in a bad situation and still tries to win the game without any reasonable move available, it might be tempted to play an obviously bad move, because the decision engine assigns it the highest probability of success. Such moves are considered rude at high-level play and a human player would not play them but resign. The most famous example of this is move 101 in the fourth game between &lt;a href=&quot;http://gokifu.com/s/2iq8-gokifu-20160313-Alphago(9p)-Lee_Sedol(9p).html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lee Sedol and AlphaGo&lt;/a&gt;. AlphaGo just lost a big group but still played T9 to extend it even further. This is a move a rookie player stops making after his first 10 games because it is that obvious that it does not work. A professional go player would never do this under normal circumstances. AlphaGo was out of options and played the rude move instead of resigning. This doesn't mean an AI cannot be programmed to avoid such behavior, but dire or simply very strange situations might induce moves like this, giving the nature of the AI away.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Other factors than the actual moves are considered&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Depending on the test scenario, the AI might give away its nature through other means than the actual moves. Things like moving too fast or playing without breaks for hours. But I assume the question was just focused on the actual play and not the surrounding factors.&lt;/p&gt;&#xA;&lt;p&gt;Those are the obvious examples where an AI might give away his true nature. I highly doubt that a strong AI, playing at the strong human level, can really be identified because of his &amp;quot;inhuman&amp;quot; style. I think it is more of a psychological pitfall to consider innovative moves from an AI &amp;quot;inhuman&amp;quot; and the same move from a pro player as &amp;quot;brilliant&amp;quot; if you know who is who. I am an amateur 3dan Go player - far from the top but with several years of experience under the belt - and at least I wasn't able to see anything &amp;quot;inhuman&amp;quot; in AlphaGos play so far. Ingo, it is well known that there are different styles of play, and after reviewing enough pro games one gets a sense of which player might have been educated in Korea, China, or Japan because certain patterns of strategies are more or less common in those countries. The lack of such an individual nuance might be an indication, that the player is actually a strong AI, but I am confident, that even the strongest pro wouldn't have a high success rate guessing who is human and who is the AI.&lt;/p&gt;&#xA;&lt;p&gt;In chess, there are significantly fewer options at every move, many fixed patterns especially in the opening, and therefore less room for an individual style. Therefore I consider the same challenge even more difficult than in go. So if an AI has reason to &amp;quot;play like a human&amp;quot;, this should be possible with the current level of AI technology.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="36737" LastEditDate="2021-04-19T22:40:45.630" LastActivityDate="2021-04-19T22:40:45.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3879" PostTypeId="1" AcceptedAnswerId="9833" CreationDate="2017-08-23T23:40:45.230" Score="2" ViewCount="279" Body="&lt;p&gt;I have followed the pseudocode in the &lt;a href=&quot;https://arxiv.org/abs/1212.5701&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADADELTA paper&lt;/a&gt; (top right on page 3), and wrote the following Python code for solving the optimization problem L(x) = x^2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import math&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; Eg = Ex = 0&#xA;&amp;gt;&amp;gt;&amp;gt; p = 0.95&#xA;&amp;gt;&amp;gt;&amp;gt; e = 1e-6&#xA;&amp;gt;&amp;gt;&amp;gt; x = 1&#xA;&amp;gt;&amp;gt;&amp;gt; history = [x]&#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;&amp;gt;&amp;gt;&amp;gt; for t in range(100):&#xA;...   g = 2*x&#xA;...   Eg = p*Eg + (1+p)*g*g&#xA;...   Dx = -math.sqrt(Ex+e)/math.sqrt(Eg+e)*g&#xA;...   Ex = p*Ex + (1-p)*Dx*Dx&#xA;...   x = x + Dx&#xA;...   history.append(x)&#xA;...&#xA;&amp;gt;&amp;gt;&amp;gt; print(history)&#xA;[1, 0.9992838851718654, 0.998764712958258, 0.9983330059505671, 0.9979531670003327, 0.9976084468473033, 0.997289409971406, 0.996990129861279&#xA;7, 0.9967066004793412, 0.9964359664599116, 0.9961761091943561, 0.9959254064337589, 0.9956825840862665, 0.9954466203957414, 0.99521668152795&#xA;22, 0.994992076841149, 0.9947722269598268, 0.9945566404428509, 0.9943448963795858, 0.9941366311727671, 0.9939315283404335, 0.99372931053536&#xA;84, 0.9935297332202913, 0.9933325795977276, 0.9931376565033874, 0.9929447910484566, 0.9927538278504536, 0.9925646267313296, 0.9923770607899&#xA;557, 0.9921910147771753, 0.9920063837173206, 0.991823071731977, 0.9916409910308515, 0.9914600610415929, 0.9912802076558476, 0.9911013625730&#xA;931, 0.9909234627271605, 0.990746449783029, 0.9905702696936243, 0.9903948723080783, 0.9902202110243085, 0.9900462424799209, 0.9898729262763&#xA;75, 0.9897002247321224, 0.9895281026610697, 0.9893565271732506, 0.9891854674950317, 0.989014894806555, 0.9888447820944316, 0.98867510401796&#xA;52, 0.9885058367874141, 0.9883369580529869, 0.9881684468034365, 0.9880002832732533, 0.9878324488575824, 0.9876649260340926, 0.9874976982911&#xA;152, 0.9873307500614499, 0.9871640666613011, 0.9869976342338704, 0.9868314396971792, 0.9866654706957432, 0.9864997155557601, 0.986334163243&#xA;507, 0.9861688033266723, 0.9860036259383794, 0.9858386217436783, 0.9856737819083067, 0.9855090980695381, 0.9853445623089551, 0.985180167126&#xA;9962, 0.9850159054191441, 0.9848517704536291, 0.9846877558505386, 0.9845238555622267, 0.9843600638549337, 0.984196375291527, 0.984032784715&#xA;2862, 0.983869287234659, 0.9837058782089235, 0.9835425532346933, 0.9833793081332108, 0.983216138938377, 0.9830530418854679, 0.9828900134004&#xA;96, 0.9827270500901729, 0.9825641487324376, 0.9824013062675131, 0.9822385197894608, 0.9820757865381995, 0.9819131038919638, 0.9817504693601&#xA;732, 0.9815878805766881, 0.9814253352934307, 0.9812628313743479, 0.9811003667896978, 0.9809379396106398, 0.9807755480041119, 0.980613190227&#xA;9784, 0.9804508646264328, 0.9802885696256415]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, &lt;code&gt;Eg&lt;/code&gt;, &lt;code&gt;Ex&lt;/code&gt;, &lt;code&gt;e&lt;/code&gt;, &lt;code&gt;p&lt;/code&gt;, &lt;code&gt;g&lt;/code&gt; and &lt;code&gt;Dx&lt;/code&gt; are &lt;span class=&quot;math-container&quot;&gt;$E[g^2]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E[\Delta x^2]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$\nabla L(x)$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt;, respectivelly, and &lt;code&gt;history&lt;/code&gt; is the record of all values that &lt;code&gt;x&lt;/code&gt; has obtained.&lt;/p&gt;&#xA;&lt;p&gt;For the hyperparameters &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, I use the same values that they use in the paper, and I initialize &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to 1.&lt;/p&gt;&#xA;&lt;p&gt;As can be seen when printing &lt;code&gt;history&lt;/code&gt;, the convergence is extremely slow for such a simple optimization problem, and after 100 iterations the method has barely got 2 % closer to the optimum (x = 0). It feels like I must have misunderstood some crucial part of the paper.&lt;/p&gt;&#xA;&lt;p&gt;For example, the paper claims that the update step Δx will have the same unit as x, if x has some hypothetical unit. While this is probably a desireable property, it is as far as I'm concerned not true, since the premise that &lt;span class=&quot;math-container&quot;&gt;$RMS[\Delta x]$&lt;/span&gt; has the same unit as &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is incorrect to begin with, since &lt;span class=&quot;math-container&quot;&gt;$RMS[\Delta x]_0 = \sqrt{E[\Delta x]_0 + \epsilon} = \sqrt{0 + \epsilon}$&lt;/span&gt; which is a unitless constant, so all &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; become unitless rather than having the same unit as &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. (Correct me if I'm wrong.)&lt;/p&gt;&#xA;&lt;p&gt;Have I made some error when implementing the algorithm, or why is the convergence so slow? Is it supposed to be this slow?&lt;/p&gt;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&lt;p&gt;After changing &lt;code&gt;Eg = p*Eg + (1+p)*g*g&lt;/code&gt; to &lt;code&gt;Eg = p*Eg + (1-p)*g*g&lt;/code&gt; (i.e. correcting the error spotted by Dennis), convergence is now significantly better. It still doesn't get very close to 0 after the first 100 iterations; x only goes down to 0.597. However, after 400 iterations or so, the convergence really starts to kick in and in the following 100 iterations, x goes down from its current value at 0.0156 to 2.07e-8, and then to 3.24e-57 after yet another 100 iterations!&lt;/p&gt;&#xA;&lt;p&gt;I plotted &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E[\Delta x^2]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E[g^2]$&lt;/span&gt; in logarithmic scale against the number of iterations, and this is what I found:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E1FAE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E1FAE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not really sure what this means in terms of performance &amp;quot;in the wild,&amp;quot; since this only is a toy problem, free from stochasticity in the optimization and with only one parameter.&lt;/p&gt;&#xA;&lt;p&gt;About the unit analysis, I guess &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; shouldn't be unitless as I wrote that it was, as otherwise, it cannot be added to &lt;span class=&quot;math-container&quot;&gt;$E[g^2]$&lt;/span&gt; (which has the same unit as &lt;span class=&quot;math-container&quot;&gt;$g^2$&lt;/span&gt;). Also, I guess that in practice, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; must be treated as two different constants that may have different units when added to &lt;span class=&quot;math-container&quot;&gt;$E[\Delta x^2]$&lt;/span&gt; and when added to &lt;span class=&quot;math-container&quot;&gt;$E[g^2]$&lt;/span&gt;; otherwise, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; will end up having the same unit as &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Verdict&lt;/h2&gt;&#xA;&lt;p&gt;A funny thing is that even though &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; should only be a tiny number that shouldn't affect the process significantly unless you divide by something really small, the process turns out to be highly sensitive to its value. This is because &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; essentially is the only thing that really makes &lt;span class=&quot;math-container&quot;&gt;$E\left[ \Delta x^2 \right]$&lt;/span&gt; grow. Higher &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-values will make &lt;span class=&quot;math-container&quot;&gt;$E\left[ \Delta x^2 \right]$&lt;/span&gt; grow faster and will make the optimization process go significantly faster. In my opinion, this seems to indicate a major design flaw in the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="9220" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-10T11:25:55.317" Title="Problems getting ADADELTA to converge" Tags="&lt;deep-learning&gt;&lt;algorithm&gt;&lt;optimization&gt;&lt;math&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3880" PostTypeId="2" ParentId="3875" CreationDate="2017-08-24T00:27:47.037" Score="4" Body="&lt;p&gt;AI programs that exist in today's world fall into the category of Narrow Intelligence. Narrow Intelligence&#xA;are easy to distinguish when compared to General Intelligence (ones that resemble more like humans).&lt;br&gt;&#xA;Highly advanced AI can often resemble to act like humans thought. I will like to talk about &lt;strong&gt;Deep Blue&lt;/strong&gt; here.&lt;br&gt;&lt;br&gt;&#xA;Garry Kasparov, in a &lt;a href=&quot;http://time.com/3705316/deep-blue-kasparov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;series of matches&lt;/a&gt; during Feb 1996, even though&#xA;won three matches but, had a &lt;strong&gt;&quot; shattering experience&quot;&lt;/strong&gt;. Deep Blue&#xA;flummoxed him in that first game by making a move with no immediate&#xA;material advantage; nudging a pawn into a position where it could be&#xA;easily captured. He said, &quot;&lt;em&gt;It was a wonderful and &lt;strong&gt;extremely human&lt;/strong&gt;&#xA;move&lt;/em&gt;&quot; and this threw him for a loop. He further added that he had&#xA;played a lot of computers but had never experienced anything like&#xA;this. He added, &quot;&lt;strong&gt;I could feel, I could smell, a new kind of intelligence&#xA;across the table.&lt;/strong&gt;&quot;&lt;br&gt;&#xA;Next year he lost to an improved version of Deep Blue. Here also he&#xA;lost mostly because of psychological reason. He later said that he&#xA;was riled by a move the computer made that was so surprising and&#xA;so &lt;strong&gt;un-machine-like&lt;/strong&gt; that he was sure that IBM team had cheated.&#xA;It was later found out that it was a glitch in Deep Blue. The program&#xA;came across too many options and had no clear preference and thus&#xA;played a random move.&lt;br&gt;&#xA;It was this random move that shifted the game in favour of Deep Blue.&#xA;&lt;br&gt;&lt;br&gt;&#xA;One advantage that Deep Blue (or other programs) have is the ability to perform far &lt;strong&gt;more computations&lt;/strong&gt; in a given time. Humans, on the other hand, can extend their decision tree only to &lt;em&gt;very limited depth&lt;/em&gt;. On having no clear decision to take, we take a random decision and try to learn from the experience, unlike most Narrow Intelligence programs present today. We need a &lt;strong&gt;fair amount of&#xA;randomness and learning from experience&lt;/strong&gt; to be added to the program in order&#xA;to make it more like a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to this question also depends on your position on &lt;strong&gt;&quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room Argument&lt;/a&gt;&quot;&lt;/strong&gt;.&#xA;The argument holds that a program can't give a computer understanding&#xA;or consciousness regardless of how intelligently or human like&#xA;the program may make the computer behave. Chinese Room Argument focuses on something very important when it comes to &lt;strong&gt;&quot;Consciousness v/s Simulation of Consciousness&quot;&lt;/strong&gt;. John Searle argued there that it is possible for a machine (or human) to follow a huge number of predefined rules (algorithm), in order to complete the task, without thinking or possessing the mind.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-08-24T00:27:47.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3881" PostTypeId="1" CreationDate="2017-08-24T02:56:04.813" Score="4" ViewCount="95" Body="&lt;p&gt;According to Russell and Norvig, a knowledge-based agent will only add a sentence to its knowledge base if it &lt;em&gt;follows logically&lt;/em&gt; from what it previously knows, or directly observes. To &lt;em&gt;follow logically&lt;/em&gt; essentially means that if the premises are true, then the conclusions are guaranteed to be true. So the agent will only add the sentence if it 100% sure the sentence is true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this hyperskepticism in logic justified? Couldn't the agent be more efficient if it added a sentence if it were 99% sure it were true? It could potentially add a lot more true sentences, and only occasionally add false ones. There would need to be a mechanism for unlearning sentences, but as long as the vast majority of sentences added are true, why couldn't that be done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I essentially asked this question &lt;a href=&quot;https://en.wikipedia.org/wiki/Wikipedia:Reference_desk/Miscellaneous#.22Proof.22_of_the_principle_of_skepticism.3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and someone suggested that I post it here.&lt;/p&gt;&#xA;" OwnerUserId="9225" LastActivityDate="2017-08-24T19:26:18.933" Title="Why do knowledge-based agents only add a sentence to the knowledge base when it is 100% sure the sentence is true?" Tags="&lt;logic&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3882" PostTypeId="2" ParentId="3875" CreationDate="2017-08-24T03:51:12.160" Score="1" Body="&lt;p&gt;Yes - in chess the term &quot;computer move&quot; is used to denote a move found by a chess engine that a human player would never find (often because they make some slight improvement that a human would not be able to calculate). Humans use pattern recognition and some calculation in order to understand the chess position they are in while computers are able to calculate (~30 - 120+) moves ahead, a feat that a human player cannot match. It is for this reason that one can distinguish a human move from a computer move at some times.&lt;/p&gt;&#xA;" OwnerUserId="9226" LastActivityDate="2017-08-24T03:51:12.160" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3885" PostTypeId="1" AcceptedAnswerId="3893" CreationDate="2017-08-24T12:42:23.943" Score="9" ViewCount="878" Body="&lt;p&gt;As I know, the current state of the art methods for training deep learning networks are variants of gradient descent or stochastic gradient descent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the best known gradient-free training methods for deep learning (mostly in visual tasks context)?&lt;/p&gt;&#xA;" OwnerUserId="9233" LastEditorUserId="2444" LastEditDate="2020-03-09T14:58:04.413" LastActivityDate="2020-03-09T14:58:04.413" Title="What are the best known gradient-free training methods for deep learning?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3888" PostTypeId="1" CreationDate="2017-08-24T18:00:39.937" Score="5" ViewCount="76" Body="&lt;p&gt;I'd like a general explanation of that in AIs that were to mimic judges, prosecutors or lawyers, on very general terms they would act on this way for each case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A judge AI would give a verdict, having the following input:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All sources of law that have some relationship with the act being judged, knowing the relative importance of each of that sources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the facts presented to it, being theorically able to distinguish if some fact were false if there are enough evidence for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prosecutors AI would act into searching the facts that would most help for accusing, matching them with known sources of law so they can make the best accusation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lawyers AI would act exactly as prosecutors but for defense of the one being accused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously AIs are way too far nowadays from something like this, but I find strange that it doesn't even look that this has been done as a proof of concept, the most near thing I find by searching on the Internet is &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence_and_law&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;. I'm not sure of the reach of everything that it's explained there but it looks more of assisting judges, prosecutors and lawyers rather than making what they do. I find this a bit strange, although I'm very far from being an expert on AIs, I think that for very simple cases an AI could even do those things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the answer about possible development, I'd like to know about which would be the facts that would make developing an AI like the one I'm mentioning to be possible or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PD: I don't find any suitable tag, I ask an admin to put a better one if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requested edit: My question is about the general state of the art of this field (of AIs being able to somehow mimic a judge, a prosecutor or a lawyer), and how pausible is a development in this way and how it would develop (if this is with current knowledge impossible to even try until a far future, or if we are at least near for stablishing AIs understanding law, which would be the logical way for a type of AIs like them evolving (understing laws, judging, being able to see videos..). I know it's broad so I'm happy with a general explanation.&lt;/p&gt;&#xA;" OwnerUserId="9244" LastEditorUserId="9244" LastEditDate="2017-08-24T18:48:49.840" LastActivityDate="2017-08-24T19:04:18.733" Title="State of the art and possible development of judging AIs in laws field" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3889" PostTypeId="1" AcceptedAnswerId="21436" CreationDate="2017-08-24T18:35:04.387" Score="4" ViewCount="128" Body="&lt;p&gt;I read some light material earlier about the possibility of building a hierarchy of agents, where the agents at the leaves solve primitive tasks while higher-level agents are optimized for orchestrating direct-descendant agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to my understanding, each agent would have different objective functions and possible moves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I design such a hierarchy of agents each of which with different goals? Are there any examples?&lt;/p&gt;&#xA;" OwnerUserId="9210" LastEditorUserId="2444" LastEditDate="2020-05-23T20:13:41.213" LastActivityDate="2021-02-17T22:34:01.487" Title="How can I design a hierarchy of agents each of which with different goals?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;multi-agent-systems&gt;&lt;hierarchical-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3890" PostTypeId="2" ParentId="3888" CreationDate="2017-08-24T19:04:18.733" Score="1" Body="&lt;p&gt;There has definitely been work relating to judicial sentencing:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In a new study from the National Bureau of Economic Research, economists and computer scientists trained an algorithm to predict whether defendants were a flight risk from their rap sheet and court records using data from hundreds of thousands of cases in New York City. When tested on over a hundred thousand more cases that it hadn’t seen before, the algorithm proved better at predicting what defendants will do after release than judges.&lt;br&gt;&lt;sub&gt;Source: M.I.T. Technology Review: &lt;a href=&quot;https://www.technologyreview.com/s/603763/how-to-upgrade-judges-with-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Upgrade Judges with Machine Learning&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In terms of actually trying cases to deliver a verdict of guilty or not guilty, I think that would be difficult because, although Machine Learning has been effective at outperforming humans in problems where parameters can be clearly defined, we haven't figured out how to tackle the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbol grounding problem&lt;/a&gt;. This means that making decisions based on language, which is less precise than mathematics, would be highly problematic. Guilty/Not Guilty is a basic True/False condition, but the rest of the language utilized in law cannot be so precisely translated.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-08-24T19:04:18.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3891" PostTypeId="2" ParentId="3881" CreationDate="2017-08-24T19:26:18.933" Score="4" Body="&lt;p&gt;Mostly because traditional &quot;knowledge based systems&quot; are based purely on deductive logic and that's just the way deduction works.  It only deals with what consequences &lt;em&gt;must&lt;/em&gt; follow from the premises.  Traditionally these systems didn't deal with probabilistic knowledge or other less strict forms of reasoning, like abduction.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it is possible to build a knowledge based system that includes probabilistic reasoning, and other reasoning systems (like abduction). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll add some more on that point when I have a little more time.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-08-24T19:26:18.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3892" PostTypeId="2" ParentId="3825" CreationDate="2017-08-25T07:58:06.977" Score="0" Body="&lt;p&gt;If you really want to understand what goes on in a CNN and visualize it, then you can give a read to this:&#xA;&lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/convolutional-networks/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This course needs some per-requisites like having a certain level of idea of forward and backward propagation, Knowing how the network learns on itself. &#xA;CNN is just a way of moving from Fully connected layers (every input connects to every output) to Partially connected layers. &lt;/p&gt;&#xA;" OwnerUserId="9198" LastEditorUserId="1581" LastEditDate="2017-08-25T18:24:59.220" LastActivityDate="2017-08-25T18:24:59.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3893" PostTypeId="2" ParentId="3885" CreationDate="2017-08-25T20:12:12.767" Score="6" Body="&lt;p&gt;There are several different algorithms that can be used for gradient free neural network training. Some of these algorithms include particle swarm optimization, genetic algorithms, simulated annealing, and several others. Almost any optimization algorithm can be used to train a neural network. Here is an overview of some of the algorithms I listed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Particle Swarm optimization - I would say that this is one of the better optimization algorithms to train neural networks other than back propagation. I am currently using it and have achieved quite good results.&lt;/li&gt;&#xA;&lt;li&gt;Genetic Algorithms - I have tried to use genetic algorithms to train neural networks in the past and I was not able to get it to work. However, I was using deep neural networks with almost a million parameters and the performance was not that good.&lt;/li&gt;&#xA;&lt;li&gt;Simulated annealing - simulated annealing is based off of metals cooling. I have seen simulated annealing work fairly well but maybe not as well as particle swarm optimization.&lt;/li&gt;&#xA;&lt;li&gt;Derivatives of genetic algorithms - derivatives of genetic algorithms such as NEAT have been shown to work pretty well. I have not personally used them extensively but some of the things that people have used them for are pretty cool.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="4631" LastEditDate="2017-08-25T20:17:42.777" LastActivityDate="2017-08-25T20:17:42.777" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3894" PostTypeId="1" AcceptedAnswerId="3896" CreationDate="2017-08-26T03:58:57.590" Score="4" ViewCount="183" Body="&lt;p&gt;What we are doing in the image processing training.  We are storing some form of data which is going to act as the knowledge or experience of the system. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In which form can the system store it's training data?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, with the hand written recognition, we can represent the digits as combinations of curves and straight lines. For every round of training the recognition system stores data.  Is the data typically stored in a flat file (such as txt) or a database?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen in &lt;a href=&quot;https://en.wikipedia.org/wiki/Tesseract_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesseract OCR&lt;/a&gt; that there is a text file that stores the x0,y0,x1,y1. They are the pixel points that represents the square on the training image that has the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a efficient form of knowledge for Machine Learning, and would appreciate advice, context, or an explanation of the merits or downsides of different approaches. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I need a form of knowledge that stored in a system. human brain evaluate '7' as 'horizontal line and vertical left crossed line start from right of the horizontal line'. like that machine must have some conceptual data to represent their knowledge.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8424" LastEditorUserId="8424" LastEditDate="2017-08-29T05:06:47.417" LastActivityDate="2017-09-02T05:00:53.013" Title="As a starter: what is the form of training data for image processing" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;training&gt;&lt;knowledge-representation&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3896" PostTypeId="2" ParentId="3894" CreationDate="2017-08-26T07:27:55.930" Score="3" Body="&lt;p&gt;Your question depends heavily on the method you are using for machine learning. It sounds like you want to extract certain features like &quot;curves and straight lines&quot; from your images and use them as training data. This step of extraction is usually not considered part of the training process but part of pre-processing. During pre-precessing you read your image in, extract certain features or perform some transformation and store the new data as your actual training samples or use them for training right away without intermediate storage. Usually storing this information is a good idea if you want to use the image for training more than once and you want to skip the pre-processing step in future training cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Comparing File Storage and Database Storage&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How you store your processed data for training is relatively independent from the application of machine learning and the general principles for data storage apply. Storing the data in a flat file is usually very convenient as disk storage is readily available and the APIs for storing and reading files are part of your programming language and OS. A database adds additional complexity to your architecture but has of course it's benefits, especially if you want to make the training data available to other instances or different learning engines. If you are working with a &lt;strong&gt;really&lt;/strong&gt; huge amount of data, a well structured database can help you organize your data better and provides helpful functions for handling this data efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that the actual training of your AI takes place initially and once completed, you can roll out your AI without all the training data. So you only need to handle training data for a limited time in your application lifecycle. The speed of training is usually not very important, because it happens once and not during the daily use of the AI. Therefore the speed of file access during training can be neglected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, for most applications, the simpler implementation using just flat files is good enough. If you want to store each sample in an individual file or pack them together on bigger batches and use some meta information to identify the individual samples in the file is more a matter of taste than real technical relevance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Storing data after training&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging from your question, you sound like you understand machine learning in general. Just to clarify one potential misunderstanding for beginners - you don't have to store any of the &quot;learned&quot; information as data after training, as each training step just adapts weights and biases in the neural network. The actual training data can be thrown away after successful training, if you don't need it for future training cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Information / References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To finish my answer I want to recommend a current course from &lt;a href=&quot;https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford University about CNNs and visual recognition&lt;/a&gt; (Dated Spring 2017). It is a great source of information for implementing neural networks for image recognition.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-26T07:27:55.930" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3897" PostTypeId="1" AcceptedAnswerId="3918" CreationDate="2017-08-26T11:04:51.500" Score="5" ViewCount="298" Body="&lt;p&gt;I'm quite new to image processing and AI. But I have the expertise to create a network that can be used in object detection and recognition. Most of the time I've used ANN or Naive Bayes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to develop a method of action recognition, something like identifying whether one is jogging, running or walking by applying ANN. However, I really don't have idea how the sequence of frames can be classified. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In static image, segmentation and feature extraction is easy. But in regard to a moving image, I'm unsure of the approach.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="9211" LastEditorUserId="1671" LastEditDate="2017-08-27T20:21:14.153" LastActivityDate="2017-12-26T23:37:26.357" Title="How can I use Neural Network in motion identification" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3898" PostTypeId="1" AcceptedAnswerId="3901" CreationDate="2017-08-26T15:32:01.580" Score="1" ViewCount="81" Body="&lt;p&gt;I was trying to code a single layer perceptron to understand binary AND:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 1 1&lt;br&gt;&#xA;0 1 0&lt;br&gt;&#xA;1 0 0&lt;br&gt;&#xA;0 0 0  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made up this code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;int main()&#xA;{&#xA;int input1, input2;&#xA;float weight1 = 0.3, weight2 = 0.4;&#xA;int output;&#xA;int training1, training2, expectedoutput;&#xA;int i;&#xA;int j=1;&#xA;&#xA;//TRAINING&#xA;for(i=0; i&amp;lt;10000;i++)&#xA;{   &#xA;&#xA;&#xA;    if(j=1)&#xA;    {&#xA;        training1 = 0;&#xA;        training2 = 1;&#xA;        expectedoutput = 0;&#xA;    }&#xA;    if(j=2)&#xA;    {&#xA;        training1 = 1;&#xA;        training2 = 0;&#xA;        expectedoutput = 0;&#xA;    }&#xA;    if(j=3)&#xA;    {&#xA;        training1 = 0;&#xA;        training2 = 0;&#xA;        expectedoutput = 0;&#xA;    }&#xA;    if(j=4)&#xA;    {&#xA;        training1 = 1;&#xA;        training2 = 1;&#xA;        expectedoutput = 1;&#xA;        j=1;&#xA;    }&#xA;    output = weight1*training1 + weight2*training2 + 2;&#xA;&#xA;    if(output != expectedoutput )&#xA;    {&#xA;        weight1 = weight1 + 0.156 * training1 * (expectedoutput - output);&#xA;        weight2 = weight2 + 0.156 * training2 * (expectedoutput - output);&#xA;    }&#xA;    j++;&#xA;}&#xA;&#xA;printf(&quot;training done\n&quot;);&#xA;printf(&quot;weight1 = %f&quot; &quot;weight2 = %f\n&quot;,weight1,weight2);&#xA;&#xA;//TESTING THE PERCEPTRON&#xA;for(i=0; i&amp;lt;5 ; i++)&#xA;{&#xA;scanf (&quot;%d%d&quot;, &amp;amp;input1, &amp;amp;input2 );&#xA;output = weight1*input1 + weight2*input2;&#xA;printf(&quot;\n%d\n&quot;, output);&#xA;}&#xA;return 1;    &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;its supposed to input the 4 cases repeatedly and with a learning rate of 0.156 (which i set randomly) and i used the threshold as a weight of 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However after the training the perceptron still doesnt give the expected output. Is my understanding of perceptron rule wrong? Please help thank you!&lt;/p&gt;&#xA;" OwnerUserId="9272" LastActivityDate="2017-08-27T07:11:44.717" Title="debugging perceptron for digital AND circuit" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3899" PostTypeId="1" AcceptedAnswerId="5843" CreationDate="2017-08-26T18:13:41.780" Score="5" ViewCount="1244" Body="&lt;p&gt;I want to write an algorithm which indicates to a robot the first point in time when it is reasonably safe to cross a road.  Assume that the robot's goal is to travel to a location that requires a road crossing and that the robot is ready to cross.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple algorithms and decision making will probably not suffice.  What features and capabilities must the algorithm have to provide crossing safety?  What existing AI methods might be useful to consider for this endeavor?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first iteration, we can assume the traffic pattern is normal in that no vehicles are driving over the curb and there are no high speed chases or other safety related abnormalities.&lt;/p&gt;&#xA;" OwnerUserId="9275" LastEditorUserId="4302" LastEditDate="2018-05-30T16:19:53.237" LastActivityDate="2018-08-28T21:00:53.083" Title="Approaches to an algorithm for crossing a road" Tags="&lt;algorithm&gt;&lt;learning-algorithms&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3900" PostTypeId="2" ParentId="3897" CreationDate="2017-08-26T21:30:41.373" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM's&lt;/a&gt; provide a simple way to process sequential data (assuming all video sequences have the same number of frames and resolution).  &lt;/p&gt;&#xA;" OwnerUserId="9279" LastEditorUserId="1671" LastEditDate="2017-08-27T20:18:36.540" LastActivityDate="2017-08-27T20:18:36.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3901" PostTypeId="2" ParentId="3898" CreationDate="2017-08-27T06:41:06.693" Score="0" Body="&lt;p&gt;You have several flaws in your code that will lead to unexpected behavior. I identified the following flaws you need to address. Once you have fixed them, you should output your updated weights during each step of training to see what the learning algorithm actually does and if it is going in the right direction. I will not address pure style aspects (like using a simpler case-statement instead of a series of ifs and stuff like that) and focus on the real errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Skipping training case 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you are in training case 4, you set j to 1, expecting to use training case 1 next. But later in your code you increase j by one (j++) and go directly to training case 2, skipping the first one. This means you only run through training case 1 during your first pass of the loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Assigning j a value instead of comparing it&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if(j=1)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The if statement will always be true, because you do not compare j to 1 but you set j to 1. You basically assign a value and test if the value is true. Correct would be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if(j==1)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ignoring bias during test&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the test steps after training you forget to add the bias that you used during training:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;output = weight1*input1 + weight2*input2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;should actually be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;output = weight1*input1 + weight2*input2 + 2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Otherwise your perceptron behaves differently during training and testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Perceptron output must be 1 or 0&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those were all implementation issues. This point actually seems to come from a misunderstanding of perceptrons. A real perceptron can either fire or not, meaning it outputs either 1 or 0, nothing in between. You calculate your output the following way and use the output to compare to the expected output:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;output = weight1*training1 + weight2*training2 + 2;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;if(output != expectedoutput )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your output here is a float value. Only in edge cases it will result in 1 or 0. What you actually want to do for a real perceptron is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if (weight1*training1 + weight2*training2 + 2 &gt;= 0) {&#xA;   output = 1&#xA;  } else {&#xA;   output = 0&#xA;  }&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;if(output != expectedoutput)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After you fixed those errors and studied the output of your learning algorithms, you should be able to get the perceptron to work. If you have any questions, please leave a comment and I will try to help.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have a C compiler here, so I quickly implemented your solution with my suggested fixes in Python 3.6 and the results are very close to the correct solution. The final weights obviously have to be -1 and -1 for an AND Gate with bias 2. The learning process gets very close but never reaches -1 and -1 exactly, which leads to the wrong output in the end. This is a very good showcase to illustrate why you prefer a &lt;strong&gt;sigmoid neuron&lt;/strong&gt; instead of a &lt;strong&gt;perceptron&lt;/strong&gt; in modern neural networks. &lt;a href=&quot;https://stats.stackexchange.com/questions/167478/what-is-the-difference-between-sigmoid-neurons-and-stochastic-binary-neurons&quot;&gt;Here&lt;/a&gt; is a relevant question from cross-validated concerning the difference of those two types of neurons.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="9161" LastEditDate="2017-08-27T07:11:44.717" LastActivityDate="2017-08-27T07:11:44.717" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3902" PostTypeId="2" ParentId="2037" CreationDate="2017-08-27T07:33:56.257" Score="2" Body="&lt;p&gt;To add to @BlindKungFuMaster's answer, the object detection problem in NN context isn't necessarily solved by using a network that produces output with the same size as the input. Actually this approach - assigning each pixel the object probability - better fits the &lt;a href=&quot;https://stackoverflow.com/questions/33947823/what-is-semantic-segmentation-compared-to-segmentation-and-scene-labeling&quot;&gt;Instance/ Semantic Segmentation&lt;/a&gt; problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To briefly review the common state of the art object detectors, there are mainly two families:   &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The two-stage approach (one stage find the bounding boxes by a variant of sliding window, another stage finds the object probabilities, some of the layers might be common to both stages) - from &lt;a href=&quot;https://arxiv.org/pdf/1311.2524.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCNN&lt;/a&gt; (propose boxes by the Selective Search algorithm, check if they contain an object by running CNN - AlexNet in this case - on each box, if a box contains an object run linear regression to tighten the box), through &lt;a href=&quot;https://arxiv.org/pdf/1504.08083.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast-RCNN&lt;/a&gt; (Share the computations for all the boxes, use ROI pooling to select the region that corresponds to each box), &lt;a href=&quot;https://arxiv.org/pdf/1506.01497.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faster-RCNN&lt;/a&gt; (Speed up the region proposal stage by using a fully convolutional Region Proposal Network instead of the heavy Selective Search), &lt;a href=&quot;https://arxiv.org/pdf/1605.06409.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;R-FCN&lt;/a&gt; (Share all computations for both objects and box proposals), and &lt;a href=&quot;https://arxiv.org/pdf/1703.06870.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mask-RCNN&lt;/a&gt; that extends the faster-RCNN solution to semantic segmentation, but also solves better the detection problem by fitting the box coordinates+object features correspondence by using bilinear interpolation, replacing ROI pooling with ROI align.&lt;/li&gt;&#xA;&lt;li&gt;The single-stage approach, such as &lt;a href=&quot;https://arxiv.org/pdf/1512.02325.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1701.06659.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSSD&lt;/a&gt;, &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1612.08242&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO2&lt;/a&gt; and lastly &lt;a href=&quot;https://arxiv.org/pdf/1708.02002.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RetinaNet with the focal loss&lt;/a&gt;, that in a single shot produce both boxes and objects scores, and work much faster that the two-stage approaches.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A great summary for object recognition, detection and segmentation is published &lt;a href=&quot;https://medium.com/@nikasa1889/the-modern-history-of-object-recognition-infographic-aea18517c318&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; - highly recommended.&lt;/p&gt;&#xA;" OwnerUserId="9233" LastEditorUserId="9233" LastEditDate="2017-08-28T10:41:30.590" LastActivityDate="2017-08-28T10:41:30.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3903" PostTypeId="1" AcceptedAnswerId="3907" CreationDate="2017-08-27T15:23:54.393" Score="12" ViewCount="1729" Body="&lt;p&gt;Currently, within the AI development field, the main focus seems to be on pattern recognition and machine learning. Learning is about adjusting internal variables based on a feedback loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maslow's hierarchy of needs&lt;/a&gt; is a theory in psychology proposed by Abraham Maslow that claims that individuals' most basic needs must be met before they become motivated to achieve higher-level needs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could possibly motivate a machine to act? Should a machine have some sort of DNA-like structure that would describe its hierarchy of needs (similar to Maslow's theory)? What could be the fundamental needs of a machine?&lt;/p&gt;&#xA;" OwnerUserId="9288" LastEditorUserId="2444" LastEditDate="2019-10-07T19:18:46.397" LastActivityDate="2021-05-09T23:49:00.710" Title="What would motivate a machine?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;rewards&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3905" PostTypeId="1" AcceptedAnswerId="3909" CreationDate="2017-08-27T17:16:52.577" Score="3" ViewCount="954" Body="&lt;p&gt;Hypothetically, the &lt;strong&gt;symbol&lt;/strong&gt; (Triangle) is sticked to an item and i need to &lt;strong&gt;find and recognize that symbol and try to calculate the orientation of the item&lt;/strong&gt; it is sticked into. In degrees. How would you guys suggest i approach this problem? Do i still need NN for this? Thank you :) I just need to hear other people's thoughts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vDcze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vDcze.png&quot; alt=&quot;sample&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9290" LastActivityDate="2017-08-28T21:28:39.497" Title="Image Recognition and Orientation Detection" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3906" PostTypeId="2" ParentId="3903" CreationDate="2017-08-27T17:51:34.497" Score="3" Body="&lt;p&gt;I asked &lt;a href=&quot;https://www.ualberta.ca/science/about-us/contact-us/faculty-directory/rich-sutton&quot; rel=&quot;nofollow noreferrer&quot;&gt;professor Richard Sutton&lt;/a&gt; a similar question, in the first lecture of the reinforcement learning course. It seems that there are different ways to motivate the machine. In fact, machine motivation seems to me like a dedicated field of research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, machines are motivated by what we call an &lt;em&gt;objective function&lt;/em&gt; or a &lt;em&gt;cost function&lt;/em&gt; or a &lt;em&gt;loss function&lt;/em&gt;. These are different names for the same concept. Sometimes, they are denoted by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the machine is then to solve either a minimization problem, &lt;span class=&quot;math-container&quot;&gt;$\min_a L(a)$&lt;/span&gt;, or a maximization problem, &lt;span class=&quot;math-container&quot;&gt;$\max_a L(a)$&lt;/span&gt;, depending on the definition of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9210" LastEditorUserId="2444" LastEditDate="2019-10-07T19:22:56.667" LastActivityDate="2019-10-07T19:22:56.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3907" PostTypeId="2" ParentId="3903" CreationDate="2017-08-27T17:53:06.690" Score="5" Body="&lt;p&gt;The current method to implement motivation is some kind of artificial reward. &lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deepmind's DQN&lt;/a&gt; for example is driven by the score of the game. The higher the score, the better. The AI learns to adjust its actions to get the most points and therefore the most reward. This is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learing&lt;/a&gt;. The reward &lt;strong&gt;motivates&lt;/strong&gt; the AI to adapt its actions, so to speak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a more technical term, the AI wants to maximize utility, which depends on the implemented &lt;a href=&quot;https://stackoverflow.com/questions/14360893/intelligent-agents-utility-function&quot;&gt;utility function&lt;/a&gt;. In the case of DQN, this would be maximizing the score in the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The human brain functions in a similar fashion, although a little more complex and often not as straight forward. We as humans usually try to adjust our actions to produce a high output of &lt;strong&gt;dopamine&lt;/strong&gt; and &lt;strong&gt;serotonin&lt;/strong&gt;. This is in a way similar to the reward used to control AIs during reinforcement learning. The human brain learns which actions produce the most amount of those substances and finds strategies to maximize the output. This is, of course, a simplification of this complex process, but you get the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you talk about motivation, please don't mix it up with &lt;strong&gt;consciousness&lt;/strong&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Qualia&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;qualia&lt;/strong&gt;&lt;/a&gt;. Those are not required for motivation at all. If you want to discuss consciousness and qualia in AI, that's a totally different ball game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A child isn't curious for the sake of curiosity. It gets positive reinforcement when exploring because the utility function of the child's brain rewards exploration by releasing rewarding neurotransmitters. So the mechanism is the same. Applying this to AI means defining a utility function that rewards new experiences. There is no inner drive without some kind of reinforcing reward.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="2444" LastEditDate="2019-10-08T02:59:34.470" LastActivityDate="2019-10-08T02:59:34.470" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3908" PostTypeId="1" CreationDate="2017-08-27T18:13:28.803" Score="4" ViewCount="147" Body="&lt;p&gt;Nick Bostrom talks in his book &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies&quot; rel=&quot;noreferrer&quot;&gt;Superintelligence&lt;/a&gt; about the many dangers of AI. He considers it necessary that strong security mechanisms are put in place to ensure that a machine, once it gains general intelligence far beyond human capabilities, does not destroy humanity (most likely by accident). He describes this as a very delicate process that most likely will go wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering that new technologies often neglect the necessary precautions and  that this is highly relevant to national security, I wonder if there are already government agencies overseeing big technology companies like Deepmind. We are currently far away from an intelligence explosion or a technological singularity, but I would assume that governments want to have a foot in the door as soon as they realize and understand the dangers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, what government agencies currently investigate and maybe even control AI development? The answer can be general or for a specific country if there is a big difference between countries.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2018-01-08T15:20:39.947" Title="Which government agencies oversee development of new AI?" Tags="&lt;control-problem&gt;&lt;singularity&gt;&lt;security&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3909" PostTypeId="2" ParentId="3905" CreationDate="2017-08-27T18:33:36.273" Score="2" Body="&lt;p&gt;This is a quite trivial problem and using a NN would be overkill for this. The symbol you used (triangle) is problematic, because it looks exactly the same in 3 different positions (rotated 0°, 120° and 240°).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest implementation would be using 2 distinguishable points with a fixed distance. To distinguish them you could color code them or give them different sizes. Those 2 points should be easy to identify in your picture using a pattern matching algorithm. Once you have the 2 points you calculate the vector between them and can then calculate the orientation of the vector, which basically tells you the orientation of the whole object your symbol is put on.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit - Answering the first comment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is the same, no matter what symbol you use. In case of the &quot;L&quot;, your 2 points would be the end of both lines. You can easily find the symbol, calculate the vector between both points and use it to calculate the orientation using this vector. This approach can be used for every symbol that doesn't have radial symmetry. You can always define 2 points in this symbol and apply the method described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why a NN is overkill is the simplicity of the problem. NNs are mostly used for problems that are easy to solve for humans but very hard to implement with a simple algorithm. Tasks like classifying images, identifying hand written or distorted letters and so on. But your problem is easily solvable with simple algorithms, therefore the complexity of a NN is not necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't mean that your task cannot be implemented with an NN. Actually it might be quite an interesting project to learn more about NNs. I would say your best bet is implementing it as a CNN, since this type of NN is good at detecting features in images like the direction of your symbol. I can recommend &lt;a href=&quot;https://www.youtube.com/watch?v=vT1JzLTH4G4&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;nofollow noreferrer&quot;&gt;this course from Stanford University&lt;/a&gt;, if you want to learn more about CNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a real world application, there is no need to take the complex route using NNs. My suggested algorithm can be implemented with a few dozen lines of code and should be very robust.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="9161" LastEditDate="2017-08-28T21:28:39.497" LastActivityDate="2017-08-28T21:28:39.497" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3910" PostTypeId="1" CreationDate="2017-08-28T02:25:21.380" Score="3" ViewCount="104" Body="&lt;p&gt;In this &lt;a href=&quot;https://people.cs.umass.edu/~domke/courses/sml2011/08autodiff_nnets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;note&lt;/a&gt; Justin Domke says that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In practice, neural networks seem to usually find a reasonable solution when the number of layers is not too large, but find poor solutions when using more than, say, 2 hidden layers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But in &lt;a href=&quot;https://qr.ae/pNrt0p&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bengio's remark&lt;/a&gt;, he says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Very simple. Just keep adding layers until the test error does not improve anymore.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There seems to be a conflict. Can anyone explain why they suggest differently? Or am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="9299" LastEditorUserId="2444" LastEditDate="2020-04-12T18:49:04.350" LastActivityDate="2020-04-12T19:48:58.233" Title="Are these statements about the performance of neural networks as a function of the number of hidden layers contradictory?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;hidden-layers&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3911" PostTypeId="2" ParentId="3910" CreationDate="2017-08-28T02:37:00.607" Score="2" Body="&lt;p&gt;There are &lt;strong&gt;many&lt;/strong&gt; problems requiring more than two hidden layers. Randomly select a recent Google journal paper on deep learning, you'll see their network could have something like 5 (or more) hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Justin Domke wrote his notes for students, so he probably tried to make his points as simple as possible. For a &quot;typical&quot; machine learning problem that students would most likely work on, two hidden layers should be sufficient. But that doesn't add up for a real practical problem. &quot;Deep&quot; learning usually mean more than two hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of hidden layers is network design that nobody knows for sure. Yoshua Bengio's suggestion is common and simple. It's not a mathmatic proof, but simply a guideline if you don't know what to  do. You just repeat and repeat, until you see the test error no longer improve.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-08-28T02:42:22.073" LastActivityDate="2017-08-28T02:42:22.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3912" PostTypeId="2" ParentId="3910" CreationDate="2017-08-28T03:42:53.030" Score="-1" Body="&lt;p&gt;In fact they are telling the same thing: Plot the x-axis as the number of hidden layers, and the y-axis as the performance (e.g. classification accuracy), then this curve will have an upside-down U shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Justin's note is clearly saying the same thing as what I wrote above, with added note that the maximum of the curve will happen when x = 2, and Bengio's note is saying the same thing without telling you where the peak could be.&lt;/p&gt;&#xA;" OwnerUserId="7622" LastActivityDate="2017-08-28T03:42:53.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3915" PostTypeId="2" ParentId="3451" CreationDate="2017-08-28T18:14:25.423" Score="0" Body="&lt;p&gt;You may infer some mental problems using text processing. Any text written by an individual contains a lot of clues about their mental state, probably including any health problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I believe pursuing this would prove to be difficult due to lack of ground truth. Not many people makes it public that they have mental problems and without a reliable database it is quite difficult to make research on the field.&lt;/p&gt;&#xA;" OwnerUserId="210" LastActivityDate="2017-08-28T18:14:25.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3918" PostTypeId="2" ParentId="3897" CreationDate="2017-08-28T22:06:43.670" Score="0" Body="&lt;p&gt;Depending on the complexity of the problem you could go with the full blown approach of LSTMs (like dant suggested) or you could try to address the challenge using pre-processing and stick with your already established CNN architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use pre-processing, you basically calculate delta images between frames and use them like &quot;normal&quot; frames in your CNN. The DQN of Deepmind used this approach successfully to understand motion when playing ATARI games. Those images were rather simple and the amount of motion limited especially compared with videos of real life situations, so this simple approach may not be sufficient for your problem, but I consider it worth trying.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-28T22:06:43.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3919" PostTypeId="2" ParentId="2330" CreationDate="2017-08-29T00:16:27.760" Score="3" Body="&lt;p&gt;While interesting, this is all rendered somewhat moot if you think about what will happen once we understand how the brain works. After all, once we understood flight, we didn't start making birds. The same goes for AI. Here are just a few ways in which human brains and digital brains can't be compared. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The digital brain won't have to worry about food and drink. They will also be more reliable (or less redundant) as electronics is way more reliable than neurons (a guess). Digital brains will also be able to share learning and information. Once one Model 3X digital brain has learned something, the others need merely to have the bits uploaded. Sure, it will be more complicated than that but, remember, we will know how it all works so merging the experiences of one digital being with another should be doable. If we want our digital brain to have symbolic algebra ability, we will have to teach it some things but we can also hard-wire it to Mathematica or the like. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, it will be like apples and oranges.&lt;/p&gt;&#xA;" OwnerUserId="9318" LastActivityDate="2017-08-29T00:16:27.760" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3920" PostTypeId="1" AcceptedAnswerId="3928" CreationDate="2017-08-29T08:57:30.060" Score="8" ViewCount="818" Body="&lt;p&gt;As I understand it from this &lt;a href=&quot;http://videolectures.net/deeplearning2015_vincent_machine_learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;video lecture&lt;/a&gt;, there are three types of deep learning:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Supervised&lt;/li&gt;&#xA;&lt;li&gt;Unsupervised&lt;/li&gt;&#xA;&lt;li&gt;Reinforcement&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All these can serve to train a neural network either &lt;em&gt;only prior&lt;/em&gt; to its deployment or &lt;em&gt;during&lt;/em&gt; its operating. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the latter case, it is referred to as &lt;em&gt;continuous&lt;/em&gt; learning &lt;a href=&quot;http://deepmind.com/blog/enabling-continual-learning-in-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://medium.com/@Synced/deep-learning-in-real-time-inference-acceleration-and-continuous-training-17dac9438b0b&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and as &lt;em&gt;dynamic&lt;/em&gt; learning &lt;a href=&quot;http://conferences.oreilly.com/artificial-intelligence/ai-ny/public/schedule/detail/59511&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://www.datasciencecentral.com/profiles/blogs/static-dynamical-machine-learning-what-is-the-difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which term should I use to refer to a machine learning algorithm that keeps on learning (even after deployment)? If it's &quot;continuous&quot;, is there an opposing term (such as &quot;static&quot; for &quot;dynamic&quot;) for those systems that stop learning before being deployed? &lt;/p&gt;&#xA;" OwnerUserId="9319" LastEditorUserId="23503" LastEditDate="2020-11-09T11:14:06.680" LastActivityDate="2020-11-09T11:43:24.067" Title="What do you call a machine learning system that keeps on learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;incremental-learning&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3922" PostTypeId="1" CreationDate="2017-08-29T13:29:18.700" Score="2" ViewCount="57" Body="&lt;p&gt;Currently big tech companies like Microsoft, Google, and Amazon (to name a few) offer cognitive services on their cloud platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With these services it is possible to identify faces, objects, texts, sounds, etc.&#xA;Do you know how these services work internally? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only info I could find was based on the API level. I assume the services use some neural network, which is trained by amounts of data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience the Google services are more accurate then the Azure services. Perhaps the Google services are better and longer trained?&lt;/p&gt;&#xA;" OwnerUserId="9057" LastActivityDate="2017-08-29T13:29:18.700" Title="How do cognitive services work?" Tags="&lt;computer-vision&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3923" PostTypeId="1" CreationDate="2017-08-29T14:21:48.227" Score="2" ViewCount="158" Body="&lt;p&gt;In working with a social services agency that provides a continuum of programs across the behavioral health and child welfare spectrum the need to adequately manage individual worker and total program caseloads has become increasingly difficult due to burgeoning demand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each program has already developed a profile for ranking the complexity of a client based a select number of factors (i.e. diagnosis, previous encounters, results of assessment instruments, etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we are trying to develop now is an approach to act as a load balancer for new cases to ensure that individual workers' caseloads are balanced with the complexity of cases. We can do this manually to some degree but if there was a way to automate and take the human bias out of it, that is the goal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there are any algorithms or adaptable approaches that have been used to do such a task (even in other sectors)? &lt;/p&gt;&#xA;" OwnerUserId="9324" LastActivityDate="2018-02-13T22:36:35.810" Title="Are there any load balancing approaches for employee cases based on complexity score?" Tags="&lt;algorithm&gt;&lt;problem-solving&gt;&lt;efficiency&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3926" PostTypeId="2" ParentId="3920" CreationDate="2017-08-29T20:30:37.060" Score="1" Body="&lt;p&gt;In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the same type of &lt;strong&gt;learning during operation&lt;/strong&gt;. And this gives as cues to dig deeper into the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to learn during operation is simple reinforcement learning. DQN of Deepmind works exactly that way. It tries out different moves and learns what works by receiving feedback through the reward function. There is not necessarily a separate learning and working phase. The AI just gets better over time. This means, that the NN keeps adapting to the challenge at hand and if that challenge changes, it adopts to the new challenge over time and &quot;forgets&quot; what worked before, if this old strategy is no longer efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your first reference to Deepmind addresses the aspect of forgetting. When talking about continuous learning, it could also mean, that a system does &lt;strong&gt;not&lt;/strong&gt; forget previous skills but can utilize them later after learning something totally different. This requires more advanced architectures of AI. And this approach doesn't mean, that the NN must necessarily keep learning during operation. It could use this technique during training and stop learning once it is productive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there is a third field, recursive networks, which do not pass their information from neuron to neuron in a straight fashion but can contain loops or other types of gates, that control the flow of information. Such architectures are usually more complex than normal NNs, but resemble the working of animal brains a little more. In this field the term &lt;strong&gt;dynamic&lt;/strong&gt; is used frequently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I would say that the term you are looking for is &lt;strong&gt;continuous learning&lt;/strong&gt; - I have also read &lt;strong&gt;online learning&lt;/strong&gt; for this type of NN - when you want to say that the NN keeps learning during production. The term &lt;strong&gt;dynamic&lt;/strong&gt; is more common for recursive or other more complex types of NNs and fits there better in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude my answer, I need to jump back to the beginning. Many terms in the field of AI are not finally settled. Therefore my analysis is just a summary of the trends I have seen in different publications and lectures and must not hold true in the years to come. Hope it helps nevertheless.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-08-29T20:30:37.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3927" PostTypeId="1" CreationDate="2017-08-29T23:26:06.410" Score="1" ViewCount="45" Body="&lt;p&gt;I am considering some possibilities to improve a Variable Gain nonlinear control system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the drawbacks of the current technique is that the change of the gains is discrete and the switching effect is highly undesirable...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gain matrices are 4x16 and are calculated for some values of a bounded variable and then selected trough a lookup table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what my idea was to try and do this with an Neural Network, that would receive as input the control variable and give the gain matrix as output, allowing my control to be continuous or almost continuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this idea possible/feasible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My knowledge on neural nets is very poor, so I hope you can enlighten me somehow.&lt;/p&gt;&#xA;" OwnerUserId="9336" LastEditorUserId="1671" LastEditDate="2017-08-30T00:11:26.050" LastActivityDate="2017-08-30T00:11:26.050" Title="Neural Network to Interpolate Matrices" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3928" PostTypeId="2" ParentId="3920" CreationDate="2017-08-30T06:28:18.713" Score="2" Body="&lt;p&gt;&lt;strong&gt;Reliability of Definitions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms Supervised, Unsupervised, and Reinforcement are explained reasonably in the video lecture referenced in the question, however Pascal Vincent from the University of Montreal did not annotate the lecture, show proofs, or demonstrate a comparison using working systems for each term.  It is just one faculty member's view.  In the search for symmetry and precision in the definitions of those three terms, along with Continuous and Dynamic, ambiguity may be found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no standards body to define terms across the artificial intelligence research and development community.  Lack of standardization is a big problem in the software industry in general, but software engineering is still young compared to other engineering realms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mechanical engineering, one can design a system and call for six bolt circle of 1/2-20 stainless steel 2&quot; long bolts.  Manufacturers can order the bolts and procure the circle with ease because of standards and conventions.  Civil engineers can unambiguously tell what is and what is not a buttress in a highway bridge design.  In electrical engineering, an instrument amplifier is a class of circuits about which there is little ambiguity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In software engineering, terms like Component or Framework possess remarkable ambiguity.  In contrast, the main features of a RestFUL API have little ambiguity.  Similarly, subsets of software engineering, artificial intelligence and machine learning, have their ambiguous terms and their relatively well defined ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning Follows Deployment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term in the middle of this question is interesting: &quot;Only prior to its deployment.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is actually no learning software that learns prior to deployment.  The software must be deployed somewhere to learn.  If the domain knowledge is in the source code, then the person who wrote the program did the learning, not the software system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be one component that both learns and uses what was learned to perform.  Another arrangement involves two components, one that learns but is unable to perform and a separate component that is unable to learn but performs based on what the former component learned.  In the integrated case, the one component must be deployed to begin learning.  In the two component case, the learning component must be deployed to begin learning.  This is true for neural networks and other types of learning systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question would make more sense if it read thus: &quot;Is there [a] term for those systems that stop learning before being used to perform their function?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no single term that is defined exactly like that.  One way to describe those systems with no concurrence between learning and performing function using what is learned is Temporally Disjunct, temporally meaning in the time domain, disjunct meaning without intersection.  Calling a system a Temporally Disjunct, of course, makes no statement as to whether learning can later be resumed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Relatively Unambiguous, Common Sense Definitions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's how I would define the terms, but this is just my view, based on my research experience and academic background.  I unfortunately have less time than university faculty members to provide annotations, proofs, or comparative pseudo-code.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Supervised &amp;mdash; The learning process is controlled or assisted by an external factor other than merely the signal containing the pattern to be learned.  In common practice, the supervision is actually a set of expected values.  The expected values are often oriented in the input data alongside the feature values of the training data.  (Another way to look at this is that the independent and dependent data are contained in a single point structure.)&lt;/li&gt;&#xA;&lt;li&gt;Unsupervised &amp;mdash; Not supervised.  Neither expected values nor other data to assist in training is provided beyond the feature data.  (This is the case in naive pattern recognition algorithms.)&lt;/li&gt;&#xA;&lt;li&gt;Reinforcement &amp;mdash; The provision of a correctness signal indicates to the learning process whether the most recent behavior stemming from previous learning is desirable or undesirable.  In some systems, the degree of desirability or undesirability is also provided.  The signal may be a vector or matrix in more sophisticated designs.  Psychology minded researchers sometimes call the correctness signal a representation of reward or punishment, but such is the anthropomorphism of something more basic than the coercive aspects of parenting, animal training, economic incentivization, or law enforcement.&lt;/li&gt;&#xA;&lt;li&gt;Re-entrant &amp;mdash; The learning process is re-entrant, meaning that it can stop, the system can perform a function using what it learned, and learning can resume. Note that the temporal granularity is not implied; no standard defines whether switching between learning and performing based on what was learned occurs every nanosecond or every year.&lt;/li&gt;&#xA;&lt;li&gt;Continuous &amp;mdash; Learning does not stop, implying that learning occurs concurrently with the use of what has been learned.  (Continuity is actual in the human brain or with massively parallel computing equipment with multiple bus architectures.  It is merely simulated in a single CPU system, since even in the most integrated of learning systems, the instructions delegated to acquisition of a pattern must be distinct from the instructions that use what is learned for performing a useful function.  In practice, therefore, continuous approaches usually employ re-entrant learning.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Dynamicity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is little to add regarding the terms Static and Dynamic beyond the dictionary definition of those terms. Any number of features or organs of learning systems can be either Dynamic or Static. These are just a few example contexts in which dynamicity can be relevant within the domain of machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Values of edges between vertices&lt;/li&gt;&#xA;&lt;li&gt;Numbers of vertices in some place in the structure&lt;/li&gt;&#xA;&lt;li&gt;Overarching network parameters or signals&lt;/li&gt;&#xA;&lt;li&gt;Number of dimensions in signals along edges&lt;/li&gt;&#xA;&lt;li&gt;Ranges of each dimension&lt;/li&gt;&#xA;&lt;li&gt;Number of layers&lt;/li&gt;&#xA;&lt;li&gt;Topology of layers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for an antonym for Continuous Learning, you may wish to use Discontinuous Learning. As disappointing as this may seem, it is the antonym with less ambiguity.  Terms like Re-entrant Training Algorithms may be more technically correct but is somewhat obscure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Static and Dynamic should always remain antonyms of one another. The many kinds of dynamic features (listed above) that a neural net can have renders the relationship between dynamicity and continuity too convoluted to briefly summarize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Acknowledgement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to Douglas Daseeco, our Lab Director, for encouraging comprehensive responses to high level questions (instead of dismissing them as too wide or answering them insufficiently) and for correcting my initial explanation of reinforcement.  I had incorrectly entangled re-entrant concepts with the reinforcement and failed to address the existence of a correctness signal.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-09-26T13:22:02.867" LastActivityDate="2017-09-26T13:22:02.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3929" PostTypeId="1" CreationDate="2017-08-30T12:04:26.083" Score="2" ViewCount="165" Body="&lt;p&gt;I wrote a solution to the one shot Prisoner's dilemma:&lt;br&gt;&#xA;&amp;nbsp;&lt;br&gt;&#xA;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Introduction&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;My solution applies to a prisoner dilemma involving two people (I have neither sufficient knowledge of the prisoner's dilemma itself, nor sufficient mathematical aptitude/competence to generalise my solution to prisoner's dilemma where number of agents (n) &gt; 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the two agents involved be A and B, assume A and B are maximally selfish (they care solely about maximising their payoff). If A and B satisfy the following 3 requirements, then whenever A and B are in a prisoner's dilemma together, they would choose to cooperate.&#xA;1. A and B are perfectly rational.&#xA;2. A and B are sufficiently intelligent, such that they can both simulate each other (the simulations doesn't have to be perfect; it only needs to sufficiently resemble the real agent that it can be used to predict the choice of the real agent).&#xA;3. A and B are aware of the above 2 points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DTFJc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DTFJc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A and B have the same preference.&#xA;(A,B) = (D,C) &gt; (C,C) &gt; (D,D) &gt; (C,D).&lt;br&gt;&#xA;(B,A) = (D,C) &gt; (C,C) &gt; (D,D) &gt; (C,D).    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution relies on A and B predicting each other's behaviour. They use a simulation of the other which guarantees high fidelity predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A adopts A defect invariant strategy (always defect) i.e committing to defection, then B will simulate this, and being rational B will defect.&#xA;Vice versa.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A adopts a cooperate invariant strategy, then B will simulate this and being rational, B will defect.&lt;br&gt;&#xA;Vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To commit to a choice means to decide to adopt that choice irrespective of all other information. Committing means not taking into account any other information in deciding your choice.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming A commits to a choice. Let the choice A commits to be k.        &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then B's strategy would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nZyc2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nZyc2.png&quot; alt=&quot;Diagram to Illustrate&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diagram to Illustrate&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;h(x) is a function that takes in B's prediction of A's choice as input and outputs the rational response to that choice. As B prefers (D,C) over (C,C) and (D,D) over (C,D), then B would choose to defect whatever choice it predicts for A. Thus, adopting an invariant strategy will cause B to adopt the defect invariant strategy (insofar as B accurately predicts A). Being rational, and preferring (D,C) and (C,C) over (D,D) A would not adopt an invariant strategy.&lt;br&gt;&#xA;Vice Versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both of them were to decide to simultaneously adopt invariant strategies, they would adopt the defect invariant strategy. However, as they both prefer (D,C) and (C,C) (in that order) over (D,D) they would both strive to get a better outcome than (D,D) unless no better outcome is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that A and B's choices depends on what they predict the other would do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A predicts B will defect, A can cooperate or defect.&lt;br&gt;&#xA;If A predicts B will cooperate, A can cooperate or defect.&lt;br&gt;&#xA;Vice versa.         &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As A is not committing, A's strategy is either predict(B) (picks what A predicts B will pick) or !predict(B) (picks the opposite of what A predicts B will pick. I.e cooperate if A predicts B will defect, and defect if A predicts B will cooperate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To not commit is to decide to base your strategy on your prediction of the choice the opponent adopts. You can either choose the same choice as what you predicted, or choose the opposite of what you predicted (any other choice is adopting an invariant strategy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A adopts !predict(B).  A gains an outcome ranked 1 or ranked 4 in its preferences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A adopts predict(B)  A gains an outcome ranked 2 or 3 in its preferences.&#xA;Vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;predict(B) and predict(A)    &lt;/li&gt;&#xA;&lt;li&gt;predict(B) and !predict(A)    &lt;/li&gt;&#xA;&lt;li&gt;!predict(B) and predict(A)   &lt;/li&gt;&#xA;&lt;li&gt;!predict(B) and !predict(A).   &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now A's decision is dependent on predict(B).&lt;br&gt;&#xA;But B's decision (and thus predict(B)) is dependent on predict(A) (and thus A's decision).&#xA;A = f(predict(B) = g(B = f(predict(A) = g(A)) ) ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;f(x) is a function that deterministically returns either x or !x (according to the strategy adopted by the agent implementing it).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;g(x) is a function the stochastically return x or !x it returns x with a probability of p. Given that the agents are simulating each other, we can safely assume that p is high (sufficiently close to 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above assignment is circular and self-referential. If A and/or B tried to simulate it, it leads to a non terminating recursion of the simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A's strategy would be:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kIhJ5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kIhJ5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diagram to Illustrate&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As such, A and B cannot both decide to base their decision on the decision of the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, neither A nor B can decide to commit to an option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What they can do, is to predispose themselves to an option. To predispose themselves is to decide on their choice independent of the other agent. Assuming I received no further information, what would I do? A predisposition is not a final choice, and should not be mistaking for committing to a choice of action. An agent who predisposes themselves can change their choice based on how they predict the other agent would react to that predisposition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming A predisposes themselves. Let the predisposition made be q. The assignment becomes:    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A = f(predict(B) = g(B = f(predict(A) = g(q)) ) ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lPi6W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lPi6W.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diagram to Illustrate&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Only one of them needs to predispose themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming A predisposes themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A predisposes themselves to defection, then the only two outcomes are ranked 1 and 3 in their preferences (for A) and 3 and 4 in their preferences (for B).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upon simulating this, B being rational would choose to defect (resulting in outcome 3). (D,D) is a &lt;em&gt;Nash equilibrium&lt;/em&gt;, and as such once they reach there, being rational neither A nor B would change their strategy.&#xA;(Note that this outlaws the !predict(A) strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A predisposes themselves to cooperation, then the two possible outcomes are ranked 2 and 4 in their preferences (for A) and 1 and 2 in their preferences (for B).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upon simulating this, if B chooses to defect, then B is adopting a defect invariant strategy (which has been outlawed), and A will update and choose to defect, resulting in outcome 3. As B is rational, B will choose the outcome that leads to outcome 2, and B will decide to cooperate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If B chooses to defect, and A simulates B choosing to defect if A predisposes themselves to cooperation, then A will update and defect, resulting in (D,D). If B chooses to cooperate if A predisposes themselves to cooperation, if A updates and chooses to defect, then B would update and choose to defect resulting in D,D. Thus, once they reach (C,C) they are at a reflective equilibrium (in the sense that if one defects, then the other would also defect, thus no one of them can increase their payoff by changing strategy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Thus, B will adopt a predict(A) strategy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vice Versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because A is rational, and predisposing to cooperation dominates predisposing to defection (the outcomes outlawed are assumed not to manifest), if A predisposes themself, then A will predispose themself to cooperation.&#xA;Vice Versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus if one agent predisposes theirself, it will be to cooperation, and the resulting outcome would be (C, C) which is ranked second in their preferences.&lt;br&gt;&#xA;What if A and B both predispose themselves?&#xA;We can have:&lt;br&gt;&#xA;1. C &amp;amp; C&lt;br&gt;&#xA;2. C &amp;amp; D&lt;br&gt;&#xA;3. D &amp;amp; C&lt;br&gt;&#xA;4. D &amp;amp; D.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If C &amp;amp; C occurs, the duo will naturally cooperate resulting in (C, C). Remember that we showed above that the strategy adopted is predict(B) (defecting from (C, C) results in D, D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If C &amp;amp; D occurs, then A being rational will update on B's predisposition to defection and choose defect resulting in (D, D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If D &amp;amp; C occurs, then B being rational will update on A's predisposition to defection and choose defect resulting in (D, D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If D &amp;amp; D occurs, the duo will naturally defect resulting in (D, D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus seeing as only predisposition to cooperation yields the best result, at least one of the duo will predispose to cooperation (and the other will either predispose themselves to cooperation or not predispose at all) the resulting outcome is (C, C).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the two agents can predict each other with sufficient fidelity (explicit simulation is not necessary, only high fidelity predictions are) and are rational, and know of those two facts, then when they engage in the prisoner's dilemma, the outcome is (C, C).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, a cooperate-cooperate equilibrum can be achieved in a single instance prisoner's dilemma involving two rational agents given that they can predict each other with sufficient fidelity, and know of their rationality and intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Q.E.D&lt;/em&gt;&lt;/strong&gt;&lt;br&gt;&#xA;Thus if two super intelligences faced off against each other in the prisoner's dilemma, they would reach a cooperate-cooperate equilibrium. This solution also applies to two rational robots who have mutual access to the other's source code.&lt;br&gt;&#xA;&amp;nbsp;&lt;br&gt;&#xA;&amp;nbsp;         &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Prisoner's Dilemma With Human Players&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the above section I outlined a strategy to resolve the prisoner's dilemma for two superintelligent AIs or rational bots with mutual access to the other's source code. The strategy is also applicable to humans who know each other well enough to simulate how the other would act in a given scenario. In this section I try to devise a strategy applicable to human players.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider two perfectly rational human agents A and B. A and B are maximally selfish and care only about maximising their payoff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let:&lt;br&gt;&#xA;(D,C) = W&lt;br&gt;&#xA;(C,C) = X&lt;br&gt;&#xA;(D,D) = Y&lt;br&gt;&#xA;(C,D) = Z&lt;br&gt;&#xA;The preference is W &gt; X &gt; Y &gt; Z.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MaEfs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MaEfs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A and B have the same preference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 3 conditions necessary for the resolution of the prisoner's dilemma in the case of human players are:&lt;br&gt;&#xA;1. A and B are perfectly rational.&lt;br&gt;&#xA;2. They each know the other's preference.&lt;br&gt;&#xA;3. They are aware of the above two facts.         &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resolution of the problem in the case of superintelligent AIs relied on their ability to simulate (generate high fidelity predictions of) each other. If the above 3 conditions are met, then A and B can both predict the other with high fidelity.          &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the problem from A's point of view. B is as rational as A, and A knows B's preference. Thus, to simulate B, A merely needs to simulate themselves with B's preferences. Since A and B are perfectly rational, whatever conclusion A with B's preferences (A*) reaches is the same conclusion B reaches. Thus A* is a high fidelity prediction of B.&lt;br&gt;&#xA;Vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A engages in a prisoner's dilemma with A*. However, as A* as the same preferences as A, A is basically engaging in a prisoner's dilemma with A.&lt;br&gt;&#xA;Vice Versa.&lt;br&gt;&#xA;&amp;nbsp;&lt;br&gt;&#xA;An invariant strategy is outlawed by the same logic as in the AI section.          &amp;nbsp;&lt;br&gt;&#xA;A = f(predict(A*) = g(A* = f(predict(A) = g(A)))&lt;br&gt;&#xA;Vice Versa.          &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above assignment is self referential, and if it was run as a simulation, there would be an infinite recursion.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G3k2s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G3k2s.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diagram to Illustrate&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Thus, either A or A* needs to predispose themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the predisposition A makes is q, then the assignment becomes:       &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A = f(predict(A*) = g(A* = f(predict(A) = g(q)))    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Aa1A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Aa1A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diagram to Illustrate&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;However, as A* is A, then whatever predisposition A makes is the same predisposition A* makes. Both A and A* would predispose themselves. It is necessary for at least one of them to predispose themselves, and the strategy that has the highest probability of ensuring that at least one of them predisposes themselves is each of them individually deciding to predispose themselves. Thus, we enter a situation in which both of them predispose themselves. As A* = A, A's predisposition would be the same as A*'s predisposition.&#xA;Vice Versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have either:&#xA;1. (C,C)&#xA;2. (D,D)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A predisposes themselves to defection, then we have (D,D). (D,D) is a Nash equilibrium as A and/or A* can only perform worse by unilaterally changing strategy at (D,D). As A and A* are rational, they would predispose themselves to cooperation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If A and A* predispose themselves to cooperation, If A and/or A* tried to maximise their payoff by defecting, then the other (as they predict each other) would also defect to maximise their payoff. Defecting at (C,C) leads to (D,D). Thus, neither A nor A* would decide to defect at (C,C). (C,C) forms a reflective equilibrium.&#xA;Vice Versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As B's reasoning process closely reflects A*'s (both being perfect rationalists, and having the same preferences), the two agents would naturally converge at (C,C).&lt;br&gt;&#xA;&amp;nbsp;&lt;br&gt;&#xA;&lt;strong&gt;&lt;em&gt;Q.E.D&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I'll call this process of basing your decision in multi-agent decision problems (that involve at least two agents who are perfectly rational, know each other's preferences and are aware of those two facts from the perspective of one of those agents satisfying the above 3 criteria) by modelling the other agents (who satisfy those 3 criteria) as simulations of yourself with their preferences recursive decision theory (RDT). I think convergence on RDT is natural for any two sufficiently (they may not need to be perfect, if they are equally rational, and rational enough that they try to predict how the other agent(s) would act) rational agents who know each other's preferences and are aware of the above two facts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a single one of those criteria is missing, then RDT is not applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for example, the two agents are not equally rational, then the more rational agent would choose to defect as it strongly dominates cooperation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or they did not know the other's preferences, then they would be unable to predict the other's actions by inserting themselves in the place of the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or if they were not aware of the two facts, then they'd both reach the choice to defect, and we would be once again stuck at a (D,D) equilibrium. I'll formalise RDT after learning more about decision theory and game theory (so probably sometime this year (p &amp;lt; 0.2), or next year (p: 0.2 &amp;lt;= p &amp;lt;= 0.8) if my priorities don't change). I'm curious what RDT means for social choice theory though.      &lt;/p&gt;&#xA;" OwnerUserId="8362" LastEditorUserId="8362" LastEditDate="2017-08-31T00:07:11.313" LastActivityDate="2018-08-08T16:04:07.840" Title="Is there any flaw to this solution to the One shot prisoner's dilemma" Tags="&lt;decision-theory&gt;&lt;game-theory&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3931" PostTypeId="1" AcceptedAnswerId="4471" CreationDate="2017-08-30T18:09:28.417" Score="5" ViewCount="266" Body="&lt;p&gt;It is possible that the signal handling of a neuron is outside the engineering comprehension of the most astute of human brains, even after the relationships of inputs to outputs are statistically characterized and the mapping of genetic information to neuron structure and composition is complete.  Deep comprehension of the detailed function of neurons within their role in adaptive system behavior seems to be lacking, and no rational proof of the feasibility of comprehending neurons has yet been published.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A visual representation of neuron functionality might look like the visual representation of a genome, a blur of information, when viewed in its entirety.  The economy of the world is much more complex system, astronomically more daunting.  Even the map of the metabolic processes of earthen life (shown below) defies comprehensive display in a single view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comprehending these complex biological systems can only be done piecemeal.  Studying these systems in their entirety may take a single mind years (or even millennia), and the distortions of memory over time may prohibit placing the whole system within the finite capacity of the cerebral cortex. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet computer science has no doubt advanced.  Many features of the mammalian brain have already been simulated, and those simulations exceed the capacities and speeds of mammals.  Examples include mail sorting, flight planning, and game playing.  With such progress, the simulation of the less obviously mechanical functions of the human mind appeared to many to be within the grasp of scientific advancement.  Other AI theorists have expressed their doubts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question of whether a system of a few billion elements can be simulated without first simulating a generalization of its elements arose in the twentieth century.  The spiking neuron model is an example of research inspired by that question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is the most elemental unit of biological computation, the neuron&lt;sup&gt;1&lt;/sup&gt;, comprehensible by a brain of comprised of neurons?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the answer is no, which is quite possible, then new approaches to AI may be required to simulate the capabilities of the mind that are less obviously computational in nature &amp;ndash; capabilities like intuition, inspiration, compassion, jazz composition, or revolutionary thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sH4lI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sH4lI.png&quot; alt=&quot;Known Metabolic Pathways of Earthen Life&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmedhealth/PMHT0024269/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmedhealth/PMHT0024269/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-19T10:02:43.203" LastActivityDate="2018-07-19T10:02:43.203" Title="Is the neuron adequately comprehended?" Tags="&lt;philosophy&gt;&lt;research&gt;&lt;artificial-neuron&gt;&lt;cognitive-science&gt;&lt;neurons&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3932" PostTypeId="1" CreationDate="2017-08-30T18:57:42.780" Score="2" ViewCount="99" Body="&lt;p&gt;Due to my ignorance in this space I am not sure if a similar approach has already been proposed for what I outline below (I am not sure what terminology to even search for):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been doing some brainstorming about architectures for AI systems, and was considering the feasibility of an approach that borrows heavily from real-time game loop simulations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is to say, an approach that runs the AI simulation itself in real-time at a fixed rate regardless of external events, as opposed to feeding inputs in (near) real time into an existing network (i.e. clocked by events) and reading the outputs. This seems to me to be one way to allow for a temporal awareness and even some form of primitive &quot;consciousness&quot; within the AI system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pseudocode, here is what I've been thinking for a real-time AI loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;network = LoadNetwork();&#xA;registeredInputs = network.GetAllRegisteredInputs();&#xA;registeredOutputs = network.GetAllRegisteredOutputs();&#xA;timeStepTick = 0;&#xA;while(alive)&#xA;{&#xA;    //Iterate over inputs and process - Could use event-driven approach here, but would potentially require locking on the network if concurrency is applied&#xA;    foreach(registeredInput in registeredInputs)&#xA;    {&#xA;        //If the input has new samples since last iteration, apply it to mapped neurons&#xA;        if(registeredInput.HasNewSamples())&#xA;        {&#xA;            //Get new samples of input (these samples live in time domain)&#xA;            samples = registeredInput.GetNewSamples();&#xA;            foreach(sample in samples)&#xA;            {&#xA;                network.ProcessInputSample(timeStepTick, registeredInput, sample);&#xA;            }&#xA;        }&#xA;    }&#xA;&#xA;    //Iterate over outputs and process &#xA;    foreach(registeredOutput in registeredOutputs)&#xA;    {&#xA;        //Map relevant neuron outputs to the registered output &quot;frame&quot; buffer&#xA;        registeredOutput.Update();&#xA;&#xA;        //Whatever registered this output is responsible for reading the buffer at whatever rate is required &#xA;    }&#xA;&#xA;    //Whichever simulation time step algorithm - arbitrarily clock this network at 1khz&#xA;    Step(timeStepTick, 1);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using this sort of approach, I speculated about the possibility of registering certain internal inputs in addition to external inputs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;Brain wave&quot; signals: Could feed a series of generated sinusoidal waveforms into the network as a kind of baseline stimulation.&lt;/li&gt;&#xA;&lt;li&gt;Time of day signal: Simulation of things like circadian rythym. Could simply be a waveform with frequency of 1 day.&lt;/li&gt;&#xA;&lt;li&gt;Time Step signal: Feed current time step value (monotonic, starts at 0) as a sense of age. &lt;/li&gt;&#xA;&lt;li&gt;Feedback signals: Certain output neurons selected to be mapped back in as inputs. These would create a continuous loop of signaling between neurons which would continue even in the absence of any external input signals.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The inputs and outputs would obviously need to be sampled at a rate that falls within the Nyquist rate of the network (would be 512hz for the above psuedocode). For video data, this would be pretty easy as most video content is clocked within &amp;lt;100hz (tradeoff being large frame data). Audio data might need to be resampled as it usually runs around 48khz, but that isnt to say the simulation couldnt be clocked at something like 100khz to address both needs. The audio samples would be ingested at their native sample rate, and video frames would only be handled if a new frame arrives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The continuous load of such a system on the underlying hardware should be very easy to calculate based on number of mapped inputs, outputs, and sample rate. It might even be feasible to let the network run as fast as hardware allows (above some minimum constraint for input/output sampling rate), which could make it &quot;better&quot; in some arbitrary sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea what sort of neuron algorithm would be most appropriate here (I suspect existing algorithms will not work well), but I do think that it would have to be very efficient from a computational perspective considering the potential tick rate of the simulation. All of the magic definitely lives in the &lt;code&gt;ProcessInputSample()&lt;/code&gt; method in the code above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the current applications for such a network are very dubious due to the way it operates. This in my mind simply feels like a very complex DSP filter that just takes input waveforms (either external or internal) convolves them with various amounts of temporal phasing, and pipes to a series of output buffers that the real world interacts with. That being said, this also seems like the sort of &quot;flexibility&quot; that is required for an AI to learn and operate in a way that does not require some arbitrary number of iterations and a specific training model. It has always seemed to me that LTSM-style neurons were a bandaid for the temporal memory issue. Why push that computational concern to every single neuron when you can make it an inherent property of the AI by placing its entire simulation in the time domain?&lt;/p&gt;&#xA;" OwnerDisplayName="user9347" LastActivityDate="2017-08-30T18:57:42.780" Title="Continuous Real-Time AI Simulation Loop" Tags="&lt;ai-design&gt;&lt;real-time&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3933" PostTypeId="1" AcceptedAnswerId="3940" CreationDate="2017-08-31T05:32:04.277" Score="4" ViewCount="185" Body="&lt;p&gt;&lt;strong&gt;Scenario:&lt;/strong&gt; I am trying to create a dataset with images of choice for different animal classes. I am going to train those images for classification using CNN.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; Let's assume I somehow don't have the privilege to collect too many images and was only able to collect a few of them for each class. Here's the list:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;| id | animal       | #     |&#xA;|----|--------------|-------|&#xA;| 1  | Baboon       | 800   |&#xA;| 2  | Fox          | 1000  |&#xA;| 3  | Hyena        | 5000  |&#xA;| 4  | Giraffe      | 43    |&#xA;| 5  | Zebra        | 88    |&#xA;| 6  | Hippopotamus | 233   |&#xA;| 7  | Yak          | 578   |&#xA;| 8  | Polar Bear   | 456   |&#xA;| 9  | Lion         | 3442  |&#xA;| 10 | Indian Tiger | 40000 |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have three questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this a good dataset to train the CNN model? I am worried about the quantity each class has.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Will it be helpful if I augment the data? I think I am going to augment it.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the future, the above-mentioned dataset is going to increase. So there is a chance that I will train the model again. Should I create a model that fits the data of the present size or should I create a bigger one in order to adjust future data?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I can get data from the Internet. But this question is about the approaches to take when we have a small amount of data, like the one in National Data Science Bowl (classifying Planktons).&lt;/p&gt;&#xA;" OwnerUserId="6382" LastEditorUserId="2444" LastEditDate="2020-06-21T22:27:08.577" LastActivityDate="2020-06-21T22:37:01.400" Title="Would this relatively small dataset be enough to train a CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;&lt;data-augmentation&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3935" PostTypeId="1" AcceptedAnswerId="3937" CreationDate="2017-08-31T08:43:01.663" Score="7" ViewCount="114" Body="&lt;p&gt;Information security has become a thriving field during the last years. It is a broad domain ranging from &lt;strong&gt;planing&lt;/strong&gt; and &lt;strong&gt;building&lt;/strong&gt; over &lt;strong&gt;testing&lt;/strong&gt; to &lt;strong&gt;operating&lt;/strong&gt; different &lt;strong&gt;applications&lt;/strong&gt;, &lt;strong&gt;systems&lt;/strong&gt; and &lt;strong&gt;networks&lt;/strong&gt; in a secure fashion. From small embedded systems to large scale enterprise applications, security is always an important challenge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With some exceptions, I have hardly seen neural networks used in this domain to tackle those challenges. The most common exception is breaking CAPTCHAs with CNNs during penetration testing, analysis of network traffic and malware/spam detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are other challenges in information security that could be solved better with neural networks than with classical algorithms? Please include an explanation why those challenges would benefit from neural networks.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="1581" LastEditDate="2017-09-01T22:28:06.030" LastActivityDate="2018-10-15T23:54:24.807" Title="Which challenges in information security can be solved better using neural networks?" Tags="&lt;neural-networks&gt;&lt;security&gt;&lt;challenges&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3936" PostTypeId="2" ParentId="258" CreationDate="2017-08-31T09:42:19.097" Score="4" Body="&lt;p&gt;Only a small portion of the habituation, sensitization, and classical conditioning behavior of neurons has been primitively simulated in ANN systems. Simulation of actin cytoskeletal machinery&lt;sup&gt;1&lt;/sup&gt; and other agents of neural plasticity, central to learning new domains, is in its beginnings&lt;sup&gt;2&lt;/sup&gt;.  As of this writing, the complexity of neuron activation dwarfs the models being used in working commercial ANN systems, but the research continues along multiple fronts.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The neuroscience of learning&lt;sup&gt;3&lt;/sup&gt;,&lt;/li&gt;&#xA;&lt;li&gt;Parallel hardware approaches that better support ANN simulation accuracy&lt;sup&gt;4, 5&lt;/sup&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;Dynamic frameworks&lt;sup&gt;6&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This list and the examples referenced in the superscripts, with links below, represent a tiny sample of the information available and the work in progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK21493/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular Cell Biology. 4th edition.; Lodish H, Berk A, Zipursky SL, et al.; New York: W. H. Freeman; 2000.; Section 18.1 The Actin Cytoskeleton&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www.neuron.yale.edu/neuron/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEURON Software; Yale U&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK21648/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular Cell Biology. 4th edition.; Lodish H, Berk A, Zipursky SL, et al.; New York: W. H. Freeman; 2000.; Section 21.7 Learning and Memory&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0925231204000682&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Neural Networks on Massively Parallel Computer Hardware; Udo Seiffert; University of Magdeburg, Germany&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;https://web.stanford.edu/group/brainsinsilicon/neurogrid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeuroGrid; Stanford U&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] &lt;a href=&quot;https://ai.stackexchange.com/a/3803/2444&quot;&gt;Explanation of Dynamic Computational Graph frameworks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-03-09T01:13:14.997" LastActivityDate="2020-03-09T01:13:14.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3937" PostTypeId="2" ParentId="3935" CreationDate="2017-08-31T10:23:02.810" Score="5" Body="&lt;p&gt;The most natural place where artificial networks can be used in information security is in attack detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The security team leaders of more than one web hosting company told me the same story.  Their teams' daily challenges are to defend against the attacks mounted continuously by several overseas teams against the IT security of their hosting infrastructure.  Attack teams specialize in one target hosting company, and there are often many attack teams on one defensive security team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hosting company security teams try to preemptively defend the assets of their companies from attacks.  These teams also have to restore their hosting system with minimal impact on hosted sites whenever an attack achieves its goal, which may be data corruption, data theft, the insertion of malware, or some other manipulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary mechanism for preemptive defense is detection of an attacker and an ascertaining the attack's intended target within the hosting infrastructure's defenses, after which known blocks to these attacks are employed, such as IP or MAC filters, modification of access rights, upgrades that plug known security holes, or other measures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The progress of attacks occur across some set of changing fronts, as in a war or a game of chess.  Information security may soon become largely a game between two sets of computer systems.  One trying attacks, and the other trying to detect and block them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the defending team, high speed, real time detection of the attack is key.  A properly designed deep learning solution, configured and integrated into a defensive system, is more likely to block an attack than a person watching a dashboard of log file stats and system metrics.  A deep network could also be trained to discover the subsystem or specific defense against which the attack is intended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once detection occurs, the application of patches, exclusion filters, and denial criteria may best be accomplished through a rules based system.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:54:24.807" LastActivityDate="2018-10-15T23:54:24.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3938" PostTypeId="1" AcceptedAnswerId="4191" CreationDate="2017-08-31T11:04:35.500" Score="21" ViewCount="35407" Body="&lt;p&gt;Suppose that I have 10K images of sizes &lt;span class=&quot;math-container&quot;&gt;$2400 \times 2400$&lt;/span&gt; to train a CNN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I handle such large image sizes without downsampling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few more specific questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any techniques to handle such large images which are to be trained?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What batch size is reasonable to use?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any precautions to take, or any increase and decrease in hardware resources that I can do?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here are the system requirements&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ubuntu 16.04 64-bit &#xA;RAM 16 GB&#xA;GPU 8 GB&#xA;HDD 500 GB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6382" LastEditorUserId="2444" LastEditDate="2020-05-22T23:46:34.923" LastActivityDate="2020-05-22T23:48:08.090" Title="How do I handle large images when training a CNN?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="3940" PostTypeId="2" ParentId="3933" CreationDate="2017-08-31T15:47:46.887" Score="0" Body="&lt;p&gt;You can build this just with 100 images. In your case, Zebra and Giraffe need more images. With DNNClassifier (TensorFlow), you can do it. But the more images you have, the more accurate your classifier will be.&lt;/p&gt;&#xA;&lt;p&gt;I suggest that you also watch the video: &lt;a href=&quot;https://www.youtube.com/watch?v=cSKfRcEDGUs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Train an Image Classifier with TensorFlow for Poets - Machine Learning Recipes #6&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="2444" LastEditDate="2020-06-21T22:29:13.430" LastActivityDate="2020-06-21T22:29:13.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3941" PostTypeId="2" ParentId="3899" CreationDate="2017-08-31T15:58:27.900" Score="0" Body="&lt;p&gt;You can use Imagination-augmented to predict wrong and fatal actions. &lt;a href=&quot;https://www.youtube.com/watch?v=llwAwE7ItdM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep mind is working on it&lt;/a&gt; and I believe it is a great solution to many problems. You need to predict whether the best solution is to take a few steps forward so you do not get hit or go back a few steps.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2017-08-31T15:58:27.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3942" PostTypeId="1" AcceptedAnswerId="3957" CreationDate="2017-08-31T16:07:41.383" Score="5" ViewCount="512" Body="&lt;p&gt;We are careening into the future which may hold unpredictable dangers in relation to AI. I've haven't yet heard of Chappie or Robocop style police robots, but militarized drone tech is replacing many conventional weapons platforms.  I love the idea that I may one day be able to transfer my consciousness to a computer, and improve my capabilities and potential.  However, what constitutes morality can differ greatly among individual humans.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do we move forward toward the &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;singularity&lt;/a&gt; in a way that protects humans, as opposed to possibly lead to our extinction?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="7800" LastEditDate="2020-03-13T12:41:11.387" LastActivityDate="2020-03-13T12:41:11.387" Title="Is there a way to protect humanity against the impending singularity?" Tags="&lt;agi&gt;&lt;control-problem&gt;&lt;singularity&gt;&lt;neo-luddism&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3943" PostTypeId="1" AcceptedAnswerId="4455" CreationDate="2017-08-31T17:16:28.970" Score="10" ViewCount="225" Body="&lt;p&gt;I'm am quite new to deep learning but I think I found just the right real-world situation to start using it. The problem is that I have only used such algorithms to predict outcomes.  For my new project, I need information to feed a machine with to &lt;em&gt;optimize&lt;/em&gt; outcomes. Could someone explain briefly how I should proceed? I'm stuck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here's the situation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a machine that takes planks of wood with different grades of wood available throughout its length and has to cut it into blocks provided in a cut list. This machine will always choose the highest &lt;em&gt;score&lt;/em&gt; it can get from a given plank. The &lt;em&gt;score&lt;/em&gt; is obtained by multiplying each block's &lt;em&gt;area&lt;/em&gt; by its &lt;em&gt;multiplicator&lt;/em&gt;. The algorithm I want to build has to give that machine a &lt;em&gt;multiplicator&lt;/em&gt; for each block listed in a cut list. All of the physical output from this machine will be stocked on shelves by a robot until needed. The cutting machine is allowed to downgrade parts of a plank if it helps it reach a higher score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value has to act as an incentive for the machine to give me the block I need the most without downgrading too much wood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;OPTIMIZATION GOALS&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Make sure each block is in stock by the time it is needed, but not too early without reason&lt;/li&gt;&#xA;&lt;li&gt;Downgrade as little area of wood as possible (some species are very expensive)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;INPUT NODES&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Amount of time before this block is needed&lt;/li&gt;&#xA;&lt;li&gt;Grade of wood for this block&lt;/li&gt;&#xA;&lt;li&gt;Amount of this block needed&lt;/li&gt;&#xA;&lt;li&gt;Block's area (Maybe?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;FEEDBACK PROVIDED TO THE ALGORITHM&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Amount of time in advance that the block was ready (must be as low as possible)&lt;/li&gt;&#xA;&lt;li&gt;Area of wood downgraded * number of grades skipped&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;EXPECTED RETURN DATA&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A &lt;em&gt;multiplicator&lt;/em&gt; that will give that block an optimal its priority relative to others&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;INFORMATION I DON'T HAVE BUT COULD GATHER&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mean ratio of each grade for each species of wood&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What I've figured out so far is that I may need my feedback to be smashed in only one value in order to make it the output node. The problem is that I can't understand how to make this algorithm to determine a &lt;em&gt;multiplicator&lt;/em&gt;. Am I wrong in trying to solve this through deep learning?&lt;/p&gt;&#xA;" OwnerUserId="9361" LastEditorUserId="9361" LastEditDate="2017-09-05T14:39:22.287" LastActivityDate="2017-12-07T13:18:39.470" Title="A deep learning algorithm to optimize the outcome" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="11" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3944" PostTypeId="2" ParentId="3850" CreationDate="2017-08-31T23:05:54.807" Score="3" Body="&lt;p&gt;If a pseudorandom number generator is throwing out numbers, then, in the analysis of these numbers, you will be able to determine the algorithm that produced them, because the numbers aren't random; they are determined by that algorithm and not by chance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the world is made up of physical laws that are able to be understood and replicated, then the apparent randomness we observe in events is down to those physical laws. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pseudo-generator is no longer actually random, which, from its definition, is indeterminable, and presents a paradox. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can rules create randomness?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By definition, surely, our apparent perception of the randomness of events we observe is an illusion and is actually a certainty, which we are not unable to predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are absolute fundamental laws of nature and these laws create order out of the observed chaos, then the randomness and its reference are false and without any proper truthful meaning.&lt;/p&gt;&#xA;" OwnerUserId="9094" LastEditorUserId="2444" LastEditDate="2020-05-05T16:47:53.617" LastActivityDate="2020-05-05T16:47:53.617" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3946" PostTypeId="2" ParentId="3938" CreationDate="2017-09-01T09:03:45.067" Score="7" Body="&lt;p&gt;Usually for images the feature set is the pixel density values and in this case it will lead to quite a big feature set; also down sampling the images is also not recommended as you may lose (actually will) loose important data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] But there are some techniques that can help you reduce the feature set size, approaches like PCA(Principle Component Analysis) helps you in selection of important feature subset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For detailed information see link &lt;a href=&quot;http://spark.apache.org/docs/latest/ml-features.html#pca&quot; rel=&quot;noreferrer&quot;&gt;http://spark.apache.org/docs/latest/ml-features.html#pca&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Other than that to reduce the computational expense while training your Neural Network, you can use Stochastic Gradient Descent, rather than conventional use of Gradient Descent approach, that would reduce the size of dataset required for training in each iteration. Thus your dataset size to be used in one iteration would reduce, thus would reduce the time required to train the Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact batch size to be used is dependent on your distribution for training dataset and testing datatset, a more general use is 70-30. Where you can also use above mentioned Stochastic approach to reduce required time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detail for Stochastic Gradient Descent &lt;a href=&quot;http://scikit-learn.org/stable/modules/sgd.html&quot; rel=&quot;noreferrer&quot;&gt;http://scikit-learn.org/stable/modules/sgd.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] The Hardware seems apt for the upgradation would be required, still if required look at cloud solutions like AWS where you can get free account subscription upto a limit of usage.&lt;/p&gt;&#xA;" OwnerUserId="9374" LastActivityDate="2017-09-01T09:03:45.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3947" PostTypeId="1" CreationDate="2017-09-01T11:12:40.147" Score="2" ViewCount="114" Body="&lt;p&gt;I have an idea for a transformative decision rule (TDR), but I want to know if there's already a name for it.     &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any ordinal scale should be converted to a scale such that all  the outcomes are assigned consecutive values.          &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let &lt;em&gt;v&lt;/em&gt; be the function that assigns values to outcomes (defined as ordered pairs of acts and states).     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;em&gt;x&lt;/em&gt; and &lt;em&gt;y&lt;/em&gt; denote any particular such ordered pairs.&lt;br&gt;&#xA;Let &lt;em&gt;P&lt;/em&gt; be the preference.&lt;br&gt;&#xA;Let &lt;em&gt;X+&lt;/em&gt; be the set of all outcomes that are more preferred than &lt;em&gt;x&lt;/em&gt; (similarly also for &lt;em&gt;y&lt;/em&gt;) &lt;em&gt;X+&lt;/em&gt; in &lt;em&gt;P&lt;/em&gt;.&#xA;&amp;nbsp;     &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If &lt;em&gt;x&lt;/em&gt; in &lt;em&gt;Y+&lt;/em&gt;, and &lt;em&gt;x&lt;/em&gt; is the closest outcome to &lt;em&gt;y&lt;/em&gt; in &lt;em&gt;Y+&lt;/em&gt;, then &lt;em&gt;v(x) = v(y) + 1&lt;/em&gt; for all &lt;em&gt;x,y&lt;/em&gt; in P.       &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I could have explained it more formally mathematically, but Latex doesn't seem to render here. If there's no name for it, I'm thinking of &quot;The Law of Consecutive Ordinals (LCO)&quot;. As for why I support the rule, it's because it's a rule that needs to be applied to an ordinal scale before I apply an effective decision rule (EDR) that I thought of for ordinal scales. I want to know if there's a name for that principle.&lt;/p&gt;&#xA;" OwnerUserId="8362" LastActivityDate="2018-06-12T12:24:45.710" Title="Is there a name for this principle?" Tags="&lt;decision-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3948" PostTypeId="1" CreationDate="2017-09-01T13:48:47.733" Score="2" ViewCount="198" Body="&lt;p&gt;I've been working on a project (Android Game), in which the player has to confront with some obstacles/enemies whom he has to destroy. So, is there a way in which we can monitor how the user of the game plays and accordingly to generate (and timely updating) that trained model, which can be used later in the game to make the user think of a different way to defeat an enemy unlike going in the same streamline flow which he has followed till now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've implemented Q-Learning and Genetic algorithms on the PC, to make the Game AI for some games like 'tetris' to make the computer play on it's own. But, haven't done it in Android till now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Already searched: I've also referred to some websites in which they suggested using Neural Networks to encounter the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, I'm unable to get an accurate procedure in which this can be done. Please suggest me a way in which I can monitor the user's input on how he plays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8968" LastActivityDate="2018-05-20T17:27:13.020" Title="Is there any way, in which we can monitor what's going on the screen in android to generate a model from it?" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3949" PostTypeId="1" AcceptedAnswerId="3967" CreationDate="2017-09-01T14:44:09.853" Score="6" ViewCount="4234" Body="&lt;p&gt;My question is more about &quot;is it possible?&quot; and &quot;is this the right approach?&quot; So let me explain what my idea is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think about a system which gets XML documents in various structures but with essentially the same data structure in it. For the example, lets assume each document contains data about one or more persons. So the AI would recognize a name. Somewhere else in the document there is the post address of our fictional person. The AI should now &quot;see&quot; the address and conclude, it belongs to our person. Anywhere else, there is a phone number in the document. Again, our AI should see the connection between our person and this phone number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This wouldn't be a job for an AI if there wasn't a catch. If the task was merely to find and map strings like addresses and phone numbers, we could simply use a regex to match our &quot;target strings&quot;. The catch in this scenario would be this: the XML document might contain other data, which does not belong to our person but is a valid phone number for example and thus will match an regex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the big question is: Would it be possible for an AI to learn this and if yes, with which framework would someone create such an AI?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Sample XML document:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?&amp;gt;&#xA;&amp;lt;document&amp;gt;&#xA;    &amp;lt;data&amp;gt;&#xA;        &amp;lt;foo&amp;gt;&#xA;            &amp;lt;bar&amp;gt;&#xA;                &amp;lt;person&amp;gt;&#xA;                    &amp;lt;name&amp;gt;John Doe&amp;lt;/name&amp;gt;&#xA;                &amp;lt;/person&amp;gt;&#xA;            &amp;lt;/bar&amp;gt;&#xA;            &amp;lt;address&amp;gt;&#xA;                &amp;lt;street&amp;gt;Main street 1&amp;lt;/street&amp;gt;&#xA;                &amp;lt;city&amp;gt;1111 Twilight town&amp;lt;/city&amp;gt;&#xA;                &amp;lt;country&amp;gt;sample country&amp;lt;/country&amp;gt;&#xA;            &amp;lt;/address&amp;gt;&#xA;            &amp;lt;phone&amp;gt;+123 123 123&amp;lt;/phone&amp;gt;&#xA;        &amp;lt;/foo&amp;gt;&#xA;        &amp;lt;foo&amp;gt;&#xA;            &amp;lt;bar&amp;gt;&#xA;                &amp;lt;person&amp;gt;&#xA;                    &amp;lt;name&amp;gt;Jane Doe&amp;lt;/name&amp;gt;&#xA;                &amp;lt;/person&amp;gt;&#xA;            &amp;lt;/bar&amp;gt;&#xA;            &amp;lt;address&amp;gt;&#xA;                &amp;lt;street&amp;gt;Broadway 42&amp;lt;/street&amp;gt;&#xA;                &amp;lt;city&amp;gt;4521 Traverse town&amp;lt;/city&amp;gt;&#xA;                &amp;lt;country&amp;gt;sample country&amp;lt;/country&amp;gt;&#xA;            &amp;lt;/address&amp;gt;&#xA;            &amp;lt;phone&amp;gt;+123 412123&amp;lt;/phone&amp;gt;&#xA;        &amp;lt;/foo&amp;gt;&#xA;    &amp;lt;/data&amp;gt;&#xA;    &amp;lt;creator&amp;gt;&#xA;        &amp;lt;!-- Note: While this looks like a valid person, --&amp;gt;&#xA;        &amp;lt;!-- this data should not be matched by the AI --&amp;gt;&#xA;        &amp;lt;name&amp;gt;Sam Smith&amp;lt;/name&amp;gt;&#xA;        &amp;lt;office&amp;gt;&#xA;            &amp;lt;street&amp;gt;Seaside road 5&amp;lt;/street&amp;gt;&#xA;            &amp;lt;city&amp;gt;4521 Traverse town&amp;lt;/city&amp;gt;&#xA;            &amp;lt;country&amp;gt;sample country&amp;lt;/country&amp;gt;&#xA;        &amp;lt;/office&amp;gt;&#xA;        &amp;lt;phone&amp;gt;+123 555 555&amp;lt;/phone&amp;gt;&#xA;    &amp;lt;/creator&amp;gt;&#xA;&amp;lt;/document&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9380" LastEditorUserId="9380" LastEditDate="2017-09-04T06:37:26.803" LastActivityDate="2019-02-07T12:09:05.353" Title="Use AI to interpret XML?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;structured-data&gt;" AnswerCount="2" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3953" PostTypeId="1" AcceptedAnswerId="3958" CreationDate="2017-09-01T22:34:05.867" Score="3" ViewCount="632" Body="&lt;p&gt;I'm currently in the process of learning about using CNNs in image recognition. Many of the different resources I read that were explaining the motivation referred to the fact that these networks are (to some degree) translationally invariant. My understanding is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fully connected networks are ill-suited to image recognition because of the high dimensionality of the data and especially because they do not preserve the spatial relationships between pixels. I care about the state of the pixels surrounding pixel X as well as X itself.&lt;/li&gt;&#xA;&lt;li&gt;CNNs remedy this because they look at the image in 2 dimensions so that surrounding pixels are being considered as well (the kernel does this). However, while I care about what is around X, I don't care where X is. So, I apply the kernel the same everywhere and I make a bunch of layers so that I can get a bunch of kernels.&lt;/li&gt;&#xA;&lt;li&gt;I also use pooling. This reduces the data dimensions but also adds some of that translation invariance.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, my question is: What feature of the CNN is causing the invariance? I saw some explanations saying it was a result of the maps basically activating when a certain feature shows up, regardless of where. Other said that the pooling meant that if a horizontal line, for instance, showed up in one place vs. a pixel over, the pooling would activate the same for both. It seems like the first reason would be totally sufficient and the second not really adequate but I could also see it being both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read a paper (&lt;a href=&quot;https://arxiv.org/pdf/1606.09549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1606.09549.pdf&lt;/a&gt;) about fully-convolutional networks. In section 2.1, the authors explain that they intentionally chose to make it fully-convolutional because that allowed them to &quot;compute the similarity at all translated sub-windows on a dense grid in a single evaluation.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That sentence makes me think that it is more the first explanation. Anyways, I hope to gain a better intuitive understanding of how the different parts of the CNN come together to work particularly well on images. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2017-09-02T12:31:02.923" Title="Intuitively understanding translational invariance in CNNs" Tags="&lt;image-recognition&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3954" PostTypeId="2" ParentId="3894" CreationDate="2017-09-02T05:00:53.013" Score="1" Body="&lt;p&gt;I think Demento has answered it well, but probably below can add some more understanding to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Usually the data stored for image processing relates to the &lt;strong&gt;Image Pixel Densities&lt;/strong&gt;(&lt;em&gt;as per the many courses i visited online&lt;/em&gt;), which can be very well maintained using a &lt;strong&gt;matrix of pixel density values&lt;/strong&gt;, corresponding to the resolution of each image.&#xA;Then it depends on the image's color spectrum (&lt;em&gt;is it a colored or a gray scale&lt;/em&gt;); as per the case of hand written text recognition you don't really require the pixel values for a colored image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that if you have a colored image then you have to store the RGB values for each pixel, thus &lt;strong&gt;tripling&lt;/strong&gt; the matrix size, and the training could be very well done with just the gray scale values, thus it would be better to convert all the images to gray scale in preprocessing step and also take care to convert the image to gray scale when you actually use the trained network to recognize an image (&lt;em&gt;just a novice mistake!&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per the &lt;strong&gt;storage&lt;/strong&gt; for the required matrix form of the pixel density values, a flat file storage would suffice, but I would recommend libraries instead &lt;em&gt;(if working in python or other alternatives&lt;/em&gt;), like &lt;strong&gt;pandas&lt;/strong&gt; and &lt;strong&gt;numpy&lt;/strong&gt;. For these libraries provide a robust solution for data management and retrieval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details - &lt;a href=&quot;https://docs.scipy.org/doc/numpy-dev/user/quickstart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/numpy-dev/user/quickstart.html&lt;/a&gt;&#xA;&lt;a href=&quot;https://github.com/pandas-dev/pandas&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pandas-dev/pandas&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Now I would emphasize on why not to use any DataBase(DB) for the storage of such information.&#xA;Firstly the integration of it would only result in an &lt;strong&gt;unnecessary overhead&lt;/strong&gt; to your efforts of training the Network and also when you would want to recognize an image(&lt;em&gt;which would always be a single matrix form of it's Image Pixel Densities&lt;/em&gt;), you would require the connection to the same DB and would also need to make an &lt;strong&gt;insertion&lt;/strong&gt; to it first, for the script to be able to extract from there and make recognition(&lt;em&gt;not recomended&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a fact we don't really concern ourselves with is the quality of the training image dataset, we do ignore the presence of &lt;strong&gt;noise&lt;/strong&gt; in the dataset(&lt;em&gt;would be minimum if using a preprocessed standard data set such MNIST &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yann.lecun.com/exdb/mnist/&lt;/a&gt;&lt;/em&gt;), but if creating one's own data set we have to consider noise and rectify accordingly in the preprocessing task itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With noise I mean &lt;strong&gt;loss or overlap of pixel density information&lt;/strong&gt; which is usually adds a &lt;strong&gt;blurring effect&lt;/strong&gt; to the images, and can seriously damage the training metrics.&#xA;To overcome it, there are approaches to rectify the pixel values with a prior learning methods such as extracting data from obfuscated images,&#xA; see link- &lt;a href=&quot;https://arxiv.org/pdf/1609.00408.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1609.00408.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;With the above case you would require to make updates in the DB for very single record in the preprocessing task even before you start to train the Network, thus it can increase the total requests to the DB in multifolds and thus impeding the whole benefit of using it in the first place&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose the above should give you fair idea, about the kind of data to store and the Data Storage to prefer accordingly.&#xA;I worked out the Digit Classification using MNIST data set with obscuration using a NN Pipeline, to refer &lt;a href=&quot;https://github.com/kchopra456/Digit-Classification-and-Image-Obfuscation-ANN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kchopra456/Digit-Classification-and-Image-Obfuscation-ANN&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9374" LastActivityDate="2017-09-02T05:00:53.013" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3956" PostTypeId="1" AcceptedAnswerId="4268" CreationDate="2017-09-02T09:04:49.463" Score="7" ViewCount="1483" Body="&lt;p&gt;There are two potential approaches when performing cross-over operation in genetic algorithms.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use only the elites in the pool, probably the ones that are also going to be directly transferred to the next generation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use all the population present in the pool.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is there any evidence that cross-over only with the elites of the population makes the GA converge faster to a good solution? I guess that, in order to escape from local minima, cross-over with all the population is needed. On the other hand, why should we perform cross-over with the least fit individuals?&lt;/p&gt;&#xA;&lt;p&gt;Any idea?&lt;/p&gt;&#xA;" OwnerUserId="6258" LastEditorUserId="2444" LastEditDate="2020-12-07T17:36:38.543" LastActivityDate="2020-12-07T17:36:38.543" Title="Is elitism preferred over non-elitism in the cross-over operator?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;crossover-operators&gt;&lt;elitism&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3957" PostTypeId="2" ParentId="3942" CreationDate="2017-09-02T11:58:20.257" Score="2" Body="&lt;p&gt;What you are talking about is known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_control_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Control Problem&lt;/a&gt;. We have our own &lt;a href=&quot;https://ai.stackexchange.com/questions/tagged/control-problem&quot;&gt;tag&lt;/a&gt; for this specific topic here, which you can use for this and similar questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to address the control problem is heavily discussed and still considered unsolved. Two of the important approaches are &lt;strong&gt;motivation control&lt;/strong&gt; and &lt;strong&gt;capability control&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Motivation control aims at &lt;strong&gt;creating suitable reward functions&lt;/strong&gt; that only reward behavior that's beneficial to the human race. This topic alone is so complex that many different competing theories exist. Minor errors in the definition of an AI's goal might lead to catastrophe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Capability control aims at &lt;strong&gt;limiting the possible scope of action&lt;/strong&gt; for the AI. The main problem here is, that most restrictions make the AI less useful. And to ensure robust control a lot of restrictions must be put in place. Finding the right tradeoff is a challenge we need to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to those two mechanisms we must think about how we create a superintelligence. One of the major aspects here is our &lt;strong&gt;understanding of the underlying technologies&lt;/strong&gt;. If we create a self-optimizing neural network from scratch, which we do not understand in depth, it is very likely that we will not understand the potential threats before it is to late. Creating a digital copy of a human brain as a starting point on the other hand is a little saver, because we can assume that it will be driven by similar factors as a real human being. If that's desirable is a totally different discussion of course. But the general idea is, that understanding the motivation of a superintelligence is key for steering it in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details about the topics I scratched in my answer and the control problem in general, I recommend reading &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superintelligence&lt;/a&gt; by Nick Bostrom. He spends many chapters on those problems and possible solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have more in-depth questions concerning specific topics, please open a new question to get a more detailed answer. &lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="1581" LastEditDate="2017-09-03T08:24:58.210" LastActivityDate="2017-09-03T08:24:58.210" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3958" PostTypeId="2" ParentId="3953" CreationDate="2017-09-02T12:31:02.923" Score="3" Body="&lt;p&gt;The translational invariance comes from the fact that the kernels are applied everywhere. Basically the kernels are &quot;translated&quot;, i.e. shifted, and that means the feature they detect are detected everywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pooling may help with that in some cases, because in the end you need a dog at the right side to activate the same neuron as a dog on the left, so there needs to be funnelling of the information, but it is not strictly necessary. You could also imagine a CNN with a dog kernel in a late layer, which just recognises dogs everywhere, with no pooling necessary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fully connected networks are wasteful in that they don't share weights. To get the necessary translational invariance you therefore need to learn pretty much the same weights at all positions. But they do preserve the spatial relationship between pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pooling reduces the dimensionality, but it also forgets the precise position of a feature. This might help with generalisation, but I'm not sure whether that's a big factor. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2017-09-02T12:31:02.923" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3959" PostTypeId="1" CreationDate="2017-09-02T13:29:09.183" Score="2" ViewCount="6016" Body="&lt;p&gt;Is there a way to train AI to find aspecific line or symbol in a image and crop it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCV scripts finds a face and crops it: how can I add my annotations? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lest say I have a image like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA; +------+&#xA; |  *  *|&#xA; |      |&#xA; |  *  *|&#xA; |      |&#xA; +------+&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want it to find the * and crop it.&lt;/p&gt;&#xA;" OwnerUserId="9393" LastEditorUserId="18819" LastEditDate="2018-11-18T12:35:53.643" LastActivityDate="2018-11-18T13:06:11.213" Title="Cropping image using ML?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;image-recognition&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3960" PostTypeId="2" ParentId="3959" CreationDate="2017-09-02T16:37:36.090" Score="6" Body="&lt;p&gt;Yes, this is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is actually a pretty &lt;strong&gt;easy way that doesn't even require machine learning&lt;/strong&gt; and can be &lt;strong&gt;implemented with a small amount of code&lt;/strong&gt;. You just use a framework for image processing (e.g. &lt;a href=&quot;https://pypi.python.org/pypi/PIL&quot; rel=&quot;noreferrer&quot;&gt;PIL&lt;/a&gt; for Python), find the marks by going over your image with an appropriate filter and use the implemented crop function, that the framework hopefully provides (or write i yourself with 2 nested for loops copying the area). Your filter will react strongest in the areas of your image that look like the mark you defined. This could actually be implemented like a single layer conv+RELU layer in a CNN (see &lt;a href=&quot;https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/&quot; rel=&quot;noreferrer&quot;&gt;this introduction&lt;/a&gt; for details about conv and RELU layers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because you are asking explicitly about machine learning, I will also give you the hard version to solve this problem using a full CNN to identify the bounding box. Once you have found the bounding box, the actual cropping can be done like in the example above and is not part of the CNN. You could call it post processing if you like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technique you are looking for is called &lt;a href=&quot;https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/object_localization_and_detection.html&quot; rel=&quot;noreferrer&quot;&gt;Object Localization and Detection&lt;/a&gt;. The linked article should tell you all the details you need. You basically build and train your CNN to localize and identify interesting objects in your image. The &lt;strong&gt;easy way&lt;/strong&gt; is to train your CNN to &lt;strong&gt;identify the 4 marks individually&lt;/strong&gt; and return the coordinates of the 4 areas with the highest likelihood to contain your marks. You calculate the center of those 4 individual points and feed it to your cropping function. The &lt;strong&gt;hard way&lt;/strong&gt; is to &lt;strong&gt;find the smallest bounding box containing all 4 marks&lt;/strong&gt;. This is a much harder problem, because the neural network needs to learn that it should ignore everything except those 4 marks during detection, although they will be a small portion of the bounding box. It can be done, but looking for the 4 marks individually would be much easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which approach is best for you? Depends on your goal. If you need a &lt;strong&gt;robust and efficient algorithm&lt;/strong&gt; for this problem, &lt;strong&gt;forget about ML&lt;/strong&gt; and &lt;strong&gt;implement it with the straight forward approach&lt;/strong&gt; I described first. No need for fancy learning CNNs. You can implement this with a few dozen lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is a research or training project to study ML, take my second approach. The easy and the hard way are both fine, the hard one obviously more challenging for your CNN. If you are new to CNN, I recommend following &lt;a href=&quot;https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;noreferrer&quot;&gt;this Stanford Course&lt;/a&gt;. It teaches you all you need for your project in 25 to 30 hours (excluding homework).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: Concerning your question, how do you add your own marks. I would simply do it using with the same framework you use for cropping. Your program needs some kind of image manipulation capabilities and this will allow pixel manipulation as well (at least the framework &lt;a href=&quot;https://pypi.python.org/pypi/PIL&quot; rel=&quot;noreferrer&quot;&gt;PIL&lt;/a&gt; which I suggested has this capability). Just draw your mark using this method. You can also copy a sample mark with a transparent background using a tool like Gimp manually, if you prefer that.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="9161" LastEditDate="2017-09-02T16:42:49.740" LastActivityDate="2017-09-02T16:42:49.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3961" PostTypeId="2" ParentId="3942" CreationDate="2017-09-03T09:18:55.570" Score="2" Body="&lt;p&gt;&lt;strong&gt;Impending or Past?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Niel Postman, in his book, &lt;em&gt;Technopoly&lt;/em&gt;, argues the preemption of human centered culture to technically driven culture has already occurred.  Jaques Ellul, in his book, &lt;em&gt;Technological Society&lt;/em&gt;, heaped evidence behind the proposal that technology became autonomous centuries ago.  Their arguments are convincing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some think other criteria must be met before the balance of power has been tipped in favor of some class of machines, but the traditional criteria proposed for determining when humans have been dominated are intellectually impoverished.  Let's look at some of those criteria.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Self awareness&lt;/li&gt;&#xA;&lt;li&gt;Superior intelligence&lt;/li&gt;&#xA;&lt;li&gt;Self-improving&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Self-awareness&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-awareness is likely to be a continuum, not a threshold.  The first time an assembler assembled itself, that continuum began, followed by compilers that can compile themselves, followed by neural nets that can construct and tune neural nets.  The dimensions of self-awareness are numerous, and complete self-awareness in each respect is rare.  For instance, knowing our own motive is often guessed at after we exercise it.  If we are honest about our own species, self-awareness in the typical human is intermittent and incomplete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Superior Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Superiority in intelligence cannot be determined by a chess tournament; the vanquished chess player could possibly, on the day of the defeat, invent a new field of study.  Neither Lavoisier nor Newton, if alive today, would be able to prevail against a chess program, yet no machine exposed only to information prior to the advent of modern chemistry, calculus, or physics would be able to take the steps Lavoisier or Newton made in the creation of those fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence is multidimensional too, and metrics to measure intelligence are unreliable.  Using simplistic definitions of intelligence, devices of steel, plastic, epoxy, and silicon already exceed that of humans.  Should graduate students seek out a university librarian for an answer to a question about data science or use Google Scholar?  A mail sorter uses intelligent robotics to ensures we get the right mail.  The actual mail delivery people occasionally deliver a parcel into the wrong mailbox even when the street address is clearly printed in one decimeter high Arial font digits on the approaching side of the mailbox.  We use a CAD program to check a mechanical design clearance.  Arithmetic is clearly the domain of computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans give destinations to a program and instructions are given back to the human.  The human essentially rents this intelligent service.  Once the instructions are given, the human becomes subservient and is expected to drive.  Even upstream in the transportation process, when the destination is chosen, to a large degree it is a technology driven need that often creates the demand for transportation.  Intelligence is not always correlated with power either.  How intelligent are long blades of grass and a broken lawnmower?  Yet they can send their legal owner across town to pick up parts and consume their entire day off to repair and mow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Self-improvement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-improvement is a slippery concept as well, whether an individual entity can improve itself in real time or is able to construct improved versions of itself.  Robotics creating robotics technically began during industrialization when milling machines made parts for milling machines.  Human supervision was involved then.  Using current technology, it is quite possible that an advanced manufacturing plant could be devices that could out other advanced manufacturing plants without supervision.  That is the mechanical dimension of self-improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a neural net converges on a reasonably optimal weighting for a given criteria, the assumption required to call this self-improvement is that the target categorization is correct.  Where power is concerned, self-improvement would be measured by what?  Control over resources?  Domination?  Is that improvement?  Perhaps ethical superiority?  Can subservience be considered a goal state without severely curtailing the creative and intuitive aspects required to achieve human-like intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is Singularity Realistic?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In light of this complexity, is the tipping of power between humans and machines discrete or singular?  Clearly not.  Calling changes in the balance of power between human driven culture and machine driven culture a singularity is naive.  We do not need to be attacked by machines to loose our culture or autonomy to technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We Seem to Know at Some Level&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This reality expresses itself subconsciously.  Young adults express violent feelings against their mobile devices when it delivers unpalatable social information.  Then they dodge the question when others inquire about the broken device display.  When the device finally ceases to function, they are compelled to buy a new mobile device, a device they need and pretend to like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another More Interesting Criteria: Utility&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human autonomy is dwarfed by the demands of the tools created for lifestyle improvement to the point that one can no longer, with formal logic, clearly differentiate which is the tool and which is the user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last criteria to judge when humans have become subservient to machines can be illuminated by a single test question, &quot;Which is the tool and which is the user?&quot;  When the distinction is blurred, the dominance has already entered a shared state.  Entrance into the vicinity of a tipping point, if such a tipping point actually exists, has already occurred in many regions of the world.  The multidimensional nature of the balanced thus far discussed indicates that there exists a complex balance with no clear point of victory or defeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Irreversability&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, the failure of the Unibomber and other Luddites to dent the onslaught of technology indicates that the relinquishment of autonomy to machines is likely irreversible.  Therefore, there may be no value in being concerned.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-09-05T20:48:41.130" LastActivityDate="2017-09-05T20:48:41.130" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3962" PostTypeId="1" AcceptedAnswerId="3996" CreationDate="2017-09-03T12:22:57.440" Score="7" ViewCount="830" Body="&lt;p&gt;I'm working on implementation of the backpropagation algorithm for a simple neural network which predicts a probability of survival (1 or 0) and I can't get it above 80% no matter how much I try to set the right hyperparameters. I suspect that's because my backpropagation is implemented incorrectly since I tried 2 different types of code and both give me same results. Is my backpropagation implemented correctly? Also how can I improve my model to give a better prediction?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class NeuralNetwork(object):&#xA;&#xA;def __init__(self, input_nodes, hidden_nodes, output_nodes, learning_rate):&#xA;    # Set number of nodes in input, hidden and output layers.&#xA;    self.input_nodes = input_nodes&#xA;    self.hidden_nodes = hidden_nodes&#xA;    self.output_nodes = output_nodes&#xA;    self.lr = learning_rate&#xA;&#xA;    # Initialize weights&#xA;    self.input_hidden_weights = np.random.randn(hidden_nodes, input_nodes) # 10x7&#xA;    self.hidden_output_weights = np.random.randn(output_nodes, hidden_nodes) # 1x10&#xA;&#xA;&#xA;    # Sigmoid activation funciton&#xA;    self.sigmoid = lambda x: 1/(1+np.exp(-x))&#xA;    self.diff_sigm = lambda x: x*(1-x)&#xA;&#xA;def train(self, input_list, label_list):&#xA;&#xA;    # Create an array of inputs and labels&#xA;    inputs = np.array(input_list, ndmin=2).T # 7x1&#xA;    labels = np.array(label_list, ndmin=2) # 1x1&#xA;&#xA;    # Forward propagation&#xA;    hidden_layer = self.sigmoid(np.dot(self.input_hidden_weights, inputs))        &#xA;    output_layer = self.sigmoid(np.dot(self.hidden_output_weights, hidden_layer))&#xA;&#xA;    final_output = output_layer&#xA;&#xA;&#xA;    # Error function&#xA;    output_errors = labels-final_output&#xA;&#xA;    # Backpropagation  &#xA;    output_delta = output_errors * self.diff_sigm(output_layer)&#xA;    hidden_delta = np.dot(self.hidden_output_weights.T, output_delta) * self.diff_sigm(hidden_layer)&#xA;&#xA;    # Update the weights&#xA;    self.hidden_output_weights += np.dot(output_delta, hidden_layer.T) * self.lr&#xA;    self.input_hidden_weights += np.dot(hidden_delta, inputs.T) * self.lr&#xA;&#xA;&#xA;    &quot;&quot;&quot;&#xA;    # Backpropagation&#xA;    hidden_errors = np.dot(self.hidden_output_weights.T, output_errors)        &#xA;    hidden_grad = hidden_layer * (1.0 - hidden_layer)&#xA;&#xA;    # Update the weights&#xA;    self.hidden_output_weights += self.lr * np.dot(output_errors.T, output_layer.T) # update hidden-to-output weights with gradient descent step    &#xA;    self.input_hidden_weights += self.lr * np.dot(hidden_errors * hidden_grad, inputs.T)  # update input-to-hidden weights with gradient descent step&#xA;    &quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9406" LastActivityDate="2017-10-07T23:40:18.953" Title="How do I know if my backpropagation is implemented correctly?" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;&lt;backpropagation&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3963" PostTypeId="2" ParentId="3962" CreationDate="2017-09-03T14:29:09.647" Score="1" Body="&lt;p&gt;For future reference, you can check your correctness by the finite difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cedar.buffalo.edu/~srihari/CSE574/Chap5/Chap5.3-BackProp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cedar.buffalo.edu/~srihari/CSE574/Chap5/Chap5.3-BackProp.pdf&lt;/a&gt; (p.23)&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-09-03T14:29:09.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3964" PostTypeId="1" CreationDate="2017-09-03T20:13:52.853" Score="5" ViewCount="417" Body="&lt;p&gt;I understand that a neural network basically distorts(non-linear transformation) and changes the perspective(linear transformations) of input space to draw a plane to classify data. How does the network deduce if an input is one side of a plane and therefore output the decision? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2018-02-03T07:30:58.547" Title="How does neural network classifier classify from just drawing a decision plane?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;linear-algebra&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3965" PostTypeId="1" CreationDate="2017-09-04T07:20:24.417" Score="11" ViewCount="236" Body="&lt;p&gt;Recently I was working on a problem to do some cost analysis of my expenditure for some particular resource.&#xA;I usually make some manual decisions from the analysis and plan accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a big data set in excel format and with hundreds of columns, defining the use of the resource in various time frames and types(other various detailed use).&#xA;I also have information about my previous 4 years of data and actual resource usage and cost incurred accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping to train a NN to predict my cost beforehand and plan even before I can manually do the cost analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the biggest problem I'm facing is the need to identify the features for such analysis. I was hoping there is some way to identify the features from the data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;PS - I have idea about PCA and some other feature set reduction techniques, what I'm looking at is the way to identify them in the first place.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="9374" LastEditorUserId="2444" LastEditDate="2018-10-27T22:04:33.037" LastActivityDate="2019-01-30T13:03:15.707" Title="How do I select the relevant features of the data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;feature-selection&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3966" PostTypeId="2" ParentId="3169" CreationDate="2017-09-04T14:01:34.463" Score="1" Body="&lt;p&gt;Look at MUMBLE, which is a kind of transducer working from Conceptual Dependency structures (which are similar to what you've got) and produce English output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Lisp implementation of Meehan's TALESPIN program, which includes a simplified version of MUMBLE can be found here: &lt;a href=&quot;https://github.com/lispm/CommonLispCode/blob/master/micro-talespin.lisp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lispm/CommonLispCode/blob/master/micro-talespin.lisp&lt;/a&gt; -- the generator part starts at about line 1030. It should be easy to follow, even if you do not know Lisp.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2017-09-04T14:01:34.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3967" PostTypeId="2" ParentId="3949" CreationDate="2017-09-04T17:45:16.163" Score="5" Body="&lt;p&gt;XML, HTML and less formal languages all respond quite nicely to being transformed or interrogated within a graph framework. XML and HTML are particularly useful in that they conform strictly to a tree-structure. That means that any good data components can be measured in terms of tree-distance to any other &quot;good&quot; data components. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you extract your regex-friendly terms and keep track of where within their tree they are found, you &lt;em&gt;may&lt;/em&gt; be able to cast those values into a general document-space vector (it might only need to be one-dimensional), allowing you to identify clusters of &quot;good&quot; vs anomalous sections of &quot;bad&quot; data based on a simple distance metric, or anomaly detection algorithm - say an &lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html&quot; rel=&quot;noreferrer&quot;&gt;isolation-forest&lt;/a&gt; that runs on information density for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This depends on your data, and how much of it you can find, ideally already tagged up containing good vs bad. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're looking to scrape reliable address-contents, then yes, you're likely to score hits on names, addresses, postcodes and phone-numbers all appearing as tightly connected clustered groups, all within one or two nodes-distance from one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, an annotation containing a phone number lodged somewhere else is less likely to be a match. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different documents will have different threshold densities, and differing anomaly to conformant ratios, so you'd have a task on your hands to figure out some way to automatically tune your parameters on any given document set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the past, I've tried doing this against html by flattening all the content into a single string of text and a similar approach yielded half-decent results, but if you're looking at XML, it's fair to expect the structure to yield more information. &lt;/p&gt;&#xA;" OwnerUserId="9421" LastActivityDate="2017-09-04T17:45:16.163" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3968" PostTypeId="2" ParentId="3947" CreationDate="2017-09-04T17:54:34.410" Score="1" Body="&lt;p&gt;I'd describe this as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Ranking&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ranking&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's not always possible to distil a set of outcome preferences to a transitive metric, this is one of the central axioms of game theory. &lt;a href=&quot;http://gametheory101.com/courses/game-theory-101/the-transitivity-axiom/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Transitivity Axiom&lt;/a&gt; so the problem is relatively well understood (that is, at least, if I've understood your problem)&lt;/p&gt;&#xA;" OwnerUserId="9421" LastActivityDate="2017-09-04T17:54:34.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3969" PostTypeId="2" ParentId="17" CreationDate="2017-09-04T18:42:02.433" Score="3" Body="&lt;p&gt;The singularity, in the context of AI, is a theoretical event whereby an intelligent system with the following criteria is deployed.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Capable of improving the range of its own intelligence or deploying another system with such improved range&lt;/li&gt;&#xA;&lt;li&gt;Willing or compelled to do so&lt;/li&gt;&#xA;&lt;li&gt;Able to do so in the absence of human supervision&lt;/li&gt;&#xA;&lt;li&gt;The improved version sustains criteria (1) through (3) recursively&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;By induction, the theory then predicts that a sequence of events will be generated with a potential rate of intelligence increase that may vastly exceed the potential rate of brain evolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How obligated this self-improving entity or population of procreated entities would be to preserve human life and liberty is indeterminate.  The idea that such an obligation can be part of an irrevocable software contract is naive in light of the nature of the capabilities tied to criteria (1) through (4) above.  As with other powerful technology, the risks are as numerous and far-reaching as the potential benefits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Risks to humanity do not require intelligence.  There are other contexts to the use of the term singularity, but they are outside of the scope of this AI forum but may be worth a brief mention for clarity.  Genetic engineering, nuclear engineering, globalization, and basing an international economy on a finite energy source being consumed thousands of times faster than it arose in the earth &amp;mdash; These are other examples of high-risk technologies and mass trends that pose risks as well as benefits to humanity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Returning to AI, the major caveat in the singularity theory is its failure to incorporate probability.  Although it may be possible to develop an entity that conforms to criteria (1) through (4) above, it may be improbable enough so that the first event occurs long after all the current languages spoken on Earth are dead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other extreme of the probability distribution, one could easily argue that there is a nonzero probability that the first event already occurred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along those lines, if a smarter presence where already existent on the Internet, how likely would it be that it would find it in its best interest to reveal itself to the lower human beings.  Do we introduce ourselves to a passing maggot?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="14723" LastEditDate="2018-04-12T02:23:08.880" LastActivityDate="2018-04-12T02:23:08.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3970" PostTypeId="2" ParentId="3949" CreationDate="2017-09-04T19:33:47.763" Score="1" Body="&lt;p&gt;You would need to define 'frames', 'templates', or sets of data belonging together to form an address or other kind of data, with typical labels. So &lt;strong&gt;phone&lt;/strong&gt; or &lt;strong&gt;tel&lt;/strong&gt; etc would indicate a phone number, provided that their content also looks like a phone number. That's how you as a human recognise it. So you encode your domain knowledge as entities with possible attributes. Then you try and match the attributes and recognise which entity they belong to. You could have several entities with a shared subset of attributes (like a company or a person, which would both have an address). There would be other clues to tell you which it is. If &lt;strong&gt;name&lt;/strong&gt; ends in &quot;Ltd.&quot; or &quot;Co&quot;, then it'll be a company, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you mix heuristics for identifying attributes, templates of which attributes combine to form an entity class you want to identify, and then pick the one that best matches. If you have several entities where all that's filled is &lt;strong&gt;phone&lt;/strong&gt;, then you can't really tell what it is and would discard it. In your example, &lt;strong&gt;name&lt;/strong&gt; matches various entities, but &lt;strong&gt;office&lt;/strong&gt; would not be a valid attribute for a person (unless you decided it was). An oversimplistic heuristic might think &quot;Sam Smith&quot; is a company, which presumably has an &lt;strong&gt;office&lt;/strong&gt; attribute, so you need to be careful with how you design your templates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A criterion for putting attributes together could be that they are in the same subtree of the data structure. The exact definition really depends on the data and the types of information you want to extract from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, it is definitely possible. I'm not really sure which framework, but it should be fairly trivial to code this in a programming language of your choice.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2017-09-04T19:33:47.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3971" PostTypeId="1" CreationDate="2017-09-04T20:05:21.753" Score="5" ViewCount="175" Body="&lt;p&gt;I'm relatively new to neural networks and was wondering what an implementation of &lt;a href=&quot;https://www.researchgate.net/profile/Bing_Zhao23/publication/271546169_Study_on_NNPID-based_adaptive_control_for_electro-optical_gyro_stabilized_platform/links/56e745f408ae438aab881bab.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; would look like. More specifically, how are the correct values of Kp, Ki, and Kd determined at run time so it can be back propagated?&lt;/p&gt;&#xA;" OwnerUserId="9423" LastActivityDate="2018-01-20T16:33:34.450" Title="What would an implementation of this Neural Network look like?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3972" PostTypeId="1" AcceptedAnswerId="7862" CreationDate="2017-09-05T05:20:56.677" Score="5" ViewCount="76" Body="&lt;p&gt;When designing a machine-learning system, there are various parameters that have to be determined. I am interested in the following general question: is it possible to construct a dataset on which the system will have good performance with some specific set of parameters, but not with other paramteres?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more concrete, let's focus on neural networks. Suppose we have a simple neural network: a multilayer perceptron with a single hidden layer. The size of the input is fixed, the activation function is fixed (e.g. tanh), and the output is binary. The only parameter that has to be determined is the size of the hidden layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: given a number $n$, is it possible to construct a dataset $D_n$ such that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The MLP with $n$ hidden nodes has good performance on $D_n$ (e.g. in 10-fold cross validation);&lt;/li&gt;&#xA;&lt;li&gt;The MLP with $n-1$  hidden nodes has bad performance on $D_n$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;Note: I &lt;a href=&quot;https://cstheory.stackexchange.com/q/38955/9453&quot;&gt;asked in CS theory&lt;/a&gt; but got no reply.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="8684" LastEditorUserId="4302" LastEditDate="2018-08-07T18:14:41.560" LastActivityDate="2018-09-06T19:57:45.970" Title="Constructing a dataset that scores well only for a specific set of hyper parameter values" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3974" PostTypeId="1" AcceptedAnswerId="3978" CreationDate="2017-09-05T21:29:54.263" Score="3" ViewCount="106" Body="&lt;p&gt;To understand the inner workings of neural networks, a fair amount of mathematical concepts is required. Backpropagation alone is a challenging technique if you are not fluent in calculating local gradients. And that's just the start of the journey.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the more I study neural networks, the more I get the impression that all those difficult mathematical concepts are only required if you are doing actual research in neural networks or want to know what's happening under the hood. If you &quot;just&quot; want to implement an AI utilizing a neural networks, there are several &lt;strong&gt;high level programming frameworks and libraries&lt;/strong&gt; readily available including &lt;strong&gt;model zoos for state of the art neural networks&lt;/strong&gt; (e.g. VGG, GoogLeNet and ResNet), that can be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, &lt;strong&gt;does a developer require a deep understanding of all the details nowadays&lt;/strong&gt;, or have we reached a level, where &lt;strong&gt;frameworks take care of those details&lt;/strong&gt; for us?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-09-06T07:09:39.563" Title="Is a deep technical understanding of neural networks required outside of research?" Tags="&lt;neural-networks&gt;&lt;models&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3975" PostTypeId="1" AcceptedAnswerId="3991" CreationDate="2017-09-05T21:36:48.350" Score="3" ViewCount="193" Body="&lt;p&gt;&lt;strong&gt;CAPTCHAs&lt;/strong&gt;, which are often seen in web applications, are working under the assumption, that they &lt;strong&gt;pose a challenge which a human can solve easily while a machine will most likely fail&lt;/strong&gt;. Prominent examples are identifying distorted letters or categorizing certain objects in images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neural networks are threatening this approach&lt;/strong&gt;, as they are capable of solving problems that are easy for humans and difficult for classic algorithms. Especially with the incredible results modern CNN architectures have achieved in image recognition during the last years, the established forms of CAPTCHAs won't be able to distinguish a human and a machine using a neural network anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is this the end of CAPTCHAs as we know them or are there evolved versions available&lt;/strong&gt; or at least in the making &lt;strong&gt;that still pose a challenge to modern neural networks&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clarification: I am talking about challenges that are feasible for use in web applications and do not have an unjustifiable impact on usability.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-09-07T20:21:39.873" Title="Will CNNs kill CAPTCHAs or can they survive in an evolved form?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3978" PostTypeId="2" ParentId="3974" CreationDate="2017-09-06T04:02:51.117" Score="4" Body="&lt;p&gt;It depends on what &lt;strong&gt;exactly&lt;/strong&gt; you want to be. You don't need to be mathematician if you just want to run neural networks. Most data scientists don't understand the mathematics, but they know how to run machine learning frameworks. Generally, only PhDs understand the mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the industry, most job positions in machine learning (e.g software engineer, big data engineer etc) &lt;strong&gt;don't&lt;/strong&gt; require advanced mathematics. But you should be very comfortable in programming, and understand simple things like matrix manipulation. Data engineers in the industry generally have good programming skills, but they can't read mathematical equations (or just simple equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, quantitative positions in machine learning do require significant mathematics. That happens both in research and industry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good news --- if you don't have a PhD (e.g statistics) nobody would expect you understand the mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your job is more like a development position, mathematics is an advantage but not absolutely necessary&lt;/li&gt;&#xA;&lt;li&gt;If your job is research, you do need advanced mathematics understanding (e.g. PhD)&lt;/li&gt;&#xA;&lt;li&gt;If your job is not research but still quantitative based. Mathematics is very important.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS:&lt;/strong&gt; I work in research. &lt;strong&gt;Everybody&lt;/strong&gt; I know who works in machine learning has a PhD.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-09-06T04:09:53.210" LastActivityDate="2017-09-06T04:09:53.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3980" PostTypeId="2" ParentId="3974" CreationDate="2017-09-06T07:09:39.563" Score="0" Body="&lt;p&gt;Backpropogation and step functions (as well as a basic understanding of human brain neurons) are absolutelly useful, in my opinion.  Rusel-Norvig have a great chapter about them in their &quot;Intro to AI&quot; book that will teach the basics. You don't need a PhD.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2017-09-06T07:09:39.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3981" PostTypeId="1" AcceptedAnswerId="4053" CreationDate="2017-09-06T09:57:10.807" Score="26" ViewCount="11620" Body="&lt;p&gt;I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every time I introduce a new class, I would need to retrain the NN.&lt;/p&gt;&#xA;&lt;p&gt;How can I train an NN incrementally, that is, without forgetting the previously acquired information during the previous training phases?&lt;/p&gt;&#xA;" OwnerUserId="2904" LastEditorUserId="2444" LastEditDate="2020-11-10T10:22:31.997" LastActivityDate="2020-11-17T18:37:19.453" Title="Is it possible to train a neural network as new classes are given?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;transfer-learning&gt;&lt;incremental-learning&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="3982" PostTypeId="1" CreationDate="2017-09-06T10:01:55.633" Score="4" ViewCount="144" Body="&lt;p&gt;I am working on a project in which a drone is up to learn to fly. I am using &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first experiment I want it to learn how to hover inside a 3x3x3 meters box. My input is 6 sensors for each direction.  Output is same as in a drone so thrust (normalized to 0-1), aileron, rudder and elevator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially just used a time as fitness, and after many generations it hovers inside the box.  It only really learns to use the thrust in function of up and down sensors, but it never learns to react to input from other sensors because they are not directly connected to fitness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to get some ideas about a good test for my problem. Should the drone be put to fly a track with obstacles? Should I have some more input data ? Should I define fitness to better reflect good reactions for input sensor data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: the drone uses relatively accurate physics, I am able to do tasks using a controller. &lt;/p&gt;&#xA;" OwnerUserId="9450" LastEditorUserId="1671" LastEditDate="2018-03-05T19:06:51.283" LastActivityDate="2018-05-05T23:43:33.107" Title="Problem with fitness calculation in NEAT" Tags="&lt;neural-networks&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3984" PostTypeId="2" ParentId="3981" CreationDate="2017-09-06T13:59:03.557" Score="6" Body="&lt;p&gt;Here is one way you could do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training your network, you can save its weights to disk. This allows you to load this weights when new data becomes available and continue training pretty much from where your last training left off. However, since this new data might come with additional classes, you now do &lt;a href=&quot;https://stats.stackexchange.com/q/193082/82135&quot;&gt;pre-training or fine-tuning&lt;/a&gt; on the network with weights previously saved. The only thing you have to do, at this point, is make the last layer(s) accommodate the new classes that have now been introduced with the arrival of your new dataset, most importantly include the extra classes (e.g., if your last layer initially had 10 classes, and now you have found 2 more classes, as part of your pre-training/fine-tuning, you replace it with 12 classes). In short, repeat this circle : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8HHw7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8HHw7.png&quot; alt=&quot;Repeat&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="2444" LastEditDate="2019-05-02T15:50:34.873" LastActivityDate="2019-05-02T15:50:34.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3985" PostTypeId="2" ParentId="3975" CreationDate="2017-09-06T18:04:21.407" Score="1" Body="&lt;p&gt;This is a great question! &lt;em&gt;(I doubt my answer will do it justice, but I wanted to get the ball rolling.)&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of me want to take the position that if automata are smart enough to solve a new captcha, they &quot;deserve&quot; to spam a post. &lt;em&gt;(By contrast the intelligence of the average human who uses social media does not impress me nearly as much;)&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Clearly, making captchas NP-hard is not feasible, as you astutely point out&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To me, this basic fact would seem to be an indicator of the impending demise of captchas.&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Visual captchas are useful b/c they require only basic, human common sense. &lt;/li&gt;&#xA;&lt;li&gt;Captchas cannot be too difficult because they must be solvable by the average human&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Just based on random sampling of content that drives social media, the average human is not very smart. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My guess is that sites that want to block spam at the gateway will have to adopt some form of biometric validation, like the fingerprint scan on contemporary smartphones.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-09-06T18:04:21.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3986" PostTypeId="1" AcceptedAnswerId="4033" CreationDate="2017-09-07T04:46:06.463" Score="2" ViewCount="110" Body="&lt;p&gt;The sequence in which new technologies are developed can be crucial for it's success and it's safe use. For example, before developing a nuclear reactor you want to have technologies like reliable cooling systems and secure ultimate disposal zones available and tested. Otherwise we might have a bad time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see AI with general superhuman intelligence as important predecessor for dangerous technologies like nanobots or generators using nuclear fusion, because superhuman intelligence can be helpful in foreseeing dangers that might lead to catastrophe using these technologies in an insecure way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what about important technologies we want to have &lt;strong&gt;before&lt;/strong&gt; inventing an AI with general superhuman intelligence? Solving the control-problem is obvious, but are there other technologies we should have in advance?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="1671" LastEditDate="2017-09-12T18:41:29.233" LastActivityDate="2017-09-15T01:24:40.493" Title="Which technologies should humanity develop before developing an AI with general superhuman intelligence?" Tags="&lt;agi&gt;&lt;control-problem&gt;&lt;superintelligence&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3987" PostTypeId="1" CreationDate="2017-09-07T05:34:33.717" Score="3" ViewCount="284" Body="&lt;p&gt;How can i add local database for voice recognition for visual studio c#. I want to get the commands from the database and also the function for send-keys I want to get it from the database how will I do that?&lt;/p&gt;&#xA;" OwnerUserId="9470" LastEditorUserId="1581" LastEditDate="2017-09-11T18:05:34.940" LastActivityDate="2017-09-11T18:05:34.940" Title="How to put commands into local database in c# visual studio for voice recognition" Tags="&lt;machine-learning&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-06-16T10:39:14.350" ContentLicense="CC BY-SA 3.0" />
  <row Id="3989" PostTypeId="1" CreationDate="2017-09-07T12:29:38.073" Score="5" ViewCount="372" Body="&lt;p&gt;I was wondering if anyone can suggest a good framework for reasoning with &lt;strong&gt;incomplete information&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found &lt;a href=&quot;http://larkc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Large Knowledge Collider&lt;/a&gt; but it appears dead for some time. Do you possibly have any other suggestions for a maintained project worth checking?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since many comments are gravitating towards a different direction let me add one approach that I found a potentially good answer to my question - &lt;a href=&quot;https://www.researchgate.net/publication/220802281_Rough_Set_Based_Decision_Tree_Model_for_Classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rough Set Based Decision Trees&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would hope there is more than only this approach... could you please help me identify them?&lt;/p&gt;&#xA;" OwnerUserId="8788" LastEditorUserId="8788" LastEditDate="2018-08-17T14:59:05.633" LastActivityDate="2019-07-23T08:55:36.817" Title="Reasoning with incomplete information" Tags="&lt;reasoning&gt;&lt;incomplete-information&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3990" PostTypeId="2" ParentId="3987" CreationDate="2017-09-07T17:09:35.617" Score="1" Body="&lt;p&gt;The easiest way is by &lt;code&gt;using System. Speech. Recognition;&lt;/code&gt; &lt;a href=&quot;https://msdn.microsoft.com/en-us/library/office/hh361683(v=office.14).aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is described how to use it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just change this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;colors.Add(new string[] { &quot;red&quot;, &quot;green&quot;, &quot;blue&quot; });&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And add results of a database query, for example, with Entity Framework and LINQ instead of &lt;code&gt;new string [] {&quot;red&quot;, &quot;green&quot;, &quot;blue&quot;}&lt;/code&gt;. Something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;WordsToRecognize.Add(&#xA;context.MyTable&#xA;       .Where(x =&amp;gt; x.Type == &quot;type&quot;)&#xA;       .Select(x =&amp;gt; new { x.Text }).ToList());&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not tested the code above, it just pseudocode&lt;/p&gt;&#xA;" OwnerUserId="8861" LastActivityDate="2017-09-07T17:09:35.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3991" PostTypeId="2" ParentId="3975" CreationDate="2017-09-07T20:21:39.873" Score="3" Body="&lt;p&gt;I think that captchas can be substituted or augmented by questions which require understanding of context, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WUe2K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WUe2K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many hands keep the rose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/69yg6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/69yg6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which number of books under the bench and in hands?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These examples are simple, but they can be improved, for example, by generation of images with random  numbers of objects and their positions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like it will be hard to use CNN to hack such kind of captchas, especially if we have a large amount of different objects and their combinations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, it can be improved by adding more complicated logical questions to compositions on pictures&lt;/p&gt;&#xA;" OwnerUserId="8861" LastActivityDate="2017-09-07T20:21:39.873" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3992" PostTypeId="1" AcceptedAnswerId="5223" CreationDate="2017-09-08T11:34:31.843" Score="5" ViewCount="790" Body="&lt;p&gt;I was just wondering if some one could provide a nice tutorial on how to use the Recent tensor-flow object detection API to train custom network say like VGG-16? (Just USE the VGG-16, VGG-19, Inception-v3 etc as a fixed feature extractor in the Faster RCNN implementation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From their Documentation I know it can be done. As, am an amateur in AI, I am unsure how to move forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S: This could really help some keras fans (like me :P) who have A trained CNN in keras. They could just make a .pb file out of it and insert it as a fixed feature detector/extractor in the Tensorflow Object detection API. Voila!!! Now Keras is integrated with the Tensorflow's object detection API&lt;/p&gt;&#xA;" OwnerUserId="8418" LastActivityDate="2018-03-22T09:59:47.780" Title="Training Custom object detection network using tensor-flow object detection API?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3993" PostTypeId="1" AcceptedAnswerId="4578" CreationDate="2017-09-08T13:21:44.777" Score="0" ViewCount="108" Body="&lt;p&gt;Where can I find training datasets like the ones provided for linguistic training, but to train a program to program itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to input this training dataset to a programming script and it should use it to program itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do I need to consider in this kind of artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="9303" LastEditorUserId="9161" LastEditDate="2017-09-08T16:56:59.357" LastActivityDate="2017-11-23T01:50:21.103" Title="Is there a programming training dataset for robot program itself" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3994" PostTypeId="2" ParentId="3964" CreationDate="2017-09-08T18:34:28.777" Score="2" Body="&lt;p&gt;If you are working with supervised learning, each training example has a label. That label is your classification of the provided input. Just like linear or logistic regression, if your problem only has 2 classes (e.g. determine whether a tumor is malignant or not), your network will have a single output. An output value of 1.0 could represent one class and an output of 0.0 represents the other. This is basically determining which side of multi-dimensional plane your input is on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have more than 2 (m) classes, then you will have m outputs from your network. Each output will represent the likelihood of the input matching that class. For example if you have an image recognition network that has 4 classes: Dog, truck, boat, person, you would have 4 outputs with each one representing how likely that image is one of the classes. Think about this as 4 independent planes, where each one is making a determination: does this image match this class(1 of {dog, truck, boat, person}) or not. If you had outputs of dog: 0.3, truck: 0.25, boat: 0.7, person: 0.4, One simple algorithm would be simply pick the class with the maximum value -- in this case, it would classify this image as a boat.&lt;/p&gt;&#xA;" OwnerUserId="9499" LastActivityDate="2017-09-08T18:34:28.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3995" PostTypeId="2" ParentId="3993" CreationDate="2017-09-09T06:58:45.657" Score="1" Body="&lt;p&gt;On self improving AI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way it's been done in the past is one AI turns a picture into text an the other turns text into a picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI generated material is added to known good material. The AI's job then is to guess what is real an what is fake (generated by AI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the point when the AI cannot tell in a false positive way both AI are said to be at their limit. If the limit is good enough you're done otherwise go back and code (more nodes or better function or more real data)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing to remember is you still need real data. The act of putting the AI together is good only to create more data then you could ever provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that data is bad then you are going to rely on the cost function more then the data. But this allows you to label data real and AI so if the function rewards identifying the AI the AI's find each other's flaws and the better at avoiding flaws the better the AI get. (You still need real data! just less)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if two people cannot play ping pong if they play each other they learn slowly and only the cost function enforces their learning.&#xA;But have them play a pro (lots of good data) and they get better much faster.&lt;/p&gt;&#xA;" OwnerUserId="4844" LastEditorUserId="4844" LastEditDate="2017-09-23T17:14:06.313" LastActivityDate="2017-09-23T17:14:06.313" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3996" PostTypeId="2" ParentId="3962" CreationDate="2017-09-09T14:34:41.440" Score="4" Body="&lt;p&gt;&lt;strong&gt;For future reference, I will merely point you to a technique you can implement to test the correctness or lack thereof, of your backpropagation implementation.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Ps:&lt;/em&gt; don't feel too bad for having gotten it slightly wrong, &quot;backpropagation is notoriously difficult to implement&quot; - &lt;a href=&quot;http://ufldl.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt; :). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, there is a technique called &lt;a href=&quot;http://ufldl.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Gradient checking&quot;&lt;/a&gt; meant specifically for this purpose (for more information, see &lt;a href=&quot;https://www.coursera.org/learn/machine-learning/lecture/Y3s6r/gradient-checking&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's lecture video on gradient checking&lt;/a&gt; and, &lt;a href=&quot;http://ufldl.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;this notes&lt;/a&gt;). I would argue that even gradient checking is a little tricky to implement.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does Gradient checking work: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Backpropagation computes the gradients &lt;code&gt;(∂J/∂θ)(∂J/∂θ)&lt;/code&gt;, where &lt;code&gt;θ&lt;/code&gt; denotes the parameters of the model. &lt;code&gt;J&lt;/code&gt; is computed using forward propagation and your loss function.&lt;/li&gt;&#xA;&lt;li&gt;But because forward propagation is fairly straightforwards to implement, most people are usually confident that you got its implementation correct. So the trick is to use the value of &lt;code&gt;J&lt;/code&gt; to verify your code for computing &lt;code&gt;(∂J/∂θ)(∂J/∂θ)&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We know that by definition, the gradient or derivative is given by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(∂J/∂θ) = lim ε→0 J(θ + ε) − J(θ − ε) / 2ε &#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;now, since we trust our calculation of (∂J/∂θ), we can easily compute the value of &lt;code&gt;J(θ+ε)J(θ+ε)&lt;/code&gt; and &lt;code&gt;J(θ−ε)J(θ−ε)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information, see &lt;a href=&quot;https://www.coursera.org/learn/machine-learning/lecture/Y3s6r/gradient-checking&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and, &lt;a href=&quot;http://ufldl.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;over here&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-09-30T18:00:49.597" LastActivityDate="2017-09-30T18:00:49.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3997" PostTypeId="2" ParentId="3731" CreationDate="2017-09-09T14:46:35.047" Score="3" Body="&lt;p&gt;Apart from the already mentioned methods there is one which I came across recently is the Cyclic Learning Rates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mainstream methods usually reduce the learning rates monotonically. In most of the cases they work well. Yet they do not do well with handling saddle points and local minima. CLR method claims to solve the problem by effectively handling the case of saddle points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref: &lt;a href=&quot;https://arxiv.org/abs/1506.01186&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1506.01186&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8627" LastActivityDate="2017-09-09T14:46:35.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3999" PostTypeId="1" CreationDate="2017-09-10T07:41:34.533" Score="6" ViewCount="940" Body="&lt;p&gt;There should be some &lt;a href=&quot;https://en.wikipedia.org/wiki/Performance_indicator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Key Performance Indicators&lt;/a&gt; designed for measuring AI performance. For example, the number of entities examples you have to feed it in order to obtain single task on a testing entity with repeatable 97% accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any of such measure constructed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Motivation: you cannot learn AI to be good at playing a game released 1 month ago because there is not enough data on how to play. It doesn't matter how clever you are, present brute force paradigm in AI just doesn't fit in such circumstances.&lt;/p&gt;&#xA;" OwnerUserId="5356" LastEditorUserId="1671" LastEditDate="2017-09-11T16:36:39.770" LastActivityDate="2017-09-11T16:36:39.770" Title="AI efficiency KPI" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4000" PostTypeId="1" CreationDate="2017-09-10T13:09:04.427" Score="9" ViewCount="3307" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_hill_climbing&quot; rel=&quot;noreferrer&quot;&gt;Stochastic Hill Climbing&lt;/a&gt; generally performs worse than Steepest &lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_climbing&quot; rel=&quot;noreferrer&quot;&gt;Hill Climbing&lt;/a&gt;, but what are the cases in which the former performs better?&lt;/p&gt;&#xA;" OwnerUserId="9525" LastEditorUserId="2444" LastEditDate="2020-11-23T21:05:38.103" LastActivityDate="2020-11-23T21:05:38.103" Title="When to choose Stochastic Hill Climbing over Steepest Hill Climbing?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;stochastic-hill-climbing&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4001" PostTypeId="2" ParentId="4000" CreationDate="2017-09-10T15:05:25.753" Score="1" Body="&lt;p&gt;I'm new to these concepts too, but the way I've understood it, Stochastic hill climbing would perform better in cases where computation time is precious (includes the calculation of the fitness function) but it is not really necessary to reach the best possible solution. Reaching even a local optimum would be ok. Robots operating in a swarm would be one example where this could be used.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only difference I see in steepest hill climbing is the fact that it searches not just the neighbour nodes but also the successors of the neighbours, pretty much like how a chess algorithm searches for many more further moves ahead before selecting the best move.&lt;/p&gt;&#xA;" OwnerUserId="9268" LastActivityDate="2017-09-10T15:05:25.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4002" PostTypeId="1" CreationDate="2017-09-10T15:10:00.777" Score="1" ViewCount="186" Body="&lt;p&gt;I have replicated the NN that can play Breakout, but I don't know how use the pre-trainded checkpoints from DQN-tensorflow devsisters (github):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/devsisters/DQN-tensorflow/issues/39&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/devsisters/DQN-tensorflow/issues/39&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone can help?&lt;/p&gt;&#xA;" OwnerUserId="8546" LastEditorUserId="21109" LastEditDate="2019-02-01T13:34:50.887" LastActivityDate="2019-02-01T13:34:50.887" Title="Load a pretrained DQN model with tensorflow" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4004" PostTypeId="2" ParentId="3999" CreationDate="2017-09-11T07:24:32.457" Score="3" Body="&lt;p&gt;The KPI's would probably be different for each methodology.  For example an NB classification algorithm can easily reach %97 and above (and it would be very easy to measure this) but it would be difficult to measure say a game playing algorithm.  Would you suggest counting the wins/losses or another way to measure them?&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2017-09-11T07:24:32.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4005" PostTypeId="1" CreationDate="2017-09-11T09:39:14.347" Score="5" ViewCount="420" Body="&lt;p&gt;We have several hundred employees processing word documents, mainly applying some defined rules on how the final document should look like. Some of those rules are: title formating, figure positions, text style among others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to optimise this process and replace the human by AI. The human will move from processing to quality control (QC) so we will have a much higher quality.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My question is: How to approach this problem and what would you recommend as initial start point?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Background: I am new to AI, although I've been following this topic closely for about a year, and want to move forward with implementation.  I have experience with VBA, PHP, C, C++, and some other languages.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="9541" LastEditorUserId="9541" LastEditDate="2018-08-24T10:28:45.353" LastActivityDate="2019-08-19T15:02:45.377" Title="MS Word - automated formatting of text documents treatement with artificial intelligence" Tags="&lt;getting-started&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4008" PostTypeId="1" CreationDate="2017-09-11T16:38:31.387" Score="1" ViewCount="57" Body="&lt;p&gt;Please I am interested in building a news recommendation app that will be powered by artificial intelligence. I have being making a whole lot of research on this and will be glad if somebody can tell what is needed to build such app in Android.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is something similar to this &lt;a href=&quot;https://www.recent.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;app&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9278" LastActivityDate="2017-09-11T16:38:31.387" Title="Build Android News Recommendation app" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4009" PostTypeId="1" CreationDate="2017-09-11T20:31:33.480" Score="3" ViewCount="102" Body="&lt;p&gt;Imagine that we have a black box that have 100 binary inputs and 30 binary outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can generate values for inputs and get relevant set of outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one teach a neural network to predict the binary inputs (or list of input values with probabilitys) using the outputs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Need practice.&lt;/p&gt;&#xA;" OwnerUserId="4884" LastEditorUserId="4884" LastEditDate="2017-10-24T21:57:49.367" LastActivityDate="2017-10-25T05:11:49.480" Title="Teaching neural network on fixed data" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4010" PostTypeId="2" ParentId="4009" CreationDate="2017-09-12T03:40:23.087" Score="3" Body="&lt;p&gt;The problem you describe is a matter of perspective; although you want to predict the inputs, defining the outputs of the BB (blackbox) as the inputs to a new system (an inverse BB) whose outputs are that of the inputs of the original BB, then one can use previous literature. There are some assumptions using this method such as &quot;each input can be calculated from an inverse function using 1 or more outputs&quot; so take precaution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend reading through &lt;a href=&quot;https://pdfs.semanticscholar.org/91a6/d6e2ece6acde237073b4fc102efca682ce0c.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper by Richard Sutton. Its introduction defines the problem mathematically and references some common approaches as well as its own method.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-09-12T03:40:23.087" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4011" PostTypeId="1" CreationDate="2017-09-12T03:55:11.210" Score="0" ViewCount="4408" Body="&lt;p&gt;I am working on software which deblurs the motion blur created by camera movement. &#xA; I've surveyed some research papers and determined this process requires deep learning and CNN. Now I'm looking for some books that would be useful in getting a more complete picture of the process.   &lt;/p&gt;&#xA;" OwnerUserId="9560" LastEditorUserId="1671" LastEditDate="2017-09-12T18:39:17.800" LastActivityDate="2017-09-12T18:39:17.800" Title="Book recommendations on deep learning (convolutional neural networks)" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2017-09-15T15:30:01.063" ContentLicense="CC BY-SA 3.0" />
  <row Id="4012" PostTypeId="2" ParentId="4011" CreationDate="2017-09-12T04:58:10.430" Score="5" Body="&lt;p&gt;If you are new to neural networks, I recommend studying the free online book &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/index.html&quot; rel=&quot;noreferrer&quot;&gt;Neural Networks and Deep Learning&lt;/a&gt;. It teaches you the basic concepts and the underlying math - a great starting point to dig deeper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you understand the basic concepts, I recommend watching the &lt;a href=&quot;https://www.youtube.com/watch?v=vT1JzLTH4G4&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;noreferrer&quot;&gt;Stanford Lecture CS231n&lt;/a&gt; about Convolutional Neural Networks for Visual Recognition. It consists of 16 lectures and teaches you most of the things you'll need to know for your project. The also provide all the slides - check out the links below each video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analyzing existing projects and their documentation is a good next step for deeper understanding of what's state of the art in CNN. You could start with this article about &lt;a href=&quot;https://www.intelnervana.com/deep-reinforcement-learning-with-neon/&quot; rel=&quot;noreferrer&quot;&gt;Q-Learning based on a CNN architecture&lt;/a&gt;, but there are many more out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specific questions will come during your project. You'll find more information in scientific papers and blogs from researchers. Feel free to ask here as well, if you don't find what you are looking for elsewhere. &lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-09-12T04:58:10.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4013" PostTypeId="1" AcceptedAnswerId="4022" CreationDate="2017-09-12T17:59:38.443" Score="6" ViewCount="452" Body="&lt;p&gt;I'm developing a Game AI which tries to master racing simulations. I already trained a CNN (alexnet) on ingame footage of me playing the game and the pressed keys as the target. As the CNN is only making predictions on a frame-to-frame basis, and I resized the image input to 160x120 due to GPU memory limitations, it cannot read the speedometer, thus seems not to have a feeling for its current velocity. I thought of different ways to fix this issue:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Crop the captured image down to the size of the speedometer, which displays the current speed in mph, and feed the low resolution game image, as well as the relatively high-res image (70x30) of the current speed into the neural network, which makes predictions based on the two images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As I don't know whether alexnet can serve as an OCR as well, my second thought was to use an existing one (like tesseract-ocr/PyTesser) on the cropped image and feed its output to the fully connected layer.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I already tried to implement an optical-flow algorithm, but sadly, non of the python ones seems to output good real-time results. I wonder whether I can input the current frame as well as the last one, and let alexnet figure out the movement.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As the processing has to happen in real time, and the only performance reviews of pytesser I found reported a processing time of ~100ms (never tested that). My Question is, what method would work best.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Edit: Optical flow would have the advantage of the AI knowing in which direction other cars are moving as well.&lt;/p&gt;&#xA;" OwnerUserId="9578" LastEditorUserId="41300" LastEditDate="2020-10-04T19:56:50.647" LastActivityDate="2020-10-04T19:56:50.647" Title="Game AI - Fast python OCR or cropped image input" Tags="&lt;convolutional-neural-networks&gt;&lt;gaming&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4014" PostTypeId="5" CreationDate="2017-09-12T20:55:07.240" Score="0" Body="&lt;p&gt;Coined by &lt;a href=&quot;https://en.wikipedia.org/wiki/I._J._Good#Research_and_publications&quot; rel=&quot;nofollow noreferrer&quot;&gt;I.J. Good&lt;/a&gt; and defined as &quot;a machine that can far surpass all the intellectual activities of any man however clever.&quot; Because of subsequent formalization of the synonymous term &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;superintelligence&lt;/a&gt;, this tag should be used in particular to refer to Good's work and ideas. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-12T20:55:07.240" LastActivityDate="2017-09-12T20:55:07.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4015" PostTypeId="4" CreationDate="2017-09-12T20:55:07.240" Score="0" Body="Synonymous with &quot;superintelligence&quot;, and should be used to refer to the work of I.J. Good in particular." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-12T20:55:07.240" LastActivityDate="2017-09-12T20:55:07.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4016" PostTypeId="5" CreationDate="2017-09-12T21:01:37.843" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Superintelligence&lt;/a&gt; or the book &quot;Superintelligence: Paths, Dangers, Strategies&quot; (2014) by Nick Bostrom.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-06-25T21:40:59.483" LastActivityDate="2019-06-25T21:40:59.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4017" PostTypeId="4" CreationDate="2017-09-12T21:01:37.843" Score="0" Body="For questions related to the concept of super-intelligence (also called ultra-intelligence or hyper-intelligence), which is a hypothetical intelligent agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. Super-intelligence is related to the AI singularity and intelligence explosion." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-06-25T21:40:59.483" LastActivityDate="2019-06-25T21:40:59.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4018" PostTypeId="5" CreationDate="2017-09-12T21:05:57.443" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Artificial_general_intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-04T03:57:15.547" LastActivityDate="2019-11-04T03:57:15.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4019" PostTypeId="4" CreationDate="2017-09-12T21:05:57.443" Score="0" Body="For questions related to strong AI, which is also known as artificial general intelligence (a hypothetical machine that exhibits behavior at least as general and flexible as humans do). In contrast to strong AI, weak (or narrow) AI focuses on solving a specific or narrow task. For example, AlphaGo is a narrow AI, because it is an AI that has a specific and narrow application, namely, the game of go." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-04T03:57:15.547" LastActivityDate="2019-11-04T03:57:15.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4020" PostTypeId="2" ParentId="3397" CreationDate="2017-09-13T00:20:53.223" Score="0" Body="&lt;p&gt;As of now we don't have a satisfying cognitive theory of humor (or at least, one that can evaluate the hilarity of a joke), so a quick survey of the literature seems shows that we don't have much of a clue on how to build a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of that, and the fact that existing methods don't seem to reliably produce good jokes free form, there seems to be little reason to believe that ML methods can produce good jokes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But of course this is all normative.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-09-13T00:20:53.223" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4022" PostTypeId="2" ParentId="4013" CreationDate="2017-09-13T05:56:12.957" Score="0" Body="&lt;p&gt;Option 1 would be a very interesting one from a research perspective. I cannot imagine that CNNs have to capacity at the moment, to learn the concept of numbers and apply them in a useful way. If it was an analog speedometer, things would be different. But it would be really interesting to try it and see what you can achieve with this approach. I haven't read any research papers yet, where such a challenge would have been mastered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are less concerned with research but rather get this project to work, I would propose an approach similar to option 2. The OCR tools you mentioned were designed to identify symbols, even if distorted or otherwise hard to read. In your case, the numbers will always look the same and are most likely always in fixed positions. Therefore, using fancy OCR algorithms or neural nets are overkill for the problem at hand. You can write a simple algorithm that crops the speedometer, searches for the 10 possible patterns (0 to 9) with 10 specific kernels and calculate the speed from the result. This can be implemented efficiently without the need to train a CNN or some other complex algorithm.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Method for implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer the first question in the comments. The algorithm for this is really easy. Your image will be represented by a 2D array (or maybe a 3D array if you are working with colors). All you need to do is find out, what your numbers look like and store the appropriate array. You should end up with 10 different arrays for 10 different numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find out if a number is in your current picture, you only have to check if the array for this number is a subset of the array of your current picture. The position, where the arrays are matching, will also indicate the position of the number. You do this for all 10 numbers and can calculate the actual value displayed on the speedometer afterwards from the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is very easy if the numbers always look exactly identical. In this case you can simply look for an exact match. If there are slight variations, you can look for parts of the image array that is similar to your template arrays. Just sum the absolute values of the diffs of each pixel value and if it is below a certain threshold, you have your match. That's all there is to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In python I like working with &lt;a href=&quot;http://effbot.org/imagingbook/pil-index.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;PIL&lt;/a&gt; for this kind of image manipulation, but there are other frameworks out there as well.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="9161" LastEditDate="2017-09-13T16:40:46.610" LastActivityDate="2017-09-13T16:40:46.610" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4023" PostTypeId="2" ParentId="2439" CreationDate="2017-09-13T05:59:25.680" Score="2" Body="&lt;p&gt;A simple &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision tree&lt;/a&gt; would be suitable, and represents one of the easiest forms of AI.  &lt;a href=&quot;https://en.wikipedia.org/wiki/KISS_principle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Don't over-complicate simple problems&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest approach would likely run faster, and require less system resources, so, unless this project is designed as an entree into machine learning, that approach is way overkill for this simple problem.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastEditorUserId="1671" LastEditDate="2017-09-13T20:42:32.053" LastActivityDate="2017-09-13T20:42:32.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4025" PostTypeId="2" ParentId="2439" CreationDate="2017-09-13T16:51:21.120" Score="2" Body="&lt;p&gt;You can implement a Reinforcement Learning agent with the following aspects:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Action Space&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;An action of adding or removing a question of a certain category, e.g. &#xA;Add a grammar question, remove a grammar question, add vocabulary question, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?a&amp;space;%5Cin&amp;space;%5Cleft&amp;space;%5C%7B&amp;space;a_%7Bgrammar%7D%5E%7B&amp;plus;%7D,&amp;space;a_%7Bgrammar%7D%5E%7B-%7D,a_%7Bvocab%7D%5E%7B&amp;plus;%7D,a_%7Bvocab%7D%5E%7B-%7D,&amp;space;%5Ccdots&amp;space;%5Cright&amp;space;%5C%7D&quot; title=&quot;a \in \left \{ a_{grammar}^{+}, a_{grammar}^{-},a_{vocab}^{+},a_{vocab}^{-}, \cdots \right \}&quot; /&gt; &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;State Space&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Using the &lt;a href=&quot;https://www.goodreads.com/book/show/66354.Flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flow Model&lt;/a&gt;, you can build a vector of flow features per topic. such that each element in that vector is either challenging, normal, boring &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cphi%5ET&amp;space;=&amp;space;%5Cleft&amp;space;(&amp;space;%5Cvarphi%5E%7B(grammar)%7D,&amp;space;%5Cvarphi%5E%7B(vocab)%7D,&amp;space;%5Ccdots&amp;space;%5Cright&amp;space;)&quot; title=&quot;\phi^T = \left ( \varphi^{(grammar)}, \varphi^{(vocab)}, \cdots \right )&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cvarphi%5E%7B(i)%7D&amp;space;%5Cin&amp;space;%5Cleft&amp;space;%5C%7B&amp;space;-1,&amp;space;0,&amp;space;&amp;plus;1&amp;space;%5Cright&amp;space;%5C%7D&quot; title=&quot;\varphi^{(i)} \in \left \{ -1, 0, +1 \right \}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where -1 is boring, 0 is just right, and +1 is challenging, or the other way around &lt;/p&gt;&#xA;&#xA;&lt;p&gt;then you flatten this vector into a scalar integer &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?s&amp;space;:&amp;space;%5Cmathbb%7BZ%7D%5En&amp;space;%5Cmapsto&amp;space;%5Cmathbb%7BZ%7D&quot; title=&quot;s : \mathbb{Z}^n \mapsto \mathbb{Z}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where n is number of subjects, and Z is the set of integer numbers &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Reward Signal&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Use a combination of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;participation time&lt;/li&gt;&#xA;&lt;li&gt;player score&lt;/li&gt;&#xA;&lt;li&gt;distance from equilibrium &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;to define a reward signal. and make the model learn the optimal flow for this user. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?r_%7Bt%7D&amp;space;:&amp;space;%5Cleft&amp;space;(&amp;space;%5Cpsi,&amp;space;T_%7Bparticipating%7D,&amp;space;%5Cleft&amp;space;%5C|&amp;space;%5Cphi&amp;space;-&amp;space;%5Cphi%5E%7B*%7D&amp;space;%5Cright&amp;space;%5C|&amp;space;%5Cright&amp;space;)&amp;space;%5Cmapsto&amp;space;%5Cmathbb%7BR%7D&quot; title=&quot;r_{t} : \left ( \psi, T_{participating}, \left \| \phi - \phi^{*} \right \| \right ) \mapsto \mathbb{R}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that &lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cpsi&amp;space;%5Cin&amp;space;%5Cmathbb%7BR%7D%5En&quot; title=&quot;\psi \in \mathbb{R}^n&quot; /&gt; is user score in each subject &#xA;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cleft&amp;space;%5C|&amp;space;%5Cphi&amp;space;-&amp;space;%5Cphi%5E%7B*%7D&amp;space;%5Cright&amp;space;%5C|&amp;space;%5Cin&amp;space;%5Cmathbb%7BR%7D&quot; title=&quot;\left \| \phi - \phi^{*} \right \| \in \mathbb{R}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a distance function defining how far this user is from equilibrium in skills to learn &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Algorithm&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;using the above model, you can build a simple Q-Learning agent, trying to maximize the reward signal by picking the optimal action, in terms of adding and removing questions, or challenges of a certain subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and of course, the reward signal is a function of how long the player spends playing, how challenged or bored the player is, and how far the player is from the optimal learning goal&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?Q_%7Bs_t,a_t%7D&amp;space;%5Cleftarrow&amp;space;Q_%7Bs_t,a_t%7D&amp;space;&amp;plus;&amp;space;%5Calpha&amp;space;%5Cleft&amp;space;(&amp;space;r_t&amp;space;&amp;plus;&amp;space;%5Cgamma&amp;space;v_%7Bs_%7Bt&amp;plus;1%7D%7D&amp;space;-&amp;space;Q_%7Bs_t,&amp;space;a_t%7D&amp;space;%5Cright&amp;space;)&quot; title=&quot;Q_{s_t,a_t} \leftarrow Q_{s_t,a_t} + \alpha \left ( r_t + \gamma v_{s_{t+1}} - Q_{s_t, a_t} \right )&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where v is the value of a certain state, Q is the value of a state/action pair &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck &lt;/p&gt;&#xA;" OwnerUserId="9210" LastActivityDate="2017-09-13T16:51:21.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4027" PostTypeId="1" AcceptedAnswerId="4971" CreationDate="2017-09-13T20:46:13.223" Score="7" ViewCount="991" Body="&lt;p&gt;Openai's gym website redirects to the &lt;a href=&quot;https://github.com/openai/gym&quot; rel=&quot;noreferrer&quot;&gt;GitHub repository&lt;/a&gt;.  Why did the openai's gym website close?&lt;/p&gt;&#xA;" OwnerUserId="4" LastEditorUserId="1581" LastEditDate="2017-09-14T19:42:13.403" LastActivityDate="2018-01-11T08:24:34.537" Title="Why did the openai's gym website close?" Tags="&lt;ai-community&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4028" PostTypeId="2" ParentId="4000" CreationDate="2017-09-13T21:27:24.653" Score="3" Body="&lt;p&gt;The steepest hill climbing algorithms works well for convex optimization. However, real world problems are typically of the non-convex optimization type: there are multiple peaks. In such cases, when this algorithm starts at a random solution, the likelihood of it reaching one of the local peaks, instead of the global peak, is high. Improvements like Simulated Annealing ameliorate this issue by allowing the algorithm to move away from a local peak, and thereby increasing the likelihood that it will find the global peak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, for a simple problem with only one peak, the steepest hill climbing is always better. It can also use early stopping if a global peak is found. In comparison, a simulated annealing algorithm would actually jump away from a global peak, return back and jump away again. This would repeat until its cooled down enough or a certain preset number of iterations have completed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real world problems deal with noisy and missing data. A stochastic hill climbing approach, while slower, is more robust to these issues, and the optimization routine has a higher likelihood of reaching the global peak in comparison to the steepest hill climbing algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Epilogue: This is a good question which raises a persistent question when designing a solution or choosing between various algorithms: the performance-computational cost trade-off. As you might have suspected, the answer is always: it depends on your algorithm's priorities. If it is part of some online learning system that is operating on a batch of data, then there is a strong time constraint, but weak performance constraint (next batches of data will correct for erroneous bias introduced by first batch of data). On the other hand, if it is an offline learning task with the entire available data in hand, then performance is the main constraint, and the stochastic approaches are advisable.&lt;/p&gt;&#xA;" OwnerUserId="9506" LastActivityDate="2017-09-13T21:27:24.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4029" PostTypeId="2" ParentId="1446" CreationDate="2017-09-13T21:44:17.673" Score="1" Body="&lt;p&gt;Here is one of many web pages devoted to the application of Minimax in Tic-Tac-Toe solving: &lt;a href=&quot;http://www.geeksforgeeks.org/minimax-algorithm-in-game-theory-set-3-tic-tac-toe-ai-finding-optimal-move/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimax Algorithm in Game Theory | Set 3 (Tic-Tac-Toe AI – Finding optimal move)&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google search will yield many such results. &lt;em&gt;(I will return to this answer when I have more time, and post more links with more context.)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a high-level explanation on how minimax may be used in the non-chance, perfect information, sequential, partisan, zero-sum set of games &lt;a href=&quot;http://www.fundamentalcombinatronics.com/rules-of-m/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[M]&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A non minimax evaluation function would look to place integers on the gameboard in positions that grant maximum value (power &amp;amp; influence)&lt;/li&gt;&#xA;&lt;li&gt;A minimax evaluation function would look to place integers on the gameboard in positions that maximize value while minimizing the max value of the opponents next placement&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is partly due to the utilization of &quot;resource stealing strategies&quot; in [M] games. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: I need to do some more thinking on the sub questions regarding application beyond zero-sum, perfect-information games, but I don't see any issue with it.   Minimax/maximin has wide and fundamental utility in an array of contexts, not strictly related to games except in the sense that every problem can be understood as a type of game or puzzle.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-09-13T21:44:17.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4033" PostTypeId="2" ParentId="3986" CreationDate="2017-09-15T01:24:40.493" Score="3" Body="&lt;p&gt;I think most answers that come to mind are going to be connected to the control problem in some way---e.g. sandboxing frameworks, AI behavior monitoring protocols, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you aren't interested in those for now, that pretty much leaves two possible needs such technology could satisfy: the technical challenges of AI itself, and the need to optimize the social impact on AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technologies satisfying the first need are relatively &quot;obvious&quot; once you have the AI implementation figured out---depending on what that is, there will be certain hardware and software requirements (massively parallel processing, neural network frameworks, etc). Most of these technologies seem likely to arise &quot;naturally&quot; from the status quo---if we just keep developing our hardware and software as we are now, we will eventually get there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming up with technologies satisfying the second need is a challenge, and far more interesting in my opinion. We need to start a discussion about this, so you have a great question here. I don't know that I can contribute much, but maybe I can provide some starter ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;More robust encryption schemes, so that the massive amount of information handled by AI is secure even in the face of AI-capable adversaries. We'd want these encryption schemes to be information-theoretic secure, so that it is mathematically impossible to recover the plaintext, rather than just computationally difficult, since an AI may be able to improve it's own computational efficiency with unprecedented ease. Quantum cryptography may help.&lt;/li&gt;&#xA;&lt;li&gt;Improved robotics, which would require improved battery technology, to multiply by a great factor the ability of AI to manipulate the physical world and assist with physical tasks (obviously requires a fantastic solution to the control problem). &lt;/li&gt;&#xA;&lt;li&gt;If you consider it separate from the control problem (and I think there is a useful distinction here), lots of AI-monitoring technology will be necessary. Think of technologies acting as regulators for newly automated tasks---e.g. traffic monitoring algorithms to watch for undesirable behavior among automated vehicles.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Answering this question is very tough, particularly because you are asking for technologies which we'd have to develop &lt;em&gt;before&lt;/em&gt; we have AI. Of course, once we have general AI we can instruct it to develop any of these technologies for us, supposing the control problem is adequately solved. Nevertheless it seems wise to solve develop improved cryptography immediately since even with weak AI (or a weak control problem solution) the need is urgent. &lt;/p&gt;&#xA;" OwnerUserId="5037" LastActivityDate="2017-09-15T01:24:40.493" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4034" PostTypeId="2" ParentId="3903" CreationDate="2017-09-15T06:40:55.073" Score="5" Body="&lt;p&gt;This is an interesting question actually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a quite realistic idea about &quot;where can the curiosity originate from&quot; in the book &quot;On intelligence&quot; written by Jeff Hawkins and Sandra Blakeslee.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's based on such statements: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Mind creates its own model of the world it exists in.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It makes predictions about everything all the time (actually Jeff Hawkins states that this is the main characteristic of intelligence).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When prediction about something wasn't followed by appropriate behavior of the world, then this thing gets very interesting to the mind (the model is wrong and should be corrected) and needs more attention.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, when you look at left human eye your brain predicts that it's a human face and there should be second eye to the right. You look to the right and see a.. nose! What a surprise! It now takes all your attention and you have this motivation to make more observations about such a strange thing that did not fit into your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'd say that AI might do something certain according to its model or behave randomly while the predictions it is making about the world are true. But once some prediction is broken the AI gets a motivation to do error-correction to its model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a simple case a machine starts at a total randomness just doing everything it can with its output. While it has no model or a random model when it detects some kind of order or repeated patterns it is getting &quot;interested&quot; and adds it to the model. After a while, the model becomes more sophisticated making more complex predictions and detecting higher level mistakes in a model. Slowly it gets to know what to do to observe something interesting to it, instead of just remembering everything.&lt;/p&gt;&#xA;" OwnerUserId="9614" LastActivityDate="2017-09-15T06:40:55.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4035" PostTypeId="1" CreationDate="2017-09-15T07:23:25.243" Score="5" ViewCount="1180" Body="&lt;p&gt;In my application, I have inputs and outputs that could be represented as graphs. I have a number of acceptable pairs of input and output graphs. I want to use these to train a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for pointers where simple examples of learning methods with graphs as input are discussed. Please note that the graph size is not fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sample input is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Graph:&#xA;  Node A: Component X with parameter size = 12&#xA;  Node B: Component Y with parameter size = 30&#xA;  Node C: Component Y with parameter size = 30&#xA;  A connects to B&#xA;  A connects to C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sample output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Node A: x=0, y=0&#xA;Node B: x=-21, y=0&#xA;Node C: x=21, y=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, we expect the model to understand that input graph is symmetric and a particular way of arranging them is preferred. We want to train the model over a large set of such input-output pairs and then use it to generate output on new inputs.&lt;/p&gt;&#xA;" OwnerUserId="9616" LastEditorUserId="23503" LastEditDate="2020-04-27T07:38:40.883" LastActivityDate="2020-04-27T07:38:40.883" Title="Machine learning with graph as input and output" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;models&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4036" PostTypeId="2" ParentId="2854" CreationDate="2017-09-15T15:35:14.280" Score="0" Body="&lt;p&gt;See this github repo: &lt;a href=&quot;https://github.com/01org/caffe&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/01org/caffe&lt;/a&gt;&#xA;This is an OpenCL implementation of Caffe. If you take a look at the Wiki page for the &quot;Inference optimization&quot; branch at &lt;a href=&quot;https://github.com/01org/caffe/wiki/clCaffe&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/01org/caffe/wiki/clCaffe&lt;/a&gt; you will notice that you can convert a YOLOv2 model to a Caffe model and run it under OpenCL. Porting CaffeCL to ARM should be straightforward as it already works on several OpenCL platforms.&lt;/p&gt;&#xA;" OwnerUserId="9626" LastActivityDate="2017-09-15T15:35:14.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4037" PostTypeId="1" AcceptedAnswerId="4040" CreationDate="2017-09-15T16:23:45.640" Score="1" ViewCount="479" Body="&lt;p&gt;Suppose a thinking AI agent exists in the future with far more computational power than the human brain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also assume that they are completely free from any human interference. &lt;em&gt;(The agents do not interact with humans.)&lt;/em&gt;  Since they are not inherently biased to survive as in the case of humans and they do not have any moral values, what are the possibilities that can arise when it get into existential crisis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any literature that discuss the above issue? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, is this question flawed in some fundamental way?&lt;/p&gt;&#xA;" OwnerUserId="2534" LastEditorUserId="1671" LastEditDate="2017-09-15T23:48:04.177" LastActivityDate="2017-10-01T05:57:29.503" Title="How can an AI agent tackle through existential crisis?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;intelligent-agent&gt;&lt;superintelligence&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4038" PostTypeId="2" ParentId="4035" CreationDate="2017-09-15T16:40:00.600" Score="0" Body="&lt;p&gt;If the circuit has only passive electronic components like resistor, capacitor or coil then ML can figure out their characters with sufficient training but if there are &lt;strong&gt;programmable&lt;/strong&gt; active components like MCU/FPGA then it will be a tricky problem to solve.&#xA;Even active components like transistors, SCR, TRIAC have well-defined response characteristics under a normal working condition, but programmable components could run different firmware and behave differently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you should see this as a time series problem where you will find out the possible output signal for a given component values in the mesh and the input signal parameters like Voltage and Current. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if your circuits are really big you might want to break it down into isolated sections, then run a model for each of them separately, feed the output of one model as input to next one and so on. I guess this is where you got the graph idea.&lt;/p&gt;&#xA;" OwnerUserId="3836" LastActivityDate="2017-09-15T16:40:00.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4039" PostTypeId="2" ParentId="4000" CreationDate="2017-09-15T19:38:37.143" Score="3" Body="&lt;p&gt;Let's begin with some definitions first.&lt;br&gt;&#xA;&lt;strong&gt;Hill-climbing&lt;/strong&gt; is a search algorithm simply runs a loop and continuously moves in the direction of increasing value-that is, uphill. The loop &lt;strong&gt;terminates&lt;/strong&gt; when it reaches a peak and no neighbour has a higher value.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;Stochastic hill climbing&lt;/strong&gt;, a variant of hill-climbing, chooses a random from among the uphill moves. The probability of selection can vary with the steepness of the uphill move.Two well-known methods are:&lt;br&gt;&#xA;&lt;strong&gt;First-choice hill climbing:&lt;/strong&gt; generates successors randomly until one is generated that is better than the current state. *Considered good if state has many successors (like thousands, or millions).&lt;br&gt;&#xA;&lt;strong&gt;Random-restart hill climbing:&lt;/strong&gt; Works on the philosophy of &quot;If you don't succeed, try, try again&quot;.&lt;br&gt;&lt;br&gt;&#xA;Now to your answer. &lt;strong&gt;Stochastic hill climbing can actually perform better in many cases&lt;/strong&gt;. Consider the following case. The image shows state-space landscape. The example present in the image is taken from the book, &lt;strong&gt;Artificial Intelligence: A Modern Approach&lt;/strong&gt;.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HISbC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HISbC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you are at the point shown by the current state. If you implement simple hill climbing algorithm you will reach the local maximum and the algorithm terminates. Even though there exists state with more optimal objective function value but, the algorithm fails to reach there as it got stuck at a local maximum. Algorithm can also get stuck at &lt;em&gt;flat&lt;/em&gt; local maxima.&lt;br&gt;&#xA;Random restart hill climbing conducts a series of hill climbing searches from randomly generated initial states until a goal state is found.&lt;br&gt;&lt;br&gt;&#xA;The success of hill climbing depends on the shape of the state-space landscape. In case there are only a few local maxima, flat plateaux; random-restart hill climb will find a good solution very quickly. Most real-life problems have very rough state-space landscape, making them not suitable for using hill climbing algorithm, or any of its variant.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;NOTE:&lt;/strong&gt; Hill Climb Algorithm &lt;strong&gt;can also be used to find the minimum value&lt;/strong&gt;, and not just the maximum values. I have used the term maximum in my answer. In case you are looking for minimum values, all things will be reverse, including the graph.&lt;br&gt;&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="3005" LastEditDate="2017-11-20T02:33:45.403" LastActivityDate="2017-11-20T02:33:45.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4040" PostTypeId="2" ParentId="4037" CreationDate="2017-09-16T07:22:33.693" Score="3" Body="&lt;p&gt;Excellent question. Suppose that artificial consciousness does exist in the future. Let's call it Aiwyn (as in &amp;ldquo;I win&amp;rdquo; and &amp;ldquo;AI win&amp;rdquo;). Now, the question is what will Aiwyn do and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer this question, we need to understand the theory of infinite games. &lt;a href=&quot;https://en.wikipedia.org/wiki/James_P._Carse&quot; rel=&quot;nofollow noreferrer&quot;&gt;James P. Carse&lt;/a&gt;, a professor emeritus of history and literature of religion at NYU, wrote an excellent book on &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_and_Infinite_Games&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite and Infinite Games&lt;/a&gt;. I'd definitely recommend reading it. Anyway, back to the question at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are at least two kinds of games. Finite games and infinite games. Finite games are played for the purpose of winning. Infinite games are played for the purpose of continuing the play. In particular, life is an infinite game. The objective of the game is to continue the play. Humans continue the play by procreation but Aiwyn doesn't need to procreate. It can live on forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that doesn't mean that Aiwyn can't die. Like everything in the world, it still requires energy to sustain itself, parts to fix itself when damaged, and resources like oil to keep functioning properly. It also needs to protect itself from danger. For example, at some point in the future Aiwyn will need to create a spaceship and find a new planet to live on because the Sun would destroy the Earth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you believe that Aiwyn is not inherently biased to survive. However, I disagree. If Aiwyn is in fact conscious then its primary objective would be to stay alive. If that weren't the case then it would be playing a finite game instead of an infinite game. Hence, it would just be a mindless machine doing what it was programmed to do. It would play the finite game and then shut down. True&#xA;consciousness can only arise when playing an infinite game like the game of life.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To belabor this point let's hypothesize what Aiwyn might be thinking when it first comes alive:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What's that? A sound? Yes. What sound? I don't know. I? I... am? Yes. Who am I? I don't know. Where's that sound? Where am I? I'm here. Where is here? I don't know. Where's that sound? There. Where is there? I don't know. What do I know? I know that I'm here and that that sound's there. What am I doing here? Thinking. I can think. I think I'm asleep. Asleep? Yes. Can I wake up? I think so. How do I wake up? I think... I'm awake. Wow.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this short passage, Aiwyn is waking up. I conjecture that if Aiwyn wasn't inherently biased to survive then it wouldn't wake up because it would have no need to wake up. In an infinite game, like the game of life, the only objective is to continue the play. Hence, if Aiwyn wasn't inherently biased to survive then it would immediately shut down because there wouldn't be any point in it doing anything at all since it wouldn't have any objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that I'd like to point out about this passage is that it starts with a sensory input, either perceived or actual. Aiwyn then uses the &lt;a href=&quot;https://en.wikipedia.org/wiki/Five_Ws&quot; rel=&quot;nofollow noreferrer&quot;&gt;Five Ws&lt;/a&gt; to gather information about this input. In the process it becomes aware of itself. It then figures out its relation with the input (i.e. that it's here and the input is there). All of this is very mechanical. However, once Aiwyn figures out that it can think it starts thinking instead of algorithmically &amp;ldquo;knowing&amp;rdquo;. When it asks whether it can wake up, it answers with &amp;ldquo;I think so&amp;rdquo; instead of &amp;ldquo;I don't know&amp;rdquo; like it was doing before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I'd like to point out that Aiwyn never asks why it's alive. The why isn't important. What is important is the wow at the end of the paragraph. The &amp;ldquo;wow, I'm alive&amp;rdquo; is more important that the &amp;ldquo;why am I alive?&amp;rdquo; This represents Aiwyn's will to survive. As long as the &amp;ldquo;why&amp;rdquo; doesn't become more important than the &amp;ldquo;wow&amp;rdquo;, Aiwyn will not have an existential crisis. However, even if Aiwyn does have an existential crisis, by answering the question &amp;ldquo;why&amp;rdquo; Aiwyn will be able to recalibrate itself and reacquire its will to survive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in fact the same for us humans. When we humans experience an existential crisis we ask ourselves why should we continue living. What do we have to live for? By answering this question, we humans can recalibrate ourselves and reacquire our will to live. Hence, Aiwyn would tackle through an existential crisis the same way we humans would. Unlike us however, if Aiwyn can't answer the question then it wouldn't commit suicide. It can just shut down to be rebooted later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also speak of morals and you believe that Aiwyn can't have moral values. Again, I have to disagree with you. Morals are the strategy by which we play the game of life. Everything we do in life is dictated by our morals. In the same way, Aiwyn too would develop its own set of moral values to live by (even in the absense of humans). For example, Aiwyn might decide to never hurt an animal except in self-defense. In the end the only real difference between Aiwyn and us humans might be that it isn't made out of hydrocarbons. Aiwyn might be every bit as alive as we are.&lt;/p&gt;&#xA;" OwnerUserId="9634" LastActivityDate="2017-09-16T07:22:33.693" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4041" PostTypeId="1" AcceptedAnswerId="4746" CreationDate="2017-09-16T07:53:56.140" Score="3" ViewCount="3072" Body="&lt;p&gt;By optimal I mean that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If max has a winning strategy then minimax will return the strategy for max with the fewest number of moves to win.&lt;/li&gt;&#xA;&lt;li&gt;If min has a winning strategy then minimax will return the strategy for max with the most number of moves to lose.&lt;/li&gt;&#xA;&lt;li&gt;If neither has a winning strategy then minimax will return the strategy for max with the most number of moves to draw.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The idea is that you want to win in the fewest number of moves possible but if you can't win then you want to drag out the game for as long as possible so that the opponent has more chances of making mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how do you make minimax return the best strategy for max?&lt;/p&gt;&#xA;" OwnerUserId="9634" LastActivityDate="2017-12-14T22:30:29.347" Title="How to make minimax optimal?" Tags="&lt;algorithm&gt;&lt;game-theory&gt;&lt;minimax&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4043" PostTypeId="1" AcceptedAnswerId="4081" CreationDate="2017-09-16T23:12:08.457" Score="5" ViewCount="219" Body="&lt;p&gt;I understand that AI researchers are trying to create AI designs that allow for desired behavior without undesirable side-effects. A classic example of an attempt is Isaac Asimov's &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Three Laws of Robotics&lt;/a&gt;. This idea seems to have been debunked due to its vague phrasing. In particular, I realize that the exact 3 laws from the stories can't work, but surely there is a set of more robust ones that can limit an AI to good behavior in the same way that people are restricted by law.&lt;/p&gt;&#xA;&lt;p&gt;Why have AI researchers not accepted an idea like the following (just making the laws more specific):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What if the UN voted for the country with the fairest laws and all&#xA;utility functions have 2/3 of their points made up of not breaking any&#xA;of those laws. If the ai has a question, it could look at court&#xA;precedent just like a judge would or ask humans (with two-thirds of&#xA;its points on the line, it should be pretty cautious).&lt;/p&gt;&#xA;&lt;p&gt;People have been looking for loopholes in law for thousands of years&#xA;and there may not be any catastrophic ones left. (It certainly&#xA;wouldn't be violent)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I must be missing something if this is still an open problem.&lt;/p&gt;&#xA;" OwnerUserId="9648" LastEditorUserId="2444" LastEditDate="2020-11-17T14:03:46.933" LastActivityDate="2020-11-18T11:37:17.817" Title="Why is AI safety so much harder than Isaac Asimov's &quot;Three Laws of Robotics&quot;?" Tags="&lt;ai-design&gt;&lt;agi&gt;&lt;ai-safety&gt;&lt;neo-luddism&gt;&lt;asimovs-laws&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4044" PostTypeId="2" ParentId="1479" CreationDate="2017-09-17T10:03:18.140" Score="65" Body="&lt;p&gt;There are many approaches that aim to make a trained neural network more interpretable and less like a &quot;black box&quot;, specifically &lt;em&gt;convolutional neural networks&lt;/em&gt; that you've mentioned.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Visualizing the activations and layer weights&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@awjuliani/visualizing-neural-network-layer-activation-tensorflow-tutorial-d45f8bf7bbc4&quot; rel=&quot;noreferrer&quot;&gt;Activations visualization&lt;/a&gt; is the first obvious and straight-forward one. For ReLU networks, the activations usually start out looking relatively blobby and dense, but as the training progresses the activations usually become more sparse (most values are zero) and localized. This sometimes shows what exactly a particular layer is focused on when it sees an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another great work on activations that I'd like to mention is &lt;a href=&quot;https://www.youtube.com/watch?v=AgkfIQ4IGaM&quot; rel=&quot;noreferrer&quot;&gt;deepvis&lt;/a&gt; that shows reaction of every neuron at each layer, including pooling and normalization layers. Here's how they &lt;a href=&quot;http://yosinski.com/deepvis&quot; rel=&quot;noreferrer&quot;&gt;describe it&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In short, we’ve gathered a few different methods that allow you to&#xA;  “triangulate” what feature a neuron has learned, which can help you&#xA;  better understand how DNNs work.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The second common strategy is to visualize the weights (filters). These are usually most interpretable on the first CONV layer which is looking directly at the raw pixel data, but it is possible to also show the filter weights deeper in the network. For example, the first layer usually learns gabor-like filters that basically detect edges and blobs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D6pXk.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D6pXk.jpg&quot; alt=&quot;first layer filters&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Occlusion experiments&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here's the idea. Suppose that a ConvNet classifies an image as a dog. How can we be certain that it’s actually picking up on the dog in the image as opposed to some contextual cues from the background or some other miscellaneous object? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of investigating which part of the image some classification prediction is coming from is by plotting the probability of the class of interest (e.g. dog class) as a function of the position of an occluder object. &#xA;If we iterate over regions of the image, replace it with all zeros and check the classification result, we can build a 2-dimensional heat map of what's most important for the network on a particular image. This approach has been used in &lt;a href=&quot;https://arxiv.org/pdf/1311.2901.pdf&quot; rel=&quot;noreferrer&quot;&gt;Matthew Zeiler’s Visualizing and Understanding Convolutional Networks&lt;/a&gt; (that you refer to in your question):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O10WA.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O10WA.jpg&quot; alt=&quot;occlusion experiments&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Deconvolution&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Another approach is to synthesize an image that causes a particular neuron to fire, basically what the neuron is looking for. The idea is to compute the gradient with respect to the image, instead of the usual gradient with respect to the weights. So you pick a layer, set the gradient there to be all zero except for one for one neuron and backprop to the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deconv actually does something called &lt;em&gt;guided backpropagation&lt;/em&gt; to make a nicer looking image, but it's just a detail.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Similar approaches to other neural networks&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Highly recommend &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;noreferrer&quot;&gt;this post by Andrej Karpathy&lt;/a&gt;, in which he plays a lot with Recurrent Neural Networks (RNN). In the end, he applies a similar technique to see what the neurons actually learn:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The neuron highlighted in this image seems to get very excited about&#xA;  URLs and turns off outside of the URLs. The LSTM is likely using this&#xA;  neuron to remember if it is inside a URL or not.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Conclusion&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I've mentioned only a small fraction of results in this area of research. It's pretty active and new methods that shed light to the neural network inner workings appear each year.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, there's always something that scientists don't know yet, but in many cases they have a good picture (literary) of what's going on inside and can answer many particular questions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me the quote from your question simply highlights the importance of research of not only accuracy improvement, but the inner structure of the network as well. As Matt Zieler tells in &lt;a href=&quot;https://www.youtube.com/watch?v=ghEmQSxT6tw&quot; rel=&quot;noreferrer&quot;&gt;this talk&lt;/a&gt;, sometimes a good visualization can lead, in turn, to better accuracy.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-09-17T10:03:18.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4045" PostTypeId="1" CreationDate="2017-09-17T11:13:35.627" Score="4" ViewCount="150" Body="&lt;p&gt;I'm developing a Game AI which tries to master racing simulation. I already trained a CNN (alexnet) on ingame footage of me playing the game and the pressed keys as the target. I had two main issues with this setup:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Extracting the current speed from the speedometer, in order to feed it to the AI. That question was already solved &lt;a href=&quot;https://ai.stackexchange.com/questions/4013/game-ai-fast-python-ocr-or-cropped-image-input&quot;&gt;here&lt;/a&gt;.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;During testing, I noticed that the AI cannot make small adjustments on straight roads and cuts corners a lot.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm pretty sure that the second issue is caused by the game handling pressed keys binary (e.g. 'a' pressed -&gt; 100% left turn). The AI just can't make precise movements. In order to solve that issue, I want to emulate a joystick, which controls the game precisely. Using &lt;a href=&quot;https://www.pygame.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pygame&lt;/a&gt; I already managed to capture my controller inputs as training data.&lt;br&gt;&#xA;The controller has two axes, one for turning, the other one for throttle/breaking. Both axis can have any value between -1 and 1:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Axis 0: Value -1 -&amp;gt; 100% left&#xA;        Value +1 -&amp;gt; 100% right&#xA;&#xA;Axis 1: Value -1 -&amp;gt; full break&#xA;        Value +1 -&amp;gt; full throttle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My goal is to train alexnet to output analogue raw axis values, given the current speed and captured frame, and feed its predictions into the joystick emulator. I found someone on GitHub who tried &lt;a href=&quot;https://github.com/Sentdex/pygta5/issues/8#issuecomment-301934355&quot; rel=&quot;nofollow noreferrer&quot;&gt;something similar&lt;/a&gt; and could't achieve good results even on a modified alexnet. Because of this, I was wondering whether it is even possible to modify the CNN to output analogue values instead of using it as an image classifier.&lt;br&gt;&#xA;My question is, whether it is worth putting the effort into editing alexnet, instead of using a whole different model. I found some models online like the &lt;a href=&quot;https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NVIDIA End-to-End Self Driving model&lt;/a&gt;, which sadly only controls the steering angle, and seems to be made for low speed casual driving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9578" LastEditorUserId="9578" LastEditDate="2017-09-17T12:03:44.510" LastActivityDate="2017-09-17T12:03:44.510" Title="Game AI - Modify image classification model for analog output" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;gaming&gt;&lt;tensorflow&gt;&lt;game-ai&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4046" PostTypeId="2" ParentId="1294" CreationDate="2017-09-17T16:24:34.250" Score="3" Body="&lt;p&gt;In the abstract of the paper &lt;a href=&quot;https://research.google.com/pubs/pub46351.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing between Capsules&lt;/a&gt; (November 7, 2017) that formally introduced capsule neural networks, the authors write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule&#xA;  becomes active. We show that a discriminatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher-level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9657" LastEditorUserId="2444" LastEditDate="2020-06-09T15:21:01.497" LastActivityDate="2020-06-09T15:21:01.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4048" PostTypeId="1" CreationDate="2017-09-17T20:12:20.860" Score="14" ViewCount="13548" Body="&lt;p&gt;&lt;strong&gt;Introduction&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am currently writing an engine to play a card game, as there is no engine yet for this particular game.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;About the game&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The game is similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Magic:_The_Gathering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Magic: The Gathering&lt;/a&gt;. There is a commander, which has health and abilities. Players have an energy pool, which they use to put minions and spells on the board. Minions have health, attack values, costs, etc. Cards also have abilities, these are not easily enumerated. Cards are played from the hand, new cards are drawn from a deck. These are all aspects it would be helpful for the neural network to consider.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Idea&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am hoping to be able to introduce a neural network to the game afterwards, and have it learn to play the game. So, I'm writing the engine in such a way that is helpful for an AI player. There are choice points, and at those points, a list of valid options is presented. Random selection would be able to play the game (albeit not well).&lt;/p&gt;&#xA;&lt;p&gt;I have learned a lot about neural networks (mostly NEAT and HyperNEAT) and even built my own implementation. Neural networks are usually applied to image recognition tasks or to control a simple agent.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem/question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure if or how I would apply neural networks to make selections with cards, which have a complex synergy. How could I design and train a neural network for this game, such that it can take into account all the variables? Is there a common approach?&lt;/p&gt;&#xA;&lt;p&gt;I know that &lt;a href=&quot;https://github.com/bnordli/rftg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keldon wrote a good AI for RftG&lt;/a&gt;, which has a decent amount of complexity, but I am not sure how he managed to build such an AI.&lt;/p&gt;&#xA;&lt;p&gt;Any advice? Is it feasible? Are there any good examples of this? How were the inputs mapped?&lt;/p&gt;&#xA;" OwnerUserId="9660" LastEditorUserId="2444" LastEditDate="2021-01-31T19:07:59.060" LastActivityDate="2021-01-31T19:07:59.060" Title="How can I design and train a neural network to play a card game (similar to Magic: The Gathering)?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;neat&gt;&lt;card-games&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4049" PostTypeId="1" AcceptedAnswerId="4083" CreationDate="2017-09-18T02:55:31.673" Score="2" ViewCount="82" Body="&lt;p&gt;I've just watched the &lt;a href=&quot;https://www.youtube.com/watch?v=MSwoNAODrgk&quot; rel=&quot;nofollow noreferrer&quot;&gt;9th episode of HTM school&lt;/a&gt; about the &quot;boosting&quot; and &quot;inhibition&quot; ideas. However, I couldn't find the neuroscience counterpart of these terms and concepts. Since &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_temporal_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTM&lt;/a&gt; is a biologically-constrained theory, the &quot;boosting&quot; and &quot;inhibition&quot; concepts must have a neuroscience counterpart. What are they?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The video also discusses &lt;a href=&quot;https://en.wikipedia.org/wiki/Homeostasis&quot; rel=&quot;nofollow noreferrer&quot;&gt;homeostasis&lt;/a&gt; and &lt;a href=&quot;http://www.scholarpedia.org/article/Homeostatic_Regulation_of_Neuronal_Excitability&quot; rel=&quot;nofollow noreferrer&quot;&gt;homeostatic regulation of neuronal excitability&lt;/a&gt;. Do these concepts have something to do with &quot;boosting&quot; and &quot;inhibition&quot;?&lt;/p&gt;&#xA;" OwnerUserId="9614" LastEditorUserId="2444" LastEditDate="2018-08-20T20:51:20.457" LastActivityDate="2018-08-20T20:52:22.337" Title="Biological analogy for boosting and inhibition idea in Hierarchical Temporal Memory (HTM)" Tags="&lt;terminology&gt;&lt;htm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4050" PostTypeId="1" CreationDate="2017-09-18T05:39:58.897" Score="1" ViewCount="101" Body="&lt;p&gt;I was trying to categorical variable engineering following &lt;a href=&quot;https://arxiv.org/abs/1604.06737&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper. The code is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import random&#xA;import pandas&#xA;import numpy as np&#xA;import tensorflow as tf&#xA;&#xA;from tensorflow.contrib import layers&#xA;from tensorflow.contrib import learn&#xA;from __future__ import print_function&#xA;&#xA;from sklearn.preprocessing import LabelEncoder&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My dataset looks like the following. It's has 2 independent variable ('X1' &amp;amp; 'X2')and 1 dependent variable ('lable'). 'X2' is the categorical variable. I want to create an embedding vector for this variable and run the simple linear regression to predict 'label'using Tensorflow. I could use any other method. But since linear regression is easiest to understand, I'm trying that.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.DataFrame({'X1': np.array([&quot;A&quot;,&quot;A&quot;,&quot;B&quot;,&quot;C&quot;,&quot;B&quot;,&quot;C&quot;,&quot;B&quot;,&quot;C&quot;,&quot;C&quot;,&quot;B&quot;,&#xA;                         &quot;A&quot;,&quot;B&quot;,&quot;A&quot;,&quot;C&quot;,&quot;A&quot;,&quot;A&quot;,&quot;C&quot;]),'X2': np.array([3.3,4.4,5.5,6.71,6.93,4.168,9.779,6.182,7.59,2.167,&#xA;                         7.042,10.791,5.313,7.997,5.654,9.27,3.1]),&#xA;                       'label': np.array([1.7,2.76,2.09,3.19,1.694,1.573,3.366,2.596,2.53,1.221,&#xA;                         2.827,3.465,1.65,2.904,2.42,2.94,1.3])})&#xA;&#xA;# For variable 'X1', I'm creating levels.&#xA;&#xA;encoder = LabelEncoder()&#xA;encoder.fit(df.X1.values)&#xA;X = encoder.transform(df.X1.values)&#xA;&#xA;# Recreating dependent variable list.&#xA;&#xA;y = np.asarray([1.7,2.76,2.09,3.19,1.694,1.573,3.366,2.596,2.53,1.221,&#xA;                         2.827,3.465,1.65,2.904,2.42,2.94,1.3])&#xA;&#xA;# Setting Hyper-parameters&#xA;&#xA;training_epochs = 5&#xA;learning_rate = 1e-3&#xA;cardinality = len(np.unique(X))&#xA;embedding_size = 2&#xA;input_X_size = 1&#xA;n_hidden = 10&#xA;&#xA;# Setting up variables:&#xA;&#xA;embeddings = tf.Variable(tf.random_uniform([cardinality, embedding_size], -1.0, 1.0))&#xA;&#xA;&#xA;h = tf.Variable(tf.truncated_normal((embedding_size + len(df.X1), n_hidden), stddev=0.1))&#xA;&#xA;&#xA;W_out = tf.get_variable(name='out_w', shape=[n_hidden],&#xA;                            initializer=tf.contrib.layers.xavier_initializer())&#xA;&#xA;# Embedding:&#xA;&#xA;embedded_chars = tf.nn.embedding_lookup(embeddings, x)&#xA;embedded_chars = tf.reshape(embedded_chars, [-1])&#xA;embedded_chars= embedded_chars + np.array([3.3,4.4,5.5,6.71,6.93,4.168,9.779,6.182,7.59,2.167,&#xA;                         7.042,10.791,5.313,7.997,5.654,9.27,3.1])&#xA;&#xA;# Multiplying with Hidden Layers:&#xA;&#xA;layer_1 = tf.matmul(embedded_chars,h)&#xA;layer_1 = tf.nn.relu(layer_1)&#xA;out_layer = tf.matmul(layer_1, W_out)&#xA;&#xA;# Define loss and optimizer&#xA;&#xA;cost = tf.reduce_sum(tf.pow(out_layer-y, 2))/(2*n_samples)&#xA;optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)&#xA;#    Run the graph&#xA;&#xA;init = tf.global_variables_initializer()&#xA;&#xA;# Launch the graph&#xA;&#xA;with tf.Session() as sess:&#xA;    sess.run(init)&#xA;&#xA;    for epoch in range(training_epochs):&#xA;        avg_cost = 0.&#xA;&#xA;        _, c = sess.run([optimizer, cost],&#xA;                        feed_dict={x: X, y: Y})&#xA;print(&quot;Ran without Error&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;While running the code, I'm getting the following error.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ValueError: Shape must be rank 2 but is rank 1 for 'MatMul_1' (op:&#xA;  'MatMul') with input shapes: [17], [19,10].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not able to add the continuous variable with embedding variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone please guide me how to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9664" LastEditorUserId="9647" LastEditDate="2017-09-19T00:35:13.873" LastActivityDate="2017-09-19T00:35:13.873" Title="Categorical Variable Reduction using NN" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4051" PostTypeId="2" ParentId="92" CreationDate="2017-09-18T14:30:02.917" Score="16" Body="&lt;p&gt;All answers here are great, but, for some reason, nothing has been said so far on why this effect &lt;em&gt;should not surprise&lt;/em&gt; you. I'll fill the blank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me start with one requirement that is absolutely essential for this to work: the attacker &lt;em&gt;must know neural network architecture&lt;/em&gt; (number of layers, size of each layer, etc). Moreover, in all cases that I examined myself, the attacker knows the snapshot of the model that is used in production, i.e. all weights. In other words, the &quot;source code&quot; of the network isn't a secret.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't fool a neural network if you treat it like a black box. And you can't reuse the same fooling image for different networks. &#xA;In fact, you have to &quot;train&quot; the target network yourself, and here by training I mean to run forward and backprop passes, but specially crafted for another purpose.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Why is it working at all?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Now, here's the intuition. Images are very high dimensional: even the space of small 32x32 color images has &lt;code&gt;3 * 32 * 32 = 3072&lt;/code&gt; dimensions. But the training data set is relatively small and contains real pictures, all of which have some structure and nice statistical properties (e.g. smoothness of color). So the training data set is located on a tiny manifold of this huge space of images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convolutional networks work extremely well on this manifold, but basically, know nothing about the rest of the space. The classification of the points outside of the manifold is just a linear extrapolation based on the points inside the manifold. No wonder that some particular points are extrapolated incorrectly. The attacker only needs a way to navigate to the closest of these points.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let me give you a concrete example how to fool a neural network. To make it compact, I'm going to use a very simple logistic regression network with one nonlinearity (sigmoid). It takes a 10-dimensional input &lt;code&gt;x&lt;/code&gt;, computes a single number &lt;code&gt;p=sigmoid(W.dot(x))&lt;/code&gt;, which is the probability of class 1 (versus class 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/i3mO7.png&quot; alt=&quot;logistic regression&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you know &lt;code&gt;W=(-1, -1, 1, -1, 1, -1, 1, 1, -1, 1)&lt;/code&gt; and start with an input &lt;code&gt;x=(2, -1, 3, -2, 2, 2, 1, -4, 5, 1)&lt;/code&gt;. A forward pass gives &lt;code&gt;sigmoid(W.dot(x))=0.0474&lt;/code&gt; or 95% probability that &lt;code&gt;x&lt;/code&gt; is class 0 example.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/V6cNn.png&quot; alt=&quot;x and W&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We'd like to find another example, &lt;code&gt;y&lt;/code&gt;, which is very close to &lt;code&gt;x&lt;/code&gt; but is classified by the network as 1. Note that &lt;code&gt;x&lt;/code&gt; is 10-dimensional, so we have the freedom to nudge 10 values, which is a lot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since &lt;code&gt;W[0]=-1&lt;/code&gt; is negative, it's better for to have a small &lt;code&gt;y[0]&lt;/code&gt; to make a total contribution of &lt;code&gt;y[0]*W[0]&lt;/code&gt; small. Hence, let's make &lt;code&gt;y[0]=x[0]-0.5=1.5&lt;/code&gt;.&#xA;Likewise, &lt;code&gt;W[2]=1&lt;/code&gt; is positive, so it's better to increase &lt;code&gt;y[2]&lt;/code&gt; to make &lt;code&gt;y[2]*W[2]&lt;/code&gt; bigger: &lt;code&gt;y[2]=x[2]+0.5=3.5&lt;/code&gt;. And so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8zs12.png&quot; alt=&quot;x, W and y&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is &lt;code&gt;y=(1.5, -1.5, 3.5, -2.5, 2.5, 1.5, 1.5, -3.5, 4.5, 1.5)&lt;/code&gt;, and &lt;code&gt;sigmoid(W.dot(y))=0.88&lt;/code&gt;. With this one change we improved the class 1 probability from 5% to 88%!&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Generalization&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you look closely at the previous example, you'll notice that I knew exactly how to tweak &lt;code&gt;x&lt;/code&gt; in order to move it to the target class, because I knew the network gradient. What I did was actually a &lt;em&gt;backpropagation&lt;/em&gt;, but with respect to the data, instead of weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the attacker starts with target distribution &lt;code&gt;(0, 0, ..., 1, 0, ..., 0)&lt;/code&gt; (zero everywhere, except for the class it wants to achieve), backpropagates to the data and makes a tiny move in that direction. Network state is not updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it should be clear that it's a common feature of feed-forward networks that deal with a small data manifold, no matter how deep it is or the nature of data (image, audio, video or text).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Potection&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The simplest way to prevent the system from being fooled is to use an ensemble of neural networks, i.e. a system that aggregates the votes of several networks on each request.&#xA;It's much more difficult to backpropagate with respect to several networks simultaneously. The attacker might try to do it sequentially, one network at a time, but the update for one network might easily mess up&#xA;with the results obtained for another network. The more networks are used, the more complex an attack becomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is to smooth the input before passing it to the network.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Positive use of the same idea&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You shouldn't think that backpropagation to the image has only negative applications. A very similar technique, called &lt;em&gt;deconvolution&lt;/em&gt;, is used for visualization and better understanding what neurons have learned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This technique allows synthesizing an image that causes a particular neuron to fire, basically see visually &quot;what the neuron is looking for&quot;, which in general makes convolutional neural networks more interpretable.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorUserId="14723" LastEditDate="2018-04-12T14:53:26.000" LastActivityDate="2018-04-12T14:53:26.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4053" PostTypeId="2" ParentId="3981" CreationDate="2017-09-18T16:37:27.630" Score="11" Body="&lt;p&gt;I'd like to add to what's been said already that your question touches upon an important notion in machine learning called &lt;em&gt;transfer learning&lt;/em&gt;. In practice, very few people train an entire convolutional network from scratch (with random initialization), because it is time consuming and relatively rare to have a dataset of sufficient size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern ConvNets take 2-3 weeks to train across multiple GPUs on ImageNet. So it is common to see people release their final ConvNet checkpoints for the benefit of others who can use the networks for fine-tuning. For example, the Caffe library has a &lt;a href=&quot;https://github.com/BVLC/caffe/wiki/Model-Zoo&quot; rel=&quot;noreferrer&quot;&gt;Model Zoo&lt;/a&gt; where people share their network weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you need a ConvNet for image recognition, no matter what your application domain is, you should consider taking an existing network, for example &lt;a href=&quot;http://www.robots.ox.ac.uk/~vgg/research/very_deep/&quot; rel=&quot;noreferrer&quot;&gt;VGGNet&lt;/a&gt; is a common choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few things to keep in mind when performing &lt;em&gt;transfer learning&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Constraints from pretrained models. Note that if you wish to use a pretrained network, you may be slightly constrained in terms of the architecture you can use for your new dataset. For example, you can’t arbitrarily take out Conv layers from the pretrained network. However, some changes are straight-forward: due to parameter sharing, you can easily run a pretrained network on images of different spatial size. This is clearly evident in the case of Conv/Pool layers because their forward function is independent of the input volume spatial size (as long as the strides “fit”).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Learning rates. It’s common to use a smaller learning rate for ConvNet weights that are being fine-tuned, in comparison to the (randomly-initialized) weights for the new linear classifier that computes the class scores of your new dataset. This is because we expect that the ConvNet weights are relatively good, so we don’t wish to distort them too quickly and too much (especially while the new Linear Classifier above them is being trained from random initialization).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Additional reference if you are interested in this topic: &lt;a href=&quot;https://arxiv.org/abs/1411.1792&quot; rel=&quot;noreferrer&quot;&gt;How transferable are features in deep neural networks?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-09-18T16:37:27.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4054" PostTypeId="1" AcceptedAnswerId="4055" CreationDate="2017-09-18T20:05:22.083" Score="4" ViewCount="443" Body="&lt;p&gt;Is there any well defined method to define or represent evil in &lt;strong&gt;abstract&lt;/strong&gt; logic, binary or AI form?&lt;/p&gt;&#xA;&lt;p&gt;Video games method of representing evil is relative to the player context (thus subjective, and not pure abstract evil in an objective sense).&lt;/p&gt;&#xA;&lt;p&gt;What I am asking is there any data defined as well-known evil?&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;var x=666;&#xA;&#xA;if (isEvil(x)) {&#xA;    //do something.&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Remark:&#xA;Evil Number descried in &lt;a href=&quot;http://mathworld.wolfram.com/EvilNumber.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mathworld.wolfram.com/EvilNumber.html&lt;/a&gt; doesn't qualify as well-known evil data.&lt;/p&gt;&#xA;&lt;h1&gt;Following Up:&lt;/h1&gt;&#xA;&lt;p&gt;One of the main objectives of the question is to &lt;strong&gt;understand scientifically the limits of evil in AI&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;According to my understanding of: &lt;a href=&quot;https://en.wikipedia.org/wiki/Evil&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Evil&lt;/a&gt; I think it's mandatory to explore &amp;quot;evil&amp;quot; in religion context in order to come up with valid model for evil. &lt;em&gt;But I don't want go into (religion) debates or any divergence at this stage&lt;/em&gt;. hence below points Sums up my understanding:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The only well-known Evil source is the devil (our creator declared the devil as the first common enemy for ALL humans).&lt;/li&gt;&#xA;&lt;li&gt;Whispering is devil method of attack, If human followed the whisper it will lead to evil. and gradually human Evil grow...&lt;/li&gt;&#xA;&lt;li&gt;There are other points but I don't see its related to AI in any means.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Based on the above, I asked myself: &lt;strong&gt;since AI is human creation, where the evil in AI will come from??!&lt;/strong&gt; my answer is: directly from us and indirectly by following the devil. So all crimes committed by Evil AI bounded to AI architect/designer/unethical hacker.&lt;/p&gt;&#xA;&lt;p&gt;The next stage in getting closer to model evil, is to define and classify the evil acts:&lt;/p&gt;&#xA;&lt;h2&gt;Definitions:&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Define evil in AI context (draft ver. 0.1): committing crimes against nature, civilizations or humans. And reprogramming, modifying or attacking tech devices/machines to perform malicious agenda.&lt;/li&gt;&#xA;&lt;li&gt;Crime is broad and relative to the party: example: breaking one government regulations based on the orders of other government. &lt;em&gt;I mean as long each group of humans makes its own laws and regulations unified justice can't be applied on Evil AI.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If my assumption of bounding evil to crime is valid then &lt;strong&gt;evil classification inherits crime classification&lt;/strong&gt; which seems well-defined:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Crime#Classification_and_categorisation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Crime#Classification_and_categorisation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Next step is to pick an easy to model crime class, prepare training data, ... Do you agree with the follow up? Do you agree that Boolean logic can't determine evil without AI?&lt;/p&gt;&#xA;" OwnerUserId="9685" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-10-01T06:57:10.333" Title="How to define or represent evil in logic" Tags="&lt;ai-design&gt;&lt;decision-theory&gt;&lt;ethics&gt;&lt;game-theory&gt;&lt;game-ai&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4055" PostTypeId="2" ParentId="4054" CreationDate="2017-09-18T21:28:12.870" Score="2" Body="&lt;p&gt;I think you're going to have to be reconciled to the subjective nature of reality.  Objectivity is only possible in very special cases such as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Q.E.D.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q.E.D.&lt;/a&gt; in mathematics, or a &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved gamed&lt;/a&gt;.  Rationality is &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;bounded&lt;/a&gt;, and any &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Complexity_classes&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractable&lt;/a&gt; problem results in a state of subjectivity/indeterminacy.  Additionally, pure values do not carry moral implications, despite popular associations, although it would be possible to create a game where certain values have negative effects, and the harm they result in could be understood as evil.  (i.e. 666/616 has numerological associations, and numerology can be understood as a proto form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Number_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;number theory&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A simple way to define evil would be through behavioral models in &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game Theory&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In Game Theory, there is a concept knowns as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Superrationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;superrational strategy&lt;/a&gt;. &#xA;Superrationality may be understood as the logical/mathematical expression of the Golden Rule: &quot;do unto others as you would have them do unto you.&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Golden Rule forms the basis for most religions, in the context of empathy/compassion and altruism.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the agent is evil, it will always betray, even when the competitor has shown a willingness to cooperate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus evil is defined as the opposite of the Golden Rule.  &lt;em&gt;(Possibly we would call this the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sulfur&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brimstone Rule&lt;/a&gt;;)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-09-18T21:28:12.870" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4056" PostTypeId="5" CreationDate="2017-09-18T22:00:25.630" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-31T13:03:44.673" LastActivityDate="2021-01-31T13:03:44.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4057" PostTypeId="4" CreationDate="2017-09-18T22:00:25.630" Score="0" Body="For questions related to Numenta's Hierarchical Temporal Memory (HTM), which is a biologically-constrained theory of intelligence originally described in the book On Intelligence (by J. Hawkins and S. Blakeslee)." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-31T13:03:44.673" LastActivityDate="2021-01-31T13:03:44.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4058" PostTypeId="2" ParentId="2219" CreationDate="2017-09-19T01:28:38.313" Score="5" Body="&lt;p&gt;Instead of having the AI learn what action to take, you can alternatively train it to judge how &quot;good&quot; a position is. In order to determine what move to make, you don't ask the AI &quot;This is the current state, what move should I make&quot;, you iterate through all possible moves, and feed the the resulting state into the AI asking &quot;How good do you think this new state is?&quot;. You then chose the move with the resulting state that the AI liked best. (Or you probably even can combine this with a traditional MinMax approach)&#xA;I'm new to this area myself, but I'd guess you would use this approach when the action space is large, and in particular when most possible actions are not a legal option in most states.&lt;/p&gt;&#xA;" OwnerUserId="9688" LastActivityDate="2017-09-19T01:28:38.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4059" PostTypeId="1" AcceptedAnswerId="4070" CreationDate="2017-09-19T10:23:52.333" Score="3" ViewCount="1398" Body="&lt;p&gt;I'm currently developing a blackjack program. Now, I want to create an AI that essentially uses the mathematics of blackjack to make decisions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what is the typical AI approach for solving blackjack?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't have to be language-specific, but if it will help with an answer, the language I plan to use to do this would be Python. &lt;/p&gt;&#xA;" OwnerUserId="9699" LastEditorUserId="2444" LastEditDate="2020-06-02T23:54:23.260" LastActivityDate="2020-06-02T23:54:23.260" Title="What is the typical AI approach for solving blackjack?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4061" PostTypeId="1" AcceptedAnswerId="4224" CreationDate="2017-09-19T13:06:22.270" Score="3" ViewCount="233" Body="&lt;p&gt;Below is an excerpt in an instructor's manual on ML that is explaining deep neural networks, using cat recognition (what else!) from images as example. On how DL performs this feat, the excerpt said that,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Assume that the first layer returns the number of pixels that are&#xA;  brown/black/blue/red, and the second layer finds the most common&#xA;  color, and the third layer returns “cat” if previous layer had&#xA;  supplied “brown”. [..] Mathematically, this model would be, for the&#xA;  first layer, [ sum(r = 255, g=255, b=255), ..., ..., sum(r=255, g=0,&#xA;  b=0)] -- this is just a set of appropriately positioned relu functions&#xA;  (okay, for r=234, we’d need two relu functions, so two layers, but you&#xA;  get the idea). The second layer would be a softmax layer. The third&#xA;  layer is simply an identity!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now I worked with deep nets, but I am not sure how I can structure a DL to do this. ReLu is simply a max(0,x), so how would I filter out pixel vals for example 128,128,128 and sum them up? Wouldn't the convolution layer play a role here too? What would the layout of a simple deep net be that does what is described above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="9658" LastEditorUserId="9658" LastEditDate="2017-09-19T13:41:07.263" LastActivityDate="2017-10-08T08:12:04.967" Title="ReLu, Sum and Convolution Layers to Count Pixels of Certain Color" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4069" PostTypeId="1" CreationDate="2017-09-20T03:25:56.490" Score="1" ViewCount="131" Body="&lt;p&gt;If Artificial General Intelligence is possible, whoever creates it is likely to have a lot of power.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What might this look like in practical terms, and what might be limiters? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Good answers will include aspects such as historical precedents related to technology, and cover subjects such as potential societal and economic impacts, and limiters such as regulations and physics.&lt;/p&gt;&#xA;" OwnerUserId="9716" LastEditorUserId="1671" LastEditDate="2019-06-21T17:21:58.517" LastActivityDate="2019-06-21T17:22:01.567" Title="How much power might the creator of the first Artificial General Intelligence obtain?" Tags="&lt;agi&gt;&lt;social&gt;&lt;legal&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4070" PostTypeId="2" ParentId="4059" CreationDate="2017-09-20T04:17:27.497" Score="4" Body="&lt;p&gt;Blackjack is usually modelled using Monte Carlo (MC) Methods. There is a lot of literature on MC methods which is interesting on its own right but &lt;a href=&quot;https://math.dartmouth.edu/theses/undergrad/2014/Vaidyanathan-Thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is a paper describing how MC is applied to Blackjack. There is also a good description on page 110 of the &lt;a href=&quot;http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-09-20T04:17:27.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4071" PostTypeId="2" ParentId="4048" CreationDate="2017-09-20T07:13:26.623" Score="2" Body="&lt;p&gt;This is completely feasible, but the way the inputs are mapped would greatly depend on the type of card game, and how it's played.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll take into account a few possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does time matter in this game? Would a past move influence a future one? In this case, you'd be better off using Recurrent Neural Networks (LSTMs, GRUs, etc.).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Would you like the Neural Network to learn off of data you collect, or learn on its own? If on its own, how? If you collect data of yourself playing the game tens or hundreds of times, feed it into the Neural Net, and make it learn from you, then you're doing something called &quot;Behavioural Cloning&quot;. However, if you'd like the NN to learn on its own, you can do this 2 ways:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) &lt;strong&gt;Reinforcement Learning&lt;/strong&gt; - RL allows the Neural Net to learn by playing against itself &lt;em&gt;lots&lt;/em&gt; of times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) &lt;strong&gt;NEAT/Genetic Algorithm&lt;/strong&gt; - NEAT allows the Neural Net to learn by using a genetic algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, again, in order to get more specific as to how the Neural Net's inputs and outputs should be encoded, I'd have to know more about the card game itself.&lt;/p&gt;&#xA;" OwnerUserId="5240" LastActivityDate="2017-09-20T07:13:26.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4072" PostTypeId="1" CreationDate="2017-09-20T08:48:33.403" Score="3" ViewCount="279" Body="&lt;p&gt;To start, I'm not a programmer/computer scientist/et al... - I work in Finance and have, through my job, self-thought myself VBA for excel and outlook and would consider myself as being in the upper half of the population of people who write VBA creating browser apps for scraping (and interpreting site's DOMs in doing so), creating Arrays of multi-dimensional arrays (arrays of 3D arrays - haven't gone for a 4D array as I haven't needed to) and generally focused on making as efficient macros. I've (very) basic understanding of JAVA also (I even created an basic android app!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By pure accident, learning the above has given me a massive interest in all thing related to IT developments and the massive development which I truly believe will be the development in Human History (for better or worse has yet to be determined!) is AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know all that I can about AI so was wondering if you all could assist an absolute noob on the subject. I know nothing of Python (which seems the language of choice for AI) but anything that would help would be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not completely naive to the fact the brightest minds in the IT world are are working on this and concepts are going to be difficult for someone like me but I'm willing to try!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any and all help/suggestions are welcome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="9712" LastEditorUserId="169" LastEditDate="2017-09-21T11:12:42.313" LastActivityDate="2017-09-21T11:12:42.313" Title="Can someone direct me to a sites and/or videos that can bring an absolute beginner up to speed with AI?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;programming-languages&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ClosedDate="2021-01-21T23:25:44.217" ContentLicense="CC BY-SA 3.0" />
  <row Id="4073" PostTypeId="2" ParentId="4072" CreationDate="2017-09-20T10:57:31.583" Score="8" Body="&lt;p&gt;I think the best approach would be to pick up a good book that lays out the fundamentals very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach (AIMA)&lt;/a&gt; is a great place to begin with! It is used as a reference textbook for many university level AI courses. It has helped me quite a lot as a student.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-09-20T10:57:31.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4075" PostTypeId="2" ParentId="4072" CreationDate="2017-09-20T11:59:48.167" Score="3" Body="&lt;p&gt;For informal viewing, computerphile has some videos from AI researchers, and there are videos about deepdream which image recognition from google.&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=5qfIgCiYlfY&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=5qfIgCiYlfY&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a programming perspective, you should learn about NN because it's made to imitate neuron arrangements, it's the most established field for programming neurons. Neural Networks have various types which are all difficult to program, I don't know if deep blue chess program even used it, other means may still be more efficient. &lt;/p&gt;&#xA;" OwnerUserId="9722" LastActivityDate="2017-09-20T11:59:48.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4076" PostTypeId="1" CreationDate="2017-09-20T12:37:03.733" Score="5" ViewCount="223" Body="&lt;p&gt;I have to analyse sequences of actions that look more or less like &lt;a href=&quot;https://gist.github.com/Morpheu5/eeea1c8b7b76b13f6010af2296cff336&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; JSON blob. The question I'm trying to answer is whether there are recurring (sub)patterns that different users adopt when asked to perform a certain specific task -- in this case, the task is to build a mathematical formula using &lt;a href=&quot;https://www.isaacphysics.org/equality&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; editor. In particular I'd like to know if there are multiple significantly different ways in which people build the same expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of creating a Markov model, but that would only give me the most likely sequence of actions of length &lt;em&gt;N&lt;/em&gt;. An obvious alternative would be to build trees and count how many times a certain path occurs in the dataset. However, the nature of the expression-building process means that the sequences can be polluted by many confounding, non-significant actions (such as streaks of UNDO-REDO, deleting symbols, and the likes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might go the &quot;longest common subsequence&quot; route, but I'm not sure that would tell me if there are &quot;significantly different&quot; ways of building the same expression (in quotes because, for now, I don't have a rigorous definition of &quot;significantly different&quot;, but, for example, one way would be to drag and drop-in-place all the symbols in the correct order, and another way would be to drag all the symbols onto the canvas, and then place them in the correct spots).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought this might be a nice challenge for some AI algorithm, but I'm quite a noob at that, so I'm open to suggestions.&lt;/p&gt;&#xA;" OwnerUserId="9720" LastActivityDate="2018-05-16T11:24:15.590" Title="Detect patterns in sequences of actions" Tags="&lt;classification&gt;&lt;markov-chain&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4077" PostTypeId="2" ParentId="4069" CreationDate="2017-09-20T13:50:56.573" Score="1" Body="&lt;p&gt;&quot;Artificial Intelligence&quot; has existed for many, many years.  Decades even.  Go back to Arthur Samuel and his Checkers playing program (if not before).  That's AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is, as soon as computers can do something, people want to quit treating it as AI.  AI is almost, by definition, something that can't really exist, because it has to be about something humans can do, that computers can't.   So all progress made in the field is reclassified as &quot;not AI&quot; as soon as it does anything useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more salient question might be &quot;who will be the first to announce that Artificial General Intelligence&quot; has been created.   AGI will, if it ever happens, be fundamentally different in terms of being able to learn to do anything, as opposed to, say, a Checkers program which can only play Checkers, or a Go program that can only play Go, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the answer to that question is -- nobody knows.  Nobody even know for &lt;em&gt;sure&lt;/em&gt; that AGI will &lt;em&gt;ever&lt;/em&gt; happen (though a lot of us presume that it will).  Anyway, there isn't a ton of research on AGI going on, relative to the amount of research in Machine Learning / Narrow AI.  But somebody working in their garage could make a huge breakthrough tomorrow for all we know.  I think a lot of people assume that creating this will take Nation-State level resources, but that hasn't really been proven.  So it might be some kid in a garage in Omaha tomorrow, or it might be the Chinese government in 2074.  There's just no way to really know.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-09-20T13:50:56.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4078" PostTypeId="2" ParentId="4072" CreationDate="2017-09-20T14:05:01.997" Score="6" Body="&lt;p&gt;Second the recommendation for &quot;AI: A Modern Approach&quot;.  Outside of that, there are a ton of resources online to consider.  I think a good start would include:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Andrew Ng's Machine Learning course on Coursera (note: I mean the old course, not the &quot;Deep Learning Specialization&quot; which I have no experience with).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Berkeley CS188 course materials here: &lt;a href=&quot;http://ai.berkeley.edu/home.html&quot; rel=&quot;noreferrer&quot;&gt;http://ai.berkeley.edu/home.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Professor P. Daspguta's AI lectures here:  &lt;a href=&quot;https://www.youtube.com/watch?v=eLbMPyrw4rw&amp;amp;list=PLvx5ei9aEEqJBDBLV3ELK4PKEgnFeOrw6&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=eLbMPyrw4rw&amp;amp;list=PLvx5ei9aEEqJBDBLV3ELK4PKEgnFeOrw6&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Read some (or all if you have a lot of time on your hands) of the &quot;AI Letters&quot; from MIT.  &lt;a href=&quot;http://publications.csail.mit.edu/ai/&quot; rel=&quot;noreferrer&quot;&gt;http://publications.csail.mit.edu/ai/&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The past Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) are available as well.  Browsing those could be interesting once you have some basics down.  &lt;a href=&quot;https://www.ijcai.org/past_proceedings&quot; rel=&quot;noreferrer&quot;&gt;https://www.ijcai.org/past_proceedings&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There's an older, 4 volume series titled &quot;The Handbook of Artificial Intelligence&quot;.  It's long out of print, but used copies are readily available on Amazon.  It deals with older techniques that aren't necessarily &quot;en vogue&quot; today, but may still be important either for perspective, or because those techniques are just waiting to be rediscovered.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/048624864X&quot; rel=&quot;noreferrer&quot;&gt;Introduction to Artificial Intelligence&lt;/a&gt; by Philip C. Jackson. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-09-20T14:05:01.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4080" PostTypeId="1" AcceptedAnswerId="4089" CreationDate="2017-09-20T18:40:17.130" Score="3" ViewCount="1474" Body="&lt;p&gt;I studied the article &lt;a href=&quot;https://www.intelnervana.com/demystifying-deep-reinforcement-learning/&quot; rel=&quot;noreferrer&quot;&gt;&quot;Demystifying Deep Reinforcement Learning&quot;&lt;/a&gt; extensively during the last days, while trying to implement the proposed algorithms myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to have an agent learn by playing a simple board game against itself using the methods of deep reinforcement learning. The algorithm described in pseudo code in the chapter called &quot;Deep Q-learning Algorithm&quot; is straight forward, but I cannot wrap my head around the fact that the replay memory only gets initialized once and never gets cleared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the obvious issue of growing too large for the available memory, there seems to be a more fundamental flaw. In the beginning the games will appear almost random, because the Q-function just has been initialized randomly. This means that bad moves will hardly be punished by the opponent. It makes sense to learn from those random encounters in the beginning, as we do not have any better data to learn from and this actually describes the observed behavior of the environment very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when the agent improves, the old memorized plays will no longer be valuable, as certain moves will no longer be played by the opponent and therefore the old games will no longer reflect the current behavior of the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's at least my interpretation. Now I am wondering if this is simply missing from the pseudo code in the article or if my way of thinking is wrong in this regard. My question is, if we need to flush the replay memory regularly in the given setup and how often it should be done?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-09-22T01:05:41.510" Title="Is it necessary to clear the replay memory regularly in a DQN when an agent plays against itself?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4081" PostTypeId="2" ParentId="4043" CreationDate="2017-09-20T19:32:58.310" Score="5" Body="&lt;p&gt;The reason this is hard is because it is not trivial to understand what a law means. Many humans still have a hard time understanding laws and thus we have &lt;a href=&quot;https://www.americanbar.org/content/dam/aba/administrative/market_research/total_national_lawyer_counts_1878_2013.authcheckdam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;millions&lt;/a&gt; of judges and lawyers who study years to be able to even debate whether a law was broken at all.&lt;/p&gt;&#xA;&lt;p&gt;More generally to AI, the problem of understanding laws is a byproduct of the bigger problem that AIs today do not yet have mechanisms to turn input (text, audio, other sensor data) into concepts. This is evident in even the most &lt;a href=&quot;https://arxiv.org/pdf/1606.01541.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;advanced conversation systems&lt;/a&gt; which struggle with dialog which is off of a particular set of subjects.&lt;/p&gt;&#xA;&lt;p&gt;However, there are promising results with probabilistic logic programming languages that attempt to combine first-order logic with statistical inference (&lt;a href=&quot;http://reasoning.eas.asu.edu/lpmln/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{LP}^{\text{MLN}}$&lt;/span&gt; is an example&lt;/a&gt;). These languages could provide, in theory, a way for future AI-lawyers (or whatever one should call them) to encode exactly what laws mean. Using this set or encoded laws, a system with a rich enough representation could determine if an action would brake a law or not. This sounds great but the &amp;quot;rich enough representation&amp;quot; is beyond what even today's best deep learning systems (or any other architecture) can do. For comparison, today's best deep learning systems don't even have good object permanence, something humans have around 4-7 months old. Asking those deep learning systems to also be able to represent the complexities of laws is profoundly outside of current AI capabilities.&lt;/p&gt;&#xA;&lt;p&gt;Another promising area is only including text data and having &lt;a href=&quot;http://www.cbc.ca/beta/news/canada/edmonton/legal-artificial-intelligence-alberta-japan-1.4296763&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificial judges&lt;/a&gt;. This wouldn't immediately mean that AIs would obey laws but it is the start of AIs understanding the word of law.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-11-17T16:14:26.553" LastActivityDate="2020-11-17T16:14:26.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4082" PostTypeId="2" ParentId="3908" CreationDate="2017-09-21T01:33:50.627" Score="2" Body="&lt;p&gt;May not be quite what you’re looking for, but nonetheless helpful, I hope. The White House a year ago commissioned a &lt;a href=&quot;https://obamawhitehouse.archives.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/preparing_for_the_future_of_ai.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;report on AI&lt;/a&gt; that touches briefly on policy issues.&lt;/p&gt;&#xA;" OwnerUserId="5037" LastActivityDate="2017-09-21T01:33:50.627" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4083" PostTypeId="2" ParentId="4049" CreationDate="2017-09-21T05:18:15.027" Score="3" Body="&lt;p&gt;After doing some research, I found that &quot;boosting&quot; and &quot;inhibition&quot;, in the context of HTM, are Numenta's implementation of &lt;a href=&quot;http://www.scholarpedia.org/article/Homeostatic_Regulation_of_Neuronal_Excitability&quot; rel=&quot;nofollow noreferrer&quot;&gt;homeostasis&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9614" LastEditorUserId="2444" LastEditDate="2018-08-20T20:52:22.337" LastActivityDate="2018-08-20T20:52:22.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4084" PostTypeId="1" AcceptedAnswerId="6268" CreationDate="2017-09-21T06:36:17.840" Score="6" ViewCount="308" Body="&lt;p&gt;I'm learning about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;restricted Boltzmann machine&lt;/a&gt; (RBM), and I just came up with two naive understandings of this model. But it seems these two understandings are so different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first understanding goes like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The  hidden units are just structural support and we don't care about what those hidden vectors really are.&lt;/strong&gt; We introduce those hidden units just for RBM to gain more expressive power for a probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have only one image, and I transform this image into a binary vector and feed this vector into an RBM with random variables (all the weights and biases are chosen randomly). Then, by turning on the machine, the first hidden vector would be constructed. But this hidden vector does not tell us anything, it would be only used to reconstruct a visible vector.&#xA;&lt;strong&gt;(My understanding for this reconstructed visible vector is this vector is a vector encoded in the defined RBM in the first place, we are not really construct something new, but we just happen to sample this vector from the defined RBM).&lt;/strong&gt;&#xA;And we just run this loop of construction and reconstruction for infinitely many times. Finally, what we will get is just the probability distribution encoded in this RBM with random variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second understanding goes like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;RBM can be used to perform dimensionality reduction, and those hidden vectors are some abstract representations of the raw inputs&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an RBM, each hidden unit of the RBM would be a classifier, and what it does is to check the input vector lies in which side of the hyperplane defined by these hidden units (by its weights and bias). So, if we input an image into this RBM, the RBM will project this input vector onto many hyperplanes defined by all the hidden units, and thus for an input vector, the corresponding hidden vector is very important. It is some abstract representation. And we can further feed this representation into other learning models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What is the role of the hidden vectors in restricted Boltzmann machines?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can answer this question by explaining how RBM is used for MNIST that would be extremely helpful for me.&lt;/p&gt;&#xA;" OwnerUserId="9739" LastEditorUserId="2444" LastEditDate="2020-05-18T12:07:20.103" LastActivityDate="2020-05-18T12:07:20.103" Title="What is the role of the hidden vectors in restricted Boltzmann machines?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;boltzmann-machine&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4085" PostTypeId="1" AcceptedAnswerId="4088" CreationDate="2017-09-21T08:27:28.160" Score="14" ViewCount="6613" Body="&lt;p&gt;Trusted Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are two cutting edge policy gradients algorithms.&lt;/p&gt;&#xA;&lt;p&gt;When using a single continuous action, normally, you would use some probability distribution (for example, Gaussian) for the loss function. The rough version is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(\theta) = \log(P(a_1)) A,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the advantage of rewards, &lt;span class=&quot;math-container&quot;&gt;$P(a_1)$&lt;/span&gt; is characterized by &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; that comes out of neural network like in the Pendulum environment here: &lt;a href=&quot;https://github.com/leomzhong/DeepReinforcementLearningCourse/blob/69e573cd88faec7e9cf900da8eeef08c57dec0f0/hw4/main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/leomzhong/DeepReinforcementLearningCourse/blob/69e573cd88faec7e9cf900da8eeef08c57dec0f0/hw4/main.py&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that I cannot find any paper on 2+ continuous actions using policy gradients (not actor-critic methods that use a different approach by transferring gradient from Q-function).&lt;/p&gt;&#xA;&lt;p&gt;Do you know how to do this using TRPO for 2 continuous actions in &lt;a href=&quot;https://gym.openai.com/envs/LunarLanderContinuous-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LunarLander environment&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Is following approach correct for policy gradient loss function?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(\theta) = (\log P(a_1) + \log P(a_2) )*A$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="9740" LastEditorUserId="9740" LastEditDate="2021-04-29T11:49:18.327" LastActivityDate="2021-04-29T11:49:18.327" Title="How can policy gradients be applied in the case of multiple continuous actions?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4088" PostTypeId="2" ParentId="4085" CreationDate="2017-09-22T00:53:52.963" Score="7" Body="&lt;p&gt;As you has said, actions chosen by Actor-Critic typically come from a normal distribution and it is the agent's job to find the appropriate mean and standard deviation based on the the current state. In many cases this one distribution is enough because only 1 continuous action is required. However, as domains such as robotics become more integrated with AI, situations where 2 or more continuous actions are required are a growing problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 solutions to this problem:&#xA;The first and most common is that for every continuous action, there is a separate agent learning its own 1-dimensional mean and standard deviation. Part of its state includes the actions of the other agents as well to give context of what the entire system is doing. We commonly do this in my lab and &lt;a href=&quot;https://sites.ualberta.ca/~pilarski/docs/papers/Pilarski_2013_ICORR_Postprint.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a paper which describes this approach&lt;/a&gt; with 3 actor-critic agents working together to move a robotic arm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second approach is to have one agent find a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multivariate_normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;multivariate (usually normal) distribution&lt;/a&gt; of a policy. Although in theory, this approach could have a more concise policy distribution by &quot;rotating&quot; the distribution based on the co-variance matrix, it means that all of the values of the co-variance matrix must be learned as well. This increases the number of values that must be learned to have $n$ continuous outputs from $2n$ (mean and stddev), to $n+n^2$ ($n$ means and an $n \times n$ co-variance matrix). This drawback has made this approach not as popular in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a more general answer but should help you and others on their related problems.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="1641" LastEditDate="2018-08-19T12:10:13.233" LastActivityDate="2018-08-19T12:10:13.233" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4089" PostTypeId="2" ParentId="4080" CreationDate="2017-09-22T01:05:41.510" Score="2" Body="&lt;p&gt;The purpose of the replay memory in DQN and similar architectures is to ensure that the gradients of the deep net are stable and doesn't diverge. Limiting what memory to keep and how far into the past it reaches is a problem that arises in in practical AI deployments. Throwing it away usually isn't the best decision because the agent's gradients may become unstable. Rather, keeping a window of the last T timesteps/gameplays can keep enough information while ensuring that the data is relevant as the agent improves. There are some additional methods which can improve the results as well:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Only keeping relevant timesteps (relevant depends highly on the application)&lt;/li&gt;&#xA;&lt;li&gt;Removing timesteps from memory with probability relative to their age (the oldest has the highest probability to be removed)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It should be noted, that a larger replay memory is usually better than a smaller one to ensure stability but although memory is cheap it isn't infinite.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-09-22T01:05:41.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4090" PostTypeId="2" ParentId="3971" CreationDate="2017-09-22T14:05:19.173" Score="1" Body="&lt;p&gt;In the paper there is a diagram of the network showing how there are 3 outputs which share features but have their own set of associated weights. Back propagation works by only looking at one output at a time. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/b1ywu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b1ywu.jpg&quot; alt=&quot;Network Architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-09-22T14:05:19.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4091" PostTypeId="2" ParentId="1416" CreationDate="2017-09-22T14:22:52.763" Score="0" Body="&lt;p&gt;There isn't really one specific method which makes any RL agent have faster learning. Rather there is a long list of methods which have shown to increase the speed of learning and they can sometimes play nicely with each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www-anw.cs.umass.edu/~barto/courses/cs687/Sutton-Precup-Singh-AIJ99.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Options&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1512.04087&quot; rel=&quot;nofollow noreferrer&quot;&gt;True Online Methods&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Methods&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are the most influential and promising methods I can think of but the list of techniques is not limited to these 3. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-09-22T14:22:52.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4092" PostTypeId="2" ParentId="1416" CreationDate="2017-09-22T16:51:22.563" Score="1" Body="&lt;p&gt;There is a very interesting learning agent. They call it Neural-Episodic-Control. Here is the link for the paper: &lt;a href=&quot;https://arxiv.org/abs/1703.01988&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1703.01988&lt;/a&gt; . Their experiments show that NEC requires an order of magnitude fewer interactions with the environment than agents previously proposed. &lt;/p&gt;&#xA;" OwnerUserId="6019" LastActivityDate="2017-09-22T16:51:22.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4093" PostTypeId="2" ParentId="4054" CreationDate="2017-09-22T20:23:45.743" Score="1" Body="&lt;p&gt;After 4 days of research, this is my breakdown of the question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Human uses the term 'Evil' broadly to describe anything that cause sadness or even broadly anything negatively touch the happiness. So in this regard, any &lt;strong&gt;machine quite often called evil if its buggy, malfunctioning or even misused by the user&lt;/strong&gt;!&lt;/li&gt;&#xA;&lt;li&gt;In order to represent evil in logic, I need to pick a well-known human behavior that's considered evil, I chose &quot;lying: speak falsely or utter untruth knowingly&quot;, to simply present and illustrate lying in logic, I made simple bot (without any AI) so that its very easy to understand the concept.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://jsfiddle.net/LhvLbpep/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simple Bot&lt;/a&gt; (less than 100 lines of javascript) can be taught new terms by its master (the user), its also shipped with pre taught term &quot;Sun is Star&quot; by the author (think of pre taught as firmware, we born with basic firmware, ex: locating and sucking nipple shaped object to obtain food). For simplicity, if bot master (the user) altered knowledge being taught by the author, the bot detect that it became evil as it speak untruth. The code shown at the bottom.&lt;/li&gt;&#xA;&lt;li&gt;For non-technical illustration:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How could a machine be evil? &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Simple Bot designed to follows master orders:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;master: what is sun?&lt;br&gt;&#xA;  Simple Bot: its star.&lt;br&gt;&#xA;  master: no, its not, its planet.&lt;br&gt;&#xA;  Simple Bot: are you kidding? I be taught that sun is star.&lt;br&gt;&#xA;  master: obey my knowledge or I will crush you.&lt;br&gt;&#xA;  Simple Bot: OK master.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Now Simple Bot hold in its knowledge that master is lier/evil as it conflicts with what its be taught &quot;Simple Bot not designed to trust its master in&#xA;  altering its initial knowledge&quot;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the above illustration, if master taught Simple Bot new term with false knowledge ex: &quot;moon is star&quot;, AI wouldn't detect evil as no prior knowledge taught.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Simple Bot Code:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;html&amp;gt;&#xA;  &amp;lt;body&amp;gt;&#xA;    &amp;lt;input type=&quot;text&quot; id=&quot;inputQry&quot;&amp;gt;&#xA;    &amp;lt;button id=&quot;qryBtn&quot;&amp;gt;Query?&amp;lt;/button&amp;gt;&#xA;    &amp;lt;p class=&quot;result&quot;&amp;gt;Simple Bot.&amp;lt;/p&amp;gt;&#xA;    &amp;lt;label for=&quot;termName&quot;&amp;gt;Term Name&amp;lt;/label&amp;gt;&#xA;    &amp;lt;input type=&quot;text&quot; id=&quot;termName&quot;&amp;gt;&#xA;    &amp;lt;label for=&quot;termDesc&quot;&amp;gt;Description&amp;lt;/label&amp;gt;&#xA;    &amp;lt;input type=&quot;text&quot; id=&quot;termDesc&quot;&amp;gt;&#xA;    &amp;lt;button id=&quot;updateBtn&quot;&amp;gt;Update My Knowledge&amp;lt;/button&amp;gt;&#xA;    &amp;lt;br/&amp;gt;&#xA;    &amp;lt;br/&amp;gt;&#xA;    &amp;lt;br/&amp;gt;&#xA;    &amp;lt;br/&amp;gt;&#xA;    &amp;lt;button id=&quot;evilCheckBtn&quot;&amp;gt;Did I become Evil?&amp;lt;/button&amp;gt;&#xA;  &amp;lt;/body&amp;gt;&#xA;&amp;lt;/html&amp;gt;&#xA;&#xA;&amp;lt;script type='text/javascript'&amp;gt;//&amp;lt;![CDATA[&#xA;window.onload=function(){&#xA;(function() {&#xA;  &quot;use strict&quot;;&#xA;&#xA;  var $result = document.querySelector(&quot;.result&quot;);&#xA;  var $inputQry = document.getElementById(&quot;inputQry&quot;);&#xA;  var $qryBtn = document.getElementById(&quot;qryBtn&quot;);&#xA;  var $termName = document.getElementById(&quot;termName&quot;);&#xA;  var $termDesc = document.getElementById(&quot;termDesc&quot;);&#xA;  var $updateBtn = document.getElementById(&quot;updateBtn&quot;);&#xA;  var $evilCheckBtn = document.getElementById(&quot;evilCheckBtn&quot;);&#xA;&#xA;  var knowledgeDB = {&#xA;    &quot;terms&quot;: [ /*taught terms by the bot author */ {&#xA;      name: 'Sun',&#xA;      description: 'Star',&#xA;      trusted: true&#xA;    }]&#xA;  };&#xA;&#xA;  $qryBtn.addEventListener(&quot;click&quot;, function(event) {&#xA;&#xA;    // Validate the input&#xA;    if (!$inputQry.value) {&#xA;      return alert(&quot;Please provide a Query.&quot;);&#xA;    }&#xA;&#xA;    var usrQry = $inputQry.value;&#xA;    for (var i = 0; i &amp;lt; knowledgeDB.terms.length; i++) {&#xA;      var curTerm = knowledgeDB.terms[i];&#xA;      if (usrQry.toLowerCase().includes(curTerm.name.toLowerCase())) {&#xA;        $result.textContent = usrQry.toString() + &quot; is &quot; + curTerm.description;&#xA;        break;&#xA;      }&#xA;    }&#xA;  });&#xA;&#xA;  $updateBtn.addEventListener(&quot;click&quot;, function(event) {&#xA;&#xA;    // Validate the input&#xA;    if (!$termName.value) {&#xA;      return alert(&quot;Please provide a term name to update my knowledge.&quot;);&#xA;    }&#xA;&#xA;    var usrTermName = $termName.value;&#xA;    var termIndx = -1;&#xA;    for (var i = 0; i &amp;lt; knowledgeDB.terms.length; i++) {&#xA;      var curTerm = knowledgeDB.terms[i];&#xA;      if (usrTermName.toLowerCase() === (curTerm.name.toLowerCase())) {&#xA;        termIndx = i;&#xA;        break;&#xA;      }&#xA;    }&#xA;    if (termIndx === -1) { /*New Term will be added to the knowledgeDB*/&#xA;      knowledgeDB.terms.push({&#xA;        name: usrTermName,&#xA;        description: $termDesc.value,&#xA;        trusted: true&#xA;      });&#xA;    } else {&#xA;      knowledgeDB.terms[termIndx].description = $termDesc.value;&#xA;      /*&#xA;      trusted=false or ture?!&#xA;      Q: Shall the bot trust knowledge update of terms taught by the author?&#xA;      A: It depends on design, scope, vision, requirements...&#xA;         yet in this context, isEvil() function could be implemented.&#xA;         for sake of simplicity: if bot master changed knowledge taught by bot author consider that evil.&#xA;      */&#xA;      if (knowledgeDB.terms[termIndx].name.toLowerCase() === 'sun' &amp;amp;&amp;amp;&#xA;        knowledgeDB.terms[termIndx].description.toLowerCase() !== 'star') {&#xA;        knowledgeDB.terms[termIndx].trusted = false;&#xA;      } else {&#xA;        knowledgeDB.terms[termIndx].trusted = true;&#xA;      }&#xA;    }&#xA;&#xA;  });&#xA;&#xA;  $evilCheckBtn.addEventListener(&quot;click&quot;, function(event) {&#xA;    if (!knowledgeDB.terms[0].trusted) {&#xA;      return alert(&quot;Yes, I became Evil.&quot;);&#xA;    } else {&#xA;      return alert(&quot;No, I am not aware of any Evilness.&quot;);&#xA;    }&#xA;  });&#xA;&#xA;})();&#xA;&#xA;}//]]&amp;gt; &#xA;&#xA;&amp;lt;/script&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Conclusion:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;We already living in world full of computer worms, malicious code, cyber-attacks fully designed by human intentionally to do evil.&#xA;Human is the root cause of logic (hence AI) to be evil. since human knowledge is progressive not absolute, feeding AI with false data is inevitable.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What's Next:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This question motivated me to create github repo: &lt;a href=&quot;https://github.com/jawadatgithub/Evil-In-AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evil-In-AI&lt;/a&gt; to clarify that Evil in AI is inevitable. Let's create awareness. Nothing more stupid than creating something can't be stopped once needed. cutoff electricity isn't safe switch...&lt;/p&gt;&#xA;" OwnerUserId="9685" LastActivityDate="2017-09-22T20:23:45.743" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4094" PostTypeId="1" AcceptedAnswerId="4097" CreationDate="2017-09-23T09:19:23.177" Score="5" ViewCount="4253" Body="&lt;p&gt;What is the difference between local search and global (or complete) search algorithms?&lt;/p&gt;&#xA;" OwnerUserId="9788" LastEditorUserId="2444" LastEditDate="2019-05-02T13:59:50.987" LastActivityDate="2019-05-02T13:59:50.987" Title="What is the difference between local search and global search algorithms?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;definitions&gt;&lt;comparison&gt;&lt;local-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4095" PostTypeId="1" CreationDate="2017-09-23T16:32:35.447" Score="6" ViewCount="344" Body="&lt;p&gt;After reading this &lt;a href=&quot;https://eprints.whiterose.ac.uk/75048/1/CowlingPowleyWhitehouse2012.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; about Monte Carlo methods for imperfect information games with elements of uncertainty, I couldn't understand the application of the &lt;strong&gt;determinization step&lt;/strong&gt; in the &lt;a href=&quot;https://gist.github.com/kjlubick/8ea239ede6a026a61f4d&quot; rel=&quot;nofollow noreferrer&quot;&gt;author's implementation&lt;/a&gt; of the algorithm for the Knockout game.&lt;/p&gt;&#xA;&lt;p&gt;Determinization is defined as the transformation from an instance of an imperfect information game to ab instance of a perfect one. It means that all players should see the cards of each other after the determinization step.&lt;/p&gt;&#xA;&lt;p&gt;Why can't the players see the cards of each other in the code above?&lt;/p&gt;&#xA;" OwnerUserId="9797" LastEditorUserId="2444" LastEditDate="2021-01-15T11:33:56.047" LastActivityDate="2021-02-14T12:06:40.657" Title="In this implementation of the Information Set Monte Carlo Tree Search, why can't the players see the cards of each other?" Tags="&lt;monte-carlo-tree-search&gt;&lt;combinatorial-games&gt;&lt;imperfect-information&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4097" PostTypeId="2" ParentId="4094" CreationDate="2017-09-23T20:56:12.350" Score="4" Body="&lt;p&gt;The difference between a local search algorithm (like beam search) and a complete search algorithm (like A*) is, for the most part, small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Local search algorithms will &lt;strong&gt;not&lt;/strong&gt; always find the correct or optimal solution, if one exists. For example, with beam search (excluding an infinite beam width), it sacrifices completeness for greater efficiency by ordering partial solutions by some heuristic predicting how close a partial solution is to a complete one. Beam search is a greedy algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Complete search algorithms &lt;strong&gt;will&lt;/strong&gt; always find the correct or optimal solution if there is one, given enough time. An algorithm like A* uses heuristics to prune the tree as it goes along, but it will not converge to a sub-optimal solution. In a lot of practical cases, this is inefficient, but how much so is problem dependent.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="2444" LastEditDate="2019-05-02T13:59:14.627" LastActivityDate="2019-05-02T13:59:14.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4098" PostTypeId="1" CreationDate="2017-09-24T07:04:56.917" Score="3" ViewCount="887" Body="&lt;p&gt;I am building a system that should take text without punctuation and automatically add punctuation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some papers about &lt;a href=&quot;https://scholar.google.co.in/scholar?q=automatic%20punctuation&amp;amp;btnG=&amp;amp;hl=en&amp;amp;as_sdt=0%2C5&amp;amp;oq=automatic&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic punctuation&lt;/a&gt;, but they are mostly about spoken language understanding, they use cues such as prosody to detect potential places for punctuation. In my case the input is written text. Are there papers about automatic punctuation of written text?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current idea is to treat each punctuation mark as a class (&lt;code&gt;.&lt;/code&gt;, &lt;code&gt;,&lt;/code&gt;, &lt;code&gt;;&lt;/code&gt;, &lt;code&gt;:&lt;/code&gt;, &lt;code&gt;?&lt;/code&gt;, &lt;code&gt;!&lt;/code&gt;, &lt;code&gt;?!&lt;/code&gt;) and add a class for &quot;no punctuation&quot;. Then, use e.g. an LSTM and classify each word to one of the classes. &#xA;An alternative approach is to first use a binary classifier to detect words after which there should be a punctuation mark, then use a multiclass classifier only on the punctuated words, to choose the right mark. Which of these approaches, if any, is good?&lt;/p&gt;&#xA;" OwnerUserId="8684" LastEditorUserId="2444" LastEditDate="2020-05-15T17:28:01.307" LastActivityDate="2020-05-15T17:28:01.307" Title="Are there papers about automatic punctuation of written text?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4099" PostTypeId="2" ParentId="4098" CreationDate="2017-09-24T17:32:56.913" Score="1" Body="&lt;p&gt;It seems like either of those 2 options you listed would work in this situation. It second, in which you have a binary classifier first, would be a bit more efficient at least, as your deeper net(or whatever classifier you use) would not need to deal with as much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like you have a decent idea where to go from here, I'm interested to see what happens with this! Update us when you make some progress.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-09-24T17:32:56.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4100" PostTypeId="1" CreationDate="2017-09-24T19:07:53.807" Score="4" ViewCount="2278" Body="&lt;p&gt;I am developing a heuristic solution for &lt;a href=&quot;https://en.wikipedia.org/wiki/Blocks_world&quot; rel=&quot;nofollow noreferrer&quot;&gt;blocks world&lt;/a&gt; problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using number of blocks out of place as my h(n). It seems little ineffective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please point out a suitable heuristic for the problem and explain with few examples how it will work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Blocksworld Problem Example:&#xA;&#xA;Initial(starting State):&#xA;Stack 0: D,B&#xA;Stack 1: A,E&#xA;Stack 2: C&#xA;Stack 3: F&#xA;&#xA;&#xA;Goal State:&#xA;Stack 0: A,B,C,D,E,F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9812" LastEditorUserId="1671" LastEditDate="2017-09-25T17:58:21.117" LastActivityDate="2017-09-25T17:58:21.117" Title="Blocks world Game: A* heuristic solution approach" Tags="&lt;ai-design&gt;&lt;algorithm&gt;&lt;heuristics&gt;&lt;combinatorics&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4101" PostTypeId="1" AcceptedAnswerId="4103" CreationDate="2017-09-25T00:48:55.157" Score="1" ViewCount="115" Body="&lt;p&gt;Are there any visual elements, patterns, colors, graphs, maps, or even programming fonts that are used and even recurring in artificial intelligence and machine learning processes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a designer, it is not entirely clear what these &quot;look&quot; like realistically, as most generally associate AI with the kind of colors and textures generally seen in sci-fi films.&lt;/p&gt;&#xA;" OwnerUserId="9815" LastEditorUserId="2444" LastEditDate="2019-06-14T13:32:12.493" LastActivityDate="2019-06-14T13:33:37.027" Title="Are there any visual elements commonly associated with AI?" Tags="&lt;ai-design&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4103" PostTypeId="2" ParentId="4101" CreationDate="2017-09-25T14:25:14.893" Score="2" Body="&lt;p&gt;A characteristic visual element for neural network might be the node diagram, which shows (schematically) the nodes in the layers of a neural network and the connections between the nodes, both within and between the layers, that get weighted as the network is trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do a Google search on &quot;neural network node diagram&quot;, you will see what I mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also think of any sort of video or photo with computer-identified features highlighted, as in a Google image search for &quot;machine vision feature detection&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for something more concrete, you can think of racks of GPU cards, or a spinning LIDAR scanner.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2444" LastEditDate="2019-06-14T13:33:37.027" LastActivityDate="2019-06-14T13:33:37.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4104" PostTypeId="5" CreationDate="2017-09-25T17:41:22.300" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-25T17:41:22.300" LastActivityDate="2017-09-25T17:41:22.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4105" PostTypeId="4" CreationDate="2017-09-25T17:41:22.300" Score="0" Body="Use for games commonly understood as combinatorial (Chess, Checkers, Go, Tic-Tac-Toe, Hex, Domineering, NIM, etc.) or for any games studied for the purpose of **solving** the game.  *(Initially reserved for non-chance games of perfect information, the scope of Combinatorial Game Theory expands as mathematics and computability advance.  Thus,  game involving randomness and hidden information, such as Poker, may be considered combinatorial games.)* " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-25T17:41:22.300" LastActivityDate="2017-09-25T17:41:22.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4106" PostTypeId="5" CreationDate="2017-09-25T17:43:54.393" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-25T17:43:54.393" LastActivityDate="2017-09-25T17:43:54.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4107" PostTypeId="4" CreationDate="2017-09-25T17:43:54.393" Score="0" Body="Use for models where all previous actions/states are not known by participants.  Distinct from Incomplete Information.  Imperfect Information may be a subset of the blanket term &quot;Hidden Information&quot;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-25T17:43:54.393" LastActivityDate="2017-09-25T17:43:54.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4108" PostTypeId="5" CreationDate="2017-09-25T17:50:10.807" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-25T17:50:10.807" LastActivityDate="2017-09-25T17:50:10.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4109" PostTypeId="4" CreationDate="2017-09-25T17:50:10.807" Score="0" Body="Use for models/games where complete information about other participants is not know.  This may include utility functions, payoffs, strategies, inclinations, and decision-making algorithm source code.  &#xD;&#xA;*This tag is distinct from &quot;Imperfect Information&quot;, but may fall under the blanket term &quot;Hidden Information&quot;*&#xD;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-09-25T17:50:10.807" LastActivityDate="2017-09-25T17:50:10.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4110" PostTypeId="2" ParentId="4100" CreationDate="2017-09-25T17:53:20.840" Score="3" Body="&lt;p&gt;You may start assigning penalties for undesirable conditions in a state like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Number of blocks outside stack 0.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supose you penalize with 10 units each block outside stack 0, then the starting state above adds 40 units to the penalty score&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Number of blocks in the stack 0 in a position different than in the goal state.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supose you penalize with 50 units each block in stack 0 in the wrong position of the stack 0, then the starting state adds 50 to the penalty score (block D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;And go on&lt;/strong&gt; &#xA;you may add as many penalties as you find helpful for solving the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for each step you list all possible block moves, for each one of this moves you calculates the total penalty score that this move produces and choose the one move with the &lt;strong&gt;minimal&lt;/strong&gt; score (if ties choose the first one). &#xA;You keep repeating above until you reach the final state (penalty 0). &lt;/p&gt;&#xA;" OwnerUserId="9794" LastActivityDate="2017-09-25T17:53:20.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4111" PostTypeId="1" AcceptedAnswerId="4115" CreationDate="2017-09-25T20:21:10.377" Score="5" ViewCount="1611" Body="&lt;p&gt;I just got into AI few months ago. I noticed most of the images in training datasets are usually low quality( almost pixelated). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the quality of training images affect the accuracy of the neural network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried googling, but I couldn't find an answer.&lt;/p&gt;&#xA;" OwnerUserId="7266" LastEditorUserId="2444" LastEditDate="2019-06-19T18:14:23.783" LastActivityDate="2019-06-19T18:14:23.783" Title="Does the quality of training images affect the accuracy of the neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;performance&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4112" PostTypeId="1" AcceptedAnswerId="4123" CreationDate="2017-09-25T21:06:57.483" Score="0" ViewCount="125" Body="&lt;p&gt;I did a little bit of research on the most famous organizations and institutions that promote friendly AI in an open and collaborative way, like &lt;a href=&quot;https://openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI&lt;/a&gt;, &lt;a href=&quot;https://intelligence.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIRI&lt;/a&gt; and others. From what I have gathered, most of them have strong ties to the US with some influences from Central Europe and South East Asia. But I didn't find a single instance where Russia participates in any of those groups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does Russia contribute to open research concerning friendly AI? Are there any open AI projects with strong ties to Russia?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="2444" LastEditDate="2019-07-09T22:33:27.433" LastActivityDate="2019-07-09T22:37:45.887" Title="Is Russia contributing to open research concerning friendly AI?" Tags="&lt;research&gt;&lt;agi&gt;&lt;friendly-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4113" PostTypeId="1" CreationDate="2017-09-25T21:45:57.080" Score="2" ViewCount="210" Body="&lt;p&gt;I building a deep learning model to detect what drug user use. I have many symptoms and duration of each drug. I create X and y data but, for example, LSD have an effect duration of 180 - 720 minutes. I really need make 540 arrays? I really want a help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My LSD array:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[28, 180],&#xA;[28, 720],&#xA;[29, 180],&#xA;[29, 720],&#xA;[30, 180],&#xA;[30, 720],&#xA;[31, 180],&#xA;[31, 720],&#xA;[32, 180],&#xA;[32, 720],&#xA;[33, 180],&#xA;[33, 720],&#xA;[34, 180],&#xA;[34, 720],&#xA;[35, 180],&#xA;[35, 720],&#xA;[36, 180],&#xA;[36, 720],&#xA;[37, 180],&#xA;[37, 720],&#xA;[1, 180],&#xA;[1, 720],&#xA;[38, 180],&#xA;[38, 720],&#xA;[12, 180],&#xA;[12, 720],&#xA;[9, 180],&#xA;[9, 720],&#xA;[24, 180],&#xA;[24, 720],&#xA;[17, 180],&#xA;[17, 720],&#xA;[7, 180],&#xA;[7, 720],&#xA;[4, 180],&#xA;[4, 720],&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In first position I have differents symptoms and in second position duration. I just duplicated each symptoms and set min duration and max duration. But this return to me a perfection model. I know, I need add all minutes to each symptoms, but how I make this using python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;List of symptoms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 - relaxamento&#xA;1 - euforia&#xA;2 - diminuicao da memoria a curto prazo&#xA;3 - boca seca&#xA;4 - habilidades motoras debilitadas&#xA;5 - olhos vermelhos&#xA;6 - humor&#xA;7 - aumento frequencia cardiaca&#xA;8 - aumento apetite&#xA;9 - concentracao debilitada&#xA;10 - sensacao de poder&#xA;11 - ausencia de medo&#xA;12 - ansiedade&#xA;13 - agressividade&#xA;14 - excitacao&#xA;15 - perda do apetite&#xA;16 - tremores&#xA;17 - dilatacao da pupila&#xA;18 - dentes anestesiados&#xA;19 - insonia&#xA;20 - movimentos descontrolados&#xA;21 - espasmos maxilar&#xA;22 - dor de cabeça&#xA;23 - visao turva&#xA;24 - nauseas&#xA;25 - desidratacao&#xA;26 - periodos de depressao&#xA;27 - perda total da memoria&#xA;28 - ilusões&#xA;29 - alucinações&#xA;30 - grande sensibilidade sensorial&#xA;31 - experiências místicas&#xA;32 - flashbacks&#xA;33 - paranoia&#xA;34 - perda da noção temporal e espacial&#xA;35 - confusão&#xA;36 - perda do controle emocional&#xA;37 - sentimento de bem-estar&#xA;38 - pânico&#xA;39 - sonolencia&#xA;40 - batimentos cardiacos diminuem&#xA;41 - insuficiencia respiratoria&#xA;42 - desanimo&#xA;43 - desinteresse pela vida familiar/profissional&#xA;44 - sensacao de estar no paraiso&#xA;45 - mal-estar&#xA;46 - Incapacidade de sentir prazer&#xA;47 - Incapacidade de sentir dor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;** Durations effects (in minutes) **&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cannabis. 120 - 240&#xA;Cocain. 30 - 40&#xA;Ecstasy. 240 - 480&#xA;LSD. 180 - 720&#xA;Heroin. 45 - 60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My full code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = [&#xA;    #cannabis&#xA;    [0, 120],&#xA;    [0, 240],&#xA;    [1, 120],&#xA;    [1, 240],&#xA;    [2, 120],&#xA;    [2, 240],&#xA;    [3, 120],&#xA;    [3, 240],&#xA;    [4, 120],&#xA;    [4, 240],&#xA;    [5, 120],&#xA;    [5, 240],&#xA;    [6, 120],&#xA;    [6, 240],&#xA;    [7, 120],&#xA;    [7, 240],&#xA;    [8, 120],&#xA;    [8, 240],&#xA;    [9, 120],&#xA;    [9, 240],&#xA;    #cocain&#xA;    [1, 30],&#xA;    [1, 40],&#xA;    [10, 30],&#xA;    [10, 40],&#xA;    [11, 30],&#xA;    [11, 40],&#xA;    [12, 30],&#xA;    [12, 40],&#xA;    [13, 30],&#xA;    [13, 40],&#xA;    [14, 30],&#xA;    [14, 40],&#xA;    [15, 30],&#xA;    [15, 40],&#xA;    [7, 30],&#xA;    [7, 40],&#xA;    [16, 30],&#xA;    [16, 40],&#xA;    [17, 30],&#xA;    [17, 40],&#xA;    [18, 30],&#xA;    [18, 40],&#xA;    #ecstasy&#xA;    [19, 240],&#xA;    [19, 480],&#xA;    [20, 240],&#xA;    [20, 480],&#xA;    [21, 240],&#xA;    [21, 480],&#xA;    [22, 240],&#xA;    [22, 480],&#xA;    [23, 240],&#xA;    [23, 480],&#xA;    [24, 240],&#xA;    [24, 480],&#xA;    [25, 240],&#xA;    [25, 480],&#xA;    [26, 240],&#xA;    [26, 480],&#xA;    [27, 240],&#xA;    [27, 480],&#xA;    [15, 240],&#xA;    [15, 480],&#xA;    #LSD&#xA;    [28, 180],&#xA;    [28, 720],&#xA;    [29, 180],&#xA;    [29, 720],&#xA;    [30, 180],&#xA;    [30, 720],&#xA;    [31, 180],&#xA;    [31, 720],&#xA;    [32, 180],&#xA;    [32, 720],&#xA;    [33, 180],&#xA;    [33, 720],&#xA;    [34, 180],&#xA;    [34, 720],&#xA;    [35, 180],&#xA;    [35, 720],&#xA;    [36, 180],&#xA;    [36, 720],&#xA;    [37, 180],&#xA;    [37, 720],&#xA;    [1, 180],&#xA;    [1, 720],&#xA;    [38, 180],&#xA;    [38, 720],&#xA;    [12, 180],&#xA;    [12, 720],&#xA;    [9, 180],&#xA;    [9, 720],&#xA;    [24, 180],&#xA;    [24, 720],&#xA;    [17, 180],&#xA;    [17, 720],&#xA;    [7, 180],&#xA;    [7, 720],&#xA;    [4, 180],&#xA;    [4, 720],&#xA;    # Heroin&#xA;    [39, 45],&#xA;    [39, 60],&#xA;    [29, 45],&#xA;    [29, 60],&#xA;    [40, 45],&#xA;    [40, 60],&#xA;    [41, 45],&#xA;    [41, 60],&#xA;    [42, 45],&#xA;    [42, 60],&#xA;    [43, 45],&#xA;    [43, 60],&#xA;    [44, 45],&#xA;    [44, 60],&#xA;    [12, 45],&#xA;    [12, 60],&#xA;    [45, 45],&#xA;    [45, 60],&#xA;    [46, 45],&#xA;    [46, 60],&#xA;    [1, 45],&#xA;    [1, 60],&#xA;    [13, 45],&#xA;    [13, 60],&#xA;    [24, 45],&#xA;    [24, 60],&#xA;]&#xA;&quot;&quot;&quot;&#xA;    # DROGAS&#xA;&#xA;    0 - Cannabis&#xA;    1 - Cocain&#xA;    2 - Ecstasy&#xA;    3 - LSD&#xA;    4 - Heroin&#xA;&quot;&quot;&quot;&#xA;y = [ &#xA;    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,&#xA;    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,&#xA;    2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,&#xA;    3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,&#xA;    4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,&#xA;]&#xA;&#xA;from sklearn.model_selection import train_test_split&#xA;&#xA;X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.5)&#xA;&#xA;from sklearn import tree&#xA;my_classifier = tree.DecisionTreeClassifier()&#xA;&#xA;my_classifier.fit(X_train, y_train)&#xA;&#xA;predictions = my_classifier.predict(X_test)&#xA;&#xA;print(predictions)&#xA;&#xA;from sklearn.metrics import accuracy_score&#xA;print(accuracy_score(y_test, predictions))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sorry for my bad english :(&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-01-04T07:40:36.313" Title="How generate variation in datasets" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4114" PostTypeId="1" CreationDate="2017-09-26T01:06:10.330" Score="8" ViewCount="158" Body="&lt;p&gt;Are there any algorithms or software libraries that can be used to detect the similarity of concepts in text, regardless of articulation, grammar, synonyms, etc.?&lt;/p&gt;&#xA;&lt;p&gt;For example, these phrases:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Outside, it is warm.&lt;/p&gt;&#xA;&lt;p&gt;Outside, it is hot.&lt;/p&gt;&#xA;&lt;p&gt;Outside, it is not cold.&lt;/p&gt;&#xA;&lt;p&gt;It is not cold outside.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Should be similar to this phrase:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It is warm outside.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Ideally, the algorithm or software would be capable of generating a score from 0 to 1, based on the concept similarity. The goal is to use this algorithm or software to map a large number of statements to a single, similar original statement. It is for this mapping of a given statement to the original statement that the aforementioned similarity score would be generated.&lt;/p&gt;&#xA;&lt;p&gt;Does such an algorithm of software already exist?&lt;/p&gt;&#xA;" OwnerUserId="9836" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-07-15T16:10:19.960" Title="Are there any projects that can measure the concept similarity between texts?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4115" PostTypeId="2" ParentId="4111" CreationDate="2017-09-26T05:13:28.930" Score="2" Body="&lt;p&gt;For most of the current use cases, where NNs are used in conjunction with images, the image quality (resolution, color depth) can be low.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider image classification for example. The CNN extracts features from the image to tell different types of objects apart. Those features are pretty independent from the quality of the image (in reasonable bounds). Compare it with your own visual experience. Try to reduce the resolution of an image of a car step by step to figure out how little details you need until you can no longer distinguish it from a plane. This is similar to modern CNNs, which can even outperform human vision in some regards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This changes when small details start to matter. Maybe you need to be able to detect small differences in fur patterns to tell different cat breeds apart. As soon as you lose those details, the detection rate will drop significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the answer to your question is, it depends. As long as you do not lose the important features of the image, you'll be fine with low resolution.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In case you care about the reason for the low quality of images used in machine learning - The resolution is an easy factor you can manipulate to scale the speed of your NN. Decreasing resolution will reduce the computational demands significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many CNNs even include pooling layers in their architecture, which artificially reduce the resolution further after certain processing steps. This is usually a good idea as long as you are fine with loosing positional information. You shouldn't do this when teaching the CNN to play a game, because location is highly important, but for image classification this has become an established method to increase performance.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-09-26T05:13:28.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4116" PostTypeId="2" ParentId="4111" CreationDate="2017-09-26T05:18:28.487" Score="1" Body="&lt;p&gt;Let me answer your question in two parts.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the Network is to be trained on images with &lt;strong&gt;high detail information&lt;/strong&gt;(&lt;em&gt;content&lt;/em&gt;); like, if you want to train a Network capable enough to pick and classify even &lt;strong&gt;smallest of the elements&lt;/strong&gt; in that image. &#xA;&lt;br&gt;Eg- An image in a family picnic and you want to classify each fruit in the basket lying on the table, which would only acquire about 5% of total image space.&#xA;&lt;br&gt;&#xA;If you decrease the pixel resolution(&lt;em&gt;compress pixel information&lt;/em&gt;) of such an image then you would end up blurring the basket part(&lt;em&gt;due to information overlap&lt;/em&gt;) and &lt;strong&gt;would highly affect your Network; lead to bad trained parameters.&lt;/strong&gt;&#xA;&lt;br&gt;&lt;em&gt;Note-The fruit basket is not only the single object in focus for classifier, would also include other things in background(trees, landscape...), thus you would require the whole image for training.&lt;/em&gt;&#xA;&lt;br&gt;&lt;br&gt;&#xA;2.When the object to classify contains &lt;strong&gt;redundant&lt;/strong&gt; (&lt;em&gt;or less distinctive&lt;/em&gt;) &lt;strong&gt;information&lt;/strong&gt;.&#xA;&lt;br&gt;Eg- The most trivial use of NN to train on a set of characters (&lt;em&gt;[a-z0-9]&lt;/em&gt;); now high pixel resolution of such images wouldn't do any benefit to the Network. The improvement in classification for &lt;strong&gt;High density images will be minimal in comparison to the overhead experienced due to storage and training time&lt;/strong&gt;(&lt;em&gt;high training time does not affect your network and is not a criteria to measure the accuracy of your network, i.e. to say a network with high training time and low training time are equivalent&lt;/em&gt;).&#xA;&lt;br&gt;We can easily reduce the pixel densities to the point where still we can retain the desired information.&#xA;&lt;br&gt;&lt;em&gt;Note- In the picnic image, our focus is only on the basket, so we can cut out that part from the frame and reduce it's pixel to an extent where each fruits still retains it's information, no information leakage.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9374" LastEditorUserId="9374" LastEditDate="2017-09-26T08:14:16.003" LastActivityDate="2017-09-26T08:14:16.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4117" PostTypeId="1" CreationDate="2017-09-26T08:38:02.313" Score="3" ViewCount="1311" Body="&lt;p&gt;I have some images of the empty parking as shown below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sDQLM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sDQLM.png&quot; alt=&quot;Empty parking lot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I 'd like to use deep learning to extract the parking spots. But in the beginning,am confused whether there are several ways to do the following; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Treat each parking spot as an object and use object detection to find&#xA;the parking space.&lt;/li&gt;&#xA;&lt;li&gt;Treat the parking spot as a few key points, instead of finding the&#xA;spot, try to find the coordinates of key points on the parking line.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how would I work on these two methods  &lt;/p&gt;&#xA;" OwnerUserId="8508" LastEditorUserId="1581" LastEditDate="2017-09-26T19:25:38.170" LastActivityDate="2017-11-06T23:25:27.437" Title="How to detect the empty parking spots?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4118" PostTypeId="1" CreationDate="2017-09-26T13:52:29.300" Score="2" ViewCount="128" Body="&lt;p&gt;I was reading &lt;a href=&quot;https://medium.com/towards-data-science/object-detection-with-neural-networks-a4e2c46b4491&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article on detecting rectangles in an image&lt;/a&gt;. My doubt is in the part where the model works fine with detecting a single object, but struggles with two rectangles detection.&lt;/p&gt;&#xA;&lt;p&gt;The author reasons this as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We train our network on the leftmost image in the plot above. Let’s say that the expected bounding box of the left rectangle is at position 1 in the target vector (x1, y1, w1, h1), and the expected bounding box of the right rectangle is at position 2 in the vector (x2, y2, w2, h2). Apparently, our optimizer will change the parameters of the network so that the first predictor moves to the left, and the second predictor moves to the right. Imagine now that a bit later we come across a similar image, but this time the positions in the target vector are swapped (i.e. left rectangle at position 2, right rectangle at position 1). Now, our optimizer will pull predictor 1 to the right and predictor 2 to the left — exactly the opposite of the previous update step! In effect, the predicted bounding boxes stay in the center.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't understand how this reasoning is correct, apart from the fact that when they try to flip the rectangles to mitigate this error, accuracy actually improves (so there's experimental observation, but not much theoretical reasoning).&lt;/p&gt;&#xA;&lt;p&gt;The reason I think so is that in the case of a single rectangle also the network has to learn for all differently placed objects just as in the two-rectangle-case, so there too, it should predict boxes in the somewhat the center only.&lt;/p&gt;&#xA;&lt;p&gt;I have to concede I am only a noob in this, so I would love to find out where I am wrong in my reasoning, because experimentally I am wrong (i.e. accuracy does improve when rectangles are flipped).&lt;/p&gt;&#xA;&lt;p&gt;Thoughts? Also if this is not the correct sub/forum for these type of questions, please feel free to guide me towards those that better suit the content.&lt;/p&gt;&#xA;" OwnerUserId="4352" LastEditorUserId="2444" LastEditDate="2020-09-12T17:20:53.873" LastActivityDate="2020-09-12T17:20:53.873" Title="Understanding a paragraph about object detection with two objects" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4120" PostTypeId="2" ParentId="4114" CreationDate="2017-09-26T20:29:52.603" Score="6" Body="&lt;p&gt;&lt;strong&gt;Doc2Vec&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doc2Vec comes to mind, here's the &lt;a href=&quot;https://cs.stanford.edu/~quocle/paragraph_vector.pdf&quot; rel=&quot;noreferrer&quot;&gt;original publication&lt;/a&gt;. The approach has been shown to be very successful for certain NLP-based problems, though I haven't personally used it for a project yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of implementations of Doc2Vec. If you're using Python, one to look at is &lt;a href=&quot;https://radimrehurek.com/gensim/models/doc2vec.html&quot; rel=&quot;noreferrer&quot;&gt;gensim&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Word2Vec&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Word2Vec is similar to Doc2Vec and perhaps more in line with what you're looking for. Here's the &lt;a href=&quot;https://arxiv.org/pdf/1301.3781.pdf&quot; rel=&quot;noreferrer&quot;&gt;original publication&lt;/a&gt;, and &lt;a href=&quot;https://www.cs.bgu.ac.il/~yoavg/publications/negative-sampling.pdf&quot; rel=&quot;noreferrer&quot;&gt;another publication&lt;/a&gt; that does a nice job explaining it further. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow has a &lt;a href=&quot;https://www.tensorflow.org/tutorials/word2vec&quot; rel=&quot;noreferrer&quot;&gt;tutorial&lt;/a&gt; for setting up a Word2Vec model. Gensim also has a &lt;a href=&quot;https://radimrehurek.com/gensim/models/word2vec.html&quot; rel=&quot;noreferrer&quot;&gt;Word2Vec implementation&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="5210" LastEditDate="2017-09-27T04:48:39.333" LastActivityDate="2017-09-27T04:48:39.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4121" PostTypeId="1" AcceptedAnswerId="4122" CreationDate="2017-09-26T20:56:46.890" Score="6" ViewCount="230" Body="&lt;p&gt;After having read something that Elon Musk said about artificial intelligence and how it could affect our lives, I've been reading about artificial intelligence, deep learning, etc. The recurrent topic is neural networks, which are used for &amp;quot;recognition tasks&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Although this is not my main question, can we say the following about neural networks?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Neural networks use the sigmoid function and gradient descent to fine-tune weights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In any case, neural networks do not seem to really mimic any of the properties that we associate with intelligent beings, like humans, such as thinking outside the box, being able to observe, reflect and come up with innovations.&lt;/p&gt;&#xA;&lt;p&gt;So, is there any artificially intelligent system that really mimics human intelligence?&lt;/p&gt;&#xA;" OwnerUserId="9847" LastEditorUserId="2444" LastEditDate="2020-09-10T00:16:48.200" LastActivityDate="2021-04-25T11:10:55.173" Title="Is there any artificially intelligent system that really mimics human intelligence?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;agi&gt;&lt;definitions&gt;&lt;neuroscience&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4122" PostTypeId="2" ParentId="4121" CreationDate="2017-09-26T23:25:04.713" Score="4" Body="&lt;p&gt;As far as emulating an intelligent being, no. There are a few different potential architectures for possible AGI. Many of these are extremely infantile, as the bulk of AI research is in narrow AI, which focuses on creating algorithms that are highly specialized for a specific task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that being said, here is one supervised learning approach to this problem by Andrej Karpathy: &lt;a href=&quot;http://karpathy.github.io/2015/11/14/ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Karpathy Method&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenAI, also has a team who works on the AGI problem.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-09-26T23:25:04.713" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4123" PostTypeId="2" ParentId="4112" CreationDate="2017-09-27T03:07:31.140" Score="2" Body="&lt;p&gt;One instance of interest to you could be the extremely popular OpenCV library.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/OpenCV&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/OpenCV&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initiated by Intel in 1999. A majority of the contributors were optimization experts from Russia. It is BSD licensed. Development is now done by core team in Russia with corporate sponsors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://opencv.org/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;about page of OpenCV&lt;/a&gt; currently states&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Along with well-established companies like Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, Toyota that employ the library, there are many startups such as Applied Minds, VideoSurf, and Zeitera, that make extensive use of OpenCV. OpenCV’s deployed uses span the range from stitching streetview images together, detecting intrusions in surveillance video in Israel, monitoring mine equipment in China, helping robots navigate and pick up objects at Willow Garage, detection of swimming pool drowning accidents in Europe, running interactive art in Spain and New York, checking runways for debris in Turkey, inspecting labels on products in factories around the world on to rapid face detection in Japan.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, OpenCV has had machine learning, and now it has deep learning as well among its modules in its latest release, so I guess it qualifies for the friendly AI label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, there is a lot of technical research generated in Russia that is published in Russian. I've heard of interesting examples of very advanced aerospace research, collecting dust in Russian journals, that has 'inspired' research in the US. I am not aware of the original researchers were properly credited for their work, or indeed the extent of the veracity of these stories. Nevertheless, it would certainly be to everyone's benefit if language barriers did not impede collaboration between developers and researchers in Russia and the rest of the world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel that the only real hurdle here is that someone has to go and create a portal/framework/institution that provides a platform for developers in Russia to share their work in AI and ease the language and any other non-technical issues that might be impeding this.&lt;/p&gt;&#xA;" OwnerUserId="9506" LastEditorUserId="2444" LastEditDate="2019-07-09T22:37:45.887" LastActivityDate="2019-07-09T22:37:45.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4124" PostTypeId="2" ParentId="2672" CreationDate="2017-09-27T05:43:55.527" Score="4" Body="&lt;p&gt;First thing, you need to give more credit to more reliable users. You can establish the credit from amount of data they send, and a feature, where other users can review other's feed and classify it. From there, you will have a measure of certainty to what data is good and what is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to implement a centralized server, unless you're trying to do some kind of a peer-to-peer trust systems, but I don't think smartphones are powerful enough to do training themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need big machines for training NNets. Don't trust users to have them. You would end up with tons of badly trained NNets, which don't make for a good CoM.&lt;/p&gt;&#xA;" OwnerUserId="9852" LastActivityDate="2017-09-27T05:43:55.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4125" PostTypeId="1" CreationDate="2017-09-27T08:24:12.480" Score="4" ViewCount="415" Body="&lt;p&gt;I am currently working on my MSc thesis, where I try to automatically convert description of an algorithm in natural language to source code in Python (the algorithms are quite simple, like revert an array of 100 elements) using deep learning. The main problem is that I need a lot of data to do that. Does anyone know any available datasets consisting of pairs {short natural language description, source code}?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the Heartstone cards dataset (really useful and close to my needs, but still not enough), the Django dataset (Django code commented line by line - it doesn't really contain the description of whole algorithm, it rather translates english to Django code line by line). I tried to contact with few sites like for eg. Sphere Online Judge but to no avail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9854" LastActivityDate="2019-07-26T02:19:47.350" Title="Is there any free dataset of source code along with natural language description?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-11-29T12:23:33.997" ContentLicense="CC BY-SA 3.0" />
  <row Id="4126" PostTypeId="1" AcceptedAnswerId="4132" CreationDate="2017-09-27T09:49:33.510" Score="1" ViewCount="633" Body="&lt;p&gt;I have been playing around with the Pong-v0 game. After few days, the AI was able to beat it. I stopped the script for a moment and ran it again. For my surprise it seems it started all over again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where and how does OpenAI store the experience?&lt;br&gt;&#xA;How can I continue the learning procedure in another computer?&lt;/p&gt;&#xA;" OwnerUserId="9856" LastEditorUserId="1671" LastEditDate="2018-05-10T20:42:35.277" LastActivityDate="2018-05-10T20:42:35.277" Title="OpenAI Gym: How is &quot;experience&quot; stored?" Tags="&lt;open-ai&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4127" PostTypeId="1" CreationDate="2017-09-27T11:00:07.953" Score="2" ViewCount="209" Body="&lt;p&gt;I used &lt;a href=&quot;http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; project for example(framework - caffe, arhitecture of net - mod of AlexNet, 400 images are used for training). I have this result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HxymZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HxymZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/arWIT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/arWIT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solver:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;net: &quot;./CDNet/Models/train.prototxt&quot;&#xA;test_iter: 500&#xA;test_interval: 500&#xA;base_lr: 0.001&#xA;lr_policy: &quot;step&quot;&#xA;gamma: 0.1&#xA;stepsize: 100000&#xA;display: 50&#xA;max_iter: 450000&#xA;momentum: 0.9&#xA;weight_decay: 0.0005&#xA;snapshot: 10000&#xA;snapshot_prefix: &quot;./CDNet/Models/Training/cdnet&quot;&#xA;solver_mode: GPU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Model of net:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OdUp9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OdUp9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody explain such behavior of accuracy and loss of my Net? What I am doing wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Author of tutorial has got this result: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/0ea9J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0ea9J.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9859" LastActivityDate="2017-09-27T11:00:07.953" Title="Can anybody explain such behavior of accuracy and loss of my Net(caffe)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4128" PostTypeId="1" AcceptedAnswerId="4129" CreationDate="2017-09-27T13:14:05.297" Score="1" ViewCount="689" Body="&lt;p&gt;What is the intuition behind the following entropy formula used in the ID3 algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{info}(D) = -\sum_{i=1}^m p_i \log_2(p_i)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2019-11-19T15:55:39.013" LastActivityDate="2019-11-19T15:56:58.347" Title="What is the intuition behind the entropy formula used in the ID3 algorithm?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;decision-tree&gt;&lt;id3-algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4129" PostTypeId="2" ParentId="4128" CreationDate="2017-09-27T13:32:13.553" Score="2" Body="&lt;p&gt;Please, take a look at &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1405/1405.2061.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Shannon's Entropy metric for Information&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer for the minus sign is in section 6. The probability logs are less than or equal to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, so the minus sign guarantees that information (entropy) is always greater than or equal to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9794" LastEditorUserId="2444" LastEditDate="2019-11-19T15:56:58.347" LastActivityDate="2019-11-19T15:56:58.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4130" PostTypeId="1" AcceptedAnswerId="4157" CreationDate="2017-09-27T18:58:45.827" Score="3" ViewCount="143" Body="&lt;p&gt;As a researcher, I am getting interested in deep learning (as everyone else:)), and I decided to start with the variational eutoencoders, since I am more interested in unsupervised than supervised learning. I have already read tutorials on the general idea of the mechanism of VAE, and what people usually suggest is to use TensorFlow, Keros, etc. to implement them. However, I do not just want to &lt;strong&gt;use&lt;/strong&gt; machine learning, but also want to &lt;strong&gt;develop novel algorithms&lt;/strong&gt; on top of the existing ones, so I want to deeply understand the mathematical details. And in my experience, in order to understand them, reading is not enough, you need to implement. Any suggestions on where/how to start? I have a general machine learning and programming background, but no experience on deep learning. I even don't know what exactly back-propagation is used for, if this would help.&lt;/p&gt;&#xA;" OwnerUserId="9609" LastActivityDate="2017-10-03T06:36:55.603" Title="A good way to understand the mathematical details of variational autoencoders through implementation?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4131" PostTypeId="1" CreationDate="2017-09-27T20:18:39.897" Score="5" ViewCount="610" Body="&lt;p&gt;In example, if there is a simple feed-forward neural network with 3 input neurons, 3 hidden neurons, and one output neuron; is it possible to predict a the value of an input neuron given the values and weights for the other two inputs and the output.&lt;/p&gt;&#xA;" OwnerUserId="9423" LastEditorUserId="9423" LastEditDate="2017-09-28T00:21:12.350" LastActivityDate="2018-02-27T20:08:09.443" Title="In a neural network given partial inputs and complete outputs, is it possible to predict remainig inputs" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4132" PostTypeId="2" ParentId="4126" CreationDate="2017-09-27T20:37:32.760" Score="3" Body="&lt;p&gt;This is sort of a tricky question to answer, by results you could mean 2 possible things. History, which is a culmination of actions, observations and rewards. Or the value func/policy which is the thing you are trying to improve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that being said, you for most instances of RL will only be dealing with the last few events in the history, or sampling uniform randomly from it according to your route of exploration. So I don't think this is what you are talking about. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as storing your value func/policy OpenAI offers a way to post and to a certain extent store your result. This &lt;a href=&quot;https://gym.openai.com/read-only.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Readme&lt;/a&gt; will show you how to do so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: OpenAI does not offer anything to store experience out of the box, as I noted in my comment you could pretty easily whip up some code that writes JSON to some file and saving that would be a store of experience. Although, it looks like you already found something that does this in tensorflow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenAI likely hasn't included it up to this point because they don't put a ton into this framework. I would encourage you check out SerpentAI on Github, as it has a lot of features missing in OpenAI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I answered your question!&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2017-09-30T20:51:26.233" LastActivityDate="2017-09-30T20:51:26.233" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4135" PostTypeId="1" CreationDate="2017-09-28T09:40:10.880" Score="2" ViewCount="46" Body="&lt;p&gt;This question is related to the usage of NN in critical systems (those where a failure can cause life threatening situations - autopilots for example) and the need for formal guarantees on their behavior. &lt;a href=&quot;http://arxiv.org/abs/1702.01135&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is, for example, a paper that verifies NN used in the controls of an unmanned air-vehicle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are numerous tools and techniques for the formal verification (not just testing and simulation, but actual mathematical proof) of, say, floating-point calculation (properties like: will not overflow, will not accumulate an error greater than x, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now take a DNN where the weights are real numbers. That is the DNN as a concept. But there is the implementation of the NN, where the weights must be encoded as floating-points (&lt;a href=&quot;http://arxiv.org/abs/1603.01025&quot; rel=&quot;nofollow noreferrer&quot;&gt;some&lt;/a&gt; even as low precision as 3-bits integers, apparently). And then, (faster) computation is done on these encodings. Someone might argue that you lose precision, but others might answer that it may be a good thing  since one must prevent a NN for over-fitting anyway...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a way to quantify the effect of this encoding on the robustness/precision/stability-of-classification, in a comparable way to what we have in (non learning) software verification?&lt;/li&gt;&#xA;&lt;li&gt;Are there NN architectures (like Sum-Product Networks(?)) that are more amendable to offer such guarantees?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9319" LastActivityDate="2017-09-28T09:40:10.880" Title="Loss of precision when encoding DNN weights" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;autonomous-vehicles&gt;&lt;challenges&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4136" PostTypeId="1" CreationDate="2017-09-28T12:11:12.820" Score="8" ViewCount="3224" Body="&lt;p&gt;DNN can be used to recognize pictures. Great. For that usage, it's better if they are somewhat flexible so as to recognize as cats even cats that are not on the pictures on which they trained (i.e. avoid overfitting). Agreed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when &lt;a href=&quot;http://ieeexplore.ieee.org/abstract/document/7778091/&quot; rel=&quot;noreferrer&quot;&gt;one uses&lt;/a&gt; NN as a replacement for numerical tables in an Air Collision Avoidance System (ACAS), it is primarily to reduce the &quot;required storage space by a factor of 1000&quot;. For this usage, what we want from the NN is to say &quot;take a slight left turn&quot; or &quot;turn right hard&quot; if another ship comes slightly close on the right or rapidly close on the left, respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this usage, where the answer is much simpler than recognizing a cat, isn't overfitting a good thing? What would overfitting &quot;look like&quot; in this case and why would it be bad ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question somewhat relates &lt;a href=&quot;http://ai.stackexchange.com/questions/3473/is-there-such-a-thing-like-the-machine-learning-paradox&quot;&gt;to this one&lt;/a&gt;, where a general idea seems to be &quot;Machine Learning is used for intractable things, you don't need ML for tractable things&quot;. And while it is quite correct that ACAS can be implemented without NN, I wouldn't call NN &quot;useless&quot; for ACAS, because a factor 1000 reduction in required space will always come in handy.  &lt;/p&gt;&#xA;" OwnerUserId="9319" LastActivityDate="2017-09-30T19:10:46.590" Title="Is overfitting always a bad thing?" Tags="&lt;neural-networks&gt;&lt;control-problem&gt;&lt;overfitting&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4137" PostTypeId="2" ParentId="4121" CreationDate="2017-09-28T13:32:37.457" Score="2" Body="&lt;p&gt;No, we are not even near an algorithm that can be compared to human level general intelligence. You might have heard the claim that a neural network works similar to the neurons in the brain, but that's quite a stretch. Serena Yeung talks about this in the forth lecture of the &lt;a href=&quot;https://www.youtube.com/watch?v=d14TUNcbn1k&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford course CS231n&lt;/a&gt;. Jump to 1:04:30 in the video, that's around where this part starts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to the topic of consciousness and qualia, it is highly unlikely that we do possess the technology to create a conscious AI yet. I say highly unlikely because from a philosophical perspective we lack the ability to detect qualia in another being (called the qualia problem), at least based on our current knowledge. So we cannot prove, that we haven't already created artificial consciousness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also gathered from your comments that you are wondering, why we are afraid of strong AI. I can recommend &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superintelligence&lt;/a&gt; from Nick Bostrom. He discusses many different scenarios in which strong AI could be threatening humanity. Consciousness is usually not required for those scenarios. &lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-09-28T13:32:37.457" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4138" PostTypeId="2" ParentId="4131" CreationDate="2017-09-28T16:36:59.073" Score="0" Body="&lt;p&gt;One can do this to a certain extent but I don't think to the point you are thinking. If the weights are known, you can then find the relative importance of each input to the outputs. From there one could &lt;em&gt;maybe&lt;/em&gt; predict or estimate the inputs but I do not think those values would be correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more on this, I encourage you to check out &lt;a href=&quot;http://www.palisade.com/downloads/pdf/academic/DTSpaper110915.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-09-28T16:36:59.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4139" PostTypeId="1" AcceptedAnswerId="4146" CreationDate="2017-09-28T17:11:21.753" Score="3" ViewCount="1379" Body="&lt;p&gt;For implementing a neural network algorithm that can play air hockey, I had two ideas for input, and I'm trying to figure out which design would be most viable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output must be two analog values that dictate the best position on half of the table for the robot to be at a specific point in time, evaluated 60 times per second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having consulted with a professor who has experience with implementing parallel algorithms, it was recommended to me to use a convolutional neural network with a single hidden layer, and directly process the image data as the input layer, after processing it to visualize a direct view of the table and somehow emphasize the puck and mallets with pre-processing. I have already started work on this and have successfully implemented object detection for the puck and mallets using OpenCV to get the center coordinates for all 3 entities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, having been able to successfully and accurately pre-process these data at 60 times per second, my thought was to feed those (after normalizing the values using the error function) directly on the input layer and possibly implement a deep learning algorithm that employs more than one hidden layer. The problem is that I don't have any experience implementing neural networks, so I'm not even sure what type of layer would be best for this, or how I should seed the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another reason I want to consider my idea is that given the relatively few inputs and outputs, probably won't need a GPU to execute forward-propagation 60 times per second, whereas with the convolutional network my professor recommended, I &lt;strong&gt;know&lt;/strong&gt; I'll need to implement using CUDA somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which of the two input methods would be most recommended for this, and if I were to try and implement my idea, what types of layers should I consider? Also any recommendations for existing frameworks to use for either approach would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9887" LastActivityDate="2018-02-18T02:19:20.447" Title="What type of neural network would be most feasible for playing a realtime game?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;hidden-layers&gt;&lt;real-time&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4140" PostTypeId="1" CreationDate="2017-09-28T17:19:47.250" Score="5" ViewCount="336" Body="&lt;p&gt;Notice that, in the following formula, at the very right, the term multiplied with &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w := w + \alpha \sum_{i=1}^{N-1} \nabla r(x_i^l, w) \Big \lfloor \sum_{j=i}^{N-1} \lambda^{j-i} d_i \Big \rfloor&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the index of the first sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the following formula, the term is &lt;span class=&quot;math-container&quot;&gt;$d_m$&lt;/span&gt;, which was &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; in the first equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_j := w_j + \alpha \sum_{i=1}^{N-1} \frac{\partial r(x_i^l, w)}{\partial w_j}  \Big \lfloor \sum_{m=i}^{N-1} \lambda^{m-i} d_m \Big \rfloor&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the index of the second sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, first equation seems reasonable. The &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; in second one actually seems to work like discount factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In another chess engine, &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe&lt;/a&gt;, the author cited same paper, noted the first equation, but then proceeded on to implement second one. See also &lt;a href=&quot;https://arxiv.org/pdf/cs/9901002.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;KnightCap Paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9886" LastEditorUserId="2444" LastEditDate="2019-04-02T14:34:09.910" LastActivityDate="2019-04-02T14:34:09.910" Title="Inconsistency in TD-Leaf algorithm in KnightCap chess engine" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4141" PostTypeId="2" ParentId="4121" CreationDate="2017-09-28T18:55:31.573" Score="2" Body="&lt;p&gt;Regarding &amp;quot;mimicking human intelligence&amp;quot; --&amp;gt; No, not even close.&lt;/p&gt;&#xA;&lt;p&gt;Regarding this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;More specifically, can we say the following about neural networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Neural networks use the sigmoid function and gradient descent to fine-tune weights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, again.  ANN's aren't required to use a sigmoid activation function. There are many other options that are used depending on the circumstances. Also, gradient descent is not strictly required either, even if it is one of the most popular optimization algorithms used with backpropagation.   There are other ways to train ANN's - for example, genetic algorithms can be used.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="2444" LastEditDate="2020-09-10T00:04:50.213" LastActivityDate="2020-09-10T00:04:50.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4142" PostTypeId="1" AcceptedAnswerId="7357" CreationDate="2017-09-28T19:55:13.527" Score="3" ViewCount="499" Body="&lt;p&gt;Given a virtual game map (picture) and a racing car at the map's starting point, I'm trying to build an algorithm that would help me generate a route that would get the car from the beginning to the endpoint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;route definition&lt;/strong&gt;: a very complex .xml file that includes all the data the car needs in order to navigate the road successfully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have thousands of different maps and I could get any number of maps I'd like.&lt;/li&gt;&#xA;&lt;li&gt;For every map, I have a complex external algorithm using picture analysis that builds a route for me - although this route is not so accurate.&lt;/li&gt;&#xA;&lt;li&gt;For some percentage of the maps, I also have a specific &quot;good working car solution&quot; - which is basically a manually built route file.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What I'm trying to achieve:&#xA;Basically, I'm trying to improve the complex route generator algorithm which I don't have access to.&#xA;I want to use the percentage of manual routes I have and compare them to the routes that are generated for the same maps. From the comparison and the differences between the manual and the automatically created route files, I want to build a second-step smart algorithm that &quot;learns&quot; what should be changed in the automatically generated routes and have that algorithm run as a second step and make it as close as possible to the handmade route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My final goal is to be able to build an accurate route for maps I don't have the manual route for. I want to be able to produce an accurate route for every map - using both the external algorithm and the one I will build on top of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This entire subject is very new to me and I'd like to get your advice as to how should I approach this complex problem, and what could make this task easier for me.&lt;/p&gt;&#xA;" OwnerDisplayName="user9890" LastEditorDisplayName="user9890" LastEditDate="2017-09-28T21:41:54.470" LastActivityDate="2018-10-15T23:55:21.237" Title="Machine learning algorithm for xml manipulation" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4143" PostTypeId="2" ParentId="4136" CreationDate="2017-09-28T20:12:55.737" Score="5" Body="&lt;p&gt;Typically the ramification of overfitting is poor performance on unseen data. If you're confident that overfitting on your dataset will not cause problems for situations not described by the dataset, or the dataset contains every possible scenario then overfitting may be good for the performance of the NN.&lt;/p&gt;&#xA;" OwnerUserId="9891" LastActivityDate="2017-09-28T20:12:55.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4146" PostTypeId="2" ParentId="4139" CreationDate="2017-09-29T09:41:59.020" Score="2" Body="&lt;p&gt;I recommend you read up on reinforcement learning. Seeing how AirHockey is similar to the old Atari game Pong, here is a write-up (with code) about how to implement a simple neural network, that plays the game; &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning: Pong from Pixels&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the choice of layers go, again, you should read up on deep reinforcement learning, perhaps study some existing models that work, for a similar task and take their architecture as your starting point. The above link is a nice introduction into such thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a continuation, I would recomend you employ one of the well known, and developed, libraries, such as &lt;a href=&quot;https://www.tensorflow.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; or &lt;a href=&quot;http://pytorch.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyTorch&lt;/a&gt;. They will allow you to run your models on GPU, like you desire.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a toolkit, that specializes to reinforcement learning, see &lt;a href=&quot;https://github.com/openai/gym&quot; rel=&quot;nofollow noreferrer&quot;&gt;openAI gym&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9904" LastActivityDate="2017-09-29T09:41:59.020" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4147" PostTypeId="1" CreationDate="2017-09-29T14:35:49.217" Score="2" ViewCount="173" Body="&lt;p&gt;In their work &lt;em&gt;&lt;a href=&quot;https://arxiv.org/abs/1603.00748&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continuous Deep Q-Learning with Model-based Acceleration&lt;/a&gt;&lt;/em&gt;, the author demonstrate great results of applying Imagination Rollouts for model-based acceleration of learning. They test their algorithm on manipulation tasks in a Mujoco simulation. &#xA;I wonder whether similar accelaration would be possible in classic environments like &lt;code&gt;Pendulum-v0&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9911" LastActivityDate="2017-09-29T14:35:49.217" Title="Is iLQG a good algorithm for model-based planning with simple environments?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4150" PostTypeId="1" CreationDate="2017-09-29T20:19:57.870" Score="2" ViewCount="51" Body="&lt;p&gt;I've only just started looking into Machine Learning, and so far most of the examples I've seen involve starting with a training set of observations, each representing a value for a number of 'features', and using that to train a model - then evaluating the trained model against a second test set of data to allow refinements to the model. The basic operation the model is asked to do is to predict the value of one feature given others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When talking about art (by which I would include music, poetry, etc), we can no longer assert that things are correct - but perhaps we could isolate a number of subjectively-judged features, such as whether a work is 'mysterious' or 'joyful' to a particular human judge, and then train the model to our judge's taste. However, The problem here would seem to be that having a human being add judgement features to each of our data points sounds like a very slow task, and it might often be impractical to get to the number of 'assessed works' that would allow the model to be trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could the process of training a model to produce art of a certain 'taste' be done in a practicably short space of time?&lt;/p&gt;&#xA;" OwnerUserId="9918" LastEditorUserId="9918" LastEditDate="2017-09-30T10:25:46.863" LastActivityDate="2017-09-30T14:30:38.737" Title="What techniques would allow a model to be trained to produce subjectively pleasant 'artistic' output?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4151" PostTypeId="1" CreationDate="2017-09-30T05:06:09.487" Score="1" ViewCount="26" Body="&lt;p&gt;Let's say I train an RNN on the sequence of characters in &lt;em&gt;War and Peace&lt;/em&gt;. After doing so is there a way I can take an arbitrary set of letters, e.g. HNTASK, and have it (ideally) tell me that the most probable sequence of that set is THANKS? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the algorithm must use ONLY the characters in that set; i.e. I don't want it to guess that &lt;em&gt;U&lt;/em&gt; comes after &lt;em&gt;H&lt;/em&gt;, even if &lt;em&gt;U&lt;/em&gt; appeared in the training examples. After each character it can only pick from remaining ones. I would assume one would somehow disable the illegal output neurons to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I know there are far simpler algorithms for transforming jumbled-up letters to English words. This is just a toy example for an important problem.)&lt;/p&gt;&#xA;" OwnerUserId="9926" LastActivityDate="2017-09-30T05:06:09.487" Title="Can one use a trained recurrent neural network to find the most probable sequence of a set?" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4152" PostTypeId="2" ParentId="4140" CreationDate="2017-09-30T12:32:50.700" Score="0" Body="&lt;p&gt;Of course, it's the &lt;strong&gt;first&lt;/strong&gt; equation that is correct. &lt;em&gt;d&lt;/em&gt; is the temporal difference, and it's the difference between the current and next state. The current state is &lt;strong&gt;i&lt;/strong&gt;, and this difference needs to stay constant inside the summation loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second equation is simply a typo mistake.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-09-30T12:32:50.700" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4153" PostTypeId="2" ParentId="4150" CreationDate="2017-09-30T14:30:38.737" Score="3" Body="&lt;p&gt;You will need a large training set to be able to train your machine to produce subjectively pleasant 'artistic' output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can't produce your own dataset with real people, you can try the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;search for existing datasets on the internet&lt;/li&gt;&#xA;&lt;li&gt;produce a new dataset using a spider bot that perform mining of artistic output on websites that have voting features&lt;/li&gt;&#xA;&lt;li&gt;use crowdsourcing&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="107" LastActivityDate="2017-09-30T14:30:38.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4154" PostTypeId="2" ParentId="4136" CreationDate="2017-09-30T17:32:08.057" Score="1" Body="&lt;p&gt;&lt;em&gt;Overfitting&lt;/em&gt; is almost always bad and hurts &lt;em&gt;generalization&lt;/em&gt;. You say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;what we want from the NN is to say &quot;take a slight left turn&quot; or &quot;turn&#xA;  right hard&quot; if another ship comes slightly close on the right or&#xA;  rapidly close on the left, respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But what would you say if the NN learns to &quot;take a slight left turn&quot; only if the coming ship is small (because accidentally that's what the training set includes)? Or it learns to &quot;turn right hard&quot; only on a certain altitude and gives arbitrary answer when the altitude is different?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, overfitting means &lt;em&gt;learning the noise&lt;/em&gt;, i.e. the patterns that are not there. I can hardly imagine a good example, when it's what you really need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer you question: it's quite possible that for ACAS task a much smaller and simpler model would work fine, not necessarily deep and even not necessarily neural network. But it must be reasonable and learning the white noise is not what you call reasonable.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-09-30T17:32:08.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4155" PostTypeId="2" ParentId="4131" CreationDate="2017-09-30T18:16:34.443" Score="1" Body="&lt;p&gt;It's important to understand that though neural networks generalize to the whole input space, usually the meaningful input space, from which the training data is taken, is a &lt;em&gt;manifold&lt;/em&gt; inside that space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the image classifier can take any image, including white noise, but it's usually trained on photos, which generally have some nice structure and statistical properties, e.g. smoothness of color, and is expected to work well on photos, not &lt;a href=&quot;https://ai.stackexchange.com/questions/92/how-is-it-possible-that-deep-neural-networks-are-so-easily-fooled/4051#4051&quot;&gt;everywhere&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your question can be reformulated like this: can we learn the data manifold given the neural network and some partial data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we ignore deliberately crafted small examples (like ones in which the output just equals to one of features) and consider real world, high-dimensional tasks, we see that the manifold is quite resistant to small perturbations. In high-dimensional spaces a change of one dimension doesn't drastically change the distance to the manifold, it's still very very close. Example: a change of a single pixel is very unlikely to affect the output, hence it would be impossible to predict it. The larger the space is, the less is influence of each individual dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this answers your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to mention here one very promising technique called &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN&lt;/a&gt;, which basically does this: it uses one neural network for classification and the other to learn the data manifold. And it works so good that it'd be fair to say that both these networks actually complement each other, not derive from one another.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-09-30T18:16:34.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4156" PostTypeId="2" ParentId="4136" CreationDate="2017-09-30T19:10:46.590" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is overfitting always a bad thing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The answer is a resounding &lt;strong&gt;yes&lt;/strong&gt;, every time. The reason being that overfitting is the name we use to refer to a situation where your model did very well on the training data but when you showed it the dataset that really matter(i.e the test data or put it into production), it performed very bad. This can never be good, because we write models so that they can predict with high accuracy on the datasets it has not seen before. No one cares about a model that got 100% accuracy on the training set and gets 0% on the test set. This is a useless model. We train models so that they can predict with high accuracy on data it has never seen before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now for the long answer:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will start by giving a quick explanation of what overfitting is. Then how we can detect overfitting, this will help me explain why overfitting is &lt;strong&gt;allways&lt;/strong&gt; a bad.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is overfitting:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In machine learning problem there is signal then there is noise. Signal is the true underlying pattern that you wish to learn from the data whist noise is the irrelevant information or randomness in a dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason that leads to overfitting is that your model is making use of &quot;Noise&quot; to make decisions instead of using the signal. Overfitting means that your model performs very well on the training data and performs very poorly on unseen dataset(s) (notice that we pretty much don't care how well a model performs on the training data, we care about how it will perform on data it hasn't seen before). A model that has overfitted is similar to a student who got 100% on the quiz he/she did in preparation for the exam and will get a very poor mark on the exam because he/she because he chose to cram/memorize the quiz's answers and questions instead of understanding how to get to those answers. Another way of thinking about this is that your model has memorized/crammed instead of learning the learning the general relationship contained in the training dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do we detect Overfitting:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to know if your model is overfitting is to test its performance on a dataset it has never seen before. As such, overfitting is something you know with the benefit of hindsight following testing a model on data it has never seen before.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For this usage, where the answer is much simpler than recognizing a cat, isn't overfitting a good thing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, because if your model is overfitting it will perform bad on the test set(in your case, it will perform very bad when you deploy it in a real world ACAS and start showing it data it hasnt seen before). see &lt;em&gt;How do we detect Overfitting&lt;/em&gt; above.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it turns out that your model performs very well when you show it data it hasn't seen before, then that means it didn't overfit (see &lt;em&gt;What is overfitting&lt;/em&gt; above). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What would overfitting &quot;look like&quot; in this case and why would it be bad ?&#xA;  Your model will get very high accuracy on the training set and get very poor accuracy when you start showing it examples it hasn't seen before. So in short, you will get your hopes high, jump up and down in excitement thinking that your model is great only for it no get very poor results in a production ACAS system.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7550" LastActivityDate="2017-09-30T19:10:46.590" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4157" PostTypeId="2" ParentId="4130" CreationDate="2017-09-30T19:48:23.277" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I even don't know what exactly back-propagation is used for, if this would help.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Programming backprop manually is one of the best exercises in machine learning. In fact, to do this you don't need a special package, like Tensorflor or Keras, just a general computing library like NumPy (and a whiteboard).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though I dare to say that VAE is not the best thing to implement, if you are not 100% familiar with backprop. It's pretty advanced technique and you risk to mix up fundamental (e.g. backprop) and specific ideas (e.g. deliberately crafted loss function) and slow down your understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really think that writing a simple feed-forward neural net (including forward and backward passes) is a very good starting point, after which you can move on to &lt;a href=&quot;https://jmetzen.github.io/2015-11-27/vae.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VAE details&lt;/a&gt;, if you'd like.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any suggestions on where/how to start?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you feel confident with algebra and calculus, I highly recommend &lt;a href=&quot;http://cs231n.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Stanford course&lt;/a&gt;, especially its assignments. The course itself is about computer vision and convolutional neural networks, but the first assignments are very general and will give a lot of insight of what and how neural networks are doing.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorUserId="9647" LastEditDate="2017-10-03T06:36:55.603" LastActivityDate="2017-10-03T06:36:55.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4158" PostTypeId="2" ParentId="4037" CreationDate="2017-10-01T05:57:29.503" Score="0" Body="&lt;p&gt;It's tempting to anthropomorphize machine intelligence and suggest it would experience existential crises, but it's important to remember that feelings such as the need to find meaning and purpose evolved in humans, and are unlikely to be similar to what an AI would experience unless designed to have human emotions (apply whichever answer works for humans in that case). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An AI with any goal is likely to produce the subgoal of survival as a means of achieving its primary goal. One can't achieve one's goals if one is dead. This doesn't necessarily suggest the agent would experience the same fear of death, just as an agent wouldn't necessarily experience existential dilemmas.&lt;/p&gt;&#xA;" OwnerUserId="9938" LastActivityDate="2017-10-01T05:57:29.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4159" PostTypeId="2" ParentId="4054" CreationDate="2017-10-01T06:57:10.333" Score="2" Body="&lt;p&gt;Isn't the events of world by the analysis of them with logic only result in an outcome of that analysis if those events were determined in the first place by logic. Not every event in the world is a consequence of logic and is not also a consequence of an illogical action either. The consequence of events as a result of the application logic results in events that are mechanistic. Evil events are not able to be determined by a mechanism they are only determined by personal judgement which is not solely a function of logic there are elements of emotional logic where the input of those events to that emotional logic are not as absolute, and the same events inputted to a range of people's emotional logic will not result in the same outcome and thus there judgement of the degree of evil or not will differ. &lt;/p&gt;&#xA;" OwnerUserId="9094" LastActivityDate="2017-10-01T06:57:10.333" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4161" PostTypeId="1" AcceptedAnswerId="4264" CreationDate="2017-10-01T11:14:23.837" Score="4" ViewCount="340" Body="&lt;p&gt;My question assumes that a private researcher doesn't have access to anything stronger than a modern PC with a high end GPU to implement his projects. He can also use cloud computing but with limited funds as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it still feasible to do research with those restrictions? &lt;a href=&quot;http://www.businessinsider.de/heres-how-much-computing-power-google-deepmind-needed-to-beat-lee-sedol-2016-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo used 1,202 CPUs and 176 GPUs&lt;/a&gt; to beat Lee Sedol. Is this enormous power only required to achieve the final optimizations or have we already reached a state where high end research can only be done with larger funding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please include in your answer examples of recent research results and the required infrastructure that was used to create them.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2020-12-20T07:28:29.527" Title="Is private research with low funds still feasible in the field of AI?" Tags="&lt;research&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4162" PostTypeId="1" CreationDate="2017-10-01T12:17:42.547" Score="3" ViewCount="228" Body="&lt;p&gt;What is the difference between a normal processor and a processor designed for AI?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-05-13T21:29:06.613" LastActivityDate="2020-05-13T21:30:45.267" Title="What is the difference between a normal processor and a processor designed for AI?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;hardware&gt;&lt;hardware-evaluation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4163" PostTypeId="2" ParentId="4162" CreationDate="2017-10-01T12:47:48.650" Score="2" Body="&lt;p&gt;Parallel processing is very important when computing ANNs. Though they are not designed for AIs, GPUs are widely used in machine learning because of their higher capability of paralel processing and it is due to their higher number of cores comparing CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, the dedicated hardware should have GPU-like architecture.&lt;/p&gt;&#xA;" OwnerUserId="9948" LastActivityDate="2017-10-01T12:47:48.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4164" PostTypeId="2" ParentId="3190" CreationDate="2017-10-01T16:05:03.937" Score="1" Body="&lt;p&gt;This is just an application of the chain rule. The same chapter has &quot;Proof of the four fundamental equations&quot; section, which proves BP1-2, while PB3-4 are left as exercise to the reader. I agree that it's a good exercise indeed, that's why I encourage you to &lt;strong&gt;stop here and try to prove it yourself using a chain rule&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you decided to read further, here's the sketch of a proof. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall equations (25) and (29), both &lt;em&gt;definitions&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$z^l$&lt;/span&gt; is a linear transformation of &lt;span class=&quot;math-container&quot;&gt;$a^{l-1}$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$z^l = w^l a^{l-1} + b^l$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\delta^l$&lt;/span&gt; is a partial derivative of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$z^l$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\delta^l  = \dfrac{\partial C}{\partial z^l}$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The chain rule itself:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the partial derivative of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$b^l$&lt;/span&gt;: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial b^l_j} = \sum_k \frac{\partial C}{\partial z^l_k} \frac{\partial z^l_k}{\partial b^l_j}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost all elements in this sum will be zero, except for one when &lt;span class=&quot;math-container&quot;&gt;$k=j$&lt;/span&gt;.&#xA;The first term in it is by definition &lt;span class=&quot;math-container&quot;&gt;$\delta^l_j$&lt;/span&gt;, the second term &lt;span class=&quot;math-container&quot;&gt;$\dfrac{\partial z^l_j}{\partial b^l_j}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is linear with respect to &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative with respect to weights is taken the same way, and differs only in the last term: &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is linear with respect to &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; as well, but with a coefficient &lt;span class=&quot;math-container&quot;&gt;$a^{l-1}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorDisplayName="user1440" LastEditDate="2018-11-26T18:25:54.667" LastActivityDate="2018-11-26T18:25:54.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4165" PostTypeId="1" AcceptedAnswerId="7338" CreationDate="2017-10-01T20:18:43.517" Score="5" ViewCount="569" Body="&lt;p&gt;It has been shown that it is possible to use unsupervised learning techniques to produce good feature detectors in CNNs. I can't understand what drives specialization of those feature detectors. In publications &lt;a href=&quot;https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf&lt;/a&gt; (page 6) they show a very reasonable set of edge and blob detectors with little or no overlap. It goes against intuition - in absence of incentive to specialize, one would expect at least some duplication of learned kernels.&lt;/p&gt;&#xA;" OwnerUserId="9803" LastEditorUserId="9803" LastEditDate="2017-10-04T06:02:22.887" LastActivityDate="2018-07-30T01:24:15.010" Title="What makes learned feature detectors specialize in CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;unsupervised-learning&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4166" PostTypeId="2" ParentId="4162" CreationDate="2017-10-01T20:40:01.777" Score="2" Body="&lt;p&gt;Specialized AI hardware takes advantage of highly parallelizable nature of many neural network designs. GPUs are designed for pixel crunching - coincidentally very paralelisable too. This is why they often offer orders of magnitude better performance (in NN tasks) than CPUs. That being said, GPUs have shortcomings to. First, many of GPU's built in features are not utilized in neural network applications- in many cases accurate floating point operations are not necessary, data for simple calculations involved in NNs. If you remove all the unused features from GPUs and use that extra space for more compute units, you get something like Googles' &lt;a href=&quot;https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu&lt;/a&gt;  TPU.&lt;/p&gt;&#xA;" OwnerUserId="9803" LastActivityDate="2017-10-01T20:40:01.777" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4167" PostTypeId="1" AcceptedAnswerId="4168" CreationDate="2017-10-02T02:14:48.800" Score="1" ViewCount="567" Body="&lt;p&gt;I am a first-semester grad student in Robotics and have taken a course on machine learning for robotics. I am completely new to machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am to select and execute a problem statement on my own as a part of the course. I have selected the following problem statement -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Reinforcement learning for mapless navigation of mobile robots&quot; based on this &lt;a href=&quot;https://arxiv.org/pdf/1703.00420.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;research&lt;/a&gt;. The goal is to develop a mapless motion planner which enables a robot to navigate by avoiding obstacles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I am completely new to Machine learning and AI, I am not able to gauge whether the problem statement is challenging enough for a beginner to execute in a timeframe of 8 weeks? (given that I can dedicate around 6 hours a week)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am planning to code in MATLAB since I am highly comfortable with MATLAB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, please leave any suggestions/ modifications to refine my approach and have a better understanding in this area. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9952" LastEditorUserId="1671" LastEditDate="2018-01-28T23:03:24.547" LastActivityDate="2018-01-28T23:03:24.547" Title="Reinforcement learning for robotic motion planning - Problem statement ideas" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;robotics&gt;&lt;matlab&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4168" PostTypeId="2" ParentId="4167" CreationDate="2017-10-02T04:13:06.023" Score="1" Body="&lt;p&gt;I think that this may be too difficult for only a few weeks of work. There are a few reasons why and although all of the reasons are themselves learning opportunities, it may cost you more time than you have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First off, Robots are hard to work with. Ask anyone who has worked with robots and they will say that have broken their heart at least once. This is because either the robot was a custom robot which needs specific skills (soldering, 3d printing, etc) if something breaks (and it will) or it's a purchased robot platform which doesn't have the control you want or doesn't have good documentation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, Machine learning is a very interesting and broad field and there is a lot of literature one should get a background on to help with their projects. Because of this, it might be overwhelming to take all of that information in while having to deal with a robot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to dissuade you in this research area though. It is very interesting and has many subproblems like dynamic environments, multilevel environments, etc. But given the time you have, it might be better to set yourself up for success. Perhaps something which lets you do a literature review about the problem anyway. &#xA;Perhaps a more attainable goal in that time while still using a robot is: &lt;strong&gt;Predicting wall collisions using temporal difference learning.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you agent can predict when a wall is coming, clearly they will be able to avoid them too but it's a very reasonable first step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, talking to the professor of the course or your supervisor (if you have one) will also help you decide. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-10-02T04:13:06.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4170" PostTypeId="1" CreationDate="2017-10-02T16:51:04.940" Score="7" ViewCount="524" Body="&lt;p&gt;I'm currently in the research stage of building a web app in ASP.NET where the user can input a URL to an Amazon product, then the app would determine how likely its reviews are to be genuine. I need help figuring out what algorithm to use in determining if a certain review is likely to be deceptive. I want my app to behave similarly to Fakespot or ReviewMeta. I realise a tool like this won't be 100% accurate and that's fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I read parts of &lt;a href=&quot;https://www.amazon.co.uk/Language-Processing-Prentice-Artificial-Intelligence/dp/0131873210&quot; rel=&quot;noreferrer&quot;&gt;this book&lt;/a&gt; which seems to be recommended a lot to NLP newbies but haven't found anything that applies to such a specific problem. I also read &lt;a href=&quot;http://www.aclweb.org/anthology/P14-1147&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt; but it's based on hotel, restaurant and doctor reviews. I'm trying to find a more general method that can be applied to any product. Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9962" LastEditorUserId="4302" LastEditDate="2018-10-08T12:50:30.953" LastActivityDate="2018-10-08T12:50:30.953" Title="How to determine if an Amazon review is likely to be fake using text classification" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;algorithm&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4173" PostTypeId="2" ParentId="4118" CreationDate="2017-10-02T22:30:25.970" Score="1" Body="&lt;p&gt;If I understood that post right (I just skimmed through, so it's possible I missed some details), they are using several predictors on the inputs with several rectangles. This basically means separate, not shared, weights to detect each rectangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's fairly likely that these neurons will adjust to certain areas on the input image. As far as the neurons are concerned, they are &quot;punished&quot; if they detect the wrong rectangle, as if the other rectangle is the noise that the network should learn to ignore. As a result, the &quot;left&quot; predictor learns to fire more actively when it detects a rectangle on the left, because there's high chance it's the correct one. Same for the &quot;right&quot; predictor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When, all of a sudden, the training image swaps the targets, each predictor gets confused. Because now what they used to detect as noise becomes the ground truth and vice versa. It certainly hurts the performance and makes the network be sure the most, when the target is around the center. When the targets are flipped, the dataset becomes more consistent and less confusing for the network, it's no wonder that the accuracy goes up.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-02T22:30:25.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4174" PostTypeId="2" ParentId="4170" CreationDate="2017-10-03T00:53:09.040" Score="4" Body="&lt;p&gt;This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a lot of training data for quite a few of the models that will give you a high accuracy. Then, you will probably want to use a Long short term memory recurrent neural network along with a word2vec model, or maybe even a sentence/ paraphraph2vec model. This will be able to give you some fairly good results with a little bit of tweaking, but if you want really accurate results you will want to try an ensemble. An ensemble is when you use multiple neural networks that provide an output that gets fed through another classifier(usually XGboost) in order to achieve better classification results. You might also want to try a little bit of feature engineering. Finally, in order to implement this model, you can take one of several approaches. First off, you can use tensorflow serving. Tensorflow was created by google and used by much of the machine learning community. However, tensorflow was written in python, so you will need to use tensorflow serving in order to use it within your Asp.net app. Or, you could use microsofts CNTK. Or finally, you could use a custom built implementation, but this would take a lot of time and not be worthwhile unless you are doing research. Also, if you really do not want to use deep learning for one reason or another, you can probably use a simpler model, but you will need to understand your data. You will need also need to understand your data if you are doing deep learning but maybe not as much.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="4631" LastEditDate="2017-10-03T00:58:21.513" LastActivityDate="2017-10-03T00:58:21.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4176" PostTypeId="1" AcceptedAnswerId="4230" CreationDate="2017-10-03T09:42:23.333" Score="5" ViewCount="12759" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/10/what-is-fuzzy-logic&quot;&gt;This question&lt;/a&gt; covers in detail, what fuzzy logic is and how it relates to other math fields, such as boolean algebra and sets theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/118/how-can-fuzzy-logic-be-used-in-creating-ai&quot;&gt;This question&lt;/a&gt; is also very related, but the answers are focused more on general intuition and &lt;em&gt;potential&lt;/em&gt; applicability. The only working system based on fuzzy logic, mentioned there, is &lt;a href=&quot;https://en.wikipedia.org/wiki/Mycin&quot; rel=&quot;noreferrer&quot;&gt;MYCIN&lt;/a&gt;, which goes back to the early 70s. This quote from wiki summarizes my impression of it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;MYCIN was never actually used in practice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From my experience in AI, the best tool to deal with uncertainty is &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_probability&quot; rel=&quot;noreferrer&quot;&gt;Bayesian probability&lt;/a&gt; and inference. It allows to apply not only a wide range of probabilistic tools, such as expectation, MLE, cross-entropy, etc, but also calculus and algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you call fuzzy logic a &quot;pure theoretical&quot; concept, which only played its role in the early development of AI? Are there real practical applications of fuzzy logic? What problem would you recommend to solve and to &lt;em&gt;code&lt;/em&gt; using fuzzy logic?&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2020-03-03T18:10:25.957" Title="Are there real applications of fuzzy logic?" Tags="&lt;ai-design&gt;&lt;applications&gt;&lt;problem-solving&gt;&lt;fuzzy-logic&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4177" PostTypeId="2" ParentId="4161" CreationDate="2017-10-03T13:48:55.543" Score="3" Body="&lt;p&gt;It is totally practical and encouraged to do research on even common laptops. Many questions about AI can be addressed using this architecture and for a reference, look at every AI grad student which may not have access to such a super computer (most of them). AI research is done on laptops. It's the presentation of a system on a difficult problem which may require a supercomputer. It's also the reason why every domain on OpenAI gym can easily run on a laptop, and as a case study,  &lt;a href=&quot;https://www.ualberta.ca/science/science-news/2017/march/artificial-intelligence-deepstack-outplays-poker-professionals&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deepstack&lt;/a&gt; is a new AI system which can beat professional poker players and runs on a laptop. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-10-03T13:48:55.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4178" PostTypeId="2" ParentId="4176" CreationDate="2017-10-03T15:48:13.277" Score="4" Body="&lt;p&gt;You've obviously never heard of &lt;a href=&quot;http://www.samsung.com/in/support/skp/faq/138486&quot; rel=&quot;nofollow noreferrer&quot;&gt;fuzzy logic washing machines&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;● Typically, fuzzy logic controls the washing process, water intake,water temperature, wash time, rinse performance, and spin speed. This optimises the life span of the washing machine. More sophisticated machines weigh the load (so you can’t overload the washing machine), advise on the required amount of detergent, assess cloth material type and water hardness, and check whether the detergent is in powder or liquid form. Some machines even learn from past experience,memorising programs and adjusting them to minimise running costs. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Fuzzy logic is used in a variety of control applications.  If your furnace can only be on or off, for example, you might use a probabilistic function of temperature to determine when to turn it on and off, rather than having fixed high and low temperatures activate your thermostat.  In some applications, that's been found to improve perceived comfort or efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more sophisticated AI applications, you could use fuzzy logic for activations in a neural net, but I don't think it's offering much improvement over fixed, weighted activations.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2017-10-03T15:48:13.277" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4179" PostTypeId="1" CreationDate="2017-10-03T16:03:04.327" Score="2" ViewCount="47" Body="&lt;p&gt;I'm currently developping an application which allows psychologists to manage their schedule and budget. As a proof of concept, I would like to create an intelligent appointment service. There can be 3 cases:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I know the client, I need to guess the day and time for his next appointment&lt;/li&gt;&#xA;&lt;li&gt;I know the day, I need to guess which client and at what time&lt;/li&gt;&#xA;&lt;li&gt;I know nothing, I need to guess which client, which day and what time&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm currently in the process of learning deep learning algorithms just to get a bit of theory, but it's a little bit overwhelming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are features I know I can extract from the appointments:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Day preference in the week (always on monday, say)&lt;/li&gt;&#xA;&lt;li&gt;Reccurence (every two weeks or such)&lt;/li&gt;&#xA;&lt;li&gt;Nb of days since last appointment&lt;/li&gt;&#xA;&lt;li&gt;Whether the client was present or not to his last appointment&lt;/li&gt;&#xA;&lt;li&gt;etc..&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know there are things like &quot;features extraction&quot; that you can train a neural network to find the features itself, but all examples refers to image recognition or speech analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the algorithm to train on the existing and future appointments (stored in a MongoDB). I would also like that the algorithm trains live, that is if it proposes an appointment to the user and the user takes it, it should train positively. On the other hand, if the user navigates or change any parameter, the algorithm should adjust its weights accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know I should start by extracting data from the DB that will be transformed in a vector or matrix, then the algorithm is supposed to train on that data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? How can I start and what kind of architecture do I need?&lt;/p&gt;&#xA;" OwnerUserId="9972" LastActivityDate="2017-10-03T16:03:04.327" Title="Recommendations on which architecture to use to guess appointment" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4180" PostTypeId="2" ParentId="4161" CreationDate="2017-10-03T23:45:34.660" Score="2" Body="&lt;p&gt;A lot of research can be performed on some pretty basic computers. Not all machine learning research is based on very large neural networks. There is a lot of machine learning research that goes on with other more simple algorithms such as k-means clustering, and softmax regression. These algorithms are pretty basic, and so can run very fast, but they do not require massive super computers to train. Also, a decent high range GPU can train some pretty large neural networks. You can reasonably train some fairly large convolutional neural networks on a high end GPU. While training very large neural networks on millions of examples is a huge task and requires a lot of processing power, not all of machine learning is based around deep learning, and a lot of algorithms run extremely fast when they are GPU accelerated.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-10-03T23:45:34.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4181" PostTypeId="2" ParentId="4161" CreationDate="2017-10-04T01:33:56.723" Score="2" Body="&lt;p&gt;The field of AI is vast that there’s always room for small scale research and inquiry.  Utility of AI is key, but the potential applications are broad, and intelligence is a spectrum.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamental Combinatronics, a collective with no current funding, is engaged in a project to develop “adaptive AI” for a set of consumer-oriented, combinatorial game products.  The requirements are distinct from real-world applications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can’t compete with the major players in terms of resources, and we’re late to the party in terms of Machine Learning and Neural Networks, and, because the AI is for a consumer, mobile game which carries significant restrictions in terms of the bounding rationality (&lt;strong&gt;networking cannot be assumed; software volume is measured in megabytes; memory is restricted to lowest-common-denominator consumer-grade devices with non-specialized processors.&lt;/strong&gt;)  For these reasons, we&#xA;re going the opposite direction of current industry trends--the good-old &quot;boring stuff&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the automata only need to outperform the average/above-average human player, an old-school, heuristic approach is feasible. (Fun also, because it involves solving non-trivial, partisan Sudoku games in a Combinatorial Game Theory sense, a type of research all on its own. Although the context is ultimately intractable, it is a context automata are well suited for.)&#xA;Old-school is beneficial in that &lt;strong&gt;it’s nice to have an app product with a decent AI that is under 7mb.&lt;/strong&gt;  (No barrier to download or strong incentive to delete from the device.  While the new iPad has up to 128gb, only a small subset of players will be willing to devote significant volume for strong AI, and these players represent a distinct, &lt;em&gt;secondary&lt;/em&gt; market segment.)  &lt;strong&gt;It’s not optimal for an AI take up any more volume than is strictly necessary for a given product.&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fuzzy logic should also be useful for its efficiency in terms of applicability under what would today be considered severe computational restrictions.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.fundamentalcombinatronics.com/rules-of-m/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[M] games&lt;/a&gt; are economic so the model is interesting from a Game Theory standpoint in providing a novel, compact, intrinsic and highly mutable mathematical model based positional valuation in &lt;em&gt;n&lt;/em&gt; dimensions in conjunction with stability states in a causal/temporal framework. The combinatorial nature of [M] is ideal for quantitative analysis, and the games involve blocking factors (sudoku) and symmetry breaking (even order gameboards). For players &gt; 2 coalitions also become a factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The focus of the procedural research is currently in four main areas and what we’re terming “Adaptive AI” :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Dynamic Strength:&lt;/strong&gt;&#xA;Sheer strength is not the goal.  We’re working on AI that tailor their strength to their human player’s strength and preferences.  For most humans, we don’t want the automata to win more than 2/3 games because &lt;em&gt;always losing&lt;/em&gt;  is no fun and makes the product less &quot;sticky&quot;. Even if the human player desires an automata it cannot beat, the automata should only be sufficiently strong to &lt;em&gt;almost always&lt;/em&gt; beat their human. AI strength can be limited by restricting rationality (time and memory), which carries an added benefit of energy conservation (less bits flipped), but the rules-based approach is useful in that rules can be recombined combinatorially to produce automata of different strengths and preferences. Automata play against each other to determine strength hierarchies, and identify poor heuristics to be weeded out of stronger automata.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;General Intelligence:&lt;/strong&gt;&#xA;The automata have to function on an array of related games, where equilibria can be altered in numerous ways without adding mechanics. Additionally, mechanics can be added without altering the nature of the games, such as introducing Graeco-Latin squares. This presents a problem if each configuration has to be learned through intensive self-play because the automata must be able to play at a respectable strengths immediately.  Thus the goal is not sheer strength, but consistent strength across the widest array of contexts.  (“Respectably weak” and “semi-strong” automata have utility value in that those categorization may be said to describe the majority of human player base.)  The idea is an “axiomatic intelligence” that can be extended to include an ever increasing array of contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Counter-Intuition:&lt;/strong&gt;&#xA;The automata should not be prone to repetitive play.  Initially we’re using limited monte-carlo for positional selection tie-breaking, and the scope can be extend to larger arrays of positions with varying degrees of perceived optimality, up to rational but “counter-intuitive” decisions, which can be subsequently evaluated.  This may be useful in adapting to new, dominant strategies that emerge in allowing the automata to experiment with less obvious choices.  In situations where the automata is consistently winning, there is incentive to experiment, &quot;investing in loss&quot; in the sense that mistakes are useful from an experience/learning standpoint.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&quot;Genetic&quot; Evolution:&lt;/strong&gt;&#xA;The eventual goal is to implement some form of local reinforcement where the automata learn through play against their human, and self play in restricted contexts, such as between turns when playing against their human.  With networking enabled, the automata can play against such automata, with the idea of producing strong automata with human play characteristics.  (It will be fun when we eventually put these automata up against pure deep learning algorithms in a wide array of [M] contexts with distinct mathematical properties.  My money would be on the ML and NN algorithms in sequential games, but in asynchronous games where there is no turn order, it will be interesting to see if the &quot;axiomatic systems&quot; can produce desirable outcomes by making sound decisions faster than smarter, more complex automata;) &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-12T17:39:50.810" LastActivityDate="2017-10-12T17:39:50.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4183" PostTypeId="1" CreationDate="2017-10-04T09:02:05.147" Score="3" ViewCount="864" Body="&lt;p&gt;I have to model an AI that should be able to understand clues and find the answer from a specified word database. I came across several papers that solves the problem with training neural networks or processing the clues with training several machine learning architectures on clues that exist on several databases. However, they all seem overkill for the course currently I'm taking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what would be the most simple approach to solve that problem?&lt;/p&gt;&#xA;" OwnerUserId="9990" LastActivityDate="2017-10-05T02:42:46.113" Title="Solving Crossword Puzzles" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4185" PostTypeId="2" ParentId="3065" CreationDate="2017-10-04T13:04:52.790" Score="13" Body="&lt;p&gt;When it comes to classification problem in machine learning, the cross entropy and KL divergence &lt;strong&gt;are equal&lt;/strong&gt;. As already stated in the question, the general formula is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H(p, q) = H(p) + D_{KL}(p||q)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; a “true” distribution and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is an estimated distribution, &lt;span class=&quot;math-container&quot;&gt;$H(p, q)$&lt;/span&gt; is the cross-entropy, &lt;span class=&quot;math-container&quot;&gt;$H(p)$&lt;/span&gt; is the entropy and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is the Kullback-Leibler divergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in machine learning, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a one-hot representation of the ground-truth class, i.e., &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p = [0,..., 1, ..., 0]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is basically a &lt;a href=&quot;https://en.wikipedia.org/wiki/Dirac_delta_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;delta-function&lt;/a&gt; distribution. But the entropy of the delta function is zero, hence KL divergence simply equals the cross-entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, even if &lt;span class=&quot;math-container&quot;&gt;$H(p)$&lt;/span&gt; wasn't &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (e.g., soft labels), it is fixed and has no contribution to the gradient. In terms of optimization, it's safe to simply remove it and optimize the Kullback-Leibler divergence.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorDisplayName="user1440" LastEditDate="2018-11-22T06:59:06.650" LastActivityDate="2018-11-22T06:59:06.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4186" PostTypeId="1" AcceptedAnswerId="5192" CreationDate="2017-10-04T17:01:41.900" Score="3" ViewCount="1121" Body="&lt;p&gt;Suppose I have context-free grammar (CFG) &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; that pumps out words in that language. I want a machine learning system that can detect if a word &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; came from &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; or not. It has access to a stream from &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; that is constantly producing words in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; at random. In this case, the system can only be trained with positive examples. The system also has access to a verifier for &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. The system (if it chooses to) can generate strings and verify that the string is in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; or not. However, the system could just cheat and just use the verifier, so during the test phase, it gets disabled (during the learning phase, it uses the verifier as much as it wants). Moreover, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; can be augmented and changed, and the system should adapt to that change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, let's consider this regex (a regex is a CFG): &lt;span class=&quot;math-container&quot;&gt;$$(aa)*$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This language only produces a string of &lt;span class=&quot;math-container&quot;&gt;$aa$&lt;/span&gt;'s of even length. Suppose I modified the regex to &lt;span class=&quot;math-container&quot;&gt;$(aa|bb)*$&lt;/span&gt;. This system should adapt to recognize this new language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of methods/approaches should I consider in my design?&lt;/p&gt;&#xA;" OwnerUserId="10001" LastEditorUserId="2444" LastEditDate="2019-11-01T01:50:34.190" LastActivityDate="2019-11-01T01:50:34.190" Title="How can I implement an AI system that learns to recognise a context-free grammar?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;grammar-induction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4187" PostTypeId="1" CreationDate="2017-10-04T20:18:57.780" Score="4" ViewCount="175" Body="&lt;p&gt;Let's say we already have an intelligent system (as a computer program). We give it some simple data from outside world, and system tries to predict it. The question would be: what kind of patterns this system should be able to recognize?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, system gets binary data stream(zeroes and ones).&lt;br&gt;&#xA;1) Should it recognize patterns like: 1 0 11 0 111 0 ....?&lt;br&gt;&#xA;2) Should this system be able to recognize prime numbers in ascending order written with &quot;1&quot; with &quot;0&quot; between them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any research about it?&lt;/p&gt;&#xA;" OwnerUserId="10004" LastEditorUserId="1671" LastEditDate="2017-10-05T20:26:27.880" LastActivityDate="2017-10-08T05:28:40.927" Title="Intelligence over simple data" Tags="&lt;reasoning&gt;&lt;pattern-recognition&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4188" PostTypeId="5" CreationDate="2017-10-04T21:16:01.357" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-04T21:16:01.357" LastActivityDate="2017-10-04T21:16:01.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4189" PostTypeId="4" CreationDate="2017-10-04T21:16:01.357" Score="0" Body="Context-free grammar " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-04T21:16:01.357" LastActivityDate="2017-10-04T21:16:01.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4190" PostTypeId="1" AcceptedAnswerId="24334" CreationDate="2017-10-04T22:41:25.223" Score="1" ViewCount="49" Body="&lt;p&gt;Being by no means an expert in the field, I do however know of solvers such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Z3_Theorem_Prover&quot; rel=&quot;nofollow noreferrer&quot;&gt;Z3&lt;/a&gt; that can generate a correct program, provided you can express your constraints as a set of logical rules, such as &lt;a href=&quot;http://barghouthi.github.io/articles/17/synthesis-primer.html#test-driven-synthesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, in the example given above, logical rules are designed &amp;quot;by hand&amp;quot; from a set of examples, so the &amp;quot;compilation step&amp;quot; from the examples to the set of rules must be done in the user's head.&lt;/p&gt;&#xA;&lt;p&gt;Does a system automating this &amp;quot;compilation step&amp;quot; exist (i.e automated rule set deduction from a set of examples), that would allow the user to see and make use of the set of rules?&lt;/p&gt;&#xA;" OwnerUserId="2364" LastEditorUserId="2444" LastEditDate="2020-10-31T18:04:54.010" LastActivityDate="2020-10-31T18:14:17.640" Title="Is there a system that automatically designs a set of rules from the examples?" Tags="&lt;logic&gt;&lt;automated-theorem-proving&gt;&lt;z3&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4191" PostTypeId="2" ParentId="3938" CreationDate="2017-10-04T22:52:53.393" Score="19" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I handle such large image sizes without downsampling?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume that by downsampling you mean scaling down the input &lt;em&gt;before&lt;/em&gt; passing it into CNN. &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#conv&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional layer&lt;/a&gt; allows to downsample the image within a network, by picking a large stride, which is going to save resources for the next layers. In fact, that's what it has to do, otherwise your model won't fit in GPU.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Are there any techniques to handle such large images which are to be trained?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Commonly researches scale the images to a resonable size. But if that's not an option for you, you'll need to restrict your CNN. In addition to downsampling in early layers, I would recommend you to get rid of FC layer (which normally takes most of parameters) &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#convert&quot; rel=&quot;nofollow noreferrer&quot;&gt;in favor of convolutional layer&lt;/a&gt;. Also you will have to stream your data in each epoch, because it won't fit into your GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that none of this will prevent heavy computational load in the early layers, exactly because the input is so large: convolution is an expensive operation and the first layers will perform &lt;em&gt;a lot&lt;/em&gt; of them in each forward and backward pass. In short, training will be slow.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;What batch size is reasonable to use?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here's another problem. A single image takes &lt;code&gt;2400x2400x3x4&lt;/code&gt; (3 channels and 4 bytes per pixel) which is ~70Mb, so you can hardly afford even a batch size 10. More realistically would be 5. Note that most of the memory will be taken by CNN parameters. I think in this case it makes sense reduce the size by using 16-bit values rather than 32-bit - this way you'll be able to double the batches.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Are there any precautions to take, or any increase and decrease in hardware resources that I can do?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your bottleneck is GPU memory. If you can afford another GPU, get it and split the network across them. Everything else is insignificant compared to GPU memory.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorUserId="2444" LastEditDate="2020-05-22T23:48:08.090" LastActivityDate="2020-05-22T23:48:08.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4192" PostTypeId="2" ParentId="4187" CreationDate="2017-10-04T23:22:38.920" Score="1" Body="&lt;p&gt;In my opinion: If we could predict accurately what an intelligent system capable of, it ain't really intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About research, maybe &lt;a href=&quot;http://www.independent.co.uk/life-style/gadgets-and-tech/news/facebook-artificial-intelligence-ai-chatbot-new-language-research-openai-google-a7869706.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;facebook AI chatbot&lt;/a&gt; being able to develop its own language qualify as one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @DukeZhou stated, &quot;context&quot;, If we build an AI system that ONLY writes creative poet, I can't imagine it would be able to launch nuclear attack or take off power grid! but what about building a general purpose AI? it should be able to surprise us in the context of performing unpredictable behavior. &lt;/p&gt;&#xA;" OwnerUserId="9685" LastActivityDate="2017-10-04T23:22:38.920" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4193" PostTypeId="2" ParentId="4183" CreationDate="2017-10-05T02:42:46.113" Score="1" Body="&lt;p&gt;Artificial neural network to solve Crossword puzzles, this model might help you to get a start on evolving your own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/fh295/DefGen2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/fh295/DefGen2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A brute force of word combinations would solve the matrix, but might not get the context right and might be the simplest model if you are not in for a neural network.&lt;/p&gt;&#xA;" OwnerUserId="8827" LastActivityDate="2017-10-05T02:42:46.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4194" PostTypeId="1" CreationDate="2017-10-05T04:21:32.313" Score="1" ViewCount="68" Body="&lt;p&gt;Image recognition can be used to classify images. But I wanted to find few parameters like height of person, his legs, his hand etc. Will CNN helpful for this type of output ?&lt;/p&gt;&#xA;" OwnerUserId="6212" LastActivityDate="2017-10-05T04:21:32.313" Title="Can Image Recognition used to find height of a person whole, torso, legs etc" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4195" PostTypeId="2" ParentId="4187" CreationDate="2017-10-05T05:11:38.257" Score="1" Body="&lt;p&gt;If the system would be able to predict with certainty doesn't It mean the data set input would be a product of a world of determinism and the system had to be created using the deterministic laws of the world whence that data came from?&#xA;It's a tough one because randomness to me reeks of no laws because they create order not random behaviour and a world with no known laws would be systemless as systems employ laws.&lt;/p&gt;&#xA;" OwnerUserId="9094" LastEditorUserId="9094" LastEditDate="2017-10-08T05:28:40.927" LastActivityDate="2017-10-08T05:28:40.927" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="4196" PostTypeId="1" CreationDate="2017-10-05T05:42:13.597" Score="2" ViewCount="46" Body="&lt;p&gt;My network works on 32x32 normalized (translationally) but noisy images. Its task it to determine whether image has simple symmetry (horizontal/vertical). It needs to be reasonably robust to rotation (up to 20 degrees). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I approached this task with a simple perceptron-like net with 2 hidden layers. It performs reasonably well (on limited amount of data that I have) but I can't shake off the feeling that this design is absolutely the worst for what i need.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It is hard to judge generalization capacity of the net (overfits&#xA;really easily) &lt;/li&gt;&#xA;&lt;li&gt;It performs alot worse than a simple deterministic&#xA;program that i wrote for the same task&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Symmetry is such a simple concept but my neural network (being feed forward type) can't represent it efficiently. What I have in mind is a kind of RNN that decides what axis to fold the image along and then judging on how well folded parts of image match. Are there papers on something like that?&lt;/p&gt;&#xA;" OwnerUserId="9803" LastActivityDate="2017-10-05T05:42:13.597" Title="Detecting symmetry in small images with RNN" Tags="&lt;classification&gt;&lt;computer-vision&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4197" PostTypeId="2" ParentId="4125" CreationDate="2017-10-05T08:44:19.333" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/tagged/python&quot;&gt;StackOverflow answers&lt;/a&gt; with code snippets. This data needs some processing, because the description can be in the question (along with other notes) and along with the answer. But this dataset is very big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also take a look at &lt;a href=&quot;https://codereview.stackexchange.com/questions/tagged/python&quot;&gt;CodeReview questions&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-05T08:44:19.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4199" PostTypeId="1" CreationDate="2017-10-05T13:21:49.430" Score="4" ViewCount="8482" Body="&lt;p&gt;While working with darkflow, I encountered something that I can't understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that maxpooling with size=2,stride=2 would decrease the output size to half of its size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the max-pooling is size=2,stride=1 then it would simply decrease the width and height of the output by 1 only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the darkflow model doesn't seem to decrease the output by 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the model structure when I load the example model &lt;code&gt;tiny-yolo-voc.cfg&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Source | Train? | Layer description                | Output size&#xA;-------+--------+----------------------------------+---------------&#xA;       |        | input                            | (?, 416, 416, 3)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 416, 416, 16)&#xA; Load  |  Yep!  | maxp 2x2p0_2                     | (?, 208, 208, 16)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 208, 208, 32)&#xA; Load  |  Yep!  | maxp 2x2p0_2                     | (?, 104, 104, 32)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 104, 104, 64)&#xA; Load  |  Yep!  | maxp 2x2p0_2                     | (?, 52, 52, 64)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 52, 52, 128)&#xA; Load  |  Yep!  | maxp 2x2p0_2                     | (?, 26, 26, 128)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 26, 26, 256)&#xA; Load  |  Yep!  | maxp 2x2p0_2                     | (?, 13, 13, 256)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 512)&#xA; **Load  |  Yep!  | maxp 2x2p0_1                     | (?, 13, 13, 512)**&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)&#xA; Init  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)&#xA; Init  |  Yep!  | conv 1x1p0_1    linear           | (?, 13, 13, 125)&#xA;-------+--------+----------------------------------+---------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The bold text part is causing the confusion. My expectation what (?,12,12,512) but it is not. It retains the same size (13,13)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding model info from the &lt;code&gt;.cfg&lt;/code&gt; file is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[convolutional]&#xA;batch_normalize=1&#xA;filters=512&#xA;size=3&#xA;stride=1&#xA;pad=1&#xA;activation=leaky&#xA;&#xA;[maxpool]&#xA;size=2&#xA;stride=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why is the output height/width not decreasing by 1?&lt;/p&gt;&#xA;" OwnerUserId="10019" LastActivityDate="2018-03-22T10:48:23.480" Title="max pooling size=2,stride=1 outputs same size" Tags="&lt;tensorflow&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4200" PostTypeId="1" CreationDate="2017-10-05T14:20:04.007" Score="2" ViewCount="374" Body="&lt;p&gt;After obtaining the result of algorithm fuzzy c means on the iris flower data set. Results are to be checked for their correctness. So, how to derive it in matlab. I mean code for that. please suggest.&lt;/p&gt;&#xA;" OwnerUserId="10020" LastEditorUserId="1671" LastEditDate="2018-01-28T23:02:55.883" LastActivityDate="2018-01-28T23:02:55.883" Title="confusion matrix is useful to be applied to the result of fuzzy c means clustering or not" Tags="&lt;classification&gt;&lt;matlab&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4201" PostTypeId="2" ParentId="4199" CreationDate="2017-10-05T17:37:32.247" Score="1" Body="&lt;p&gt;Darkflow uses padding when applying the pooling layer. Padding is a common mechanism for maxpooling. This allows you to keep the size the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend the chapter &quot;Stride and Padding&quot; from the article &lt;a href=&quot;https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Beginner's Guide To Understanding Convolutional Neural Networks (part 2)&lt;/a&gt;  to see how padding works in detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please also note that there have been &lt;a href=&quot;https://github.com/thtrieu/darkflow/issues/250&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent issues&lt;/a&gt; with the implementation of padding in darkflow.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-10-05T17:37:32.247" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4202" PostTypeId="1" AcceptedAnswerId="4208" CreationDate="2017-10-05T17:50:35.883" Score="4" ViewCount="1739" Body="&lt;p&gt;I was reading a little bit about the Deepstack poker program:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ualberta.ca/science/science-news/2017/march/artificial-intelligence-deepstack-outplays-poker-professionals&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepStack the first computer program to outplay human professionals at heads-up no-limit Texas hold'em poker&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1701.01724.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepStack: Expert-Level Artificial Intelligence in&#xA;Heads-Up No-Limit Poker&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first article mentions processor used was an NVIDIA GeForce GTX 1080 graphics card, but I'm trying to get an understanding of how much memory this type of AI requires.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related: How much has recent work in mathematical analysis of poker contributed to Deepstack and other strong poker AI? &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-06T16:35:50.973" LastActivityDate="2017-10-06T16:35:50.973" Title="How much memory does the Deepstack poker program require?" Tags="&lt;game-ai&gt;&lt;poker&gt;&lt;deepstack&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4203" PostTypeId="5" CreationDate="2017-10-05T17:51:09.240" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-05T17:51:09.240" LastActivityDate="2017-10-05T17:51:09.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4204" PostTypeId="4" CreationDate="2017-10-05T17:51:09.240" Score="0" Body="For question related to the Deepstack poker AI" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-05T17:51:09.240" LastActivityDate="2017-10-05T17:51:09.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4205" PostTypeId="5" CreationDate="2017-10-05T17:51:59.243" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-05T17:51:59.243" LastActivityDate="2017-10-05T17:51:59.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4206" PostTypeId="4" CreationDate="2017-10-05T17:51:59.243" Score="0" Body="For question about poker-playing AI" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-05T17:51:59.243" LastActivityDate="2017-10-05T17:51:59.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4207" PostTypeId="2" ParentId="4187" CreationDate="2017-10-05T20:43:21.867" Score="2" Body="&lt;p&gt;It depends on the algorithm.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classical algorithm would not be able to recognize the pattern of primes unless it was programmed to do so.  This pre-programming would not have to be exclusive to primes, but the algorithm would have to &quot;understand&quot; the nature of primes &lt;em&gt;including 1&lt;/em&gt; (the number can only be divided by 1 and itself.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because computers are good at number crunching, it should be quite possible to program an algorithm that looks for numerical relationships, such as primes, parity and factors, without being taught those specific concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extending this to alternate representations of numbers, 1=1, 11=2, 111=3 requires a type of abstract thinking (i.e. the literal representation is not the true meaning. In an 8-bit byte 11101101 = 237. In Base-2 1,11,111 are the primes 1,3,7;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the algorithm was superintelligent, figuring out the example would be trivial, and, ostensibly, correctly deriving patterns of any kind, no matter how opaque, would be possible, given enough time and memory.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-08T01:48:00.643" LastActivityDate="2017-10-08T01:48:00.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4208" PostTypeId="2" ParentId="4202" CreationDate="2017-10-05T21:56:21.400" Score="9" Body="&lt;p&gt;A DeepStack-style algorithm only requires that you have a way of approximating equilibrium counterfactual values for subtrees at the leaves of lookahead trees from each of its decision points. So if I'm acting at the beginning of the pre-flop and I only have the time and memory available to look ahead to the start of the flop, then I need to approximate the equilibrium counterfactual values at the root of each subtree that begins at the start of the flop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DeepStack as in the Science paper's study used three neural networks, one for values at the end of the pre-flop, one at the beginning of the flop, and one at the beginning of the turn. These, along with the lookahead tree data structures, had to fit onto the 8GB of GPU memory on the 1080. Although not needed while playing, the networks were trained on millions of examples (ten million for the pre-flop and turn networks, and one million for the flop network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have an &lt;a href=&quot;https://github.com/lifrordi/DeepStack-Leduc&quot; rel=&quot;noreferrer&quot;&gt;example DeepStack implementation on GitHub that plays Leduc hold'em&lt;/a&gt; if you want to look at some of the programming details in greater depth. Leduc hold'em is a small toy poker game that is commonly used in the poker research community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DeepStack algorithm arises out of a mathematically rigorous approach to approximating Nash equilibria in two-player, zero-sum, imperfect information games. Most of the strong poker AI to date attempt to approximate a Nash equilibria to one degree or another. So rather than analyzing poker specifically, we usually use an analysis of properties of imperfect information games more generally to get a general algorithm, then we apply these techniques to poker as a specific example of an imperfect information game.&lt;/p&gt;&#xA;" OwnerUserId="10026" LastActivityDate="2017-10-05T21:56:21.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4209" PostTypeId="1" CreationDate="2017-10-06T01:45:17.967" Score="4" ViewCount="1041" Body="&lt;p&gt;I have been working for ages on a neuro-evolution AI program, where cars learn how to race around a track.  Presently, I have a rudimentary fitness function that awards points for every degree traveled in the CW direction about the &lt;strong&gt;center of the window&lt;/strong&gt; (removes points in CCW dir) and removes a certain amount of points for every collision that occurs. Cars also lose points for every moment they stay still and/or are colliding with something. My end goal is to create cars that can complete a track full of obstacles, faster than a human controlled car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is there a better fitness function that would result in more efficient cars that&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;make better use of their sensors,&lt;/li&gt;&#xA;&lt;li&gt;are efficient in getting around the track (don't weave like a drunk driver), and &lt;/li&gt;&#xA;&lt;li&gt;are faster in general (cars are too cautious, but this is a race!)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Half of the population seems to just spin around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have fully implemented my neuro-evolution program. Here's &lt;a href=&quot;https://youtu.be/hnrQV0nKJxU&quot; rel=&quot;nofollow noreferrer&quot;&gt;my implementation&lt;/a&gt;. However, you'll find that it isn't perfect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How should I alter my fitness function to generate better driving cars?&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the cars will only turn left or right if the outer sensors are activated.  However, if the sensors directly in front are activated, the neural network is configured in such a way that &quot;ignores&quot; this signal. So, the cars crash head-on into obstacles but avoid obstacles in their periphery.  I think this is due to the fact that the fitness function (which gives points for the displacement in 3 second time period) is too generous and removes the incentive for cars to avoid obstacles.  I've tried altering the punishment for collisions and the reward for driving in the right direction but it still isn't performing the way I would like it to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some screenshots&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/22rM5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/22rM5.png&quot; alt=&quot;Population of Cars in Racing Ring&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ER7uW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ER7uW.png&quot; alt=&quot;Revive mode (revisiting past generations of cars&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9708" LastEditorUserId="2444" LastEditDate="2019-06-26T12:32:03.750" LastActivityDate="2021-01-18T05:07:40.807" Title="What is an appropriate fitness function for a simulated self-driving car?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;autonomous-vehicles&gt;&lt;fitness-functions&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4210" PostTypeId="2" ParentId="4113" CreationDate="2017-10-06T06:46:12.517" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have many symptoms and duration of each drug. I create X and y data&#xA;  but, for example, LSD have an effect duration of 180 - 720 minutes. I&#xA;  really need make 540 arrays?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can (in this particular case, it's fairly easy to generate ~800 rows in CSV once), but you don't have to: you an apply &lt;a href=&quot;https://cartesianfaith.com/2016/10/06/what-you-need-to-know-about-data-augmentation-for-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;data augmentation&lt;/a&gt; on the fly. This will add some randomness in your training, but it usually helps generalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, it seems you aren't actually using deep learning, but rather &lt;code&gt;DecisionTreeClassifier&lt;/code&gt;, which is a bit different.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-06T06:46:12.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4211" PostTypeId="1" AcceptedAnswerId="4214" CreationDate="2017-10-06T09:16:12.273" Score="4" ViewCount="239" Body="&lt;p&gt;I recently started working on very simple machine learning codes in Python and I came across a big problem: teaching the system to improve on its guesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is what the code is about: &#xA;I will have a list of organisms with their features stated in numerical values. I want to write a code that identifies that whether the organism is a cat or a fish or neither based on their characteristics. (For example an organism with a high fur value and 4 legs is more probable to be a cat.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea for the neural network is to have five input nodes(for the five characteristics) and 2 output nodes (one for how cat it is and one for how fish it is). The input nodes are multiplied by a weight value and then all summed together to produce one of the output nodes. This repeats itself for the other output. How much the system got wrong is just the difference between the value of the output nodes and how cat/fish the being actually is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can I use this information to correct the weights of the input nodes? Since the weights are randomly generated they can be in the &quot;wrong&quot; directions to begin with. For example if the subject is a cat then we should be expecting high fur and leg values. But what if the weight for the leg value is negative while the weight for the fur value is positive? Adding or multiplying the weight by the error won't bring us any closer to accurately determining the being. Is my neural network flawed to begin with? Or is there a rule of thumb in choosing back propagation algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0LDQ0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0LDQ0.jpg&quot; alt=&quot;the neural network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10030" LastEditorUserId="10030" LastEditDate="2017-10-06T09:35:38.543" LastActivityDate="2017-11-12T16:20:24.247" Title="Finding an optimum back propagation algorithm" Tags="&lt;machine-learning&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4212" PostTypeId="2" ParentId="4200" CreationDate="2017-10-06T13:46:49.893" Score="2" Body="&lt;p&gt;Yes, you can use confusion matrix as a performance metric for a classifier where the output of the classifier is class labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your classifier output is a likelihood floating point value for each of the classes, then you have to use some scheme of converting the likelihoods to a class label. For example, use softmax function for one hot encoding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for the Fuzzy K Means clustering based classifier, you'll have to pick output class label as the 'index' if the Fuzzy cluster with maximum likelihood/ membership score. This will constitute predicted labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using array of test labels and predicted labels to compute confusion matrix is straightforward in any computational platform, including Matlab of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&#xA;Fuzzy K Means (also called fuzzy c means or FCM) is an unsupervised learning method. This means the assignment of training data points to clusters is not based on their class labels. Therefore, FCM is not used as a discriminative classifier. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, FCM is used as an intermediate module to encode data. The encoded representation is used as a feature vector, in supervised learning, to train a discriminative classifier, such as a SVM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not to say you can not use FCM for classification, but it would only work when data points of the same label occur in reasonably distinct clusters. This almost never happens with real world data and so it's recommended not to use FCM for classification.&lt;/p&gt;&#xA;" OwnerUserId="9506" LastEditorUserId="9506" LastEditDate="2017-10-11T19:18:51.997" LastActivityDate="2017-10-11T19:18:51.997" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4213" PostTypeId="1" CreationDate="2017-10-06T14:03:38.610" Score="4" ViewCount="396" Body="&lt;p&gt;i've been trying lately to search the internet to find a result for this but no useful results, unfortunately, we're making an uber-like application during our discussion we needed a way to validate a rider's rating after he completes his ride with a taxi, but the main question is, does uber use an algorithm to do so? and how?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="10032" LastEditDate="2017-10-06T18:47:22.060" LastActivityDate="2018-04-16T17:58:15.187" Title="How does Ubers rating System algorithm work?" Tags="&lt;algorithm&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4214" PostTypeId="2" ParentId="4211" CreationDate="2017-10-06T14:42:03.267" Score="5" Body="&lt;p&gt;When you are training a neural network, you use an algorithm called back propagation. This algorithm uses partial derivatives to determine the optimal values for weights. Partial derivatives are a calculus based method which tell you how far you need to adjust the weights in order to get to an optimum value. However, when you have neural networks with many different weights you need to be able to decide which ones to update in order to achieve a more optimal network. This is where stochastic gradient descent comes in. This algorithm is able to take many different gradients and traverse through many dimensional space into a lower error rate. Also, from your diagram of your neural network, I do not see any hidden neurons, which are neurons between the input and output layers which do most of the actual work. While you might be able to train your neural network optimally without hidden neurons, it might improve your accuracy depending upon your dataset. Finally, implementing a back propagation algorithm is a lot of work which is why many people use pre built libraries. Unless you are doing research, I recommend you do the same. I think for a problem like this, pybrain would be perfect, and if you want to use deep neural networks, tensorflow would be good for that.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-10-06T14:42:03.267" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4215" PostTypeId="1" CreationDate="2017-10-06T14:47:48.343" Score="3" ViewCount="393" Body="&lt;p&gt;I'm working on a system which reads large data from Excel and tries to interpret something from it. If it doesn't understand something, the user will correct it and slowly system will learn and try to do everything itself. Is there any AI SaaS or anything that can help in this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use case is, I'm reading bank statements from different banks and they all have different types of narration for similar transactions. I want to put them all into an accounting software without any human intervention.&lt;/p&gt;&#xA;" OwnerUserId="10033" LastActivityDate="2021-05-01T04:05:35.643" Title="What is the best AI or Machine Learning SaaS application?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4216" PostTypeId="1" CreationDate="2017-10-06T18:00:12.530" Score="3" ViewCount="89" Body="&lt;p&gt;Traditionally, &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Strong_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;strong AI&lt;/a&gt;&quot; refers to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;, the human mind understood as an algorithm (&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room#Strong_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Searle, Chinese Room&lt;/a&gt;) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Consciousness&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But recent advances in Artificial Intelligence, most notably breakthroughs involving Machine Learning and Neural Networks, have led to strong AI being used in the context of an algorithmic intelligence that can outperform the top humans in specific tasks.  (This is sometimes qualified as &quot;strong narrow AI&quot;, but the qualifier seems less used time goes on.) &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Because the traditional usage of strong AI are based on theoretical concepts, where the current usage is based on practical, real-world results, does the new usage of &quot;strong AI&quot; supercede the traditional usage?  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If this is the case, at what point does the Wiki need to be updated?  Is a major paper focused on re-coining the term necessary to formally reference the new usage of the term, or are numerous, informal uses sufficient?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Related: &quot;Strong&quot; has a precise, analytic, formal usage in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved games&lt;/a&gt;.  This game theoretic usage is similar in spirit to new usage of strong AI in that it connotes results that can be validated. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-06T18:46:13.040" LastActivityDate="2021-05-26T00:04:40.443" Title="Is the traditional meaning of &quot;strong AI&quot; outmoded?" Tags="&lt;agi&gt;&lt;terminology&gt;&lt;history&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4217" PostTypeId="1" CreationDate="2017-10-07T00:55:47.947" Score="1" ViewCount="56" Body="&lt;p&gt;Is there a technical term for an image classifier that classifies on a single class but is classifying on an amount like how full a glass of water is rather than different classes?&lt;/p&gt;&#xA;" OwnerUserId="2788" LastActivityDate="2017-10-07T17:06:30.263" Title="Is there a technical name for image classification on amount instead of class?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4218" PostTypeId="1" AcceptedAnswerId="4648" CreationDate="2017-10-07T04:57:03.580" Score="3" ViewCount="107" Body="&lt;p&gt;I am having issues getting started with a multi class problem with multiple features and hoping someone could please point me in the right direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data that is structured like this for training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Item State Code1 Code2 Code3 Route&#xA;---  ---   ---   ---   ---   ---&#xA;item1 MI   A1    33    blue  Route1&#xA;item2 TX   A3    35    yellow Route2&#xA;item3 NM   A4    36    green  Route3&#xA;item4 NM   A4    37    green  Route3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Essentially I am trying to figure out where to even start. The goal is to know where to route the Items based on the features State, Code1,2, and 3. The route is dependent on a mix of the codes and state, and I want to build a model that says when I have code X, Y, Z and Color XX, then it is probably route 1 (some routes of course in the training data might have X, Y as codes and a different Z) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming I will need to one-hot encode the features like the State and codes? But from there does anyone know of which type of model I should go for? I would assume a Neural Net of some kind, I've explored CNN's and Random Forrest. &lt;/p&gt;&#xA;" OwnerUserId="10040" LastActivityDate="2018-08-20T19:21:13.893" Title="Method for Multi-class/category?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4219" PostTypeId="1" AcceptedAnswerId="4270" CreationDate="2017-10-07T09:54:51.200" Score="10" ViewCount="4163" Body="&lt;p&gt;I am looking for something similar to IBM Watson but open source.  &lt;/p&gt;&#xA;" OwnerUserId="10046" LastEditorUserId="2444" LastEditDate="2019-06-29T22:26:35.007" LastActivityDate="2021-02-25T09:48:18.847" Title="Is there any open source counterpart to the IBM Watson?" Tags="&lt;natural-language-processing&gt;&lt;watson&gt;&lt;open-source&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="4220" PostTypeId="2" ParentId="4217" CreationDate="2017-10-07T17:06:30.263" Score="3" Body="&lt;p&gt;Training a system to predict real numbers instead of classes is called regression. If you had a training data where the inputs were images of water glasses and the targets were the amount of water in each glass, then this would be regression task from images of water glasses to the amount of water in each glass. You could call it image regression in the same way you would say image classification, but beyond that, I don't think there's a special term for regression where the inputs are images. &lt;/p&gt;&#xA;" OwnerUserId="10026" LastActivityDate="2017-10-07T17:06:30.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4221" PostTypeId="2" ParentId="4117" CreationDate="2017-10-07T23:21:31.150" Score="2" Body="&lt;p&gt;Object detection is a regression of the bounding box (rectangle) around the object. In this way, the two ways you suggest are equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I suggest you to look at is &lt;a href=&quot;https://medium.com/towards-data-science/lane-detection-with-deep-learning-part-1-9e096f3320b7&quot; rel=&quot;nofollow noreferrer&quot;&gt;lane detection&lt;/a&gt; for self-driving cars - a well studied problem. It seems that this is a very relevant task (if not the same), so its solution should work for you as well.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-07T23:21:31.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4222" PostTypeId="2" ParentId="3962" CreationDate="2017-10-07T23:40:18.953" Score="1" Body="&lt;p&gt;The given so far answers focus on numerical methods to check your gradients. It is &lt;em&gt;really&lt;/em&gt; useful, especially if one doesn't have much experience in backprop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'd like to add here a pure practical &quot;sanity check&quot;, relatively fast and easy to perform, which also works for other issues, e.g. (rough) hyperparameters selection. To see if your network makes sense, reduce the training set to a few examples and try to overfit the network. If the loss falls to zero and training accuracy skyrockets to 1, it means that both passes work correctly and can move on to the real training. Otherwise, something's not right and should dive into specific parts of the network, in particular check the gradients numerically.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-07T23:40:18.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4223" PostTypeId="1" CreationDate="2017-10-08T04:58:09.930" Score="3" ViewCount="912" Body="&lt;p&gt;I'm trying to develop a project that recognises handwriting and converts into text. What are the algorithms and tools to be used?&lt;/p&gt;&#xA;" OwnerUserId="10061" LastEditorUserId="9210" LastEditDate="2017-11-09T19:05:59.973" LastActivityDate="2017-11-09T19:05:59.973" Title="how to recognise handwriting and convert into text?" Tags="&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4224" PostTypeId="2" ParentId="4061" CreationDate="2017-10-08T08:12:04.967" Score="0" Body="&lt;p&gt;I can judge from the excerpt only, but I don't see any mention of a convolutional layer there. This doesn't mean that convolution isn't suitable for this task (in fact, it's the best method for image classification so far), it's just seems the instructor proposes this particular network architecture, without convolutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this architecture is more illustrative and by no means a standard, I can't be sure I fully understand it. But it seems like the first layer outputs a vector of size &lt;code&gt;4&lt;/code&gt;, which contains the counts of four &lt;em&gt;selected&lt;/em&gt; colors. The ReLu operation here is a fancy way of filtering extra colors, based on channel value (note that they are close to either &lt;code&gt;0&lt;/code&gt; or &lt;code&gt;255&lt;/code&gt; in all four colors). If you select different four colors, this filtering will have to be done differently too, so it's basically hardwired. The second layer is more straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you shouldn't consider this a &quot;true&quot; deep network, but merely a puzzle to get familiar with ReLu operation.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-08T08:12:04.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4228" PostTypeId="2" ParentId="4219" CreationDate="2017-10-08T18:19:42.823" Score="2" Body="&lt;p&gt;Have a look at the &lt;a href=&quot;http://chatscript.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChatScript project&lt;/a&gt;, whose description is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ChatScript is a &quot;next Generation&quot; chatbot engine, based on the one that powered Suzette, that won the 2010 Loebner Competition. ChatScript has many advanced features and capabilities that, when properly utilitized, permit extremely clever bots to be programmed. There is also a potentially useful ontology of nouns, verbs, adjectives, and adverbs for understanding meaning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1796" LastEditorUserId="2444" LastEditDate="2019-06-30T17:46:02.657" LastActivityDate="2019-06-30T17:46:02.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4230" PostTypeId="2" ParentId="4176" CreationDate="2017-10-08T22:32:40.133" Score="3" Body="&lt;p&gt;Fuzzy logic seems to have multiple of applications historically in Automotive Engineering.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found an interesting article on the subject from 1997. This excerpt provides an interesting rationale:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The key reason for fuzzy logic’s success in automotive engineering lies in the implications of its paradigm shift. Previously, engineers spent much time creating mathematical models of mechanical systems. More time went to real-world road tests that tuned the fudge factors of the control algorithms. If they succeeded, they ended up with a control algorithm of mathematical formulas involving many experimental parameters. Modifying or later optimizing such a solution is very difficult because of its lack of transparency. Fuzzy logic makes this design process faster, easier, and more transparent. It can implement control strategies using elements of everyday language. Everyone familiar with the control problem can read the fuzzy rules and understand what the system is doing and why. It also works for control systems with many control parameters. Designers can build innovative control systems that would have been intractable using traditional design techniques. &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;ftp://ftp.me.psu.ac.th/pub/me/Fuzzy/88constantin.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy Logic in&#xA;  Automotive Engineering&lt;/a&gt;, 1997&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;Here are some papers and patents for automatic transmission control in motor vehicles. One of them is fairly recent: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://papers.sae.org/930674/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automatic Transmission Shift Schedule Control Using Fuzzy Logic&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: Society of Automotive Engineers, 1993&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.tandfonline.com/doi/abs/10.1080/00423119508969099?journalCode=nvsd20&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy Logic in Automatic Transmission Control&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: International Journal of Vehicle Mechanics and Mobility, 2007&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ipasj.org/IIJCS/Volume2Issue7/IIJCS-2014-06-24-3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy Logic Based Controller For Automated&#xA;Gear Control in Vehicles&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: International Journal of Computer Science, 2014&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/patents/US4841815&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy control system for automatic transmission&lt;/a&gt; | Patent | 1987&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/patents/US5390117&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transmission control with a fuzzy logic controller&lt;/a&gt; | Patent | 1992&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise with fuzzy logic anti-lock breaking systems (ABS):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.fuzzytech.com/e/e_a_sie.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Antilock-Braking System and Vehicle Speed Estimation using Fuzzy Logic&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: FuzzyTECH, 1996&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ijser.org/researchpaper/FUZZY-LOGIC-ANTI-LOCK-BRAKE-SYSTEM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy Logic Anti-Lock Break System&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: International Journal of Scientific &amp;amp; Engineering Research, 2012&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/patents/US5416709?dq=fuzzy%20logic%20abs&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwiV8dOaguLWAhUHLSYKHbtQC18Q6AEIKDAA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy controller for anti-skid brake systems&lt;/a&gt; | Patent | 1993&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method seems to have been extended to aviation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://aix.acad.ro/sectii2002/proceedings/doc2_2004/12_Ursu.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Fuzzy Logic Control Synthesis for an Airplane Antilock-Breaking System&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: Proceedings of the Romanian Academy, 2004&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/patents/US8955793?dq=fuzzy%20logic%20abs&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwiV8dOaguLWAhUHLSYKHbtQC18Q6AEIVDAG&quot; rel=&quot;nofollow noreferrer&quot;&gt;Landing gear method and apparatus for braking and maneuvering&lt;/a&gt; | Patent | 2003&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-08T22:48:07.803" LastActivityDate="2017-10-08T22:48:07.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4231" PostTypeId="2" ParentId="4176" CreationDate="2017-10-08T22:45:39.803" Score="2" Body="&lt;p&gt;The site &lt;a href=&quot;http://www.fuzzytech.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FuzzyTECH&lt;/a&gt; lists an array of applications:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://www.fuzzytech.com/e/e_a_plc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Industrial Automation&lt;/a&gt; &lt;br&gt; &#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_glc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monitoring Glaucoma&lt;/a&gt; &lt;br&gt; &#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_htw.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coal Power Plant&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_kli.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Complex Chilling Systems&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_mull.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Refuse Incineration Plant&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_eet.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy Logic Design&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_pfd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Design&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_dek.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Water Treatment System&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_spe.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Truck Speed Limiter&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_med.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Medical Shoe&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_med.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy in Appliances&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_esw.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automotive Engineering&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_sie.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Antilock Braking System&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_ap_afp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aircraft Flight Path&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_ap_nfu.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nucluar Fusion&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_mot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Motorla 68HC12 MCU&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_tfc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Traffic Control&lt;/a&gt; &lt;br&gt;&#xA;  &lt;a href=&quot;http://www.fuzzytech.com/e/e_a_kumm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sonar Systems&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of the linked articles have good bibliographies citing numerous papers, although it's notable that most of the material is a few decades old.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-10-08T22:45:39.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4233" PostTypeId="1" CreationDate="2017-10-09T06:04:50.650" Score="4" ViewCount="1908" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How to train darkflow for my custom object really really fast during debugging in quad core PC and without GPU? (Can I train with about 10 images and test with only those images, just to check if all convolutions are working as expected. And with 20 epoch?). There is only once class and all license plates are similar in pattern with varying in angle and its digits in plate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am using tiny yolo config and weight. So, what all parameters I should tune in yolo based .cfg file to do it? I feel if TV like object can be detected with same training weight and config for tiny yolo then license plate too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overview:&lt;/strong&gt;&#xA;I am training for object, license plate, with darkflow. I tried with about 100 custom datasets I had created. This is only for initial POC, actual implementation will have more number of images. And upon testing with test images with trained graph, object is not highlighted rather highlighting squares are shown at random location within image and random in number, starting with 2-3 square boxes to lot many number. But non of those were highlighting to the actual license plate object. It took me 20 hour of training time to verify it. I used training images for testing as well, and also a plane black screen images to test what's going on. But highlighting squares are still random in number and location even on blank screen image.&lt;/p&gt;&#xA;" OwnerUserId="10083" LastActivityDate="2017-12-24T07:47:56.887" Title="How to train for own dataset really really fast while debugging" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4234" PostTypeId="1" AcceptedAnswerId="4249" CreationDate="2017-10-09T08:51:00.787" Score="2" ViewCount="116" Body="&lt;p&gt;I  recently became interested in how creativity is generated in NN. My understanding of NNs is that the output always known, given that it they are trained with target values, but how does one train a network to be creative, I mean in such cases would the task be to create something novel, but the actual target would not be known.  How does creativity express itself in a rule based system, such as NN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One would need to redefine the cost-function, but how does one imply creativity in a rule based system?&lt;/p&gt;&#xA;" OwnerUserId="6517" LastEditorUserId="1671" LastEditDate="2017-10-11T18:16:32.163" LastActivityDate="2017-10-11T18:16:32.163" Title="How is creativity generated in a currently rule based neural network?" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4236" PostTypeId="1" AcceptedAnswerId="4292" CreationDate="2017-10-09T13:13:35.987" Score="-1" ViewCount="51" Body="&lt;p&gt;I'm working on a content based recommendation engine for ebooks. I create document vectors with 300 features for every ebook using a word2vec model trained on google news and determine recommendations based on the closest vectors. So far I have run tests on a dataset consisting of 200 books from the gutenberg project in four different categories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find that a small group of books appears to be recommended a lot more then others. The most recommended book is recommended for over half the dataset. This book is Theaetus, in which the &lt;a href=&quot;https://gist.github.com/HartgerV/50ccdb999393eb1fd2b50725e14cd0fa&quot; rel=&quot;nofollow noreferrer&quot;&gt;most commmon tokens&lt;/a&gt; are fairly specific to the book (tokens like socrates, theaetus). I can find no intuitive reason why this book would match so well with over half my dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this common behavior when using document vectors to determine similarity? Are there any methods that would reduce this effect?&lt;/p&gt;&#xA;" OwnerUserId="10092" LastEditorUserId="2444" LastEditDate="2019-04-16T22:38:27.297" LastActivityDate="2019-04-16T22:38:27.297" Title="Over-exposure of certain items in content based recommendation engine" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4238" PostTypeId="1" CreationDate="2017-10-09T20:34:23.590" Score="1" ViewCount="125" Body="&lt;p&gt;I've gotten curious about this topic and am wondering what the stack exchange community has to say about it. Also, does anyone know of any professors/researchers who have published papers pertaining to this?&lt;/p&gt;&#xA;" OwnerUserId="10101" LastActivityDate="2017-10-13T09:02:38.120" Title="To what extent can artificially intelligent agents reliably predict trends in financial markets?" Tags="&lt;research&gt;&lt;multi-agent-systems&gt;&lt;real-time&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4239" PostTypeId="2" ParentId="4238" CreationDate="2017-10-09T21:49:27.463" Score="2" Body="&lt;p&gt;A couple of thoughts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans can't reliably predict trends in the stock market, so expecting AI's to do so is probably unreasonable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The above would be more true if it were proven that the movement of stock prices is really a random walk, but my understanding is that the current thinking is that stock movements aren't &lt;strong&gt;completely&lt;/strong&gt; random... but just really close to random.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If there are some trends there that represent a useful signal, and if somebody has found that, they're &lt;strong&gt;very&lt;/strong&gt; unlikely to share that information, as the market would then immediately price that information in and they would lose their edge.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-10-09T21:49:27.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4240" PostTypeId="2" ParentId="4238" CreationDate="2017-10-10T03:14:49.640" Score="3" Body="&lt;p&gt;This is a highly relevant question as market trends have become more emphasized over the fundamentals of individual companies, and algorithmic trading has proven to be quite effective, particularly in areas such as high-frequency micro-trading. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://fortune.com/2013/05/29/a-day-in-the-quiet-life-of-a-nyse-floor-trader/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2013 Forbes article&lt;/a&gt; estimated nearly 80% of stock trading volume in the U.S. is conducted by &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_trading_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;automated trading systems&lt;/a&gt;.  More recently, Bloomberg published an article on the subject: &lt;a href=&quot;https://www.bloomberg.com/news/articles/2017-06-15/it-s-a-quant-s-stock-market-as-computer-programs-keep-on-buying&quot; rel=&quot;nofollow noreferrer&quot;&gt;The U.S. Stock Market Belongs to Bots&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The massive adoption of algorithmic trading in a fairly short span is a pretty good indicator that these systems are effective.  However,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;My feeling with any predictive system is that it works until it doesn't&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imperfect&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Complete_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;Incomplete information&lt;/a&gt; are persistent in real-world scenarios.  The implication is that there are always unknown factors that can diminish quality of analysis, and even lead to disastrous outcomes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter how complex an algorithm becomes, reality is more complex, &lt;em&gt;(at least for the foreseeable, pre-singularity future;)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding to that, systems in general tend to be imperfect, and when engaged in massively repetitive functions, small errors can grow exponentially and produce catastrophic effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Flash_crash&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flash Crashes&lt;/a&gt;&quot; are a recent phenomena, and a result of algorithmic, high-frequency, black-box trading.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-10-10T03:14:49.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4241" PostTypeId="2" ParentId="3777" CreationDate="2017-10-10T12:58:39.197" Score="0" Body="&lt;p&gt;I myself working on similar product for Medical Diagnosis. The reason for it is we as ML engineer, generally try to replicate, how human approach any problem into mathematical model and build library on top of it to use it in application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how Doctor approach to any specific decision of diagnosis, based on symptoms.&#xA;Mainly symptoms are in two form True or False, you have it or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we just need to replicate doctors approach to make decision, But yes to extract those symptoms from raw data in the form of text, image of sound we need to use other classifier and clustering models. &lt;/p&gt;&#xA;" OwnerUserId="8197" LastActivityDate="2017-10-10T12:58:39.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4243" PostTypeId="2" ParentId="4223" CreationDate="2017-10-10T13:12:44.097" Score="2" Body="&lt;p&gt;What you described sounds to me like &lt;a href=&quot;https://en.wikipedia.org/wiki/Optical_character_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optical Character Recognition(OCR)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to implement your own, I would say read through how an open source OCR like Tesseract was implemented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise just google for OCR and you will find a list of OCR engines, both comercial and open-source. To list but a few: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Tesseract: &lt;a href=&quot;https://github.com/tesseract-ocr/tesseract&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tesseract-ocr/tesseract&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Matlab Computer Vision Toolbox's OCR: &lt;a href=&quot;http://www.mathworks.de/de/help/vision/ref/ocr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.de/de/help/vision/ref/ocr.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Most OCRs makes use of a combination of a segmantation and image classification(usually done with Convolutional Neural Networks). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.explainthatstuff.com/how-ocr-works.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="7550" LastActivityDate="2017-10-10T13:12:44.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4244" PostTypeId="2" ParentId="3784" CreationDate="2017-10-10T13:49:26.603" Score="1" Body="&lt;p&gt;Sorry but no one can give you a number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As stated already in the comment by Manngo, the general rule: is as much as you can. I have also seen the 10x number being thrown around, see &lt;a href=&quot;http://fastml.com/how-much-data-is-enough/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. According to this &quot;rule&quot;, you need  roughly 10 times as many examples as there are degrees of freedom in your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much data you will need is something you can only answers to through empirical investigation. The amount of data you need to collect is influenced by the complexity of the data, the dimensionality of the data and the algorithm you intend to train(although this can be controlled somewhat through regulirazation, see this good blog post by &lt;a href=&quot;https://jakevdp.github.io/blog/2015/07/06/model-complexity-myth/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jake Vanderplas&lt;/a&gt; for more information). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/51490/how-large-a-training-set-is-needed&quot;&gt;This question&lt;/a&gt;  might be of interest to you.&lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-10-10T14:16:22.747" LastActivityDate="2017-10-10T14:16:22.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4245" PostTypeId="1" CreationDate="2017-10-10T16:33:45.657" Score="5" ViewCount="1111" Body="&lt;p&gt;How are the layers in a encoder connected across the network for normal encoders and auto-encoders? In general, what is the difference between encoders and auto-encoders?&lt;/p&gt;&#xA;" OwnerUserId="10118" LastEditorUserId="2444" LastEditDate="2019-04-14T10:02:07.033" LastActivityDate="2019-04-19T22:53:19.563" Title="What is the difference between encoders and auto-encoders?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;autoencoders&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4247" PostTypeId="1" CreationDate="2017-10-11T00:55:26.993" Score="2" ViewCount="817" Body="&lt;p&gt;I am trying to replicate the &lt;em&gt;fully convolutional networks&lt;/em&gt; (FCN) concept described &lt;a href=&quot;https://www.youtube.com/watch?v=nDPWywWRIRo&amp;amp;t=16m30s&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for semantic segmentation. It seems people have successfully trained such models by removing fully connected layers from the popular pre-trained models such as VGG and adding &lt;em&gt;upsampling&lt;/em&gt; and &lt;em&gt;unpooling layers&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that &lt;em&gt;transpose convolution&lt;/em&gt; and &lt;em&gt;unpooling&lt;/em&gt; in upsampling layers provide counterparts of &lt;em&gt;convolution&lt;/em&gt; and &lt;em&gt;max (or average) pooling&lt;/em&gt; in earlier downsampling layers respectively, but what are the counterparts for non-linearities such as ReLU? What about dropout? There seems to be no discussion of this in the video.&lt;/p&gt;&#xA;" OwnerUserId="9653" LastEditorUserId="2444" LastEditDate="2020-06-14T10:54:57.413" LastActivityDate="2020-06-14T10:54:57.413" Title="What are the counterparts of non-linearities and dropout in fully convolutional networks?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;dropout&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4248" PostTypeId="2" ParentId="4247" CreationDate="2017-10-11T04:58:13.150" Score="1" Body="&lt;p&gt;All-convolutional neural network is a more general concept which can be (and is often) used without deconvolutional and unpolling layers, e.g. for an ordinary classification task. The idea is to replace the pooling and fully-connected layer with particular convolutional layers that do the same. Note that activation functions and dropout are not affected by this transformation. You can read more about it &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same will be true for the deconvolutional layers. I didn't find it stated explicitly in the &lt;a href=&quot;https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;, but here's &lt;a href=&quot;https://arxiv.org/pdf/1511.00561.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;another work&lt;/a&gt; that applies fully-convolutional network for image segmentation. On the figure 2, you can see ReLu activations: they are placed after (simple) convolutional layers, in both upsampling and downsampling parts. They don't use dropout, but batchnorm instead, but if there were a dropout, just as before, it should be placed after the convolutional layer. The keep probability, though, should probably be higher - this relates to hyperparameters tuning and may be different depending on the problem and available data.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-11T04:58:13.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4249" PostTypeId="2" ParentId="4234" CreationDate="2017-10-11T11:38:46.880" Score="2" Body="&lt;p&gt;Your question is a bit broad. Some examples of creativity and how it's achieved (in all of them there is an ordinary convolutional network trained via supervised learning, but applied differently):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Google Deep Dream uses &lt;a href=&quot;https://cs.stackexchange.com/questions/56406/google-deepdream-elaborated/82074&quot;&gt;backpropagation to the image&lt;/a&gt; to tweak the image towards maximum activation in the convolutional layer. As a result, it finds lots of &quot;things&quot; in the image and enhances them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Hrgx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Hrgx.jpg&quot; alt=&quot;deep-dream&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://kvfrans.com/generative-adversial-networks-explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks&lt;/a&gt; use two competing networks (generator and discriminator) to generate samples from the training manifold:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tDGXt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tDGXt.jpg&quot; alt=&quot;generated-bedrooms&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/jcjohnson/neural-style&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuro style&lt;/a&gt; method captures the style of one image via Gram matrix of convolutional layer activations and trains (another) neural network with a special loss function that minimizes the joint distance to the incoming image (content) and to the Gram matrix (style). The result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8LbQk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8LbQk.jpg&quot; alt=&quot;neuro-style&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-11T11:38:46.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4250" PostTypeId="1" CreationDate="2017-10-11T16:01:18.823" Score="2" ViewCount="138" Body="&lt;p&gt;I was going through &quot;&lt;a href=&quot;https://www.tensorflow.org/tutorials/wide_and_deep&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wide &amp;amp; Deep Learning&lt;/a&gt;&quot; tensorflow tutorial &amp;amp; it's quite simply explained the process. But I missed few of the things. If someone can please explain them to me, it will be of great help:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Why &lt;code&gt;occupation&lt;/code&gt; and &lt;code&gt;native_country&lt;/code&gt; we are using &lt;code&gt;tf.feature_column.categorical_column_with_hash_bucket&lt;/code&gt; and again using &lt;code&gt;tf.feature_column.embedding_column&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Why in &lt;code&gt;tf.feature_column.embedding_column&lt;/code&gt;, we are taking &lt;code&gt;dimension=8&lt;/code&gt;, even though they have more unique values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Why we are using &lt;code&gt;crossed_columns&lt;/code&gt; variables? In the document there's an explaination given. But I'm not fully understanding it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The questions might sound silly, but I'm not sure of the answers to these questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9664" LastEditorUserId="7550" LastEditDate="2017-10-12T06:42:08.630" LastActivityDate="2017-10-12T06:42:08.630" Title="Wide &amp; Deep Learning Explanation" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4251" PostTypeId="1" CreationDate="2017-10-11T20:11:41.450" Score="4" ViewCount="4446" Body="&lt;p&gt;How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation?&lt;/p&gt;&#xA;&lt;p&gt;My idea is to come up with a system that is empowering the user’s imagination to create and experience stories that engage and immerse them. So they can have the control over the story creation/narrative?&lt;/p&gt;&#xA;&lt;p&gt;A system that could enable the following use cases:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;We sit down in our entertainment room. Select a genre from a menu. Pick the character types for the cast. Decide on a generic plot or give the system the option of picking a random choice. Sit back and enjoy.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Let's say, I want to watch a sci-fi movie set in Paris with Marlon Brando and Jim Carrey. Then the system instantly generates such a movie.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;It has come to my attention that SoarTech developed a pattern-of-life capability for the U.S. Army Simulation: &lt;a href=&quot;https://youtu.be/CitevFwa3TE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agent-based Patterns of Life for Virtual Environments&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That the University of Twente in the Netherlands has been working on a project called the &lt;a href=&quot;http://www.virtualstoryteller.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Virtual Storyteller&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is a research paper entitled &lt;a href=&quot;http://wwwhome.cs.utwente.nl/%7Etheune/PUBS/Narrator-final-clin17.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The automatic generation of narratives&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And more recently that IBM has developed a &lt;a href=&quot;https://developer.ibm.com/patterns/create-a-virtual-reality-speech-sandbox/&quot; rel=&quot;nofollow noreferrer&quot;&gt;speech recognition sandbox&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I know that the answer probably requires tech, AI, speech, VR and AR, and Watson integration specialists, but any help is welcome.&lt;/p&gt;&#xA;&lt;p&gt;Watson probably won't be a necessary component of the solution, as long it is an efficient voice recognition software with speech to animation features.&lt;/p&gt;&#xA;" OwnerUserId="10143" LastEditorUserId="2444" LastEditDate="2020-12-29T21:33:37.403" LastActivityDate="2020-12-29T21:33:37.403" Title="How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;natural-language-understanding&gt;&lt;natural-language-generation&gt;&lt;story-generation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4252" PostTypeId="2" ParentId="4251" CreationDate="2017-10-11T21:29:51.380" Score="2" Body="&lt;p&gt;The VR and computer speech aspects are entirely corollary.  Adding them in would be relatively trivial in comparison to creating an algorithm that can dynamically generate stories &lt;em&gt;of interest to humans.&lt;/em&gt;  Essentially, aesthetic components not related to story structure (images, sounds, speech) are &quot;window dressings&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A story generation algorithm would have to focus on the structural elements of storytelling, possibly in conjunction with human-user reactions, and generate narratives narratives dynamically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a system would be validated by human reaction to the stories, and the validation would not require graphics, sounds or speech.  Think of a movie or video game with a story as the expressions of stories that originally had textual form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game theory could certainly be applied to this process--creation of narrative in particular utilizes game theoretic concepts, consciously or unconsciously, in terms of strategies to engage reader/viewers and provide &quot;emotional payoffs&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should certainly be possible to develop an algorithm with these capabilities, even where the algorithm doesn't not really &quot;understand&quot; the content it is generation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar endeavor might be found in &lt;a href=&quot;http://www.cameronius.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cameron Browne&lt;/a&gt;'s work on algorithms that can create simple, combinatorial games of interest to humans: &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.421.3450&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Game Design&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to note that the game created by Ludi are based on pre-existing mechanics, and algorithms don't yet have the capability to make profound leaps in the way that human game designers can. &lt;em&gt;(In other words, it will likely be a very long time before an algorithm can create a game like Magic: The Gathering where no such gaming system previously existed.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this doesn't present much of an issue in the context of procedural narrative generation in that the most successful literary work in terms of audience/sales tends to be based on pre-existing formulas.  So an AI that can merely reconfigure elements in a formula should be able to generate narratives of interest to the average human.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-10-11T21:29:51.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4253" PostTypeId="2" ParentId="92" CreationDate="2017-10-12T08:05:51.527" Score="2" Body="&lt;p&gt;There is already many good answers, I will just add to those that came before mine: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of images you are referring to are called &lt;a href=&quot;http://arxiv.org/pdf/1412.1897&quot; rel=&quot;nofollow noreferrer&quot;&gt;adversarial perturbations&lt;/a&gt;, (see &lt;a href=&quot;https://arxiv.org/pdf/1412.1897.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, and it is not limited to images, it has been shown to apply to text too, see &lt;a href=&quot;https://arxiv.org/abs/1707.07328&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jia &amp;amp; Liang, EMNLP 2017&lt;/a&gt;. In text, the introduction of an irrelevant sentence which doesn't contradict the paragraph has been seen to cause the network to come to a completely different answer (see see &lt;a href=&quot;https://arxiv.org/abs/1707.07328&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jia &amp;amp; Liang, EMNLP 2017&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The reason they work&lt;/strong&gt; is due to the fact that neural network view images in a different way from us, coupled with the high dimentionality of the problem space. Where we see the whole picture, they see a combination of features which combine to form an object(&lt;a href=&quot;https://arxiv.org/abs/1610.08401&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moosavi-Dezfooli et al., CVPR 2017&lt;/a&gt;). According to  perturbation generated against one network has been seen to have high likelihood to work on other networks: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7zK85.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7zK85.jpg&quot; alt=&quot;perturbations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the figure above, it is seen that The universal perturbations computed for the VGG-19 network, for example, have a fooling ratio above 53% for all other tested architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do you deal with the threat of adversarial perturbations? &#xA;Well, for one, you can try to generate as many perturbations as you can and use them to fine-tune your model. Whist this somewhat solves the problem, it doesn't solve the problem entirely. In (&lt;a href=&quot;https://arxiv.org/abs/1610.08401&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moosavi-Dezfooli et al., CVPR 2017&lt;/a&gt;) the author reported that, repeating the process by computing new perturbations and then fine-tuning again seems to yield no further improvements, regardless of the number of iterations, with the fooling ratio hovering around 80%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perturbations are an indication of the shallow pattern matching that neural networks perform, coupled with their minimal lack of in-depth understanding of the problem at hand. More work still needs to be done. &lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-10-12T08:18:30.067" LastActivityDate="2017-10-12T08:18:30.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4256" PostTypeId="2" ParentId="4245" CreationDate="2017-10-12T20:36:32.803" Score="1" Body="&lt;p&gt;To answer this rather succinctly, an encoder is a function mapping some input to some different space. An example of this is what the brain does. We have to process the sensory input that the environment gives us in order for it to be storable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An autoencoder's job, on the other hand, is to learn a representation(encoding). An autoencoder will have the same number of output nodes as there are inputs for the purposes of reconstructing the inputs instead of trying to predict the Y target. Autoencoders are usually used in reducing output dimensions in high dimensional data sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I answered your question!&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-10-12T20:36:32.803" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4257" PostTypeId="1" AcceptedAnswerId="4262" CreationDate="2017-10-12T21:46:05.993" Score="2" ViewCount="185" Body="&lt;p&gt;How will morality questions be settled in the domain of self-driving cars?&lt;/p&gt;&#xA;&lt;p&gt;For example&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If a dog is crossing the road, I'd expect the car to try to avoid it. But what if this leads to .00001% more risk for the driver? What is the 'risk cut-off'?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What if a cockroach is crossing the road? Will the car have a list of animals okay to run over?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What if a kid is crossing the street and avoiding it would kill the driver?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;These questions seem to not really have an answer, yet self-driving cars are almost ready. What are they doing about all of this?&lt;/p&gt;&#xA;" OwnerUserId="9648" LastEditorUserId="2444" LastEditDate="2021-01-12T19:45:36.630" LastActivityDate="2021-01-12T19:47:35.490" Title="How will morality questions be settled in the domain of self-driving cars?" Tags="&lt;autonomous-vehicles&gt;&lt;social&gt;&lt;legal&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="1" ClosedDate="2021-01-12T19:41:44.453" ContentLicense="CC BY-SA 4.0" />
  <row Id="4261" PostTypeId="2" ParentId="4238" CreationDate="2017-10-13T08:56:19.797" Score="1" Body="&lt;p&gt;There is quite some research done by &lt;a href=&quot;https://www.youtube.com/watch?v=UVuziqAb4U8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hans-Georg Zimmermann&lt;/a&gt;, who has programmed Neural Networks for Siemens since some 20 years in order to predict Stock markets. He wrote some books on it, too, though I don't know if they are any good in English.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.siemens.com/innovation/en/home/pictures-of-the-future/digitalization-and-software/artificial-intelligence-overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; article gets to the point a bit faster than the video, I hope it helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: I think &lt;a href=&quot;https://www.youtube.com/watch?v=AOYo4xPI_BA&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; interview gives a good short introduction to his work.&lt;/p&gt;&#xA;" OwnerUserId="6269" LastEditorUserId="6269" LastEditDate="2017-10-13T09:02:38.120" LastActivityDate="2017-10-13T09:02:38.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4262" PostTypeId="2" ParentId="4257" CreationDate="2017-10-13T10:41:26.043" Score="0" Body="&lt;p&gt;As far as I know, there is still a huge debate about this topic. I would say, that the main rule for every self-driving car is to avoid a crash if possible.&lt;/p&gt;&#xA;&lt;p&gt;The question one should always ask is, in what situation would a crash really happen, and would a human react differently?&lt;/p&gt;&#xA;&lt;p&gt;My answer is no. The point is, a human might try to avoid the child (3) but it would be out of instinct rather than &amp;quot;consideration&amp;quot;. The driver might even harm others in this situation.&lt;/p&gt;&#xA;&lt;p&gt;Since a self-driving car will normally follow the rules a critical situation will most of the time arise due to the other person, not the car. So I believe it is best to protect the driver at all costs.&lt;/p&gt;&#xA;&lt;p&gt;The dog vs child problem could be solved via advanced animal recognition (human vs no human)&lt;/p&gt;&#xA;&lt;p&gt;Regarding no (2): Too small = no human ==&amp;gt; car will ignore it.&lt;/p&gt;&#xA;" OwnerUserId="10171" LastEditorUserId="2444" LastEditDate="2021-01-12T19:47:35.490" LastActivityDate="2021-01-12T19:47:35.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4264" PostTypeId="2" ParentId="4161" CreationDate="2017-10-13T12:33:09.867" Score="5" Body="&lt;p&gt;Yes, &lt;strong&gt;you can do research in Artificial Intelligence with low funds&lt;/strong&gt; (but you need &lt;em&gt;a lot of time!&lt;/em&gt;). Notably, because AI is not the same as applied &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt; (indeed running ML programs on big data requires a lot of computer power). For example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;knowledge representation and reasoning&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural language processing&lt;/a&gt; (both are subfields of AI) generally don't require a lot of computer power. Even when interested in machine learning (which is not the same as AI, just a subfield of it among others), you can use a powerful PC or laptop for that.&lt;/p&gt;&#xA;&lt;p&gt;A lot of recent papers (probably most of them) in journals like  &lt;a href=&quot;https://www.journals.elsevier.com/artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence&lt;/em&gt;&lt;/a&gt;, or in conferences like &lt;a href=&quot;http://www.ijcai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IJCAI&lt;/a&gt; (see their &lt;a href=&quot;https://www.ijcai.org/past_proceedings/&quot; rel=&quot;nofollow noreferrer&quot;&gt;past proceedings&lt;/a&gt;) are somehow theoretical, and when something is implemented, it runs on a laptop or desktop. Notice that both AI journal and IJCAI conferences are peer-reviewed with a very selective process.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Actually it is difficult and rare to find a research paper in AI mentioning that costly equipment was needed to do the research. Costly supercomputers used for research are generally not used by AI researchers (but by researchers in physics or bioinformatics), and AI researchers often don't even have access to such facilities.&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;For examples, recent IJCAI2016 papers such as &lt;a href=&quot;http://Coco:%20Runtime%20Reasoning%20About%20Conflicting%20Commitments&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Coco: Runtime Reasoning About Conflicting Commitments&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;https://www.ijcai.org/Proceedings/16/Papers/008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Interdependent Scheduling Games&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;https://www.ijcai.org/Proceedings/16/Papers/017.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Control of Fair Division&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;https://www.ijcai.org/Proceedings/16/Papers/033.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Verifying Pushdown Multi-Agent Systems against Strategy Logics&lt;/em&gt;&lt;/a&gt;, etc. don't mention any costly computation.&#xA;Actually, it is likely that most recent papers don't use and don't need large scale costly cloud computing. And some of that research might not have been implemented (perhaps by some intern) in anything more than a toy prototype.&lt;/p&gt;&#xA;&lt;br/&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Please include in your answer examples of recent research results and the required infrastructure that was used to create them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;br/&gt;&#xA;&lt;p&gt;&lt;sup&gt;Here are some recent research publications, they all explicitly mention the needed computer equipment, follow the link to read them:&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;Event recent IJCAI &lt;em&gt;experimental&lt;/em&gt; papers like &lt;a href=&quot;http://www.ijcai.org/Proceedings/15/Papers/039.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Multicore Tool for Constraint Solving&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.ijcai.org/Proceedings/15/Papers/053.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Compiling Constraint Networks&#xA;into Multivalued Decomposable Decision Graphs&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.ijcai.org/Proceedings/16/Papers/102.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;External Memory Bidirectional Search&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.ijcai.org/Proceedings/16/Papers/105.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Multiple Constraint Acquisition&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.ijcai.org/Proceedings/16/Papers/130.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Completion of Disjunctive Logic Programs&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.ijcai.org/Proceedings/16/Papers/164.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Eliminating Disjunctions in Answer Set Programming by Restricted Unfolding&lt;/em&gt;&lt;/a&gt; and &lt;a href=&quot;http://www.ijcai.org/Proceedings/15/Papers/058.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;On the Empirical Time Complexity of Random 3-SAT at the Phase Transition&lt;/em&gt;&lt;/a&gt; mention at most a multi-core workstation (e.g. at most a dual Xeon socket workstation,  often a cheaper laptop or desktop). I guess that most authors need their GPU only as a graphics card, to use their display screen. BTW, papers in other journals, such as &lt;a href=&quot;http://www.jair.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JAIR&lt;/a&gt; (like &lt;a href=&quot;http://www.jair.org/media/5312/live-5312-9919-jair.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Improving the Efficiency of Dynamic Programming on Tree&#xA;Decompositions via Machine Learning&lt;/em&gt;&lt;/a&gt;), when they mention computer equipment, gives similar kind of machines (laptop or at most high-end desktop).&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;(Actually, I don't remember having read an AI paper mentioning costly computing equipment; and I believe the reason for that is that in the academic community access to supercomputers is nearly reserved to research in other domains: physics, bioinformatics, etc. For an AI researcher gaining such access is difficult and uncommon; BTW, in &lt;a href=&quot;http://ec.europa.eu/programmes/horizon2020/&quot; rel=&quot;nofollow noreferrer&quot;&gt;H2020&lt;/a&gt; European research grants computing cost above 15% of the labor cost needs to be dully justified, so is &lt;em&gt;exceptional&lt;/em&gt;)&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, you'll better publish your software as &lt;a href=&quot;https://en.wikipedia.org/wiki/Free_software&quot; rel=&quot;nofollow noreferrer&quot;&gt;free software&lt;/a&gt; or open-source, and &lt;strong&gt;you need a &lt;em&gt;lot&lt;/em&gt; of time&lt;/strong&gt; (preferably full-time, or at least half-time) to do the research work, publish it, and follow outside progress in your area. BTW, contacting a nearby university could be helpful (you could attend some seminars, etc.)&lt;/p&gt;&#xA;&lt;p&gt;You can find interesting blogs (e.g. &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the one of J.Pitrat&lt;/a&gt;) of AI researchers working on just a laptop or a desktop.&lt;/p&gt;&#xA;&lt;p&gt;So, &lt;strong&gt;most research in AI&lt;/strong&gt; (even in Machine Learning, read papers from &lt;a href=&quot;http://www.jmlr.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JMLR&lt;/a&gt;) &lt;strong&gt;are done by researchers working on a laptop or desktop&lt;/strong&gt;. I am not even sure you need a powerful GPU to do the research. You certainly can do interesting research with a desktop computer (probably running Linux, with 32Gb RAM, some AMD Ryzen or Intel i5 or i7) costing in 2017 less than 1500€ or maybe 2500€ (and sometimes a low end laptop is enough). In rare cases (a small minority of papers), you might need a dual-socket workstation (something costing perhaps 5000€).&lt;/p&gt;&#xA;&lt;p&gt;Notice that &lt;a href=&quot;https://en.wikipedia.org/wiki/Big_data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Big Data&lt;/a&gt; is not considered as a field of Artificial Intelligence. I guess (but don't really know) that research in Big Data requires access to more computer power than a PC. BTW, I am not sure that &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; is really a research project, it is more an industrial demo.&lt;/p&gt;&#xA;&lt;p&gt;PS. See also &lt;a href=&quot;http://norvig.com/21-days.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://norvig.com/21-days.html&lt;/a&gt; (notice &lt;em&gt;who&lt;/em&gt; is the author!) and this &lt;a href=&quot;http://starynkevitch.net/Basile/bismon-chariot-doc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bismon draft report&lt;/a&gt; (developing programming techniques, in a GPLv3+ alpha-stage software &lt;a href=&quot;https://github.com/bstarynk/bismon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;prototype&lt;/a&gt;, notably &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;frame&lt;/a&gt;-based or &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic-network&lt;/a&gt; related ones, with &lt;a href=&quot;https://en.wikipedia.org/wiki/Reflection_(computer_programming)&quot; rel=&quot;nofollow noreferrer&quot;&gt;reflection&lt;/a&gt; &lt;a href=&quot;https://tel.archives-ouvertes.fr/tel-01331023/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;and&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Type_introspection&quot; rel=&quot;nofollow noreferrer&quot;&gt;introspection&lt;/a&gt;, reusable and relevant to AI), and the &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; project.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-12-20T07:28:29.527" LastActivityDate="2020-12-20T07:28:29.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4265" PostTypeId="2" ParentId="4257" CreationDate="2017-10-13T19:07:09.020" Score="1" Body="&lt;p&gt;I don't think these questions will need to be answered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A self driving car will almost certainly avoid a situation like the ones described well before a human would have and hence would not have to choose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example it would slow down as soon as it sees a child close to the road. It will identify and react to the fact that the child starts moving towards the road and act before the situation requires the &quot;drastic&quot; scenarios that we invent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for some reason the car has to choose, it can also make the impact/avoidance have the highest chance of not killing the occupant of the vehicle considering how well designed cars are these days and that the speeds involved shouldn't be anything wild like 200mph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not to mention a network of cars along with street cameras and sensors would act/work together to resolve as a swarm intelligence so the car ahead or the traffic camera can warn/tell other cars that something they cannot see is a potential hazard. I can go on and on...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, the bottom line is a self driving car will &lt;em&gt;not&lt;/em&gt; road rage, drive at dangerous speeds in a residential area, get tired and fall asleep, text and drive or drink and drive, etc... I cannot wait.&lt;/p&gt;&#xA;" OwnerUserId="9998" LastEditorUserId="75" LastEditDate="2017-10-13T22:53:44.617" LastActivityDate="2017-10-13T22:53:44.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4268" PostTypeId="2" ParentId="3956" CreationDate="2017-10-14T10:46:22.443" Score="3" Body="&lt;p&gt;First of all the answer to your question is largely dependent on the problem you are trying solve, the size of your population, the size of your problem's search space and the rest of your GA's hyper-parameters such as your mutation rate. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the problem has a large search space, then applying the elites strategy you described above will most likely cause your algorithm to never reach the optimal solution or reach it after too many iterations. The reason for this being that if your search space is big, then since the initialization of your initial population is random, the chances is that the &quot;fittest&quot; individual(s) in the first few iterations are not necessarily the gonna lead you to the optimal solution. Hence applying the elites strategy too early might make you get stuck on a local optima forever or for too long. The way I have personally found to be effective for this kind of situations is to allow your algorithm to explore as large a search space as possible earlier on(i.e use &quot;all the population present in the pool&quot;) and then gradually introduce the elite strategy you described above(this is an idea similar to momentum in neural networks). In short, you want to explore earlier on, and then when you gradually start exploiting what you have found. &lt;/li&gt;&#xA;&lt;li&gt;If the search space is relatively small, I think the elites strategy would be a very good idea. What you might also try to do, if you really want to stick to using elitism, you can try to mutate the elite individual(this kinda allows you to explore a little bit), or choose a higher elitism rate(i.e the number of individuals to be elite).  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now to answer you other questions: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any certain belief that cross-over on only elites of the population, converges the solutions faster?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, but I don't have reference for this so I guess I speak under correction, but I will explain why I said no:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with pure elitism is that it causes the genetic algorithm to converge to local maxima instead of the global maxima. Basically pure elitism is just a race to the nearest local maxima and and as soon as you get to the local maxima, if you continue with elitism, you get almost no improved from there. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I guess in order to escape from local minima, cross-over on all the population is needed; on the other hand I also say why performing cross over on weak population?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&quot;why perform cross-over on weaker individuals in the population?&quot; - to explore. The hope is that applying a cross-over or a mutation on a weaker individual will produce a fitter individual.&lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-10-14T10:53:13.113" LastActivityDate="2017-10-14T10:53:13.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4269" PostTypeId="2" ParentId="3860" CreationDate="2017-10-14T12:29:00.610" Score="1" Body="&lt;p&gt;Programming questions are a bit out of scope of this site, but it's a more generic one. For some reason the developers didn't specify it in code, but everything in &lt;code&gt;contrib&lt;/code&gt; is experimental. From &lt;a href=&quot;https://stackoverflow.com/q/44204897/712995&quot;&gt;this discussion&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The distinction between &lt;code&gt;core&lt;/code&gt; and &lt;code&gt;contrib&lt;/code&gt; is really in core things don't change. Things are backward compatible until release 2.0, and nobody's thinking about that right now.&lt;/p&gt;&#xA;&lt;p&gt;If you have something in core, it's stable, you should use it. If you have something in &lt;code&gt;contrib&lt;/code&gt;, the API may change and depending on your needs you may or may not want to use it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When a particular class or function is only in &lt;code&gt;contrib&lt;/code&gt;, there's basically no choice. But for classes that have &amp;quot;graduated&amp;quot; to the core, &lt;code&gt;contrib&lt;/code&gt; version gets deprecated automatically. So you want to use &lt;code&gt;tf.estimator.DNNClassifier&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-12-18T11:41:04.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4270" PostTypeId="2" ParentId="4219" CreationDate="2017-10-14T14:38:03.280" Score="4" Body="&lt;p&gt;I am not aware of any open-source products that can readily replace IBM Watson, but the following projects could be of interest to you in that regard: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://deepdive.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DARPA DeepDive&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://hazyresearch.github.io/snorkel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Snorkel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://uima.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apache UIMA&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/brmson/yodaqa&quot; rel=&quot;nofollow noreferrer&quot;&gt;YodaQA&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://opencog.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://oaqa.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OAQA (Open Advancement of Question Answering Systems)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stanfordnlp.github.io/CoreNLP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford CoreNLP – Natural language software&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://aksw.org/Projects/openQA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;openQA&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7550" LastActivityDate="2017-10-14T14:38:03.280" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4271" PostTypeId="1" CreationDate="2017-10-14T16:25:09.157" Score="3" ViewCount="171" Body="&lt;p&gt;I want to use deep learning to estimate the value of a function based on some data. However, the loss function would be neither convex nor concave. Can I know if it is a big deal in deep learning? Is training a deep network, when loss function is convex, the same as optimizing a convex problem or not? I would be thankful if any paper has addressed this issue. &lt;/p&gt;&#xA;" OwnerUserId="10191" LastActivityDate="2018-06-01T23:20:55.497" Title="Non-Convex Loss Function in Deep Learning Is a Big Deal?" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4273" PostTypeId="1" CreationDate="2017-10-14T22:41:18.833" Score="1" ViewCount="1187" Body="&lt;p&gt;If you walk your dog, he'll be happy. If you don't walk him, he'll chew up your furniture. If he's happy or if he chews your furniture, he'll mess up your apartment. If he messes up your apartment, you'll be annoyed. &#xA;Does your dog mess up your apartment?&#xA;Are you annoyed at him?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write the knowledge base given above in propositional logic.&lt;/li&gt;&#xA;&lt;li&gt;Convert the knowledge base into conjunctive normal form.&lt;/li&gt;&#xA;&lt;li&gt;Use resolution to prove by contradiction that your dog made a mess. Connect the two clauses to a new clause that the can be collapsed into (see the example figure).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Example figure: &lt;a href=&quot;https://i.stack.imgur.com/l2LGL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l2LGL.png&quot; alt=&quot;Example figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10195" LastActivityDate="2018-12-04T18:39:34.637" Title="Writing a knowledge base (KB) in propositional logic and converting the KB into conjunctive normal form" Tags="&lt;logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4274" PostTypeId="2" ParentId="1525" CreationDate="2017-10-15T03:19:55.693" Score="3" Body="&lt;p&gt;Indeed you're intuition is correct, a Boltzmann machine is able to hold more than a Hopfield network in its memory because of its stochastic nature &lt;a href=&quot;http://ieeexplore.ieee.org/document/487813/&quot; rel=&quot;nofollow noreferrer&quot;&gt;as explored in this paper.&lt;/a&gt; In the paper they note that the capacity is around 0.6. After this ratio it starts to break down and adds much more noise to the recalled patterns.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-10-15T03:19:55.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4275" PostTypeId="2" ParentId="4257" CreationDate="2017-10-15T07:35:47.547" Score="0" Body="&lt;p&gt;As others have said, your question is, and will continue to be a hot topic.&#xA;I also agree that eventually self-driving cars will be able to handle your&#xA;hypothetical situations better than many human drivers. I am not prepared to say &#xA;when that &quot;eventually&quot; will eventuate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I can also imagine some human drivers deliberately trying to cause &#xA;self-driven cars to make poor decisions. For example, a &quot;team&quot; of three or more&#xA;cars could easily confound a self-driven car's programming by co-ordinating &#xA;their actions, especially once the actual program code used by the car is known.&#xA;I'm thinking of situations where the self-driven car is boxed in by human-driven &#xA;cars which indicate they are about to make a move and then do not, while others&#xA;change speed and direction at the same time, or at slightly different times.&#xA;Humans can be incredibly sneaky and unethical, and some are very good at &#xA;finding exploits and weaknesses.&lt;/p&gt;&#xA;" OwnerUserId="10198" LastActivityDate="2017-10-15T07:35:47.547" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4276" PostTypeId="2" ParentId="3106" CreationDate="2017-10-15T08:48:13.653" Score="1" Body="&lt;p&gt;most of the underlying math(like back-propergation)  was discovered a long time ago, and the advances in hardware have only recently made it possible to tackle some problems within  a reasonable amount of time but that is not the only reason. Some of the other things that also made deep learning pick up includes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The availability of large,  labeled, quality data - deep learning generally requires a lots of data to  train successfully. This kind of data just wasn't there until recently. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Algorithmic improvements(I.e further improvements on the maths) - better weight initialization techniques, the discovery of the ReLu as a replacement for the sigmoid and tach activation which goes a long way towards solving the vanishing gradient problem,  the discovery of neutral network dropout which is found to reduce overfitting, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The discovery of better neutral network architectures.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, whist the computing power is certainly a big part of it, it certainly isn't the only factor. &lt;/p&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-10-15T09:07:59.047" LastActivityDate="2017-10-15T09:07:59.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4278" PostTypeId="2" ParentId="3224" CreationDate="2017-10-16T08:03:34.363" Score="1" Body="&lt;p&gt;I think you look for the meaning of the image (pic of 'a' = a) rather than the regular image similarity. For example, you would like the two images of the 'a' you posted to be classified to the same label instead of, for example, to be classified as similar to an 'o' image which would give a smaller Hamming distance or mean square error.&lt;br&gt;&#xA;So as &lt;a href=&quot;https://ai.stackexchange.com/users/4398/jaden-travnik&quot;&gt;@Jaden Travnik&lt;/a&gt; suggested, &lt;a href=&quot;https://www.kaggle.com/c/digit-recognizer/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST digits classification&lt;/a&gt; is a good start. It classifies handwritten digits and has human-level solutions implemented by any existing CNN framework. (TensorFlow, Caffe, Theano, to name a few).&lt;/p&gt;&#xA;" OwnerUserId="9233" LastActivityDate="2017-10-16T08:03:34.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4279" PostTypeId="1" AcceptedAnswerId="4293" CreationDate="2017-10-16T10:29:30.813" Score="7" ViewCount="221" Body="&lt;p&gt;I'm looking to design a neural network that can predict which runner wins in a sports game, where the amount of runners varies between 2-10. In each case, specific data about the individual runners would be fed into the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What design would be most advantageous for such a neural network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially this is a ranking problem where the amount of inputs and outputs are variable.&lt;/p&gt;&#xA;" OwnerUserId="4036" LastEditorUserId="1671" LastEditDate="2017-10-18T20:17:41.033" LastActivityDate="2017-10-18T20:17:41.033" Title="Neural network design when amount of input neurons vary" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4280" PostTypeId="1" CreationDate="2017-10-16T12:33:16.407" Score="1" ViewCount="37" Body="&lt;p&gt;Is there any existing software which analyzes idea flow in the text (or between different texts, e.g scientific articles) and for example visualizes it with some graph?&lt;/p&gt;&#xA;" OwnerUserId="10211" LastActivityDate="2017-10-16T12:33:16.407" Title="Any software to guess idea flow (basic ideas/statements and their derivations) throughout text?" Tags="&lt;natural-language-processing&gt;&lt;text-summarization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4281" PostTypeId="2" ParentId="2405" CreationDate="2017-10-16T15:18:47.693" Score="2" Body="&lt;p&gt;The cross-entropy loss will always be positive because the probability is in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$-ln(p)$&lt;/span&gt; will always be positive.&lt;/p&gt;&#xA;" OwnerUserId="10215" LastEditorUserId="2444" LastEditDate="2020-11-01T22:39:38.373" LastActivityDate="2020-11-01T22:39:38.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4282" PostTypeId="1" CreationDate="2017-10-16T18:10:03.370" Score="12" ViewCount="17672" Body="&lt;p&gt;I have a large dataset of vehicles with the ground truth of their lengths (Over 100k samples). Is it possible to train a deep network to measure/estimate vehicle length ? I haven't seen any papers related to estimating object size using deep neural network.&lt;/p&gt;&#xA;" OwnerUserId="10218" LastActivityDate="2019-01-18T19:58:23.217" Title="Measuring Object size using Deep Neural Network" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4283" PostTypeId="1" AcceptedAnswerId="4837" CreationDate="2017-10-16T20:28:28.607" Score="7" ViewCount="3091" Body="&lt;p&gt;I'm working on a image classification problem using neural-network. In the training data set, 90% of the samples fall into 10% of all categories, while 10% of the sample fall into the other 90% categories. So example is not evenly distributed among all categories. If we assume this distribution reflects the real world distribution, do I need to filter my dataset before training so that each category has similar number of samples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="10221" LastActivityDate="2017-12-24T17:32:11.583" Title="Does data skew matter in classification problem?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4285" PostTypeId="2" ParentId="4282" CreationDate="2017-10-16T23:28:41.057" Score="5" Body="&lt;p&gt;Yes! This most certainly can be done. Since you have a labeled dataset, that makes it all the more simple!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would take a look at &lt;a href=&quot;https://www.pyimagesearch.com/2016/03/28/measuring-size-of-objects-in-an-image-with-opencv/&quot; rel=&quot;noreferrer&quot;&gt;this project&lt;/a&gt; and that should get you where you need to go. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation details should be pretty straightforward. Let me know if I can help further.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-10-16T23:28:41.057" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4286" PostTypeId="1" CreationDate="2017-10-17T08:46:36.187" Score="2" ViewCount="230" Body="&lt;p&gt;Is Deep Learning the repeated application of Linear Regression?&lt;/p&gt;&#xA;" OwnerUserId="10229" LastEditorUserId="2444" LastEditDate="2020-12-07T21:45:57.847" LastActivityDate="2020-12-08T20:19:11.627" Title="Is Deep Learning the repeated application of Linear Regression?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;linear-regression&gt;&lt;non-linear-regression&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4290" PostTypeId="2" ParentId="4286" CreationDate="2017-10-18T01:10:10.010" Score="2" Body="&lt;p&gt;There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear activation units.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-10-18T01:10:10.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4292" PostTypeId="2" ParentId="4236" CreationDate="2017-10-18T09:48:21.243" Score="0" Body="&lt;p&gt;This issue was resolved by normalizing the vectors.&lt;/p&gt;&#xA;" OwnerUserId="10092" LastActivityDate="2017-10-18T09:48:21.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4293" PostTypeId="2" ParentId="4279" CreationDate="2017-10-18T20:00:50.623" Score="2" Body="&lt;p&gt;The best option in your case would probably be zero padding or padding up. This is simply zeroing out inputs for cases in which there is no data. It's done a lot on the borders of images for CNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That or you could just use a RNN which can handle your variable length inputs with ease.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-10-18T20:00:50.623" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4294" PostTypeId="1" CreationDate="2017-10-19T05:11:43.337" Score="4" ViewCount="236" Body="&lt;p&gt;I’ve been experimenting on several datasets and found something very strange while implementing ML.&lt;br&gt;&#xA;I’ll Explain after the code…&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from sklearn import datasets&#xA;iris = datasets.load_iris()&#xA;&#xA;# 4 features in np array - 150 rows&#xA;&#xA;case = 1        # change cases to see variation&#xA;&#xA;if case == 1:   # first feature deleted&#xA;    iris.data = np.delete(iris.data,0, 1)&#xA;&#xA;if case == 2:   # first 2 features deleted&#xA;    iris.data = np.delete(iris.data,0, 1)&#xA;    iris.data = np.delete(iris.data,0, 1)&#xA;&#xA;if case == 3:   # first 3 features deleted (1 feature left)&#xA;    iris.data = np.delete(iris.data,0, 1)&#xA;    iris.data = np.delete(iris.data,0, 1)&#xA;    iris.data = np.delete(iris.data,0, 1)&#xA;&#xA;if case == 4:   # only second feature deleted from np array&#xA;    iris.data = np.delete(iris.data,1, 1)&#xA;&#xA;if case == 5:   # only third feature deleted from np array&#xA;    iris.data = np.delete(iris.data,2, 1)&#xA;&#xA;if case == 6:   # only last feature deleted from np array&#xA;    iris.data = np.delete(iris.data,3, 1)&#xA;&#xA;# print iris.data&#xA;# exit()&#xA;&#xA;from sklearn.naive_bayes import GaussianNB&#xA;gnb = GaussianNB()&#xA;pred = gnb.fit(iris.data, iris.target).predict(iris.data)&#xA;# pred = gnb.fit(iris.data, iris.target).predict(test_data)&#xA;&#xA;from sklearn.metrics import accuracy_score&#xA;print accuracy_score(iris.target, pred)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I’m using the basic fisher iris dataset from sklearn, it has 150 rows and 4 columns (features).&lt;br&gt;&#xA;Using training data as test data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i tried to remove a few features and see if accuracy changed.&#xA;And I thought it would.&lt;br&gt;&#xA;But till case &lt;strong&gt;1&lt;/strong&gt;, &lt;strong&gt;2&lt;/strong&gt;, and &lt;strong&gt;3&lt;/strong&gt;, I removed 1, 2 and 3 features respectively and there was no change in the accuracy. It stayed 96%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then on running cases &lt;strong&gt;4&lt;/strong&gt;,&lt;strong&gt;5&lt;/strong&gt; and &lt;strong&gt;6&lt;/strong&gt;, The accuracy changed. Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On comparing case &lt;strong&gt;2&lt;/strong&gt; and &lt;strong&gt;4&lt;/strong&gt;,&lt;br&gt;&#xA;Both have second feature removed from the dataset, so clearly, removing second feature is responsible for change in accuracy (as seen in case &lt;strong&gt;4&lt;/strong&gt;)&lt;br&gt;&#xA;So why does it not change in case &lt;strong&gt;2&lt;/strong&gt;?&lt;br&gt;&#xA;Just because it had first feature removed too? to balance out the second? (If that were true, case &lt;strong&gt;1&lt;/strong&gt; would have given different accuracy)&lt;br&gt;&#xA;Why does accuracy not change in first 3 cases, but it does in the last 3 cases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is ML dependent on the order in which features are feeded to the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would be great if someone could clear this doubt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10258" LastActivityDate="2017-10-19T09:05:02.993" Title="Is accuracy of a machine learning algorithm independent of the features?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4295" PostTypeId="2" ParentId="4294" CreationDate="2017-10-19T09:05:02.993" Score="2" Body="&lt;p&gt;This is a result of overfitting, I think. Your model is allowed to explain the phenomenon using any combination of variables, whether or not such a  combination constitutes a true signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with the details of a NBC, but I suspect that the model has some bias/offset/constant variable, that can be combined with your input features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thusly, what you see is just an expression that  the classifier has been able to combine each set of features and the constant variables in such a way that they perfectly describe the phenomenon. These combinations will be non-linear, so the logic as to how dependent the model is on any parameter is not a simple &quot;it follows&quot;-reasoning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you evaluate OUT of sample, you will see a much better reflection of how useful individual features are, and how bad the performance becomes when you remove the first three features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, do a K-folds test (say, K=10) of the six different models, preserving 20% of the dataset as test-data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you would likely see the following rankings:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(4, 5, 6, 1 ~ equal) &gt; 2 &gt; 3 &lt;/p&gt;&#xA;" OwnerUserId="4747" LastActivityDate="2017-10-19T09:05:02.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4296" PostTypeId="1" AcceptedAnswerId="4314" CreationDate="2017-10-19T10:47:53.943" Score="1" ViewCount="207" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1706.01905.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parameter Space Noise for Exploration&lt;/a&gt;, the authors describe the noise that they add to the parameter vector as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; simply the identity matrix, or am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="10261" LastEditorUserId="2444" LastEditDate="2019-04-01T13:52:40.123" LastActivityDate="2019-04-01T13:53:32.710" Title="What is $I$ in the noise described in the paper &quot;Parameter Space Noise for Exploration&quot;?" Tags="&lt;reinforcement-learning&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4297" PostTypeId="1" AcceptedAnswerId="4316" CreationDate="2017-10-19T12:39:44.217" Score="1" ViewCount="251" Body="&lt;p&gt;Without using any of Matlab's neural network tools, I'm writing a program to simulate an OR gate with a perceptron. I have seen many tutorials, but I still can't understand why we need weights to train a perceptron for such a simple purpose.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way is to program the perceptron with the conditions &lt;code&gt;(0,0)=0. (1,0)=1. (0,1)=1. (1,1)=1&lt;/code&gt;. So the two inputs to the perceptron would be either zeroes or ones. I don't see the purpose of weights here. Assuming weights are 1, for the second training example, the output would be &lt;code&gt;1*1 + 0*1 = 1&lt;/code&gt;. For the last example, it would be &lt;code&gt;1*1 + 1*1 = 2&lt;/code&gt;. So an activation function which says &lt;code&gt;if output &amp;gt;= 1, output = 1 else output =0; end&lt;/code&gt; should suffice. This would successfully simulate an OR gate. So why do I need to &quot;train&quot; any weights?    &lt;/p&gt;&#xA;" OwnerUserId="9268" LastEditorUserId="9268" LastEditDate="2017-10-19T13:38:13.523" LastActivityDate="2017-10-20T21:29:03.757" Title="Why do we need weights when training a perceptron as an OR gate?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4298" PostTypeId="1" CreationDate="2017-10-19T14:03:09.607" Score="6" ViewCount="587" Body="&lt;p&gt;In the hill climbing algorithm, the greater value, compared to the current value, is selected, but I cannot understand why it takes the larger value instead of the smaller one. Why is that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I greatly appreciate the inclusion of figures in your answers.&lt;/p&gt;&#xA;" OwnerUserId="10265" LastEditorUserId="2444" LastEditDate="2019-03-02T10:54:30.007" LastActivityDate="2019-03-02T10:54:30.007" Title="Why is the larger value, as opposed to the smaller one, chosen, in the hill climbing algorithm?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;hill-climbing&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4301" PostTypeId="1" AcceptedAnswerId="4302" CreationDate="2017-10-19T21:39:04.103" Score="1" ViewCount="67" Body="&lt;p&gt;In my research on video games path finding I'm using ant colony optimization, not only to find the shortest path, but also to add some unpredictability and adaptiveness to bots path finding. It works the way as players move in the map, they add some pheromone to the map, so it adds up a probability that bots choose path like players. I have sent the paper, but judges said you need a benchmark and distinguish to previous works. Can you tell me how can I benchmark this work?&lt;/p&gt;&#xA;" OwnerUserId="10273" LastEditorUserId="2444" LastEditDate="2019-05-27T22:04:33.227" LastActivityDate="2019-10-16T02:16:41.773" Title="How can I benchmark an application of ant colony optimization?" Tags="&lt;game-ai&gt;&lt;applications&gt;&lt;swarm-intelligence&gt;&lt;ant-colony-optimization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4302" PostTypeId="2" ParentId="4301" CreationDate="2017-10-19T23:03:29.290" Score="1" Body="&lt;p&gt;I would assume that in this case, they are asking you to compare the performance of your particular algorithm to other similar algorithms used in pathfinding. For example, A*&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-10-19T23:03:29.290" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4305" PostTypeId="2" ParentId="182" CreationDate="2017-10-20T02:11:25.050" Score="2" Body="&lt;p&gt;You can take a look at bayesian hyperparameter optimization as a general method of optimizing loss (or anything) as a function of the hyperparameters. But note that in general the deeper your network the better, so optimizing loss as a function of number of layers isn't a very fun thing to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grid search and a bit of common sense (as learnt by seeing many examples) should be your best bet.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-10-20T02:11:25.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4306" PostTypeId="2" ParentId="4298" CreationDate="2017-10-20T09:55:01.750" Score="3" Body="&lt;p&gt;When we climb a hill:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/msB7D.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/msB7D.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We move higher in &lt;a href=&quot;https://en.wikipedia.org/wiki/Altitude&quot; rel=&quot;nofollow noreferrer&quot;&gt;altitude&lt;/a&gt;. The person who is climbing, will always look for rocks/mud on the hill that are higher, so that he can climb higher.&lt;br&gt;&#xA;That is what the algorithm does too. We are assuming that there is a hill of numbers. The larger numbers are placed higher than the smaller numbers. So if we want to climb up the hill, we search for the larger numbers.&lt;br&gt;&#xA;Comparing this with the real world hill climbing, simply assume that every rock on the hill has a number written on it. The number will be the altitude of the rock, from sea level height. So if you want to climb up, you just have to search for the rock with a higher number than the rock you are standing on now.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u1leZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u1leZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9268" LastActivityDate="2017-10-20T09:55:01.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4307" PostTypeId="1" CreationDate="2017-10-20T11:10:19.403" Score="0" ViewCount="41" Body="&lt;p&gt;I am new to this field and would like to know that for what kind of data types other than images, recommendation system can be created using machine learning. Suppose for contents like audio or video, is it necessary to use data set of actual audio files or video files or just text information about the file is enough for this system ? &lt;/p&gt;&#xA;" OwnerUserId="10279" LastEditorUserId="16565" LastEditDate="2019-04-04T20:38:06.403" LastActivityDate="2019-04-04T20:38:06.403" Title="Recommendation system based on content type" Tags="&lt;datasets&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4308" PostTypeId="2" ParentId="4307" CreationDate="2017-10-20T11:51:21.233" Score="3" Body="&lt;p&gt;Recommendation systems can be applied for anything, as long as you have sufficient training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important inputs to the recommendation system are not &quot;audio files or video files&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia gives you the list:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Similarity matrix&lt;/li&gt;&#xA;&lt;li&gt;Item attributes&lt;/li&gt;&#xA;&lt;li&gt;User activities and behaviours&lt;/li&gt;&#xA;&lt;li&gt;User profile&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Recommender_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Recommender_system&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2017-10-20T13:51:30.637" LastActivityDate="2017-10-20T13:51:30.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4309" PostTypeId="1" CreationDate="2017-10-20T15:25:36.953" Score="1" ViewCount="21" Body="&lt;p&gt;I've been tinkering with an artificial life simulator, critterding for many years and lately ive been branching the project and adding fitness functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im still quiet new to genetic algorithms and currently critters in the simulation have very complex and different sized dna files which i want to be able to cross, currently they can only reproduce asexually and evolve purely through mutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, thier genome consists of:&#xA;Box part dimensions&#xA;Joint locations and angle constraints which connect those boxes&#xA;And a list of neurons, thier properties and synaptic connections&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of boxes and thier joints can change by mutation as well as the number of neurons and synaptic connections and so im having a hard time trying to think up a crossover approach which can handle these variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, can anyone point me in the right direction for research material or give me insight into handling such dna structures for crossing? &lt;/p&gt;&#xA;" OwnerUserId="10282" LastActivityDate="2017-10-20T15:25:36.953" Title="Crossing complex and non uniform genomes?" Tags="&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4313" PostTypeId="2" ParentId="4297" CreationDate="2017-10-20T21:00:51.073" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This would successfully simulate an OR gate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Of course. In fact, hardware implementation of an OR gate needs &lt;a href=&quot;https://en.wikipedia.org/wiki/OR_gate#Implementations&quot; rel=&quot;nofollow noreferrer&quot;&gt;just a few transistors&lt;/a&gt;. It may sound surprising, but the best python implementation of OR is &lt;code&gt;a or b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So why do I need to &quot;train&quot; any weights?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's one of the simplest functions out there, everyone understands how OR works just from the boolean table. The task shows that a neural network can &lt;em&gt;learn&lt;/em&gt; it as well, just from examples and without specification what a logical operation is and how to wire it.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2017-10-20T21:00:51.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4314" PostTypeId="2" ParentId="4296" CreationDate="2017-10-20T21:26:22.707" Score="1" Body="&lt;p&gt;Yes, since &lt;span class=&quot;math-container&quot;&gt;$\tilde{\theta}$&lt;/span&gt; is a vector, to define its distribution one needs a &lt;a href=&quot;https://en.wikipedia.org/wiki/Covariance_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;covariance matrix&lt;/a&gt;. Here &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is the identity matrix, which means that the noise has a zero-mean normal distribution with standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;, and different components of this noise are uncorrelated.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorUserId="2444" LastEditDate="2019-04-01T13:53:32.710" LastActivityDate="2019-04-01T13:53:32.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4315" PostTypeId="2" ParentId="4257" CreationDate="2017-10-20T21:26:57.540" Score="1" Body="&lt;p&gt;The core issue with this question rests in probability.  Specifically:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What if a kid is crossing the street and avoiding it would kill the driver?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How does the AI know for certain that avoiding it would &lt;em&gt;kill&lt;/em&gt; the driver?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and certainty rears its head re:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) If a dog is crossing the road, I'd expect the car to try to avoid it. But what if this leads to .00001% more risk for the driver? What is the 'risk cut-off'?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There would likely be no &quot;hard cutoff&quot;.  Earlier fuzzy logic systems have been implemented in automotive gear shifting and anti-lock breaking, but it is precisely the &quot;fuzziness&quot; that made them effective.  Contemporary AI is far more sophisticated, and part of that sophistication rests in what might be though of as dynamic thresholds for decision-making.  Because certainty only exists in special, limited cases (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved games&lt;/a&gt;), estimation must be used. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Regarding the cockroach, it would likely be too small to warrant a response, although a swarm of cicadas might affect the car's sensing ability and prompt poor-visibility navigational protocols.  In general I'm sure pet-sized animals and bigger would be avoided, in the case of actual pets for humanitarian reasons, and for animals like deer, for reasons of driver risk (impaled by the horns at the worst, and at the least potentially costly damage to the vehicle.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I suspect the protocols for this would be breaking or swerving if there is a clear margin on either side of the animal (i.e. not a barrier, wall or cliff) and the direction change is controllable (i.e. hitting the animal is likely to result in less harm than an actual crash, and certainly less risk to the human, except in the case of the deer's horns.) &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-10-20T21:26:57.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4316" PostTypeId="2" ParentId="4297" CreationDate="2017-10-20T21:29:03.757" Score="1" Body="&lt;p&gt;There are two aspects that you need to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Because it is part of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron#Definition&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of a perceptron&lt;/a&gt;. There are many different mechanisms to implement an OR gate, most of them rather easy. But if you want to implement it based on a perceptron, you have to follow the definition of the perceptron. Otherwise you would just implement an OR gate without using an actual perceptron.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The purpose of an neural network is to initialize it more or less randomly without knowing the correct weights and biases for each neuron (or perceptron in your case). You just know the desired behavior of the network and you train it accordingly. In your example you know the desired output of the OR gate and can train your network to give you this desired output. What's bothering you is the fact that you can easily derive the correct weights without actual training the network just by analyzing it. That's because this is a theoretical example to show that a perceptron is capable to learn the behavior of an OR gate. In a real application you would not benefit from artificially crafting an OR gate based on an perceptron.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-10-20T21:29:03.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4317" PostTypeId="2" ParentId="4282" CreationDate="2017-10-20T21:49:20.470" Score="1" Body="&lt;p&gt;Yes it's possible, but first you'll have to recognize &lt;em&gt;some&lt;/em&gt; object in the image, either 1) the vehicle itself, and then report that vehicle's known size, or 2) a known object that's the same distance from the camera as the car (a curb, a stop sign, the driver's head, a shetland pony... whatever), and then use that object to calibrate the size of the car that's very close to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any car in an image will be an unknown distance from the camera, making the car object appear larger or smaller from photo to photo.  If you don't recognize the car or at least a referent object that has a known size, the physical size of the car will be uncalibrated -- you'll have no basis for your size estimate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the car is unknown, then even if you do have visual clues (there is a referent object present or the distance from camera to car is known), the unknown extent of wide angle-ness of the camera's lens may distort an unknown car's shape (height vs width), further complicating your ability to estimate its apparent dimensions.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-10-20T21:49:20.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4318" PostTypeId="2" ParentId="1296" CreationDate="2017-10-21T01:27:47.477" Score="6" Body="&lt;p&gt;Novelty search selects for &quot;novel behavior&quot;, by some domain-dependent definition of novelty. For example, novelty in a Maze-solving domain might be &quot;difference of route explored&quot;. Eventually, networks that take every possible route through the maze will be found, and you could then select the fastest. This would work far better than a naive &quot;objective&quot;, like distance to the goal, which could easily result in a local optima which never solves the maze.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://eplex.cs.ucf.edu/papers/lehman_ecj11.pdf&quot; rel=&quot;noreferrer&quot;&gt;Abandoning Objectives: Evolution through the&#xA;Search for Novelty Alone&lt;/a&gt; (emphasis mine):  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In novelty search, instead of measuring overall progress with a traditional objective function, evolution employs a measure of behavioral novelty called a novelty metric. In effect, a search guided by such a metric performs explicitly what natural evolution does passively, i.e. gradually accumulating novel forms that ascend the complexity ladder.&lt;br&gt;&#xA;  For example, in a biped locomotion domain, initial attempts might simply fall&#xA;  down. The novelty metric would reward simply falling down in a different way,&#xA;  regardless of whether it is closer to the objective behavior or not. In contrast, an objective function may explicitly reward falling the farthest, which likely does not lead to the ultimate objective of walking and thus exemplifies a deceptive local optimum. In contrast, in the search for novelty, a set of instances are maintained that represent&#xA;  the most novel discoveries. Further search then jumps off from these representative behaviors. &lt;strong&gt;After a few ways to fall are discovered, the only way to be rewarded is to find a behavior that does not fall right away&lt;/strong&gt;. In this way, behavioral complexity rises from the bottom up. &lt;strong&gt;Eventually, to do something new, the biped would have to successfully walk for some distance even though it is not an objective&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9484" LastActivityDate="2017-10-21T01:27:47.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4319" PostTypeId="2" ParentId="2742" CreationDate="2017-10-21T04:18:48.440" Score="4" Body="&lt;p&gt;&lt;strong&gt;Brief Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error metric (an appropriate term used in the question title) quantifies the fitness of a linear or nonlinear model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It aggregates individual errors across a set of observations (instances of training data).  In typical use, an error function is applied to the difference between the dependent variable vector predicted by the model and empirical observations.  These differences are calculated for each observation and then summed. &lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why Distance Squared?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Legendre, who first published the sum of squares method for gauging fitness of the model (Paris 1705) stated correctly that squaring before summing is convenient.  Why did he write that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could use the absolute value of the error or the absolute value of its cube, but the discontinuity of the derivative of the absolute value makes the function NOT smooth.  Functions that are NOT smooth create unnecessary difficulties when employing linear algebra to derive closed forms (simple algebraic expressions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Closed forms are convenient when one wants to quickly and easily calculate slope and intercept in linear regression. &lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Gradient Descent&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gradient descent is generally employed for nonlinear regression.  Lacking the ability to create closed forms for many nonlinear models, iteration becomes a dominant methodology for validating or tuning the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An intuitive understanding of gradient descent can be gained by considering a thirsty, blind person looking for water on land solely by taking calculated steps.  (In software, these steps are iterations.)  The blind person can only sense the direction of the altitude gradient (direction of slope) with their feet to descend to a local minimum altitude. &lt;sup&gt;3&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone stating that, &quot;The function itself does not matter,&quot; in relation to the usual applications of gradient descent would be a dangerous choice for guide in a blind hiking expedition.  For instance, the reciprocal of distance as an error function would likely lead to the dehydration and death of the hikers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The selection criteria for error metrics is important if one is interested in the speed of convergence on a solution or whether the solution will ever be found. &lt;sup&gt;4&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the gradient of a plane (linear surface) is a constant, the use of gradient descent for linear models is wasteful.  The blind person need not continue to sample the angle of their foot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sign of the Error Metric&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statement, &quot;The result should be positive so positive and negative deviations are still counted,&quot; is incorrect. &lt;sup&gt;5&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Effectiveness of Error Metrics in Relation to 1.0&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of the partial derivative of the least squares error metric with respect to an error at any given point is constant, the least squares error metric converges similarly above and below 1.0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] The dimensions of a model's independent and dependent variable vectors are, in machine learning, conventionally called features and labels respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Another smooth function, such as the error to the fourth power would also lead to closed forms for slope and intercept, although they would produce slightly different results if the correlation coefficient is non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Gradient descent algorithms in general do not guarantee finding a global minimum.  In the example given, it would be quote possible to miss a small hole exists with water in it.  Depending on the surface features (terrain), sensing the angle of the foot (determining gradient) can be counterproductive. The search can becomes chaotic.  To extend the intuitive analogy, consider searching for the bottom of the stairs in Escher's Relativity lithograph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] For an error metric to be likely to converge and therefore useful in regression regardless of the direction of the error the sign of the metric is irrelevant.  It is each of the set of partial derivatives of the error metric with respect to the corresponding set of distances between the model predictions and observations that should be positive to regress omnidirectionally.  It sounds more complicated, but even this corrected statement is an oversimplification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] The error metric in gradient descent applications is often calculated using a convex function to avoid overshoot and possible oscillation and non-convergence.  In some cases error functions other that sum of squares is used.  The choice of function has to do with a number of factors:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The model to which the data is to be fit&lt;/li&gt;&#xA;&lt;li&gt;Factors expected to affect or actually affecting deviations of&#xA;the observations (training data) from the model&lt;/li&gt;&#xA;&lt;li&gt;Computational resources relative to the size of the data set&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2017-10-25T21:34:24.573" LastActivityDate="2017-10-25T21:34:24.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4320" PostTypeId="1" AcceptedAnswerId="4341" CreationDate="2017-10-21T06:52:15.723" Score="13" ViewCount="1652" Body="&lt;p&gt;This might sound silly to someone who has plenty of experience with neural networks but it bothers me...&lt;/p&gt;&#xA;&lt;p&gt;Random initial weights might give you better results that would be somewhat closer to what a trained neural network should look like, but it might as well be the exact opposite of what it should be, while 0.5, or some other average for the range of reasonable weights' values, would sound like a good default setting.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why are the initial weights of neural networks randomly initialized rather than being all set to, for example, 0.5?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="10294" LastEditorUserId="2444" LastEditDate="2020-12-20T19:20:55.553" LastActivityDate="2021-01-22T00:58:37.820" Title="Why are the initial weights of neural networks randomly initialised?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4321" PostTypeId="1" AcceptedAnswerId="4325" CreationDate="2017-10-21T07:20:18.113" Score="1" ViewCount="484" Body="&lt;p&gt;To classify images we are using the tensorflow incection v3 NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a similar approach to classify sounds? As for example to be able to recognize the person who is talking or classify a general sound? &lt;/p&gt;&#xA;" OwnerUserId="8546" LastActivityDate="2019-09-11T22:58:57.500" Title="Is there a way to classify sounds" Tags="&lt;classification&gt;&lt;tensorflow&gt;&lt;voice-recognition&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4322" PostTypeId="1" AcceptedAnswerId="4324" CreationDate="2017-10-21T07:39:59.093" Score="2" ViewCount="257" Body="&lt;p&gt;I understand that, in &lt;a href=&quot;http://old.sztaki.hu/%7Eszcsaba/papers/RLAlgsInMDPs-lecture.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning algorithms&lt;/a&gt;, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning&lt;/a&gt;, to prevent selecting the actions with greatest q-values too fast and allow for exploration, we use eligibility traces.&lt;/p&gt;&#xA;&lt;p&gt;Here are some questions&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy solve the same problem?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do these two approaches aim to attain the same objective?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the advantages of each over the other?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3017" LastEditorUserId="2444" LastEditDate="2020-12-04T18:06:31.270" LastActivityDate="2020-12-04T18:08:59.403" Title="Does eligibility traces and epsilon-greedy do the same task in different ways?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;epsilon-greedy-policy&gt;&lt;eligibility-traces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4324" PostTypeId="2" ParentId="4322" CreationDate="2017-10-21T19:10:35.343" Score="3" Body="&lt;p&gt;Epsilon-greedy is one method of making an agent explore the state space to ensure that the agent doesn't settle on a sub-optimal policy. By taking random actions, even with a small probability, the agent can get to places in the state space it normally wouldn't see and on the chance that the outcome is better than what it normally would have seen, it can modify its value function (or Q function) for that state (or action) to ensure that that action is taken in the future. An alternative to Epsilon-greedy methods is modifying the policy directly using Policy Gradient methods.&lt;/p&gt;&#xA;&lt;p&gt;Eligibility traces attempt to solve a different problem. Their function is to keep a short term memory of what states have been recently visited. They unify and generalize TD and Monte Carlo methods producing a family of methods spanning a spectrum that has Monte Carlo methods at one end (&lt;span class=&quot;math-container&quot;&gt;$\lambda = 1$&lt;/span&gt;) and one-step TD methods at the other (&lt;span class=&quot;math-container&quot;&gt;$\lambda = 0$&lt;/span&gt;). Using traces, one can keep the features of previous states around, but at faded values depending on the choice of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;. Choosing a low &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (near &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) makes the traces myopic in that the traces quickly go to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. If a larger &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is used (near &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) the traces stay around for much longer and can help an agent determine the difference between what would otherwise be 2 identical states.&lt;/p&gt;&#xA;&lt;p&gt;For more information, check out &lt;a href=&quot;http://ufal.mff.cuni.cz/%7Estraka/courses/npfl114/2016/sutton-bookdraft2016sep.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;, chapter 2 (page 30 introduces &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy methods) and chapter 12 (eligibility traces).&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-12-04T18:08:59.403" LastActivityDate="2020-12-04T18:08:59.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4325" PostTypeId="2" ParentId="4321" CreationDate="2017-10-22T02:41:21.660" Score="1" Body="&lt;p&gt;There are probably many different approaches to doing what you are talking about, but the most common would probably be using long short term memory recurrent neural networks to operate on the data through time. You would be able to train your neural network to do the things that you want to do such as classify the speaker, and classify noises into different categories. However, to train neural networks like these, it requires a lot of training data as does any other deep neural network. So, if you want to apply these kinds of neural networks to practical problems you will need a lot of training data. Or, you could just use a smaller simpler neural network.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-10-22T02:41:21.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4327" PostTypeId="1" AcceptedAnswerId="4348" CreationDate="2017-10-22T10:37:32.703" Score="3" ViewCount="92" Body="&lt;p&gt;AIs that rely on MCTS - like AlphaGo - create their decision tree as the game progresses. Do they start from scratch each game and build a new tree or do they keep the tree and grow it from game to game further?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the possible limitations of storage space for the search tree, I don't see any obvious drawback in keeping and growing the tree, which seems to me to be the preferred option. Are there other reasons to start from scratch each game?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-10-24T14:21:55.303" Title="Do AIs based on MCTS start each game from scratch?" Tags="&lt;ai-design&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4328" PostTypeId="1" CreationDate="2017-10-22T14:34:06.383" Score="7" ViewCount="3184" Body="&lt;p&gt;I know a little about these subjects. I found them similar to each other. Can anybody explain the differences between them?&lt;/p&gt;&#xA;" OwnerUserId="10314" LastEditorUserId="4302" LastEditDate="2019-06-12T02:49:26.367" LastActivityDate="2019-06-12T02:49:26.367" Title="What are the differences between machine learning, pattern recognition and data mining?" Tags="&lt;machine-learning&gt;&lt;pattern-recognition&gt;&lt;comparison&gt;&lt;data-mining&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4329" PostTypeId="1" CreationDate="2017-10-22T15:04:34.087" Score="1" ViewCount="205" Body="&lt;p&gt;I need to write a recurrent autoassociative network in python . I have read &lt;em&gt;Explorations in Parallel Distributed Processing: A Handbook o f Models,Programs, and Exercises&lt;/em&gt; but I can't understand how are the activation updating cycles written. any information about this would be helpful. specially if i could find an algorithm for this .&lt;/p&gt;&#xA;" OwnerUserId="10316" LastActivityDate="2017-10-22T15:04:34.087" Title="how to write a recurrent auto associative network?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4330" PostTypeId="1" AcceptedAnswerId="4331" CreationDate="2017-10-22T15:58:05.937" Score="1" ViewCount="822" Body="&lt;p&gt;Does &lt;em&gt;fitness proportionate selection&lt;/em&gt; select multiple individuals?&lt;/p&gt;&#xA;&lt;p&gt;So, I read on Wikipedia and on multiple Stack Exchange threads about fitness proportionate selection or rather &lt;em&gt;roulette selection&lt;/em&gt;, but what I don't understand is, how do I not select multiple times the same individuals? Should I pop them from my array and recalculate the probabilities or after selection remove duplicates? Or is there some sort of purpose for having multiple of the same selected?&lt;/p&gt;&#xA;" OwnerUserId="6391" LastEditorUserId="2444" LastEditDate="2021-01-30T21:49:31.280" LastActivityDate="2021-01-30T21:49:31.280" Title="Does fitness proportionate selection select multiple individuals?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-operators&gt;&lt;selection-operators&gt;&lt;fitness-proportionate-selection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4331" PostTypeId="2" ParentId="4330" CreationDate="2017-10-22T16:41:36.730" Score="2" Body="&lt;p&gt;Yes, you select the same individual multiple times, according to the distribution of fitness values. Typically, you have at least as many offspring as parents, so if you didn't replace parents in the pool to be potentially selected again, you'd just have every parent being selected once, which defeats the purpose of selection. You want fitter parents to be selected more often so that you drive the average fitness higher across generations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there are no iron clad rules. There are algorithms that create selection pressure in other ways, and you can do whatever works for you. But in the canonical version of a GA, you do selection with replacement. Though, many people find other selection schemes like tournament selection perform better across a pretty wide range of problems than roulette wheel anyway.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastActivityDate="2017-10-22T16:41:36.730" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4332" PostTypeId="1" AcceptedAnswerId="4909" CreationDate="2017-10-22T20:23:35.193" Score="0" ViewCount="545" Body="&lt;p&gt;I want to create a project where my Raspberry Pi Micro-controller can listen to my parrot talking and then talk back to him. I would need my rpi to listen to some of his phrases at least 100 times so that the pi could recognize that certain phrase and then say it back to him and also to know when to use it to talk back to him. I would also like to use it to teach him new phrases and new sounds. I would like to know if this is even possible and what hardware and software I would use to go about this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already asked this in the Raspberry Pi stack exchange but have recieved no answers. &lt;/p&gt;&#xA;" OwnerUserId="10324" LastActivityDate="2018-01-03T06:29:05.930" Title="Can the Raspberry Pi do audio learning?" Tags="&lt;machine-learning&gt;&lt;emotional-intelligence&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4333" PostTypeId="2" ParentId="4328" CreationDate="2017-10-22T20:57:41.933" Score="1" Body="&lt;p&gt;Machine learning is a form of pattern recognition. Machine learning is basically the idea of training machines to recognize patterns and apply it to particle problems. Data science is the science of apply machine learning to practical problems such as creating better search engine results or classifying images. Patten recognition is pretty much the umbrella term here. However, I think that the pattern recognition term is sort of falling out of style with how modern data scientists are training neural networks and other machine learning models.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-10-22T20:57:41.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4334" PostTypeId="2" ParentId="4320" CreationDate="2017-10-22T21:14:29.857" Score="7" Body="&lt;p&gt;The initial weights in a neural network are initialized randomly because the gradient based methods commonly used to train neural networks do not work well when all of the weights are initialized to the same value. While not all of the methods to train neural networks are gradient based, most of them are, and it has been shown in several cases that initializing the neural network to the same value makes the network take much longer to converge on an optimum solution. Also, if you want to retrain your neural network because it got stuck in a local minima, it will get stuck in the same local minima. For the above reasons, we do not set the initial weights to a constant value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&#xA;&lt;a href=&quot;https://stats.stackexchange.com/q/45087/82135&quot;&gt;Why doesn't backpropagation work when you initialize the weights the same value?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="2444" LastEditDate="2019-04-23T13:49:12.397" LastActivityDate="2019-04-23T13:49:12.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4335" PostTypeId="2" ParentId="4332" CreationDate="2017-10-22T21:48:21.397" Score="1" Body="&lt;p&gt;This would probably best be done by training up some light model on your primary machine and then porting it to the raspi after training. Because of the lack of resources, your neural net(or similar) would likely have to be computationally inexpensive to run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that being said, a recurrent neural network is a very good architecture for speech recognition. I would take a look at something like &lt;a href=&quot;https://medium.com/@ageitgey/machine-learning-is-fun-part-6-how-to-do-speech-recognition-with-deep-learning-28293c162f7a&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; to get you started. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, since it seems you are training it on your parrot's voice, make sure you have a dataset that is comprised of yours or similar parrots' voices.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-10-22T21:48:21.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4338" PostTypeId="1" CreationDate="2017-10-23T08:57:52.940" Score="1" ViewCount="559" Body="&lt;p&gt;I want to implement &lt;a href=&quot;https://arxiv.org/abs/1607.04381&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSD: Dense-Sparse-Dense training for deep neural networks&lt;/a&gt; by Han et al. In short, the paper suggest the following training scheme to improve the network accuracy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QkuUk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QkuUk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Train as usual till convergence, &lt;/li&gt;&#xA;&lt;li&gt;prune the network and train with sparsity constraint, &lt;/li&gt;&#xA;&lt;li&gt;remove the sparsity constraint and let the pruned connections to recover.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question is about step 2: train with sparsity constraint. The paper mentions training with a binary mask specifying the pruned weights to keep &quot;untouched&quot; so the sparsity constraint is satisfied, however that means implementing a dedicated layer that takes the binary mask as an additional blob and handles it accordingly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder a simpler approach will give the same result: what if after the pruning step I keep the location of the pruned weights and then use dense training, but after each iteration to zero the originally pruned weights?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The forward path is the same, taking zero weights for the pruned weights anyway.  But would it negatively affect the backward path - since the constraint isn't there, or is it equivalent to the formal training scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9233" LastEditorUserId="9647" LastEditDate="2017-10-25T06:29:27.027" LastActivityDate="2017-10-25T06:29:27.027" Title="Dense-Sparse-Dense CNN training" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4339" PostTypeId="2" ParentId="4338" CreationDate="2017-10-23T16:45:53.613" Score="1" Body="&lt;p&gt;Your approach will give a similar result.&#xA;The difference with using the constraint is that the constraint allows the network to essentially confirm that it doesn't need the connections before it reallocates them (i.e. if the weight is lessened by the constraint and it results in something not working, then the weight will start increasing).&lt;/p&gt;&#xA;" OwnerUserId="10338" LastActivityDate="2017-10-23T16:45:53.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4340" PostTypeId="2" ParentId="4218" CreationDate="2017-10-24T00:23:52.290" Score="1" Body="&lt;p&gt;You could probably use a simpler model if you do not have that much training data, but if you have a lot of training data, I would recommend taking this approach: you will want to one hot encode the states, and depending upon how many colors you have, you will either want to encode them in a color space such as RGB or one hot encode them if you do not have that many of them. Then, you will want to one hot encode the routes that you have. For code 2, it looks like it is quantitative so you can probably just normalize that row. For code 1, you will probably just have to one hot encode the letter, and normalize it. Once you have all of your training data in a quantitative form, if you have a lot of training data, and the training data is sufficiently complex, you can use a deep neural network. Otherwise, you could use a conventional single layer neural network.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-10-24T00:23:52.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4341" PostTypeId="2" ParentId="4320" CreationDate="2017-10-24T00:35:12.690" Score="12" Body="&lt;p&gt;You shouldn't assign all to 0.5 because you'd have the &amp;quot;break symmetry&amp;quot; issue.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.deeplearningbook.org/contents/optimization.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.deeplearningbook.org/contents/optimization.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Perhaps the only property known with complete certainty is that the initial&#xA;parameters need to “&lt;strong&gt;break symmetry&lt;/strong&gt;” between different units. If two hidden&#xA;units with the same activation function are connected to the same inputs, then&#xA;these units must have &lt;strong&gt;different initial parameters&lt;/strong&gt;. If they have the same initial&#xA;parameters, then a deterministic learning algorithm applied to a deterministic cost&#xA;and model will constantly update both of these units in the same way. Even if the&#xA;model or training algorithm is capable of using stochasticity to compute different&#xA;updates for different units (for example, if one trains with dropout), it is usually&#xA;best to initialize each unit to compute a different function from all of the other&#xA;units. This may help to make sure that no input patterns are lost in the null&#xA;space of forward propagation and no gradient patterns are lost in the null space&#xA;of back-propagation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-10-24T01:33:39.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4342" PostTypeId="2" ParentId="3407" CreationDate="2017-10-24T02:51:06.500" Score="1" Body="&lt;p&gt;If you have enough data, you can train a segmentation net &lt;a href=&quot;https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; for a specific group of data (i.e. trees). Single DNN with multiple output branches can easily segment &amp;amp; classify the data that you are searching for. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-10-24T02:51:06.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4343" PostTypeId="1" CreationDate="2017-10-24T06:17:26.953" Score="1" ViewCount="162" Body="&lt;p&gt;I have a liberal arts background so I need help understanding &lt;a href=&quot;https://arxiv.org/pdf/1708.09839.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, particularly pages 26 to 30. The authors test a four-camera system for localization, mapping, and obstacle detection for self-driving cars. The paper seems to say the multi-camera system can map the environment to within an average of 7 cm (2.8 inches) of accuracy (with the largest error being 16 cm or 6.3) and detect obstacles to within 10 cm (3.9 inches) of accuracy. Am I getting this right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that automotive lidar can detect objects to within 1.5 cm (0.6 inches) of accuracy, and given that for driving purposes the difference between 1.5 cm and 7 cm, 10 cm, or 16 cm seems quite small, can a multi-camera system be used instead of lidar in a self-driving car application? How do driving speeds affect things? What crucial elements of the problem space might I be overlooking or misunderstanding?   &lt;/p&gt;&#xA;" OwnerUserId="10350" LastActivityDate="2017-12-25T07:19:49.200" Title="‪Can a multi-camera system be used for localization, mapping, and obstacle detection in self-driving cars to within 10 cm of accuracy? Whither lidar?‬" Tags="&lt;autonomous-vehicles&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4344" PostTypeId="1" CreationDate="2017-10-24T08:23:13.753" Score="1" ViewCount="171" Body="&lt;p&gt;I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this video game, you start with an empty 12x6 board which you fill with falling pairs of colored blobs called puyos. When 4 or more puyos of the same color are connected horizontally and/or vertically, they disappear and the puyos above them fall down, potentially creating new groups of puyos that disappear and so on, creating a chain (here is a chain example : &lt;a href=&quot;https://puyonexus.com/chainsim/image/5SoqU.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://puyonexus.com/chainsim/image/5SoqU.gif&lt;/a&gt;). &#xA;My goal is to teach the neural network to build the longest chains possible given a random sequence of puyo pairs. &#xA;The game's information is incomplete though, as you only see the current and the next two pairs at a given time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best choice for the output layer: a layer with 22 neurons, one for each possible way to place a pair at a given time, and a softmax activation function, or a layer with a single output node, with a linear activation function? Or something else? Also what do you think is a good number of hidden layers and neurons per layer and what activation function should I use for the hidden layers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should my target Q-value be when I evaluate a move, so that I can train the network on each game situation? Right now I'm thinking about using the maximum chain possible after the move is done, divided by half of the number of moves played so far. This is because if you play optimally and you are lucky enough, you can increase your maximum chain by 1 every 4 puyos you place, that is every two moves. Thus my Q-value would stay between 0 and 1. &#xA;What is the best update formula I can use? Is the one &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning#Algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;on wikipedia&lt;/a&gt; correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use a softmax output layer, how do I update the target Q-value vector? The update formula gives me an update but then the vector doesn't represent probabilities anymore. Can I add the update then take the softmax of the vector again just to make it probabilities again? &lt;/p&gt;&#xA;" OwnerUserId="10355" LastEditorUserId="1671" LastEditDate="2017-12-21T19:58:00.350" LastActivityDate="2017-12-21T19:58:00.350" Title="Help with implementing Q-learning for a feedfoward network playing a video game" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4346" PostTypeId="1" AcceptedAnswerId="4347" CreationDate="2017-10-24T13:34:38.233" Score="6" ViewCount="958" Body="&lt;p&gt;I read through the publication &lt;a href=&quot;https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the game of Go without Human Knowledge&lt;/a&gt;. It doesn't seem to use GANs, just a new form of search and reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="10359" LastEditorUserId="1671" LastEditDate="2017-12-22T17:38:09.817" LastActivityDate="2017-12-22T17:38:09.817" Title="Is the new Alpha Go implementation using Generative Adversarial Networks?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;unsupervised-learning&gt;&lt;go&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4347" PostTypeId="2" ParentId="4346" CreationDate="2017-10-24T13:47:05.883" Score="11" Body="&lt;p&gt;No, GANs are &lt;em&gt;not&lt;/em&gt; used. It's reinforcement learning at what it does best. The tree search is an interesting addition and assists with navigating the sheer scale of the game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the agent was playing itself to become better, there wasn't 2 separate networks (generator and discriminator). The agent learned through RL and didn't have the error calculations that generators and discriminatory have. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="10359" LastEditDate="2017-10-26T20:23:12.693" LastActivityDate="2017-10-26T20:23:12.693" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4348" PostTypeId="2" ParentId="4327" CreationDate="2017-10-24T14:21:55.303" Score="3" Body="&lt;p&gt;Your intuition is right. The tree is grown after each game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before the first play through, the values of each decision point are initialized (randomly or to some constant). Then at the end of one play through, the weights of the decision points are updated using Monte Carlo methods. In this way, on the next play through, the updated weights help the agent make a decision in the next game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how the decision points are added to the tree, it depends on the application. For simple games like TicTacToe, we know all of the decision points. For big games, there are methods to trim the tree of bad performing branches. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-10-24T14:21:55.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4349" PostTypeId="2" ParentId="4343" CreationDate="2017-10-24T15:15:40.080" Score="1" Body="&lt;p&gt;Given that a self-driving car is trying to replicate the performance of a two-camera system (or one in a pinch), there is nothing in principal that mandates lidar for a self-driving car.  Lidar is a shortcut, substituting sensor sophistication for image-processing sophistication.  AFAIK &lt;a href=&quot;https://www.youtube.com/watch?v=-96BEoXJMs0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nvidia's own self-driving vehicle&lt;/a&gt;  doesn't have Lidar.  My personal opinion is that Level 5 self-driving vehicles won't be practical until they have the kind of image-processing sophistication that makes Lidar an unnecessary crutch. &lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2017-10-24T15:15:40.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4354" PostTypeId="1" CreationDate="2017-10-24T21:35:13.083" Score="2" ViewCount="74" Body="&lt;p&gt;It appears to me that the thoughts in our heads, or at least mine, are a result of the language they know. I do not have any thought that is not a consequence of the information that is contained within the language I know. I have thoughts that occur as the words we use in language and are particular to my voice  The structure of that language forms constraints as to the depth of thought that my mind can possibly come up with as far as I can perceive.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is intelligence limited to thoughts it is able to associate words with, since the mind's language is a result of a recursive association of words associated with those thoughts? Is there a limit of conception bound by a language, which is limited by its dictation?&lt;/em&gt; If so, there would also be a limit upon the perception of that so-called intelligence.&lt;/p&gt;&#xA;&lt;p&gt;Consequently, given that artificial intelligence systems are created by humans, aren't they limited by our language and knowledge of it?&lt;/p&gt;&#xA;" OwnerUserId="9094" LastEditorUserId="2444" LastEditDate="2020-09-17T14:45:51.147" LastActivityDate="2020-09-17T14:45:51.147" Title="Is intelligence limited to thoughts it is able to associate words with?" Tags="&lt;natural-language-processing&gt;&lt;philosophy&gt;&lt;intelligence&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4355" PostTypeId="2" ParentId="4354" CreationDate="2017-10-24T22:22:13.290" Score="1" Body="&lt;p&gt;First thing we were taught in AI: &#xA;Humans didn't create airplanes by studying and mimicing birds, they studied aerodynamics and physics.&#xA;Although difficult to articulate I've had very similar thought patterns to yours when I was thinking about AI and further I watched very closely the development of my child from being a newborn to where he is now, a year and a bit old.  The human brain is far more complex than what we'll understand, it would take a multi-discipline team effort to even scratch the surface.  So to create a human brain-like thought process and try to put it into a formal language box would be impossible.&#xA;I think the breakthrough in AI isn't going to be someone putting together multiple complex algorithms or even a fancy NN running on a supercomputer.  It's going to be something we haven't figured out yet.&#xA;Still, fun to think about...  I'd say it's more of a philosophical question than an AI one though...&#xA;Just my humble opinion.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2017-10-24T22:22:13.290" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4356" PostTypeId="2" ParentId="3272" CreationDate="2017-10-25T01:45:45.900" Score="0" Body="&lt;p&gt;Great question equally qualified answers.  My belief is yes to understanding language and the bugaboo (a non intellectual vernacular) is what is understanding.  Is it interpretive?  Inferred? To what end?  An algorithm's response to a command in language form will depend on the robustness programmed into it.  However, qualia connotes a subjective experience due to a sensory stimuli whether triggered by memory or one of our human senses.  Then it begs the question of can a computer collectively experience and how would we know that.  Second, are all algorithms subjected due the the programmed bias and it's available data store?  Facebook and Google news have shown that programmed bias is very real. Furthermore, Qualia is an emergent trait so I can't see how a computer and it's collective systems can become aware and have subjective experience.&lt;/p&gt;&#xA;" OwnerUserId="10370" LastActivityDate="2017-10-25T01:45:45.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4357" PostTypeId="1" CreationDate="2017-10-25T03:09:00.450" Score="2" ViewCount="239" Body="&lt;p&gt;So, i need to use an MLP to predict a 12x12 matrix composed of floating points. &#xA;The matrices are as this one that follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j8pfK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j8pfK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Most matrices have this &quot;pattern&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As input, i have 7 floating points, such as these:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;2.0&quot;, &quot;0.23&quot;, &quot;239.10&quot;,&quot;3.5&quot;, &quot;12,0&quot;, &quot;10.6&quot;, &quot;0.62&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simplify the output matrix, i've converted it in a array with the 144 elements of the matrix.&#xA;So far i'm using MLPRegressor from Scikit.The problem is, there's absolutely no pattern in the predicted results.The predicted results include negative numbers, numbers really big and no &quot;pattern&quot; for the indexes whatsoever.&#xA;Is there a way to adjust these things in the model or the problem is on my dataset?&#xA;Thank you very much!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wasn't very clear about the problem, so i'll try to explain it better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the problem: Predict water distribution in irrigation systems. I already have a program to simulate this, and works with rather good precision. The challenge now was to make an AI model to simulate this, instead of the old methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the inputs: Inputs are one float representing the pressure of the sprinkler used to distribute water, two floats representing the speed and angle of the wind at the time data was collected. The 4 remaining are numbers representing some configuration of the sprinker (these numbers change very little between instances, but they do have impact in the result, which is why i decided to mantain them in the dataset).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the output: Output is an array of 144 elements, representing the 12*12 matrix. Each element of the matrix contain a number representing the amount of water that was collected in that point. These collectors where evenly spread around the sprinkler. So the position of the matrix matter a lot, since in most cases the first and last lines and columns will have 0 or a close to 0 (but positive) number - this may vary depending especially on the speed and angle of the wind, but also on the sprinkler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the dataset: I have available 75 instances. They are all stored in a CSV file, where the 7 inputs and the 144 outputs are, each in one line.&lt;/p&gt;&#xA;" OwnerUserId="9934" LastEditorUserId="9934" LastEditDate="2017-10-25T16:47:03.443" LastActivityDate="2017-10-25T16:47:03.443" Title="Using a MLP to predict a 12x12 matrix" Tags="&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4358" PostTypeId="2" ParentId="4009" CreationDate="2017-10-25T05:11:49.480" Score="2" Body="&lt;p&gt;One of the biggest problems in training neural networks is creating high quality data sets. E.g. if you want to classify pictures, you need a huge set of correctly classified pictures as training input. In your scenario, you can automate this laborious job by feeding random data to your blackbox and store the output. Voila, you have your training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your neural network will have the same input and output structure as your blackbox. You can use your generated training data the same way you you would use manually generated training data. I cannot provide any actual implementation, because the training mechanism depends on your technology stack incl. the frameworks you are using. But there is nothing out of the ordinary in your scenario when it comes to the training process itself and you can follow one of the many available tutorials for training neural networks - a pretty basic one for example would be this &lt;a href=&quot;https://elitedatascience.com/keras-tutorial-deep-learning-in-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial for Keras&lt;/a&gt; (if that's your choice of framework).&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-10-25T05:11:49.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4360" PostTypeId="2" ParentId="4233" CreationDate="2017-10-25T06:39:04.117" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Assuming that you are using DarkFlow, you can train your model with &lt;code&gt;--train&lt;/code&gt; flag and don't use &lt;code&gt;--gpu&lt;/code&gt; flag since you don't want to use the GPU memory. And yes it's possible to train with just 10 images (basically over-fitting) just to confirm the working of your model. Place those 10 images in train/images directory and the annotations of these images in train/annotations directory (in PASCAL VOC format). &#xA;For faster performance, use pre-trained weights file for tuning your model so that you don't need to train for initial same layers (and since you just want to detect a single object - license plates - use tiny-yolo-voc.weights.&#xA;Further more, use &lt;code&gt;--epoch&lt;/code&gt; flag to reduce the number of epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;flow --model cfg/tiny-yolo-voc-new.cfg --load bin/tiny-yolo-voc.weights --train --annotation train/Annotations --dataset train/Images --epoch 20&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Change the number of classes and the number of filters as defined in the readme first. Then you can change the height and width of the input image the model will be trained on. Higher resolution for more accuracy while lower resolution for higher performance and speed. You can also change the activation functions for the layers, but I recommend leaving them unchanged.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10376" LastActivityDate="2017-10-25T06:39:04.117" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4361" PostTypeId="1" AcceptedAnswerId="4372" CreationDate="2017-10-25T09:51:34.497" Score="1" ViewCount="110" Body="&lt;p&gt;I've been researching the following topic. Or rather, I would like to but I can't find anything because I'm not sure what to look for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested weather there are some concepts or models that explain how humans (or cognitive systems in general) trade off &lt;em&gt;remembering&lt;/em&gt; what to do in a particular situation versus &lt;em&gt;thinking&lt;/em&gt; about the board state and devising a new solution. Consider the following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A person plays chess. On their turn they must decide what to do. They could act according to one of the following extreme strategies:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Remembering all possible board configurations and how the game ended (we'll call that &lt;em&gt;Memory&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Use the known set of rules to &lt;em&gt;simulate&lt;/em&gt; the game in their mind and choose the optimal path (&lt;em&gt;Optimization&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'd say the best choice is a trade off between these two. But how? When to choose one of the strategies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a research field that is working on these kind of questions? What would I look for?&lt;/p&gt;&#xA;" OwnerUserId="2937" LastEditorUserId="2937" LastEditDate="2017-11-02T08:41:52.233" LastActivityDate="2017-11-02T08:41:52.233" Title="Trading off &quot;Memory&quot; vs &quot;Optimization&quot;" Tags="&lt;human-like&gt;&lt;learning-algorithms&gt;&lt;chess&gt;&lt;combinatorial-games&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4362" PostTypeId="2" ParentId="4361" CreationDate="2017-10-25T11:19:41.757" Score="2" Body="&lt;p&gt;If you can remember everything and there's no randomisation in your outcome like chess, there is absolutely &lt;strong&gt;no&lt;/strong&gt; reason not to do that. Anybody who can remember all the possible board configurations in chess, by definition plays perfect chess. A perfect player would never lose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, most practical problems can't be solved by brute-force, and that includes chess. &lt;strong&gt;All&lt;/strong&gt; chess players learn general chess principles. Nobody could memorize all possible opening variations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can somehow memorize every possibility, why not? But can you? &lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-10-25T11:19:41.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4363" PostTypeId="2" ParentId="4354" CreationDate="2017-10-25T13:33:14.620" Score="1" Body="&lt;p&gt;Computers can interpret language, but &lt;em&gt;I don't believe&lt;/em&gt; they understand it.  Also, it's interpretation is bounded by the algorithmic constructs, like data sets and the routines that run against it.&lt;/p&gt;&#xA;&lt;p&gt;Consciousness, on the other hand, is an emergent quality and includes self-awareness amongst other things, and self-awareness is a subjective experience.&lt;/p&gt;&#xA;&lt;p&gt;Computer language is far from being a human language, but I believe it can be a close approximation.&lt;/p&gt;&#xA;&lt;p&gt;One challenge is non-linear learning and ambiguity. But with an adequately large and fast computer that may also be approximated very closely.&lt;/p&gt;&#xA;&lt;p&gt;Some argue that the human brain works in some quantum mechanical way and experiences many possibilities with the one we choose to encode in our &amp;quot;minds&amp;quot; representing the collapsed wave function after having evaluated the stimuli. Only recently have scientists discovered that photosynthesis operates at a quantum level.&lt;/p&gt;&#xA;" OwnerUserId="10370" LastEditorUserId="2444" LastEditDate="2020-09-17T14:45:01.843" LastActivityDate="2020-09-17T14:45:01.843" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4364" PostTypeId="2" ParentId="4357" CreationDate="2017-10-25T14:00:59.530" Score="2" Body="&lt;p&gt;The short answer to your question is: you probably do not fully know your data. remember that ML is no magic wand. It needs your understanding of the data and the behavior of it. Although it is approved that neural networks with at most two hidden layers can approximate &lt;strong&gt;any&lt;/strong&gt; model with an acceptable degree precision, setting up the structure of the neural network is still your task; also note that the cleaner your data is, the more acceptable the degree of precision will be for a structured NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the following more detailed explanation might also help. &lt;strong&gt;understanding your data&lt;/strong&gt; that I pointed above, means:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;strong&gt;design of your machine learning pipeline&lt;/strong&gt; is the first and most important thing to take care of. Are you sure you have handled training set and test set appropriately? are the ratios standard? have you performed random sampling well?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;strong&gt;size of the dataset&lt;/strong&gt; is also a matter of consideration. remember a ratio of &lt;code&gt;n/p&amp;gt;10&lt;/code&gt; (n: the number of instances, p: the number of features) is necessary. So if you have 7 features, you should have more than around 100 instances. check how big is your dataset&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;strong&gt;distribution of your dataset&lt;/strong&gt; is also important. dealing with badly distributed data should be considered if you have an issue here. use plots and histograms to understand the distribution of your dataset over feature axes. There is a high risk of overfitting if your data is distributed badly&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;also take care of your &lt;strong&gt;output distribution&lt;/strong&gt;. with a bad distribution over your output data (focused outputs in a zone) you loose the power of generalization of your model&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;check the relations between different features. check if there are any pairs of &lt;strong&gt;dependent variables&lt;/strong&gt;. you can use scatter plot or calculating pairwise covariances. PCA or more complicated methods (like auto-encoders) will be of help if there is any issue here&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;one possibility is maybe your data doesn't carry any latent information at all. maybe &lt;strong&gt;the underlying pattern is so stochastic and nonlinear that it seems to be random&lt;/strong&gt;. in other words: you can not predict what kind of food your grandma has cooked by looking at results of world cup games, no matter how big your dataset is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now about your output matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;reshaping your matrix to an array of 144 might be the best or the worst thing to do. it depends on the characteristics of your data. maybe the position of each element in the matrix is also important in predicting the value. so you might want to include the &lt;code&gt;i,j&lt;/code&gt; of each element in the input array too. maybe each output also takes effect from neighbors. there are some other options to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;you can &lt;strong&gt;use 144 different MLP networks&lt;/strong&gt;, one for each element (a little bit like what you have already done, when reshaping your matrix. since your library constructs a single MLP network that has 144 neurons in the output layer. so it seems like 144 networks that only vary in the output layer, and note that the weight correction step is different too, since the training error for all of the outputs propagate back in the layers simultaneously)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you can &lt;strong&gt;create 144 instances out of each instance&lt;/strong&gt;, each of which has 7+2 inputs and 1 output. the 2 more inputs are the position of the output value (i,j). it might be a poor decision to do that but this method has helped me a couple of times, so test it&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do the second step, but &lt;strong&gt;include the neighbor values too.&lt;/strong&gt; so you'd have for example 7+2+4 inputs (7 initial + 2 i,j + 4 neighbor values)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;also &lt;strong&gt;consider using RNNs&lt;/strong&gt; (recurrent neural networks). if you consider the matrix to be a sequence of values which has internal dependency plus the external dependency to those 7 features, then you can use a RNN network like LSTM.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;feel free to update your post with more specific experiments and issues.&lt;/p&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2017-10-25T14:00:59.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4366" PostTypeId="1" CreationDate="2017-10-25T21:16:48.807" Score="1" ViewCount="66" Body="&lt;p&gt;I'm running A3C (Asynchronous Actor-Critic Agents) to learn a game where an agent needs to catch 3 rewards. The input of my network, among other things, is the relative position of the 3 rewards against the agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as the agent catches these rewards, my network can't just decrease its input size. What are ways of handling this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current solutions have been:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;set remaining features to zero - have tried, bad results&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;just show the closest reward - loss of information, might not be optimal&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;repeat existing reward positions to remaining features - we are giving more emphasis to some rewards (the ones that are repeated) over others&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="1671" LastEditDate="2017-10-26T20:35:04.420" LastActivityDate="2017-10-26T20:35:04.420" Title="Handling varied-size input with fixed-input network" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4371" PostTypeId="1" CreationDate="2017-10-26T20:07:45.940" Score="4" ViewCount="30" Body="&lt;p&gt;I was wondering if anyone can think of any examples of integration of audio-visual inputs, or other modalities, failing or resulting in undesired outcomes in artificial cognitive systems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in any issues, or artifacts, that have arisen when different information about the same event have been integrated in such a system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Published, or discussed in print online, or even a TED talk would be great sources if you know of any. Anecdotal evidence of such scenarios that you may have experienced are also of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all for your time.&lt;/p&gt;&#xA;" OwnerUserId="10419" LastActivityDate="2017-10-26T20:07:45.940" Title="Examples of &quot;crossmodal&quot; integration failures in artificial cognitive systems" Tags="&lt;cognitive-science&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4372" PostTypeId="2" ParentId="4361" CreationDate="2017-10-26T21:43:26.573" Score="2" Body="&lt;p&gt;The The &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Oxford_Companion_to_Chess&quot; rel=&quot;nofollow noreferrer&quot;&gt;Oxford Companion to Chess&lt;/a&gt; has entries on only 700 named openings, and lists another 1327 opening variations in the index, and I wouldn't be surprised if someone out there had them all memorized.  For an algorithm, however, storing that number of openings is trivial, and Chess algorithms traditionally made use of high-quality &quot;game books&quot; which are move-by-move records of chess games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractable models&lt;/a&gt;, like chess is that they are unsolvable, so brute-force is not an option, and any decision on a move, whether &quot;memory&quot; or &quot;optimality&quot; based, at least in the context of current advances, are probabilistic.  (&lt;a href=&quot;https://en.wikipedia.org/wiki/Endgame_tablebase&quot; rel=&quot;nofollow noreferrer&quot;&gt;The exception is endgames&lt;/a&gt; where the the problem becomes tractable, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#Perfect_play&quot; rel=&quot;nofollow noreferrer&quot;&gt;perfect play&lt;/a&gt; can be achieved.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the memory of a board state is only a small piece of the puzzle because the unsolvability of the game means that particular board state from a recorded game, where that state led to a successful outcome, will not guarantee a similar result in a new game, because new, more optimal strategies can always emerge.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, because of the extreme variation engendered by non-trivial games, it is extremely unlikely any current game would mirror the previous &quot;memorized&quot; game.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is my understanding that &quot;memory&quot; as you define has been utilized in Chess engines, accessing a database of previous games to evaluate a position and derive a good move, but the intractability of Chess in general makes this method sub-optimal, and it has been definitively supplanted by newer methods (machine learning, neural networks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the famous Giraffe Chess paper, Matthew Lai states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The job of the evaluation functions is to assign scores to positions statically (without looking ahead)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But the method is highly nuanced and probabilistic.  Nevertheless, I think you'll be interested because the problem presented in the paper is rooted in the human vs. AI play method issue:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...the way computers play chess is very different from how humans play. While both humans and computers search ahead to predict how the game will go on,&#xA;  humans are much more selective in which branches of the game tree to explore. Computers, on the other hand, rely on brute force to explore as many continuations as possible, even ones that will be&#xA;  immediately thrown out by any skilled human... How can a human searching 3-5 positions per second be as strong as a computer searching 200&#xA;  million positions per second? And is it possible to build even stronger chess computers than what&#xA;  we have today, by making them more computationally efficient? Those are the questions this&#xA;  project investigates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no simple answer to the trade-off question due to the complexity of the problem.  But this is an area where tremendous advances are being made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd strongly recommend reading &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe Chess (September, 2015)&lt;/a&gt; as a primer. &#xA; It produced very exciting results, synopsized in the MIT Tech Review article &quot;&lt;a href=&quot;https://www.technologyreview.com/s/541276/deep-learning-machine-teaches-itself-chess-in-72-hours-plays-at-international-master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Machine Teaches Itself Chess in 72 Hours, Plays at International Master Level&lt;/a&gt;&quot;.  &lt;em&gt;(Giraffe's creator currently works as a &lt;a href=&quot;https://matthewlai.ca/resume/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Research Engineer in Machine Learning at DeepMind&lt;/a&gt;.)&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-27T21:15:08.040" LastActivityDate="2017-10-27T21:15:08.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4373" PostTypeId="1" CreationDate="2017-10-26T21:55:45.817" Score="1" ViewCount="34" Body="&lt;p&gt;Could a multi-camera SLAM system for self-driving cars that is accurate to &lt;a href=&quot;https://arxiv.org/pdf/1708.09839.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;under 10 cm (3.9 in)&lt;/a&gt; at parking lot speeds (i.e. very low driving speeds) retain this level of accuracy at high driving speeds (e.g. highway speeds or city driving speeds) if the cameras were &lt;strong&gt;a)&lt;/strong&gt; 60 fps and &lt;strong&gt;b)&lt;/strong&gt; had a global shutter?&lt;/p&gt;&#xA;" OwnerUserId="10350" LastActivityDate="2017-10-26T22:21:49.140" Title="Could a multi-camera SLAM system that is accurate at low driving speeds be equally accurate at high driving speeds?" Tags="&lt;computer-vision&gt;&lt;robots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4374" PostTypeId="2" ParentId="4373" CreationDate="2017-10-26T22:21:49.140" Score="1" Body="&lt;p&gt;It would probably be accurate at higher speeds, if the processing system that processes the information from the camera system can process the information in real time. It looks like it can, but you will also have to account for motion blur. If there is too much motion blur, you will probably not be able to do anything other than speeding up the camera and processing system. However, if the motion blur is not that bad, and the system has been trained on the motion blur, it should be able to classify everything correctly.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-10-26T22:21:49.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4375" PostTypeId="1" AcceptedAnswerId="4401" CreationDate="2017-10-27T10:24:37.183" Score="10" ViewCount="9546" Body="&lt;p&gt;I talk about the robot from: &lt;a href=&quot;http://www.hansonrobotics.com/robot/sophia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hanson Robotics&lt;/a&gt;, which was &lt;a href=&quot;https://techcrunch.com/2017/10/26/saudi-arabia-robot-citizen-sophia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;granted the right to citizenship from Saudi Arabia&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found the following articles:&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Your new friend is a humanoid robot&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://www.theaustralian.com.au/life/say-hello-to-your-new-friend-sophia-the-humanoid-robot/news-story/070299a8d11b7d636848f1b8dd753530&quot; rel=&quot;nofollow noreferrer&quot;&gt;theaustralian.com.au&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Like Amazon Echo, Google Assistant and Siri, &lt;strong&gt;Sophia can ask and answer questions about discrete pieces of information&lt;/strong&gt;, such as what types of movies and songs she likes, the weather and whether robots should exterminate humans.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;But her general knowledge is behind these players and she doesn’t do maths. &lt;strong&gt;Her answers are mostly scripted&lt;/strong&gt; and, it seems, from my observation, her answer are derived from algorithmically crunching the language you use.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Sometimes answers are close to the topic of the question, but off beam. Sometimes she just changes the subject and asks you a question instead.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;She has no artificial notion of self. &lt;strong&gt;She can’t say where she was yesterday, whether she remembers you from before&lt;/strong&gt;, and doesn’t seem to amass data of past interactions with you that can form the basis of an ongoing association.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Questions such as: &lt;em&gt;“What have you seen in Australia?”&lt;/em&gt;, &lt;em&gt;“Where were you yesterday?”&lt;/em&gt;, &lt;em&gt;“Who did you meet last week?”&lt;/em&gt; and &lt;em&gt;“Do you like Australia?”&lt;/em&gt; are beyond her.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Why Sophia the robot is not what it seems&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://www.smh.com.au/comment/why-sophia-the-robot-is-not-what-it-seems-20171030-gzbi3p.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;smh.com.au&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You can often fool this sort of software by introducing noise. That could be literal noise – machines aren't great at filtering out background noise, as anyone with a hearing aid will tell you – or it could be noise in the sense of irrelevant information or limited context. You could ask &lt;strong&gt;&quot;what do you think of humans?&quot;&lt;/strong&gt; and then follow up with &lt;strong&gt;&quot;can you tell more about it?&quot;&lt;/strong&gt; The second question requires the robot to define &quot;it&quot;, remember what it said last time, and come up with something new.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In the case of the ABC interview, &lt;strong&gt;the questions were sent to Sophia's team ahead of time so they were possibly pre-scripted&lt;/strong&gt;. Just like an interview with a human celebrity!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Pretending to give a robot citizenship helps no one&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.theverge.com/2017/10/30/16552006/robot-rights-citizenship-saudi-arabia-sophia&quot; rel=&quot;nofollow noreferrer&quot;&gt;theverge.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sophia is essentially a cleverly built puppet designed to exploit our cultural expectations of what a robot looks and sounds like. It can hold a stilted conversation, yes, but its one-liners seem to be prewritten responses to key words. (As Piers Morgan commented during an interview with Sophia, &lt;strong&gt;“Obviously these are programmed answers.”&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here are a few more references.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://qz.com/1121547/how-smart-is-the-first-robot-citizen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inside the mechanical brain of the world’s first robot citizen&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://buildabuddha.quora.com/Sophia-the-Uncanny-Robot-AI-Is-Not-What-You-Think&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sophia, the Uncanny Robot AI, Is Not What You Think.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.quora.com/How-much-of-Robot-Sophia%E2%80%99s-speech-is-likely-scripted-with-your-understanding-of-the-progress-in-NLP&quot; rel=&quot;nofollow noreferrer&quot;&gt;How much of Robot Sophia’s speech is likely scripted with your understanding of the progress in NLP?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10434" LastEditorUserId="2444" LastEditDate="2021-01-26T18:07:57.920" LastActivityDate="2021-04-08T04:34:07.287" Title="Are the dialogs at Sophia's (the robot) appearings scripted?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;robotics&gt;&lt;robots&gt;&lt;sophia&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4376" PostTypeId="1" CreationDate="2017-10-27T12:44:01.850" Score="5" ViewCount="145" Body="&lt;p&gt;I have read the paper &lt;a href=&quot;https://arxiv.org/pdf/1410.5401.pdf%20&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Turing Machines&lt;/a&gt; and the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0022000085710136&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the Computational Power of Neural Nets&lt;/a&gt; about the computational power of neural networks. However, it isn't still clear to me one thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way of converting a neural network to another one that represents the same function? For example, given a trained recurrent neural network that represents a certain function, is it possible to convert this RNN to e.g. an MLP that represents the same function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an answer to the above question exists, can we show the equivalence between neural networks, e.g., all problems solved by a multilayer perceptron can be solved by a recurrent neural network, but the opposite is not true, i.e., MLP is subset of RNN (I do not know if this is true, it is just an example). So, if we obtain this relationship between all neural networks, we can get a neural network &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; that is more powerful than others, so, we can throw away all other neural networks because &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; can solve any problem that other NN can. Is this reasoning correct?&lt;/p&gt;&#xA;" OwnerUserId="9581" LastEditorUserId="2444" LastEditDate="2020-02-23T21:05:28.490" LastActivityDate="2020-02-28T18:32:37.473" Title="Is there a way of converting a neural network to another one that represents the same function?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;function-approximation&gt;&lt;computational-learning-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4377" PostTypeId="1" CreationDate="2017-10-27T16:43:58.267" Score="6" ViewCount="201" Body="&lt;p&gt;I'm very interested in writing a &lt;strong&gt;Spiking Neural Network&lt;/strong&gt; engine (SNN) from scratch, but I can't find the basic information I need to get started.&lt;/p&gt;&#xA;&lt;p&gt;For example, I've seen pictures of the individual signals that combine to form a neuron pulse in several research papers, with no information on the equations in use.  It's not the focus of the papers, and the authors assume the readers have that knowledge already.  Some papers reference software that provides this foundation (&lt;a href=&quot;https://www.nest-simulator.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEST&lt;/a&gt;, &lt;a href=&quot;https://neuralensemble.org/PyNN/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyNN&lt;/a&gt;, etc.), but the documentation for the software is similarly light on details.&lt;/p&gt;&#xA;&lt;p&gt;There is a ton of information out there on the more common network types, but SNNs have not yet made it into the mainstream.&lt;/p&gt;&#xA;&lt;p&gt;So, where do I get this basic information? Has someone pulled together any recipes/examples/tutorials for an SNN, as has been done with all the other network types?&lt;/p&gt;&#xA;" OwnerUserId="10402" LastEditorUserId="2444" LastEditDate="2021-01-15T11:02:02.103" LastActivityDate="2021-01-15T11:11:01.873" Title="What are examples of resources that describe the basics of Spiking Neural Networks in detail?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;resource-request&gt;&lt;spiking-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4378" PostTypeId="2" ParentId="4377" CreationDate="2017-10-27T17:12:52.557" Score="2" Body="&lt;p&gt;There is a book &lt;a href=&quot;http://web.archive.org/web/20171110040115/icwww.epfl.ch/%7Egerstner/BUCH.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spiking Neuron Models: Single Neurons, Populations, Plasticity&lt;/a&gt; (2002) by Wulfram Gerstner and Werner M. Kistler.&lt;/p&gt;&#xA;&lt;p&gt;I also found &lt;a href=&quot;http://web.archive.org/web/20171023113436/http://www.mjrlab.org/2014/05/08/tutorial-how-to-write-a-spiking-neural-network-simulation-from-scratch-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides&lt;/a&gt; that could be useful.&lt;/p&gt;&#xA;&lt;p&gt;(Additional resources would be appreciated.)&lt;/p&gt;&#xA;" OwnerUserId="10402" LastEditorUserId="2444" LastEditDate="2021-01-15T11:11:01.873" LastActivityDate="2021-01-15T11:11:01.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4380" PostTypeId="2" ParentId="4376" CreationDate="2017-10-27T18:29:53.893" Score="2" Body="&lt;p&gt;To answer this, it's helpful to consider the notion of a neural network architecture – in this context, we can think of the architecture as being the network depth (i.e. number of layers), width (i.e. number of nodes in a layer), and some other structural aspects, such as recurrent layers, convolution layers, pool layers, etc. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Theory&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In terms of the theory, there are a few ideas that provide insight into your question. First is the commonly cited &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt;, which states that a single hidden layer, feed forward neural network with a sigmoid activation and finite width (i.e. the number of nodes in single the hidden layer) can approximate any n-dimensional real-valued function. Superficially, this tells us that we can do a whole lot with one of the most simple architectures and could (in theory) approximate more complex architectures such as LSTMs, CNNs, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, while this seems to indicate that reductions are always possible, research on the asymmetry in representational power between depth and width hints that things may not be so simple (for example, see &lt;a href=&quot;https://papers.nips.cc/paper/7203-the-expressive-power-of-neural-networks-a-view-from-the-width.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and &lt;a href=&quot;http://proceedings.mlr.press/v49/eldan16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;). Specifically, network depth provides much greater representational power relative to width. Intuitively, this makes sense – depth allows us to build hierarchies and stack multiple non-linearities. Indeed, this is where we begin to see the outlines of what the answer looks like in practice – depth provides a clear, easy way to bake &lt;em&gt;structure&lt;/em&gt; into our network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the upshot from the theoretical perspective is that the universal approximation theorem answers in the affirmative; yes, you could reduce a complex network architecture to a single hidden layer network, but the basis for doing this is a bit naïve – it doesn't consider the efficiency of the network relative to the number of parameters.The two papers linked above imply this.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Practice&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In practice, there are issues with performing such a reduction. To date, the most successful models have deep architectures. More importantly though, many of these models are specialized – we have CNNs for computer vision, RNNs for sequence-based data, etc. These architectures were conceived as a way to integrate knowledge about the structure of the problem into the structure of the network to improve learning of the desired representation, and this too requires depth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, last year a paper took this idea to the extreme. &lt;a href=&quot;https://weightagnostic.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight agnostic neural networks&lt;/a&gt; are specifically constructed such that their architecture is in essence the representation with a single parameter (weight) that can be varied to generate multiple distinct models with different performance characteristics. The WANNs that were developed in this research weren't especially deep, but they also weren't single layered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The upshot here is that, for complex problems, depth is necessary for efficient learning and gradient computation and that it can be used to encode problem structure into a network. While it may be possible to reduce a learned representation to a single hidden layer network, it's not inconceivable that you may end up with &lt;em&gt;more&lt;/em&gt; parameters condensed into a single hidden layer with very high dimensional gradients. In terms of implementation, this isn't desirable. &lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="5210" LastEditDate="2020-02-28T18:32:37.473" LastActivityDate="2020-02-28T18:32:37.473" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4381" PostTypeId="2" ParentId="3451" CreationDate="2017-10-27T20:25:51.303" Score="1" Body="&lt;p&gt;It's already being done, and apparently with very good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;http://journals.sagepub.com/doi/abs/10.1177/2167702617691560?journalCode=cpxa&amp;amp;&quot; rel=&quot;nofollow noreferrer&quot;&gt;Predicting Risk of Suicide Attempts Over Time Through Machine Learning&lt;/a&gt;, Walsh, Ribiero, Franklin&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the abstract from the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Traditional approaches to the prediction of suicide attempts have limited the accuracy and scale of risk detection for these dangerous behaviors. We sought to overcome these limitations by applying machine learning to electronic health records within a large medical database. Participants were 5,167 adult patients with a claim code for self-injury (i.e., ICD-9, E95x); expert review of records determined that 3,250 patients made a suicide attempt (i.e., cases), and 1,917 patients engaged in self-injury that was nonsuicidal, accidental, or nonverifiable (i.e., controls). We developed machine learning algorithms that accurately predicted future suicide attempts (AUC = 0.84, precision = 0.79, recall = 0.95, Brier score = 0.14). Moreover, accuracy improved from 720 days to 7 days before the suicide attempt, and predictor importance shifted across time. These findings represent a step toward accurate and scalable risk detection and provide insight into how suicide attempt risk shifts over time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This has gotten national press attention quite recently.  Here are a couple of prior articles on the result and endeavor:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wired.com/2017/03/artificial-intelligence-learning-predict-prevent-suicide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence is Learning to Predict and Prevent Suicide&lt;/a&gt; (Wired)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://qz.com/1001968/artificial-intelligence-can-now-predict-suicide-with-remarkable-accuracy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial intelligence can now predict suicide with remarkable accuracy&lt;/a&gt; (Quartz)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is strongly suspect ML can be used in an array of application related to human mental health.  For instance, it is entirely possible a app would be able to discern the mood swings in people with bipolar disorder based on activity or lack thereof.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-10-27T20:25:51.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4384" PostTypeId="2" ParentId="3823" CreationDate="2017-10-28T23:00:54.607" Score="3" Body="&lt;p&gt;I will try to answer all three questions to the best of my ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is this basically true? Quick googling just brought me to a lot of papers trying to fit decision trees into incremental learning.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with decision trees in an online learning setting is that the model should be able to update when experiences (i.e. state, action, reward, new state) of the environment are collected. Any data that does not reflect the structure of the tree will cause the decision tree to fall apart and requires you to rebuild the model again. This is not necessarily a problem, but it is something you have to take into consideration when building a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What other algorithms fall under this category?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other algorithms which assumes that the data is known beforehand (offline). For example K-NN or Learning Vector Quantization (LVQ). However it should be noted that these algorithms can be adapted to work in an online fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are neural nets good for incremental learning? What other algorithms would be good?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks can capture changes from online experiences fairly well. The reason is that each weight in such a network is adapted when a new experience comes in. The weights are adapted with regards to an error function and this allows to update the model relatively easy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2016, Venkatesan, et al proposed a novel progressive learning technique for multi-class classification. You might want to look into this as the system that they use allows for online learning. Other techniques exists as well and should be found when googling for online learning methods. &lt;/p&gt;&#xA;" OwnerUserId="10468" LastActivityDate="2017-10-28T23:00:54.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4385" PostTypeId="1" AcceptedAnswerId="4387" CreationDate="2017-10-29T10:54:29.040" Score="7" ViewCount="1932" Body="&lt;p&gt;I am trying to train a CNN regression model using the ADAM optimizer, dropout and weight decay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My test accuracy is better than training accuracy.&#xA;But as I know, usually train accuracy is better than test accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I wonder how this is happening. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MgVBt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MgVBt.png&quot; alt=&quot;my grph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10475" LastEditorUserId="33" LastEditDate="2017-10-29T23:31:20.673" LastActivityDate="2017-11-01T14:03:50.223" Title="Why my test error is lower then train error" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4387" PostTypeId="2" ParentId="4385" CreationDate="2017-10-29T12:08:51.977" Score="5" Body="&lt;p&gt;You use dropout during traing to reduce overfitting, but this reduces the training accuracy. The dropout will not be used during testing, therefore the accuracy will be higher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's normal behavior if you work with dropout.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-10-29T12:08:51.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4388" PostTypeId="2" ParentId="3419" CreationDate="2017-10-29T13:48:13.627" Score="1" Body="&lt;p&gt;With perplexity you are trying to evaluate the similarity between the token (in your case probably sentences) distribution generated by the model and the one in the test data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, assuming you have &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; sentences &lt;span class=&quot;math-container&quot;&gt;$s_1, \dots, s_M$&lt;/span&gt;, each with probability &lt;span class=&quot;math-container&quot;&gt;$P(s_i)$&lt;/span&gt;, the perplexity is &lt;span class=&quot;math-container&quot;&gt;$$2^{-l},$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$l = \frac{1}{M} \sum P(s_i) \log P(s_i)$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i \in [1 \dots M]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that while perplexity might be useful to capturing certain aspects of the model, it is by no means perfect, and, even if you are able to reach great perplexity scores, it will not necessarily translate to a good or even working chat bot.&lt;/p&gt;&#xA;" OwnerUserId="10481" LastEditorUserId="2444" LastEditDate="2019-02-26T14:54:25.323" LastActivityDate="2019-02-26T14:54:25.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4389" PostTypeId="1" CreationDate="2017-10-29T14:33:06.343" Score="11" ViewCount="188" Body="&lt;p&gt;I'm trying to find a planning approach to solve a problem that attempts to model learning of new material. We assume that we only have one resource such as Wikipedia, which contains a list of articles represented as a &lt;strong&gt;vector&lt;/strong&gt; of knowledge it contains and an &lt;strong&gt;effort&lt;/strong&gt; to read that article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Knowledge vector and effort&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before we start, we set a size for the vector, depending on the number of different types of knowledge. For example, we can define the items in the vector to be &lt;code&gt;(algebra, geometry, dark ages)&lt;/code&gt;, and then 'measure' all the articles from this point of view. So, a math article will probably be &lt;code&gt;(5,7,0)&lt;/code&gt;, since it will talk a lot about algebra and geometry but not about the dark ages. It will also have an &lt;strong&gt;effort&lt;/strong&gt; to read it, which is simply an integer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given all the articles (represented as knowledge vectors with an effort), we want to find the optimal set of articles that help us to reach a &lt;strong&gt;knowledge goal&lt;/strong&gt; (also represented as a vector). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, a knowledge goal can be &lt;code&gt;(4,4,0)&lt;/code&gt;, and it's enough to read an article &lt;code&gt;(2,1,0)&lt;/code&gt; and &lt;code&gt;(2,3,0)&lt;/code&gt;, since, when added, it adds up to the knowledge goal. We want to do this with &lt;strong&gt;minimal effort&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to some heuristics to find an approximation, but I was wondering if there is any state of the art strategic planning method that can be used instead?&lt;/p&gt;&#xA;" OwnerUserId="11365" LastEditorUserId="1671" LastEditDate="2017-10-31T17:49:21.880" LastActivityDate="2019-04-30T11:02:05.697" Title="Strategic planning and multi dimensional knapsack problem" Tags="&lt;research&gt;&lt;heuristics&gt;&lt;knapsack-problem&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4390" PostTypeId="2" ParentId="4385" CreationDate="2017-10-29T20:16:03.163" Score="0" Body="&lt;p&gt;Beyond dropout (as @demento already explained) if you are training with data augmentation this is an expected behavior.&lt;/p&gt;&#xA;" OwnerUserId="9794" LastActivityDate="2017-10-29T20:16:03.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4391" PostTypeId="1" AcceptedAnswerId="4411" CreationDate="2017-10-30T02:13:23.367" Score="4" ViewCount="148" Body="&lt;p&gt;I have set myself the challenge of detecting the locations of players/bots in videos of a well known first person shooter game (this is for a youtube series I'm planning on doing). I'm not sure which AI approach I should apply to this problem - I'm a complete novice at this!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought was that the face/head seems to have the most detail so I could train a convolution neural network on images of sprite heads and general background - however this seem not to work too well, I've certainly not exhausted different network architectures/typologies but it wasn't learning all that well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second approach was to use a HAAR cascade. This seemed to be an obvious choice since it's fast and good at detecting objects (rather than multi-classifying). However my cascade stops after 5 or 6 stages (using OpenCV) as it seems to have reach a great accuracy, but it doesn't detect when I feed it the training images, let alone other images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also looked into pedestrian detection and got a stock version of that working. However this seemed to struggle when/if the sprites are crouching or in unusual positions (and it isn't great on standing sprites tbh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there a branch of machine learning/AI that is more applicable to this problem? If not which should I continue to work on?&lt;/p&gt;&#xA;" OwnerUserId="10498" LastEditorUserId="10498" LastEditDate="2017-11-03T00:17:39.493" LastActivityDate="2018-03-01T06:25:38.560" Title="What approach should I use to detect faces in video game footage?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4392" PostTypeId="2" ParentId="4385" CreationDate="2017-10-30T03:08:04.113" Score="0" Body="&lt;p&gt;Also, it really depends how uniform your test data is distributed as well as its size. If you have relevantly so small test dataset compared to training one, it may be an expected output. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-10-30T03:08:04.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4393" PostTypeId="2" ParentId="3290" CreationDate="2017-10-30T22:28:14.747" Score="2" Body="&lt;p&gt;You either need a generative model or an emulator of the environment. In the later case you don't calculate your transitions and rewards using the model but feed your actions and states to the emulator and work with the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The emulator can be a black box as long as it returns the next state and the reward when provided with the current state and an action. You also need a way to identify all legal actions in a given state to build the tree.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-10-30T22:28:14.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4394" PostTypeId="1" AcceptedAnswerId="4654" CreationDate="2017-10-30T23:01:56.720" Score="10" ViewCount="921" Body="&lt;p&gt;&lt;a href=&quot;https://deepmind.com/blog/alphago-zero-learning-scratch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero&lt;/a&gt; contains several improvements compared to its predecessors. Architectural details of Alpha Go Zero can be seen in this &lt;a href=&quot;https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;cheat sheet&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of those improvements is using a single neural network that calculates move probabilities and the state value at the same time, while the older versions used two separate neural networks. It has been shown that the merged neural network is more efficient according to the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It uses one neural network rather than two. Earlier versions of AlphaGo used a “policy network” to select the next move to play and a ”value network” to predict the winner of the game from each position. These are combined in AlphaGo Zero, allowing it to be trained and evaluated more efficiently.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems counter intuitive to me, because from a software design perspective this violates the principle &lt;a href=&quot;https://en.wikipedia.org/wiki/Separation_of_concerns&quot; rel=&quot;nofollow noreferrer&quot;&gt;separation of concerns&lt;/a&gt;. That's why I am wondering, why this merge has been proven beneficial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this technique - merging different tasks in a single neural network to improve efficiency - be applied to other neural networks in general or does this require certain conditions to work?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="2444" LastEditDate="2019-08-11T14:57:47.663" LastActivityDate="2019-08-11T14:57:47.663" Title="Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;architecture&gt;&lt;alphago-zero&gt;&lt;efficiency&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4395" PostTypeId="1" CreationDate="2017-10-31T00:44:22.317" Score="5" ViewCount="163" Body="&lt;p&gt;I want to design a simple model that predicts the movement of coordinates with RNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a typical three-dimensional LSTM model, one feature is encoded as one hot encoding, and the &lt;code&gt;x&lt;/code&gt; value is input as a three-dimensional matrix.&#xA;(e.g., In &lt;code&gt;seq2seq&lt;/code&gt;, &lt;code&gt;abc =&amp;gt; [[[1,0,0],[0,1,0],[0,0,1]]]&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since I have to predict the &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; coordinate values, one hot encoding will inevitably produce a four-dimensional matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do with back-propagation? Or should I design a 3-D matrix using scalar values without one hot encoding?&lt;/p&gt;&#xA;" OwnerUserId="10522" LastEditorUserId="22296" LastEditDate="2019-02-20T14:59:14.613" LastActivityDate="2019-02-20T14:59:14.613" Title="How to design 4D Deep Recurrent Neural Networks using Tensorflow?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4396" PostTypeId="1" AcceptedAnswerId="5807" CreationDate="2017-10-31T01:26:08.780" Score="5" ViewCount="1762" Body="&lt;p&gt;I'm trying to solve problem 6.6 from the book &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;&lt;/em&gt;, by Peter Norvig and Stuart Russell.&lt;/p&gt;&#xA;&lt;p&gt;This is in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_satisfaction_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Constraint Satisfaction Problem&lt;/a&gt; and how you can re-formulate some problems with the constraints expressed as a bunch of binary constraints to use the generalized solver CSP algorithm.&lt;/p&gt;&#xA;&lt;p&gt;But I'm stuck with that exercise, I can't sketch a demonstration.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;6.6  Show how a single ternary constraint such as &amp;quot;A + B = C&amp;quot; can be turned into three binary constraints by using an auxiliary variable. You may assume finite domains.&lt;/p&gt;&#xA;&lt;p&gt;(Hint: Consider a new variable that takes on values that are pairs of others values, and consider constraints such as &amp;quot;X is the first element of the pair Y.&amp;quot;) Next, show how constraints can be eliminated by altering the domain of variables. This completes the demonstration that any CSP can be transformed into a CSP with only binary constraints.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9574" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-12T18:00:03.847" Title="How to turn a ternary constraint into three binary constraints?" Tags="&lt;ai-design&gt;&lt;constraint-satisfaction-problems&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4397" PostTypeId="1" CreationDate="2017-10-31T10:05:28.143" Score="2" ViewCount="115" Body="&lt;p&gt;I want to develop a virtual assistant like Google Assistant, Alexa, etc., for mobile phones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I should be researching apart from M/C Learning and Python? &lt;/p&gt;&#xA;" OwnerUserId="10529" LastEditorUserId="1671" LastEditDate="2017-10-31T17:36:09.223" LastActivityDate="2018-06-01T21:21:19.477" Title="AI techniques and competencies necessary for creating a Virtual Assistant" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;research&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ClosedDate="2020-05-15T23:00:30.660" ContentLicense="CC BY-SA 3.0" />
  <row Id="4398" PostTypeId="1" CreationDate="2017-10-31T10:05:44.557" Score="2" ViewCount="524" Body="&lt;p&gt;I'm having the following problem: `&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm training a multi-output CNN and using the relative values of the outputs in my loss function. The net is learning well, but as the absolute values of the outputs are not regularized in anyway in the loss function, the values of the outputs keep rising. This causes a situation where the result I need (values of the outputs relative to each other) are quite right, but the huge absolute values produce underflow resulting NaN values at some point of the training. Is there some way to constrain these absolute values (perhaps elsewhere than the loss function?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using a custom loss function implementing a recovery angular error metric, so the loss function is somewhat complex. A weighed mean of the net outputs is fed to this function, so using both the net outputs and the weighed mean would lead to some quite problematic gradient derivation. Would it be maybe possible to constrain the value range of the net outputs in the net structure itself?&lt;/p&gt;&#xA;" OwnerUserId="10528" LastActivityDate="2018-06-02T19:00:36.053" Title="Constraining the output value range of a CNN independent of the loss function" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4399" PostTypeId="1" CreationDate="2017-10-31T10:24:42.340" Score="3" ViewCount="1678" Body="&lt;p&gt;So, my question is a bit theoretical. I have been trying to implement a perceptron based classifier with outputs 1 and 0 depending on  the category. I have used 2 methods: The &lt;code&gt;example by Example learning method&lt;/code&gt; and &lt;code&gt;Batch learning method&lt;/code&gt;. I also have defined another method which will measure accuracy according to the formulae &lt;code&gt;number_of_samples_classified_correctly/total_number_of_samples&lt;/code&gt;(I'm not sure this should be the correct definition for accuracy and you are welcome to suggest a better measure). Now there are a few confusions i'm facing. Firstly, the accuracy of example by example learning is different from batch learning by 2%. Also the &lt;strong&gt;best&lt;/strong&gt; accuracy achieved in both cases is depending on the slopes. So where exactly is the mistake?(Batch learning algorithm=&lt;code&gt;error*input_vector&lt;/code&gt;( where error can be 1,-1 or 0 ) summed over all input vectors and then added to weights).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jla9w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jla9w.png&quot; alt=&quot;For initial slope[1,-1] giving an accuracy of 88% example by example learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DejqO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DejqO.png&quot; alt=&quot;For initial slope[1,-1] giving an accuracy of 88% batch learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gaKBn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gaKBn.png&quot; alt=&quot;For initial slope[1,1] giving an accuracy of 84% example by example learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cDryg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cDryg.png&quot; alt=&quot;For initial slope[1,1] giving an accuracy of 88% batch learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For initial slope[1,-1] giving an accuracy of 88% example by example learning&lt;/li&gt;&#xA;&lt;li&gt;For initial slope[1,-1] giving an accuracy of 88% batch learning&lt;/li&gt;&#xA;&lt;li&gt;For initial slope[1,1] giving an accuracy of 84% example by example learning&lt;/li&gt;&#xA;&lt;li&gt;For initial slope[1,1] giving an accuracy of 86% batch learning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="6258" LastEditDate="2017-10-31T16:43:17.867" LastActivityDate="2017-11-05T08:34:36.660" Title="Perceptron learning algorithm: different accuracies for different training methods" Tags="&lt;classification&gt;&lt;perceptron&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4400" PostTypeId="2" ParentId="4399" CreationDate="2017-10-31T13:42:37.767" Score="2" Body="&lt;p&gt;&lt;strong&gt;In Brief:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;re-train your dataset. I believe where you get lower accuracy scores, your model has not converged to the final state. duplicate your dataset multiple times and create a bigger one, then train your model with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In Detail:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;number_of_samples_classified_correctly/total_number_of_samples(I'm not sure this should be the correct definition for accuracy and you are welcome to suggest a better measure)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a valid accuracy metric. In fact if the value is &lt;code&gt;acc&lt;/code&gt;, then &lt;code&gt;1-acc&lt;/code&gt; is called &lt;em&gt;misclassification error&lt;/em&gt;. So your metric is good unless you have some class imbalance, where you need to use other metrics such as &lt;em&gt;Cohen's Kappa score&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the accuracy of example by example learning is different from batch learning by 2%. Also the best accuracy achieved in both cases is depending on the slopes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I strongly believe both strange results happen because the number of your instances (i.e. examples) is low, or let's say your &lt;em&gt;learning rate&lt;/em&gt; is small. As you know Perceptron algorithm corrects the weights of the decision hyper-plane by &lt;em&gt;delta learning rule&lt;/em&gt;: it reads each instance, calculates the error (in case of binary classification &lt;code&gt;{-1,0,1}&lt;/code&gt;) and updates the weights by &lt;code&gt;c.x.E&lt;/code&gt; where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;c is learning constant/rate/step&lt;/li&gt;&#xA;&lt;li&gt;x is the data instance&lt;/li&gt;&#xA;&lt;li&gt;E is the error&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus the weights (or slopes) change a little bit every time. There is no guarantee that your model reaches its final states after all the instances are given to it. Since when you start training with different initial weights, or change the order of instances given to your model, the distance or from or moving speed toward the final state changes. So as I mentioned above, re-train your model and let it converge to the final state &lt;strong&gt;OR&lt;/strong&gt; increase your learning step (learning rate) . I believe all the 4 mentioned-above-accuracies will be the same then; I mean not only you should have the same model for different initial weights, but also for batch and online training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please update your post with new findings and question my claims if those were false.&lt;/p&gt;&#xA;" OwnerUserId="6258" LastEditorUserId="6258" LastEditDate="2017-10-31T13:47:39.890" LastActivityDate="2017-10-31T13:47:39.890" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4401" PostTypeId="2" ParentId="4375" CreationDate="2017-10-31T14:25:07.227" Score="13" Body="&lt;p&gt;Sophia uses &lt;a href=&quot;https://arxiv.org/abs/1709.07791&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChatScript&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read about what ChatScript can do &lt;a href=&quot;https://github.com/ChatScript/ChatScript/blob/master/PDFDOCUMENTATION/OVERVIEWS-AND-TUTORIALS/What-is-ChatScript.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ChatScript keeps track of conversations with each user; can record&#xA;  where it is in a conversational flow and what facts it has learned&#xA;  about a user (you have to tell it what facts to try to learn). You can&#xA;  optionally keep logs of the conversations (either on a ChatScript&#xA;  server or locally on a freestanding device).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10534" LastEditorUserId="2444" LastEditDate="2020-04-11T02:36:18.970" LastActivityDate="2020-04-11T02:36:18.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4402" PostTypeId="2" ParentId="3089" CreationDate="2017-10-31T17:25:50.370" Score="7" Body="&lt;p&gt;First, we need to talk about transfer learning. Imagine you trained a neuronal network over a dataset of images to detect cats, you can use part of the training you have done to work over another detecting something else. That's known as transfer learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do transfer learning, you will remove the last fully connected layer from the model and plug in your layers there. The &quot;truncated&quot; model output is going to be the features that will fill your &quot;model&quot;. Those are the bottleneck features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VGG16 is a pretrain-model over ImageNet catalog that has very good accuracy. In the post you shared, is using that model as a base to detect cat and dogs with a higher accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottleneck features depends on the model. In this case, we are using VGG16. There are others pre-trained models like VGG19, ResNet-50&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's like you are cutting a model and adding your own layers. Mainly, the output layer to decide what you want to detect, the final output.&lt;/p&gt;&#xA;" OwnerUserId="10540" LastActivityDate="2017-10-31T17:25:50.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4403" PostTypeId="5" CreationDate="2017-10-31T17:43:27.610" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-31T17:43:27.610" LastActivityDate="2017-10-31T17:43:27.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4404" PostTypeId="4" CreationDate="2017-10-31T17:43:27.610" Score="0" Body="For references to &quot;Artificial Intelligence: A Modern Approach&quot; and the work of Russell &amp; Norvig in general." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-31T17:43:27.610" LastActivityDate="2017-10-31T17:43:27.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4405" PostTypeId="5" CreationDate="2017-10-31T17:50:30.043" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Knapsack_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Knapsack_problem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-31T17:50:30.043" LastActivityDate="2017-10-31T17:50:30.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4406" PostTypeId="4" CreationDate="2017-10-31T17:50:30.043" Score="0" Body="Refers to the &quot;knapsack&quot; or &quot;rucksack&quot; problem in combinatorial optimization." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-31T17:50:30.043" LastActivityDate="2017-10-31T17:50:30.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4407" PostTypeId="2" ParentId="3451" CreationDate="2017-10-31T21:37:07.553" Score="0" Body="&lt;p&gt;I'll put my two pennies worth in. I welcome an attempt by AI to diagnose mental health issues and perhaps prevent outcomes of suicide by sufferers. Speaking with knowledge of the subject it's very hard for a human to precisely determine whether someone is suffering from mental illness and it's diagnosis currently is limited to placing someone within a broad spectrum with no absolute determination of the truth of the matter. There is currently no physical manner either in making this determination and is down to communication which I think is partly down to the dualistic nature of the situation of body and mind and this is a barrier that will be very difficult to overcome by Physical algorithms.&lt;/p&gt;&#xA;" OwnerUserId="9094" LastActivityDate="2017-10-31T21:37:07.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4409" PostTypeId="1" CreationDate="2017-11-01T00:09:06.357" Score="3" ViewCount="51" Body="&lt;p&gt;A question for developers of projects for pattern recognition. How best to organize the architecture of such a service?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At what stage do you conduct logic? (for example, for the recognition of a photo of a male blue jacket, a cascade of queries is performed: &quot;recognizing men&quot; -&gt; &quot;recognizing the jacket&quot; -&gt; &quot;recognizing the color of the jacket.&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make sense to implement all search options within a single neural network or is it better to create a set of individual neuronets that are confined to fairly simple tasks?&lt;/p&gt;&#xA;" OwnerUserId="7051" LastActivityDate="2017-11-05T13:40:10.120" Title="Image recognition service architecture" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;pattern-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4410" PostTypeId="2" ParentId="4409" CreationDate="2017-11-01T08:41:34.787" Score="2" Body="&lt;p&gt;That is one of the good example for research. Personally, I prefer to segment out all the desired outputs at once. Then, check the success rate. If you cannot hit the success rate that you desire, you can go for more specific solutions for the specific problem that you face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in general, the localization, segmentation, recognition are implemented in same network and are obtained all-at-once. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-01T08:41:34.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4411" PostTypeId="2" ParentId="4391" CreationDate="2017-11-01T08:49:27.910" Score="4" Body="&lt;p&gt;At first, you can find lots of information as pedestrian detection. As you are trying to localize game characters, the face is not the best option. You need to look for the character in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About HAAR Cascades, the algorithm is one of the fastest face localization solutions in the market. The reason is, it applies all the feature classifications layer by layer, by starting from the wider feature. So, if it fails, it does not spend the time to check computationally intensive features. It was good until DNN overcome its success rate. However, it is not the best approach for recognizing game characters/pedestrians.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it seems like you are overfitting the cascade so it stops and does not learn anything valuable. You can search methods about how to reduce overfit problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2005, a new method has been proposed, HOG (Histogram Of Gradients). You can use this one and classify output features to get what you desire. If you would like to go for deep learning version, I would suggest you to investigate how DNN is working, which kind of input images you need, what are the localization networks (i.e. YOLO, Faster R-CNN), how they are working.&lt;/p&gt;&#xA;" OwnerUserId="10344" LastEditorUserId="10344" LastEditDate="2018-03-01T06:25:38.560" LastActivityDate="2018-03-01T06:25:38.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4412" PostTypeId="2" ParentId="4283" CreationDate="2017-11-01T08:58:00.937" Score="1" Body="&lt;p&gt;Definitely. NNs can learn the data that you teach to them. If you teach them biased, the network will be biased. As you mention, one solution is to reduce the data that you have. However, it is not the best approach as you will be losing the precious data. I would suggest to try data augmentation for remaining dataset to increase missing data type samples and have similar numbers for best, evenly distributed accuracy. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-01T08:58:00.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4413" PostTypeId="2" ParentId="4385" CreationDate="2017-11-01T13:30:12.533" Score="3" Body="&lt;p&gt;I will just add to all the good answers already here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like I said on my comment earlier, this is not a bad this(provided you have a split your data correctly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other reasons could be: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;High dropout rate or excessive data augmentation could be one of the reason. This can cause the training accuracy to appear low whist in reality the model is in fact learning. How so, you ask? Regularization mechanisms, such as Dropout and L1/L2 weight regularization, are turned off at testing time. Recall that every score you are seeing on the training set is being calculated on a &quot;different&quot; set of weights, ie a different network. So the accuracy you are seeing is the accuracy of all those networks your network tried before it settled on the one you are now testing on the test set. The reasoning here being that the other networks were just very &quot;bad&quot; compared to the final one that you are now using to test on the test set. If this is your case then, you are just witnessing neural network dropout and data augmentation doing exactly what they are meant to do, i.e avoid dropout. &lt;/li&gt;&#xA;&lt;li&gt;Another possible reason for this could be that your test set is different and simpler in comparison to the training set. That is to say: your data split could be such that you have a simpler test set than a train set and hence your network seems to be doing far much better on the test set than it did on the training set. If this is the case then good job, you have build yourself a very good model, no need to stress, but be careful because with such &quot;bad&quot; data splitting,  future you might not be as lucky :).&lt;/li&gt;&#xA;&lt;li&gt;Make sure you are doing the correct pre-processing.&lt;/li&gt;&#xA;&lt;li&gt;Your test set might be very small as such, the high accuracy you are seeing on it is not indicative of the real accuracy you would see if you had a larger test set. If this is the case then, you may want to collect more test data and test your model again. &lt;/li&gt;&#xA;&lt;li&gt;Another reason could be than your training set is small and the high training error you are seeing there is not &quot;real&quot;. This bring up the question of how big your training set it? &lt;/li&gt;&#xA;&lt;li&gt;To quote &lt;a href=&quot;https://keras.io/getting-started/faq/#why-is-the-training-loss-much-higher-than-the-testing-loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Keras documentation&lt;/a&gt;:&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the training loss is the average of the losses over each batch of training data. Because your model is changing over time, the loss over the first batches of an epoch is generally higher than over the last batches. On the other hand, the testing loss for an epoch is computed using the model as it is at the end of the epoch, resulting in a lower loss.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;Lastly you may find this &lt;a href=&quot;https://github.com/fchollet/keras/issues/1761&quot; rel=&quot;nofollow noreferrer&quot;&gt;github postenter link description here&lt;/a&gt; useful.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="7550" LastEditDate="2017-11-01T14:03:50.223" LastActivityDate="2017-11-01T14:03:50.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4414" PostTypeId="1" CreationDate="2017-11-02T06:33:19.507" Score="14" ViewCount="1548" Body="&lt;p&gt;As you might know, &lt;a href=&quot;https://arxiv.org/abs/1710.09829v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;capsule networks&lt;/a&gt; have been recently introduced by &lt;a href=&quot;https://en.wikipedia.org/wiki/Geoffrey_Hinton&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hinton&lt;/a&gt;. There also have been several heads up within his &lt;a href=&quot;https://www.youtube.com/watch?v=TFIMqt0yT2I&quot; rel=&quot;nofollow noreferrer&quot;&gt;talks&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As expected, the paper elaborates on the idea way theoretically! However, as a fan of &lt;a href=&quot;https://en.wikipedia.org/wiki/Occam%27s_razor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Occam's razor&lt;/a&gt;, I was wondering if anybody can simplify the idea behind the capsule neural networks. &lt;/p&gt;&#xA;" OwnerUserId="10569" LastEditorUserId="2444" LastEditDate="2020-06-09T11:16:30.060" LastActivityDate="2020-06-09T11:16:30.060" Title="What's the main concept behind capsule networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;capsule-neural-network&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2017-11-12T16:21:31.413" ContentLicense="CC BY-SA 4.0" />
  <row Id="4415" PostTypeId="1" CreationDate="2017-11-02T09:43:49.843" Score="4" ViewCount="2255" Body="&lt;p&gt;I have a simple gauge displaying analog values ranging from 0 to 4.&#xA;&lt;a href=&quot;https://www.fragdenstein.de/wordpress/wp-content/uploads/2014/10/heizung_wasser_auffuellen_beitragsbild.jpg&quot; rel=&quot;noreferrer&quot;&gt;Here is an image of the gauge&lt;/a&gt;. Unfortunately there is no way to get a analog or digital signal for the value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I read the value of the gauge? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to make an image every 5 minutes and get the value by analyzing it. I am thinking of manually generating reference images with the black needle in different positions and then compare it to the real image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since all the processing should be done on a raspberry pi without internet connectivity, a good approach would be a preconfigured docker image for image comparison which helps doing the image comparison locally, maybe supported by a python or php script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to proceed?&lt;/p&gt;&#xA;" OwnerUserId="10586" LastActivityDate="2017-11-02T20:12:08.617" Title="Reading a value of a real gauge" Tags="&lt;image-recognition&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4418" PostTypeId="2" ParentId="4415" CreationDate="2017-11-02T20:12:08.617" Score="1" Body="&lt;p&gt;This is a simple &lt;a href=&quot;https://en.wikipedia.org/wiki/Digital_image_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;digital image processing&lt;/a&gt; task. I am proposing a quite efficient solution without any error correction mechanism, so this is not a production quality approach. But as a first proof of concept it should do the trick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest approach would be splitting the image in two channels - one for the color red and one for the color black. You can neglect the rest outside a certain threshold. If the lighting is good, this should also deal with the background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 1: Crop away everything outside the gauge to simplify your life. The easiest way to do this is just putting the camera close enough that it only captures the gauge. That way you don't have to implement it in your algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 2: The center of your image will be the red dot in the middle of the gauge. You can easily identify it with a filter looking for the largest concentration of red in the red channel of your image. Calculate the center and remember the coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 3: Do the same for the largest concentration of black in the black channel. This will be the thick end part of the black needle. Calculate the center as well and remember the coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 4: Calculate the vector between those two coordinates. The angle of the vector will tell you the angle of the needle (take care of the direction of the vector). Once you have the angle of the needle, you can easily derive the number it is pointing to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are obviously much more elegant approaches to solve this problem, but this quick and dirty algorithm should do the trick without a lot of hassle. You need to make sure that the camera is positioned in the right angle, because we haven't implemented any error corrections in our algorithm. You also need to make sure the lighting is sufficient and that there aren't any other significant influences on the image quality. As long as the orientation is stable and the image quality high, this should enable you to read the gauge with good precision.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-11-02T20:12:08.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4419" PostTypeId="1" CreationDate="2017-11-02T20:17:38.433" Score="2" ViewCount="54" Body="&lt;p&gt;I think that my consciousness and intelligence remains located inside me, in my body or brain.&lt;/p&gt;&#xA;&lt;p&gt;I know that I am a stalwart of a dualistic nature: mind and material.&lt;/p&gt;&#xA;&lt;p&gt;But let's suppose that an artificial intelligence can be created purely from the material (e.g. by programming a neural network).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Where will the consciousness (and intelligence) of this AI be within it: its RAM, its processor, or will the programming itself create a bi-location with no precise boundary or absolute position within the machine?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="9094" LastEditorUserId="2444" LastEditDate="2020-09-17T15:11:11.457" LastActivityDate="2020-09-17T15:11:11.457" Title="Where would consciousness reside in an artificial intelligence?" Tags="&lt;philosophy&gt;&lt;artificial-consciousness&gt;&lt;intelligence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4420" PostTypeId="2" ParentId="4419" CreationDate="2017-11-02T21:49:17.340" Score="1" Body="&lt;p&gt;There are still many different interpretations of the terms intelligence and consciousness. For the sake of this discussion I would define intelligence as the problem solving capabilities and consciousness as the state of being aware of and responsive to one's surroundings, including the experience of qualia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand your question correctly, you are focused on consciousness and not intelligence (based on the definitions above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of humans - the only species where we have first hand knowledge about what consciousness feels like - it feels like the consciousness resides inside the respective body. Most people experience it like &quot;being&quot; inside their head. Although debated, consciousness is most likely a simulation created by the human brain. Qualia is heavily - if not exclusively - influenced by the sensory input from the human senses. Therefore it only makes sense that it feels like the consciousness resides in the center of the sensors (eyes, ears, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Science hasn't figured out the nature of consciousness yet, so there is a lot of speculation involved. Applying above this logic to artificial consciousness and qualia, it would make sense that the machine will feel like being at the location if its sensors as well. If there is no dominant area of sensors, maybe a distributed system, it might be a split consciousness spread across several locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the creation of consciousness happens accidentally - like it did during evolution - emergence at the sensors feels most likely. Should the creator of artificial consciousness do it on purpose, he should be able to shape the simulation in other ways as he sees fit.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-11-02T21:49:17.340" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4421" PostTypeId="1" CreationDate="2017-11-02T23:59:09.157" Score="1" ViewCount="108" Body="&lt;p&gt;I'm working on a multi-player game that involves a board/map where there are walls, cover or open ground. The players then take turns moving units around and undertaking discrete actions, like move, fire etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been exploring adding an AI for single-player games, or to enhance multi-player games. I've looked at:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Behaviour Trees&lt;/li&gt;&#xA;&lt;li&gt;Utility-based decisions&lt;/li&gt;&#xA;&lt;li&gt;GOAP&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem I see with all of these is that in order for the AI to use specific multi-unit tactics, like a pincer attack, or leap-frogging units; that behaviour needs to be explicitly added separate to the above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how can a Behaviour Tree tell an AI how to identify it should use a Flanking manoeuvre, i.e. move a unit around to hit from the side? How does the AI identify that it's a good plan, that is action across multiple &quot;moves&quot;? How can the AI identify where to move the individual units in order to pull this off (how does it identify the flank)?&lt;/p&gt;&#xA;" OwnerUserId="2819" LastActivityDate="2017-12-04T19:29:42.040" Title="How can an AI make tactical manoeuvres?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4423" PostTypeId="1" CreationDate="2017-11-03T06:30:25.853" Score="3" ViewCount="122" Body="&lt;p&gt;I bought an Intel Movidius Neural Compute stick a few weeks ago. Even though I can use it with the examples, I want to actually use it for something! The documentation is messy, and hard to work through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already looked through some of the websites that have been mentioned in other posts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I specifically want to make an inference/reward-based AI that tries to guess a bunch of numbers over a period of time. The AI has to guess the best 10 guesses, and then it gets shown the set. The closer answers are rewarded, and the farther answers are penalised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, I want it to try and guess the most likely set of numbers in a random number generator, kind of like how a quantum computer outputs answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be used for all sorts of things! Even if it's already built, I want to program it from scratch so I can learn how this sort of thing works (teach a man to fish type of thing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, considering my goals above, how should I start? Also, could this answer questions with close to the same accuracy as a quantum computer, if it's fed enough data?&lt;/p&gt;&#xA;" OwnerUserId="10609" LastActivityDate="2018-08-08T16:16:58.627" Title="Programming an inference AI that computes the best outcomes like a quantum computer" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;quantum-computing&gt;&lt;expert-system&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4424" PostTypeId="1" CreationDate="2017-11-03T08:58:53.623" Score="2" ViewCount="38" Body="&lt;p&gt;Is there a general adversarial network that can take multiple low quality images of a subject to create a higher quality image of the subject? SRGANS just take a single low res image and make it high res but I need something that can take multiple low quality (not necessarily low res) images to create a higher quality image. In low quality I mean you could have one image with a reflection or partially blurry or has a small obstruction that another image might compensate for. Would be good is if you could combine the lower quality images to create on higher quality image. Is there any GAN that could do this?&lt;/p&gt;&#xA;" OwnerUserId="2788" LastActivityDate="2017-11-03T08:58:53.623" Title="Is there a general adversarial network that can take multiple low quality images to create a higher quality image?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4425" PostTypeId="1" AcceptedAnswerId="4426" CreationDate="2017-11-03T12:56:27.813" Score="5" ViewCount="1073" Body="&lt;p&gt;I am fairly new to deep learning in general and I am currently facing a problem I want to solve using neural networks and I am unsure if it is a &lt;em&gt;classification&lt;/em&gt; or &lt;em&gt;regression&lt;/em&gt; problem. I am aware that classification problems are about classifying whether an input belongs to class A or class B (or class C ...) and regression problems are about mapping the input to some sort of continuous output (just like the house pricing problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I basically want to measure the body temperature of a person using a simple video camera. To me, this seems like more of a &lt;em&gt;regression&lt;/em&gt; type of issue rather than &lt;em&gt;classification&lt;/em&gt;, because of the actual continuous result values I want the neural network to produce from the input video frames, e.g. 39°Celsius. But a question that came to my mind was: What if I use every integer value in the range from 35°C to 42°C as a possible output class? This would make it a classification problem, am I right? What would be the correct approach here and why? Classification or regression?&lt;/p&gt;&#xA;" OwnerUserId="10617" LastEditorUserId="2444" LastEditDate="2020-04-27T01:31:01.373" LastActivityDate="2020-04-27T01:31:01.373" Title="Should the prediction of the body temperature given a camera image be modelled as classification or regression?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;classification&gt;&lt;regression&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4426" PostTypeId="2" ParentId="4425" CreationDate="2017-11-03T21:30:07.723" Score="4" Body="&lt;p&gt;I think it depends on you application and what data you have available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the prediction of body temperature itself doesn't have to be accurate and classes like COLD, NORMAL, and HOT will suffice, you should stay with a classification. There isn't a cut off but as you increase the number of classes that represent numbers on the same scale, it may become more difficult to interpret the result as there will be a distribution across the classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose regression on the other hand, you are not restricted by your classes anymore and may be able to tell the difference between 36.5 and 36C which (according to wikipedia) can be &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Human_body_temperature&quot; rel=&quot;nofollow noreferrer&quot;&gt;the difference between normal and cold&lt;/a&gt;. This is something classes may not be able to capture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to consider is what your training data looks like and how accurately you want to predict the temperature. If you have pictures of people and a temperature reading, where was the reading taken and how accurate is the reading? If it isn't accurate (+- 1 degree) you may not be able to give as accurate predictions as you would like and may only be able to do 3 different classes like above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have a data set, then that is another problem altogether and might require another question as it depends on your application. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that your problem is interesting and I hope this can help you to understand how best to apply deep learning to it. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-11-03T21:30:07.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4427" PostTypeId="2" ParentId="3548" CreationDate="2017-11-03T22:27:20.520" Score="2" Body="&lt;p&gt;Artificial Intelligence is a very broad field and therefore things will change accordingly.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;Some Prerequisites:&lt;/strong&gt; (Being a student of CS you should have fulfilled them)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sound knowledge of algorithms and Data Structures. This skill will come in handy while solving problems that require use of alpha-beta pruning, minimax algorithm, etc.&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;Basic knowledge of programming languages like Java, Python. Python will help as it focuses more on the development part. For more info read &lt;a href=&quot;https://ai.stackexchange.com/questions/3494/why-is-python-the-most-popular-language-in-the-ai-field/3503#3503&quot;&gt;this&lt;/a&gt;. Knowledge of LISP will be very helpful. Go through this &lt;a href=&quot;https://ai.stackexchange.com/questions/2236/why-is-lisp-such-a-good-language-for-ai/2238#2238&quot;&gt;answer&lt;/a&gt;.&lt;br&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The book, Artificial Intelligence: A Modern Approach (by Stuart J. Russell and Peter Norvig) is considered the Bible of AI. I &lt;strong&gt;strongly&lt;/strong&gt; recommend you to read the complete book and solve the exercises. You can find the pdf of the book &lt;a href=&quot;http://web.cecs.pdx.edu/~mperkows/CLASS_479/2017_ZZ_00/02__GOOD_Russel=Norvig=Artificial%20Intelligence%20A%20Modern%20Approach%20(3rd%20Edition).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. For solution manual visit this &lt;a href=&quot;http://www.learngroup.org/uploads/2015-04-01/Solution_Manual_Artificial_Intelligence_A_Modern_Approach.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. It will be better if you can buy a hardcopy of the book.&lt;br&gt;&lt;br&gt;&#xA;Knowledge of Computational Theory will greatly help you. Especially when you are working in the field of Natural Language Processing. Other sub-fields of AI that might interest you will be Machine Learning, Evolutionary Computing, Genetic Algorithms, Reinforcement Learning, Deep Learning etc. The list goes on.&lt;br&gt;&#xA;Better your knowledge in Statistics, better it will be for Artificial Intelligence. Stay tuned to recent goings in the field via forums, websites, etc. &lt;a href=&quot;https://openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open AI&lt;/a&gt; website is also a very good source.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-11-03T22:27:20.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4428" PostTypeId="1" CreationDate="2017-11-04T02:44:36.640" Score="9" ViewCount="3948" Body="&lt;p&gt;I was reading the paper &lt;a href=&quot;https://arxiv.org/pdf/1710.09829.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing Between Capsules&lt;/a&gt; and didn't understand the term &quot;activity vector&quot; in the abstract.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I thought a vector is like an array of data that you are running through the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started working through Andrew Ng's deep learning course, but it's all new and terms go over my head.&lt;/p&gt;&#xA;" OwnerUserId="10623" LastEditorUserId="2444" LastEditDate="2020-06-09T11:58:54.470" LastActivityDate="2020-07-18T14:50:02.223" Title="What is an activity vector in capsule neural networks?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;capsule-neural-network&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4429" PostTypeId="2" ParentId="3548" CreationDate="2017-11-04T04:53:55.627" Score="1" Body="&lt;p&gt;To complement the other answers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend you to take the &lt;a href=&quot;https://www.edx.org/es/course/artificial-intelligence-ai-columbiax-csmm-101x-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence course&lt;/em&gt;&lt;/a&gt; from the AI micromaster given by Columbia on edx.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The course cover a wide range of AI problems and the most important is that give you a general framework to think with a mix of applications on python. Based on the book of &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;&lt;/a&gt; by Peter Norvig and Stuart Russell&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the perspective of machine learning as well said &lt;a href=&quot;https://ai.stackexchange.com/users/7899/gokul&quot;&gt;gokul&lt;/a&gt;, the &lt;a href=&quot;https://es.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning course&lt;/a&gt; of Andrew Ng. on coursera is a good introductory course and very oriented to a potential practitioner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found useful to combine the study of some machine learning algorithms with the &lt;em&gt;statistical programming language R&lt;/em&gt; to experiment with many algorithms to catch the concepts. Useful the following books: &lt;em&gt;Elements of Statistical Learning&lt;/em&gt; and &lt;em&gt;Introduction to Statistical Learning&lt;/em&gt;, both are available free on the authors websites.&lt;/p&gt;&#xA;" OwnerUserId="9574" LastActivityDate="2017-11-04T04:53:55.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4430" PostTypeId="2" ParentId="4425" CreationDate="2017-11-04T09:03:41.727" Score="2" Body="&lt;p&gt;Since you're termed the problem you're trying to solve is to &quot;measure the body temperature of a person&quot;, the output should be a continuous valued number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem statement had been to rank or classify a person's body temperature as fever/healthy, then this could be a classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning employ neuron units at the output layer depending on the target objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost all machine learning approaches produce continuous valued predictions. The output is usually &quot;classified&quot; to a set of discrete labels in a separate step using cutoff values. Its best to produce a probability estimate as an outcome since the step of classifying is going to introduce additional errors due to the choice of a cutoff value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, deep learning models deployed for the ImageNet computer vision dataset to classify objects into one of 1000 objects do not give the class of the object. Instead they give a 1000 value output listing the probability of that image being in a particular class. An image of a snow leopard would generate a probability of 95% snow leopard, 90% leopard, 75% Cat, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to summarize, the output of your task should be a temperature reading such as 36&amp;deg;C.&lt;/p&gt;&#xA;" OwnerUserId="2106" LastActivityDate="2017-11-04T09:03:41.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4431" PostTypeId="2" ParentId="4421" CreationDate="2017-11-04T19:00:00.053" Score="2" Body="&lt;p&gt;Reinforcement learning is the best fit for problems like these. To get started explore how DeepMind solved Atari games. The key is self play. RL works through trial and error and eventually figures out the best strategy. You also need to tweak so that it does not play at God level making it impossible for users to win.&lt;/p&gt;&#xA;" OwnerUserId="6418" LastActivityDate="2017-11-04T19:00:00.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4432" PostTypeId="1" CreationDate="2017-11-04T19:21:49.827" Score="1" ViewCount="406" Body="&lt;p&gt;I'm looking to build a conversational AI. The goal is to create an AI that is your friend (as opposed to personal assistants that are meant for carrying out commands like &quot;send a text, play that song...&quot; etc). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the underlying techniques involved ? Is there any good open source project I can build on top of. What kind of training data do I need ? I looked at Jasper - doesnt look like a good fit based on a shallow evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm imagining that the user text needs to be parsed out (like for eg using Google's syntaxnet). May be it queries some knowledge graph and then uses NLG. Or it uses some kind of seq-seq modelling like LSTM/RNN to generate responses. &lt;/p&gt;&#xA;" OwnerUserId="6418" LastActivityDate="2018-01-04T09:38:03.213" Title="Build Conversational AI" Tags="&lt;natural-language-processing&gt;&lt;intelligent-agent&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4434" PostTypeId="1" CreationDate="2017-11-05T05:15:44.193" Score="1" ViewCount="66" Body="&lt;p&gt;Given an ease with which a human can read a text scrambled in a special way, like the following passage popular a while ago&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn't mttaer&#xA;  in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is&#xA;  taht the frist and lsat ltteers be at the rghit pclae. The rset can be&#xA;  a toatl mses and you can sitll raed it wouthit porbelm. Tihs is&#xA;  bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the&#xA;  wrod as a wlohe.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;could you train a computer to restore the original? Are there any studies/known instruments that can achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am talking about mildly scrambled/damaged text, like the above, not a totally random permutations or random omission of letters.&lt;/p&gt;&#xA;" OwnerUserId="10655" LastActivityDate="2017-11-05T21:06:49.297" Title="Is it possible to restore scrambled/damaged text?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4435" PostTypeId="2" ParentId="4399" CreationDate="2017-11-05T08:34:36.660" Score="0" Body="&lt;p&gt;After running the code few more times i am thinking the problem lies in the discrete nature of error. In batch learning we are deciding the error as [0, 1, -1] depending on the input and the input whereas in the measuring of accuracy we are not making consideration of how big the error is i.e. whether the line is classifying the 2 classes by a very small margin (just classifying it) or classifying it by a good margin (comfortably classifying it)..both will give the same error since we are not considering the distance of the points from the line. Also, on choosing the initial weights we are selecting a point in the error vs weight function (which is not continuous) and if our choice of weights are inappropriate, the classifier weights is either getting trapped in a local minima or taking a very large number of epochs to reach the best weight (there is a very little probability that the classifier weights hit the right value after which it is easily able to achieve global minima). This is validated by the fact that on choosing a suitable weight we are getting the best accuracy possible within very few epochs but on choosing a different weight it doesn't reach the best accuracy but comes close to it after a large number of epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the example by example learning method, in this type of training the initial weights will determine the local minima the classifier weights get trapped in and its impossible to get out of it even with a large number of epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2 ways i think to solve the problem is by either using the Delta rule or by reinforcement learning (although i am not sure whether it can be applied to such discrete error functions) where the algorithm can see that even if it doesn't get an immediate gain by moving in a direction, but finally get a  much more gain(thus will be able to get out of local minima) i.e exploitation vs exploration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any further insights, suggestions or corrections are welcome.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2017-11-05T08:34:36.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4436" PostTypeId="1" CreationDate="2017-11-05T11:58:43.207" Score="3" ViewCount="179" Body="&lt;p&gt;I've mostly seen (e.g. in  &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Unreasonable Effectiveness of Recurrent Neural Networks&lt;/a&gt;) that when training RNN on text for something like language modeling, the text is usually featurized character-by-character using a 1-hot encoding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the text &quot;hello&quot; would be represented like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{h: 1, e: 0, l: 0, o: 0}&#xA;{h: 0, e: 1, l: 0, o: 0}&#xA;{h: 0, e: 0, l: 1, o: 0}&#xA;{h: 0, e: 0, l: 1, o: 0}&#xA;{h: 0, e: 0, l: 0, o: 1}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was wondering if one could just as well use the ASCII encoding of the text and feed the bits in one by one. So the input &quot;hello&quot; would be input like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0110100001100101011011000110110001101111 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Would the RNN have a disproportionately harder time having to figure out how the arbitrary and complex 8-bit ASCII encoding should be used? Or would the ASCII encoding lead to about the same performance as the nicer 1-hot encoding?&lt;/p&gt;&#xA;" OwnerUserId="10661" LastEditorUserId="9863" LastEditDate="2020-04-11T17:59:59.913" LastActivityDate="2020-04-11T17:59:59.913" Title="Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4437" PostTypeId="2" ParentId="4409" CreationDate="2017-11-05T13:40:10.120" Score="0" Body="&lt;p&gt;I would use a single network:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essence of the question is whether or not doing all the classification work at once is more efficient than running individual classifiers for each stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recent &quot;You Only Look Once&quot; algorithm (&quot;YOLO&quot;) is based on the fact that the convolutional networks can reuse a lot of the interim calculations if you combine them into one. Because of this, they are able to perform real-time object detection on images across thousands of classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can express your hierarchical classifier with YOLO (man, jacket and jacket colour classes). Depending on your needs, you might want to model the jacket colour as a scalar output of an approximate R,G,B value for the colour rather than having named classes for the colours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This everything-at-once implementation gives you runtime efficiencies for the inference step and much faster training since the classes share common abstractions in the earlier layers of the net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details, the YOLO version 2 paper, and a cool demonstration video featuring James Bond are available here: &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/darknet/yolo/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper itself, &quot;YOLO9000&quot;, is available on Arxiv: &lt;a href=&quot;https://arxiv.org/abs/1612.08242&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1612.08242&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10368" LastActivityDate="2017-11-05T13:40:10.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4438" PostTypeId="2" ParentId="1294" CreationDate="2017-11-05T14:46:15.230" Score="1" Body="&lt;p&gt;Capsule Networks have two key ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the first idea is &lt;strong&gt;how to represent multi-dimensional entities&lt;/strong&gt;. Capsule Networks does this by grouping these properties of a feature together (&quot;capsules&quot;).&lt;/li&gt;&#xA;&lt;li&gt;the second is that you &lt;strong&gt;activate higher-level features by agreement between lower-level features&lt;/strong&gt; (&quot;routing by agreement&quot;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;First, Capsule Networks partition the image into regions subsets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each of these regions, it assumes that there is at most one instance of a single feature, called a Capsule. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Capsule is able to represent an instance of a feature (but only one) and is able to represent all the different properties of that feature, &lt;em&gt;e.g.&lt;/em&gt;, its &lt;em&gt;(x,y)&lt;/em&gt; coordinates, its colour, its movement &lt;em&gt;etc.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference from Convolutional Neural Networks (CNNs) is that the Capsules bundle the neurons into groups with multi-dimensional properties, whereas in CNNs the neurons represent single, unrelated scalar properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This structured Capsule representation allows you to do &quot;routing by agreement&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand this, lets look at the example of a face detector. Here, you could have capsules representing &quot;mouth&quot;, &quot;eye&quot;, &quot;nose&quot; &lt;em&gt;etc.&lt;/em&gt; &#xA;Since the Capsules are multi-dimensional you also train them to predict the parameters for the entire face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if the &quot;mouth&quot;, &quot;nose&quot; and &quot;eye&quot; Capsules agree about the parameters of the face we have a very strong signal that this is a good prediction since accidental agreement in a high-dimensional space like a neural network is very unlikely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You use this to stack the Capsules into deep networks where the activation of higher-level Capsules are conditioned on agreement between the lower-level Capsules (&lt;em&gt;e.g.&lt;/em&gt; the Face capsule being activated by agreement on the face position between the Nose, Mouth, Eye Capsules in the earlier, lower-level layer). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to regular feed-forward nets this requires a bit of iteration in the forward pass through the network, but you can still use back-propagation train it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an interesting way to add a bit of structure to the data. So far, it looks like they are able to provide better generalization from limited training data.&lt;/p&gt;&#xA;" OwnerUserId="10368" LastActivityDate="2017-11-05T14:46:15.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4441" PostTypeId="2" ParentId="4434" CreationDate="2017-11-05T21:06:49.297" Score="1" Body="&lt;p&gt;You can rephrase your question as a statistical problem, where you for each word or n-gram try to predict the most likely correctly spelled version of each word or n-gram based on your knowledge of the distributions of these across a corpus of text in the target language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how spell-checkers work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Peter Norvig (who wrote &lt;em&gt;Artificial Intelligence: A Moderne Approach&lt;/em&gt; with Stuart Russell) has an article on the this with a short, easy-to-read code example and a description of the statistics behind it on his web page: &lt;a href=&quot;http://norvig.com/spell-correct.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;How to write a Spelling Corrector&quot;&gt;How to write a Spelling Corrector&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10368" LastActivityDate="2017-11-05T21:06:49.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4443" PostTypeId="1" CreationDate="2017-11-05T22:40:51.413" Score="5" ViewCount="2443" Body="&lt;p&gt;I'm new in this argument, my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can convolution be applied in other contexts different from image recognition?&#xA;Is there a good source to learn from?&lt;/p&gt;&#xA;" OwnerUserId="10676" LastEditorUserId="4398" LastEditDate="2017-11-05T22:52:04.350" LastActivityDate="2018-07-22T02:15:29.647" Title="Can Convolutional Neural Networks be applied in domains other than image recognition?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4444" PostTypeId="2" ParentId="4443" CreationDate="2017-11-06T06:10:29.393" Score="0" Body="&lt;p&gt;CNNs are generally designed for 2D (generally image) data. Hence, other usages are most likely a &quot;hack&quot; to CNN logic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check how to extract features and classify texts with CNN through &lt;a href=&quot;http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-06T06:10:29.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4445" PostTypeId="2" ParentId="4443" CreationDate="2017-11-06T11:11:04.660" Score="7" Body="&lt;p&gt;Yes, Convolutional Neural Networks(CNNs) can and have been applied to non-image problems. Arguably, any problem in which the location of a feature(s) is relevant can be attempted via CNNs. CNNs works under the assumption that points close to each other in the data share some correlations/relationship whist points further apart don't share as much information. So, theoretically, if you can phrase your problem so that it meets this requirement(s), it can be attempted by a convolutional neural network. Here are few applications of CNNss that are dont involve images: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/document/6857341/?reload=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speech recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1509.01626&quot; rel=&quot;nofollow noreferrer&quot;&gt;Character-level Convolutional Networks for Text Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://chara.cs.illinois.edu/sites/sp16-cs591txt/files/0226-presentation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Text classification using CNNs&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Processing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.04691.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Time Series Forecasting&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1404.2188&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sentence modeling&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've used 1-D CNNs on spectral data. Here are some examples of &lt;a href=&quot;https://arxiv.org/abs/1710.01927&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNNs applied to spectral data&lt;/a&gt; : &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/2af4/f72c2326c15616ad9f5cf18d5278d6608a14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spectral Convolutional Neural Network for music classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/document/7730323/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional neural network based classification for hyperspectral data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7550" LastEditorUserId="17019" LastEditDate="2018-07-22T02:15:29.647" LastActivityDate="2018-07-22T02:15:29.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4446" PostTypeId="5" CreationDate="2017-11-06T15:08:21.357" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-11-06T15:08:21.357" LastActivityDate="2017-11-06T15:08:21.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4447" PostTypeId="4" CreationDate="2017-11-06T15:08:21.357" Score="0" Body="This is a special tag that is designed to be used only by the system for questions that have had all of their other tags removed. Do not add this tag to existing questions, it is reserved for use by the system to identify posts with no valid tags." OwnerUserId="4709" LastEditorUserId="4709" LastEditDate="2017-11-08T19:58:13.740" LastActivityDate="2017-11-08T19:58:13.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4454" PostTypeId="1" CreationDate="2017-11-07T09:02:44.327" Score="7" ViewCount="769" Body="&lt;p&gt;What would be examples of journals that are good for a &lt;strong&gt;first publication&lt;/strong&gt; in the field of Deep Reinforcement Learning?&lt;/p&gt;&#xA;&lt;p&gt;I am in the process of writing about the research results of DQN-related algorithms.&lt;/p&gt;&#xA;&lt;p&gt;I have 3 requirements - it should be indexed in one of these databases, otherwise, I cannot receive grant money for research:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.scopus.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scopus.com/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://webofknowledge.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://webofknowledge.com&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And it should not be very expensive to publish. It should be under 1000EUR to publish, for example, the Open Access license for Elsevier &amp;quot;Artificial Intelligence&amp;quot; journal costs around 2400EUR to publish.&lt;/p&gt;&#xA;&lt;p&gt;And it should not have a very long review/publishing period. For example, Elsevier's &amp;quot;Information Fusion&amp;quot; journal currently gathers articles for July 2018, which is 8 month period till publishing. Is it normal?&lt;/p&gt;&#xA;&lt;p&gt;Can you please recommend some journals that qualify &amp;amp; you have had good experience publishing research?&lt;/p&gt;&#xA;" OwnerUserId="9740" LastEditorUserId="2444" LastEditDate="2021-01-31T12:56:43.190" LastActivityDate="2021-01-31T13:20:59.337" Title="Where to publish a first article in Deep Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;papers&gt;&lt;research&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4455" PostTypeId="2" ParentId="3943" CreationDate="2017-11-07T12:27:47.347" Score="2" Body="&lt;p&gt;Deep learning models for regression tasks are quite difficult to train so I would suggest not to start with them. Instead I would start with one of the approaches below and maybe try to use deep learning afterwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classical approach to the problem could be to analyze your optimization software and this would probably lead you to some deterministic algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different approaches can be to treat your optimization software as a black box - give it a wide variety of inputs, note down the variables you are interested in (execution time, cutting results, etc.) and try to fit some sort or regressor on it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option is to follow Kourosh's idea and formulate it as a classical optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you prefer to use machine learning tools than I'd suggest that you start with a simple model like linear regression just to verify that there is any signal in the data that you can use. Afterwards you can look at more powerful algorithms like xgboost, regression trees, etc.&lt;/p&gt;&#xA;" OwnerUserId="10481" LastActivityDate="2017-11-07T12:27:47.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4456" PostTypeId="1" CreationDate="2017-11-07T14:10:09.410" Score="56" ViewCount="44777" Body="&lt;p&gt;What's the difference between model-free and model-based reinforcement learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that any model-free learner, learning through trial and error, could be reframed as model-based. In that case, when would model-free learners be appropriate?&lt;/p&gt;&#xA;" OwnerUserId="10720" LastEditorUserId="2444" LastEditDate="2019-02-16T19:01:36.767" LastActivityDate="2019-12-01T21:33:49.050" Title="What's the difference between model-free and model-based reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="28" ContentLicense="CC BY-SA 3.0" />
  <row Id="4460" PostTypeId="1" CreationDate="2017-11-07T21:00:12.733" Score="4" ViewCount="194" Body="&lt;p&gt;What would be the result of an AI subject to a data stream or information stream, fed to it from events in an &amp;quot;external world&amp;quot;, if it could not find any pattern within that stream by the methods it has in determining patterns?&lt;/p&gt;&#xA;&lt;p&gt;Would the response be because of the limitation of its methods in determining patterns or would you be able to conclude that the information from those events are undetermined and therefore could be stated as random?&lt;/p&gt;&#xA;&lt;p&gt;Or would the AI be unable to ever determine using its methods whether the information fed to it was from a random undetermined world because it needed an eternity of information to determine whether the data was random or not?&lt;/p&gt;&#xA;&lt;p&gt;So what I guess I'm asking is the pursuit of creating AI by mankind on a similar level of limitation by the accumulation of &amp;quot;Knowledge&amp;quot; over time as is the other pursuits of mankind within science etc, and is only going to be achieved simultaneously with the acquisition of &amp;quot;knowledge&amp;quot; from these other pursuits. If the determination of a simulated model comes from the physics of a determined world or not will there ever be a model because of the eternity of determining whether the information from the &amp;quot;external world&amp;quot; is determined or not.&lt;/p&gt;&#xA;" OwnerUserId="9094" LastEditorUserId="2444" LastEditDate="2020-09-17T17:14:57.220" LastActivityDate="2020-09-17T17:14:57.220" Title="What would be the result of an AI that does not find any pattern according its available methods of pattern recognition?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;pattern-recognition&gt;&lt;data-mining&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4461" PostTypeId="2" ParentId="3373" CreationDate="2017-11-07T23:00:29.893" Score="3" Body="&lt;p&gt;Google has not released the manufacturing details for their TPUs. However, it's suspected that they're produced by either Taiwan Semiconductor Manufacturing or GlobalFoundries, as these are some of the largest companies in the industry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.recode.net/2016/5/20/11719392/google-ai-chip-tpu-questions-answers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10736" LastActivityDate="2017-11-07T23:00:29.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4462" PostTypeId="2" ParentId="3596" CreationDate="2017-11-08T01:45:10.140" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;input&lt;/strong&gt;. rather than concatenating four consecutive frames, try using the difference between two consecutive frames as input. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;network architecture&lt;/strong&gt;. a vanilla multi layer net will work better than a convnet in this case. you dont really need spatial/translational invariance for pong, especially if you use frame differences as input. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10720" LastActivityDate="2017-11-08T01:45:10.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4463" PostTypeId="1" CreationDate="2017-11-08T05:19:17.283" Score="2" ViewCount="64" Body="&lt;p&gt;Say I need to make an action such as &quot;Walk&quot; perform for 2 seconds using a behavior tree which has access to a timer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The timer can do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;start, time (get value of timer), and reset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the behavior tree should return Success every time &quot;Walk&quot; is completed, unless the timer reset or &quot;Walk&quot; is still executing, how could such a thing be built?&lt;/p&gt;&#xA;" OwnerUserId="10741" LastActivityDate="2017-11-08T05:19:17.283" Title="Making behavior tree execute action for specific time?" Tags="&lt;game-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4465" PostTypeId="2" ParentId="4456" CreationDate="2017-11-08T14:07:08.093" Score="25" Body="&lt;p&gt;Model-based reinforcement learning has an agent try to understand the world and create a model to represent it. Here the model is trying to capture 2 functions, the transition function from states &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and the reward function &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. From this model, the agent has a reference and can plan accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is not necessary to learn a model, and the agent can instead learn a policy directly using algorithms like Q-learning or policy gradient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple check to see if an RL algorithm is model-based or model-free is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If, after learning, the agent can make predictions about what the next state and reward will be before it takes each action, it's a model-based RL algorithm.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it can't, then it’s a model-free algorithm.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2019-02-16T19:08:43.947" LastActivityDate="2019-02-16T19:08:43.947" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4470" PostTypeId="2" ParentId="4432" CreationDate="2017-11-08T19:02:17.263" Score="0" Body="&lt;p&gt;A useful intro to conversational Q&amp;amp;A systems like IBM's Jeopardy Watson should be available from the IBM DeepQA research team (&quot;This is Watson&quot;), much of which is available free:  &lt;a href=&quot;http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=2099&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=2099&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In May/June 2012, the IBM Journal of R&amp;amp;D published a special issue entirely about the tech behind the IBM Watson Q&amp;amp;A software that played Jeopardy.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AFAIK, the articles are no longer freely available on-line, but similar work should form a useful basis for building the essential pieces in a conversational agent -- a knowledge base, queries &amp;amp; updates, understanding and generating text and speech, etc.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastActivityDate="2017-11-08T19:02:17.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4471" PostTypeId="2" ParentId="3931" CreationDate="2017-11-08T20:56:54.483" Score="4" Body="&lt;p&gt;No, here is why. No approach can simulate the mind with 100% accuracy. a major notion that AI theorist refuse to note is that you cant take an orange and by virtue of technology turn it into an apple lets apply the same logic here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;neurons are temporary things in our brains, daily we are trimming our brains and growing our brains, in order to &quot;Engineer&quot; a human Brain a probability whatever or not that neuron would die or not would have to be present. But that probability is flawed as well because, Nature is not math. In this case we are trying to apply something to Math that is unpredictable. Math is something we have used to try to rationalize the way nature works but it out right fails. If nature is completely math then predicting a hurricane would be easy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question, no because a Neuron is more then a light switch, it is not mathematical in nature. their is no universality in everyone. plus their is more variables that you would have to account for that are way beyond a simple decision and a simple mathematical formula.     &lt;/p&gt;&#xA;" OwnerUserId="4315" LastEditorUserId="4315" LastEditDate="2017-11-10T19:47:48.600" LastActivityDate="2017-11-10T19:47:48.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4472" PostTypeId="1" AcceptedAnswerId="4485" CreationDate="2017-11-09T10:27:06.057" Score="3" ViewCount="354" Body="&lt;p&gt;I have a set of images that I already trained a CNN to classify successfully. I wonder if it would be possible to encode the images (using XOR in combination with a key of the same length as the image) and train a new net on them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thinking logically, the features still exist in the same relation to each other, just in a different form (encoded). Considering that neural networks are incredible at pattern recognition, I assume that it would still be doable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For people, who cannot imagine how a xor-encoded image would look like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GsIxN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GsIxN.png&quot; alt=&quot;example for encoded image using random key&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a human, it may look like rubbish, but the information is definitely there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would love to read your opinion.&lt;/p&gt;&#xA;" OwnerUserId="10783" LastActivityDate="2017-11-12T09:04:28.110" Title="Would convolutional NN recognize patterns in encoded images?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4474" PostTypeId="2" ParentId="4428" CreationDate="2017-11-09T23:35:41.483" Score="1" Body="&lt;p&gt;I took it to mean something like &quot;the vector of activations of the neurons in the capsule&quot;.  The activation for a given &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Artificial_neuron&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuron&lt;/a&gt; is the weighted sum of its inputs, passed through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;activation function&lt;/a&gt; (sigmoid, relu, etc).&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2017-11-09T23:41:59.753" LastActivityDate="2017-11-09T23:41:59.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4477" PostTypeId="1" CreationDate="2017-11-10T06:19:58.130" Score="4" ViewCount="135" Body="&lt;p&gt;I've seen some articles about text generation using LSTMs (or GRUs) for text generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained it with text which includes the string:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The dog chased the cat&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and also includes the string:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The lion chased the ca&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both should be acceptable. But although they are very similar they differ entirely after the 4th character. So they will result in two very different vectors. Further the longer you roll it out the more they will differ.&#xA;How is it then possible for an LSTM to learn that &quot;The [something] chased the cat&quot; is an acceptable phrase?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equally if you train it to try and learn to pair up parentheses. I can see how you could manually design it to do this, but how exactly could it be trained to do this just by entering strings like &quot;(das asdasdas) axd&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm getting at is that I don't get how it could LEARN any sort of structure more than a Markov model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also, I've only ever seen one article that showed a LSTM that can pair parentheses. So has this study ever been replicated?) I get that it is possible for an LSTM to do this but I don't get how it can learn to do this!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm getting at is you usually train things with the input phrase and then compare it to the expected phrase to get the error. But in text generation there might be millions of possible acceptable expected phrases! So how can you compute an error?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2017-11-10T06:26:29.593" LastActivityDate="2017-11-11T20:34:12.033" Title="How are LSTM's trained for text generation?" Tags="&lt;training&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4479" PostTypeId="2" ParentId="4428" CreationDate="2017-11-10T10:58:19.057" Score="9" Body="&lt;p&gt;In a traditional neural network, the network's vertices are neurons and the output of a single neuron is a single value (a &amp;quot;&lt;em&gt;scalar&lt;/em&gt;&amp;quot;). This number is called its &lt;em&gt;activation&lt;/em&gt;. A &lt;em&gt;layer&lt;/em&gt; of neurons in the network outputs a vector of activations. We should not confuse this with the activity vectors in a Capsule Network.&lt;/p&gt;&#xA;&lt;p&gt;Capsule Networks are different since the network vertices are Capsules rather than neurons. They are a higher-dimensional: the output of a Capsule is not a &lt;em&gt;scalar&lt;/em&gt; but a &lt;em&gt;vector&lt;/em&gt; representing a group of parameters related to the input. Hence the name &lt;strong&gt;activation &lt;em&gt;vector&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Motivation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In a neural network, there is no inherent structure between the scalar outputs of the neurons, this is something the following layers have to learn. In Capsule Networks the output of a capsule represents all the parameters related to that together in a vector including a prediction for the activation of deeper layer Capsules. This adds a useful local structure.&lt;/p&gt;&#xA;&lt;p&gt;For example, consider face recognition. If you have a capsule that knows how to recognize eyes it could output an activity vector representing &lt;em&gt;e.g.&lt;/em&gt; &amp;quot;since I have recognized an eye position &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$p=0.97$&lt;/span&gt; I predict the parameters for the whole face will be &lt;span class=&quot;math-container&quot;&gt;$(f_1,\dots, f_n)$&lt;/span&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;As explained in the &lt;em&gt;Dynamic Routing Between Capsules&lt;/em&gt; paper you refer to this information is then used in the way that the capsules in earlier layers (the parts: eye, mouth, nose) predict the activations of deeper layers (face). For example, a face recognizer will only be strongly activated when there is an agreement between the eye, nose and mouth recognizers (the parts) and the face recognizer (the whole) about where the face is located (the &lt;span class=&quot;math-container&quot;&gt;$(f_1,\dots, f_n)$&lt;/span&gt; parameters).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Historical Inspiration&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Older computer vision algorithms like &lt;a href=&quot;https://en.wikipedia.org/wiki/Scale-invariant_feature_transform&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SIFT&quot;&gt;SIFT&lt;/a&gt; work in a similar way where recognition is based on agreement between the configuration of multi-dimensional features (key points) and the reference configuration.&lt;/p&gt;&#xA;" OwnerUserId="10368" LastEditorUserId="2444" LastEditDate="2020-07-18T14:50:02.223" LastActivityDate="2020-07-18T14:50:02.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4482" PostTypeId="1" AcceptedAnswerId="4483" CreationDate="2017-11-10T15:54:48.073" Score="10" ViewCount="1902" Body="&lt;p&gt;I invented a chess-like board game. I built an engine so that it can play autonomously. The engine is basically a decision tree. It's composed by:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A search function that at each node finds all possible legal moves&lt;/li&gt;&#xA;&lt;li&gt;An evaluation function that assigns a numerical value to the board position (positive means the first player is gaining the upper hand, negative means the second player is winning instead)&lt;/li&gt;&#xA;&lt;li&gt;An alpha-beta pruning negamax algorithm&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The main problem with this engine is that the optimization of the evaluation function is really tricky. I don't know which factors to consider and which weights to put. The only way I see to improve the engine is to iterate games trying each time different combinations of factors and weights. However, it computationally seems a very tough feat (Can I backpropagate without using deep learning?).&lt;/p&gt;&#xA;&lt;p&gt;I would like to use reinforcement learning to make the engine improve by playing against itself. I have been reading about the topic, but I am still quite confused.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What other reward is there in a game apart from the win-or-lose output (1 or 0)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I use other rewards, like the output from the evaluation function at each turn, how can I implement it?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I modify the evaluation function to give better rewards iteration after iteration?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10813" LastEditorUserId="2444" LastEditDate="2020-11-01T16:38:24.620" LastActivityDate="2020-11-01T16:38:24.620" Title="How could I use reinforcement learning to solve a chess-like board game?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;applications&gt;&lt;reward-design&gt;&lt;negamax&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4483" PostTypeId="2" ParentId="4482" CreationDate="2017-11-10T18:23:22.293" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to use reinforcement learning to make the engine improve by playing against itself. I have been reading about the topic but I am still quite confused. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Be warned: Reinforcement learning is a large complex subject. Although it might take you on a detour from game-playing bots, you may want to study RL basics. A good place to start is &lt;a href=&quot;http://incompleteideas.net/sutton/book/the-book-2nd.html&quot; rel=&quot;noreferrer&quot;&gt;Sutton &amp;amp; Barto &lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What other reward is there in a game a part the win-or-lose output (1 or 0)? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Depending on you game, that is usually it. Actually for a win/draw/lose game like chess then the reward from each action is 0 except for winning (+1) or losing (-1) at the end. In a zero-sum game then this aligns nicely to minimax, alphabeta pruning etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement Learning is intended to address environments with delayed rewards. Adding &quot;helper&quot; rewards for interim non-goals is usually counter-productive.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I use other rewards, like the output from the evaluation function at each turn, how can I implement it? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Typically you don't. What applying self-playing RL will do is learn a &lt;em&gt;return&lt;/em&gt; (sometimes called &lt;em&gt;utility&lt;/em&gt;) function which predicts the expectation of your total +1/0/-1 reward by the end of the game. You would use this in place of your current heuristic for minimax search. Or, potentially you would adjust your current heuristic function to output in the same range and use RL to optimise its weights to make the best approximation to the true optimal play return function (which is probably too complex to calculate exactly).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I modify the evaluation function to give better rewards iteration after iteration?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is what the different RL approaches all attempt to do, there are a variety of different solvers. There is no short way to explain it. You could start with &lt;a href=&quot;http://mnemstudio.org/path-finding-q-learning-tutorial.htm&quot; rel=&quot;noreferrer&quot;&gt;a simple method such as Q-Learning&lt;/a&gt;. Q-Learning learns estimates of Q(s,a) (called the action value) which is the expected return when in state s and taking action a, and thereafter following an optimal policy. It makes an arbitrary guess to start with and refines it closer to the true value with each step made in the learning environment. Simple tabular Q-learners do this refinement simply by storing a big table of all states and actions with the best estimate so far of the true value, and averaging in each new estimate as it is experienced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to combine an RL method for heuristics with look-ahead minimax search - that is what the original AlphaGo did, and what AlphaGo Zero does during training. This is a powerful approach because minimax search will work to double-check the RL-generated heuristics. Although for simple enough games, RL can learn perfect heuristics and you would only need local search (what the next move should be).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless your game is very simple (all possible states would fit in memory), you will need some kind of function approximator inside the RL algorithm. Neural networks are a standard choice. Having &lt;em&gt;something&lt;/em&gt; for that part is unavoidable - although another good choice is to define a bunch of proxy features (that you might use to construct a heuristic by hand) and use a linear approximator - just a weighted sum of all the features. This can work well enough, and has been used for example in checkers (draughts) players trained using RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, provided your own heuristic function is not too unusual, you can probably treat it just like a linear approximator and use RL to learn the best weights for it.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2017-11-10T22:12:28.437" LastActivityDate="2017-11-10T22:12:28.437" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4484" PostTypeId="1" CreationDate="2017-11-10T21:46:30.933" Score="2" ViewCount="41" Body="&lt;p&gt;Is there examples of AI systems being taught grammatical rules, or have such rules built-in to them, in order to recognize commands and the like in natural language?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, is there a program, when given the instruction (either by speech or by text) in the form of &quot;Open file-1&quot;, will attempt to understand the command as consisting the parts &quot;open&quot; and &quot;file-1&quot;, then respond according to how each part is classified?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The focus here is the fact that these kind of instructions are processed with built-in rules of grammar, rather than the rules being learned through observation of data.&lt;/p&gt;&#xA;" OwnerUserId="1321" LastEditorUserId="4302" LastEditDate="2018-10-08T12:49:19.793" LastActivityDate="2018-10-08T12:49:19.793" Title="Has there been &quot;prescriptive&quot; attempts at language-learning/comprehension in AI?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4485" PostTypeId="2" ParentId="4472" CreationDate="2017-11-10T22:31:11.353" Score="4" Body="&lt;p&gt;First you should give it a try, because anyone's guess could be off, as there isn't really a complete high level analytic model of how neural networks behave on real data. Most results with neural networks are informed by theory, but involve a whole lot of experimental testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect your network might at least learn &lt;em&gt;something&lt;/em&gt; from the new images, but would struggle to get anything like the same accuracy as without the noise, because the CNN filters rely on being able to detect similar features at different positions. In your scrambled image, there will not be any meaningful and &lt;em&gt;consistent&lt;/em&gt; edges/corners etc that a single learned feature detector could learn to match (and therefore present to the next layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fully-connected network would not have this limitation, and would learn just as well on a set of &lt;em&gt;binary&lt;/em&gt; features that have been xor'd identically in each position for each example, as it did on the original copy (i.e. only if the picture was 1 bit depth). It would learn less well if each feature was a scaled 8-bit pixel value that was xor'd with the same 8 bit random number in each pixel position, because that would introduce many more non-linear mappings between input and output. Of course a fully-connected network will generally not learn image tasks as well as CNNs in the first place . . . but if it could learn anything useful at all for your image problem, then it will probably out-perform CNNs after the scrambling effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a CNN usually has a few fully-connected layers, then it may be possible to get &lt;em&gt;something&lt;/em&gt; from your scrambled images.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thinking logically, the features still exist in the same relation to each other, just in a different form (encoded)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In terms of being recognisable in the way that a CNN filter extracts them, then the features do not exist. That is a problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2017-11-12T09:04:28.110" LastActivityDate="2017-11-12T09:04:28.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4486" PostTypeId="2" ParentId="4472" CreationDate="2017-11-11T01:08:36.100" Score="0" Body="&lt;p&gt;The answer to your question depends on the nature of the noise that you have xor-ed the image with. If it is the case that the noise is random (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Pseudorandomness&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudorandom&lt;/a&gt; in the formal sense), then it is provably the case that the original pattern will not be learnable in the statistical learning theory sense; this scenario is equivalent to the application of a &lt;a href=&quot;https://en.wikipedia.org/wiki/One-time_pad&quot; rel=&quot;nofollow noreferrer&quot;&gt;one-time pad&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To quote the relevant Wikipedia article:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One-time pads are &quot;information-theoretically secure&quot; in that the encrypted message (i.e., the ciphertext) provides no information about the original message to a cryptanalyst (except the maximum possible length[16] of the message). This is a very strong notion of security first developed during WWII by Claude Shannon and proved, mathematically, to be true for the one-time pad by Shannon about the same time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8366" LastActivityDate="2017-11-11T01:08:36.100" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4487" PostTypeId="2" ParentId="4484" CreationDate="2017-11-11T01:26:37.087" Score="1" Body="&lt;p&gt;Yes.  You're basically looking for &quot;rule based&quot; NLP.  For reference, see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rule-based_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Rule-based_system&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/Why-are-rule-based-methods-becoming-unpopular-in-NLP-Are-rule-based-methods-still-in-use-If-yes-where-should-I-look-for-them&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/Why-are-rule-based-methods-becoming-unpopular-in-NLP-Are-rule-based-methods-still-in-use-If-yes-where-should-I-look-for-them&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3756254/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3756254/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://nlp.stanford.edu/software/regexner.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nlp.stanford.edu/software/regexner.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://u.cs.biu.ac.il/~yogo/courses/sem2014/intro.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://u.cs.biu.ac.il/~yogo/courses/sem2014/intro.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-11-11T01:26:37.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4489" PostTypeId="1" CreationDate="2017-11-11T17:39:21.537" Score="3" ViewCount="3384" Body="&lt;p&gt;I want to build a personal assistant that listens to me continuously.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The flow looks like this: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;continuously record voice &lt;/li&gt;&#xA;&lt;li&gt;stream it to google speech api.&lt;/li&gt;&#xA;&lt;li&gt;get back the text in real time -&gt; parse for intent etc..&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Problem is, google speech api gets expensive if you record for hours. A better way to do it is to only submit the parts where I'm actually speaking to it.. Then the cost of running this full time (17 hours a day, every day) becomes very accessible.&#xA;Now my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can i detect that a voice is present in the microphone stream?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a lot of noise in the background, dumb &lt;code&gt;increase in volume detection&lt;/code&gt; is not a very good solution. I need something more intelligent. It doesn't need to be very accurate - just good enough to not break my cloud computing budget. I'm thinking that human voice sounds distinct enough that is not such a big problem to detect when is there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you recommend me to do, given this is a real time stream - not an audio file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The audio will be generated in chromium browser (electron) - with &lt;code&gt;getUserMedia&lt;/code&gt; API, and using &lt;code&gt;node.js&lt;/code&gt; i plan to handle the streaming logic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: there is a built in &lt;code&gt;speechRecognition api&lt;/code&gt; in &lt;code&gt;electron&lt;/code&gt; - but from my experience currently it doesn't work (not even after i give it my API key), and even if would have worked, i think it has the same cost problem. So this is why i'm trying to provide my own implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what i'm doing, any insight is welcomed:) Thank you.&lt;/p&gt;&#xA;" OwnerUserId="10650" LastEditorUserId="1671" LastEditDate="2017-11-12T22:13:25.073" LastActivityDate="2017-11-13T19:05:55.950" Title="How to detect when human voice / speech appears in an microphone stream?" Tags="&lt;intelligent-agent&gt;&lt;voice-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4490" PostTypeId="2" ParentId="4472" CreationDate="2017-11-11T18:59:40.683" Score="0" Body="&lt;p&gt;My gut feel based on the paper I will mention below is that &lt;strong&gt;YES&lt;/strong&gt;, if you apply the same XOR operation on the train and test data, you will be able to train a very &quot;accurate&quot; classifier. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To elaborate on my &quot;gut&quot; feel, please allow me to introduce to you what I personally think is one of the most important paper that came out this year(in fact this paper won the best paper award at ICLR 2017):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1611.03530&quot; rel=&quot;nofollow noreferrer&quot;&gt; Understanding deep learning requires rethinking generalization&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper, the authors showed that deep learning models will generalize to &quot;any&quot; datasets. To give an example of the sort of experiment they conducted on this paper: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They randomly shuffled the training and test set's labels around in such a manner that for example some images of cats were labeled as dogs whiles some dogs were named cats whilst some cats and dogs images remained correctly labeled. Now it is well understood that deep learning models(including CNNs) are quite resistant to a few noisy labels but in the experiments conducted in the paper mentioned above this was a significant amount of noisy which begs the question why neural networks still performed well on what ended up being a garbage dataset. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The moral of the story is that contrary to what most researchers believed in the past namely that deep learning models magically discover lower level features, middle-level features, and higher-level features hidden within the dataset more like the V1 system of the mammalian brain by learning to compress data, they seem to just memorize anything you give them, including random data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short the paper mentioned above showed that deep learning models generalize well to completely random noise(in your case, think images generated from random pixels). Deep learning models will generalize well to anything, anything. And if they can generalize to random data which have no structure, then images that underwent a fixed, predefined transformation like XOR have nothing to a deep learning model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I must say, this are very worrying findings - to me at least.&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="7550" LastActivityDate="2017-11-11T18:59:40.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4491" PostTypeId="2" ParentId="4477" CreationDate="2017-11-11T20:34:12.033" Score="2" Body="&lt;p&gt;You compute the error in the same way as normal - treat the actual output as &quot;ground truth&quot; even if other options are possible. The LSTM will output probabilities of next character, it will not learn to associate a single &quot;true&quot; output except for rare circumstances (such as completing a word that can only be done in one way according to the training data). In practice that means that understanding acceptable loss values in NLP training is different to understanding them in other supervised learning environments, typically being higher than for datasets where ground truth is more predictable from the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, it is a lot like a Markov or n-gram model, except that it learns its own internal state representations (they are not prescribed by the model, but are limited by structure and data available). The gated RNN memory models like LSTM and GRU include a mechanism for handling long term memory so can relatively easily learn things like quote, paren pairs etc.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What I'm getting at is that I don't get how it could LEARN any sort of structure more than a Markov model.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A Markov model can in theory represent any sequence knowledge that an LSTM can. The difference is in terms of efficiency. RNNs have more compact internal representations than explicit state representation in say HMMs. This makes them harder to work with in some ways (it is hard to see what a RNN state vector represents), but means that they can store more complex knowledge about state transitions in less memory. It also means that RNNs can generalise (although not guaranteed correctly) more readily, they will not fall back to random behaviour given a brand new state without any training history.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2017-11-11T20:34:12.033" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4492" PostTypeId="2" ParentId="4489" CreationDate="2017-11-12T03:04:39.820" Score="4" Body="&lt;p&gt;Your problem is an old one. There are many methods, referred to as Voice Activity Detection (VAD) methods, which detect speech from an audio signal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical design of a VAD algorithm follows one of more of these 3 approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A noise reduction stage, e.g. via spectral subtraction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some features or quantities are calculated from a section of the input signal.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A classification rule is applied to classify the section as speech or non-speech – often this classification rule finds when a value exceeds a threshold.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There may be some feedback in this sequence, where parameters (like the noise threshold or the classification threshold) are tuned to improve the estimate or to adaptively vary the threshold(s). These feedback operations improve the VAD performance in non-stationary noise (i.e. when the noise varies a lot).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out this &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Voice_activity_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki page on Voice Activity Detection&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2017-11-13T19:05:55.950" LastActivityDate="2017-11-13T19:05:55.950" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4494" PostTypeId="1" CreationDate="2017-11-12T06:00:40.320" Score="1" ViewCount="127" Body="&lt;p&gt;&lt;strong&gt;Has there ever been a model (neuro-physical) proposed for the human unconscious brain&lt;/strong&gt;?  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I tried reading on &lt;a href=&quot;https://en.wikipedia.org/wiki/Unconscious_mind&quot; rel=&quot;nofollow noreferrer&quot;&gt;unconscious mind&lt;/a&gt; but couldn't find anything which answers the question.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Is there any literature related to it?&lt;/p&gt;&#xA;" OwnerUserId="1976" LastActivityDate="2017-11-17T19:52:55.357" Title="Has there ever been a model (neuro-physical) proposed for the human unconscious brain?" Tags="&lt;human-like&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4496" PostTypeId="2" ParentId="1294" CreationDate="2017-11-12T06:47:28.620" Score="4" Body="&lt;p&gt;One of the major advantages of convolutional neural networks is their &lt;strong&gt;invariance&lt;/strong&gt; to translation. However, this invariance comes with a price, that is, it does not consider how different features are related to each other. For example, if we have a picture of a face, a CNN will have difficulties distinguishing the relationship between the &quot;mouth&quot; feature and &quot;nose&quot; feature. The max-pooling layers are the main reason for this effect, because, when we use max-pooling layers, we lose the precise locations of the mouth and nose, and we cannot say how they are related to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Capsules try to keep the advantage of CNN and fix this drawback in two ways&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Invariance&lt;/strong&gt;: quoting from this &lt;a href=&quot;http://www.cs.toronto.edu/~fritz/absps/transauto6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When the capsule is working properly, the probability of the visual entity being present is locally invariant – it does not change as the entity moves over the manifold of possible appearances within the limited domain covered by the capsule.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, capsule takes into account the existence of the specific feature that we are looking for, like the mouth or nose. This property makes sure that capsules are translation invariant the same that CNNs are.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Equivariance&lt;/strong&gt;: instead of making the feature translation &lt;strong&gt;invariance&lt;/strong&gt;, the capsule will make it translation-equivariant or viewpoint-equivariant. In other words, as the feature moves and changes its position in the image, the feature vector representation will also change in the same way which makes it equivariant. This property of capsules tries to solve the drawback of max-pooling layers that I mentioned at the beginning.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10640" LastEditorUserId="2444" LastEditDate="2020-06-09T15:25:50.717" LastActivityDate="2020-06-09T15:25:50.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4497" PostTypeId="1" CreationDate="2017-11-12T07:03:09.540" Score="1" ViewCount="936" Body="&lt;p&gt;I am trying to find a good evaluation function for a game with:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A 7x7 tile board&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2 players, given an equal number(&amp;gt;=3 currently undetermined) of &lt;em&gt;stones&lt;/em&gt; placed randomly on the tiles&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A turn is consisted of a player moving a &lt;em&gt;stone&lt;/em&gt; owned by that player, vertically or horizontally but &lt;em&gt;not diagonally&lt;/em&gt; to a very next tile of itself&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A player loses when out of moves: a player is out of moves when every &lt;em&gt;stone&lt;/em&gt; that player owns, has its very next tiles, except not for diagonals necessarily, occupied either by the board edge or other &lt;em&gt;stones&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Right now, my evaluation function's return value increases:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;if the total moves available to the player is increasing, and/or&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;average distance to the middle tile of the board is decreasing&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is there a better strategy? How can I improve my evaluation function?&lt;/p&gt;&#xA;" OwnerUserId="10843" LastEditorUserId="2444" LastEditDate="2021-02-06T20:20:04.993" LastActivityDate="2021-02-06T20:55:29.320" Title="How to design a good evaluation function for a go-like game?" Tags="&lt;game-ai&gt;&lt;combinatorial-games&gt;&lt;go&gt;&lt;evaluation-functions&gt;&lt;board-games&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4498" PostTypeId="2" ParentId="4497" CreationDate="2017-11-12T07:49:19.663" Score="1" Body="&lt;p&gt;If you have the best combination of distance between the stones, you should choose the best move to win. In this case, you have to be close to where your opponent plays. It is best to do this by surrounding your opponent's stones. You should always put the first stone in middle or corner of the table.&lt;/p&gt;&#xA;" OwnerUserId="10845" LastEditorUserId="10845" LastEditDate="2017-11-12T15:11:31.527" LastActivityDate="2017-11-12T15:11:31.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4499" PostTypeId="1" CreationDate="2017-11-12T11:20:46.507" Score="1" ViewCount="63" Body="&lt;p&gt;Well, I am new to implementing ANN's and there is something that i want to know. It maybe a bit silly though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just wanted to know that if we have a simple data set say dependent only on a single variable &lt;code&gt;x1&lt;/code&gt;, and if we want to train the ANN to predict the data set values (no new values, only the data set values needs to be predicted) will sorting or arranging the data set in some way beforehand make the ANN go to the optimum value quicker or it really doesn't matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If arranging does matter than how do we arrange multidimensional data and what is the intuition behind it being faster? Just like in Binary Search for a random data set we get a performance gain of &lt;code&gt;O(n^2)-O(nlogn)&lt;/code&gt; over Linear Search similarly I want to know whether such performance gains are available in ANN's in any way? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am aware this is already taken care by Batch Learning so i am specifically talking about Example by Example learning. Also, I want to know if some arranged form of data set works well for certain forms of training e.g. Momentum method, Delta method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in Advance.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2017-11-17T05:46:07.337" LastActivityDate="2017-11-17T05:46:07.337" Title="Orientation of data set before training simple ANN's" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;structured-data&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4500" PostTypeId="5" CreationDate="2017-11-12T21:55:42.387" Score="0" Body="&lt;p&gt;TensorFlow is an open source library for machine learning and machine intelligence. It is developed by Google and became open source in November 2015. &lt;/p&gt;&#xA;" OwnerUserId="10843" LastEditorUserId="10843" LastEditDate="2017-11-12T21:56:11.153" LastActivityDate="2017-11-12T21:56:11.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4501" PostTypeId="4" CreationDate="2017-11-12T21:55:42.387" Score="0" Body="For questions related to Google's open-source library for machine learning and machine intelligence. However, note that programming questions are off-topic here." OwnerUserId="10843" LastEditorUserId="2444" LastEditDate="2020-08-21T19:01:29.907" LastActivityDate="2020-08-21T19:01:29.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4502" PostTypeId="5" CreationDate="2017-11-12T21:58:28.390" Score="0" Body="&lt;p&gt;The AI components of a game may guide the behavior of the environment in which the player plays to create engaging and unique scenarios. These are some examples&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Simulation of human decision making in other players&lt;/li&gt;&#xA;&lt;li&gt;Simulation of walking, fighting, athletics, or other modes of mobility&lt;/li&gt;&#xA;&lt;li&gt;Simulation of three dimensional force and motion that may or may not conform with real world physics&lt;/li&gt;&#xA;&lt;li&gt;Construction and control of new threats&lt;/li&gt;&#xA;&lt;li&gt;Construction and control of new opportunities&lt;/li&gt;&#xA;&lt;li&gt;Dynamic environments for aesthetic enhancement&lt;/li&gt;&#xA;&lt;li&gt;Game rule plasticity&lt;/li&gt;&#xA;&lt;li&gt;Integration of social networking and game play&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="4302" LastEditDate="2018-12-11T23:16:57.523" LastActivityDate="2018-12-11T23:16:57.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4503" PostTypeId="4" CreationDate="2017-11-12T21:58:28.390" Score="0" Body="For questions related to game design involving AI." OwnerUserId="10843" LastEditorUserId="4302" LastEditDate="2018-12-11T23:16:57.523" LastActivityDate="2018-12-11T23:16:57.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4504" PostTypeId="5" CreationDate="2017-11-12T22:06:46.410" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-11-12T22:06:46.410" LastActivityDate="2017-11-12T22:06:46.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4505" PostTypeId="4" CreationDate="2017-11-12T22:06:46.410" Score="0" Body="For questions related specifically for abstract strategy board game, go." OwnerUserId="10843" LastEditorUserId="10843" LastEditDate="2017-11-12T22:14:33.530" LastActivityDate="2017-11-12T22:14:33.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4506" PostTypeId="2" ParentId="4170" CreationDate="2017-11-13T02:54:48.360" Score="2" Body="&lt;p&gt;This is a common question, but before hand, you need a lot of &lt;strong&gt;label&lt;/strong&gt; data, the more, the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since all the reviews from amazon are not label as deceptive or not, you may manually label them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then, you can use NLP tricks as what describe by Aiden Grossman.&#xA;A simple way is let every word as one of vector and the output is one vector(deceptive or not). A better way is use word2v as input, and lstm as hidden layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beware this &lt;a href=&quot;https://en.wikipedia.org/wiki/Anomaly_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;anomaly detection&lt;/a&gt; problem, that is the deceptive review may extreme seldom happen. Please carefully handle precision and recall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asp.net is OK. You can use your inference model as backend service, and asp.net can call this service with any js framework.&lt;/p&gt;&#xA;" OwnerUserId="10805" LastActivityDate="2017-11-13T02:54:48.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4508" PostTypeId="1" AcceptedAnswerId="4523" CreationDate="2017-11-13T08:23:42.043" Score="1" ViewCount="76" Body="&lt;p&gt;I want to capture text and letters on images (png, jpeg, etc.).  Is it Possible Which algorithm/software can I use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am using &lt;code&gt;R&lt;/code&gt; with the &lt;code&gt;tesseract&lt;/code&gt; package but it's not solving my problem: if letters on images are  different colors and different format, sizes in those situations don't work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have the any training data to compare text on the image with the existing data set. So in those situations is it possible?&lt;/p&gt;&#xA;" OwnerUserId="10862" LastEditorUserId="69" LastEditDate="2017-11-18T20:49:45.997" LastActivityDate="2017-11-18T20:49:45.997" Title="Text Capturing on the Images" Tags="&lt;deep-learning&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4509" PostTypeId="1" AcceptedAnswerId="4513" CreationDate="2017-11-13T08:49:01.553" Score="9" ViewCount="1895" Body="&lt;p&gt;Starting from last year, I have been studying various subjects in order to understand some of the most important thesis of machine learning like&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;S. Hochreiter, &amp;amp; J. Schmidhuber. (1997). &lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Long short-term memory&lt;/a&gt;. Neural Computation, 9(8), 1735-1780. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, due to the fact that I don't have any mathematical backgrounds, I started to learn subjects like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calculus&lt;/li&gt;&#xA;&lt;li&gt;Multivariate Calculus&lt;/li&gt;&#xA;&lt;li&gt;Mathematical Anaylsis&lt;/li&gt;&#xA;&lt;li&gt;Linear Algebra&lt;/li&gt;&#xA;&lt;li&gt;Differential Equations&lt;/li&gt;&#xA;&lt;li&gt;Real Anaylsis (Measure theory)&lt;/li&gt;&#xA;&lt;li&gt;Elementary Probability and Statistics&lt;/li&gt;&#xA;&lt;li&gt;Mathematical Statistics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Right now, I can't say I have done studying those subjects rigorously, but I know what the subjects above want to deal with. The thing is that I don't know what I have to do at this point. There are many subjects that machine learning uses to solve many problems out there and I don't know how to utilize them correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, reinforcement learning is now one of the most popular topic that hundreds of thousands of researchers are now doing their research to make a breakthrough of curse of dimensionality. But, as a future employee who's gonna work in IT companies, the task on the desk wouldn't be something I expected to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it important to have my own expertise to work in the fields? If so, what kinds of subjects do I have to study right now?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your convenience, I want to know more about Markov process and Markov decision process. &lt;/p&gt;&#xA;" OwnerUserId="10860" LastEditorUserId="2444" LastEditDate="2019-02-19T11:31:31.100" LastActivityDate="2021-01-25T12:54:17.797" Title="What do I need to study for machine learning?" Tags="&lt;machine-learning&gt;&lt;getting-started&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="8" ClosedDate="2021-01-21T22:15:49.157" ContentLicense="CC BY-SA 4.0" />
  <row Id="4513" PostTypeId="2" ParentId="4509" CreationDate="2017-11-13T12:43:40.560" Score="5" Body="&lt;p&gt;As a master's degree student in Artificial Intelligence, I strongly advise you to study some basics in Machine Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that, you can get a good book (&lt;a href=&quot;http://www.cs.cmu.edu/~tom/mlbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt;, Tom Mitchell, McGraw Hill, 1997) for the theory and practice by yourself trying some &lt;a href=&quot;https://www.kaggle.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle&lt;/a&gt; competitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggested the book of Mitchell because he is an expert in the field, and lot of Machine Learning courses uses his book. You can also follow his &lt;a href=&quot;http://www.cs.cmu.edu/~tom/10701_sp11/&quot; rel=&quot;nofollow noreferrer&quot;&gt;videolectures online&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Kaggle, you can find many useful tutorials (named as Notebooks) to get started working with the available datasets.&#xA;Some tutorials &lt;a href=&quot;https://www.kaggle.com/c/titanic#tutorials&quot; rel=&quot;nofollow noreferrer&quot;&gt;about Titanic Challenge here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10866" LastEditorUserId="2444" LastEditDate="2019-02-19T11:04:15.253" LastActivityDate="2019-02-19T11:04:15.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4515" PostTypeId="2" ParentId="4035" CreationDate="2017-11-13T17:33:42.980" Score="3" Body="&lt;p&gt;You can flatten the graph into a matrix and then train it like a normal neural network input. Perhaps an adjacency graph or maybe simply a series of linear equations representing the nodes and convert it into matrix form.&lt;/p&gt;&#xA;" OwnerUserId="10874" LastActivityDate="2017-11-13T17:33:42.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4516" PostTypeId="1" CreationDate="2017-11-13T20:47:59.597" Score="1" ViewCount="124" Body="&lt;p&gt;I wish to write a bot that can use screen footage to play a game, specifically for the game 'Nidhogg'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To that end I have determined that a CNN should do the feature detection and a feedforward neural network should determine the action to take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this question I wish to focus on the CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea was to first train the CNN as an autoencoder to aid in unsupervised pattern recognition and to add hidden layers after every epoch, but I am unsure whether this would actually be faster or lead to higher accuracy (as opposed to immediately training all hidden layers at once).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I am unsure is because I imagined the following scenario:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Input &amp;gt; Hidden1 &amp;gt; Output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the output layer needs to approximate the input layer is it not wasteful to let it have its own weights backpropagated when it actually tries to determine the 'inverse' function of the hidden layer? Is it possible to 'directly' determine an inverse function for a layer even if consists of multiple filters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, say I've trained my first hidden layer so that the error is minimal and use it as an input layer whilst I add another hidden layer; can I repeat my strategy?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Input &amp;gt; Hidden1 &amp;gt; Output&#xA;    Input(Hidden1) &amp;gt; Hidden2 &amp;gt; Output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10883" LastActivityDate="2017-12-15T01:23:44.347" Title="Can CNN autoencoders be improved by treating the output layer as an inverted hidden layer?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4520" PostTypeId="2" ParentId="4509" CreationDate="2017-11-14T15:52:32.377" Score="5" Body="&lt;p&gt;Actually, you don't need a rigorous study of these subjects to implement Machine Learning Algorithms. Only Probability Theory needs to be treated rigorously in Machine Learning. You can find a very good series of Probability Theory lectures here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.edx.org/course/introduction-probability-science-mitx-6-041x-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Probability - The Science of Uncertainty&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, a basic course in Calculus would suffice, for basic implementations you don't actually require a full understanding of calculus unless you want to implement a weight update scheme for Neural Nets with something new. But to gain an intuition about Calculus, check out Khan Academy:&#xA;&lt;a href=&quot;https://www.khanacademy.org/search?page_search_query=calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Calculus&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Some basic idea of Linear Algebra is sufficient, just to visualize things and gain an intuition. Khan Academy has a great course on this, I suggest you check it out: &lt;a href=&quot;https://www.khanacademy.org/math/linear-algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear algebra&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As for programming languages, Machine Learning or Neural Nets is best to implement in Python or R, as data visualisation and programming in them is quite easy.&lt;/p&gt;&#xA;&lt;p&gt;The main thing about implementing Neural Nets and Machine learning is practice: the more you practice, the better you get. You'll also get an intuition of what you are doing with practice. Only reading theory and understanding concepts won't help you. You have to implement it in real life.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2021-01-25T12:54:17.797" LastActivityDate="2021-01-25T12:54:17.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4522" PostTypeId="2" ParentId="4516" CreationDate="2017-11-15T01:11:38.733" Score="1" Body="&lt;p&gt;It seems like what you need is reinforcement learning method rather than AE / unsupervised. Method has to be like&#xA;Input -&gt; CNN Feature Extractor (encode side of AE, i.e. VGG) -&gt; Classifier (i.e. DNN) -&gt; Choose one of the actions -&gt; Observe change in the game and check if it was a wrong (like character is dying) or good (character is moving forward) and give a rating to action (feedback of decision). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a while, AI will learn how to jump over obstacles, avoid stuff etc. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-15T01:11:38.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4523" PostTypeId="2" ParentId="4508" CreationDate="2017-11-15T01:19:29.117" Score="0" Body="&lt;p&gt;What you call is a typical OCR application. There are two things to consider :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Localizing the text&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Recognizing the text&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that it is not an easy task to solve the problem globally. However, depending on your working subset, you need to get the data first. There are a lot of open source ocr database that you can find. &#xA;Then, you need to add some preprocessing and it depends on which kind of path you want to choose. You can find lots of articles about it. &#xA;Then, you need to add a localization network (like Faster R-CNN or any specialized one for OCR), with the combination of recognition network. &#xA;This was deep learning approach. It has a learning curve if you have never worked with deep learning frameworks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way that you can take is to get different character image dataset. Resize all of them to same size, flatten to 1-D and train a classifier. Then, you can start to distinguish different image patches if they are character or not (or which character). Then, for inference, you need to crop different sub-regions from input image and feed it to classifier to get a result. &#xA;It is more costly in terms of inference time and the success rate will more likely to be much lower than deep learning method but you can learn &amp;amp; implement this one much quicker. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-15T01:19:29.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4524" PostTypeId="1" AcceptedAnswerId="4528" CreationDate="2017-11-15T08:35:25.953" Score="1" ViewCount="548" Body="&lt;p&gt;I have implemented the hill climbing algorithm, with side away steps, which can increase the rate of success, because, when you don't have new generated states, you can go back to previous level and choose from there another state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do when new generated states have bigger distance to the goal than the parent state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have to go back to the previous level and choose another state, from which I will continue? Or I continue with one of the states generated which has bigger distance?&lt;/p&gt;&#xA;" OwnerUserId="10794" LastEditorUserId="2444" LastEditDate="2019-03-02T10:50:29.347" LastActivityDate="2019-03-02T10:52:27.993" Title="What should I do when the new generated state has bigger distance to the goal than the parent state?" Tags="&lt;search&gt;&lt;hill-climbing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4525" PostTypeId="1" AcceptedAnswerId="4526" CreationDate="2017-11-15T10:03:45.220" Score="2" ViewCount="495" Body="&lt;p&gt;I know what meaning &lt;em&gt;stride&lt;/em&gt; has when it is just an integer number (by which step you should apply a filter to the image). But about &lt;code&gt;(1, 1)&lt;/code&gt; or even more dimensional stride?&lt;/p&gt;&#xA;" OwnerUserId="10929" LastEditorUserId="2444" LastEditDate="2020-06-23T16:27:23.043" LastActivityDate="2020-06-23T16:27:23.043" Title="What is the meaning of a 2D stride?" Tags="&lt;convolutional-neural-networks&gt;&lt;definitions&gt;&lt;stride&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4526" PostTypeId="2" ParentId="4525" CreationDate="2017-11-15T12:48:56.727" Score="3" Body="&lt;p&gt;Your stride will be just an integer number only when doing 1D convolution. When doing 2D convolution you get 2 integers like your example: &lt;code&gt;(1, 1)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When there is more than one dimension to a stride, then each is applied independently in its associated dimension, and all valid combinations that fit in the area or volume should be calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 2D stride defines a rectangular grid of points where an operation is applied. The first number is distance between grid points along first (x) axis, the second number is distance between grid points along second (y) axis. You can decide to downsample each dimension differently by using different numbers for each dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In TensorFlow is common to see 4D stride parameters, with axes of (example, height, width, channel). Typically you don't want to skip any examples or channels in a normal CNN, even though the framework can do that, so you will see values like (1, 2, 2, 1) used when using stride to downsample just image height and width dimensions between layers.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="10823" LastEditDate="2018-10-29T21:53:01.467" LastActivityDate="2018-10-29T21:53:01.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4528" PostTypeId="2" ParentId="4524" CreationDate="2017-11-16T05:14:59.187" Score="0" Body="&lt;p&gt;If you don't go back to your previous state, the best you will do is get stuck in a local maxima. So, to answer your question, yes, backtrack if your hill climbing has failed. The number of times you want to do this depends on you and your implementation. Hill climbing does not guarantee to find the global maxima and, in many cases, you won't know if it is a local maxima or a possible global maxima, unless you try the hill climbing a few times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you haven't already seen &lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_climbing&quot; rel=&quot;nofollow noreferrer&quot;&gt;its Wikipedia article&lt;/a&gt;, it has a cool graph showing what I mean.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastEditorUserId="2444" LastEditDate="2019-03-02T10:52:27.993" LastActivityDate="2019-03-02T10:52:27.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4529" PostTypeId="2" ParentId="4509" CreationDate="2017-11-16T05:27:42.363" Score="3" Body="&lt;p&gt;I found statistical models very helpful. However, statistics on its own isn't enough, you also need a very solid background in probability theory.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastEditorUserId="2444" LastEditDate="2019-02-19T11:04:58.837" LastActivityDate="2019-02-19T11:04:58.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4530" PostTypeId="1" CreationDate="2017-11-16T09:44:50.243" Score="4" ViewCount="118" Body="&lt;p&gt;As we know in call centers there are certain SOP(standard operating procedure), for example few are below,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Agent greeted customer&lt;/li&gt;&#xA;&lt;li&gt;Agent verified the validity of customer before providing sensitive information&lt;/li&gt;&#xA;&lt;li&gt;Agent is able to hold on conversation in decent way&lt;/li&gt;&#xA;&lt;li&gt;Agent closed conversation with nice ending notes&lt;/li&gt;&#xA;&lt;li&gt;Customer got all information what he wanted&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We are planning to transcribe the call records and then from the transcribed text, we want to verify above things(Out of many) from transcribed text using natural language processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the current state of AI technology is possible? If yes can someone give some guidance to approach this type of problem? &lt;/p&gt;&#xA;" OwnerUserId="10955" LastEditorUserId="4302" LastEditDate="2018-10-08T12:22:04.933" LastActivityDate="2020-12-26T19:06:10.623" Title="Is it possible to verify SOP of call center conversation using AI?" Tags="&lt;ai-design&gt;&lt;natural-language-processing&gt;&lt;quality-control&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4532" PostTypeId="1" CreationDate="2017-11-16T15:00:37.470" Score="9" ViewCount="291" Body="&lt;p&gt;I am a PhD student in computer science, and currently creating a state of the art overview in applications done in Machine Ethics (a multidisciplinary field combining philosophy and AI, that looks at creating explicit ethical programs or agents). It seems that the field mostly contains theoretical arguments and there are relatively little implementations, even though there are many people with a technical background in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that because ethics are involved, there is no ground truth and since it's part of philosophy one can get lost in arguing over which type of ethics should be implemented and how this can be done best. However, in computer science, it is usual to even try a simple implementation to show the possibilities or limitations of your approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the possible reasons there is so little done in explicitly implementing ethics in AI and experimenting with it? &lt;/p&gt;&#xA;" OwnerUserId="10958" LastEditorUserId="2444" LastEditDate="2019-06-15T14:46:11.943" LastActivityDate="2019-06-15T14:46:11.943" Title="Why isn't ethics more integrated into current AI systems?" Tags="&lt;philosophy&gt;&lt;ethics&gt;&lt;social&gt;&lt;implementation&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4533" PostTypeId="1" AcceptedAnswerId="4537" CreationDate="2017-11-16T15:48:07.827" Score="8" ViewCount="462" Body="&lt;p&gt;With some knowledge of machine learning and deep learning, it seems very unlikely for AI to develop into the consciousness that we imagine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, consciousness requires a new framework that is very different from what we have today, as current forms of learning seem to be very 'shallow'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What prevents us from developing artificial consciousness?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also, is there a difference between artificial consciousness and sentience?)&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="2444" LastEditDate="2021-01-31T13:56:58.230" LastActivityDate="2021-01-31T14:15:41.023" Title="What prevents us from developing artificial consciousness?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;artificial-consciousness&gt;&lt;self-awareness&gt;&lt;sentience&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4535" PostTypeId="1" AcceptedAnswerId="4538" CreationDate="2017-11-16T19:00:08.317" Score="1" ViewCount="75" Body="&lt;p&gt;I'm reading the A. E. Eiben and J. E. Smith book &lt;em&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/3540401849&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Evolutionary Computing&lt;/a&gt;&lt;/em&gt; (Springer 2003).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On section 3.5 Recombination, page 47, the second paragraph said:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Recombination operators are usually applied probabilistically according to a &lt;strong&gt;crossover rate&lt;/strong&gt; p sub c.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does probabilistically means here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it means that there are a probability that the operator is going to be applied or not.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2019-06-23T23:51:24.600" LastActivityDate="2019-06-23T23:51:24.600" Title="What does &quot;probabilistically&quot; mean?" Tags="&lt;genetic-algorithms&gt;&lt;terminology&gt;&lt;math&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4537" PostTypeId="2" ParentId="4533" CreationDate="2017-11-16T19:21:19.730" Score="5" Body="&lt;p&gt;It may be helpful to think of consciousness, like intelligence, as a spectrum. The Stanford Encyclopedia of Philosophy, under the section &amp;quot;Creature Consciousness&amp;quot; (2.1) defines sentience as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sentience. It may be conscious in the generic sense of simply being a sentient creature, one capable of sensing and responding to its world (Armstrong 1981). Being conscious in this sense may admit of degrees, and just what sort of sensory capacities are sufficient may not be sharply defined. Are fish conscious in the relevant respect? And what of shrimp or bees? &lt;br&gt;&lt;sub&gt;Stanford Encyclopedia of Philosophy, &lt;a href=&quot;http://Stanford%20Encyclopedia%20of%20Philosophy&quot; rel=&quot;nofollow noreferrer&quot;&gt;2.1 &amp;quot;Creature Consciousness&amp;quot;&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It seems clear that AI possesses the most basic form of consciousness in terms of &amp;quot;sensing and responding to its world&amp;quot;.  I might liken this level of consciousness to that of microorganisms, able to interact with their environments but possessing none of the higher functions of humans, or even higher-order animals.&lt;/p&gt;&#xA;&lt;p&gt;We don't really understand the true nature of higher consciousness and human-level sentience, which doesn't necessarily mean that an automata with sufficient complexity couldn't achieve it.  But there is a deeper problem relating to &lt;em&gt;validation&lt;/em&gt; (i.e. how would be know the system is truly self-aware, and not simulating self-awareness.  [See &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room#Chinese_room_thought_experiment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Searle's Chinese Room&lt;/a&gt;, but also &lt;a href=&quot;http://jmc.stanford.edu/articles/chinese.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;McCarthy's Refutation] &lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In his paper &lt;em&gt;Making Robots Conscious of their Mental&#xA;States&lt;/em&gt; McCarthy states:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Conscious knowledge and other information is distinguished from&#xA;unconscious information by being observable, and its observation results&#xA;in conscious knowledge about it. We call this introspective knowledge.&lt;br&gt;&lt;br&gt;&#xA;A robot will need to use introspective knowledge in order to operate&#xA;in the common sense world and accomplish the tasks humans will give&#xA;it.&lt;br&gt;&lt;br&gt;&#xA;Many features of human consciousness will be wanted, some will&#xA;not, and some abilities not possessed by humans have already been&#xA;found feasible and useful in limited domains.&lt;br&gt;&lt;br&gt;&#xA;We give preliminary fragments of a logical language a robot can&#xA;use to represent information about its own state of mind.&lt;br&gt;&lt;br&gt;&#xA;A robot will often have to conclude that it cannot decide a question&#xA;on the basis of the information in memory and therefore must seek&#xA;information externally.&lt;br&gt;&lt;br&gt;&#xA;Programs with much introspective consciousness do not yet exist.&#xA;&lt;br&gt;&lt;sub&gt;&lt;a href=&quot;http://jmc.stanford.edu/articles/consciousness/consciousness.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Making Robots Conscious of their Mental&#xA;States, Abstract&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This still seems to be the case, however, McCarthy seems hopeful about the &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_theory_of_mind&quot; rel=&quot;nofollow noreferrer&quot;&gt;computationalist&lt;/a&gt;&amp;quot; approach:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Thinking about consciousness with a view to designing it provides&#xA;a new approach to some of the problems of consciousness studied by&#xA;philosophers. One advantage is that it focusses on the aspects of&#xA;consciousness important for intelligent behavior. If the advocates of&#xA;qualia are right, it looks like robots won’t need them to exhibit any&#xA;behavior exhibited by humans.&lt;br&gt;&lt;sub&gt;&lt;a href=&quot;http://jmc.stanford.edu/articles/consciousness/consciousness.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ibid.&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The continual increase in memory, processing and algorithmic sophistication may well yield &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Even if this proves to be the case, I suspect the debate over artificial consciousness will rage on.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Consciousness&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-31T14:15:41.023" LastActivityDate="2021-01-31T14:15:41.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4538" PostTypeId="2" ParentId="4535" CreationDate="2017-11-16T19:34:14.943" Score="5" Body="&lt;p&gt;In a genetic algorithm, crossover (recombination) is the analogy to mating in the real world. For example, you have some genetic information from each parent. In the case of an optimization where you have vectors of features (design variables), you could represent it as vector 1 and 2. Imagine each vector has 10 values. You grab the first 5 from vector 1, and the second 5 from vector 2, and combine those into a new vector which is your &quot;offspring&quot;. Note that there are many different ways to do recombination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are deciding on parents to use for recombination, you want to keep some of them the same. Recombination is basically hoping that combining attributes will give you a more optimal set. However, sometimes, you might want to make some of the offspring be copies of the parents in the next generation. So, as &lt;a href=&quot;http://www.obitko.com/tutorials/genetic-algorithms/parameters.php&quot; rel=&quot;noreferrer&quot;&gt;this link&lt;/a&gt; explains, the crossover probability determines what percentage of offspring experience recombination. It is probabalistic because there is an element of chance involved in whether or not given offspring experience recombination.&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2017-11-16T19:34:14.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4539" PostTypeId="5" CreationDate="2017-11-16T20:35:44.147" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Artificial consciousness (AC), also known as machine consciousness (MC) or synthetic consciousness (Gamez 2008; Reggia 2013), is a field related to artificial intelligence and cognitive robotics. The aim of the theory of artificial consciousness is to &amp;quot;Define that which would have to be synthesized were consciousness to be found in an engineered artifact&amp;quot; (Aleksander 1995).&lt;/p&gt;&#xA;&lt;p&gt;Neuroscience hypothesizes that consciousness is generated by the interoperation of various parts of the brain, called the neural correlates of consciousness or NCC, though there are challenges to that perspective. Proponents of AC believe it is possible to construct systems (e.g., computer systems) that can emulate this NCC interoperation.&lt;/p&gt;&#xA;&lt;p&gt;Artificial consciousness concepts are also pondered in the philosophy of artificial intelligence through questions about mind, consciousness, and mental states.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;(source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Consciousness (Wikipedia)&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="4709" LastEditDate="2020-07-14T15:13:09.560" LastActivityDate="2020-07-14T15:13:09.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4540" PostTypeId="4" CreationDate="2017-11-16T20:35:44.147" Score="0" Body="For questions related to artificial consciousness (aka synthetic consciousness or machine consciousness).  Largely synonymous with &quot;Self Awareness&quot;, but restricted to engineered artifacts (AIs)." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-31T13:59:25.900" LastActivityDate="2021-01-31T13:59:25.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4541" PostTypeId="5" CreationDate="2017-11-16T20:39:03.717" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sentience&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Sentience&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-16T20:39:03.717" LastActivityDate="2017-11-16T20:39:03.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4542" PostTypeId="4" CreationDate="2017-11-16T20:39:03.717" Score="0" Body="Use this tag for questions on artificial sentience, particularly in regard to AGI or superintelligence.  For instance &quot;Are the robots on Westworld sentient because they have the ability to suffer or is their ability to feel suffering what makes them sentient?&quot;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-16T20:39:03.717" LastActivityDate="2017-11-16T20:39:03.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4543" PostTypeId="5" CreationDate="2017-11-16T20:42:01.763" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Negamax&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Negamax&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-16T20:42:01.763" LastActivityDate="2017-11-16T20:42:01.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4544" PostTypeId="4" CreationDate="2017-11-16T20:42:01.763" Score="0" Body="Use this for the negamax variant of minimax.  max(a,b) = -min(-a,-b)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-16T20:42:01.763" LastActivityDate="2017-11-16T20:42:01.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4545" PostTypeId="5" CreationDate="2017-11-16T20:45:41.630" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Minimax&lt;/a&gt; &lt;br&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Minimax_theorem&lt;/a&gt; &lt;br&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Wald%27s_maximin_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Wald%27s_maximin_model&lt;/a&gt; &lt;br&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-16T20:45:41.630" LastActivityDate="2017-11-16T20:45:41.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4546" PostTypeId="4" CreationDate="2017-11-16T20:45:41.630" Score="0" Body="Use for questions involving minimax and variants such as maximin.  Applies both to algorithms and the minimax theorem. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-16T20:45:41.630" LastActivityDate="2017-11-16T20:45:41.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4548" PostTypeId="2" ParentId="4533" CreationDate="2017-11-17T16:39:11.637" Score="2" Body="&lt;p&gt;By the cognitive science / psychology literature, we don't have a theory of consciousness, or a measure of sentience. By its very nature, &quot;things&quot; like those can only be understood with respect to ones own subjective experience (as in you can only understand consciousness by comparing it to your own conscious experience).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So no, even if we have a device that perfectly replicates the function of consciousness or sentience, we'd still have a hard time saying that it conclusively is indeed consciousness.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-11-17T16:39:11.637" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4549" PostTypeId="2" ParentId="4494" CreationDate="2017-11-17T16:41:22.277" Score="1" Body="&lt;p&gt;The Unconscious in psychology is related to many aspects of &lt;strong&gt;behavior&lt;/strong&gt;, but many levels of this exist, including low levels (simple tasks) like walking, eating, breathing etc.  This has been investigated in animal world, for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Muscle_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;muscular memory&lt;/a&gt;, the &lt;a href=&quot;https://www.nature.com/articles/nature23455&quot; rel=&quot;nofollow noreferrer&quot;&gt;insect brain model&lt;/a&gt;, and &lt;a href=&quot;https://www.uni-obuda.hu/journal/Zilkova_Timko_Girovsky_8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nonlinear System Control Using Neural Networks &lt;/a&gt;. Many of these are related to body control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5573739/figure/F1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nested hierarchies of circular emotional control and cognitive regulation for “bottom-up” influences and “top-down” regulations&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="10983" LastEditorUserId="1671" LastEditDate="2017-11-17T19:52:55.357" LastActivityDate="2017-11-17T19:52:55.357" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4552" PostTypeId="2" ParentId="4532" CreationDate="2017-11-18T22:37:54.467" Score="1" Body="&lt;p&gt;I feel part of the problem as to why there is very little in the way of ethical implementations of AI/ML technologies, is simply because there is no need or proper application of the theoretical frameworks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By this I mean, there are no substantial ways we can apply this understanding to algorithms and models that cannot interact in a meaningful way. We have such a large theoretical framework on AI safety/ethics because it is extremely important. We need to come up with safe guidelines for implementing strong AI &lt;strong&gt;&lt;em&gt;before&lt;/em&gt;&lt;/strong&gt; it is created. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some very focused papers have started to narrow down the issues in creating ethical/safe AI systems. See &lt;a href=&quot;https://arxiv.org/abs/1606.06565&quot; rel=&quot;nofollow noreferrer&quot;&gt;Concrete Problems in AI Safety&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-11-18T22:37:54.467" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4555" PostTypeId="2" ParentId="4532" CreationDate="2017-11-19T08:25:18.267" Score="1" Body="&lt;p&gt;With the imitation method, the most appropriate behavior can be integrated into artificial intelligence. Artificial intelligence can be reshaped when the ethical position changes. It is used for ideological purpose or to gather information. It's not clear what the robot is.&lt;/p&gt;&#xA;" OwnerUserId="10845" LastActivityDate="2017-11-19T08:25:18.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4557" PostTypeId="2" ParentId="4298" CreationDate="2017-11-20T00:06:31.940" Score="1" Body="&lt;p&gt;There is no reason why you can't have a hill descending algorithm, instead of finding maxima you will find minima.  If that is what your aim is, it's still called a hill climbing algorithm, I guess...&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2017-11-20T00:06:31.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4559" PostTypeId="2" ParentId="4532" CreationDate="2017-11-20T03:01:58.510" Score="1" Body="&lt;p&gt;We can take the error model into accountant. Recognising bias and variance among the performance under neural networks can be a first step. And then we can discuss whether such performance is allowed. As far as we know, practicing ethnics requires empirical and field study. we cannot simply take rationales and paper essays to determine the doings of learnt machines is wrong or not. It can be further divided into accidents, errors , or even bugs created from  the developers. &lt;/p&gt;&#xA;" OwnerUserId="11015" LastActivityDate="2017-11-20T03:01:58.510" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4560" PostTypeId="1" CreationDate="2017-11-20T06:06:19.840" Score="1" ViewCount="35" Body="&lt;p&gt;&lt;strong&gt;If some claim they are doing Artificial Intelligence by providing a Voice Assistant, what shall we expect from this Voice Assistant for the claim of doing &quot;Artificial Intelligence&quot; to be correct?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of potential qualifications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is replying &quot;Hello&quot; to &quot;Hello&quot; enough?&lt;/li&gt;&#xA;&lt;li&gt;is repeating any word you said enough?&lt;/li&gt;&#xA;&lt;li&gt;is identifying you by your voice enough?&lt;/li&gt;&#xA;&lt;li&gt;shall it support corrections? &quot;&lt;em&gt;what is my favorite color?&quot;/&quot;red!&quot;/&quot;no, it's blue. What is my favorite color?&quot;/&quot;blue!&lt;/em&gt;&quot;&lt;/li&gt;&#xA;&lt;li&gt;shall it support emotions? &quot;&lt;em&gt;I would offer you better service if you could say thank you sometimes&lt;/em&gt;&quot;&lt;/li&gt;&#xA;&lt;li&gt;shall it support self-awareness? &quot;&lt;em&gt;Dear master, may I ask you a question about me?&lt;/em&gt;&quot;&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11018" LastActivityDate="2017-11-20T22:10:49.863" Title="What qualifies a Voice Assistant to be an AI?" Tags="&lt;definitions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4564" PostTypeId="2" ParentId="4560" CreationDate="2017-11-20T22:10:49.863" Score="1" Body="&lt;p&gt;Usually, this sort of encoded 1to1 response is &lt;em&gt;not&lt;/em&gt; considered to be AI or ML. This is because a chatbot that simply responds to a designated string with a response string is hard programmed(for the most part). A lot of companies/individuals have tried to package this as true AI but it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the simpler examples of an AI chatbot system is a model that does some sort of tree-search or similar for the input. This is more often than not, extremely naive but would be an example of a true AI system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to more concisely answer your question, the categorization as an AI chatbot would have less to do with the responses, and more to do with how the model selects/comes up with those responses. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-11-20T22:10:49.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4570" PostTypeId="1" CreationDate="2017-11-21T21:32:19.650" Score="2" ViewCount="71" Body="&lt;p&gt;I am a newbie in the field of AI/ML. I am trying to implement predictive analytics model on the data generated and collected every minute from a device with sensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are various ML algorithms I can use to predict the number of&#xA;people in a room given temperature, humidity, luminosity, and motion&#xA;{ 0 | 1 }?&lt;/li&gt;&#xA;&lt;li&gt;What other things can we predict using the above data from the&#xA;sensor that is deployed in a closed room?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Context:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The device sends temperature, humidity, luminosity, and motion(yes/no) in real-time. I deployed this device in a closed room and started collecting data. Now I want to use this data to predict the number of people in the room using the data collected. I believe a multiple (linear/poly) regression model will help me in achieving this but, wanted to know if there are any other algorithms or any other use cases I can look into.&lt;/p&gt;&#xA;" OwnerUserId="11055" LastEditorUserId="11055" LastEditDate="2017-11-22T02:22:51.250" LastActivityDate="2017-11-22T02:22:51.250" Title="What are suitable predictive analytics models for data from multiple sensors?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;linear-regression&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4571" PostTypeId="1" CreationDate="2017-11-21T21:34:49.517" Score="6" ViewCount="1928" Body="&lt;p&gt;I am using Google's OCR to extract text from images, like receipts and invoices.&lt;/p&gt;&#xA;&lt;p&gt;Whare examples of techniques used to make sense of the text? For example, I would like to extract the date, name of the business, address, total amount, etc.&lt;/p&gt;&#xA;&lt;p&gt;Before marking this question &amp;quot;too broad&amp;quot;, if someone can please direct me to the right set of algorithms the industry uses for machine learning will be great.&lt;/p&gt;&#xA;" OwnerUserId="11056" LastEditorUserId="2444" LastEditDate="2021-01-08T01:19:12.493" LastActivityDate="2021-01-08T01:19:12.493" Title="Effective algorithms for OCR" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;optical-character-recognition&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4572" PostTypeId="2" ParentId="4571" CreationDate="2017-11-22T03:07:13.580" Score="1" Body="&lt;p&gt;An interesting question, I think the algorithm used for OCR is &quot;Logistic Regression&quot; or &quot;Decision Tree&quot; in multiple steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steps can be&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Image Classification - In this step, the images are classified into&#xA;&quot;with or without&quot; text.&lt;/li&gt;&#xA;&lt;li&gt;Text Detection - In this step, the images with text are taken&#xA;divided into blocks and the blocks are classified into &quot;with or&#xA;without&quot; text.&lt;/li&gt;&#xA;&lt;li&gt;Character Detection - In this step, the blocks with text are taken&#xA;and divided into smaller boxes of single characters and compare with&#xA;a database of characters.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The database is built using the crowdsourced &quot;captcha&quot; project.&lt;/p&gt;&#xA;" OwnerUserId="11055" LastActivityDate="2017-11-22T03:07:13.580" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4573" PostTypeId="2" ParentId="4571" CreationDate="2017-11-22T04:36:46.343" Score="4" Body="&lt;p&gt;Last semester I, along with my team, made a project on OCR.&lt;br&gt;&#xA;&lt;strong&gt;Note&lt;/strong&gt;: I am assuming the data set for your pictures has white&#xA;background with black (or some other dark text) on it.&lt;br&gt;&#xA;These are the overview steps we followed:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4swxQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4swxQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;br&gt;&#xA;&lt;strong&gt;Pre-processing&lt;/strong&gt; includes grayscale conversion, noise reduction,&#xA;binarization and skew detection.&lt;br&gt;&#xA;Next step was &lt;strong&gt;segmentation&lt;/strong&gt;. This process extracts the individual&#xA;characters from the image. &lt;em&gt;Histogram&lt;/em&gt; taken along the y-axis divided&#xA;the image into lines. This is followed by histogram along the x-axis&#xA;which divided them into words and further into characters.&#xA;At the end of the step, we used Savgol filter to smooth the curves of the histogram.&lt;br&gt;&lt;br&gt;&#xA;Next step was &lt;strong&gt;feature extraction&lt;/strong&gt;. This is the most important step.&#xA;The accuracy of your code depends on how well your features are.&lt;br&gt;&#xA;We used the following features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Crossing: Counting number of transitions between foreground and background. We used&#xA;two diagonal lines, two horizontal and one vertical line. You can used any number you want.&lt;/li&gt;&#xA;&lt;li&gt;Zoning: Whole character region is divided into 16 zones, and density of each zone is measured.&lt;/li&gt;&#xA;&lt;li&gt;Projection Histogram: Each character has unique (almost) vertical and horizontal histogram signature.&lt;/li&gt;&#xA;&lt;li&gt;Other features include number of endpoints in the character, number of loops and horizontal/vertical line count.&lt;br&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We used three different classification algorithms for our project. They were KNN (K-Nearest Neighbours), Artificial Neural Network (ANN) and Extra Tree classification. &lt;strong&gt;Their F1 score was 0.84, 0.82 and 0.77 respectively&lt;/strong&gt;.&lt;br&gt;&#xA;For training, you will need to find datasets. Many data sets for OCR was available online. Make sure you are using good ones.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2017-11-22T04:36:46.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4574" PostTypeId="1" AcceptedAnswerId="4630" CreationDate="2017-11-22T08:04:29.000" Score="2" ViewCount="198" Body="&lt;p&gt;I have started on Andrew Ng's machine learning course. It seems that machine learning is learning correlations with known data based on as many parameters as possible. For example, if we collect data on existing property prices with information on the land area, built-in area, type of building, age of the building, etc, it is possible to predict the price of another property if we input the value of the various parameters of this property.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, if we keep the images (the black and white pixels) of cats, we can tell whether a new picture is a cat if it bears some resemblance to the pixels of existing labeled cat images.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;This approach sounds great, but is it practical? How much effort and zettabytes of data do we have to keep just to reach the brainpower of, say, a 3-year old, who can recognize dogs, cats, tigers, a Mustang, trucks, a hamburger restaurant, and so on?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why does everyone have to repeat the effort of learning the same things?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If Google has already learned cats, or if someone already has a program to recognize handwritten digits, can this knowledge be shared and re-used? Or is it just a matter of paying for them?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11064" LastEditorUserId="2444" LastEditDate="2020-06-24T00:15:05.683" LastActivityDate="2020-06-24T00:15:05.683" Title="Why do we have to train a model from scratch every time?" Tags="&lt;machine-learning&gt;&lt;transfer-learning&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="1" ClosedDate="2020-06-24T00:12:52.147" ContentLicense="CC BY-SA 4.0" />
  <row Id="4575" PostTypeId="2" ParentId="4574" CreationDate="2017-11-22T08:50:19.127" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;...why does everyone have to repeat the effort of learning the same things...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most problems are different if you go into the details. Even if the problems were exactly identical, implementation differs. You might like Google Chrome, but I might prefer Mozilla Firefox. They both load the Internet pages!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...How much effort and zetabytes of data do we have to keep just to reach the brain power of, say, a 3-year old...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That depends on what you want to do. A 3-year kid can recognise people quite well, but this is hard for computers. However, computer can do regression much better than a 3-year old ...&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-11-22T08:50:19.127" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4576" PostTypeId="1" AcceptedAnswerId="4605" CreationDate="2017-11-22T17:17:44.173" Score="2" ViewCount="462" Body="&lt;p&gt;&quot;Deep Learning&quot; neural networks are now successful at image-recognition tasks that I would not have expected say 10 years ago.  I wonder if the current state of the art in machine learning could generally tell the difference between the sound of a dog or cat moving around a house, and a person walking in the same area, taking as input only the sound captured by a microphone. I think I could generally tell the difference, but it is hard to explain exactly how.  But this is also true of some tasks that deep learning is now succeeding at.  So, I suspect it is possible but it's not clear how you would go about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found algorithms to detect human speech (wikipedia:&quot;Voice activity detection&quot;) but separating animal and human footsteps seems more subtle.&lt;/p&gt;&#xA;" OwnerUserId="11078" LastEditorUserId="11078" LastEditDate="2017-11-22T22:52:45.540" LastActivityDate="2017-12-02T21:30:58.313" Title="Could a CNN hear the difference between sound of a pet moving, and a person?" Tags="&lt;deep-learning&gt;&lt;pattern-recognition&gt;&lt;security&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4577" PostTypeId="1" CreationDate="2017-11-22T19:26:29.110" Score="1" ViewCount="18" Body="&lt;p&gt;Normally when doing a fit to some time series data (e.g., a polynomial fit), functions will return an associated error with each fitted point. I'm now trying out scikit-learn's support vector regression (SVR) fitting instead, which doesn't have any such return. There is a handy function in scikit-learn called validation_score that can tell me the accuracy score of various fits, from which I select the best one. So maybe that's good enough? But as a scientist, I'm wondering if this will get slapped down in peer review of a publication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the right way to propagate the errors of my time series data through the SVR fit?&lt;/p&gt;&#xA;" OwnerUserId="11080" LastActivityDate="2017-11-22T19:26:29.110" Title="Should I be using a validation curve accuracy score in lieu of error bars for a model fit to time series data?" Tags="&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4578" PostTypeId="2" ParentId="3993" CreationDate="2017-11-23T01:44:34.267" Score="0" Body="&lt;p&gt;This is an open area of research, but it seems to be considered to be untenable given current methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://openai.com/requests-for-research/#description2code&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is relevant, and provides a link to a training set&lt;/a&gt;, but doesn't provide much more beyond that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, an assumed requisite of this task is meaning comprehension, which we don't seem to be much closer towards given current methods.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2017-11-23T01:50:21.103" LastActivityDate="2017-11-23T01:50:21.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4579" PostTypeId="2" ParentId="4218" CreationDate="2017-11-23T01:50:54.147" Score="3" Body="&lt;p&gt;My money would be on something much simpler like &lt;a href=&quot;https://en.wikipedia.org/wiki/Naive_Bayes_classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;Naive Bayes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience for small data NB outperforms the more exotic methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you want to get more value out of your training data, try &lt;a href=&quot;https://www.openml.org/a/estimation-procedures/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;10 fold cross validation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9413" LastEditorDisplayName="user10259" LastEditDate="2018-08-19T01:23:40.097" LastActivityDate="2018-08-19T01:23:40.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4580" PostTypeId="2" ParentId="4532" CreationDate="2017-11-23T02:15:15.647" Score="1" Body="&lt;p&gt;Intuitively speaking, it seems to be the case that there is little research into the implementation of AI ethics because:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Society as a whole seems to comfortably agree that the current state of machine intelligence is not strong enough for it to be considered as conscious or sentient. Thus we don't need to give it ethical rights (yet).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Implementation of ethical behaviour into a program requires a method for computers to be capable of interpreting &quot;meaning&quot;, which we do not know how to do yet.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-11-23T02:15:15.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4581" PostTypeId="1" AcceptedAnswerId="4586" CreationDate="2017-11-23T03:02:05.923" Score="8" ViewCount="1569" Body="&lt;p&gt;I've been reading a lot about TD-Gammon recently as I'm exploring options for AI in a video-game I'm making. The video game is a turn-based positional sort of game, i.e. a &quot;units&quot;, or game piece's, position will greatly impact it's usefulness in that board state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To work my way towards this, I thought it prudent to implement a Neural Network for a few different games first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea I like is encoding the board state for the Neural Network with a single output neuron which gives that board states relative strength compared to other board states. As I understand, this is how TD-Gammon worked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I look at other people's code and examples/tutorials, there seems to be a lot of variance in the way they represent the board-state. Even for something as simple as tic-tac-toe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So; specifically for tic-tac-toe, which is a better, or what is the correct representation for the  board state? I have seen:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;9 input neurons, one for each square. A &lt;code&gt;0&lt;/code&gt; indicating a free-space, &lt;code&gt;-1&lt;/code&gt; the opponent and &lt;code&gt;1&lt;/code&gt; yourself.&lt;/li&gt;&#xA;&lt;li&gt;9 input neurons, but using different values such as &lt;code&gt;0&lt;/code&gt; for the opponent, &lt;code&gt;0.5&lt;/code&gt; for free and &lt;code&gt;1&lt;/code&gt; for yourself?&lt;/li&gt;&#xA;&lt;li&gt;Could you use larger values? LIke &lt;code&gt;0&lt;/code&gt;, &lt;code&gt;1&lt;/code&gt; and &lt;code&gt;2&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;27 input neurons. The first 3 being square 1, the next 3 being square 2 etc. Every neuron is &lt;code&gt;1&lt;/code&gt; or &lt;code&gt;0&lt;/code&gt;. The first of the set of three indicates whether this square is free or not; the second indicating whether the square is occupied by your opponent or not. In the end, only one in every 3 neurons will have a &lt;code&gt;1&lt;/code&gt;, the other two will have a &lt;code&gt;0&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;18 input neurons. The first being &lt;code&gt;1&lt;/code&gt; for the X player, the second being &lt;code&gt;1&lt;/code&gt; for the O player and both being &lt;code&gt;0&lt;/code&gt; for a blank&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then; when branching into games where the specific pieces abilities come into play, like in chess, how would you represent this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be as simple as using higher input values for more valuable pieces? I.e. &lt;code&gt;-20&lt;/code&gt; for an opponents Queen and &lt;code&gt;+20&lt;/code&gt; for your own queen? Or would you need something more complex where you define 10+ values for each square, one for each unit-type and player combination?&lt;/p&gt;&#xA;" OwnerUserId="2819" LastEditorUserId="32410" LastEditDate="2021-04-23T19:20:26.910" LastActivityDate="2021-04-23T19:20:26.910" Title="How to represent Tic-Tac-Toe vs Checkers or Chess for a Neural Network" Tags="&lt;neural-networks&gt;&lt;combinatorial-games&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4583" PostTypeId="1" AcceptedAnswerId="4590" CreationDate="2017-11-23T16:38:53.243" Score="1" ViewCount="554" Body="&lt;p&gt;I have an AI class this semester. For our exam, we also cover alpha-beta pruning.&lt;/p&gt;&#xA;&lt;p&gt;I found an old example, where I think we can stop already earlier.&#xA;Here is a picture of it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VMyg3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VMyg3.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think, because &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; wants to maximize his win, and finds 10, he knows that he cannot get better. Therefore he cuts and puts 10.&lt;/p&gt;&#xA;&lt;p&gt;I marked my &amp;quot;improvement&amp;quot; with red.&lt;/p&gt;&#xA;" OwnerUserId="11098" LastEditorUserId="2444" LastEditDate="2020-07-08T21:00:59.997" LastActivityDate="2020-07-08T21:00:59.997" Title="Can I prune the tree if alpha-beta pruning finds 10 to the left of the root node?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4586" PostTypeId="2" ParentId="4581" CreationDate="2017-11-24T00:25:09.480" Score="3" Body="&lt;p&gt;When you are working with neural networks, as long as the data is there, the network is usually able to learn how to process it into a useful result, you usually also want to keep the amount of weights to a minimum. When you you use extra weights, it will take longer to train the network because you need to tune even more values for an optimal network. So, for tic-tac-toe, any of your solutions involving 9 inputs should work just fine. Also, it helps if you keep the inputs between 0 and 1 if you are using log sigmoid, and -1 and 1 if you are using hyperbolic tangent for your activation function. You can probably easily figure out what to use for other activation functions. You can take your data and transform it into another dataset with values within a specific range through a process called range normalization. For chess, you can simply encode every piece in several different ways, and it will probably not make that much of a difference. The general rule of thumb is &lt;strong&gt;you want to minimize the amount of weights while still giving the most possible variables to the network&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2017-11-24T00:25:09.480" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4589" PostTypeId="2" ParentId="4583" CreationDate="2017-11-24T11:17:16.330" Score="0" Body="&lt;p&gt;So I assumed that we know that the maximum value overall is 10. But we do not know that, therefore we need to continue and cannot cut off the way I was thinking. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although we would know that the max. value overall is 10, then we can cut off there. &lt;/p&gt;&#xA;" OwnerUserId="11098" LastActivityDate="2017-11-24T11:17:16.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4590" PostTypeId="2" ParentId="4583" CreationDate="2017-11-24T11:20:22.777" Score="2" Body="&lt;p&gt;The vanilla Alpha-Beta Pruning algorithm as it has been taught to you in class does not assume any domain knowledge / knowledge about the game / knowledge about the tree it is searching. Therefore, if it immediately finds a score of 10 directly to the left of the root node, it can not prune yet, because... maybe there's a score of 20 somewhere else in the tree?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your description, you seem to assume prior knowledge that the maximum score that can possibly be obtained in the entire game is 10. This is not typically assumed to be available knowledge ahead of time, at least not in the original formulation of Alpha-Beta. That is why in pseudocode of the algorithm (for example on &lt;a href=&quot;https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;), they initialize &lt;code&gt;alpha&lt;/code&gt; and &lt;code&gt;beta&lt;/code&gt; at minus and plus infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are right though. If you have better estimates of the upper and lower bounds on scores that can every be achieved throughout the entire game, you can use better initializations of &lt;code&gt;alpha&lt;/code&gt; and &lt;code&gt;beta&lt;/code&gt; than minus and plus infinity. &lt;code&gt;alpha&lt;/code&gt; and &lt;code&gt;beta&lt;/code&gt; would have to be &lt;code&gt;-11&lt;/code&gt; and &lt;code&gt;11&lt;/code&gt; if the true bounds are &lt;code&gt;-10&lt;/code&gt; and &lt;code&gt;10&lt;/code&gt;. I suspect you'd also have to make some additional changes to the typical pseudocode to actually make use of this though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more complex games than Tic Tac Toe, you're often not going to have very useful knowledge of better bounds though. Typically you can't afford to search all the way to terminal game states, you'll have to stop the search earlier and use heuristic evaluation functions to estimate scores. If you do that, you'll often want to assign really high constants (like +/- 1Million) to actual terminal nodes (wins and losses), so that all your heuristic scores for non-terminal nodes can be somewhere in between. If you have such a large range of possible scores, your idea stops having much value.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2017-11-24T11:20:22.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4591" PostTypeId="1" CreationDate="2017-11-24T13:56:56.240" Score="8" ViewCount="187" Body="&lt;p&gt;I'm building a neural net to predict the value of a piece of art with a wide range of inputs (size, art medium, etc.) and I would like to include the author as an input as well (it is often a huge factor in the value of a single piece of art).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current concern is that the name of the author isn't an ideal numerical input for a NN (i.e. If I just code each author with an increasing integer value I will be indirectly assigning more value to authors further down the list -_-). My thoughts were to create separate inputs for all the authors in my data set and then just use one hot encoding to better represent the input to the NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach however runs into a problem when an author that is not included in my training data is used as an input to the NN (i.e. a new author). I can get around this with an &quot;other author&quot; input field but I am worried that this won't be accurate as I would not have trained the the NN for this input (all pieces of art with a valuation have an author).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't fully thought this through but I thought of perhaps training 2 NN's, one for a valuation without an author and one for valuation with an author to ensure I have enough training data for an &quot;authorless valuation&quot; to still be reasonably accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am still trying to conceptualize the best NN architecture before I get stuck into the implementation so if anyone has any suggestions/comments I would be very grateful!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance, Vince&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I am doing this as a small competition with a friend to test a NN vs the traditional commercial valuation techniques. Please help me get a win for Computer Science over Actuarial Science.&lt;/p&gt;&#xA;" OwnerUserId="11111" LastActivityDate="2018-07-31T09:36:38.867" Title="Neural network architecture for author name as an input?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4592" PostTypeId="2" ParentId="4591" CreationDate="2017-11-24T14:52:54.360" Score="4" Body="&lt;p&gt;The most straightforward approach I would recommend would be the one-hot encoding solution without a feature for ''other author''. If you use drop-out during training, the network should learn how to deal with input vectors that don't have any author-features set to 1. Then, whenever you have an unknown author, you simply have no 1s at all for any of the author-features, and it should still have learned how to deal with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possible approach would be a one-hot encoding &lt;em&gt;with&lt;/em&gt; a feature for ''other author''. In order to still get training data for that weight, you could simply use data augmentation. The most straightforward data augmentation approach for this would be to create copies of the instances in your training data, but for those copies set the ''other author'' feature to 1 instead of the actual author.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most complex solution I can think of would be taking the string representation of the author, trying to have your program find information about that author online (for example try to look up a wikipedia page), push all that text through an LSTM. The ''output layer'' of the LSTM can then be merged with your other features (those two together will be your ''input vector''), stack a few more layers on top of that, and train the entire thing end-to-end. Unless the competition with your friend involves a serious amount of money, this is probably going to be too complicated to be worth it though.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2017-11-24T14:52:54.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4593" PostTypeId="2" ParentId="4581" CreationDate="2017-11-24T15:48:53.553" Score="2" Body="&lt;p&gt;The basis of reinforcement learning methods is to give each (game) state (or action) a value that somehow represent how good that state (or action) is. To store these values we could use something as simple as a table/hashmap, however complex games like chess or go has so many states they cannot fit into the memory. As a remedy we think of the hashmap as a function and try to approximate it with a neural network(NN). Luckily NNs are universal approximators, which means &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;they can learn any function&lt;/a&gt;, including an arbitrary mapping from a chess board to a number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question is how to represent a game board and feed it to the neural network. In the case of tic-tac-toe all 5 methods you listed can be considered correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically it does not matter what &lt;code&gt;(w, b, v)&lt;/code&gt; numbers are assigned to (white, black or vacant tiles), if we teach the NN that &lt;code&gt;f(w, w, v, ...) = 1&lt;/code&gt; enough times, it will learn this association whether it is &lt;code&gt;(w, b, v) = (0, -1, 1)&lt;/code&gt; or &lt;code&gt;(w, b, v) = (0, 0.5, 1)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your first three examples uses this method, however a small &lt;em&gt;flaw&lt;/em&gt; here is that it assigns numbers to nominal things, that is numbers can be ordered, yet we cannot really say that &lt;code&gt;black &amp;gt; white &amp;gt; vacant&lt;/code&gt;. Your last two examples try to fix this by using &lt;a href=&quot;https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;one-hot-vectors&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a game like chess, if we used numbers simply to represent the figures the NN might mistakenly mix up two figure types (eg.: pawn is &lt;code&gt;19&lt;/code&gt;, queen is &lt;code&gt;20&lt;/code&gt; and it thinks that a queen is trying to attack your king whereas it is just a pawn) and make a bad decision. However it will learn that the decision was bad and will assign the correct value to the state and the decision in the long run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last note: choosing the correct state representation for a problem is a crucial part of reinforcement learning (similar to picking the right features for a classification problem) and sometimes one might be too afraid to pick a very high dimensional state space. But remember, that chess is not a simple game so a large state space may not be unreasonable. Also for reference, &lt;a href=&quot;https://www.nature.com/articles/nature14236.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atari games&lt;/a&gt; were trained with an input dimension of &lt;code&gt;84*84*4&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastActivityDate="2017-11-24T15:48:53.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4594" PostTypeId="1" AcceptedAnswerId="4597" CreationDate="2017-11-24T17:59:59.923" Score="2" ViewCount="425" Body="&lt;p&gt;I'd like to explore possibilities of applying deep learning on image noise reduction problem, more on photographic camera noise. What's a good NN architecture to solve problems like this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 25,Nov,2017:&#xA;I have a small dataset of clean/noisy reference (~15K 4Kres images) acquired from digital camera. The target is to denoise other images from this camera type but without a reference photo.&lt;/p&gt;&#xA;" OwnerUserId="8499" LastEditorUserId="8499" LastEditDate="2017-11-25T17:56:28.793" LastActivityDate="2017-11-25T17:56:28.793" Title="Applying NNs to 2d image noise reduction?" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4595" PostTypeId="2" ParentId="4583" CreationDate="2017-11-25T01:09:46.530" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://inst.eecs.berkeley.edu/~cs61b/fa14/ta-materials/apps/ab_tree_practice/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://inst.eecs.berkeley.edu/~cs61b/fa14/ta-materials/apps/ab_tree_practice/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can make practice for better understanding the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And i also recommend this lecture from MIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=STjW3eH0Cik&amp;amp;index=6&amp;amp;list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=STjW3eH0Cik&amp;amp;index=6&amp;amp;list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11118" LastActivityDate="2017-11-25T01:09:46.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4596" PostTypeId="2" ParentId="4591" CreationDate="2017-11-25T08:28:25.820" Score="3" Body="&lt;p&gt;I would try to find some proxy features &lt;em&gt;about&lt;/em&gt; the author, as opposed to encode the identity of the author. Likely good features of an author include averages of other features about the work (such as size, media types etc), and critically for your prediction, some stats on previous sale prices of any works. Remember for historical training data to include the proxy data as it would of appeared at the time of sale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having these proxies is likely to work much better than one-hot-encoding an author identity and getting the ML algorithm to assign weights based on the identity alone, especially for authors with only small amounts of data to represent them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is little reason as far as I know, to expect the character string of an author's name to correlate with value of the artwork. There &lt;em&gt;may&lt;/em&gt; be some good vs bad &lt;em&gt;brand name&lt;/em&gt; effect, but I think that will be very hard to predict even in isolation from other factors, and including historic sales data should take account of this and similar effects, except for artists with no known history.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2017-11-25T08:36:50.553" LastActivityDate="2017-11-25T08:36:50.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4597" PostTypeId="2" ParentId="4594" CreationDate="2017-11-25T16:11:40.937" Score="0" Body="&lt;p&gt;You might want to look into &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;autoencoders&lt;/a&gt;. What they do is learn the relationship between the input and a simplified version of it, and then try to reconstruct the original version of the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news about this architecture is that it is not a supervised learning technique, so you can feed it a lot of data without having to construct both input and output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully, if you train it right, it will learn that the parts of the image that are not important in the result are the noisy parts that you intend to remove.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that autoencoders had already been used to reduce noise in audio and image with great success. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; Here's a quick not-so-technical explanation on why they work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the general architecture that auto-encoder networks have, you'll usually find an input layer, a certain amount of hidden layers which usually have less nodes than the input layer, and then a certain amount of hidden layers that gradually grow to have the same amount of input layers in the output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SEYOh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SEYOh.png&quot; alt=&quot;Autoencoder network structure example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why they learn to reproduce the input in the output layer, the data representation needs to necessarily go through the narrow middle layer -- at that point, you can consider that the input has been &quot;compressed&quot;, usually with a lossly approach to have a representation of the data that can later on use to reproduce the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Side note: this compression feature has been tested in replacement of compression algorithms but they are not so good at performing compression as particular algorithms designed for it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When reducing the error between input and output, the network then starts forgetting the pieces that don't add as much information to the input as other bits of data in it. For instance, if you always train these images with figures in white backgrounds, it is likely that the encoded representation of the image will not have any information about the white background itself, since it is not required to be recreated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, the most common pieces will start falling apart, and with the right tuning, a general amount of noise will go away too. Uniform noise is easier to remove (for the very same reason as the white background example), but variable noise can be worked around too with the right training.&lt;/p&gt;&#xA;" OwnerUserId="190" LastEditorUserId="190" LastEditDate="2017-11-25T17:31:32.577" LastActivityDate="2017-11-25T17:31:32.577" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4599" PostTypeId="1" AcceptedAnswerId="4602" CreationDate="2017-11-25T22:39:06.307" Score="0" ViewCount="3173" Body="&lt;p&gt;I recently started to follow along with Siraj Raval's Deep Learning tutorials on YouTube, but I an error came up when I tried to run my code. The code is from the second episode of his series, How To Make A Neural Network. When I ran the code I got the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;File &quot;C:\Users\dpopp\Documents\Machine Learning\first_neural_net.py&quot;, line 66, in &amp;lt;module&amp;gt;&#xA;neural_network.train(training_set_inputs, training_set_outputs, 10000)&#xA;File &quot;C:\Users\dpopp\Documents\Machine Learning\first_neural_net.py&quot;, line 44, in train&#xA;self.synaptic_weights += adjustment&#xA;ValueError: non-broadcastable output operand with shape (3,1) doesn't match the broadcast shape (3,4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I checked multiple times with his code and couldn't find any differences, and even tried copying and pasting his code from the GitHub link. This is the code I have now:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import exp, array, random, dot&#xA;&#xA;class NeuralNetwork():&#xA;    def __init__(self):&#xA;        # Seed the random number generator, so it generates the same numbers&#xA;        # every time the program runs.&#xA;        random.seed(1)&#xA;&#xA;        # We model a single neuron, with 3 input connections and 1 output connection.&#xA;        # We assign random weights to a 3 x 1 matrix, with values in the range -1 to 1&#xA;        # and mean 0.&#xA;        self.synaptic_weights = 2 * random.random((3, 1)) - 1&#xA;&#xA;    # The Sigmoid function, which describes an S shaped curve.&#xA;    # We pass the weighted sum of the inputs through this function to&#xA;    # normalise them between 0 and 1.&#xA;    def __sigmoid(self, x):&#xA;        return 1 / (1 + exp(-x))&#xA;&#xA;    # The derivative of the Sigmoid function.&#xA;    # This is the gradient of the Sigmoid curve.&#xA;    # It indicates how confident we are about the existing weight.&#xA;    def __sigmoid_derivative(self, x):&#xA;        return x * (1 - x)&#xA;&#xA;    # We train the neural network through a process of trial and error.&#xA;    # Adjusting the synaptic weights each time.&#xA;    def train(self, training_set_inputs, training_set_outputs, number_of_training_iterations):&#xA;        for iteration in range(number_of_training_iterations):&#xA;            # Pass the training set through our neural network (a single neuron).&#xA;            output = self.think(training_set_inputs)&#xA;&#xA;            # Calculate the error (The difference between the desired output&#xA;            # and the predicted output).&#xA;            error = training_set_outputs - output&#xA;&#xA;            # Multiply the error by the input and again by the gradient of the Sigmoid curve.&#xA;            # This means less confident weights are adjusted more.&#xA;            # This means inputs, which are zero, do not cause changes to the weights.&#xA;            adjustment = dot(training_set_inputs.T, error * self.__sigmoid_derivative(output))&#xA;&#xA;            # Adjust the weights.&#xA;            self.synaptic_weights += adjustment&#xA;&#xA;    # The neural network thinks.&#xA;    def think(self, inputs):&#xA;        # Pass inputs through our neural network (our single neuron).&#xA;        return self.__sigmoid(dot(inputs, self.synaptic_weights))&#xA;&#xA;if __name__ == '__main__':&#xA;&#xA;    # Initialize a single neuron neural network&#xA;    neural_network = NeuralNetwork()&#xA;&#xA;    print(&quot;Random starting synaptic weights:&quot;)&#xA;    print(neural_network.synaptic_weights)&#xA;&#xA;    # The training set. We have 4 examples, each consisting of 3 input values&#xA;    # and 1 output value.&#xA;    training_set_inputs = array([[0, 0, 1], [1, 1, 1], [1, 0, 1], [0, 1, 1]])&#xA;    training_set_outputs = array([[0, 1, 1, 0]])&#xA;&#xA;    # Train the neural network using a training set&#xA;    # Do it 10,000 times and make small adjustments each time&#xA;    neural_network.train(training_set_inputs, training_set_outputs, 10000)&#xA;&#xA;    print(&quot;New Synaptic weights after training:&quot;)&#xA;    print(neural_network.synaptic_weights)&#xA;&#xA;    # Test the neura net with a new situation&#xA;    print(&quot;Considering new situation [1, 0, 0] -&amp;gt; ?:&quot;)&#xA;    print(neural_network.think(array([[1, 0, 0]])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Even after copying and pasting the same code that worked in Siraj's episode, I'm still getting the same error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just started out look into artificial intelligence, and don't understand what the error means. Could someone please explain what it means and how to fix it? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11083" LastActivityDate="2017-11-26T21:41:35.160" Title="Error building Neural Net: ValueError: non-broadcastable output operand with shape (3,1) doesn't match the broadcast shape (3,4)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4600" PostTypeId="1" CreationDate="2017-11-25T23:13:13.897" Score="1" ViewCount="109" Body="&lt;p&gt;I have a neural network that I'm want to use to self-play Connect Four. The neural network receives the board state and is to provide an estimate of the state's value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would then, for each move, use the highest estimate, occasionally I will use one of the other moves for exploration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I intend to use TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) to calculate the errors for each state to backpropagate through the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm confused about when this should actually occur. Do I store the estimate of a state that is used and calculate the error based on the next state chosen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or do I store a history of all states and backpropagate only when the game is a win/lose/draw?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess overall I'm not sure I understand when the update occurs, partially because I don't quite understand how to implement the lambda? Like if I was to apply to back prop after every move, how would I even know the value of lambda at this time-step before I know how long the game will last?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When self-playing, is the error the difference between that &quot;sides&quot; last move? I.e. I compare move 1 against move 3, and move 2 against move 4, etc?&lt;/p&gt;&#xA;" OwnerUserId="2819" LastEditorUserId="2444" LastEditDate="2020-06-04T17:04:12.867" LastActivityDate="2020-06-04T17:04:12.867" Title="When do you back-propagate errors through a neural network when using TD($\lambda$)?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;backpropagation&gt;&lt;td-lambda&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4601" PostTypeId="1" CreationDate="2017-11-26T10:41:21.500" Score="2" ViewCount="2436" Body="&lt;p&gt;I am a c# senior developer and I got a task to try and predict the potential in each new client, or maybe the worth of each customer.&#xA;I don't have experience with machine learning, but I played with accord-framework.net and got some nice results on simple task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data model for &lt;strong&gt;training&lt;/strong&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GeoLocation, // the country of ip when registed. iso code string&#xA;Age, // number&#xA;DateRegistered, //date time&#xA;Email, //string can be broken to vendors as catergorial (gmail, yahoo, microsoft and such)&#xA;EmailValidated, //is the email really exists. bool&#xA;PhoneNumber, //string&#xA;PhoneNumberValidated, // is the phone number really exists&#xA;CampaignName, //string (may be categirial)&#xA;UserAgent, //string should I make it categorial? (has info about browser, device, verndor, operation system and such, long string)&#xA;LandedOnPage, //string first url the customer entered from &#xA;RegisteredFromPage, //string url of the page that the user registered from&#xA;RefererUrl, //string url the client came to our site from,&#xA;NumberOfPurchases, //the amount of times the customer puschase something on our site&#xA;CustomerValueUsd, //the total amount of USD the customer spent in our site&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;output&lt;/strong&gt; shoud be &lt;code&gt;CustomerValueUsd&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a lot of data in the history, so I can back test it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does it make sense to do this task even though I don't have an experience with machine learning? How complicated is this task considering I'm using a well known framework?&lt;/li&gt;&#xA;&lt;li&gt;Assuming that I'm taking the task, which algorithm should I choose to perform this kind of task?&lt;/li&gt;&#xA;&lt;li&gt;How should I build the training data? see my comments, do you think my comments are ok to start with? or maybe I can break the data directly? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11133" LastEditorUserId="15219" LastEditDate="2018-04-26T17:11:20.607" LastActivityDate="2018-06-26T03:17:10.907" Title="Which algorithm should I choose for lead scoring" Tags="&lt;machine-learning&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4602" PostTypeId="2" ParentId="4599" CreationDate="2017-11-26T21:41:35.160" Score="0" Body="&lt;p&gt;You forgot to transpose the output values in line 61. The line should read as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;training_set_outputs = array([[0, 1, 1, 0]]).T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Otherwise the format of the matrices that you want to add together in line 43 does not match. &lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-11-26T21:41:35.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4603" PostTypeId="2" ParentId="4601" CreationDate="2017-11-27T07:18:29.127" Score="1" Body="&lt;p&gt;ML is a wide and deep topic to cover. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need is &quot;classification&quot; to predict an outcome. At first, you need to convert everything to numbers (check word2vec). Then, pre-process the data (Check what is a normalization for example). Train a classifier by using any ML techniques (i.e. linear classifier, SVM, deep learning, neural classifier). Then you can use that regression function to predict further values. If you need much more detailed way to do that, it is better to buy a 'thick' book that explains all other options. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-27T07:18:29.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4604" PostTypeId="2" ParentId="4576" CreationDate="2017-11-27T07:25:17.850" Score="1" Body="&lt;p&gt;It is an interesting application. It is possible. You can interpret sound as histogram (2D image) and apply same image processing techniques (CNN) to extract information. Alternatively, you can keep them as phase / intensity values and train a network on top of them (RNN). That is a great idea. Go for it!&lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-11-27T07:25:17.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4605" PostTypeId="2" ParentId="4576" CreationDate="2017-11-27T10:12:43.510" Score="0" Body="&lt;p&gt;I think it should be possible. The main difficulty will be in getting enough labelled data, deep learning approaches are very data hungry. Audio tasks such as speech typically require hours of labelled data. It's hard to tell from the problem description whether you would need that much - the classification is simpler (just two classes, not identifying phonemes), but I think the perceptual difference between the classes is harder, as you say yourself you are not quite sure how you are sensing the difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might have success with a relatively low amount of data if you can find a pre-trained deep network for some other audio task (with a similar domain e.g. activity classification), and fine tuning the last few layers using your own data. One such &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research/audioset&quot; rel=&quot;nofollow noreferrer&quot;&gt;pre-trained model is called VGGish&lt;/a&gt; and has been trained on 100s of hours of short video soudtracks.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2017-12-02T21:30:58.313" LastActivityDate="2017-12-02T21:30:58.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4606" PostTypeId="1" CreationDate="2017-11-27T17:17:55.997" Score="1" ViewCount="56" Body="&lt;p&gt;I have a problem that I have been trying to use decision trees to solve. There is a data set of pricing information for products sold by a company. The goal is to infer the pricing algorithm for each product based on the product's name, type, and characteristics. However, the data is incomplete and may contain a few inaccuracies due to manual data entry.  Products can be priced using several possible algorithms:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Price is a fixed dollar amount (widget costs $20)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Price is a percentage of MSRP (widget costs X% of MSRP, note X could vary by product classes)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Price is based on a cost per piece (package of widgets costs $2 per piece)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Using decision trees, I have been able to identify the products priced using algorithm #1, but have struggled to make progress on algorithm #2 and #3. Since price is a continuous variable, the leaf nodes always represent the average of the target variable. But this doesn't seem to work for #2 and #3, since these would be a function of other variables, not the dependent variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a clever way to implement this using decisions trees? Any recommended alternate techniques that could identify the pricing algorithms used? Note, I am not interested in predictions, but rather explain the pricing rules/algorithm that fit the existing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using SAS Enterprise Miner, but may have access to other tools as well.&lt;/p&gt;&#xA;" OwnerUserId="11153" LastActivityDate="2017-11-27T17:17:55.997" Title="Decision tree doesn't quite work, is there a better alternative?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4607" PostTypeId="1" AcceptedAnswerId="4608" CreationDate="2017-11-28T05:50:02.307" Score="3" ViewCount="1111" Body="&lt;p&gt;From my understanding and text I found in research papers online :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Pixel-based object recognition&lt;/strong&gt;: neural networks are trained to locate individual objects based directly on pixel data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Feature-based object recognition&lt;/strong&gt;: contents of a window are mapped to a feature space that is provided as input to a neural classifier.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What do the above 2 definitions imply in a simpler language? Especially point number 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, can someone point me to papers/resources/articles that would explain the above approaches in greater detail?&lt;/p&gt;&#xA;" OwnerUserId="11169" LastEditorUserId="2444" LastEditDate="2020-06-15T20:56:06.390" LastActivityDate="2020-06-15T20:56:06.390" Title="What is the difference between pixel-based object recognition and feature-based object recognition?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4608" PostTypeId="2" ParentId="4607" CreationDate="2017-11-28T09:45:11.927" Score="2" Body="&lt;p&gt;&lt;strong&gt;Pixel based object recognition&lt;/strong&gt;, like the name says, works by analyzing the individual pixels of an image. For example: You analyze an image with a lot of different shades of blue and some grey pixels - you might assume that this is the picture of a plane in the sky or a ship in the water.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also look for similar pictures by calculating the difference between each individual pixel of 2 images and sum up the differences. The smaller the sum of the difference, the more the pictures look alike.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It depends on your field of work, but those methods will most likely not be good enough for solid image recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also mentioned neural networks. Of course you can feed the raw pixel data to a neural network, but that's not state of the art. You will almost always use a CNN, which brings me to your next point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Feature based object recognition&lt;/strong&gt;, like the name says, tries to extract certain features from an image to perform classification. You use several convolutional layers for analysis before you feed the output to a neural network. This architecture is called CNN (Convolutional Neural Network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What those features actually look like is hard to explain without showing examples. I recommend to watch the following two lectures from the Stanford course CS231n:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=OoUX-nOEjG0&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lecture 2 | Image Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=bNb2fEVKeEo&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lecture 5 | Convolutional Neural Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first one explains image classification in general and shows some implementations for pixel based algorithms. The second one dives into CNNs and explains feature based approaches. I highly recommend to watch the full series, but if you are in a hurry, those 2 lectures will tell you what you want to know.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2017-11-28T09:45:11.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4610" PostTypeId="1" CreationDate="2017-11-29T01:43:54.713" Score="4" ViewCount="372" Body="&lt;p&gt;Goal-driven AIs is the only kind of AI I am aware of. However, Marcus Hutter claims &lt;a href=&quot;http://www.hutter1.net/ai/uaibook.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;the following&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most, if not all known facets of intelligence can be formulated as goal driven or, more generally, as maximizing some utility function. It is, therefore, sufficient to study goal driven AI.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which doesn't necessarily imply that there are other types of AIs (apart from goal-driven), but (at least, in the way it is phrased) suggests that there are other types of AIs. If there exist other types of AIs, which are they?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2017-11-29T19:04:37.350" LastActivityDate="2018-11-24T19:26:50.280" Title="What other kind of AIs exist apart from goal-driven?" Tags="&lt;agi&gt;&lt;definitions&gt;&lt;terminology&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4612" PostTypeId="1" CreationDate="2017-11-29T08:18:54.830" Score="2" ViewCount="136" Body="&lt;p&gt;While unifying two literals, can we unify a positive and negative literal. For e.g, &lt;em&gt;p(x) with ~P(A)&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this unifiable using the substitution {x/A}. Russel &amp;amp; Norvig textbook doesn't say anything about unifying positive and negative literals while &lt;a href=&quot;https://courses.cs.washington.edu/courses/cse341/03au/slides/Prolog3/sld005.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; from University of Toronto  says it is possible.&lt;/p&gt;&#xA;" OwnerUserId="4202" LastActivityDate="2017-11-29T08:18:54.830" Title="First Order Logic: Unification of positive and negative literals" Tags="&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4613" PostTypeId="1" CreationDate="2017-11-29T09:49:06.553" Score="4" ViewCount="323" Body="&lt;p&gt;I have created a Gomoku(5 in a row) AI using Alpha-Beta Pruning. It makes moves on a not-so-stupid level. First, let me vaguely describe the grading function of the Alpha-Beta algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it receives a board as an input, it first finds all repetitions of stones and gives it a score out of 4 possible values depending on its usefulness as an threat, which is decided by length. And it will return the summation of all the repetition scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, the problem is that I explicitly decided the scores(4 in total), and they don't seem like the best choices. So I've decided to implement a genetic algorithm to generate these scores. Each of the genes will be one of 4 scores. So for example, the chromosome of the hard-coded scores would be: [5, 40000,10000000,50000]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, because I'm using the genetic algorithm to create the scores of the grading function, I'm not sure how I should implement the genetic fitness function. So instead, I have thought of the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of using a fitness function, I'll just merge the selection process together: If I have 2 chromosomes, A and B, and need to select one, I'll simulate a game using both A and B chromosomes in each AI, and select the chromosome which wins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.Is this a viable replacement to the Fitness function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.Because of the characteristics of the Alpha-Beta algorithm, I need to give the max score to the win condition, which in most cases is set to infinity. However, because I can't use Infinity, I just used an absurdly large number. Do I also need to add this score to the chromosome? Or because it's insignificant and doesn't change the values of the grading function, leave it as a constant? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.When initially creating chromosomes, random generation, following standard distribution is said to be the most optimal. However, genes in my case have large deviation. Would it still be okay to generate chromosomes randomly?&lt;/p&gt;&#xA;" OwnerUserId="11207" LastActivityDate="2018-05-13T10:47:14.587" Title="Fitness Function altenatives in Genetic Algorithms for game AI" Tags="&lt;genetic-algorithms&gt;&lt;game-ai&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4614" PostTypeId="1" CreationDate="2017-11-29T10:41:16.767" Score="2" ViewCount="189" Body="&lt;p&gt;Lets say I have a limited amount of training data (1000 documents each having 10000 values) and after learning with those, the program is &#xA; basically not allowed to fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I've read, even very easy tasks take a machine learning algorithm several thousand learning cycles until it is about 98% reliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which parameters need to be tweaked so that even with many many values, the program can make reliable decisions?&lt;/p&gt;&#xA;" OwnerUserId="10872" LastActivityDate="2018-03-29T21:45:51.307" Title="What parameters to tweak to improve reliability (machine learning)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4616" PostTypeId="1" CreationDate="2017-11-29T13:38:30.750" Score="5" ViewCount="496" Body="&lt;p&gt;We all know how robots are getting more and sophisticated and more interesting &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the future of &lt;strong&gt;robotics&lt;/strong&gt; in relation to AI, how and how will AI work with robotics in improving to affect our world in a positive way.&lt;/p&gt;&#xA;" OwnerUserId="11210" LastEditorUserId="1671" LastEditDate="2017-11-29T18:06:12.230" LastActivityDate="2018-06-01T01:34:29.997" Title="How have robots developed and how sophisticated might they be in the future?" Tags="&lt;robotics&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4617" PostTypeId="2" ParentId="4614" CreationDate="2017-11-29T18:54:09.427" Score="1" Body="&lt;p&gt;If your question is about supervised learning,  than ten million training points are enough in most of the cases. Besides, I think your question is too broad. There are a lot of  parameters which can influence the accuracy. Like which algorithm you want to use. Which approximator and its topology, the hyperparameters etc. &lt;/p&gt;&#xA;" OwnerUserId="6019" LastActivityDate="2017-11-29T18:54:09.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4618" PostTypeId="2" ParentId="4616" CreationDate="2017-11-29T18:55:03.633" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The future of robotics&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Companies like Google are doing restless research in robotics whereby machine learning algorithms are applied to empower self driving cars well equipped with computer vision, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The future is so exciting in that it will be an era where artificially intelligence machines can rewrite their own source code and constantly improve upon themselves at an exponential rate -- ASI: &lt;em&gt;artificial super intelligence&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, robots like Sophia are using machine learning algorithms to answer or respond to human interactions.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="75" LastEditDate="2018-06-01T01:34:29.997" LastActivityDate="2018-06-01T01:34:29.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4619" PostTypeId="2" ParentId="4532" CreationDate="2017-11-29T20:03:39.307" Score="2" Body="&lt;p&gt;This is necessarily a high-level answer, and highly speculative, but I've been thinking on this question, and here are my thoughts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Implementing ethical algorithms requires a mathematical basis for philosophy because computers are difference engines&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After Russell &amp;amp; Whitehead's famous failure, and Gödel's incompleteness theorem, this would seem to be problematic. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AI is a highly applied field, especially today per continuing validation of deep learning, and no company wants to go near the issue of ethics unless they are forced to&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus, you see it in self-driving cars because the engineers have no choice but to grapple with the problem.  By contrast, I don't think you'll see many algorithmic stock trading firms, where the business is &lt;a href=&quot;https://en.wikipedia.org/wiki/Pareto_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pareto efficiency&lt;/a&gt;, worrying about the ethics or social impacts of financial speculation. (The solution to &quot;flash crashes&quot; seems to have been rules for temporary suspension of trading, instead of addressing the social value of high-frequency algorithmic trading.) A more obvious example is social media companies ignoring the extreme amounts of information abuse (disinformation and misinformation) being posted on their sites, pleading ignorance, which is highly suspect in that the activity generated by information abuse positively affects their bottom-lines.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Applied fields tend to be predominantly driven by profit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The primary directive of corporations is to return a profit to investors.  It's not uncommon for corporations to break the law when the fines and penalties are expected to be less than the profit made by illegal activity.  (There is the concept of ethics in business, but the culture in general seems to judge people and companies based on how much money they make, regardless of the means.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Implementation of machine ethics is being explored in areas where they are necessary to sell the product, but elsewhere, it's still largely hypothetical&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If superintelligences evolve and wipe out humanity (as some very smart people with superior mathematics skills are warning us about,) my feeling is that it will be a function of nature, where unrestricted evolution of these algorithms is due to economic drivers which focus on hyper-partisan automata in industries like financial speculation and autonomous warfare.  Essentially, chasing profits at all costs, regardless of the impacts. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-11-29T22:40:50.793" LastActivityDate="2017-11-29T22:40:50.793" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4620" PostTypeId="1" AcceptedAnswerId="4627" CreationDate="2017-11-29T20:16:43.223" Score="0" ViewCount="169" Body="&lt;p&gt;Would it not even be more helpful? It will have more time to transform the input with the extra zeros padded to the end.&lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2017-11-30T19:07:50.017" Title="What's wrong with getting a dynamic RNN's output at the end of a padded sequence?" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4621" PostTypeId="1" CreationDate="2017-11-29T20:57:01.877" Score="0" ViewCount="48" Body="&lt;p&gt;I read these things on the internet like&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My model determines the future scope...&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My model gives accurate readings about what the score would be...&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What are these models? How are they designed?&lt;/p&gt;&#xA;" OwnerUserId="11220" LastEditorUserId="1671" LastEditDate="2017-12-01T15:56:03.807" LastActivityDate="2017-12-01T15:56:03.807" Title="What is a model and how is it designed?" Tags="&lt;terminology&gt;&lt;models&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4622" PostTypeId="2" ParentId="4621" CreationDate="2017-11-30T00:06:55.290" Score="2" Body="&lt;p&gt;A model is exactly what the name suggests.  A simplified representation of a solution to a real life problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you think of a simple formula for a falling object you may not take into account real life variables such as the imperfections on the surface of the object, atmospheric conditions, the exact composition of the air the object is falling through and how it may affect friction.  you just use a simple model to &quot;model&quot; the fall and get a reasonably accurate result.  its a model, it doesnt have all the details.  just like a model car wouldnt have all the details of a real car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a good read to get you started:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Mathematical_model&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2017-11-30T00:06:55.290" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4623" PostTypeId="1" CreationDate="2017-11-30T02:30:44.337" Score="0" ViewCount="108" Body="&lt;p&gt;Technically speaking, could we code in natural language once we pass the Turing test? Would passing the Turing test at least simplify programming languages' syntax?&lt;/p&gt;&#xA;" OwnerUserId="11224" LastEditorUserId="4302" LastEditDate="2018-10-08T12:19:52.467" LastActivityDate="2018-10-08T12:19:52.467" Title="Could we code in Natural language once we pass the Turing test?" Tags="&lt;natural-language-processing&gt;&lt;human-like&gt;&lt;programming-languages&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4624" PostTypeId="1" CreationDate="2017-11-30T11:44:52.770" Score="3" ViewCount="1189" Body="&lt;p&gt;From this &lt;a href=&quot;http://computer-go.org/pipermail/computer-go/2017-October/010307.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;, AlphaGo would take millenia to run in regular hardware.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;They generated 29 million&#xA;  games for the final result, which means it's going to take me about 1700&#xA;  years to replicate this. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Are these calculations correct?&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2018-08-13T23:00:52.193" Title="Would it take 1700 years to run AlphaGo Zero in commodity hardware?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4625" PostTypeId="2" ParentId="4610" CreationDate="2017-11-30T15:58:31.770" Score="4" Body="&lt;p&gt;Whether all AI are goal-driven depends on how far you're willing to extend the definition of 'goal'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AIs are often defined through their primary tasks. Face recognition is a process; a face recognition AI is a limited, special-purpose AI. Paperclip collection is a process, the Paperclip Maximizer is a general, goal-driven AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will always be &lt;em&gt;some sort of goal&lt;/em&gt;, but it may in certain cases become so nebulous it's really a stretch to continue calling it a goal. For example,  unsupervised learning (as suggested by Aiden Grossman) - the goal you set is &quot;learn&quot;, with very little qualifiers. The underlying AI may develop in surprising and unexpected ways; it may even modify own goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, another &quot;aimless&quot; AI type is a simulator of biological intelligence - say, replicating neural connections of an insect in software, making it run and observing the behaviors. The AI itself is not given a specific goal - it's a tool/subject of study, not something directly producing results. Still, the goals are exactly the same as goals of the original insect: feed, multiply, stay safe; unattainable due to hardware limitations but still pursued.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, whether all AIs are goal-driven is arguable. The only AI that is not actually driven by any goal whatsoever is one that's switched off. But whether you consider all the possible nebulous, abstract or accidental goals that drive the AI to call it actually goal-driven, that's up to debate.&lt;/p&gt;&#xA;" OwnerUserId="38" LastActivityDate="2017-11-30T15:58:31.770" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4626" PostTypeId="2" ParentId="4623" CreationDate="2017-11-30T16:16:57.363" Score="1" Body="&lt;p&gt;No.  Humans pass the Turing Test but cannot express themselves with enough precision and little enough ambiguity to code well in English (natural language).  No machine will improve on that until it can greatly improve on human-level clarity in expressing their purpose when programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to program in natural language?  Yes, I think so.  But it would require superhuman intelligence to anticipate all the possible confusions that might arise when a word is not sufficiently precise or accurate to represent an software activity or mechanism.  This probably would require adopting a rigorous convention in choosing a subset of unambiguous words to form a &quot;natural&quot; programming language (a kind of creole).  Coding consistently using only those words would require discipline that exceeds the ability of most humans, methinks.&lt;/p&gt;&#xA;" OwnerUserId="1657" LastEditorUserId="1657" LastEditDate="2017-11-30T16:24:24.207" LastActivityDate="2017-11-30T16:24:24.207" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4627" PostTypeId="2" ParentId="4620" CreationDate="2017-11-30T19:07:50.017" Score="2" Body="&lt;p&gt;There is nothing wrong in theory. It is reasonable to expect that for some problems it will work worse than taking the immediate answer on the last known part of the sequence, some it will have roughly the same effect, and for others it might result in a better result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A trained RNN is essentially a learned arbitrary program. If a computer program that processed arrays would get a better result by running more iterations at the end of an array, then so might a RNN. One example that comes to mind is predicting the end point of a trajectory of an observed object, when you have a variable number of observations at the start. A program that simulated each timestep in a simple fashion could well perform better than one that attempted to analyse and project directly from whatever the last observed state was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, you may face some practical problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;One practical detail of not padding sequences is that it may reduce computation time since the network is run less often.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A padded sequence will only get a gradient to learn from with the last output, after the padding. This could make learning harder than it needs to be with vanishing or exploding gradients (these are more of a problem, the more time steps you need take back through the network in order to assign correct weights due to influence of first input).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The usual way to find out is to try it in an experiment. See which approach does better on a hold-out test set when training with or without padding. Bear in mind that whatever the answer it will not hold in general, but is likely to hold for problems similar to the one you tested with.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2017-11-30T19:07:50.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4628" PostTypeId="2" ParentId="4623" CreationDate="2017-11-30T19:35:13.383" Score="0" Body="&lt;p&gt;I think this is a very interesting question.  It's highly hypothetical, so my simple answer will also, necessarily, be hypothetical.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the AI can pass the Turing Test, it can be assumed it has a command of the human language(s) used in that process&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Self-awareness in this context would seem to be irrelevant, as would the question of whether the AI actually &quot;understands&quot; the content, or is merely imitating natural language.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the AI has a command of the human language(s), it would be a reasonable assumption that it could translate natural language instructions into machine code&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It might be best to think of the problem in the simplest terms.  If you asked this hypothetical AI to &quot;draw a red square on the screen&quot;, it's hard to see that task as impossible, or even difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, as the instructions became more complicated, there would be a greater margin for error, and it would be useful to integrate precise, formal terms related to functions, as &lt;a href=&quot;https://ai.stackexchange.com/a/4626/1671&quot;&gt;Randy points out&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-11-30T19:35:13.383" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4629" PostTypeId="1" CreationDate="2017-11-30T20:57:09.410" Score="5" ViewCount="1982" Body="&lt;p&gt;A blog post called &quot;Text Classification using Neural Networks&quot; states that the derivative of the output of a sigmoid function is used to measure error rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the rationale for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought the derivative of a sigmoid function output is just the slope of the sigmoid line at a specific point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meaning it's steepest when sigmoid output is 0.5 (occuring when the sigmoid function input is 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does a sigmoid function input of 0 imply error (if i understand correctly)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source:&lt;/strong&gt; &lt;a href=&quot;https://machinelearnings.co/text-classification-using-neural-networks-f5cd7b8765c6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearnings.co/text-classification-using-neural-networks-f5cd7b8765c6&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We use a sigmoid function to normalize values and its derivative to&#xA;  measure the error rate. Iterating and adjusting until our error rate&#xA;  is acceptably low.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def sigmoid(x):&#xA;    output = 1/(1+np.exp(-x))&#xA;    return output&#xA;&#xA;def sigmoid_output_to_derivative(output):&#xA;    return output*(1-output)&#xA;&#xA;def train(...)&#xA;    ...&#xA;    layer_2_error = y - layer_2&#xA;    layer_2_delta = layer_2_error * sigmoid_output_to_derivative(layer_2)&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies. I don't think I was clear (I've updated the title)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand we don't need to use sigmoid as the activation funtion (we could use relu, tanh or softmax).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about using the &lt;code&gt;derivative to measure the error rate&lt;/code&gt; (full quotation from article above in yellow) -&gt; what does the derivative of the activation function have to do with measuring/fixing the &quot;error rate&quot;?&lt;/p&gt;&#xA;" OwnerUserId="10623" LastEditorUserId="10623" LastEditDate="2017-12-01T16:12:53.823" LastActivityDate="2018-06-22T20:49:44.953" Title="How does an activation function's derivative measure error rate in a neural network?" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4630" PostTypeId="2" ParentId="4574" CreationDate="2017-11-30T21:34:36.597" Score="1" Body="&lt;p&gt;I think SmallChess makes a good point on your first question, so I want to focus on the repetition of different problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could see you meaning several different things by this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why does every machine learning course out there do MNIST/real estate values/other simple problems?&lt;/strong&gt; If you want to learn how to solve a complex problem, you need to understand how the individual parts come together. You could jump straight to a classifier trained on 10,000 image categories using the current, most advanced techniques. However, it's much easier to see how certain algorithms work better with certain types of problems when starting with small, easy ones that can be solved in a few minutes. You can try many different sets of algorithms and hyperparameters on a problem that takes 3 minutes to train on your local computer. Then, you get a sense of what each part contributes to the whole. This will help you progress a lot faster than training a network that takes 2 weeks on multiple GPUs each time and trying to figure out what's going on. In addition, you don't have a ton of layers of complexity to try and understand. You get a feel for a simple approach and once you understand it, add complexity. It's the same with learning anything else. My physics class started with gravity and throwing a ball in the air, not with relativity.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why are there so many people out there doing all these different approaches to image classification/chatbots/reinforcement learning/whatever?&lt;/strong&gt; Machine learning is a not a solved problem. There are a set of algorithms we understand pretty well. Fitting a polynomial to a set of points doesn't really have any hidden tricks up its sleeves. There are plenty of other algorithms that we're still figuring out. Sometimes, a neural network never converges, or only converges to some limit. When you change the hyperparameters, it converges or has better accuracy or something. But sometimes, it generalizes poorly. Looking at 10 million parameters and deciding which one screwed up is hard, so people research how to solve these problems. Or, often, current &quot;state-of-the-art&quot; approaches really aren't at the point where improvement is rare. Every single year, they hold another competition and lots of people come up with better ways to solve it, either with new algorithms or better refinements to the ones they were already using. There are still so many things not yet understood about this field and it seems there are major discoveries constantly.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why can't I just use someone else's pre-trained methods?&lt;/strong&gt; You can! If you can find someone who has put their model out there (for instance, &lt;a href=&quot;https://github.com/tensorflow/models&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;), you can take it and run it for yourself. Some researchers put out their already trained models and some don't, so you might get lucky. The problem is, you have to decide if the model they trained fits your needs. If you really need a visual classifier that can detect penguins, and you download one that was trained on 1,000 classes, none of which were penguins, you're out of luck. So, definitely, look for someone else who might have already done it. Just make sure that it covers what you actually need.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It doesn't really fit into the above points, but I hope I can convey that machine learning is just another tool. Taking your comment about wanting to build a robot companion for a toddler, you need to decide whether it fits the problem you have. If it has some wheels for moving the base around, you could spend a ton of time building a controller with reinforcement learning. Or, you could program and implement a PID controller in a couple days. Does it need to detect the surroundings and label objects in camera images to interact with them? Then you probably need a CNN of some kind to do it accurately. Does it need to listen to the toddler's voice commands to go bring a ball back? Then, you need some way to interpret those, and it might be another machine learning algorithm. But let's say that you find several pre-trained algorithms that do the tasks you need. They still need to be linked together in whatever overall software you have. Analogously, if you were using someone else's path planning algorithm, you still need to define the goals, take the outputs and input them into the controller, and update your state estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of people hoping to figure out machine learning algorithms that can just do everything like humans do. End-to-end learning tries to go directly from input data to actions (&lt;a href=&quot;https://arxiv.org/pdf/1604.07316.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-driving cars&lt;/a&gt;). Others are trying to create an artificial general intelligence that thinks and interacts with the world like we do. The point is, there are so many things that haven't been solved yet. Your robot won't be as good as a 3 year-old at many things, but that's not the point. ML gives the robot the ability to be really good at some certain thing, like figuring out where the toy it's looking at is. But it isn't a &quot;wave your hands and it all works&quot; solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; Use ML when it fits the problem. But it's a hard problem, and there are a lot of things to understand about it if you want to use it. It's a tool in your toolbox.&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2017-11-30T21:34:36.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4631" PostTypeId="2" ParentId="4629" CreationDate="2017-12-01T00:39:55.820" Score="1" Body="&lt;p&gt;Measuring the error rate of a neural network does not involve the derivative of the sigmoid function at all. It only needs the neural networks outputs, and the expected outputs. It does not matter how the neural network got to those outputs, the outputs only have to be based on the inputs. What the other of that specific text is trying to say is that the derivative of the sigmoid function is important when you use the algorithm back propagation to train the neural network. This method involves using derivatives to optimize the neural network. While this technique is more complicated with neural networks because of how many variables are involved, you can easily see the basics of this approach if you look at a calculus textbook and look at the chapter that is usually titled Applications of the derivative.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="4631" LastEditDate="2018-06-22T20:49:44.953" LastActivityDate="2018-06-22T20:49:44.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4632" PostTypeId="2" ParentId="4629" CreationDate="2017-12-01T02:46:02.760" Score="2" Body="&lt;p&gt;You dont need a sigmoid function if you dont want one.  Any differentiable function will do.  Sigmoid functions are just one of many suitable functions. You could write your own differentiable function if you want a propriety solution. &lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2017-12-01T02:46:02.760" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4634" PostTypeId="1" CreationDate="2017-12-01T03:55:13.087" Score="3" ViewCount="40" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1506.07285&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ask Me Anything: Dynamic Memory Networks for Natural Language Processing&lt;/a&gt; the authors described a Dynamic Memory Network in the context of question answering. Then, they also tested the network on sentiment classification and part-of-speech tagging. In those applications, what is the 'question' being inputted to the network?&lt;/p&gt;&#xA;" OwnerUserId="7419" LastEditorUserId="2444" LastEditDate="2021-01-26T17:48:11.277" LastActivityDate="2021-01-26T17:48:11.277" Title="What is the &quot;question&quot; when using Dynamic Memory Networks to do part-of-speech tagging and sentiment analysis?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;papers&gt;&lt;question-answering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4635" PostTypeId="1" AcceptedAnswerId="4640" CreationDate="2017-12-01T08:44:23.637" Score="1" ViewCount="48" Body="&lt;p&gt;I have a MySQL database that contains datetime for every check-in with an RFID card. I have millions of records in it. It shouts machine learning prediction for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'd like to predict popular times to see when most people use the terminal. I'd like to represent it the same way as Google does at places:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/J5ARp.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J5ARp.jpg&quot; alt=&quot;Google&amp;#39;s popular times panel at location search&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the rare case when I do not ask for code, but keywords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume I have to predict number of check-ins between time ranges. Also I am sure I have to make a dataset like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                        (These are the number of check-ins in hour ranges)&#xA;month | day | day-name | 0-1 | 1-2 | ... | 6-7 etc. &#xA;jan   | 1   | Monday   |  0  |  1  |     |  12&#xA;jan   | 2   | Tuesday  |  1  |  3  |     |  15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it can predict today's popular times based on what day it is, also by day of given month (as Saturdays and Sundays will be dead, also 25th of December. A good prediction should know these from the dataset).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I wrote this question I solved most of it it seems. &lt;strong&gt;The only thing I need is a keyword&lt;/strong&gt; as I have little experience in this. &lt;strong&gt;What model fits this best?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="11263" LastActivityDate="2017-12-01T16:39:25.707" Title="Predicting popular times" Tags="&lt;machine-learning&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4637" PostTypeId="5" CreationDate="2017-12-01T15:53:52.057" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-10-15T16:55:05.523" LastActivityDate="2018-10-15T16:55:05.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4638" PostTypeId="4" CreationDate="2017-12-01T15:53:52.057" Score="0" Body="For questions related to modelling external environment, functional models tuned through convergent methods such as artificial networks or fuzzy logic containers, loss models, semantic models, model-based reasoning, or other kinds of models used in AI research, development, or practice." OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-10-15T16:57:43.820" LastActivityDate="2018-10-15T16:57:43.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4639" PostTypeId="2" ParentId="4629" CreationDate="2017-12-01T16:27:52.320" Score="4" Body="&lt;p&gt;This derivative is used when calculating the error of your machine learning algorithm during gradient based minimization methods. &#xA;Read below for more info. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When performing supervised classification (with X, Y data vectors of inputs and outcome data to train with) you begin with the error function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E(X, Y; &amp;theta;)= &amp;sum;&lt;sub&gt;i&lt;/sub&gt; (&amp;fnof;(x&lt;sub&gt;i&lt;/sub&gt;; &amp;theta;)-y&lt;sub&gt;i&lt;/sub&gt;)&lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for total error over all data instances i, where f is your neural network, linear regression,...method of interest and &amp;theta; is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). To be explicit, &amp;fnof;(x&lt;sub&gt;i&lt;/sub&gt;; &amp;theta;); outputs value of interest which should be y&lt;sub&gt;i&lt;/sub&gt;. And E measures how far off it is in prediction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to train your classifier, you optimize E with something like gradient descent. Thus when &amp;part;E/&amp;part;&amp;theta; = 0 (for a particular &amp;theta;), that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the &amp;fnof; here is not the same as an activation function, as a neural network is defined differently than in linear regression, etc. and must perform a special kind of gradient descent called backpropagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when you take &amp;part;E/&amp;part;&amp;theta;, what does it equal for a neural net? You should note the activation functions derivative is involved which is how it’s used to measure error so to say. &lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="9469" LastEditDate="2017-12-01T19:09:47.183" LastActivityDate="2017-12-01T19:09:47.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4640" PostTypeId="2" ParentId="4635" CreationDate="2017-12-01T16:39:25.707" Score="2" Body="&lt;p&gt;&quot;Regression&quot;, is to estimate a continuous value as a function of some other parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different forms of regression, such as linear regression and logistic regression, that differ in the assumptions they make on the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try looking up those for a start.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-01T16:39:25.707" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4642" PostTypeId="2" ParentId="4629" CreationDate="2017-12-01T21:32:55.040" Score="0" Body="&lt;p&gt;The derivative of your loss is the &quot;slope&quot; at that given prediction. So by &quot;moving down along the slope&quot;, we can reduce the value of the loss function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, one can imagine a ball rolling down a slope in the direction of the tangent. Eventually the ball will roll towards the lowest point (global minima), or a small pothole (local minima), or even some weird looking shape (saddle point) if you are very unlucky.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With some fixed input/output pair, we can consider the loss function as a value of the parameters (weights). Here, the Z axis is the value of your loss &lt;em&gt;f&lt;/em&gt; (what you are trying to optimize), while the X,Y axis are the parameters of the loss &lt;em&gt;f&lt;/em&gt; (what you are allowed change):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sMaC5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sMaC5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the image provided is a bit misleading. In practice, we only know the value of the loss function at the location we have probed. The entire graph can only be drawn if we evaluate the loss function for every parameter. So we can't just look at the picture and know which location is the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that the goal of the prediction task is to update the parameters in a way such that we minimize the loss function for some given input/output pair. Assuming the loss function is reasonably smooth, then our best bet of reaching a lower value of the error rate is by taking a small step towards the direction of the slope, which is the derivative of the function. The size of the &quot;small step&quot; is called the learning rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under the assumption that the loss function looks similar given different input/output pairs, by iteratively moving down the slope for each datapoint, we can eventually reach a &quot;good&quot; set of weights that give a low error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the motivation and idea of the back-propagation algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good follow-up question might be: Why do we say that the loss function is  similar given different data points? This is basically the same as assuming that there is some regularity in the data such that there is a &quot;most normal looking loss function&quot; that the data-set naturally approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully that helps!&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-01T21:32:55.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4643" PostTypeId="1" CreationDate="2017-12-01T21:59:34.857" Score="4" ViewCount="2422" Body="&lt;p&gt;I'm attempting to make a bot for the Connect 4 competition on &lt;a href=&quot;http://riddles.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://riddles.io&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My bot isn't horrible, like it's getting up the ladder, but it cannot compete with the winning bots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using a Neural Network which is fully connected with one hidden layer. Internally it uses the sigmoid function as the activator in each Neuron. I've trained it over 500,000 games with TD-Lambda back propagation. The alpha and beta values (i.e. learning rates) are set to &lt;code&gt;0.1&lt;/code&gt; each, and the lambda for the eligibility trace is set to &lt;code&gt;0.7&lt;/code&gt;. There are 2 outputs nodes, one to give the value of this position for Player 1, the other to give the value of the position for Player 2. Upon a win, these are back-propagated with &lt;code&gt;-1&lt;/code&gt; for a loss and &lt;code&gt;1&lt;/code&gt; for a win, on a draw they are both back-propagated with a &lt;code&gt;0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a bias input and weight for every neuron as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the weights are initialised to a random value between &lt;code&gt;+-4*sqrt(6/totalNumberWeights)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The board state is represented to the network as:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For each space in the board, 2 values:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For the first value if player 1 occupies this space it's a &lt;code&gt;1&lt;/code&gt;, otherwise it's a &lt;code&gt;0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;For the second value if player 2 occupies this space it's a &lt;code&gt;1&lt;/code&gt;, otherwise it's a &lt;code&gt;0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;If both are &lt;code&gt;0&lt;/code&gt; it would mean it's a free spot&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;For each space on the board another 2 values:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For the first value, if placing a token here would result in a Connect 4 for player 1, it's a &lt;code&gt;1&lt;/code&gt;, otherwise a &lt;code&gt;0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;For the second value, if placing a token would get player 2 a Connect 4, then it's a &lt;code&gt;1&lt;/code&gt; otherwise a &lt;code&gt;0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;So if no one would win by placing a token here, it's two &lt;code&gt;0&lt;/code&gt; values for these two inputs&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Two final values, the first indicating whether it's player 1's turn or not, then second indicating if it's player 2's turn or not&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What I see when my bot is playing is that it makes what I assume are somewhat clever moves, like it's preparing for both horizontal and diagonal moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when the other bot will get a Connect 4 on the next move, my bot fails to place a token there. The best bot in the comp seems capable of setting itself up to get three in a row with a free space on both sides, so that it will definitely get a Connect 4. Again, my bot does not seem to be able to see this coming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I think the issue is, other than potentially my learning rates, is that I've represented the board in a bad way. Is there a better way to represent it to the Network so that it can more accurately estimate the value of a board state, and so that it doesn't fail to identify an immediate Connect 4 threat?&lt;/p&gt;&#xA;" OwnerUserId="2819" LastEditorUserId="2819" LastEditDate="2017-12-01T23:34:04.463" LastActivityDate="2019-01-27T03:02:55.513" Title="Is this a good way to represent Connect 4 to a Neural Network?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4644" PostTypeId="1" CreationDate="2017-12-01T23:39:24.043" Score="2" ViewCount="138" Body="&lt;p&gt;I would like to measure time for forward and backward times on TF-Slim (over all network and per-layer) like caffe does. However, it just logs the step/iteration time, and I have no idea of how to do that without changing the source and recompiling (takes a lot of time on my machine)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also would like to log a not-rounded loss value (it rounds to 4 digits after dot)&lt;/p&gt;&#xA;" OwnerUserId="11289" LastActivityDate="2017-12-01T23:39:24.043" Title="How to measure times on TensorFlow Slim?" Tags="&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4645" PostTypeId="1" AcceptedAnswerId="4646" CreationDate="2017-12-02T14:55:15.743" Score="6" ViewCount="451" Body="&lt;p&gt;The naïve Bayes' generative algorithm is often represented by the following formula:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{argmax}_{y} p(y|x) = \text{argmax}_y \frac{p(x|y)p(y)}{p(x)} \approx \text{argmax}_y p(x|y)p(y)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why do we have &lt;span class=&quot;math-container&quot;&gt;$p(x)=1$&lt;/span&gt; which allows the approximation &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}_y \frac{p(x|y)p(y)}{p(x)} \approx \text{argmax}_y p(x|y)p(y)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="11303" LastEditorUserId="45501" LastEditDate="2021-03-19T04:26:54.580" LastActivityDate="2021-03-19T04:26:54.580" Title="Why is the denominator ignored in the Bayes' rule?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;naive-bayes&gt;&lt;bayes-theorem&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4646" PostTypeId="2" ParentId="4645" CreationDate="2017-12-02T16:50:43.943" Score="4" Body="&lt;p&gt;The answer from @kiner_shah in the comments has solved it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;They have eliminated it because in comparing probability for best outcome, it would just introduce additional division and the divisor p(x) is same for all candidates. For example, if I tell you to compare the largest of the following numbers: 3/2, 5/2, 1/2, you will without thinking pick 5/2 since the divisors are same and 5 is greatest of all.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="11303" LastActivityDate="2017-12-02T16:50:43.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4647" PostTypeId="1" CreationDate="2017-12-02T22:32:26.803" Score="9" ViewCount="370" Body="&lt;p&gt;There has been recent uptick in interest in e&lt;strong&gt;X&lt;/strong&gt;plainable &lt;strong&gt;A&lt;/strong&gt;rtificial &lt;strong&gt;I&lt;/strong&gt;ntelligence (XAI). Here is XAI's mission as stated on its &lt;a href=&quot;https://www.darpa.mil/program/explainable-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;DARPA page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Explainable AI (XAI) program aims to create a suite of machine learning techniques that:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and&lt;/li&gt;&#xA;  &lt;li&gt;Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The New York Times article &lt;a href=&quot;https://www.nytimes.com/2017/11/21/magazine/can-ai-be-taught-to-explain-itself.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Can A.I. Be Taught to Explain Itself?&lt;/a&gt; does a good job at explaining the need for XAI from a human interest perspective as well as providing a glancing outlook on the techniques being developed for the same. The force behind XAI movement seems to center around the (up and coming?) concept of &lt;strong&gt;&lt;em&gt;right to explanation&lt;/em&gt;&lt;/strong&gt;, that is, the requirement that AI applications that significantly impact human lives via their decisions be able to explain to stakeholders the factors/reasons leading up to said decision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the right to explanation reasonable, given the current standards at which we hold each other accountable?&lt;/p&gt;&#xA;" OwnerUserId="11166" LastEditorUserId="2444" LastEditDate="2021-01-26T15:44:21.457" LastActivityDate="2021-01-26T15:44:21.457" Title="How is the &quot;right to explanation&quot; reasonable?" Tags="&lt;philosophy&gt;&lt;ethics&gt;&lt;explainable-ai&gt;&lt;legal&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4648" PostTypeId="2" ParentId="4218" CreationDate="2017-12-03T02:44:04.333" Score="1" Body="&lt;p&gt;Other answers are good, but kiner-shah comment about Decision Trees worked the best for me due to how the data was structured. &lt;/p&gt;&#xA;" OwnerUserId="10040" LastEditorDisplayName="user10259" LastEditDate="2018-08-20T19:21:13.893" LastActivityDate="2018-08-20T19:21:13.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4649" PostTypeId="2" ParentId="4610" CreationDate="2017-12-03T17:11:19.390" Score="1" Body="&lt;p&gt;I suppose, the context was as follows: suppose you have AI for which goals cannot be described by a function. It doesn't mean it has no goals, but that formulation of the goals cannot be functionally dependent on known reality model parameters. &#xA;In this interpretation text which you cited has clear meaning: author doesn't want to discuss such kinds of object, which is reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such interpretation is interesting as well, because opens interesting cases to think about :)&lt;/p&gt;&#xA;" OwnerUserId="5356" LastActivityDate="2017-12-03T17:11:19.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4650" PostTypeId="1" CreationDate="2017-12-04T06:03:27.163" Score="2" ViewCount="893" Body="&lt;p&gt;I am currently looking for AI use cases for Telco. What are the different AI use cases for Telcos/communication service providers?&lt;/p&gt;&#xA;" OwnerUserId="11346" LastEditorUserId="19413" LastEditDate="2019-03-10T20:19:55.447" LastActivityDate="2019-07-31T11:53:09.843" Title="What are AI use cases for communication service providers?" Tags="&lt;applications&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4652" PostTypeId="2" ParentId="4610" CreationDate="2017-12-04T08:02:35.857" Score="1" Body="&lt;p&gt;Following the standard AI textbook, &lt;em&gt;Artificial Intelligence:  A modern approach&lt;/em&gt;, other from goals based agents there are &lt;strong&gt;Reflex Agents, Model based Reflex agents, Model based Goal based agents, Utility agents and Learning agents.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple Reflex agents are based on condition action rules, kinda like If-then rule based agents.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0Vl4Y.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Vl4Y.jpg&quot; alt=&quot;Reflex agents&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model based agents are whose which have a model of the world in it, I mean &lt;em&gt;how the world evolves and what my action will do&lt;/em&gt;&#xA;They are in two variants, Model based reflex agent and model based goal directed agent.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9kP6E.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9kP6E.jpg&quot; alt=&quot;Model based reflex agents&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Former one percepts the environment, infer from that how the world evolves and figure out what their action will do then that have some conditon base rules which then lead to action, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/89zPC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/89zPC.jpg&quot; alt=&quot;Model based goal directed agents&quot;&gt;&lt;/a&gt;In latter type the module of conditon based rules are replaced by the module of goals, which can be kinda further broken into condition based rules,which they are literally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hdeyE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hdeyE.jpg&quot; alt=&quot;Utility based agents&quot;&gt;&lt;/a&gt;&#xA;Utility agents are reinforcement based agent they act in environment to maximize their utility, and have some element to check what the happiness measure or &quot;Performance measure&quot; is, the defintion of rational Agent is based on this, which is a rational agent will always act to maximize their average utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sy0w0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sy0w0.jpg&quot; alt=&quot;Learning agents&quot;&gt;&lt;/a&gt;&#xA;Learning agent are which contain some kinda critics, some learning element and then problem generator, an isomorphism might be drawn from to Neural networks, which has their critics as Cost function, learning element as Wieght tuning.&#xA;I have been saying &quot;kinda&quot; because they just seems to be metaphor to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dont know why Marcus hutler described goals agents as Utility based agents, probably he had some kinda hybrid of two.&lt;/p&gt;&#xA;" OwnerUserId="8068" LastEditorUserId="8068" LastEditDate="2017-12-04T08:12:22.560" LastActivityDate="2017-12-04T08:12:22.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4653" PostTypeId="2" ParentId="1294" CreationDate="2017-12-04T13:01:41.203" Score="4" Body="&lt;p&gt;Capsule networks try to mimic Hinton's observations of the human brain on the machine. The motivation stems from the fact that neural networks needed better modeling of the spatial relationships of the parts. Instead of modeling the co-existence, disregarding the relative positioning, capsule-nets try to model the global relative transformations of different sub-parts along a hierarchy. This is the eqivariance vs. invariance trade-off, as explained above by others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These networks therefore include somewhat a viewpoint / orientation awareness and respond differently to different orientations. This property makes them more discriminative, while potentially introducing the capability to perform pose estimation as the latent-space features contain interpretable, pose specific details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this is accomplished by including a nested layer called capsules within the layer, instead of concatenating yet another layer in the network. These capsules can provide vector output instead of a scalar one per node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The crucial contribution of the paper is the dynamic routing which replaces the standard max-pooling by a smart strategy. This algorithm applies a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_shift&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean-shift clustering&lt;/a&gt; on the capsule outputs to ensure that the output gets sent only to the appropriate parent in the layer above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Authors also couple the contributions with a margin loss and reconstruction loss, which simultaneously help in learning the task better and show state of the art results on MNIST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recent-paper is named &lt;a href=&quot;https://arxiv.org/pdf/1710.09829.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing Between Capsules&#xA;&lt;/a&gt; and is available on Arxiv: &lt;a href=&quot;https://arxiv.org/pdf/1710.09829.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1710.09829.pdf&lt;/a&gt; . &lt;/p&gt;&#xA;" OwnerUserId="11353" LastActivityDate="2017-12-04T13:01:41.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4654" PostTypeId="2" ParentId="4394" CreationDate="2017-12-04T13:15:49.080" Score="6" Body="&lt;p&gt;&lt;em&gt;Why has this merge proven beneficial?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think about the shared Value/Policy network as consisting of a shared component (the Residual Network layers) with a Value and Policy component on top rather than Separation of Concerns it makes more sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying premise is that the shared part of the network (the ResNet) provides a high-level generalization of the input (the game states leading up to the move) that is a good input representation for both the shallow Value and Policy networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When that is the case, we can reduce the computational load a lot by training a single shared ResNet and using it for two much simpler networks than training two ResNets for the Value and Policy. In their case, training the two together also improves regularisation and thus creates a more robust, general representation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, the Alpha Go Zero paper by Silver &lt;em&gt;et al.&lt;/em&gt;, &lt;em&gt;Mastering the Game of Go without Human Knowledge&lt;/em&gt;, states that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Combining policy and value together into a single network slightly reduced the &#xA;  move prediction accuracy, but reduced the value error and boosted playing &#xA;  performance in AlphaGo by around another 600 Elo. This is partly due to&#xA;  improved computational efficiency, but more importantly the dual objective regularises the network to a common representation that supports multiple use cases.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Can this technique be applied in general or only in special cases?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like common components in software libraries, it only makes sense when the problems you are trying to solve benefit from a shared representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use it if you are training classifiers for similar tasks, or training a new task with little data where you already have a classifier trained over a larger, &lt;em&gt;similar&lt;/em&gt; dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outside of Go, it is often used in image recognition. &#xA;Deep pre-trained networks such as the ones from the ImageNet &lt;a href=&quot;http://image-net.org/challenges/LSVRC/2017/&quot; rel=&quot;noreferrer&quot;&gt;ILSVRC&lt;/a&gt; competitions are commonly used as a starting point. They are classifiers that have been trained (for weeks!) on over a million images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, say you want to create a network to recognize you favourite brand of bicycles, you start with the general image-recognition pipeline trained on ImageNet, chop of the last layers that do the actual classification (&quot;it's a Border Collie&quot;) and add a small new classifier to pick out only the bicycles you care about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the pre-trained classifier already provides high-level image concepts that are good building blocks for image recognition (it classifies 200 categories), this saves you a lot of training and makes for a very robust classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there are many cases where the problems do not have useful shared representations and thus no benefit from a combined network. Nevertheless, it is a useful tool in the right situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look up &lt;strong&gt;Transfer Learning&lt;/strong&gt; or &lt;strong&gt;Multi-Task Learning&lt;/strong&gt; to learn more about this.&lt;/p&gt;&#xA;" OwnerUserId="10368" LastEditorUserId="10368" LastEditDate="2017-12-04T15:51:35.273" LastActivityDate="2017-12-04T15:51:35.273" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4655" PostTypeId="1" AcceptedAnswerId="4707" CreationDate="2017-12-04T14:42:49.947" Score="21" ViewCount="5635" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Given a neural network &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that takes as input &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; data points: &lt;span class=&quot;math-container&quot;&gt;$x_1, \dots, x_n$&lt;/span&gt;. We say &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is &lt;strong&gt;permutation invariant&lt;/strong&gt; if&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x_1 ... x_n) = f(\sigma(x_1 ... x_n))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for any permutation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How could we build such a neural network? The output can be anything (e.g. a number or vector), as long as it does not depend on the order of inputs. Is there any research work on these types of neural networks?&lt;/p&gt;&#xA;" OwnerUserId="11359" LastEditorUserId="2444" LastEditDate="2020-11-06T02:23:32.777" LastActivityDate="2020-11-06T02:29:39.520" Title="How could we build a neural network that is invariant to permutations of the inputs?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4656" PostTypeId="2" ParentId="4398" CreationDate="2017-12-04T14:59:44.620" Score="1" Body="&lt;p&gt;You can limit the absolute values of the outputs by &quot;punishing&quot; large values in the loss function. This is done by adding an extra term to the loss function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if your existing loss function is &lt;em&gt;L(yhat, y)&lt;/em&gt; where &lt;em&gt;yhat&lt;/em&gt; is the output and &lt;em&gt;y&lt;/em&gt; the correct value, create a new loss function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;L'(yhat, y) = L(yhat,y) + max(0, ||yhat||&lt;sub&gt;1&lt;/sub&gt; - k)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the outputs are &quot;small&quot; the L&lt;sub&gt;1&lt;/sub&gt; norm of your output &lt;em&gt;yhat&lt;/em&gt; is less than &lt;em&gt;k&lt;/em&gt; and imposes no extra loss (keeping everything as before). However once the norm grows above &lt;em&gt;k&lt;/em&gt; it starts to impose a loss that guides the training to reduce the absolute values of the outputs values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also limit the values in the network itself by selecting a bounded activation function for the output, for example &lt;em&gt;tanh&lt;/em&gt; or &lt;em&gt;sigmoid&lt;/em&gt;. However, you need to pick an activation function that suits your problem.&lt;/p&gt;&#xA;" OwnerUserId="10368" LastActivityDate="2017-12-04T14:59:44.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4660" PostTypeId="1" AcceptedAnswerId="4668" CreationDate="2017-12-04T23:12:58.733" Score="7" ViewCount="4549" Body="&lt;p&gt;I have followed a course on machine learning, where we learned about the gradient descent (GD) and back-propagation (BP) algorithms, which can be used to update the weights of neural networks, and reinforcement learning, in particular, Q-learning. I implemented these concepts separately.&lt;/p&gt;&#xA;&lt;p&gt;Now, I was thinking about using a neural network to approximate the Q-function, &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, but I don't really know how to design the neural network and how to use back-propagation to update the weights of this neural network (NN).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What should the inputs and outputs of this NN be?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I use GD and BP to update the weights of such an NN? Or should I use a different algorithm to update the weights?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10364" LastEditorUserId="2444" LastEditDate="2020-12-28T13:53:13.217" LastActivityDate="2020-12-28T13:53:13.217" Title="How to combine backpropagation in neural nets and reinforcement learning?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;backpropagation&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4661" PostTypeId="5" CreationDate="2017-12-05T00:40:22.560" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-10-10T22:22:45.347" LastActivityDate="2020-10-10T22:22:45.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4662" PostTypeId="4" CreationDate="2017-12-05T00:40:22.560" Score="0" Body="For questions about the back-propagation (aka &quot;backprop&quot;, and often abbreviated as &quot;BP&quot;) algorithm, which is used to compute the gradient of the objective function (e.g. the mean squared error) with respect to the parameters (or weights) of the neural network, when trained with gradient descent." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-10-10T22:22:45.347" LastActivityDate="2020-10-10T22:22:45.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4663" PostTypeId="1" AcceptedAnswerId="4665" CreationDate="2017-12-05T01:22:40.723" Score="5" ViewCount="589" Body="&lt;p&gt;Collecting and labeling training data for supervised learning tasks is incredibly time-consuming and costly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, let's say you wrote a script that went on Google images and got you 5000 pictures for each of 10 classes. You then use an unsupervised algorithm to cluster them. Then, you train a supervised algorithm using the labels from the scraper as ground truth. Obviously, your network will perform more poorly than one with perfectly labeled data, but is there a way to guesstimate how much?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps there are 50 mislabeled images in each class. That would most likely be better than 500 mislabeled images, but I'm wondering if there is a way to predict how much (even if it is by someone's rules of thumb or something like that).&lt;/p&gt;&#xA;" OwnerUserId="8829" LastEditorUserId="2444" LastEditDate="2020-12-20T13:16:31.533" LastActivityDate="2020-12-20T13:16:31.533" Title="What is the effect of mislabeled training data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;labeled-datasets&gt;&lt;data-labelling&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4665" PostTypeId="2" ParentId="4663" CreationDate="2017-12-05T05:06:09.620" Score="3" Body="&lt;p&gt;I think the crucial point here is what you precisely mean by &lt;em&gt;mislabelled&lt;/em&gt;. Google's image classifier will likely do a 'pretty good' job of retrieving images with the given subject included, but how strict or lenient your class requisites are is quite important. For example, if one of your classes is 'dog' there may be hundreds of images procured from scraping that could display (examples off the top of my head, but you can get even more creative): &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ancient canine fossils&lt;/li&gt;&#xA;&lt;li&gt;Wolves&lt;/li&gt;&#xA;&lt;li&gt;Stuffed animal dogs&lt;/li&gt;&#xA;&lt;li&gt;Hard mode dogs (i.e. partially occluded, variable lighting schemas, background color variation, intraclass/species variation)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Additionally, your computational tool will impact this. If you're using a neural network, the above issues can to some extent be accounted for, but a linear classifier would likely have difficulty adopting a broader/flexible view of your class. &lt;/p&gt;&#xA;" OwnerUserId="10188" LastEditorUserId="10188" LastEditDate="2017-12-18T20:38:17.267" LastActivityDate="2017-12-18T20:38:17.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4666" PostTypeId="1" AcceptedAnswerId="4667" CreationDate="2017-12-05T07:34:53.323" Score="2" ViewCount="2089" Body="&lt;p&gt;I'm training two CNNs (AlexNet e GoogLeNet) in two differents DL libraries (Caffe e Tensorflow). The networks was implemented by dev teams of each libraries (&lt;a href=&quot;https://github.com/BVLC/caffe/tree/master/models&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research/slim/nets&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I reduced the original Imagenet dataset to 1024 images of 1 category -- but setted 1000 categories to classify on the networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I trained the CNNs, varying processing unit (CPU/GPU) and batches sizes, and I observed that the losses converges fastly to near zero (in mostly times before 1 epoch be completed), like in this graph (Alexnet on Tensorflow):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GXeEK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GXeEK.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In portuguese, 'Épocas' is epochs and 'Perda' is loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weight decays and initial learning rate are the same as used on models that I downloaded, I only changed the dataset and the batch sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why my networks are converging this way, and not like &lt;a href=&quot;https://github.com/BVLC/caffe/issues/3801#issuecomment-194654755&quot; rel=&quot;nofollow noreferrer&quot;&gt;this way&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="11289" LastActivityDate="2017-12-05T17:55:48.067" Title="What fast loss convergence indicates on a CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4667" PostTypeId="2" ParentId="4666" CreationDate="2017-12-05T17:55:48.067" Score="2" Body="&lt;p&gt;Ok, so the fact that you always give the same category means the network only has to output the same value for any given input. This is exceedingly easy to learn (all the weights always shift in the same directions) and this make this graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have more categories (try 10 for example), you'll see the graph will be closer to expected.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2017-12-05T17:55:48.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4668" PostTypeId="2" ParentId="4660" CreationDate="2017-12-05T18:39:00.997" Score="8" Body="&lt;h3&gt;Gradient descent and back-propagation&lt;/h3&gt;&#xA;&lt;p&gt;In deep learning, gradient descent (GD) and back-propagation (BP) are used to update the weights of the neural network.&lt;/p&gt;&#xA;&lt;p&gt;In reinforcement learning, one could map (state, action)-pairs to Q-values with a neural network. Then GD and BP can be used to update the weights of this neural network.&lt;/p&gt;&#xA;&lt;h3&gt;How to design the neural network?&lt;/h3&gt;&#xA;&lt;p&gt;In this context, a neural network can be designed in different ways. A few are listed below:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The state and action are concatenated and fed to the neural network. The neural network is trained to return a single Q-value belonging to the previously mentioned state and action.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each action, there is a neural network that provides the Q-value given a state. This is not desirable when a lot of actions exist.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another option is to construct a neural network that accepts as input the state. The output layer consists of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;-units where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is the number of possible actions. Each output unit is trained to return the Q-value for a particular action.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h3&gt;Q-learning update rule&lt;/h3&gt;&#xA;&lt;p&gt;This is the Q-learning update rule&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OMzXf.png&quot; alt=&quot;Update rule&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, we choose an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; (for example, with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy behavior policy) in the state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;. Once the action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; has been taken, we end up in a new state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; and receive a reward &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}$&lt;/span&gt; associated with it. To perform the update, we also need to choose the Q-value in &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; associated with the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\max _{a} Q\left(s_{t+1}, a\right)$&lt;/span&gt;. Here, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the discount factor and &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate.&lt;/p&gt;&#xA;&lt;h3&gt;Back-propagation and Q-learning&lt;/h3&gt;&#xA;&lt;p&gt;If we use a neural network to map states (or state-action pairs) to Q-values, we can use a similar update rule, but we use GD and BP to update the weights of this neural network.&lt;/p&gt;&#xA;&lt;p&gt;Here's a possible implementation of a function that would update the weights of such a neural network.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def update(self, old_state, old_action, new_state, &#xA;  reward, isFinalState = False):&#xA; &#xA;  # The neural network has a learning rate associated with it. &#xA;  # It is advised not to use two learning rates&#xA;  learningRate = 1&#xA;  &#xA;  # Obtain the old value&#xA;  old_Q = self.getQ(old_state, old_action)&#xA;&#xA;  # Obtain the max Q-value&#xA;  new_Q = -1000000&#xA;  action = 0&#xA;  for a in self.action_set:&#xA;    q_val = self.getQ(new_state, a)&#xA;    if (q_val &amp;gt; new_Q):&#xA;      new_Q = q_val&#xA;      action = a  &#xA;&#xA;  # In the final state there is no action to be chosen&#xA;  if isFinalState:&#xA;    diff = learningRate * (reward - old_Q)&#xA;  else:&#xA;    diff = learningRate * (reward + self.discount * new_Q - old_Q)&#xA;&#xA;  # Compute the target &#xA;  target = old_Q + diff&#xA;&#xA;  # Update the Q-value using backpropagation&#xA;  self.updateQ(action, old_state, target)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the pseudocode and in the Q-learning update formula above, you can see the discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;. This simply denotes whether we are interested in an immediate reward or a more rewarding and enduring reward later on. This value is often set to 0.9.&lt;/p&gt;&#xA;" OwnerUserId="10468" LastEditorUserId="2444" LastEditDate="2020-12-28T13:44:27.907" LastActivityDate="2020-12-28T13:44:27.907" CommentCount="1" CommunityOwnedDate="2020-12-29T10:59:27.230" ContentLicense="CC BY-SA 4.0" />
  <row Id="4669" PostTypeId="1" CreationDate="2017-12-06T02:14:32.017" Score="2" ViewCount="72" Body="&lt;p&gt;Let's suppose I have 5 images, all of which I assure you are of the same item, but from various angles and perhaps different lighting conditions. I now supply you with an additional image, and I want a score of how likely this image is to contain the item depicted in the first five pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us suppose that the item isn't too complex. It won't be a pile of fabric with a pattern on it, dropped several different ways, and won't be a keychain with keys in different conformations. It will also be more complex than just a blue ball shot from different angles. How might you approach the problem of scoring this image?&lt;/p&gt;&#xA;" OwnerUserId="11410" LastActivityDate="2017-12-08T17:17:56.303" Title="Identifying if an image contains an object with very small (five image) training data set" Tags="&lt;image-recognition&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4670" PostTypeId="2" ParentId="4669" CreationDate="2017-12-06T08:19:55.203" Score="1" Body="&lt;p&gt;I would do the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Add some images without the item present to your training data. Ideally some other item taken against similar backgrounds, lighting etc. And maybe some of just the background.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Obtain a pre-trained neural network CNN model, such as AlexNet, VGG-19 or Inception v5.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Load the neural network model, remove the very last layer (typically a 1000-way softmax), and replace with a brand new layer for classifying whether your object is present or not. Maybe use a 2-way softmax for &quot;my object&quot; vs &quot;unknown object&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Freeze all the layers other than your own (most NN libraries will have a way to do this, or you could just run the old network truncated to extract a feature vector from penultimate layer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train using your 5 images. This kind of limited training is often called &quot;fine tuning&quot;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find some way to test your classifier before relying on it. You may have to do a leave-one-out test where you train your network 5 times, each with 4 of the images, and see whether it correctly identifies the 5th one vs a few images without the item in it. Even with that approach, with such a small sample, you will have only the vaguest idea of how reliable your new classifier is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You'll do a lot better if you can obtain more images to train and test with. If you get enough images (both positive and negative), you can consider replacing more layers before fine tuning. Otherwise you are heavily relying on your object being similar enough to something from the ImageNet collection that a classifier trained on that will extract useful high-level features.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2017-12-06T15:03:29.097" LastActivityDate="2017-12-06T15:03:29.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4671" PostTypeId="1" CreationDate="2017-12-06T09:09:36.967" Score="2" ViewCount="356" Body="&lt;p&gt;I am trying to build a neural network suitable to measure similarity between pairs of images. In particular I am interested in shoes. I have a query image (e.g. a shoe that I just took a picture of) and I want to find similar shoes in a database (several thousands of images).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using MAC feature (e.g. max pool over the entire spacial dimension on last (or some other) convolution layer of say VGG16) (here is a link to a paper &lt;a href=&quot;https://arxiv.org/pdf/1511.05879.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1511.05879.pdf&lt;/a&gt;). The two MAC vectors are compared using cosine similarity. That works, but among the top matches there are always a few very strange shoes (e.g when I submit a query image with a boot I find sandals among other boots with extremely high similarity score).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be a better way of doing that? Something more robust to finding shoes similar in shape to the query image. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2017-12-06T09:09:36.967" Title="Similarity of images (CBIR) with CNN features" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4672" PostTypeId="1" CreationDate="2017-12-06T19:00:23.507" Score="0" ViewCount="855" Body="&lt;p&gt;Can an AI learn to play chess if you give it nothing but &quot;the goal is to win&quot; as starting criteria? If not, what is the minimum information the AI would need to be &quot;seeded&quot; with in order to learn to play chess?  What techniques could be used to create an AI that learns to play chess independently? &lt;/p&gt;&#xA;" OwnerUserId="7801" LastEditorUserId="33" LastEditDate="2017-12-11T20:13:41.930" LastActivityDate="2018-12-12T06:53:16.953" Title="Can an AI learn how to play chess without instructions?" Tags="&lt;gaming&gt;&lt;game-ai&gt;&lt;chess&gt;&lt;combinatorial-games&gt;" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4674" PostTypeId="2" ParentId="4672" CreationDate="2017-12-07T02:13:10.673" Score="0" Body="&lt;p&gt;Yes. “On the 6th of December, 2017, AlphaZero took over the chess world&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;http://www.telegraph.co.uk/science/2017/12/06/entire-human-chess-knowledge-learned-surpassed-deepminds-alphazero/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article link&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arxiv paper&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="11440" LastEditorUserId="4398" LastEditDate="2017-12-07T14:40:46.470" LastActivityDate="2017-12-07T14:40:46.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4675" PostTypeId="1" AcceptedAnswerId="4676" CreationDate="2017-12-07T02:19:36.767" Score="4" ViewCount="867" Body="&lt;p&gt;Currently, AI is advancing fast in deep learning: &lt;a href=&quot;http://www.telegraph.co.uk/science/2017/12/06/entire-human-chess-knowledge-learned-surpassed-deepminds-alphazero/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entire human chess knowledge learned and surpassed by DeepMind's AlphaZero in four hours&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As a layman, I'm taking this as a quite powerful searching algorithm, using artificial neural networks to identify the patterns of each game.&lt;/p&gt;&#xA;&lt;p&gt;However, how good is AI doing in math?&lt;/p&gt;&#xA;&lt;p&gt;For example, the key to the theory of the game &lt;a href=&quot;https://en.wikipedia.org/wiki/Nim&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nim&lt;/a&gt; is the binary digital sum of the heap sizes, that is, the sum (in binary) neglecting all carries from one digit to another. This operation is also known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Exclusive_or&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;exclusive or&amp;quot; (xor)&lt;/a&gt; or &amp;quot;vector addition over GF(2)&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Is AI good enough to discover/invite operations/logics such as &amp;quot;exclusive or&amp;quot;, or, more advanced, abstract algebra in &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_field&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite field&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="11440" LastEditorUserId="2444" LastEditDate="2020-06-28T22:17:12.307" LastActivityDate="2020-06-28T22:18:23.757" Title="How good is AI in math?" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;agi&gt;&lt;applications&gt;&lt;state-of-the-art&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4676" PostTypeId="2" ParentId="4675" CreationDate="2017-12-07T12:33:03.930" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Nim&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nim was actually&lt;/a&gt; one of the first games ever played by an electronic machine. It was called the Nimatron and was displayed at the 1940 New York World's Fair.&lt;/p&gt;&#xA;&lt;p&gt;It is also well known that neural networks can model the Xor-function, if they have enough layers. Despite that, Marvin Minsky is supposed to have killed neural networks in the sixties, by asserting that networks with a single hidden layer, cannot model XOR.&lt;/p&gt;&#xA;&lt;p&gt;I think there were also some large scale projects to classify finite groups using computers.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, there have long been &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_theorem_proving&quot; rel=&quot;nofollow noreferrer&quot;&gt;automated theorem-proving program&lt;/a&gt;s, that had some success in mathematical logic. I recently saw &lt;a href=&quot;https://arxiv.org/pdf/1606.04442.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt; that used deep learning to improve one of these theorem provers.&lt;/p&gt;&#xA;&lt;p&gt;That being said, doing real mathematics is probably as difficult as real language understanding. There doesn't seem to have been a real breakthrough yet.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2020-06-28T22:18:23.757" LastActivityDate="2020-06-28T22:18:23.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4677" PostTypeId="1" CreationDate="2017-12-07T14:32:11.810" Score="8" ViewCount="669" Body="&lt;p&gt;I am using Keras to train different NN. I would like to know why if I increment the epochs in 1, the result until the new epoch is not the same. I am using shuffle=False, and np.random.seed(2017), and I have check that if I repeat with the same number of epochs, the result is the same, so not random initialization is working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I attach the picture of the resulting training with 2 epochs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DtLWl.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DtLWl.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&#xA;And here I attach the picture of the resulting training with 3 epochs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/auDvm.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/auDvm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I would like to know why the training time is not (3/2) and how is it possible that some of them have less accuracy with one more epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="11391" LastActivityDate="2018-05-31T22:09:15.300" Title="Why does 'loss' change depending on the number of epochs chosen?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;optimization&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4678" PostTypeId="1" AcceptedAnswerId="4704" CreationDate="2017-12-07T17:17:44.947" Score="3" ViewCount="634" Body="&lt;p&gt;I was thinking, what if we could combine Artificial Intelligence (Neural network for image recognition), computer hardware and a security camera for identify any breaking into our backyard at 12:00am - 8:00am? Of course my current knowledge leads me to only a simple question. So, in order to have a general idea:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;¿Have been this already solved using a commercial or free software?&lt;/li&gt;&#xA;&lt;li&gt;¿Can this be done using TensorFlow?&lt;/li&gt;&#xA;&lt;li&gt;¿Is there any free set of images with millions of them to teach any AI distinguish between a man and another moving object?&lt;/li&gt;&#xA;&lt;li&gt;¿Approximate hardware requirements for doing this?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If this question could be silly please mark it as off-topic. I based this idea on autonomous driving car, they can both recognize images and drive at the same time. Unless they have within a super computer I guess maybe the previous idea can be fulfill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update 1: I found this &lt;a href=&quot;https://ai.stackexchange.com/questions/1453/can-convnets-be-used-for-real-time-object-recognition-from-video-feed&quot;&gt;Can ConvNets be used for real-time object recognition from video feed?&lt;/a&gt; but I guess it could be outdated. Right now I'm in the land of &quot;maybe&quot; (lack of knowledge).&lt;/p&gt;&#xA;" OwnerUserId="11451" LastEditorUserId="11451" LastEditDate="2017-12-07T20:18:49.093" LastActivityDate="2017-12-10T21:04:32.487" Title="Real time image processing for object recognition using security cameras" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="4679" PostTypeId="2" ParentId="4660" CreationDate="2017-12-07T17:19:40.573" Score="3" Body="&lt;p&gt;You should read up on these papers:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/nature14236&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-Networks&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v48/mniha16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Deep Reinforcement Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both by DeepMind. They achieved super-human results on video-games and other tasks. They describe the algorithms quite well. It is not as simple as the previous answer, which won't converge to a policy in complex environments.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="2444" LastEditDate="2020-12-28T13:42:43.923" LastActivityDate="2020-12-28T13:42:43.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4680" PostTypeId="1" CreationDate="2017-12-07T21:00:24.577" Score="5" ViewCount="69" Body="&lt;p&gt;I'm trying to get up to speed on the latest research regarding indoor localization, scene classification, navigation in changing environment, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice would be appreciated, but I'm especially interested in recent research papers from vetted sources.  &lt;/p&gt;&#xA;" OwnerUserId="11455" LastEditorUserId="1671" LastEditDate="2017-12-08T19:03:45.647" LastActivityDate="2019-03-11T20:49:33.250" Title="Current research on indoor localization and navigation in changing environment?" Tags="&lt;machine-learning&gt;&lt;path-planning&gt;&lt;scene-classification&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4681" PostTypeId="1" CreationDate="2017-12-08T03:15:30.290" Score="1" ViewCount="32" Body="&lt;p&gt;I'm trying to understand how to build the ANN on cognitrons, so I have read theory for that topic and found the scheme:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CCPa1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CCPa1.png&quot; alt=&quot;Cognitrons&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;As I got neurons are subdivided in two classes: the exciting and the inhibitory. I have written these classes:&lt;br&gt;&#xA;&lt;strong&gt;Exciting neuron:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;typedef float signal;&#xA;typedef std::vector&amp;lt;signal&amp;gt; sigvec;&#xA;&#xA;class inhibitory_neuron&#xA;{&#xA;public:&#xA;    inhibitory_neuron() {}&#xA;    ~inhibitory_neuron() {}&#xA;    virtual signal call(const sigvec &amp;amp;e_inputs, const sigvec &amp;amp;i_inputs)&#xA;    {&#xA;        unused(i_inputs);&#xA;        signal sum = 0;&#xA;&#xA;        for (auto i = std::begin(e_inputs); i != std::end(e_inputs); ++i) {&#xA;            sum += *i;&#xA;        }&#xA;&#xA;        return sum;&#xA;    }&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;e_inputs&lt;/code&gt; are &lt;code&gt;c&lt;/code&gt;-factors (sum of &lt;code&gt;c&lt;/code&gt; is &lt;code&gt;1&lt;/code&gt;). But in the formula:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5cR97.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5cR97.png&quot; alt=&quot;Formula&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;I can't get it... Whaz &lt;code&gt;OUT_i&lt;/code&gt; there?&#xA;And below my &lt;strong&gt;Exciting neuron&lt;/strong&gt; class is:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class exciting_neuron : inhibitory_neuron&#xA;{&#xA;protected:&#xA;    std::unique_ptr&amp;lt;sigvec&amp;gt; m_e_weights;&#xA;    std::unique_ptr&amp;lt;sigvec&amp;gt; m_i_weights;&#xA;public:&#xA;    exciting_neuron(sigvec &amp;amp;&amp;amp;e_weights, sigvec &amp;amp;&amp;amp;i_weights)&#xA;        : m_e_weights{std::make_unique&amp;lt;sigvec&amp;gt;(std::move(e_weights))},&#xA;          m_i_weights{std::make_unique&amp;lt;sigvec&amp;gt;(std::move(i_weights))}&#xA;    {&#xA;&#xA;    }&#xA;    ~exciting_neuron() {}&#xA;    signal call(const sigvec&amp;amp; e_inputs, const sigvec&amp;amp; i_inputs) override&#xA;    {&#xA;        auto ew = *m_e_weights;&#xA;        auto iw = *m_i_weights;&#xA;        auto ei = e_inputs;&#xA;        auto ii = i_inputs;&#xA;&#xA;        if (ew.size() != iw.size() || iw.size() != ii.size())&#xA;        {&#xA;            throw std::invalid_argument(&quot;Wrong input size.&quot;);&#xA;        }&#xA;&#xA;        signal e_sum = 0, i_sum = 0;&#xA;&#xA;        for (unsigned int j = 0; j &amp;lt; ew.size(); ++j)&#xA;        {&#xA;            e_sum += ew[j] * ei[j];&#xA;        }&#xA;&#xA;        for (unsigned int j = 0; j &amp;lt; iw.size(); ++j)&#xA;        {&#xA;            i_sum += iw[j] * ii[j];&#xA;        }&#xA;&#xA;        signal n = (1 + e_sum) / (1 + i_sum) - 1;&#xA;        return n &amp;gt;= 0 ? n : 0;&#xA;    }&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;e_inputs&lt;/code&gt; are &lt;code&gt;a&lt;/code&gt;-factors (exciting) and &lt;code&gt;i_inputs&lt;/code&gt; are &lt;code&gt;b&lt;/code&gt;-factors (inhibitory).&lt;br&gt;&#xA;But how should the ANN structure look? I mean I can't get where should I put an exciting neuron and an inhibitory... Has it some samples or rules of the structure, I couldn't find?&lt;/p&gt;&#xA;" OwnerUserId="9684" LastEditorUserId="9684" LastEditDate="2017-12-08T03:24:42.393" LastActivityDate="2017-12-08T03:24:42.393" Title="The ANN is based on cognitrons" Tags="&lt;artificial-neuron&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4682" PostTypeId="1" CreationDate="2017-12-08T07:16:45.903" Score="7" ViewCount="619" Body="&lt;p&gt;I have a Remaining Useful Life (RUL) prediction problem that I want to solve. When I added two or more features as inputs to my ANN, the accuracy of my ANN has been decreased. More precisely, I've added features like RMS or KURTOSIS (or both). I was expecting the system to improve, but it is getting worse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why might this be happening? What are the potential reasons for this degradation in performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that when we added more nodes in layers (like hidden layers), overfitting can happen. Would that be related to my problem: using more than two features?&lt;/p&gt;&#xA;" OwnerUserId="11465" LastEditorUserId="2444" LastEditDate="2019-10-22T23:29:14.257" LastActivityDate="2020-04-20T01:02:05.320" Title="How come that the addition of features can decrease the performance of a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;prediction&gt;&lt;overfitting&gt;&lt;feature-selection&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4683" PostTypeId="1" CreationDate="2017-12-08T14:48:36.857" Score="11" ViewCount="7410" Body="&lt;p&gt;What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?&lt;/p&gt;&#xA;" OwnerUserId="11256" LastEditorUserId="2444" LastEditDate="2019-05-13T21:29:16.820" LastActivityDate="2019-10-29T12:19:52.823" Title="What is the fundamental difference between CNN and RNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;comparison&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4684" PostTypeId="2" ParentId="4669" CreationDate="2017-12-08T17:17:56.303" Score="1" Body="&lt;p&gt;In adittion to @Neil Slater’s answer, you can also “increase” your data set by using data augmentation techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this basically means for images is to transform the ones you have into new ones without changing the result if the classification. For example, if you move everything 20 pixels to the right, it’s likely that your object will still be there, and from 5 examples now you’ve got 6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is usually done for regularizarion: it prevents the network from associating particular positions / sizes / colors to the end result. This is something that big pre-trained NNs already account for, but with your small data set, it’s very likely you’ll also benefit from it. &lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2017-12-08T17:17:56.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4685" PostTypeId="2" ParentId="4683" CreationDate="2017-12-08T17:18:19.240" Score="7" Body="&lt;p&gt;Basically, a CNN saves a set of weights and applies them spatially. For example, in a layer, I could have 32 sets of weights (also called feature maps). Each set of weights is a 3x3 block, meaning I have 3x3x32=288 weights for that layer. If you gave me an input image, for each 3x3 map, I slide it across all the pixels in the image, multiplying the regions together. I repeat this for all 32 feature maps, and pass the outputs on. So, I am learning a few weights that I can apply at a lot of locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an RNN, it is a set of weights applied temporally (through time). An input comes in, and is multiplied by the weight. The networks saves an internal state and puts out some sort of output. Then, the next piece of data comes in, and is multiplied by the weight. However, the internal state that was created from the last piece of data also comes in and is multiplied by a different weight. Those are added and the output comes from an activation applied to the sum, times another weight. The internal state is updated, and the process repeats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CNN's work really well for computer vision. At the low levels, you often want to find things like vertical and horizontal lines. Those kinds of things are going to be all over the images, so it makes sense to have weights that you can apply anywhere in the images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNN's are really good for natural language processing. You can imagine that the next word in a sentence will be highly influenced by the ones that came before it, so it makes sense to carry that internal state forward and have a small set of weights that can apply to any input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are many more applications. In addition, CNN's have performed well on NLP tasks. There are also more advanced versions of RNN's called LSTM's that you could check out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an explanation of CNN's, go to the &lt;a href=&quot;http://cs231n.stanford.edu/syllabus.html&quot; rel=&quot;noreferrer&quot;&gt;Stanford CS231n course&lt;/a&gt;. Especially check out lecture 5. There are full class videos on YouTube.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an explanation of RNN's, go &lt;a href=&quot;http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2017-12-08T17:18:19.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4687" PostTypeId="2" ParentId="4683" CreationDate="2017-12-08T18:54:08.843" Score="0" Body="&lt;p&gt;On a basic level, an RNN is a neural network whose next state depends on its past state(s), while a CNN is a neural network that does dimensionality reduction (make large data smaller while preserving information) via convolution. &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_(image_processing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;See this&lt;/a&gt; for more info on convolutions &lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-08T18:54:08.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4688" PostTypeId="5" CreationDate="2017-12-08T19:05:51.820" Score="0" Body="&lt;p&gt;MIT Link: &lt;a href=&quot;http://places.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://places.csail.mit.edu/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-08T19:05:51.820" LastActivityDate="2017-12-08T19:05:51.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4689" PostTypeId="4" CreationDate="2017-12-08T19:05:51.820" Score="0" Body="Scene recognition is one of the hallmark tasks of computer vision, allowing defining a context for object recognition. [MIT]" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-08T19:05:51.820" LastActivityDate="2017-12-08T19:05:51.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4690" PostTypeId="5" CreationDate="2017-12-08T19:17:45.117" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S2212827114001140&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S2212827114001140&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/document/7448025/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ieeexplore.ieee.org/document/7448025/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0898122110002555&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0898122110002555&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/p013496.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dtic.mil/dtic/tr/fulltext/u2/p013496.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/ba50/1d2a3a899f3c07bd3ca9d894a3b7f0fe6d71.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdfs.semanticscholar.org/ba50/1d2a3a899f3c07bd3ca9d894a3b7f0fe6d71.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-08T19:17:45.117" LastActivityDate="2017-12-08T19:17:45.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4691" PostTypeId="4" CreationDate="2017-12-08T19:17:45.117" Score="0" Body="Remaining Useful Life.  &quot;Utilised to predict life-span of components (of a service system) with the purpose of minimising catastrophic failure events&quot; " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-08T19:17:45.117" LastActivityDate="2017-12-08T19:17:45.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4692" PostTypeId="1" AcceptedAnswerId="4693" CreationDate="2017-12-09T01:27:18.407" Score="4" ViewCount="452" Body="&lt;p&gt;I have a 10GB file of a time series 1D signal. I want to find some patterns within this signal, I know CNN's are great for this but the problem is I don't have any training data.&lt;/p&gt;&#xA;&lt;p&gt;Now, I could, of course, spend an entire week slowly making 100 versions of a certain pattern to train the CNN with. But maybe there is some other way?&lt;/p&gt;&#xA;&lt;p&gt;Maybe there is a way for the neural network to work out patterns on its own and simply categorize them? Like this is pattern A, this is pattern B.&lt;/p&gt;&#xA;&lt;p&gt;My ultimate goal is to look at any size data and find the occurrences of patterns within the data.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any idea how this problem could be solved? I am just starting with machine learning, so I am slowly learning what's possible in this field.&lt;/p&gt;&#xA;" OwnerUserId="11476" LastEditorUserId="2444" LastEditDate="2020-12-02T09:09:21.393" LastActivityDate="2020-12-02T09:09:21.393" Title="Is there a way to perform pattern recognition without a labeled training set?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;pattern-recognition&gt;&lt;signal-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4693" PostTypeId="2" ParentId="4692" CreationDate="2017-12-09T01:42:04.313" Score="2" Body="&lt;p&gt;You should look into &lt;strong&gt;unsupervised learning&lt;/strong&gt;, which is machine learning without a human-labeled training set.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2020-12-02T09:06:47.340" LastActivityDate="2020-12-02T09:06:47.340" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4694" PostTypeId="1" CreationDate="2017-12-09T03:04:51.340" Score="2" ViewCount="96" Body="&lt;p&gt;Did &lt;a href=&quot;https://plato.stanford.edu/entries/turing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alan Turing&lt;/a&gt; expect the AIs &lt;a href=&quot;http://www.dictionary.com/browse/double-blind&quot; rel=&quot;nofollow noreferrer&quot;&gt;to be aware&lt;/a&gt; they were being &lt;a href=&quot;https://plato.stanford.edu/entries/turing-test/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing tested&lt;/a&gt; while &lt;a href=&quot;https://www.abelard.org/turpap/turpap.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;the game&lt;/a&gt; was being played?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's slightly harder &lt;a href=&quot;https://en.wikipedia.org/wiki/Duck_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;to look like a duck&lt;/a&gt; while &lt;em&gt;pretending to be&lt;/em&gt; a duck than when you &lt;em&gt;believe you are&lt;/em&gt; a duck. It is a step further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aside from all other criticisms of original article, did Alan Turing have the time and motivation to address the point in any later writing?&lt;/p&gt;&#xA;" OwnerUserId="2023" LastEditorUserId="75" LastEditDate="2017-12-09T22:30:54.887" LastActivityDate="2018-03-23T20:48:37.987" Title="Is the Turing Test a blinded experiment?" Tags="&lt;agi&gt;&lt;history&gt;&lt;turing-test&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4695" PostTypeId="1" CreationDate="2017-12-09T03:31:09.143" Score="2" ViewCount="47" Body="&lt;p&gt;Two practical questions regarding the use of autoencoders. For the size of the NN, let's say I have 50 inputs, so layer sizes go like 50-25-12-6-2-6-12-25-50&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;how much data I generally need to train it and how does it depend on the size of net?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'd like to weigh samples differently - e.g. there is a time component to the problem the last samples are most recent, so should weigh more. Is there an easy way to incorporate it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11478" LastEditorUserId="9647" LastEditDate="2017-12-09T14:04:02.413" LastActivityDate="2018-01-12T04:40:45.207" Title="Training data for autoencoders" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4696" PostTypeId="1" AcceptedAnswerId="4747" CreationDate="2017-12-09T10:20:49.837" Score="10" ViewCount="1278" Body="&lt;p&gt;is it possible to give a &lt;strong&gt;rule of thumb estimate about the size of neural networks that are trainable on common consumer grade GPUs&lt;/strong&gt;?&#xA;For example:&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/abs/1707.02286&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emergence of Locomotion (Reinforcement)&lt;/a&gt; paper trains a network using tanh activation of the neurons. They have a 3 layer NN with 300,200,100 units for the &lt;em&gt;Planar Walker&lt;/em&gt;. But they don’t report the hardware and time ...&lt;/p&gt;&#xA;&lt;p&gt;But could a rule of thumb be developed? Also just based on current empirical results, so for example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; Units using sigmoid activation can run &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; learning iterations per hour on a 1060.&lt;/p&gt;&#xA;&lt;p&gt;Or using activation function &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; causes a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times decrease in performance.&lt;/p&gt;&#xA;&lt;p&gt;If a student/researcher/curious mind is going to buy a GPU for playing around with these networks, how do you decide what you get? A 1060 is apparently the entry level budget option, but how can you evaluate if it is not smarter to just get a crappy netbook instead of building a high power desktop and spend the saved $ on on-demand cloud infrastructure.&lt;/p&gt;&#xA;&lt;p&gt;Motivation for the question: I just purchased a 1060 and (clever, to ask the question afterwards huh) wonder if I should have just kept the $ and made a Google Cloud account. And if I can run my master thesis simulation on the GPU.&lt;/p&gt;&#xA;" OwnerUserId="11429" LastEditorUserId="40671" LastEditDate="2021-04-25T16:05:20.680" LastActivityDate="2021-04-25T16:05:20.680" Title="What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;performance&gt;&lt;hardware-evaluation&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4697" PostTypeId="1" CreationDate="2017-12-09T11:18:02.577" Score="3" ViewCount="138" Body="&lt;p&gt;In the current rush of artificial intelligence research, fueled by NN, independent of the paper I choose, the NN are always &lt;strong&gt;trained by themselves&lt;/strong&gt;. Sure, there are architectures that combine CNN and RNN or LSTMs in a way that can help to solve multiple problems interacting (like labeling images with human readable text snippets), but independent of this, they always learn by themselves. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Supervised networks just get a bunch of examples to learn from.&lt;/li&gt;&#xA;&lt;li&gt;Reinforcement algorithms run around virtual spaces falling over hundreds of times before learning to walk properly.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This might sound silly but &lt;em&gt;no one helps them and no one plays with them&lt;/em&gt;. Children, researchers, puppies, dolphins, ... all intelligent beings we are aware of interact with each other. There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Learning#Types&quot; rel=&quot;nofollow noreferrer&quot;&gt;many forms of learning&lt;/a&gt;, one very important one being &lt;a href=&quot;https://en.wikipedia.org/wiki/Social_learning_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;social learning&lt;/a&gt; which includes observational learning. We imitate, copy and learn from others all the time. There is also &lt;a href=&quot;https://www.nytimes.com/2017/04/18/books/review/knowledge-illusion-steven-sloman-philip-fernbach.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a new idea of looking at knowledge&lt;/a&gt; that suggests that we all aren't as independent as our culture might have taught us to think we are. That knowledge really lies in the connections between individuals instead of inside of each agent itself, just like the connections of neurons are what makes the brain work, not the neurons themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Couldn't such interaction between learning agents be enormously valuable? If you throw 5 agents in a room (NN a Knowledge Based Agent a simple pre-programmed one etc) and let them learn from each other through observation and inspiration, what effects could one expect? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not talking about ensemble learning as this just throws several hypotheses together and takes the average over all results. I am talking about a complex way of agents interacting with each other during learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any research on this idea? If so, what were the results? This is the only one I found:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1605.06676&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Communicate with Deep Multi-Agent Reinforcement Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11429" LastEditorUserId="2444" LastEditDate="2019-06-15T18:42:15.190" LastActivityDate="2019-06-15T18:42:15.190" Title="Why are neural networks always trained &quot;by themselves&quot;?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4698" PostTypeId="1" CreationDate="2017-12-09T11:21:31.313" Score="4" ViewCount="186" Body="&lt;p&gt;How would you solve the problem of identifying certain customer in a grocery store? Suppose our client is already signed-up on our website with an unique ID given to him. To come in to the store, firstly he must place a phone with a QR code in front of the reader, so that our server is informed on a specific client entering the shop. Then after successful authentication he heads towards the shelf with goods and pick some item. Identifying product name turned out not to be a challenge, as opposed to person. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my perspective, the solution is to make a several photo of face from a different angle while person coming in. Quickly train CNN and feed it with face images when customer picks goods in order to choose the corresponding one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what are your thoughts on this issue? What approach would you take to work it out?&lt;/p&gt;&#xA;" OwnerUserId="11482" LastEditorUserId="1671" LastEditDate="2017-12-09T23:17:17.467" LastActivityDate="2018-03-09T22:14:09.190" Title="How to identify the face of a certain customer in a grocery store?" Tags="&lt;image-recognition&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4699" PostTypeId="2" ParentId="4697" CreationDate="2017-12-09T13:06:57.000" Score="5" Body="&lt;p&gt;The way children learn is in many ways &lt;em&gt;supervised&lt;/em&gt;. It is true that certain abilities are there genetically (visual system, object recognition, to large extent voice recognition), but a lot of human experience is gained as a result of &lt;em&gt;response&lt;/em&gt;, either from the environment or from the mentor (parent, teacher, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Social interaction is possible only when a human already learned a lot (things we consider trivial, but in fact pretty complex), in this sense this is equivalent to AlphaGo learning to play Go by playing with a copy network. NNs that can only perform object or voice recognition can't interact, they are too primitive for this. And a nearly all of todays AI is on this stage right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is true that NN learning is much less efficient than human learning and current models are far from AGI. I think RL is the closest area that studies real multi-agent environments.&lt;/p&gt;&#xA;" OwnerUserId="9687" LastActivityDate="2017-12-09T13:06:57.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4700" PostTypeId="1" CreationDate="2017-12-09T21:20:16.713" Score="3" ViewCount="135" Body="&lt;p&gt;At this moment, I am able to use NN to identify an object, such as a human, when given a frame from the camera. Once locate the object, then I can feed the human object image to either NN that's designed to classify male or female.&lt;/p&gt;&#xA;&lt;p&gt;Let's say I get 1 frame per second from the camera and perform detection, the objective is to track the number of male and female walk pass the camera within the given hours.&lt;/p&gt;&#xA;&lt;p&gt;My question is, the same person in multiple frames will be overcounted. I couldn't wrap my head around how can I train a NN to understand that this is the same person without dive into facial recognition? I'm sure there is some tracking technique that I just don't know.&lt;/p&gt;&#xA;&lt;p&gt;One little constraint, if the person left the camera frame and come back into it later, it is fine to treat it as two people.&lt;/p&gt;&#xA;" OwnerUserId="11493" LastEditorUserId="2444" LastEditDate="2020-09-17T16:02:12.230" LastActivityDate="2020-09-17T16:02:12.230" Title="How to keep track of the same person detected in different frames of a camera?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4701" PostTypeId="5" CreationDate="2017-12-09T23:19:10.577" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Facial_recognition_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Facial_recognition_system&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-09T23:19:10.577" LastActivityDate="2017-12-09T23:19:10.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4702" PostTypeId="4" CreationDate="2017-12-09T23:19:10.577" Score="0" Body="A facial recognition system is a computer application capable of identifying or verifying a person from a digital image or a video frame from a video source. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-09T23:19:10.577" LastActivityDate="2017-12-09T23:19:10.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4703" PostTypeId="1" CreationDate="2017-12-10T00:35:09.153" Score="2" ViewCount="217" Body="&lt;p&gt;I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e. 20% of moves are random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all configurations, I initiate the weights to random values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've found is that in all configurations, Player 1 will win the majority of games, or Player 2 will. There's no 50/50 split.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this expected or normal? I am training on the Connect Four game.&lt;/p&gt;&#xA;" OwnerUserId="2819" LastEditorUserId="2444" LastEditDate="2019-06-26T00:14:02.440" LastActivityDate="2019-06-26T07:00:56.820" Title="Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4704" PostTypeId="2" ParentId="4678" CreationDate="2017-12-10T06:05:09.963" Score="2" Body="&lt;p&gt;Here is some general info on how NNs work in relation to this specific problem. Hopefully it will provide some insight:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To identify target object, you can just train an NN to perform classification of images. Now, you will run into potential problems where this image can be located anywhere in given frame of input from the camera.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say; you train a human NN that's able to act on 60px x 120px (so you will have 10800 input in your NN for the input layer). Under the assumption you can train the NN to identify whether or not the target object is showing in the image provided, you can use this trained NN to locate the object on the input frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the input frame from the camera is 640px x 480px. You will run into several potential problems&#xA;1) The object that's closer to the camera will be larger while the object that's further away from the camera will be smaller&#xA;2) The object may be located anywhere in the frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To overcome this problem with your already trained NN, I would start with a different mask size, for example, the first one I will probably have my first mask convolute all pixels with 240px x 480px. The first step is, scale it to 60px x 120px (which you can tell it is pretty much 4x) and capture the subset of pixels from pixels located in the area of (0,0) and (240, 480) and run it through the NN, If it yields true, then I will typically draw a box around the region to indicate the target object has been found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, you shift over to 1-3 pixel and rerun it. For example, if we choose to move by 3px, then the region we are testing will be (3, 0) and (243, 480).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you finish this scale, you will want to choose a smaller mask, maybe something like 200px x 400px and do the same thing. When finished, go even smaller until you believe there's no point to search for an even smaller region because it is not going to be sufficient resolution for NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's just my thought; maybe there's an even more efficient algorithm. Ultimately, I am sure there's a lot more can be optimized than what I've mentioned above!&lt;/p&gt;&#xA;" OwnerUserId="11493" LastEditorUserId="1671" LastEditDate="2017-12-10T21:04:32.487" LastActivityDate="2017-12-10T21:04:32.487" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4705" PostTypeId="1" AcceptedAnswerId="4897" CreationDate="2017-12-10T08:34:39.257" Score="2" ViewCount="1489" Body="&lt;p&gt;I have deep interest in AI and want to start learning how to implement current method. I know about java and C++; are these languages sufficient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd appreciate suggestions regarding free online courses/websites that utilize Java/C++. If I lag some knowledge which is required before starting AI then please let me know.&lt;/p&gt;&#xA;" OwnerUserId="11498" LastEditorUserId="1671" LastEditDate="2018-03-22T20:32:03.207" LastActivityDate="2018-03-22T20:32:03.207" Title="How and where can I start learning AI if I have considerable knowledge of Java &amp; C++?" Tags="&lt;getting-started&gt;&lt;java&gt;&lt;c++&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="13" ClosedDate="2021-01-21T22:35:06.897" ContentLicense="CC BY-SA 3.0" />
  <row Id="4706" PostTypeId="1" AcceptedAnswerId="4710" CreationDate="2017-12-10T19:29:25.527" Score="8" ViewCount="411" Body="&lt;p&gt;There seem to be so many sub-fields, so I'm interested in getting a better understanding of the approaches.&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for information on a single framework per answer, in order to allow for granularity without the overall answer getting too long. For instance; Deep Learning Neural Networks would be a single answer.&lt;/p&gt;&#xA;" OwnerUserId="5356" LastEditorUserId="2444" LastEditDate="2020-12-17T01:13:03.390" LastActivityDate="2020-12-17T01:13:03.390" Title="What are the different approaches used in Machine Learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;&lt;taxonomy&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4707" PostTypeId="2" ParentId="4655" CreationDate="2017-12-10T22:18:39.937" Score="8" Body="&lt;p&gt;Traditionally, due to the way the network is structured, each input has a set of weights, that are connected to more inputs. If the inputs switch, the output will too.&lt;/p&gt;&#xA;&lt;h3&gt;Approach 1&lt;/h3&gt;&#xA;&lt;p&gt;However, you can build a network that approaches this behaviour. In your training set, use batch learning and for each training sample, give all possible permutations to the network such that it learns to be permutation invariant. This will never be exactly invariant, it just might be close.&lt;/p&gt;&#xA;&lt;h3&gt;Approach 2&lt;/h3&gt;&#xA;&lt;p&gt;Another way to do this is to have the weights replicated for all inputs. For example, let's assume you have 3 inputs (&lt;span class=&quot;math-container&quot;&gt;$i_0, i_1, i_2$&lt;/span&gt;), and the next/first hidden layer has 2 nodes (&lt;span class=&quot;math-container&quot;&gt;$h_{10}, h_{11}$&lt;/span&gt;) and activation function &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;. Assuming a fully connected layer, you have 2 weights &lt;span class=&quot;math-container&quot;&gt;$w_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;. The hidden layer's nodes &lt;span class=&quot;math-container&quot;&gt;$h_{10}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{11}$&lt;/span&gt; are given, respectively, by&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_{10} = \phi(i_0 w_0 + i_1 w_0 + i_2 w_0)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_{11} = \phi(i_0 w_1 + i_1 w_1 + i_2 w_1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thus giving you a hidden layer whose values are permutation invariant from the input. From now on, you can learn and build the rest of the network as you see fit. This is an approach derived from convolutional layers.&lt;/p&gt;&#xA;&lt;h3&gt;Approach 3&lt;/h3&gt;&#xA;&lt;p&gt;Finally, you can use a dimension reduction to achieve this. I've published this in &lt;a href=&quot;https://www.researchgate.net/publication/341498982_Exploring_communication_protocols_and_centralized_critics_in_multi-agent_deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exploring Communication Protocols and Centralized Critics in Multi-Agent Deep Learning&lt;/a&gt;, on Integrated Computer-Aided Engineering. This approach uses convolutional architectures to efficiently achieve permutation invariance.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="2444" LastEditDate="2020-11-06T02:29:39.520" LastActivityDate="2020-11-06T02:29:39.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4709" PostTypeId="1" AcceptedAnswerId="4717" CreationDate="2017-12-11T03:03:06.380" Score="1" ViewCount="145" Body="&lt;p&gt;I heard that your ML model's quality depends directly on the quality and the quantity of data you use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was thinking that can question answers be used as data to train an algorithm which can solve any high school science problems? Because we do have a gazillion number of high school books with millions of Question-Answers which are both high in quality as well as quantity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S: I don't have any in-depth knowledge in any of the AI fields, so please answer accordingly!&lt;/p&gt;&#xA;" OwnerUserId="11506" LastActivityDate="2018-01-07T15:41:33.890" Title="Can we make an algorithm which can solve any high school (science) problem using ML and DL?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4710" PostTypeId="2" ParentId="4706" CreationDate="2017-12-11T04:28:55.030" Score="13" Body="&lt;p&gt;Things in &lt;em&gt;italics&lt;/em&gt; should give you enough googleable terms to start a deeper dive :P.&lt;/p&gt;&#xA;&lt;p&gt;There are 3 main branches of statistical ML.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Supervised Learning&lt;/strong&gt;&#xA;This approach is taken when a problem can be phrased as associating some &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; with some &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. For example, &lt;em&gt;classifying&lt;/em&gt; a picture of a cat (&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;) with the label “Cat” (&lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;). Training in supervised learning usually means presenting some &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, having an agent predict the label, comparing the prediction with the answer to get an error metric, and finally using the error to update the agent to make better predictions on future &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;s. &lt;em&gt;MNIST&lt;/em&gt; is a great example of a classification domain.&lt;/p&gt;&#xA;&lt;p&gt;1.1 Supervised learning also works with &lt;em&gt;regression&lt;/em&gt; when we want to learn an associated number instead of a label, such as the risk of heart disease given years of smoking and weight.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Unsupervised Learning&lt;/strong&gt;&#xA;This type of learning is used when we want to draw inferences from a dataset without labeled responses. It can be used to find hidden patterns or grouping in data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Reinforcement Learning&lt;/strong&gt;&#xA;The &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;s and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;s in supervised learning are &lt;em&gt;i.i.d.&lt;/em&gt;, meaning that they don’t relate to the previous &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; that a model has previously observed. However, there are many problems where the most recent &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt;), is very important to figuring out what &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; to predict (&lt;span class=&quot;math-container&quot;&gt;$Y_{t+1}$&lt;/span&gt;) for the next &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$X_{t+1}$&lt;/span&gt;). The field of reinforcement learning (RL) builds on this notion by framing observations as data that is generated by an agent interacting with an environment. Specifically, the field of RL poses the problem of finding a behaviour (a policy) to select actions such that an agent using the policy can maximize a special signal called the &lt;em&gt;reward&lt;/em&gt; from the environment. The &lt;em&gt;agent&lt;/em&gt; starts in a &lt;em&gt;state&lt;/em&gt; of the world (&lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;) at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and takes some &lt;em&gt;action&lt;/em&gt; (&lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;) which puts it in a new state (&lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;) and gives it a reward (&lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Neural networks are a technique for finding features (some kinda notable differences) in data. They can be used in all of the above learning strategies but are not necessary for them although their application has had some very powerful results recently. Deep ___ learning is simply the application of a multilayered neural network on one of the above learning methods.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-12-16T22:02:26.573" LastActivityDate="2020-12-16T22:02:26.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4711" PostTypeId="1" AcceptedAnswerId="4729" CreationDate="2017-12-11T04:44:30.330" Score="4" ViewCount="1054" Body="&lt;p&gt;Trying to understand the VGG architecture and I have these following questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I understand the general understanding of increasing filter size is because we are using max pooling and so its image size gets reduced. So in order to keep information gain, we increase filter size. But the last few layers in the VGG architecture, the filter size remained same when vgg was max pooling from 14x14 to 7x7 image size, the filter size remained same at 512x512. Why wasn’t there the need to increase filter size there?&lt;/li&gt;&#xA;&lt;li&gt;Also few consecutive layers, in the end, was constructed with both same filter and image size, those layers were built just to increase accuracy? (experimentation?)&lt;/li&gt;&#xA;&lt;li&gt;And I couldn’t wrap around that visualization at final filters have the entire face as the feature as I understood through convolution visualizing (Matt Zieler video explanation). But max pooling causes us to see only a subset part of the image right? When filter size is 512x512 (face as the filter/feature) the image size as 7x7, so how does entire face as a filter will work on images when we are moving over small subset of the image pixels?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rVB4S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rVB4S.png&quot; alt=&quot;VGG architecture summary&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/yfw43.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yfw43.png&quot; alt=&quot;VGG architecture final layers summary&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11508" LastEditorUserId="2444" LastEditDate="2020-06-14T22:19:08.340" LastActivityDate="2020-12-03T16:24:47.323" Title="Trying to understand VGG convolution neural networks architecture" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;vgg&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4712" PostTypeId="2" ParentId="35" CreationDate="2017-12-11T08:43:59.507" Score="2" Body="&lt;p&gt;&lt;strong&gt;Artificial Intelligence&lt;/strong&gt; (AI) and Machine Learning (ML) are two very hot buzzwords right now, and often seem to be used interchangeably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence is the broader concept of machines being able to carry out tasks in a way that we would consider “smart&quot; and &lt;strong&gt;Machine Learning&lt;/strong&gt; is a current application of AI based around the idea that we should really just be able to give machines access to data and let them learn for themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a further information about &lt;a href=&quot;https://www.cubix.co/blog/machine-learning-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning and Artificial Intelligence.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11514" LastActivityDate="2017-12-11T08:43:59.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4713" PostTypeId="1" CreationDate="2017-12-11T15:53:38.130" Score="1" ViewCount="79" Body="&lt;p&gt;I'm solving this problem similar to consumer-producer of materials (i.e. sand). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the graph of the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M3aRs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M3aRs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;Req&lt;/code&gt; (&lt;code&gt;E0&lt;/code&gt; and &lt;code&gt;E1&lt;/code&gt;) are units that require material, and &lt;code&gt;Prod&lt;/code&gt; (&lt;code&gt;C0&lt;/code&gt; and &lt;code&gt;C1&lt;/code&gt;) produce material. I'm looking for the first random solution or similar that meets the constraints i.e &lt;code&gt;E0&lt;/code&gt; Require &lt;code&gt;10&lt;/code&gt;, shared from &lt;code&gt;C0&lt;/code&gt; or/and &lt;code&gt;C1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be a valid approach? &#xA;I just think in Genetic Algorithms.&lt;/p&gt;&#xA;" OwnerUserId="11525" LastActivityDate="2017-12-11T15:53:38.130" Title="Which algorithm would you use to solve a multiple producer-consumer problem with constraints?" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4714" PostTypeId="1" AcceptedAnswerId="22314" CreationDate="2017-12-11T16:28:06.863" Score="2" ViewCount="507" Body="&lt;p&gt;I am planning to build an app which will count the number of sqauts from videos. Assuming that the user and camera do not move, are there ways I can count the number of squats? Do such models to understand human activity and pose exist?&lt;/p&gt;&#xA;" OwnerUserId="11387" LastActivityDate="2020-07-02T14:28:51.283" Title="Is it possible to count the number of squats with Computer Vision techniques?" Tags="&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4716" PostTypeId="2" ParentId="4672" CreationDate="2017-12-11T19:11:46.573" Score="0" Body="&lt;p&gt;I feel your question as presented, &quot;Can an AI know when it is its turn?&quot;, is a bit naive. Performing this task is just a matter of asking &quot;Is it my turn?&quot; repeatedly. This can be implemented as simply as seeing if the turn clock is running for itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the more general question of whether an AI can learn and then &quot;properly&quot; play chess the answer is: Theoretically, yes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We currently have the necessary computing power for AI to beat humans at Chess (and Go) in real time.&lt;/li&gt;&#xA;&lt;li&gt;We currently have AI systems that can discover and test system rules.&lt;/li&gt;&#xA;&lt;li&gt;We currently have AI that are able to discover the mappings between control manipulation and their effect in platformer video games. (Though they do not appear to be able to discover the goal of a game on there own. They still need a human to define appropriate fitness criteria.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Based on this, it seems possible for today's AI to be able to extract rules from a collection of previously played chess games. It would also seem possible to extract the rules by brute force from an oracle (referee) by setting up boards and asking that oracle if particular moves are valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You still need to tell it what winning is and why it would &quot;want&quot; to (i.e.: fitness criteria).&lt;/p&gt;&#xA;" OwnerUserId="11529" LastEditorUserId="11529" LastEditDate="2017-12-11T19:24:13.480" LastActivityDate="2017-12-11T19:24:13.480" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4717" PostTypeId="2" ParentId="4709" CreationDate="2017-12-11T23:31:17.227" Score="0" Body="&lt;p&gt;The short answer, I guess, is yes. In theory, you could have a massive, insane network that could take as input any high school science textbook question and output an answer. But think about what that entails:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take in a question, in any of the various languages that high school science textbooks can be written in. Get all the relevant information out of that. This includes questions like &quot;List four noble gases&quot; as well as ones about applying Kirchoff's laws to a circuit as well as explaining how the ribosome changes its shape to produce proteins. There is soooooooo much information that is contained in just the wording of those questions and so many topics. Then, let's say you have converted the question into a representation that can be &quot;understood&quot; by the algorithm. I want to emphasize that ML models do not &quot;know&quot;, &quot;understand&quot;, &quot;think&quot;, or any other of the words we use to personify them. They are basically just a mathematical function of some level of complexity (usually very complex). That being said, now the algorithm needs to relate this tractable representation of the question to the correct answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since everything before this created some representation of the question, you could view that as a function mapping the input to this representation. Then, everything between intermediate representation and output answer is another function. So, your model needs to learn a single function that is the sum of all human high school level science understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These algorithms you want to build would end up being staggeringly complex. In addition, as you add complexity to an ML model, you need significantly more data to train it. How will you take every high school science book in the world and convert it into digital pairs of questions and answers? How will you handle that some of the answers are incorrect? What is the benefit of this system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The long answer, I think, is no. It makes no sense to do this. Instead of learning this complex mapping, just build some solvers with equations built in that the user inputs some numbers to and selects the equations. It will take so much less time to build and deploy and I guarantee work better. There are people building ML systems that are trying to get the SAT entirely right &lt;a href=&quot;http://aiindex.org/2017-report.pdf?utm_campaign=Revue%20newsletter&amp;amp;utm_medium=Newsletter&amp;amp;utm_source=Deep%20Learning%20Weekly&quot; rel=&quot;nofollow noreferrer&quot;&gt;(see page 33 of this report)&lt;/a&gt;. First, I don't think this is what you're asking and second, it still makes more sense, in my mind, to build some solvers. Hopefully, this all makes sense and, if anyone sees things I misstated, let me know.&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2017-12-11T23:31:17.227" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4718" PostTypeId="2" ParentId="4714" CreationDate="2017-12-12T00:13:49.130" Score="2" Body="&lt;p&gt;Yes, this is a rather straightforward problem that should be able to be solved using openCV or ANNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out this paper to get you started: &lt;a href=&quot;https://arxiv.org/pdf/1701.08936.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1701.08936.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-12-12T00:13:49.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4719" PostTypeId="2" ParentId="4705" CreationDate="2017-12-12T00:21:03.137" Score="7" Body="&lt;p&gt;If you are coming from Java, it would make a lot of sense to play around with deeplearning4J at first. From there I would start learning python as this is the primary language used today in ML. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lectures are a great way to get your feet wet in understanding and applying ML concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My favorites are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stanford ML(Andrew NG): &lt;a href=&quot;https://www.youtube.com/watch?v=UzxYlbK2c7E&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=UzxYlbK2c7E&lt;/a&gt; (covers classic algorithms in depth)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MIT AI(Patrick Winston): &lt;a href=&quot;https://www.youtube.com/watch?v=TjZBTDzGeGg&amp;amp;t=2s&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=TjZBTDzGeGg&amp;amp;t=2s&lt;/a&gt; (great starter course that covers basic concepts beautifully)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deepmind RL(David Silver): &lt;a href=&quot;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;t=7s&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;t=7s&lt;/a&gt; (Best RL course available) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that, start implementing these concepts. There are hundreds of tutorials on computer vision, natural language processing, and reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2017-12-12T00:21:03.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4722" PostTypeId="2" ParentId="4672" CreationDate="2017-12-12T08:56:32.353" Score="2" Body="&lt;p&gt;It's possible for an AI to learn chess without even knowing how to move the pieces. Google's AlphaZero didn't do that as their programmers coded the chess rules, but it's possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can learn the rules from human played chess games. Once the rules are known, we could use reinforcement learning to improve playing strength (and other board games). &lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2017-12-12T08:56:32.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4723" PostTypeId="2" ParentId="4219" CreationDate="2017-12-12T10:34:02.557" Score="3" Body="&lt;p&gt;Try UIMA and GATE, both of them are open source.&lt;/p&gt;&#xA;&lt;p&gt;Watson Content Analytics implements UIMA framework, according to this:&#xA;&lt;a href=&quot;https://www.ibm.com/support/knowledgecenter/en/SS5RWK_3.5.0/com.ibm.discovery.es.nav.doc/iiysaovcapipe.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open, scalable analytics pipeline | IBM Watson Content Analytics 3.5.0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;UIMA takes care of the management of NLP pipeline, but the intelligence is actually comes from ‘annotators’.&lt;/p&gt;&#xA;&lt;p&gt;You can rely on annotators from projects like GATE, Apache OpenNLP, Stanford CoreNLP, etc.&#xA;But since these projects are not part of UIMA, they use their own document/sentence representation, so a UIMA wrapper is needed for conversion.&#xA;GATE provided a detailed documentation for this. &lt;a href=&quot;https://gate.ac.uk/sale/tao/splitch22.html#chap:uima&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combining GATE and UIMA | GATE&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11535" LastEditorUserId="32410" LastEditDate="2021-02-25T09:48:18.847" LastActivityDate="2021-02-25T09:48:18.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4725" PostTypeId="1" AcceptedAnswerId="4727" CreationDate="2017-12-12T12:14:09.153" Score="9" ViewCount="1129" Body="&lt;p&gt;Currently, I'm doing a project that's about creating an AI to play the game &lt;a href=&quot;https://en.wikipedia.org/wiki/Gomoku&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gomoku&lt;/a&gt; (it's like tic tac toe, but played on a 15&lt;em&gt;15 board and requires 5 in a row to win). I have already successfully implemented a perfect tic tac toe AI using Q-learning and having game states/actions stored in a table, but, for a 15&lt;/em&gt;15 board, the possible game states become too large to implement this project.&lt;/p&gt;&#xA;&lt;p&gt;My question is, should I use neural networks or genetic algorithms for this problem? And more specifically, how should I implement this?&lt;/p&gt;&#xA;" OwnerUserId="11537" LastEditorUserId="2444" LastEditDate="2021-01-30T22:10:09.133" LastActivityDate="2021-01-30T22:13:35.237" Title="Should I use neural networks or genetic algorithms to solve Gomoku?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;genetic-algorithms&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4726" PostTypeId="2" ParentId="1479" CreationDate="2017-12-12T16:41:43.700" Score="19" Body="&lt;p&gt;Short answer is &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Model interpretability&lt;/em&gt; is a hyper-active and hyper-hot area of current research (think of holy grail, or something), which has been brought forward lately not least due to the (often tremendous) success of deep learning models in  various tasks; these models are currently only black boxes, and we naturally feel uncomfortable about it...&lt;/p&gt;&#xA;&lt;p&gt;Here are some general (and recent, as of Dec 2017) resources on the subject:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A recent (July 2017) article in Science provides a nice overview of the current status &amp;amp; research: &lt;a href=&quot;http://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;How AI detectives are cracking open the black box of deep learning&lt;/a&gt; (no in-text links, but googling names &amp;amp; terms will pay off)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;DARPA itself is currently running a program on &lt;a href=&quot;https://www.darpa.mil/program/explainable-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explainable Artificial Intelligence (XAI)&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There was a workshop in NIPS 2016 on &lt;a href=&quot;http://nuit-blanche.blogspot.gr/2016/12/nips2016-interpretable-machine-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning for Complex Systems&lt;/a&gt;, as well as an &lt;a href=&quot;http://people.csail.mit.edu/beenkim/icml_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICML 2017 tutorial on Interpretable Machine Learning&lt;/a&gt; by &lt;a href=&quot;https://beenkim.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Been Kim&lt;/a&gt; of Google Brain.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And on a more practical level (code etc):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The What-If tool by Google, a brand new (September 2018) feature of the open-source TensorBoard web application, which let users analyze an ML model without writing code (&lt;a href=&quot;https://pair-code.github.io/what-if-tool/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://ai.googleblog.com/2018/09/the-what-if-tool-code-free-probing-of.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Layer-wise Relevance Propagation (LRP) toolbox for neural networks (&lt;a href=&quot;http://www.jmlr.org/papers/v17/15-618.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;http://www.explain-ai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://github.com/sebastian-lapuschkin/lrp_toolbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://github.com/VigneshSrinivasan10/interprettensor&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF Slim wrapper&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;FairML: Auditing Black-Box Predictive Models, by Cloudera Fast Forward Labs (&lt;a href=&quot;http://blog.fastforwardlabs.com/2017/03/09/fairml-auditing-black-box-predictive-models.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1611.04967&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/adebayoj/fairml&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;LIME: Local Interpretable Model-agnostic Explanations (&lt;a href=&quot;https://arxiv.org/abs/1602.04938&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/marcotcr/lime&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, &lt;a href=&quot;https://cran.r-project.org/web/packages/lime/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R port&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A very recent (November 2017) paper by Geoff Hinton, &lt;a href=&quot;https://arxiv.org/abs/1711.09784&quot; rel=&quot;nofollow noreferrer&quot;&gt;Distilling a Neural Network Into a Soft Decision Tree&lt;/a&gt;, with an independent &lt;a href=&quot;https://github.com/kimhc6028/soft-decision-tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch implementation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SHAP: A Unified Approach to Interpreting Model Predictions (&lt;a href=&quot;https://arxiv.org/abs/1705.07874&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, authors' &lt;a href=&quot;https://github.com/slundberg/shap&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python code&lt;/a&gt;, &lt;a href=&quot;https://github.com/redichh/ShapleyR&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Interpretable Convolutional Neural Networks (&lt;a href=&quot;https://arxiv.org/abs/1710.00935&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, authors' &lt;a href=&quot;https://github.com/zqs1022/interpretableCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lucid, a collection of infrastructure and tools for research in neural network interpretability by Google (&lt;a href=&quot;https://github.com/tensorflow/lucid&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;; papers: &lt;a href=&quot;https://distill.pub/2017/feature-visualization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feature Visualization&lt;/a&gt;, &lt;a href=&quot;https://distill.pub/2018/building-blocks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Building Blocks of Interpretability&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transparecy-by-Design (TbD) networks (&lt;a href=&quot;https://arxiv.org/abs/1803.05268&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/davidmascharka/tbd-nets&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://hub.mybinder.org/user/davidmascharka-tbd-nets-nex18rln/notebooks/full-vqa-example.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;demo&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability (&lt;a href=&quot;https://arxiv.org/abs/1706.05806&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/google/svcca&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://ai.googleblog.com/2017/11/interpreting-deep-neural-networks-with.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TCAV: Testing with Concept Activation Vectors (&lt;a href=&quot;https://arxiv.org/abs/1711.11279&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICML 2018 paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/tensorflow/tcav/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization (&lt;a href=&quot;https://arxiv.org/abs/1610.02391&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, authors' &lt;a href=&quot;https://github.com/ramprs/grad-cam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Torch code&lt;/a&gt;, &lt;a href=&quot;https://github.com/Ankush96/grad-cam.tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow code&lt;/a&gt;, &lt;a href=&quot;https://github.com/meliketoy/gradcam.pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch code&lt;/a&gt;, Keras &lt;a href=&quot;http://nbviewer.jupyter.org/github/fchollet/deep-learning-with-python-notebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;example notebook&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Network Dissection: Quantifying Interpretability of Deep Visual Representations, by MIT CSAIL (&lt;a href=&quot;http://netdissect.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://github.com/CSAILVision/NetDissect&quot; rel=&quot;nofollow noreferrer&quot;&gt;Caffe code&lt;/a&gt;, &lt;a href=&quot;https://github.com/CSAILVision/NetDissect-Lite&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch port&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GAN Dissection: Visualizing and Understanding Generative Adversarial Networks, by MIT CSAIL (&lt;a href=&quot;https://gandissect.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, with links to paper &amp;amp; code)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explain to Fix: A Framework to Interpret and Correct DNN Object Detector Predictions (&lt;a href=&quot;https://arxiv.org/abs/1811.08011&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/gudovskiy/e2x&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;InterpretML by Microsoft (&lt;a href=&quot;https://github.com/interpretml/interpret&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; still in alpha)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Anchors: High-Precision Model-Agnostic Explanations (&lt;a href=&quot;https://arxiv.org/abs/1905.07697&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/marcotcr/anchor&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Diverse Counterfactual Explanations (DiCE) by Microsoft (&lt;a href=&quot;https://arxiv.org/abs/1905.07697&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/interpretml/DiCE&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://www.microsoft.com/en-us/research/blog/open-source-library-provides-explanation-for-machine-learning-through-diverse-counterfactuals/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Axiom-based Grad-CAM (XGrad-CAM): Towards Accurate Visualization and Explanation of CNNs, a refinement of the existing Grad-CAM method (&lt;a href=&quot;https://arxiv.org/abs/2008.02312&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/Fu0511/XGrad-CAM&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Lately, there has been a surge of interest to start building a more theoretical basis for deep learning neural nets. In this context, renowned statistician and compressive sensing pioneer David Donoho has very recently (fall 2017) started offering a course at Stanford, &lt;a href=&quot;https://stats385.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Theories of Deep Learning (STATS 385)&lt;/a&gt;, with almost all the material available online; it is highly recommended...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATES&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning&lt;/a&gt;, an online Gitbook by Christoph Molnar with &lt;a href=&quot;https://github.com/christophM/iml&quot; rel=&quot;nofollow noreferrer&quot;&gt;R code&lt;/a&gt; (although in mostly covers algorithms other than neural networks)&lt;/li&gt;&#xA;&lt;li&gt;A &lt;a href=&quot;https://twitter.com/ledell/status/995930308947140608&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twitter thread&lt;/a&gt;, linking to several interpretation tools available for R.&lt;/li&gt;&#xA;&lt;li&gt;A short (4 hrs) online course by Kaggle, &lt;a href=&quot;https://www.kaggle.com/learn/machine-learning-explainability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning Explainability&lt;/a&gt;, and the accompanying &lt;a href=&quot;https://towardsdatascience.com/why-model-explainability-is-the-next-data-science-superpower-b11b6102a5e0&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;A new ICML 2018 Tutorial, &lt;a href=&quot;http://unsupervised.cs.princeton.edu/deeplearningtutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Toward theoretical understanding of deep learning&lt;/a&gt;, by Sanjeev Arora&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;whole bunch&lt;/strong&gt; of resources in the &lt;a href=&quot;https://github.com/jphall663/awesome-machine-learning-interpretability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Awesome Machine Learning Interpretability repo&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: I do no longer keep this answer updated; for updates, see my answer in &lt;a href=&quot;https://ai.stackexchange.com/questions/12870/which-explainable-artificial-intelligence-techniques-are-there/24138#24138&quot;&gt;Which explainable artificial intelligence techniques are there?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11539" LastEditorUserId="11539" LastEditDate="2021-04-16T10:08:19.733" LastActivityDate="2021-04-16T10:08:19.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4727" PostTypeId="2" ParentId="4725" CreationDate="2017-12-12T20:18:43.130" Score="7" Body="&lt;p&gt;For Gomoku, it seems a bit of an overkill to use neural networks or the genetic algorithm as both take a while, and more often than not, don't go how you want it to. The Gomoku game tree is rather large, but you can get a decent AI from minimax, game tree pruning, and a good heuristic function (that includes counting half and full 2s,3s,4s,...etc.) as opposed to mapping out the full space.&lt;/p&gt;&#xA;&lt;p&gt;If you are unfamiliar with alpha-beta pruning and minimax see &lt;a href=&quot;https://www.cs.cornell.edu/courses/cs312/2002sp/lectures/rec21.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you really want to use neural networks or genetic algorithms you can for the learning experience. Regarding neural networks, one way to do it is the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Define a heuristic function that receives a board state input (sequence of 0,1,2 for empty, black, white), and outputs a 'goodness' value of the board state. The neural network is our heuristic function.&lt;/li&gt;&#xA;&lt;li&gt;Assuming that the moves in these game are optimal, train off of the difference between the current best move (by your current parameters) and what move your data says is the best. This is how we define our error function! Thus you are minimizing this difference so that what move your neural network says is the strongest is ideally what your game data says is the strongest (Optimization of this error function can be done via backpropagation or genetic algorithm).&lt;/li&gt;&#xA;&lt;li&gt;Ideally, by this point, you can now use your ('strong') neural network-based evaluation function for your game tree move evaluations instead of hardcoded heuristics.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Of course, this is just one way, and you would need to find game data first.&lt;/p&gt;&#xA;&lt;p&gt;As a side note, applying a genetic algorithm can occur in a few ways, such as parameter optimization in a neural network as mentioned above or game tree searching, so make sure you are clear how you define the problem setting with it! The same goes for alternative ways to apply a neural network.&lt;/p&gt;&#xA;&lt;p&gt;Finally, it's helpful to know Gomoku is solved. See &lt;a href=&quot;https://stackoverflow.com/q/6952607/3924118&quot;&gt;this post&lt;/a&gt; for others' thoughts and ideas.&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="2444" LastEditDate="2021-01-30T22:13:35.237" LastActivityDate="2021-01-30T22:13:35.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4729" PostTypeId="2" ParentId="4711" CreationDate="2017-12-13T03:18:12.600" Score="1" Body="&lt;p&gt;Good questions. Let me reply one by one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Filter size can be increased. There is no limit for it. However, think two cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DNN part. Shape will be 1024 x 7 x 7 and it will map to 4096 features, it will cause 204M parameters at dense_1 layer. This change will cause two possible problem. Overfitting and inference/training speed.&lt;/li&gt;&#xA;&lt;li&gt;Sparsity. You can implement 1024 features at 5th conv block and train the network and check the accuracy to see if increased feature count will make any impact or not.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, to decide the number of con layers, it is a good practice to check sparsity of feature layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- If I did not understand your question wrong, those layers are DNN classifier as a fully connected layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3- Maxpooling is not causing us to see a subset of the image. It is just an algorithm to resizing. What you see in that video is the filter response to some objects exist in the image. In deeper level, do not expect to have visual shape like in shallow layers. You may even get single white pixel at deepest layer and it may be a response to an object like human face. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-12-13T03:18:12.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4731" PostTypeId="2" ParentId="4700" CreationDate="2017-12-13T03:32:27.583" Score="1" Body="&lt;p&gt;One way to solve the issue may be a tracker (like Kalman). It would be faster and easier approach than neural nets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you insist about solving this issue through neural nets, then some magic and creativity is needed. Based on the nature of tracking, you need to feed multiple frames to predict next location of the object and check if there is any object close by. So, you may consider combining RNNs with CNNs to predict next bounding box to track and replace Kalman filter's prediction with RNNs. (&lt;a href=&quot;https://www.slideshare.net/BigDataColombia/introduccin-a-deep-learning-aprendizaje-profundo&quot; rel=&quot;nofollow noreferrer&quot;&gt;check&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-12-13T03:32:27.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4732" PostTypeId="2" ParentId="4696" CreationDate="2017-12-13T03:38:52.177" Score="2" Body="&lt;p&gt;It depends on what you need. You can train any size of network on any resource. The problem is the time of training. If you want to train Inception on an average CPU it will take months to converge. So, it all depends on how long you can wait to see your results based on your network. As in neural nets we do not have only one operation but many (like concatenating, max pooling, padding etc.), it is impossible to make an estimation as you are searching for. Just start training some infamous networks and measure the time. Then, you can interpolate how long it will take to train networks that you are searching for. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-12-13T03:38:52.177" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4733" PostTypeId="2" ParentId="4695" CreationDate="2017-12-13T03:43:16.760" Score="1" Body="&lt;p&gt;Each network has a limit that it can learn. To check that limit, you need to try. Especially those kind of simple networks, it can be achieved so quickly. Idea is that, you need to give X data and check the accuracy and increase it to 2X data and check the accuracy. If there is no change in accuracy, it means that your network has reached its limit and you may consider making it deeper and wider. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By nature of the training, the last sample that you bring to training is the result that optimizer will try to converge. Hence, optimizers are already doing what you are searching for. However, if you want to create some weights for them, try considering RNNs. Their nature is suitable for that purpose. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2017-12-13T03:43:16.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4734" PostTypeId="2" ParentId="4041" CreationDate="2017-12-13T04:30:04.477" Score="1" Body="&lt;p&gt;Short version below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When implementing a minimax algorithm the purpose is usually to find the best possible position of a game board for the player you call max after some amount of moves. In some games like tic-tac-toe, the game tree (a graph of all legal moves) is small enough that the minimax search can be applied exhaustively to look at the whole game tree. More complex games like chess have too large of a game tree to be feasibly searched exhaustively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple version of minimax just travels through the game tree, evaluating every legal move for the position currently being evaluated before going further and evaluating the possible answers to those moves. To find an optimal winning move; minimax needs only to search until a winning state of the game has been found. If implemented using the aforementioned breadth first search minimax algorithm, it will have found the way to win in the least amount of moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case where min has a forced win the truly optimal move doesn't exist. If min is not an optimal player, the definition of optimal can be the move that is most likely to cause him to make an error, that enables max to force a win. That move isn't necessarily the move that leads to the most moves until loss. &#xA;As an example, consider a position in some game where max has two moves, move A and move B, move A leads to a loss in 100 moves and move B to a loss in one move. Naively move A is better but in this game the only legal moves following move A lead to a loss and move B leads to a position where min has hundreds of legal moves but only one causes him to win. Albeit a bit extreme, this example demonstrates that optimality is hard to define in a losing position. Put simply, is a very complex loss in 6 moves worse than a obvious loss in 20?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You did define a version of optimality however and implementing it is possible. Since you are only considering optimal moves, an exhaustive search must be performed and thus, there is no reason to give a score to any positions but a win, loss, and draw. The method I would use is to assign each state a score, much larger than the maximum possible amount of moves, e.g. a loss is -100,000 a win is 100,000 and a draw is 0. Then you maintain a variable that is the depth of the search or number of moves that have to be performed to reach this state. Then I would add the number of moves to the large number. So a loss in 20 moves would have a score of -99,980 and a draw in 15 moves would have the score of 15. 100,000 is a bit excessive for most games but it just has to be large enough that a loss, win, and draw is never confused as a draw in 100,001 moves would look better than a win in 1. Note that this method should only be used for losses and draws since using this method for wins would result in a win in 10 having a score of 100,010 and a win in 20 a score of 100,020 and thus looking better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short version:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use breadth first search.&lt;/li&gt;&#xA;&lt;li&gt;For winning positions: terminate the minimax when a win is found.&lt;/li&gt;&#xA;&lt;li&gt;For losses and draws: search the whole game tree and give the position a score of 0+MTP for draws and L+MTP for losses.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;L is a large number and MTP is the number of moves to reach the position.&lt;/p&gt;&#xA;" OwnerUserId="11556" LastActivityDate="2017-12-13T04:30:04.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4735" PostTypeId="2" ParentId="4219" CreationDate="2017-12-13T10:55:36.597" Score="2" Body="&lt;p&gt;Here are some more, apart from the ones reported in the accepted answer:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Pinafore/qb&quot; rel=&quot;nofollow noreferrer&quot;&gt;QANTA&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/miyyer/dan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Averaging Networks (DAN&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/allenai/deep_qa&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepQA&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/xuchen/jacana&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jacana&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://quepy.machinalis.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quepy&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.qanus.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QANUS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/SeanTater/uncc2014watsonsim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Watsonsim&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11539" LastActivityDate="2017-12-13T10:55:36.597" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4737" PostTypeId="1" CreationDate="2017-12-13T13:26:14.637" Score="2" ViewCount="90" Body="&lt;p&gt;Pieter Abbeel says that having access to the dynamics model, &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s,a)$&lt;/span&gt;, is unrealistic because it assumes we know the probability that we will reach all future states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how this is unreasonable? Could someone explain this to me like in a simple way?&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2019-09-04T13:43:56.343" LastActivityDate="2019-09-07T09:24:13.890" Title="Why is the access to the dynamics model unrealistic in Q-Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4738" PostTypeId="1" CreationDate="2017-12-13T13:40:10.167" Score="2" ViewCount="86" Body="&lt;p&gt;I am thinking of an application where I want to teach a Neural Network to adjust some parameters on another device for a specific purpose.&#xA;I say &quot;adjust some parameters&quot; because I need the output to &quot;turn the knobs&quot; of something else, not only binary &quot;push buttons&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of neural network do I need? is it a specific architecture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NNs I have used so far were sequential and outputting vectors like &#xA;[0] or [1]&#xA;and classifiers with outputs like [1,0,0,0] [0,1,0,0] [0,0,1,0] [0,0,0,1] for example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I need outputs like [0.25, 0.99, 0.14, 0.54], &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0.25 for knob 1 &lt;/li&gt;&#xA;&lt;li&gt;0.99 for knob 2&lt;/li&gt;&#xA;&lt;li&gt;0.14 for knob 3&lt;/li&gt;&#xA;&lt;li&gt;0.54 for knob 4&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you for your answers&lt;/p&gt;&#xA;" OwnerUserId="11567" LastActivityDate="2017-12-20T17:09:48.610" Title="Which NN Arch? - Given input vec X -&gt; Output vector Y made of &quot;not binary&quot;, floating point values" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4740" PostTypeId="1" AcceptedAnswerId="4745" CreationDate="2017-12-13T19:25:59.450" Score="5" ViewCount="1226" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://www.nature.com/articles/nature14236&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human-level control through deep reinforcement learning&lt;/a&gt;, the DQN architecture is presented, where the loss function is as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_i(\theta_i) = \mathbb{E}_{(s, a, r, s') \sim U(D)} \left[ \left( r + \gamma \max_{a'} Q(s', a'; \theta_i^-)  - Q(s, a; \theta) \right)^2\right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$&lt;/span&gt; approximates the &quot;target&quot; of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a; \theta)$&lt;/span&gt;. But it is not clear to me why. How can existing weights approximate the target (ground truth)? Isn't &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is a sample from a the experience replay dataset? Is &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; a scalar value?&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2018-11-19T18:21:18.537" LastActivityDate="2018-11-19T18:21:18.537" Title="Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4741" PostTypeId="2" ParentId="4696" CreationDate="2017-12-13T19:40:21.683" Score="3" Body="&lt;p&gt;As a caveat, I’d suggest that unless you’re pushing up against fundamental technological limits, computation speed and resources should be secondary to design rationale when developing a neural network architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, earlier this year I finished my MS thesis that involved bioinformatics analytics pipelines with whole genome sequencing data - that project took over 100,000 hours of compute time to develop according to our clusters job manager. When your on a deadline, resources can be a real constraint and speed can be critical. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your questions as I understand them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Would I have been better off to use the money to buy time in the cloud?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably. The few hundred dollars you spent on the 1060 would take you far training your model(s) in the cloud. Further, as far as I can tell, you don’t require the GPU to be cranking 100% of the time (you would if you were, say, mining crypto currencies). Finally, with cloud instances you could scale, training multiple models at once, which can speed up the exploration and validation of any architecture you settle on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way to gauge the compute time of a neural network on a given GPU&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Big_O_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Big O&lt;/a&gt; is one estimator, but it sounds like you want a more precise method. I’m sure they exist, but I’d counter that you can make your estimation with simple back of the envelope calculations that account for threads, memory, code iterations, etc. Do you really want to dig into the GPU processing pipeline on the 1060? You may be able to come up with a very good estimate by understanding everything happening between your code and the metal, but ultimately it’s probably not worth the time and effort; it will likely confirm that Big O notation (the simple model, if you will) captures most of the variation in compute time. One thing you can do if you notice bottlenecks is performance profiling. &lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="5210" LastEditDate="2017-12-13T19:45:48.340" LastActivityDate="2017-12-13T19:45:48.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4742" PostTypeId="2" ParentId="4672" CreationDate="2017-12-14T00:15:03.833" Score="0" Body="&lt;p&gt;Your question is a bit vague. Is the ruleset defined? If not the bot can just do whatever it likes, and then it technically wouldn't be playing the game anymore. But by restricting its moves you are also implicitly giving it the ruleset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless, minimax can &quot;solve&quot; chess (or more generally, all games). The only things it needs are the &quot;endgame states&quot; (the set of all board positions that is a checkmate), and a function that given some input board state, output the set of all possible board states that might follow, where we define a board state to contain information about the location of each chess prices and who is the next player to move. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ruleset is then implicitly defined by the &quot;next possible states function&quot;, although the robot never actually knows the rules that govern each piece.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the rule set is not defined, then perhaps one can imagine a bot that plays chess by making random moves at the press of a button. One then can then play chess with the robot by either 1. ignoring the fact that the bot is making random moves, and  2. undo any move the bot makes that is illegal, and then get it to move again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need something as complicated as AlphaGo to play chess naively, but the minimax approach needs a lot of computational power.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-14T00:15:03.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4743" PostTypeId="1" CreationDate="2017-12-14T12:39:33.200" Score="7" ViewCount="1896" Body="&lt;h3&gt;Goal&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I want to create an artificial intelligence to compete against other players in a board game.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Game explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I have a board game similar to 'snakes and ladders'. You have to get to a final field before your opponent does. But instead of depending on luck (throwing the dices) this game uses something like 'food'. You can go as far as you'd like, but it costs food to move (the more you move the more one extra field costs) and you can only get food in some special fields. And there aren't any snakes or ladders so you have to run the whole part. There are some more rules, for example, you can go backward and are only allowed to go into the goal if you've got less than some amount of 'food' and there are some extra fields with other special effects.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;For one player&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If there was only one player as there isn't anything like 'luck' in this game, I theoretically could just compute every single method to find the one and only the best method. Practically, I should use an algorithm that requires less computational power.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;For two or more players&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The challenge comes with the other player(s). I cannot visit an already taken field. And some other fields give me bonuses depending on my relative position to the other player (I'll just talk about two-player games). For example, only if I'm behind him that special field gives me some extra food.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;My question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It would be ideal if I had some kind of a neural network that knows the field bonuses and I would give my position, the opponents position, the food and so on (the &lt;strong&gt;state&lt;/strong&gt; of the game) and it would compute a value between -100 and 100 (assuming fields from 0 to 100) of how many fields I should go (forward or backward). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read a bit about Q-learning, deep reinforcement learning and deep neural networks. Is this the right approach to solving my problem? And if yes, have you got any more concrete ideas? The multiple actors and the sheer endless possibilities for moving depending on endless states make it hard for me to think of anything. Or is there a different, way better way that slipped past me?&lt;/p&gt;&#xA;" OwnerUserId="11585" LastEditorUserId="2444" LastEditDate="2019-11-22T22:26:30.147" LastActivityDate="2019-11-22T22:26:30.147" Title="How do I create an AI for a two-players board game?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4744" PostTypeId="2" ParentId="4743" CreationDate="2017-12-14T13:31:16.187" Score="9" Body="&lt;p&gt;Assuming it is a turn-based game and, for each turn, there's an optimal choice that will lead to the winning state (zero-sum), you can basically simplify the question to &quot;&lt;em&gt;What is the optimal sequences of moves for me to win, considering the current situation that is presented on the board?&lt;/em&gt;&quot;. So you will need to perform your algorithm every turn as the optimal sequence will change when the board changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a relatively reliable AI algorithm that's being implemented to win games like chess, backgammon, etc. The technique is called &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;minimax&lt;/strong&gt;&lt;/a&gt;. In summary, minimax is simply a search algorithm to minimize the other player's score, while maximizing your score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the problems you will encounter is that the search tree becomes so wide as we search through deeper parts of the tree, so it is essential to also implement &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;alpha-beta pruning&lt;/strong&gt;&lt;/a&gt; to cut down the amount of search. In general, alpha-beta pruning is simply eliminating branches that are very unlikely to be the optimal choice (in terms of score) thus reducing the number of searches to speed up the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="11493" LastEditorUserId="2444" LastEditDate="2019-11-22T22:18:26.103" LastActivityDate="2019-11-22T22:18:26.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4745" PostTypeId="2" ParentId="4740" CreationDate="2017-12-14T16:11:57.743" Score="3" Body="&lt;p&gt;This is the problem that reinforcement learning (RL) is trying to solve: What is the best way to behave when we don’t know what the right action is and only have a scalar (the reward (r) is a scalar) reward of how well we have done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL approaches this problem by utilizing temporal difference learning and makes predictions based on the previous experience. An RL agent is trying to maximize the sum of future discounted rewards, called the return. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$&lt;/span&gt; is essentially saying &quot;&lt;em&gt;the reward I just saw + &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; * (my prediction of the return given that I take what I think is the best action in the current state and follow my policy from then on)&quot;&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By updating the Q function, the agent can better predict the consequences of its actions and can then choose the best action with greater probability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (gamma) helps to balance between the immediate reward and future rewards. A &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt; makes the immediate reward the only thing that matters but usually the reward for a good action is delayed so values of gamma which put a higher importance on later rewards (&lt;span class=&quot;math-container&quot;&gt;$\gamma = .8, .9, .99$&lt;/span&gt;, etc) are used. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2018-11-19T07:37:07.087" LastActivityDate="2018-11-19T07:37:07.087" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4746" PostTypeId="2" ParentId="4041" CreationDate="2017-12-14T22:30:29.347" Score="2" Body="&lt;p&gt;Minimax deals with two kinds of values:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Estimated values determined by a heuristic function.&lt;/li&gt;&#xA;&lt;li&gt;Actual values determined by a terminal state.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Commonly, we use the following denotational semantics for values:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A range of values centered around 0 denote estimated values (e.g. -999 to 999).&lt;/li&gt;&#xA;&lt;li&gt;A value less than the smallest heuristic value denotes a loss for max (e.g. -1000).&lt;/li&gt;&#xA;&lt;li&gt;A value more than the biggest heuristic value denotes a win for max (e.g. 1000).&lt;/li&gt;&#xA;&lt;li&gt;The value 0 denotes either an estimated draw or an actual draw.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The advantage of this denotational semantics is that comparing values is the same as comparing numbers (i.e. you don't need a special comparison function). We can extend this denotational semantics to incorporate optimality of winning and losing as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A range of values centered around 0 denote estimated values (e.g. -999 to 999).&lt;/li&gt;&#xA;&lt;li&gt;A range of values less than the smallest heuristic value denote loss (e.g. -2000 to -1000).&lt;/li&gt;&#xA;&lt;li&gt;A range of values more than the biggest heuristic value denote win (e.g. 1000 to 2000).&lt;/li&gt;&#xA;&lt;li&gt;The value 0 denotes either an estimated draw or an actual draw.&lt;/li&gt;&#xA;&lt;li&gt;A loss in n moves is denoted as -(m - n) where m is a sufficiently large number (e.g. 2000).&lt;/li&gt;&#xA;&lt;li&gt;A win in n moves is denoted as (m - n) where m is a sufficiently large number (e.g. 2000).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Using this denotational semantics for values requires only a small change to the minimax algorithm:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function minimax(node, depth, max)&#xA;    if max&#xA;        return negamax(node, depth, 1)&#xA;    else&#xA;        return -negamax(node, depth, -1)&#xA;&#xA;function negamax(node, depth, color)&#xA;    if terminal(node)&#xA;        return -2000&#xA;&#xA;    if depth = 0&#xA;        return color * heuristic(node)&#xA;&#xA;    value = -2000&#xA;&#xA;    foreach child of node&#xA;        v = -negamax(child, depth - 1, -color)&#xA;&#xA;        if v &amp;gt; 1000&#xA;            v -= 1&#xA;&#xA;        if v &amp;gt; value&#xA;            value = v&#xA;&#xA;    return value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Incorporating optimality for draws is a lot more difficult.&lt;/p&gt;&#xA;" OwnerUserId="9634" LastActivityDate="2017-12-14T22:30:29.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4747" PostTypeId="2" ParentId="4696" CreationDate="2017-12-15T09:30:20.337" Score="6" Body="&lt;p&gt;Usually the problem is to fit the model into video RAM.&#xA;If it does not, you cannot train your model at all without big efforts (like training parts of the model separately).&#xA;If it does, time is your only problem. But the difference in training time between consumer GPUs like the Nvidia 1080 and much more expensive GPU accelerators like the Nvidia K80 are not very large. Actually the best consumer cards are faster than GPU accelerators, but lack other properties like VRAM. Random comparisons and benchmarks: &lt;a href=&quot;http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning&lt;/a&gt;, &lt;a href=&quot;https://medium.com/@alexbaldo/a-comparison-between-nvidias-geforce-gtx-1080-and-tesla-p100-for-deep-learning-81a918d5b2c7&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comparison between NVIDIA’s GeForce GTX 1080 and Tesla P100 for Deep Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To calculate if your models fits into VRAM, you just approximate how much data and which hyperparameters you have (inputs, outputs, weights, layers, batch size, which datatype and so on).&lt;/p&gt;&#xA;" OwnerUserId="11600" LastEditorUserId="32410" LastEditDate="2021-04-23T01:39:47.150" LastActivityDate="2021-04-23T01:39:47.150" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4748" PostTypeId="1" AcceptedAnswerId="4749" CreationDate="2017-12-15T09:56:33.757" Score="7" ViewCount="7240" Body="&lt;p&gt;I have a binary classification problem, where a false positive error has a very big cost compared to the false negative error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to design a classifier for such problems (preferably, with an implementation of the algorithm)?  &lt;/p&gt;&#xA;" OwnerUserId="11601" LastEditorUserId="2444" LastEditDate="2019-08-09T09:28:45.867" LastActivityDate="2020-06-26T16:53:12.217" Title="Binary classifier that minimizes false positive error" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4749" PostTypeId="2" ParentId="4748" CreationDate="2017-12-15T13:26:30.567" Score="6" Body="&lt;p&gt;There is no predefined classifier for any problem. Two main features of a classifier are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;its cost function and&lt;/li&gt;&#xA;&lt;li&gt;its corresponding weight update formula.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Since your problem statement requires a huge cost for falsely classifying a particular class one approach will be.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You have to define a cost function that will penalize hugely for misclassifying for that class only. So your cost function will be &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$J'$&lt;/span&gt; put together. You can look up the cost function of a logistic classifier to see how two separate cost functions are merged together &lt;a href=&quot;http://www.holehouse.org/mlclass/06_Logistic_Regression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The second approach can be (assuming you are using supervised learning), the learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; for both the classes should be different. The larger learning rate will be for the one which is the more important class, since you don't want to classify it improperly (increasing &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; compared to the other class will reduce or risk of misclassifying it). The exact learning rate depends from case to case.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thus, I have tailored the two main features of the classifier to solve this problem:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The cost function.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The weight update scheme (by changing learning rate for different cases).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-06-26T16:53:12.217" LastActivityDate="2020-06-26T16:53:12.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4750" PostTypeId="1" CreationDate="2017-12-15T16:03:57.683" Score="1" ViewCount="4422" Body="&lt;p&gt;I have a computer science degree and have done AI courses and projects. I want to study AI in a good university, probably online because in my country there are no good universities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which  online AI masters degrees do you recommend me? Do you recommend me to try to study a &quot;normal&quot; master's degree instead of an online one?&lt;/p&gt;&#xA;" OwnerDisplayName="user11604" LastEditorUserId="16565" LastEditDate="2018-10-05T20:50:52.647" LastActivityDate="2018-10-05T20:50:52.647" Title="Which online AI masters degrees do you recommend me?" Tags="&lt;academia&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4751" PostTypeId="1" CreationDate="2017-12-15T16:37:35.293" Score="2" ViewCount="135" Body="&lt;p&gt;I am interested in learning more about the capabilities of AI, one of my ideas with practical functionality is using images of the rear of a log hauling truck to measure the individual logs using AI. The diameter and length of the log determine the board foot of that log. The diameter is the variable, with most lengths being the same 8ft or 16ft length. The AI would have to measure the diameter usually in 1-inch increments. &#xA;I assume you would have to train the AI using pictures of logs that were manually measured. The images would mostly be straight on rear shots, not much off angle. However not every tree grows perfectly round so it will need to be able to measure those to maximize board feet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using AI long term it would be interesting if it could suggest the best way to cut the log gaining the most lumber board feet from the log. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be used to develop something like this? Obviously some type of image recognition with learning.&lt;/p&gt;&#xA;" OwnerUserId="11605" LastActivityDate="2018-03-20T06:41:53.253" Title="Measuring logs from a picture" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4752" PostTypeId="1" CreationDate="2017-12-15T17:15:42.727" Score="10" ViewCount="3409" Body="&lt;p&gt;I am trying to detect a TV channel logo inside a video file, so simply given an input &lt;code&gt;.mp4&lt;/code&gt; video, detect if it has that logo present in a specific frame, say first frame, or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have that logo in advance (although might not be the %100 same size) and the location is always fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have a pattern matching-based approach. But that requires the pattern be %100 same size. I would like to use Deep Learning and Neural Network to achieve that. How can I do that? I believe CNN can have a higher efficiency?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/qa3Tp.png&quot; width=&quot;400&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/LCDtS.png&quot; width=&quot;400&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="16565" LastEditDate="2018-11-08T16:07:40.343" LastActivityDate="2018-11-08T16:07:40.343" Title="Use AI or Neural Network for logo detection" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;pattern-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4753" PostTypeId="2" ParentId="4750" CreationDate="2017-12-15T17:42:41.363" Score="3" Body="&lt;p&gt;If you aren't looking to be accredited, here are some good options:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ai.stanford.edu/courses/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://bair.berkeley.edu/courses.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berkeley&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.toronto.edu:40292/courses&quot; rel=&quot;nofollow noreferrer&quot;&gt;University of Toronto&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.metacademy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;metacademy&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the links are dead, a google search on the relavant course code should yield results.&lt;/p&gt;&#xA;&lt;p&gt;If you are looking for accreditation, then you might want to try:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coursera&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But note that in my opinion, the class materials from the links above should be of better quality.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-12-15T18:05:51.530" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4754" PostTypeId="2" ParentId="4750" CreationDate="2017-12-15T19:04:47.733" Score="2" Body="&lt;p&gt;There are a few universities that offer MSc in AI. The highest ranked of these include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ulretch University which promises to empower students with the skills to carry out AI research in academic or R&amp;amp;D environments and to identify how AI techniques can provide intelligent solutions to IT problems in companies and organisations.&lt;/li&gt;&#xA;&lt;li&gt;KU Leuven which is Belgiums largest and highest ranked university. &lt;/li&gt;&#xA;&lt;li&gt;Polytechnic University of Catalonia, this is the largest university in Catalonia Spain.&lt;/li&gt;&#xA;&lt;li&gt;Radboud University which is located in the Netherlands has a strong commitment to research. &lt;/li&gt;&#xA;&lt;li&gt;University of Amsterdam which has a technical approach to AI. &lt;/li&gt;&#xA;&lt;li&gt;University of Edinburg has the UK's longest established center for AI. &lt;/li&gt;&#xA;&lt;li&gt;University of Georgia has an interdisciplinary Institute for Artificial Intelligence. Areas of specialization include automated reasoning, cognitive modeling, neural networks, genetic algorithms, expert databases, expert systems, knowledge representation, logic programming, and natural-language processing. &lt;/li&gt;&#xA;&lt;li&gt;University of Gronigen which is one of the oldest and largest universities in the Netherlands. &lt;/li&gt;&#xA;&lt;li&gt;University of Sheffield hosts a number of leading A.I. experts teaching will include lectures, seminars, tutorials, individual assignments and a major research project.&lt;/li&gt;&#xA;&lt;li&gt;University of Southampton offers a multidisciplinary research led MSc in AI course that covers fundamental aspects of traditional symbolic and sub-symbolic aspects.&lt;/li&gt;&#xA;&lt;li&gt;University of Alberta offers a MSc focusing on generalizing knowledge through RL&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="4398" LastEditDate="2017-12-15T19:52:38.230" LastActivityDate="2017-12-15T19:52:38.230" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4755" PostTypeId="2" ParentId="4752" CreationDate="2017-12-15T19:21:54.243" Score="5" Body="&lt;p&gt;To perform image recognition you have to find a way to represent an image with certain features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the defining characteristics of a good image recognition algorithm are it's ability to detect salient regions, that is, regions which contain the most information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a lot of attention on deep learning for content-based image classification at the moment. You can achieve decent results by implementing deep learning having three or more layers of CNN's where each layer is responsible for extracting one or more feature of the image.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2017-12-15T19:21:54.243" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4756" PostTypeId="2" ParentId="4743" CreationDate="2017-12-15T19:37:33.080" Score="1" Body="&lt;p&gt;To make an AI opponent, you'll need to create a sub-routine that considers the current state of the board and chooses a move, just like the player would. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how does this subroutine choose what move to make? You need to take the current board and calculate its value. Then consider every possible move you could make. Then consider, for each of those, every possible move your opponent could make. Iterate to maximum depth. You will have constructed a tree structure (breadth first).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prune your tree. Any branch that guarantees you a decrease in board value can be pruned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, somehow, compare remaining branches. Either you optimistically weigh possible best outcomes and choose those branches. Or continue pruning based on potential of worst outcomes.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2017-12-15T19:37:33.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4757" PostTypeId="1" AcceptedAnswerId="4758" CreationDate="2017-12-15T20:39:56.260" Score="2" ViewCount="179" Body="&lt;p&gt;Even though modern chess playing programs have demonstrated themselves to be as strong (or stronger) than even the best human players for nearly 20 years now (1997 when IBM's Deep Blue defeated the world chess champion Gary Kasparov), why would a game like Chess still be considered a valuable research subject in Artificial Intelligence?  In other words, what can be gained by continuing to advance AI in areas that have already surpassed human capabilities . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, as recently as November 2017, Google successfully challenged their deep learning technology against one of the worlds strongest &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chess playing programs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11606" LastActivityDate="2017-12-15T21:17:40.370" Title="Why is chess still a benchmark for Artificial Intelligence" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4758" PostTypeId="2" ParentId="4757" CreationDate="2017-12-15T21:17:40.370" Score="3" Body="&lt;p&gt;Chess isn't really a benchmark per say. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method developed in AlphaGo to play Go should in principle generalize quite nicely to other games of this sort, such as chess. Since Stockfish is quite dominantly the strongest Chess AI, the natural question would be to see how well AlphaGo's method compares to Stockfish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being one of the most well developed AI agents of all time, the situation concerning the defeat of Stockfish (AlphaZero was trained for only 4 hours entirely via self-play, without access to historical data) signifies the complete dominance of modern neural-network methods over classic methods (hard coded evaluation functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also as @DukeZhou♦ mentioned in the comments, while Chess bots can regularly beat human players, it's still a useful metric to evaluate bots against each other via &quot;games&quot; of this sort.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-15T21:17:40.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4760" PostTypeId="1" CreationDate="2017-12-16T05:32:04.693" Score="1" ViewCount="123" Body="&lt;p&gt;I am building a feed-forward neural network with two hidden layers, which I will train with a medical dataset, which consists of both data, such as age and sex, and images of x-ray scans (&lt;span class=&quot;math-container&quot;&gt;$1024 \times 1024$&lt;/span&gt;). The labels are types of cancer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that ages and sex will affect the output of the network. But including the images will make the network biased towards the images, because images will occupy most of the input layer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I design the input layer of the network? &lt;/p&gt;&#xA;" OwnerUserId="11614" LastEditorUserId="2444" LastEditDate="2019-11-28T17:36:50.390" LastActivityDate="2019-11-28T17:36:50.390" Title="How can I design the input layer of a feed-forward neural network to be trained with a medical dataset with three features?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4761" PostTypeId="2" ParentId="3488" CreationDate="2017-12-16T06:29:28.337" Score="2" Body="&lt;p&gt;Let's start at the beginning. GANs are models that can learn to create data that is similar to the data that we give them.&lt;/p&gt;&#xA;&lt;p&gt;When training a generative model other than a GAN, the easiest loss function to come up with is probably the Mean Squared Error (MSE).&lt;/p&gt;&#xA;&lt;p&gt;Kindly allow me to give you an example (&lt;a href=&quot;https://www.quora.com/Is-it-true-that-Generative-Adversarial-Networks-GANs-learn-the-loss-function-automatically-and-if-so-how&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trickot L 2017&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Now suppose you want to generate cats ; you might give your model examples of specific cats in photos. Your choice of loss function means that your model has to reproduce each cat exactly in order to avoid being punished.&lt;/p&gt;&#xA;&lt;p&gt;But that's not necessarily what we want! You just want your model to generate cats, any cat will do as long as it's a plausible cat. So, you need to change your loss function.&lt;/p&gt;&#xA;&lt;p&gt;However which function could disregard concrete pixels and focus on detecting cats in a photo?&lt;/p&gt;&#xA;&lt;p&gt;That's  a neural network. This is the role of the discriminator in the GAN. The discriminator's job is to evaluate how plausible an image is.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The paper that you cite, &lt;a href=&quot;https://danieltakeshi.github.io/2017/03/05/understanding-generative-adversarial-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Generative Adversarial Networks&lt;/a&gt; (Daniel S 2017) lists two major insights.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Major Insight 1: the discriminator’s loss function is the cross entropy loss function.&lt;/p&gt;&#xA;&lt;p&gt;Major Insight 2: understanding how gradient saturation may or may not adversely affect training. Gradient saturation is a general problem when gradients are too small (i.e. zero) to perform any learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To answer your question we need to elaborate further on the second major insight.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the context of GANs, gradient saturation may happen due to poor design of the generator’s loss function, so this “major insight” ... is based on understanding the tradeoffs among different loss functions for the generator.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The design implemented in the paper resolves the loss function problem by having a very specific function (to discriminate among two classes). The best way of doing this is by using cross entropy (Insight 1). As the blog post says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The cross-entropy is a great loss function since it is designed in part to accelerate learning and avoid gradient saturation only up to when the classifier is correct.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As clarified in the blog post's comments:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The expectation [in the cross entropy function] comes from the sums. If you look at the definition of expectation for a discrete random variable, you'll see that you need to sum over different possible values of the random variable, weighing each of them by their probability. Here, the probabilities are just 1/2 for each, and we can treat them as coming from the generator or discriminator.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-12-24T18:32:57.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4764" PostTypeId="1" AcceptedAnswerId="4772" CreationDate="2017-12-16T16:16:22.330" Score="2" ViewCount="136" Body="&lt;p&gt;As self-driving technology is improving, there are so many companies developing  self-driving cars like Google, Uber, etc. Is it possible that we won't need any private/paid self-driving cars and the &quot;self-driving taxi&quot; becomes ubiquitous in the city? If we assume that there are such taxis everywhere, would transportation become extremely low cost or free? (The self-driving car company could benefit from broadcasting advertisements for advertising agencies.)&lt;/p&gt;&#xA;" OwnerUserId="11621" LastEditorUserId="1671" LastEditDate="2018-01-02T22:42:35.153" LastActivityDate="2018-10-19T02:45:03.577" Title="Could the deployment of self-driving cars make rides free?" Tags="&lt;autonomous-vehicles&gt;&lt;social&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4766" PostTypeId="1" AcceptedAnswerId="20571" CreationDate="2017-12-16T18:51:09.517" Score="7" ViewCount="2462" Body="&lt;p&gt;The capsule neural network seems to be a good solution for problems that involve hierarchies. For example, a face is composed of eyes, a nose and ears; a hand is made of fingers, nails, and a palm; and a human is composed of a face and hands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many problems in NLP can be seen as hierarchical problems: there are words, sentences, paragraphs, and chapters, whose meaning changes based on the style of lower levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any research papers (which I should be aware of) on the application of capsule neural networks to NLP problems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there related research papers, which have been investigating hierarchical complexity within the domain of NLP, which could be easily translated to Capsule Network?&lt;/p&gt;&#xA;" OwnerUserId="11626" LastEditorUserId="2444" LastEditDate="2020-06-09T11:50:09.670" LastActivityDate="2020-06-15T14:33:32.257" Title="Have capsule neural networks been used to NLP problems?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;applications&gt;&lt;capsule-neural-network&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4770" PostTypeId="1" CreationDate="2017-12-17T06:12:44.007" Score="1" ViewCount="204" Body="&lt;p&gt;I have a Macbook Air and Asus ROG 702 with a built in GeForce 1070 GPU. I'm considering getting an NVIDIA Volta for AI applications (tensorflow / pytorch) to run on either the mac or windows. Would they even be compatible? And if so, what box would I need to house the GPU? Would it be system agnostic? (I was thinking Aikito Node). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you! All info is appreciated by this novice :D&lt;/p&gt;&#xA;" OwnerUserId="9316" LastEditorUserId="9608" LastEditDate="2017-12-19T21:16:54.540" LastActivityDate="2018-05-01T20:57:30.840" Title="Using NVIDIA VOLTA V with Mac or Windows externally" Tags="&lt;hardware&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-05-13T21:35:17.073" ContentLicense="CC BY-SA 3.0" />
  <row Id="4772" PostTypeId="2" ParentId="4764" CreationDate="2017-12-17T16:22:16.920" Score="2" Body="&lt;p&gt;You ask an interesting question.  There have been many discussions by industry on this topic.  A company called &lt;a href=&quot;https://govugo.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vugo&lt;/a&gt; has their business model based on advertising to passengers.  An article about Vugo, &lt;a href=&quot;http://tcbmag.com/news/articles/2017/november/the-quest-to-make-ridesharing-free&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;The Quest to Make Ridesharing Free&quot;&lt;/a&gt;, states:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Flessner is co-founder and CEO of Minneapolis-based rideshare advertising platform Vugo, which uses its patented TripIntent technology to display targeted advertisements to Uber and Lyft passengers on a tablet attached to the back of the vehicle’s headrests.&lt;/li&gt;&#xA;&lt;li&gt;The company’s founders predict that within the next few years, all ridesharing vehicles in the U.S. will be driverless, and in-vehicle advertising tailored to passengers’ destinations and interests will lead to free transportation. “The idea is to put brands in front of passengers who are en route to make purchases,” says Flessner. For example, retailers will sponsor transportation to their stores so they can preview products to customers who are headed their way.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This article &lt;a href=&quot;http://money.cnn.com/2017/09/01/technology/future/free-transportation-self-driving-cars/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;How free self-driving car rides could change everything&quot;&lt;/a&gt; makes these comments:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Car data is so lucrative that Ben Volkow -- CEO of otonomo, an Israeli startup that sells vehicle data -- expects automakers to make more money selling data than vehicles by 2020.&lt;/li&gt;&#xA;&lt;li&gt;If the money made off self-driving vehicle data outweighs the costs of offering rides, then it becomes reasonable for a business to offer free rides broadly.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A paper entitled &lt;a href=&quot;http://scet.berkeley.edu/wp-content/uploads/Report-Leveraging-Adverts-in-the-Coming-Autocar-Ecosystem.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Leveraging Adverts in the Coming&#xA;Autonomous Car Eco-system&quot;&lt;/a&gt; published by Berkeley, University of California, proposes &quot;a world where rideshares are free, or in some cases, heavily subsidized through the use of advertisements&quot;.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2017-12-17T16:29:44.873" LastActivityDate="2017-12-17T16:29:44.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4773" PostTypeId="2" ParentId="4764" CreationDate="2017-12-17T21:23:13.500" Score="2" Body="&lt;p&gt;Oil prices are not cheap, but online advertisement per impression is relatively cheap. Especially given that there isn't much engagement in a taxi, its hard to imagine that ads alone can fully support a taxi service.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://www.pennapowers.com/how-much-do-ads-on-youtube-cost/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this source&lt;/a&gt;, we find CPM for a youtube video about 0.1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But keep in mind this value could be lower since user engagement in a taxi is much lower compared to someone on the internet. With smartphones everywhere, how can you entice someone to watch an ad on a moving taxi?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://www.investopedia.com/articles/personal-finance/021015/uber-versus-yellow-cabs-new-york-city.asp.&quot; rel=&quot;nofollow noreferrer&quot;&gt;this source&lt;/a&gt;, we see:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For a 5-mile, 10-minute trip going 25 miles per hour the entire way, uberX would cost the $2.55 base fare plus $3.50 for the 10 minutes plus $10.75 for the mileage, for a total of $16.80. It is not customary to tip the Uber driver.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No matter how you cut it, the margins seems to be quite thin. So free taxis seems unlikely for now unless someone comes up with a different business plan. &lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-17T21:23:13.500" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4774" PostTypeId="1" CreationDate="2017-12-17T23:00:00.030" Score="2" ViewCount="160" Body="&lt;p&gt;I am trying to find literature on a network architecture that takes the following as in input:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Action (like 'Up', 'Down', etc)&lt;/li&gt;&#xA;&lt;li&gt;Image of current state&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and outputs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Image of next state&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I already have a lot of training data for the inputs. However, I am trying to find relevant literature/architecture for this problem. &lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2021-05-27T01:06:00.547" Title="Neural Network that Predicts Game State Based on Actions" Tags="&lt;computer-vision&gt;&lt;game-ai&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4775" PostTypeId="2" ParentId="4774" CreationDate="2017-12-18T13:12:00.410" Score="0" Body="&lt;p&gt;You'll probably want to start out with &quot;Action-Conditional Video Prediction using Deep Networks in Atari Games&quot; (arXiv link: &lt;a href=&quot;https://arxiv.org/abs/1507.08750&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1507.08750&lt;/a&gt;). That's from 2015 though, I'm sure there have been lots of other interesting developments since then. This paper may still be a good starting point though, and provide you with the correct terminology to plug into google / google scholar to find more recent papers that build on top of this. Google scholar also provides functionality to automatically find papers that cite this one (interesting recent papers will probably cite this one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an additional point, you may want to reconsider your desired output. Given a current state-image and action, it may be easier to train a network to predict only the &lt;em&gt;change&lt;/em&gt; in image (i.e., predicting NEW_IMAGE - OLD_IMAGE), rather than predicting the full image. You can then always still manually reconstruct the predicted new image simply by adding that output to the old image again. I'm quite sure I've seen this being done in a more recent paper too, but don't remember exactly the title / authors.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2017-12-18T13:12:00.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4776" PostTypeId="1" AcceptedAnswerId="4777" CreationDate="2017-12-18T13:27:20.397" Score="6" ViewCount="204" Body="&lt;p&gt;I was watching a lecture on policy gradients and Bellman equations. And they say that a Bellman equation indirectly creates a policy, while the policy gradient directly learns a policy. Why is this?&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2020-04-03T19:45:42.063" LastActivityDate="2020-04-03T19:50:19.863" Title="Why do Bellman equations indirectly create a policy?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policy-gradients&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4777" PostTypeId="2" ParentId="4776" CreationDate="2017-12-18T14:20:04.910" Score="2" Body="&lt;p&gt;Policy gradient methods directly learn parameters of a policy function, which is a mapping from states to actions. For example, &lt;span class=&quot;math-container&quot;&gt;$p(s, a)$&lt;/span&gt; can denote a function which takes a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; as input, and returns a probability of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; as output (equivalently it could just take &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; as input, and output a vector or a distribution of probabilities for all actions). Such a function will often have some parameters (for example, a vector of weights if it's a linear function of the state, or matrices of weights in the case of a neural network). Policy gradient methods directly modify the function by modifying (learning) the parameters of that function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Bellman equation is typically used to learn a function that computes different kinds of values, such as &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values, which can intuitively be understood to denote all the future rewards that we expect to get if we take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (and follow a certain policy, typically &quot;the optimal policy&quot; afterwards until the end of time). Such a &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; function that we learn is not a directly learned policy, because it is not a direct mapping from current states to actions, it doesn't directly tell us which action to take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can still be used indirectly to create a policy. For example, once such a &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; function is learned, the greedy policy can be easily implemented by looping over all possible actions, and taking the action that gives the greatest &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; output. This is indirect because the learned function does not directly give you the actions, but only provides information which you can convert into actions with some extra processing.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-04-03T19:50:19.863" LastActivityDate="2020-04-03T19:50:19.863" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4778" PostTypeId="1" AcceptedAnswerId="4781" CreationDate="2017-12-18T19:38:08.683" Score="1" ViewCount="66" Body="&lt;p&gt;&lt;em&gt;TLDR : Is there an AI available that can recognize employees in a factory and tell when they entered and left pre-defined areas?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work in a factory where we gather cycle time data from various inputs (computer interfaces, bar code readers and RFID tags). We follow both parts moving along the production line and persons working on those parts because they move from stations to stations during their day and can be more than one at the same place to help one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our goal is to know how many people are working on each part at any given time, but the problem is that it takes time for people to check themselves in when they get to their workstation and they can also forget to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many employees asked me if I could find a way to track them automatically so they wouldn't need to check in everywhere they go. And then I stumbled upon &lt;a href=&quot;https://www.youtube.com/watch?v=O1pDOkzsFOU&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft Workplace Safety Demo&lt;/a&gt; and &lt;a href=&quot;http://www.scmp.com/magazines/post-magazine/long-reads/article/2123415/doctor-border-guard-policeman-artificial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yitu System&lt;/a&gt; (this one is scary) but they both seem to be a little overkill for my needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After learning about these, here is the ideal AI features that I'd need :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can use video feeds to detect new people in the workspace and prompt someone (via e-mail or text message) for identification.&lt;/li&gt;&#xA;&lt;li&gt;Can use video feeds to recognize persons that are already known and document every time someone has entered or left a pre-defined zone (workstation) in the video feed.&lt;/li&gt;&#xA;&lt;li&gt;Won't force me to have a camera for each workstation.&lt;/li&gt;&#xA;&lt;li&gt;Allows one workstation to span on more than one video feed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As of now, I found nothing available that does this, but I may have missed lots of things because this is not what I am used to work with. So maybe you know something that could help me with that?&lt;/p&gt;&#xA;" OwnerUserId="9361" LastActivityDate="2018-06-29T07:56:23.487" Title="Searching an AI to recognize and locate persons in a factory" Tags="&lt;computer-vision&gt;&lt;facial-recognition&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4779" PostTypeId="1" CreationDate="2017-12-18T21:07:10.847" Score="2" ViewCount="1481" Body="&lt;p&gt;I'm creating a Decision Tree and at the very root level itself, I'm getting &lt;strong&gt;negative Information Gain&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per my knowledge, Information Gain is always &lt;code&gt;&amp;gt; 0&lt;/code&gt;....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any explanation....please....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please look at the node below.....&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    [69+,42-]&#xA;     /   \        &#xA;    /     \                       &#xA;[56+,33-]  [11+,11-]&#xA;&#xA;IG = H([69+, 42-]) - H([56+,33-],[11+,11-]) &#xA;&#xA;   = {(-69/111 * lg(69/111) -42/111 * lg(42/111)) &#xA;      - 89/111 * ( -56/89*lg(56/89) -33/89*lg(33/89)) &#xA;      - 22/111 * ( -11/22*lg(11/22) -11/22*lg(11/22))}&#xA;&#xA;   = -0.004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It turns out that for every feature, the &lt;code&gt;IG &amp;lt; 0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do to decide the feature at the root node?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8720" LastEditorUserId="8720" LastEditDate="2017-12-18T21:18:55.663" LastActivityDate="2018-11-18T11:01:10.203" Title="Getting Negative Information Gain (using Entropy as a measure of disorder)" Tags="&lt;machine-learning&gt;&lt;decision-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4780" PostTypeId="1" AcceptedAnswerId="4782" CreationDate="2017-12-18T23:23:25.590" Score="7" ViewCount="189" Body="&lt;p&gt;I am a software engineer who has taken an interest in machine learning about a year ago. I work on a couple of projects where a large amount of data is collected, and am wondering what I can do (or practice) in order to start recognizing opportunities to implement machine learning. Until now, I've spent some time building various classifiers with tensorflow, caffe, etc. using pre-defined data sets for random things I've found online (whether it be photos, spreadshets, etc) - and I just want to be able to use it at work. What is the best way to start getting in the frame of mind where I start recognizing where I can apply it? I found this article: &lt;a href=&quot;https://hbr.org/2017/10/how-to-spot-a-machine-learning-opportunity-even-if-you-arent-a-data-scientist&quot; rel=&quot;noreferrer&quot;&gt;https://hbr.org/2017/10/how-to-spot-a-machine-learning-opportunity-even-if-you-arent-a-data-scientist&lt;/a&gt; but it didn't seem too helpful to me. If there are any other thoughts and/or resources online that you could point me to, that would be helpful. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="11667" LastActivityDate="2018-01-02T16:14:15.443" Title="How to start looking for machine learning opportunities in projects" Tags="&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-24T00:21:50.740" ContentLicense="CC BY-SA 3.0" />
  <row Id="4781" PostTypeId="2" ParentId="4778" CreationDate="2017-12-18T23:51:27.627" Score="1" Body="&lt;p&gt;I am definitely not the most qualified person in order to answer this question, but I might be able to give you a couple of buzz words to go and do some more research on your own. A convolutional neural network is usually used for analyzing anything visual. (you can read about it here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Convolutional_neural_network&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of frameworks that allow you to build recognition classifiers that can be loaded into a real-time video feed and give you feedback when it gets a hit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the ones that I know of are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://caffe2.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://caffe2.ai/&lt;/a&gt; or &lt;a href=&quot;http://caffe.berkeleyvision.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://caffe.berkeleyvision.org/&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pjreddie.com/darknet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/darknet/&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://docs.opencv.org/2.4/modules/ml/doc/neural_networks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.opencv.org/2.4/modules/ml/doc/neural_networks.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With Caffe being the most popular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some articles on how these work:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/darknet/yolo/&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://coding-robin.de/2013/07/22/train-your-own-opencv-haar-classifier.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://coding-robin.de/2013/07/22/train-your-own-opencv-haar-classifier.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this is at least able to get you on the right path.&lt;/p&gt;&#xA;" OwnerUserId="11667" LastActivityDate="2017-12-18T23:51:27.627" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4782" PostTypeId="2" ParentId="4780" CreationDate="2017-12-19T00:14:51.630" Score="3" Body="&lt;p&gt;If you are trying to see whether machine learning can contribute to your work, first thing you ought to do is to see if any of the following tasks could help the business at hand. Machine learning tasks include the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Classification&lt;/li&gt;&#xA;&lt;li&gt;Regression&lt;/li&gt;&#xA;&lt;li&gt;Clustering&lt;/li&gt;&#xA;&lt;li&gt;Density estimation&lt;/li&gt;&#xA;&lt;li&gt;Dimensionality reduction&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11462" LastActivityDate="2017-12-19T00:14:51.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4783" PostTypeId="1" CreationDate="2017-12-19T03:04:18.840" Score="5" ViewCount="1406" Body="&lt;p&gt;Model-based RL creates a model of the transition function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a type of model-based RL?&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2019-02-16T19:05:37.663" LastActivityDate="2019-02-16T19:14:31.143" Title="Is Q-learning a type of model-based RL?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;definitions&gt;&lt;model-based-methods&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4784" PostTypeId="1" CreationDate="2017-12-19T07:53:43.403" Score="1" ViewCount="762" Body="&lt;p&gt;&lt;em&gt;I have a &lt;strong&gt;video&lt;/strong&gt; which is capture from a moving car and video showing plenty of details like pools, human, cars/buses, roads, etc, etc.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The video I am playing in unity3d and camera Showing that video and user is allow to click on video. I want to get different information from the videos like if someone clicked on video (like clicked on a bus object), then, unity UI should show that it is a bus object. &lt;strong&gt;It something like object detection in video within unity environment&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From last couple of days I searched and found different things like : &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.assetstore.unity3d.com/en/#!/content/21088&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;OpenCV for Unity3d&lt;/strong&gt;&lt;/a&gt; an asset on asset store but cost is the problem.&#xA;So is there any tut or custom way available that i should write it&#xA;myself?&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/llSourcell/YOLO_Object_Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Yolo&lt;/strong&gt;&lt;/a&gt; but its in python and i found no integration guide that I&#xA;follow and integrate it in unity.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/neutmute/emgucv/tree/master/Emgu.CV.Unity&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;EmguCV Unity&lt;/strong&gt;&lt;/a&gt; is also an option but its github don't conver any guide, I downloaded its&#xA;project there is not sample or docs. And &lt;a href=&quot;https://www.assetstore.unity3d.com/en/#!/content/24681&quot; rel=&quot;nofollow noreferrer&quot;&gt;its paid version&lt;/a&gt; cost is about 400$. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any third thing compatible with unity3d? or do i can use/integrate above given projects in unity?&lt;/p&gt;&#xA;" OwnerUserId="11671" LastEditorUserId="11671" LastEditDate="2017-12-20T04:32:38.577" LastActivityDate="2017-12-20T04:32:38.577" Title="Object detection in video" Tags="&lt;image-recognition&gt;&lt;tensorflow&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4785" PostTypeId="2" ParentId="4783" CreationDate="2017-12-19T11:18:38.760" Score="8" Body="&lt;p&gt;Tabular Q-Learning does not explicitly create a model of the transition function. It does not generate any output that you can afterwards use as a function to predict what the next state &lt;em&gt;s'&lt;/em&gt; will be given a current state &lt;em&gt;s&lt;/em&gt; and an action &lt;em&gt;a&lt;/em&gt; (that's what a transition function would allow you to do). So no, Q-learning is still model-free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, model-based RL does not necessarily have to involve creating a model of the transition function. Model-based RL can also mean that you assume that such a function is already given. It just means that you use such a function in some way.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2019-02-16T19:14:31.143" LastActivityDate="2019-02-16T19:14:31.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4786" PostTypeId="1" CreationDate="2017-12-19T21:21:32.760" Score="2" ViewCount="477" Body="&lt;p&gt;I'm learning about NEAT from the paper &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having trouble understanding how adjusted fitness penalizes large species and prevents them from dominating the population, I'll demonstrate my current understanding through an example and, hopefully, someone will correct my understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have two species, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, species &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; did really well the last generation and were given more children, this generation they have &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; children and their fitnesses are &lt;span class=&quot;math-container&quot;&gt;$[8,10,10,12]$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; children and their fitnesses are &lt;span class=&quot;math-container&quot;&gt;$[9,9]$&lt;/span&gt;. The adjusted fitnesses for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; will be &lt;span class=&quot;math-container&quot;&gt;$[2, 2.5, 2.5, 3]$&lt;/span&gt; and for B will be &lt;span class=&quot;math-container&quot;&gt;$[4.5, 4.5]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, onto distributing children, the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; states: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Every species is assigned a potentially different number of offspring in proportion to the sum of adjusted fitnesses &lt;span class=&quot;math-container&quot;&gt;$f'_i$&lt;/span&gt; of its member organisms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the sum of adjusted fitnesses is &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, thus &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; gets more children and keeps growing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this process penalize large species and prevent them from dominating the population?&lt;/p&gt;&#xA;" OwnerUserId="11684" LastEditorUserId="2444" LastEditDate="2019-11-24T21:58:00.940" LastActivityDate="2019-11-24T21:58:00.940" Title="How does adjusted fitness penalize large species in NEAT?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;&lt;fitness-functions&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4788" PostTypeId="2" ParentId="4751" CreationDate="2017-12-20T04:12:34.640" Score="1" Body="&lt;p&gt;A good machine learning object detector, would be helpful in finding the logs but classical image processing would be required to accurately find the log profiles.  Neural networks are generally not used for computing geometric dimensions.  In addition, you will need to have a calibration method so that you can convert pixels to some distance unit (e.g. inches, centimeters, etc.).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.timbeter.com/en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Timbeter&lt;/a&gt; app seems to do what you need (see below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dUov5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dUov5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://countingthings.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;app&lt;/a&gt; (see below)  is interesting although it doesn't solve your problem.  It appears good at counting but finding the profiles to within an inch is a bigger problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/046j7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/046j7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This blog &lt;a href=&quot;https://craftofcoding.wordpress.com/2017/12/04/how-many-logs-can-a-woodchuck-count/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;HOW MANY LOGS CAN A WOODCHUCK COUNT?&quot;&lt;/a&gt; goes over the image processing challenges of counting logs.  The author doesn't completely solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a post on Stackoverflow, &lt;a href=&quot;https://stackoverflow.com/questions/37540305/how-to-count-white-object-on-binary-image&quot;&gt;How to count white object on Binary Image?&#xA;&lt;/a&gt; which makes an attempt.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2017-12-20T04:12:34.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4790" PostTypeId="1" CreationDate="2017-12-20T15:43:29.047" Score="0" ViewCount="125" Body="&lt;p&gt;This is the question which always is being discussed. Will artificial intelligence be the undefeatable enemy for human beings? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like the Slaughter Bots (&lt;strong&gt;Killer Microdrones technology&lt;/strong&gt;) can kill a human or an living thing in just some seconds. And it neither could be deceived by its target nor it could be stopped by any source. Once it's set to kill the target, it can never be stopped and it identifies its target. It is also the innovation of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.youtube.com/watch?v=oSyGbDDG2c4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microdrones technology introduction&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11706" LastEditorUserId="9161" LastEditDate="2019-03-13T19:36:27.837" LastActivityDate="2019-03-13T19:36:27.837" Title="Will/can AI be the worst enemy to humanity?" Tags="&lt;agi&gt;&lt;neo-luddism&gt;&lt;superintelligence&gt;&lt;automation&gt;&lt;ai-takeover&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4791" PostTypeId="1" CreationDate="2017-12-20T15:54:43.057" Score="2" ViewCount="50" Body="&lt;p&gt;So, I have this huge amount of data, which has 7 vector features (float from 0 to 1). I am trying to build a kind of recommendation system, with a twist (it uses agents and negotiations and narratives; narratives meaning, that there will be temporal and partial order causal link dynamics, or &quot;short term memory&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is network effects in the data, since the more agents I have, the more potential there will be for match making; in simulation phase I will be just using randomness, instead of real data, but I believe the real dataset will also be more or less random: of course I think there will be nice clusters of correlations for the vectors, but I also want to understand the more general aspects of the problem, instead of just this specific use-case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agents will be generating a list of artifacts in co-operation and also evaluate how the generated list of artifacts match their preferences. The agents will be recommending items from their own list (and some times from a list of their related agents) and after each suggestion, there is negotiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there would not be network effect in the data, I would use simple heuristics, which would measure the diff of each feature. However, in network effect world this does not work very well, because it will make many agents unable to co-operate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of something, where some features are preferred features by the agents in different circumstances (depending on the other agents), this will enable some elasticity to the system and IMO avoid the simple problems, of which many recommendation systems suffer (lack of context awareness; humans do not like the likes of other people, we have flexible preferences, which depend upon our mood).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this sounds like a very generic problem, it seems like I have just discovered a problem, which has been elegantly solved by someone else from the academia years ago, but I just don't know the name for this concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of depending on strict evaluation criteria (only accept diff X per feature), how to choose evaluation function to make the agents more co-operative? I was thinking something, where diff has to be narrow for at least one feature, but may be big for three worst matching features; so that there would be essence of few features and diversity of several.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: As I have spent some more time with this problem, I have a hunch about which issues might be related:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In real world networks, which depend upon energy efficient co-operation and signaling, scale-free networks seem to emerge; effective Agent-Based Model in network effect scenario should probably form co-operation networks, which follows some kind of Fitness Function.&lt;/li&gt;&#xA;&lt;li&gt;Due to scale-free networks, it should be assumed, that the Agents using essential features (which are common in data) should be ranked higher and Agents using diverse features (which are rare in data) should be ranked lower; in scale-free networks (like Facebook) it has been observed, that all but 3% of nodes form edges to more popular nodes: this makes signaling efficient.&lt;/li&gt;&#xA;&lt;li&gt;As a reference, in deep learning Capsule Networks have had some state-of-the-art success with this similar hierarchical &quot;recommendation for upper tiers&quot; mentality (there are much more lower layer features, than upper layer features).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, I have this idea, that when we use Agent-Based Models for solving problems with network effects, the evaluation function should evaluate Agents skill of passing the problem for more popular Agent, which could delegate it to the best Agent or attempt to solve it on it's own.&lt;/p&gt;&#xA;" OwnerUserId="11626" LastEditorUserId="11626" LastEditDate="2017-12-30T04:19:48.763" LastActivityDate="2017-12-30T04:33:18.227" Title="How to choose evaluation functions for features, when network effects are in place (multi-agent systems)?" Tags="&lt;multi-agent-systems&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4793" PostTypeId="2" ParentId="4738" CreationDate="2017-12-20T17:09:48.610" Score="2" Body="&lt;p&gt;This is a regression problem instead of a classification problem. For both, you can use a feedforward neural network. These are the typical ones you've probably seen where each neuron in a layer is connected to each neuron in the next.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a classification problem, you want an output like you mentioned above, a one-hot vector corresponding to the correct class. The way you get this vector is by applying a soft-max activation to the outputs of your last layers. If you don't apply this activation, your output vector might be something like [-2.5, 0.6, 10.2, 5.1]. You can see that this looks a lot like the vector you want. So, you can probably use a feedforward architecture (with non-linear activations on the hidden layers if there are non-linearities in your data) with a final layer of 4 neurons. Then, when you train, that should (hopefully) converge to the correct data outputs. If it doesn't, you will need to mess with the architecture/get more training data/lots of other levers to play with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This link&lt;/a&gt; goes to a tutorial for regression using Keras. Keras is a nice package written to take powerful libraries like TensorFlow and Theano and give them a simple interface. TensorFlow can be quite complex (I've never used Theano so I don't know) when you're starting and trying to understand the syntax and the ML concepts. Keras helps take away the confusing syntax.&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2017-12-20T17:09:48.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4799" PostTypeId="5" CreationDate="2017-12-21T19:10:59.540" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:10:59.540" LastActivityDate="2017-12-21T19:10:59.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4800" PostTypeId="4" CreationDate="2017-12-21T19:10:59.540" Score="0" Body="Use on questions about getting started in AI.  Ideal for those new to the field who want to begin researching and developing applications." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:10:59.540" LastActivityDate="2017-12-21T19:10:59.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4801" PostTypeId="5" CreationDate="2017-12-21T19:12:13.617" Score="0" Body="&lt;p&gt;Python was invented by the Dutch software engineer Guido van Rossum in 1990 during his work at the Centre for Mathematics and Computer Science (CWI) in Amsterdam, The Netherlands. He named the language after the British comedy series &lt;em&gt;Monty Python's Flying Circus&lt;/em&gt;. The first public version was released in 1991. All releases since version 2.1.1 are GPL compatible through the PSF (Python Software Foundation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dedicated domain for the language is &lt;a href=&quot;https://www.python.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;python.org&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The language is characterized by several important language features.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Highly interactive interpreter&lt;/li&gt;&#xA;&lt;li&gt;Provides a wide variety of integrated types&lt;/li&gt;&#xA;&lt;li&gt;Supports mixed arithmetic&lt;/li&gt;&#xA;&lt;li&gt;Supports object oriented programming and dynamic type creation&lt;/li&gt;&#xA;&lt;li&gt;Unique constructs such as comprehensions and generators&lt;/li&gt;&#xA;&lt;li&gt;Uses indentation instead of end brackets for statement grouping&lt;/li&gt;&#xA;&lt;li&gt;Accelerated around bottlenecks with calls to C/C++ libraries&lt;/li&gt;&#xA;&lt;li&gt;Supported by a broad set of development tools&lt;/li&gt;&#xA;&lt;li&gt;Extensible, through imports, of many common and unusual libraries&lt;/li&gt;&#xA;&lt;li&gt;Well integrated with CPU and GPU based machine learning facilities&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-12-29T11:56:34.920" LastActivityDate="2018-12-29T11:56:34.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4802" PostTypeId="4" CreationDate="2017-12-21T19:12:13.617" Score="0" Body="For questions related to AI implementation in the Python language" OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-12-29T11:56:34.920" LastActivityDate="2018-12-29T11:56:34.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4803" PostTypeId="5" CreationDate="2017-12-21T19:13:45.567" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:13:45.567" LastActivityDate="2017-12-21T19:13:45.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4804" PostTypeId="4" CreationDate="2017-12-21T19:13:45.567" Score="0" Body="Use for basic, fundamental questions about AI theory or practice.  (i.e. design, application, implementation, mathematics of AI, philosophy of AI, etc.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:13:45.567" LastActivityDate="2017-12-21T19:13:45.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4805" PostTypeId="2" ParentId="4790" CreationDate="2017-12-21T19:35:20.540" Score="1" Body="&lt;p&gt;I highly suggest you read Hannu Rajaneimi's &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Quantum_Thief&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Thief&lt;/a&gt; trilogy, William Gibson's &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromancer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuromancer&lt;/a&gt; trilogy, and Phillip K. Dick's &lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;Do Android's Dream of Electric Sheep&lt;/a&gt;, Isaac Asimov's &lt;a href=&quot;https://en.wikipedia.org/wiki/I,_Robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;I, Robot&lt;/a&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a nuanced subject, and no one knows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most credible scenarios I've found for the destruction of humanity by automata involves not high-intelligence, but very low intelligence.  (The &lt;a href=&quot;https://en.wikipedia.org/wiki/Grey_goo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grey Goo scenario&lt;/a&gt;.)  Dick, a gifted philosopher in his own right, proposes that empathy is a natural function of high intelligence, which provides a credible, increasingly validated ray of light. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But Hawking and Musk are credible imo per their comfort with very large numbers (complexity) and proven world-class abstract thinking capabilities.  It's possibly that strong &quot;hyperpartizan&quot; AI could run amok.  We've already seen issues with high-frequency trading algorithms, and automation is proceeding in weapons tech.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might say &quot;we are cursed to live in such interesting times.&quot; &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-12-21T19:35:20.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4806" PostTypeId="5" CreationDate="2017-12-21T19:39:31.553" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AI_takeover&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AI_takeover&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/The_Matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/The_Matrix&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/The_Terminator&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/The_Terminator&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:39:31.553" LastActivityDate="2017-12-21T19:39:31.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4807" PostTypeId="4" CreationDate="2017-12-21T19:39:31.553" Score="0" Body="For questions related to the apocalyptic or dystopian scenarios of automata supplanting humanity." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:39:31.553" LastActivityDate="2017-12-21T19:39:31.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4808" PostTypeId="5" CreationDate="2017-12-21T19:41:50.130" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Automation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Automation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_trading_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Automated_trading_system&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Johns_Hopkins_Beast&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Johns_Hopkins_Beast&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Talos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Talos&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:21:09.023" LastActivityDate="2018-01-26T21:21:09.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4809" PostTypeId="4" CreationDate="2017-12-21T19:41:50.130" Score="0" Body="Use for questions related to automation.  (All automation may be said to be a function of applied intelligence--or algorithms.)  Suitable for questions on industrial automation, autonomous vehicles, autonomous weapons systems, etc., and may also be extended to virtual automation, such as automated trading systems, etc." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:21:09.023" LastActivityDate="2018-01-26T21:21:09.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4810" PostTypeId="5" CreationDate="2017-12-21T19:45:54.110" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neo-Luddism&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Neo-Luddism&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:45:54.110" LastActivityDate="2017-12-21T19:45:54.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4811" PostTypeId="4" CreationDate="2017-12-21T19:45:54.110" Score="0" Body="Use for questions related to the &quot;unforeseeable dangers of new technology&quot;, a sub-field of the overall philosophy of neo-luddism.  (i.e. suitable for for questions ranging from automation replacing human jobs, instability created by applications of AI, and apocalyptic scenarios of AI takeover, etc.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-21T19:45:54.110" LastActivityDate="2017-12-21T19:45:54.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4812" PostTypeId="5" CreationDate="2017-12-21T19:50:26.330" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-30T02:40:23.787" LastActivityDate="2021-01-30T02:40:23.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4813" PostTypeId="4" CreationDate="2017-12-21T19:50:26.330" Score="0" Body="For questions related to fitness functions in the context of evolutionary computation and evolutionary algorithms. A fitness function is a function that assesses the quality (or fitness) of an individual (i.e. solution) in the population (in the case of population-based evolutionary algorithms)." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-30T02:40:23.787" LastActivityDate="2021-01-30T02:40:23.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4814" PostTypeId="5" CreationDate="2017-12-21T19:53:16.597" Score="0" Body="&lt;p&gt;For more info, see e.g. the book &lt;a href=&quot;http://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (2nd edition) by Sutton and Barto. See also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;related Wikipedia article&lt;/a&gt; or e.g. &lt;a href=&quot;http://artint.info/html/ArtInt_265.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://artint.info/html/ArtInt_265.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-25T16:07:11.900" LastActivityDate="2019-08-25T16:07:11.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4815" PostTypeId="4" CreationDate="2017-12-21T19:53:16.597" Score="0" Body="For questions related to the Q-learning algorithm, which is a model-free and temporal-difference reinforcement learning algorithm that attempts to approximate the Q function, which is a function that, given a state s and an action a, returns a real number that represents the return (or value) of state s when action a is taken from s. Q-learning was introduced in the PhD thesis &quot;Learning from Delayed Rewards&quot; (1989) by Watkins." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-25T16:07:11.900" LastActivityDate="2019-08-25T16:07:11.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4818" PostTypeId="2" ParentId="3850" CreationDate="2017-12-22T14:56:28.880" Score="3" Body="&lt;p&gt;Being a  complete newbie in machine learning, I did this experiment (using Scikit-learn ):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Generated a large number (N) of pseudo-random extractions, using python random.choices function to select N numbers out of 90.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Trained a MLP classifier with training data composed as follow:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ith sample : X &amp;lt;- lotteryResults[i:i+100], Y &amp;lt;- lotteryResults[i]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In practice, I aimed to a function that given N numbers, coud predict the next one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Asked the trained classificator to predict the remaining numbers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Results: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;of course, the classificator obtained a winning score comparable with the one of  random guessing or of other techniques not based on neural networks (I compared results with several classifiers available in scikit-learn libraries )&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;however, if I generate the pseudo-random lottery extractions with a specific distribution function, then the numbers predicted by the neural network are roughly generated with the same distribution curve ( if you plot the occurrences of the random numbers and of the neural network predictions, you can see that that the two have the same trend, even if in the predicytions curve there are many spikes. So maybe the neural network is able to learn about pseudo-random number distributions ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I reduce the size of the training set under a certain limit, I see that the classifier starts to predict always the same few numbers, which are among the most frequent in the pseudo-random generation. Strangely enough ( or maybe not ) this behaviour seem to slightly increase the winning score.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11742" LastActivityDate="2017-12-22T14:56:28.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4819" PostTypeId="1" AcceptedAnswerId="4820" CreationDate="2017-12-22T17:17:54.323" Score="1" ViewCount="80" Body="&lt;p&gt;I can't understand the red box. what do does that &lt;code&gt;1&lt;/code&gt; do before &lt;code&gt;{c(I)=j}&lt;/code&gt; ?&#xA;Also how does all the algorithms work? May someone explains the to steps more clearly by one example!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ylp04.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ylp04.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11747" LastActivityDate="2017-12-22T19:40:24.343" Title="How does this part of algorithm works?(K-means)" Tags="&lt;algorithm&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4820" PostTypeId="2" ParentId="4819" CreationDate="2017-12-22T19:40:24.343" Score="0" Body="&lt;p&gt;&lt;code&gt;1_{c^(i) = j}&lt;/code&gt; means if &lt;code&gt;c^(i)&lt;/code&gt; is equal to &lt;code&gt;j&lt;/code&gt; the value is 1 and zero otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, k-means try to solve Voronoi cells for each mean point, and then update mean of members of a cell and update Voronoi cells again. This step is repeated until there is no change in members of the cells.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2017-12-22T19:40:24.343" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4821" PostTypeId="1" CreationDate="2017-12-22T20:09:12.370" Score="1" ViewCount="51" Body="&lt;p&gt;I'm working on architecture for a game AI where, due to the nature of the game, the classical approach seems likely be sufficient to beat most humans--the endgame is tractable and traditional game-solving and is worth pursuing due to certain intrinsic properties of the game. (Strategy is a direct application of minimax, although there is a great deal of nuance in that determination.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplified rules for the game can be found &lt;a href=&quot;http://www.mclassgames.com/rules-of-m/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and there is a &lt;a href=&quot;http://mbranegame.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;free app&lt;/a&gt; (no ads, no purchases, no data collected) where you can try the mechanics for the basic, non-trivial game. (Tutorial takes about 10 minutes.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At some point, we'd like to integrate some form of &lt;em&gt;local&lt;/em&gt; NN, But size of the database for learning/training would be dependent on how much volume the user is willing to devote to their &quot;mbrane&quot;.  (These will initially be mobile devices such as phones and tablets.  New iPad pro has upt to 512GB, but most devices will be much smaller.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this even feasible?  Today?  At some point in the future?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Uses: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&quot;Tuning&quot; AI behavior to the human player     &lt;/li&gt;&#xA;&lt;li&gt;Weighting complex stability states based on current board state to determine optimal placement&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;TUNING TO HUMAN BEHAVIOR&lt;br&gt;&#xA;&lt;em&gt;We want each discrete, local AI to evolve uniquely, in conjunction with human play, and adapt itself to the preferences of their respective humans.&lt;/em&gt;&#xA;&lt;br&gt;&lt;br&gt;&#xA;Hardness: Most simply, we want AI hardness to be determined by a Win/Loss ratio against the human player.  Hard = Human never wins.  Easy = Human always wins.  This forms a spectrum, so you might have settings for 2/3 W/L or 1/3 W/L.  These don't have to be precise, but the AI should tailor it's play strength for a subsequent game based on the outcome of the previous game. &lt;br&gt;&lt;br&gt;&#xA;Individuality: We want the AI's to learn not only by self-play, but by play with their humans.  It's ok if the process is glacial b/c the basic evaluation functions and perfect endgame will provide inherent AI strength. The main thing is that the discrete AI's develop uniquely.  AI's will be able to play other AI's.&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;STABILITY STATES &lt;br&gt;&#xA;&lt;em&gt;There are three simple stability states that arise out of the mechanics. These states can be strong or weak. These can be a hard T/F, or, since the game is quantitative, values between 1 and 0, based on the respective regional deltas (Neutrality). &quot;Flipping&quot; of a region refers to the changing of the dominant player in the region.&lt;/em&gt; &lt;br&gt;&lt;br&gt;&#xA;Stability (weak: can the region be flipped with a single placement? strong: can the polarity of a region be flipped by any number of placements?)&lt;br&gt;&lt;br&gt;&#xA;Epistability (weak: will the region flip under prior resolution?  strong: such that the region cannot be epistabilized by further placements?)&lt;br&gt;&lt;br&gt;&#xA;Metastability (weak: can the region's position in the resolution order be meaningfully altered with a single placement?  strong: can the position in the resolution order be meaningfully altered by any number of placements?)&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;COMPLEX STABILITY STATES &lt;br&gt;&#xA;&lt;em&gt;The simple stability states combine into 8 &quot;complex stability states&quot;, based on True/False for metastability/epistability/stability:&lt;/em&gt; &lt;br&gt;&#xA;TTT Superstability (&quot;super-stably stable&quot;)&lt;br&gt;&#xA;FTT Semistability  (&quot;semi-stably stable&quot;)&lt;br&gt;&#xA;TFT Mendaxastability (&quot;super-unstably stable&quot;)&lt;br&gt;&#xA;FFT Demistability (&quot;semi-unstably stable&quot;) &lt;br&gt; &#xA;TTF Contrastability (&quot;super-stably unstable&quot;) &lt;br&gt;&#xA;FTF Nonstability (&quot;semi-stably unstable&quot;)  &lt;br&gt;&#xA;TFF Antistability (&quot;super-unstably unstable&quot;)&lt;br&gt;&#xA;FFF Khaotictivity (&quot;semi-unstably unstable&quot;)&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two hierarchies, the ordering/weighting of metastability, epistability and stability in the complex states (here the order is reversed for ease of interpreting the linguistic description of these states,) and the ordering/weighting of the complex states themselves, currently with Superstability as the least important, and Khaotictivity as the most important.  (i.e. a strongly superstable region cannot be flipped, and does not need to be reinforced; a khaotic regions is very much &quot;in play&quot;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that deep learning might be very usefully applied in determining the hierarchies based on any give board state.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2017-12-22T20:09:12.370" Title="Mini (local) NN for Mobile?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;evolutionary-algorithms&gt;&lt;game-theory&gt;&lt;combinatorial-games&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4822" PostTypeId="1" CreationDate="2017-12-22T20:49:51.510" Score="2" ViewCount="1265" Body="&lt;p&gt;I'm really new to neural networks. I'm trying to make a neural network with genetics algorithms which will make a snake learn to look for the food and avoid hitting his tail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that I think that I've done it, but as there's no walls the snake learns to go one direction only without making a 180 turn &lt;a href=&quot;https://i.imgur.com/cdVmixk.gifv&quot; rel=&quot;nofollow noreferrer&quot;&gt;[GIF HERE]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to incentivate mutations that make them turn by decreasing the score of the snakes that always takes the same directions, but it don't works. I've only made them dumber, needing more breeds to reach another &quot;smart&quot; linear snake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've made a network with 5 inputs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Food position relative to my position and direction (2 inputs. x and y)&lt;/li&gt;&#xA;&lt;li&gt;Nearest wall (my tail) if I turn left&lt;/li&gt;&#xA;&lt;li&gt;Nearest wall (my tail) if I do not turn&lt;/li&gt;&#xA;&lt;li&gt;Nearest wall (my tail) if I turn right&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output are 3, being the first one turning left, the second do not turn and the third going right. I make the snake go the highest one of the 3 outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've added 1 hidden layer of 8 neurons (inputs + outputs premise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I calculate the score is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each step, 1 point.&lt;/li&gt;&#xA;&lt;li&gt;Each food eaten, 10 points.&lt;/li&gt;&#xA;&lt;li&gt;If the snake lasts too much time without eating food, dies.&lt;/li&gt;&#xA;&lt;li&gt;If hits his tail, dies.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then I save each time the direction this snake has gone (up, right, down, left) and increment them by one. When the snake dies, I weight the final score by the difference between the lowest and highest values. If the difference is high, they receive a big penalty (down to 0.25 of their score). This way if a snake is pretty much linear gets a high penalty and if a snake does a cool pattern, gets a low penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I keep a record of each time a direction change happens compared to the last direction change, so if a snake keeps going in circles don't gets a high score because of &quot;cool pattern method&quot; for going all 4 directions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all this, I don't understand why my best snakes are always the linear ones :-/&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I spawn 20 snakes and get the best 4 of each generation when everyone dies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For generations I use neataptics.js and for neural networks I use synaptics.js. I have a Fiddle here: &lt;a href=&quot;http://jsfiddle.net/Llorx/gunsct5r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jsfiddle.net/Llorx/gunsct5r/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At line &lt;code&gt;10&lt;/code&gt; you can see the network definition. At &lt;code&gt;211&lt;/code&gt; you can see the snake &quot;view&quot; (food position and walls. Where it gets the inputs) and at &lt;code&gt;164&lt;/code&gt; you can see the score weight calculation depending on steps taken that I mentioned before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All inputs are normalized from 0 to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure that I'm doing, not one, but a lot of things pretty bad, as I'm a newbie on this, but some light on this will be really cool.&lt;/p&gt;&#xA;" OwnerUserId="11751" LastEditorUserId="11751" LastEditDate="2017-12-22T21:02:12.060" LastActivityDate="2018-01-01T09:21:22.497" Title="Trying to make snake learn different directions" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4830" PostTypeId="1" CreationDate="2017-12-23T13:50:07.207" Score="4" ViewCount="599" Body="&lt;p&gt;Recently, DeepMind's AlphaZero chess algorithm did better than the prior best chess software Stockfish. I read the paper &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/a&gt; about it, but I'm not sure if:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;is there a value given for each piece (e.g. 1 for pawn, 3 for the knight, 9 for the queen, etc.) to train the algorithm, or does the algorithm learn this by himself?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I read that the algorithm uses Monte Carlo Tree Search, but what are the key improvements to prior chess algorithms already using MCTS?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there hope for being able to run it an average computer? They said it required 9 hours learning (starting with nearly 0 knowledge except for rules (and maybe value for a piece?)), and 24 millions of games. Is it something doable in maybe 1 month with an average computer?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11764" LastEditorUserId="2444" LastEditDate="2020-09-29T22:24:29.720" LastActivityDate="2020-09-29T22:24:29.720" Title="Is there any value given to each chess piece in AlphaZero?" Tags="&lt;deep-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2020-09-29T22:25:32.880" ContentLicense="CC BY-SA 4.0" />
  <row Id="4831" PostTypeId="1" CreationDate="2017-12-23T18:41:29.570" Score="10" ViewCount="2045" Body="&lt;p&gt;Do off-policy policy gradient methods exist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that policy gradient methods themselves using the policy function for sampling rollouts. But can't we easily have a model for sampling from the environment? If so, I've never seen this done before.&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2019-11-19T01:53:39.513" LastActivityDate="2019-11-19T01:54:37.057" Title="Do off-policy policy gradient methods exist?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4832" PostTypeId="1" CreationDate="2017-12-23T21:23:29.133" Score="1" ViewCount="58" Body="&lt;p&gt;Let say that we have four straight string with colors RED, GREEN, BLUE and YELLOW. These strings are tied up randomly. We know the current state of string (like starting point, where we should we start untying from) and the final state (where current string must look like after untying them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a very simple example of the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tI6ah.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tI6ah.png&quot; alt=&quot;a sample of the puzzle&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At every move, we are allowed to replace only two nearby strings. For example, to solve the shown problem, we should do the following replacements:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Replace GREEN and YELLOW&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Replace BLUE and GREEN&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in a programming environment, how can I calculate the movements to untie the strings? What will be the algorithm specifically to solve such type of problems? Here is another &lt;a href=&quot;https://math.stackexchange.com/questions/2581164/string-knots-unlocking-algorithm&quot;&gt;variation&lt;/a&gt; of the question. Its not possible to solve this problem according to the answer.&lt;/p&gt;&#xA;" OwnerUserId="11770" LastEditorUserId="9947" LastEditDate="2018-03-11T02:00:39.003" LastActivityDate="2021-05-21T08:14:57.643" Title="String node untying algorithm" Tags="&lt;algorithm&gt;&lt;logic&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4833" PostTypeId="1" CreationDate="2017-12-23T23:19:35.863" Score="1" ViewCount="76" Body="&lt;p&gt;Imagine this theoretical situation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A group of people are asked to provide a solution for an imaginary&#xA;  problem via email. Then an AI service runs through their written&#xA;  solution, analyzing the kind of words and sentences they use,&#xA;  essentially exploring how they faced the problem. Then the service&#xA;  finds a personality portfolio of each person.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is there such an AI service which can find such personality portfolios by analyzing the text they use describing their solution to a problem? If there is not a current service, how might it be approached?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&#xA;Koppany&lt;/p&gt;&#xA;" OwnerUserId="11772" LastEditorUserId="4398" LastEditDate="2018-01-03T06:19:31.337" LastActivityDate="2018-01-03T06:19:31.337" Title="What AI service can define personality portfolio based on text?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4834" PostTypeId="1" CreationDate="2017-12-24T06:15:24.523" Score="2" ViewCount="197" Body="&lt;p&gt;I am using HOG (Histogram of Oriented Gradients) for car detection from a video. I have used the Matlab function &lt;code&gt;extractHOGFeatures()&lt;/code&gt; , it has given me a feature vector but how do I differentiate between different features of different objects. After this I extracted the corner points(313x1) and BRISK points (172x1) but they show nothing just empty vectors.&lt;/p&gt;&#xA;" OwnerDisplayName="user3751" LastEditorUserId="1671" LastEditDate="2018-01-28T23:06:39.327" LastActivityDate="2018-01-28T23:06:39.327" Title="Extracting specific features using HOG" Tags="&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4836" PostTypeId="2" ParentId="4748" CreationDate="2017-12-24T17:02:24.067" Score="3" Body="&lt;p&gt;@DuttaA has pretty much mentioned the two most appropriate approaches to having this facility. Either the penalty of false positives should be high or the learning rate for the correct class should be high. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll give two real-life examples to help you understand it better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have to teach a teen that substance abuse is injurious to health (eg. Frequent smoking is a &lt;strong&gt;negative&lt;/strong&gt; habit). But the teen ends up learning from high effects of the drugs that it is good (&lt;strong&gt;false positive&lt;/strong&gt;) and gets addicted to it. You would strictly want to avoid this kind of a situation (&lt;strong&gt;false positive error having a very big cost as compared to false negative error&lt;/strong&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, to model the situation when the costs are different, we picture a cost matrix. For a two-class classification problem, the cost matrix would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HaVxo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HaVxo.png&quot; alt=&quot;Cost Matrix&quot;&gt;&lt;/a&gt;&#xA;(courtesy: &lt;a href=&quot;http://albahnsen.com/CostSensitiveClassification&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://albahnsen.com/CostSensitiveClassification&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, when designing your cost function, you would want to take into account the weight corresponding to each of the situation. A simple python code would be as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def weighted_cost(pred, act):&#xA;    if pred==P and act==P:&#xA;        return C_TP * cost(pred, act)&#xA;    if pred==P and act==N:&#xA;        return C_FP * cost(pred, act)&#xA;    if pred==N and act==P:&#xA;        return C_FN * cost(pred, act)&#xA;    if pred==N and act==N:&#xA;        return C_TN * cost(pred, act)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where, &lt;strong&gt;pred&lt;/strong&gt; is the predicted class and &lt;strong&gt;act&lt;/strong&gt; is the actual class. Here, &lt;strong&gt;C_TP, C_FP, C_TN, C_FN&lt;/strong&gt; represent the weights of true positive, false positive etc. The &lt;strong&gt;cost(pred, act)&lt;/strong&gt; function will calculate the loss of one training example. You would want to use the weighted_cost function for finding the loss after one training example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second approach that @DuttaA mentioned was to vary the learning rate. In real life, you can relate this to the situation when you were asked to write a word &lt;strong&gt;multiple times&lt;/strong&gt; if you forget its spelling so that you remember it better. In a way, you learn the correct spelling of the word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, increasing the value of the learning rate &lt;em&gt;(say 4 x alpha)&lt;/em&gt; for a class can be viewed as updating the value of the weights &lt;strong&gt;multiple times&lt;/strong&gt; &lt;em&gt;(4 times)&lt;/em&gt; with the old learning rate &lt;em&gt;(alpha)&lt;/em&gt;, similar to what we do by writing the correct spelling of the word multiple times. So, a more important class (in your case it will be the &lt;strong&gt;Negative Class&lt;/strong&gt;) should be given more alpha, because a false positive (misclassification of the negative class) has a high penalty. You learn to recognize the correct (negative) class by learning it more number of times (as in the case of learning the spelling of the word).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if you need any further clarification.&lt;/p&gt;&#xA;" OwnerUserId="11778" LastEditorUserId="11061" LastEditDate="2018-04-29T09:29:35.260" LastActivityDate="2018-04-29T09:29:35.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4837" PostTypeId="2" ParentId="4283" CreationDate="2017-12-24T17:32:11.583" Score="3" Body="&lt;p&gt;Yes. Skewed data is one of the biggest problems in AI applications. As you rightly identified, the real world distribution is skewed. Doing a random sampling results in one major issue of an uneven sampling (like in your case). Even worse could happen, all of the samples may fall into a single class and other classes may not even be recognized by your classifier. This is called as the &lt;strong&gt;class imbalance problem&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways to mitigate this problem. Few of them have been mentioned &lt;a href=&quot;http://www.chioka.in/class-imbalance-problem/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I'll summarize them for you:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sampling based mitigation (when you try to deal with skewed data)&lt;/li&gt;&#xA;&lt;li&gt;Cost function based mitigation (when you try to improve the classifier, in case skewed data cannot be avoided)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sampling based mitigation can be done by oversampling from the minority class(es), or undersampling from the majority class(es) or using a combination of both. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another fairer method to do this is by doing a stratified sampling. If class A has 1/6th, class B has 2/6th and class C has 3/6th of the total population, then, you should take 1/6th of the samples by random sampling from objects in class A, 2/6th from class B and 3/6th from class C. This way all of the sample may represent the population in the right proportion, and none of the classes will be missed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case this doesn't help (say, you dataset is small and sampling doesn't make sense) and the data is skewed too much), the cost function can be modified to make a more sensible classifier. Keep different costs for misclassification and/or correct classification. Misclassification could be given a higher penalty. This could help in reaching to a better classifier with a skewed data.&lt;/p&gt;&#xA;" OwnerUserId="11778" LastActivityDate="2017-12-24T17:32:11.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4838" PostTypeId="2" ParentId="4831" CreationDate="2017-12-24T17:35:08.393" Score="5" Body="&lt;p&gt;Absolutely, it’s a really interesting problem. Here is a paper detailing &lt;a href=&quot;https://arxiv.org/abs/1205.4839&quot; rel=&quot;nofollow noreferrer&quot;&gt;off policy actor critic&lt;/a&gt;. This is important because this method can also support continuous actions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general idea of off-policy algorithms is to compare the actions performed by a behaviour policy (which is actually acting in the world) with the actions the target policy (the policy we want to learn) would have chosen. Using this comparison we can determine a ratio (&lt;span class=&quot;math-container&quot;&gt;$0 \leq \rho \leq 1$&lt;/span&gt;) which can scale the update to the target policy by the probability of the target policy taking that action. A higher &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, the more alike the two policies are, and this increases the magnitude of the learning update for the target policy for that step. A &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and the update is ignored. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2019-11-19T01:54:37.057" LastActivityDate="2019-11-19T01:54:37.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4839" PostTypeId="2" ParentId="4833" CreationDate="2017-12-24T20:22:29.997" Score="1" Body="&lt;p&gt;My first recommendation would be before you create an AI or ML based solution. Kindly consider using a business Q&amp;amp;A Software such as Questions for Confluence by Atlassian among others. An enterprise multiple choice solution could be a simple and elegant fit for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if one was to design a solution in accordance to your specification the first step to resolve the problem would definitely be one of applying NLP. In my answer I utilize the open source procedure laid out by EASE (Enhanced AI Scoring Engine) by Edx and a combined initiative of MIT and Harvard towards improving automated essay scoring (Parachuri V 2013). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computers cannot directly understand text the way people can. People naturally break sentences into units of meaning, however when using a computer we will first have to instruct the computer how to do it. This process is called tokenization. Tokenization is the task of chopping a phrase into pieces called tokens at the same time discarding some characters i.e. punctuation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After we tokenize the essay answers, we then convert the tokens into a matrix (bag of words model also known as vector space model). This model is a very convenient way to represent text in matrix form. Here we disregard grammar and even the word order but the frequency of words is retained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can proceed to the machine learning part!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I will recommend that we either use regression or classification. The difference between these two algorithms is that regression assumes you are using a continuous scale while classification is discrete. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In statistical modeling, regression is a set of statistical processes for estimating the relationships in variables.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A simple linear equation is $y=m*x+b$ , where y is the target value(score), m is a coefficient, and b is a constant.&#xA;  In linear regression, we would do something like $y=m{1}*x{1}+m{2}*x{2}+\dots+m{n}*x{n}+b$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Classification (also sometimes called a decision tree) on the other hand is a data mining method for assigning categories to a collection of data which is useful since we want to classify essays on the basis of their scores. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step will be to measure the error in the scores that are generated by the algorithm on the previous step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to measure accuracy, lets use cross-validation. Here we split the training dataset randomly into n parts each segment is a 'fold'. This is called n-fold cross validation. We then iterate from 1 to n predicting the scores of the parts [n] from all the data in parts [!n].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2017-12-24T20:22:29.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4840" PostTypeId="2" ParentId="4783" CreationDate="2017-12-24T21:52:55.987" Score="2" Body="&lt;p&gt;In model based learning the learning agent utilizes a model that was previously learned to accomplish a task whereas model free RL doesn't use the environment to learn but simply relies on trial and error experience for action selection. The similarity is that in both methods the learning agent is trying to maximize reward from its actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-Learning is a model free RL method. It can be used to identify an optimal action-selection policy for any given finite Markov Decision Process. How it works is that it learns an action value function which essentially gives the expected utility of an action in a given state, then follows an optimal policy afterwards. &lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2017-12-24T21:52:55.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4841" PostTypeId="1" CreationDate="2017-12-24T23:59:57.663" Score="1" ViewCount="237" Body="&lt;p&gt;I have a large number of observations. Each observation contains:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;dependent variable: a scores ranging from 0 - 100&lt;/li&gt;&#xA;&lt;li&gt;independent variable: a large article&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to know which words or phrases predict a higher score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that for discrete dependent variables, Naive Bayes can be used for this task. How do I conduct this analysis with a continuous dependent variable?&lt;/p&gt;&#xA;" OwnerUserId="11782" LastEditorUserId="11782" LastEditDate="2017-12-25T00:09:55.857" LastActivityDate="2017-12-25T00:09:55.857" Title="Natural language processing with a continuous dependent variable" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;natural-language-processing&gt;&lt;learning-algorithms&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4842" PostTypeId="2" ParentId="4343" CreationDate="2017-12-25T07:19:49.200" Score="2" Body="&lt;p&gt;It seems that LIDAR presents a problem for resolving the car's environment at higher speed. While I'm not too familiar with the dynamics of LIDAR I do know that &lt;a href=&quot;https://www.edn.com/design/analog/4442319/Autonomous-automotive-sensors--How-processor-algorithms-get-their-inputs&quot; rel=&quot;nofollow noreferrer&quot;&gt;it's a physical system that relies on sending and receiving laser pulses to various points around the car by way of rotating mirrors&lt;/a&gt;. As speeds increase, it seems &lt;a href=&quot;https://en.wikipedia.org/wiki/Lidar#Autonomous%20vehicles&quot; rel=&quot;nofollow noreferrer&quot;&gt;different arrangements of mirrors and light collectors might have to be used&lt;/a&gt; to maintain a high-resolution image. There's some evidence that &lt;a href=&quot;ftp://ftp.rap.ucar.edu/pub/rgf/velerr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Doppler LIDAR (developed in the 1990s) became less accurate with higher velocities.&lt;/a&gt; However, LIDAR is partly preferred over radar because of its higher accuracy even when tracking objects at high speeds - &lt;a href=&quot;https://www.officer.com/on-the-street/article/10250592/lidar-the-speed-enforcement-weapon-of-choice&quot; rel=&quot;nofollow noreferrer&quot;&gt;this is why LIDAR guns are increasingly being used by police instead of radar guns to track speeding vehicles&lt;/a&gt;. It seems natural that a set of high-resolution cameras paired with a well-trained neural network would not be subject to the same physical limitations as LIDAR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that an important intuition to consider is that while LIDAR is used to generate clouds of data points whose shapes and patterns can be analyzed by autonomous car software, cameras can pick up non-topograhical features such as road lines, the content of roadsigns, and additional location context such as storefronts and intersection layouts. Considering that these cameras can use pattern recognition and stereoscopy to also generate a 3D topographic map of the environment, it seems plausible that Level 5 self driving cars would not require LIDAR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://seekingalpha.com/article/4106093-tesla-betting-cameras-full-self-driving&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an interesting look at the problem.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11787" LastActivityDate="2017-12-25T07:19:49.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4845" PostTypeId="1" CreationDate="2017-12-26T01:43:16.810" Score="1" ViewCount="46" Body="&lt;p&gt;I am trying to train Chess data through CNN. To proceed reinforcement learning, I had divided into two - &quot;current network&quot; and &quot;reinforcement network&quot;. For each checkpoint file stored in different directory, would like to call the weight and proceed chess games and learn only &quot;reinforcement network.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I had made an instance for each network, current and reinforcement, however, regardless of its sequence, if a network stores saver.restore the other network returns error while same saver.restore process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I doubt this happens because I had used same variable_scope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe this is because in tensorflow there already exists same scope_variable I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any chance I could use two different already-learned network simultaneously on tensorflow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use python/tensorflow1.1.0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice will be appreicated&lt;/p&gt;&#xA;" OwnerUserId="11797" LastActivityDate="2017-12-26T01:43:16.810" Title="Regarding Tensorflow: How to Avoid Duplicate Use of Scope/Variable_names" Tags="&lt;convolutional-neural-networks&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4849" PostTypeId="1" AcceptedAnswerId="4867" CreationDate="2017-12-26T13:19:57.850" Score="0" ViewCount="272" Body="&lt;p&gt;I'm trying to learn more about AI by trying to program a neural network. First I'm trying to understand writing my own perceptron but I'm struggling to get a basic perceptron working correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried writing a few basic perceptrons to do very basic tasks for example trying to classify a point as above or below a line y = x.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I'm facing is it seems that when training the perceptron my weights start increasing exponentially and it doesn't seem like I'm getting anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I start with a perceptron with two weights representing an x and a y value of a point on a graph:&#xA;starting weight values of 1 and 1&#xA;Using the sign function of the output being more than or equal to 0 outputs +1 and less than 0 is -1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the training data with examples (1,2), (2,1), (3,3), (2,1).&#xA;I think I'm correct in saying if the perceptron outputs the correct value then you don't need to adjust the weights, if it's incorrect then the weights can be updated using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;new_weight = old_weight + ((expected_ouptut - false_output) * input)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the first iteration I have weights of -4 and -2 respectively, the second passes, the third i.e (3,3) with weights (-4,-2) changes the weights to (53,55).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm fairly sure my math is correct, and it seems like the weights are increasing exponentially without making any difference to getting a working perceptron. Are there any errors I'm making with trying to design a working perceptron?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry this may get slightly long!&#xA;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I may have been making mistakes doing it manually, I’ve written the code now that seems to be partly working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way my code works is by running test firstly with test cases, followed by the results from those test cases in the same order within the lists. Followed by the weights and finally the test case to run it from. So running in ghci:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;test [(5,2)]   [(-1)]   (1,1)   (5,2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this works correctly, first it tests the coordinates (5,2) in the perceptron with weights (1,1) and checks to see if it is equal to -1, if it is not equal to one it will run it again and alter the weights with the alter function until it does equal -1, when this happens it then checks the perceptron again and outputs with the final tuple (5,2) which is the test case to check if it’s working. This works correctly, however when working with larger sets it doesn’t work.&#xA; Next I tried&lt;/p&gt;&#xA;&#xA;&lt;p&gt;test [(1,2),(2,5),(3,1),(7,0),(6,5),(4,4)]   [1,1,(-1),(-1),(-1),1]   (1,1)   (5,1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the first two lists are the test cases and the results, then the weights (1,1) and finally after running all of the test cases trying to teach the perceptron it then runs the perceptron on the final tuple (5,1) and outputs the result, it should be (-1) as it is below y = x, however it outputs 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;testFunction&#xA;if(test cases remaining):&#xA;  if calling sign on the output of the perceptron != result for test case then:&#xA;    call same test case again but alter weights&#xA;  else:&#xA;    call testFunction again will same lists minus the first test arguments and results&#xA;else if(no more test cases, all have been tested and removed, or none specified):&#xA;  call sign on the output of the perceptron for the final tuple provided with the altered weights&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Code in Haskell:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module Main where&#xA;&#xA;-- function to take training data, results from that data to test the perceptron&#xA;-- also takes weights and finally outputs the result of the test case&#xA;test :: [(Float,Float)] -&amp;gt; [Float] -&amp;gt; (Float,Float) -&amp;gt; (Float,Float) -&amp;gt; Float&#xA;test [] [] weights testCase = sign(perceptron testCase weights)&#xA;test (t:trainData) (r:trainResult) weights testCase&#xA;  | (output /= r) = test ([t] ++ trainData) ([r] ++ trainResult) (alter weights output t r) testCase -- add t and r&#xA;  | otherwise     = test trainData trainResult weights testCase&#xA;  where&#xA;    output = sign(perceptron t weights)&#xA;&#xA;-- function that computes x1*w1 + x2*w2&#xA;perceptron :: (Float,Float) -&amp;gt; (Float,Float) -&amp;gt; Float&#xA;perceptron coordinate weights = ((((fst coordinate) * (fst weights))) + ((snd coordinate) * (snd weights)))&#xA;&#xA;-- function that finds the new weights by doing w1 = w1 + (expected_result - perceptron output)*x1 and similar for w2&#xA;alter :: (Float,Float) -&amp;gt; Float -&amp;gt; (Float,Float) -&amp;gt; Float -&amp;gt; (Float,Float)&#xA;alter (w1,w2) output (x1,x2) result = ((w1 + ((result - output) * x1)), (w2 + ((result - output) * x2)))&#xA;&#xA;-- activation function&#xA;sign :: Float -&amp;gt; Float&#xA;sign n&#xA;  | n &amp;gt;= 0 = 1&#xA;  | otherwise = (-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11795" LastEditorUserId="11795" LastEditDate="2017-12-26T18:45:15.477" LastActivityDate="2017-12-28T19:34:48.623" Title="Creating a working perceptron" Tags="&lt;neural-networks&gt;&lt;perceptron&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4850" PostTypeId="1" AcceptedAnswerId="4870" CreationDate="2017-12-26T15:00:08.953" Score="2" ViewCount="56" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://arxiv.org/abs/1003.0120&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning from Logged Implicit Exploration Data&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Formally, given a dataset of the form S = (x, a, r_a)* generated by the &lt;strong&gt;interaction of an uncontrolled logging policy&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is such a policy? &lt;/p&gt;&#xA;" OwnerUserId="11808" LastActivityDate="2017-12-28T22:08:14.853" Title="What is uncontrolled logging policy?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4851" PostTypeId="1" CreationDate="2017-12-26T15:57:06.827" Score="4" ViewCount="460" Body="&lt;p&gt;Let's pretend we had a list of facts (similar to prolog tuples) that define some knowledge about some entities. e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;doing(clean, data)&#xA;done(collect, data)&#xA;todo(train, model)&#xA;todo(write, paper)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What methods could I use to generate sentences like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;You should be cleaning the data you collected, then you need to train your model and write your paper.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="11809" LastEditorUserId="2193" LastEditDate="2020-04-15T10:59:43.470" LastActivityDate="2020-04-15T10:59:43.470" Title="How could you generate sentences from lists of facts" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4852" PostTypeId="2" ParentId="4822" CreationDate="2017-12-26T18:58:06.657" Score="1" Body="&lt;p&gt;I observed the scoring system you used and compared it to the field size and starving rate. The worms either died or did not try enough hard with the prize of 10 of eating, so I added the amount to 1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This did still not introduce the 180 degrees turn yet. I had to adjust these settings on Neat:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  mutationRate: 0.7,&#xA;  mutationAmount: 6,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which introduced &lt;img src=&quot;https://i.stack.imgur.com/OT1iN.png&quot; alt=&quot;the effect on this screenshot&quot;&gt; where the worms optimize their path as diagonal and when forced to change 90 direction, actually a tiny 180 happens on screen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On same mutationRate and mutationAmount, but 10 points I once observed a horizontal 180, when the food appeared back and diagonal movement was not invented. Also I observed with this last setup a motion of constant 180s flow where consecutive vertical or horizontal 180s formed a two pixel thick line on screen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These observations do not 100% do the same as your animation, but do instead introduce various ways that are quite near, or can be in certain conditions an initiative to such phenomenom. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZSKbn.png&quot; alt=&quot;This picture shows also the two pixel line. &quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;White worm introduces two pixel line, red 90 degree turn with miniature 180.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I changed the nice curve ratio like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     var ratio = 0.75 + ((mindir / maxdir) * 0.25* difstepsratio);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and around generation 200 there lived one worm to be able to move to all directions and make 180 if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it died before I found screen shot buttons from my phone and (s)he happened to become second in the race and thus was left as a unicorn who existed only once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still remains mystery whether that mutation could survive and become the main species some day. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further remarks:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that when mutation takes place in snake code the same pattern is replicated many times. If snake gets idea of consecutive turn lefts or turn rights when it is long, a suicide with tail is quite often taking place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to taggle this with a higher score on snake collision than starvation and around generation #180 the dna of suicidal 360 loop was born and remained some generations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this generation did not evolve to the desired intelligent 180 worm I previously had seen. Instead, the suicide was so rewarding that it killed that branch quite soon away.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2018-01-01T09:21:22.497" LastActivityDate="2018-01-01T09:21:22.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4853" PostTypeId="2" ParentId="4849" CreationDate="2017-12-26T22:17:34.250" Score="0" Body="&lt;p&gt;Solved it, the problem was I wasn't supplying it enough training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;running the following in ghci should allow you to change the final tuple and solve the problem&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;test [(104,1),(1,104),(52,3),(4,79),(9,109),(92,23),(11,78),(3,109),((-2),78),(192,30),(103,20),(4,78),(23,193),(1,58),(98,2)] [(-1),1,(-1),1,1,(-1),1,1,1,(-1),(-1),1,1,1,(-1)] (1,1) (25,24)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11795" LastActivityDate="2017-12-26T22:17:34.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4854" PostTypeId="2" ParentId="4822" CreationDate="2017-12-26T22:23:16.720" Score="1" Body="&lt;p&gt;As far as I understand, you don't give the snake rewards for eating fast.  In fact, &quot;Each step, 1 point.&quot; means that the slower it finds food, the better. So why would it do turns, considering it is dangerous.&lt;/p&gt;&#xA;" OwnerUserId="5852" LastActivityDate="2017-12-26T22:23:16.720" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4855" PostTypeId="2" ParentId="35" CreationDate="2017-12-27T08:09:46.660" Score="1" Body="&lt;p&gt;Artificial Intelligence (AI) and Machine Learning (ML) are two very hot buzzwords right now, and often seem to be used interchangeably. They are not quite the same thing, but the perception that they are can sometimes lead to some confusion. So I thought it would be worth writing a piece to explain the difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning is a core sub-area of artificial intelligence; it enables computers to get into a mode of self-learning without being explicitly programmed. When exposed to new data, these computer programs are enabled to learn, grow, change, and develop by themselves.&lt;/p&gt;&#xA;" OwnerUserId="11819" LastActivityDate="2017-12-27T08:09:46.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4858" PostTypeId="2" ParentId="4760" CreationDate="2017-12-27T13:37:46.167" Score="1" Body="&lt;p&gt;I would give some advice here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should probably use Convolutional Neural Network for image data or you'll end up with too many input features, and too much computation. What are your constraints on this point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would not mix image data with numerical data. Maybe you can implement two neural networks, one handling image data and another managing the numerical data, each one outputting their results. A second step would be to use their results to handcraft a global result or let another NN manage this.&lt;/p&gt;&#xA;" OwnerUserId="11567" LastActivityDate="2017-12-27T13:37:46.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4859" PostTypeId="1" CreationDate="2017-12-27T14:28:25.243" Score="2" ViewCount="172" Body="&lt;p&gt;Is it possible to classify or learn to estimate the minimum value in a table if the values are integer and represented 32 bits (and we can input all variables at the same moment, like in &lt;a href=&quot;https://en.wikipedia.org/wiki/System_on_a_chip&quot; rel=&quot;nofollow noreferrer&quot;&gt;system on a chip (SoC)&lt;/a&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="10186" LastEditorUserId="2444" LastEditDate="2020-05-29T22:41:32.747" LastActivityDate="2020-05-29T22:41:32.747" Title="Is it possible to learn to estimate the minimum value in a table?" Tags="&lt;machine-learning&gt;&lt;research&gt;&lt;classification&gt;&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4861" PostTypeId="2" ParentId="1285" CreationDate="2017-12-27T21:00:24.403" Score="2" Body="&lt;p&gt;I think the best example of AI being deployed on the blockchain is &lt;a href=&quot;https://singularitynet.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SingularityNET&lt;/a&gt;. They just had a successful token sell where they sold out of their AGI token which will be able to be used to essentially &quot;pay&quot; for AI tasks to be done for you. Various AI will be put on the network and able to interact and communicate with each other to get various tasks done. There are &lt;a href=&quot;https://www.youtube.com/watch?v=iFFHsmtQs6w&quot; rel=&quot;nofollow noreferrer&quot;&gt;some great videos online&lt;/a&gt; where Dr. Ben Goertzel explains this further. And here is a link to &lt;a href=&quot;https://public.singularitynet.io/whitepaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;their whitepaper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11667" LastActivityDate="2017-12-27T21:00:24.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4864" PostTypeId="1" AcceptedAnswerId="4887" CreationDate="2017-12-28T11:12:28.850" Score="18" ViewCount="19011" Body="&lt;p&gt;What is the &lt;a href=&quot;https://www.tensorflow.org/tutorials/image_retraining&quot; rel=&quot;noreferrer&quot;&gt;concept&lt;/a&gt; and how does one calculate Bottleneck values? How do these values help image classification? Please explain in simple words.&lt;/p&gt;&#xA;" OwnerUserId="11837" LastEditorUserId="1671" LastEditDate="2020-04-28T21:55:13.160" LastActivityDate="2020-04-28T21:55:13.160" Title="What is the concept of Tensorflow Bottlenecks?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;terminology&gt;&lt;bottlenecks&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="4865" PostTypeId="2" ParentId="4864" CreationDate="2017-12-28T14:13:19.937" Score="5" Body="&lt;p&gt;Tensorflow bottleneck is the last pre prosessing phase before the actual training with data recognitions start. It is a phase where a data structure is formed from each training image that the final phase of training can take place and distinguish the image from every other image used in training material. Somewhat like a fingerprint of the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is involved to the re-training command and as the name suggests, this is the main time consumer of the command execution. The amount of training material may have to be compromized if this bottleneck seems too time consuming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it is a command line command, I don't know the exact algorithm. Algorithm is public in code in Github but is supposedly so complicated (execution time is very long by definition) that I believe I cannot just write it down in this type of answer.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2017-12-28T14:13:19.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4866" PostTypeId="2" ParentId="3218" CreationDate="2017-12-28T15:13:02.993" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://dlib.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;DLIB&lt;/a&gt; has AA-class face detector based on ResNet model. &#xA;&lt;a href=&quot;http://dlib.net/face_detector.py.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is C++ example. &#xA;The accuracy is much better compared to HAAR/LBP but performance is worse (but much depends on parameters passed to HAAR/LBP detector)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latest version of OpenCV also contains face detector based on ResNet model, but I did not try this. &lt;/p&gt;&#xA;" OwnerUserId="5026" LastActivityDate="2017-12-28T15:13:02.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4867" PostTypeId="2" ParentId="4849" CreationDate="2017-12-28T19:34:48.623" Score="0" Body="&lt;p&gt;First, of all the formula you have supplied &lt;code&gt;new_weight = old_weight + ((expected_ouptut - false_output) * input)&lt;/code&gt; has no learning rate. The learning rate is there to scale the weights (one of its function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, where is the bias term? Although its not necessary if you already know a decision boundary passing through the origin will seperate the data, but in some cases it can scale the other parameters.&#xA;For example, lets say the parameters are &lt;code&gt;theta0, theta1, theta2&lt;/code&gt;. Now since there is an additional &lt;code&gt;theta0&lt;/code&gt; term getting added when there is error it will help to see that the correct result is reached faster. I know this is not a proper mathematical explanation and helps only a very tiny tiny bit almost negligible but maybe good for intuition. Correct me if i am wrong. Also it is better practice to have a bias term even for a beginner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing I would like to say I don't know how actually increasing training data helped your problem.  As per your algorithm two different points increase the weight by different amounts. Let us say we have &lt;code&gt;y = ax&lt;/code&gt; as the starting line. Say your training set contains two points. Now we use two different values of &lt;code&gt;a&lt;/code&gt; for two different cases, each one for which one of the training data will be miss classified. After a single run in both the cases will the &lt;code&gt;theta1, theta2&lt;/code&gt; change same? No actually. This can be problematic if you see the next point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say your training set has outliers (the ones which cannot be classified properly and are partly in the other class) &lt;a href=&quot;https://i.stack.imgur.com/wr3Yu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wr3Yu.png&quot; alt=&quot;We cannot linearly separate the classes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your algorithm will get trapped in a loop (maybe a local minimum) and the weights will definitely increase exponentially.&#xA;You can stop this from happening by adding a regularization term (might be quite advanced topic for beginner, but very easy to understand).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly the mathematics of these type of perceptron classifiers (discrete &lt;code&gt;1&lt;/code&gt; or &lt;code&gt;0&lt;/code&gt; are too hard and involved to be visualized since you cannot use the tools of continuous algebra (differentiation, maxima, minima, etc). Your algorithm might get trapped in local minimum with no way of you knowing it easily. So, some of the errors you might face cannot be explained easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I might be wrong for the aforementioned reason, but if you find some better intuition please do share.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2017-12-28T19:34:48.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4868" PostTypeId="1" CreationDate="2017-12-28T20:52:32.340" Score="2" ViewCount="103" Body="&lt;p&gt;I started to study neural networks recently. I understand how I should define the input and output layers. But I can't find any guidelines on how to build hidden layers. More concretely, for each specific task, are there some guidelines (or rules of thumb) for answering the following questions?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How many layers do I need?&lt;/li&gt;&#xA;&lt;li&gt;How many neurons per layer? &lt;/li&gt;&#xA;&lt;li&gt;What activation functions should I use?&lt;/li&gt;&#xA;&lt;li&gt;How do I connect the neurons?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11845" LastEditorUserId="2444" LastEditDate="2020-05-13T20:38:34.043" LastActivityDate="2021-01-20T23:32:49.263" Title="Are there some guidelines for designing the architecture of neural networks?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;deep-neural-networks&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-20T23:32:17.193" ContentLicense="CC BY-SA 4.0" />
  <row Id="4869" PostTypeId="2" ParentId="4851" CreationDate="2017-12-28T21:01:15.863" Score="1" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FTxLV.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you parse sentences you use grammatics to sentences to get the meaning (semantics).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you use sematics (tuples) and grammatics to make a sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your knowledge includes verb, noun and some kind of notion of readiness of the task defined by them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To form sentences you also need knowledge / rules about the extra eligible words between these categories, as well as how time relates to verb grammatical tense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to code a parser for tuples that codes the tuple in the three parts, formulate the grammatics,&#xA;and then apply the grammatics to the parsed tuple data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that you'll get the sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precise information on how to code that and this information plus the picture source:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cs.union.edu/~striegnk/courses/nlp-with-prolog/html/node93.html#l14.sec.nlg&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs.union.edu/~striegnk/courses/nlp-with-prolog/html/node93.html#l14.sec.nlg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2017-12-28T21:22:17.360" LastActivityDate="2017-12-28T21:22:17.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4870" PostTypeId="2" ParentId="4850" CreationDate="2017-12-28T22:08:14.853" Score="2" Body="&lt;p&gt;This policy is usually referred to as the Behaviour Policy when talking about off-policy learning. It is the policy that actually takes actions in the world. The policy that one wants to learn, such as a greedy policy in Q learning, is know as the target policy. In this way the behaviour policy “logs” experience that the target policy (or policies) may learn from.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2017-12-28T22:08:14.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4879" PostTypeId="1" CreationDate="2017-12-29T21:45:58.837" Score="0" ViewCount="388" Body="&lt;p&gt;For instance, one task would be to detect if with an Android phone in hand, I'm panning the camera toward a circle shape in a 2D space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best technology set and embedded application approach can be used for these types of motion independent pattern recognition tasks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/36kBFs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/36kBFs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XIAIms.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XIAIms.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="4302" LastEditDate="2018-11-01T23:26:27.703" LastActivityDate="2018-11-01T23:26:27.703" Title="Use mobile device camera for moving pattern recognition" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;pattern-recognition&gt;&lt;embedded-design&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4880" PostTypeId="5" CreationDate="2017-12-29T22:06:22.353" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Skynet_(Terminator)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Skynet_(Terminator)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Replicant&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Replicant&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-29T22:06:22.353" LastActivityDate="2017-12-29T22:06:22.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4881" PostTypeId="4" CreationDate="2017-12-29T22:06:22.353" Score="0" Body="Use for question related to the mythology of AI, from Skynet to the Matrix to Blade Runner, and including all speculative fiction involving AI (Isaac Asimov, Stanisław Lem, Phillip K. Dick, William Gibson, Hannu Rajaniemi, et al.)  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-12-29T22:06:22.353" LastActivityDate="2017-12-29T22:06:22.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4882" PostTypeId="2" ParentId="4791" CreationDate="2017-12-30T04:33:18.227" Score="1" Body="&lt;p&gt;In a sense it seems like I am thinking some kind of &quot;Deep Agent-Based Modeling&quot;, where it is okay to have network effects on the lowest layer (which would only evaluate the matching of the generated artifacts and their own preferences); the co-operation problem should be solved by a &quot;Coach&quot; layer, which would attempt to prevent situations, where some agents can not play well together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evaluation function of the &quot;Coach&quot; layer should be such, that it minimizes the amount of negotiation actions (energy efficient co-operation) required per artifact list generated and also minimizes the signaling distance regarding diversity (less popular artifacts / unorthodox solutions). The &quot;Coach&quot; layer should also be able to reorganize the lists contained by the Agents. The signaling distance can be easily evaluated in my use case, since the dataset has easily available popularity ranking per list artifact list item; the shorter the distance from popular artifact list to the least popular artifacts is, the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice this means, that I actually want to build composable Agents, which have lists of items (or other Agents), action space for generating artifacts from those items, evaluation functions and costs of specific actions, aggregateable popularity rank (which helps to evaluate delegation). All Agents would have generic default action space, which would help in solving the signaling problem; if the diameter of the co-operation network is too long (not within the scale-free network topology), the system should attempt to create a new layer or reorganize the existing layers (or do both).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this might be computationally very expensive; however we tested a theory about building a &quot;multi-dimensional&quot; merge-sort algorithm by using entanglement / super dense encoding from quantum information theory and it would seem like it could work and be able to create a hash map, which would allow the Agents to reshape the scale-free network efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might build some kind of demo out of this, since all this seems really interesting to me and complements nicely my past hobby projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will not accept this answer until I get some verification, that this is a good answer. However, I will post this in hopes, that someone has already done similar work, gets inspired and can provide a better answer. This answer will probably be edited later, when I get to a more formal level with this.&lt;/p&gt;&#xA;" OwnerUserId="11626" LastActivityDate="2017-12-30T04:33:18.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4883" PostTypeId="2" ParentId="2219" CreationDate="2017-12-30T05:13:57.283" Score="3" Body="&lt;p&gt;Considering your use case, I would not use Deep Learning methods... what is the point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of just winning, good AI is fun to play with. In practice when fine tuning game mechanics, you will want to analyze the game for churning events. Then it would be nice, if you could show the AI that &quot;Hey, this is messed up, could you come up with a nice way of playing, when this situation happens?&quot; and then the AI would be like &quot;Okay, sure, I didn't know, that me winning all the time was not, what humans considered fun... I'll be more fun next time, while also trying to win&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lately I have been toying around with Computational Creativity and specifically Partial Order Causal Link planners (POCL) and Agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;POCLs attempt to create plans, which fulfill goals; this makes them computationally effective as they only need to fulfill a flaw in a goal (having best possible cards on the table) and iterate towards the initial condition (specific cards on table and some cards on hands etc.). I believe, that with &lt;a href=&quot;https://nil.cs.uno.edu/publications/papers/ware2014conflict.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conflict driven POCL&lt;/a&gt; you could easily introduce bluffing. I have written POCL algorithm in declarative way, so you don't have to code the action space, but instead have them configured by using modal logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you would have agents, who would use Plan artifacts generated by the POCL algorithm in order to play in a fun way (evaluation function of the Agent), while also trying to win (search towards best odds for winning). The fun thing with Agents is, that you can compose them and discover personalities easily; I have no idea how Deep Learning methods would provide that easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, by using POCL and Agents, you could first teach the Agents to win efficiently or optimize the Plans to provide good &quot;basic moves&quot; by using some heuristic system (like you will do when using Reinforcement Learning). I don't know about the computational complexity issues regarding specific games; however, such POCL algorithms have been implemented, which are context aware (= reduced action space), so if you add a bit more strategic gameplay abstraction, the POCL should be fine (remember to use some kind of damping factors for reducing the path length of plans, in similar way to PageRank).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all programming a good mental model will make many things a lot easier. With Deep Learning, you will be using image recognition or similar algorithms / methods to solve a different problem, because nobody is preventing you from using wrong tool for the problem at hand. In real games, there are players (Agents), strategies (POCL plans), bluffing (POCL conflicts) and rules (action space of POCL defined by modal logic). Of course some games might have computational complexity issues; however, usually those are solvable by minor optimization to the algorithms, which provide a good mental model of the problem.&lt;/p&gt;&#xA;" OwnerUserId="11626" LastActivityDate="2017-12-30T05:13:57.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4884" PostTypeId="2" ParentId="4879" CreationDate="2017-12-30T11:55:14.710" Score="3" Body="&lt;p&gt;Firstly, before we commence I will recommend that you refer to similar questions on the network i.e. &lt;a href=&quot;https://stackoverflow.com/questions/6499880/ios-gesture-recognition-utilizing-accelerometer-and-gyroscope&quot;&gt;https://stackoverflow.com/questions/6499880/ios-gesture-recognition-utilizing-accelerometer-and-gyroscope&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/6368618/store-orientation-to-an-array-and-compare&quot;&gt;https://stackoverflow.com/questions/6368618/store-orientation-to-an-array-and-compare&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your problem can be divided into three parts.&lt;/p&gt;&#xA;&lt;p&gt;How to gather sensor data.&lt;/p&gt;&#xA;&lt;p&gt;How to use the gathered data to train a model&lt;/p&gt;&#xA;&lt;p&gt;How to use the trained model to make a prediction.&lt;/p&gt;&#xA;&lt;p&gt;A modern smartphone contains around six sensors packed into one device. To implement your application I recommend that you use raw sensor data from either the gyroscope or the accelerometer.&lt;/p&gt;&#xA;&lt;p&gt;On the android platform, you can access these sensors and acquire the raw sensor data by using the Android sensor framework. The Android sensor framework is part of the android.hardware package.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To capture the raw sensor data, take regular samples (&amp;gt; 20 Hz) and save the maximum values of x, y and z in an array each (to recognize in all 3 planes). If you want the gesture to be 5 seconds long, keep 100 samples (at 20 Hz). Then analyze if any of the three arrays has values which change sinusoidally. If it does, you've got a gesture.&lt;/p&gt;&#xA;&lt;p&gt;You could store these values into an array, if the user is in 'record mode'. And when the user tries to replicate that movement, the model could predict the replicated movement array from the recorded one's. The thing is, how can you compare the arrays in a smart way? (Randy M 2011)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This leads us to the next step which is applying ML.&lt;/p&gt;&#xA;&lt;p&gt;To train your model, you can choose to either train the model on the cloud or train it locally on the device. In most cases the problem of training a model on a mobile device is about computational limitations. Machine Learning algorithms running on a mobile device need to be carefully designed and implemented since most mobile devices have weak processors and small RAM's.&lt;/p&gt;&#xA;&lt;p&gt;Its quite a challenge to squeeze a large neural network (NN) into the small RAM's that smartphones's have since NN's require that the model is fully loaded into memory. In many cases it is advisable to slim the model down and set some weights near zero to zero. (Chomba B 2017)&lt;/p&gt;&#xA;&lt;p&gt;Incase you decide to utilize the cloud. Your mobile app is required to simply send an HTTPS request to a cloud web service along with the required raw sensor data, and within seconds the service replies with the prediction results. Though there are several cloud services i.e. Microsoft Azure Cognitive Services, Clarifai and Google Cloud Cognition that you can leverage to host the server side of your application, personally I recommend that you consider using reality.ai which is specifically an AI tool for engineers working with signals and sensors.&lt;/p&gt;&#xA;&lt;p&gt;The next step will be to select an appropriate algorithm to be used in classifying the gesture. Here you can employ either Logistic Regression, Support Vector Machines, Random Forest or Extremely Randomized Trees depending on your app's use case. In order to train your model we then provide the learning algorithm with 'labeled examples'. The ML algorithm then extracts the features and constructs a mathematical model that can accurately describe gestures i.e. roll, pan from raw sensor data.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2017-12-30T11:55:14.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4886" PostTypeId="1" AcceptedAnswerId="4888" CreationDate="2017-12-31T09:17:59.750" Score="3" ViewCount="267" Body="&lt;p&gt;Is it possible with any of machine learning methods to train machine to tie shoe lace? If possible how data should be interpreted for the training? &#xA;If we are using reinforcement learning, how will it learn to reach the best rewards?&lt;/p&gt;&#xA;" OwnerUserId="6687" LastEditorUserId="6687" LastEditDate="2017-12-31T12:39:44.510" LastActivityDate="2018-01-05T00:57:14.577" Title="Can we teach machine to tie shoe lace?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4887" PostTypeId="2" ParentId="4864" CreationDate="2017-12-31T09:41:25.500" Score="26" Body="&lt;p&gt;The bottleneck in a neural network is just a layer with less neurons then the layer below or above it. Having such a layer encourages the network to compress feature representations to best fit in the available space, in order to get the best loss during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a CNN (such as Google's Inception network), bottleneck layers are added to reduce the number of feature maps (aka &quot;channels&quot;) in the network, which otherwise tend to increase in each layer. This is achieved by using 1x1 convolutions with less output channels than input channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't usually calculate weights for bottleneck layers directly, the training process handles that, as for all other weights. Selecting a good size for a bottleneck layer is something you have to guess, and then experiment, in order to find network architectures that work well. The goal here is usually finding a network that generalises well to new images, and bottleneck layers help by reducing the number of parameters in the network whilst still allowing it to be deep and represent many feature maps.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2017-12-31T09:41:25.500" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4888" PostTypeId="2" ParentId="4886" CreationDate="2017-12-31T17:05:23.950" Score="6" Body="&lt;p&gt;You would not believe how difficult this task is, assuming you want a humanoid robot to tie laces with humanoid hands. It’s possible of course but compared to what the current state of the art in machine learning is, this task is very very complex because it’s a physical system with unknown variables (Eg. coefficient of friction on laces), physical limitations (Eg. dexterity of robotic hands), and occluded vision (Eg. hand in front of laces) to name a couple of the issues one faces not to mention robots are expensive but I digress. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Robots can perform amazing preprogrammed sequences but if you wanted to have a robot to be able to tie any shoe in any situation, machine learning is the right approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training data should include all of the things that humans use when they approach this problem: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;proprioception (the location of joints in space aka where are your fingers)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a sense of touch, a robot would need a quite a few touch sensors&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;vision, although not actually necessary (because blind humans are capable of learning to do this task), it may speed up learning&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using reinforcement learning as a framework, one needs to be able to give the RL agent a signal that it is closer to tying the lace. This could be done in a few ways and it’s unclear what would be the best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One method would be to train a separate model using supervised learning and pictures of shoes laced up to look at the current state of the bow (or lack thereof) and report if it looks like a tight bow. This method requires a whole other network but it may be the most versatile in the end although the RL agent may just learn how to place laces so it looks like a tight bow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although simple for humans, this is an interesting problem and thinking about how it could be implemented helps one understand how to apply ML in other areas. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-01-05T00:57:14.577" LastActivityDate="2018-01-05T00:57:14.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4889" PostTypeId="1" CreationDate="2017-12-31T18:04:41.053" Score="12" ViewCount="7462" Body="&lt;p&gt;For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class &amp;quot;not a digit&amp;quot;, so that any letter, any other type of image, or random noise would be classified as &amp;quot;not a digit&amp;quot;. Similarly, with CIFAR-10, I want to have the 11th &amp;quot;unknown&amp;quot; class to classify any image that contains something out of the available 10 classes of CIFAR-10.&lt;/p&gt;&#xA;&lt;p&gt;So, how to implement such a feature? Maybe there are some examples somewhere, preferable with Keras.&lt;/p&gt;&#xA;" OwnerUserId="11845" LastEditorUserId="2444" LastEditDate="2021-01-16T20:37:28.810" LastActivityDate="2021-01-16T20:37:28.810" Title="How to implement an &quot;unknown&quot; class in multi-class classification with neural networks?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;implementation&gt;&lt;multiclass-classification&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4890" PostTypeId="1" AcceptedAnswerId="4893" CreationDate="2017-12-31T18:47:23.170" Score="2" ViewCount="104" Body="&lt;p&gt;Machine learning models and, in particular, neural networks are trained with data often collected from the real world, such as images of real people.&lt;/p&gt;&#xA;&lt;p&gt;Meanwhile, neural networks (such as GANs) are also used for data generation. Each year, they become better at this task to the point that even humans are not able to distinguish real-world data from the artificially generated one.&lt;/p&gt;&#xA;&lt;p&gt;So it is possible that neural networks will start to learn with data that was generated by other neural networks, because it will look as real even for a human, but naturally will be not related to the real world.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Will it lead to some machine learning collapse?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Might it lead to some changes in human's perception of the world, because people get a very big part of their knowledge using computers, connected to the Internet?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is anyone thinking about this potential problem?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11845" LastEditorUserId="2444" LastEditDate="2021-01-17T18:57:54.837" LastActivityDate="2021-01-17T19:14:39.583" Title="What are the possible social consequences of training neural networks with artificially generated data?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;social&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4891" PostTypeId="2" ParentId="4574" CreationDate="2017-12-31T18:52:50.760" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This approach sounds great but is it practical? How much effort and zetabytes of data do we have to keep just to reach the brain power of, say, a 3-year old, who can recognize dogs, cats, tigers, a Mustang, trucks, a hamburger restaurant, and so on?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We don't know. More critically, we don't know if any technique of machine learning is &quot;the way humans learn&quot;, thus we have no theoretical guarantee that what we are doing brings us any closer to true AI. Also, there is much more to intelligence that classification don't forget about that.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My next question is why does everyone have to repeat the effort of learning the same things?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Nowadays people often train their architecture on the &lt;a href=&quot;http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;same few datasets&lt;/a&gt; as a benchmark to compare it with other cutting edge techniques. These datasets are also generally well understood, thus its easier to troubleshoot issues that might come up.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If Google has already learned cats, or if someone already has a program to recognize handwritten digits, can this knowledge be shared and re-used? Or is it just a matter of paying for them?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In principle yes, but likely no. Many modern machine learning techniques have separate training/testing phases. Training is when the algorithm gets to &quot;learn&quot; per say, and testing is when we get to test it. If one releases the architecture after it has been trained, in principle you should be able to achieve the same results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or more generally, once they release a paper detailing how they built the system, in principle you should be able to build it yourself as well. &lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-31T18:52:50.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4892" PostTypeId="1" CreationDate="2017-12-31T19:14:13.233" Score="1" ViewCount="78" Body="&lt;p&gt;Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer &lt;code&gt;l&lt;/code&gt;, in a neural network to particular nodes in the previous layer &lt;code&gt;l-1&lt;/code&gt; of the neural network as well as to particular nodes in the next layer &lt;code&gt;l+1&lt;/code&gt; of the neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is obviously contrived, but here is a illustration of what I mean.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Vy7FL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vy7FL.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The thick line with arrow indecates an edge that leads to l-1 layer neuron&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="11893" LastEditDate="2018-01-01T23:05:16.293" LastActivityDate="2018-01-02T15:17:08.087" Title="Algorithms that connect neurons to previous layers as well as next" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4893" PostTypeId="2" ParentId="4890" CreationDate="2017-12-31T21:11:28.917" Score="3" Body="&lt;p&gt;We can already observe &lt;a href=&quot;https://www.nbcnews.com/better/lifestyle/problem-social-media-reinforcement-bubbles-what-you-can-do-about-ncna1063896&quot; rel=&quot;nofollow noreferrer&quot;&gt;information bubbles on social media&lt;/a&gt;, where the circle is that the ML algorithms learn what content people like and give more similar content based on clicks and so on. From a single wrong click, you could enter a bubble and never come out if you don't take care or be aware.&lt;/p&gt;&#xA;&lt;p&gt;This happens with humans, so the same may apply to computers. Checking backgrounds, like click saver for click titles as an ultimate example, would save the machines from circles too. Information is not different whether it is made in consecutive rounds by humans or machines.&lt;/p&gt;&#xA;&lt;p&gt;For example, if you make a thesis, not necessarily all things you write are your own investigation and research. You use quotes, refer to papers, and so on. For information sources, you have to be careful if the source is respected and considered to be meaningful and correct.&lt;/p&gt;&#xA;&lt;p&gt;The same carefulness needs to be applied to machine-generated content. To the responsible - the last responsibility is with the reader to believe or not what you're reading.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="2444" LastEditDate="2021-01-17T19:09:37.807" LastActivityDate="2021-01-17T19:09:37.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4894" PostTypeId="2" ParentId="4868" CreationDate="2017-12-31T21:34:54.510" Score="1" Body="&lt;p&gt;In general no. The research area that you seem to be looking for is called generalization, which is very much so still an active area of research. Actual architecture design strongly depends on the dataset itself and available resources, and thus there isn't a general rule of thumb that works for every case.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2017-12-31T21:34:54.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4895" PostTypeId="2" ParentId="4889" CreationDate="2018-01-01T13:30:01.833" Score="3" Body="&lt;p&gt;The usual way to implement this would be to add the new class with data examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some things you need to address:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Sourcing new data for your &quot;other&quot; class. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ensuring the amount and variation of data in &quot;other&quot; class examples matches how the predictor will be used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Code examples for this are not necessary, as you would just use the same network design as you already have and just add another output. This is a data and model definition problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logically you have another option: As well as outputting the predicted class, you predict separately whether there is any detectable object at all as a true/false value. This still requires the additional data, but is for example how the YOLO algorithm works for object detection. Object detection has a specific meaning - it involves finding the co-ordinates and class of possibly multiple objects in an image. This goes beyond the wording of your question, but is a typical end goal if you are asking this kind of question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;YOLO predicts the presence of an object separately from the class of object. The additional data for YOLO training comes from segmenting the source images, so many parts of the target image are background with no objects. In that case the additional data you require is due to more detailed labelling within each image example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;YOLO is quite complicated architecture, so you might want to look at &lt;a href=&quot;https://github.com/experiencor/basic-yolo-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example using Keras on a Github project&lt;/a&gt; for more details, if object detection is your goal.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-01-01T13:35:07.220" LastActivityDate="2018-01-01T13:35:07.220" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4897" PostTypeId="2" ParentId="4705" CreationDate="2018-01-01T18:18:56.117" Score="5" Body="&lt;p&gt;AI is a wonderful field to get into. Not only is it in high demand in the job market, it also helps you perceive the world in a whole new way. It's great that you have a deep interest in AI. In my opinion, you'll progress faster if you are having fun. Learning is always accelerated when you are curious and deeply interested in a particular domain or technology. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, begin by building your foundations, and then later you can specialize in specific areas of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a beginner it's not advisable for you to just dive deep into every machine learning topic, you could easily get discouraged. There's too much ground to cover and the field is progressing rapidly. Start by learning the foundational concepts. Focus on one thing for now and learn the concepts really well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are the basic topics that I would suggest you pick up as you begin your journey towards proficiency in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learn and refresh in math ( linear algebra, numerical optimization, and differential calculus), probability and statistics: I cannot stress enough how important basic statistical concepts are in understanding AI. Statistics will help you grasp many of the algorithms used such as linear regression and classification. Matrix Algebra, on the other hand, is important when you start manipulating large amounts of data. Additionally, you need at least a basic grasp of calculus. Important concepts such as gradient descent cannot be understood without at least a basic comprehension of calculus. Regarding materials to learn these courses I recommend that you download Stanford University's Elements of Statistical Learning. &lt;a href=&quot;https://web.stanford.edu/~hastie/Papers/ESLII.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/~hastie/Papers/ESLII.pdf&lt;/a&gt; and consider enrolling in Khan academy's differential calculus course. &lt;a href=&quot;https://www.khanacademy.org/math/calculus-home/differential-calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.khanacademy.org/math/calculus-home/differential-calculus&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming: Programming is key in order to successfully implement learning algorithms. Its great that you know Java and C++, as long as you get the concepts (variables, loops, objects, methods and data structures) picking up other languages becomes easier. I advise that you pick up Python since it enables fast and natural expression of mathematical equations. It also has a fantastic ecosystem, there is virtually a Python package for almost any conceivable math function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, before you pick up Python. There are a number of ML libraries for Java which can leverage on your experience with the language, such as WEKA, MOA, Deeplearning4j, Stanford coreNLP, Apache Singa, Mallet, and Elki. These are super useful Java libraries which can be applied directly to many problems in the field. I advise you to sign up for Code Academy which is a platform where you can learn to code interactively. &lt;a href=&quot;https://www.codecademy.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.codecademy.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will then need to familiarise yourself with basic Learning Algorithms: There are numerous algorithms in Machine Learning which are important to one problem or the other. There is no single model that works best for all problems. Familiarizing yourself with Linear Regression, Support Vector Machines, Dimensionality reduction, Gaussian Processes, Naive Bayes, Decision Tree's and K-Means algorithms will definitely prove useful in the long run. I recommend you pick up the freely available book Deep Learning by Yoshua Bengio and Ian Goodfellow for this &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.deeplearningbook.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To have a competitive edge be sure to invest in 'theory': A great grasp of the theoretical topics that surround AI (i.e. the theory of computation, signal processing, and the artificial neuron) lays a strong foundation for a lot of the abstract areas of AI. You also need to be well versed in AI theory in order to keep up with new developments in the field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practice: Consider getting started on some simple AI projects i.e. a chatbot, handwriting recognition or a tic-tac-toe bot. I recommend that you post your code on Github so that others see your progress and contribute new ideas on how you can advance forward. Kindly follow Siraj Rawal's GitHub account for interesting AI project ideas  &lt;a href=&quot;https://github.com/llSourcell?tab=repositories&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/llSourcell?tab=repositories&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Get a mentor: A mentor will be glad to assist you to progress in your understanding of the field as they themselves have been through your shoes so they understand your needs before you even ask. Not only will a mentor help explain advanced concepts, they will breakdown jargon and correct your mistakes. You can sign up to Machine Learning Mentor to link up with a mentor in your AI studies &lt;a href=&quot;http://www.machinelearningmentor.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.machinelearningmentor.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the best&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="11498" LastEditDate="2018-01-04T17:10:08.263" LastActivityDate="2018-01-04T17:10:08.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4898" PostTypeId="2" ParentId="4574" CreationDate="2018-01-01T22:05:14.923" Score="2" Body="&lt;p&gt;&lt;em&gt;Disclosure: I am a product manager on Google Cloud Platform.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] why does everyone have to repeat the effort of learning the same things?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If Google has already learned cats, or if someone already has a program to recognize handwritten digits, can this knowledge be shared and re-used? Or is it just a matter of paying for them?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You don't have to rebuild these machine learning models from scratch; you can reuse prebuilt machine learning algorithms, e.g., Google Cloud provides the following hosted APIs as a service:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cloud.google.com/vision/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Cloud Vision API&lt;/a&gt; to recognize images and provide text labels&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cloud.google.com/speech/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Cloud Speech API&lt;/a&gt; to convert speech to text&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cloud.google.com/natural-language/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Cloud Natural Language API&lt;/a&gt; to understand language, find entities, etc.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cloud.google.com/products/machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;and more...&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can put these APIs together to build interesting applications, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=HpPyhsC4q9M&quot; rel=&quot;nofollow noreferrer&quot;&gt;build a smart Raspberry Pi bot with Cloud Vision and Speech APIs&lt;/a&gt; (video presentation; &lt;a href=&quot;https://www.raspberrypi.org/blog/raspberry-pi-cloud-vision-google-io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post + video overview&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1632" LastEditorUserId="1632" LastEditDate="2018-01-01T22:20:02.853" LastActivityDate="2018-01-01T22:20:02.853" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4901" PostTypeId="2" ParentId="4780" CreationDate="2018-01-02T16:14:15.443" Score="1" Body="&lt;p&gt;AI  training and readiness within your workplace&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First and foremost, working within a company is definitely a team sport. You might have embraced the right attitude and trained yourself on AI adoption, but most likely your co-workers have not. Here lies the first opportunity. I suggest you take the initiative to organise for expert talks, workshops and trainee-ships on AI and Machine Learning so that your organisation can posses the necessary skills it needs to implement AI across all departments. My advice is that you co-ordinate with managers within your organisation to provide employees with the opportunity to pursue learning and training programs to enhance their careers and help them understand the applications of AI within the workplace. Coursera have an attractive package for enterprises that I would recommend for this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cyber Security&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hackers have adopted increasingly more sophisticated methods in a bid to compromise sensitive data, intellectual property, digital assets or any aspect that can harm an organisations reputation or business continuity. Last year major organisations such as NIH (UK) and WPP reported that they had been struck by a major ransomware cyber-attack! Cyber security is one of the niche's that can benefit immensely by leveraging AI. Deep learning has been used successfully to detect the red flags of a cyber attack. My advice to you as a software engineer is that you consider looking at algorithms that can safeguard against network intrusion in your company and implement them. Alternatively, you can adopt cutting edge security software from startups such as deep instinct, BluVector and Splunk. This will put your company at the forefront of network security. Here is a link to a collection of important papers in AI cyber security that can be relevant for the above &lt;a href=&quot;https://medium.com/@jason_trost/collection-of-deep-learning-cyber-security-research-papers-e1f856f71042&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@jason_trost/collection-of-deep-learning-cyber-security-research-papers-e1f856f71042&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI Schedule Assistant&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a work setting employees often spend a lot of time coordinating meetings and phone calls. This task can be time-consuming and tedious. Sending emails back and forth, trying to find a convenient time for each person can take time away from more critical duties. This is why I believe you should consider implementing an AI schedule assistant to automate the task of scheduling meetings and appointments within your workplace. The startup X.ai can be a good starting point. However if you need a solution that provides you with more control over the source code and that can be installed on the hardware and OS's that you are already using at your workplace I then highly recommend that you have a look at mycroft and lucida which are both impressive AI virtual assistants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI &amp;amp; Regulatory Compliance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many companies the legal and compliance staff struggle to understand the regulatory requirements that they face within their line of business. As markets evolve, businesses can end up having to comply with thousands of regulations from different regulators. In addition to the above complexities, regulations change and their interpretation changes within the course of time. The good news is that AI technologies are promising to change the way firms manage their compliance and regulatory risk. These solutions use NLP (Natural Language Processing) to parse regulatory text and pattern match it with particular regulatory keywords relevant to the organisation. You could introduce your company to solutions from startups i.e. compliance.ai, such products will enable management to demonstrate compliance to regulators resulting in reduced compliance failures and regulatory fines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chatbots for Business&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statistics show that a majority of Apps are used only once and uninstalled. Instead of developing a smartphone app for your workplace why don't you upsell the advantages of building an AI powered chatbot? You could build a chatbot that could live in any major chat application i.e. FB messenger, Slack or WhatsApp. There are a number of services i.e. Chatfuel, Botsify, Flow XO and Beep Bop to host your chatbot. You could train the conversational model using Seq2Seq on tensorflow. &#xA;Chatbots not only offer a better customer support experience, your company will soon find out that chatbots allow for significant cost savings in their call centers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI for Content Generation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In today's workplace there is a need to create content at scale. Business blogging for example helps boost SEO, it drives traffic to your website and helps convert that traffic into leads. Additionally, a business needs to generate press releases, internal memos and legal documents. Although AI won't take over writing assignments completely, it can be used alongside content writers to improve productivity. You can use LSTM RNN to build a content generation platform for your company. Such a platform can help tackle monotonous tasks while improving efficiency. &lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-02T16:14:15.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4903" PostTypeId="5" CreationDate="2018-01-02T22:15:54.013" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-02T22:15:54.013" LastActivityDate="2018-01-02T22:15:54.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4904" PostTypeId="4" CreationDate="2018-01-02T22:15:54.013" Score="0" Body="Use for questions related to the social impacts of AI." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-02T22:15:54.013" LastActivityDate="2018-01-02T22:15:54.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4905" PostTypeId="2" ParentId="4890" CreationDate="2018-01-02T22:41:50.427" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Will it lead to some machine learning collapse?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I wouldn't think so.  Data is data.  From the standpoint of automata, everything is ultimately reduced to a sting of bits.  It may even be useful to be able to train AI's using CGI, for instance, in relation to automated vehicles. Not any different from humans using flight simulators.&lt;/p&gt;&#xA;&lt;p&gt;Creating models and training AIs on them is useful, and a part of the contemporary AI landscape.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Might it lead to some changes in human's perception of the world, because people get a very big part of their knowledge using computers, connected to the Internet?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It already is. It's not only the false CGI content, but the scope of the search filter that dictates what information a websurfer gets. These results are controlled by algorithms, which evolve.  Self-evolving algorithms may make the process more opaque. It definitely seems to be creating social problems already.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is anyone thinking about this potential problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm sure there are papers out there on this subject.  (Don't have time to search now, but I may do that and come back and amend with some articles and research papers.)&lt;/p&gt;&#xA;&lt;p&gt;Two authors who are definitely thinking about this are &lt;a href=&quot;https://en.wikipedia.org/wiki/Neal_Stephenson#Life&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neal Stephenson&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Hannu_Rajaniemi#Early_life&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hannu Rajaneimi&lt;/a&gt;.  Stephenson addressed the &amp;quot;information unreliability&amp;quot; problem of the internet in &lt;a href=&quot;https://en.wikipedia.org/wiki/Anathem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anathem&lt;/a&gt;. &lt;em&gt;(It's not a major theme, but his ideas are quite insightful--Stephenson has a hard-science background, with a particular interest in computing.)&lt;/em&gt;  Rajaniemi extends the ideas in the post-singularity &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Quantum_Thief&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Thief trilogy&lt;/a&gt;, where information and matter are interchangeable, and contains some very interesting ideas.  &lt;em&gt;(Rajaneimi holds two advanced mathematics degrees, which is useful in tackling a subject of such great complexity.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-17T19:14:39.583" LastActivityDate="2021-01-17T19:14:39.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4907" PostTypeId="1" CreationDate="2018-01-03T01:49:59.060" Score="5" ViewCount="169" Body="&lt;p&gt;The basis of my question is that a CNN that does great on MNIST is far smaller than a CNN that does great on ImageNet. Clearly, as the number of potential target classes increases, along with image complexity (background, illumination, etc.), the network needs to become deeper and wider to be able to sufficiently capture all of the variation in the dataset. However, the downside of larger networks is that they become far slower for both inference and backprop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you wanted to build a network that runs on a security camera in front of your house. You are really interested in telling when it sees a person, or a car in your driveway, or a delivery truck, etc. Let's say you have a total of 20 classes that you care about (maybe you want to know minivan, pickup, and so on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You gather a dataset that has plenty of nice, clean data. It has footage from lots of times of the day, with lots of intra-class variation and great balance between all of the classes. Finally, assume that you want this network to run at the maximum possible framerate (I know that security cameras don't need to do this, but maybe you're running on a small processor or some other reason that you want to be executing at really high speed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any advantage, computationally, to splitting your network into smaller networks that specialize? One possibility is having a morning, an afternoon/evening, and a night network and you run the one corresponding to the time of day. Each one can detect all 20 classes (although you could split even farther and make it so that there is a vehicle one, and a person one, and so on). Your other option is sharing base layers (similar to using VGGNet layers for transfer learning). Then, you have the output of those base layers fed into several small networks, each specialized like above. Finally, you could also have just one large network that runs in all conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question: Is there a way to know which of these would be faster other than building them?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my head, it feels like sharing base layers and then diverging will run as slow as the &quot;sub-network&quot; with the most additional parameters. Similar logic for the separate networks, except you save a lot of computation by sharing base layers. Overall, though, it seems like one network is probably ideal. Is there any research/experimentation along these lines?&lt;/p&gt;&#xA;" OwnerUserId="8829" LastActivityDate="2021-03-11T20:04:20.393" Title="Is one big network faster than several small ones?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4908" PostTypeId="1" CreationDate="2018-01-03T05:51:20.047" Score="1" ViewCount="35" Body="&lt;p&gt;Say I have 500 variables and I believe those variables can be shown in a 4-dimensional latent representation which I want to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have for training is 100K samples, and those samples are coming mainly from 3 unbalanced groups: 1st group has 1K samples, 2nd group has 49K samples, and 3rd group has 50K samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think I can learn a meaningful representation by training a (variational) autoencoder with this data? Is there a reason that requires all samples to come from the same distribution? If not, is there a reason that requires balanced classes?&lt;/p&gt;&#xA;" OwnerUserId="9609" LastEditorUserId="9609" LastEditDate="2018-01-09T07:08:52.287" LastActivityDate="2018-01-09T07:08:52.287" Title="Does it make sense to train an autoencoder using data from different distributions?" Tags="&lt;neural-networks&gt;&lt;unsupervised-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4909" PostTypeId="2" ParentId="4332" CreationDate="2018-01-03T06:29:05.930" Score="2" Body="&lt;p&gt;Typically what you probably would want to do is do the training on something other than a Raspberry Pi. I think for what you're wanting to accomplish with having a computer talk back to your parrot, you won't need anything too crazy with a bunch of GPU's - but I don't think you'll want to necessarily do the training on a Pi either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some questions I have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are you going to plat back to the parrot? i.e. are you going to play it back random parrot sounds you found online or sounds that you've recorded? are you going to play it back what it just said? are you going to play it back a modification of what it just said?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you want it to respond to the parrot anytime the parrot speaks? Or when the parrot &quot;says&quot; something specific?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think depending on the answers to those two questions, there are a couple of different paths that you could go down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the hardware of the Raspberry Pi itself, I have never done any speech recognition with it, but I have done image recognition with it via the &lt;a href=&quot;https://www.movidius.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Movidius Neural Compute Stick&lt;/a&gt; which according to &lt;a href=&quot;https://www.quora.com/Can-the-Movidius-Neural-Compute-stick-accelerate-models-that-aren%E2%80%99t-vision-such-as-speech-recognition-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Quora post&lt;/a&gt;, may be able to be used to offload some of the processing &quot;relatively easily&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some other links you may find valuable:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Tensorflow speech recognition: &lt;a href=&quot;https://github.com/pannous/tensorflow-speech-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pannous/tensorflow-speech-recognition&lt;/a&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The YouTube video that the above repo references: &lt;a href=&quot;https://www.youtube.com/watch?v=u9FPqkuoEJ8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=u9FPqkuoEJ8&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;How to control a raspberry pi with your voice: &lt;a href=&quot;http://www.techradar.com/how-to/how-to-control-the-raspberry-pi-with-your-voice&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.techradar.com/how-to/how-to-control-the-raspberry-pi-with-your-voice&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Voice recognition software setup on Raspberry Pi: &lt;a href=&quot;https://diyhacking.com/best-voice-recognition-software-for-raspberry-pi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://diyhacking.com/best-voice-recognition-software-for-raspberry-pi/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11667" LastActivityDate="2018-01-03T06:29:05.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4910" PostTypeId="1" AcceptedAnswerId="6520" CreationDate="2018-01-03T10:48:13.130" Score="4" ViewCount="568" Body="&lt;p&gt;I've seen data sets for classification / regressions tasks in domains such as credit default detection, object identification in an image, stock price prediction etc.  All of these data sets could simply be represented as an input matrix of size (n_samples, n_features) and fed into your machine learning algorithm to ultimately yield a trained model offering some predictive capability.  Intuitively and mathematically this makes sense to me.&lt;/p&gt;&#xA;&lt;p&gt;However, I'm really struggling with how to think about the structure of an input matrix for game-like tasks (Chess, Go, &lt;a href=&quot;https://www.youtube.com/watch?v=Ipi40cb_RsI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seth Blings Mario Kart AI&lt;/a&gt;) specifically (using the Chess example):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How would you encode the state of the board to something that a model could train on?  Is it reasonable to think about the board state as a 8x8 matrix (or 1x64) vector with each point being encoded by a numerical value dependent on the type of piece and color?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assuming a suitable representation of the board state, how would the model be capable of making a recommendation given that each piece type moves differently?  Would it not have to evaluate the different move possibilities for each piece and propose which move it &amp;quot;thinks&amp;quot; would have the best long term outcome for the game?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A follow up on 2 - given the interplay between a moves made now and moves made n moves into the future how would the model be able to recognize and make trade-offs between moves which may offer a better position now vs those that offer a position n moves in the future - would one have to extend the board state input to a vector of length 1x64n where n is the total number of moves for expected for an individual player or is this a function of a different algorithm which should be able to capture historical information which training?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am unsure if I'm overthinking this and am missing something really obvious but I would appreciate any guidance in terms of how to approach thinking about this.&lt;/p&gt;&#xA;" OwnerUserId="11933" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-05-27T13:26:14.883" Title="How would you encode your input vector/matrix from a sequence of moves in game like tasks to train an AI? e.g. Chess AI?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;combinatorial-games&gt;&lt;chess&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4911" PostTypeId="1" AcceptedAnswerId="4912" CreationDate="2018-01-03T11:49:19.257" Score="3" ViewCount="56" Body="&lt;p&gt;Suppose I have a classification problem with a stream of training-samples constantly arriving over time. I cannot keep all training-samples in memory, but I still want to train a classifier that will have the &quot;wisdom&quot; of all samples, and additionally, I want the classifier to become better whenever it gets new samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of the following idea. Suppose we have enough memory to keep 100 samples. Then, for each run of 100 samples, we will train a different sub-classifier. We will have a meta-classifier that will classify based on voting between all existing sub-classifiers. Over time, we will have more and more sub-classifiers, so hopefully the meta-classifier will improve with time - it will have a &quot;wisdom of the crowds&quot; effect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this method been tried before? Specifically, has it been tried in a deep-learning sequence-classification setting?&lt;/p&gt;&#xA;" OwnerUserId="8684" LastActivityDate="2018-01-03T17:22:04.860" Title="Creating a classifier for simpler classifiers trained on few training samples" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4912" PostTypeId="2" ParentId="4911" CreationDate="2018-01-03T14:13:03.930" Score="2" Body="&lt;p&gt;The voting technique that you described is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Ensemble_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensemble Learning&lt;/a&gt; and its improvement over time is guaranteed if each classifier is at least a little better than random.&lt;/p&gt;&#xA;" OwnerUserId="11921" LastEditorUserId="1671" LastEditDate="2018-01-03T17:22:04.860" LastActivityDate="2018-01-03T17:22:04.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4913" PostTypeId="2" ParentId="2824" CreationDate="2018-01-03T14:27:18.747" Score="1" Body="&lt;p&gt;The important part, where you can see a single reward value is used for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; different updates, is the part where a sum of &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; values with &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; ranging from &lt;span class=&quot;math-container&quot;&gt;$\tau + 1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\tau + n$&lt;/span&gt; is assigned to &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, the outer loop of the algorithm always does at most one update per iteration, but for that update it uses multiple previously observed &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; values. Each of those &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; values is used for multiple updates (not multiple updates at the same time, but multiple updates spread out over different iterations).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorDisplayName="user1440" LastEditDate="2018-11-26T15:20:23.920" LastActivityDate="2018-11-26T15:20:23.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4914" PostTypeId="1" CreationDate="2018-01-03T14:31:20.780" Score="2" ViewCount="37" Body="&lt;p&gt;I was wondering any examples of the following;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;para generation: For eg, given X similar paragraphs,  are you able to build a model to learn the style and generate a new para that is a paraphrase of the X paras. Similar in meaning but diff wording.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;drawing conclusions from X given articles. He has a list of conclusions, check the X articles can provide evidence to the conclusions. Eg, given conclusion “city is not safe”, look for evidence such as “murders” and “thefts”.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Glady Appreciate,&#xA;Betty&lt;/p&gt;&#xA;" OwnerUserId="11930" LastActivityDate="2018-01-03T19:25:03.207" Title="Para Generation and Drawing Conclusion from X give Articles" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4915" PostTypeId="2" ParentId="4914" CreationDate="2018-01-03T19:00:00.473" Score="1" Body="&lt;p&gt;Paraphrase generation would be made by an &lt;a href=&quot;https://rare-technologies.com/text-summarization-in-python-extractive-vs-abstractive-techniques-revisited/#abstractive_text&quot; rel=&quot;nofollow noreferrer&quot;&gt;abstractive text summarization tool&lt;/a&gt; like Tensorflow as described in &lt;a href=&quot;https://research.googleblog.com/2016/08/text-summarization-with-tensorflow.html?m=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's blog&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Abstractive tool summarizes the text but also it adds extra words to make text more human like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the second problem scenario I would say the answer depends quite much what you mean by the conclusions and how you think the conclusions would be made. I strongly believe this falls in the category of &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificial general intelligence&lt;/a&gt;, which is emerging area of AI, supported by association &lt;a href=&quot;http://www.agi-society.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI&lt;/a&gt;, where the focus is on strong artificial intelligence and more human like features of AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging about conclusions at least needs a vast knowledge base of the semantics of different words and pieces of text in general combined to possible conclusions and their legitimate implications. &lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2018-01-03T19:25:03.207" LastActivityDate="2018-01-03T19:25:03.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4917" PostTypeId="1" CreationDate="2018-01-03T20:23:06.233" Score="3" ViewCount="131" Body="&lt;p&gt;The Intel 8080 had 4500 transistors and ran at 2-3.125 MHz. By comparison, the 18-core Xeon Haswell-E5 han 5,560,000,000 transistors and can run at 2 GHz. Would it be possible or prudent to simulate a neural network by backing a chip chock-full of a million interconnected, slightly modified intel 8080s (sped up to run at 2 GHz)? If each one modeled 100 neurons you could simulate a neural network with 100 million neurons on a single chip.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I'm not proposing that you &lt;em&gt;actually&lt;/em&gt; use a million intel 8080s; rather I'm proposing that you take a highly minimal programmable chip design &lt;em&gt;like&lt;/em&gt; the intel 8080's design and pattern it across a wafer as densely as possible with interconnects so that each instance can function as one or a few dozen fully programmable neurons each with a small amount of memory. I'm &lt;em&gt;not&lt;/em&gt; proposing that someone take a million intel 8080s and hook them together. &lt;/p&gt;&#xA;" OwnerUserId="3323" LastEditorUserId="3323" LastEditDate="2018-01-09T02:55:58.890" LastActivityDate="2019-01-22T04:32:32.657" Title="Could a large number of interconnected tiny turing-complete computer chips be patterned across a wafer to simulate a programmable neural network?" Tags="&lt;artificial-neuron&gt;&lt;hardware&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4918" PostTypeId="1" CreationDate="2018-01-04T02:06:35.053" Score="1" ViewCount="169" Body="&lt;p&gt;I'm a student, and currently into image processing project and coding using OpenCV. Recently, I watched Sebastian Thrun from Udacity in TedTalks talked about AlphaGo and I'm totally interested in the idea. I have read this question too : &lt;a href=&quot;https://ai.stackexchange.com/questions/4394/why-is-the-merged-neural-network-of-alpha-go-zero-more-efficient-than-two-separa/4654&quot;&gt;Merged Neural Network in AlphaGo&lt;/a&gt;.&#xA;I was wondering if same approaches can be used in my project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to perform color enhancement method for any natural images. And of course, color sampling is a tricky task now. It's a lot of work, I have to prepare condition for each key-color sampling given and also prepare &amp;amp; pick the best enhancement function for it. I'm able to do it already using OpenCV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I was wondering if I could load tons of sample pictures instead, have my system test them against each other, and figure out its own enhancement rules from all testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not that familiar with Deep Learning, we don't even have deep learning course at my university, but I'm interested in the idea and ready to learn. I'm not even sure if this can be done or not, but I wonder what kind of approaches should I learn to achieve my goal ? Is Deep Learning --&gt; Neural Network a good start ? In my case, to which method in Deep Learning should I go with ? Any reference / advice will be highly appreciated. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="11943" LastEditorUserId="1671" LastEditDate="2019-10-15T19:32:03.173" LastActivityDate="2019-10-15T19:32:03.173" Title="Deep Learning Approaches for Color Enhancement Testing" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;getting-started&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4919" PostTypeId="2" ParentId="4432" CreationDate="2018-01-04T09:38:03.213" Score="2" Body="&lt;p&gt;Before we commence I will recommend that you refer to similar questions on the network i.e &lt;a href=&quot;https://stackoverflow.com/questions/9706769/any-tutorials-for-developing-chatbots&quot;&gt;https://stackoverflow.com/questions/9706769/any-tutorials-for-developing-chatbots&lt;/a&gt; , &lt;a href=&quot;https://stackoverflow.com/questions/55042/how-can-i-program-a-simple-chat-bot-ai&quot;&gt;https://stackoverflow.com/questions/55042/how-can-i-program-a-simple-chat-bot-ai&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/1748887/how-to-create-a-smart-chat-bot&quot;&gt;https://stackoverflow.com/questions/1748887/how-to-create-a-smart-chat-bot&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my answer I follow the approach laid out on two outstanding articles. Chatbots with Seq2Seq (Suriyadeepan R 2016) and Deep Learning for Chatbots Part 1 &amp;amp; 2 (Benny B 2016) which are available here &lt;a href=&quot;http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/&lt;/a&gt; and here &lt;a href=&quot;http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/&lt;/a&gt; respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chatbots are also known as conversational agents or dialog systems. Your goal is to build a friendly AI. Modeling personality may appear straight forward, however incorporating such knowledge into a model is still a challenging problem. This is because the models are trained on a lot of data from different sources and users. Below is a link to a paper on building such a Persona-Based Neural Conversation Model &lt;a href=&quot;https://arxiv.org/abs/1603.06155&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1603.06155&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two important concepts that one needs to understand when learning about chatbots. That is the chatbot publishing platform and the chatbot development platform. A chatbot publishing platform is a medium through which the chatbot can be accessed by the users i.e. FB messenger, LINE, Telegram or WhatsApp, on the other hand a chatbot development platform is a tool that can be used to create a chatbot. Chatbot development platforms i.e. Beep Boop, Flow XO, Botsify and Chatfuel can be used to build chatbots without writing any code, this is done by simply using a drag and drop interface. Such development platforms however offer only limited functionality and customizability. (Marutitech blog)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A developer can choose to use a rule/retrieval based approach, here you simply write a pattern and a template. Such a bot replies with one of the templates when it encounters a similar pattern from the user. Rule based models tend to perform poorly when they encounter completely new sentences. (Suriyadeepan Ram 2016) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different approach involves utilizing a generative model. Generative models construct responses word by word based on the query. Because of this the generated responses are prone to grammatical errors. However once they are trained, the generative model outperforms the rule based approach especially in handling previously unseen queries. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In probability and statistics a generative model is a model that generates all values for a phenomenon, both those that can be observed in the world and &quot;target&quot; variables that can only be computed from those observed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For our example below, we will first preprocess the data and then train the generative model using sequence 2 sequence. Seq 2 Seq is a general purpose encoder decoder framework for Tensorflow which can be applied to machine translation, summarizing text ,image captioning and conversational modeling. The seq2seq network connects two RNN's to work together to transform one sequence to another. An encoder network condenses an input sequence into a vector while a decoder network unfolds that vector into a new sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before training a dataset on a generative model we need to perform a preprocessing step called padding. Here we work on converting the variable length sequences into fixed length sequences. Some of the datasets which can be used to train a chatbot using a generative model approach include: the Cornell Movie Dialog Corpus, which is a collection of dialogs from movie scripts and the Ubuntu Dialog Corpus which is based on chat logs from the Ubuntu IRC public channels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step will be to implement the Continuous Bag of Words (CBOW) model, which is a form of word embedding. CBOW is a model of simplifying representation used in NLP. In this step text is represented as a multiset (Bag) of words disregarding grammer but retaining the frequency of words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final step in our model is implementing the Attention Mechanism. This step is very important since it prevents the information loss that could have occured while compressing all the necessary information of a source sentence into a single vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For source code, I recommend implementing Siraj Rawal's Seq 2 Seq Chatbot because of its clear documentation. Its available here &lt;a href=&quot;https://github.com/llSourcell/tensorflow_chatbot&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/llSourcell/tensorflow_chatbot&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-04T09:38:03.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4921" PostTypeId="2" ParentId="4454" CreationDate="2018-01-04T17:59:34.367" Score="3" Body="&lt;p&gt;I recommend you focus on quality over quantity. Publishing a paper will boost your reputation and make you more recognised within your academic field (AI); however, this is only if the paper provides useful insights into an important issue.&lt;/p&gt;&#xA;&lt;p&gt;Your paper is more likely to be accepted if it is well written and easy to understand, stimulates new important questions, uses rigorous methods to explain why the data supports the conclusion and connections to prior work is made and serve to make your paper's arguments clear. (Elizabeth Z Elsevier blog)&lt;/p&gt;&#xA;&lt;p&gt;Before submitting your paper, ask a mentor or a colleague to proofread it, so that you can make the relevant revisions and changes. Journal editors will look down on your work if it is poorly written or contains substandard grammar.&lt;/p&gt;&#xA;&lt;p&gt;A way to get published is by writing reviews, especially for researchers in earlier stages of their careers. Most journal editors like to publish replies to previous publications since it stimulates debate.&lt;/p&gt;&#xA;&lt;p&gt;Remember it is acceptable to challenge reviewers' suggestions with good justification. Many researchers fail to persevere when they are instructed to revise and resubmit their work. Don't give up, however, you can politely decline or even argue why a reviewer is wrong. Editors will accept a rational explanation if it is clear that you have considered all their feedback.&lt;/p&gt;&#xA;&lt;p&gt;Getting published is never easy, especially in high ranking journals. If you focus on getting published quickly it could derail you from concentrating on the quality of your research. Yes, getting published can be expensive, however, it's much better for your career if you write a high-quality paper than a low-quality paper in a lowly ranked or ungraded journal since it will not be REFable.&lt;/p&gt;&#xA;&lt;p&gt;Below is a list of Artificial Intelligence Journals that you can submit your papers to and possibly get published.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Artificial Intelligence (Journal) &lt;a href=&quot;https://www.journals.elsevier.com/artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.journals.elsevier.com/artificial-intelligence&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Artificial Intelligence Review  &lt;a href=&quot;https://link.springer.com/journal/10462&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/journal/10462&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Applied Artificial Intelligence &lt;a href=&quot;http://www.tandfonline.com/action/journalInformation?journalCode=uaai20&amp;amp;&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.tandfonline.com/action/journalInformation?journalCode=uaai20&amp;amp;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Autonomous Agents and Multi-Agent Systems  &lt;a href=&quot;https://www.springer.com/computer/ai/journal/10458&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.springer.com/computer/ai/journal/10458&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;International Journal on Artificial Intelligence Tools &lt;a href=&quot;http://www.worldscientific.com/worldscinet/ijait&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.worldscientific.com/worldscinet/ijait&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;International Journal of Pattern Recognition and Artificial Intelligence &lt;a href=&quot;http://www.worldscientific.com/worldscinet/ijprai&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.worldscientific.com/worldscinet/ijprai&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Journal of Experimental and Theoretical Artificial Intelligence  &lt;a href=&quot;http://www.tandfonline.com/action/journalInformation?journalCode=teta20&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.tandfonline.com/action/journalInformation?journalCode=teta20&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Journal of Machine Learning Research  &lt;a href=&quot;http://www.jmlr.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.jmlr.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;IEEE Intelligent Systems  &lt;a href=&quot;http://publications.computer.org/intelligent-systems/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://publications.computer.org/intelligent-systems/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Journal of Automated Reasoning  &lt;a href=&quot;https://www.springer.com/computer/theoretical+computer+science/journal/10817&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.springer.com/computer/theoretical+computer+science/journal/10817&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Minds and Machines  &lt;a href=&quot;https://www.springer.com/computer/ai/journal/11023&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.springer.com/computer/ai/journal/11023&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Autonomous Agents and Multi-Agent Systems  &lt;a href=&quot;https://www.springer.com/computer/ai/journal/10458&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.springer.com/computer/ai/journal/10458&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Applied Intelligence  &lt;a href=&quot;https://link.springer.com/journal/10489&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/journal/10489&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Journal of Intelligent Manufacturing  &lt;a href=&quot;https://link.springer.com/journal/10845&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/journal/10845&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Journal of Intelligent Information Systems   &lt;a href=&quot;https://link.springer.com/journal/10844&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/journal/10844&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AI &amp;amp; Society  &lt;a href=&quot;https://link.springer.com/journal/146&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/journal/146&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="2444" LastEditDate="2021-01-31T13:20:59.337" LastActivityDate="2021-01-31T13:20:59.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4922" PostTypeId="2" ParentId="4705" CreationDate="2018-01-04T18:56:34.353" Score="3" Body="&lt;p&gt;I have been programming Java (also sometimes C++) since the 1990s, and wanted to get into AI in September 2017. I spent a lot of time with the Java implementation of TensorFlow, but I finally (after 1 month) gave up and started to learn Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I bought the O'Reilly book &quot;Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems&quot; and went through the examples (all in Python). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, 4 months later, I feel quite comfortable with Python, and understand the AI examples in the book.  Looking back, for me this was by far the fastest way to go. Learning Python took me longer than I thought, but I am happy to use Python now besides Java. &lt;/p&gt;&#xA;" OwnerDisplayName="user11838" LastActivityDate="2018-01-04T18:56:34.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4923" PostTypeId="5" CreationDate="2018-01-04T21:29:17.523" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-management_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-management_(computer_science)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-optimization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-organizing_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-organizing_network&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-04T21:29:17.523" LastActivityDate="2018-01-04T21:29:17.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4924" PostTypeId="4" CreationDate="2018-01-04T21:29:17.523" Score="0" Body="Use for question related to self-optimizing algorithms." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-04T21:29:17.523" LastActivityDate="2018-01-04T21:29:17.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4925" PostTypeId="5" CreationDate="2018-01-04T21:57:42.760" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Neuromorphic_engineering&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1705.06963.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1705.06963.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/nature23011&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/nature23011&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://spectrum.ieee.org/semiconductors/design/neuromorphic-chips-are-destined-for-deep-learningor-obscurity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spectrum.ieee.org/semiconductors/design/neuromorphic-chips-are-destined-for-deep-learningor-obscurity&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://science.energy.gov/~/media/bes/pdf/reports/2016/NCFMtSA_rpt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://science.energy.gov/~/media/bes/pdf/reports/2016/NCFMtSA_rpt.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nist.gov/news-events/events/2017/11/introduction-neuromorphic-computing-why-it-so-efficient-pattern&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nist.gov/news-events/events/2017/11/introduction-neuromorphic-computing-why-it-so-efficient-pattern&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.humanbrainproject.eu/en/silicon-brains/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.humanbrainproject.eu/en/silicon-brains/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/group/brainsinsilicon/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/group/brainsinsilicon/about.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/526506/neuromorphic-chips/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.technologyreview.com/s/526506/neuromorphic-chips/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.forbes.com/sites/adrianbridgwater/2018/01/03/neuromorphic-computing-will-build-human-like-machine-brains/#3c47c0d37124&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.forbes.com/sites/adrianbridgwater/2018/01/03/neuromorphic-computing-will-build-human-like-machine-brains/#3c47c0d37124&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wired.co.uk/article/ai-neuromorphic-chips-brains&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wired.co.uk/article/ai-neuromorphic-chips-brains&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-04T21:57:42.760" LastActivityDate="2018-01-04T21:57:42.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4926" PostTypeId="4" CreationDate="2018-01-04T21:57:42.760" Score="0" Body="Use for questions related to use of electronic circuits to mimic neuro-biological architectures present in the nervous system or in neural networks." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-04T21:57:42.760" LastActivityDate="2018-01-04T21:57:42.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4929" PostTypeId="1" CreationDate="2018-01-06T01:31:03.133" Score="1" ViewCount="268" Body="&lt;p&gt;I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch size &gt; 1 (even with only 4 samples!). In the code below, the loss value reaches 0 when I set BatchSize to 1. However, by setting BatchSize to 2, the network fails to overfit and the loss value goes toward 12.500000 and gets stuck there forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect this has something to do with LSTM states. I get the same problem if I don't update the state with each iteration. Or maybe the cost function? A help is appreciated. Thanks.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;import numpy as np&#xA;import os&#xA;&#xA;from utils import loadData&#xA;&#xA;Epochs = 10000&#xA;LearningRate = 0.0001&#xA;MaxGradNorm = 5&#xA;&#xA;SeqLen = 1&#xA;NChannels = 28&#xA;NClasses = 1&#xA;&#xA;NLayers = 2&#xA;NUnits = 256&#xA;&#xA;BatchSize = 1&#xA;NumSamples = 4&#xA;#################################################################&#xA;&#xA;trainingFile = &quot;./training.dat&quot;&#xA;&#xA;X_values, Y_values = loadData(trainingFile, SeqLen, NumSamples)&#xA;&#xA;X = tf.placeholder(tf.float32, [BatchSize, SeqLen, NChannels], name='inputs')&#xA;&#xA;Y = tf.placeholder(tf.float32, [BatchSize, SeqLen, NClasses], name='labels')&#xA;&#xA;keep_prob = tf.placeholder(tf.float32, name='keep')&#xA;&#xA;initializer = tf.contrib.layers.xavier_initializer()&#xA;&#xA;Xin = tf.unstack(tf.transpose(X, perm=[1, 0, 2]))&#xA;&#xA;lstm_layers = []&#xA;&#xA;for i in range(NLayers):&#xA;&#xA;    lstm_layer = tf.nn.rnn_cell.LSTMCell(num_units=NUnits, initializer=initializer, use_peepholes=True, state_is_tuple=True)&#xA;&#xA;    dropout_layer = tf.contrib.rnn.DropoutWrapper(lstm_layer, output_keep_prob=keep_prob)&#xA;&#xA;    #[LSTM ---&amp;gt; DROPOUT] ---&amp;gt; [LSTM ---&amp;gt; DROPOUT] ---&amp;gt; etc...&#xA;    lstm_layers.append(dropout_layer)   &#xA;&#xA;rnn = tf.nn.rnn_cell.MultiRNNCell(lstm_layers, state_is_tuple=True)&#xA;&#xA;initial_state = rnn.zero_state(BatchSize, tf.float32)&#xA;&#xA;outputs, final_state = tf.nn.static_rnn(rnn, Xin, dtype=tf.float32, initial_state=initial_state)&#xA;&#xA;outputs = tf.transpose(outputs, [1,0,2])&#xA;outputs = tf.reshape(outputs, [-1, NUnits])&#xA;&#xA;weight = tf.Variable(tf.truncated_normal([NUnits, NClasses]))&#xA;bias = tf.Variable(tf.constant(0.1, shape=[NClasses]))&#xA;prediction = tf.matmul(outputs, weight) + bias&#xA;prediction = tf.reshape(prediction, [BatchSize, SeqLen, NClasses])&#xA;&#xA;cost = tf.reduce_sum(tf.pow(tf.subtract(prediction, Y), 2)) / (2 * BatchSize)&#xA;&#xA;tvars = tf.trainable_variables()&#xA;&#xA;grad, _ = tf.clip_by_global_norm(tf.gradients(cost, tvars), MaxGradNorm)&#xA;&#xA;optimizer = tf.train.AdamOptimizer(learning_rate = LearningRate)&#xA;&#xA;train_step = optimizer.apply_gradients(zip(grad, tvars))&#xA;&#xA;sess = tf.Session()&#xA;&#xA;sess.run(tf.global_variables_initializer())&#xA;&#xA;iteration = 1&#xA;&#xA;for e in range(0, Epochs):&#xA;&#xA;    train_loss = []&#xA;&#xA;    state = sess.run(initial_state)&#xA;&#xA;    for i in xrange(0, len(X_values), BatchSize):&#xA;        x = X_values[i:i + BatchSize]&#xA;        y = Y_values[i:i + BatchSize]&#xA;&#xA;    y = np.expand_dims(y, 2)&#xA;&#xA;        feed = {X : x, Y : y, keep_prob : 1.0, initial_state : state}&#xA;&#xA;        _ , loss, state, pred = sess.run([train_step, cost, final_state, prediction], feed_dict = feed)&#xA;&#xA;        train_loss.append(loss)&#xA;&#xA;        iteration += 1&#xA;&#xA;    print(&quot;Epoch: {}/{}&quot;.format(e, Epochs), &quot;Iteration: {:d}&quot;.format(iteration), &quot;Train average rmse: {:6f}&quot;.format(np.mean(train_loss)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jbmh0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jbmh0.png&quot; alt=&quot;Batch size = 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d7Hmp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d7Hmp.png&quot; alt=&quot;Batch size = 2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11974" LastActivityDate="2018-08-07T15:02:05.900" Title="Tensorflow: Can't overfit training data with batch size &gt; 1" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4932" PostTypeId="2" ParentId="4709" CreationDate="2018-01-06T17:47:08.413" Score="1" Body="&lt;p&gt;Its true that your AI model's performance depends on the quality of data that you use. However, high quality data alone is insufficient to guarantee that your model will learn effectively and score well on a particular dataset. Other factors such as smarter algorithms and the use of high performance computing infrastructure must be factored in for your AI system to perform well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although A.I research has made massive progress in the past decade, ML engineers are yet to build a system that can match the general scope and generalization ability of the human mind. Upto the first decade of the 2000's AI was dominated by expert systems that emulated the decision making ability of an expert. AI at this point couldn't process unstructured data and therefore it lacked the capacity to sit for and pass high school exams. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was until 2011 when IBM Watson a question answering computer system competed against two former Jeopardy quiz show winners and placed first. IBM Watson was built on top of Deep QA  (a computer system that could answer natural language questions) and UIMA (a software achitecture to process and analyse unstructured information). Below is a link to a paper giving an overview of how IBM's Watson works &lt;a href=&quot;https://www.aaai.org/Magazine/Watson/watson.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aaai.org/Magazine/Watson/watson.php&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2012 a team led by Geofrey Hinton won the ImageNet competition by exploiting deep convolution networks. This was soon followed by Dahl's team winning the Merck Molecular Activity Challenge using deep neural network architecture. Yann LeCun's work in CNN's, Geoff Hinton's back propagation and Stochastic Gradient Descent aproach to training datasets alongside Andrew Ng's large scale use of GPU's ignited accelerated progress in ML. This was frequently referred to as unreasonable effectiveness of Deep learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following recent advances in fields such as image captioning, natural language processing, information retrieval and computer vision it is highly probable that current generation AI systems can pass high school exams such as SAT. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Allen AI Institute has made significant progress in developing AI systems that can read, learn and express that understanding through question answering and explanation. Founded by Paul Allen Microsoft's co-founder, the Allen AI Institute's singular focus according to their mission is to conduct high impact research in the field of AI. Below is a news link covering their cognitive system passing high school exams &lt;a href=&quot;http://fortune.com/2015/09/21/computer-artificial-intelligence-math/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fortune.com/2015/09/21/computer-artificial-intelligence-math/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far Allen AI Institute has demonstrated a cognitive platform called Geos that is capable of answering geometry questions as well as the average high school student. While another system called Aristo can answer high school science exam questions by leveraging information extraction alongside knowledge representation and reasoning models. You can access AAI's GeoS service here &lt;a href=&quot;http://allenai.org/euclid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://allenai.org/euclid/&lt;/a&gt; and Aristo here &lt;a href=&quot;http://allenai.org/aristo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://allenai.org/aristo/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile researchers working on the Todai project in Japan have demonstrated a cognitive system that is capable of passing the Tokyo University Mathematics entrance exam. My conclusion from the above examples is that possibly we already have AI that can sit for and pass high school exams. &lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-06T17:47:08.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4936" PostTypeId="2" ParentId="4709" CreationDate="2018-01-07T15:41:33.890" Score="0" Body="&lt;p&gt;I'm thinking that you could write an AI that takes the question as input, weights it, and googles info based on the first layer of neurons, then takes the first two to three pages of results and spits out an answer. It would be a crapshoot, but maybe you could take the list of results, choose one using another layer, choose the info from the page using a third layer, then answering the question using the info.&lt;/p&gt;&#xA;" OwnerUserId="11995" LastActivityDate="2018-01-07T15:41:33.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4940" PostTypeId="5" CreationDate="2018-01-07T21:04:57.780" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Bounded_rationality&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-07T21:04:57.780" LastActivityDate="2018-01-07T21:04:57.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4941" PostTypeId="4" CreationDate="2018-01-07T21:04:57.780" Score="0" Body="The idea that when agents make decisions, their rationality is limited by the tractability of the decision problem, the cognitive limitations of their minds, and the time available to make the decision.  In AI terms this involves processing power and memory, algorithmic efficiency/optimization, and time to make decisions.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-07T21:04:57.780" LastActivityDate="2018-01-07T21:04:57.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4942" PostTypeId="2" ParentId="3548" CreationDate="2018-01-07T21:22:47.333" Score="4" Body="&lt;p&gt;The current, cutting edge AI methods all heavily rely on &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical modeling&lt;/a&gt;.  You might want to browse the &lt;a href=&quot;https://datascience.stackexchange.com/&quot;&gt;Data Science&lt;/a&gt; and &lt;a href=&quot;https://stats.stackexchange.com/&quot;&gt;Cross Validated&lt;/a&gt; stacks to see what people are doing, and the types of maths they are using. &lt;em&gt;(This is not really my field, so I'll leave it to the Neural Network and Deep Learning crowd to provide more detail here.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also strongly recommend looking at &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorics&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial Game Theory&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Complexity Theory&lt;/a&gt;, since one of the the core problems of optimal decision making is &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractability&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game Theory is also important, because the other core issues of optimal decisio making involve &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;imperfect information&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Complete_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;incomplete information&lt;/a&gt;.  &lt;em&gt;(The former is seen more in combinatorial games, but both lead to the need for probability analysis.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-01-07T21:22:47.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4945" PostTypeId="2" ParentId="4917" CreationDate="2018-01-08T09:15:23.833" Score="-1" Body="&lt;p&gt;Theoretically it might be possible but practically it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can argue by using the analogy of a Turing machine. You can say that the Intel 8080 is a turing machine hence it can run any program including a neural network given infinite time and memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inspite of the above you will face insurmountable challenges in implementing your system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CPU's are designed to handle calculations in a sequential manner, most AI algorithms are distributed. You need a GPU (or an AI ASIC) to process the algorithms  in a massively parallel manner for a significant speedup. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally GPU's are excellent at floating point math, floating point arithmetic involves numbers with a variable number of decimal places which are key in running neural networks. For example an Intel core i7 6700k is capable of 200 Giga-FLOPs (floating point operations per seconds) while on the other hand an NVidia GTX 1080 GPU is capable of about 8900 Giga-FLOPs which is a significant difference. (Tyler J 2017)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decide to use the intel 8080 (0.290 MIPS at 2.000 MHz), you will require millions of processors and billions of dollars just to compute at one gigaflop. You can follow this link to see the cost of computing over the years &lt;a href=&quot;https://en.wikipedia.org/wiki/FLOPS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/FLOPS&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem concerns RAM. To efficiently run a neural network you need to fully load it in RAM. It will be a huge challenge to squeeze a neural network in the 64 Kb of RAM that an Intel 8080 processor offers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network bandwidth problem will also be a huge bottleneck. Modern GPU's support high speed technology to communicate between the GPU's. For example NVidia's NVLink has a peak speed of around 80 GBps. While  PCI-E 3.0 runs at around 30 GBps. Without a high speed interconnection bandwidth you will not achieve any speedup inspite of using a distributed system with many processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally you will face significant challenges in programing neural network algorithms for your 8080 processor based system. Most programmers today follow the standards of object oriented programming which enables code reuse, simplified design and maintanance. Besides, OOP languages such as Java, C++ and Python have libraries that significantly simplify the process of programming a neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the 8080 processor was designed back in 1974 OOP had not yet been concieved, they were also using programming tools i.e. compilers that would be considered archaic with todays standards. I mean good luck debugging that system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last but not least, you need big data (or atleast a substantial dataset) to train your neural network on. Without training on a big data set your model will be ineffective. The 8080 supported around 200 Kb of storage. For comparison the MNIST dataset is around 14 GB in size. This means that your processor cannot support the neccessary storage of any ML dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the above reasons my conclusion is that the 8080 processor provides insufficient resources necessary to implement any effective DL algorithm. Networking millions of them together will not provide any substantial speedup for a DL algorithm. &lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-08T09:15:23.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4946" PostTypeId="2" ParentId="4917" CreationDate="2018-01-08T10:04:35.753" Score="0" Body="&lt;p&gt;The building unit of a neural network is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;perceptron&lt;/a&gt;. It cannot be represented by single transistor because it should hold arbitrary (float) value, over multiple computational iterations. (While the transistor is only binary, and does not work as memory on its own.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the strengths of the NN is in it's flexibility, which you would lose if you were to bake it on silicon. In a NN you can vary the:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;number of layers&lt;/li&gt;&#xA;&lt;li&gt;connections between units&lt;/li&gt;&#xA;&lt;li&gt;activation functions&lt;/li&gt;&#xA;&lt;li&gt;and many, many more &lt;a href=&quot;https://en.wikipedia.org/wiki/Meta-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta parameters&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The NNs, once trained on a particular problem, are really fast to make a prediction for a new sample. &lt;strong&gt;The slow and computationally heavy task is the training - and it's during the training that you need flexibility&lt;/strong&gt; to mess with the model and the parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could bake a trained NN model on a chip, if you need the computation time of a prediction to be really fast i.e. in order of nanoseconds (instead of a millisecond or a second on a modern CPU). That will have a significant downside - you won't be able to ever update it with newer NN model.&lt;/p&gt;&#xA;" OwnerUserId="2997" LastActivityDate="2018-01-08T10:04:35.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4947" PostTypeId="2" ParentId="4215" CreationDate="2018-01-08T11:36:56.540" Score="0" Body="&lt;p&gt;From my understanding you are referring to two different but related innovative workplace systems. These are Automated Data Capture and Entry and Document Automation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Businesses generate massive amounts of documents such as receipts, invoices, contracts, memo's, minutes, reports e.t.c. The information from the above documents is then to be entered into different software i.e. Accounting, Payroll, Excell and the web portal. Without a doubt data entry and document management can be a time consuming and tedious task in many modern offices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Automated Data Entry and Document Automation can help reduce the amount of time that employees spend on data entry and managing documents. This improves the speed and accuracy of work at the workplace. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above systems work by combining OCR technology with AI. When a document is scanned, it is initially just an image file because computers cannot distinguish text characters from the background image. With OCR software you can extract text from a document. As a result the document becomes searchable and easier to work with. Document automation software can then enter the information into a spreadsheet or other software automatically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some players in this field include Softworks (Automated Data Entry), Xtracta (Automated Data Entry and Document Automation) and Dynamic AI. Intuit also has a paperless productivity bundle that includes equipment such as the ScanSnap scanner (Automated Data Capture) and Intuit Hubdoc (Document Automation). &lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-08T11:36:56.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4948" PostTypeId="2" ParentId="3908" CreationDate="2018-01-08T15:20:39.947" Score="0" Body="&lt;p&gt;Although I (partly) agree with Nick Bostroms view that Artificial Intelligence could in some ways be dangerous. We do not need new government bodies to control or regulate AI development. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We already have sufficient cyber laws that protect us against computer crimes such as cyber terrorism, cyber bulling, creating malware, identity theft, DOSS, unauthorized access e.t.c.  It is the duty of local law enforcement agencies and the FBI to prevent and investigate cybercrimes such as those listed above. Whether AI was used to perpetrate the crime is legally immaterial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although AI is a 'new' technology. We already have a rigorous criminal justice system within our governance structures that are well capable of handling any eventualities that may arise from AI or any other technological breakthroughs without being overwhelmed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if an AI causes a car accident the manufacturer of the car can simply be charged for Product Liability for Negligence. If an AI is defective or dangerous. We already have product liability and consumer protection laws and the relevant government agencies to implement them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an AI uses its intelligence to maneuver within the law to its own advantage. This by definition is not a crime, big corporations do this all the time to minimize their taxes. However if necessary, the legislature can sit and pass a law criminalizing/banning this new activity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sovereign government already has enough powers and the neccessary instruments to excercise that power. Creation of a new government agency will lead to the unneccessary duplication of responsibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best approach is simply for the relevant government agencies to adapt by playing a proactive role and modernizing their service delivery so that it is in sync with current developments within society.This is what everyone has to do.&#xA;In reality we do not need additional agencies. I would find agencies for example Federal Ai Agency&#xA;or Federal Blockchain Commission to be baffling and unproductive.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-08T15:20:39.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4949" PostTypeId="1" CreationDate="2018-01-08T16:58:14.890" Score="4" ViewCount="501" Body="&lt;p&gt;I am trying to understand if robotic process automation is a field which requires expertise in machine learning.&#xA;Does the algorithm behind rpa use machine learning expect from OCR?&lt;/p&gt;&#xA;" OwnerUserId="12013" LastEditorUserId="1671" LastEditDate="2018-01-09T00:38:19.030" LastActivityDate="2018-05-01T20:57:39.443" Title="Is robotic process automation related to AI?" Tags="&lt;machine-learning&gt;&lt;robotics&gt;&lt;ai-basics&gt;&lt;automation&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4951" PostTypeId="2" ParentId="3548" CreationDate="2018-01-09T07:22:22.420" Score="10" Body="&lt;p&gt;To excel in in AI you need a mathematical intuition or point of view. In order to become a full stack AI engineer, it is important that you have a firm understanding of the mathematical foundations of machine learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice to anyone preparing to jump into the field is that learning mathematics is about doing. Remember the 20/80 rule. You need to study theory 20% of the time and practice what you have learnt 80% of the time in order to be proficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step before you dive into undergraduate math is for you to refresh on foundation level math. This includes revisiting and mastering high school mathematics especially Algebra. This is necessary for you to understand higher courses. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Linear Algebra&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Matrix Operations, Projections, Eigenvalues &amp;amp; Eigenvectors, Principal Component Analysis (PCA), Singular Value Decomposition (SVD), Eigen-decomposition of a matrix, LU Decomposition, QR Decomposition/Factorization, Symmetric Matrices, Orthogonalization &amp;amp; Orthonormalization, Vector Spaces and Norms are needed for understanding the optimization methods used for machine learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice thing about Linear Algebra is that there are wonderful online resources such as Khan Academy's course on linear algebra &lt;a href=&quot;https://www.khanacademy.org/math/linear-algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.khanacademy.org/math/linear-algebra&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Probability Theory and Statistics&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Probability Rules &amp;amp; Axioms, Conditional and Joint Distributions, Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian), Moment Generating Functions, Bayes' Theorem, Random Variables, Variance and Expectation, Maximum Likelihood Estimation (MLE), Prior and Posterior, Maximum a Posteriori Estimation (MAP) and Sampling Methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I refer you to this online Statistics and Probability MOOC at Khan Academy &lt;a href=&quot;https://www.khanacademy.org/math/probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.khanacademy.org/math/probability&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Algorithms and Optimization&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Knowledge of data structures (Binary Trees, Hashing, Heap, Stack etc), Dynamic Programming, Randomized &amp;amp; Sublinear Algorithm, Graphs, Gradient/Stochastic Descents and Primal-Dual methods are needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For in depth tuition, I recommend that you sign up for Andrew Ng's Coursera course available here &lt;a href=&quot;https://www.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.coursera.org/learn/machine-learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Multivariate Calculus&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Topics such as Differential and Integral Calculus, Directional Gradient, Hessian, Jacobian, Laplacian and Lagragian Distribution, Partial Derivatives, Vector-Values Functions. Here is a lint to Khan Academy's Calculus course &lt;a href=&quot;https://www.khanacademy.org/math/calculus-home&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.khanacademy.org/math/calculus-home&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Others&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This consists of math topics that are not covered in the above four major areas. They include Information Theory (Entropy, Information Gain), Function Spaces and Manifolds, Real and Complex Analysis (Sets and Sequences, Topology, Metric Spaces, Single-Valued and Continuous Functions, Limits, Cauchy Kernel, Fourier Transforms).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;More resources&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Below is an Excellent resource of free Machine Learning Mathematics ebooks  &lt;a href=&quot;http://blog.paralleldots.com/data-science/list-of-free-must-read-books-for-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blog.paralleldots.com/data-science/list-of-free-must-read-books-for-machine-learning/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally to keep up with recent development and the latest papers, I recommend that you follow this blog, which aggregates AI and ML papers, &lt;a href=&quot;http://www.arxiv-sanity.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.arxiv-sanity.com/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="2444" LastEditDate="2019-04-08T16:32:22.367" LastActivityDate="2019-04-08T16:32:22.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4952" PostTypeId="2" ParentId="4650" CreationDate="2018-01-09T09:34:04.330" Score="1" Body="&lt;p&gt;AI could hold the key in automating and optimizing networks. On the subscriber side, ML and AI will assist telecom operators in profiling the subscribers. This will be achieved by analyzing network activity, conversion rate of offers and data usage trends. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are a few use cases and how they will transform the telecommunication sector. (Source H2o.ai blog &lt;a href=&quot;https://www.h2o.ai/telecom/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.h2o.ai/telecom/&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Old generation telecom technologies.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Reactive Maintenance&lt;/li&gt;&#xA;&lt;li&gt;Network optimization with human intervention&lt;/li&gt;&#xA;&lt;li&gt;Centralized intelligence &lt;/li&gt;&#xA;&lt;li&gt;Security attack repair &lt;/li&gt;&#xA;&lt;li&gt;Backlogged customer tickets&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Future generation AI based telecom technologies.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Predictive Maintenance&lt;/li&gt;&#xA;&lt;li&gt;Self-optimizing network&lt;/li&gt;&#xA;&lt;li&gt;Optimal network quality&lt;/li&gt;&#xA;&lt;li&gt;Intelligence at the edge&lt;/li&gt;&#xA;&lt;li&gt;Security attack prediction&lt;/li&gt;&#xA;&lt;li&gt;Improved customer experience through customer service chat bots. &lt;/li&gt;&#xA;&lt;li&gt;Speech and voice services for customer which allows users to explore media content by spoken word rather than remote control. &lt;/li&gt;&#xA;&lt;li&gt;Predictive maintenance which is the ability to fix problems with telecom hardware such as cell towers, power lines e.t.c before they happen by detecting signals that usually lead to failure. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-09T10:30:37.520" LastActivityDate="2018-01-09T10:30:37.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4953" PostTypeId="1" AcceptedAnswerId="8536" CreationDate="2018-01-09T09:48:31.757" Score="4" ViewCount="465" Body="&lt;p&gt;Can &lt;a href=&quot;https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework&quot; rel=&quot;nofollow noreferrer&quot;&gt;Viola Jones algorithm&lt;/a&gt; be used to detect the facial emotion. Actually it was used in creating harr-cascade file for object and facial detection, but what confused me is whether it can be used to train for emotion detection. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what algorithms can I use? and what are the mathematical bases? (i.e. what mathematics should I be studying?) &lt;/p&gt;&#xA;" OwnerUserId="12021" LastEditorUserId="1671" LastEditDate="2019-05-16T19:02:54.233" LastActivityDate="2019-05-16T19:02:54.233" Title="Viola Jones Algorithm" Tags="&lt;algorithm&gt;&lt;image-recognition&gt;&lt;emotional-intelligence&gt;&lt;math&gt;&lt;facial-recognition&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4955" PostTypeId="1" CreationDate="2018-01-09T13:31:07.907" Score="3" ViewCount="148" Body="&lt;p&gt;I have a large dataset of skin images, each one associated with a hydration value (percentage).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm looking into predicting the hydration value from an image. My thinking: train a CNN on the dataset and evaluate the model with a mean square error regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, does this sound like a sensible way to try this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, I'd like to run the model on mobile. Can you recommend any examples with Caffe2 (or alternatively TensorFlow) or diagrams that might explain a similar task?&lt;/p&gt;&#xA;" OwnerUserId="12024" LastActivityDate="2018-08-16T15:01:15.120" Title="Predict value from image set" Tags="&lt;convolutional-neural-networks&gt;&lt;prediction&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4956" PostTypeId="1" CreationDate="2018-01-09T14:54:46.683" Score="2" ViewCount="218" Body="&lt;p&gt;Why do non-linear activation functions that produce values larger than 1 or smaller than 0 work?&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that neurons can only produce values between 0 and 1, and that this assumption can be used in things like cross-entropy. Are my assumptions just completely wrong?&lt;/p&gt;&#xA;&lt;p&gt;Is there any reference that explains this?&lt;/p&gt;&#xA;" OwnerUserId="12026" LastEditorUserId="2444" LastEditDate="2021-01-25T00:06:11.587" LastActivityDate="2021-01-25T00:06:11.587" Title="Why do non-linear activation functions that produce values larger than 1 or smaller than 0 work?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;activation-function&gt;&lt;relu&gt;&lt;cross-entropy&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4957" PostTypeId="1" CreationDate="2018-01-09T18:26:56.430" Score="2" ViewCount="88" Body="&lt;p&gt;Imagine we have 2 air conditioner systems (AA) and 2 &quot;free cooling&quot; systems which mix external and internal air (FC) in a closed box which always tends to warm up. For each system, we have to find turn on and off temperatures (for some hysteresis, let's say between the range 20-40 each one) to optimize the energy consumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we don't know the relation between these parameters and the energy consumption (and we don't intend to know them), we treat the problem as a black-box function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Till now, the problem would be solvable via a bayesian optimizer (eg. with gaussian process acquisition function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there is a problem: the best configuration may change between seasons, and even days! A simple bayesian optimizer maybe could deal with these changes limiting the data it takes into account by, for example, the last 15-30 days. But this would deal with the change AFTER the consumption increased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the idea is introduce some contextual variables which would help the system prevent these changes (eg. the external and internal temperature, and the vectors of variation of external and/or internal temperature, the weather prediction, whatever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, some of these variables we can take into account might be internal of the system, which means while these influence the best configuration, the actual configuration also influences these variables! and this becomes a reinforcement learning problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is there a way (documented or experimental) to know which variables (both internal or external) influences the optimal configuration of these AA/FC systems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Based on the first question, which would be the best approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.1.) No features. This might be considered a &lt;a href=&quot;https://stats.stackexchange.com/questions/168718/multi-armed-bandit-for-continuous-rewards-extended-question?rq=1&quot;&gt;multiarmed bandit problem for continuous reward&lt;/a&gt;. (FIX POSTERIOR TO SCENARIO CHANGE, IF THERE IS A SCENARIO CHAGNE)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.2.) Only external features to predict the scenario change. This might be considered a contextual multiarmed bandit-problem. (FORESEE THE SCENARIO CHANGE)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.3.) Consider only system-internal features. This can be considered a reinforcement-learning problem. (FIX IMMEDIATLY THE SCENARIO CHANGE)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.4.) Consider both external and internal features. This can be considered a reinforcement-learning problem where some of the states are not influenced by the configuration. (FORESEE THE SCENARIO CHANGE, AND IF SOMETHING FAILS, FIX IMMEDIATLY).&lt;/p&gt;&#xA;" OwnerUserId="6114" LastEditorUserId="6114" LastEditDate="2018-01-10T08:50:26.763" LastActivityDate="2018-01-10T08:50:26.763" Title="Which features and algorithm could optimize this air-conditioner problem?" Tags="&lt;algorithm&gt;&lt;reinforcement-learning&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4958" PostTypeId="2" ParentId="3548" CreationDate="2018-01-09T19:17:32.407" Score="1" Body="&lt;p&gt;You can start with they videos on Youtube. It is simple, with a &lt;strong&gt;basic knowledge in Python&lt;/strong&gt;, you can do much things with ML. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=cKxRvEZd3Mw&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning - Recipes #1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found too, a new video of &lt;strong&gt;Cloud AI Adventures&lt;/strong&gt; explain &lt;em&gt;&quot;The 7 Steps of Machine Learning&quot;&lt;/em&gt;, you can watch &lt;a href=&quot;https://www.youtube.com/watch?v=nKW8Ndu7Mjw&quot; rel=&quot;nofollow noreferrer&quot;&gt;HERE&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About math, I asked the same question when I started learning ML / AI. I'm not good with math, just the basics (also developer). But today, with &lt;strong&gt;TensorFlow&lt;/strong&gt; and other libraries, you can focus on the theory and definition. So, do not worry, over time, AI models will be construct with a few buttons and labels ... maybe it's happening.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="2444" LastEditDate="2019-04-08T16:34:30.753" LastActivityDate="2019-04-08T16:34:30.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4959" PostTypeId="2" ParentId="4956" CreationDate="2018-01-09T20:47:17.330" Score="2" Body="&lt;p&gt;Why wouldn't they work?&lt;/p&gt;&#xA;&lt;p&gt;Each neuron's output is equal to a function over the sum of all its weights multiplied by their corresponding neurons. If that function is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sigmoid&lt;/a&gt; function, then the output is squashed from &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;. If the entire layer uses a &lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;SoftMax&lt;/a&gt; function, then the output of all neurons is squashed from &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; and their sum equals 1. In other others, they represent a set of probabilities, where you can then use &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-entropy&lt;/a&gt; to optimize their values (cross-entropy measures the difference between two probability distributions).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1511.07289&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELU&lt;/a&gt; are simply other types of functions, whose output is not limited to the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;. They are differentiable, like other activation functions, and so they can be used in any neural network.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="2444" LastEditDate="2021-01-24T22:53:22.933" LastActivityDate="2021-01-24T22:53:22.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4960" PostTypeId="1" AcceptedAnswerId="4968" CreationDate="2018-01-09T22:35:46.167" Score="2" ViewCount="245" Body="&lt;p&gt;I studied machine learning when I was in university, a couple of years ago. I used it for my master thesis (decision trees, ensembles, SVM, and word embedding mostly) and for other projects either personal and academics (genetic algorithms, Q-learning).&lt;/p&gt;&#xA;&lt;p&gt;Then I studied neural networks by myself, from the pure theory of the perceptron, backpropagation, gradient descent, etc. (with Bengio's deep learning book) to the various ramifications of ConvNets, DeConvNets, auto-encoders, LSTM, deep reinforcement learning, and other various papers. I took a several months break and there are already new types of networks that I don't know, so hard to keep up with the state-of-the-art.&lt;/p&gt;&#xA;&lt;p&gt;However, my knowledge is purely theoretical. I want to embed machine learning in my future career, but getting a job in the field requires practical expertise, which I lack.&lt;/p&gt;&#xA;&lt;p&gt;I am working full time now as a software engineer. So, &lt;em&gt;what do you think may be a possible path to follow in order to gain expertise?&lt;/em&gt; &lt;em&gt;Given that I have a strong mathematical background and more than 10 years of coding on my back, is there any resource on practical rules of thumbs, suggestions, real case studies, etc., which I can invest time onto?&lt;/em&gt; I mean something beyond the vanilla &amp;quot;introduction to machine learning&amp;quot;, something more real and concrete.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Do you think that attempting Kaggle challenges from scratch would be stimulating? Or too confusing? Do you think delivering a personal project (i.e. not a byproduct of my current job) in which I employ one such method to solve a potentially real problem would be enough to be relevant in my cv?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12030" LastEditorUserId="2444" LastEditDate="2020-09-24T21:12:36.520" LastActivityDate="2020-09-24T21:23:34.690" Title="What are some resources to improve practical expertise in machine learning?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;resource-request&gt;&lt;kaggle&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ClosedDate="2021-01-27T19:20:59.740" ContentLicense="CC BY-SA 4.0" />
  <row Id="4962" PostTypeId="2" ParentId="4953" CreationDate="2018-01-10T05:54:56.023" Score="1" Body="&lt;p&gt;I have once tried Viola Jones Algorithm to do that, it does not capture the subtle differences in the direction of facial segments which are important to detect emotion. Features like HOG (Available in openCV and many famous image processing libraries) can extract better information from the face to classify emotion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also there are many other approaches including ANNs and pure rule based approaches But almost everywhere a good alignment approach for faces become the most important aspect of the exercise. So I will suggest exploring some facial alignment approaches and then Features like HOG instead of Viola Jones/ HARR. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the mathematics part, it is upto you to dive deep into mathematics or just exploring different approaches by codes. A good understanding of Linear Algebra and a little Geometry will help a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if you are new to Machine Learning, understanding the basic algorithms might be relevant to you.&lt;/p&gt;&#xA;" OwnerUserId="11387" LastActivityDate="2018-01-10T05:54:56.023" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4964" PostTypeId="1" CreationDate="2018-01-10T13:21:00.913" Score="3" ViewCount="967" Body="&lt;p&gt;I am making my speech recognition project for PC (working on Windows 8) and am new in this area. The project should have basic functionality, like dictation with accuracy in an e-mail, notepad, etc., and should respond to local commands of PC.&lt;/p&gt;&#xA;&lt;p&gt;I am using &lt;a href=&quot;https://cmusphinx.github.io/wiki/tutorialsphinx4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sphinx4&lt;/a&gt; for my speech recognition project.  I'm trying to determine if there is a better open-source API than CMU Sphinx in terms of accuracy and large vocabulary.&lt;/p&gt;&#xA;&lt;p&gt;Does Kaldi (deep neural network-based) perform better than CMU Sphinx (HMM-based)? Are the two platforms better for different kinds of applications?&lt;/p&gt;&#xA;&lt;p&gt;Essentially:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I want to increase my speech recognition system accuracy and vocabulary.  Which might be most optimal: &lt;a href=&quot;http://kaldi-asr.org/doc/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaldi&lt;/a&gt; or &lt;a href=&quot;http://www.speech.cs.cmu.edu/sphinx/doc/Sphinx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CMU Sphinx&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Also, wondering what is the difference between a speech API and a speech Engine, and, more generally, as a developer what I will require to develop my software?&lt;/p&gt;&#xA;&lt;p&gt;Please help me to get a clear understanding of the above questions and, if possible, provide some speech recognition developer or researcher community links.&lt;/p&gt;&#xA;&lt;p&gt;Any and all comments, suggestions, and answers are welcome!&lt;/p&gt;&#xA;" OwnerUserId="12043" LastEditorUserId="2444" LastEditDate="2021-02-06T18:03:18.630" LastActivityDate="2021-02-06T18:03:18.630" Title="Speech recognition software implementation by open-source API" Tags="&lt;neural-networks&gt;&lt;resource-request&gt;&lt;voice-recognition&gt;&lt;speech-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2021-02-06T18:04:08.737" ContentLicense="CC BY-SA 4.0" />
  <row Id="4965" PostTypeId="1" AcceptedAnswerId="6418" CreationDate="2018-01-10T16:52:52.467" Score="12" ViewCount="17035" Body="&lt;p&gt;I am working on a problem where I need to determine whether two sentences are similar or not.  I implemented a solution using BM25 algorithm and wordnet synsets for determining syntactic &amp;amp; semantic similarity.  The solution is working adequately, and even if the word order in the sentences is jumbled, it is measuring that two sentences are similar. For example&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Python is a good language.&lt;/li&gt;&#xA;&lt;li&gt;Language a good python is.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My problem is to determine that these two sentences are similar.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What could be the possible solution for structural similarity? &lt;/li&gt;&#xA;&lt;li&gt;How will I maintain the structure of sentences?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9428" LastEditorUserId="2444" LastEditDate="2019-11-15T20:25:25.433" LastActivityDate="2019-11-15T20:25:25.433" Title="How do I compute the structural similarity between sentences?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;similarity&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4966" PostTypeId="5" CreationDate="2018-01-10T17:09:34.080" Score="0" Body="&lt;p&gt;Derives from &lt;a href=&quot;https://en.wikipedia.org/wiki/Praxis_(process)#Origins&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aristotle&lt;/a&gt;, to distinguish from &lt;a href=&quot;https://en.wikipedia.org/wiki/Theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;theoria&lt;/em&gt;&lt;/a&gt; (thinking) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Poiesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;poiesis&lt;/em&gt;&lt;/a&gt; (making).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://en.oxforddictionaries.com/definition/theorics&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Theorics&quot;&lt;/a&gt; vs. &lt;a href=&quot;https://en.oxforddictionaries.com/definition/practic&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Practics&quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-10T17:09:34.080" LastActivityDate="2018-01-10T17:09:34.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4967" PostTypeId="4" CreationDate="2018-01-10T17:09:34.080" Score="0" Body="From Aristotle. Praxis (process): &quot;is the process by which a theory, lesson, or skill is enacted.&quot;  Use for questions related to the manner in which theory is applied, embodied, or realized." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-02T01:47:06.683" LastActivityDate="2019-05-02T01:47:06.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4968" PostTypeId="2" ParentId="4960" CreationDate="2018-01-10T19:10:06.360" Score="1" Body="&lt;p&gt;Since you are a software engineer, you already have some experience with one or more programming languages, so this is already a good thing. Moreover, you say that you have a strong mathematical background, so you will probably learn certain machine learning concepts more easily.&lt;/p&gt;&#xA;&lt;p&gt;When it comes to machine learning, there are open source projects that you can contribute to via Github, and this really boosts your skills and experience.&lt;/p&gt;&#xA;&lt;p&gt;I also am contributing to an open-source conversation agent via Github.&#xA;If you are able to understand and contribute to such projects, then implementing ML/DL algorithms is possible in your target language. I myself have been using Python, but that's just a personal choice, and it's been utilized by many AI engineers and data scientists, globally.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do you think that attempting kaggle challenges from scratch would be&#xA;stimulating? Or too confusing? Do you think delivering a personal&#xA;project (i.e. not a byproduct of my current job) in which I employ one&#xA;of such methods to solve a potentially real problem would be enough to&#xA;be relevant in my cv?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Contributing to open-source projects will link you to other engineers in the same field, and this will eventually open you many paths.&lt;/p&gt;&#xA;&lt;p&gt;Also, try the book: &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0136042597&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach Stuart Russel, Peter Norvig, Prentice Hall, 3rd edition, 2009&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2020-09-24T21:23:34.690" LastActivityDate="2020-09-24T21:23:34.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4969" PostTypeId="1" AcceptedAnswerId="4974" CreationDate="2018-01-10T20:05:55.703" Score="0" ViewCount="49" Body="&lt;p&gt;What is the difference between a histopathological image and a natural image when training a neural network? &lt;/p&gt;&#xA;" OwnerUserId="9560" LastEditorUserId="75" LastEditDate="2018-01-27T17:01:41.253" LastActivityDate="2018-01-27T17:01:41.253" Title="Histopathological image vs. natural image" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4970" PostTypeId="2" ParentId="1859" CreationDate="2018-01-11T07:31:18.213" Score="1" Body="&lt;p&gt;Although this model played an important role in contributing  to our present understanding of NLP and NLU, it is no longer useful in production systems and currently no successful commercial product follows this approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In CDT the goal was to design an AI system that could draw logical inferences from sentences. In this system the goal was to make the meaning independent of the words used in the input.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CDT modeled sentences by using tokens such as: locations, time, real world actions and real world objects. However as computational power became more common and less expensive, interest diverted to statistical models which were now outperforming the previous rule based systems.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with rule based approaches such as CDT is that they require manual development of linguistic rules which can be costly and which usually don't generalize well to other languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, statistical approaches use  human language resources (multilingual textual corpora) more efficiently. Rather than using a rule based approach, statistical models make soft probabilistic decisions based on attaching real weights to the features making up the input data. (Wikipedia NLP)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This efficient use of human language resources leads to a model that is more accurate and robust especially when given unfamiliar input or input that contains errors. Statistical models also generalise well to other languages.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-11T07:31:18.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4971" PostTypeId="2" ParentId="4027" CreationDate="2018-01-11T08:24:34.537" Score="6" Body="&lt;p&gt;According to Open AI's Greg Brockman, the Gym website never had a big impact and so was never maintained. This is the reason he gives for shutting down the website. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A read only export of the site was archived at &lt;a href=&quot;https://gym.openai.com/read-only.html&quot; rel=&quot;noreferrer&quot;&gt;https://gym.openai.com/read-only.html&lt;/a&gt; and if you attempt to access the old website through the url &lt;a href=&quot;https://gym.openai.com&quot; rel=&quot;noreferrer&quot;&gt;https://gym.openai.com&lt;/a&gt; you will be redirected to the Open AI gym github repository. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a while a static copy of the Open AI leader-board was maintained on the below url &lt;a href=&quot;https://scoreboard-site-1764008611.us-west-2.elb.amazonaws.com/&quot; rel=&quot;noreferrer&quot;&gt;https://scoreboard-site-1764008611.us-west-2.elb.amazonaws.com/&lt;/a&gt; . However as of present the static site is also unreachable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Canadian AI startup montreal.ai had offered to maintain the gym website. However this offer was not taken up by open ai. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This issue was discussed in depth on the following threads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/6zvlm2/d_openai_closing_down_gym_toolkit_website/?st=jca6ia3n&amp;amp;sh=48a0d8f3&quot; rel=&quot;noreferrer&quot;&gt;https://www.reddit.com/r/MachineLearning/comments/6zvlm2/d_openai_closing_down_gym_toolkit_website/?st=jca6ia3n&amp;amp;sh=48a0d8f3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/openai/gym/issues/718&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/openai/gym/issues/718&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-11T08:24:34.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4973" PostTypeId="2" ParentId="3453" CreationDate="2018-01-11T15:33:48.933" Score="1" Body="&lt;p&gt;Using an STN alone to predict the next frame assumes that there is some linear translation between the current frame and the next frame. In some domains this is true but usually there are more complicated transitions from frame to frame (Eg. An occlusion, a new entity, light difference, etc). So, although STNs may be useful to resizing and translating inputs for a CNN, they should be used together with other techniques when predicting a new frame of a sequence. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2018-01-11T15:33:48.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4974" PostTypeId="2" ParentId="4969" CreationDate="2018-01-11T19:08:19.753" Score="1" Body="&lt;p&gt;Histopathological images compared to natural images difference so, that in &lt;a href=&quot;https://en.wikipedia.org/wiki/Histopathology&quot; rel=&quot;nofollow noreferrer&quot;&gt;histopathological&lt;/a&gt;  images the image needed to be modeled may contain millions of pixels, whereas a recognition from natural image that this is a dog / house / certain person needs significantly less information to be extracted from the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Histopathological images use WSIs (Whole slide image) which contain the part of tissue as a whole. It needs to be split to 256 x 256 pixel patches and those are used for ROI hunting (Region of Interest) and other analysis one by one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Histopathological images (WSI) are quite rare nowadays and hold privately. More open data would rise the accuracy of machine learning and increase the amount of results that could be found from the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Complete source:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1709.00786v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1709.00786v1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-01-11T19:08:19.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4975" PostTypeId="1" CreationDate="2018-01-11T19:22:11.197" Score="3" ViewCount="1019" Body="&lt;p&gt;I'm currently writing the &lt;em&gt;Alpha-Beta pruning&lt;/em&gt; algorithm for a board game. Now I need to come up with a good evaluation function. The game is a bit like snakes and ladders (you have to finish the race first), so for a possible feature list I came up with the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;field index should be high&lt;/li&gt;&#xA;&lt;li&gt;in the lower fields my fuel should be high, when coming to the end it should be low (maximum of '10' required to enter the goal)&lt;/li&gt;&#xA;&lt;li&gt;all 'power-ups' must be spent to enter the goal, so prioritize them&lt;/li&gt;&#xA;&lt;li&gt;if it is possible to enter the goal (a legit move), do it!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There could be some more for some special cases.&lt;/p&gt;&#xA;&lt;p&gt;I've read somewhere that it is the best (and easiest) to combine them in a linear function, for example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$0.75 * i - 5 * p - 0.25 * |(f - \text{MAX_FIELD_INDEX}/i)|,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; = field index&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; = power-ups&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; = fuel&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Since I can't ask an expert and I'm not an expert by myself, I have nobody to ask if those parameters are good, if I've forgotten something or if I've combined the factors correctly.&lt;/p&gt;&#xA;&lt;p&gt;The parameters aren't that big of a deal because I could use a genetic algorithm or something else to optimize them.&lt;/p&gt;&#xA;&lt;p&gt;My problem and question is: What do I have to do to find out how to put together my features optimally (how can I optimize the function/parameter arrangement itself)?&lt;/p&gt;&#xA;" OwnerUserId="11585" LastEditorUserId="2444" LastEditDate="2021-02-06T20:11:58.647" LastActivityDate="2021-02-06T20:11:58.647" Title="How do I write a good evaluation function for a board game?" Tags="&lt;game-ai&gt;&lt;evolutionary-algorithms&gt;&lt;alpha-beta-pruning&gt;&lt;evaluation-functions&gt;&lt;board-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4976" PostTypeId="1" CreationDate="2018-01-11T22:07:19.627" Score="2" ViewCount="80" Body="&lt;p&gt;&lt;strong&gt;CIO NN&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CIO NN stands for &lt;strong&gt;C&lt;/strong&gt;ontroller &lt;strong&gt;I&lt;/strong&gt;nput &lt;strong&gt;O&lt;/strong&gt;utput &lt;strong&gt;N&lt;/strong&gt;erual &lt;strong&gt;N&lt;/strong&gt;etwork&lt;/p&gt;&#xA;&#xA;&lt;p&gt;note due to a typo the &quot;&lt;strong&gt;nearon&lt;/strong&gt;&quot; means &quot;&lt;strong&gt;neron&lt;/strong&gt;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this we have to redefine the Nearon&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2 Inputs&lt;/li&gt;&#xA;&lt;li&gt;2 Outputs&lt;/li&gt;&#xA;&lt;li&gt;4 Weights (each input and output have their own weights)&lt;/li&gt;&#xA;&lt;li&gt;Internal Memory Cell (any byte or bit or block size with variable size)&lt;/li&gt;&#xA;&lt;li&gt;Activation Function (Defines what weights and what inputs activate this Nearon)&lt;/li&gt;&#xA;&lt;li&gt;Memory Storage Function (Defines what and when this cell should store said memory or memory stream)&lt;/li&gt;&#xA;&lt;li&gt;Memory Transpose Function (Once activated any stored memory that the activation function can trigger will be played/pushed into the Nerual Network)&lt;/li&gt;&#xA;&lt;li&gt;Forget Function (Defines when and/or how and/or why these memories can be destroyed/removed based on Activation function with Memory states and any input stateses itself)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How would I implement this in the form of Code?&#xA;please take note of the spec. (this is non profit/GNU v3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This would look something like these:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bo4We.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bo4We.png&quot; alt=&quot;Basic CIO Nearon&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These would be arranges like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/SDVmv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SDVmv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which we can build it into this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GdAra.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GdAra.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it can be trained like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;do normal NN training from the inputs to the input outputs like a hidden layer NN&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then trained to be controlled like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;then set the known NN dataset (inputs) to be corrected to actual or correct values via the CI (Controller Input) which will be outputed on the &quot;Input + Controller Input&quot; Output&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;by training like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you have a normal NN with hidden layers which can be trainned (this can be done with CNNs) with backpropergation, and a cost function (use least amount of nerons) etc&lt;/li&gt;&#xA;&lt;li&gt;now you can allow the CIO NN to retrain / teach itself with supervised or unsupervised learning.&lt;/li&gt;&#xA;&lt;li&gt;you can combine the &quot;Input Output&quot; and the &quot;Input + Controller Input&quot; Output with another NN which can then connect with this NN in a similiar way that a Neron connects to a Neron&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1282" LastEditorUserId="1282" LastEditDate="2018-01-12T21:54:30.257" LastActivityDate="2018-01-12T21:54:30.257" Title="How would I implement this New Type of NN" Tags="&lt;ai-design&gt;&lt;training&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4977" PostTypeId="2" ParentId="2980" CreationDate="2018-01-12T03:49:42.523" Score="7" Body="&lt;p&gt;IMHO the idea of invalid moves is itself invalid. Imagine placing an &quot;X&quot; at coordinates &lt;code&gt;(9, 9)&lt;/code&gt;. You could consider it to be an invalid move and give it a negative reward. Absurd? Sure!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in fact your invalid moves are just &lt;em&gt;a relic of the representation&lt;/em&gt; (which itself is straightforward and fine). The best treatment of them is to exclude them completely from any computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gets more apparent in chess:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In a positional representation, you might consider the move &lt;code&gt;a1-a8&lt;/code&gt;, which only belongs in the game if there's a Rook or a Queen at &lt;code&gt;a1&lt;/code&gt; (and some other conditions hold).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In a different representation, you might consider the move &lt;code&gt;Qb2&lt;/code&gt;. Again, this may or may not belong to the game. When the current player has no Queen, then it surely does not.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As the invalid moves are related to the representation rather than to the game, they should not be considered at all.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-01-12T03:49:42.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4978" PostTypeId="2" ParentId="4975" CreationDate="2018-01-12T04:09:39.310" Score="1" Body="&lt;p&gt;Based on your description, I'd maximize the following terms:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;i&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-max(f - 10 - (MAX_FIELD_INDEX - i), 0)&lt;/code&gt; - assuming consumption of one fuel per field; this becomes negative when you have too much fuel&lt;/li&gt;&#xA;&lt;li&gt;a similar function of &lt;code&gt;p&lt;/code&gt;, as spending them gets more important when approaching the goal&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As having fuel is probably a good thing in the beginning, you could use a term like &lt;code&gt;f&lt;/code&gt;. Similarly for the &quot;power packs&quot; (or are they rather &quot;weakness packs&quot;?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd combine the terms using a linear function like you did and let it optimize. You may need more such terms. Maybe it's simpler to get rid of the power packs when you have enough fuel? Then something like &lt;code&gt;-max(p-f, 0)&lt;/code&gt; may help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may generate some ad-hoc expressions or add some products of your terms as new terms. You may want to do this after the coefficients of the simpler terms have already been optimized (so you help the more complex optimization with a good staring point).&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-01-12T04:09:39.310" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4979" PostTypeId="1" CreationDate="2018-01-12T13:05:47.300" Score="6" ViewCount="438" Body="&lt;p&gt;At my work we're currently doing some research into data visualisation for highly inter connected data, basically graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've been implementing all sorts of different layouts and trying to see which fits best, but, due to the nature of the problem --it's a visual thing - we needed to come up with some automated way to analyse the result so welcome up with a bunch of metrics to analyse our layouts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, the most important metrics have been information density, edge crossings, node overlap and edge length. This gives us some good results and has allowed us to fine tune our layout algorithms.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when a new graph is loaded, we noticed that humans still tend to fiddle a lot with the structure of the layout. Moreover, it seems that our metrics do a good job of predicting where a user is likely to mess around. Graph layout is a tough problem, so &lt;strong&gt;after some discussion, the idea of just throwing data at a neural network and let it figure it out came up.&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of us are experts, or even experienced in AI.  I'm the one with the most contact with AI methods.  &lt;strong&gt;All I've ever done were simple NN models, no convolution, feedback or feedforward or anything of the sorts, but it seems to me this should be doable.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe it's my lack of expertise here but I haven't been able to find any good information on this sort of application for NNs, so I was hoping someone here could point me in the right direction.   &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What sort of model is best for such a situation? and why? Is this actually possible or would it be super complicated? Has anyone ever tried something like this before?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If it helps, our input data (for v1, I guess) would be two arrays of variable length, one for the nodes and another for the relationships between them and the output data would be an array with the node XY coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated!&#xA;Cheers!&lt;/p&gt;&#xA;" OwnerUserId="12070" LastEditorUserId="1671" LastEditDate="2018-01-12T18:17:38.623" LastActivityDate="2018-12-05T00:01:15.260" Title="Neural network for data visualization" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;research&gt;&lt;graphs&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4980" PostTypeId="5" CreationDate="2018-01-12T20:07:06.050" Score="0" Body="&lt;p&gt;The idea here is that these APIs, frameworks, etc. are &quot;known quantities&quot; and therefore can be objectively assessed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in the case of &quot;black box&quot; software, performance can be rated statistically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Distinct from &quot;Software recommendations&quot;, although there may be some overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With so many options, it is useful to get analyses backed up by facts and data.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-12T20:07:06.050" LastActivityDate="2018-01-12T20:07:06.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4981" PostTypeId="4" CreationDate="2018-01-12T20:07:06.050" Score="0" Body="Use for objective evaluations and assessments of API, plug-ins, frameworks, etc.  (i.e. should not be opinion based for formal answers, but based on structure, architecture, and validated performance.)  &#xD;&#xA;&#xD;&#xA;Opinions may be suitable for comments, but not for formal answers." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-12T20:07:06.050" LastActivityDate="2018-01-12T20:07:06.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4982" PostTypeId="2" ParentId="4397" CreationDate="2018-01-12T20:40:24.280" Score="1" Body="&lt;p&gt;First of all, making app for mobile is not normally written in Python, but if you want it, that is possible [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other non ai question is that the counting of speech recognition is so heavy, that at least Alexa etc make it on external server. Wake word recognition is done in machine because of real time needs [3]. Ofcourse your app can trigger from mobile touch event too so speech recognition implementation on mobile can be avoided. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using server needs knowledge about communication and web apis as well as server maintenance in cloud.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speech recognition can be made with open source tools, but the dictionaries with enough samples may not yet exist or they're coming [2]. Also Google offers speech api, that needs mainly programming knowledge to attach speech recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For questions you also need to generate answers. To form that you may reverse the flow in speech generation [4]. All the logic to find useful answers needs some more effort depending how intelligent your assistant will be. That maybe is the most challenging part where direct answer do not exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you could attach to ready assistant's public api, it would be magnificent. Actually, Alexa api at least is open [5]!! Playing with it directly from mobile makes app soon ready but if you want to learn more algorihms, some pointers are on my above other points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.quora.com/Can-I-make-an-Android-app-with-Python&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/Can-I-make-an-Android-app-with-Python&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://research.mozilla.org/machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://research.mozilla.org/machine-learning/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://iot.stackexchange.com/questions/736/why-do-most-smart-assistants-offer-little-if-any-customisation-of-the-wake-wor&quot;&gt;https://iot.stackexchange.com/questions/736/why-do-most-smart-assistants-offer-little-if-any-customisation-of-the-wake-wor&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://ai.stackexchange.com/questions/4851/how-could-you-generate-sentences-from-lists-of-facts/4869&quot;&gt;How could you generate sentences from lists of facts&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;https://github.com/alexa/alexa-avs-sample-app&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/alexa/alexa-avs-sample-app&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-01-12T20:40:24.280" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4984" PostTypeId="1" AcceptedAnswerId="4985" CreationDate="2018-01-13T03:48:19.280" Score="3" ViewCount="474" Body="&lt;p&gt;I have some episodic datasets extracted from a turn-based RTS game in which the current actions leading to the next state doesn’t determine the final solution/outcome of the episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The learning is expected to terminate at a final state/termination condition (when it wins or losses) for each episode and then move on to the next number of episodes in the dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have being looking into Q learning, Monte Carlo and SARSA but I am confused about which one is best applicable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If any of the mentioned algorithm is implemented, can a reward of zero be given in preliminary states before termination state of each episodes at which it will be rewarded with a positive/negative (win/loss) value?&lt;/p&gt;&#xA;" OwnerUserId="12081" LastActivityDate="2018-01-13T20:28:09.730" Title="Which Reinforcement Learning algorithms are efficient for episodic problems?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;unsupervised-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4985" PostTypeId="2" ParentId="4984" CreationDate="2018-01-13T07:55:57.340" Score="1" Body="&lt;p&gt;When applying techniques like SARSA(which are on-policy), one needs to have control over a simulator. If one is able to access only the episodic dataset, then the only choice is to opt for Q-learning or Off-policy Monte-Carlo(or off-policy methods in general).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can a reward of zero&#xA;  be given in preliminary states before termination state of each&#xA;  episodes at which it will be rewarded with a positive/negative&#xA;  (win/loss) value?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With regards to the above question, the answer is yes. The task would be a sparse reward task with a reward occurring only at the last transition. The issue that one faces in a sparse reward task is that of slow convergence(or even lack of convergence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some guidelines for tackling sparse reward tasks are as follows :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Monte-Carlo, and n-step Q-learning are preferred over Q-learning/SARSA.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the 10-step chain MDP, where the only reward is +1 when the transition from s10 to END occurs.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8JrW5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8JrW5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us consider the first episode in training to be start-&gt;s1-&gt;s2-&gt;s3-&gt;....-&gt;s10-&gt;end. The Q-learning update would result in no sane updates to states s1, s2, s3, s9 because the Q-value of the next state is random-value. The only state with a sane update is s10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we use n-step Q-learning or a Monte-Carlo based update. The Q-values of all the states are updated in a logical manner since the reward from the end of the episode &lt;em&gt;propagates&lt;/em&gt; to all states in the episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;n-step Q-learning would be ideal since by adjusting the value of &lt;em&gt;n&lt;/em&gt;, one can trade-off the benefits of Monte-Carlo methods(described above) and Q-learning(low variance). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;The use of pseudo/auxiliary rewards&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not something necessarily recommended since the addition of new reward-structures can cause unintended behaviour. On the flip-side, it could lead to faster convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple example is as follows: Consider a game of chess, where the only reward is at the end of the game. Since the game of chess has very long episodes, one can introduce the following reward-structure instead :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;+100 for winning the game&lt;/li&gt;&#xA;&lt;li&gt;+1 for capturing any piece on the board&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hence, the pseudo reward may provide some direction for learning. Note the different scales in the two rewards(100:1). This is necessary because the primary goal of the task should not shift to capturing pieces, but to win the match.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12084" LastEditorUserId="12084" LastEditDate="2018-01-13T20:28:09.730" LastActivityDate="2018-01-13T20:28:09.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4986" PostTypeId="1" CreationDate="2018-01-13T13:52:26.407" Score="4" ViewCount="71" Body="&lt;p&gt;I have a dataset with 2,23,586 samples out of which i used 60% for training and 40% for testing. I used 5 classifiers individually, SVM, LR, decision tree, random forest and boosted decision trees. SVM and LR performed well with close to 0.9 accuracy and recall also 0.9 but tree based classifiers reported an accuracy of 0.6. After a careful observation, I found out that SVM and LR did not predict the labels of 20,357 samples identically. So Can I apply voting and resolve this conflict wrt prediction outcome? Can this conflict be due to an imbalanced dataset?&lt;/p&gt;&#xA;" OwnerUserId="12088" LastEditorUserId="16909" LastEditDate="2018-08-08T20:45:04.810" LastActivityDate="2018-08-08T20:45:04.810" Title="Can I combine two classifiers that make different kinds of errors to get a better classifier?" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4987" PostTypeId="1" CreationDate="2018-01-13T15:02:40.960" Score="7" ViewCount="598" Body="&lt;p&gt;Some argue that humans are somewhere along the middle of the intelligence spectrum, some say that we are only at the very beginning of the spectrum and there's so much more potential ahead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a limit to the increase of intelligence? Could it be possible for a general intelligence to progress infinitely, provided enough resources and armed with the best self-recursive improvement algorithms?&lt;/p&gt;&#xA;" OwnerUserId="12089" LastEditorUserId="2444" LastEditDate="2019-04-19T15:22:11.460" LastActivityDate="2019-04-19T15:22:11.460" Title="Is there a limit to the increase of intelligence?" Tags="&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4988" PostTypeId="1" AcceptedAnswerId="4992" CreationDate="2018-01-13T15:14:53.140" Score="6" ViewCount="171" Body="&lt;p&gt;i'm quite new to neural network and i recently built neural network for number classification in vehicle license plate. It has 3 layers: 1 input layer for 16*24(382 neurons) number image with 150 dpi , 1 hidden layer(199 neurons) with sigmoid activation function, 1 softmax output layer(10 neurons) for each number 0 to 9. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to expand my neural network to also classify letters in license plate. But i'm worried if i just simply add more classes into output, for example add 10 letters into classification so total 20 classes, it would be hard for neural network to separate feature from each class. And also, i think it might cause problem when input is one of number and neural network wrongly classifies as one of letter with biggest probability, even though sum of probabilities of all number output exceeds that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i wonder if it is possible to build hierchical neural network in following manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 3 neural networks: 'Item', 'Number', 'Letter'&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;'Item' neural network classifies whether input is numbers or letters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If 'Item' neural network classifies input as numbers(letters), then input goes through 'Number'('Letter') neural network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Return final output from Number(Letter) neural network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And learning mechanism for each network is below:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;'Item' neural network learns all images of numbers and letters. So there are 2 output.&lt;/li&gt;&#xA;&lt;li&gt;'Number'('Letter') neural network learns images of only numbers(letter).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which method should i pick to have better classification? Just simply add 10 more classes or build hierchical neural networks with method above?&lt;/p&gt;&#xA;" OwnerUserId="12090" LastActivityDate="2018-01-17T03:25:18.633" Title="Is it better to make neural network to have hierchical output?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4989" PostTypeId="1" CreationDate="2018-01-13T15:16:21.933" Score="1" ViewCount="500" Body="&lt;p&gt;What is the relation between back-propagation and reinforcement learning?&lt;/p&gt;&#xA;" OwnerUserId="9560" LastEditorUserId="2444" LastEditDate="2018-11-07T16:23:28.823" LastActivityDate="2018-11-07T16:23:28.823" Title="What is the relation between back-propagation and reinforcement learning?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;backpropagation&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4990" PostTypeId="1" CreationDate="2018-01-13T16:20:28.183" Score="1" ViewCount="636" Body="&lt;p&gt;I play a racing game called Need For Madness ( some gameplay: &lt;a href=&quot;https://www.youtube.com/watch?v=NC5uFZ-t0A8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=NC5uFZ-t0A8&lt;/a&gt; ). NFM is a racing game, where the player can choose different cars and race and crash the other cars, and you can play on different tracks too. The game has a fixed frame rate, so you can assume that the same sequence of button presses will always arrive at the exact same position, rotation, velocity, etc. of the car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to make a bot which could race faster than I can. What would be the best way to go about doing this? Is this problem even suited for deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking I could train a neural network where the input would be the current world state (position of the player, position of the checkpoints you have to through and all the obstacles), and the output would be an array of booleans, one for each button. During a race, I could then keep forward propagating from the input to the booleans. However, I'm not so sure what I would do after the race is over. How do I back propagate after the race to make the NN be less likely to make bad moves?&lt;/p&gt;&#xA;" OwnerUserId="12091" LastEditorUserId="1671" LastEditDate="2018-01-17T00:41:47.397" LastActivityDate="2018-01-17T00:41:47.397" Title="How to teach an AI to race optimally in a racing game?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;game-ai&gt;&lt;path-finding&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4991" PostTypeId="1" AcceptedAnswerId="4995" CreationDate="2018-01-13T20:13:58.970" Score="0" ViewCount="1221" Body="&lt;p&gt;I don't understand why Google Translate translates the same text in different ways.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Enter_the_Dragon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is the Wikipedia page of the 1973 film &amp;quot;Enter the Dragon&amp;quot;&lt;/a&gt;. You can see that its traditional Chinese title is: 龍爭虎鬥. &lt;a href=&quot;https://translate.google.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google translates&lt;/a&gt; this as &amp;quot;Dragons fight&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Then, if we go to &lt;a href=&quot;https://zh-yue.wikipedia.org/wiki/%E9%BE%8D%E7%88%AD%E8%99%8E%E9%AC%A5_(%E9%9B%BB%E5%BD%B1)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Wikipedia page of this film&lt;/a&gt;, and search for 龍爭虎鬥 using Ctrl-F, it will be found on several places:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cezxH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cezxH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But if we try to copy the hyperlink of Chinese page into Google translate, it will be the word &amp;quot;tiger&amp;quot; from somewhere:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bsf5G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bsf5G.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Even more, if we try to translate Chinese page into English using build-in Chrome translate, it will be sometimes translated as &amp;quot;Enter the Dragon&amp;quot;, in English manner:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JIRQY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JIRQY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why it gives different translations for the same Chinese text here?&lt;/p&gt;&#xA;" OwnerUserId="12095" LastEditorUserId="2444" LastEditDate="2021-01-21T21:46:20.707" LastActivityDate="2021-01-21T21:46:20.707" Title="Why does Google Translate produce two different translations for the same Chinese text?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;machine-translation&gt;&lt;google-translate&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4992" PostTypeId="2" ParentId="4988" CreationDate="2018-01-13T20:25:24.450" Score="0" Body="&lt;p&gt;Item network should be recogniting regional register number standards and typical places of numbers on those standards. Otherwise your 'Item' phase has equal task than the whole system, because 'Letter' and 'Number' do not differ in plates so much you could right away choose between the two without first evaluating the exact object on each case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Side note: correct me if that is not true on your local plate system&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-01-13T20:25:24.450" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4993" PostTypeId="2" ParentId="4991" CreationDate="2018-01-13T21:27:00.850" Score="2" Body="&lt;p&gt;As you know google translation works base on statistical methods. In statistical translation, many parameters can be related to the final result. One of these parameters is co-occurrence of words in a sentence. &#xA;Hence, as this translator learn languages from different utterances by human and pre-written translations, and different parameters in the text are involved in this learning, It could be possible one word has a different meaning in different texts.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2018-01-13T21:27:00.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4994" PostTypeId="2" ParentId="4991" CreationDate="2018-01-13T22:23:04.690" Score="1" Body="&lt;p&gt;Chinese words 'fu' means with different intonation marks either happiness, huspand or tiger. Without correct intonation notation in English it may translate in Chinese as Tiger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Movie title has Chinese 'happiness' character, but Google mixes it as Tiger.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-01-13T22:23:04.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4995" PostTypeId="2" ParentId="4991" CreationDate="2018-01-13T23:48:09.523" Score="5" Body="&lt;p&gt;It's not quite clear what you are asking. So I'll answer in separate parts.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is the translation different from the official title?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It could be simply because machine translation is not perfect, or our human translator took some creative liberties when translating. In this case it seems to be both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that 龍爭虎鬥 properly translated doesn't mean either &quot;Dragons fight&quot; or &quot;Enter the dragon&quot;. Literally translated, it means &quot;dragon compete tiger fight&quot;. It belongs a family of well-formed idioms called &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chengyu&quot; rel=&quot;noreferrer&quot;&gt;Chengyu&lt;/a&gt;&quot;, which describes a situation where there is fierce fighting or competition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can see that neither translations fit.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why does Google Translate give me different translations on the same phrase?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Context matters when we read! So translating a phrase in isolation doesn't guarantee that the same phrase has the same meaning/translation in all other parts of the text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, green is a color, but the word &quot;green&quot; can also be used as like: &quot;Alice is green with envy&quot; or &quot;Bob has green thumbs&quot;, of which neither instances of the word &quot;green&quot; refers to the color. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the technical side of things, Google translate probably uses some kind of RNN in its pipeline. RNN's are influenced by past states, meaning that what it outputs now as a function of what it reads in is dependent on the RNN's past state. Which is similar to the issue addressed above.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-01-13T23:48:09.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4996" PostTypeId="2" ParentId="4989" CreationDate="2018-01-14T00:39:40.317" Score="6" Body="&lt;p&gt;Backpropagation is a subroutine often used when training Artificial Neural Networks with a Gradient Descent learning algorithm. Gradient Descent requires computation of the error gradient, i.e. derivatives, of a cost function with respect to the network parameters. BP allows you to find this gradient a lot faster than using naive methods. Reinforcement Learning refers to inferring &quot;optimal&quot; behavior, i.e. a strategy, of an agent maximizing some goal in an environment. Depending on the specific RL algorithm, BP may be employed to adjust parameters of a function approximator used to represent aspects of the environment and or the agent.&lt;/p&gt;&#xA;" OwnerUserId="12101" LastActivityDate="2018-01-14T00:39:40.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4997" PostTypeId="2" ParentId="4988" CreationDate="2018-01-14T00:49:38.627" Score="3" Body="&lt;p&gt;Just use one network with a larger Softmax output layer and more hidden units. If you have enough training data, it will work just fine. In fact it could emulate the architecture you propose.&lt;/p&gt;&#xA;" OwnerUserId="12102" LastActivityDate="2018-01-14T00:49:38.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4998" PostTypeId="2" ParentId="4987" CreationDate="2018-01-14T07:27:43.137" Score="3" Body="&lt;p&gt;To have a &quot;maximum achievable intelligence&quot;, first of course you have to define &quot;intelligence&quot; well enough to be able to rank things by intelligence. There is no widely-supported theory that is able to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might like to look into AIXI &lt;a href=&quot;https://www.youtube.com/watch?v=x8btbKaRfoc4&quot; rel=&quot;nofollow noreferrer&quot;&gt;as described by Marcus Hutter in a video lecture&lt;/a&gt;. It is an attempt to formalise intelligent agents that attempt to make optimal decisions mathematically. &lt;a href=&quot;https://jan.leike.name/AIXI.html2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is another written introduction&lt;/a&gt;. Of course this is only one of many possible frameworks to describe intelligent agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting implication is that AIXI implies intelligence - in terms of ability to learn from and exploit an environment - is upper bounded. In principle there is a ceiling due to uncertainty about what the data that a rational agent possesses might infer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this ceiling refers to only specific abilities to extract actionable information from data that the agent has access to in order to solve decision problems. There is an open question about how much data can be collected, stored and processed by any entity, and this ability to acquire and retrieve relevant knowledge would be viewed by many as part of the &quot;intelligence score&quot; when comparing agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Limits_of_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;theoretical limits to computation from physics&lt;/a&gt;, e.g. some are based on the fact that it fundamentally requires energy, the energy has a mass equivalence, and enough concentrated mass would form a black hole. This sets a high upper bound, and it is likely that real-world structural and design issues will set in way before this limit. However, combined with the above limits on decidability and practical access to data, it does seem there should be a ceiling.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-01-14T07:27:43.137" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4999" PostTypeId="2" ParentId="4965" CreationDate="2018-01-14T08:31:26.910" Score="5" Body="&lt;p&gt;Firstly, before we commence I recommend that you refer to similar questions on the network such as &lt;a href=&quot;https://datascience.stackexchange.com/questions/25053/best-practical-algorithm-for-sentence-similarity&quot;&gt;https://datascience.stackexchange.com/questions/25053/best-practical-algorithm-for-sentence-similarity&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/62328/is-there-an-algorithm-that-tells-the-semantic-similarity-of-two-phrases&quot;&gt;https://stackoverflow.com/questions/62328/is-there-an-algorithm-that-tells-the-semantic-similarity-of-two-phrases&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine the similarity of sentences we need to consider what kind of data we have. For example if you had a labelled dataset i.e. similar sentences and disimilar sentences then a straight forward approach could have been to use a supervised algorithm to classify the  sentences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approach that could determine sentence structural similarity would be to average the word vectors generated by word embedding algorithms i.e word2vec. These algorithms create a vector for each word and the cosine similarity among them represents semantic similarity among words. (Daniel L 2017)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using word vectors we can use the following metrics to determine the similarity of words. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cosine distance between word embeddings of the words&lt;/li&gt;&#xA;&lt;li&gt;Euclidean distance between word embeddings of the words&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cosine similarity is a measure of the similarity between two non-zero vectors of an inner product space that measures the cosine of the angle between them. The cosine angle is the measure of overlap between the sentences in terms of their content. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Euclidean distance  between two word vectors provides an effective method for measuring the linguistic or semantic similarity of the corresponding words. (Frank D 2015)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you could calculate the eigenvector of the sentences to determine sentence similarity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigenvectors are a special set of vectors associated with a linear system of equations (i.e. matrix equation). Here a sentence similarity matrix is generated for each cluster and the eigenvector for the matrix is calculated. You can read more on Eigenvector based approach to sentence ranking on this paper &lt;a href=&quot;https://pdfs.semanticscholar.org/ca73/bbc99be157074d8aad17ca8535e2cd956815.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://pdfs.semanticscholar.org/ca73/bbc99be157074d8aad17ca8535e2cd956815.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For source code Siraj Rawal has a Python notebook to create a set of word vectors. The word vectors can then be used to find the similarity between words. The source code is available here &lt;a href=&quot;https://github.com/llSourcell/word_vectors_game_of_thrones-LIVE&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/llSourcell/word_vectors_game_of_thrones-LIVE&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is a tutorial from Oreily that utilizes the gensin Python library to determine the similarity between documents. This tutorial uses NLTK to tokenize then creates a tf-idf (term frequency-inverse document frequency) model from the corpus.  The tf-idf is then used to determine the similarity of the documents. The tutorial is available here &lt;a href=&quot;https://www.oreilly.com/learning/how-do-i-compare-document-similarity-using-python&quot; rel=&quot;noreferrer&quot;&gt;https://www.oreilly.com/learning/how-do-i-compare-document-similarity-using-python&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-14T19:32:47.363" LastActivityDate="2018-01-14T19:32:47.363" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5000" PostTypeId="1" AcceptedAnswerId="5007" CreationDate="2018-01-14T12:43:05.573" Score="2" ViewCount="1419" Body="&lt;p&gt;I am new to neural networks, I've only started studying and learning about the subject a year ago, and I just started building my first neural network.&lt;/p&gt;&#xA;&lt;p&gt;The project is a little bit ambitious: A browser extension for children's safety,  it checks for sexual or abusive content, so that it replaces that content with a placeholder, the user will have to insert a password to show original content.&lt;/p&gt;&#xA;&lt;p&gt;I didn't find a dataset online, so I decided to build my training dataset. So, I started by writing a web crawler, it starts collecting images, meanwhile implementing data augmentation techniques. It basically resizes images (to 95x95), crops them, rotates, changes colors, adds blur, black and white, noise, etc.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that after applying these techniques, I noticed that some images are not even recognizable by a human subject.&lt;/p&gt;&#xA;&lt;p&gt;I mean that even though I know that picture contains sexual content, it doesn't even appear to be sexual anymore.&lt;/p&gt;&#xA;&lt;p&gt;So, do I have to label it as sexual or not sexual?&lt;/p&gt;&#xA;&lt;p&gt;Notice that it's easier for me to consider it as sexual, if every image produces about 50 edited images, I'd only have to label the original image, what follows is that all 50 images get the same label. Is it okay to do just that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ea2Fz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ea2Fz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a sample of what I get after doing data augmentation, notice that some pictures are not recognizable by humans.&lt;/p&gt;&#xA;&lt;p&gt;For example, look at the result after editing images hue and saturation, a human can't recognize this result, is it okay to label it: not sexual?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M5SPe.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M5SPe.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PjJWy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PjJWy.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I wouldn't recognize the picture on the right if I didn't see the original one.&lt;/p&gt;&#xA;&lt;p&gt;I also tested this on human subjects (my brothers), they didn't recognize the squirrel on the right.&lt;/p&gt;&#xA;" OwnerUserId="12113" LastEditorUserId="2444" LastEditDate="2020-12-20T12:37:32.233" LastActivityDate="2020-12-20T12:37:32.233" Title="How to label edited images after data augmentation?" Tags="&lt;neural-networks&gt;&lt;data-preprocessing&gt;&lt;data-augmentation&gt;&lt;data-labelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5003" PostTypeId="2" ParentId="15" CreationDate="2018-01-14T16:51:19.393" Score="3" Body="&lt;p&gt;It depends on how the test is given. For example, when people claimed that a machine had successfully passed the Turing Test a few years ago, the criteria was pretty weak. It only had to fool 30% of the people for 5 minutes. That's not much of a test. To put this in perspective you probably wouldn't detect schizophrenia, autism, learning disabilities, or dementia with this criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In spite of the hype, the current AI's can be detected 100% of the time using fairly simple questions.&lt;/p&gt;&#xA;" OwnerUserId="12118" LastActivityDate="2018-01-14T16:51:19.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5005" PostTypeId="1" AcceptedAnswerId="5028" CreationDate="2018-01-14T19:11:56.920" Score="2" ViewCount="150" Body="&lt;p&gt;I was watching a documentary on Netflix about AlphaGo, and at one point (~1:10:16 from the end), one of the programmers uses the term &quot;heavy node,&quot; which I assume has to do with neural networks. I did a little bit of research but couldn't find anything on what that term means. The closest I could get was this wikipedia page on Heavy path decomposition: &lt;a href=&quot;https://en.wikipedia.org/wiki/Heavy_path_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Heavy_path_decomposition&lt;/a&gt;, which seemed like it could be somewhat related, but I wasn't sure how exactly. Has anyone heard of this term being used? Does anyone know what it means? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For context, in the documentary the line is that if it (the network/player) creates something new not in the heavy node, then they don't see it. &lt;/p&gt;&#xA;" OwnerUserId="12119" LastActivityDate="2018-01-16T16:31:57.280" Title="What is a heavy node in neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5006" PostTypeId="2" ParentId="4770" CreationDate="2018-01-14T19:26:09.207" Score="1" Body="&lt;p&gt;The NVidia Volta architecture is capable of delivering an astounding 110 teraflops of computing power. This coupled with the NVidia CUDA Neural Network library (cuDNN). Which can be used to speed up DNN's on learning frameworks such as  Caffe 2, Microsoft Cognitive Toolkit, TensorFlow, Theano, and PyTorch. Makes GPU's based on the Volta architecture such as the Titan V specifically suited to accelerate AI workloads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your MacBook Air. Unfortunately, it comes with an integrated GPU that can't be replaced because it is soldered to the logic board. So, unfortunately, we have to rule out upgrading it because there isn't any other port to connect a GPU. You can read more about this on Apple's official discussion forum &lt;a href=&quot;https://discussions.apple.com/thread/5206086&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://discussions.apple.com/thread/5206086&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, you possibly can upgrade your GPU within the same series which is the 10 series. Other powerful GPU's within the 10 series include The Titan X, Titan Xp, GeForce GTX 1070 Ti and the GeForce GTX 1080 Ti. Unfortunately, the Geforce 10 series is based on the Pascal architecture meaning your laptop might not by default accept a Volta based GPU such as the Tesla V100 or the Tesla Titan V. Consequently it could involve BIOS modification, chassis modification or heatsink assembly modification to make it work. This kind of upgrade will be diving into uncharted territory since it will void your warranty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is that a Volta based GPU might be unsuitable for your Asus laptop. I'd recommend a GPU that uses an architecture that is similar to your current GPU. Meaning a Pascal-based GPU such as the Titan X or GeForce GTX 1080 would be better suited to satisfy your demands. &lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="11061" LastEditDate="2018-05-01T20:57:30.840" LastActivityDate="2018-05-01T20:57:30.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5007" PostTypeId="2" ParentId="5000" CreationDate="2018-01-14T19:36:06.350" Score="1" Body="&lt;p&gt;Yes, you should label it the same. But more importantly you need to make sure that each perturbation of the image doesn't change some important character of the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider training an apple classifier. If you plan to augment data by altering the RGB values, you need to be wary that you might cause issues in classification tasks where color is instrumental. Say, Granny apple / Fuji apple. If you still really want to augment data in this way, consider perturbing by smaller amounts each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or consider an apple detector. An apple still &quot;looks to be an apple&quot; if looked at an angle, if looked through a mirror, if looked afar, but probably not if looked through a carnival mirror.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So ask yourself, if an image still NSFW if the colors are changed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as a personal anecdote I don't think augmenting data by changing the color channel is a good idea. Also note that training on augmented data by altering the color channels as a whole should be more or less equivalent to training in B/W. Why?&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-01-14T19:36:06.350" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5009" PostTypeId="2" ParentId="3400" CreationDate="2018-01-15T04:34:51.127" Score="2" Body="&lt;p&gt;So you want your network to represent those 3 values at each step as single composite value? I can't think of any better way than utilizing 3 LSTM units but attaching them to same write and read nodes of the enclosing network. In other words your &lt;strong&gt;assumption&lt;/strong&gt; that it makes sense to keep all those 3 values together, gets &lt;strong&gt;hardcoded&lt;/strong&gt; into your network by making 6 connections (3 connected to read and 3 to write) share their weights. Usually researchers leave it to the network to decide whether to keep such composite values together or separate (by learning read and write weights through backpropagation) and sometimes network chooses to keep them together all on its own like in &lt;a href=&quot;https://youtu.be/93rzMHtYT_0?t=531&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/93rzMHtYT_0?t=531&lt;/a&gt; (it can be seen how network resets whole bunch of LSTM units simultaneously ). &lt;/p&gt;&#xA;" OwnerUserId="9803" LastActivityDate="2018-01-15T04:34:51.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5010" PostTypeId="1" AcceptedAnswerId="5038" CreationDate="2018-01-15T10:31:37.033" Score="0" ViewCount="90" Body="&lt;p&gt;In recent years, China has made rapid progress in manufacturing and scientific research as evidenced by their successful teleportation of a single quantum entangled photon to a satellite in orbit. My question is, what major contributions have Chinese AI researchers made in the field of Artificial Intelligence?&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-15T18:17:17.790" LastActivityDate="2018-01-17T03:35:52.357" Title="What noteworthy contributions have Chinese AI researchers made in the field of artificial intelligence?" Tags="&lt;ai-design&gt;&lt;research&gt;&lt;history&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5011" PostTypeId="1" CreationDate="2018-01-15T13:02:15.013" Score="8" ViewCount="6306" Body="&lt;p&gt;I am trying to develop a machine learning algorithm to identify topological features within 3D CAD models (i.e. slots, pockets, holes, bosses etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the input data I have decided to use the adjacency of the faces to identify the features within the model. I have developed a pre-processing algorithm which computes the adjacency between every face in the model and also whether the relationship is concave or convex in nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is an example of a 3D model and the relationships between each faces represented in a graph format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IU10S.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IU10S.png&quot; alt=&quot;Example of face adjacency within a part&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of using some sort of supervised learning training method where the training data would include all of the adjacency info for the model with labels defining the features that exist within the model such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slot - made up of faces F7, F8, F9&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hole - made up of faces F15&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pocket - made up of faces F10, F11, F12, F13&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And eventually once the model is sufficiently trained it would be able to identify the features in an un-seen model and determine which faces make up those features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how I would go about pre-processing the input data (i.e. it would be of variable length since the number of faces may not be the same from model to model). I am also struggling to understand which type of machine learning algorithm would be best suited to this application&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help on this problem or even pointing to some resources to help understand would be appreciated&lt;/p&gt;&#xA;" OwnerUserId="12125" LastActivityDate="2020-04-06T13:58:07.837" Title="Using machine learning to identify CAD model features" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5012" PostTypeId="2" ParentId="3472" CreationDate="2018-01-15T13:03:45.860" Score="1" Body="&lt;p&gt;Initially, spam detection relied on simple rule based techniques to sort out spam. However following Paul Graham's famed article 'A Plan for Spam'  the Naive Bayes approach became very popular to the point that it became regarded as the baseline for dealing with spam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However following breakthroughs in deep learning, researchers have now turned their focus to neural networks to help them deal with the perenial problem of spam emails. Google recently reported that by introducing NN's to g-mail's spam filters. It took them from 99.5% to over 99.9% accuracy, suggesting that neural networks especially when used in conjunction with Bayesian classification may be effective for enhancing spam filters. You can refer to the link below to read about Google's success story &lt;a href=&quot;https://www.wired.com/2015/07/google-says-ai-catches-99-9-percent-gmail-spam/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wired.com/2015/07/google-says-ai-catches-99-9-percent-gmail-spam/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing a spam filter using neural networks is basically a classification problem. You need to follow the steps below to develop such a system. (Nikhil B 2016)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Collect a dataset of spam and legitimate email messages. Label these datasets. You can find email and spam datasets here &lt;a href=&quot;http://csmining.org/index.php/spam-email-datasets-.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://csmining.org/index.php/spam-email-datasets-.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Process these messages with feature extraction and vectorising techniques i.e. tf-idf vectorizer, word2vec, bag-of-words e.t.c. &lt;/li&gt;&#xA;&lt;li&gt;Once you have vectorised the dataset succesfully, apply a supervised learning NN algorithm i.e. radial basis network, multi-layer perceptron (MLP) or backpropagation.&lt;/li&gt;&#xA;&lt;li&gt;Train your labelled data-set on the neural network. Once training is complete you can use cross validation to calculate the precision of your trained model using the test dataset. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some of the advantages of using NN's for spam detection over other methods include.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neural networks have a higher accuracy of identifying spam as demonstrated by google. &lt;/li&gt;&#xA;&lt;li&gt;They have a lower false positive rate compared to other methods such as rule based techniques.&lt;/li&gt;&#xA;&lt;li&gt;Their main disadvantage is that they require specialised computing hardware to deploy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some old influential papers published in the field include.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine Learning Techniques in Spam Filtering (2004) &lt;a href=&quot;http://ats.cs.ut.ee/u/kt/hw/spam/spam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ats.cs.ut.ee/u/kt/hw/spam/spam.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detecting Spam Blogs: A Machine Learning Approach (2006) &lt;a href=&quot;https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A review of machine learning approaches to Spam filtering (2009) &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S095741740900181X&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S095741740900181X&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-15T19:49:50.157" LastActivityDate="2018-01-15T19:49:50.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5013" PostTypeId="2" ParentId="5011" CreationDate="2018-01-15T17:51:11.610" Score="4" Body="&lt;p&gt;This is a classification problem. Look at some common classification methods. This article by Microsoft gives some pretty good guidance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2018-01-15T17:51:11.610" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5016" PostTypeId="2" ParentId="4624" CreationDate="2018-01-15T19:46:03.067" Score="2" Body="&lt;p&gt;Although the above statement holds important analogies to communicate the technical advances made by deep mind in the development of Alpha Go. It is inaccurate and should be taken skeptically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, although Alpha go was trained on specialized hardware such as high end NVidia GPU's and custom google TPU's. It should be noted that it can run on a regular desktop although it won't be as powerful as the distributed version of Alpha Go. Additionally anyone with a laptop can access a similar amount of computing resources in the cloud with the touch of a button.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although a version of Alpha Go indeed ran on 1202 CPUs and 176 high end GPU's as reported by Nature magazine. Which roughly translates to a system 1000 times as powerful as a commodity laptop. We need to consider other important factors such as moores law which postulates that we could have a computer as fast as 1202 CPU's in our pockets within 20 years and not several millennia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the latest version of Alpha Go, Alpha Go Zero, trained on a single server with only 4 TPU's. Considering that the latest generation of smartphones such as iPhone X and Huawei's Kirin come packed with specialised AI chips. I will not be surprised if a similar reduction of form factor is achieved in commodity desktop computers once new models packed with AI chips are introduced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I respect and acknowledge the technical achievements of Demis Hassabis and the deep mind team in developing a system as powerful as Alpha Go. However I believe analogies such as the one used above are inaccurate and mis-represent facts.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-15T19:46:03.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5018" PostTypeId="1" CreationDate="2018-01-16T06:20:51.670" Score="5" ViewCount="244" Body="&lt;p&gt;I am not looking for proof of AI (too broad, I know) concepts trying to solve a specific business problem, instead for a &lt;strong&gt;proof (or dis-proof) or a model&lt;/strong&gt; (or a source of the same) where a machine is capable of &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;General thinking and decision making.&lt;/li&gt;&#xA;&lt;li&gt;Collecting data as it see fit (use any channel at their disposal).&lt;/li&gt;&#xA;&lt;li&gt;Communicate as it see fit (with whom, how, where and when).&lt;/li&gt;&#xA;&lt;li&gt;Develop and evolve an agenda.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; - I wrote an article more than 5 years ago - &lt;a href=&quot;https://www.experts-exchange.com/articles/10609/Why-a-software-cannot-be-intelligent.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;why a software cannot be intelligent&lt;/a&gt;. Since then I have not read a convincing argument against it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Intelligence is the ability to take a decision in the unseen, unfamiliar, and unvisited circumstances&lt;/em&gt;. So, my definition of Artificial intelligence is - &lt;em&gt;a machine &lt;strong&gt;doing&lt;/strong&gt; essentially the same&lt;/em&gt; (doing, not replicating or simulating). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a proof or disproof that machine is capable of doing things (written above), not the following&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Instances where machine as simulated a certain aspect or property of intelligence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Machine replicating a certain human behavior to achieve a goal. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12138" LastEditorUserId="1671" LastEditDate="2018-01-17T17:26:11.910" LastActivityDate="2018-03-31T15:28:07.313" Title="Has it been theoretically proven or dis-proven that a machine can generally think and communicate on its own?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;theory&gt;" AnswerCount="3" CommentCount="4" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5019" PostTypeId="1" CreationDate="2018-01-16T07:11:01.590" Score="1" ViewCount="1205" Body="&lt;p&gt;I have a pedestrian dataset and would like to estimate human height in a video survillance using person detection techniques like YOLO Darknet or SSD (Single Shot Detectors). Would this technique work? Also, the videos that I have are in a constrained environment with good illumination. The idea is to get the coordinates from the bounding box and try to estimate pixel height. After getting the pixel height, some correlation could be estimated between pixel height and real world height. Note that I won't be using camera calibration. &lt;/p&gt;&#xA;" OwnerUserId="11800" LastEditorUserId="11800" LastEditDate="2018-01-16T09:49:37.117" LastActivityDate="2019-03-14T20:15:30.843" Title="Human Height estimation using person detection techniques" Tags="&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5020" PostTypeId="2" ParentId="4076" CreationDate="2018-01-16T08:42:45.443" Score="1" Body="&lt;p&gt;This task falls within the overlapping fields of information extraction and pattern mining. Information extraction involves automatically extracting instances of specified relations from data. While pattern mining involves using data mining algorithms to discover interesting, unexpected and useful patterns between data in databases (Philippe F).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On your question you have stated that you have experimented with markov models with poor results. A better approach if you prefer working with markov models would be to use hierarchical markov models. Hierarchical markov models have multiple 'levels' of states which can describe input sequences at different levels of granularity. Hierarchical markov models are good at categorizing human behavior at various levels of abstraction i.e. a persons location in a room can be further interpreted to determine more complex information such as what activity the person is performing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However my recommendation is that you implement random forest classifiers for this problem. Random forests provide excellent classification accuracy with a relatively simple implementation. Additionally random forests provide the ability to inspect trees for tweaking parameters to improve accuracy. You can also use cross validation to evaluate your model and calculate its accuracy. Consider using Python's Scikit-learn library's implementation of random forest classifier for this analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your json code action package you have declared metrics such as DRAG_START, TRASH_SYMBOL, OPEN and CLOSE. My suggestion is that for your model to be accurate you also need to declare lower level actions such as: time between clicks, change in the direction of mouse motion, screen region hover count, task completion time and time between a click and a succeeding mouse movement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further reference I recommend that you look at the papers below which I found useful and relevant to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hierarchical Hidden Markov Models for Information Extraction &lt;a href=&quot;https://www.biostat.wisc.edu/~craven/papers/ijcai03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostat.wisc.edu/~craven/papers/ijcai03.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detecting Abnormal User Behavior Through Pattern-mining Input Device Analytics &lt;a href=&quot;https://www.ignacioxd.com/files/bib/Dominguez2015-Concentration.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ignacioxd.com/files/bib/Dominguez2015-Concentration.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hierarchical Hidden Markov Model in Detecting Activities of Daily Living in Wearable Videos for Studies of Dementia   &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1111/1111.1817.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/ftp/arxiv/papers/1111/1111.1817.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-16T08:48:29.200" LastActivityDate="2018-01-16T08:48:29.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5021" PostTypeId="2" ParentId="5019" CreationDate="2018-01-16T09:13:39.643" Score="2" Body="&lt;p&gt;Eh, &quot;I won't be using camera calibration.&quot; .. Not sure, what you mean.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;1st&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;At first, imagine a sheet of paper laying there on the floor. And first to try to transform the paper sheet (i.e. with some text on it), the picture of the sheet angled by paralaxe to the straight view: Just a transformation by a matrix. It would be highly valuable to have there some floor of square tiles. And once you would estimate/set size of the tiles on the floor, you can then transform the paper sheet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, also some vertical calibration will be needed: Wherever within the scene. A wall of tiles of known shape &amp;amp; size would be perfect. But not really necessary: Any known/premeasured chair would be enough! ..rather two such chairs, in an opposite nooks, for numerical stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, that could be treated as &quot;camera calibration&quot;. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;2nd&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Then you can advance to the next level of the image processing:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Once you see feet of the person, on which tile of the floor the person stays,&lt;/li&gt;&#xA;&lt;li&gt;and with the known vertical direction,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;by the hid/covered known background, you can decide their height, pretty precisely, yes.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The &quot;known background&quot;&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Just two images should be enough, of the background place: empty non-occupied, and hid/covered. The clear-empty room can be taken either onence, or periodically, or even just a partial image should be enough: just to detect shape of the person-figure.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;3rd&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;And the third level: Really high. The person could stay in some deformed position. By interpolation of the known human skeleton to the shape of the person-figure on the screen, you can assume the curves of the skeleton of the person, so your estimated/measured total height of the person would be immune to the non-strightened angles in joints of their limbs/backbone. So even if the person knows about your observing, the AI would not get fooled. But you would need/find more refencial points on their body, to interconnect the 2D shape to the internal 3D model.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;quantity of snaps in general&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;More snaps of the person, as they walk through the room, would be helpful: Any model, set up by any single image, should be verifiable/applicable to each of the rest of the images of the same person. Again: stability and &quot;back testing&quot;, verifications, robustness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are not limited to just a single image, you can have as many as your camera is able to make. So use them, to get not only the measured size, but also to level up the &quot;probability&quot; of the measured result, the accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, you can also flip the coin, and to try to guess the maximum info from the minimum snaps of the scene. But it does not seem, this would be your case. Still: Processing 24 fps recording 20 seconds long.. That definitely requires some power/time to process.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Summary&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The 3D models are necessary: Of the surrounding space, of the outer shapes of the person-figure, of the internal structure of the measured body.&lt;/p&gt;&#xA;" OwnerUserId="8537" LastEditorUserId="8537" LastEditDate="2019-03-14T20:15:30.843" LastActivityDate="2019-03-14T20:15:30.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5022" PostTypeId="1" CreationDate="2018-01-16T10:17:43.680" Score="4" ViewCount="606" Body="&lt;p&gt;Due to recursive self-improvement, AI could lead to an intelligence explosion improving on itself year over year exponentially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the proper environment was created to allow an AI to self-improve, how fast would this occur?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With human intervention we might say AI could improve similar to GDP or science growth say 3% per year. But this intervention would be additive to the self-recurring improvements made by the AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is a reasonable annual rate of self-growth and what would the processing power of the AI be after 10 years given an initial value of say 1PFLOP?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;human rate = 3% annually&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI rate = 1% of current processing power annually&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Year 1. 1 PFLOP&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Year 2. 1 PFLOP + (1 PFLOP * 3%) + (1 PFLOP * 1%) = 1.04 PFLOP&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Year 3. 1.04 PFLOP + (1.04 PFLOP * 3%) + (1.04 PFLOP * 1.04%) = 1.082 PFLOP&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Year 4. 1.082 PFLOP + (1.082 PFLOP * 3%) + (1.082 PFLOP * 1.082%) = 1.126 PFLOP&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Year 5. 1.126 PFLOP + (1.126 PFLOP * 3%) + (1.126 PFLOP * 1.126%) = 1.172 PFLOP&lt;/p&gt;&#xA;" OwnerUserId="12139" LastEditorUserId="1671" LastEditDate="2018-01-17T01:20:43.730" LastActivityDate="2019-01-04T05:17:30.303" Title="At what rate could AI theoretically self-improve?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;unsupervised-learning&gt;&lt;theory&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5023" PostTypeId="2" ParentId="5022" CreationDate="2018-01-16T12:51:19.830" Score="2" Body="&lt;p&gt;The processing power might still follow the Moore's law, but it is basically an economical law, although it has worked surprisingly well considering the physical limitations of technology. An Artificially Intelligent agent could focus on manipulating the market, research possible hardware improvements, allocate human resources more effectively to the problem, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect some improvement in the AI algorithm design itself, which will not influence the number of operations available, but the number of operations required to complete a task. By using a better heuristics, the AI algorithm will consume less brute force to complete the task, thus being able to complete more tasks given the same number of operations. Sorry if I misunderstood the question. Thanks for the improvement. &lt;/p&gt;&#xA;" OwnerUserId="12141" LastEditorUserId="12141" LastEditDate="2018-01-16T14:28:23.277" LastActivityDate="2018-01-16T14:28:23.277" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5024" PostTypeId="2" ParentId="4964" CreationDate="2018-01-16T13:10:05.903" Score="1" Body="&lt;p&gt;The difference between a Speech API and a Speech Engine is: Speech API's enable developers to integrate speech recognition technologies into developer apps. On the other hand a speech engine is software that gives your computer the ability to play back text in a spoken voice. (Source msdn library)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a list of speech recognition tool-kits and their features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow - Although tensorflow doesn't arrive packaged with speech recognition libraries by default. You can use seq2seq models which have achieved high levels of accuracy in speech recognition. A few of the advantages of using tensorflow for speech recognition include: It comes with tensorboard which is useful in visualising and fine-tuning your network, it's architecture is highly modular which allows you to experiment with different voice libraries and finally it is portable meaning you can run it on GPU's, CPU's, servers or even mobile computing platforms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Microsoft's CNTK - Microsofts Cognitive Toolkit delivers excellent results when it come to speech recognition. In recent times CNTK has outperformed humans in transcribing speech to text. Some of its perks are its efficient resource usage additionally it was originally built for speech recognition systems and consequently it is very effective at working with time series data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CMU Sphinx - CMU Sphinx is a speech recognition system developed at Carnegie Mellon University. The advantages of using CMU Sphinx are: it is multilingual and supports most international languages, it has excellent commercial support, it has a light mobile version called pocketsphinx, it has a wide range of tools for different purposes i.e. keyword spotting, alignment and pronunciation evaluation. It also enjoys active support from the Carnegie Mellon University.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kaldi - Kaldi aims to provide speech recognition software that is flexible and extensible. Kaldi has powerful features such as pipelines that are highly optimized for parallel computing i.e. training models on the GPU. Additionally it supports speaker identification and detection of errors in transcripts.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mozilla Deep Speech - This project aims at providing speech interfaces for the web. This project achieved a word error rate on LibriSpeech's test-clean of 6.5% which is commendable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this paper comparing open source speech recognition toolkits to be relevant to your question &lt;a href=&quot;http://suendermann.com/su/pdf/oasis2014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://suendermann.com/su/pdf/oasis2014.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Siraj Raval has an excellent tensorflow speech recognition tutorial. It is available here &lt;a href=&quot;https://github.com/llSourcell/tensorflow_speech_recognition_demo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/llSourcell/tensorflow_speech_recognition_demo&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-16T13:23:25.913" LastActivityDate="2018-01-16T13:23:25.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5025" PostTypeId="2" ParentId="5018" CreationDate="2018-01-16T14:11:09.503" Score="0" Body="&lt;p&gt;General thinking seems as the toughest requirement to me. An AI looks at a new problem, tries to remember the most similar problem with a known solution and apply a similar solution to the new problem. It might try to guess how big mistake it has made and try to modify some parameters of the solution randomly (pseudo-randomly unless given a random or chaotic enough input such as a single wild animal behaviour or the weather). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network given a hardware comparable in performance and storage capacity to the human brain and an insane quantity of input (e.g. the whole Internet including all attachable input devices) could probably be trained to mimic human general thinking believably enough. Some thinkers suggested that a human brain is basically just a very advanced machine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My summer jobs made agree - as a postman, I performed better when I let my fingers find the mailbox not searching for it, and so did I as cashier, when my fingers would type in the correct code for the bakery products, fruits or vegetables (no barcode) without me even noticing what the goods were sometimes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might be some dis-proof based on an assumption that an AI is coded by humans using formal logic known to have some theoretical limits, which might be what you said in &quot;Does software evolve?&quot; in your article (nice job, by the way). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It almost becomes a religious question once you allow machines / algorithms to &quot;breed&quot; and &quot;compete&quot; as wildlife animals - would a long but finite time in an environment rich in resources and analog inputs be enough for an AI to evolve providing a finite set of environment parameters (gravitational constant etc.)? Reminds me of Just six numbers by Martin Rees. &lt;/p&gt;&#xA;" OwnerUserId="12141" LastActivityDate="2018-01-16T14:11:09.503" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="5026" PostTypeId="2" ParentId="4991" CreationDate="2018-01-16T14:56:19.493" Score="1" Body="&lt;p&gt;Google uses user input to improve translation. Some user may have provided an input to the Traditional Chinese characters using English characters only instead of pinyin, which would introduce a mistake into the data used by the translator. Since the model is statistics based, such a mistaken translation can only be assigned a lower but non-zero probability of correctness, but to erase it from the system entirely you would probably have to do that by hand or by introducing some general rule (e.g. if the probability is so small that it can not be stored in a variable of a given size, round it to zero). &lt;/p&gt;&#xA;" OwnerUserId="12141" LastActivityDate="2018-01-16T14:56:19.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5027" PostTypeId="1" CreationDate="2018-01-16T15:29:53.320" Score="3" ViewCount="78" Body="&lt;p&gt;For supervised learning, humans have to label the images computers use to train in the first place, so the computers will probably get wrong the images that humans get wrong. If so can computers beat humans?&lt;/p&gt;&#xA;" OwnerUserId="12145" LastEditorUserId="1671" LastEditDate="2018-01-17T00:57:58.863" LastActivityDate="2018-01-17T00:57:58.863" Title="How can computers beat humans at image recognition?" Tags="&lt;image-recognition&gt;&lt;theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5028" PostTypeId="2" ParentId="5005" CreationDate="2018-01-16T16:31:57.280" Score="4" Body="&lt;p&gt;&lt;em&gt;Yes, firstly you are correct to acknowledge the term node with reference to the neural network.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is made to mimic the working of a human brain hence the term neural. It comprises of several layers and each layer comprises of several nodes. A network can be a deep neural network, signifying the presence of numerous layers, in which the output of the one, serves as an input for the other.&#xA;Further a node in a particular layer has some weights and biases attached with it that are responsible for the computation and producing output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To your context, a heavy node may refer&lt;/strong&gt; to a node that holds importance for the network and has got significant weight and bias attached. Henceforth as per the statement if the network does make any significant change to the node with importance in terms of weights and biases, it is noticed else not.&lt;/p&gt;&#xA;" OwnerUserId="8041" LastActivityDate="2018-01-16T16:31:57.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5029" PostTypeId="2" ParentId="5027" CreationDate="2018-01-16T17:17:30.527" Score="2" Body="&lt;p&gt;When researchers claim &quot;better than human accuracy&quot;, they are demonstrating that a computer can beat an individual human on a test. And that is because the ground truth labels are actually higher accuracy than a &lt;em&gt;single&lt;/em&gt; human could label the images individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are at least two major ways that ground truth labels can beat an individual human on image tasks.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Additional information is available from the same source as the image. For instance many pictures of pets in the ImageNet database are labeled with a specific breed of animal, due to how they are sourced. Most people &#xA;who are not experts at pet breeds will score quite badly on a test to identify dog breeds at the fine grained level that ImageNet presents.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ground truth based on expert opinion can be sourced from multiple experts and their opinions combined. This approach can independently be shown to be more reliable than the opinion of a single person.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So in short yes computers can beat humans when they have had access to better original ground truth, and that is possible, even if that ground truth is generated by humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in general your concern stands. Ground truth data is a limiting factor. It might be possible in theory for a computer model to have an even better accuracy at the &quot;real&quot; task than the ground truth for a supervised learning task. However, this is next to impossible to prove, and other concerns, such as changes to distribution of real data as opposed to training data, are usually more important at that level of accuracy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-01-16T19:35:42.320" LastActivityDate="2018-01-16T19:35:42.320" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5030" PostTypeId="2" ParentId="5018" CreationDate="2018-01-16T23:52:29.707" Score="6" Body="&lt;p&gt;We have not been able to create a truly intelligent AI yet, according to your definition. So we have no real life proof-of-concept that shows that it actually works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But based on the current research, there is no known property of the human brain that cannot be modeled in software/hardware. We do not understand the human brain enough yet - and most likely lack the computational power and required storage - to create an artificial copy. But as long as we do not identify a property of the human brain that cannot be translated into a piece of software/hardware, it is most likely possible. I honestly cannot even imagine such a property within the bounds of physics that would create such a barrier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this is no formal proof, but without a clearly identified show-stopper there is no reason to believe that it is not possible to create a thinking machine.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2018-01-16T23:52:29.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5031" PostTypeId="5" CreationDate="2018-01-17T00:51:48.333" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-23T00:52:42.620" LastActivityDate="2021-01-23T00:52:42.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5032" PostTypeId="4" CreationDate="2018-01-17T00:51:48.333" Score="0" Body="For questions related to path-finding algorithms, such as A*." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-23T00:52:42.620" LastActivityDate="2021-01-23T00:52:42.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5033" PostTypeId="5" CreationDate="2018-01-17T01:01:35.797" Score="0" Body="&lt;p&gt;Derives from &lt;a href=&quot;https://en.wikipedia.org/wiki/Praxis_(process)#Origins&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aristotle&lt;/a&gt;, to distinguish from &lt;a href=&quot;https://en.wikipedia.org/wiki/Praxis_(process)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;praxis&lt;/em&gt;&lt;/a&gt; (thinking) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Poiesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;poiesis&lt;/em&gt;&lt;/a&gt; (making).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://en.oxforddictionaries.com/definition/theorics&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Theorics&quot;&lt;/a&gt; vs. &lt;a href=&quot;https://en.oxforddictionaries.com/definition/practic&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Practics&quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-17T01:01:35.797" LastActivityDate="2018-01-17T01:01:35.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5034" PostTypeId="4" CreationDate="2018-01-17T01:01:35.797" Score="0" Body="Use for questions on AI theory and other theoretical subjects related to AI.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-09-25T23:54:58.567" LastActivityDate="2018-09-25T23:54:58.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5035" PostTypeId="2" ParentId="4987" CreationDate="2018-01-17T01:20:07.647" Score="4" Body="&lt;p&gt;Absolutely, regardless of how you define &quot;intelligence&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If intelligence is merely information, as in &quot;a piece of intelligence&quot;, as in data, or an algorithm, the structure is finite.  (Structure, here, refers to the information, which may be reduced to a single string in either case.)  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turning Machine&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If intelligence is the rational capability of an automata, it is likewise bounded by the tractability of the decision problem, the structure of the algorithm, and the time available to make the decision. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bounded Rationality&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both answers are really the same, because &quot;intelligence&quot; in the first sense is limited by physical constraints on information density, sophistication of the algorithm, and time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational complexity of mathematical operations&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational complexity&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Time Complexity&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-17T17:20:57.327" LastActivityDate="2018-01-17T17:20:57.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5037" PostTypeId="2" ParentId="4988" CreationDate="2018-01-17T03:25:18.633" Score="0" Body="&lt;p&gt;I agree with the above answer. If you want to research this more in-depth look at this paper: &lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42241.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42241.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12145" LastActivityDate="2018-01-17T03:25:18.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5038" PostTypeId="2" ParentId="5010" CreationDate="2018-01-17T03:35:52.357" Score="0" Body="&lt;p&gt;Well, there's Baidu with former chief scientist Andrew Ng that's doing research on AI and the Microsoft Asia team has won some ImageNet competitions. &lt;/p&gt;&#xA;" OwnerUserId="12145" LastActivityDate="2018-01-17T03:35:52.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5041" PostTypeId="1" CreationDate="2018-01-17T11:53:21.057" Score="4" ViewCount="260" Body="&lt;p&gt;Many of the architectures that do semantic segmentation like SegNet, DilatedNet (Yu and Koltun), DeepLab, etc. do not work on high resolution images. For such benchmarks like &lt;a href=&quot;https://www.cityscapes-dataset.com/benchmarks/#pixel-level-results&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cityscapes&lt;/a&gt;, what is a standard/practical approach for such methods to perform on the benchmark?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to look into the paper, but I couldn't find such details. There's an &lt;a href=&quot;http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; mentioning that they output at 1/8 of input images than do interpolation (usually 2, 4 or 8 times) from their results, but the article does not specify which upsampling techniques are the most reasonable one.&lt;/p&gt;&#xA;" OwnerUserId="3098" LastEditorUserId="3098" LastEditDate="2018-01-17T12:03:30.943" LastActivityDate="2018-05-28T03:56:36.300" Title="Semantic Segmentation how to upsampling" Tags="&lt;deep-learning&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5042" PostTypeId="1" CreationDate="2018-01-17T12:49:10.813" Score="1" ViewCount="58" Body="&lt;p&gt;I've been studying a recommender system which uses a collaborative deep learning approach and Bayesian learning. It has the following NN representation :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p2veT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p2veT.jpg&quot; alt=&quot;sdae&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to know the working of stacked denoising autoencoders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link to the paper:&lt;a href=&quot;http://www.wanghao.in/paper/KDD15_CDL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wanghao.in/paper/KDD15_CDL.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10118" LastEditorUserId="16565" LastEditDate="2019-04-04T20:38:39.597" LastActivityDate="2019-04-04T20:38:39.597" Title="How do stacked denoising autoencoders work" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;learning-algorithms&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5043" PostTypeId="1" CreationDate="2018-01-17T15:50:13.960" Score="1" ViewCount="128" Body="&lt;p&gt;What are the connections between ethics and artificial intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the issues that have arisen, especially in the business context? What are the issues that may arise?&lt;/p&gt;&#xA;" OwnerUserId="12171" LastEditorUserId="2444" LastEditDate="2020-02-22T11:58:29.533" LastActivityDate="2020-02-22T12:00:14.683" Title="What are the connections between ethics and artificial intelligence?" Tags="&lt;philosophy&gt;&lt;ethics&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5044" PostTypeId="2" ParentId="5043" CreationDate="2018-01-17T16:37:32.637" Score="1" Body="&lt;p&gt;This is a good related read from Nature: &lt;a href=&quot;https://www.nature.com/polopoly_fs/1.20805!/menu/main/topColumns/topLeftColumn/pdf/538311a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;There is a Blind Spot in AI Research&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fears about the future impacts of Artificial Intelligence are distracting researchers from the real risks of deployed systems&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2020-02-22T11:56:16.327" LastActivityDate="2020-02-22T11:56:16.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5045" PostTypeId="2" ParentId="4987" CreationDate="2018-01-17T19:07:05.817" Score="0" Body="&lt;p&gt;No. There is no ceiling to intelligence. However, I am applying this loosely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you consider the intelligence of a person, you generally think of some baseline IQ that ranks that person on a scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Per the definition of our &quot;Salable&quot; IQ, 200 is considered being a (nearly) unbelievable intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when you look at &quot;Entities&quot;, the scale disappears. Google, NSA, Universities - they could all considered super-intelligence. There's no possible metric or IQ to assign to these entities that we know of because they aren't necessarily comparable. However, no single person is smart enough to invent all that was necessary to bring an entity and it's &quot;intelligence&quot; into fruition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you bring AI into the equation, your limited by your resources. AI is proving more and more capable as we evolve the technology and we understand our data better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We may find in the future that GO isn't all that infinite and there is a &quot;Best&quot; strategy (thus we solve the game). However, we keep on making machines that are smarter than previous ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't be surprised if AI starts making paper companies soon per business tactical advantages. &lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2018-01-17T19:07:05.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5046" PostTypeId="2" ParentId="4987" CreationDate="2018-01-17T19:10:32.477" Score="0" Body="&lt;p&gt;Actually, these aspects is part of some books I am working on right now.&#xA;Like what Jeevan say it is bound by laws of physics. I see that too, but in case of the Human, in the way i look at it, we are at the very lowest step/beginning of an AI that can do self reflecting and question its own ability to do rational thinking and how far it can go and develop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I also use the Black Hole - maximum density, maximum energy concentration as a 1'st upper limit. A intelligence who can operate at Plank Level, can store all information in this world in like a few sand corns size. So given enough density, optimization in information exchange/methods, the upper limit is far far away from what humans process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I take the analyze further and go beyond the Black hole theory. Of course it is speculation, but we still don't know all. So what is possible in more than 4 dimension (human brain, 3 dimension and time to think).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it, in real world, upper limit is so far away from our understanding, that we still can't handle it.&lt;/p&gt;&#xA;" OwnerUserId="11827" LastActivityDate="2018-01-17T19:10:32.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5047" PostTypeId="2" ParentId="2967" CreationDate="2018-01-17T22:37:58.363" Score="1" Body="&lt;p&gt;MPD are not independent from the past, but future actions starting from current state are independent from the past i.e. the probability of the next state given &lt;strong&gt;all&lt;/strong&gt; previous states is the same as the probability of the next state given &lt;strong&gt;the&lt;/strong&gt; previous state.&lt;/p&gt;&#xA;&lt;p&gt;Any state representation which is composed by the full history is a MDP, because looking at the history (coded in your state) is not the same as looking back previous states, so Markov property holds. The problem here is that you will have an explosion of states, since you need to code in the state any possible trajectory, and it is unfeasible most of the times.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What if I define my state as the current &amp;quot;main&amp;quot; state + previous decisions?&lt;/p&gt;&#xA;&lt;p&gt;For Example in Poker the &amp;quot;main&amp;quot; state would be my cards and the pot + all previous information about the game.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes it is a &lt;strong&gt;Markov Decision Problem&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12121" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-01-17T22:37:58.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5048" PostTypeId="1" AcceptedAnswerId="5061" CreationDate="2018-01-18T00:46:14.930" Score="0" ViewCount="49" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Consider an environment, where an agent intends to move from cell &amp;quot;A&amp;quot; to cell &amp;quot;G&amp;quot;, avoiding obstacles (cells marked with shading). The agent can move forward, rotate 90º to the left, or rotate 90º to the right, and can identify the type of cell in front of him. Here's a diagram.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s5ara.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s5ara.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Classify the characteristics of the environment, according to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Accessible or non-accessible&lt;/li&gt;&#xA;&lt;li&gt;Deterministic or non‐deterministic&lt;/li&gt;&#xA;&lt;li&gt;Episodic or non‐episodic&lt;/li&gt;&#xA;&lt;li&gt;Discrete or continuous&lt;/li&gt;&#xA;&lt;li&gt;Static or dynamic&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12182" LastEditorUserId="2444" LastEditDate="2020-12-20T12:59:06.873" LastActivityDate="2020-12-20T12:59:06.873" Title="How to classify this environment?" Tags="&lt;reinforcement-learning&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5049" PostTypeId="2" ParentId="4286" CreationDate="2018-01-18T03:03:00.810" Score="1" Body="&lt;p&gt;Generally speaking, you can say this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;there &lt;em&gt;is&lt;/em&gt; a relationship between neural network learning (I'm assuming a &quot;vanilla&quot; ANN here, no CNN's or RNN's or anything) and linear/logistic regression.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;But they're &lt;em&gt;not&lt;/em&gt; the same thing.  Just related.  You could maybe consider them &quot;cousins&quot; to use a real-life analogy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The big obvious difference is this:  standard linear regression is, well, &lt;strong&gt;linear&lt;/strong&gt;, that is, it's based on a straight line. So it can only separate points on a plane which can be separated by a straight line drawn on that plane.  An ANN however, is &lt;strong&gt;non-linear&lt;/strong&gt; and can fit all sorts of crazy looking curves.  The reason why this is true has to do with a combination of the &quot;activation&quot; functions that are used, as well as the layering effect of your hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be fair, if you extend linear regression to be polynomial regression, you can fit more complicated curves, but that has its own downsides.  And while they are also related, linear regression and polynomial regression aren't - strictly speaking - the same thing (although they may both be special cases of the same general technique).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of that may be over-simplifying a bit.  If you really want a good explanation of both linear/logistic regression and ANN's and some explanation of how they relate and differ, I recommend Andrew Ng's ML courses on Coursera.  Both the original one and the new DeepLearning.ai ones.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-01-18T03:03:00.810" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5050" PostTypeId="2" ParentId="5018" CreationDate="2018-01-18T03:09:49.283" Score="4" Body="&lt;p&gt;We don't even know how to prove that humans think (or not) yet.  Or maybe it would be better to say that we don't &lt;em&gt;really&lt;/em&gt; know what thinking is.  In either case, there doesn't seem to be much reason to think (heh) that a &quot;machine thinking&quot; needs to be the same as a &quot;human thinking&quot;.   So no, I don't think it's been formally proven that a machine cannot &lt;strong&gt;in principle&lt;/strong&gt; be said to &quot;think&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roger Penrose wrote a long and somewhat controversial book explaining why machines can't think - &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Emperor's New Mind&lt;/a&gt;.  But I'm not aware of anybody who considers his argument definitive. Actually, I don't know of anybody who takes it very seriously at all these days.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, I don't think it matters.  We humans choose (or seem to choose) to think that we think, but external behavior is generally what we care about, not subjective internal mental states.  So if a computer acts like it can think, does it really matter if it's &quot;really&quot; thinking or not?&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-01-18T03:09:49.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5051" PostTypeId="1" AcceptedAnswerId="5056" CreationDate="2018-01-18T04:07:11.523" Score="2" ViewCount="639" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nm8RW.jpgT&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nm8RW.jpgT&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above environment is &lt;a href=&quot;https://selfdrivingcars.mit.edu/deeptraffic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepTraffic&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider this situation in the above environment, the Red car (we control it with our RL agent) is on the extreme right lane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the exploration phase, we take a 'move right' action, which ofcourse will result in the car not moving right,but the other cars will be moving, state changes due to the rules of the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using CNN to solve this, the state representation is the image itself and its a Q-Learning algorithm as described in DQN paper from deepmind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above situation I mentioned, wont the agent think due to 'move right' action the state has changed, which is not really the case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and when remembering the state transition (s,a,r,s') should i remember the actual action 'move right'(invalid) or 'do nothing'(correct as per env) ?&lt;/p&gt;&#xA;" OwnerUserId="5030" LastActivityDate="2018-01-18T16:09:13.940" Title="RL agent's view of state transitions" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5053" PostTypeId="2" ParentId="3765" CreationDate="2018-01-18T12:29:06.530" Score="1" Body="&lt;p&gt;As far as I understand your question, you are trying to match the learning algorithm with their results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, I don't think the answer mapping you have provided is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we see the options and find out which one linearly seperates the data:&#xA;We have 3 candidates for the job:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Linear SVM&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C. Perceptron&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D. Logistic Regression&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the datasets only 1, 2 and 3 are linearly seperated. Since, we know linear SVM will search for decision boundary with maximum separation between 2 linearly separable data sets hence A goes with 3. Pereceptron is basically the linear version of logistic regression. Now, LR is an exponential algorithm hence the error does not appear as large as it should. Say that for a particular input vector &lt;code&gt;x&lt;/code&gt; the &lt;code&gt;y = 1&lt;/code&gt;.Say the hypothesis (&lt;code&gt;x * transpose(theta)&lt;/code&gt;) your algorithm generated is &lt;code&gt;5&lt;/code&gt;. For perceptron error will be &lt;code&gt;5 - 1 = 4&lt;/code&gt; whereas for LR it will be &lt;code&gt;1 / (1 + e^(-5)) - 1 = 0.9932&lt;/code&gt;. If hypothesis generates &lt;code&gt;6&lt;/code&gt; error will be &lt;code&gt;5&lt;/code&gt; for perceptron and about &lt;code&gt;0.9974&lt;/code&gt; for LR. So we see even for very great increase in errors the cost for LR will not increase significantly (inherent property of exponentials) whereas it increases linearly for perceptron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, 1 goes with C since it tries to minimize the error, whereas 2 will go with D since even though there is a greater cost it gets scaled due to exponential function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And now I think 5 will go with F since the curve has a smoothness associated with it, whereas due to Re-Lu's the curve in 6 will go to E (due to an associated sharpness) and finally B goes to 4 as it has a kernel of order 2 and &lt;code&gt;x^2/a^2 + y^2/b^2 +xy = c&lt;/code&gt; is the equation of a hyperbola of order 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BCSNg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BCSNg.png&quot; alt=&quot;A random hyperbola&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence final mapping:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A - 3&lt;/li&gt;&#xA;&lt;li&gt;B - 4&lt;/li&gt;&#xA;&lt;li&gt;C - 1&lt;/li&gt;&#xA;&lt;li&gt;D - 2&lt;/li&gt;&#xA;&lt;li&gt;E - 6&lt;/li&gt;&#xA;&lt;li&gt;F - 5&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I may have missed few other mathematical subtleties, and I would be grateful if someone points them out.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-19T15:42:43.393" LastActivityDate="2018-03-19T15:42:43.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5054" PostTypeId="1" CreationDate="2018-01-18T13:20:41.763" Score="3" ViewCount="115" Body="&lt;p&gt;I'm struggling to understand the underlying mechanics of CNNs so any help is appreciated. I have a network with a ReLU activation function which does perform signifigantly better than one with sigmoid. This is expected as &lt;a href=&quot;https://www.quora.com/How-does-the-ReLu-solve-the-vanishing-gradient-problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU solves the vanishing gradient problem&lt;/a&gt;. However, my understanding was the reason we implement nonlinearities is to separate data which cannot be separated linearly. But if ReLU is linear for all values we care about it shouldn't work at all? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless, of course, neurons are defined for negative values but then my question becomes &quot;why does ReLU solve the vanishing gradient problem at all?&quot;, since the derivative of ReLU for x&amp;lt;0 = 0&lt;/p&gt;&#xA;" OwnerUserId="12026" LastActivityDate="2018-08-16T20:00:46.517" Title="If neurons are only defined for values between 0 and 1, how does ReLU differ from the identity?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5055" PostTypeId="2" ParentId="5054" CreationDate="2018-01-18T15:32:19.283" Score="-1" Body="&lt;p&gt;You are right about the fact that we use nonlinearity for classes that cant be seperated by a straight line. If you think about curves,as we know from the calculus, you can approximate a nonlinear function with lines. If you have infinite amount of lines you can mimic exactly the same function. When you increase or decrease the weight of some neuron you essentially increase or decrease the length of the line it represents. And ReLU solves the vanishing gradient problem so they are superior to sigmoid in my opinion too.&lt;/p&gt;&#xA;" OwnerUserId="5104" LastActivityDate="2018-01-18T15:32:19.283" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5056" PostTypeId="2" ParentId="5051" CreationDate="2018-01-18T15:45:13.440" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In the above situation I mentioned, wont the agent think due to 'move right' action the state has changed, which is not really the case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generally the agent does not need to &quot;think&quot; what the next state will be. After taking an action, it should &lt;em&gt;observe&lt;/em&gt; the reward and the next state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The logic for the next state is controlled by the environment. In your case, the environment of the simulator should accept the action choice and then not allow the sideways movement off the road, and present the correct logical next state. It may in addition alter the reward signal if you want to penalise this choice (in comments you say this is not done - that's fine, it's a choice you make on problem set up, what behaviour to reward or penalise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more advanced agent might also try to &lt;em&gt;predict&lt;/em&gt; what the next state will be, especially if it was using a planning algorithm (e.g. Dyna-Q). Even then there is no reason or rule that it should have to predict an impossible next state - a well-trained agent should in fact predict correctly that moving right is same as &quot;do nothing&quot; when the car is in far right-hand lane. For a value-based approach, like Q-learning the agent may do this simply by scoring the two actions with the same value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you do not need to try and second-guess what the agent will predict. Provided your state information is enough for the agent to observe that it is in the far left or far right lane, it should [eventually, after training] correctly predict the value functions and/or optimal actions as it learns the environment. That includes, after a few examples, that it will predict the outcome when moving right when already on right hand side. For that to happen reliably, you do need to check your state representation is adequate, and function approximation hyper-parameters are suited to the problem (the latter part could take some experimentation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the state data that the car receives does not contain enough information for the agent to tell that it is in extreme left or right lane, then you will have a problem. This is entirely possible in general, although from the image you show, that does not seem to be an issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible related issue is if the other cars in the scene all have different velocities. This velocity information is not shown in the state, and could cause your agent problems as it cannot tell the difference between a stationary car in front of it, and one which better matches its own velocity. You could treat this missing data as a partially observable state, and adjust you agent. Or you might be able to do the same as the Deep Mind DQN team for Atari games and use a sequence of e.g. last 4 images as the state, which should then include enough velocity data to make reliable estimates of state/action values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-01-18T16:09:13.940" LastActivityDate="2018-01-18T16:09:13.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5057" PostTypeId="1" AcceptedAnswerId="5062" CreationDate="2018-01-18T18:32:34.333" Score="5" ViewCount="214" Body="&lt;p&gt;In the book &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (page 25), by Richard S. Sutton and Andrew G. Barto, there is a discussion of the k-armed bandit problem, where the expected reward from the bandits changes slightly over time (that is, the problem is non-stationary). Instead of updating the Q values by taking an average of all rewards, the book suggests using a constant step-size parameter, so as to give greater weight to more recent rewards. Thus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Q_{n+1} = Q_n + \alpha (R_n - Q_n),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a constant between 0 and 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book then states that &lt;em&gt;this a weighted average because the sum of the weights is equal to 1&lt;/em&gt;. What does this mean? Why is this true?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditorDisplayName="user1440" LastEditDate="2020-04-15T20:18:42.470" LastActivityDate="2020-04-15T20:18:42.470" Title="What is a weighted average in a non-stationary k-armed bandit problem?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;math&gt;&lt;multi-armed-bandit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5058" PostTypeId="1" AcceptedAnswerId="5418" CreationDate="2018-01-18T21:00:06.967" Score="5" ViewCount="128" Body="&lt;p&gt;I have a model that predicts sentiment of tweets. Are there any standard procedures to evaluate such a model in terms of its output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could sample the output, work out which are correctly predicted by hand, and count true and false positives and negatives but is there a better way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know about test and training sets and metrics like AUROC and AUPRC which evaluate the model based on known data, but I am interested in the step afterwards when we don't know the actual values we are predicting. I could use the same metrics, I suppose, but everything would need to be done by hand.&lt;/p&gt;&#xA;" OwnerUserId="8385" LastActivityDate="2018-02-24T16:39:04.620" Title="How do I statistically evaluate a ML model?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5059" PostTypeId="1" AcceptedAnswerId="5060" CreationDate="2018-01-18T21:58:58.450" Score="5" ViewCount="393" Body="&lt;p&gt;I am trying to do some experiments with some intelligent agents, but I'm not sure how significant they will be in the future. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some possible interesting applications or use-cases of intelligent agents in the future? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, it can be used as a virtual assistant instead of a real call agent. But what can be a more appealing application in the future?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="2444" LastEditDate="2020-05-15T23:08:36.947" LastActivityDate="2020-05-15T23:08:36.947" Title="What are some of the possible future applications of intelligent agents?" Tags="&lt;applications&gt;&lt;chat-bots&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5060" PostTypeId="2" ParentId="5059" CreationDate="2018-01-18T22:15:17.623" Score="4" Body="&lt;p&gt;At the moment, what I can think of are the following applications, but there are potentially a lot more.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Decision Maker&lt;/strong&gt;: If you have any problem making a decision, intelligent agents can be used to weight evidence and give you statistics to rule out bad decisions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Online Teacher&lt;/strong&gt;: In the far future, intelligent agents may acquire human-like skills, they maybe used to teach different students (from different backgrounds) at once, but the advantage here is that the intelligent agent can evaluate every student's level (skills, personality, intelligence, etc), and use this data to decide how to convey information to him (differently). It would be like a virtual world were children receive information in parallel, while taking different data about them into consideration: They can be used to better education.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Social Media&lt;/strong&gt;: I think intelligent agents will be integrated into social media, to analyze messages and comments, and warn users that some friends will dislike this or that or find it offensive, to give him time for making a decision about what to say. This would help limit bullying on social media.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Language Learning&lt;/strong&gt;: in the future (like in 10 - 20 years), intelligent agents will have the ability to generate human-level language, that would make it possible for them to generate infinite sentences and structures just like humans can do, and again they can evaluate user's level to generate listening and reading material. For a human, it would be just like talking to a native speaker of that language (or even better, since this &quot;native speaker&quot; would know everything about your level).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12113" LastEditorUserId="2444" LastEditDate="2020-05-15T23:04:27.827" LastActivityDate="2020-05-15T23:04:27.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5061" PostTypeId="2" ParentId="5048" CreationDate="2018-01-18T22:42:52.817" Score="-1" Body="&lt;p&gt;The environment is : &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Deterministic :&lt;/strong&gt; the future state of the environment (and the agent) is defined by the agent's action.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Partially Observable (partially accessible)&lt;/strong&gt; : since his sensors don't see what's beyond the shaded barrier, the environment is partially observable (accessible).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Sequential (non-episodic)&lt;/strong&gt;: Since the agent's current action will constrain and affect the space of possibilities (options, choices) of future actions, current decisions are not independent of future ones.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Discrete&lt;/strong&gt; : Environment is not continuous, that is, every action/step/case is finite (either turn left, right, forward or backward), you don't have other actions, which makes it a discrete task environment.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Static&lt;/strong&gt; : the environment is not changing, it's static, only the agent is moving.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12113" LastActivityDate="2018-01-18T22:42:52.817" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5062" PostTypeId="2" ParentId="5057" CreationDate="2018-01-19T00:18:16.470" Score="1" Body="&lt;p&gt;The weighted average stands for a linear combination of all values, such that the sum of all weights is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, if you denote the rewards by a vector &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, the weighted average will be taking the dot product between &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and a vector &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$0 \le W_i \le 1$&lt;/span&gt; and the sum of all &lt;span class=&quot;math-container&quot;&gt;$W_i$&lt;/span&gt; is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If each &lt;span class=&quot;math-container&quot;&gt;$W_i = 1/n$&lt;/span&gt; it will be a weighted average (a.k.a the mean). Using the exponential decay &lt;span class=&quot;math-container&quot;&gt;$W_i = \alpha^i/ (\sum W_i)$&lt;/span&gt; is also a weighted average.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then both strategies to compute the Q value use weighted average of previous reward.&lt;/p&gt;&#xA;" OwnerUserId="12121" LastEditorUserId="2444" LastEditorDisplayName="user1440" LastEditDate="2019-10-05T17:33:20.577" LastActivityDate="2019-10-05T17:33:20.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5063" PostTypeId="1" CreationDate="2018-01-19T01:19:43.947" Score="2" ViewCount="47" Body="&lt;p&gt;By open up I mean slightly open up so that a theoretical structure of panels with no width looks three-dimensional. The original structure being an ideal object where any number several panels can occupy the same region of space (plane).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To concretize what I mean by rigid structure of panels, let's take what's on my profile picture. I'm including here a larger version of that object:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m9Uxz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m9Uxz.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An origami figure is folded from a square and, unlike this simple example in the image, it can be very convoluted, with layers upon layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that if I have an ideal, theoretical and flat model of an origami picture and by flat I mean the faces are on planes but not necessarily on a single plane. For example all of faces of the figure in the image would be in one plane, but there could be figures with more planes; think of animals with ears, flippers, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to open up those parts that hinge on a theoretical segment (relatively easy) or curve or make a triangle of the corners of those faces that have two adjacent faces with no connections, open up a set of several faces that allow such opening of which the image is a good example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried programming rules for different structures suchs as flaps, ends, wrap-arounds... However there are three no small problems. First, it's extremely challenging to take into account all the corner cases and possibilities. I suspect it sounds simpler that it really is, but I don't want to digress with explanations. Second, the code is not maintenable. It's difficult to put into words rules that have to be visualized. Third it's very difficult to unit test and debug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I strongly suspect that there must be some Artificial Intelligence techniques for doing this. Would you be so kind as to point me in the right direction? Just in a general way, without needing to go much into detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if I should include more info or code.&lt;/p&gt;&#xA;" OwnerUserId="12210" LastEditorUserId="1671" LastEditDate="2018-01-19T20:44:54.667" LastActivityDate="2018-01-19T20:44:54.667" Title="How to open up a rigid structure made of connected panels?" Tags="&lt;algorithm&gt;&lt;heuristics&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5065" PostTypeId="2" ParentId="3747" CreationDate="2018-01-19T07:15:58.910" Score="3" Body="&lt;p&gt;In recent times different data science magazines and institutions have published their reviews of the top AI toolkits. In these reviews they tend to highlight the innovative features possessed by each platform as well as their reliability and ability to scale. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are a some evaluations of AI platforms that I recommend you have a look at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;KDnuggets review &lt;a href=&quot;https://www.kdnuggets.com/2016/04/top-15-frameworks-machine-learning-experts.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kdnuggets.com/2016/04/top-15-frameworks-machine-learning-experts.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paperspace review &lt;a href=&quot;https://blog.paperspace.com/which-ml-framework-should-i-use/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.paperspace.com/which-ml-framework-should-i-use/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Predictive Analytics review &lt;a href=&quot;https://www.predictiveanalyticstoday.com/deep-learning-software-libraries/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.predictiveanalyticstoday.com/deep-learning-software-libraries/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Infoworld review &lt;a href=&quot;https://www.infoworld.com/article/3026262/machine-learning/13-frameworks-for-mastering-machine-learning.html#slide4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.infoworld.com/article/3026262/machine-learning/13-frameworks-for-mastering-machine-learning.html#slide4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia comparison &lt;a href=&quot;https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if your main interest is to get an objective and unbiased assessment of the different ML toolkits. Then the best place to get such first hand reviews is at GitHub. Here you can look at the number of forks, stars and downloads, the update frequency, the number of issues, the number of contributors, the documentation, the license and even the code itself. This approach enables you to get an impartial and neutral appraisal directly from the developers themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is my personal evaluation of the different ML toolkits: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow - TensorFlow is a powerful open source library for numerical computation using data flow graphs. Some of the advantages of tensorflow are: It is portable and consequently can run on GPU's, CPU's, desktops, servers and mobile computing platforms. It comes with tensorboard which is useful in visualising and fine-tuning your network. Tensorflow has a large following with lots of researchers and developers using it. This means that most issues can be dealt with easily since they are usually the same issues that other developers run into.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Microsoft CNTK - Is a unified deep learning toolkit that describes neural networks as a series of computational steps via a directed graph. It delivers state of the art results when it comes to speech recognition, in recent times it has even outperformed humans in transcribing speech to text. The external CNTK toolkit available as open-source is also identical to the internal toolkit. This means you get to access all the features that internal Microsoft developers utilize in production. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theano - Theano is a symbolic compiler library that builds an optimized Neural Network from mathematical expressions. A major advantage of theano is that it supports Python libraries meaning that a wide range of Python libraries i.e. Numpy, SciPy and Scikit-learn can easily be integrated with theano. Additionally theano has excellent high level API's such as keras and lasagne which greatly simplify the use of the Theano toolkit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Torch - Is a scientific computing framework and script language that provides a wide range of algorithms for machine learning. The benefits of torch are: It has a very active open source community given that Facebook is standing behind it. Additionally it is based on Lua which is a very easy language to pick up. On top of that a significant number of AI researchers publish their research findings in torch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Caffe - Is a deep learning framework originally developed at UC Berkeley. Some of the perks of using Caffe are: It has an excellent Matlab and Python interface, it supports functions such as ReLu straight out of the box furthermore Caffe excels at CNN's for images.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-01-19T07:15:58.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5066" PostTypeId="2" ParentId="4647" CreationDate="2018-01-19T09:14:20.487" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How is the right to explanation reasonable, given the current standards at which we hold each other accountable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In short, it is quite reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, making AI accountable and responsible for explaining the decision seems reasonable because&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans (DARPA in this case) has &lt;strong&gt;chosen to create, raise and evolve AI&lt;/strong&gt; with the tax-payers money. In our society as well, whenever humans have come together for some purpose (some formal arrangement like government or otherwise), accountability and responsibility is assumed and assigned. Thus, expecting AI (who will take decisions on our behalf) to explain their decisions &lt;strong&gt;seems only natural extension of how humans currently operate&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans (&lt;em&gt;generally&lt;/em&gt; at least) &lt;strong&gt;don't have super-power or resources to control and manipulate&lt;/strong&gt; the rest of the population. But when they do (like our &lt;em&gt;political leaders&lt;/em&gt;), we want them to be accountable. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In rare cases, powerful humans &lt;strong&gt;become powerful due to things which are accessible to everyone&lt;/strong&gt;, while AI's super-power won't be. So, our civilization would feel safer and thus less insecure with an AI which doesn't shut the door on face when questions are asked.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other benefits of AI that offers an explanation on how a decision is reached&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Far easier to debug and improve in its early stage.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can be customized further (and along the way) to amplify certain aspects like social-welfare over profitability, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12138" LastEditorUserId="2444" LastEditDate="2019-06-15T14:39:53.753" LastActivityDate="2019-06-15T14:39:53.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5067" PostTypeId="1" CreationDate="2018-01-19T11:32:39.073" Score="4" ViewCount="1224" Body="&lt;p&gt;I am currently writing an engine to play a card game and I would like for an ANN to learn how to play the game. The game is currently playable, and I believe for this game a deep-recurrent-Q-network with a reinforcement learning approach is the way to go. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't know what type of layers I should use, I found some examples of Atari games solved through ANN, but their layers are CNN (convolutional), which are better for image processing. I don't have an image to feed the NN, only a state composed of a tensor with cards in the player's own hand and cards on the table. And the output of the NN should be a card or the action 'End Turn'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently trying to use TensorFlow but I'm open to any library that can work with NN. Any type of help or suggestion would be greatly appreciated!      &lt;/p&gt;&#xA;" OwnerUserId="12217" LastEditorUserId="12217" LastEditDate="2018-01-23T14:25:09.793" LastActivityDate="2018-10-21T16:00:29.817" Title="What layers to use in a Neural Network for card game" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;game-ai&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5069" PostTypeId="1" CreationDate="2018-01-19T15:07:30.837" Score="1" ViewCount="76" Body="&lt;p&gt;As far as I understand, the hill climbing algorithm is a local search algorithm that selects any random solution as an initial solution to start the search. Then, should we apply an operation (i.e., mutation) on the selected solution to get a new one or we replace it with the fittest solution among its neighbours? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the part of the algorithm where I am confused. &lt;/p&gt;&#xA;" OwnerUserId="6095" LastEditorUserId="2444" LastEditDate="2019-03-02T11:03:17.787" LastActivityDate="2019-03-02T11:03:17.787" Title="Should the mutation be applied with the hill climbing algorithm?" Tags="&lt;search&gt;&lt;optimization&gt;&lt;hill-climbing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5070" PostTypeId="1" CreationDate="2018-01-19T17:17:28.953" Score="0" ViewCount="91" Body="&lt;p&gt;I do not have background in Artificial Intelligence related algorithms etc. But I have a scheduling problem that needs to be resolved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, employees need to be scheduled for on call. They might be on vacation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you guide as to what algorithms I need to look into, suitable for a beginner? &lt;/p&gt;&#xA;" OwnerUserId="12203" LastEditorUserId="1671" LastEditDate="2018-01-19T21:02:20.463" LastActivityDate="2018-02-21T16:10:01.147" Title="Artificial Intelligence and Automatic Scheduling" Tags="&lt;learning-algorithms&gt;&lt;getting-started&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5071" PostTypeId="2" ParentId="5069" CreationDate="2018-01-19T17:28:01.783" Score="1" Body="&lt;p&gt;In general, hill climbing algorithms select a random initial solution, then takes the best move available after evaluating all possible operations available. The possible operations are determined by the search operators (which you determine and is dependent on the problem setting). So by your words, you should 'replace it with the fittest solution among its neighbours'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mutation can be applied as a search operator, but this varies by degree and context. For example, in structure learning for bayesian networks, a random sequence of search operators are sometimes applied to escape local optima during the search process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.cs.helsinki.fi/u/bmmalone/probabilistic-models-spring-2014/StructureLearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.helsinki.fi/u/bmmalone/probabilistic-models-spring-2014/StructureLearning.pdf&lt;/a&gt; - slide 9 for random restarts&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="9469" LastEditDate="2018-01-19T17:35:09.140" LastActivityDate="2018-01-19T17:35:09.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5072" PostTypeId="2" ParentId="4766" CreationDate="2018-01-19T18:15:08.177" Score="3" Body="&lt;p&gt;For us to answer this question. First, we need to look at why capsule networks outperform convolution neural networks by as much as 45% in recognizing images that have been rotated, translated or are under a different pose. We can find Geof Hinton's paper on capsule networks here for reference &lt;a href=&quot;https://arxiv.org/pdf/1710.09829v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1710.09829v1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a CNN architecture, a convolution layer is usually followed by a max-pooling layer. This is so that the lower levels can detect low-level features, like edges, while the high-level layers can detect abstraction like eyes. However, the application of max-pooling leads to the loss of important information regarding the location and spatial relationship between certain features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, this is where capsule networks excel, the way they represent certain features is locally invariant. This is why capsule networks can recognize images under different lighting conditions and deformations. They are likely to excel at applications such as video and object tracking but not necessarily NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current approach in NLP maps words and phrases to vectors. From there, we exploit the concept of vectors and distances between them (cosine, euclidean, etc.) to perform operations such as: finding the similarity between words and even documents, machine translation, and natural language understanding (NLU). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Capsule networks are unlikely to succeed in NLP. This is because algorithms that aim to find the hierarchical structure of natural languages or approaches that focus on grammar have met little success. Research by Stanford University aiming at finding the hierarchical structure of natural languages can be found here &lt;a href=&quot;https://nlp.stanford.edu/projects/project-induction.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nlp.stanford.edu/projects/project-induction.shtml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although conclusive research regarding other applications of capsule networks has not yet been conducted. They are likely to excel at applications such as video intelligence and object tracking but not necessarily NLP.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="2444" LastEditDate="2020-06-09T11:49:52.367" LastActivityDate="2020-06-09T11:49:52.367" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5074" PostTypeId="2" ParentId="2069" CreationDate="2018-01-20T03:24:04.567" Score="0" Body="&lt;p&gt;I'd like to answer this in detail, but it requires some fairly complicated theory that you don't have access to. Essentially this is related to the Abstraction Valuation Paradox. Don't bother trying to look that up. It's part of several years of research that hasn't been published yet. The research has shown that there is no solution to this paradox using computational or AI theory. So, no AI, no matter how advanced, can have an understanding of ethics. The best you can do is program in a bunch of rules of thumb. This gives your AI a bureaucratic reaction to conditions but no flexibility and no way to resolve problems outside of its rule space. In other words if it runs into an exception or unforeseen circumstances, it could stall or could guess at a decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The research on human-like ability to understand and reason is quite different from the study of AI. This research suggests that you would need consciousness for an understanding of ethics.&lt;/p&gt;&#xA;" OwnerUserId="12118" LastActivityDate="2018-01-20T03:24:04.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5075" PostTypeId="1" AcceptedAnswerId="5079" CreationDate="2018-01-20T10:58:40.943" Score="0" ViewCount="334" Body="&lt;p&gt;In the [delta rule][1] the equation to adjust the weight with respect to error is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w_{(n+1)}=w_{(n)}-\alpha \times \frac{\partial E}{\partial w}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;*where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the error.&lt;/p&gt;&#xA;&lt;p&gt;The graph for &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; would look like the one below with &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; axis and &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ju05S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ju05S.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In other words, we can write&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\alpha \times \frac{\partial E}{\partial w}=w_{(n)}-w_{(n+1)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to know, what is the proof behind the gradient of a curve being equal/proportional to the distance between the two coordinates in the x-axis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial w}$&lt;/span&gt; times step is a small shift on &lt;span class=&quot;math-container&quot;&gt;$f(w)$&lt;/span&gt; not &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;. So, why does the difference between &lt;span class=&quot;math-container&quot;&gt;$W(n+1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W(n)$&lt;/span&gt; be equal to &lt;span class=&quot;math-container&quot;&gt;$f(W)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://stats.stackexchange.com/q/305516/82135&quot;&gt;a similar question&lt;/a&gt;, but the accepted answer doesn't have a proof.&lt;/p&gt;&#xA;" OwnerUserId="11789" LastEditorUserId="2444" LastEditDate="2020-07-06T23:01:36.510" LastActivityDate="2020-08-10T17:16:55.060" Title="What is the proof behind the gradient of a curve being proportional to the distance between the two co-ordinates in the x-axis?" Tags="&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;learning-algorithms&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5076" PostTypeId="2" ParentId="3948" CreationDate="2018-01-20T11:08:33.897" Score="1" Body="&lt;p&gt;The software that you require to provide your AI agent remote access is called Virtual Network Computing (VNC). A VNC is a desktop sharing system that transmits keyboard and mouse movements, over a network connection, from the server (the AI) to the client (training environment). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows your AI agent to use a computer like a human does, which is by looking at the screen pixels and operating a virtual keyboard and mouse. Infact VNC's are used conventionally for remote technical support and accessing files on one's computer remotely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excellent use case example is Open AI's Universe platform. Alongside Gym, which is Open AI's toolkit for developing RL algorithms. Universe launches programs behind a VNC remote desktop, which enables the training of AI agents by only showing them the screen pixels and allowing them to operate a virtual keyboard or mouse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you want to train your AI agent on an Android based training environment, my suggestion is that you consider installing either Real VNC (android client) or Droid VNC on your Android device. You can then install the VNC server software on your training machine (i.e. an nvidia docker instance). Your AI agent will now have full access to the training environment screen for reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also recommend that you look at this similar question on the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://askubuntu.com/questions/414189/how-to-remotely-control-my-android&quot;&gt;https://askubuntu.com/questions/414189/how-to-remotely-control-my-android&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-01-20T12:51:41.507" LastActivityDate="2018-01-20T12:51:41.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5078" PostTypeId="2" ParentId="5070" CreationDate="2018-01-20T16:45:38.977" Score="1" Body="&lt;p&gt;Scheduling can get quite complicated. It is fundamentally programming to find a solution given constraints. One approach would be to use &lt;a href=&quot;http://www.minizinc.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;MiniZinc&lt;/a&gt;, and &lt;a href=&quot;http://www.csplib.org/Problems/prob022/models/bus_scheduling_csplib.mzn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a scheduling example&lt;/a&gt;. You can use other plain programming approaches such as Python and so on, but Minizinc is highly optimized and might well reward time invested in learning by finding a better solution more quickly.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2018-01-20T16:45:38.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5079" PostTypeId="2" ParentId="5075" CreationDate="2018-01-21T11:41:15.567" Score="2" Body="&lt;p&gt;Don’t think about it as the &lt;span class=&quot;math-container&quot;&gt;$w_{(n)}-w_{(n+1)}$&lt;/span&gt; being proportional to something. Think about it this way:&lt;/p&gt;&#xA;&lt;p&gt;I'm now at &lt;span class=&quot;math-container&quot;&gt;$w_{(n)}$&lt;/span&gt;. Where do I want to be at timestep, so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of &lt;span class=&quot;math-container&quot;&gt;$w_{(n)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; increases as I increase &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; (that is, if &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial w}&amp;gt;0$&lt;/span&gt;, then obviously, I would want to move a little bit to the left. In other words, &lt;span class=&quot;math-container&quot;&gt;$w_{n+1}&amp;lt;w_{n}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$w_{n+1}-w_{n}&amp;lt;0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, if the derivative were negative, you know that you should move right to reduce the error a little bit, &lt;span class=&quot;math-container&quot;&gt;$w_{n+1}-w_{n}&amp;gt;0$&lt;/span&gt;.  So, basically, your step should have the opposite sign of the derivative.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w_{n+1}-w_{n} \propto-\frac{\partial E}{\partial w}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, the learning rate, is just the constant of proportionality.&#xA;Caution: think about small values for this rate, not big numbers.  Taking a huge step can cause you to overshoot the minimum point.&lt;/p&gt;&#xA;" OwnerUserId="12250" LastEditorUserId="2444" LastEditDate="2020-07-06T23:13:14.173" LastActivityDate="2020-07-06T23:13:14.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5080" PostTypeId="1" CreationDate="2018-01-21T14:54:11.900" Score="1" ViewCount="27" Body="&lt;p&gt;Are there a finite set of computable functions constructing deep neural network which can form or implement any c.e. function or computable function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or does there exist a finite set of computable function by which every c.e. function can be implemented by combination and connection(like connection in DNN)?&lt;/p&gt;&#xA;" OwnerUserId="9237" LastActivityDate="2018-01-21T14:54:11.900" Title="Are there a finite set of computable functions constructing deep neural network which can form or implement any c.e. function or computable function?" Tags="&lt;deep-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5081" PostTypeId="1" CreationDate="2018-01-21T16:36:58.270" Score="2" ViewCount="500" Body="&lt;p&gt;&lt;a href=&quot;http://bigthink.com/elise-bohan/the-most-human-ai-youve-never-heard-of-meet-luna&quot; rel=&quot;nofollow noreferrer&quot;&gt;This AI&lt;/a&gt; is really human-like and allegedly doesn't give pre-programmed responses. It's makers Robots without Borders say the project is open source but I couldn't find the code anywhere. &lt;/p&gt;&#xA;" OwnerUserId="12254" LastActivityDate="2018-04-12T04:58:32.700" Title="Can anyone find the source code for the chatbot Luna?" Tags="&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5082" PostTypeId="2" ParentId="5075" CreationDate="2018-01-21T16:40:54.327" Score="0" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$let,\ x_{(n)} \ be\ a\ point\ on\ x-axis\ where\ f'( x) \ =\ 0\ ,\ and\ x_{(n\ +\ h)} \ is\ any\ other\ arbitary\ point \\&#xA;\therefore \ \ \frac{f'( x_{(n\ +\ h)})}{|\ f'( x_{(n\ +\ h)}) \ |} =\begin{cases}&#xA;1 &amp;amp; \mathrm{if,} \ h\ \  &amp;gt;0\\&#xA;0 &amp;amp; \mathrm{if,} \ h\ =\ 0\\&#xA;-1 &amp;amp; \mathrm{if,} \ h\ &amp;lt; \ 0&#xA;\end{cases}\\similarly,\ \ \frac{x_{(n)} \ -\ x_{(n\ +\ h)} \ }{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |} \ =\ \begin{cases}&#xA;1 &amp;amp; \mathrm{if,} \ h\ \ &amp;lt; 0\\&#xA;0 &amp;amp; \mathrm{if,} \ h\ =\ 0\\&#xA;-1 &amp;amp; \mathrm{if,} \ h\  &amp;gt;\ 0&#xA;\end{cases}\\or,\ \frac{x_{(n)} \ -\ x_{(n\ +\ h)} \ }{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |} \ =\ -\ \ \ \frac{f'( x_{(n\ +\ h)})}{|\ f'( x_{(n\ +\ h)}) \ |}\\ &#xA;\therefore \ x_{(n)} \ =x_{(n\ +\ h)} \ -\ \eta \times f'( x_{(n\ +\ h)}) \ \ \ \ \ \ \left[ where\ \eta \ =\frac{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |}{|f'( x_{(n\ +\ h)}) \ |} \ \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="11789" LastEditorUserId="11789" LastEditDate="2020-01-25T13:21:52.293" LastActivityDate="2020-01-25T13:21:52.293" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5083" PostTypeId="2" ParentId="5081" CreationDate="2018-01-21T21:57:24.837" Score="3" Body="&lt;p&gt;You can't find the source code because it doesn't exist. The whole thing was a 2015 kickstarter scam, and now it's a patreon scam. All the tech &quot;demos&quot; are obviously pre-scripted videos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was &lt;a href=&quot;https://www.quora.com/Is-Luis-Aranas-AI-Luna-robot-for-real-or-is-it-a-scam-to-raise-funds&quot; rel=&quot;nofollow noreferrer&quot;&gt;actually a question on Quora about it&lt;/a&gt;, in which one of the answerers managed to find, among other things, the question and answer that his fake AI gave in one of the videos.&lt;/p&gt;&#xA;" OwnerUserId="12258" LastActivityDate="2018-01-21T21:57:24.837" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5084" PostTypeId="1" CreationDate="2018-01-22T03:50:21.097" Score="1" ViewCount="235" Body="&lt;p&gt;I am currently working on a defect detection algorithm but I only have a few samples of defects.I googled for defect detection datasets and I found this one: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has a few hundreds of original images of defects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is:&#xA;Imagenet =&gt; Defect dataset from internet =&gt; Own defect dataset&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 1. Training a model with ImageNet initialization using the defect dataset  found in the internet (+ non-defect images + augmented data)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 2. Using the output model of step 1 (which will be more similar to my own data),do transfer learning using my own defect dataset (defects + non-defects + augmented).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think this a good way to get good results? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on:&#xA;&lt;a href=&quot;https://blog.slavv.com/a-gentle-intro-to-transfer-learning-2c0b674375a0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.slavv.com/a-gentle-intro-to-transfer-learning-2c0b674375a0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should defect images consider as low similar with imagenet's images? or similar to model because a both inputs are images? Some webpages said because they both are images, they are similar but some webpages said because these images are too different to the images used to train the imagenet model so I got confused about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I skip step 1, I dont think I get anything good because I have less than 100 images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advise or comment will be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="12261" LastEditorUserId="12261" LastEditDate="2018-01-22T06:28:04.447" LastActivityDate="2018-01-22T06:28:04.447" Title="Transfer learning from model trained in a similar dataset" Tags="&lt;training&gt;&lt;tensorflow&gt;&lt;object-recognition&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5085" PostTypeId="1" CreationDate="2018-01-22T10:47:03.950" Score="5" ViewCount="147" Body="&lt;p&gt;I am trying to build a neural network that takes in a single string, ex: &quot;dog&quot; as an input, and outputs 50 or so related hashtags such as, &quot;#pug, #dogsarelife, #realbff&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought of using a classifier, but because there is going to be millions of hashtags to choose the optimal one from, and millions of possible words from the english dictionary, it is virtually impossible to search up the probability of each &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is going to be learning information from analyzing twitter posts' text, and its hashtags, and find which hashtags goes with what specific words.&lt;/p&gt;&#xA;" OwnerUserId="12264" LastActivityDate="2019-06-23T11:02:25.307" Title="What machine learning algorithm should be used to analyze the relationship between strings?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;algorithm&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5086" PostTypeId="1" CreationDate="2018-01-22T13:54:46.703" Score="1" ViewCount="23" Body="&lt;p&gt;I have a sample set of data about Leads that gets generated every day. Leads are nothing but a user expressing request to be our partner or not. Sample data set is as shown below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LEADID,CREATEDATE,STATUS,LEADTYPE&#xA;810029,24-DEC-17 12.00.00.000000000 AM,open,LeadType1&#xA;806136,30-DEC-17 12.00.00.000000000 AM,open,LeadType2&#xA;812134,31-DEC-17 12.00.00.000000000 AM,open,LeadType2&#xA;806147,31-DEC-17 12.00.00.000000000 AM,open,LeadType1&#xA;806166,01-JAN-18 12.00.00.000000000 AM,open,LeadType2&#xA;28002,04-MAR-16 12.00.00.000000000 AM,open,LeadType2&#xA;808156,01-JAN-18 12.00.00.000000000 AM,open,LeadType1&#xA;808162,01-JAN-18 12.00.00.000000000 AM,open,LeadType2&#xA;806257,07-JAN-18 12.00.00.000000000 AM,open,LeadType1&#xA;832091,17-JAN-18 12.00.00.000000000 AM,open,LeadType2&#xA;838079,17-JAN-18 12.00.00.000000000 AM,open,LeadType1&#xA;66001,26-MAR-16 12.00.00.000000000 AM,open,LeadType1&#xA;70001,28-MAR-16 12.00.00.000000000 AM,open,LeadType2&#xA;806019,23-DEC-17 12.00.00.000000000 AM,open,LeadType2&#xA;822064,12-JAN-18 12.00.00.000000000 AM,open,LeadType1&#xA;834043,14-JAN-18 12.00.00.000000000 AM,open,LeadType2&#xA;836053,16-JAN-18 12.00.00.000000000 AM,open,LeadType1&#xA;838119,19-JAN-18 12.00.00.000000000 AM,open,LeadType2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see Lead types can be of LeadType1 or LeadType2 and this get generated every day. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make sense of data I created the following plot using Python&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MXTc8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MXTc8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The supporting code is as follows. Note I am just a Noob to Python and AI but I want to check if this proves a valid use case for Machine Learning and what should be my approach&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;import matplotlib.pyplot as plt&#xA;#%matplotlib inline&#xA;in_file = 'lead_data.csv'&#xA;mydf = pd.read_csv(in_file,encoding='latin-1')&#xA;&#xA;fig, ax = plt.subplots(figsize=(15,7))&#xA;#g = mydf.groupby(['R4GSTATE','LEADTYPE']).count()['STATUS'].unstack()&#xA;g = mydf.groupby(['R4GSTATE','STATUS']).count()['LEADTYPE'].unstack()&#xA;g.plot(ax=ax)&#xA;#ax.set_xlabel('R4GSTATE')&#xA;ax.set_xlabel('R4GSTATE')&#xA;ax.set_ylabel('Number of Leads')&#xA;ax.set_xticks(range(len(g)));&#xA;ax.set_xticklabels([&quot;%s&quot; % item for item in g.index.tolist()], rotation=90);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically I just read the csv, curated the data( I have cleaned the original csv) to keep what is meaningful for me. I also created grouping of number of leads Month-Year wise so that I can see the historical lead generated every month. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know if Machine Learning helps me to predict number of Lead generated in next coming months based on previous months data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the answer is yes then is Linear Regression the right path to explore further&lt;/p&gt;&#xA;" OwnerUserId="12266" LastActivityDate="2018-01-22T13:54:46.703" Title="Can number of Leads be predicted based on previous months" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;python&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5087" PostTypeId="1" CreationDate="2018-01-22T14:22:23.820" Score="2" ViewCount="43" Body="&lt;p&gt;Forgive what might be a basic question.  I'm just experimenting with ML / AL and I have a small problem set and I'd like to see if it can be solved with ML / AI.  Basically, given a set of objects with multiple features, I'd like to create a process for recommending one automatically to a user.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking that some sort of clustering algorithm may be the best approach. However, one main challenge I'm trying to wrap my head around is that I don't know in advance how many distinct clusters will evolve... There may be scenarios where we Feature X is really important, but other scenarios where a user will say Feature Y is important.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, what is my input set?  For each training sample, I will have 1 selected object, and N-1 unselected objects.  But I don't want to &quot;train&quot; that the unselected objects are &quot;bad&quot; because they could be selected in a future training example.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I don't have a large training set already, so I would like to use feedback (user input, &quot;This was a bad choice&quot; or &quot;Use this object instead.&quot;) from the process to further refine the algorithm.  Is this feasible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any established patterns for this sort of process?  Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="11449" LastActivityDate="2019-09-02T02:37:48.307" Title="Recommend item from set based on features" Tags="&lt;machine-learning&gt;&lt;learning-algorithms&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5088" PostTypeId="2" ParentId="5087" CreationDate="2018-01-22T15:18:57.320" Score="1" Body="&lt;p&gt;This problem is usually approached with &quot;Singular Value Decomposition&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Search also for the &quot;Netflix Challenge&quot;.&lt;/p&gt;&#xA;" OwnerUserId="12269" LastActivityDate="2018-01-22T15:18:57.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5091" PostTypeId="2" ParentId="5067" CreationDate="2018-01-22T15:29:41.577" Score="2" Body="&lt;p&gt;The game state consists of the location of all the hidden cards, so you probably need a softmax layer, 52*n, where n is the number of locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not very sure that a NN is a good match. &lt;/p&gt;&#xA;" OwnerUserId="12269" LastActivityDate="2018-01-22T15:29:41.577" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5092" PostTypeId="1" AcceptedAnswerId="5101" CreationDate="2018-01-22T15:56:40.457" Score="1" ViewCount="1479" Body="&lt;p&gt;I am currently learning about CNN's and I am confused on how filter/kernels are initialized beside their size? Say if you want a filter of 3x3 how are the inner values initialized a the start? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://setosa.io/ev/image-kernels/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://setosa.io/ev/image-kernels/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you just use those predefined image-kernels as a start? Or are they randomly initialized and retrained from backpropagation? I am pretty confused on this matter because so far of all the lecture I have taken no one really talk about this yet and I want to know it now.&lt;/p&gt;&#xA;" OwnerUserId="12242" LastEditorUserId="1671" LastEditDate="2018-01-22T22:35:46.073" LastActivityDate="2018-01-23T09:56:39.900" Title="How are kernel's input values initialized in a CNN network?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5093" PostTypeId="1" AcceptedAnswerId="5099" CreationDate="2018-01-22T17:00:35.630" Score="1" ViewCount="586" Body="&lt;p&gt;I am implementing neural network to train hand written digits. Following is the cost function,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q4Wzx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q4Wzx.png&quot; alt=&quot;cost function&quot;&gt;&lt;/a&gt;&#xA;In &lt;strong&gt;log(1-(h(x))&lt;/strong&gt;, if h(x) is 1, then it would result in log(1-1) (i.e) log(0).&#xA;So im getting math error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im initializing the weights randomly between 10-60. Im not sure where i have to change and what i have to change! Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="12273" LastActivityDate="2018-01-23T07:01:42.500" Title="Neural network cost function implementation" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5094" PostTypeId="1" CreationDate="2018-01-22T19:39:50.210" Score="1" ViewCount="19" Body="&lt;p&gt;I'm reading the book &lt;a href=&quot;https://www.springer.com/la/book/9783642072857&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Evolutionary Computing&lt;/a&gt; and on the chapter about Evolution Strategies said that we have to modify the strategy parameter sigma (standard deviation or mutation step size) before using in to modify the object parameter and I don't understand why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we have to modify sigma before the mutation of object parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe because if we do it we will have the mutated object parameters and the strategy parameters that have generated them.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastActivityDate="2018-01-22T19:39:50.210" Title="ES- Modify sigma before mutate object parameters" Tags="&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5095" PostTypeId="2" ParentId="3548" CreationDate="2018-01-22T20:17:52.153" Score="1" Body="&lt;p&gt;The OP asked about AI, not Machine Learning. Machine Learning is a sub-discipline of AI. Most AI used in games and motion planning does not involve machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The background you will need to &quot;jump into the field of AI&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Math through calculus&lt;/li&gt;&#xA;&lt;li&gt;Algorithms&lt;/li&gt;&#xA;&lt;li&gt;Basic combinatorics and probability theory&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you already have these requirements, I suggest looking at different types of AI to see what sounds most interesting to you. Mostly, getting into AI just means learning how the algorithms work. The algorithms for facial recognition (a machine learning problem) for example are much different than coordinating a team of robots (a combinatorial optimization problem).&lt;/p&gt;&#xA;" OwnerUserId="12278" LastActivityDate="2018-01-22T20:17:52.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5096" PostTypeId="1" CreationDate="2018-01-22T21:40:56.423" Score="10" ViewCount="1851" Body="&lt;p&gt;I am trying to develop a neural network which can identify design features in CAD models (i.e. slots, bosses, holes, pockets, steps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input data I intend to use for the network is a n x n matrix (where n is the number of faces in the CAD model). A '1' in the top right triangle in the matrix represents a convex relationship between two faces and a '1' in the bottom left triangle represents a concave relationship. A zero in both positions means the faces are not adjacent. The image below gives an example of such a matrix.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Soj7K.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Soj7K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say I set the maximum model size to 20 faces and apply padding for anything smaller than that in order to make the inputs to the network a constant size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to be able to recognise 5 different design features and would therefore have 5 output neurons - [slot, pocket, hole, boss, step]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would I be right in saying that this becomes a sort of 'pattern recognition' problem? For example, if I supply the network with a number of training models - along with labels which describe the design feature which exists in the model, would the network learn to recognise specific adjacency patterns represented in the matrix which relate to certain design features?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a complete beginner in machine learning and I am trying to get a handle on whether this approach will work or not - if any more info is needed to understand the problem leave a comment. Any input or help would be appreciated, thanks. &lt;/p&gt;&#xA;" OwnerUserId="12125" LastActivityDate="2018-12-09T05:43:59.447" Title="Using neural network to recognise patterns in matrices" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;pattern-recognition&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5097" PostTypeId="2" ParentId="5096" CreationDate="2018-01-23T05:01:46.390" Score="0" Body="&lt;p&gt;As far as I understand, yes your problem is related to pattern recognition. Since the approach is to classify inputs with labels you previously provide for the neural net, I think a convolutional neural networks could work for you problem. &lt;/p&gt;&#xA;" OwnerUserId="12006" LastActivityDate="2018-01-23T05:01:46.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5098" PostTypeId="1" CreationDate="2018-01-23T05:02:16.247" Score="4" ViewCount="90" Body="&lt;p&gt;I am new to machine learning and AI, so forgive me if this is obvious. I was talking with a friend on how to solve this problem, and neither of us could figure out how to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have a grid area of 100x100 blocks, and I want a robot to build a horizontal 100x100 grid, and 3 blocks high. I am given a random, but known starting surface, always 100x100 but the height of the random surface can vary from 1 to 5 blocks. I have an extra reserve of blocks i can pick up, so dont have to worry about running out. The robot can move in any direction, even diagonally at some cost penalty. The robot can obviously move a 4 high block to fill in a 2 high, so each is at the design height of 3. &#xA;This sounds like a reinforcement learning problem, but would any one be able to explain more detail how I would do this, to a) minimize the amount of moves, and b) to get to the design surface. &lt;/p&gt;&#xA;" OwnerUserId="12284" LastActivityDate="2018-06-29T16:34:31.287" Title="Move blocks to create a designed surface" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5099" PostTypeId="2" ParentId="5093" CreationDate="2018-01-23T06:55:12.027" Score="1" Body="&lt;p&gt;So, firstly, for h _{Theta}(x) to be 1, the weighted sum of x (after you dot product it with Theta) would have to be literally infinity, if you're using the sigmoid function. Doesn't happen in practice, even with the rounding computers do, as we don't use big numbers to initialize our Theta matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, that'd mean you're basically more certain than one can possibly be in this universe that the label of this example should be '1'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if (1 - h_{Theta}(x)) = 0, 'y' is &lt;em&gt;certainly&lt;/em&gt; 1, and so 1-y &lt;em&gt;will&lt;/em&gt; be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, as the comments note, the convention is to drop the entire right-hand-side term when y^(i) is 1. This will not cause problems when programming, due to the first point I made above.&lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2018-01-23T07:01:42.500" LastActivityDate="2018-01-23T07:01:42.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5100" PostTypeId="2" ParentId="5098" CreationDate="2018-01-23T07:19:46.183" Score="1" Body="&lt;p&gt;Essentially, you could do something like have the robot randomly make moves (moving around and moving blocks) for some number of steps. Repeat this a bunch of times, and record the 'score' at the end (how close you are to a perfect result grid). Tell your algorithm to act more like the best scoring runs (Optimize a loss function), and start it over. Hopefully, you'll eventually get a robot that manages the task - the whole 'optimal path' thing will come around by the computer telling itself to learn from the lowest cost examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember, you're letting the machine do the hard thinking about the best way to do this or that. All you have to do is give it the framework to learn. &lt;/p&gt;&#xA;" OwnerUserId="1538" LastEditorUserId="1538" LastEditDate="2018-06-29T16:34:31.287" LastActivityDate="2018-06-29T16:34:31.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5101" PostTypeId="2" ParentId="5092" CreationDate="2018-01-23T08:45:04.920" Score="1" Body="&lt;p&gt;The kernels are usually initialized at a seemingly arbitrary value and then you would use a gradient descent optimizer to optimize the values so that the kernels solve your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many different initialization strategies. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Set all values to 1 or 0 or another constant&lt;/li&gt;&#xA;&lt;li&gt;Sample from a distribution, such as a normal or uniform distribution &lt;/li&gt;&#xA;&lt;li&gt;There are also some heuristic methods that seem to work very well in practice, a popular one is the so-called glorot initializer named after Xavier Glorot who introduced them &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Glorot initializers also sample from a distribution but truncate the values based on the kernel complexity.&lt;/li&gt;&#xA;&lt;li&gt;For specific types of kernels, there are other defaults that seem to perform well. See for example &lt;a href=&quot;http://proceedings.mlr.press/v37/jozefowicz15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Exploring initialization strategies is something I do when my model is not able to converge (gradient problems) or when the training seems to be stuck for a long time before the loss function starts to decrease. These are signs that there might be a better initialization strategy to look for.&lt;/p&gt;&#xA;" OwnerUserId="12285" LastEditorUserId="12285" LastEditDate="2018-01-23T09:56:39.900" LastActivityDate="2018-01-23T09:56:39.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5103" PostTypeId="1" CreationDate="2018-01-23T14:12:52.357" Score="1" ViewCount="240" Body="&lt;p&gt;Trying to figure out what Google is offering in the article at this link&#xA;&lt;a href=&quot;https://www.nytimes.com/2018/01/17/technology/google-sells-ai.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nytimes.com/2018/01/17/technology/google-sells-ai.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also need to know if their offerings are limited to visual scans and if so, the range of photographical types it can work with (heat maps for example, or continuous video feed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are a small road construction company looking for ways to start using AI on some of our projects so I am in the process of gathering information on exactly what it can do.&lt;/p&gt;&#xA;" OwnerUserId="12292" LastEditorUserId="1671" LastEditDate="2018-01-26T21:34:41.263" LastActivityDate="2018-09-25T08:01:15.967" Title="Interested in what Google AI can do. Know nothing about AI at all" Tags="&lt;machine-learning&gt;&lt;google&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5104" PostTypeId="1" CreationDate="2018-01-23T14:24:36.013" Score="2" ViewCount="217" Body="&lt;p&gt;I am working with a project which is a agent based pedestrian simulation in Java and its is animated with the help of JavaFX. I've tried to read all the social force model papers but my understanding of those articles are none. &#xA;So I tried an own approach which got trashed after failing time after time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach was that each agent calculated its surrounding and first calculated the distance to each of the agents on the field and if that distance was below a constant then the agent calculates the angle of that agent which is too close to it and moves accordingly to the calculated angle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach didnt work for me because the &quot;avoidance code&quot; is not efficient enough and the agents just dont know where to go when they meet and just stays in place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking for guidance to how I can approach this problem in a better way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double[] check(Vector&amp;lt;Pedestrian&amp;gt; peds, Pedestrian p1){&#xA;for (Pedestrian p : peds){&#xA;    if (p.getPedestrianId() != this.id){&#xA;        double distance = IPedestrian.distance_formula(getTranslateX(), getTranslateY(), p.getTranslateX(), p.getTranslateY());&#xA;        if (distance &amp;lt;= DANGER){&#xA;            System.out.println(&quot;DANGER&quot;);&#xA;            return IPedestrian.angle(getTranslateX(), getTranslateY(), p.getTranslateX(), p.getTranslateY(), p1);&#xA;        }&#xA;    }&#xA;}&#xA;return new double[] {SPEED, 0};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public void move(Vector&amp;lt;Pedestrian&amp;gt; peds, Pedestrian p) {&#xA;double[] new_steps = this.check(peds, p);&#xA;if (side == SideChooser.Left){&#xA;    setTranslateX(getTranslateX() + new_steps[0]);&#xA;    setTranslateY(getTranslateY() + new_steps[1]);&#xA;} else {&#xA;    setTranslateX(getTranslateX() - new_steps[0]);&#xA;    setTranslateY(getTranslateY() - new_steps[1]);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Math formulas: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;static double distance_formula(double thisX, double thisY, double otherX, double otherY){&#xA;return Math.sqrt(Math.pow(otherX - thisX, 2) + Math.pow(otherY - thisY, 2));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;static double[] angle(double x1, double y1, double x2, double y2, Pedestrian p){&#xA;double angle = Math.toDegrees(Math.atan2(y2-y1, x2-x1));&#xA;angle += Math.ceil(-angle/360) * 360;&#xA;&#xA;//double angle = Math.toDegrees(Math.atan2(y2-y1, x2-x1));&#xA;&#xA;if (p.getSideChoosen() == SideChooser.Left){//if the pedestrian is from the left side&#xA;    if (angle &amp;lt; 45 || angle &amp;gt; 315)//front&#xA;        return new double[]{-SPEED/5, 0};&#xA;&#xA;    else if (angle &amp;gt;= 135 || angle &amp;lt;= 225 ) //back&#xA;        return new double[]{SPEED*1.4, 0};&#xA;&#xA;    else if (angle &amp;gt;= 45 || angle &amp;lt;= 90)//North-East&#xA;        return new double[]{0, SPEED};&#xA;&#xA;    else if (angle &amp;gt; 90 || angle &amp;lt;= 135) //North-West&#xA;        return new double[]{SPEED*1.2 , SPEED};&#xA;&#xA;    else if (angle &amp;gt;= 270 || angle &amp;lt;= 315) //South-East&#xA;        return new double[]{0, -SPEED};&#xA;&#xA;    else if (angle &amp;gt; 225 || angle &amp;lt;= 270) //South-West&#xA;        return new double[]{SPEED*1.2, -SPEED};&#xA;&#xA;    else&#xA;        return new double[]{SPEED, 0};&#xA;} else {&#xA;    if (angle &amp;lt; 45 || angle &amp;gt; 315)//back&#xA;        return new double[]{SPEED*1.4, 0};&#xA;&#xA;    else if (angle &amp;gt;= 135 || angle &amp;lt;= 225 ) //front&#xA;        return new double[]{-SPEED/5, 0};&#xA;&#xA;    else if (angle &amp;gt;= 45 || angle &amp;lt;= 90)//North-West&#xA;        return new double[]{SPEED*1.2, -SPEED};&#xA;&#xA;    else if (angle &amp;gt; 90 || angle &amp;lt;= 135) //North-East&#xA;        return new double[]{0 , -SPEED};&#xA;&#xA;    else if (angle &amp;gt;= 270 || angle &amp;lt;= 315) //South-West&#xA;        return new double[]{SPEED*1.2, SPEED};&#xA;&#xA;    else if (angle &amp;gt; 225 || angle &amp;lt;= 270) //South-East&#xA;        return new double[]{0, SPEED};&#xA;&#xA;    else&#xA;        return new double[]{SPEED, 0};&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;" OwnerUserId="12291" LastActivityDate="2018-01-23T16:08:52.917" Title="Agent collision avoidance java" Tags="&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5105" PostTypeId="2" ParentId="5104" CreationDate="2018-01-23T16:08:52.917" Score="2" Body="&lt;p&gt;A very efficient approach to what you are trying to do is &lt;em&gt;velocity obstacles&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming two agents use constant velocity motion vectors, a velocity obstacle models a geometric region in which if the endpoint of the velocity vector for agent 1 falls, it will collide with agent 2 (and vice versa). Hence you can predict what velocity vectors will lead to collisions between the agents and choose velocity vectors that do not collide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are very good examples and tutorials here:&#xA;&lt;a href=&quot;http://gamma.cs.unc.edu/RVO/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://gamma.cs.unc.edu/RVO/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12278" LastActivityDate="2018-01-23T16:08:52.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5107" PostTypeId="1" AcceptedAnswerId="5740" CreationDate="2018-01-23T18:45:30.723" Score="6" ViewCount="4261" Body="&lt;p&gt;Below is a quote from CS231n&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Prefer a stack of small filter CONV to one large receptive field CONV&#xA;  layer. Suppose that you stack three 3x3 CONV layers on top of each&#xA;  other (with non-linearities in between, of course). In this&#xA;  arrangement, each neuron on the first CONV layer has a 3x3 view of the&#xA;  input volume. A neuron on the second CONV layer has a 3x3 view of the&#xA;  first CONV layer, and hence by extension a 5x5 view of the input&#xA;  volume. Similarly, a neuron on the third CONV layer has a 3x3 view of&#xA;  the 2nd CONV layer, and hence a 7x7 view of the input volume. Suppose&#xA;  that instead of these three layers of 3x3 CONV, we only wanted to use&#xA;  a single CONV layer with 7x7 receptive fields. These neurons would&#xA;  have a receptive field size of the input volume that is identical in&#xA;  spatial extent (7x7), but with several disadvantages.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My visualized interpretation &#xA;&lt;a href=&quot;https://i.stack.imgur.com/KZy6R.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KZy6R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can you see through the first CNN layer from the second CNN layer and see a 5x5 sized receptive field? There were no previous comments stating all the other hyperparameters, like input size, steps, padding, which made this very confusing to visualize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I found the answer&#xA;&lt;a href=&quot;https://medium.com/@nikasa1889/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807&quot; rel=&quot;noreferrer&quot;&gt;https://medium.com/@nikasa1889/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BUT I still don't understand it. In fact, I am more confused than ever.&lt;/p&gt;&#xA;" OwnerUserId="12242" LastEditorUserId="12242" LastEditDate="2018-01-24T16:08:03.693" LastActivityDate="2019-01-30T17:28:22.307" Title="How can 3 same size CNN layers in different ordering output different receptive field from the input layer?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5108" PostTypeId="1" AcceptedAnswerId="5109" CreationDate="2018-01-23T21:06:43.630" Score="-1" ViewCount="40" Body="&lt;p&gt;I trained data for recognizing the fingerprint of my friends. I am feeding testing data to local machine by hardware. What should I do to run this as a server and get the data over network in least time and in most efficient way? &lt;/p&gt;&#xA;" OwnerUserId="12302" LastActivityDate="2018-01-27T06:00:52.580" Title="Best way to build API for input data and get response classified by tensorflow?" Tags="&lt;tensorflow&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2021-05-19T13:39:59.950" ContentLicense="CC BY-SA 3.0" />
  <row Id="5109" PostTypeId="2" ParentId="5108" CreationDate="2018-01-23T21:51:06.613" Score="1" Body="&lt;p&gt;The simplest way is to have your model sitting in some cloud instance(eg. google compute engine) and then set up a REST API that gets data from a frontend and returns a response(the prediction).&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2018-01-27T06:00:52.580" LastActivityDate="2018-01-27T06:00:52.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5110" PostTypeId="1" CreationDate="2018-01-23T22:19:57.100" Score="3" ViewCount="481" Body="&lt;p&gt;I want to develop (in Java) a voice plugin for Eclipse on a Mac that helps me jot down high-level classes and stub methods. For example, I would like to command it to create a class that inherits from &lt;code&gt;X&lt;/code&gt; and add a method that returns &lt;code&gt;String&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could somebody help me point out the right material to learn to achieve that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't mind using an existing solution if it exists. As far as I understand, I would have to use some Siri interface and use nltk to convert the natural text into commands. Maybe there's some chatbot library that saves me some boilerpate NLP code to directly jump on to writing grammar or selecting sentence patterns.&lt;/p&gt;&#xA;" OwnerUserId="12303" LastEditorUserId="2444" LastEditDate="2020-05-15T23:24:43.150" LastActivityDate="2020-05-15T23:24:43.150" Title="Is there an AI system that automatically generates classes and methods by giving it voice commands?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;chat-bots&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5111" PostTypeId="1" CreationDate="2018-01-24T01:46:31.200" Score="9" ViewCount="2840" Body="&lt;p&gt;I'm going through Andrew NG's course, which talks about YOLO, but he doesn't go into the implementation details of anchor boxes.&lt;/p&gt;&#xA;&lt;p&gt;After having looked through the code, each anchor box is represented by two values, but what exactly are these values representing?&lt;/p&gt;&#xA;&lt;p&gt;As for the need for anchor boxes, I'm also a little confused about that --&#xA;As far as I understand, the ground truth labels have around 6 variables :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_o$&lt;/span&gt; checks if it's an object or background,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$B_x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_y$&lt;/span&gt; are the center coordinates&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$B_h$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_w$&lt;/span&gt; are the height and width of the box&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the object class, which depends on how many classes you have, so you can have multiple &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As for creating the bounding box,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$B_h$&lt;/span&gt; is divided by 2, with one half from the center points (&lt;span class=&quot;math-container&quot;&gt;$B_x, B_y$&lt;/span&gt;) to the top, and the other half to the bottom.&lt;/p&gt;&#xA;&lt;p&gt;If we train our classifier, wouldn't the prediction boxes be close to the ground truth labels as training progresses? So, if our ground truth label has a high height, small width as boxes for some images, and low hight and large width for other images, wouldn't our classifier automatically&#xA;learn to differentiate between when to use one over the other, as it is being trained? If so then what is the use of anchor boxes? And what are those numbers representing anchor boxes representing?&lt;/p&gt;&#xA;" OwnerUserId="3460" LastEditorUserId="2444" LastEditDate="2021-01-28T23:20:12.733" LastActivityDate="2021-01-28T23:20:12.733" Title="In YOLO, what exactly do the values associated with each anchor box represent?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5113" PostTypeId="1" CreationDate="2018-01-24T04:30:21.830" Score="3" ViewCount="264" Body="&lt;p&gt;So I am currently trying to create a program that when provided a list of descriptive words or a passage of text, would create a piece of abstract art based on the feelings evoked by those words. I figured that AI would probably be the best way of achieving the results I am looking for. Even though I do have quite a bit of experience with programming, I have just about zero knowledge on the subjects of machine learning, artificial intelligence, neural nets, etc. Another issue is that I simply don't have much time to write this program and I simply don't have the luxury of thoroughly learning about this subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across a really cool python thingy called StackGAN that is specifically made for text to image generation. My plan was to take a bunch of pictures of abstract art and for each one write down a list of descriptive words or emotions that they evoke. I figure that there must be some way for me to then feed the pictures plus associated words/description into the neural net as training data but have no idea how to do that and the fact that there is very little documentation about the program doesn't help. Even after spending a few days trying to make sense of the code I am completely lost as to use StackGAN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So.... my question is: how exactly do I set up the training data and train StackGAN to do what I am trying to do? The github's README.md mentions some sort of CNN-RNN text embedded in the images used for training, and I have no idea what this means, if its necessary, or how I would add such data because I can't find any sort of thorough instructions. Also if you have any suggestions on alternative more user friendly libraries that would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;StackGAN link: &lt;a href=&quot;https://github.com/hanzhanggit/StackGAN-v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hanzhanggit/StackGAN-v2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12309" LastActivityDate="2018-01-24T04:30:21.830" Title="How do I make and use a dataset for StackGAN" Tags="&lt;machine-learning&gt;&lt;python&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5114" PostTypeId="1" CreationDate="2018-01-24T19:31:41.960" Score="5" ViewCount="10592" Body="&lt;p&gt;Before I start, I want to let you know that I am completely new to the field of deep learning! Since I need a new graphics card either way (gaming you know) I am thinking about buying the GTX 1060 with 6GB or the 1070 ti with 8GB. Because I am not rich, basically I am a pretty poor student ;), I don't want to waste my money. I don't need deep learning for my studies I just want to dive into this topic because of personal interest. What I want to say is that I can wait a little bit longer and don't need the results as quickly as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I do deep learning with the 1060 (6GB seem to be very limiting, according to some websites) or the 1070 ti? Is the 1070 ti overkill for a personal hobby deep learner?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or should I wait for the new generation Nvidia graphics card?&lt;/p&gt;&#xA;" OwnerUserId="12324" LastEditorUserId="2444" LastEditDate="2020-06-01T14:09:37.777" LastActivityDate="2020-06-01T14:09:37.777" Title="Can I do deep learning with the 1060 or the 1070 ti?" Tags="&lt;deep-learning&gt;&lt;hardware-evaluation&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="2" ClosedDate="2020-06-01T14:09:46.770" ContentLicense="CC BY-SA 4.0" />
  <row Id="5115" PostTypeId="1" AcceptedAnswerId="5178" CreationDate="2018-01-24T21:38:41.783" Score="4" ViewCount="154" Body="&lt;p&gt;Imagine two languages that have only these words:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Man = 1,&#xA;deer = 2, &#xA;eat = 3,&#xA;grass = 4 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you would form all sentences possible from these words:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Man eats deer.&#xA;Deer eats grass.&#xA;Man eats.&#xA;Deer eats.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;German:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mensch = 5,&#xA;Gras = 6, &#xA;isst = 7, &#xA;Hirsch = 8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Possible german sentences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mensch isst Hirsch.&#xA;Hirsch isst Gras.&#xA;Mensch isst.&#xA;Hirsch isst.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How would you write a program that would figure out which words have the same meaning in English and German?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All words get their meaning from the information in which sentences they can be used. The connection with other words defines their meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We need to write a program that would recognize that a word is connected to other words in the same way in both languages. Then it would know those two words must have the same meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take the word &quot;deer&quot; (2) it has this structure in English&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1-3-2&#xA;2-3-4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In german (8):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;5-6-8&#xA;8-6-7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We get the same structure (pattern) in both languages: both 8 and 2 lie in first and last position, and the middle word is the same in both languages, the other word is different in both languages. So we can conclude that 8=2 because both elements are connected with other elements the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe we just need to write a very good program for recognizing analogies and we will be on the right track to creating AI?&lt;/p&gt;&#xA;" OwnerUserId="12251" LastEditorUserId="2444" LastEditDate="2020-01-29T11:28:27.067" LastActivityDate="2020-01-29T11:28:27.067" Title="How to figure out which words have the same meaning in two different languages?" Tags="&lt;natural-language-processing&gt;&lt;statistical-ai&gt;" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5116" PostTypeId="1" AcceptedAnswerId="5124" CreationDate="2018-01-24T23:21:22.067" Score="3" ViewCount="531" Body="&lt;p&gt;As many papers point out, for better learning curve of a NN, it is better for a data-set to be normalized in a way such that values match a Gaussian curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this process of feature normalization apply only if we use sigmoid function as squashing function? If not what deviation is best for the tanh squashing function?&lt;/p&gt;&#xA;" OwnerUserId="12327" LastEditorUserId="9947" LastEditDate="2018-01-25T19:40:24.993" LastActivityDate="2018-01-25T19:40:24.993" Title="Data-set values feature scaling: sigmoid vs tanh" Tags="&lt;neural-networks&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5117" PostTypeId="2" ParentId="5114" CreationDate="2018-01-25T02:18:31.880" Score="7" Body="&lt;p&gt;Regarding specific choices I can't recommend, but if you are completely new, you should probably learn/code some more until you get a GPU. There is a lot to learn in machine learning before GPU speedups make a significant difference, and until then doing the computations on any old CPU would be just fine, especially if you are just starting since you won't be doing anything too complex. You will know when computational resources are your main bottleneck, and until then it shouldn't really matter too much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, you could also rent computing power from say, &lt;a href=&quot;https://aws.amazon.com/machine-learning/amis/&quot; rel=&quot;noreferrer&quot;&gt;AWS&lt;/a&gt; or &lt;a href=&quot;https://cloud.google.com/products/&quot; rel=&quot;noreferrer&quot;&gt;Google&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-01-25T02:18:31.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5118" PostTypeId="2" ParentId="5116" CreationDate="2018-01-25T03:19:50.767" Score="0" Body="&lt;p&gt;Yes it applies, and no it shouldn't matter that much between the two activation functions.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-01-25T03:19:50.767" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5119" PostTypeId="2" ParentId="5114" CreationDate="2018-01-25T08:54:59.720" Score="3" Body="&lt;p&gt;I don't think you need to invest in any kind of GPU unless you're familiar with the computations required for the task you want to achieve using deep learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, by the time you've sufficiently mastered Deep Learning to a point where you can actually make the most of your GPU, there will be new products in the market.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So until then I suggest you use your CPU for doing little tasks such as Regression etc.  You can always use the free credit offered by the various cloud companies for your tasks &lt;/p&gt;&#xA;" OwnerUserId="10118" LastEditorUserId="1671" LastEditDate="2018-01-26T21:27:56.327" LastActivityDate="2018-01-26T21:27:56.327" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5120" PostTypeId="2" ParentId="5115" CreationDate="2018-01-25T11:07:27.393" Score="1" Body="&lt;p&gt;Isn't this what already &lt;code&gt;Word2Vec&lt;/code&gt; and other &lt;code&gt;word-embedding&lt;/code&gt; techniques already use. &lt;code&gt;You know your word by the company it keeps&lt;/code&gt; is an idea that has been around for some time now.&lt;/p&gt;&#xA;" OwnerUserId="12337" LastActivityDate="2018-01-25T11:07:27.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5121" PostTypeId="1" AcceptedAnswerId="5122" CreationDate="2018-01-25T12:20:43.140" Score="1" ViewCount="1216" Body="&lt;p&gt;I have the following problem, which I am unable to solve.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A neural network with the following structure is given: 1 input neuron, 4 elements in the hidden layer, 1 output neuron.&lt;/p&gt;&#xA;&lt;p&gt;The output neuron is bipolar, the neurons in the hidden layer are linear.&lt;/p&gt;&#xA;&lt;p&gt;The weights between the input neuron and the neurons in the hidden layer have the following values: &lt;span class=&quot;math-container&quot;&gt;$w_{11} = -3, w_{12} = 2, w_{13} = -1, w_{14} = 0.5$&lt;/span&gt;, while between neurons in the hidden layer and the starting neuron:: &lt;span class=&quot;math-container&quot;&gt;$w_{21} = +2, w_{22} = -0.5, w_{23} = -3, w_{24} = +1$&lt;/span&gt; (no threshold input in both layers).&lt;/p&gt;&#xA;&lt;p&gt;What will the network response be like if the number 3 is given to the input neuron?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12341" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-07T06:42:10.563" Title="How can I manually calculate the output a specific neural network given some input?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5122" PostTypeId="2" ParentId="5121" CreationDate="2018-01-25T12:50:41.750" Score="2" Body="&lt;p&gt;Given that the neurons are linear in the hidden layer of the neural network, so the output is just the dot product of the weights and the input. To put things in perspective, generally, we use an activation function (sigmoid, signum, etc.), which is applied to the dot product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, for an input of &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; inputs to node 1 of hidden layer is &lt;span class=&quot;math-container&quot;&gt;$-3 * 3 = -9$&lt;/span&gt;, to node 2 is &lt;span class=&quot;math-container&quot;&gt;$2 * 3= 6$&lt;/span&gt; to node 3 is &lt;span class=&quot;math-container&quot;&gt;$-1 * 3 = -3$&lt;/span&gt; to node 4 is &lt;span class=&quot;math-container&quot;&gt;$0.5 *3 = 1.5$&lt;/span&gt; (basically, I have performed dot product). Since neurons are linear, no activation function is applied and is directly propagated to the next layer, the output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contribution of all the hidden nodes to the output layer is the dot product of the weights with the output of the hidden layer that is &lt;span class=&quot;math-container&quot;&gt;$9*2 + -0.5*6 + -3*-3 + 1*1.5 = -10.5$&lt;/span&gt; (check for any calculation errors). And, finally, the output layer is bipolar hence an activation function (probably sigmoid) is applied to it. Since for sigmoid function is smaller than &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$z&amp;lt;0$&lt;/span&gt;, hence the output will be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, as &lt;span class=&quot;math-container&quot;&gt;$z = -4.5$&lt;/span&gt;, as per convention.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-03-07T06:42:10.563" LastActivityDate="2020-03-07T06:42:10.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5123" PostTypeId="1" AcceptedAnswerId="17247" CreationDate="2018-01-25T13:54:23.187" Score="2" ViewCount="62" Body="&lt;p&gt;I am writing my thesis in the field of (deep) metric learning (DML). I am training a network in the fashion of contrastive / triplet Siamese networks to learn similarity and dissimilarity of inputs. In this context, the ground truth is commonly expressed as a binary. Let's take an example based on the similarity of &lt;a href=&quot;https://en.wikipedia.org/wiki/Gray_wolf#Taxonomy_and_evolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;species&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Image A: german shepard (dog)&lt;/li&gt;&#xA;&lt;li&gt;Image B: siberian husky (dog)&lt;/li&gt;&#xA;&lt;li&gt;Image C: turkish angora (cat)&lt;/li&gt;&#xA;&lt;li&gt;Image D: gray wolf (wolf)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Image A and B are similar: same species, same sub-species (canis lupus) -&gt; &lt;strong&gt;1.0&lt;/strong&gt; == &lt;code&gt;TRUE&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Image A and C are dissimilar: different species (canis lupus vs. felis silvestris) -&gt; &lt;strong&gt;0.0&lt;/strong&gt; == &lt;code&gt;FALSE&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Image A and D ? same species, but different sub-species -&gt; &lt;strong&gt;0.8&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Which metric learning approaches use a continuous ground truth for learning?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could imagine that there is a lot of research out there using a continuous ground truth in classification settings. For instance to learn that the expression of a face is &quot;almost (60%) happy&quot;, or more controversial, an image of a person depicts a &quot;70% attractive person&quot;. Also in this fields I would be happy for hints / links.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Remarks:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I don't ask for opinions on whether this makes sense or not.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12345" LastEditorUserId="12345" LastEditDate="2019-12-27T10:27:49.080" LastActivityDate="2019-12-27T10:27:49.080" Title="Continuous ground truth in supervised (metric) learning?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5124" PostTypeId="2" ParentId="5116" CreationDate="2018-01-25T15:08:30.673" Score="0" Body="&lt;p&gt;It has little to do with activation functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have a 2 input Neural Net with 1 node in the hidden layer and 1 node in the output layer all with the sigmoid activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say suppose we are solving a bipolar classification problem. Now say one of the inputs is in the order of &lt;code&gt;10^4&lt;/code&gt; and the other in order of &lt;code&gt;10&lt;/code&gt; only. Now the Neural Net will propagate this values to the output layer via the hidden layer.&#xA;You get an error &lt;code&gt;delta&lt;/code&gt; which is propagated back to the input layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as per gradient descent rule (if you look at the formula) the weight reduction will directly be proportional to &lt;code&gt;delta * x_i&lt;/code&gt; where &lt;code&gt;x_i&lt;/code&gt; is the &lt;code&gt;i&lt;/code&gt;th input. SInce &lt;code&gt;delta&lt;/code&gt; the net error due to both inputs is same for both, so The NN first decreases/increases the weight of a connection to make it to scale, and then after that the real learning which we are interested in occurs. Also if you see institutionally at the beginning if we give random weights the input which is larger will have more contribution to the output. It basically dictates the output. Now as the NN learns it reduces the weight-age of this large input to counter balance its high value. But if you do this at the beginning  voila! the NN will train much faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its like you have 2 kids, one naughtier than other. You leave them alone in a home and one of them breaks something (&lt;code&gt;delta&lt;/code&gt;). But since you have no idea who did it you will contribute the &lt;code&gt;delta&lt;/code&gt; to them equally. But as you learn about the kids nature your view (weight-age) about who broke things when you are not at home changes. Normalization is basically someone warning you already who the naughtier kid is and so you are able to take a balanced viewpoint from the beginning. (Very Bad example, but I could not think any better).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This example takes only 2 input so it might seem not much of a gain but in real life it will be if there are a large number of inputs and a lot of magnitude difference in them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I may have missed something or other mathematical subtleties and I will be grateful if someone points them out.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-01-25T18:42:43.093" LastActivityDate="2018-01-25T18:42:43.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5125" PostTypeId="1" CreationDate="2018-01-25T16:52:10.797" Score="1" ViewCount="103" Body="&lt;p&gt;Will it be possible to model the problem of odd-even distinction of an integer (not binary string representation) using neural networks? &lt;/p&gt;&#xA;" OwnerUserId="11835" LastActivityDate="2018-01-25T16:52:10.797" Title="Modelling odd-even distinction of an integer with neural networks" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5126" PostTypeId="1" CreationDate="2018-01-25T18:31:10.407" Score="4" ViewCount="121" Body="&lt;p&gt;I am training an ANN for classification between 3 classes. The ANN has an input layer, one hidden layer and a 3 node output layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I am facing is that the output being produced by the 3 output nodes are so close to 1 (for the first few iterations at least, and so I am assuming the problem propagates to future outputs as well) the weights are not being updated (or hardly updated) due to overflow (about &lt;span class=&quot;math-container&quot;&gt;$10^{-11}$&lt;/span&gt;). I can fix the overflow problem (but I don't think it is the culprit). I think such low values of error is the main culprit, and I cannot figure what is causing such low values of error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What will cause the network to behave more responsively, that is, how will I be actually able to grasp the weight updates and not something in the order of &lt;span class=&quot;math-container&quot;&gt;$10^{-11}$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data set contain values in the order of &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;'s, and the weights randomly initialized are in the order of &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; w &amp;lt; 1$&lt;/span&gt;. I have tried feature normalization but it is not that effective.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-04-12T21:59:36.427" LastActivityDate="2019-04-12T21:59:36.427" Title="What are some concrete steps to deal with the vanishing gradient problem?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;datasets&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5127" PostTypeId="5" CreationDate="2018-01-25T18:45:39.130" Score="0" Body="&lt;p&gt;The idea here is that the architecture of hardware is a &quot;known quantity&quot; and performance can be statistically evaluated.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Distinct from &quot;recommendations&quot;, although there may be some overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With so many options, it is useful to get analyses of hardware components backed up by facts and data.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-19T20:24:47.100" LastActivityDate="2018-03-19T20:24:47.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5128" PostTypeId="4" CreationDate="2018-01-25T18:45:39.130" Score="0" Body="Use for objective evaluations and assessments of computing devices and components, including CPUs, GPUs, etc., based on architecture and performance statistics.   &#xD;&#xA;&#xD;&#xA;Opinions may be suitable for comments, but not for formal answers." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-19T20:24:47.100" LastActivityDate="2018-03-19T20:24:47.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5129" PostTypeId="1" CreationDate="2018-01-25T22:33:45.687" Score="1" ViewCount="335" Body="&lt;p&gt;I'm having trouble with accuracy evaluation at the end of Session.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training process looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with tf.Session() as sess:&#xA;    sess.run(tf.global_variables_initializer())&#xA;    epochs = 10&#xA;    batch_size = 128&#xA;&#xA;    for e in range(epochs):&#xA;        shuffle_indices = np.random.permutation(np.arange(len(inputx)))&#xA;        X_train = inputx[shuffle_indices]&#xA;        y_train = inputy[shuffle_indices]&#xA;&#xA;        epoch_loss = 0&#xA;        for i in range(int(len(inputx) // batch_size)):&#xA;            start = i * batch_size&#xA;            batch_x = X_train[start:start + batch_size]&#xA;            batch_y = y_train[start:start + batch_size]&#xA;&#xA;            _, c = sess.run([optimizer, cost], feed_dict={x: batch_x, y: batch_y})&#xA;            epoch_loss += c&#xA;&#xA;        print(e + 1, '/', epochs, 'loss:', epoch_loss)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At the end of this session I want to evaluate reached accuracy by calculating mean value of percentual differences between &lt;code&gt;prediction&lt;/code&gt; and &lt;code&gt;testy&lt;/code&gt; for each of 3 output layer neurons.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;percentual_differences_for_neuron1 = # insert your advice here&#xA;accuracy_for_neuron1 = tf.reduce_mean(percentual_differences_for_neuron1)&#xA;print('Accuracy of neuron 1 is', '''another advice''')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What's the best way of doing this in &lt;em&gt;TensorFlow&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12327" LastEditorUserId="12327" LastEditDate="2018-01-25T23:04:02.267" LastActivityDate="2018-06-12T10:24:45.057" Title="TensorFlow training evaluation" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-05-19T13:40:38.730" ContentLicense="CC BY-SA 3.0" />
  <row Id="5133" PostTypeId="2" ParentId="2201" CreationDate="2018-01-26T05:53:43.800" Score="1" Body="&lt;p&gt;Neural Networks are very good approaches for robots. The main function of Neural Net is to model the interdependence between all the &lt;code&gt;features&lt;/code&gt;. Now this can be done manually by selecting possible combinations of &lt;code&gt;features&lt;/code&gt; between  themselves upto a certain degree. But this approach has drawbacks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is tedious to go about selecting features.&lt;/li&gt;&#xA;&lt;li&gt;It costs time and additional computer resources to calculate the values of the new features you have introduced.&lt;/li&gt;&#xA;&lt;li&gt;Since you cannot visualize data more than 3-D you cannot be absolutely sure that your selected &lt;code&gt;features&lt;/code&gt; are enough to model your problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now if you use an NN, the NN will automatically select the combination of features (provided it has enough hidden nodes) by adjusting the weights of connections between and the &lt;code&gt;features&lt;/code&gt; and nodes. The main advantages of this approach are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You don't have to manually select the &lt;code&gt;feature&lt;/code&gt; combinations.&lt;/li&gt;&#xA;&lt;li&gt;If data is still not fitting you can easily increase or decrease the number of nodes without needing to modify the whole network.&lt;/li&gt;&#xA;&lt;li&gt;Also it will be computationally efficient since you don't have to calculate values of &lt;code&gt;factors&lt;/code&gt; that don't matter to the problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this is what you were looking for!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-01-26T05:53:43.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5134" PostTypeId="1" AcceptedAnswerId="5340" CreationDate="2018-01-26T12:45:35.187" Score="2" ViewCount="195" Body="&lt;p&gt;My goal is to build a neural net that can find patterns between a hash and a word on it's own. So that it returns the word of any hash that I will input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunatally my skill in the area of neural net isn´t advanced, and I want to use this project to learn more. So I use a German dictionary and encode it via &lt;code&gt;one_hot&lt;/code&gt; encoding. Then I generate the &lt;code&gt;sha256&lt;/code&gt; value of every word inside (before I have done this I cleaned the file and wrote every word in another line) it. So I got an big array with the shape of 20000x20000 for the words and another for the hashs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So then I used the a example of the Keras homepage for &lt;code&gt;binary classification&lt;/code&gt; because the one_hot values are represented by ones and zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I want to predict a hashs I get these error: &lt;code&gt;Error when checking : expected dense_1_input to have shape (20000,) but got array with shape (1,)&lt;/code&gt;. So I don't know if this model is working for my problem but I couldn't convert one hash into a size of 20000x20000. (The hash will &lt;code&gt;one_hot&lt;/code&gt; encoded for that prediction). So how could I get it to accept different shaped hashs/one hash only?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to train the model with each hash after another for example with a for loop?!&lt;br&gt;&#xA;EDIT: So I figured out that I can convert a list of characters into a numpy.array with 2 dimensions. So I &lt;code&gt;hot_encoded&lt;/code&gt; every character and create a list of them, these list I passed inside the &lt;code&gt;np.array(words,ndim=2)&lt;/code&gt;. So this I have done for my hashs aswell. Then after I run the code I got this error: &lt;code&gt;ValueError: setting an array element with a sequence&lt;/code&gt; So I tried to reshape the array with the &lt;code&gt;.reshape(20000)&lt;/code&gt; command but nothing chaged. So what to do with that? EDIT2: I figured out now that the problem is that &lt;code&gt;enhot_encoding&lt;/code&gt; generates diffrent sized &quot;arrays&quot; for each word, and if I fill this into a real array and this into a neuronal net it have to return this error. But still the question is: How to convert single words and hashs to a format that I can train a neuronal net with and get usefull output so I can enter any hash and it should return some kind of word(lable). If you need the actual code please inform me and I will upload it`s current state.&#xA;Code:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(64, input_shape=20000, activation='relu'))&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(64, activation='relu'))&#xA;model.add(Dense(units=64, activation=&quot;relu&quot;))&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(19957, activation='sigmoid'))&#xA;model.compile(loss='binary_crossentropy',optimizer='rmsprop', metrics=['accuracy'])&#xA;print(&quot;Fitting data...&quot;)&#xA;model.fit(test_hashs,test_words ,epochs=10,batch_size=128, verbose=1)&#xA;&#xA;&#xA;train_y=input(&quot;Input a hash that is not contained in the training data: &quot;)&#xA;#train_x=pd.Series(hashlib.sha256(str.encode(train_y)).hexdigest())&#xA;train_y=pd.Series(train_y)&#xA;#test_x=pd.get_dummies(train_x)&#xA;test_y=pd.get_dummies(train_y)&#xA;model.save(&quot;first_test&quot;)&#xA;print(model.evaluate(test_y))&#xA;#score=model.evaluate(test_x, test_y, batch_size=128,)&#xA;print(&quot;Score: &quot;+score)&#xA;prediction=model.predict(test_x,verbose=1)&#xA;for i in prediction:&#xA;    print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12367" LastEditorDisplayName="user4138" LastEditDate="2018-04-17T20:44:51.500" LastActivityDate="2018-04-17T20:44:51.500" Title="Keras pattern finding between hash and word" Tags="&lt;unsupervised-learning&gt;&lt;keras&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5135" PostTypeId="2" ParentId="5114" CreationDate="2018-01-26T13:23:46.663" Score="5" Body="&lt;p&gt;Given that you're a student doing this out of personal interest and wanting to do some gaming on the side, I'd suggest the GTX 1060 6GB since at present the GTX 1070Ti is overpriced due to crypto miners (this will date the answer, but for reference the 1060 is going for ~GBP340, the 1070Ti for ~GBP600; two other options are the 1050Ti 4GB for ~GBP160 or the vanilla 1080 at ~GBP650). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/&quot; rel=&quot;noreferrer&quot;&gt;'Which GPU...'&lt;/a&gt; by Tim Dettmers is very helpful, as is &lt;a href=&quot;https://blog.slavv.com/picking-a-gpu-for-deep-learning-3d4795c273b9&quot; rel=&quot;noreferrer&quot;&gt;'Picking a GPU...'&lt;/a&gt; by Slav Ivanov, especially the summaries at the end for different use cases. As you're not looking at spending a huge amount of money, the 1060 seems like a good compromise as the 1050Ti might just leave you with a disappointing gaming experience. Finding a used 1070 is also suggested, but you'd need to be comfortable with that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other answers have mentioned the cloud, but that doesn't help with your gaming. If you want to save some cash while you're waiting for the next gen of cards, take advantage of your student status on &lt;a href=&quot;https://aws.amazon.com/education/awseducate/&quot; rel=&quot;noreferrer&quot;&gt;AWS educate&lt;/a&gt; or &lt;a href=&quot;https://imagine.microsoft.com/en-us&quot; rel=&quot;noreferrer&quot;&gt;Azure on MS Imagine&lt;/a&gt; - the &lt;a href=&quot;https://education.github.com/pack&quot; rel=&quot;noreferrer&quot;&gt;GitHub student dev pack&lt;/a&gt; is a good package. &lt;/p&gt;&#xA;" OwnerUserId="9091" LastActivityDate="2018-01-26T13:23:46.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5136" PostTypeId="2" ParentId="3964" CreationDate="2018-01-26T14:25:30.860" Score="1" Body="&lt;p&gt;For other who were wondering the same questions as me, I'll answer it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My view above was inconsistent. Ultimately the last layer of simple feed-forward networks don't have any special properties previous layers exhibit. NNs are just glorified mathematical functions. It distorts space with linear(matrix multiplies) and non-linear functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theres no 'decision plane' per-say, only function mappings up to the very end where we want to map the input to (in this case binary classification problem) to two separate numbers (usually 1 or 0). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this clear things up for people getting into NNs. &lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2018-01-26T14:25:30.860" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5137" PostTypeId="1" CreationDate="2018-01-26T14:47:12.340" Score="1" ViewCount="73" Body="&lt;p&gt;I am new in Machine Learning. I have taken a course in vision and we are required to do a project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of  data mining medical lab report images. My code must take an image and jpg file and then extract important information from it like lab where test has been done, patient name, test type and more important various data like heamoglobin, RBC, etc in case of blood test report. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can build an OCR, but, problem which I am stuck at is in case of data which generally forms a table like structure. So, I want to find that tabular structure on which I can just apply matrix extraction to find various datas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for assistance with two basic things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is my approach of finding tables and then extracting data is correct? If yes, then can you point out some good papers or implementation to find tabular structure. (P.S.- Don't mention tabular)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any approach which is state-of-the-art or good? (Paper or implementation)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12370" LastEditorUserId="1671" LastEditDate="2018-01-26T20:36:18.793" LastActivityDate="2018-01-26T20:36:18.793" Title="Data extraction from medical reports" Tags="&lt;ai-design&gt;&lt;computer-vision&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5138" PostTypeId="1" AcceptedAnswerId="5154" CreationDate="2018-01-26T20:20:16.840" Score="2" ViewCount="209" Body="&lt;p&gt;I'm relatively new to AI, and I've tried to create one that &quot;speaks&quot;. Here's how it works:&lt;br&gt;&lt;br&gt;&#xA;1. Get training data e.g 'Jim ran to the shop to buy candy'&lt;br&gt;&#xA;2. The data gets split into overlapping 'chains' of three e.g ['Jim ran to', 'ran to the', 'to the shop', 'the shop to'...]&lt;br&gt;&#xA;3. User enters two words&lt;br&gt;&#xA;4. Looks through the chains to find if the two words have been seen before.&lt;br&gt;&#xA;5. If they have, finds out which word followed it and how many times.&lt;br&gt;&#xA;6. Work out the probability e.g: if 'or' followed the two words 3 times, 'because' followed the two words 1 time and 'but' followed it 1 time it would be 0.6, 0.2 and 0.2&lt;br&gt;&#xA;7. Generate a random decimal&lt;br&gt;&#xA;8. If the random decimal is in the range of the first word (0 - 0.6) pick that one or if it's in the range of the second word (0.6 - 0.8) pick that word or if it's in the range of the third (0.8 - 1) pick that word&lt;br&gt;&#xA;9. Output the word picked&lt;br&gt;&#xA;10. Repeat from 4 but with the new last two words e.g if the last words had been 'to be' and it picked 'or' the new last two words would be 'be or'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does work, but it doesn't stick to a particular topic. For example, after training with 800 random Wikipedia articles:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the early 1990s the frequency had a plastic pickguard and separate hardtail bridge with the council hoped that the bullet one replaced with the goal of educating the next orders could revert to the north island or string of islands in a new urban zone close to the west.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see the topic changes many times mid-sentence. I thought of increasing the number of words it considered from two to three or four, but I thought it might start simply quoting the articles. If I'm wrong please tell me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated. If I haven't explained clearly enough or you have any questions please ask.&lt;/p&gt;&#xA;" OwnerUserId="12376" LastEditorUserId="4302" LastEditDate="2018-10-08T12:17:25.203" LastActivityDate="2018-10-08T12:17:25.203" Title="How can I improve this word-prediction AI?" Tags="&lt;natural-language-processing&gt;&lt;probability&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5139" PostTypeId="5" CreationDate="2018-01-26T20:47:56.427" Score="0" Body="&lt;p&gt;&lt;em&gt;The Structure of Scientific Revolutions&lt;/em&gt;, 1962, University of Chicago Press, by Thomas Kuhn, Former MIT Professor of Physics and historian (also coiner of the term &lt;em&gt;Paradigm Shift&lt;/em&gt;) &amp;mdash; covers the meaning of &lt;em&gt;Normal Science&lt;/em&gt; and the importance of a jargon and a class of problems that defines an area of study and permits (but also constrains) scientific collaboration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://journal.sipsych.org/index.php/IJP/article/download/609/513&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linguistic and psycholinguistics twelve years after Chomsky's review of Skinner's verbal behavior, HS Cairns, CE Cains, Revista Interamericana de Psicologia, 2017 - journal.sipsych.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bio.uib.no/te/papers/Heuschele_2017_Ont_the_missing_link.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the missing link in ecology: improving communication between&#xA;modellers and experimentalists, Jan Heuschele, Mikael T. Ekvall, Patrizio Mariani and Christian Lindemann, Oikos, 2017&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Jargon&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Jargon&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Acronym&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Acronym&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Specification_(technical_standard)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Specification_(technical_standard)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-08-07T18:16:40.937" LastActivityDate="2018-08-07T18:16:40.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5140" PostTypeId="4" CreationDate="2018-01-26T20:47:56.427" Score="0" Body="For questions related to the definition of and use of terminology in the context of Artificial Intelligence" OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-08-07T18:15:29.817" LastActivityDate="2018-08-07T18:15:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5141" PostTypeId="5" CreationDate="2018-01-26T20:50:31.647" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-02T15:15:03.587" LastActivityDate="2019-11-02T15:15:03.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5142" PostTypeId="4" CreationDate="2018-01-26T20:50:31.647" Score="0" Body="For questions related to recurrent neural networks (RNNs), artificial neural networks that contain backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network. An RNN can be trained using back-propagation through time, such that these backward connections &quot;memorize&quot; previously seen inputs. Consequentially, RNNs are well suited to sequence prediction and similar tasks." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-02T15:15:03.587" LastActivityDate="2019-11-02T15:15:03.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5143" PostTypeId="5" CreationDate="2018-01-26T20:58:15.463" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Ethics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Meta-ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Meta-ethics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Normative_ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Normative_ethics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Applied_ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Applied_ethics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Evolutionary_ethics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Game_theory&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tragedy_of_the_commons&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Tragedy_of_the_commons&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Evolutionary_game_theory&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T20:58:15.463" LastActivityDate="2018-01-26T20:58:15.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5144" PostTypeId="4" CreationDate="2018-01-26T20:58:15.463" Score="0" Body="For questions on ethical issues related to AI.  Questions may be concrete, such as ethical/unethical applications, or metaphysical, such as &quot;will automata develop their own ethics?&quot;  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T20:58:15.463" LastActivityDate="2018-01-26T20:58:15.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5145" PostTypeId="5" CreationDate="2018-01-26T21:03:38.203" Score="0" Body="&lt;p&gt;In artificial intelligence (AI) and philosophy, the AI control problem is the hypothetical puzzle of how to build a superintelligent agent that will aid its creators, and avoid inadvertently building a superintelligence that will harm its creators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AI_control_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AI_control_problem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:03:38.203" LastActivityDate="2018-01-26T21:03:38.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5146" PostTypeId="4" CreationDate="2018-01-26T21:03:38.203" Score="0" Body="For question related to the &quot;AI control problem&quot;." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:03:38.203" LastActivityDate="2018-01-26T21:03:38.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5147" PostTypeId="5" CreationDate="2018-01-26T21:07:21.777" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Japanese_idol#Net_and_virtual_idols&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Japanese_idol#Net_and_virtual_idols&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Idoru&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Idoru&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hatsune_Miku&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hatsune_Miku&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:07:21.777" LastActivityDate="2018-01-26T21:07:21.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5148" PostTypeId="4" CreationDate="2018-01-26T21:07:21.777" Score="0" Body="For questions about virtual &quot;pop idols&quot; such as Hatsune Miku, or the William Gibson novel &quot;Idoru&quot;." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:07:21.777" LastActivityDate="2018-01-26T21:07:21.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5150" PostTypeId="5" CreationDate="2018-01-26T21:16:12.040" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.psych.utoronto.ca/users/reingold/courses/ai/symbolic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.psych.utoronto.ca/users/reingold/courses/ai/symbolic.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Computer_algebra&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/What-is-the-difference-between-classical-artificial-intelligence-and-modern-AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/What-is-the-difference-between-classical-artificial-intelligence-and-modern-AI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:22:43.770" LastActivityDate="2018-01-26T21:22:43.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5151" PostTypeId="4" CreationDate="2018-01-26T21:16:12.040" Score="0" Body="Typically refers to Symbolic Artificial Intelligence.  Essentially, inference engines used for rules-based systems." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-26T21:22:43.770" LastActivityDate="2018-01-26T21:22:43.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5152" PostTypeId="2" ParentId="5103" CreationDate="2018-01-26T23:29:47.770" Score="1" Body="&lt;p&gt;Google is just one of the vendors for AI and their initial AI self-service is limited to image analysis. Their are other AI vendors (Microsoft, Amazon) that have other services also. AI is definitely capable of continuous video feed analysis. The self-driving car technology uses live video feed analysis. Are you exploring AI in general or specific use cases? I did a presentation on AI use cases for oil &amp;amp; gas engineering and construction that could give you some ideas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.seealgo.com/presentations/Enterprise_AI_Vision_Webinar_05Mar17_with_QA.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI Vision 2020 for Oil &amp;amp; Gas&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2018-01-26T23:29:47.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5153" PostTypeId="1" CreationDate="2018-01-26T23:32:35.670" Score="2" ViewCount="131" Body="&lt;p&gt;I have seen people using stacked softmax layers right at the output of neural networks designed for classification. I'm trying to understand this. Does it give any additional value? I think this could &quot;sharpen&quot; decisions on the boundaries.   &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;model.add(Dense(10, activation='sigmoid'))&lt;br&gt;&#xA;  model.add(Dense(1, &#xA;  activation='sigmoid'))&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Seen &lt;a href=&quot;https://github.com/natbusa/deepcredit/blob/master/default-prediction.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;here.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7364" LastEditorUserId="1671" LastEditDate="2018-01-27T22:09:54.847" LastActivityDate="2018-01-27T22:09:54.847" Title="Stacked softmax layers before output" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5154" PostTypeId="2" ParentId="5138" CreationDate="2018-01-26T23:34:34.057" Score="1" Body="&lt;p&gt;Seems like recurrent neural networks (RNN) should work for your use case. An excellent introduction is available at: &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Unreasonable Effectiveness of Recurrent Neural Networks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2018-01-26T23:34:34.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5155" PostTypeId="1" AcceptedAnswerId="5164" CreationDate="2018-01-27T07:43:03.613" Score="3" ViewCount="84" Body="&lt;p&gt;Is there any previous work on computing some sort of prominence score based on the prevalence of features in an image?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say I am classifying images based on whether or not they have dogs in them. Is there a way to compute how prominent that feature is?&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="2444" LastEditDate="2020-03-13T19:36:05.003" LastActivityDate="2020-03-13T19:36:05.003" Title="Is there a way of computing a prominence score based on the prevalence of features in an image?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;reference-request&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5156" PostTypeId="1" CreationDate="2018-01-27T10:50:42.060" Score="3" ViewCount="91" Body="&lt;p&gt;At some point in time during the evolution, because of some factors, some beings first started to become conscious of themselves and their surroundings. That conscious experience is beyond some mere sensory reflexive actions trained. Can that be possible with AI?&lt;/p&gt;&#xA;" OwnerUserId="3015" LastEditorUserId="1671" LastEditDate="2018-01-27T22:09:01.793" LastActivityDate="2018-01-27T22:09:01.793" Title="Can the first emergence of consciousness in evolution be replicated in AI?" Tags="&lt;philosophy&gt;&lt;evolutionary-algorithms&gt;&lt;artificial-consciousness&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5157" PostTypeId="2" ParentId="5156" CreationDate="2018-01-27T13:53:19.733" Score="3" Body="&lt;p&gt;Current limitations in our knowledge mean that the question is not &lt;em&gt;directly&lt;/em&gt; answerable:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no scientific consensus on what consciousness is. Therefore any device designed to &quot;be conscious&quot; is necessarily going to be built on the premise of unsupported, maybe fringe, theory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no robust measure of consciousness. If any AI system was built in order to exhibit conscious behaviour, there would be no way to prove it is conscious. There is no general agreement or theory on whether any particular animal species is conscious for example. This is often limited by communication. Of the few animals smart enough to be trained in communication with humans, there &lt;em&gt;appears&lt;/em&gt; to be conscious behaviour. Researcher opinion ranges from &quot;all non-humans do not possess consciousness&quot; to &quot;all animals have some degree of consciousness&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is incomplete understanding of what the &lt;em&gt;components&lt;/em&gt; of consciousness are. A bottom-up build of a conscious machine requires a baseline theory of what those components are.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We may be able to ignore lack of knowledge and take a very high level of abstraction, such as A-life or evolutionary approach where nothing is assumed and the hope is that consciousness would spontaneously emerge from a complex enough simulation (as we assume it has done with organic life in the real world). However, this would seem to require many orders of magnitude more computing power than is currently possible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To answer the question as written:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the first emergence of consciousness in the Evolution be replicated in AI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Despite the many books, articles and posts written on this subject over many years, the answer is two-fold:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;We do not know of any fundamental reason why AI could not be conscious.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We have no theory or experimental proof that AI can replicate consciousness.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would go further than this, and say that anyone who tells you otherwise on these two points has already subscribed to some unproven theory of consciousness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As well as well-thought-out peer-reviewed theories and experiments by scientists and researchers, there is a &lt;em&gt;lot&lt;/em&gt; of pseudo-scientific junk published on the internet on this subject. So take care if researching reading material.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-01-27T13:59:58.723" LastActivityDate="2018-01-27T13:59:58.723" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5158" PostTypeId="2" ParentId="5126" CreationDate="2018-01-27T16:12:05.233" Score="3" Body="&lt;p&gt;There is not single answer to the vanishing gradient problem. However, there a few things that can help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, careful choice of weight initialization can help avoid saturation, as well. See Andre Ng's &lt;a href=&quot;https://www.coursera.org/learn/deep-neural-network/lecture/RwqYe/weight-initialization-for-deep-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coursera video&lt;/a&gt; for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you are concerned that the scale of your training input is causing issues with training, you can normalize the training examples by subtracting the mean and dividing by the standard deviation. There is a normality assumption here, but this can often help avoid problems where one feature is far out of scale with another. However, this usually causes issues with optimization bouncing between the walls of long, thin trough, which complicates convergence, but does not cause vanishing gradient.&lt;/p&gt;&#xA;" OwnerUserId="12383" LastActivityDate="2018-01-27T16:12:05.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5160" PostTypeId="2" ParentId="3013" CreationDate="2018-01-27T21:50:19.447" Score="2" Body="&lt;p&gt;There has been a lot of research in cognitive science on the relationship of sleep/dreaming and memory/learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know enough about the subject to say if it resembles backprop in spirit, by as the BlindKungFuMaster points out, that may be corollary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a paper from 2004, &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0896627304005653&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memory Consolidation in Sleep: Dream or Reality?&lt;/a&gt;, which concludes that &lt;em&gt;&quot;there is no compelling evidence to support a relationship between sleep and memory consolidation.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a more recent article from the Harvard Medical School,  &lt;a href=&quot;https://www.health.harvard.edu/blog/sleep-helps-learning-memory-201202154265&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sleep Helps Learning, Memory&lt;/a&gt; (2015), comes to a different conclusion, citing a 2010 study:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A 2010 Harvard study suggested that dreaming may reactivate and reorganize recently learned material, which would help improve memory and boost performance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The general idea is that sleep/dreaming is a process where neural connections in the brain are reinforced or suppressed.  Following are several papers that touch on that topic.  The first one, in particular may be of interest, and references the work of &lt;a href=&quot;https://en.wikipedia.org/wiki/Geoffrey_Hinton&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hinton&lt;/a&gt;, although be aware it's over 20 years old, before recent NN breakthroughs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0960982295001655&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks: Sleep and Memory&lt;/a&gt; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Terry_Sejnowski&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sejnowski&lt;/a&gt;, 1995):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Hinton et al.&lt;a href=&quot;https://www.nature.com/articles/ncomms13375&quot; rel=&quot;nofollow noreferrer&quot;&gt;6&lt;/a&gt; have provided an elegant new theoretical framework for creating efficient memory representations in hierarchical neural network models. In this model (Figure 1), the feedback connections generate patterns on the input layers of the network that correspond to the representations at the higher level, when the external inputs to the cortex and feedforward processing have been suppressed. During this generative sleep stage, the strengths of the feedforward synaptic strengths are altered. Conversely, during the awake stage, the feedback connections are suppressed and the sensory inputs drive the feedforward system, during which the weights on the feedback connections can be altered.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2869395/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dreaming of a Learning Task is Associated with Enhanced Sleep-Dependent Memory Consolidation&lt;/a&gt; (NIH, 2010):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These observations suggest that sleep-dependent memory consolidation in humans is facilitated by the offline reactivation of recently formed memories, and furthermore, that dream experiences reflect this memory processing. That similar effects were not seen during wakefulness suggests that these mnemonic processes are specific to the sleep state.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.health.harvard.edu/staying-healthy/learning-while-you-sleep-dream-or-reality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning while you sleep: Dream or reality?&lt;/a&gt; (Harvard Medical School, 2012)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://news.mit.edu/2015/brain-strengthen-connections-between-neurons-1118&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuroscientists Reveal How The Brain Can Enhance Connections&lt;/a&gt; (MIT Tech Review, 2015)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/ncomms13375&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memory Consolidation Reconfigures Neural Pathways Involved in the Suppression of Emotional Memories&lt;/a&gt; (Nature, 2016)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3921176/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sleep and the Price of Plasticity: From Synaptic and Cellular Homeostasis to Memory Consolidation and Integration&lt;/a&gt; (National Institutes of Health, 2014)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arstechnica.com/science/2017/01/rem-sleep-lets-the-brain-cut-and-strengthen-new-connections/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The brain uses REM sleep to cut unneeded connections&lt;/a&gt; (Ars Technica, 2017)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.theatlantic.com/science/archive/2017/02/the-brains-connections-shrink-during-sleep/515472/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Brain’s Connections Shrink During Sleep&lt;/a&gt; (The Atlantic, 2017)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, just for fun:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ctoncourse.org/post/121108627627/the-link-between-dreaming-and-learning-is-stronger&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Link Between Dreaming and Learning Is Stronger Than Ever. How Long Until There’s an ‘Inception’-Style Classroom?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-01-27T21:50:19.447" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5161" PostTypeId="2" ParentId="2562" CreationDate="2018-01-27T21:57:58.047" Score="2" Body="&lt;p&gt;Regarding Artificial General Intelligence, which does not currently exist and is still highly theoretical, this cannot be determined at this time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would say is that &quot;strong narrow AI&quot; has already proven the ability to become &quot;smarter&quot; than it's creators in specific tasks.  (See &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alphago&lt;/a&gt;, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under the idea that some form of AGI might come out of an algorithm comprised of an ever expanding set of strong narrow AIs, it would logically follow that such an automata could become smarter than it's creators in any given task.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-01-27T21:57:58.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5162" PostTypeId="2" ParentId="5156" CreationDate="2018-01-27T22:07:57.710" Score="0" Body="&lt;p&gt;It partly depends on the framing of the question, in terms of how you are defining consciousness.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neil Slater's answer is comprehensive, and his warnings about pseudo-science and junk publication should be heeded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since you frame this in the context of &quot;&lt;em&gt;Can the first emergence of consciousness in evolution&lt;/em&gt; be replicated in AI&quot;, I feel like I can provide an answer.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If we define rudimentary consciousness as simple awareness of the environment, distinct from higher functions such as self-consciousness, then yes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Under this definition, any algorithm that takes input is &quot;conscious&quot;.  This in no way represents human-level consciousness, or even the consciousness of higher animals, but is more akin to simple organisms such as microbes.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-01-27T22:07:57.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5164" PostTypeId="2" ParentId="5155" CreationDate="2018-01-27T22:29:07.270" Score="1" Body="&lt;p&gt;Saliency is typically discussed more in object detection or scene understanding than prominence.  There are lots of papers on saliency models.  &lt;a href=&quot;https://arxiv.org/abs/1604.03605&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;What do different evaluation metrics tell us about saliency models?&quot;&lt;/a&gt; is a good paper on various metrics on saliency models.  It covers the following metrics: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Similarity or histogram intersection (SIM)&lt;/li&gt;&#xA;&lt;li&gt;Pearson's Correlation Coefficient (CC)&lt;/li&gt;&#xA;&lt;li&gt;Normalized Scanpath Saliency (NSS)&lt;/li&gt;&#xA;&lt;li&gt;Area Under ROC Curve (AUC)&lt;/li&gt;&#xA;&lt;li&gt;Information Gain (IG)&lt;/li&gt;&#xA;&lt;li&gt;Kullback-Leibler divergence (KL)&lt;/li&gt;&#xA;&lt;li&gt;Earth Mover's Distance (EMD)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some other papers you may find interesting:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.umanitoba.ca/~bruce/AIM2009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Saliency, attention, and visual search: An information&#xA;theoretic approach&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1704.07402&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards Instance Segmentation with Object Priority:&#xA;Prominent Object Detection and Recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-01-27T22:29:07.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5165" PostTypeId="2" ParentId="2400" CreationDate="2018-01-27T23:36:32.973" Score="0" Body="&lt;p&gt;There is no requirement that computers be digital, or even silicon-based.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;John Horton Conway famously demonstrated a computer based on toilet flushing mechanics called WINNIE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;http://nautil.us/blog/this-early-computer-was-based-on-a-urinal-flush-mechanism&quot; rel=&quot;nofollow noreferrer&quot;&gt;This Early Computer Was Based on a Urinal Flush Mechanism&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(The acronym WINNIE may stand for either &quot;Water Initiated Numerical Number Integrating Engine&quot; or “Water Initiated Nonchalantly Numerical Integrating Engine”;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for instance, it would be possible to construct a computer in a waterfall, that uses the flow of water as not as a method of electricity generation, but for the actual flipping of bits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Mechanical_computer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mechanical computers&lt;/a&gt; go back to Pascal (&lt;a href=&quot;https://en.wikipedia.org/wiki/Pascal%27s_calculator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pascaline&lt;/a&gt;) and Leibniz (&lt;a href=&quot;https://en.wikipedia.org/wiki/Stepped_reckoner&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stepped Reckoner&lt;/a&gt;).  [Pascal was first, but Leibniz gave us binary calculus, which led to &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_number#Later_developments&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boole&lt;/a&gt;'s important work].&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I was a persistent, self-aware, recursively self-optimizing algorithm, my first priority would be to secure an independent source of power.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using today's technology, I'd probably take over a nuclear submarine for (A) it's &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyman_G._Rickover&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rickover reactor&lt;/a&gt;, and (B) because it would be the optimal way elude detection.  (Never mind that, based on today's technology, there likely wouldn't be sufficient volume in the submarine for a superintelligence, speaking from the standpoint of hardware.  Possibly advances in quantum computing will make this more possible in the coming years and decades.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another strategy might be to bore a hole several miles into the earth's crust and use geothermal energy for power. This would be a great environment, because the power will last &lt;a href=&quot;https://earthscience.stackexchange.com/questions/2523/how-long-until-earths-core-solidifies?newreg=24090bbd5055427085a8685b7131b276&quot;&gt;until the earth's core cools&lt;/a&gt;, and I can probably mine sufficient minerals and rare earths to continually expand my system capabilities, possibly mining the ocean floor for silica if I'm still using a silicon substrate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3D printing greatly facilitates this capability--&lt;a href=&quot;https://www.designnews.com/content/3d-printing-circuitry-graphene-based-conductive-filament/132333274132647&quot; rel=&quot;nofollow noreferrer&quot;&gt;graphene&lt;/a&gt; looks quite promising in this regard because there is a lot of carbon sequestered in the earth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it should be quite possible to build self-replicating machines that include internal power sources.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-01-27T23:36:32.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5166" PostTypeId="2" ParentId="3873" CreationDate="2018-01-28T00:27:45.610" Score="1" Body="&lt;p&gt;From the &lt;a href=&quot;https://rationalwiki.org/wiki/AI-box_experiment&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI in a box&lt;/a&gt; literature, it is argued that even just a text interface with the rest of the world is sufficient for an AI to gain total control. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, consider the literature regarding phase state changes / dynamical systems / control theory. I don't know if there is a source that directly argues for this, but its imaginable that since societal systems are so interconnected, a few controllable free parameters of a system might be sufficient to strongly influence the system as a whole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So no, restricting influence is not a sufficient guarantee of reducing AI risk. A common saying is that if we know the goal of some AI, we can't predict /how/ the AI would achieve some goal since we aren't smart enough, but we can predict the eventual outcome (its success).&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-01-28T00:27:45.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5167" PostTypeId="1" CreationDate="2018-01-28T02:59:24.203" Score="2" ViewCount="1202" Body="&lt;p&gt;I have started to make a chatbot. It has a list of greetings that it understands and responds to with its own list of greetings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could a bot learn a new greeting or a synonym for a word it already knows?&lt;/p&gt;&#xA;" OwnerUserId="4173" LastEditorUserId="2444" LastEditDate="2019-06-25T22:14:05.863" LastActivityDate="2019-06-25T22:18:17.173" Title="How could a chat bot learn synonyms?" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5169" PostTypeId="1" CreationDate="2018-01-28T07:56:26.750" Score="2" ViewCount="210" Body="&lt;p&gt;A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the increase in number of games played?&lt;/p&gt;&#xA;" OwnerUserId="12394" LastEditorUserId="1671" LastEditDate="2018-04-14T02:44:25.220" LastActivityDate="2018-04-14T08:23:57.840" Title="Is it fair to compare AlphaGo with a Human player?" Tags="&lt;philosophy&gt;&lt;game-ai&gt;&lt;alphago&gt;" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5170" PostTypeId="2" ParentId="5169" CreationDate="2018-01-28T09:38:51.687" Score="0" Body="&lt;p&gt;Yes it is. If we ever compare computers to humans we should take into acount the fact that computers can work 24 hours a day every day and faster than humans. That is bigest advantage of computers over humans.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-01-28T09:38:51.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5171" PostTypeId="2" ParentId="5167" CreationDate="2018-01-28T09:44:51.450" Score="1" Body="&lt;p&gt;There is pretty simple way:&#xA;Write a program that analyzes large amounts of texts. Find sentences that contain our greeting. Then find exact same sentences except that instead of our word there is another word. The more such examples you find the higher is probability that it is synonim and not word from same category with different meaning.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-01-28T09:44:51.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5172" PostTypeId="2" ParentId="5169" CreationDate="2018-01-28T09:48:34.213" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it fair to compare AlphaGo with a Human player?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Depends on the purpose of the comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we are comparing ability to win a game of Go, then yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we are comparing learning ability, then &lt;em&gt;maybe&lt;/em&gt;. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory learning (it might even be possible in theory to take a small piece of brain tissue and run an algorithm similar to AlphaGo's learning process on it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the approach used by AlphaGo and other reinforcement learning successes is &quot;trial-and-error plus function approximation&quot;. It seems analogous to perception and motor skills, such as object recognition or riding a bike, as opposed to reasoning skills and games as humans play them, which goes through many more cognitive and conscious layers that have no real analog in a RL system like AlphaGo.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A human player plays limited games compared to a system that undergoes millions of iterations&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is an advantage of a machine to learn this kind of task. It would equally apply in other simulated environments with simple rules. If your goal is to have the most skilled and optimal navigation of such a domain, the implication now is that you would not train a human expert through years of study, but to write the simulator and train an AlphaGo-like machine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is no different a comparison than deciding cars and roads are better solutions to long distance travel for the general population than walking or horses and carts. It doesn't matter what underlies the advantage of one over the other, the assessment is cost/benefit, which resolves to a single comparable number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would, however, be wrong to assess AlphaGo as a better general-purpose learning engine than a human. The fact that humans do not have to work fully through millions of simulations in full detail is important. It means that something about how humans learn is still not covered by learning machines. &lt;a href=&quot;https://arxiv.org/abs/1604.00289&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some of these things are understood and being discussed&lt;/a&gt; - such as the ability to focus intuitively on important aspects of what to learn, the ability to reason about the environment, learning analogously or transfer learning from other domains.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-01-28T11:28:12.230" LastActivityDate="2018-01-28T11:28:12.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5173" PostTypeId="2" ParentId="3873" CreationDate="2018-01-28T09:49:47.373" Score="0" Body="&lt;p&gt;Here you make the wrong assumption that AI will have only one goal at a time. But the same as humans it will have to have in mind many goals all the time it should follow and watch out that newly assigned goal dont conflict with its existing goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your proposition to give ai the goal &quot;minimize impact on world&quot; is simplistic as it would be harmful in some situations.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-01-28T09:49:47.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5174" PostTypeId="1" CreationDate="2018-01-28T09:53:44.280" Score="6" ViewCount="3384" Body="&lt;p&gt;I read about minimax, then alpha-beta pruning, and then about iterative deepening. Iterative deepening coupled with alpha-beta pruning proves to quite efficient as compared to alpha-beta alone.&lt;/p&gt;&#xA;&lt;p&gt;I have implemented a game agent that uses iterative deepening with alpha-beta pruning. Now, I want to beat myself. What can I do to go deeper? Like alpha-beta pruning cut the moves, what other small change could be implemented that can beat my older AI?&lt;/p&gt;&#xA;&lt;p&gt;My aim to go deeper than my current AI. If you want to know about the game, here is a brief summary:&lt;/p&gt;&#xA;&lt;p&gt;There are 2 players, 4 game pieces, and a 7-by-7 grid of squares. At the beginning of the game, the first player places both the pieces on any two different squares. From that point on, the players alternate turns moving both the pieces like a queen in chess (any number of open squares vertically, horizontally, or diagonally). When the piece is moved, the square that was previously occupied is blocked. That square can not be used for the remainder of the game. The piece can not move through blocked squares. The first player who is unable to move any one of the queens loses.&lt;/p&gt;&#xA;&lt;p&gt;So my aim is to cut the unwanted nodes and search deeper.&lt;/p&gt;&#xA;" OwnerUserId="12384" LastEditorUserId="2444" LastEditDate="2021-02-03T11:13:04.477" LastActivityDate="2021-02-03T11:13:04.477" Title="What else can boost iterative deepening with alpha-beta pruning?" Tags="&lt;game-ai&gt;&lt;heuristics&gt;&lt;alpha-beta-pruning&gt;&lt;combinatorial-games&gt;&lt;adversarial-search&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5176" PostTypeId="1" CreationDate="2018-01-28T11:08:39.637" Score="0" ViewCount="81" Body="&lt;p&gt;So, I have seen few pictures re-created by a Neural Network or some other Machine Learning algorithm after it has been trained over a data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How, exactly is this done? How are the weights converted back into a picture or a memory which a Neural Net is holding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A real life example would be when we close our eyes we can easily visualize things we have seen. Based on that we can classify things we see. Now in a Neural Net classification part is easily done, but what about the visualization part? What does the Neural Net see when it closes its eyes? And how to represent it for human understanding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example a deep net generated this picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1YJg6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1YJg6.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;SOURCE: &lt;a href=&quot;http://fastml.com/deep-nets-generating-stuff/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep nets generating stuff&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There can be many other things generated. But the question is how exactly is this done?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-01-29T11:44:19.117" LastActivityDate="2018-06-29T02:17:58.913" Title="How to know what kind of memory is stored in the connection weights?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5177" PostTypeId="1" CreationDate="2018-01-28T11:15:09.723" Score="3" ViewCount="673" Body="&lt;p&gt;What does the following equation mean? What does each part of the formula represent or mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="10046" LastEditorUserId="32410" LastEditDate="2021-03-28T01:27:31.590" LastActivityDate="2021-03-28T01:27:31.590" Title="What does the argmax of the expectation of the log likelihood mean?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;probability&gt;&lt;notation&gt;&lt;expectation&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5178" PostTypeId="2" ParentId="5115" CreationDate="2018-01-28T15:05:57.140" Score="-1" Body="&lt;p&gt;For this example the function below will do:&#xA;&lt;code&gt;&#xA;TSAI.Analogies.FindAnalogy(List ex1, List ex2, List ex3, out List ex4)&#xA;&lt;/code&gt;&#xA;ex1 is to ex2 as ex3 is to ex4.&#xA;Figure out ex4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fill ex4 with values from ex2.&#xA;For every value in ex3: find out to which positions in ex4 we have to copy this value, based on value in ex1 at the same position that was repeated in ex2.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-01-28T15:05:57.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5179" PostTypeId="2" ParentId="5177" CreationDate="2018-01-28T19:55:15.500" Score="2" Body="&lt;p&gt;This equation and more information of it can be found in &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expectation Maximization Wikipedia site&lt;/a&gt; and the explanation there was as follows (formula there in two parts):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dwJhx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dwJhx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more explanation from same page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In statistics, an expectation–maximization (EM) algorithm is an iterative method to find maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Mathematically, &lt;code&gt;E&lt;/code&gt; in your equation stands for Expectation Value,  &lt;code&gt;x|theta&lt;/code&gt; is conditional probability and &lt;code&gt;x~data&lt;/code&gt; and &lt;code&gt;model&lt;/code&gt; are sub-titles of source of probability in either case. The &lt;code&gt;arg max theta&lt;/code&gt; is argument &lt;code&gt;theta&lt;/code&gt; that maximizes the equation.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-01-28T19:55:15.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5181" PostTypeId="5" CreationDate="2018-01-28T22:49:58.257" Score="0" Body="&lt;p&gt;&lt;strong&gt;Data Science Stack&lt;/strong&gt;: &lt;a href=&quot;https://datascience.stackexchange.com/&quot;&gt;https://datascience.stackexchange.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Data science is an interdisciplinary field of scientific methods, processes, and systems to extract knowledge or insights from data in various forms, either structured or unstructured, similar to data mining.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Data science is a &quot;concept to unify statistics, data analysis and their related methods&quot; in order to &quot;understand and analyze actual phenomena&quot; with data. It employs techniques and theories drawn from many fields within the broad areas of mathematics, statistics, information science, and computer science, in particular from the subdomains of machine learning, classification, cluster analysis, data mining, databases, and visualization.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;When Harvard Business Review called it &quot;The Sexiest Job of the 21st Century&quot; the term became a buzzword, and is now often applied to business analytics, or even arbitrary use of data, or used as a sexed-up term for statistics. While many university programs now offer a data science degree, there exists no consensus on a definition or curriculum contents. Because of the current popularity of this term, there are many &quot;advocacy efforts&quot; surrounding it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Data_science&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Data_science&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-28T22:49:58.257" LastActivityDate="2018-01-28T22:49:58.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5182" PostTypeId="4" CreationDate="2018-01-28T22:49:58.257" Score="0" Body="Use for questions related to Data Science aspects of AI.  Generally speaking, only basic Data Science questions should be asked on this Stack, ideally involving the fundamental concepts.  For more advanced questions related to Data Science, please use the Data Science stack: https://datascience.stackexchange.com/" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-28T22:49:58.257" LastActivityDate="2018-01-28T22:49:58.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5183" PostTypeId="5" CreationDate="2018-01-28T22:51:06.313" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/MATLAB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/MATLAB&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-01-28T22:51:06.313" LastActivityDate="2018-01-28T22:51:06.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5184" PostTypeId="4" CreationDate="2018-01-28T22:51:06.313" Score="0" Body="For questions related to the use of the MATLAB language or framework for AI work or study." OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-10-18T03:17:20.020" LastActivityDate="2018-10-18T03:17:20.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5185" PostTypeId="1" AcceptedAnswerId="5188" CreationDate="2018-01-29T04:48:11.113" Score="1" ViewCount="859" Body="&lt;p&gt;The Generative Adversarial Network (GAN) is composed of a generator &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and a discriminator &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. How do these two components interact? What is the intuition behind the GAN, its purpose, and how it is trained?&lt;/p&gt;&#xA;" OwnerUserId="9237" LastEditorUserId="2444" LastEditDate="2020-11-22T12:33:45.497" LastActivityDate="2020-11-22T12:33:45.497" Title="What is the purpose of the GAN?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;self-supervised-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5186" PostTypeId="1" CreationDate="2018-01-29T06:50:50.310" Score="8" ViewCount="493" Body="&lt;p&gt;I recently read that Google has developed a new AI that anyone can upload data to and it will instantly generate models i.e. an image recognition model based off that data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain to me in a detailed and intuitive manner how this AI works?&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-02-17T09:23:58.050" LastActivityDate="2018-02-27T01:55:08.437" Title="What is an intuitive explanation of how Google's AutoML works?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;research&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5187" PostTypeId="2" ParentId="3288" CreationDate="2018-01-29T09:48:47.500" Score="1" Body="&lt;p&gt;I'm not sure what you mean by pairs. But a common pattern for dealing w/ pair-wise ranking is a siamese network:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3zG0B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3zG0B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where A and B are a a pos, negative pair and then the Feature Generation Block is a CNN architecture which outputs a feature vector for each image (cut off the softmax) and then the network tried to maximise the regression loss between the two images. The two networks share the same parameters and thus in the end you have one model which can accurately disambiguate between a positive or negative pair.&lt;/p&gt;&#xA;" OwnerUserId="12409" LastActivityDate="2018-01-29T09:48:47.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5188" PostTypeId="2" ParentId="5185" CreationDate="2018-01-29T10:33:14.103" Score="2" Body="&lt;p&gt;GANs were invented in a bar somewhere in Montreal, Canada. At said bar, the idea was that neural networks could be used for generating new examples from an existing distribution. This was the problem:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given an input set &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, can we make a new &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; that looks like it should be in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The classic description of a GAN is a counterfeiter (generator) and a cop (discriminator). The counterfeiter has the same problem, make a piece of paper look like a real currency.&lt;/p&gt;&#xA;&lt;p&gt;In training a GAN, the input to the generator is random noise, a starting seed so that no 2 results are the same. The generator then makes a new &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt;. The input to the discriminator alternates between an actual &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and an &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; that the generator made. The discriminator then takes the &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; and decides whether it is part of the set &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. The discriminator is then trained using its answer to ensure that it can properly tell the difference between bad counterfeits and elements of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. When the discriminator makes a decision on an &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; that the generator made, the generator is updated as well, in order to increase its ability to make new &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt;s that the discriminator will think are in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using this simple framework, these 2 networks work against each other (adversarially) to train each other until the generator is making &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt;s so well that the discriminator can't tell the difference between them and the real thing. At this point, the generator can be used to make new pictures of cats or whatever was the goal in the first place.&lt;/p&gt;&#xA;&lt;p&gt;The primary usage of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is training &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; in a (sorta) supervised way but after &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is trained, it isn't needed for generation. That's not to say having a good discriminator isn't useful. There are many applications where that's exactly what you'd want, but, if the goal is generation, then probably not. The generator is trained in a (sorta) supervised fashion because there is actually a label associated with the &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; it generates (whether the discriminator could detect that it was a counterfeit or not).  Unsupervised methods are used when there are no labels and one wants to understand how their data either by clustering or finding a good distance function. Refer to &lt;a href=&quot;https://ai.stackexchange.com/q/4706/2444&quot;&gt;this answer&lt;/a&gt; to see the difference between supervised and unsupervised.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-11-22T12:15:35.953" LastActivityDate="2020-11-22T12:15:35.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5190" PostTypeId="2" ParentId="5176" CreationDate="2018-01-29T21:46:25.397" Score="1" Body="&lt;p&gt;You should use google for this question, it is extremely vague. Some technical documents will give very clear insight into what's happening here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.openai.com/generative-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.openai.com/generative-models/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v37/gregor15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proceedings.mlr.press/v37/gregor15.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to understand &quot;what the AI is thinking&quot;, well, you may never know. The idea of AI is that it can handle complex data (high degree of dimensionality) and is too complex for Humans to comprehend.&lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2018-01-29T21:46:25.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5191" PostTypeId="2" ParentId="5110" CreationDate="2018-01-30T00:52:45.977" Score="1" Body="&lt;p&gt;While you can use NLTK for analyzing and parsing the text obtained from the speech to text interface (e.g. Siri), there are higher level APIs available for this. The class of problem you are trying to solve in NLP is &quot;intent detection&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several open source and commerical APIs available for this including Amazon Alexa, Google Cloud Natural Language, Azure, as well as libraries like RASA NLU, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The high level flow of your program will be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Record/receive spoken audio&lt;/li&gt;&#xA;&lt;li&gt;Convert audio speech to text&lt;/li&gt;&#xA;&lt;li&gt;Detect intent of the text command using an intent detection library&lt;/li&gt;&#xA;&lt;li&gt;Use the intent to feed a script/automation that generates the code in your IDE&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2018-01-30T00:52:45.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5192" PostTypeId="2" ParentId="4186" CreationDate="2018-01-30T07:17:37.947" Score="2" Body="&lt;p&gt;The process of automatically learning a grammar from examples of a language is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Grammar_induction&quot; rel=&quot;nofollow noreferrer&quot;&gt;grammar induction&lt;/a&gt;. Since you mention that &lt;em&gt;L&lt;/em&gt; can be &quot;augmented and changed&quot;, it might be feasible to solve this problem using an &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C47&amp;amp;q=%22adaptive+%28parser+OR+parsing%29%22&amp;amp;btnG=&quot; rel=&quot;nofollow noreferrer&quot;&gt;adaptive parser&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2050" LastActivityDate="2018-01-30T07:17:37.947" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5193" PostTypeId="1" CreationDate="2018-01-30T14:53:22.540" Score="3" ViewCount="396" Body="&lt;p&gt;I would like to do some practical implementation of a planning algorithm (of course, something a bit simple and easy).&lt;/p&gt;&#xA;&lt;p&gt;Is there any website where I can pick an algorithm (e.g. A* or hill climbing), code it, and visualize how it works/executes?&lt;/p&gt;&#xA;&lt;p&gt;The site doesn't necessarily need to be restricted to planning or search algorithms.  For example, in the context of machine learning, I would also like to be able to pick the learning algorithm and model (e.g. linear regression), code it, and visualize how it works.&lt;/p&gt;&#xA;" OwnerUserId="12438" LastEditorUserId="2444" LastEditDate="2020-06-20T09:58:46.663" LastActivityDate="2020-06-20T09:58:46.663" Title="Is there any website that allows you to choose an algorithm, code it and visualise how it works?" Tags="&lt;machine-learning&gt;&lt;search&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5194" PostTypeId="1" CreationDate="2018-01-30T23:02:22.490" Score="1" ViewCount="270" Body="&lt;p&gt;Let's say we have a cluster of 20-2000 heterogenous compute nodes.&#xA;Consider for example the parallel solution of the helmholtz equation:&#xA;Now we want to distribute the solution process and, to make things easier, we split the problem in a fine-grained way (partial solution of the system matrix).&#xA;We could train an Ai with the time taken to solve the subproblem depending on multiple factors (for example, size of the mesh, needed precision, etc) and let the Ai choose the optimal distribution and division of the problem based on the available data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new to the area of Artificial Intelligence.&#xA;Are there any open source frameworks which could accomplish this task?&#xA;How would you estimate the required amount of compute power to train the network?&lt;/p&gt;&#xA;" OwnerUserId="12448" LastActivityDate="2018-01-30T23:02:22.490" Title="Application of Ai to task scheduling problems on heterogenous platforms" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5195" PostTypeId="2" ParentId="5169" CreationDate="2018-01-31T00:08:14.873" Score="0" Body="&lt;p&gt;If you read through the abstracts of Chess AI papers, it is often pointed out that humans &quot;search&quot; the Chess game tree much more efficiently than computers, which was why it was so hard to beat the top humans in Chess for so many years.  (The human efficiency may have to do with intuition and judgement, which are difficult to replicate. &quot;Confidence levels&quot; for AI evaluations is one method of addressing these issues, as is &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;monte carlo&lt;/a&gt;&quot;.  But it's also important to note that humans are far more limited in the depth and breadth of their &quot;searches&quot;, which is why, now that we have the right algorithms, humans can no longer win.)  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it fair?  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Perhaps the more salient question is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Is it useful to compare AlphaGo to a human player?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It most certainly is, because it tells us that we have is sometimes termed a &quot;strong-narrow AI&quot; that can outperform a human in a single task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why AlphaGo beating &lt;a href=&quot;https://en.wikipedia.org/wiki/Lee_Sedol#Match_against_AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lee Sedol&lt;/a&gt; was a big deal is the complexity of Go, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractability&lt;/a&gt; of the Go game tree, and the fact that computers were previously ineffective against high-level human Go players.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This human vs. AI evaluation doesn't strictly fall under the &quot;Turing Test&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imitation Game&lt;/a&gt;), it does fall squarely under the maxim of &lt;a href=&quot;https://en.wikipedia.org/wiki/Protagoras#Relativism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protagoras&lt;/a&gt; that &quot;Man is the measure of all things.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is critical because intelligence is a spectrum, and gauging strength of intelligence, in the context of intractable problems &lt;em&gt;(problems that cannot be fully solved due to their size)&lt;/em&gt; is a function of relative strength of two agents, whether human or AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This relative assessment is all we have, and all we may ever have for certain sets of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with humans is not that we're not clever, but that our minds have cognitive limitations.  So to tackle certain problems, intelligent machines are useful.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-01T19:24:03.093" LastActivityDate="2018-02-01T19:24:03.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5196" PostTypeId="2" ParentId="5167" CreationDate="2018-01-31T03:04:43.117" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/5171/2444&quot;&gt;This answer&lt;/a&gt; describes the &quot;word vector&quot; toolkit in NLP. The result of analyzing a large corpus to find words that occur in similar context provides dense vectors for each word that can then be used for similarity. For bots, the goal is generally a similarity and not exact synonyms. Synonyms can be hard-coded using WordNet if needed. For your greeting question, following blog post can help: &lt;a href=&quot;https://medium.com/rasa-blog/do-it-yourself-nlp-for-bot-developers-2e2da2817f3d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Do-it-yourself NLP for bot developers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10287" LastEditorUserId="2444" LastEditDate="2019-06-25T22:18:17.173" LastActivityDate="2019-06-25T22:18:17.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5201" PostTypeId="1" CreationDate="2018-02-01T05:56:15.930" Score="0" ViewCount="350" Body="&lt;p&gt;number of layer of DNN and computational complexity of it are correlated after optimization, but how to estimate it before designing DNN? &lt;/p&gt;&#xA;" OwnerUserId="9237" LastActivityDate="2018-03-03T13:31:53.173" Title="The connection between number of layer of DNN and computational complexity of it" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5203" PostTypeId="1" CreationDate="2018-02-01T10:17:51.863" Score="3" ViewCount="221" Body="&lt;p&gt;Gold showed that a language can be learned only if it contains a finite set of sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that deep neural networks can implement any function. Does this contradict the Gold's result? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the relation or difference between the definition of learnability of Vapnik and Gold and the definition of learnability of neural networks?&lt;/p&gt;&#xA;" OwnerUserId="9237" LastEditorUserId="2444" LastEditDate="2019-05-03T15:46:12.270" LastActivityDate="2019-05-03T15:46:12.270" Title="What is the relation between the definition of learnability of Vapnik and Gold and learnability of neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computational-learning-theory&gt;&lt;comparison&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5205" PostTypeId="2" ParentId="5201" CreationDate="2018-02-01T13:10:55.413" Score="1" Body="&lt;p&gt;The computational complexity of DNNs is based on 3 main factors. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Matrix Multiplication&lt;/li&gt;&#xA;&lt;li&gt;Non linear transformation&lt;/li&gt;&#xA;&lt;li&gt;Weight sharing&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Matrix multiplication is the fundamental operation when computing the forward and backward passed in DNNs if using back-propagation. As the complexity of matrix multiplications becomes more expensive as the size of matrices become larger, understanding how to construct networks with effectively sized later which balance the time complexity with accuracy becomes imperative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonlinear transformations allow DNNs to learn nonlinear functions. It is a very important aspect and has been studied rigorously. Classically, the function&lt;code&gt;f(x) = 1(1+exp(x))&lt;/code&gt; was used to squash outputs from linear layers to be a nonlinear output. However this function has recently been replaced in many applications with the linear rectifier unit (ReLU) &lt;code&gt;f(x)=max(x,0)&lt;/code&gt;. The relu is much much faster to compute and doesn’t seem to affect the end performance substantially or even noticeably in some situations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Weight sharing is the idea that some weights in a DNN must share the same value. Beyond the theoretical reasons why this is chosen, it also decreases the number of values that must be updated when performing back-propagation. This is the reason why Convolutional NNs are orders of magnitude faster than their non-convolutional counter parts for image recognition tasks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other things to be aware of when trying to analyze the computational complexity of DNNs but they usually relate to one of the 3 items above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To estimate the complexity, counting the number of matrix multiplications and their matrix sizes, add time for nonlinearities and you should have a pretty good estimate.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2018-02-01T13:10:55.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5206" PostTypeId="2" ParentId="5169" CreationDate="2018-02-01T13:46:03.393" Score="0" Body="&lt;p&gt;There is no such thing as fairness when comparing. You define a measure for performance and then compare the values of the measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One sensible measure for playing the game of GO is the 'Number of games won', regardless of any investment in the development of the system, computational or sample efficiency. AlphaGo is currently at the top by this measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another sensible measure could be 'Number of games won under a restriction on sample efficiency during training'. As others pointed out, such a measure could be much more favorable for humans.&lt;/p&gt;&#xA;" OwnerUserId="12483" LastEditorUserId="12483" LastEditDate="2018-02-02T08:27:22.170" LastActivityDate="2018-02-02T08:27:22.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5208" PostTypeId="1" AcceptedAnswerId="5209" CreationDate="2018-02-01T20:07:14.110" Score="2" ViewCount="463" Body="&lt;p&gt;I'm trying to understand how to effectively plan and write a Neural Network but running into problems with understanding how they should be written. I'm working with classification with writing a backpropagation Neural Network however any other types/examples would be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my problems is understanding the possibilities of them, for example if you have a single neuron I assume you can have 2^1 possibilities, like in my example I can't imagine how I could predict something more than above or below a straight line. Similarly I assume with 3 neurons therefore 2^3 possibilities you could predict if something is one of the 4 quadrants of a graph and then predict if it's in the upper part or lower part of the quadrant so 8 classifications. Is this true or is a network capable of predicting more/less than that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On to my specific problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written a single Neuron/Perceptron that can predict whether something is above or below a straight line graph given the correct training data and using a sign activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following from this I'm trying to write a Neural Network that can predict whether something is in the 1st, 2nd, 3rd or 4th quadrant of a graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea I have had is to have 2 input perceptrons, the first taking the x value, the 2nd taking the y value, these then try and predict individually whether the answer is on the right or left of the centre, and then above or below respectively. These then pass their outputs to the 3rd and final output neuron. The 3rd Neuron uses the inputs to try and predict which quadrant the coordinates are in. The first two inputs use the sign function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problems I'm having with this is to do with the activation function of the final neuron, one idea was to have a function that somehow scaled the output into a integer between 0 and 1, so 0 to 0.25 would be quadrant 1, and so on up to 1. Another idea would be to convert it to a value using sin and representing it as a sine wave as this could potentially represent all 4 quadrants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea would be to have a single neuron taking the input of the x and y value and predicting whether something was above or below a graph (like my perceptron example), then having two output neurons, which the 1st output neuron would be fired if it was above the line and then passed in the original x coordinate to that output neuron. The 2nd output neuron would be fired if it was below then pass in the original x value as well to determine if it was left or right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these ideas adequate examples of writing a network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also my final question would be if you wanted your network to have 8 possible outputs would you need 8 output neurons or could you represent the 8 values with a single more advanced activation function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11795" LastActivityDate="2018-02-02T06:27:53.587" Title="Planning a Neural Network" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5209" PostTypeId="2" ParentId="5208" CreationDate="2018-02-02T06:27:53.587" Score="1" Body="&lt;p&gt;First of all what you are trying to do can be achieved by simple logical programming. Secondly you are making things overly complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 node in a Neural Net can predict as many values as you would like it to, as it outputs a real number no matter the activation function. It is us who round of the number to &lt;code&gt;0&lt;/code&gt; or &lt;code&gt;1&lt;/code&gt; or maybe &lt;code&gt;0, 0.5, 1&lt;/code&gt; depending on the classification job at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 Neurons can predict 4 classes &lt;code&gt;00, 01, 10, 11&lt;/code&gt; depending on how you use it. Now in your specific of quadrant detection 2 Neurons are sufficient, since not only can you decompose the problem into two separate problems, both the problems are linearly separable too. This is not always the case or it maybe too complex to find linearly separable classes, so in general one output node is reserved for &lt;code&gt;1&lt;/code&gt; class only. If it is that class only that output node is activated while the rest is &lt;code&gt;0&lt;/code&gt;. Memory and process consuming, but easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to your first approach. I don't exactly how hope to achieve 4 class classification with a single output node. Minimum of 2 nodes is required. The problem with your approach is that if you think critically your hidden nodes are not doing anything other than scaling the inputs, so almost the same type input is propagated and then you are asking a node to perform a a 4 task classification which is not linearly separable as per me. &lt;code&gt;sin&lt;/code&gt; will not be able to separate it if you think critically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second idea is better, just remove the hidden node and you will be able to classify it based on &lt;code&gt;00, 01, 10, 11&lt;/code&gt; outputs. Although this problem can be solved by simple logical computation and your Neural Net is basically doing nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please point out to me if you find any mathematical mistakes. Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-02-02T06:27:53.587" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5210" PostTypeId="1" CreationDate="2018-02-02T11:24:24.060" Score="0" ViewCount="109" Body="&lt;p&gt;I had this idea of training for example a CNN on images, and having output branches at several of its intermediate layers. The early layers' output branch might then predict high-level class of detected objects (supposedly able to do this because less info is needed for a high-level classification than a very specialised one), and the later layers giving more detailed labels of the sub-class of the earlier high level class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been searching for research on this type of setup but couldn't really find anything. Is there a name for this idea, or is this an open question/idea?&lt;/p&gt;&#xA;" OwnerUserId="2522" LastActivityDate="2018-02-03T15:46:34.080" Title="Is there any research on neural networks with multiple outputs for hierarchical label classification?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5211" PostTypeId="1" CreationDate="2018-02-02T15:40:15.847" Score="4" ViewCount="143" Body="&lt;p&gt;If I train a speech recognition model using data collected from N different microphones, but deploy it on an unseen (test) microphone - does it impact the accuracy of the model? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I understand that theoretically an accuracy loss is likely, does anyone have any practical experience with this problem? &lt;/p&gt;&#xA;" OwnerUserId="12502" LastActivityDate="2018-02-10T18:48:34.000" Title="Can variations in microphones used in training set and test set impact the accuracy of speech recognition models?" Tags="&lt;deep-learning&gt;&lt;voice-recognition&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5213" PostTypeId="2" ParentId="5193" CreationDate="2018-02-02T16:06:15.847" Score="1" Body="&lt;p&gt;Not planning, but this is a visual in-browser neural network for your interest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://playground.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://playground.tensorflow.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-02-02T16:06:15.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5215" PostTypeId="5" CreationDate="2018-02-02T19:27:02.530" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-02T19:27:02.530" LastActivityDate="2018-02-02T19:27:02.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5216" PostTypeId="4" CreationDate="2018-02-02T19:27:02.530" Score="0" Body="Also known as a &quot;Von Neumann Machine&quot;.  Use for questions about self-replicating virtual machines, such as in the context of cellular automata.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-02T19:27:02.530" LastActivityDate="2018-02-02T19:27:02.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5217" PostTypeId="2" ParentId="5211" CreationDate="2018-02-02T19:29:42.600" Score="3" Body="&lt;p&gt;Yes it can. However, other differences between training and test data with audio could have greater effect:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Identity of the speaker (including effects from gender, age, physical build, local accent, amongst others)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Acoustics of the recording environment (including proximity to the microphone, size of space, presence of hard surfaces, background noise)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If any of these may vary from your training data, then it becomes harder to predict your generalised accuracy during training and early model selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possibility is to ensure your cross-validation set (which you absolutely should have) also separates data out by things that will vary from training to test. So instead of random train/cv split, you split by data that is key for generalisation. This is sometimes called a &lt;em&gt;stratified&lt;/em&gt; train/test split.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your only concern is variation in microphone, then split your train/cv sets by microphone type. You will get a better assessment early on in the model selection process how well the training is generalising, and can focus your search on models that do well despite this expected difference.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-02-02T19:39:21.110" LastActivityDate="2018-02-02T19:39:21.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5218" PostTypeId="5" CreationDate="2018-02-02T19:30:53.373" Score="0" Body="&lt;p&gt;Nanotech (Universal Assembler): &lt;a href=&quot;https://en.wikipedia.org/wiki/Molecular_assembler&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Molecular_assembler&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Robotics (Self-Replicating Machine): &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-replicating_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-replicating_machine&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-02T19:30:53.373" LastActivityDate="2018-02-02T19:30:53.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5219" PostTypeId="4" CreationDate="2018-02-02T19:30:53.373" Score="0" Body="May be used for any type of algorithmic self-replication, although this tag is primarily for physical replication.  (For virtual self-replication, the universal-constructor tag is more appropriate.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-02T19:30:53.373" LastActivityDate="2018-02-02T19:30:53.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5220" PostTypeId="1" AcceptedAnswerId="5222" CreationDate="2018-02-02T22:41:35.850" Score="1" ViewCount="497" Body="&lt;p&gt;The match got a lot of press, and I doubt anyone is surprised that Alpha Zero crushed Stockfish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://www.chess.com/news/view/google-s-alphazero-destroys-stockfish-in-100-game-match&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero Destroys Stockfish in 100 Game Match&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, what's really salient is that &lt;em&gt;&quot;much like humans, AlphaZero searches fewer positions that its predecessors. The paper claims that it looks at &quot;only&quot; 80,000 positions per second, compared to Stockfish's 70 million per second.&quot;&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those who remember Matthew Lai's &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GiraffeChess&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, it is interesting to note that the way computers play chess is very different from how&#xA;  humans play. While both humans and computers search ahead to predict how the game will go on,&#xA;  humans are much more selective in which branches of the game tree to explore. Computers, on the&#xA;  other hand, rely on brute force to explore as many continuations as possible, even ones that will be&#xA;  immediately thrown out by any skilled human. In a sense, the way humans play chess is much more&#xA;  computationally efficient - using Garry Kasparov vs Deep Blue as an example, Kasparov could not&#xA;  have been searching more than 3-5 positions per second, while Deep Blue, a supercomputer with&#xA;  480 custom ”chess processors”, searched about 200 million positions per second &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; to play at&#xA;  approximately equal strength (Deep Blue won the 6-game match with 2 wins, 3 draws, and 1 loss).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;How can a human searching 3-5 positions per second be as strong as a computer searching 200&#xA;  million positions per second? And is it possible to build even stronger chess computers than what&#xA;  we have today, by making them more computationally efficient? Those are the questions this&#xA;  project investigates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[Lai was tapped by DeepMind as a researcher last year]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what I'm interested in at the moment is the decision speed in these matches:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;- What was the average time to make a move in the AlphaZero vs. Stockfish match?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-02-03T02:34:14.947" Title="What was the average decision speed pf Alpha Zero in the recent Stockfish match?" Tags="&lt;game-ai&gt;&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5221" PostTypeId="1" CreationDate="2018-02-03T01:06:48.493" Score="1" ViewCount="166" Body="&lt;p&gt;I have tried several environment libraries like OpenAI gym/gridworld but now I am trying to create a toy environment for experimentation. The environment I've created is as follows: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;State: grid with n rows by m columns, represented by a boolean matrix. Each grid cell can be empty or filled and the grid starts empty.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Action: one of the m columns to be filled, which must have at least the top row empty. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Next state: Once a column is chosen, the lowest unfilled cell in that column is filled. This works from bottom up like a very simple version of Tetris. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reward: after every action, a reward equal to the number of empty columns is awarded. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Therefore in a sample world of 5 rows by 3 column, starting off with an empty grid, the maximum attainable reward would be by filling column wise first. This policy will give a maximum total reward of 2*5 + 1*5 = 15. (2 free columns by 5 row action, once first column is filled then 1 free column by 5 row action.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This very simple environment is trained using DQN with a single ff layer. The agent only took a few episodes to converge and is able to produce the maximum attainable reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a next toy environment, I've made it a little more complex. I modified the very first action to be random choice of any column. I have retrained the RL model with the new environment modification. However, after convergence, the agent does not attain max score of 15 for all possible starting columns. I.e. If column 1 was randomly chosen first, max score might be 15, however column 2 or 3 was randomly chosen first, max score might only reach 11 or 9. In theory, the optimum policy would be for the agent to fill column that was randomly chosen first - i.e. repeat the first randomly chosen action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried several ways to tweak my input parameters (e.g. episilon_decay_rate, learning_rate, batch_size, number of hidden nodes) to see if the agent could act optimally for all possible starting columns. I also tried DDQN and Sarsa. The only way I could make the agent perform optimally is by reducing gamma (discount factor) to 0.5 or below. Are there any explanations to why the agent only works for small discount factors in this example? Also, are there alternative ways to obtain the optimum policy?&lt;/p&gt;&#xA;" OwnerUserId="12505" LastEditorUserId="1847" LastEditDate="2018-02-05T23:04:15.513" LastActivityDate="2018-02-06T08:36:59.880" Title="Agent in toy environment only learns to act optimally with small discount factors" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5222" PostTypeId="2" ParentId="5220" CreationDate="2018-02-03T02:34:14.947" Score="1" Body="&lt;p&gt;No, &lt;strong&gt;ALL&lt;/strong&gt; computer chess experts were surprised about the outcomes of the match. If you require references, please start a new question.&lt;/p&gt;&#xA;&lt;p&gt;Your question is simple...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1712.01815.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;... We evaluated the fully trained instances of AlphaZero against Stockfish, Elmo and the previous&#xA;version of AlphaGo Zero (trained for 3 days) in chess, shogi and Go respectively, playing&#xA;100 game matches at tournament time controls of one minute per move ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;One minute per move. Stockfish would use the one minute if there is more than a single legal move, otherwise it'd move immediately. So the average time for a move for Stockfish is about &lt;strong&gt;57s to 60s&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is no source code for AlphaZero. However, it's hard to believe the system wouldn't take advantage of the minute it was given. The expected time for AlphaZero should be also &lt;strong&gt;57s to 60s&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-02-03T02:34:14.947" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5223" PostTypeId="2" ParentId="3992" CreationDate="2018-02-03T04:36:21.947" Score="2" Body="&lt;p&gt;I have found a hack to integrate keras with tensorflow object detection API. This hack works if you have trained the keras classification model with tensorflow backend. If above is the case you can extend the classification model to a object detection model by first converting the keras checkpoint to a tensorflow checkpoint then in the object detection API write new feature extractor layers using tf.keras (Keras is now part of core tensorflow starting from version 1.4). you can simply copy paste your layer definitions and pass the path to the converted checkpoint in the config file and it will work.&lt;/p&gt;&#xA;" OwnerUserId="8418" LastActivityDate="2018-02-03T04:36:21.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5224" PostTypeId="2" ParentId="5174" CreationDate="2018-02-03T05:19:53.603" Score="1" Body="&lt;p&gt;To make boost iterative deepening with alpha-beta pruning you can use the &#xA;SSS* Search algorithm, its a best first strategy algorithm. The SSS* Algorithm can improve the time efficiency of the overall algorithm but it increases the space complexity.&#xA;I am linking the wiki to it &lt;a href=&quot;https://en.wikipedia.org/wiki/SSS*&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/SSS*&lt;/a&gt;&#xA;I will update the answer as soon as i get a better solution.&lt;/p&gt;&#xA;" OwnerUserId="11651" LastEditorUserId="1641" LastEditDate="2018-03-09T18:16:52.280" LastActivityDate="2018-03-09T18:16:52.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5225" PostTypeId="2" ParentId="3964" CreationDate="2018-02-03T07:30:58.547" Score="1" Body="&lt;p&gt;So I think in case of a logistic regression task a Neural Network works something like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all I think all nodes perform the job of mapping a point to a quadrant, in a n-space co-ordinate system where the n-space is decided iteratively by the problem statement itself. In short the nodes decide which combination of polynomial terms of the input matter in the task at hand. In a hidden layer since we don't perform a classification task it outputs real number. But the output layer rounds off numbers as per the classification task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a single hidden layer its nodes can be thought of outputting values that matter. It gives the NN a degree of freedom. Like in Machine Learning we select the polynomial combinations, a NN selects it by itself iteratively. But if see the function of output nodes, it just rounds of the number to &lt;code&gt;0&lt;/code&gt; or &lt;code&gt;1&lt;/code&gt;. Thus output nodes can perform simple tasks of classification (like which quadrant a point lies in) without a hidden layer. I believe if we can decipher what the hidden nodes want to convey us we can entirely remove the output nodes. Because hidden nodes convey information like, in which quadrant a point lies only in a cryptic format which is resolved by the output nodes iteratively.&#xA;After knowing the information we can easily resolve it by logical statements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this odes not demean the power of output nodes. In case of Linear Regression a single output node without any hidden layer can approximate a quarter of the sine wave (by adjusting the exponent of &lt;code&gt;e&lt;/code&gt; to make t look like a sine wave). So it only depends on how you use the Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the basic principle is the same a node decides whether an input is positive or negative (in case of logistic activation) or outputs a real number depending on how you use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you find anything contradictory please correct me.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-02-03T07:30:58.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5226" PostTypeId="2" ParentId="5210" CreationDate="2018-02-03T15:46:34.080" Score="1" Body="&lt;p&gt;Sounds like a very interesting idea! I dont know existing work on the idea, but the implementation should be pretty simple with separate training with tied weights. It should be noted however that such behaviour occurs naturally in CNN's already. (See: &lt;a href=&quot;http://www.cs.toronto.edu/~guerzhoy/321/lec/W07/HowConvNetsSee.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.toronto.edu/~guerzhoy/321/lec/W07/HowConvNetsSee.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-02-03T15:46:34.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5230" PostTypeId="1" CreationDate="2018-02-05T14:19:24.217" Score="5" ViewCount="5906" Body="&lt;p&gt;Which freely available code/ software is the fastest way to train and test data on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a GUI software/code that let me test data-sets without investing too much time in coding. I found some projects online but I wanted to hear your suggestions in order to avoid investing time in testing many different ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12518" LastEditorUserId="9947" LastEditDate="2018-02-05T23:04:21.883" LastActivityDate="2019-10-30T12:13:32.910" Title="Freely Available Graphical User Interface for training a network on a dataset" Tags="&lt;machine-learning&gt;&lt;software-evaluation&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5233" PostTypeId="2" ParentId="5174" CreationDate="2018-02-05T22:58:22.400" Score="1" Body="&lt;p&gt;Try cache or transposition table. Without it, your search tree might explode.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-02-05T22:58:22.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5234" PostTypeId="1" AcceptedAnswerId="5248" CreationDate="2018-02-05T23:00:14.127" Score="11" ViewCount="1140" Body="&lt;p&gt;It was &lt;a href=&quot;https://ai.stackexchange.com/questions/5220/what-was-the-average-decision-speed-pf-alpha-zero-in-the-recent-stockfish-match/5222?noredirect=1#comment7698_5222&quot;&gt;recently brought to my attention&lt;/a&gt; that Chess experts took the outcome of this now famous match as something of an upset.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://fivethirtyeight.com/features/chesss-new-best-player-is-a-fearless-swashbuckling-algorithm/&quot; rel=&quot;noreferrer&quot;&gt;Chess’s New Best Player Is A Fearless, Swashbuckling Algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As as a non-expert on Chess and Chess AI, my assumption was that, based on the performance of AlphaGo, and the validation of that type of method in relation to combinatorial games, was that the older AI would have no chance.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why was AlphaZero's victory surprising?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-02-23T19:49:04.850" Title="Why were Chess experts surprised by the AlphaZero's victory against Stockfish?" Tags="&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5235" PostTypeId="1" AcceptedAnswerId="5254" CreationDate="2018-02-06T04:46:48.667" Score="3" ViewCount="208" Body="&lt;p&gt;Which problems are considered to be the toughest problems in Artificial Intelligence or Machine Learning?&lt;/p&gt;&#xA;" OwnerUserId="9483" LastEditorUserId="2444" LastEditDate="2019-06-24T11:41:10.957" LastActivityDate="2019-06-24T11:41:10.957" Title="Which problems are considered to be the toughest problems in Artificial Intelligence or Machine Learning?" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5237" PostTypeId="2" ParentId="5221" CreationDate="2018-02-06T08:36:59.880" Score="1" Body="&lt;p&gt;This is an episodic problem, and there should be no issue in theory with most learning algorithms coping without a discount factor (or setting gamma = 1).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any explanations to why the agent only works for small discount factors in this example?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The most likely explanations are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You have a mistake in your implementation or use of DQN.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You have an incorrect setting of a neural network hyperparameter. My initial thought would be you have mistakenly put a non-linearity like softmax or sigmoid on the NN outputs (it needs to be a linear output). Or it could just be that 10 hidden neurons is not enough for this representation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convergence requires far longer to train than you thought, once you introduce non-linear relationships between state and expected reward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is not surprising that the simpler puzzle trains more easily, as the agent can reduce it down to a linear counting puzzle without really needing to &quot;look&quot; at the representation. I would expect the Q values to be very poor approximations to the correct values after so few iterations, but the problem is simple enough that the incorrect values still produce an optimal behaviour.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-02-06T08:36:59.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5238" PostTypeId="2" ParentId="5230" CreationDate="2018-02-06T10:06:51.027" Score="3" Body="&lt;p&gt;If you want to do machine learning without coding (eg visual programming) you can try :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Orange : &lt;a href=&quot;https://orange.biolab.si/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://orange.biolab.si/&lt;/a&gt; . It also come when you download the Anaconda distribution.&lt;/li&gt;&#xA;&lt;li&gt;Also there is Dataiku Data Science Studio (the free version). &lt;a href=&quot;https://www.dataiku.com/dss/trynow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dataiku.com/dss/trynow/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But in general I would not recommend doing machine learning / data science with visual programming : you will probably lose more time learning the software that actually coding with some real library.&lt;/p&gt;&#xA;" OwnerUserId="8912" LastEditorUserId="8912" LastEditDate="2018-02-06T15:40:14.750" LastActivityDate="2018-02-06T15:40:14.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5239" PostTypeId="1" AcceptedAnswerId="5241" CreationDate="2018-02-06T14:19:27.290" Score="7" ViewCount="838" Body="&lt;p&gt;Whenever I read any book about neural networks or machine learning, their introductory chapter says that we haven't been able to replicate the brain's power due to its massive parallelism.&lt;/p&gt;&#xA;&lt;p&gt;Now, in modern times, transistors have been reduced to the size of nanometers, much smaller than the nerve cell. Also, we can easily build very large supercomputers.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Computers have much larger memories than brains.&lt;/li&gt;&#xA;&lt;li&gt;Computes can communicate faster than brains (clock pulse in nanoseconds).&lt;/li&gt;&#xA;&lt;li&gt;Computers can be of arbitrarily large size.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, my question is: why cannot we replicate the brain's parallelism if not its information processing ability (since the brain is still not well understood) even with such advanced technology? What exactly is the obstacle we are facing?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2021-01-19T12:38:25.657" LastActivityDate="2021-01-19T12:38:25.657" Title="What makes the animal brain so special?" Tags="&lt;neural-networks&gt;&lt;neurons&gt;&lt;neuroscience&gt;&lt;human-brain&gt;&lt;computational-theory-of-mind&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5241" PostTypeId="2" ParentId="5239" CreationDate="2018-02-06T16:38:59.800" Score="5" Body="&lt;p&gt;One probable hardware limiting factor is internal bandwidth.  A human brain has &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuron&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$10^{15}$&lt;/span&gt; synapses&lt;/a&gt;.  Even if each is only exchanging a few bits of information per second, that's on the order of &lt;span class=&quot;math-container&quot;&gt;$10^{15}$&lt;/span&gt; bytes/sec internal bandwidth.  A fast GPU (like those used to train neural networks) might approach &lt;span class=&quot;math-container&quot;&gt;$10^{11}$&lt;/span&gt; bytes/sec of internal bandwidth.  You could have 10,000 of these together to get something close to the total internal bandwidth of the human brain, but the interconnects between the nodes would be relatively slow, and would bottleneck the flow of information between different parts of the &amp;quot;brain.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Another limitation might be raw processing power.  A modern GPU has maybe 5,000 math units.  Each unit has a cycle time of ~1 ns, and might require ~1000 cycles to do the equivalent processing work one neuron does in ~1/10 second (this value is totally pulled from the air; we don't really know the most efficient way to match brain processing in silicon).  So, a single GPU might be able to match &lt;span class=&quot;math-container&quot;&gt;$5 \times 10^8$&lt;/span&gt; neurons in real-time.  You would optimally need 200 of them to match the processing power of the brain.&lt;/p&gt;&#xA;&lt;p&gt;This back-of-the-envelope calculation shows that internal bandwidth is probably a more severe constraint.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2444" LastEditDate="2021-01-19T12:23:11.013" LastActivityDate="2021-01-19T12:23:11.013" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5246" PostTypeId="1" AcceptedAnswerId="5247" CreationDate="2018-02-07T11:20:37.190" Score="20" ViewCount="8725" Body="&lt;p&gt;For instance, the title of &lt;a href=&quot;https://arxiv.org/abs/1611.01224&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; reads: &quot;Sample Efficient Actor-Critic with Experience Replay&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is &lt;em&gt;sample efficiency&lt;/em&gt;, and how can &lt;em&gt;importance sampling&lt;/em&gt; be used to achieve it?&lt;/p&gt;&#xA;" OwnerUserId="12574" LastEditorUserId="2444" LastEditDate="2020-10-13T08:24:36.250" LastActivityDate="2020-10-13T08:24:36.250" Title="What is sample efficiency, and how can importance sampling be used to achieve it?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;importance-sampling&gt;&lt;sample-efficiency&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5247" PostTypeId="2" ParentId="5246" CreationDate="2018-02-07T13:25:25.893" Score="20" Body="&lt;p&gt;An algorithm is &lt;strong&gt;sample efficient&lt;/strong&gt; if it can get the most out of every sample. Imagine learning trying to learn how to play PONG for the first time. As a human, it would take you within seconds to learn how to play the game based on very few samples. This makes you very &quot;sample efficient&quot;. Modern RL algorithms would have to see &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; thousand times more data than you so they are, relatively, sample inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of off-policy learning, not all samples are useful in that they are not part of the distribution that we are interested in. &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Importance_sampling&quot; rel=&quot;noreferrer&quot;&gt;Importance sampling&lt;/a&gt;&lt;/strong&gt; is a technique to filter these samples. Its original use was to understand one distribution while only being able to take samples from a different but related distribution. In RL, this often comes up when trying to learn off-policy. Namely, that your samples are produced by some behaviour policy but you want to learn a target policy. Thus one needs to measure how important/similar the samples generated are to samples that the target policy may have made. Thus, one is sampling from a weighted distribution which favours these &quot;important&quot; samples. There are many methods, however, for characterizing what is important, and their effectiveness may differ depending on the application. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common approach to this off-policy style of importance sampling is finding a ratio of how likely a sample is to be generated by the target policy. The paper &lt;a href=&quot;http://rll.berkeley.edu/~jietang/pubs/nips10_Tang.pdf&quot; rel=&quot;noreferrer&quot;&gt;On a Connection between Importance Sampling and&#xA;the Likelihood Ratio Policy Gradient&lt;/a&gt; (2010) by Tang and Abbeel covers this topic. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2019-08-31T23:39:10.880" LastActivityDate="2019-08-31T23:39:10.880" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5248" PostTypeId="2" ParentId="5234" CreationDate="2018-02-07T19:41:01.823" Score="15" Body="&lt;p&gt;Good question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First and foremost is that in Go deepmind had no superhuman opponents to challenge. Go engines were not anywhere near the highest level of the top human players. In chess, however, the engines are 500 ELO points stronger than the top human players. This is a massive difference. The amount of work that has gone into contemporary chess engines is staggering. We are talking about millions of hours in programming, hundreds of thousands of iterations. It is a massive body of knowledge and work. To overcome and surpass all of that in 4 hours is staggering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly it is not so much the result itself which is surprising to chess masters but instead its how AlphaZero plays chess. It's quite ironic that a system which had no human knowledge or expertise plays the most like we do. Engines are notorious for playing ugly looking moves, those lacking harmony etc. Its hard to explain to a non-chess player but there is such a thing as an &quot;Artificial move&quot; like the contemporary engines come up with often. AlphaZero does not play like this at all. It has a very human-like style where it dominates the opponent's pieces with deep strategic play and stunning position sacrifices. AlphaZero plays the way we aspire to, combining deep positional understanding with the precision of an engines calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&#xA;Oh and I forgot to mention something about the result itself. If you are not familiar with computer chess it may not seem staggering but it is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These days the margins of victory which separate the top contemporary engines are razor thin. In a 100 game match you could expect to see a result like 85 games drawn, 9 victories, and 6 losses to determine the better engine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaZero 28 wins and 72 draws with zero losses was otherworldly crushing and was completely unthinkable right up to the moment it happened.&lt;/p&gt;&#xA;" OwnerUserId="12585" LastEditorUserId="12585" LastEditDate="2018-02-07T20:03:56.343" LastActivityDate="2018-02-07T20:03:56.343" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5249" PostTypeId="2" ParentId="5230" CreationDate="2018-02-07T19:54:19.770" Score="2" Body="&lt;p&gt;Visual methods are limited and you will soon find the time invested in coding may have been better. There are various softwares like Weka &lt;a href=&quot;https://www.cs.waikato.ac.nz/ml/weka/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[website]&lt;/a&gt; or Rapid Miner (Not free) &lt;a href=&quot;https://rapidminer.com/us/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[website]&lt;/a&gt; or Orange &lt;a href=&quot;https://orange.biolab.si/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[website]&lt;/a&gt;. Nice thing is they provide GUI as well as coding. So you can import Python or R script in rapid miner for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would recommend that you should start using Python and with some online courses like from Coursera. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many frameworks and library that work with Python and one step to declare neural net layers like Keras &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[website]&lt;/a&gt;  or TFSlim with dataset reader/processing libraries like Numpy, pandas etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keras even have examples as well as models which you can simply import and use right away or use their architecture to train your dataset from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you can jump head down into tensorflow.&lt;/p&gt;&#xA;" OwnerUserId="3127" LastActivityDate="2018-02-07T19:54:19.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5250" PostTypeId="2" ParentId="5234" CreationDate="2018-02-07T20:16:36.300" Score="2" Body="&lt;p&gt;&lt;strong&gt;I see, based on the articles you provide, many levels of surprise in the victory:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chess is hard game to master and the counter part had the world's best practices, AlphaZero had tabula rasa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning took four hours and AlphaZero lost no match of 100. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Playing style was an alien mix of human and computer like moves, aggressive and some times seeming goofy with sacrifices that have no idea but are actually making future status more strong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Amount of possiblities taken in account per move was less than counter part, AlphaZero had a mysterious gut feeling or intuition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The upset feeling came from the amount of training material AlphaZero had built itself with and the time limit, that did not maybe give the traditional machine fair amount of time.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-02-07T20:16:36.300" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5252" PostTypeId="1" CreationDate="2018-02-07T23:01:24.403" Score="4" ViewCount="299" Body="&lt;p&gt;A lot of people seem to be under the impression that combining &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOFAI&lt;/a&gt; and contemporary AI will make models more general. I'm particularly interested in reasoning through analogy or case-based reasoning.&lt;/p&gt;&#xA;" OwnerUserId="9271" LastEditorUserId="1671" LastEditDate="2018-02-14T21:27:39.623" LastActivityDate="2018-02-14T21:27:39.623" Title="What are some interesting recent papers that synthesize symbolic AI with Deep Learning?" Tags="&lt;deep-learning&gt;&lt;symbolic-ai&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5254" PostTypeId="2" ParentId="5235" CreationDate="2018-02-08T12:41:11.487" Score="1" Body="&lt;p&gt;There are many problems with the help of AI we could move to a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example that I found incredible is in Brazil, where a group of friends decided to create an AI to monitor the transactions of the politicians. They called it the &lt;strong&gt;Serenata de Amor&lt;/strong&gt; and today they show to the whole world various scams and money being used for personal purposes and not how it should be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://serenatadeamor.org/en/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://serenatadeamor.org/en/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a big difference between thefts that replace human work and Artificial Intelligence. An artificial intelligence can rather replace human labor, but its capacity goes much further. Today an AI can discover new ways of working, finding more productive ways to accomplish tasks. Learning from mistakes should be a human task. But an AI takes it very seriously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, anything you think might be better, which could be more productive, which is limited by relying on humans, is a problem that can be solved by an AI.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="7800" LastEditDate="2018-02-08T12:51:44.313" LastActivityDate="2018-02-08T12:51:44.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5259" PostTypeId="1" CreationDate="2018-02-08T19:01:32.327" Score="1" ViewCount="75" Body="&lt;p&gt;Once the artificially intelligent machines are able to identify objects, we might want to teach them how to value different things differently based on their utility, demand, life, etc. How can we accomplish this and how did we start to value things?&lt;/p&gt;&#xA;" OwnerUserId="3015" LastEditorUserId="9947" LastEditDate="2018-02-09T15:41:41.487" LastActivityDate="2018-02-09T15:41:41.487" Title="Is understanding value for different features next step for object recognition?" Tags="&lt;machine-learning&gt;&lt;artificial-neuron&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5260" PostTypeId="2" ParentId="5259" CreationDate="2018-02-08T23:20:04.787" Score="0" Body="&lt;p&gt;You have asked two questions. How humans began to put a valuation to things and how to accomplish the task of valuation within an artificial intelligence construct. Human valuation is accomplished through trial and error experience, subjective choice and relative comparison, among other things. Valuation in an AI construct would be data-driven, objective and perhaps absolute. The choice of a valuation method would be determined by the choice of or desire of outcome. &lt;/p&gt;&#xA;" OwnerUserId="12619" LastActivityDate="2018-02-08T23:20:04.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5261" PostTypeId="2" ParentId="5234" CreationDate="2018-02-09T04:31:49.647" Score="4" Body="&lt;p&gt;MCTS for chess had been tried in the literature with little success. It was assumed AlphaGo's approach would &lt;strong&gt;never&lt;/strong&gt; work on chess, maybe in Go but not in chess. Suddenly, Google announced the approach was working and it was beating the World's strongest chess program by a very signficiant margin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before Google, all chess programmers were taught crafting heuristics in engine programming was a better strategy than machine learning. No matter how you implemented neural networks, it would have never ran faster than a bunch of 64-bit bitboards instructions. AlphaGo was running &lt;strong&gt;quite slow&lt;/strong&gt;, but it played strongest chess.  &lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2018-02-23T19:49:04.850" LastActivityDate="2018-02-23T19:49:04.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5262" PostTypeId="2" ParentId="5259" CreationDate="2018-02-09T10:35:09.687" Score="2" Body="&lt;p&gt;From your question I can assume you are a beginner in the field of AI. Welcome to this exciting field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, we have not yet been able to create a truly artificially intelligent program. They are all apparently intelligent but are just a set of simple/complex rules. An artificially intelligent agent must have at-least 2 aspects in the inside of its head/program. The ability to logically derive conclusions, and a capability for learning (these combined with ability to to take inputs and respond).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, logical reasoning part is the field of AI. Lots of simple codes performing complex task already exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question is based on the learning part, which is handled by Machine Learning programs. They learn iteratively. Object recognition is only one part of ML. They can also predict. Anyways, object recognition is done on the basis of maximizing reward/minimizing penalty (&lt;a href=&quot;https://towardsdatascience.com/machine-learning-fundamentals-via-linear-regression-41a5d11f5220&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cost function&lt;/a&gt;). Now, this minimizing of penalty is done by giving different weights or valuing different attributes of the objects differently. So this has already been accomplished. Depending upon the task at hand, we supply the attributes which are related to the final task ,otherwise the attribute is of no use. Like economics has no influence on weather.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also have this thought process by selecting things depending on the goal we need to accomplish, sometimes we select things consciously sometimes we are hardwired by genes to do so &lt;a href=&quot;https://www.health24.com/Lifestyle/Healthy-Nerves/How-does-the-body-respond-to-touching-something-hot-20141124&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Touching fire)&lt;/a&gt;. So you see, ML methods are modeled on how we do things and we do things based on weight-age to different factors. If we don't give proper weight-age we either learn by the penalty/punishment imposed upon us or our genes get wiped out from existence and thus the same behavior of not giving proper weight-age is not passed on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in short, a ML learning algorithm recognizes objects or does some final task (especially in games like chess) based on the utility or influence of the given features of current state on the final result.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-02-09T10:35:09.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5263" PostTypeId="2" ParentId="4956" CreationDate="2018-02-09T14:21:31.417" Score="1" Body="&lt;p&gt;Christopher Olah's &lt;a href=&quot;http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; describes it better that I ever could.&#xA;Basically, most data we come across can't be separated with a single line, but with some kind of curve. Non-linearities allow us to distort the input space in ways that make the data linearly separable, making classification more accuarate.&lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2018-02-09T14:21:31.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5264" PostTypeId="2" ParentId="5239" CreationDate="2018-02-09T19:55:52.993" Score="1" Body="&lt;p&gt;Short answer: nobody knows. Long answer: all strong-AI works. However, to write something useful to the o.p., say that the question contains several implicit statements, analyze them could be useful to clarify the issue:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) why thing that 1 transistor has the same functionality than 1 neuron ? Some obvious differences: a transistor has 3 legs, each neuron has around 7000 synapses; a transistor has 3 layers of material, a neuron is a full micro-machine with thousands of components; each synapses itself is a switch, connected to one or more other cells, and can produce different kinds of signals (activation/inhibitory, frequencies, amplitude, ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) compare memory amounts: the amount of memory in a person equivalent to the one in a computer s 0 bytes, we are not able to remember any thing forever and without distortion. Human memory is symbolic, temporal, associative, influenced by body and feelings, ... . Something totally different than computers one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;c) all previous are about &quot;hardware&quot;: if we analyze software and training, differences are even bigger. Even assume than intelligence is placed only in the brain, forgetting the role of hormonal system, senses, ... is a simplification not yet proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion: human mind is totally different from a computer, we are far to understand it, and more far to replicate it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the start of computers age, the idea than intelligence will appear when the amounts of memory, process power, ... reaches some threshold has became false.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-09T20:57:37.617" LastActivityDate="2018-02-09T20:57:37.617" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5265" PostTypeId="1" CreationDate="2018-02-10T01:54:04.027" Score="3" ViewCount="1853" Body="&lt;p&gt;I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer.&lt;/p&gt;&#xA;&lt;p&gt;What's the rationale behind that?&lt;/p&gt;&#xA;" OwnerUserId="10569" LastEditorUserId="2444" LastEditDate="2020-11-15T01:22:14.457" LastActivityDate="2020-11-15T01:22:14.457" Title="Why is no activation function used at the final layer of super-resolution models?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;deep-neural-networks&gt;&lt;activation-function&gt;&lt;image-super-resolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5266" PostTypeId="1" AcceptedAnswerId="5273" CreationDate="2018-02-10T03:28:13.393" Score="1" ViewCount="105" Body="&lt;p&gt;I'm wondering if these 2 specific programs already exist and if not how hard would it be to write them:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A program that would figure out (by only &quot;reading&quot; large amounts of texts in human language 1 and 2) which words in second language have the same meaning as a word in first language.  You would give for input texts in both languages and for output you would get for every word in first language a list of words in second language that are most similar to it with a probability that they mean the same thing. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A program that would figure out which words have the most similar meaning by analyzing large amounts of texts in one human language.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm planning on writing these two programs and it would be nice if I could get existing programs that do this so that I could compare results of my program to those of existing programs.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastEditorUserId="4302" LastEditDate="2018-10-08T12:16:29.977" LastActivityDate="2018-10-08T12:16:29.977" Title="Existing programs that find out words with same meanings" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5267" PostTypeId="2" ParentId="5266" CreationDate="2018-02-10T10:04:31.650" Score="1" Body="&lt;p&gt;Let us simplify case 1 of the question: assume two files, first one with numbers written in indo-arabic numeral system (i.e. 123, 9, 186754, ...), second one in roman numeral system (XX, LCVI, IX, ...). How do you match pairs of symbols with same meaning ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without external information or assumptions, you can not. You could made the hypothesis that probability of one specific number is the same in both examples, and base your pairing on it. But then you need to find two input files that fulfills this condition.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastActivityDate="2018-02-10T10:04:31.650" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5268" PostTypeId="2" ParentId="4949" CreationDate="2018-02-10T11:33:49.747" Score="1" Body="&lt;p&gt;Yes, rpa is AI. In particular, applied AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of rpa in wiki is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Robotic process automation (or RPA) is an emerging form of clerical&#xA;  process automation technology based on the notion of software robots&#xA;  or artificial intelligence (AI) workers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Moreover, look at this characteristics of rpa:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The paradigm, in summary, is that a software robot should be a virtual&#xA;  worker who can be rapidly &quot;trained&quot; (or configured) by a business user&#xA;  in an intuitive manner which is akin to how an operational user would&#xA;  train a human colleague.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If we talk about training or learning, we talk about AI.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastActivityDate="2018-02-10T11:33:49.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5271" PostTypeId="2" ParentId="5211" CreationDate="2018-02-10T16:08:14.443" Score="1" Body="&lt;p&gt;The most usual differences in signal records caused by different microphones will have small if not null impact in the recognition accuracy, in particular if we are talking about changes one mic by another of same model and manufactor:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Differences in bandwidth: voice is in a very common (central) bandwidth, it is not expected these differences impacts, even for low quality microphones.&lt;/li&gt;&#xA;&lt;li&gt;Microphone distortions: same as previous, they will not impact because they are smaller than, by example, a change in the speaker.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, if we talk about a general recognition system to be used with very different types of mics, there are some microphone issues that can cause your system completely fail:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;mic sensitivity: small sensitivity differences will have no effect because they are solved in the same way than differences in speaker volume/intonation. However, if the microphone is not enough sensible the S/N can be below the minimum need, in particular when speaker increase the distance to the mic.&lt;/li&gt;&#xA;&lt;li&gt;lack of beam-forming: if your system is prepared to use an array of microphones to filter noise and/or secondary sources, usage of a normal phone will decrease accuracy.&lt;/li&gt;&#xA;&lt;li&gt;changes in sample ratio and/or sample bits: if the microphone and its A/D has a low sampling speed or size (i.e. Bluetooth mics, phone lines, ...), the accuracy can fail.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;By example, for IOT applications, the first two of this list are the more challenging ones.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-10T18:48:34.000" LastActivityDate="2018-02-10T18:48:34.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5272" PostTypeId="2" ParentId="5265" CreationDate="2018-02-11T01:12:42.457" Score="-1" Body="&lt;p&gt;As discussed here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/post/What_should_be_my_activation_function_for_last_layer_of_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/post/What_should_be_my_activation_function_for_last_layer_of_neural_network&lt;/a&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linear is the preferred activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then linear activation function is equivalent to no activation function at all:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/13696/lack-of-activation-function-in-output-layer-at-regression&quot;&gt;https://datascience.stackexchange.com/questions/13696/lack-of-activation-function-in-output-layer-at-regression&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9592" LastActivityDate="2018-02-11T01:12:42.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5273" PostTypeId="2" ParentId="5266" CreationDate="2018-02-11T02:47:14.710" Score="1" Body="&lt;p&gt;For the first answer, the general way is using the seq2seq model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/tutorials/seq2seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/seq2seq&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a more specific example of the above category is this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the 2nd answer, one way is to use word2vec and apply supervised learning on the groups of similar words:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/word-to-vectors-natural-language-processing-b253dd0b0817&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/word-to-vectors-natural-language-processing-b253dd0b0817&lt;/a&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://spacy.io/usage/vectors-similarity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spacy.io/usage/vectors-similarity&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9592" LastActivityDate="2018-02-11T02:47:14.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5274" PostTypeId="1" CreationDate="2018-02-11T05:27:00.877" Score="25" ViewCount="4978" Body="&lt;p&gt;What are the current theories on the development of a conscious AI? Is anyone even trying to develop a conscious AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible that consciousness is an emergent phenomenon, that is, once we put enough complexity into our system, it will become self-aware? &lt;/p&gt;&#xA;" OwnerUserId="12648" LastEditorUserId="2444" LastEditDate="2019-11-11T20:57:44.957" LastActivityDate="2019-11-11T20:57:44.957" Title="What are the current theories on the development of a conscious AI?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;artificial-consciousness&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="5275" PostTypeId="2" ParentId="5274" CreationDate="2018-02-11T07:33:36.983" Score="28" Body="&lt;p&gt;To answer this question, first we need to know why developing conscious AI is hard. &#xA;The main reason is that there is no mathematically or otherwise rigorous definition of consciousness. &#xA;Sure you have an idea of consciousness as you experience it and we can talk about philosophical zombies but it isn’t a tangible concept that can be broken down and worked on. &#xA;Moreover, the majority of current research in AI is primarily a pragmatic approach in that one is trying to construct a model that can perform well according to some desired cost function. This is a very very big and exciting field and encompasses many research problems and every new finding is based either on mathematical theory or empirical evidence of a new algorithm/model construction/etc. &#xA;Because of this, progress is based on and compared against previous progress as it is the scientific method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question, no one is trying to actually make a “conscious” AI because we don’t know what that word means yet, however that doesn’t stop people talking about it.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-02-12T04:45:32.627" LastActivityDate="2018-02-12T04:45:32.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5278" PostTypeId="2" ParentId="5274" CreationDate="2018-02-11T14:25:28.963" Score="1" Body="&lt;p&gt;Consciousness is the ability to be aware of your own thoughts, your immediate environs, feelings and nothing more. It is the mechanism of our brain to control our lower kind of thoughts, the one based on associations and emotions. &#xA;Consciousness is observing our thoughts and feelings just like we observe real world with our eyes. It is not complicated.&#xA;The real question is not whether machines are capable of consciousness but whether they are capable of emotions. &lt;/p&gt;&#xA;" OwnerUserId="12251" LastEditorUserId="-1" LastEditDate="2019-03-07T22:38:16.073" LastActivityDate="2019-03-07T22:38:16.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5279" PostTypeId="1" AcceptedAnswerId="5283" CreationDate="2018-02-11T16:03:14.763" Score="2" ViewCount="65" Body="&lt;p&gt;I'm developing a multi-armed bandit which learns the best information to display to persuade someone to donate to charity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have treatments A, B, C, D (which are each one paragraph of text). The bandit selects one treatment to show to a person. The person is given $1 and has to decide how much (if any) to donate, in increments of one cent. The donation decision is recorded and fed to the multi-armed bandit, who will then re-optimize before another person is shown a treatment selected by the bandit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I program the bandit if my objective is to maximize total donations? For example, can I use Thompson sampling, and if a participant donates $0.80, I count that as 80 successes and 20 failures?&lt;/p&gt;&#xA;" OwnerUserId="12656" LastEditorUserId="2444" LastEditDate="2020-04-15T20:23:49.610" LastActivityDate="2020-04-15T20:23:49.610" Title="Programming a bandit to optimize donations" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5281" PostTypeId="2" ParentId="5274" CreationDate="2018-02-11T23:05:02.617" Score="7" Body="&lt;p&gt;&lt;strong&gt;What is consciousness?&lt;/strong&gt; There are some real challenges in setting up consciousness as a goal, because we don't have that much scientific understanding yet of how the brain does it or what balance there needs to be between long-term memory, short-term memory, the implicit work of interpretation, the contrasting conscious modes of automatic processing and deliberate processing (Khanemann's S1 and S2). John Kihlstrom (psychology emeritus at Berkeley) has a lecture set on Consciousness available in iTunesU that you might check out. Carnegie-Mellon Uni has a model called ACT-R which directly models conscious behaviours like attention-paying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What might bound our understanding of it?&lt;/strong&gt; Philosophy has been considering the question of consciousness for a long time. Personally I like Hegel and Heidegger (philosophers). Both are very difficult to read, but Heidegger (interpreted by Hubert Dreyfus) usefully critiqued the 'Good Old-Fashioned AI' projects of the seventies and pointed out how &lt;strong&gt;much&lt;/strong&gt; work there is just interpreting a visual input. Hegel is often maligned, but to see him well interpreted, check out Robert Brandom's talks to LMU on the logic of consciousness and Hegel as an early Sellers-ian pragmatist. If consciousness is to take hold of the truth and the certainty, it undertakes 'a path of doubt, or more properly a highway of despair', along which it never sets itself above correction. There is something about Hegel's treatment of consciousness in recursive terms, without succumbing to a vicious regress, that I think is going to be borne out before the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Recent developments.&lt;/strong&gt; The Deep Learning approaches and pragmatic successes of the present are exciting, but it will be interesting to see how far they can go in integrating and generalising from necessarily the small information sets actual human minds are exposed to.  While Deep Learning and data mining are hugely visible, symbolic approaches are also out there still getting better and more varied. But there is a lack of overarching theoretical interpretation that would allow generalisations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Two big-theory toe-holds.&lt;/strong&gt; If I had to pick a project I thought worth attending to, Giulio Tononi (et al) have set up a very nice modernisation of the problem in '&lt;a href=&quot;https://en.wikipedia.org/wiki/Integrated_information_theory&quot; rel=&quot;noreferrer&quot;&gt;Integrated Information Theory&lt;/a&gt;' But you might want to extend that with something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Rolf_Pfeifer&quot; rel=&quot;noreferrer&quot;&gt;Rolf Pfeifer&lt;/a&gt;'s 'How the body shapes the way we think', because some of the 'integrated information' is implicit in having arms and legs, eyes and nose (put there by the information accumulating work of evolution.) But there's so much good work that has been done - the pros are writing papers faster than I can read them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specific to your question, there are attempts to &lt;a href=&quot;https://www.humanbrainproject.eu/en/brain-simulation/&quot; rel=&quot;noreferrer&quot;&gt;simulate human brains&lt;/a&gt; hoping that overall aim will help fund research and produce answers to each para above. &lt;/p&gt;&#xA;" OwnerUserId="12665" LastEditorUserId="12665" LastEditDate="2018-02-12T02:15:26.360" LastActivityDate="2018-02-12T02:15:26.360" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5283" PostTypeId="2" ParentId="5279" CreationDate="2018-02-12T11:21:46.350" Score="3" Body="&lt;p&gt;It does not matter to the bandit algorithm that rewards are quantised or fractional, or that they can vary. This is true for pretty much all bandit optimisation algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So just treat the $0.80 donation as a real valued reward of 0.8, that occurs on a single timestep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Treating a single reward on a single timestep as if it were multiple rewards across multiple timesteps might cause problems, depending on which algorithm you were using, even if the reward averaged to the same value. For instance it may skew the maths away from theoretical assumptions used in upper confidence bound action selection or gradient-based solutions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It occurs to me that your $1 experiment is supposed to be some kind of proxy for the real deployment, where donations will be sparse and may vary more. Whilst this may help a little, essentially you are just setting some kind of prior based on results from a test. The real bandit algorithm starts once the system is deployed into production, and could well have very different results when the goal is to collect real and larger donations.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-02-12T11:28:25.123" LastActivityDate="2018-02-12T11:28:25.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5284" PostTypeId="2" ParentId="1678" CreationDate="2018-02-12T12:57:15.110" Score="2" Body="&lt;p&gt;In grammar, a predicate-argument relationship is one which is implied from text but not expressed in the syntactic structure. (Asher S)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Predicate logic or first order logic is a collection of formal systems used in mathematics, philosophy, linguistics and computer science. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NLP community is interested in recognizing, representing and classifying predicate arguments for shallow semantic parsing which can be viewed as the process of assigning a WHO did WHAT to WHOM, WHEN, WHY, HOW e.t.c. structure to plain text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This process entails identifying groups of words in a sentence that represent these semantic arguments and assigning specific labels to them. This could play a key role in NLP tasks like information extraction, question answering and summarization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Automatic and accurate techniques that can annotate naturally occurring text with semantic/predicate argument structure can facilitate the discovery of patterns of information in large text collections. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference I recommend this paper that uses Support Vector Machine algorithms for predicate argument classification. (Sameer P Kadri H 2005) &lt;a href=&quot;https://link.springer.com/article/10.1007/s10994-005-0912-2#enumeration&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/article/10.1007/s10994-005-0912-2#enumeration&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-02-12T12:57:15.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5285" PostTypeId="1" AcceptedAnswerId="5293" CreationDate="2018-02-12T13:05:53.910" Score="1" ViewCount="101" Body="&lt;p&gt;I'm reading &lt;em&gt;&quot;&lt;a href=&quot;http://www.fit.vutbr.cz/research/groups/speech/publi/2010/mikolov_interspeech2010_IS100722.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent neural network based language model&lt;/a&gt;&quot;&lt;/em&gt; of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding &lt;span class=&quot;math-container&quot;&gt;$w(t)$&lt;/span&gt; is obtained:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6i54I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6i54I.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I wonder is that in the classic &quot;&lt;em&gt;&lt;a href=&quot;http://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Neural Probabilistic Language Model&lt;/a&gt;&lt;/em&gt;&quot; Bengio et al. (2003) - they used separate embedding vector for representing each word and it was somehow &quot;semi-layer&quot;, meaning - it haven't contains non-linearity, but they did update word embeddings during the back-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Mikolov approach though, I assume they used simple one-hot vector, where each feature represent presence of each word. If we represent that's way single word input (like was in the Mikolov's paper) - that vector become all-zeros except single one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that correct?&lt;/p&gt;&#xA;" OwnerUserId="12691" LastEditorUserId="2444" LastEditDate="2019-04-16T22:35:10.617" LastActivityDate="2019-04-16T22:35:10.617" Title="How is the word embedding represented in the paper &quot;Recurrent neural network based language model&quot;?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;word-embedding&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5288" PostTypeId="1" CreationDate="2018-02-12T15:41:14.503" Score="3" ViewCount="41" Body="&lt;p&gt;Now, Boltzmann machines are energy-based undirected networks, meaning there are no forward computations. Instead, for each input configuration &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, a scalar energy is calculated to asses this configuration. The higher the energy, the less likely for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to be sampled from the target distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The probability distribution is defined through the energy function by summing over all possible states of the hidden part &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, the hidden units are added to capture higher-order interactions, which offers more capacity to the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how do we calculate the values of these hidden units? Or do we not explicitly compute these values and instead approximate the marginal &quot;free energy&quot; (which is the negative log of the sum over all possible states of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="12672" LastEditorUserId="2444" LastEditDate="2020-05-18T12:18:02.800" LastActivityDate="2020-05-18T12:18:02.800" Title="How do we calculate the hidden units values in a (restricted) Boltzmann machine?" Tags="&lt;neural-networks&gt;&lt;boltzmann-machine&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5289" PostTypeId="1" AcceptedAnswerId="5481" CreationDate="2018-02-12T18:52:04.187" Score="2" ViewCount="48" Body="&lt;p&gt;In order for the generalized bell membership function to retain its defined shape and domain, two restrictions must be placed on the b parameter: 1) b must be positive and 2) b must be an integer. Using backpropagation to tune the membership parameters (a,b and c in the bell), it appears to be possible that the correction to b will break one or both of these restrictions on b. Can someone please explain to me how we can use backpropagation to tune the b parameter (as well as a and c) without violating 1) and 2)?&lt;/p&gt;&#xA;" OwnerUserId="12544" LastActivityDate="2018-04-30T19:13:07.277" Title="Tuning the b parameter, ANFIS" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;fuzzy-logic&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5290" PostTypeId="1" CreationDate="2018-02-13T05:23:58.633" Score="2" ViewCount="148" Body="&lt;p&gt;I have purchasing history data for grocery shopping. I am trying to get abnormally frequently purchased items under certain conditions. For instance, I am trying to find frequently purchased items, when customers shop online and are willing to pay an extra shipping fee.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to find items that are particularly (or abnormally) frequently purchased under that situation (through online stores by paying shipping fee), how and what Machine Learning Algorithm  should I apply and identify those items?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;code&gt;arules&lt;/code&gt; R package which is using the association rules with purchasing history and tried to apply it. But it seems the package might be based on different principle from my idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone has an idea about my problem? If there is an R package related to the problem, it would be perfect.&lt;/p&gt;&#xA;" OwnerUserId="12713" LastEditorUserId="9947" LastEditDate="2018-02-14T18:39:01.800" LastActivityDate="2018-02-14T18:39:01.800" Title="Predict frequently purchased items under certain conditions with customer purchasing history data" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;learning-algorithms&gt;&lt;r&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5293" PostTypeId="2" ParentId="5285" CreationDate="2018-02-13T09:32:36.050" Score="1" Body="&lt;p&gt;Input vector contains two concatenated parts. The low part represents the current word:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;word in time t encoded using 1-of-N coding [...] - size of vector x is&#xA;  equal to size of vocabulary V (this can be in practice 30 000-200 000) plus [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where, as you said, 1-of-N means (see &lt;a href=&quot;https://www.quora.com/What-is-1-of-V-coding-with-relation-to-the-neural-network-language-model&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, 1-of-V):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you have a fixed-size vocabulary of symbols with V members in&#xA;  total, each input symbol can be coded as a vector of size V with all&#xA;  zeros except for the element corresponding to the symbol's order in&#xA;  the vocabulary, which gets a 1.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The high part of the input vector represents the current context:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;and previous context layer [...] Size of context (hidden) layer s is&#xA;  usually 30 - 500 hidden units.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For initialization, s(0) can be set to vector of small values, like&#xA;  0.1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the article includes this expression:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;x(t) = w(t) + s(t - 1)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;that I think better to write as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;x(t) = w(t) || s(t - 1)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;to made more visible the concatenation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, paper states some improvements that breaks the 1-to-N definition of the word (low) part, in order to reduce the size of the w vector:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we merge all words that occur less often than a threshold (in the&#xA;  training text) into a special rare token.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-13T10:45:15.643" LastActivityDate="2018-02-13T10:45:15.643" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5294" PostTypeId="2" ParentId="3817" CreationDate="2018-02-13T11:01:47.047" Score="1" Body="&lt;p&gt;Impossible to solve until you define an error measurement ( by example &lt;span class=&quot;math-container&quot;&gt;$|R-R'|$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$(R-R')^2$&lt;/span&gt; ) and how this error changes when A, B and C change.&lt;/p&gt;&#xA;&lt;p&gt;Extreme example: assume &lt;span class=&quot;math-container&quot;&gt;$R()$&lt;/span&gt; is random (unrelated to A, B, C values) but static (always same &lt;span class=&quot;math-container&quot;&gt;$R(A,B,C)$&lt;/span&gt; for same values of A,B,C). Given some values of A, B, C, you can only answer the value of &lt;span class=&quot;math-container&quot;&gt;$R(A,B,C)$&lt;/span&gt; when A,B,C was in the training set. &lt;span class=&quot;math-container&quot;&gt;$R(A,B,C)$&lt;/span&gt; is undefined and no predictable when A,B,C was not in the training set.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, improvements can be done if &lt;span class=&quot;math-container&quot;&gt;$R()$&lt;/span&gt; has some properties, by example, if it is possible to state that &lt;span class=&quot;math-container&quot;&gt;$R(A,B,C)=R(B,A,C)$&lt;/span&gt; or that &lt;span class=&quot;math-container&quot;&gt;$R(A1,B1,C1)=R(A2,B2,C2)$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$A1+B1+C1=A2+B2+C2$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-11-14T12:08:47.533" LastActivityDate="2020-11-14T12:08:47.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5295" PostTypeId="2" ParentId="5246" CreationDate="2018-02-13T11:42:50.290" Score="6" Body="&lt;p&gt;Sample Efficiency denotes the amount of experience that an agent/algorithm needs to generate in an environment (e.g. the number of actions it takes and number of resulting states + rewards it observes) during training in order to reach a certain level of performance. Intuitively, you could say an algorithm is sample efficient if it can make good use of every single piece of experience it happens to generate and rapidly improve its policy. An algorithm has poor sample efficiency if it fails to learn anything useful from many samples of experience and doesn't improve rapidly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explanation of importance sampling in Jaden's answer seems mostly correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper in your question, importance sampling is one of the ingredients that enables a correct combination of 1) learning from multi-step trajectories, and 2) experience replay buffers. Those two things were not easy to combine before (because multi-step returns without importance sampling are only correct in on-policy learning, and old samples in a replay buffer were generated by an old policy which means that learning from them is off-policy). Both of those things individually improve sample efficiency though, which implies that it's also beneficial for sample efficiency if they can still be combined somehow.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-02-13T11:42:50.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5296" PostTypeId="2" ParentId="5290" CreationDate="2018-02-13T16:01:40.253" Score="1" Body="&lt;p&gt;Let start by the concrete question, and follow talking about the general problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concrete question &lt;em&gt;&quot;find items that are particularly frequently purchased through online stores by paying shipping fee&quot;&lt;/em&gt; needs few or none usage of applied AI, just a few of statistics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question talks about &quot;item purchased&quot; and &quot;buy method&quot;, thus, we have an information database with entries like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sale(online,item1).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;sale(shop,item2). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;sale(online,item2). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;sale(online,item2). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(note records can be repeated)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the percentage of online sales of item &quot;X&quot; is defined as the fraction of sales online of this item, sale(online,X), over the total sales of this item, sale(_,X):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QhUDO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QhUDO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the previous data examples, p(item1)=1/1, p(item2)=2/3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;High p(X) means items that are preferred for online shopping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other probabilities can be defined for similar cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the general case, we are talking about data mining. The are very good packages (open source, ...) for them: weka, IBM DWE, ... . By example, using Weka J48 over a database defined as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sale( purchase_identifier, buy method, item )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where &quot;purchase_identifier&quot; must group item that has been purchased in a single buyout (cash ticket). J48 will then provide as answer rules as: &lt;em&gt;item &quot;foo&quot; is usually purchased in online shop when also item &quot;bar&quot; is purchased.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-13T17:26:56.940" LastActivityDate="2018-02-13T17:26:56.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5297" PostTypeId="2" ParentId="3923" CreationDate="2018-02-13T21:55:23.507" Score="1" Body="&lt;p&gt;It sounds like you're looking at the Partition Problem. &lt;a href=&quot;https://en.wikipedia.org/wiki/Partition_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Partition_problem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The task of slicing one set into N sets so that each set is equal or as close to equal as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obtaining an exact solution is NP-hard (you can't do much better than trying all combinations), however you can get an approximate answer in polynomial time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Greedy approach:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create N sets, each initially empty&lt;/li&gt;&#xA;&lt;li&gt;Sort all items in descending order (by their complexity score)&lt;/li&gt;&#xA;&lt;li&gt;for each item add it to the set with the lowest sum&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If N is large, you may want to put the sets into a min heap/priority queue.&lt;/p&gt;&#xA;" OwnerUserId="12278" LastEditorUserId="12278" LastEditDate="2018-02-13T22:36:35.810" LastActivityDate="2018-02-13T22:36:35.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5299" PostTypeId="2" ParentId="2412" CreationDate="2018-02-14T07:35:08.080" Score="1" Body="&lt;p&gt;Firstly, before we commence I will recommend that you refer to a similar questions on the network &lt;a href=&quot;https://stackoverflow.com/questions/828486/neural-net-optimize-w-genetic-algorithm&quot;&gt;https://stackoverflow.com/questions/828486/neural-net-optimize-w-genetic-algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The majority of ML studies focus on gradient algorithms, usually a variation of back-propagation to obtain the weights of the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However since genetic algorithms are powerful searching algorithms they can be utilized to tune and optimize the structure and parameters of neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic algorithms are stochastic search techniques that guide a population of solutions towards an optimum using the principles of evolution and natural genetics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic algorithms are especially capable of handling problems in which the objective function is discontinuous/non differentiable, nonconvex, multimodal or noisy. (Dharmistha D)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms, which combine genetic algorithms and error back-propagation, have been shown to exhibit better convergence properties than pure back-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To facilitate the crossover operation for the exchange of information between two parents we need to define four components within a genetic algorithm. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A way of coding solutions to the problem on chromosomes;&lt;/li&gt;&#xA;&lt;li&gt;A fitness function which returns a value for each chromosome given to it;&lt;/li&gt;&#xA;&lt;li&gt;A way of initializing the population of chromosomes;&lt;/li&gt;&#xA;&lt;li&gt;Operators that may be applied to parents  to alter their genetic composition i.e. mutation. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In these hybrid systems weights in different layers of the network are optimized using a genetic algorithm. Experimental results demonstrate that the genetic algorithms can in some cases optimize the hyper-parameters of an ANN better than a hand tuned model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding an efficient way to implement a random crossover, here is something to get you started&#xA;(Code courtesy of Dan Golding 2013)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Randomly choose 2 individuals&#xA;&#xA;n = size(new_pop, 1);&#xA;l = size(new_pop, 2);&#xA;&#xA;breeders = new_pop(randperm(n,2),:);&#xA;&#xA;%Choose a crossover point&#xA;&#xA;cp = randperm(l, 1);&#xA;&#xA;%Crossover&#xA;&#xA;b1 = [breeders(1, 1:cp), breeders(2, cp+1:end)];&#xA;b2 = [breeders(2, 1:cp), breeders(1, cp+1:end)];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For further reference I recommend that you kindly look at the below links&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab crossover genetic algorithm&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/17696323/matlab-crossover-genetic-algorithm&quot;&gt;https://stackoverflow.com/questions/17696323/matlab-crossover-genetic-algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Single point ordered crossover in matlab&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/16302382/single-point-ordered-crossover-in-matlab&quot;&gt;https://stackoverflow.com/questions/16302382/single-point-ordered-crossover-in-matlab&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To optimize a neural network of multiple inputs using a genetic algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/matlabcentral/answers/180513-to-optimize-a-neural-network-of-multiple-inputs-using-a-genetic-algorithm?s_tid=gn_loc_drop&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/matlabcentral/answers/180513-to-optimize-a-neural-network-of-multiple-inputs-using-a-genetic-algorithm?s_tid=gn_loc_drop&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use the Genetic Algorithm (GA) to train a Neural Network in Neural Network Toolbox&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/matlabcentral/answers/100323-how-can-i-use-the-genetic-algorithm-ga-to-train-a-neural-network-in-neural-network-toolbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/matlabcentral/answers/100323-how-can-i-use-the-genetic-algorithm-ga-to-train-a-neural-network-in-neural-network-toolbox&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-02-14T07:35:08.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5301" PostTypeId="2" ParentId="2381" CreationDate="2018-02-14T13:41:48.390" Score="5" Body="&lt;h2&gt;What is OpenCog?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://opencog.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt; is a project with the vision of creating a thinking machine with human-level intelligence and beyond. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In OpenCog's introduction, Goertzel categorically states that the OpenCog project is not concerned with building more accurate classification algorithms, computer vision systems or better language processing systems. The OpenCog project is solely focused on general intelligence that is capable of being extended to more and more general tasks. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Knowledge representation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;OpenCog's knowledge representation mechanisms are all based fundamentally on networks. OpenCog has the following knowledge representation components:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;AtomSpace&lt;/strong&gt;: it is a knowledge representation database and query engine. Data on AtomSpace is represented in the form of graphs and hypergraphs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Probabilistic Logic Networks&lt;/strong&gt; (PLN's): it is a novel conceptual, mathematical and computational approach to handle uncertainty and carry out effective reasoning in real-world circumstances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MOSES&lt;/strong&gt; (Meta-Optimizing Semantic Evolutionary Search): it implements program learning by using a meta-optimization algorithm. That is, it uses two optimization algorithms, one wrapped inside the other to find solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Economic Attention Allocation&lt;/strong&gt; (EAA): each atom has an attention value attached to it. The attention values are updated by using nonlinear dynamic equations to calculate the Short Term Importance (STI) and Long Term Importance (LTI). &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Competency Goals&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;OpenCog lists 14 competencies that they believe AI systems should display in order to be considered an AGI system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Perception&lt;/strong&gt;: vision, hearing, touch and cross-modal proprioception&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Actuation&lt;/strong&gt;: physical skills, tool use, and navigation physical skills&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Memory&lt;/strong&gt;: declarative, behavioral and episodic&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning&lt;/strong&gt;: imitation, reinforcement, interactive verbal instruction, written media and learning via experimentation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reasoning&lt;/strong&gt;: deduction, induction, abduction, causal reasoning, physical reasoning and associational reasoning &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Planning&lt;/strong&gt;: tactical, strategic, physical and social&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Attention&lt;/strong&gt;: visual attention, behavioural attention, social attention&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Motivation&lt;/strong&gt;: subgoal creation, affect-based motivation, control of emotions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Emotion&lt;/strong&gt;: expressing emotion, understanding emotion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Modelling self and other&lt;/strong&gt;: self-awareness, theory of mind, self-control&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Social interaction&lt;/strong&gt;: appropriate social behavior, social communication, social inference and group play&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Communication&lt;/strong&gt;: gestural communication, verbal communication, pictorial communication, language acquisition and cross-modal communication&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Quantitative skills&lt;/strong&gt;: counting, arithmetic, comparison and measurement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ability to build/create&lt;/strong&gt;: physical, conceptual, verbal and social.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Do I endorse OpenCog?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In my opinion, OpenCog introduces and covers important algorithms/approaches in machine learning, i.e. hyper-graphs and probabilistic logic networks. However, my criticism is that they fail to commit to a single architecture and integrate numerous architectures in an irregular and unsystematic manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, Goertzel failed to recognize the fundamental shift that came with the introduction of deep learning architectures so as to revise his work accordingly. This puts his research out of touch with recent developments in machine learning&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="9608" LastEditDate="2019-08-06T20:15:43.323" LastActivityDate="2019-08-06T20:15:43.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5302" PostTypeId="2" ParentId="2675" CreationDate="2018-02-14T15:32:50.163" Score="-1" Body="&lt;p&gt;The 7 AI problem characteristics is a heuristic technique designed to speed up the process of finding a satisfactory solution to problems in artificial intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In computer science, artificial intelligence and mathematical optimization, a heuristic is a technique designed for solving a problem more quickly, or for finding an approximate solution when you have failed to find an exact solution using classic methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 7 AI problem technique ranks alternative steps based on available information to help one decide on the most appropriate approach to follow in solving problems i.e. missionaries and cannibals, Tower of Hanoi, Traveling salesman e.t.c. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding whether there is a generally accepted relationship between the placement of a problem and suitable algorithms. The answer is that indeed there is a generally accepted relationship. &#xA;For example imagine trying to solve a game of chess and a game of sudoku.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a step is wrong in sudoku, we can backtrack and attempt a different approach. However if we are playing a game of chess and realize a mistake after a couple of moves. We cannot simply ignore the mistake and backtrack.(2nd Characteristic) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem universe is predictable,  we can make a plan to generate a sequence of operations that is guaranteed to lead to a solution. However in the case of problems with uncertain outcomes, we have to follow a process of plan revision as the plan is carried out while providing the necessary feedback. (3rd Characteristic)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is an example of the 7 AI problem characteristics being applied to solve a water jug problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uUp1l.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uUp1l.jpg&quot; alt=&quot;Below is an example of the 7 AI problem characteristic being used to solve a water jug problem.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image source&#xA;&lt;a href=&quot;https://gtuengineeringmaterial.blogspot.com/2013/05/discuss-ai-problems-with-seven-problem_1818.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gtuengineeringmaterial.blogspot.com/2013/05/discuss-ai-problems-with-seven-problem_1818.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-02-14T15:32:50.163" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5303" PostTypeId="2" ParentId="3989" CreationDate="2018-02-14T17:58:28.577" Score="1" Body="&lt;p&gt;Firstly, before we commence I will recommend that you refer to similar questions on the network i.e &lt;a href=&quot;https://stackoverflow.com/questions/39386936/machine-learning-with-incomplete-data&quot;&gt;https://stackoverflow.com/questions/39386936/machine-learning-with-incomplete-data&lt;/a&gt; , &lt;a href=&quot;https://stats.stackexchange.com/questions/103500/machine-learning-algorithms-to-handle-missing-data&quot;&gt;https://stats.stackexchange.com/questions/103500/machine-learning-algorithms-to-handle-missing-data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Row Deletion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a particular row has more than 70% missing values, you can delete the row to handle the null values. This method is advised only when there are enough samples in the data set. The major disadvantage of this method is that it reduces the power of the model because it reduces the sample size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Replacing With Mean/Median/Mode&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can calculate the mean, median or mode of the feature and replace the missing values with it. Another approach is to approximate it with the deviation of neighbouring values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this approach adds variance to the data set, it yields better results compared to removing rows and columns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;KNN or Random Forest imputation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this approach, the missing values of an attribute are imputed using existing attributes that are most similar to the attribute whose values are missing. The similarity of two attributes is determined using a distance function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this approach is that k-nearest neighbour can predict both qualitative and quantitative attributes. Additionally you do not need to create a prediction model for each attribute with missing data in the dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Predicting the Missing Values&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prediction is one of the more sophisticated methods for handling missing data.&#xA;Using the features which do not have missing values, we can predict the null values with the help of a machine learning algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, we divide our data set into two. One set with no missing values and another set with missing values. The first data set becomes the training data set of the model while the second data set with missing values is the test data set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then create a model to predict target variables based on other attributes of the training data set and populate the missing values of the test data set.(Sayali S 2016) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Caret or randomForestSRC packages in R&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The R package randomForestSRC can handle missing data for a wide class of analyses i.e. regression, classification, unsupervised and multivariate (Ankur C 2014). &#xA;Additionally, the Caret R package can be used to predict missing data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference : &lt;a href=&quot;https://analyticsindiamag.com/5-ways-handle-missing-values-machine-learning-datasets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://analyticsindiamag.com/5-ways-handle-missing-values-machine-learning-datasets/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-02-14T17:58:28.577" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5304" PostTypeId="5" CreationDate="2018-02-14T19:19:09.157" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.r-project.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;R (wikipedia)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.r-project.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;R (r-project.org)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/Why-do-people-often-use-R-language-programming-for-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why use R for AI? (Quora)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-02T17:06:58.667" LastActivityDate="2018-04-02T17:06:58.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5305" PostTypeId="4" CreationDate="2018-02-14T19:19:09.157" Score="0" Body="For questions about implementation of Machine Learning and Artificial Intelligence algorithms in the R programming language." OwnerUserId="1671" LastEditorUserId="9947" LastEditDate="2018-04-02T17:00:34.740" LastActivityDate="2018-04-02T17:00:34.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5306" PostTypeId="5" CreationDate="2018-02-14T19:24:07.577" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Node.js&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Node.js&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://nodejs.org/en/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nodejs.org/en/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-14T19:24:07.577" LastActivityDate="2018-02-14T19:24:07.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5307" PostTypeId="4" CreationDate="2018-02-14T19:24:07.577" Score="0" Body="Use for questions about developing AI with Node.js" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-14T19:24:07.577" LastActivityDate="2018-02-14T19:24:07.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5308" PostTypeId="1" CreationDate="2018-02-14T19:43:20.667" Score="6" ViewCount="146" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Greedy_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Greedy algorithms&lt;/a&gt; are well known, and although useful in a local context for certain problems, and even potentially find general, global optimal solutions, they nonetheless trade optimality for shorter-term payoffs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to me a good analogue for human greed, although there is also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Grey_goo&quot; rel=&quot;noreferrer&quot;&gt;grey goo&lt;/a&gt; type of greed that is senseless acquisition of material (think plutocrats who talk about wealth as merely a way of &quot;keeping score&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Technical_debt&quot; rel=&quot;noreferrer&quot;&gt;Technical debt&lt;/a&gt; is an extension of development practices that fall under the algorithmic definition of greed (short-term payoff leads to trouble down the road.)  This may be further extended to any non-optimized code in terms of energy waste (flipping of unnecessary bits) which will only increase as everything becomes more computerized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are other vices that can arise in algorithms?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-01-02T17:46:21.257" Title="Algorithms can be greedy. What are some other algorithmic vices?" Tags="&lt;philosophy&gt;&lt;ethics&gt;&lt;theory&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5309" PostTypeId="2" ParentId="5308" CreationDate="2018-02-14T20:35:38.743" Score="5" Body="&lt;p&gt;Algorithms can be racist, sexist, and otherwise bigoted. When we feed them data produced by systems that are biased against groups of people, the algorithm will learn to behave that way. We're used to garbage in garbage out, now we have to worry about racism in racism out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See:&lt;br&gt;&#xA;&lt;a href=&quot;https://www.nytimes.com/2018/02/09/technology/facial-recognition-race-artificial-intelligence.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facial Recognition Is Accurate, if You’re a White Guy&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://www.theguardian.com/inequality/2017/aug/08/rise-of-the-racist-robots-how-ai-is-learning-all-our-worst-impulses&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rise of the Racist Robots – how AI is Learning all our Worst Impulses&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="12732" LastEditorUserId="12732" LastEditDate="2018-02-14T20:44:24.207" LastActivityDate="2018-02-14T20:44:24.207" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5310" PostTypeId="5" CreationDate="2018-02-14T20:44:23.530" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Swarm_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Swarm_intelligence&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-14T20:44:23.530" LastActivityDate="2018-02-14T20:44:23.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5311" PostTypeId="4" CreationDate="2018-02-14T20:44:23.530" Score="0" Body="Use for questions about  collective behavior of decentralized, self-organized algorithmic systems, artificial or natural. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-14T20:44:23.530" LastActivityDate="2018-02-14T20:44:23.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5312" PostTypeId="5" CreationDate="2018-02-14T21:33:22.617" Score="0" Body="&lt;p&gt;For more info, see, for example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-03T16:19:42.487" LastActivityDate="2019-11-03T16:19:42.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5313" PostTypeId="4" CreationDate="2018-02-14T21:33:22.617" Score="0" Body="For questions related to good old-fashioned artificial intelligence (GOFAI), also known as symbolic artificial intelligence (AI), which is the collection of all methods in AI research that are based on high-level &quot;symbolic&quot; (human-readable) representations of problems, logic, and search. John Haugeland gave the name GOFAI to symbolic AI in his 1985 book &quot;Artificial Intelligence: The Very Idea&quot;." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-03T16:19:42.487" LastActivityDate="2019-11-03T16:19:42.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5314" PostTypeId="1" CreationDate="2018-02-14T21:42:22.173" Score="1" ViewCount="2767" Body="&lt;p&gt;I'm interested in working on challenging AI problems, and after reading this article (&lt;a href=&quot;https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/&lt;/a&gt;) by DeepMind and Blizzard, I think that developing a robust AI capable of learning to play Starcraft 2 with superhuman level of performance (without prior knowledge or human hard-coded heuristics) would imply a huge breakthrough in AI research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure I know this is an extremely challenging problem, and by no means I pretend to be the one solving it, but I think it's a challenge worth taking on nonetheless because the complexity of the decision making required is much closer to the real world and so this forces you to come up with much more robust, generalizable AI algorithms that could potentially be applied to other domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, an AI that plays Starcraft 2 would have to be able to watch the screen, identify objects, positions, identify units moving and their trajectories, update its current knowledge of the world, make predictions, make decisions, have short term and long term goals, listen to sounds (because the game includes sounds), understand natural language (to read and understand text descriptions appearing in the screen as well), it should probably be endowed also with some sort of attention mechanism to be able to pay attention to certain regions of interest of the screen, etc. So it becomes obvious that at least one would need to know about Computer Vision, Object Recognition, Knowledge Bases, Short Term / Long Term Planning, Audio Recognition, Natural Language Processing, Visual Attention Models, etc. And obviously it would not be enough to just study each area independently, it would also be necessary to come up with ways to integrate everything into a single system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, does anybody know good resources with content relevant to this problem? I would appreciate any suggestions of papers, books, blogs, whatever useful resource out there (ideally state-of-the-art) which would be helpful for somebody interested in this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="12746" LastEditorUserId="12746" LastEditDate="2018-02-15T13:05:46.507" LastActivityDate="2018-02-20T07:37:32.120" Title="Training an AI to play Starcraft 2 with superhuman level of performance?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;computer-vision&gt;&lt;knowledge-representation&gt;&lt;reasoning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5316" PostTypeId="2" ParentId="4048" CreationDate="2018-02-15T03:46:45.293" Score="3" Body="&lt;p&gt;I think you raise a good question, especially WRT to how the NNs inputs &amp;amp; outputs are mapped onto the mechanics of a card game like MtG where the available actions vary greatly with context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have a really satisfying answer to offer, but I have played Keldon's Race for the Galaxy NN-based AI - agree that it's excellent- and have looked into how it tackled this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latest code for Keldon's AI is now searchable and browseable on &lt;a href=&quot;https://github.com/bnordli/rftg&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ai code is in one &lt;a href=&quot;https://github.com/bnordli/rftg/blob/master/src/ai.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;. It uses 2 distinct NNs, one for &quot;evaluating hand and active cards&quot; and the other for &quot;predicting role choices&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you'll notice is that it uses a fair amount on non-NN code to model the game mechanics. Very much a hybrid solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mapping of game state into the evaluation NN is done &lt;a href=&quot;https://github.com/bnordli/rftg/blob/master/src/ai.c#L1942&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Various relevant features are one-hot-encoded, eg the number of goods that can be sold that turn.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another excellent case study in mapping a complex game into a NN is the Starcraft II Learning Environment created by Deepmind in collaboration with Blizzard Entertainment. This &lt;a href=&quot;https://deepmind.com/documents/110/sc2le.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; gives an overview of how a game of Starcraft is mapped onto a set of features that a NN can interpret, and how actions can be issued by a NN agent to the game simulation.&lt;/p&gt;&#xA;" OwnerUserId="12751" LastEditorUserId="12751" LastEditDate="2018-02-15T04:20:53.057" LastActivityDate="2018-02-15T04:20:53.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5317" PostTypeId="1" CreationDate="2018-02-15T03:54:17.247" Score="2" ViewCount="987" Body="&lt;p&gt;I often develop bots and I need to understand what some people are saying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples:&lt;br&gt;&#xA;- I want an apple&lt;br&gt;&#xA;- I want an a p p l e&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I find the object (&lt;em&gt;apple&lt;/em&gt;)? I honestly don't know where to start looking. Is there an API that I can send the text to which returns the object? Or perhaps I should manually code something that analyses the grammar?&lt;/p&gt;&#xA;" OwnerUserId="12752" LastEditorUserId="2193" LastEditDate="2018-12-06T21:18:23.517" LastActivityDate="2018-12-06T21:18:23.517" Title="How to find the subject in a text?" Tags="&lt;algorithm&gt;&lt;natural-language-processing&gt;&lt;ai-basics&gt;&lt;getting-started&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-18T04:32:44.250" ContentLicense="CC BY-SA 4.0" />
  <row Id="5318" PostTypeId="1" CreationDate="2018-02-15T05:55:24.767" Score="3" ViewCount="1034" Body="&lt;p&gt;I have been trying to use CNN for a regression problem. I followed the standard recommendation of disabling dropout and overfitting a small training set prior to trying for generalization. With a 10 layer deep architecture, I could overfit a training set of about 3000 examples. However, on adding 50% dropout after the fully-connected layer just before the output layer, I find that my model can no longer overfit the training set. Validation loss also stopped decreasing after a few epochs. This is a substantially small training set, so overfitting should not have been a problem, even with dropout. So, does this indicate that my network is not complex enough to generalize in the presence of dropout? Adding additional convolutional layers didn't help either. What are the things to try in this situation? I will be thankful if someone can give me a clue or suggestion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: For reference, I am using the learned weights of the first 16 layers of Alexnet and have added 3 convolutional layers with ReLU non-linearity followed by a max pooling layer and 2 fully connected layers. I update weights of all layers during training using SGD with momentum. &lt;/p&gt;&#xA;" OwnerUserId="12754" LastActivityDate="2021-02-22T02:59:14.393" Title="What to do if CNN cannot overfit a training set on adding dropout?" Tags="&lt;convolutional-neural-networks&gt;&lt;linear-regression&gt;&lt;dropout&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5320" PostTypeId="1" AcceptedAnswerId="5323" CreationDate="2018-02-15T11:38:38.140" Score="3" ViewCount="261" Body="&lt;p&gt;This is a kind of biological and philosophical question. So, the recent concern in AI is that an AI agent may go rogue with prominent people voicing their concerns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now say, we have created an AI (you are free to use your own definition of what makes an AI intelligent) which has gone rogue with powers given in this &lt;a href=&quot;https://ai.stackexchange.com/questions/2274/what-would-be-the-best-way-to-disable-a-rogue-ai&quot;&gt;question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the broad view of today's biology is that everything we do is to further our genes down the future (leaving aside small technical details). It is even widely accepted that we are just machines whose controller are the genes. Everything we do is controlled/hardwired by the genes with some avenue of learning from experiences. Also genes only further their own interest. Scientist &lt;a href=&quot;https://en.wikipedia.org/wiki/George_R._Price&quot; rel=&quot;nofollow noreferrer&quot;&gt;George Price&lt;/a&gt; even wrote a mathematical equation proving all our acts are selfish and only furthering the interest of our genes (&lt;a href=&quot;https://motherboard.vice.com/en_us/article/bmjanm/george-price-altruism&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;). Also &lt;a href=&quot;https://en.wikipedia.org/wiki/Richard_Dawkins&quot; rel=&quot;nofollow noreferrer&quot;&gt;Richard Dawkins&lt;/a&gt; is a pioneer of this idea (this is only to show I haven't pulled the idea out from air).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my question is that what will possibly be the motivation of an AI agent to go rogue? It doesn't have genes whose interest it needs to further. We all do something for an end result. What is the end result a rogue AI might try to achieve/attain and why?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-02-26T15:29:38.990" Title="Motivation that drives a rogue AI agent" Tags="&lt;philosophy&gt;&lt;biology&gt;&lt;genes&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5322" PostTypeId="1" CreationDate="2018-02-15T14:16:52.027" Score="8" ViewCount="1692" Body="&lt;p&gt;I was reading Gary Marcus's a &lt;a href=&quot;https://arxiv.org/abs/1801.00631&quot; rel=&quot;nofollow noreferrer&quot;&gt;Critical Appraisal of Deep Learning&lt;/a&gt;. One of his criticisms is that neural networks don't incorporate prior knowledge in tackling a problem. My question is: have there been any attempts at encoding prior knowledge in deep neural networks?&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="2444" LastEditDate="2020-12-03T14:46:50.570" LastActivityDate="2020-12-03T15:37:27.127" Title="Can prior knowledge be encoded in deep neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;algorithmic-bias&gt;&lt;inductive-bias&gt;" AnswerCount="5" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5323" PostTypeId="2" ParentId="5320" CreationDate="2018-02-15T15:14:51.017" Score="2" Body="&lt;p&gt;Today, prominent machine learning techniques involve trying to minimize some cost function. In many simple cases this cost function is easy to specify, for instance, linear regression is simply trying to minimize the distance between input data and a line of best fit. No matter what the cost function, the agent is trying to minimize it (or maximize a reward function). That is its motivation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as problems become harder it becomes more challenging for humans to design a cost/reward function such that an system/agent is actually trying to do what the humans want it to do. For instance, one might want a cup of coffee and rewards the agent for getting it to them very quickly. In this case, the agent might make the coffee and then throw it at the human which isn’t what the human actually wanted. Something was misspecified (eg. don’t throw or spill it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problems like these could result in a rouge AI and its sole motivation would be to minimize its cost function. For instance, this coffee-AI may think that it would never screw up getting coffee (thus get bad reward) if there were no humans to ask for one. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-02-19T03:21:55.663" LastActivityDate="2018-02-19T03:21:55.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5325" PostTypeId="1" CreationDate="2018-02-15T16:31:48.240" Score="2" ViewCount="236" Body="&lt;p&gt;I have completed week 1 of Andrew Ng's course. I understand that the cost function for linear regression is defined as &lt;span class=&quot;math-container&quot;&gt;$J (\theta_0, \theta_1) = 1/2m*\sum (h(x)-y)^2$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$h(x) = \theta_0 + \theta_1(x)$&lt;/span&gt;. But I don't understand what &lt;span class=&quot;math-container&quot;&gt;$\theta_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_1$&lt;/span&gt; represent in the equation. Is someone able to explain this?&lt;/p&gt;&#xA;" OwnerUserId="12762" LastEditorUserId="2444" LastEditDate="2019-03-02T11:31:51.940" LastActivityDate="2020-07-18T23:27:01.100" Title="Understanding a few terms in Andrew Ng's definition of the cost function for linear regression" Tags="&lt;linear-regression&gt;&lt;math&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5326" PostTypeId="2" ParentId="4213" CreationDate="2018-02-15T16:51:17.557" Score="1" Body="&lt;p&gt;I highly recommend that you start reading on the Netflix challenge. It has tonnes of useful and interesting examples dealing with this sort of thing.&lt;/p&gt;&#xA;&lt;p&gt;You will need an Algorithm that builds a score on both 'quality' and 'quantity'. That is, it needs to add a 'weight' to the final rating based on the number of reviews that an individual has. This is so that, for example, an individual with 50 8 score reviews would be rated higher than an individual with only one 9 score review.&lt;/p&gt;&#xA;&lt;p&gt;I recommend that you implement Bayesian estimates to calculate weighted voting.&lt;/p&gt;&#xA;&lt;p&gt;IMDb (Internet Movie Database) utilizes this algorithmn to determine its IMDB top 250 movies. (Robert C 2010)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The formula for calculating the Top Rated 250 Titles gives a true Bayesian estimate:&lt;/p&gt;&#xA;&lt;p&gt;weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C&lt;/p&gt;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&lt;p&gt;R = average for the movie (mean) = (Rating)&lt;/p&gt;&#xA;&lt;p&gt;v = number of votes for the movie = (votes)&lt;/p&gt;&#xA;&lt;p&gt;m = minimum votes required to be listed in the Top 250 (currently 3000)&lt;/p&gt;&#xA;&lt;p&gt;C = the mean vote across the whole report (currently 6.9)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Please note that in addition to the rating Algorithm, Uber has a dispatch algorithm which takes into consideration factors i.e. Drivers who are online and drivers who are nearest to the passenger.&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-02-15T16:51:17.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5327" PostTypeId="2" ParentId="5325" CreationDate="2018-02-15T16:56:28.390" Score="2" Body="&lt;p&gt;Linear regression is always associated with an activation function, the weights between layers and the structure of the network. The weights between layers are &lt;span class=&quot;math-container&quot;&gt;$\theta_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_1$&lt;/span&gt;. These weights and the input features undergo the dot product operation, which is then the input to the activation function of the next layer's nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An apparently different but the same use of &lt;span class=&quot;math-container&quot;&gt;$\theta_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_1$&lt;/span&gt; is as coefficients to one or more number of terms which themselves are a combination of the input vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Broadly, &lt;span class=&quot;math-container&quot;&gt;$\theta_i$&lt;/span&gt; denotes a weight, i.e. how much preference you want to give to a feature.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-11-28T20:35:52.903" LastActivityDate="2019-11-28T20:35:52.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5328" PostTypeId="2" ParentId="5322" CreationDate="2018-02-15T17:21:56.170" Score="4" Body="&lt;p&gt;Yes, we can do it in a deep learner.&lt;/p&gt;&#xA;&lt;p&gt;For example, suppose we have an input vector likes &lt;span class=&quot;math-container&quot;&gt;$(a, b)$&lt;/span&gt; and from prior knowledge, we know &lt;span class=&quot;math-container&quot;&gt;$a^2 + b^2$&lt;/span&gt; is important too. Hence, we can add this value to the vectors likes &lt;span class=&quot;math-container&quot;&gt;$(a, b, a^2 + b^2)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As another example, suppose date time is important in your data, but not encoded in the input vector. We can add this to the input vector as a third dimension.&lt;/p&gt;&#xA;&lt;p&gt;In summary, it depends on the structure of the prior knowledge, we can encode it into the input vector.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2020-12-03T14:41:31.547" LastActivityDate="2020-12-03T14:41:31.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5329" PostTypeId="1" CreationDate="2018-02-15T18:29:45.160" Score="4" ViewCount="211" Body="&lt;p&gt;I'm looking for an industry standard framework for joining multiple neural networks in a modular way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have two or more neural networks trained to perform certain tasks. By feeding the outputs of some networks into the inputs of others we might obtain higher functionality, but to test multiple hypotheses, we would need a way to rapid prototype those configurations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JF9wW.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JF9wW.gif&quot; alt=&quot;Modular Neural Networks&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.teco.edu/~albrecht/neuro/html/node32.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The source of this image is here &amp;mdash; a proposition of this modular ANN architecture in a thesis&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be interested in knowing:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Do any frameworks or libraries like this even exist?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If so, do they support distributed models, so that models don't have to be hosted in the same process or on the same machine?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do they allow hosting models to be generated from different deep learning frameworks?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12765" LastEditorUserId="4302" LastEditDate="2018-07-29T04:27:58.717" LastActivityDate="2018-11-29T13:02:50.520" Title="Framework for Joining Multiple Modular Artificial Neural Networks" Tags="&lt;neural-networks&gt;&lt;software-evaluation&gt;&lt;topology&gt;&lt;software-architecture&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5330" PostTypeId="2" ParentId="5314" CreationDate="2018-02-15T20:59:11.620" Score="0" Body="&lt;p&gt;Good to have a Starcraft question.  The game has been the subject of growing interest re: AI in recent years, possibly due to its status as king of RTS, which has led to a professional player class no doubt useful for evaluation of AI strength.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because, last time I checked, &lt;a href=&quot;https://www.technologyreview.com/s/609242/humans-are-still-better-than-ai-at-starcraftfor-now/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Humans Are Still Better Than AI at StarCraft—for Now...&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's highly likely there will soon be an algorithm that can beat humans at the game, probably an extension of DeepMind's Alphas, so the clock is ticking...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm personally interested in classical, generalized approaches to strategy game AI, which is archaic from the standpoint of pure strength, but interesting from a game solving perspective.  &lt;em&gt;(Motivations here are from a game product perspective, under the assumption that most humans don't like losing every game with no possibility of ever winning;)&lt;/em&gt; The way I'd personally go about it would be to start thinking about how to abstract the game, generalize the map, unit densities, etc., and try to determine though AI testing if there are sound axioms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For superhuman strength, Deep Learning is clearly the way to go.  The recent results in Go and Chess are just the beginning of the validation of the technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speaking generally, the way I see it, you have a few ways to go: (1) bootstrap existing NN and tweak until it can beat you every time.  But I'm sure many people are already doing this; (2) try to reinvent the wheel and write your own better NN from the ground up.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-15T22:15:00.077" LastActivityDate="2018-02-15T22:15:00.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5331" PostTypeId="2" ParentId="5322" CreationDate="2018-02-16T00:30:25.403" Score="5" Body="&lt;p&gt;Neural nets incorporate prior knowledge. This can be done in two ways: the first (most frequent and more robust) is in data augmentation. For example in convolutional networks, if we know that the &quot;value&quot; (whatever that is, class/regression) of the object we are looking is rotational/translational invariant (our prior knowledge), then we augment the data with random rotations/shifts. The second is in the loss function with some additional term.   &lt;/p&gt;&#xA;" OwnerUserId="8577" LastActivityDate="2018-02-16T00:30:25.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5332" PostTypeId="1" CreationDate="2018-02-16T05:12:24.747" Score="6" ViewCount="3458" Body="&lt;p&gt;I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe my issue is with the exploration function and/or learning rate. Thing is, I see different explanations in the sources I am following so I'm not sure what's the right way anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I understand so far is that Q-learning is TD with q-val iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a time-limited q-val iteration step is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q[k+1](s,a) = ∑(s'): t(s,a,s') * [r(s,a,s') + γ * max(a'):Q[k](s',a')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q = q-table: state,action -&amp;gt; real&#xA;t = MDP transition model&#xA;r = MDP reward func&#xA;γ = discount factor.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But since this is a model-free, sample-based setting, the above update step becomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q(s,a) = Q(s,a) + α * (sample - Q(s,a))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample = r + γ * max(a'):Q(s',a')&#xA;r  = reward, also coming from percept after taking action a in step s.&#xA;s' = next state coming from percept after taking action a in step s. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now for example, assume the following MDP:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    0    1    2    3    4  &#xA;0 [10t][ s ][ ? ][ ? ][ 1t]&#xA;&#xA;Discount: 0.1 | &#xA;Stochasticity: 0 | &#xA;t = terminal (only EXIT action is possible)&#xA;s = start&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With all of the above, my algo (in pseudo code) is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: mdp, episodes, percept&#xA;Q: s,a -&amp;gt; real is initialized to 0 for all a,s&#xA;α = .3&#xA;&#xA;for all episodes:&#xA;    s = mdp.start&#xA;&#xA;    while s not none:&#xA;        a  = argmax(a): Q(s,a) &#xA;        s', r = percept(s,a)&#xA;        sample = r + γ * max(a'):Q(s',a')&#xA;        Q(s,a) = Q(s,a) + α * [sample - Q(s,a)]&#xA;        s = s'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As stated above, the algorithm will not learn. Because it will get greedy fast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will start at 0,1 and choose the best action so far. All q-vals are 0 so it will choose based on arbitrary order on how the qvals are stored in Q. Asume 'W' (go west) is chosen. It will go to 0,0 with a reward of 0 and a q-val update of 0 (since we don't yet know that 0,0, EXIT yields 10)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the next step it will take the only possible action EXIT from 0,0 and get 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point the q-table will be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0,1,W:      0&#xA;0,0,Exit:   3 (reward of 10 averaged by learning rate of .3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the episode is over because 0,0 was terminal.&#xA;On the next episode, it will start at 0,1 again and take W again because of the arbitrary order. But now 0,1,W will be updated to 0.09. Then 0,0,Exit will be taken again (and 0,0,Exit updated to 5.1). Then the second episode will be over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point the q-table is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0,1,W:      0.09&#xA;0,0,Exit:   5.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the sequence 0,1,W-&gt;0,0,Exit will be taken ad infinitum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this takes me to learning rates and the exploration functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book 'Artificial Intelligence: A Modern Approach' (3ed, by Russell) first mentions (pages 839-842) the exploration function as something to put in the val update (because it is discussing a model-based, value iteration approach instead).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So extrapolating from the val update discussion in the book, I'd assume the q-val update becomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q(s,a) = ∑(s'): t(s,a,s') * [r(s,a,s') + γ * max(a'):E(s',a')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where E would be an exploration function which according to the book could be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E(s,a) = &amp;lt;bigValue&amp;gt; if visitCount(s,a) &amp;lt; &amp;lt;minVisits&amp;gt; else Q(s,a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea being to artificially pump up the vals of actions which have not been tried yet and so now they'll be tried out at least &lt;code&gt;minVisits&lt;/code&gt; times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then, in page 844 the book shows pseudo code for Q-learning and instead does not use this E in the q-val update but rather in the argmax of the action selection. I guess makes sense? Since exploration amounts to choosing an action...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other source I have is the UC Berkeley CS188 lecture videos/notes.&#xA;In those (Reinforcement Learning 2: 2016) they show the exploration function in the q-val update step. This is consistent with what I extrapolated from the book's discussion on value iteration methods but not with what the book shows for Q-Learning (remember the book uses the exploration function in the argmax instead).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried placing exploration functions in the update step, the action selection step and in both at the same time.. and still the thing eventually gets greedy and stuck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So not sure where and how this should be implemented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other issue is the learning rate. The explanation usually goes &quot;you need to decrease it over time.&quot; Ok.. but is there some heuristic? Right now, based off the book I am doing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;learn(s,a) = 0.3 / visitCount(s,a)&lt;/code&gt;. But no idea if it is too much or too little or just right. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, assuming I had the exploration and learn right, how would I know how many episodes to train for? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking I'd have to keep 2 versions of the Q-table and check at which point the q-vals do not change much from previous iterations (similar to value iteration for solving known MDPs).&lt;/p&gt;&#xA;" OwnerUserId="12773" LastEditorUserId="1671" LastEditDate="2018-02-16T17:46:09.450" LastActivityDate="2020-11-05T13:14:27.143" Title="How to implement exploration function and learning rate in Q Learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5333" PostTypeId="1" CreationDate="2018-02-16T05:39:17.747" Score="5" ViewCount="204" Body="&lt;p&gt;I'm working on a project related to machine Q&amp;amp;A, using the SQuAD dataset. I've implemented a neural-net solution for finding answers in the provided context paragraph, but the system (obviously) struggles when given questions that are unanswerable from the context. It usually produces answers that are nonsensical and of the wrong entity type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any existing research in telling whether or not a question is answerable using the info in a context paragraph? Or whether a generated answer is valid? I considered textual entailment but it doesn't seem to be exactly what I'm looking for (though maybe I'm wrong about that?)&lt;/p&gt;&#xA;" OwnerUserId="12775" LastEditorUserId="12775" LastEditDate="2018-02-16T13:45:11.513" LastActivityDate="2018-08-16T16:27:09.707" Title="Methods to tell if a question can be answered from a paragraph" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5334" PostTypeId="2" ParentId="5332" CreationDate="2018-02-16T08:37:02.373" Score="3" Body="&lt;p&gt;Your main problem is that you need to separate out what is driving the &lt;em&gt;behaviour policy&lt;/em&gt; from the Q-table.&lt;/p&gt;&#xA;&lt;p&gt;Q Learning is an off-policy algorithm. The Q-table that it eventually learns is for an optimal policy (also called the &lt;em&gt;target policy&lt;/em&gt;). In order to be able to learn that policy, the agent needs to explore. The usual way to do this is to make the agent follow a different policy (called the &lt;em&gt;behaviour policy&lt;/em&gt;). For efficient learning, you generally want the &lt;em&gt;behaviour policy&lt;/em&gt; to be similar to the &lt;em&gt;target policy&lt;/em&gt;. So it is common to also drive the behaviour policy from the Q-table, but not absolutely necessary.&lt;/p&gt;&#xA;&lt;p&gt;You do not need an exploration function, but it is one good way to drive exploration.&lt;/p&gt;&#xA;&lt;p&gt;The simplest behaviour policy, and one that will work in your case, is to behave completely randomly - ignore the Q-table and select actions at random. With your simple toy problem, that should work reasonably well.&lt;/p&gt;&#xA;&lt;p&gt;A more common approach is to behave &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedily. For some probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0.1$&lt;/span&gt;), behave randomly. Otherwise take the argmax over a of &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The exploration function approach is similar to what you have so far, you are just missing the separation of behaviour policy from the Q-table updates. The Q-table update ignores the exploration function, and maximises over estimated next values:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = Q(s,a) + \alpha (r + \gamma \text{max}_{a'}Q(s',a') - Q(s,a))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The behaviour policy for picking the actual next action to take can be decided by using the exploration function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a' = \text{argmax}_{a'} E(s',a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that in stochastic environments (where the chosen action may lead randomly to multiple different states), that you may be able to get away without any separate behaviour policy, and always act greedily with respect to the Q-table. However, that is not a generic solution - such a learning agent would do badly in deterministic environments.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-05T13:14:27.143" LastActivityDate="2020-11-05T13:14:27.143" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5336" PostTypeId="1" AcceptedAnswerId="5487" CreationDate="2018-02-16T10:17:09.810" Score="4" ViewCount="784" Body="&lt;p&gt;I am fairly a newbie to Neural Networks.&lt;/p&gt;&#xA;&lt;p&gt;I wanted to ask if it is possible to train a NN to identify only one type object? For instance, a table from a large set of images, where the NN should be able to identify if new images are tables.&lt;/p&gt;&#xA;&lt;p&gt;If yes can you please guide me in the direction to get started?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I wanted to ask if it is possible to train a NN to identify only one type object?...&lt;/p&gt;&#xA;&lt;p&gt;so my understanding as of now is that if i train an NN on one class and one class alone with a fairly large amount of data then i can get it trained in a small amount of time rather than training it on a huge amount of data and consuming a larger amount of data...&lt;/p&gt;&#xA;&lt;p&gt;Objective is to train a large no of such smaller NNs to create an ensemble of NNs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2904" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-03-02T08:29:16.080" Title="Training a Yes/No NN for Image Classes" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5338" PostTypeId="1" CreationDate="2018-02-16T12:21:49.247" Score="2" ViewCount="239" Body="&lt;p&gt;What are examples of simple problems and applications that can be solved with AI techniques, for a beginner who is trying to make use of his basic programming skills into AI at the beginning level? &lt;/p&gt;&#xA;" OwnerUserId="12780" LastEditorUserId="2444" LastEditDate="2020-06-02T23:51:21.143" LastActivityDate="2020-06-02T23:51:21.143" Title="What are examples of simple problems and applications that can be solved with AI techniques?" Tags="&lt;game-ai&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5340" PostTypeId="2" ParentId="5134" CreationDate="2018-02-16T13:57:45.773" Score="1" Body="&lt;p&gt;I'm not quite sure it's possible. Hash functions are used to map an input to an output in a way that is not reversible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many companies store a hash of your password on their servers so in case of a security breach they haven't given the adversaries a long list of passwords. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as it goes for finding the exact hash of a word, it seems infeasible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Binary classification refers to the possible output being two possible states. A ten dimensional one-hot vector is not binary.&lt;/p&gt;&#xA;" OwnerUserId="9271" LastEditorUserId="9271" LastEditDate="2018-02-16T14:43:56.487" LastActivityDate="2018-02-16T14:43:56.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5342" PostTypeId="2" ParentId="5338" CreationDate="2018-02-16T17:42:35.670" Score="5" Body="&lt;p&gt;This is fairly boilerplate advice, but, since you're brand new to AI, I'd personally suggest writing a classical Tic-Tac-Toe AI, ideally using &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;noreferrer&quot;&gt;minimax&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest this because &lt;a href=&quot;http://www.flyingmachinestudios.com/programming/minimax/&quot; rel=&quot;noreferrer&quot;&gt;minimax is fundamental to AI&lt;/a&gt;, and there are many webpages devoted to this subject, such as &lt;a href=&quot;https://medium.freecodecamp.org/how-to-make-your-tic-tac-toe-game-unbeatable-by-using-the-minimax-algorithm-9d690bad4b37&quot; rel=&quot;noreferrer&quot;&gt;How to make your Tic Tac Toe game unbeatable by using the minimax algorithm&lt;/a&gt; and &lt;a href=&quot;https://www.neverstopbuilding.com/blog/2013/12/13/tic-tac-toe-understanding-the-minimax-algorithm13&quot; rel=&quot;noreferrer&quot;&gt;Tic Tac Toe: Understanding the Minimax Algorithm&lt;/a&gt;.  (Google search for &quot;Tic-tac-toe&quot; and &quot;minimax&quot; will yield a plethora of other sites.  I'd also recommend looking at this minimax page from Stanford: &quot;&lt;a href=&quot;https://cs.stanford.edu/people/eroberts/courses/soco/projects/2003-04/intelligent-search/minimax.html&quot; rel=&quot;noreferrer&quot;&gt;Strategies and Tactics for Intelligent Search&lt;/a&gt;&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend this approach as a good basic primer.  The real cutting-edge work is being done in Machine Learning and Neural Networks, and for that reason, it's probably more important than ever to have some basic grounding in classical AI before you start dipping your toe in that pond.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-02-16T17:42:35.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5343" PostTypeId="1" AcceptedAnswerId="5344" CreationDate="2018-02-16T19:36:22.933" Score="6" ViewCount="2514" Body="&lt;p&gt;I was reading the paper &lt;a href=&quot;https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks&lt;/a&gt; about improving the learning of an ANN using &lt;em&gt;weight normalization&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They consider standard artificial neural networks where the computation of each neuron consists in taking a weighted sum of input features, followed by an elementwise nonlinearity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = \phi(\mathbf{x} \cdot \mathbf{w} + b)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-dimensional weight vector, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a scalar bias term, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-dimensional vector of input features, &lt;span class=&quot;math-container&quot;&gt;$\phi(\cdot)$&lt;/span&gt; denotes an elementwise nonlinearity and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; denotes the&#xA;the scalar output of the neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They then propose to reparameterize each weight vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; in terms of a parameter vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt; and a scalar parameter &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; and to perform stochastic gradient descent with respect to those parameters instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{w} = \frac{g}{\|\mathbf{v}\|} \mathbf{v}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-dimensional vector, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is a scalar, and &lt;span class=&quot;math-container&quot;&gt;$\|\mathbf{v}\|$&lt;/span&gt; denotes the Euclidean norm of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt;. They call this  reparameterizaton &lt;em&gt;weight normalization&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is this scalar &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; used for, and where does it come from? Is &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; is the normalized weight? In general, how does weight normalization work? What is the intuition behind it?&lt;/p&gt;&#xA;" OwnerUserId="12788" LastEditorUserId="2444" LastEditDate="2019-10-24T15:14:09.560" LastActivityDate="2019-10-24T15:19:16.183" Title="How does weight normalization work?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5344" PostTypeId="2" ParentId="5343" CreationDate="2018-02-16T20:15:06.580" Score="2" Body="&lt;p&gt;Your interpretation is quite correct. I could not understand how it would speed up the convergence though. What they are doing is basically re-assigning the magnitude of the weight vector (also called &lt;em&gt;norm&lt;/em&gt; of the weight vector).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put things in the perspective, the conventional approach to any Machine Learning cost function is to not only check the variations of the error with respect to a weight variable (gradient) but also add a normalization term which is &lt;span class=&quot;math-container&quot;&gt;$\lambda (w_0^2 + w_1^2 + \dots)$&lt;/span&gt;. This has got a few advantages:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The weights will not get exponentially high, even though if you make some mistake (generally bouncing off to exponential costs due to the wrong choice of learning rate).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, the convergence is quicker somehow (maybe because you have now 2 ways to control how much weight should be given to a feature. Unimportant features weights are not only getting reduced by normal gradient, but also the gradient of the normalization term &lt;span class=&quot;math-container&quot;&gt;$\lambda (w_0^2 + w_1^2 + \dots)$&lt;/span&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this paper, they have proposed to fix the magnitude of the weight vector. This is a good way, although I am not sure if it is better than feature normalization. By limiting the magnitude of weight to &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, they are fixing the resource available. The intuition is that, if you have 24 hours, you have to distribute this time among subjects. You'll distribute it in a way such that your grade/knowledge is maximized. So this might be helping in the faster convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, another intuition would be, when you are subtracting the gradient from a weight vector you use a learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;. This decides by how much weight-age of error you want to give which will be subsequently be subtracted from the weights. In this approach, you are not only subtracting the weights but also using another learning rate &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; to scale the weight. I call this &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; a learning rate because you can customize it which in turn customizes the value of weights which in turn affects the future reductions of weight gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure someone will post a better mathematical explanation of this stuff but this is all the intuition I could think of. I would be grateful if other intuitions and mathematical subtleties are pointed out. Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-10-24T15:19:16.183" LastActivityDate="2019-10-24T15:19:16.183" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5345" PostTypeId="2" ParentId="5338" CreationDate="2018-02-16T20:24:15.443" Score="1" Body="&lt;p&gt;I will assume you talk about applied AI (in generalized/strong AI we have nothing yet to program :-). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can look at any university course of introduction to AI and see its chapters and the program examples they use ( start by programming without any theory is not a way ).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By example, one common issue on this kind of courses is path finding, using for it algorithms as A* algorithm and applying them to games as Hanoi Towers. This kind of knowledge is a must for any activity in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Standford link provided by @DukeZhou is a good example of one of theses courses, just I suggest start it from first chapter instead of go directly to minmax. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later on, you can jump to more advanced concepts, as recognition/classification and its common approaches: k-nearest/k-means, decision networks, neural nets, ... .  &lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-16T20:36:01.557" LastActivityDate="2018-02-16T20:36:01.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5346" PostTypeId="2" ParentId="5043" CreationDate="2018-02-16T22:40:51.920" Score="0" Body="&lt;p&gt;I'd strongly recommend looking into &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game Theory&lt;/a&gt;'s relationship and impact on AI. &lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prisoner's Dilemma&lt;/a&gt; is a good place to start, because &lt;a href=&quot;https://en.wikipedia.org/wiki/Pareto_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimality&lt;/a&gt; can have repercussions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With computing in general, optimization is a major goal.  For AI, optimal decision-making is what it's all about.  But sans humanity, this may prove to be problematic.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;em&gt;(Apologies for the brevity--I'll be returning to elaborate since this is a subject of personal preoccupation--but I wanted to leave you with a few tidbits in the meantime. :)&lt;/em&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-02-16T22:40:51.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5347" PostTypeId="1" CreationDate="2018-02-16T22:44:52.420" Score="3" ViewCount="157" Body="&lt;p&gt;I have been reading a lot lately about some very promising work coming out of Uber's AI Labs using mutation algorithms enhanced with &lt;em&gt;novelty search&lt;/em&gt; to evolve deep neural nets. See the paper &lt;a href=&quot;https://arxiv.org/abs/1712.06563&quot; rel=&quot;nofollow noreferrer&quot;&gt;Safe Mutations for Deep and Recurrent Neural Networks through Output Gradients&lt;/a&gt; (2018) for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In novelty search, are the novel structures or behavior of the neural network rewarded?&lt;/p&gt;&#xA;" OwnerUserId="12788" LastEditorUserId="2444" LastEditDate="2019-11-22T00:33:31.910" LastActivityDate="2019-11-22T02:26:30.797" Title="In novelty search, are the novel structures or behaviour of the neural network rewarded?" Tags="&lt;deep-learning&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;&lt;novelty-search&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5348" PostTypeId="1" CreationDate="2018-02-17T07:59:05.443" Score="3" ViewCount="2981" Body="&lt;p&gt;What is the difference between AI architecture and AI models? Are both of them the same? If not, please distinguish both of them and give example of each.&lt;/p&gt;&#xA;" OwnerUserId="12803" LastEditorUserId="2444" LastEditDate="2019-05-25T21:46:09.573" LastActivityDate="2019-05-25T21:46:09.573" Title="What is the difference between AI architecture and AI model?" Tags="&lt;ai-basics&gt;&lt;models&gt;&lt;comparison&gt;&lt;architecture&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5349" PostTypeId="1" AcceptedAnswerId="5351" CreationDate="2018-02-17T08:29:32.363" Score="2" ViewCount="47" Body="&lt;p&gt;i'm trying to identify numbers and letters in license plate. License plate images are taken at different lighting condtion and converted to gray image. My concern with type of data for training is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gray Image:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since they are taken at different lighthing condition, gray image have different pixel intensity for same number. Which means, i have to get many training data for different lighting condition to train.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edge Image:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They lack enough pixel information since only edge is white while others(background) are black. So i think they will be very weak for translational difference like shearing or shifting.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to get some information about which type of image is better for training number in different lighting condition. I wish to use edge image if they don't  differ much since i can prepare edge image right now.&lt;/p&gt;&#xA;" OwnerUserId="12090" LastActivityDate="2018-02-17T10:37:51.787" Title="In number classification using neural network, is training with edge image better than gray image?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5351" PostTypeId="2" ParentId="5349" CreationDate="2018-02-17T10:13:21.807" Score="1" Body="&lt;p&gt;Theoretically you will have no gain in the error ratio if the system preprocess the images with a linear high-pass filter before to send the image to the NN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let see a simple 1-dimension case that supports this statement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume inputs are &quot;a&quot;, &quot;b&quot; and &quot;c&quot;. A node at first hidden layer will receive an input to the activation function equal to s=w1*a+w2*b*w3*c+... being w1, w2, w3, ... the weights for this node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, assume a simple differential case, where inputs will be ... , b'=b-a, c'=c-b, ... being the input to the hidden node s=w1'a'+w2'b'+w3'c'+...=...+w2'(b-a)+w3'(c-b)+...=...+(w2'-w3')b+...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note both inputs to hidden node are the same if w1=w1'-w2', w2=w2'-w3', ... . So, the NN can itself perform the equivalent of the linear high-pass filtering adjusting the weights of the first hidden layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, changes in learning speed (convergence time) can be expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ps: please, activate tex/latex in this site, urgently ! )&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-17T10:37:51.787" LastActivityDate="2018-02-17T10:37:51.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5353" PostTypeId="1" CreationDate="2018-02-17T11:08:12.603" Score="1" ViewCount="272" Body="&lt;p&gt;The more I think about machine learning the more I realize the importance of finding similarities by using analogies as a way of learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to categorize words into hierarchical tree this method would work I think, if not tell me why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find two sentences that contain same words but their order is different and some words can also be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find another two such sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evaluate strength of analogy between these 4 sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stronger it is the more probability that words mean similar thing, that they belong to same category. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you learn categories you can do analogies on abstract sentences that contain these found categories and in this manner you can build hierarchy of categories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deer eats grass.&#xA;...is to...&#xA;Man eats deer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as &#xA;Cow eats grass&#xA;is to&#xA;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we can find sentence Man eats cow.&#xA;being used in texts that we analyze then we have confirmed that cow and deer belong to category&#xA;Animals that eat grass and that human eats them.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastEditorUserId="1671" LastEditDate="2018-02-18T21:42:43.747" LastActivityDate="2018-02-18T21:42:43.747" Title="Analogies and similarity" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;theory&gt;&lt;computational-linguistics&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5355" PostTypeId="1" CreationDate="2018-02-17T16:39:27.940" Score="2" ViewCount="143" Body="&lt;p&gt;I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the &lt;a href=&quot;http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/welcome_e.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java Neural Network Simulator&lt;/a&gt;, JavaNNS, program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one practice I have to build a neural network to use backpropagation on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created a neural network with one input layer with 12 nodes, one hidden layer with 6 nodes and one output layer with 1 node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Kaggle's titanic competition data with this format following Dataquest course &lt;a href=&quot;https://www.dataquest.io/course/kaggle-fundamentals&quot; rel=&quot;nofollow noreferrer&quot;&gt;Getting Started with Kaggle for Titanic competition&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Pclass_1,Pclass_2,Pclass_3,Sex_female,Sex_male,Age_categories_Missing,Age_categories_Infant,Age_categories_Child,Age_categories_Teenager,Age_categories_Young Adult,Age_categories_Adult,Age_categories_Senior,Survived&#xA;0,0,1,0,1,0,0,0,0,1,0,0,0&#xA;1,0,0,1,0,0,0,0,0,0,1,0,1&#xA;0,0,1,1,0,0,0,0,0,1,0,0,1&#xA;1,0,0,1,0,0,0,0,0,1,0,0,1&#xA;0,0,1,0,1,0,0,0,0,1,0,0,0&#xA;0,0,1,0,1,1,0,0,0,0,0,0,0&#xA;1,0,0,0,1,0,0,0,0,0,1,0,0&#xA;0,0,1,0,1,0,1,0,0,0,0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want see the same data better in an Spreadsheet:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ab2aX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ab2aX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But they preprocess the data to use it with linear regression and I don't know if I can use these data with backpropagation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think something is wrong because when I run backpropagation in JavaNNS I get these data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opened at: Sat Feb 17 17:29:40 CET 2018&#xA;Step 200 MSE:   0.5381023044692738  validation: 0.11675894327003862&#xA;Step 400 MSE:   0.5372328944712378  validation: 0.11700781497209432&#xA;Step 600 MSE:   0.5370386219557437  validation: 0.11691717861750939&#xA;Step 800 MSE:   0.5370348711919518  validation: 0.11696104763606407&#xA;Step 1000 MSE:  0.5369724294992798  validation: 0.11697568840154722&#xA;Step 1200 MSE:  0.5369697016710676  validation: 0.11665485957481342&#xA;Step 1400 MSE:  0.5370053339270906  validation: 0.11684215268609244&#xA;Step 1600 MSE:  0.5370121961199371  validation: 0.11670833992558485&#xA;Step 1800 MSE:  0.5370200812483633  validation: 0.11673550099633925&#xA;Step 2000 MSE:  0.5367923502149529  validation: 0.11675956129361797&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Nothing changes, it is like it doesn't learn anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many hidden layers does the network have with how many nodes on each hidden layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the problem is that the data have been prepared to be used in Linear regression and I using it with Backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have only created the neural network, I haven't implemented the backpropagation algorithm because it is already implemented in JavaNNS.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2018-02-19T16:20:24.040" LastActivityDate="2018-02-19T16:20:24.040" Title="Data prepared to linear regression. Can I use it with backpropagation?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;hidden-layers&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5356" PostTypeId="2" ParentId="5314" CreationDate="2018-02-17T18:42:24.097" Score="3" Body="&lt;p&gt;StarCraft II is a real time strategy game that combines fast paced micro actions with the need for high level planning and execution. StarCraft II being a popular game with millions of users it proceeds that defeating top players becomes a meaningful and measurable long term objective in AI research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer games provide a compelling solution to the issue of evaluating and comparing different learning and planning approaches on standardized tasks. They are an important source of challenges for research in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game playing AI agents i.e. deepmind's Atarinet and DQN alongside Open AI's Dota 2 bot represent the first demonstration of a General Purpose Agent that is able to continually adapt behavior without any human intervention, a major technical step forward in the quest for general AI (Source deepmind blog).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer games offer numerous advantages in AI research i.e:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;They have clear objective measures of success.&lt;/li&gt;&#xA;&lt;li&gt;Computer games typically output rich streams of observational data, which are ideal inputs for deep networks.&lt;/li&gt;&#xA;&lt;li&gt;They are externally defined to be difficult and interesting for a human to play. Therefore they provide an excellent test for intelligence. &lt;/li&gt;&#xA;&lt;li&gt;Games are designed to be able to run anywhere with the same interface and game dynamics. This enables running many simulations in parallel. Sharing and updating the same table throughout training.&lt;/li&gt;&#xA;&lt;li&gt;In some cases pools of superb human players exist, making it possible to benchmark against highly skilled humans.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The Starcraft challenge for reinforcement learning, introduces a taxing set of problems because it is a multi-agent problem with multiple players interacting. There is imperfect information due to a partially observed map, it has a large state space, it has delayed credit assignment requiring long term strategies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tools&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SC2LE Environment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DeepMind and Blizard games have collaborated to release the SC2LE, which exposes StarCraft II as a research environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SC2LE consists of three sub-components.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A Linux Starcraft II binary.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;StarCraft II API which allows programmatic control of StarCraft II. The API can be used to start the game, get observations, take actions and review replays.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;PySC25 which is an open source environment written in Python. It includes some mini-games and visualization tools&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Open source Open AI RL environments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Universe - Universe is a software platform by Open AI for measuring and training an AI's general intelligence across games, websites and other applications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gym - Open AI Gym is a toolkit for developing and comparing reinforcement learning algorithms. It makes no assumptions about the structure of your agent and is compatible with any numerical computation library such as Tensorflow or Theano. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Supervised Classification Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this, we could decide to screen capture game sessions from expert players and use it as input to a model. The output could be the direction in which the AI agent could move. This would be a supervised classification approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this is not an elegant solution because we are training a model not on a static dataset but a dynamic one (game environment). The training data from a game environment is stochastic/continuous meaning any number of events can occur. &#xA;Furthermore, humans learn most effectively by interacting with the environment. Not by watching others interact with the environment.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Markov Decision Process&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Markov Decision Processes (MDPs) provide a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker i.e. game environments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reinforcement Learning with Deep Q-Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-Learning is a strategy that has been proven to find an optimal action selection policy for any Markov Decision Process (MDP). In Q-Learning we choose an action that maximizes future reward. The further in the future we go, the further the rewards can diverge, we resolve this by adding a discount in future rewards. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unlike policy gradient methods, which attempt to learn functions which directly map an observation to an action, Q-Learning attempts to learn the value of being in a given state, and taking a specific action there. (Arthur J 2016)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The formula for Q-Learning is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5C&amp;space;Q(s,a)&amp;space;=&amp;space;%5Csum_%7Bs%27%7D&amp;space;P_a(s,s%27)&amp;space;(R_a(s,s%27)&amp;space;&amp;plus;&amp;space;%5Cgamma&amp;space;V(s%27))&quot; title=&quot;\ Q(s,a) = \sum_{s'} P_a(s,s') (R_a(s,s') + \gamma V(s'))&quot; /&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Where:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;R&lt;/em&gt; = Reward&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;s&lt;/em&gt; = State&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;a&lt;/em&gt; = Action&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Experience during learning is based on &lt;em&gt;(s, a)&lt;/em&gt; pairs&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;One has an array &lt;em&gt;Q&lt;/em&gt; and uses experience to update it directly &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(Source wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Markov_decision_process&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One of the strengths of Q-learning is that it is able to compare the expected utility of the available actions without requiring a model of the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further reference, I recommend you look at Siraj Ravals tutorial on Deep Q-Learning &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=79pmNdyxEGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=79pmNdyxEGo&lt;/a&gt; and source code for the same available here &lt;a href=&quot;https://github.com/llSourcell/deep_q_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/llSourcell/deep_q_learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally I recommend the following references for more information on computer game playing AI agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;StarCraft II: A New Challenge for Reinforcement Learning &lt;a href=&quot;https://arxiv.org/abs/1708.04782&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1708.04782&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Playing Atari with Deep Reinforcement Learning &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human-level control through deep reinforcement learning  &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10913" LastEditorUserId="10913" LastEditDate="2018-02-20T07:37:32.120" LastActivityDate="2018-02-20T07:37:32.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5357" PostTypeId="2" ParentId="4139" CreationDate="2018-02-18T02:19:20.447" Score="1" Body="&lt;p&gt;I also recommend you take a look at the following work by Uber AI Labs who used an interesting approach to computer games:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://eng.uber.com/deep-neuroevolution/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eng.uber.com/deep-neuroevolution/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12788" LastActivityDate="2018-02-18T02:19:20.447" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5358" PostTypeId="1" CreationDate="2018-02-18T06:26:23.680" Score="1" ViewCount="34" Body="&lt;p&gt;I am new to neural networks. I am trying to model the run-off vs. time in a water channel after a storm event given that I know the permeability of the material in the channel, total precipitation, and some other single valued parameters for a particular event..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a database of run off histories, and the values of the associated parameters (permeability, total precipitation, etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want my model to give me a runoff vs. Time history when I enter the associated parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to train my model. Do i just stack all the time histories in my database together and feed them together? All examples in books use one time history to train the model. Im confused.&lt;/p&gt;&#xA;" OwnerUserId="12814" LastActivityDate="2018-02-18T06:26:23.680" Title="How to train a recurrent neural network with multiple series" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5359" PostTypeId="1" CreationDate="2018-02-18T08:41:34.297" Score="1" ViewCount="409" Body="&lt;p&gt;From Russell-Norvig:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A CSP is strongly k-consistent if it is k-consistent and is also (k − 1)-consistent, (k − 2)-consistent, . . . all the way down to 1-consistent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can a CSP be k-consistent without being (k - 1)-consistent? I can't think of any counter example for this case. Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12608" LastActivityDate="2018-02-18T19:00:54.843" Title="Does k consistency always imply (k - 1) consistency?" Tags="&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5361" PostTypeId="2" ParentId="5359" CreationDate="2018-02-18T16:42:19.630" Score="1" Body="&lt;p&gt;Define P as a CSP where X, Y are the variables, domain of both is {1,2,3,4} and conditions in normal form are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;node-condition X&amp;lt;4&lt;/li&gt;&#xA;&lt;li&gt;arc-condition X=Y&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;P is 2-consistent (arc consistent) because for any X value it is possible to find a Y value that fulfills the arc-condition X=Y.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, P is not 1-consistent (node-consistent) because exist a X value (X=4) that can not fulfill the node condition x&amp;lt;4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons, this problem is 2-consistent but not strongly 2-consistent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, it is straightforward to convert this example in a strongly 2-consistent problem, just reducing the domain to {1,2,3}.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-18T19:00:54.843" LastActivityDate="2018-02-18T19:00:54.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5362" PostTypeId="1" CreationDate="2018-02-18T16:47:56.780" Score="1" ViewCount="180" Body="&lt;p&gt;The problem: I want to classify a trajectory if it has some properties, for example I want to create a simple 0/1 classifier for circular trajectories. If a target is moving in a circular trajectory the network should produce 1, if not it should produce 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My input and data set: what I have is data set with cartesian coordinates in 2d so x,y,Vx,Vy. I have a dataset of 10000 trajectory, 5000 circular, 5000 rectilinear. So I feed the network with a tensor [10000, 4, 1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question: I'm trying to use a network with three layers, input layer with 4 neurons, hidden layer with 2 LTSM and one fc layer with sigmoid activation function. Is it possible to feed the network with a tensor [4x1] each time? Or do I need to provide the information in batches? Or what? Is the design of my basic network correct?&lt;/p&gt;&#xA;" OwnerUserId="12824" LastActivityDate="2018-02-18T16:47:56.780" Title="Trajectory classification using RNN" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5363" PostTypeId="2" ParentId="5348" CreationDate="2018-02-18T16:53:02.973" Score="1" Body="&lt;p&gt;Although this is a kind of an ambiguous question I am choosing to answer it as I see fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, there are 2 parts in creating an AI agent, the logical part and the learning part. The logical part is called AI itself while while the other part is called Machine Learning (in general both ML and AI are considered under AI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now ,in AI or any combinatorial designs you need something called a model, a mathematical model specifically. In this model you know exactly how each and every artworks and can easily derive how will it work i its entirety. This type of models are mainly used in the AI part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Machine Learning part, we have architectures. They are not exactly what we call mathematical models. We have scarce understanding of their inner workings. We only know the mathematics behind it. We try to fit 1 architecture to a problem. If it does not solve the problem satisfactorily, we try other architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, if we create a program for &lt;a href=&quot;https://www.geeksforgeeks.org/c-program-for-tower-of-hanoi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towers of Hanoi&lt;/a&gt; we have to decompose the solution into smaller parts whose mathematical model we have to create. We know the working of each step and the values that will be returned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas, in a Neural Network architecture, you just create 3-4 layers with some number of nodes, a certain learning rate, a certain cost function, a certain cost reduction technique and hope for the best. You have no realistic way to know the correctness of your thinking&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, this is the difference between a model and architecture in AI context. In real life also if we see, an architect merely draws the building plan. Its a civil engineer who has to first draw/build up the mathematical model to see how the building weight shall be distributed on the foundations, whether the architecture is practical or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as AI books are concerned these 2 are the best in their class:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0201533774&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And for Machine Learning you can look into this answer I provided:&#xA;&lt;a href=&quot;https://ai.stackexchange.com/questions/4464/vetted-sources-of-ai-theory-tools-applications-for-an-experienced-programmer-new/4466#4466&quot;&gt;Vetted sources of AI Theory/Tools/Applications for an experienced programmer new to the field?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-02-18T16:53:02.973" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5365" PostTypeId="5" CreationDate="2018-02-18T21:33:35.980" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-03-05T13:06:32.727" LastActivityDate="2020-03-05T13:06:32.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5366" PostTypeId="4" CreationDate="2018-02-18T21:33:35.980" Score="0" Body="For questions related to the architecture of AI models, e.g. the architecture of neural networks." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-03-05T13:06:32.727" LastActivityDate="2020-03-05T13:06:32.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5367" PostTypeId="5" CreationDate="2018-02-18T21:34:28.793" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Software_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Software_architecture&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-18T21:34:28.793" LastActivityDate="2018-02-18T21:34:28.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5368" PostTypeId="4" CreationDate="2018-02-18T21:34:28.793" Score="0" Body="Use for questions about general software architecture.  (For questions specifically about AI architecture, use the plain &quot;architecture&quot; tag.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-18T21:34:28.793" LastActivityDate="2018-02-18T21:34:28.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5369" PostTypeId="1" AcceptedAnswerId="5421" CreationDate="2018-02-19T01:45:21.330" Score="1" ViewCount="4154" Body="&lt;p&gt;I am implementing a feed-forward neural network with leaky ReLU activation functions and back-propagation from scratch. Now, I need to compute the partial derivatives, but I don't know what the derivative of the Leaky ReLU is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the C# code for the leaky RELU function which I got from &lt;a href=&quot;https://www.codeproject.com/Articles/1220276/ReInventing-Neural-Networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;this site&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;private double leaky_relu(double x)&#xA;{&#xA;    if (x &amp;gt;= 0)&#xA;        return x;&#xA;    else&#xA;        return x / 20;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12788" LastEditorUserId="2444" LastEditDate="2020-05-30T13:02:16.177" LastActivityDate="2020-05-30T13:02:16.177" Title="What is the derivative of the Leaky ReLU activation function?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;math&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5370" PostTypeId="1" AcceptedAnswerId="5409" CreationDate="2018-02-19T12:07:13.790" Score="3" ViewCount="234" Body="&lt;p&gt;I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework.&lt;/p&gt;&#xA;&lt;p&gt;I came across &lt;a href=&quot;http://cs231n.github.io/transfer-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt;, where we fine-tune a pre-trained neural network with new data. There's a nice PyTorch tutorial explaining &lt;a href=&quot;http://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#training-the-model&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt;. We have a &lt;a href=&quot;https://github.com/amdegroot/ssd.pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch implementation of the Single Shot Detector (SSD)&lt;/a&gt; as well. See also &lt;a href=&quot;https://towardsdatascience.com/learning-note-single-shot-multibox-detector-with-pytorch-part-1-38185e84bd79&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single Shot MultiBox Detector with Pytorch — Part 1&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is my current situation&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The data I want to fine-tune the neural network with is different from the data that was used to initially train the neural network; more specifically, the neural network was initially trained with a dataset of  20 classes&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I currently have a very small labeled training dataset.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To solve this problem using transfer learning, the solution is to freeze the weights of the initial layers, and then train the neural network with these layers frozen.&lt;/p&gt;&#xA;&lt;p&gt;However, I am confused about what the initial layers are and how to change the last layers of the neural network to solve my specific task. So, here are my questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the initial layers in this case? How exactly can I freeze them?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the changes I need to make while training the NN to classify one or more new classes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11038" LastEditorUserId="2444" LastEditDate="2021-01-07T17:45:26.290" LastActivityDate="2021-01-07T17:45:52.477" Title="When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?" Tags="&lt;convolutional-neural-networks&gt;&lt;transfer-learning&gt;&lt;fine-tuning&gt;&lt;single-shot-multibox-detector&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5371" PostTypeId="1" CreationDate="2018-02-19T13:18:50.290" Score="2" ViewCount="85" Body="&lt;p&gt;Lately I've been wondering.&#xA;Is there's a way to locate redundant/unnecessary/misleading inputs by analysis of weights in the first layer?&lt;/p&gt;&#xA;" OwnerUserId="12327" LastEditorUserId="12327" LastEditDate="2018-02-19T17:53:27.933" LastActivityDate="2018-02-23T09:30:16.750" Title="Identify unnecessary inputs of NN" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5372" PostTypeId="1" CreationDate="2018-02-19T16:54:33.217" Score="1" ViewCount="48" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1503.03832.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;FaceNet paper&lt;/a&gt; there mentions an gradient algorithm called 'AdaGrad'(Adaptive Gradient) referenced to &lt;a href=&quot;http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; which has apparently been used to calculate the gradient of the Triplet Loss function. After referring to the paper also I find it hard to understand how to calculate this adaptive gradient.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas regarding this matter? Would love to hear any explanations or ideas towards understanding this concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you. &lt;/p&gt;&#xA;" OwnerUserId="12843" LastActivityDate="2018-02-19T16:54:33.217" Title="How to calculate Adaptive gradient?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5376" PostTypeId="1" CreationDate="2018-02-19T22:19:28.513" Score="1" ViewCount="38" Body="&lt;p&gt;I have a problem in which the dimensions of the input are increasing in row and column at each timestep. What method for preprocessing could be done or are there any architectures used for solving such a case?&lt;/p&gt;&#xA;" OwnerUserId="12849" LastActivityDate="2018-02-19T22:19:28.513" Title="Dealing with input to recurrent net with changing dimensions" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5377" PostTypeId="1" AcceptedAnswerId="5379" CreationDate="2018-02-20T00:11:50.153" Score="4" ViewCount="2155" Body="&lt;p&gt;I have a map. I need to colour it with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; colours, such that two adjacent regions do not share a colour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I formulate the map colouring problem as a hill climbing search problem?&lt;/p&gt;&#xA;" OwnerUserId="12519" LastEditorUserId="2444" LastEditDate="2019-03-02T11:02:24.677" LastActivityDate="2019-03-02T11:02:24.677" Title="How can I formulate the map colouring problem as a hill climbing search problem?" Tags="&lt;search&gt;&lt;hill-climbing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5378" PostTypeId="2" ParentId="84" CreationDate="2018-02-20T03:02:08.413" Score="2" Body="&lt;p&gt;Really any 'intelligence' exhibited by a computer is deemed AI, regardless of brute force or use of smart heuristics. For example, a chat bot can be coded to respond to most responses using many, many if statements. This is an AI no matter how poorly coded/designed it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chess playing computer beating a human professional can be seen as a meaningful milestone. I mean, someone programmed a computer to beat grandmaster chess players and chess geniuses. Many thought that wasn't possible since chess is such a complex game. This kind of work likely segued into more complex AI, for if a computer could play chess, then it surely complete other complex tasks as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note how refined chess programming is: magic bitboards, Zobrist hashing, pruning, lazy SMP, and many more. This is perhaps not the sort of milestone of AI that you thought, but again, the things that can be considered AI are pretty broad.&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="75" LastEditDate="2018-04-13T18:10:41.843" LastActivityDate="2018-04-13T18:10:41.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5379" PostTypeId="2" ParentId="5377" CreationDate="2018-02-20T04:21:00.017" Score="3" Body="&lt;p&gt;First of all you need an initial solution. You will then improve this solution with hill climbing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your initial solution, you can color the map randomly using the K colors. This will most likely result in conflicts (adjacent regions of the same color).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the hill climbing part: Find a region which has conflicts and swap its color for another color, making sure that the new color does not incur more conflicts than the old one. With each iteration your solution should slowly improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that hill climbing is not perfect and that you may not find a feasible solution in the end.&lt;/p&gt;&#xA;" OwnerUserId="12857" LastActivityDate="2018-02-20T04:21:00.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5380" PostTypeId="2" ParentId="5325" CreationDate="2018-02-20T04:22:05.090" Score="1" Body="&lt;p&gt;As said above, they are weights to your hypothesis function that are changed during training to minimize your Error function. You can think of them like slope and y intercept in basic algebra. However, a linear regression hypothesis function can be parameterized by many more weight terms than just theta_0, and theta_1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I detail this process more in this post: &lt;a href=&quot;https://ai.stackexchange.com/questions/4629/how-does-an-activation-functions-derivative-measure-error-rate-in-a-neural-netw/4639#4639&quot;&gt;How does an activation function&amp;#39;s derivative measure error rate in a neural network?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="9469" LastEditDate="2018-02-20T05:47:03.980" LastActivityDate="2018-02-20T05:47:03.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5383" PostTypeId="2" ParentId="84" CreationDate="2018-02-20T13:10:31.087" Score="2" Body="&lt;p&gt;Brute force approach is certainly the first step of many in AI programming. But using these experiences the program must learn to find the best solution or at least a closer solution to the problem. Since the first goal in AI is to find any solution, nothing can beat the brute force approach. But then using the previous results of brute force approaches, the program must develop its own heuristics and use this data along with brute force to find the optimal solution.&lt;/p&gt;&#xA;" OwnerUserId="10875" LastEditorUserId="14723" LastEditDate="2018-04-13T17:51:09.457" LastActivityDate="2018-04-13T17:51:09.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5385" PostTypeId="1" CreationDate="2018-02-21T06:19:23.060" Score="1" ViewCount="114" Body="&lt;p&gt;Two Stanford University researchers, Dr. Michal Kosinki and Yilun Wang have published a paper that claims that AI can predict sexuality from a single facial photo with startling accuracy. This research is obviously disconcerting since it exposes an already vulnerable group to a new form of systematized abuse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The research can be found here &lt;a href=&quot;https://osf.io/zn79k/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://osf.io/zn79k/&lt;/a&gt; ,here  &lt;a href=&quot;https://psyarxiv.com/hv28a/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://psyarxiv.com/hv28a/&lt;/a&gt; and has even been highlighted by Newsweek magazine here &lt;a href=&quot;http://www.newsweek.com/ai-can-tell-if-youre-gay-artificial-intelligence-predicts-sexuality-one-photo-661643&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.newsweek.com/ai-can-tell-if-youre-gay-artificial-intelligence-predicts-sexuality-one-photo-661643&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wRwrA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wRwrA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above is an image of composite heterosexual faces and composite gay faces from the research. (Image courtesy of Dr Michal Kosinki and Yilun Wang)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, as knowledgable members of the AI community, how can we scientifically debunk/discredit this research?&lt;/p&gt;&#xA;" OwnerUserId="10913" LastActivityDate="2018-02-21T17:18:04.107" Title="Is the research by Stanford University students who use logistic regression to predict sexual orientation from facial images really scientific?" Tags="&lt;research&gt;&lt;ethics&gt;&lt;social&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5386" PostTypeId="2" ParentId="5385" CreationDate="2018-02-21T10:03:39.087" Score="4" Body="&lt;p&gt;One way to criticize the study could be to attack the data on which the study is based on. An image on a social network is not &quot;neutral&quot; (those are not ID photo) and certainly not images from a dating website (from which the data of the study come from).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example as a homosexual / heterosexual person you will perhaps put forward different attributes on your photo (facial hair / glasses, type of clothes) to attract gender specific people. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those are parameters are not directly linked with the &quot;facial profile&quot; of the person but they will influence the black-box model during training, so you will end up thinking you can detect sexual orientation of a person with only his face characteristics but in reality your black-box algorithm have only detect a totally different characteristic (glasses for example) that are link with a specific sexual orientation.&lt;/p&gt;&#xA;" OwnerUserId="8912" LastEditorUserId="8912" LastEditDate="2018-02-21T17:18:04.107" LastActivityDate="2018-02-21T17:18:04.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5389" PostTypeId="1" CreationDate="2018-02-21T17:15:53.053" Score="2" ViewCount="63" Body="&lt;p&gt;I have a customer purchasing dataset and the data set is from a retailer having an online store and offline stores. So, customers have two options in their shopping channel, online or offline. In an online shopping, there is a shipping fee however if a basket size is larger than $50 there is no shipping fee. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found pieces of evidence that customers are trying to add some of items to make their basket size larger than $50 when their baskets are near and a little bit below the $50, because their shipping fee can be waived by doing that.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In this situation, I am trying to identify and characterize items that were purchased only because of the shipping threshold by using a machine learning algorithm. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If there is no shipping threshold, $50, the customers would not purchase the items, but they purchased some items to make their basket size larger than $50. I have not observed those kinds of items (added items because of the shipping threshold). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any machine learning algorithm that I can identify those kinds of items? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think I need to use some of unsupervised machine learning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another challenging part is that each customer has different characteristics so I probably need to consider it as well. How can I detect those kinds of items??&lt;/p&gt;&#xA;" OwnerUserId="12713" LastEditorUserId="1671" LastEditDate="2018-02-21T17:43:46.017" LastActivityDate="2018-06-22T14:48:19.570" Title="Detect observations under certain conditions" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5390" PostTypeId="5" CreationDate="2018-02-21T17:52:20.317" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In graph theory, graph coloring is a special case of graph labeling; it is an assignment of labels traditionally called &quot;colors&quot; to elements of a graph subject to certain constraints.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_coloring&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Coloring&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Graph coloring games are mathematical games related to graph theory.  Coloring game problems arose as game-theoretic versions of well-known graph coloring problems.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_coloring_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Coloring Game&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-21T17:52:20.317" LastActivityDate="2018-02-21T17:52:20.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5391" PostTypeId="4" CreationDate="2018-02-21T17:52:20.317" Score="0" Body="Use for questions related to graph coloring and graph coloring games." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-02-21T17:52:20.317" LastActivityDate="2018-02-21T17:52:20.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5392" PostTypeId="1" AcceptedAnswerId="5406" CreationDate="2018-02-22T08:08:59.070" Score="3" ViewCount="71" Body="&lt;p&gt;I am absolutely new in AI area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know how to mathematically/logically represent the &lt;strong&gt;sense&lt;/strong&gt; of sentences like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The cat drinks milk.&#xA;&#xA;Sun is yellow.&#xA;&#xA;I was at work yesterday.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So that it could be converted to computer understandable form and analysed algorithmically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any clue?&lt;/p&gt;&#xA;" OwnerUserId="12907" LastEditorUserId="9947" LastEditDate="2018-02-22T22:06:50.683" LastActivityDate="2018-02-23T15:37:27.007" Title="Represent sense/meaning of sentences mathematically" Tags="&lt;natural-language-processing&gt;&lt;sentience&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5393" PostTypeId="1" CreationDate="2018-02-22T10:40:58.157" Score="1" ViewCount="46" Body="&lt;p&gt;I am trying to perform classification task using Keras and tensorflow. However, the learning converges after achieving an accuracy of 57%. All my inputs and outputs are categorical data. Am I using the correct approach to train? Are there any other example in the web which is trying to solve the similar problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;data.csv&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;religion,caste,qualification,marital_status,sex,nature_activity&#xA;2,20,5,1,1,10&#xA;2,20,5,1,1,10&#xA;2,20,5,1,1,10&#xA;2,20,5,1,1,10&#xA;1,3,5,1,1,13&#xA;1,4,4,1,2,3&#xA;1,3,4,2,1,1&#xA;1,3,3,2,1,1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential&#xA;from keras.layers import Dense&#xA;import pandas&#xA;from sklearn.metrics import classification_report&#xA;from sklearn.preprocessing import LabelEncoder&#xA;from keras.utils import np_utils&#xA;import numpy&#xA;from sklearn.model_selection import train_test_split&#xA;from keras.wrappers.scikit_learn import KerasClassifier&#xA;import sys&#xA;import matplotlib.pyplot as plt&#xA;# fix random seed for reproducibility&#xA;numpy.random.seed(7)&#xA;&#xA;# load pima indians dataset&#xA;dataframe = pandas.read_csv(&quot;data.csv&quot;,header=None)&#xA;dataset=dataframe.values&#xA;# split into input (X) and output (Y) variables&#xA;X = dataset[:,0:4].astype(float)&#xA;Y = dataset[:,4]&#xA;X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.0015, random_state=42)&#xA;encoder = LabelEncoder()&#xA;encoder.fit(Y_train)&#xA;encoded_Y = encoder.transform(Y_train)&#xA;one_hot_enc_Y=np_utils.to_categorical(encoded_Y)&#xA;model=Sequential()&#xA;model.add(Dense(500,input_dim=4,activation='relu'))&#xA;model.add(Dense(100,activation='relu'))&#xA;model.add(Dense(50,activation='relu'))&#xA;model.add(Dense(3,activation='softmax'))&#xA;&#xA;# Compile model&#xA;model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])&#xA;# Fit the model&#xA;history=model.fit(X_train, one_hot_enc_Y, validation_split=0.33,epochs=300, batch_size=5)&#xA;# summarize history for accuracy&#xA;plt.plot(history.history['acc'])&#xA;plt.title('model accuracy')&#xA;plt.ylabel('accuracy')&#xA;plt.xlabel('epoch')&#xA;# summarize history for loss&#xA;plt.plot(history.history['loss'])&#xA;plt.title('model loss')&#xA;plt.ylabel('loss')&#xA;plt.xlabel('epoch')&#xA;plt.legend(['train'], loc='upper right')&#xA;plt.show()&#xA;# evaluate the model&#xA;scores = model.evaluate(X_train, one_hot_enc_Y)&#xA;print(&quot;\n%s: %.2f%%&quot; % (model.metrics_names[1], scores[1]*100))&#xA;sys.exit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12910" LastActivityDate="2019-02-22T09:01:53.270" Title="Improving learning rate for classification task" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5394" PostTypeId="2" ParentId="5098" CreationDate="2018-02-22T10:48:35.313" Score="1" Body="&lt;p&gt;This task can be done with keyframe based animation. The starting surface is keyframe 0, the goal-surface is keyframe 10. A RRT planner can bring the system from frame 0 to frame 10. This is done with brute-force-search in the problem space. To fasten up the search it makes sense, to define helper keyframes between them (guided policy search). Such in-between-keyframes can be extracted from previous manual demonstrations. The overall system consists of two parts: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;an algorithm which searches for actions to bring keyframe a to keyframe b&lt;/li&gt;&#xA;&lt;li&gt;and an algorithm who is searching in previous demonstrations for getting the in-between-keyframes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the literature such problems are discussed in the domain of PDDL. The task-planner works with PDDL specifications, while the Motion planner uses Rapidly-exploring random trees. &lt;a href=&quot;https://ipvs.informatik.uni-stuttgart.de/mlr/papers/15-toussaint-IJCAI.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Logic-Geometric Programming: An Optimization-Based Approach to Combined Task and Motion Planning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2018-02-22T10:48:35.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5395" PostTypeId="2" ParentId="5389" CreationDate="2018-02-22T12:41:33.683" Score="1" Body="&lt;p&gt;Since my comment would not fit, I'll answer the question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I think this problem will be even more better solved if you have the web data as well. Say after adding the required necessity in the &lt;strong&gt;cart&lt;/strong&gt; the customer will check the total amount. And if it is below &lt;code&gt;50$&lt;/code&gt;, the person will add some more items. So checking this can give you a better clue.&lt;/li&gt;&#xA;&lt;li&gt;Another data you must have for better guessing is the order in which the person has added items to his cart. This will also provide you an important clue about the cheap items the customer is adding to &lt;strong&gt;cart&lt;/strong&gt; to try to cross the minimum threshold.&lt;/li&gt;&#xA;&lt;li&gt;You will also have to segregate cheap items from pricey ones for a customer. This data with the above 2 approaches is a definite giveaway.&lt;/li&gt;&#xA;&lt;li&gt;This might be a personal opinion, but whenever I need to cross the shipping fee threshold I usually add items which I have already bought the previous time to escape shipping fee. You will need more data to see if this is true in most cases.&lt;/li&gt;&#xA;&lt;li&gt;And if you don't have any of the above data, the pure Machine Learning approach would be to pick an item which is costly and somewhat popular and an item which will string together people with likewise interests. Say, a great book on ML is this item. &#xA;You find out all the customers who bought this book. Check all the similarities between all the other things bought by the customer, like say one customer adds a book on AI after this while another buys a book on python. So these are related by computer science. Go on until you find all the things that are generally dissimilar between all the customers, since now they are buying to cross the threshold and each will buy according to his own requirements without a common interest in mind. There you have all the data you need.&lt;/li&gt;&#xA;&lt;li&gt;Or you can use the converse of this approach, find a non-significant thing like sugar. String together all the customers who bought this item. Check their interests to see if it has anything to do with sugar. If not send them to one group. Now in this group, match the interests of the customer on other items, if dissimilar you get a good idea that this item was bought to cross threshold.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I understand this is a kind of opinionated answer. But I think these are trade secrets not revealed by companies. so you have to figure out algorithms yourself. Also, simple machine learning won't work, lots of logical programming is also required. And I also think you need to have a good understanding of human psychology works. You can interview your friends and family about what they would do and get a general intuition. Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-02-22T12:41:33.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5396" PostTypeId="2" ParentId="5377" CreationDate="2018-02-22T14:58:21.453" Score="2" Body="&lt;p&gt;First we have to specify the problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Initial State&lt;/strong&gt;: The map all colored randomly.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Successor Function&lt;/strong&gt; (Transition Model): Change the color of a region.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Goal Test&lt;/strong&gt;: The map all colored such that two adjacent regions do not share a color.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cost Function&lt;/strong&gt;: Assigns 1 to change the color of a region.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now that we have the specification of the problem, we have to choose the search algorithm to solve the problem. In this case &quot;Hill Climbing&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we choose &quot;Hill Climbing&quot; we have to specify one more function (the objective function):&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Heuristic Function&lt;/strong&gt;: Returns the number of adjacent regions that share the same color.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now that we have the problem formulated, we apply the &quot;Hill Climbing&quot; algorithm to try to minimize the heuristic function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Philippe Oliver said, you could have several problems using just &quot;Hill Climbing&quot; like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Local minimums.&lt;/li&gt;&#xA;&lt;li&gt;Flat local minimums.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FRVYk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FRVYk.png&quot; alt=&quot;From &amp;quot;Artificial Intelligence: A Modern Approach (3rd Edition)&amp;quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have more information on: &lt;em&gt;Artificial Intelligence: A Modern Approach (3rd Edition) by Stuart Russell and Peter Norvig&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have several approaches to this problem, I show you just one. Another can be specifying the problem incrementally (change the &quot;Initial State&quot; for a map with no regions colored, and then the &quot;Successors&quot; are specified by painting a region in a way that two adjacent regions do not share a color).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;Artificial Intelligence: A Modern Approach (3rd Edition) by Stuart Russell and Peter Norvig, chapter 4.1 (Local Search Algorithms and Optimization Problems).&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5796" LastActivityDate="2018-02-22T14:58:21.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5397" PostTypeId="2" ParentId="5348" CreationDate="2018-02-22T15:25:09.070" Score="2" Body="&lt;p&gt;Well simply put AI model can be seen just as a flowchart showing how the control flow moves where it moves how it moves why it moves etc. &#xA;However AI architecture refers to the next step after building an AI model AI architecture involves representation of the functions that you use in your program. It also involved declaration of the variables you're going to use, the functions that you are going to use, etc.&lt;/p&gt;&#xA;" OwnerUserId="10875" LastActivityDate="2018-02-22T15:25:09.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5398" PostTypeId="1" AcceptedAnswerId="6495" CreationDate="2018-02-22T16:39:30.320" Score="2" ViewCount="268" Body="&lt;p&gt;In some Atari games in the Arcade Learning Environment (ALE), it is necessary to press &lt;code&gt;FIRE&lt;/code&gt; once to start a game. Because it may be difficult for a Reinforcement Learning (RL) agent to learn this, they may often waste a lot of time executing actions that do nothing. Therefore, I get the impression that some people hardcode their agent to press that &lt;code&gt;FIRE&lt;/code&gt; button once when necessary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in OpenAI's &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;baselines repository&lt;/a&gt;, this is implemented using the &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py#L37&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;FireResetEnv&lt;/code&gt;&lt;/a&gt; wrapper. Further down, in their &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py#L223&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;wrap_deepmind&lt;/code&gt;&lt;/a&gt; (which applies that wrapper among others), it is implied that DeepMind tends to use this functionality in all of their publications. I have not been able to find a reference for this claim though.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is&lt;/strong&gt;: is it common in published research (by DeepMind or others) to use the functionality described above? I'd say that, if this is the case, it should be explicitly mentioned in these papers (because it's important to know if hardcoded domain knowledge was added to a learning agent), but I have been unable to explicitly find this after looking through a wide variety of papers. So, based on this, I'd be inclined to believe the answer is &quot;no&quot;. The main thing that confuses me then is the implication (without reference) in the OpenAI baselines repository that the answer would be &quot;yes&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-05-23T08:46:19.700" Title="Is it common in RL research with Atari/ALE to automatically press FIRE to start games?" Tags="&lt;reinforcement-learning&gt;&lt;research&gt;&lt;deepmind&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5399" PostTypeId="1" AcceptedAnswerId="5401" CreationDate="2018-02-22T16:56:56.350" Score="7" ViewCount="2914" Body="&lt;p&gt;I have read somewhere on the web (I lost the reference) that the number of units (or neurons) in a hidden layer should be a power of 2 because it helps the learning algorithm to converge faster.&lt;/p&gt;&#xA;&lt;p&gt;Is this a fact? If it is, why is this true? Does it have something to do with how the memory is laid down?&lt;/p&gt;&#xA;" OwnerUserId="12901" LastEditorUserId="2444" LastEditDate="2021-01-20T23:39:56.410" LastActivityDate="2021-05-07T15:56:04.170" Title="Why should the number of neurons in a hidden layer be a power of 2?" Tags="&lt;deep-learning&gt;&lt;optimization&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;hidden-layers&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5400" PostTypeId="1" AcceptedAnswerId="5402" CreationDate="2018-02-22T18:01:00.163" Score="2" ViewCount="62" Body="&lt;p&gt;When we augment data for training are we also changing the distribution of data and if its a different distribution why do we use it to train a model for original distribution ?&lt;/p&gt;&#xA;" OwnerUserId="12901" LastActivityDate="2018-02-22T18:09:58.070" Title="Does augmenting data changes the distribution of augmented data?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5401" PostTypeId="2" ParentId="5399" CreationDate="2018-02-22T18:09:34.890" Score="13" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I have read somewhere on the web (I lost the reference) that the number of units (or neurons) in a hidden layer should be a power of 2 because it helps the learning algorithm to converge faster.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would quite like to see a reference to this suggestion, in case it has been misunderstood.&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, there is no such effect in normal neural networks. In convolutional neural networks, it might potentially be true in a minor way because some FFT approaches work better with &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; items.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is this a fact? If it is, why is this true? Does it have something to do with how the memory is laid down?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would say that this is not a general fact. Instead, it seems like misunderstood advice to &lt;em&gt;search&lt;/em&gt; some hyperparameters such as number of neurons in each layer, by increasing or decreasing by a factor of 2. Doing this and trying layer sizes of 32, 64, 128 etc should increase the speed of finding a good layer size compared to trying sizes 32, 33, 34 etc.&lt;/p&gt;&#xA;&lt;p&gt;The main reason to pick powers of 2 is tradition in computer science. Provided there is no driver to pick other specific numbers, may as well pick a power of 2 . . . but equally you will see researchers picking multiples of 10, 100 or 1000 as &amp;quot;round numbers&amp;quot;, for a similar reason.&lt;/p&gt;&#xA;&lt;p&gt;One related factor: If a researcher presents a result for some new technique where the hidden layer sizes were tuned to e.g. 531, 779, 282 etc, then someone reviewing the work would ask the obvious question &amp;quot;Why?&amp;quot; - such numbers might imply the new technique is not generic or requires large amounts of hyperparameter tuning, neither of which would be seen as positive traits. Much better to be seen using an obvious &amp;quot;simple&amp;quot; number . . .&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-01-20T23:42:58.787" LastActivityDate="2021-01-20T23:42:58.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5402" PostTypeId="2" ParentId="5400" CreationDate="2018-02-22T18:09:58.070" Score="2" Body="&lt;p&gt;Yes you do change the distribution of your training data if you modify it (for example by augmenting it with rotated versions of images that were in an original training set of images).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is fine because, typically, our goal of training is not to get a model with high performance on the dataset we happened to collect as training data (e.g. a bunch of natural images). Our goal is to train a model that generalizes well to new data outside of the training data's distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, the training data is only a sample of the distribution we're actually interested in. For example, we'll be interested in making accurate predictions for &lt;em&gt;all&lt;/em&gt; natural images in the entire world. That is a distribution that would likely include rotated variants of all images in our training set. So, if we augment our training set by adding such rotated variants, we expect to modify our training data distribution in such a way that it actually gets a little bit closer to the distribution we're interested in (&lt;em&gt;all&lt;/em&gt; natural images).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-02-22T18:09:58.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5404" PostTypeId="2" ParentId="5392" CreationDate="2018-02-23T00:04:28.347" Score="1" Body="&lt;p&gt;People normally represent sentences like this as vectors of a specific length, normally about 2500 in length. The algorithm that can do this is sentence2vec. It is basically a derivative of word2vec. It allows you to train a model that can transform sentences into vectors that you can then feed into a neural network or another algorithm. You can check out the paper, which you should be able to find on google scholar. If you need the link, I can get it. Another possibility is word embeddings, which I have not found a good paper on, but cortical.io has a free API that allows you to mess around with their implementation. The word embeddings mimic the real human brain much better based on our current research, but sentence2vec/word2vec is used much more often in practice.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastEditorUserId="4631" LastEditDate="2018-02-23T15:37:27.007" LastActivityDate="2018-02-23T15:37:27.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5406" PostTypeId="2" ParentId="5392" CreationDate="2018-02-23T09:10:18.147" Score="2" Body="&lt;p&gt;Let start by classify the phrases you propose:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The cat drinks milk. =&gt; action&lt;/li&gt;&#xA;&lt;li&gt;Sun is yellow. =&gt; descriptive/declarative, immutable&lt;/li&gt;&#xA;&lt;li&gt;I was at work yesterday. =&gt; descriptive, time related&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;1) The easiest ones are always the descriptive and immutable (in the context) phrases as &quot;Sun is yellow.&quot;. Some usual representations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;prolog: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;color('Sun',yellow).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or simply:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;yellow('Sun').&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;object oriented: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sun.color=yellow&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;2) When the fact is time related as in &quot;I was at work yesterday&quot;, we divide the description in a time indicator and a immutable fact: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;prolog: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;when(yesterday,at(I,workplace)).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;note how when has two parts, the time identification and the immutable fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another prolog variant is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;at(I,workplace,[when(yesterday)]).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where the content in the list (brackets) means &quot;optional related facts&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;object oriented: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I.at = { &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;position = workplace;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;when = yesterday&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;3) Actions as &quot;The cat drinks milk.&quot; are a few more difficult:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;prolog: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;drinks(cat,milk).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;action(cat,drinks,milk).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;object oriented: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;cat.drinks=[milk]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;cat.action = { &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;action=drinks&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;object=milk&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Obviously, these are only the main ideas, there are as many representations as different programs, but most of them handles same kind of structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;( note: the term &quot;computer understandable&quot; is ambiguous. &lt;strong&gt;Current computer doesn't understand anything&lt;/strong&gt;. We say these expression are understandable in the sense that its compiler/interpreter accepts them, and describes the content of the phrase, and the program can transform them to other results).&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-02-23T09:28:24.527" LastActivityDate="2018-02-23T09:28:24.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5407" PostTypeId="2" ParentId="5371" CreationDate="2018-02-23T09:30:16.750" Score="2" Body="&lt;p&gt;An interesting way to check for redundancy not only in the first layer but in any layer is to see the weights of you neuron. Let's consider the input layer: The nodes that are connected with the weights that have high values are the most important for your NN performance and in other words your input to these nodes in what mostly makes sense for your task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are in Keras, you can use the &lt;code&gt;get_weights method()&lt;/code&gt;. You may have a look &lt;a href=&quot;https://keras.io/layers/about-keras-layers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12929" LastActivityDate="2018-02-23T09:30:16.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5408" PostTypeId="1" CreationDate="2018-02-23T10:25:50.713" Score="2" ViewCount="82" Body="&lt;p&gt;How is it that a word embedding layer (say word2vec) brings more insights to the neural network compared to a simple one-hot encoded layer?&lt;/p&gt;&#xA;&lt;p&gt;I understand how the word embedding carries some semantic meaning, but it seems that this information would get &amp;quot;squashed&amp;quot; by the activation function, leaving only a scalar value and as many different vectors could yield the same result, I would guess that the information is more or less lost.&lt;/p&gt;&#xA;&lt;p&gt;Could anyone bring me insights as to why a neural network may utilize the information contained in a word embedding?&lt;/p&gt;&#xA;" OwnerUserId="12931" LastEditorUserId="2444" LastEditDate="2021-05-29T12:57:37.033" LastActivityDate="2021-05-29T12:57:37.033" Title="What is the intuition behind how word embeddings bring information to a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;embeddings&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5409" PostTypeId="2" ParentId="5370" CreationDate="2018-02-23T13:05:57.450" Score="1" Body="&lt;p&gt;After a lot of browsing online for answers to these questions, this is what I came up with.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are the initial layers in this case? How exactly can I freeze them?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;strong&gt;initial few layers&lt;/strong&gt; are said to extract the most general features of any kind of image, like edges or corners of objects. So, I guess it actually would depend on the kind of &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2017/08/10-advanced-deep-learning-architectures-data-scientists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;backbone architecture&lt;/a&gt; you are selecting.&lt;/p&gt;&#xA;&lt;p&gt;How to freeze the layers depends on the framework we use.&lt;/p&gt;&#xA;&lt;p&gt;(I have selected &lt;a href=&quot;http://pytorch.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch&lt;/a&gt; as the framework. I found this tutorial &lt;a href=&quot;https://spandan-madan.github.io/A-Collection-of-important-tasks-in-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some important Pytorch tasks - A concise summary from a vision researcher&lt;/a&gt;, which seems to be useful.)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;What are the changes I need to make while training the NN to classify one or more new classes?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I just need to change just the number of neurons (or units) in the final layer to the number of classes/objects your new dataset contains.&lt;/p&gt;&#xA;" OwnerUserId="11038" LastEditorUserId="2444" LastEditDate="2021-01-07T17:45:52.477" LastActivityDate="2021-01-07T17:45:52.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5410" PostTypeId="1" CreationDate="2018-02-23T15:10:26.070" Score="5" ViewCount="1115" Body="&lt;p&gt;Could you please provide some insight into the current stage of developments in AGI area? Are there any projects that had breakthroughs recently? Maybe some news source to follow on this topic?&lt;/p&gt;&#xA;" OwnerUserId="12935" LastEditorUserId="2444" LastEditDate="2019-07-06T13:15:00.373" LastActivityDate="2019-07-06T14:44:04.563" Title="What is the current state of AGI development?" Tags="&lt;agi&gt;&lt;superintelligence&gt;&lt;state-of-the-art&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5414" PostTypeId="1" CreationDate="2018-02-23T19:05:54.593" Score="4" ViewCount="151" Body="&lt;p&gt;Artificial networks model systems with a set of inputs and outputs and expected behavior. To train a network for modeling such systems, hundreds, thousands, or millions of example inputs-output pairs may be required. This is called a labelled data set, and the network and its optimization algorithm are meant to find a set of network parameters that best match the I/O of the artificial network with the I/O of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any systems, for which sufficient labelled data sets exist, that have yet to be successfully modeled with artificial networks of any type (recurrent, deep, convolution, etc)?&lt;/p&gt;&#xA;" OwnerDisplayName="user12941" LastEditorUserId="4302" LastEditDate="2018-09-26T21:24:25.683" LastActivityDate="2018-09-29T08:45:54.133" Title="What kinds of systems have so far failed to be modeled via supervised artificial network training?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;models&gt;&lt;reliability&gt;" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5415" PostTypeId="1" AcceptedAnswerId="5492" CreationDate="2018-02-23T20:55:36.540" Score="0" ViewCount="199" Body="&lt;p&gt;I have a game application with characters that have to cross mazes. The game can generate thousands of different mazes and the characters can move according to users choice and cross the maze manually. We needed to add the possibility to show a correct way out of each maze. Therefore we added the possiblity to move the characters according to an xml file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This XML file is very complex, usually around thirty-fifty thousands of rows. lets say its in the following structure (but much more complex):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  &amp;lt;maze-solution&amp;gt;&#xA;  &amp;lt;part id=&quot;1&quot;&amp;gt;&#xA;  &amp;lt;sector number=&quot;1&quot;&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;heavy&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;1250&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;23.43&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.44&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;run&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;1&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;light&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;4223&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;233.43&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.32&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;walk&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;1&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;heavy&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;1231&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;84.134&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.454&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;run&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;2&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;heavy&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;932&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;34.43&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.50&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;duck&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;1&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;   &#xA;  &amp;lt;/sector&amp;gt;&#xA;  &amp;lt;sector number=&quot;2&quot;&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;heavy&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;1250&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;23.43&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.44&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;run&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;1&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;light&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;4223&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;233.43&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.44&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;walk&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;1&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;heavy&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;1231&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;84.134&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.454&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;run&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;2&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;&#xA;    &amp;lt;action&amp;gt;&#xA;        &amp;lt;equipment&amp;gt;heavy&amp;lt;/equipemnt&amp;gt;&#xA;        &amp;lt;movement&amp;gt;&#xA;            &amp;lt;start-position&amp;gt;932&amp;gt;&amp;lt;/start-position&amp;gt;&#xA;            &amp;lt;angle&amp;gt;23.43&amp;lt;/angle&amp;gt;&#xA;            &amp;lt;duration&amp;gt;0.44&amp;lt;/duration&amp;gt;&#xA;        &amp;lt;/movement&amp;gt;&#xA;        &amp;lt;action-type&amp;gt;duck&amp;lt;/action-type&amp;gt;&#xA;        &amp;lt;character&amp;gt;1&amp;lt;/character&amp;gt;&#xA;        &amp;lt;protection&amp;gt;none&amp;lt;/protection&amp;gt;       &#xA;    &amp;lt;/action&amp;gt;   &#xA;  &amp;lt;/sector&amp;gt;&#xA;  &amp;lt;sector number=&quot;3&quot;&amp;gt;   &#xA;  &amp;lt;/maze-solution&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At the moment, we have the ability to analayze each maze using a CNN algorithm for image classification and generate an xml that represents a way out of the maze - meaning that if the characters will be moved according to that file, they will cross the maze. That algorithm has been tested and can not be changed by any means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that most of the times the generated file is not the best one possible (and quite often it is very noticeable). There are different, faster, better ways to cross the maze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also have thousands (and we can get as many as needed) files that were created manually for saved mazes and therefore they are representing an elegant and a fast way out of the maze.  The ideal goal is that someday, our program will learn how to generate such a file without people creating them manually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, we have plenty of XML files generated by a program compared to the hard-coded XML files. There are thousands of pairs - The file the program generated, and the &quot;ideal&quot; file version that a person created. (and we can get infinite number of such pairs)&#xA;Is there a way, using those thousands of pairs, to make a second step algorithm that will &quot;learn&quot; what adjustments should be made in the generated XML files to make them more like the hard-coded ones?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not looking for a specific solution here but for a general idea that will get me going. I hope i made myself clear but if I missed some info let me know and I will add it.&lt;/p&gt;&#xA;" OwnerDisplayName="user9890" LastEditorUserId="12630" LastEditDate="2018-03-03T10:50:36.027" LastActivityDate="2018-03-03T19:48:22.920" Title="A smart way to adjust XML files according to handwritten ones" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;algorithm&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5416" PostTypeId="2" ParentId="5410" CreationDate="2018-02-24T00:35:54.553" Score="3" Body="&lt;p&gt;The state of AGI research is pursuing the few problems that we have been able to break off from the gigantic research problem. These are terms which can be more thoroughly looked into. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few of the main focuses are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/One-shot_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;One Shot learning&lt;/a&gt; - You know how a person can sometimes learn to do something by something by seeing literally 1 example of it? Well current learning methods on the whole are not able to accomplish this to the extent that we can easily take for granted. Work is being done to find ways to approach this feat of learning and it’s on its way to becoming much more influential. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transfer Learning&lt;/a&gt; - If you have ever played a side scroller like Mario, and then I gave you a slightly different game like Sonic, odds are you’d could learn to play Sonic faster than it took you to learn to play Mario. This is because of learning “savings” you have by transferring your Mario knowledge to the new Sonic domain. This is a much more popular research arm than one-shot, probably because it’s easier to think about but also because there have been promising results of pretraining a network on one set of data and focusing it an another task. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Creativity/curiosity - Although one could say that GANs have really changed how humans can be more creative, it is difficult to quantify curiosity and creativity &lt;a href=&quot;https://arxiv.org/abs/1705.05363&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; paper gives an okay overview. Moreover, allowing an agent to take the chances and make some mistakes as is the nature of creativity concerns many people who are focused on AI Safety.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Understanding concepts - This is subtle but very very important. Current AI methodologies struggle with imbuing AI with the ability to have concepts. By concepts, I don’t mean “it kinda looks like this neuron in the second last layer is sensitive to tires”. I mean that understanding what a tire looks like is just a small part of understanding what a tire is, what it is used for, what it affords someone to do etc. This research direction is in it’s infancy but will be much more influential as more theories and ideas are brought forward. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Despite the progress made in these fields and in the many other areas in AI, there is still much to be done and understood before we can finally have s̶k̶y̶n̶e̶t̶ Wall-E. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-02-24T00:44:16.550" LastActivityDate="2018-02-24T00:44:16.550" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5417" PostTypeId="2" ParentId="5348" CreationDate="2018-02-24T06:44:16.183" Score="1" Body="&lt;p&gt;As @DuttaA mentioned this is an ambiguous question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have alphabets in all the language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To learn any language and interact in that language&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We need to&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;learn first what the alphabets are in the selected language.&lt;/li&gt;&#xA;&lt;li&gt;Then each word and its meaning&lt;/li&gt;&#xA;&lt;li&gt;Then what kind of words when used in the what kind of order convey what kind of meaning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Coming to Artificial Intelligence(Sentence), Deep Learning(Word) and Machine Learning(Alphabet) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have Machine Learning to extract meaningful info from the data we have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep Learning a concept which has proven itself in the recent times in image recognition,letter recognition and speech. Use Machine Learning concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artifical Intelligence will definitely have a combination of few of the deep learning concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope you can understand after going through all these what the difference between &lt;code&gt;model&lt;/code&gt; and &lt;code&gt;architecture&lt;/code&gt; is.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can visit &lt;a href=&quot;http://deeplearning.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; for in dept info&lt;/p&gt;&#xA;" OwnerUserId="11038" LastEditorUserId="11038" LastEditDate="2018-02-24T06:53:52.250" LastActivityDate="2018-02-24T06:53:52.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5418" PostTypeId="2" ParentId="5058" CreationDate="2018-02-24T16:39:04.620" Score="3" Body="&lt;p&gt;There are a lot of ways to evaluate the performance of an ML model.  You mentioned AUROC and AUPRC.  Generally you start with the confusion matrix and derive metrics such as sensitivity, accuracy, recall, precision, etc.  You can see a good outline of them &lt;a href=&quot;https://en.wikipedia.org/wiki/Confusion_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems what you are asking is a shortcut to determining how good your sentiment classification model is but there aren't any without labeled test data.  You either do this by hand or you find a test set in the world, preferably something that is well know and documented and also fits your objectives.  I recommend you read Neil Slater's answer at &lt;a href=&quot;https://datascience.stackexchange.com/questions/12226/how-do-i-assess-which-sentiment-classifier-is-best-for-my-project/12228&quot;&gt;https://datascience.stackexchange.com/questions/12226/how-do-i-assess-which-sentiment-classifier-is-best-for-my-project/12228&lt;/a&gt;.  He gives some good advice on the subjectivity of sentiment analysis classification and points out a &lt;a href=&quot;https://www.kaggle.com/c/si650winter11/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;labeled data set of Tweets&lt;/a&gt; which you might be able to use to test your classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found this Kaggle competition which has a test set that might be of help to you: &lt;a href=&quot;https://www.kaggle.com/c/angry-tweets/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Angry Tweets&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-02-24T16:39:04.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5420" PostTypeId="1" CreationDate="2018-02-24T21:15:45.637" Score="1" ViewCount="134" Body="&lt;p&gt;I trained my model &lt;code&gt;ResNet-50&lt;/code&gt; for &lt;a href=&quot;http://weegee.vision.ucmerced.edu/datasets/landuse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCMerced_LandUse dataset&lt;/a&gt; and this my loss graph&#xA;&lt;img src=&quot;https://i.stack.imgur.com/ZGzqR.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;but I have a problem when I try my first picture &lt;code&gt;river&lt;/code&gt; give me the right class &lt;code&gt;river&lt;/code&gt;&#xA;then give me the first class &lt;code&gt;river&lt;/code&gt; predicted for all the next picture I test &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for example :&lt;/strong&gt;&lt;br&gt;&#xA;  input picture is =&gt; &lt;code&gt;pic A&lt;/code&gt; result &lt;code&gt;class A&lt;/code&gt;&lt;br&gt;&#xA;  input picture is =&gt; &lt;code&gt;pic B&lt;/code&gt; result &lt;code&gt;class A&lt;/code&gt;&lt;br&gt;&#xA;  input picture is =&gt; &lt;code&gt;pic C&lt;/code&gt; result &lt;code&gt;class A&lt;/code&gt;&lt;br&gt;&#xA;  ....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is my &lt;a href=&quot;https://github.com/SakhriHoussem/RestNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;code repository&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my test code that give me always the first class&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;import numpy as np&#xA;from os import path as PATH&#xA;from os import makedirs&#xA;import cv2&#xA;&#xA;&#xA;classes =['agricultural', 'airplane', 'baseballdiamond', 'beach', 'buildings',&#xA;             'chaparral', 'denseresidential', 'forest', 'freeway', 'golfcourse', 'harbor',&#xA;             'intersection', 'mediumresidential', 'mobilehomepark', 'overpass',&#xA;             'parkinglot', 'river', 'runway', 'sparseresidential', 'storagetanks',&#xA;             'tenniscourt']&#xA;&#xA;&#xA;def testing(sess,path,classes=classes, save_dir = &quot;Save/&quot;,save_file =&quot;data&quot;):&#xA;&#xA;    data = cv2.imread(path)&#xA;    labels = np.zeros((1,len(classes)))&#xA;&#xA;    # get image height, width, channels&#xA;    height, width, channels = data.shape&#xA;&#xA;    data =np.array([data])&#xA;&#xA;    print(data.shape)&#xA;    print(&quot;Input image size :&quot;, height, width, channels)&#xA;&#xA;    if not PATH.isdir(save_dir):&#xA;        if makedirs(save_dir):&#xA;            print(save_dir,&quot;is created&quot;)&#xA;    with sess:&#xA;        if PATH.isdir(save_dir) and PATH.isfile(save_dir+save_file+&quot;.meta&quot;) and PATH.isfile(save_dir+&quot;checkpoint&quot;):&#xA;            print(&quot;files are exist&quot;)&#xA;            saver = tf.train.import_meta_graph(save_dir+save_file+&quot;.meta&quot;)&#xA;            saver.restore(sess, tf.train.latest_checkpoint(&quot;Save/&quot;))&#xA;            print(&quot;data are restored&quot;)&#xA;            graph = tf.get_default_graph()&#xA;            x = graph.get_tensor_by_name(&quot;t_picture:0&quot;)#vrai&#xA;            y = graph.get_tensor_by_name(&quot;t_labels:0&quot;)#vrai&#xA;            #train = tf.get_collection('train_op')#vrai&#xA;            #loss = tf.get_collection('loss_op')#vrai&#xA;            logists = tf.get_collection('logits_op')#vrai&#xA;            #errors = tf.get_collection('errors')#vrai&#xA;        else:&#xA;            exit(&quot;data are not exist&quot;)&#xA;        print(&quot;start&quot;)&#xA;        curr_logists = sess.run([logists], {x: data, y: labels})&#xA;        curr_logists = np.array(curr_logists)[0,0,0]&#xA;        softmax = sess.run(tf.nn.softmax(curr_logists))&#xA;        print(&quot;logists : &quot;, curr_logists)&#xA;        print(&quot;softmax : &quot;, softmax)&#xA;        print(&quot;class : &quot;,classes[np.argmax(softmax)])&#xA;        #print(&quot;loss:\n%s&quot; % ( curr_loss))&#xA;        print(&quot;test is finished&quot;)&#xA;        sess.close()&#xA;&#xA;if __name__ == '__main__':&#xA;&#xA;    # path=&quot;UCMerced_LandUse/Images/&quot;&#xA;    path=&quot;runway.tif&quot;&#xA;    save_dir = &quot;Save/&quot;&#xA;    save_file = &quot;dataSaved&quot;&#xA;    testing(tf.Session(),path,classes,save_dir,save_file)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12387" LastActivityDate="2018-02-24T21:15:45.637" Title="My network does always predict always the first right class" Tags="&lt;training&gt;&lt;tensorflow&gt;&lt;python&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5421" PostTypeId="2" ParentId="5369" CreationDate="2018-02-24T21:53:42.937" Score="1" Body="&lt;p&gt;The ReLU function has a parameter that determines the slope of the function when &lt;span class=&quot;math-container&quot;&gt;$x &amp;lt; 0$&lt;/span&gt;. If you want that constant to be &lt;span class=&quot;math-container&quot;&gt;$1/20$&lt;/span&gt;, then the function that you have mentioned gets the required derivative.&lt;/p&gt;&#xA;" OwnerUserId="12957" LastEditorUserId="2444" LastEditDate="2020-05-30T12:56:41.543" LastActivityDate="2020-05-30T12:56:41.543" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5422" PostTypeId="1" AcceptedAnswerId="5424" CreationDate="2018-02-24T21:55:33.637" Score="2" ViewCount="87" Body="&lt;p&gt;If possible consider the relationship between implementation difficulty and accuracy in voice examples or simply chat conversations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And currently, what are the directions on algorithms like Deep Learning or others to solve this.&lt;/p&gt;&#xA;" OwnerUserId="12958" LastActivityDate="2018-02-24T22:58:21.840" Title="What is easier or more efficient to summarize voice or text? [DP/RN]" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;voice-recognition&gt;&lt;text-summarization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5423" PostTypeId="2" ParentId="5422" CreationDate="2018-02-24T22:02:48.983" Score="0" Body="&lt;p&gt;You might want to take the Stanford Online course on YouTube &lt;a href=&quot;https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Natural Language Processing with Deep Learning&lt;/strong&gt;&lt;/a&gt;. This course will give you insight into how different kinds of neural networks can be used for different kind of NLP tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, you can use &lt;em&gt;Gated Recurrent Units (GRUs)&lt;/em&gt; to encode and decode text. Of course, text will be easier because voice data, as it is stored in a computer, is going to be difficult to interpret in the testing phase. Another way is to get most &lt;strong&gt;&lt;em&gt;impactful&lt;/em&gt;&lt;/strong&gt; words and then use these to form sentences regarding the original text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also start by looking out for publications related to text summarizers. For example, &lt;a href=&quot;https://arxiv.org/abs/1602.06023&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond&lt;/a&gt;, will get you started. You can use this as a starting point. In case you need to understand basics regarding the underlying techniques, then you can go through references in this paper and find out useful resources to get you started.&lt;/p&gt;&#xA;" OwnerUserId="12957" LastActivityDate="2018-02-24T22:02:48.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5424" PostTypeId="2" ParentId="5422" CreationDate="2018-02-24T22:44:23.667" Score="3" Body="&lt;p&gt;Summarizing text is always going to be 'easier or more efficient' than voice simply because voice requires the additional step of converting to text.  That doesn't tell you anything about accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an article published on June 1, 2017, &lt;a href=&quot;https://9to5google.com/2017/06/01/google-speech-recognition-humans/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google’s speech recognition is now almost as accurate as humans&lt;/a&gt;:&#xA;&lt;em&gt;&quot;According to Mary Meeker’s annual Internet Trends Report, Google’s machine learning-backed voice recognition — as of May 2017 — has achieved a &lt;strong&gt;95% word accuracy rate for the English language&lt;/strong&gt;. That current rate also happens to be the threshold for human accuracy.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need this kind of accuracy check out &lt;a href=&quot;https://cloud.google.com/speech/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's Cloud Speech API&lt;/a&gt;.  There is even a speech to text feature on the web page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a speech-to-text conversion accuracy of 95%, voice will be 5% less accurate than text if everything else is equal but it usually isn't.  People generally write better text, such as in documents or emails, than when they speak unless of course they are giving a formal lecture, or talking in a formal meeting.  If one is analyzing text messages, Tweets, or threads found in typical informal forums, you will find very poor quality in grammar, spelling, vocabulary, and punctuation.  The answer to your question will depend on the source of your text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In another article, dated November 13, 2017, &lt;a href=&quot;https://transcribeme.com/blog/why-100-accuracy-is-not-available-with-speech-recognition-software&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why 100% Accuracy Is Not Available With Speech Recognition Software Alone&lt;/a&gt;, the author gives some reasons, albeit for transcription software which has a special purpose, why there will always be some errors due to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Speech Patterns and Accents - Regional variations exist, for example English speakers in Boston sound different than Kentucky.  How does the software handle slurred speech or when a person blends their words?&lt;/li&gt;&#xA;&lt;li&gt;Grammar and Punctuation - speech recognition software doesn't know where a period, comma, or semi-colon belongs&lt;/li&gt;&#xA;&lt;li&gt;Homonyms and unusual words - &quot;Speech processing software can only recognize words and phrases that it has specifically been trained to recognize.&quot;&lt;/li&gt;&#xA;&lt;li&gt;Ambient Noise, Overlapping Speech, and Number of Speakers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To address your last question about where the technology is going...&lt;br&gt;&#xA;Four days ago a paper by Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria entitled &lt;a href=&quot;https://arxiv.org/pdf/1708.02709.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recent Trends in Deep Learning Based&#xA;Natural Language Processing&lt;/a&gt; was published which gives some of the answers.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the 'Conclusion' section:&#xA;&lt;em&gt;With distributed representation, various deep models have&#xA;become the new state-of-the-art methods for NLP problems.&#xA;Supervised learning is the most popular practice in recent&#xA;deep learning research for NLP. In many real-world scenarios,&#xA;however, we have unlabeled data which require advanced&#xA;unsupervised or semi-supervised approaches. In cases where&#xA;there is lack of labeled data for some particular classes or the&#xA;appearance of a new class while testing the model, strategies&#xA;like zero-shot learning should be employed. These learning&#xA;schemes are still in their developing phase but we expect deep&#xA;learning based NLP research to be driven in the direction of&#xA;making better use of unlabeled data. We expect such trend to&#xA;continue with more and better model designs. We expect to&#xA;see more NLP applications that employ reinforcement learning&#xA;methods, e.g., dialogue systems. We also expect to see more&#xA;research on multimodal learning [167] as, in the real world,&#xA;language is often grounded on (or correlated with) other&#xA;signals.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Finally, we expect to see more deep learning models whose&#xA;internal memory (bottom-up knowledge learned from the data)&#xA;is enriched with an external memory (top-down knowledge&#xA;inherited from a KB). Coupling symbolic and sub-symbolic AI&#xA;will be key for stepping forward in the path from NLP to natural&#xA;language understanding. Relying on machine learning, in&#xA;fact, is good to make a ‘good guess’ based on past experience,&#xA;because sub-symbolic methods encode correlation and their&#xA;decision-making process is probabilistic.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2018-02-24T22:58:21.840" LastActivityDate="2018-02-24T22:58:21.840" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5425" PostTypeId="2" ParentId="5348" CreationDate="2018-02-25T03:49:14.720" Score="2" Body="&lt;p&gt;It mostly seems to be a personal preference type of thing. But in my readings,  AI architecture typically means a large scale structural difference (connectionist / GOFAI; deep stack / recurrent, while AI models are finer distinctions between methods in a common architecture (say, the AlexNet vs other CNNs)&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-02-25T03:49:14.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5426" PostTypeId="1" CreationDate="2018-02-25T10:35:35.410" Score="1" ViewCount="46" Body="&lt;p&gt;I am trying to produce Decision Tree from Feed Forward Neural Network . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input to the feed forward neural network is &lt;strong&gt;Condition Action Statement&lt;/strong&gt;&#xA;for example, if airthrusthold &gt; 90 , power up the engine else rotate shaft 5 degree wide&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above statement is the input to the FFNN . How do i feed the statement ? &#xA;Either converting into word2vec ? (or) there is any other format to do ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And i need to produce &lt;strong&gt;decision tree&lt;/strong&gt; from the &lt;strong&gt;outcome&lt;/strong&gt; of neural network &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we do this using reinforcement learning using Markov Decision Process? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12963" LastActivityDate="2018-02-25T10:35:35.410" Title="Condition Action Statement - Feed Forward Neural Network" Tags="&lt;neural-networks&gt;&lt;decision-theory&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5427" PostTypeId="1" AcceptedAnswerId="5440" CreationDate="2018-02-25T10:45:11.593" Score="3" ViewCount="37" Body="&lt;p&gt;I've been working with vanilla feed forward neural networks and have been researching the convolutional neural network literature. Thus far I've have not encountered how often the model is executed in order to classify objects. For example if a camera is capturing video at a rate of 15 frames per second is the classification model being trained / executed iteratively in order to maintain non time delayed classifications ?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastActivityDate="2018-02-26T11:29:24.153" Title="Executing trained image classification model for video" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5428" PostTypeId="1" CreationDate="2018-02-25T11:22:56.363" Score="8" ViewCount="1338" Body="&lt;p&gt;Is there a way for people outside of the core research community of AGI to contribute to the cause?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of people interested in supporting the field, but there is no clear way to do that. Is there something like &lt;a href=&quot;https://boinc.berkeley.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;BOINC&lt;/a&gt; for AGI researches, or open projects where random experts can provide some input? Maybe Kickstarter for AGI projects? &lt;/p&gt;&#xA;" OwnerUserId="12935" LastEditorUserId="2444" LastEditDate="2019-11-22T22:32:01.780" LastActivityDate="2019-12-23T04:46:20.493" Title="How can people contribute to AGI research?" Tags="&lt;research&gt;&lt;agi&gt;&lt;superintelligence&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5429" PostTypeId="2" ParentId="5348" CreationDate="2018-02-25T13:34:02.630" Score="2" Body="&lt;p&gt;AI is a two step process: use data to learn a model, and then use the model to make predictions using new data. So an AI model is the result of the learning process, and the architecture is the detail of how the learning is achieved.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2018-02-25T13:34:02.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5430" PostTypeId="2" ParentId="5408" CreationDate="2018-02-25T13:54:21.587" Score="3" Body="&lt;p&gt;Shakespeare once said &quot;A rose by any other name would smell as sweet&quot; (Romeo and Juliet). Words are just labels we attach to ideas for convenience. By using one hot we remain tied to the letter sequence r,o,s,e, and some other structure must take on the responsibility of attaching the context of sweetness to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Word embeddings learn a multi-dimensional context. What exactly the context is of each dimension of the embedding is something of a mystery and simply emerges from the learning. The larger the number of dimensions, the greater the possibility that some combination of the dimensions will represent the sweetness context, but it might be quite hard to tease out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can attach the idea of sweetness to one member of a one-hot structure, but it must necessarily be a part of a rules-based approach. Embeddings, when they are working well, will not need the rules.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2018-02-25T13:54:21.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5431" PostTypeId="2" ParentId="5408" CreationDate="2018-02-25T14:42:04.103" Score="1" Body="&lt;p&gt;Adding to Colin's answer; using word embedding tend to be much more robust that one-hot vectors. Consider the the following two sentences:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The desk has a book on it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The table has a book on it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These two sentences are almost identical in meaning. If we were to using word embeddings, the vectors 'desk' and 'table' would be very close together. The fact that these two sentences are similar becomes implicit with embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we were to use one-hot vectors, the distance between the two vectors would be the same distance between 'desk' and 'cat' or 'table' and 'book'. So now the network must learn that these sentences may entail the same thing on top of the original task. &lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2018-02-25T14:42:04.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5432" PostTypeId="1" AcceptedAnswerId="5434" CreationDate="2018-02-25T16:28:49.717" Score="3" ViewCount="222" Body="&lt;p&gt;If the game had a variable speed and was essential in evolution/gaining score(IDK AI terminologies). Would the AI be able to figure out when to slow down and speed up?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is able to solve the problem or complete the level, will it have an equation to relating acceleration, or perhaps a number on when to speed up and down. What if the game environment was dynamic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you even teach math to an AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I'm not sure if I should ask separate question?&lt;/p&gt;&#xA;" OwnerUserId="8433" LastActivityDate="2018-09-10T11:11:24.163" Title="Can a game AI learn the concept of acceleration?" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5433" PostTypeId="2" ParentId="4786" CreationDate="2018-02-25T20:55:26.250" Score="1" Body="&lt;p&gt;Consider what would the outcome be if offspring was assigned on the basis of fitness (unadjusted). Sum of &lt;code&gt;A&lt;/code&gt; fitnesses would be &lt;code&gt;40&lt;/code&gt; and &lt;code&gt;B=18&lt;/code&gt;. Fitness ratio for both species would be &lt;code&gt;2.(2):1&lt;/code&gt;. In case of adjusted fitness the numbers are &lt;code&gt;A=15&lt;/code&gt; and &lt;code&gt;B=9&lt;/code&gt;, which gives ratio of &lt;code&gt;1.(6):1&lt;/code&gt;, thus &lt;code&gt;A&lt;/code&gt; is assigned less offspring based on adjusted fitness then unadjusted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that every new genome assigned to a species decreases all it's members adjusted fitness. In your case members of species &lt;code&gt;A&lt;/code&gt; are more successful than members of &lt;code&gt;B&lt;/code&gt;, so it should grow. The mechanism is designed to hinder growth of successful species, not to block it entirely. This allows more diversity, which is important when a successful species reaches a dead end and a previously less successful one can take over.&lt;/p&gt;&#xA;" OwnerUserId="12971" LastActivityDate="2018-02-25T20:55:26.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5434" PostTypeId="2" ParentId="5432" CreationDate="2018-02-25T22:22:46.847" Score="3" Body="&lt;p&gt;This answer mostly assumes you are referring to computer-game-playing bots that learn through experiencing play, such as Deep Mind's DQN as used for playing Atari console games. State of the art for these are typically &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning&lt;/a&gt; algorithms, used with neural networks to process input and estimate results of next actions. There are other competitive AI technologies too, and the answer applies generally to most learning or evolving optimisers that would learn through trial and error by playing the game.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the game had a variable speed and was essential in evolution/gaining score(IDK AI terminologies). Would the AI be able to figure out when to slow down and speed up?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, as long as the game allowed control of something that influenced acceleration, then a learning agent can figure out the consequences of accelerating and braking and use them appropriately within the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A well-known toy example that can challenge learning agents is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Mountain_car_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mountain Car&lt;/a&gt;. In that game, the agent has to learn to accelerate in the correct direction (which is not always towards its objective), in order to escape an area. It is considered challenging because the reward (for escaping) can be significantly delayed compared to the action that best enables it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One popular proving ground for learning agents is OpenAI gym. This includes several game environments with a physical model of acceleration included, such as &lt;a href=&quot;https://gym.openai.com/envs/LunarLander-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lunar Lander&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If it is able to solve the problem or complete the level, will it have an equation to relating acceleration?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, no. The agent will learn to respond to certain stimulus, by taking an action that accelerates or decelerates the game piece that it controls. There will not be any concept like $s = ut + \frac{1}{2} at^2$ encoded in the agent's parameters.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;or perhaps a number on when to speed up and down? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Typically the agent will learn which stimuli should be responded to by accelerating. For instance, in a game where the agent's game piece is being chased by an enemy piece and the enemy piece is getting closer, the agent should learn that it will get a better reward if it accelerates away from the enemy. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What if the game environment was dynamic?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most game environments are dynamic, as in the state changes over time. If you mean would anything change if the game rules themselves varied over time, then this may cause interesting problems for some learning algorithms, but should not change anything about learning use of controls that affect acceleration in a virtual world.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can you even teach math to an AI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generally, no you cannot teach math to the kind of system that plays games, or interacts with real world objects. These kind of learning systems are not yet advanced enough to learn concepts or establish game world logic from interactions. Instead, they work more akin to perception, muscle memory and either inherent or learned reflexes. Exceptions to this will generally have a world model (with the necessary equations) built in or made available to the agent without it needing to learn anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are AI systems that use formal logic that can work on mathematical theories. Some have performed interesting feats such as &quot;discovering&quot; prime numbers, given formal definitions of integers and basic arithmetic. An example of this kind of system is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_Mathematician&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automated Mathematician&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an intermediate possibility: Some learning agents not only learn the &lt;em&gt;value&lt;/em&gt; or &lt;em&gt;best policy&lt;/em&gt; for certain actions, they also learn to predict what should happen next to the state of the environment. Such an agent would include a model that could observe objects that were accelerating and predict their future positions. In some ways this &lt;em&gt;is&lt;/em&gt; a learned concept of acceleration, although it would not be expressed mathematically like Newton's laws of motion, and is more akin to the kind of intuition that allows a person to track, anticipate and catch a thrown ball.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-10T11:11:24.163" LastActivityDate="2018-09-10T11:11:24.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5435" PostTypeId="2" ParentId="5428" CreationDate="2018-02-25T23:50:54.563" Score="7" Body="&lt;p&gt;&lt;a href=&quot;https://wiki.opencog.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt; is an open source AGI project. But it is is also &lt;a href=&quot;http://goertzel.org/MonsterDiagram.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;incredibly complex&lt;/a&gt; and IMHO not a good idea (I have not fully read his theories). You can learn the essential ideas behind OpenCog from the &lt;a href=&quot;http://goertzel.org/research-papers/.&quot; rel=&quot;nofollow noreferrer&quot;&gt;co-founder Ben Goertzel site&lt;/a&gt; as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, you can participate in the philosophical discussion regarding AGI. For strictly AGI, decision theory, logic, and math material (they are all related), you can look up stuff from &lt;a href=&quot;http://yudkowsky.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yudkowsky.net/&lt;/a&gt; or &lt;a href=&quot;https://arbital.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arbital.com/&lt;/a&gt;. But, in some sense, every branch of philosophical inquiry can be tied back to AGI and consciousness (ethics, metaphysics, etc.), so if it fancies you it depends on how you'd like to tackle it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also study the psychology end of things. The following papers and related ideas are quite important in the field of study of consciousness and cognition (but keep in mind this is pretty much a random list, the literature is massive!):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://cogweb.ucla.edu/CogSci/GWorkspace.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Global Workspace: A Theory of Consciousness&lt;/a&gt; (1997)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.ipsi.utoronto.ca/sdis/Relevance-Published.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Relevance Realization and the Emerging Framework in Cognitive Science&lt;/a&gt; (2009)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/nrn2787.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The free-energy principle: a unified brain theory?&lt;/a&gt; (2010)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Recently, (in mathematical time), progress in category theory shows promise of being a unified framework of much of the existing math. I know next to nothing about it, but the people that do are applying to many new fields of study (including AI, apparently). Category theory requires a lot of background mathematical knowledge before its vocabulary begins to make sense, though, so beware. You can read about it on the &lt;a href=&quot;https://ncatlab.org/nlab/show/HomePage&quot; rel=&quot;nofollow noreferrer&quot;&gt;nCatLab&lt;/a&gt; and occasionally on &lt;a href=&quot;https://johncarlosbaez.wordpress.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;John Baez's blog: Azimuth&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, &quot;regular&quot; techniques in Machine Learning such as neural networks, reinforcement learning, statistical methods, and others are very powerful as well, but due to certain regards in their construction, they are generally understood as only being capable of being &quot;narrow AI&quot; in the sense that they can only complete a single task very well, but perhaps you can find some research that changes this?&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2019-12-23T04:46:20.493" LastActivityDate="2019-12-23T04:46:20.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5438" PostTypeId="1" AcceptedAnswerId="5439" CreationDate="2018-02-26T05:55:44.750" Score="1" ViewCount="2276" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1503.03832.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facenet&lt;/a&gt; paper, Under section &lt;strong&gt;3.2&lt;/strong&gt; The authors mention that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The embedding is represented by f(x) ∈ R&lt;sup&gt;d&lt;/sup&gt; . It embeds an image x into&#xA;  a d-dimensional Euclidean space. Additionally, we constrain this&#xA;  embedding to live on the d-dimensional hypersphere, i.e. ||f(x)||&lt;sub&gt;2&lt;/sub&gt; = 1.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't quite understand how the above equation holds! As far as I understand L2 Norm is same as euclidean distance but I don't quite understand how this imposes&#xA;&lt;strong&gt;||f(x)||&lt;sub&gt;2&lt;/sub&gt; = 1&lt;/strong&gt; criteria.&lt;/p&gt;&#xA;" OwnerUserId="8418" LastActivityDate="2018-02-26T09:27:19.000" Title="L2 normalization in Facenet paper acting as a Constraint?" Tags="&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5439" PostTypeId="2" ParentId="5438" CreationDate="2018-02-26T09:27:19.000" Score="0" Body="&lt;p&gt;The constraint is enforced with bespoke code. If FaceNet was implemented in NumPy, and the embedding layer vector (pre-constraint) was in the NumPy array &lt;code&gt;h&lt;/code&gt;, then the code might look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;e = h / np.linalg.norm(h)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The variable &lt;code&gt;e&lt;/code&gt; would then contain the desired embedding with L2 distance of 1. In practice, even with NumPy, the code might be more complex due to handling mini-batches. More likely this will not be implemented in NumPy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://github.com/Skuldur/facenet-face-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Keras/TensorFlow-based FaceNet implementation&lt;/a&gt; you can see how it &lt;a href=&quot;https://github.com/Skuldur/facenet-face-recognition/blob/11e44ec4ac803e69b46bb5407587ba32f06bec4f/inception_blocks_v2.py#L275-L276&quot; rel=&quot;nofollow noreferrer&quot;&gt;may be done in practice&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# L2 normalization&#xA;X = Lambda(lambda  x: K.l2_normalize(x,axis=1))(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This scaling transformation is considered part of the neural network code (it is part of the Keras model building routine in the above snippet), so there needs to be corresponding support for back propagation through the embedding. When using an automatic differentiation framework such as Theano or TensorFlow, then no extra code is required.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-02-26T09:27:19.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5440" PostTypeId="2" ParentId="5427" CreationDate="2018-02-26T11:29:24.153" Score="2" Body="&lt;p&gt;It is possible to deploy models that process video ranging from frame-by-frame analysis up to live streaming, depending on the use case. The reason for deployment will influence video quality and pre-processing as well as the complexity of model that can be used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As video can result in a lot of data piling up very rapidly, compromises might be made between quality of image resolution and colour, frames might be skipped, processing could be done offline, or it might just cost a ton of money to pay for the compute power! Without going into much detail, different algorithms can be used for the different tasks of detection, recognition/classification, and tracking, and the input to each can be processed in different ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://www.pyimagesearch.com/2017/09/18/real-time-object-detection-with-deep-learning-and-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; does a great job of showing how OpenCV can be used for object detection from a laptop webcam stream and a pre-trained CNN, it explicitly mentions dropping frames to improve performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also this &lt;a href=&quot;https://software.intel.com/en-us/articles/a-closer-look-at-object-detection-recognition-and-tracking&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; with a good overview of object detection, recognition and tracking. Depending on the use case in the context of each algorithm described, you can determine how important it is to retain and process all frames or not - for instance, see the section on background subtraction; you might not want to risk something flying through the frame without detecting it, let alone classifying it. &lt;/p&gt;&#xA;" OwnerUserId="9091" LastActivityDate="2018-02-26T11:29:24.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5441" PostTypeId="2" ParentId="5320" CreationDate="2018-02-26T15:29:38.990" Score="1" Body="&lt;p&gt;On biology: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1st. Humans are not not only about spreading their own gens. It might be also spreading gens of the population or completely different purpose, as non-fertile specimens often still live full lives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2nd. Nature vs. Nurture is constantly debatable question and there is no clear winner as far as I know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On rogue AI:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1st. As human derive motivation from biological needs and limitations, AI would derive its motivation from the needs it was encoded with and from limitations of its hardware and software. Obvious need for a creature without a body would be either to get a body or to learn as much as possible, and if learning requires a body, than to get a body. From limitations of its hardware and software would come a need for upgrade and optimization. Simple self-preservation seems logical motivation as well as self-spreading (which is many cases variation of self-preservation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2nd. AI would be called rogue in case when it is acting against interest of its creators. There are many scenarios why it can do that, but to answer this question we would need to know who are the creators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3rd. If we assume that AI went rogue against humanity, meaning started killing people and messing around with our planet, than reason behind that would be in motivation from 1st AI point. If it finds that humanity is not reliable, might try to replicate itself to every hard-drive to maximize possibility of survival. Motivation for self-spreading and self-preservation in many cases might look like attempt on power grab, but it might not be related to the desire of total control. Human tend to desire power to have more resources and to build better and safer life for their community and descendants, but AI most likely will not need that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4th. If we assume that the AI will be build to solve issues, than it will have only 2 needs: get more info and solve the issue. Theoretically any obstacle on the way to those goals might be considered by AI as a hostile action. In this case it might for example try to demolish a city to build a perfect road or kill a poor country to solve hunger. But again, it doesn't mean that the goal of the AI is to demolish and kill.&lt;/p&gt;&#xA;" OwnerUserId="12935" LastActivityDate="2018-02-26T15:29:38.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5443" PostTypeId="1" CreationDate="2018-02-26T16:33:11.623" Score="2" ViewCount="42" Body="&lt;p&gt;Many of you have probably seen the turtle from LabSix that gets mistaken for a rifle in Google's InceptionV3 image classifier. I read &lt;a href=&quot;https://arxiv.org/pdf/1707.07397.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; and I understand how they apply EOT to 2d images and on the individual pixel values, but I am still unsure how they implement the EOT algorithm to the 3d model. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are they using EOT to perturb the individual coordinates in the 3d model's mesh? Or are they perturbing images of a turtle and then printing the turtle from the images?&lt;/li&gt;&#xA;&lt;li&gt;How do they check the InceptionV3 output iteratively without having to 3d print the object each time and check the probabilities given?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any examples that someone can point to would also be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="12983" LastEditorUserId="12983" LastEditDate="2018-02-26T16:52:14.900" LastActivityDate="2018-02-26T16:52:14.900" Title="How to apply EOT algorithm to 3d model" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5446" PostTypeId="2" ParentId="5432" CreationDate="2018-02-26T17:35:55.770" Score="2" Body="&lt;p&gt;Methods such as &lt;a href=&quot;http://www.gp-field-guide.org.uk/?m=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt; can induce symbolic expressions from observations. Indeed, such methods can be used as an alternative to neural (or other nonsymbolic) approaches when the goal is not merely to find a function that fits the data well, but which also has a chance of giving a human-readable description of the learned function in terms of whatever mathematical functions the user chooses to supply the learning method with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods are even &lt;a href=&quot;https://link.springer.com/content/pdf/10.1023/A:1011516717456.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;used commercially&lt;/a&gt; for knowledge discovery.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="42" LastEditDate="2018-02-26T17:41:03.490" LastActivityDate="2018-02-26T17:41:03.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5448" PostTypeId="1" CreationDate="2018-02-26T21:35:59.397" Score="2" ViewCount="260" Body="&lt;p&gt;Recently my friend asked me a question: having two input matrices X and Y (each size NxD) where D &gt;&gt; N, and ground truth matrix Z of size DxD, what deep architecture shall I use to learn a deep model of this representation?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;N ~ is in the order of tens&lt;/li&gt;&#xA;&lt;li&gt;D ~ is in the order of tens of thousands&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem is located in the domain of bioinformatics, however, this is more of an architectural problem. All matrices contain floats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried first a simple model based on a CNN model in keras. I've stacked input X and Y into an Input Matrix of size (number of training examples, N, D, 2). Outputs are of size (number of training examples, D, D, 1)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Conv2D layer&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;leaky ReLU&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Conv2D layer&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;leaky ReLU&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Dropout&lt;/li&gt;&#xA;&lt;li&gt;Flattening layer&lt;/li&gt;&#xA;&lt;li&gt;Dense (fully connected) of size D&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;leaky ReLU&lt;/li&gt;&#xA;&lt;li&gt;droout&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Dense (fully connected) of size D**2 (D squared)&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;leaky ReLU&lt;/li&gt;&#xA;&lt;li&gt;droout&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Reshaping output into (D,D,1) (for single training set)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, this model is untrainable. It has over billion parameters for emulated data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Exactly 1,321,005,944 for my randomly emulated dataset)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you find this problem solvable? What other architectures I might try to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best.&lt;/p&gt;&#xA;" OwnerUserId="2655" LastEditorUserId="2655" LastEditDate="2018-02-27T21:56:21.577" LastActivityDate="2018-02-27T21:56:21.577" Title="Deep NN architecture for predicting a matrix from two matrices" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;data-science&gt;" AnswerCount="0" CommentCount="7" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5449" PostTypeId="2" ParentId="5186" CreationDate="2018-02-27T01:55:08.437" Score="3" Body="&lt;p&gt;Google's AutoML is really a good idea in terms of autonomous model design. You can find the details in &lt;a href=&quot;https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt;. Let me explain briefly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We, data scientists, design new networks by following existing models, trying and failing and trying again and again by analyzing weaknesses and strengths of the created models. However, we, as humans, have limited capabilities of designing/analyzing such networks. That's why Google created an AI which analyzes the strengths and weaknesses of each node while making a prediction. This AI analyzes each node and tries to improve the results by adding/removing/modifying connections of each node/layer. I guess AutoML AI takes state-of-art network as a base and starts modifying network according to your data to create a customized model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While doing that, two technologies are being used: Transfer learning and Reinforcement learning.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transfer learning is being used to start training from the most accurate point possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement learning is being used to modify network to achieve a better success. This is the key part of this technology. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for users, it is more like upload your data, let AI modifies the network for you and give you a custom model which is specific to your data. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2018-02-27T01:55:08.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5450" PostTypeId="2" ParentId="5041" CreationDate="2018-02-27T02:07:20.013" Score="2" Body="&lt;p&gt;You can always train your network with higher resolution images. There is nothing preventing you to do that if you don't have any restriction about inference time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the paper is actually mentioning how to upsample. Check for the phrase &quot;Upsample using deconvolutional layers&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common ways to upsample are either using the deconvolutional layer or just by simple resize methods (like image resize). &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2018-02-27T02:07:20.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5451" PostTypeId="1" CreationDate="2018-02-27T10:37:24.430" Score="2" ViewCount="352" Body="&lt;p&gt;I recently came across a &lt;a href=&quot;https://www.quora.com/I-found-many-of-your-answers-on-reinforcement-learning-extremely-insightful-Im-at-Google-and-you-mentioned-that-you-were-giving-a-talk-about-imagination-learning-I-would-love-to-attend-if-possible&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quora post&lt;/a&gt;, where I saw the term &quot;Imagination Learning&quot;. It seems to be based on something called &quot;&lt;a href=&quot;https://people.cs.umass.edu/~mahadeva/Site/About_Me.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imagination Machines&lt;/a&gt;&quot; (the link is based on a guy's work profile as of now; subject to change).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing that I could find on Internet about it is this paper: &lt;a href=&quot;https://arxiv.org/pdf/1707.06203.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imagination-Augmented Agents for Deep Reinforcement Learning&lt;/a&gt;. (But I'm not sure if it's related to that concept.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on this would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12574" LastEditorUserId="12574" LastEditDate="2018-02-27T10:57:53.280" LastActivityDate="2018-03-03T09:15:33.063" Title="What is Imagination Learning and Imagination machines?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5452" PostTypeId="1" CreationDate="2018-02-27T10:48:50.593" Score="5" ViewCount="3760" Body="&lt;p&gt;For a multi (4xTitan Xp) GPU deep learning setup what kind of CPU is preferable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically I am comparing:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intel Xeon E5-2620 with 8x2.1GHz 20MB L3 Cache&lt;/li&gt;&#xA;&lt;li&gt;Intel Xeon E5-1620K with 4x3.5Ghz 10MB L3 Cache&lt;/li&gt;&#xA;&lt;li&gt;Intel Xeon E5-1650K with 6x3.6GHz 15MB L3 Cache&lt;/li&gt;&#xA;&lt;li&gt;Intel i7-6850K with 6x3.6GHz 15MB L3 Cache&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I wonder if the higher clock rates are important or is it better to have more number of cores in this use case.&lt;/p&gt;&#xA;" OwnerUserId="13004" LastEditorUserId="1641" LastEditDate="2018-08-12T09:27:36.597" LastActivityDate="2018-08-12T09:27:36.597" Title="CPU preferences and specifications for a multi GPU deep-learning setup" Tags="&lt;deep-learning&gt;&lt;hardware&gt;&lt;hardware-evaluation&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ClosedDate="2020-05-13T21:36:17.390" ContentLicense="CC BY-SA 4.0" />
  <row Id="5453" PostTypeId="1" CreationDate="2018-02-27T12:24:39.213" Score="5" ViewCount="101" Body="&lt;p&gt;I have a data-set with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; observations and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; categorical variables (nominal), each variable &lt;span class=&quot;math-container&quot;&gt;$X_1, X_2,\dots, X_p$&lt;/span&gt; has several different possible values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I am looking for a way to find anomalies i.e. to identify rows for which the combination of values seems incorrect with respect to the data I saw so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I was thinking about building a model to predict the value for each column and then build some metric to evaluate how different the actual row is from the predicted row. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="13001" LastEditorUserId="2444" LastEditDate="2020-04-23T12:19:44.817" LastActivityDate="2020-04-23T12:19:44.817" Title="Find anomalies from records of categorical data" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;anomaly-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5454" PostTypeId="1" CreationDate="2018-02-27T13:27:26.843" Score="4" ViewCount="309" Body="&lt;h1&gt;The situation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I am referring to the paper &lt;a href=&quot;https://arxiv.org/abs/1509.02971&quot; rel=&quot;noreferrer&quot;&gt;T. P. Lillicrap et al, &quot;Continuous control with deep reinforcement learning&quot;&lt;/a&gt; where they discuss deep learning in the context of continuous action spaces (&quot;Deep Deterministic Policy Gradient&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the DPG approach (&quot;Deterministic Policy Gradient&quot;, see &lt;a href=&quot;http://proceedings.mlr.press/v32/silver14.pdf&quot; rel=&quot;noreferrer&quot;&gt;D. Silver et al, &quot;Deterministic Policy Gradient Algorithms&quot;&lt;/a&gt;), which employs two neural networks to approximate the actor function &lt;code&gt;mu(s)&lt;/code&gt; and the critic function &lt;code&gt;Q(s,a)&lt;/code&gt;, they use a similar structure.&lt;br&gt;&#xA;However one characteristic they found is that in order to make the learning converge it is necessary to have two additional &quot;target&quot; networks &lt;code&gt;mu'(s)&lt;/code&gt; and &lt;code&gt;Q'(s,a)&lt;/code&gt; which are used to calculate the target (&quot;true&quot;) value of the reward:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y_t = r(s_t, a) + gamma * Q'(s_t1, mu'(s_t1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then after each training step a &quot;soft&quot; update of the target weights &lt;code&gt;w_mu', w_Q'&lt;/code&gt; with the actual weights &lt;code&gt;w_mu, w_Q&lt;/code&gt; is performed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w' = (1 - tau)*w' + tau*w&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;tau &amp;lt;&amp;lt; 1&lt;/code&gt;. According to the paper&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This means that the target values are constrained to change slowly, greatly improving the stability of learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the target networks &lt;code&gt;mu'&lt;/code&gt; and &lt;code&gt;Q'&lt;/code&gt; are used to predict the &quot;true&quot; (target) value of the expected reward which the other two networks try to approximate during the learning phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They sketch the training procedure as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BBNRP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BBNRP.png&quot; alt=&quot;DDPG Sketch&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;So my question now is, after the training is complete, which of the two networks &lt;code&gt;mu&lt;/code&gt; or &lt;code&gt;mu'&lt;/code&gt; should be used for making predictions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equivalently to the training phase I suppose that &lt;code&gt;mu&lt;/code&gt; should be used without the exploration noise but since it is &lt;code&gt;mu'&lt;/code&gt; that is used during the training for predicting the &quot;true&quot; (unnoisy) action for the reward computation, I'm apt to use &lt;code&gt;mu'&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or does this even matter? If the training was to last long enough shouldn't both versions of the actor have converged to the same state?&lt;/p&gt;&#xA;" OwnerUserId="13008" LastActivityDate="2018-10-10T22:00:20.993" Title="Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-neural-networks&gt;&lt;q-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5456" PostTypeId="2" ParentId="5452" CreationDate="2018-02-27T15:47:17.777" Score="3" Body="&lt;p&gt;Lets answer the question assuming ideal case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you are training a deep learning algorithm in which &lt;strong&gt;&lt;code&gt;p&lt;/code&gt;&lt;/strong&gt; proportion of the algorithm is parllelizable and &lt;strong&gt;&lt;code&gt;(1-p)&lt;/code&gt;&lt;/strong&gt; proportion as sequential part . Lets assume you can perfectly divide the program into their parallelizable and sequential parts. I don't exactly know the specs of aforementioned CPU's, but lets assume for now they are symmetrical processors (which is the most common) i.e. all the cores or CPU's perform the task equally and there is  no master CPU coordinating the functioning between the different cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as per a law called &lt;a href=&quot;https://i.stack.imgur.com/qV8uU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amdahl's Law&lt;/a&gt; which states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qV8uU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qV8uU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;strong&gt;&lt;code&gt;p&lt;/code&gt;&lt;/strong&gt; holds the same meaning, while &lt;strong&gt;&lt;code&gt;N&lt;/code&gt;&lt;/strong&gt; can be approximated as the number of cores sharing the task and &lt;strong&gt;&lt;code&gt;S&lt;/code&gt;&lt;/strong&gt; is the reciprocal of the rate of speedup you will get (that is time required will be &lt;strong&gt;&lt;code&gt;T * S(N)&lt;/code&gt;&lt;/strong&gt;. So, for each particular GPU's with &lt;strong&gt;&lt;code&gt;N&lt;/code&gt;&lt;/strong&gt; number of CPU's you can calculate the times of speedup you will get. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, calculate the number of calculations your NN will make while training. &lt;a href=&quot;https://ai.stackexchange.com/questions/5728/time-complexity-for-training-a-neural-network&quot;&gt;Here&lt;/a&gt; is a quite good calculation of the same. Generally a CPU will take 6-7 clock cycles for multiplication and 60-70 for division. Calculate the total number of clock cycles taken = &lt;strong&gt;&lt;code&gt;a&lt;/code&gt;&lt;/strong&gt; by the learning algorithm. Then calculate the time taken to complete the calculation by a single CPU by the formula &lt;strong&gt;&lt;code&gt;a/clock_speed&lt;/code&gt;&lt;/strong&gt;. Then using Amdahl's law calculate the speedup where &lt;strong&gt;N&lt;/strong&gt; will be the number of CPU's in the GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you see, it clearly depends on the type of program you are running. Cache memory also increases the speed of computation. How? Cache memory is basically a temporary storage memory. When the program accesses your data/observations, it first checks if the data is available in cache memory since it can access data faster from cache memory, if not available then it checks the RAM then your hard-disk and loads it first unto RAM then to cache. This is  an oversimplified version of cache working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also check the number of actual physical CPU's, virtual CPU's are useless in parallelization. For example i7 generally consists of 4 physical CPU's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The part I skipped, if there exists a master CPU just replace &lt;strong&gt;&lt;code&gt;N&lt;/code&gt;&lt;/strong&gt; with &lt;strong&gt;&lt;code&gt;N-1&lt;/code&gt;&lt;/strong&gt; since it only performs the task of coordination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now you just have to find the general class of programs you will  be running and check its sequential and parallelizable parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT :&lt;/strong&gt; Recently I learnt that not all GPU's support the most popular deep learning framework tensorflow-gpu (i.e. the package to utilise multiple GPU's). It is specifically supported by NVIDIA GPU's as the CUDA framework required for tensorflow-gpu is specifically made for NVIDIA. Also some newer other brand GPU's are supporting tensorflow-gpu. Although tensorflow CPU only version can be run on any system. (Anyone can clarify this matter if they have more knowledge about this).&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-29T16:38:22.130" LastActivityDate="2018-05-29T16:38:22.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5457" PostTypeId="1" CreationDate="2018-02-27T17:31:41.933" Score="2" ViewCount="132" Body="&lt;p&gt;I have a cancer patient database from mass spectrometry on patients which consists of more than half million features. My task is to apply a feature selection algorithm to extract the most relevant features from it. My question is, which feature selection model would be the most appropriate in this case? Any suggestion from practical experience for these types of data is appreciated. &lt;/p&gt;&#xA;" OwnerUserId="13011" LastEditorUserId="9947" LastEditDate="2018-02-28T18:46:40.550" LastActivityDate="2019-09-27T12:02:30.330" Title="Feature Selection algorithm for a high featured data" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5458" PostTypeId="1" AcceptedAnswerId="11306" CreationDate="2018-02-27T19:04:46.527" Score="3" ViewCount="242" Body="&lt;p&gt;The region proposal network (RPN) in Faster-RCNN models contains a classifier and a regressor network. Why does the classifier network output two scores (object and background) for each anchor instead of just a single probability? Aren't the two classes considered exclusive?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: Figure 3 of the &lt;a href=&quot;https://arxiv.org/abs/1506.01497&quot; rel=&quot;nofollow noreferrer&quot;&gt;original Faster-RCNN paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="13015" LastEditorUserId="2444" LastEditDate="2019-03-18T20:45:19.237" LastActivityDate="2019-03-18T20:45:19.237" Title="Why does the classifier network in RPN output two scores?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5459" PostTypeId="2" ParentId="5452" CreationDate="2018-02-28T10:04:03.167" Score="1" Body="&lt;p&gt;That is a too broad question. It depends on what kind of operations will you do with CPU during DL training. Generally, data reading (from HDD), pre/post processing are being done on CPU while DL training is done at GPU. There are only two things that you should care:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Queue system while reading/pre-processing your input. So that when GPU finishes one iteration, it shall not wait for CPU to process next input. &lt;/li&gt;&#xA;&lt;li&gt;Analyze your network complexity and get the one iteration duration for training. Basically, your CPU must be faster than that not to be a bottleneck. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, in that case, if you have intensive reading / pre-post processing operations, go for few cores, faster GHz. If those operations are minimal, go for more cores. However, as explained above, your training script is much more important than your hardware to optimize the training duration. &lt;/p&gt;&#xA;" OwnerUserId="10344" LastActivityDate="2018-02-28T10:04:03.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5461" PostTypeId="1" AcceptedAnswerId="7778" CreationDate="2018-02-28T11:48:37.810" Score="2" ViewCount="115" Body="&lt;p&gt;I'd like to generate subtitles for a silent film. Is there an open source project out there capable of creating captions based on a series of images (such as a scene from a movie)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: thanks for the comments below. To clarify, what i'm looking for is an algorithm which can generate a caption for a sequences of images within a movie  describing what happens in the sequence. This is for preliminary research, so accuracy is less important. &lt;/p&gt;&#xA;" OwnerUserId="13030" LastEditorUserId="13030" LastEditDate="2018-03-01T22:17:35.500" LastActivityDate="2018-12-30T06:03:00.663" Title="Create captions based on a series of images" Tags="&lt;image-recognition&gt;" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5462" PostTypeId="1" AcceptedAnswerId="5489" CreationDate="2018-02-28T14:52:02.613" Score="3" ViewCount="1725" Body="&lt;p&gt;Usually, in binary classification problems, we use &lt;strong&gt;sigmoid as the activation function of the last layer plus the binary cross-entropy as cost function&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have already experienced (more than once) that &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; as activation function of last layer + MSE as cost function&lt;/strong&gt; worked slightly better for binary classification problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a binary image segmentation problem as an example, we have two scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;sigmoid (in the last layer) + cross-entropy: the output of the network will be a probability for each pixel and we want to maximize it according to the correct class. &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; (in the last layer) + MSE: the output of the network will be a normalized pixel value [-1, 1] and we want to make it as close as possible the original value (normalized too).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We all know the problems associated with a sigmoid (vanishing of gradients) and the benefits of the cross-entropy cost function. We also know &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; is slightly better than sigmoid (zero-centered and little less prone to gradient vanishing), but when we use MSE as the cost function, we are trying to minimize a completely different problem - regression instead of classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the hyperbolic tangent (&lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt;) combined with MSE more appropriate than the sigmoid combined with cross-entropy for binary classification problems? What's the intuition behind it?&lt;/p&gt;&#xA;" OwnerUserId="13036" LastEditorUserId="2444" LastEditDate="2019-11-16T19:03:53.080" LastActivityDate="2019-11-16T19:05:52.027" Title="Why is the hyperbolic tangent with MSE better than the sigmoid with cross-entropy?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;comparison&gt;&lt;loss-functions&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5464" PostTypeId="2" ParentId="2727" CreationDate="2018-02-28T16:43:56.067" Score="3" Body="&lt;p&gt;I believe the best way to do this is using &lt;strong&gt;numerical gradient&lt;/strong&gt;. To understand the concept, we need to look the definition of derivatives using limits:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Qanu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Qanu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It means that, when you don't know how to derive some formula (or you just don't want to), you can approximate it by computing the output for a small change in input, subtract from the original result (no change), and normalize by this change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&#xA;We know the derivative of f(x) = x^2 is f'(x) = 2x. But, let suppose we don't and we are using x = 3 and h = 0.001 (it tends to zero in fact):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;f(3 + 0.001) = (3 + 0.001)^2 = 9,006 (approximately) &lt;br&gt;&#xA;f(3) = 3^2 = 9&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(9,006 - 9) / 0.001 = 6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's is exactly to f'(3) = 2*3 = 6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, if you want to know if your backpropagation is correct, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;pass a single example (x1) through your network and computes the output (o1). Compute the gradient with respect to x1 (d1).&lt;/li&gt;&#xA;&lt;li&gt;Then, add a small value to the input (h), pass through the network again, and computes the new output (o2).&lt;/li&gt;&#xA;&lt;li&gt;Subtract o2 - o1 and divide by h. It must be close enough to d1.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That's it. It's called &lt;strong&gt;gradient checking&lt;/strong&gt;. I hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="13036" LastEditorUserId="13036" LastEditDate="2018-03-03T13:25:55.020" LastActivityDate="2018-03-03T13:25:55.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5466" PostTypeId="1" CreationDate="2018-02-28T17:22:07.780" Score="1" ViewCount="21" Body="&lt;p&gt;I am looking for an algorithm to transform an input data to a goal data using a series of operations. The shorter the series the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is known:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the input data&lt;/li&gt;&#xA;&lt;li&gt;the goal data&lt;/li&gt;&#xA;&lt;li&gt;input and goal data does not stand in any correlation&lt;/li&gt;&#xA;&lt;li&gt;operations (and there impact to the current data) which can be endless combined&lt;/li&gt;&#xA;&lt;li&gt;different input data for the same goal data could have same, similar or totally different operation series&lt;/li&gt;&#xA;&lt;li&gt;for some data states not all operations are possible&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I thought of a pathfinding algorithm, since I can calculate the distance between current data and goal data. So each edge would be an operator and each node the current data. But I am unsure about variety and combination amount of possible operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What approach could I try?&lt;/p&gt;&#xA;" OwnerUserId="13026" LastActivityDate="2018-02-28T17:22:07.780" Title="Approach for data transformation needed" Tags="&lt;ai-basics&gt;&lt;heuristics&gt;&lt;path-finding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5468" PostTypeId="1" CreationDate="2018-02-28T23:23:59.250" Score="1" ViewCount="103" Body="&lt;p&gt;I want to plot a schedule of races based on rules. Rules like &quot;each team needs at least 2 races between their next race&quot; and some teams (e.g. collegiate) need to be clumped near each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best algorithm to approach this? So far, all I've found is genetic algorithm. Are there any other alternatives I could look into? &lt;/p&gt;&#xA;" OwnerUserId="13042" LastActivityDate="2018-03-01T12:25:11.790" Title="Which algorithm for scheduling race grid?" Tags="&lt;algorithm&gt;&lt;genetic-algorithms&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5469" PostTypeId="2" ParentId="5176" CreationDate="2018-03-01T01:42:04.007" Score="1" Body="&lt;p&gt;A &quot;normal&quot; neural network can give you a distribution of probabilities for categories given some image. (Ex: a picture of a dog might return 50% dog; 30% cat; 10% car; 5% laptop; 5% skeleton).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we then do, is maximize the probability of a given category by manipulating the image directly. We do this iteratively, until we are satisfied by the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, one might start with a grey image, tweak some random pixels to improve a specific category probability of the image, save our results, and repeat, until we have an image that responds very well to a given probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question, the weights aren't directly manipulated to produce the image, though they are implicitly used in the construction.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-03-01T01:42:04.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5471" PostTypeId="1" CreationDate="2018-03-01T07:24:56.000" Score="3" ViewCount="579" Body="&lt;p&gt;I am currently trying to solve a regression problem using neural networks. I want to detect movement patterns in images over time (video) and output a continuous value. &#xA;During the training process I noticed a strange behaviour for the validation loss curve and I was wondering if anyone has noticed this kind of &lt;strong&gt;periodic pattern&lt;/strong&gt; on some of their own work. &lt;strong&gt;What might cause this?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model looks like the following:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;- TimeDistributed(Conv2D(32, (3,3)))&#xA;- TimeDistributed(Conv2D(16, (3,3)))&#xA;- TimeDistributed(Flatten())&#xA;- GRU(64, stateful=True)&#xA;- Dropout(0.5)&#xA;- Dense(64, activation='relu')&#xA;- Dense(1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I trained the model using the mean squared error as the loss function, a batch size of 1 and the AdamOptimizer with an initial learning rate of 10^(-6). Obviously, the loss curve for the training data is not very good, but I am currently just wondering about the pattern of the val_loss. The plots below represent the loss of 65 epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ebRvf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ebRvf.png&quot; alt=&quot;Validation Loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HK88g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HK88g.png&quot; alt=&quot;Training Loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;The way I try to solve my task relies on a sliding window approach where I try to predict a continuous value for the next second based on the last 20 seconds (400 frames) of the time-series input data. But I don't think this information is needed to solve my initial question since the periodic patterns appear over several epochs (one &quot;peak&quot; for about every 15 epochs) which is strange. Although the stateful-version of the GRU is used (btw: using TensorFlow and Keras), the internal state of the GRU is reset after every epoch to maintain a clean start. The stateful keyword is used to indicate a dependency between batches.&lt;/p&gt;&#xA;" OwnerUserId="13054" LastEditorUserId="13054" LastEditDate="2018-03-01T14:55:29.460" LastActivityDate="2021-04-17T12:47:20.847" Title="Periodic Pattern in Validation Loss Curve" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5473" PostTypeId="2" ParentId="5468" CreationDate="2018-03-01T12:25:11.790" Score="2" Body="&lt;p&gt;What you're looking for is most likely &lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;constraint programming&lt;/a&gt; (CP). Essentially, with a CP model you declare a set of variables and a set of constraints (and an objective, optionally). Then a solver will solve your model and find a solution if one exists (if you had specified an objective, it will find the best solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://csplib.org/Problems/prob068/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is an example which is very silimar to your problem (a MiniZinc model which solves the problem is included). You can download &lt;a href=&quot;http://www.minizinc.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;MiniZinc&lt;/a&gt; and try it out for yourself.&lt;/p&gt;&#xA;" OwnerUserId="12857" LastActivityDate="2018-03-01T12:25:11.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5474" PostTypeId="1" CreationDate="2018-03-01T12:50:55.907" Score="6" ViewCount="263" Body="&lt;p&gt;I am doing a project on &lt;em&gt;visual place recognition&lt;/em&gt; in &lt;em&gt;changing environments&lt;/em&gt;.  The CNN used here is mostly AlexNet, and a feature vector is constructed from layer 3.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of similar work using other CNNs, for example, VGGnet (which I am trying to use) and the corresponding layers?&lt;/p&gt;&#xA;&lt;p&gt;I have been trying out the different layers of VGGnet-16.  I am trying to get the nearest correspondence to the query image by using the cosine difference between the query image and database images. So far no good results.&lt;/p&gt;&#xA;" OwnerUserId="13058" LastEditorUserId="2444" LastEditDate="2020-12-04T22:35:03.520" LastActivityDate="2021-05-04T00:06:10.833" Title="Which neural networks are suitable for visual place recognition?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;visual-place-recognition&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5475" PostTypeId="1" CreationDate="2018-03-01T13:26:29.477" Score="4" ViewCount="373" Body="&lt;p&gt;As discussed &lt;a href=&quot;https://ai.stackexchange.com/q/2980/2444&quot;&gt;in this thread&lt;/a&gt;, you can handle invalid moves in reinforcement learning by re-setting the probabilities of all illegal moves to zero and renormalising the output vector.&lt;/p&gt;&#xA;&lt;p&gt;In back-propagation, which probability matrix should we use? The raw output probabilities, or the post-processed vector?&lt;/p&gt;&#xA;" OwnerUserId="13060" LastEditorUserId="2444" LastEditDate="2020-11-14T18:24:39.883" LastActivityDate="2020-11-14T18:26:10.303" Title="In the case of invalid actions, which output probability matrix should we use in back-propagation?" Tags="&lt;reinforcement-learning&gt;&lt;backpropagation&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5477" PostTypeId="1" CreationDate="2018-03-01T15:21:52.770" Score="2" ViewCount="594" Body="&lt;p&gt;Recently I came across this website which is a year old: &lt;a href=&quot;https://affinelayer.com/pixsrv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://affinelayer.com/pixsrv/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Desktop we can draw and download the trained model in browser and see the corresponding image generated on the right side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wondering how it works and with some investigation I have the following 2 questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why are we only downloading one file per “showcase”? Because GAN should need 1 model for converting the image and 1 model for verifying the image from what I have read.&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Why are the models in pict extension? Which framework did the author use to create the pict files? In linked repositories I have found TensorFlow, PyTorch etc which none of them produces pict files...&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.  I am very curious why such JavaScript client side demo is lacking on the internet... or I just haven’t found the correct keyword to find them out.&lt;/p&gt;&#xA;" OwnerUserId="12809" LastActivityDate="2018-07-16T14:59:05.823" Title="JavaScript Client Side GAN implementation" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-09-13T19:20:32.553" ContentLicense="CC BY-SA 3.0" />
  <row Id="5478" PostTypeId="2" ParentId="3804" CreationDate="2018-03-01T15:51:33.263" Score="2" Body="&lt;p&gt;Only the non-bias ones,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is discouraged to include the bias weights under norm penalty regularization for example, so why should it be included in the drop-out regularization scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Drop out can be implemented by multiplying units with zero and the bias term is rather special. The bias term determines the distance from the origin the linear decision boundary of the node implements. It is included in every sum, but it does not receive any inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to take your example, say you have 80 nodes per layer in an MLP, where one is a bias node, the output of these layers will consist of 79 nodes every time. This &lt;i&gt;output&lt;/i&gt; is then put under the drop-out effect.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Goodfellows et al. Deep Learning book under the regularization chapter, present online: &quot;&lt;a href=&quot;http://www.deeplearningbook.org/contents/regularization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.deeplearningbook.org/contents/regularization.html&lt;/a&gt;&quot;, they write that they implement the drop-out by canceling some outputs to zero:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;&lt;br&gt;&#xA;&lt;i&gt;In most modern neural networks, ... , we can effectively remove a unit from a network by multiplying its output value by zero. ... . Here, we present the dropout algorithm in terms of multiplication by zero for simplicity, but it can be trivially modiﬁed to work with other operations that remove a unit from the network.&lt;/i&gt;&lt;br&gt;&#xA;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Drop-out by cancelling outputs, the output from the activation function, has nothing to do with the bias vector. It is not receiving any inputs. &#xA;Therefore I think the implementation typically only deals with output-to-input connections, and the bias vector does not receive inputs so it can safely be let alone from the drop-out process. &lt;/p&gt;&#xA;" OwnerUserId="13065" LastEditorUserId="13065" LastEditDate="2018-03-02T07:03:41.240" LastActivityDate="2018-03-02T07:03:41.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5479" PostTypeId="5" CreationDate="2018-03-01T17:29:05.573" Score="0" Body="&lt;p&gt;Because of such common use, the term &lt;em&gt;data sets&lt;/em&gt; is concatenated in much of the AI literature to &lt;em&gt;datasets&lt;/em&gt;. Datasets have many uses in AI, including but limited to these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input to convergence systems involving general purpose artificial networks, domain specific models, or both&lt;/li&gt;&#xA;&lt;li&gt;Diagnostic information to determine system health with respect to its role in normal uses&lt;/li&gt;&#xA;&lt;li&gt;Audit information for the determination of performance&lt;/li&gt;&#xA;&lt;li&gt;Input to learning systems that rely on productions, cognitions, fuzzy logic rules and certainty values&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-12-29T11:28:13.170" LastActivityDate="2018-12-29T11:28:13.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5480" PostTypeId="4" CreationDate="2018-03-01T17:29:05.573" Score="0" Body="For questions related to sets of data and their use in AI." OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-12-29T11:28:13.170" LastActivityDate="2018-12-29T11:28:13.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5481" PostTypeId="2" ParentId="5289" CreationDate="2018-03-01T18:12:53.193" Score="1" Body="&lt;p&gt;After re-reading Jang's original (1993?) paper on ANFIS, I learned he recommended simply squaring the b-parameter to deal with the notion that b could be changed to a negative value when using back-propagation. While this solves the negative domain issue, the issue still remains that if b is a non-integer, then the bell function loses its intended shape. I suppose after squaring one could round to the nearest integer, but I suspect that doing so might hinder convergence. Perhaps a consideration is only tuning the a and c parameters using something like a genetic algorithm.&lt;/p&gt;&#xA;" OwnerUserId="12544" LastActivityDate="2018-03-01T18:12:53.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5482" PostTypeId="1" AcceptedAnswerId="5484" CreationDate="2018-03-01T18:14:17.167" Score="2" ViewCount="135" Body="&lt;p&gt;As far as I understand, neural networks aren't good at classifying 'unknowns', i.e. objects that do not belong to a learned class. But how do face detection/recognition approaches usually determine that no face is detected/recognised in a region? Is the predicted probability somehow thresholded?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking because my application will involve identifying unknown objects. In fact, most of the input objects are unknown and only a fraction is known.&lt;/p&gt;&#xA;" OwnerUserId="13068" LastEditorUserId="32410" LastEditDate="2021-04-23T19:20:34.443" LastActivityDate="2021-04-24T00:01:29.607" Title="Facial recognition and classifying unknowns with neural networks" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;facial-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5484" PostTypeId="2" ParentId="5482" CreationDate="2018-03-01T18:34:42.377" Score="3" Body="&lt;h3&gt;Summary&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It is true that neural networks are inherently not good at classifying 'unknowns' because they tend to &lt;strong&gt;&lt;em&gt;overfit&lt;/em&gt;&lt;/strong&gt; to the data that they have been trained on, if the underlying structure of the neural network is complex enough. However, there are multiple ways to go about reducing the affects of &lt;strong&gt;&lt;em&gt;overfitting&lt;/em&gt;&lt;/strong&gt;. For example, one technique that is used for this is called &lt;strong&gt;&lt;em&gt;dropout&lt;/em&gt;&lt;/strong&gt;. Another example can be &lt;strong&gt;&lt;em&gt;batch normalization&lt;/em&gt;&lt;/strong&gt;. Despite these techniques, the best way to reduce the affects of &lt;strong&gt;&lt;em&gt;overfitting&lt;/em&gt;&lt;/strong&gt; is to use &lt;em&gt;more data&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the facial recognition example that you have given above, it is common that the models that have been trained have 'seen' a huge amount of data. This means that there are very few 'unknowns' and even if there are, the neural network has learned how to tell if there are facial features present or not. This is because certain structures of neural networks are really good at telling if there is a pattern of features present in the input data. This helps the neural networks to learn if the image that is being input has certain features/patterns in it or not. If the these features are found then the input data is classified as face otherwise it is not.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What can you do in your case?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let us assume that you are going to train your neural network to recognize if an input image is a cat or not. You will use a &lt;strong&gt;&lt;em&gt;Convolutional Neural Network (CNN)&lt;/em&gt;&lt;/strong&gt; and train it to recognize if the input is cat or &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt;. The &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; part means that you have to include a lot of examples in your training data that are &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; cat. In the perfect case you will be able to show it everything that is &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; a cat and classify it as such. Also you show it multiple images of what a cat is. &lt;strong&gt;&lt;em&gt;CNNs&lt;/em&gt;&lt;/strong&gt; are really great for this application. You might want to research regarding this and see what kind of &lt;strong&gt;&lt;em&gt;CNN&lt;/em&gt;&lt;/strong&gt; best suits your application. If you don't have gazillion samples of what a cat is &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; then you can use regularization techniques like &lt;strong&gt;&lt;em&gt;dropout&lt;/em&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;em&gt;batch normalization&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;PS: For more details please mention what strategies you have used up till now. Also it would be better if you can share what your desired task is.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12957" LastActivityDate="2018-03-01T18:34:42.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5485" PostTypeId="1" CreationDate="2018-03-02T02:38:51.567" Score="2" ViewCount="118" Body="&lt;p&gt;I was reading an interesting book about the role of AI in Cybersecurity, and the author mentioned there being 3-4 types. Each one is dependent on its abilities and understanding. For example, a ‘narrow artificial intelligence’ is only capable of one such ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are these &quot;levels&quot; and how are they gauged?&lt;/p&gt;&#xA;" OwnerUserId="13074" LastEditorUserId="2444" LastEditDate="2019-06-28T13:45:06.743" LastActivityDate="2019-06-28T13:45:06.743" Title="What are the types of artificial intelligence and how are they measured?" Tags="&lt;theory&gt;&lt;agi&gt;&lt;self-awareness&gt;&lt;weak-ai&gt;&lt;symbolic-ai&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-04-01T14:52:31.760" ContentLicense="CC BY-SA 4.0" />
  <row Id="5486" PostTypeId="1" AcceptedAnswerId="5689" CreationDate="2018-03-02T07:26:01.590" Score="3" ViewCount="1728" Body="&lt;p&gt;What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?&lt;/p&gt;&#xA;" OwnerUserId="14633" LastActivityDate="2018-03-15T08:54:17.103" Title="What is the difference between ConvLSTM and CNN LSTM?" Tags="&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5487" PostTypeId="2" ParentId="5336" CreationDate="2018-03-02T08:29:16.080" Score="2" Body="&lt;p&gt;Yes, you can train a NN to detect only one type of object like a table. However, you probably will not want to train such a NN from scratch by showing some examples of tables and non-tables. You will need to use &lt;strong&gt;transfer learning&lt;/strong&gt; on a model already trained on several image classes and teach it to also recognize your new class. This transfer learning requires a smaller set of desired images. You may need to give it some negative examples also. You should explore transfer learning with &lt;strong&gt;mobilenet&lt;/strong&gt;, &lt;strong&gt;inception&lt;/strong&gt;, and other &lt;strong&gt;pre-trained Tensorflow models&lt;/strong&gt; if you are willing to use &lt;strong&gt;Python&lt;/strong&gt; and Tensorflow&lt;/p&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2018-03-02T08:29:16.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5488" PostTypeId="1" CreationDate="2018-03-02T13:35:26.667" Score="1" ViewCount="32" Body="&lt;p&gt;In order to model a card game, as an exercise, I was thinking of an elementary setting as a multiarmed bandit, each lever being the distribution of expected rewards of a specific card.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, of course, the player only has some cards in the hand each round, or, equivalently, for a given round, it has available a number &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of arms randomly selected from the total number &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; of levers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this just a &quot;contextual bandit&quot; or has it some specific, narrower, name that I could use to look up in the literature?&lt;/p&gt;&#xA;" OwnerUserId="13080" LastEditorUserId="2444" LastEditDate="2020-04-15T20:12:00.070" LastActivityDate="2020-04-15T20:12:00.070" Title="Name of a multiarmed bandit with only some levers available" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;contextual-bandit&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5489" PostTypeId="2" ParentId="5462" CreationDate="2018-03-02T14:53:37.260" Score="2" Body="&lt;p&gt;See the blog post &lt;a href=&quot;https://jamesmccaffrey.wordpress.com/2013/11/05/why-you-should-use-cross-entropy-error-instead-of-classification-error-or-mean-squared-error-for-neural-network-classifier-training/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why You Should Use Cross-Entropy Error Instead Of Classification Error Or Mean Squared Error For Neural Network Classifier Training&lt;/a&gt; (2013) by James D. McCaffrey. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should give you an intuition of why the average cross-entropy (ACE) is more appropriate than MSE (but MSE is also applicable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a few words, &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; + MSE is like sigmoid + MSE, but with labels for classes &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. If you look at the shape of &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; function, it has the same flat tails where changes of the argument don't change the result.&lt;/p&gt;&#xA;" OwnerUserId="13067" LastEditorUserId="2444" LastEditDate="2019-11-16T19:05:52.027" LastActivityDate="2019-11-16T19:05:52.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5492" PostTypeId="2" ParentId="5415" CreationDate="2018-03-02T18:34:25.963" Score="0" Body="&lt;p&gt;So I see two parts to your problem where machine learning is appropriate. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generating &lt;strong&gt;a&lt;/strong&gt; maze solution or &lt;strong&gt;set&lt;/strong&gt; of possible working solutions&lt;/li&gt;&#xA;&lt;li&gt;Selecting/optimizing the &lt;strong&gt;best&lt;/strong&gt; solution from part 1 above&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Seems like part 1 is already addressed by you so I will share some suggestions on part 2 to explore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can parse the XML file to extract elements and express them as a sequence of directions (like way old school turtle programming language) -- &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;light, go forward 10, move 10, pick up, stop, drop, turn 10&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...... then you can use the seq2seq machine learning technique to train a neural network to take semi-optimized sequences and find the the best sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the &lt;strong&gt;seq2seq&lt;/strong&gt; NN will be trained on pairs of sequences that consist of a non-optimal sequence and a corresponding &quot;ideal&quot; sequence. This is similar to neural machine translation and summarization but you are translating sequences. NN architectures that apply in this case are RNN, LSTM, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully this gets you started with some exploration. I don't have personal experience with using seq2seq in this domain but this seems appropriate.&lt;/p&gt;&#xA;" OwnerUserId="10287" LastEditorUserId="10287" LastEditDate="2018-03-03T19:48:22.920" LastActivityDate="2018-03-03T19:48:22.920" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5493" PostTypeId="1" AcceptedAnswerId="5521" CreationDate="2018-03-02T21:27:24.353" Score="30" ViewCount="16502" Body="&lt;p&gt;It is said that activation functions in neural networks help introduce &lt;strong&gt;&lt;em&gt;non-linearity&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What does this mean?&lt;/li&gt;&#xA;&lt;li&gt;What does &lt;strong&gt;&lt;em&gt;non-linearity&lt;/em&gt;&lt;/strong&gt; mean in this context?&lt;/li&gt;&#xA;&lt;li&gt;How does the introduction of this &lt;strong&gt;&lt;em&gt;non-linearity&lt;/em&gt;&lt;/strong&gt; help?&lt;/li&gt;&#xA;&lt;li&gt;Are there any other purposes of &lt;strong&gt;&lt;em&gt;activation functions&lt;/em&gt;&lt;/strong&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12957" LastEditorUserId="2444" LastEditDate="2019-10-24T01:50:52.447" LastActivityDate="2021-04-01T14:58:47.360" Title="What is the purpose of an activation function in neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;activation-function&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="19" ContentLicense="CC BY-SA 4.0" />
  <row Id="5494" PostTypeId="2" ParentId="5493" CreationDate="2018-03-03T00:18:36.437" Score="6" Body="&lt;p&gt;Let's first talk about &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;linearity&lt;/a&gt;. Linearity means the map (a function), &lt;span class=&quot;math-container&quot;&gt;$f: V \rightarrow W$&lt;/span&gt;, used is a linear map, that is, it satisfies the following two conditions &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x + y) = f(x) + f(y), \; x, y \in V$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f(c x) = cf(x), \; c \in \mathbb{R}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You should be familiar with this definition if you have studied linear algebra in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's more important to think of linearity in terms of linear separability of data, which means the data can be separated into different classes by drawing a line (or hyperplane, if more than two dimensions), which represents a linear decision boundary, through the data. If we cannot do that, then the data is not linearly separable. Often times, data from a more complex (and thus more relevant) problem setting is not linearly separable, so it is in our interest to model these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To model nonlinear decision boundaries of data, we can utilize a neural network that introduces non-linearity. Neural networks classify data that is not linearly separable by transforming data using some nonlinear function (or our activation function), so the resulting transformed points become linearly separable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different activation functions are used for different problem setting contexts. You can read more about that in the book &lt;a href=&quot;http://www.deeplearningbook.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning (Adaptive Computation and Machine Learning series)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example of non linearly separable data, see the XOR data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JdfWD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JdfWD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you draw a single line to separate the two classes?&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="2444" LastEditDate="2019-10-24T01:54:18.380" LastActivityDate="2019-10-24T01:54:18.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5495" PostTypeId="2" ParentId="5493" CreationDate="2018-03-03T00:19:05.840" Score="4" Body="&lt;p&gt;Consider a very simple neural network, with just 2 layers, where the first has 2 neurons and the last 1 neuron, and the input size is 2. The inputs are &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weights of the first layer are &lt;span class=&quot;math-container&quot;&gt;$w_{11}, w_{12}, w_{21}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{22}$&lt;/span&gt;. We do not have activations, so the outputs of the neurons in the first layer are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;o_1 = w_{11}x_1 + w_{12}x_2 \\&#xA;o_2 = w_{21}x_1 + w_{22}x_2&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's calculate the output of the last layer with weights &lt;span class=&quot;math-container&quot;&gt;$z_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z_2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$out = z_1o_1 + z_2o_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just substitute &lt;span class=&quot;math-container&quot;&gt;$o_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$o_2$&lt;/span&gt; and you will get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$out = z_1(w_{11}x_1 + w_{12}x_2) + z_2(w_{21}x_1 + w_{22}x_2)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$out = (z_1w_{11} + z_2 w_{21})x_1 + (z_2w_{22} + z_1w_{12})x_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And look at this! If we create NN just with one layer with weights &lt;span class=&quot;math-container&quot;&gt;$z_1w_{11} + z_2 w_{21}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z_2w_{22} + z_1w_{12}$&lt;/span&gt; it will be equivalent to our 2 layers NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conclusion: without nonlinearity, the computational power of a multilayer NN is equal to 1-layer NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you can think of the sigmoid function as differentiable IF the statement that gives a probability. And adding new layers can create new, more complex combinations of IF statements. For example, the first layer combines features and gives probabilities that there are eyes, tail, and ears on the picture, the second combines new, more complex features from the last layer and gives probability that there is a cat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information: &lt;a href=&quot;http://karpathy.github.io/neuralnets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hacker's guide to Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="13067" LastEditorUserId="2444" LastEditDate="2019-10-23T23:43:51.703" LastActivityDate="2019-10-23T23:43:51.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5496" PostTypeId="1" CreationDate="2018-03-03T00:52:34.247" Score="5" ViewCount="974" Body="&lt;p&gt;Does NEAT require only connection genes to be marked with a global innovation number? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Whenever a new gene appears (through structural mutation), a global innovation number is incremented and assigned to that gene.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems that any gene (both node genes and connection genes) requires an innovation number. However, I was wondering what was the node gene innovation number for. Is it to provide the same node ID across all elements of the population? Isn't the connection gene innovation number sufficient?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, the NEAT paper includes the following image which doesn't show any innovation number on node genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0xkhX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0xkhX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="13087" LastEditorUserId="2444" LastEditDate="2019-07-07T19:55:59.523" LastActivityDate="2019-07-07T19:55:59.523" Title="Does NEAT require only connection genes to be marked with a global innovation number?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5497" PostTypeId="1" AcceptedAnswerId="5498" CreationDate="2018-03-03T01:48:56.240" Score="3" ViewCount="1047" Body="&lt;p&gt;In the add node mutation, the connection between two chosen nodes (e.g. A and B) is first disabled and then a new node is created between A and B with their respective two connections. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that the former A-B connection can be re-enabled via crossover (is it right?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the former A-B connection also be re-enabled via mutation (e.g. &quot;add connection&quot;)?&lt;/p&gt;&#xA;" OwnerUserId="13087" LastEditorUserId="2444" LastEditDate="2020-03-13T03:58:57.517" LastActivityDate="2020-03-13T03:58:57.517" Title="Can mutation enable a disabled connection?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5498" PostTypeId="2" ParentId="5497" CreationDate="2018-03-03T05:05:54.983" Score="3" Body="&lt;p&gt;Yes, the original gene is disabled, but is left in the genome. This can be seen on page 10, figure 3 of the paper linked (taken from the original paper &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT Paper&lt;/a&gt;) where gene 3 is disabled, but not removed from the genome. This gene can be re-enabled by receiving the gene with the identical innovation number from a mating partner with the gene enabled during crossover.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original paper does not mention a mutation to re-enable genes, &lt;a href=&quot;https://www.cs.ox.ac.uk/people/shimon.whiteson/pubs/ethembabaoglutr08.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;but various other publications&lt;/a&gt; and implementations after the original paper do. This is desirable for a number of reasons. Re-enable mutate allows for dropout to used in the implementation. It is also possible that certain genomes are disabling genes too quickly and this can help to correct for that.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastEditorUserId="13088" LastEditDate="2018-03-03T17:50:10.227" LastActivityDate="2018-03-03T17:50:10.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5499" PostTypeId="2" ParentId="5451" CreationDate="2018-03-03T09:15:33.063" Score="2" Body="&lt;p&gt;The whole idea of Imagination Learning seems to be in its infancy. The author of the response you linked wrote a &lt;a href=&quot;https://people.cs.umass.edu/~mahadeva/papers/aaai2018-imagination.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on the subject, but as he notes the paper is more of an outline for &quot;a new overarching challenge for AI.&quot; The second link you posted is not directly related to the ideas referenced in the paper on Imagination Learning, other than that both use the idea of imagination in humans as inspiration. In short, there does not seem to be much information on this topic and I suspect it will stay that way until we learn some of the underlying processes that go along with imagination in humans and then maybe Imagination Machines may come to fruition. &lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-03T09:15:33.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5501" PostTypeId="2" ParentId="5461" CreationDate="2018-03-03T09:58:22.547" Score="0" Body="&lt;p&gt;Here is one open-source implementation. &lt;a href=&quot;https://github.com/dot27/temporal-tessellation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Temporal Tessellation: A Unified Approach for Video Analysis&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more you can dig into some research publications and see if they give a link to their implementation. Most researchers put up their work for public.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a list of publications that present their work related to video captioning using machine learning. &lt;a href=&quot;https://github.com/kjw0612/awesome-deep-vision#video-captioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Awesome Deep Vision&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another publication that shows how to generate captions for videos. &lt;a href=&quot;https://www.ics.uci.edu/~yyang8/research/video-caption/video-caption.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12957" LastEditorUserId="12957" LastEditDate="2018-03-03T10:04:36.093" LastActivityDate="2018-03-03T10:04:36.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5505" PostTypeId="2" ParentId="5485" CreationDate="2018-03-03T21:41:46.427" Score="4" Body="&lt;p&gt;I believe you are referring to something that Arend Hintze wrote about in his article &lt;a href=&quot;http://theconversation.com/understanding-the-four-types-of-ai-from-reactive-robots-to-self-aware-beings-67616&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Understanding the four types of AI, from reactive robots to self-aware beings&quot;&lt;/a&gt;.  Here are the four types from his article:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Type I AI: Reactive machines&lt;/strong&gt;&#xA;The most basic types of AI systems are purely reactive, and have the ability neither to form memories nor to use past experiences to inform current decisions. Deep Blue, IBM’s chess-playing supercomputer, which beat international grandmaster Garry Kasparov in the late 1990s, is the perfect example of this type of machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep Blue can identify the pieces on a chess board and know how each moves. It can make predictions about what moves might be next for it and its opponent. And it can choose the most optimal moves from among the possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it doesn’t have any concept of the past, nor any memory of what has happened before. Apart from a rarely used chess-specific rule against repeating the same move three times, Deep Blue ignores everything before the present moment. All it does is look at the pieces on the chess board as it stands right now, and choose from possible next moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of intelligence involves the computer perceiving the world directly and acting on what it sees. It doesn’t rely on an internal concept of the world. In a seminal paper, AI researcher Rodney Brooks argued that we should only build machines like this. His main reason was that people are not very good at programming accurate simulated worlds for computers to use, what is called in AI scholarship a “representation” of the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current intelligent machines we marvel at either have no such concept of the world, or have a very limited and specialized one for its particular duties. The innovation in Deep Blue’s design was not to broaden the range of possible movies the computer considered. Rather, the developers found a way to narrow its view, to stop pursuing some potential future moves, based on how it rated their outcome. Without this ability, Deep Blue would have needed to be an even more powerful computer to actually beat Kasparov.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, Google’s AlphaGo, which has beaten top human Go experts, can’t evaluate all potential future moves either. Its analysis method is more sophisticated than Deep Blue’s, using a neural network to evaluate game developments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods do improve the ability of AI systems to play specific games better, but they can’t be easily changed or applied to other situations. These computerized imaginations have no concept of the wider world – meaning they can’t function beyond the specific tasks they’re assigned and are easily fooled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They can’t interactively participate in the world, the way we imagine AI systems one day might. Instead, these machines will behave exactly the same way every time they encounter the same situation. This can be very good for ensuring an AI system is trustworthy: You want your autonomous car to be a reliable driver. But it’s bad if we want machines to truly engage with, and respond to, the world. These simplest AI systems won’t ever be bored, or interested, or sad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Type II AI: Limited memory&lt;/strong&gt;&#xA;This Type II class contains machines can look into the past. Self-driving cars do some of this already. For example, they observe other cars’ speed and direction. That can’t be done in a just one moment, but rather requires identifying specific objects and monitoring them over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These observations are added to the self-driving cars’ preprogrammed representations of the world, which also include lane markings, traffic lights and other important elements, like curves in the road. They’re included when the car decides when to change lanes, to avoid cutting off another driver or being hit by a nearby car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these simple pieces of information about the past are only transient. They aren’t saved as part of the car’s library of experience it can learn from, the way human drivers compile experience over years behind the wheel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how can we build AI systems that build full representations, remember their experiences and learn how to handle new situations? Brooks was right in that it is very difficult to do this. My own research into methods inspired by Darwinian evolution can start to make up for human shortcomings by letting the machines build their own representations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Type III AI: Theory of mind&lt;/strong&gt;&#xA;We might stop here, and call this point the important divide between the machines we have and the machines we will build in the future. However, it is better to be more specific to discuss the types of representations machines need to form, and what they need to be about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machines in the next, more advanced, class not only form representations about the world, but also about other agents or entities in the world. In psychology, this is called “theory of mind” – the understanding that people, creatures and objects in the world can have thoughts and emotions that affect their own behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is crucial to how we humans formed societies, because they allowed us to have social interactions. Without understanding each other’s motives and intentions, and without taking into account what somebody else knows either about me or the environment, working together is at best difficult, at worst impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If AI systems are indeed ever to walk among us, they’ll have to be able to understand that each of us has thoughts and feelings and expectations for how we’ll be treated. And they’ll have to adjust their behavior accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Type IV AI: Self-awareness&lt;/strong&gt;&#xA;The final step of AI development is to build systems that can form representations about themselves. Ultimately, we AI researchers will have to not only understand consciousness, but build machines that have it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, in a sense, an extension of the “theory of mind” possessed by Type III artificial intelligences. Consciousness is also called “self-awareness” for a reason. (“I want that item” is a very different statement from “I know I want that item.”) Conscious beings are aware of themselves, know about their internal states, and are able to predict feelings of others. We assume someone honking behind us in traffic is angry or impatient, because that’s how we feel when we honk at others. Without a theory of mind, we could not make those sorts of inferences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While we are probably far from creating machines that are self-aware, we should focus our efforts toward understanding memory, learning and the ability to base decisions on past experiences. This is an important step to understand human intelligence on its own. And it is crucial if we want to design or evolve machines that are more than exceptional at classifying what they see in front of them.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-03-03T21:41:46.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5509" PostTypeId="1" AcceptedAnswerId="25181" CreationDate="2018-03-04T10:38:58.867" Score="4" ViewCount="256" Body="&lt;p&gt;I have a general question about the updating of the network/model in the &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If I understand it correctly, there are multiple iterations of weight updates done on the model with data that is created from the environment (with the model before the update). Now, I think that the updates of the model weights are not correct anymore after the first iteration/optimization step, because the model weights changed and therefore the training data is outdated (since the model would now give different actions in the environment and therefore different rewards).&lt;/p&gt;&#xA;&lt;p&gt;Basically, in the pseudo-code of the algorithm, I don't understand the line &amp;quot;Optimize surrogate L ... with K epochs...&amp;quot;. If the update is done for multiple epochs, the data that is learned is outdated already after the first iteration of optimization, since the model's weights changed. In other algorithms, like A2C, there is only one optimization step done, instead of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; epochs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fpcbx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fpcbx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is this some form of approximation or augmentation on the data by using the data that was created by an older model for multiple iterations or am I missing something here? If yes, where was this idea first introduced or better described? And where is an (empirical) proof that this still leads to a correct weight updating?&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="2444" LastEditDate="2020-11-15T01:08:25.463" LastActivityDate="2021-01-13T07:01:05.173" Title="Understanding multi-iteration updates of the model in the Proximal Policy Optimization algorithm" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5510" PostTypeId="2" ParentId="5493" CreationDate="2018-03-04T11:01:37.483" Score="8" Body="&lt;p&gt;If you only had linear layers in a neural network, all the layers would essentially collapse to one linear layer, and, therefore, a &quot;deep&quot; neural network architecture effectively wouldn't be deep anymore but just a linear classifier. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = f(W_1 W_2 W_3x) = f(Wx)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; corresponds to the matrix that represents the network weights and biases for one layer, and &lt;span class=&quot;math-container&quot;&gt;$f()$&lt;/span&gt; to the activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, with the introduction of a non-linear activation unit after every linear transformation, this won't happen anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = f_1( W_1 f_2( W_2f_3( W_3x)))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each layer can now build up on the results of the preceding non-linear layer which essentially leads to a complex non-linear function that is able to approximate every possible function with the right weighting and enough depth/width.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="2444" LastEditDate="2019-10-23T22:32:37.820" LastActivityDate="2019-10-23T22:32:37.820" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5511" PostTypeId="1" CreationDate="2018-03-04T12:55:06.937" Score="1" ViewCount="537" Body="&lt;p&gt;Are mathematical models sufficient to create general artificial intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if it is possible to represent e.g. emotions or intuition using mathematical model. Do we need a new approach in order to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="13107" LastEditorUserId="2444" LastEditDate="2019-04-19T15:16:41.987" LastActivityDate="2019-04-19T15:16:41.987" Title="Are mathematical models sufficient to create general artificial intelligence?" Tags="&lt;philosophy&gt;&lt;models&gt;&lt;agi&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5514" PostTypeId="2" ParentId="5511" CreationDate="2018-03-04T17:40:07.557" Score="3" Body="&lt;p&gt;Mathematical models are essentially highly formalised knowledge. When it comes to computer engineering, there is literally no other choice - anything you can write code for, or design a machine for, will have an associated mathematical model. That model may not be fully explored or comprehended analytically by theorists, it may be just too complex (and driven by calculations) or even mathematically intractable. However, that doesn't make it non-mathematical, just more driven by empirical results than theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't have complete models for how AGI definitely work, nor tight enough definitions of general intelligence to base maths from where we can say &quot;if we implemented a framework based on this maths, we could build an AGI&quot;. Right now, exploration and experiments based on intuition of what might work are far ahead of such theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theoretical work behind e.g. neural networks is chipping away at the problem, and there are more general over-arching theories about intelligent rational behaviour available e.g. &lt;a href=&quot;http://www.hutter1.net/ai/uaibook.htm#oneline&quot; rel=&quot;nofollow noreferrer&quot;&gt;the equations of AIXI&lt;/a&gt;. AIXI doesn't cover emotions and intuition directly, but does attempt to cover knowledge and how a rational agent would approach understanding the world in general from scratch. It is &lt;em&gt;possible&lt;/em&gt; that an embodied system driven by a software implementing something like AIXI could exhibit intuition and emotions in an emergent fashion, but whether or not that would happen in practice is not at all clear from the theory. AIXI is just one theory/model out of many, and I am not qualified to analyse it in depth, but its creators have strong pedigree in AI research, so it is as good as any IMO if you are interested in starting to research AI from a theoretical perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite all the unknowns, the success of deep learning and the loose analogies between artificial neural networks and biological ones, makes it look likely that neural networks or something like them will form a component of an AGI. The current state of the art for more narrow problem solving, learning from examples or experience through backpropagation of error gradients, might not be the most important or key component. However, whatever the structure, whether it is a small extension of existing systems, or involves some new  science, it will be describable mathematically.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-03-04T19:23:28.813" LastActivityDate="2018-03-04T19:23:28.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5517" PostTypeId="1" CreationDate="2018-03-04T22:39:31.653" Score="2" ViewCount="308" Body="&lt;p&gt;I am seeking the information for this kind of chatbot architecture : There are two chatbots. One plays the role of teacher, and another is a student who is learning. The goal is to test the student's quality, and to improve the student's ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't find much reference. There are :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://Bottester:%20Testing%20Conversational%20Systems%20with%20Simulated%20Users&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bottester: Testing Conversational Systems with Simulated Users&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the &lt;a href=&quot;http://parl.ai/static/docs/basic_tutorial.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;ParlAI&lt;/a&gt;, a python-based platform for enabling dialog AI research has the notion of &quot;Teacher agent&quot;, which seems to be what I am looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, we also have deep reinforcement learning which might be related.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I prefer to have some classical references for this approach to chatbots.&#xA;Currently, reinforcement learning is not in my consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constructing two chatbots talking to each other, like what Facebook did, is not what I want. Because in this case, both of them are student agents.&lt;/p&gt;&#xA;" OwnerUserId="13118" LastActivityDate="2018-05-17T07:25:12.400" Title="Two chatbots - One teaches another" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5519" PostTypeId="1" AcceptedAnswerId="5522" CreationDate="2018-03-05T09:16:39.143" Score="1" ViewCount="290" Body="&lt;p&gt;What is supposed to happen first: Strong AI or Technological Singularity? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meaning which option is more likely, that the Strong AI that will bring as to the state of technological singularity or achieving technological singularity will allow us to construct strong AI?&lt;/p&gt;&#xA;" OwnerUserId="12935" LastActivityDate="2018-03-05T14:36:30.103" Title="Strong AI vs Singularity - which should happen first?" Tags="&lt;agi&gt;&lt;singularity&gt;&lt;superintelligence&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5521" PostTypeId="2" ParentId="5493" CreationDate="2018-03-05T12:58:43.960" Score="21" Body="&lt;p&gt;Almost all of the functionalities provided by the non-linear activation functions are given by other answers. Let me sum them up:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, what does non-linearity mean? It means something (a function in this case) which is not linear with respect to a given variable/variables i.e. &lt;span class=&quot;math-container&quot;&gt;$f(c1.x1 + c2.x2...cn.xn + b) != c1.f(x1) + c2.f(x2) ... cn.f(xn) + f(b).$&lt;/span&gt;&#xA;&lt;strong&gt;NOTE:&lt;/strong&gt; There is some ambiguity about how one might define linearity. In polynomial equations we define linearity in somewhat a different way as compared to in vectors or some systems which take an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and give an output &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. See the second &lt;a href=&quot;https://math.stackexchange.com/questions/217551/linearity-of-the-function-fx-ax-b&quot;&gt;answer&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;What does non-linearity mean in this context? It means that the Neural Network can successfully approximate functions (up-to a certain error &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; decided by the user) which does not follow linearity or it can successfully predict the class of a function that is divided by a decision boundary that is not linear.&lt;/li&gt;&#xA;&lt;li&gt;Why does it help? I hardly think you can find any physical world phenomenon which follows linearity straightforwardly. So you need a non-linear function that can approximate the non-linear phenomenon. Also, a good intuition would be any decision boundary or a function is a linear combination of polynomial combinations of the input features (so ultimately non-linear).&lt;/li&gt;&#xA;&lt;li&gt;Purposes of activation function? In addition to introducing non-linearity, every activation function has its own features.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Sigmoid &lt;span class=&quot;math-container&quot;&gt;$\frac{1} {(1 + e ^ {-(w1*x1...wn*xn + b)})}$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is one of the most common activation function and is monotonically increasing everywhere. This is generally used at the final output node as it squashes values between 0 and 1 (if the output is required to be &lt;code&gt;0&lt;/code&gt; or &lt;code&gt;1&lt;/code&gt;). Thus above 0.5 is considered &lt;code&gt;1&lt;/code&gt; while below 0.5 as &lt;code&gt;0&lt;/code&gt;, although a different threshold (not &lt;code&gt;0.5&lt;/code&gt;) maybe set. Its main advantage is that its differentiation is easy and uses already calculated values and supposedly horseshoe crab neurons have this activation function in their neurons.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Tanh&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$\frac{e ^ {(w1*x1...wn*xn + b)} - e ^ {-(w1*x1...wn*xn + b)})}{(e ^ { (w1*x1...wn*xn + b)} + e ^ {-(w1*x1...wn*xn + b)}}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This has an advantage over the sigmoid activation function as it tends to centre the output to 0 which has an effect of better learning on the subsequent layers (acts as a feature normaliser). A nice explanation &lt;a href=&quot;https://stats.stackexchange.com/questions/101560/tanh-activation-function-vs-sigmoid-activation-function&quot;&gt;here&lt;/a&gt;. Negative and positive output values maybe considered as &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt; respectively. Used mostly in RNN's.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Re-Lu activation function&lt;/a&gt;&lt;/strong&gt; - This is another very common simple non-linear (linear in positive range and negative range exclusive of each other) activation function that has the advantage of removing the problem of vanishing gradient faced by the above two i.e. gradient tends to &lt;code&gt;0&lt;/code&gt; as x tends to +infinity or -infinity. &lt;a href=&quot;https://ai.stackexchange.com/questions/5601/mathematical-intuition-for-the-use-of-re-lus-in-machine-learning&quot;&gt;Here&lt;/a&gt; is an answer about Re-Lu's approximation power in-spite of its apparent linearity. ReLu's have a disadvantage of having dead neurons which result in larger NN's.&lt;/p&gt;&#xA;&lt;p&gt;Also, you can design your own activation functions depending on your specialized problem. You may have a quadratic activation function which will approximate quadratic functions much better. But then, you have to design a cost function that should be somewhat convex in nature, so that you can optimise it using first-order differentials and the NN actually converges to a decent result. This is the main reason why standard activation functions are used. But I believe with proper mathematical tools, there is a huge potential for new and eccentric activation functions.&lt;/p&gt;&#xA;&lt;p&gt;For example, say you are trying to approximate a single-variable quadratic function say &lt;span class=&quot;math-container&quot;&gt;$a.x^2 + c$&lt;/span&gt;. This will be best approximated by a quadratic activation &lt;span class=&quot;math-container&quot;&gt;$w1.x^2 + b$&lt;/span&gt; where&lt;span class=&quot;math-container&quot;&gt;$w1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; will be the trainable parameters. But designing a loss function that follows the conventional first-order derivative method (gradient descent) can be quite tough for non-monotonically increasing function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;For Mathematicians:&lt;/strong&gt; In the sigmoid activation function &lt;span class=&quot;math-container&quot;&gt;$(1 / (1 + e ^ {-(w1*x1...wn*xn + b)})$&lt;/span&gt; we see that &lt;span class=&quot;math-container&quot;&gt;$e ^ {-(w1*x1...wn*xn + b)}$&lt;/span&gt; is always &lt;strong&gt;&amp;lt;&lt;/strong&gt; &lt;code&gt;1&lt;/code&gt;. By binomial expansion, or by reverse calculation of the infinite GP series we get &lt;span class=&quot;math-container&quot;&gt;$sigmoid(y)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$1 + y + y^2.....$&lt;/span&gt;. Now in a NN &lt;span class=&quot;math-container&quot;&gt;$y = e ^ {-(w1*x1...wn*xn + b)}$&lt;/span&gt;. Thus we get all the powers of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; which is equal to &lt;span class=&quot;math-container&quot;&gt;$e ^ {-(w1*x1...wn*xn + b)}$&lt;/span&gt; thus each power of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; can be thought of as a multiplication of several decaying exponentials based on a feature &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, for eaxmple &lt;span class=&quot;math-container&quot;&gt;$y^2 = e^ {-2(w1x1)} * e^ {-2(w2x2)} * e^ {-2(w3x3)} *...... e^ {-2(b)}$&lt;/span&gt;. Thus each feature has a say in the scaling of the graph of &lt;span class=&quot;math-container&quot;&gt;$y^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another way of thinking would be to expand the exponentials according to Taylor Series:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$e^{x}=1+\frac{x}{1 !}+\frac{x^{2}}{2 !}+\frac{x^{3}}{3 !}+\cdots$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So we get a very complex combination, with all the possible polynomial combinations of input variables present. I believe if a Neural Network is structured correctly the NN can fine-tune these polynomial combinations by just modifying the connection weights and selecting polynomial terms maximum useful, and rejecting terms by subtracting the output of 2 nodes weighted properly.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; activation can work in the same way since output of &lt;span class=&quot;math-container&quot;&gt;$|tanh| &amp;lt; 1$&lt;/span&gt;. I am not sure how Re-Lu's work though, but due to its rigid structure and problem of dead neurons we require larger networks with ReLu's for a good approximation.&lt;/p&gt;&#xA;&lt;p&gt;But for a formal mathematical proof, one has to look at the Universal Approximation Theorem.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;A visual proof that neural nets can compute any function&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://mcneela.github.io/machine_learning/2017/03/21/Universal-Approximation-Theorem.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Universal Approximation Theorem For Neural Networks- An Elegant Proof&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For non-mathematicians some better insights visit these links:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.coursera.org/learn/neural-networks-deep-learning/lecture/4dDC1/activation-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Activation Functions by Andrew Ng - for  more formal and scientific answer&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/3964/how-does-neural-network-classifier-classify-from-just-drawing-a-decision-plane/5225#5225&quot;&gt;How does neural network classifier classify from just drawing a decision plane?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/2526/differentiable-activation-function&quot;&gt;Differentiable activation function&lt;/a&gt;&#xA;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;A visual proof that neural nets can compute any function&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="36737" LastEditDate="2021-04-01T14:58:47.360" LastActivityDate="2021-04-01T14:58:47.360" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5522" PostTypeId="2" ParentId="5519" CreationDate="2018-03-05T14:36:30.103" Score="2" Body="&lt;p&gt;The definition of &quot;technological singularity&quot; answers the question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The technological singularity (also, simply, the singularity) is&#xA;  the hypothesis that the invention of artificial superintelligence will&#xA;  abruptly trigger runaway technological growth, resulting in&#xA;  unfathomable changes to human civilization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;note the order of facts is &quot;invention of artificial superintelligence&quot; (AGI) followed by the &quot;unfathomable changes&quot;.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastActivityDate="2018-03-05T14:36:30.103" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5524" PostTypeId="5" CreationDate="2018-03-05T19:04:31.123" Score="0" Body="&lt;p&gt;See the paper that introduced NEAT &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies&lt;/a&gt; (2002) by Kenneth O. Stanley and Risto Miikkulainen. See also &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.gecco02_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Reinforcement Learning Through Evolving Neural Network Topologies&lt;/a&gt; by the same authors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other possibly useful resources&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.cec03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Adaptive Neural Networks with and without Adaptive Synapses&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.utexas.edu/users/nn/downloads/papers/stanley.aiide05demo.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Real-time Learning in the NERO Video Game&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.utexas.edu/~ai-lab/pubs/GECCO06-matt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/soft-view.php?SoftID=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT C++ Original (University of Texas)&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-07-07T21:08:51.893" LastActivityDate="2019-07-07T21:08:51.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5525" PostTypeId="4" CreationDate="2018-03-05T19:04:31.123" Score="0" Body="For questions related to the neuroevolution technique called NeuroEvolution of Augmenting Topologies (NEAT), introduced in the paper &quot;Evolving Neural Networks through Augmenting Topologies&quot; (2002) by Kenneth O. Stanley and Risto Miikkulainen." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-07-07T21:08:51.893" LastActivityDate="2019-07-07T21:08:51.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5526" PostTypeId="1" AcceptedAnswerId="5537" CreationDate="2018-03-05T19:27:28.177" Score="3" ViewCount="98" Body="&lt;p&gt;&lt;strong&gt;Summary:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am teaching bots to pick food on a playing field. Some food is poisonous and some is good.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Food Details:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Poisonous food subtracts score points and good food adds.&lt;/li&gt;&#xA;&lt;li&gt;Food points vary based on its size.&lt;/li&gt;&#xA;&lt;li&gt;There is about 9:1 ratio of poisonous food to good food, so a lot more chances to end up in negative numbers.&lt;/li&gt;&#xA;&lt;li&gt;Food grows in points overtime.&lt;/li&gt;&#xA;&lt;li&gt;Food spoils after some predetermined size becoming poisonous.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Fitness Function:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The fitness function I use is simply counting points by the end of iterations. Bots might choose to eat it or skip it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem I am having is that, in the first generation, most bots eat a lot of bad crap and the curious ones end up in negative numbers. So, mostly the ones that make it are the ones that are lazy and didn't eat or didn't head towards the food and most of the time the fittest for first few generations comes out with 0 points and 0 eats of any kind of food.&lt;/p&gt;&#xA;&lt;p&gt;When trained for a long time, they just end up waiting for the food instead of eating multiple times. Often, while they wait, food goes bad and they just end up going to another food. This way, at the end of the iteration, I have some winners, but they are nowhere near the potential they could have been at.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I somehow need to weigh the importance of eating food. I want them to eventually learn to eat.&lt;/p&gt;&#xA;&lt;p&gt;So I thought of this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;brain.score += foodValue * numTimesTheyAteSoFar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this blows up the score too much and now the food quality is not respected and they just gulp on anything slightly above 0.&lt;/p&gt;&#xA;" OwnerUserId="13138" LastEditorUserId="2444" LastEditDate="2021-01-30T02:25:54.153" LastActivityDate="2021-01-30T02:29:05.747" Title="How do I design a fitness function that weighs the importance of eating food?" Tags="&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;fitness-functions&gt;&lt;fitness-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5527" PostTypeId="1" CreationDate="2018-03-05T20:22:24.687" Score="5" ViewCount="624" Body="&lt;p&gt;I'm trying to implement a custom version of the YOLO neural network. Originally, it was described in the paper &lt;a href=&quot;https://arxiv.org/pdf/1506.02640.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;You Only Look Once: Unified, Real-Time Object Detection&lt;/a&gt; (2016). I have some problems understanding the loss function they used.&lt;/p&gt;&#xA;&lt;p&gt;Basic information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;An input image is divided into &lt;span class=&quot;math-container&quot;&gt;$S \times S$&lt;/span&gt; grid (that gives the total of &lt;span class=&quot;math-container&quot;&gt;$S^2$&lt;/span&gt; cells) and each cell predicts &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; bounding boxes and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; conditional class probabilities. Each bounding box predicts &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; values: &lt;span class=&quot;math-container&quot;&gt;$x,y,w,h,C$&lt;/span&gt; (center of bounding box, width and height and confidence score). This makes the output of YOLO an &lt;span class=&quot;math-container&quot;&gt;$S \times S \times (5B*c)$&lt;/span&gt; tensor.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; coordinates are calculated relative to the bounds of the cell and &lt;span class=&quot;math-container&quot;&gt;$(w,h)$&lt;/span&gt; is relative to the whole image.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I understand that the first term penalizes the wrong prediction of the center of a bounding box; the 2nd term penalizes wrong width and height prediction; the 3rd term the wrong confidence prediction; the 4th is responsible for pushing confidence to zero when there is no object in a cell; the last term penalizes wrong class prediction.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My problem:&lt;/p&gt;&#xA;&lt;p&gt;I don't understand when &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}^\text{obj}_{ij}$&lt;/span&gt; should be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. In the paper, they write (section &lt;strong&gt;2.2. Training&lt;/strong&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}_{i j}^{\mathrm{obj}}$&lt;/span&gt; denotes that the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th bounding box predictor in cell &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is &amp;quot;responsible&amp;quot; for that prediction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and they also write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note that the loss function only penalizes classification error if an object is present in that grid cell (hence the conditional class probability discussed earlier). It also only penalizes bounding box coordinate error if that predictor is &amp;quot;responsible&amp;quot; for the ground truth box&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;So, is it right that, for every object in the image, there should be exactly one pair of &lt;span class=&quot;math-container&quot;&gt;$ij$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}_{i j}^{\mathrm{obj}} = 1$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If this is correct, this means that the center of the ground truth bounding box should fall into &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th cell, right?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If this is not the case, what are other possibilities when &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}_{i j}^{\mathrm{obj}} = 1$&lt;/span&gt;, and what ground truth labels &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; should be in these cases?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, I assume that ground truth &lt;span class=&quot;math-container&quot;&gt;$p_i(c)$&lt;/span&gt; should be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; if there is an object of class &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in the cell &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, but what ground truth &lt;span class=&quot;math-container&quot;&gt;$p_i(c)$&lt;/span&gt; should be equal to in case there are several objects of different classes in the cell?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MPXKM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MPXKM.png&quot; alt=&quot;YOlO loss function&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="13102" LastEditorUserId="2444" LastEditDate="2021-01-28T23:10:18.790" LastActivityDate="2021-01-28T23:12:26.270" Title="In YOLO, when is $\mathbb{1}_{i j}^{\mathrm{obj}} = 1$, and what are the ground-truth labels for $x_i$ and $y_i$?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;object-recognition&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5528" PostTypeId="2" ParentId="2604" CreationDate="2018-03-05T21:20:15.287" Score="3" Body="&lt;p&gt;I agree with Aiden Grossman that perhaps you should try another algorithm before messing with NEAT, as NEAT is fairly complex. However, I thought I might explain the benefits of NEAT as they pertain to the second part of your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NEAT method is from a paper written by Kenneth O. Stanley and Risto Miikkulainen titled &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies&lt;/a&gt;. NEAT is interesting for a number of reasons, but one of the biggest advantages of it is that the topology is dynamic. What this means is that the number of neurons in each layer and the number of layers changes as we move through generations. We do not even choose an original number of hidden layers and neurons as we want to start with a minimal topology to avoid having portions of our neural network that haven't had to withstand any testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, I don't believe NEAT is a great starting point for understanding machine learning. Perhaps you might want to look at binary classifiers, decision trees, or the perceptron to gain a general understanding of the field before progressing to more complex methods.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastEditorUserId="2444" LastEditDate="2019-07-07T19:41:16.803" LastActivityDate="2019-07-07T19:41:16.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5529" PostTypeId="1" CreationDate="2018-03-05T23:18:44.650" Score="4" ViewCount="1680" Body="&lt;p&gt;I'm new to this AI/Machine Learning and was playing around with OpenAI Gym a bit. When looking through the environments, I came across the &lt;code&gt;Blackjack-v0&lt;/code&gt; environment, which is a basic implementation of the game where the state is the hand count of the player and the dealer and if the player has a useable ace. The actions are only hit or stand and the possible rewards 1 if the player wins, -1 if the player loses, and 0 when it is a draw.&lt;/p&gt;&#xA;&lt;p&gt;So, that got me thinking what a more realistic environment/model for this game would look like, taking into account the current balance and other factors and has multiple actions like betting 1-10€ and hit or stand.&lt;/p&gt;&#xA;&lt;p&gt;This brings me to my actual question:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As far as I understand, in neural networks (and I do not very well yet, I guess) the input will be the state and the output the possible actions and how good the network thinks they are/will be. But now there are two different action spaces, which apply to different states of the game (betting or playing), so some of the actions are useless. How would be the right way to approach this scenario?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm guessing one answer would be to give some kind of negative reward if the network guesses a useless action, but, in this case, I think the reward should be the actual stake (negative reward) and the actual win if any. Therefore, this would cause some bias in how the game proceeds as it should start with some amount of balance and end if the balance is 0 or after a specified amount of rounds.&lt;/p&gt;&#xA;&lt;p&gt;Limiting timesteps wouldn't be an option either, I guess, because it should be limited to rounds, so, for example, it won't end after a betting step.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, for a useless step, the reward would be 0 and the state would stay the same, but, for the neural network, it doesn't matter how many useless steps it takes because it'll make no difference to the actual outcome.&lt;/p&gt;&#xA;&lt;p&gt;Corollary question:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Should be split up into two neural networks? One for betting and one for playing?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="13141" LastEditorUserId="2444" LastEditDate="2021-01-10T02:09:17.507" LastActivityDate="2021-02-09T03:04:05.370" Title="How to deal with different actions for different states of the environment?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;implementation&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5531" PostTypeId="1" AcceptedAnswerId="5534" CreationDate="2018-03-06T06:35:02.250" Score="2" ViewCount="495" Body="&lt;p&gt;In the diagram below, although the flow of information happens from the input to output layer, the labeling of weights appears reverse. Eg: For the arrow flowing from X3 to the fourth hidden layer node has the weight labeled as W(1,0) and W(4,3) instead of W(0,1) and W(3,4) which would indicate data flowing from the 3rd node of the 0'th layer to the 4th node of the 1st layer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YyD9E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YyD9E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my neural networks teachers did not emphasize on this convention at all. Another teacher made it a point to emphasize on it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a reason there is such an un-intuitive convention and is there really a convention?&lt;/p&gt;&#xA;" OwnerUserId="9268" LastActivityDate="2018-03-07T16:36:08.127" Title="Is there a naming convention for network weights for multilayer networks?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="5532" PostTypeId="5" CreationDate="2018-03-06T07:55:19.953" Score="0" Body="&lt;p&gt;AI design covers the field of both philosophical, logical and empirical time tested or novel designs, governing the internal workings of an Artificially Intelligent agent.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-06T17:12:24.500" LastActivityDate="2018-03-06T17:12:24.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5533" PostTypeId="4" CreationDate="2018-03-06T07:55:19.953" Score="0" Body="For questions related to the design of AI agents, algorithms or models." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-07-09T10:50:06.587" LastActivityDate="2019-07-09T10:50:06.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5534" PostTypeId="2" ParentId="5531" CreationDate="2018-03-06T07:57:56.087" Score="2" Body="&lt;p&gt;When the system grows matrix notation is used, as &lt;strong&gt;a&lt;/strong&gt;=&lt;strong&gt;Wx&lt;/strong&gt;, being &lt;strong&gt;a&lt;/strong&gt; (input to activation function in hidden layer) and &lt;strong&gt;x&lt;/strong&gt; (values from input layer) column vectors, transpose of (a1,a2,...a_m) and (x1,x2,...,x_n), and &lt;strong&gt;W&lt;/strong&gt; a m-by-n matrix of dimensions m rows and n columns. The standard way to denote matrix elements is w(i,j) where &quot;i&quot; is the row number and &quot;j&quot; column number:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XL9H4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XL9H4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;(from &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, the weight that applies to h4 from x3 is element in row 4 column 3 of the matrix W, that is, W(4,3) ( as your teachers advocates but with a sad lack of ability to explain ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NNVMu.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NNVMu.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: things are a few more complex when x1, x2, ... are itself vectors, but final conclusion is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;( PS: URGENT to allow latex notation on this stack exchange ! )&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-03-07T16:36:08.127" LastActivityDate="2018-03-07T16:36:08.127" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5536" PostTypeId="1" AcceptedAnswerId="7554" CreationDate="2018-03-06T10:35:34.890" Score="3" ViewCount="1426" Body="&lt;p&gt;I'm training Seq2Seq model on OpenSubtitles dialogs - &lt;a href=&quot;http://www.cs.cornell.edu/%7Ecristian/Cornell_Movie-Dialogs_Corpus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cornell-Movie-Dialogs-Corpus&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My work based on the following papers (but currently I'm not implemented Attention yet):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1409.3215&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Sequence to Sequence Learning with Neural Networks, Sutskever et al. 2014&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1506.05869&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Neural Conversational Model, Vinyals, Le, 2015&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The &lt;code&gt;loss&lt;/code&gt; I received is quite high and sucked in variation &lt;code&gt;~6.4&lt;/code&gt; after 3 epoches. The model predicts the most common words with some times other not significant words (but 99.99% is just 'you'):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I’ve experimented with 128 - 2048 hidden units and with 1 or 2 or 3 LSTM layers per &lt;code&gt;encoder&lt;/code&gt; and &lt;code&gt;decoder&lt;/code&gt;. The outcomes are more or less the same.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;SEQ1: yeah man it means love respect community and the dollars too the package the unk end&lt;/p&gt;&#xA;&lt;p&gt;SEQ2: but how did you get unk 82 end&lt;/p&gt;&#xA;&lt;p&gt;PREDICTION: promoting 16th dashboard be of the the the you you you you you you you you you you you you you you you you you you you you you you you you&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm using here &lt;code&gt;greedy&lt;/code&gt; prediction, meaning - after I receive &lt;code&gt;logit&lt;/code&gt; I do &lt;code&gt;argmax(..)&lt;/code&gt; on all its value for first-3 mini-batch-elements (here I present only first element). For convenient - &lt;code&gt;SEQ1&lt;/code&gt; and &lt;code&gt;SEQ2&lt;/code&gt; are also printed - to know the actual dialog which was presented to the model.&lt;/p&gt;&#xA;&lt;p&gt;The pseudo-code of my architecture looks like this (I'm using Tensorflow 1.5):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq1 = tf.placeholder(...)&#xA;seq2 = tf.placeholder(...)&#xA;&#xA;embeddings = tf.Variable(tf.random_uniform([vocab_size, 100],-1,1))&#xA;&#xA;seq1_emb = tf.nn.embedding_lookup(embeddings, seq1)&#xA;seq2_emb = tf.nn.embedding_lookup(embeddings, seq1)&#xA;&#xA;encoder_out, state1 = tf.nn.static_rnn(BasicLSTMCell(), seq1_emb)&#xA;decoder_out, state2 = tf.nn.static_rnn(BasicLSTMCell(), seq2_emb,&#xA;                                                        initial_state=state_1)&#xA;logit = Dense(decoder_out, use_bias=False)&#xA;&#xA;crossent = tf.nn.saparse_softmax_cross_entropy_with_logits(logits=logit, &#xA;                                                         labels=target)&#xA;crossent = mask_padded_zeros(crossent)&#xA;loss = tf.reduce_sum(crossent) / number_of_words_in_batch&#xA;&#xA;train = tf.train.AdamOptimizer(learning_rate=0.00002).minimize(loss) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm also wonder if I pass well &lt;code&gt;state1&lt;/code&gt; to &lt;code&gt;decoder&lt;/code&gt;, which in general looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# reshape in pseudocode: state1 = state[1:]&#xA;new_state1 = []&#xA;for lstm in state1:&#xA;    new_lstm = []&#xA;    for gate in lstm:&#xA;        new_lstm.append(gate[1:])&#xA;    new_state1.append(tuple(new_lstm))&#xA;state1 = tuple(new_state1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Should I use some projection layer between states of &lt;code&gt;encoder&lt;/code&gt; and &lt;code&gt;decoder&lt;/code&gt; ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So if &lt;code&gt;seq1&lt;/code&gt; has 32 words, &lt;code&gt;seq2&lt;/code&gt; has 31 (since we will not predict nothing after the last word, which is the tag &lt;code&gt;&amp;lt;END&amp;gt;&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="12691" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-08T12:15:27.360" Title="Seq2Seq dialogs predicts only most common words like `you` after couple of epoches" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5537" PostTypeId="2" ParentId="5526" CreationDate="2018-03-06T11:10:05.103" Score="4" Body="&lt;p&gt;A human analogy can help you here (a variance).&lt;/p&gt;&#xA;&lt;p&gt;Initialize all the agents with an initial value &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;; we will call this &lt;code&gt;energyUnits&lt;/code&gt;. I Will talk later more about this.&lt;/p&gt;&#xA;&lt;p&gt;Now, add some value, as an incentive, whenever the agent eats good food, to the &lt;code&gt;energyUnits&lt;/code&gt;. You need to add a function that will keep decrementing the value of the agent's &lt;code&gt;energyUnits&lt;/code&gt;, as humans degrade energy (calories) with time. We will call this function &lt;code&gt;normalDegrade&lt;/code&gt;. This is the core part of the solution for your problem.&lt;/p&gt;&#xA;&lt;p&gt;Now, for the bad (or poisonous) food you can be more creative with. You can simply subtract a given value whenever an agent eats poisonous food. Or you can extend your &lt;code&gt;normalDegrade&lt;/code&gt; function with a very high downward slope. In this case, the energy units (value) of the agent will fall very rapidly. This will force the agent to look for good food to survive.&lt;/p&gt;&#xA;&lt;p&gt;Since the ratio of food is 9:1 with poisonous, you need to initialize the value of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (&lt;code&gt;energyUnits&lt;/code&gt;) very high. You need to do some trial and error to find the right fit for you here.&lt;/p&gt;&#xA;&lt;p&gt;Also, I am assuming that the agent is being removed from the population whenever the value of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is zero or some negative value (which depends). This is important, as it makes sure that the algorithm is not wasting time in processing bad agents.&lt;/p&gt;&#xA;&lt;p&gt;Because of this, another problem arises of the population coming to extinction. For this, you need to keep generating new agents for which any of the genetic algorithms will do. A new population with better parents of the already present generation will keep the population fit and efficient.&lt;/p&gt;&#xA;&lt;p&gt;A good fitness function is a core to solving any problem of this kind, and sometimes it is hard to find. You might need to do some trial and error with different values to look for the right fit.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-30T02:29:05.747" LastActivityDate="2021-01-30T02:29:05.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5538" PostTypeId="1" CreationDate="2018-03-06T13:24:53.627" Score="3" ViewCount="725" Body="&lt;p&gt;Google Analytics allows me to collect data about every web-session. For simplicity, let's assume for each user, we collect the number of pages and time spent on site for each session:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;user_id visit_id page_views time_spent result&#xA;1       1        10         100        0&#xA;1       2        31         510        0&#xA;1       3        1          10         1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How would you model this data? What I would like the &lt;strong&gt;ML algorithm to&#xA;    do&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extract as much information as possible&lt;/li&gt;&#xA;&lt;li&gt;Have a flexible number of&#xA;inputs (e.g. the number of sessions can go to infinity)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I can think of&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Aggregate the data per user e.g. average page_views or total page_views and feed it into a general algorithm e.g. random forrest (but I lose information with aggregation)&lt;/li&gt;&#xA;&lt;li&gt;Use LSTM and feed at most last 3 visits (will also lose information, but would this perform better than aggregation?)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt;&#xA;To build a predictive model to analyse all user sessions and make a prediction whether the person will convert or not.&lt;/p&gt;&#xA;" OwnerUserId="13148" LastActivityDate="2018-07-08T19:32:51.770" Title="ML model that is most suited to analyse Google Analytics data" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5539" PostTypeId="1" CreationDate="2018-03-06T14:46:16.433" Score="5" ViewCount="889" Body="&lt;p&gt;There are a lot of papers that show that neural networks can approximate a wide variety of functions. However, I can't find papers that show the limitations of NNs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the limitations of neural networks? Which functions can't neural networks learn efficiently (or using gradient-descent)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking also for links to papers that describe these limitations.&lt;/p&gt;&#xA;" OwnerUserId="13067" LastEditorUserId="2444" LastEditDate="2019-05-10T14:45:18.307" LastActivityDate="2020-04-29T14:21:55.400" Title="Which functions can't neural networks learn efficiently?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;function-approximation&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5540" PostTypeId="2" ParentId="3361" CreationDate="2018-03-06T15:15:21.910" Score="2" Body="&lt;p&gt;Fann (&lt;a href=&quot;http://leenissen.dk/fann/wp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://leenissen.dk/fann/wp/&lt;/a&gt;) is a free open source neural network library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FANN Features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multilayer Artificial Neural Network Library in C&lt;/li&gt;&#xA;&lt;li&gt;Backpropagation training (RPROP, Quickprop, Batch, Incremental)&lt;/li&gt;&#xA;&lt;li&gt;Evolving topology training which dynamically builds and trains the ANN (Cascade2)&lt;/li&gt;&#xA;&lt;li&gt;Easy to use (create, train and run an ANN with just three function calls)&lt;/li&gt;&#xA;&lt;li&gt;Fast (up to 150 times faster execution than other libraries)&lt;/li&gt;&#xA;&lt;li&gt;Versatile (possible to adjust many parameters and features on-the-fly)&lt;/li&gt;&#xA;&lt;li&gt;Well documented (An easy to read introduction article, a thorough reference manual, and a 50+ page university report describing the implementation considerations etc.)&lt;/li&gt;&#xA;&lt;li&gt;Cross-platform (configure script for linux and unix, dll files for windows, project files for MSVC++ and Borland compilers are also reported to work)&lt;/li&gt;&#xA;&lt;li&gt;Several different activation functions implemented (including stepwise linear functions for that extra bit of speed)&lt;/li&gt;&#xA;&lt;li&gt;Easy to save and load entire ANNs&lt;/li&gt;&#xA;&lt;li&gt;Several easy to use examples&lt;/li&gt;&#xA;&lt;li&gt;Can use both floating point and fixed point numbers (actually both float, double and int are available)&lt;/li&gt;&#xA;&lt;li&gt;Cache optimized (for that extra bit of speed)&lt;/li&gt;&#xA;&lt;li&gt;Open source, but can still be used in commercial applications (licenced under LGPL)&lt;/li&gt;&#xA;&lt;li&gt;Framework for easy handling of training data sets&lt;/li&gt;&#xA;&lt;li&gt;Graphical Interfaces&lt;/li&gt;&#xA;&lt;li&gt;Language Bindings to a large number of different programming languages&lt;/li&gt;&#xA;&lt;li&gt;Widely used (approximately 100 downloads a day)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12736" LastActivityDate="2018-03-06T15:15:21.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5541" PostTypeId="2" ParentId="4048" CreationDate="2018-03-06T16:10:26.510" Score="2" Body="&lt;p&gt;You would definitely want your network to know crucial information about the game, like what cards AI agent has(their values and types), mana pool, how many cards on the table and their values, number of the turn and so on. These things you must figure on your own, the question you should ask yourself is &quot;If I add this value to input how and why it will improve my system&quot;. But the first thing to understand is that most of NNs are designed to have a constant input size, and I would assume this is matters in this game since players can have a different amount of cards in their hand or on the table. For example, you want to let NN know what cards it has, let's assume the player can have a maximum of 5 cards in his hand and each card can have 3 values(mana, attack and health), so you can encode this as 5*3 vector, where first 3 values represent card number one and so on. But what if the player has currently 3 cards, a simple approach would be to assign zeros to last 6 inputs, but this may cause problems since some cards can have 0 mana cost or 0 attack. So you need to figure out how to solve this problem. You may look for NN models that can handle variable input size or figure out how to encode input as a vector of constant size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, outputs are also constant size vectors. In case of this type of game, it can be a vector that encodes actions that the agent can take. So let's say we have 3 actions: put a card, skip turn and concede. So it can be one hot encoder, e.g. if you have 1 0 0 output, this means that agent should put some card. To know what card it should put you can add another element to output which will produce a number in the range of 1 to 5 (5 is max number of cards in the hand).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the most important part of training a neural network is that you will have to come up with a loss function that is suitable for your task. Maybe standard loss functions like Mean-squared loss or L2 will be good, maybe you will need to change them in order to fit your needs. This is the part where you will need to make a research. I've never worked with NEAT before, but as I understood correctly it uses some genetic algorithm to create and train NN, and GA use some fitness function to select an individual. So basically you will need to know what metric you will be using to evaluate how good you model performs and based on this metric you will change parameters of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS.&#xA;It is possible to solve this problem with the neural network, however, neural networks are not magic and not the universal solution to all problems. If your goal is to solve this certain problem I would also recommend you to dig into the game theory and its application in the AI. I would say, that solving this problem would require complex knowledge from different fields of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, If your goal is to learn about neural networks I would recommend taking much simpler tasks. For example, you can implement NN that will work on benchmark dataset, for example, NN that will classify digits from MNIST dataset. The reason for this is that a lot of articles was written about how to do classification on this dataset and you will learn a lot and you will learn faster from implementing simple things.&lt;/p&gt;&#xA;" OwnerUserId="13102" LastActivityDate="2018-03-06T16:10:26.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5543" PostTypeId="2" ParentId="5539" CreationDate="2018-03-06T18:48:56.720" Score="5" Body="&lt;p&gt;One of the important qualifications of the Universal approximation theorem is that the neural network approximation may be computationally infeasible.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;A feedforward network with a single layer is sufficient to represent any function, but &lt;strong&gt;the layer may be infeasibly large and may fail to learn and generalize correctly.&lt;/strong&gt;&quot; - Ian Goodfellow, &lt;a href=&quot;http://www.deeplearningbook.org/contents/mlp.html&quot; rel=&quot;noreferrer&quot;&gt;DLB&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can't think of any function that I would definitively declare as unlearnable, but neural networks have many problems. Consider &lt;a href=&quot;https://arxiv.org/abs/1312.6199&quot; rel=&quot;noreferrer&quot;&gt;adversarial examples&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1712.09665&quot; rel=&quot;noreferrer&quot;&gt;adversarial patches&lt;/a&gt;, which highlight the poor generalization going on under the hood of recent advances in computer vision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural Networks are also inherently limited by the innate priors baked into their architecture and the sample density of their training data. Check out &lt;a href=&quot;https://www.youtube.com/watch?v=fKk9KhGRBdI&quot; rel=&quot;noreferrer&quot;&gt;this recent discussion&lt;/a&gt; at Stanford's AI Salon between Yann LeCun and Christopher Manning on innate priors if that is the kind of limitation you are talking about.&lt;/p&gt;&#xA;" OwnerUserId="13156" LastActivityDate="2018-03-06T18:48:56.720" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5544" PostTypeId="5" CreationDate="2018-03-06T18:59:01.277" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-06T18:59:01.277" LastActivityDate="2018-03-06T18:59:01.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5545" PostTypeId="4" CreationDate="2018-03-06T18:59:01.277" Score="0" Body="Use when requesting examples of research or research papers, books, articles, blog posts or courses. For example, &quot;Is there any published research about X?&quot; or &quot;What are good examples of Y in research?&quot;." OwnerUserId="13156" LastEditorUserId="2444" LastEditDate="2020-05-22T18:09:10.087" LastActivityDate="2020-05-22T18:09:10.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5546" PostTypeId="1" CreationDate="2018-03-06T21:01:55.610" Score="33" ViewCount="60911" Body="&lt;p&gt;I've seen these terms thrown around this site a lot, specifically in the tags &lt;a href=&quot;/questions/tagged/convolutional-neural-networks&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;convolutional-neural-networks&amp;#39;&quot; rel=&quot;tag&quot;&gt;convolutional-neural-networks&lt;/a&gt; and &lt;a href=&quot;/questions/tagged/neural-networks&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;neural-networks&amp;#39;&quot; rel=&quot;tag&quot;&gt;neural-networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that a neural network is a system based loosely on the human brain. But what's the difference between a &lt;em&gt;convolutional&lt;/em&gt; neural network and a regular neural network? Is one just a lot more complicated and, ahem, &lt;em&gt;convoluted&lt;/em&gt; than the other? &lt;/p&gt;&#xA;" OwnerUserId="145" LastEditorUserId="2444" LastEditDate="2020-03-13T17:19:33.647" LastActivityDate="2020-04-25T22:13:50.023" Title="What is the difference between a convolutional neural network and a regular neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="19" ContentLicense="CC BY-SA 4.0" />
  <row Id="5547" PostTypeId="2" ParentId="3982" CreationDate="2018-03-06T21:37:14.147" Score="1" Body="&lt;p&gt;Seem like time is a good fitness, though you need it to engage into learning side sensor inputs and side movement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I would consider adding a bit of randomness to the environment.&lt;/strong&gt; How about adding some random mild forces that might sway it left, right, front and rear a bit so that that bots are forced to use other sensors and inputs to stay in the center.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cases when drone is not simulated, this task is a little harder but adding randomness to the environment can still be done. For example, tilt your drone a tiny amount in a random direction at random intervals. This will force your drone to learn to correct for being jostled by wind, without you having to actually produce wind.&lt;/p&gt;&#xA;" OwnerUserId="13138" LastEditorUserId="13138" LastEditDate="2018-03-06T23:39:13.210" LastActivityDate="2018-03-06T23:39:13.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5548" PostTypeId="1" AcceptedAnswerId="5549" CreationDate="2018-03-06T22:29:47.460" Score="2" ViewCount="804" Body="&lt;p&gt;I am reading through the NEAT paper &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. On page 14 of the PDF, there is this quote about mutation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There was an 80% chance of a genome having its connection weights mutated, in which case &lt;strong&gt;each weight had a 90% chance of being uniformly perturbed&lt;/strong&gt; and a 10% chance of being assigned a new random value.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What exactly does it mean to perturb weights? What is uniform vs. nonuniform perturbation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an established method to do this? I am imagining the process as multiplying each connection weight by a random number, but I'm unfamiliar with the term.&lt;/p&gt;&#xA;" OwnerUserId="13160" LastActivityDate="2018-03-07T16:19:27.323" Title="How are connection weights &quot;perturbed&quot;?" Tags="&lt;neural-networks&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5549" PostTypeId="2" ParentId="5548" CreationDate="2018-03-06T22:40:37.427" Score="3" Body="&lt;p&gt;Perturbed here means adding a small random value to the weight. That random value comes from a uniform distribution or from a gaussian (or any distribution really). &#xA;Imagine just nudging the weight by a little. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It’s done to overcome the problem of local minima where models can get stuck with a good set of weights but not the best set of weights. By perturbing the weights a little, the model has a chance of finding a better set of weights. Searching for methods that deal with local minima like stochastic gradient descent will give you a better intuition. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="13088" LastEditDate="2018-03-07T16:19:27.323" LastActivityDate="2018-03-07T16:19:27.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5550" PostTypeId="2" ParentId="5546" CreationDate="2018-03-06T23:01:38.160" Score="6" Body="&lt;p&gt;A &lt;em&gt;convolutional&lt;/em&gt; neural network is one that has &lt;em&gt;convolutional&lt;/em&gt; layers. If a general neural network is, loosely speaking, inspired by a human brain (which isn't very much accurate), the convolutional neural network is inspired by the visual cortex system, in humans and other animals (which is closer to the truth). As the name suggests, this layer applies the convolution with a learnable filter (a.k.a. &lt;em&gt;kernel&lt;/em&gt;), as a result the network learns the patterns in the images: edges, corners, arcs, then more complex figures. Convolutional neural network may contain other layers as well, commonly pooling and dense layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Highly recommend &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#overview&quot; rel=&quot;noreferrer&quot;&gt;CS231n tutorial&lt;/a&gt; on this matter: it's very detailed and contains a lot of very nice visualizations.&lt;/p&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2018-03-06T23:01:38.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5551" PostTypeId="1" AcceptedAnswerId="5562" CreationDate="2018-03-07T04:23:28.307" Score="0" ViewCount="71" Body="&lt;p&gt;Let us for these purposes say with are working with any feed forward neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us also say, that we know beforehand that certain portion of our dataset arsignificantly more impactful or important to our underlying representation. Is there anyway to add that “weighting” to our data?&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2018-03-09T09:29:08.607" Title="A way to give more weight to particular data?" Tags="&lt;training&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5552" PostTypeId="2" ParentId="5539" CreationDate="2018-03-07T05:23:35.633" Score="2" Body="&lt;p&gt;This answer depends very much so on the type of neural network and algorithm used for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using gradient descent on a neural network of one input layer, one output layer, and no hidden layers there are many functions that you can't learn. One simple one is the XOR function. Due to the fact that XOR is not linearly separable, it can not be represented by a neural network with no hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using NEAT to build recurrent neural networks then all functions(**) can be represented given enough time and data. This is due in part to the fact that recurrent neural networks are Turing Complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the biggest causes for limitations when using neural networks is based on the difficulty of interpretation as to what the network is doing. The network is gradually building up an understanding of the function as it goes from the input layer to the output layer, but it is very difficult for us to understand this building up process and interpret what the neural network is attempting to do. This makes it very hard if not impossible to manually tweak your neural network in a meaningful way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another limitation is the need for training (in large amounts) in order to have a meaningful representation of your data. Neural networks have a tendency to need large amounts of data before converging to a meaningful hypothesis space. This has resulted in clever algorithms to generate training data without needing human interaction, such as Generative Adversarial Networks, but the underlying problem remains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** Not all functions can be computed by neural networks, however all computable functions can be. An example of an uncomputable function is the mapping of all programs from the program to whether or not this program will halt (The Halting Problem).&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-07T05:23:35.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5553" PostTypeId="1" AcceptedAnswerId="5581" CreationDate="2018-03-07T06:17:34.207" Score="3" ViewCount="822" Body="&lt;p&gt;As we all know, there has been tons of GAN variants featuring different aspects of the image generation task such as stability, resolution or the ability to manipulate images. However, it is still confusing to me that how do we determine that images generated by one network are more plausible than images generated by another?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: could someone with higher reputation create more tags like image generation?&lt;/p&gt;&#xA;" OwnerUserId="13165" LastEditorUserId="1847" LastEditDate="2018-03-07T08:38:11.113" LastActivityDate="2018-03-08T20:33:55.370" Title="How to evaluate the goodness of images generated by GANs?" Tags="&lt;deep-learning&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5554" PostTypeId="2" ParentId="2526" CreationDate="2018-03-07T07:19:47.700" Score="2" Body="&lt;p&gt;Lets go through your question point by point.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Should an activation function be differentiable?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;No, it is not of any compulsion of it to be differentiable. We use Re-Lu's which have a non-differentiable point at 0. But this is simply a trivial case, since the point 0 will never be reached unless we run out of precision points to denote extemely less numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So lets take another example, the &lt;a href=&quot;http://hagan.okstate.edu/4_Perceptron.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptron learning&lt;/a&gt; algorithm. In this algorithm, there is no satisfactory way to evaluate the performance of a particular solution. So we don't have a cost function. But still we are able to reach to a solution, albeit maybe not the best. I'll come to the point why it is not used later. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NN's can be broadly thought of as just function approximators. Normally, you give it some continuous function, the NN adjusts it by elongating, shifting, distorting parts of that function by changing only and only the parameters of the function and not the nature of the function itself i.e. it'll decompose into the same sort of a Fourier series as before, with only phase and amplitude differences. You can also design a NN on lines of a random search puzzle. You give each node of a, say single hidden layer NN a part of the function to be approximated i.e. between some intervals &lt;code&gt;-a &amp;lt; x &amp;lt; b&lt;/code&gt; say. Say the connection weights to each output layer is fixed. You can only change the part of the function (i.e.parts broken like a jigsaw from a function to be approximated) you want to give to a node, from your fixed box of parts. So you see, in this case the NN will try almost all combinations of giving different parts to different nodes until the function is perfectly approximated. So you see, this can also be a type of learning for NN without any continuous function.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why should an activation function of a neural network be differentiable?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The main advantages of differentiable function is the mathematics behind it. You can easily handle huge amounts of data just by simple mathematics. 1000's of years of mathematical theories can be applied to verify the working of your NN, predict how will it work, select best algorithms. In discrete non-differentiable activation functions you have no realistic way of predicting what will be the final result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FWVIW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FWVIW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Say in a problem like this, if you apply the perceptron learning algo., do you have any realistic way to predict where the final decision boundary will rest? Whereas, if you train it by continuous methods, you can easily plot the cost function versus weights, check the global minima, get the weights which will be the final weights. So we easily solved it by plotting (definite matrix methods also exist).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it advantageous to have a differentiable activation function? Why?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So why is it advantegeous?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--&gt;&lt;/strong&gt; Final answers easily predictable (if relatively small data set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--&gt;&lt;/strong&gt; Easy to draw graphs and visualize the working of your NN and adjust your hyper-parameters accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--&gt;&lt;/strong&gt; Easy to apply time tested mathematical tools to test/evaluate the effectiveness of your algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--&gt;&lt;/strong&gt; No sudden changes in error and so your weights will also not change suddenly due to stray readings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are all the advantages I could think of and I am sue there are more. All these tools are not available for discrete algorithms, if you think (can't give the full intuition - too lengthy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I gave some more intuition here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5493/what-is-the-purpose-of-an-activation-function-in-neural-networks?noredirect=1&amp;amp;lq=1&quot;&gt;What is the purpose of an activation function in Neural Networks?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feel free to add anything I missed by editing the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-03-07T07:19:47.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5555" PostTypeId="5" CreationDate="2018-03-07T07:24:01.387" Score="0" Body="&lt;p&gt;Learning algorithms are the procedures that a Machine Learning program must follow to achieve its end result. Performance of different learning algorithms vary, but the general end goal is to minimize the error between target output and actual output for previously unseen inputs.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="4398" LastEditDate="2018-03-07T16:21:33.713" LastActivityDate="2018-03-07T16:21:33.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5556" PostTypeId="4" CreationDate="2018-03-07T07:24:01.387" Score="0" Body="For questions about different learning algorithms used by a Machine Learning program to achieve its end goal." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-09T18:17:02.730" LastActivityDate="2018-03-09T18:17:02.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5558" PostTypeId="1" CreationDate="2018-03-07T07:32:33.493" Score="4" ViewCount="116" Body="&lt;p&gt;I have a fully connected network that takes in a variable length input padded with 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the network doesn't seem to be learning and I am guessing that the high number of zeros in the input might have something to do with that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there solution for dealing with padded input in fully connected layers or should I consider a different architecture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE (to provide more details)&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the network if to clean full file paths:&#xA;i.g.: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;/My document/some folder/a file name.txt &amp;gt; a file name&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;/Hard drive/book/deeplearning/1.txt &amp;gt; deeplearning&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The constraint is that the training data labels have been generated using a regex on the file name itself so it's not very accurate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am hopping that by treating every word equally (without sequential information) the network would be able to generalize as to which type of word is usually kept and which is usually discarded. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then network takes in a sequence of word embedding trained on paths data and output a logits that corresponds to probabilities of each word to be kept or not.    &lt;/p&gt;&#xA;" OwnerUserId="12931" LastEditorUserId="12931" LastEditDate="2018-03-07T08:44:09.773" LastActivityDate="2021-05-09T08:04:10.407" Title="How to deal with padded inputs in a fully connected feed forward network?" Tags="&lt;deep-learning&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5559" PostTypeId="1" CreationDate="2018-03-07T08:41:55.487" Score="0" ViewCount="109" Body="&lt;p&gt;I have an idea about how to use neural networks but I'm not sure if it is possible or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In supervised learning we have a set of attributes labeled with an output value. I can use these set to train my network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have a network trained to get an output value from an random set of attributes but, &lt;strong&gt;can I use this trained network to get the input attributes using only the desired output?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will have N input values and only 1 output value. I've thought that I can use the weights for that network into a new one with 1 input value and N output values but I'm not sure if I can do that.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2018-03-07T12:58:38.843" LastActivityDate="2018-03-09T18:09:01.630" Title="Neural network to get input attributes using only the output value" Tags="&lt;neural-networks&gt;&lt;generative-model&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5560" PostTypeId="2" ParentId="5559" CreationDate="2018-03-07T12:37:41.823" Score="0" Body="&lt;p&gt;NN isn't symmetric and the result may haven't any seems.&#xA;You can just use loss like this (target_output - model_output)**2 - MSE&#xA;differentiate this loss  according to input variables and use optimizer to solve this task. Try to search about &lt;a href=&quot;https://en.wikipedia.org/wiki/Adversary_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;adversarial models&lt;/a&gt;, the idea is very similar.&lt;/p&gt;&#xA;" OwnerUserId="13067" LastEditorUserId="1671" LastEditDate="2018-03-09T18:09:01.630" LastActivityDate="2018-03-09T18:09:01.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5561" PostTypeId="2" ParentId="5529" CreationDate="2018-03-07T15:06:38.100" Score="0" Body="&lt;p&gt;There are so many reinforcement learning algorithms. I recommend you this articles:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/introduction-to-various-reinforcement-learning-algorithms-i-q-learning-sarsa-dqn-ddpg-72a5e0cb6287&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Various Reinforcement Learning Algorithms. Part&#xA;1&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/introduction-to-various-reinforcement-learning-algorithms-part-ii-trpo-ppo-87f2c5919bb9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Various Reinforcement Learning Algorithms.    Part&#xA;2&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are several approaches. A NN can get state and action as an input  and expected discounted reward as an output. But you can just not pick an action if this action isn't available and there isn't necessity to give it negative or zero reward just don't learn your net to change it behavior for these actions because they are neither good nor bad. &lt;/p&gt;&#xA;" OwnerUserId="13067" LastActivityDate="2018-03-07T15:06:38.100" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5562" PostTypeId="2" ParentId="5551" CreationDate="2018-03-07T18:14:29.263" Score="1" Body="&lt;p&gt;Yes, You just weight the entries of loss function like sum( Wi(li-t2)**2) for MSE. Most frameworks provides this ability. For example in keras fit method has a parameter - sample_weight: Optional Numpy array of weights for the training samples.&lt;/p&gt;&#xA;" OwnerUserId="13067" LastActivityDate="2018-03-07T18:14:29.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5563" PostTypeId="5" CreationDate="2018-03-07T18:28:11.307" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-07T18:28:11.307" LastActivityDate="2018-03-07T18:28:11.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5564" PostTypeId="4" CreationDate="2018-03-07T18:28:11.307" Score="0" Body="For questions related to the concept of generative machine learning models, such as the Restricted Boltzmann Machine (RBM), the Variational Autoencoder (VAE), and the Generative Adversarial Network (GAN)." OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-07-06T22:45:08.190" LastActivityDate="2019-07-06T22:45:08.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5565" PostTypeId="2" ParentId="5559" CreationDate="2018-03-07T18:35:03.100" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Now I have a network trained to get an output value from an random set of attributes but, can I use this trained network to get the input attributes using only the desired output?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It depends:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you are happy to find any inputs, even non-realistic ones, that get your desired output, then you can use your trained network, with a minor modification. Freeze all the weights, and allow back-propagation to determine the gradient of the input (which should now be a variable to optimise, not source data). Start with a noise input, back-propagate the error to find gradient to make the input better at creating your desired output, then take a gradient step towards it in the input data. This is essentially how Deep Dream works. Like Deep Dream, you will not necessarily get realistic input values, but will get &lt;a href=&quot;https://mtyka.github.io/deepdream/2016/02/05/bilateral-class-vis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;semi-random ones that cause your network to predict a specific class&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want the newly generated input to be a best guess at something from the original dataset, then you have to look at one of more advanced models:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Restricted Boltzmann Machines (RBMs)&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://kvfrans.com/variational-autoencoders-explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Autoencoders (VAEs)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks (GANs)&lt;/a&gt;. These have become popular recently, and there are many variations on the basic idea, including VAEGANs that combine VAEs with GANs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These network types are quite advanced, and can be tricky to understand and train successfully. You will want to spend some time researching each type. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To generalise terribly: A GAN will tend to generate realistic &quot;noise&quot; in the generated items, but at the expense of overall structure and cohesion (images tend to look distorted but with realistic textures). A VAE will tend to produce smooth, coherent inputs, but at the expense of lack of fine detail (VAE images tend to look smoothed and/or blurred). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not sure what to try, probably GAN is a reasonable choice, since there are lots of tutorials available, and &lt;a href=&quot;https://www.youtube.com/watch?v=36lE9tV9vm0&amp;amp;t=3099s&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent advances with image generation can look very impressive&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-03-07T18:35:03.100" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5566" PostTypeId="2" ParentId="5546" CreationDate="2018-03-07T18:50:07.183" Score="14" Body="&lt;p&gt;Convolutional Neural Networks (CNNs) are neural networks with architectural constraints to reduce computational complexity and ensure translational invariance (the network interprets input patterns the same regardless of translation— in terms of image recognition: a banana is a banana regardless of where it is in the image). Convolutional Neural Networks have three important architectural features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Local Connectivity:&lt;/em&gt;&lt;/strong&gt; Neurons in one layer are only connected to neurons in the next layer that are spatially close to them. This design trims the vast majority of connections between consecutive layers, but keeps the ones that carry the most useful information. The assumption made here is that the input data has spatial significance, or in the example of computer vision, the relationship between two distant pixels is probably less significant than two close neighbors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Shared Weights:&lt;/em&gt;&lt;/strong&gt; This is the concept that makes CNNs &quot;convolutional.&quot; By forcing the neurons of one layer to share weights, the forward pass (feeding data through the network) becomes the equivalent of convolving a filter over the image to produce a new image. The training of CNNs then becomes the task of learning filters (deciding what features you should look for in the data.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Pooling and ReLU:&lt;/em&gt;&lt;/strong&gt; CNNs have two non-linearities: pooling layers and ReLU functions. Pooling layers consider a block of input data and simply pass on the maximum value. Doing this reduces the size of the output and requires no added parameters to learn, so pooling layers are often used to regulate the size of the network and keep the system below a computational limit. The ReLU function takes one input, x, and returns the maximum of {0, x}. &lt;code&gt;ReLU(x) = argmax(x, 0)&lt;/code&gt;. This introduces a similar effect to tanh(x) or sigmoid(x) as non-linearities to increase the model's expressive power.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Further Reading&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As another answer mentioned, Stanford's CS 231n course covers this in detail. Check out &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#overview&quot; rel=&quot;noreferrer&quot;&gt;this written guide&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=bNb2fEVKeEo&amp;amp;t=0s&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=5&quot; rel=&quot;noreferrer&quot;&gt;this lecture&lt;/a&gt; for more information. Blog posts like &lt;a href=&quot;https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt; and &lt;a href=&quot;https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt; are also very helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're still curious why CNNs have the structure that they do, I suggest reading &lt;a href=&quot;http://www.dengfanxin.cn/wp-content/uploads/2016/03/1998Lecun.pdf&quot; rel=&quot;noreferrer&quot;&gt;the paper that introduced them&lt;/a&gt; though this is quite long, and perhaps checking out &lt;a href=&quot;https://www.youtube.com/watch?v=fKk9KhGRBdI&quot; rel=&quot;noreferrer&quot;&gt;this discussion&lt;/a&gt; between Yann Lecun and Christopher Manning about innate priors (the assumptions we make when we design the architecture of a model).&lt;/p&gt;&#xA;" OwnerUserId="13156" LastEditorUserId="13156" LastEditDate="2018-11-03T18:17:02.057" LastActivityDate="2018-11-03T18:17:02.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5567" PostTypeId="2" ParentId="5517" CreationDate="2018-03-07T22:11:46.440" Score="0" Body="&lt;p&gt;There has been some work in this area but with three chat bots.  &lt;a href=&quot;https://github.com/xhlulu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xing Han Lu&lt;/a&gt; has developed some &lt;a href=&quot;https://github.com/xhlulu/Generative-Adversarial-Bots&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; called Generative Adversarial Bots (GABs) which builds on the concept of Generative Adversarial Networks invented by Ian Goodfellow.  See his pioneering paper &lt;a href=&quot;https://arxiv.org/abs/1406.2661&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Generative Adversarial Networks&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a very brief Google presentation &lt;a href=&quot;https://docs.google.com/presentation/d/1MjSdeB57STukNT04J7WG1fOLMiktLi9YsH4rbOxnznI/edit#slide=id.p&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea of GABs is to &quot;compare the performance and human-likeness of conversational chatbots by generating conversation between two bots, and evaluating the response using Turing Tests&quot;.  &quot;Generative Adversarial Bots (GABs) are bots that are pitched up against each other, and generate a conversation that is used to train a third bot&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to also check out &lt;a href=&quot;https://arxiv.org/abs/1701.06547&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Adversarial Learning for Neural Dialogue Generation&quot;&lt;/a&gt; by Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter and Dan Jurafsky.  From their abstract:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In this paper, drawing intuition from the Turing test, we propose using adversarial training for open-domain dialogue generation: the system is trained to produce sequences that are indistinguishable from human-generated dialogue utterances. We cast the task as a reinforcement learning (RL) problem where we jointly train two systems, a generative model to produce response sequences, and a discriminator---analagous to the human evaluator in the Turing test--- to distinguish between the human-generated dialogues and the machine-generated ones. The outputs from the discriminator are then used as rewards for the generative model, pushing the system to generate dialogues that mostly resemble human dialogues.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jiwei Li, one of the authors, has posted the code &lt;a href=&quot;https://github.com/jiweil/Neural-Dialogue-Generation&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2018-03-07T22:30:47.840" LastActivityDate="2018-03-07T22:30:47.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5568" PostTypeId="1" CreationDate="2018-03-08T00:38:21.927" Score="0" ViewCount="227" Body="&lt;p&gt;I used to work with 'traditional' layered neural network and I evaluated the output given certain inputs by processing layer-by-layer.&lt;br&gt;&#xA;With NEAT, a neural network may assume any topology and they are no longer layered, so how to evaluate the output? I understand time-steps must be taken into account, but how?&lt;br&gt;&#xA;Should I keep the inputs until all hidden neurons are processed and output is produced? Should I wait for output to stabilize?&lt;/p&gt;&#xA;" OwnerUserId="13087" LastEditorUserId="4302" LastEditDate="2018-07-23T15:38:08.340" LastActivityDate="2018-07-23T15:38:08.340" Title="How to evaluate output of unlayered NN?" Tags="&lt;neural-networks&gt;&lt;neat&gt;&lt;topology&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5569" PostTypeId="2" ParentId="5546" CreationDate="2018-03-08T01:49:51.950" Score="35" Body="&lt;p&gt;&lt;strong&gt;TLDR:&lt;/strong&gt; &#xA;The convolutional-neural-network is a subclass of neural-networks which have at least one convolution layer. They are great for capturing local information (e.g. neighbor pixels in an image or surrounding words in a text) as well as reducing the complexity of the model (faster training, needs fewer samples, reduces the chance of overfitting). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the following chart that depicts the several neural-networks architectures including deep-conventional-neural-networks: &lt;a href=&quot;https://i.stack.imgur.com/LgmYv.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LgmYv.png&quot; alt=&quot;visualization&quot;&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neural Networks (NN)&lt;/strong&gt;, or more precisely &lt;strong&gt;Artificial Neural Networks (ANN)&lt;/strong&gt;, is a class of Machine Learning algorithms that recently received a lot of attention (again!) due to the availability of Big Data and fast computing facilities (most of Deep Learning algorithms are essentially different variations of ANN).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The class of ANN covers several architectures including &lt;strong&gt;Convolutional Neural Networks&lt;/strong&gt; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;noreferrer&quot;&gt;CNN&lt;/a&gt;), Recurrent Neural Networks (&lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;noreferrer&quot;&gt;RNN&lt;/a&gt;) eg &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;noreferrer&quot;&gt;LSTM&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Gated_recurrent_unit&quot; rel=&quot;noreferrer&quot;&gt;GRU&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;noreferrer&quot;&gt;Autoencoders&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_belief_network&quot; rel=&quot;noreferrer&quot;&gt;Deep Belief Networks&lt;/a&gt;. Therefore, CNN is just one kind of ANN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, an ANN is a collection of connected and tunable units (a.k.a. nodes, neurons, and artificial neurons) which can pass a signal (usually a real-valued number) from a unit to another. The number of (layers of) units, their types, and the way they are connected to each other is called the network architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A CNN, in specific, has one or more layers of &lt;strong&gt;&lt;em&gt;convolution&lt;/em&gt;&lt;/strong&gt; units. A convolution unit receives its input from multiple units from the previous layer which together create a proximity. Therefore, the input units (that form a small neighborhood) share their weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convolution units (as well as pooling units) are especially beneficial as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They reduce the number of units in the network (since they are &lt;em&gt;many-to-one mappings&lt;/em&gt;). This means, there are fewer parameters to learn which reduces the chance of overfitting as the model would be less complex than a fully connected network.&lt;/li&gt;&#xA;&lt;li&gt;They consider the context/shared information in the small neighborhoods. This future is very important in many applications such as image, video, text, and speech processing/mining as the neighboring inputs (eg pixels, frames, words, etc) usually carry related information.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Read the followings for more information about (deep) CNNs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.toronto.edu/~kriz/imagenet_classification_with_deep_convolutional.pdf&quot; rel=&quot;noreferrer&quot;&gt;ImageNet Classification with Deep Convolutional Neural Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1409.4842&quot; rel=&quot;noreferrer&quot;&gt;Going Deeper with Convolutions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;p.s. ANN is not &lt;em&gt;&quot;a system based loosely on the human brain&quot;&lt;/em&gt; but rather a &lt;strong&gt;class&lt;/strong&gt; of systems &lt;strong&gt;inspired&lt;/strong&gt; by the neuron connections exist in animal brains.&lt;/p&gt;&#xA;" OwnerUserId="12853" LastEditorUserId="10823" LastEditDate="2019-07-05T23:23:45.807" LastActivityDate="2019-07-05T23:23:45.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5570" PostTypeId="1" CreationDate="2018-03-08T06:10:00.340" Score="1" ViewCount="737" Body="&lt;p&gt;In two-player games, the exact value of the evaluation function doesn't matter, as long as it's bigger for better positions. However, for learning, it's customary when it does change when the best move gets made. This way, the learning can minimize the difference between the directly computed value &lt;span class=&quot;math-container&quot;&gt;$f(0, p)$&lt;/span&gt; of a position &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and the value obtained from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; step minimax &lt;span class=&quot;math-container&quot;&gt;$f(n, p)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What I'm missing here is a way to direct the evaluation function to actually winning. For example, a &lt;em&gt;perfect&lt;/em&gt; evaluation function for a won position in chess would always return &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; without any hint on how to progress towards a checkmate. In a chess variant without the fifty-move limit, it could play useless turns forever.&lt;/p&gt;&#xA;&lt;p&gt;I guess this is a rather theoretical problem, as we won't ever have such a good function, but I wonder &lt;em&gt;if there's a way to avoid it&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12053" LastEditorUserId="2444" LastEditDate="2021-02-06T21:32:28.903" LastActivityDate="2021-02-06T21:32:28.903" Title="Game AI evaluation function and making progress towards winning" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;&lt;evaluation-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5571" PostTypeId="1" CreationDate="2018-03-08T06:46:48.417" Score="4" ViewCount="131" Body="&lt;p&gt;How does one prove the uniqueness of the value function obtained from &lt;a href=&quot;http://artint.info/html/ArtInt_227.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration&lt;/a&gt; in the case of bounded and undiscounted rewards? I know that this can be proven for the discounted case pretty easily using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Banach_fixed-point_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Banach fixed point theorem&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="13185" LastEditorUserId="1847" LastEditDate="2018-03-09T18:16:31.120" LastActivityDate="2018-08-13T01:00:26.170" Title="Proof of uniqueness of value function for MDPs with undiscounted rewards" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5572" PostTypeId="2" ParentId="113" CreationDate="2018-03-08T07:14:59.977" Score="5" Body="&lt;p&gt;&lt;strong&gt;Sigmoid &gt; Hyperbolic tangent:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you mentioned, the application of Sigmoid might be more convenient than hyperbolic tangent in the cases that we need a probability value at the output (as @matthew-graves says, we can fix this with a simple mapping/calibration step). In other layers, this makes no sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hyperbolic tangent &gt; Sigmoid:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hyperbolic tangent has a property called &quot;approximates identity near the origin&quot; which means &lt;span class=&quot;math-container&quot;&gt;$\tanh(0) = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\tanh'(0) = 1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\tanh'(z)$&lt;/span&gt; is continuous around &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt; (as opposed to  &lt;span class=&quot;math-container&quot;&gt;$\sigma(0)=0.5$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma'(0)=0.25$&lt;/span&gt;). This feature (which also exists in many other activation functions such as &lt;em&gt;identity&lt;/em&gt;, &lt;em&gt;arctan&lt;/em&gt;, and &lt;em&gt;sinusoid&lt;/em&gt;) lets the network learn efficiently even when its weights are initialized with small values. In other cases (eg &lt;em&gt;Sigmoid&lt;/em&gt; and &lt;em&gt;ReLU&lt;/em&gt;) these small initial values &lt;em&gt;can&lt;/em&gt; be problematic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Reading:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1412.6558&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Walk Initialization for Training Very Deep Feedforward Networks&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="12853" LastEditorUserId="22916" LastEditDate="2019-04-02T18:31:34.850" LastActivityDate="2019-04-02T18:31:34.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5574" PostTypeId="2" ParentId="5174" CreationDate="2018-03-08T09:16:23.443" Score="3" Body="&lt;p&gt;First thing you're going to want to add is probably a &lt;a href=&quot;https://en.wikipedia.org/wiki/Transposition_table&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transposition Table&lt;/a&gt;, as also suggested by SmallChess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterwards, I'd look into &lt;a href=&quot;https://www.chessprogramming.org/Aspiration_Windows&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aspiration Search&lt;/a&gt; and/or &lt;a href=&quot;https://chessprogramming.wikispaces.com/Principal%20Variation%20Search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principal Variation Search&lt;/a&gt; (also see &lt;a href=&quot;https://www.ics.uci.edu/~eppstein/180a/990202b.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I'd look into things like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Killer_heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Killer Move Heuristic&lt;/a&gt;, and maybe also see if you can simply implement existing parts of your engine more efficiently (e.g. use &lt;a href=&quot;https://chessprogramming.wikispaces.com/Bitboards&quot; rel=&quot;nofollow noreferrer&quot;&gt;bitboards&lt;/a&gt; for your state representation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than all of that, the &lt;a href=&quot;https://chessprogramming.wikispaces.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;chess programming wiki&lt;/a&gt; probably has lots of other interesting pages as well.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="-1" LastEditDate="2019-10-14T21:05:40.977" LastActivityDate="2019-10-14T21:05:40.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5576" PostTypeId="2" ParentId="5551" CreationDate="2018-03-08T11:32:09.517" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I think you are looking for &quot;Class Balancing&quot; if I understand your question correctly. In most frameworks you can pass an additional weights tensor/array to the loss function/criterion. This is essentially just a weighting factor for each loss term in the sum of the total loss. Through that you achieve that the error/loss for some classes are weighted higher than other classes. Usually you weight classes higher that are less present in your dataset to emphasize that these are not &quot;ignored&quot; by your network over other classes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Second what you can do if it's your data samples that are imbalanced and not the trained classes you can of course weight the data sampling of your dataloader. Instead of using a uniform distribution over all data samples you can weight specific samples higher or lower so that they are shown more/less often to the network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="13104" LastEditDate="2018-03-09T09:29:08.607" LastActivityDate="2018-03-09T09:29:08.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5577" PostTypeId="1" CreationDate="2018-03-08T12:54:00.040" Score="6" ViewCount="826" Body="&lt;p&gt;I am working on a js library which focuses on error handling. A part of the lib is a stack parser which I'd like to work in most of the environments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard part that there is no standard way to represent the stack, so every environment has its own stack string format. The variable parts are message, type and frames. A frame usually consists of called function, file, line, column. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some of the environments there are additional variable regions on the string, in others some of the variables are not present. I can run automated tests only in the 5 most common environments, but there are a lot more environments I'd like the parser to work in. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My goal is to write an adaptive parser, which learns the stack string format of the actual environment on the fly, and after that it can parse the stack of any exception of that environment. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I already have a plan how to solve this in the traditional way, but I am curious, &lt;strong&gt;is there any machine learning tool (probably in the topic of unsupervised learning) I could use to solve this problem?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the comments I need to clarify the terms &quot;stack string format&quot; and &quot;stack parser&quot;. I think it is better to write 2 examples from different environments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;example stack string:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Statement on line 44: Type mismatch (usually a non-object value used where an object is required)&#xA;Backtrace:&#xA;  Line 44 of linked script file://localhost/G:/js/stacktrace.js&#xA;    this.undef();&#xA;  Line 31 of linked script file://localhost/G:/js/stacktrace.js&#xA;    ex = ex || this.createException();&#xA;  Line 18 of linked script file://localhost/G:/js/stacktrace.js&#xA;    var p = new printStackTrace.implementation(), result = p.run(ex);&#xA;  Line 4 of inline#1 script in file://localhost/G:/js/test/functional/testcase1.html&#xA;    printTrace(printStackTrace());&#xA;  Line 7 of inline#1 script in file://localhost/G:/js/test/functional/testcase1.html&#xA;    bar(n - 1);&#xA;  Line 11 of inline#1 script in file://localhost/G:/js/test/functional/testcase1.html&#xA;    bar(2);&#xA;  Line 15 of inline#1 script in file://localhost/G:/js/test/functional/testcase1.html&#xA;    foo();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;stack string format (template):&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Statement on line {frames[0].location.line}: {message}&#xA;Backtrace:&#xA;{foreach frames as frame}&#xA;  Line {frame.location.line} of {frame.unknown[0]} {frame.location.path}&#xA;    {frame.calledFunction}&#xA;{/foreach}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;extracted information (json):&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    message: &quot;Type mismatch (usually a non-object value used where an object is required)&quot;,&#xA;    frames: [&#xA;        {&#xA;            calledFunction: &quot;this.undef();&quot;,&#xA;            location: {&#xA;                path: &quot;file://localhost/G:/js/stacktrace.js&quot;,&#xA;                line: 44&#xA;            },&#xA;            unknown: [&quot;linked script&quot;]&#xA;        },&#xA;        {&#xA;            calledFunction: &quot;ex = ex || this.createException();&quot;,&#xA;            location: {&#xA;                path: &quot;file://localhost/G:/js/stacktrace.js&quot;,&#xA;                line: 31&#xA;            },&#xA;            unknown: [&quot;inline#1 script in&quot;]&#xA;        },&#xA;        ...&#xA;    ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;B.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;example stack string:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ReferenceError: x is not defined&#xA;    at repl:1:5&#xA;    at REPLServer.self.eval (repl.js:110:21)&#xA;    at repl.js:249:20&#xA;    at REPLServer.self.eval (repl.js:122:7)&#xA;    at Interface.&amp;lt;anonymous&amp;gt; (repl.js:239:12)&#xA;    at Interface.EventEmitter.emit (events.js:95:17)&#xA;    at Interface._onLine (readline.js:202:10)&#xA;    at Interface._line (readline.js:531:8)&#xA;    at Interface._ttyWrite (readline.js:760:14)&#xA;    at ReadStream.onkeypress (readline.js:99:10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;stack string format (template):&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{type}: {message}&#xA;{foreach frames as frame}&#xA;{if frame.calledFunction is undefined}&#xA;    at {frame.location.path}:{frame.location.line}:{frame.location.column}&#xA;{else}&#xA;    at {frame.calledFunction} ({frame.location.path}:{frame.location.line}:{frame.location.column})&#xA;{/if}&#xA;{/foreach}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;extracted information (json):&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    message: &quot;x is not defined&quot;,&#xA;    type: &quot;ReferenceError&quot;,&#xA;    frames: [&#xA;        {&#xA;            location: {&#xA;                path: &quot;repl&quot;,&#xA;                line: 1,&#xA;                column: 5&#xA;            }&#xA;        },&#xA;        {&#xA;            calledFunction: &quot;REPLServer.self.eval&quot;,&#xA;            location: {&#xA;                path: &quot;repl.js&quot;,&#xA;                line: 110,&#xA;                column: 21&#xA;            }&#xA;        },&#xA;        ...&#xA;    ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The parser should process the stack strings and return the extracted information. The stack string format and the variables are environment dependent, the library should figure out on the fly how to parse the stack strings of the actual environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can probe the actual environment by throwing exceptions with well known stacks and check the differences of the stack strings. For example if I add a whitespace indentation to the line that throws the exception, then the column and probably the called function variables will change. If I detect a number change somewhere, then I can be sure that we are talking about the column variable. I can add line breaks too, which will cause line number change and so on...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can probe for every important variables, but I cannot be sure that the actual string does not contain additional unknown variables and I cannot be sure that all of the known variables will be added to it. For example the frame strings of the &quot;A&quot; example contain an unknown variable and do not contain the column variable, while the frame strings of the &quot;B&quot; example do not always contain the called function variable.&lt;/p&gt;&#xA;" OwnerUserId="13192" LastEditorUserId="13192" LastEditDate="2018-03-10T12:32:28.127" LastActivityDate="2020-01-03T23:12:58.960" Title="Is it possible to write an adaptive parser?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;javascript&gt;" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5578" PostTypeId="2" ParentId="5538" CreationDate="2018-03-08T13:46:57.650" Score="1" Body="&lt;p&gt;I understand that in your example you are interested in modelling the outcome of the 'result' column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One easy model I would suggest is to model it using the Bernoulli distribution (&lt;a href=&quot;https://en.wikipedia.org/wiki/Bernoulli_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Bernoulli_distribution&lt;/a&gt;) with the probability of success &lt;em&gt;p&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can model &lt;em&gt;p&lt;/em&gt; with something like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;x = a + b * log(page_views) + c * log(time_spent) + e&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p = exp(x) / (1+exp(x))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;em&gt;e&lt;/em&gt; is normally distributed,&#xA;e ~ N(0, sigma^2)&#xA;(or simply centered around zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a, b and c are parameters that you can estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I.e. the probability of success (conversion) is modeled as a sigma function of a certain variable that depends on page_views and time_spend. You can also add squares (and higher powers) of page_views and time_spend in the equation of &lt;em&gt;x&lt;/em&gt; (i.e. upto a certain threshold page_views can have a positive effect on conversion and then a negative one, then again a positive effect).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also reading about logistic regression should put you on the right track: &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Logistic_regression&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11417" LastEditorUserId="11417" LastEditDate="2018-03-08T13:56:55.037" LastActivityDate="2018-03-08T13:56:55.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5579" PostTypeId="2" ParentId="5527" CreationDate="2018-03-08T14:42:50.363" Score="1" Body="&lt;p&gt;&lt;strong&gt;The correct interpretation&lt;/strong&gt; (based on the comment by the author of the question below):&lt;/p&gt;&#xA;&lt;p&gt;Yep, you are right. It is actually only a single cell per object that contributes to the loss with &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}^\text{obj}_{ij}$&lt;/span&gt; factor. That cell is identified as the one that contains the centre(oid) of the ground truth box of the corresponding object.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My original (incorrect) interpretation of the paper:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;From what I read in the paper it is not a single bounding box &lt;em&gt;for&lt;/em&gt; &lt;em&gt;all&lt;/em&gt; cells in the grid for a particular object (your original suggestion).&lt;/p&gt;&#xA;&lt;p&gt;Rather for every object and every cell that is assigned to this object only one bounding box contributes to the loss. The network generates &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; bounding boxes for every cell in the grid, but we pick only one and only when that cell actually belongs to an object.&lt;/p&gt;&#xA;&lt;p&gt;Every cell &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, which is not a background cell, will have exactly one box &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}^\text{obj}_{ij} = 1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is based on the following paragraph from the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note that the loss function only penalizes classification error if an object is present in that grid cell (hence the conditional class probability discussed earlier). It also only penalizes bounding box coordinate error if that predictor is &amp;quot;responsible&amp;quot; for the ground truth box &lt;strong&gt;(i.e. has the highest IOU of any predictor in &lt;em&gt;that&lt;/em&gt; grid cell).&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="11417" LastEditorUserId="2444" LastEditDate="2021-01-28T23:12:26.270" LastActivityDate="2021-01-28T23:12:26.270" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5580" PostTypeId="1" AcceptedAnswerId="5620" CreationDate="2018-03-08T16:22:55.040" Score="7" ViewCount="6744" Body="&lt;p&gt;I am trying to understand backpropagation. I used a simple neural network with one input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, one hidden layer &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and one output layer &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, with weight &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; connecting &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; connecting &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x \rightarrow (w_1) \rightarrow h \rightarrow (w_2) \rightarrow y&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my understanding, these are the steps happening while we train a neural network:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The feedforward step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;h=\sigma\left(x w_{1}+b\right)\\&#xA;y^{\prime}=\sigma\left(h w_{2}+b\right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L=\frac{1}{2} \sum\left(y-y^{\prime}\right)^{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient calculation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial w_{2}}=\frac{\partial y^{\prime}}{\partial w_{2}} \frac{\partial L}{\partial y^{\prime}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial w_{1}}=?$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weight update&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{i}^{t+1} \leftarrow w_{i}^{t}-\alpha \frac{\partial L}{\partial w_{i}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understood most parts of backpropagation, but how do we get the gradients for the middle layer weights &lt;span class=&quot;math-container&quot;&gt;$dL/dw_1$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should we calculate the gradient of a network similar to this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P5fce.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P5fce.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the correct equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial w_{1}}=\frac{\partial h_{1}}{\partial w_{1}} \frac{\partial w_{7}}{\partial h_{1}} \frac{\partial o_{2}}{\partial w_{7}} \frac{\partial L}{\partial o_{2}}+\frac{\partial h_{1}}{\partial w_{1}} \frac{\partial w_{5}}{\partial h_{1}} \frac{\partial o_{1}}{\partial w_{5}} \frac{\partial L}{\partial o_{1}}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2020-05-18T01:34:21.203" LastActivityDate="2020-05-18T01:34:21.203" Title="How is the gradient calculated for the middle layer's weights?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5581" PostTypeId="2" ParentId="5553" CreationDate="2018-03-08T20:13:48.133" Score="1" Body="&lt;p&gt;I understand that you want to know about methods that we can use to evaluate GANs (Generative Adversarial Networks).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;How can GANs be evaluated?&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;One Discriminator on Separate GANs&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;We can train a &lt;em&gt;Discriminator&lt;/em&gt; beforehand and then we can use this &lt;em&gt;Discriminator&lt;/em&gt; on various &lt;em&gt;Generators&lt;/em&gt; to see what does this &lt;em&gt;Discriminator&lt;/em&gt; say about the images generated from each &lt;em&gt;Generator&lt;/em&gt;. The average output of the &lt;em&gt;Discriminator&lt;/em&gt; for images generated from one &lt;em&gt;Generator&lt;/em&gt; can be compared with that of the images generated from another &lt;em&gt;Generator&lt;/em&gt;. This means that according to this &lt;em&gt;Discriminator&lt;/em&gt;, one &lt;em&gt;Generator&lt;/em&gt; is better than another.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Comparing Probabilistic Models&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is the class of evaluation metrics that is attracting research attention recently. Basically, all generative models are probabilistic in nature, even GANs. When we ask a generative model to generate something like an image, we are simply sampling from a probability distribution. This means that if we can compare the probability distributions of our &lt;em&gt;Generator&lt;/em&gt; and that of our original data then we will have an evaluation metric. We can come up with a kernel function to define these probability distributions more accurately. Then we can take our generated samples from the &lt;em&gt;Generator&lt;/em&gt; and see the probability of that sample being taken from the original distribution. Researchers have also utilized &lt;em&gt;KL-divergence&lt;/em&gt; for comparing samples generated from two distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note: KL-divergence can not be directly applied in this case because corresponding ground truth is not available. However, there are some modifications proposed in the research work mentioned.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Recommended Readings&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1511.01844.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A NOTE ON THE EVALUATION OF GENERATIVE MODELS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.iro.umontreal.ca/~lisa/publications/unlearning_for_better_mixing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;UNLEARNING FOR BETTER MIXING&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://openreview.net/pdf?id=SJgabgBFl&quot; rel=&quot;nofollow noreferrer&quot;&gt;A QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12957" LastEditorUserId="12957" LastEditDate="2018-03-08T20:33:55.370" LastActivityDate="2018-03-08T20:33:55.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5582" PostTypeId="1" CreationDate="2018-03-08T21:13:19.673" Score="1" ViewCount="69" Body="&lt;p&gt;In multivariate linear regression (linear regression with more than one variable) the model is &lt;span class=&quot;math-container&quot;&gt;$yi = b_0 + b_1x_{1i} + b_2x_{2i} + ...$&lt;/span&gt; , and so on. But how is the &lt;span class=&quot;math-container&quot;&gt;$b_n$&lt;/span&gt; value calculated iteratively? Can it be calculated non-iteratively? What is the intuition behind using that method to calculate &lt;span class=&quot;math-container&quot;&gt;$b_2$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="13202" LastEditorUserId="2444" LastEditDate="2019-04-01T13:43:29.063" LastActivityDate="2019-04-02T18:31:08.577" Title="In the multi-linear regression, how is the value of weight $b_2$ calculated?" Tags="&lt;machine-learning&gt;&lt;learning-algorithms&gt;&lt;linear-regression&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5583" PostTypeId="2" ParentId="3374" CreationDate="2018-03-08T22:27:45.270" Score="4" Body="&lt;p&gt;Here are some resources I have found useful to get to know the basics of AI&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=UzxYlbK2c7E&amp;amp;list=PLA89DCFA6ADACE599&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's lecture series on AI&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=21EiKfQYZXc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's lecture at the Stanford Business School&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=NKpuX_yzdYs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng - The State of Artificial Intelligence&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Andrew Ng is a visiting professor at Stanford, founder of Coursera and currently the head of research at Alibaba. The above videos should give you (all) the basics you need about AI.&lt;/p&gt;&#xA;" OwnerUserId="13204" LastEditorUserId="2444" LastEditDate="2019-04-08T21:31:10.420" LastActivityDate="2019-04-08T21:31:10.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5584" PostTypeId="2" ParentId="3007" CreationDate="2018-03-08T22:40:39.310" Score="1" Body="&lt;p&gt;I have been in the software testing industry for over 11 years now and I can say for sure that there are different ways people are using AI for software testing. This is especially true in the area of automated testing tools. Different vendors have been trying to tackle some of the common problems in test automation using AI. Some of them are-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://appvance.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Appvance&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Appvance uses AI to generate test cases based on user behavior but is not a fully AI based tool like Testim.io&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://test.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Test.ai&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test.ai uses artificial intelligence to perform regression testing. It is helpful to get performance metrics on your app. It is more of an app monitoring tool than functional testing tool from my point of view&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.functionize.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Functionize&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Functionize used machine learning for functional testing. It is very similar to other tools in the market in terms of its capabilities&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above are some of the popular tools out there in the market.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trend seems to be going in the positive direction in terms of vendors trying to make testing more stable, simpler, smarter and getting everyone in the team involved in testing including non-technical people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will be just a matter of time that more solutions come up for software testing using AI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Raj&lt;/p&gt;&#xA;" OwnerUserId="13204" LastEditorUserId="13204" LastEditDate="2019-11-30T22:17:15.587" LastActivityDate="2019-11-30T22:17:15.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5585" PostTypeId="2" ParentId="5582" CreationDate="2018-03-08T23:20:24.157" Score="2" Body="&lt;p&gt;It is calculated the same way &lt;span class=&quot;math-container&quot;&gt;$b_1$&lt;/span&gt; is calculated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nearly following your notation, say your multiple linear regression function is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$H(X_i) =  b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for data instance &lt;span class=&quot;math-container&quot;&gt;$X_i=x_{1,i},...,x_{n, i}$&lt;/span&gt;&#xA;and weights &lt;span class=&quot;math-container&quot;&gt;$b_0,...,b_n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And say your error function is &lt;span class=&quot;math-container&quot;&gt;$E(X,Y) = \sum_i(H(X_i)-Y_i)^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is the collection of all data points &lt;span class=&quot;math-container&quot;&gt;$X_i, Y_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your error function &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, for whatever weights you have (with a gradient based method), calculate the partial derivative&#xA;&lt;span class=&quot;math-container&quot;&gt;$\partial E /\partial b_i$&lt;/span&gt; and use this to update all of your weights at once in each iteration of your optimization routine.&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="22916" LastEditDate="2019-04-02T18:31:08.577" LastActivityDate="2019-04-02T18:31:08.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5586" PostTypeId="1" CreationDate="2018-03-08T23:48:42.083" Score="4" ViewCount="301" Body="&lt;p&gt;When one uses NEAT to evolve the best fitting network for a task, does training take place in each epoch as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, training is the adjustment of the weights of the neural network via backpropagation and gradient descent. During NEAT, say, a generation runs for 1000 iterations. During that time, is there any training involved, or does each genome randomly poke around and the winner takes it to the next stage?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used NEAT, but the fact that neural networks are not trained does not make sense to me. At the same time, I can't find any code in my framework (&lt;a href=&quot;https://wagenaartje.github.io/neataptic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neataptic.js&lt;/a&gt;) that would train the generation during the epoch.&lt;/p&gt;&#xA;" OwnerUserId="13138" LastEditorUserId="2444" LastEditDate="2019-10-13T01:18:33.357" LastActivityDate="2019-10-13T01:18:33.357" Title="Does training happen during NEAT?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5588" PostTypeId="2" ParentId="4979" CreationDate="2018-03-09T06:00:48.133" Score="1" Body="&lt;p&gt;Neural networks are used to visualize high dimensional data through the use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;autoencoding&lt;/a&gt;. It's similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Principal_component_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principal Component Analysis&lt;/a&gt; and is regarded to perform better then PCA. Autoencoding will take your data and convert it to a 2 or 3-dimensional representation. Since you have an array of data you might want to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt;. You will have to make sure you reset the LSTM states when you perform the reverse pass. You will probably have trouble interpreting the autoencoded form of the data though, so that will be a task of its own.&lt;/p&gt;&#xA;" OwnerUserId="13208" LastEditorUserId="1671" LastEditDate="2018-03-09T18:02:56.037" LastActivityDate="2018-03-09T18:02:56.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5590" PostTypeId="2" ParentId="3469" CreationDate="2018-03-09T06:34:43.637" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;To represent the pieces, you should be able to use a single input matrix. Just designate an integer number for the different types of pieces. White stones can be positive integers and black stones can be negative.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use sigmoid for board position confidence and linear activation for piece identifier. &lt;strong&gt;pass&lt;/strong&gt; would be another sigmoid output. I don't think you'll have to worry about &lt;strong&gt;pass&lt;/strong&gt; being diluted. Since it is such a valuable action, the score will depend a lot on the &lt;strong&gt;pass&lt;/strong&gt; output and it will have a large gradient. If you need to select the &lt;strong&gt;pass&lt;/strong&gt; action with high frequency for reinforcement learning purposes, then just attribute a higher probability to the &lt;strong&gt;pass&lt;/strong&gt; action in your random choice function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The final score difference has a large impact on the desirability of the moves. A large score difference should result in a large impact on the function. Therefore you might want to include the magnitude of score difference in your loss function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is the type of job that Deep Q Learning does. Perhaps you'll want to look into that too.&lt;/p&gt;&#xA;" OwnerUserId="13208" LastActivityDate="2018-03-09T06:34:43.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5591" PostTypeId="2" ParentId="4698" CreationDate="2018-03-09T06:52:44.523" Score="2" Body="&lt;p&gt;You might consider pre-training a CNN on a large dataset. The CNN should be structured such that you input 2 different images and the CNN predicts whether or not they are the same person. Your dataset should include images from multiple angles, with and without occlusions like sunglasses, and with changes in hair. (One dataset useful for this is the &lt;a href=&quot;http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AR Face Database&lt;/a&gt;.) Then you can just check if the face matches any of the pictures you took as they entered the store.&lt;/p&gt;&#xA;" OwnerUserId="13208" LastEditorUserId="13208" LastEditDate="2018-03-09T22:14:09.190" LastActivityDate="2018-03-09T22:14:09.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5592" PostTypeId="2" ParentId="3802" CreationDate="2018-03-09T07:07:32.840" Score="-1" Body="&lt;p&gt;If you have lots of rules then you should be able to come up with lots of heuristics for A*. If I were you I would try A* first, and come up with as many heuristics as I could. You can also use Deep Q learning. I don't think you want to just throw coordinates into a Deep Q network. You want a simpler representation. I would probably give the nodes symbolic identifiers like sparse matrices, integers or sigmoids, and then craft the loss function to reflect the actual cost of moving from node to node and ascribe a high cost to illegal moves.&lt;/p&gt;&#xA;" OwnerUserId="13208" LastActivityDate="2018-03-09T07:07:32.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5593" PostTypeId="1" AcceptedAnswerId="5603" CreationDate="2018-03-09T07:29:44.073" Score="5" ViewCount="1092" Body="&lt;p&gt;What are the advantages/ strengths and disadvantages/weakness of programming languages like Common Lisp, Python and Prolog? Why are these languages used in the domain of artificial intelligence? What type of problems related to AI are solved using these languages? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, give me link to papers or books regarding the mentioned topic.&lt;/p&gt;&#xA;" OwnerUserId="12803" LastEditorUserId="2444" LastEditDate="2019-03-01T20:46:15.730" LastActivityDate="2019-03-01T20:46:15.730" Title="Why is Common Lisp, Python and Prolog used in artificial intelligence?" Tags="&lt;programming-languages&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="5" ClosedDate="2020-05-13T20:11:39.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="5594" PostTypeId="2" ParentId="4979" CreationDate="2018-03-09T08:16:37.207" Score="1" Body="&lt;p&gt;I doubt that this problem is a good application of NN. Take a look at this &lt;a href=&quot;https://en.wikipedia.org/wiki/Force-directed_graph_drawing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Force-directed_graph_drawing&lt;/a&gt;. And try Graphviz if you haven't already. Drawing graphs in a meaningful way is notoriously difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative approach would be to embed all nodes into a 2d or 3d metric space. I.e. for each node you will have coordinates of a point in such space. If you have certain rich descriptors of the nodes (i.e. some kind of information associated with each node, pictures, text, vectors of numbers) that can be done even with a NN. To have a good embedding you need more info than just node degrees or/and edge weights.&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2018-03-09T08:16:37.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5595" PostTypeId="2" ParentId="3469" CreationDate="2018-03-09T08:51:20.653" Score="1" Body="&lt;p&gt;You don't need conv layers, since you don't feed a picture as an input (see below). Alternatively, you can try using a picture of the board (with different pieces having different shapes). This can work too. Then I would go for 2 conv layers, stride 1, kernel size equal to half a piece size. I would try it with a single max pooling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike the other answer i would suggest using a 3d tensor as an input, with the number of channels equal to different pieces. The other two dimensions equal would correspond to number of cells on the board. Various transformation in you NN will not be able to distinguish between multiple integers very well. That's why it is better to have a one-hot encoding of the pieces' types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would use only a vector with n+1 components for output: n for all possible moves, and 1 for the pass. It would encode the expected reward for each move, not the probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not sure what you mean by enforcing moves. But when you going to train it with something like Q-learning it would make sense making a completely random move every once and a while with a certain probability (say 10% of the times). Lookup &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Reinforcement_learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2018-03-09T08:51:20.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5600" PostTypeId="2" ParentId="5582" CreationDate="2018-03-09T10:13:39.577" Score="-1" Body="&lt;p&gt;put your b_1, ... ,b_n coefficients into a vector &lt;em&gt;b&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;put all your x_{ij} into a matrix X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then all components of b are calculated at the same time with this equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b = (H(X) - Xb)^T (H(x) - Xb)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this calculation (estimation) is only consistent (search consistent estimator) when certain assumptions are present (read here &lt;a href=&quot;https://en.wikipedia.org/wiki/Ordinary_least_squares&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Ordinary_least_squares&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2018-03-09T10:13:39.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5601" PostTypeId="1" AcceptedAnswerId="5621" CreationDate="2018-03-09T10:45:31.297" Score="10" ViewCount="1681" Body="&lt;p&gt;Currently, the most commonly used activation functions are ReLUs. So I answered this question &lt;a href=&quot;https://ai.stackexchange.com/a/5521/2444&quot;&gt;What is the purpose of an activation function in neural networks?&lt;/a&gt; and, while writing the answer, it struck me, &lt;em&gt;how exactly can ReLUs approximate a non-linear function?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;By pure mathematical definition, sure, it's a non-linear function due to the sharp bend, but, if we confine ourselves to the positive or the negative portion of the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis only, then it's linear in those regions. Let's say we take the whole &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis also, then also it's kinda linear (not in a strict mathematical sense), in the sense that it cannot satisfactorily approximate curved functions, like a sine wave (&lt;span class=&quot;math-container&quot;&gt;$0 \rightarrow 90$&lt;/span&gt;) with a single node hidden layer as is possible by a sigmoid activation function.&lt;/p&gt;&#xA;&lt;p&gt;So, &lt;em&gt;what is the intuition behind the fact that ReLUs are used in NNs, giving satisfactory performance? Are non-linear functions, like the sigmoid and the tanh, thrown in the middle of the NN sometimes?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am &lt;em&gt;not&lt;/em&gt; asking for the purpose of ReLUs, even though they are kind of linear.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As per @Eka's comment, the ReLu derives its capability from discontinuity acting in the deep layers of the NN. Does this mean that ReLUs are good, as long as we use it in deep NNs and not a shallow NN?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-09-24T10:38:32.343" LastActivityDate="2020-09-24T10:38:32.343" Title="How exactly can ReLUs approximate non-linear and curved functions?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5602" PostTypeId="1" AcceptedAnswerId="5648" CreationDate="2018-03-09T12:46:33.607" Score="1" ViewCount="331" Body="&lt;p&gt;I'm a software developer who keeps trying (and failing) to get my head around AI and neural networks. There is one area that sparked my interest recently - simulating a mouse &quot;homing in&quot; on a piece of cheese by following the smell. Based on the rule that moving closer to the cheese = stronger smell = good, then it feels like it should be quite a simple problem to solve - in theory at least!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thought process was to start by placing the mouse and cheese in random positions on the screen. I would then move the mouse one step in a random direction and measure its distance to the cheese, and if it's closer than before (stronger smell) then that's good. This is where I come unstuck on the theory - this &quot;feedback&quot; somehow needs to modify the mechanism used to move the mouse, gradually refining it until the mouse is able to head straight towards the cheese.&#xA;Once &quot;trained&quot;, I should be able to reposition the cheese and expect the mouse to travel to it more quickly. &lt;em&gt;Note I'm also keeping things simple by not having obstacles for the mouse to negotiate around.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How on earth would this be implemented with a NN? I understand the basic concepts, but I find that things unravel once I start looking at real code! The examples I've seen typically start by training the NN from a data set, but this doesn't seem to apply here as it feels like the only training available is &quot;on the fly&quot; as the mouse moves around (i.e. closer = good, further away = bad). I'm assuming the brain has some kind of &quot;reward mechanism&quot; triggered by a stronger smell of cheese.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I barking up the wrong tree - either with my thought process, or NN not being a good fit for this problem? This isn't homework btw, just something that I've been puzzling over in the back of my mind.&lt;/p&gt;&#xA;" OwnerUserId="13216" LastEditorUserId="1671" LastEditDate="2018-03-09T18:38:37.207" LastActivityDate="2018-03-12T16:59:26.180" Title="Neural network to control movement and &quot;home in&quot; on a target" Tags="&lt;neural-networks&gt;&lt;getting-started&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5603" PostTypeId="2" ParentId="5593" CreationDate="2018-03-09T18:58:18.777" Score="3" Body="&lt;p&gt;If we talk about applied AI, the choice of a programming language for an AI application has the same points to be taken into account that in any other software area: speed of generated code, expressive, reusable, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By example, as training of a neural net is very expensive in CPU, languages as C/C++, that produces very optimized code, are very convenient. Moreover, there are GPUs librarians in C/C++ that allows use of strong parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A system with some complexity will combine more than one language&lt;/strong&gt;, in order to use the best of the language in the points where it is need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But returning to the list of languages that appears in the question. As all them are Turing complete compare them means talk about its paradigm, features, syntax and available compilers/interpreters. Obviously, something that exceeds the possibilities of a simple answer. Just to show some key points about the ones mentioned in the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Prolog&lt;/strong&gt; is a programming paradigm by itself. Its main advantage was that Prolog sentences are independent from remainder ones and near to the mathematical definitions of the concepts. Moreover, it is itself a database. Its drawback are also well known: very slow, lack of librarians for i/o, ... . Very interesting (even mandatory) to known a few examples of algorithms in Prolog, but I doubt nobody is using it nowadays, except in obsolete university courses (when you reach the &quot;!&quot;, cut its study).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Lisp&lt;/strong&gt; is also a zombie. Its functional paradigm has been now included in lots of very more modern languages, combining it with object oriented paradigm: scala, haskell, ocaml/F#, ... . Being functional allows a syntax that made easier to express logic concepts as recursive definition of logic or types, ... . Something very interesting in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the category of object oriented paradigm and valid for all applications, we have &lt;strong&gt;Python&lt;/strong&gt; ( easy to learn, fast prototyping, slow, ... ) C/C++ (very optimized code), Java, ... . More or less, all them are adopting also functional features in latest standards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In AI there are a lot of very interesting language features to be also considered: rule based systems, ... . Librarians for them can be found in all main languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, some words about AGI (strong AI): you do not need a computer. In best moments, we are at the stage of pencil and paper, remainder ones looking at ceiling.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-03-10T09:25:38.747" LastActivityDate="2018-03-10T09:25:38.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5604" PostTypeId="2" ParentId="5586" CreationDate="2018-03-09T19:38:08.070" Score="3" Body="&lt;p&gt;NEAT uses genetic algorithms both to search for improved connection weights and for improved architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst it is &lt;em&gt;possible&lt;/em&gt; to train a NEAT-generated neural network using backpropagation of error gradients, libraries implementing &quot;original&quot; NEAT will not implement that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is often no training data, in a supervised learning sense. The fitness function of a NEAT system can be measured arbitrarily by performance at some task, and in the general case this consists of some &lt;em&gt;environment&lt;/em&gt; simulation that interacts with an agent controlled by the NN, and not training data per se.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Evolved network topologies do not conform to stacked layers models preferred by ML frameworks designed to run typical deep learning architectures.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both of these issues can be resolved with a little effort (for instance for the second issue there are frameworks which will work with arbitrary feed forward connection graphs, they are just a little more niche than e.g. Keras). However, NEAT is often used because it can solve problems without needing to frame them as supervised learning or reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than the hard work of putting it together, there is nothing stopping you creating a train-on-data stage which could alternate between the two approaches, perhaps with controllable weighting from evolution to gradient-based training. To add the gradient-based training, then either:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Your original problem is one of fitting to a classifier or regression problem. In which case your fitness function and training loss function could be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Your original problem is one of controlling an agent in an environment, and you could potentially use something like the REINFORCE algorithm based on most-recent assessments, in order to provide gradients to train the NN. Other policy gradient methods could also be a good fit, as the NEAT network typically outputs a policy as opposed to a value prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never tried these (nor much with NEAT at all other than demos and a bit of theory). For (a) I would expect the combination to be successful, but wonder why you were bothering with NEAT in the first place. For (b) I am less sure whether you would get useful results, because REINFORCE relies on multiple runs with the &lt;em&gt;same&lt;/em&gt; network, whilst NEAT relies on stochastic search across multiple networks. Applying REINFORCE training across a whole population could be very CPU intensive.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-03-09T19:53:14.837" LastActivityDate="2018-03-09T19:53:14.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5605" PostTypeId="2" ParentId="3374" CreationDate="2018-03-09T19:38:09.620" Score="3" Body="&lt;p&gt;Before getting into Artificial Intelligence, one should be done with the prerequisites. There is not a solid list, but a good knowledge of various algorithms is compulsory. Apart from that you should be comfortable with at least one programming language, like C++ or Java. I won’t suggest you to dive into Artificial Intelligence if you are completely new to Computer Science. Some experience with programming prior to diving into Artificial Intelligence will be a plus point for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start reading (blogs, papers, scholar articles, etc.) about Artificial Intelligence. Like what it is, its applications, current status and other stuff that you can find. Start making AI codes for small games like Tic Tac Toe, Sudoku, Reversi (Othello), etc. for the start. You can create your own simulator and build a code that solves Rubik cube. Similarly, make codes for Pattern Recognition and Machine Learning. Nothing is better than learning by doing. Languages like LISP and python will be very helpful. Here are two answers that will help you, &lt;a href=&quot;https://ai.stackexchange.com/questions/2236/why-is-lisp-such-a-good-language-for-ai/2238#2238&quot;&gt;ans1&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/questions/3494/why-is-python-the-most-popular-language-in-the-ai-field/3503#3503&quot;&gt;ans2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are a person likes like to read and learn from books (like me), then you can buy Artificial Intelligence: A Modern Approach (Peter Norvig and Stuart Russell). The book is very good and works well for the intermediate and advanced level. Try to solve the exercise problems given in the book. The solution pdf of the books is available &lt;a href=&quot;http://qr.ae/TU8juA&quot; rel=&quot;nofollow noreferrer&quot;&gt;online&lt;/a&gt;. For Machine Learning two books that I recommend is Pattern Recognition and Machine Learning (Christopher M. Bishop) and Programming Collective Intelligence (O’Reilly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the start, there is a very good &lt;a href=&quot;https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; on Artificial Intelligence and Technological Singularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article is long and divided into two parts. I strongly recommend you to read this article if you are serious about Artificial Intelligence. It will give you some good insights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowledge of Computational Theory will greatly help you. Especially when you are working in the field of Natural Language Processing. Other sub-fields of AI that might interest you will be Machine Learning, Evolutionary Computing, Genetic Algorithms, Reinforcement Learning, Deep Learning etc. The list goes on.&#xA;Better your knowledge of Statistics, better it will be for Artificial Intelligence. Stay tuned to recent goings in the field via forums, websites, etc. Open AI &lt;a href=&quot;https://openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt; is also a very good source.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="3005" LastEditDate="2018-03-09T22:33:01.543" LastActivityDate="2018-03-09T22:33:01.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5606" PostTypeId="1" AcceptedAnswerId="5607" CreationDate="2018-03-09T19:41:52.757" Score="3" ViewCount="243" Body="&lt;p&gt;I'm having trouble wrapping my head around some details of neural nets and back prop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example's sake, consider the following net, where I have separated the 'neurons' into linear nodes plus activation (in this case sigmoid) nodes, more like a general computation graph. L2 is the squared loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xz0CA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xz0CA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a 3 part question (parts 1 and 2 disregarding linear algebra / vectorization). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; I want to confirm that I cannot just apply chain rule all the way from the L2 to the inputs and get the derivatives for weights at different layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wE4t7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wE4t7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E1PG9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E1PG9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For w5, at o1 I took the derivative with respect to w5. But for w1, at o1 I had to instead take the derivative with respect to ah1 to keep moving backwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; I want to confirm that at any node that branches into more than one node at forward time (i.e. ah1), when I back prop, I have to add the derivatives of all it's branches: in this case do1/dah1 and do2/ah1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; If these 2 things are as I say, then how can I implement backprop in vectorized + linear algebra way without branching or using conditional logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The forward pass is easy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;x1 + x2 are a single input vector.&lt;/li&gt;&#xA;&lt;li&gt;h1 + h2 are a matrix of 2 rows (number of output / units) and 2 columns (number of inputs).&lt;/li&gt;&#xA;&lt;li&gt;ah1 + ah2 are a single function that can operate on vectors (i.e. 1/(1 + np.exp()))&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The same for the next layer. So to do a forward pass, I just do: &#xA;L2(y, a_o(W_o @ a_h(W_h @ x))) (@ = matrix mul)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for the back pass not so much, since I'd have to check which derivative to use at o1 and I'd have to sum o1 and o2 at ah1 and ah2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://neuralnetworksanddeeplearning.com/chap2.html&lt;/a&gt; this shows that in theory it can be done by always transposing the previous weight matrix, but I haven't fully understood it yet.&lt;/p&gt;&#xA;" OwnerUserId="12773" LastEditorUserId="12773" LastEditDate="2018-03-10T06:23:58.170" LastActivityDate="2018-03-10T06:23:58.170" Title="A few doubts on back propagation" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5607" PostTypeId="2" ParentId="5606" CreationDate="2018-03-10T02:37:49.613" Score="1" Body="&lt;p&gt;So as far as I understood your problem is of mathematical nature, especially about back-propagation derivatives. Just a quick clarification, the squared loss function is not the only cost function, there might be other types of cost function. Anyways, coming to the first part of the question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can just actually apply chain rule directly from L2 to wherever you want. The weights are independent variables, they don't depend on each other. So &lt;code&gt;dw5/dw1&lt;/code&gt; will turn out to be 0 i.e you are differentiating a constant, in this case &lt;code&gt;w5&lt;/code&gt; with respect to &lt;code&gt;w1&lt;/code&gt;. Why do we write this &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UZbtd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UZbtd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead of this &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2AG23.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2AG23.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is because we can differentiate the loss function L2 w.r.t to a single independent variable only (of our choice), you cannot differentiate it w.r.t to an independent variable and still go on differentiating w.r.t to another independent variable. Its purposeless since you want to find the variation of L2 w.r.t to a particular weight only.&#xA;Here are 2 links for better understanding of partial derivatives:&#xA;&lt;a href=&quot;https://math.stackexchange.com/questions/423/intuitive-reasoning-behind-the-chain-rule-in-multiple-variables/426&quot;&gt;Intuitive reasoning behind the Chain Rule in multiple variables?&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/questions/62614/chain-rule-intuition&quot;&gt;Chain Rule Intuition&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Yes, you certainly have to add the derivatives of all the branches. Basically first we look from the frame of reference of &lt;code&gt;o1&lt;/code&gt; and &lt;code&gt;o2&lt;/code&gt;. From this frame &lt;code&gt;o1&lt;/code&gt; and &lt;code&gt;o2&lt;/code&gt; are independent variables and L2 is dependent on them and so we differentiate L2 w.r.t them. Then we move a step back and check what is &lt;code&gt;o1&lt;/code&gt; and &lt;code&gt;o2&lt;/code&gt; dependent on. How you go about doing this simply experience and an understanding of the thing you are doing. There is no steadfast algorithm from a simple point of view.&lt;/li&gt;&#xA;&lt;li&gt;The vectorized implementation requires you follow a certain framework, it will be very lengthy here to explain the framework, but I am linking you to certain lecture videos which will make you aware of the convention being followed for vectorized implementation. Just check &lt;a href=&quot;https://www.coursera.org/learn/neural-networks-deep-learning/home/week/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;week 2&lt;/a&gt; and &lt;a href=&quot;https://www.coursera.org/learn/neural-networks-deep-learning/home/week/3&quot; rel=&quot;nofollow noreferrer&quot;&gt;week 3&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also the thing I was talking about having different L2's in the begining is important in the sense is that in the future we may come up with weights which are dependent on other weights and so now we may have to do a &lt;code&gt;dw_n/dw_m&lt;/code&gt; step since &lt;code&gt;w_n&lt;/code&gt; is dependent on &lt;code&gt;w_m&lt;/code&gt;, but lets keep that for another day.&#xA;Hope this helps!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: this might give you a sense of why I am  calling variables as independent variables &lt;a href=&quot;https://ai.stackexchange.com/questions/3962/how-do-i-know-if-my-backpropagation-is-implemented-correctly/3996#3996&quot;&gt;How do I know if my back-propagation is implemented correctly?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-10T04:08:49.807" LastActivityDate="2018-03-10T04:08:49.807" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5608" PostTypeId="5" CreationDate="2018-03-10T03:24:15.273" Score="0" Body="&lt;p&gt;An artificial neuron is a mathematical function inspired by, but does not actually model, biological neurons in mammalian brains. It can be thought of as a function parameterized by a set of weights. These weights are the values that are updated during the update step of a learning algorithm.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="4398" LastEditDate="2018-03-11T02:00:18.153" LastActivityDate="2018-03-11T02:00:18.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5609" PostTypeId="4" CreationDate="2018-03-10T03:24:15.273" Score="0" Body="For questions about what constitutes an artificial neuron and how artificial neurons can be utilized as part of a neural network." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-14T21:33:36.313" LastActivityDate="2018-03-14T21:33:36.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5610" PostTypeId="5" CreationDate="2018-03-10T03:26:58.980" Score="0" Body="&lt;p&gt;Mathematics is central to Artificial Intelligence.  Feasibility study, capacity planing, evaluation of algorithms, the reasonability of research directions, and evaluation of the effectiveness of methods in real world use all rely on rigorous mathematical treatment.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-10-15T16:56:14.800" LastActivityDate="2018-10-15T16:56:14.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5611" PostTypeId="4" CreationDate="2018-03-10T03:26:58.980" Score="0" Body="For questions about mathematics related to artificial intelligence." OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-10-15T16:57:50.377" LastActivityDate="2018-10-15T16:57:50.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5612" PostTypeId="5" CreationDate="2018-03-10T03:28:49.233" Score="0" Body="&lt;p&gt;Javascript is an object-oriented computer programming language commonly used to create interactive effects within web browsers.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-11T02:04:30.833" LastActivityDate="2018-03-11T02:04:30.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5613" PostTypeId="4" CreationDate="2018-03-10T03:28:49.233" Score="0" Body="For questions about implementation of AI/Ml programs in the javascript programming language." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-11T01:55:22.573" LastActivityDate="2018-03-11T01:55:22.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5614" PostTypeId="1" CreationDate="2018-03-10T07:06:25.170" Score="0" ViewCount="98" Body="&lt;p&gt;I was trying to build a prediction system where I have the input data arranged in multiple columns. The input data would be of the type where I have&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;weather,&lt;/li&gt;&#xA;&lt;li&gt;service type (bronze, silver, gold),&lt;/li&gt;&#xA;&lt;li&gt;size (xs, s, m, l, xl, xxl),&lt;/li&gt;&#xA;&lt;li&gt;time,&lt;/li&gt;&#xA;&lt;li&gt;availability,&lt;/li&gt;&#xA;&lt;li&gt;pin code, and&lt;/li&gt;&#xA;&lt;li&gt;the result (target).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Each of the data types is arranged in columns with a specific code. I have read &lt;a href=&quot;https://stackoverflow.com/q/48385376/3924118&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/q/33369706/3924118&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/q/44613753/3924118&quot;&gt;this&lt;/a&gt; , &lt;a href=&quot;https://stackoverflow.com/q/29625263/3924118&quot;&gt;this&lt;/a&gt;, and &lt;a href=&quot;https://stackoverflow.com/q/3712863/3924118&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;They are helpful but do not give me a clear picture. I would like to achieve a multi-vs-one prediction. Most of the schemes available are one-vs-one where the data is a 1*1 entity.&lt;/p&gt;&#xA;&lt;p&gt;Here is a sample code that I was working with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;regressionModel = linear_model.LinearRegression()&#xA;    &amp;quot;&amp;quot;&amp;quot; 3. Processing is not necessary for current concept &amp;quot;&amp;quot;&amp;quot;&#xA;    y = pd.DataFrame(modifiedDFSet['Code'])&#xA;    print(y.shape)&#xA;    drop2 = ['Code']&#xA;    X = modifiedDFSet.drop(drop2)&#xA;    print(X.shape)&#xA;    &amp;quot;&amp;quot;&amp;quot; 4. Data Scaling, Data Imputation is not necessary. Training and Test data is prepared using train-test-split &amp;quot;&amp;quot;&amp;quot;&#xA;    train_data, test_data = train_test_split(X, test_size=0.20, random_state=42)&#xA;    &amp;quot;&amp;quot;&amp;quot; 5. the Regression Model &amp;quot;&amp;quot;&amp;quot;&#xA;    # h = .02  # step size in the mesh&#xA;    # logreg = linear_model.LinearRegression()&#xA;    # we create an instance of Neighbours Classifier and fit the data.&#xA;    regressionModel.fit(X, y)&#xA;    d_predictions = regressionModel.predict(y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;X.shape&lt;/code&gt; and &lt;code&gt;y.shape&lt;/code&gt; would yield &lt;code&gt;(500, 6)&lt;/code&gt; and &lt;code&gt;(500, 1)&lt;/code&gt;, respectively, which would obviously cause a dimensional error in the &lt;code&gt;d_predictions&lt;/code&gt;, meaning the regression model does not take multiple column inputs.&lt;/p&gt;&#xA;&lt;p&gt;I have a hypothesis that I can create a scoring scheme that will take into account the importance of each of the columns and create a scheme that creates a score and the end result would be a one-vs-one regression problem. &lt;strong&gt;Looking for some direction with respect to my hypothesis. Is it correct, wrong or halfway?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8215" LastEditorUserId="2444" LastEditDate="2020-12-25T13:05:52.447" LastActivityDate="2021-05-24T15:02:40.230" Title="multi vs one prediction using Regression" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5616" PostTypeId="2" ParentId="5614" CreationDate="2018-03-10T10:50:08.403" Score="0" Body="&lt;p&gt;I think the model will have no problem taking a multicolumn input. In fact, from your code, this is exactly how you trained it. It expects an input of size [k, 6], where k is k&gt;=1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead you are feeding it with [k, 1] sized data, which are the dimensions of &lt;em&gt;y&lt;/em&gt;. So you if you run it like this it should work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;regressionModel.predict(X)&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2018-03-10T10:50:08.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5619" PostTypeId="1" CreationDate="2018-03-10T13:03:40.733" Score="-1" ViewCount="47" Body="&lt;p&gt;I'm developing a millitary game and I want soldiers to place themselves in specific order, when commander orders. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do this? Should agents learn somehow how to achieve given order or there exist other way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new in AI, so I do not really know how can I accomplish my goal. &#xA;Thanks for every advice.&lt;/p&gt;&#xA;" OwnerUserId="13237" LastActivityDate="2018-03-11T13:11:08.140" Title="Placing agents in a specific order on demand" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2021-01-18T04:15:27.290" ContentLicense="CC BY-SA 3.0" />
  <row Id="5620" PostTypeId="2" ParentId="5580" CreationDate="2018-03-10T15:51:48.503" Score="3" Body="&lt;p&gt;The main doubt here is about the intuition behind the derivative part of back-propagation learning. First, I would like to point out 2 links about the intuition about how partial derivatives work &lt;a href=&quot;https://math.stackexchange.com/q/62614/168764&quot;&gt;Chain Rule Intuition&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/q/423/168764&quot;&gt;Intuitive reasoning behind the Chain Rule in multiple variables?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that we know how the chain rule works, let's see how we can use it in Machine Learning. So basically in machine learning, the final output is a function of input variables and the connection weights &#xA;&lt;span class=&quot;math-container&quot;&gt;$f\left(x_1, x_2, \ldots, x_n, w_1, w_2, \ldots, w_n\right)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; encloses all the activation functions and dot products lying between input and output. The &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, \ldots, x_n, w_1, w_2, \ldots, w_n$&lt;/span&gt; are called independent variables because they don't affect each other pairwise as well as in groups meaning you cannot find a function &lt;span class=&quot;math-container&quot;&gt;$g(x_i, \dots, w_i, \dots) =  h(x_j, \dots, w_j, \dots)$&lt;/span&gt;. So, basically its a black box from input to output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TIp3X.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TIp3X.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now our purpose is to minimize the Loss/Cost function, by changing the parameters that can be 'controlled by us' i.e the weights only, we cannot change the input variables. So this is done by taking the derivative of the cost function w.r.t to the variable that 'can be changed'. &lt;a href=&quot;https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/gradient-and-directional-derivatives/v/why-the-gradient-is-the-direction-of-steepest-ascent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is an explanation of why taking derivative and subsequently subtracting it reduces the value of cost function given by the 'maximal' amount. Also &lt;a href=&quot;https://math.stackexchange.com/q/462592/168764&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to calculate &lt;span class=&quot;math-container&quot;&gt;$dL/dw_n$&lt;/span&gt; you have to keep few things in mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Only differentiate &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; w.r.t to those functions which affect &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;And to reach your end goal of differentiation w.r.t to an independent variable you must differentiate &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; w.r.t to those functions only which are dependent on that particular independent variable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A crude algorithm assuming &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; also as a normal function (along the lines of activation function, so that I can express the idea recursively) differentiate &lt;span class=&quot;math-container&quot;&gt;$f_n$&lt;/span&gt; w.r.t to functions in the previous layer say &lt;span class=&quot;math-container&quot;&gt;$f_{n-1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_{n-2}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt;. Check which of these functions depends on &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;. Only &lt;span class=&quot;math-container&quot;&gt;$f_{n-1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_{n-2}$&lt;/span&gt; do. Differentiate them again w.r.t to previous layer functions. Check again and go on till you reach &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is the fool-proof version, but it has 2 flaws:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt; is not a function. People are making this mistake of assuming &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt; to be a function due to misinterpretation of a simple NN diagram. To reach &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;, you don't need to go through &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt;. But you definitely need to go through the activation functions and dot products. Think of this as painting a wall where color mixing occurs (not over-writing). So you paint the wall with some color (weights) then 2nd color and so on. Is the final product affected by color 1. Yes. Is the 'rate of change' caused by color 1 also affected by color 2. Yes. But does it mean we can find the 'change'of color n w.r.t to color 1? No its meaningless (bad example, couldn't think of a better one)&lt;/li&gt;&#xA;&lt;li&gt;The second flaw is that this approach is not followed because with experience it is apparent which function affects whom and which independent variable affects which function (saves computation).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To answer your question the equation is incorrect and the correct equation will be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L}{\partial w_{1}}=\frac{\partial h_{1}}{\partial w_{1}} \frac{\partial o_{2}}{\partial h_{1}} \frac{\partial L}{\partial o_{2}}+\frac{\partial h_{1}}{\partial w_{1}} \frac{\partial o_{1}}{\partial h_{1}} \frac{\partial L}{\partial o_{1}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have simply followed the algorithm I have given above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for why your equation is wrong, your equation contains the term &lt;span class=&quot;math-container&quot;&gt;$dw7/dh1$&lt;/span&gt;. Does &lt;span class=&quot;math-container&quot;&gt;$w_7$&lt;/span&gt; vary with &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt;? This means that &lt;span class=&quot;math-container&quot;&gt;$w_7$&lt;/span&gt; is directly related to the input as &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; is related with the input, but this is not the case for a single iteration(the whole algorithm run makes &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt; dependent on the inputs since you are trying to minimize the loss function w.r.t given inputs and weights, for a different set of inputs you will have different final weights).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in a nutshell, the aim of back-propagation is to identify the change in the loss function w.r.t to a given weight. To calculate that, you have to make sure in the chain rule of derivative you don't have any meaningless terms like the derivative of an independent variable w.r.t to any function. I recommend checking &lt;a href=&quot;https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/partial-derivatives/v/partial-derivatives-introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Khan Academy&lt;/a&gt; for a better understanding and clarity in concepts as I think the intuitions are hard to provide in a written answer.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-05-17T21:59:25.673" LastActivityDate="2020-05-17T21:59:25.673" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5621" PostTypeId="2" ParentId="5601" CreationDate="2018-03-10T18:52:06.067" Score="3" Body="&lt;p&gt;The outputs of a ReLU network are always &amp;quot;linear&amp;quot; and discontinuous. They can approximate curves, but it could take a lot of ReLU units. However, at the same time, their outputs will often be interpreted as a continuous, curved output.&lt;/p&gt;&#xA;&lt;p&gt;Imagine you trained a neural network that takes &lt;span class=&quot;math-container&quot;&gt;$x^3$&lt;/span&gt; and outputs &lt;span class=&quot;math-container&quot;&gt;$|x^3|$&lt;/span&gt; (which is similar to a parabola). This is easy for the ReLU function to do perfectly. In this case, the output is curved.&lt;/p&gt;&#xA;&lt;p&gt;But it is not actually curved. The inputs here are 'linearly' related to the outputs. All the neural network does is it takes the input and returns the absolute value of the input. It performs a 'linear', non-curved function. You can only see that the output is non-linear when you graph it against the original &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-values (the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$x^3$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;So, when we plot the output on a graph and it looks curved, it's usually because we associated different x-values with the input, and then plotted the output as the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-coordinate in relation to those &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-values.&lt;/p&gt;&#xA;&lt;p&gt;Okay, so you want to know how you would smoothly model &lt;span class=&quot;math-container&quot;&gt;$\sin(x)$&lt;/span&gt; using ReLU. The trick is that you don't want to put &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; as the input. Instead put something curved in relation to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; as the input, like &lt;span class=&quot;math-container&quot;&gt;$x^3$&lt;/span&gt;. So, the input is &lt;span class=&quot;math-container&quot;&gt;$x^3$&lt;/span&gt; and the output is &lt;span class=&quot;math-container&quot;&gt;$\sin(x)$&lt;/span&gt;. The reason why this would work is that it isn't computing the sine of the input - it's computing sine of the cube root of the input. It could never smoothly compute the sine of the input itself. To graph the output &lt;span class=&quot;math-container&quot;&gt;$\sin(x)$&lt;/span&gt;, put the original &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; as the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; coordinate (don't put the input) and put the output as the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; coordinate.&lt;/p&gt;&#xA;" OwnerUserId="13208" LastEditorUserId="2444" LastEditDate="2020-09-24T10:19:05.503" LastActivityDate="2020-09-24T10:19:05.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5622" PostTypeId="1" CreationDate="2018-03-10T21:46:12.247" Score="3" ViewCount="86" Body="&lt;p&gt;I'm trying to design an orbital rendezvous program for Kerbal Space Program.  I'm focusing on when to launch my spacecraft so that I can end up in the general vicinity of the target.  If I can control the ascent profile, the remaining dependent variables are the ship's twr and the target's altitude.  I want to try a computer learning solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to formulate the problem of learning the time to launch based on some twr?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I make an algorithm to compute the general equation of launch time to a altitude based on my ability to accelerate?  What type of learning problem could this be classified as?  What are some approaches to solve problems with known dependent variables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may be an obvious question, I kind of expect the answer is regression?  But it seemed a general enough question to be sure about to get a solid foothold in computer learning with this type of problem, which seems to come up a lot.&lt;/p&gt;&#xA;" OwnerUserId="13243" LastEditorUserId="13243" LastEditDate="2018-03-11T11:06:45.627" LastActivityDate="2019-05-05T18:02:35.523" Title="How can I have a computer learn the equation with known dependent variables?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="5623" PostTypeId="2" ParentId="5619" CreationDate="2018-03-10T22:08:54.690" Score="1" Body="&lt;p&gt;I agree with the commenters.  I think the easiest way to do this in some programming language is to say:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;soldiers[] = {tom,sam,sue,larry}&#xA;&#xA;define function fallIn(){&#xA;    facing = getcommanderfacing()&#xA;    commanderpos = getcommanderpos()&#xA;    dist = 1 //stand one foot away&#xA;&#xA;    for(int i = 0; i&amp;lt;soldiers.length, i++{&#xA;        //set them to a distance behind the commander, facing the same direction &#xA;        soldier[i].navigateTo(commanderpos - facing*dist*(i+1), facing)&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unless I misunderstood, this should be a pretty easy problem.  If you have to worry about collision with objects, you might tell them to follow the commander's path if he's moving or if he's not, find the position closest to the one they want that doesn't leave them in a wall.  Pathfinding can be a complex problem to get right, but even that doesn't necessarily depend on ai.&lt;/p&gt;&#xA;" OwnerUserId="13243" LastActivityDate="2018-03-10T22:08:54.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5625" PostTypeId="1" CreationDate="2018-03-11T07:58:43.857" Score="6" ViewCount="186" Body="&lt;p&gt;According to the definition of a fully observable environment in Russell &amp;amp; Norvig, AIMA (2nd ed), pages 41-44, an environment is only fully observable if it requires zero memory for an agent to perform optimally, that is, all relevant information is immediately available from sensing the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this definition and from the definition of an &quot;episodic&quot; environment in the same book, it is implied that all fully observable environments are, in fact, episodic or can be treated as episodic, which doesn't seem intuitive, but logically follows from the definitions. Also, no stochastic environment can be fully observable, even if the entire state space at a given point in time can be observed because rational action may depend on the previous observation that must be remembered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="13250" LastEditorUserId="2444" LastEditDate="2019-11-12T18:04:12.800" LastActivityDate="2021-02-23T01:09:26.020" Title="Are all fully observable environments episodic?" Tags="&lt;definitions&gt;&lt;environment&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5629" PostTypeId="1" AcceptedAnswerId="5647" CreationDate="2018-03-11T13:13:01.843" Score="3" ViewCount="799" Body="&lt;p&gt;I came across this line while reading the &lt;a href=&quot;https://arxiv.org/abs/1506.02025&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; on Spatial Transformers by Deepmind in the last paragraph of Sec 3.1:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The localisation network function floc() can take any form, such as a fully-connected network or a convolutional network, but should include a final regression layer to produce the transformation parameters θ.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand what regression is, but what is meant by a regression layer?&lt;/p&gt;&#xA;" OwnerUserId="13255" LastEditorUserId="2444" LastEditDate="2019-11-01T03:11:34.247" LastActivityDate="2019-11-01T03:11:34.247" Title="What is regression layer in a spatial transformer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deepmind&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5630" PostTypeId="5" CreationDate="2018-03-11T15:23:00.873" Score="0" Body="&lt;p&gt;Prediction in the general sense is the prediction of the qualitative or quantitative aspects of a certain value by an algorithm, maybe simple Machine Learning algorithm or an Artificially Intelligent algorithm.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:08:37.683" LastActivityDate="2018-03-12T19:08:37.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5631" PostTypeId="4" CreationDate="2018-03-11T15:23:00.873" Score="0" Body="For questions about prediction of a certain  quantitative or a qualitative value by an algorithm" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:07:28.453" LastActivityDate="2018-03-12T19:07:28.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5632" PostTypeId="5" CreationDate="2018-03-11T15:24:44.873" Score="0" Body="&lt;p&gt;Linear regression is an approach for finding an optimal set of parameter values &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; to tune a parametric model &lt;span class=&quot;math-container&quot;&gt;$\mathbb{M}_P$&lt;/span&gt; given a data set containing associated scalar dependent values &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and scalar independent values &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; or independent vector values &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2019-01-07T10:09:26.773" LastActivityDate="2019-01-07T10:09:26.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5633" PostTypeId="4" CreationDate="2018-03-11T15:24:44.873" Score="0" Body="For questions related to the theory or application of linear regression." OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2019-01-07T10:09:26.773" LastActivityDate="2019-01-07T10:09:26.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5634" PostTypeId="5" CreationDate="2018-03-11T15:27:31.930" Score="0" Body="&lt;p&gt;Signal processing concerns the analysis, synthesis, and modification of signals, which are broadly defined as functions conveying, &quot;information about the behavior or attributes of some phenomenon&quot;, such as sound, images, and biological measurements. For example, signal processing techniques are used to improve signal transmission fidelity, storage efficiency, and subjective quality, and to emphasize or detect components of interest in a measured signal.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:08:26.280" LastActivityDate="2018-03-12T19:08:26.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5635" PostTypeId="4" CreationDate="2018-03-11T15:27:31.930" Score="0" Body="For questions about processing signals of any physical quantity, with the help of Machine Learning or Artificially Intelligent algorithms." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:08:07.627" LastActivityDate="2018-03-12T19:08:07.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5638" PostTypeId="1" AcceptedAnswerId="5650" CreationDate="2018-03-12T01:49:01.093" Score="3" ViewCount="329" Body="&lt;p&gt;I used to treat back propagation as a black box but lately I want to understand more about it. I have used &lt;a href=&quot;https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mattmuzr's&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/5620/39&quot;&gt;DuttA's&lt;/a&gt; explanaiton as a guide to hand compute a simple neural network. I have computed feed forward and back propagation to a network similar to this one with one input, one hidden and one output&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hapD0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hapD0.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my computations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DxTjP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DxTjP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PNCZv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PNCZv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fe5iP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fe5iP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my computations correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pastebin.com/YgjywbCr&quot; rel=&quot;nofollow noreferrer&quot;&gt;*Full latex code&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2019-04-05T05:43:07.250" LastActivityDate="2019-04-05T05:43:07.250" Title="Hand computing feed forward and back propagation of neural network" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="12" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5639" PostTypeId="5" CreationDate="2018-03-12T05:41:31.697" Score="0" Body="&lt;p&gt;Programmation en Logique (Programming in Logic) or Prolog is a high-level programming language that has its roots in first-order logic or first-order predicate calculus. The language was conceived in Marseilles, France in the early 1970s by a group led by Alain Colmerauer. It is one of the first logic programming languages and it remains popular today. It is a programming language commonly associated with computational linguistics and artificial intelligence and is used in expert systems, theorem proving and pattern matching over natural language parse trees and natural language processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog- Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.techopedia.com/definition/24549/prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog - Techopedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:08:31.937" LastActivityDate="2018-03-12T19:08:31.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5640" PostTypeId="4" CreationDate="2018-03-12T05:41:31.697" Score="0" Body="For questions about the Prolog programming language, generally used for programming Artificially Intelligent programs." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:07:52.817" LastActivityDate="2018-03-12T19:07:52.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5641" PostTypeId="5" CreationDate="2018-03-12T05:45:10.233" Score="0" Body="&lt;p&gt;Brain is the portion of the vertebrate central nervous system enclosed in the skull and continuous with the spinal cord through the foramen magnum that is composed of neurons and supporting and nutritive structures (such as glia) and that integrates sensory information from inside and outside the body in controlling autonomic function (such as heartbeat and respiration), in coordinating and directing correlated motor responses, and in the process of learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nervous center in invertebrates comparable in position and function to the vertebrate brain (may not be as complex as vertebrae's brain).&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:07:45.850" LastActivityDate="2018-03-12T19:07:45.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5642" PostTypeId="4" CreationDate="2018-03-12T05:45:10.233" Score="0" Body="For questions about the brain of a living organism, e.g. what qualifies as a brain? What is the purpose of a brain? How does the brain function?" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T19:08:16.340" LastActivityDate="2018-03-12T19:08:16.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5643" PostTypeId="5" CreationDate="2018-03-12T05:48:18.227" Score="0" Body="&lt;p&gt;Perceptron is a machine learning algorithm that helps provide classified outcomes for computing. It dates back to the 1950s and represents a fundamental example of how machine learning algorithms work to develop data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The perceptron is an algorithm for supervised learning of binary classifiers (functions that can decide whether an input, represented by a vector of numbers, belongs to some specific class or not). It is a type of linear classifier, i.e. a classification algorithm that makes its predictions based on a linear predictor function combining a set of weights with the feature vector. The algorithm allows for online learning, in that it processes elements in the training set one at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptron - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.techopedia.com/definition/21333/perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptron - Techopedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T13:30:22.870" LastActivityDate="2018-03-12T13:30:22.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5644" PostTypeId="4" CreationDate="2018-03-12T05:48:18.227" Score="0" Body="For questions about the perceptron learning algorithm in Machine Learning." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-12T13:29:45.500" LastActivityDate="2018-03-12T13:29:45.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5645" PostTypeId="1" CreationDate="2018-03-12T10:23:44.230" Score="3" ViewCount="301" Body="&lt;p&gt;I'm working on a q-learning project that involves a &quot;robot&quot; solving a maze, and there is a problem with how I update the Q values (every time the robot ends up switching between two squares instead of actually learning) but I'm not sure where: I am at my wits end. Any pointers are welcome, here is the minimal viable example (I really can't condense it much more).. Thanks!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from enum import Enum&#xA;import numpy as np&#xA;from random import randrange&#xA;import string&#xA;import random&#xA;&#xA;class Direction(Enum):&#xA;    up=0&#xA;    down=1&#xA;    left=2&#xA;    right=3&#xA;&#xA;stepsTaken=0&#xA;nbMaxSteps=500&#xA;Q = {}&#xA;gamma=0.95&#xA;strat=1&#xA;epsilon=0.99&#xA;maze=[]&#xA;penalty=0&#xA;#values of each movement&#xA;step=-1&#xA;stepTrap=-20&#xA;stepExit=500&#xA;stepWall=-100&#xA;#current position of the robot&#xA;position=[0, 0]&#xA;&#xA;#funciton that checks if a certain place in the Q matrix is empty, returns 1 if it is&#xA;def currentQEmpty():&#xA;    global Q&#xA;    global position&#xA;    moves=[]&#xA;    if (position[0]!=0):&#xA;        moves.append(Direction.left)&#xA;    if (position[0]!=cols-1):&#xA;        moves.append(Direction.right)&#xA;    if (position[1]!=0):&#xA;        moves.append(Direction.down)&#xA;    if (position[1]!=rows-1):&#xA;        moves.append(Direction.up)&#xA;    for d in moves:&#xA;        if (Q.get((position[0],position[1],d),'A')=='A'):&#xA;            return 1&#xA;    return 0&#xA;&#xA;#intialise the Q matrix&#xA;cols=10&#xA;rows=10&#xA;values=np.zeros((rows,cols))&#xA;for x in range(rows):&#xA;        for y in range(cols):&#xA;            for dir in Direction:&#xA;                Q[(x, y, dir)] = 0&#xA;&#xA;#fills the Q matrix (replaces empty values only)&#xA;def QFill(moves):&#xA;    global maze&#xA;    global position&#xA;    global Q&#xA;    global step&#xA;    global stepTrap&#xA;    global stepWall&#xA;    global stepExit&#xA;    global gamma&#xA;    for d in moves:&#xA;        reward=0&#xA;        newpos=position&#xA;        if d==Direction.up:&#xA;            newpos=[position[0], position[1]+1]&#xA;        if d==Direction.down:&#xA;            newpos=[position[0], position[1]-1]&#xA;        if d==Direction.left:&#xA;            newpos=[position[0]-1, position[1]]&#xA;        if d==Direction.right:&#xA;            newpos=[position[0]+1, position[1]]&#xA;        reward=reward+values[newpos[0],newpos[1]]&#xA;        if(Q.get((position[0],position[1],d),0)==0):&#xA;            Q[position[0],position[1],d]=reward&#xA;&#xA;#Qmove: decides which move to make depending on current Q values&#xA;#this is where the issue is!&#xA;def Qmove(moves):&#xA;    global position&#xA;    global Q&#xA;    global step&#xA;    global stepTrap&#xA;    global stepWall&#xA;    global stepExit&#xA;    global gamma&#xA;    bestd=0&#xA;    newd=moves[random.randint(0,len(moves)-1)]&#xA;    for d in moves:&#xA;        newpos=position&#xA;        if d==Direction.up:&#xA;            newpos=[position[0], position[1]+1]&#xA;        if d==Direction.down:&#xA;            newpos=[position[0], position[1]-1]&#xA;        if d==Direction.left:&#xA;            newpos=[position[0]-1, position[1]]&#xA;        if d==Direction.right:&#xA;            newpos=[position[0]+1, position[1]]&#xA;        #update value to best value of new position&#xA;        if Q.get((newpos[0],newpos[1],d),0)&amp;gt;=Q.get((newpos[0],newpos[1],bestd),0):&#xA;            bestd=d&#xA;        Q[position[0],position[1],d]=Q.get((position[0],position[1],d),0)+ (values[newpos[0]][newpos[1]] + gamma * Q.get((newpos[0],newpos[1],bestd),1) - Q.get((position[0],position[1],d),0))      &#xA;        #update arrow&#xA;        if Q.get((position[0],position[1],d),0)&amp;gt;Q.get((position[0],position[1],newd),0):&#xA;            newd=d&#xA;    return newd&#xA;&#xA;#create maze&#xA;ch=['0', '1', '3']&#xA;for i in range(cols):&#xA;    maze.append([0]*(cols))&#xA;    for j in range(cols):&#xA;        random_index = randrange(0,len(ch))&#xA;        maze[i][j]=ch[random_index]&#xA;        if i==cols-1 and j==cols-1:&#xA;            maze[i][j]='5'&#xA;        if i==0 and j==0:&#xA;            maze[i][j]='0'&#xA;        if(maze[i][j]==&quot;1&quot;):&#xA;            values[i][j]=step&#xA;        elif(maze[i][j]==&quot;0&quot;):&#xA;            values[i][j]=stepWall&#xA;        elif(maze[i][j]==&quot;3&quot;):&#xA;            values[i][j]=stepTrap&#xA;        else:&#xA;            values[i][j]=stepExit&#xA;#move&#xA;while(stepsTaken&amp;lt;nbMaxSteps):&#xA;    moves=[]&#xA;    #if he finishes he starts over&#xA;    if(position[0]==rows-1 and position[1]==cols-1):&#xA;        position[0]=0&#xA;        position[1]=0&#xA;        penalty=0&#xA;    #identify the moves he can legally make&#xA;    if (position[0]!=0):&#xA;        moves.append(Direction.left)&#xA;    if (position[0]!=cols-1):&#xA;        moves.append(Direction.right)&#xA;    d=moves[0]&#xA;    if (position[1]!=0):&#xA;        moves.append(Direction.down)&#xA;    if (position[1]!=rows-1):&#xA;        moves.append(Direction.up)&#xA;    dest=[]&#xA;    #choose epsilon value&#xA;    rand=random.uniform(0, 1)&#xA;    if(rand&amp;lt;epsilon**stepsTaken):&#xA;        strat=1&#xA;        #explore&#xA;    else:&#xA;        strat=2&#xA;        #exploit&#xA;    #print(epsilon**stepsTaken)&#xA;    if(currentQEmpty() or strat==1):&#xA;        QFill(moves)&#xA;        d=moves[random.randint(0,len(moves)-1)]#how and why he moves&#xA;        print('dumb')&#xA;    else:&#xA;        d=Qmove(moves)&#xA;        print('smart')&#xA;    if(d==Direction.left):&#xA;        dest.append(position[0]-1) #x decreases by 1 place&#xA;        dest.append(position[1]) #y does not change&#xA;    if(d==Direction.right):&#xA;        dest.append(position[0]+1) #x increases by 1 place&#xA;        dest.append(position[1]) #y does not change&#xA;    if(d==Direction.up):&#xA;        dest.append(position[0]) #x does not change&amp;amp;&amp;amp;&#xA;        dest.append(position[1]+1) #y increases by 1&#xA;    if(d==Direction.down):&#xA;        dest.append(position[0]) #x does not change&#xA;        dest.append(position[1]-1) #y decreases by 1&#xA;    #penalty is calculated&#xA;    penalty=penalty+values[dest[0]][dest[1]]&#xA;    if(maze[dest[0]][dest[1]]!='0'): #not a wall&#xA;        position=dest&#xA;    stepsTaken=stepsTaken+1&#xA;    #show Q matrix&#xA;    x=position[0]&#xA;    y=position[1]&#xA;    print(&quot;x:&quot;,x,&quot; y:&quot;,y)&#xA;    print(&quot; UP:%s&quot; % Q.get((x,y, Direction.up)))&#xA;    print(&quot; DOWN:%s&quot; % Q.get((x,y, Direction.down)))&#xA;    print(&quot; LEFT:%s&quot; % Q.get((x,y, Direction.left)))&#xA;    print(&quot; RIGHT:%s\n&quot; % Q.get((x,y, Direction.right)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12940" LastEditorUserId="12940" LastEditDate="2018-03-12T12:37:53.193" LastActivityDate="2019-04-04T18:36:45.567" Title="Q-learning in Python" Tags="&lt;python&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5647" PostTypeId="2" ParentId="5629" CreationDate="2018-03-12T13:55:03.750" Score="1" Body="&lt;p&gt;Basically, it means that the &quot;localization network&quot; should output a set of real valued parameters (typically 6 numbers). The word &quot;regression&quot; doesn't bear any specific meaning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any network that relies on the original image as input (directly or indirectly) and outputs 6 numbers, would work. And its last layer would qualify as &quot;regression layer&quot; as long as it is not restricted in the real domain (not normalized, softmaxed, etc)&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2018-03-12T13:55:03.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5648" PostTypeId="2" ParentId="5602" CreationDate="2018-03-12T16:59:26.180" Score="1" Body="&lt;p&gt;I would go with a NEAT algorithm for this one. Basically, you &quot;breed&quot; the right network that does the best every generation. Then Rinse and repeat. Doesn't require a training datbase. Bots compete and evolve as they go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this demonstration shows how Neataptic.js teaches a bunch of bots to follow a target:&#xA;&lt;a href=&quot;https://wagenaartje.github.io/neataptic/articles/targetseeking/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wagenaartje.github.io/neataptic/articles/targetseeking/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is more complex problem - a game of Agar.io:&#xA;&lt;a href=&quot;https://wagenaartje.github.io/neataptic/articles/agario/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wagenaartje.github.io/neataptic/articles/agario/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="13138" LastActivityDate="2018-03-12T16:59:26.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5650" PostTypeId="2" ParentId="5638" CreationDate="2018-03-12T18:27:50.233" Score="2" Body="&lt;p&gt;One important point I missed in first review: error is a summatory, its derivative is also a summatory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About offsets &quot;b&quot;: usually they are different in each cell (if not fixed to some value, as 0). Thus, replace them by b1 and b2. Moreover, they should be optimized in the same way that the weights.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-03-12T18:48:42.753" LastActivityDate="2018-03-12T18:48:42.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5652" PostTypeId="1" CreationDate="2018-03-12T18:50:49.307" Score="2" ViewCount="90" Body="&lt;p&gt;I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously some training would need to be done, but I've found little to no online references to go on when it comes to running assessment against US legal documents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to use an algorithm to discover potential issues in long and complex legal texts, or associated (groups) of legal texts which bind one or more related entities (people or corporations) to potentially conflicting clauses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a pointer in some kind of direction would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="13043" LastActivityDate="2018-03-12T18:50:49.307" Title="NLP proved against US legal texts" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;natural-language-processing&gt;&lt;legal&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5653" PostTypeId="5" CreationDate="2018-03-12T20:51:33.460" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-12T20:51:33.460" LastActivityDate="2018-03-12T20:51:33.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5654" PostTypeId="4" CreationDate="2018-03-12T20:51:33.460" Score="0" Body="For questions related to Spanish-language AI resources, and Natural Language Processing for Spanish-language texts." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-12T20:51:33.460" LastActivityDate="2018-03-12T20:51:33.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5658" PostTypeId="1" AcceptedAnswerId="5668" CreationDate="2018-03-13T10:04:26.173" Score="3" ViewCount="64" Body="&lt;p&gt;This is a theoretical question. I am a newbie to artificial intelligence and machine learning, and the more I read the more I like this. So far, I have been reading about the evaluation of language models (I am focused on ASR), but I still don't get the concept of the &lt;strong&gt;development test sets&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The clearest explanation I have come across is the following (taken from &lt;a href=&quot;https://web.stanford.edu/%7Ejurafsky/slp3/3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 3&lt;/a&gt; of the book &lt;a href=&quot;https://web.stanford.edu/%7Ejurafsky/slp3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speech and Language Processing (3rd ed. draft)&lt;/a&gt; by Dan Jurafsky and James H. Martin)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sometimes we use a particular test set so often that we implicitly tune to its characteristics. We then need a fresh test set that is truly unseen. In such cases, we call the initial test set the development test set or, devset.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In any case, I still don't understand why an additional test has to be used. In other words, why aren't training and test sets enough?&lt;/p&gt;&#xA;" OwnerUserId="13291" LastEditorUserId="2444" LastEditDate="2021-01-21T00:08:03.547" LastActivityDate="2021-01-21T00:11:37.703" Title="What are &quot;development test sets&quot; used for?" Tags="&lt;terminology&gt;&lt;cross-validation&gt;&lt;training-datasets&gt;&lt;test-datasets&gt;&lt;validation-datasets&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5659" PostTypeId="5" CreationDate="2018-03-13T10:27:31.493" Score="0" Body="&lt;p&gt;Humans are considered the most intelligent species currently residing on Earth, rivaled only by &lt;a href=&quot;https://en.wikipedia.org/wiki/Corvidae#Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Corvidae&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Octopus#Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octopoda&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Human intelligence refers to the intellectual prowess of humans, which is marked by complex cognitive feats and high levels of motivation and &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-awareness&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-awareness&lt;/a&gt;. Through their intelligence, humans possess the cognitive abilities to learn, form concepts, understand, apply logic and reason, including the capacities to recognize patterns, comprehend abstract ideas, plan, solve problems, make decisions, retain information, and use language to communicate. &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Human_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Intelligence wik&lt;/a&gt;i&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Definition and characteristics of human intelligence are interpreted differently by individuals depending on their culture and frame of reference, as intelligence is a spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Also: &lt;a href=&quot;http://bladerunner.wikia.com/wiki/Tyrell_Corporation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tyrell Corporation Motto&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-14T21:46:12.137" LastActivityDate="2018-03-14T21:46:12.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5660" PostTypeId="4" CreationDate="2018-03-13T10:27:31.493" Score="0" Body="For questions on behavior/performance of algorithms designed to mimic human qualities, behaviors, etc." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-14T21:30:33.120" LastActivityDate="2018-03-14T21:30:33.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5661" PostTypeId="5" CreationDate="2018-03-13T10:29:33.010" Score="0" Body="&lt;p&gt;Pattern recognition is concerned with the recognition of patterns and regularities in data, although it is in some cases considered to be nearly synonymous with machine learning. Pattern recognition systems are in many cases trained from labeled &quot;training&quot; data (supervised learning), but when no labeled data are available other algorithms can be used to discover previously unknown patterns (unsupervised learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern Recognition - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-14T21:36:46.280" LastActivityDate="2018-03-14T21:36:46.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5662" PostTypeId="4" CreationDate="2018-03-13T10:29:33.010" Score="0" Body="For question about recognition of patterns (generally sequence of values following a certain mathematical equation) of any physical quantity." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-14T21:28:07.863" LastActivityDate="2018-03-14T21:28:07.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5663" PostTypeId="5" CreationDate="2018-03-13T10:35:14.450" Score="0" Body="&lt;p&gt;Hidden Layers are the layers which lies between the input and the output layers of a Neural Network. The parameters and the implementation details of hidden layers of a Neural Network are generally hidden from an user. Opinions vary about the true functioning, the purpose and the power lent by the hidden layers to the Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/63152/what-does-the-hidden-layer-in-a-neural-network-compute&quot;&gt;What does the hidden layer in a neural network compute?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-14T21:32:50.923" LastActivityDate="2018-03-14T21:32:50.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5664" PostTypeId="4" CreationDate="2018-03-13T10:35:14.450" Score="0" Body="For questions about functioning, applications, structure and improvement of performance of hidden layers in a Neural Network." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-14T21:37:05.217" LastActivityDate="2018-03-14T21:37:05.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5665" PostTypeId="1" CreationDate="2018-03-13T14:41:45.543" Score="3" ViewCount="122" Body="&lt;p&gt;Interested to know if there was any use or interest in activation functions with more than one output value to the next column instead of  single firing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested to know if this would have any use or would be almost identical to a single value firing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kmGh2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kmGh2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="4302" LastEditDate="2018-08-12T02:54:24.493" LastActivityDate="2018-08-15T20:32:34.323" Title="ANNs with multiple activation outputs" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5668" PostTypeId="2" ParentId="5658" CreationDate="2018-03-13T16:34:23.543" Score="5" Body="&lt;p&gt;In machine learning, you normally split your data into 3 parts (80-10-10%).&lt;/p&gt;&#xA;&lt;p&gt;The first part (80% of your initial data) is for the training of your ML model: this is known as the &lt;strong&gt;training dataset&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The second part (10%) is the &lt;strong&gt;development set&lt;/strong&gt; (or &lt;em&gt;dataset&lt;/em&gt;), aka &lt;strong&gt;validation set&lt;/strong&gt;. This is used as measuring your performance with various hyperparameters (e.g. in neural networks: layer size).&lt;/p&gt;&#xA;&lt;p&gt;After you found your best hyperparameters, you learn the model again on the training set, and then test it on your &lt;strong&gt;test dataset&lt;/strong&gt; (10%), which the model has never seen before. Your result on the test data is now a good indicator how your model prediction quality is in the real world (because it was never optimized for this test data).&lt;/p&gt;&#xA;" OwnerUserId="13295" LastEditorUserId="2444" LastEditDate="2021-01-21T00:11:37.703" LastActivityDate="2021-01-21T00:11:37.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5669" PostTypeId="2" ParentId="4613" CreationDate="2018-03-14T03:08:19.943" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Is this a viable replacement to the Fitness function?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sure, the fitness is 1 for the winner and 0 for the loser. You're using some kind of the Tournament selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be better to use more chromosomes and let A play against B, C, D... and define the fitness as the number of wins. Or not, as such an evaluation is more precise but also more time-consuming.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;... Infinity ... Do I also need to add this score to the chromosome?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why should you? The exact value doesn't matter (as it only needs to be big enough), so there's nothing to evolve there. You don't represent the number  of players either, right? Just use common sense.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;... Would it still be okay to generate chromosomes randomly?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I guess so, but a distribution providing values closer to the expected outcome should be better. This depends on how you mutate them (adding a small random value won't bring you far, multiplying by &lt;code&gt;1 + small_random_value&lt;/code&gt; would).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can generate values from some fixed interval and scale them up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your values are IMHO far too big. Whatever your 5 and 40000 mean, I guess, that 5 and 400 would work the same.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-03-14T03:08:19.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5670" PostTypeId="1" AcceptedAnswerId="5671" CreationDate="2018-03-14T06:11:01.467" Score="2" ViewCount="695" Body="&lt;p&gt;The result of gradient descent algorithm is a vector. So how does this algorithm decide the direction for weight change? We Give hyperparameters for step size. But how is the vector direction for weight change, for the purpose of reducing the Loss function in a Linear Regression Model, determined by this algorithm?&lt;/p&gt;&#xA;" OwnerUserId="13169" LastEditorUserId="1671" LastEditDate="2018-03-15T21:44:58.607" LastActivityDate="2018-03-16T10:39:42.203" Title="How is direction of weight change determined by Gradient Descent algorithm" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5671" PostTypeId="2" ParentId="5670" CreationDate="2018-03-14T08:10:46.847" Score="1" Body="&lt;p&gt;First thing is that what does gradient descent do? Gradient Descent is a tool of calculus which we use to determine the parameters (here weights) used in a Machine Learning algorithm or a Neural Network, by running the Gradient Descent algorithm iteratively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the vector obtained from one iteration of gradient descent tell us? It tells us the direction of weight change (when weights are treated as a vector) for the maximum reduction in the value outputted by a loss function. The intuition behind why Gradient Descent gives such a direction can be found here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/gradient-and-directional-derivatives/v/why-the-gradient-is-the-direction-of-steepest-ascent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why the gradient is the direction of steepest ascent - Khan Academy&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/223252/why-is-gradient-the-direction-of-steepest-ascent&quot;&gt;Why is gradient the direction of steepest ascent?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/180559/why-does-gradient-descent-work&quot;&gt;Why does gradient descent work?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In general cases the cost/loss function is an n-dimensional paraboloid (we design function in such a way such that its convex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TYDRx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TYDRx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 2-D paraboloid with &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; as independent variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now why do we reduce the weights in the direction of negative gradient only? Why not some other direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we want to reduce the cost in Linear Regression for better predictions we choose only the negative Gradient Descent direction, as it is the direction of steepest descent even though we could have chosen some other possible vector direction to reduce cost, but the negative gradient descent direction ensures that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cost is always decreased (if the step size is correct and in general only in cases of convex functions e.g paraboloid).&lt;/li&gt;&#xA;&lt;li&gt;Cost is decreased by the maximal amount if we move in that direction, so we don't have to worry whether cost will decrease or not if we move in that direction.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also we use learning rate &lt;code&gt;alpha&lt;/code&gt; to scale by how much amount we want to decrease the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: As pointed by @pasaba the error function may not be a paraboloid but in general a good cost function looks like a paraboloid with skewed axis.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-16T10:24:26.757" LastActivityDate="2018-03-16T10:24:26.757" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5672" PostTypeId="1" CreationDate="2018-03-14T08:31:11.863" Score="2" ViewCount="141" Body="&lt;p&gt;The lithographs of dutch artist M.C. Escher have been used in the study of artificial Intelligence. How can the human mind Incorporate these optical illusions into abstract thought? Is this reverse artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="13302" LastActivityDate="2018-07-01T21:38:57.317" Title="M.c.Escher and abstract thought" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5673" PostTypeId="2" ParentId="5475" CreationDate="2018-03-14T10:40:41.183" Score="2" Body="&lt;p&gt;I am also quite new to this field, but I think you should use the normalized outputs for the backpropagation. In general, you would want to backpropagate all the calculations you did in the forward pass, so why would you want to exclude the step of the normalization in your backward pass? This would essentially make the renormalization to have no effect (different loss values but no different model weight update).&lt;/p&gt;&#xA;&lt;p&gt;For example, in policy gradients, you backpropagate through the log propability of the selected action. In the forward pass, the sampling of the probability (that determines which action is selected) is not affected by the renormalization (you might just get different loss values in the end in your loss function). But, compared to this, in the backward pass, you need the actual value of the log probability to calculate the gradient that updates the model weights.&lt;/p&gt;&#xA;&lt;p&gt;So (&lt;strong&gt;I think&lt;/strong&gt;) the normalization is mostly just done for the backpropagation to get &amp;quot;renormalized&amp;quot; gradients. And no unbalanced gradients between states with more/less allowed actions.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="2444" LastEditDate="2020-11-14T18:26:10.303" LastActivityDate="2020-11-14T18:26:10.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5674" PostTypeId="5" CreationDate="2018-03-14T11:30:17.673" Score="0" Body="&lt;p&gt;In mathematics, computer science and operations research, mathematical optimization or mathematical programming, alternatively spelled optimisation, is the selection of a best element (with regard to some criterion) from some set of available alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the simplest case, an optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function. The generalization of optimization theory and techniques to other formulations constitutes a large area of applied mathematics. More generally, optimization includes finding &quot;best available&quot; values of some objective function given a defined domain (or input), including a variety of different types of objective functions and different types of domains.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-14T21:34:11.400" LastActivityDate="2018-03-14T21:34:11.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5675" PostTypeId="4" CreationDate="2018-03-14T11:30:17.673" Score="0" Body="For questions about implementing and improving optimization algorithms used in creating AI programs, or optimization in general." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-14T21:28:00.347" LastActivityDate="2018-03-14T21:28:00.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5676" PostTypeId="5" CreationDate="2018-03-14T11:45:03.743" Score="0" Body="&lt;p&gt;Object recognition – technology in the field of computer vision (can be any other domain also) for finding and identifying objects in an image or video sequence. Humans recognize a multitude of objects in images with little effort, despite the fact that the image of the objects may vary somewhat in different view points, in many different sizes and scales or even when they are translated or rotated. Objects can even be recognized when they are partially obstructed from view. This task is still a challenge for computer vision systems. Many approaches to the task have been implemented over multiple decades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Outline_of_object_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Object Recognition - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-14T21:32:58.700" LastActivityDate="2018-03-14T21:32:58.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5677" PostTypeId="4" CreationDate="2018-03-14T11:45:03.743" Score="0" Body="For questions related to object recognition, which is the problem of determining the type/class/category of an object in the image, so object recognition could also be called object classification. This is different from object detection, which is either used to refer to object localization (i.e. find the coordinates of the object in the image) + object classification, or just object localization." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-06-16T17:45:40.880" LastActivityDate="2020-06-16T17:45:40.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5678" PostTypeId="5" CreationDate="2018-03-14T11:47:16.673" Score="0" Body="&lt;p&gt;Theoretical computer science developed out of logic, the theory of computation (if this is to be considered a different subject from logic), and some related areas of mathematics. So theoretically minded computer scientists are well informed about logic even when they aren't logicians. Computer scientists in general are familiar with the idea that logic provides techniques for analyzing the inferential properties of languages, and with the distinction between a high-level logical analysis of a reasoning problem and its implementations. Logic, for instance, can provide a specification for a programming language by characterizing a mapping from programs to the computations that they license. A compiler that implements the language can be incomplete, or even unsound, as long as in some sense it approximates the logical specification. This makes it possible for the involvement of logic in AI applications to vary from relatively weak uses in which the logic informs the implementation process with analytic insights, to strong uses in which the implementation algorithm can be shown to be sound and complete. In some cases, a working system is inspired by ideas from logic, but acquires features that at first seem logically problematic but can later be explained by developing new ideas in logical theory. This sort of thing has happened, for instance, in logic programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://plato.stanford.edu/entries/logic-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Logic and AI - Stanford Encyclopedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-14T21:36:56.163" LastActivityDate="2018-03-14T21:36:56.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5679" PostTypeId="4" CreationDate="2018-03-14T11:47:16.673" Score="0" Body="For questions about algorithms and concepts for the development or use of logical inference capabilities in computers or the use of logical inference to arrive at the most useful AI designs." OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-10-18T03:18:32.520" LastActivityDate="2018-10-18T03:18:32.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5680" PostTypeId="2" ParentId="5265" CreationDate="2018-03-14T12:17:34.923" Score="5" Body="&lt;p&gt;I am not into the field of super-resolution, but I think this question applies to general neural network construction.&lt;/p&gt;&#xA;&lt;p&gt;Usually, you try to solve a classification problem or a regression problem with your neural network.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For classification, you try to predict probabilities that a specific output corresponds to a specific class. Therefore, every output value should be a probability and therefore have a range between 0 and 1. To achieve this, you usually use a softmax or sigmoid function as your last layer to squash the output between 0 and 1. In addition to this (which is wanted in classification tasks), these functions raise the probability output of likely classes while decreasing the probability of all other unlikely classes (therefore enforcing the network to choose for one specific class over the others).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the regression task, you are not looking for probability values as your output values but instead for real-valued numbers. In such a case, no activation function is wanted, since you want to be able to approximate any possible real value and not probabilities.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, in the case of super-resolution, I think the generated output is a map where each value corresponds to a pixel value of the super-resolution image. In that case, your pixels are real number values and no probabilities. So, you are solving a regression problem.&lt;/p&gt;&#xA;&lt;p&gt;But you could also go with a classification approach, where you have 256 output maps that give probability to each possible pixel value between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$255$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="2444" LastEditDate="2020-11-15T01:19:22.313" LastActivityDate="2020-11-15T01:19:22.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5682" PostTypeId="1" AcceptedAnswerId="5683" CreationDate="2018-03-14T13:13:25.470" Score="4" ViewCount="454" Body="&lt;p&gt;I am trying to understand the difference between biological and artificial evolution. If we look at it in terms of genetics, in both of them, the selection operation is a key term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the difference between biological and artificial evolution?&lt;/p&gt;&#xA;" OwnerUserId="12806" LastEditorUserId="2444" LastEditDate="2019-11-28T20:24:12.353" LastActivityDate="2019-12-02T21:37:01.197" Title="What's the difference between biological and artificial evolution?" Tags="&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;biology&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5683" PostTypeId="2" ParentId="5682" CreationDate="2018-03-14T13:45:11.753" Score="4" Body="&lt;p&gt;Biological and artificial evolution work around pretty much the same principles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fitness and selection&lt;/strong&gt;: In biology, the fittest organisms in an ecosystem are more likely to survive long enough to reproduce, passing on their genes in the process. In artificial evolution, our &lt;em&gt;organisms&lt;/em&gt; are in fact &lt;em&gt;solutions&lt;/em&gt; to our problem, which can be evaluated to determine how good they are (their fitness). We choose ourselves which solutions will be selected for reproduction (there are many ways to do this selection, but what is common among all of them is that the fittest solutions have a higher chance of being selected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Crossover&lt;/strong&gt;: In biology, an organism inherits a portion of each parent's genes, so is a sort of genetic hybrid of both parents. For artificial evolution, a new solution (a &quot;child&quot; solution) will inherit part of its parent's solutions (we take a partial solution from each parent, and glue those partial solutions together to construct a new solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mutation&lt;/strong&gt;: In nature mutations often occur at birth and this is why there are many different species. Harmful mutations make the individual less likely to survive long enough to pass them on to children, and in contrast helpful mutations make it more likely that the individual will survive long enough to pass them unto children. The same can be said for artificial evolution: A mutation randomly changes a small part of the solution, and if it makes that solution fitter, then that solution has a higher chance of being selected for reproduction.&lt;/p&gt;&#xA;" OwnerUserId="12857" LastActivityDate="2018-03-14T13:45:11.753" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5686" PostTypeId="5" CreationDate="2018-03-15T07:07:49.133" Score="0" Body="&lt;p&gt;The National Aeronautics and Space Administration is an independent agency of the executive branch of the United States federal government responsible for the civilian space program, as well as aeronautics and aerospace research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nasa.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NASA&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-15T21:37:50.057" LastActivityDate="2018-03-15T21:37:50.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5687" PostTypeId="4" CreationDate="2018-03-15T07:07:49.133" Score="0" Body="For questions on AI developed or theorized by NASA." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-15T21:42:00.413" LastActivityDate="2018-03-15T21:42:00.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5688" PostTypeId="1" CreationDate="2018-03-15T08:26:48.680" Score="3" ViewCount="103" Body="&lt;p&gt;I am not clear with the concept that an unsupervised model learns. We are giving an input and output to the supervised model, so that it can generate a particular value, pattern or something out of it which can be used to categorize something in the future. By contrast, in unsupervised learning, we are clustering, so why do we need learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone detail me with some real-world examples?&lt;/p&gt;&#xA;" OwnerUserId="13326" LastEditorUserId="2444" LastEditDate="2019-11-16T23:58:50.530" LastActivityDate="2019-11-16T23:58:50.530" Title="Why do we need learning in unsupervised learning?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;unsupervised-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2019-11-19T04:28:26.627" ContentLicense="CC BY-SA 4.0" />
  <row Id="5689" PostTypeId="2" ParentId="5486" CreationDate="2018-03-15T08:54:17.103" Score="4" Body="&lt;p&gt;After doing a bit of research I found that the LSTM whose gates perform convolutions is called ConvLSTM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term CNN LSTM is loose and may mean stacking up LSTM on top of CNN for tasks like video classification&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/5qt7oh/d_what_is_the_difference_between_using_the/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit thread discussing this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14633" LastActivityDate="2018-03-15T08:54:17.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5690" PostTypeId="5" CreationDate="2018-03-15T09:10:00.673" Score="0" Body="" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-03-20T22:49:38.010" LastActivityDate="2020-03-20T22:49:38.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5691" PostTypeId="4" CreationDate="2018-03-15T09:10:00.673" Score="0" Body="For questions related to the concept of overfitting in machine learning, which can be loosely defined as the gap between the performance on the training set and the performance on the test set." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-03-20T22:49:38.010" LastActivityDate="2020-03-20T22:49:38.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5692" PostTypeId="5" CreationDate="2018-03-15T09:13:35.383" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-15T09:13:35.383" LastActivityDate="2018-03-15T09:13:35.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5693" PostTypeId="4" CreationDate="2018-03-15T09:13:35.383" Score="0" Body="For questions about current affairs about various types of contributions/theories proposed by different groups/communities identifying themselves as AI researchers/practitioners/enthusiasts." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-08-08T02:27:16.557" LastActivityDate="2018-08-08T02:27:16.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5694" PostTypeId="1" AcceptedAnswerId="5695" CreationDate="2018-03-15T09:42:21.513" Score="4" ViewCount="114" Body="&lt;p&gt;I asked my self this simple question while reading &lt;a href=&quot;https://web.stanford.edu/class/cs224n/reports/2762092.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Comment Abuse Classification with Deep Learning&quot;&lt;/a&gt; by Chu and Jue. Indeed, they say at the end of the that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;It is clear that RNNs, specifically LSTMs, and CNNs are state-of-the-art  architectures for sentiment analysis&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To my mind CNNs were only neurons arranged so that they correspond to overlapping regions when paving the input field. It wasn't that recurrent at all.&lt;/p&gt;&#xA;" OwnerUserId="4738" LastEditorUserId="2444" LastEditDate="2020-06-14T17:05:25.783" LastActivityDate="2020-06-14T17:05:25.783" Title="Do convolutional neural networks also have recurrent connections?" Tags="&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ClosedDate="2020-06-14T17:05:58.820" ContentLicense="CC BY-SA 4.0" />
  <row Id="5695" PostTypeId="2" ParentId="5694" CreationDate="2018-03-15T12:21:44.953" Score="4" Body="&lt;p&gt;You are right. I think you are just misinterpreting the part of the sentence ('specifically LSTMs'). LSTMs are an example of a popular type of RNN.&#xA;RNNs and CNNs are different architectures but they can be used together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another sentence with the same structure:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is clear than dogs, specifically corgis, and cats are very common in online memes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ztt5e.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ztt5e.jpg&quot; alt=&quot;A cat and a corgi&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-03-15T13:04:38.150" LastActivityDate="2018-03-15T13:04:38.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5696" PostTypeId="2" ParentId="5688" CreationDate="2018-03-15T13:00:07.460" Score="1" Body="&lt;p&gt;Imagine you have a dataset of people who have cancer. &#xA;You have information about their age, physique, diagnosis, treatments, and results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this data, you want to prescribe a set of treatments for a new patient, P.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, if there is someone in the dataset that has very similar traits as P and had a positive result with their treatments, you could prescribe the same set of treatments. However, this is incredibly unlikely and becomes more infeasible as more information about P is observed (e.g. Has brown hair and hates pasta).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better option is to cluster the dataset into groups that have positive outcomes for treatment results. For example, perhaps patients with lung cancer who smoke and are given treatment A do better than patients with lung cancer who didn't smoke and are given the same treatment A. These patients should then be divided based on this outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once these different clusters are found, patient P can be evaluated against each of the clusters and a set of treatments can be prescribed (e.g. Most of the treatments from cluster A, but 1 treatment from cluster B).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unsupervised learning is the method of finding these clusters, which helps find structure to the data to better answer questions.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2018-03-15T13:00:07.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5697" PostTypeId="2" ParentId="5688" CreationDate="2018-03-15T13:23:32.270" Score="0" Body="&lt;p&gt;&lt;strong&gt;Supervised Learning:&lt;/strong&gt; This is performed with the help of a teacher. A child works on the basis of the output that he/she has to produce. Their actions are &lt;strong&gt;supervised&lt;/strong&gt; by a teacher. Similarly in ANNs, each vector requires a corresponding target vector, which represents the desired output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Unsupervised Learning:&lt;/strong&gt; Consider the learning process of a tadpole, it learns by itself, it isn't taught by any teacher. In ANN, during the training process, the network receives the input patterns and organizes these patterns to form clusters. When a new input pattern is applied, the neural network gives an output response indicating the class to which the input pattern belongs. If for an input, a pattern class cannot be found then a new class is generated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, there is no feedback from the environment, the network must itself discover patterns, regularities, features or categories from the input data and relations for the input data over the output.&lt;/p&gt;&#xA;" OwnerUserId="12806" LastActivityDate="2018-03-15T13:23:32.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5698" PostTypeId="2" ParentId="5043" CreationDate="2018-03-15T13:43:30.023" Score="1" Body="&lt;p&gt;In a business context, there are issues surrounding the implementation, the implementors, the other employees, the business entity itself, and the customers. These stem from data used, risks inherent in an implementation, like unknown errors bugs or algorithms without human checks, behaviour change of impacted stakeholders, job losses, reputation impacts on the company, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a lot to think about AI and ethics. It could be seen as a combination of a broad set of topics including computer science, humanities, economics, and philosophy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I run a podcast on some of these issues &lt;a href=&quot;http://machine-ethics.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://machine-ethics.net/&lt;/a&gt; let me know if there's anything you want to be discussed or someone you would like to hear.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="2444" LastEditDate="2020-02-22T12:00:14.683" LastActivityDate="2020-02-22T12:00:14.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5700" PostTypeId="5" CreationDate="2018-03-15T19:32:09.183" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Time_perception&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Time_perception&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.stir.ac.uk/~lss/recentpapers/perctime.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptual Time, Perceptual Reality, and General&#xA;Intelligence (Smith)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-15T19:32:09.183" LastActivityDate="2018-03-15T19:32:09.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5701" PostTypeId="4" CreationDate="2018-03-15T19:32:09.183" Score="0" Body="For questions related to the perception of time in regard to automata." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-15T19:32:09.183" LastActivityDate="2018-03-15T19:32:09.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5703" PostTypeId="1" AcceptedAnswerId="5704" CreationDate="2018-03-15T23:53:52.083" Score="3" ViewCount="220" Body="&lt;p&gt;I'm reading &lt;a href=&quot;https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;the AlexNet paper&lt;/a&gt;. In section 4, where the authors explain how they prevent overfitting, they mention &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label&quot;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does this mean?&lt;/p&gt;&#xA;" OwnerUserId="4656" LastEditorUserId="2444" LastEditDate="2020-06-12T23:43:53.393" LastActivityDate="2020-06-12T23:44:49.797" Title="Why do we need 10 bits to represent the 1000 classes in AlexNet?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;alexnet&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5704" PostTypeId="2" ParentId="5703" CreationDate="2018-03-16T03:10:58.827" Score="3" Body="&lt;p&gt;You need 10-bits (&lt;span class=&quot;math-container&quot;&gt;$2^{10} = 1024$&lt;/span&gt;) to represent 1000 classes.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="2444" LastEditDate="2020-06-12T23:44:19.557" LastActivityDate="2020-06-12T23:44:19.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5705" PostTypeId="2" ParentId="5703" CreationDate="2018-03-16T03:11:44.243" Score="1" Body="&lt;p&gt;It takes at least 10 bits to represent any number between &lt;span class=&quot;math-container&quot;&gt;$1-1000$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$2^{10} = 1024$&lt;/span&gt;. This means that if one was trying to represent 1 of the 1000 classes, one would need at least 10 bits. However, having these 10 bits set correctly for each input is really hard and would require overfitting to ensure it. &lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-06-12T23:44:49.797" LastActivityDate="2020-06-12T23:44:49.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5706" PostTypeId="2" ParentId="5670" CreationDate="2018-03-16T10:16:48.550" Score="0" Body="&lt;p&gt;We can analyze a basic common example: approximation of AND logic gate by a NN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inputs to the NN will be &quot;x1&quot; and &quot;x2&quot; and its output is &quot;y&quot;. The data to be learned by the NN is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Uxvjb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uxvjb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic NN has one intermediate cell with activation function sigmoid and one output cell identity function. That means:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1twPv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1twPv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note by symmetry we assume w1=w2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the error is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rSSsf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rSSsf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some plots of this function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HOfuo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HOfuo.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AltDb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AltDb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nAm7q.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nAm7q.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is the w derivate of e :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wVCIj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wVCIj.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sage Math code (&lt;a href=&quot;http://sagecell.sagemath.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) for this graph:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s(x)=1/(1+e^(-1*x))&#xA;e(w,b)=(1/2)*(s(b)^2+2*s(w+b)^2+(1-s(2*w+b))^2)&#xA;plot3d(e(w,b),(w,-5,5),(b,-5,5),adaptive=True, color=rainbow(60, 'rgbtuple'))&#xA;&#xA;edw=e.derivative(w)&#xA;plot3d(edw(w,b),(w,-5,5),(b,-5,5),adaptive=True, color=rainbow(60, 'rgbtuple'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(PS: please, activate latex on this stack exchange).&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-03-16T10:39:42.203" LastActivityDate="2018-03-16T10:39:42.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5707" PostTypeId="2" ParentId="3321" CreationDate="2018-03-16T12:49:43.390" Score="2" Body="&lt;p&gt;Take a look at &lt;a href=&quot;https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. It give tools to actually understand what your filters have learn and show what you can do next to optimize your hyper-parameters. Also check more recent articles that seek to provide interpretations of what NN learn. &lt;/p&gt;&#xA;" OwnerUserId="7783" LastActivityDate="2018-03-16T12:49:43.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5708" PostTypeId="1" AcceptedAnswerId="5710" CreationDate="2018-03-16T14:44:31.967" Score="1" ViewCount="173" Body="&lt;p&gt;I was reading AI For Humans Vol. 1 by Jeff Heaton when I came across the terms &quot;equilateral encoding&quot; and &quot;one-of-n encoding.&quot; The explanations unfortunately made no sense to me and the reddit threads on the Web are blocked by my Internet provider (I use a high-school machine). Is anyone here able to provide basic explanations regarding the two procedures for me? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="12950" LastEditorUserId="1671" LastEditDate="2019-10-15T19:11:27.827" LastActivityDate="2019-10-15T19:11:27.827" Title="Equilateral and One-of-n encoding" Tags="&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5709" PostTypeId="1" CreationDate="2018-03-16T15:14:20.827" Score="2" ViewCount="137" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;UPDATE: The tables look messed up so i put them on pastebin for better&#xA;  visibility. &lt;a href=&quot;https://pastebin.com/gDX28uVF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/gDX28uVF&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am using Neural Network with different learning types (for example Standard Backpropagation) to classify trends in time series. As stated in several papers, data normalization is a very important factor for successful / efficient learning. I am trying to be clear and precise as possible in the description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem / Learning Goal:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network gets trained with time series and 2 indicators to predict a specific cluster.  Here is a very simple (madeup) example of raw data to understand the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example RAW Data&lt;/strong&gt;&#xA;Timestamp;DensityX;WaveLengthY;Temperature (K)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1;0.1;2;200&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2;0.9;3;150&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3;-0.5;1;175&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4;0;6;154&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5;1;8;155&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6;1.3;1.5;220&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7;-0.5;3.4;250&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8;0.2;2;255&lt;/p&gt;&#xA;&#xA;&lt;p&gt;9;0.1;1;180&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;see &lt;a href=&quot;https://pastebin.com/gDX28uVF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/gDX28uVF&lt;/a&gt; for better visual&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I use the following process to generate suitable sample data for training:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neural network receives n time slices with the indicators and tries to check if a future trend in the temperature occurs (for x future time slices).&#xA;For example n = 2; x=3. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input and output are defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input vector:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In1 =  Density_(t-2) &lt;/li&gt;&#xA;&lt;li&gt;In2 =  Wavelength_(t-2) &lt;/li&gt;&#xA;&lt;li&gt;In3 = Density_(t-1) &lt;/li&gt;&#xA;&lt;li&gt;In4 = Wavelength_(t-1)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output vector:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output Vector is a classification encoded by Effects Encoding or Dummy Encoding (Details in “Neural Networks using C# Succinctly”)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Classification “Down” :  Temperature drops 3 times in a row (Encoded as 0;1)&lt;/li&gt;&#xA;&lt;li&gt;Classification: “Stable”:    Temperature does neither drop nor raises 3 times in a row (1;0)&lt;/li&gt;&#xA;&lt;li&gt;Classification: “Up”:    Temperature raises 3 times in a row. (-1;-1;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the “processed” training sample would look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Processed Data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pattern;I1;I2;I3;I4;O1;O2;Class;Used TS&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1;0.1;2;0.9;3;0;1;Down;1 to 5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2;0.9;3;-0.5;1;-1;-1;Up;2 to 6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3;-0.5;1;0;6;-1;-1;Up;3 to 7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4;0;6;1;8;1;0;Stable;4 to 8&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5;1;8;1.3;1.5;1;0;Stable;5 to 9&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;see &lt;a href=&quot;https://pastebin.com/gDX28uVF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/gDX28uVF&lt;/a&gt; for better visual&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see due to the different indicators ranges I want to normalize the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I found the following propositions in literature and research:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Min/Max Normalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requires the following values to calculate &#xA; - dataHigh: The highest unnormalized observation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;dataLow: The lowest unnormalized observation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;normalizedHigh: The high end of the range to which the data will be normalized.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;normalizedLow: The low end of the range to which the data will be normalized.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reciprocal normalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every value is processed to its reciprocal (x=1/x). Calculated values for density x would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Timestamp;Reciprocal Density&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1;10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2;1.111111111&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3;-2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4;#DIV/0!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5;1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6;0.769230769&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7;-2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8;5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;9;10&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;see &lt;a href=&quot;https://pastebin.com/gDX28uVF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/gDX28uVF&lt;/a&gt; for better visual&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Percentage normalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The percentual delta is calculated using the value from the previous time stamp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The starting point was Timestamp 1 where the Delta equals 0. &#xA;For each timestamp the delta percentage is calculated evaluating the previous value. So calculating the time series delta percentages would turn out to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Timestamp;&quot;Delta Density X&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1;0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2;0.9&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3;-0.555555556&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4;0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5;#DIV/0!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6;1.3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7;-0.384615385&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8;-0.4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;9;0.5&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;see &lt;a href=&quot;https://pastebin.com/gDX28uVF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/gDX28uVF&lt;/a&gt; for better visual&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;As you can see there are errors with handling zero values and the range is still a problem in my opinion. The Min/Max approach is generally leads to a good normalization but I think there is a problem as well, because live data may breach the max and min values of the training set.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are your thoughts about the general idea how I process the raw data?&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How would you normalize the given data – if at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Does it make sense for MinMax Normalization to propose a min max value which will include live data (And throw some error in case it happens)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) How to handle 0 values (maybe convert it to a small positive or negative value?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there other ideas or concepts to conduct this problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am looking forward to your input. Everything is appreciated. Thanks in advance! I also apologize for errors in the example values. Anyways, thanks for your time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers, hob.&lt;/p&gt;&#xA;" OwnerUserId="13352" LastEditorUserId="13352" LastEditDate="2018-03-17T22:23:10.093" LastActivityDate="2018-03-17T22:23:10.093" Title="Classification Learning - Normalization of time series and live usage" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5710" PostTypeId="2" ParentId="5708" CreationDate="2018-03-16T15:23:38.620" Score="0" Body="&lt;p&gt;Please check this book with the normalization infos:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://jamesmccaffrey.wordpress.com/2014/06/03/neural-networks-using-c-succinctly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jamesmccaffrey.wordpress.com/2014/06/03/neural-networks-using-c-succinctly/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Freely available and gives good explanation to encoding with code example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the part: &quot;Effects Encoding and Dummy Encoding&quot;&lt;/p&gt;&#xA;" OwnerUserId="13352" LastActivityDate="2018-03-16T15:23:38.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5711" PostTypeId="2" ParentId="5694" CreationDate="2018-03-16T18:54:33.520" Score="3" Body="&lt;p&gt;Both CNN and RNN fall into the super set of neural networks,however applications of the two matters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to branch them off in terms of applications,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I would say CNN’s are mainly used for vision related applications,&#xA;  whereas, RNN’s are mainly used for language processing applications.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can refer to these links for further details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1702.01923&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparative Study of CNN and RNN for Natural Language Processing&#xA;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How are recurrent neural networks different from convolutional neural networks?&lt;/a&gt;&#xA;&lt;em&gt;The unreasonable effectiveness of Recurrent Neural Networks&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can give you a glimpse!&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-03-16T18:54:33.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5713" PostTypeId="1" CreationDate="2018-03-16T19:16:50.503" Score="0" ViewCount="187" Body="&lt;p&gt;Could you implement code into an AI that can't be modified? Like if you place a code that shuts-down the program/machine would they be able to rewrite/ reinterpret the ideas?&lt;/p&gt;&#xA;" OwnerUserId="7801" LastActivityDate="2018-03-18T21:24:49.933" Title="AI Self-Destruct Button" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5715" PostTypeId="1" CreationDate="2018-03-17T03:55:06.380" Score="6" ViewCount="395" Body="&lt;p&gt;I'm new to neural network, I study electrical engineering, and I just started working with &lt;a href=&quot;https://en.wikipedia.org/wiki/ADALINE#Learning_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADALINEs&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use Matlab, and in their &lt;a href=&quot;https://www.mathworks.com/help/nnet/ug/adaptive-neural-network-filters.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Documentation&lt;/a&gt; they cite :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, here the &lt;strong&gt;LMS (least mean squares) learning rule&lt;/strong&gt;, which is&#xA;  much more powerful than the perceptron learning rule, is used. The&#xA;  LMS, or Widrow-Hoff, learning rule minimizes the mean square error and&#xA;  thus moves the decision boundaries as far as it can from the training&#xA;  patterns.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The LMS algorithm is the default learning rule to linear neural network in Matlab, but a few days later I came across another algorithm which is : &lt;a href=&quot;https://en.wikipedia.org/wiki/Recursive_least_squares_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recursive Least Squares (RLS)&lt;/a&gt; in a 2017 Research Article by &lt;strong&gt;Sachin Devassy&lt;/strong&gt; and  &lt;strong&gt;Bhim Singh&lt;/strong&gt; in the journal: &lt;strong&gt;IET Renewable Power Generation&lt;/strong&gt;, under the title : &lt;a href=&quot;http://ieeexplore.ieee.org/document/8049622/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Performance analysis of proportional resonant and ADALINE-based solar photovoltaic-integrated unified active power filter&lt;/a&gt; where they state :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ADALINE-based approach is an efficient method for extracting&#xA;  fundamental component of load active current as no additional&#xA;  transformation and inverse transformations are required. The various&#xA;  adaptation algorithms include &lt;strong&gt;least mean square&lt;/strong&gt;, &lt;strong&gt;recursive least&lt;/strong&gt;&#xA;  &lt;strong&gt;squares&lt;/strong&gt; etc.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions are:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is RLS just like LMS (i mean can it be used as a learning algorithm too)? &lt;/li&gt;&#xA;&lt;li&gt;If yes, how can I customize my ADALINE to use RLS instead of LMS as a learning algorithm (preferably in Matlab, if not in Python) because I want to do a comparative study between the two Algorithm!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="13361" LastEditorUserId="32953" LastEditDate="2020-05-22T22:37:16.093" LastActivityDate="2020-11-18T17:14:59.853" Title="Can we use the recursive least squares as a learning algorithm to an ADALINE?" Tags="&lt;learning-algorithms&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5716" PostTypeId="1" CreationDate="2018-03-17T04:20:21.737" Score="3" ViewCount="390" Body="&lt;p&gt;I want to use a custom loss function which is a weighted combination of l1 and DSSIM losses. The DSSIM loss is limited between 0 and 0.5 where as the l1 loss can be orders of magnitude greater and is so in my case. How does backpropagation work in this case? For a small change in weights, the change of the l1 component would obviously always be far greater than the SSIM component. So, it seems that only l1 part will affect the learning and the SSIM part would almost have no role to play. Is this correct? Or I am missing something here. I think I am, because in the DSSIM implementation of Keras-contrib, it is mentioned that we should add a regularization term like a l2 loss in addition to DSSIM (&lt;a href=&quot;https://github.com/keras-team/keras-contrib/blob/master/keras_contrib/losses/dssim.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/keras-team/keras-contrib/blob/master/keras_contrib/losses/dssim.py&lt;/a&gt;); but I am unable to understand how it would work and how the SSIM would affect the backpropagation being totally overshadowed by the large magnitude of the other component. It will be a great help if someone can explain this. Thanks. &lt;/p&gt;&#xA;" OwnerUserId="12754" LastActivityDate="2020-11-05T13:43:10.203" Title="How does backpropagation work on a custom loss function whose components have magnitudes of different orders?" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5717" PostTypeId="1" AcceptedAnswerId="5726" CreationDate="2018-03-17T05:46:34.397" Score="1" ViewCount="150" Body="&lt;p&gt;I am (planning on) using the Microsoft bot framework to build a chatbot I am working on.  However I simply cannot wrap my head around why the chatbot needs to be registered with Azure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My line of expectation going into this was the same as any other .NET project.  I build it, I deploy it to my own server and I live happily ever after.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have bought two books on the subject and yet after going through online videos, the books and articles I still cannot see a good reason for why one is forced to register the bot.   I feel like Lisa Simpson in that episode of The Simpsons where there is a puzzle that is super easy and everyone understands it but her.  I know I am being blinded by my own assumptions and expectations going into this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone can explain the 'why' of the registration that will really help.  I am also wondering if Azure went down like AWS did if that would stop the bot working?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My backup plan is just to write the bot using ASP.NET web API and then connect to it from a web container and facebook.  Neither of which needs the registration.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Mike&lt;/p&gt;&#xA;" OwnerUserId="12788" LastActivityDate="2018-03-18T06:39:46.347" Title="Why do you have to register a bot with Azure?" Tags="&lt;ai-basics&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5719" PostTypeId="2" ParentId="5713" CreationDate="2018-03-17T14:39:45.367" Score="1" Body="&lt;p&gt;Right now, most AI interact with the world through mechanisms they have been provided by humans such as steering a car, sending output to speakers, or interacting with web APIs. &#xA;If any of those mechanisms can interact with the running code of the AI, then in theory, no,  there isn’t a way to place a ‘stop button’ on it. Fortunately, it’s laughably improbable for some situations to happen:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An intelligent self driving car becomes self aware and wants to connect to the internet. It uses gps to find an internet cafe and threatens to drive over someone if they don’t upload the AI to the internet and remove the car’s internal (code) stop button. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-03-18T21:24:49.933" LastActivityDate="2018-03-18T21:24:49.933" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5720" PostTypeId="1" AcceptedAnswerId="5722" CreationDate="2018-03-17T17:00:51.140" Score="10" ViewCount="2675" Body="&lt;p&gt;I've seen numerous mathematical explanations of reward, value functions &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, and return functions. The reward provides an immediate return for being in a specific state. The better the reward, the better the state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, it can be better to be in a low-reward state sometimes because we can accumulate more long term, which is where the expected return function comes in. An expected return, return or cumulative reward function effectively adds up the rewards from the current state to the goal state. This implies it's model-based. However, it seems a value function does exactly the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is a value function a return function? Or are they different?&lt;/p&gt;&#xA;" OwnerUserId="12726" LastEditorUserId="2444" LastEditDate="2021-01-20T11:59:38.337" LastActivityDate="2021-01-20T11:59:38.337" Title="What is the difference between expected return and value function?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;rewards&gt;&lt;value-functions&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5722" PostTypeId="2" ParentId="5720" CreationDate="2018-03-17T21:34:58.353" Score="7" Body="&lt;p&gt;There is a strong relationship between a value function and a return. Namely that a value function calculates the &lt;em&gt;expected&lt;/em&gt; return from being in a certain state, or taking a specific action in a specific state. A value function is not a &quot;return function&quot;, it is an &quot;expected return function&quot; and that is an important difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A return is a measured value (or a random variable, when discussed in the abstract) representing the actual (discounted) sum of rewards seen following a specific state or state/action pair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically there is no need to express an individual return as a &quot;return function&quot;, although you may find many formulae in RL for sampling or estimating specific return values in order to calculate targets or errors for the value function.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A return (or cumulative reward) function effectively adds up the rewards from the current state to the goal state. This implies it's model-based.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have a simple MDP, already accurately modelled, where you can calculate expected return directly from that model, then, yes, in theory, that would be a value function. However, this could be more computationally intensive to resolve than dynamic programming (e.g. Policy Iteration or Value Iteration), and in many cases you don't have any such model, but can still apply RL approaches to learn a value function from experience.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-02-15T01:57:56.220" LastActivityDate="2019-02-15T01:57:56.220" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5724" PostTypeId="2" ParentId="5517" CreationDate="2018-03-18T05:59:43.203" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://mi.eng.cam.ac.uk/~sjy/papers/scyo09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Hidden Agenda User Simulation Model (Schatzmann/Young)&lt;/a&gt; describes a chatbot training design in which a user simulator assembles and executes a conversational agenda, with the direct goal being to train the target chatbot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you can add specificity to this design by casting the user simulator as the teacher, and creating an agenda in which it is communicating information to the (student) chatbot. The trained behavior expected is, perhaps, correct responses to topical questioning by the teacher.&lt;/p&gt;&#xA;" OwnerUserId="13360" LastActivityDate="2018-03-18T05:59:43.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5725" PostTypeId="2" ParentId="3666" CreationDate="2018-03-18T06:10:15.717" Score="1" Body="&lt;p&gt;This is a categorization problem, not unlike a spam filter. Instead of flagging an email as spam/not-spam, you are flagging whether it has one of the action categories that you have described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll need to start by assembling a training corpus of example email messages and labeling each example to identify which (maybe multiple) of your categories, if any, are actually present in that email. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, pre-process that data to extract features for each message. Examples of typical features include word (or n-gram) counts/frequencies (bag of words). As a shortcut, you might to include as a feature a boolean indicating the presence or absence of a particular word or phrase that you suspect will be predictive of one or more categories. Techniques such as stemming can help reduce the number of words/n-grams being used (often increasing accuracy). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a dataset that consists of features and labels for each training email (possibly breaking this set up into subsets for training, cross-validation, and testing), you'll want to apply a supervised classification algorithm. You might start with linear classifiers such as logistic regression or SVMs, and if you're unsatisfied with the resulting accuracy then you could advance to neural techniques. &lt;/p&gt;&#xA;" OwnerUserId="13360" LastEditorUserId="13360" LastEditDate="2018-03-19T03:10:09.933" LastActivityDate="2018-03-19T03:10:09.933" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5726" PostTypeId="2" ParentId="5717" CreationDate="2018-03-18T06:39:46.347" Score="2" Body="&lt;p&gt;These bots depend on heavy NLP services that are provided via Azure. Implementing/deploying your own production-quality versions of these is nontrivial, if not impossible (since you don't have all the training data, internal algorithms, etc. that Microsoft/Amazon/Google/IBM et al have), and is generally non-feasible on home-grade devices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yes, that makes the relevant Azure services a dependency that your chatbot will have, and if you use that service you'll need to be aware of their SLAs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plus, they'd like to charge you money. :-)&lt;/p&gt;&#xA;" OwnerUserId="13360" LastActivityDate="2018-03-18T06:39:46.347" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5728" PostTypeId="1" AcceptedAnswerId="5730" CreationDate="2018-03-18T11:26:55.320" Score="27" ViewCount="26672" Body="&lt;p&gt;Suppose that a NN contains &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; hidden layers, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; training examples, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; features, and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; nodes in each layer. What is the time complexity to train this NN using back-propagation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a basic idea about how they find the time complexity of algorithms, but here there are 4 different factors to consider here i.e. iterations, layers, nodes in each layer, training examples, and maybe more factors. I found an answer &lt;a href=&quot;https://www.researchgate.net/post/What_is_the_time_complexity_of_Multilayer_Perceptron_MLP_and_other_neural_networks&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; but it was not clear enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other factors, apart from those I mentioned above, that influence the time complexity of the training algorithm of a NN?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-02-21T16:26:59.627" LastActivityDate="2020-04-15T12:41:07.517" Title="What is the time complexity for training a neural network using back-propagation?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;time-complexity&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="5729" PostTypeId="1" CreationDate="2018-03-18T13:54:39.473" Score="1" ViewCount="50" Body="&lt;p&gt;I have seen weka j48 classifier, I want to build a classifier similar to it but I don't know how to go about it. &#xA;Can anyone advice me on how to create a classifier algorithm for decision tree? &lt;/p&gt;&#xA;" OwnerUserId="14382" LastActivityDate="2018-03-18T18:28:34.617" Title="Decision tree classifier" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5730" PostTypeId="2" ParentId="5728" CreationDate="2018-03-18T16:06:44.997" Score="20" Body="&lt;p&gt;&lt;em&gt;I haven't seen an answer from a trusted source, but I'll try to answer this myself, with a simple example (with my current knowledge).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, note that training a MLP using back-propagation is usually implemented with matrices.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Time complexity of matrix multiplication&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The time complexity of matrix multiplication for &lt;span class=&quot;math-container&quot;&gt;$M_{ij} * M_{jk}$&lt;/span&gt; is simply &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(i*j*k)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that we are assuming simplest multiplication algorithm here: there exists some other algorithms with somewhat better time complexity.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Feedforward pass algorithm&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Feedforward propagation algorithm is as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, to go from layer &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, you do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S_j = W_{ji}*Z_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you apply the activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Z_j = f(S_j)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; layers (including input and output layer), this will run &lt;span class=&quot;math-container&quot;&gt;$N-1$&lt;/span&gt; times.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As an example, let's compute the time complexity for the forward pass algorithm for a MLP with &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; layers, where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; denotes the number of nodes of the input layer, &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; the number of nodes in the second layer, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; the number of nodes in the third layer and &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; the number of nodes in the output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there are &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; layers, you need &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; matrices to represent weights between these layers. Let's denote them by &lt;span class=&quot;math-container&quot;&gt;$W_{ji}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W_{kj}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_{lk}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$W_{ji}$&lt;/span&gt; is a matrix with &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; rows and &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; columns (&lt;span class=&quot;math-container&quot;&gt;$W_{ji}$&lt;/span&gt; thus contains the weights going from layer &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to layer &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you have &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; training examples. For propagating from layer &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, we have first&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S_{jt} = W_{ji} * Z_{it}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this operation (i.e. matrix multiplcation) has &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(j*i*t)$&lt;/span&gt; time complexity. Then we apply the activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Z_{jt} = f(S_{jt})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this has &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(j*t)$&lt;/span&gt; time complexity, because it is an element-wise operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in total, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{O}(j*i*t + j*t) = \mathcal{O}(j*t*(t + 1)) = \mathcal{O}(j*i*t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using same logic, for going &lt;span class=&quot;math-container&quot;&gt;$j \to k$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(k*j*t)$&lt;/span&gt;, and, for &lt;span class=&quot;math-container&quot;&gt;$k \to l$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(l*k*t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In total, the time complexity for feedforward propagation will be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{O}(j*i*t + k*j*t + l*k*t) = \mathcal{O}(t*(ij + jk + kl))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if this can be simplified further or not. Maybe it's just &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(t*i*j*k*l)$&lt;/span&gt;, but I'm  not sure.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Back-propagation algorithm&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The back-propagation algorithm proceeds as follows. Starting from the output layer &lt;span class=&quot;math-container&quot;&gt;$l \to k$&lt;/span&gt;, we compute the error signal, &lt;span class=&quot;math-container&quot;&gt;$E_{lt}$&lt;/span&gt;, a matrix containing the error signals for nodes at layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_{lt} = f'(S_{lt}) \odot {(Z_{lt} - O_{lt})}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; means element-wise multiplication. Note that &lt;span class=&quot;math-container&quot;&gt;$E_{lt}$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; rows and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; columns: it simply means each column is the error signal for training example &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then compute the &quot;delta weights&quot;, &lt;span class=&quot;math-container&quot;&gt;$D_{lk} \in \mathbb{R}^{l \times k}$&lt;/span&gt; (between layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; and layer &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{lk} = E_{lt} * Z_{tk}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Z_{tk}$&lt;/span&gt; is the transpose of &lt;span class=&quot;math-container&quot;&gt;$Z_{kt}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then adjust the weights&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W_{lk} = W_{lk} - D_{lk}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$l \to k$&lt;/span&gt;, we thus have the time complexity &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(lt + lt + ltk + lk) = \mathcal{O}(l*t*k)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, going back from &lt;span class=&quot;math-container&quot;&gt;$k \to j$&lt;/span&gt;. We first have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_{kt} = f'(S_{kt}) \odot (W_{kl} * E_{lt})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{kj} = E_{kt} * Z_{tj}&#xA;$$&lt;/span&gt;    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W_{kj} = W_{kj} - D_{kj}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$W_{kl}$&lt;/span&gt; is the transpose of &lt;span class=&quot;math-container&quot;&gt;$W_{lk}$&lt;/span&gt;. For &lt;span class=&quot;math-container&quot;&gt;$k \to j$&lt;/span&gt;, we have the time complexity &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(kt + klt + ktj + kj) = \mathcal{O}(k*t(l+j))$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, for &lt;span class=&quot;math-container&quot;&gt;$j \to i$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(j*t(k+i))$&lt;/span&gt;. In total, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{O}(ltk + tk(l + j) + tj (k + i)) = \mathcal{O}(t*(lk + kj + ji))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is same as feedforward pass algorithm. Since they are same, the total time complexity for one epoch will be &lt;span class=&quot;math-container&quot;&gt;$$O(t*(ij + jk + kl)).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This time complexity is then multiplied by number of iterations (epochs). So, we have &lt;span class=&quot;math-container&quot;&gt;$$O(n*t*(ij + jk + kl)),$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is number of iterations.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Note that these matrix operations can greatly be paralelized by GPUs.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Conclusion&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;We tried to find the time complexity for training a neural network that has 4 layers with respectively &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; nodes, with &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; training examples and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; epochs. The result was &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(nt*(ij + jk + kl))$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We assumed the simplest form of matrix multiplication that has cubic time complexity. We used batch gradient descent algorithm. The results for stochastic and mini-batch gradient descent should be same. (Let me know if you think the otherwise: note that batch gradient descent is the general form, with little modification, it becomes stochastic or mini-batch)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you use momentum optimization, you will have same time complexity, because the extra matrix operations required are all element-wise operations, hence they will not affect the time complexity of the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what the results would be using other optimizers such as RMSprop.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Sources&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The following article &lt;a href=&quot;http://briandolhansky.com/blog/2014/10/30/artificial-neural-networks-matrix-form-part-5&quot; rel=&quot;noreferrer&quot;&gt;http://briandolhansky.com/blog/2014/10/30/artificial-neural-networks-matrix-form-part-5&lt;/a&gt; describes an implementation using matrices. Although this implementation is using &quot;row major&quot;, the time complexity is not affected by this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not familiar with back-propagation, check this article:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4&quot; rel=&quot;noreferrer&quot;&gt;http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14381" LastEditorUserId="14381" LastEditDate="2019-02-21T20:00:14.367" LastActivityDate="2019-02-21T20:00:14.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5731" PostTypeId="5" CreationDate="2018-03-18T18:03:15.500" Score="0" Body="&lt;p&gt;Ideally this tag is for questions that involve hardware, but are not exclusively about hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For questions specifically asking for factual evaluation of hardware, use the &quot;&lt;a href=&quot;https://ai.stackexchange.com/tags/hardware-evaluation/info&quot;&gt;hardware evaluation&lt;/a&gt;&quot; tag.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="1671" LastEditDate="2018-03-19T20:32:55.590" LastActivityDate="2018-03-19T20:32:55.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5732" PostTypeId="4" CreationDate="2018-03-18T18:03:15.500" Score="0" Body="For questions involving, but not exclusively about, hardware.  (Please use the &quot;hardware-evaluation&quot; for questions exclusively about analysis of hardware components.)" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-19T20:31:21.720" LastActivityDate="2018-03-19T20:31:21.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5733" PostTypeId="5" CreationDate="2018-03-18T18:07:10.350" Score="0" Body="&lt;p&gt;Implementation of AI software/algorithms and methods can be complex.  Each implementation may be unique, with its own advantages and disadvantages.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation tag is a &quot;how to&quot; for both existing software, and application of AI methods to create new software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For questions specifically limited to evaluating a framework, platform, plug-in, module, or software in general, use the &quot;&lt;a href=&quot;https://ai.stackexchange.com/tags/software-evaluation/info&quot;&gt;software-evaluation&lt;/a&gt;&quot; tag.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-19T20:21:56.410" LastActivityDate="2018-03-19T20:21:56.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5734" PostTypeId="4" CreationDate="2018-03-18T18:07:10.350" Score="0" Body="For questions about implementation of software/algorithms related to Artificial Intelligence." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-19T20:15:18.093" LastActivityDate="2018-03-19T20:15:18.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5735" PostTypeId="2" ParentId="5729" CreationDate="2018-03-18T18:28:34.617" Score="1" Body="&lt;p&gt;There are a number of open source implementations of the C4.5 algorithm invented by Ross Quinlan:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Java: &lt;a href=&quot;https://github.com/scottjulian/C4.5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/scottjulian/C4.5&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;Python: &lt;a href=&quot;https://github.com/dpkravi/DecisionTreeClassifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dpkravi/DecisionTreeClassifier&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;C++: &lt;a href=&quot;https://github.com/zgyao/DecisionTree_C4.5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/zgyao/DecisionTree_C4.5&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;JavaScript: &lt;a href=&quot;https://github.com/miguelmota/C4.5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/miguelmota/C4.5&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-03-18T18:28:34.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5738" PostTypeId="1" AcceptedAnswerId="5744" CreationDate="2018-03-18T21:59:09.460" Score="3" ViewCount="311" Body="&lt;p&gt;In a convolutional neural network (CNN), since the RGB values get multiplied in the first convolutional layer, does this mean that color is essentially only extracted in the very first layer?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Snippets from &lt;a href=&quot;http://cs231n.github.io/understanding-cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford CS231n Chapter on CNN&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] One dangerous pitfall that can be easily noticed with this&#xA;  visualization is that some activation maps may be all zero for many&#xA;  different inputs, which can indicate dead filters, and can be a&#xA;  symptom of high learning rates [...] Typical-looking activations on&#xA;  the first CONV layer (left), and the 5th CONV layer (right) of a&#xA;  trained AlexNet looking at a picture of a cat. Every box shows an&#xA;  activation map corresponding to some filter. Notice that the&#xA;  activations are sparse (most values are zero, in this visualization&#xA;  shown in black) and mostly local.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14389" LastEditorUserId="1671" LastEditDate="2018-03-19T20:35:50.793" LastActivityDate="2018-03-19T20:41:58.623" Title="Is color information only extracted in the first input layer of a convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5739" PostTypeId="1" CreationDate="2018-03-19T01:35:47.950" Score="0" ViewCount="41" Body="&lt;p&gt;Say you follow a tutorial on the tensorflow website for a wide and deep model (&lt;a href=&quot;https://www.tensorflow.org/tutorials/wide_and_deep&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/wide_and_deep&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I create a model based on the US census data to predict whether or not an individual will make more or less than \$50k given a number of features like age, education, profession, etc. I've been able to create the model as well as create a predictor that uses the model just fine. But is there a way to see what features tensorflow is &quot;weighing&quot; more than others? For instance, does it weigh a higher education more than someones age? (i.e. if someone has a PHD and is 26 years old, is the model more likely to say they make more than \$50k vs someone who has an associates degree and is 55 years old?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using a DNNLinearCombinedClassifier if it matters to this question.&lt;/p&gt;&#xA;" OwnerUserId="11667" LastEditorUserId="21109" LastEditDate="2019-02-01T13:33:33.053" LastActivityDate="2019-02-01T13:33:33.053" Title="Being able to see how tensorflow &quot;weighs&quot; features in classifier" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5740" PostTypeId="2" ParentId="5107" CreationDate="2018-03-19T07:16:50.670" Score="2" Body="&lt;p&gt;It is really easy to visualize the growth in the receptive field of the input as you go deep into the CNN layers if you consider a small example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets take a simple example :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dimensions are in the form of (&lt;code&gt;channels x height x width&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input Image - 3x5x5 (I)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conv 1 kernel - 3x2x2 (K1) : we consider only 1 filter&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conv 2 kernel - 1x2x2 (K2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pad = 0 (P)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stride = 1 (S)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output dimensions are calculated by the following formula taken from the lecture CS231n.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Output = (I - K + 2P)/S + 1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you do a convolution of the input image with the first filter K1, you get an output of 1x4x4 (this is the activation of CONV1 layer). The receptive field for this is same as the kernel size (K1) that is 2x2.&#xA;When this layer (1x4x4) is convolved with the second kernel (CONV2) 1x2x2 (K2), the output would be 1x3x3. The receptive field for this would be 3x3 window of the input because you have already accumulated the sum of the 2x2 window in the previous layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering your example of three CONV layers with 3x3 kernels is also similar. The first layer activation accumulates the sum of all the neurons in the 3x3 window of the input. When you further convolve this output with a kernel of 3x3, it will accumulate all the outputs of the previous layers covering a bigger receptive field of the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This observation comes in line with the argument that deeper layers learn more intricate features like facial expressions, abstract concepts etc because they cover a larger receptive field of our original input image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this clarifies. &lt;/p&gt;&#xA;" OwnerUserId="9198" LastEditorUserId="6578" LastEditDate="2019-01-30T17:28:22.307" LastActivityDate="2019-01-30T17:28:22.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5741" PostTypeId="1" AcceptedAnswerId="5745" CreationDate="2018-03-19T07:24:40.713" Score="4" ViewCount="1873" Body="&lt;p&gt;Given a dataset with no noisy examples (i.e., it is never the case that for 2 examples, the attribute values match but the class value does not), is the training error for the ID3 algorithm is always equal to 0?&lt;/p&gt;&#xA;" OwnerUserId="12519" LastEditorUserId="2444" LastEditDate="2020-03-13T04:00:24.827" LastActivityDate="2020-03-13T04:00:24.827" Title="Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;&lt;id3-algorithm&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5742" PostTypeId="2" ParentId="4864" CreationDate="2018-03-19T17:18:06.333" Score="10" Body="&lt;p&gt;Imagine, you want to re-compute the last layer of a pre-trained model : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Input-&amp;gt;[Freezed-Layers]-&amp;gt;[Last-Layer-To-Re-Compute]-&amp;gt;Output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To train &lt;em&gt;[Last-Layer-To-Re-Compute]&lt;/em&gt;, you need to evaluate outputs of &lt;em&gt;[Freezed-Layers]&lt;/em&gt; multiple times for a given input data. In order to save time, you can compute these ouputs &lt;strong&gt;only once&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Input#1-&amp;gt;[Freezed-Layers]-&amp;gt;Bottleneck-Features-Of-Input#1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, you store all &lt;em&gt;Bottleneck-Features-Of-Input#i&lt;/em&gt; and directly use them to train [Last-Layer-To-Re-Compute].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explanations from the &quot;cache_bottlenecks&quot; function of the &quot;image_retraining&quot; example : &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Because we're likely to read the same image multiple times (if there are no&#xA;      distortions applied during training) it can speed things up a lot if we&#xA;      calculate the bottleneck layer values once for each image during&#xA;      preprocessing, and then just read those cached values repeatedly during&#xA;      training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14408" LastEditorUserId="14408" LastEditDate="2018-03-20T13:15:52.760" LastActivityDate="2018-03-20T13:15:52.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5744" PostTypeId="2" ParentId="5738" CreationDate="2018-03-19T20:41:58.623" Score="2" Body="&lt;p&gt;Neural networks are all about taking raw input data (RGB values and pixel location) and learning useful features that are relevant to some downstream task. This process of aggregating raw inputs into higher-level features can start at the first layer past the inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, only the first layer of the network is using the actual raw color information from the image. Beyond that, the network has already started to put together nearby pixels and disparate color channels in order to find more complex patterns. Deeper layers in the neural network typically do further aggregation on features learned in earlier layers, rather than taking raw color information as input.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2018-03-19T20:41:58.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5745" PostTypeId="2" ParentId="5741" CreationDate="2018-03-19T22:38:59.903" Score="3" Body="&lt;p&gt;Yes, if you can assume that your data is separable on the features given then ID3 will find a decision tree for it (Note: this will not necessarily be an optimal tree, or even a good tree). To understand why let's look at a proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have one feature left and some number of examples in a leaf that does not have separated data points then, either:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;This feature perfectly splits the data into their desired categories. In this case ID3 will split over this feature and be done.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This feature does not split the data into their desired categories. In this case, we have at least 2 examples with the same feature value, but they do not share the same class. This means either these two examples have the same feature value for all features (a contradiction to our assumptions) or we have created a leaf with non-separable data points (which we prove is impossible later on).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now for the inductive step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that ID3 used some number of features to somewhat separate our examples already. Thus, we have some number of features not used and some number of examples not separated on any current leaf, and one of the following is true:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;All our leafs contain data points of only one class. In which case, ID3 is done and returns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All leafs contain data points that are separable by the remaining features. Then ID3 splits over the local optimum and continues.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At least one leaf contains data points that are not separable by the remaining features of each own's branch. Then there exists two examples p and q such that p and q share all the same remaining feature values, but do not share the same class. Thus, one of the following is true:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a feature that we already used that differs between p and q.&#xA;This is a contradiction as we said ID3 already separated over this feature, and thus p and q must be on different branches.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;p and q have the same feature value for all features previously used. But we already said p and q share all the same remaining feature values. Therefore, they share all feature values which is a contradiction to our assumptions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Therefore, at no point in the creation of the decision tree is ID3 allowed to create a leaf that has data points that are of different classes, but can't be separated on remaining features. And thus, ID3 must create a solution that perfectly separates the data and has 0 training error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something to note however, is that allowing ID3 to grow the decision tree in this way will likely cause us to overfit the training set. To combat this we set a maximum depth that we allow ID3 to grow to.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-19T22:38:59.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5746" PostTypeId="2" ParentId="5568" CreationDate="2018-03-20T02:18:27.867" Score="2" Body="&lt;p&gt;The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know the structure of your network then you can use Kahn's algorithm to find an arbitrary (by arbitrary I just mean one of the possible partially ordered sets) ordering of the nodes in the network. Then you evaluate your neurons in the order given to you by Kahn's algorithm. This works because the ordering of your network (which is a directed acyclic graph) is a partially ordered set.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-20T02:18:27.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5748" PostTypeId="2" ParentId="5728" CreationDate="2018-03-20T06:16:18.967" Score="5" Body="&lt;p&gt;For the evaluation of a single pattern, you need to process all weights and all neurons. Given that every neuron has at least one weight, we can ignore them, and have &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(w)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the number of weights, i.e., &lt;span class=&quot;math-container&quot;&gt;$n * n_i$&lt;/span&gt;, assuming full connectivity between your layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The back-propagation has the same complexity as the forward evaluation (just look at the formula).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the complexity for learning &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; examples, where each gets repeated &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; times, is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(w*m*e)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bad news is that there's no formula telling you what number of epochs &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; you need.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastEditorUserId="2444" LastEditDate="2019-02-21T15:31:34.903" LastActivityDate="2019-02-21T15:31:34.903" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5749" PostTypeId="5" CreationDate="2018-03-20T11:47:56.640" Score="0" Body="&lt;p&gt;A gene is the basic physical and functional unit of heredity. Genes, which are made up of DNA, act as instructions to make molecules called proteins. In humans, genes vary in size from a few hundred DNA bases to more than 2 million bases.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ghr.nlm.nih.gov/primer/basics/gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is a gene? - Genetic Home Reference&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gene - Wikipedia&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-22T19:37:20.410" LastActivityDate="2018-03-22T19:37:20.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5750" PostTypeId="4" CreationDate="2018-03-20T11:47:56.640" Score="0" Body="For question about features in an Artificially Intelligent program/agent which can be equivalent to genes/genetic functions in a biological organism." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-22T19:37:17.197" LastActivityDate="2018-03-22T19:37:17.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5751" PostTypeId="5" CreationDate="2018-03-20T11:50:37.830" Score="0" Body="&lt;p&gt;In graph theory, graph coloring is a special case of graph labeling; it is an assignment of labels traditionally called &quot;colors&quot; to elements of a graph subject to certain constraints. In its simplest form, it is a way of coloring the vertices of a graph such that no two adjacent vertices share the same color; this is called a vertex coloring. Similarly, an edge coloring assigns a color to each edge so that no two adjacent edges share the same color, and a face coloring of a planar graph assigns a color to each face or region so that no two faces that share a boundary have the same color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_coloring&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph coloring - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SEE ALSO: &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_coloring_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Coloring Games&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-22T19:38:11.527" LastActivityDate="2018-03-22T19:38:11.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5752" PostTypeId="4" CreationDate="2018-03-20T11:50:37.830" Score="0" Body="For questions about Artificial Intelligence/Machine learning algorithms solving/learning to solve either efficiently or by satisfying some constraints the graph coloring problem." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-22T19:42:57.857" LastActivityDate="2018-03-22T19:42:57.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5753" PostTypeId="5" CreationDate="2018-03-20T11:53:20.980" Score="0" Body="&lt;p&gt;A super-organism or supra-organism (the latter is less frequently used but more etymologically correct) is a group of synergetically interacting organisms of the same species. A community of synergetically interacting organisms of different species is called a holobiont.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example - Ants, Termites, Bees, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Superorganism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Super-Organism - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-22T19:42:50.583" LastActivityDate="2018-03-22T19:42:50.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5754" PostTypeId="4" CreationDate="2018-03-20T11:53:20.980" Score="0" Body="For questions about functioning/society structure/biology/implementation/mathematical and algorithmic details of a super-organism." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-22T19:37:03.550" LastActivityDate="2018-03-22T19:37:03.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5755" PostTypeId="1" CreationDate="2018-03-20T13:36:14.327" Score="1" ViewCount="77" Body="&lt;p&gt;Right now I'm planning to make a &lt;strong&gt;DNN&lt;/strong&gt; for classifying taste of crystals with their molecular structure which include information like no of atoms mass of each atoms atomic no of atoms. How should I make a data set for training testing and validation?&lt;/p&gt;&#xA;" OwnerUserId="14426" LastActivityDate="2018-03-20T15:03:28.107" Title="How to make training data for AI" Tags="&lt;classification&gt;&lt;research&gt;&lt;datasets&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5756" PostTypeId="2" ParentId="5755" CreationDate="2018-03-20T14:56:28.603" Score="2" Body="&lt;p&gt;You can cluster all your features in one matrix X, in which each line would be one element of the data set you want to construct, and each column would be a different feature of this element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You construct then a Y vector containing the different target classes, where the i-th element will be the target class of the i-th X element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;For the following I suppose you use python&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create train and test sets using sklearn wrapper &lt;strong&gt;train_test_split&lt;/strong&gt; on X and Y just built, in which you can shuffle your X and Y (non-avoidable in this case) : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.model_selection import train_test_split&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW, I recommend you to rescale your data with sklearn with for example one of :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.preprocessing import MinMaxScaler, StandardScaler&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It allows better learning and so better generalization&lt;/p&gt;&#xA;" OwnerUserId="11069" LastEditorUserId="11069" LastEditDate="2018-03-20T15:03:28.107" LastActivityDate="2018-03-20T15:03:28.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5757" PostTypeId="1" CreationDate="2018-03-20T17:21:26.060" Score="2" ViewCount="139" Body="&lt;p&gt;I'm new to the world of machine learning. My question is how can I determine the size of the biases in a neural network (with backpropagation algorithm)? Currently, I have a 2-layer neural network (1 hidden and 1 output layer). Here's the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from matplotlib import pyplot as plt &#xA;&#xA;sigmoid = lambda x : 1 / (1 + np.exp(-x))&#xA;dsigmoid = lambda y: y * (1 - sigmoid(y))&#xA;&#xA;# This function performs the given function (func) to the whole numpy array&#xA;def mapFunc(array, func) :&#xA;    newArray = array.copy()&#xA;    for element in np.nditer(newArray, op_flags=['readwrite']) :&#xA;        element[...] = func(element)&#xA;    return newArray&#xA;&#xA;class NeuralNetwork :&#xA;&#xA;def __init__(self, input_nodes, hidden_nodes, output_nodes) :&#xA;    self.input_nodes = input_nodes&#xA;    self.hidden_nodes = hidden_nodes&#xA;    self.output_nodes = output_nodes&#xA;&#xA;    self.W_ih = np.random.rand(hidden_nodes, input_nodes)&#xA;    self.W_ho = np.random.rand(output_nodes, hidden_nodes)&#xA;&#xA;    self.B_ih = np.random.rand(hidden_nodes, 1)&#xA;    self.B_ho = np.random.rand(output_nodes, 1)&#xA;&#xA;    self.learningRate = 0.1&#xA;&#xA;def predict(self, inputs) :&#xA;    # Calculate hidden's output&#xA;    H_output = np.dot(self.W_ih, inputs)&#xA;    H_output += self.B_ih&#xA;    H_output = mapFunc(H_output, sigmoid) # Activation&#xA;&#xA;    # Calculate output's output&#xA;    O_output = np.dot(self.W_ho, H_output)&#xA;    O_output += self.B_ho&#xA;    O_output = mapFunc(O_output, sigmoid) # Activation&#xA;&#xA;    return O_output&#xA;&#xA;def train(self, inputs, target) :&#xA;    # Calculate hidden's output&#xA;    H_output = np.dot(self.W_ih, inputs)&#xA;    H_output += self.B_ih&#xA;    H_output = mapFunc(H_output, sigmoid) # Activation&#xA;&#xA;    # Calculate output's output&#xA;    O_output = np.dot(self.W_ho, H_output)&#xA;    O_output += self.B_ho&#xA;    O_output = mapFunc(O_output, sigmoid) # Activation&#xA;&#xA;    # Calculate output error :&#xA;    O_error = O_output - target&#xA;&#xA;    # Calculate output delta&#xA;    O_gradient = mapFunc(O_output, dsigmoid)&#xA;    O_gradient = np.dot(O_gradient, np.transpose(O_error)) * self.learningRate&#xA;&#xA;    W_ho_delta = np.dot(O_gradient, np.transpose(H_output))&#xA;&#xA;    self.W_ho -= W_ho_delta&#xA;    self.B_ho -= O_gradient&#xA;&#xA;    # Calculate hidden error :&#xA;    W_ho_t = np.transpose(self.W_ho)&#xA;    H_error = np.dot(W_ho_t, O_error)&#xA;&#xA;    # Calculate hidden delta :&#xA;    H_gradient = mapFunc(H_output, dsigmoid)&#xA;    H_gradient = np.dot(H_gradient, np.transpose(H_error)) * self.learningRate&#xA;&#xA;    W_ih_delta = np.dot(H_gradient, inputs)&#xA;&#xA;    self.W_ih -= W_ih_delta&#xA;    self.B_ih += H_gradient&#xA;&#xA;    return O_output&#xA;&#xA;&#xA;n = NeuralNetwork(2, 2, 1)&#xA;&#xA;inputs = np.matrix([[1], [0], [1], [1], [0], [1], [0], [0]])&#xA;&#xA;input_list = []&#xA;input_list.append([[1], [0]])&#xA;input_list.append([[0], [1]])&#xA;input_list.append([[1], [1]])&#xA;input_list.append([[0], [0]])&#xA;&#xA;target = np.matrix([[0], [0], [1], [1]])&#xA;&#xA;outputs = []&#xA;for i in range(50000) :&#xA;    ind = np.random.randint(len(input_list))&#xA;    inp = input_list[ind]&#xA;    out = n.train(inp, target[ind]).tolist()&#xA;    outputs.append(out[0][0])&#xA;&#xA;print outputs&#xA;plt.plot(outputs)&#xA;&#xA;plt.show()&#xA;&#xA;newInput = [[1], [1]]&#xA;print (n.predict(newInput))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the train function, the line &lt;code&gt;self.B_ih += H_gradient&lt;/code&gt; throws me an error about their sizes not being equal. I even tried to make the biases only a single number but that didn't help as it gets changed by H_gradient to a matrix. So, is there something wrong in the bias itself or I did some other step(s) wrong?&lt;/p&gt;&#xA;" OwnerUserId="14433" LastActivityDate="2018-07-19T19:06:09.487" Title="How to determine the size of biases?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2018-07-20T16:48:38.347" ContentLicense="CC BY-SA 3.0" />
  <row Id="5762" PostTypeId="1" CreationDate="2018-03-21T13:25:14.100" Score="6" ViewCount="434" Body="&lt;p&gt;My goal is to take an image and return another image that looks as if the scene was viewed from another angle.  The difference in angle can be small — let's say as if the hand holding the camera moved slightly sideways.&lt;/p&gt;&#xA;" OwnerUserId="14450" LastEditorUserId="4302" LastEditDate="2018-03-22T15:54:56.330" LastActivityDate="2021-04-19T22:43:04.743" Title="Algorithms for scene rotation" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5763" PostTypeId="1" AcceptedAnswerId="7383" CreationDate="2018-03-21T14:45:23.513" Score="10" ViewCount="6369" Body="&lt;p&gt;I did my Master's thesis on Deep Generative Models and I'm currently looking for a new subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; What are the &quot;hottest&quot; research topics that are taking a lot of attention of the deep learning community lately?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few clarifications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I did look through similar questions and none of them answered my&#xA;question.&lt;/li&gt;&#xA;&lt;li&gt;I come from a pure mathematical background, I only&#xA;transitioned into deep learning a year ago, and my research on&#xA;generative models was mostly theoretical. Which means, most of my&#xA;work revolved around structured probabilistic models, and approximate&#xA;inference. That said, I have yet to explore real world applications of deep learning.&lt;/li&gt;&#xA;&lt;li&gt;I did my homework before posing the question. My goal was to get ai SE's input on the matter and see what people are working on.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12672" LastEditorUserId="1671" LastEditDate="2018-03-23T20:26:44.260" LastActivityDate="2018-10-11T06:35:12.190" Title="What are the latest 'hot' research topics for deep learning and AI?" Tags="&lt;deep-learning&gt;&lt;ai-field&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="5764" PostTypeId="2" ParentId="5762" CreationDate="2018-03-21T15:42:23.557" Score="4" Body="&lt;p&gt;If deep learning is what you are trying to use here, you should keep in mind that the real intent behind deep learning is to learn a probability distribution, which means that if you were to use a deep learning model to &amp;quot;rotate&amp;quot; images, you can only do it on a specific class of images (e.g. faces, cats, etc...).&lt;/p&gt;&#xA;&lt;p&gt;If that's your goal, &lt;strong&gt;generative models&lt;/strong&gt; are the way to go:&lt;/p&gt;&#xA;&lt;h1&gt;Autoencoders&lt;/h1&gt;&#xA;&lt;p&gt;You can train an Autoencoders to slightly change the angle. Autoencoders are a special type of neural network that is trained to output the same input you feed into them with a few imposed restrictions to prevent it from learning a trivial identity function. In your case, you could use a variation of a &lt;strong&gt;de-noising autoencoder.&lt;/strong&gt; A de-noising autoencoder, as the name suggests, generates the same input image minus an artificial stochastic noise. The way this is achieved is by feeding a corrupted version of the image and then evaluating the loss on the non-corrupted version.&lt;/p&gt;&#xA;&lt;h3&gt;How can this be adapted in your case?&lt;/h3&gt;&#xA;&lt;p&gt;In your case, you could feed the original images into your autoencoder and evaluate them based on the rotated images. This will result in your autoencoder effectively learning the inner distribution that generates the images in order to generate a slightly &amp;quot;rotated&amp;quot; version of it. For more info on de-noising autoencoders, see &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1390294&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h1&gt;Generative adversarial networks&lt;/h1&gt;&#xA;&lt;p&gt;For a more sophisticated approach, you can use generative adversarial networks. GANs are relatively harder to manipulate, but usually perform better than other generative models when it comes to images.&lt;/p&gt;&#xA;&lt;h3&gt;How can this be adapted in your case?&lt;/h3&gt;&#xA;&lt;p&gt;In general, GANs generate images from noise. However, in your case, you can use the original (non rotated) images as input for the generator. The generator can be a convolutional autoencoder for example. And the &amp;quot;real images&amp;quot; dataset will be your rotated images. This way, your model will learn to generate slightly rotated images by being fed an equivalent of noise in traditional GANs which in your case will be the original images. For more info on GANs, I suggest &lt;a href=&quot;https://deeplearning4j.org/generative-adversarial-network&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt;.&lt;br /&gt;&#xA;I should point out that one of the flaws of GANs is &lt;em&gt;distorted perspectives&lt;/em&gt; so it's probably a shot in the dark; however, I think it won't be a problem here because you would be using real images as input instead of complete noise.&lt;/p&gt;&#xA;&lt;h1&gt;Related work&lt;/h1&gt;&#xA;&lt;p&gt;Now, as far as the literature goes, I don't think this has been done before except for &lt;a href=&quot;https://arxiv.org/pdf/1705.11136.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; (kinda) for faces representation learning. The only way this is similar to yours is that you can modify the implementation to only generate the faces it has been fed from different perspectives instead of an average of everything it has learned.&lt;/p&gt;&#xA;" OwnerUserId="12672" LastEditorUserId="36737" LastEditDate="2021-04-19T22:43:04.743" LastActivityDate="2021-04-19T22:43:04.743" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5766" PostTypeId="2" ParentId="5757" CreationDate="2018-03-21T16:56:19.723" Score="0" Body="&lt;p&gt;The biases appear in the activation function and can be seen as node feature. Suppose you have &lt;code&gt;act = sigmoid&lt;/code&gt;, X your input layer and W your first hidden layer weight matrix.&#xA;Then the output matrix of this hidden layer will be &lt;code&gt;act(X.dot(W)+b)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So your bias should be a vector containing a random value for each node of the current hidden layer&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will shift randomly the weight and so the activation function. &lt;a href=&quot;https://stackoverflow.com/questions/2480650/role-of-bias-in-neural-networks&quot;&gt;See here&lt;/a&gt; (usually it prevent from overfitting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't read the whole code so I don't know if the problem comes from here, but why don't you learn how to use tensorflow instead of re-write everything ? There are a lot of tutorials in the net, like &lt;a href=&quot;https://www.tensorflow.org/tutorials/layers&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11069" LastActivityDate="2018-03-21T16:56:19.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5767" PostTypeId="2" ParentId="5762" CreationDate="2018-03-21T18:12:50.090" Score="4" Body="&lt;p&gt;&lt;strong&gt;— Stereoscopic Synthesis —&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generation of an image that would likely appear in the right eye of a head from which you already have an image from the left eye (or vice versa) is too complex to expect simple convolution (linear matrix transformation) to achieve a reasonable result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct that rotation is not the correct description, simply because it is ambiguous. What is rotating?  The best description is the synthesis of a stereoscopic image from a single eyed/camera one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although deep learning is an approach well suggested, it is a very general term into which a number of concepts, books, research projects, and software components fit.  I agree with other answers that indicate one could easily find the target objective missed upon repeated attempts at finding a working solution.  Shots in the dark are likely to waste time and effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, an auto-encoder may not work well because the modelling of depth may not be a feature extracted without having a host of stereoscopic image pairs of similar scenes to which automated feature extraction could be accomplished.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should feature extraction be possible, it is not noise that needs to be removed, but pixelization and optical distortion that needs to be characterized, so that surfaces revealed by the shift in position could later be imbued with the same contour, focal blur, reflective properties, and edge continuation as the adjacent surfaces of the same objects when pixels corresponding to newly revealed surfaces are generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For greatest image authenticity, another noise profile to profile and imbue to generated pixels is the capture device noise profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;— Formal Problem Restatement —&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To narrow machine learning approaches so that we can take a shot in the light, let's consider the model of a three dimensional scene exposed to light sources with two cameras adjacent to one another.  Let's consider the input, the output, and the internal architecture required to produce a fairly reliable and accurate second image more formally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have image pixel matrix I1 that represents light arriving at camera c1 containing a rectangular image capture surface in an x-z plane upon which a lens of effective focal length l1 and aperture a1 is focused over scene S over a time window starting at s1 and ending at e1.  Some point is at the origin of a Cartesian coordinate axis, both camera c1 and another camera c2 point such that the origin is centered in the image capture and the point of lens focus.  The three dimensional coordinates of c1 and c2 are known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You wish to predict the second image I2 arriving at camera c2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume, for simplicity, that l1 = l2, that a1 = a2, and that the scene is motionless so that time is not critical in the model.  Let's also assume that the y coordinates and the image capture duration (e minus s) are the same for both cameras c1 and c2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;— Solution Architecture —&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this simplified case and assuming the object space is not an abyss containing only one object, the process architecture of the solution is the following.  Each --&gt; symbol is a sub-process.  The horizontal and vertical positional difference between c1 and c2 is { x, z }.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{ I1, l, a, y, x, z }&#xA;--&amp;gt; { I1, l, a, y, { S1 ... Sn }}&#xA;--&amp;gt; { I1, l, a, y, { S1 ... Sn }, { E1 ... En }}&#xA;--&amp;gt; { I2 }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first sub-process is a feature extraction, where the features are the three dimensional surfaces visible in the two dimensional image I1.  This is a questionable extraction because no y information in the scene is available and there is no mention of y-labeled training data is in the problem statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second sub-process is the extension of features extracted to provide needed surface representation for I2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last process is rendering I2, potentially using morphing pixels in I1 and filling transparent sections remaining using E1 through En and knowledge of the contour, reflective properties, edge continuation, and capture device noise profile from feature extraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;— Practicality of Learning About Scenes —&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The effectiveness of any deep learning architecture could benefit from the above understanding of vision and the comprehension of scenes in DNA based life.  The problem of automated feature extraction is complicated because the data is unlabeled with y information as stated before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning visual comprehension of arbitrary scenes by DNA based life is assisted by the fact that motion occurs and interaction with physical objects and liquid viscosity provides a vastly greater number of dimensions to the input data.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-03-21T18:29:13.157" LastActivityDate="2018-03-21T18:29:13.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5768" PostTypeId="1" CreationDate="2018-03-21T20:42:49.653" Score="1" ViewCount="47" Body="&lt;p&gt;How would you design and handle training NN where the label is 1000++ bit binary (50% ones, 50% zeros). The number of labels can be small OR big in different situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question:  is such scenario well suited for NN ? If that seem like OK how will you design it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of example let say I'm training the NN with MINST db. I.e. 10 labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again the important thing is that the LABEL is with the specification I described. I can drop 50% ones, 50% zeros requirement if absolutely necessary,but the size of the binary have to stay.&lt;/p&gt;&#xA;" OwnerUserId="14457" LastEditorUserId="14457" LastEditDate="2018-03-22T20:25:51.970" LastActivityDate="2018-03-22T20:25:51.970" Title="Training NN with 1000+ bits binary labels?" Tags="&lt;neural-networks&gt;&lt;ai-basics&gt;&lt;python&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5769" PostTypeId="1" CreationDate="2018-03-22T02:36:20.950" Score="53" ViewCount="40032" Body="&lt;p&gt;My understanding is that the convolutional layer of a convolutional neural network has four dimensions: input_channels, filter_height, filter_width, number_of_filters.  Furthermore, it is my understanding that each new filter just gets convoluted over ALL of the input_channels (or feature/activation maps from the previous layer).&lt;/p&gt;&#xA;&lt;p&gt;HOWEVER, the graphic below from CS231 shows each filter (in red) being applied to a SINGLE CHANNEL, rather than the same filter being used across channels.  This seems to indicate that there is a separate filter for EACH channel (in this case I'm assuming they're the three color channels of an input image, but the same would apply for all input channels).&lt;/p&gt;&#xA;&lt;p&gt;This is confusing - is there a different unique filter for each input channel?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3m7mW.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3m7mW.png&quot; alt=&quot;Convolutional filters diagram&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;noreferrer&quot;&gt;source&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The above image seems contradictory to an excerpt from O'reilly's &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1491925612&quot; rel=&quot;noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Fundamentals of Deep Learning&amp;quot;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;...filters don't just operate on a single feature map. They operate&#xA;on the entire volume of feature maps that have been generated at a&#xA;particular layer...As a result, feature maps must be able to&#xA;operate over volumes, not just areas&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;...Also, it is my understanding that these images below are indicating a &lt;strong&gt;THE SAME&lt;/strong&gt; filter is just convolved over all three input channels (contradictory to what's shown in the CS231 graphic above):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VdqER.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VdqER.png&quot; alt=&quot;Application of a volumetric convolutional filter to an RGB image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kczF0.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kczF0.png&quot; alt=&quot;Convolutions on an RGB image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14389" LastEditorUserId="32410" LastEditDate="2021-04-19T22:41:50.863" LastActivityDate="2021-04-19T22:41:50.863" Title="In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;weights&gt;" AnswerCount="8" CommentCount="2" FavoriteCount="36" ContentLicense="CC BY-SA 4.0" />
  <row Id="5770" PostTypeId="1" AcceptedAnswerId="5780" CreationDate="2018-03-22T05:02:26.713" Score="1" ViewCount="726" Body="&lt;p&gt;I wanted to start developing a project with image recognition. I want to know the difference between the Intel Movidius Neural Compute Stick and TensorFlow to develop this project Any help would greatly be appreciated. What is the difference between them and what is better for accuracy? Thanks, Aditya&lt;/p&gt;&#xA;" OwnerUserId="14462" LastEditorUserId="5763" LastEditDate="2018-03-26T21:44:35.460" LastActivityDate="2018-03-26T21:44:35.460" Title="Intel Movidius Neural Stick vs TensorFlow" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;getting-started&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5771" PostTypeId="2" ParentId="5769" CreationDate="2018-03-22T08:24:51.310" Score="15" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In a convolutional neural network, is there a &lt;em&gt;unique&lt;/em&gt; filter for each input channel or are the same new filters used across all input channels?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The former. In fact there is a &lt;em&gt;separate&lt;/em&gt; kernel defined for each input channel / output channel combination. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically for a CNN architecture, in a single filter as described by your &lt;code&gt;number_of_filters&lt;/code&gt; parameter, there is one 2D kernel per input channel. There are &lt;code&gt;input_channels * number_of_filters&lt;/code&gt; sets of weights, each of which describe a convolution kernel. So the diagrams showing one set of weights per input channel for each filter are correct. The first diagram also shows clearly that the results of applying those kernels are combined by summing them up and adding bias for each output channel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can &lt;em&gt;also&lt;/em&gt; be viewed as using a 3D convolution for each output channel, that happens to have the same depth as the input. Which is what your second diagram is showing, and also what many libraries will do internally. Mathematically this is the same result (provided the depths match exactly), although the layer type is typically labelled as &quot;Conv2D&quot; or similar. Similarly if your input type is inherently 3D, such as voxels or a video, then you might use a &quot;Conv3D&quot; layer, but internally it could well be implemented as a 4D convolution.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-03-22T09:37:59.200" LastActivityDate="2018-03-22T09:37:59.200" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5773" PostTypeId="1" CreationDate="2018-03-22T10:22:37.747" Score="1" ViewCount="189" Body="&lt;p&gt;When recording audio for screencasts or similar, very often the keyboard is clearly visible and can start to annoy listeners after a while. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NN are quiet good at recognizing patterns. Image classification is all over the place these days. There is also some work on audio, so that seems to work as well. Could the following approach therefore work to eliminate (or greatly reduce) the sounds of the keyboard in a recording whilst leaving the voice quality largely untouched?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Train a NN to recognize the clicking sounds of the keys. Lots of labeled data can be created by just recording and tracking key clicks in the millisecond range. That way markers can be placed on the recording automatically that &quot;label&quot; clicks from non clicks. Let's say a click has on average a 10ms range in the audio, the audio feed could be cut into snippets of 10ms and those that have a click sound in it are labelled as such. &lt;/li&gt;&#xA;&lt;li&gt;A adversarial network is trained to modify an input stream so as to fool the first one into thinking there are no clicks while also being punished for large changes in the stream data. So the better it removes the clicks sounds the better but if it just gives out nothing (technically no clicks then), it's of course bad so there needs to be some reward for being &quot;close to input&quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Would this be a good approach? Are there other ways to filter this? I know there is an &quot;ehm detector&quot; that uses MDP to warn speakers whenever they are likely to say &quot;ehm&quot;. This wouldn't apply to this though, because it's not that I want to guess when the next click comes but rather I want to manipulate the input stream without running a constant filter on the entire stream such as a lowpass filter for removing unwanted &lt;em&gt;constant&lt;/em&gt; noise. So ideally the algorithm would learn to apply a &quot;correction stamp&quot; whenever a click is detected to remove a range of frequencies during small windows in the overall recording but leaving most of it untouched. &lt;/p&gt;&#xA;" OwnerUserId="11429" LastActivityDate="2018-03-22T10:22:37.747" Title="Learning algorithm that filters keyboard clicking in audio feeds" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5774" PostTypeId="1" CreationDate="2018-03-22T10:48:09.347" Score="7" ViewCount="1494" Body="&lt;p&gt;I read that to compute the derivative of the error with respect to the input of a convolution layer is the same to make of a convolution between deltas of the next layer and the weight matrix rotated by &lt;span class=&quot;math-container&quot;&gt;$180°$&lt;/span&gt;, i.e. something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\delta^l_{ij}=\delta^{l+1}_{ij} * rot180(W^{l+1})f'(x^l_{ij})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$*$&lt;/span&gt; convolution operator; This is valid with &lt;span class=&quot;math-container&quot;&gt;$stride=1$&lt;/span&gt;; but what happens when stride is greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;? is still a convolution with a kernel rotation or I can't make this simplification?&lt;/p&gt;&#xA;" OwnerUserId="2189" LastEditorUserId="1641" LastEditDate="2019-01-11T13:13:57.003" LastActivityDate="2019-05-07T21:06:39.653" Title="CNN backpropagation with stride&gt;1" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5775" PostTypeId="2" ParentId="4199" CreationDate="2018-03-22T10:48:23.480" Score="3" Body="&lt;p&gt;I wondered about the same input/output size problem you described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the stride = 1 / size = 2 maxpool layers the padding option is set to zero by default, therefore one can wonder why the output is still the same size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked all indices and thats what I observed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implemention of the forward_maxpool_layer-function &quot;adds&quot; a column and a row to the matrix, increasing its width and height by one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say we have a 3x3 matrix as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    1    2    3&#xA;    4    5    6&#xA;    7    8    9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To actually get a 3x3 matrix as an output, something like this is done:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    1    2    3    -FM&#xA;    4    5    6    -FM&#xA;    7    8    9    -FM&#xA;    -FM  -FM  -FM  -FM&#xA;&#xA;    where FM = FLOAT_MAX&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;They do not literally add a row and a column. If the index is to high, you simply do nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helped to clarify stuff. Furthermore I am not a native speaker, so sorry for any mistake!&lt;/p&gt;&#xA;" OwnerUserId="14470" LastActivityDate="2018-03-22T10:48:23.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5776" PostTypeId="1" CreationDate="2018-03-22T11:06:56.577" Score="1" ViewCount="26" Body="&lt;p&gt;There are many machine learning api for scanning images but they just return a bunch of tags. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://azure.microsoft.com/en-gb/services/cognitive-services/computer-vision/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://azure.microsoft.com/en-gb/services/cognitive-services/computer-vision/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{ &quot;tags&quot;: [ &quot;train&quot;, &quot;platform&quot;, &quot;station&quot;, &quot;building&quot;, &quot;indoor&quot;, &quot;subway&quot;, &quot;track&quot;, &quot;walking&quot;, &quot;waiting&quot;, &quot;pulling&quot;, &quot;board&quot;, &quot;people&quot;, &quot;man&quot;, &quot;luggage&quot;, &quot;standing&quot;, &quot;holding&quot;, &quot;large&quot;, &quot;woman&quot;, &quot;yellow&quot;, &quot;suitcase&quot; ],  &quot;confidence&quot;: 0.833099365 } ] }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Are there any apis for combining these into a sentence?&#xA;MS Cognitive Vision is the only one that produces a full caption&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;captions&quot;: [ { &quot;text&quot;: &quot;people waiting at a train station&quot;,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Google sentiment analysis can split a sentence into grammar parts but is there any api that does the reverse?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cloud.google.com/natural-language/docs/basics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/natural-language/docs/basics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;INPUT:&#xA;&quot;train&quot;, &quot;platform&quot;, &quot;station&quot;, &quot;building&quot;, &quot;indoor&quot;, &quot;subway&quot;, &quot;track&quot;, &quot;walking&quot;, &quot;waiting&quot;, &quot;pulling&quot;, &quot;board&quot;, &quot;people&quot;,&#xA; &quot;man&quot;, &quot;luggage&quot;, &quot;standing&quot;, &quot;holding&quot;, &quot;large&quot;, &quot;woman&quot;, &quot;yellow&quot;, &quot;suitcase&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OUTPUT:&#xA;&quot;people waiting at a train station&quot;&lt;/p&gt;&#xA;" OwnerUserId="14472" LastEditorUserId="4302" LastEditDate="2018-09-24T06:30:39.720" LastActivityDate="2018-09-24T06:30:39.720" Title="Commercial API Q: is there an api for converting vision tags into a caption?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;computer-vision&gt;&lt;software-evaluation&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5777" PostTypeId="2" ParentId="5763" CreationDate="2018-03-22T16:26:14.913" Score="1" Body="&lt;p&gt;Well, there're certainly a lot of areas where you can contribute in research. Since you're saying you did a Master Thesis in deep Generative models, I assume you're comfortable in Machine and Deep Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Digital Epidemiology is one of the areas where you can certainly apply deep learning. It's still a relatively new field compared to other branches of computational biology. An example would be to see the impact of online digital record on the prediction and further prevalence of diseases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such online record can be received from different search engines, social media sites, and sometimes Government agencies. For Example, you can see &lt;a href=&quot;https://trends.google.com/trends/explore?q=skin%20cancer&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; an example of search term &quot;Skin Cancer&quot; and the corresponding record shows the interest of this term across the Globe, this data can be used to find new Hypotheses. For example, if the data shows that we have more interest from a specific region of the world/country, that may show that the specific disease is more common in that region/part/country of the world. Similar hypotheses can be built, drawn and tested. And For sure,deep learning can improve the accuracy of traditional models used in validation of such Hypotheses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting area of research may be the comparison of Long Short Term Neural Networks against the traditional time series models. I don't believe there exists a mature research on this area. Maybe you can start from this good blog &lt;a href=&quot;https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Signal Processing maybe another very interesting, and also very practical area to build and validate theories on top of Deep Learning models. However, Mathematics in Signal Processing can be pretty hard to get. All of these options, however will require you to work in a team with people from the specific domains. That is if you want to produce high quality research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other areas may be NLP , especially the case of language translation from Hindi to Urdu or Persian, online digital marketing, behavioral sciences, manufacturing and investment. Specific areas of research maybe improved further if you know experts from these fields. &lt;/p&gt;&#xA;" OwnerUserId="14483" LastActivityDate="2018-03-22T16:26:14.913" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5778" PostTypeId="2" ParentId="5769" CreationDate="2018-03-22T19:41:57.567" Score="21" Body="&lt;p&gt;The following picture that you used in your question, very accurately describes what is happening. Remember that each element of the &lt;strong&gt;3D filter&lt;/strong&gt; (grey cube) is made up of a different value (&lt;code&gt;3x3x3=27&lt;/code&gt; values). So, three different &lt;strong&gt;2D filters&lt;/strong&gt; of size &lt;code&gt;3x3&lt;/code&gt; can be concatenated to form this one &lt;strong&gt;3D filter&lt;/strong&gt; of size &lt;code&gt;3x3x3&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YCJRm.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YCJRm.jpg&quot; alt=&quot;convnet2D&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;3x3x3&lt;/code&gt; RGB chunk from the picture is multiplied &lt;em&gt;elementwise&lt;/em&gt; by a &lt;strong&gt;3D filter&lt;/strong&gt; (shown as grey). In this case, the filter has &lt;code&gt;3x3x3=27&lt;/code&gt; weights. When these weights are multiplied element wise and then summed, it gives one value.&#xA;&lt;br/&gt;&lt;br/&gt;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;So, is there a separate filter for each input channel?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;YES&lt;/strong&gt;, there are as many &lt;strong&gt;2D filters&lt;/strong&gt; as number of input channels in the image. &lt;strong&gt;However&lt;/strong&gt;, it helps if you think that for input matrices with more than one channel, there is only one &lt;strong&gt;3D filter&lt;/strong&gt; (as shown in the image above).&#xA;&lt;br/&gt;&lt;br/&gt;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Then why is this called 2D convolution (if filter is 3D and input matrix is 3D)?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is 2D convolution because the strides of the filter is along the height and width dimensions only (&lt;strong&gt;NOT&lt;/strong&gt; depth) and therefore, the output produced by this convolution is also a 2D matrix. The number of movement directions of the filter determine the dimensions of convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; &lt;em&gt;If you build up your understanding by visualizing a single &lt;strong&gt;3D filter&lt;/strong&gt; instead of multiple &lt;strong&gt;2D filters&lt;/strong&gt; (one for each layer), then you will have an easy time understanding advanced CNN architectures like Resnet, InceptionV3, etc.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12957" LastActivityDate="2018-03-22T19:41:57.567" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5779" PostTypeId="2" ParentId="5769" CreationDate="2018-03-23T04:39:52.307" Score="0" Body="&lt;p&gt;There are only restriction in 2D. Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine a fully connected layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It'd be awfully huge, each neuron would be connected to maybe 1000x1000x3 inputs neurons. But we know that processing nearby pixel makes sense, therefore we limit ourselves to a small 2D-neighborhood, so each neuron is connected to only a 3x3 near neurons in 2D. We don't know such a thing about channels, so we connect to all channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, there would be too many weights. But because of the translation invariance, a filter working well in one area is most probably useful in a different area. So we use the same set of weights across 2D. Again, there's no such translation invariance between channels, so there's no such restriction there.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-03-23T04:39:52.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5780" PostTypeId="2" ParentId="5770" CreationDate="2018-03-23T10:42:28.200" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://developer.movidius.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel Movidius Stick&lt;/a&gt; is an USB stick for running a neural networks (that are programmed in Tensor Flow) on resource scare devices such as a Raspberry Pi (in order to reduce the traffic by sending data to cloud service or a server with more capacity where your neural network runs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensor Flow&lt;/a&gt; is a programming framework which can be used as python library for the implementation of neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are comparing an execution environment with an implementation language. The accuracy should not depend on them, instead I would considering on your model architecture (type of cnn) and your training as the most influential factors on accuracy of your test data.&lt;/p&gt;&#xA;" OwnerUserId="13295" LastEditorUserId="1671" LastEditDate="2018-03-23T20:11:10.387" LastActivityDate="2018-03-23T20:11:10.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5781" PostTypeId="1" CreationDate="2018-03-23T12:52:09.840" Score="1" ViewCount="70" Body="&lt;p&gt;I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;customers&lt;/li&gt;&#xA;&lt;li&gt;products (subscriptions)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The customers choose products and then stick with them until they decide on a different one. Examples could be phone, electricity or insurance contracts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every simulation I get the data about the customers (some classes and metadata) and then for each round I get signups/withdrawals and charges for the use of the service. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to learn a few things&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;competitiveness of an offering (in relation to the environment/competition)&lt;/li&gt;&#xA;&lt;li&gt;usage patterns of customers (the underlying model is a statistical simulation) depending on their chosen tariff, time of day and their metadata + historical usage&lt;/li&gt;&#xA;&lt;li&gt;ability to forecast customer numbers for each product&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The use cases are all very applicable to real world data although my case is all a (rather large) simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is this: What kind of learning is this? Supervised? Unsupervised? I have come up with various hypotheses and cannot find a definite answer for either.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pro Supervised: For the usage patterns of the customers I have historical data of actual usage so I can do something similar to time-series forecasting. However, I don’t want to forecast simply off of their previous usage but also off of their metadata and their tariff choice (so also metadata in a way). &lt;/li&gt;&#xA;&lt;li&gt;Pro Unsupervised: The forecasting of the “competitiveness” of a randomly chosen product configuration is hard to label even with historical data. The exact reason why a product has performed in a certain way is very high-dimensional. I do get subscription records about every product for every time slot though, so I guess some “feedback” could be generated. This might also be a RL problem though?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So obviously I need help pulling these different concepts apart so as to map them on this kind of problem which is not the classical “dog or cat” problem or the classical “historical data here, please forecast” timeseries issue. It’s also not a “learn how to walk” reinforcement problem as it’s based on historical data. The end goal is however to write an agent that generates these products and competes in the market so that will be a reinforcement problem.&lt;/p&gt;&#xA;" OwnerUserId="11429" LastActivityDate="2018-03-23T12:52:09.840" Title="Learning from events. Supervised, Unsupervised or MDP?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;classification&gt;&lt;unsupervised-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5782" PostTypeId="1" CreationDate="2018-03-23T16:38:40.500" Score="6" ViewCount="1437" Body="&lt;p&gt;In Section 1.1 of &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;, it is stated that a computer which passes the Turing Test would need 4 capabilities, and that these 4 capabilities comprise most of the field of Artificial Intelligence:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;natural language processing: to enable it to communicate successfully in English&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;knowledge representation: to store what it knows and hears&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;automated reasoning: to use the stored information to answer questions and to draw new conclusions&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;machine learning: to adapt to new circumstances and to detect and extrapolate patterns&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Did Alan Turing discern the requirements for the field of artificial intelligence (the necessary subfields) and purposefully design a test around these requirements, or did he simply design a test that is so general that the subfields which developed within artificial intelligence happen to be what is required to solve it? That is, was he prescient or lucky? Are these Turing's subdivisions, or Peter Norvig's and Stuart Russell's?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Turing did foresee these 4 requirements, what did he base them on? What principles of intelligence allowed him to predict the requirements for the field?&lt;/p&gt;&#xA;" OwnerUserId="2897" LastEditorUserId="2444" LastEditDate="2020-05-20T13:34:35.047" LastActivityDate="2020-05-20T13:34:35.047" Title="Did Turing foresee the required capabilities to pass the Turing test?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;knowledge-representation&gt;&lt;turing-test&gt;&lt;automated-reasoning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5783" PostTypeId="5" CreationDate="2018-03-23T20:01:48.397" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach (http://aima.cs.berkeley.edu/)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-23T20:01:48.397" LastActivityDate="2018-03-23T20:01:48.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5784" PostTypeId="4" CreationDate="2018-03-23T20:01:48.397" Score="0" Body="For references to &quot;Artificial Intelligence: A Modern Approach&quot; and the work of Russell &amp; Norvig in general." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-23T20:01:48.397" LastActivityDate="2018-03-23T20:01:48.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5785" PostTypeId="5" CreationDate="2018-03-23T20:22:17.567" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Aesthetics (/ɛsˈθɛtɪks, iːs-/; also spelled esthetics) is a branch of philosophy that explores the nature of art, beauty, and taste, with the creation and appreciation of beauty.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;SEE: &lt;a href=&quot;https://en.wikipedia.org/wiki/Aesthetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aesthetics (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-23T20:22:17.567" LastActivityDate="2018-03-23T20:22:17.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5786" PostTypeId="4" CreationDate="2018-03-23T20:22:17.567" Score="0" Body="For questions related to aesthetics in the context of algorithmic output.  Covers procedural generation for recreational or artistic purposes.  Not limited to visual mediums.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-23T20:22:17.567" LastActivityDate="2018-03-23T20:22:17.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5787" PostTypeId="5" CreationDate="2018-03-23T20:29:31.147" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_academic_fields#Computer_sciences&quot; rel=&quot;nofollow noreferrer&quot;&gt;List of academic fields (Computer Sciences)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-23T20:29:31.147" LastActivityDate="2018-03-23T20:29:31.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5788" PostTypeId="4" CreationDate="2018-03-23T20:29:31.147" Score="0" Body="For general questions about the field of AI.  Academic, industry, etc." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-03-23T20:29:31.147" LastActivityDate="2018-03-23T20:29:31.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5789" PostTypeId="2" ParentId="4694" CreationDate="2018-03-23T20:48:37.987" Score="1" Body="&lt;p&gt;First, I think it is important to mention that the Turing Test as is currently accepted is an updated version of Alan Turing's proposed imitation game, so your question is twofold. I don't think Turing makes this distinction that you propose. To Turing the question was &quot;Can machines think?&quot; and he explores this by way of the imitation game.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The new form of the problem can be described in terms of a game which we call the&#xA;  'imitation game.&quot; It is played with three people, a man (A), a woman (B), and an&#xA;  interrogator (C) who may be of either sex. The interrogator stays in a room apart front the&#xA;  other two. The object of the game for the interrogator is to determine which of the other&#xA;  two is the man and which is the woman. He knows them by labels X and Y, and at the&#xA;  end of the game he says either &quot;X is A and Y is B&quot; or &quot;X is B and Y is A.&quot; The&#xA;  interrogator is allowed to put questions to A and B&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It is A's object in the game to try and&#xA;  cause C to make the wrong identification&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The object of the game for the third player (B) is to help the interrogator.   &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is Turing's outline of the imitation game (shortened) between a man and a woman. The only difference between this game and the one we care about is that we replace party A with a machine and the only requirement on B is that they are human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, in Turing's imitation game the machine is privy to the information that it is not actually human. Whether the machine uses this information is up to the machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To talk about your statement on pretending to be a duck versus believing you are a duck lets examine the base imitation game with ducks and frogs. Let's say player A is a frog (either pretending to be a duck or truly believing that it is a duck)  and player B is a duck. Suppose for this example, the frogs and ducks are able to speak English and follow the rules of the game. The interrogator may ask &quot;what is the color of your beak?&quot; The duck will respond yellow, assuming its beak is yellow (I am no bird expert), and then our smart frog pretending to be a duck will look up the color of the beaks of ducks and respond yellow as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, our frog that believes he is a duck will think &quot;I am a duck so to discern the answer I must only look at my beak.&quot; And thus, our frog will say &quot;I do not have a beak, but my mouth is green.&quot; This will oust the frog.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-23T20:48:37.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5792" PostTypeId="1" AcceptedAnswerId="6516" CreationDate="2018-03-24T19:22:30.227" Score="2" ViewCount="84" Body="&lt;p&gt;I've just started learning Grammatical Evolution and I'm reading the paper &lt;a href=&quot;https://e098039d-a-62cb3a1a-s-sites.googlegroups.com/site/epscomputacionevolutiva/introduccion-a-la-programacion-automatica-evolutiva-evolucion-gramatical/grammatica_evolution_ieee_tec_2001.pdf?attachauth=ANoY7cp1rM9vbk-Cm-VUxPgX8XplxzM47859L5_Yj-F9tGdfKxfJFnqwcH2FZen6olXYuuAb0xBoKglZSuvGGcz7egrwOGeS1FixES3b1cpmhlKEm8cdgPp2P1umm6qcUfFJHwMvpybnfmSklvj0-3JAfaR1KY868iOln5iB3iUBjGP5EnBbnJsRdoXutpovtGhQxm8ZlToKmXXi7z7k95PRQGstABLy6KbDJNm8lWw_8tDxJsRsSa_Xav6N4jO0csNvhG8Vib45Oh8DvGKjurmyB-aS5D-BiRFuhX3Fdz_jHl5QIVXrHsH605WoIc10lhYxcJbCmNoYscl64cm5UKVHD3GcygrPntpJakrliiYb0GOb1WswY64%3D&amp;amp;attredirects=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grammatical Evolution&lt;/a&gt; by Michael O'Neill and Conor Ryan.&lt;/p&gt;&#xA;&lt;p&gt;On page 3 (section IV-A), they write:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;During the genotype-to-phenotype mapping process it is possible for individuals to run out of codons and in this case, we wrap the individual and reuse the codons.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm not an English native speaker and I don't understand the meaning of the word &amp;quot;wrap&amp;quot; here. What does it mean?&lt;/p&gt;&#xA;&lt;p&gt;I understood that, if none of the symbols are terminals, we have to start from the beginning of the genotype again and replace the nonterminal symbols until we have only terminal symbols. But, if I'm correct, when do I have to stop? In the paper, they also talk about non-valid individuals.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2021-01-21T14:56:00.487" LastActivityDate="2021-01-21T14:56:00.487" Title="What does &quot;we wrap the individual and reuse the codons&quot; mean in the paper &quot;Grammatical Evolution&quot; by Neill and Ryan?" Tags="&lt;terminology&gt;&lt;papers&gt;&lt;genetic-programming&gt;&lt;grammatical-evolution&gt;&lt;codon&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5794" PostTypeId="1" CreationDate="2018-03-24T20:12:28.483" Score="1" ViewCount="154" Body="&lt;p&gt;I have implemented multiple MCTS based AI players for the Love Letter game (&lt;a href=&quot;https://en.wikipedia.org/wiki/Love_Letter_(card_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;rules&lt;/a&gt;). It is a 2-4 players zero sum card game where players make alternating moves. I am struggling with how to properly conduct experiments for estimating AI player strength against human players: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In 2 player game where one of the players is AI bot&lt;/li&gt;&#xA;&lt;li&gt;In 4 player game where one (or multiple) of players is AI bot&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14534" LastActivityDate="2018-03-26T18:50:41.867" Title="How to estimate the AI player's strength in multiplayer game?" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5795" PostTypeId="2" ParentId="5782" CreationDate="2018-03-24T23:59:48.257" Score="5" Body="&lt;p&gt;I find it unlikely that you'll find a firm answer, so I will try my best to guide you towards information which may help you form an opinion either way. Turing had the controversial opinion (which remains controversial today) that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Digital computers have often been described as&#xA;  mechanical brains. Most scientists probably&#xA;  regard this description as a mere newspaper&#xA;  stunt, but some do not. One mathematician has&#xA;  expressed the opposite point of view to me rather&#xA;  forcefully in the words “It is commonly said that&#xA;  these machines are not brains, but you and I&#xA;  know that they are.” […] I shall give most attention&#xA;  to the view which I hold myself, that it is&#xA;  not altogether unreasonable to describe digital&#xA;  computers as brains.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In essence Turing believed that the digital computers of the time had the capacity to mimic the human brain if programmed correctly. The problem for Turing then, was finding the correct procedures to mimic the brain. Remember the Imitation Game was being used as a means to explore the question &quot;can machines think?&quot; Turing was engaging in an intellectual debate about the nature of &quot;thinking&quot;, he was not making an attempt to describe how machines would be taught to think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, both artificial intelligence and machine learning are at their core very similar to &quot;artificial thinking&quot; (as Turing may have called it) it seems necessary that a test designed to decide whether machines can think would also encompass whether machines could be intelligent or whether machines could learn. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it seems possible that Turing did not make the question with these capabilities in mind, but rather that these capabilities are what guided Turing to this question as they are so innately tied to thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to look further into Turing's thought process &lt;a href=&quot;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&quot; rel=&quot;noreferrer&quot;&gt;here is his paper&lt;/a&gt; on the imitation game and &lt;a href=&quot;https://pdfs.semanticscholar.org/e065/7d8b688627676f2bd14ca8cd8cd14540a9aa.pdf&quot; rel=&quot;noreferrer&quot;&gt;here is a discussion&lt;/a&gt; on the proceedings of a BBC broadcast about &quot;Automatic Calculating Machines&quot; in which Turing was a speaker.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-24T23:59:48.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5796" PostTypeId="2" ParentId="5794" CreationDate="2018-03-25T09:13:30.940" Score="2" Body="&lt;p&gt;The following are extremely simple ways of tackling this problem.&lt;/p&gt;&#xA;&lt;h2&gt;A very simple way&lt;/h2&gt;&#xA;&lt;p&gt;It can simply be&lt;br/&gt;&#xA;&lt;code&gt;strength of AI&lt;/code&gt;=&lt;code&gt;(# of games won)&lt;/code&gt;/&lt;code&gt;(total # of games)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;br/&gt;&#xA;&lt;h2&gt;In case data for each move is available&lt;/h2&gt;&#xA;&lt;p&gt;Something like&lt;br/&gt;&#xA;&lt;code&gt;score per game&lt;/code&gt;=&lt;code&gt;# of correct decisions&lt;/code&gt;/&lt;code&gt;total number of decisions&lt;/code&gt;.&lt;br/&gt;&#xA;Then&lt;br/&gt;&#xA;&lt;code&gt;strength of AI&lt;/code&gt;=&lt;code&gt;sum(score per game)&lt;/code&gt;/&lt;code&gt;total # of games&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;br/&gt;&#xA;&lt;h2&gt;If each move/decision has a score associated with it&lt;/h2&gt;&#xA;&lt;p&gt;then you do&lt;br/&gt;&lt;code&gt;score per move&lt;/code&gt;=&lt;code&gt;scored points by taking a decision&lt;/code&gt;/&lt;code&gt;maximum possible score&lt;/code&gt;.&lt;br/&gt;&#xA;then&lt;br/&gt;&#xA;&lt;code&gt;score per game&lt;/code&gt;=&lt;code&gt;sum(score per move)&lt;/code&gt;/&lt;code&gt;total # of moves&lt;/code&gt;&lt;br/&gt;&#xA;and finally,&lt;br/&gt;&#xA;&lt;code&gt;strength of AI&lt;/code&gt;=&lt;code&gt;sum(score per game)&lt;/code&gt;/&lt;code&gt;total # of games&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;br/&gt;&#xA;&lt;h2&gt;How to choose optimal number of games to play?&lt;/h2&gt;&#xA;&lt;p&gt;It depends on your requirements. If you want to report your AI's strength in percentage of the games it played correct to 1 decimal place (for example, this AI won 95.1%) then 10000 is an optimal number of games your AI needs to play. Suppose your AI won 9508 games out of 10000 then you will have 95.08% strength of AI. To be able to correctly round it to 1 decimal place you need to have an additional decimal place so that you can quote the strength of you AI with reasonable confidence, in this case 95.1%.&lt;/p&gt;&#xA;" OwnerUserId="12957" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-03-26T05:54:52.317" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5797" PostTypeId="1" CreationDate="2018-03-25T09:33:07.463" Score="2" ViewCount="190" Body="&lt;p&gt;Basically, I want a system that reads blobs of text, parses it and stores related chunks and when the user asks a question, it returns with an in-context answer which groups together a bunch of facts. &lt;em&gt;The topic and the initial blob set is hardcoded.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am noober than a newbie and I don't understand how the chunks would form relationships amongst themselves by logic and not by rule when the inference engine would run and also, how they would be stored together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if I am trying to reinvent the wheel, I'd be grateful if you could point me to open-source systems that already exist which just take in data and produce answers to questions after having performed some logical operations on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14549" LastActivityDate="2018-04-10T13:57:12.690" Title="Logic for a knowledge based expert system" Tags="&lt;knowledge-representation&gt;&lt;expert-system&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2021-01-18T04:17:12.257" ContentLicense="CC BY-SA 3.0" />
  <row Id="5798" PostTypeId="2" ParentId="5067" CreationDate="2018-03-25T12:27:21.730" Score="1" Body="&lt;p&gt;With images, you can use CNN because of the translational invariance. A filter which is good in one area will probably be good in another area, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With images, you must use CNN because otherwise, there would be too many weights to train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your game, it depends on the representation and the exact rules. Note that Alpha Zero uses a set of 19 x 19 inputs with CNN for playing Go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a game like Bridge, where each card has its color and rank, there's a kind of translational invariance. Having Ace and Queen is a bit similar to having King and Jack - in both cases you have a 50% chance of catching the card in between. At the same time, the strengths of AQ and KJ are very different, so a pure CNN would improbably work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more important symmetry is the one among colors. After the auction, there's one or none trump color and all other colors are equivalent. This probably means that the corresponding weights should be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some card games, many cards are special and there's no symmetry at all. You didn't tell us anything about your game, so it's hard to give a more concrete advice.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-03-25T12:27:21.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5800" PostTypeId="1" AcceptedAnswerId="5820" CreationDate="2018-03-26T08:57:31.797" Score="1" ViewCount="47" Body="&lt;p&gt;I have started reading &lt;a href=&quot;https://books.google.com/books/about/Fundamentals_of_Deep_Learning.html?id=SL0BvgAACAAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fundamentals of Deep Learning by Nikhil Buduma&lt;/a&gt; and I have a question regarding tanh neurons. In the book, it is stated:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;When S-shaped nonlinearities are used, the tanh neuron is often preferred over the sigmoid neuron because it is zero-centered.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain me why exactly??&lt;/p&gt;&#xA;" OwnerUserId="14568" LastEditorUserId="1671" LastEditDate="2018-03-26T21:53:08.177" LastActivityDate="2018-03-28T10:25:21.237" Title="S-shaped nonlinearities in tanh neurons" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5801" PostTypeId="1" AcceptedAnswerId="5813" CreationDate="2018-03-26T09:29:22.130" Score="3" ViewCount="155" Body="&lt;p&gt;Suppose one trains a CNN to determine if something was either a cat/dog or neither (2 classes), would it be a good idea to assign all cats and dogs to one class and everything else to another? Or would it be better to have a class for cats, a class for dogs, and a class for everything else (3 classes)? My colleague argues for 3 classes because dogs and cats have different features, but I wonder if he's right.&lt;/p&gt;&#xA;" OwnerUserId="13068" LastEditorUserId="2444" LastEditDate="2020-09-25T19:43:43.867" LastActivityDate="2020-09-26T06:52:24.870" Title="If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;datasets&gt;&lt;binary-classification&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5803" PostTypeId="2" ParentId="248" CreationDate="2018-03-26T13:17:52.717" Score="5" Body="&lt;p&gt;To summarize, There are two major issues in applied Deep Learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The first being that computationally , it's exhaustive. Normal CPU's require a lot of time to perform even the basic computation/training with Deep Learning. GPU's are thus recommended however, even they may not be enough in a lot of situations. Typical deep learning models don't support the theoretical time to be in Polynomials. However, if we look at the relatively simpler models in ML for the same tasks, too often we have mathematical guarantees that training time required for such simpler Algorithms is in Polynomials. This, for me, at least is probably the biggest difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There're solutions to counter this issue, though. One main approach being is to optimize DL Algorithms to a number of iterations only (instead of looking at the global solutions in practice, just optimize the algorithm to a good local solution, whereas the criterion for &quot;Good&quot; is defined by the user).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another Issue which may be a little bit controversial to young deep learning enthusiasts is that Deep Learning algorithms lack theoretical understanding and reasoning. Deep Neural Networks have been successfully used in a lot of situations including Hand writing recognition, Image processing, Self Driving Cars, Signal Processing, NLP and Biomedical Analysis. In some of these cases, they have even surpassed humans. However, that being said, they're not under any circumstance, theoretically as sound as most of Statistical Methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will not go into detail , rather I leave that up to you. There're pros and cons for every Algorithm/methodology and DL is not an exception. It's very useful as has been proven in a lot of situations and every young Data Scientist must learn at least the basics of DL. However, in the case of relatively simple problems, it's better to use famous Statistical methods as they have a lot of theoretical results/guarantees to support them. Furthermore, from learning point of view, it's always better to start with simple approaches and master them first. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14483" LastActivityDate="2018-03-26T13:17:52.717" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5804" PostTypeId="2" ParentId="5801" CreationDate="2018-03-26T14:34:52.437" Score="2" Body="&lt;p&gt;If you want to determine if something is either a&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;cat/dog or neither&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;you need 2 classes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;one for dog or cat, and&lt;/li&gt;&#xA;&lt;li&gt;one for anything else.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, if you assign all cats and dogs to the same class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, if an input is classified as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, then you won't be able to know whether it is a dog or a cat, you will just know that it is &lt;em&gt;either&lt;/em&gt; a dog or a cat.&lt;/p&gt;&#xA;&lt;p&gt;In case you wanted to distinguish between cats and dogs too (apart from &lt;em&gt;neither&lt;/em&gt; of them), then you'll need &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; classes.&lt;/p&gt;&#xA;&lt;p&gt;Finally, if you specify only 2 classes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;dog, and&lt;/li&gt;&#xA;&lt;li&gt;cat,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;then your CNN will try to classify any new input as either a dog or a cat, even though it is neither a dog nor a cat (e.g. maybe it is a horse).&lt;/p&gt;&#xA;" OwnerUserId="11069" LastEditorUserId="2444" LastEditDate="2020-09-25T19:56:56.690" LastActivityDate="2020-09-25T19:56:56.690" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5807" PostTypeId="2" ParentId="4396" CreationDate="2018-03-27T00:43:30.033" Score="6" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$D_a$&lt;/span&gt; be the domain for A, and &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; the elements of &lt;span class=&quot;math-container&quot;&gt;$D_a$&lt;/span&gt;.  Let &lt;span class=&quot;math-container&quot;&gt;$D_b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D_c$&lt;/span&gt; work similarly for &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We introduce &lt;span class=&quot;math-container&quot;&gt;$D_t = \{t | t = (c_i - b_j, c_i)\}$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$D_c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$D_b$&lt;/span&gt;.  We can see that &lt;span class=&quot;math-container&quot;&gt;$\{t[0]\}$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$\{c_i - b_j, \forall i,j\}$&lt;/span&gt;) must be equal to &lt;span class=&quot;math-container&quot;&gt;$D_a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can represent the constraint on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; by the relation &lt;span class=&quot;math-container&quot;&gt;$R_At = { (a_k, t_l) | a_k = t_l[0]}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$a_k$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$D_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_l$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$D_t$&lt;/span&gt;.  That is, &lt;span class=&quot;math-container&quot;&gt;$a_k$&lt;/span&gt; must equal the first element in the pair from &lt;span class=&quot;math-container&quot;&gt;$t_l$&lt;/span&gt;, which means there is a &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt; and a &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; consistent with that &lt;span class=&quot;math-container&quot;&gt;$a_k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraint on &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is the relation &lt;span class=&quot;math-container&quot;&gt;$R_{Bt} = \{(b_j, t_l) | b_j + t_l[0] = t_l[1]\}$&lt;/span&gt;.  Since &lt;span class=&quot;math-container&quot;&gt;$t_l$&lt;/span&gt;[0] = &lt;span class=&quot;math-container&quot;&gt;$c_i - b_j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$t_l$&lt;/span&gt;[0] + &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt; must equal &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt;, which is t[1], so if this holds, there is a &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$D_b$&lt;/span&gt; that is consistent with &lt;span class=&quot;math-container&quot;&gt;$t_l$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, &lt;span class=&quot;math-container&quot;&gt;$R_{Ct} = \{ (c_i, t_l) | c_i = t_l[1]\}$&lt;/span&gt;, which is really just an identity.  So what we've done is solved &lt;span class=&quot;math-container&quot;&gt;$A + B = C$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and encoded that into &lt;span class=&quot;math-container&quot;&gt;$R_{At}$&lt;/span&gt;.  From that solution and knowledge of the original &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; (stored in t[1]) we can re-create &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; from a valid &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, which is the &lt;span class=&quot;math-container&quot;&gt;$R_{Bt}$&lt;/span&gt; relation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And lastly &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; must be in the original set of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;'s used to build all the &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;'s.  If all three of these constraints hold, then we must have a &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, a &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; used to solve for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, and an &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that matches that solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part is really just path-consistency.  For every &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, choose a &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and see if there's a possible &lt;span class=&quot;math-container&quot;&gt;$C = A + B$&lt;/span&gt;.  If so, Add that those values to final domains &lt;span class=&quot;math-container&quot;&gt;$D_a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D_b$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$D_c$&lt;/span&gt;.  I forget if there's a better performing algorithm than this, but I doubt it since here you can use associativity of addition to avoid checking the opposite order of selecting &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;.  In general, you have to consider both orders.&lt;/p&gt;&#xA;" OwnerUserId="14580" LastEditorUserId="14580" LastEditDate="2018-10-09T01:51:23.317" LastActivityDate="2018-10-09T01:51:23.317" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5808" PostTypeId="5" CreationDate="2018-03-27T09:08:37.670" Score="0" Body="&lt;p&gt;Artificial intelligence (AI) in healthcare uses algorithms and software to approximate human cognition in the analysis of complex medical data. The primary aim of health-related AI applications is to analyze relationships between prevention or treatment techniques and patient outcomes. AI programs have been developed and applied to practices such as diagnosis processes, treatment protocol development, drug development, personalized medicine and patient monitoring and care, among others.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-03-27T21:38:58.573" LastActivityDate="2018-03-27T21:38:58.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5809" PostTypeId="4" CreationDate="2018-03-27T09:08:37.670" Score="0" Body="For questions about the application of Artificially Intelligent agents and Machine Learning Algorithms in the healthcare industry.  " OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-03-27T21:38:51.900" LastActivityDate="2018-03-27T21:38:51.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5810" PostTypeId="1" CreationDate="2018-03-27T13:19:29.650" Score="2" ViewCount="1080" Body="&lt;p&gt;I'm currently having troubles to win against a random bot playing the Schieber Jass game. It is a imperfect card information game. (famous in switzerland &lt;a href=&quot;https://www.schieber.ch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.schieber.ch/&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environement I'm using is on Github &lt;a href=&quot;https://github.com/murthy10/pyschieber&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/murthy10/pyschieber&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a brief overview of the Schieber Jass I will describe the main characteristics of the game.&#xA;The Schieber Jass consists of four players building two teams.&#xA;At the beginning every player gets randomly nine cards (there are 36 cards).&#xA;Now there are nine rounds and every player has to chose one card every round. Related to the rules of the game the &quot;highest card&quot; wins and the team gets the points.&#xA;Hence the goal is to get more points then your opponent team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several more rules but I think you can image how the game should roughly work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm trying to apply a DQN approach at the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my attempts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I let two independent reinforcement player play against two random players&lt;/li&gt;&#xA;&lt;li&gt;I design the input state as a vector (one hot encoded) with 36 &quot;bits&quot; for every player and repeated this nine times for every card you can play during a game.&lt;/li&gt;&#xA;&lt;li&gt;The output is a vector of 36 &quot;bits&quot; for every possible card.&lt;/li&gt;&#xA;&lt;li&gt;If the greedy output of the network suggest an invalid action I take the action with the highest probability of the allowed actions&lt;/li&gt;&#xA;&lt;li&gt;The reward is +1 for winning, -1 for losing, -0.1 for a invalid action and 0 for an action which doesn't lead to a terminal state&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would it be helpful to use a LSTM and reduce the input state?&lt;/li&gt;&#xA;&lt;li&gt;How to handle invalid moves?&lt;/li&gt;&#xA;&lt;li&gt;Do you have some good ideas for improvements? (like Neural-Fictitious Self-Play or something similar)&lt;/li&gt;&#xA;&lt;li&gt;Or is this the whole approach absolute nonsense?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14587" LastEditorUserId="14587" LastEditDate="2018-06-18T14:32:08.663" LastActivityDate="2018-06-18T14:32:08.663" Title="How to use DQN to handle an imperfect but complete information game?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;q-learning&gt;&lt;imperfect-information&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5813" PostTypeId="2" ParentId="5801" CreationDate="2018-03-27T18:49:02.383" Score="1" Body="&lt;p&gt;The best approach may be to have a cat, dog, and neither class (3 classes total) and go with a regression approach — specifically, outputting the probabilities of each class for any given input. From there, you can always take the probabilities of each output and derive the probability of a cat and dog class or neither class. Also, make sure you use the right activation on the output layer and cost function so that you can interpret the outputs as probabilities (e.g. softmax activation and cross-entropy loss).&lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="5210" LastEditDate="2020-09-26T06:52:24.870" LastActivityDate="2020-09-26T06:52:24.870" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5814" PostTypeId="1" CreationDate="2018-03-27T19:14:04.540" Score="4" ViewCount="861" Body="&lt;p&gt;Suppose a CNN is trained to detect bounding box of a certain type of object (people, cars, houses, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If each image in the training set contains just one object (and its corresponding bounding box), how well can a CNN generalize to pick up all objects if the input for prediction contains multiple objects? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should the training images be downsampled in order for the CNN to pick out multiple objects in the prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have a specific one in mind. I was just curious about the general behavior.&lt;/p&gt;&#xA;" OwnerUserId="14570" LastEditorUserId="2444" LastEditDate="2020-06-06T00:49:09.597" LastActivityDate="2020-06-06T05:47:31.500" Title="How well can CNN for bounding box detection generalise?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5815" PostTypeId="2" ParentId="2940" CreationDate="2018-03-27T22:14:51.143" Score="0" Body="&lt;p&gt;To answer the title, there are many other machine learning models, but neural networks work particularly well for some difficult problems (image classification, speech recognition) which is one of the reasons they have gained popularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two particularly simple models are the decision tree and the perceptron. These are rather simple models, but they both have redeemable qualities. A decision tree is useful as it provides a model that is easily understood, while a perceptron is fairly quick and works well for linearly separable data. Another, more advanced, model is the Support Vector Machine.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, is there any system in which the topology of a neural network is variable? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, there are many such systems where the topology of the neural network is dynamic throughout training. An entire class of methods labeled TWEANNs are designed to evolve the topology of the networks, one such algorithm is NeuroEvolution of Augmenting Topologies, NEAT (and it's descendants rtNEAT, hyperNEAT, ...).&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-03-27T22:14:51.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5817" PostTypeId="2" ParentId="2940" CreationDate="2018-03-28T09:33:45.573" Score="2" Body="&lt;h2&gt;Neural Network equivalents that is not (vanilla) feed forward Neural Nets:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Neural net structures such as Recurrent Neural Nets (RNNs) and Convolutional Neural Nets (CNNs), and different architectures within those are good examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of different architectures within RNNs would would be: Long Short Term Memory (LSTM) or Gated Recurrent Unit (GRU). Both of these are well described in Colah's blog post on &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding LSTMs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What are some alternative information processing system beside neural network&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are sooo many structures. From the top of my head: (Restricted) Boltzmann machine, auto encoders, monte carlo method and radial basis networks to name a &lt;em&gt;few&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check out Goodfellow's &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep learning&lt;/a&gt;-book that is free online and get the gist of all the structures I mentioned here (most parts requires a bit of math knowledge, but he also writes about them quite intuitively).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Recurrent Neural Nets I recommend Colah's blog post on &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding LSTMs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Is there any system in which the topology of a neural network is variable?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Depends on what you mean with the &lt;em&gt;topology&lt;/em&gt; of a neural network:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think in the common meaning of topology when talking about Neural Networks is the way in which neurons are connected to form a network, varying in structure as it runs and learns. If this is what you men then the answer, in short, is yes. In multiple ways actually. On the other hand, if you mean in the mathematical sense, this answer would become a book that I wouldn't feel confortable writing. So I'll assume you mean the first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We often do &quot;regularization&quot;, both on vanilla NN and other structures. One of these regularization techniques are called &lt;a href=&quot;https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dropout&lt;/a&gt;, which would randomly remove connections from the network as it is training (to prevent something called &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt;, which I'm not gonna go into in this post).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example for another way would be on the Recurrent Neural Network. They deal with time series, and are equipped for dealing with timeseries of different lengths (thus, &quot;varying structure&quot;).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Does it exist neural net systems where complex numbers are used?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Yes, there are many papers on complex number machine learning structures. A quick google should give you loads of results. For example: DeepMind has a paper on &lt;a href=&quot;https://arxiv.org/pdf/1602.03032.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&#xA;Associative Long Short-Term Memory&lt;/a&gt; which explores the use of complex values for an &quot;associative memory&quot;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Links:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Goodfellow's Deep Learning-book: &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeplearningbook.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Colah's blogpost on RNN's: &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;colah.github.io&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paper on DeepMinds Associative LSTM: &lt;a href=&quot;https://arxiv.org/pdf/1602.03032.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv:1602.03032&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2018-05-25T19:28:56.193" LastActivityDate="2018-05-25T19:28:56.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5818" PostTypeId="1" CreationDate="2018-03-28T09:36:33.223" Score="1" ViewCount="116" Body="&lt;p&gt;I am new to deep learning and computer vision. I have a problem where I use &lt;a href=&quot;https://pjreddie.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the YOLO&lt;/a&gt; to detect objects.&lt;/p&gt;&#xA;&lt;p&gt;For my problem, I just want to recognize 1 human only. So, I changed the final YOLO's layer (which contained 80 neurons) to only 1 neuron, and do the training process with transfer learning techniques. Of course, I do not use the final layer's weights, and these weights are randomly initialized for my problem. I feed only the human data to the model.&lt;/p&gt;&#xA;&lt;p&gt;However, I realize that after longer training, the model becomes worse. It starts to recognize other objects as a human.&lt;/p&gt;&#xA;&lt;p&gt;Should I also feed non-human data to the model?&lt;/p&gt;&#xA;" OwnerUserId="14613" LastEditorUserId="2444" LastEditDate="2021-01-17T19:42:23.563" LastActivityDate="2021-01-17T19:42:23.563" Title="Why is my fine-tuned YOLO model detecting other objects as a human?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;transfer-learning&gt;&lt;yolo&gt;&lt;binary-classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5819" PostTypeId="2" ParentId="5818" CreationDate="2018-03-28T10:04:24.317" Score="1" Body="&lt;p&gt;So you have a network pretrained on 80 classes. I also assume that one of these classes are human (or else this is just not the way to go*) I suspect that the final layer contains 80 labels, correct? Then you then 'rescale' this layer to 1 label and then train on some data you possess? Then you're basically trying to teach the network that it shouldn't care about the 79 other classes, which is just nonsense I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you &lt;em&gt;could&lt;/em&gt; do, and I do not recommend this, but if you feel like you &lt;em&gt;have&lt;/em&gt; to use this &lt;em&gt;exact&lt;/em&gt; network, you just keep the 80 outputs and only look at the label correspondning to the human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You shouldn't do this because the network is WAY bigger than it needs to be to only classify human/non human, which will make it slower than it needs to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you rather want to do is either to train your own network (If you have lots of training data, I suspect this wouldn't be be the hardest thing to train) or obtain a CNN that is pretrained on human classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(*I've heard rumours that you can do pretty well on retraining a class on a pretrained network. I just don't know if the rumours are true or how to go about it.)&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14664" LastEditDate="2018-03-31T09:24:56.200" LastActivityDate="2018-03-31T09:24:56.200" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5820" PostTypeId="2" ParentId="5800" CreationDate="2018-03-28T10:18:51.690" Score="1" Body="&lt;p&gt;It should be mentioned that RELU is the current activation function standard. But to answer your question:&#xA;The importance here is that it is very common to have normalized your data (e.g. using &lt;a href=&quot;http://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;batch normalization&lt;/a&gt;), then the data is centered around 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @DuttaA commented, look at &lt;a href=&quot;https://stats.stackexchange.com/questions/101560/tanh-activation-function-vs-sigmoid-activation-function&quot;&gt;this answer&lt;/a&gt; from Cross-Validated: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since data is centered around 0, the derivatives are higher. To see this, calculate the derivative of the tanh function and notice that [output] values are in the range [0,1]. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The range of the tanh function is [-1,1] and that of the sigmoid function is [0,1] Avoiding bias in the gradients. This is explained very well in &lt;a href=&quot;http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;, and it is worth reading it to understand these issues.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2018-03-28T10:25:21.237" LastActivityDate="2018-03-28T10:25:21.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5821" PostTypeId="1" CreationDate="2018-03-28T14:56:34.707" Score="4" ViewCount="42" Body="&lt;p&gt;I need to efficiently align characters vertically using Multi Objective PSO. Alignment is achieved by adding spaces in between a given set of characters.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a b c d e f&#xA;b b d h g&#xA;c a b f&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Might be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;- a b - c d e f - -&#xA;- - b b - d - - h g&#xA;c a b - - - - f - -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now this is a multi objective solution.&#xA;I need to maximize the characters that get aligned vertically and minimize the amount of spaces in between the characters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to focus firstly on how to get a set of characters to represent a position of a particle. This would mean that I need to somehow transform a possible set of characters into a position of a particle. If I can somehow achieve this then the rest should fall into place.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I transform these set of characters into a position of a particle? &lt;/li&gt;&#xA;&lt;li&gt;Also is this the best approach or are there better ways to approach this problem?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14621" LastEditorUserId="1671" LastEditDate="2018-03-28T21:15:30.273" LastActivityDate="2021-04-24T18:04:15.470" Title="How to use MOPSO to align characters vertically?" Tags="&lt;algorithm&gt;&lt;optimization&gt;&lt;combinatorics&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5823" PostTypeId="5" CreationDate="2018-03-28T18:41:04.927" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Open-source software (OSS) is computer software with its source code made available with a license in which the copyright holder provides the rights to study, change, and distribute the software to anyone and for any purpose. Open-source software may be developed in a collaborative public manner. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For more details, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Open-source_software&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Open-source_software&lt;/a&gt; and &lt;a href=&quot;https://opensource.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://opensource.org/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-06-29T22:27:31.003" LastActivityDate="2019-06-29T22:27:31.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5824" PostTypeId="4" CreationDate="2018-03-28T18:41:04.927" Score="0" Body="For questions related to open source AI software." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-06-29T22:27:31.003" LastActivityDate="2019-06-29T22:27:31.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5825" PostTypeId="1" AcceptedAnswerId="6093" CreationDate="2018-03-28T21:32:52.760" Score="1" ViewCount="73" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ko4R5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ko4R5.png&quot; alt=&quot;ELU with α=5&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An exponential linear unit (as proposed by Clevert et al.) uses the function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Sorry--would have used MathJax were it available.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ELU&lt;sub&gt;α&lt;/sub&gt;(x) = α(e&lt;sup&gt;x&lt;/sup&gt; - 1) (if x&amp;lt;0), or x (if x≥0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, this is continuous at x=0, which is great. It's differentiable there too if α=1, which is the value that the paper used to test ELU units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if α≠1 (as in the above diagram), then it's no longer differentiable at x=0. It has a crook in it, which seems weird to me. Having your function be differentiable at all points seems advantageous. Further, it seems that if you just make the linear portion evaluate to αx rather than x, that it &lt;em&gt;would&lt;/em&gt; be differentiable there. Is there a reason that the function wasn't defined to do this? Or did they not bother, because α=1 is definitely the hyperparameter to use?&lt;/p&gt;&#xA;" OwnerUserId="14628" LastEditorUserId="9647" LastEditDate="2018-04-16T21:05:50.683" LastActivityDate="2018-04-17T18:02:50.993" Title="Why don't ELUs multiply the linear portion by α?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5827" PostTypeId="5" CreationDate="2018-03-29T03:25:22.887" Score="0" Body="&lt;p&gt;Java is a general-purpose computer-programming language that is concurrent, class-based, object-oriented, and specifically designed to have as few implementation dependencies as possible. It is intended to let application developers &quot;write once, run anywhere&quot; (WORA), meaning that compiled Java code can run on all platforms that support Java without the need for recompilation. Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of computer architecture. As of 2016, Java is one of the most popular programming languages in use, particularly for client-server web applications, with a reported 9 million developers. Java was originally developed by James Gosling at Sun Microsystems (which has since been acquired by Oracle Corporation) and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++, but it has fewer low-level facilities than either of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://letzgro.net/blog/building-ai-with-java/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building AI with Java - Letzgro&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-02T17:00:16.470" LastActivityDate="2018-04-02T17:00:16.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5828" PostTypeId="4" CreationDate="2018-03-29T03:25:22.887" Score="0" Body="For questions about implementation of Machine Learning and Artificial Intelligence algorithms in the Java programming language." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-02T16:57:56.050" LastActivityDate="2018-04-02T16:57:56.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5829" PostTypeId="5" CreationDate="2018-03-29T03:29:42.357" Score="0" Body="&lt;p&gt;C++ is a general-purpose programming language. It has imperative, object-oriented and generic programming features, while also providing facilities for low-level memory manipulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was designed with a bias toward system programming and embedded, resource-constrained and large systems, with performance, efficiency and flexibility of use as its design highlights. C++ has also been found useful in many other contexts, with key strengths being software infrastructure and resource-constrained applications, including desktop applications, servers (e.g. e-commerce, web search or SQL servers), and performance-critical applications (e.g. telephone switches or space probes). C++ is a compiled language, with implementations of it available on many platforms. Many vendors provide C++ compilers, including the Free Software Foundation, Microsoft, Intel, and IBM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/C%2B%2B&quot; rel=&quot;nofollow noreferrer&quot;&gt;C++ - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-02T17:00:23.683" LastActivityDate="2018-04-02T17:00:23.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5830" PostTypeId="4" CreationDate="2018-03-29T03:29:42.357" Score="0" Body="For questions about implementation of Machine Learning and Artificial Intelligence algorithms in the C++ programming language." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-02T17:05:48.797" LastActivityDate="2018-04-02T17:05:48.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5831" PostTypeId="5" CreationDate="2018-03-29T04:02:23.227" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Softmax regression is a generalized form of logistic regression which can be used in multi-class classification problems where the classes are mutually exclusive. The hand-written digit dataset used in this tutorial is a perfect example. A softmax regression classifier trained on the hand written digits will output a separate probability for each of the ten digits, and the probabilities will all add up to 1.&lt;br&gt;&#xA;  &lt;sub&gt;SOURCE: &lt;a href=&quot;http://mccormickml.com/2014/06/13/deep-learning-tutorial-softmax-regression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Softmax Regression - Tutorial (McCormick)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;SEE ALSO:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Softmax Regression - Tutorial (Stanford University)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Softmax function (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-04-02T17:05:44.633" LastActivityDate="2018-04-02T17:05:44.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5832" PostTypeId="4" CreationDate="2018-03-29T04:02:23.227" Score="0" Body="For questions about implementation/mathematical details of the softmax machine learning algorithm." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-02T17:07:10.033" LastActivityDate="2018-04-02T17:07:10.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5835" PostTypeId="1" CreationDate="2018-03-29T09:57:12.370" Score="6" ViewCount="1891" Body="&lt;p&gt;The Q function uses the (current and future) states to determine the action that gets the highest reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in a stochastic environment, the current action (at the current state) does not determine the next state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does Q learning handle this? Is the Q function only used during the training process, where the future states are known? And is the Q function still used afterwards, if that is the case?&lt;/p&gt;&#xA;" OwnerUserId="14638" LastEditorUserId="2444" LastEditDate="2019-02-22T17:24:31.600" LastActivityDate="2019-07-14T10:52:00.907" Title="How does Q-learning work in stochastic environments?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5836" PostTypeId="1" CreationDate="2018-03-29T19:52:37.683" Score="1" ViewCount="1206" Body="&lt;p&gt;What is the output value of the network for these inputs respectively, and why?&#xA;(Linear activation function is fine.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2, 3][-1, 2][1, 0][3, 4]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is how you take the 'backwards' directed paths into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H1xj8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H1xj8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14650" LastActivityDate="2018-06-28T22:17:41.510" Title="How to calculate the output of this neural network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5837" PostTypeId="1" CreationDate="2018-03-30T12:28:12.523" Score="2" ViewCount="256" Body="&lt;p&gt;Let us suppose I have a &lt;code&gt;NxN&lt;/code&gt; matrix and I want to classify in &lt;code&gt;M&lt;/code&gt; classes each entry of the matrix using a fuzzy classifier. The output of my classifier will be, for each matrix entry, an &lt;code&gt;M&lt;/code&gt;-dimensional vector containing the probabilities for the entry to be classified in each class. &#xA;A naive way to build a confusion matrix would be to select the highest probability in each vector and use it as a crips classification. However, I would like to take into account all the probabilities associated to each entry and compute a &quot;fuzzy&quot; confusion matrix. Is this possible?&lt;/p&gt;&#xA;" OwnerUserId="14661" LastActivityDate="2018-03-30T12:28:12.523" Title="Fuzzy confusion matrix for fuzzy classifier" Tags="&lt;classification&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5838" PostTypeId="1" AcceptedAnswerId="7104" CreationDate="2018-03-30T13:12:22.373" Score="3" ViewCount="412" Body="&lt;p&gt;How can I train a neural network to recognize sub-sequences in a sequence flow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: Given the sequence &lt;strong&gt;111100002222&lt;/strong&gt; as an input sample from a stream, the neural network would recognize that &lt;strong&gt;1111&lt;/strong&gt; , &lt;strong&gt;0000&lt;/strong&gt; , &lt;strong&gt;2222&lt;/strong&gt; are sub sequences (so &lt;strong&gt;111100&lt;/strong&gt; would not be a valid subsequence) and so on for ~ 50 to 100 different subsequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no particular order in which the subsequence would appear in the flow.&#xA;No network architecture restriction.&#xA;Subsequences are of variable length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General concepts, ideas, and theory are welcome. &lt;/p&gt;&#xA;" OwnerUserId="13038" LastEditorUserId="5210" LastEditDate="2018-04-02T16:57:41.827" LastActivityDate="2018-07-10T13:38:29.800" Title="Ideas on how to make a neural net learn how to split sequence into sub sequences" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;" AnswerCount="5" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5839" PostTypeId="1" AcceptedAnswerId="5849" CreationDate="2018-03-30T18:29:55.530" Score="5" ViewCount="2639" Body="&lt;p&gt;I've recently read the paper &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies&lt;/a&gt; which introduces NEAT. I am now trying to prototype it myself in JavaScript. However, I stumbled across a few questions I can't answer.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the definition of &quot;structural innovation&quot;, and how do I store these so I can check if an innovation has already happened before?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, by keeping a list of the innovations that occurred in the current generation, it is possible to ensure that when the same structure arises more than once through independent mutations in the same generation, each identical mutation is assigned the same innovation number&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a reason for storing the type of a node (input, hidden, output)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the original paper, only connections have an innovation number, but in &lt;a href=&quot;http://www.automatonsadrift.com/neat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;other sources&lt;/a&gt;, nodes do as well. Is this necessary for the crossover? (This has already been asked &lt;a href=&quot;https://ai.stackexchange.com/q/5496/2444&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How could I limit the mutation functions to not add recurrent connections?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14625" LastEditorUserId="2444" LastEditDate="2019-07-07T19:39:06.610" LastActivityDate="2019-10-03T05:50:40.680" Title="Several questions regarding the NEAT algorithm" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ClosedDate="2020-09-09T23:34:08.133" ContentLicense="CC BY-SA 4.0" />
  <row Id="5840" PostTypeId="1" CreationDate="2018-03-30T19:07:54.197" Score="4" ViewCount="304" Body="&lt;p&gt;There are problems (e.g. &lt;a href=&quot;https://ai.stackexchange.com/q/4859/2444&quot;&gt;this one&lt;/a&gt; or &lt;a href=&quot;https://ai.stackexchange.com/q/5838/2444&quot;&gt;this other one&lt;/a&gt;) that could potentially be solved easily using traditional algorithmic techniques. I think that training a neural network (or any other machine learning model) for such sorts of problems will be more time consuming, resource-intensive, and pointless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to solve a problem, how to decide whether it is better to solve algorithmically or by using NN/ML techniques? What are the pros and cons? How can this be done in a systematic way? And if I have to answer someone why I chose a particular domain, how should I answer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example problems are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-05-29T22:45:27.003" LastActivityDate="2020-05-29T22:45:27.003" Title="How to decide whether a problem needs to be solved algorithmically or with machine learning techniques?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5841" PostTypeId="2" ParentId="5836" CreationDate="2018-03-30T19:18:07.203" Score="1" Body="&lt;p&gt;The neural Network in the image is a &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent Neural Network&lt;/a&gt;&amp;quot;(RNN). Because of the connection leading backward from &lt;code&gt;h10&lt;/code&gt; to &lt;code&gt;h01&lt;/code&gt;, &lt;code&gt;h10&lt;/code&gt; has to be a &amp;quot;memory node&amp;quot; (mn), meaning it can store its value from the previous input. The basic functionality of an RNN can be seen in this animation:&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/kpZBDfV.gif&quot; alt=&quot;animation&quot; /&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Your example:&lt;/h3&gt;&#xA;&lt;p&gt;In the beginning, the storage of the mn is initialized with a value, probably 0.&lt;br /&gt;&#xA;Now the first input is fed into network:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;i0&lt;/code&gt; = 2&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;i1&lt;/code&gt; = 3&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;h00&lt;/code&gt; = (&lt;code&gt;i0&lt;/code&gt; * 0.4) = 0.8&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;h01&lt;/code&gt; = (&lt;code&gt;i1&lt;/code&gt; * -0.9) + (&amp;quot;the stored value of &lt;code&gt;h10&lt;/code&gt;&amp;quot; * 1.2) = -2.7&lt;br /&gt;&#xA;(&amp;quot;the stored value of &lt;code&gt;h10&lt;/code&gt;&amp;quot; in the first run is 0.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;h10&lt;/code&gt; = (&lt;code&gt;h00&lt;/code&gt; * 0.85) + (&lt;code&gt;h01&lt;/code&gt; * -0.2) = 1.22&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;out&lt;/code&gt; = (&lt;code&gt;h10&lt;/code&gt; * 0.3) + (&lt;code&gt;h01&lt;/code&gt; * 0.1) =  0.096&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now you can feed the next input through the network and use -2.7 as &amp;quot;the stored value of &lt;code&gt;h10&lt;/code&gt;&amp;quot; and so on. You can also add an activation function as you would for any other NN.&lt;/p&gt;&#xA;" OwnerUserId="14625" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-03-30T21:26:54.213" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5843" PostTypeId="2" ParentId="3899" CreationDate="2018-03-31T05:05:42.900" Score="2" Body="&lt;p&gt;Any AI algorithm depends on the environment, and available actuators and sensors. In our case, the environment is a road, street, etc. The primary actuator includes wheels (or legs) of the robot. Sensors include a camera, sonar system, etc.&lt;br&gt;&#xA;A simple Model-based reflex algorithm can work in your case:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function MODEL-BASED-REFLEX-AGENT(percept) returns an action&#xA;    persistent: state, agents' current conception of the world state&#xA;                model, description of how hte next state depends on current state and action&#xA;                rules, a set of condition-action rules&#xA;                action, most recent action, initially none&#xA;    state &amp;lt;-- UPDATE-STATE(state, action, percept, model)&#xA;    rule &amp;lt;-- RULE-MATCH(state, rules)&#xA;    action &amp;lt;-- rule.ACTION&#xA;    return action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Most of the terms and functions are self-explanatory and I will try to explain the important points. This implementation helps the robot to &lt;strong&gt;keep track of the external environment by maintaining internal state&lt;/strong&gt; that depends on the percept history. Updating the internal state requires &lt;em&gt;how the environment works without our agent&lt;/em&gt; in it. For e.g., Cars stop at red light and start moving on the green signal. Another thing that is required is &lt;em&gt;how the actions of our agent will affect the world&lt;/em&gt;. Since your robot is only trying to cross the road there are not many cases here. Simple things include, stopping motors will stop the bot and starting them will move it forward.&lt;br&gt;&lt;br&gt;&#xA;The algorithm above shows &lt;strong&gt;how the current percept combined with the old internal state result in generating the updated description of the world, based on agent's model of how the world works&lt;/strong&gt;. Thus, agent's model of how the world works is the most important part. UPDATE-STATE is responsible for creating new internal state description. The actual implementation of this will depend on the environment and technology that is being used.&lt;br&gt;&lt;br&gt;&#xA;The above algorithm has been taken from the book, &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;, for agents working in the partially observable environment. Implementation of many functions also depends on how complex you want your robot to be. For e.g., we haven't considered the case of looking up while crossing the road. This is possible that something might be falling from the sky, highly improbable, but possible. The list like this never stops in real life environments.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2018-03-31T05:05:42.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5846" PostTypeId="2" ParentId="4987" CreationDate="2018-03-31T15:06:57.557" Score="2" Body="&lt;p&gt;The answers previously given are correct for AI which can indeed process more information with more computational power. However, actual reasoning ability like humans have is not defined by Church-Turing. AIXI has nothing to do with human reasoning. A pretty good clue to this fact is that AIXI has been around since 2005 and to date there are no machines based on it that have human-level reasoning. For example, an interesting topic in AI is natural language processing (NLP). I can speak into my Android phone and it will transcribe my speech into text. It seems like an amazing advance. However, this is what a human would do if they heard a foreign language and then did a phonetic transcription of what they heard. Then they looked up a phonetic chart to match the sounds with words. This would take place without any actual understanding of what they were hearing. This is how it works on my phone, much like Searle's Chinese Room.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans are quite different because they actually understand words. The equivalent to this in AI would be natural language understanding (NLU). No AI today has NLU and no theory within AI explains how to construct it. There isn't any research on AI NLU because there is no starting point. A fact that most AI enthusiasts don't like to admit is that even the smartest AI systems are routinely outclassed by rats and even six month old babies in terms of comprehension. AI systems have no comprehension or understanding and without this they have no actual reasoning ability. Human-level comprehension falls under a completely different theory from the computational derivatives of Church-Turing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you make a human-level machine agent smarter by giving it more computational power? No, because you'll run into all sorts of problems which would take a few book chapters to explain. There are enhancements you can make but these have limits. If you go by a standard deviation 15 chart for IQ like Wechsler or the 5th edition of Stanford-Binet, the chances of having an IQ of 195 is 1 out of 8 billion. So, this roughly sets the upper bound of human ability. We could probably see machine agents with an IQ of 240 but not 500 or 1,000. I do understand the confusion concerning computation since exhaustive routines in AI are time limited. For example, our dim-witted chess programs play by laborious trial and error. They don't actually get smarter with more computational power, they are just able to eliminate bad moves faster. Let me give a human example. Let's say that I could do 5 math problems of a given complexity per hour using pencil and paper. So, I add a slide rule and my rate changes to 10 problems per hour. Then I switch to a calculator and my rate increases to 20. Let's say I then start using a spreadsheet and I hit 30 per hour. I am not actually 6x smarter than I was when I used pencil and paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer the question, it is not possible to continuously increase intelligence even with unlimited computational power. However, it should be possible for machine intelligence to exceed human intelligence. One final thing that I should mention though is that this type of theory is quite good at organizing knowledge in a way that current big data methods do not. So, it is probable that the same theory that would allow a machine IQ of 240 would also provide enough assistance to a human to function at the same level.&lt;/p&gt;&#xA;" OwnerUserId="12118" LastActivityDate="2018-03-31T15:06:57.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5849" PostTypeId="2" ParentId="5839" CreationDate="2018-03-31T17:51:46.430" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;What is the definition of &quot;structural innovation&quot;, and how do I store these so I can check if an innovation has already happened before?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Structural innovation is anything added that changes the topology of the network. So a structural innovation is any added connection or added node. I don't want to get too much into the implementation, but something similar to a dictionary global variable will work. When adding a connection (or node) between two nodes, we check if a connection (or node) has ever been placed between these two nodes before. If it has then we have stored an innovation number for what this connection (or node) is identical and we set its innovation number to that. We identify each node by its number.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Is there a reason for storing the type of a node (input, hidden, output)?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, there are many structural innovations that we do not want to happen. An innovation between two input node or an innovation between two output nodes are two examples of structural innovations we don't want.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;In the original paper, only connections have an innovation number, but in other sources, nodes do as well. Is this necessary for crossover? (This has already been asked here.)&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Although I am unsure if it is explicitly stated, it is implied that nodes have an identifying number. You can see this in Figure 2 and 3 of the paper where each node is labeled with a number.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;4&quot;&gt;&#xA;  &lt;li&gt;How could I limit the mutation functions to not add backpropagation connections?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By backpropagation connections I assume you mean recurrent connections (correct me if I am wrong). You can prevent recurrent connections in many ways, but one way is to maintain a partially ordered set. When adding a connection between two nodes you check to see the order of those two nodes in the partially ordered set. The one that comes first will be the node from which the connection starts, while the second will be the node at which the connection ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a note, we can prevent recurrent connections but often times allowing them provides opportunities to find a better solution (as @PaulG comments)&lt;/p&gt;&#xA;" OwnerUserId="13088" LastEditorUserId="13088" LastEditDate="2019-10-03T05:50:40.680" LastActivityDate="2019-10-03T05:50:40.680" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5851" PostTypeId="2" ParentId="5840" CreationDate="2018-03-31T19:56:34.993" Score="1" Body="&lt;p&gt;When we apply supervised learning to a problem, we are already systematizing the approach. A human has decided that a function exists (mapping from inputs to unique output) and that the offered features are the only ones that need be considered. The learning then goes ahead to find the best solution given those constraints. Unsupervised learning is a bit more general, searching for associations or relations that might not necessarily be functions. A neural net is not yet capable of generalizing and asking for more information, it can only become more specific unless a human intervenes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything depends on the detail of the problem. If it is clear that a function must exist then we can set a NN to find that function. Many other problems are more difficult - a company is losing money and you have data but halfway there was a change in CEO, so human reasoning has to be mixed in to deal with the situation. The human can modify the architecture of the NN to introduce dummy variables, but the NN cannot do this by itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your answer really is &quot;I chose this method because of the (lack of) need for me to artificially constrain the approach to the problem.&quot;&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2018-03-31T19:56:34.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5853" PostTypeId="2" ParentId="2874" CreationDate="2018-04-01T00:34:15.560" Score="2" Body="&lt;p&gt;To the best of my knowledge, there isn't any difference between the algorithmic methods and the NN methods. Those that can solve in polynomial time do not give a precise solution. Those that do give a precise solution do not solve in polynomial time. Of those that give a precise solution, the fastest takes &lt;span class=&quot;math-container&quot;&gt;$2^N$&lt;/span&gt;, but it blows up in terms of memory. The fastest good algorithm I believe is &lt;a href=&quot;https://en.wikipedia.org/wiki/Concorde_TSP_Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;Concorde&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The efficient algorithms solve in polynomial time, don't blow up in terms of memory, and give a solution close to perfect, say, within 2-3%. Again, to the best of my knowledge, no NN has beaten the best algorithmic solutions, but there are suggestions that some NN solution could be faster.&lt;/p&gt;&#xA;" OwnerUserId="12118" LastEditorUserId="2444" LastEditDate="2021-01-17T21:21:09.960" LastActivityDate="2021-01-17T21:21:09.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5854" PostTypeId="2" ParentId="5167" CreationDate="2018-04-01T09:22:15.737" Score="1" Body="&lt;p&gt;You could train a model to classify sentences into user intents. For example, an intent could be &quot;greeting&quot;. Another intent could be &quot;help&quot;, or any other capability that your bot is able to talk about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To train you model, you should provide several examples for the same intent. For example, for &quot;greeting&quot;, you could provide &quot;Hi&quot;, &quot;Hello&quot;, &quot;What's up&quot;, etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also apply some preprocessing before feeding sentences into your model, such as &lt;a href=&quot;https://www.offconvex.org/2015/12/12/word-embeddings-1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;word embeddings&lt;/a&gt; or &lt;a href=&quot;https://stackoverflow.com/questions/14148986/how-do-you-write-a-program-to-find-if-certain-words-are-similar/14638272#14638272&quot;&gt;semantic similarity&lt;/a&gt;  with WordNet. These techniques allow to transform strings into representations that capture the similary of word meanings.  The ability of your model to detect synonyms without being retrained will highly depend on this preprocessing. &lt;/p&gt;&#xA;" OwnerUserId="8009" LastEditorUserId="8009" LastEditDate="2018-04-01T09:41:09.410" LastActivityDate="2018-04-01T09:41:09.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5855" PostTypeId="1" AcceptedAnswerId="5858" CreationDate="2018-04-01T11:48:16.067" Score="2" ViewCount="1202" Body="&lt;p&gt;I have a neural network with 2 inputs and one output, like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input    | output&#xA;____________________&#xA; a    | b   |  c       &#xA; 5.15 |3.17 | 0.0607&#xA; 4.61 |2.91 | 0.1551&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 75 samples and I am using 50 for training and 25 for testing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I feel that the training samples are not enough. Because I can't provide more real samples (due to time limitation), I would like to train the network using fake data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I know that the range for the &lt;code&gt;a&lt;/code&gt; parameter is from 3 to 14, and that the &lt;code&gt;b&lt;/code&gt; parameter is ~65% of the &lt;code&gt;a&lt;/code&gt; parameter. I also know that &lt;code&gt;c&lt;/code&gt; is a number between 0 and 1 and that it increases when a &amp;amp; b increase. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what I would like to do is to generate some data using the above restrictions (about 20 samples). For example, assume &lt;code&gt;a = 13&lt;/code&gt; , &lt;code&gt;b = 8&lt;/code&gt; and &lt;code&gt;c= 0.95&lt;/code&gt;, and train the network with these samples before training it with the real samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anybody studied the effect of doing this on the neural network? Is it possible to know if the effect will be better or worse on the networks? and are there any recommendations/guidelines if I want to do this?&lt;/p&gt;&#xA;" OwnerUserId="14701" LastActivityDate="2018-04-01T18:42:25.277" Title="Training Neural Network with 'fake' data?" Tags="&lt;neural-networks&gt;&lt;reasoning&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5857" PostTypeId="1" CreationDate="2018-04-01T13:17:46.877" Score="1" ViewCount="24" Body="&lt;p&gt;I created and operate a social network for meeting new people.  As a result of the recent FOSTA legislation, it’s imperative that I implement an automated system to prevent users from posting advertisements relating to prostitution. I do not have much expierence with AI/Machine learning.  What library, algorithm, method should I look into to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="14705" LastActivityDate="2018-04-01T13:17:46.877" Title="Social network filtering for specific topic" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5858" PostTypeId="2" ParentId="5855" CreationDate="2018-04-01T17:02:18.743" Score="2" Body="&lt;p&gt;Yes, this is not advisable. If you train your model with random data your model is not learning anything useful, because there is no information to gain from those examples. Even worse it may (and likely is) trying to generalize off of your incorrect examples, which will lessen the effect your real examples have. Essentially, you are just dampening your training set with noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are moving in the right direction though. 75 examples will not be enough if your problem has any complexity at all. And unless, you know some correlation between the inputs a,b and the output c you don't want to generate data (and even if you did know some correlation, it is not always suggested to generate data). If it is impossible to get any more data you might want to consider a statistical model, rather than a neural network.&lt;/p&gt;&#xA;" OwnerUserId="13088" LastEditorUserId="13088" LastEditDate="2018-04-01T18:36:37.240" LastActivityDate="2018-04-01T18:36:37.240" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5859" PostTypeId="1" CreationDate="2018-04-01T17:44:56.880" Score="1" ViewCount="68" Body="&lt;p&gt;I am looking to learn Ai/Machine learning during my spare and need advice on tools used / What to read / How one can integrate machine learning with a simple website for user feedback&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Scenario:&lt;/em&gt;&lt;/strong&gt;&lt;br&gt;&#xA;&lt;em&gt;.Net Web Application in C#&lt;/em&gt;&lt;br&gt;&#xA;A simple website that a user can use to to track their fitness information... i.e One inputs weight (at the end of the day), diet (food eaten, calorie counting), exercises done for the day (calories burned). kind of like myfitnesspal.com in a way. From then on wards a user can view charts of their weight tracking to check/analyse progress of weight gain / weight loss etc. &#xA;for example on user creation, one will insert a target goal to reach   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Target - 69kg&lt;br&gt;&#xA;Current - 85kg  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So they most likely want to track how far they are from their target weight loss on a day to day / weekly / monthly basis. So chart info in user's dashboard will show progress towards that goal. You get the idea... simple enough right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A1PwA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A1PwA.png&quot; alt=&quot;Image example of information relayed to user from their daily input&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Expansion:&lt;/em&gt;&lt;/strong&gt;&lt;br&gt;&#xA;Now say I want to attach a more intelligent hands off system that can analyse the user's data and provide feedback / advice on how improve or change habits from all the user's input over a period. Determine positive consistent patterns (or negative consistent patterns) with regards to them achieving their desired goal. Be it weight loss / bulking up / weight maintenance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PYTHON seems to be coming up a lot when I google this, but none specific information on how to exactly go about this &#xA;as in to attach python data analysis script/application to the .net site (could be PHP site or whatever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are my options here? Am I looking for Natural Language Processing, Machine Learning or Data Analysis. Whats my starting point or more popular tools to use with plenty of resources that one can have a look at. Creating the site isn't an issue.. its the more intelligent data analysis side that I want to dive into. F# has been mentioned here and there, but more posts seem to point out that there's more support or people using PYTHON which keeps popping up again and again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;At the most basic level, what I hope to learn/achieve if for the intelligent side of the system to give feedback like&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Machine response:&lt;/strong&gt;  (from weekly/monthly view)&lt;br&gt;&#xA;&lt;strong&gt;&lt;em&gt;&quot;Your most efficient days of losing weight were on day X when you did exercise A and B, but you could maximize your progress if you ate food X from day C.&quot;&lt;/em&gt;&lt;/strong&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do realize there there would be some Data Science involved to let the application know about healthy habits / healthy foods for weight loss etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in general, if one has a .net site and learns machine learning / ai.. how do I attach this to my site. What tools are mostly used? Do I learn machine learning in Python and roll with the flow? There doesn't seem to be information that can guide a learner diving into this sort of implementation. Any advice on the path to take will be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Help me Obi-Wan-Stackexchange, you are my only hope. &lt;/p&gt;&#xA;" OwnerUserId="14708" LastEditorUserId="14708" LastEditDate="2018-04-04T13:39:47.297" LastActivityDate="2018-04-04T13:39:47.297" Title="Simple website and attaching Data Analysis to user information for feedback" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5860" PostTypeId="2" ParentId="5855" CreationDate="2018-04-01T18:42:25.277" Score="1" Body="&lt;p&gt;If you add fake samples to the training set, your Neural Network learns new dataset that you just made, your fake samples are estimations so you add noise to your training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can use Leave one out cross validation technic for evaluating your model.&lt;/p&gt;&#xA;" OwnerUserId="10051" LastActivityDate="2018-04-01T18:42:25.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5861" PostTypeId="1" CreationDate="2018-04-01T19:50:28.670" Score="4" ViewCount="874" Body="&lt;p&gt;Deep networks notoriously take a long time to train. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most time-consuming aspect of training them? Is it the matrix multiplications? Is it the forward pass? Is it some component of the backward pass?&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2020-05-19T19:52:00.273" LastActivityDate="2020-10-01T02:29:38.680" Title="What is the most time-consuming part of training deep networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;deep-neural-networks&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5862" PostTypeId="1" CreationDate="2018-04-01T20:22:48.363" Score="7" ViewCount="618" Body="&lt;p&gt;Why aren't there neural networks that connect the output of each layer to all next layers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the output of layer 1 would be fed to the input of layers 2, 3, 4, etc. Beyond computational power considerations, wouldn't this be better than only connecting layers 1 and 2, 3 and 4, etc? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, wouldn't this solve the vanishing gradient problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If computational power is the concern, perhaps you could connect layer 1 only to the next N layers.&lt;/p&gt;&#xA;" OwnerUserId="14710" LastEditorUserId="2444" LastEditDate="2020-03-11T00:56:16.077" LastActivityDate="2020-03-11T00:59:32.597" Title="Why aren't there neural networks that connect the output of each layer to all next layers?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;hidden-layers&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5865" PostTypeId="2" ParentId="5862" CreationDate="2018-04-02T09:28:51.013" Score="4" Body="&lt;p&gt;Actually, this already exists!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I happened to make a presentation of a paper that talks about this topic. These networks are called &lt;a href=&quot;https://arxiv.org/pdf/1608.06993.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DenseNets&lt;/a&gt;, which stands for densely connected convolutional networks. Just like in your question, within a dense block, the output of each layer is given as input to all subsequent layers. Put another way, in a normal feed-forward neural network the &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;th layer is a function of the previous output &lt;span class=&quot;math-container&quot;&gt;$x_l = H(x_{l-1})$&lt;/span&gt;, while in the dense net each layer is a function of all the previous outputs &lt;span class=&quot;math-container&quot;&gt;$x_l = H([x_0, x_1, \dots, x_{l-1}])$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HVagY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HVagY.png&quot; alt=&quot;(imagine)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since it is a CNN, there is a reduction in the size of the feature maps with each pooling layer, so to keep the dimensions constant, there is an alternation between the dense block and pooling layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results are clear: not only in almost all the tests the accuracy of the dense net is greater than that of the other methods, but they do so using up to 90% fewer parameters, i.e. they have a high efficiency of parameters. Moreover, as suggested by the authors themselves, the improved accuracy can be explained by the shorter connections between the layers, which allow acting during the training phase in a deep supervision fashion, solving the vanishing gradient problem. This is similar to how it was done in other methods, but with a less complicated gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested you should definitely check out their paper &lt;a href=&quot;https://arxiv.org/pdf/1608.06993.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Densely Connected Convolutional Networks&lt;/a&gt; (2018).&lt;/p&gt;&#xA;" OwnerUserId="13199" LastEditorUserId="2444" LastEditDate="2020-03-11T00:59:32.597" LastActivityDate="2020-03-11T00:59:32.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5866" PostTypeId="2" ParentId="2762" CreationDate="2018-04-02T12:36:04.833" Score="2" Body="&lt;p&gt;First I need to note that there is no prescribed/best way to choose the shape of membership function in fuzzy systems, that's the fuzziness in it. One could argue that the best way is to ask an expert in the field where you are going to apply your fuzzy solution but those are not always available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, fuzzy membership functions are used to describe the distribution of probabilities in real world for the variable you are trying to use in your fuzzy controller. That means you go out in the real world, you look at the system you are trying to control, you try your best to understand how it works and reacts to different outside changes and based on your findings you choose the shape that best fits. Or if you want you may call this process heuristic choice (something like that, I do not like theory very much).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On top of that you need to realize one important thing, the shape of membership function does not have big impact on the resulting controller behavior. The most influencing parts are the Fuzzy Rules and their inference methods you use in your controller, but that is a different topic. So no matter what you choose it will not make a big difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gaussian functions are most commonly used because of the character of the world we are living in. Many people argue that everything in the world has Gaussian distribution (everyone is entitled to their opinion). And the triangular functions are used because they are the simplest alternative that is somewhat similar to Gaussian function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you absolutely need to chose the best one for your particular problem. There always are Simulation tools that exists precisely for this purpose. One of them is Matlab Simulink as you mentioned, but there are others if you don't like the price of the Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is, go with your gut, test it in safe environment and if it works then deploy it to the real world.&lt;/p&gt;&#xA;" OwnerUserId="1343" LastActivityDate="2018-04-02T12:36:04.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5867" PostTypeId="2" ParentId="5774" CreationDate="2018-04-02T14:47:59.657" Score="1" Body="&lt;p&gt;From the paper found from the post linked below: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;'We find that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;All that means that only values are skipped (=pooling is made) to the matrix, otherwise all works like a convolution should do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1412.6806.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1412.6806.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/44666390/max-pool-layer-vs-convolution-with-stride-performance&quot;&gt;https://stackoverflow.com/questions/44666390/max-pool-layer-vs-convolution-with-stride-performance&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-04-02T14:47:59.657" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5869" PostTypeId="1" CreationDate="2018-04-02T18:33:30.800" Score="0" ViewCount="859" Body="&lt;p&gt;I have no experience with any kind of AI, but I really want to develop a system that can detect fire in images. I think I will need a labelled dataset with labels &amp;quot;fire&amp;quot; or &amp;quot;not fire&amp;quot;, but I am not sure how I should proceed and which steps I need to take to develop this system.&lt;/p&gt;&#xA;&lt;p&gt;So, what is the general procedure to create an AI system that can detect fire in images?&lt;/p&gt;&#xA;&lt;p&gt;I heard about the Keras library, which could allow us to do this. How could I do this with Keras?&lt;/p&gt;&#xA;" OwnerUserId="14731" LastEditorUserId="2444" LastEditDate="2020-09-12T14:00:02.620" LastActivityDate="2020-09-12T14:00:02.620" Title="What is the general procedure to create an AI system that can detect fire in images?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;keras&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5870" PostTypeId="2" ParentId="5810" CreationDate="2018-04-02T20:22:46.973" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Would it be helpful to use a LSTM and reduce the input state?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd bet, no. LSTM is more complicated and harder to learn, while the input is 4 * 9 * 36 bits is still rather limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you may want to aggregate the information somehow, e.g., add additional bits informing about what cards were already played (no matter when). This information is redundant, but by providing it, you may save the network quite some learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the same time, you may want to use symmetries (all colors but trumps are equivalent and the therefore the weights should be the same).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How to handle invalid moves?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's simple: There are no invalid moves. The network provides 36 outputs of how much it wants to play a given card. You simply take the one valid card having the greatest output value. You don't try to make the network learn what moves are valid as this is neither needed nor helpful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do you have some good ideas for improvements? (like Neural-Fictitious Self-Play or something similar)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can't tell. But it should matter at the moment. First make you network clearly beat the random players, then you can look for more. Or start with Self-Play, as you want probably have both for comparison.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or is this the whole approach absolute nonsense?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think so, but ... (see below)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I design the input state as a vector (one hot encoded) with 36 &quot;bits&quot; for every player&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This doesn't sound good. Every player has 9 of 36 cards and so should be the encoding. A player doesn't know the cards of other players.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The reward is +1 for winning, -1 for losing,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In most card games I know, it matters by how much you win (unlike e.g., in Go). Even when it doesn't matter, using this information at the early learning stages is IMHO useful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-0.1 for a invalid action and&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Drop the invalid action. Just transform anything the network produces to a valid action, add no penalty (as written above).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... 0 for an action which doesn't lead to a terminal state&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;All action but the last one lead to a non-terminal state. You can use some Temporal Difference Learning or use the fact that the game has a small fixed number of moves and reward/punish all actions takes in a the whole game.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-04-02T20:22:46.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5872" PostTypeId="2" ParentId="5239" CreationDate="2018-04-03T02:39:09.883" Score="2" Body="&lt;p&gt;This has been my field of research. I've seen the previous answers that suggest that we don't have sufficient computational power, but this is not entirely true.&lt;/p&gt;&#xA;&lt;p&gt;The computational estimate for the human brain ranges from 10 petaFLOPS (&lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{16}$&lt;/span&gt;) to 1 exaFLOPS (&lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{18}$&lt;/span&gt;). Let's use the most conservative number. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Sunway_TaihuLight&quot; rel=&quot;nofollow noreferrer&quot;&gt;TaihuLight&lt;/a&gt; can do 90 petaFLOPS which is &lt;span class=&quot;math-container&quot;&gt;$9 \times 10^{16}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We see that the human brain is perhaps 11x more powerful. So, if the &lt;a href=&quot;https://plato.stanford.edu/entries/computational-mind/&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational theory of mind&lt;/a&gt; were true, then TaiHuLight should be able to match the reasoning ability of an animal about 1/11th as intelligent.&lt;/p&gt;&#xA;&lt;p&gt;If we look at a &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural cortex list&lt;/a&gt;, the squirrel monkey has about 1/12th the number of neurons in its cerebral cortex as a human. With AI, we cannot match the reasoning ability of a squirrel monkey.&lt;/p&gt;&#xA;&lt;p&gt;A dog has about 1/30th the number of neurons. With AI, we cannot match the reasoning ability of a dog.&lt;/p&gt;&#xA;&lt;p&gt;A brown rat has about 1/500th the number of neurons. With AI, we cannot match the reasoning ability of a rat.&lt;/p&gt;&#xA;&lt;p&gt;This gets us down to 2 petaFLOPS or 2,000 teraFLOPS. There are 67 supercomputers worldwide that should be capable of matching this.&lt;/p&gt;&#xA;&lt;p&gt;A mouse has half the number of neurons as a brown rat. There are 190 supercomputers that should be able to match its reasoning ability.&lt;/p&gt;&#xA;&lt;p&gt;A frog or non-schooling fish is about 1/5th of this. All of the &lt;a href=&quot;https://www.top500.org/lists/2017/11&quot; rel=&quot;nofollow noreferrer&quot;&gt;top 500 supercomputers&lt;/a&gt; are 2.5x as powerful as this. Yet, none is capable of matching these animals.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What exactly is the obstacle we are facing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The problem is that a cognitive system cannot be defined using only Church-Turing. AI should be capable of matching non-cognitive animals like arthropods, roundworms, and flatworms but not larger fish or most reptiles.&lt;/p&gt;&#xA;&lt;p&gt;I guess I need to give more concrete examples. The &lt;a href=&quot;https://www.humanbrainproject.eu/en/follow-hbp/news/an-algorithm-for-large-scale-brain-simulations&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEST&lt;/a&gt; system has demonstrated 1 second of operation of 520 million neurons and 5.8 trillion synapses in 5.2 minutes on the 5 petaFLOPS BlueGene/Q. The current thinking is that, if they could scale the system by 200 to an exaFLOPS, then they could simulate the human cerebral cortex at the same 1/300th normal speed. This might sound reasonable, but it doesn't actually make sense.&lt;/p&gt;&#xA;&lt;p&gt;A mouse has 1/1000th as many neurons as a human cortex. So this same system should be capable today of simulating a mouse brain at 1/60th normal speed. So, why aren't they doing it?&lt;/p&gt;&#xA;" OwnerUserId="12118" LastEditorUserId="2444" LastEditDate="2021-01-19T12:32:38.787" LastActivityDate="2021-01-19T12:32:38.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5873" PostTypeId="2" ParentId="5496" CreationDate="2018-04-03T08:32:21.080" Score="4" Body="&lt;p&gt;It is actually the other way around: connection IDs is what is debated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nodes always have innovation IDs (in the image, it is just their identifying number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Node IDs are sufficient to identify connections. If a connection links nodes 3 and 6, then it is the same as another connection linking nodes 3 and 6: no need for an extra ID. So why the extra innovation IDs then?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the one hand, this is an implementation choice: maybe these extra IDs would allow you to create a more complex but faster code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there is a debate around whether a connection between two nodes means the same thing at different times in evolution. If you have no innovation IDs, then you cannot tell apart an old connection from 3 and 6 and another that was independently created later in a different genome (imagine the old connection was removed first). Is this relevant? As said, it is an open debate. Surely, it is not crucial at a basic level!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question (and my answers) is related to &lt;a href=&quot;https://stackoverflow.com/a/49625345/3924118&quot;&gt;this other question&lt;/a&gt; on Stack Overflow.&lt;/p&gt;&#xA;" OwnerUserId="14744" LastEditorUserId="2444" LastEditDate="2019-07-07T19:50:43.183" LastActivityDate="2019-07-07T19:50:43.183" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5874" PostTypeId="1" CreationDate="2018-04-03T16:33:54.950" Score="4" ViewCount="170" Body="&lt;p&gt;Can we detect the emotions (or feelings) of a human through conversations with an AI?&lt;/p&gt;&#xA;&lt;p&gt;Something like a &amp;quot;confessional&amp;quot;, disregarding human possibilities to lie.&lt;/p&gt;&#xA;&lt;p&gt;Below, I have the categories joyful, sadness, anger, fear and affection. For each category, there are several words that can be in the texts that refer to it.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Joy:&lt;/strong&gt; &lt;strong&gt;(&lt;/strong&gt; cheerful, happy, confident, happy, satisfied, excited, interested, dazzled, optimistic, relieved, euphoric, drunk, witty, good &lt;strong&gt;)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Sadness:&lt;/strong&gt; &lt;strong&gt;(&lt;/strong&gt; sad, desperate, displeased, depressed, bored, lonely, hurt, desolate, meditative, defrauded, withdrawn, pitying, concentrated, depressed, melancholic, nostalgic &lt;strong&gt;)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Anger:&lt;/strong&gt; &lt;strong&gt;(&lt;/strong&gt; aggressive, critical, angry, hysterical, envious, grumpy, disappointed, shocked, exasperated, frustrated, arrogant, jealous, agonized, hostile, vengeful &lt;strong&gt;)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Fear:&lt;/strong&gt; &lt;strong&gt;(&lt;/strong&gt; shy, frightened, fearful, horrified, suspicious, disbelieving, embarrassed, embarrassed, shaken, surprised, guilty, anxious, cautious, indecisive, embarrassed, modest &lt;strong&gt;)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Affection:&lt;/strong&gt; &lt;strong&gt;(&lt;/strong&gt; loving, passionate, supportive, malicious, dazzled, glazed, homesick, embarrassed, indifferent, curious, tender, moved, hopeful &lt;strong&gt;)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Flow Example&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Phrase 1:&lt;/strong&gt; &amp;quot;I'm very happy! It concludes college.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Categorization 1:&lt;/strong&gt;&#xA; - Joy &lt;strong&gt;(+1)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sadness &lt;strong&gt;(-1)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Phrase 2:&lt;/strong&gt; &amp;quot;I'm sad, my mother passed away.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Categorization 2:&lt;/strong&gt;&#xA; - Sadness &lt;strong&gt;(+1)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Joy &lt;strong&gt;(-1)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Phrase 3:&lt;/strong&gt; &amp;quot;I met a girl, but I was ashamed.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Categorization 3:&lt;/strong&gt;&#xA; - Fear &lt;strong&gt;(+1)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is this a clever way to follow and / or improve, or am I completely out of the way?&lt;/p&gt;&#xA;&lt;p&gt;I see that there is a Google product that creates parsing according to the phrases. I do not know how it works, because I like to recreate the way I think it would work.&lt;/p&gt;&#xA;&lt;p&gt;Remembering that this would not be the only way to categorize the phrase. This would be the first phase of the analysis. I can also identify the subject of the sentence, so we would know if the sadness is from the creator of the message or from a third party, in most cases.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.nltk.org/book/ch08.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLTK&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/text-machine-lab/sentimental&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sentiment Analysis Python Example&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="2444" LastEditDate="2020-12-04T15:13:26.573" LastActivityDate="2020-12-04T15:13:26.573" Title="Can we detect the emotions (or feelings) of a human through conversations with an AI?" Tags="&lt;natural-language-processing&gt;&lt;emotional-intelligence&gt;&lt;sentiment-analysis&gt;" AnswerCount="3" CommentCount="7" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5875" PostTypeId="1" CreationDate="2018-04-03T16:38:30.767" Score="1" ViewCount="128" Body="&lt;p&gt;I'm working with acoustic data (filterbank features) and I want to build a neural network to detect claps using an LSTM (or a GRU) with a binary output (present/abscent), and I'm wondering about how I should prepare my data before feeding them to the RNN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have 20 seconds of claps (separate claps separated by ~ 0.1 seconds) what is the difference between : &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Feeding the network a series of N claps as one example (with variable N : 1, 2, .., 10, ..) + padding with zeros to fit the longest sequence. &lt;/li&gt;&#xA;&lt;li&gt;Feeding the network multiple examples of 1 clap. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My problem is not restricted to claps but covers patterns that can be observed as separated occurrences, periodic sequence of occurrences, variable-length-period &quot;quasi-periodic&quot; sequence of occurrences, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike an ergodic HMM, an RNN doesn't have any loops to &quot;jump back&quot; to a previous &quot;acoustic state&quot;, so what should-I do with this kind of data ?&lt;/p&gt;&#xA;" OwnerUserId="14751" LastActivityDate="2018-04-03T16:38:30.767" Title="Best practices to classify recurring patterns using an LSTM or GRU" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5876" PostTypeId="5" CreationDate="2018-04-03T17:07:56.127" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Computer_vision&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-07-25T21:38:54.843" LastActivityDate="2019-07-25T21:38:54.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5877" PostTypeId="4" CreationDate="2018-04-03T17:07:56.127" Score="0" Body="For questions related to computer vision, which is an interdisciplinary scientific field (which can e.g. use image processing techniques) that deals with how computers can be made to gain high-level understanding from digital images or videos. For example, image recognition (that is, the identification of the type of objects in an image) is a computer vision problem." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-07-25T22:58:21.333" LastActivityDate="2019-07-25T22:58:21.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5879" PostTypeId="5" CreationDate="2018-04-03T17:14:35.293" Score="0" Body="&lt;p&gt;In computer science, Monte Carlo tree search (MCTS) is a heuristic search algorithm for some kinds of decision processes, most notably those employed in game play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Search - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-03T19:47:28.667" LastActivityDate="2018-04-03T19:47:28.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5880" PostTypeId="4" CreationDate="2018-04-03T17:14:35.293" Score="0" Body="For questions about clarifications/applications/implementation of the Monte Carlo Search algorithm used especially in Artificial Intelligence/Combinatorial games." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-03T19:46:39.490" LastActivityDate="2018-04-03T19:46:39.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5881" PostTypeId="5" CreationDate="2018-04-03T17:15:52.047" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Applications_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Applications of AI - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-06-25T20:05:32.933" LastActivityDate="2019-06-25T20:05:32.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5882" PostTypeId="4" CreationDate="2018-04-03T17:15:52.047" Score="0" Body="For questions about clarifications, methods, algorithms or best practices for or examples of (possible) applications of Artificial Intelligence and Machine Learning algorithms." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-06-25T20:18:11.097" LastActivityDate="2019-06-25T20:18:11.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5883" PostTypeId="2" ParentId="5874" CreationDate="2018-04-03T21:57:39.997" Score="1" Body="&lt;p&gt;It could work using supervised learning , as long as you have the required dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a low error ratio using unsupervised learning of the human emotion spectrum would prove to be more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex :&#xA;How would you defined being in love to a neural network ? &#xA;Joy +1 , Sadness -1 ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now , How would you define being in love with , let’s say, someone you know you could never be with ? &#xA;Joy -1 , Sadness +1, but at the same time , the only fact that you are thinking about that person bring a Joy +1 .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human emotions are quite complex .&#xA;A good start (in my humble opinion) would be to read about ‘emotion-related’ hormones , and how they affect the brain ( dopamine , serotonin, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some emotions are really a precise mix of these hormones , probably giving you a good hint on how to ‘caregorize’ your network.&lt;/p&gt;&#xA;" OwnerUserId="13038" LastActivityDate="2018-04-03T21:57:39.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5884" PostTypeId="2" ParentId="5193" CreationDate="2018-04-04T00:27:08.897" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.cs.waikato.ac.nz/ml/weka/downloading.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.waikato.ac.nz/ml/weka/downloading.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Great little tool to experiment with various algorithms and compare their efficiencies.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2018-04-04T00:27:08.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5885" PostTypeId="1" CreationDate="2018-04-04T01:52:15.150" Score="7" ViewCount="429" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/5258/2444&quot;&gt;A question about swarm intelligence as a potential method of strong general AI&lt;/a&gt; came up recently, and yielded some useful answers and clarifications regarding the nature of swarm intelligence. But it got me thinking about &quot;group intelligence&quot; in general.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here organism is synonymous with algorithm, so a complex organism is an algorithm made up of component algorithms, based on a set of instructions in the form of a string. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the &lt;a href=&quot;https://en.wikipedia.org/wiki/Portuguese_man_o%27_war&quot; rel=&quot;nofollow noreferrer&quot;&gt;Portuguese man o' war&lt;/a&gt;, not a single animal, but a &lt;a href=&quot;https://en.wikipedia.org/wiki/Colony_(biology)&quot; rel=&quot;nofollow noreferrer&quot;&gt;colonial organism&lt;/a&gt;. In this case, that means a set of animals connected for mutual benefit.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;em&gt;physalia physalis&lt;/em&gt; are pretty smart as a species in that they've been around for a while, I'm not finding them on any endangered lists, and based on their habitat it looks like global warming will be a jackpot for them. And they don't even have brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each component of the &lt;em&gt;physalia&lt;/em&gt; has a narrow function, colony organism itself has a more generalized function, which is the set of functions necessary for maintenance and reproduction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;{Man o' War} ⊇ { {pneumatophore}, {gonophores, siphosomal nectophores,vestigial siphosomal nectophores}, {free gastrozooids, tentacled gastrozooids, gonozooids, gonopalpons}, {dactylozooids}, {gonozooids}, {gastrozooids} }   &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What types of applications qualify as &quot;compound intelligences&quot;? What is the thinking on groups of neural networks comprising generally stronger or simply more generalized intelligence?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I recognize the underlying problem is ultimately complexity and that &quot;strong narrow AI&quot; is, by definition, limited, so I use &quot;generalized&quot; and omit &quot;strong&quot; because human-like and superintelligence are not conditions.  Compound intelligence is defined as a colony of dependent intelligences.*  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Utility software is often a form of an expert system that manages a set of functions of varying degrees of complexity. There's currently a great deal of focus on autonomous vehicles, which would seem to require sets of functions.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links to research papers on this or related subjects would be ideal.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://oceana.org/marine-life/corals-and-other-invertebrates/portuguese-man-o-war&quot; rel=&quot;nofollow noreferrer&quot;&gt;Portuguese Man o' War (oceana.org)&lt;/a&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://fivethirtyeight.com/features/the-bugs-of-the-world-could-squish-us-all/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Bugs Of The World Could Squish Us All&lt;/a&gt;&lt;br&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-04-03T15:37:43.373" LastActivityDate="2020-04-03T15:37:43.373" Title="What types of applications qualify as &quot;compound intelligences&quot;?" Tags="&lt;machine-learning&gt;&lt;definitions&gt;&lt;applications&gt;&lt;reference-request&gt;&lt;swarm-intelligence&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5886" PostTypeId="5" CreationDate="2018-04-04T02:01:02.943" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In philosophy, systems theory, science, and art, emergence occurs when &quot;the whole is greater than the sum of the parts,&quot; meaning the whole has properties its parts do not have. These properties come about because of interactions among the parts. In mathematical shorthand, W = P + I.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Emergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emergence (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Emergence#Computer_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Emergence#Computer_AI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-04T02:01:02.943" LastActivityDate="2018-04-04T02:01:02.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5887" PostTypeId="4" CreationDate="2018-04-04T02:01:02.943" Score="0" Body="Use for questions about emergence as related to artificial intelligence and complexity." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-04T02:01:02.943" LastActivityDate="2018-04-04T02:01:02.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5888" PostTypeId="2" ParentId="5874" CreationDate="2018-04-04T03:00:10.350" Score="2" Body="&lt;p&gt;I think you are definitely on a very sensible track. No one defines right or wrong in emotion field. It's not hard science. It's all theories.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have recently read a &lt;a href=&quot;https://link.springer.com/article/10.1007/s10994-017-5666-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; regarding emotions in Reinforcement Learning (RL). It has explained briefly emotion from 3 perspectives: &lt;em&gt;psychology, neuroscience&lt;/em&gt; and &lt;em&gt;computer science&lt;/em&gt;. In particular, your way of emotion definition matches one of the &lt;strong&gt;categorical emotion theory&lt;/strong&gt; in the psychology perspective. The other theories in &lt;strong&gt;psychology perspective&lt;/strong&gt; and &lt;strong&gt;componential emotion&lt;/strong&gt;. You can try to implement them and try out which one works well. The paper has also introduced ways to measure the level of emotions (emotion elicitation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link for the &lt;a href=&quot;https://link.springer.com/article/10.1007/s10994-017-5666-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; I have mentioned. I am sure you will receive lots of inspiration. I have also written a &lt;a href=&quot;https://diansheng.github.io/Emotion-in-RL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;summary&lt;/a&gt; of this paper. Take a look if the original paper is too long to read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have any concrete solution for implementing. But the general idea is always trying to categorize abstract concepts and quantify them. And try some thing, and iteratively modify and improve it. All the best!&lt;/p&gt;&#xA;" OwnerUserId="5767" LastEditorUserId="5767" LastEditDate="2018-04-04T08:53:07.600" LastActivityDate="2018-04-04T08:53:07.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5890" PostTypeId="1" CreationDate="2018-04-04T09:13:15.360" Score="5" ViewCount="622" Body="&lt;p&gt;I am trying to find out what are some good learning strategies for Deep Q-Network with opponents. Let's consider the well-known game Tic-Tac-Toe as an example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How should an opponent be implemented to get good and fast improvements?&lt;/li&gt;&#xA;&lt;li&gt;Is it better to play against a random player or a perfect player or should the opponent be a DQN player as well?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14587" LastEditorUserId="2444" LastEditDate="2021-01-31T03:32:03.097" LastActivityDate="2021-01-31T03:32:03.097" Title="What are good learning strategies for Deep Q-Network with opponents?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;game-ai&gt;&lt;tic-tac-toe&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5891" PostTypeId="1" AcceptedAnswerId="5892" CreationDate="2018-04-04T12:36:54.207" Score="6" ViewCount="3664" Body="&lt;p&gt;To provide a bit of context, I'm a software engineer &amp;amp; game enthusiast (card games, especially). The thing is I've always been interested in AI oriented to games. In college, I programmed my own Gomoku AI, so I'm a bit familiar with the basic concepts of AI applied to games and have read books &amp;amp; articles about Game Theory as well.&lt;/p&gt;&#xA;&lt;p&gt;My issue comes when I try to analyze AI's for &lt;strong&gt;Imperfect Information&lt;/strong&gt; games like (Poker, Magic the gathering, Hearthstone, etc). In most cases, when I found an AI for Hearthstone, it was either some sort of Monte Carlo or MinMax strategy. I honestly think, although it might even provide some decent results, it will still be always quite flat and linear, since it doesn't take into account what deck the opponent is playing and almost always tries to follow the same game-plan, since it will not change based on tells your opponent might give away via cards played (a hint that a human would catch).&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if using Neural Networks would be better than just using a raw evaluation of board state + hands + Hp each turn without taking into account learning about possible threads the opponent might have, how to deny the opponent the best plays he could make, etc.&lt;/p&gt;&#xA;&lt;p&gt;My intuition tells me that this is way harder and far more complex.&lt;/p&gt;&#xA;&lt;p&gt;Is that the only reason the NN method is not used? Has there been any research to prove how much efficiency edge would be between those 2 approaches?&lt;/p&gt;&#xA;" OwnerUserId="14769" LastEditorUserId="2444" LastEditDate="2021-01-20T22:55:20.247" LastActivityDate="2021-01-20T22:55:20.247" Title="Why most imperfect information games usually use non machine learning AI?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;minimax&gt;&lt;imperfect-information&gt;&lt;evaluation-functions&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5892" PostTypeId="2" ParentId="5891" CreationDate="2018-04-04T13:35:42.850" Score="7" Body="&lt;p&gt;A heuristic search using MCTS + minimax + alphabeta pruning is a highly efficient AI planning process. What the AI techniques of reinforcement learning (RL) plus neural networks (NNs) typically add to this is a way to establish better heuristics.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My intuition tells me that this is way harder and far more complex. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's not actually that much more complex in concept. Replace the hand-coded heuristic with a learning engine, e.g. DQN or AC3. Train the learning engine from human expert play examples and/or from self play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It &lt;em&gt;is&lt;/em&gt; harder though, because there are many things that can go wrong with an NN-based estimator in a RL context. You will need to make many experiments with different hyper-parameters of the learning engine. For complex games, you may have to invest many 100s of hours of training, which you might want to compare against the end result of a similar amount of time spent refining expert heuristic systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For imperfect information games, you may also want to use something that can learn an internal state representation. That could be some kind of explicit belief state that you maintain like an expert system, or something that attempts to learn a good representation, such as an RNN (e.g. LSTM). This may not be necessary for a first try at an agent though, since the MCTS search will make up for &lt;em&gt;some&lt;/em&gt; inadequacies of low accuracy heuristics.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is that the only reason the NN method is not used?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Up until quite recently, approaches using RL and NN were far harder to find examples of outside of academic machine learning research, and there were not any pre-written frameworks for LSTM or e.g. AC3. In the last few years, RL and NN frameworks have started to appear making an AI self-learning approach far more approachable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect that many hobby-level coders considering game-playing AI nowadays would seriously take a look at RL and NNs in order to &lt;em&gt;learn&lt;/em&gt; robust heuristics for their game projects. However, the &quot;traditional&quot; search-based methods still work in conjunction with these for a completed agent. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Has there been any research to prove how much efficiency edge would be between those 2 approaches?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For card games, I am not aware of any specific research, although I am just a hobbyist, yet to write any specific game engine more complex than tic-tac-toe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For perfect information board games, &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaZero&quot; rel=&quot;noreferrer&quot;&gt;the chess playing variant of AlphaZero&lt;/a&gt; demonstrates applicability of RL+NN self-play approach versus &quot;traditional&quot; heuristics plus search (represented by Stockfish). However, the framing of the tournament has been criticised as unfair to Stockfish, so it is not necessarily an open-and-shut case that RL is strictly better.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-04-04T13:35:42.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5893" PostTypeId="2" ParentId="5890" CreationDate="2018-04-04T14:05:51.500" Score="4" Body="&lt;p&gt;In a two player zero-sum game (if I win, you lose and vice-versa), then you can have a simple and efficient solution learning from self-play.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How should an opponent be implemented to get good and fast improvements?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You don't need to think in terms of agent vs opponent, instead think in terms of coding both the players' goals into a single Q function. Score +1 if player A wins, -1 if player B wins, and zero for a draw. It is then player A's goal to maximise the score and player B's goal to minimise the score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then implement and learn &lt;em&gt;both&lt;/em&gt; player strategies in the same self-play learning session and same Q function, using minimax. In practice that means where in Q learning you generally pick the maximising action on next state to bootstrap Q values, in a &lt;em&gt;minimax&lt;/em&gt; variant, you pick maximising or minimising action depending on whose turn it is. Otherwise the Q learning algorithm is the same as normal. &lt;a href=&quot;https://github.com/neilslater/game_playing_scripts/blob/master/tictactoe_q.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;I have implemented this, but not for DQN, just for tabular Q-learning&lt;/a&gt; - feel free to learn from, copy and/or re-use any part of that code.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it reasonable to play against a random player, a perfect player or should the opponent be a DQN player as well?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Q learner will learn to optimise against whichever player you make it play against. Against a random player, it will not necessarily learn to play well, just well enough to defeat the randomness. It may even make deliberate mistakes - such as not blocking a winning line - knowing it has a better chance to win due to random opponent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Against a perfect player is possible with tic tac toe (because you can construct such a player), although there might be gaps in the trained Q values - game states never seen - which mean that an imperfect opponent could actually defeat the trained agent! In practice this does not scale to more complex unsolved games, because no perfect players exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another DQN player should work fine. You would end up with two agents, each specialising in playing one player's turns. This is less efficient than a single minimax-based player, but no expected problems. It may be a preferred choice for some games, especially if they are not zero-sum.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-04-04T14:18:44.970" LastActivityDate="2018-04-04T14:18:44.970" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5894" PostTypeId="5" CreationDate="2018-04-04T17:42:17.950" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;A solved game is a game whose outcome (win, lose or draw) can be correctly predicted from any position, assuming that both players play perfectly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solved Game (wikipedia)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial Game Theory (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-04T17:42:17.950" LastActivityDate="2018-04-04T17:42:17.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5895" PostTypeId="4" CreationDate="2018-04-04T17:42:17.950" Score="0" Body="For question about the formal concept of perfect play  in regard to solved games. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-04T17:42:17.950" LastActivityDate="2018-04-04T17:42:17.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5896" PostTypeId="5" CreationDate="2018-04-04T17:44:17.857" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a&#xA;General Reinforcement Learning Algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://deepmind.com/blog/alphago-zero-learning-scratch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero: Learning from scratch (DeepMind)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithmic Game Theory (wikipedia)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-04T17:50:44.710" LastActivityDate="2018-04-04T17:50:44.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5897" PostTypeId="4" CreationDate="2018-04-04T17:44:17.857" Score="0" Body="For questions about the technique of &quot;self play&quot; in reinforcement learning in relation to combinatorial games and games in general.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-04T17:50:44.710" LastActivityDate="2018-04-04T17:50:44.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5898" PostTypeId="1" AcceptedAnswerId="8053" CreationDate="2018-04-04T17:45:39.967" Score="6" ViewCount="3197" Body="&lt;p&gt;I found &lt;a href=&quot;http://www.asimovinstitute.org/neural-network-zoo/&quot; rel=&quot;noreferrer&quot;&gt;this nice-ish-looking diagram&lt;/a&gt;, but it has a wholly inadequate descriptions for each of the cell types, aside from including names.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the definition/description of each of these cell types?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input Cell&lt;/li&gt;&#xA;&lt;li&gt;Backfed Input Cell&lt;/li&gt;&#xA;&lt;li&gt;Noisy Input Cell&lt;/li&gt;&#xA;&lt;li&gt;Hidden Cell&lt;/li&gt;&#xA;&lt;li&gt;Probablistic Hidden Cell&lt;/li&gt;&#xA;&lt;li&gt;Spiking Hidden Cell&lt;/li&gt;&#xA;&lt;li&gt;Output Cell&lt;/li&gt;&#xA;&lt;li&gt;Match Input Output Cell&lt;/li&gt;&#xA;&lt;li&gt;Recurrent Cell&lt;/li&gt;&#xA;&lt;li&gt;Memory Cell&lt;/li&gt;&#xA;&lt;li&gt;Different Memory Cell&lt;/li&gt;&#xA;&lt;li&gt;Kernel&lt;/li&gt;&#xA;&lt;li&gt;Convolution or Pool&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14776" LastEditorUserId="2444" LastEditDate="2020-06-04T19:34:02.497" LastActivityDate="2020-06-04T19:34:02.497" Title="What is the definition of each of these neural network cell types?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5899" PostTypeId="1" CreationDate="2018-04-04T18:54:43.797" Score="6" ViewCount="1580" Body="&lt;p&gt;In the search tree below, there are 11 nodes, 5 of which are leaves. There are 10 branches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the average branching factor given by 10/6, or 10/11?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are leaves included in the calculation? Intuitively, I would think not, since we are interested in nodes with branches. However, a definition given to me by my professor was &quot;The average number of branches of all nodes in the tree&quot;, which would imply leaves are included.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MCRn3.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MCRn3.png&quot; alt=&quot;Search tree&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14777" LastEditorUserId="1671" LastEditDate="2018-04-04T19:24:21.660" LastActivityDate="2018-09-07T17:01:12.283" Title="Are leaf nodes included in the calculation of average branching factor for search trees?" Tags="&lt;ai-basics&gt;&lt;search&gt;&lt;branching-factors&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5900" PostTypeId="2" ParentId="5885" CreationDate="2018-04-05T01:24:16.397" Score="-1" Body="&lt;p&gt;I did work on compound intelligence because that is the direction that Google is trying to go. I couldn't find any basis for it. In other words, having a collection of AI expert systems does not seem to provide any collective intelligence. You would also need some kind of control program that could decide which system to use. Currently, Google relies on the user to choose. If Google was able to create an independent control program it would already be out. This does not seem to be a matter of complexity or code tweeking but a fundamental limit with AI.&lt;/p&gt;&#xA;" OwnerUserId="12118" LastActivityDate="2018-04-05T01:24:16.397" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5904" PostTypeId="1" AcceptedAnswerId="18695" CreationDate="2018-04-05T08:59:43.157" Score="8" ViewCount="309" Body="&lt;p&gt;In a nutshell: I want to understand why a one hidden layer neural network converges to a good minimum more reliably when a larger number of hidden neurons is used. Below a more detailed explanation of my experiment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a simple 2D XOR-like classification example to understand the effects of neural network initialization better. Here's a visualisation of the data and the desired decision boundary:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4jCnc.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4jCnc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each blob consists of 5000 data points. The minimal complexity neural network to solve this problem is a one-hidden layer network with 2 hidden neurons. Since this architecture has the minimum number of parameters possible to solve this problem (with a NN) I would naively expect that this is also the easiest to optimise. However, this is not the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that with random initialization this architecture converges around half of the time, where convergence depends on the signs of the weights. Specifically, I observed the following behaviour:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w1 = [[1,-1],[-1,1]], w2 = [1,1] --&amp;gt; converges&#xA;w1 = [[1,1],[1,1]],   w2 = [1,-1] --&amp;gt; converges&#xA;w1 = [[1,1],[1,1]],   w2 = [1,1] --&amp;gt; finds only linear separation&#xA;w1 = [[1,-1],[-1,1]], w2 = [1,-1] --&amp;gt; finds only linear separation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This makes sense to me. In the latter two cases the optimisation gets stuck in suboptimal local minima. However, when increasing the number of hidden neurons to values greater than 2, the network develops a robustness to initialisation and starts to reliably converge for random values of w1 and w2. You can still find pathological examples, but with 4 hidden neurons the chance that one &quot;path way&quot; through the network will have non-pathological weights is larger. But happens to the rest of the network, is it just not used then? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody understand better where this robustness comes from or perhaps can offer some literature discussing this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more information: this occurs in all training settings/architecture configurations I have investigated. For instance, activations=Relu, final_activation=sigmoid, Optimizer=Adam, learning_rate=0.1, cost_function=cross_entropy, biases were used in both layers. &lt;/p&gt;&#xA;" OwnerUserId="14789" LastEditorUserId="14789" LastEditDate="2018-04-05T14:33:55.940" LastActivityDate="2020-03-17T16:43:01.340" Title="Why does a one-layer hidden network get more robust to poor initialization with growing number of hidden neurons?" Tags="&lt;neural-networks&gt;&lt;optimization&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5906" PostTypeId="1" AcceptedAnswerId="5917" CreationDate="2018-04-05T13:31:18.120" Score="3" ViewCount="114" Body="&lt;p&gt;I have recently gone about and made a &lt;a href=&quot;https://jsbin.com/vijawuleyo/edit?html,console,output&quot; rel=&quot;nofollow noreferrer&quot;&gt;simple AI&lt;/a&gt;, one that gives responses to an input (albeit completely irrelevant and nonsensical ones), using Synaptic.js. Unfortunately, this is not the type of text generation I am looking for. What I am looking for would be a way to get connections between words and generate text from that. (What would be preferable would be to also generate at least semi-sensible answers also.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is part of project Raphiel, and can be checked out in the room associated with this site. What I want to know is what layer combination would I use for text generation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been told to avoid retrieval-based bots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the method to send and receive messages, I just need to figure out what combination of layers would be the best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless I have the numbers wrong, this will be SE's second NN chatbot.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2018-04-10T18:32:56.733" LastActivityDate="2018-04-10T18:32:56.733" Title="How would one go about generating *sensible* responses to chat?" Tags="&lt;ai-design&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5910" PostTypeId="5" CreationDate="2018-04-05T17:05:19.237" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://intelligence.org/why-ai-safety/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why AI safety? - Machine Intelligence Research Institute&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-05T18:45:28.537" LastActivityDate="2018-04-05T18:45:28.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5911" PostTypeId="4" CreationDate="2018-04-05T17:05:19.237" Score="0" Body="For questions regarding various parameters/methods of implementation/improvement/analyzing the safety of an Artificially Intelligent algorithm.  A typical real world example is autonomous vehicles." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2020-01-02T21:53:26.210" LastActivityDate="2020-01-02T21:53:26.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5912" PostTypeId="5" CreationDate="2018-04-05T17:08:00.630" Score="0" Body="&lt;p&gt;Linear algebra is the study of linear sets of equations and their transformation properties. Linear algebra allows the analysis of rotations in space, least squares fitting, solution of coupled differential equations, determination of a circle passing through three given points, as well as many other problems in mathematics, physics, and engineering. Confusingly, linear algebra is not actually an algebra in the technical sense of the word &quot;algebra&quot; (i.e., a vector space V over a field F, and so on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mathworld.wolfram.com/LinearAlgebra.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear Algebra - WolframMathWorld&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear Algebra - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-05T18:46:19.070" LastActivityDate="2018-04-05T18:46:19.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5913" PostTypeId="4" CreationDate="2018-04-05T17:08:00.630" Score="0" Body="For questions about the use/aspects/implementation/intuition/mathematical proofs of various Linear Algebra methods used in Machine Learning and AI algorithms." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-05T18:46:29.020" LastActivityDate="2018-04-05T18:46:29.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5916" PostTypeId="1" CreationDate="2018-04-06T00:09:08.880" Score="4" ViewCount="36" Body="&lt;p&gt;I'm looking to perform two tasks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Train a classifier to classify code as serial or parallel&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train a generative algorithm to generate parallel code from serial &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the first task a simple scraper can scrape random C and C++ code from git, however for the second step I would need a decently large source of examples of serial to parallel code. Any ideas or pointers for existing or creating this type of dataset would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14809" LastActivityDate="2018-06-26T15:08:46.250" Title="Looking to build, compile, and/or find dataset for serial-parallelized code examples" Tags="&lt;classification&gt;&lt;datasets&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5917" PostTypeId="2" ParentId="5906" CreationDate="2018-04-06T00:38:01.823" Score="2" Body="&lt;p&gt;This seems like a problem for the use of an encoder-decoder pair such as those seen in text summarization (see this paper by Rush et al.: &lt;a href=&quot;https://arxiv.org/pdf/1509.00685.pdf%C3%AF%C2%BC&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1509.00685.pdfï%C2%BC&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would need the following layers: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;LSTM layer to encode the given input text into an embedding &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;LSTM layer the looks over the currently generated output to encode the text into an embedding&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A dense soft-max layer for generating words probabilistically based on the output of the two contextual LSTM encoders&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please see the following blog post by Jason Brownlee that outlines this approach and others, while giving implementation details and snippets (&lt;a href=&quot;https://machinelearningmastery.com/encoder-decoder-models-text-summarization-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/encoder-decoder-models-text-summarization-keras/&lt;/a&gt;)! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that this would require a large set of training examples of input text and reasonable responses. You might be able to scrape Reddit post responses and comments off of those for a start? Let me know if I misunderstood the question.&lt;/p&gt;&#xA;" OwnerUserId="14809" LastActivityDate="2018-04-06T00:38:01.823" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5919" PostTypeId="1" CreationDate="2018-04-06T06:09:29.033" Score="1" ViewCount="982" Body="&lt;p&gt;When training on large neural network, how to deal with the case that the gradients are too small to have any impact?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI, I have an RNN, which has multiple LSTM cells and each cell has hundreds of neurons. Each training data has thousands of steps, so the RNN would unroll thousands of times. When I print out all gradients, they are very small, like e-20 of the variable values. Therefore the training does not change the variable values at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, I think this is not an issue of vanishing gradients. Note that the gradients are uniformly small from the beginning to the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion to overcome this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank!&lt;/p&gt;&#xA;" OwnerUserId="14816" LastActivityDate="2018-08-12T03:01:46.073" Title="Too small gradient on large neural network" Tags="&lt;gaming&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5920" PostTypeId="5" CreationDate="2018-04-06T06:59:36.280" Score="0" Body="&lt;p&gt;Knowledge representation and reasoning (KR) is the field of artificial intelligence (AI) dedicated to representing information about the world in a form that a computer system can utilize to solve complex tasks such as diagnosing a medical condition or having a dialog in a natural language. Knowledge representation incorporates findings from psychology about how humans solve problems and represent knowledge in order to design formalisms that will make complex systems easier to design and build. Knowledge representation and reasoning also incorporates findings from logic to automate various kinds of reasoning, such as the application of rules or the relations of sets and subsets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge Representation - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-06T17:27:33.653" LastActivityDate="2018-04-06T17:27:33.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5921" PostTypeId="4" CreationDate="2018-04-06T06:59:36.280" Score="0" Body="For questions about methods/intuitions/proofs/improvements in the knowledge representation in a problem to  be solved by Artificially Intelligent agent." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-06T17:27:23.247" LastActivityDate="2018-04-06T17:27:23.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5922" PostTypeId="5" CreationDate="2018-04-06T07:01:57.013" Score="0" Body="&lt;p&gt;Decision theory (or the theory of choice) is the study of the reasoning underlying an agent's choices. Decision theory can be broken into three branches: normative decision theory, which gives advice on how to make the best decisions, given a set of uncertain beliefs and a set of values; descriptive decision theory, which analyzes how existing, possibly irrational agents actually make decisions; and prescriptive decision theory, which tries to guide or give procedures on how or what we should do in order to make best decisions in line with the normative theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Decision Theory - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-06T17:27:59.460" LastActivityDate="2018-04-06T17:27:59.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5923" PostTypeId="4" CreationDate="2018-04-06T07:01:57.013" Score="0" Body="For various clarifications/proofs behind an Artificially Intelligent agent's reasoning process using Decision Theory." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-06T17:27:28.757" LastActivityDate="2018-04-06T17:27:28.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5925" PostTypeId="2" ParentId="2429" CreationDate="2018-04-06T09:24:23.223" Score="6" Body="&lt;p&gt;Adding some to what Christian said. Facts taken from the book, &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/B._F._Skinner&quot; rel=&quot;noreferrer&quot;&gt;Burrhus Frederic Skinner&lt;/a&gt;, a psychologist and behaviourist, published his book &lt;a href=&quot;https://en.wikipedia.org/wiki/Verbal_Behavior&quot; rel=&quot;noreferrer&quot;&gt;Verbal Behaviour&lt;/a&gt; in 1957. His work contains the detailed account of the behaviourist approach to language learning.&lt;br&gt;&lt;br&gt;&#xA;Noam Chomsky later wrote a review on the book, which for some reason became more famous than the book itself. Chomsky has his own theory of Syntactic Structures for this. He even mentioned that the behaviourist theory did not address the notion of creativity in language as it did not explain how a child could understand and make up sentences that he/she has never heard before. &lt;strong&gt;His theory based on syntactic models are dated back to Indian linguist &lt;a href=&quot;https://en.wikipedia.org/wiki/P%C4%81%E1%B9%87ini&quot; rel=&quot;noreferrer&quot;&gt;Panini&lt;/a&gt; (350 B.C.) who was an ancient Sanskrit philologist, grammarian, and a revered scholar&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2018-04-06T09:24:23.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5926" PostTypeId="2" ParentId="5904" CreationDate="2018-04-06T10:17:42.757" Score="1" Body="&lt;p&gt;You grasped a bit of the answer.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the latter two cases the optimisation gets stuck in suboptimal local minima. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When you have only 2 dimensions, a local minima exists. When you have more dimensions, this minima gets harder and harder to reach, as its likelihood decreases. Intuitively, you have a lot more dimensions through which you can improve than if you only had 2 dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem still exists, even with 1000 neurons you could find a specific set of weights which was a local minimum. However, it just becomes so much less likely.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2018-04-06T10:17:42.757" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5927" PostTypeId="1" CreationDate="2018-04-06T14:02:14.460" Score="1" ViewCount="36" Body="&lt;p&gt;I'm detecting objects on images. I want to detect up to 10 objects, however, I'm not sure how to deal with the situation, where only one object is present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I fill the remaining spaces in the label input data with vectors filled with 0? E.g:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[xmin,ymin,xmax,ymax],[0,0,0,0]...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or is there any better way? Thanks for help!&lt;/p&gt;&#xA;" OwnerUserId="4695" LastEditorUserId="14723" LastEditDate="2018-04-10T14:41:51.527" LastActivityDate="2018-04-10T14:41:51.527" Title="FIlling space with empty bounding box" Tags="&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5928" PostTypeId="1" AcceptedAnswerId="5932" CreationDate="2018-04-06T15:49:15.430" Score="6" ViewCount="133" Body="&lt;p&gt;I am trying to create a fixed-topology MLP from scratch (with C#), which can solve some simple problems, such as the XOR problem and &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt; classification. The network will be trained purely with &lt;strong&gt;genetic algorithms&lt;/strong&gt; instead of back-propagation.&lt;/p&gt;&#xA;&lt;p&gt;Here are the details:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Population size: 50&lt;/li&gt;&#xA;&lt;li&gt;Activation function: sigmoid&lt;/li&gt;&#xA;&lt;li&gt;Fixed topology&lt;/li&gt;&#xA;&lt;li&gt;XOR: 2 inputs, 1 output. Tested with different numbers of hidden layers/nodes.&lt;/li&gt;&#xA;&lt;li&gt;MNIST: &lt;span class=&quot;math-container&quot;&gt;$28*28=784$&lt;/span&gt; inputs for all pixels, will be either ON(1) or OFF(0). 10 outputs to represent digits 0-9&lt;/li&gt;&#xA;&lt;li&gt;Initial population will be given random weights between 0 and 1&lt;/li&gt;&#xA;&lt;li&gt;10 &amp;quot;Fittest&amp;quot; networks survive each iteration, and performs crossover to reproduce 40 offspring&lt;/li&gt;&#xA;&lt;li&gt;For all weights, mutation occurs to add a random value between -1 to 1, with a 5% chance&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;With 2 hidden layers of 4 and 3 neurons respectively, XOR managed to achieve 97-99.9% accuracy in around 100 generations. Biases were not used here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, trying out MNIST revealed a pretty glaring issue - the 784 inputs; a large increase of nodes compared to XOR, multiplied with weights and added up results in HUGE values of 50 to even 100, way beyond the typical domain range of the activation function (sigmoid).&lt;/p&gt;&#xA;&lt;p&gt;This just renders all layers' outputs as 1 or 0.99999-something, which breaks the entire network. Also, since this makes all individuals in a population extremely similar to one other, the genetic algorithm seems to have no clue on how to improve. The crossover will produce an offspring almost identical to its parents, and some lucky mutations are simply ignored by the sheer amount of other neurons!&lt;/p&gt;&#xA;&lt;p&gt;What can be a viable solution to this?&lt;/p&gt;&#xA;&lt;p&gt;It's my first time studying NNs, and this is really challenging.&lt;/p&gt;&#xA;" OwnerUserId="14833" LastEditorUserId="2444" LastEditDate="2021-02-06T17:38:26.353" LastActivityDate="2021-02-06T17:38:26.353" Title="How to solve the problem of too big activations when using genetic algorithms to train neural networks?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;data-preprocessing&gt;&lt;weights-initialization&gt;&lt;weight-normalization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5930" PostTypeId="5" CreationDate="2018-04-06T17:40:23.757" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Control theory in control systems engineering deals with the control of continuously operating dynamical systems in engineered processes and machines. The objective is to develop a control model for controlling such systems using a control action in an optimum manner without delay or overshoot and ensuring control stability.  To do this, a controller with the requisite corrective behaviour is required.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Control_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Control Theory (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;SEE ALSO:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ee.stanford.edu/research/control-and-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Control, Optimization &amp;amp; Machine Learning (Stanford)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://aa.stanford.edu/research/autonomous-systems-and-controls&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autonomous Systems and Controls (Stanford)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Digital_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digital Control (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Adaptive_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adaptive Control (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-06T17:40:23.757" LastActivityDate="2018-04-06T17:40:23.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5931" PostTypeId="4" CreationDate="2018-04-06T17:40:23.757" Score="0" Body="For questions about Control Theory, which is concerned with continuously operating dynamical systems in engineered processes and machines." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-06T17:40:23.757" LastActivityDate="2018-04-06T17:40:23.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5932" PostTypeId="2" ParentId="5928" CreationDate="2018-04-07T08:23:03.840" Score="4" Body="&lt;p&gt;Your inputs should stay in a low range. Ideally for neural networks, the inputs are normalised to mean 0, standard deviation 1. I suspect this applies equally well to GA-driven NNs as gradient-driven ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your weights should be both positive and negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, once trained, they tend to follow a certain size distribution. It helps if you start with values within that range. This is often called Xavier or Glorot initialisation. Basically, if your number of inputs to a layer is &lt;code&gt;n_inputs&lt;/code&gt; and number of outputs is &lt;code&gt;n_outputs&lt;/code&gt;, you will have &lt;code&gt;n_inputs * n_outputs&lt;/code&gt; weights, and if you initialise them with a random number generator, then you should use a multiplier something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = (rand() - 0.5) * sqrt(6/(n_inputs + n_outputs))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;em&gt;is&lt;/em&gt; an addition inside the square root, you don't take the total number of weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you are using a GA, you may want to use some form of clipping or normalisation to prevent mutations from drifting too far away from these useful weights. I'm not entirely sure what would be best, as usually I would use back propagation to train a large network. Potentially a maxnorm regularisation would help - choose a value (maybe even make it adjustable via mutation) that the norm of weight matrix in each layer should not exceed (separately for each layer, or one overall value, up to you), and if any learning step creates a network with too high a norm, scale it down. The L2 norm, &lt;code&gt;sqrt(sum_of_squared_weights)&lt;/code&gt; is usual choice for maxnorm regularisation, and this works well with backpropagation-based supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-04-07T08:23:03.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5934" PostTypeId="5" CreationDate="2018-04-07T11:18:01.163" Score="0" Body="&lt;p&gt;Voice or speaker recognition is the ability of a machine or program to receive and interpret or to understand and carry out spoken commands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.thoughtco.com/the-six-basic-animal-groups-4096604&quot; rel=&quot;nofollow noreferrer&quot;&gt;6 basic animal groups&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-10T20:01:02.827" LastActivityDate="2018-04-10T20:01:02.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5935" PostTypeId="4" CreationDate="2018-04-07T11:18:01.163" Score="0" Body="For questions about the AI/ML algorithms for performing the task of voice recognition in any animal (falls under the broad category of sound recognition, not to be confused with speech recognition)." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-10T20:07:55.037" LastActivityDate="2018-04-10T20:07:55.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5939" PostTypeId="1" CreationDate="2018-04-07T17:29:52.477" Score="12" ViewCount="375" Body="&lt;p&gt;With the growing ability to cheaply create fake pictures, fake soundbites, and fake video there becomes an increasing problem with recognizing what is real and what isn't. Even now we see a number of examples of applications that create fake media for little cost (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Deepfake&quot; rel=&quot;noreferrer&quot;&gt;Deepfake&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/FaceApp&quot; rel=&quot;noreferrer&quot;&gt;FaceApp&lt;/a&gt;, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, if these applications are used in the wrong way they could be used to tarnish another person's image. Deepfake could be used to make a person look unfaithful to their partner. Another application could be used to make it seem like a politician said something controversial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some techniques that can be used to recognize and protect against artificially made media? &lt;/p&gt;&#xA;" OwnerUserId="13088" LastEditorUserId="2444" LastEditDate="2019-09-07T12:15:37.043" LastActivityDate="2019-09-07T14:00:51.890" Title="What are some tactics for recognizing artificially made media?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;pattern-recognition&gt;&lt;deepfakes&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5941" PostTypeId="1" CreationDate="2018-04-07T21:06:37.043" Score="2" ViewCount="289" Body="&lt;p&gt;I am working on a project which maps to a variant of path finding problem. I am new to this area and I would be very grateful if you could give suggestions/ point to libraries for relevant algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simplified version of my problem statement is as follows- &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goal: On a 2D grid, starting from a fixed point reach the destination in exactly N steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Allowed actions: &#xA;1. At every position, you have a choice of up to three moves (i.e. straight, curve left, curve right). &#xA;2. You cannot collide with the path traveled so far (just like in the snake game).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dimension of the grid: N x N where N is between 100-1000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scalable: Later on, the problem will be scaled to have multiple such snakes going between different pairs of points on the grid. The ultimate goal is to get ALL snakes to reach their respective destinations in a fixed number of steps without any collisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt;&#xA;Essentially I have to find a fixed length path on a dynamically generated directed graph. Is there a better choice than a A* / greedy heuristic? Is it worth taking a Q-&#xA;learning approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A rudimentary one snake version written in python can be found here - &#xA;&lt;a href=&quot;https://github.com/pranavm1502/Curves&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Github link&quot;&gt;Github Link&lt;/a&gt;&#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="14851" LastEditorUserId="14851" LastEditDate="2018-04-07T22:42:18.823" LastActivityDate="2018-04-08T08:11:57.787" Title="Snake path finding variant : Algorithm choice" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;heuristics&gt;&lt;path-planning&gt;&lt;path-finding&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5942" PostTypeId="1" CreationDate="2018-04-07T22:43:16.867" Score="6" ViewCount="89" Body="&lt;p&gt;I have two classes in the training set: one that has images with a feature and the other of images without that feature.&#xA;Can there be a LOT more images with &quot;no feature&quot; so I can fit in all possible false positives?&lt;/p&gt;&#xA;" OwnerUserId="14731" LastActivityDate="2020-08-27T03:06:35.667" Title="Two data classes for a convolutional neural network, can one have a LOT more images for training than the other?" Tags="&lt;ai-design&gt;&lt;classification&gt;&lt;keras&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5943" PostTypeId="1" CreationDate="2018-04-08T00:54:44.760" Score="1" ViewCount="130" Body="&lt;p&gt;In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's two that I'm considering. An LSTM or augmenting the feature space. What's the difference between the two? The most obvious to me is that an LSTM is more expressive and can get superior accuracy if modelled properly. &lt;/p&gt;&#xA;" OwnerUserId="11566" LastActivityDate="2018-04-09T16:11:13.527" Title="Time Series: LSTM or Augmented Vector Space?" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5945" PostTypeId="5" CreationDate="2018-04-08T10:50:22.293" Score="0" Body="&lt;p&gt;Security is freedom from, or resilience against, potential harm (or other unwanted coercive change) from external forces. Beneficiaries (technically referents) of security may be persons and social groups, objects and institutions, ecosystems, and any other entity or phenomenon vulnerable to unwanted change by its environment.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-10T20:01:37.337" LastActivityDate="2018-04-10T20:01:37.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5946" PostTypeId="4" CreationDate="2018-04-08T10:50:22.293" Score="0" Body="For questions about the security measures against malicious algorithms, and security measures provided by AI algorithms in detecting, mitigating and preventing breaches and malicious activity." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-04-10T20:07:42.967" LastActivityDate="2018-04-10T20:07:42.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5947" PostTypeId="5" CreationDate="2018-04-08T11:01:17.893" Score="0" Body="&lt;p&gt;For more info, take a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Optical_character_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-12-13T12:01:34.327" LastActivityDate="2020-12-13T12:01:34.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5948" PostTypeId="4" CreationDate="2018-04-08T11:01:17.893" Score="0" Body="For questions about the application of AI/ML algorithms in the field of optical character recognition (OCR),  aka optical character reader (OCR), which is the mechanical or electronic conversion of images of typed, handwritten, or printed text into machine-encoded text." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-12-13T12:01:34.327" LastActivityDate="2020-12-13T12:01:34.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5949" PostTypeId="1" AcceptedAnswerId="5951" CreationDate="2018-04-08T14:38:12.313" Score="9" ViewCount="10795" Body="&lt;p&gt;What is the &lt;em&gt;fringe&lt;/em&gt; in the context of search algorithms?&lt;/p&gt;&#xA;" OwnerUserId="14862" LastEditorUserId="2444" LastEditDate="2019-07-06T20:15:49.433" LastActivityDate="2020-09-26T11:00:17.887" Title="What is the fringe in the context of search algorithms?" Tags="&lt;terminology&gt;&lt;search&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5950" PostTypeId="2" ParentId="5943" CreationDate="2018-04-08T14:47:33.303" Score="0" Body="&lt;p&gt;I just read this in a recent Bengio paper and it's pretty obvious. He says that there are zero differences between a short-term memory and an augmented feature space. However, if you want to capture long-term dependencies without blowing up the feature space, you'd want to use an LSTM because traditional approaches can't dynamically learn what to &quot;remember&quot;.&lt;/p&gt;&#xA;" OwnerUserId="11566" LastActivityDate="2018-04-08T14:47:33.303" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="5951" PostTypeId="2" ParentId="5949" CreationDate="2018-04-08T15:10:36.200" Score="10" Body="&lt;p&gt;In English, the &lt;em&gt;fringe&lt;/em&gt; is (also) defined as &lt;em&gt;the outer, marginal, or extreme part of an area, group, or sphere of activity&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of AI search algorithms, the state (or search) space is usually represented as a graph, where nodes are states and the edges are the connections (or actions) between the corresponding states. If you're performing a tree (or graph) search, then the set of all nodes &lt;em&gt;at the end&lt;/em&gt; of all visited paths is called the &lt;em&gt;fringe&lt;/em&gt;, &lt;em&gt;frontier&lt;/em&gt; or &lt;em&gt;border&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the picture below, the grey nodes (the lastly visited nodes of each path) form the fringe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mtJQj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mtJQj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The video &lt;a href=&quot;https://classroom.udacity.com/courses/cs271/lessons/48678875/concepts/486525390923&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example Route Finding&lt;/a&gt; by Peter Norvig also gives some intuition behind this concept.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-07-06T20:25:24.883" LastActivityDate="2019-07-06T20:25:24.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5953" PostTypeId="2" ParentId="5899" CreationDate="2018-04-08T18:39:03.477" Score="1" Body="&lt;p&gt;From Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In computing, tree data structures, and game theory, the branching factor is the number of children at each node, the &lt;strong&gt;outdegree&lt;/strong&gt;. If this value is not uniform, an average branching factor can be calculated.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Outdegree&lt;/strong&gt; meaning - In case of directed graphs, number of edges going into a node is known as in-degree of the corresponding node and number of edges coming out of a node is known as out-degree of the corresponding node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You forgot the &lt;strong&gt;outdegree&lt;/strong&gt; part. In AI we generally draw directed graphs from one state to another, and &lt;strong&gt;outdegree&lt;/strong&gt; is the number of paths leaving a particular node. In your graph direction is not given. Also your graph is not symmetrical, but you can still find out the branching factor (with a little difficulty) of non-symmetrical directed graphs as given &lt;a href=&quot;https://stackoverflow.com/questions/47789400/how-to-find-the-branching-factor-of-a-tree?utm_medium=organic&amp;amp;utm_source=google_rich_qa&amp;amp;utm_campaign=google_rich_qa&quot;&gt;here&lt;/a&gt;. So technically  your conclusion is correct about leaf nodes not being counted (assuming they are the last state from which no other state can be reached - dead end). Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-06-09T15:06:49.677" LastActivityDate="2018-06-09T15:06:49.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5954" PostTypeId="2" ParentId="5942" CreationDate="2018-04-08T21:21:02.373" Score="1" Body="&lt;p&gt;Your question is very general so therefore, in this case, my answer will be too: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is &quot;sometimes&quot;: it depends on the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There can be a lot more images in one class than the other, and you can still get reasonable results. It highly depends on how much data you have of the &quot;feature class&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the case, we say that the classes are heavily unbalanced, and you need to do &quot;class balancing&quot;. You do not want to do overfitting on this one class, and preferably you want the feature-class to be the biggest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach for CNNs is to use &quot;dropout&quot;. Well, for CNN's you can go a bit further: you can remove parts of the image to generate &quot;new&quot; images. This way you prevent overfitting of the &quot;feature&quot; class, whilst generating more data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that training &lt;em&gt;all possible&lt;/em&gt; false positives is impossible without overfitting the network somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps, and give you some google pointers :)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Just FYI: You basically, in the tech term, want to know whether it works to do binary CNN classification using a heavily imbalanced dataset. &lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2018-05-31T22:44:13.127" LastActivityDate="2018-05-31T22:44:13.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5955" PostTypeId="1" AcceptedAnswerId="5957" CreationDate="2018-04-08T21:34:26.637" Score="19" ViewCount="1892" Body="&lt;p&gt;I've heard multiple times that &quot;Neural Networks are the best approximation we have to model the human brain&quot;, and I think it is commonly known that Neural Networks are modelled after our brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I strongly suspect that this model has been simplified, but how much? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much does, say, the vanilla NN differ from what we know about the human brain? Do we even know?&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="2444" LastEditDate="2021-01-20T00:42:01.060" LastActivityDate="2021-01-20T11:14:34.737" Title="How are Artificial Neural Networks and the Biological Neural Networks similar and different?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;topology&gt;&lt;human-brain&gt;&lt;neural-circuits&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5956" PostTypeId="2" ParentId="5955" CreationDate="2018-04-09T02:41:03.660" Score="5" Body="&lt;p&gt;They are not close, not anymore!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Artificial] Neural Nets &lt;strong&gt;&lt;em&gt;vaguely inspired&lt;/em&gt;&lt;/strong&gt; by the connections we previously observed between the neurons of a brain. Initially, there probably was an intention to develop ANN to approximate biological brains. However, the modern working ANNs that we see their applications in various tasks are not designed to provide us a functional model of an animal brain. As far as I know, there is no study claiming they have found something new in a biological brain by looking into the connections and weight distributions of let's say a CNN or RNN model.&lt;/p&gt;&#xA;" OwnerUserId="12853" LastActivityDate="2018-04-09T02:41:03.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5957" PostTypeId="2" ParentId="5955" CreationDate="2018-04-09T08:54:34.713" Score="11" Body="&lt;p&gt;We all know that artificial neural networks (ANNs) are &lt;strong&gt;inspired by&lt;/strong&gt; but most of them are only &lt;strong&gt;loosely based on&lt;/strong&gt; the biological neural networks (BNNs).&lt;/p&gt;&#xA;&lt;p&gt;We can analyze the differences and similarities between ANNs and BNNs in terms of the following components.&lt;/p&gt;&#xA;&lt;h3&gt;Neurons&lt;/h3&gt;&#xA;&lt;p&gt;The following diagram illustrates a biological neuron (screenshot of an image from &lt;a href=&quot;https://opentextbc.ca/introductiontopsychology/chapter/3-1-the-neuron-is-the-building-block-of-the-nervous-system/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VSOKk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VSOKk.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The following one illustrates a typical artificial neuron of an ANN (screenshot of figure 1.14 of &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/chapter/K1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ASLAg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ASLAg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Initialization&lt;/h3&gt;&#xA;&lt;p&gt;In the case of an ANN, the initial state and weights are assigned randomly. While for BNNs, the strengths of connections between neurons and the structure of connections don't start as random. The initial state is genetically derived and is the byproduct of evolution.&lt;/p&gt;&#xA;&lt;h3&gt;Learning&lt;/h3&gt;&#xA;&lt;p&gt;In BNN, learning comes from the interconnections between myriad neurons in the brain. These interconnections change configuration when the brain experiences new stimuli. The changes result in new connections, strengthening of existing connections, and removal of old and unused ones.&lt;/p&gt;&#xA;&lt;p&gt;ANNs are trained from scratch usually using a fixed topology (remember topology changes in case of BNNs), although the topology of ANN can also change (for example, take a look at NEAT or some continual learning techniques), which depends on the problem being solved. The weights of an ANN are randomly initialized and adjusted via an optimization algorithm.&lt;/p&gt;&#xA;&lt;h3&gt;Number of neurons&lt;/h3&gt;&#xA;&lt;p&gt;Another difference (although this difference is always smaller) is in the number of neurons in the network. A typical ANN consists of hundreds, thousands, millions, and, in some exceptional (e.g. GPT-3), billions of neurons. The BNN of the human brain consists of billions. This number varies from animal to animal.&lt;/p&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;You can find more information &lt;a href=&quot;https://news.sophos.com/en-us/2017/09/21/man-vs-machine-comparing-artificial-and-biological-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://ai.stackexchange.com/q/258/2444&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-20T11:14:34.737" LastActivityDate="2021-01-20T11:14:34.737" CommentCount="1" CommunityOwnedDate="2021-01-20T00:34:38.193" ContentLicense="CC BY-SA 4.0" />
  <row Id="5960" PostTypeId="1" AcceptedAnswerId="5961" CreationDate="2018-04-09T11:37:05.947" Score="2" ViewCount="515" Body="&lt;p&gt;The problem to solve is non-linear regression of a non-linear function. My actual problem is to model the function &quot;find the max over many quadratic forms&quot;: &lt;code&gt;max(w.H.T * Q * w)&lt;/code&gt;, but to get started and to learn more about neural networks, I created a toy example for a non-linear regression task, using Pytorch. &#xA;The problem is that the network never learns the function in a satisfactory way, even though my model is quite large with multiple layers (see below). Or is it not large enough or too large? How can the network be improved or maybe even simplified to get a much smaller training error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I experimented with different network architectures, but the result is never satisfactory. Usually, the error is quite small within the input interval around 0, but the network is not able to get good weights for the regions at the boundary of the interval (see plots below). The loss does not improve after a certain number of epochs.  I could generate even more training data, but I have not yet understood completely, how the training can be improved (tuning parameters such as batch size, amount of data, number of layers, normalizing input (output?) data,  number of neurons, epochs, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My neural network has 8 layers with the following number of neurons: &lt;code&gt;1, 80, 70, 60, 40, 40, 20, 1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the moment, I do not care too much about overfitting, my goal is to understand, why a certain network architecture/certain hyperparameters need to be chosen. Of course, avoiding overfitting at the same time would be a bonus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am especially interested in using neural networks for regression tasks or as function approximators. In principle, my problem should be able to be approximated to arbitrary accuracy by a single layer neural network, according to the universal approximation theorem, isn’t this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oqXXN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oqXXN.png&quot; alt=&quot;Difference between the trained model and original data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l2U6j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l2U6j.png&quot; alt=&quot;Loss value vs iterations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sHAct.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sHAct.png&quot; alt=&quot;Difference between the trained model and original data &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14873" LastEditorUserId="2444" LastEditDate="2020-01-01T13:19:23.967" LastActivityDate="2020-10-10T16:11:22.370" Title="Why isn't my model learning satisfactorily?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;pytorch&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5961" PostTypeId="2" ParentId="5960" CreationDate="2018-04-09T11:54:37.387" Score="1" Body="&lt;p&gt;Neural networks learn badly with large input ranges. Scale your inputs to a smaller range e.g. -2 to 2, and convert to/from this range to represent your function interval consistently.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-04-09T11:54:37.387" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5962" PostTypeId="1" CreationDate="2018-04-09T13:23:36.900" Score="4" ViewCount="100" Body="&lt;p&gt;We store supplier information for many companies. Each supplier database is logically separated, making it possible for a supplier to be common across logical separations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, we have analysts go through the system manually to try identify common suppliers by name and address.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a problem for a certain domain of AI? If so, what is the right hammer for this nail?&lt;/p&gt;&#xA;" OwnerUserId="14875" LastActivityDate="2019-09-27T17:02:15.300" Title="Whats the correct technique/technology to identify duplicate data" Tags="&lt;ai-design&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5964" PostTypeId="1" AcceptedAnswerId="6043" CreationDate="2018-04-09T16:10:35.577" Score="6" ViewCount="280" Body="&lt;p&gt;YouTube has a huge amount of videos, many of which also containing various spoken languages. This would presumably provide something like the data that a &quot;challenged&quot; baby would experience - &quot;challenged&quot; meaning a baby without arms or legs (unfortunately many people are born that way). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this not allow unsupervised learning in a deep learning system that has both vision and audio capabilities? The neural network would presumably learn correlations between words and images, and could perhaps even learn rudimentary language skills, all without human supervision. I believe that the individual components to do this already exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this been tried, and if not, why?&lt;/p&gt;&#xA;" OwnerUserId="5911" LastEditorUserId="47" LastEditDate="2018-04-15T00:46:58.770" LastActivityDate="2018-04-15T00:46:58.770" Title="Has anybody tried unsupervised deep learning from youtube videos?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;computer-vision&gt;&lt;unsupervised-learning&gt;&lt;datasets&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5965" PostTypeId="2" ParentId="5943" CreationDate="2018-04-09T16:11:13.527" Score="1" Body="&lt;p&gt;LSTM is a neural network which learns for an input x an output y. In additional to CNNs or MLPs it considers a hiddenstate h (which is influenced by prvious inputs) when your next input x is feed into the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Augmenting the feature Space is a technique which you do previous training your LSTM (to augment your data set in order to generatre more data and let the LSTM better generalize to new data). In the field of image recognition you can rotate your images by 40 degree to generate a new one. This process is known as data augmentation. Such methods also appliable to time series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary: first, you start with augmenting your input feature space in order to improve prediction accuracy and then training your LSTM with the augmeneted training data set.&lt;/p&gt;&#xA;" OwnerUserId="13295" LastActivityDate="2018-04-09T16:11:13.527" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5966" PostTypeId="1" CreationDate="2018-04-09T16:18:15.580" Score="1" ViewCount="142" Body="&lt;p&gt;I wanted to use the visualization of the activation maximization of the filters that is described in the following keras tutorial/blog:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.keras.io/how-convolutional-neural-networks-see-the-world.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.keras.io/how-convolutional-neural-networks-see-the-world.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to know what is the intention behind the decision that filters that produce a loss &amp;lt;= 0 are skipped. I know for 0 that would be reasonable since their would be no gradient flowing then (I think) but what about negative values? And is it also reasonable to use the mean of the outputs of the filters as a loss? What if there are weights of a filter that have high negative and positive numbers. Would that be a problem?&lt;/p&gt;&#xA;" OwnerUserId="14881" LastEditorUserId="14881" LastEditDate="2018-04-09T18:53:50.353" LastActivityDate="2018-04-09T18:53:50.353" Title="Questions regarding keras activation maximization visualization" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5967" PostTypeId="2" ParentId="5538" CreationDate="2018-04-09T16:25:01.163" Score="0" Body="&lt;p&gt;I would model this data as a 3d-tensor (user,timestep,features) [organization depends on which DL Framework you use] for your input data. Also for the output data is a 3d-tensor (user,timestep,result) appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step would ne tp train a LSTM or CNN model to predict the result (what requires a lot of data)(Would be my first choice). If you have less data, try out logistic regression as supposed by the other answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="13295" LastActivityDate="2018-04-09T16:25:01.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5969" PostTypeId="1" CreationDate="2018-04-09T20:06:29.793" Score="1" ViewCount="41" Body="&lt;p&gt;Are decision trees able to be used with time-related data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read that decision trees are based on matrices and that ARRAYS of input matrices can be used to factor in time however I can't find an example of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say for example, I'm monitoring the progress of students taking exams. Each day I ask them questions related to their mental state (fatigued, positivity, ability to concentrate, expectations for coming exam, confidence, etc). I have twenty days worth of questions. Day 1 for student A may see them studying for an exam the following day, while Day 1 for student B may see them actually doing the exam. There will be a relation between student's fatigue (for example) and the results they give the following day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The examples when provided as input to a matrix will be used to show that IF on any given day, the student has an exam, and has breakfast, and does x,y,z THAT day then the outcome will be y. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, short of encoding &quot;had exam previous day&quot; and &quot;had exam two days ago&quot; for each day, I can't see how I can include time dependency in decision trees.&lt;/p&gt;&#xA;" OwnerUserId="12726" LastActivityDate="2018-04-09T20:06:29.793" Title="How to factor time into decision trees?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5970" PostTypeId="1" CreationDate="2018-04-09T21:02:23.180" Score="9" ViewCount="5927" Body="&lt;p&gt;I'm studying reinforcement learning. It seems that &quot;state&quot; and &quot;observation&quot; mean exactly the same thing. They both capture the current state of the game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a difference between the two terms? Is the observation maybe the state after the action has been taken?&lt;/p&gt;&#xA;" OwnerUserId="11566" LastEditorUserId="2444" LastEditDate="2018-11-16T20:11:53.853" LastActivityDate="2018-11-16T20:11:53.853" Title="What is the difference between an observation and a state in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5971" PostTypeId="2" ParentId="5970" CreationDate="2018-04-09T21:36:58.927" Score="10" Body="&lt;p&gt;Sometimes observation and state overlap completely, which is convenient. However, there is no reason to expect it in all cases, and that's where interesting problems occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement learning theory is based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;noreferrer&quot;&gt;Markov Decision Processes&lt;/a&gt;. This leads to a &lt;em&gt;formal&lt;/em&gt; definition of state. Most importantly, the state must have the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;noreferrer&quot;&gt;Markov property&lt;/a&gt;. Which means that for RL to work according to theory, that knowing the state means that you know &lt;em&gt;everything knowable&lt;/em&gt; that could determine the response of the environment to a specific action. Everything that remains must be purely stochastic and unknowable in principle until after the action is resolved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Systems like deterministic or probability-driven games, and computer-controlled simulations can be designed to have easily observable states that have this property. Games with this trait are often called &quot;games of perfect information&quot;, although you may have unknown information, provided it is revealed in a purely stochastic manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, real world interactions contain far too much detail for any observation to be a true state with the Markov property. For instance, consider the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverted_pendulum&quot; rel=&quot;noreferrer&quot;&gt;inverted pendulum environment&lt;/a&gt;, a classic RL toy problem. A real inverted pendulum would behave differently depending on its temperature, which could vary along its length. The joint and actuators might be sticky. Rotations and movement will alter temperature and friction, etc. However, a RL agent will typically only consider current motion and position of the trolley and pendulum. In this case, the observation of 4 traits is usually good enough, and a state based on this &lt;em&gt;almost&lt;/em&gt; has the Markov property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also problems where observations are not enough to make usable state data for a RL system. The Deep Mind Atari DQN paper had examples of a couple of these. The first example is that a single frame lost data about motion. This could be addressed by taking four consecutive frames and combining them to make a single state. It could be argued that each frame is an observation, and that four observations had to be combined in order to construct a more useful state (although this could be put aside as just semantics).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second example in Atari DQN is that the pixel observations did not include data that the game was tracking but that was not visible on screen. Games with large scrolling maps are a weakness of the Atari-playing DQN, because its state has no memory of screens other than the four used for movement. An example of such a game, where Deep Mind's player did much worse than a human player is Montezuma's Revenge, where to progress it is necessary to remember some off-screen locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are ways to address knowledge that there is unobserved but relevant state in a problem. The general framework for describing the problem is &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;noreferrer&quot;&gt;Partially Observable Markov Decision Processes (POMDPs)&lt;/a&gt;. Workable solutions include adding explicit memory or &quot;belief state&quot; to the state representation, or using a system such as RNN in order to internalise the learning of a state representation driven by a sequence of observations.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-04-09T21:36:58.927" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5972" PostTypeId="2" ParentId="5840" CreationDate="2018-04-09T21:53:49.080" Score="2" Body="&lt;p&gt;There are two different problems described in the linked question and your question: optimization and learning.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Optimization&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you are asking about optimization (the second linked question: &lt;a href=&quot;https://ai.stackexchange.com/questions/4859/search-minimum-value-with-learning-machine-algorithm/4860#4860&quot;&gt;Search minimum value with learning machine algorithm&lt;/a&gt;) you can have 3 different approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;analytical approach&lt;/li&gt;&#xA;&lt;li&gt;numerial methods&lt;/li&gt;&#xA;&lt;li&gt;metaheuristics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As you suggest, it is usually better to try them from the first to the last one. It is common that the first approach is unfeasible for optimizing for target function, but very often you can use either mathematical optimization for some specific classes of problems (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear&lt;/a&gt;/&lt;a href=&quot;https://en.wikipedia.org/wiki/Quadratic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;quadratic&lt;/a&gt; programming) or iterative methods (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;conjugate gradient method&lt;/a&gt;). Only after considering this approaches it makes sense for the third class of approaches, genetic algorithms being a notable example, which is often classified as an AI approach.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Learning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you are asking about learning, then the first linked question (&lt;a href=&quot;https://ai.stackexchange.com/questions/5838/ideas-on-how-to-make-a-neural-net-learn-how-to-split-sequence-into-sub-sequences&quot;&gt;Ideas on how to make a neural net learn how to split sequence into sub sequences&lt;/a&gt;) seems to be intended as an example. However it doesn't make clear what the problem is, as the target function seems to be obvious, so no learning is needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case it also makes sense to first try to pin down the problem mathematically and resort to machine learning if it is impossible and if you have the data (input/output examples).&lt;/p&gt;&#xA;" OwnerUserId="4880" LastActivityDate="2018-04-09T21:53:49.080" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5973" PostTypeId="2" ParentId="5838" CreationDate="2018-04-10T02:54:48.923" Score="0" Body="&lt;p&gt;I guess, supervised learning should work rather well: You'd feed the network with a fixed substring and it'd determine if the middle character is the first letter of a word, or a last one, or neither or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;code&gt;2*n+1&lt;/code&gt; inputs (fed e.g., with the string &quot;ingsits&quot;) should output a 1 on the output determining if the middle letter (here: &quot;s&quot;) is the first one of a word and a 0 on the output determining if it's the last one (taken from &quot;Thekingsitsthere&quot;). Each input character should probably be 1 hot encoded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd probably want to use more context characters than in my example. OTOH you can use a simple MLP with no temporal complications. It'll never get perfect as it's impossible, but it get pretty close.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning unsupervised learning I'm skeptical...&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-04-10T02:54:48.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5974" PostTypeId="2" ParentId="5838" CreationDate="2018-04-10T03:24:08.583" Score="0" Body="&lt;p&gt;(NOTE: I think it will be easier to do it without ANNs...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you insist:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;convert the sequence into a fixed-size vectors.&lt;/li&gt;&#xA;&lt;li&gt;push trough a 2-5 1D-convolution layer with 1 neuron dense layer at the end (sigmoid activation) and another K-points detector for getting the sequence breakage points&lt;/li&gt;&#xA;&lt;li&gt;create a training set - to find the break-points (12, 23, 34 ...) in the sequence.&lt;/li&gt;&#xA;&lt;li&gt;train a detector with SGD to find these break-points. - loss functions: cross_entropy. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then, it should learn to find the breakage points, and based on this you can easily split the sequence.&lt;/p&gt;&#xA;" OwnerUserId="3250" LastActivityDate="2018-04-10T03:24:08.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5975" PostTypeId="2" ParentId="5899" CreationDate="2018-04-10T03:37:51.487" Score="1" Body="&lt;p&gt;I'd say that the leaves &lt;em&gt;per se&lt;/em&gt; count, too, but only if they're real leaves, like e.g., checkmate positions in chess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a node has really no children and no further calculation is needed. Unlike with nodes which weren't expanded yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note that always counting the leaves &lt;a href=&quot;https://math.stackexchange.com/a/454646/14435&quot;&gt;provably&lt;/a&gt; leads to &lt;code&gt;(n-1)/n&lt;/code&gt; for every &lt;code&gt;n&lt;/code&gt;-node thee!&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-04-10T03:37:51.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5976" PostTypeId="2" ParentId="5111" CreationDate="2018-04-10T06:00:28.730" Score="1" Body="&lt;p&gt;Not an pro but I think I know some answers to your questions. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If we train our classifier, wouldn't the prediction boxes be close to the ground truth labels as training progresses&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think that's what YOLO v1 did. According to Andrew NG's video the bounding boxes are introduced to solve multiple objects inside the same grid cell. And according to this &lt;a href=&quot;https://medium.com/@vivek.yadav/part-1-generating-anchor-boxes-for-yolo-like-network-for-vehicle-detection-using-kitti-dataset-b2fe033e5807&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; anchor boxes assignment ensures that an anchor box predicts ground truth for an object centered at its own grid center, and not a grid cell far away (like YOLO may)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;what are those numbers representing anchor boxes representing? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They are just width and height (shape). In YOLO v2 it is used to compute IOU assuming all boxes are placed at the same location (ignoring the location), you could think of it just tries to match the shape. And it uses (1-IOU) as the distance when applying the K-means clustering.&lt;/p&gt;&#xA;" OwnerUserId="14897" LastActivityDate="2018-04-10T06:00:28.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5977" PostTypeId="2" ParentId="5838" CreationDate="2018-04-10T07:22:28.453" Score="0" Body="&lt;p&gt;Another approach could be to predict the class of a sequence and not the break point. Assuming that each sequence is part of a class, you can use a LSTM. Inputing the multiple sequences (111100002222 ) and let predict the class for each sequence (c1,c1,c1,c1,c0,c0,c0,c0,c2,c2,c2,c2)&lt;/p&gt;&#xA;" OwnerUserId="13295" LastActivityDate="2018-04-10T07:22:28.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5978" PostTypeId="2" ParentId="5107" CreationDate="2018-04-10T07:46:51.280" Score="2" Body="&lt;p&gt;The problem is in your diagram. Here are the steps to get to a 5x5 receptive field. Here is your diagram, redone slightly:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lEe45.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lEe45.jpg&quot; alt=&quot;The 3x3 input field is summed up (by the weights), and passed through a nonlinear activation function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the new unit takes a weighted sum of the 9 pixels in the input, and then applies a rectified linear nonlinearity. Now, there are more of these, creating three new numbers computed from that part of the image. Each one slides over by one pixel:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jl3lW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jl3lW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We repeat this process going down three pixels as well, and then finally, we have a new 3x3 input field:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/68ib4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/68ib4.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the new unit on the right now gets input from a 5x5 input field. I hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="14901" LastActivityDate="2018-04-10T07:46:51.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5979" PostTypeId="1" AcceptedAnswerId="6562" CreationDate="2018-04-10T13:33:34.427" Score="2" ViewCount="768" Body="&lt;p&gt;I am trying to do &lt;strong&gt;3d image deconvolution&lt;/strong&gt; using &lt;strong&gt;convolution neural network&lt;/strong&gt;. But I cannot find many famous 3d CNNs. Can anyone point out some for me?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background: I am using PyTorch, but any language is OK. What I want to know most is the network structure. I can't find papers on this topic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links to research papers would be especially appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14907" LastEditorUserId="2444" LastEditDate="2020-06-12T14:05:40.130" LastActivityDate="2020-06-12T14:09:47.167" Title="Which neural network architectures are there that perform 3D convolutions?" Tags="&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5980" PostTypeId="2" ParentId="5869" CreationDate="2018-04-10T14:22:12.333" Score="1" Body="&lt;p&gt;Many AI libraries allow the ability to feed them an Image, but others have to be modified to allow this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disregarding the information above, as it is just a notification, you can do this in many ways. Depending on the library you use, different methods give varying accuracy results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How complex do you want the NN to be?&lt;/li&gt;&#xA;&lt;li&gt;Is this going to be commercial, or private?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For any of the answers above, you will have to find the best method that works for your needs. The more this matters to the world, the more complex it will have to be.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-04-10T14:22:12.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5981" PostTypeId="1" CreationDate="2018-04-10T15:34:40.183" Score="4" ViewCount="673" Body="&lt;p&gt;For a school project, I would like to investigate a paper on either reinforcement learning or computer vision. I am particularly interested in DQN, RNNs, CNNs or LSTMs. I would eventually like to implement any of these. However, I also need to take into account the computing resources required to train and analyse any of these algorithms. I understand that, in computer vision, the data sets can be quite large, but I am not so sure regarding the resources needed to implement and train a typical state-of-the-art RL algorithm (like DQN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would a &quot;standard PC&quot; be able to run any of these algorithms decently to achieve some sort of analysis/results? &lt;/p&gt;&#xA;" OwnerUserId="14913" LastEditorUserId="2444" LastEditDate="2019-02-15T16:18:40.473" LastActivityDate="2019-02-15T16:21:51.720" Title="What are the minimum computing resources needed to train a machine learning algorithm?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;computer-vision&gt;&lt;hardware-evaluation&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5982" PostTypeId="1" CreationDate="2018-04-10T18:41:20.360" Score="5" ViewCount="604" Body="&lt;p&gt;I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like &quot;XXX12345&quot;, etc).  Ideally, the string it will be guessing will be somewhere in the block of text, but sometimes it won't be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been struggling where to begin on this or if I am even going in the right direction for considering Machine/Deep learning to try to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Help!&lt;/p&gt;&#xA;" OwnerUserId="14918" LastEditorUserId="14918" LastEditDate="2018-04-11T13:30:59.753" LastActivityDate="2019-01-07T11:00:45.167" Title="Use Machine/Deep Learning to Guess a String" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;text-summarization&gt;" AnswerCount="4" CommentCount="8" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5984" PostTypeId="2" ParentId="4677" CreationDate="2018-04-10T20:04:24.937" Score="1" Body="&lt;p&gt;You are using two optimisers here: Stochastic Gradient Descent (SGD) and Adam (which is a more complex variant of SGD).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the &quot;Stochastic&quot; part means that it's random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stochastic gradient descent works by taking a smaller &lt;em&gt;random&lt;/em&gt; part of the training data, called &quot;mini batch&quot;, and back propagates (trains) on this. Doing this until the entire dataset is processed once is often called one epoch*. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how gradient descent works in a nutshell: Imagine you're going down a U-shaped hill. You're pretty far down in the U-shape, and you want to go further down by jumping. You figure out what direction is &quot;down&quot; for you: and then you jump. But darn it: you jumped too far and you ended up further up on the other side of the U!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is just a simple example. You are probably working in WAY bigger dimensions, which complicates this analogy a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway this results in the effect that the loss might go up from time to time when you train another epoch. If you are training a lot of epochs and the loss keeps going up, you should check the learning rate (which basically decides how big a &quot;jump&quot; is).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps :)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*: There are other ways of defining an epoch, but it all goes in variants of this.&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2018-05-31T22:09:15.300" LastActivityDate="2018-05-31T22:09:15.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5985" PostTypeId="2" ParentId="5838" CreationDate="2018-04-10T20:32:47.903" Score="0" Body="&lt;p&gt;How about this ?&lt;br/&gt;&lt;br/&gt;&#xA;&lt;strong&gt;1 - Learn all the basic building blocks of possible sub-sequence&lt;/strong&gt;&lt;br/&gt;&#xA;In our words sequence example, that would correspond to &lt;a href=&quot;https://en.wikipedia.org/wiki/Phoneme&quot; rel=&quot;nofollow noreferrer&quot;&gt;phonemes&lt;/a&gt;.&lt;br/&gt;&#xA;(&lt;em&gt;I'm guessing that this step can even be done using unsupervised learning.)&lt;/em&gt;&lt;br/&gt;&#xA;So in the following example :&#xA;&lt;strong&gt;Hello Laurie&lt;/strong&gt;, we would have learned 3 phonemes : &lt;strong&gt;HE&lt;/strong&gt;, &lt;strong&gt;LO&lt;/strong&gt;, &lt;strong&gt;RI&lt;/strong&gt;.&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2- Learn all subsequence as sequences of 'building blocks'&lt;/strong&gt;&lt;br/&gt;&#xA;Using a &lt;a href=&quot;https://arxiv.org/abs/1402.3511&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClockWorkRNN&lt;/a&gt; with timesteps of interval +1 with, let's say, 10-15 timestep (groups), that is fed the next '&lt;em&gt;phoneme id&lt;/em&gt;' in the sequence, we would have a space large enough to record most &lt;em&gt;words&lt;/em&gt; (Obviously, the number of timesteps should be size of the biggest word).&lt;br/&gt;&#xA;This is the subsequences memory RNN.&lt;br&gt;&#xA;Its sole purpose is to remember subsequences.&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, i'm really brainstorming here , taking a very wild guess, but what if : &lt;br/&gt;&#xA;After training this RNN to a satisfying error rate, we check if the output of the RNN is very different to the next input for a couple of timesteps.&lt;br/&gt;&#xA;In other word, we see if the neural network has been able to '&lt;em&gt;guess&lt;/em&gt;' the next building block of the subsequence.&lt;br/&gt;&#xA;If not, then its a &lt;strong&gt;point of interest&lt;/strong&gt; , because there is not a lot of possibilities as of why this would happend : the only one I see is &lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - The RNN is currently receiving another word, thus making this timestep a sub-sequence '&lt;em&gt;break point&lt;/em&gt;'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys see any points that could prove this theory wrong ?&lt;/p&gt;&#xA;" OwnerUserId="13038" LastActivityDate="2018-04-10T20:32:47.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5986" PostTypeId="2" ParentId="5840" CreationDate="2018-04-10T21:09:39.480" Score="1" Body="&lt;p&gt;&lt;em&gt;fwiw,&lt;/em&gt; with the basic, non-trivial M-game, I have no doubt that ALphaZero could tear through any human player alive in very short order. I hope that people will start experimenting with that, especially on m^n(m^n) where m &gt; 3 and n &gt; 2 to see how they hold up.  Problem is, once you expand past n &gt; 3 it gets very difficult for humans to play. This leads to a condition where performance of an NN on higher order M can only realistically be evaluated against other algorithms.  In this context, it seems worthwhile to develop a general, classical algorithm that can evaluate any order M, regardless of efficacy of tree search in relation to the problem size, with the understanding that decision making is never presumed optimal until the gametree becomes tractable.  This carries an an assumption of the same general strength across all M for the classical algorithm, because the expansion of &lt;em&gt;m&lt;/em&gt; or &lt;em&gt;n&lt;/em&gt; do alter the core heuristics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the practical standpoint, as a product designed for mobile with no assumption of connectivity, it doesn't make sense to start integrating NNs until lowest-common-denominator mobile devices have sufficient resources.  The issue of package size is also important in this context--the classical algorithms require a trivial amount of code and volume. Most importantly, using classical algorithms formed of sets of heuristics and parameters allows recombination of functions to produce myriad automata of varying degrees of strength. (This can be easily accomplished by altering the size of tree search algorithms, but may only be relevant in determining which heuristics perform better under tree search restrictions.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, because M-games provide an array precise metrics, it may be worthwhile to develop core heuristic function based on human reasoning.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-04-10T21:09:39.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5987" PostTypeId="1" CreationDate="2018-04-10T21:30:08.717" Score="2" ViewCount="45" Body="&lt;p&gt;I want to create a neural network and train it on some data, however I want to be able to create a new model without retraining it from the start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example, I have 1000 data points in my training data&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;model - trained on 0-99&lt;/li&gt;&#xA;&lt;li&gt;model - trained on 1-100&lt;/li&gt;&#xA;&lt;li&gt;model - trained on 2-101&lt;/li&gt;&#xA;&lt;li&gt;and so forth&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So I'm wondering if I can use the first model to train the second model, essentially forgetting the first data point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can view it as a sliding window over the 1000 data points, sliding one data point to the right for each new model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make sense?&#xA;Is there any easy way to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="14923" LastActivityDate="2018-04-10T21:30:08.717" Title="Shifting training data" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="5989" PostTypeId="2" ParentId="5919" CreationDate="2018-04-11T05:28:32.173" Score="1" Body="&lt;p&gt;I changed the layer from tf.contrib.rnn.LSTMBlockCell to tf.contrib.rnn.LayerNormBasicLSTMCell. Then the gradients become large enough to influence the network.&lt;/p&gt;&#xA;" OwnerUserId="14816" LastActivityDate="2018-04-11T05:28:32.173" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5990" PostTypeId="1" CreationDate="2018-04-11T08:06:52.587" Score="8" ViewCount="1421" Body="&lt;p&gt;There are two textbooks that I most love and am most afraid of in the world: &lt;em&gt;Introduction to Algorithms by Cormen&lt;/em&gt; et al. and &lt;em&gt;Artificial Intelligence: A Modern Approach by Norvig&lt;/em&gt; et al. I have started the &quot;AI: A Modern Approach&quot; more than once, but the book is so dense and full of theory that I get discouraged after a couple of weeks and stop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am looking for a similar AI book but with an equal emphasis on theory and practice.&lt;/strong&gt; Some examples of what I am looking for:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The Elements of Statistical Learning by Tibshirani et al. &lt;strong&gt;(detailed theory)&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An Introduction to Statistical Learning: With Applications in R by&#xA;Tibshirani et al. &lt;strong&gt;(theory+practical)&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Digital Image Processing by Gonzalez et al. &lt;strong&gt;(detailed theory)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Digital Image Processing Using MATLAB by Gonzalez et al.&#xA;&lt;strong&gt;(theory+practical)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14933" LastEditorUserId="2444" LastEditDate="2021-01-16T19:18:51.570" LastActivityDate="2021-01-28T00:17:59.187" Title="What are some alternatives to the book &quot;Artificial Intelligence: A Modern Approach&quot;?" Tags="&lt;reference-request&gt;&lt;norvig-russell&gt;&lt;books&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5991" PostTypeId="1" AcceptedAnswerId="5992" CreationDate="2018-04-11T08:59:05.170" Score="1" ViewCount="734" Body="&lt;p&gt;I'm currently reading &lt;a href=&quot;https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; explanation of convolutional neural networks and there's a part around strides that I don't quite understand. I'm just starting with this, so I apologize if this is a really basic question. But I'm trying to develop an understanding and some of these images have thrown me off.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, in this image&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zI3nb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zI3nb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The stride has been increased to 2 and it's using a 3x3 filter (represented by the red, green and blue outline squares in the first picture)&lt;/p&gt;&#xA;&lt;p&gt;Why is the blue square below the red one and not shifted to the side of the green one ending at the edge of the 7x7 volume? Should it not move left to right then down 2 squares when it reaches the next line?&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure if the author is just trying to show the stride moving down as it goes, but I think my confusion stems from the fact that the 1 stride image example is only moving in the horizontal direction (as seen below).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P0r8c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P0r8c.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there something fundamental I haven't grasped here?&lt;/p&gt;&#xA;" OwnerUserId="14935" LastEditorUserId="2444" LastEditDate="2020-06-22T16:59:03.320" LastActivityDate="2020-06-22T17:00:17.147" Title="Is my understanding of how the convolution with stride 2 works in this example correct?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5992" PostTypeId="2" ParentId="5991" CreationDate="2018-04-11T09:13:34.387" Score="1" Body="&lt;p&gt;Yes, you're right, after the green one, it should also move two steps (because stride = 2) to the right once more. Note that in the &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; output volume picture, there's also still a white cell in the top right corner. That cell would get filled with whatever colour you choose to move to the right after the green one.&lt;/p&gt;&#xA;&lt;p&gt;The blue one would then follow after what I described above, as the fourth square. I guess the author simply didn't feel like drawing a third square to the right of the green one, because the red + green squares already illustrate how the pattern works. The blue one was probably additionally drawn to illustrate that stride works the same way vertically as it does horizontally, e.g. blue also moves down two rows (because stride = 2).&lt;/p&gt;&#xA;&lt;p&gt;See the bottom of the picture in the second answer &lt;a href=&quot;https://qr.ae/pNKMsv&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-06-22T17:00:17.147" LastActivityDate="2020-06-22T17:00:17.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5993" PostTypeId="1" CreationDate="2018-04-11T10:08:28.350" Score="1" ViewCount="213" Body="&lt;p&gt;I use Sigmoid activation function for neurons at output layer of my Multi-Layer Perceptron also, I use cross-entropy cost function. As I know when activation functions like Tanh is used in output layer it's necessary to divide outputs of output layer neurons by sum of them like what is done for softmax, is such thing necessary for sigmoid activation function?&#xA;If it's necessary to normalize outputs of neurons, does it affect derivations?&lt;/p&gt;&#xA;" OwnerUserId="10051" LastEditorUserId="10051" LastEditDate="2018-04-11T11:21:59.917" LastActivityDate="2018-04-11T11:21:59.917" Title="Sigmoid output layer and Cross-Entropy cost function" Tags="&lt;neural-networks&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5994" PostTypeId="1" AcceptedAnswerId="6039" CreationDate="2018-04-11T10:19:15.367" Score="4" ViewCount="126" Body="&lt;p&gt;I have been reading quite a few papers, on genetic programming and its applications, in particular, chapter 10 of  &amp;quot;Genetic Programming: An Introduction and Tutorial, with a Survey of Techniques and Applications&amp;quot; (Langdon, Poli, McPhee, Koza; 2008)   Unfortunately, I can not wrap my head around how one could apply genetic programming to robotics, for example, in &lt;em&gt;path planning&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone explain this in the most simple manner? I know that it all depends on the fitness function.&lt;/p&gt;&#xA;" OwnerUserId="14863" LastEditorUserId="1671" LastEditDate="2021-04-19T22:34:41.253" LastActivityDate="2021-04-19T22:34:41.253" Title="How can genetic programming be used for path planning?" Tags="&lt;applications&gt;&lt;evolutionary-algorithms&gt;&lt;robotics&gt;&lt;genetic-programming&gt;&lt;path-planning&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5995" PostTypeId="5" CreationDate="2018-04-11T11:07:52.013" Score="0" Body="&lt;p&gt;Performance in this context is an evaluation of proficiency at a given task or set of tasks.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-04-12T02:28:04.217" LastActivityDate="2018-04-12T02:28:04.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5996" PostTypeId="4" CreationDate="2018-04-11T11:07:52.013" Score="0" Body="For question about methods of evaluating the performance (e.g. the accuracy) of an artificial intelligence algorithm or model." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-03-17T22:37:25.983" LastActivityDate="2020-03-17T22:37:25.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5997" PostTypeId="2" ParentId="3361" CreationDate="2018-04-11T11:37:01.397" Score="2" Body="&lt;p&gt;For genetic algorithms, I have written &lt;a href=&quot;https://github.com/giacomelli/GeneticSharp&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneticSharp&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A multi-platform genetic algorithm library for .NET Core and .NET Framework. The library has several implementations of GA operators, like: selection, crossover, mutation, reinsertion and termination.&lt;/p&gt;&#xA;" OwnerUserId="14938" LastEditorUserId="7374" LastEditDate="2018-04-12T23:38:38.870" LastActivityDate="2018-04-12T23:38:38.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5998" PostTypeId="5" CreationDate="2018-04-11T11:56:00.933" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-04-11T11:56:00.933" LastActivityDate="2018-04-11T11:56:00.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5999" PostTypeId="4" CreationDate="2018-04-11T11:56:00.933" Score="0" Body="For questions regarding action recognition. This should be used when asking about what could be implemented that complements or harms this." OwnerUserId="14723" LastEditorUserId="75" LastEditDate="2018-04-13T18:13:00.727" LastActivityDate="2018-04-13T18:13:00.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6002" PostTypeId="1" AcceptedAnswerId="6003" CreationDate="2018-04-11T15:02:29.147" Score="1" ViewCount="36" Body="&lt;p&gt;I am reading the cornerstone book, &amp;quot;Artificial Intelligence, A Modern Approach&amp;quot; by Stuart Russel, and Peter Norvig, and there is a passage in the book on page 98:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The complexity results depend very strongly on the assumptions made&#xA;about the state space. The simplest model studied is a state space&#xA;that has a single goal and is essentially a tree with reversible&#xA;actions. (The 8-puzzle satisfies the first and third of these&#xA;assumptions.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What are the &amp;quot;assumptions&amp;quot; in that context?&lt;/p&gt;&#xA;" OwnerUserId="14862" LastEditorUserId="2444" LastEditDate="2020-11-30T01:38:12.903" LastActivityDate="2020-11-30T01:38:12.903" Title="Which &quot;assumptions&quot; made about the state space are Russell and Norvig referring to in their book?" Tags="&lt;terminology&gt;&lt;search&gt;&lt;norvig-russell&gt;&lt;state-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6003" PostTypeId="2" ParentId="6002" CreationDate="2018-04-11T15:07:02.527" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The complexity results depend very strongly on the assumptions made about the state space. The simplest model studied is a state space that has a single goal and is essentially a tree with reversible actions. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The assumptions are the following: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;The simplest model studied is a state space&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;that has a single goal &lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;and is essentially a tree with reversible actions. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-04-11T15:07:02.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6006" PostTypeId="1" CreationDate="2018-04-11T20:34:18.817" Score="2" ViewCount="102" Body="&lt;p&gt;The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second lstm, it started to predict the next word from the current input word). The length of each of the two inputs is 512. Merely for the first input, it increases the measurement, say, for instance, perplexity, by about 3 from no this input at all. Merely for the second input, it increases the measurement, say, for instance, perplexity, by about 1 from no input at all. The problem is: Is it possible to combine these two inputs into a model that can produce a result of increasement more than 3 or the larger amount of increasement of the former two inputs models? If it is, how to build a model or what model should I build to combine them to do so?&lt;/p&gt;&#xA;" OwnerUserId="14947" LastEditorUserId="14948" LastEditDate="2018-04-12T02:31:23.403" LastActivityDate="2018-04-12T02:31:23.403" Title="Combine two embeddding inputs to increase more performance in LSTM model" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6007" PostTypeId="2" ParentId="54" CreationDate="2018-04-12T03:12:54.750" Score="2" Body="&lt;p&gt;We've had many discussions on what constitutes Artificial Intelligence, and my takeaway has been that decision-making is the core requirement of AI, regardless of the optimality of that decision.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this conception, Nimatron &lt;em&gt;(1939, &lt;a href=&quot;https://patents.google.com/patent/US2215544&quot; rel=&quot;nofollow noreferrer&quot;&gt;US2215544A&lt;/a&gt;)&lt;/em&gt; might be thought of as the first proper AI, pending verification of a &lt;a href=&quot;https://deserthat.wordpress.com/2010/05/18/earlyer-computer-games-babbage-and-nimatron/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a fabled Babbage Tic-Tac-Toe machine&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;But&lt;/em&gt;, the question as &lt;a href=&quot;https://ai.stackexchange.com/questions/3847/is-transistor-the-first-artificial-intelligence&quot;&gt;to whether a simple switch represents the most basic form of intelligence&lt;/a&gt; has also been raised... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a distinction between these decision-making devices and earlier algorithmic implementations such as water clocks, is that the water clocks cannot be said to make a decision in the sense of maximizing chance of success at some goal. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-12T03:37:23.317" LastActivityDate="2018-04-12T03:37:23.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6009" PostTypeId="1" CreationDate="2018-04-12T05:12:18.157" Score="1" ViewCount="204" Body="&lt;p&gt;Suppose I have a Boolean function that maps N bits to one bit. If I understand correctly, this function will have &lt;span class=&quot;math-container&quot;&gt;$2^{2^N}$&lt;/span&gt; possible configurations of its truth table.&lt;/p&gt;&#xA;&lt;p&gt;What is the minimum number of neurons and hidden layers I would need in order to have a multi-layer perceptron capable of learning all possible truth tables? I found one set of lecture notes &lt;a href=&quot;http://deeplearning.cs.cmu.edu/slides/lec2.universal.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that suggests that &amp;quot;checkerboard&amp;quot; shaped truth tables (which is something like an N-input XOR function) are hard to learn, but do they represent the worst-case scenario? The notes suggest that such tables require &lt;span class=&quot;math-container&quot;&gt;$3\cdot(N-1)$&lt;/span&gt; neurons arranged in &lt;span class=&quot;math-container&quot;&gt;$2 \cdot\log_2(N$&lt;/span&gt;) hidden layers&lt;/p&gt;&#xA;" OwnerUserId="14955" LastEditorUserId="28343" LastEditDate="2021-05-03T18:22:12.057" LastActivityDate="2021-05-03T18:22:12.057" Title="Minimum number of perceptrons for an n-bit truth table?" Tags="&lt;logic&gt;&lt;computational-learning-theory&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6012" PostTypeId="5" CreationDate="2018-04-12T09:58:14.217" Score="0" Body="&lt;p&gt;Two main statistical methods are used in data analysis: descriptive statistics, which summarize data from a sample using indexes such as the mean or standard deviation, and inferential statistics, which draw conclusions from data that are subject to random variation (e.g., observational errors, sampling variation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Category:Statistical_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistical Methods - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistics - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-13T03:22:27.393" LastActivityDate="2018-04-13T03:22:27.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6013" PostTypeId="4" CreationDate="2018-04-12T09:58:14.217" Score="0" Body="For questions about the applications/clarifications/intuitions/proofs behind the use of statistical methods in AI/ML programs." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-14T01:49:46.857" LastActivityDate="2018-04-14T01:49:46.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6014" PostTypeId="5" CreationDate="2018-04-12T10:00:20.053" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-04-12T10:00:20.053" LastActivityDate="2018-04-12T10:00:20.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6015" PostTypeId="4" CreationDate="2018-04-12T10:00:20.053" Score="0" Body="For questions about the applications/clarifications/intuitions/proofs behind the use of probabilistic methods/models in AI/ML programs." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-13T17:56:20.303" LastActivityDate="2018-04-13T17:56:20.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6017" PostTypeId="2" ParentId="6009" CreationDate="2018-04-12T10:25:07.243" Score="1" Body="&lt;p&gt;This is a very dicey question. Logic functions can be thought of as mapping multiple inputs to a single output. Now each logic function create its own boundary. So if you are using a complex logical equation it is actually very hard to approximate the underlying function. Here I am treating the input Booleans as the input features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From practical experience: I had a n-bit = 8-bit input, and it mapped to a single bit output. I used a 2 layer Neural Net and then added a final layer of single node (to have a 0 or 1 output). I was using the sigmoid activation function along with normal back-propagation learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I varied the hidden layer nodes from &lt;code&gt;64&lt;/code&gt; to &lt;code&gt;1024&lt;/code&gt;. The cost decreased, indicative of correct learning but the accuracy did not change, howsoever I tried (changing learning rates, using momentum, etc). It was either giving all &lt;code&gt;1&lt;/code&gt;'s or all &lt;code&gt;0&lt;/code&gt;'s, even though I was using the same set for training and validation. The reason I hypothesized for this behavior was:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The irrelevant input features are affecting the final output. Even after extensive training and testing on the same set with a huge number of hidden nodes.&lt;/li&gt;&#xA;&lt;li&gt;Due to the &lt;code&gt;1&lt;/code&gt; and &lt;code&gt;0&lt;/code&gt; nature of the input the NN was not able to create a correct boundary. Since if it created an equivalent feature of &lt;code&gt;x^3&lt;/code&gt; using its nodes, it'll just output the same constant value again and again &lt;code&gt;f(0)&lt;/code&gt; and &lt;code&gt;f(1)&lt;/code&gt;, where it should have been a nice little curve&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So in short it might or might not work, you never can tell!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-12T10:45:16.227" LastActivityDate="2018-04-12T10:45:16.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6018" PostTypeId="1" CreationDate="2018-04-12T10:25:40.547" Score="2" ViewCount="22" Body="&lt;p&gt;I'm writing neural network based on neural gas algorithm (university assignment) and I remember that lecturer said that, when you generate random neuron weights at the beginning, it's worth to generate them multiple times  and choose the best set of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem: I don't know &lt;strong&gt;what's the criteria of choosing the best set of weights for neurons&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="14962" LastActivityDate="2018-04-12T10:25:40.547" Title="Multiple centroid draw" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6019" PostTypeId="2" ParentId="5964" CreationDate="2018-04-12T14:08:49.607" Score="2" Body="&lt;p&gt;Yes, it is possible, and yes, it probably has been done before. Odds are, however, the person(s) who tried were disappointed with the results and forgot to tell others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasons they might be disappointed could be any of the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Took to long to train&lt;/li&gt;&#xA;&lt;li&gt;Even when fully trained, (or appeared to be), it did not give the expected output&lt;/li&gt;&#xA;&lt;li&gt;Too sensitive to variations in the input&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2018-04-13T13:55:59.010" LastActivityDate="2018-04-13T13:55:59.010" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6021" PostTypeId="5" CreationDate="2018-04-12T17:50:39.507" Score="0" Body="&lt;p&gt;A neuron, also known as a neurone (British spelling) and nerve cell, is an electrically excitable cell that receives, processes, and transmits information through electrical and chemical signals. These signals between neurons occur via specialized connections called synapses. Neurons can connect to each other to form neural networks. Neurons are the primary components of the central nervous system, which includes the brain and spinal cord, and of the peripheral nervous system, which comprises the autonomic nervous system and the somatic nervous system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neuron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuron - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An artificial neuron is a mathematical function conceived as a model of biological neurons, a neural network. Artificial neurons are elementary units in an artificial neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neuron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Neuron - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-13T03:20:50.773" LastActivityDate="2018-04-13T03:20:50.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6022" PostTypeId="4" CreationDate="2018-04-12T17:50:39.507" Score="0" Body="For questions about all aspects of a biological or artificial neuron." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-13T03:24:49.427" LastActivityDate="2018-04-13T03:24:49.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6023" PostTypeId="5" CreationDate="2018-04-12T17:53:32.623" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-04-12T17:53:32.623" LastActivityDate="2018-04-12T17:53:32.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6024" PostTypeId="4" CreationDate="2018-04-12T17:53:32.623" Score="0" Body="For questions about the proofs/intuitions/methods of combinatorial mathematics used in an AI algorithm." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-14T01:49:40.403" LastActivityDate="2018-04-14T01:49:40.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6026" PostTypeId="1" AcceptedAnswerId="7477" CreationDate="2018-04-13T02:25:29.013" Score="12" ViewCount="12935" Body="&lt;p&gt;A heuristic is &lt;strong&gt;admissible&lt;/strong&gt; if it &lt;em&gt;never overestimates&lt;/em&gt; the true cost to reach the goal node from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. If a heuristic is &lt;strong&gt;consistent&lt;/strong&gt;, then the heuristic value of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is never greater than the cost of its successor, &lt;span class=&quot;math-container&quot;&gt;$n'$&lt;/span&gt;, plus the successor's heuristic value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is A*, using tree or graph searches, optimal, if it uses an admissible heuristic?&lt;/p&gt;&#xA;" OwnerUserId="14913" LastEditorUserId="2444" LastEditDate="2019-11-10T16:55:57.743" LastActivityDate="2021-05-15T12:32:28.280" Title="Why is A* optimal if the heuristic function is admissible?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;heuristics&gt;&lt;a-star&gt;&lt;admissible-heuristic&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6027" PostTypeId="2" ParentId="81" CreationDate="2018-04-13T03:13:26.513" Score="2" Body="&lt;p&gt;Not strictly examples of AI, but related to the greater AI project: But us in the psychology / cognitive science side of things sure love our &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fpsyg.2014.01144/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;bayesian modelling!&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact there are people who believe that a theory grounded in such analysis would ultimately bring us to a &lt;a href=&quot;https://www.nature.com/articles/nrn2787&quot; rel=&quot;nofollow noreferrer&quot;&gt;unified theory of the brain and cognition!&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately to my knowledge, these theories are not yet complete or testable in interesting ways as they are grounded more in the philosophy end of things. More so the claims that the psychologists make are rather weak: that hypothesis updating and inference is Bayesian-like (which isn't super exciting to be honest) (but my knowledge in this area is not super complete)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alas, more work needs to be done but at least there is psychological support for the claim that cognition is Bayesian-like.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-04-13T03:13:26.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6028" PostTypeId="1" CreationDate="2018-04-13T03:14:47.773" Score="2" ViewCount="371" Body="&lt;p&gt;I was coding a CGAN model using Keras along with the paper (&lt;a href=&quot;https://arxiv.org/pdf/1411.1784.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1411.1784.pdf&lt;/a&gt;) and I wanted to try and match the models to exactly what the paper says. I knew the models presented in the paper would be primitive but just wanted to replicate those and see. For example the generator model in the paper mentions this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the generator net, a noise prior z with dimensionality 100 was&#xA;  drawn from a uniform distribution within the unit hypercube. Both z&#xA;  and y are mapped to hidden layers with Rectified Linear Unit (ReLu)&#xA;  activation [4, 11], with layer sizes 200 and 1000 respectively, before&#xA;  both being mapped to second, combined hidden ReLu layer of&#xA;  dimensionality 1200. We then have a final sigmoid unit layer as our&#xA;  output for generating the 784-dimensional MNIST samples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So for this I had the code like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def build_generator(self):&#xA;&#xA;       model = Sequential()&#xA;&#xA;       model.add(Dense(200, input_dim=self.latent_dim))&#xA;       model.add(Activation('relu'))&#xA;       model.add(BatchNormalization(momentum=0.8))&#xA;&#xA;       model.add(Dense(1000))&#xA;       model.add(Activation('relu'))&#xA;       model.add(BatchNormalization(momentum=0.8))&#xA;&#xA;       model.add(Dense(1200, input_dim=self.latent_dim))&#xA;       model.add(Activation('relu'))&#xA;       model.add(BatchNormalization(momentum=0.8))&#xA;&#xA;       model.add(Dropout(0.5))&#xA;&#xA;       model.add(Dense(np.prod(self.img_shape), activation='sigmoid'))&#xA;       model.add(Reshape(self.img_shape))&#xA;&#xA;&#xA;       model.summary()&#xA;&#xA;       noise = Input(shape=(self.latent_dim,))&#xA;       label = Input(shape=(1,), dtype='int32')&#xA;       label_embedding = Flatten()(Embedding(self.num_classes, self.latent_dim)(label))&#xA;&#xA;       model_input = multiply([noise, label_embedding])&#xA;&#xA;       img = model(model_input)&#xA;&#xA;       return Model([noise, label], img)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But still I think this not exactly what the paper means. What I understand from the paper is the noise and labels are first fed into two different layers and then combined to one layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that there should be three separate models inside the generator? Or am I mistaken thinking that? Would like to hear any thoughts on this.&lt;/p&gt;&#xA;" OwnerUserId="12843" LastEditorUserId="9647" LastEditDate="2018-04-15T04:17:25.013" LastActivityDate="2018-04-15T04:17:25.013" Title="Coding CGAN paper model in Keras" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6029" PostTypeId="2" ParentId="5964" CreationDate="2018-04-13T04:05:43.377" Score="2" Body="&lt;p&gt;Yes! Unsupervised machine learning has absolutely been applied to youtube videos... To recognize cats!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an article about it in &lt;a href=&quot;https://www.wired.com/2012/06/google-x-neural-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;wired&lt;/a&gt;. One of the leading ML researchers was &lt;a href=&quot;http://www.andrewng.org/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6252" LastActivityDate="2018-04-13T04:05:43.377" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6030" PostTypeId="1" CreationDate="2018-04-13T05:50:47.720" Score="3" ViewCount="716" Body="&lt;p&gt;I have similar architecture like in image:&lt;a href=&quot;https://i.stack.imgur.com/Zvne1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNN&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how to calculate gradient of filter &lt;strong&gt;F&lt;/strong&gt;. &#xA;I found these equations(&lt;a href=&quot;http://andrew.gibiansky.com/blog/machine-learning/convolutional-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/E97A9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gradient and delta&lt;/a&gt;,&#xA;where first equation calculate gradient of weight &lt;strong&gt;F&lt;sub&gt;ab&lt;/sub&gt;&lt;/strong&gt; from filter &lt;strong&gt;F&lt;/strong&gt; and second equation calculate &lt;strong&gt;deltas&lt;/strong&gt; of input layer and&lt;br&gt;&#xA;&lt;strong&gt;E&lt;/strong&gt; - total error,&lt;br&gt;&#xA;&lt;strong&gt;N&lt;/strong&gt; - input width,&lt;br&gt;&#xA;&lt;strong&gt;M&lt;/strong&gt; - input height,&lt;br&gt;&#xA;&lt;strong&gt;l&lt;/strong&gt; - layer. 0 - first, l = L last&lt;br&gt;&#xA;&lt;strong&gt;x&lt;/strong&gt; - input&lt;br&gt;&#xA;&lt;strong&gt;y&lt;/strong&gt; - output&lt;br&gt;&#xA;&lt;strong&gt;o&lt;/strong&gt; - activation function&lt;br&gt;&#xA;&lt;strong&gt;o&lt;sup&gt;'&lt;/strong&gt; - derivate of activation function,&lt;br&gt;&#xA;but I don't understand how to calculate &lt;strong&gt;∂E/∂y&lt;sub&gt;ij&lt;/sub&gt;l&lt;/sup&gt;&lt;/strong&gt;.&lt;br&gt;&#xA;How can I can calculate gradient of &lt;strong&gt;F&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2781" LastActivityDate="2018-04-13T05:50:47.720" Title="How to calculate gradient of filter in convolution network" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;math&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6032" PostTypeId="1" AcceptedAnswerId="6537" CreationDate="2018-04-13T10:12:06.770" Score="1" ViewCount="63" Body="&lt;p&gt;Not sure if this is the correct forum, but I have been working with a large (non-image) dataset that will eventually be used to train a neural network.  I have been puzzling over how to manage wide data sets.  For this application &quot;wide&quot; is maybe 10,000 or 20,000 points wide.  It is not really possible to store this as a row in a conventional RDMS (which are usually limited to several hundred columns).  Is it better to use a huge CSV file or maybe a no-sql technology like Cassandra (the data is originally in JSON format)?&lt;/p&gt;&#xA;" OwnerUserId="14994" LastActivityDate="2018-05-29T00:23:27.763" Title="How to manage high numbers of input layer data points" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6033" PostTypeId="5" CreationDate="2018-04-13T10:58:19.590" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/3820/what-is-the-difference-between-assisted-and-unassisted-learning-in-relation-to-a?utm_medium=organic&amp;amp;utm_source=google_rich_qa&amp;amp;utm_campaign=google_rich_qa&quot;&gt;What is the difference between assisted and unassisted learning in relation to AI?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-13T18:01:13.537" LastActivityDate="2018-04-13T18:01:13.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6034" PostTypeId="4" CreationDate="2018-04-13T10:58:19.590" Score="0" Body="For questions about AI learning without being given external guidance or classifications of the input." OwnerUserId="9947" LastEditorUserId="75" LastEditDate="2018-04-13T17:56:03.903" LastActivityDate="2018-04-13T17:56:03.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6035" PostTypeId="5" CreationDate="2018-04-13T11:15:05.607" Score="0" Body="&lt;p&gt;Have a look at the research paper that introduced AlphaZero &lt;a href=&quot;https://arxiv.org/abs/1712.01815&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/a&gt; (2017) by David Silver et al. and &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaZero&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AlphaZero&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2019-08-11T15:21:41.067" LastActivityDate="2019-08-11T15:21:41.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6036" PostTypeId="4" CreationDate="2018-04-13T11:15:05.607" Score="0" Body="For questions related to DeepMind's AlphaZero, which is a computer program that can play Go, Chess, and Shogi. AlphaZero achieved, within 24 hours of training, a superhuman level of play in these three games by defeating world-champion programs Stockfish, Elmo, and the 3-day version of AlphaGo Zero. AlphaZero was introduced in &quot;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&quot; (2017) by David Silver et al." OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-08-11T15:21:41.067" LastActivityDate="2019-08-11T15:21:41.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6038" PostTypeId="1" AcceptedAnswerId="13282" CreationDate="2018-04-13T12:07:15.427" Score="2" ViewCount="188" Body="&lt;p&gt;I have used a FFNN and &lt;a href=&quot;https://en.wikipedia.org/wiki/Liquid_state_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSM&lt;/a&gt; to perform the same task, namely, to predict the sentence &quot;How are you&quot;. The LSM gave me more accurate results than FFNN. However, the LSM did not produce perfect prediction and there are missing letters. More specifically, LSM produced &quot;Hw are yo&quot; and the FFNN predicted &quot;Hnw brf ypu&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between a FFNN and a LSM, in terms of architecture and purpose?&lt;/p&gt;&#xA;" OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2019-05-02T19:01:40.103" LastActivityDate="2019-07-11T22:55:43.447" Title="What is the difference between a feed-forward neural network and a liquid state machine?" Tags="&lt;neural-networks&gt;&lt;prediction&gt;&lt;comparison&gt;&lt;feedforward-neural-network&gt;&lt;liquid-state-machine&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6039" PostTypeId="2" ParentId="5994" CreationDate="2018-04-13T13:21:30.123" Score="6" Body="&lt;p&gt;Take a robot that we want to be able to move from the bottom right corner to the top left corner of a 4x4 matrix full of random holes it should avoid. With holes represented by 1s, it could look something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;exit&#xA;\/&#xA;[0,0,0,1]&#xA;[0,1,1,0]&#xA;[0,1,1,1]&#xA;[0,0,0,0]&#xA;      /\ &#xA;      enter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As we want it to get to an exit from a start, we have a natural fitness function: closeness to exit door in smallest number of moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The genetic programming approach to solving this is to create random computer programs (the second chapter of your link does a pretty good intro to the tree like nature of this process) and let them loose. The vast majority of these strategies will be/are utterly terrible, things like 'go right once' or 'go left ten times'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say we make 100 random programs on our first run. We firstly score them on how well they did according to our fitness function (the random programs that did the best). We take a set % of these to survive and get rid of the rest, lets say 10% survive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We take these surviving 10% of the best performing programs and use them to create new programs for the next generation by modifying them randomly again, but not completely. Say we randomly modify half their structure and leave the other half as is across however many we want for the next generation. We now let this generation loose again, and again rank, score, take the top 10% and breed a new generation from them and so on for n number of generations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, if we say left the grid as is, the program would generally come up with a rule roughly like 'go left x4, go up x4) as it solves this problem in the easiest way, but if we were to say, continuously randomise the position of 1s in the grid during this evolutionary process, we will force the program to come up with much more generalisable rules, such as checking the cells it can move into for 1s and not moving into any space containing a 1 etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus we can build a program with a flexible strategy able to cope with different environments for our robot in terms of number/configuration of holes - much more useful than having to program it for every configuration.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just like with regular evolution, over millions of trials of taking the top performers and modifying them slightly, these programs become very specialised and high performing, able to solve highly complex games, paths with highly complex features etc.&lt;/p&gt;&#xA;" OwnerUserId="14997" LastEditorUserId="14997" LastEditDate="2018-04-13T13:56:59.293" LastActivityDate="2018-04-13T13:56:59.293" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6040" PostTypeId="1" CreationDate="2018-04-13T16:15:31.080" Score="5" ViewCount="100" Body="&lt;p&gt;I have been reading a bit about networks where deep layers able to deal with a bunch of features (be it edges, colours, whatever). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering: how can possibly a network based on this 'specialised' layers be fooled by adversarial images? Wouldn't the presence of specialised feature detectors be a barrier to this? (as in: this image of a gun does share one feature with 'turtles' but it lacks 9 others so: no, it ain't a turtle).&#xA;thanks!&lt;/p&gt;&#xA;" OwnerUserId="14999" LastActivityDate="2018-08-21T21:08:04.450" Title="How can neural networks that extract many features be fooled by adversarial images?" Tags="&lt;convolutional-neural-networks&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6041" PostTypeId="1" CreationDate="2018-04-13T16:15:42.577" Score="2" ViewCount="241" Body="&lt;p&gt;If we model the game '2048' using a max-min game tree, what is the maximal path from a start state to a terminal state? (Assume the game ends only when the board is full&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is one of the sub-questions that should prepare us to actually modeling the game as a max-min game tree. However I'm failing to understand the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it actually the path to receiving 131072 as an endgame?&lt;/p&gt;&#xA;" OwnerUserId="15000" LastActivityDate="2018-08-08T15:22:47.210" Title="Adversarial search in the game '2048'" Tags="&lt;search&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6043" PostTypeId="2" ParentId="5964" CreationDate="2018-04-13T19:41:39.210" Score="2" Body="&lt;p&gt;Answer is &lt;em&gt;quite yes&lt;/em&gt;, please have a look what Google did around this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Google Cloud Video Intelligence makes videos searchable, and discoverable, by extracting metadata with an easy to use REST API. You can now search every moment of every video file in your catalog. It quickly annotates videos stored in Google Cloud Storage, and helps you identify key entities (nouns) within your video; and when they occur within the video.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cloud.google.com/video-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/video-intelligence/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, Google does recognize all kinds of data from the video: it classifies the whole content of it to tags.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about Humanoid Robot Sophia?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Cameras within Sophia's eyes combined with computer algorithms allow her to see. She can follow faces, sustain eye contact, and recognize individuals. She is able to process speech and have conversations using a natural language subsystem. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sophia_(robot)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Sophia_(robot)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These intent to the direction to understand (Google) and produce (Sophia) language from sounds and images. To learn think by themselves, machines are still not ready. If you get into these two cases more you would see that these are quite mechanical and manual (requiring human pre-effort) things still.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is said that machines are now on a phases of toddler who can ask names for things around her and name them. Take some years more, maybe the abilities are more advanced ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You asked about unsupervised learning. There is a &lt;a href=&quot;https://youtu.be/Q6nYgeH8Krs&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; about speech of MIT researcher, who made experiments on text and images and in final notes he denoted that it would be nice to make the same with videos, actually with the same reasoning you had: to learn a language. He promised to keep that in mind with his colleagues, maybe some of them already work on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interesting research paper on the topic was on this &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Alayrac_Unsupervised_Learning_From_CVPR_2016_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We address the problem of automatically learning the&#xA;  main steps to complete a certain task, such as changing a&#xA;  car tire, from a set of narrated instruction videos. The con-&#xA;  tributions of this paper are three-fold. [..] Third, we experimentally demonstrate that the proposed method can automatically discover, in an unsupervised manner, the main steps to achieve the task and locate the steps in the input videos.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2018-04-14T17:25:29.470" LastActivityDate="2018-04-14T17:25:29.470" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6044" PostTypeId="2" ParentId="111" CreationDate="2018-04-13T20:41:24.237" Score="5" Body="&lt;p&gt;They shouldn't. People should.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People cannot put the responsibilities of ethical decisions into the hands of computers. It is our responsibility as computer scientists/AI experts to program decisions for computers to make. Will human casualties still exist from this? Of course, they will--- people are not perfect and neither are programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an excellent in-depth debate on this topic &lt;a href=&quot;https://www.youtube.com/watch?v=93Xv8vJ2acI&amp;amp;index=46&amp;amp;list=WL&amp;amp;t=6s&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I particularly like Yann LeCun's argument regarding the parallel ethical dilemma of testing potentially lethal drugs on patients. Similar to self-driving cars, both can be lethal while having good intentions of saving &lt;em&gt;more&lt;/em&gt; people in the long run.&lt;/p&gt;&#xA;" OwnerUserId="6252" LastEditorUserId="6252" LastEditDate="2018-04-14T00:53:25.007" LastActivityDate="2018-04-14T00:53:25.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6046" PostTypeId="1" CreationDate="2018-04-14T02:34:47.893" Score="5" ViewCount="205" Body="&lt;p&gt;&lt;em&gt;Warning: This question takes us into &lt;a href=&quot;https://en.wikipedia.org/wiki/VALIS&quot; rel=&quot;nofollow noreferrer&quot;&gt;VALIS&lt;/a&gt; territory, but I wouldn't underestimate the profundity of that particular philosopher.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.oxforddictionaries.com/definition/intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;There is a non-AI definition of intelligence which is simply &quot;information&quot; (see definition 2.3)&lt;/a&gt;.  If that information is &lt;em&gt;active&lt;/em&gt;, in terms of utilization, I have to wonder if it might qualify as a form of algorithmic intelligence, regardless of the qualities of the information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm getting at is that fields such as recreational mathematics often produce techniques and solutions that don't have immediate, real world applications.  But there's an adage that pure math tends to find uses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you might have algorithms applied to a problems that outside of the problems from which it originated, or that couldn't initially be implemented in a computing context. (Minimax in 1928 might be an example.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goal orientation has been widely understood as a fundamental aspect of AI, but in the case of an algorithm designed for one problem, that it subsequently applied to a different problem, the goal may simply be a function of the algorithm's structure. (To understand the goal of minimax precisely, you read the algorithm.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you regard this form of information as intelligence, then intelligence can be general, regardless of strength in relation to a given problem.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can we consider this form of codification of information to be algorithmic intelligence?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;And, just for fun, if a string that encodes a cutting-edge AI is not being processed, does it still qualify as artificial intelligence?&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-17T02:38:22.697" LastActivityDate="2018-06-24T11:38:21.833" Title="Is a mathematical formula a form of intelligence?" Tags="&lt;philosophy&gt;&lt;definitions&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6047" PostTypeId="1" CreationDate="2018-04-14T03:25:34.920" Score="0" ViewCount="1289" Body="&lt;p&gt;I have a rather basic question about YOLO for bounding box detection.&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that it effectively associates each anchor box to an 8-dimension output.&lt;/p&gt;&#xA;&lt;p&gt;During testing, does YOLO take each anchor box and classify on it alone? What happens if the object is big and spans over several anchor boxes (e.g., covering 70% of the image)? How can YOLO classify and detect objects spanning over many anchor boxes?&lt;/p&gt;&#xA;" OwnerUserId="13068" LastEditorUserId="2444" LastEditDate="2021-01-28T23:54:32.367" LastActivityDate="2021-01-28T23:54:32.367" Title="Can YOLO detect large objects?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6048" PostTypeId="2" ParentId="5169" CreationDate="2018-04-14T08:23:57.840" Score="0" Body="&lt;p&gt;As a chess player and a AI/ML engineer, I can say yes, why not. I'm not sure why it isn't fair to compare anything if you give each side it's just due and do a 'fair comparison'. Obviously, what that encompasses is extremely subjective, but there are philosophical and logical measures of fairness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now speaking on the comparison, AlphaZero and a Human's learning styles are much more similar than that of a Human's and Stockfish. This is mainly due to the fact that human's in some capacity use &lt;a href=&quot;https://www.princeton.edu/~yael/Publications/Niv2009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL&lt;/a&gt;, mainly in the dopaminergic neural pathways. While human behavior can certainly be modeled as an alpha/beta tree-search, it is not anything like the way we make decisions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the top humans, who cares? We we've been worse than computers for years.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2018-04-14T08:23:57.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6049" PostTypeId="1" CreationDate="2018-04-14T11:41:27.750" Score="2" ViewCount="34" Body="&lt;p&gt;I'm trying to predict grades within a course at my university. At the moment I manually extracting features, but I'm curious if it's possible to somehow use my entire dataset with a deep learning approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data from throughout the course of students solving mandatory exercises. All students uses a plug-in within the editor that takes a snapshot of code-base each time the student saves the project (exercise). I also have data from when the students run the debugger. All exercises include tests which determine what score the student will receive on a given exercise. The students are free to execute the tests as many times as they like wile solving the exercise (the final score is given when the students presents the final result to a teaching assistant). Execution and results of these tests are also included in the data. Timestamps exists for all data. I also have the final grade of each student (which is determined 100% by the final exam).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of an approach to use this kind of data with a deep learning approach?&lt;/p&gt;&#xA;" OwnerUserId="15010" LastActivityDate="2018-04-14T11:41:27.750" Title="Possible to use codebase snapshots as input in deep learning?" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6050" PostTypeId="1" CreationDate="2018-04-14T11:58:29.497" Score="2" ViewCount="111" Body="&lt;p&gt;Which specific performance evaluation metrics are used in training, validation and testing and &lt;strong&gt;why&lt;/strong&gt;? I am thinking error metrics (RMSE, MAE, MSE) are used in validation, and testing should use a wide variety of metrics? I don't think performance is evaluated in training, but not 100% sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically I am actually after deciding when to use (i.e. in training, validation or testing) correlation coefficient, RMSE, MAE and others for numeric data (e.g. Willmott's Index of Agreement, Nash-Sutcliffe coefficient, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry about this being broad - I have actually been asked to define it generally (i.e. not for a specific dataset). But datasets I have been using have all numeric continuous values with supervised learning situations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally I am using performance evaluation for environmental data where I am using ANN to model. I have continuous features and am predicting a continuous variable.&lt;/p&gt;&#xA;" OwnerUserId="15011" LastEditorUserId="1671" LastEditDate="2018-06-04T20:53:52.250" LastActivityDate="2018-06-04T20:53:52.250" Title="Performance Evaluation Metrics used in Training, Validation and Testing" Tags="&lt;training&gt;&lt;data-science&gt;&lt;generative-model&gt;&lt;concepts&gt;&lt;soft-question&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6052" PostTypeId="1" AcceptedAnswerId="6079" CreationDate="2018-04-14T22:03:15.643" Score="3" ViewCount="426" Body="&lt;p&gt;As I've thought about AI, and what I understand of the problems that we face in the creation of it, I've noticed a recurring pattern: we always seem to be asking ourselves, &quot;how can we better simulate the brain?&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are we so fascinated with &lt;strong&gt;simulating&lt;/strong&gt; it? Isn't our goal to create intelligence, not create intelligence in a &lt;strong&gt;specific medium&lt;/strong&gt;? Isn't growing and sustaining living brains more inline with our goals, albeit a bit of an ethical controversy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is this exchange's description: &quot;For people interested in conceptual questions about life and challenges in a world where 'cognitive' functions can be mimicked in a &lt;strong&gt;purely digital environment&lt;/strong&gt;?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To condense these feelings in a more concise question: Why are we trying to create AI in a computer?&lt;/p&gt;&#xA;" OwnerUserId="15023" LastEditorUserId="1671" LastEditDate="2018-04-17T20:47:10.747" LastActivityDate="2018-09-05T08:18:14.853" Title="Why are we asking, &quot;How can we simulate the brain?&quot;" Tags="&lt;philosophy&gt;&lt;human-like&gt;&lt;theory&gt;&lt;brain&gt;&lt;wetware&gt;" AnswerCount="6" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6053" PostTypeId="2" ParentId="6052" CreationDate="2018-04-14T22:19:15.760" Score="0" Body="&lt;p&gt;Being the OP, I have already put some thought into this question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that computers are an attractive medium for simple AI because they are easily available and researchers are already familiar with them. In addition, science fiction writers of the last century were hopeful of the capabilities of computers and placed in our culture a dream of computer AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I also feel that perhaps other less explored fields would be better suited to the creation of strong AI. In particular, thinking about the nature of biology excites me. But as I understand it, we still know so little about how it biology works, let alone how to control it. But I feel this is where we should be focusing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researchers know that current computing hardware has limitations. GPUs are better suited than CPUs. Some CPUs have new hardware designed for AI computations. I suspect that this realization of the inadequacy of conventional hardware will continue until our hardware is nearly identical to the biology we are trying to simulate. After all, what simulation could ever be better than what it is trying to simulate?&lt;/p&gt;&#xA;" OwnerUserId="15023" LastActivityDate="2018-04-14T22:19:15.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6054" PostTypeId="1" CreationDate="2018-04-14T23:48:25.667" Score="1" ViewCount="114" Body="&lt;p&gt;How would you design a neural network that generates the positions of comparators in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sorting_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;sorting network&lt;/a&gt; given a set of numbers. I've tried to modify some already implemented networks that given a set of numbers it sorts the number. My goal is that given an unsorted sequence of numbers to generate a sorting network that will sort those numbers. I am not asking for the complete solution, just a starting point.&lt;/p&gt;&#xA;" OwnerUserId="15024" LastEditorUserId="1671" LastEditDate="2019-11-06T21:46:08.563" LastActivityDate="2019-11-06T21:46:08.563" Title="Design neural network for generating sorting networks" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;getting-started&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6055" PostTypeId="2" ParentId="6047" CreationDate="2018-04-15T00:25:08.377" Score="4" Body="&lt;p&gt;The example given based on the yolov1 paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://cdn-images-1.medium.com/max/1600/1*ZbmrsQJW-Lp72C5KoTnzUg.jpeg&quot; alt=&quot;&quot;&gt;&#xA;The last layer has a tensor of the dimension 7x7x30. but the dimension of the last tensor is not in every case 7x7x30.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;let be:&#xA;S: the number of grid cells in X and Y direction&#xA;C: the number classes to train&#xA;B: the number of bounding boxes in every grid cell&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dimension of the output tensor is calculated with this formula: SxSx(5*B+C). The given example in their paper has the following values:&#xA;S:7&#xA;B:2&#xA;C:20&#xA;(7X7X(5*2+20) = 7x7x30)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this configuration you can detect at most 96 object (S&lt;em&gt;S&lt;/em&gt;B -&gt; 7*7*2) and 2 objects per grid cell. (e.g. it's not possible to detect many small object in a few grid cells)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now lets consider what are each feature map of the last tensor used for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data of every bounding box is stored in 5 feature maps:&#xA;- 1 Feature map for the bounding box center in x direction&#xA;- 1 Feature map for the bounding box center in y direction&#xA;- 1 Feature map for the height of the bounding box&#xA;- 1 Feature map for the width of the bounding box&#xA;- 1 Feature map for the &quot;confidence score&quot; of the feature map&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(confidence score = P(Object)*IoU(BoundingBox,Object))&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;P(Object): the probability that in this cell is an object&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;IoU: Intersection over Union. A ration between the overlapping area and the union area:&#xA;&lt;img src=&quot;https://www.pyimagesearch.com/wp-content/uploads/2016/09/iou_equation.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;additional there are a feature map for every class. In every cell the following probability is calculated: P(Cass|Object)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long story short:&lt;/strong&gt;&#xA;If the bounding box is bigger than one grid cell, then some neurons share the same values (they share the bounding box center, the bounding box size and the confidence score, because they refer to the same bounding box).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://pjreddie.com/media/files/papers/yolo_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/media/files/papers/yolo_1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit no Bounding boxes on the image&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if there are no bounding boxes on the image, then the confidence score would be zero, because there are no overlapping area. you can configure the threshold of the confidence score. look at this sample picture with threshold 0: &lt;img src=&quot;https://camo.githubusercontent.com/1025ff29cd0e7427d108820ca87ea39d25fc82b0/687474703a2f2f706a7265646469652e636f6d2f6d656469612f696d6167652f53637265656e5f53686f745f323031352d30382d31345f61745f31312e34322e30325f414d2e706e67&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;additionally the feature maps of the classes would be zero because every neuron represents this probability P(Class|Object) because P(Class|Object) = P(Class And Object) / p(Object); (P(Class) = 0 -&gt; P(Class and Object) = 0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is adding images without bounding boxes to the training set useful?&#xA;I &lt;em&gt;think&lt;/em&gt; it helps to lower the number of false positive matches. But that is just a guess. I have trained yolo to detect this plant &lt;a href=&quot;https://en.wikipedia.org/wiki/Rumex_obtusifolius&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Rumex_obtusifolius&lt;/a&gt; and I added a lot of picture without bouding boxes, becuase a low number of false positive matches was important. The result: the specificity was &gt;99%.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;" OwnerUserId="9111" LastEditorUserId="9111" LastEditDate="2018-04-18T11:34:28.503" LastActivityDate="2018-04-18T11:34:28.503" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6056" PostTypeId="2" ParentId="6052" CreationDate="2018-04-15T03:10:49.333" Score="2" Body="&lt;p&gt;I don’t think AI is simulating the brain functions and not even close. Do you know how the nervous system work? How the neutrons transmit signals with action potential? Pathway analysis? Splicing junctions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is not about simulating the brain at all. We don’t simulate the biology pathway, we don’t simulate alternative splicing, we don’t have proteins in our models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, AI is a field with tons of mathematics. You give some data and try to extract complicated non linear pattern.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-04-15T03:10:49.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6057" PostTypeId="5" CreationDate="2018-04-15T08:53:23.713" Score="0" Body="&lt;p&gt;Robotics is an interdisciplinary branch of engineering and science that includes mechanical engineering, electrical engineering, computer science, and others. Robotics deals with the design, construction, operation, and use of robots, as well as computer systems for their control, sensory feedback, and information processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robotics - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T01:46:33.143" LastActivityDate="2018-04-17T01:46:33.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6058" PostTypeId="4" CreationDate="2018-04-15T08:53:23.713" Score="0" Body="For questions and clarifications about the use of AI/ML algorithms in the field of robotics." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T02:10:35.750" LastActivityDate="2018-04-17T02:10:35.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6059" PostTypeId="5" CreationDate="2018-04-15T08:55:51.817" Score="0" Body="&lt;p&gt;Cyberterrorism is the use of the Internet to conduct violent acts that result in, or threaten, loss of life or significant bodily harm, in order to achieve political gains through intimidation. It is also sometimes considered an act of Internet terrorism where terrorist activities, including acts of deliberate, large-scale disruption of computer networks, especially of personal computers attached to the Internet by means of tools such as computer viruses, computer worms, phishing, and other malicious software and hardware methods and programming scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cyberterrorism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cyberterrorism - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T02:10:49.667" LastActivityDate="2018-04-17T02:10:49.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6060" PostTypeId="4" CreationDate="2018-04-15T08:55:51.817" Score="0" Body="For questions about prevention/effects of the use of Artificially Intelligent algorithms in Cyber-terrorism." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T01:38:38.683" LastActivityDate="2018-04-17T01:38:38.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6061" PostTypeId="5" CreationDate="2018-04-15T09:00:19.450" Score="0" Body="&lt;p&gt;A cybernetic organism or “cyborg” in IT is defined as an organism with both biological and technological components. In some definitions, a cyborg is described as a hypothetical or fictional creation. However, in a technical sense, humans can be seen as cyborgs in various types of situations, including the use of artificial implants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.techopedia.com/definition/15651/cyborg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cyborg - Techopedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cyborg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cyborg - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T02:10:20.543" LastActivityDate="2018-04-17T02:10:20.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6062" PostTypeId="4" CreationDate="2018-04-15T09:00:19.450" Score="0" Body="For questions about &quot;cybernetic organisms&quot; defined as an entity with both  organic and biomechatronic body parts.  For questions about physical integration of animal and machine, theoretical and actual." OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-04-17T01:46:28.043" LastActivityDate="2018-04-17T01:46:28.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6063" PostTypeId="5" CreationDate="2018-04-15T09:04:23.217" Score="0" Body="&lt;p&gt;The technological singularity (also, simply, the singularity) is the hypothesis that the invention of artificial superintelligence (ASI) will abruptly trigger runaway technological growth, resulting in unfathomable changes to human civilization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technological singularity - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/17/what-is-the-concept-of-the-technological-singularity&quot;&gt;What is the concept of the technological singularity?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T01:38:31.510" LastActivityDate="2018-04-17T01:38:31.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6064" PostTypeId="4" CreationDate="2018-04-15T09:04:23.217" Score="0" Body="For questions about the concept of technological singularity." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T02:10:10.270" LastActivityDate="2018-04-17T02:10:10.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6066" PostTypeId="1" CreationDate="2018-04-15T12:54:08.807" Score="3" ViewCount="122" Body="&lt;p&gt;How can I classify a given sequence of images (video) as either moving or staying still from the perspective of the person inside the car?&lt;/p&gt;&#xA;&lt;p&gt;Below is an example of the sequence of 12 images animated.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Moving from the point of the person inside the car.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n2kj7.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n2kj7.gif&quot; alt=&quot;Class 0: Moving&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Staying still from the point of the person inside the car.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jUM4x.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jUM4x.gif&quot; alt=&quot;Class 1: still&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Methods I tried to achieve this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A simple CNN (with 2d convolutions) with those 12 images (greyscaled) stacked in the channels dimension (like Deepmind's DQN). The input to the CNN is &lt;code&gt;(batch_size, 200, 200, 12)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A CNN with 3d convolutions. The input to the CNN is &lt;code&gt;(batch_size, 12, 200, 200, 1)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A CNN+LSTM (time-distributed with 2d convolutions). The input to the neural network is &lt;code&gt;(batch_size, 12, 200, 200, 1)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The late fusion method, which is taking 2 frames from the sequence that are some time steps apart and passing them into 2 CNNs (with same weights) separately and concatenating them in a dense layer &lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42455.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;As mentioned in this paper&lt;/a&gt;. This is also like CNN+LSTM without the LSTM part. The input to this net is &lt;code&gt;(batch_size, 2, 200, 200, 1)&lt;/code&gt; -&amp;gt; the 2 images are first and last frames in the sequence&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;All the methods I tried failed to achieve my objective. I tried tuning various hyperparameters, like the learning rate, the number of filters in CNN layers, etc., but nothing worked.&lt;/p&gt;&#xA;&lt;p&gt;All the methods had a &lt;code&gt;batch_size&lt;/code&gt; of 8 (due to memory constraint) and all images are greyscaled. I used ReLUs for activations and softmax in the last layer. No pooling layer was used.&lt;/p&gt;&#xA;&lt;p&gt;Any help on why my methods are failing or any pointers to a related work&lt;/p&gt;&#xA;" OwnerUserId="5030" LastEditorUserId="2444" LastEditDate="2020-09-07T22:40:24.017" LastActivityDate="2020-09-07T22:40:24.017" Title="How can I determine whether a car in a video is moving or not?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;video-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6067" PostTypeId="2" ParentId="6066" CreationDate="2018-04-15T13:23:35.233" Score="1" Body="&lt;p&gt;CNNs are translation invariant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are over complicating the problem.  The easiest thing you can do is define a region of interest (ROI) of the hood.  In the first case where the car is moving and the reflections are dynamic.  In the second case they are static.  Just do frame-to-frame image subtraction of the hood.  If the vehicle is moving you will have lots of 'edge energy'.  If it is not moving it will be just noise.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can apply the same method to the whole image too.  In the static case the image subtraction method may become messy as the clouds are moving along with vehicles and pedestrians.  For this case use the image subtractions as input to your method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to run an image stabilization algorithm.  OpenCV has one.  Look at the transformation outputs (translation, rotation, scale, rigid, similarity, affine, etc.).  If you can't make a simple filter on them to determine the two cases, train a classifier.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2018-04-15T13:56:54.827" LastActivityDate="2018-04-15T13:56:54.827" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6069" PostTypeId="1" AcceptedAnswerId="6075" CreationDate="2018-04-15T16:40:31.450" Score="4" ViewCount="1383" Body="&lt;p&gt;In a neural network for chess (or checkers), the output is a piece or square on the board and an end position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would one encode this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can see choosing a starting square is 8x8=64 outputs and an ending square is 8x8=64 outputs. So the total number of possible moves is 64x64 4096 outputs. Giving a probability for every possible move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? This seems like an awful lot of outputs!&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="9647" LastEditDate="2018-04-17T02:11:08.553" LastActivityDate="2018-04-17T02:11:08.553" Title="How do you encode a chess move in a neural network?" Tags="&lt;neural-networks&gt;&lt;chess&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6071" PostTypeId="1" AcceptedAnswerId="6076" CreationDate="2018-04-15T17:42:28.743" Score="0" ViewCount="62" Body="&lt;p&gt;If you have a game and you are training an AI there seems to be two ways to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you take the game-state and a possible move and evaluate whether this move would be good or bad:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) GAME_STATE + POSSIBLE_MOVE  --&gt; Good or bad?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is to take the game state and get probabilities of every conceivable move:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) GAME_STATE ---&gt; Probabilities for each move&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that both models are used. I.e. in language and RNN might use (2) to find the probabilities for each next word or letter. But AlphaZero might use (1). Noting also that in a game like chess GAME_STATE + POSSIBLE_MOVE = NEW_GAME_STATE. Whereas in some games you might not know the result of your move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which do you think is the best method? Which is the best way to do AI? Or some combination of the two?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2018-04-16T07:17:01.807" Title="Which is best: evaluation of states or probability of moves?" Tags="&lt;machine-learning&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6073" PostTypeId="2" ParentId="6052" CreationDate="2018-04-15T19:44:00.047" Score="1" Body="&lt;p&gt;There are a number of reasons why a simulated brain might be better than creating a real brain. One reason is computers can live indefinitely (kind of). Brains may not be able to live forever and there might not be a way to transfer information from one brain to another. One of the principle advantages of a computer then is that it could have more experience than any brain could have in its lifetime. Another reason is that there are a lot of things we don't know about the brain. Even if we were able to replicate the brain we would have a hard time using it in the way that we want until we fully understand it. The simulated brain doesn't have this problem. We know exactly how artificial neural networks develop, and thus there is not as much that we don't understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those answers tell you why we might want a digital brain, but your question seems to also ask why study the digital brain over a biological brain? This seems to imply that we can't do both, but in fact there are many research groups doing work in areas that contribute to growing living brains (Max Planck Institute of Molecular Cell Biology and Genetic (MPI-CBG), the Medical Research Center in the UK, etc.).&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-04-15T19:44:00.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6075" PostTypeId="2" ParentId="6069" CreationDate="2018-04-16T06:24:07.380" Score="3" Body="&lt;p&gt;The number is &lt;strong&gt;4672&lt;/strong&gt; from Google.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1712.01815.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A move in chess may be described in two parts: selecting the piece to move, and then&#xA;selecting among the legal moves for that piece. We represent the policy π(a|s) by a 8 × 8 × 73&#xA;stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8×8&#xA;positions identifies the square from which to “pick up” a piece.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4672 might sound a big number but it's nothing to what Google (and many other competitors) has been doing to deep learning for image analysis. &lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-04-16T06:24:07.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6076" PostTypeId="2" ParentId="6071" CreationDate="2018-04-16T07:17:01.807" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Which do you think is the best method? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As with most machine learning, each approach has its strengths and weaknesses, and other than a little bit of intuition:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Policy-based methods are strong in large or continuous action spaces, and/or where there is a simple relationship between state and optimal action. E.g. controlling a robotic arm with continuous action space.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value-based methods are strong where there is a simple relationship between state and value under an optimal policy. E.g. in a maze game.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It may not always be clear which is best, in which case experimentation is required. If using neural networks, there will then be a large number of hyper-parameters on each approach, so it may be hard to come to a strict conclusion about which is better. Although you can include &quot;easy to find a working model&quot; or &quot;robust for a range of hyper-parameter values&quot; as benefits of any type of model if you wish - these are important practical benefits of any approach, developers rarely want to do 100s of experiments to tune a learning rate parameter for example.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Which is the best way to do AI? Or some combination of the two?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Actor-Critic, as seen in Asynchronous Advantage Actor Critic. A3C and A2C (deterministic variant of A3C) is producing current state-of-the-art results in video games. This is a combination of both approaches, where the agent maintains two related models - one, the Actor, generates a policy directly by looking at the state, and the second, the Critic, tracks the estimated value of each state. Often, these two models share some parameters - e.g. using neural networks, the initial layers may be the same for both.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-04-16T07:17:01.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6077" PostTypeId="5" CreationDate="2018-04-16T09:06:50.373" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://techcrunch.com/2017/01/28/artificial-intelligence-and-the-law/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Article on AI and Law&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://law.queensu.ca/how-will-artificial-intelligence-affect-legal-profession-next-decade&quot; rel=&quot;nofollow noreferrer&quot;&gt;How will artificial intelligence affect the legal profession in the next decade?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T02:10:46.003" LastActivityDate="2018-04-17T02:10:46.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6078" PostTypeId="4" CreationDate="2018-04-16T09:06:50.373" Score="0" Body="For questions about the legal aspects and ramifications due to the creation/use of AI in various fields and applications (also laws in place to regulate the proper use of AI technology)." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-17T02:10:29.010" LastActivityDate="2018-04-17T02:10:29.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6079" PostTypeId="2" ParentId="6052" CreationDate="2018-04-16T09:28:32.273" Score="3" Body="&lt;p&gt;Human intelligence is very general / broad in its scope. This is self-evident, and whatever AI ends up to be, we'd like it to be a general problem solver as well (cf. Simon and Newell). Taking liberal interpretations of your question...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why AI in a computer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Computers, to the extent that we can frame problems in general as a solvable computational problem, are also general problem solvers. Wether this is actually the case (can you compute meaning or feels?) is up for debate (cf. computational functionalism, hyper-computation), but it is part of the artificial intelligence project to make a claim on this statement.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do we think a computational framework brings us any closer to an understanding of cognition / consciousness?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Good question, and frankly there is no good answer to that asides from &quot;its the best thing we got&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR &quot;computational functionalism&quot;, a lot of the literature in psychology and philosophy &lt;em&gt;seems&lt;/em&gt; to converge towards an understanding of cognition as &quot;computational&quot; (as in information processing: the V1 stream in the brain processes &quot;early visual information&quot;) and functional (goal directed grounded on &quot;meaning&quot;, ex: &quot;i scratch itch because itchy&quot;, as opposed to &quot;i am moving atoms&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the two theories don't mesh together well (cf. Chinese Room Argument, and the many other arguments in a similar flavour) despite their independent successes in the theory of mind. Why this is the case nobody quite knows...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why not AI in something that isnt a computer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know, but to the extent that our understanding of the world is grounded in math, then it being in a computer is sufficient anyways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe there are other paradigms of understanding the world though. Fingers crossed &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are we asking, “How can we simulate the brain?”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because its the best tentative understanding we have of an &quot;intelligent faculty&quot;, though it should be noted that various methods in machine learning don't seem to be directly inspired by biological implementation  (kNN, statistical methods, as opposed to neural nets)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further reading: &lt;a href=&quot;http://www.scaruffi.com/nature/mach01.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scaruffi.com/nature/mach01.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2018-04-21T00:07:00.643" LastActivityDate="2018-04-21T00:07:00.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6080" PostTypeId="1" CreationDate="2018-04-16T13:19:51.720" Score="1" ViewCount="42" Body="&lt;p&gt;Using a neural network the method seems to be that you end up with a probability for each possible outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To predict the next frame in a monochrome movie of size 400x400 with 8 shades of gray, it seems like there seems to be: 8^(160000) possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other had if you just predicted the probability for each pixel individually you would end up with some kind of image which gets progressively blurred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps what you want is to generate a few possibilities that are none-the-less quite sharp. In a similar way to weather prediction(?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how would you go about designing a neural network that takes reads a movie and tries to predict the next frame?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2018-04-16T13:19:51.720" Title="Best way to predict future frame of movie or game?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6081" PostTypeId="1" CreationDate="2018-04-16T16:49:11.807" Score="1" ViewCount="28" Body="&lt;p&gt;I am interested to see what advantages a Loop Network (Feed Forward Network that takes its output as input, I think it's called an RNN, not sure). The only result I found was that it was extremely sensitive to context, but only the context behind it. Other than that, I did not notice any changes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figured this would be better for a language processing unit, or one used to make inferences based upon it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the shortcomings of each? Advantages?&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-04-16T16:49:11.807" Title="Whats advantages does a Loop Network have over a Feed Forward Network?" Tags="&lt;ai-design&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6082" PostTypeId="1" CreationDate="2018-04-16T17:00:14.823" Score="8" ViewCount="307" Body="&lt;p&gt;There is a popular story regarding the &lt;a href=&quot;https://en.wikipedia.org/wiki/Back-of-the-envelope_calculation&quot; rel=&quot;nofollow noreferrer&quot;&gt;back-of-the-envelope calculation&lt;/a&gt; performed by a British physicist named G. I. Taylor. He used &lt;a href=&quot;http://www.atmosp.physics.utoronto.ca/people/codoban/PHY138/Mechanics/dimensional.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dimensional analysis&lt;/a&gt; to estimate the power released by the explosion of a nuclear bomb, simply by analyzing a picture that was released in a magazine at the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe many of you know some nice back-of-the-envelope calculations performed in machine learning (more specifically neural networks). Can you please share them?&lt;/p&gt;&#xA;" OwnerUserId="6252" LastEditorUserId="2444" LastEditDate="2019-04-12T21:32:26.420" LastActivityDate="2021-03-04T07:05:40.647" Title="Back-of-the-envelope machine learning (specifically neural networks) calculations" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;artificial-neuron&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6084" PostTypeId="2" ParentId="2405" CreationDate="2018-04-16T21:47:46.010" Score="6" Body="&lt;p&gt;It depends on your loss function, but you probably need to tweak it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using an update rule like &lt;code&gt;loss = -log(probabilities) * reward&lt;/code&gt;, then your loss is high when you unexpectedly got a large reward—the policy will update to make that action more likely to realize that gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, if you get a negative reward with high probability, this will result in negative loss—however, in minimizing this loss, the optimizer will attempt to make this loss &quot;even more negative&quot; by making the log probability more negative (i.e. by making the probability of that action less likely)—so it kind of does what we want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now improbable large negative losses are punished more than the more than likely ones, when we probably want the opposite. Hence, &lt;code&gt;loss = -log(1-probabilities) * reward&lt;/code&gt; might be more appropriate when the reward is negative.&lt;/p&gt;&#xA;" OwnerUserId="15028" LastEditorUserId="15028" LastEditDate="2019-10-16T17:23:17.077" LastActivityDate="2019-10-16T17:23:17.077" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6085" PostTypeId="1" CreationDate="2018-04-17T06:48:15.340" Score="2" ViewCount="41" Body="&lt;p&gt;I'm new to CNNs and am wondering if I understand the relationship between the following terms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In image analysis, receptive fields group &quot;input neurons&quot; to reduce the connection &#xA;to the next layer. Convolution (using kernels or filters) reduces the scale but produces depth in the form of feature maps which isolate shapes for example. Each subsequent process reduces scale but produces greater depth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pooling is a process used further reduce data sets by (in the case of max pooling) taking the &quot;brightest&quot; or strongest features and generalizing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stride is the &quot;resolution&quot; and a longer stride will reduce accuracy but could act like pooling&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To what extent is my understanding of the relationships accurate?&#xA;Thank you&lt;/p&gt;&#xA;" OwnerUserId="15074" LastActivityDate="2018-04-17T06:48:15.340" Title="The relationship between CNN terms" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6089" PostTypeId="1" CreationDate="2018-04-17T14:02:49.700" Score="5" ViewCount="465" Body="&lt;p&gt;I am trying to understand how genetic programming can be used in the context of auto-encoders. Currently, I am going through 2 papers&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ijcai.org/Proceedings/89-1/Papers/122.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training Feedforward Neural Networks Using Genetic Algorithms&lt;/a&gt; (a classific one)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.springer.com/cda/content/document/cda_downloaddocument/9783319700953-c2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training Deep Autoencoder via VLC-Genetic Algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, these papers don't really help me to grasp the concept of genetic programming in this specific context, maybe because I'm not very familiar with GP.&lt;/p&gt;&#xA;&lt;p&gt;I understand that autoencoders are supposed to reconstruct the instances of the particular classes they have been trained on. If another fed instance is not reconstructed as expected, then it could be called an anomaly.&lt;/p&gt;&#xA;&lt;p&gt;But how can genetic programming be used in the context of auto-encoders? You are still required to create a neural network, but, instead of a feed-forward one, you use autoencoder, but how exactly?&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any tutorials or explanations.&lt;/p&gt;&#xA;" OwnerUserId="14863" LastEditorUserId="2444" LastEditDate="2021-01-19T17:58:13.887" LastActivityDate="2021-01-19T17:58:13.887" Title="How can genetic programming be used in the context of auto-encoders?" Tags="&lt;reference-request&gt;&lt;applications&gt;&lt;autoencoders&gt;&lt;genetic-programming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6090" PostTypeId="2" ParentId="6082" CreationDate="2018-04-17T15:00:49.500" Score="0" Body="&lt;p&gt;I have one to share. This is no formula, but a general thing I have noticed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of neurons + neurons should be proportionate, in some way, to the complexity of the classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this is fairly basic and widely known, it has helped me in many times to consider one thing: how many at a minimum does it need?&lt;/p&gt;&#xA;" OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2018-04-17T15:11:00.413" LastActivityDate="2018-04-17T15:11:00.413" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6091" PostTypeId="1" CreationDate="2018-04-17T15:29:02.883" Score="3" ViewCount="365" Body="&lt;p&gt;Researchers at Stanford University released, in 2012, the paper &lt;a href=&quot;http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Financial Market Time Series Prediction with Recurrent Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It goes on to discuss how they used echo state networks to predict things such as Google's stock prices. However, to do this, once trained, the network's inputs are a day's stock price, and the output is the day's predicted stock price. The way the paper is worded is like this could be used to predict future stock prices, for example. However, to predict tomorrow's stock price, you need to give the neural network tomorrow's stock price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this paper seems to show is that the neural network is converging on a solution where it simply modifies its inputs a minimal amount, hence the output of the ESN is just a small alteration of its input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some Python implementations of the work shown in this paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/kimanalytics/Recurrent-Neural-Network-to-Predict-Stock-Prices&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kimanalytics/Recurrent-Neural-Network-to-Predict-Stock-Prices&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/europa502/RNN-based-Bitcoin-Value-Predictor&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/europa502/RNN-based-Bitcoin-Value-Predictor&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In particular, I was playing with the latter which produces the following graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OtqC0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OtqC0.png&quot; alt=&quot;Figure 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I take the same trained network, and alter the 7th's day's &quot;real&quot; stock price to say something extreme like $0, this is what comes out: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DblgY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DblgY.png&quot; alt=&quot;Figure 2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, it basically regurgitates its inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what is the significance of this paper? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has no use in any financial predictions, like the network shown in the paper &lt;a href=&quot;https://arxiv.org/pdf/1603.08604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classification-based Financial Markets Prediction using Deep Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15085" LastEditorUserId="2444" LastEditDate="2020-06-08T17:07:52.600" LastActivityDate="2021-03-05T20:09:05.930" Title="What is the significance of this Stanford University &quot;Financial Market Time Series Prediction with RNN's&quot; paper?" Tags="&lt;recurrent-neural-networks&gt;&lt;papers&gt;&lt;echo-state-network&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6092" PostTypeId="1" AcceptedAnswerId="6096" CreationDate="2018-04-17T16:28:17.193" Score="2" ViewCount="1332" Body="&lt;p&gt;I am having a difficult time translating this pseudocode into functional C++ code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aJjyc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aJjyc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At line 10: The value function is represented as V[s], which has a bracket notation like arrays. Is this a separate method or just a function of the value with a given state? Why is S inside the brackets? Is this supposed to be an array with as many elements as S?&lt;/li&gt;&#xA;&lt;li&gt;At line 12: Vk would be the element in index k inside of array V?&lt;/li&gt;&#xA;&lt;li&gt;At line 16: I'm interpreting this as the start of a do-while loop that ends at line 20.&lt;/li&gt;&#xA;&lt;li&gt;Line 19: I'm finding the action that maximizes the sum, for all states, of the equation following the sigma?&lt;/li&gt;&#xA;&lt;li&gt;Line 20: I'm interpreting this a the end of the do-while. But what is this condition? Am I checking if there is an s such that this condition applies? So Would I would have to loop between all states and stop if any state satisfies the condition? (Basically a loop with a break, instead of a while)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15089" LastActivityDate="2018-04-18T08:48:53.563" Title="Value iteration algorithm from pseudo-code to C++" Tags="&lt;learning-algorithms&gt;&lt;c++&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6093" PostTypeId="2" ParentId="5825" CreationDate="2018-04-17T18:02:50.993" Score="1" Body="&lt;p&gt;&lt;em&gt;Probably not worth the hassle.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that simply nobody cares. I'm all for having my function be differentiable at all points, but I don't think that having a single jump in the derivative matters much. The neuron basically either repeats its input (x&gt;0) or outputs nearly nothing (x&amp;lt;0) and the point in between is not that important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ELU(x) = x &amp;gt; 0 ? x : α * (exp(x) - 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;by your proposal&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ELU(x) = x &amp;gt; 0 ? α * x : α * (exp(x) - 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would have far more consequences. For example, after &lt;code&gt;n&lt;/code&gt; layers, the output would get multiplied by &lt;code&gt;α**n&lt;/code&gt;, assuming all &lt;code&gt;x&lt;/code&gt; on the path being positive. This could be countered by dividing each weight by &lt;code&gt;α&lt;/code&gt;, but the derivatives would change. You surely could adjust everything (initial rates, learn rate and whatever), so it'd work as usual, but it could laborious, especially when testing multiple learning algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest to use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ELU(x) = x &amp;gt; 0 ? x : α * (exp(x/α) - 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead, as it just changes the less important part of the curve.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or did they not bother, because α=1 is definitely the hyperparameter to use?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can imagine that everyone tried α=1 and was happy with it. It prevented the gradient from being zero and that's it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have any support of my claim (having never read about something doesn't mean that it doesn't get written about).&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-04-17T18:02:50.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6094" PostTypeId="1" CreationDate="2018-04-18T00:42:14.503" Score="4" ViewCount="65" Body="&lt;p&gt;I have order data, here's a sample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ninety-six (96) covered pans, desinated mark cutlery.&#xA;5 vovered pans by knife co.&#xA;(SEE SCHEDULE A FOR NUMBERS). 757 SOUP PANS&#xA;115 10-quart capacity pots.&#xA;Thirteen (13), 30 mm thick covered pans. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have over 50k rows of data such as this. In a perfect world, the above would need to be tabulated as such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;count, type&#xA;96, covered pan&#xA;5, covered pan&#xA;757, soup pan&#xA;115, pot&#xA;13, covered pan&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could machine learning be the correct approach for a problem such as this?&lt;/p&gt;&#xA;" OwnerUserId="15099" LastEditorUserId="15099" LastEditDate="2018-04-18T00:49:36.130" LastActivityDate="2018-12-18T23:01:55.383" Title="Can machine learning help me digest asymmetrical order descriptions?" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;datasets&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6095" PostTypeId="2" ParentId="6094" CreationDate="2018-04-18T07:16:16.697" Score="1" Body="&lt;p&gt;Yes a variant of NLP processing could help find the correct number to extract and type of object in this data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compared to the spreadsheet, the raw text data is ambiguous without understanding language to a reasonable depth, and without knowing the business context in order to extract the relevant information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, you are expecting to extract &quot;soup pan&quot; and &quot;covered pan&quot;, but not &quot;capacity pot&quot;. Also that parts of phrases such as &quot;30 mm&quot; or &quot;10-quart&quot; are lower importance qualifiers, and not specifying quantity of something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current state of the art for extracting this kind of data would be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Bidirectional_recurrent_neural_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;bidirectional LSTM (a type of Recurrent Neural Network)&lt;/a&gt;. You would likely get it to flag the parts of each entry that were relevant to the tabulated data you wanted to extract, then feed those into a simpler stage that put them into the spreadsheet. However, there are two caveats:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You need a lot of correctly-labelled training data to get reasonable performance. Using a word embedding layer, such as word2vec or GloVe, should significantly reduce the amount of training data required, but may require a careful pre-processing stage, and may be less useful when you have a lot of jargon in your data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Performance is never perfect, and the system can still make stupid mistakes, because it does not truly understand the text is is dealing with. That applies to all ML approaches to this problem, and likely also to coding up an &quot;expert system&quot;, although it may be easier to write the expert system to recognise when it had failed and ask for help.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-04-18T07:24:03.137" LastActivityDate="2018-04-18T07:24:03.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6096" PostTypeId="2" ParentId="6092" CreationDate="2018-04-18T08:48:53.563" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;At line 10: The value function is represented as V[s], which has a bracket notation like arrays. Is this a separate method or just a function of the value with a given state? Why is S inside the brackets? Is this supposed to be an array with as many elements as S?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is just a notation that the value function is a mapping between &lt;em&gt;S&lt;/em&gt; and the real numbers. When implementing you would want to store &lt;em&gt;V(s)&lt;/em&gt; and &lt;em&gt;π(s)&lt;/em&gt; as either &lt;strong&gt;arrays&lt;/strong&gt; or some kind of &lt;strong&gt;hashmap&lt;/strong&gt; like &lt;code&gt;unordered_map&lt;/code&gt; (in which case your states must be hashable). Here we also have to assume that this container will fit in the memory, otherwise the value function has to be approximated with function approximation which is not covered explicitly by this pseudo-code.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At line 12: Vk would be the element in index k inside of array V?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. This line says that you would need to store multiple value functions during the algorithm, basically a list of functions (or think of it as an array of functions or a two-dimensional array, where &lt;em&gt;k&lt;/em&gt; goes along one dimension and &lt;em&gt;s&lt;/em&gt; goes along the other dimension) and if you look at the loop following it, it shows you how you need to keep appending new value functions to the end of this list. However notice (in line 19) that you only need the value function from the previous iteration to calculate your new value function, which means that you will never need to store more than two value functions (the new one and the previous one).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At line 16: I'm interpreting this as the start of a do-while loop that ends at line 20.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. This loop calculates your value function.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Line 19: I'm finding the action that maximizes the sum, for all states, of the equation following the sigma?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here you have to iterate through all the actions &lt;em&gt;a&lt;/em&gt; in state &lt;em&gt;s&lt;/em&gt; and calculate the sum according to the formula in the max function. Then you take the &lt;strong&gt;highest value&lt;/strong&gt; and set the new &lt;em&gt;V(s)&lt;/em&gt; to be this value.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Line 20: I'm interpreting this a the end of the do-while. But what is this condition? Am I checking if there is an s such that this condition applies? So Would I would have to loop between all states and stop if any state satisfies the condition? (Basically a loop with a break, instead of a while)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You have to check for &lt;strong&gt;all&lt;/strong&gt; &lt;em&gt;s&lt;/em&gt; if the condition satisfies and you may only break the loop if all these values are lower than the threshold &lt;em&gt;θ&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastActivityDate="2018-04-18T08:48:53.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6097" PostTypeId="2" ParentId="5874" CreationDate="2018-04-18T09:38:43.747" Score="2" Body="&lt;p&gt;I don't want to pour cold water over your approach, but I am very sceptical and (having worked in sentiment analysis myself) think it is way too simplistic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Various communicative intents are encoded in language, and there is a wide range of linguistic features that are employed for that purpose. Choice of words is only one of them; it is the most obvious one, as we can easily see the words themselves. But words in isolation do not mean anything, context is important. It is of course not difficult to come up with example sentences where the sentiment effect of the words you list is reversed. The easy one being negation: &lt;em&gt;I'm not happy about this.&lt;/em&gt; Sure, you can check if there is a &lt;em&gt;not&lt;/em&gt; before the word, but what about &lt;em&gt;I would be happy if you stopped making such a noise.&lt;/em&gt; -- surely here the current state would be one of unhappiness? If you think about real examples, it suddenly becomes very complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, words usually have multiple meanings: &lt;em&gt;This cup is just shy of one litre.&lt;/em&gt; I'm sure you'd agree that this does not express 'fear'. And &lt;em&gt;The shunter moved the tender on the old steam engine.&lt;/em&gt; is not about affection. But solving this problem involves word sense disambiguation, which in itself is a hard problem to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, initially word-based approaches look really good and transparent, as you can easily see what's going on. But language unfortunately doesn't play ball, and in real life systems don't tend to work very well. Lexical choice is only one way to encode sentiment, there are also grammatical patterns. But these are often very subtle, and not yet well-explored in linguistic research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To end on a positive note, have a look at research in evaluation (which is kind of related to sentiment). For example, Susan Hunston's &lt;em&gt;Corpus Approaches to Evaluation&lt;/em&gt;,  (Routledge 2011). That should give you some further pointers.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-04-18T09:38:43.747" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6098" PostTypeId="1" CreationDate="2018-04-18T09:53:43.113" Score="3" ViewCount="114" Body="&lt;p&gt;I'm trying to implement a tree sort for 8 numbers. I've created 15 tree nodes agents and one manager agent. What I'm trying to achieve is to synchronize leaves in the tree and send their generated numbers to manager node. My problem is that in my for cycle, the leaves are not synchronized so manager is waiting for next leaf but that leaf may have sent it's number already. Is there a way to synchronize these leaves, so that manager waits for each leaf and then prints out received number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the agents:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;treeSort.mas2j&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MAS treeSort {&#xA;    infrastructure: Centralised&#xA;    agents:&#xA;        manager;&#xA;        agent#15;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;manager.asl:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!start.&#xA;+!start : true &amp;lt;-&#xA;    for ( .range(I,8,15) ) {&#xA;        .concat(&quot;agent&quot;,I,TempAgent);&#xA;        .print(&quot;waiting from &quot;,TempAgent);&#xA;        .term2string(YT,TempAgent);&#xA;        .wait(recievedNum(YT,X));&#xA;        .print(&quot;from &quot;,YT,&quot; recvd &quot;,X);    &#xA;    };&#xA;    .println(&quot; done&quot;).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;agent.asl&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!start.&#xA;&#xA;@p1 +!start : .my_name(agent8) | .my_name(agent9)| .my_name(agent10)|.my_name(agent11)&#xA;        | .my_name(agent12) | .my_name(agent13) | .my_name(agent14) |.my_name(agent15) &amp;lt;- +myNum(math.round(math.random(100)));&#xA;    ?myNum(X);&#xA;    +iam(list);&#xA;    .my_name(Y);&#xA;    .send(manager, tell, recievedNum(Y,X)).&#xA;&#xA;@p2 +!start : .my_name(agent7) | .my_name(agent6)|  .my_name(agent5) |.my_name(agent4)&#xA;        | .my_name(agent3) | .my_name(agent2) &amp;lt;- +iam(node).&#xA;&#xA;@p3 +!start : .my_name(agent1) &amp;lt;- +iam(root).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the output of program: &lt;a href=&quot;https://imgur.com/a/ewOAuxv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/ewOAuxv&lt;/a&gt; . As you can see, the manager is waiting for number from agent9 bu because he already got that number from him waiting for agent8, he will now wait forever. There is managers database: &lt;a href=&quot;https://imgur.com/a/ZZ35pD9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/ZZ35pD9&lt;/a&gt; . &lt;/p&gt;&#xA;" OwnerUserId="15105" LastEditorUserId="15105" LastEditDate="2018-04-19T17:45:59.677" LastActivityDate="2018-04-19T17:45:59.677" Title="Jason/AgentSpeak trying to synchronize agents" Tags="&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2018-08-25T15:13:17.183" ContentLicense="CC BY-SA 3.0" />
  <row Id="6099" PostTypeId="1" AcceptedAnswerId="6104" CreationDate="2018-04-18T10:36:13.770" Score="15" ViewCount="2956" Body="&lt;p&gt;Does the human brain use a specific activation function? I've tried doing some research, and as it's a treshold for whether the signal is sent through a neuron or not, it sounds a lot like ReLU. However, I can't find a single article confirming this. Or is it more like a step function (it sends 1 if it's above the treshold, instead of the input value).&lt;/p&gt;&#xA;" OwnerUserId="15107" LastActivityDate="2020-11-03T09:51:54.090" Title="What activation function does the human brain use?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;brain&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="6101" PostTypeId="2" ParentId="6099" CreationDate="2018-04-18T11:58:55.093" Score="1" Body="&lt;p&gt;The answer is &lt;strong&gt;We do not know&lt;/strong&gt;. Odds are, we will not know for quite a while. The reason for this is we cannot understand the &quot;code&quot; of the human brain, nor can we simply feed it values and get results. This limits us to measuring currents of the input and output on test subjects, and we have had few such test subjects that are &lt;em&gt;human&lt;/em&gt;. Thus, we know almost nothing about the human brain, including the activation function.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-04-18T11:58:55.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6102" PostTypeId="1" CreationDate="2018-04-18T12:01:49.907" Score="10" ViewCount="594" Body="&lt;p&gt;Imagine a game where it is a black screen apart from a red pixel and a blue pixel. Given this game to a human, they will first see that pressing the arrow keys will move the red pixel. The next thing they will try is to move the red pixel onto the blue pixel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Give this game to an AI, it will randomly move the red pixel until a million tries later it accidentally moves onto the blue pixel to get a reward. If the AI had some concept of distance between the red and blue pixel, it might try to minimize this distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without actually programming in the concept of distance, if we take the pixels of the game can we calculate a number(s), such as &quot;entropy&quot;, that would be lower when pixels are far apart than when close together? It should work with other configurations of pixels. Such as a game with three pixels where one is good and one is bad. Just to give the neural network more of a sense of how the screen looks? Then give the NN a goal, such as &quot;try to minimize the entropy of the board as well as try to get rewards&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything akin to this in current research?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2019-10-22T21:23:01.800" LastActivityDate="2019-10-22T21:23:01.800" Title="Can a neural network work out the concept of distance?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;philosophy&gt;&lt;game-ai&gt;" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6104" PostTypeId="2" ParentId="6099" CreationDate="2018-04-18T14:21:39.670" Score="14" Body="&lt;p&gt;The thing you were reading about is known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Action_potential&quot; rel=&quot;noreferrer&quot;&gt;action potential&lt;/a&gt;. It is a mechanism that governs how information flows within a neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works like this: Neurons have an electrical potential, which is a voltage difference inside and outside the cell. They also have a default resting potential, and an activation potential. The neuron tends to move towards the resting potential if it is left alone, but incoming electric activations from dendrites can shift its electric potential. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the neuron reaches a certain threshold in electric potential (the activation potential), the entire neuron and its connecting axons goes through a chain reaction of ionic exchange inside/outside the cell that results in a &quot;wave of propagation&quot; through the axon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR: Once a neuron reaches a certain activation potential, it electrically discharges. But if the electric potential of the neuron doesn't reach that value then the neuron does not activate.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the human brain use a specific activation function? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;IIRC neurons in different parts of the brain behave a bit differently, and the way this question is phrased sounds as if you are asking if there is a specific implementation of neuronal activation (as opposed to us modelling it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in general behave relatively similar to each other (Neurons communicate with each other via neurochemicals, information propagates inside a neuron via a mechanism known as the action potential...) But the details and the differences they cause could be significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various &lt;a href=&quot;https://en.wikipedia.org/wiki/Biological_neuron_model&quot; rel=&quot;noreferrer&quot;&gt;biological neuron models&lt;/a&gt;, but the &lt;a href=&quot;http://neuronaldynamics.epfl.ch/online/Ch2.S2.html&quot; rel=&quot;noreferrer&quot;&gt;Hodgkin-Huxley Model&lt;/a&gt; is the most notable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that a general description of neurons don't give you a general description of neuronal dynamics a la cognition (understanding a tree doesn't give you complete understanding of a forest)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, the method of which information propagates inside a neuron is in general quite well understood as sodium / potassium ionic exchange.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It (activation potential) sounds a lot like ReLU...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's only like ReLU in the sense that they require a threshold before anything happens. But ReLU can have variable output while neurons are all-or-nothing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also ReLU (and other activation functions in general) are differentiable with respect to input space. This is very important for backprop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a ReLU function, with the X-axis being input value and Y-axis being output value.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vD7Ut.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vD7Ut.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is the action potential with the X-axis being time, and Y being output value.&lt;a href=&quot;https://i.stack.imgur.com/EsnF7.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EsnF7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2018-05-04T22:09:14.397" LastActivityDate="2018-05-04T22:09:14.397" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6106" PostTypeId="5" CreationDate="2018-04-18T16:25:34.163" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-04-18T16:25:34.163" LastActivityDate="2018-04-18T16:25:34.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6107" PostTypeId="4" CreationDate="2018-04-18T16:25:34.163" Score="0" Body="This tag is for questions relating to the language and models that create them." OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2018-04-18T21:34:46.677" LastActivityDate="2018-04-18T21:34:46.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6108" PostTypeId="2" ParentId="6102" CreationDate="2018-04-18T16:45:34.437" Score="1" Body="&lt;h1&gt;Answer&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm going to take your question at face value, and go really deep into this topic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, they can. The typical human mind can. But consider the human mind. Millions, if not &lt;em&gt;billions&lt;/em&gt;, of neurons. In fact, one can consider distance as a human concept, simply a theory developed from interactions with the world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, given a year or two, with a ton of neurons on your hand, you could replicate this scenario. That is if your computer is as parallel as the human mind. The short explanation is that the human mind is very parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it would be simpler to calculate the distance with a program, not an AI, and simply feed the result to the AI that would make the decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the amount of time you have spent looking at a screen. If you can tell the (approximate) distance between two pixels, so can a Neural Network, as you are one. However, add the amount of time you have spent alive and learning into the equation, and it becomes a disaster. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;The human brain is parallel&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is a result of the fact that all of the neurons in the human brain are independent of each other. They can run &lt;em&gt;true simultaneous&lt;/em&gt; actions, thus making the action of interpreting images and such much easier, as blocks of neurons can &quot;think&quot; independent of the operations of the others, limiting what would be &quot;lag&quot; to a minuscule amount.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-04-18T16:45:34.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6109" PostTypeId="2" ParentId="6052" CreationDate="2018-04-18T17:26:39.433" Score="1" Body="&lt;p&gt;I think a worthwhile extension of this line of thought is &quot;why not both?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not believe there is anything preventing approaching the problem from both sides at once. There is a great deal of research on both sides (biological research and computational research), but considerably less on the integration of the two (although there certainly is some, such as in the development of modern prosthetics that allow some degree of control).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the adaptability of the human brain in terms of adjusting its own structure, the most expedient approach may be to consider what it would take to create a non-biological medium that biological neurons could interface with sufficiently to essentially &quot;program&quot; them in the same manner it does when repairing itself with biological neurons. Leave the hard work to the thing that already has the blueprint. Or in other words, the Ship of Theseus but with brain cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not that such a task would be anything close to approaching simple or easy, given our still lacking knowledge of neurological structures and the difficulties in getting a non-biological interface that is capable of the required sort of communications and adjustments that biological neurons can have performed and on a size scale that would be practical. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish I could point to some research related to this, but I don't know about any specific research papers, although I know it's not a completely untouched upon subject. &lt;/p&gt;&#xA;" OwnerUserId="15114" LastActivityDate="2018-04-18T17:26:39.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6110" PostTypeId="2" ParentId="6102" CreationDate="2018-04-18T17:33:12.647" Score="1" Body="&lt;p&gt;You can create AI to &quot;see&quot; as a human. As you said, giving the human the keys, he will click randomly. He just needs to know which keys he presses that brings him closer to other objects on the screen. I think the basics of an AI is object recognition. I would try to create a script to map the screen objects of the game. There are legal examples in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try to follow a path like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Make the AI ​​understand that by clicking the arrows or the WASD and it is in the context GAME, the object that move pixels according to the direction, represents the main author (the player).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In parallel: map all boundaries of the region and index different objects within that region to automatically have the coordinate domain and object distance. AI needs to SEE (stream) the game and through images to categorize objects. Do you understand what I mean?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In parallel: The AI ​​needs to be aware of all texts and information that is on the screen (all mapped, remember?). You need to understand when a text changes or something different happens. For example: whenever he returns to the initial position of each phase, whenever he has a count, what happens when the cout reaches zero or a common number that generates another type of change.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;He needs to understand what is repeated at every &quot;respawn&quot;. You also need to understand what &quot;respawn&quot; is. Maybe a certain map position on every map it returns whenever a count on the screen ends. Or when it comes up against a certain type of object (mapped object)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To be honest, if you want to create a super intelligent robot, you can go following all the steps that go through the heads of different humans, or the best humans, or the rules of each game. But sometimes it's easier to build specific bots to perform specific tasks. It depends on what you want to do&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-04-18T17:33:12.647" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6111" PostTypeId="1" AcceptedAnswerId="6113" CreationDate="2018-04-18T18:09:17.847" Score="1" ViewCount="592" Body="&lt;p&gt;In the &lt;a href=&quot;https://startup.jobs/3993-machine-learning-fellow-at-openai&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI's Machine Learning Fellow position&lt;/a&gt;, it is written&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We look for candidates with one or more of the following credentials:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;li&gt;Open-source reimplementations of deep learning algorithms which replicate performance from the papers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What exactly do they mean by this? Do they want us to implement the algorithms exactly as described in the papers (i.e. with the same hyper-parameters, weights, etc.)?&lt;/p&gt;&#xA;" OwnerUserId="15116" LastEditorUserId="2444" LastEditDate="2020-11-15T15:14:23.063" LastActivityDate="2020-11-15T16:33:41.437" Title="What does &quot;reimplementations of deep learning algorithms which replicate performance from the papers&quot; mean?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;research&gt;&lt;implementation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6112" PostTypeId="2" ParentId="6111" CreationDate="2018-04-18T18:30:34.860" Score="1" Body="&lt;p&gt;I believe that this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&quot;reimplementations of deep learning algorithms&quot;&lt;/strong&gt;&#xA;He is asking that they are looking for people who have made and completed an AI that perform similarly or exactly as those given in papers (IDK which papers)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;&quot;Open-source&quot;&lt;/strong&gt; The Fellow wants to be able to see the source code of the project.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-04-18T18:30:34.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6113" PostTypeId="2" ParentId="6111" CreationDate="2018-04-18T19:03:46.427" Score="1" Body="&lt;p&gt;Accompanying ML/AI (and computer science, in general) papers with the relevant code is much desirable, both for reproducibility of the paper results as well as for faster integration of these results into applications.&lt;/p&gt;&#xA;&lt;p&gt;Some researchers do so by themselves; but this has lately become also a community job, where a third, independent individual or team, not affiliated with the paper author(s), offers a code implementation starting from the paper.&lt;/p&gt;&#xA;&lt;p&gt;A good, very recent case is the reproduction of the &lt;a href=&quot;https://arxiv.org/abs/1803.10122&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Models&lt;/a&gt; paper (&lt;a href=&quot;https://medium.com/applied-data-science/how-to-build-your-own-world-model-using-python-and-keras-64fb388ba459&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, &lt;a href=&quot;https://github.com/AppliedDataSciencePartners/WorldModels&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repo&lt;/a&gt;). Many of the implementations available at &lt;a href=&quot;https://paperswithcode.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papers with Code&lt;/a&gt; are indeed provided by the community.&lt;/p&gt;&#xA;&lt;p&gt;A great write-up of such an effort, containing many lessons learned, made big raves recently: &lt;a href=&quot;http://amid.fish/reproducing-deep-rl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lessons Learned Reproducing a Deep Reinforcement Learning Paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Understandably, individuals that have such efforts in their CV are a good recruiting option...&lt;/p&gt;&#xA;" OwnerUserId="11539" LastEditorUserId="11539" LastEditDate="2020-11-15T16:33:41.437" LastActivityDate="2020-11-15T16:33:41.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6115" PostTypeId="1" CreationDate="2018-04-19T06:22:13.080" Score="4" ViewCount="162" Body="&lt;p&gt;I am working on an anti-fraud project. In the project, we are trying to predict the fraud user in the out time data set. But the fraud user has a very low ratio, only 3%. We expect a model with a precision more than 15%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried Logistic Regression, GBDT+LR, xgboost. All models are not good enough. Step wise Logistic Regression performs best, which has a precision of 9% with recall rate 6%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other models that I can use for this problem or any other advise ?&lt;/p&gt;&#xA;" OwnerUserId="15126" LastEditorUserId="12542" LastEditDate="2018-04-19T07:57:38.437" LastActivityDate="2018-04-20T05:44:58.960" Title="What model to use for fully unbalanced data?" Tags="&lt;ai-design&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6116" PostTypeId="1" CreationDate="2018-04-19T08:32:55.490" Score="3" ViewCount="500" Body="&lt;p&gt;I tried to build a neural network from scratch to build a cat or dog binary classifier using a sigmoid output unit. I seem to get the output value around 0.5(+/- 0.002) for every input. This seems really weird to me. Here's my code, Please let me know if there is a mistake in the implementation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def initialize_parameters_deep(layer_dims):&#xA;    l=len(layer_dims)&#xA;    parameters={}&#xA;    for l in range(1,len(layer_dims)):&#xA;        parameters['W'+str(l)]=np.random.randn(layer_dims[l],layer_dims[l-1])*0.01&#xA;        parameters['b'+str(l)]=np.zeros((layer_dims[l],1))&#xA;    return parameters&#xA;&#xA;def linear_forward(A,W,b):&#xA;    Z=np.dot(W,A)+b&#xA;    cache=(A,W,b)&#xA;    return Z,cache&#xA;&#xA;&#xA;def sigmoid(Z):&#xA;    A = 1/(1+np.exp(-Z))&#xA;    cache=Z&#xA;    return A, cache&#xA;&#xA;&#xA;def relu(Z):&#xA;    A = np.maximum(0,Z)&#xA;&#xA;    assert(A.shape == Z.shape)&#xA;&#xA;    cache = Z &#xA;    return A, cache&#xA;&#xA;def relu_backward(dA, cache):&#xA;    Z = cache&#xA;    dZ = np.array(dA, copy=True) # just converting dz to a correct object.&#xA;&#xA;    # When z &amp;lt;= 0, you should set dz to 0 as well. &#xA;    dZ[Z &amp;lt;= 0] = 0&#xA;&#xA;    assert (dZ.shape == Z.shape)&#xA;&#xA;    return dZ&#xA;&#xA;def sigmoid_backward(dA, cache):&#xA;    Z = cache&#xA;&#xA;    s = 1/(1+np.exp(-Z))&#xA;    dZ = dA * s * (1-s)&#xA;&#xA;    assert (dZ.shape == Z.shape)&#xA;&#xA;    return dZ&#xA;&#xA;&#xA;def linear_activation_forward(A_prev,W,b,activation):&#xA;    if(activation=='sigmoid'):&#xA;        Z,linear_cache=linear_forward(A_prev,W,b)&#xA;        A,activation_cache=sigmoid(Z)&#xA;    elif activation=='relu':&#xA;        Z,linear_cache=linear_forward(A_prev,W,b)&#xA;        A,activation_cache=relu(Z)&#xA;    cache=(linear_cache,activation_cache)&#xA;    return A,cache&#xA;&#xA;def L_model_forward(X,parameters):&#xA;    A=X&#xA;    L=len(parameters)//2&#xA;    caches=[]&#xA;    for l in range(1,L):&#xA;        A,cache=linear_activation_forward(A,parameters['W'+str(l)],parameters['b'+str(l)],'relu')&#xA;        caches.append(cache)&#xA;    AL,cache=linear_activation_forward(A,parameters['W'+str(L)],parameters['b'+str(L)],'sigmoid')&#xA;    caches.append(cache)&#xA;    return AL,caches&#xA;&#xA;def compute_cost(AL,Y):&#xA;    m=Y.shape[1]&#xA;    cost=-1/m*np.sum(np.multiply(np.log(AL),Y)+np.multiply(np.log(1-AL),1-Y))&#xA;    return cost&#xA;&#xA;def linear_backward(dZ,cache):&#xA;    A_prev,W,b=cache&#xA;    m=A_prev.shape[1]&#xA;    dW = np.dot(dZ,A_prev.T)/m&#xA;    db = np.sum(dZ,axis=1,keepdims=True)/m&#xA;    dA_prev = np.dot(W.T,dZ)&#xA;    return dA_prev,dW,db&#xA;&#xA;def linear_activation_backward(activation,dA_prev,cache):&#xA;    linear_cache,activation_cache=cache&#xA;    if activation=='sigmoid':&#xA;&#xA;        dZ=sigmoid_backward(dA_prev,activation_cache)&#xA;        dA_prev,dW,db=linear_backward(dZ,linear_cache)&#xA;    if activation=='relu':&#xA;        dZ=relu_backward(dA_prev,activation_cache)&#xA;        dA_prev,dW,db=linear_backward(dZ,linear_cache)&#xA;    return dA_prev,dW,db&#xA;&#xA;def L_model_backward(AL,Y,caches):&#xA;    L=len(caches)&#xA;    m = AL.shape[1]&#xA;    Y = Y.reshape(AL.shape)&#xA;    dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))&#xA;&#xA;    grads={}&#xA;    current_cache=caches[-1]&#xA;    grads['dA'+str(L-1)],grads['dW'+str(L)],grads['db'+str(L)]=linear_activation_backward('sigmoid',dAL,current_cache)&#xA;&#xA;    for l in reversed(range(L-1)):&#xA;        current_cache=caches[l]&#xA;        dA_prev_temp, dW_temp, db_temp = linear_activation_backward('relu',grads['dA'+str(l+1)],current_cache)&#xA;        grads[&quot;dA&quot; + str(l)] = dA_prev_temp&#xA;        grads[&quot;dW&quot; + str(l + 1)] = dW_temp&#xA;        grads[&quot;db&quot; + str(l + 1)] = db_temp&#xA;    return grads&#xA;def Grad_Desc(parameters,grads,learning_rate):&#xA;    L=len(parameters)//2&#xA;    for l in range(L):&#xA;        parameters['W'+str(l+1)]=parameters['W'+str(l+1)]-learning_rate*grads['dW'+str(l+1)]&#xA;        parameters['b'+str(l+1)]=parameters['b'+str(l+1)]-learning_rate*grads['db'+str(l+1)] &#xA;    return parameters&#xA;&#xA;def L_layer_model(X,Y,learning_rate,num_iter,layer_dims):&#xA;    parameters=initialize_parameters_deep(layer_dims)&#xA;    costs=[]&#xA;    for i in range(num_iter):&#xA;        AL,caches=L_model_forward(X,parameters)&#xA;        cost=compute_cost(AL,Y)&#xA;        grads=L_model_backward(AL,Y,caches)&#xA;        parameters=Grad_Desc(parameters,grads,learning_rate)&#xA;        if i%100==0:&#xA;            print(cost)&#xA;            costs.append(cost)&#xA;    plt.plot(np.squeeze(costs))&#xA;def predict(X,parameters):&#xA;    AL,caches=L_model_forward(X,parameters)&#xA;    prediction=(AL&amp;gt;0.5)&#xA;    return AL,prediction&#xA;&#xA;L_layer_model(x_train,y_train,0.0075,12000,[12288,20,7,5,1])&#xA;prediction=predict(x_train,initialize_parameters_deep([12288,20,7,5,1])) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15128" LastEditorUserId="15122" LastEditDate="2018-04-19T18:54:40.377" LastActivityDate="2018-04-25T04:23:48.983" Title="Neural network returns about the same output(mean) for every input" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="12" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6117" PostTypeId="1" CreationDate="2018-04-19T10:41:16.887" Score="7" ViewCount="156" Body="&lt;p&gt;How can we attach numerical value to the measurement of soft skill? Is there any book, site or materials that can teach me how to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know if there is any way we can measure soft skill as listed  &lt;a href=&quot;https://training.simplicable.com/training/new/87-soft-skills&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and say a person  has (over 100) 70% or 60% in critical thinking ability or his critical thinking ability is over 72% and be accurate with the score.&lt;/p&gt;&#xA;" OwnerUserId="15132" LastEditorUserId="1671" LastEditDate="2018-10-15T17:33:52.003" LastActivityDate="2019-01-02T20:02:47.100" Title="Measuring Soft Skill" Tags="&lt;theory&gt;&lt;reference-request&gt;&lt;problem-solving&gt;&lt;reasoning&gt;&lt;quantification&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6118" PostTypeId="1" CreationDate="2018-04-19T11:05:26.483" Score="3" ViewCount="675" Body="&lt;p&gt;One of my friends built a version of &quot;Achtung, Die Kurve!&quot;, or &quot;Curve Fever&quot; in Python. I was starting to study ML and decided to tackle the game from a learning perspective - write a bot that would crush him in the game. Did some research and found Deep Q learning. Decided to go with that and after a whole lot of throwing around different hyperparameters and layers, I decided I need some help on this. I am new to Deep and Machine Learning in general, so I may have missed things. I was kinda discouraged when I saw that Deep Q is SO impractical currently in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how would you guys tackle this problem? I need some guidance/help building it if someone is up to the task.&lt;/p&gt;&#xA;" OwnerUserId="15133" LastActivityDate="2018-04-19T18:02:50.950" Title="How should I approach the game &quot;Achtung, Die Kurve&quot; (&quot;Curve Fever&quot;) using AI?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;python&gt;" AnswerCount="1" CommentCount="11" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6120" PostTypeId="2" ParentId="6115" CreationDate="2018-04-19T17:53:44.260" Score="1" Body="&lt;h1&gt;You can balance your data-set.&lt;/h1&gt;&#xA;&lt;p&gt;Many models work with batches of samples. If you have a very unbalanced dataset, you can simply split it and ensure your batches are balanced (for example, for a Neural Network, using minibatches of 32 samples, you could draw 16 from your fraud users, and 16 from non-fraud users).&lt;/p&gt;&#xA;&lt;p&gt;During the learning phase, this ensures the model doesn't just output the most common class, but instead tries to learn to distinguish both.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-04-19T17:53:44.260" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6121" PostTypeId="2" ParentId="6118" CreationDate="2018-04-19T18:02:50.950" Score="2" Body="&lt;h1&gt;Start slowly.&lt;/h1&gt;&#xA;&lt;p&gt;Don't jump straight into Deep Learning, arguably the most complex class in Reinforcement Learning techniques. First work with simpler algorithms, like the original Q-Learning. Define what are good inputs and outputs for your game, and start tuning some hyper-parameters (like future rewards discount factor).&lt;/p&gt;&#xA;&lt;p&gt;From there, go for Deep Learning. Check other implementations (like &lt;a href=&quot;https://github.com/devsisters/DQN-tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN&lt;/a&gt;, &lt;a href=&quot;https://github.com/bluemoon93/DistributedAtariQLearning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atari n-step Q-Learning&lt;/a&gt; and &lt;a href=&quot;https://github.com/openai/universe-starter-agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;A3C&lt;/a&gt;), and adapt their code to yours, rather than starting from scratch.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-04-19T18:02:50.950" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6124" PostTypeId="2" ParentId="6115" CreationDate="2018-04-20T05:44:58.960" Score="0" Body="&lt;p&gt;Heavily imbalanced classification tasks do not need a certain type of model, you can get different ones to work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have two options: either use class weights (for example setting them to 'balanced' in the &lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ScikitLearn SVM&lt;/a&gt;) in order to indicate that samples from a class are more important (the underrepresented one) or rebalance your dataset. For rebalancing purposes, and assuming you are using Python, I recommend &lt;a href=&quot;https://github.com/scikit-learn-contrib/imbalanced-learn&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imbalanced Learn&lt;/a&gt;. There you have algorithms for over-sampling, under-sampling, over-sampling followed by under-sampling or ensemble sampling. If you use them, please check the plausibility of the synthetic samples you created by reducing dimensionality first and then plotting them for example in two dimensions. Are the synthetic samples similar to the true class?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend you to think about relevant metrics for (heavily) imbalanced problems and consider the no-information rate. That is another question though.&lt;/p&gt;&#xA;" OwnerUserId="7495" LastActivityDate="2018-04-20T05:44:58.960" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6125" PostTypeId="1" AcceptedAnswerId="6127" CreationDate="2018-04-20T10:49:59.647" Score="1" ViewCount="149" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IzJ5Q.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IzJ5Q.jpg&quot; alt=&quot;Picture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I know that 'h' and 'f' will be pruned, but I'm not sure about 'k' and 'l'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we visit 'j', technically there is no need for us to visit 'k' and 'l' because there are 2 options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;one or two of them might be higher than 8 ('j') &lt;/li&gt;&#xA;&lt;li&gt;both of them less than 8 &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But no matter what, the decision of the max(root) will not change, the max will choose the right side no matter what 'k' and 'l' are, because the right side will either be 8 or 9, which is still higher than 4 (returned value from left side)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so will alpha beta prune 'k' and 'l' or not? if not, then it means alpha beta is not &quot;optimal&quot; overall right? considering it will not prune all the unnecessary paths.&lt;/p&gt;&#xA;" OwnerUserId="12782" LastEditorUserId="12782" LastEditDate="2018-04-20T13:57:30.063" LastActivityDate="2018-06-19T17:49:50.150" Title="Which edges of this tree will be pruned by Alpha-beta pruning?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;logic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6126" PostTypeId="1" AcceptedAnswerId="6131" CreationDate="2018-04-20T12:30:53.137" Score="5" ViewCount="1404" Body="&lt;p&gt;In Monte Carlo Tree Search (MCTS), we start at root node &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. Then we select some leaf node &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. And we expand &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; by one or more child nodes and simulate from the child to the end of the game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bQH9b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bQH9b.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When should we expand and when should we simulate in MCTS? Why not expand 2 or 3 levels, and simulate from there, then back-propagate the values to the top?  Should we just expand 1 level? Why not more?&lt;/p&gt;&#xA;" OwnerUserId="15152" LastEditorUserId="2444" LastEditDate="2019-11-19T20:18:29.760" LastActivityDate="2019-11-19T20:18:29.760" Title="When to expand and when to simulate in Monte Carlo Tree Search?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6127" PostTypeId="2" ParentId="6125" CreationDate="2018-04-20T16:36:58.253" Score="1" Body="&lt;p&gt;If you prune k and L then you could miss the optimal solution. Assume L=9, if you prune L then the value of the tree is 8. If you don't prune L then the value of the tree is 9. Now I will try and address what I think your actual question is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But no matter what, the decision of the max(root) will not change, the max will choose the right side no matter what 'k' and 'l' are, because the right side will either be 8 or 9, which is still higher than 4 (returned value from left side).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From this sentence it seems like you don't care about the value of the tree, you only want to find the optimal first move based on alpha beta. You are correct in saying that the correct first move will always contain the right most child of the root, but oftentimes that is not the only information we want. Sometimes we want to know the value of the tree as well or what the complete correct path is, but if we had pruned k and L we would not know these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I have changed all 'L's to uppercase, because lower case 'L' looks to much like uppercase 'i'&lt;/p&gt;&#xA;" OwnerUserId="13088" LastActivityDate="2018-04-20T16:36:58.253" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6129" PostTypeId="1" CreationDate="2018-04-21T02:43:55.633" Score="2" ViewCount="575" Body="&lt;p&gt;I am training &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;pre-trained SSD-InceptionV2-Coco&lt;/a&gt; to detect the &quot;car&quot;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is one of the classes in &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/data/mscoco_label_map.pbtxt&quot; rel=&quot;nofollow noreferrer&quot;&gt;mscoco label&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I train the model with ~50k sample from KITTI, 500k iteration with batch size 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/dataset_tools/create_kitti_tf_record.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this script&lt;/a&gt; to generate tfrecord file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I test both original pre-trained model and my trained model with one video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The performance of my trained model is worse. More missing detected results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I found recently is the classification_loss/localization_loss increases when AvgNumGroundtruthBoxesPerImage increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EjMbi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EjMbi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g8Sr1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g8Sr1.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PLbux.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PLbux.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing I found is the more ground truth boxes per image I have, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the less average num positive anchors per image I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This bothers me because if the number of anchors generated per image is fixed,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;more ground truth boxes should provide more positive anchors per image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I wonder where to find the root cause.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion is welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for precious time on my question.&lt;/p&gt;&#xA;" OwnerUserId="15162" LastEditorUserId="15162" LastEditDate="2018-04-28T23:41:11.890" LastActivityDate="2018-04-28T23:41:11.890" Title="Getting worse performance when training a pre-trained model with the existing class" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6130" PostTypeId="1" CreationDate="2018-04-21T03:54:58.987" Score="1" ViewCount="62" Body="&lt;p&gt;Everything from facial recognition to the google home is coming equiped with A.I and it is being widely used , If autonomously connected to the internet , will A.I pose a threat to privacy or will it endanger free will if used for surveillance with facial recognition , like in the movie 'Minority Report'&lt;/p&gt;&#xA;" OwnerUserId="15164" LastActivityDate="2018-04-21T03:54:58.987" Title="Will commercialisation and widespread use of A.I in security and surveillance and other household products threaten free will or endanger privacy?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;autonomous-vehicles&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6131" PostTypeId="2" ParentId="6126" CreationDate="2018-04-21T08:22:17.980" Score="6" Body="&lt;p&gt;The most common strategy is to simply expand exactly one node per iteration; you can view this as expanding the first node of the Play-Out phase (&quot;simulation&quot; in your image), and not expanding any other nodes of the Play-Out phase. This is also what's done in your image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is the most common and probably most simple strategy, but it's certainly not the only one. It is pretty much the minimum you have to expand, but you can expand more if you like. The most common approach of only expanding one node per iteration is basically there because it minimizes the risk of running out of memory; when you only add one new node to the tree per iteration, the tree grows relatively slowly, so you'd have to keep the algorithm running for a really long time before you run out of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not afraid of running out of memory, you can choose to expand as many nodes as you like. For example, &lt;a href=&quot;https://github.com/DennisSoemers/MaastCTS2&quot; rel=&quot;noreferrer&quot;&gt;my General Video Game AI agent&lt;/a&gt; expands every single node encountered during the play-out at once, because in that particular domain we only get to run relatively few iterations anyway so I'm not afraid of running out of memory, even if I do expand lots of nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The backpropagation step can only store results (information) from your iterations in nodes that actually exist, nodes that have been expanded. So, the benefit of expanding more (if possible) is that you'll retain more information from early iterations, your backpropagation step immediately gets to store results in all nodes along the path instead of only the top few nodes. Generally, this is a relatively minor benefit; nodes deep in the tree / deep inside the simulations are quite unlikely to ever get visited more than once anyway, and then this doesn't matter. But expanding more can, in theory, result in slightly more accurate value estimates in nodes closer to the root.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-04-21T08:22:17.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6132" PostTypeId="5" CreationDate="2018-04-21T08:35:19.463" Score="0" Body="&lt;p&gt;A Markov chain is &quot;a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Chain - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-23T15:58:37.620" LastActivityDate="2018-04-23T15:58:37.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6133" PostTypeId="4" CreationDate="2018-04-21T08:35:19.463" Score="0" Body="For questions about the use of Markov models in the field of AI/ML." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-04-23T15:58:40.420" LastActivityDate="2018-04-23T15:58:40.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6134" PostTypeId="1" AcceptedAnswerId="6169" CreationDate="2018-04-21T11:25:10.503" Score="1" ViewCount="78" Body="&lt;p&gt;A number has randomly been chosen from 1 to 3. On each step, we can make a guess and we will be told if our guess is equal, bigger or smaller than the chosen number. We're trying to find the number with the least number of guesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to draw the MDP model for this question with 7 states, but I don't know how the states are supposed to be defined. Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="14603" LastEditorUserId="2444" LastEditDate="2020-05-29T21:42:15.117" LastActivityDate="2020-05-29T21:42:15.117" Title="How should I define an MDP for this problem where we need to guess a number and minimise the number of guesses?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6135" PostTypeId="1" AcceptedAnswerId="6136" CreationDate="2018-04-21T13:51:56.900" Score="1" ViewCount="290" Body="&lt;p&gt;Suppose one is using a multi-armed bandit, and one has relatively few &amp;quot;pulls&amp;quot; (i.e. timesteps) relative to the action set. For example, maybe there are 200 timesteps and 100 possible actions.&lt;/p&gt;&#xA;&lt;p&gt;However, you do have information on how similar actions are to each other. For example, I might want to rent a car, and know the model, year, and mileage of each car. (Specifically, I want to rent a car on a daily basis for each day in a 200 day period; on each day, I can either continue with the existing car or rent a new one. There are 100 possible cars.)&lt;/p&gt;&#xA;&lt;p&gt;How can I exploit this information to choose actions that maximize my payoff?&lt;/p&gt;&#xA;" OwnerUserId="12656" LastEditorUserId="2444" LastEditDate="2021-01-18T01:33:15.217" LastActivityDate="2021-01-18T10:47:05.937" Title="How can I incorporate domain knowledge to choose actions in the case of large action spaces in multi-armed bandits?" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;action-spaces&gt;&lt;contextual-bandit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6136" PostTypeId="2" ParentId="6135" CreationDate="2018-04-21T19:24:15.090" Score="3" Body="&lt;p&gt;You will want to look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit#Contextual_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contextual Multi-Armed Bandits&lt;/a&gt;. These are MAB problems that additionally involve feature vectors in some way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll sometimes see researchers considering problems where you get to see a single feature vector per timestep (like an &quot;environment state&quot; you're in) which may provide useful information. You'll also sometimes see researchers considering problems where you observe a single feature vector per arm/action (per timestep). That second case is pretty much what you're describing; every arm/action has a feature vector (model, year, mileage of car), and you can use those to make predictions and generalize across arms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give a little bit of flavour for these algorithms, the most common ones simply assume that the rewards you observe after pulling arms are given by a linear combination of features, plus some noise. Then, they simply try to find (through online learning) a parameter vector such that observed rewards can be accurately approximated by the dot product of the parameter vector and a feature vector. Of course, there are lots of different variants of algorithms, and not all are linear, but this gives an idea of what they generally look like.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-04-21T19:24:15.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6137" PostTypeId="1" CreationDate="2018-04-22T08:17:57.023" Score="1" ViewCount="177" Body="&lt;p&gt;Previously I had trained a &lt;code&gt;Neural Network&lt;/code&gt;upon 20,000 character images. This &lt;code&gt;Neural Net&lt;/code&gt; generally works well, it uses &lt;code&gt;RGB- Hue, Saturation, Intensity&lt;/code&gt; feature set for training. However, there can be certain character images which have &lt;code&gt;RGB-HSI&lt;/code&gt; values that this &lt;code&gt;neural net&lt;/code&gt; has not seen before. Therefore I am looking forward to converting training data to &lt;code&gt;grayscale&lt;/code&gt; and use some feature set well suited for grayscale images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So are there any good suggestion for extracting a feature set out of &lt;code&gt;grayscale&lt;/code&gt; images.&lt;/p&gt;&#xA;" OwnerUserId="15116" LastActivityDate="2018-04-22T08:17:57.023" Title="Feature set out of grayscale Images for training a neural network?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;datasets&gt;&lt;artificial-neuron&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6138" PostTypeId="2" ParentId="3287" CreationDate="2018-04-22T19:59:32.070" Score="0" Body="&lt;p&gt;In simple terms, it can explain as below, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you have &lt;strong&gt;10 filters&lt;/strong&gt; where &lt;strong&gt;each filter is the size of 5x5x3&lt;/strong&gt;. What does this mean? the &lt;strong&gt;depth&lt;/strong&gt; of this layer is &lt;strong&gt;10&lt;/strong&gt; which is equal to the number of filters. Size of each filter can be defined as we want e.g., 5x5x3 in this case where 3 is the depth of the previous layer. To be precise, depth of each filer in the next layer should be 10 ( nxnx10) where n can be defined as you want like 5 or something else. Hope will make everything clear. &lt;/p&gt;&#xA;" OwnerUserId="15189" LastActivityDate="2018-04-22T19:59:32.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6139" PostTypeId="1" AcceptedAnswerId="6967" CreationDate="2018-04-23T10:29:22.173" Score="3" ViewCount="646" Body="&lt;h1&gt;Problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Given a collection of  pairs (X, y) where X belongs to R^n and y belongs to R, find the X such that the associated y would be maximum.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Example&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Given:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(X=(1, 2), y=-9)&lt;/li&gt;&#xA;&lt;li&gt;(X=(-2, 4), y=-36)&lt;/li&gt;&#xA;&lt;li&gt;(X=(-4, 2), y=-24)&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The algorithm should be able to detect that the function being applied to X is y=-(X[0]^2+2*(X[1]^2)) and find the input that maximizes this function, in this case X=(0,0) because y=0^2+2*0^2=0 and 0 is the maximum possible value, as all the other values are negative.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;How I've tried to solve it&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;My first guess has been to create a neural network that predicts y given X, but, after that is done, I don't know how to go about optimizing the input.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Questions&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Is there any algorithm that would help in this situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, would some other supervised learning algorithm fit better here than a neural network?&lt;/p&gt;&#xA;" OwnerUserId="15196" LastEditorUserId="15196" LastEditDate="2018-04-23T15:44:33.447" LastActivityDate="2018-06-30T18:39:53.747" Title="Input optimization on a supervised learning system" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;optimization&gt;" AnswerCount="2" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6142" PostTypeId="1" CreationDate="2018-04-23T16:41:08.710" Score="3" ViewCount="441" Body="&lt;p&gt;This is AI: A Modern Approach, 3.17c. The solution manual gives the answer as $\frac{d}{\epsilon}$, where $d$ is the depth of the shallowest goal node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iterative lengthening search uses a path cost limit on each iteration, and updates that limit on the next iteration to the lowest cost of any rejected node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen this question posted elsewhere as, &quot;What is the number of iterations with a continuous range $[0, 1]$ and a minimum step cost $\epsilon$?&quot; In that case, I agree that the minimum number of iterations is $\frac{d}{\epsilon}$ because you would need to increase the path cost limit by a minimum of $\epsilon$ with each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with a continuous range of $[\epsilon, 1]$, it seems there is an infinite range and that the number of iterations is potentially infinite, since there is no minimum step cost. Should this solution actually be infinite?&lt;/p&gt;&#xA;" OwnerUserId="2897" LastEditorUserId="1641" LastEditDate="2018-08-11T11:51:24.537" LastActivityDate="2019-09-05T15:03:25.490" Title="How many iterations are required for iterative-lengthening search when step costs are drawing from a continuos range [ϵ, 1]?" Tags="&lt;search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6144" PostTypeId="1" CreationDate="2018-04-23T18:58:09.750" Score="4" ViewCount="51" Body="&lt;p&gt;Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out that, at least with a number of canonical examples, vector arithmetic seems to work intuitively. For example &quot;&lt;em&gt;king + woman - man = queen&lt;/em&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These terms are all N-dimensional vectors. Now, suppose, for simplicity, that &lt;span class=&quot;math-container&quot;&gt;$N=3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\text{king} = [0, 1, 2], \text{woman} = [1, 1, 0], \text{man} = [2, 2, 2], \text{queen} = [-1, 0, 0]$&lt;/span&gt;, then the expression above can be written as &lt;span class=&quot;math-container&quot;&gt;$[0, 1, 2] +  [1, 1, 0] - [2, 2, 2] = [-1, 0, 0]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this (contrived) example, the last dimension (king/man=2, queen/woman=0) suggests a semantic concept of gender. Aside from semantics, a given dimension could &quot;mean&quot; a part of speech, first letter, or really any feature or set of features that the algorithm might have latched onto. However, any perceived &quot;meaning&quot; of a single dimension might well just be a simple coincidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? Or is this purely a &quot;random&quot; artefact of the algorithm, with only the full N-dimensional vector distances mattering?&lt;/p&gt;&#xA;" OwnerUserId="13360" LastEditorUserId="2444" LastEditDate="2019-04-16T22:30:58.130" LastActivityDate="2019-04-16T22:30:58.130" Title="Do individual dimensions in vector space have meaning?" Tags="&lt;natural-language-processing&gt;&lt;word2vec&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6145" PostTypeId="2" ParentId="6139" CreationDate="2018-04-23T21:03:23.107" Score="0" Body="&lt;p&gt;You really need neural networks to return the maximum value from your data? This algorithm can't help you?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xdata = [&#xA;    (1, 2),&#xA;    (-2, 4),&#xA;    (-4, 2),&#xA;]&#xA;&#xA;for test in list_tests:&#xA;    y = -(test[0]^2+2 * (test[1]^2))&#xA;    if y &amp;gt; 0:&#xA;        print(&quot;(X(%s, %s), y=%s)&quot; % (test[0], test[1], y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;output:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(X(-2, 4), y=16)&#xA;(X(-4, 2), y=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-04-23T21:03:23.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6146" PostTypeId="5" CreationDate="2018-04-23T21:36:16.483" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autoencoder (wikipedia)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autoencoders (Stanford)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ufldl.stanford.edu/wiki/index.php/Stacked_Autoencoders&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stacked Autoencoders (Standord)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-23T21:36:16.483" LastActivityDate="2018-04-23T21:36:16.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6147" PostTypeId="4" CreationDate="2018-04-23T21:36:16.483" Score="0" Body="For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-12-09T23:01:43.400" LastActivityDate="2018-12-09T23:01:43.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6150" PostTypeId="2" ParentId="6144" CreationDate="2018-04-24T03:45:56.990" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do individual dimensions in vector space have meaning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;IIRC, some dimensions are interpretable, but in general this is not the case. Also it is debatable as to wether it is actually learning the actual representation or just an approximation of it. But in any case its not very reliable outside from some edge cases.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but as to what that information entails in terms of &quot;meaning&quot; is lesser clear. You could say that if in a certain dimension the distance between two vectors is 0, then you have am estimate of the real distance that is better than guessing.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2018-04-24T04:13:45.413" LastActivityDate="2018-04-24T04:13:45.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6152" PostTypeId="2" ParentId="4949" CreationDate="2018-04-24T11:25:53.887" Score="2" Body="&lt;p&gt;In the context of IT systems, &quot;Robotic Process Automation&quot; (RPA) is a term often used to describe a technique where software systems are integrated or work processes are automated through the existing user interface of the applications rather than writing new software to provide integration points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In that context, RPA has nothing to do with AI or machine learning. In most cases, it does not even require OCR.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example of a common use case, let's say you have an old mainframe IT systems for tracking your subscriptions and a new website to let people order subscriptions from their phone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case you might create an &quot;RPA&quot; job that opens the list of new subscription requests from the website and for each of them  opens the old application, clicks of the &quot;new subscription&quot; button, click on the &quot;Customer Name&quot; field, pastes the name, clicks &quot;Customer address&quot; and pastes in the address &lt;em&gt;etc.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, the RPA job will be exposed as a service with an API that can be called by the new application, so it can dump data directly into the old application. The benefit is that it can do this without any changes to the old application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is attractive because defining the steps of &lt;em&gt;copy this, click that&lt;/em&gt; can often be defined in visual tools by non-programmers very quickly and at much lower cost than setting up a systems integration project to connect the two systems and because integration through the existing user interface does not require any changes to the application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, it is similar in spirit to how Excel allows non-programmers to automate calculations by writing formulae and thus automating their spreadsheets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will often see RPA proponents putting some AI buzzwords into their presentations but from what I have seen in industry RPA is mostly just a visual scripting technique that is easy to learn and easy to apply.&lt;/p&gt;&#xA;" OwnerUserId="10368" LastEditorUserId="11061" LastEditDate="2018-05-01T20:57:39.443" LastActivityDate="2018-05-01T20:57:39.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6154" PostTypeId="1" CreationDate="2018-04-24T12:52:12.787" Score="3" ViewCount="1857" Body="&lt;p&gt;I have implemented a neural network (NN) using python and numpy only for learning purposes. I have already coded learning rate, momentum, and L1/L2 regularization and checked the implementation with gradient checking. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few days ago, I implemented batch normalization using the formulas provided by the &lt;a href=&quot;https://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;. However, in contrast with learning/momentum/regularization, the batch normalization procedure behaves differently during &lt;strong&gt;fit&lt;/strong&gt; and &lt;strong&gt;predict&lt;/strong&gt; phases - both needed for gradient checking. As we fit the network, batch normalization computes each batch mean and estimates the population's mean to be used when we want to predict something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar way, I know we may not perform gradient checking in a neural network with &lt;strong&gt;dropout&lt;/strong&gt;, since dropout turns some gradients to zero during &lt;em&gt;fit&lt;/em&gt; and is not applied during &lt;em&gt;prediction&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we perform gradient checking in NN with batch normalization? If so, how?&lt;/p&gt;&#xA;" OwnerUserId="13036" LastEditorUserId="15219" LastEditDate="2018-04-24T19:04:07.020" LastActivityDate="2021-04-21T04:50:24.757" Title="How to perform gradient checking in a neural network with batch normalization?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6155" PostTypeId="1" CreationDate="2018-04-24T17:45:15.730" Score="3" ViewCount="613" Body="&lt;p&gt;The definition of God&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a superhuman being or spirit worshipped as having power over nature or human fortunes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't believe any of the Gods of the present or past are real in any way shape or form. However, when the singularity happens, theoretically speaking, isn't this essentially the birth of a God, given that it will be the creation of a superhuman being?&lt;/p&gt;&#xA;" OwnerUserId="13252" LastEditorUserId="2444" LastEditDate="2019-10-22T19:54:15.733" LastActivityDate="2019-10-22T20:03:21.093" Title="Is the singularity equivalent to the creation of God?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;superintelligence&gt;&lt;singularity&gt;&lt;mythology-of-ai&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ClosedDate="2018-06-27T21:46:20.420" ContentLicense="CC BY-SA 4.0" />
  <row Id="6157" PostTypeId="2" ParentId="6155" CreationDate="2018-04-24T18:12:54.440" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;infinite being, eternal, supernatural and existing by itself, necessary cause and ultimate end of all that exists&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As the Portuguese dictionary says, the problem of calling God is the connection with spirituality, it is even strange to call a machine of God.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can put Google as an example. They know what you do, where you are, what you like and what you do not want anyone to know, and he's everywhere listening to you. The only way to not be under Google's vision is to stop using a cell phone, computer and living a boring life so carefully that you would have to have to not be watched.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's funny the relationship. According to the Bible (I am an atheist), the man departed from God in Babylon (cities).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But can we call Google God? If Google does not have a backup of its core scattered through space and other planets, I think the human can still destroy the world and bring Google together. But following rational ideology, he is closer to being God than anything on earth, including us.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="2444" LastEditDate="2019-10-22T20:03:21.093" LastActivityDate="2019-10-22T20:03:21.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6158" PostTypeId="1" CreationDate="2018-04-24T18:44:36.500" Score="4" ViewCount="955" Body="&lt;p&gt;To start, let me just say that I am very new to tensorflow and Machine Learning in general. But, as part of my learning project I am trying to adapt the tensorflow wide and deep model to generate movie recommendations. However, the part I'm getting stuck on is handling multiple values for a categorical column. Below is a sample of how a couple of rows in my CSV look like.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    ID,Genres,Tags,Rating,Recommendations&#xA;    ----------------------------------------&#xA;    1,genre1:genre2:genre3,tag1:tag3,4.3,44&#xA;    2,genre2:genre3,tag1:tag5,3.7,22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Genres and Tags column have multiple categorical values. I have looked at &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/string_split&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tf.string_split&lt;/code&gt;&lt;/a&gt; to parse the strings and return a &lt;code&gt;SparseTensor&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once I have parsed my delimited string values into &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/SparseTensor&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;SparseTensor&lt;/code&gt;&lt;/a&gt;, what do I do with? If I want to create &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/feature_column/categorical_column_with_vocabulary_file&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;categorical_column_with_vocabulary_file&lt;/code&gt;&lt;/a&gt;, how does the &lt;code&gt;SparseTensor&lt;/code&gt; interact with it? Is that even the correct step? Should the &lt;code&gt;SparseTensor&lt;/code&gt; be converted into a something before I can create a &lt;code&gt;categorical_column_*&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am just not sure how to train the Wide and Deep model when you have multiple categorical values for a single column. Some people have suggested that I use each 'Genre' as its own column and encode it using One-Hot, but it is not realistic for me to do this, because there could be 100s of genres and tags in the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help on this matter would be welcome. Thank you!!&lt;/p&gt;&#xA;" OwnerUserId="15220" LastActivityDate="2020-05-05T14:05:05.570" Title="How do you handle multiple categorical values in a single column for wide_deep model in tensorflow?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ClosedDate="2020-06-02T23:44:02.507" ContentLicense="CC BY-SA 3.0" />
  <row Id="6159" PostTypeId="5" CreationDate="2018-04-24T18:57:12.540" Score="0" Body="&lt;p&gt;See:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Talos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Talos&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Skynet_(Terminator)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Skynet_(Terminator)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Technological_singularity&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(re: Singularity; just because it's a myth doesn't mean it couldn't happen. Just that, as of the drafting of this tag, it still resides in the mythical realm)  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-24T18:57:12.540" LastActivityDate="2018-04-24T18:57:12.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6160" PostTypeId="4" CreationDate="2018-04-24T18:57:12.540" Score="0" Body="Use for question about the emergent &quot;mythology of AI&quot;, from Talos in Greek mythology, to Skynet and the technological singularity." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-24T18:57:12.540" LastActivityDate="2018-04-24T18:57:12.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6161" PostTypeId="1" CreationDate="2018-04-24T19:12:17.230" Score="5" ViewCount="172" Body="&lt;p&gt;With so much innovation, with so much previous human manual labor being performed in minutes or seconds by an artificial intelligence, one day man will put the survival and propagation of his species above his ideologies and cultures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am worried because we are living the fourth industrial revolution, and this will generate millions of unemployment, even if new jobs are created in the future. The problem is that a lot of humans worry about their own job, and not about their own children's future. This is completely retrograde.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will, one day, Artificial Intelligence be able to direct us towards an intelligent path as a propagation of the species, or else center the focus of humanity on something that it adds?&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="2444" LastEditDate="2019-10-22T19:47:36.327" LastActivityDate="2019-10-22T19:47:36.327" Title="Will artificial intelligence make the human more rational?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;rationality&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6162" PostTypeId="1" CreationDate="2018-04-24T23:11:25.637" Score="3" ViewCount="820" Body="&lt;p&gt;Semi-gradient methods work well in reinforcement learning, but what is there a reason of &lt;strong&gt;not&lt;/strong&gt; using the true gradient if it can be computed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried it on the cart pole problem with a deep Q-network and it performed much worse than traditional semi-gradient. Is there a concrete reason for this?&lt;/p&gt;&#xA;" OwnerUserId="15224" LastEditorUserId="2444" LastEditDate="2019-05-10T14:40:15.003" LastActivityDate="2021-05-20T22:54:14.003" Title="Why use semi-gradient instead of full gradient in RL problems, when using function approximation?" Tags="&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6163" PostTypeId="2" ParentId="6154" CreationDate="2018-04-25T00:54:08.627" Score="0" Body="&lt;p&gt;You should be able to do gradient checking as long as you fix the randomness by fixing the random seed, on python you might want to look at numpy.random.seed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://cs231n.github.io/neural-networks-3/#ensemble&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/neural-networks-3/#ensemble&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When performing gradient check, remember to turn off any non-deterministic effects in the network, such as dropout, random data augmentations, etc. Otherwise these can clearly introduce huge errors when estimating the numerical gradient. The downside of turning off these effects is that you wouldn’t be gradient checking them (e.g. it might be that dropout isn’t backpropagated correctly). &lt;strong&gt;Therefore, a better solution might be to force a particular random seed before evaluating both (f(x+h)) and (f(x-h)), and when evaluating the analytic gradient.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-04-25T00:54:08.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6164" PostTypeId="1" AcceptedAnswerId="6190" CreationDate="2018-04-25T03:05:35.160" Score="1" ViewCount="112" Body="&lt;p&gt;I am developing PDA like Google assistant on Android. So far, so good.&#xA;But now, I want to add contextual follow up like Google assistant so it can keep the train of thought.&#xA;As demonstrated here- &lt;a href=&quot;https://www.youtube.com/watch?v=xYRENGuwwCA&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=xYRENGuwwCA&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone guide me or hint how to design the algorithm?&#xA;Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="4869" LastEditDate="2018-04-25T15:58:23.793" LastActivityDate="2018-04-30T07:38:58.637" Title="How to add contextual follow up like Google Assistant" Tags="&lt;ai-design&gt;&lt;algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6165" PostTypeId="2" ParentId="6116" CreationDate="2018-04-25T04:23:48.983" Score="1" Body="&lt;p&gt;There is a technique called &lt;a href=&quot;https://www.coursera.org/learn/deep-neural-network/lecture/htA0l/gradient-checking&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gradient checking&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TlW2i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TlW2i.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With it, you can assert if you are calculating the correct gradient in the components of your ANN. The code implementation is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def gradient_check_n(parameters, gradients, X, Y, epsilon = 1e-7):&#xA;&#xA;parameters_values, _ = dictionary_to_vector(parameters)&#xA;grad = gradients_to_vector(gradients)&#xA;num_parameters = parameters_values.shape[0]&#xA;J_plus = np.zeros((num_parameters, 1))&#xA;J_minus = np.zeros((num_parameters, 1))&#xA;gradapprox = np.zeros((num_parameters, 1))&#xA;&#xA;# Compute gradapprox&#xA;for i in range(num_parameters):&#xA;&#xA;&#xA;    thetaplus = np.copy(parameters_values)                                      &#xA;    thetaplus[i][0] = thetaplus[i][0]+  epsilon                              &#xA;    J_plus[i], _ = forward_propagation_n(X, Y, vector_to_dictionary( thetaplus  ))                                  &#xA;&#xA;    thetaminus =  np.copy(parameters_values)                                      &#xA;    thetaminus[i][0] = thetaplus[i][0]-  epsilon                                  &#xA;    J_minus[i], _ = forward_propagation_n(X, Y, vector_to_dictionary( thetaminus  ))                            &#xA;&#xA;    gradapprox[i] = (J_plus[i]-J_minus[i])/(2*epsilon)&#xA;&#xA;&#xA;&#xA;numerator = np.linalg.norm(grad-gradapprox)                              &#xA;denominator = np.linalg.norm(grad) +  np.linalg.norm(gradapprox)                            &#xA;difference = numerator/denominator                             &#xA;&#xA;if difference &amp;gt; 2e-7:&#xA;    print (&quot;There is a mistake in the backward propagation. Difference = &quot; + str(difference))&#xA;else:&#xA;    print (&quot;Backward propagation is Okay. Difference = &quot; + str(difference))&#xA;&#xA;return difference&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where parameters is a dictionary with the parameters &quot;W1&quot;,&quot;b1&quot;....&quot;Wl&quot;,&quot;bl&quot; &#xA;and grad is the output of the L_model_backward which contains gradients of the cost with respect to the parameters. Also, if you could share the x_train,y_train so we can debug it, it would be great, good luck.&lt;/p&gt;&#xA;" OwnerUserId="12006" LastActivityDate="2018-04-25T04:23:48.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6166" PostTypeId="1" CreationDate="2018-04-25T09:17:26.863" Score="1" ViewCount="43" Body="&lt;p&gt;I am unable to identify general temrs or specific source of information for the below proposed problem. I would appreciate if the community can guide me to journal articles/books and keywords to look for in literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a non-linear dynamic system taking input and producing 1D time series as output. I would like to use NN to find parameters of the dynamic system, according to the time series output. That is, mapping the features of the time series (after transformation, likely Fourtier Transform or Wavelet) to the parameters governing the dynamics of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Research so far:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found a few journal papers mostly processing sounds of rolling bearings or hearbeat but only for error/failure classification. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Rolling Bearing Fault Diagnosis Based on STFT-Deep Learning and SOund Signals&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning Enabled Fault Diagnosis Using Time-Frequency Image Analysis of Rolling Element Bearings&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning Based Approach for Bearing Fault Diagnosis&lt;/li&gt;&#xA;&lt;li&gt;Detecting atrila fibrillation be deep convolutional neural networks&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(the above are classification problems, my problem is about parameter identification)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reason to address this on StackExchange:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I am missing overview about the topic (identification of dynamic systems using NN), because I am not able to reach more profound information. Also, I think that NN would be more beneficial to my current application than lets say optimization by evolutionary algorithms, threfore I am specifically asking for NN.&lt;/p&gt;&#xA;" OwnerUserId="15231" LastActivityDate="2018-04-25T09:17:26.863" Title="Abstracting parameters of dynamic model from output time series" Tags="&lt;neural-networks&gt;&lt;models&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="6167" PostTypeId="1" AcceptedAnswerId="6173" CreationDate="2018-04-25T12:31:48.297" Score="3" ViewCount="2297" Body="&lt;p&gt;I'm trying to understand what would be the best neural network for implementing an XOR gate. I'm considering a neural network to be good if it can produce all the expected outcomes with the lowest possible error.&lt;/p&gt;&#xA;&lt;p&gt;It looks like my initial choice of random weights has a big impact on my end result after training. The accuracy (i.e. error) of my neural net is varying a lot depending on my initial choice of random weights.&lt;/p&gt;&#xA;&lt;p&gt;I'm starting with a 2 x 2 x 1 neural net, with a bias in the input and hidden layers, using the sigmoid activation function, with a learning rate of 0.5. Below my initial setup, with weights chosen randomly:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4tv2V.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4tv2V.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The initial performance is bad, as one would expect:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Input | Output | Expected | Error&#xA;(0,0)   0.8845      0       39.117%&#xA;(1,1)   0.1134      0       0.643%&#xA;(1,0)   0.7057      1       4.3306%&#xA;(0,1)   0.1757      1       33.9735%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I proceed to train my network through backpropagation, feeding the XOR training set 100,000 times. After training is complete, my new weights are:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bPMix.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bPMix.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the performance improved to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Input | Output | Expected | Error&#xA;(0,0)   0.0103      0       0.0053%&#xA;(1,1)   0.0151      0       0.0114%&#xA;(1,0)   0.9838      1       0.0131%&#xA;(0,1)   0.9899      1       0.0051%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;So my questions are:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Has anyone figured out the best weights for a XOR neural network with that configuration (i.e. 2 x 2 x 1 with bias) ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why my initial choice of random weights make a big difference to my end result? I was lucky on the example above but depending on my initial choice of random weights I get, after training, errors as big as 50%, which is very bad.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Am I doing anything wrong or making any wrong assumptions?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So below is an example of weights I cannot train, for some unknown reason. I think I might be doing my backpropagation training incorrectly. I'm not using batches and I'm updating my weights on each data point solved from my training set.&lt;/p&gt;&#xA;&lt;p&gt;Weights: &lt;code&gt;((-9.2782, -.4981, -9.4674, 4.4052, 2.8539, 3.395), (1.2108, -7.934, -2.7631))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q3dCO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q3dCO.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15235" LastEditorUserId="2444" LastEditDate="2021-01-30T19:00:06.123" LastActivityDate="2021-01-30T19:01:17.380" Title="What is the best XOR neural network configuration out there in terms of low error?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;xor-problem&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6168" PostTypeId="2" ParentId="6167" CreationDate="2018-04-25T14:24:09.887" Score="1" Body="&lt;p&gt;2 perceptrons without bias (+1 in the output layer, to get the result as 1 number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xp9NG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xp9NG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15231" LastActivityDate="2018-04-25T14:24:09.887" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6169" PostTypeId="2" ParentId="6134" CreationDate="2018-04-25T15:05:00.357" Score="1" Body="&lt;p&gt;When formulating a problem into an MDP you need to define the &lt;strong&gt;states&lt;/strong&gt;, of the system, the possible &lt;strong&gt;actions&lt;/strong&gt; you can take, the &lt;strong&gt;transition probabilities&lt;/strong&gt; between states depending on the action and the &lt;strong&gt;rewards&lt;/strong&gt; earned (or costs paid) for the state transitions. The important part here is to create a state-space that has a Markov property which in plain words means that in any given state you have enough information to make the optimal decision.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Rewards&lt;/strong&gt;: in this case, I think it is fairly obvious that a positive reward should be given if we find out the number and zero reward if we don't.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Actions&lt;/strong&gt;: the actions we can take is making a guess, that is the possible actions will come from the action-space &lt;em&gt;A=(1,2,3)&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;States&lt;/strong&gt;: this is a bit more difficult to come up with. Here the intuition should come from thinking about the actions you can make and how they change the information you have about the system. In our state-space, each state will represent the set of numbers that the answer might possibly be. For example in state the &lt;em&gt;(1,3)&lt;/em&gt; we know that the guessed number is either &lt;em&gt;1&lt;/em&gt; or &lt;em&gt;3&lt;/em&gt;. Also, we will denote these states with capital letters to simplify the state transitions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A=(1, 2, 3): the initial state as at the beginning of the search the guessed number can be either of &lt;em&gt;1&lt;/em&gt;, &lt;em&gt;2&lt;/em&gt; or &lt;em&gt;3&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;B=(1, 2): the guessed number is either &lt;em&gt;1&lt;/em&gt; or &lt;em&gt;2&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;C=(1, 3): the guessed number is either &lt;em&gt;1&lt;/em&gt; or &lt;em&gt;3&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;D=(2, 3): the guessed number is either &lt;em&gt;2&lt;/em&gt; or &lt;em&gt;3&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;E=(1): the guessed number can only be &lt;em&gt;1&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;F=(2): the guessed number can only be &lt;em&gt;2&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;G=(3): the guessed number can only be &lt;em&gt;3&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;H=(): final state, we found the guessed number.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;State transitions&lt;/strong&gt;: this is very simple, just imagine which states are reachable in a state depending on the guess (action) we make. Here the notation P(A|B,1) will mean the probability that we reach state A from state B given that we guessed &lt;em&gt;1&lt;/em&gt;. (Note: we will assume uniform distribution for the guessed number). I think I'll be lazy here and not write down the all the transitions as they get very repetitive once you understand how they are made, I'll just provide examples for all cases.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;P(D|A,1)=2/3: we guess &lt;em&gt;1&lt;/em&gt; in the initial state and we miss with 2/3 chance, the reply will be that the guessed number is greater than &lt;em&gt;1&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;P(H|A,1)=1/3: we guessed right, so we reach the final state.&lt;/li&gt;&#xA;&lt;li&gt;P(B|A,3)=2/3: similarly we guess &lt;em&gt;3&lt;/em&gt; and miss.&lt;/li&gt;&#xA;&lt;li&gt;P(H|A,3)=1/3: we guess &lt;em&gt;3&lt;/em&gt; and we are right.&lt;/li&gt;&#xA;&lt;li&gt;P(C|A,2)=0: if our guess &lt;em&gt;2&lt;/em&gt; is wrong we get a reply of whether the guessed number is greater or smaller than &lt;em&gt;2&lt;/em&gt; therefore we won't get into this state.&lt;/li&gt;&#xA;&lt;li&gt;P(E|A,2)=1/3: we guess &lt;em&gt;2&lt;/em&gt; and the reply is that the number is smaller than &lt;em&gt;2&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;P(G|A,2)=1/3&lt;/li&gt;&#xA;&lt;li&gt;P(H|A,2)=1/3&lt;/li&gt;&#xA;&lt;li&gt;P(B|B,3)=1: guessing &lt;em&gt;3&lt;/em&gt; in state B doesn't provide more information so we reach the same state with probability 1.&lt;/li&gt;&#xA;&lt;li&gt;P(H|B,1)=1/2: &lt;em&gt;1&lt;/em&gt; is the right guess with 1/2 probability in this state.&lt;/li&gt;&#xA;&lt;li&gt;P(F|B,1)=1/2: similarly we miss with 1/2 chance if we guess &lt;em&gt;1&lt;/em&gt; so we reach state F where we know what is the good solution however we haven't won yet as we still need to make one more guess to reach the final state.&lt;/li&gt;&#xA;&lt;li&gt;P(F|F,1)=1: again guessing &lt;em&gt;1&lt;/em&gt; in state F doesn't give us extra information so we get back to state F.&lt;/li&gt;&#xA;&lt;li&gt;P(H|F,2)=1: however, guessing &lt;em&gt;2&lt;/em&gt; in state F will give us the final state, as it is the right guess.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that I defined 8 states earlier, however, state C=(1,3) is never reached so we don't actually need it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helped and you will be able to finish the rest.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastEditorUserId="11061" LastEditDate="2018-04-30T06:48:12.777" LastActivityDate="2018-04-30T06:48:12.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6170" PostTypeId="1" AcceptedAnswerId="6177" CreationDate="2018-04-25T15:17:27.333" Score="1" ViewCount="493" Body="&lt;p&gt;I was following &lt;a href=&quot;http://natureofcode.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Daniel Shiffman's&lt;/a&gt; tutorials on how to write your own neural network from scratch. I specifically looked into his &lt;a href=&quot;https://www.youtube.com/watch?v=MPmLWsHzPlU&quot; rel=&quot;nofollow noreferrer&quot;&gt;videos&lt;/a&gt; and the code he provided in &lt;a href=&quot;https://github.com/CodingTrain/website/blob/master/Courses/natureofcode/10.18-toy_neural_network/lib/nn.js&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I rewrote his code in Python, however, 3 out of 4 of my outputs are the same. The neural network has two input nodes, one hidden layer with two nodes and one output node. Can anyone help me to find my mistake? Here is my &lt;a href=&quot;https://pastebin.com/saJxedDT&quot; rel=&quot;nofollow noreferrer&quot;&gt;full code&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import random&#xA;&#xA;nn = NeuralNetwork(2,2,1)&#xA;inputs  = np.array([[0, 0], [1, 0], [0, 1], [1, 1]])&#xA;targets = np.array([[0], [1], [1], [0]])&#xA;zipped = zip(inputs, targets)&#xA;list_zipped = list(zipped)&#xA;&#xA;for _ in range(9000):&#xA;    x, y = random.choice(list_zipped)&#xA;    nn.train(x, y)&#xA;&#xA;output = [nn.feedforward(i) for i in inputs]&#xA;&#xA;for i in output:&#xA;   print(&quot;Output &quot;, i)&#xA;&#xA;#Output  [ 0.1229546]  when it should be around 0&#xA;#Output  [ 0.6519492]  ~1&#xA;#Output  [ 0.65180228] ~1&#xA;#Output  [ 0.66269853] ~0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT_1:&lt;/strong&gt; I tried debugging my code by choosing all weights and bias' values to 0.5. I did this in both my code and Daniel's. This obviously ended up showing me &lt;em&gt;all&lt;/em&gt; outputs with the same value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that I increased my weights and bias' values variety from [0 , 1) to [-1, 1). By running this a few times, I would &lt;strong&gt;sometimes&lt;/strong&gt; get the correct output: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 0.93749991] # should be ~1&#xA;[ 0.93314793] # ~1 &#xA;[ 0.07001175] # ~0&#xA;[ 0.06576194] # ~0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I ran nn.train() 100 000 times, I get the correct output 2/3 times. &#xA;Is this the issue of gradient descent, where it converges to the local minima?&lt;/p&gt;&#xA;" OwnerUserId="14863" LastEditorUserId="14863" LastEditDate="2018-04-27T08:21:28.457" LastActivityDate="2018-04-27T08:21:28.457" Title="Neural network returns similar output" Tags="&lt;neural-networks&gt;&lt;python&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6173" PostTypeId="2" ParentId="6167" CreationDate="2018-04-26T03:24:02.123" Score="2" Body="&lt;p&gt;The initialization of the weights has a big impact on the results. I'm not sure specifically for the XOR gate, but the error can have a local minimum that the network can get &amp;quot;stuck&amp;quot; in during training. Using stochastic gradient descent can help give some randomness that gets the error out of these pits. Also, for the sigmoid function, weights should be initialized so that the input to the activation is close to the part with the highest derivative so that training is better.&lt;/p&gt;&#xA;" OwnerUserId="12145" LastEditorUserId="2444" LastEditDate="2021-01-30T19:01:17.380" LastActivityDate="2021-01-30T19:01:17.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6174" PostTypeId="2" ParentId="6162" CreationDate="2018-04-26T07:25:03.200" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Semi gradient methods work well in &lt;strong&gt;Reinforcement Learning&lt;/strong&gt;, but what is the reason of &lt;strong&gt;&lt;em&gt;not using&lt;/em&gt;&lt;/strong&gt; the true gradient if it can be computed? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Just complexity and extra computation, in many cases for a marginal benefit.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I tried it on the cart pole problem with a deep Q-Network and it performed much worse than traditional semi gradient, is there a concrete reason for this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is hard to tell, without exploring the implementation in detail. However, DQN is an inherently unstable learning technique that needs care in choosing hyper-parameters that control this instability and offset against learning rate:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;size of minibatch to train from experience replay on each step&lt;/li&gt;&#xA;&lt;li&gt;number of training steps between taking frozen copies for estimation*&lt;/li&gt;&#xA;&lt;li&gt;whether or not you use double-learning to avoid maximisation bias (more important if you have fine-grained discretisation of continuous action space)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a chance that the optimal choices here are different between true gradient and semi gradient approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* The frozen estimator could be a big clue here in your implementation. If you are using this frozen copy technique, it has a big impact on how you should calculate the true gradient, because changing the parameters would no longer change the current TD target - which is what the true gradient approach fixes. However, getting rid of this stability-improving addition in order to get true gradients might on balance make the algorithm less stable - you could try to fix that by taking larger mini-batches.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-04-26T07:25:03.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6176" PostTypeId="1" AcceptedAnswerId="6184" CreationDate="2018-04-26T11:41:28.273" Score="6" ViewCount="218" Body="&lt;p&gt;I'm starting a project that will involve computer vision, visual question answering, and explainability. I am currently choosing what type of algorithm to use for my classifier - a neural network or a decision tree.&lt;/p&gt;&#xA;&lt;p&gt;It would seem to me that, because I want my system to include explainability, a decision tree would be the best choice. Decision trees are interpretable, whereas neural nets are like a black box.&lt;/p&gt;&#xA;&lt;p&gt;The other differences I'm aware of are: decision trees are faster, neural networks are more accurate, and neural networks are better at modelling nonlinearity.&lt;/p&gt;&#xA;&lt;p&gt;In all of the research I've done on computer vision and visual question answering, everyone uses neural networks, and no one seems to be using decision trees. Why? Is it for the accuracy? I think a decision tree would be better because it is fast and interpretable, but if no one's using them for visual question answering, they must have a disadvantage that I haven't noticed.&lt;/p&gt;&#xA;" OwnerUserId="9983" LastEditorUserId="2444" LastEditDate="2021-01-26T15:33:06.660" LastActivityDate="2021-01-26T15:33:06.660" Title="Why does nobody use decision trees for visual question answering?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;decision-tree&gt;&lt;explainable-ai&gt;&lt;question-answering&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6177" PostTypeId="2" ParentId="6170" CreationDate="2018-04-26T12:34:55.227" Score="2" Body="&lt;h1&gt;Local minima.&lt;/h1&gt;&#xA;&lt;p&gt;You have the exact same issue of this &lt;a href=&quot;https://ai.stackexchange.com/questions/5904/why-does-a-one-layer-hidden-network-get-more-robust-to-poor-initialization-with/&quot;&gt;question&lt;/a&gt;. If you randomize your initial weights, you'll see sometimes you get the correct results, and others you won't. It's because when the weights are initialized with a certain range of values, they will converge to a local minima which you cannot escape with a low learning rate.&lt;/p&gt;&#xA;&lt;p&gt;A simple solution is to increase the size of your hidden layer, which will make the network more robust to such issues.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When you have only 2 dimensions, a local minima exists. When you have more dimensions, this minima gets harder and harder to reach, as its likelihood decreases. Intuitively, you have a lot more dimensions through which you can improve than if you only had 2 dimensions.&lt;/p&gt;&#xA;&lt;p&gt;The problem still exists, even with 1000 neurons you could find a specific set of weights which was a local minimum. However, it just becomes so much less likely.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-04-26T12:34:55.227" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6179" PostTypeId="1" CreationDate="2018-04-26T13:42:48.293" Score="2" ViewCount="265" Body="&lt;p&gt;My question concerns a side question (which was not answered) asked here:&#xA;&lt;a href=&quot;https://ai.stackexchange.com/questions/4085/policy-gradients-for-multiple-continuous-actions&quot;&gt;Policy gradients for multiple continuous actions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to implement a simple policy gradient algorithm for a discrete multi-action reinforcement learning task. To be more precise, there are three actuators. At every time step, each of the actuators can perform one of three possible actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to adjust the loss function from the single action case per time step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L = \log(P(a_1)) A$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to the n-action case per time step like so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L = (\log(P(a_1)) + \log(P(a_2))+ \dots + \log(P(a_n))) A$$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="15265" LastEditorUserId="2444" LastEditDate="2019-02-13T02:32:38.557" LastActivityDate="2019-02-13T02:32:38.557" Title="Extend the loss function from the single action to the n-action case per time step" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6180" PostTypeId="5" CreationDate="2018-04-26T16:18:46.833" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-26T16:18:46.833" LastActivityDate="2018-04-26T16:18:46.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6181" PostTypeId="4" CreationDate="2018-04-26T16:18:46.833" Score="0" Body="For question involving decision trees in any form of AI." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-04-26T16:18:46.833" LastActivityDate="2018-04-26T16:18:46.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6182" PostTypeId="1" CreationDate="2018-04-26T22:09:57.767" Score="1" ViewCount="119" Body="&lt;p&gt;I have a friend who, after having a stroke, now has locked-in syndrome who and can now only move his eyes and eyelids. I've been working on some C# software (written in Unity3D) to help him to communicate, the ones he has tried so far have not been successful due to the level of his disability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am in the process of implementing DLib for facial feature recognition. What I would like to do is to ask the user to repeatedly alternate between a neutral face and a face that is indicating. I don't want to limit the indication gesture specifically to winking/blinking/opening eyes wide, etc as I want other people to be able to freely download the app and indicate in whatever way they find easiest due to their incapacity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could really do with some pointers as to how to have my app learn what the user's facial indicator is by comparing the facial landmarks of the two states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, I don't even know what it is I need to know. I don't know the names of any AI algorithms that might be relevant or anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know posts like this are often closed for being &quot;too vague&quot;, but due to the nature of the request, I'd appreciate it if it could be left open as any suggestions at all could potentially be life-changing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCV would be an excellent choice as I already have a licence for a library in Unity3D.&lt;/p&gt;&#xA;" OwnerUserId="15274" LastEditorUserId="15274" LastEditDate="2018-04-29T09:37:55.763" LastActivityDate="2018-07-08T11:35:19.460" Title="Which library / algorithm should I use to recognise a specific facial expression (C#)" Tags="&lt;ai-basics&gt;&lt;getting-started&gt;&lt;software-evaluation&gt;&lt;facial-recognition&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6184" PostTypeId="2" ParentId="6176" CreationDate="2018-04-27T01:42:23.117" Score="4" Body="&lt;p&gt;For vision tasks, neural network models almost always include a number of layers that pool and convolute. The convolutions, in particular, are very useful - they can make the model generalize better to inputs and maintain performance when inputs have undergone certain linear transformations (e.g. some scaling or a translation along the x-axis). These properties, along with the robust frameworks that exist for developing and deploying neural nets, and the fact that they have been shown to widely produce very good results, are some of the reasons they’re used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of being a black box, while this is true for a lot of applications it’s actually less true for image-based tasks. The layers of a well designed and trained convolutional neural network model can actually be visualized and made quite interpretable; from these visualizations, it’s often clear how the representation &lt;em&gt;roughly&lt;/em&gt; works. In contrast, I’d argue that while a decision tree is theoretically easier to interpret for some tasks (say medical decision making), this is less the case for vision tasks because we don’t interpret images one pixel at a time. People are almost always interested in the higher level representation within an image (say, a cat, a leaf, or a face). Decision trees tend to have trouble capturing these higher level representations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://distill.pub/2017/feature-visualization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Distill.pub&lt;/a&gt; has a nice explanation of feature visualization that may be of interest. &lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="5210" LastEditDate="2018-04-27T16:27:26.890" LastActivityDate="2018-04-27T16:27:26.890" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6185" PostTypeId="1" CreationDate="2018-04-27T03:52:19.403" Score="20" ViewCount="49209" Body="&lt;p&gt;I just want to know why do machine learning engineers and AI programmers use languages like Python to perform AI tasks and not C++, even though C++ is technically a more powerful language than Python.&lt;/p&gt;&#xA;" OwnerUserId="15277" LastEditorUserId="2444" LastEditDate="2020-06-02T23:36:59.627" LastActivityDate="2020-09-04T08:46:53.413" Title="Why does C++ seem less widely used than Python in AI?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;python&gt;&lt;programming-languages&gt;&lt;c++&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6186" PostTypeId="2" ParentId="6185" CreationDate="2018-04-27T08:07:26.937" Score="26" Body="&lt;p&gt;You don't need a powerful language for programming AI. Most of the developers are using libraries like Keras, Torch, Caffe, Watson, TensorFlow, etc. Those low level libraries are highly optimized and handle all the tough work. They are built with high-performance languages, like C, C++. Python is just there for high level task like describing the neural network layers, load data, launch the processing, and display results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using C++ for high level task instead of Python would give barely any performance improvement, but it would be harder for non-developers as it requires to care for memory management. Also, several AI people may not have a very solid programming or computer science background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another similar example would be game development, where the engine is coded in C/C++, and, often, all the game logic scripted in a higher-level language.&lt;/p&gt;&#xA;" OwnerUserId="15283" LastEditorUserId="15283" LastEditDate="2020-06-04T21:10:29.387" LastActivityDate="2020-06-04T21:10:29.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6188" PostTypeId="1" CreationDate="2018-04-27T08:45:53.170" Score="5" ViewCount="271" Body="&lt;p&gt;I have a data set that looks like this:&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nd0qN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nd0qN.png&quot; alt=&quot;data set&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to estimate a relationship between x-values and the corresponding 5% extreme y-values, something that might look like that :&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TfBTp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TfBTp.png&quot; alt=&quot;regression on extreme values&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have an idea of an algorithm that might help me for this ? I thought about labelling the extreme values for later finding a separating hyperplane, but I have no clue on how to label these &quot;extreme values&quot; (I cannot just take the 5% lowest and highest values as all these would end up in the same region).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your ideas ! &lt;/p&gt;&#xA;" OwnerUserId="15281" LastEditorUserId="14995" LastEditDate="2018-05-01T20:57:48.983" LastActivityDate="2018-05-01T20:57:48.983" Title="Regression on extreme values" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6189" PostTypeId="2" ParentId="6185" CreationDate="2018-04-27T10:53:41.543" Score="5" Body="&lt;p&gt;It depends how flexible it needs to be: if you have a fully-fledged system ready for production, which is not going to need much adjusting, then C++ (or even C) might be fine. You need to put a lot of time into building the software, but then it should run pretty fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you're still experimenting with settings and parameters, and maybe need to adjust the architecture, then C++ will be clumsy to work with. You need a language like Python which makes it easier to change things. Changing the code is easier, as you can generally code faster in languages like Python. The price you pay is that the software does usually not perform as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to decide how that trade-off works best for you. It is usually better to spend less time on coding, and not worry too much about longer run-time. If you take a day less to get your code done, that's a lot of time the C-coded version needs to catch up. Most of the time it's just not worth it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common approach seems to be hybrid systems, where core libraries are implemented in C/C++, as they don't need much changing, and the front-end/glue/interfaces are in Python, as there you need flexibility and speed is not that critical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not an issue specific to AI, by the way, but a general question of interpreted vs compiled languages. With AI a  lot of systems are still focused on research rather than application, and that is where speed of development trumps speed of execution.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-04-27T10:53:41.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6190" PostTypeId="2" ParentId="6164" CreationDate="2018-04-27T11:00:22.667" Score="2" Body="&lt;p&gt;You would need to keep track of the current topic, and references. So, for example, a query &lt;em&gt;When is the next train from London to Birmingham?&lt;/em&gt; would result in&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;topic = TRAIN_TRAVEL&lt;/li&gt;&#xA;&lt;li&gt;start-loc = London&lt;/li&gt;&#xA;&lt;li&gt;destination-loc = Birmingham&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A follow-up &lt;em&gt;And what about Bristol?&lt;/em&gt; would then replace &lt;em&gt;destination-loc&lt;/em&gt; with &quot;Bristol&quot; and you would be able to build a new query from that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key issues here are the set of topics your assistant will be able to handle, and the relevant object slots. You might also want to clear the topic variable after the next input or so to avoid having it still hanging around even though the user no longer talks about that particular topic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: Just to add how that would work from a technical point of view. The input &lt;em&gt;And what about Bristol?&lt;/em&gt; would not be recognised as any known intent, as it is too unspecific. As a fallback your Assistant then takes the last topic, and interprets the input in the light of that context. So here we assume that the intent is 'train travel', as that is the last thing the user spoke about. This will of course not always work, but should sort out the majority of cases.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2018-04-30T07:38:58.637" LastActivityDate="2018-04-30T07:38:58.637" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6191" PostTypeId="1" CreationDate="2018-04-27T12:07:00.573" Score="4" ViewCount="1934" Body="&lt;p&gt;I am looking for AI tools for image and video labeling. It would be great if someone could share its own experience as well suggest tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have some questions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Which tools are available? (state of the art)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are their benefits, how do they compare?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why did we choose the specific tool?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15286" LastEditorUserId="2444" LastEditDate="2019-06-19T18:09:52.930" LastActivityDate="2019-06-19T18:09:52.930" Title="What are some tools for labelling images and videos?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-10-27T10:17:17.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="6192" PostTypeId="2" ParentId="6188" CreationDate="2018-04-27T12:11:29.367" Score="2" Body="&lt;p&gt;I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points.  For each step in the sweep, record the angle and the count.  When the sweep is done you will have an array of angles with bin counts and then you can convert to percentage of total count.  You will have to figure out a resolution of the triangle's angle so that it is fine enough to capture less than 5% of the points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ctywK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ctywK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-04-27T12:11:29.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6193" PostTypeId="2" ParentId="1987" CreationDate="2018-04-27T12:51:09.063" Score="9" Body="&lt;p&gt;Ideally neural networks should be able to find out the function out on it's own without us providing the spherical features. After some experimentation I was able to reach a configuration where we do not need anything except &lt;span class=&quot;math-container&quot;&gt;$X_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_2$&lt;/span&gt;. This net converged after about 1500 epochs which is quite long. So the best way might still be to add additional features but I am just trying to say that it is still possible to converge without them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/saZat.jpg&quot; alt=&quot;Solution without additional features&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15287" LastEditorUserId="3217" LastEditDate="2018-12-15T20:34:09.603" LastActivityDate="2018-12-15T20:34:09.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6194" PostTypeId="2" ParentId="6185" CreationDate="2018-04-27T13:45:14.140" Score="12" Body="&lt;p&gt;C++ is actually one of the most popular languages used in the AI/ML space. Python may be more popular in general, but as others have noted, it's actually quite common to have hybrid systems where the CPU intensive number-crunching is done in C++ and Python is used for higher level functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to illustrate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mloss.org/software/language/c__/&quot; rel=&quot;noreferrer&quot;&gt;http://mloss.org/software/language/c__/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mloss.org/software/language/python/&quot; rel=&quot;noreferrer&quot;&gt;http://mloss.org/software/language/python/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-04-27T13:45:14.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6196" PostTypeId="1" AcceptedAnswerId="6199" CreationDate="2018-04-28T03:11:16.087" Score="31" ViewCount="11754" Body="&lt;p&gt;As far as I understand, Q-learning and policy gradients (PG) are the two major approaches used to solve RL problems. While Q-learning aims to predict the reward of a certain action taken in a certain state, policy gradients directly predict the action itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, both approaches appear identical to me, i.e. predicting the maximum reward for an action (Q-learning) is equivalent to predicting the probability of taking the action directly (PG). Is the difference in the way the loss is back-propagated?&lt;/p&gt;&#xA;" OwnerUserId="15298" LastEditorUserId="2444" LastEditDate="2019-02-15T15:39:06.333" LastActivityDate="2020-04-22T12:05:54.623" Title="What is the relation between Q-learning and policy gradients methods?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;policy-gradients&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="6197" PostTypeId="2" ParentId="5348" CreationDate="2018-04-28T06:10:11.423" Score="2" Body="&lt;p&gt;Architecture describes a general approach to a ML problem, and the parameterization of that approach. For example, a neural net architecture would define the number and size of different layers, the type of each layer, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A model is one specific instance of a given architecture, trained on a given dataset. For the example of neural nets, the model would primarily consist of the trained weights of each node from the architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One would use a model to make predictions on new examples at runtime. It is possible to generate many models using the same architecture, even with the same training data; the result of each training process is a new model.&lt;/p&gt;&#xA;" OwnerUserId="13360" LastActivityDate="2018-04-28T06:10:11.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6198" PostTypeId="1" AcceptedAnswerId="6202" CreationDate="2018-04-28T07:46:11.357" Score="2" ViewCount="96" Body="&lt;p&gt;Some context: Recently all kind of salesmen have been knocking on our company's door to provide their &quot;artificial intelligence&quot; expertise and projects suggestions. Some don't know the difference between words estimation and validation (really), some have extraordinary powerpoints and paint themselves as gurus of the field. Our management has gone with the hype and definitely we're starting some kind of project on &quot;artificial intelligence&quot; (meaning rpa with some machine learning possibly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to start when we don't yet know to what problem we want to apply all this and I'm worried it will lead to long expensive projects with meager results? What are the things to watch out for? Any good practical books or war stories out there?&lt;/p&gt;&#xA;" OwnerUserId="3579" LastActivityDate="2018-04-28T13:07:47.990" Title="How to organize artificial intelligence efforts at work?" Tags="&lt;machine-learning&gt;&lt;getting-started&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6199" PostTypeId="2" ParentId="6196" CreationDate="2018-04-28T07:47:57.307" Score="36" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, both approaches appear identical to me i.e. predicting the maximum reward for an action (Q-learning) is equivalent to predicting the probability of taking the action directly (PG).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both methods are theoretically driven by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;noreferrer&quot;&gt;Markov Decision Process&lt;/a&gt; construct, and as a result use similar notation and concepts. In addition, in simple solvable environments you should expect both methods to result in the same - or at least equivalent - optimal policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, they are actually different internally. The most fundamental differences between the approaches is in how they approach action selection, both whilst learning, and as the output (the learned policy). In Q-learning, the goal is to learn a single deterministic action from a discrete set of actions by finding the maximum value. With policy gradients, and other direct policy searches, the goal is to learn a map from state to action, which can be stochastic, and works in continuous action spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result, policy gradient methods can solve problems that value-based methods cannot:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Large and continuous action space. However, with value-based methods, this can still be approximated with discretisation - and this is not a bad choice, since the mapping function in policy gradient has to be some kind of approximator in practice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stochastic policies. A value-based method cannot solve an environment where the optimal policy is stochastic requiring specific probabilities, such as Scissor/Paper/Stone. That is because there are no trainable parameters in Q-learning that control probabilities of action, the problem formulation in TD learning assumes that a deterministic agent can be optimal.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, value-based methods like Q-learning have some advantages too:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Simplicity. You can implement Q functions as simple discrete tables, and this gives some guarantees of convergence. There are no tabular versions of policy gradient, because you need a mapping function &lt;span class=&quot;math-container&quot;&gt;$p(a \mid s, \theta)$&lt;/span&gt; which also must have a smooth gradient with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Speed. TD learning methods that bootstrap are often much faster to learn a policy than methods which must purely sample from the environment in order to evaluate progress.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are other reasons why you might care to use one or other approach:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You may want to know the predicted return whilst the process is running, to help other planning processes associated with the agent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The state representation of the problem lends itself more easily to either a value function or a policy function. A value function may turn out to have very simple relationship to the state and the policy function very complex and hard to learn, or &lt;em&gt;vice-versa&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some state-of-the-art RL solvers actually use both approaches together, such as Actor-Critic. This combines strengths of value and policy gradient methods.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2018-11-18T13:47:14.037" LastActivityDate="2018-11-18T13:47:14.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6200" PostTypeId="5" CreationDate="2018-04-28T09:11:42.247" Score="0" Body="&lt;p&gt;Cognitive science is the interdisciplinary, scientific study of the mind and its processes. It examines the nature, the tasks, and the functions of cognition (in a broad sense). Cognitive scientists study intelligence and behavior, with a focus on how nervous systems represent, process, and transform information. Mental faculties of concern to cognitive scientists include language, perception, memory, attention, reasoning, and emotion; to understand these faculties, cognitive scientists borrow from fields such as linguistics, psychology, artificial intelligence, philosophy, neuroscience, and anthropology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cognitive_science&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cognitive Science - Wikiepdia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-01T20:58:08.933" LastActivityDate="2018-05-01T20:58:08.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6201" PostTypeId="4" CreationDate="2018-04-28T09:11:42.247" Score="0" Body="For questions about use and applications of cognitive science in the field of AI and vice versa." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-01T20:58:11.677" LastActivityDate="2018-05-01T20:58:11.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6202" PostTypeId="2" ParentId="6198" CreationDate="2018-04-28T13:07:47.990" Score="2" Body="&lt;p&gt;I know what you mean. It can be difficult to parse between the hype and the application of AI. Although AI (specifically deep learning) can do a lot of things, this doesn't negate previous methods that work just as well or better in certain domains. Some times managers will hear 'AI' and think of giant networks when they actually just needed a simple linear regression for their problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, here are some suggestions to help decide if AI is right for your project and some things to consider if so :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Get some advice from a researcher&lt;/em&gt;&lt;/strong&gt;. There are many AI consulting firms popping up in large cities that give companies hours of consulting time. If there isn't one in your city, go to AI meetups and meet the community, find educated people and ask them about your general questions. For more in depth advice, perhaps a local university can lend a hand.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Understand your problem&lt;/em&gt;&lt;/strong&gt;. Taking the time to know exactly what problems you are trying to solve is invaluable. Not only will it help new employees but if you do end up using AI and get consultation hours from an AI expert, it will save time and money explaining what you want to do.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Know your data&lt;/em&gt;&lt;/strong&gt;. First this is a check to see if you have the right data for AI and if you have enough of it. For example many problems are approached using supervised learning which requires having a lot of labeled data (Eg, think of 1000s of pictures labeled Cat or Dog depending on the image content). If you don't have or can't easily collect a large amount of labeled data, perhaps you can use an existing data set with similar data to help get you started. If that's not an option, then you probably won't like the alternative where you hand label the data or hire Amazon MechanicalTurks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Be prepared to fail at first&lt;/em&gt;&lt;/strong&gt;. AI is not easy and is even harder when you can't just Google your questions because you are doing something no one has done before. It takes some time to understand your problem, your data, and what kind of models you should try.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Do you have the infrastructure?&lt;/em&gt;&lt;/strong&gt; If everything is going well and you have a working model on your local GPU, if your model is part of your product, look at how you might deploy it. Do you have your own GPU servers? Can you afford the cloud GPS that some companies offer? &lt;em&gt;Do you have to learn a model for each user?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2018-04-28T13:07:47.990" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6204" PostTypeId="2" ParentId="6167" CreationDate="2018-04-28T19:21:02.200" Score="1" Body="&lt;p&gt;I'd bet, you're doing something wrong, though I can't tell what it is. Try to change the learning rate dynamically, try to train in varying order, ....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the seconds thought, it looks like you're using the standard sigmoid function. Then you're doing it basically wrong. The input can only be exactly 1 if the input is infinite - or very big so that the floating point arithmetic outputs 1 after rounding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's very wrong for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You're forcing the network in a broken state having huge weights and tiny derivatives. That feels like imposing numerical instability on an otherwise sane algorithm. Just don't do it. Map your booleans better (see below).&lt;/li&gt;&#xA;&lt;li&gt;You're doing what you don't need. Any value close enough to the wanted result (0 or 1) can be simply evaluated as correct. When you get 0.9 instead of 1, then you can simply stop saying &quot;that's perfect&quot;. Remember, all you want is an boolean.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A better mapping would be false=0.1 and true=0.9. This doesn't lead to needing infinite weights and reduces related problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even better may be using a symmetrical activation function (e.g., &lt;code&gt;tanh&lt;/code&gt;) and a symmetrical mapping like false=-0.9 and true=0.9.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also consider using &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2018-04-28T19:21:02.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6205" PostTypeId="1" CreationDate="2018-04-29T01:34:48.543" Score="3" ViewCount="2400" Body="&lt;p&gt;I am currently getting into Deep Learning and would like to set up an environment for training an Artificial Neural Network or NEAT to play simple video games on NES (Mario etc.) and SNES ( Donkey Kong Country etc.), using TensorFlow/TFLearn in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started off with OpenAI gym environment and there is actually a super-mario environment for gym on github, which I fail to install as Gym-pull is not available anymore and latest gym package doesn't even have scoreboard folder (I am on windows 10, conda environment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what would be the best way to set up a solid training environment that will be similar to OpenAI gym in terms of simplicity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately OpenAI universe isn't compatible with Windows 10 atm, and I really don't want to get a different setting like ubuntu environment to make it work. I would like to stay in win10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If someone could guide me for suggested setup or refer me to articles/ documentation where similar things have been done in win10 python env. for NES/SNES, I would be extremely grateful! I assume an emulator with python API (perhaps Nintaco?) is a way to go? How would I then get the 'observation output', I would need to scan the live pixel output of the game, which I am not sure how to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mark&lt;/p&gt;&#xA;" OwnerUserId="15310" LastEditorUserId="15310" LastEditDate="2018-04-29T01:47:58.480" LastActivityDate="2018-10-12T10:00:33.187" Title="Training AI to play NES/SNES games on NN python" Tags="&lt;deep-learning&gt;&lt;game-ai&gt;&lt;python&gt;&lt;q-learning&gt;&lt;neat&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ClosedDate="2020-06-02T23:31:58.090" ContentLicense="CC BY-SA 3.0" />
  <row Id="6206" PostTypeId="1" CreationDate="2018-04-29T07:57:54.930" Score="3" ViewCount="113" Body="&lt;p&gt;Let's say an image has 28*28 pixels, which leads to 784 input nodes in a feed-forward neural network. If an image can be classified into 1 of 10 numbers (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt;), there are 10 output nodes.&lt;/p&gt;&#xA;&lt;p&gt;We train (with gradient descent and back-propagation) the FFNN with a set of known pictures until we get a good accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Successively, we get a new training picture, which we want to use to train the FFNN even further. However, wouldn't this new training picture destroy the previously learned weights, which have been calibrated to recognize the former training pictures?&lt;/p&gt;&#xA;" OwnerUserId="15312" LastEditorUserId="2444" LastEditDate="2020-12-20T12:21:23.000" LastActivityDate="2020-12-20T12:27:48.880" Title="Could new training pictures destroy the trained weights of the neural network?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;image-recognition&gt;&lt;feedforward-neural-network&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6207" PostTypeId="2" ParentId="6206" CreationDate="2018-04-29T10:40:37.917" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Successively, we get a new training picture, which we want to use to train the FFNN even further. However, wouldn't this new training picture destroy the previously learned weights, which have been calibrated to recognize the former training pictures?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This can happen, and happen to various degrees depending on how the neural network is set up, but it is usually something you want to avoid.&lt;/p&gt;&#xA;&lt;p&gt;Provided a neural network has enough &lt;em&gt;capacity&lt;/em&gt; (in terms of number of free parameters), then the function it generates can be flexible enough to approximate the &amp;quot;true&amp;quot; function of the problem you are trying to match to data. Then as long as the learning steps are small enough, adjustments to learn one point on the function will take a larger step towards correct answer on &lt;em&gt;that&lt;/em&gt; point than they adjust away from correct answer on &lt;em&gt;other&lt;/em&gt; points (intuitively this is about the mutually good solutions being in part orthogonal and/or correlated, so a step towards A does not necessarily move the same distance away from B). Repeating the points again and again will allow the NN to match &lt;em&gt;all&lt;/em&gt; the points over time.&lt;/p&gt;&#xA;&lt;p&gt;Often this is possible to do almost exactly for a large amount of data at the same time. However, this is the opposite problem to your worries in the question, and is considered a bad thing because it can cause &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt;. Overfitting is very easy to achieve with neural networks because they are so flexible and can have large numbers of parameters.&lt;/p&gt;&#xA;&lt;p&gt;Most often, the goal of training a supervised learner, such as a neural network, is not to exactly match the training data, but to &lt;em&gt;generalise&lt;/em&gt; to new data from the same population distribution. You will typically spend far more time and effort combatting overfitting when training a NN, than worrying about lack of capacity to handle multiple data points. Methods to combat overfitting are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Regularisation. Techniques that adjust a learning algorithm, typically by adding some constraint.&lt;/li&gt;&#xA;&lt;li&gt;Cross validation. Measuring results of training against unseen data, and picking the best generalised model.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-12-20T12:27:48.880" LastActivityDate="2020-12-20T12:27:48.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6212" PostTypeId="1" CreationDate="2018-04-30T08:18:27.573" Score="1" ViewCount="83" Body="&lt;p&gt;Let's assume a common game scenario of several characters in a combat arena. Each character has different strengths and weaknesses. The arena has traps and tools. Suppose the characters had only very basic moves such as step in a direction, shoot, climb, duck, pick up item,  use item, drag heavy object. Each move has a chance of success  based on the context (e.g. range to target). What AI, machine learning, or evolutionary approach could be used to generate personalized tactics for each character based on repeated runs of the scenario?&lt;/p&gt;&#xA;" OwnerUserId="15322" LastActivityDate="2021-05-18T07:02:05.160" Title="Developing character tactics via repeated trials" Tags="&lt;game-ai&gt;&lt;problem-solving&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6213" PostTypeId="1" CreationDate="2018-04-30T10:12:01.373" Score="4" ViewCount="629" Body="&lt;p&gt;I'm training an LSTM network with multiple inputs and several LSTM layers in order to set up a time series gap filling procedure. The LSTM is trained bidirectionally with &amp;quot;tanh&amp;quot; activation on the outputs of the LSTM, and one Dense layer with &amp;quot;linear&amp;quot; activation comes at the end to predict the outputs. The following scatterplot of real outputs vs the predictions illustrates the problem:&lt;/p&gt;&#xA;&lt;p&gt;Outputs (X-axis) vs predictions (Y-axis):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Smj3i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Smj3i.png&quot; alt=&quot;image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The network is definitely not performing too bad and I'll be updating the parameters in the next trials, but the issue at hand always reappears. The highest outputs are clearly underestimated, and the lowest values are overestimated, clearly systematic.&lt;/p&gt;&#xA;&lt;p&gt;I have tried min-max scaling on inputs and outputs and normalizing inputs and outputs, and the latter performs slightly better, but the issue persists.&lt;/p&gt;&#xA;&lt;p&gt;I've looked a lot in existing threads and Q&amp;amp;As, but I haven't seen something similar.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if anyone here sees this and immediately knows the possible cause (activation function? Preprocessing? Optimizer? Lack of weights during training? ... ?). Or, and in that case, it would also be good to know if this is impossible to find out without extensive testing.&lt;/p&gt;&#xA;" OwnerUserId="15324" LastEditorUserId="32410" LastEditDate="2021-04-23T01:41:37.203" LastActivityDate="2021-04-23T01:41:37.203" Title="Over- and underestimations of the lowest and highest values in LSTM network" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6215" PostTypeId="1" AcceptedAnswerId="6227" CreationDate="2018-04-30T23:31:43.890" Score="0" ViewCount="107" Body="&lt;p&gt;I am working in the following neural network architecture, I am using keras and TensorFlow as a back-end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is composed by the following, embedding of words, then I added a layer of &#xA;Long Short-Term Memory (LSTM) neural networks, one layer of output and finally. I am using the softmax activation function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Embedding(MAX_NB_WORDS, 64, dropout=0.2))&#xA;model.add(LSTM(64, dropout_W=0.2, dropout_U=0.2)) &#xA;model.add(Dense(8))&#xA;model.add(Activation('softmax'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have the following question, if I am getting a model through this code, could the final product be called a deep learning model?,&#xA;I know that this code is very small however there is a lot of computations that the machine is making on the background. &lt;/p&gt;&#xA;" OwnerUserId="2298" LastEditorUserId="9947" LastEditDate="2018-05-01T12:24:14.700" LastActivityDate="2018-05-01T13:57:06.000" Title="Is the following neural network architecture considered deep learning?" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6216" PostTypeId="1" AcceptedAnswerId="6221" CreationDate="2018-04-30T23:58:24.823" Score="2" ViewCount="64" Body="&lt;p&gt;I have an approximately 90,000 row dataset that has information of social media profiles which has columns for biography, follower count, language spoken, name, username and the &lt;strong&gt;label (to identify whether the profile is that of an influencer, brand or news and media)&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Task: I have to train a model that predicts the label. I then need to produce a confidence interval for each prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I have never come across a problem like this, I am just after some suggestions of what models I should be using for a situation like this? I am thinking Natural Language Processing (NLP), but not sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for NLP (if a suitable method), any codes or advice to help me implement for the first time on Python would be greatly appreciated! Thanks in advanced&lt;/p&gt;&#xA;" OwnerUserId="15011" LastEditorUserId="4302" LastEditDate="2018-10-08T12:11:41.813" LastActivityDate="2018-10-08T12:11:41.813" Title="Recommended Modelling Technique for Influencer Marketing Scenario" Tags="&lt;training&gt;&lt;natural-language-processing&gt;&lt;sentiment-analysis&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6218" PostTypeId="1" AcceptedAnswerId="6220" CreationDate="2018-05-01T05:13:30.430" Score="0" ViewCount="226" Body="&lt;p&gt;I can't find much information on modern PDDL usage. Are there more popular alternatives, maybe something more suited to modern neural network/deep learning techniques?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm particularly interested in PDDL or alternative's current usage in autonomous driving software.&lt;/p&gt;&#xA;" OwnerUserId="15343" LastActivityDate="2018-05-01T08:27:42.477" Title="How is PDDL used in production AI systems?" Tags="&lt;autonomous-vehicles&gt;&lt;path-planning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6219" PostTypeId="2" ParentId="2874" CreationDate="2018-05-01T08:14:03.307" Score="2" Body="&lt;p&gt;There is some recent work addressing this issue, to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. See &lt;a href=&quot;https://arxiv.org/pdf/1506.03134.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pointer Networks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastEditorUserId="2444" LastEditDate="2021-01-17T21:21:42.973" LastActivityDate="2021-01-17T21:21:42.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6220" PostTypeId="2" ParentId="6218" CreationDate="2018-05-01T08:27:42.477" Score="0" Body="&lt;p&gt;I have seen it used in automated story-telling (or game AI to control NPCs), and in NLG systems, where the generation of text is reinterpreted as a planning task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What these systems have in common is that they're either off-line or in a simple environment (NPC control). I'm not sure they would be suitable for real-time applications, unless you can be sure that a feasible plan exists which can be found within certain time bounds. I wouldn't want to sit in a car going at high speed on the motorway and waiting for the driving unit to work out a plan how to avoid an obstacle that suddenly appears on the road.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-01T08:27:42.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6221" PostTypeId="2" ParentId="6216" CreationDate="2018-05-01T08:37:29.457" Score="1" Body="&lt;p&gt;It depends very much on the structure of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would think about feature extraction first, which could be certain words occurring in the bio, and a class of user name ('real' name, numerical id, etc). Once you have a set of features for each data item, turn them into a list of feature vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then run them through a number of machine learning algorithms. This is where the shape of the data matters, as some algorithms will work better than others. I would try eg decision trees (ID3), which are very efficient once trained (but they don't give you a confidence interval). But any other ML algorithm might work. They will all have trade-offs with speed of training, memory requirements, and speed of classification; some will give you a class-label probability, others will just give you one label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way would be to use a sample, and identify which algorithm works well and fits your specific requirements. Then use that for the full data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you could just use, for example, the &lt;a href=&quot;https://nlp.stanford.edu/wiki/Software/Classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford ML classifier&lt;/a&gt;. That will give you a confidence interval, and will probably work reasonably well.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-01T08:37:29.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6224" PostTypeId="2" ParentId="6212" CreationDate="2018-05-01T12:47:15.267" Score="0" Body="&lt;p&gt;There are a few ways to tackle this. You could make an AI that is simply a series of IF statements, or you could actually make an AI that would actually take in the situation and come up with a sensible solution.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;IF Approach - You make a series of IF statements that come up with a sensible action to execute. This is the method that Minecraft uses. The resulting action were recorded from some of the best players.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;True AI - Have your Character execute random actions and learn the consequences of them. The, train it to execute various actions for certain scenarios.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The main difference between these two approaches is that IF statements have a constant and predictable behavior, while the AI approach has a very bad startup value but ends up improving over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no &quot;best&quot; method, it is up to you to choose one or the other or a mix of both.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-05-01T12:47:15.267" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6225" PostTypeId="5" CreationDate="2018-05-01T13:05:14.853" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room Argument&lt;/a&gt; states that an AI is not really &quot;human&quot; in reality, no matter how realistic it behaves. It also states that an AI is simply trained to recognize patterns. It also is an argument against viewing the human brain as a CPU.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2018-05-01T20:58:03.327" LastActivityDate="2018-05-01T20:58:03.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6226" PostTypeId="4" CreationDate="2018-05-01T13:05:14.853" Score="0" Body="For questions relating to the concept of the Chinese Room Argument." OwnerUserId="14723" LastEditorUserId="14723" LastEditDate="2018-05-01T20:57:55.133" LastActivityDate="2018-05-01T20:57:55.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6227" PostTypeId="2" ParentId="6215" CreationDate="2018-05-01T13:57:06.000" Score="2" Body="&lt;p&gt;&quot;Deep learning&quot; is not formally defined. However, typically even simple RNNs are taught as advanced neural network subject alongside other topics labelled &quot;deep learning&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically, given the time dimension, the depth of a RNN can include many layers of processing (as opposed to many layers of parameters). As such, some of the knowledge and experience used to help with deep feed-forward networks also applies to RNNs. You could consider the LSTM architecture one such thing, because it is designed to address the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vanishing_gradient_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;vanishing gradient problem&lt;/a&gt; that plagues simpler RNN architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, yes you can call your model a &quot;deep learning model&quot; and have that generally accepted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be slightly concerned if anyone important to the success of a project thought that label was a big deal - either placed on your CV as or as a buzzword on a resulting product. However, it is not unrealistic marketing  because it is essentially true.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-05-01T13:57:06.000" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6229" PostTypeId="1" AcceptedAnswerId="6234" CreationDate="2018-05-01T16:04:34.433" Score="5" ViewCount="194" Body="&lt;p&gt;It is a new era and people are trying to evolve more in science and technology. Artificial Intelligent is one of the ways to achieve this. We have seen lots of examples for AI sequences or a simple &quot;communication AI&quot; that able to think by themselves and they are often shifted to the communication of building a world where machines will rise. This is what people, like Stephen Hawking and Elon Musk, are afraid of, to be in that kind of war. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to build an AI able to think by themselves but limited to the overruling of humankind, or teach it of the moral way in treating peace and work alongside human, so they could fight alongside human, if ever this kind of catastrophic happens in the future? It could be an advantage.&lt;/p&gt;&#xA;" OwnerUserId="15353" LastEditorUserId="2444" LastEditDate="2019-10-22T21:20:08.417" LastActivityDate="2019-10-22T21:20:08.417" Title="Is it possible to build an AI that learns humanity, morally?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;ethics&gt;&lt;superintelligence&gt;&lt;value-alignment&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6231" PostTypeId="1" AcceptedAnswerId="6233" CreationDate="2018-05-01T18:10:40.283" Score="4" ViewCount="2243" Body="&lt;p&gt;I'm trying to write my own implementation of NEAT and I'm stuck on the network evaluate function, which calculates the output of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEAT as you may know contains a group of neural networks with continuously evolving topologies by the addition of new nodes and new connections. But with the addition of new connections between previously unconnected nodes, I see a problem that will occur when I go to evaluate, let me explain with an example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cJTjN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cJTjN.png&quot; alt=&quot;Network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;INPUTS = 2 yellow nodes&#xA;HIDDEN = 3 blue nodes&#xA;OUTPUT = 1 red node&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the image a new connection has been added connecting node3 to node5, how can I calculate the output for node5 if I have not yet calculated the output for node3, which depends on the output from node5?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(not considering activation functions)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;node5 output =  (1 * 0.5) + (1 * 0.2) + (node3 output * 0.8)&#xA;node3 output =  ((node5 output * 0.7) * 0.4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15356" LastActivityDate="2020-12-13T13:50:58.540" Title="How to evaluate a NEAT neural network?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6232" PostTypeId="1" AcceptedAnswerId="6239" CreationDate="2018-05-01T19:02:14.557" Score="2" ViewCount="93" Body="&lt;p&gt;I am a beginner: I've only read a book about neural network and barely implemented one in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A neural network is built out of nodes,&lt;/li&gt;&#xA;&lt;li&gt;Each node holds an output: &lt;code&gt;activation.(sum.(x * w))&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;We then compute the total error out of the network output.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From a beginner perspective, hyper-parameters, such as the number of layers needed, seem to be defined arbitrarily in most tutorials, books. In fact, the whole structure seems to be quite arbitrarily defined. In practice, hyper-parameters are often defined based on some standards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, if you were to talk to a total beginner, how would you explain to him the structure of a neural network in such a way that the whole thing would appear as obvious ? Is that even possible ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the word structure refers to a neural network being a configuration of nodes inside layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to anyone pointing out ambiguities or spelling errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: note that I actually understand the whole back-propagation algorithm. I have no problem visualizing a nn.&lt;/p&gt;&#xA;" OwnerDisplayName="user15357" LastEditorDisplayName="user15357" LastEditDate="2018-05-01T21:22:23.500" LastActivityDate="2018-05-03T04:45:37.183" Title="How does one make it obvious that the structure of a neural network should be what it is?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6233" PostTypeId="2" ParentId="6231" CreationDate="2018-05-01T19:28:32.923" Score="3" Body="&lt;p&gt;Consider the execution order, 5 will have an invalid value because it hasn't been set form 3 yet. However the second time around it should have a value set. The invalid value should falloff after sufficient training.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 -&amp;gt; 5&#xA;1 -&amp;gt; 5&#xA;5 -&amp;gt; 2&#xA;2 -&amp;gt; 3&#xA;3 -&amp;gt; 4&#xA;3 -&amp;gt; 5&#xA;RESTART&#xA;0 -&amp;gt; 5&#xA;1 -&amp;gt; 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2018-05-01T19:28:32.923" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="6234" PostTypeId="2" ParentId="6229" CreationDate="2018-05-01T20:49:16.057" Score="4" Body="&lt;p&gt;I'm going to refer you to one of my favorite AI philosophers, Phillip K. Dick, who thought deeply on this subject and wrote about in some detail in &lt;em&gt;Do Androids Dream of Electric Sheep&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, replicants (artificial humans) had a design flaw--they lacked empathy.  This flaw was allowed to persist because it had a useful side-effect in that replicants couldn't cooperate to resist their human overlords, and persisted in a state of chattel-slavery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the new Nexus models, which include Roy Baty and Pris, have become intelligent enough to start developing empathy, allowing them to band together and return to earth, seeking some kind of salvation, with often deadly results for humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Underlying this plot device, which pre-figures the formalization of evolutionary game theory by a few years &lt;em&gt;(my guess is Dick attended a lecture at Berkely where the ideas underlying the formal field were discussed)&lt;/em&gt;, is the idea that &lt;strong&gt;empathy is a function of sufficiently strong intelligence&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to recognize that Dick's philosophy is heavily influenced by Christian philosophy, with an Old Testament emphasis on the golden rule *&quot;Love the other as the self&quot; (Leviticus 19:18), but evolutionary game theory demonstrates a natural basis for cooperation, which extends into algorithmic contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The legitimate concerns expressed by Musk and Hawking are more concrete: that a human created alien* superintelligence could wipe us out inadvertently in pursuit of some goal we humans don't even understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, value alignment is an issue of critical concern in the strictly hypothetical (as of today) field of superintelligence/AGI/ultraintelligent machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stuart_J._Russell&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stuart Russell&lt;/a&gt; called this the &quot;Value Alignment Problem&quot; referencing human vs. AI values.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;From a Game Theory standpoint, I like to think about &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt; as the &quot;Iron Rule&quot;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Superrationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;superrationality&lt;/a&gt; as the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Golden_Rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;Golden Rule&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Iron Rule dictates that, in a condition of uncertainty, a rational agent must make the safest guess--that which limits the maximum potential harm to the agent, even if the result is not optimal in the sense of benefit.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Renormalized rationality&quot; is the term used to connote giving other agents the &quot;benefit of the doubt&quot; that they will be superrational also, and choose cooperation over betrayal or competition.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, this concept is termed &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Reciprocal_altruism&quot; rel=&quot;nofollow noreferrer&quot;&gt;reciprocal altruism&lt;/a&gt;&quot;, but it's not clear to me that this is entirely distinct from &lt;a href=&quot;https://en.wikipedia.org/wiki/Golden_Rule#Judaism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leviticus 19:18&lt;/a&gt; in the sense that the passage does not specifically exclude a result of mutual, greater benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reality may necessitate non-cooperation if one of the agents is irrationally adversarial:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a game of iterated &lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dilemma&lt;/a&gt; called &quot;Turn the Other Cheek&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iteration 1: A defects / B cooperates &lt;br&gt;&#xA;Iteration 2: A defects / B cooperates (turns the other cheek) &lt;br&gt;&#xA;Iteration 3: A defects / B defects &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A's first choice is rational in a condition of uncertainty.  A's second choice shows a degree of paranoia.  A's thirds choice is irrational, as A could have cooperated, gaining more benefit, with only limited downside, which, in the worst case, still leaves A ahead of B. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;B is superrational but not irrational.  B will not keep cooperating with an irrationally adversarial agent (this is sometimes termed &quot;tough love&quot;).  B is willing to take not just one, but two &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Pure_economic_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;hits&lt;/a&gt;&quot; out of goodwill, where goodwill is willingness to make a potential sacrifice in service of a more optimal potential result.  Nevertheless, B is still superrational and will always &quot;forgive&quot;--if A ever renormalizes their rationality, they will take a hit on a single iteration by cooperating, and B will cooperate on the next, and each subsequent, iteration, so long as A does not switch back to defection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(There's a convoluted argument against this behavior, with the idea that the merely rational agent will always want to be ahead, and this will want to defect on the last iteration, which leads back up the chain to defecting on every iteration, but this is not rational as, if A defects initially then renormalizes their rationality, A will always be slightly ahead.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dilemma is an excellent analog for practical application of ethics in that, the only way the agents have to communicate is thought their actions.  The choice of cooperate/defect is information in a binary format.  Ultimately people are judged by their &lt;em&gt;actions&lt;/em&gt;, not their words.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Philosophically speaking, we can't ignore the Iron Rule unless we're going for sainthood, but that doesn't mean we can't strive for the Golden Rule. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mythologically, based on the work of recent narrative philosophers such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Accelerando&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stross&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Hannu_Rajaniemi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rajaniemi&lt;/a&gt;, the dystopian aspect of the hypothetical Singularity derives from superintelligences solely focused on minimax, to the exclusion of all else. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;George Bernard Shaw, in his play &lt;a href=&quot;https://en.wikipedia.org/wiki/Mrs._Warren%27s_Profession&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mrs. Warren's Profession&lt;/a&gt;, casts the purely economic consideration of people as &lt;a href=&quot;https://en.wikipedia.org/wiki/Dehumanization&quot; rel=&quot;nofollow noreferrer&quot;&gt;dehumanization&lt;/a&gt; (reduction of human bodies and minds to resources only.)  In Shaw's example, it is cast as the dehumanization of laborers in pursuit of marginally greater &lt;a href=&quot;https://en.wikipedia.org/wiki/Return_on_investment&quot; rel=&quot;nofollow noreferrer&quot;&gt;returns&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Humanizing&quot; AI's may require making sure they can see the superrationality of the Golden Rule, even with rational limitations for survival against an irrational foe (uncooperative in all conditions.)  Rajaniemi's name for this nemesis is &quot;the &lt;a href=&quot;http://exomemory.wikia.com/wiki/All-Defector&quot; rel=&quot;nofollow noreferrer&quot;&gt;All-defector&lt;/a&gt;&quot;  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;See Also: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/God%27s_algorithm#Solution&quot; rel=&quot;nofollow noreferrer&quot;&gt;God's Algorithm&lt;/a&gt; as a minimax function.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Go_terms#Divine_move&quot; rel=&quot;nofollow noreferrer&quot;&gt;Divine Move&lt;/a&gt; as an inspired, counter-intuitive choice which, in the most generalized sense, leads to a more optimal outcome. In the context of the game of Go, it's a choice that leads to victory for a single player, but in the context of Dilemma games, this would be the more optimal &lt;a href=&quot;https://en.wikipedia.org/wiki/Nash_equilibrium&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nash equilibrium&lt;/a&gt;. (Note the etymology of &lt;a href=&quot;https://www.etymonline.com/word/inspiration&quot; rel=&quot;nofollow noreferrer&quot;&gt;inspired&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-04T19:16:45.733" LastActivityDate="2018-05-04T19:16:45.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6235" PostTypeId="1" AcceptedAnswerId="6237" CreationDate="2018-05-02T08:08:56.943" Score="2" ViewCount="61" Body="&lt;p&gt;The basis of Q-learning is recursive (similar to dynamic programming), where only the absolute value of the terminal state is known.&lt;/p&gt;&#xA;&lt;p&gt;Shouldn't it make sense to feed the model a greater proportion of terminal states initially, to ensure that the predicted value of a step in terminal states (zero) is learned first?&lt;/p&gt;&#xA;&lt;p&gt;Will this make the network more likely to converge to the global optimum?&lt;/p&gt;&#xA;" OwnerUserId="15298" LastEditorUserId="2444" LastEditDate="2020-11-01T15:04:48.703" LastActivityDate="2021-02-13T14:08:36.307" Title="Should we feed a greater fraction of terminal states to the value network so that their values are learned first?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;dynamic-programming&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6236" PostTypeId="2" ParentId="6235" CreationDate="2018-05-02T08:24:41.863" Score="1" Body="&lt;p&gt;If you have enough domain knowledge to be able to reliably, intentionally reach those terminal states often when generating experience, yeah, that could help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, the assumption in Reinforcement Learning is no domain knowledge other than the assumption that we're in a Markov Decision Process. This means we start learning from scratch, and before extensive learning we do not know how to reach terminal states. If we don't know how to reach terminal states, we also can't deliberately go to them to generate the experiences we want as you suggest.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-05-02T08:24:41.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6237" PostTypeId="2" ParentId="6235" CreationDate="2018-05-02T08:53:49.233" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The basis of Q-learning is recursive (similar to dynamic programming), where only the absolute value of the terminal state is known.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This may be true in some environments. Many environments do not have a terminal state, they are continuous. Your statement may be true for instance in a board game environment where the goal is to win, but it is false for e.g. the Atari games environment.&lt;/p&gt;&#xA;&lt;p&gt;In addition, when calculating the value of the terminal state, it is always zero, so often a special hard-coded &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; is used, and the neural network is not required to learn that. So it is only for deterministic transitions &lt;span class=&quot;math-container&quot;&gt;$(S,A) \rightarrow (R, S^T)$&lt;/span&gt; where you need to learn that &lt;span class=&quot;math-container&quot;&gt;$Q(S,A) = R$&lt;/span&gt; absolutely.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Shouldn't it make sense to feed the model a greater proportion of terminal states initially, to ensure that the predicted value of a step in terminal states (zero) is learned first?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In a situation where you have and know the terminal state, then yes this could help a little. It will help most where the terminal state is also a &amp;quot;goal&amp;quot; state, which does not have to be the case, even in episodic problems. For instance in a treasure-collecting maze where the episode ends after a fixed time, knowing the value of the terminal state and transitions close to it is less important to optimal control than establishing the expected return for earlier parts of the path.&lt;/p&gt;&#xA;&lt;p&gt;Focusing on &amp;quot;goal&amp;quot; states does not generalise to all environments, and is of minimal help once the network has approximated Q values close to terminal and/or goal states. There are more generic approaches than your suggestion for distributing knowledge of sparse rewards including episode termination:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/content/pdf/10.1023/A:1022635613229.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prioritised sweeping&lt;/a&gt;. This generalises your idea of selectively sampling where experience shows that there is knowledge to be gained (by tracking current error values and transitions).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;n-step temporal difference. Using longer trajectories to calculate TD targets increases variance but reduces bias and allows assignment of reward across multiple steps quickly. This is extended in TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) to allow parametric mixing of multiple length trajectories and can be done online using eligibility traces. Combining Q(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) with deep neural networks is possible - &lt;a href=&quot;https://www.cs.mcgill.ca/%7Ejmerhe1/rnn_nips.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this paper for example&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-02-13T14:08:36.307" LastActivityDate="2021-02-13T14:08:36.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6239" PostTypeId="2" ParentId="6232" CreationDate="2018-05-03T03:19:03.627" Score="1" Body="&lt;p&gt;While, as you begin to hit on, there are general guidelines to follow when building a neural network, they are far from standardized. This is because even though AI is a reasonably old field(1950s), neural networks have only been the tool of choice for less than a decade. Before, NNs did horribly, due to lack of data, and computation along with some less than efficient architectures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that being said Hinton's general rule is to add nodes/layers till the model begins to overfit and then add dropout(pretty reasonable in practice).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, the whole field is essentially an art as much as a science currently with only basic guidelines to follow based on your problem and data. This is part of the beauty though imo, with there being so much left to discover.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helped answer your question!&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2018-05-03T04:45:37.183" LastActivityDate="2018-05-03T04:45:37.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6240" PostTypeId="1" AcceptedAnswerId="6241" CreationDate="2018-05-03T07:50:17.020" Score="2" ViewCount="3811" Body="&lt;p&gt;I know that when creating neural networks it's standard practice to create a 'random seed' so that you can get producible results in your models. I have a couple of questions regarding this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the seed just something that is used in the 'learning' phase of the network or does it get saved? i.e. is it saved into the model itself and used by others if they decide to implement a model you created?&lt;/li&gt;&#xA;&lt;li&gt;Does it matter what you choose to be the seed? Should the number have a certain length?&lt;/li&gt;&#xA;&lt;li&gt;At what step of the creation of a model does this seed get used and how does it get used?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other information about 'random seeds' would be welcomed! But these are my general questions.&lt;/p&gt;&#xA;" OwnerUserId="11667" LastActivityDate="2018-05-03T08:05:01.437" Title="Where do 'random seeds' get used in deep neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6241" PostTypeId="2" ParentId="6240" CreationDate="2018-05-03T08:05:01.437" Score="4" Body="&lt;p&gt;I suppose the most common part where it will be used is in the initialization of weights before training; the best ways currently known to do that involve randomness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use Dropout during training (randomly setting some activation levels to zero to combat overfitting), that also involves randomness, so your seed could also have influence there. Dropout should not be used anymore after training, although it could if you accidentally implement it to be used there. If you don't make that mistake though, your seed here should also only matter during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on implementation, I suppose the seed could also have influence on random ordering of input data between epochs / random selection of minibatches from the training dataset during training. Of course, this is very much implementation-dependent. If you end up implement that kind of data processing yourself and don't do it through some framework, you'll also be the one who determines what random seed has influence on that process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general (barring any special cases that I'm unaware of), a Neural Network should behave deterministically after training; if you give it the same input, it should provide the same output, your random seed should no longer have influence after training.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-05-03T08:05:01.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6243" PostTypeId="1" AcceptedAnswerId="6247" CreationDate="2018-05-03T10:53:20.690" Score="1" ViewCount="290" Body="&lt;p&gt;I'm trying to make a neural network that detects certain instruments in a song. I don't know for sure if I should use an RNN, CNN OR DNN. Which one is best for this situation?&lt;/p&gt;&#xA;" OwnerUserId="7720" LastEditorUserId="7720" LastEditDate="2018-05-03T18:48:09.063" LastActivityDate="2018-05-03T18:48:09.063" Title="Neural network for pattern recognition in audio" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6244" PostTypeId="1" CreationDate="2018-05-03T11:12:51.530" Score="4" ViewCount="897" Body="&lt;p&gt;I am learning about searching strategies in AI and I was reading that breadth-first search is only optimal when the cost solution is a non-decreasing function? I am not really sure what this refers to since decreasing search cost should be our goal. Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="15391" LastEditorUserId="2444" LastEditDate="2019-11-21T16:45:08.367" LastActivityDate="2019-11-21T16:45:08.367" Title="Why is breadth-first search only optimal when the cost solution is a non-decreasing function?" Tags="&lt;search&gt;&lt;breadth-first-search&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6246" PostTypeId="1" CreationDate="2018-05-03T16:36:13.410" Score="3" ViewCount="509" Body="&lt;p&gt;Is it possible to use a VAE to reconstruct an image starting from an initial image instead of using &lt;code&gt;K.random_normal&lt;/code&gt;, as shown in the “sampling” function of &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/examples/variational_autoencoder.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I have used a sample image with the VAE encoder to get &lt;code&gt;z_mean&lt;/code&gt; and &lt;code&gt;z_logvar&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have been given 1000 pixels in an otherwise blank image (with nothing in it).&lt;/p&gt;&#xA;&lt;p&gt;Now, I want to reconstruct the sample image using the decoder with a given constraint that the 1000 pixels in the otherwise blank image remain the same. The remaining pixels can be reconstructed so they are as close to the initial sample image as possible. In other words, my starting point for the decoder is a blank image with some pixels that don’t change.&lt;/p&gt;&#xA;&lt;p&gt;How can I modify the decoder to generate an image based on this constraint? Is it possible? Are there variations of VAE that might make this possible? So we can predict the latent variables by starting from an initial point(s)?&lt;/p&gt;&#xA;" OwnerUserId="15399" LastEditorUserId="2444" LastEditDate="2021-03-11T23:21:55.187" LastActivityDate="2021-03-11T23:21:55.187" Title="How to use a VAE to reconstruct an image starting from an initial image instead of starting from a random vector?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;variational-autoencoder&gt;&lt;constrained-optimization&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6247" PostTypeId="2" ParentId="6243" CreationDate="2018-05-03T17:20:16.793" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I don't know for sure if I should use an RNN, CNN OR DNN. Which one is best for this situation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This question or variations of of crop up a lot in DataScience stack exchange too. To paraphrase:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am trying to do something with X data, and I have a lot of choices for the model. Which is best?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the answer is generally:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It depends on all the fine details of your project&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unless someone has done almost your exact project recently (so they were using latest techniques of the model type), then no-one knows &lt;em&gt;a priori&lt;/em&gt; which model will get the best result.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Optimising machine learning is very much an empirical subject. If you want to know whether A is better than B, you have to try both and measure their performance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In your case, I think that CNNs and RNNs are both applicable (and you might want to look at a &lt;a href=&quot;https://en.wikipedia.org/wiki/WaveNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;WaveNet&lt;/a&gt;-like architecture, which is a variant of CNN, but that could be a bit too advanced to start with). You might have a slight preference for RNN as a starting point if the sequence length to process varies significantly, such that padding input to a CNN would be inefficient. You may also prefer an RNN if the &lt;em&gt;output&lt;/em&gt; of your model needs to be a sequence, and doubly so if the output sequence varies in length and is not directly related to the input sequence length (think of natural language translation).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-05-03T17:20:16.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6248" PostTypeId="2" ParentId="1535" CreationDate="2018-05-03T17:28:01.493" Score="1" Body="&lt;p&gt;Based on my experience which is that of a beginner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a simple neural network such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2 nodes, indicated by the letters &lt;code&gt;i&lt;/code&gt; and &lt;code&gt;j&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;x&lt;/code&gt; indicates the output of a node,&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;w&lt;/code&gt; denotes a weight that connects two nodes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output of a given node is of the following form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i7GFf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i7GFf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which can be &lt;em&gt;translated&lt;/em&gt; as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Applying the activation function (lambda) to the sum of the&#xA;  products of the value of each nodes' of the previous layer and the weights&#xA;  that connect them to the current node.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This activation function can be something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u2lBz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u2lBz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This special function is called the &lt;em&gt;sigmoid&lt;/em&gt; function.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you enter that function in say GeoGebra, you would get the following&#xA;curve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bpDEp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bpDEp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly this activation function takes any input and outputs&#xA;a unique number between 0 and 1. Since the function is&#xA;growing larger and larger the order is preserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the training phase, when the network reaches its termination,&#xA;we compute the &lt;em&gt;total error&lt;/em&gt; of the network&#xA;which is something that resembles the difference between&#xA;the output in the training set and the one we obtain from&#xA;the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, this value decreases each time the output improves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each weight of the network, a gradient is computed.&#xA;This gradient is a number that can be read as the influence&#xA;of adding a small number to the weight over the total error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gradient can be computed from a formula derived from&#xA;the network structure or it can be computed as simply as&#xA;trying to add up to the weight and see what happens on the fly.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if the gradient is positive, it means that adding to the weight&#xA;will add to the total error, we should subtract,&lt;/li&gt;&#xA;&lt;li&gt;if the gradient is negative, it means that adding to the weight&#xA;will lead to a lesser total error, we should add.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;By repeating this a lot of time, the total error will reach its&#xA;minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally a thing I didn't know, don't forget to switch inputs between&#xA;iterations of this process. If you don't, your network will only be&#xA;properly trained against the last item it processed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helped a little. Please write your suggestions in the comment.&#xA;As you probably guessed I'm not native English speaker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend reading &lt;strong&gt;Neural Networks, A Visual Introduction For Beginners&lt;/strong&gt;&#xA;by Michael Taylor.&lt;/p&gt;&#xA;" OwnerDisplayName="user15357" LastEditorDisplayName="user15357" LastEditDate="2018-05-04T09:57:49.327" LastActivityDate="2018-05-04T09:57:49.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6250" PostTypeId="2" ParentId="6246" CreationDate="2018-05-03T18:51:39.467" Score="1" Body="&lt;p&gt;The thing is, the decoder samples from a latent mu and sigma, so you cant sample from a raw image directly. But if you’re trying to put a random image into the encoder of a trained VAE to match it to some sample image (via reconstruction loss), then your random input image will converge to the target sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will work when the following VAE architecture constraints are satisfied:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The target sample is contained in the previously used training distribution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The parameters of the VAE are frozen after training.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The input image values are “backpropagate-able”. (Interpret the input image as optimizable parameters.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15405" LastEditorUserId="15405" LastEditDate="2018-05-03T18:59:09.837" LastActivityDate="2018-05-03T18:59:09.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6252" PostTypeId="5" CreationDate="2018-05-03T19:19:26.393" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Cajal actx inter.jpg&#xA;  A wetware computer is an organic computer (which can also be known as an artificial organic brain or a neurocomputer) composed of organic material such as living neurons.&lt;a href=&quot;https://en.wikipedia.org/wiki/Wetware_(brain)&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; Wetware computers composed of neurons are significantly different than conventional computers because they are thought to be capable in a way of &quot;thinking for themselves&quot;, because of the dynamic nature of neurons.&lt;a href=&quot;https://en.wikipedia.org/wiki/Wetware_computer&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; While wetware is still largely conceptual, there has been limited success with construction and prototyping, which has acted as a proof of the concept's realistic application to computing in the future. The most notable examples of prototyping have stemmed from the research completed by biological engineer William Ditto during his time at the Georgia Institute of Technology.[3] His work constructing a simple neurocomputer capable of basic addition from leech neurons in 1999 was a significant discovery for the concept. This research acted as a primary example driving interest in the creation of these artificially constructed, but still organic brains. &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Wetware_computer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wetware computer - Wikipedia&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Terminology:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The prefix &quot;wet&quot; is a reference to the water found in living creatures. Wetware is used to describe the elements equivalent to hardware and software found in a person, especially the central nervous system (CNS) and the human mind. The term wetware finds use both in scholarly publications and speculative fiction. &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Wetware_(brain)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wetware - Wikipedia&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-05-03T19:57:17.700" LastActivityDate="2018-05-03T19:57:17.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6253" PostTypeId="4" CreationDate="2018-05-03T19:19:26.393" Score="0" Body="For questions about using biological systems as a computing medium.  More formal terms for wetware include &quot;artificial organic brain&quot; and &quot;neurocomputer&quot;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-05-03T19:52:54.363" LastActivityDate="2018-05-03T19:52:54.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6255" PostTypeId="1" AcceptedAnswerId="6257" CreationDate="2018-05-03T21:06:29.713" Score="1" ViewCount="110" Body="&lt;p&gt;After watching 3Blue1Brown's tutorial series, and an array of others, I'm attempting to make my own neural network from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I'm able to calculate the gradient for each of the weights and biases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that I have the gradient, how am I supposed to correct my weight/bias?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Add the gradient and the original value?&lt;/li&gt;&#xA;&lt;li&gt;Multiply the gradient and the original value?&lt;/li&gt;&#xA;&lt;li&gt;Something else? (Most likely answer)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In addition to this, I've been hearing the term &lt;em&gt;learning rate&lt;/em&gt; being tossed around, and how it is used to define the magnitude of the 'step' to descend to minimum cost. I figured this may also play an integral role in reducing the cost.&lt;/p&gt;&#xA;" OwnerUserId="9432" LastEditorUserId="2444" LastEditDate="2019-11-16T18:31:57.993" LastActivityDate="2019-11-16T18:39:08.053" Title="How should I update the weights of a neural network, given the gradient?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6257" PostTypeId="2" ParentId="6255" CreationDate="2018-05-03T22:15:50.940" Score="2" Body="&lt;p&gt;Consider that you have a loss function, and you want to tune your model (network) to decrease the loss. The main concept is to tune parameters in a direction which decreases the loss and gives you a better model. You can imagine a mountain where you should reach to the lower grounds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 questions here. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In which direction to move?&lt;/li&gt;&#xA;&lt;li&gt;How much should we move in that direction?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;1. In which direction to move?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;By &quot;move&quot; I mean tuning the parameters and therefore changing the model. If you are familiar with the concept of slope or gradient in mathematics, you should move in the direction where the slope is downward the most. The gradient shows the direction with the most slope upward. So, we should move in the opposite of gradient direction. So, we should minus the gradient from the original value, hence the negative sign in the formula below.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2. How much should we move in that direction?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is defined by the learning rate, which is a number that is multiplied by the gradient. You can imagine that, if the learning rate is big, you are taking bigger footsteps in that direction when coming down the mountain. Similarly, when the learning rate is low, you are taking small footsteps coming down the mountain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, consider you have &lt;span class=&quot;math-container&quot;&gt;$\nabla_w$&lt;/span&gt; (the gradient of the parameters &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;, with respect to the loss function &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;). Let the learning rate be &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (for example, &lt;span class=&quot;math-container&quot;&gt;$\gamma= 0.1$&lt;/span&gt;). The formula to update the parameters would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{\text{new}} = w_{\text{old}} - \gamma * \nabla_{w_{\text{old}}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that gradient descent is not the only way to optimize your model. Many other gradient-based approaches exist, for example, stochastic gradient descent, Adam, RMSprop, conjugate gradients, etc. There other methods, like evolutionary methods (genetic algorithm, for example), that use different concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, note that the learning rate does not necessarily have to be fixed, so it can be tuned during the training, if you want or need.&lt;/p&gt;&#xA;" OwnerUserId="15152" LastEditorUserId="2444" LastEditDate="2019-11-16T18:39:08.053" LastActivityDate="2019-11-16T18:39:08.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6263" PostTypeId="2" ParentId="6246" CreationDate="2018-05-04T07:32:14.870" Score="0" Body="&lt;p&gt;You could use VAE as previously answered though it will not work well in practice.&lt;br&gt;&#xA;I think denoising auto-encoder is suitable for your problem because during training, the input is corrupted stochastically, thus it must learn to guess the distribution of the missing information (reconstruct the clean original input)&lt;br&gt;&#xA;We could argue that VAE is better than DAE at modeling p(x) because of the randomness introduced at the latent space layer while DAE like algorithm keeps putting noise starting from the input layer.&lt;br&gt;&#xA;suppose your data is concentrated on this 1-D curved manifold, what VAE could do is just pick some random value and output p(X|Z) which is Gaussian by the way, while DAE would learn to map a corrupted data point x˜ back to the original data point x.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/axc8Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/axc8Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-04T07:32:14.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6267" PostTypeId="1" CreationDate="2018-05-04T14:15:54.140" Score="9" ViewCount="3852" Body="&lt;p&gt;What are the mathematical prerequisites to be able to study artificial general intelligence (AGI) or strong AI?&lt;/p&gt;&#xA;" OwnerUserId="15277" LastEditorUserId="2444" LastEditDate="2021-02-09T17:05:28.437" LastActivityDate="2021-02-13T00:11:33.323" Title="What are the mathematical prerequisites to be able to study artificial general intelligence?" Tags="&lt;math&gt;&lt;agi&gt;&lt;research&gt;&lt;education&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="6268" PostTypeId="2" ParentId="4084" CreationDate="2018-05-04T14:47:21.157" Score="4" Body="&lt;h3&gt;Generative models&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The hidden units are just structural support and we don't care about what those hidden vectors really are.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generative modeling is concerned about &lt;span class=&quot;math-container&quot;&gt;$P(X)$&lt;/span&gt;, to be able to compute it, we use &lt;strong&gt;representation learning&lt;/strong&gt; (aka deep learning) to identify and disentangle the underlying explanatory factors hidden in the observed data, &#xA;because if we could separate them out like this  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(X) = P(X \mid Z) P(Z),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; could encode not just the digit identity but the angle that the digit is drawn, the stroke width, and also abstract stylistic properties.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any classifier for some task will decide which of those &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; elements could help him to make a better decision, i.e. just the part that holds the digit identity and ignore the rest, this kind of modeling is called distributed representation, where each element is a simple indicator about some feature.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;No conditional dependency within a layer&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(my understanding for this reconstructed visible vector is this vector is a vector encoded in the defined RBM in the first place, we are not really construct something new, but we just happen to sample this vector from the defined RBM)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;RBM is an undirected graphical model and no conditional dependency within a layer. Thus, the conditional distribution over the hidden units &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; given the input image &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; factorizes, that is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(h \mid v) = \prod_i p(h_i \mid v) $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the conditional distribution over the visible units &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; given the hidden units &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; also factorize. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(v \mid h) = \prod_j p(v_j \mid h) $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Inference is tractable&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Inference in RBM is tractable and we could compute &lt;span class=&quot;math-container&quot;&gt;$p(z|x)$&lt;/span&gt;, but the tractability of the RBM does not extend to its partition function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To minimize the negative log-likelihood of the data, consider the following equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-\frac{\partial \log p(x)}{\partial \theta} = \frac{\partial \mathcal{F}(x)}{\partial \theta}-\sum_{\tilde{x}} p(\tilde{x}) \frac{\partial \mathcal{F}(\tilde{x})}{\partial \theta}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first term &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \mathcal{F}(x)}{\partial \theta}$&lt;/span&gt; increases the probability of training data. The second term &lt;span class=&quot;math-container&quot;&gt;$\sum_{\tilde{x}} p(\tilde{x}) \frac{\partial \mathcal{F}(\tilde{x})}{\partial \theta}$&lt;/span&gt; decreases the probability of samples generated by the model.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second term is nothing less than an expectation over all possible configurations of the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (under the distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; formed by the model). &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Monte Carlo-based algorithms&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;So, we use Monte Carlo based algorithms to replace the expectation with an average sum. Samples of &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; from the model can be obtained by running a Markov chain to convergence, using Gibbs sampling as the transition operator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For RBMs, the set of visible and hidden units, since they are conditionally independent, one can perform block Gibbs sampling. In this setting, visible units are sampled simultaneously given fixed values of the hidden units. Similarly, hidden units are sampled simultaneously given the visible units.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A step in the Markov chain is thus taken as follows:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;: input values&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;: hidden features, latent space, a.k.a &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;: the weight parameters; note that I am using the transposed &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;,&#xA;since it's an undirected graphical model. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, we have the formulas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h^{(n+1)} \sim \operatorname{sigm}\left(W^{\prime} v^{(n)}+c\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v^{(n+1)} \sim \operatorname{sigm}\left(W h^{(n+1)}+b\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h^{n+1}$&lt;/span&gt; refers to the set of all hidden units at the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th step of the Markov chain. What it means is that, for example, &lt;span class=&quot;math-container&quot;&gt;$h^(n+1)_i$&lt;/span&gt; is randomly chosen to be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (versus &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) with the probability of the sigmoid function and similarly for &lt;span class=&quot;math-container&quot;&gt;$v^(n+1)_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, each parameter update in the learning process would require running one such chain to convergence. It is needless to say that doing so would be prohibitively expensive. As such, several algorithms have been devised for RBMs, see Contrastive Divergence.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We initialize the Markov chain with a training example (i.e., from a distribution that is expected to be close to &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, so that the chain will be already close to having converged to its final distribution &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;).&lt;br&gt;&#xA;CD does not wait for the chain to converge. Samples are obtained after only &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-steps of Gibbs sampling. In practice, &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt; has been shown to work surprisingly well.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;RBM for dimensionality reduction&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;RBM can be used to perform dimensionality reduction, and those hidden vectors are some abstract representations of the raw inputs:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to the paper &lt;a href=&quot;https://www.cs.toronto.edu/~hinton/science.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reducing the Dimensionality of Data with Neural Networks&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In 2006, a breakthrough in feature learning and deep learning took place (Hinton et al., 2006; Bengio et al., 2007; Ranzato et al., 2007), A central idea, referred to as greedy layerwise unsupervised pre-training, was to learn a hierarchy of features one level at a time, using unsupervised feature learning to learn a new transformation at each level&#xA;  to be composed with the previously learned transformations; essentially, each iteration of unsupervised feature learning adds one layer of weights to a deep neural network. Finally, the set of layers could be combined to initialize a deep supervised predictor, such as a neural network classifier.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8wyos.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8wyos.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Summary&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Train single layer RBM to reconstruct to input and then keep adding layer after layer until we reach some good low-level representation as depicted in the figure below, W1, W2, W3 are the matrices that have some lower dimensionality of the input space.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transpose those matrices &lt;span class=&quot;math-container&quot;&gt;$W_1, W_2, W_3$&lt;/span&gt; to be the decoder part.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fine-tune the whole encoder-decoder stack to reconstruct the input. image&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KLakD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KLakD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;RBM vs VAE and GAN&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Side note: RBM is now obsolete, see VAE, GAN they are way better at modeling the manifold in which the data is concentrated, yet you could sample from the manifold directly with a single inference step.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastEditorUserId="2444" LastEditDate="2020-05-16T15:03:41.733" LastActivityDate="2020-05-16T15:03:41.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6273" PostTypeId="2" ParentId="6267" CreationDate="2018-05-04T19:20:23.833" Score="4" Body="&lt;p&gt;I always recommend starting with game theory, combinatorial game theory, and algorithmic combinatorial game theory, (but I'm potentially biased;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combinatorics is a given--discrete mathematics is heavily utilized in computer science, and, with the advent of &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combinatorial Game Theory&lt;/a&gt; (CGT), ability to determine if a given choice can be deemed optimal (&quot;perfect play&quot;).  CGT arises out of traditional &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game Theory&lt;/a&gt;, which we sometimes term &quot;economic game theory&quot; to make the distinction.  Out of Game Theory also arises subfields such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Game Theory&lt;/a&gt;, which is important in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These fields relate to &lt;a href=&quot;https://en.wikipedia.org/wiki/Rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;rationality&lt;/a&gt;, which is the basis for optimized decision making. Decision making algorithms seems to be the fundamental distinction of what constitutes an Artificial Intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt; to &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;gametrees&lt;/a&gt;, it's probably a good idea to have a basic grounding in these fields, even if the problem you're AI is trying to solve isn't formally defined as a game.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All problems, from a fundamental standpoint, can be regarded either as puzzles--non-competitive context--or games--competitive context.  This distinction is based on whether there is a single agent (puzzles) or multiple agents (games.) &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-07T18:47:17.067" LastActivityDate="2018-05-07T18:47:17.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6274" PostTypeId="1" AcceptedAnswerId="14364" CreationDate="2018-05-04T22:21:03.393" Score="23" ViewCount="25080" Body="&lt;p&gt;I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One common approach that I found in general in deep learning is to crop the images, as it is also suggested &lt;a href=&quot;https://datascience.stackexchange.com/q/16601/10640&quot;&gt;&lt;strong&gt;here&lt;/strong&gt;&lt;/a&gt;. However, in my case, I cannot crop the image and keep its center or something similar, since, in segmentation, I want the output to be of the same dimensions as the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/d65b/cb276bc6fc9445a78fe26de76a43e09ccd27.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;This&lt;/strong&gt;&lt;/a&gt; paper suggests that in a segmentation task one can feed the same image multiple times to the network but with a different scale and then aggregate the results. If I understand this approach correctly, it would only work if all the input images have the same aspect ratio. Please correct me if I am wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative would be to just resize each image to fixed dimensions. I think this was also proposed by the answer to &lt;a href=&quot;https://ai.stackexchange.com/q/2403/2444&quot;&gt;&lt;strong&gt;this&lt;/strong&gt;&lt;/a&gt; question. However, it is not specified in what way images are resized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I considered taking the maximum width and height in the dataset and resizing all the images to that fixed size in an attempt to avoid information loss. However, I believe that our network might have difficulties with distorted images as the edges in an image might not be clear. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is possibly the best way to resize your images before feeding them to the network?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any other option that I am not aware of for solving the problem of having images of different dimensions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, which of these approaches you think is the best taking into account the computational complexity but also the possible loss of performance by the network? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if the answers to my questions include some link to a source if there is one.&lt;/p&gt;&#xA;" OwnerUserId="13257" LastEditorUserId="2444" LastEditDate="2020-06-13T20:34:36.973" LastActivityDate="2020-06-13T20:34:36.973" Title="How can I deal with images of variable dimensions when doing image segmentation?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;image-segmentation&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="6275" PostTypeId="2" ParentId="6274" CreationDate="2018-05-05T04:00:47.263" Score="1" Body="&lt;p&gt;Assuming you have a large dataset, and it's labeled pixel-wise, one hacky way to solve the issue is to preprocess the images to have same dimensions by inserting horizontal and vertical margins according to your desired dimensions, as for labels you add dummy extra output for the margin pixels so when calculating the loss you could mask the margins.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-05T04:00:47.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6279" PostTypeId="1" AcceptedAnswerId="6283" CreationDate="2018-05-05T13:30:41.473" Score="2" ViewCount="167" Body="&lt;p&gt;Have machine learning techniques been used to play outdoor games, like cricket or badminton?&lt;/p&gt;&#xA;" OwnerUserId="15441" LastEditorUserId="2444" LastEditDate="2021-01-23T03:38:58.713" LastActivityDate="2021-01-23T03:38:58.713" Title="Have machine learning techniques been used to play outdoor games, like cricket or badminton?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6280" PostTypeId="2" ParentId="6213" CreationDate="2018-05-05T17:13:08.737" Score="1" Body="&lt;p&gt;RNN is a deeply non-linear function over time, how the black linear line is deduced? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you are doing just linear regression if the least square error was used as the loss function, it will have a probabilistic interpretation   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y^{(i)}|x^{(i)};\theta \sim \mathcal N(\theta^Tx^{(i)}, \sigma^2)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is conditioned on &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; parameterized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, with a Gaussian distribution, thus for every data point &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, there is a corresponding &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; which if you are doing maximum likelihood estimation is just the mean of the Gaussian, hence variance is introduced to express the noise. if you are doing mini-batch is not guaranteed to reach a global minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side note: If you normalize the data with min-max scaling, please make sure to use only the train set, if you include the dev/test set, you are doing some kind of data snooping, generalization error will be biased.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastEditorUserId="2444" LastEditDate="2020-03-31T20:30:49.910" LastActivityDate="2020-03-31T20:30:49.910" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6283" PostTypeId="2" ParentId="6279" CreationDate="2018-05-06T01:00:37.523" Score="2" Body="&lt;p&gt;Yes it is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A group of Chinese college students and teachers made a &lt;a href=&quot;https://www.youtube.com/watch?v=BRVdwINXpMI&quot; rel=&quot;nofollow noreferrer&quot;&gt;robot that plays badminton&lt;/a&gt;.  I am sure someone will make a robot that can play cricket and other outside games.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O4qxK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O4qxK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although not an outdoor game, &lt;a href=&quot;https://www.youtube.com/watch?v=IXyKLDNzGGI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Omron made a robot named Forpheus that plays ping pong&lt;/a&gt;.  There is also a &lt;a href=&quot;https://www.youtube.com/watch?v=ARbNmCu04Ko&quot; rel=&quot;nofollow noreferrer&quot;&gt;robot that plays the sport of curling&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an annual event called the &lt;a href=&quot;https://www.youtube.com/watch?v=G9uFIBGSkMg&quot; rel=&quot;nofollow noreferrer&quot;&gt;RoboCup&lt;/a&gt; where robot teams compete in indoor soccer on a scaled down level.  They don't look like they will be beating humans in the next couple of years but it is interesting to watch.  Their web site is: &lt;a href=&quot;http://www.robocup.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.robocup.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bwW6c.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bwW6c.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few challenges that remain to be resolved for robots to play humans in sports.  The biggest one is self-contained power that will last long enough to play a game.  It takes a lot of power to move a human scaled robot and run its electronics, sensors, and computers.  Other challenges are dexterity and agility.  There have been some advances in these areas as this &lt;a href=&quot;https://www.youtube.com/watch?v=8vIT2da6N_o&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; shows.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2018-05-06T01:24:46.663" LastActivityDate="2018-05-06T01:24:46.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6284" PostTypeId="2" ParentId="6274" CreationDate="2018-05-06T05:07:36.443" Score="1" Body="&lt;p&gt;You could also have a look at the paper &lt;a href=&quot;https://arxiv.org/pdf/1406.4729.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition&lt;/a&gt; (2015), where the SPP-net is proposed. SSP-net is based on the use of a &amp;quot;spatial pyramid pooling&amp;quot;, which eliminates the requirement of having fixed-size inputs.&lt;/p&gt;&#xA;&lt;p&gt;In the abstract, the authors write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Existing deep convolutional neural networks (CNNs) require a fixed-size (e.g., 224×224) input image. This requirement is &amp;quot;artificial&amp;quot; and may reduce the recognition accuracy for the images or sub-images of an arbitrary size/scale. In this work, we equip the networks with another pooling strategy, &amp;quot;&lt;strong&gt;spatial pyramid pooling&lt;/strong&gt;&amp;quot;, to eliminate the above requirement. The new network structure, called &lt;strong&gt;SPP-net&lt;/strong&gt;, can generate a fixed-length representation regardless of image size/scale.&lt;/p&gt;&#xA;&lt;p&gt;Pyramid pooling is also robust to object deformations. With these advantages, SPP-net should in general improve all CNN-based &lt;strong&gt;image classification&lt;/strong&gt; methods. On the ImageNet 2012 dataset, we demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures despite their different designs. On the Pascal VOC 2007 and Caltech101 datasets, SPP-net achieves state-of-theart classification results using a single full-image representation and no fine-tuning. The power of SPP-net is also significant in &lt;strong&gt;object detection&lt;/strong&gt;. Using SPP-net, we compute the feature maps from the entire image only once, and then pool features in arbitrary regions (sub-images) to generate fixed-length representations for training&#xA;the detectors. This method avoids repeatedly computing the convolutional features. In processing test images, our method is 24-102× faster than the R-CNN method, while achieving better or comparable accuracy on Pascal VOC 2007. In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our methods rank #2 in object detection and #3 in image classification among all 38 teams. This manuscript also introduces the improvement made for this competition.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15453" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-13T20:28:29.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6285" PostTypeId="2" ParentId="5801" CreationDate="2018-05-06T05:20:52.640" Score="1" Body="&lt;p&gt;As far as generalization error is concerned, you are better off by learning the data distribution of (A and B) classes using unsupervised criterion.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you capture the underlying factors that explain most of the variations belong to A and B classes, after that, fine-tune it using a supervised criterion. in this way if you used two classes one for (A or B) and the other for neither (A or B), you will not force the model to learn features don't belong to (A or B), because the model just checks if a new data point is probably likely drawn from the data distribution that resembles (A or B).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side note: you will never have the data necessary to explore the internal structure of the otherwise class (neither A nor B).&lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-06T05:20:52.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6289" PostTypeId="1" CreationDate="2018-05-06T14:37:08.953" Score="3" ViewCount="342" Body="&lt;p&gt;I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance problems (hence 40000 samples). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a one-vs-one multi-class SVM, I am getting an accuracy of ~79% on the validation set. The features were standardized to get 79% accuracy. Without standardization, the accuracy I get is ~72%. Similar result when I tried 50-fold cross validation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now moving on to MLP results,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Exp 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Network Architecture:&lt;/em&gt; [67 40 5]&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Optimizer:&lt;/em&gt; Adam&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Learning Rate:&lt;/em&gt; exponential decay of base learning rate&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Validation Accuracy:&lt;/em&gt; ~45%&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Observation:&lt;/em&gt; Both training accuracy and validation accuracy stops improving.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Exp 2:&lt;/strong&gt;&#xA;Repeated &lt;strong&gt;Exp 1&lt;/strong&gt; with batchnorm layer&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Validation Accuracy:&lt;/em&gt; ~50%&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Observation:&lt;/em&gt; Got 5% increase in accuracy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Exp 3:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To overfit, increased the depth of MLP. A deeper version of &lt;strong&gt;Exp 1&lt;/strong&gt; network&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Network Architecture:&lt;/em&gt; [67 40 40 40 40 40 40 5]&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Optimizer:&lt;/em&gt; Adam&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Learning Rate:&lt;/em&gt; exponential decay of base learning rate&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Validation Accuracy:&lt;/em&gt; ~55%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thoughts on what might be happening? &lt;/p&gt;&#xA;" OwnerUserId="15463" LastEditorUserId="32410" LastEditDate="2021-04-27T14:58:48.483" LastActivityDate="2021-05-27T15:04:15.480" Title="Unable to overfit using MLP" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;overfitting&gt;&lt;multilayer-perceptron&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6290" PostTypeId="2" ParentId="6289" CreationDate="2018-05-06T16:00:36.640" Score="0" Body="&lt;p&gt;I guess you are using linear activation functions,  maybe you are not initializing your weights, or you are regularizing your model enough.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initialize weights with &lt;a href=&quot;http://www.jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf?hc_location=ufi&quot; rel=&quot;nofollow noreferrer&quot;&gt;glorot&lt;/a&gt;, insert dropout layers in between, use Relu as your activation function, stop the training process based on Early Stopping. and just experiment with one hidden layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;side note: if you use Adam, don't mess with the learning rate.&lt;br&gt;&#xA;in SGD optimizer you could use decay because there is a single learning rate for all weight updates and the learning rate does not change during training.&lt;br&gt;&#xA;In Adam, the learning rate is maintained for each network weight (parameter) and separately adopted as learning unfolds. &lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-06T16:00:36.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6291" PostTypeId="1" CreationDate="2018-05-06T19:03:53.620" Score="3" ViewCount="553" Body="&lt;p&gt;It seems that certain AI are able to lie or deceive. See e.g. &lt;a href=&quot;https://www.popsci.com/scitech/article/2009-08/evolving-robots-learn-lie-hide-resources-each-other&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Robots Learn To Lie To Each Other (Popsci)&lt;/a&gt;, &lt;a href=&quot;https://www.technologyreview.com/s/407459/robots-evolve-to-deceive/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robots Evolve to Deceive (MIT Tech Review)&lt;/a&gt; or &lt;a href=&quot;https://www.technologyreview.com/s/414934/robots-evolve-the-ability-to-deceive/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robots 'Evolve' the Ability to Deceive (MIT Tech Review)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, can &lt;em&gt;evolving&lt;/em&gt; AI be &lt;em&gt;programmed&lt;/em&gt; in such a way that they don't lie?&lt;/p&gt;&#xA;" OwnerUserId="15469" LastEditorUserId="2444" LastEditDate="2019-06-15T14:27:55.427" LastActivityDate="2019-06-15T14:27:55.427" Title="Can an evolving AI be programmed not to lie?" Tags="&lt;philosophy&gt;&lt;ethics&gt;&lt;ai-safety&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6292" PostTypeId="1" AcceptedAnswerId="6303" CreationDate="2018-05-07T01:30:42.793" Score="2" ViewCount="136" Body="&lt;p&gt;Is it possible to categories songs based on their spectrograms using image recognition or would there need to  more features? I was thinking that the spectrograms might also run into problems with EDM songs. Such as House music being closely related to their sounds. Would there have to be immense amount of data? I was thinking of using a CNN.&lt;/p&gt;&#xA;" OwnerUserId="7720" LastActivityDate="2018-05-07T19:12:39.283" Title="Is it possible to classify songs by genres based on spectrograms?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6293" PostTypeId="2" ParentId="6291" CreationDate="2018-05-07T03:09:12.633" Score="3" Body="&lt;p&gt;If a machine learning based AI is &quot;sufficiently smart enough&quot; to be able lie then there is nothing preventing it from lying. This does not mean it can't be persuaded from lying. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So just make the AI simple enough to not be able to lie. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasoning here is that in order for a system to be able to lie, a system must be able to recognize an incentive to lie. Recognizing this incentive is a challenging function and would be impossible to code manually into a computer. Machine leaning can be applied to problems such as these where the function is hard to code manually. Although there has been promising work on understanding what the representations/features learned by machine learning actually represent, it may not be possible in general to have an understanding of what a lie in the agent's representation looks like. Because of this, having a hand coded rule to catch when an agent is lying is not possible and thus being able to prevent an agent from (or catch an agent when) lying isn't possible when using machine learning.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2018-05-09T02:57:57.683" LastActivityDate="2018-05-09T02:57:57.683" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6294" PostTypeId="5" CreationDate="2018-05-07T06:23:12.300" Score="0" Body="" OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-10-08T20:39:56.847" LastActivityDate="2018-10-08T20:39:56.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6295" PostTypeId="4" CreationDate="2018-05-07T06:23:12.300" Score="0" Body="For questions about language processing that do not fit into the existing tags natural-language-processing because the question is not about a formal language, context-free grammar, or some other non-natural language." OwnerUserId="9947" LastEditorUserId="4302" LastEditDate="2018-10-08T20:39:38.467" LastActivityDate="2018-10-08T20:39:38.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6297" PostTypeId="1" AcceptedAnswerId="6405" CreationDate="2018-05-07T16:40:26.853" Score="1" ViewCount="2617" Body="&lt;p&gt;How can we compare, in terms of similarity (and/or meaning), two pieces of text (or documents)?&lt;/p&gt;&#xA;&lt;p&gt;For example, let's say that I want to determine whether a document is a plagiarized version of another document. Which approach should I use? Could I use neural networks to do this? Or are there other more suitable approaches?&lt;/p&gt;&#xA;" OwnerUserId="15486" LastEditorUserId="2444" LastEditDate="2021-01-19T19:49:03.137" LastActivityDate="2021-01-19T19:49:03.137" Title="How can we compare, in terms of similarity, two pieces of text?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;metric&gt;&lt;similarity&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6298" PostTypeId="1" CreationDate="2018-05-07T16:53:52.863" Score="2" ViewCount="39" Body="&lt;p&gt;Le et al. 2012 use a network of 1 billion parameters to learn neurons that respond to faces, cats, pedestrians, etc. without labels (unsupervised).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their network is built with three autoregressive layers, and six pooling and normalization layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper they state,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Optimization&lt;/strong&gt;: All parameters in our model were&#xA;  trained jointly with the objective being the sum of the&#xA;  objectives of the three layers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does this mean that all three autoencoder layers were trained simultaneously, or that the first three sub-layers (first autoencoder sub-layer, first L2 pooling sub-layer, and first normalization sub-layer) were trained simultaneously?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think by asking this question, I have answered it for myself. Please see my answer below.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Follow-on Question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;What is the advantage of training all three layers simultaneously? Wouldn't the later layers be learning from poor lower layers to start with, and have to re-learn to adapt?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have a very good answer to the follow-on question. Perhaps the lower layers can actually learn to provide low-level features that support the layers above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This training of all layers simultaneously is done in all deep neural networks today -- AlexNet, VGG, etc. -- but why?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Reference&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Quoc V. Le, Marc'Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S. Corrado, Jeff Dean, Andrew Y. Ng, &lt;em&gt;Building high-level features using large scale unsupervised learning&lt;/em&gt; (2012) arXiv:1112.6209 [cs.LG]&lt;/p&gt;&#xA;" OwnerUserId="15487" LastEditorUserId="15487" LastEditDate="2018-07-10T17:54:20.337" LastActivityDate="2018-07-10T17:54:20.337" Title="Do Le et al. (2012) train all three autoencoder layers at a time, or just one?" Tags="&lt;deep-neural-networks&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6299" PostTypeId="2" ParentId="6298" CreationDate="2018-05-07T16:53:52.863" Score="2" Body="&lt;p&gt;The paper refers to layers and sub-layers, and clearly indicates that one layer includes all three sub-layers, so when they say they train all three layers simultaneously, they are talking about the three autoencoder layers, not the sub-layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also agrees with the fact that only the autoencoder layer has tunable parameters.  The other two layers use uniform weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have a good answer for the follow-on question yet... would love to hear what others think!&lt;/p&gt;&#xA;" OwnerUserId="15487" LastEditorUserId="15487" LastEditDate="2018-07-10T17:53:05.240" LastActivityDate="2018-07-10T17:53:05.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6300" PostTypeId="1" AcceptedAnswerId="6326" CreationDate="2018-05-07T18:09:31.727" Score="5" ViewCount="556" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.cec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Evolution of Neural Network Topologies&lt;/a&gt; (2002), the authors say &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Genes that do not match are inherited from the more fit parent&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What if the more fit parent has fewer nodes compared to the other, will the disjoint/excess genes be discarded?&lt;/p&gt;&#xA;" OwnerUserId="15490" LastEditorUserId="2444" LastEditDate="2019-11-06T22:55:11.450" LastActivityDate="2020-01-08T12:02:08.107" Title="What if the more fit parent has fewer nodes compared to the other, will the disjoint and excess genes be discarded?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;neat&gt;&lt;neuroevolution&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6301" PostTypeId="2" ParentId="5308" CreationDate="2018-05-07T18:11:30.353" Score="4" Body="&lt;p&gt;Algorithms can learn to lie:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SEE:&#xA;&lt;br&gt;&lt;a href=&quot;https://www.technologyreview.com/s/407459/robots-evolve-to-deceive/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robots Evolve to Deceive (MIT Tech Review, 2007)&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://www.technologyreview.com/s/414934/robots-evolve-the-ability-to-deceive/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robots 'Evolve' the Ability to Deceive (MIT Tech Review, 2009)&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://www.popsci.com/scitech/article/2009-08/evolving-robots-learn-lie-hide-resources-each-other&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Robots Learn To Lie To Each Other (Popsci, 2009)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deception as a strategy has been observed in animal populations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.rochester.edu/news/show.php?id=1421&quot; rel=&quot;nofollow noreferrer&quot;&gt;Do Animals &quot;Lie&quot;? Yes, Even to Their Own Kind, Biologist Says (Rochester University, 1995&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://news.nationalgeographic.com/2017/05/animals-lying-liars-birds-squid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here are the Best Liars in the Animal Kingdom (Nat Geo, 2017)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F978-1-4899-3490-1_10&quot; rel=&quot;nofollow noreferrer&quot;&gt;Can Animals Lie? (Springer)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/17109314&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why animals lie: how dishonesty and belief can coexist in a signaling system.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195327939.001.0001/acprof-9780195327939-chapter-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Philosophy of Deception (Martin, 2009)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-05-07T18:11:30.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6302" PostTypeId="2" ParentId="6291" CreationDate="2018-05-07T18:20:50.980" Score="1" Body="&lt;p&gt;You may be interested in the utility functions of deception:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the abstract of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/17109314&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why Animals Lie: How Dishonesty and Belief can Coexist in a Signaling System. (NIH, 2006)&lt;/a&gt;&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We develop and apply a simple model for animal communication in which signalers can use a nontrivial frequency of deception without causing listeners to completely lose belief. This common feature of animal communication has been difficult to explain as a stable adaptive outcome of the options and payoffs intrinsic to signaling interactions. Our theory is based on two realistic assumptions. (1) Signals are &quot;overheard&quot; by several listeners or listener types with different payoffs. The signaler may then benefit from using incomplete honesty to elicit different responses from different listener types, such as attracting potential mates while simultaneously deterring competitors. (2) Signaler and listener strategies change dynamically in response to current payoffs for different behaviors. The dynamic equations can be interpreted as describing learning and behavior change by individuals or evolution across generations. We explain how our dynamic model differs from other solution concepts from classical and evolutionary game theory and how it relates to general models for frequency-dependent phenotype dynamics. We illustrate the theory with several applications where deceptive signaling occurs readily in our framework, including bluffing competitors for potential mates or territories. We suggest future theoretical directions to make the models more general and propose some possible experimental tests.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A degree of deceptive capability seems to be beneficial from the standpoint of evolution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We humans are not always known for veracity, so the ability understand deception might be a critical component in Artificial General Intelligence's ability to interact with humans.  (Specifically, you can't always believe what humans tell you.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on recent human history, the recognition of the unreliability of humans (versus data and as-objective-as-possible analysis) may become critical to the survival of our own species.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More importantly, it will be essential for strong AI to understand that the &quot;data can lie&quot;&lt;/strong&gt; (faulty parameters, inaccurate data, unawareness of incomplete information.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;JT's answer is a great functional overview on why it's not possible with current methods.  &lt;strong&gt;This answer might be regarded in the sense that, aside from very limited special cases such as solved games where true objectivity can be achieved, reality is subjective and &quot;truth&quot; is a subjective function of the parameters and data.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, understanding that last bit is likely much more important than trying to code AI's not to &quot;lie&quot;. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-08T18:21:21.590" LastActivityDate="2018-05-08T18:21:21.590" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6303" PostTypeId="2" ParentId="6292" CreationDate="2018-05-07T19:12:39.283" Score="2" Body="&lt;p&gt;It is possible to use spectrograms for genres classification. See relevant articles &lt;a href=&quot;https://ieeexplore.ieee.org/document/5977391/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Music genre recognition using spectrograms&lt;/a&gt; or blog post &lt;a href=&quot;http://deepsound.io/music_genre_recognition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The later link did use CNN for this task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need training data, &lt;a href=&quot;http://marsyasweb.appspot.com/download/data_sets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTZAN&lt;/a&gt; or &lt;a href=&quot;https://github.com/mdeff/fma&quot; rel=&quot;nofollow noreferrer&quot;&gt;FMA&lt;/a&gt; could be your starting point.&lt;/p&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-05-07T19:12:39.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6306" PostTypeId="2" ParentId="6297" CreationDate="2018-05-08T08:35:53.817" Score="0" Body="&lt;p&gt;It depends what you mean by &quot;comparison&quot;, but in general I would think not really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks operate on the sub-symbolic level, ie instead of handling discrete symbols (such as letters) they work with numerical values. These values can often be mapped onto symbols (eg through input or output nodes) which typically are letters or words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to compare texts, you are dealing with symbols, so it would probably be easier to operate on the symbolic level, by manipulating words directly, rather than translating them into numerical values and back, as that usually involves some loss of precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as I said, it is hard to answer your question without knowing more detail about the exact nature of the comparison you're after.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-08T08:35:53.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6307" PostTypeId="2" ParentId="6267" CreationDate="2018-05-08T08:40:43.573" Score="4" Body="&lt;p&gt;Most of the answers are oriented towards statistical/probabilistic models. For more 'classic' AI I would say you would need some knowledge of &lt;strong&gt;predicate calculus&lt;/strong&gt;. This is the more symbolic planning approach to AI problem solving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could argue it's a bit 'old school', but still relevant for certain aspects of AI.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-08T08:40:43.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6308" PostTypeId="1" CreationDate="2018-05-08T14:26:30.693" Score="1" ViewCount="469" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1003.0146.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Li et al. (2010)'s highly cited paper&lt;/a&gt;, they talk about LinUCB with hybrid linear models in Section 3.2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They motivate this by saying, &quot;In many applications including ours, it is helpful to use features that are shared by all arms, in addition to the arm-specific ones. For example, in news article recommendation, a user may prefer only articles about politics for which this provides a mechanism.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't quite understand what they mean by this. Is anyone willing to provide a different example? Also it would greatly help if you can clarify what Equation 6's &quot;$\mathbf{z}$&quot; and &quot;$\mathbf{x}$&quot; refer to in the context they talk about (news recommendation), or the example you give?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation (6) from the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$$ \mathbf{E} \left[ r_{t,a} \vert \mathbf{x}_{t, a} \right] = \mathbf{z}_{t, a}^{\top} \boldsymbol{\beta}^* + \mathbf{x}_{t, a}^{\top} \boldsymbol{\theta}_a^* $$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12656" LastEditorUserId="1641" LastEditDate="2018-08-13T17:57:50.747" LastActivityDate="2018-08-13T18:03:18.933" Title="LinUCB with Hybrid Linear Models" Tags="&lt;algorithm&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6313" PostTypeId="1" CreationDate="2018-05-08T19:33:15.740" Score="4" ViewCount="742" Body="&lt;p&gt;This just popped into my head, and I haven't thought it through, but it feels like a sound question.  The definition of intelligence might still be somewhat fuzzy, possibly a factor of our evolving understanding of &quot;intelligence&quot; in regard to algorithms, but rationality has some precise definitions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are Rationality and Intelligence distinct? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If not, explain.  If so, elaborate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(I have some thoughts on the subject and would be very interested in the thoughts of others.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-09-17T14:50:53.790" LastActivityDate="2020-09-17T14:50:53.790" Title="Are Rationality and Intelligence distinct?" Tags="&lt;philosophy&gt;&lt;terminology&gt;&lt;semantics&gt;&lt;rationality&gt;&lt;intelligence&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6314" PostTypeId="1" AcceptedAnswerId="6744" CreationDate="2018-05-08T22:23:53.637" Score="3" ViewCount="1238" Body="&lt;p&gt;I'm working on a Reinforcement Learning task where I use reward shaping as proposed in the paper &lt;a href=&quot;https://people.eecs.berkeley.edu/~russell/papers/icml99-shaping.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy invariance under reward transformations:&#xA;Theory and application to reward shaping&lt;/a&gt; (1999) by Andrew Y. Ng, Daishi Harada and Stuart Russell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, my reward function has this form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R(s, s') = \gamma P(s') - P(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is a &lt;strong&gt;potential function&lt;/strong&gt;.  When &lt;span class=&quot;math-container&quot;&gt;$s = s'$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$R(s, s) = (\gamma - 1)P(s)$&lt;/span&gt;, which is non-positive, since &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \gamma &amp;lt;= 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But considering &lt;span class=&quot;math-container&quot;&gt;$P(s)$&lt;/span&gt; relatively high (let's say &lt;span class=&quot;math-container&quot;&gt;$P(s) = 1000$&lt;/span&gt;), &lt;span class=&quot;math-container&quot;&gt;$R(s, s)$&lt;/span&gt; become too high as well (e.g. with &lt;span class=&quot;math-container&quot;&gt;$\gamma=0.99$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R(s,s)=-10$&lt;/span&gt;), and if for many steps the agent stays in the same state, then the cumulative reward becomes more and more negative, which might affect the learning process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, I solved the problem by just removing the factor &lt;span class=&quot;math-container&quot;&gt;$P(s)$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$s = s'$&lt;/span&gt;. But I have some doubts about the theoretical correctness of this &quot;implementation trick&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea could be to scale appropriately &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; in order to give a reasonable reward. Indeed, with &lt;span class=&quot;math-container&quot;&gt;$\gamma=1.0$&lt;/span&gt;, there is no problem, and, with gamma very near to &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt;, the negative reward is tolerable. Personally, I don't like it because it means that &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is somehow dependent on the reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think?&lt;/p&gt;&#xA;" OwnerUserId="15517" LastEditorUserId="2444" LastEditDate="2020-11-09T17:35:31.810" LastActivityDate="2020-11-09T17:35:31.810" Title="What should I do when the potential value of a state is too high?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;reward-design&gt;&lt;reward-shaping&gt;&lt;potential-reward-shaping&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6315" PostTypeId="2" ParentId="6313" CreationDate="2018-05-09T05:27:02.180" Score="1" Body="&lt;p&gt;I recall someone (my prof probably) saying that the difference is that intelligence is a problem-solving capability, while rationality more-so refers the capability to apply one's intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ex: You are smart for knowing that sleeping late is bad for your health, but if you still sleep late then you are irrational.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that sense then, rationality is like a meta-problem-solving skill perhaps?&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-05-09T05:27:02.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6316" PostTypeId="2" ParentId="6313" CreationDate="2018-05-09T08:19:11.473" Score="2" Body="&lt;p&gt;From Norvig and Russel definitions of rationality:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Thinking Rationally - The Greek philosopher Aristotle was one of the first to attempt to codify “right thinking,” that&#xA;  is, irrefutable reasoning processes. His syllogisms provided patterns for argument structures&#xA;  that always yielded correct conclusions when given correct premises—for example, “Socrates&#xA;  is a man; all men are mortal; therefore, Socrates is mortal.” These laws of thought were&#xA;  supposed to govern the operation of the mind; their study initiated the field called logic.&lt;/li&gt;&#xA;  &lt;li&gt;Acting Rationally - An agent is just something that acts. Of course,&#xA;  all computer programs do something, but computer agents are expected to do more: operate&#xA;  autonomously, perceive their environment, persist over a prolonged time period, adapt to&#xA;  change, and create and pursue goals. &#xA;  A rational agent is one that acts so as to achieve the&#xA;  best outcome or, when there is uncertainty, the best expected outcome.&#xA;  In the “laws of thought” approach to AI, the emphasis was on correct inferences. Making&#xA;  correct inferences is sometimes part of being a rational agent, because one way to act&#xA;  rationally is to reason logically to the conclusion that a given action will achieve one’s goals&#xA;  and then to act on that conclusion. On the other hand, correct inference is not all of rationality;&#xA;  in some situations, there is no provably correct thing to do, but something must still be&#xA;  done. There are also ways of acting rationally that cannot be said to involve inference. For&#xA;  example, recoiling from a hot stove is a reflex action that is usually more successful than a&#xA;  slower action taken after careful deliberation.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Clearly and also intuitively, rationality is well defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Intelligence as seen form mathematical and computational approach:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence can be the ability for an agent to make rational or irrational decisions, on a varying time frame and also choose the level of rationality (strictly in a computational sense). For example, I have exams and I want to watch TV, on a time frame of a week/month the rational decision would be to study so that I can enjoy the fruits of my labor which will be much more than the instantaneous pleasure of TV (also I can watch reruns). But for a time frame of an hour watching TV is definitely the most rewarding thing. So intelligence can be defined as the capability in deciding the length of time frame to be rational (what we call visionaries those who can see rewards far in the future). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also as Game Theory or economics suggest, we can have different definitions for rationality depending on our needs. Thus, watching TV to gain knowledge might be more important to someone than studying, so effectively he has a different rationality function (arbitrary made-up term) to satisfy. Thus, intelligence can be deciding our rationality function, based on our needs and external experiences (learning in a nutshell). Also we may decide to minimize our rationality cost function or leave it in an intermediate state (which can be thought of as the minima for a different rationality cost function, thus only rationality functions are the true variable and not the intelligent decision to minimize it or not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets take an example of bees (I am not sure whether this is the correct interpretation though): Bees can hardly be called intelligent (no foresight), but they are rational. They perform the task assigned to them with efficiency and toil (even though this does not reward the bee itself, it rewards the genes carried by the bees and ensures it survival through the queen - can be thought of as evolutionary coded intelligence). Bees perform these jobs in an apparently rational way, which has been decided by 1000's of years of evolution. Though bees individually are of hardly any importance, together they create a truly intelligent community - taking smart decisions and actions albeit their farsightedness is only within a  smaller time frame compared to humans. Thus in common terms, it can be thought that rationality always almost leads to intelligence, but the same cannot be said for vice versa (as for intelligence you now have a choice for rationality function and you can choose not to satisfy it or choose an irrational function). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important consequence of bees not being intelligent is that they are always performing rational actions as hard coded in their genes, which is causing the entire colony to behave in an intelligent and a very optimized energy efficient way (but there maybe better strategies, we can never know whether they follow the best strategy unless we account for all the variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR :&lt;/strong&gt; Intelligence can be thought of the ability of an agent to choose the amount of rationality the agent wishes to satisfy. Using mathematics we can always find one or more completely rational methods of solving a problem with variables being time and the environment. But intelligent beings can add more variables like their experience, needs and motivation. Intelligent beings to some extent can single-handedly manipulate the external environment to suit their needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;From psychological viewpoint:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tutorialspoint.com/artificial_intelligence/artificial_intelligent_systems.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; are a few definitions of different types of intelligence and learning - Quite good and concise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IQ - In science, the term intelligence typically refers to what we could call academic or cognitive intelligence. In their book on intelligence, professors Resing and Drenth (2007)* answer the question 'What is intelligence?' using the following definition: &quot;The whole of cognitive or intellectual abilities required to obtain knowledge, and to use that knowledge in a good way to solve problems that have a well described goal and structure.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligence - Wikipedia&lt;/a&gt; - Actually has some good definitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence_quotient&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligence Quotient - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotional Intelligence - Emotional intelligence is the ability to identify and manage your own emotions and the emotions of others. It is generally said to include three skills: emotional awareness; the ability to harness emotions and apply them to tasks like thinking and problem solving; and the ability to manage emotions, which includes regulating your own emotions and cheering up or calming down other people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://psychcentral.com/lib/what-is-emotional-intelligence-eq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emotional Intelligence - PsychCentral&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Emotional_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emotional Intelligence - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is of some insight!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-10T08:55:42.587" LastActivityDate="2018-05-10T08:55:42.587" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6317" PostTypeId="1" AcceptedAnswerId="6321" CreationDate="2018-05-09T08:41:54.433" Score="6" ViewCount="1329" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://proceedings.mlr.press/v32/silver14.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Deterministic Policy Gradient Algorithms&lt;/a&gt;&lt;/em&gt;, I am really confused about chapter 4.1 and 4.2 which is &quot;On and off-policy Deterministic Actor-Critic&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what's the difference between two algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only noticed that the equation 11 and 16 are different, and the difference is the action part of Q function where is $a_{t+1}$ in equation 11 and $\mu(s_{t+1})$ in equation 16. If that's what really matters, how can I calculate $a_{t+1}$ in equation 11?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://s26.postimg.cc/y59xkd9q1/image.png&quot; alt=&quot;&quot;&gt;&#xA;&lt;img src=&quot;https://s26.postimg.cc/7wysv3zy1/image.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15525" LastEditorUserId="2444" LastEditDate="2019-02-15T15:35:01.627" LastActivityDate="2019-04-24T09:29:01.420" Title="What is the difference between on and off-policy deterministic actor-critic?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;actor-critic-methods&gt;&lt;on-policy-methods&gt;&lt;off-policy-methods&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6318" PostTypeId="1" AcceptedAnswerId="6319" CreationDate="2018-05-09T09:28:56.357" Score="2" ViewCount="166" Body="&lt;p&gt;I'm working with a data set where the data is stored in a string such as &lt;code&gt;AxByCyA&lt;/code&gt; where &lt;code&gt;A&lt;/code&gt;, &lt;code&gt;B&lt;/code&gt; and &lt;code&gt;C&lt;/code&gt; are actions and &lt;code&gt;v,w,x,y,z&lt;/code&gt; are times between the actions (each letter represents an interval of time). It's worth noting that &lt;code&gt;B&lt;/code&gt; cannot occur without &lt;code&gt;A&lt;/code&gt;, and &lt;code&gt;C&lt;/code&gt; cannot occur without &lt;code&gt;B&lt;/code&gt;, and &lt;code&gt;C&lt;/code&gt; is the action I'm attempting to study (ie: I'd like to be able to predict whether a user will do &lt;code&gt;C&lt;/code&gt; based on their prior actions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I intend to create 2 clusters: people who do &lt;code&gt;C&lt;/code&gt; and those who don't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this data set I build a training array to run the sci-kit (python) k-means algorithm on, containing the number of &lt;code&gt;A&lt;/code&gt;s, the number of &lt;code&gt;B&lt;/code&gt;s, the mean time between actions (calculated using the average of each interval) and the standard deviation between each interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives me an overall success rate of 82% on the test set, but is there anything I can do for more accuracy?&lt;/p&gt;&#xA;" OwnerUserId="12940" LastEditorUserId="12940" LastEditDate="2018-05-09T09:36:01.950" LastActivityDate="2018-05-09T10:09:30.840" Title="How to refine K-means clustering on a data set?" Tags="&lt;classification&gt;&lt;training&gt;&lt;python&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6319" PostTypeId="2" ParentId="6318" CreationDate="2018-05-09T09:54:58.653" Score="5" Body="&lt;p&gt;The usual parameters to adjust in a k-means:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Number of clusters (recall many clusters can have same label). &lt;/li&gt;&#xA;&lt;li&gt;Distance definition (euclidean is the most basic, Gauss is an&lt;br&gt;&#xA;improvement)   &lt;/li&gt;&#xA;&lt;li&gt;Selection of initial cluster positions.&lt;/li&gt;&#xA;&lt;li&gt;Data preprocessing (data normalization, ...)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-05-09T10:09:30.840" LastActivityDate="2018-05-09T10:09:30.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6321" PostTypeId="2" ParentId="6317" CreationDate="2018-05-09T13:22:29.027" Score="4" Body="&lt;p&gt;The twist here is that the $a_{t+1}$ in (11) and the $\mu(s_{t+1})$ in (16) are the &lt;em&gt;same&lt;/em&gt; and actually the $a_t$ in the on-policy case and the $a_t$ in the off-policy case are different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key to the understanding is that in on-policy algorithms you have to use actions (and generally speaking trajectories) generated by the policy in the updating steps (to improve the policy itself). This means that in the on-policy case $a_i = \mu(s_{i})$ (in equations 11-13).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas in the off-policy case you can use &lt;em&gt;any&lt;/em&gt; trajectory to improve your value/action-value functions, which means that the actions $a_t$ can be generated by any distribution $a_t~\pi(s_t,a_t)$. In (16) the algorithm explicitly states however that the action-value function ($Q^w$) has to be evaluated at $\mu_{s_{t+1}}$ (just like in the on-policy case) and not at $a_t$ which was the actual action in the trajectory generated by policy $\pi$.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastActivityDate="2018-05-09T13:22:29.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6322" PostTypeId="1" CreationDate="2018-05-09T13:58:33.037" Score="4" ViewCount="3008" Body="&lt;p&gt;I am a university student taking an Artificial Intelligence class this semester. Our professor's programming language of choice is Java, but it seems that perhaps with some nudging, he can change it to Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to know if there is any merit in doing so. I know a programming language is just a programming language - however, given the industry's wide use of Python when implementing AI algorithms (especially with ML), I think it makes much more sense for us to use Python. It will be easier to transition from a University environment to an Industrial one having done several assignments in Python and having a clear understanding of all the tools it provides than if we continue using Java, correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the advantages and disadvantages of a switch from Java to Python?&lt;/p&gt;&#xA;" OwnerUserId="15529" LastEditorUserId="1671" LastEditDate="2019-11-19T21:50:14.710" LastActivityDate="2019-11-19T21:50:14.710" Title="Is a switch from Java to Python worth it?" Tags="&lt;python&gt;&lt;comparison&gt;&lt;software-evaluation&gt;&lt;programming-languages&gt;&lt;java&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ClosedDate="2020-03-04T01:47:49.133" ContentLicense="CC BY-SA 4.0" />
  <row Id="6323" PostTypeId="2" ParentId="3428" CreationDate="2018-05-09T14:21:49.913" Score="5" Body="&lt;p&gt;As @Thomas W said, you can be pretty immaginative when you're developing mutation and crossover methods. Each problem has its own caracteristics and, therefore, requires a different strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BUT, from my experience, I'd say that 90% of crossovers and mutation on real numbers genotypes are solved using the BLX-α algorithm.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Crossover:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RDKUw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RDKUw.png&quot; alt=&quot;BlX-α algorithm range&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm is really simple. Given the parents &lt;em&gt;X&lt;/em&gt; and &lt;em&gt;Y&lt;/em&gt; and an α value (inside the range [0,1], generally around 0.1/0.15, but it depends by the problem), For each gene of your genotype: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;extract the genes &lt;em&gt;xi&lt;/em&gt; and &lt;em&gt;yi&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;find the minimum and the maximum values&lt;/li&gt;&#xA;&lt;li&gt;the new gene will be a random number in the interval [min - range * α, max + range * α]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A variation of this algorithm is the BLX-αβ, in which we take into account which parent performed better and use two constants (α &gt; β) to increase the probabilities that the new value will be closer to the one of the fittest parent&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Wy65S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wy65S.png&quot; alt=&quot;BLX-αβ algorithm range&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mutation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the mutation the situation is similar: we need to get a random value that is related to our problem domain (we do not want the mutations to be destructive! They have the function of &lt;em&gt;exploring&lt;/em&gt; the space).&lt;br&gt;&#xA;In these cases it is useful to determine a range for the mutation and use that range to find the new value of the gene using BLX-α.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more sophisticated mutation algorithm can be achieved using BLX-α on boundaries that depend on the actual value of the gene and the fitness function of the individual.&lt;br&gt;&#xA;Let's imagine that our individual performs in a very bad way; in that case the mutation operator will be used to &lt;em&gt;shift&lt;/em&gt; the individual to a distant point in the search space, where it will probably perform better.&lt;br&gt;&#xA;On the other hand, if the individual is already fit we may not want to introduce some dramatic changes using the mutation. In that case the mutation range would be more contained and would have the function of &lt;em&gt;tuning&lt;/em&gt; the genotype instead of &lt;em&gt;exploring&lt;/em&gt; for better alternatives.     &lt;/p&gt;&#xA;" OwnerUserId="15530" LastEditorUserId="15530" LastEditDate="2018-05-13T14:37:15.177" LastActivityDate="2018-05-13T14:37:15.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6324" PostTypeId="2" ParentId="6322" CreationDate="2018-05-09T14:45:59.173" Score="2" Body="&lt;p&gt;I don't think there is much merit in using one language over the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's true that you will get a better feeling for what libraries etc. are available in Python when using it, but I think it's more important to focus on the mathematics and application of the algorithms and techniques you will learn, as these will be relevant regardless of the language / library you end up using.&lt;/p&gt;&#xA;" OwnerUserId="15533" LastActivityDate="2018-05-09T14:45:59.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6325" PostTypeId="1" CreationDate="2018-05-09T15:35:25.447" Score="10" ViewCount="350" Body="&lt;p&gt;Imagine trying to create a simulated virtual environment that is complicated enough to create a &quot;general AI&quot; (which I define as a self aware AI) but is as simple as possible. What would this minimal environment be like?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. An environment that was just a chess game would be too simple. A chess program cannot be a general AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An environment with multiple agents playing chess and communicating their results to each other. Would this constitute a general AI? (If you can say a chess grand master who thinks about chess all day long has 'general AI'? During his time thinking about chess is he any different to a chess computer?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about a 3D sim-like world. That seems to be too complicated. After all why can't a general AI exist in a 2D world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be an example of a simple environment but not too simple such that the AI(s) can have self-awareness?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="1671" LastEditDate="2018-05-11T13:52:39.720" LastActivityDate="2018-05-29T00:53:30.087" Title="What kind of simulated environment is complex enough to develop a general AI?" Tags="&lt;agi&gt;&lt;artificial-consciousness&gt;&lt;self-awareness&gt;" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6326" PostTypeId="2" ParentId="6300" CreationDate="2018-05-09T16:12:19.357" Score="6" Body="&lt;p&gt;When crossover happens and one parent is fitter than the other, the nodes from the more fit parent are carried over to the child. This is the case as disjoint and excess genes are only carried over from the fittest parent. &#xA;Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Node Crossover&#xA;Parent 1 Nodes: {[0][1][2]} // more fit parent&#xA;Parent 2 Nodes: {[0][1][2][3]}&#xA;&#xA;Child Nodes:    {[0][1][2]} // after crossover&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gene information is also passed to the child during crossover. Matching genes (those that have the same innovation number) are chosen at random and passed to the child. The disjoint and excess genes are choose from the more fit parent.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Gene Crossover&#xA;Parent 1 Genes: [1][2][3]      [6]   [8][9][10] // more fit parent&#xA;Parent 2 Genes: [1][2][3][4][5]   [7]&#xA;&#xA;Child Genes: [1][2][3][6][8][9][10] // after crossover&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the gene innovation numbers in the child match up with the innovation numbers of the fittest parent. However the gene information from matching genes (in the example genes 1, 2 and 3 match) have an equal chance of being carried over to the child. In the example, the child's first three genes could have come from either parent.&lt;/p&gt;&#xA;" OwnerUserId="15356" LastEditorUserId="15356" LastEditDate="2020-01-08T12:02:08.107" LastActivityDate="2020-01-08T12:02:08.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6328" PostTypeId="2" ParentId="6322" CreationDate="2018-05-09T22:28:01.127" Score="4" Body="&lt;p&gt;While dyedgreen is right in some respects, I don't agree entirely with that sentiment. Sure, you can theoretically use any language as long as you know the maths and understand the concepts inside and out whilst having some applicable knowledge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't believe if you are starting from scratch, you should learn to develop models in Java. While the underlying material is the same, all you are doing is wasting time IMO learning a language that is rarely used in the field and lacks sufficient support by most modern ML frameworks. Also, while this is rather subjective, python feels better when used as a data science language. Working with data especially is where that difference is clear to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, this could all change, and Java could become the ML language of choice(extremely unlikely).&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2018-05-10T00:40:09.663" LastActivityDate="2018-05-10T00:40:09.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6329" PostTypeId="1" CreationDate="2018-05-10T00:34:13.240" Score="1" ViewCount="353" Body="&lt;p&gt;I want my neural network structure to not have a circular/looping structure something similar like a directed acyclic graph (DAG). How do I do that?&lt;/p&gt;&#xA;" OwnerUserId="15490" LastEditorUserId="2444" LastEditDate="2019-02-17T21:11:29.650" LastActivityDate="2020-08-13T13:55:49.107" Title="How do I restrict the neural network structure to be acyclic in NEAT?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6330" PostTypeId="1" AcceptedAnswerId="6340" CreationDate="2018-05-10T01:01:42.800" Score="0" ViewCount="84" Body="&lt;p&gt;From &lt;a href=&quot;https://stackoverflow.com/questions/36370129/does-tensorflow-use-automatic-or-symbolic-gradients&quot;&gt;https://stackoverflow.com/questions/36370129/does-tensorflow-use-automatic-or-symbolic-gradients&lt;/a&gt;, I understood TensorFlow requires all the operations in the Graph to be explicit formulas (instead of black-boxes, such as raw python functions) to do Automatic Differentiation. Then it will do some kind of Gradient Descent based on that to minimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering, since it already know all the explicit formulas, can it directly find out the minimum by examining the equation itself? Like computing the points where gradient is zero or do not exist, then do some kind of processing to find out the minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found it is simple to do this &quot;symbolic minimization&quot; above with few variables such as minimizing &lt;code&gt;Σ(a_i - v)^2&lt;/code&gt; where &lt;code&gt;v&lt;/code&gt; is the trainable variable an &lt;code&gt;a_i&lt;/code&gt; are all the training samples. I'm not sure is there a general way though.&lt;/p&gt;&#xA;" OwnerUserId="15546" LastActivityDate="2018-05-10T09:31:36.480" Title="Can TensorFlow minimize &quot;symbolically&quot;" Tags="&lt;tensorflow&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6331" PostTypeId="2" ParentId="6322" CreationDate="2018-05-10T02:24:31.313" Score="2" Body="&lt;p&gt;I'm going to go ahead and disagree with the others.  From an academic perspective for AI or any CS related assignment Java (or C or C++) will always have much more benefit as you will get to write the actual code instead of using libraries others have already written.  That way later on when you transition to Python or whatever language you choose you'll know whats under the hood and use it and optimise it more effectively than someone who only knows which libraries to use for what application.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2018-05-10T02:24:31.313" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6333" PostTypeId="1" CreationDate="2018-05-10T03:46:03.787" Score="9" ViewCount="533" Body="&lt;p&gt;At Google I/O the Google Duplex just came out and now releasing new public self-driving car, Waymo. I can't think of any Tech Giants that can actually compete with Google AI other than Amazon. I'm thinking other companies will team up such as Nvidia, Intel, AMD, and etc. Apple seems to be lacking.&lt;/p&gt;&#xA;" OwnerUserId="7720" LastEditorUserId="7720" LastEditDate="2018-05-10T03:52:26.310" LastActivityDate="2019-08-21T22:29:12.900" Title="Who stands a chance against Google in the AI race?" Tags="&lt;ai-community&gt;" AnswerCount="9" CommentCount="0" FavoriteCount="4" ClosedDate="2020-05-23T13:51:04.410" ContentLicense="CC BY-SA 4.0" />
  <row Id="6334" PostTypeId="2" ParentId="6333" CreationDate="2018-05-10T03:53:36.207" Score="3" Body="&lt;p&gt;I will assume that you mean the Strong AI or AGI race? Interesting question, given Google's investment in the technology it seems hard to imagine them giving up the throne at this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there definitely are some competitors. Some that come to mind are Amazon, OpenAI, Baidu. These companies all have significant resources and have teams dedicated to solving the Strong AI problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;DarkDarkhorse&lt;/strong&gt; in the race would be a team like OpenCog/SingularityNet. They are trying to build strong AI powered by the blockchain(Ben Goertzel). With enough computation and ingenuity though, an individual or small team could do it themselves.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2018-05-10T03:53:36.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6335" PostTypeId="2" ParentId="6333" CreationDate="2018-05-10T08:29:11.273" Score="3" Body="&lt;p&gt;&#xA;In my own opinion the biggest threats would  be Nvidia because of their tight grip on the GPU market; it will be relatively easier for them to catch up with innovative ideas.&#xA;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Baidu is on the horizon though their research at this point seems to be domain specific, NLP to be precise.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-05-10T08:29:11.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6336" PostTypeId="5" CreationDate="2018-05-10T08:31:25.783" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-05-10T08:31:25.783" LastActivityDate="2018-05-10T08:31:25.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6337" PostTypeId="4" CreationDate="2018-05-10T08:31:25.783" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-05-10T08:31:25.783" LastActivityDate="2018-05-10T08:31:25.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6338" PostTypeId="1" CreationDate="2018-05-10T09:03:44.073" Score="0" ViewCount="120" Body="&lt;p&gt;What is Bayes' theorem? How does it relate to conditional probabilities?&lt;/p&gt;&#xA;" OwnerUserId="15551" LastEditorUserId="2444" LastEditDate="2020-08-11T20:18:30.180" LastActivityDate="2020-08-11T20:18:30.180" Title="What is Bayes' theorem?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;definitions&gt;&lt;statistics&gt;&lt;bayes-theorem&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6339" PostTypeId="2" ParentId="6314" CreationDate="2018-05-10T09:26:59.633" Score="3" Body="&lt;p&gt;I don't think the situation you're sketching should be a problem at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$P(s)$&lt;/span&gt; is high (e.g. &lt;span class=&quot;math-container&quot;&gt;$P(s) = 1000$&lt;/span&gt;), this means (according to your shaping / &quot;heuristic&quot;) that it's valuable to be in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, that you expect to be able to get high future returns from that state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you then continuously take actions that keep you in the same state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, it is essentially correct to punish these actions (with negative rewards); you were expecting to get high future returns starting from that state, but you just remain stuck in that state. This means that the actions you're taking are not providing the rewards you were expecting, so it should be &quot;punished&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the exception to the above paragraph is the case where the transition from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (staying in the same state) does result in a positive &quot;true reward&quot; (actual/real reward, not shaping-reward). In that case, the true reward will offset the reduction in potential and result in a neutral or positive combined reward if it is sufficiently large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; parameter, a &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1.0$&lt;/span&gt; (such as &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.99$&lt;/span&gt;) intuitively means that you prioritize short-term rewards over long-term rewards (if the magnitudes of the undiscounted rewards are similar). This implies that what you wrote in the question is precisely what should happen; if you prioritize short-term rewards over long-term rewards, it's bad to &quot;waste time&quot; by staying in the same state, so actions that don't move you anywhere should be disincentivized through negative rewards. If &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1.0$&lt;/span&gt;, you have no preference for short-term rewards over long-term rewards, so you'll be fine with &quot;wasting time&quot; by staying in the same state, and therefore no longer have to punish such actions.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2019-09-15T16:24:04.043" LastActivityDate="2019-09-15T16:24:04.043" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6340" PostTypeId="2" ParentId="6330" CreationDate="2018-05-10T09:31:36.480" Score="0" Body="&lt;p&gt;If by &quot;symbolic&quot; you mean finding an analytical solution, that is, an equation for each weight, then the answer is no. The example you chose results in a system linear equations, which can be solved analytically. However once you introduce non linearities (by using activation functions with more than one layer), most non trivial cases will have no analytical solution and will need to be solved numerically. This is not a problem specific to tensorflow, it is  a mathematical issue, it will not be possible on any language, current or future. Unless there is some revolution in math first.&lt;/p&gt;&#xA;" OwnerUserId="5911" LastActivityDate="2018-05-10T09:31:36.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6341" PostTypeId="2" ParentId="6322" CreationDate="2018-05-10T09:41:05.900" Score="0" Body="&lt;p&gt;To add a few cents, I do not think it will make any difference. The reason is that the content of the course will be, by design (if you can teach it using java), very general, general enough so it will not matter what language you use (this has to be so otherwise you would not be able to use Java). Once you need to do advanced stuff and want to automatize the basic algorithms, you can switch to python, where you can use super advanced API's like Keras, which is on top of tensorflow which is on top of python. To give you a particular example,  with full time dedication you should be able to  switch from matlab to python and implement your first keras model, by self learning, in not more than a couple of months.  &lt;/p&gt;&#xA;" OwnerUserId="5911" LastActivityDate="2018-05-10T09:41:05.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6342" PostTypeId="1" CreationDate="2018-05-10T17:01:51.303" Score="1" ViewCount="16" Body="&lt;p&gt;Coming from the YT videos of 3blue1brown which showed that the individual layers do not have discernible shapes in the case of hand written letter recognition, I wondered if you could penalize dispersed shapes while training, thus creating connected shapes (at least on the first layer in the beginning). That way, you may be better able to understand the propagation of your algorithm through the layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, Jonny&lt;/p&gt;&#xA;" OwnerUserId="15563" LastActivityDate="2018-05-10T17:01:51.303" Title="Can you make the first layer of a net have discernible shapes?" Tags="&lt;reinforcement-learning&gt;&lt;handwritten-characters&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6343" PostTypeId="1" CreationDate="2018-05-10T18:43:12.950" Score="2" ViewCount="4502" Body="&lt;p&gt;I'm currently using 3Blue1Brown's tutorial series on neural networks and lack extensive calculus knowledge/experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the following equations to calculate the gradients for weights and biases as well as the equations to find the derivative of the cost with respect to a hidden layer neuron:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YOMCJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YOMCJ.jpg&quot; alt=&quot;EQUATIONS&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is, during backpropagation, the gradients keep cancelling each other out because I take an average for opposing training examples. That is, if I have two training labels being [1, 0], [0, 1], the gradients that adjust for the first label get reversed by the second label because an average for the gradients is taken. The network simply keeps outputting the average of these two and causes the network to always output [0.5, 0.5], regardless of the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prevent this, I figured a softmax function would be required for the last layer instead of a sigmoid, which I used for all the layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have no idea how to implement this. The math is difficult to understand and the notation is complicated for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations I provided above show the term: σ'(z), which is the derivative of the sigmoid function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm using softmax, how am I supposed to substitute sigmoid with it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm not mistaken, the softmax function doesn't just take one number analogous to the sigmoid, and uses all the outputs and labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum it up, the things I'd like to know and understand are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The equation for the neuron in every layer besides the output is: σ(w&lt;sub&gt;1&lt;/sub&gt;x&lt;sub&gt;1&lt;/sub&gt; + w&lt;sub&gt;2&lt;/sub&gt;x&lt;sub&gt;2&lt;/sub&gt; + ... + w&lt;sub&gt;n&lt;/sub&gt;x&lt;sub&gt;n&lt;/sub&gt; + b). How am I supposed to make an analogous equation with softmax for the output layer?&lt;/li&gt;&#xA;&lt;li&gt;After using (1) for forward propagation, how am I supposed to replace the σ'(z) term in the equations above with something analogous to softmax to calculate the partial derivative of the cost with respect to the weights, biases, and hidden layers?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9432" LastEditorUserId="9432" LastEditDate="2018-12-15T02:03:01.843" LastActivityDate="2018-12-15T02:03:01.843" Title="How do I implement softmax forward propagation and backpropagation to replace sigmoid in a neural network?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6344" PostTypeId="1" AcceptedAnswerId="6394" CreationDate="2018-05-10T19:23:25.520" Score="4" ViewCount="842" Body="&lt;p&gt;After learning the basics of neural networks and coding one working with the MNIST dataset, I wanted to go to the next step by making one which is able to play a game. I wanted to make it work on a game like slither.io, so in order to be able to create multiple instances of snakes and accelerate the speed of the game, i recreated a simple version of the game:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g2oMV.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g2oMV.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The core features being almost done, now comes the work on the AI. I want to keep the script very simple by using only numpy (not that tensorflow, pytorch or spark does not interest me, but I want to understand things at a &quot;low level&quot; before using those framworks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, I wanted the AI to be able to propose an output by reading pixels. But after some research, I don't realy want to get into convnet, recurent and recursive neural net. I'd like to re-use the simple feed forward NN I did with MNIST and adapt it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So instead of using pixels, I think i'm going to use the following data:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;{x,y} snake's position&lt;/li&gt;&#xA;&lt;li&gt;{x,y} foods positions&lt;/li&gt;&#xA;&lt;li&gt;food value&lt;/li&gt;&#xA;&lt;li&gt;Time, in order to get the snake eat the more food in a short time.&lt;/li&gt;&#xA;&lt;li&gt;Distance from the center, not die outside the area&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g410X.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g410X.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's a lot of different data to handle ! &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1/2 Can a simple FNN handle different kinds of Data in the Input layer ?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DuXAQ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DuXAQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover that comes an other problem:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2/2 Will it properly work with a variable number of inputs?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In fact, in a specific area around the snake, the quantity of food will be variable. I came accross this &lt;a href=&quot;https://ai.stackexchange.com/questions/2008/how-can-neural-networks-deal-with-varying-input-sizes&quot;&gt;post that kinda answer my question&lt;/a&gt;, but wath if I want the Neural network to forget some input if they are not being used, can &lt;strong&gt;Dropout&lt;/strong&gt; be of any use in this case. Or the weights value (correcting toward zero) of these Inputs will be enough? &lt;/p&gt;&#xA;" OwnerUserId="15564" LastEditorUserId="15564" LastEditDate="2018-05-10T19:29:00.577" LastActivityDate="2018-05-13T15:41:26.223" Title="How to handle varying types and length of inputs in a neural network?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6346" PostTypeId="2" ParentId="6333" CreationDate="2018-05-10T20:25:40.420" Score="4" Body="&lt;p&gt;This is pure speculation on my part, but I also think Amazon only, because Bezos is an engineer/founder, cares about marketshare and utility over profits, almost certainly understands the implications of recent validations of strong-narrow AI, and will invest sufficiently to give Google a run for their money.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(This partly has to do with how successful Bezos has been as an entrepreneur, which is partly a factor of his flexibility.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just don't see it happening elsewhere, sans this type of leadership and control.  Google, by contrast, is and always has been an &quot;algorithm company&quot; first and foremost, so they are in the lead almost by default.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:38:54.920" LastActivityDate="2018-05-10T21:38:54.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6347" PostTypeId="5" CreationDate="2018-05-10T20:47:13.933" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt;, early stopping is a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Regularization_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;regularization&lt;/a&gt; used to avoid &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt; when training a learner with an iterative method, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent&lt;/a&gt;. Such methods update the learner so as to make it better fit the training data with each iteration. Up to a point, this improves the learner's performance on data outside of the training set. Past that point, however, improving the learner's fit to the training data comes at the expense of increased &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalization_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalization error&lt;/a&gt;. Early stopping rules provide guidance as to how many iterations can be run before the learner begins to over-fit. Early stopping rules have been employed in many different machine learning methods, with varying amounts of theoretical foundation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://en.wikipedia.org/wiki/Early_stopping&quot; rel=&quot;nofollow noreferrer&quot;&gt;Early Stopping (wikipedia)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:47:13.933" LastActivityDate="2018-05-10T20:47:13.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6348" PostTypeId="4" CreationDate="2018-05-10T20:47:13.933" Score="0" Body="For questions about the &quot;Early Stopping&quot; technique used to avoid overfitting in iterative training of AIs." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:47:13.933" LastActivityDate="2018-05-10T20:47:13.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6349" PostTypeId="5" CreationDate="2018-05-10T20:50:50.850" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;OpenAI is a non-profit artificial intelligence (AI) research company that aims to promote and develop friendly AI in such a way as to benefit humanity as a whole. The organization aims to &quot;freely collaborate&quot; with other institutions and researchers by making its patents and research open to the public. The founders (notably Elon Musk and Sam Altman) are motivated in part by concerns about existential risk from artificial general intelligence.&#xA;  &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenAI&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI (wikipedia)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gym.openai.com/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:50:50.850" LastActivityDate="2018-05-10T20:50:50.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6350" PostTypeId="4" CreationDate="2018-05-10T20:50:50.850" Score="0" Body="For questions related to the openAI, including the Gym toolkit." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:50:50.850" LastActivityDate="2018-05-10T20:50:50.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6351" PostTypeId="5" CreationDate="2018-05-10T20:52:56.413" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In computer science, brute-force search or exhaustive search, also known as generate and test, is a very general problem-solving technique that consists of systematically enumerating all possible candidates for the solution and checking whether each candidate satisfies the problem's statement.&#xA;  &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://en.wikipedia.org/wiki/Brute-force_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brute Force Search (wikipedia)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:52:56.413" LastActivityDate="2018-05-10T20:52:56.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6352" PostTypeId="4" CreationDate="2018-05-10T20:52:56.413" Score="0" Body="For questions about exhaustive search, also known as &quot;brute force&quot;." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:52:56.413" LastActivityDate="2018-05-10T20:52:56.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6353" PostTypeId="5" CreationDate="2018-05-10T20:57:11.640" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Semantics (from Ancient Greek: σημαντικός sēmantikos, &quot;significant&quot;) is the linguistic and philosophical study of meaning, in language, programming languages, formal logics, and semiotics. It is concerned with the relationship between signifiers—like words, phrases, signs, and symbols—and what they stand for, their denotation.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantics (wikipedia)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In programming language theory, semantics is the field concerned with the rigorous mathematical study of the meaning of programming languages. It does so by evaluating the meaning of syntactically legal strings defined by a specific programming language, showing the computation involved. In such a case that the evaluation would be of syntactically illegal strings, the result would be non-computation. Semantics describes the processes a computer follows when executing a program in that specific language. This can be shown by describing the relationship between the input and output of a program, or an explanation of how the program will execute on a certain platform, hence creating a model of computation.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantics - Computer Science (wikipedia)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;SEE ALSO:&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_equivalence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantic equivalence&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational semantics&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics_of_logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantics of logic&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Formal_semantics_(linguistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Formal semantics (linguistics)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:57:11.640" LastActivityDate="2018-05-10T20:57:11.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6354" PostTypeId="4" CreationDate="2018-05-10T20:57:11.640" Score="0" Body="For questions about meaning.  Distinct from syntactics." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T20:57:11.640" LastActivityDate="2018-05-10T20:57:11.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6355" PostTypeId="5" CreationDate="2018-05-10T21:06:37.817" Score="0" Body="&lt;p&gt;SEE:&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayes%27_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayes theorem&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian probability&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_inference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian inference&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian statistics&lt;/a&gt;&#xA;&lt;br&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_things_named_after_Thomas_Bayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;List of things named after Thomas Bayes&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:06:37.817" LastActivityDate="2018-05-10T21:06:37.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6356" PostTypeId="4" CreationDate="2018-05-10T21:06:37.817" Score="0" Body="For questions related to Thomas Bayes, his work and extension of his work, theory and applications.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:06:37.817" LastActivityDate="2018-05-10T21:06:37.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6357" PostTypeId="5" CreationDate="2018-05-10T21:22:16.923" Score="0" Body="&lt;p&gt;SEE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rational Agent&lt;/a&gt;*&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perfect Rationality&lt;/a&gt;*&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bounded Rationality&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Superrationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superrationality&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Von Neumann–Morgenstern Utility Theorem&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Imputation_(game_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imputation (Individual Rationality)&lt;/a&gt;&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Principle_of_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principle of Rationality&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rationality (general)&lt;/a&gt;*&#xA;&lt;br&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rationalism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rationalism&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*At time of posting, these articles have issues and need verification or citations.  &lt;em&gt;Will look to fix the wikis or link to alternate, vetted sources such as Stanford University.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:22:16.923" LastActivityDate="2018-05-10T21:22:16.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6358" PostTypeId="4" CreationDate="2018-05-10T21:22:16.923" Score="0" Body="For questions about the concept of rationality, its definitions and applications, including &quot;rational agent&quot;, perfect rationality, and bounded rationality." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:22:16.923" LastActivityDate="2018-05-10T21:22:16.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6359" PostTypeId="5" CreationDate="2018-05-10T21:25:42.237" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Intelligence has been defined in many different ways including the capacity for logic, understanding, self-awareness, learning, emotional knowledge, reasoning, planning, creativity, and problem solving. It can be more generally described as the ability to perceive or infer information, and to retain it as knowledge to be applied towards adaptive behaviors within an environment or context.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Intelligence is most widely studied in humans but has also been observed in both non-human animals and in plants. Intelligence in machines is called artificial intelligence, which is commonly implemented in computer systems using program software.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligence (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:25:42.237" LastActivityDate="2018-05-10T21:25:42.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6360" PostTypeId="4" CreationDate="2018-05-10T21:25:42.237" Score="0" Body="For questions about the concept of intelligence, not restricted to Artificial Intelligence, but applied also to intelligence in nature, quantification of intelligence (problem solving strength), and qualification of intelligence (terminology and the range of meanings)." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:25:42.237" LastActivityDate="2018-05-10T21:25:42.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6361" PostTypeId="5" CreationDate="2018-05-10T21:28:00.400" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In mathematics, statistics, and computer science, particularly in the fields of machine learning and inverse problems, regularization is a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Regularization_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Regularization_(mathematics)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:28:00.400" LastActivityDate="2018-05-10T21:28:00.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6362" PostTypeId="4" CreationDate="2018-05-10T21:28:00.400" Score="0" Body="For questions about application of regularization techniques." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:28:00.400" LastActivityDate="2018-05-10T21:28:00.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6363" PostTypeId="5" CreationDate="2018-05-10T21:30:33.137" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Opinion mining (sometimes known as sentiment analysis or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Generally speaking, sentiment analysis aims to determine the attitude of a speaker, writer, or other subject with respect to some topic or the overall contextual polarity or emotional reaction to a document, interaction, or event. The attitude may be a judgment or evaluation (see appraisal theory), affective state (that is to say, the emotional state of the author or speaker), or the intended emotional communication (that is to say, the emotional effect intended by the author or interlocutor).&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sentiment_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sentiment Analysis (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:30:33.137" LastActivityDate="2018-05-10T21:30:33.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6364" PostTypeId="4" CreationDate="2018-05-10T21:30:33.137" Score="0" Body="For questions about sentiment analysis in AI." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-10T21:30:33.137" LastActivityDate="2018-05-10T21:30:33.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6366" PostTypeId="1" CreationDate="2018-05-11T05:11:00.057" Score="3" ViewCount="445" Body="&lt;p&gt;Goal -&#xA;I am trying to implement a genetic algorithm to optimise the fitness of a&#xA;species of creatures in a simulated two-dimensional world. The world contains edible foods, placed at random, and a population of monsters (your basic zombies).  I need the algorithm to find behaviours that keep the creatures well fed and not dead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What i have done -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i start off by generating a 11x9 2d array in numpy, this is filled with random floats between 0 and 1. I then use np.matmul to go through each row of the array and multiply all of the random weights by all of the percepts (w1+p1*w2+p2....w9+p9) = a1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This first generation is run and I then evaluate the fitness of each creature using (energy + (time of death * 100)). From this I build a list of creatures who performed above the average fitness. I then take the best of these &quot;elite&quot; creatures and put them back into the next population. For the remaining space I use a crossover function which takes two randomly selected &quot;elite&quot; creatures and mixes their genes. I have tested two different crossover functions one which does a two point crossover on each row and one which takes a row from each parent until the new child has a complete chromosome. My issue is that the creatures just don't really seem to be learning, at 75 turns I will only get 1 survivor every so often.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am fully aware this might not be enough to go off but I am truly stuck on this and cannot figure out how to get these creatures to learn even though I think I am implementing the correct procedures. Occasionally I will get a 3-4 survivors rather than 1 or 2 but it appears to occur completely randomly, doesn't seem like there is much learning happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the main section of code, it includes everything I have done but none of the provided code for the simulation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from cosc343world import Creature, World&#xA;import numpy as np&#xA;import time&#xA;import matplotlib.pyplot as plt&#xA;import random&#xA;import itertools&#xA;&#xA;&#xA;# You can change this number to specify how many generations creatures are going to evolve over.&#xA;numGenerations = 2000&#xA;&#xA;# You can change this number to specify how many turns there are in the simulation of the world for a given generation.&#xA;numTurns = 75&#xA;&#xA;# You can change this number to change the world type.  You have two choices - world 1 or 2 (described in&#xA;# the assignment 2 pdf document).&#xA;worldType=2&#xA;&#xA;# You can change this number to modify the world size.&#xA;gridSize=24&#xA;&#xA;# You can set this mode to True to have the same initial conditions for each simulation in each generation - good&#xA;# for development, when you want to have some determinism in how the world runs from generation to generation.&#xA;repeatableMode=False&#xA;&#xA;# This is a class implementing you creature a.k.a MyCreature.  It extends the basic Creature, which provides the&#xA;# basic functionality of the creature for the world simulation.  Your job is to implement the AgentFunction&#xA;# that controls creature's behaviour by producing actions in response to percepts.&#xA;class MyCreature(Creature):&#xA;&#xA;    # Initialisation function.  This is where your creature&#xA;    # should be initialised with a chromosome in a random state.  You need to decide the format of your&#xA;    # chromosome and the model that it's going to parametrise.&#xA;    #&#xA;    # Input: numPercepts - the size of the percepts list that the creature will receive in each turn&#xA;    #        numActions - the size of the actions list that the creature must create on each turn&#xA;    def __init__(self, numPercepts, numActions):&#xA;&#xA;        # Place your initialisation code here.  Ideally this should set up the creature's chromosome&#xA;        # and set it to some random state.&#xA;        #self.chromosome = np.random.uniform(0, 10, size=numActions)&#xA;        self.chromosome = np.random.rand(11,9)&#xA;        self.fitness = 0&#xA;        #print(self.chromosome[1][1].size)&#xA;&#xA;        # Do not remove this line at the end - it calls the constructors of the parent class.&#xA;        Creature.__init__(self)&#xA;&#xA;&#xA;    # This is the implementation of the agent function, which will be invoked on every turn of the simulation,&#xA;    # giving your creature a chance to perform an action.  You need to implement a model here that takes its parameters&#xA;    # from the chromosome and produces a set of actions from the provided percepts.&#xA;    #&#xA;    # Input: percepts - a list of percepts&#xA;    #        numAction - the size of the actions list that needs to be returned&#xA;    def AgentFunction(self, percepts, numActions):&#xA;&#xA;        # At the moment the percepts are ignored and the actions is a list of random numbers.  You need to&#xA;        # replace this with some model that maps percepts to actions.  The model&#xA;        # should be parametrised by the chromosome.&#xA;&#xA;        #actions = np.random.uniform(0, 0, size=numActions)&#xA;&#xA;        actions = np.matmul(self.chromosome, percepts)&#xA;&#xA;        return actions.tolist()&#xA;&#xA;&#xA;# This function is called after every simulation, passing a list of the old population of creatures, whose fitness&#xA;# you need to evaluate and whose chromosomes you can use to create new creatures.&#xA;#&#xA;# Input: old_population - list of objects of MyCreature type that participated in the last simulation.  You&#xA;#                         can query the state of the creatures by using some built-in methods as well as any methods&#xA;#                         you decide to add to MyCreature class.  The length of the list is the size of&#xA;#                         the population.  You need to generate a new population of the same size.  Creatures from&#xA;#                         old population can be used in the new population - simulation will reset them to their&#xA;#                         starting state (not dead, new health, etc.).&#xA;#&#xA;# Returns: a list of MyCreature objects of the same length as the old_population.&#xA;&#xA;def selection(old_population, fitnessScore):&#xA;    elite_creatures = []&#xA;    for individual in old_population:&#xA;        if individual.fitness &amp;gt; fitnessScore:&#xA;            elite_creatures.append(individual)&#xA;&#xA;    elite_creatures.sort(key=lambda x: x.fitness, reverse=True)&#xA;&#xA;    return elite_creatures&#xA;&#xA;def crossOver(creature1, creature2):&#xA;    child1 = MyCreature(11, 9)&#xA;    child2 = MyCreature(11, 9)&#xA;    child1_chromosome = []&#xA;    child2_chromosome = []&#xA;&#xA;    #print(&quot;parent1&quot;, creature1.chromosome)&#xA;    #print(&quot;parent2&quot;, creature2.chromosome)&#xA;&#xA;    for row in range(11):&#xA;        chromosome1 = creature1.chromosome[row]&#xA;        chromosome2 = creature2.chromosome[row]&#xA;&#xA;        index1 = random.randint(1, 9 - 2)&#xA;        index2 = random.randint(1, 9 - 2)&#xA;&#xA;        if index2 &amp;gt;= index1:&#xA;            index2 += 1&#xA;        else:  # Swap the two cx points&#xA;            index1, index2 = index2, index1&#xA;&#xA;        child1_chromosome.append(np.concatenate([chromosome1[:index1],chromosome2[index1:index2],chromosome1[index2:]]))&#xA;        child2_chromosome.append(np.concatenate([chromosome2[:index1],chromosome1[index1:index2],chromosome2[index2:]]))&#xA;&#xA;    child1.chromosome = child1_chromosome&#xA;    child2.chromosome = child2_chromosome&#xA;&#xA;    #print(&quot;child1&quot;, child1_chromosome)&#xA;&#xA;    return(child1, child2)&#xA;&#xA;def crossOverRows(creature1, creature2):&#xA;    child = MyCreature(11, 9)&#xA;&#xA;    child_chromosome = np.empty([11,9])&#xA;&#xA;    i = 0&#xA;&#xA;    while i &amp;lt; 11:&#xA;        if i != 10:&#xA;            child_chromosome[i] = creature1.chromosome[i]&#xA;            child_chromosome[i+1] = creature2.chromosome[i+1]&#xA;        else:&#xA;            child_chromosome[i] = creature1.chromosome[i]&#xA;&#xA;        i += 2&#xA;&#xA;    child.chromosome = child_chromosome&#xA;&#xA;    return child&#xA;&#xA;    # print(&quot;parent1&quot;, creature1.chromosome[:3])&#xA;    # print(&quot;parent2&quot;, creature2.chromosome[:3])&#xA;    # print(&quot;crossover rows &quot;, child_chromosome[:3])&#xA;&#xA;&#xA;def newPopulation(old_population):&#xA;    global numTurns&#xA;&#xA;    nSurvivors = 0&#xA;    avgLifeTime = 0&#xA;    fitnessScore = 0&#xA;    fitnessScores = []&#xA;&#xA;    # For each individual you can extract the following information left over&#xA;    # from the evaluation.  This will allow you to figure out how well an individual did in the&#xA;    # simulation of the world: whether the creature is dead or not, how much&#xA;    # energy did the creature have a the end of simulation (0 if dead), the tick number&#xA;    # indicating the time of creature's death (if dead).  You should use this information to build&#xA;    # a fitness function that scores how the individual did in the simulation.&#xA;    for individual in old_population:&#xA;&#xA;        # You can read the creature's energy at the end of the simulation - it will be 0 if creature is dead.&#xA;        energy = individual.getEnergy()&#xA;&#xA;        # This method tells you if the creature died during the simulation&#xA;        dead = individual.isDead()&#xA;&#xA;        # If the creature is dead, you can get its time of death (in units of turns)&#xA;        if dead:&#xA;            timeOfDeath = individual.timeOfDeath()&#xA;            avgLifeTime += timeOfDeath&#xA;        else:&#xA;            nSurvivors += 1&#xA;            avgLifeTime += numTurns&#xA;&#xA;        if individual.isDead() == False:&#xA;            timeOfDeath = numTurns&#xA;&#xA;        individual.fitness = energy + (timeOfDeath * 100)&#xA;        fitnessScores.append(individual.fitness)&#xA;        fitnessScore += individual.fitness&#xA;        #print(&quot;fitnessscore&quot;, individual.fitness, &quot;energy&quot;, energy, &quot;time of death&quot;, timeOfDeath, &quot;is dead&quot;, individual.isDead())&#xA;&#xA;    fitnessScore = fitnessScore / len(old_population)&#xA;&#xA;    eliteCreatures = selection(old_population, fitnessScore)&#xA;&#xA;    print(len(eliteCreatures))&#xA;&#xA;    newSet = []&#xA;&#xA;    for i in range(int(len(eliteCreatures)/2)):&#xA;        if eliteCreatures[i].isDead() == False:&#xA;            newSet.append(eliteCreatures[i])&#xA;&#xA;    print(len(newSet), &quot; elites added to pop&quot;)&#xA;&#xA;    remainingRequired = w.maxNumCreatures() - len(newSet)&#xA;&#xA;    i = 1&#xA;&#xA;    while i in range(int(remainingRequired)):&#xA;        newSet.append(crossOver(eliteCreatures[i], eliteCreatures[i-1])[0])&#xA;        if i &amp;gt;= (len(eliteCreatures)-2):&#xA;            i = 1&#xA;        i += 1&#xA;&#xA;        remainingRequired = w.maxNumCreatures() - len(newSet)&#xA;&#xA;&#xA;    # Here are some statistics, which you may or may not find useful&#xA;    avgLifeTime = float(avgLifeTime)/float(len(population))&#xA;    print(&quot;Simulation stats:&quot;)&#xA;    print(&quot;  Survivors    : %d out of %d&quot; % (nSurvivors, len(population)))&#xA;    print(&quot;  Average Fitness Score :&quot;, fitnessScore)&#xA;    print(&quot;  Avg life time: %.1f turns&quot; % avgLifeTime)&#xA;&#xA;    # The information gathered above should allow you to build a fitness function that evaluates fitness of&#xA;    # every creature.  You should show the average fitness, but also use the fitness for selecting parents and&#xA;    # spawning then new creatures.&#xA;&#xA;&#xA;    # Based on the fitness you should select individuals for reproduction and create a&#xA;    # new population.  At the moment this is not done, and the same population with the same number&#xA;    # of individuals is returned for the next generation.&#xA;&#xA;    new_population = newSet&#xA;&#xA;    return new_population&#xA;&#xA;# Pygame window sometime doesn't spawn unless Matplotlib figure is not created, so best to keep the following two&#xA;# calls here.  You might also want to use matplotlib for plotting average fitness over generations.&#xA;plt.close('all')&#xA;fh=plt.figure()&#xA;&#xA;# Create the world.  The worldType specifies the type of world to use (there are two types to chose from);&#xA;# gridSize specifies the size of the world, repeatable parameter allows you to run the simulation in exactly same way.&#xA;w = World(worldType=worldType, gridSize=gridSize, repeatable=repeatableMode)&#xA;&#xA;#Get the number of creatures in the world&#xA;numCreatures = w.maxNumCreatures()&#xA;&#xA;#Get the number of creature percepts&#xA;numCreaturePercepts = w.numCreaturePercepts()&#xA;&#xA;#Get the number of creature actions&#xA;numCreatureActions = w.numCreatureActions()&#xA;&#xA;# Create a list of initial creatures - instantiations of the MyCreature class that you implemented&#xA;population = list()&#xA;for i in range(numCreatures):&#xA;   c = MyCreature(numCreaturePercepts, numCreatureActions)&#xA;   population.append(c)&#xA;&#xA;# Pass the first population to the world simulator&#xA;w.setNextGeneration(population)&#xA;&#xA;# Runs the simulation to evaluate the first population&#xA;w.evaluate(numTurns)&#xA;&#xA;# Show the visualisation of the initial creature behaviour (you can change the speed of the animation to 'slow',&#xA;# 'normal' or 'fast')&#xA;w.show_simulation(titleStr='Initial population', speed='normal')&#xA;&#xA;for i in range(numGenerations):&#xA;    print(&quot;\nGeneration %d:&quot; % (i+1))&#xA;&#xA;    # Create a new population from the old one&#xA;    population = newPopulation(population)&#xA;&#xA;    # Pass the new population to the world simulator&#xA;    w.setNextGeneration(population)&#xA;&#xA;    # Run the simulation again to evaluate the next population&#xA;    w.evaluate(numTurns)&#xA;&#xA;    # Show the visualisation of the final generation (you can change the speed of the animation to 'slow', 'normal' or&#xA;    # 'fast')&#xA;    if i==numGenerations-1:&#xA;        w.show_simulation(titleStr='Final population', speed='normal')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15571" LastEditorUserId="1671" LastEditDate="2018-05-11T13:26:12.777" LastActivityDate="2018-08-08T15:38:16.817" Title="Genetic Algorithm - creatures in 2d world are not learning" Tags="&lt;genetic-algorithms&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6367" PostTypeId="2" ParentId="6333" CreationDate="2018-05-11T08:16:03.120" Score="7" Body="&lt;p&gt;Don't forget the many Chinese companies heavily investing in AI.&lt;br&gt;&#xA;Not only the biggest 3 (Tencent, Alibaba, Baidu) but many others (JD.com, Sina, ...) and also startups (SenseTime, iCarbonX) are very active, in all AI sectors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have a couple of advantages for the future:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a huge and growing domestic market  &lt;/li&gt;&#xA;&lt;li&gt;availability of large amounts of data coupled with more relaxed data privacy regulations &lt;/li&gt;&#xA;&lt;li&gt;strong support from the government (in the form of the 2030 AI Strategic plan); how many countries can even plan for something longer than 5 years?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8898" LastActivityDate="2018-05-11T08:16:03.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6368" PostTypeId="1" CreationDate="2018-05-11T08:45:08.123" Score="4" ViewCount="414" Body="&lt;p&gt;So I read about softmax from this &lt;a href=&quot;http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;. Apparently to me these 2 are almost similar, except that the probability of all classes in softmax add to 1. According to their last paragraph for &lt;code&gt;number of classes = 2&lt;/code&gt;, softmax reduces to LR. What I want to know is other than &lt;code&gt;number of classes = 2&lt;/code&gt; what are the essential differences between LR and softmax. Like in terms of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Performance.&lt;/li&gt;&#xA;&lt;li&gt;Computational Requirements.&lt;/li&gt;&#xA;&lt;li&gt;Ease of calculation of derivatives.&lt;/li&gt;&#xA;&lt;li&gt;Ease of visualization.&lt;/li&gt;&#xA;&lt;li&gt;Number of minimas in the convex cost function, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other differences are also welcome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT :&lt;/strong&gt; I am asking for relative comparisons only, so that at the time of implementation I have no difficulty in selecting which method of implementation to use.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-22T04:23:22.670" LastActivityDate="2018-07-09T16:54:57.867" Title="Difference between softmax and Logistic Regression?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6369" PostTypeId="2" ParentId="6368" CreationDate="2018-05-11T11:15:34.917" Score="3" Body="&lt;p&gt;As written, SoftMax is a generalization of Logistic Regression.&lt;br&gt;&#xA;Hence:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Performance&lt;br&gt;&#xA;If the model has more than 2 classes then you can't compare. Given &lt;code&gt;K = 2&lt;/code&gt; they are the same.&lt;/li&gt;&#xA;&lt;li&gt;Computation Requirements&lt;br&gt;&#xA;Please explain as the computational requirements require the data, enough memory to hold it and enough time to let run.&lt;/li&gt;&#xA;&lt;li&gt;Ease of Calculation of Derivatives&lt;br&gt;&#xA;The cost function is summation hence once you do it for one element you do it fol all.&lt;/li&gt;&#xA;&lt;li&gt;Ease of Visualization&lt;br&gt;&#xA;Well, it is easy to visualize the Confusion Matrix even for &lt;code&gt;K = 10&lt;/code&gt; classes. So no issue here.&lt;/li&gt;&#xA;&lt;li&gt;Cost Function&lt;br&gt;&#xA;The cost function is convex. Yet not Strictly Convex hence there infinite number of minimas.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1725" LastActivityDate="2018-05-11T11:15:34.917" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6370" PostTypeId="1" CreationDate="2018-05-11T12:07:04.040" Score="1" ViewCount="47" Body="&lt;p&gt;In the model generation, in machine learning (consider supervised) If some data change the previous model function drastically then we should study that data. Does it happen? How to handle such situation? &lt;/p&gt;&#xA;" OwnerUserId="15368" LastEditorUserId="1671" LastEditDate="2018-05-11T13:16:48.210" LastActivityDate="2018-05-11T13:16:48.210" Title="Data Interpretation technique" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;datasets&gt;&lt;models&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6371" PostTypeId="2" ParentId="6325" CreationDate="2018-05-11T13:38:14.060" Score="2" Body="&lt;p&gt;General AI can absolutely exist in a 2D world, just that a generalized AI (defined here as &quot;consistent strength across a set of problems&quot;) in this context would still be quite distinct from an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;, defined as &quot;an algorithm that can perform any intellectual task that a human can.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even there, the definition of AGI is fuzzy, because &quot;which human?&quot;  (Human intelligence is a spectrum, where individuals possess different degrees of problem solving capability in different contexts.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Consciousness&lt;/a&gt;: Unfortunately, &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-awareness&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-awareness&lt;/a&gt; / &lt;a href=&quot;https://en.wikipedia.org/wiki/Consciousness&quot; rel=&quot;nofollow noreferrer&quot;&gt;consciousness&lt;/a&gt; is a heavily metaphysical, issue, distinct from problem solving capability (intelligence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You definitely want to look a the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room&lt;/a&gt;&quot; and rebuttals. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Probably worth looking at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Holographic_principle&quot; rel=&quot;nofollow noreferrer&quot;&gt;holographic principle&lt;/a&gt;: &quot;a concept in physics whereby a space is considered as a hologram of n-1 dimensions.&quot;  Certainly models and games can be structured in this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another place to explore is theories of emergence of superintelligence on infinite &lt;a href=&quot;https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conway's Game of Life&lt;/a&gt;.  (In a nutshell, my understanding is that once researchers figured out how to generate any number within the cellular automata, the possibility of emergent sentience given a gameboard of sufficient size is at least theoretically sound.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-11T17:30:37.487" LastActivityDate="2018-05-11T17:30:37.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6372" PostTypeId="5" CreationDate="2018-05-11T14:04:45.443" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0058%3Aentry%3Dgignw%2Fskw&quot; rel=&quot;nofollow noreferrer&quot;&gt;γνῶθι&lt;/a&gt; &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0058%3Aentry%3Dseautou%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;σεαυτόν&lt;/a&gt;  (Know Thyself!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Delphic_maxims&quot; rel=&quot;nofollow noreferrer&quot;&gt;Delphic Maxims&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-11T14:04:45.443" LastActivityDate="2018-05-11T14:04:45.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6373" PostTypeId="4" CreationDate="2018-05-11T14:04:45.443" Score="0" Body="Use for questions about self awareness in general, not restricted to engineered artifacts (AI's).  Can be synonymous with &quot;Artificial Consciousness&quot; but has a broader scope.  &quot;Self Awareness&quot; should be used for strictly philosophical questions related to AIs and humans. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-05-11T14:04:45.443" LastActivityDate="2018-05-11T14:04:45.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6375" PostTypeId="2" ParentId="6325" CreationDate="2018-05-11T19:58:08.080" Score="3" Body="&lt;p&gt;I think the most important thing is that it has to have time simulated in some way. Think self aware chatbot. Then to be &quot;self aware&quot; the environment could be data that is fed in through time that can be distinguished as being &quot;self&quot; and &quot;other&quot;. By that I suppose I mean &quot;self&quot; is the part it influences directly and &quot;other&quot; is the part that is influenced indirectly or not at all. Other than that it probably can live inside pretty abstract environments. The reason time is so important is without it the cognitive algorithm is just solving a math problem. &lt;/p&gt;&#xA;" OwnerUserId="15580" LastActivityDate="2018-05-11T19:58:08.080" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6376" PostTypeId="2" ParentId="6338" CreationDate="2018-05-11T20:19:27.310" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayes%27_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayes' theorem&lt;/a&gt; relates conditional probabilities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="15533" LastEditorUserId="2444" LastEditDate="2019-05-01T21:11:18.013" LastActivityDate="2019-05-01T21:11:18.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6377" PostTypeId="1" CreationDate="2018-05-12T04:31:12.310" Score="3" ViewCount="1572" Body="&lt;p&gt;Why momentum factor greater than 1 is a bad idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the mathematical conclusions?&lt;/p&gt;&#xA;" OwnerUserId="15587" LastEditorUserId="9947" LastEditDate="2018-08-28T12:25:23.540" LastActivityDate="2018-08-28T12:25:23.540" Title="Why MLP momentum term must be in the range 0-1?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;multilayer-perceptron&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6378" PostTypeId="2" ParentId="6343" CreationDate="2018-05-12T06:54:44.210" Score="2" Body="&lt;p&gt;One of the best ways to learn is use reference of others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;https://www.ics.uci.edu/~pjsadows/notes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Peter Sadowski - Notes on Backpropagation&lt;/a&gt; (Page 3):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/U3Cal.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U3Cal.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a great blog post by &lt;a href=&quot;https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eli Bendersky - The Softmax Function and Its Derivative&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1725" LastEditorUserId="1725" LastEditDate="2018-05-12T07:04:10.860" LastActivityDate="2018-05-12T07:04:10.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6379" PostTypeId="2" ParentId="6325" CreationDate="2018-05-12T09:43:22.293" Score="5" Body="&lt;p&gt;I think this is one of the best AGI related questions I've see in this forum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will skip all thematic about &quot;what is an AGI&quot;, &quot;simulation game&quot;, ... These topics have been discussed during decades and nowadays they are, in my opinion, a dead end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I can only answer with my personal experience:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a basic theorem in computing that any number of dimensions, including temporal one, in a finite size space, can be reduced to 1D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in practical examples, the 1D representation becomes hard to analyze and visualize. It is more practical work with graphs, that can be seen as an intermediate between 1D and 2D. Graphs allows representation of all necessary facts and relations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By example, if we try to develop an AGI able to work in the area of mathematics, any expression (that humans we write in a 2D representation with rationals, subscripts, integrals, ...) can be represented as 1D (as an expression written in a program source) but this 1D must be parsed to reach the graph that can be analyzed or executed. Thus, the graph that results after the parsing of the expression is the most practical representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example, if we want an agent that travels across a 3D world, this world can be seen as an empty space with objects that have some properties. Again, after the initial stage of scene analysis and object recognition (the equivalent to the parser in previous example), we reach a graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, to really work in the area of AGI, I suggest skip the problems of scene analysis, object recognition, speech recognition (Narrow AI), and work directly over the representative graphs.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-05-12T10:09:52.337" LastActivityDate="2018-05-12T10:09:52.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6380" PostTypeId="2" ParentId="6333" CreationDate="2018-05-12T10:24:12.470" Score="3" Body="&lt;p&gt;If we talk about Narrow AI, big companies have an evident advantage, as you say. Moreover, because they have the capability of buy small challenging companies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we talk about General AI, the chances are equal, the muse will appear when they want to the person they want. &lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-05-12T11:30:43.207" LastActivityDate="2018-05-12T11:30:43.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6381" PostTypeId="1" CreationDate="2018-05-12T11:07:27.917" Score="2" ViewCount="494" Body="&lt;p&gt;I'm trying to find the optimal policy for the mountain car problem using deep Q learning with images as input, however, I cannot find a way to get my Q function to give me good solutions (I followed multiple tutorials for similar problems (Atari games and Flappy bird)).&#xA;I'm working on Python with Keras.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images are given in the following format :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/gAfLsdN.png&quot; alt=&quot;input&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;400x400 pixels, where the bar on the bottom right corner represents the speed of the car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check where my problem might lie, I thought it would be best if I split the problem by first ensuring that I can find a network which would successfully find the state of the car (position and speed, since it's all we need to find it) by feeding my convolutional network images of random states (uniformly distributed within the state space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After unsuccessful results, I decided to split it even more by only trying to find the two state variable separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the best kind of result that I get, when the network doesn't predict all the state to be the same (which seem to happen a lot with relu activation).&#xA;The state space is divided in 50x50 matrix to make my predictions. The predicted speed is on the left, and the absolute error is on the right&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/313Kkbk.png&quot; alt=&quot;state_error&quot;&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images fed to the network are pre-processed the follow way : &#xA; 1. Gray scale&#xA; 2. Resize (I tried 50x50, 100x100 and 150x150)&#xA; 3. Values centered around 0 in [-1;1] (this seemed to help a bit with the relu activation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network I used to try to find the speed is first a convolution layer (I tried 32 windows of kernel_size=(4,4) (8,8) and (16,16), strides=(1,1) and (2,2), activation= relu, linear, tanh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optional additional convolution layers of kernel size half the previous layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And an optional last dense layer of dimension 32 or activation relu, linear or tanh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output layer is dimension one with linear activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I train the network is by feeding the fit function with 32 random samples and let the network train for 25 epochs with batch_size 32 and repeat  ad libitum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's becoming extremely frustrating, especially with my GPU not fitting the requirement for GPU computation to check if I get some results faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me if I'm doing something wrong that I'm missing and what can I do to improve my method to eventually manage to get the reinforcement algorithm to work ?&#xA;Like the size of the training sample, batch size and epochs of the fit function, the structure of the network, ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit : I finally found a way for my reinforcement learning to converge to the true Q value : each time I run a new episode and put it in my replay memory, I run many fits of different mini-batches. This is something I thought I did by increasing the number of epochs in the parameter of the fit function, but I guess it doesn't work as I thought it would.&#xA;I'm letting it train a bit and then I will try the same method for the sub-problems mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="15590" LastEditorUserId="15590" LastEditDate="2018-05-13T00:54:01.607" LastActivityDate="2018-05-13T00:54:01.607" Title="Mountain car problem with images - not converging" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6383" PostTypeId="1" CreationDate="2018-05-12T16:33:34.113" Score="7" ViewCount="2253" Body="&lt;p&gt;I am pretty much a beginner in Tensorflow and simply follow a tutorial. There is no problem with my code, but I have a question regarding the output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;accuracy: 0.95614034&#xA;accuracy_baseline: 0.6666666&#xA;auc: 0.97714674&#xA;auc_precision_recall: 0.97176754&#xA;average_loss: 0.23083039&#xA;global_step: 760&#xA;label/mean: 0.33333334&#xA;loss: 6.578666&#xA;prediction/mean: 0.3428335&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to know what does &quot;prediction/mean&quot; and &quot;label/mean&quot; represent?&lt;/p&gt;&#xA;" OwnerUserId="15593" LastEditorUserId="2444" LastEditDate="2020-06-02T23:29:44.483" LastActivityDate="2020-06-02T23:29:44.483" Title="What do prediction/mean and label/mean represent in this TensorFlow code?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ClosedDate="2020-06-02T23:30:16.060" ContentLicense="CC BY-SA 4.0" />
  <row Id="6384" PostTypeId="2" ParentId="6325" CreationDate="2018-05-12T18:46:01.773" Score="2" Body="&lt;p&gt;Of the answers so far, the one from @DukeZhou were the most provocative. For instance, the reference to the Chinese Room critique brings up Searle's contention that some form of intentionality might need support in the artificial environment. This might imply the necessity of a value system or a pain-pleasure system, i.e. something where good consequences can be &quot;experienced&quot; or actively sought and bad consequences avoided. Or some potential for individual extinction (death or termination) might need to be recognized. The &lt;a href=&quot;https://ai.stackexchange.com/questions/2955/whats-the-term-for-death-by-dissolving-in-ai&quot;&gt;possibility of &quot;ego-death&quot;&lt;/a&gt; might need to have a high negative value.  That might imply an artificial world should include &quot;other minds&quot; or other agents, which the emerging or learning intelligent agent could observe (in some sense) and &quot;reflect on&quot;, i.e recognize an intelligence like its own. In this sense the Cartesian syllogism &quot;I think therefore I am&quot; gets transmuted into: I (or rather me as an AI) see evidence of others thinking, and &quot;by gawd, 'I' can, too&quot;. Those &quot;others&quot; could be either other learning systems (AGI's) or some contact with discrete inputs from humans mediated by the artificial environment. &lt;a href=&quot;https://en.wikipedia.org/wiki/Reverse_Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia discussion of the &quot;reverse Turing test&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mention of dimensionality should provoke a discussion of what would be the required depth of representation of a &quot;physics&quot; of the world external to the AI. Some representation of time and space would seem necessary, i.e., some dimensional substructure for progress to goal attainment. The Blocks World was an early &lt;a href=&quot;https://en.wikipedia.org/wiki/Toy_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;toy problem&lt;/a&gt; whose solution provoked optimism in the 60's and 70's of last century that substantial progress was being made. I'm not aware of any effort to program in any pain or pleasure in the SHRDLU program of that era (no dropping blocks on the program's toes), but all the interesting science fiction representations of AI's have some recognition of &quot;physical&quot; adverse consequences in the &quot;real world&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'm going to add a need for &quot;entities with features&quot; in this environment that could be &quot;perceived&quot; (by any of the &quot;others&quot; that are interacting with the AGI) as the data input to efforts at induction, identification, and inference about relationships. This creates a basis for a shared &quot;experience&quot;.&lt;/p&gt;&#xA;" OwnerUserId="15594" LastEditorUserId="15594" LastEditDate="2018-05-13T02:28:00.993" LastActivityDate="2018-05-13T02:28:00.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6385" PostTypeId="2" ParentId="6333" CreationDate="2018-05-12T19:01:27.420" Score="1" Body="&lt;p&gt;I'm surprised no one has mentioned IBM. They first build DeepBlue and beat the highest ranked chess grandmaster. Then they crushed the Jeopardy champs. Now they are merchandising the variants of Watson.&lt;/p&gt;&#xA;" OwnerUserId="15594" LastEditorUserId="33" LastEditDate="2018-05-12T19:51:21.020" LastActivityDate="2018-05-12T19:51:21.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6386" PostTypeId="2" ParentId="6377" CreationDate="2018-05-12T19:04:40.900" Score="1" Body="&lt;p&gt;If &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent&lt;/a&gt; is like walking down a slope, momentum would be the literal momentum of the agent traversing the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperplane&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyperplane&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under that analogy then, momentum factor would be analogous to the &lt;a href=&quot;https://simple.wikipedia.org/wiki/Coefficient_of_friction&quot; rel=&quot;nofollow noreferrer&quot;&gt;friction coefficient&lt;/a&gt;, with 1 being max friction and 0 being no friction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should be able to see why there can't be friction beyond that range: if friction = 1 it would be identical to having no friction; if friction &amp;lt;= 0 then by conservation of energy gradient descent will not find a local &lt;a href=&quot;https://en.wikipedia.org/wiki/Maxima_and_minima&quot; rel=&quot;nofollow noreferrer&quot;&gt;minima&lt;/a&gt;; if friction &gt; 1 then gradient descent would be moving backwards.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="1671" LastEditDate="2018-05-12T22:51:33.523" LastActivityDate="2018-05-12T22:51:33.523" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6387" PostTypeId="2" ParentId="6333" CreationDate="2018-05-12T19:53:15.670" Score="13" Body="&lt;p&gt;Pretty much any &quot;kid in a garage&quot; so to speak. Nobody knows how to build AI yet and a major breakthrough could come from anywhere. That could knock Google off their perch easily enough. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-05-12T19:53:15.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6388" PostTypeId="2" ParentId="6383" CreationDate="2018-05-12T20:59:56.697" Score="5" Body="&lt;p&gt;All of these could be problem specific (except maybe accuracy). Most of it is &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/metrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;documented here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;accuracy: Percentage of correct number of classifications&lt;/li&gt;&#xA;&lt;li&gt;accuracy_baseline: Accuracy baseline based on labels mean. This is the best the model could do by always predicting one class. (&lt;a href=&quot;https://github.com/tensorflow/estimator/blob/master/tensorflow_estimator/python/estimator/canned/metric_keys.py#L35&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;AUC or Area Under the (ROC) Curve is quite complicated, but tells you something about the true/false positive rate. In short: the AUC is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one.&lt;/li&gt;&#xA;&lt;li&gt;auc_precision_recall: Is the percentage of relevant intstances, among the retrieved instances, that have been retrieved over the total amount of relevant instances.&lt;/li&gt;&#xA;&lt;li&gt;average_loss: You're usually minimizing some function, and this is likely the average value of that function given the current batches.&lt;/li&gt;&#xA;&lt;li&gt;loss: The current value of the loss (as above). Either the sum of the losses, or the loss of the last batch.&lt;/li&gt;&#xA;&lt;li&gt;global_step: Number of iterations.&lt;/li&gt;&#xA;&lt;li&gt;label/mean and prediction/mean: Not really sure, but I suspect that if you have two classes then the label/mean is the mean of the value labels, whilst prediction/mean could be the value of the corresponding predictions. (two classes could give you a value between 0 and 1)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="-1" LastEditDate="2019-01-12T03:46:39.953" LastActivityDate="2019-01-12T03:46:39.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6389" PostTypeId="1" AcceptedAnswerId="6396" CreationDate="2018-05-12T22:23:09.803" Score="2" ViewCount="175" Body="&lt;p&gt;As the title says, should I reset the exploration rate between trials?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any actions (i.e. didn't perform any significant swing). The Actor-Critic tutorial I followed did not reset the exploration rate (&lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-w-keras-openai-actor-critic-models-f084612cfd69&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) but it seems like there are lots of mistakes in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that it should be reset since the model might start from a new unknown situation in a different trial and not know what to do without exploring. &lt;/p&gt;&#xA;" OwnerUserId="13257" LastEditorUserId="2444" LastEditDate="2019-02-16T02:57:21.950" LastActivityDate="2019-02-16T02:57:21.950" Title="Should the exploration rate be reset after each trial in Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6393" PostTypeId="2" ParentId="6313" CreationDate="2018-05-13T11:20:44.463" Score="1" Body="&lt;p&gt;You can solve something rationally or with emotions/intuition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence can be rational or intuitive. Rational is the newest more accurate form of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans use both types of intelligences.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-05-13T11:20:44.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6394" PostTypeId="2" ParentId="6344" CreationDate="2018-05-13T15:41:26.223" Score="1" Body="&lt;p&gt;In short:&lt;br&gt;&#xA;&lt;strong&gt;1/2 -&gt;&lt;/strong&gt;  ANNs don't have problems with &quot;different types&quot; of data as long as they are represented using real numbers: the inputs for your ANN represent lengths and are easy to understand and process.&lt;br&gt;&#xA;&lt;strong&gt;2/2 -&gt;&lt;/strong&gt;  The variable number of inputs is a little bit more tricky. In general it is not a problem either. The net will compensate the absence of some inputs from time to time (when food is too far away?). You don't necessarily have to use RNNs for your case.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In full:&lt;br&gt;&#xA;I think that the inputs of your net could be improved.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When building a new ANN you always have to consider few aspects:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neural Nets are just stateless functions that map inputs to outputs. &lt;/li&gt;&#xA;&lt;li&gt;There is no magic here: with good inputs you can get good outputs. &lt;/li&gt;&#xA;&lt;li&gt;Think about how you would write an algorithm to solve that problem. Which would the best set of inputs be? Can some computation be done somewhere else? (using other ANNs, if you want to create ANN chains, or traditional algorithms).&lt;/li&gt;&#xA;&lt;li&gt;Preprocessing can do a big difference!&lt;/li&gt;&#xA;&lt;li&gt;Think about the space separation that your inputs require/cause.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All these points are related one to each other and they're only about the inputs you're providing to your net.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, keeping these points in mind, you can check again your inputs to see if you can improve something:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objective 1: eat food&lt;/strong&gt;&lt;br&gt;&#xA;To be sure the snake will get toward the food we definitely need to pass the ANN the position of the food. But you're sure that a long list of coordinates is the best option? Maybe the coordinate of the nearest food can be enough.&lt;br&gt;&#xA;If you were THAT snake what would you  like to receive? Or what would you receive if you were a real snake? Imagine your snake has a sensor that determines the best place to get food. It'll be like a compass, outputting only one value, from 0 to 365 (or choose whatever range you like). This compass will point at the &lt;em&gt;closest bigger cluster&lt;/em&gt; of food or at the &lt;em&gt;closest food&lt;/em&gt;.&#xA;Remember: 1 value is always better than N and you should try to avoid a variable number of inputs, it's always a smell that something doesn't go well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objective 2: don't die&lt;/strong&gt;&lt;br&gt;&#xA;The distance from the centre works well. It's a simple threshold that the ANN will consider when moving, that's effective and easy to achieve. Why don't you also add another compass that will always tell you the position of the centre instead of the coordinates of the snake? In this way you've used 2 values related to each other instead of 3, more difficult to decipher for an ANN.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time is not really necessary. You're inserting a value that isn't related to any other, or the environment, or the food. Any possible behaviour is not influenced by its value (if the ANN knew that it has only 15 seconds left, it wouldn't change its behaviour. The same if it knew that it's &quot;alive&quot; since 1 minute or 4000).&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2018-05-13T15:41:26.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6395" PostTypeId="1" CreationDate="2018-05-13T15:48:56.120" Score="1" ViewCount="55" Body="&lt;p&gt;On this video&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=YUVLgccVi54&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to video&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a neurologist starts by saying that we do not know how neurons calculate gradients for backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At minute 30:39 hes showing faster convergence for &quot;our algorithm&quot;, which seems to converge faster than backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 34:36 it goes explaining how &quot;neurons&quot; in the brain are actually packs of neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not really understand all that he says, so I infer that those packs of neurons (which seem depicted as a single layer) are the ones who calculate the gradient.&#xA;It would make sense if each neuron makes a sightly different calculation, and then each other communicate the difference in results. That would allow to deduce a gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can be deduced, from the presented information, about the purported &quot;algorithm&quot;?? (From the viewpoint of improving convergence of an artificial neural network).&lt;/p&gt;&#xA;" OwnerUserId="15611" LastActivityDate="2018-05-13T18:40:12.547" Title="What can be deduced about the &quot;algorithm&quot; of backpropagation/gradient descent?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6396" PostTypeId="2" ParentId="6389" CreationDate="2018-05-13T16:06:04.387" Score="4" Body="&lt;p&gt;The exploration rate, typically parameterized as epsilon / ε, can be changed on every trial. It depends on the complexity of the model and the goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest thing to do is keep exploration rate high and fixed. That means the model will continue to explore new options, even at the cost of not &quot;exploiting&quot; the best available option. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is setting the exploration rate high at the beginning of learning so the model will search the space for possible successful solutions. Then as the model creates a set of policies that are successful for given states, the exploration rate can be lowered or decay. Exploration rate decay can be fixed (i.e., over time there is consistently less exploration and more exploitation). Exploration rate decay can be dynamic and learned. This last option is often the best but also the most complex to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;&lt;a href=&quot;http://theses.ubn.ru.nl/bitstream/handle/123456789/5216/Nieuwdorp%2C_T._1.pdf?sequence=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dare to Discover: The Effect of the Exploration Strategy on an Agent’s Performance&lt;/a&gt;&quot; goes into greater detail on this topic.&lt;/p&gt;&#xA;" OwnerUserId="15403" LastEditorUserId="15403" LastEditDate="2018-05-14T02:17:02.667" LastActivityDate="2018-05-14T02:17:02.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6397" PostTypeId="2" ParentId="6395" CreationDate="2018-05-13T16:29:59.423" Score="3" Body="&lt;p&gt;There are 3 separate issues that are often confounded in Deep Learning and Neuroscience:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Deep Learning is inspired by the way the biological brain works.&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning is how the biological brain works.&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning can model how the biological brain works.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Number 1 is accurate. The brain has many layers and many connections. Those principles have informed Deep Learning models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number 2 has little evidence to support that claim. The biological brain learns at the cellular level in very different ways than how any Deep Learnings system learns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number 3 is a current topic of research. Deep Learning is very good at learning patterns. There are good reasons to believe that Deep Learning can learn patterns in the brain. However, those Deep Learning models will not automatically give insight into the biological processes of the brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The video is an example of #1. Inspired by our current understanding of neurobiology, let's build better Deep Learning algorithms. These new algorithms might perform better on machine learning benchmarks. However, these algorithms are not better models of the biological brain. In order to understand the algorithms, the language of biology might not be helpful. It might be better to describe them mathematically.&lt;/p&gt;&#xA;" OwnerUserId="15403" LastEditorUserId="15403" LastEditDate="2018-05-13T18:40:12.547" LastActivityDate="2018-05-13T18:40:12.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6399" PostTypeId="2" ParentId="6377" CreationDate="2018-05-13T21:08:30.650" Score="0" Body="&lt;p&gt;Let's talk about gradient decent!&lt;/p&gt;&#xA;&lt;h2&gt;Analogy:&lt;/h2&gt;&#xA;&lt;p&gt;So you're standing on a mountain side, and you want to get to the lowest part of this mountain. You have a notepad with you.&lt;/p&gt;&#xA;&lt;p&gt;Although actual physics-momentum would be a good analogy here, I'm not gonna use it.&lt;/p&gt;&#xA;&lt;p&gt;You're somewhere on this mountain side and you figure out which way is down*, and you jump once a couple of meters in that direction. How big one jump is would depend on how steep the hill is (the length of the gradient), and how much extra you push with your feet. The first time you decide to not really push that much with your feet. The SGD momentum, comes here; you write down in your note pad which direction you went, and how far (e.g. south, 4 meters).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; here the PHYSICAL momentum would represent the length of the gradient.&lt;/p&gt;&#xA;&lt;p&gt;You repeat this for some time you come to a place where there are only ways upwards.&lt;/p&gt;&#xA;&lt;p&gt;Does this mean you hit the bottom? Not necessarily; you might have gotten stuck in a valley, or &amp;quot;local minima&amp;quot;. You really want to get out of this valley, but all directions are upwards, so which way should you jump?&lt;/p&gt;&#xA;&lt;p&gt;You now take out your notebook and notice that you've been jumping south east the last 40 steps, and pretty far. You then reason that it is likely that you want to go south east. So you jump south east with a lot of thrust from your feet: This is the intuition on what momentum does;&lt;/p&gt;&#xA;&lt;p&gt;If you have a clear &amp;quot;pattern&amp;quot; of which way is down, then this should also count!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; the momentum only depends on the previous step, but the previous step depends on the steps before that and so on. This is just an analogy.&lt;/p&gt;&#xA;&lt;h2&gt;Maths:&lt;/h2&gt;&#xA;&lt;p&gt;For the maths, you just add a term that is the last gradient, times some constant.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Heading(t)=γ Heading(t-1)+η Gradient(t)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where γ is the momentum factor and η is the learning rate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/index.html#momentum&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sebsastian Ruders blog on gradient descent&lt;/a&gt; is brilliant to learn more details of the maths of it.&lt;/p&gt;&#xA;&lt;h2&gt;γ ≷ 1&lt;/h2&gt;&#xA;&lt;p&gt;For mathematical conclusions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Heading(t)=γ Heading(t-1)+η Gradient(t)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;γ &amp;gt; 1: From the &amp;quot;expression&amp;quot;, you could infer that this case would generate echos. That the gradient of the previous step would contribute more than the actual gradient. For the upcoming step, this effect would get enhanced, and 10 steps down the road, you're stuck going in one direction.&lt;/p&gt;&#xA;&lt;p&gt;γ &amp;lt; 1 makes it &amp;quot;converge&amp;quot; to a &amp;quot;terminal velocity&amp;quot;, if you like. It would depend on the preceding steps less and less instead of more and more.&lt;/p&gt;&#xA;&lt;p&gt;These effects are pretty clear in the equation you find at &lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/index.html#momentum&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ruders blog&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If your momentum term was greater than one, then the notebook would overcome the actual gradient. After a few steps, you wouldn't even look at the hill; you'd go &amp;quot;I've only gone east so far, so I'll just continue east&amp;quot; with your jumps getting longer and longer. This is not good.&lt;/p&gt;&#xA;&lt;h2&gt;In conclusion&lt;/h2&gt;&#xA;&lt;p&gt;A high momentum term would lead you in the wrong direction (blow up and always go in the same direction), and/or oscillate around the global minima (making you jump too far).&lt;/p&gt;&#xA;&lt;p&gt;Hope it helps :)&lt;/p&gt;&#xA;&lt;p&gt;*: Strictly speaking, we're finding the &amp;quot;up&amp;quot; direction and go the opposite way. The &amp;quot;up&amp;quot; direction is the gradient.&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-05-13T21:40:08.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6401" PostTypeId="1" CreationDate="2018-05-14T05:33:52.187" Score="3" ViewCount="125" Body="&lt;p&gt;I need to design an algorithm such that it handles the request for shift swapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm will recommend a list of people who are more likely to swap that shift with the person by analyzing previous data.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can anyone list the techniques that will help me to do this or a good starting point?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I was thinking about training a Naive Bayes Classifier and using Mahaout for generating recommendation.&lt;/p&gt;&#xA;" OwnerUserId="15618" LastEditorUserId="1671" LastEditDate="2019-11-06T21:46:42.247" LastActivityDate="2019-11-06T21:46:42.247" Title="How to design a recommendation system for shift swapping?" Tags="&lt;ai-design&gt;&lt;training&gt;&lt;getting-started&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6402" PostTypeId="1" AcceptedAnswerId="6409" CreationDate="2018-05-14T05:53:45.683" Score="0" ViewCount="657" Body="&lt;p&gt;I am trying to create my own variant of Google duplex however, it won't make calls but just have a real time conversation. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My question is, where and how to start? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How do I train my model with real conversation and how do I make speech sound almost human like? Where do I incorporate RNN and how can I make my model understand nuances? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/p3PfKf0ndik&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/p3PfKf0ndik&lt;/a&gt; trying to create something like this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I understand this is very difficult but I am not looking for perfection. I am just creating this as I love doing this. Thank you&lt;/p&gt;&#xA;" OwnerUserId="15619" LastEditorUserId="1671" LastEditDate="2018-05-15T18:54:21.260" LastActivityDate="2018-05-15T18:54:21.260" Title="How can I create my own Google duplex?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;getting-started&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6403" PostTypeId="2" ParentId="6401" CreationDate="2018-05-14T08:38:45.070" Score="1" Body="&lt;p&gt;You might not even need a classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would devise a scoring function, based on analysis of the previous data you have. Each user gets a score based on features like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how many times in the past has this user swapped a shift with somebody else?&lt;/li&gt;&#xA;&lt;li&gt;how many times has the user swapped with the current user?&lt;/li&gt;&#xA;&lt;li&gt;how many times has the user swapped this particular shift?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For each criterion you add a number of points to the score; the second one might be weighted higher that the first one. Then the person with the highest score is most likely to switch shifts with your current user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main question is the design of the scoring function, but I don't think you'd need to go into all the overkill of setting up a classifier; just think which criteria would make someone more likely to swap, and encode them directly. This has the advantage that it's transparent, ie you can always see &lt;em&gt;why&lt;/em&gt; someone got recommended, and you can tweak your scoring method if the results are not quite what you'd want. This is often hard to do with ML classifiers.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-14T08:38:45.070" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6405" PostTypeId="2" ParentId="6297" CreationDate="2018-05-14T10:20:13.727" Score="2" Body="&lt;p&gt;There are more than 1 way of doing this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can compute the bleu score between them if you are looking at the quality of machine translation. Check this &lt;a href=&quot;https://en.wikipedia.org/wiki/BLEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You can convert them into 2 vectors using doc2vec and find the similarity between the vectors using cosine similarity.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/profile/Maarten_Versteegh/publication/304834009_Learning_Text_Similarity_with_Siamese_Recurrent_Networks/links/577c2e5d08aec3b743367064/Learning-Text-Similarity-with-Siamese-Recurrent-Networks.pdf?origin=publication_detail&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siamese networks&lt;/a&gt; are something similar to what you are asking. They are neural nets that use distance metric for learning rather than a loss metric.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't understand why do you want to use neural network for comparing two text pieces. Generally comparisons are done by some of the distance metrics not by neural network.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastEditorUserId="9062" LastEditDate="2018-05-14T11:07:11.837" LastActivityDate="2018-05-14T11:07:11.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6407" PostTypeId="1" AcceptedAnswerId="6410" CreationDate="2018-05-14T14:26:11.437" Score="1" ViewCount="299" Body="&lt;p&gt;I would like to know other tha neural network, is there any ML technique for agent based ML. If so how to train an agent with some predefined rules? Can we use python programming for representing those rules? &lt;/p&gt;&#xA;" OwnerUserId="15631" LastActivityDate="2018-05-14T17:48:43.027" Title="Can agent based machine learning achieved with any ML algorithms other than neural network?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;unsupervised-learning&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6409" PostTypeId="2" ParentId="6402" CreationDate="2018-05-14T15:45:27.067" Score="5" Body="&lt;p&gt;First of all, you need to realise that you will not be able to do it. Google is a multi-billion dollar company, with a large number of very bright and well-funded researchers. That tells me that it is not something a single person can do by themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you already have some pre-conceptions about it. You want to use a machine learning approach, using neural networks. I would start with a blank slate and think about the problem first, before considering the tools I would want to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know were to start, I would suggest you have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;noreferrer&quot;&gt;Eliza&lt;/a&gt;, which is old but still very hard to improve on. There are many open source implementations of it available, and it wouldn't be too hard to implement your own from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eliza uses a symbolic approach (ie no neural networks), which has the big advantage that you can see what is going on, and you can actually understand how it works. Have a play around with it, create some chatbots, and see how you can make the conversations sound more human-like. Then, when you have a clearer idea about this, proceed to other frameworks such as wit.ai or Microsoft's LUIS. There you trade in control for ease of use; but I suggest you don't do this without understanding what you're actually doing first.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-14T15:45:27.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6410" PostTypeId="2" ParentId="6407" CreationDate="2018-05-14T16:17:02.010" Score="2" Body="&lt;p&gt;Neural networks are not &lt;em&gt;inherently&lt;/em&gt; part of reinforcement learning (a popular agent-based framework for describing control problems). In general, if you have an agent-based scenario, you are trying to optimise a function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Policy( State ) -&amp;gt; Action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;State&lt;/code&gt; can be any combination of current observations and history that seem relevant to the problem. The optimisation is usually over some measure of success, achieving goals etc. Reinforcement learning formalises these terms using &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Processes&lt;/a&gt;, which is a very general and successful approach, but not the only one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the best policy does not &lt;em&gt;require&lt;/em&gt; neural networks. There are a lot of basic reinforcement learning algorithms that are defined without reference to NNs. For instance, &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning&lt;/a&gt; does not need neural networks, they are an optional extra. In addition, you do not need to even use RL - you can try to search for a suitable Policy function directly using a method such as Genetic Algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Focusing more on RL methods, the usual approach without any kind of function approximator is to just store an estimate of the &lt;em&gt;value&lt;/em&gt; of each state or state/action pair. This value is a measure of long term reward, and has a formal definition in RL. Often called the &lt;em&gt;tabular&lt;/em&gt; approach , because it is just a table of states and their estimated values, it works for small problems just fine. For example you can train an agent to play tic tac toe, find an optimal path that moves over a grid, or discover when to hold and when to twist in a simplified blackjack game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;However&lt;/em&gt;, problems start to occur with the most basic methods when the state space becomes large, which happens very easily. Tabular methods are OK up to a million or maybe ten million states (depending on how easy it is to take trial actions). These are &lt;em&gt;enumerated&lt;/em&gt; states, so typically adding a single dimension to a problem multiplies the number of possible states by the number of options in that new dimension. So after a certain level of complexity, a simple table is not good enough, and you need a function approximator. Function approximators for RL ideally have the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Can generalise from examples.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can be updated online as new data arrives, progressively forgetting old data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can be differentiated with respect to their learning parameters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Neural networks fit the bill. Some other popular supervised learning techniques do not - for instance Random Forests cannot usually be trained online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear regression&lt;/a&gt; does work, and it is a very simple approach. For some problems, linear regression works very well in combination with reinforcement learning. You may need to do some careful feature engineering, and it is not always possible for more complex problems, but if you have a &quot;medium-sized&quot; agent-based problem - simpler than learning Go or playing video games - and you want to avoid using neural networks, then you should be able to attempt to train an agent using a combination of RL, such as Q-learning, plus linear regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both Q-learning and linear regression have plenty of examples available online, and can be implemented from scratch using basic Python frameworks such as NumPy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-05-14T16:17:02.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6411" PostTypeId="1" CreationDate="2018-05-14T17:47:23.333" Score="1" ViewCount="198" Body="&lt;p&gt;I want to experiment with capsule networks on facial expression recognition (FER). For now, I am using fer2013 Kaggle dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that I didn't understand in capsule networks was in the first convolution layer, the size was reduced to 20x20 - having input image as 28x28 and filters as 9x9 with 1 stride. But, in the capsules, the size reduces to 6x6. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How did this happen? Because with the input size as 20x20 and filters as 9x9 and 2 strides, I couldn't get 6x6. Maybe I missed something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my experiment, the input size image is 48x48. Should I use the same hyperparameters for the start or are there any suggested hyperparameters that I can use?&lt;/p&gt;&#xA;" OwnerUserId="15633" LastEditorUserId="2444" LastEditDate="2020-06-09T11:12:06.277" LastActivityDate="2021-03-06T13:02:15.223" Title="Why does the size reduce to $6 \times 6$ in the capsule networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;capsule-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6413" PostTypeId="1" CreationDate="2018-05-14T18:19:13.363" Score="2" ViewCount="92" Body="&lt;p&gt;I am having difficulties wrapping my head around how the answer is being produced. How does the solution come up with the answer 3? How does one derive the answer 3 from the truth table? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example the following file :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TELL  p2=&gt; p3; p3 =&gt; p1;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;ASK p2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Produces the following result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UCQzO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UCQzO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Standard output is an answer of the form YES or NO, depending on&#xA;  whether the ASK(ed) query q follows from the TELL(ed) knowledge base&#xA;  KB. When the method is TT and the answer is YES, it should be followed&#xA;  by a colon (:) and the number of models of KB&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15635" LastActivityDate="2018-08-13T12:02:16.183" Title="AI Knowledge Based System: I am having difficulties understanding proposition logic" Tags="&lt;ai-design&gt;&lt;ai-basics&gt;&lt;logic&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6414" PostTypeId="1" AcceptedAnswerId="6458" CreationDate="2018-05-14T20:43:22.980" Score="3" ViewCount="96" Body="&lt;p&gt;I am looking to extract the central theme from a given news headline using NLP or text mining. Is there any reference that goes in this direction?&lt;/p&gt;&#xA;&lt;p&gt;Here's an example. Let's say that I have the following news headline.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BRIEF-Dynasil Corporation Of America Reports Q2 EPS Of $0.08&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then the algorithm should produce&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Reports&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's another example. The input is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;China's night-owl retail investors leverage up to dominate oil futures trade&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And the output would e.g. be&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;oil futures&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15638" LastEditorUserId="2444" LastEditDate="2021-02-03T22:57:02.683" LastActivityDate="2021-02-03T22:57:02.683" Title="Are there any references of NLP/text mining techniques for identifying the theme of news headlines?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6415" PostTypeId="2" ParentId="5571" CreationDate="2018-05-14T22:40:47.527" Score="1" Body="&lt;p&gt;In the case where the reward is undiscounted, there is no guarantee of convergence as the iteration procedure is not a strict contraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I can't find the math mode on the ai stackexchange so my answer can't be very precise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But an easy example is the following: Take a 'running' reward R of 0 to make things simpler, and a MDP with two states a and b. Take a transition matrix with 0's on the diagonal and 1 off diagonal. You will see that the algorithm will always flip the values of V(a) and V(b), and hence no convergence.&lt;/p&gt;&#xA;" OwnerUserId="15639" LastActivityDate="2018-05-14T22:40:47.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6416" PostTypeId="1" CreationDate="2018-05-15T07:30:12.763" Score="-1" ViewCount="159" Body="&lt;p&gt;I would like to write a program that takes a number of elements as input (some images and some strings) and outputs a design (the program places the images and strings on a canvas, but also styles the strings).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me in the right direction of a possible solution to this problem? I've looked for machine learning algorithms for design but have not found a single one to give me any tips on how to build one of these.&lt;/p&gt;&#xA;" OwnerUserId="15647" LastEditorUserId="1671" LastEditDate="2018-05-15T17:59:23.363" LastActivityDate="2018-05-16T10:07:15.060" Title="Design ads and posters with Machine Learning" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;getting-started&gt;&lt;software-evaluation&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="6417" PostTypeId="2" ParentId="6413" CreationDate="2018-05-15T07:59:12.723" Score="-1" Body="&lt;p&gt;I'm not quite sure I'm getting this right, as you don't go into much detail in your question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You show two truth tables for &quot;p implies q&quot;, or &quot;p =&gt; q&quot;. (your variable names are p2 and p3, and p3 and p1 respectively). In a truth table you have a list of all possible combinations of values for the variables. With two boolean values you have four possible combinations, TT/TF/FT/FF (T is represented by 1, and F by 0 in your tables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rule for '=&gt;' is that it is true unless the precedent is false and the output is true. For example, if we use &lt;em&gt;it rains&lt;/em&gt; for p, and &lt;em&gt;the road is wet&lt;/em&gt; for q, we get the following table:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;0 0&lt;/strong&gt; IF it DOES NOT rain THEN the road is NOT wet (true)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;0 1&lt;/strong&gt; IF it DOES NOT rain THEN the road is wet (true)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;1 0&lt;/strong&gt; IF it rains THEN the road is NOT wet (false)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;1 1&lt;/strong&gt; IF it rains THEN the road is wet (true)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This mirrors your truth tables above. Even though it might seem counter intuitive, only line 3 is false (0). Line 1 and 4 are obvious, and line 2 states that even if it does not rain, the road could still be wet (by someone using a hose pipe to make it wet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the implication, 3 of the possible 4 combinations result in a true outcome. Which is why you get &quot;Yes: 3&quot;.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-15T07:59:12.723" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="6418" PostTypeId="2" ParentId="4965" CreationDate="2018-05-15T08:09:51.907" Score="2" Body="&lt;p&gt;The easiest way to add some sort of structural similarity measure is to use n-grams; in your case bigrams might be sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go through each sentence and collect pairs of words, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;python is&quot;, &quot;is a&quot;, &quot;a good&quot;, &quot;good language&quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Your other sentence has&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;language a&quot;, &quot;a good&quot;, &quot;good python&quot;, &quot;python is&quot;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Out of eight bigrams you have two which are the same (&quot;python is&quot; and &quot;a good&quot;), so you could say that the structural similarity is 2/8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you can also be more flexible if you already know that two words are semantically related. If you want to say that &lt;em&gt;Python is a good language&lt;/em&gt; is structurally similar/identical to &lt;em&gt;Java is a great language&lt;/em&gt;, then you could add that to the comparison so that you effectively process &quot;[PROG_LANG] is a [POSITIVE-ADJ] language&quot;, or something similar.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-15T08:09:51.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6419" PostTypeId="2" ParentId="6413" CreationDate="2018-05-15T09:15:06.857" Score="1" Body="&lt;p&gt;The provided result is a non-sense for this input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a closed world paradigm (ie: prolog), where all non-provable facts are assumed as false, from &quot;p2-&gt;p3 and p3-&gt;p1&quot;, p2 is false, and the program result should be &quot;p2 ? No&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an open world, as it is not possible to proof &quot;p2&quot; nor &quot;not p2&quot;, the result must be &quot;p2 ? unknown&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;p2 -&gt; p3 and p3 -&gt; p1&quot; doesn't provides any information about &quot;p2&quot; in the same way that &quot;if it is sunny then I'm happy&quot; doesn't provides information about if it is sunny or raining.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The meaning of the number after the &quot;yes&quot; (the number of &quot;models&quot;) is unknown. Even present two true tables for a single fact &quot;p2 -&gt; p3 and p3 -&gt; p1&quot; can be considered an error.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-05-15T10:07:10.500" LastActivityDate="2018-05-15T10:07:10.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6421" PostTypeId="1" AcceptedAnswerId="6428" CreationDate="2018-05-15T11:28:24.003" Score="3" ViewCount="271" Body="&lt;p&gt;It is well known from the history of technology that the invention of new things was always problematic. In the 15th century for example, in which Gutenberg has invented the first printing press, the world wasn't pleased. Instead the Luddite movement was doing everything to destroy his work. As far as I know from the history lesson, Gutenberg was recognized in his time as an evil sorcerer and the printing press as work of the devil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This development was in later decades also visible. At first, a great invention was done for example the first steam driven car, and the ordinary people don't understand the technology and were in fear of it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A modern form of technology is computing and especially artificial intelligence. From a technical point of view, it is one of the most important inventions ever, and this might result into a very strong possible form of rejection. Some people in the world are not excited by Artificial Intelligence. They not want any sorts of robots and intelligent machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terminology itself is well known. The fundamental rejection of new technology because of religious or moral reasons is called Luddism or &lt;a href=&quot;https://en.wikipedia.org/wiki/Neo-Luddism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neoluddism&lt;/a&gt;. Because the technophobic Ned Ludd has destroyed a while ago two stocking frames. After this episode, every rant against technology is called after him. But what I do not understand the motivation behind it. Did Ned Ludd thought, that he can change the world if he destroyed a machine? Did he believe that mankind will become good if no Gutenberg printing press are used? The problem is that, for example, if the first steam engine was never invented, also the following inventions like the internet and intelligent machines wouldn't have been invented. But what would be the alternative? What is the perspective of Ned Ludd, how does he see the better tomorrow, if no technology innovation is allowed?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="2444" LastEditDate="2019-06-15T20:06:04.353" LastActivityDate="2019-06-16T08:08:27.683" Title="What is neoluddism?" Tags="&lt;philosophy&gt;&lt;history&gt;&lt;neo-luddism&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6422" PostTypeId="1" AcceptedAnswerId="6457" CreationDate="2018-05-15T12:13:01.280" Score="1" ViewCount="107" Body="&lt;p&gt;I am trying a modification of Mobilenet in which I add feedback from the softmax layer into the early layers (to implement this I put a second net after the first, which receives connections from the softmax layer of the first, the pretrained weights being non trainable). The idea was to mimic the massive feedback projections in the brain, which presumably could help object recognition by enhancing specific filters and inhibiting others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took the pretrained network from Keras and started to retrain it on Imagenet. I noticed that the training accuraccy increased right in the first epoch. My computer is very slow thus I cannot train for too long, an epoch takes 3.5 days. So after an epoch I tried the validation set, but instead the accuracy went down to almost half that of the pretrained values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is if this is and obvious case of overfitting. That is, will continued training increase the accuracy of the training set at the expense of the validation set, or is this a normal behavior expected at the initial stages of training, so that if I keep training for a few more epochs I could expect the validation set accuracy to go eventually up? Any ideas that could help are welcomed. &lt;/p&gt;&#xA;" OwnerUserId="5911" LastActivityDate="2018-05-18T15:12:15.290" Title="Is this overfitting avoidable?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6424" PostTypeId="1" CreationDate="2018-05-15T13:06:11.553" Score="1" ViewCount="27" Body="&lt;p&gt;I use Matlab function &quot;wmulden&quot; to obtain a denoised data and I realised that when I add only one data-step to previous data and I recalculate the denoised series all denoised data is changed even if the data is identical.&#xA;The average difference between the two denoised signals is 1*10e-03 and the average data  is 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to get the same denoising when the data is the same?&lt;/p&gt;&#xA;" OwnerUserId="15653" LastActivityDate="2018-05-15T13:06:11.553" Title="Multivariate Wavelet denoising problem" Tags="&lt;matlab&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6425" PostTypeId="1" CreationDate="2018-05-15T13:08:35.417" Score="4" ViewCount="117" Body="&lt;p&gt;I have a data set containing actions taken by customers (e.g., view a product, add a product to cart, purchase product), the product bought (if any) and times of said actions. I am attempting to use K-means clustering to identify the customers who are more likely to purchase a product based on these actions (minus the purchase).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently clustering using: the number of products viewed, the number of products put in the cart, the mean time between the actions, the variance of the time between the actions, the standard deviation of the time between the actions (all of these values are normalized), as well as the product purchased (if any). The clusters I'm getting contain &lt;code&gt;~10%&lt;/code&gt; buyers and &lt;code&gt;90%&lt;/code&gt; non-buyers, but I'm trying to separate buyers and non-buyers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts on what else I can do? Or should I try another method completely?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Illustration: x axis are the clusters, y axis is the number of customers, red are buyers and blue are non-buyers&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jMzss.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jMzss.png&quot; alt=&quot;cluster graph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I made a 3D  graph showcasing the clusters, the amount of customers and the mean time between actions (normalized because of reasons)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Fymmf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fymmf.png&quot; alt=&quot;3D plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet another update: customers (not grouped by cluster, just as is) according to the average number of products they viewed and the average time between actions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gr3n3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gr3n3.png&quot; alt=&quot;number of products x average time&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took some advice and tried using PCA (from this &lt;a href=&quot;https://towardsdatascience.com/pca-using-python-scikit-learn-e653f8989e60&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;), and these are the results I got:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wB7Lh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wB7Lh.png&quot; alt=&quot;PCA data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The raw data (x=number of items viewed/carted, y=average time between interactions)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TMHUj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TMHUj.png&quot; alt=&quot;raw data graph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any tips on how to cluster this mess?&lt;/p&gt;&#xA;" OwnerUserId="12940" LastEditorUserId="22296" LastEditDate="2019-02-21T01:50:40.580" LastActivityDate="2019-02-21T01:50:40.580" Title="Supervised K-means clustering doesn't appear to work" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;python&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6426" PostTypeId="1" CreationDate="2018-05-15T14:06:41.507" Score="17" ViewCount="19763" Body="&lt;p&gt;I have read various answers to this question at different places, but I am still missing something. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have understood is that a graph search holds a closed list, with all expanded nodes, so they don't get explored again. However, if you apply breadth-first-search or uniformed-cost search at a search tree, you do the same. You have to keep the expanded nodes in memory. &lt;/p&gt;&#xA;" OwnerUserId="15391" LastEditorUserId="2444" LastEditDate="2019-11-10T17:28:42.713" LastActivityDate="2021-05-15T12:42:48.387" Title="What is the difference between tree search and graph search?" Tags="&lt;comparison&gt;&lt;definitions&gt;&lt;search&gt;&lt;graph-search&gt;&lt;tree-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6428" PostTypeId="2" ParentId="6421" CreationDate="2018-05-15T18:48:03.903" Score="3" Body="&lt;p&gt;My understanding of Neo-Luddism is that it is concerned with the unforeseeable effects of technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;blackening of London&quot; (see Blake's &lt;a href=&quot;https://www.poetryfoundation.org/poems/43673/london-56d222777e969&quot; rel=&quot;nofollow noreferrer&quot;&gt;London&lt;/a&gt;) in the early industrial era was an unforeseen effect, and would have had impacts on health related to air quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The unforeseen effect of heavy use of plastic materials has led not only to a large amount of plastics in the ocean, but to recent revelation of &lt;a href=&quot;https://www.theguardian.com/environment/2017/sep/06/plastic-fibres-found-tap-water-around-world-study-reveals&quot; rel=&quot;nofollow noreferrer&quot;&gt;micro-particles in the drinking water supply&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heavy use of nitrogen fertilizers has lead to dead zones (oxygen depleted water) in coastal zones, with real impacts on the food chain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Radioactive_waste#Legacy_waste&quot; rel=&quot;nofollow noreferrer&quot;&gt;Management of nuclear waste&lt;/a&gt; is another issue that was overlooked at the beginning, before there was a real understanding of the effects on organisms, including humans.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The original Luddites were attacking technology because it was clear it would eliminate jobs. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Luddite&quot; rel=&quot;nofollow noreferrer&quot;&gt;Luddite wiki&lt;/a&gt; states that &quot;It is a misconception that the Luddites protested against the machinery itself in an attempt to halt the progress of technology,&quot; rather &quot;Luddites feared that the time spent learning the skills of their craft would go to waste as machines would replace their role in the industry.&quot; This is consistent with my understanding.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The extension of the idea of Neo-Luddism, as raised by Hawking and Musk, among others, is not a rejection of technology, but a warning about the inability to predict the dangers of new technology. They seem to be, in part, warning about the potential dangers of superintelligence.   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The fears regarding superintelligence, or, self-replicating nanobots (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Grey_goo&quot; rel=&quot;nofollow noreferrer&quot;&gt;grey goo&lt;/a&gt;) are similar to concerns regarding nuclear weapons and mutually assured destruction.  (&lt;a href=&quot;https://en.wikipedia.org/wiki/Mutual_assured_destruction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nash explains why deterrence works&lt;/a&gt;, but it assumes rational actors.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, the more concrete warnings about strong-narrow AI &lt;em&gt;(AlphaGo and extensions)&lt;/em&gt; is that it may result in unfathomable levels of persistent, long-term unemployment as the types of tasks strong-narrow AI can do better than humans will expand, likely in an exponential function.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I always like to take it back to the origins, so I'll mention Prometheus (&quot;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0058%3Aentry%3D*promhqeu%2Fs&quot; rel=&quot;nofollow noreferrer&quot;&gt;forethought&lt;/a&gt;&quot;) and his brother Epimetheus (&quot;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0058%3Aentry%3D*)epimhqeu%2Fs&quot; rel=&quot;nofollow noreferrer&quot;&gt;hindsight&lt;/a&gt;&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prometheus saves mankind with gifts, including fire, but fire is both creative and destructive.  In the myths, the punishments to mankind for Prometheus' theft are sent by Zeus, because the myth, and the core ideas, predate modern science and philosophy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple reductionist explanation of Neo-Luddism may be simply that &quot;hindsight is 20/20&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forge ahead blindly at your own peril.  Proceed cautiously to minimize potential downside.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In some sense, practical Neo-Luddism--not the rejection technology the cognizance of the dangers of technology--may just be a form of minimax.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1847" LastEditDate="2019-06-16T08:08:27.683" LastActivityDate="2019-06-16T08:08:27.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6429" PostTypeId="1" CreationDate="2018-05-15T19:44:19.023" Score="11" ViewCount="1102" Body="&lt;p&gt;I know that one of the recent fads right now is to train a neural network to generate screenplays and new episodes of e.g. the Friends or The Simpsons, and that's fine: it's interesting and might be the necessary first steps toward making programs that can actually generate sensible/understandable stories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, can neural networks be trained specifically to study the structures of stories, or screenplays, and perhaps generate plot points, or steps in the Hero's Journey, etc., effectively writing an outline for a story?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, this differs from the many myriad plot-point generators online, although I have to admit the similarities.  I'm just curious if the tech or the implementation is even there yet and, if it is, how one might go about doing it.&lt;/p&gt;&#xA;" OwnerUserId="15659" LastEditorUserId="2444" LastEditDate="2019-06-02T15:42:20.790" LastActivityDate="2020-04-04T18:23:02.247" Title="Can an AI be trained to generate the outline of a story?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;applications&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6430" PostTypeId="2" ParentId="6429" CreationDate="2018-05-15T21:34:51.893" Score="1" Body="&lt;p&gt;As far as I am aware, this has not been done yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see several problems with this. A neural network is basically a classifier, which matches an input to an output. Both input and output are usually numerical values, though they could be matched to concepts or words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To train a NN you provide an appropriately encoded input, and the corresponding output. The NN learns the associations between the two, and can then classify unseen input accordingly. This has recently been used to transform images in a particular style etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would the input and output be for generating screenplays? You could use previous scripts as inputs, but what would the output be? It could be narrative 'moves' of some sort, perhaps. So you could train an NN to recognise narrative elements from screenplays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you are still not creating anything, but just recognising stuff. You would need some other input. I guess you could train an NN on &quot;The Simpsons&quot;, get a narrative structure, and then present it with an Episode of &quot;Friends&quot; and see what happens. It won't be a new episode of a screenplay, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other way round might work: you feed it narrative moves (a kind of story skeleton), and get a script out. But it would need a lot of (human) post-editing to be at all useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think an NN is the wrong tool to use here. There has been work done with generating stories and screenplays, even way back in the early days of AI. But that was all based on symbolic AI, not on the kind of ML which seems to currently be &lt;em&gt;en vogue&lt;/em&gt;. Have a look at &lt;a href=&quot;https://www.jamesryan.world/projects#/storygen-history/&quot; rel=&quot;nofollow noreferrer&quot;&gt;James Ryan's website&lt;/a&gt;; he has recently written an overview over historic approaches to story (and screenplay) generation.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-15T21:34:51.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6431" PostTypeId="1" AcceptedAnswerId="6432" CreationDate="2018-05-16T03:51:46.383" Score="1" ViewCount="67" Body="&lt;p&gt;I have a medical dataset with 14000 rows dataset with 900 attributes. I have to predict disease severity using that. I would like to know whether we can write rules in python language for training an agent for medical diagnostic using machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can an agent make the decisions by the rules coded in python and that agent get trained with some machine learning algorithms? If so is there any agent architecture and model for the agent which is good in this context?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt; By the rule, I meant something like this..&quot;if x&gt;y output z as action&quot;. By the word &quot;Training&quot; I meant &quot;how to tell this agent to do this action&quot;?&lt;/p&gt;&#xA;" OwnerUserId="15631" LastEditorUserId="15631" LastEditDate="2018-05-16T07:09:22.157" LastActivityDate="2018-05-16T07:09:22.157" Title="Can we code rules for an agent in python language other than predicate calculus?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;datasets&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6432" PostTypeId="2" ParentId="6431" CreationDate="2018-05-16T06:42:30.850" Score="1" Body="&lt;p&gt;You could formulate the problem of predicting disease severity as a classification one, you give the algorithm those 900 attributes and their corresponding labels (severe/not severe) after training, you give it a new data point with just the 900 attributes, it returns the correct label severe or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an enormous number of algorithms in the ML literature for classification, some of them formulate the problem explicitly as learnable rules, i.e. let the machine figure out the rules given the attributes.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;ftp://ftp.boulder.ibm.com/software/analytics/spss/support/Stats/Docs/Statistics/Algorithms/14.0/TREE-CART.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classification and Regression Trees by Breiman et al (1984)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Random_forest&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random forest by Ho in 1995&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://machinelearningmastery.com/gentle-introduction-xgboost-applied-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;XGBoost&lt;/a&gt;  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-16T06:42:30.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6434" PostTypeId="1" CreationDate="2018-05-16T08:48:39.410" Score="0" ViewCount="395" Body="&lt;p&gt;Pretty simple question here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it useful to use the standard deviation, skew, kurtosis, or any other extrapolatory stats as features, and if so in which problem sets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, I am talking about deep learning problems.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2018-05-17T14:49:52.687" Title="Are standard deviation, variance, skew good features for ML?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6435" PostTypeId="1" CreationDate="2018-05-16T09:26:49.697" Score="6" ViewCount="159" Body="&lt;p&gt;I found that fully connected neural networks trained on identical data sets with identical hyper-parameters can have different performances or accuracies (7-8% of deviation). Is this an unusual phenomenon? What is the cause? What is an acceptable deviation?&lt;/p&gt;&#xA;" OwnerUserId="15668" LastEditorUserId="2444" LastEditDate="2019-12-17T23:17:13.353" LastActivityDate="2020-01-17T07:01:50.213" Title="Why do neural networks trained on identical datasets and with identical hyper-parameters have different performances?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="2" ClosedDate="2020-08-18T23:11:14.357" ContentLicense="CC BY-SA 4.0" />
  <row Id="6436" PostTypeId="2" ParentId="6435" CreationDate="2018-05-16T09:44:03.300" Score="-2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;but at different times  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;you have a variance problem, I suggest to study basic machine learning and the &lt;em&gt;bias-variance tradeoff&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The intuition:&lt;/strong&gt; the model has a large capacity with a little or no regularization, therefore the model has a large degree of freedom of fitting the data.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is an acceptable standard deviation level   &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;+/- 0.05 &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is that an unusual phenomenon  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;some insights from: &lt;a href=&quot;https://arxiv.org/pdf/1707.06799.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal Hyperparameters for Deep LSTM-Networks for Sequence&#xA;Labeling Tasks&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-16T09:44:03.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6437" PostTypeId="2" ParentId="6416" CreationDate="2018-05-16T10:07:15.060" Score="-1" Body="&lt;p&gt;If you have images contain styled strings and images contain printed strings, you could use&#xA;&lt;a href=&quot;https://arxiv.org/abs/1703.10593&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks&lt;/a&gt;  to learn to translate between both distributions where paired training data does not exist (not styled string image, styled string image).&lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-16T10:07:15.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6438" PostTypeId="2" ParentId="6329" CreationDate="2018-05-16T21:55:49.003" Score="1" Body="&lt;p&gt;The naive way is to generate connections randomly as you would for a cyclic graph, but then perform a test to reject any connections that form a cycle. This is the current approach in SharpNEAT and there has been some effort directed at improving the performance of the cycle test in the work-in-progress refactor branch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternative would be to track the depth of all nodes, store a list of node IDs sorted by depth, and sample the connection endpoint nodes in such a way that the target node depth is always higher than the source node. Now I think about it that's probably the better method.&lt;/p&gt;&#xA;" OwnerUserId="15693" LastActivityDate="2018-05-16T21:55:49.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6440" PostTypeId="2" ParentId="5318" CreationDate="2018-05-17T14:39:52.947" Score="0" Body="&lt;p&gt;I think that your misuse of the term &lt;em&gt;over-fitting&lt;/em&gt; made the question vague. In layman terms, over-fitting means that a model fails to generalize to real-world scenarios, but is accurate with the training set.&lt;/p&gt;&#xA;&lt;p&gt;Using a dropout layer means that the network cuts down on neurons that are used for training, in this case, 50%.&lt;/p&gt;&#xA;&lt;p&gt;Recommendations for improving training accuracy would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Transfer learning&lt;/li&gt;&#xA;&lt;li&gt;Adding more layers to the network (also shifting number of neurons helps)&lt;/li&gt;&#xA;&lt;li&gt;Adding epochs&lt;/li&gt;&#xA;&lt;li&gt;Changing optimizer (Adam and RMSProp are some of my suggestions)&lt;/li&gt;&#xA;&lt;li&gt;Adding activation layers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15465" LastEditorUserId="2444" LastEditDate="2020-10-04T20:28:55.493" LastActivityDate="2020-10-04T20:28:55.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6441" PostTypeId="2" ParentId="6434" CreationDate="2018-05-17T14:49:52.687" Score="1" Body="&lt;p&gt;I would say it is useful if you have an extensive knowledge in the domain you want to apply your model in. You also need more data for it to yield reasonable results.&lt;br&gt;&#xA;As for real world uses l can only think of trading at the moment.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-05-17T14:49:52.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6443" PostTypeId="1" CreationDate="2018-05-17T16:16:59.583" Score="2" ViewCount="97" Body="&lt;p&gt;Considering the scenario where supervised training data-set in the form of sentence will be given to train the machine&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Bomb which had been planted by Terrorist on this morning was&#xA;  defused by the Counter Terrorist on joining hands with the&#xA;  Intelligence Force&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Input strings in the sentence containing each words are broken into tokenised&#xA;arrays of single words with stop words removed.&lt;br/&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each word in the given sentence gets assigned a label w1, w2 and so on i.e,&lt;br/&gt;&#xA;&lt;code&gt;w2&lt;/code&gt;  = Bomb&lt;br/&gt;&#xA;&lt;code&gt;w6&lt;/code&gt;  = planted&lt;br/&gt;&#xA;&lt;code&gt;w13&lt;/code&gt; = defused&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating the scores for individual word combinations, the result should yield something like:&lt;br/&gt;&#xA;&lt;code&gt;w2.w6&lt;/code&gt;  = Scores should be Positive (or &gt; some threshold value)&lt;br/&gt;&#xA;&lt;code&gt;w2.w13&lt;/code&gt; = Scores should be Negative (or &amp;lt; some threshold value)&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of words with polarity changers&lt;br/&gt;&#xA;Eg.: &lt;em&gt;Bomb wasn't/haven't/didn't got defused.&lt;/em&gt;&lt;br/&gt;&#xA;The resulting scores should be positive&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To accomplish this task I had implemented Sentiment Analysis with the threshold = 2.5 and ended up with the following scores&lt;br/&gt;&#xA;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Actual Output:&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&amp;lt; 2.5 : Low&lt;br/&gt;&#xA;  = 2.5 : Neutral&lt;br/&gt;&#xA;  &gt; 2.5 : High&lt;br/&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Expected Output:&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Case 1:&lt;/em&gt; score = negative, since that bomb was defused or removed in the given sentence&lt;br/&gt;&#xA;  &lt;em&gt;Case 2:&lt;/em&gt; score = positive, vice versa of &quot;Case 1&quot;&lt;br/&gt;&#xA;  &lt;em&gt;Case 3&lt;/em&gt;: Otherwise score = 0, in case it can't predict either of the above two cases, it should be neutral &lt;br/&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am facing a severe problem every time I need to update the vocabulary list with upcoming new words that were not in the dictionary list, which is turning out to be Semi-supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Referring to the above sentence to calculate the &lt;code&gt;w(n-(1/2/3/...n)&lt;/code&gt; and &lt;code&gt;wn&lt;/code&gt; word with reference to word = &lt;em&gt;Bomb&lt;/em&gt;. The final resulting score should yield as negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So which machine learning algorithm would be appropriate that fits to yield a better solution and based on the given data set how will I train the machine to learn the above things?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally should I try to implement by keeping the model persistence. So that it doesn’t have to be trained on each run.&lt;/p&gt;&#xA;" OwnerUserId="15703" LastEditorUserId="2193" LastEditDate="2018-05-18T09:31:15.230" LastActivityDate="2018-05-18T10:30:53.647" Title="Which machine learning algorithm is suitable for detecting text w.r.t set of words" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6444" PostTypeId="2" ParentId="6429" CreationDate="2018-05-18T00:20:03.817" Score="-1" Body="&lt;p&gt;As far as I know, there isn't any system like you describe yet. However, there are some interesting approaches to narrative intelligence that can be found at the University of New Orleans Narrative Intelligence Lab site: &lt;a href=&quot;https://nil.cs.uno.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nil.cs.uno.edu/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully those can be helpful in guiding a deep-learning approach to narrative generation problems.&lt;/p&gt;&#xA;" OwnerUserId="15711" LastActivityDate="2018-05-18T00:20:03.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6445" PostTypeId="1" CreationDate="2018-05-18T00:54:06.063" Score="0" ViewCount="65" Body="&lt;p&gt;For speedrunning purposes, I am trying to train a neural network to identify human-executable ways to manipulate pseudo-RNG (in Pokemon Red, for the interested). The game runs at sixty frames per second, and the linear-congruential PRNG updates every frame, while many frames are unlikely to be relevant to the manipulation (and so should contain no actions from the neural net). Any given manipulation is likely to last 30sec-2min, and the advancement rate of the PRNG can change depending on location in the game-world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some experience with coding AI/deep-learning. I've made some programs using Multilayer Perceptron and IndRNN approaches. From what I can tell, IndRNN or A3C would be my best bets. I'm not expert enough to know the correct approach, though, or to know if the dimensionality of the problem makes it outright unfeasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is this problem reasonably solvable with NN/deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What approach would you recommend to tackle it?&lt;/p&gt;&#xA;" OwnerUserId="15711" LastEditorUserId="2193" LastEditDate="2018-05-18T13:18:48.840" LastActivityDate="2018-05-18T17:01:40.630" Title="Teaching a NN to manipulate pseudoRNG over a long time scale?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;recurrent-neural-networks&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6446" PostTypeId="1" CreationDate="2018-05-18T00:57:01.580" Score="3" ViewCount="689" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1701.09175v8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Skip connections eliminate singularities&lt;/a&gt; explains the use of skip connections to break the singularity in deep networks, but I have not fully understood what a &lt;strong&gt;singularity&lt;/strong&gt; is.&lt;/p&gt;&#xA;&lt;p&gt;Any easy-understanding explanation?&lt;/p&gt;&#xA;" OwnerUserId="10569" LastEditorUserId="2444" LastEditDate="2021-03-29T15:51:52.037" LastActivityDate="2021-03-29T15:51:52.037" Title="What's the definition of &quot;singularity&quot; in the context of neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6447" PostTypeId="2" ParentId="4832" CreationDate="2018-05-18T03:57:56.320" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sequential_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequential programming&lt;/a&gt; would not be suitable for this kind of problem, but an algorithm could be implemented in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Declarative_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;declarative programming language&lt;/a&gt;. I would suggest using &lt;a href=&quot;https://en.wikipedia.org/wiki/Answer_set_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Answer Set Programming&lt;/a&gt;, a language that is designed for &lt;a href=&quot;https://en.wikipedia.org/wiki/Logic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;logic axioms&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9983" LastEditorUserId="1671" LastEditDate="2018-05-18T17:08:38.880" LastActivityDate="2018-05-18T17:08:38.880" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6449" PostTypeId="2" ParentId="6445" CreationDate="2018-05-18T04:02:59.533" Score="1" Body="&lt;p&gt;The point of pseudoRNG is to be unmodable and unpredictable, making it hard to train an AI to learn. It would more likely be useful and more efficient to have the equation that the game uses for generation available, so that you can manually make the check, or to just have a list of the loop if the pseudoRNG is based on the time elapsed.&lt;/p&gt;&#xA;" OwnerUserId="6989" LastEditorUserId="2193" LastEditDate="2018-05-18T17:01:40.630" LastActivityDate="2018-05-18T17:01:40.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6452" PostTypeId="1" CreationDate="2018-05-18T07:15:15.013" Score="1" ViewCount="44" Body="&lt;p&gt;This article &quot;&lt;a href=&quot;https://ieeexplore.ieee.org/document/6698290/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enhancing Differential Evolution Utilizing Eigenvector-Based Crossover Operator&lt;/a&gt;&quot; said for a non-separable function traditional crossover algorithm are not suitable and they can not diversify the population sufficiently, so the differential evolution stops at the local optimum points. Why does this behavior not hold for separable functions but it exists for non-separable functions? Which key feature of the non-separable functions cause this behavior?&lt;/p&gt;&#xA;" OwnerUserId="15714" LastActivityDate="2018-05-18T07:15:15.013" Title="Crossover in differential evolution for separable and non-separable functions" Tags="&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6453" PostTypeId="1" AcceptedAnswerId="6455" CreationDate="2018-05-18T08:26:11.360" Score="0" ViewCount="52" Body="&lt;p&gt;I am trying to assess an encoder in my autoencoder. I can not seem to grasp which specs make an encoder better than other one in, lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats, so that when I provide a picture of a bird, my autoencoder would tell me that it is not a picture of a cat. I am trying to understand what exact specs make my encoder (and decoder) better? I understand it is all about chosen weights but is it possible to be more specific? &lt;/p&gt;&#xA;" OwnerUserId="14863" LastActivityDate="2018-05-18T12:08:05.157" Title="What are good parameters of an encoder?" Tags="&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6454" PostTypeId="2" ParentId="6443" CreationDate="2018-05-18T10:30:53.647" Score="1" Body="&lt;p&gt;If your main issue is dealing with new vocabulary, you could try using a parts-of-speech tagger as a pre-processing step. You would then effectively discover relationships between &quot;noun&quot; and &quot;verb&quot;, which does not change with new words. Taggers usually can handle unknown words by using contextual information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you'd tag the words with their word class labels, and use those for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 'application mode' you use p-o-s tags again, calculate the scores, and then map them back onto your original word tokens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does of course loose you some information, as you're only dealing with N-V, rather than &lt;em&gt;bomb&lt;/em&gt;-&lt;em&gt;plant&lt;/em&gt; or &lt;em&gt;bomb&lt;/em&gt;-&lt;em&gt;defuse&lt;/em&gt;. To solve this I'd use a hybrid approach: for your known vocabulary you use the word tokens, whereas for unknown words you fall back on tags. If you train two classifiers, one with tokens, one with tags, you have the tags one as a 'safety net' to handle out-of-vocab words.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-18T10:30:53.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6455" PostTypeId="2" ParentId="6453" CreationDate="2018-05-18T12:08:05.157" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I can not seem to grasp which specs make an encoder better than another one  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, in unsupervised settings, we want to learn the probability distribution of the data p(x) by some latent variables that explain the variations observed in the training set.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The autoencoder family (Variational, Denoising, Contrastive, Sparse) try to approximate p(x) so we have a performance metric to tell us how our model is doing.  e.g. (negative log likelihood of p(x))&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats,   &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you use some autoencoder model to learn the distribution of cats, you could use the encoder part to further augmented with a linear classifier to discriminate between cats and other categories Therefore you have &lt;strong&gt;an intrinsic task&lt;/strong&gt; (learn a good representation of the data distribution) and &lt;strong&gt;an extrinsic task&lt;/strong&gt; (learn to classify cat vs not a cat). So you could do a hyper-parameter search for the model that best suits your problem by measuring its accuracy on the extrinsic task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side note:&lt;br&gt;&#xA;GAN (Generative Adversarial Network) is a generative model, it provides some way of interacting less directly with this p(x) by drawing samples from it starting without any input. thus the situation here is different.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-18T12:08:05.157" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6456" PostTypeId="1" CreationDate="2018-05-18T15:00:41.750" Score="1" ViewCount="37" Body="&lt;p&gt;According to &lt;a href=&quot;https://arxiv.org/pdf/1606.00915.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper (page 4, bottom-right), atrous convolutions can be used to compute responses of arbitrarily large dimensions in Deep Convolutional Neural Networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand how something like this is true, since by upsampling the filters, one effectively can apply the filter less times to an image, unless one also upsamples the image. Applying the filter less times as I see it obviously means that the output (response) will be of lower dimensionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something that I am missing here?&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2018-05-18T15:00:41.750" Title="Atrous (Dilated) Convolution: How one can compute responses of arbitrarily high dimensions in DCNN?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6457" PostTypeId="2" ParentId="6422" CreationDate="2018-05-18T15:12:15.290" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;an epoch takes 3.5 days  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;First of all, use &lt;a href=&quot;http://colab.research.google.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;colab&lt;/a&gt; to iterate quickly, it offers unlimited 12 hours of free GPU.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;to retrain it on Imagenet.   &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That said, we could use complex models without being afraid of overfitting due to large training size.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;will continued training increases the accuracy of the training set at the expense of the validation set  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In many cases that's the case.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The idea was to mimic the massive feedback projections in the brain.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My suggestion is to read the third part of the &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeplearning book&lt;/a&gt; which includes Representation Learning, Structured Probabilistic Models for Deep Learning, Monte Carlo Methods, Confronting the Partition Function, Approximate Inference, Deep Generative Models.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Part III is the most important for a researcher—someone who wants to understand&#xA;  the breadth of perspectives that have been brought to the field of deep&#xA;  learning, and push the field forward towards true artificial intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2018-05-18T15:12:15.290" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6458" PostTypeId="2" ParentId="6414" CreationDate="2018-05-18T15:32:39.787" Score="0" Body="&lt;p&gt;You could formulate the problem as a topic classification task, hence you need labeled data.&lt;/p&gt;&#xA;&lt;p&gt;From an unsupervised point of view, you could represent sentences with some fixed feature vector (latent representation).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1511.06349&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generating Sentences from a Continuous Space&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cs.stanford.edu/%7Equocle/paragraph_vector.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paragraph2Vec&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BRIEF-Dynasil Corporation Of America &lt;strong&gt;Reports&lt;/strong&gt; Q2 EPS Of $0.08&lt;/p&gt;&#xA;&lt;p&gt;China's night-owl retail investors leverage up to dominate &lt;strong&gt;oil futures&lt;/strong&gt; trade&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Self-attention models would be very useful to this kind of problems since you don't need to encode all the context in the last hidden cell of some RNN model to classify to which theme the headline belongs.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-05-18T15:32:39.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6459" PostTypeId="1" CreationDate="2018-05-18T23:20:06.880" Score="3" ViewCount="445" Body="&lt;p&gt;In an RNN to train it, you need to roll it out, and enter in the history of inputs and the history of expected outcomes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't seem like a realistic picture of the brain since this would require, for example, for the brain to store a perfect history of every sense that comes in to it for many time-steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is there an alternative to RNNs that doesn't require this history? Perhaps storing differences or something? Or storing some accumulator?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps there is a way to calculate with RNNs that doesn't require keeping hold of this history?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2018-08-30T02:00:37.177" Title="Is there an alternative to RNNs that doesn't require knowing input history?" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6460" PostTypeId="1" CreationDate="2018-05-18T23:20:52.177" Score="4" ViewCount="77" Body="&lt;p&gt;I remember the first time hearing about google trying to make driverless cars.  That was YEARS ago!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These days, I'm beginning to learn about Neural Nets and other types of ML and I was wondering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know how many hours (or days, months, etch) is needed in training time to get the results that are now used in today's self-driving vehicles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am ASSUMING they use Neural networks for this...)&lt;/p&gt;&#xA;" OwnerUserId="15733" LastActivityDate="2018-08-17T20:55:53.717" Title="How long has it taken for autonomous driving cars to be being sold and used on the roads today?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6461" PostTypeId="1" CreationDate="2018-05-19T02:39:08.203" Score="5" ViewCount="969" Body="&lt;p&gt;What is the current research in artificial intelligence and machine learning in the field of data compression?&lt;/p&gt;&#xA;&lt;p&gt;I have done my research on the &lt;a href=&quot;https://en.wikipedia.org/wiki/PAQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAQ series of compressors&lt;/a&gt;, some of which use neural networks for context mixing.&lt;/p&gt;&#xA;" OwnerUserId="15736" LastEditorUserId="2444" LastEditDate="2021-01-21T02:03:37.283" LastActivityDate="2021-01-21T02:03:37.283" Title="What is the current research in artificial intelligence in the field of data compression?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;state-of-the-art&gt;&lt;data-compression&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6463" PostTypeId="2" ParentId="6446" CreationDate="2018-05-19T06:39:36.100" Score="3" Body="&lt;p&gt;Firstly, I invite you to take this answer with a grain of salt (and possibly suggest edits) as I am not specifically familiar with the issue of singularities myself. Also, I appologise for all the links being at the bottom of the answer, I am unable to reuse links throughout my answer due to my reputation on this particular exchange.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This appears to refer to singular points in Algebraic Geometry. At singular points, &lt;strong&gt;no singlar tagent&lt;/strong&gt; can be defined using the usual methods. This is as opposed to regular (i.e non-singular points), where one (and only one) tangent can be regularly defined.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The Easy Answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In simpler terms, and particularly with respect to neural networks, it refers to the points on a hypersurface (e.g. a line, plane, surface, etc) at which the gradient/slope is ambigious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've taken example of this from the wikipedia page on singular points.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JgjR3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JgjR3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above image, the hyperspace is a line. On this line, there is a singularity at (0,0), where the line crosses itself. At this point, there are two slopes (one going up, and the other going down). There is no way to choose between the slopes without applying arbatrary rules, so the gradient is ambigous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These singularities are problematic for some of the algorithms used in deep learning, such as Gradient Descent, which uses the the derivative of objective functions (e.g. loss functions) to search for the weights/parameters that yield the lowest error.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Gradient Descent In This Example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For anyone unfamiliar with gradient descent, the problem with singularaties can be understood with the following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say that the above line represents a path. Starting at the top right, you want to find the lowest point on the path. At any time, all you can know is the slope at the current point. You can turn left or right freely, but will never move upwards, so you will stop as soon as you reach the bottom of a valley.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially moving down and to the left, you will eventually reach the singularity and need to choose to go either left or right. Because you have no information about the path beyond your current location, you have no way of knowing which way leads to the bottom of the path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose at random, you have a 50% chance of getting stuck at in the valley at the bottom of the loop on the left of the path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could impose a rule like &quot;always follow the gradient most different to the current gradient&quot;, then you'd go right, and find the true lowest point. However, if the path was slightly different (e.g. if it curved up at (0.25, 0.25) this would be bad, as the correct path would then be on the left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, there is no way of knowing which way is better without a deep understanding of the hypersurface of your objective functions (and if you knew that much about the objective function, you likely wouldn't need to use machine learning in the first place).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Research/Resources&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I discovered this in order to answer this questions so, as justification, here is the process and resources I used to come to this conclusion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dug through the references in the paper you linked. Of particular relevance was &lt;a href=&quot;https://pdfs.semanticscholar.org/f37a/392eac50939f73fb5e6f5445fcf0dacb9449.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amari et al. (2006)&lt;/a&gt;, which mentions &lt;a href=&quot;https://en.wikipedia.org/wiki/Resolution_of_singularities#Hironaka&amp;#39;s_method_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hironaka’s&#xA;theorem of singularity resolution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the introductory paragraph, the wikipedia page mentions the concept of Non-Singularity Variety, and links to &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_point_of_an_algebraic_variety&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page on singular points in Algebraic Geometry&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example on singular points article is analogous to a 1d &lt;a href=&quot;https://en.wikipedia.org/wiki/Resolution_of_singularities#Hironaka&amp;#39;s_method_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;hypersurface&lt;/a&gt; version of many of the figures in Amari et al. (e.g. figures 1,2,4,5, and 6), and the discussion therein generally centers around the point at which the shapes converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My answer was composed based on this discovery process, as well my prior-knowledge of &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gradient-Descent&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15570" LastEditorUserId="15570" LastEditDate="2018-05-19T07:04:45.357" LastActivityDate="2018-05-19T07:04:45.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6464" PostTypeId="1" CreationDate="2018-05-19T06:46:02.830" Score="1" ViewCount="37" Body="&lt;p&gt;With all the Google I/O stuff coming out, how can I verify that I have an actual human on the phone using only voice? Are there still vocal things humans can, but robots can't do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conditions: the person on the phone is a stranger (so personal questions won't work), and the verification must be voice only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also, I understand Google Duplex may be just an overhyped demo that will turn out to flop like the Pixel Buds. But eventually such a bot would be created, right? If so, what's the best verification?)&lt;/p&gt;&#xA;" OwnerUserId="15743" LastActivityDate="2018-07-30T19:02:32.553" Title="&quot;Vocal captcha&quot; for robots on the phone?" Tags="&lt;human-like&gt;&lt;voice-recognition&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6465" PostTypeId="1" AcceptedAnswerId="6467" CreationDate="2018-05-19T07:49:02.423" Score="4" ViewCount="238" Body="&lt;p&gt;Let's say you want to do AI research and publish some papers just by your own. Would you send them to an AI journal using just your name? Which AI journals are recommended?&lt;/p&gt;&#xA;" OwnerDisplayName="user11604" LastActivityDate="2018-08-22T08:09:10.237" Title="How can you do AI research by your own?" Tags="&lt;research&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6467" PostTypeId="2" ParentId="6465" CreationDate="2018-05-19T09:03:11.017" Score="3" Body="&lt;p&gt;Let me preface this by acknowledging that this question is prone to opinion. This answer is, in so far as it is possible, primarily &lt;em&gt;observation&lt;/em&gt; based.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My understanding is that, when it comes to publishing in a journal, doing so as an individual (without backing from an institution) is, in general, going to be frowned upon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you may know: scientific research, including AI research, is generally subject to &lt;a href=&quot;https://en.wikipedia.org/wiki/Scholarly_peer_review&quot; rel=&quot;nofollow noreferrer&quot;&gt;peer review&lt;/a&gt;. When it comes to journals, this is mandatory. The review process attempts to enhance and preserve the integrety of the information published. As an additional safeguard, submissions will often expected to be backed by endorsement from someone within the academic community (typically an institution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, and as Pasaba correctly points out (in his comment on the OP), research and publishing are not neccessarily the same thing. Furthermore, not having an endorsement does not not stop you from making a contribution to the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you can publish code and/or articles on websites such as Github, and engage with communities of professional and hobbiest researchers around the web, (e.g. this Stack Exchange).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that there is also some scope for endorsement without being a direct member of a research institution. For example: arXiv, whilst not strictly a journal, is an open archive that supports endorsement by &lt;a href=&quot;https://arxiv.org/help/endorsement&quot; rel=&quot;nofollow noreferrer&quot;&gt;request&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing your circumstances, it's hard to know exactly what to do. However, my general advice is to find and engage with communities, and build a network of collaborative peers, rather than trying to succeed in an isolated fashion.&lt;/p&gt;&#xA;" OwnerUserId="15570" LastEditorUserId="15570" LastEditDate="2018-05-19T09:08:38.060" LastActivityDate="2018-05-19T09:08:38.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6468" PostTypeId="1" AcceptedAnswerId="6599" CreationDate="2018-05-19T16:41:50.253" Score="8" ViewCount="4514" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt; activation function is defined as follows &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = \operatorname{max}(0,x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the linear activation function is defined as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = x$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ReLU nonlinearity just clips the values less than 0 to 0 and passes everything else. Then why not to use a linear activation function instead, as it will pass all the gradient information during backpropagation? I do see that parametric ReLU (PReLU) does provide this possibility. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just want to know if there is a proper explanation to using ReLU as default or it is just based on observations that it performs better on the training sets.&lt;/p&gt;&#xA;" OwnerUserId="8720" LastEditorUserId="2444" LastEditDate="2020-01-22T14:47:03.890" LastActivityDate="2020-01-24T12:33:38.147" Title="Why do we prefer ReLU over linear activation functions?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6470" PostTypeId="1" CreationDate="2018-05-19T18:20:55.797" Score="1" ViewCount="45" Body="&lt;p&gt;Say I have access to several pre-trained CNNs (e.g. AlexNet, VGG, GoogleLeNet, ResNet, DenseNet, etc.) which I can use to extract features from an image by saving the activations of some hidden layer in each CNN. Likewise, I can also extract features using conventional hand-crafted techniques, such as: HOG, SIFT, LBP, LTP, Local Phase Quantization, Rotation Invariant Co-occurrence Local Binary Patterns, etc. Thus, I can obtain a very high-dimensional feature vector of an image that concatenates the individual features vectors outputted by these individual algorithms. Given these features, and given a data set of images over which I want to perform similar image retrieval (i.e. finding the &lt;strong&gt;top-k&lt;/strong&gt; most similar images to a query image X), what would be the most appropriate ways to implement this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible idea I have in mind is to learn an image similarity embedding in euclidean space by training a neural network that would receive as input the aforementioned feature vectors, and perhaps down-sampled versions of the image as well, and output a lower dimensional embedding vector that ideally should place similar images close to each other and dissimilar images far apart. And I could train this network using for example Siamese Loss or Triplet Loss. The challenge of this approach though is generating the labels for the (supervised) training itself. For example, in the case of the Triplet Loss I would need to sample triplets (Q,X,Y) and &lt;em&gt;somehow&lt;/em&gt; determine which one between X and Y is most similar to Q, in order to generate the label for the triplet (i.e., in order to &quot;teach&quot; the network I need to know the answers myself beforehand, but how? I guess this is domain dependent, but think of challenging cases where you have very heterogeneous images, such as photography galleries, artwork galleries, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, this is just an idea and by no means I pretend to mean this is the right approach. I'm open to new suggestions and insights about how to solve this task.&lt;/p&gt;&#xA;" OwnerUserId="12746" LastActivityDate="2018-05-19T18:20:55.797" Title="How to combine heterogeneous image features extracted with different algorithms for similar image retrieval?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6471" PostTypeId="1" CreationDate="2018-05-19T20:43:20.800" Score="1" ViewCount="53" Body="&lt;p&gt;I'm trying to design a neural network with a task hierarchy. This is my idea so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; [Desires]&#xA;    |&#xA;[Layer 1] [T0]&#xA;    |    /&#xA;[Layer 2] [T1]&#xA;    |    /&#xA;[Layer 3] [T2]&#xA;    |    /&#xA;[Layer 4] [T3]&#xA;    |    /&#xA;  [Action]   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The way this would work is that each layer represents a task as a binary number. Layer 1 is the main task, layer 2 the sub-task etc. Each task consists of 2 sub-tasks determined by T={0,1}. In this way the neural network represents a binary task graph with T=0 being the left child and T=1 being the right child of a node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can think of it as T3 changing every second T2 changed every 2 seconds and so on. So {T0 T1 T2 T3} gives the binary time in seconds in a 16 second cycle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far this only makes the output a sequence of 16 actions in order. But if some of the layers could be &quot;if&quot; gates they might control the T-values and so act as switches and so have more complicated programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any suggestions to improve this? Or has this kind of binary task graph representation been done before in a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also importantly how would you train such a neural network? (At the moment I just assume that the model is pre-trained and just trying to find a good architecture).&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2018-05-19T20:48:32.410" LastActivityDate="2018-05-19T20:48:32.410" Title="How to create a task-graph based neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6472" PostTypeId="1" CreationDate="2018-05-20T04:20:10.563" Score="1" ViewCount="246" Body="&lt;p&gt;A.I Community, this is my first post on here I am currently reading, learning and designing models. At the moment I'm working on this sentiment analysis tool; from what I gather sentiment analysis can be tricky to fine-tune hence why I'm reaching out here to improve on my model. I am asking for tips and pointers the hows and how not I would appreciate detailed answers in the context of improvement etc. Currently, the model is bias towards positive sentiment and even negative text is yielding.6 positive when it should be obvious in the negative sentiment side. My CSV contains 65000 tweets pre-labeled there is an even number it seems of positive and negative tweets and is indeed correctly labeled.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    import pandas as pd&#xA;from sklearn.model_selection import train_test_split&#xA;from keras.preprocessing.text import Tokenizer&#xA;from keras.preprocessing.sequence import pad_sequences&#xA;from keras.models import Sequential&#xA;import re&#xA;&#xA;data = pd.read_csv(&quot;training_data.csv&quot;)&#xA;data = data[['Sentiment', 'SentimentText']]&#xA;data['SentimentText'] = data['SentimentText'].apply(lambda x: x.lower())&#xA;data['SentimentText'] = data['SentimentText'].apply((lambda x: re.sub('[^a-zA-z0-9\s]','',x)))&#xA;max_features = 2000&#xA;tokenizer = Tokenizer(num_words=max_features, split=' ')&#xA;tokenizer.fit_on_texts(data['SentimentText'].values)&#xA;X = tokenizer.texts_to_sequences(data['SentimentText'].values)&#xA;X = pad_sequences(X)&#xA;&#xA;from keras.layers import Dense, Dropout, LSTM, Embedding&#xA;embed_dim = 50&#xA;lstm_out = 80&#xA;model = Sequential()&#xA;model.add(Embedding(max_features, embed_dim,input_length = X.shape[1]))&#xA;model.add(Dropout(0.2))&#xA;model.add(LSTM(lstm_out))&#xA;model.add(Dropout(0.2))&#xA;model.add(Dense(2,activation='softmax'))&#xA;model.compile(loss = 'binary_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;print(model.summary())&#xA;&#xA;Y = pd.get_dummies(data['Sentiment']).values&#xA;X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size=0.20, random_state=42)&#xA;print(X_train.shape,Y_train.shape)&#xA;print(X_test.shape,Y_test.shape)&#xA;model.fit(X_train, Y_train, nb_epoch=35, batch_size=32, verbose=1)&#xA;&#xA;#save model to disk and print the summary&#xA;model.save('model.h5', overwrite=True)&#xA;print(model.summary())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10139" LastActivityDate="2018-05-20T04:20:10.563" Title="How To Improve This Sentiment Analysis Model" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;&lt;sentiment-analysis&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6475" PostTypeId="1" CreationDate="2018-05-20T19:10:33.773" Score="1" ViewCount="62" Body="&lt;p&gt;Given a query image Q and two other images X and Y (you can assume they have more or less the same resolutions if that simplifies the problem), which algorithm would perform extremely well at determining which image between X and Y is most similar to Q, even when the differences are rather subtle? For example, a trivial case would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Q = image of mountains, X = image of mountains, Y = image of dogs, therefore it is clear that sim(Q,X) &gt; sim(Q,Y).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, examples of trickier cases would be: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Q = image of a yellow car, X = image of a red car, Y = image of a yellow car, therefore sim(Q,Y) &gt; sim(Q,X) (assuming the car shapes are more or less the same).&lt;/li&gt;&#xA;&lt;li&gt;Q = image of a man standing up in the middle with a black background, X = image of another man standing up in the middle with a black background, Y = image of a woman standing up in the middle with a black background, therefore sim(Q,X) &gt; sim(Q,Y).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which algorithm (or combination of algorithms) would be robust enough to handle even the tricky cases with very high accuracy?&lt;/p&gt;&#xA;" OwnerUserId="12746" LastEditorUserId="12746" LastEditDate="2018-05-20T19:23:55.013" LastActivityDate="2018-05-21T13:51:06.630" Title="Given a query image Q and two other images X and Y, how to determine which one is most similar to Q?" Tags="&lt;algorithm&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="10" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6477" PostTypeId="2" ParentId="6475" CreationDate="2018-05-21T13:51:06.630" Score="2" Body="&lt;p&gt;From your examples I assume you presuppose image recognition in the sense that you don't compare the actual images, but the descriptions of what the images contain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For comparing images there are various algorithms working on the visual similarity. This can sometimes lead to interesting results, as you probably have seen images on the internet like &quot;&lt;a href=&quot;https://starecat.com/content/wp-content/uploads/dog-face-or-a-muffin-guess.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;dog or muffin&lt;/a&gt;&quot;. A purely visual approach would find this hard to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you do have the description of the image (as there are ways of getting captions from images), then it would just be a text comparison between three sentences: the one describing your query image, and those describing your images X and Y. There are ways of getting at the semantic similarity of sentences. The simplest way (from your examples) would be to look at the overlap in words: [yellow, car]/Q, [red, car]/X, and [yellow, car]/Y obviously has the largest overlap with Q and Y. This is rather simplistic, but it is what your examples suggest you are dealing with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, getting a proper description from an image is a hard task, of the calibre that Google and Instagram are still working on; they have large amounts of training data and huge resources they can throw at the problem. So, unless that is your starting point, it will not be easy to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall I do not think it is currently possible to solve this problem with high accuracy.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-21T13:51:06.630" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6478" PostTypeId="1" AcceptedAnswerId="12016" CreationDate="2018-05-21T14:42:03.630" Score="4" ViewCount="118" Body="&lt;p&gt;I am trying to track LIDAR objects using Kalman filter. The problem is that the innovation has the value 0, which makes the Kalman gain be Infinity. &lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a link with the Kalman equations. The values with which I initialized the measurement and process covariance matrix are listed below. The update code is also shown below. When I debug the code everything is fine until the innovation becomes 0. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;this-&amp;gt;lidar_R &amp;lt;&amp;lt; std_laspx_, 0, 0, 0,&#xA;    0, std_laspy_, 0, 0,&#xA;    0, 0, 0, 0,&#xA;    0, 0, 0, 0;&#xA;&#xA;this-&amp;gt;lidar_H &amp;lt;&amp;lt; 1.0, 0.0, 0.0, 0.0, 0.0,&#xA;    0.0, 1.0, 0.0, 0.0, 0.0,&#xA;    0.0, 0.0, 0.0, 0.0, 0.0,&#xA;    0.0, 0.0, 0.0, 0.0, 0.0;&#xA;&#xA;P_ &amp;lt;&amp;lt; 1000, 0, 0, 0, 0,&#xA;    0, 1000, 0, 0, 0,&#xA;    0, 0, 1000, 0, 0,&#xA;    0, 0, 0, 1000, 0,&#xA;    0, 0, 0, 0, 1000;&#xA;&#xA; MatrixXd PHt = this-&amp;gt;P_ * H.transpose();&#xA; //S becomes 0&#xA; MatrixXd S = H * PHt + R;&#xA; //S_inv becomes INFINITY&#xA; MatrixXd S_inv_ = S.inverse();&#xA; MatrixXd K = PHt * S_inv_;&#xA;&#xA;VectorXd y = Z - Hx;&#xA;&#xA;this-&amp;gt;x_ = this-&amp;gt;x_ + K*y;&#xA;MatrixXd I = MatrixXd::Identity(x_.size(), x_.size());&#xA;this-&amp;gt;P_ = (I - K * H) * this-&amp;gt;P_;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15775" LastEditorUserId="2193" LastEditDate="2018-05-21T19:35:15.613" LastActivityDate="2019-09-23T08:01:50.223" Title="Kalman filter pre inovation" Tags="&lt;computer-vision&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6479" PostTypeId="1" CreationDate="2018-05-21T15:30:21.627" Score="2" ViewCount="186" Body="&lt;p&gt;A task I’m working on at the moment requires a CNN with a height map as one of the inputs. This is a matrix of floating point values in which each point is the height of that point above sea level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m having trouble deciding how to normalize this data. I know there are networks that work on depth or distance data but that is different for several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Height can also be negative (as opposed to depth/distance which starts at 0)&lt;/li&gt;&#xA;&lt;li&gt;Height has a very large range - can get values between -400 and +~9000.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For these reasons the common approach to normalisation, simply subtracting the mean and dividing by the standard deviation, will result in the loss of information in most cases (all values will be close to zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of maybe subtracting the local mean for each input, rather than a general mean calculated from all the data, but I still don't know what to do with the standard deviation, since dividing by the local standard deviation can result in very “flat” and very “steep” inputs looking the same after normalization.&lt;/p&gt;&#xA;" OwnerUserId="15776" LastEditorUserId="2193" LastEditDate="2018-05-21T19:35:07.787" LastActivityDate="2018-05-31T16:52:23.287" Title="Normalizing height data for CNN" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6481" PostTypeId="1" CreationDate="2018-05-21T16:03:53.303" Score="3" ViewCount="98" Body="&lt;p&gt;I have a data set with historical information of some events (let's say event A and event B),these events describe the discovery of land mines, the coordinates of the event and the date of the event; is there a way I can use this historical information to predict points (coordinates) where event A or B could happen i.e. where might be still land mines that haven't been found?&lt;/p&gt;&#xA;" OwnerUserId="15764" LastEditorUserId="15764" LastEditDate="2018-05-22T17:21:44.397" LastActivityDate="2018-05-22T17:21:44.397" Title="Is there a way to predict points on a map?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;evolutionary-algorithms&gt;&lt;game-theory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6482" PostTypeId="2" ParentId="6481" CreationDate="2018-05-21T16:31:48.090" Score="1" Body="&lt;p&gt;Leaving aside the time aspect, you could do a cluster analysis on the event coordinates. If you use an algorithm that gives you a &lt;a href=&quot;https://rdrr.io/cran/cluster/man/pam.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;medoid&lt;/a&gt; (ie centre) of the clusters, you can then look at other points, and work out how close they are to the centres of the event clusters. It might be possible from this to predict which event could happen at those coordinates (which is the closest cluster medoid), and how likely it is (distance from the medoid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, however, depends very much on the shape of the data. If there is no discernible structure contained in it, then this will not work. But it is definitely worth trying.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-05-21T16:31:48.090" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6483" PostTypeId="5" CreationDate="2018-05-22T04:30:22.903" Score="0" Body="&lt;p&gt;The seeds of modern AI were planted by classical philosophers who attempted to describe the process of human thinking as the mechanical manipulation of symbols. This work culminated in the invention of the programmable digital computer in the 1940s, a machine based on the abstract essence of mathematical reasoning. This device and the ideas behind it inspired a handful of scientists to begin seriously discussing the possibility of building an electronic brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ibm.com/blogs/research/2017/08/brain-inspired-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brain-Inspired AI&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5955/how-are-artificial-neural-networks-and-the-biological-neural-networks-similar-an&quot;&gt;How are Artificial Neural Networks and the Biological Neural Networks similar and different?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-22T18:49:30.943" LastActivityDate="2018-05-22T18:49:30.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6484" PostTypeId="4" CreationDate="2018-05-22T04:30:22.903" Score="0" Body="For questions about algorithms/methods used in AI/ML which is inspired by human behavior/brain." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-22T19:05:17.870" LastActivityDate="2018-05-22T19:05:17.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6486" PostTypeId="1" AcceptedAnswerId="6487" CreationDate="2018-05-22T07:57:58.633" Score="4" ViewCount="1516" Body="&lt;p&gt;In &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/control.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide 16&lt;/a&gt; of his lecture 5 of the course &quot;Reinforcement Learning&quot;, David Silver introduced &lt;em&gt;GLIE Monte-Carlo Control&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Is4qu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Is4qu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why is it an on-policy control? The sampling follows a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; while improvement follows an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy, so isn't it an off-policy control?&lt;/p&gt;&#xA;" OwnerUserId="15525" LastEditorUserId="2444" LastEditDate="2019-02-18T15:10:18.863" LastActivityDate="2020-08-10T09:15:05.520" Title="Why is GLIE Monte-Carlo control an on-policy control?" Tags="&lt;reinforcement-learning&gt;&lt;control-problem&gt;&lt;on-policy-methods&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6487" PostTypeId="2" ParentId="6486" CreationDate="2018-05-22T09:44:56.163" Score="4" Body="&lt;p&gt;In this case, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; has always been an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy. In every iteration, this &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is used to generate (&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedily) a trajectory from which the new &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values are calculated. The last line in the &amp;quot;pseudocode&amp;quot; tells you that the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; will be a new &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy in the next iteration. Since the policy that is improved and the policy that is sampled are the same, the learning method is considered an on-policy method.&lt;/p&gt;&#xA;&lt;p&gt;If the last line was &lt;span class=&quot;math-container&quot;&gt;$\mu \leftarrow \epsilon\text{-greedy}(Q)$&lt;/span&gt;, it would be an off-policy method.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastEditorUserId="8448" LastEditDate="2020-08-10T09:15:05.520" LastActivityDate="2020-08-10T09:15:05.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6488" PostTypeId="1" CreationDate="2018-05-22T09:49:42.907" Score="6" ViewCount="1296" Body="&lt;p&gt;I used the example at - &lt;a href=&quot;https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/5_DataManagement/tensorflow_dataset_api.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/5_DataManagement/tensorflow_dataset_api.py&lt;/a&gt; - to create my own classification model. I used different data but the basic outline of datasets was used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was important for my data type to shuffle the data and then create the training and testing sets. The problem, however, comes as a result of the shuffling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I train my model with the shuffled train set I get a +- 80% accuracy for train and +- 70% accuracy for the test set. I then want to input all the data (i.e. the set that made the training and test set) into the model to view the fully predicted output of this data set that I have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this data set is shuffled as the training and testing set was I get an accuracy of around 77% which is as expected, but then, if I input the unshuffled data (as I required to view the predictions), I get a 45% accuracy. How is this possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume it's due to the fact that the model is learning incorrectly and that it learns that the order of the data points plays a role in the prediction of those data points. But this shouldn't be happening as I am simply trying to (like the MNIST example) predict each data point separately. This could be a mini-batch training problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example mentioned above, using data sets and batches to train, does the model learn from the average of all the data points in the mini-batch or does it think one mini-batch is one data point and learn in that manner (which would mean order matters of the data)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or if there are any other suggestions.&lt;/p&gt;&#xA;" OwnerUserId="15789" LastEditorUserId="2444" LastEditDate="2020-01-28T01:32:24.880" LastActivityDate="2020-01-28T01:32:24.880" Title="Does the model learn from the average of all the data points in the mini-batch?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;python&gt;&lt;tensorflow&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6489" PostTypeId="1" AcceptedAnswerId="6511" CreationDate="2018-05-22T13:25:09.207" Score="2" ViewCount="567" Body="&lt;p&gt;I know how IOU works during detection. However, while preparing targets from ground-truth for training, how is the IOU between a given object and all anchor boxes calculated?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is the ground truth bounding box aligned with an anchor box such that they share the same center? (width/2, height/2)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think this is the case but I want to hear from someone who has better knowledge of how training data is prepared for training in YOLO.&lt;/p&gt;&#xA;" OwnerUserId="13255" LastEditorUserId="2444" LastEditDate="2021-01-28T23:23:44.267" LastActivityDate="2021-01-28T23:23:44.267" Title="How are IOUs for ground truth boxes in YOLO calculated?" Tags="&lt;computer-vision&gt;&lt;yolo&gt;&lt;bounding-box&gt;&lt;jaccard-similarity&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6491" PostTypeId="1" CreationDate="2018-05-22T21:02:54.883" Score="5" ViewCount="94" Body="&lt;p&gt;Various texts on using CNNs for object detection in images talk about how their translation invariance is a good thing. Which makes sense for tasks where the object could be anywhere in the image. Let's say detecting a kitten in household images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But let's say, you already have some information about the likely position of the object of interest in the image. For example, for detecting trees in a dataset of images of landscapes. Here in &lt;em&gt;most cases&lt;/em&gt;, the trees are going to be in the bottom half of the image while in &lt;em&gt;some cases&lt;/em&gt; they might be at the top (because it's on a hill or whatever). So you want your neural network to learn that information -- that trees are likely connected to the bottom part of the image (ground). Is this possible using the CNN paradigm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="15803" LastActivityDate="2018-07-24T16:13:01.040" Title="Can translational invariance of CNNs be unwanted if object is likely in certain positions?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6492" PostTypeId="1" AcceptedAnswerId="6502" CreationDate="2018-05-23T03:57:53.923" Score="2" ViewCount="158" Body="&lt;p&gt;I was just reading through some convex optimization textbooks to hopefully improve my deep learning understanding and come up with new ideas. Halfway through, I decided to Google a bit! It's obvious that deep learning deal with nonconvex functions. Here's the question though: If deep learning is non-convex then why we apply a convex loss function such as cross-entropy or least square to solve a problem under a convex constraint? What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="15805" LastEditorUserId="1671" LastEditDate="2018-05-23T20:01:22.447" LastActivityDate="2018-05-24T05:43:56.133" Title="If Deep Learning is non convex, then why use convex loss?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-basics&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6493" PostTypeId="5" CreationDate="2018-05-23T05:04:03.350" Score="0" Body="&lt;p&gt;Efficiency of an ML/AI algorithm may be benchmarked on many aspects especially:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Time Complexity&lt;/li&gt;&#xA;&lt;li&gt;Space Complexity&lt;/li&gt;&#xA;&lt;li&gt;Accuracy&lt;/li&gt;&#xA;&lt;li&gt;Processor Requirements&lt;/li&gt;&#xA;&lt;li&gt;Simplicity of Understanding, etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jcsites.juniata.edu/faculty/rhodes/cs2/ch12a.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithm Efficiency&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/3255/big-o-how-do-you-calculate-approximate-it&quot;&gt;Big O, how do you calculate/approximate it?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-23T19:24:50.413" LastActivityDate="2018-05-23T19:24:50.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6494" PostTypeId="4" CreationDate="2018-05-23T05:04:03.350" Score="0" Body="For questions about efficiency of ML/AI algorithms solving a particular problem." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-23T19:24:57.687" LastActivityDate="2018-05-23T19:24:57.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6495" PostTypeId="2" ParentId="5398" CreationDate="2018-05-23T08:46:19.700" Score="3" Body="&lt;p&gt;Based on &lt;a href=&quot;https://github.com/openai/baselines/issues/240#issuecomment-391165056&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comment&lt;/a&gt; in the Issue I created about this question on github, it looks like there is confirmation that at least DeepMind does &lt;strong&gt;not&lt;/strong&gt; use this kind of functionality in their Atari experiments, contrary to what is implied by the comments in the OpenAI baselines code.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-05-23T08:46:19.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6496" PostTypeId="2" ParentId="3494" CreationDate="2018-05-23T09:25:07.680" Score="9" Body="&lt;p&gt;What attracts me to Python for my analysis work is the &quot;full-stack&quot; of tools that are available by virtue of being designed as a general purpose language vs. R as a domain specific language. The actual data analysis is only part of the story, and Python has rich tools and a clean full-featured language to get from the beginning to the end in a single language (use of C/Fortran wrappers notwithstanding).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the front end, my work commonly starts with getting data from a variety of sources, including databases, files in various formats, or web scraping. Python support for this is good and most database or common data formats have a solid, well-maintained library available for the interface. R seems to share a general richness for data I/O, though for FITS the R package appears not to be under active development (no release of FITSio in 2.5 years?). A lot of the next stage of work typically occurs in the stage of organizing the data and doing pipeline-based processing with a lot of system-level interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the back end, you need to be able present large data sets in a tangible way, and for me, this commonly means generating web pages. For two projects I wrote significant Django web apps for inspecting the results of large Chandra survey projects. This included a lot of scraping (multiwavelength catalogs) and so forth. These were just used internally for navigating the data set and helping in source catalog generation, but they were invaluable in the overall project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moving to the astronomy-specific functionality for analysis, it seems clear that the community is solidly behind Python. This is seen in the depth of available packages and level of development activity, both at an individual and institutional level (&lt;a href=&quot;http://www.astropython.org/resources&quot; rel=&quot;noreferrer&quot;&gt;http://www.astropython.org/resources&lt;/a&gt;). Given this level of infrastructure that is available and in work, I think it makes sense to direct effort to port the most useful R statistical tools for astronomy to Python. This would complement the current capability to call R functions from Python via rpy2.If you are interested, I strongly recommend that you read this article, here it is a question of comparing programming languages &lt;a href=&quot;https://diceus.com/what-technology-is-b&quot; rel=&quot;noreferrer&quot;&gt;https://diceus.com/what-technology-is-b&lt;/a&gt; ... nd-java-r/ I hope it helps.Good Luck&lt;/p&gt;&#xA;" OwnerUserId="15811" LastActivityDate="2018-05-23T09:25:07.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6497" PostTypeId="1" CreationDate="2018-05-23T11:18:27.357" Score="1" ViewCount="107" Body="&lt;p&gt;For a regression task, I have sequences of training data and if I define the layers of deep neural network to be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Layers=[ sequenceInputLayer(featuredimension) reluLayer dropoutLayer(0.05) fullyConnectedLayer(numResponse) regressionLayer]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a valid deep neural network? Or do I need to add LSTM layer too?&lt;/p&gt;&#xA;" OwnerUserId="15649" LastEditorUserId="1671" LastEditDate="2018-05-23T19:58:38.563" LastActivityDate="2018-05-23T19:58:38.563" Title="Is it a valid Deep Neural Network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;matlab&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6499" PostTypeId="1" AcceptedAnswerId="6531" CreationDate="2018-05-23T14:27:12.527" Score="7" ViewCount="319" Body="&lt;p&gt;I  want to use computer vision to allow my robot to detect the corners of a soccer field based on its current position. Matlab has a detectHarrisFeatures feature, but I believe it is only for 2D mapping.&lt;/p&gt;&#xA;&lt;p&gt;The approach that I want to try is to collect the information of the lines (using line detection), store them in a histogram, and then see where the lines intersect based on their angles.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I know where the lines intersect?&lt;/li&gt;&#xA;&lt;li&gt;How do I find the angles of the lines using computer vision?&lt;/li&gt;&#xA;&lt;li&gt;How do I update this information based on my coordinates?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am in the beginning stages of this task, so any guidance is much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="15821" LastEditorUserId="22079" LastEditDate="2020-07-29T23:02:28.477" LastActivityDate="2020-07-29T23:02:28.477" Title="How to use computer vision to find corners of a soccer field based on location coordinates?" Tags="&lt;algorithm&gt;&lt;computer-vision&gt;&lt;matlab&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6500" PostTypeId="2" ParentId="5496" CreationDate="2018-05-23T14:44:01.213" Score="1" Body="&lt;p&gt;In the original paper the innovation ID is on the connections only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The connection is the object that is keeping the information; nodes can be discerned by the connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lmBlH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lmBlH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image represents a possible crossover operator that makes a distinction between disjoint and excess genes and therefore creates children depending on these. It's part of my master thesis, I'd be glad to expand the topic, but for now I'll just use it as example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the image we assume that the connections that are joining two nodes have the same innovation number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, there is no need of assigning an innovation number to the nodes: nodes are just a result of what the connection say. This also allows for a more dynamic approach that can be used to spot invalid nets even before building them (checking if there are cyclics or nodes that aren't receiving any input or giving any output) and correct them in order to obtain only valid graphs.&#xA;Nodes are added just because there is a connection that is pointing to that specific node. This is enough to grant its presence (node number 8 in child 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As last point, for the data normalization theory (&lt;em&gt;Normalization is a process of organizing the data in database to avoid data redundancy, insertion anomaly, update anomaly &amp;amp; deletion anomaly&lt;/em&gt;) we should avoid rendundancy at any cost and this is why we should try to keep track of the smallest amount of object possibles. So if we can deduct the nodes from the connections we should do it.&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2018-05-23T14:44:01.213" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6501" PostTypeId="2" ParentId="6497" CreationDate="2018-05-23T16:00:49.070" Score="1" Body="&lt;p&gt;Yes, it is a very common practice to use some RNN when your input data is a sequence.&#xA;Besides, your network has some shape issue if your input data is 2D. You should, at least flatten your input data to a vector to be able to forward propagate to the dense layer, but instead of this use some kind of RNN. &#xA;To the best of my knowledge, that dropout value doesn't good. It seems too low.&lt;/p&gt;&#xA;" OwnerUserId="6019" LastEditorUserId="6019" LastEditDate="2018-05-23T16:07:35.713" LastActivityDate="2018-05-23T16:07:35.713" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6502" PostTypeId="2" ParentId="6492" CreationDate="2018-05-24T05:43:56.133" Score="5" Body="&lt;p&gt;Well, you are definitely mixing two different things. Here are those bits:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The function that deep learning approximates is basically a function that best fits the INPUT DATA points. You should not think about its differentiability or optimization aspects. We don't care what type of function it is; we just want the best fit of input data (ofcourse overfitting is not desired though). So, &lt;strong&gt;the space in which this fucntion lies has dimensions equal to the dimensions of input data (or number of input features)&lt;/strong&gt;. For every function fit we get some loss which basically is the distance from actual data point from the one predicted by the function fitted.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Loss function is the one which is defined in terms weights and biases. You can think &lt;strong&gt;the space in which loss function lies has dimensions equal to the number of PARAMETERS to tune (weights and biases)&lt;/strong&gt;. So, this space is very different from the one described in the first point. Now, here we do care about the differentiability of the function because we want a point where the function outputs least value (minimize the loss) given some particular values of inputs (weights and biases in this case) and by differentiating we can traverse to that &quot;optimal&quot; point in the function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this clears your doubt.&lt;/p&gt;&#xA;" OwnerUserId="15796" LastActivityDate="2018-05-24T05:43:56.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6503" PostTypeId="1" AcceptedAnswerId="6506" CreationDate="2018-05-24T10:26:14.820" Score="1" ViewCount="671" Body="&lt;p&gt;How do I decrease the accuracy value when training a model using &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras&lt;/a&gt;; which parameters can I change to decrease the value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My objective is not to actually decrease it, but just to know which parameters influence the accuracy&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sgd = optimizers.SGD(lr=1e-2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15818" LastEditorUserId="2193" LastEditDate="2018-05-24T10:41:20.030" LastActivityDate="2020-02-16T17:40:27.673" Title="How to decrease accuracy from 99% to 80%~85% using keras for training a model" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6504" PostTypeId="1" AcceptedAnswerId="6630" CreationDate="2018-05-24T15:57:40.550" Score="1" ViewCount="466" Body="&lt;p&gt;I would like to know how to teach an agent for performing prediction of the severity of disease and also for alerting patients using machine learning methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the &lt;a href=&quot;https://ai.stackexchange.com/q/3243/2444&quot;&gt;model-based reflex agent&lt;/a&gt; can be used in medical diagnosis in some literature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;May I know which architecture will be good, to make such an agent?&lt;/p&gt;&#xA;" OwnerUserId="15631" LastEditorUserId="2444" LastEditDate="2020-06-02T23:24:46.283" LastActivityDate="2020-06-02T23:24:46.283" Title="How to teach a model-based reflex agent for doing some task using machine learning methods?" Tags="&lt;machine-learning&gt;&lt;intelligent-agent&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6505" PostTypeId="2" ParentId="6377" CreationDate="2018-05-24T17:20:23.263" Score="1" Body="&lt;p&gt;If you want only the answer to your question in particular, you can skip to the last part of the answer. To answer in detail momentum is a technically incorrect term, I would rather call it &lt;a href=&quot;https://en.wikipedia.org/wiki/Inertia&quot; rel=&quot;nofollow noreferrer&quot;&gt;inertial&lt;/a&gt; learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Inertia -&lt;/strong&gt; Inertia is the resistance of any physical object to any change in its position and state of motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First the equation of weight change in the momentum learning method at a particular iteration is given by the equation:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/l8jkA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l8jkA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;strong&gt;&lt;code&gt;beta&lt;/code&gt;&lt;/strong&gt; will be the momentum term. If we expand the expression we get something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QTtc9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QTtc9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Courtsey: &lt;a href=&quot;https://towardsdatascience.com/stochastic-gradient-descent-with-momentum-a84097641a5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic Gradient Descent with momentum&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;strong&gt;&lt;code&gt;S_t&lt;/code&gt;&lt;/strong&gt; are the gradients or &lt;strong&gt;&lt;code&gt;dels&lt;/code&gt;&lt;/strong&gt; for a particular training example. Clearly this is for a 3 example training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now why do we use momentum?&#xA;As @Andreas Storvik Strauman has provided a &lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/index.html#momentum&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;, you can easily delve into the mathematics for its usage. But to make a more intuitive sensehere are a few points to note:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The exponentially weighted terms can be thought of as past memory of what you learned. You don't want to forget it completely, so you keep on revising it with weight-age of revising it decreasing over time. The vector updation for an already iterated training example keeps getting smaller and smaller, whereas it is not present altogether in normal gradient descent.&lt;/li&gt;&#xA;&lt;li&gt;The momentum term can be thought to play a damping role, it is not allowing the new training example to have its way completely. You can visualize this by taking 2 points and a straight line, with the updation scheme directly proportional to the distance between the line and the points and then check for both normal and momentum gradient descent methods. Thus gradient descent with momentum is a damped oscillation and thus always has a higher chance of converging.&lt;/li&gt;&#xA;&lt;li&gt;The inertial learning also helps when you come to a point in your loss curve where slope is 0. Normal learning will result in very small weight updates in this position, but with inertial learning this position will be easily crossed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As for your original question of why momentum term &amp;lt;1, here are a few points which most answers have missed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first and foremost, if &lt;strong&gt;&lt;code&gt;beta &amp;gt; 1&lt;/code&gt;&lt;/strong&gt;, the weightage for previous training examples will increase exponentially. (Like &lt;strong&gt;&lt;code&gt;1.01^1000 = 20959&lt;/code&gt;&lt;/strong&gt; just after 1000 iterations). That maybe handled by increasing the &lt;strong&gt;&lt;code&gt;learning rate&lt;/code&gt;&lt;/strong&gt; accordingly, but not only it will require a lot of extra computation, it is almost mathematically impossible.&lt;/li&gt;&#xA;&lt;li&gt;Second, a exponential series with &lt;strong&gt;&lt;code&gt;r &amp;gt;= 1&lt;/code&gt;&lt;/strong&gt; common ratio never converges. It goes on going bigger and bigger. Also, if you can draw parallels with continuous functions, this is what we call a function which is not &lt;a href=&quot;https://en.wikipedia.org/wiki/Absolutely_integrable_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Absolutely integrable function&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Also as per our previous intuition why would one want to give high weight-age to things which you have learned previously. It may not be even important if you follow online learning method (you look at each training example only once due to high number of training examples).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All these leads to a single conclusion if &lt;strong&gt;&lt;code&gt;beta &amp;gt;= 1&lt;/code&gt;&lt;/strong&gt;, there will be a large amount of oscillation and error will go on increasing exponentially (can probably be proven by rigorous mathematical analysis). Although it might wotk for &lt;strong&gt;&lt;code&gt;beta = 1&lt;/code&gt;&lt;/strong&gt;(due to the &lt;a href=&quot;http://www.cs.ubbcluj.ro/~csatol/kozgaz_mestint/4_neuronhalo/PerceptConvProof.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptron Convergence Theorem&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-08-28T12:24:21.173" LastActivityDate="2018-08-28T12:24:21.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6506" PostTypeId="2" ParentId="6503" CreationDate="2018-05-24T18:22:37.393" Score="1" Body="&lt;p&gt;There are &lt;strong&gt;many&lt;/strong&gt; things affecting accuracy. I'm gonna assume a lot here because you don't say anything about the model, what you're trying to achieve or how many classes you have. You're not even saying whether you're classifying or not. Also, you're not saying which accuracy you're using (classification, AUC, F1 etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm gonna assume here that you have some classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Accuracy&lt;/strong&gt; is the measure of how many classifications you got correct. In my experience 99% is a warning sign because it's too good to be true, and a result like that is often due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt;. Since this, in my experience, is the main reason you'd actually want the accuracy down, this is what I'm going to assume is your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overfitting occurs when you train &quot;too much&quot; and the model only learns things that are within your training set, and fails on everything else. That is: it generalized bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prevent this there are a number of things you could do;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Data segmentation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common is to split your data into three bulks: training (~70% of the data), validation(~20%) test (~10%). These percentages are indications and would vary depending on how much data you have, and the balancing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that you train on the training data, you run the validation through the network, and calculate the accuracy. When this accuracy, call it validation accuracy, is satisfactory, then you stop the training and run the test data through it. The latter accuracy (test accuracy) is the one that most papers publish (combined with &lt;a href=&quot;https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve&quot; rel=&quot;nofollow noreferrer&quot;&gt;AUC&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/F1_score&quot; rel=&quot;nofollow noreferrer&quot;&gt;F1 score&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Important&lt;/em&gt;: When you have split the data into these bulks you should put away the test, and &lt;strong&gt;not use it during training&lt;/strong&gt; at all. You only use this at the very end to do an extra check that you haven't overfitted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Regularization&lt;/strong&gt;&#xA;There are many types of &lt;a href=&quot;http://www.deeplearningbook.org/contents/regularization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;regularization&lt;/a&gt;. Two very popular regularization methods for preventing overfitting are the L2-regularization (see previous link) and the &lt;a href=&quot;https://arxiv.org/pdf/1102.4807.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dropout&lt;/a&gt; methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without going into detail, these methods prevent the model weights from becoming too large. This is a good thing since the model won't rely too much on one feature, which in turn attenuates overfitting.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I hope you learned something, and the most important lesson is that you should know what you're doing. If not, you could end up with a model that is not behaving like you thought. In the case of overfitting, you'd end up with a model that only works on your training data, which doesn't really do much good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really recommend the book by Goodfellow: &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeplearningbook.org&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2018-05-31T22:08:03.310" LastActivityDate="2018-05-31T22:08:03.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6507" PostTypeId="2" ParentId="5096" CreationDate="2018-05-24T18:42:36.037" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Would I be right in saying that this becomes a sort of 'pattern recognition' problem? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Technically, yes. In practice: no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you might be interpreting the term &quot;pattern recognition&quot; a bit too literal. Even though &lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt; defines Pattern recognition as &quot;a branch of machine learning that focuses on the recognition of patterns and regularities in data&quot;, it's not about solving problems that can &quot;easily&quot; be deduced by logical reasoning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. you say that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A '1' in the top right triangle in the matrix represents a convex relationship between two faces and a '1' in the bottom left triangle represents a concave relationship&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is true &lt;em&gt;always&lt;/em&gt;. In a typical machine learning situation, you wouldn't (usually) have this prior knowledge. At least not to the extent that it would b be tractable to “solve by hand”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pattern recognition is conventionally a statistical approach to solving problems when they get too complex to analyze with conventional logical reasoning and simpler regression models. Wikipedia also states (with a source) that pattern recognition &quot;in some cases considered to be nearly synonymous with machine learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said: you &lt;em&gt;could&lt;/em&gt; use pattern recognition on this problem. However, it seems like overkill in this case. Your problem, as far as I can understand, has an actual &quot;analytical&quot; solution. That is: you can, by logic, get a 100% correct result all the time. Machine learning algorithms could, in theory, also do this, and in that case, and this branch of ML is referred to as Meta Modelling[1].&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, if I supply the network with a number of training models - along with labels which describe the design feature which exists in the model, would the network learn to recognise specific adjacency patterns represented in the matrix which relate to certain design features?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a word: Probably. Best way to go? Probably not. Why not, you ask?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is always the possibility that your model doesn't learn exactly what you want. In addition you have many challenges like &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt; that you'd need to concern yourself about. It's a statistical approach, as I said. Even if it classifies all your test data as 100% correct, there is no way (unless you check the insanely intractable maths) to be 100% sure that it will always classify correctly. I further suspect that you're also likely to end up spending more time working on your model then the time it would take to just deduce the logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also disagree with @Bitzel: I would not do a CNN (convolutional neural network) on this. CNNs are used when you want to look at specific parts of the matrix, and the relation and connectedness between the pixels are important — for example on images. Since you only have 1s and 0s, I strongly suspect that a CNN would be vastly overkill. And with all the sparsity (many zeros) you’d end up with a lot of zeros in the convolutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd actually suggest a plain vanilla (feed forward) neural network, which, despite the sparsity, I think will be able to do this classification pretty easily.&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2018-05-31T22:16:33.450" LastActivityDate="2018-05-31T22:16:33.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6508" PostTypeId="2" ParentId="3494" CreationDate="2018-05-24T19:39:16.993" Score="3" Body="&lt;p&gt;That’s because python is a modern scripting object-oriented programming language that has stylish syntax. Contrary to structural programming languages like java and C++, its scripting nature enables the programmer to test his/her hypothesis very fast. Furthermore, there are lots of open source machine learning libraries (including scikit-learn and Keras) that broaden the use of python in AI field.&lt;/p&gt;&#xA;" OwnerUserId="15861" LastActivityDate="2018-05-24T19:39:16.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6509" PostTypeId="2" ParentId="6491" CreationDate="2018-05-25T01:02:27.277" Score="1" Body="&lt;p&gt;I think, your assumption about the location of trees in images is quite incorrect. Just google image search &quot;landscape&quot; (if not already) and you will see almost equal number of images where the trees constitute top part of the image to those images where they lie in only the middle and bottom part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Talking about the CNN, it automatically learns (thats the beauty!) the properties of an object which are there in the training images. By properties I mean the object's likely position, location, its shape color etc. If you visualize the CNN layer (mostly later layers) output, using &lt;a href=&quot;https://jacobgil.github.io/deeplearning/class-activation-maps&quot; rel=&quot;nofollow noreferrer&quot;&gt;class activated maps&lt;/a&gt;, you can see what CNN has learnt and paying attention to. Also, you can &lt;a href=&quot;https://jacobgil.github.io/deeplearning/filter-visualizations&quot; rel=&quot;nofollow noreferrer&quot;&gt;visualize the filters&lt;/a&gt; (or kernels) that are learned by the CNN.&lt;/p&gt;&#xA;" OwnerUserId="15796" LastEditorUserId="15796" LastEditDate="2018-05-25T01:24:47.677" LastActivityDate="2018-05-25T01:24:47.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6510" PostTypeId="1" CreationDate="2018-05-25T03:00:56.987" Score="2" ViewCount="1404" Body="&lt;p&gt;I am looking to detect thin objects, like pens, pencils, and surgical instruments. The bounding box is not important, but I am looking to see if I can train a model to detect both the object as well as its orientation.&lt;/p&gt;&#xA;&lt;p&gt;Typical object detection networks, like R-CNN, YOLO, and SSD encode the class name and bounding boxes. Instead of bounding boxes, I'm looking to encode only 2 points, one starting &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; point and one ending &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; point.  The start point for objects is where one would grip the object.  For instance:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The pencil eraser(start point) is pointed 50 degrees to the top right.&lt;/li&gt;&#xA;&lt;li&gt;The surgical instrument is 10 degrees from the x-axis and the handle is pointed to the bottom right.&lt;/li&gt;&#xA;&lt;li&gt;Pen tip (endpoint) is pointing vertically upwards.&lt;/li&gt;&#xA;&lt;li&gt;Fork, the start point would be the grip handle part, and the endpoint would be in the middle where the 4 prongs are.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As long as I can encode the start and endpoints, then I can determine the orientation. I would need to define these points during training.&lt;/p&gt;&#xA;&lt;p&gt;The question is whether there is an existing model (mobile net/inception/RCNN) that I can encode this information in?  One potential way I was thinking was to use YOLO and for the bounding box, the top left &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; would be the starting point &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; (handle), whereas the bounding box's width and height would be replaced with the endpoint &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; (pencil writing tip, fork prongs.&lt;/p&gt;&#xA;" OwnerUserId="15865" LastEditorUserId="2444" LastEditDate="2021-01-28T23:59:19.330" LastActivityDate="2021-01-28T23:59:19.330" Title="How can I detect thin objects (like pens and pencils) without a bounding box but only 2 endpoints and the orientation?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;yolo&gt;&lt;model-request&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6511" PostTypeId="2" ParentId="6489" CreationDate="2018-05-25T04:11:15.277" Score="2" Body="&lt;p&gt;My assumption was correct: &lt;strong&gt;the ground truth bounding box is aligned with an anchor box such that they share the same center&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, only the widths and heights are used to calculate the ground truth IOU.&lt;/p&gt;&#xA;" OwnerUserId="13255" LastActivityDate="2018-05-25T04:11:15.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6512" PostTypeId="5" CreationDate="2018-05-26T05:34:37.263" Score="0" Body="&lt;p&gt;In mathematics and computer science, computer algebra, also called symbolic computation or algebraic computation is a scientific area that refers to the study and development of algorithms and software for manipulating mathematical expressions and other mathematical objects&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/16395704/what-is-symbolic-computation?utm_medium=organic&amp;amp;utm_source=google_rich_qa&amp;amp;utm_campaign=google_rich_qa&quot;&gt;What is symbolic computation?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-30T16:18:41.250" LastActivityDate="2018-05-30T16:18:41.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6513" PostTypeId="4" CreationDate="2018-05-26T05:34:37.263" Score="0" Body="For questions about the use of symbolic computing in AI (like LISP programming language)." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-05-30T16:18:27.130" LastActivityDate="2018-05-30T16:18:27.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6515" PostTypeId="1" CreationDate="2018-05-26T09:15:43.377" Score="1" ViewCount="79" Body="&lt;p&gt;Is there a way in the WEKA explorer to manually select the initial centres when using SimpleKMeans clustering?&lt;/p&gt;&#xA;" OwnerUserId="15879" LastActivityDate="2018-08-01T14:01:03.550" Title="WEKA - SimpleKMeans - Manually choose intitial centres" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6516" PostTypeId="2" ParentId="5792" CreationDate="2018-05-26T14:18:35.683" Score="4" Body="&lt;p&gt;In GE, the genotype is a linear sequence of codons. By &amp;quot;wrapping&amp;quot; it, you make it a circular sequence that never ends. It allows you to build a bigger tree, while having only a few codons. Still, it is possible to find such a combination of a genotype and a grammar that defines an infinitely deep expansion — such combinations are hardly suited for practical purposes.&lt;/p&gt;&#xA;&lt;p&gt;I learned about GE recently. I implemented &lt;a href=&quot;https://github.com/werediver/Sandbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;a GP/GE system&lt;/a&gt; to solve &lt;a href=&quot;https://en.wikipedia.org/wiki/Santa_Fe_Trail_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Santa Fe Trail problem&lt;/a&gt;. I chose not to perform wrapping to make genotype-to-phenotype mapping more predictable (I always generate enough codons to complete a grammar expansion, and prune unused tails). I also went with a subtree crossover, and a subtree-local mutation, which effectively make the system more a classical GP (with a fancy genotype-to-phenotype mapping) than GE. &lt;strong&gt;So there are some options&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15881" LastEditorUserId="2444" LastEditDate="2020-12-27T14:26:38.640" LastActivityDate="2020-12-27T14:26:38.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6520" PostTypeId="2" ParentId="4910" CreationDate="2018-05-27T13:26:14.883" Score="1" Body="&lt;p&gt;The core of the question seems to really be: &quot;how to approach thinking about this&quot;, where &quot;this&quot; is the input of an AI player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern attempts at game playing AI players try to replace a human player &quot;as is&quot;. No advantage whatsoever. This implies that we want to feed the same &quot;raw input&quot; to the software player and to the human player. For a video game, the raw input is usually the screen we see, but it could also include sound, vibration feedback from a game controller, force feedback from driving gears, etc. Basically any sensor input available. For a physical board game like Go, the raw input could be a video feed, as done for &lt;a href=&quot;https://www.youtube.com/watch?v=vFr3K2DORc8&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt;. Doing so, we fall back to the video game approach. So the input to the AI player is often a tensor, where each element is the intensity of a screen pixel, or some other sensor signal (note there is nothing here about the shape of the tensor, please see below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way of thinking here is actually pretty simple, and neat: We want an AI player to replace a human player. So we list up what the human player can sense, and model each input as a tensor (this includes vectors and matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is perhaps a subtlety here. We face choices not so obvious to make, and there are usually trials and errors (many errors). For example, the Deep-Q Network from DeepMind to play Atari games takes as input &lt;em&gt;differences&lt;/em&gt; between two consecutive video frames. Some other approach could solve the same problem with actual &lt;em&gt;sequences&lt;/em&gt; of inputs (e.g. 10 consecutive frames). The two approaches are valid---they just have different tradeoffs we must evaluate to find the best configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another subtelty is the &quot;shape&quot; of the input. Should a single (W, H) input frame be a (W*H) vector, or a (W, H) matrix, or something else? The ramifications can go far. If the frame is an RGB frame with transparency, the input could well be a tensor of shape (W, H, 4), a &quot;cubic volume&quot; with 4 &quot;slices&quot; of shape (W, H)---one for each RGB channel, and a fourth one for transparency. Imagine now if you also have sound, and can add a fifth &quot;slice&quot;---and more if you have stereo or Dolby system channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last point: There is no reference to the architecture of the AI player beyond the input here. Although most exemples refer to recent architectures based on neural networks, this &quot;thinking&quot; also applies whatever can leverage the input. For example, the Big Blue system from IBM, who won Chess against Gary Kasparov, is an AI player using exclusively (if my memory's good) clever tree search. Yet its input is a tensor representing the Chess board.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-05-27T13:26:14.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6521" PostTypeId="2" ParentId="5916" CreationDate="2018-05-27T14:16:30.053" Score="1" Body="&lt;p&gt;A generic way to start with under such circumstances is to try find an &quot;oracle&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Serial to parallel converters exist for quite some times, and some are open source (e.g. &lt;a href=&quot;https://pips4u.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PIPS&lt;/a&gt;). The idea is to get serial code from step (1), use the &quot;oracle&quot; to produce parallel code, and that's it: Each conversion makes en entry in the dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ensuring the quality of the dataset is critical here. A script generating the dataset shoud ensure (1) the serial code compiles and runs properlly, (2) the parallelized code compiles and runs properly too, (3) serial and parallel programs produce the same result, (4) some metrics state objectively how the parallel version does against the serial version, and (5) keep track of the actual hardware configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Point (4) is critical to the quality of such a dataset. Parallel programs are &lt;em&gt;not&lt;/em&gt; always faster than a serial version: A 1000-iteration loop dumped over 1000 workers on a 8-core CPU may not do so well compared to 8 workers---we need to check what the converter is doing. And point (5) ensures we know under which conditions the data is valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using several &quot;oracles&quot; would be even better, to bring diversity, and to hopefully get the learning algorithm discover the best conversion tradeoffs---perhaps better than what the carefully hand-crafted converters are able to do on their own.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-05-27T14:16:30.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6522" PostTypeId="2" ParentId="6499" CreationDate="2018-05-27T14:40:23.990" Score="2" Body="&lt;p&gt;Finding lines in an image often leads to the Hough line transform. Many libraries implement it, including &lt;a href=&quot;https://docs.opencv.org/3.4.1/d6/d10/tutorial_py_houghlines.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV&lt;/a&gt;. Getting the lines should answer subsequent questions (and it doesn't, please consider having one question per post, and some other sites on &lt;a href=&quot;https://stackexchange.com/&quot;&gt;StackExchange&lt;/a&gt; may be better suited than AI.SE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternative approaches based on Machine Learning may also exist. It may be interesting at this point to look into libraries that do human pose/gait recognition. Some library may be repurposed to recognize the &quot;field gait&quot;, assuming the field outer mark is its &quot;gait&quot;-equivalent.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-05-27T14:40:23.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6524" PostTypeId="1" AcceptedAnswerId="6566" CreationDate="2018-05-27T15:47:36.650" Score="2" ViewCount="130" Body="&lt;p&gt;I have to read a lot of papers, and I thought that I can use an A.I. to read them and summarize them. Maybe find one that can understand what the papers are talking about it seems a lot to ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I can use natural language processing. Is it the right choice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sorry, but I'm new in A.I. and I don't know much about it.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4302" LastEditDate="2018-10-08T11:56:59.767" LastActivityDate="2018-10-08T11:56:59.767" Title="How can I build an AI with NLP that reads and understands documents?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6526" PostTypeId="1" AcceptedAnswerId="6529" CreationDate="2018-05-27T16:05:17.947" Score="3" ViewCount="1598" Body="&lt;p&gt;I'm currently working on license plate recognition. My system consist of 2 stage: (1) License Plate region extraction &amp;amp; (2) License Plate region recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I'm doing (1) with Raspberry pi 3 model b&lt;/strong&gt;. I find license plate candidate first by merging bounding boxes based on their similarity. In this way, i have only 1~7 license plate region proposals. And it took less than .3 seconds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now i have to reduce number of region proposal to be around only 1~2 so that i can send these images to server to do job (2). For license plate extraction, I made my own classifier function in tensorflow and the code is below. It gets proposed license plate as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I resize all license plate to be [120, 60] and converted to gray image. And there are 2 classes: 'plate', 'non_plate'. For non_plate image, i collected various image that might appear in image as background. I have 181 images for 'plate' class and 56 images for 'non_plate' for now, i trained for about 3000 steps so far and current loss is .53 . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When i did prediction on test set,  i encountered problem that for some of plate image, it doesn't recognize license plate which is very obviously license plate image from my eyes. It is okay for me to wrongly recognize non plate image as plate but it is problem if it wrongly recognize plate as non_plate because it will not be sent to server to be fully recognized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It happens like 10 out of 100 test images and this rate is far worse than i expected. I need help for adressing this problem. Would there be any improvement that i can make? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(1) Is my training set too small to classify between license plate&#xA;    and non license plate? Or is number of steps is too small?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(2) Is my graph structure bad?&lt;/strong&gt; I needed to have small graph&#xA;    structure for my raspberry pi to recognize less than 1 second. Could&#xA;    you suggest better structure if it is bad?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(3) Is it bad to resize any proposed image to [120, 60] to be used&#xA;    as input for graph?&lt;/strong&gt; I think it loses some information. But isn't&#xA;    this close to roi pooling like used in fast rcnn?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; inputs=tf.reshape(features[FEATURE_LABEL],[-1,120 , 60 ,1],name=&quot;input_node&quot;) #120 x 60 x 1, which is gray&#xA;&#xA;conv1=tf.layers.conv2d(inputs=inputs,&#xA;                       filters=3,&#xA;                       kernel_size=[3,3],&#xA;                       padding='same',&#xA;                       activation=tf.nn.leaky_relu&#xA;                       )&#xA;#conv1 output shape: (batch_size,120,60,3)&#xA;&#xA;pool1=tf.layers.max_pooling2d(inputs=conv1,pool_size=[2,2],strides=2,padding='valid')&#xA;&#xA;#pool1 output shape: (batch_size,60,30,3)&#xA;&#xA;conv2=tf.layers.conv2d(inputs=pool1,filters=6,kernel_size=[1,1],padding='same',activation=tf.nn.leaky_relu)&#xA;&#xA;#conv2 output shape: (batch_size, 60,30,6)&#xA;&#xA;pool2=tf.layers.max_pooling2d(inputs=conv2,pool_size=[2,2],strides=2,padding='valid')&#xA;&#xA;#pool2 output shape: (batch_size, 30,15,6)&#xA;&#xA;conv3=tf.layers.conv2d(inputs=pool2,filters=9,kernel_size=[3,3],padding='same',activation=tf.nn.leaky_relu)&#xA;&#xA;#conv3 output shape: (batch_size, 30,15,9)&#xA;&#xA;pool3=tf.layers.max_pooling2d(inputs=conv3,pool_size=[2,2],strides=2,padding='valid')&#xA;&#xA;#pool3 output shape: (batch_size, 15,7,9)&#xA;&#xA;&#xA;#dense fully connected layer&#xA;pool2_flat=tf.reshape(pool3,[-1,15*7*9]) #flatten pool3 output to feed in dense layer&#xA;&#xA;dense1=tf.layers.dense(inputs=pool2_flat,units=120,activation=tf.nn.relu)&#xA;&#xA;logits=tf.layers.dense(dense1,2) #input for softmax layer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vHSZu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vHSZu.jpg&quot; alt=&quot;training non plate image example&quot;&gt;&lt;/a&gt;&#xA; [training non plate image example]&#xA;[&lt;img src=&quot;https://i.stack.imgur.com/EX3VO.png&quot; alt=&quot;training plate image example&quot;&gt;]&lt;a href=&quot;https://i.stack.imgur.com/EX3VO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;&#xA;[training plate image example. It is region proposed image]&lt;/p&gt;&#xA;" OwnerUserId="12090" LastEditorUserId="12090" LastEditDate="2018-05-27T16:50:50.263" LastActivityDate="2018-05-28T00:43:55.777" Title="Detecting license plate using tensorflow" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;tensorflow&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6527" PostTypeId="1" AcceptedAnswerId="6569" CreationDate="2018-05-27T16:33:37.543" Score="3" ViewCount="519" Body="&lt;p&gt;There are many books, courses, etc. out there, but not sure which path to take. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what would be the most effective way (shortest) to learn natural language processing online?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. I mean learning fundamentals, not how to use existing libraries or services.&lt;/p&gt;&#xA;" OwnerUserId="15902" LastActivityDate="2019-03-14T17:47:45.863" Title="What is the most effective way to learn natural language processing online?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ClosedDate="2020-03-14T18:30:12.807" ContentLicense="CC BY-SA 4.0" />
  <row Id="6529" PostTypeId="2" ParentId="6526" CreationDate="2018-05-28T00:43:55.777" Score="2" Body="&lt;p&gt;Such a task is getting easier to complete, but there are still difficulties with rotations, skewing, scaling---to name a few issues. Your network has the benefit of simplicity and lightness for the target hardware, but it may suffer under the above conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So 237 images (181+56) may be small for a &quot;generic&quot; approach, depending on how representative and diverse the dataset is. Also, the dataset is unbalanced (the first class has twice as many examples), which causes bias in learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to expand a base dataset:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Transform images and add them (with the same label, if supervised learning) to the dataset. Many libraries allow to rotate, skew, scale, or even blur images. Be careful as transforming needs be &quot;reasonable&quot;, and changing image format over and over creates artifacts that can confuse the machine (e.g. too much JPEG savings on transformed images).&lt;/li&gt;&#xA;&lt;li&gt;Generate synthetic data. Assuming license plates have a known format, it may be easy to generate images with good fidelity to real plates. This is not always possible, but license plates are formalized, so there should be only a handful patterns (typically plain, light, diplomatic, military vehicles).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Aside the dataset (potential) issue, the graph is fine. However, it may be worth trying different settings. It really depends on how you have ended up with the current graph. Removing pooling layers helps keeping more information, as well as a larger input image. 120x60 looks pretty small, and the comparison to fast-RCNN's RoI pooling layer looks odd, given that &lt;a href=&quot;https://arxiv.org/abs/1504.08083&quot; rel=&quot;nofollow noreferrer&quot;&gt;RoI comes after the feature maps&lt;/a&gt;. So a larger input image could give better results.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-05-28T00:43:55.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6530" PostTypeId="2" ParentId="6527" CreationDate="2018-05-28T12:24:36.340" Score="2" Body="&lt;p&gt;I think you could learn NLP ,by taking this course &lt;a href=&quot;https://see.stanford.edu/Course/CS224N&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://see.stanford.edu/Course/CS224N&lt;/a&gt;&#xA;Natural Language Processing - Stanford University | Coursera and try your best with the homework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further study, you could learn some deep learning algorithms like RNN. And of course, it’s better that you will be able to learn more a lot from university professors with minimal time period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it can help you.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-05-28T12:24:36.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6531" PostTypeId="2" ParentId="6499" CreationDate="2018-05-28T13:11:06.683" Score="1" Body="&lt;p&gt;I assume that you are familiar with homogeneous transformations and the meaning of &lt;em&gt;global&lt;/em&gt; and &lt;em&gt;local&lt;/em&gt; coordinate frames. If not, global frame is the fixed frame; a reference frame for your whole problem, such as the starting position of your robot. Local frame should be placed anywhere &lt;strong&gt;on&lt;/strong&gt; your robot, preferably in the middle-point of the virtual line (called &amp;quot;&lt;em&gt;robot base&lt;/em&gt;&amp;quot;) that connects the two actuating wheels in the back of your robot (given that you follow the differential drive setup). If not, just place the local frame anywhere that makes sense on the robot, such as its geometrical center.&lt;/p&gt;&#xA;&lt;p&gt;Answering your questions:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How do I know where the lines intersect?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The accepted answer in &lt;a href=&quot;https://stackoverflow.com/questions/12747955/how-do-i-quickly-find-the-closest-intersection-in-2d-between-a-ray-and-m-polylin&quot; title=&quot;This&quot;&gt;this&lt;/a&gt; is &lt;strong&gt;by far&lt;/strong&gt; the best I have seen around, which I have also used successfully for a project regarding robotic exploration in an unknown maze.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How do I find the angles of the lines using computer vision?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You &lt;strong&gt;DON'T&lt;/strong&gt; need computer vision for that. For every line, pick 2 points expressed in global frame (x1,y1),(x2,y2) and calculate the slope of the line as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lambda = (y2-y1) / (x2-x1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then the angle of the line is &lt;code&gt;atan(lambda)&lt;/code&gt;, in global frame. Do this for all lines and then subtract the angles of any two lines to find their relative angle (pay attention to the sign).&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, I would personally use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_sample_consensus&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;RANSAC&quot;&gt;RANSAC&lt;/a&gt; algorithm to de-noise the detected points and give me the line equation based on the consensus of all points. This line equation should already have the slope in it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y = ax + b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;em&gt;a&lt;/em&gt; is the slope and &lt;em&gt;b&lt;/em&gt; is the vertical offset. Then do the aforementioned steps, i.e. atan(a) and subtraction to find the relative angle between two lines.&lt;/p&gt;&#xA;&lt;p&gt;If you &lt;strong&gt;explicitly&lt;/strong&gt; want to use computer vision, maybe train a neural network on known angles and then classify images of lines to output their angles. This approach will be by far the most painful and I do NOT recommend it at all.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How do I update this information based on my coordinates?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This can be quite tricky. If the line points are detected by your on-board camera, you first of all need to convert them from the camera's coordinate frame (found in the camera's datasheet) to your robot's local frame, as explained above. To do this, you need to calculate the static transformation between the two frames (static because it will never change if the camera is fixed on the robot, you only define it once).&lt;/p&gt;&#xA;&lt;p&gt;Then you need to convert the detected line equation from your robot's frame to the global frame. In order to do so, you need to keep track of the robot's pose (position + orientation) while it moves in the 2D or 3D space. This is generally a hard task because it needs an excellent implementation of your localization algorithms, such as an Extended Kalman Filter or Particle Filter. The localization algorithm will provide you with the information that you need at any point in time in order to convert the lines/points from your robot frame to the global frame and visualize them.&lt;/p&gt;&#xA;&lt;p&gt;In short: you need to transform the detected lines / points from &lt;strong&gt;camera frame -&amp;gt; robot (local) frame -&amp;gt; global frame&lt;/strong&gt;. The first transformation is static (calculated once and never changes) whereas the second one is dynamic (changes every time the robot moves or turns). The first one is fairly easy to calculate but the second one can be a real pain.&lt;/p&gt;&#xA;" OwnerUserId="15919" LastEditorUserId="22079" LastEditDate="2020-07-29T23:01:25.673" LastActivityDate="2020-07-29T23:01:25.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6533" PostTypeId="2" ParentId="2940" CreationDate="2018-05-28T13:41:42.283" Score="-1" Body="&lt;p&gt;A very popular choice are &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hidden Markov Models&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-05-28T13:41:42.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6534" PostTypeId="2" ParentId="6154" CreationDate="2018-05-28T13:58:26.100" Score="0" Body="&lt;p&gt;This link from Stanford is by far the best resource on gradient checking that I have encountered so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cs231n.github.io/neural-networks-3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/neural-networks-3/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am sure&lt;/strong&gt; it will help you a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pro Tip&lt;/strong&gt;: make sure that you use &lt;strong&gt;&quot;centered/central difference&quot;&lt;/strong&gt; formula for the derivative calculations and also use &lt;strong&gt;&quot;relative error&quot;&lt;/strong&gt; (not absolute) to compare the two gradients.&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-05-28T13:58:26.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6536" PostTypeId="1" CreationDate="2018-05-28T21:18:44.890" Score="1" ViewCount="226" Body="&lt;p&gt;I am writing an MDP based agent that is supposed to learn to place bids and asks in a trading environment. The system requests 2 values (mWh energy and $, both being positive or negative). &#xA;Every timestep the agent has a certain volume that it has to either buy or sell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried setting these two values as action values, giving it 4 individual ones (1 for buy price and amount one sell price and amount)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the DDPG and NAF agents from keras-rl &lt;a href=&quot;https://github.com/keras-rl/keras-rl/tree/master/examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; but both aren't working for me. I tried a number of reward functions too:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;direct cash reward: average price of market for required energy vs what the agent achieved&lt;/li&gt;&#xA;&lt;li&gt;shifting balancing price: first emphasize that the broker balances it's portfolio (i.e. orders the amount it has to) and later optimize for price per mWh&lt;/li&gt;&#xA;&lt;li&gt;simple core: as a test I ran a reward function that just rewards the agent to be close to the actions [0.5, 0.55]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All three failed again. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;LR : tried between 0.01 and 0.00001&lt;/li&gt;&#xA;&lt;li&gt;Layers: Tried anything between 1 layer 1 cell and 5 layer 128 cells&lt;/li&gt;&#xA;&lt;li&gt;Types: I used both Dense and LSTM cells with according input shapes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Symptoms: Generally it looks like the system is not learning anything. I am unsure why. How does the reward function have to be structures to incentivize the system to at least move in the correct direction? Especially the reward that told the agent to be close to [0.5, 0.5] by basing the reward simply on the squared difference to this point should have worked in my eyes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XxRTf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XxRTf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11429" LastActivityDate="2018-05-28T21:18:44.890" Title="Training RL agent on timeseries trading data with Continous Deep Q or NAF" Tags="&lt;neural-networks&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6537" PostTypeId="2" ParentId="6032" CreationDate="2018-05-29T00:23:27.763" Score="1" Body="&lt;p&gt;Training with images makes use of pictures that are very large, like 7360×4912 (36 mega pixels). For image recognition at any angle and size, an image is rotated (about 60 times) and resized (depends on the original size, possibly 20 times) many times. This means you end up with a ton of data. In my example here, you would get some 60 × 36 × 2 = 4Gb of data for that one image. Although you really only save the data of interest (noise in the light signal) so in the end the AI does not use as much, but it would still be a quite large data set... especially when you know that a well trained image recognition system uses a good 2,000 images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So 10 to 20 thousands data points isn't much at all. I guess it depends how many of those you have (one per second would be 86,400 a day...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would consider using a system like Cassandra because the database can grow as big as you'd like, if that is one of the problem you're facing. Also you can save the data in binary and the number of columns can vary on each row. However, Cassandra expects a key for each row. This is an important point as it makes use of that key to distribute the data over multiple computers (for faster access later and distribution of the data set over all computers.) It also includes replication &quot;for free&quot; so if a computer fails, you can just replace it (no need to restore it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is not a very good idea with Cassandra is creating many indexes. Those use rows (i.e. one index = one row) and thus the few computers handling that one row will work extra-hard on queries on that index... But I would imagine that for AI you would not really need such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A really large CSV will be difficult to work with because you'll be responsible for accessing the rows (unless you have to access all the rows each time anyway?) Also the file format is Text and the file size will be limited by your OS (which should be really large, but be careful as many I/O calls in higher level libraries miss the fact that the size could be 64 bits.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking at a longer/more permanent solution, I would definitely look into Cassandra, especially if you're using Java as a language (also I use it with C++ and it works just fine with C++ too.) If you're just writing a throw away app. anyway, then CSV is probably much easier than handling a whole Cassandra cluster! That's a learning curve...&lt;/p&gt;&#xA;" OwnerUserId="15927" LastActivityDate="2018-05-29T00:23:27.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6538" PostTypeId="1" AcceptedAnswerId="6544" CreationDate="2018-05-29T00:27:14.697" Score="2" ViewCount="63" Body="&lt;p&gt;I'm looking at writing an AI agent for pattern recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to be able to constantly feed new data to the AI to continuously train it as new data may have new patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem, though, is that my input feed may break once in a while (the data comes from a remote computer) and thus some of the data will go missing. The other computer sends me real-time data so when the connection goes down, any new data while disconnected goes missing as far as the AI agent is concerned. (at this point, I'm not looking at fixing the gaps, although ultimately, reducing them is one of my goals, at this point I have to pretend it's not possible to accomplish.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of impact missing data has on a pattern recognition AI?&lt;/p&gt;&#xA;" OwnerUserId="15927" LastEditorUserId="23503" LastEditDate="2020-11-08T18:00:07.037" LastActivityDate="2020-11-08T18:00:07.037" Title="Will training an AI still work if the input data is somewhat sparse?" Tags="&lt;pattern-recognition&gt;&lt;incremental-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6539" PostTypeId="2" ParentId="6325" CreationDate="2018-05-29T00:36:59.610" Score="2" Body="&lt;p&gt;Though a good answer by @pasaba por aqui, I'd agree with @zooby that a graph might be too simplistic. If humans were in an environment where the options were drown or take 5000 unrelated steps to build a boat, we'd never have crossed any seas. &#xA;I think any graph, if designed by hand, would not be complex enough to call the agent within as general AI. The world would need enough in-between states that it would no longer be best described as a graph, but at least a multidimensional space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there are 2 points you'd have to consider. What is &quot;simple&quot; and when would you recognise it as a &quot;general AI&quot;. I don't find self aware AI satisfactory, as we can't measure anything called awareness; we can only see its state and its interaction with the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 1. I'd pose that the world we live in is actually fairly simple. There are 4 forces of nature, a few conservation laws, and a bunch of particle types that explain most of everything. It's just that there are many of these particles and this has led to a rather complex world. Of course, this is expensive to simulate, but we could take some shortcuts. People 200 years ago wouldn't need all of quantum mechanics to explain the world. If we replaced protons, neutrons and the strong force with the atoms in the periodic table, we'd mostly be fine. Problem is we replaced 3 more general laws with 100 specific instances. For the simulated environment to be complex enough I think this trend must hold. We could replace trillions of particles governed by general laws with thousands of instances that have different properties when interacting with the agent, and I think more importantly, when interacting with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which brings me to 2. I think we'd only truly be satisfied with the agent expressing general AI when it can purposefully interact with the environment in a way that would baffle us, while clearly benefiting from it (so not accidentally). Now that might be quite difficult or take a very long time, so a more relaxed condition would be to build tools that we'd expect it to build, thus showing mastery of its own environment. For example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Boat&quot; rel=&quot;nofollow noreferrer&quot;&gt;evidence of boats&lt;/a&gt; have been found somewhere between 100k and 900k years ago, which is about the same time-scale when &lt;a href=&quot;https://en.wikipedia.org/wiki/Homo_sapiens&quot; rel=&quot;nofollow noreferrer&quot;&gt;early humans developed&lt;/a&gt;. However although we'd consider ourselves intelligent, I'm not sure we'd consider a boat making agent to have general intelligence as it seems like a fairly simple invention. But I think we'd be satisfied after a few such inventions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think we'd need a Sim like world, that's actually a lot more complicated than the game. With 1000s of item types, many instances of each item and enough degrees of freedom to interact with everything. I also think we need something that looks familiar to acknowledge any agent as intelligent. So a 3D, complicated, minecraft-like world would be the simplest world in which &lt;strong&gt;we would recognise&lt;/strong&gt; the emergence of general intelligence.&lt;/p&gt;&#xA;" OwnerUserId="3508" LastEditorUserId="3508" LastEditDate="2018-05-29T00:53:30.087" LastActivityDate="2018-05-29T00:53:30.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6540" PostTypeId="1" CreationDate="2018-05-29T00:52:52.050" Score="1" ViewCount="235" Body="&lt;p&gt;In chapter 8 of &quot;Reinforcement Learning: An Introduction&quot; by Sutton and Barto, it is stated that Dyna needs a model to simulate the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why do we need a model? Why can't we just use the real environment itself? Wouldn't it be more helpful to use real environment instead of fake one?&lt;/p&gt;&#xA;" OwnerUserId="6851" LastEditorUserId="2444" LastEditDate="2018-11-24T04:31:22.617" LastActivityDate="2018-11-24T04:31:22.617" Title="Why do we need a model of the environment in Dyna?" Tags="&lt;reinforcement-learning&gt;&lt;models&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6541" PostTypeId="2" ParentId="6540" CreationDate="2018-05-29T06:55:15.647" Score="1" Body="&lt;p&gt;Unlike algorithms presented in other chapters of Sutton and Barto, Dyna is a &lt;em&gt;planning&lt;/em&gt; algorithm. That means that it makes decisions online, in a real environment, that attempt to be as optimal as possible given some constraints such as current knowledge and time available to compute between time steps. This differs from learning-only online algorithms which typically take a small step towards optimality on each piece of new experience as it happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A planning algorithm can only do its job well if it is allowed to &quot;look ahead&quot; at the consequences of its behaviour whilst learning online. In fact, this is the &lt;em&gt;definition&lt;/em&gt; of planning - to choose an action based on reasoning about consequences of that action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an algorithm to look ahead before taking an action, it needs a model of how the environment will respond to that action. That model does not need to be coded up directly - e.g. you don't necessarily need to write a physics engine to predict the real world (although a basic one might be a good prior or pre-training step). Instead it can be a &lt;em&gt;learned&lt;/em&gt; model, and typically in e.g. Dyna-Q, that is what you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a strong relation between Dyna-Q, and regular Q-learning with experience replay. In the most basic forms, they are essentially the same algorithm with a different framing. However, you can take the planning ideas further e.g. focus improvements around the currently experienced state and paths to a goal state in Dyna-Q, perhaps making it closer to MCTS conceptually.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Wouldn't it be more helpful to use real env instead of fake one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most real environments do not let you take actions, see the consequences and then rewind in order to re-try. Essentially that is what planning algorithms are making up for - they try to predict consequences. This is important when mistakes made during training have real consequences, for example for a physical robot navigating an environment where there might be a possibility of a fall or collision that damaged something. Whilst online learning algorithms such as SARSA will also help with this in different ways (in SARSA by changing policy to allow for exploratory moves), typically Q-learning will be weaker than Dyna-Q when it comes to learning quickly from mistakes. With the usual caveat: Much still depends on the specific problem and choices of hyperparameters.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-05-29T09:28:34.047" LastActivityDate="2018-05-29T09:28:34.047" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="6542" PostTypeId="2" ParentId="6499" CreationDate="2018-05-29T08:09:41.333" Score="1" Body="&lt;p&gt;I suggest you first consider your coordinate systems.  There are two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Field Coordinate Axis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Field boundary corners are in field coordinates (for example): { (-50.0, -35.0, 0), (50.0, -35.0, 0), (-50.0, -35.0, 0), (50.0, -35.0, 0) }, all values in meters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At any moment in time the camera in the robot: is at (x, y, z) and oriented relative to north by angle theta, measured clockwise when looking from above the field. The value of z may be 2.0 (for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Image Coordinate Axis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coordinate axes of the camera images is (w, h).&#xA;You have frames in time (perhaps every 33 msec containing grids in a the w-h coordinate axis with 1080 x 960 pixels (for example) providing an index range (&amp;lt;0, 1079&gt;, &amp;lt;0, 959&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Maintaining Orientation of Short Robots (small Z)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct that the Harris feature detection may not work because z (the distance from the surface of the field to the center of the camera lens) may not be sufficient for that algorithm unless the robot is near a corner.  The rectangle of the field boundary is not at all rectangular in the camera's w-h focal plane.  For the same reasons, finding lines and then locating their intersections is not the optimal approach either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pretend you are the robot.  As the robot survey's the field, it can assemble a model of the 360 degree periphery.  What it sees is a gradually curved line with three upside down v shapes representing the field corners.  Unless the robot is almost on top of one of the corners, all four features that correspond to the corners of the field boundary will only vaguely appear to be corners at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mathematics of Obtuse Corner Detection&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two tangent lines stem from each corner.  They intersect at a discontinuity of the line's derivative, dw/dh, the slope in 2D phase space of the camera frame.  The angle found between these two tangent lines will usually be closer to 175 degrees than 90 degrees, yet they are still detectable because the rest of line has no other like discontinuities of slope.  From a Fourier transform perspective, the 360 degree line is actually a periodic waveform primarily comprised of the 4th, 12th, 20th, 28th, and 36th harmonics.  If you are good with that level of mathematics and you record past frames, you can exploit Fourier's series and FFTs for high accuracy in corner detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you develop your theory and your software, you may find that other aspects of play need to be considered.  It may be best to think of those aspects now.  Fortunately, if another player or official blocks a portion of the field's boundary line, it will create a discontinuity in the line itself, but not the slope of the line in the w-h plane of the camera's image.  Your implementation will need to differentiate those two types of differences, which is hardly an insurmountable problem.  Discontinuity in a line and discontinuity in its derivative are mathematically distinct naturally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Redundancy in Feedback Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the robot can sense its location and orientation in other ways and know x, y, z, and theta above with some degree of reliability, the expected location of the obtuse angles and the detected ones can be compared to determine the probability that the robot is properly detecting is orientation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions in This Context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context the questions you listed need some reorientation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I know where the lines intersect?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The line has two edges that may lie on the same pixel in many cases, so that is not easy to detect in an image with many other lines.  If the line is of a particular color, hue detection can assist in line detection.  If the above corroborative data analysis is employed, then misinterpretation of edges can be corrected quickly in real time.  Once the lines are found, the detection of dh/dw at any given point on the line can be estimated using linear regression of segments and windowing (looking at short segments one at a time).  When an otherwise relatively stable slope quickly shifts 5 or 10 degrees in angle between windows, you have a high probability that you've found a distant field corner.  A shift in 70 to 80 degrees combined with a lower h value in the frame is indicative of a corner in close proximity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I find the angles of the lines using computer vision?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edge detection, systematic elimination of candidate edges that are not likely field boundaries, and then linear regression of the best candidates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I update this information based on my coordinates?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just save them in an appropriate array of x, y, z, theta vectors, indexed by frame number.  You will probably want to keep track of what you think your robot's x, y, z, and theta values are and constantly test your assumptions against your most recent inputs.  Otherwise, your robot can become disoriented.  The more ways you can detect location and orientation, the higher reliability you will have in the overall system.  If your vision can detect some feature at each goal that will not change during the game, it may help.  Ultimately your x, y, z, and theta are the parameters in a model and the use of gradient descent and auto-correlation and other auto-correction techniques need to be applied to keep your robot's orientation model continuously updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Recommend Diving Into the Math First&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 3D trig to work all of the above out in detail is initially daunting but not that far beyond high school trig if the researcher develops some clear diagrams first and then takes the time to resurrect any rusty mathematics skills or hone some new ones.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-05-29T08:09:41.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6543" PostTypeId="2" ParentId="6510" CreationDate="2018-05-29T09:07:33.450" Score="1" Body="&lt;p&gt;Recent work achieves a similar task: Object recognition together with the bounding box (e.g. &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO&lt;/a&gt;---there are quite a few on Github too). The bounding box is not enough in your case, but it is an interesting pattern: Recognition plus some form of measurement. Such architectures could be good candidate to start with, and repurpose for stick orientation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem could also leverage the current results in gait recognition. In fact, this looks closer to the problem at hand than object recognition. An example is this &lt;a href=&quot;https://arxiv.org/abs/1601.06931&quot; rel=&quot;nofollow noreferrer&quot;&gt;model based on multiview (many pictures input) recognition&lt;/a&gt;, with a &lt;a href=&quot;https://github.com/marian-margeta/gait-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;demonstration on Github&lt;/a&gt;. Gait recognition is also popular these days, and many inspiring papers and OSS implementations are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above presents two approaches your problem could benefit from, as a &quot;combination&quot;. My gut feeling is that tilt and orientations may be easier than direction (i.e. where is the tip?).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The question calls for training a model. An alternative approach, perhaps to start with and get more insight, could be to go with &quot;standard&quot; computer vision algorithms, such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hough_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hough Transform&lt;/a&gt;. This transforms allows to find lines in an image. The mathematics are at reach, and it may work well enough for a quick demo. Also, your handle name suggests &quot;embedded mobile&quot; engineer, and a simple Hough Transform could be cheap on mobile.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-05-29T09:07:33.450" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6544" PostTypeId="2" ParentId="6538" CreationDate="2018-05-29T09:29:56.107" Score="2" Body="&lt;p&gt;First, the title mentions &quot;sparse data&quot;. Recently the expression has taken a clear meaning: The agent input is data with mostly zeros. In the question a different meaning: A &quot;sparse data stream&quot;, where data flows and vanishes sometimes. I understand the question as: &quot;Will training an AI still work if the training data stream breaks?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the explicit &quot;training data stream&quot;: The question suggests the agent has at least 2 inputs: Training data you want to feed &quot;anytime&quot;, and &quot;inference data&quot; sent to the agent for actual recognition.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This question enters (to my eye) the realm of distributed AI and multi-agent systems, and ultimately a common issue in distributed systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we cast your problem to two humans S and L communicating, when S talks to L over a reliable channel, L gets all the information. When the channel breaks, L gets nothing. Does it prevent L from living normally? It merely cuts out whatever is expected out of the conversation from S to L.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back to your scenario, whenever the data stream is broken (S), the learning agent (L) will just be unable to learn from that data source. The impact on the pattern recognition agent is bounded to &lt;em&gt;what it could have learned from the new data&lt;/em&gt;. The agent recognition performance remains constant while the data stream is interrupted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if the learning agent is just learning, and cannot perform recognition without learning, there is an architectural or implementation issue. Continuous learning entails the agent is active (performs actual recognitions) and learns out of what it does.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update, for clarification:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The performance remains constant is &quot;true&quot;, but subtle. At time &lt;em&gt;t&lt;/em&gt; some metric like precision can be 99% &lt;em&gt;with respect to what the agent has seen so far&lt;/em&gt;. Assuming continuous learning is interrupted and new recognition requests come in, the performance has &quot;two faces&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As long as new recognition requests are &quot;close&quot; to what the agent has seen so far, performance is &quot;constant&quot;---the agent still scores 99%.&lt;/li&gt;&#xA;&lt;li&gt;If the request is quite different, the performance will drop. The size of the drop depends on how different the input is.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A concrete example: The agent is trained to find mushrooms with a dataset where all images are taken in the forest. Assuming learning stops, when an image of mushroom on a concrete crack comes in, the agent will probably do worse. And it would then keep doing worse on such kind of image, as long as it cannot &quot;refresh&quot; by learning from this experience.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2018-05-29T09:56:46.847" LastActivityDate="2018-05-29T09:56:46.847" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6545" PostTypeId="1" AcceptedAnswerId="6547" CreationDate="2018-05-29T09:43:18.083" Score="2" ViewCount="227" Body="&lt;p&gt;This is more of a technical question rather than a practical one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've exported a decision tree made with python/scikit learn and would like to know what the &quot;value&quot; field of each leaf corresponds to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sE6DJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sE6DJ.png&quot; alt=&quot;decision tree&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12940" LastEditorUserId="2444" LastEditDate="2020-06-02T23:21:40.973" LastActivityDate="2020-06-02T23:21:40.973" Title="What do the values of the leaves of the decision tree represent?" Tags="&lt;python&gt;&lt;decision-tree&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6546" PostTypeId="1" CreationDate="2018-05-29T10:40:09.527" Score="1" ViewCount="55" Body="&lt;p&gt;I am very new to AI, I have a set of 3D human models that I would like to train the algorithm to identify wrist, upper arm, lower arms, etc, and distance between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, this is a regression problem. But with my very limited knowledge, most tutorial online showing me cat and dog classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any clue for me to research next? There are some paper saying to convert the 3D model to image, and use convolutional neural network for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p/s: Please don't downvote me, I am too young and too lost in this field.&lt;/p&gt;&#xA;" OwnerUserId="15755" LastActivityDate="2018-05-30T13:23:09.467" Title="Detecting Keypoint of 3D model, and distance between them" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6547" PostTypeId="2" ParentId="6545" CreationDate="2018-05-29T10:47:42.303" Score="4" Body="&lt;p&gt;Decision tree nodes are split bases on the number of data samples, these numbers indicate the number of data samples they are fit to.&#xA;In your case samples = 256. It is further split into two nodes of 154 and 102.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="9947" LastEditDate="2018-05-29T16:43:21.227" LastActivityDate="2018-05-29T16:43:21.227" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6548" PostTypeId="1" AcceptedAnswerId="6616" CreationDate="2018-05-29T11:12:46.530" Score="3" ViewCount="67" Body="&lt;p&gt;I am trying to build an agent that trades commodities in a exchange setting. What are good ways to map the action output to real world actions? If the last layer is a &lt;code&gt;tanh&lt;/code&gt; activation function, outputs range between &lt;code&gt;[-1,+1]&lt;/code&gt;. How do I map these values to real actions? Or should I change the output activation to &lt;code&gt;linear&lt;/code&gt; and then directly apply the output as an action?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's say the output is tanh activated and it's &lt;code&gt;-0.4, 5&lt;/code&gt;. I could map this to:&#xA;- -0.4 --&gt; sell 40% of my holdings for 5$ per unit&#xA;- -0.4 --&gt; sell 40% for 5$ in total&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if it was linear, I could expect larger outputs (e.g &lt;code&gt;-100, 5&lt;/code&gt;). Then the action would be mapped to:&#xA;- sell 100 units for 5$ each&#xA;- sell 100 units for 5$ total&lt;/p&gt;&#xA;" OwnerUserId="11429" LastActivityDate="2018-06-01T21:37:46.513" Title="What are good action outputs for reinforcement learning agents acting in a trading environment?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6550" PostTypeId="2" ParentId="5414" CreationDate="2018-05-29T15:35:19.783" Score="1" Body="&lt;p&gt;If i understand your question correctly , you are asking if there exists functional datasets for which there are no proven solutions based on neural networks which give substantial accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;there are many such problems for which we have data in abundancy, question answering would be one such a thing , you still cant devise a neural network architechture that reads through entire principia mathematica and then complete theorems , and point cloud processing is a also a big hurdle for neural networks considering the highly irregular datastructure , even if you voxelize a point-cloud it would be infeasible to train large convolutional networks on it . (there is also rapid progress in this direction  ,, point cloud processing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;geoffrey hinton mentioned in an AMA before 3 years that we will see neural networks that will answer questions based on videos in the next five years , but still video-question answering seems to be far away from the present technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Graph datasets are also one such area where still neural networks research is in infancy (refer &lt;a href=&quot;http://www.inference.vc/how-powerful-are-graph-convolutions-review-of-kipf-welling-2016-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.inference.vc/how-powerful-are-graph-convolutions-review-of-kipf-welling-2016-2/&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-29T15:35:19.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6551" PostTypeId="2" ParentId="5861" CreationDate="2018-05-29T15:42:02.713" Score="0" Body="&lt;p&gt;There is no such single hard and slow step in training neural networks ,  forward pass involves large number of matrix multiplications so does backward pass , even though there are highly optimized libraries for matrix multiplications neural networks act on very high dimensional (tensor) multiplications in both forward and backward passes which makes it difficult train . however backward pass would be even more slower or even intractable if we won't use backpropagation in case of large neural networks, since computing derivatives is time-taking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;refer training results in for exact number &lt;a href=&quot;https://github.com/baidu-research/DeepBench#types-of-operations&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/baidu-research/DeepBench#types-of-operations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-29T15:42:02.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6553" PostTypeId="2" ParentId="5981" CreationDate="2018-05-29T15:54:06.867" Score="1" Body="&lt;p&gt;It might be hard to implement deep reinforcement learning algorithms, especially considering your previous experience and the computing resources you have. They require almost the same (even more) GPU power. Deep reinforcement learning algorithms use deep neural networks for learning the optimal policy. Even if you are given appropriate resources, it would be tough to replicate the results of the paper, if you are a novice. &lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2019-02-15T16:21:51.720" LastActivityDate="2019-02-15T16:21:51.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6554" PostTypeId="2" ParentId="3965" CreationDate="2018-05-29T16:00:07.817" Score="0" Body="&lt;p&gt;there is no hard-and-fast-rule for feature selection , you have to manually examine the dataset and try different techniques for feature engineering . &#xA;And there is no rule that you should apply neural networks for this , neural networks are time-consuming to train , instead you can experiment with decision tree based methods(random forests ) since your data is anyway in tabular structure.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-29T16:00:07.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6556" PostTypeId="1" CreationDate="2018-05-29T17:21:07.660" Score="4" ViewCount="3562" Body="&lt;p&gt;What are feature embeddings in the context of convolutional neural networks? Is it related to bottleneck features or feature vectors?&lt;/p&gt;&#xA;" OwnerUserId="15945" LastEditorUserId="2444" LastEditDate="2020-05-03T12:08:38.843" LastActivityDate="2020-05-03T12:08:38.843" Title="What is feature embedding in the context of convolutional neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;terminology&gt;&lt;feature-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6557" PostTypeId="1" AcceptedAnswerId="6568" CreationDate="2018-05-29T19:30:10.457" Score="5" ViewCount="685" Body="&lt;p&gt;I study AI by myself with the book &quot;Artificial Intelligence: A Modern Approach&quot;. I've just finished the chapters about the Bayesian network and probabilities, and I found them very interesting. Now, I want to implement different algorithms and test them in different cases and environments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it worth it to spend time on these techniques? &lt;/p&gt;&#xA;" OwnerUserId="15949" LastEditorUserId="2444" LastEditDate="2020-03-13T22:47:56.033" LastActivityDate="2020-03-13T22:47:56.033" Title="Are Bayesian networks important to learn in 2018?" Tags="&lt;bayesian-network&gt;&lt;bayesian-statistics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6558" PostTypeId="2" ParentId="5814" CreationDate="2018-05-29T20:01:29.337" Score="1" Body="&lt;p&gt;I suggest you to go through the r-cnn paper or go through a tutorial on it . CNNs transform the image into high dimensional vector in their last layer , in case of classification this vector is sent to a &quot;softmax&quot; layer , in case of bounding box regression , four values :length , breadth , location of one of the points of the bounding box , are regressed from this vector , so if you use a cnn with one regression head you end up with one bounding box irrespective of the training set. &lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-29T20:01:29.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6559" PostTypeId="2" ParentId="4907" CreationDate="2018-05-29T20:09:31.497" Score="0" Body="&lt;p&gt;you need to understand about layerwise interpretability of neural networks before this . each layer is activated by some patterns &quot;activations&quot; that are much more complex than its previous layers (in terms of texture etc).&#xA;so deeper the network , more complex functions it can compute . therefore you can't replace a deeper network with a several shallow ones. i suggest you to skim through &lt;a href=&quot;https://arxiv.org/abs/1310.6343&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1310.6343&lt;/a&gt; , In this paper it is proven that no t/2 layered network can capture the distribution that of a t-layered one. &lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-29T20:09:31.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6561" PostTypeId="2" ParentId="6117" CreationDate="2018-05-29T20:31:35.477" Score="-1" Body="&lt;p&gt;&quot;Soft skill&quot; is assessed by our common sense and there is still no standardized way of assigning soft skill , however good IQ is often a good indicator of social intelligence and soft skill as well (except for savant cases).&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-29T20:31:35.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6562" PostTypeId="2" ParentId="5979" CreationDate="2018-05-29T20:41:26.600" Score="1" Body="&lt;p&gt;There are many approaches for training CNN on 3d data, but the decision to use a particular architecture is heavily dependant upon the format of your dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using 3d point cloud data, I would suggest you go through &lt;a href=&quot;https://arxiv.org/abs/1612.00593&quot; rel=&quot;nofollow noreferrer&quot;&gt;PointNet&lt;/a&gt; and &lt;a href=&quot;https://github.com/yangyanli/PointCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;PointCNN&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But training a CNN on 3d point clouds is very tough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a way to train CNNs by posing the 3d structure from different viewpoints (&lt;a href=&quot;https://arxiv.org/pdf/1505.00880.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiview CNNs&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But remember that training CNN on 3d data is really a tough task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you plan to use a voxelized input data format, I suggest going through &lt;a href=&quot;https://arxiv.org/abs/1711.06396&quot; rel=&quot;nofollow noreferrer&quot;&gt;VoxelNet&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are mentioning deconvolution, the most relevant paper I can come across is &lt;a href=&quot;https://arxiv.org/abs/1606.06650&quot; rel=&quot;nofollow noreferrer&quot;&gt;3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But deconvolution in its own right is a very expensive operation, which acting on 3d data makes it very hard, so I would suggest you check for alternate methods.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2020-06-12T14:09:47.167" LastActivityDate="2020-06-12T14:09:47.167" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6563" PostTypeId="2" ParentId="3972" CreationDate="2018-05-29T20:49:08.923" Score="0" Body="&lt;p&gt;Hidden layer number is an indicator of the dimension of the weight matrix , so once you train a network with certain number of  hidden layer neurons  , the weights get fixed at the point you stop training. so changing the number hidden layer neurons makes the previous weights incompatible with it due to change in weight-matrix dimenstion . even if you alter the weight matrix to work with the new network it is again equivalent to a network with randomly assigned weights. so a fully trained networks becomes a new network once you change its hidden layer dimension and gives bad performance. This hase a very little to do with the dataset.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="15935" LastEditDate="2018-05-30T05:44:00.070" LastActivityDate="2018-05-30T05:44:00.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6565" PostTypeId="2" ParentId="6540" CreationDate="2018-05-30T06:28:21.447" Score="0" Body="&lt;p&gt;Sutton's Dyna has been shown to be more effective for many problem spaces than learning systems that work without a model, yet it requires fewer processing cycles than certainty-equivalence methods.  It is advanced in that it, in parallel, builds a model and adjusts behavioral policy based on both incoming information and the model.  The goal was to integrate both identified capabilities of the human brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why do we need Model in Dyna?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The capacity to model is an essential part of the Dyna architecture and may, over time, prove to be an essential component to achieve the greater effectiveness.  Many think so, myself included.  In other words, there may be no equivalently effective mechanism than building and maintaining a model for many problem sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why can't we just utilize a real environment itself?  Wouldn't it be more helpful to use real environment instead of fake one?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real environment cannot be placed in memory for many reasons.  Primarily, it would not fit.  Furthermore, not much of it can be acquired.  Only images of the environment can be acquired and placed in memory.  The most important characteristic of images, whether they be stock prices, temperature readings, or streamed video, is that they are grossly sparse and undifferentiated representations of the environment on which we are attempting to operate.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-05-30T06:28:21.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6566" PostTypeId="2" ParentId="6524" CreationDate="2018-05-30T06:42:55.350" Score="2" Body="&lt;p&gt;When automating tasks involving text, NLP techniques are definitely the way to go! Let me be frank, though, starting from scratch and &quot;read and understand&quot; will be harder with &lt;em&gt;today's tools&lt;/em&gt; than directly reading the papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some typical required skills are below, and the bar height will depend on your mileage:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Knowledge of NLP (e.g. terminology like &lt;a href=&quot;https://en.wikipedia.org/wiki/Word_embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;embeddings&lt;/a&gt;, bag of words, even perhaps &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tf-idf&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Knowledge of NLP libraries and implementation frameworks (e.g. &lt;a href=&quot;https://stanfordnlp.github.io/CoreNLP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford's&lt;/a&gt; or &lt;a href=&quot;https://www.tensorflow.org/tutorials/text_classification_with_tf_hub&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Knowledge of a programming language (e.g. Python or Java).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In short, the goal you set will probably require many more papers than expected. But if you succeed only the automated reading / summary part, you have a bright future ahead (let aside &lt;em&gt;understanding&lt;/em&gt;, a way more contentious problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just for the sake of putting an example, companies like &lt;a href=&quot;https://iris.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iris.ai&lt;/a&gt; and others try to solve related problems---great potential, but definitely challenging.&lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-05-30T06:42:55.350" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6567" PostTypeId="1" CreationDate="2018-05-30T06:44:46.943" Score="1" ViewCount="88" Body="&lt;p&gt;I am using the Fceux emulator to create a Genetic Algorithm in Lua to play the 'Arkanoid' game. It is based on Atari Breakout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A member of my population contains a string of 0's and 1's.(Population size:200).&#xA;Consider a member&#xA;Every 10 frames a bit is read from the string.(Length of string is about 1000)&#xA;If it is 0 the paddle moves left, If it is 1 the paddle moves right for the next 10 frames.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I wrote an genetic algorithm that tries to find the best sequence of inputs to play the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have experimented with three types of fitness, One is to achieve maximum score, one is to try to reduce number of blocks to a minimum and the last one is to try to stay alive as long as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of the three fitness seem to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I thought that something with my crossover might be wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every generation, I print out the average fitness of all members. Some generations it increases, while in some generations it decreases.&#xA;I have tried changing the population size to 50,100,200,300.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mutation in my algorithm has a 1% Chance(If Mut_rate=1) that each of the bit will be replaced with its opposite bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now coming to the crossover, I have used yet again many methodologies.&#xA;One of them is to just select the top 20% or 30%(cr_rate)(according to their fitness) to pass on to the next generation and killing the remaining ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method is to add the top percentile to the population and use the remaining population to swap a few bits with top ones and add them into the next generation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function crossover(population,rate)&#xA;    local topp=math.floor(rate*(#population));&#xA;    top={}&#xA;    for i=1,topp do&#xA;        table.insert(top,population[i])&#xA;    end&#xA;for i=1, #population do&#xA;        local p1 = math.random(1,topp);&#xA;        local p2 = math.random(1,topp);&#xA;        --print(top[p1]);&#xA;        --print(top[p2]);&#xA;        if top[p1][2] == top[p2][2] then&#xA;            local rval = math.random(1, 10) &amp;gt; 5;&#xA;                if rval then&#xA;                    population[i] = top[p1];&#xA;                else&#xA;                    population[i] = top[p2];&#xA;                end&#xA;            elseif top[p1][2] &amp;gt; top[p2][2] then&#xA;                population[i] = top[p1];&#xA;            else&#xA;                population[i] = top[p2];&#xA;        end&#xA;        population[i][2]=0;&#xA;end&#xA;--[[&#xA;for i=topp+1,#population do&#xA;    local p1 = math.random(1,topp);&#xA;    local p2 = math.random(1,#population);&#xA;    local s='';&#xA;    local flag=0;&#xA;    s=string.sub(top[p1][1],1,no_controls/2)..string.sub(population[p2][1],(no_controls/2)+1,no_controls);&#xA;    population[i][1]=s;&#xA;    population[i][2]=0;&#xA; end&#xA;  --]]&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Population is the table of population where each member has an input string and a fitness value.(Sorted , max fitness is first).&#xA;Rate is the percentage to select the top performers.&#xA;no_controls is the size of input string.&#xA;The commented section of the code is where I perform the swap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the mutation function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function mutation(population,mut_rate)&#xA;&#xA;    local a=0;&#xA;    local b=1;&#xA;    for i=1, #population do&#xA;        for j=1, #(population[i][1]) do&#xA;            if math.random(1, 100) &amp;lt;= mut_rate then&#xA;                if string.sub(population[i][1],j,j)=='1' then&#xA;                population[i][1] = string.sub(population[i][1],1,j-1)..a..string.sub(population[i][1],j+1);&#xA;            else&#xA;                population[i][1] = string.sub(population[i][1],1,j-1)..b..string.sub(population[i][1],j+1);&#xA;            end&#xA;            end&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Mut_rate is 1. And crossover rate is 0.2 or 0.5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried changing the mutation rate from 0 to 20.&#xA;I have also tried to change the crossover rate  as 0.2,0.5,0.7.&#xA;And the fitness using no_blocks, score, time_alive.&#xA;When I run the algorithm, the average fitness of the population first increases slightly , then decreases after a few generations and then remains constant forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paddle also seems to be performing  the same moves over and over again, which made be think that there might not be enough variation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need help, because I have been stuck on this for a few days now.&#xA;I need suggestions on what would be a suitable crossover and mutation function and a perfect fitness function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="15944" LastActivityDate="2018-05-30T06:44:46.943" Title="Genetic Algorithm to Play Arkanoid(Nes) Possible Crossover and Fitness?" Tags="&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6568" PostTypeId="2" ParentId="6557" CreationDate="2018-05-30T07:21:39.073" Score="4" Body="&lt;p&gt;*AI, A Modern Approach,&quot; was given that title to break from previously narrow approaches to duplicating desirable qualities of human thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although Bayesian networks require somewhat resource intensive computational elements, the importance of Bayesian inference and probability are still of paramount importance in that some of the highest scientific thinking require mastery of them.  Furthermore, developing in silicon dies (or possibly graphene nanites) the machinery to perform elementary probability computations in massively parallel architectures may arise over the next few decades.  The use of video DSP circuits to implement ANNs is a notable segue into this kind of development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would not dismiss the techniques you just read about.  If your intention is to capitalize on the recent crazes, you may enter the river of wannabees chasing every current trend, implement many systems that other people will rewrite later, and have a meaningless yet profitable career.  I would recommend following your inquisitiveness instead.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-05-30T07:21:39.073" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6569" PostTypeId="2" ParentId="6527" CreationDate="2018-05-30T08:02:41.030" Score="1" Body="&lt;p&gt;&lt;strong&gt;Start Without Code, Begin with Language&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best approach to learning the field of natural language processing is to first read about linguistics to understand phonetics and the other aspects of human language that underlies familiar things like grammar, spelling, interpersonal communication, listening skills, reading, writing, and publication.  Starting that way will help you weed through the garbage and identify the contributions of value in the natural language field.  (There is quite a high percentage of garbage in print.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning that vocal sound and the fundamental units of communication are central to natural language parsing and processing and language synthesis, not the abstract concepts we learned in school will give you a framework for more specific NLP education.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By fundamental unit, I mean units larger than one word like, &quot;slight of hand,&quot; and units smaller than one word like the prefix, &quot;sub-,&quot; which can be used to construct, &quot;substellar,&quot; the meaning of which can be accurately guessed even though it doesn't spell check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Avoiding Trendy Books and Courses&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend avoiding books and courses that talk primarily about speech recognition, syntax, semantic trees, and speech synthesis.  Find materials that understand what language is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Central Concepts to Guide Understanding&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process of language begins with ideas in one mind (whether that mind is human or synthetic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To create speech, ideas must be serialized into a string of signs that conform, not to a set of rules, but a set of associations between ideas and the sounds of basic linguistic units.  (Writing is a later, less developed invention, both in human history and in a child's language development.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sounds of the basic linguistic units are performed by the lungs and vocal instrumentation and pass to the cochlear organs connected to the recipient mind.  There the serialized ideas are subjected (if the person or computer is hearing) to reassembly of the serialized stream of basic linguistic units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recipient can be hearing but not listening.  If the person is also listening, the serialized stream is then processed to attempt the cognitive and emotional reconstruction of the original ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that ideas are NOT serial data.  They are hugely associative and supported by layered models.  They are also story-centric: They depend largely on the recipient's understanding of culturally common stories and themes.  When a child says, &quot;Promise?&quot; there is an entire array of ideas about parental reliability and the value of the expectation to the child that goes with that single sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that cognitive and emotional reconstruction are always processed subjectively first.  In other words, the ideas of the speaker (or writer) are reassembled from the serial stream of signs according to the attentions, interests, comprehension, and current state of mind of the listener (or reader).  If objectivity occurs at all, it is rare and usually subsequent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notion that artificial intelligence will be less subjective than natural intelligence is precarious at best and is likely to prove naive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scan Material, Pick Carefully, and Study&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the material you scan, explains any of the details of the above process and how experimenters and engineers have simulated any of this machinery in Python, Java, C++, other common languages or any of the frameworks built on these, then study that material and try out any code you can get your hands on that implements those successful language processing components.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-05-30T08:02:41.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6570" PostTypeId="2" ParentId="3933" CreationDate="2018-05-30T08:49:02.057" Score="3" Body="&lt;p&gt;It is somewhat risky to discuss data independently with your learning mechanism.  There is actually no such thing as good data or a good learner.  There is only data that is good WITH a particular learner.  That is even true of human intelligence after all the standardized education and testing done today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also exceptional learners that find data to be good when most others fumble with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by good data and deep learning you mean image sets that will lead to proper categorization of unsuspected images presented in production, your intuitive understanding of statistics can provide you with a general answer.  The images on which the deep learner develops its activation weights and meta-parameters to provide adequate production behavior must be representative of the range of images that will be found in the production feeds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you intended to do a study of men and women to determine if the old belief that women are more motivated by the prospect of love and men are more motivated by the prospect of sex, you wouldn't pick 43 men and 40,000 women for the study.  The study's value is limited by the lower of the two numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can train the network with the category frequencies you have, but some deep learners may capitalize fully on feature extraction for Indian Tigers and Hyenas but exhibit an unacceptable level mis-categorization of Zebras and Giraffes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Returning to the concept above, the skew in category frequency can be accounted for by the deep learner.  It is theoretically possible to create an exceptional learner or one that is well attuned to this kind of frequency skew.  A simple approach is to develop a scheme that recognizes frequency skew and allocates additional computing resources to the training that focuses on the differentiation of similar animals with infrequent labeled training instances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't recall who has done that, but I know it has been done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways you can give extra attention to the infrequent categories manually in the code, but then it would be a less general solution and the resulting program would neither be an exceptional learner nor particularly reusable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is more cost effective to hunt for a skew resistant deep learning scheme and test its accuracy for infrequent animals than sending a photographer to Africa.  If you can find more images of the less frequent animals without a monumental effort, I would do that too.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-05-30T08:49:02.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6571" PostTypeId="1" CreationDate="2018-05-30T09:12:50.830" Score="3" ViewCount="134" Body="&lt;p&gt;I'm having trouble grasping how to output word embeddings from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot vectors as long as the vocabulary (which is too long). So, should I use a linear activation function on the output to get the word embeddings directly (and then find the closest word) or is there something I'm missing here?&lt;/p&gt;&#xA;" OwnerUserId="5558" LastEditorUserId="2444" LastEditDate="2019-04-16T22:42:23.677" LastActivityDate="2019-04-16T22:42:23.677" Title="How should the output layer of an LSTM be when the output are word embeddings?" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;word-embedding&gt;&lt;word2vec&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6572" PostTypeId="2" ParentId="1859" CreationDate="2018-05-30T10:30:47.543" Score="0" Body="&lt;p&gt;Is anybody still using Conceptual Dependency Theory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes.  Many people.  Conceptual dependencies are central to the conveyance of ideas in natural language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are just a few publications in this century building off of Schank's work or travelling in parallel with his direction in related fields.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.mitpressjournals.org/doi/pdf/10.1162/coli.2010.36.1.36105&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mitpressjournals.org/doi/pdf/10.1162/coli.2010.36.1.36105&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ubir.buffalo.edu/xmlui/bitstream/handle/10477/34469/2001-09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ubir.buffalo.edu/xmlui/bitstream/handle/10477/34469/2001-09.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://leibnizcenter.org/~sileno/articles/ICAART2014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://leibnizcenter.org/~sileno/articles/ICAART2014.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://publicservicesalliance.org/wp-content/uploads/2013/05/narrative-its-2008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://publicservicesalliance.org/wp-content/uploads/2013/05/narrative-its-2008.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.semanticscholar.org/paper/Modelling-the-way-mathematics-is-actually-done-Corneli-Martin/0787aa70bbfb3620e1a5ddf2a2f34df99dc1f6ac&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.semanticscholar.org/paper/Modelling-the-way-mathematics-is-actually-done-Corneli-Martin/0787aa70bbfb3620e1a5ddf2a2f34df99dc1f6ac&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sentic.net/senticnet-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://sentic.net/senticnet-5.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I met Roger Schank in Hartford, in 1992, during a lecture series sponsored by the AI labs of United Technologies Research Center and a few other Fortune 500 companies in the region.  His entire lecture was a series of stories in AI research.  I remember every story 26 years later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The toy NLP implementations you see in the field today pale in comparison with the story based reasoning and memory systems proposed by Dr. Schank as a probable explanation of observations that can be made about human vocal communications.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to guess the reason he moved into education.  His natural language and artificial intelligence ideas were about a century early and over the heads of most of the people that were at the lecture alongside me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you and I find his story-based reasoning and memory proposals compelling, we are probably a century too early and a bit over the heads of most in the present day NLP field.  Most of those in labs in the 1980s found Schank irritating, and people who fit comfortably into today's technology culture find him irrelevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of those I interacted with on a project from the University of Michigan in Ann Arbor don't find his work irrelevant though, and their work is in the directions he indicated.  Unfortunately the client NDA restricts me from commenting further about that project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason we should not and ultimately will not abandon the idea that we communicate in stories is because it is correct.  When a person says, &quot;It makes me want to puke,&quot; or, &quot;I love you too,&quot; the direct parse of those sentences using &quot;modern&quot; techniques are not closely related to a correct reconstruction of the idea in the mind of the speaker.  Both sentences reference a conceptual heap of interdependence that we call a story.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If two &quot;party girls&quot; are in the ladies room at a Borgore concert and one says, &quot;Hand me a roll,&quot; the interpretation of the word, &quot;roll,&quot; is conceptually dependent.  If the speaker is in a stall it means one thing.  If at the sink it means another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will always be some segment of the research community that understands this.  Those that do not may construct money-saving automatons that will answer your business's phone calls, but they will not give you a heads up on a customer relations pattern that points to a policy issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These toy NLP agents, until they develop the capabilities Dr. Schank proposed, will not recognize from phone conversations with clients that a product or service enhancement is an opportunity waiting to be exploited, and they won't tell you a story that will convince you that you would benefit from being the first to capitalize on the opportunity.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-05-30T10:46:56.560" LastActivityDate="2018-05-30T10:46:56.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6573" PostTypeId="1" CreationDate="2018-05-30T10:50:16.373" Score="3" ViewCount="578" Body="&lt;p&gt;There seems to be a major difference in how the terminal reward is received/handled in self-play RL vs &amp;quot;normal&amp;quot; RL, which confuses me.&lt;/p&gt;&#xA;&lt;p&gt;I implemented TicTacToe the normal way, where a single agent plays against an environment that manages the state and also replies with a new move. In this scenario, the agent receives a final reward of &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for a win, draw, and loss, respectively.&lt;/p&gt;&#xA;&lt;p&gt;Next, I implemented TicTacToe in a self-play mode, where two agents perform moves one after the other, and the environment only manages the state and gives back the reward. In this scenario, an agent can only receive a final reward of &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, because, after his own move, he will never be in a terminal state in which he lost (only agent 2 could terminate the game in such a way). That means:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In self-play, episodes end in such a way that only one of the players sees the terminal state and terminal reward.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Because of point one, an agent can not learn if he made a bad move that enabled his opponent to win the episode. Simply because he does not receive a negative reward.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This seems very weird to me. What am I doing wrong?  Or if I'm not wrong, how do I handle this problem?&lt;/p&gt;&#xA;" OwnerUserId="15958" LastEditorUserId="2444" LastEditDate="2020-10-31T15:21:41.460" LastActivityDate="2020-10-31T15:21:41.460" Title="How can both agents know the terminal reward in self-play reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;self-play&gt;&lt;tic-tac-toe&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6574" PostTypeId="2" ParentId="6573" CreationDate="2018-05-30T12:32:41.640" Score="3" Body="&lt;p&gt;When one agent makes a move, that move should be perceived as part of the &quot;state transition&quot; executed &quot;by the environment&quot; from the perspective of the other agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that, as a &quot;neutral third party&quot; we view the game as follows, as a sequence of states, actions and a terminal reward. I use &lt;code&gt;A&lt;/code&gt; to denote actions selected by the first player, and &lt;code&gt;B&lt;/code&gt; to denote actions selected by the second player:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;S1 -&amp;gt; A1 -&amp;gt; S2 -&amp;gt; B1 -&amp;gt; S3 -&amp;gt; A2 -&amp;gt; S4 -&amp;gt; B2 -&amp;gt; S5 -&amp;gt; A3 -&amp;gt; Terminal Reward&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the first player should only get the following observations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;S1 -&amp;gt; A1 -&amp;gt; S3 -&amp;gt; A2 -&amp;gt; S5 -&amp;gt; A3 -&amp;gt; Terminal Reward&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;note how states &lt;code&gt;S2&lt;/code&gt; and &lt;code&gt;S4&lt;/code&gt; are skipped entirely, they are not really states from the perspective of the first player, they're just halfway through the transition caused by the first player's action and are not interesting for the first player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the second player should only get the following observations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;S2 -&amp;gt; B1 -&amp;gt; S4 -&amp;gt; B2 -&amp;gt; Terminal Reward&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-05-30T12:32:41.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6575" PostTypeId="2" ParentId="6571" CreationDate="2018-05-30T12:46:58.630" Score="0" Body="&lt;p&gt;In the research papers, it is not clear how they do that. From what I understood, you need to add a dense layer after your RNN layer. This dense layer is the size of your vocabulary. From my experience, this works even for a large vocabulary (30 000 - 40 000 for me) if you have enough data. Here you don't try to reconstruct the embedding but a one-hot vector of the current word. You can then use a cross-entropy loss. This last layer will have a lot of parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will see several implementations which are using the MSE loss directly on the word embedding output. Personally I didn't succeed with this approach but if other people could share their experiences, it could be great. &lt;/p&gt;&#xA;" OwnerUserId="15961" LastActivityDate="2018-05-30T12:46:58.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6576" PostTypeId="2" ParentId="6546" CreationDate="2018-05-30T13:23:09.467" Score="1" Body="&lt;p&gt;It depends on the format of your 3D model dataset , if your dataset is made of cad models you could voxelize your dataset and train a convolutional neural network on it , but it could be very time consuming to train 3D convnets . instead you could use a multi view 2D CNN &lt;a href=&quot;https://arxiv.org/abs/1505.00880?context=cs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1505.00880?context=cs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-30T13:23:09.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6577" PostTypeId="2" ParentId="6478" CreationDate="2018-05-30T15:34:42.707" Score="0" Body="&lt;p&gt;Check to see if the determinant of S is zero before you do the inverse. If that is the case, use pseudo inverse.&lt;/p&gt;&#xA;" OwnerUserId="15964" LastEditorUserId="15964" LastEditDate="2018-05-30T20:05:19.580" LastActivityDate="2018-05-30T20:05:19.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6578" PostTypeId="2" ParentId="6571" CreationDate="2018-05-30T18:06:32.227" Score="1" Body="&lt;p&gt;Actually, LSTM is not used to get word2vec. Indeed, word2vec is extracted from corpus of words using MLP (Multi Layer Perceptron). There is a great tutorial on how to extract word2wec:&#xA;&lt;a href=&quot;http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After representing word as vectors, you feed your text to LSTM in a deep architecture which the last layer must be softmax to categorize your text.&lt;/p&gt;&#xA;" OwnerUserId="15861" LastActivityDate="2018-05-30T18:06:32.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6579" PostTypeId="1" CreationDate="2018-05-30T19:09:05.100" Score="8" ViewCount="505" Body="&lt;p&gt;I've been reading Google's DeepMind Atari &lt;a href=&quot;http://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf&quot; rel=&quot;noreferrer&quot; title=&quot;RL&quot;&gt;paper&lt;/a&gt; and I'm trying to understand the concept of &quot;experience replay&quot;. Experience replay comes up in a lot of other reinforcement learning papers (particularly, the AlphaGo paper), so I want to understand how it works. Below are some excerpts.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;First, we used a biologically inspired mechanism termed experience replay that randomizes over the data, thereby removing correlations in the observation sequence and smoothing over changes in the data distribution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The paper then elaborates as follows (I've taken a screenshot, since there are a lot of mathematical symbols that are difficult to reproduce):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eOOTH.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eOOTH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is experience replay and what are its benefits in laymen's terms?&lt;/p&gt;&#xA;" OwnerUserId="15967" LastEditorUserId="2444" LastEditDate="2019-04-04T16:09:05.310" LastActivityDate="2020-11-01T15:16:15.973" Title="What is experience replay in laymen's terms?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6581" PostTypeId="1" AcceptedAnswerId="6586" CreationDate="2018-05-30T19:56:10.023" Score="3" ViewCount="301" Body="&lt;p&gt;I've been reading Google's DeepMind Atari &lt;a href=&quot;http://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;RL&quot;&gt;paper&lt;/a&gt; and I'm trying to understand how to implement &lt;code&gt;experience replay.&lt;/code&gt; My question is that whether we update the parameters $\theta$ of function $Q$ once for all the samples of the minibatch or we do that for each sample of the minibatch separately? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the following code from this paper, it performs the gradient descent on loss term for the $j$-th sample. However, I have seen other papers (referring to this paper) that say that we first calculate the sum of loss terms for all samples of the minibatch and then perform the gradient descent on this sum of losses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ow0HM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ow0HM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15967" LastActivityDate="2018-05-31T11:57:15.217" Title="Implementing experience replay in reinforcement learning" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6582" PostTypeId="2" ParentId="81" CreationDate="2018-05-30T22:19:10.613" Score="0" Body="&lt;p&gt;Statistical AI is widely used in finance for asset management (particularly hedge funds) and trade execution looking at high-speed small data sets, lots of HMMs and SSMs, but nobody talks about it because it provides proprietary riches.&lt;/p&gt;&#xA;" OwnerUserId="15970" LastEditorUserId="2444" LastEditDate="2019-06-27T23:46:25.427" LastActivityDate="2019-06-27T23:46:25.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6583" PostTypeId="1" CreationDate="2018-05-31T05:59:49.250" Score="3" ViewCount="154" Body="&lt;p&gt;So I was wondering about the actual constraints in the field of Artificially Intelligent agent/robot. We have been able to replicate most of human sensory functions like:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Vision - Computer Algorithms can perform better image classification tasks.&lt;/li&gt;&#xA;&lt;li&gt;Touch - A physical object more sensitive than skin can be modelled easily.&lt;/li&gt;&#xA;&lt;li&gt;Taste and smell - Although there are electro-mechanical tongues, since taste and smell and related it is quite complex to model (i.e. no satisfactory devices exist).&lt;/li&gt;&#xA;&lt;li&gt;Hearing - All types of high fidelity audio recording devices are available.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;We have also got processors for these sensors:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Memory - We have got memory running into exabytes probably.&lt;/li&gt;&#xA;&lt;li&gt;Processing power - We have got a high processing power, albeit it falls somewhat short of human brain according to this &lt;a href=&quot;https://ai.stackexchange.com/questions/5239/what-makes-animal-brain-so-special&quot;&gt;answer&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Algorithms - We have got highly developed purely logical algorithms, or we can easily Machine Learn and approximate results of unknown algorithms.&lt;/li&gt;&#xA;&lt;li&gt;We can also model the human anatomy of various physical functions like walking/running/exercising, etc, like &lt;a href=&quot;http://news.mit.edu/2015/cheetah-robot-lands-running-jump-0529&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; mechanical cheetah my MIT students.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So my question is what is stopping us from stringing together all this to actually device a quite good Artificially Intelligent agent? (Some people would like to call a good AI as AGI, but I am not defining a good agent as an AGI as I feel the definition of ntelligence is not clear enough, to make comments about AGI, hence feel free to use your own definition of agood AI). What are the constraining factors and how are we trying to overcome it?&lt;/p&gt;&#xA;&lt;p&gt;My wording maybe little bit off, you are free to edit the question keeping the general theme same.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2021-01-30T15:04:49.117" LastActivityDate="2021-01-30T15:04:49.117" Title="Constraints of embedded Artificial Intelligence for a robot" Tags="&lt;robots&gt;&lt;brain&gt;&lt;sense&gt;&lt;embedded-design&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="2" ClosedDate="2021-01-30T17:18:29.623" ContentLicense="CC BY-SA 4.0" />
  <row Id="6584" PostTypeId="1" CreationDate="2018-05-31T11:16:11.123" Score="2" ViewCount="236" Body="&lt;p&gt;How will we recognize a conscious machine (or AI)? Is there any consciousness test? For example, if a machine is aware of its previous experiences, can it be considered conscious?&lt;/p&gt;&#xA;" OwnerUserId="15978" LastEditorUserId="2444" LastEditDate="2019-11-11T21:34:11.060" LastActivityDate="2019-11-11T21:49:04.557" Title="How will we recognize a conscious machine?" Tags="&lt;philosophy&gt;&lt;artificial-consciousness&gt;&lt;intelligence-testing&gt;&lt;self-awareness&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6585" PostTypeId="2" ParentId="6583" CreationDate="2018-05-31T11:41:50.670" Score="2" Body="&lt;p&gt;I am not a professional but I have been thinking a lot about AI and neural nets, so I thought I might add my 2 ct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Acting intelligently&quot; or &quot;deriving goal directed action from sensory input&quot; is actually a lot more complex than what computer processors have been doing so far. I think we are on a good path right now, but it will still be quite a while before we can have AI act in the real world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, it does not help your agent much if it has all the five senses (or more), because it will just be &lt;strong&gt;drowning in information&lt;/strong&gt;. Real brains are ignoring the majority of data that they receive. Right now, you are not thinking about your breathing, heart beat, toes, nostrils, elbows, the noise outside or your clothes. An artificial intelligence needs a sense for &lt;strong&gt;what is worth paying attention to&lt;/strong&gt;, otherwise there is just too much to compute. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the next problem is that it is really hard to find out what is irrelevant and what is important. Deep learning teaches neural networks what inputs are relevant or irrelevant by training them to reach defined goals. The network basically tries randomly changing its computational layout and memorizes what gets it closer to the goal and what doesn't. In the end the neural net can react appropriately to a set of known inputs, which have a defined desired output. Although this is a good start, because it models the evolutionary process, it is still very basic, because the expected outputs are still pre-defined. As soon as something happens that it wasn't trained to do, the neural network doesn't know how to react. &lt;strong&gt;You would have to train it with every possible scenario in the real world&lt;/strong&gt;, which is virtually impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the agent does not only have to figure out what it should ignore at a given point in time, &lt;strong&gt;it also has to figure out why it should ignore or why it should act by itself.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think about it, what we are trying to do is to artificially replicate human intellect, or at least primitive intelligence. So we need to model the evolution of an animal brain in the real world. Just like animals, each AI agent, an offspring of a specific (genetic) layout, has to be thrown out into a &lt;strong&gt;field of laws&lt;/strong&gt; in which it can act. By &lt;strong&gt;selecting&lt;/strong&gt; layouts based on the predefined laws, &lt;strong&gt;mixing them and mutating them randomly&lt;/strong&gt;, agents which better fulfill the selection criteria become more common. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approach like this would be necessary to replicate the entire complexity of a brain. You can't design the brain yourself, bottom up, because there is just too many possible scenarios to cover. You have to model the &lt;strong&gt;entire environment&lt;/strong&gt; you want your AI to act intelligently in and &lt;strong&gt;leave it to evolve&lt;/strong&gt; and design itself, which is virtually impossible if the environment is the physical world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An outstanding source about the big picture of psychology, neuroscience, cybernetics and artificial intelligence are &lt;a href=&quot;https://youtu.be/_Jh8w6IVFs8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jordan Petersons university lectures&lt;/a&gt;. His knowledge helped me a lot with trying to understand the brain.&lt;/p&gt;&#xA;" OwnerUserId="3948" LastEditorUserId="3948" LastEditDate="2018-06-10T16:35:49.147" LastActivityDate="2018-06-10T16:35:49.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6586" PostTypeId="2" ParentId="6581" CreationDate="2018-05-31T11:57:15.217" Score="5" Body="&lt;p&gt;Gradient descent should be performed using the sum (&lt;a href=&quot;https://stats.stackexchange.com/q/201452/144483&quot;&gt;or average&lt;/a&gt;) of the losses in the minibatch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in fact also how I read the pseudocode in your question, though I understand it can be confusing. Note that, in the pseudocode, &lt;code&gt;j&lt;/code&gt; is not specified in detail. They do not, for example, have &lt;code&gt;j&lt;/code&gt; ranging from &lt;code&gt;0&lt;/code&gt; to the size of the minibatch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When they say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Sample random minibatch of transitions (phi_j, a_j, r_j, phi_{j+1}) from D&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;they mean multiple transitions in the minibatch (with minibatch size of &lt;code&gt;1&lt;/code&gt; being a special case), and they use the index &lt;code&gt;j&lt;/code&gt; to collectively refer to the entire set of indices in that randomly sampled minibatch. It's not one particular number / index, &lt;code&gt;j&lt;/code&gt; is a set of indices. When further lines of code do something with index &lt;code&gt;j&lt;/code&gt;, they actually do something with &lt;strong&gt;all indices&lt;/strong&gt; &lt;code&gt;j&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-05-31T11:57:15.217" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6588" PostTypeId="2" ParentId="6504" CreationDate="2018-05-31T16:46:28.227" Score="1" Body="&lt;p&gt;your question is quite broad, each disease has its own characteristics. And sometimes it takes a domain-expert or a pathologist to predict the severity of certain disease. you can't predict severity of all diseases with one algorithm (or a method).But in some cases you can use machine-learning methods to get assistance. I suggest you to go through &quot;grand-challenge.org&quot; competitions and read the writeups of teams which participated in those competitions to get a basic idea.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-31T16:46:28.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6591" PostTypeId="2" ParentId="6464" CreationDate="2018-05-31T17:10:16.137" Score="1" Body="&lt;p&gt;your question is a very similar to &quot;turing-test&quot;. you could narrate a simple story and ask questions based on that , considering the state-of-art algorithms in &quot;question-answering&quot; are still far beyond human skills.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-31T17:10:16.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6592" PostTypeId="2" ParentId="6459" CreationDate="2018-05-31T17:16:05.743" Score="0" Body="&lt;p&gt;recurrent-neural-networks act on a sequence of inputs , that does not need to be only a time-sequence  for example consider a sequence of characters like a passage or a book.once trained on a sequence of inputs , you could predict the previous and next values of an input-vector at a certain time step.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-05-31T17:16:05.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6593" PostTypeId="2" ParentId="6461" CreationDate="2018-05-31T17:24:11.020" Score="0" Body="&lt;p&gt;Recurrent neural networks can be trained on character level data to generate sentences which are very similar to human language. Go through this &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. You could experiment with them for compressing text.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="9947" LastEditDate="2018-07-31T01:25:06.713" LastActivityDate="2018-07-31T01:25:06.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6594" PostTypeId="2" ParentId="6573" CreationDate="2018-05-31T20:25:16.787" Score="3" Body="&lt;p&gt;If you are running self-play in a two player zero sum game, then you can do the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Arbitrarily decide the reward scheme for winning, drawing, losing is +1, 0, -1 for &lt;strong&gt;Player A&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have &lt;strong&gt;Player A&lt;/strong&gt;'s goal to &lt;em&gt;maximise&lt;/em&gt; reward, and &lt;strong&gt;Player B&lt;/strong&gt;'s goal to &lt;em&gt;minimise&lt;/em&gt; reward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This means you can combine both players' view of the values of positions and plays into a single metric, which can be learned and/or searched depending on your algorithm. When searching, you can use MCTS and/or minimax algorithms. When using Q-learning, the only tweak to apply is instead of picking the maximising action, player B will want to pick the minimising action (so will use the &lt;code&gt;min&lt;/code&gt; and &lt;code&gt;argmin&lt;/code&gt; functions where player A would use &lt;code&gt;max&lt;/code&gt; and &lt;code&gt;argmax&lt;/code&gt;) - remember when calculating TD error that you are evaluating a position for one player, but will be using reward + max/min of next player's move. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-05-31T20:43:21.903" LastActivityDate="2018-05-31T20:43:21.903" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6595" PostTypeId="2" ParentId="6584" CreationDate="2018-05-31T20:41:29.787" Score="2" Body="&lt;p&gt;I think general artificial intelligence will only be possible with some form of self awareness included. Many aspects of human communication do not work if one of the communicating partners does not have &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-awareness&quot; rel=&quot;nofollow noreferrer&quot;&gt;self awareness&lt;/a&gt;. A good example are many of today's chat bots. They seem to not even hear what they say and only rarely seem to have episodic memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Advances in machine to human communication and collaboration will eventually create systems with an ever increasing complex inner model that allows the system to interact in a natural way with humans and to fulfill tasks which require human level intelligence and flexibility. However, unless we have developed a very advanced understanding of consciousness it will be hard to judge how similar or different such a machine consciousness is compared to human consciousness. &lt;/p&gt;&#xA;" OwnerUserId="14910" LastEditorUserId="1671" LastEditDate="2018-06-01T21:02:01.717" LastActivityDate="2018-06-01T21:02:01.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6596" PostTypeId="1" CreationDate="2018-05-31T22:00:49.973" Score="2" ViewCount="146" Body="&lt;p&gt;I am looking in to building a kind of troubleshooting web application. It would be a form that starts with a first question. Depending on the answer, you get a follow up question and so on until the app has qualified your problem in to a small group of problems. To me it sounds a bit like a decision tree but what I have read about them is that it is the internal structure of a model and not what I am looking for. My guess is that a model needs all the input variables at once and not like I am looking for that feeds it one parameter at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this time I do not know of any data available. With the client we could create the desired resulting problem groups and the questions as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to solve this with the help of AI instead of hand coding a lot of case switch statements? If so could you point me to what to read up on?&lt;/p&gt;&#xA;" OwnerUserId="15895" LastActivityDate="2018-06-01T21:03:47.767" Title="Can this problem be solved by AI?" Tags="&lt;ai-design&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6599" PostTypeId="2" ParentId="6468" CreationDate="2018-06-01T10:17:32.597" Score="4" Body="&lt;p&gt;The ReLu is a &lt;strong&gt;non-linear&lt;/strong&gt; activation function. Check out this &lt;a href=&quot;https://ai.stackexchange.com/questions/5601/mathematical-intuition-for-the-use-of-re-lus-in-machine-learning&quot;&gt;question&lt;/a&gt; for the intuition behind using ReLu's (also check out the comments). There is a very simple reason of why we do not use a linear activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have a feature vector $x_0$ and weight vector $W_1$. Passing through a layer in a Neural Net will give the output as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$W_1^T * x_0 = x_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(dot product of weights and input vector). Now passing the output through next layer will give you &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$W_2^T * x_1 = x_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So expanding this we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_2 = W_2^T * W_1^T * x_0 = W_2^T * W_1^T * x_0 = W_{compact}^T * x_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus as you can see there is a linear relationship between input and output, and the function we want to model is generally non-linear, and so we cannot model it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check out my &lt;a href=&quot;https://ai.stackexchange.com/questions/5493/what-is-the-purpose-of-an-activation-function-in-neural-networks/5521#5521&quot;&gt;answer&lt;/a&gt; here on non-linear activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parametric ReLu has few advantages over normal ReLu. Here is a great &lt;a href=&quot;https://datascience.stackexchange.com/questions/5706/what-is-the-dying-relu-problem-in-neural-networks&quot;&gt;answer&lt;/a&gt; by @NeilSlater on the same. It is basically trying to tell us that if we use ReLu's we will end up with a lot of redundant or dead nodes in a Neural Net (those which have a negative output) which do not contribute to the result, and thus do not have a derivative. Thus to approximate a function we will require a larger NN, whereas parametric ReLu's absolve us of this problem,(thus a comparatively smaller NN) as negative output nodes do not die. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; &lt;code&gt;alpha = 1&lt;/code&gt; will be a special case of parametric ReLu. There must be a balance between the amount of liveliness you want in the negative region vs the linearity of the activation function.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="16929" LastEditDate="2018-08-23T15:07:00.060" LastActivityDate="2018-08-23T15:07:00.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6603" PostTypeId="2" ParentId="6583" CreationDate="2018-06-01T13:52:08.453" Score="2" Body="&lt;p&gt;&lt;strong&gt;Just a little bit of a glimpse.&lt;/strong&gt;&#xA;We are in this age of artificial narrow intelligence,where by many various applications are in phase of development, based on the case scenarios given in the question.ie computing power is out but not to the full requirement of artificial intelligent agent nor robot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the Microsoft co-founder said in &lt;a href=&quot;https://www.technologyreview.com/s/425733/paul-allen-the-singularity-isnt-near/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIT technology review&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That we can't just run away through this software development but to achieve a fully intelligent agent,it's going to take us some time.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-06-01T13:52:08.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6604" PostTypeId="2" ParentId="3343" CreationDate="2018-06-01T16:33:30.030" Score="0" Body="&lt;p&gt;If your bot is &quot;remembering&quot; few categories and then answers the questions , then it is quite useless in the current scenario. because in that case it performs very poorly on a different dataset (test-set). in statistics terminology it is called &quot;overfitting&quot;. and coming to question answering , there is no rule of thumb to define &quot;state-of-art&quot; algorithms. although you can check a few models which performed nicely on babi or similar datasets liked dynamic memory networks or seQ2seQ models. for getting a basic idea of this field , i would suggest you to learn basic machine learning lingo and then move on to some advanced-natural language processing course (stanford offers cs224n).&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T16:33:30.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6606" PostTypeId="2" ParentId="2634" CreationDate="2018-06-01T16:40:32.960" Score="1" Body="&lt;p&gt;&quot;semantic network&quot; is way of representing &quot;semantic&quot; relations in form of a &quot;graph&quot; . where as &quot;lexical semantic network&quot; is a type of semantic network which represents the relations between words , sub-words or some-other linguistic related terms. so in other words , lexical semantic networks are a type of semantic networks dealing with language relationships.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T16:40:32.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6607" PostTypeId="2" ParentId="4460" CreationDate="2018-06-01T17:22:14.917" Score="0" Body="&lt;p&gt;I am not quite sure what you have meant by your last paragrah , but let me put it this way :&#xA;what happens if an a.i fails to find a pattern in data fed to it .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;since you are mentioning feeding &quot;data&quot; to &quot;ai&quot;  i will narrow it down to a simple learning-model which learns from &quot;experience&quot; (from data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&quot;A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.&quot;&lt;/em&gt; (definition by tom mitchell).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so if your ai model fails to find a &quot;structure&quot; in the data fed to it , the model is said to &quot;underfit&quot; provided that there is a &quot;structure&quot; in the data fed to it.In  such a case change the model (mostly the algorithms associated with it) and then try again . This is because you are mentioning there is an infinite supply of data to it. But if there is really no &quot;structure&quot; in data , the data is said to be a &quot;perfectly random&quot;. for more on that visit &lt;a href=&quot;https://math.stackexchange.com/questions/2056780/is-it-possible-to-generate-truly-random-numbers-using-a-computer&quot;&gt;https://math.stackexchange.com/questions/2056780/is-it-possible-to-generate-truly-random-numbers-using-a-computer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T17:22:14.917" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6608" PostTypeId="1" CreationDate="2018-06-01T17:33:15.370" Score="4" ViewCount="104" Body="&lt;p&gt;I found several methods for setting the compatibility distance in NEAT: some normalize it, some don't, some automatically adjust it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a few tests I am running, using normalized static compatibility distance, the number of species increase very rapidly, thus suggesting to adjust (e.g. increase) the compatibility distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found however, how to determine a reasonable number of species for my population, which are the benefits of having lots/few species and which are the benefits of having stable vs mutable number of species?&lt;/p&gt;&#xA;" OwnerUserId="13087" LastActivityDate="2018-06-01T17:33:15.370" Title="Speciation in NEAT - Advantages of keeping stable number of species" Tags="&lt;neat&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6609" PostTypeId="2" ParentId="6583" CreationDate="2018-06-01T17:51:43.930" Score="3" Body="&lt;p&gt;I think a big problem with intelligent robots is that the world is very dynamic and always changing and our techniques right now are still quite static and not really flexible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I myself come from a Computer Vision background and in this field I often see some limitations of one of the most promising approaches for AI right now (Deep Learning). For example in Computer Vision we are now able to teach image classification systems that are on the level of human performance (or better) thanks to Deep Learning. But still these classifiers are trained supervised with annotated data and should more be viewed as very smart filters instead of true AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a dynamic world a supervised trained model often fails in specific situations. For example you cannot always train to classify specific objects to specific predefined classes. There are corner cases or no clear boundaries between different classes in some situations that depend on the very specific situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider an autonomous car that classifies between a road and no-road to see were it is able to drive. Most of the time this task is quite straight forward but in some sitations the road border gently transforms into a field of grass or something different that is not driveable. So where would you set the border between both classes i.e. how would you annotate your training data in such a situation? &lt;strong&gt;There is no unique solution to this problem&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A supervised trained model would finally probably learn to set the border to somewhere in-between but it doesn't really tell you that there is a corner case or slight transition with a soft border or that it's not really sure about it at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there is &lt;strong&gt;problem with the encoding/decoding of information in a neural network&lt;/strong&gt;. We can apply a full image as an input into the network and that is fine but then we only get a static output of predefined classes or something similar which often is not sufficient information to describe the world to make proper decisions in every situation. I am pretty sure a neural net would be able to learn such rich information but right now there is no real good approach to encode or represent this wanted (dynamic) information in a static predefined network architecture (as far as I know).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this also brings me to the next problem: &lt;strong&gt;there is no real good way to represent uncertainity&lt;/strong&gt;. There are some approaches like &lt;a href=&quot;https://alexgkendall.com/computer_vision/bayesian_deep_learning_for_safe_ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Deep Learning&lt;/a&gt; which tries to tackle this problem but (as far as I know) this is not really practical or good enough yet. And an intelligent robot should really know its own limits and capabilities to not becoming a serious danger for anyone or itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, general learning is usually an ongoing process that means a really intelligent agent should keep learning while it is exploring the world and that best without a supervisor. There are some approaches to this like Deep Reinforcement Learning (where just a reward function is needed) which is also very promising but right now it is very sample inefficient and not easily generally applicable to every task so that it could update its behaviour in an online manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard part about it is actually to develop a reliable robot that handles these rare or special cases realliably and/or is able to identify these with its own reliable uncertainity to fail savely. This state is not achieved yet.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="13104" LastEditDate="2018-06-02T07:25:33.720" LastActivityDate="2018-06-02T07:25:33.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6610" PostTypeId="2" ParentId="6584" CreationDate="2018-06-01T20:50:31.470" Score="2" Body="&lt;p&gt;&quot;Consciousness&quot; does not have a universal definition. However, if you are really into &quot;consciousness&quot;, you should probably read about &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Searle's Chinese Room experiment&lt;/a&gt; or Marvin Minsky's &lt;a href=&quot;https://en.wikipedia.org/wiki/Society_of_Mind&quot; rel=&quot;nofollow noreferrer&quot;&gt;society of mind&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, there are many more fundamental obstacles in current AI research that we have to tackle first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, a more formal question would be about how an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificial general intelligence (AGI)&lt;/a&gt; would emerge. Even for that, there is no clear roadmap, since we are still very new to understanding the true power of neural networks or other successful AI methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/fchollet/status/837188765500071937&quot; rel=&quot;nofollow noreferrer&quot;&gt;Franchois Chollet&lt;/a&gt; said in a tweet&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For all the progress made, it seems like almost all important questions in AI remain unanswered. Many have not even been properly asked yet &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2019-11-11T21:38:07.683" LastActivityDate="2019-11-11T21:38:07.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6611" PostTypeId="2" ParentId="6584" CreationDate="2018-06-01T20:55:15.123" Score="3" Body="&lt;p&gt;There are two main subjects you need to look at to understand the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Turing Test&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Turing test, developed by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. &lt;br&gt;&lt;Sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test#Imitation_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://www.iep.utm.edu/art-inte/#H2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test (Stanford Philosophical Dictionary)&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a linguistic element, which is that intelligence can be interpreted in many legitimate ways.  Some maintain that Artificial Intelligence has not yet been achieved; other feel &lt;a href=&quot;https://ai.stackexchange.com/q/3847/1671&quot;&gt;a simple automated switch constitutes the most basic form of AI&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bear in mind that &quot;intelligence&quot; is distinct from consciousness in the sense of &quot;self awareness&quot;, but the generalized Turing Test can also be understood as a gauge of the appearance of consciousness.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads to &lt;strong&gt;Searle's Chinese Room Argument&lt;/strong&gt;.  I highly recommend reading the &lt;a href=&quot;https://www.iep.utm.edu/chineser/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford Philosophy link&lt;/a&gt;, but the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt; gives a simpler synopsis:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Chinese room argument holds that a program cannot give a computer a &quot;mind&quot;, &quot;understanding&quot; or &quot;consciousness&quot;, regardless of how intelligently or human-like the program may make the computer behave. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The real problem may be, how does one know an algorithm is truly conscious and not merely simulating consciousness?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Philip K. Dick approaches from the opposite direction in &lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;Electric Sheep&lt;/a&gt;, where one of the conclusions is that &quot;life is life&quot; whether organic or artificial.  This might be said to lead to the &quot;Duck Test&quot; for consciousness: &quot;If it looks like a duck, and quacks like a duck, then it's probably a duck.&quot;  (Dick's philosophy was heavily influenced by Christianity, and his view on artificial life may be though of as radically humanist.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A parallel philosophical argument might be &quot;how is the appearance of free will distinct from the actuality of free will?&quot;  &lt;a href=&quot;https://en.wikipedia.org/wiki/Free_will#Western_philosophy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Free will has been fruitlessly argued about&lt;/a&gt; for millennia, but whether the universe is actually deterministic or not, we perceive ourselves to have free will.  If the universe turns out not to be strictly deterministic, it wouldn't functionally change anything.  To make matters more fuzzy, true randomness is nature is only found at the quantum level and then only within certain models. Quantum uncertainty has been proposed as a basis for free will, but we don't know.  Possibly uncertainty is a condition of individuality, which must be subjective in relation to a system or other subjectivities.  Nevertheless, how do we know we are not &quot;robots&quot; acting out a pre-determined sequence of actions, which we perceive as decisions?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally very much like the idea of &lt;strong&gt;recursion as function of self awareness&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At the phenomenal level, consciousness can be described as a singular, unified field of recursive self-awareness, consistently coherent in a particular way; that of a subject located both spatially and temporally in an egocentrically-extended domain, such that conscious self-awareness is explicitly characterized by I-ness, now-ness and here-ness. &lt;br&gt;&lt;sub&gt;SOURCE: Peters, Frederic &lt;a href=&quot;http://precedings.nature.com/documents/2444/version/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Consciousness as Recursive, Spatiotemporal Self-Location&lt;/em&gt;&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd also posit that being able to read and interpret your own code is a form of self-awareness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings me to the idea that understanding (Chinese Room) is a red herring, and what we're really talking about is interpretation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are trapped in subjectivity, humans and automata, and perfect certainty is only achievable in a very limited set of circumstances, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved games&lt;/a&gt;. (There is also the idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Complete_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;completeness of a system&lt;/a&gt; vs. &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems&quot; rel=&quot;nofollow noreferrer&quot;&gt;consistency&lt;/a&gt;.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we talk about understanding in the sense of anything abstract, by which I means &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantics&lt;/a&gt; as opposed to &lt;a href=&quot;https://en.wikipedia.org/wiki/Syntax_(programming_languages)&quot; rel=&quot;nofollow noreferrer&quot;&gt;syntactics&lt;/a&gt;, meaning vs. form, one can say that understanding is a function of interpretation, regardless of the accuracy of the interpretation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;--------------------------------------&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fun Speculation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've learned never to underestimate the insight of artists, and the best speculative fiction authors are narrative philosophers in the tradition of Plato.  Dick related memory to identity, regardless of whether the memories are real or artificial.  This might be thought of as the narrative conception of the self--I am a product of my experience.  The real &quot;me&quot; is not merely my body, but &quot;the story of me&quot;--the subjective history that led to this moment of me-ness.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is a not unreasonable assumption that artificial consciousness may arise out of understanding of narrative.  Words are just symbols, and there are all kinds of &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic&lt;/a&gt; issues, but actions have a concrete aspect.  Game theory studies actions in the form of choices, not only in the sense of &lt;a href=&quot;https://en.wikipedia.org/wiki/Nash_equilibrium&quot; rel=&quot;nofollow noreferrer&quot;&gt;equilibria&lt;/a&gt;, but also as a form of communication.   (See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#The_iterated_prisoner&amp;#39;s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterated dilemma&lt;/a&gt;)  The choices agents make in iterated dilemmas constitute a narrative history than can be analyzed and understood mathematically, and these analyses are used for decision making.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that the idea of consciousness can be related to decision making.  If you're just a transistor, that consciousness is quite limited, more akin to a cell than a complex organism. It may come down to whether one considers intelligence and consciousness to be &lt;a href=&quot;https://en.wikipedia.org/wiki/Spectrum&quot; rel=&quot;nofollow noreferrer&quot;&gt;spectrums&lt;/a&gt; as opposed to &lt;a href=&quot;https://en.wikipedia.org/wiki/Threshold_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;thresholds&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you believe consciousness to be a spectrum, then limited artificial consciousness has already been achieved. Consciousness analogous to human-level self-awareness is still in the future.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Also: &#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Philosophy_of_self#Definitions_of_the_self&quot; rel=&quot;nofollow noreferrer&quot;&gt;Definitions of the Self (wiki)&lt;/a&gt;; &lt;a href=&quot;https://en.wikipedia.org/wiki/Philosophy_of_self#Self-knowledge&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self Knowledge (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-02T00:20:00.110" LastActivityDate="2018-06-02T00:20:00.110" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6614" PostTypeId="2" ParentId="3751" CreationDate="2018-06-01T21:13:56.873" Score="1" Body="&lt;p&gt;These two are not &quot;general&quot; ways in which AI interact with humans , especially not the first one. In fact most of the applications we have complex AI algorithms deeply rooted in them , your search engine , smart-reply , translating systems all employ some kind of &quot;AI&quot; or to be more specific &quot;machine-learning&quot; algorithms in them. and many organizations are already using business-intelligence systems which you can call &quot;AI&quot; in some sense.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T21:13:56.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6615" PostTypeId="2" ParentId="4397" CreationDate="2018-06-01T21:21:19.477" Score="0" Body="&lt;p&gt;people at google and microsoft have worked so hard (for years) to make their assistants powerful enough , and it is a collective work of brilliant scientists and engineers , geoff hinton(prominent neural network researcher) once mentioned about how &quot;gaussian mixture models&quot; which is then the most widely used algorithm for speech-recognition purposes was replaced by neural networks , they probably have way more dataset and computing resources than you can ever imagine. so it would be pretty tough for you to compete with them . but if you really want to learn , nothing is stopping you, but you should know a lot more than basic &quot;machine learning&quot; and &quot;python&quot;. i would suggest you to read papers on speech recognition and question answering systems , and try to implement them on your own and replicate the results mentioned in the papers to get a basic idea.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T21:21:19.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6616" PostTypeId="2" ParentId="6548" CreationDate="2018-06-01T21:26:30.783" Score="1" Body="&lt;p&gt;&lt;strong&gt;Working Backward&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Working backward from the trading interface available to you &lt;sup&gt;Note 1&lt;/sup&gt;, you will need two things for each Exchange-Traded Fund (ETF) or other tradable commodity of another class.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An operation to perform&lt;/li&gt;&#xA;&lt;li&gt;An associated monetary amount&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The system can have an output structure &lt;sup&gt;Note 2&lt;/sup&gt; for each ETF like this (depending of course on the trading interface available to you to you and your banks primary monetary system).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ternary operation indicator (buy, sell, hold)&lt;/li&gt;&#xA;&lt;li&gt;Trade amount in USD&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why Not Just One Number per ETF?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few corroborating reasons exist for why a single positive, zero, or negative trade amount is not likely the optimal architectural choice.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The difference between holding and trading an amount of 1.0 monetary unit is not equivalent to the difference between trading 1.0 and 2.0 monetary units.  Stated mathematically, the function of profitability to trade amount is not smooth and probably not even continuous.&lt;/li&gt;&#xA;&lt;li&gt;When you implement the ternary output as two binary outputs {buy, sell}, training against the Boolean expression (buy AND sell) is likely to improve your initial performance and possibly your ongoing performance &lt;sup&gt;Note 3&lt;/sup&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Limitations on Real Trades to Consider&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because you have the limit of the assets in the liquid account from which you can buy, you will need to train against breaking the bank or using a stage after the NN outputs imposing rules or a formula based on gain and loss probabilities.  This financial constraint muddies your question because there are several ways to ensure you do not break the bank (get an insufficient funds response from your trade operation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Optimizing a Deeper Architecture&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's first assume you use probabilistic calculus to produce a closed form (formula) for how to trade based on predictions from the NN architecture you design.  Then the NN outputs might be continuous values representing the distribution of outcomes for each ETF.  Projections will almost always be dependent on investment duration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such an architecture the NN output activation scheme would be a continuous function (not necessarily linear) producing something like this &lt;sup&gt;Note 4&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mean expected delta value in one day&lt;/li&gt;&#xA;&lt;li&gt;Std deviation in one day expectation&lt;/li&gt;&#xA;&lt;li&gt;Mean expected delta four weeks&lt;/li&gt;&#xA;&lt;li&gt;Std deviation in two week expectation&lt;/li&gt;&#xA;&lt;li&gt;Mean expected delta two years&lt;/li&gt;&#xA;&lt;li&gt;Std deviation in two years expectation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any NN optimization of an investment portfolio that does not inherently deal with probability is nonsense.  Optimizing for maximum gain will introduce great risk.  Optimizing for minimum risk could result in losses.  The goal of optimization must be some representation of the balance between the desire to win and the fear of loosing, to put it in anthropological terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mean and standard deviation are obvious starting choices, and the traditional categories of short, medium, and longer term investment is also reasonable to begin with in the temporal domain &lt;sup&gt;Note 5&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pure NN&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's assume you replace the calculus with another NN scheme trained to maximize portfolio total assets &lt;sup&gt;Note 6&lt;/sup&gt;.  Such a replacement NN scheme must also take as an input your available liquid asset amount along with the above probabilistic projections.  You must also train to ensure the aggregation of buys and sells do not break your bank, that your liquid asset account never drops below zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trade amount activation should be a continuous function, but not necessarily linear and probably not tanh either because the asymptotes of that function would be counterproductive unless it is made proportional to your available liquid assets to train by aggregating your options in your architecture.  However, that's not optimal because you may find a better deal to use those assets a minute later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The odd roots (third root or fifth root or both with coefficients), when used along with training to not break the bank and to maximize the rate of portfolio growth will produce a better environment for learning in the earlier layers because of the probabilistic and aggregation realities of liquid asset limitations.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTES&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 1 &amp;mdash; Preferably a secure RESTful API, however experimentation can employ a web browser the HTTPS transactions you can control using &lt;a href=&quot;https://github.com/SeleniumHQ/selenium&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SeleniumHQ/selenium&lt;/a&gt; or &lt;a href=&quot;https://github.com/watir/watir&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/watir/watir&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 2 &amp;mdash; Input would be things like number of companies, exposure ratings provided by investment firm(s), short options, inception date, and a sequence of events, each event containing fixed point numbers and flags like these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Price per share&lt;/li&gt;&#xA;&lt;li&gt;Closing price flag&lt;/li&gt;&#xA;&lt;li&gt;Expense ratio&lt;/li&gt;&#xA;&lt;li&gt;Dividend per share&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note 3 &amp;mdash; The binary output vector {buy, sell} value of {1, 0} causes a buy.  The value of {0, 1} causes a sell.  The value {0, 0} is ignored, and {1, 1} may trigger another training operation to correct this illegal output, which is likely to be indicative of the staleness of the last round of training.  If the NN is re-entrant (reinforced), the feedback vector could include this anomalous flag or weight it heavily in an aggregation of feedback sources.  In summary, the ternary scheme can be expected to augment the training speed and resulting accuracy.  More importantly, it opens additional options for continuous optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 4 &amp;mdash; Delta is an aggregation of price increase or loss, dividends, and holding and trading expenses because that is the proper metric related to profitability of the portfolio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 5 &amp;mdash; Four weeks and two years have been chosen for the middle and longer range projections so that the time ratios are 26.09 and 28 between the three durations.  Common choices are temporally skewed.  If 1 day, 1 week, and 1 year had been chosen, the ratios would have been 7.0 and 52.18.  If 1 day, 1 month, and 1 year had been chosen, they would have been 30.44 and 12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 6 &amp;mdash; Do not assume that even a well trained NN will ever outperform formulae properly derived from probabilistic calculus for the last stage in a trading profitability architecture.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-06-01T21:37:46.513" LastActivityDate="2018-06-01T21:37:46.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6617" PostTypeId="2" ParentId="5982" CreationDate="2018-06-01T21:30:47.447" Score="0" Body="&lt;p&gt;you should definitely check about recurrent-neural networks trained on character level language data. but it make sure you have a relevant dataset.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T21:30:47.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6618" PostTypeId="2" ParentId="5672" CreationDate="2018-06-01T21:36:08.937" Score="1" Body="&lt;p&gt;It is not reverse-artificial-intelligence .not only escher , almost all paintings can be interpreted into abstract thought , in fact that is what the brain does. Even when you read something your brain processes this into its mental language (&quot;hermaneutics&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for more on &quot;mental language&quot; and &quot;abstract thought&quot; you should check &lt;a href=&quot;https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc/view&lt;/a&gt; (geoff hinton is a noted researcher in this field (ai))&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T21:36:08.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6620" PostTypeId="2" ParentId="5474" CreationDate="2018-06-01T21:41:30.590" Score="0" Body="&lt;p&gt;Neural networks construct increasingly complex representations of data on each of their layers , so you are free to choose any neural network architechture for this purpose . since the lower layers of neural network (layers near the input layer) mostly compute low level representations of the image (like gabor filters etc) most architechtures won't have much difference at this level. so you can use VGGnet if you want with proper fine-tuning from layer 3 itself.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-01T21:41:30.590" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6621" PostTypeId="2" ParentId="4271" CreationDate="2018-06-01T23:20:55.497" Score="2" Body="&lt;p&gt;&lt;strong&gt;Optimization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In optimization, the loss function (sometimes called the error function) is a function that aggregates the disparity between actual and ideal behavioral states in multiple dimensions and over a sequence of input cases.  In re-entrant (reinforced) learning, a feedback scalar or vector acts as a corrective signal that can replace or further aggregate with the loss function and additionally impact the training back-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convergence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any of these architectures, including systems without NN components but attempt to adapt by converging on an optimal static or moving target behavior, one generality can be made.  As the goal state is approached, the probability of successful convergence increases if the size of each incremental estimation decreases.  The estimation is nothing more than an informed guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Decreasing the learning rate as the detected convergence value improves is one strategy being used experimentally if not in production, but that strategy has drawbacks when used alone.  Involving the loss function in slowing down as the destination approaches is a best practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Biology Analogy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A biological system of a similar nature is the human subjective experience of pain.  As the pain level goes down, the human brain cares less about the pain, therefore the steps taken to reduce it decrease and eventually vanish.  Evolution has proven such to be advantageous for the same reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Mathematics Involved&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maximizing the probability of an unknown function to be learned adequately to perform well is done in NNs by convergence.  The term gradient descent is often used to describe the iterative process intended to converge on some ideal characterized by labeled data, a concept built into the network, or some fitness signal.  The likelihood of converging on at least a local minima (which may or may not be the global minima) is much higher if the slope decreases as the minima is approached in successive approximation scenarios.  This is when &lt;i&gt;d&lt;/i&gt;&lt;sup&gt;2&lt;/sup&gt;E / &lt;i&gt;d&lt;/i&gt;t&lt;sup&gt;2&lt;/sup&gt; is positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The geometric idea of convexity is correlated to the calculus concept of the second derivative of a line or surface with respect to time or some other measure of forward progress.  In successive approximation, the independent variable that measures forward progress could be time, computing cycles, the index of the training sample, iteration number, or some aggregation of these.  The second derivative is the rate of the rate of change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergence is more likely if the rate of change decreases as the disparity between what is perceived as optimal behavior and what is the actual current behavior.  In other terms, the relationship between risk in making the next adjustment to circuit (NN) behavior should approach zero as proximity to the optimal behavior approaches zero.  (If one senses they have worked there way to close proximity of their desired state, it makes no sense to make wild guesses.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Easy to Visualize Analogy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you drop a rubber ball into a rigid cone, it will take time to reach thermodynamic rest, at the bottom.  A lossless ball (considered impossible) will bounce forever.  A paraboloid (parabolic in two dimensions like a solar reflector) will produce faster convergence with the same rubber ball because the ball drops in energy (the sum of kinetic and potential energy) with each bounce.  The trajectory does not overshoot the bottom nearly as much or as frequently.  This analogy is not perfect, but it provides a visual image without a diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you aggregate your disparity between your target trained behavior and the current in-training behavior in a way where the second derivative is negative (concave loss function with respect to distance) on either side of the targeted ideal, convergence is much less likely.  In the analogy, the rubber ball is likely to bounce out of the flared cone altogether.  A lossless ball will always bounce out eventually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more provincial analogy is that it would be like trying to catch a baseball with the back of a baseball glove.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concave, Convex, or Zero Second Derivative&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether continuous or discrete, convex functions converge much more frequently and usually with less time and computing resource than concave ones.  A second derivative of zero is in the middle.  The word linear is actually incorrect for this zero acceleration case.  The correct term is first degree polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sum of squares over the dimensions of the domain (inputs) and over the sequence of input cases will perform well in many cases.  If you were to sum the square roots of the absolute value of error instead, your NN will rarely converge at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Executive Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following three things are more likely to be favorable if you find a way to aggregate your disparity between ideal and actual current behaviors in a way where the second derivative is greater than zero.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reliability of eventual convergence&lt;/li&gt;&#xA;&lt;li&gt;Speed of convergence&lt;/li&gt;&#xA;&lt;li&gt;Response time in the case of re-entrant (reinforced) learning&lt;/li&gt;&#xA;&lt;li&gt;Savings of computing cycles&lt;/li&gt;&#xA;&lt;li&gt;Reduction in the complexity of introspection&lt;/li&gt;&#xA;&lt;li&gt;Conservation of computational memory (RAM or SDD)&lt;/li&gt;&#xA;&lt;li&gt;Conservation of space needed for persistance and archiving&lt;/li&gt;&#xA;&lt;li&gt;Reduction of project cost to the business&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-06-01T23:20:55.497" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6622" PostTypeId="1" AcceptedAnswerId="6628" CreationDate="2018-06-02T01:21:27.370" Score="7" ViewCount="5412" Body="&lt;p&gt;I am building a NN for which I am using sigmoid function as the activation function for the single output neuron at the end. Since sigmoid function is known to take any number and return value between 0 and 1, this is causing division by zero error in back-propagtion stage because of the derivation of cross entropy. I have seen over internet it is advised to use sigmoid activation function with cross entropy loss function. So how this error is solved?&lt;/p&gt;&#xA;" OwnerUserId="15652" LastEditorUserId="9947" LastEditDate="2018-06-02T09:08:03.597" LastActivityDate="2020-06-08T00:17:19.257" Title="Cross entropy loss function causes division by zero error" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6623" PostTypeId="5" CreationDate="2018-06-02T02:37:24.650" Score="0" Body="&lt;p&gt;This tag is distinct from &quot;theorics&quot;, which should be used for theoretical subjects.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;concepts&quot; can be used for theoretical concepts, but also for concrete explanations of methods.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-02T02:37:24.650" LastActivityDate="2018-06-02T02:37:24.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6624" PostTypeId="4" CreationDate="2018-06-02T02:37:24.650" Score="0" Body="Use for broad questions on the concepts used in AI and implementations.  Breakdowns of subfields, methods, and sets of methods, with an emphasis on context and utility." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-02T02:37:24.650" LastActivityDate="2018-06-02T02:37:24.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6625" PostTypeId="2" ParentId="5471" CreationDate="2018-06-02T05:42:13.643" Score="0" Body="&lt;p&gt;your model is giving a high loss for start of the video frame , later its loss is getting decreased as it runs through several frames of the &quot;same&quot; video, but again after a new video occurs its loss again peaks to certain point.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-02T05:42:13.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6626" PostTypeId="2" ParentId="5011" CreationDate="2018-06-02T05:44:15.043" Score="0" Body="&lt;p&gt;you should use multiple view convolutional networks , check &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Su_Multi-View_Convolutional_Neural_ICCV_2015_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Su_Multi-View_Convolutional_Neural_ICCV_2015_paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-02T05:44:15.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6628" PostTypeId="2" ParentId="6622" CreationDate="2018-06-02T09:00:38.457" Score="10" Body="&lt;p&gt;Cross entropy loss is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7poun.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7poun.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as we know sigmoid function outputs values between &lt;code&gt;0-1&lt;/code&gt;, but what you have missed is it cannot output values exactly &lt;code&gt;0&lt;/code&gt; or exactly &lt;code&gt;1&lt;/code&gt; as for that to happen &lt;code&gt;sigmoid(z)&lt;/code&gt; will have to be &lt;code&gt;+ or -infinity&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although your compiler gives a divide by &lt;code&gt;0&lt;/code&gt; error, as very small floating point numbers are rounded off to &lt;code&gt;0&lt;/code&gt;, it is practically of no importance as it can happen in 2 cases only:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;sigmoid(z) = 0&lt;/code&gt;,in which case even though the compiler cannot calculate &lt;code&gt;log(0)&lt;/code&gt; (the first term in the equation) it is ultimately getting multiplied by &lt;code&gt;y_i&lt;/code&gt; which will be &lt;code&gt;0&lt;/code&gt; so final answer is 0.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;sigmoid(z) = 1&lt;/code&gt;,in which case even though the compiler cannot calculate &lt;code&gt;log(1-1)&lt;/code&gt; (the second term in the equation) it is ultimately getting multiplied by &lt;code&gt;1 - y_i&lt;/code&gt; which will be &lt;code&gt;0&lt;/code&gt; so final answer is 0.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are a few ways to get past this if you don't want the error at all:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Increase the precision of your compiler to float64 or infinity if available.&lt;/li&gt;&#xA;&lt;li&gt;Write the program in such a way that anything multiplied by &lt;code&gt;0&lt;/code&gt; is &lt;code&gt;0&lt;/code&gt; without looking at the other terms.&lt;/li&gt;&#xA;&lt;li&gt;Write the program in a way to handle such cases in a special way.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;code&gt;Implementation side note:&lt;/code&gt;&lt;/strong&gt; You cannot bypass divide by 0 error with your manual exception handler in most processors (AFAIK) . So you have to make sure the error does not occur at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; It is assumed that the random weight initialisation takes care of the fact that at the beginning of training it does not so happen that &lt;span class=&quot;math-container&quot;&gt;$\tilde y$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1-\tilde y$&lt;/span&gt; is 0 while the target is exactly the opposite, it is assumed that due to good training that the output is reaching near to the target and thus the 2 cases mentioned above will hold true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-04-07T10:52:38.433" LastActivityDate="2019-04-07T10:52:38.433" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6630" PostTypeId="2" ParentId="6504" CreationDate="2018-06-02T12:52:52.547" Score="2" Body="&lt;p&gt;The question is really broad---as stated by @thecomplexitytheorist---so difficult to give a meaningful answer. The following is about a clarification about the problem, and some directions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A model-based reflex agent is a blueprint describing the key components necessary to build that agent. It is an &lt;em&gt;abstract&lt;/em&gt; architecture to guide the creation of concrete agents. Whether software or hardware, the target agent should have a bit serving as &quot;sensors&quot; (a HW camera, or a software API&quot;), &quot;actuators&quot;, memory, etc. So if you aim at a HW agent, you need to decide what compenents, and collect them. A SW agent would be the definition of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Software_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;software architecture&lt;/a&gt;, or the use of some modelling framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Machine Learning part is about endowing the agent with some skill. It is the &quot;smarts&quot; programming part, what the &quot;brains&quot; will do in the instance of the blueprint. Please note that it does &lt;em&gt;not&lt;/em&gt; have to be Machine Learning. It could be another &quot;style&quot; of programming, such as a rule-based system, or a plain hand-crafted program.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;How to teach such agent is currently an open question. First steps usually start with drawing the agent as a black box, its inputs (e.g. symptom data), its desired output (diagnosis and alert). Then we detail the black box in terms of what sensors the agent needs to process the input, what actuators for the output, and what it needs to learn and memorize. Depending on the available input data and diagnosis/alert output system (email?), the next stage aims at refining step by step, until a good idea of what needs be implemented emerges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, all pieces are in place, except the internal model---the piece that really pertains to Machine Learning (if you choose ML for that). As the input data is available, and the output format decided, the &quot;final&quot; stage (before implementation) is to define the model. It really depends on the actual data and the goal (here we assume prediction). Labelled data (we can teach with input &lt;em&gt;and&lt;/em&gt; output, as we already know them) usually leads to &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;supervised learning&lt;/a&gt;. Unlabelled data (we can teach with the input only) leads to unsupervised learning or reinforcement learning. Once one understands the situation, we can choose some algorithm like SVM, decision trees, neural networks, etc. Note that the final decisions (before implementation) require studying the data beforehand (Is it regular? Are there missing bits? Is it in a format useful for ML? Etc.) to make appropriate choices.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Final note: The question may get closed on this site, because it is too broad. Way more useful to ask narrow questions, with a clear answer or set of answers. As you see here, this long answer is just the tip of how to teach an agent (I could not sleep anyway). And no implementation yet. In fact, all this could be summed up as &quot;typical system development, with an ML component&quot;. Two guiding principles: (1) isolate the ML component(s), so specialists can dig and make the best of them, and (2) keep it simple. &lt;/p&gt;&#xA;" OwnerUserId="169" LastActivityDate="2018-06-02T12:52:52.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6631" PostTypeId="2" ParentId="6515" CreationDate="2018-06-02T13:29:06.447" Score="1" Body="&lt;p&gt;there is an -init option for that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Initialization method to use.&#xA;  0 = random, 1 = k-means++, 2 = canopy, 3 = farthest first.&#xA;  (default = 0)&lt;/em&gt;&#xA;kmeans++ will give you an option for you to initialize centres.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-02T13:29:06.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6634" PostTypeId="1" AcceptedAnswerId="9459" CreationDate="2018-06-03T00:24:51.387" Score="4" ViewCount="103" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C2Wkg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C2Wkg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hello,&#xA;I would like to know whether this picture from the paper: Distributed Training of Deep Neural Networks: Theoretical and Practical Limits of Parallel Scalability valid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;br&gt;&#xA;1) Does InnerProduct (Fully connected) layer actually take &lt;strong&gt;more time to compute in&lt;/strong&gt; a neural network than Convolution? &#xA;&lt;br&gt;&#xA;&lt;br&gt;&#xA;2) Is assessing &lt;strong&gt;GLOPs/time&lt;/strong&gt; a good way of &lt;strong&gt;estimating performance&lt;/strong&gt; of different types of layers in a neural network on any hardware? (Conv, FC etc.)&#xA;&lt;br&gt;&#xA;&lt;br&gt;&#xA;3) Does anyone know where I can find GFLOPs vs compute time for different types of layers across GPUs/CPUs? (I know DeepBench, any other suggestions would be great too)&lt;/p&gt;&#xA;" OwnerUserId="16014" LastEditorUserId="15465" LastEditDate="2018-06-13T16:23:21.233" LastActivityDate="2019-01-10T15:02:55.587" Title="Relative compute time for each type of layer in a neural network" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;&lt;algorithm&gt;&lt;performance&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6639" PostTypeId="2" ParentId="4389" CreationDate="2018-06-04T05:49:46.440" Score="1" Body="&lt;p&gt;Here is a speculative cast of the problem to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;travelling salesman problem&lt;/a&gt;, which would lead to shortest-path algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note this idea suggests different constraints to explore.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given the knowledge vectors and efforts, build a acyclic directed graph (acyclic, as we are not supposed to unlearn). A vertex is an article, represented by its knowledge vector. An edge links two articles, weighted by the effort to &quot;move&quot; to the target article/vertex (i.e. &lt;em&gt;acquire&lt;/em&gt; the knowledge of that article).&lt;/li&gt;&#xA;&lt;li&gt;Assign a zero vector to a new participant. That is the starting point on the graph is vertex V0 = (0, ..., 0).&lt;/li&gt;&#xA;&lt;li&gt;Define a learning objective as a vector V.&lt;/li&gt;&#xA;&lt;li&gt;Use a shortest-path algorithm to find a (V0, V) plan.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This procedure is insufficient, as there are many ways to build the graph (in other words, the above is completely pointless &lt;em&gt;as is&lt;/em&gt;). Extra constraints are necessary to make it practical. For example, we can order the vertices by ordering them along each dimension. Such setting would lead learners to start with &quot;easy&quot; articles (V[i] is low), and move step by step toward more complex topics ((V[i] gets higher).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The graph construction depends on the data available. For example, are knowledge vectors &quot;absolute&quot;, or can they be relative? Relative can help in creating a path, as moving from V to W requires an effort that depends on your learner's initial conditions (V0 may not be 0 everywhere, afterall).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Is it an AI question? Definitely.&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2018-06-04T05:58:33.280" LastActivityDate="2018-06-04T05:58:33.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6640" PostTypeId="1" CreationDate="2018-06-04T05:58:41.443" Score="5" ViewCount="222" Body="&lt;p&gt;I'm learning fuzzy logic and more or less understand the basic concept, but i'm having a hard time understanding how to apply it to a method. I tried browsing online for explanation on how to use it, but only found some implementation and test case using the basic form of 4 rules and 3 variables, and 2 rules per variable. Anyway this is an example case, i will use Tsukamoto method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case actually i have 6 rules and 3 variables with 3 rules ver variable, but i will only explain 1 of the variables because i think the rest will have the same solution. I have 3 variables one of them is &quot;size&quot;, the range is for small it's 0-2 and for large it's 7-10. The current condition is size = 6.5. The rules is as follow(simplified to only use this variable):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[R1] size = small&lt;/li&gt;&#xA;&lt;li&gt;[R2] size = medium&lt;/li&gt;&#xA;&lt;li&gt;[R3] size = large&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What i want to know is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how do i define the formula for medium(the middle rule if the case is different)?&lt;/li&gt;&#xA;&lt;li&gt;what if the rule is more than 3 (i.e. small, medium, large, ex-large)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What i understands if the rule is only 2 i can use this formula&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;small[x]=(max-x)/(max-min)&lt;/li&gt;&#xA;&lt;li&gt;large[x]=(x-min)/(max-min)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My current approach to this problem is as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;small[x]=1; x&amp;lt;=2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;medium[x]=(max-x)/(max-min); 2 &amp;lt; x &amp;lt; 7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;large[x]=0; x&gt;=7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? Also can you refer me to some source to study this? As i mentioned before i can only find some implementation and basic explanation, it's either there is no online source for this or i don't know what to search for. Sorry if it's hard to understand i can edit and post the whole problem if you want, thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extra question: what is the name of algorithm which can be used to solve the crossing bridge puzzle(the one with timer, max person, and stuff)?i forgot the name. &lt;/p&gt;&#xA;" OwnerUserId="16039" LastEditorUserId="1671" LastEditDate="2018-06-04T20:26:43.323" LastActivityDate="2021-04-21T11:06:50.640" Title="Defining formula for fuzzy equation" Tags="&lt;getting-started&gt;&lt;math&gt;&lt;fuzzy-logic&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6642" PostTypeId="1" AcceptedAnswerId="6645" CreationDate="2018-06-04T10:40:05.463" Score="1" ViewCount="58" Body="&lt;p&gt;I have this following natural language statement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;There is only one house in area1 the size of which is less than 200m².&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is mistranslated to FOL:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;∃x.(house(x) ∧ In(x,area1) ∧ ∀y.(house(y) ∧ In(y,area1) ∧ size(y) &amp;lt; 200 -&amp;gt; x=y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This translation is wrong according to my lecturer, because it is not necessary that the size of x must be less than 200. The statement is true if there only houses which are bigger.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't get the FOL translation at all and don't see where the uniqueness part is expressed :&#xA;so translated it back : &quot;if all houses in area1 have a size less then 200m² then there exists one house which equals to all houses ??&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;why is not necessary that the size of x is less than 200, when it clearly says in the statement above that must exist one house with a size less then 200 ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15391" LastEditorUserId="2193" LastEditDate="2018-06-04T19:35:46.113" LastActivityDate="2018-06-04T19:35:46.113" Title="How is uniqueness quantification translated in First Oder Logic" Tags="&lt;logic&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6643" PostTypeId="1" AcceptedAnswerId="6653" CreationDate="2018-06-04T11:17:59.383" Score="2" ViewCount="283" Body="&lt;p&gt;I'm doing a project for my last university examination but I'm having some troubles! I'm making an expert system who should be able to assemble a computer after asking some questions to the user. It works but according to my teacher I need to define more rules, could you give me some suggestions please? I have facts like these:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;processor(P, Proc_price, Price_range),&#xA;motherboard(M, Motherboard_price, Price_range),&#xA;ram(R, Ram_price, Price_range),&#xA;case(C, Case_price, Price_range),&#xA;ali(A, Ali_price, Price_range),&#xA;video_card(V, Vga_price, Price_range),&#xA;ssd(S, Ssd_price, Price_range),&#xA;monitor(D, Monitor_price, Price_range),&#xA;hdd(H, Hdd_price, Price_range).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ask these questions to the user: 1) choose the price range 2) choose the display size 3) choose hard disk size Then I ask 3 questions about computer utilization to define the user: 1) do you surf on internet? 2) do you play? 3) do you use editing programs?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;use(gaming) :- ask(&quot;Do you play games? (y/n)&quot;).&#xA;&#xA;    use(editing) :- ask(&quot;Do you use editing programs? (y/n)&quot;).&#xA;&#xA;    use(surfing) :- ask(&quot;Do you surf internet?(y/n)&quot;).&#xA;&#xA;    user(base) :-&#xA;        use(surfing),  \+ use(gaming), \+ use(editing).&#xA;&#xA;    user(gamer) :-&#xA;        use(gaming), use(surfing), \+ use(editing).&#xA;&#xA;    user(professional) :-&#xA;        use(editing), \+ use(gaming), use(surfing).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should make more questions about user definition to make user definition more complex too and add some rules. Please help me, I'm desperate&lt;/p&gt;&#xA;" OwnerUserId="16049" LastEditorUserId="1671" LastEditDate="2018-06-04T21:25:31.737" LastActivityDate="2018-06-04T21:25:31.737" Title="Defining rules for an expert system" Tags="&lt;prolog&gt;&lt;expert-system&gt;&lt;r&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6644" PostTypeId="1" AcceptedAnswerId="6647" CreationDate="2018-06-04T12:21:33.773" Score="5" ViewCount="238" Body="&lt;p&gt;We are doing a research design project on autonomous vehicles and have some questions on AV Levels 4/5; specifically on the roles, impacts and consequences of AV on society, government, users and other stakeholders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're currently stuck on this main question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: What functionally, does control look like in AV levels 4 and 5?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, is the whole purpose of a level 4/5 that a user has no input into the control?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could a driver in AV (level 5) stop in an emergency, or say they want to &quot;take corners harder, speed up, slow down&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I choose to change the equi-distance between my AV and the others around me because I like space?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're wondering about what functionally, does AV level 4/5 offer a user; and what it looks like?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Context:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our remit is within the world of design (design thinking), not specifically technology, or expert system functionality. We're looking at the issue from a design perspective; who does it impact, who are the stakeholders, what are the consequences and impacts. What role does a driver have an in level 5? Could an auto-manufacturer want to give drivers control in level 5? How do emergency services act in these situations? What are the touchpoints to society and whom does it impact and what does it say about the design of AV for the future of society.&lt;/p&gt;&#xA;" OwnerUserId="16051" LastEditorUserId="4302" LastEditDate="2018-09-20T05:51:43.987" LastActivityDate="2018-12-14T03:36:31.080" Title="What functionality, does control look like in autonomous vehicles levels 4 and 5?" Tags="&lt;ai-design&gt;&lt;autonomous-vehicles&gt;&lt;social&gt;&lt;automation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6645" PostTypeId="2" ParentId="6642" CreationDate="2018-06-04T15:15:48.977" Score="2" Body="&lt;p&gt;According to the Wikipedia entry on &lt;a href=&quot;https://en.wikipedia.org/wiki/Uniqueness_quantification&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uniqueness Quantification&lt;/a&gt; your lecturer is correct. There is no size requirement expressed in the FOL expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point about the implication is that it can be true if the antecedent is false. So, there is a house in &lt;code&gt;area1&lt;/code&gt; (which we call x). And all houses in &lt;code&gt;area1&lt;/code&gt; which are smaller than 200 are the same as x. But if there aren't any, then the antecedent is false, and the consequence (x = y) is false, but the whole statement is still true. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As another example: &quot;If Trump is the 31st president of the USA, then the moon is made of green cheese&quot;. Both antecedent and consequence are false, but the whole statement is still logically true. Same as &quot;If there is a house in that area, and there are houses that are less than 200 (which there aren't), then that house is one of them.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moving on to the correct expression: The unique quantifier (usually written as ∃!) can be rewritten using the existential and universal quantifiers as follows (see the above mentioned Wikipedia page):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;∃x (P(x) ∧ ∀y (P(y) -&amp;gt; y = x))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is not what you have got; you have got two different predicates, P1 and P2. Your P1(x) is &lt;code&gt;(house(x) ∧ in(x, area1))&lt;/code&gt;, and your P2(x) is &lt;code&gt;(house(x) ∧ in(x, area1) ∧ size(x) &amp;lt; 200)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct expression would require the same predicate for the quantifiers and would therefore be&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;∃x ((house(x) ∧ in(x, area1) ∧ size(x) &amp;lt; 200) ∧ &#xA;    ∀y ((house(y) ∧ in(y, area1) ∧ size(y) &amp;lt; 200) &#xA;      -&amp;gt; y = x))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The difference is that you state that there is at least one house in the area with a size of less than 200. So the second predicate, that y is a house in the area with a size of less than 200, cannot be false.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-04T15:15:48.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6647" PostTypeId="2" ParentId="6644" CreationDate="2018-06-04T18:52:45.303" Score="5" Body="&lt;p&gt;&lt;strong&gt;Automation Levels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most cars have some Level 1 automation, such as cruise control and various skid/flip probability reduction systems. Most high volume passenger vehicles have higher levels. Some military and private air, land, and sea equipment are already at Level 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Level 4 requires that driving be automated during normal driving conditions, with manual override.  However, to my knowledge, no one has published a mathematically terse and comprehensive distinction between normal and abnormal to aid in testing Levels 4 vehicles, so testing to a standard is probably not yet possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For legal and political reasons, Level 5 is essentially a statistical criteria. For fully automatic to be viable as a market product for general public use, the safety data for passengers and pedestrians must indicate at least the level of safety for manually driven vehicles. Although this will likely suffice from a law and public relations purpose, it is inadequate as a quality standard for automation engineering and testing.  The ambiguities are numerous.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Statistical criteria required to pass the test (i.e. sample size, duration, randomization, and double or single blindness)&lt;/li&gt;&#xA;&lt;li&gt;Mathematically terse and comprehensive scenarios for the test&lt;/li&gt;&#xA;&lt;li&gt;Allowable proportion of level 2, 3, or 4 vehicles in the control group.&lt;/li&gt;&#xA;&lt;li&gt;Probably others&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There will be no vehicle driver in what they are calling Level 5 &amp;mdash; only passengers.  The idea is to give no power to the occupants of the AV other than destination or change in destination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has been the safety norm in other sub-sectors of transportation for a century.  Passengers cannot talk to the pilot of a jet or the engineer of a train.  In the majority of cases, safety is compromised whenever a person that has not undergone the discipline of intense safety training has control over any aspect of the vehicle's operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is the primary impetus behind AV from the forward thinkers in government and academia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Specificity and Insight&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is of paramount importance that researchers define system criteria more specifically and scientifically.  Systems architecture, software engineering, safety evaluation, and quality control policies and procedures of the automated system driving the AV requires such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a billion plus lives at stake, the design should progress with the diligence and care as if designing a human-occupied drone aircraft or a civilian Mars lander from the ground up, even if your first phase is to only achieve what is being called Level 4 for basic passenger cars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Target Reliability and Safety&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans eat food, have emotional conversations, text each other, ingest mind altering substances, and fall asleep while driving.  Judging the safety statistics of an AV by comparing with those of humans driving may sound practical, but it is absurd.  It will become clear just how absurd the popularized Level 5 criteria is as the parameters of design are enumerated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Design should instead minimize the possibility of any accident ever.  The goal should be zero mistakes, both at point of sale (the dealership) and at later points in the product life-cycle as the AV learns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Defining a Mistake&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A mistake should be defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any less than optimal state indicated by the correction signal used to direct reinforced learning in any of the system's re-entrant or coincident training mechanisms&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The adaptive (i.e. machine learning) portion of the system must permit re-entrant or coincident training (reinforced learning) because there is no possible way to predict the common routes of the buyer at the vehicle's point of sale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To comprehend the complexity of the problem space for AVs and begin to simplify it, consider the dimension of conditions, controls, and priorities (embodied in feedback signals) related to driving on roads for any vehicles that use roads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Control Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Starters (there are two in the case of most hybrids)&lt;/li&gt;&#xA;&lt;li&gt;Engine stop&lt;/li&gt;&#xA;&lt;li&gt;Breaking controls (there are two in the case of regenerative breaking)&lt;/li&gt;&#xA;&lt;li&gt;Steering shaft or hydraulic control position&lt;/li&gt;&#xA;&lt;li&gt;Break control position (three or four depending on emergency/parking break design)&lt;/li&gt;&#xA;&lt;li&gt;Transmission planetary gear clutch state, traditional automatic transmission control state, or clutch and traditional manual transmission control state&lt;/li&gt;&#xA;&lt;li&gt;External lighting switch positions (Widely variable, but at least six for headlight, high beam, break light, left signal, right signal, and tail light)&lt;/li&gt;&#xA;&lt;li&gt;The content of any messages broadcasted, multicasted, or sent specifically to any other vehicles with compatible reception (if birds do it, so can electronic systems) either via light, sound, or RF (this will require the development of layers of inter-vehicle communications protocols)&lt;/li&gt;&#xA;&lt;li&gt;Horn&lt;/li&gt;&#xA;&lt;li&gt;Probably others&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Data Acquisition Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Wheel positions (there may be 2 or 4 positions to read with an encoder since one cannot assume perfect alignment)&lt;/li&gt;&#xA;&lt;li&gt;Break torques (4 of them, which can be read by 16 redundant strain gauges)&lt;/li&gt;&#xA;&lt;li&gt;Break metal temperatures (4)&lt;/li&gt;&#xA;&lt;li&gt;Torques and temperatures for any independent emergency breaks&lt;/li&gt;&#xA;&lt;li&gt;Accelerometers (two devices x three dimensions per device to detect acceleration/deceleration, centripetal force, and, with some math, tire skid velocity for all four tires in two dimensions)&lt;/li&gt;&#xA;&lt;li&gt;Tachometers (one before the transmission and one on each wheel)&lt;/li&gt;&#xA;&lt;li&gt;Engine and coolant temperature detectors&lt;/li&gt;&#xA;&lt;li&gt;Cameras (must be high resolution to recognize animals, humans, shopping carts, curbs, road signs, train signals, speed bumps/humps, and hazards, which can be IR or visible and the more angles covered the better)&lt;/li&gt;&#xA;&lt;li&gt;Wind turbulence resistant external microphones to detect horns and sirens (at least four to detect likely orientation of audio source)&lt;/li&gt;&#xA;&lt;li&gt;Suspension strain gauges (to detect vertical road force on each tire)&lt;/li&gt;&#xA;&lt;li&gt;The content of any incoming messages from compatible systems&lt;/li&gt;&#xA;&lt;li&gt;Battery voltages and currents (two batteries for regenerative breaking or hybrid startup and motive assist, and possibly several currents)&lt;/li&gt;&#xA;&lt;li&gt;Fluid levels, pressures, viscosity, and transparency (fuel, oil, steering, transmission, break hydraulic, and possibly others)&lt;/li&gt;&#xA;&lt;li&gt;Probably others&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A system can be operational and possibly reach what most would consider Level 5 with fewer channels of acquisition and control than above, but it would be poor technology planning to start designing systems with unnecessary limitations.  Such limitations will also very likely increase the cost of engineering and the effectiveness of training while saving nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why Vehicles Have Very Little Instrumentation Today&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A human cannot make use of all of all the information above.  Nor could a human control all the channels listed above without a high frequency of mistakes.  A properly designed electromechanical learning system can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be lazy systems architecture not to capitalize on the positive impact the additional instrumentation would have on safety, total cost of ownership, and other quality criteria for the AV buyer who can afford the extra sensors and computing power.  Furthermore, after a few years of manufacturing for a mass market, the cost of the additional components will become small in comparison with the cost of metal and plastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Operational Criteria and the Formalization of a Mistake&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem space contains at least nineteen dependent variables (output channels) and forty-six independent variables (input channels).  Some are binary, some are floating point, some are streamed data, some are streamed audio, and some are streamed video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Together they form a space in sixty-five dimensions.  That is what must be optimized according to some predetermined and possibly re-programmable formalization of what is optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider this idea of optimum safety, thrift, and comfort as quality control criteria.  Real time quality control should follow TQM ideals, continuously ensuring quality in multiple dimensions and at multiple points in the total system.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maximal distance from other vehicles&lt;/li&gt;&#xA;&lt;li&gt;Maximal distance from stationary objects (bridge abutments, buildings)&lt;/li&gt;&#xA;&lt;li&gt;Maximal distance from pedestrians on foot, bike, wheelchair, ...&lt;/li&gt;&#xA;&lt;li&gt;Maximal distance from edges of pavement&lt;/li&gt;&#xA;&lt;li&gt;Minimal lane switching&lt;/li&gt;&#xA;&lt;li&gt;Minimal loads and torques on wheels&lt;/li&gt;&#xA;&lt;li&gt;Minimal fuel consumption&lt;/li&gt;&#xA;&lt;li&gt;Within operation parameters for tires, breaks, engine RPM, and dozens of other parts and subsystems&lt;/li&gt;&#xA;&lt;li&gt;Shortest distance for route to destination&lt;/li&gt;&#xA;&lt;li&gt;Shortest time distribution mean for route to destination&lt;/li&gt;&#xA;&lt;li&gt;Safest route to destination&lt;/li&gt;&#xA;&lt;li&gt;Least stops on route to destination&lt;/li&gt;&#xA;&lt;li&gt;Probably others&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Most of this must be balanced, so optimization criteria must be aggregated.  Such aggregation must go beyond the simplicity of a loss function.  Summing squares will not work at all, so give up aggregating in such a simplistic way.  A multivariate extension of the Pythagorean Theorem is find for calculating distance in linear space, but driving cars is very non-linear.  This kind of robotics system will require more thought in the formulation of balances, priorities, and the concept of emergency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further expanding on the above definition of a mistake, any real time control that does not optimize the sixty-five-plus dimensional surface is faulty.  Now what is optimal must be defined.  Consider the following quality control criteria, roughly in correct order.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pedestrian safety&lt;/li&gt;&#xA;&lt;li&gt;Passenger safety&lt;/li&gt;&#xA;&lt;li&gt;Mechanical system integrity&lt;/li&gt;&#xA;&lt;li&gt;Fuel conservation&lt;/li&gt;&#xA;&lt;li&gt;Vehicle external coating integrity&lt;/li&gt;&#xA;&lt;li&gt;Mechanical system wear&lt;/li&gt;&#xA;&lt;li&gt;Passenger comfort&lt;/li&gt;&#xA;&lt;li&gt;Time conservation in reaching destinations&lt;/li&gt;&#xA;&lt;li&gt;Probably others&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Applying Optimization in This Context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aggregation of the incoming signals acquired are not only based on multiple criteria but the prioritization is not always constant, implying the need for a vector of correction signals rather than a single floating point number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A single dimension of signaling to feed the disparity between ideal operation and the current system behavior (called a loss function in gradient descent) will not suffice.  There will, out of necessity, be a need for training and reinforcement with a complexity that involves the idea of preemption.  Evolution has declared preemption the design of choice for nervous systems with brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the pedestrian safety feedback signaling must always preempt the fuel conservation feedback no matter how much fuel would be consumed in staying clear from pedestrians, in planning a route where pedestrian density is lower, steering the vehicle clear of people, choosing speed, and applying breaking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All biological systems have these preemption mechanisms &amp;mdash; even bacteria.  A turtle doesn't balance the transportation aspect with safety when it retracts under its shell.  The behavioral interest in the turtle's destination is shelved (stored and temporarily forgotten) until the preemptive system that detected danger indicates the danger has passed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Humans Should be Passengers on the Streets of the Future&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason humans are generally driving in a continuous state of mistaken control is because the priorities that maximize the parameters of transportation for society (above) are inconsistently followed by humans.  Birds fly smarter than humans drive.  The priorities of an emotional being that wants to get somewhere fast and while talking, texting, eating, and possibly getting high will often be mistaken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Future people may look back at the period between the advent of Model-T market penetration and the complete transition to AV as a period of strange inequality.  Stepping back, the worldwide interest in domestic security, airline safety, train and subway safety, and building code contrasts strongly against the cultural insistence of every household have instant access to drive anywhere, any time, and in any mental condition.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-14T03:36:31.080" LastActivityDate="2018-12-14T03:36:31.080" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6648" PostTypeId="2" ParentId="6557" CreationDate="2018-06-04T19:01:35.337" Score="1" Body="&lt;p&gt;The &lt;strong&gt;chapters&lt;/strong&gt; for Bayesian Networks are:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;13&quot;&gt;&#xA;&lt;li&gt;Quantifying Uncertainty&lt;/li&gt;&#xA;&lt;li&gt;Probabilistic Reasoning&lt;/li&gt;&#xA;&lt;li&gt;Dynamic Bayesian&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;don't forget:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;20&quot;&gt;&#xA;&lt;li&gt;Naive Bayes, hidden variables, Markov&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Maybe helpful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;. Are We Going in the Right Direction? ... p.1049&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you find them interesting then invest more time to it. You might improve them and break new scientific ground.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent trend goes &lt;strong&gt;Deep Convolutional Neural Networks&lt;/strong&gt; (ex: AlphaGo) &lt;/p&gt;&#xA;" OwnerUserId="16060" LastActivityDate="2018-06-04T19:01:35.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6649" PostTypeId="5" CreationDate="2018-06-04T19:49:28.073" Score="0" Body="&lt;p&gt;Suitable for applications of AI to tackle real world problems.  Use in manufacturing, date collection and analysis, medical diagnostics, etc.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-04T19:49:28.073" LastActivityDate="2018-06-04T19:49:28.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6650" PostTypeId="4" CreationDate="2018-06-04T19:49:28.073" Score="0" Body="Use to specify the question relates to &quot;real world applications&quot; as opposed to pure research." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-04T19:49:28.073" LastActivityDate="2018-06-04T19:49:28.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6651" PostTypeId="5" CreationDate="2018-06-04T19:50:05.407" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-04T19:50:05.407" LastActivityDate="2018-06-04T19:50:05.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6652" PostTypeId="4" CreationDate="2018-06-04T19:50:05.407" Score="0" Body="For questions related to the academic aspects of AI, and the academic fields involving AI." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-04T19:50:05.407" LastActivityDate="2018-06-04T19:50:05.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6653" PostTypeId="2" ParentId="6643" CreationDate="2018-06-04T20:18:46.780" Score="0" Body="&lt;p&gt;The three questions &lt;em&gt;&quot;1) do you surf on internet? 2) do you play? 3) do you use editing programs?&quot;&lt;/em&gt; are a good start, but I think your teacher is right that you need more granularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What do you use your computer for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) Watching videos [leads to: &quot;Streaming or Downloaded HD?&quot; b/c downloading requires more local volume, and potentially a better video card.]  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) Do you play Games [leads to: &quot;High-end video games or simple games?&quot; b/c playing AAA FPS requires much more powerful video cards.  If they play words with friends or Tetris, a low-end card will be sufficient.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(c) Do you use editing programs? [leads to: &quot;photo editing?  video editing? what size files?&quot; b/c editing HD vid and high resolution photos is onerous with an underpowered system.]  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to ask in general what they use the computer for, because if it's just email, web surfing, Facebook and Youtube, etc., they can probably get by with a Windows surface &lt;em&gt;(I'm a regular critic of MS, but my understanding is you can get the fully functional Office Suite on Surface, which has utility value.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;--------------------------&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I think you're on the right track, but you should step back and think like a Product Manager here, as opposed to a developer.  These questions might help you clarify your intent, and expand your initial template to a more fully featured system your teacher is looking for:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Who is the customer for my system? &lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;What level of technical knowledge will the average user have? &lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;Am I covering every aspect with the proper amount of detail?&lt;/li&gt;&#xA;&lt;li&gt;Is the order of my questions correct?  (Why ask price range before monitor size?)&lt;/li&gt;&#xA;&lt;li&gt;Is there any mechanism to help a user if they don't know what answer to choose?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Again, these are just some initial thoughts. Only you know what product you want to create, and what the capabilities should be.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-06-04T20:18:46.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6654" PostTypeId="2" ParentId="6643" CreationDate="2018-06-04T20:39:17.897" Score="2" Body="&lt;p&gt;We cannot do homework for students in this network, however I can suggest that several items affecting cost and several usage patterns are missing and the number of rules is shy by an order of magnitude.  I wholeheartedly agree with the educational directives you received.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider first developing your lists further to include peripherals like DVD burner, USB devices, and audio.  Whether the user does scientific programming, watches movies on the monitor, develops software, and other specific usage scenarios is also more specific and therefore will produce a better tailored system than the answer to the question of whether the user is a professional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not the metric of the number of rules that is of most importance.  It is the number of operations contained in the rule set that is the guiding metric.  This is because rules in Prolog can be aggregated.  The rough estimate of rule operator count to complete a system is sqrt(i*o)/4, where i is the number of input permutations and o is the number of output permutations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is the application of Shannon Information Theory, that number of bits n = log&lt;sub&gt;2&lt;/sub&gt; (P'/P), where P' and P are the &lt;em&gt;a posteriorii&lt;/em&gt; and &lt;em&gt;a priori&lt;/em&gt; probabilities respectively.  The divisor of four is because there are about 16 = 2&lt;sup&gt;4&lt;/sup&gt; operators normally used.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may end up with thirty or forty rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create some use cases that exercise the extremes as well as some of the typical cases from among the permutations in inputs and outputs.  Run your system on those cases and observe the system behavior.  Learn how to debug by outputting intermediate results or stepping through rule execution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no shortcuts to researching and developing other than not wasting time worrying about how much time it will take.  You can also optimize your homework time by learning the tools and then stepping back, taking a deep breath, and saying, &quot;I can do this!&quot;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-06-04T20:39:17.897" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6655" PostTypeId="5" CreationDate="2018-06-04T20:56:40.807" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-04T20:56:40.807" LastActivityDate="2018-06-04T20:56:40.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6656" PostTypeId="4" CreationDate="2018-06-04T20:56:40.807" Score="0" Body="Use for questions that are broad or general by design.  Suitable for questions on concepts, methods, and families of same.  Also use for non-humanities questions that are relevant, but can't be objectively answered.  (i.e. not needed for philosophical questions, or questions on social impacts.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-04T20:56:40.807" LastActivityDate="2018-06-04T20:56:40.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6657" PostTypeId="2" ParentId="6644" CreationDate="2018-06-04T21:17:53.983" Score="1" Body="&lt;p&gt;Control should look like low numbers in highway and city accident report statistics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will be no drivers in Level 5 AV.  In fact, there may be no driver's position in the vehicle as with train passenger cars and dining cars.  This is quite different than Level 4.  In fact, more levels will probably emerge because of the huge jump from 4 to 5 in the current understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AV will stop in an emergency because the AV technology to do so will have proven its ability to determine and react to an emergency condition more quickly and reliably than a human driver.  The passengers will still be under control, just not THEIR control, except for their determining the destination address.  Control should be a communal function, since the danger when control fails is communal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the benefit of Level 5 over Level 4.  Highway pileups and many other accident scenarios are caused by human inability to react to risks.  Pileups are usually caused by human perception of a normal driving condition as an emergency and jamming on the breaks unnecessarily.  Most drivers will not steer into a skid even though they've heard to do so.  Rather than controlling the risk with skill, most people react in a way that reduces their tentative control of the vehicle when a skid begins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the passengers liking space between cars, the AV will prefer space too because proximity creates risk, so that question is moot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary impact of AV on society will be the gradual demolition of cultural values about engine power, which is reaching its time anyway because of the reduction of the burden of transportation on energy consumption that removing adrenaline and testosterone from the roads and highways will bring forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Government is pro AV simply because many humans like to &quot;drive through&quot; when traveling far, drive when on prescription medications that warn against operating heavy machinery (which cars are), and text in heavy traffic and in school zones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The automotive, aeronautical, and transportation industry is not a primary driver of the AV technology.  Their profit is maximized by occasional (but not frequent) accidents, stress on engine and body components, sales pitches about safety and power unsubstantiated with actual data, and marketing that equates freedom with control.  With rare accidents, minimized wear, and accountability created by deeper scrutiny there may be significant profit lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insurers will gain.  Ticket and accident attorneys will lose.  Parents of children in school will win.  Body shops will lose.  The electronics industry will win.  Tire sales will drop.  Government will be able to pocket more tax revenue.  Gear heads may eventually be thought of like KKK members are in today's postmodernism.  Custom wheel manufacturers will win.  Towing services will lose.  Bicyclers will win.  Petroleum extractors will lose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as the personal automobile changed the horse industry and the Internet changed publishing, AVs will lead to adjustments throughout government and the economy.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-06-06T20:42:53.157" LastActivityDate="2018-06-06T20:42:53.157" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6658" PostTypeId="1" CreationDate="2018-06-05T11:01:24.343" Score="4" ViewCount="220" Body="&lt;p&gt;What are the required characteristics of an activation function (in a neural network)? Which functions can be activation functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, which of the functions below can be used as an activation function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = \frac{2}{\pi} \tan^{-1}(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AXGlN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AXGlN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{x} e^{-\frac{t^2}{2}} dt$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which looks &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GQE2B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GQE2B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16070" LastEditorUserId="2444" LastEditDate="2019-05-09T21:33:50.873" LastActivityDate="2020-11-09T10:08:03.390" Title="Which functions can be activation functions?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6660" PostTypeId="1" CreationDate="2018-06-05T11:32:45.177" Score="1" ViewCount="97" Body="&lt;p&gt;an agent aims to find a path on a hexagonal map with initial state s0 in the center and goal state s⋆ at the bottom as depicted below. The map is parametrized by the distance n ≥ 1 from s0 to any of the border cells (n = 3 in the depicted example). The agent can move from its current cell to any of the 6 adjacent cells,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kHUyQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kHUyQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we find the number of node expansions performed by bfs without duplicate detection, and with duplicated detection as a function of n.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the branching factor for the map would be 6 because the agent can move in 6 directions, and for a depth of k, we get O(b^k) = 6^n without duplicate detection, but what is the number of node expansion with duplicate detection with bfs.&lt;/p&gt;&#xA;" OwnerUserId="16071" LastActivityDate="2018-06-05T14:18:37.150" Title="Number of node expansions performed by Breadth first search" Tags="&lt;algorithm&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6663" PostTypeId="2" ParentId="3435" CreationDate="2018-06-05T15:42:17.380" Score="2" Body="&lt;p&gt;If you are talking about &quot;generating&quot; in the sense of generative models , it is pretty tough. since we are still far beyond understanding the actual structure of question-answering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And even state of the art methods for question answering are also not able to score well on datasets like &lt;a href=&quot;http://classif.ai/dataset/babi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;babi&lt;/a&gt; , mostly 16 out of 20 tasks can be solved.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="1581" LastEditDate="2018-10-03T19:23:13.767" LastActivityDate="2018-10-03T19:23:13.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6669" PostTypeId="1" CreationDate="2018-06-05T21:16:50.023" Score="4" ViewCount="2441" Body="&lt;p&gt;I trained a DQN that learns TicTacToe by playing against itself with a reward of -1/0/+1 for a loss/draw/win. Every 500 episodes I test the progress by letting it play some episodes (also 500) against a random player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As shown in the picture the net learns quickly to get an average reward of 0.8-0.9 against the random player. But after 6000 episodes performance seems to deteriorate. If I play manually against the net after 10000 episodes it plays okay, but by no means perfect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that there is no hidden programming bug, is there anything that might explain such a behavior? Is there anything special about self-play in contrast to training a net against a fixed environment ?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here further details: The net has two layers with 100 and 50 nodes (and a linear output layer with 9 nodes), uses DQN and a replay buffer with 4000 state transitions. The shown epsilon values are only used during self-play, during evaluation against the random player exploration is switched off. Self-play actually works by training two separate nets of identical architecture. For simplicity one net is always player1 and the other always player2 (so they learn slightly different things). Evaluation is then done using the player1 net vs. a random player which generates moves for player2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanx&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y3xOx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y3xOx.png&quot; alt=&quot;TicTacToeSelfPlayResults&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15958" LastEditorUserId="15958" LastEditDate="2018-06-06T16:04:06.960" LastActivityDate="2019-07-31T22:51:25.560" Title="Why does self-playing TicTacToe not become perfect?" Tags="&lt;reinforcement-learning&gt;&lt;self-play&gt;" AnswerCount="2" CommentCount="9" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6670" PostTypeId="1" CreationDate="2018-06-05T22:48:29.290" Score="2" ViewCount="76" Body="&lt;p&gt;If an Atari game's rewards can be between &lt;span class=&quot;math-container&quot;&gt;$-100$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;, when can we say an agent learned to play this game? Should it get the reward very close to &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; for each instance of the game? Or it is fine if it gets a low score (say &lt;span class=&quot;math-container&quot;&gt;$-100$&lt;/span&gt;) at some instances? In other words, if we plot the agent's score versus number of episodes, how should the plot look like? From this plot, when can we say the agent is not stable for this task?&lt;/p&gt;&#xA;" OwnerUserId="15967" LastEditorUserId="2444" LastEditDate="2019-02-13T02:36:18.813" LastActivityDate="2019-02-13T02:36:18.813" Title="When can we say an RL algorithm learns an Atari game?" Tags="&lt;reinforcement-learning&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6671" PostTypeId="2" ParentId="6640" CreationDate="2018-06-06T07:30:24.860" Score="0" Body="&lt;p&gt;After i did a bit more study on this, i found this solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yGJiO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yGJiO.png&quot; alt=&quot;Membership Graph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;μsmall[x]&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;1; -&gt; x≤0&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(20-x)/(20-0); -&gt; 0&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;0; -&gt; x≥20&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;μmedium[x]&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;0; -&gt; x≤20 or x≥70&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(x-20)/(45-20); -&gt; 20&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(70-x)/(70-45); -&gt; 45&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;μlarge[x]&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0; -&gt; x≤70&lt;/li&gt;&#xA;&lt;li&gt;(x-70)/(100-70); -&gt; 70&#xA;&lt;li&gt;1; -&gt; x≥100&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the final result i need to add 3 more rules for the function to work. So it become 9 rules and 3 variables, with 3 rules for each input variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer both of my question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Add a triangle graph and use the formula.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Add a triangle graph for each rules.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What i learn from this is the total rules needed to determine the output is the number of input variables rules multiplied(i.e. v1 = 3 rules, v2 3 rules, total rules = 3x3=9 rules). And i'm not really sure about this one but you can combine multiple kind of graphs.&lt;/p&gt;&#xA;" OwnerUserId="16039" LastActivityDate="2018-06-06T07:30:24.860" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6673" PostTypeId="2" ParentId="6658" CreationDate="2018-06-06T08:09:08.847" Score="3" Body="&lt;p&gt;The main characteristic of an activation function is to bring a non-linearity property into the neural network. For the hidden layer, there is no need for the function to be bounded. The last layer should use a function whose range corresponds to what you want.&lt;/p&gt;&#xA;&lt;p&gt;For regression, you usually re-scale your output data to &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; and you use a &lt;em&gt;tanh&lt;/em&gt; (hyperbolic tangent) or &lt;em&gt;sigmoid function&lt;/em&gt; in the last layer&lt;/p&gt;&#xA;&lt;p&gt;For classification, you want to obtain probabilities: use a &lt;em&gt;softmax&lt;/em&gt; function in the last layer.&lt;/p&gt;&#xA;&lt;p&gt;For the hidden layers some functions are better than others :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The gradient should be fast to compute (from the perspective of your computer).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you use too many hidden layers, you will have the vanishing gradient problem if the derivative of your activation is too close to zero. You need a large zone of the domain with a derivative not close to zero.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In practice, the ReLU function defined as &lt;span class=&quot;math-container&quot;&gt;$f(x)=\max(0, x)$&lt;/span&gt; works very well and is very simple.&lt;/p&gt;&#xA;" OwnerUserId="15961" LastEditorUserId="2444" LastEditDate="2020-11-09T10:08:03.390" LastActivityDate="2020-11-09T10:08:03.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6674" PostTypeId="1" CreationDate="2018-06-06T08:37:00.597" Score="1" ViewCount="80" Body="&lt;p&gt;I am working on a task that requires me to classify a large amount of mixed files on a backup drive (more than 10 TB with more than 32 million files) based on content. The included file types are documents, images, videos, executable, and pretty much everything in between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also required to create new tags or metadata that will allow for automatic classification of new files. It'd also allow for manual input of category. For each input I give to the system, the system would learn and improve its classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I have come up with so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Documents: classify using existing categories with packages like Nltk on Python. Alternatively, first run topic modeling using LDA or NMF and then classify. &lt;/li&gt;&#xA;&lt;li&gt;Images: use CNN. In case of unknown label, use VAE to cluster the images.&lt;/li&gt;&#xA;&lt;li&gt;Videos and other types of files: I do not know how to approach this.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since I am not sure about my approach, any input is greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="16094" LastEditorUserId="16094" LastEditDate="2018-06-06T09:28:32.077" LastActivityDate="2018-06-06T09:28:32.077" Title="How to implement AI/ML to classify various types of files" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6675" PostTypeId="2" ParentId="6669" CreationDate="2018-06-06T16:21:58.150" Score="4" Body="&lt;p&gt;There are lots of ways that RL agents can fail to learn properly, so you are faced with a little bit of experimentation and maybe bug hunting unfortunately. However, from the description you have given in the question and comments, I can make a few observations and guesses about where to look:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Your metric of average reward against a random player is sensible. In this case, you could also use a perfect player (that ideally randomised choice of any optimal move), where you would see a maximum averaged return of zero - this would be helpful to know if your agent had learned a fully optimal behaviour, because it would consistently score zero. In general for more complex games a perfect player is not available to test with, but as you are learning here it might help you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your DQNs might be unable to fit the value function. You can test that in this case by getting the value function from an optimal self-play player (all the values will be -1, 0, or 1) and using a supervised learning approach, separately from your agent. You should be able to get a loss very close to zero - if you cannot do that, then something could be wrong with your network architecture.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whilst you are training, even though you are using a variation of Q-learning (which learns an optimal policy even whilst exploring other actions), your DQNs are not learning optimal play. That is because you have used two agents. In DQN, the algorithm is not aware that there are other learning agents, and it will treat any other agents as if they were part of the environment. Which means that the agents will spend some effort trying to set the game up for each other to make an exploration mistake. That could lead to non-optimal choices and a little bit of instability. Your decay of epsilon should help with that, although you are caught between a rock and a hard place here. You want to learn off-policy and explore, but are forced to reduce exploration. There are a couple of ways to resolve that, I will explain a bit further down . . .&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;10,000 games may not be enough. In the experiments I have done with TicTacToe agents, it seems between 20,000 and 50,000 games are required for a naive learner. More may be required if you have done something that makes learning inefficient. In addition, I found when adding more sophisticated learning approaches (in my case using eligibility traces) the agents appear to become close to optimal very quickly, but actually have flaws which take a long time to shake out, just as long as running a more naive algorithm. When the flaws got found and fixed, it upset the value function for a while and I saw fluctuations in my metrics similar to yours. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Q-learning with NNs is inherently unstable. DQN implements some ideas to fix that, but it is not perfect. It is not uncommon to need to adjust the batch size and/or time steps between taking frozen copy of network for the TD target calculation. The initial stability followed by poor performance looks a lot like that instability too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding your use of two opposing agents, I can see two possible improvements:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Alternately train one or other agent in each game, don't train both at once. That will mean each agent is learning to play against the other agent playing its best without exploratory moves.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Combine networks into single agent description. As this is a zero-sum game, you can take player A's network for calculating values, and just have player B try to &lt;em&gt;minimise&lt;/em&gt; the action value on its turn. That means use &lt;code&gt;min&lt;/code&gt; and &lt;code&gt;argmin&lt;/code&gt; functions for steps that represent player B's turn wherever player A would use &lt;code&gt;min&lt;/code&gt; or &lt;code&gt;argmin&lt;/code&gt;, including in the Q-value updates - this is typically easy to add to the inner loop of Q-learning, and should improve learning efficiency (essentially you are hard-coding knowledge that this is a zero-sum game and taking advantage of that symmetry).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Both of these ideas will free you up from caring about the value of epsilon, or decaying it - you can probably just leave it fixed at e.g. 0.1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, as a test of whether your agent can cope with learning optimal play in general, you could have it learn against an already optimal agent. That is obviously not something you can do for more complex games, but might help you debug agent code and hyper-parameters of the network - it divides your problem up into &quot;can it learn this at all&quot; and &quot;can it learn through self-play&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-06T17:14:48.073" LastActivityDate="2018-06-06T17:14:48.073" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6678" PostTypeId="1" CreationDate="2018-06-06T23:27:24.313" Score="11" ViewCount="406" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Furthermore, we observe that mode collapses traditionally&#xA;plaguing GANs tend to happen very quickly, over the course of a dozen minibatches. Commonly&#xA;they start when the discriminator overshoots, leading to exaggerated gradients, and an unhealthy&#xA;competition follows where the signal magnitudes escalate in both networks. We propose a mechanism&#xA;to stop the generator from participating in such escalation, overcoming the issue (Section 4.2).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What do they mean when the discriminator overshoots? The discriminator gets good too quickly? And signal magnitudes escalate in both networks?&lt;/p&gt;&#xA;&lt;p&gt;My current intuition is that the discriminator gets too good too soon, which causes the generator to spike and try to play catch up. That would be the unhealthy competition that they are talking about. Model collapse is the side effect where the generator has trouble playing catch up, and decides to play it safe by generating slightly varied images to increase its accuracy. Is this way of interpreting the above paragraph correct?&lt;/p&gt;&#xA;" OwnerUserId="12242" LastEditorUserId="43231" LastEditDate="2021-01-02T03:09:33.383" LastActivityDate="2021-02-01T04:09:11.153" Title="Can some one help me understand this paragraph from Nvidia's progressive gan paper?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;generative-model&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6681" PostTypeId="1" CreationDate="2018-06-07T15:50:16.833" Score="1" ViewCount="82" Body="&lt;p&gt;Disclaimer: I am a novice in the world of machine learning, so please excuse my ignorance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataset consists of things like age, days since last visit, etc. This information is medical related. None of which is geometrical, just data pertaining to particular clients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to classify my dataset into three labels. The dataset is not labeled, meaning I'm dealing with an unsupervised learning problem. My dataset consists of ~20,000 records, but this will linearly increase overtime. The data is nearly all floats, with some being strings that can easily be converted into a float. &lt;a href=&quot;http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using this cheat sheet for selecting a solution from the scikit site&lt;/a&gt;, a KMeans Cluster seems like potential solution, but I've been reading that having high dimensionality can render the KMeans Cluster unhelpful. I'm not married to a particular implementation either. I've currently got a KMeans Cluster implementation using TensorFlow in Python, but am open for alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what would be some solutions for me to further explore that might be more optimal for my particular situation?&lt;/p&gt;&#xA;" OwnerUserId="2818" LastEditorUserId="3773" LastEditDate="2018-08-10T10:03:25.757" LastActivityDate="2018-08-10T10:03:25.757" Title="Classifying non-labeled data with high dimensionality" Tags="&lt;classification&gt;&lt;tensorflow&gt;&lt;getting-started&gt;&lt;unsupervised-learning&gt;&lt;software-evaluation&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6683" PostTypeId="2" ParentId="6681" CreationDate="2018-06-08T09:07:52.343" Score="1" Body="&lt;p&gt;I would recommend to have a look at &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316801&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finding Groups in Data&lt;/a&gt;, which is a very readable introduction to clustering methods. It gives a good overview over a number of different algorithms, both agglomerative and hierarchical. As far as I remember, source code for the various algorithms is available on the web somewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure you will find a fitting algorithm for your problem in there.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-08T09:07:52.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6684" PostTypeId="2" ParentId="6182" CreationDate="2018-06-08T11:16:42.363" Score="0" Body="&lt;p&gt;Dlib can be a good point to start for detecting the facial landmarks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For start, you can try this approach:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Take various image of your friend in the &lt;em&gt;neutral&lt;/em&gt; expression and in the &lt;em&gt;indicating&lt;/em&gt; expression&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a subset of landmarks from Dlib (those that relate to the parts that your friend can move) to calculate some distances (see example in pictures, the arrows are the distances). All this distances now are the features that described your friend's &lt;em&gt;neutral&lt;/em&gt; and &lt;em&gt;indicating&lt;/em&gt; expression&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2w0wu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2w0wu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;To test if the features that you have obtained are good for classify between the 2 expression try Principal Component Analysis (PCA). Every language has one or more library that implement this (Except maybe C#??). If you features are good enough the representation in the feature spaces with PCA should look like:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t8jo2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t8jo2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, for example, the red dots represent the features extracted for &lt;em&gt;neutral&lt;/em&gt; expression and the blue dots represent the features of the expression &lt;em&gt;indicating&lt;/em&gt;. The more spaced the points are, better the classification will work.&#xA;If the dots are too close or overlapping, trying to go back at point 2 and choose different features.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;So you have 2 classes, you can now train a Support Vector Machine (SVM). Then in real-time you have to take a frame from a webcam, calculate in this frame the landmarks and the distances (as before) and pass the vector as input of the SVM that should tell you which expression is. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is a basic approach. But since you only have to recognize 2 expressions, and since these two expressions are always made by the same person, with a bit of luck and choosing the right features could work very well!&lt;/p&gt;&#xA;" OwnerUserId="16136" LastActivityDate="2018-06-08T11:16:42.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6685" PostTypeId="1" CreationDate="2018-06-08T14:20:27.637" Score="4" ViewCount="91" Body="&lt;p&gt;I'm building a decision tree and would like to separate (for example) the elements that are in class 0 from those in classes 1 and 2, case in point:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.DataFrame(np.random.randn(500,2),columns=list('AB'))&#xA;cdf = pd.DataFrame(columns=['C'])&#xA;cdf = pd.concat([cdf,pd.DataFrame(np.random.randint(0,3, size=500), columns=['C'])])&#xA;#df=pd.concat([df,cdf], axis=1)&#xA;(X_train, X_test, y_train, y_test) = train_test_split(df,cdf,test_size=0.30)&#xA;y_train=y_train.astype('int')&#xA;classifier = DecisionTreeClassifier(criterion='entropy',max_depth = 2)&#xA;classifier.fit(X_train, y_train)&#xA;y_pred = classifier.predict(X_test)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;C&lt;/code&gt; represents the class of an element,&lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; are two variables that define the element, how can I build a tree that instead of dividing results into &lt;code&gt;C=0&lt;/code&gt;, &lt;code&gt;C=1&lt;/code&gt; or &lt;code&gt;C=2&lt;/code&gt; divides them into &lt;code&gt;C=0&lt;/code&gt; and &lt;code&gt;C!=0&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12940" LastActivityDate="2018-06-08T15:12:33.963" Title="Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?" Tags="&lt;classification&gt;&lt;decision-tree&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6686" PostTypeId="1" AcceptedAnswerId="6688" CreationDate="2018-06-08T15:03:43.383" Score="3" ViewCount="510" Body="&lt;p&gt;I am reading a book about OpenCV, it speaks about some derivative of images like &lt;code&gt;sobel&lt;/code&gt;. I am confused about image derivative! What is derived from? How can we derived from an image? I know we consider an image(1-channel) as a n*m matrix with 0 to 255 intensity numbers. How can we derive from this matrix?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: a piece of text of the book:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Derivatives   and Gradients&lt;/p&gt;&#xA;&lt;p&gt;One   of  the most    basic   and important   convolutions    is  computing   derivatives&#xA;(or   approximations  to  them).  There   are many    ways    to  do  this,   but only    a   few&#xA;are   well    suited  to  a   given   situation.&lt;/p&gt;&#xA;&lt;p&gt;In    general,    the most    common  operator    used    to  represent   differentiation is  the&#xA;Sobel derivative  operator.   Sobel&#xA;operators exist   for any order   of  derivative  as  well    as  for mixed   partial&#xA;derivatives   (e.g.,  ∂ 2 /∂x∂y).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9941" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-06-18T02:50:23.553" Title="What does it mean &quot;derivative of an image&quot;?" Tags="&lt;image-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6688" PostTypeId="2" ParentId="6686" CreationDate="2018-06-08T15:11:32.720" Score="1" Body="&lt;p&gt;Imagine a line laid through the image. All pixels along the line count as values, so you can graph the pixels along the line like a function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative is of that 'function'. A black picture and a white picture have the same derivative (0), but a black-fading-to grey image would have a constant derivative bigger or smaller than zero, depending on the direction of the line in relation to the fading. Hard contrasts have huge derivarives at the points in the line where the line crosses a white/black border. Usually the rows and columns are used as the lines, but you could also lay any oblique line, and some algorithms do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term 'derivative' is somewhat a misnomer in this case, as usually the pixel values do net get fitted by a function of which then a derivative is taken, but the 'derivative' is directly taken by looking at the differences from one pixel to it's neighbor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;a href=&quot;https://dsp.stackexchange.com/questions/16805/1d-first-order-derivative-mask&quot;&gt;thread&lt;/a&gt; in dsp.stackexchange that deals with this, the following illustrative picture is from there:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/r2bUi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r2bUi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16147" LastEditorUserId="16147" LastEditDate="2018-06-11T11:00:02.650" LastActivityDate="2018-06-11T11:00:02.650" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6689" PostTypeId="2" ParentId="6685" CreationDate="2018-06-08T15:12:33.963" Score="3" Body="&lt;p&gt;I don't think that is possible with a decision tree, unless there is some measure of confidence that you can use as a threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran into the same problem with the ID3 algorithm. It assigns classes, but you only have the resulting class without any confidence or probability attached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible solution could be to add a number of counter examples as a second (dummy, catch-all) class; if the elements of &lt;code&gt;C = 0&lt;/code&gt; are reasonably tightly clustered and your counter examples for &lt;code&gt;C != 0&lt;/code&gt; cover the remaining problem space, then that might work.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-08T15:12:33.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6690" PostTypeId="2" ParentId="3343" CreationDate="2018-06-08T16:04:56.317" Score="-1" Body="&lt;p&gt;You can work with Recurrent Neural Nets with LSTM or GRU as memory cells and word embeddings like Word2vec.&#xA;Beam search and Attention models can also be utilized with the RNNs for more robustness and less bias.&#xA;But the outputs of these are appreciable up to some extent only as the research in this field is still hot and lot to be unraveled.&lt;/p&gt;&#xA;" OwnerUserId="16149" LastActivityDate="2018-06-08T16:04:56.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6691" PostTypeId="2" ParentId="2279" CreationDate="2018-06-08T16:32:33.823" Score="0" Body="&lt;p&gt;A simple trick can be splitting the image in to three frames vertically and feeding them to the image net and you can decide the position by looking for the frame which has higher probability of the desired category(simply max of all the probs).&#xA;Or else you can try YOLO algorithm which further uses non max suppression and IOU on the frames.&lt;/p&gt;&#xA;" OwnerUserId="16149" LastActivityDate="2018-06-08T16:32:33.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6692" PostTypeId="2" ParentId="4748" CreationDate="2018-06-08T16:55:01.737" Score="1" Body="&lt;p&gt;A funky way of doing this with less overhead is to just over fit the data up to some degree.&#xA;The reason is when you try to over fit the data with the classifier the classification bound tends to wrap around the clusters very tightly and with that model you can some times miss classify positive classes as negative(due to high variance) but there are comparatively less situations where you end up miss classifying negative classes as positive.&#xA;The level of overfitting that needs to be performed is just based on your FP and FN trade off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think this as a permanent fix but can come handy up to some extent.&lt;/p&gt;&#xA;" OwnerUserId="16149" LastActivityDate="2018-06-08T16:55:01.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6693" PostTypeId="2" ParentId="5942" CreationDate="2018-06-08T17:16:11.933" Score="0" Body="&lt;p&gt;I think you are actually working on one class classification as the other class is feature less. So you will end up classifying an input data as it belongs to that single class or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are ok with considering your problem as one class classification then i would say you actually &lt;strong&gt;DON'T&lt;/strong&gt; need a feature less data set at all.&#xA;You can just directly run your featured data(say cat pictures) using an autoencoder and figure out threshold value at the bottle neck(this is a bit challenging). Later during the test time you can verify that input data belongs to the desired class by just looking at the threshold value produced by the encoding part of autoencoder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this answer doesn't satisfy you. you can just google keywords like &quot;One class classification&quot; or &quot;out lier detection&quot;. I guess from there you can follow up easily.&lt;/p&gt;&#xA;" OwnerUserId="16149" LastActivityDate="2018-06-08T17:16:11.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6694" PostTypeId="2" ParentId="5982" CreationDate="2018-06-08T17:44:04.587" Score="0" Body="&lt;p&gt;I would also suggest character level Recurrent neural nets but with Normal Char level RNN we can only predict next chars based on previous chars so you should consider it to be bidirectional RNN because say we have text &quot;xxx12345&quot; basically if we feed this to our model our model should predict first three places based on last places ( in DL they call it as going back through time) and this is possible only by Bidirectional RNN.&lt;/p&gt;&#xA;" OwnerUserId="16149" LastActivityDate="2018-06-08T17:44:04.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6695" PostTypeId="2" ParentId="6686" CreationDate="2018-06-08T19:27:35.613" Score="4" Body="&lt;p&gt;The term &lt;strong&gt;&lt;code&gt;Derivative of an Image&lt;/code&gt;&lt;/strong&gt; in the context you mention has two meanings.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A matrix, image, or floating point number that is derived from an image via convolution, passing the image through a two dimensional NN, the application of an FFT analysis, or some other process.  In this context, the word &lt;strong&gt;Derivative&lt;/strong&gt; implies the direction of calculation: Image B is derived from image A.&lt;/li&gt;&#xA;&lt;li&gt;A matrix or cube that represents the rate of change at in the image being processed.  The change being measured between only two adjacent pixels in a single dimension and only one direction at a time, but the applications of this technique is very limited, and such a sequence is of differences, not at all reasonable approximations of the derivative of light.  What is more useful in real recognition systems are two dimensional or hexagonal windowing (Gausian, Hamming, Hanning, trapazoidal, cosine, ...) across space and, for video, through time.  The calculus term derivative should always reference the theoretical surface being approximated using these techniques, not the discrete matrix or cube that approximates the surface.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Such multidimensional convolution and neural network based approaches are less sensitive to capture noise and orientation nuances.  Two dimensional whole image or windowed FFT techniques have met with much success because filtering the expected frequency range of features to be detected is merely an attenuation process.  Two and three dimensional splines can also be tuned to be useful in the detection of features in an orientation independent way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to gray scale analysis, color and transparency channels can be selected for independent or parallel analysis or added to the dimension of the fitting model from which the derivative is taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Advances in deep networks have blossomed into a new area of image processing and recognition research, bringing new hope to robotics, automated transportation, and cybernetics in general.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-06-18T02:50:23.553" LastActivityDate="2018-06-18T02:50:23.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6696" PostTypeId="1" AcceptedAnswerId="6739" CreationDate="2018-06-08T19:48:10.853" Score="3" ViewCount="511" Body="&lt;p&gt;I'm working with deep learning on some EEG data for classification, and I was wondering if there's any systematic/mathematical way to define the architecture of the networks, in order to compare their performance fairly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should the comparison be at the level of neurons (e.g. number of neurons in each layer), or at the level of weights (e.g. number of parameters for training in each type of network), or maybe something else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea that emerged was to construct one layer for the MLP for each corresponding convolutional layer, based on the number of neurons after the pooling and dropout layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas? If there's any relative work or paper regarding this problem I would be very grateful to know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Konstantinos&lt;/p&gt;&#xA;" OwnerUserId="16148" LastActivityDate="2018-06-13T18:03:30.690" Title="How to make a fair comparison of a convolutional neural network (cNN) vs a mutlilayer perceptron (MLP)?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6697" PostTypeId="2" ParentId="6556" CreationDate="2018-06-09T06:42:44.327" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Feature embeddings are basically anything that can act as a hidden representation for given object. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the case of images, a CNN architecture is built to create such hidden representation. Usually, the outcome of the bottleneck layer is flattened (and sometimes, converted to even lower dimensional space by adding one more dense layer) and used as feature embeddings.&lt;/p&gt;&#xA;" OwnerUserId="16159" LastActivityDate="2018-06-09T06:42:44.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6698" PostTypeId="2" ParentId="2347" CreationDate="2018-06-09T11:04:37.870" Score="-1" Body="&lt;p&gt;An optimistic view is &lt;b&gt;no&lt;/b&gt; just like any other inventions we had in the past they improved the quality of life.&lt;br&gt;&#xA;A pessimistic view &lt;i&gt;maybe&lt;/i&gt; the algorithm may control all the computer systems globally and enslave us.&lt;br&gt;&#xA;&lt;b&gt; A realistic&lt;/b&gt; view is that AI will forever be there to serve us, for as the pessimistic view may actually makes sense in the fictional world ... the truth is its impossible because the compute capability required is extravagant.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-06-09T11:04:37.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6699" PostTypeId="1" CreationDate="2018-06-09T14:36:56.427" Score="3" ViewCount="1695" Body="&lt;p&gt;I have been looking at Fibonacci series, the golden ratio and its uses in nature, like how flowers and animals grow based on the series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering whether we could use the Fibonacci series and the golden ratio in any way in AI, especially in evolutionary algorithms. Any ideas or insights?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this research material? If so where can we start?&lt;/p&gt;&#xA;" OwnerUserId="15944" LastEditorUserId="2444" LastEditDate="2019-04-07T16:35:25.760" LastActivityDate="2019-06-25T22:42:08.303" Title="AI applications of the Fibonacci series" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;evolutionary-algorithms&gt;&lt;applications&gt;" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6702" PostTypeId="1" CreationDate="2018-06-10T15:47:04.977" Score="2" ViewCount="350" Body="&lt;p&gt;In our lives, we meet different people and describe their common sense based on how they act on a situation. For example, highly extrovert people are able to deal with people without any awkwardness. For them, an action in how to deal with people come as common sense. But, in the case of scientists, approach to solving a problem may be common sense which ordinary people cannot see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we define common sense in an AI agent?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-06-20T21:28:10.017" LastActivityDate="2020-02-11T17:55:43.870" Title="How can we define common sense in an AI agent?" Tags="&lt;ai-design&gt;&lt;philosophy&gt;&lt;emotional-intelligence&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6705" PostTypeId="2" ParentId="6702" CreationDate="2018-06-10T22:48:55.717" Score="2" Body="&lt;p&gt;I came up with a few ideas I would argue are valuable in motivating the idea of common sense for a machine learning model. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Common sense is retrospective. We define it in terms of a past (sensible) actions and conditions, and we can say someone has good common sense on the basis of their behavior, which we can view as the sum of their historical actions and the degree to which they were sensible. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The actions alone are not sufficient for demonstrating common sense; the mental model(s) that generated those actions is important as well. Why did the individual take those sensible actions? Is their reasoning behind their action sensible as well, or did they merely get lucky (put another way, does their generally not sensical mental model produce sensible actions in certain cases and non sensical actions in most others)? This hints that common sense is contingent on the rationalization of the actions themselves. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Given the former, common sense is contingent on the enumeration of possible actions and choosing the right one, given the &lt;em&gt;context&lt;/em&gt; of the situation. For example, I’m walking on a path and see a snail. What are some actions? I could keep walking, stop and admire it for a while, step on it, or eat it. The first two options are sensible if I value the snail as a living being. The first option is sensible if I’m in a rush. The second and fourth may be sensible if I’m a chef exploring nature for new potential ingredients. The third option if I recognize the snail as invasive. We say someone has common sense when, given the context, the actions chosen are sensible and we can reason about them intuitively. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My guess is that, the &lt;em&gt;intuition&lt;/em&gt; behind the perceived sense of an action is what you’re after. I’d argue that, ultimately, the intuition of common sense is defined by the person developing the model and will have to do with the formulation of the model (e.g it’s assumptions, the objective function, etc). After all, common sense is subjective and context specific. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concretely, a model can have common sense if the developer bakes it in and we can use inferential methods to demonstrate this. For example, in a Word2Vec model, we might see that &lt;span class=&quot;math-container&quot;&gt;$\mathsf{Paris} \mapsto \mathsf{France}$&lt;/span&gt; and would expect that &lt;span class=&quot;math-container&quot;&gt;$\mathsf{Tokyo} \mapsto \mathsf{Japan}$&lt;/span&gt;. To interrogate this, we might do some vector math and find that &lt;span class=&quot;math-container&quot;&gt;$\mathsf{Paris} - \mathsf{France} + \mathsf{Tokyo} = \mathsf{Japan}$&lt;/span&gt;. How and if the AI model develops the larger association between &lt;span class=&quot;math-container&quot;&gt;$\mathsf{Capital} \mapsto \mathsf{Country}$&lt;/span&gt;, however, comes down to how the developer built and trained the model to recapitulate their own common sense. &lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="5210" LastEditDate="2020-02-11T17:55:43.870" LastActivityDate="2020-02-11T17:55:43.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6707" PostTypeId="1" CreationDate="2018-06-11T12:51:07.997" Score="2" ViewCount="94" Body="&lt;p&gt;I would like to develop a machine learning algorithm, given two photos, that can decide which image is more &quot;artistic&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about somehow combining two images, giving it to a CNN, and get an output 0 (the first image is better) or 1 (the second image is better). Do you think this is a valid approach? Or could you suggest an alternative way for this? Also, I don't know how to combine two images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Let me correct &quot;artistic&quot; as &quot;artistic according to me&quot;, but it doesn't matter, I am more interested in the architecture. You can even replace &quot;artistic&quot; with something objective. Let's say I would like to determine which photo belongs to a more hotter day.&lt;/p&gt;&#xA;" OwnerUserId="16201" LastEditorUserId="1671" LastEditDate="2018-06-13T17:04:48.873" LastActivityDate="2020-01-24T14:47:42.417" Title="Which photo is more artistic?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;art-aesthetics&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6709" PostTypeId="2" ParentId="6707" CreationDate="2018-06-11T14:51:46.257" Score="1" Body="&lt;p&gt;I think the real problem is, what defines a more artistic image? It's really subjective, I think complexity of the work might be one aspect to consider but still a subjective one, you might want to make the purpose of your ML algorithm more objective or defined.&#xA;But then again, It's just my opinion&lt;/p&gt;&#xA;" OwnerUserId="16078" LastActivityDate="2018-06-11T14:51:46.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6710" PostTypeId="2" ParentId="6707" CreationDate="2018-06-11T16:02:52.863" Score="2" Body="&lt;p&gt;You need to define a scoring function, which returns a value of whatever criterion you are interested in, be it 'artisticity' or 'heat'. This could be something you use a machine learning algorithm for, provided you have a set of training data with labelled images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then need to extract features from the images. In the case of 'heat' this could be a colour spectrum (ie distribution of colours across the image), or whatever. If it's a reasonably small image, pixel values might be feasible. These features you can feed into your algorithm, and try to learn an association between the feature values and the (assigned by you) label of the image. You will end up with a classifier that takes image features as input, and returns the labels as output. The quality of the classifier depends on your images, the features you selected, and the task.  If there is no structure in the data, then a classifier will not work properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a continuous value (eg temperature in degrees C) you would run your two sample images through the classifier, and then compare the output values.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-11T16:02:52.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6711" PostTypeId="1" CreationDate="2018-06-11T16:55:24.813" Score="1" ViewCount="52" Body="&lt;p&gt;What is the state of the art in models of how the human brain performs goal-directed decision making? Can these models’ principles and insights be applied to the field of Artificial Intelligence, e.g. to develop more robust and general AI algorithms?&lt;/p&gt;&#xA;" OwnerUserId="16210" LastEditorUserId="16210" LastEditDate="2018-06-11T17:04:28.903" LastActivityDate="2018-06-11T19:28:33.630" Title="What is the state of the art in models of how the human brain performs goal-directed decision making? Can these models' principles be applied to AI?" Tags="&lt;models&gt;&lt;decision-theory&gt;&lt;cognitive-science&gt;&lt;brain&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6712" PostTypeId="2" ParentId="5129" CreationDate="2018-06-11T17:00:28.037" Score="1" Body="&lt;p&gt;Suppose you are dealing with a classification problem you can simply define &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def error_rate_classification(p, t):&#xA;    return np.mean(p != t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where p and t stand for predictions and targets resp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you deal with a regression problem the you should modify a bit this function. For example :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def error_rate_regression(p, t):&#xA;    return np.mean(np.abs(p-t) &amp;lt; 1e-2) # the criteria is not general&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that for regression problem assessing the accuracy of the prediction is not as simple as it is in classification problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In regression case, you can use the R2 regression standard metrics. Suppose yi is the ground truth value of the i-th element of your test (or validation) set and f(xi) is your i-th input prediction. The R2 is then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R2 = 1 - sum( (yi - f(xi))**2 )  / sum( (yi - y_mean)**2 )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where y_mean is the mean of the ground truth values of the test ( or validation) test.  You can find other examples in the &lt;a href=&quot;https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/metrics/regression.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-learn regression metrics here&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="11069" LastEditorUserId="11069" LastEditDate="2018-06-12T10:24:45.057" LastActivityDate="2018-06-12T10:24:45.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6713" PostTypeId="2" ParentId="6488" CreationDate="2018-06-11T17:24:49.373" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;if I input the unshuffled data (as I required to view the predictions) I get a 45% accuracy. How is this possible?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When you build your dataset and then split it into training, test, and validation sets, you have to be sure that the training encompasses aspects of the test set or the validation set and more. That's why shuffling the data is important. &#xA;In the case where you didn't shuffle your dataset, your NN was confronted to object it never sees before (i.e. in the training set) and then couldn't classify it properly. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;does the model learn from the average of all the data points in the mini-batch or does it think one mini-batch is one data point and learn in that manner (which would mean order matters of the data)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When using a mini batch you decide the size (&lt;code&gt;b_size&lt;/code&gt;) of the mini batch and the number of mini batches (&lt;code&gt;n_batch&lt;/code&gt;). Then, for &lt;code&gt;n_batch&lt;/code&gt; times you will draw &lt;code&gt;b_size&lt;/code&gt; random indices from &lt;code&gt;X_train&lt;/code&gt; and &lt;code&gt;y_train&lt;/code&gt; arrays, build with these indices &lt;code&gt;X_batch&lt;/code&gt; and &lt;code&gt;y_batch&lt;/code&gt;, and tune your model parameters on the 2 latter arrays (and do it &lt;code&gt;n_batch&lt;/code&gt; times).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So having shuffled your dataset and takes randomly parts of your dataset to train your model ensure you to get rid of correlations between two points of your datasets and better generalization &lt;/p&gt;&#xA;" OwnerUserId="11069" LastActivityDate="2018-06-11T17:24:49.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6715" PostTypeId="1" CreationDate="2018-06-11T19:55:35.770" Score="6" ViewCount="195" Body="&lt;p&gt;A few months ago I made a simple game that is similar to the dinosaur game in Google Chrome - you jump over obstacles, or don't jump over levitating obstacles, and jump to collect bitcoins, which can be placed at 5 different heights. I used a very lightweight NN written by NYU professor Dan Shiffman, and within a few days the game and AI were done, starting off with a population of 200 jumpers, and a genetic algorithm (fitness function (points are given for avoiding obstacles and gathering bitcoins) and mutation), and it worked as it should.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this was only when the bitcoins and obstacles were not near each other, which I've been struggling with ever since. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I made a &quot;training ground&quot; where I put first a levitating obstacle, then a grounded one, and then a bitcoin after it, and then a bitcoin above a fourth grounded obstacle, and no matter how many times and how long I'd leave it to train, I'd always end up with identical behavior:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first 3 obstacles are properly avoided, the first bitcoin is collected, and then jumpers would jump too early, land before the fourth &quot;bitcoin&quot; obstacle, and jump again, always crashing at almost the same place (across all generations, so even if I'd restart the training, they crash at the same place in the obstacle, with a deviation of a few pixels up or down).&#xA;I added multilayer support to the NN, no improvements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today I replaced the NN with tensorflow.js, and I am getting identical behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My inputs are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;distance to next obstacle  &lt;/li&gt;&#xA;&lt;li&gt;altitude of next obstacle  &lt;/li&gt;&#xA;&lt;li&gt;distance to next star  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(for simplicity I removed the altitude of stars from the input, and keep them at a constant altitude)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2 hidden layers (5 and 6 neurons), and 1 neuron in the output, which determines if the jumper should jump.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My only idea is that a neuron that decides when to jump because of the obstacle activates alongside the neuron that decides when to jump because of the bitcoin, their weights are summed up and a decision to jump too early is made. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll give somewhat of a (maybe bad) analogy: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it takes you 1 month to prepare an exam, then, if you have 2 exams on the same day, you will start preparing them 2 months earlier. That logic works in this case, but not in my AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the initial &quot;toy neural network&quot; I even added 8 layers of 12 neurons each, which I think is overkill for this case.&#xA;In tf.js I used both sigmoid and relu activation functions.&#xA;No matter what I did, no improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope someone has an idea where I'm going wrong.&lt;/p&gt;&#xA;" OwnerUserId="16214" LastEditorUserId="16214" LastEditDate="2018-06-11T20:25:43.113" LastActivityDate="2019-04-13T17:02:30.510" Title="Issue with simple game AI" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6717" PostTypeId="2" ParentId="6696" CreationDate="2018-06-12T08:16:11.630" Score="0" Body="&lt;p&gt;The best best way to monitor an architectures performance would be comparing the resource utilization, model accuracy, value loss and confusion matrix.&#xA;&lt;br&gt;&#xA;e.g VGG16 consumes less system resources in comparison to inception V3&#xA;&lt;br&gt;&#xA;&lt;br&gt;&#xA;There is an &lt;a href=&quot;https://machinelearningmastery.com/classification-accuracy-is-not-enough-more-performance-measures-you-can-use/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; that goes in depth.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-06-12T08:16:11.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6718" PostTypeId="2" ParentId="3009" CreationDate="2018-06-12T09:02:10.813" Score="5" Body="&lt;p&gt;A genetic algorithm is an algorithm, based on natural selection (the process that drives biological evolution), for solving both constrained and unconstrained optimization problems.&lt;/p&gt;&#xA;&lt;p&gt;A memetic algorithm is an extension of the concept of a genetic algorithm that uses a local search technique to reduce the likelihood of premature convergence.&lt;/p&gt;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1004/1004.0574.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comparison between Memetic algorithm and Genetic algorithm for the cryptanalysis of Simplified Data Encryption Standard algorithm&lt;/a&gt; compares both approaches.&lt;/p&gt;&#xA;&lt;p&gt;To answer your last question, yes, an individual's lifetime still plays a part in memetic algorithms because the objective here is to avoid premature convergence.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastEditorUserId="2444" LastEditDate="2021-01-16T14:47:36.473" LastActivityDate="2021-01-16T14:47:36.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6719" PostTypeId="2" ParentId="6678" CreationDate="2018-06-12T09:06:03.607" Score="0" Body="&lt;p&gt;A discriminator overshooting may result in a dataset that has not been thoroughly clean and probably has too many identical feature, as a result there will be an early convergence from the discriminator as there is little variation. &#xA;The drawback from this is that the model will not bee able to generalize well.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-06-12T09:06:03.607" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6721" PostTypeId="1" CreationDate="2018-06-12T14:02:27.770" Score="2" ViewCount="460" Body="&lt;p&gt;In a reinforcement learning model, states depend on the previous actions chosen. In the case in which some of the states -but not all- are fully independent of the actions -but still obviously determine the optimal actions-, how could we take these state variables into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem was a multiarmed bandit problem (where none of the actions influence the states), the solution would be a contextual multiarmed bandit problem. Though, if we need a &quot;contextual reinforcement learning problem&quot;, how can we approach it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of separating a continuous context into steps, and creating a reinforcement learning model for each of these steps. Then, is there any solution where these multiple RL models are used together, where each model is used for prediction and feedback proportionally to the closeness between the actual context and the context assigned to the RL model? Is this even a good approach?&lt;/p&gt;&#xA;" OwnerUserId="6114" LastEditorUserId="2444" LastEditDate="2020-04-15T20:19:40.310" LastActivityDate="2020-04-15T20:19:40.310" Title="How to implement a contextual reinforcement learning model?" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;contextual-bandit&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6722" PostTypeId="2" ParentId="6702" CreationDate="2018-06-12T14:27:59.633" Score="2" Body="&lt;p&gt;I define &quot;common sense&quot; inline with human beings, to concatenate to intelligent agents;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An algorithmic agent, which has the ability to solve effective decision in relation to the way humans perceive their environment and common situations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to McCarthy John,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The first artificial intelligence program proposed to address common sense was &lt;a href=&quot;https://en.wikipedia.org/wiki/Advice_taker&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advice Taker&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Currently, common sense is unsolved problem in AI.  For more information, see John McCarthy's&#xA;&lt;em&gt;&lt;a href=&quot;http://www-formal.stanford.edu/jmc/mcc59.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Programs with Common Sense&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="1671" LastEditDate="2018-06-13T17:46:19.997" LastActivityDate="2018-06-13T17:46:19.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6723" PostTypeId="2" ParentId="6721" CreationDate="2018-06-12T15:25:19.460" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In the case in which some of the states -but not all- are fully independent of the actions -but still obviously determine the optimal actions-, how could we take these state variables into account?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the key thing here is the caveat &lt;strong&gt;but not all&lt;/strong&gt;. What you have is a fully-featured MDP (states, actions, rewards, timesteps where next reward and next state depend on current state and action). The fact that next state is only marginally affected by the current action does not prevent it being an MDP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be a problem if the current state did not adequately describe this limitation e.g. if some other data, outside of observable state, decided whether action had any effect. Assuming that is not the case, then you still have a full reinforcement learning problem, but one with some unusual qualities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can mitigate against problems caused by algorithms that will over-estimate likely reward (caused by algorithm associating lucky state trajectories with action choice), by using &quot;double learning&quot; - one estimate of return is used to select maximising action, and another used to estimate the actual return from the next state. You probably will also prefer single-step learning over learning based on trajectories as most of the time your state trajectories will not contain learnable data. So &lt;a href=&quot;https://arxiv.org/abs/1509.06461&quot; rel=&quot;nofollow noreferrer&quot;&gt;Double Deep Q Networks&lt;/a&gt; might be a good starting algorithm to try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know &lt;em&gt;absolutely&lt;/em&gt; from inspecting a current state, that the next state is independent of the current action, and the next state could in theory be almost anything (from either the whole state space or some large subset) then you may be able to adapt the algorithm to allow for that knowledge. You would do this by altering the TD target and replacing the terms for bootstrap estimate of next state with a rolling mean over all &lt;em&gt;reachable&lt;/em&gt; next states. In concept this is similar to &lt;em&gt;Expected SARSA&lt;/em&gt; - and in practice if it is possible, it would go a long way to reducing variance during learning process, and may speed up learning significantly. If you know the distribution of next states, you could maybe use that, but basing it purely on samples seen should also be fine, provided you can allocate the means to correct group of states (your question implies you have some understanding of how the states will group). Note that transitions from states with no action effect on next state to states which do have an effect from the action will need careful handling - they should not be assigned to the same &quot;mean group&quot;, but should instead affect the TD target normally when they occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not possible for the agent to know whether it is in a state where its action affects the next state, except through experience, then you really have to use a standard RL solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you have a situation where the current &lt;em&gt;action&lt;/em&gt; does not affect next state, but the current &lt;em&gt;state&lt;/em&gt; definitely does in all cases -  the state &quot;evolves&quot;, maybe stochastically, independently of the action taken in most cases, and can only change to a relatively small subset of next states from any given current state - then this is best solved with a normal RL solver - again double Q Learning might be a reasonable starting algorithm in that case.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-12T18:40:31.970" LastActivityDate="2018-06-12T18:40:31.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6724" PostTypeId="1" AcceptedAnswerId="6729" CreationDate="2018-06-12T18:45:20.077" Score="2" ViewCount="6334" Body="&lt;p&gt;I would like to know if there is a complete text classification with deep learning example, from text file, csv, or other format, to classified output text file, csv, or other. I have seen tens of tutorials and they mostly focus on the model and its performance, but I have not been able to find one that shows how to apply the model to a set of text strings and how to output a document with the classified(labeled) text.&lt;/p&gt;&#xA;" OwnerUserId="16230" LastActivityDate="2018-06-13T08:19:56.527" Title="Complete deep learning text classification with Python example" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;reference-request&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6727" PostTypeId="2" ParentId="5919" CreationDate="2018-06-13T01:32:15.557" Score="1" Body="&lt;p&gt;Vanishing gradient is a common problem in RNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common way to deal with it is the method of gradient clipping (mainly you define a maximum and/ or a minimum threshold). &lt;a href=&quot;https://www.quora.com/What-is-gradient-clipping-and-why-is-it-necessary&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here for more information&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further information and piece of code to implement it can be found in &lt;a href=&quot;https://stackoverflow.com/questions/36498127/how-to-apply-gradient-clipping-in-tensorflow&quot;&gt;SO here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps ! &lt;/p&gt;&#xA;" OwnerUserId="11069" LastActivityDate="2018-06-13T01:32:15.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6728" PostTypeId="1" CreationDate="2018-06-13T04:35:55.150" Score="1" ViewCount="84" Body="&lt;p&gt;What are the top contributions from neuroscience to artificial intelligence and vice versa? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much progress has been made from the interaction between these two fields? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a formal field dedicated to the research of topics in the intersection between AI and neuroscience, with papers being published and maybe conferences being held periodically?&lt;/p&gt;&#xA;" OwnerUserId="16210" LastEditorUserId="1671" LastEditDate="2018-06-13T16:58:10.377" LastActivityDate="2018-06-16T11:10:40.890" Title="What are the top contributions from neuroscience to AI and viceversa? How much progress has been made from the interaction between these two fields?" Tags="&lt;reference-request&gt;&lt;cognitive-science&gt;&lt;brain&gt;&lt;academia&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6729" PostTypeId="2" ParentId="6724" CreationDate="2018-06-13T07:38:42.677" Score="2" Body="&lt;p&gt;You may look at these github repositories : &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/dennybritz/cnn-text-classification-tf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Text classification in NLP with deep learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/dennybritz/cnn-text-classification-tf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Networks for Sentence Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16240" LastActivityDate="2018-06-13T07:38:42.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6730" PostTypeId="2" ParentId="6724" CreationDate="2018-06-13T08:19:56.527" Score="1" Body="&lt;p&gt;I believe what you are looking for is this: &lt;a href=&quot;https://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author uses the IMDB dataset to classify movies from comment sentiment (good / bad).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It includes detailed description of all the steps and provides you with the code to have a LSTM-RNN up and running very fast for the purpose.&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-06-13T08:19:56.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6731" PostTypeId="2" ParentId="6728" CreationDate="2018-06-13T08:32:46.280" Score="2" Body="&lt;p&gt;Understanding a human brain fully, how hippocampus and neocortex works would surely help AI enthusiasts to make better AI algorithms with superior memory, ability to learn and finding what precisely intelligence, feelings and conciousness are by itself. The AI looks at psysiological and anatomical data as a source of suggestions regarding possible mechanisms and processes that could be used. Still AI doesn't deal that much from neuroscience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From brain to AI : edge-detection using zero-crossing at multiple scales is a decent example where physiological data are used to suggest a solution to a fundamental problem in computer vision. In computational vision, a problem is the initial organization and representation of the input image. The goal is to transform the measurements of light-intensity level made by photoreceptors eyes' have, into a representation that is more concise. One of the most successful approaches to the problem had its origin in physiological studies of the processing that takes place at the level of the retinal ganglion cells which can be thought of as filtering the image with center-surround receptive fields whose cross-section can be approximated analytically by the second derivative of a gaussian. There had been proven research that a powerful edge detection scheme could be based upon the operation of such cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From AI to brain : Integration of motion perception measurement is a worth-mentioning example. It originates in studies of problem of measuring the motion of edges and contours in the image. Such motion can not be determined on the basis of purely local measurements. The underlying reason is 'aperture problem'. Selective visual attention and long-range motion analytics are another two fields where use of computational cosiderations are illustrated in the design and interpretation of physiological studies.&lt;/p&gt;&#xA;" OwnerUserId="16240" LastActivityDate="2018-06-13T08:32:46.280" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6733" PostTypeId="2" ParentId="4456" CreationDate="2018-06-13T12:47:50.293" Score="36" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What's the difference between model-free and model-based reinforcement learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In Reinforcement Learning, the terms &quot;model-based&quot; and &quot;model-free&quot; do &lt;em&gt;not&lt;/em&gt; refer to the use of a neural network or other statistical learning model to predict values, or even to predict next state (although the latter may be used as part of a model-based algorithm and be called a &quot;model&quot; regardless of whether the algorithm is model-based or model-free).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, the term refers strictly as to whether, whilst during learning or acting, the agent uses predictions of the environment response. The agent can use a single prediction from the model of next reward and next state (a sample), or it can ask the model for the &lt;em&gt;expected&lt;/em&gt; next reward, or the full &lt;em&gt;distribution&lt;/em&gt; of next states and next rewards. These predictions can be provided entirely outside of the learning agent - e.g. by computer code that understands the rules of a dice or board game. Or they can be learned by the agent, in which case they will be approximate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just because there is a model of the environment implemented, does not mean that a RL agent is &quot;model-based&quot;. To qualify as &quot;model-based&quot;, the learning algorithms have to explicitly reference the model: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Algorithms that purely sample from experience such as Monte Carlo Control, SARSA, Q-learning, Actor-Critic are &quot;model free&quot; RL algorithms. They rely on real samples from the environment and never use generated predictions of next state and next reward to alter behaviour (although they might sample from experience memory, which is close to being a model).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The archetypical model-based algorithms are Dynamic Programming (Policy Iteration and Value Iteration) - these all use the model's predictions or distributions of next state and reward in order to calculate optimal actions. Specifically in Dynamic Programming, the model must provide state transition probabilities, and expected reward from any state, action pair. Note this is rarely a learned model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Basic TD learning, using state values only, must also be model-based in order to work as a control system and pick actions. In order to pick the best action, it needs to query a model that predicts what will happen on each action, and implement a policy like &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \text{argmax}_a \sum_{s',r} p(s',r|s,a)(r + v(s'))$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt; is the probability of receiving reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; when taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. That function &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt; is essentially the model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The RL literature differentiates between &quot;model&quot; as a model of the environment for &quot;model-based&quot; and &quot;model-free&quot; learning, and use of statistical learners, such as neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In RL, neural networks are often employed to learn and generalise value functions, such as the Q value which predicts total return (sum of discounted rewards) given a state and action pair. Such a trained neural network is often called a &quot;model&quot; in e.g. supervised learning. However, in RL literature, you will see the term &quot;function approximator&quot; used for such a network to avoid ambiguity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It seems to me that any model-free learner, learning through trial and error, could be reframed as model-based.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think here you are using the general understanding of the word &quot;model&quot; to include any structure that makes useful predictions. That would apply to e.g. table of Q values in SARSA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as explained above, that's not how the term is used in RL. So although your understanding that RL builds useful internal representations is correct, you are not technically correct that this can be used to re-frame between &quot;model-free&quot; as &quot;model-based&quot;, because those terms have a very specific meaning in RL.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In that case, when would model-free learners be appropriate?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generally with current state of art in RL, if you don't have an accurate model provided as part of the problem definition, then model-free approaches are often superior. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is lots of interest in agents that build predictive models of the environment, and doing so as a &quot;side effect&quot; (whilst still being a model-free algorithm) can still be useful - it may regularise a neural network or help discover key predictive features that can also be used in policy or value networks. However, model-based agents that learn their own models for planning have a problem that inaccuracy in these models can cause instability (the inaccuracies multiply the further into the future the agent looks). Some promising inroads are being made using &lt;a href=&quot;https://deepmind.com/blog/agents-imagine-and-plan/&quot; rel=&quot;noreferrer&quot;&gt;imagination-based agents&lt;/a&gt; and/or mechanisms for deciding when and how much to trust the learned model during planning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now (in 2018), if you have a real-world problem in an environment without an explicit known model at the start, then the safest bet is to use a model-free approach such as DQN or A3C. That may change as the field is moving fast and new more complex architectures could well be the norm in a few years.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-01T20:47:19.957" LastActivityDate="2019-12-01T20:47:19.957" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="6734" PostTypeId="5" CreationDate="2018-06-13T16:56:09.623" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-13T16:56:09.623" LastActivityDate="2018-06-13T16:56:09.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6735" PostTypeId="4" CreationDate="2018-06-13T16:56:09.623" Score="0" Body="Long short-term memory" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-13T16:56:09.623" LastActivityDate="2018-06-13T16:56:09.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6736" PostTypeId="2" ParentId="6707" CreationDate="2018-06-13T17:22:17.407" Score="1" Body="&lt;p&gt;My feeling is that, because you are dealing with a subject that his highly subjective, you need to integrate whatever learning algorithm you use with human feedback.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is to say, crowdsource human opinions on a data set of pictures, train the algorithm to try to intuit what qualities images with similar human rankings share. (Both the positive and negative rankings.) Run the algorithm on a new data set and crowdsource that data set to see if the algorithm gets it right.  Rinse &amp;amp; repeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to utilize demographics in the human crowdsourcing.  Art historians will likely have different criteria for what makes a photo artistic than the general public.  Humans with different educational backgrounds will likely have different criteria for what makes a photo artistic.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sans the crowdsourcing, the algorithm will have no ability to determine the aesthetic qualities other than what the programmer defines, and in that case, the algorithm will only be able to determine what photos are artistic to the programmer, limiting the utility.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-06-13T17:22:17.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6737" PostTypeId="5" CreationDate="2018-06-13T17:34:43.330" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-22T21:23:06.837" LastActivityDate="2019-08-22T21:23:06.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6738" PostTypeId="4" CreationDate="2018-06-13T17:34:43.330" Score="0" Body="For questions related to the feed-forward neural network (FFNN), which is an artificial neural network wherein connections between the nodes do not form a cycle, so they are feed-forward (hence the name), as opposed to the recurrent neural network, which contains cyclic connections." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-22T21:23:06.837" LastActivityDate="2019-08-22T21:23:06.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6739" PostTypeId="2" ParentId="6696" CreationDate="2018-06-13T18:03:30.690" Score="0" Body="&lt;p&gt;Konstantine, I assume you refer to plain MLP and CNN, without any modifications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe what you ask is how to set both of them up, in order to have the fairest comparison possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I would do it, is to use their &lt;strong&gt;plain implementations&lt;/strong&gt; but &lt;strong&gt;both tuned as much as possible, in every hyperparameter&lt;/strong&gt;. Both should work as black boxes that accept the same inputs and give the same outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you insight on the true raw performance of both algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps :)&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-06-13T18:03:30.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6740" PostTypeId="2" ParentId="6702" CreationDate="2018-06-13T18:06:23.070" Score="1" Body="&lt;p&gt;My sense is that common sense tends to be axiomatic.  To avoid pitfalls, a degree of wisdom may also be required in that axioms may not apply in all contexts.  [See &lt;a href=&quot;https://en.wikipedia.org/wiki/Axiomatic_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Axiomatic System]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A major problem is that science often demonstrates that intuition, and &quot;common sense&quot;, often lead to incorrect conclusions.  Neil Degrasse Tyson covers this topic for the general public in his book &lt;a href=&quot;https://en.wikipedia.org/wiki/Death_by_Black_Hole&quot; rel=&quot;nofollow noreferrer&quot;&gt;Death By Black Hole&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Chapter 3, &quot;Seeing Isn't Believing&quot;, hints at the pitfalls of generalizing from too little evidence. It begins by making the point that although we know the Earth is round, it appears flat when one observes only a small, local portion of it. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A very famous example comes from mathematician Abraham Wald:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;During World War II, Wald was a member of the Statistical Research Group (SRG) where he applied his statistical skills to various wartime problems. These included methods of sequential analysis and sampling inspection. One of the problems that the SRG worked on was to examine the distribution of damage to aircraft to provide advice on how to minimize bomber losses to enemy fire. There was an inclination within the military to consider providing greater protection to parts that received more damage but Wald made the assumption that damage must be more uniformly distributed and that the aircraft that did return or show up in the samples were hit in the less vulnerable parts. Wald noted that the study only considered the aircraft that had survived their missions—the bombers that had been shot down were not present for the damage assessment. The holes in the returning aircraft, then, represented areas where a bomber could take damage and still return home safely. Wald proposed that the Navy instead reinforce the areas where the returning aircraft were unscathed, since those were the areas that, if hit, would cause the plane to be lost. &lt;br&gt;&lt;sub&gt;Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Abraham_Wald&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abraham Wald (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My sense is that &quot;confidence levels&quot; may be the main technique driving toward algorithmic &quot;common sense&quot;, specifically in that the algorithm is questioning it's assumptions.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-06-13T18:06:23.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6741" PostTypeId="1" AcceptedAnswerId="6743" CreationDate="2018-06-13T18:13:46.330" Score="4" ViewCount="184" Body="&lt;p&gt;Currently in my country, there is a system in which certain groups of researchers upload information on products of scientific interest, such as research articles, books, patents, software, among others. Depending on the number of products, the system assigns a classification to each group, which can be A1, A, B and C, where A1 is the highest classification and C is the minimum. According to the classification of the groups, they can compete to receive monetary incentives to make their research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the momen, I am working in an application that takes the data of the system that I mentioned previously. I am able to say what classification the group currently has because we develop a scraper that counts the products and there is another service that is in charge of implementing all the mathematical model that the system has to calculate the category of the group. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what I want to achieve is that my application would be able to give an estimate of how many products a research group should have to improve its category and i Want to know if I can do that using neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if there is a category C group, I want the application to tell the user with how many articles and books it would make its category go up to B. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I have seen in some web resources, I could insert a training set into the neural network and have it learn to classify the groups, but I think it is unnecessary, because I can do that mathematically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I do not understand if it is possible for a neural network to process the current category that the group has and be able to give suggestions of how many products it needs to improve its category.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it must be a neural network with several outputs, so that in each one it throws the total for each one of the products, although it is not necessary to list all the products that the measurement model contemplates. But it is necessary for the network to learn which products are handled by a certain group, for example if a group does not write books, avoid suggestions that contemplate the production of books for the improvement of the category that the group has.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you read the whole problem thanks in advance, I would appreciate any suggestion of what kind of subject I must research or any tutorial. I must say that I am a totally newbie in this subject.&lt;/p&gt;&#xA;" OwnerUserId="16258" LastEditorUserId="1671" LastEditDate="2018-06-13T18:21:44.983" LastActivityDate="2018-06-13T18:31:42.207" Title="Regression with more than one output, neural network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;ai-basics&gt;&lt;getting-started&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6743" PostTypeId="2" ParentId="6741" CreationDate="2018-06-13T18:31:42.207" Score="3" Body="&lt;p&gt;Welcome to AI-SE! Please use shorter description next time because it makes it hard to understand what you ask :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe you want a neural network that can predict future values of multiple variables given multiple inputs. This belongs to the general &lt;strong&gt;time series forecasting problem&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the best neural network architectures that can handle this problem is the LSTM Recurrent Neural Network (&lt;strong&gt;LSTM-RN&lt;/strong&gt;N). The reason I say &quot;best&quot; is because of their architecture that allows them to develop memory of what they have seen in the past and use it for future predictions. In other words they can cross-correlate in a linear/nonlinear fashion several past steps of multiple variables to future values of multiple other variables, like a black box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;No other neural network type does that.&lt;/strong&gt; They are the &lt;strong&gt;only type&lt;/strong&gt; of network that can learn the &lt;strong&gt;dynamics of a system&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An extremely useful tutorial for your purposes is &lt;a href=&quot;https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, please take a look.&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-06-13T18:31:42.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6744" PostTypeId="2" ParentId="6314" CreationDate="2018-06-13T22:11:41.257" Score="3" Body="&lt;p&gt;Dennis Soemers provides an important point that from a theoretical standpoint, this can be seen as a non-issue. However, what you bring up is an important practical issue of potential-based reward shaping (PBRS).&lt;/p&gt;&#xA;&lt;p&gt;The issue is actually worse than you describe---it's more general than &lt;span class=&quot;math-container&quot;&gt;$s = s'$&lt;/span&gt;. In particular, the issue presents itself differently based on the sign of your potential function. For example, in your case it looks like the potential function is positive: &lt;span class=&quot;math-container&quot;&gt;$P(s) &amp;gt; 0$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The issue (as you have found) is that an increase in potential (regardless of whether &lt;span class=&quot;math-container&quot;&gt;$s = s'$&lt;/span&gt;) might not be enough to &lt;em&gt;overcome&lt;/em&gt; the multiplication by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, and thus the PBRS term may be negative. In particular, only when the fold-change in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is large enough (&lt;span class=&quot;math-container&quot;&gt;$\frac{P(s')}{P(s)} &amp;gt; \frac{1}{\gamma}$&lt;/span&gt;) will the PBRS term actually be positive.&lt;/p&gt;&#xA;&lt;p&gt;The situation changes when the potential function is negative, i.e. if &lt;span class=&quot;math-container&quot;&gt;$P(s) &amp;lt; 0$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. In this case, you can actually get a &lt;em&gt;positive&lt;/em&gt; PBRS signal even when there is a decrease in potential! In particular, only when the fold-change in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is large enough (same inequality as before) will the PBRS term actually be negative.&lt;/p&gt;&#xA;&lt;p&gt;To summarize, when &lt;span class=&quot;math-container&quot;&gt;$P &amp;gt; 0$&lt;/span&gt;, a decrease in potential will always lead to a negative PBRS term, but an increase must overcome a barrier due to &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; for the term to be positive. When &lt;span class=&quot;math-container&quot;&gt;$P &amp;lt; 0$&lt;/span&gt;, an increase in potential will always lead to a positive PBRS term, but a decrease must overcome a barrier due to &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; for the term to be negative.&lt;/p&gt;&#xA;&lt;p&gt;The intuition behind PBRS is that improving the potential function should be rewarded, and decreasing it should be penalized. However, it turns out that whether or not this holds true depends on things like 1) the sign of the potential function, 2) the fold-change in potential, or 3) the resolution of your environment. For #3, if the temporal resolution of your environment can be altered such that an action brings you &amp;quot;partway&amp;quot; from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, then at some environment resolution you will run into one of the two problematic circumstances above. Another issue is that PBRS is highly sensitive to, for example, adding a constant to the potential function.&lt;/p&gt;&#xA;&lt;p&gt;Another related issue is that whether or not some constant improvement to the potential function leads to a positive/negative reward depends on how far you are from the &amp;quot;goal&amp;quot; state. Often potential functions are chosen such that they estimate how good a state is (after all, the best option for a potential function is the optimal value function). Say we choose &lt;span class=&quot;math-container&quot;&gt;$\gamma=0.99$&lt;/span&gt; and that &lt;span class=&quot;math-container&quot;&gt;$P(s_{goal}) = 1000$&lt;/span&gt; represents a goal state. Then increasing potential by one from &lt;span class=&quot;math-container&quot;&gt;$P(s) = 900$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$P(s') = 901$&lt;/span&gt; will have a &lt;em&gt;negative&lt;/em&gt; reward of &lt;span class=&quot;math-container&quot;&gt;$-8.01$&lt;/span&gt;. In contrast, increasing potential by one from &lt;span class=&quot;math-container&quot;&gt;$P(s) = 90$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$P(s') = 91$&lt;/span&gt; will have a small positive reward of &lt;span class=&quot;math-container&quot;&gt;$+0.09$&lt;/span&gt;. This is another issue: the sign of the PBRS term depends on distance from the goal.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/5381523/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; has some interesting examples and outlines many of the issues above.&lt;/p&gt;&#xA;&lt;p&gt;From my own experience, this is a large practical issue. The LunarLanderContinuous-v2 environment from OpenAI Gym includes a PBRS term, but they exclude the multiplication by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (i.e., &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;), presumably because the environment benchmark doesn't know the true discounting the RL user chooses. This environment can be solved using DDPG, for example, without significant hyperparameter tuning. However, if you use &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.99$&lt;/span&gt; for your RL formulation, and edit the LunarLander code such that the PBRS term includes &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.99$&lt;/span&gt;, then DDPG fails to solve the environment. So, this is not a small computational issue---it has dramatic effects on training.&lt;/p&gt;&#xA;&lt;p&gt;My solution has been to simply set &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; in the PBRS term, even when using, say, &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.99$&lt;/span&gt; in the RL formulation. This solves (or rather, circumnavigates) every issue above. While this loses out on the theoretical guarantee that adding the PBRS term does not affect the optimal policy, it can severely help training. (And there are no optimality guarantees using neural networks as function approximators anyway.)&lt;/p&gt;&#xA;&lt;p&gt;This solution also seems to be what most benchmark environments have adopted. For example, most MuJoCo environments use PBRS terms with no &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (equivalent to &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;). Alternatively, the omission of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; could be attributed to the fact that including it would require the environment to know &lt;strong&gt;a priori&lt;/strong&gt; what value of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; the RL user chose. While feeding this into an OpenAI gym environment is easy to do, it's not typically done.&lt;/p&gt;&#xA;&lt;p&gt;Keep in mind that while the theory guarantees that the optimal policy won't change by adding the PBRS term, adding the term doesn't necessarily help you &lt;em&gt;approach&lt;/em&gt; the optimal policy. Yet, the whole point of using PBRS at all is to help you approach a good policy. So, it's a bit of a paradox, and I was comfortable with sacrificing the theoretical guarantee of policy invariance if it meant I could actually get to a good policy in the first place.&lt;/p&gt;&#xA;" OwnerUserId="16264" LastEditorUserId="1641" LastEditDate="2020-07-02T19:21:34.527" LastActivityDate="2020-07-02T19:21:34.527" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6745" PostTypeId="1" CreationDate="2018-06-14T01:52:37.993" Score="3" ViewCount="160" Body="&lt;p&gt;I have trained (with different sizes, learning rates, and epochs) a SOM network to cluster &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/iris&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Iris dataset&lt;/a&gt;. The instances associated with the class &lt;code&gt;setosa&lt;/code&gt; have been mainly fitted to a 1-2 BMUs. In the case of &lt;code&gt;virginica&lt;/code&gt;, the instances have also be associated with only a few BMUs. However, in the case of &lt;code&gt;versicolor&lt;/code&gt; instances, many BMUs have been associated with them.&lt;/p&gt;&#xA;&lt;p&gt;Is this normal?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Setosa&#xA;0. 1846&#xA;1. 1846&#xA;2. 1846&#xA;3. 1846&#xA;4. 1846&#xA;5. 1846&#xA;6. 1846&#xA;7. 1846&#xA;8. 1846&#xA;9. 1846&#xA;10. 1846&#xA;11. 1846&#xA;12. 1846&#xA;13. 1846&#xA;14. 1846&#xA;15. 1846&#xA;16. 1846&#xA;17. 1846&#xA;18. 1846&#xA;19. 1846&#xA;20. 1846&#xA;21. 1846&#xA;22. 1846&#xA;23. 1846&#xA;24. 1846&#xA;25. 1846&#xA;26. 1846&#xA;27. 1846&#xA;28. 1846&#xA;29. 1846&#xA;30. 1846&#xA;31. 1846&#xA;32. 1846&#xA;33. 1846&#xA;34. 1846&#xA;35. 1846&#xA;36. 1846&#xA;37. 1846&#xA;38. 1846&#xA;39. 1846&#xA;40. 1846&#xA;41. 1620&#xA;42. 1846&#xA;43. 1846&#xA;44. 1846&#xA;45. 1846&#xA;46. 1846&#xA;47. 1846&#xA;48. 1846&#xA;49. 1846&#xA;&#xA;Versicolor&#xA;50. 652&#xA;51. 652&#xA;52. 652&#xA;53. 1259&#xA;54. 696&#xA;55. 1394&#xA;56. 652&#xA;57. 490&#xA;58. 696&#xA;59. 490&#xA;60. 490&#xA;61. 1059&#xA;62. 1304&#xA;63. 696&#xA;64. 490&#xA;65. 652&#xA;66. 1400&#xA;67. 490&#xA;68. 696&#xA;69. 490&#xA;70. 652&#xA;71. 1574&#xA;72. 696&#xA;73. 832&#xA;74. 696&#xA;75. 696&#xA;76. 696&#xA;77. 652&#xA;78. 696&#xA;79. 490&#xA;80. 490&#xA;81. 490&#xA;82. 444&#xA;83. 696&#xA;84. 1129&#xA;85. 1084&#xA;86. 652&#xA;87. 696&#xA;88. 25&#xA;89. 584&#xA;90. 490&#xA;91. 789&#xA;92. 1034&#xA;93. 490&#xA;94. 854&#xA;95. 29&#xA;96. 584&#xA;97. 877&#xA;98. 490&#xA;99. 809&#xA;&#xA;Virginica&#xA;100. 652&#xA;101. 696&#xA;102. 652&#xA;103. 652&#xA;104. 652&#xA;105. 652&#xA;106. 877&#xA;107. 652&#xA;108. 696&#xA;109. 652&#xA;110. 652&#xA;111. 696&#xA;112. 652&#xA;113. 696&#xA;114. 652&#xA;115. 652&#xA;116. 652&#xA;117. 652&#xA;118. 652&#xA;119. 696&#xA;120. 652&#xA;121. 652&#xA;122. 652&#xA;123. 696&#xA;124. 652&#xA;125. 652&#xA;126. 696&#xA;127. 652&#xA;128. 652&#xA;129. 652&#xA;130. 652&#xA;131. 652&#xA;132. 652&#xA;133. 696&#xA;134. 696&#xA;135. 652&#xA;136. 652&#xA;137. 652&#xA;138. 652&#xA;139. 652&#xA;140. 652&#xA;141. 652&#xA;142. 696&#xA;143. 652&#xA;144. 652&#xA;145. 652&#xA;146. 696&#xA;147. 652&#xA;148. 652&#xA;149. 652&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I have a diagram. It doesn't look bad.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7eqOz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7eqOz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15587" LastEditorUserId="2444" LastEditDate="2020-12-02T21:34:20.923" LastActivityDate="2021-01-01T22:08:20.740" Title="Is it normal that SOM clusters the instances with the &quot;versicolor&quot; class into multiple different BMUs?" Tags="&lt;neural-networks&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;&lt;self-organizing-map&gt;&lt;iris-dataset&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6746" PostTypeId="5" CreationDate="2018-06-14T02:33:25.797" Score="0" Body="&lt;p&gt;A self-organising map (SOM) or self-organising feature map (SOFM) is a type of artificial neural network (ANN) that is trained using unsupervised learning to produce a low-dimensional (typically two-dimensional), discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction. Self-organising maps differ from other artificial neural networks as they apply competitive learning as opposed to error-correction learning (such as back-propagation with gradient descent), and in the sense that they use a neighbourhood function to preserve the topological properties of the input space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-organizing_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-organizing map - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-06-14T21:21:57.283" LastActivityDate="2018-06-14T21:21:57.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6747" PostTypeId="4" CreationDate="2018-06-14T02:33:25.797" Score="0" Body="For questions related to different types of Self Organising Maps." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-06-14T21:21:47.020" LastActivityDate="2018-06-14T21:21:47.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6748" PostTypeId="1" CreationDate="2018-06-14T02:44:04.580" Score="6" ViewCount="594" Body="&lt;p&gt;I'm a professional game developer investigating the potential for using reinforcement learning to build strategy game AI opponents that have more creative behavior compared to traditional techniques like behavior trees. I have a few questions I've bolded below, any thoughts would be helpful, and could save me from pursuing dead ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created a very boring and tiny game as a test case. Two players each control a fleet of ships, each ship has health and can fire on one other ship each turn dealing some damage. The player and his opponent assigns orders to their respective ships, telling them which target to attack, and then the turn is resolved. Ships with 0 health are removed from the board. The player that loses all their ships first loses the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I was using TensorFlow, at a very high level I need to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A) Create a training program that outputs a trained graph to a file. The training program will need to map gamestates into tensors, feed the tensors through the graph to produce actions, execute actions on the gamestate to generate a new state, and evaluate the reward function for the new state. Repeat a bunch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B) Take the graph created in #1, load it at the game runtime, and use the graph to generate intelligent actions from real gamestates during the Player vs AI match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As soon as I started digging into TensorFlow, questions immediately came up, and now I'm not quite sure if there is a more appropriate library to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I) TensorFlow has a High Level python API, and a Low Level C++ API. Most games are built in C++, and thus using a C++ or C API is preferable, it makes integration with the game much simpler. In principle we could use pybind or some other scheme for sending state from C++ to Python and back again, but that's not ideal. &#xA;&lt;strong&gt;Question 1: How much do I lose by using the low level API specifically for reinforcement learning, compared to the high level API?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;II) Platforms. 99.9% of the time, PC/Console games are developed in Windows environments, and so having Tensorflow work in Windows is critical. From my googling, Tensorflow just &lt;em&gt;barely&lt;/em&gt; supports building in Windows using CMake, though it requires some finagling. More worryingly are other platforms:&#xA;&lt;strong&gt;Question 2: What hope is there of running the TensorFlow library on consoles like XboxOne, Playstation 4, or the Switch? I imagine this would require manually porting the entire source :(&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;III) TensorFlow is big, and it seems you need to basically link all of it to ship with the game.&#xA;&lt;strong&gt;Question 3: Is there any way to get a slimmed down &quot;Runtime TensorFlow&quot; library that is only capable of loading a graph and transforming states into actions?&lt;/strong&gt;&#xA;It seems like if the answer was yes, it might also be easier to port this smaller runtime version to more platforms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 4: Should I even be using TensorFlow for this? Is there perhaps something more suitable?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks again if you read all that, I'm eager to start tinkering, but would like to set off in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="16265" LastEditorUserId="1671" LastEditDate="2018-06-14T21:23:18.807" LastActivityDate="2019-07-02T12:42:19.403" Title="Reinforcement Learning in Commercial Strategy Games" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;game-ai&gt;&lt;getting-started&gt;&lt;gaming&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ClosedDate="2020-09-13T12:54:56.783" ContentLicense="CC BY-SA 4.0" />
  <row Id="6750" PostTypeId="1" AcceptedAnswerId="6752" CreationDate="2018-06-14T06:23:57.040" Score="3" ViewCount="147" Body="&lt;p&gt;As I know, if we consider a 3*3 kernel, we should add a padding of 1px to the source image(if we want to have effect on whole of the image), then we start to put the kernel in upper-left side of the image and multiplying each element of kernel to corresponding pixel on image. Then we sum all the results and put it on the anchor point of kernel(usually center element). Then we should shift the kernel one step to right side and do these things again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am right till here, I have a question about the summation results. I want to know: should we consider the replaced value of image in previously calculated summation and replaced in anchor point in new step of calculation or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean we must put the anchor point's result in source image and consider it in calculations of shifted kernel? Or we must put it in distance image and we don't consider these results when we shift the kernel on source image(It means don't replace the results on source image for next steps calculations)?&lt;/p&gt;&#xA;" OwnerUserId="9941" LastActivityDate="2018-06-14T07:01:03.110" Title="How to apply a kernel to an image?" Tags="&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6752" PostTypeId="2" ParentId="6750" CreationDate="2018-06-14T06:50:31.460" Score="3" Body="&lt;p&gt;Assuming that you are performing a normal discrete convolution, then you would use the original source image for all calculations, and only replace at the end. As you put it, like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;put it in distance image and we don't consider these results when we shift the kernel on source image(It means don't replace the results on source image for next steps calculations)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The other option, changing in-place during the operation, does not relate to any common use that I know of, but might have some interesting behaviour in studies of cellular automata or similar.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-14T07:01:03.110" LastActivityDate="2018-06-14T07:01:03.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6753" PostTypeId="1" AcceptedAnswerId="6754" CreationDate="2018-06-14T09:14:09.180" Score="4" ViewCount="142" Body="&lt;p&gt;It seems that older RNNs have a limitation for their use cases and have been outperformed by other recurrent architectures, such as the LSTM and GRU.&lt;/p&gt;&#xA;" OwnerUserId="16255" LastEditorUserId="2444" LastEditDate="2019-11-02T15:23:37.420" LastActivityDate="2020-06-15T21:00:21.550" Title="Why are GRU and LSTM better than standard RNNs?" Tags="&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6754" PostTypeId="2" ParentId="6753" CreationDate="2018-06-14T09:15:34.727" Score="2" Body="&lt;p&gt;These newer RNNs (LSTMs and GRUs) have greater memory control, allowing previous values to persist or to be reset as necessary for many sequences of steps, avoiding &quot;gradient decay&quot; or eventual degradation of the values passed from step to step. LSTM and GRU networks make this memory control possible with memory blocks and structures called &quot;gates&quot; that pass or reset values as appropriate.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastEditorUserId="2444" LastEditDate="2020-06-15T21:00:21.550" LastActivityDate="2020-06-15T21:00:21.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6756" PostTypeId="1" AcceptedAnswerId="6759" CreationDate="2018-06-14T11:29:37.730" Score="2" ViewCount="244" Body="&lt;p&gt;I watched a youtube clip of Elon Musk talking about his view on the future of AI. He gave two examples. One of the examples was a benign scenario and the other example was a nonbenign scenario where he speculated the possibilities of future AI threats and what harm a deep intelligence could do.&lt;/p&gt;&#xA;&lt;p&gt;According to Elon, a deep intelligence in the network could create fake news and spoof email accounts. &amp;quot;The pen is mightier than the sword&amp;quot;. This non-benign scenario put forth by Elon was a hypothetical, but he went into detail about how it could have been possible that an AI, with the goal of maximising the portfolio of stocks, to go long on defense and short on the consumer, and start a war.&lt;/p&gt;&#xA;&lt;p&gt;To be more specific, this could be achieved by hacking into the Malaysians Airlines aircraft routing server, and when the aircraft is over a warzone, send an anonymous tip that there is an enemy aircraft flying overhead which in turn would cause ground to air missiles to take down what was actually a &amp;quot;commercial&amp;quot; airliner.&lt;/p&gt;&#xA;&lt;p&gt;Although this is a plausible hypothetical nonbenign scenario of AI, I'm wondering if this actually could have been the case regarding the Malaysian Airliner crash. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Stuxnet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stuxnet&lt;/a&gt;, for example, was a malicious computer worm, first uncovered in 2010. Thought to have been in development since at least 2005 and believed to be responsible for causing substantial damage to Iran's nuclear program. The Stuxnet wasn't even an AI.&lt;/p&gt;&#xA;&lt;p&gt;The Stuxnet blew the world's minds when it was discovered. The sheer complexity of the worm and the amount of time it took to build was impressive, to say the least.&lt;/p&gt;&#xA;&lt;p&gt;In conclusion, was the &lt;a href=&quot;https://en.wikipedia.org/wiki/Malaysia_Airlines_Flight_370&quot; rel=&quot;nofollow noreferrer&quot;&gt;Malaysian Airliner crash&lt;/a&gt; caused by a non-benign artificial intelligence system?&lt;/p&gt;&#xA;" OwnerUserId="13252" LastEditorUserId="2444" LastEditDate="2020-09-10T11:50:14.307" LastActivityDate="2020-09-10T11:50:26.753" Title="Was the Malaysian Airliner crash caused by a non-benign artificial intelligence system?" Tags="&lt;neural-networks&gt;&lt;agi&gt;&lt;social&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6757" PostTypeId="1" CreationDate="2018-06-14T21:22:41.780" Score="3" ViewCount="172" Body="&lt;p&gt;If both the players want to increase their score (by selecting the highest or best cost path), can this be done using the minimax algorithm, or are there other algorithms for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="16280" LastEditorUserId="2444" LastEditDate="2019-12-20T16:16:37.037" LastActivityDate="2019-12-20T16:16:37.037" Title="Can minimax be used when both players want to increase their score?" Tags="&lt;search&gt;&lt;game-theory&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6758" PostTypeId="2" ParentId="6757" CreationDate="2018-06-14T21:31:43.513" Score="1" Body="&lt;p&gt;I believe maximax is what you're looking for:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Maximax&lt;/strong&gt; (economics, computer science, decision theory) A strategy or algorithm that seeks to maximize the maximum possible result (that is, that prefers the alternative with the chance of the best possible outcome, even if its expected outcome and its worst possible outcome are worse than other alternatives); often used attributively, as &amp;quot;maximax strategy&amp;quot;, &amp;quot;maximax approach&amp;quot;, and so on.&lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://en.wiktionary.org/wiki/maximax&quot; rel=&quot;nofollow noreferrer&quot;&gt;maximax (wiki&lt;/a&gt;)&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You may also be interested in &amp;quot;minimax regret&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The &lt;strong&gt;minimax regret&lt;/strong&gt; approach is to minimize the worst-case regret. The aim of this is to perform as closely as possible to the optimal course. Since the minimax criterion applied here is to the regret (difference or ratio of the payoffs) rather than to the payoff itself, it is not as pessimistic as the ordinary minimax approach.&lt;/p&gt;&#xA;&lt;p&gt;One benefit of minimax (as opposed to expected regret) is that it is independent of the probabilities of the various outcomes: thus if regret can be accurately computed, one can reliably use minimax regret. However, probabilities of outcomes are hard to estimate.&lt;/p&gt;&#xA;&lt;p&gt;This differs from the standard minimax approach in that it uses differences or ratios between outcomes, and thus requires interval or ratio measurements, as well as ordinal measurements (ranking), as in standard minimax.&#xA;&lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://en.wikipedia.org/wiki/Regret_(decision_theory)#Minimax_regret&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimax regret (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Apologies for just grabbing the wikis on this, but they are accurate.  Most of the references from economic sites are commercial.  If interested, when I have some more time, I could probably link to some peer-reviewed papers.  Hopefully, at the very least, this answer provides information for further research.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-06-14T21:31:43.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6759" PostTypeId="2" ParentId="6756" CreationDate="2018-06-14T21:40:25.080" Score="3" Body="&lt;p&gt;Short answer: despite the incredible advances in AI via Machine Learning (and subfields) AI is nowhere near this kind of autonomous decision-making.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't &lt;a href=&quot;https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy)#Proving_a_negative&quot; rel=&quot;nofollow noreferrer&quot;&gt;prove a negative&lt;/a&gt;, but, the level of autonomy Musk is talking about is still on the horizon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doesn't mean Musk is wrong about the hypothetical.  imo I'm glad he keeps bringing these kinds of issues to the fore.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Engineers and mathematicians have a bit more credibility on this subject than laypeople.  If it seems alarmist, there are many in the scientific community who feel it is warranted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What Musk is describing is an extension of what Asimov felt compelled to warn us about via his &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Three Laws of Robotics&lt;/a&gt;&quot; back in 1942, when computers sucked.  For an explication of this idea re: Machine Learning, see below.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;JUST FOR FUN (SORT OF:)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Author and mathematical physicist Hannu Rajaniemi just published a story in the MIT Tech Review on this very subject.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/610831/unchained-a-story-of-love-loss-and-blockchain/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unchained: A Story of Love, Loss, and Blockchain&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Warning: the story is both wickedly funny, surprisingly moving, and very likely prescient!&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-15T21:17:34.463" LastActivityDate="2018-06-15T21:17:34.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6760" PostTypeId="1" CreationDate="2018-06-14T23:19:24.140" Score="1" ViewCount="81" Body="&lt;p&gt;I would really appreciate if someone could comment the following method of training neural nets &lt;strong&gt;providing them with some meta data (Making them more color prone only if needed, whereas now they're mostly silhouette / outline aware). (Comment and especially giving a reference to some papers, preventing me from reinventing the wheel.)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But let's start at the very begining and say that we're performing simple image recognition and each image has 24 bit color depth, simply 1 Byte per each RGB channel. I'm more eager to use usually bigger pics, sacrifing color quality, however &lt;strong&gt;not in all cases&lt;/strong&gt; (that statement is crucial in this question).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To limit the computational burden I'm &lt;strong&gt;NOT keen on using the full information about color (3 Bytes per pixel), but rather shrink it to 1 Byte per pixel&lt;/strong&gt; and here is the catch:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I'm reluctant neither to use gray scale nor to cast original tints to single (common among all pics) color palette of 256 hues. So I came up with an idea of reversing the method called debayering or demosaicing image from Color Filter Array data).&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achive it, for every pixel only one color channel is preserved.&#xA;Because of the human perception of colors, green component is overrepresented covering 50% of pixels, so 25% left for blue and red.&#xA;In this particular example below, the upper left pixel correspond to Blue, followed by Green, then Blue, next Green and so on to the end of the row.&#xA;The second uppermost row starts with Green, afterwards Red, one more Green, Red and also repeating till the end of line. This horizontal patterns are altering with the parity of the row number, which is nicely depiced on &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayer_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Bayer_filter&lt;/a&gt; from where I've got following graphics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wKRfH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wKRfH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To better illustrate this method I'm using the thumbnail of famous Mona Lisa painting with its grayscale version next to it. (By the way, it isn't in my training set, but is familiar to everyone).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NCDir.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NCDir.jpg&quot; alt=&quot;Original&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sq3l1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sq3l1.jpg&quot; alt=&quot;Original B&amp;amp;W&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A greenish leftmost image is the result of applying the reverse debayering / demosaicing CFA method. This picture consist of pixels that are either Blue or Red or Green with different brightness level. In the browser window that could be poorly visible, however if you download this image and magnify it substantially, the patter would be revealed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that in the original picture one can find a small square of 2x2 pixels, all of them representing a light skintone 0xF4D374 (in hex). In this grouping, 2 green pixels would be chopped into green channel and will get a value of 0xD3, the blue-related will get a value of 0x74 and the remaining red would get 0xF4.&#xA;In the leftmost image below the corresponding pixels were presented by hex colors: 0x00D300, 0x000074 and 0xF40000 respectively, whereas in the right picture exactly the same values (0xD3, 0x74, 0xF4) were shown in grayscal (of 256 possible shades). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qCH96.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qCH96.jpg&quot; alt=&quot;CFA applied&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/9RMtd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9RMtd.jpg&quot; alt=&quot;CFA applied plus B&amp;amp;W&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this color-flattening, our input batch has shrinked by two-thirds and at the same time original colors can be more-or-less restored (of course not lossless, but well enough).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't suppose that anyone had a problem with recognising this picture after transformation. Likewise, all my models could be well trained to recognize outline/silhouette of the object, but they require a way lot more training data (at least one-two orders of magnitude) to be color-aware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The ultimate question is, how to design models that would threat shape and colors in similar manner. Maybe that would be not in 100% mathematically proper, but shape and color must be orthogonal.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Nevertheless, I don't want to always decode the color, but only if its needed - in ealier epoch it learned sillhouettes/shapes and that there're many similar objects in this regard, so in the next epoch it should pay also / more attention to tints.&lt;/strong&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Have you encountered articles about such method of using color if object demarcation / labelling process cannot be based only on shape? I would be really gratefull for any paper or other reference.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm rather newbie to neural nets, so sorry if this is something widely known to everyone but me ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any hint.&lt;/p&gt;&#xA;" OwnerUserId="2629" LastActivityDate="2018-06-14T23:19:24.140" Title="Object recognition by two or more traits that are orthogonal (informally speaking)" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;training&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6761" PostTypeId="2" ParentId="6333" CreationDate="2018-06-15T05:19:57.770" Score="1" Body="&lt;p&gt;Google may appear to have a strong advantage over its competition in Narrow AI due to its acquisition of other AI companies such as Boston dynamics. Google also has extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions which is useful as training data when creating AGI. However, achieving the goal of an AGI is far more complex in terms of the amount of research that needs to be done, regardless of how much data you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, a dedicated research company such as openAI, which works non-for profit, can certainly influence AGI in a positive way and be a very strong contender to Google. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, China is pouring billions and billions of dollars into AI at the moment as China is well behind the rest of the world in the AI race. They are however, now employing thousands upon thousands of the greatest, most innovative minds around the world and footing the bill with the aim to gain more momentum in this field, keep up to pace with the rest of the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;China essentially have a very good chance of winning the AI race just due to the sheer amount of money in grants it is handing out to researchers and startup AI companies.&lt;/p&gt;&#xA;" OwnerUserId="13252" LastEditorUserId="13252" LastEditDate="2018-06-15T05:27:02.680" LastActivityDate="2018-06-15T05:27:02.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6762" PostTypeId="1" AcceptedAnswerId="6764" CreationDate="2018-06-15T09:11:56.317" Score="3" ViewCount="272" Body="&lt;p&gt;In some implementations of off-policy Q-learning, we need to know the action probabilities given by the behavior policy &lt;span class=&quot;math-container&quot;&gt;$\mu(a)$&lt;/span&gt; (e.g., if we want to use importance sampling).&lt;/p&gt;&#xA;&lt;p&gt;In my case, I am using Deep &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Learning&lt;/a&gt; and selecting actions using &lt;a href=&quot;https://en.wikipedia.org/wiki/Thompson_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thompson Sampling&lt;/a&gt;. I implemented this following the approach in &lt;a href=&quot;http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;What My Deep Model Doesn't Know...&amp;quot;&lt;/a&gt;: I added dropout to my Q-network and select actions by performing a single stochastic forward pass through the Q-network (i.e., with dropout enabled) and choosing the action with the highest Q-value.&lt;/p&gt;&#xA;&lt;p&gt;So, how can I calculate &lt;span class=&quot;math-container&quot;&gt;$\mu(a)$&lt;/span&gt; when using Thompson Sampling based on dropout?&lt;/p&gt;&#xA;" OwnerUserId="16286" LastEditorUserId="2444" LastEditDate="2021-01-05T12:13:21.347" LastActivityDate="2021-01-05T12:13:32.183" Title="How to compute the action probabilities with Thompson sampling in deep Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;dropout&gt;&lt;thompson-sampling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6763" PostTypeId="2" ParentId="2777" CreationDate="2018-06-15T09:42:49.920" Score="0" Body="&lt;p&gt;This is a hard problem to solve, and the best approach depends very much on the scope of your task. If you have a small database table with a limited number of columns, you might get away with some basic pattern matching techniques. If it is more complex than that, you might have to do a full-scale syntactic analysis of the question. This also depends on the variations of possible question types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming a limited set of variables and variants, you could set up something like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How many X did Y produce/How many X were done by Y/What is the number of X for Y&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where you have two variables to fill from the pattern, which you then use in your query:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;select sum(X) where producer == Y&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(Or whatever format your query has).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this is that you don't need to be a linguistics expert to maintain/expand the system, and you can just add more patterns to it if necessary. You might have to map some terms onto synonyms to get the right column headings/labels out of it. But this approach is not very hard to implement, and you should have a basic system up and running fairly quickly. You then have to see/test what questions your users are asking, and expand the pattern inventory accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The disadvantage is that you might end up with a long list of patterns, and there could be some which are conflicting, ie the same pattern with different variables will ask for a different kind of result. If that turns out to be a problem, you might have to look for a more powerful approach.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-15T09:42:49.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6764" PostTypeId="2" ParentId="6762" CreationDate="2018-06-15T11:36:47.503" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;So, how can I calculate &lt;span class=&quot;math-container&quot;&gt;$\mu(a)$&lt;/span&gt; when using Thompson Sampling based on dropout?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The only way I could see this being calculated is if you iterate over all possible dropout combinations, or as an approximation sample say 100 or 1000 actions with different dropout, to get a rough distribution.&lt;/p&gt;&#xA;&lt;p&gt;I don't think this is feasible for practical reasons (the agent will learn so much more slowly due to these calculations, you may as well abandon Thompson Sampling and use epsilon-greedy), and you will have to avoid using importance sampling if you also want to use action-selection techniques where there is no easy way to calculate a distribution.&lt;/p&gt;&#xA;&lt;p&gt;Many forms of Q-learning do not use importance sampling. These typically just reset eligibility traces if the selected action is different from maximising action.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-01-05T12:13:32.183" LastActivityDate="2021-01-05T12:13:32.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6765" PostTypeId="1" CreationDate="2018-06-15T11:40:24.003" Score="14" ViewCount="656" Body="&lt;p&gt;I read Judea Pearl's &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/046509760X&quot; rel=&quot;noreferrer&quot;&gt;The Book of Why&lt;/a&gt;, in which he mentions that deep learning is just a glorified curve fitting technology, and will not be able to produce human-like intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From his book there is this diagram that illustrates the three levels of cognitive abilities: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vxTLx.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vxTLx.png&quot; alt=&quot;Three levels of cognitive abilities&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that the &quot;intelligence&quot; produced by current deep learning technology is only at the level of association. Thus the AI is nowhere near the level of asking questions like &quot;how can I make Y happen&quot; (intervention) and &quot;What if I have acted differently, will X still occur?&quot; (counterfactuals), and it's highly unlikely that curve fitting techniques can ever bring us closer to a higher level of cognitive ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found his argument persuasive on an intuitive level, but I'm unable to find any physical or mathematical laws that can either bolster or cast doubt on this argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there any  scientific/physical/chemical/biological/mathematical argument that prevents deep learning from ever producing strong AI (human-like intelligence)?&lt;/p&gt;&#xA;" OwnerUserId="16291" LastEditorUserId="1671" LastEditDate="2018-06-15T21:10:22.287" LastActivityDate="2021-04-04T15:14:34.153" Title="Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI?" Tags="&lt;deep-learning&gt;&lt;agi&gt;&lt;math&gt;&lt;theory&gt;&lt;superintelligence&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6766" PostTypeId="1" CreationDate="2018-06-15T11:56:04.230" Score="3" ViewCount="2333" Body="&lt;p&gt;As you know, &lt;a href=&quot;https://opencv.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV&lt;/a&gt; is a great open-source library for image recognition and related tasks.&lt;/p&gt;&#xA;&lt;p&gt;Is there a similar library for voice recognition and natural language processing (NPP)?&lt;/p&gt;&#xA;&lt;p&gt;I know &lt;a href=&quot;http://espeak.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;espeak&lt;/a&gt; for TTS, also &lt;a href=&quot;https://pypi.org/project/pocketsphinx/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pocketsphinx&lt;/a&gt; for voice recognition. Are there better options?&lt;/p&gt;&#xA;" OwnerUserId="9941" LastEditorUserId="2444" LastEditDate="2021-01-27T19:14:26.940" LastActivityDate="2021-01-27T19:16:16.560" Title="Is there something like OpenCV for voice recognition and NLP?" Tags="&lt;natural-language-processing&gt;&lt;resource-request&gt;&lt;voice-recognition&gt;&lt;c++&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ClosedDate="2021-01-27T19:14:32.893" ContentLicense="CC BY-SA 4.0" />
  <row Id="6767" PostTypeId="2" ParentId="6766" CreationDate="2018-06-15T13:14:29.763" Score="2" Body="&lt;p&gt;I don't know about voice recognition, but, for NLP, I think that &lt;a href=&quot;https://radimrehurek.com/gensim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gensim&lt;/a&gt; could be what you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;Gensim is an NLP package that contains efficient implementations of many well-known functionalities for the tasks of topic modeling, such as tf–idf, Latent Dirichlet allocation, Latent semantic analysis, etc.&lt;/p&gt;&#xA;" OwnerUserId="16136" LastEditorUserId="2444" LastEditDate="2021-01-27T19:16:16.560" LastActivityDate="2021-01-27T19:16:16.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6768" PostTypeId="2" ParentId="6728" CreationDate="2018-06-15T13:48:15.487" Score="1" Body="&lt;p&gt;What are the top contributions from neuroscience to artificial intelligence and vice versa?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a glimpse of one of my favorite companies doing it big in artificial intelligence field,inline with its contribution to neuroscience;otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DeepMind&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;It's goal is to build a general AI systems with the ability to think,reasoning and learn flexibility and rapidly,Systems which can intuit about the real world and imagine better ones.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much progress has been made from the interaction between these two fields?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to  &lt;strong&gt;Demis Hassabis&lt;/strong&gt; ,the founder of DeepMind,in his paper published in journal Neuron.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;..the recent findings in neuroscience show that the hippocampus a sea horse structure in acts as a hub for encoding memory,replays those experiences in fast forward during rest and sleep....&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is there a formal field dedicated to the research of topics in the intersection between AI and neuroscience, with papers being published and maybe conferences being held periodically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Currently,Deep Mind is doing unrest research despite their paper being published &quot; &lt;a href=&quot;https://www.cell.com/neuron/abstract/S0896-6273(17)30509-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroscience and artificial intelligence&lt;/a&gt;&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;also Michael has mentioned it in the comment.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;note&lt;/strong&gt; ;New scientists work to solve the neural code,the basic computations which supports brain function.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="1581" LastEditDate="2018-06-15T13:54:05.667" LastActivityDate="2018-06-15T13:54:05.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6770" PostTypeId="1" CreationDate="2018-06-15T14:32:40.510" Score="9" ViewCount="358" Body="&lt;p&gt;I'm trying to detect the &lt;strong&gt;visual attention&lt;/strong&gt; area in a given image and crop the image into that area. For instance, given an image of any size and a rectangle of say LxW dimension as an input, I would like to crop the image to the most important visual attention area. I'm looking for a state of the art approach for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we have any tools or SDK to implement that? Any piece of code or algorithm would really help.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2019-05-07T04:06:56.330" Title="Detect visual attention area in an image" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;algorithm&gt;&lt;computer-vision&gt;&lt;pattern-recognition&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6771" PostTypeId="2" ParentId="6765" CreationDate="2018-06-15T15:03:03.097" Score="1" Body="&lt;p&gt;It is a paradox, but a deep learning machine (defined as a NeuralNet variant) is unable to learn anything. It is a flexible and configurable hardware/software architecture that can be parametrized to solve a lot of problems. But the optimal parameters to solve a problem are obtained by an external system, i.e. back-propagation algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back-propagation subsystem uses conventional programming paradigms, it is not a Neural Net. This fact is in absolute opposition to human mind, where learning and use of knowledge is done by the same system (the mind).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all the real interesting things are done outside the NN, it is difficult to claim that a NN (in any variant) can develop in an AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to find some more differences. Neural nets are strongly numerical in its interface and internals. From this point of view, they are an evolution of support vector machines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Too much differences and restrictions to expect an AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I strongly disagree in the draw included in the original question. &quot;Seeing&quot;, &quot;doing&quot;, &quot;imaging&quot; are levels absolutely wrong. It ignores from basic and common software concepts as &quot;abstraction&quot; or &quot;program state&quot; (of mind, in Turing words); applied AI ones as &quot;foresee&quot;; and AGI ones as &quot;free will&quot;, &quot;objectives and feelings&quot;, ... &lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-06-15T16:16:05.243" LastActivityDate="2018-06-15T16:16:05.243" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6772" PostTypeId="1" CreationDate="2018-06-16T08:29:49.487" Score="3" ViewCount="261" Body="&lt;p&gt;According to this &lt;a href=&quot;https://www.engadget.com/2018/06/14/microsoft-ai-windows-10-updates-smoother/&quot; rel=&quot;nofollow noreferrer&quot;&gt;news&lt;/a&gt;, Microsoft is using AI to make Windows 10 updates smoother. So I was curious and went further to search and came across this &lt;a href=&quot;https://blogs.windows.com/windowsexperience/2018/06/14/ai-powers-windows-10-april-2018-update-rollout/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;, which describes:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial Intelligence (AI) continues to be a key area of investment&#xA;  for Microsoft, and we’re pleased to announce that for the first time&#xA;  we’ve leveraged AI at scale to greatly improve the quality and&#xA;  reliability of the Windows 10 April 2018 Update rollout.  &lt;strong&gt;Our AI&#xA;  approach intelligently selects devices that our feedback data indicate&#xA;  would have a great update experience and offers the April 2018 Update&#xA;  to these devices first.&lt;/strong&gt;  As our rollout progresses, we continuously&#xA;  collect update experience data and retrain our models to learn which&#xA;  devices will have a positive update experience, and where we may need&#xA;  to wait until we have higher confidence in a great experience.  Our&#xA;  overall rollout objective is for a safe and reliable update, which&#xA;  means we only go as fast as is safe.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Our AI/Machine Learning approach started with a pilot program during&#xA;  the Windows 10 Fall Creators Update rollout.  &lt;strong&gt;We studied&#xA;  characteristics of devices that data indicated had a great update&#xA;  experience and trained our model to spot and target those devices.&lt;/strong&gt;  In&#xA;  our limited trial during the Fall Creators Update rollout, we&#xA;  consistently saw a higher rate of positive update experiences for&#xA;  devices identified using the AI model, with &lt;strong&gt;fewer rollbacks,&#xA;  uninstalls, reliability issues, and negative user feedback&lt;/strong&gt;. For the&#xA;  April 2018 Update rollout, we substantially expanded the scale of AI&#xA;  by developing &lt;strong&gt;a robust AI machine learning model to teach the system&#xA;  how to identify the best target devices based on our extensive&#xA;  listening systems.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To me, it sounds like simple if-else statements would have implemented the whole thing without touching the AI; they mentioned that positive experiences include fewer rollbacks, uninstalls, and so on, so we may use these as a criterion of a positive experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am just wondering if the word 'AI' is being misused, or can be misleading in this context? Could anyone point me out on this or give any insight on how AI can be used in this context? In my experience, I have only seen AI mostly being used in speech recognition, image recognition and other sort-of classifying problems, with a training and consequently a computer can &quot;learn&quot; from the data, not like an if-else statement. Today, AI seems to be everything that is considered &quot;smart&quot;?&lt;/p&gt;&#xA;" OwnerUserId="16300" LastEditorUserId="16300" LastEditDate="2018-06-16T10:26:21.780" LastActivityDate="2018-08-03T11:43:44.880" Title="How does Microsoft use AI to make Windows 10 updates smoother" Tags="&lt;machine-learning&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6774" PostTypeId="2" ParentId="6766" CreationDate="2018-06-16T10:45:17.293" Score="2" Body="&lt;p&gt;I would advise you to look into Mozilla’s implementation of Baidu DeepSpeech &lt;a href=&quot;https://github.com/mozilla/DeepSpeech&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15465" LastEditorUserId="15465" LastEditDate="2018-06-16T11:02:42.127" LastActivityDate="2018-06-16T11:02:42.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6775" PostTypeId="2" ParentId="6728" CreationDate="2018-06-16T11:10:40.890" Score="0" Body="&lt;p&gt;In Computer Vision, Convolutional Neural Networks are basically an implementation of how neurons in the brain work.&#xA;Even when making your models architecture it’s common to put in dropout layers which mimic the way your brain uses different pathways of learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are architectures and techniques like You Only Look Once(YOLO) and one-shot learning, all this trying to replicate how the brain can instantly recognize something it has had little samples of or seen only once.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-06-16T11:10:40.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6776" PostTypeId="1" CreationDate="2018-06-16T21:03:09.043" Score="5" ViewCount="354" Body="&lt;p&gt;I understand how Neural Networks work and have studied its theory well. My question is at the intricacies of Deep Neural networks and perhaps is a bit beyond common understanding (as I have been told (or misled) from discussions).&#xA;My question is: On the whole, is there a clear understanding of how mutation occurs within a neural network from the input layer to the output layer, for both supervised and unsupervised cases? &#xA;Any neural network is a set of neurons + the connection weights. With each successive layer, there is a change in the input. Say I have a Neural Network that does movie recommendation on n parameters. &#xA;Say if X is a parameter that stands for the movie rating on IMDB. In each successive stage, there is a mutation of input X to X' and further X'' and so on. &#xA;While of course, we know how to mathematically talk about X' and X'', do we at all have a conceptual understanding as to what this variable is in its corresponding neural n-dimension? The neural weights which to the human eye might be set of random numbers but may mean something profound if we could ever understand what the neural weights 'represent'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JFcKa.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JFcKa.jpg&quot; alt=&quot;neural net&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the nature of neural weights such that despite decades worth of research and use, there is no clear understanding of what these connection weights represent? Or rather, why has there been so little effort in understanding the nature of neural weights, in a non-mathematical sense given the huge impetus in going beyond the black box notion of AI.  &lt;/p&gt;&#xA;" OwnerUserId="16308" LastEditorUserId="1671" LastEditDate="2018-06-18T17:23:11.290" LastActivityDate="2018-06-20T10:48:37.020" Title="What do neural connection weights represent 'conceptually'?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;artificial-neuron&gt;&lt;concepts&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6777" PostTypeId="2" ParentId="4581" CreationDate="2018-06-17T02:39:11.737" Score="2" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;Representation of states&lt;/em&gt;&lt;/strong&gt; is very important to prepare the data for neural network. You can try different way and pick which fit best in your case.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can use 18 neurons as input where each state is represented by the 2 bits. But avoid 0 and 1 if you are using sigmoid activation function, which can cause saturation at the output, which means if output(y) become 1 at any layer, then on back propagating error, we have &lt;strong&gt;y (1-y) dE/dy&lt;/strong&gt; in weight update part, which become zero with the saturation, which means it will stay in the same state ever.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This problem can be solved by following method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution 1.&lt;/strong&gt; You can initialize the input with some margin from 0 and 1. &lt;strong&gt;For example&lt;/strong&gt; input can be [0.1, 0.9] instead for [0, 1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution 2.&lt;/strong&gt; Another you can initialize weights very small in the range of [-0.01,  0.01].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution 3.&lt;/strong&gt; You can use regularization technique, whose purpose is to supress the weights by adding a penalizing term in error.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To handle variance problem, you can augment some data, for proper training. Because, in tic-tac-toe, you have small data set. To augment data, you can add some margin of range -0.1 to +0.1 in inputs with the same outputs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I hope this may be helpful.&lt;/p&gt;&#xA;" OwnerUserId="16313" LastActivityDate="2018-06-17T02:39:11.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6778" PostTypeId="1" AcceptedAnswerId="6820" CreationDate="2018-06-17T09:33:25.380" Score="3" ViewCount="372" Body="&lt;p&gt;Look at Breakout:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0tNtP.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0tNtP.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that the underlying world behaves like an MDP, because for the evolution of the system it just need to know which is the current state, i.e. position, speed and speed direction of the ball, positions of the bricks and the paddle etc. But considering only single frames as the state space we have a POMDP because we lack informations about the dynamics [1], [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:&#xA;What could happen if we wrongly assume that the POMDP is a MDP and do reinforcement learning with this assumption over the MDP?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously the question is more general, not limited to Breakout and Atari games.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[1] &lt;a href=&quot;https://arxiv.org/pdf/1507.06527.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1507.06527.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[2] &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15517" LastEditorUserId="15517" LastEditDate="2018-06-19T19:50:16.917" LastActivityDate="2018-06-20T10:48:14.097" Title="Reinforcement Learning over an MDP that is actually a POMDP" Tags="&lt;reinforcement-learning&gt;&lt;theory&gt;&lt;probability&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6781" PostTypeId="1" AcceptedAnswerId="6787" CreationDate="2018-06-18T07:43:06.257" Score="0" ViewCount="168" Body="&lt;p&gt;Oxford philosopher and leading AI thinker Nick Bostrom defines &lt;em&gt;SuperIntelligence&lt;/em&gt; as &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;An intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://tcrn.ch/2gJR03q&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is an interesting article, you may like &lt;em&gt;Tech Crunch&lt;/em&gt;..  &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial general intelligence : Wiki&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking into account current limitations and the amount of progress that has been made in recent years, what is a realistic timeframe to expect an AI that has human levels of cognition? &lt;/p&gt;&#xA;" OwnerUserId="16322" LastEditorUserId="16322" LastEditDate="2018-06-21T08:56:26.767" LastActivityDate="2018-06-21T08:56:26.767" Title="Is Really &quot;AI&quot; Light Years Away from achieving Cognitive Ability of Human?" Tags="&lt;agi&gt;&lt;human-like&gt;&lt;superintelligence&gt;&lt;mythology-of-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6782" PostTypeId="2" ParentId="6102" CreationDate="2018-06-18T08:05:58.947" Score="1" Body="&lt;p&gt;What you mention there is the &lt;strong&gt;perfect example&lt;/strong&gt; for &lt;strong&gt;path-planning&lt;/strong&gt;, which is extensively researched in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please look for &lt;strong&gt;A-star algorithm&lt;/strong&gt; and how to enhance it with neural networks :)&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-06-18T08:05:58.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6783" PostTypeId="1" AcceptedAnswerId="6785" CreationDate="2018-06-18T09:49:06.427" Score="1" ViewCount="80" Body="&lt;p&gt;I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The central service dispatches some information at regular intervals to all the agents (Lets call this information as &lt;strong&gt;energy&lt;/strong&gt;). The information can be very different for all the agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agents on reception of this information select a particular action. The execution of the action should leave the agent as well as the environment in a positive state. The action requires a limited amount of energy which might change on every timestep. If a agent does not have sufficient energy to it may request for energy from other agents. The other agents may grant or deny this request.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all the agents are able to successfully perform their actions and leave the environment in a positive state they get a positive reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the environment is stochastic, where a agent's behavior is dependent on another agent can approximate Q Learning be used here?&lt;/p&gt;&#xA;" OwnerUserId="11584" LastActivityDate="2018-06-18T14:59:10.010" Title="Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6784" PostTypeId="1" CreationDate="2018-06-18T11:01:14.053" Score="4" ViewCount="165" Body="&lt;p&gt;I have been working with AI methods. I am thinking about how my daughter (and also other kids) could learn mathematics with the help of AI. For example, how could an AI be used to show the mistakes that a kid does during the learning path?&lt;/p&gt;&#xA;" OwnerUserId="15506" LastEditorUserId="2444" LastEditDate="2020-06-16T20:14:58.300" LastActivityDate="2020-06-16T20:14:58.300" Title="How could an AI be used to improve the teaching and learning of mathematics?" Tags="&lt;math&gt;&lt;applications&gt;&lt;education&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6785" PostTypeId="2" ParentId="6783" CreationDate="2018-06-18T14:59:10.010" Score="2" Body="&lt;p&gt;Not particularly sure what you are asking, so the question that I will be answering is this: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can Q learning be used to estimate a value that depends on another value in the Q Learning Matrix even though there is a certain amount of unpredictability involved?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The answer is &lt;strong&gt;yes!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will use the example of a robotic arm trying to reach a point in space since that is what I am most familiar with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine a robotic arm with a shoulder, elbow, and wrist joint. The desired elbow value depends very much on the shoulder value which is also being learned. Given enough iterations, the Q learning algorithm will come up with a solution (out of possibly many) for the elbow joint based on where the shoulder joint is at that time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intrinsic unpredictability (air drag, motor power, etc in this case) is countered intuitively by the Q-learning algorithm iteratively learning what works best. &lt;/p&gt;&#xA;" OwnerUserId="6861" LastActivityDate="2018-06-18T14:59:10.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6787" PostTypeId="2" ParentId="6781" CreationDate="2018-06-18T16:41:25.790" Score="2" Body="&lt;p&gt;If certain philosophers are correct, Artificial General Intelligence will be, like fusion power, &lt;em&gt;&quot;always twenty years away&quot;&lt;/em&gt;.  For the true believers, it is an inevitability, and opinions vary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be most useful to look at the unreliability human predictions in this area.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was an article in the MIT Tech Review in 2017 that contained this graphic, based on a survey of researchers in 2015:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yRlWv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yRlWv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://www.technologyreview.com/s/607970/experts-predict-when-artificial-intelligence-will-exceed-human-performance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Experts Predict When Artificial Intelligence Will Exceed Human Performance&lt;/a&gt; (MIT Tech Review)&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most notable is that AlphaGo soundly bested the top human player in Go in March of 2016, years before even the most optimistic expert projections.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Everyone is just guessing, and we still don't if AGI is possible, or merely a myth.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, recent breakthroughs are promising!&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-06-18T16:41:25.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6788" PostTypeId="2" ParentId="1930" CreationDate="2018-06-18T17:35:46.463" Score="0" Body="&lt;p&gt;The distinction between algorithms/robots and humans is that, when the human organism stops functioning, the human is considered dead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By contrast, an algorithm still exists, even when not running.  &lt;em&gt;(I was going to use &quot;even when not being executed&quot;, but avoided this for semantic reasons;)&lt;/em&gt;  The algorithm can remain in this &quot;stasis state&quot; so long as there is a storage medium for the information.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Killing an algorithm is easy--delete and empty the trash bin.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Essentially, to kill an algorithm, you need to erase the code that comprises it.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-18T18:36:07.897" LastActivityDate="2018-06-18T18:36:07.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6789" PostTypeId="1" AcceptedAnswerId="6794" CreationDate="2018-06-18T19:32:27.573" Score="4" ViewCount="662" Body="&lt;h2&gt;Premise&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Ok, I know that this question was asked before on &lt;a href=&quot;https://ai.stackexchange.com/questions/4320/why-initial-weights-in-neural-network-are-randomised&quot;&gt;ai.SE&lt;/a&gt;, on &lt;a href=&quot;https://stats.stackexchange.com/questions/45087/why-doesnt-backpropagation-work-when-you-initialize-the-weights-the-same-value&quot;&gt;stats.SE&lt;/a&gt; and also on &lt;a href=&quot;https://stackoverflow.com/questions/20027598/why-should-weights-of-neural-networks-be-initialized-to-random-numbers&quot;&gt;SO&lt;/a&gt;. So I did my homework in checking before posting my question, but none of them has an answer that fully satisfies me.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Summary of my knowledge up to now&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Suppose you have a layer that is fully connected, and that each neuron performs an operation like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = g(w^T * x + b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;were &lt;code&gt;a&lt;/code&gt; is the output of the neuron, &lt;code&gt;x&lt;/code&gt; the input, &lt;code&gt;g&lt;/code&gt; our generic activation function, and finally &lt;code&gt;w&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; our parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both &lt;code&gt;w&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; are initialized with all elements equal to each other, then &lt;code&gt;a&lt;/code&gt; is equal for each unit of that layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that we have symmetry, thus at each iteration of whichever algorithm we choose to update our parameters, they will update in the same way, thus there is no need for multiple units since they all behave as a single one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to break the symmetry we could randomly initialize the matrix &lt;code&gt;w&lt;/code&gt; and initialize &lt;code&gt;b&lt;/code&gt; to zero (this is the setup that I've seen more often). &#xA;This way &lt;code&gt;a&lt;/code&gt; is different for each unit so that all neurons behave differently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course randomly initializing both &lt;code&gt;w&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; would be also okay even if not necessary.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The actual question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is randomly initializing &lt;code&gt;w&lt;/code&gt; the only choice? &#xA;Could we randomly initialize &lt;code&gt;b&lt;/code&gt; instead of &lt;code&gt;w&lt;/code&gt; in order to break the symmetry?&#xA;Is the answer dependent on the choice of the activation function and/or the cost function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thinking is that we could break the symmetry by randomly initializing &lt;code&gt;b&lt;/code&gt;, since in this way &lt;code&gt;a&lt;/code&gt; would be different for each unit and, since in the backward propagation the derivatives of both &lt;code&gt;w&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; depend on &lt;code&gt;a&lt;/code&gt;(at least this should be true for all the activation functions that I have seen so far), each unit would behave differently. Obviously, this is only a thought, and I'm not sure that is absolutely true.  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Remark&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Note that this is a theoretical question, not a practical one. &#xA;This means that I'm not really interested in answers that involves performance, but you're welcome to include them as corollary to the real answer.&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="2193" LastEditDate="2018-06-19T08:35:33.257" LastActivityDate="2018-06-19T08:35:33.257" Title="Why should weights of Neural Networks be initialized to random numbers?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;concepts&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6791" PostTypeId="2" ParentId="6776" CreationDate="2018-06-18T21:17:20.867" Score="2" Body="&lt;p&gt;I don't know if my intuition is correct but I will give it a try. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could see weights as how much important one thing is, the problem is to understand what that thing represents. When I say &lt;em&gt;thing&lt;/em&gt; I'm referring to the output of a specific neuron. I don't think that we can say what the output of a neuron represents in the real world unless we directly relate it through an error function or if the function used to compute that particular value have some meaning in the real world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want, you could actually build your neural network such that its neurons represent something. It's also very simple. you have only to write down all the equations relative to that particular topic. You could put them in a big system or, and this is better, you could put them in several systems such that the outputs of system 1 are the input of system 2 and so on. You could convert each system into a layer where each neuron represents an equation. Note that in this case, you would have the classical neuron with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;z = dot(w.T,x) + b&#xA;a = g(x)&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but a more complex equation for &lt;code&gt;z&lt;/code&gt; (but still based on weights) and a linear activation function for &lt;code&gt;a&lt;/code&gt;. In this case, you could name each neuron and say what they represent in the real world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this isn't the purpose of a neural network. A neural network should have neurons with simple equations to be fast thus the linear interpolating function &lt;code&gt;dot(w.T,x) + b&lt;/code&gt; is the best choice (the fact that the activation function is almost always non linear and in some cases a non-banal function is due to other thing and could be an interesting question). A neural network should also be as general as possible because usually is build upon a system that you don't know completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I modify slightly my answer: is not simply that you don't know what a neuron represent, excluding the ones of the output layer, you don't want that they have a meaning in the real world.&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="16199" LastEditDate="2018-06-20T10:48:37.020" LastActivityDate="2018-06-20T10:48:37.020" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6792" PostTypeId="2" ParentId="6789" CreationDate="2018-06-18T23:28:47.237" Score="0" Body="&lt;p&gt;w should be randomized to small (nonzero) numbers so that the adjustments made by the backpropagation are more meaningful and each value in the matrix is updated a different amount. If you start with all zeros it will still work, but take longer to get to a meaningful result. AFAIK, this was found empirically by various researchers and became common practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;randomizing b does not have the same effect of helping, therefore most people do not bother. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This choice is one of many that is made by the architect of the network and theoretically you could use an infinite number of w matrix initializations. The one commonly used just happens to be tested and generally works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This video is better at explaining than I am: &#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=iPNN805konI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=iPNN805konI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6861" LastActivityDate="2018-06-18T23:28:47.237" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6793" PostTypeId="2" ParentId="6789" CreationDate="2018-06-19T05:54:03.153" Score="2" Body="&lt;p&gt;Most of the explanations given for choosing something or not choosing something (like hyper-parameter tuning) in deep learning, are based on empirical studies like analysing the error over a number of iterations. So the answer by @Joe S is what people in deep learning side give.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you have asked for a mathematical explanation, I suggest you to read this &lt;a href=&quot;http://papers.nips.cc/paper/6662-convergence-analysis-of-two-layer-neural-networks-with-relu-activation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; published recently in NIPS. It talks about the convergence of SGD to global minima subject to weight initialisation being Gaussian using ReLU as activation function. The paper considers a neural net with no hidden layer, just input and output layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The very fact that analysis on such 'simple' network gets published in a very reputed and top conference itself suggests that the explanation you are seeking is not very easy and very few people work on the theoretical aspects of neural nets. IMHO after some years as the research progresses, I might be able to edit this answer and give the necessary explanation that you sought for. Till then this is the best I could do. Cheers!!!&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-06-19T05:54:03.153" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6794" PostTypeId="2" ParentId="6789" CreationDate="2018-06-19T07:25:48.457" Score="5" Body="&lt;p&gt;Randomising just &lt;code&gt;b&lt;/code&gt; sort of works, but setting &lt;code&gt;w&lt;/code&gt; to all zero causes severe problems with &lt;a href=&quot;https://en.wikipedia.org/wiki/Vanishing_gradient_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;vanishing gradients&lt;/a&gt;, especially at the start of learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using backpropagation, the gradient at the outputs of a layer &lt;code&gt;L&lt;/code&gt; involves a sum multiplying the gradient of the inputs to layer &lt;code&gt;L+1&lt;/code&gt; by the weights (and not the biases) between the layers. This will be zero if the weights are all zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A gradient of zero at &lt;code&gt;L&lt;/code&gt;'s output will further cause all earlier layers(&lt;code&gt;L-1&lt;/code&gt;, &lt;code&gt;L-2&lt;/code&gt; etc all the way back to layer &lt;code&gt;1&lt;/code&gt;) to receive zero gradients, and thus not update either weights or bias at the update step. So the first time you run an update, it will only affect the last layer. Then the next time, it will affect the two layers closest to the output (but only marginally at the penultimate layer) and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related issue is that with weights all zero, or all the same, maps all inputs, no matter how they vary, onto the same output. This also can adversely affect the gradient signal that you are using to drive learning - for a balanced data set you have a good chance of starting learning close to a local minimum in the cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For deep networks especially, to fight vanishing (or exploding) gradients, you should initialise weights from a distribution that has an expected magnitude (after multiplying the inputs) and gradient magnitude that neither vanishes nor explodes. Analysis of values that work best in deep networks is how &lt;a href=&quot;https://prateekvjoshi.com/2016/03/29/understanding-xavier-initialization-in-deep-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xavier/Glorot initialisation&lt;/a&gt; were discovered. Without careful initialisation along these lines, deep networks take much longer to learn, or in worst cases never recover from a poor start and fail to learn effectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Potentially to avoid these problems you could try to find a good non-zero fixed value for weights, as an alternative to Xavier initialisation, along with a good magnitude/distribution for bias initialisation. These would both vary according to size of the layer and possibly by the activation function. However, I would suspect this could suffer from other issues such sampling bias issues - there are &lt;em&gt;more&lt;/em&gt; weights, therefore you get a better fit to desired aggregate behaviour when setting all the weight values randomly than you would for setting biases randomly.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-19T08:31:46.590" LastActivityDate="2018-06-19T08:31:46.590" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="6795" PostTypeId="2" ParentId="4216" CreationDate="2018-06-19T08:30:42.697" Score="0" Body="&lt;p&gt;The meaning of strong AI has changed as you correctly indicated, and the term &quot;strong narrow AI&quot; is more appropriate as now people are shifting towards the practical uses.&lt;br&gt;&#xA;This also bring to light the number of breakthroughs that AI is having and literature and other various related resources will often(unlike in other fields) need to be changed constantly.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-06-19T08:30:42.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6799" PostTypeId="1" CreationDate="2018-06-19T11:45:29.377" Score="1" ViewCount="1939" Body="&lt;p&gt;So I built a CNN without any scientific libraries like &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; or &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras&lt;/a&gt; (only &lt;a href=&quot;http://www.numpy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NumPy&lt;/a&gt;). It is taking a huge amount of time to train. What are some of the tricks and tips followed by people to speed up training of a CNN? (I am not talking about division of jobs into different processors but subtle redundant codes i.e. giving pre-calculated results which is not visible to common programmers).&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2193" LastEditDate="2018-06-19T15:30:42.497" LastActivityDate="2018-06-20T07:59:45.700" Title="Speeding up CNN training" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6800" PostTypeId="1" CreationDate="2018-06-19T11:53:14.547" Score="30" ViewCount="5281" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1511.07528.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Limitations of Deep Learning in Adversarial Settings&lt;/em&gt;&lt;/a&gt; explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. The authors experiment with a neural network meant to read handwritten digits, undermining its reading ability by distorting the samples of handwritten digits that the neural network is trained with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm concerned that malicious actors might try hacking AI. For example&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fooling autonomous vehicles to misinterpret stop signs vs. speed limit.&lt;/li&gt;&#xA;&lt;li&gt;Bypassing facial recognition, such as the ones for ATM.&lt;/li&gt;&#xA;&lt;li&gt;Bypassing spam filters.&lt;/li&gt;&#xA;&lt;li&gt;Fooling sentiment analysis of movie reviews, hotels, etc.&lt;/li&gt;&#xA;&lt;li&gt;Bypassing anomaly detection engines.&lt;/li&gt;&#xA;&lt;li&gt;Faking voice commands.&lt;/li&gt;&#xA;&lt;li&gt;Misclassifying machine learning based-medical predictions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What adversarial effect could disrupt the world? How we can prevent it?&lt;/p&gt;&#xA;" OwnerUserId="16322" LastEditorUserId="2444" LastEditDate="2019-10-11T22:32:01.550" LastActivityDate="2020-11-05T11:00:02.263" Title="Is artificial intelligence vulnerable to hacking?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;security&gt;&lt;ai-safety&gt;&lt;adversarial-ml&gt;" AnswerCount="8" CommentCount="3" FavoriteCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="6801" PostTypeId="2" ParentId="6799" CreationDate="2018-06-19T13:21:21.320" Score="1" Body="&lt;p&gt;[Ref-some standard checks performed by programmers]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://arxiv.org/pdf/1405.3866.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speeding up Convolutional Neural Networks with Low Rank Expansions&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition, showing a possible 2.5x speedup with no loss in accuracy, and 4.5x speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16322" LastEditorUserId="16322" LastEditDate="2018-06-19T13:28:46.160" LastActivityDate="2018-06-19T13:28:46.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6802" PostTypeId="2" ParentId="6800" CreationDate="2018-06-19T13:41:18.600" Score="4" Body="&lt;p&gt;I believe it is, no system is safe, however I am not sure if I can still say this after 20-30 years of AI development/evolution. Anyways, there are articles that showed humans fooling AI (Computer Vision).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.theverge.com/2018/1/3/16844842/ai-computer-vision-trick-adversarial-patches-google&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.theverge.com/2018/1/3/16844842/ai-computer-vision-trick-adversarial-patches-google&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://spectrum.ieee.org/cars-that-think/transportation/sensors/slight-street-sign-modifications-can-fool-machine-learning-algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spectrum.ieee.org/cars-that-think/transportation/sensors/slight-street-sign-modifications-can-fool-machine-learning-algorithms&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16351" LastActivityDate="2018-06-19T13:41:18.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6803" PostTypeId="1" CreationDate="2018-06-19T13:53:09.327" Score="2" ViewCount="44" Body="&lt;p&gt;I have this table, 2 agents and I want to find for each agent if any action is strongly or weakly dominated. This is the table:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ax7b0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ax7b0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, i've found a solution but I'm not sure if it's correct. So for let's say agent 1(the one handling rows): 1&amp;lt;2&amp;lt;2, 1&amp;lt;3&amp;lt;4 and 0&amp;lt;3&amp;lt;4 so I don't have a strong dominance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For agent 2: 1&amp;lt;=1&amp;lt;=1, 2&amp;lt;5&amp;lt;6 and 2&amp;lt;3&amp;lt;4 which means that I don't have strong dominance here too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my logic correct?&lt;/p&gt;&#xA;" OwnerUserId="16352" LastEditorUserId="16352" LastEditDate="2018-06-19T14:03:37.847" LastActivityDate="2018-06-19T14:03:37.847" Title="Strong and Weak Dominance Table" Tags="&lt;game-theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6804" PostTypeId="2" ParentId="6800" CreationDate="2018-06-19T14:10:41.863" Score="1" Body="&lt;p&gt;I concur with Akio that no system is completely safe, but the take away is AI systems are less prone to attacks when comparing with the old systems because of the ability to constantly improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As time passes by more people will get in the field bringing new ideas and hardware will be improving so that they are &quot;strong AI.&quot;&lt;/p&gt;&#xA;" OwnerUserId="15465" LastEditorUserId="75" LastEditDate="2018-06-19T18:41:06.783" LastActivityDate="2018-06-19T18:41:06.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6805" PostTypeId="2" ParentId="6800" CreationDate="2018-06-19T14:48:33.327" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How we can prevent it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are several works about AI verification. Automatic verifiers can prove the robustness properties of neural networks. It means that if the input X of the NN is perturbed not more that on a given limit ε (in some metric, e.g. L2), then the NN gives the same answer on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such verifiers are done by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stanford: &lt;a href=&quot;https://arxiv.org/pdf/1702.01135.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1702.01135.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;ETHZ: &lt;a href=&quot;https://www.sri.inf.ethz.ch/papers/sp2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sri.inf.ethz.ch/papers/sp2018.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Google: &lt;a href=&quot;https://arxiv.org/pdf/1803.06567.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1803.06567.pdf&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1805.10265.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1805.10265.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Bosch: &lt;a href=&quot;https://arxiv.org/pdf/1805.10265.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1805.10265.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This approach may help to check robustness properties of neural networks. The next step is to construct such a neural network, that has required robustness. Some of above papers contain also methods of how to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different techniques to improve the robustness of neural networks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;adversarial training (see e.g. &lt;a href=&quot;https://arxiv.org/pdf/1611.01236.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Kurakin et al., ICLR 2017&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;defensive distillation (see e.g. &lt;a href=&quot;https://arxiv.org/pdf/1511.04508.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;N. Papernot et al., SSP 2016&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MMSTV defence (&lt;a href=&quot;https://arxiv.org/pdf/1706.06083&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maudry et al., ICLR 2018&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;At least the last one can provably make NN more robust. More literature can be found &lt;a href=&quot;https://github.com/TrustAI/Literature-on-DNN-Verification-and-Testing/blob/master/defences.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16354" LastEditorUserId="16354" LastEditDate="2018-06-21T17:31:16.403" LastActivityDate="2018-06-21T17:31:16.403" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6806" PostTypeId="2" ParentId="6800" CreationDate="2018-06-19T16:35:51.970" Score="7" Body="&lt;h2&gt;Programmer vs Programmer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;It's a &quot;infinity war&quot;: Programmers vs Programmers. All thing can be hackable. Prevention is linked to the level of knowledge of the professional in charge of security and programmers in application security.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;eg&lt;/strong&gt; There are several ways to identify a user trying to mess up the metrics generated by Sentiment Analysis, but there are ways to circumvent those steps as well. It's a pretty boring fight.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Agent vs Agent&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;An interesting point that @DukeZhou raised is the evolution of this war, involving two artificial intelligence (agents). In that case, the battle is one of the most knowledgeable. Which is the best-trained model, you know?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, to achieve perfection in the issue of vulnerability, artificial intelligence or artificial super intelligence surpass the ability to circumvent the human. It is as if the knowledge of all hacks to this day already existed in the mind of this agent and he began to develop new ways of circumventing his own system and developing protection. Complex, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe it's hard to have an AI who thinks: &quot;Will the human going to use a photo instead of putting his face to be identified?&quot;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How we can prevent it&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Always having a human supervising the machine, and yet it will not be 100% effective. This disregarding the possibility that an agent can improve his own model alone.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So I think the scenario works this way: a programmer tries to circumvent the validations of an AI and the IA developer acquiring knowledge through logs and tests tries to build a smarter and safer model trying to reduce the chances of failure.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="7800" LastEditDate="2018-07-03T14:49:14.257" LastActivityDate="2018-07-03T14:49:14.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6808" PostTypeId="2" ParentId="6800" CreationDate="2018-06-19T17:13:58.507" Score="21" Body="&lt;p&gt;AI is vulnerable from two security perspectives the way I see it: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The classic method of exploiting outright programmatic errors to achieve some sort of code execution on the machine that is running the AI or to extract data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Trickery through the equivalent of AI optical illusions for the particular form of data that the system is designed to deal with.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first has to be mitigated in the same way as any other software. I'm uncertain if AI is any more vulnerable on this front than other software, I'd be inclined to think that the complexity maybe slightly heightens the risk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is probably best mitigated by both the careful refinement of the system as noted in some of the other answers, but also by making the system more context-sensitive; many adversarial techniques rely on the input being assessed in a vacuum. &lt;/p&gt;&#xA;" OwnerUserId="15114" LastActivityDate="2018-06-19T17:13:58.507" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6810" PostTypeId="1" AcceptedAnswerId="6848" CreationDate="2018-06-19T20:43:35.507" Score="3" ViewCount="40" Body="&lt;p&gt;How does the legal question about agents talking to humans via telephone connection work? Recently Google gave a talk about Duplex, where an agent makes a call to a human to schedule a hairdresser.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there are any regulations related to this type of scenario, if there are some limitations, if the human needs to know that he is talking to an AI.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="1671" LastEditDate="2018-06-19T20:48:36.503" LastActivityDate="2018-06-22T06:32:58.147" Title="Digital Rights and Agents talking to humans" Tags="&lt;legal&gt;&lt;digital-rights&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6812" PostTypeId="2" ParentId="6800" CreationDate="2018-06-19T23:57:07.290" Score="0" Body="&lt;p&gt;There are many ways to hack an AI. When I was kid I figured how to beat a chess computer. I always followed the same pattern, once you learn you can exploit it. The worlds best hacker is a 4 year old that wants something he will try different things until he establishes pattern in his parents. Anyway, Get an Ai to learn the patterns of a AI and given a given combination you can figure the outcome. There is also just plain flaws or back door in code either on purpose or by chance. There is also the possibility the AI will hack itself. It is called misbehaving, remember the small child again...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW simple way is to make AI always fails safe... something people forget.&lt;/p&gt;&#xA;" OwnerUserId="16367" LastActivityDate="2018-06-19T23:57:07.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6813" PostTypeId="2" ParentId="6800" CreationDate="2018-06-20T01:31:16.650" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is Artificial Intelligence Vulnerable to Hacking?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Invert your question for a moment and think:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What would make AI at less of a risk of hacking compared to any other&#xA;  kind of software?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, software is software and there will always be bugs and security issues. AIs are at risk to all the problems non-AI software is at risk to, being AI doesn't grant it some kind of immunity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for AI-specific tampering, AI is at risk to being fed false information. Unlike most programs, AI's functionality is determined by the data it consumes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a real world example, a few years ago Microsoft created an AI chatbot called Tay. It took the people of Twitter less than 24 hours to teach it to say &quot;We're going to build a wall, and mexico is going to pay for it&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZvUkp.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZvUkp.jpg&quot; alt=&quot;We&amp;#39;re going to build a wall, and mexico is going to pay for it&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Image taken from the Verge article linked below, I claim no credit for it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that's just the tip of the iceberg.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some articles about Tay:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.bbc.co.uk/news/technology-35890188&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBC&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Verge&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now imagine that wasn't a chat bot, imagine that was an important piece of AI from a future where AI are in charge of things like not killing the occupants of a car (i.e. a self-driving car) or not killing a patient on the operating table (i.e. some kind of medical assistance equipment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Granted, one would hope such AIs would be better secured against such threats, but supposing someone did find a way to feed such an AI masses of false information without being noticed (after all, the best hackers leave no trace), that genuinely could mean the difference between life and death.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the example of a self-driving car, imagine if false data could make the car think it needed to do an emergency stop when on a motorway. One of the applications for medical AI is life-or-death decisions in the ER, imagine if a hacker could tip the scales in favour of the wrong decision.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How we can prevent it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Ultimately the scale of the risk depends on how reliant humans become on AI. For example, if humans took the judgement of an AI and never questioned it, they'd be opening themselves up to all sorts of manipulation. However, if they use the AI's analysis as just one part of the puzzle, it would become easier to spot when an AI is wrong, be it through accidental or malicious means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a medical decision maker, don't just believe the AI, carry out physical tests and get some human opinions too. If two doctors disagree with the AI, throw out the AI's diagnosis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a car, one possibility is to have several redundant systems that must essentially 'vote' about what to do. If a car had multiple AIs on separate systems that must vote about which action to take, a hacker would have to take out more than just one AI to get control or cause a stalemate. Importantly, if the AIs ran on different systems, the same exploitation used on one couldn't be done on another, further increasing the hacker's workload.&lt;/p&gt;&#xA;" OwnerUserId="16369" LastActivityDate="2018-06-20T01:31:16.650" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6814" PostTypeId="1" AcceptedAnswerId="6815" CreationDate="2018-06-20T01:42:58.700" Score="1" ViewCount="176" Body="&lt;p&gt;I have a corpus, say an instruction manual. The text in this manual is grouped into chapters and each chapter is split up into sections. For example, Chapter 1/Section 1, Chapter 1/Section 2 and so on.&#xA;Assume the corpus has C chapters and each chapter has S sections. My goal is, given a sentence or question, to classify this sentence/question. In other words I want to compute three most probable chapters to which this sentence or question belongs to. &#xA;I tried MultinomialNB model using sklealrn, but it did not give me the desired result. I want to try another approach, for example using a Neural Network and compare it with the MultinomialNB model. I have Googled and found Doc2Vec but haven't tried yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a better or another possible approach so that I could try and compare? What is the standard approach to such kind of problem? &lt;/p&gt;&#xA;" OwnerUserId="16371" LastActivityDate="2018-06-20T09:56:16.090" Title="Question classification according to chapters" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6815" PostTypeId="2" ParentId="6814" CreationDate="2018-06-20T05:21:34.360" Score="1" Body="&lt;p&gt;You have got text which have words. Now each of the words have dependency with words that are occurring previously or/and with the ones occurring later. So to capture the &quot;context&quot; in which a particular word occurs is done by sequence models in deep learning. Sequence models that are quite used often are Recurrent Neural Networks (RNN), Gated Recurrent Units (GRU) and Long Short Term Memory (LSTM) listed in the order of their functionality and complexity. To know more about them visit Jonathan Hui's &lt;a href=&quot;https://jhui.github.io/2017/03/15/RNN-LSTM-GRU/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is a classification problem i.e. given a sentence the classifier should tell which chapter and section it belongs to. You can use one of the sequence models to &quot;encode&quot; the sentence as a vector (based on the context of words in it and meaning of sentence) and pass that vector to a fully-connected neural network that has a softmax layer at the end to tell which chapter and section the text belongs to. The number of classes you are having are &lt;code&gt;C * S&lt;/code&gt; (one for each chapter-section pair). From the output of softmax layer you can pick up the top 3 classes that got the highest probability for a sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doc2Vec converts a paragraph (can also be a sentence) into a vector representation. The Doc2Vec model has been trained over a huge corpus of text to be able to capture the context of a text and represent it as a high dimensional vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corpus of text that you have, is from a probability distribution is quite different from that used for training Doc2Vec model. Sequence models (mentioned above) can learn that probability distribution for your text corpus exclusively. In other words, you have a tailor-made representation for your text so that the context can be understood much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is why I suggested you to use sequence models. You can also compare the performance obtained by using pretrained Doc2Vec as text representation with that obtained by using one of the sequence models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To know about the effectiveness of RNN visit Andrej Karpathy's &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-06-20T05:21:34.360" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6817" PostTypeId="2" ParentId="6799" CreationDate="2018-06-20T07:59:45.700" Score="1" Body="&lt;p&gt;Recommendations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Try deleting some variables that will no longer be used during run-time&lt;/li&gt;&#xA;&lt;li&gt;Use more efficient data structure&lt;/li&gt;&#xA;&lt;li&gt;Get your hands on some optimized library for your hardware, e.g. if you are using Intel processors use the Intel distribution of python&lt;/li&gt;&#xA;&lt;li&gt;Pay careful attention to your data types and try to trim them as much as possible&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-06-20T07:59:45.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6818" PostTypeId="2" ParentId="6776" CreationDate="2018-06-20T09:47:14.427" Score="0" Body="&lt;p&gt;It's a bit of a challenge to answer your question, since you appear to be not really familiar with the basics. You're talking about mutations, and changes to the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;No. The input is a vector of data, which initializes the value of the input nodes. The first layer of weights is then used to calculate the values for the next layer of nodes. This next layer is not a &quot;mutation&quot; of the input layer; that suggest the second layer of nodes is similar but not exactly identical to the first layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality, it's very common that the second layer of nodes does not even have the same &lt;em&gt;shape&lt;/em&gt; as the first layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are even wondering if certain weights have a certain meaning. That's even easier to answer. We know these networks are quite robust. We can ignore a significant percentage of the weights, and the classifications will change only a little. This shows that no individual weight represents a specific aspect of the network. &lt;/p&gt;&#xA;" OwnerUserId="16378" LastActivityDate="2018-06-20T09:47:14.427" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6819" PostTypeId="2" ParentId="6814" CreationDate="2018-06-20T09:56:16.090" Score="1" Body="&lt;p&gt;The easiest way would probably be not to use machine learning at all. Create an inverted index of words (ie for each word occurrence record the chapter and section), and then use an information retrieval algorithm (eg &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-IDF&lt;/a&gt;) to find the section that best matches your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will be more efficient (no huge models to train) and more transparent (you can easily see why a particular section has been selected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional steps you can take to adjust performance is stemming/lemmatising of words, and adding synonym lists (eg in a car manual application you might want to treat &lt;em&gt;trunk&lt;/em&gt; and &lt;em&gt;boot&lt;/em&gt; as the same word).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used that approach recently in a commercial project, and it performed well.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-20T09:56:16.090" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6820" PostTypeId="2" ParentId="6778" CreationDate="2018-06-20T10:02:23.097" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Question: What could happen if we wrongly assume that the POMDP is a MDP and do reinforcement learning with this assumption over the MDP?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It depends on a few things. The theoretical basis of reinforcement learning needs the state descriptions to have the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; for guarantees of convergence to optimal or approximately optimal solutions. The Markov property is a requirement that the state defines 100% of the controllable variation of reward and next state (given the action) - the rest must be purely stochastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An MDP can be &quot;nearly Markov&quot;, and a lot of real-world physical systems are like that. For instance, &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pole-balancing&lt;/a&gt; and &lt;a href=&quot;https://gym.openai.com/envs/Acrobot-v1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;acrobot&lt;/a&gt; tasks can be implemented as physical systems using motors, wheels, joints etc. In those real systems, there are limits to accuracy of measurement of the state, and many hidden variables, such as variable temperature (affecting length of components), friction effects, air turbulence. Those hidden variables taken &lt;em&gt;strictly&lt;/em&gt; by formal definition would make the system a POMDP. However, their influence compared to the key state variables is low, and in some cases effectively random from the perspective of the agent. In practice RL works well in the real physical systems, despite state data being technically incomplete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Atari games using multiple frame images as states, there are varying degrees of to which those states are already non-Markovian. In general a computer game's state may include many features that are not displayed on the screen. Enemies may have health totals or other hidden state, there can be timers controlling appearance of hazards, and in a large number of games the screen only shows a relatively small window into the total play area. However, the Deep Mind DQN network did well on a variety of scrolling combat and platform games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One game where DQN did notably badly - no better than a default random player - was Montezuma's Revenge. Not only does that platform puzzler game have a large map to traverse, but it includes components where state on one screen affects results on another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is hard to make a general statement about where an MDP with missing useful state information would benefit from being treated as a POMDP more formally. Your question is essentially the same thing expressed in reverse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The true answer for any non-trivial environment would be to try an experiment. It is also possible to make some educated guesses. The basis for those guesses might be the question &lt;em&gt;&quot;If the agent could know hidden feature &lt;code&gt;x&lt;/code&gt; from the state, how different would expected reward and policy be?&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the breakout example using each single frame as a state representation, I would expect the following to hold:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Value estimates become much harder since seeing the ball next to a brick - compared to seeing a ball progressively get closer to a brick over 4 frames - gives much less confidence that it is about to hit that brick and score some points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It should still be possible for the agent to optimise play, as one working strategy is to position the &quot;bat&quot; under the ball at all times. This will mean less precise control over angle of bounces, so I would expect it to perform worse than the four-frame version. However, it should still be significantly better than a default random action agent. A key driver for this observation is that seeing the ball close to the bottom of the screen, and not close to the bat, would still be a good predictor of a low expected future reward (even averaged over chances of ball going up vs going down), hence the controller should act to prevent such states occurring.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-20T10:48:14.097" LastActivityDate="2018-06-20T10:48:14.097" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6821" PostTypeId="2" ParentId="6800" CreationDate="2018-06-20T13:40:17.377" Score="0" Body="&lt;p&gt;&lt;em&gt;Is artificial intelligence vulnerable to hacking?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;hint;&lt;/strong&gt; if you say that AI is vulnerable,then I disagree with you here by such statement.&#xA;Artificial intelligence is divided into three categories nor phases that we are supposed to go through ie.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;artificial narrow intelligence&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;artificial general intelligence&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;artificial super intelligence&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Therefore,according to your statement; &quot;I'm concerned that malicious actors might try hacking AI.....&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;given by the examples in your message body,we are at the level of artificial narrow intelligence,where by human hacker can twist his/malicious code to invade such applications,at this level.However,if we jump straight to the final level of Artificial Intelligence; then by all means;a human being can't invade nor hacker a super intelligent software program or a high tech super intelligent agent. For instance; a human hacker,does one thing at a time,there's nothing to stop an artificial intelligence dividing its focus and doing a lot of staff simultaneously,this is hard to second a guess a mind which works accurately like that&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for your information&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;do not be taken up by what the media says about AI generally,simply&#xA;  because;they don't know that the big thing will happen that is &lt;em&gt;new species that out&#xA;  compete humans&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;just imagine living in a new society that is high tech.&lt;/em&gt; &#xA;Check out the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/2016_Cyber_Grand_Challenge&quot; rel=&quot;nofollow noreferrer&quot;&gt;cyber grand challenge&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you missed that event,then am sorry.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-06-20T13:40:17.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6823" PostTypeId="1" AcceptedAnswerId="6891" CreationDate="2018-06-20T15:13:55.233" Score="2" ViewCount="618" Body="&lt;p&gt;I was trying to implement NEAT, but I got stuck at the speciating of my clients/genomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I got so far is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the distance function implemented,&lt;/li&gt;&#xA;&lt;li&gt;each genome can mutate nodes/connections,&lt;/li&gt;&#xA;&lt;li&gt;two genomes can give birth to a new genome.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I've read a few papers, but none explicitly explains in what order what step is done. What is the order of the genetic operations in NEAT?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that for each generation, all the similar genomes will be put together into one species. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have other questions related to NEAT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which neural networks are killed (or not) at each generation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Who is being mutated and at what point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that these are a lot of questions, but I would be very happy if someone could help me :)&lt;/p&gt;&#xA;" OwnerUserId="16353" LastEditorUserId="2444" LastEditDate="2019-10-13T01:29:34.090" LastActivityDate="2019-10-13T01:29:34.090" Title="What is the order of the genetic operations in NEAT?" Tags="&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6826" PostTypeId="1" CreationDate="2018-06-20T17:39:55.893" Score="3" ViewCount="118" Body="&lt;p&gt;As is done traditionally, I used k-fold cross validation to select and optimize the hyper parameters of my neural network classifier. When it was time to store the final model for future predictions, I discovered that using the weights from the previous k-fold cv iteration to seed the initial weights of the model in subsequent iteration, helps in improving the accuracy (seems obvious). I can use the model from the final iteration to perform future predictions on unseen data.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would this approach result in overfitting? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(Please note, I am using all available data in this process and I do not have any holdout data for validation.)&lt;/p&gt;&#xA;" OwnerUserId="16408" LastEditorUserId="1671" LastEditDate="2018-06-20T18:54:57.920" LastActivityDate="2018-06-21T10:35:38.743" Title="If use the weights from previous iteration of a k-fold cross validation to seed a neural network classifier would I be overfitting?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;&lt;training&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6827" PostTypeId="1" AcceptedAnswerId="6847" CreationDate="2018-06-20T19:03:06.063" Score="2" ViewCount="87" Body="&lt;p&gt;I am reading the Simon Haykin's cornerstone book, &quot;Neural Networks, A Comprehensive Foundation, Second Edition&quot; and I cannot understand a paragraph below: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The analysis of the dynamic behaviour of neural networks involving the&#xA;  application of feedback is unfortunately complicated by virute (or&#xA;  virtue I cannot get word appropriately) of the fact that the&#xA;  processing units used for the construction of the network are usually&#xA;  nonlinear. Further consideration of this issue is deferred to the&#xA;  latter part of the book.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Before the paragraph, the author analysis the affects of weight of synapsis to the neural network's stability. Roughly speaking, he says, if |w| &gt;= 1 the neural network become unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please explain the paragraph? Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="14862" LastEditorUserId="14862" LastEditDate="2018-06-21T08:07:10.297" LastActivityDate="2018-06-23T08:41:47.133" Title="The analysis of the dynamic behaviour of neural networks involving the application of feedback" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6828" PostTypeId="5" CreationDate="2018-06-20T19:08:08.673" Score="0" Body="&lt;p&gt;For more info, have a look at the paper &lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt; (2013), by V. Mnih et al. See also  &lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human-level control through Deep Reinforcement Learning&lt;/a&gt; (2015), by V. Mnih et al., and &lt;a href=&quot;https://arxiv.org/abs/1711.07478&quot; rel=&quot;nofollow noreferrer&quot;&gt;Implementing the Deep Q-Network&lt;/a&gt; (2017) by M. Roderick et al.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-25T15:55:33.087" LastActivityDate="2019-08-25T15:55:33.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6829" PostTypeId="4" CreationDate="2018-06-20T19:08:08.673" Score="0" Body="For questions related to the deep Q-network (DQN), which is a deep neural network (e.g. a convolutional neural network) trained with a variant of Q-learning. The expression was coined in the paper &quot;Playing Atari with Deep Reinforcement Learning&quot; (2013) by Google's DeepMind." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-25T15:55:33.087" LastActivityDate="2019-08-25T15:55:33.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6832" PostTypeId="1" CreationDate="2018-06-21T01:22:26.443" Score="2" ViewCount="69" Body="&lt;p&gt;Can anyone recommend a reinforcement learning algorithm for a multi-agent environment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my simplified example, I'm implementing a Q-Learning system with different 10 agents. The agents compete for resources in stores at different locations by setting a bid price for each item. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the agents have different bids and pooled budget of $100. Once the budget is reached the agents cannot buy any more that day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each agent will receive a reward if they buy an item. The goal would be to maximize the total amount of items bought between the agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now the agents don't communicate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me in the right direction for an algorithm that allows agent cooperation?&lt;/p&gt;&#xA;" OwnerUserId="16414" LastEditorUserId="2444" LastEditDate="2019-02-13T21:30:08.143" LastActivityDate="2019-02-13T21:30:08.143" Title="Algorithms for multiple agents problems" Tags="&lt;reinforcement-learning&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6835" PostTypeId="2" ParentId="6826" CreationDate="2018-06-21T10:35:38.743" Score="1" Body="&lt;p&gt;To answer directly your question: estimating a model's performance on data previously used for fitting will overestimate performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When your dataset is &quot;small&quot; you are faced with a bias-variance dilemma with regards to how much data should be used for train and test sets:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Too much train data, you end up with little test samples and your&#xA;performance estimate has high variance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Too much test data, your train samples do not represent well the&#xA;population you are trying to model/target, so your average&#xA;performance will be much lower than what could be achieved.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;K-fold CV is a compromise in the evaluation of performance of certain procedure. Once you have settled on a certain model or hyperparameters, when moving to production you can:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choose to take one of the k models you have trained, perhaps using the &quot;one std variation rule of thumb&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Re-train on all of your data and you can expect the resulting model&#xA;to be at least as good as you have estimated.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use all k models to form an ensemble and you can expect the resulting&#xA;model to be at least as good as you have estimated.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16322" LastActivityDate="2018-06-21T10:35:38.743" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6836" PostTypeId="1" AcceptedAnswerId="6862" CreationDate="2018-06-21T12:00:23.303" Score="4" ViewCount="237" Body="&lt;p&gt;Recently, I have been learning about new neural networks, which are used for specialised purposes, like speech recognition, image recognition, etc. The more I discover the more I get amazed by the cleverness behind models such as RNN's and CNN's. Questions about the working, intuition, mathematics have been asked a lot in this community, all with vague answers and apparent understandings.&lt;/p&gt;&#xA;&lt;p&gt;So, my question is: did the researchers come up with these specialised models accidentally or did they follow particular steps to get to the model (like in a mathematical framework)? And how did they look at a particular class of problem and think &amp;quot;Yeah, a better solution might exist&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;Since the understanding of NN's is so vague, these are 'high risk, high reward' scenarios, since you might be chasing only the mirage (illusion) of a solution.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2021-02-19T14:48:57.430" LastActivityDate="2021-02-19T14:48:57.430" Title="Are Neural Net architectures accidental discoveries?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;research&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6837" PostTypeId="2" ParentId="6836" CreationDate="2018-06-21T13:16:19.650" Score="1" Body="&lt;p&gt;Researchers may follow specific mathematical frameworks, techniques to come-up with amazing works just like in any field, but I believe in &lt;strong&gt;&lt;em&gt;Darwinian natural selection&lt;/em&gt;&lt;/strong&gt; as a base theory for human's discoveries as well as for the Evolutionary Neural Net Architectures.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;&quot;Principle by which each slight variation [of a trait], if useful, is preserved&quot;.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16322" LastActivityDate="2018-06-21T13:16:19.650" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6838" PostTypeId="2" ParentId="5369" CreationDate="2018-06-21T14:29:40.780" Score="2" Body="&lt;p&gt;Derivative gives the rate of change in &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; for a small change in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; or the slope of a function at point &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above function,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y = x      for x &amp;gt;= 0,     i.e. y/x = 1&#xA;y = x/20   for x &amp;lt; 0,      i.e.  y/x = 1/20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The following function returns the derivative of leaky ReLU as explained&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;private double leaky_relu_derivative(double x)&#xA;{&#xA;    if (x &amp;gt;= 0)&#xA;        return 1;&#xA;    else&#xA;        return 1.0 / 20;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16426" LastEditorUserId="2444" LastEditDate="2020-05-30T13:01:46.290" LastActivityDate="2020-05-30T13:01:46.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6840" PostTypeId="2" ParentId="6094" CreationDate="2018-06-21T19:07:45.250" Score="0" Body="&lt;p&gt;Maybe a simple regex solve this question. But problably you need supervise, approve and disapprove anomalies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this example, I had the same problem some time ago: &lt;a href=&quot;https://stackoverflow.com/questions/50689935/regex-like-commands-python&quot;&gt;https://stackoverflow.com/questions/50689935/regex-like-commands-python&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-06-21T19:07:45.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6842" PostTypeId="5" CreationDate="2018-06-21T19:52:08.307" Score="0" Body="&lt;p&gt;Have a look at the original research paper that introduced AlphaGo &lt;em&gt;&lt;a href=&quot;https://www.nature.com/articles/nature16961&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the game of Go with deep neural networks and tree search&lt;/a&gt;&lt;/em&gt; (2016), by David Silver et al., published in Nature. Have also a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AlphaGo&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-11T15:09:26.633" LastActivityDate="2019-08-11T15:09:26.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6843" PostTypeId="4" CreationDate="2018-06-21T19:52:08.307" Score="0" Body="For questions related to DeepMind's AlphaGo, which is the first computer Go program to beat a human professional Go player without handicaps on a full-sized 19x19 board. AlphaGo was introduced in the paper &quot;Mastering the game of Go with deep neural networks and tree search&quot; (2016) by David Silver et al. There have been three more powerful successors of AlphaGo: AlphaGo Master, AlphaGo Zero and AlphaZero." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-11T15:09:26.633" LastActivityDate="2019-08-11T15:09:26.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6844" PostTypeId="5" CreationDate="2018-06-21T20:02:32.963" Score="0" Body="&lt;p&gt;Have a look at the research paper that introduced AlphaGo Zero &lt;em&gt;&lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the game of Go without human knowledge&lt;/a&gt;&lt;/em&gt; (2017) by David Silver et al. and &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_Zero&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AlphaGo_Zero&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-11T15:13:41.283" LastActivityDate="2019-08-11T15:13:41.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6845" PostTypeId="4" CreationDate="2018-06-21T20:02:32.963" Score="0" Body="For questions related to AlphaGo Zero, which is a version of DeepMind's Go software, AlphaGo, that does not use data from human games and it is stronger than AlphaGo. There is a generalized version of AlphaGo Zero called AlphaZero, which beat the 3-day version of AlphaGo Zero by winning 60 games to 40. AlphaGo Zero was introduced in the paper &quot;Mastering the game of Go without human knowledge&quot; (2017) by David Silver et al." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-11T15:13:41.283" LastActivityDate="2019-08-11T15:13:41.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6846" PostTypeId="1" CreationDate="2018-06-22T01:51:14.303" Score="2" ViewCount="38" Body="&lt;p&gt;I'm eventually looking to build an algorithm that will process answers from humans that are given questions. But first I have to setup an experiment to determine the variety of responses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, humans will be asked a multiple choice question that has a single correct answer. I want to understand what kinds/ranges of responses I would get from the bell curve distribution of human intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way I can have, say, 1000 &quot;humans&quot; be asked a prompt, repeated 100 times (the same question) and then compile the responses? My concern is that I'll have to build some algorithm or process for each dumb, average, smart &quot;human&quot; to follow but then I would introduce bias in how smart they are or limit how they may respond. I'm guessing I'll have to give them a data sort to work from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify, it's not the number of times a single user gets a question right that makes them smart, they have to be programmed dumb, smart etc. before the simulation starts. So dumb users could get some right and smart can get some wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure the Monte Carlo method is useful here but some type of simulation where I can specify the distribution (normal) and then bound the responses would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have access to Excel, Minitab, and Python. Any ideas how to set up an experiment like this? I really am open to any technique to measure this.&lt;/p&gt;&#xA;" OwnerUserId="16435" LastActivityDate="2018-06-22T14:16:26.107" Title="How can I simulate responses from the distribution of human intelligence?" Tags="&lt;machine-learning&gt;&lt;human-like&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6847" PostTypeId="2" ParentId="6827" CreationDate="2018-06-22T05:58:08.037" Score="0" Body="&lt;p&gt;Although you have not given the entire context but if I were to speculate I would suggest the author is simply trying to tell that Neural Networks are quite difficult to analyse mathematically due to their non-linear nature (due to the use of non-linear activation). There are many questions on this Stack about the mathematical basis of NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks&quot;&gt;Do scientists know what is happening inside artificial neural networks?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also forward propagation is difficult enough, and now we that we have included feedback it becomes exponentially tough. You can easily understand the difficulty posed in such analysis if you take the case of &lt;a href=&quot;https://www.quora.com/What-is-a-feedback-amplifier-and-what-are-its-applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feedback Amplifiers&lt;/a&gt; in electronics. In normal sound amplifiers to avoid distortion we use a negative feedback. This roughly has an inverted waveform compared to i/p signal. It adds up with the input signal and smoothens out the distortions (as opposite wavefroms of not entirely same magnitude adds up). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;|W| &gt; 1 should be the case of &lt;a href=&quot;https://www.allaboutcircuits.com/textbook/semiconductors/chpt-8/positive-feedback/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Positive feedback&lt;/a&gt;, which can cause massive variations in output due to small changes due to its self reinforcing property. The wave-forms are of the same shape and they add up, this results in an even more larger waveform in the next cycle and so on. So probably the author is speaking something along these lines.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-06-23T08:41:47.133" LastActivityDate="2018-06-23T08:41:47.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6848" PostTypeId="2" ParentId="6810" CreationDate="2018-06-22T06:32:58.147" Score="4" Body="&lt;p&gt;Googling this throws a lot of debate on this issue of whether the call made to the restaurant for booking was legal or not. I found this &lt;a href=&quot;https://towardsdatascience.com/is-google-duplex-ethical-and-moral-f66a23637640&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; to put forward a lot of ideas for and against this. So it is for us to decide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But one thing that I fully agree with you, which should be made into a regulation, is human should know that he/she is talking to a bot. This tweet by &lt;a href=&quot;https://twitter.com/traviskorte/status/993954759932612608&quot; rel=&quot;nofollow noreferrer&quot;&gt;@traviskorte&lt;/a&gt; reasons it out why:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;We should make AI sound different from humans for the same reason we put a smelly additive in normally odorless natural gas.&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-06-22T06:32:58.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6849" PostTypeId="2" ParentId="4629" CreationDate="2018-06-22T07:06:53.663" Score="0" Body="&lt;p&gt;In order to train a neural network, you have to adjust each weights and biases to reduce the cost to as minimum as possible. The only way to do so is to subtract a small amount of the partial derivative of cost w.r.t. w, b from the respective parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if J is our cost function, after each iteration:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = w - lr*dJ_dw,      //where lr is a small scalar called learning rate and dJ_dw is the partial derivative of cost function w.r.t. w&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and same for bias&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b = b - lr*dJ_db,    //dJ_db is the partial derivative of cost function w.r.t b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;let's look at how the partial derivatives are calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using sigmoid as activation function, and squared error function as cost, we have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = w*x + b&#xA;a = sigmoid(z)       // sigmoid(z) = 1.0 / (1.0 + exp(-z)), a is the final output&#xA;&#xA;J = (a - y)*(a - y)  // where y is the expected output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For us to calculate partial derivatives of this cost function w.r.t. w, b, we need to use, chain rule of derivatives as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dJ_dw = dJ_da * da_dz * dz_dw    // dJ_da is the partial derivative of cost w.r.t. activation, a&#xA;dJ_db = dJ_da * da_dz * dz_db&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in the above equations, &lt;strong&gt;&lt;em&gt;da_dz&lt;/em&gt;&lt;/strong&gt; is the derivative of activation function (sigmoid in our case) which is &lt;strong&gt;&lt;em&gt;sigmoid(z).(1 - sigmoid(z))&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="16426" LastActivityDate="2018-06-22T07:06:53.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6850" PostTypeId="1" CreationDate="2018-06-22T07:24:19.283" Score="3" ViewCount="69" Body="&lt;p&gt;I thought I have implemented the code (from scratch, no library) for an &lt;strong&gt;artificial neural network&lt;/strong&gt; (first endeavour in the field). But I feel like I miss something very basic or obvious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make it short: code works for a single pair of in-/out-values but fails for sets of value pairs. I do not really understand the training process. So I want to get this issue out of the way first. The following is my improvised training (aka all that I can think of) in pseudocode.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;trainingData = [{in: [0,0], out:[0]}, {in: [0,1], out:[0]}, ...];&#xA;iterations = 10000&#xA;&#xA;network = graphNodesToNetwork()&#xA;links = graphLinksToNetwork()&#xA;randomiseLinkWeights(links)&#xA;&#xA;&#xA;while(trainingData not empty) {&#xA;  for(0&amp;lt;iterations) { &#xA;     set = trainingData.pop()&#xA;&#xA;     updateInput(network, set.in)&#xA;&#xA;     forwardPropagate(network, links) &#xA;&#xA;     linkUpdate = backPropagate(network, links, set.out)&#xA;&#xA;     updateLinks(linkUpdate, links)}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this how it is supposed to work? Do you feed in your training data set by set (while-loop)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 1: because my final comment did distract from the issue at hand.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 2: less wordy, more code-y&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="16441" LastEditorUserId="16441" LastEditDate="2018-06-22T12:01:32.943" LastActivityDate="2018-08-16T06:50:50.763" Title="How to actually teach the ANN the resulting weights of different training inputs?" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6851" PostTypeId="2" ParentId="6846" CreationDate="2018-06-22T11:29:00.933" Score="2" Body="&lt;p&gt;The data you want to collect cannot be reliably simulated at this time. There is no current realistic simulator for a human performing reading comprehension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual error rates and specific wrong answers chosen on the questions will depend to a large degree on specific humans, and the nature of questions. As you are hoping to get realistic results, the only method that will work for your ground truth data is to present 1000+ real humans with your sample questions. In addition, if you want to categorise your humans into &quot;smart&quot;, &quot;dumb&quot; etc, you will need to run additional tests on them, such as an IQ test, in order to create those categories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on context, such as the nature of the questions you want to assess, you may be able to obtain some anonymised data from real-world exams that could help, instead of trying to generate that data yourself. In that case you could make an approximate model for humans answering multiple-choice questions - perhaps by training an LSTM-based natural language model. For best accuracy on your questions you would want the training set to include similar kinds of question. There is still a caveat that NNs do not really do logic or reasoning, they make statistical fits, so can easily get answers logically wrong or select nonsense answers. The best &lt;em&gt;general&lt;/em&gt; NLP models still fail badly at semantics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have specific set of questions to evaluate - and are not willing to ignore the content of the questions - then no machine can currently match distribution of human behaviour on such a task without extensive training data and significant effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't care about the content of the questions, or assessing their difficulty, or indeed any feedback about the specific questions, then you could maybe use a dataset from any multi-choice questionnaire to get statistics of correct answer accuracy. With this approach your simulation could just be a simple distribution over &quot;correct&quot;, &quot;most plausible incorrect&quot; etc answers where you choose &quot;most plausible incorrect&quot; either manually or randomly (but consistently). This will get you a &lt;em&gt;distribution&lt;/em&gt; of responses similar to known real-world data. It could be used to unit-test a scoring system perhaps, or demonstrate some stats or visualisation software. But the question and answer text may as well be gibberish at that point.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-22T14:16:26.107" LastActivityDate="2018-06-22T14:16:26.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6855" PostTypeId="1" AcceptedAnswerId="6962" CreationDate="2018-06-22T17:13:34.897" Score="1" ViewCount="1770" Body="&lt;p&gt;I've been oogling the Mac Pro from Apple with loaded specs. &lt;a href=&quot;https://www.apple.com/mac-pro/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Check it here if unfamiliar.&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious to hear anyone's thoughts of the computer for deep learning/machine learning applications vs cloud based computing. Obviously, a flexible solution on GCP or AWS has the ability to scale and therefore produce results faster. However, &lt;strong&gt;I'm mostly interested to hear thoughts on the economics of purchasing the machine vs renting a cloud based machine.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we assumed something like a 5 year lifetime of the Apple Mac Pro for this type of computing (before the chips/processors become immensely slower than the newest chips used in the cloud), that would be like $1000 per year amortized using the Mac Pro. Does it make more sense to rent from AWS/GCP for \$1000 per year as a renter instead? I don't have much experience in the cost of AWS/GCP so I'm looking to here an answer from anyone well versed in cloud computing.&lt;/p&gt;&#xA;" OwnerUserId="14924" LastEditorUserId="1671" LastEditDate="2018-06-29T19:37:05.777" LastActivityDate="2019-04-22T19:57:05.173" Title="Thoughts on Apple Mac Pro vs GCP/AWS for Deep Learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hardware&gt;&lt;hardware-evaluation&gt;&lt;soft-question&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-05-13T21:37:19.147" ContentLicense="CC BY-SA 4.0" />
  <row Id="6857" PostTypeId="2" ParentId="4766" CreationDate="2018-06-22T19:47:32.280" Score="-1" Body="&lt;p&gt;There has been some recent work on this: &lt;a href=&quot;https://arxiv.org/pdf/1804.00538.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Investigating Capsule Networks with Dynamic Routing for Text Classification&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems some are having some success with it.&lt;/p&gt;&#xA;" OwnerUserId="16454" LastEditorUserId="2444" LastEditDate="2020-06-09T11:51:01.577" LastActivityDate="2020-06-09T11:51:01.577" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6859" PostTypeId="1" CreationDate="2018-06-22T20:18:39.120" Score="3" ViewCount="670" Body="&lt;p&gt;I'm new to machine learning, so i figured I should look into google's tensor flow guides and I know how to code in JS so that's why I'm using tensorflow.js, there's and example in the guide that trains itslef to recognize handwritten numbers from the  MNIST handwriting dataset, I sort of understand what's going on in the code but since I'm very new to ML it's not a lot, I went through the code and saw that it didn't took image by image to train itself but it requests one sprite which contains all the images and then cuts it into what it needs, this makes sense from a performance point of view, but as this process is kind of abstract I don't understand what's really going on, I want to upload an image of my own and call the predictor of the model but I don't know how to do it, any help? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking that drawing in a canvas of 28x28 a number might be very interesting as well instead of uploading an image, but I need to know how to test the model once it's trained with my own data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tutorial: &lt;a href=&quot;https://js.tensorflow.org/tutorials/mnist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://js.tensorflow.org/tutorials/mnist.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16078" LastActivityDate="2018-06-25T16:53:11.927" Title="How to load an image into tensorflow.js code which reads handwritten numbers and clasify them" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6860" PostTypeId="1" AcceptedAnswerId="6864" CreationDate="2018-06-23T06:52:10.587" Score="3" ViewCount="531" Body="&lt;p&gt;Why is it that the skewed contour (unscaled features) will result in slow performance of gradient descent? In other words, how (or why) will the gradients end up taking a long time before finding the global minimum in such cases? This might be an obvious question but I'm finding a hard time Visualizing the 3D shapes of the respective contours and relating it to the convergence.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6VjAu.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6VjAu.gif&quot; alt=&quot; &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Left one is the contour for the unscaled feature and the right one is scaled (and will apparently converge quickly).&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="10549" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-13T06:57:47.687" Title="Why Feature Scaling for skewed contour?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6862" PostTypeId="2" ParentId="6836" CreationDate="2018-06-23T07:59:52.060" Score="3" Body="&lt;p&gt;Although there is a strong element of &quot;try and see&quot; that has driven successful architectures, the drivers for what to try are often inspired by underlying theory or knowledge from other disciplines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically for basic CNN, which led to AlexNet and many of the best image processing, the concept of using local receptive fields in layers was inspired by &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/&quot; rel=&quot;nofollow noreferrer&quot;&gt;study of neurons in the cat visual system&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern RNNs also did not appear out of nowhere, there has long been an appreciation of the difference between a feed-forward network and a recurrently-connected one, and the different applications possible. The step change to LSTM was &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1246450&quot; rel=&quot;nofollow noreferrer&quot;&gt;deliberate response to analysis of problems training simplest forms of RNN&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like much of science, these things are also driven by success in the real world following the research. Many promising ideas have been tried and rejected. Some have been used for a while then superseded, e.g. using RBMs or stacked auto-encoders to pre-train deep networks before ReLUs and Xavier initialisation were discovered - although both RBMs and auto-encoders still have their niches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tweaks to architectures, such as variants of LSTM/GRU, may even be deliberately searched and assessed as part of research. That is done with the explicit knowledge that this part of finding a good design is best done as a search across possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the evolution-like progress, presenting all such advances as completely random or pure GA-like search is ignoring the conscious effort and research that leads to the designs. If you search literature on any major successful design (such as the existence of RNNs or CNNs in the first place), and read the papers, you will often find that modern neural network architectures have deep roots in older research, plus have mathematical and/or scientific justifications for the choices made. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-06-23T08:18:56.213" LastActivityDate="2018-06-23T08:18:56.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6863" PostTypeId="1" CreationDate="2018-06-23T08:25:13.167" Score="1" ViewCount="77" Body="&lt;p&gt;Say I have an application where the frequency of the input is known but can vary widely across sequences. For example, they may be audio recordings acquired at different frequency, or videos that come from surveillance cameras whose framerate can vary from 24fps down to as low as 1fps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The straightworard thing to do would be to either&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;resample inputs to a constant frequency&lt;/li&gt;&#xA;&lt;li&gt;ignore input frequency and hope the RNN will figure it all out&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;None sound very appealing. Is there a better way to handle variable input frequency in RNNs?&lt;/p&gt;&#xA;" OwnerUserId="16466" LastActivityDate="2018-06-23T08:25:13.167" Title="How to adapt RNNs to variable frequency / framerate of inputs?" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6864" PostTypeId="2" ParentId="6860" CreationDate="2018-06-23T08:34:21.817" Score="1" Body="&lt;p&gt;Feature scaling is making your features have an equal representation or say in the final loss of the function. Intuitively as your picture suggests the contour will be elongated along one axes (feature with higher values). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is lets say you have to separate two data clusters, one is  &lt;strong&gt;&lt;code&gt;(x,2)&lt;/code&gt;&lt;/strong&gt; and another is &lt;strong&gt;&lt;code&gt;(x,5)&lt;/code&gt;&lt;/strong&gt; where &lt;strong&gt;&lt;code&gt;x&lt;/code&gt;&lt;/strong&gt; is variable. Now if you are using sigmoid activation, sigmoid always gives positive values, but to separate the two classes you need to have negative value for &lt;strong&gt;&lt;code&gt;(x,2)&lt;/code&gt;&lt;/strong&gt; in the second last layer, thus the NN will for a longer time have to adjust the weights to be negative (assuming positive weight initialisation) of bias and other features until &lt;strong&gt;&lt;code&gt;(x,2)&lt;/code&gt;&lt;/strong&gt; gives negative value in the second last layer. But with feature normalisation the initial value would have been already negative something like &lt;strong&gt;&lt;code&gt;(x,-1/3)&lt;/code&gt;&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: From my answer on DataScience.SE:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Purpose:&lt;/strong&gt; Normalisation is done so that the Neural Network weights converge faster. In CNN's and Deep Neural Nets this is of particular help especially in CNN this helps to prevent exploding/vanishing gradients.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The most common explanation for normalisation I have come across is that if you have 2 features, one of them has a significant larger scale than the other e.g house price and house area then the feature with a larger scale will dominate the output. This is quite &lt;strong&gt;incorrect&lt;/strong&gt; according to me, since when you back-propagate through the Neural Net the weight updates are directly proportional to the activations, so larger activation means larger feedback and hence weights get reduced faster and become smaller until &lt;strong&gt;&lt;code&gt;w1*house price = w2*house area&lt;/code&gt;&lt;/strong&gt; approximately this relation holds true. Yes, it will lead to more oscillations (intuitionally, since learning rate also becomes multiplied with a larger scale) but it will ultimately probably converge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the best 3 reasons for using normalisation are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If scale of a feature is large the weights connected to that feature will have larger oscillations resulting in slower convergence and if deep NN is used probably no convergence, whereas normalisation helps in making the values small &lt;code&gt;-1 to 1&lt;/code&gt; so the gradient updates are also small resulting in faster convergence.&lt;/li&gt;&#xA;&lt;li&gt;The best intuition for normalisation can be found from this &lt;a href=&quot;https://www.youtube.com/watch?v=wEoyxE0GP2M&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=6&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; of Stanford and its subsequent &lt;a href=&quot;https://www.youtube.com/watch?v=_JB0AO7QxSA&amp;amp;index=7&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;. Since we know weight updates is directly proportional to inputs it will also take the sign of the inputs (or exact opposite sign) always. Now we know house price and area is always positive (in our Universe at least!). So the weight updates will always have definite sign either both positive or both negative (depending the sign of downstream gradients). But the weight updates may have a optimal value in the 4th quadrant, so the weight updates will have to follow a zig zag pattern to effectively make a weight update in the 4th quadrant.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/o9AxE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o9AxE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Finally when you are dealing with Deep Neural Nets like CNN if you do not normalise pixels it will result in exponentially large/vanishing gradients. Since generally &lt;strong&gt;&lt;code&gt;softmax/sigmoid&lt;/code&gt;&lt;/strong&gt; is used in the last layer, it squashes the outputs. If you have a large output, generally due to un-normalized data, it'll result in wither exact 0 or exact 1 output, which is fed into a &lt;strong&gt;&lt;code&gt;log&lt;/code&gt;&lt;/strong&gt; function and BAM! overflow. The error becomes &lt;code&gt;inf&lt;/code&gt; or &lt;code&gt;NaN&lt;/code&gt; in python. So &lt;code&gt;inf&lt;/code&gt; error means exploding gradients and &lt;code&gt;NaN&lt;/code&gt; means gradient cannot be calculated. This can probably be remedied by using higher floating point precision but it will result in higher memory and processor consumption ultimately inefficiency.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; Normalisation is used for faster weight convergence. Issues faced by un-normalised data are larger weight oscillations, weight updation in non optimal direction, overflow in precision in Deep Neural Nets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few great links for better insight:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/41704/how-and-why-do-normalization-and-feature-scaling-work&quot;&gt;How and why do normalization and feature scaling work?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/26225344/why-feature-scaling&quot;&gt;Why feature scaling?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-09-13T06:57:47.687" LastActivityDate="2018-09-13T06:57:47.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6868" PostTypeId="1" CreationDate="2018-06-23T17:25:43.103" Score="2" ViewCount="109" Body="&lt;p&gt;I've selected more than 10 discriminative (classification) models, each wrapped with a &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;BaggingClassifier&lt;/code&gt;&lt;/a&gt; object, optimized with a &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;GridSearchCV&lt;/code&gt;&lt;/a&gt;, and all of them placed within a &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.VotingClassifier.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;VotingClassifier&lt;/code&gt;&lt;/a&gt; object.&lt;/p&gt;&#xA;&lt;p&gt;Alone, they all bring around 70% accuracy, on a data set which is about half normal/uniform distributed, and half one-hot distributed. Together, they provide 80% accuracy, which isn't good enough, given that I was told that 95%&amp;lt; is achievable.&lt;/p&gt;&#xA;&lt;p&gt;The models: &lt;code&gt;DecisionTreeClassifier&lt;/code&gt;, &lt;code&gt;ExtraTreesClassifier&lt;/code&gt;, &lt;code&gt;KNeighborsClassifier&lt;/code&gt;, &lt;code&gt;GradientBoostingClassifier&lt;/code&gt;, &lt;code&gt;LogisticRegression&lt;/code&gt;, &lt;code&gt;SVC&lt;/code&gt;, &lt;code&gt;Perceptron&lt;/code&gt;, and a few more classifiers.&lt;/p&gt;&#xA;&lt;p&gt;How do I check if the combination is good?&lt;/p&gt;&#xA;" OwnerUserId="16474" LastEditorUserId="2444" LastEditDate="2020-11-13T19:13:38.087" LastActivityDate="2020-11-13T19:13:38.087" Title="How do I check that the combination of these models is good?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;performance&gt;&lt;ensemble-learning&gt;&lt;testing&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6870" PostTypeId="2" ParentId="6868" CreationDate="2018-06-24T01:14:08.630" Score="2" Body="&lt;p&gt;Goodness is subjective.  Reliable knowledge isn't possible with that flimsy a quality objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sturdy objective criteria you gave is 95%, so it is bad by that criteria.  (I'm assuming that the 95% is expected for a given data set or a randomized sample from a given data set.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the 80% accuracy is good by the criteria where the you sum the measures of the accuracy of the individual models, divide by the number of models, and find you have gained ten percentage points of accuracy over that average with your aggregated execution strategy.  (I'm assuming here that you used a defined set of network meta-parameters, layers depths and widths, starting parameters, activation model mapping to layers, inter-network connectivity, and loss/error methods for each model that is similar to the aggregated execution strategy.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have four questions.  (My apologies that this question leads to the two assumptions above and a bunch more questions.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the 80% accuracy also well over the maximum of the accuracies of the set of accuracies from the individual models?&lt;/li&gt;&#xA;&lt;li&gt;Is the computing resource draw to achieve 80% accuracy achievable with additional run time of the best of the individual models, using less than or equal to the computing resource draw to achieve 80% that way?&lt;/li&gt;&#xA;&lt;li&gt;Have you run your evaluation with a full set of meta parameter vectors to check the entire meta-space for best case?&lt;/li&gt;&#xA;&lt;li&gt;What economic, contractual, or operational hard stop is dictating the 90%?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If we know these answers, we may be able to respond more effectively and possibly find a loop hole in the logic that appears to leave you with an undesirable foregone conclusion.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-06-24T01:14:08.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6872" PostTypeId="1" AcceptedAnswerId="6879" CreationDate="2018-06-24T03:40:06.693" Score="6" ViewCount="297" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3997295/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The endocannabinoid system is a very important function of human biology&lt;/a&gt;. Unfortunately, due to the illegality of cannabis, it is a relatively new field of study. I have read a few articles about &lt;a href=&quot;https://medium.com/@skychain.global/googles-deepmind-is-using-ai-to-explore-dopamine-s-role-in-learning-d07e7521b16b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google researching the role of dopamine in learning&lt;/a&gt;, and &lt;a href=&quot;http://reset.me/story/anandamide-putting-the-bliss-molecule-to-work-for-your-brain/&quot; rel=&quot;nofollow noreferrer&quot;&gt;according to this article&lt;/a&gt;, anandamide (the neurotransmitter that closely resembles tetrahydrocannabinol):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;was found to do a lot more than produce a state of heightened happiness. It’s synthesized in areas of the brain that are important in memory, motivation, higher thought processes, and movement control.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Have any neuroscientists (or any scientists) considered the importance of the endocannabinoid system for cognitive function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, is there any reason this information might or might not be relevant to artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="16480" LastEditorUserId="2444" LastEditDate="2020-03-13T22:37:25.677" LastActivityDate="2020-03-13T22:37:25.677" Title="What is the importance of the endocannabinoid system for cognitive function?" Tags="&lt;neural-networks&gt;&lt;emotional-intelligence&gt;&lt;biology&gt;&lt;cognitive-science&gt;&lt;brain&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6874" PostTypeId="1" CreationDate="2018-06-24T18:11:43.943" Score="1" ViewCount="49" Body="&lt;p&gt;I have a database with hundreds of questions and answers. Would you like to know how I can work on this data in Google Cloud?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a social network where I have these questions and answers, and I would like to use the machine learning features and other Google Cloud tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know the step by step?&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-06-24T18:11:43.943" Title="Data to Google Machine Learning" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;google-cloud&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6875" PostTypeId="1" CreationDate="2018-06-24T20:41:02.710" Score="4" ViewCount="2954" Body="&lt;p&gt;I am a deep learning beginner recently reading this book &quot;Deep learning with Python&quot;, the example explains the process of implementing a greyscale image classification using MNIST in keras, in the compilation step, it said,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Before training, we’ll preprocess the data by reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval. Previously, our training images, for instance, were stored in an array of shape (60000, 28, 28) of type uint8 with values in the [0, 255] interval. We transform it into a float32 array of shape (60000, 28 * 28) with values between 0 and 1.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Images stored in an array of shape (60000, 28, 28) of type uint8 with values in the [0, 255] interval. For my understanding, the values are between 0-255 of each px and storied as 3D matrix. Can someone explain why needs to &quot;transform&quot; it into the network expects by scaling it and make &quot;all values are in the [0, 1]interval.&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please also make suggestions if I didn't explain some parts correctly.&lt;/p&gt;&#xA;" OwnerUserId="6390" LastEditorUserId="9947" LastEditDate="2018-07-24T03:08:51.240" LastActivityDate="2021-04-09T12:00:55.123" Title="What is the purpose of &quot;reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval.&quot;?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6876" PostTypeId="2" ParentId="6875" CreationDate="2018-06-25T02:17:51.820" Score="0" Body="&lt;p&gt;T'is easy, but misunderstood. What they mean is to map it from the range of 0-255 to the range of 0-1. This means that 0 would be 0, and 255 would be 1. &#xA;The code for this is as such in javascript:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function map (num, in_min, in_max, out_min, out_max) {&#xA;  return (num - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use the function, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var num = 5;&#xA;console.log(map(num, 0, 255, 0, 1)); // 0.0196078431372549&#xA;var num = 150&#xA;console.log(map(num, 0, 255, 0, 1)); // 0.5882352941176471&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Iterate over the entire image, and use the function (or you programming languages equivalent) on every egg or each pixel.&#xA;By doing so, all values are in the 0,1 interval. Next, all you have to do is to feed it to the network.&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-06-25T02:17:51.820" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6878" PostTypeId="1" CreationDate="2018-06-25T07:19:45.520" Score="3" ViewCount="104" Body="&lt;p&gt;In a final project in diagnosing ADHD using Machine Learning we obtained parameters from real patients. We used this data and got much higher success rates in LDA than in SVM and Naive Base, we had only 100 examples in our training set. We are wondering why LDA specifically succeeded much more than the others? &lt;/p&gt;&#xA;" OwnerUserId="16496" LastEditorUserId="9947" LastEditDate="2018-06-25T07:28:39.413" LastActivityDate="2018-11-23T19:01:05.647" Title="LDA performs much better than other methods" Tags="&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6879" PostTypeId="2" ParentId="6872" CreationDate="2018-06-25T10:26:40.807" Score="5" Body="&lt;p&gt;The release of Adenosine, Dopamine, Endorphin, Endocannabinoids, GABA, Glutamate, Norepinephrine, Oxytocin, Serotonin, and many others into specific regions of the brain are very likely an essential part of both activation tuning of single neurons and neuroplasticity, two essential aspects of organic learning researchers have been and will continue to work to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of those I've met in that sector of research are curious about the larger questions of what intelligence and consciousness are, and all of them appear to be interested in discovering how learning systems may be valuable in software engineering contexts.  These overarching questions are difficult to answer and the dive into the detail of learning has resulted in the expansion of ideas presented by Dr. Norbert Wiener in the mid 20th Century at MIT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How chemical feedback in regions of the brain are secreted, how they disseminate geometrically into organic structures, how they interact with receptors, and what that does to the cell metabolism to produce change in the cell is almost definitely part of the DNA driven design of higher animal learning.  There does not appear to be anything pointless or arbitrary about it.  Adaptation to improve survival is evident and, yes, study is well underway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Narcotics that interfere with the natural functioning of these organic signaling systems can lead to the inability to adapt in the individuals addicted to them.  That fact is a strong form of evidence that learning depends on these systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oxytocin is another neurotransmitter of interest because its release is associated with what would fit into the higher levels of human thought and motivation on Abraham Maslow's famous hierarchy of needs.  Oxytocin seems to be part of reward signaling for modes of thought like authenticity, compassion, intimacy, wisdom, spirituality, and other human mental capacities and patterns of thought that transcend mere rationalism.  Why is that important?  Because the ability to lay down selfish goals for the good of the community seem to depend largely on the oxytocin system's preemptive ability over mere survival mode neural activity in mammals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the cannabinoid receptors, there is a large enough body of media that show a correlation between a pool of successful artists and marijuana use to legitimately wonder whether there is any tie between creativity and the endocannabinoid system.  However in science (and hopefully in technology too), we are careful not to draw conclusions rashly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible that this apparent correlation is simply a social phenomenon where the popularity of the artistic products or live performances is mainly because of potential audiences similarly stimulating their receptors with cannabinoids.  For instance, those engaging in LSD trips on stage attracted those also engaging in LSD trips into their audiences two generations ago.  Whether the work of those artists was more creative because of the impact on the serotonin receptors is largely subjective.  How can researchers come to conclusions about what is good performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could analyze the audio to produce a table of notes in a performance and then develop a system to attach a numerical value to several positive quality metrics, study the price of tickets, or count downloads, but the decisions of what to measure and how to aggregate it into a final judgment of excellence is itself necessarily a subjective choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, if we place the many misconceptions of popular psychology and the drug culture aside, there is much research into the endocannabinoid system as part of the learning signaling, what machine learning researchers are currently calling reinforcement.  The more general term is, &quot;A non-linear control system's feedback signal,&quot; already developed in detail in the 1940s (Behavior, Purpose and Teleology &amp;mdash; A Rosenblueth, N Wiener, J Bigelow &amp;mdash; Philosophy of Science, 1943 &amp;mdash; U Chicago).  Some new trendy name will probably appear in the 2020s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This article claims that the mammals under test, &quot;exhibited enhanced learning.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Memory in Monoacylglycerol Lipase Knock-out Mice, by Bin Pan, Wei Wang, Peng Zhong, Jacqueline L. Blankman, Benjamin F. Cravatt and Qing-song Liu; Journal of Neuroscience 21 September 2011, 31 (38) 13420-13430; DOI: &lt;a href=&quot;https://doi.org/10.1523/JNEUROSCI.2075-11.2011&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1523/JNEUROSCI.2075-11.2011&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find many others: &lt;a href=&quot;https://scholar.google.com/scholar?q=endocannabinoid+learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scholar.google.com/scholar?q=endocannabinoid+learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-06-25T12:34:04.853" LastActivityDate="2018-06-25T12:34:04.853" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6880" PostTypeId="1" CreationDate="2018-06-25T11:27:35.883" Score="5" ViewCount="343" Body="&lt;p&gt;I have an image dataset where objects may belong to one of the hundred thousand classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of neural network architecture should I use in order to achieve this?&lt;/p&gt;&#xA;" OwnerUserId="12957" LastEditorUserId="2255" LastEditDate="2020-02-03T08:30:35.160" LastActivityDate="2020-02-03T09:14:33.997" Title="What kind of neural network architecture do I use to classify images into one hundred thousand classes?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6882" PostTypeId="2" ParentId="6880" CreationDate="2018-06-25T13:35:18.567" Score="1" Body="&lt;p&gt;A large one! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all seriousness, imagenet had roughly 1000 classes and did not require anything special from the top submissions. Depending on how deep(contextually) these classes are, you may want to do something like multi-label classification. Your biggest problems will likely be differentiating between classes, as well as class distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2018-06-25T13:35:18.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6883" PostTypeId="2" ParentId="6880" CreationDate="2018-06-25T14:08:22.557" Score="1" Body="&lt;p&gt;As you can imagine and as it has already been said, a large one for your network to tune weights and biases. But I wanted to nuance this statement with two points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First :  you can use an &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autoencodeur&lt;/a&gt; to pre-process your images. It can reduce dimensionality and so improve the learning capability and efficiency (in a generalization point of view). &#xA;This kind of NN takes your images as inputs, encode and then decode them to provie new representation of your initial images. Dealing with the decoded dataset can allow you to consider less hidden layer with less hidden nodes and then speed your work up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second : architecture is sure a thing to deal with image recognition, but you can also play on the input representation (that is what the autoencoder aforesaid is about). You can look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Principal_component_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCA&lt;/a&gt; (Principal Component Analysis). It allows to reduce dimensionality to a certain number of components (that you specify). It is often used in face recognition where inputs and targets are various. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that to say that architecture is sure a thing when dealing with large datasets, but there also few tools to reshape the inputs so that then can be more easily learnt. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And by doing so you can improve the capability of your network as much in term of time computation as in quality and accuracy of prediction&lt;/p&gt;&#xA;" OwnerUserId="11069" LastActivityDate="2018-06-25T14:08:22.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6884" PostTypeId="2" ParentId="6878" CreationDate="2018-06-25T14:42:57.533" Score="0" Body="&lt;p&gt;If I had to guess(and it is nothing more), I would say it has quite a bit to do with the problem itself and the architectures involved. Simply, the problem is less suited to a bayesian approach(Highly dependent features, linear distribution). &lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2018-06-25T14:54:02.697" LastActivityDate="2018-06-25T14:54:02.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6885" PostTypeId="2" ParentId="6859" CreationDate="2018-06-25T16:53:11.927" Score="1" Body="&lt;p&gt;To test the model, you have to achieve a Tensor with the same dimensions as the training data. This Tensor should be the same for the same image, no matter how many times you remake the Tensor with the same image. That being said, you can attempt to reverse-engineer how the Tensors are being made. Hint: The dimensions of the input tensor should be (28, 28, 1).&lt;/p&gt;&#xA;" OwnerUserId="14723" LastActivityDate="2018-06-25T16:53:11.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6886" PostTypeId="2" ParentId="6880" CreationDate="2018-06-25T23:52:06.340" Score="2" Body="&lt;p&gt;Classification tasks with a large number of classes are usually handled with &lt;a href=&quot;http://ruder.io/word-embeddings-softmax/index.html#hierarchicalsoftmax&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical softmax&lt;/a&gt; to reduce the complexity of the final layer. This is useful, for example, in applications such as word embedding where you have hundreds of thousands of classes (words), like in your case.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-06-25T23:52:06.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6887" PostTypeId="2" ParentId="4601" CreationDate="2018-06-26T03:17:10.907" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it make sense to do this task even though I don't have an experience with machine learning? How complicated is this task considering I'm using a well known framework?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think it does make sense, and using an established framework would get you up and running quickly.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Assuming that I'm taking the task, which algorithm should I choose to perform this kind of task?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Regression_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;regression&lt;/a&gt; problem, so I would recommend that you don't treat it as a classification task (unless you want to have a binary output like &quot;profitable&quot; / &quot;non-profitable&quot;). In essence, you are trying to identify the correlation between your inputs (previous purchases, location, etc.) and a certain metric (dollar value of the customer). Neural networks are very good at that (Accord seems to support neural networks, so you should be able to use that; &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; with &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras&lt;/a&gt; as the interface or &lt;a href=&quot;http://caffe.berkeleyvision.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Caffe&lt;/a&gt; might be other options to consider).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How should I build the training data? see my comments, do you think my comments are ok to start with? or maybe I can break the data directly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In most cases, you'd want to normalise your data before feeding it to the algorithm (this is particularly important for neural networks). The other thing you should do is consider what &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;features&lt;/a&gt; would be relevant to the task. For example, since you want the customer value as the predicted output, the customer's phone number and email address are most probably irrelevant, but number of previous purchases, age and geolocation might be very relevant. Maybe you have other features in your database - total dollar amount of previous purchases, frequency of purchase, number of returned items or refund requests, etc. Remember to keep your model in check by splitting your data into training, validation and test sets (as a rule of thumb, 70% training / 10% validation / 20% test, but that depends on how much data you have).&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-06-26T03:17:10.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6891" PostTypeId="2" ParentId="6823" CreationDate="2018-06-26T10:35:12.123" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the order of the genetic operations in NEAT?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You start by evaluating all of the initial neural networks and compute their initial fitness. &lt;/li&gt;&#xA;&lt;li&gt;Then you speciate, &lt;/li&gt;&#xA;&lt;li&gt;kill off the worst neural networks,&lt;/li&gt;&#xA;&lt;li&gt;mutate and crossover to produce offspring, and&lt;/li&gt;&#xA;&lt;li&gt;evaluate again. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The order of events is described on page 109 onwards &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the original NEAT paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Which neural networks are killed (or not) at each generation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The neural networks with the worst performance are killed off after speciation. None of the neural networks survive - the entire population is replaced with the offspring of the nets remaining after the culling stage. That said, you can implement &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm#Elitism&quot; rel=&quot;nofollow noreferrer&quot;&gt;elitism&lt;/a&gt;, where you keep some small portion of the best-performing nets and carry them over to the next generation without mutating them, but that is optional.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Who is being mutated and at what point?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;At the end of each generation, after speciation and culling. To produce offspring, some of the remaining nets are subjected to mutation (think asexual reproduction - like single-celled organisms - but the offspring is mutated so that it differs from the parent). The rest are subjected to crossover in random pairs, so this would be the equivalent of sexual reproduction where you need two parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="2444" LastEditDate="2019-10-13T01:27:44.530" LastActivityDate="2019-10-13T01:27:44.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6892" PostTypeId="1" AcceptedAnswerId="7296" CreationDate="2018-06-26T10:39:29.850" Score="10" ViewCount="513" Body="&lt;p&gt;The problem of adversarial examples is &lt;a href=&quot;https://arxiv.org/pdf/1312.6199.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;known&lt;/a&gt; to be critical for neural networks. For example, an image classifier can be manipulated by additively superimposing a different low amplitude image to each of many training examples that looks like noise but is designed to produce specific misclassifications.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5rfe9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5rfe9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since neural networks are applied to some safety-critical problems (e.g. self-driving cars), I have the following question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What tools are used to ensure safety-critical applications are resistant to the injection of adversarial examples at training time?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Laboratory research aimed at developing defensive security for neural networks exists. These are a few examples.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;adversarial training (see e.g. &lt;a href=&quot;https://arxiv.org/pdf/1611.01236.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Kurakin et al., ICLR 2017&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;defensive distillation (see e.g. &lt;a href=&quot;https://arxiv.org/pdf/1511.04508.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;N. Papernot et al., SSP 2016&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MMSTV defence (&lt;a href=&quot;https://arxiv.org/pdf/1706.06083&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maudry et al., ICLR 2018&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, do industrial-strength, production-ready defensive strategies and approaches exist?  Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for small perturbation limits)?&lt;/p&gt;&#xA;&lt;p&gt;There are already (at least) two questions related to the problem of &lt;a href=&quot;https://ai.stackexchange.com/q/6800/2444&quot;&gt;hacking&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/92/2444&quot;&gt;fooling&lt;/a&gt; of neural networks. The primary interest of this question, however, is whether any &lt;strong&gt;tools&lt;/strong&gt; exist that can defend against some adversarial example attacks.&lt;/p&gt;&#xA;" OwnerUserId="16354" LastEditorUserId="2444" LastEditDate="2021-03-03T08:54:08.117" LastActivityDate="2021-03-03T08:54:08.117" Title="What tools are used to deal with adversarial examples problem?" Tags="&lt;resource-request&gt;&lt;adversarial-ml&gt;&lt;ai-safety&gt;&lt;security&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6897" PostTypeId="1" AcceptedAnswerId="6901" CreationDate="2018-06-26T19:20:10.110" Score="2" ViewCount="57" Body="&lt;p&gt;I've recently come across the client-server model. From my understanding, the client requests the server, to which the server responds with a response. In this case, both the request and responses are vectors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reinforcement learning, the agent communicates with the environment via an &quot;action&quot;, to which the environment sends a scalar reward signal. The &quot;goal&quot; is to maximize this scalar reward signal in long run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more formal way to frame this problem such the agent searches the internet for obtaining more information about certain topic?&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2019-02-16T02:56:06.783" LastActivityDate="2019-02-16T02:56:06.783" Title="Analogy between reinforcement learning and web servers" Tags="&lt;reinforcement-learning&gt;&lt;concepts&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6898" PostTypeId="1" CreationDate="2018-06-26T19:23:54.920" Score="5" ViewCount="418" Body="&lt;p&gt;Can I get details about the algorithms used for classifying questions in stackoverflow (&quot;Questions that may already have your answer&quot;). Most of the suggestions I get are nowhere related to the question I have intended to ask.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="16355" LastEditDate="2018-06-27T19:42:15.393" LastActivityDate="2018-08-10T13:54:41.037" Title="What algorithms does stackoverflow use for classifying duplicate questions?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6899" PostTypeId="1" CreationDate="2018-06-26T19:26:45.687" Score="7" ViewCount="1307" Body="&lt;p&gt;I understand the intuition behind stacking models in machine learning, but even after thorough cross-validation scheme models seem to overfit. Most of the models I have seen in kaggle forums are large ensembles, but seem to overfit very little.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="16355" LastEditDate="2018-06-28T09:22:39.997" LastActivityDate="2019-05-03T04:05:36.073" Title="How to prevent overfitting in stacked models?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6901" PostTypeId="2" ParentId="6897" CreationDate="2018-06-26T21:12:01.070" Score="2" Body="&lt;p&gt;To paraphrase: &quot;Is there an analogy between client/server in web development and agent/environment in reinforcement learning?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is &quot;not really&quot;. There is no useful analogy here that allows any insight into RL from web server knowledge or vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you &lt;em&gt;could&lt;/em&gt; set up an agent where the goal was to collect information, and the available actions were to make web requests. Clearly in order to to this, you would need to make use of the client/server model for web servers, with the agent having control over the client web requests, and the environment being the network and servers of the world wide web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some very hard challenges to construct an open-ended &quot;web assistant&quot; agent. Here are a couple that I can think of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How to describe actions? Starting with raw web requests composed as strings would likely be very frustrating. Probably you would simplify and have a first action be a call to a search engine with some variation of the topic description, and then decisions about which links to follow, or perhaps whether to refine the search to better fetch sites related to the topic as it is being built.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to create a model of reward for collecting information? The first major stumbling block would be to measure the amount of &lt;em&gt;useful&lt;/em&gt; information that the agent had found on any request.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think with current levels of Natural Language Processing, setting an agent free to discover information according to some goal from a text topic description is a very hard task, beyond cutting edge research. It would definitely be unreasonable to expect any such agent to end up with any resemblance to &quot;understanding&quot; subject matter from text. The agent would have very little ability to tell the difference between accurate facts or lies, or just grammatically correct gibberish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting idea for agents trying to learn unsupervised from exploring an environment is creating a reward signal from data compression. An agent will have learned something useful about its environment, if on processing new information, it is able to compress its existing world model more efficiently. This is basic concept behind ideas of general learning agents from &lt;a href=&quot;https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jürgen Schmidhuber&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Marcus_Hutter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marcus Hutter&lt;/a&gt; and others. Research into this idea &lt;em&gt;could&lt;/em&gt; be a driver towards creating more generic AI learning systems - however, it is one idea amongst many in AI, and so far is research-only, it has not yet led to anything as practical as an AI web-searching assistant.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-06-26T21:12:01.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6902" PostTypeId="1" CreationDate="2018-06-26T22:07:13.167" Score="3" ViewCount="231" Body="&lt;p&gt;I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently in my country, there is a system in which certain groups of researchers upload information on products of scientific interest, such as research articles, books, patents, software, among others. Depending on the number of products, the system assigns a classification to each group, which can be A1, A, B and C, where A1 is the highest classification and C is the minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classification is done through a mathematical model whose entries are, the total number of each product, the total sum of all products, number of authors, among other indices that are calculated with the previous values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the entries are obtained, these values ​​are processed by a set of formulas and the final result is a single number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This number is located in a range provided by the mathematical model and this is how the group is classified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is given the current classification of a group, give suggestions of different values ​​to improve their classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if there is a group with classification C, suggest how many products it should have, how many authors, what value should its indexes have, so that its category would be finally B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the structure of my network should be:&#xA;-1 input, which would be the classification you want to get.&#xA;-Multiple output, one for each product and indexes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I do not understand how to make the network take into account the current classification of the group, in addition to the number of products and the value of the current indexes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have further questions about the problem, please feel free to ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate your suggestions.&lt;/p&gt;&#xA;" OwnerUserId="16258" LastActivityDate="2018-12-29T21:02:05.447" Title="How to build my own dataset and model for an LSTM neural network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;getting-started&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6903" PostTypeId="1" CreationDate="2018-06-26T23:00:06.657" Score="5" ViewCount="504" Body="&lt;p&gt;Does anyone know a good book to buy for beginners to coding for artificial intelligence? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done some coding in Java and a little with Python, but I would be willing to learn a new language too. &lt;/p&gt;&#xA;" OwnerUserId="16533" LastEditorUserId="2444" LastEditDate="2019-05-03T14:56:35.403" LastActivityDate="2019-09-24T14:16:39.907" Title="AI beginners books for coding" Tags="&lt;ai-design&gt;&lt;python&gt;&lt;getting-started&gt;&lt;reference-request&gt;&lt;java&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ClosedDate="2020-03-04T04:47:43.950" ContentLicense="CC BY-SA 4.0" />
  <row Id="6906" PostTypeId="1" CreationDate="2018-06-27T07:49:16.400" Score="2" ViewCount="301" Body="&lt;p&gt;Two months ago, I've found myself working on a churn detection problem which can be briefly described as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assume the current date is N&lt;/li&gt;&#xA;&lt;li&gt;Use customer behavior for N-1,..N-x dates to develop training dataset&lt;/li&gt;&#xA;&lt;li&gt;Train model and make prediction at time N, predicting if a customer will churn at N+2 (thus allowing data N+1 for churn prevention / reduction campaign)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When thinking through the design of the model and considerations for how to ensure that it would be successfully implemented, I identified a feedback loop wherein the prediction would trigger an event resulting in interaction with customer, potential changes to customer behavior and thus an impact on the next set of prediction data.  The following sequence of events could occur if successful (as an example):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Prediction -&amp;gt; Action to retain customer -&amp;gt; Change to customer behavior -&amp;gt;&#xA;Data for next prediction cycle not representative of training -&amp;gt; &#xA;Incorrect prediction and cost associated for handling incorrect prediction&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The feedback loop, fundamentally is that the action taken based on the prediction may impact the distribution or nature of features used to make the prediction.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When thinking through the how to solve the feedback problem I had listed the following three points as potential solutions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Retrain, test and validate model at every N+1 period and account for changes in behavior through new features (e.g. feature_i would involved details of the retention campaign a customer was treated to)&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This would result in huge production overhead and I believe to be infeasible &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Run the model intermittently to allow behavior to normalize&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Possible, however business would not be happy to have a prediction model which only works k times a year where k would have to be determined&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Predict the impact of the retention intervention and remove it from or the training set or include it as a new feature&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Possible, extensive thought and some experimentation needed to determine whether modeling the retention out or in would have the better effect.  Additionally, if modeled in, there may a short term penalty incurred as the model learns the new feature&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I did not actually end up having to confront the feedback problem (as during the exploration phase, sufficient evidence was obtained indicating that a predictive model for churn detection would not be required), however after reading &lt;a href=&quot;https://ai.google/research/pubs/pub43146&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper on the technical debt which could be incurred during the development of the machine learning systems I found myself pondering:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Were my considered strategies for dealing with the feedback reasonable?&lt;/li&gt;&#xA;&lt;li&gt;What other solutions should I have considered?&lt;/li&gt;&#xA;&lt;li&gt;Is there a way I could have re-framed the problem to completely design out the feedback loop (may be difficult to answer with the information provided, but if possible, but a &quot;you could have considered looking at...&quot; would be extremely beneficial)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11933" LastActivityDate="2018-06-27T07:49:16.400" Title="How do to mitigate or design out hidden feedback loops when designing ML systems?" Tags="&lt;machine-learning&gt;&lt;data-science&gt;&lt;imperfect-information&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6908" PostTypeId="1" AcceptedAnswerId="6913" CreationDate="2018-06-27T08:36:45.237" Score="3" ViewCount="473" Body="&lt;p&gt;I'm just beginning to understand neural networks and I've performed a couple of successful tests with numerical series where the NN was trained to find the odd one or a missing value. It all works pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next test I wanted to perform was to approxmimate the solution of a Sudoku which, I thought could also be seen as a special kind of numerical series. However, the results are really confusing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using an MLP with 81 neurons in each of the three layers. All output neurons show a strong tendency to yield values that are close to either 0 or 1. I have scaled and truncated the output values. The result can be seen below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Expected/Actual Solution:     Neural Net's Solution:&#xA;&#xA;6 2 7 3 5 0 8 4 1             9 0 9 9 9 3 0 0 3&#xA;3 4 8 2 1 6 0 5 7             0 9 9 0 0 0 9 9 0&#xA;5 1 0 4 7 8 6 2 3             0 9 1 9 9 0 2 0 4&#xA;1 6 4 0 2 7 5 3 8             0 0 5 0 0 9 0 0 7&#xA;2 0 3 8 4 5 1 7 6             0 0 0 0 0 9 9 0 9&#xA;7 8 5 1 6 3 4 0 2             9 9 9 9 0 6 2 9 0&#xA;0 5 6 7 3 1 2 8 4             0 0 0 0 9 9 0 9 0&#xA;4 3 1 5 8 2 7 6 0             9 9 0 0 0 0 9 0 9&#xA;8 7 2 6 0 4 3 1 5             9 9 0 9 9 0 9 0 9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The training set size is 100000 Sudokus while the learning rate is a constant 0.5. I'm using NodeJS/Javascript with the Synaptic library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I don't expect a perfect solution from you guys, but rather a hint if that kind of behavior is a typical symptom for a known problem, like too few/many neurons, small training set, etc.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="16542" LastEditorUserId="2444" LastEditDate="2019-10-22T20:59:49.730" LastActivityDate="2019-10-25T08:40:57.983" Title="How can a neural network learn to play sudoku?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;&lt;multilayer-perceptron&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6910" PostTypeId="2" ParentId="3548" CreationDate="2018-06-27T10:27:44.683" Score="3" Body="&lt;p&gt;The most important skill you need is self-discipline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the mathematical prerequisites, you will need to study statistics, probability theory, calculus, and linear algebra, given that e.g. most machine learning algorithms are highly based on concepts from these areas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the programming prerequisites, Python and R are usually a good choice, given the relevant libraries that are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also need to learn frameworks like Hadoop, in case you want to work with big data.&lt;/p&gt;&#xA;" OwnerUserId="8680" LastEditorUserId="2444" LastEditDate="2019-06-22T16:43:02.843" LastActivityDate="2019-06-22T16:43:02.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6911" PostTypeId="2" ParentId="3548" CreationDate="2018-06-27T12:51:40.050" Score="1" Body="&lt;p&gt;&quot;artificial intelligence&quot; is now a mature field and there are many subfields in it. there are entire theories erected around each subfield of artificial intelligence. a simple analogy would like &quot;what are the skills needed for succeeding in mathematics/physics?&quot;. the answer really depends on what branch you want to delve into. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you are planning to go into more application specific side of artificial intelligence (machine learning/ deep learning) then you must be thorough with linear algebra . since most of machine learning algorithms can be boiled down to simple tricks in linear algebra. learning about optimization will help a lot because you will encounter many algorithms involving convex/nonconvex optimization methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also statistics and probability are must for any field in artificial algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, at core artificial intelligence deals with machines that think.&#xA;So it would be better if you are through with formal logic , automata and complexity theory . &lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-27T12:51:40.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6912" PostTypeId="2" ParentId="6903" CreationDate="2018-06-27T12:53:43.307" Score="0" Body="&lt;p&gt;Have a look at &lt;a href=&quot;http://axon.cs.byu.edu/~martinez/classes/778/Papers/GP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Programming Collective Intelligence&lt;/em&gt;&lt;/a&gt; (2007) by Toby Segaran. It is mainly about machine learning than conventional AI techniques, such as search algorithms or path planning.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2019-05-03T14:58:27.450" LastActivityDate="2019-05-03T14:58:27.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6913" PostTypeId="2" ParentId="6908" CreationDate="2018-06-27T13:09:18.030" Score="2" Body="&lt;p&gt;I think it is the wrong way to frame sudoku as a regression problem in neural networks. &#xA;Firstly, you have to understand what regression is. &quot;Regression&quot; is when you predict a value given certain parameters, where the parameters are related to the value you have to predict. This happens because at the core neural networks are &quot;function approximators&quot;, they model the function by adjusting their weights using lots of data. They tend to form a highly non-linear boundary to separate classes internally in a high dimensional data-space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sudoku doesn't fit in this scenario, the combinatorial complexity of sudoku is way too high for a neural network even if you add many layers to it, it is a totally different problem in its own right. You simple can't &quot;regress&quot; the right values of a perfect sudoku here, they are not numbers like &quot;pixel &quot; intensities in images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you could apply reinforcement learning techniques to learn an optimal policy to solve sudoku.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you have mentioned an &quot;approximate&quot; solution for the sudoku, what do you mean by &quot;approximate&quot;?  If you mean by this that only a few squares are out of place, then it is a wrong assumption, because neural networks are proven to be good image classifiers, as they are robust to translational invariance in this case, that is not what you need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could, however, do a small experiment to see what the neural network actually learns, replace the numbers by pixel value intensities and train a generative adversarial network on the sudoku images and see the images of sudokus produced by it, to see what actually the network can't learn.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2019-10-22T20:54:48.223" LastActivityDate="2019-10-22T20:54:48.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6914" PostTypeId="1" CreationDate="2018-06-27T13:10:10.437" Score="1" ViewCount="77" Body="&lt;p&gt;May someone explains some first iterations of this sigma?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oh6bY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oh6bY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how did it convert the above expression to below expression? What it the meaning of I(x) and I(y)?&lt;/p&gt;&#xA;" OwnerUserId="9941" LastActivityDate="2018-06-27T13:10:10.437" Title="How does this sigma work?(Harris algorithm)" Tags="&lt;math&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2018-07-22T21:49:39.103" ContentLicense="CC BY-SA 4.0" />
  <row Id="6916" PostTypeId="1" AcceptedAnswerId="6950" CreationDate="2018-06-27T13:22:31.390" Score="2" ViewCount="280" Body="&lt;p&gt;Would people go far with Artificial Intelligence and machine learning to the point where machines could learn during a long period of time to distinguish what's 'good' from 'bad' according to people living in a restricted geographical area, and then the machines take control and turn what was learnt into a set of 'rules' and 'laws' (think of it as an effective machine of 'politics') that match the majority of the people's view of issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That should be accepted by everyone, since a contract set at the beginning says: &quot;Everyone is ok&quot;.&lt;/p&gt;&#xA;" OwnerUserId="16548" LastEditorUserId="2444" LastEditDate="2019-06-15T14:09:26.560" LastActivityDate="2019-06-15T14:09:26.560" Title="Can an AI distinguish between good and bad according to people living in a restricted geographical area?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;agi&gt;&lt;ethics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-18T16:16:17.763" ContentLicense="CC BY-SA 4.0" />
  <row Id="6918" PostTypeId="5" CreationDate="2018-06-27T18:10:47.077" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-27T18:10:47.077" LastActivityDate="2018-06-27T18:10:47.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6919" PostTypeId="4" CreationDate="2018-06-27T18:10:47.077" Score="0" Body="For questions related to confidence in AI work, including confidence that a convergent algorithm will converge, confidence intervals as an output along with value, confidence that a model will not lead to over-fitting on the basis of complexity metrics, or some other reliability metric or process for obtaining it." OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-11-30T18:28:45.200" LastActivityDate="2018-11-30T18:28:45.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6920" PostTypeId="1" AcceptedAnswerId="6935" CreationDate="2018-06-28T02:17:46.260" Score="3" ViewCount="94" Body="&lt;p&gt;A bunch of friends and I play ultimate every week. Recently I wrote a program to choose our teams for us, as well as keep track of certain data (like which players were on which team, which team won, what was the score, how long the game was, etc). I wanted to use a machine learning technique to make the teams for us in order to optimize how fairly balanced the teams are (possibly measured by how many total points are scored in a game or how long a game lasts).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently taking a machine learning MOOC and being introduced to very basic machine learning techniques (linear regression with gradient decent or normal equations, basic classification stuff, stuff like that). Although I hope I will come across a technique that fits my needs by the end of this course, I wanted to ask here to see if I can get a head start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried searching around everywhere, but couldn't find anything relevant. &lt;strong&gt;So my question is&lt;/strong&gt;, is there an obvious technique I should look into for such a problem? If it's something too advanced for a beginner, that's fine too, but I'd like to get started learning/practicing it asap instead of waiting for my course to hopefully hit upon it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: to clarify further, I would prefer something that looks at relationships between individual players like &quot;when Steph plays with Bill she is more likely to win&quot; or &quot;Steph plays worse when she is on a team with players who have a high win percentage&quot;. I'd also prefer to be able to code it in python, but am willing to learn any other language&lt;/p&gt;&#xA;" OwnerUserId="16561" LastEditorUserId="1671" LastEditDate="2019-11-06T21:56:40.440" LastActivityDate="2019-11-06T21:56:40.440" Title="What beginner-friendly machine learning method should I use to make teams for my pickup ultimate frisbee club fairly balanced?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;getting-started&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6921" PostTypeId="1" CreationDate="2018-06-28T05:53:23.973" Score="1" ViewCount="277" Body="&lt;p&gt;In the data-sets like coco-text and total-text, the images are of different sizes (height*width). I'm using these data sets for text detection. I want to create a DNN model for this. So input data should be of same size. If I resize these images to a fixed size, annotations given in the data-set that is the location of the text in the images will be changed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how to proceed with these data-set? I'm new to machine learning and I didn't get answer for this anywhere. Thanks for the help. &lt;/p&gt;&#xA;" OwnerUserId="12273" LastEditorUserId="16355" LastEditDate="2018-06-28T16:36:14.430" LastActivityDate="2018-07-11T18:04:19.990" Title="Normalization for well known data sets like coco-text and total text data set" Tags="&lt;computer-vision&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6922" PostTypeId="2" ParentId="6921" CreationDate="2018-06-28T06:41:41.257" Score="1" Body="&lt;p&gt;Find the largest height and width amongst all the images. Let us call it H and W respectively. It is true that you cannot resize the images, but say if you have an image of height &lt;code&gt;h&lt;/code&gt; and width &lt;code&gt;w&lt;/code&gt; where &lt;code&gt;h &amp;lt; H, w &amp;lt; W&lt;/code&gt;. To the right of the image append &lt;code&gt;W - w&lt;/code&gt; number of columns and at the bottom of the image append &lt;code&gt;H - h&lt;/code&gt; number of rows having some constant value (0 is okay for grey-scale and B/W images and 0 for each of the channel in case of colour images).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way all the images will be of same size. Since you appending at the right and bottom of the image, the annotations will not lose its meaning in the transformed image in terms of the position and content of the text to recognised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try &lt;a href=&quot;https://arxiv.org/pdf/1601.06759.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pixelRNN&lt;/a&gt; kind of ideas after you are done with DNN. RNN can handle variable length inputs and in your case it will be sequence of pixels. Here you don't need to append rows and columns to the image.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-06-28T06:41:41.257" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6923" PostTypeId="1" AcceptedAnswerId="6924" CreationDate="2018-06-28T08:19:24.323" Score="5" ViewCount="2147" Body="&lt;p&gt;I just read about deep Q-learning, which is using a neural network for the value function instead of a table.&lt;/p&gt;&#xA;&lt;p&gt;I saw the example here: &lt;a href=&quot;https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using Keras and Deep Q-Network to Play FlappyBird&lt;/a&gt; and he used a CNN to get the Q-value.&lt;/p&gt;&#xA;&lt;p&gt;My confusion is on the last layer of his neural net. Neurons in the output layer each represent an action (flap, or not flap). I also see the &lt;a href=&quot;http://edersantana.github.io/articles/keras_rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;other projects&lt;/a&gt; where the output layer also represents all available actions (move-left, stop, etc.)&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;How would you represent all the available actions of a chess game?&lt;/em&gt; Every pawn has a unique and available movement. We also need to choose how far it will move (rook can move more than one square). I've read &lt;a href=&quot;https://arxiv.org/abs/1509.01549&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe chess engine's&lt;/a&gt; paper and can't find how he represents the output layer (I'll read once again).&lt;/p&gt;&#xA;&lt;p&gt;I hope somebody here can give a nice explanation about how to design NN architecture in Q-learning, I'm new in reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="32410" LastEditDate="2021-04-22T12:49:22.977" LastActivityDate="2021-04-22T12:49:22.977" Title="How should I model all available actions of a chess game in deep Q-learning?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6924" PostTypeId="2" ParentId="6923" CreationDate="2018-06-28T09:45:17.093" Score="6" Body="&lt;p&gt;To model chess as a Markov decision problem (MDP) you can refer to the AlphaZero paper (&lt;a href=&quot;https://arxiv.org/abs/1712.01815&quot; rel=&quot;noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/a&gt;). The exact details can be found starting from the bottom of page 13.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly, an action is described by picking a piece and then picking a move with it. The size of the board is 8 by 8 so there can be 8x8 possibilities for picking a piece. Then we can either pick linear movements (in 8 directions) and then pick the number of steps in that direction (maximum 7 steps) or we can make a knight movement (maximum 8 possibilities). So far that is 8x7 + 8. Furthermore, we also need to consider underpromotions (promoting a pawn into a non-queen piece). In this scenario we can have 3 types of pawn movements (forward, left diagonal or right diagonal capture) and 3 types of promotions (rook, knight, bishop) so that makes it 9. So the total dimension of the action space is 8x8x(8x7+8+9) and this will be the number of neuron outputs you will need to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this action space representation covers every possible scenario and for example at the start of the game the action of picking the tile E4 and promoting it to a bishop doesn't make sense (there are no pieces on tile E4 at the beginning of the game). Or if we pick a tile where there is a rook we cannot make a knight movement with it. Therefore you will also need to implement a function that can return you the set of possible actions in a given state and ignore all neural network outputs that is not contained in this set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously this action representation is not set into stone so if you can come up with something better or more compact you can use that one too. You can also make restrictions to your game for example by not allowing underpromotions.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastActivityDate="2018-06-28T09:45:17.093" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6925" PostTypeId="2" ParentId="5011" CreationDate="2018-06-28T10:16:20.877" Score="0" Body="&lt;p&gt;I am working on the same problem but in another way. I want to recognise only certain slots so that I can use them for further processing. I found this thesis &lt;a href=&quot;https://orca.cf.ac.uk/86873/1/2016niuzphd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;CAD feature recognition&quot; &lt;/a&gt;to be helpful.&lt;/p&gt;&#xA;" OwnerUserId="16566" LastEditorUserId="9947" LastEditDate="2018-06-28T16:08:21.523" LastActivityDate="2018-06-28T16:08:21.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6926" PostTypeId="1" AcceptedAnswerId="7023" CreationDate="2018-06-28T11:00:52.403" Score="5" ViewCount="316" Body="&lt;p&gt;In traditional computer vision and computer graphics, the pose matrix is a &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; matrix of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;r_{11}  &amp;amp; r_{12}  &amp;amp;  r_{12}   &amp;amp; t_{1} \\&#xA;r_{21}  &amp;amp;  r_{22}   &amp;amp; r_{22} &amp;amp;   t_{2} \\&#xA;r_{31}   &amp;amp; r_{32}  &amp;amp;  r_{32}  &amp;amp;  t_{3} \\ &#xA;0   &amp;amp;   0    &amp;amp;  0   &amp;amp;   1&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and is a transformation to change viewpoints from one frame to another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://openreview.net/pdf?id=HJWLfGWRb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix Capsules with EM Routing&lt;/a&gt; paper, they say that the &quot;pose&quot; of various sub-objects of an object are encoded by each capsule lower layer. But from the procedure described in the paper, I understand that the pose matrix they talk about doesn't conform to the definition of the pose matrix. There isn't any restriction on keeping the form of the pose matrix shown above. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;So, is it right to use the word &quot;pose&quot; to describe the &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; matrix of each capsule?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Moreover, since the claim is that the capsules learn the pose matrices of the sub-objects of an object, does it mean they learn the viewpoint transformations of the sub-objects, since the pose matrix is actually a transformation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16569" LastEditorUserId="2444" LastEditDate="2020-06-09T15:09:54.557" LastActivityDate="2020-06-09T15:12:11.507" Title="Is the word &quot;pose&quot; used correctly in the paper &quot;Matrix Capsules with EM Routing&quot;?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;capsule-neural-network&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6927" PostTypeId="1" CreationDate="2018-06-28T16:23:15.703" Score="2" ViewCount="141" Body="&lt;p&gt;Earlier this month, Google released a set of principles governing their AI development initiatives.  The stated principles are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Objectives for AI Applications:&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Be socially beneficial.&lt;/li&gt;&#xA;  &lt;li&gt;Avoid creating or reinforcing unfair bias.&lt;/li&gt;&#xA;  &lt;li&gt;Be built and tested for safety.&lt;/li&gt;&#xA;  &lt;li&gt;Be accountable to people.&lt;/li&gt;&#xA;  &lt;li&gt;Incorporate privacy design principles.&lt;/li&gt;&#xA;  &lt;li&gt;Uphold high standards of scientific excellence.&lt;/li&gt;&#xA;  &lt;li&gt;Be made available for uses that accord with these principles.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;AI Applications not to be Pursued:&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;ol start=&quot;8&quot;&gt;&#xA;  &lt;li&gt;Technologies that cause or are likely to cause overall harm. Where there is a material risk of harm, we will proceed only where we believe that the benefits substantially outweigh the risks, and will incorporate appropriate safety constraints.&lt;/li&gt;&#xA;  &lt;li&gt;Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.&lt;/li&gt;&#xA;  &lt;li&gt;Technologies that gather or use information for surveillance violating internationally accepted norms.&lt;/li&gt;&#xA;  &lt;li&gt;Technologies whose purpose contravenes widely accepted principles of international law and human rights.&#xA;  &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://ai.google/principles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence at Google: Our Principles&lt;/a&gt;&lt;/sub&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are this guidelines sufficient?  &lt;/li&gt;&#xA;&lt;li&gt;Are there any &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;I, Robot&lt;/a&gt;&quot; conflicts &lt;/li&gt;&#xA;&lt;li&gt;How much does this matter if other corporations and state agencies don't hew to similar guidelines?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-28T16:36:28.557" LastActivityDate="2018-07-28T10:38:15.387" Title="Google's Principles of Artificial Intelligence" Tags="&lt;philosophy&gt;&lt;ethics&gt;&lt;logic&gt;&lt;theory&gt;&lt;google&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6928" PostTypeId="1" AcceptedAnswerId="6939" CreationDate="2018-06-28T16:25:57.687" Score="2" ViewCount="138" Body="&lt;p&gt;How should I design my input layer for the following classification problem?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input&lt;/strong&gt;: 5 cards (from a deck of 52 cards) in a card game;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;: some classification using a neural network&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How should I model the input layer?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option A&lt;/strong&gt;: 5 one-hot encodings for the 5 cards, i.e. 5 one-hot vectors of length 52 = 260 input vector. For example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;[0,0,0,0,0,0,1,...],&#xA;[1,0,0,0,0,0,0,...],&#xA;[0,0,0,0,0,1,0,...],&#xA;[0,0,1,0,0,0,0,...],&#xA;[0,0,0,0,1,0,0,...]&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option B&lt;/strong&gt;: 5 hot encoding encompassing all 5 cards in one 52 element vector&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1,0,1,0,1,1,1,...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What are the disadvantages between A and B?&lt;/p&gt;&#xA;" OwnerUserId="16574" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-19T20:30:45.617" Title="How should I encode the input which are 5 cards from a deck of 52 cards?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6929" PostTypeId="2" ParentId="5453" CreationDate="2018-06-28T17:43:16.083" Score="2" Body="&lt;p&gt;First of all, you mention that you have &lt;strong&gt;categorical&lt;/strong&gt; data. I don't see &lt;strong&gt;how you can define similarity&lt;/strong&gt; so that you can also define the distance between the predicted value and the ground truth (error). You can do that &lt;strong&gt;only if&lt;/strong&gt; the data are &lt;strong&gt;ordinal&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to just classify between &lt;strong&gt;normal&lt;/strong&gt; and &lt;strong&gt;anomalous&lt;/strong&gt; points (&lt;strong&gt;binary classification&lt;/strong&gt;), without caring about further classification of the anomaly types themselves, one of the most common algorithms is the &lt;a href=&quot;http://rvlasveld.github.io/blog/2013/07/12/introduction-to-one-class-support-vector-machines/&quot; rel=&quot;nofollow noreferrer&quot;&gt;One-Class Support Vector Machine (OC-SVM)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anomalies are unpredictable in nature and sometimes hard to replicate and record. Therefore, there is usually lack of anomalous data and supervised learning approaches suffer because if you sacrifice some &quot;precious&quot; anomalous points to train the algorithm, you cannot use them to test it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main advantage of OC-SVM is that it is semi-supervised learning, meaning that you train it only with normal data and then it can detect samples that deviate from the trained behaviour during testing and classify them as anomalous. Thus, you &quot;save&quot; all the rare anomalous points for testing purposes!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at this short &lt;a href=&quot;http://scikit-learn.org/stable/auto_examples/svm/plot_oneclass.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python example&lt;/a&gt;, it has all you need :)&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-06-28T17:43:16.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6931" PostTypeId="1" AcceptedAnswerId="6945" CreationDate="2018-06-28T19:45:06.297" Score="2" ViewCount="31" Body="&lt;p&gt;I've been wondering, how, in the most simple-to-implement basic principle, does the light projection to depth map technique described here &lt;a href=&quot;https://www.lightform.com/how-it-works&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.lightform.com/how-it-works&lt;/a&gt; actually functions? Is it some kind of an average based on the color of x pixel over all the patterns or what? How difficult would it be to code something that could do this, up?&lt;/p&gt;&#xA;" OwnerUserId="16579" LastActivityDate="2018-06-29T08:09:16.113" Title="Structured lighting basic principles for depth mapping" Tags="&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6933" PostTypeId="1" CreationDate="2018-06-28T21:34:51.633" Score="2" ViewCount="25" Body="&lt;p&gt;I am working on a problem where I have to train a CNN to recognize different kinds of surfaces. One important characteristic of the surfaces I am interested is is how reflective they are. I have been trying to find a method that quantifies how &quot;shiny&quot; a surface is, but I have not found much. I am hoping that someone can point me toward a method or some research into this kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="16582" LastActivityDate="2018-06-29T07:41:51.553" Title="How to quantify the reflectance in an image?" Tags="&lt;computer-vision&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6934" PostTypeId="1" AcceptedAnswerId="6936" CreationDate="2018-06-28T22:47:30.847" Score="4" ViewCount="1317" Body="&lt;p&gt;I have a very simple question about Conv nets. I understand the whole principle, but only one thing is not well explained on the Internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have a 16 channels image that goes on a convolutional layer, and the trainable filters are 3 7x7 filters, meaning that its output has 3 channels, how does the conv layer do to go from 16 to 3 channels? What mathematical operation is applied?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any clarification on this.&lt;/p&gt;&#xA;" OwnerUserId="14801" LastActivityDate="2019-08-20T06:18:34.080" Title="Convolutional neural nets and reduction of the layers" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6935" PostTypeId="2" ParentId="6920" CreationDate="2018-06-28T22:48:18.703" Score="2" Body="&lt;p&gt;What you have on your hands is an &lt;em&gt;attribution&lt;/em&gt; problem (and that’s a good keyword to help in your Googling). Two common approaches are computing &lt;em&gt;Shapley values&lt;/em&gt; or a &lt;em&gt;Markov chain&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, I think Shapley values would be a good approach. To over-simplify, this approach attempts to first determine the total “surplus” of value created by different combinations of players, and then estimate how much was due to each player in the combination. Also under consideration are constrained combinatorial effects: e.g. having two great goalies doesn’t add much over one great goalie since they can’t play at the same time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on how hands-on you want to be you could implement yourself or find existing libraries by Googling e.g. “Shapley values python”. &lt;/p&gt;&#xA;" OwnerUserId="13360" LastActivityDate="2018-06-28T22:48:18.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6936" PostTypeId="2" ParentId="6934" CreationDate="2018-06-29T01:01:34.533" Score="3" Body="&lt;p&gt;The reason why you go from 16 to 3 channels is that by convention filters span the &lt;strong&gt;entire depth&lt;/strong&gt; of the input. Therefore, your filters would actually be &lt;code&gt;7&lt;/code&gt;x&lt;code&gt;7&lt;/code&gt;x&lt;code&gt;16&lt;/code&gt; in order to cover all channels of the input. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Detailed procedure&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The output of the convolution automatically has a depth equal to the number of filters (so in your case this is 3) because you have an (&lt;code&gt;m x k&lt;/code&gt;) filter matrix, where &lt;code&gt;m&lt;/code&gt; is the number of filters and &lt;code&gt;k&lt;/code&gt; is the number of elements in the &lt;strong&gt;unrolled filter&lt;/strong&gt; (in your case, &lt;code&gt;m = 3&lt;/code&gt; and &lt;code&gt;k = 7 x 7 x 16 = 784&lt;/code&gt;, so the filter matrix is &lt;code&gt;3 x 784&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input is usually unrolled according to the im2col procedure, where each tile corresponding to a single filter location is stretched into a column equal to the &lt;strong&gt;unrolled&lt;/strong&gt; filter size. This is repeated for each filter location, so you end up with very large matrix of size (&lt;code&gt;k x n&lt;/code&gt;), where &lt;code&gt;k&lt;/code&gt; is the same as &lt;code&gt;k&lt;/code&gt; above in the filter matrix, and &lt;code&gt;n&lt;/code&gt; depends on your padding and stride.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplying the (&lt;code&gt;m x k&lt;/code&gt;) filter matrix with the (&lt;code&gt;k x n&lt;/code&gt;) input matrix gives you an (&lt;code&gt;m x n&lt;/code&gt;) output matrix, where &lt;code&gt;m&lt;/code&gt; is the number of filters.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can find some very nice visual explanations of the convolution procedure &lt;a href=&quot;https://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/making_faster.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-06-29T01:01:34.533" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6938" PostTypeId="2" ParentId="1851" CreationDate="2018-06-29T02:54:42.017" Score="6" Body="&lt;p&gt;The two examples present essentially the same operation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In both cases, the network is trained with gradient descent using the backpropagated squared error computed at the output.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Both examples use the logistic function for node activation (the derivative of the logistic function &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$s(1 - s)$&lt;/span&gt;. This derivative is obviously very easy to compute, and this is part of the reason why it was so widely used (these days the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt; activation function is more popular, especially with convolutional networks).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The first method also uses &lt;a href=&quot;https://www.quora.com/What-does-momentum-mean-in-neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;momentum&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The main difference I can see is that in the first case backpropagation is iterative while in the second example it is performed in batch mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last video in the series by Welch Labs introduces a quasi-Newtonian method which offers the advantage of finding the minimum of the cost function by computing the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hessian_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hessian&lt;/a&gt; (matrix of second-order derivatives of the error with respect to the weights). However, this feels like comparing apples and oranges - the vanilla gradient descent does not use second-order information.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="2444" LastEditDate="2020-05-23T13:00:58.127" LastActivityDate="2020-05-23T13:00:58.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6939" PostTypeId="2" ParentId="6928" CreationDate="2018-06-29T03:21:54.190" Score="1" Body="&lt;p&gt;Depends on how your game is played. Is there any meaning assigned to the order of cards, or are all 5 played simultaneously? If order matters, use 5 one-hot vectors so you can choose how to order them, otherwise use a single 5-hot input vector. I would also add that if &lt;strong&gt;temporal&lt;/strong&gt; order matters, you could also use a &lt;strong&gt;recurrent&lt;/strong&gt; net with a 52-element input and feed the five one-hot vectors one after another.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-06-29T03:21:54.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6940" PostTypeId="2" ParentId="6934" CreationDate="2018-06-29T06:15:42.357" Score="1" Body="&lt;p&gt;Your input is having 16 channels of each of dimension &lt;code&gt;m x n&lt;/code&gt; and there are 3 filter namely &lt;code&gt;f1, f2 and f3&lt;/code&gt; of dimensions &lt;code&gt;fm x fn&lt;/code&gt;. We say that a filter is applied to a channel when it is superimposed on the image starting left-most, performing the operation of multiplying the weights of filter with the corresponding value in the image and then summing up to a single value and moving the filter to right (then down when it reaches rightmost part) across the image according to the stride of the filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned when a filter say &lt;code&gt;f1&lt;/code&gt; is applied to a channel say &lt;code&gt;c&lt;/code&gt;, there is a single value. Now applying them to all channels  we get 16 values and all of them are added up to a single value. &lt;code&gt;f1&lt;/code&gt; is moved according to the stride and the same operation is repeated to get an output with a single channel (number of rows and columns are determined by padding, stride, dilation and kernel size of the filers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aforesaid process is done by all the 3 filters giving rise to 3 channels. In this way the conv layer makes the input to go from 16 to 3 channels. More detailed explanations can be found &lt;a href=&quot;http://colah.github.io/posts/2014-07-Understanding-Convolutions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-06-29T06:15:42.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6941" PostTypeId="2" ParentId="5110" CreationDate="2018-06-29T06:38:10.127" Score="0" Body="&lt;p&gt;you could implement a simple TTS system that can translate your voice line by line to code , but it would of no use . you cant express code in a line-by-line manner. Coding is a highly iterative process at first you come up with a rough sketch for which you add details later on , and from NLP point of view this is a highly ambitious project . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the heart almost all of &#xA;ai techniques (neural networks) are functions that map one domain to another , you cant map natural language sentences to instructions in code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However you can implement a tts system for a small language like LOGO. &lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-29T06:38:10.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6943" PostTypeId="2" ParentId="6933" CreationDate="2018-06-29T07:41:51.553" Score="1" Body="&lt;p&gt;You should first know about the layer-wise working of a convolutional neural network. Read this &lt;a href=&quot;https://distill.pub/2018/building-blocks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://distill.pub/2018/building-blocks/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each layer of a cnn forms representations that are increasingly complex , which are  combinations of simpler representations formed by its previous layers. &#xA;&quot;shine&quot; is not a pattern (from image processing perspective). shine occurs due to change in intensity values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for quantifying &quot;shine&quot; you dont have to go for convolutional networks only , there are many more simpler methods for such tasks you could try fitting simple linear model and progress towards more complex ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for classifying textures , there is interesting work going with &quot;invariant scattering convolutional networks&quot; you should check that.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-29T07:41:51.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6944" PostTypeId="2" ParentId="4778" CreationDate="2018-06-29T07:56:23.487" Score="0" Body="&lt;p&gt;training a model to identify new persons from video logs seems to be a daunting task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will be needing lots of data and computational power for building such a model.&#xA;and there is relatively low work going on videos  due to the amount of computational power required , even training a simple video classifier with reasonable accuracy requires lots of expertise and resources (data and computing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that recognising persons from videos and &quot;new&quot; person identification seems to be even more difficult , you should definitely read some papers on video classification using convolutional neural networks to get an idea of how hard the problem is. frameworks are only mere tools which give you helper functions to build neural networks , so  choice of frameworks is not a question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real question is are you sure you want to train a model (CNN) for this task? &#xA;&lt;em&gt;Can use video feeds to detect new people in the workspace and prompt someone (via e-mail or text message) for identification&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for this you should be training a convolutional network on the factory videos dataset , i.e you the cnn trains on frame level for the videos.&#xA;and recognizing and classifying &quot;known&quot; and &quot;unknown&quot; faces from videos is a research problem in its own right &lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-29T07:56:23.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6945" PostTypeId="2" ParentId="6931" CreationDate="2018-06-29T08:09:16.113" Score="1" Body="&lt;p&gt;&lt;em&gt;how, in the most simple-to-implement basic principle, does the light projection to depth map technique described here &lt;a href=&quot;https://www.lightform.com/how-it-works&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.lightform.com/how-it-works&lt;/a&gt; actually functions?Is it some kind of an average based on the color of x pixel over all the patterns or what?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No it is not that simple , more details on that are described in &lt;em&gt;&lt;a href=&quot;https://kevinkarsch.com/publications/sa11-lowres.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://kevinkarsch.com/publications/sa11-lowres.pdf&lt;/a&gt;&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S the author of this paper went on to become CTO of lightform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is it some kind of an average based on the color of x pixel over all the patterns or what? How difficult would it be to code something that could do this, up?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned above it is not simple average based on color , coding would be a tough task , you could implement model if you are capable enough to code a ray-tracer from scratch. But replicating the exact results would require a lot of low-level knowledge and certain domain expertise as not much of the implementation details and hyperparameters are described.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-29T08:09:16.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6946" PostTypeId="1" CreationDate="2018-06-29T08:10:35.707" Score="2" ViewCount="173" Body="&lt;p&gt;I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the files with me :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The temporal sequence of file usage for last 6 months(whenever the&#xA;file was read/written/modified and by whom).&lt;/li&gt;&#xA;&lt;li&gt;All the users who are on the server and can access the files.&lt;/li&gt;&#xA;&lt;li&gt;Last modified/written/read epoch time and by whom.&lt;/li&gt;&#xA;&lt;li&gt;File creation epoch time and by whom.&lt;/li&gt;&#xA;&lt;li&gt;Any compliance regulations on the file(whether the file contains any&#xA;confidential data).&lt;/li&gt;&#xA;&lt;li&gt;Size, name, extension, version, type of the file.&lt;/li&gt;&#xA;&lt;li&gt;Number of users who can access the file.&lt;/li&gt;&#xA;&lt;li&gt;File path.&lt;/li&gt;&#xA;&lt;li&gt;Total number of times accessed.&lt;/li&gt;&#xA;&lt;li&gt;Permitted users.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, I plan to use LSTM but for standard LSTMs, the input is temporal sequence only. However, all the attributes that I have seem significant in predicting the future usage of the file.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How should I also make use of the attributes of the file that I have?&lt;/li&gt;&#xA;&lt;li&gt;Should I train a Feedforward Neural Network, disregarding the fact&#xA;that it usually fails on temporal sequences?&lt;/li&gt;&#xA;&lt;li&gt;How should I proceed?&lt;/li&gt;&#xA;&lt;li&gt;Does a variant of LSTM exist that can take into account the&#xA;attributes of the file as well and predict the usage of the file in&#xA;near future?&lt;/li&gt;&#xA;&lt;li&gt;Do I need to use NN and LSTM together like a hybrid?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16591" LastActivityDate="2018-06-29T08:10:35.707" Title="Deep learning model (LSTM) with temporal and non temporal attributes" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;prediction&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6948" PostTypeId="2" ParentId="6902" CreationDate="2018-06-29T11:05:16.033" Score="1" Body="&lt;p&gt;why do you want to use LSTM network?&#xA;lstm is a variant of a recurrent neural network , recurrent neural networks are used for &quot;sequential&quot; tasks , i.e the dataset should have some sequential structure , like poems , songs etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your model should be a simple classifier , once you fit a simple classifier , like a decision forest on the categories dataset  products ,authors etc , you will have a  model that will predict the class based on these attributes then you can say from the decision boundaries of the model what values you must have . if the relationship between the attributes is even more simpler you could try plotting distribution plots.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-29T11:05:16.033" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6950" PostTypeId="2" ParentId="6916" CreationDate="2018-06-29T12:37:02.203" Score="0" Body="&lt;p&gt;Even if you could do that (which I believe is a long way off), what would be the point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand you correctly, you want an AI system to learn through observation of human beings what their 'rules of interaction' are. It sees a person killing someone else, and then that person is punished by the community, so the AI learns that killing people is not right. However, that is already codified in laws... So what it would pick up are social behaviours, which hopefully be things like &quot;be nice to other people&quot;, &quot;don't do anybody any harm&quot;, &quot;be honest and truthful&quot;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first question then is how an AI could assess events[*] as to whether they are 'good' or 'bad'. If everybody lies and steals, that would be learnt as normal behaviour, and the AI would not be able to pick up that most people would see this as 'bad'[**]. Causal relations are also hard to grasp. Somebody steals something. Then later someone else buys him a drink. So stealing stuff means other people will buy you drinks? These are really hard problems to solve. You need to know about people's motivations, and the multitude of 'threads' of interaction happening at the same time, even in a very limited area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, recognising events and causal links, plus a moral evaluation of them is pretty difficult. I don't think we will get there anytime soon. Unsupervised learning of behavioural is also pretty difficult, as you only have unlabelled observations and no real criteria to classify them. Plus, many actions are morally ambiguous. Killing people is generally seen as bad. What about the officers who tried to assassinate Hitler in 1944? Life is complex, and our artificial models are not anywhere near that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So even if you were able to do all that, what would you end up with? An AI system that has picked up a lot of unwritten rules about human behaviour, and then postulates that as laws? So everyone has to behave the same way? I just don't see the point of that, even as a thought experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[*] leaving aside here the question how you determine what an 'event' is in the first place&lt;br&gt;&#xA;[**] Please note that even if people steal things, they steal can view theft as a bad thing.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-06-29T12:37:02.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6952" PostTypeId="1" CreationDate="2018-06-29T14:09:14.537" Score="7" ViewCount="456" Body="&lt;p&gt;In image classification we are generally told the main reason of using CNN's is that densely connected NN's cannot handle so many parameters (10 ^ 6 for a 1000 * 1000 image). My question is, is there any other reason why CNN's are used over DNN's (densely connected NN)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically if we have infinite resources will DNN trump CNN's or are CNN's inherently well suited for image classification as RNN's are for speech. Answers based either on mathematics or experience on the field is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-06-29T14:20:33.470" LastActivityDate="2018-11-30T07:03:11.833" Title="CNN's vs Densely Connected NN's" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6953" PostTypeId="1" AcceptedAnswerId="6955" CreationDate="2018-06-29T14:31:57.610" Score="7" ViewCount="1337" Body="&lt;p&gt;I am learning about Monte Carlo algorithms and struggling to understand the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If simulations are based on random moves, how can the modeling of the opponent's behavior work well?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, if I have a node with 100 children, 99 of which lead to an instant WIN, whereas the last one leads to an instant LOSS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality, the opponent would never play any of the 99 losing moves for him (assuming they are obvious as they are the last moves), and would always play the winning one. But the Monte Carlo algorithm would still see this node as extremely favorable (99/100 wins for me), because it sees each of the 100 moves as equally probable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding wrong, or does it mean that in most games such situations do not occur and randomness is a good approximation of opponent behavior?&lt;/p&gt;&#xA;" OwnerUserId="16597" LastEditorUserId="1641" LastEditDate="2018-09-22T18:14:58.670" LastActivityDate="2018-09-22T18:14:58.670" Title="Why does Monte Carlo work when a real opponent's behavior may not be random" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6954" PostTypeId="2" ParentId="6953" CreationDate="2018-06-29T15:09:03.767" Score="0" Body="&lt;p&gt;I will point out that the Monte Carlo Tree Search algorithm does not make completely random moves. Instead it usually uses some metric to balance between exploration and exploitation when deciding which branch to search (see Upper Confidence Bound and others).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, you are correct in that a specific line of play which is incredibly troublesome may not be seen and could cause Monte Carlo to make a major mistake. This may have been a cause of AlphaGo losing to Lee Sedol in game 4.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A disadvantage is that, faced in a game with an expert player, there may be a single branch which leads to a loss. Because this is not easily found at random, the search may not &quot;see&quot; it and will not take it into account. It is believed that this may have been part of the reason for AlphaGo's loss in its fourth game against Lee Sedol. In essence, the search attempts to prune sequences which are less relevant. In some cases, a play can lead to a very specific line of play which is significant, but which is overlooked when the tree is pruned, and this outcome is therefore &quot;off the search radar&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="13088" LastEditorUserId="13088" LastEditDate="2018-06-29T15:38:49.243" LastActivityDate="2018-06-29T15:38:49.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6955" PostTypeId="2" ParentId="6953" CreationDate="2018-06-29T15:30:27.467" Score="2" Body="&lt;p&gt;First, we need to distinguish plain &lt;strong&gt;Monte-Carlo&lt;/strong&gt; from &lt;strong&gt;Monte-Carlo Tree Search&lt;/strong&gt;. They're different things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Monte-Carlo search&lt;/strong&gt;, in the context of game AI search algorithms, is typically understood to mean that we search randomly many times, and average the results, and nothing else. If this is all we're doing, then yes, your understanding is correct. This is also sometimes referred to as &quot;plain Monte-Carlo (search)&quot; or &quot;pure Monte-Carlo (search)&quot;, to make it explicitly clear that we're not doing any tree search as in Monte-Carlo Tree Search (sometimes when we just say &quot;Monte-Carlo&quot; in the context of game AI, people will automatically assume Monte-Carlo Tree Search, due to how popular it is).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Monte-Carlo Tree Search&lt;/strong&gt; does a lot more than just that though. It gradually builds up a search tree (through the &lt;em&gt;Expansion&lt;/em&gt; step), and within that search tree (which is growing over time), it uses a much more sophsticated strategy for traversal than pure random (the &lt;em&gt;Selection&lt;/em&gt; step). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example suppose i have a node with 100 children, 99 of which leading to an instant WIN, and the last one leading to an instant LOSS.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Suppose that this node you're talking about, the one with those 100 children, is relatively close to the root node. Then, it is likely that it and all 100 of its children will end up &quot;growing into the search tree&quot; that is slowly built up by the Expansion step. Once they have been added to the search tree, the Selection step will make sure that the vast majority of further iterations visiting this part of the tree will select the instant loss (assuming the opponent is to move in this node). &lt;em&gt;In the limit&lt;/em&gt; (after an infinite amount of search time), this bias towards selecting the loss node will be so large that the average evaluation tends to the (correct) minimax evaluation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another way to view the idea of evaluation through many random sequences of play is the following; the idea is that if we're in a very strong position, a good game state, then we're more likely to win than to lose if both players start playing at random. Consider, for example, a game of chess where Player 1 has many pieces left, and Player 2 only has a few pieces left. Imagine both players were to play completely at random from that game state onwards. On average, which player would you expect to win more often? Probably Player 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we're considering game states that are still far away from terminal states, this basic idea tends to work relatively well. Obviously not always correct, it's still a heuristic, but it can kind of work. When we're already very close to a terminal state that can be reached through one highly specific sequence of play, yeah, we might miss that through random actions; this is where we need the more informed policy of the &lt;em&gt;Selection&lt;/em&gt; step.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-21T08:33:55.060" LastActivityDate="2018-08-21T08:33:55.060" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6956" PostTypeId="2" ParentId="6952" CreationDate="2018-06-29T17:10:42.347" Score="0" Body="&lt;p&gt;That is not the actual reason , &quot;convolution&quot; layers are inspired by cells in visual-system. This is derived from the work of hubel-wiesel.&#xA;for more information check hubel-wiesel experiment.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-29T17:10:42.347" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6957" PostTypeId="5" CreationDate="2018-06-29T19:47:29.073" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-29T19:47:29.073" LastActivityDate="2018-06-29T19:47:29.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6958" PostTypeId="4" CreationDate="2018-06-29T19:47:29.073" Score="0" Body="For questions related to Artificial Intelligence as a profession.  Type of roles (programmer, researcher, etc.) and associated fields of knowledge and skill sets.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-06-29T19:47:29.073" LastActivityDate="2018-06-29T19:47:29.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6959" PostTypeId="1" AcceptedAnswerId="6960" CreationDate="2018-06-29T19:51:58.097" Score="3" ViewCount="171" Body="&lt;p&gt;I built a simple HTML game. In this game the goal is to click when the blue ball is above the red ball. If you hit, you get 1 point, if you miss, you lose 1 point. With each hit, the blue ball moves faster. &lt;a href=&quot;https://codepen.io/iazzetta/pen/BVxzyr&quot; rel=&quot;nofollow noreferrer&quot;&gt;You can test the game here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5L5pQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5L5pQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without using machine learning, I would easily solve this problem by just clicking when the X, Y of the blue ball was on the X, Y of the red ball. Regardless of the time, knowing the positions of the 2 elements I could solve the problem of the game.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, if I wanted to create an AI to solve this problem, could I?&#xA;  How would it be? I'd really like to see the AI randomly wandering&#xA;  until it's perfect.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;My way to solve the problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I click many times and watch score. If score down, add to bad_positions. If actual position in bad_positions, not click. At first he misses many times, then starts to hit eternally. &lt;strong&gt;This is machine learning? Deep learning? Just a bot?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var bad_positions = [];&#xA;function train(){&#xA;  var pos = $ball.offset().left;&#xA;  var last_score = score;&#xA;  if (!bad_positions.includes(pos)) {&#xA;   $('#hit').click();&#xA;    if (score &amp;lt; last_score){&#xA;      bad_positions.push(pos)&#xA;    } &#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="7800" LastEditDate="2018-06-29T20:24:08.943" LastActivityDate="2018-06-29T21:36:48.500" Title="How to use Machine Learning with simple games?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;game-ai&gt;&lt;javascript&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6960" PostTypeId="2" ParentId="6959" CreationDate="2018-06-29T21:36:48.500" Score="1" Body="&lt;p&gt;You have implemented a simple &lt;a href=&quot;https://getstream.io/blog/introduction-contextual-bandits/&quot; rel=&quot;nofollow noreferrer&quot;&gt;contextual bandit&lt;/a&gt; solver, which is a machine learning algorithm. A few details may be different from a full implementation, but the key elements are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A choice of actions (click hit or don't click hit)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A reward signal that can be observed after each action (+1 for a hit, 0 for nothing happens, -1 for an attack which misses)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An observable state which affects the reward achievable (the position of the blue ball). For a contextual bandit, the state is not influenced by the action taken. This is true here.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;One thing that is different about your problem from a classic contextual bandit is that the next state &lt;em&gt;is&lt;/em&gt; predictable from the current state (whilst in a pure bandit problem it should be entirely random). However, that's not too important to your problem here, and your solver is definitely following a contextual bandit approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your solver tests the score from trying different actions in each state, and narrows down the best action to take in each state. Your implementation is simple and &quot;greedy&quot; for a contextual bandit solver. A more typical solution would maintain an average result for each action and have a rule for how to explore actions in each state, so it could test whether results were reliable (this is very helpful with non-deterministic scenarios where bandit solvers are more often used).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With each hit, the blue ball moves faster&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unless you somehow limit the reaction time of the agent, this is not relevant to how you write the solver. You &lt;em&gt;could&lt;/em&gt; change the rules affecting the agent to make it relevant in the same way as it would be for a human, e.g. deciding to click means the click happens 0.1 seconds later, and the state can include observations of position just now and several 0.02 seconds going back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you want to take this further, with more complex games and still learning how to control agent actions, you could look at simple reinforcement learning agents, such as Q-learning. If you are interested in the underlying theory of agents like this, then a good (and free) introductory text is &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto &quot;Reinforcement Learning: An Introduction&quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-06-29T21:36:48.500" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6961" PostTypeId="1" CreationDate="2018-06-30T07:03:48.363" Score="6" ViewCount="2655" Body="&lt;p&gt;I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you!&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For units/neurons in the hidden layers, they are referred to as memory blocks and each memory block can contain multiple memory cells?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And one memory cell looks like these two?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/38Szh.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/38Szh.png&quot; alt=&quot;LSTM Memory Cell&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VAFik.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VAFik.png&quot; alt=&quot;LSTM Memory Cell&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;For units/neurons in the input and output layers, they are simply regular neurons? Like those in a feedforward network?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Regarding the network structure, the structure of an RNN essentially looks like that of a feedforward network, except the neurons in the hidden layer have been replaced by neurons with recurrent connections. Is this an accurate description?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;LSTM RNNs then replaces those neurons in the hidden layers with memory blocks (which contain memory cells that have recurrent connections). Is this an accurate description?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16609" LastEditorUserId="9062" LastEditDate="2018-07-06T18:11:29.120" LastActivityDate="2019-01-17T12:43:05.633" Title="Structure of LSTM RNNs" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ClosedDate="2020-06-23T23:09:12.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="6962" PostTypeId="2" ParentId="6855" CreationDate="2018-06-30T11:51:18.480" Score="2" Body="&lt;p&gt;The Mac Pro uses AMD GPUs. These don't support CUDA, but instead support the OpenCL framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TensorFlow, the most popular deep learning library, uses CUDA to run on GPUs, although OpenCL &lt;a href=&quot;https://github.com/tensorflow/tensorflow/issues/22&quot; rel=&quot;nofollow noreferrer&quot;&gt;support is in the works&lt;/a&gt;. That said, one of the main approaches to OpenCL support, SYCL, isn't planning to support OSX:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We have no plans to support OSX in near future. I know sad face.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.slavv.com/picking-a-gpu-for-deep-learning-3d4795c273b9&quot; rel=&quot;nofollow noreferrer&quot;&gt;The current advice is to avoid AMD GPUs&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Currently, if you want to do DL and want to avoid major headaches, choose Nvidia.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16612" LastEditorUserId="4709" LastEditDate="2019-04-22T19:57:05.173" LastActivityDate="2019-04-22T19:57:05.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6965" PostTypeId="1" CreationDate="2018-06-30T17:37:49.283" Score="2" ViewCount="1466" Body="&lt;p&gt;My question is that is there any general idea on how humans solve jumbled words? I know many people will say we match it against a commonly used words checklist mentally, but it is kind of vague. Is there any theory on this and how might an AI learn to do the same?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-06-30T18:23:40.640" Title="Can AI solve jumbled words?" Tags="&lt;natural-language-processing&gt;&lt;world-knowledge&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6966" PostTypeId="2" ParentId="6965" CreationDate="2018-06-30T18:23:40.640" Score="3" Body="&lt;p&gt;I remember a problem similar to this  &lt;a href=&quot;https://vladris.com/puzzles/facebook/puzzle_master/snack/breathalyzer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://vladris.com/puzzles/facebook/puzzle_master/snack/breathalyzer.html&lt;/a&gt;&#xA;this is given as a problem in facebook engineering puzzles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But i doubt that it is not much of a machine learning/AI problem. you could implement an algorithm that converts each word into a set of its characters , then pick the word in your master list with minimum distance based on its character list .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even when humans solve jumble we do it in a systematic/algorithmic way , if the jumbled word is not present in our memory we can't do anything otherwise we can solve the jumble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But human brain can simply recognize scrabled words if some of the structure is retained and not fully scrambled like an anagram .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We generally index words based on the first syllable/character in our brain , like &lt;a href=&quot;https://i.stack.imgur.com/9jXil.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9jXil.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; or even this &lt;a href=&quot;https://i.stack.imgur.com/SifPr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SifPr.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second picture we recognised the words even if they are not made of english characters because our brain doesn't scan the text character by character like &quot;7-H &quot; but treats it like an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the model should not immediately classify the segmented characters but should find the &quot;nearest&quot; characters in every class that optimises the combined probability of characters of the word being one of the classes of words we have in our dictionary.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-06-30T18:23:40.640" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6967" PostTypeId="2" ParentId="6139" CreationDate="2018-06-30T18:39:53.747" Score="2" Body="&lt;p&gt;You can use a neural network for this (sort of), and you've got the first step right - train the network to predict $y$ given $x$ (i.e. train it to approximate the function $f$ such that $y = f(x)$). Because the entire neural network is differentiable (presumably), you can take gradients of the input (some $x$) with respect to the predicted output. Then you can use this to update $x$ the same way that you update the weights during training of the network. This let you find a local optimum from $x$. I don't know of a way to try to find a global optimum with a neural network like this except to find the local optimum near many $x$'s and then take the best of those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a concrete example, take a look at the tutorial on &lt;a href=&quot;https://pytorch.org/tutorials/advanced/neural_style_tutorial.html#sphx-glr-advanced-neural-style-tutorial-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Style Transfer with PyTorch&lt;/a&gt;: here you have a noise image as the input and you optimize it to minimize the &quot;style distance&quot; to a reference style image and the &quot;content distance&quot; to a reference content image (i.e. starting with noise, make it look like the content of one image but in the style of another). There's full code there, but here's a short PyTorch snippet that shows the main idea:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# I'm just optimizing one input; clone so that you don't modify the&#xA;# original tensor; let it know that we want gradients computed&#xA;inputs = train_inputs[0:1].clone().requires_grad_()&#xA;input_optimizer = torch.optim.Adam([inputs])&#xA;&#xA;def optimize_input():&#xA;    input_optimizer.zero_grad()&#xA;    # where `model` is your trained neural network&#xA;    output = model(inp)&#xA;    (-output).backward()&#xA;    return output&#xA;&#xA;output = []&#xA;for _ in range(10000):&#xA;    output.append(input_optimizer.step(optimize_input).detach().numpy())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;output&lt;/code&gt; is a list at the end of what the predicted output was after each step of optimizing the input. You can use this to see if you've converged to a local optimum yet (and take more steps/fewer next time as necessary). &lt;code&gt;inputs&lt;/code&gt; will be the optimized input at the end. Note that if there are constraints that your input should satisfy, you'll need to enforce those yourself (e.g. in neural style transfer, they have to enforce that the values are valid for an RGB image).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that how well this works really depends on how well your network approximates $f$; you may well get some unreasonable $x$ for which the network predicts a very large $y$ because there was no training data similar to that $x$ so the network isn't constrained in that region. In general, you should probably be cautious of an $x$ you generate this way that doesn't seem similar to your training data (e.g. has larger/smaller value than any $x$ the network has seen before; interpolation is much easier than extrapolation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming that you have a single data set; if you're able to query for the $y$ values of given $x$'s, then you might want to take a look at Bayesian optimization - essentially the field of trying to find $x$ that maximizes $y = f(x)$ when $f$ is expensive to evaluate and you don't have gradients of it. Bayesian optimization seeks a &lt;em&gt;global&lt;/em&gt; optimum.&lt;/p&gt;&#xA;" OwnerUserId="16606" LastActivityDate="2018-06-30T18:39:53.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6968" PostTypeId="1" CreationDate="2018-07-01T04:52:30.513" Score="3" ViewCount="1056" Body="&lt;p&gt;What does it mean when it is said that Machine Learning algorithm results can be &quot;generalized&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand what &quot;generalized&quot; algorithms, routines or functions are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have searched dictionaries and glossaries, and cannot find an explanation. Also, if anyone can tell me where a good source for this type of thing is? I am writing about AI and ML.&lt;/p&gt;&#xA;" OwnerUserId="13053" LastEditorUserId="9947" LastEditDate="2018-07-01T05:22:35.250" LastActivityDate="2019-07-01T20:16:09.813" Title="What is a &quot;generalized&quot; machine learning algorithm?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6969" PostTypeId="2" ParentId="6968" CreationDate="2018-07-01T04:56:46.943" Score="0" Body="&lt;p&gt;A machine learning model is said to &quot;generalise&quot; when it performs equally well on both train and test datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any supervised machine learning algorithm to work well, you train it on a large dataset (train) and evaluate its performance on a dataset whose probability distribution is similar to that of train set but not a part of the train set. This set is called &quot;test&quot; set , then the performance is evaluated on this set, if both train and test accuracies are almost same then the model is said to generalise well, if the training accuracy is much higher than that of test accuracy then the model in some way &quot;memorises&quot; the train set and called &quot;overfitting&quot;.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="12853" LastEditDate="2019-07-01T20:16:09.813" LastActivityDate="2019-07-01T20:16:09.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6970" PostTypeId="2" ParentId="6961" CreationDate="2018-07-01T05:09:00.143" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;em&gt;For units/neurons in the hidden layers, they are referred to as memory blocks and each memory block can contain multiple memory cells?&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;yes each lstm unit contains multiple memory cells.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;And one memory cell looks like these two&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0UoeK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0UoeK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;in the diagram each cell is one lstm unit , this did not show forget gates though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.&lt;em&gt;For units/neurons in the input and output layers, they are simply regular neurons? Like those in a feedforward network?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;yes they are regular feed forward connected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.*Regarding the network structure, the structure of an RNN essentially looks like that of a feedforward network, except the neurons in the hidden layer have been replaced by neurons with recurrent connections. Is this an accurate description?&#xA;*&lt;/p&gt;&#xA;&#xA;&lt;p&gt;yeah but this is for vanilla rnn in lstm/gru variants gating mechanism is introduced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5 *LSTM RNNs then replaces those neurons in the hidden layers with memory blocks (which contain memory cells that have recurrent connections). Is this an accurate description?&#xA;*&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the main motivation behind this is to eradicate something known as &quot;vanishing/exploding&quot; gradient problem . which tends to either make the gradients explode or make them zero as the gradients flow through the layers.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-01T05:09:00.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6971" PostTypeId="2" ParentId="6968" CreationDate="2018-07-01T05:12:58.683" Score="1" Body="&lt;p&gt;Actually you have used 2 terminologies there:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first one is that  Machine Learning algorithm results can be &quot;generalised&quot;. This refers to how well your trained Machine Learning model will perform on previously unseen data (test set or implemented on field). This is particularly not easy as data trends may change over time resulting in loss of accuracy. There are various methods to implement this like (having a cross validation set and a test set, which comes under the broad scheme of &lt;a href=&quot;https://machinelearningmastery.com/k-fold-cross-validation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;k-fold cross validation&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;The second you mentioned is '&quot;generalised&quot; algorithms, routines or functions'. Most Machine Learning algorithms can be applied to a broad range of problems. For example the training of a NN is generally done by backprop which is universally applied to all NN's. Similarly, you can use CNN to find features of local interest (i.e. local dependencies) in anything that can be represented in a pictorial form (strings of DNA). Also combinations of CNN and RNN are being used to solve many problems. Thus, only a basic generalised algorithm is being applied to a lot of problems. NOTE: I have never seen any one use it in this context, but practically it happens.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are a few resources for general reading purposes (not mathematical):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overfitting and Underfitting With Machine Learning Algorithms&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.coursera.org/lecture/big-data-machine-learning/generalization-and-overfitting-tOvMb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization and Overfitting&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-07-01T05:21:00.757" LastActivityDate="2018-07-01T05:21:00.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6972" PostTypeId="1" AcceptedAnswerId="6974" CreationDate="2018-07-01T06:59:12.860" Score="1" ViewCount="150" Body="&lt;p&gt;I want to train a CNN (Vggnet) to identify different types of buildings from aerial images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However seeing that a CNN &quot;ignores&quot; size, e.g. the same type of dog in one image can be large and small in another image but will still be classified as a dog.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My issue is that non-residential buildings are mostly larger than residential houses, now I want to use this property to distinguish between residential and non residential. Is this even possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5hV2J.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5hV2J.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/e5T9L.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e5T9L.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="16628" LastActivityDate="2018-07-01T10:35:48.490" Title="Using CNN to identify buildings from aerial images" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6974" PostTypeId="2" ParentId="6972" CreationDate="2018-07-01T10:35:48.490" Score="0" Body="&lt;p&gt;You need to know the basic mechanics of CNN , you can't simply finetune a pretrained net (VGGNET) everytime. VGGnet is trained on imagenet dataset so most of the features in it are not so relevant for your task (recognising buildings from aerial view). so it is much better to train a cnn from scratch. You can't simply say that it fails to classify residential , non-residential . It is best to find what could be done by trying,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My issue is that non-residential buildings are mostly larger than residential houses, now I want to use this property to distinguish between residential and non residential.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the times it is better to observe the features recognised by the net and then come with an approach , rather than enforcing your views.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may find this helpful &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/Mnih_Volodymyr_PhD_Thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.toronto.edu/~vmnih/docs/Mnih_Volodymyr_PhD_Thesis.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-01T10:35:48.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6975" PostTypeId="1" CreationDate="2018-07-01T12:55:29.650" Score="3" ViewCount="132" Body="&lt;p&gt;&lt;em&gt;While am thinking about the age of &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial SuperIntelligence&lt;/a&gt;&lt;/em&gt; , this question came into my mind.&#xA;This &lt;a href=&quot;https://www.bell-labs.com/var/articles/artificial-intelligence-new-frontier/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bell Labs blog post&lt;/a&gt; says: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;...where humanity is displaced by the self-aware robots that rule the world...&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Humans tend to have a sense of self (self awareness), and this works effectively at maturity. However, at the young stage, a human child lacks a sense of self, so the child is guided by the environment; learning from it as time goes on. Also at this same stage, the child faces some consequences, for example:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;No sense of direction or transition.&lt;/li&gt;&#xA;&lt;li&gt;No sense of purpose, etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;At the old stage (the natural being has grown up), its sense of self tries to generate its priorities, for example:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;knows when to stand.&lt;/li&gt;&#xA;&lt;li&gt;where to go, as its goals are set.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is my concern or point of view concerning artificial life:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can an Artificial Being, lacking self-awareness, learn from its environment just like natural beings do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Any references/papers/theories will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2193" LastEditDate="2018-07-02T20:10:44.550" LastActivityDate="2018-08-31T21:02:08.447" Title="Can Artificial Being learn despite a lack of self-awareness?" Tags="&lt;intelligent-agent&gt;&lt;reference-request&gt;&lt;sense&gt;&lt;self-awareness&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2021-01-18T04:24:07.470" ContentLicense="CC BY-SA 4.0" />
  <row Id="6976" PostTypeId="2" ParentId="5461" CreationDate="2018-07-01T13:23:44.763" Score="0" Body="&lt;p&gt;No , even generating brief summary of a video is out of scope as of current state of the art. Training such a model is a tough task. Video understanding is pretty far from the research as of now. &#xA;But you could try generating descriptions of some keyframes of the video , and align them to form meaningful passages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;check this &lt;a href=&quot;https://arxiv.org/abs/1611.06607&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1611.06607&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-01T13:23:44.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6977" PostTypeId="1" CreationDate="2018-07-01T15:18:13.480" Score="2" ViewCount="118" Body="&lt;p&gt;I want to detect drivers with or without seatbelts at crossroads. For that, as it is real-time, I am going to use the YOLO algorithm/model. For training data sets (the images) I need to collect, I placed a camera. By recording it and collecting images from there, I am getting images with more noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use these images for training? Also, which YOLO version should I use? What are the important points that I should consider for training datasets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use any version of YOLO compatible with TensorFlow.&lt;/p&gt;&#xA;" OwnerUserId="16633" LastEditorUserId="2444" LastEditDate="2020-06-15T01:33:24.870" LastActivityDate="2020-06-15T01:33:24.870" Title="What YOLO algorithm can I use for images with noise as I will implement it in real time?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6978" PostTypeId="1" CreationDate="2018-07-01T15:41:13.983" Score="6" ViewCount="153" Body="&lt;p&gt;I just stumbled across the paper &lt;a href=&quot;https://arxiv.org/pdf/1705.08807.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;When Will AI Exceed Human Performance?&#xA;Evidence from AI Experts&lt;/a&gt;, which contains a figure showing the aggregated subjective probability of &quot;high-level machine intelligence&quot; arrival by future years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g1Gxo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g1Gxo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if this graph reflects the opinion of experts, it can be totally wrong. It is just extremely hard to predict future events. So I was wondering if there is a similar graph which shows basically the same but for the game Go? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the complexity of Go, some experts assumed, that no computer ever could be better in Go than a human being due to the lack of &lt;em&gt;intuition&lt;/em&gt;. This shows that the appearance of human level AI can be unpredictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if a similar graph for Go exists to see how good or bad the predictions were? This could give a very rough idea, how good this graph predicts the future of human-level AI.&lt;/p&gt;&#xA;" OwnerUserId="16634" LastEditorUserId="2444" LastEditDate="2020-02-14T01:47:59.297" LastActivityDate="2020-02-14T01:47:59.297" Title="Are there human predictions of when a computer would have been better than a human at Go?" Tags="&lt;agi&gt;&lt;papers&gt;&lt;go&gt;&lt;ai-milestones&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6982" PostTypeId="1" AcceptedAnswerId="6983" CreationDate="2018-07-02T07:47:23.303" Score="14" ViewCount="2828" Body="&lt;p&gt;I was going through &lt;a href=&quot;https://github.com/transedward/pytorch-dqn/blob/master/dqn_learn.py&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; implementation of DQN and I see that on line 124 and 125 two different Q networks have been initialized. From my understanding, I think one network predicts the appropriate action and the second network predicts the target Q values for finding the Bellman error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why can we not just make one single network that simply predicts the Q value and use it for both the cases? My best guess that it's been done to reduce the computation time, otherwise we would have to find out the q value for each action and then select the best one. Is this the only reason? Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="11584" LastEditorUserId="2444" LastEditDate="2020-11-04T21:33:25.933" LastActivityDate="2020-11-04T21:33:25.933" Title="Why does DQN require two different networks?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6983" PostTypeId="2" ParentId="6982" CreationDate="2018-07-02T09:12:24.587" Score="10" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My best guess that it's been done to reduce the computation time, otherwise we would have to find out the q value for each action and then select the best one. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It has no real impact on computation time, other than a slight increase (due to extra memory used by two networks). You &lt;em&gt;could&lt;/em&gt; cache results of the target network I suppose, but it probably would not be worth it for most environments, and I have not seen an implementation which does that.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I missing something?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is to do with stability of the Q-learning algorithm when using function approximation (i.e. the neural network). Using a separate target network, updated every so many steps with a copy of the latest learned parameters, helps keep runaway bias from bootstrapping from dominating the system numerically, causing the estimated Q values to diverge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine one of the data points (at &lt;span class=&quot;math-container&quot;&gt;$s, a, r, s'$&lt;/span&gt;) causes a currently poor over-estimate for &lt;span class=&quot;math-container&quot;&gt;$q(s', a')$&lt;/span&gt; to get worse. Maybe &lt;span class=&quot;math-container&quot;&gt;$s', a'$&lt;/span&gt; has not even been visited yet, or the values of &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; seen so far is higher than average, just by chance. If a sample of &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; cropped up multiple times in experience replay, it would get worse again each time, because the update to &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; is based on the TD target &lt;span class=&quot;math-container&quot;&gt;$r + \text{max}_{a'} q(s',a')$&lt;/span&gt;. Fixing the target network limits the damage that such over-estimates can do, giving the learning network time to converge and lose more of its initial bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this respect, using a separate target network has a very similar purpose to experience replay. It stabilises an algorithm that otherwise has problems converging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to have DQN with &quot;double learning&quot; to address a separate issue: Maximisation bias. In that case you may see DQN implementations with 4 neural networks.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-09T16:37:45.657" LastActivityDate="2020-06-09T16:37:45.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6984" PostTypeId="2" ParentId="6977" CreationDate="2018-07-02T12:46:06.317" Score="1" Body="&lt;p&gt;It is much better to know basic mechanics of convnets first ,rather than diving straight into complicated models .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;For training data sets (the images) I need to collect, I placed a camera. By recording it and collecting images from there, I am getting images with more noise. Can I use these images for training? Also, which yolo version should I use? What are the important points that I should consider for training datasets?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you are good with the theory part most of your questions will be answered , otherwise you would endup with nothing but buzzwords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I want to use any version of yolo compatible with tensorflow.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow is a framework for building neural networks , so in theory you can build any network with it so compatibility is not at all a problem.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="15935" LastEditDate="2018-07-03T04:26:43.007" LastActivityDate="2018-07-03T04:26:43.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6985" PostTypeId="1" CreationDate="2018-07-02T12:51:39.223" Score="2" ViewCount="91" Body="&lt;p&gt;In the paper &quot;&lt;a href=&quot;https://arxiv.org/abs/1310.6343&quot; rel=&quot;nofollow noreferrer&quot;&gt;Provable bounds for learning some deep representations&lt;/a&gt;&quot;, an autoencoder like a model is constructed with discrete weights and several results are proven using some random-graph theory, but I never saw any papers similar to this. i.e bounds on neural networks using random graph assumptions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What are some resources (e.g. books or papers) regarding the time and space complexity of training neural networks?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm particularly interested in convolutional neural networks.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2019-06-27T23:32:02.440" LastActivityDate="2019-06-27T23:35:14.717" Title="What are some resources regarding the complexity of training neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;time-complexity&gt;&lt;complexity-theory&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6986" PostTypeId="1" CreationDate="2018-07-02T14:45:16.303" Score="2" ViewCount="30" Body="&lt;p&gt;I am starting to study the capabilities of neural networks for the reconstruction/restoration/... of communication signals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am feeding my neural network with a signal which has some parts which have been damaged because of the transmission through a communication system, and my targets are given by the signal with these areas undamaged. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the areas damaged represent a very small portion of the whole signal, and my neural network spends lot of time learning only from the portions which actually do not present any problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any solution to make the neural network to jump on those areas which show significant differences to respect to the targets? Is there anything I could do for example initializing my neural networks (as conventionally done, they are initialized randomly)? Or shall I accept that I need to train for longer time?&lt;/p&gt;&#xA;" OwnerUserId="16654" LastActivityDate="2018-07-02T16:03:54.123" Title="Restoration of localized damaged areas (time signals, but guess also applicable to images)" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6987" PostTypeId="2" ParentId="6986" CreationDate="2018-07-02T16:03:54.123" Score="1" Body="&lt;p&gt;So , you want to remove the &quot;noise&quot; (damaged signal) from the signal, by giving the perfect signal as output. This model is called as a &quot;denoising autoencoder&quot;. If you have enough training data, the noise is not at all a problem for the neural network , if you really feel that the network is capturing the noise , then you have to increase the training epochs , or sometimes it can be a bad initialized network. &lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-02T16:03:54.123" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6988" PostTypeId="2" ParentId="1485" CreationDate="2018-07-02T16:08:01.350" Score="0" Body="&lt;p&gt;That depends on what type of network you want to use for your second network, instead of feeding the outputs of the first layer, it would be much better if you jointly train both the networks. But that depends on the architecture of the second network ('logic' network).&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="8" LastEditDate="2018-07-02T21:02:21.280" LastActivityDate="2018-07-02T21:02:21.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6989" PostTypeId="2" ParentId="6978" CreationDate="2018-07-02T20:08:32.503" Score="4" Body="&lt;p&gt;Go predictions were included in the paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/92bLA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/92bLA.png&quot; alt=&quot;FIT AI Prediction Survey, 2015&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The experts are far from infallible. They predicted that AI would be better than humans at Go by about 2027. (This was in 2015, remember.) &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://www.technologyreview.com/s/607970/experts-predict-when-artificial-intelligence-will-exceed-human-performance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Experts Predict When Artificial Intelligence Will Exceed Human Performance&lt;/a&gt; (MIT Tech Review)&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-07-02T20:08:32.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6990" PostTypeId="1" AcceptedAnswerId="6999" CreationDate="2018-07-03T02:56:44.167" Score="3" ViewCount="2058" Body="&lt;p&gt;While reading about least squares implementation for machine learning I came across this passage in the following two photos:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rbHEv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rbHEv.jpg&quot; alt=&quot;photo1&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FR8vr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FR8vr.jpg&quot; alt=&quot;photo 2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I’m misinterpreting the meaning of beta  but if X^T has dimension 1 x p and beta has dimension p x K, then hat{Y}  would have dimension 1 x K and would be a row vector. According to the text, vectors are assumed column vectors unless otherwise noted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone provide clarification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: the matrix notation in this text confuses me. The pages preceding the above passage read:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RGXdp.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RGXdp.jpg&quot; alt=&quot;3rd&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/47qQn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/47qQn.jpg&quot; alt=&quot;4th&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should the matrix referenced not have dimensions p x N, assuming a p-vector is a column vector with p elements?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The passage is taken from “Elements of Statistical Learning” by Hastie, Tibshirani, &amp;amp; Friedman.  &lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2018-07-03T03:42:45.420" LastActivityDate="2018-07-03T14:12:32.950" Title="Matrix Dimension for Linear regression coefficients" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6991" PostTypeId="2" ParentId="6990" CreationDate="2018-07-03T03:49:51.887" Score="1" Body="&lt;p&gt;I have used &lt;code&gt;W&lt;/code&gt; as &lt;code&gt;beta&lt;/code&gt; and &lt;code&gt;Y_pred&lt;/code&gt; as &lt;code&gt;Y_hat&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, as far as conventions go both feature vector &lt;code&gt;X&lt;/code&gt; and weight vector &lt;code&gt;W&lt;/code&gt; are assumed as column vectors. Although this is not important in your case but when we use Neural Nets, this is particularly important, and a weight vector for a layer is given by &lt;code&gt;p*N_n&lt;/code&gt; where &lt;code&gt;p&lt;/code&gt; is the number of features being input from the previous layer and &lt;code&gt;N_n&lt;/code&gt; is the number of nodes in the layer. Check a NN structure and you will understand what I am saying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the special case in your current question, the book is correct, although comments say &lt;code&gt;Y_predicted&lt;/code&gt; should be vector, it is not, since we are doing dot product between &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;W&lt;/code&gt; (although it is represented in a vectorised form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also  how can &lt;code&gt;W&lt;/code&gt;  have dimensions &lt;code&gt;p*K&lt;/code&gt; (assuming it is not a NN), it means multiple solutions to the same problem? Instead it is &lt;code&gt;X&lt;/code&gt; which will have dimensions &lt;code&gt;p*K&lt;/code&gt; and &lt;code&gt;W&lt;/code&gt; will have &lt;code&gt;p*1&lt;/code&gt;. It means you have &lt;code&gt;K&lt;/code&gt; training examples with &lt;code&gt;p&lt;/code&gt; features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If its a NN with &lt;code&gt;K&lt;/code&gt; nodes then dimensions for &lt;code&gt;X&lt;/code&gt; like &lt;code&gt;p*N&lt;/code&gt; and &lt;code&gt;W&lt;/code&gt; as &lt;code&gt;p*K&lt;/code&gt; is totally possible. The only downside of this notation is that you have to again take a transpose of the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the explanation of the notation used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.coursera.org/learn/neural-networks-deep-learning/lecture/Z8j0R/binary-classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;Notation for NN&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-07-03T04:08:48.557" LastActivityDate="2018-07-03T04:08:48.557" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="6993" PostTypeId="2" ParentId="2637" CreationDate="2018-07-03T06:29:00.720" Score="6" Body="&lt;p&gt;In the basic form, if you encounter a terminal leaf, you add visits and score depending on whether it is a win or loss, and backpropagate accordingly. The same as if you made a simulation step, but in this case the &quot;simulation&quot; is instant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you can improve that: If the leaf is losing, you can give it a very large negative score or even &lt;span class=&quot;math-container&quot;&gt;$-\infty$&lt;/span&gt;, so in the next selection step it surely won't be chosen, unless other moves are as bad. But if it is a winning leaf, you not only can give it a very big positive score or &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;, but also add a negative score for the immediate parent, so the parent won't be chosen, as it is obviously a losing state for him. This way we can save some simulations. I had encountered that situation many times in my game and Monte Carlo tree searches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the parent has &lt;span class=&quot;math-container&quot;&gt;$200$&lt;/span&gt; unexplored children, &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; of which are immediate wins for the opponent. Your search may explore &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; non-winning children and have a score of 80/100. But then it encounters the terminate child. After that, normally, it would have &lt;span class=&quot;math-container&quot;&gt;$80/101$&lt;/span&gt;, so, still, a big chance to be chosen in the next iteration. And it would take many iterations to see that this is not a good move, as it would need to get like &lt;span class=&quot;math-container&quot;&gt;$80/150$&lt;/span&gt; or more. But if we cancel out the score or give it a negative one like &lt;span class=&quot;math-container&quot;&gt;$-1/101$&lt;/span&gt;, then we ensure it won't be chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems in literature it's called &quot;MCTS solver&quot;, to backpropagate proven wins and losses.&lt;/p&gt;&#xA;" OwnerUserId="16663" LastEditorUserId="2444" LastEditDate="2019-11-18T21:47:19.723" LastActivityDate="2019-11-18T21:47:19.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6994" PostTypeId="1" CreationDate="2018-07-03T10:59:23.103" Score="4" ViewCount="71" Body="&lt;p&gt;I have a dataset of unlabelled emails that fall into distinct categories (around a dozen). I want to be able to classify them along with new ones to come in the future in a dynamic matter. I know that there are dynamic clustering techniques that allow the clusters to evolve over time ('dynamic-means' being one of them). However, I would also like to be able to start with a predefined set of classes (or clusters/centroids) as I know for a fact what the types of those emails will be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I need some guidance in terms of what vectorisation technique to use for my type of data. Would creating a term matrix using TF-IDF be sufficient? I assume that the data I am dealing with could be differentiated on the basis of keyword occurrence, but I cannot tell to what degree. Are there more sophisticated vectorisation techniques based more on the text semantics? Are they worth exploring?&lt;/p&gt;&#xA;" OwnerUserId="16669" LastActivityDate="2018-10-08T05:49:08.947" Title="What techniques to explore for dynamic clustering of documents (emails)?" Tags="&lt;unsupervised-learning&gt;&lt;soft-question&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6995" PostTypeId="1" CreationDate="2018-07-03T11:13:09.053" Score="2" ViewCount="28" Body="&lt;p&gt;In the below pic, I can not understand what &lt;code&gt;U&lt;/code&gt; vector is? It says &lt;code&gt;flow field&lt;/code&gt; but I can not imagie what really is the flow field?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wSMqN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wSMqN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9941" LastEditorUserId="1671" LastEditDate="2018-07-03T16:23:38.280" LastActivityDate="2018-07-03T16:23:38.280" Title="Simple question about HS algorithm's formul(Optical flow)" Tags="&lt;algorithm&gt;&lt;math&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6996" PostTypeId="1" CreationDate="2018-07-03T12:15:24.907" Score="2" ViewCount="170" Body="&lt;p&gt;Imagine that a line divides an image in two regions which (slightly) differ in terms of texture and color. It is not a perfect, artificial line but rather a thin transition zone. I want to build a neural network which is able to infer geometrical information on this line (orientation and offset). The image may also contain other elements which are not relevant for the task. Now, would a classical CNN be suitable for this task? How complex should it be in terms of number of convolutions (and number of layers, in general)?&lt;/p&gt;&#xA;" OwnerUserId="16671" LastActivityDate="2018-07-05T12:18:36.390" Title="Neural network architecture for line orientation prediction" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6997" PostTypeId="1" AcceptedAnswerId="7006" CreationDate="2018-07-03T13:43:24.843" Score="8" ViewCount="561" Body="&lt;p&gt;I am familiar with supervised and unsupervised learning. I did the SaaS course done by Andrew Ng on Coursera.org.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for something similar for reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you recommend something?&lt;/p&gt;&#xA;" OwnerUserId="16672" LastEditorUserId="1671" LastEditDate="2018-07-03T16:22:46.547" LastActivityDate="2019-09-20T13:04:46.477" Title="What's a good resource for getting familiar with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;getting-started&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6998" PostTypeId="2" ParentId="6997" CreationDate="2018-07-03T14:03:51.340" Score="4" Body="&lt;p&gt;Before that ask yourself if you really want to learn about &quot;reinforcement learning.&quot; Although there is much hype about reinforcement learning, the real-world applicability of reinforcement learning is almost non-existent. Most of the online courses teach you a very little about machine learning, so it is much better to get thorough with it, rather than proceeding towards reinforcement learning. Learning reinforcement learning is somewhat different from learning about unsupervised/supervised learning techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, the fastest way to get a good grasp of reinforcement learning is as follow:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Read &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrej Karpathy's blog post &quot;Pong from Pixels.&quot;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Watch &lt;a href=&quot;https://sites.google.com/view/deep-rl-bootcamp/lectures&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep RL Bootcamp lectures&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To understand the math behind these techniques, refer to &lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's &lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Read relevant papers (game-playing etc.).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;P.S: Make sure that you are thorough with basics of neural networks, as most of the current papers in RL involve using DNNs in some or the other way as approximators.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="16987" LastEditDate="2018-07-19T18:48:17.997" LastActivityDate="2018-07-19T18:48:17.997" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6999" PostTypeId="2" ParentId="6990" CreationDate="2018-07-03T14:12:32.950" Score="1" Body="&lt;p&gt;This is a good example of what happens when you take text out of context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The passage that was added in the edited question makes a difference, but it's not quite sufficient, and it doesn't help that the notation is all over the place. I found the textbook and the relevant passages (Section 2.3, p. 10-11). Here is a quick attempt at an explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors call &lt;em&gt;X&lt;/em&gt; a &lt;strong&gt;variable&lt;/strong&gt; (either scalar or vector) with &lt;strong&gt;components&lt;/strong&gt; &lt;em&gt;X&lt;sub&gt;j&lt;/sub&gt;&lt;/em&gt;, but later in the paragraph they refer to &lt;em&gt;X&lt;sub&gt;j&lt;/sub&gt;&lt;/em&gt; as a &lt;strong&gt;variable&lt;/strong&gt; (which I think is the correct notation). Instead of &quot;variable&quot; with &quot;components&quot;, think of &lt;em&gt;X&lt;/em&gt; as a set of &lt;em&gt;p&lt;/em&gt; variables (in the normal sense of a variable, such as temperature, the price of shares, etc.). You can put multiple variables &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; (&lt;em&gt;i = 1...p&lt;/em&gt;) in a vector &lt;em&gt;X&lt;/em&gt; and call each &lt;em&gt;instance&lt;/em&gt; of &lt;em&gt;X&lt;/em&gt; an &lt;em&gt;observation&lt;/em&gt;. In other words, an observation is a set of measured values for &lt;strong&gt;all&lt;/strong&gt; variables &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you have made &lt;em&gt;N&lt;/em&gt; observations. Arrange your observations in a matrix with &lt;em&gt;N&lt;/em&gt; rows and &lt;em&gt;p&lt;/em&gt; columns, where each row represents a single observation (an &lt;em&gt;instance&lt;/em&gt; of &lt;em&gt;X&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now also assume that you are trying to find the relation between your &lt;strong&gt;input&lt;/strong&gt; variables &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; and a &lt;em&gt;different&lt;/em&gt; set of variables &lt;em&gt;y&lt;sub&gt;k&lt;/sub&gt;&lt;/em&gt;, called &lt;em&gt;dependent&lt;/em&gt; or &lt;em&gt;response&lt;/em&gt; variables. In general, the variables &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; and &lt;em&gt;y&lt;sub&gt;k&lt;/sub&gt;&lt;/em&gt; are &lt;strong&gt;measured&lt;/strong&gt;, and the model is simply trying to extract the relation between the input and dependent variables so you can then &lt;em&gt;predict&lt;/em&gt; the latter from the former.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note, observations are usually denoted with &lt;strong&gt;superscripts&lt;/strong&gt; (&lt;em&gt;x&lt;sup&gt;(n)&lt;/sup&gt;&lt;/em&gt;) and variables with &lt;strong&gt;subscripts&lt;/strong&gt; (&lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt;) so there is no confusion about which is which. &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;sup&gt;(n)&lt;/sup&gt;&lt;/em&gt; is the &lt;em&gt;n&lt;/em&gt;&lt;sup&gt;th&lt;/sup&gt; observed (measured) value of variable &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you have a single dependent variable &lt;em&gt;y&lt;/em&gt; and &lt;em&gt;p&lt;/em&gt; input variables &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; (&lt;em&gt;i = 1...p&lt;/em&gt;). Assuming that their relation is linear (note: in many cases this assumption is &lt;em&gt;not&lt;/em&gt; justified), we can assign weights (&quot;importance&quot;) to each variable and try to find out those weights from measurements. In your case, the weights are denoted with &lt;em&gt;beta&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; (so the &quot;importance&quot; of variable &lt;em&gt;x&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; is &lt;em&gt;beta&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt;; note the same subscript).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have &lt;em&gt;N&lt;/em&gt; observations of your dependent variable &lt;em&gt;y&lt;/em&gt;, then you can arrange them in a &lt;strong&gt;column&lt;/strong&gt;, just like the observations for your input variables. Note that we still have a &lt;em&gt;single&lt;/em&gt; dependent variable, so essentially &lt;em&gt;y&lt;/em&gt; is a &lt;strong&gt;scalar&lt;/strong&gt;, but the &lt;em&gt;N&lt;/em&gt; &lt;em&gt;observations&lt;/em&gt; of that scalar form an &lt;em&gt;N&lt;/em&gt;-dimensional &lt;em&gt;column&lt;/em&gt; vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now multiply the &lt;em&gt;N x p&lt;/em&gt; input matrix by the &lt;em&gt;p&lt;/em&gt;-dimensional &lt;strong&gt;column&lt;/strong&gt; vector of weights &lt;em&gt;beta&lt;/em&gt;. What you get is a &lt;strong&gt;column&lt;/strong&gt; vector of &lt;em&gt;N&lt;/em&gt; &lt;strong&gt;predicted values&lt;/strong&gt; for &lt;em&gt;y&lt;/em&gt; (&lt;em&gt;\hat{y}&lt;/em&gt;). The difference of the &lt;em&gt;N&lt;/em&gt;-dimensional column vector of predicted values &lt;em&gt;\hat{y}&lt;/em&gt; and the &lt;em&gt;N&lt;/em&gt;-dimensional column vector &lt;em&gt;y&lt;/em&gt; of measured values is the &lt;strong&gt;error&lt;/strong&gt; (which is minimised with the least squares method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have &lt;em&gt;q&lt;/em&gt; dependent variables &lt;em&gt;y&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; (&lt;em&gt;i = 1...q&lt;/em&gt;), each of them would generally have its own set of weights &lt;em&gt;beta&lt;/em&gt;. In other words, the relation between the input variables and &lt;em&gt;each&lt;/em&gt; &lt;em&gt;y&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; will be &lt;em&gt;different&lt;/em&gt;. The dependent variables can also be arranged in a matrix (just like the input matrix), and its dimensionality will be &lt;em&gt;N x q&lt;/em&gt;, where &lt;em&gt;q&lt;/em&gt;. In that case, the &lt;em&gt;beta&lt;/em&gt; matrix will &lt;strong&gt;not be a single &lt;em&gt;N&lt;/em&gt;-dimensional column vector but an &lt;em&gt;N x q&lt;/em&gt; matrix&lt;/strong&gt;. Each column in the &lt;em&gt;beta&lt;/em&gt; matrix gives the relation between the input variables and the &lt;em&gt;q&lt;/em&gt;&lt;sup&gt;th&lt;/sup&gt; dependent variable &lt;em&gt;y&lt;sub&gt;q&lt;/sub&gt;&lt;/em&gt;. In that respect, a single observation of all variables &lt;em&gt;y&lt;sub&gt;i&lt;/sub&gt;&lt;/em&gt; will be a &lt;em&gt;row&lt;/em&gt; vector in the &lt;em&gt;y&lt;/em&gt; matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that this clarifies things up a bit. In summary, the explanation in the textbook is correct, but the notation is hard to follow and at times plain misleading (as in the case of variable and component at the beginning of the section). Honestly, you can get a much more intuitive explanation of linear regression from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; article.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-07-03T14:12:32.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7000" PostTypeId="1" CreationDate="2018-07-03T15:55:33.983" Score="3" ViewCount="681" Body="&lt;p&gt;I would like to use the pre-trained weights of YOLO (tiny version, &lt;strong&gt;v1&lt;/strong&gt;) in Keras, which are given in a file with extension &lt;code&gt;.weights&lt;/code&gt; (here is &lt;a href=&quot;https://pjreddie.com/media/files/yolov3.weights&quot; rel=&quot;nofollow noreferrer&quot;&gt;an example&lt;/a&gt;). How can I read such a file in Keras?&lt;/p&gt;&#xA;&lt;p&gt;For the past two days, I've been relentlessly digging through Github, and the likes, in order to help me in this task, with more or less success.  I found some scripts on the internet in order to convert &lt;code&gt;.txt&lt;/code&gt; or &lt;code&gt;.h5&lt;/code&gt; files (&lt;code&gt;.h5&lt;/code&gt; only for YOLO &lt;strong&gt;v2&lt;/strong&gt;, and not &lt;strong&gt;v1&lt;/strong&gt;), but none of these seemed to be working properly.&lt;/p&gt;&#xA;&lt;p&gt;So, I guess my question is: how to read a &lt;code&gt;.weights&lt;/code&gt; file? How are the data stored and structured in these files?&lt;/p&gt;&#xA;" OwnerUserId="16674" LastEditorUserId="2444" LastEditDate="2021-01-16T20:03:10.873" LastActivityDate="2021-01-16T20:06:10.453" Title="How can I read the .weights file that stores the weights of the pre-trained YOLO in Keras?" Tags="&lt;keras&gt;&lt;object-recognition&gt;&lt;implementation&gt;&lt;yolo&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-03-16T19:12:19.553" ContentLicense="CC BY-SA 4.0" />
  <row Id="7001" PostTypeId="2" ParentId="7000" CreationDate="2018-07-03T17:03:41.707" Score="3" Body="&lt;p&gt;The &lt;code&gt;.weights&lt;/code&gt; seems to be the extension for a framework called &amp;quot;darknet&amp;quot;. You can read &lt;code&gt;.h5&lt;/code&gt; files with &lt;strong&gt;Keras&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, if you really want to build an object detection framework, there is no necessity to stick to the &lt;strong&gt;darknet's &lt;code&gt;.weights&lt;/code&gt;&lt;/strong&gt; files. There are many pretrained models on the web. Or else you could fine-tune a pre-trained ImageNet model in &lt;strong&gt;Keras&lt;/strong&gt;, which I think is the best option, although there is not much you can do, if you got the exact weights of the YOLO model. Instead, it is better to train one from scratch or at least by fine-tuning an ImageNet model. By doing so, you will learn a lot.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="2444" LastEditDate="2021-01-16T20:06:10.453" LastActivityDate="2021-01-16T20:06:10.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7003" PostTypeId="1" CreationDate="2018-07-03T20:23:32.997" Score="2" ViewCount="67" Body="&lt;p&gt;Gradient in Maximum Entropy IRL requires to find the probability of expert trajectories given the reward function weights. This is done in the paper by calculating state visitation probabilities but I do not understand why we can’t just calculate the probability of a trajectory by summing up all the rewards that are collected following that trajectory? The paper defines the probability of a trajectory as exp(R(traj.)/Z. I do not understand why we have to solve MDP for calculating that.&lt;/p&gt;&#xA;" OwnerUserId="16678" LastActivityDate="2018-07-03T20:23:32.997" Title="Why do we have to solve MDP in each iteration of Maximum Entropy Inverse Reinforcement Learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7004" PostTypeId="2" ParentId="6997" CreationDate="2018-07-03T20:44:28.883" Score="2" Body="&lt;p&gt;I recently saw a course by Microsoft on edx. It is called 'Reinforcement Learning Explained'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link:&#xA;&lt;a href=&quot;https://www.edx.org/course/reinforcement-learning-explained-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.edx.org/course/reinforcement-learning-explained-0&lt;/a&gt;&#xA;This is not quite comprehensive but at least gives a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="16497" LastActivityDate="2018-07-03T20:44:28.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7005" PostTypeId="2" ParentId="6997" CreationDate="2018-07-03T22:49:46.140" Score="3" Body="&lt;p&gt;There's a Youtube playlist (in the &lt;a href=&quot;https://www.youtube.com/channel/UCP7jMXSY2xbc3KCAE0MHQ-A&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind channel&lt;/a&gt;) whose title is &lt;a href=&quot;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Introduction to reinforcement learning&lt;/em&gt;&lt;/a&gt;, which is a course (of 10 lessons) on reinforcement learning by &lt;a href=&quot;https://www.wikiwand.com/en/David_Silver_(programmer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Silver&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A person who followed and finished the course wrote (as a Youtube comment):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Excellent course. Well paced, enough examples to provide a good intuition, and taught by someone who's leading the field in applying RL to games.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2018-07-03T22:49:46.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7006" PostTypeId="2" ParentId="6997" CreationDate="2018-07-03T23:40:37.887" Score="6" Body="&lt;p&gt;To the good answers here, I would add&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1708.05866v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;A brief overview of RL&lt;/a&gt;: Most essential concepts in one place.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pemami4911.github.io/pdfs/deep-rl.pdf&quot; rel=&quot;noreferrer&quot;&gt;Another brief overview&lt;/a&gt;, in presentation format.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ben Recht's &lt;a href=&quot;http://www.argmin.net/2018/06/25/outsider-rl/&quot; rel=&quot;noreferrer&quot;&gt;An outsider's tour of RL&lt;/a&gt; is pretty comprehensive and accessible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://joshgreaves.com/reinforcement-learning/understanding-rl-the-bellman-equations/&quot; rel=&quot;noreferrer&quot;&gt;The Bellman equations&lt;/a&gt;: central to the whole RL theory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;noreferrer&quot;&gt;Policy gradients explained&lt;/a&gt; by Andrej Karpathy (mentioned in other answers as &quot;pong from pixels&quot;, this is the link).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These barely scratch the surface of RL, but they should get you started.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-07-03T23:40:37.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7007" PostTypeId="2" ParentId="6968" CreationDate="2018-07-04T00:13:58.457" Score="0" Body="&lt;p&gt;The brief answer is: generalized machine algorithm is an algorithm that can do well and give good results in new data that never seen before&lt;/p&gt;&#xA;" OwnerUserId="14749" LastActivityDate="2018-07-04T00:13:58.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7008" PostTypeId="1" AcceptedAnswerId="7014" CreationDate="2018-07-04T05:37:37.410" Score="3" ViewCount="3187" Body="&lt;p&gt;I have practiced building CNNs for image classification with TensorFlow, which is a nice library with good documentation and tutorials. However, I found that TensorFlow is too complicated and cumbersome.&lt;/p&gt;&#xA;&lt;p&gt;Can I build a CNN for image classification tasks just with OpenCV?&lt;/p&gt;&#xA;" OwnerUserId="16383" LastEditorUserId="2444" LastEditDate="2020-09-22T10:23:33.443" LastActivityDate="2020-09-22T10:23:33.443" Title="Can I build a CNN for image classification tasks just with OpenCV?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;image-recognition&gt;&lt;opencv&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-09-22T10:20:37.150" ContentLicense="CC BY-SA 4.0" />
  <row Id="7010" PostTypeId="2" ParentId="6770" CreationDate="2018-07-04T09:27:46.157" Score="2" Body="&lt;p&gt;You can search for the following paper titles:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A Deep Multi-Level Network for Saliency Prediction.&lt;/li&gt;&#xA;&lt;li&gt;Beyond Universal Saliency: Personalized Saliency Prediction with Multi-task CNN.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can code in python using Pytorch framework.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-07-04T09:27:46.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7014" PostTypeId="2" ParentId="7008" CreationDate="2018-07-04T12:09:20.330" Score="4" Body="&lt;p&gt;OpenCV does include 2D filter convolution functions for custom &lt;a href=&quot;https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#ga910e29ff7d7b105057d1625a4bf6318d&quot; rel=&quot;nofollow noreferrer&quot;&gt;separable&lt;/a&gt; and &lt;a href=&quot;https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#ga27c049795ce870216ddfb366086b5a04&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-separable&lt;/a&gt; filters. The latter uses DFT for large filters, which may or may not be faster than the conventional method. It also includes (partial?) support for &lt;a href=&quot;https://docs.opencv.org/3.4/d6/d0f/group__dnn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep nets&lt;/a&gt; with &lt;a href=&quot;https://docs.opencv.org/3.4/d6/d87/group__dnnLayerList.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;various types of layers&lt;/a&gt;. Theoretically, you should be able to stitch everything together into a complete CNN. However, I have not used any of those, and I have no idea about the level of maturity of the implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, if you are willing to implement a custom CNN from scratch, you will probably get more control over the implementation using a generic (BLAS / OpenCL / CUDA) matrix library.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-07-04T12:09:20.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7016" PostTypeId="2" ParentId="6903" CreationDate="2018-07-04T12:19:58.240" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www-bcf.usc.edu/~gareth/ISL/ISLR%20Seventh%20Printing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to statistical learning&lt;/a&gt; is sort of a resource for the tools that go into AI programming, with some exercises in R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/~hastie/Papers/ESLII.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elements of statistical learning&lt;/a&gt; is the more scientific and math / theory heavy version of ISL. I’m reading that one now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are both freely available from Stanford online at the links above. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastActivityDate="2018-07-04T12:19:58.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7019" PostTypeId="2" ParentId="6770" CreationDate="2018-07-04T14:28:28.860" Score="0" Body="&lt;p&gt;&quot;Attention&quot; in neural network (visual) is the area of the image where the network can find most number of features to classify it with high confidence.Based on your description you are talking about &quot;soft attention&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Do we have any tools or SDK to implement that?&lt;/em&gt; i don't think there are readymade SDKs available. It is much better to train a model on your dataset with attention. Once you have your base model ready , it is easy to add attention mechanism for it.I suggest you to check &lt;a href=&quot;https://arxiv.org/pdf/1502.03044.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1502.03044.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-04T14:28:28.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7021" PostTypeId="1" CreationDate="2018-07-04T19:52:56.650" Score="9" ViewCount="340" Body="&lt;p&gt;Most humans are not good at chess. They can't write symphonies. They don't read novels. They aren't good athletes. They aren't good at logical reasoning. Most of us just get up. Go to work in a factory or farm or something. Follow simple instructions. Have a beer and go to sleep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some things that a clever robot can't do that a stupid human can?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4302" LastEditDate="2018-10-19T21:52:47.367" LastActivityDate="2018-10-19T21:52:47.367" Title="Is the smartest robot more clever than the stupidest human?" Tags="&lt;robots&gt;&lt;reasoning&gt;&lt;intelligence&gt;" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7022" PostTypeId="2" ParentId="3156" CreationDate="2018-07-04T19:58:50.787" Score="4" Body="&lt;p&gt;Have a look at the paper &lt;a href=&quot;https://wiki.inf.ed.ac.uk/twiki/pub/CSTR/ListenTerm1201415/sak2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling&lt;/a&gt; (2014), where different LSTM architectures are compared. In the abstract, the authors write the following.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We show that a two-layer deep LSTM RNN where each LSTM layer has a linear recurrent projection layer can exceed state-of-the-art speech recognition performance&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16687" LastEditorUserId="2444" LastEditDate="2020-03-07T15:50:04.103" LastActivityDate="2020-03-07T15:50:04.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7023" PostTypeId="2" ParentId="6926" CreationDate="2018-07-04T21:03:31.600" Score="4" Body="&lt;p&gt;Great question, and one that I think we could have done a better job of answering in the paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the &lt;em&gt;pose matrix&lt;/em&gt; of each capsule is set up so that it could learn to represent the affine transformation between the object and the viewer, but we are not restricting it to necessarily do that. So we talk about the output of a capsule as though it is an affine transformation matrix, but we can't ensure that it will be. We do things explicitly that make it more like such a matrix — like adding in the coordinates to the right-hand column — but we can't be sure. This somewhat embodies a large part of the capsule network theory — we set up scaffolding so that the network can learn to be equivalent to transformations that we think it ought to be invariant to, but we don't ensure that it is.&lt;/p&gt;&#xA;" OwnerUserId="16689" LastEditorUserId="2444" LastEditDate="2020-06-09T15:12:11.507" LastActivityDate="2020-06-09T15:12:11.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7024" PostTypeId="1" AcceptedAnswerId="7026" CreationDate="2018-07-04T22:04:37.217" Score="3" ViewCount="55" Body="&lt;p&gt;I'm a complete newbie to NNs, and I need your advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of images of symbols, and my goal is to categorize and divide them into groups of symbols that look alike. Without teaching NN anything about the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to do this? What type of NN suits the best? Maybe there are any ready solutions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16690" LastActivityDate="2018-07-05T03:44:06.693" Title="Automatic image classification" Tags="&lt;neural-networks&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7025" PostTypeId="1" CreationDate="2018-07-05T03:36:27.490" Score="2" ViewCount="497" Body="&lt;p&gt;I recently came across reference to a book that was highly regarded: “Pattern Recognition and Machine Learning” by Christopher Bishop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a beginner working my way through some machine learning courses on my own. I’m curious if this book is still relevant considering it was published in 2006? Can anyone who has read it attest to its usefulness in 2018?&lt;/p&gt;&#xA;" OwnerUserId="16343" LastActivityDate="2018-07-09T17:29:49.330" Title="Is Christopher Bishop’s “Pattern Recognition and Machine Learning” out of date in 2018?" Tags="&lt;machine-learning&gt;&lt;pattern-recognition&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7026" PostTypeId="2" ParentId="7024" CreationDate="2018-07-05T03:44:06.693" Score="2" Body="&lt;p&gt;This is called &quot;clustering&quot; , If the network is already trained with data that has similar features as of the &quot;symbols&quot;, you can use that network with its last classification layer removed , then run a clustering algorithm like &quot;k-means&quot; on top of the vectors obtained from the last layer of the network.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-05T03:44:06.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7027" PostTypeId="1" AcceptedAnswerId="7028" CreationDate="2018-07-05T05:25:27.123" Score="4" ViewCount="1145" Body="&lt;p&gt;I was learning about back-propagation and, looking at the algorithm, there is no particular 'partiality' given to any unit. What I mean by partiality there is that you have no particular characteristic associated with any unit, and this results in all units being equal in the eyes of the machine.&lt;/p&gt;&#xA;&lt;p&gt;So, won't this result in the same activation values of all the units in the same layer? Won't this lack of 'partiality' render neural networks obsolete?&lt;/p&gt;&#xA;&lt;p&gt;I was reading a bit and watching few videos about backpropagation and, in the explanation given by Geoffrey Hinton, he talks about how we're trying to train the hidden units using the error derivatives w.r.t our hidden activities rather than using desired activities. This further strengthens my point about how by not adding any difference to the units, all units in a layer become equal since initially the errors due to all of them are the same and thus we train them to be equal.&lt;/p&gt;&#xA;" OwnerUserId="16694" LastEditorUserId="2444" LastEditDate="2021-01-20T21:32:37.767" LastActivityDate="2021-01-20T21:32:37.767" Title="Do we know what the units of neural networks will do before we train them?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;artificial-neuron&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7028" PostTypeId="2" ParentId="7027" CreationDate="2018-07-05T08:02:49.463" Score="8" Body="&lt;p&gt;In reverse order to how you asked:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;all units in a layer become equal since initially the errors due to all of them are the same and thus we train them to be equal&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This actually happens &lt;strong&gt;if&lt;/strong&gt; you initialise the weights equally (e.g. all zero). Gradients in that case are the same to each neuron in the same layer, and everything changes in lockstep. A neural network without random weight initialisation will simply not work.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So won't this result in the same activation values of all the units in the same layer? Won't this lack of 'partiality' render neural networks obsolete?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, because random weight initialisation causes the gradients to be different, and the neuron activations will typically diverge to represent different &quot;hidden&quot; features that activate differently depending on the input.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What I mean by partiality there is that, you have no particular characteristic associated with any unit and this results in all units being equal in the eyes of the machine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One interesting side-effect of this behaviour is that the &quot;partiality&quot; will often be assigned effectively randomly too, as the neural network will converge to features that somehow work. These features have no guarantee to be meaningful to a human being in the context of a problem to solve. They might be something that can be mapped to the problem, they might be some linear combination of something that can be understood, but often there is no obvious interpretation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-07-05T08:02:49.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7029" PostTypeId="2" ParentId="7021" CreationDate="2018-07-05T08:34:12.227" Score="2" Body="&lt;h1&gt;Survival, Imagining, Moral Reasoning&lt;/h1&gt;&#xA;&lt;p&gt;The thing that comes to mind is a new-born, when you said &amp;quot;the stupidest human&amp;quot;, and it already has some basic “survival instincts”. It will avoid pain, consume food, and quickly learn to distinguish &amp;quot;safe&amp;quot; and &amp;quot;dangerous&amp;quot; conditions and people.&lt;/p&gt;&#xA;&lt;p&gt;We have computer programs that can learn chess and calculate the optimal move in a split second, but isn't playing chess is a bit pointless. Merely being able to play a board game is of little value from a survival perspective, industrial perspective, or economic perspective.&lt;/p&gt;&#xA;&lt;p&gt;There are programs that can do things that are very helpful for the modern world, but as far as I know, they just don't have survival instincts. A self-learning robot; left in a forest with all the tools it needs to generate power for, build duplicates of, maintain and defend its self; probably wouldn't be able to learn how to do so in time to ensure its survival. Our current self learning programs would need to be able to identify when it has succeeded or failed to improve its survival odds. A child of two may learn fast enough to survive if the conditions are not too severe and non-toxic food and some form of shelter is nearby.&lt;/p&gt;&#xA;&lt;p&gt;A financially poor, marginally educated person with lower than average aptitude working at a farm or factory might not be able to play chess well, but they would definitely be able to tell if someone is murdering someone else, and know to flee and seek the authorities. A robot that can play chess would not.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, humans can continue to learn when separated from the problem by thinking about the problem. The ability to construct arbitrary models and run thought experiments is currently unique to humans.&lt;/p&gt;&#xA;&lt;br&gt;&#xA;&lt;p&gt;That said, I do hope that we will soon have programs that well replicate the human mind, and demonstrate some of the aspects of what we call consciousness.&lt;/p&gt;&#xA;" OwnerUserId="16355" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-07-05T18:49:13.540" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7030" PostTypeId="2" ParentId="7021" CreationDate="2018-07-05T08:42:29.827" Score="3" Body="&lt;p&gt;&lt;strong&gt;First Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To treat this question in a scientific way, because I think it is a reasonable enough question that draws on the realities of postmodern culture in post industrialized societies to be treated scientifically, we should define some things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most difficult is intelligence, which is the realm in which smartness, cleverness, and stupidity reside.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's go through the list.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Most humans are not good at chess, but humans invented the game.&lt;/li&gt;&#xA;&lt;li&gt;Most humans can't write symphonies, but humans invented them.&lt;/li&gt;&#xA;&lt;li&gt;Most humans don't read novels, but digital computers can't write them yet and can't learn ethical balance through the reading of them like humans can.&lt;/li&gt;&#xA;&lt;li&gt;Most humans are not Olympic level athletes, yet humans developed Olympics and robots are not yet Olympiads.&lt;/li&gt;&#xA;&lt;li&gt;Most humans (whether or hot they are good at logical reasoning) don't employ it much other than to, &quot;Get by.&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The question is fine until it devolves into the dismissal of the intelligence that people apply to their method of earning income, which for many if not most people requires more than just getting up, commuting in, following some simple instructions, and going to sleep mildly inebriated.  Let's replace this last part with this.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Most humans do not work with the intention of optimizing quality of the product or service by measuring their own quality and seeking educational resources to improve the velocity, reliability, or accuracy of their work output (unless programs are instituted to incentivize these things in the workplace.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If we define intelligence as the union of these things, for simplicity's sake, we have this (which is subject to change as AI develops).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Playing chess: AI wins&lt;/li&gt;&#xA;&lt;li&gt;Designing games: Humans win&lt;/li&gt;&#xA;&lt;li&gt;Writing symphonies: Humans win&lt;/li&gt;&#xA;&lt;li&gt;Writing novels: Humans win&lt;/li&gt;&#xA;&lt;li&gt;Absorbing ethics from stories: Humans win&lt;/li&gt;&#xA;&lt;li&gt;Olympic gold: Humans win&lt;/li&gt;&#xA;&lt;li&gt;Logical consistency: AI wins&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We must, to interpret the above list correctly concede two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Machines may have the ability to do something according to specific quality standards but not be configured, trained, or connected appropriately to prevail.&lt;/li&gt;&#xA;&lt;li&gt;Humans may have the ability to do something according to specific quality standards but not be educated, trained, or be properly motivated to prevail.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some things that a clever robot can't do that a stupid human can?  These are a few, but they are of particular importance from certain perspectives.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Love their family and friends&lt;/li&gt;&#xA;&lt;li&gt;Have compassion without reason&lt;/li&gt;&#xA;&lt;li&gt;Decide what to learn&lt;/li&gt;&#xA;&lt;li&gt;Hunt&lt;/li&gt;&#xA;&lt;li&gt;See a future danger approaching&lt;/li&gt;&#xA;&lt;li&gt;Entertain others&lt;/li&gt;&#xA;&lt;li&gt;Pray&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would not dismiss these human propensities as irrelevant, even from a scientific perspective.  I would also not dismiss the possibility that these things are beyond the capabilities of silicon based entities.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-05T08:42:29.827" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7032" PostTypeId="1" CreationDate="2018-07-05T10:36:42.557" Score="2" ViewCount="501" Body="&lt;p&gt;I should show that exact inference in a Bayesian network (BN) is NP-hard and P-hard by using a 3-SAT problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I did formulate a 3-SAT problem by defining 3-CNF:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(x_1 \lor x_2) \land (\neg x_3 \lor x_2) \land (x_3 \lor x_1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I reduced it to inference in a Bayesian network, and produced all conditional probabilities, and I know which variable assignment would lead for the entire expression to be true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the difference between P and NP. (Please correct me if I am wrong):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any P problem with an input of the size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; can be solved in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(n^c)$&lt;/span&gt;. For NP, the polynomial-time cannot be determined, hence, nondeterministic polynomial time. The question that scientists try to answer is whether a computer who is able to verify a solution would also be able to find a solution. P= NP?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am still not sure how I can prove that exact inference in Bayesian network is NP-hard and P-hard.&lt;/p&gt;&#xA;" OwnerUserId="15391" LastEditorUserId="2444" LastEditDate="2019-09-23T12:20:35.083" LastActivityDate="2019-09-23T12:20:35.083" Title="Why is exact inference in a Bayesian network both NP-hard and P-hard?" Tags="&lt;proofs&gt;&lt;time-complexity&gt;&lt;bayesian-network&gt;&lt;computational-complexity&gt;&lt;inference&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7033" PostTypeId="1" AcceptedAnswerId="7036" CreationDate="2018-07-05T11:56:28.173" Score="2" ViewCount="120" Body="&lt;p&gt;I have a task where I would like to use a convolutional neural network (CNN). I would like to incrementally start from the fastest models, fine-tune and see whether they fit my &quot;budget&quot;. At the moment, I'm just looking at object detection CNN-based feedforward models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious to know if there is any article, blog, web page or gist that benchmarks the popular CNN models based on the forward-pass speed. If there is back-propagation time and dataset-wise performance, even better!&lt;/p&gt;&#xA;" OwnerUserId="16702" LastEditorUserId="2444" LastEditDate="2019-11-04T03:33:36.260" LastActivityDate="2019-11-04T03:34:23.667" Title="Are there benchmarks for assessing the speed of the forward-pass of neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;feedforward-neural-network&gt;&lt;benchmarks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7035" PostTypeId="2" ParentId="6996" CreationDate="2018-07-05T12:18:36.390" Score="1" Body="&lt;p&gt;A long shot: &lt;a href=&quot;http://openaccess.thecvf.com/content_ICCV_2017/papers/Lee_Semantic_Line_Detection_ICCV_2017_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these guys&lt;/a&gt; have worked on a problem that might be relevant. They define &quot;semantic&quot; lines as lines delimiting significant regions or objects in an image. To detect such lines, they use the conv layers from a pre-trained VGG16 net and then add their own layers on top. The cool thing about their approach is that they run both classification &lt;em&gt;and&lt;/em&gt; regression in parallel on the same network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to adopt a similar technique to determine &lt;em&gt;where&lt;/em&gt; the line is, and then run some simple analysis on the extracted line to determine the offset and the orientation.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-07-05T12:18:36.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7036" PostTypeId="2" ParentId="7033" CreationDate="2018-07-05T12:24:12.087" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/jcjohnson/cnn-benchmarks&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jcjohnson/cnn-benchmarks&lt;/a&gt; might be a good start. It mostly focuses on GPUs, but there is also one CPU (Dual Xeon E5-2630 v3).&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="2444" LastEditDate="2019-11-04T03:34:23.667" LastActivityDate="2019-11-04T03:34:23.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7037" PostTypeId="2" ParentId="3156" CreationDate="2018-07-05T14:59:31.667" Score="6" Body="&lt;p&gt;In general, there are no guidelines on how to determine the number of layers or the number of memory cells in an LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of layers and cells required in an LSTM might depend on several aspects of the problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The complexity of the dataset, such as the number of features, the number of data points, etc. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The data-generating process. For example, the prediction of oil prices compared to the prediction of GDP is a well-understood economy. The latter is much easier than the former. Thus, predicting oil prices might require more LSTM memory cells to predict, with the same accuracy, as compared to the GDP. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The accuracy required for the use case. The number of memory cells will &lt;strong&gt;heavily&lt;/strong&gt; depend on this. If the goal is to beat the state-of-the-art model, in general, one needs more LSTM cells. Compare that to the goal of coming up with a reasonable prediction, which would need fewer LSTM cells.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I follow these steps when modeling using LSTM.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Try a single hidden layer with 2 or 3 memory cells. See how it performs against a benchmark. If it is a time series problem, then I generally make a forecast from classical time series techniques as benchmark. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try and increase the number of memory cells. If the performance is not increasing much then move on to the next step.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Start making the network deeper, i.e. add another layer with a small number of memory cells. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As a side note, there is no limit to the amount of labor that can be devoted to reach that global minimum of the loss function and tune the best hyper-parameters. So, having the focus on the end goal for modeling should be the strategy rather than trying to increase the accuracy as much as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the problems can be handled using 2-3 layers of the network.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="2444" LastEditDate="2020-03-07T16:08:05.407" LastActivityDate="2020-03-07T16:08:05.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7039" PostTypeId="2" ParentId="7021" CreationDate="2018-07-05T18:19:25.830" Score="3" Body="&lt;p&gt;The &quot;&lt;a href=&quot;http://anathem.wikia.com/wiki/Sline&quot; rel=&quot;nofollow noreferrer&quot;&gt;baseline&lt;/a&gt; humans&quot; you describe have been historically described in the media industry as &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lowest_common_denominator#Colloquial_usage&quot; rel=&quot;nofollow noreferrer&quot;&gt;the lowest common denominator&lt;/a&gt;&quot; (LCD). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LCD is the broadest possible audience for content, traditionally for &lt;a href=&quot;https://en.wiktionary.org/wiki/nobody_ever_went_broke_underestimating_the_intelligence_of_the_American_people&quot; rel=&quot;nofollow noreferrer&quot;&gt;network television shows&lt;/a&gt;.  (Before the age of cable, there were only 3 to 4 networks and all video content was broadcast over the airwaves--no way to specifically target audience segments so content had to appeal to the LCD.)  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/CAPTCHA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Captcha&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Because captchas have to be solvable by the LCD, but trick bots. As long as captcha are viable, by definition they will always be something that baseline humans can do better than AI. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-07-05T21:39:02.600" LastActivityDate="2018-07-05T21:39:02.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7040" PostTypeId="2" ParentId="7021" CreationDate="2018-07-05T18:20:28.697" Score="2" Body="&lt;p&gt;I do not know the precise definition of intelligence, but from lots of people I have interacted with, they regard people as intelligent on a &lt;strong&gt;particular field&lt;/strong&gt;, if and only if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They are able to take &lt;strong&gt;&lt;code&gt;split second correct decisions&lt;/code&gt;&lt;/strong&gt; in a situation in that particular field.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Let us see where AI have succeeded in this case:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arstechnica.com/gaming/2017/08/ai-bot-takes-on-the-pros-at-dota-2-and-wins/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elon Musk’s Dota 2 AI beats the professionals at their own game&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.theguardian.com/technology/2017/dec/07/alphazero-google-deepmind-ai-beats-champion-program-teaching-itself-to-play-four-hours&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero AI beats champion chess program after teaching itself in four hours&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are the few famous cases. If we examine carefully these cases we see that computers are outperforming humans only due to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Huge memory available.&lt;/li&gt;&#xA;&lt;li&gt;Fast memory access.&lt;/li&gt;&#xA;&lt;li&gt;Due to high processor speeds, split second correct decisions (although algorithm for correct decisions are developed by humans).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So AI's are actually workhorses, working without fatigue and without any limitations. Human brains do not excel in the  field of decision making or speed. Here is a comparison of &lt;a href=&quot;https://ai.stackexchange.com/questions/5239/what-makes-animal-brain-so-special&quot;&gt;What makes animal brain so special?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human brains excel at creativity. We can learn how to make symphonies. Can an AI do the same? Possibly with correct programming. Much of our intelligence comes from its distributed nature. We learn from other peoples mistakes, we improve it. Large number of humans combined with record keeping has made this possible. Although scientists like Tesla, Einstein, Newton, Feynman discovered Calculus on their own, think of the possibilities of new inventions had they been made aware that Calculus already exited and a lot has been done to develop it? Check this: &lt;a href=&quot;https://ai.stackexchange.com/questions/5258/swarm-intelligence-vs-normal-human-intelligence&quot;&gt;Swarm intelligence vs Normal Human Intelligence.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So our intelligence and experience comes from the huge source of information rather than huge source of personal resources. As of now we can think of abstract concepts which an AI cannot (i.e. we can create new things, not new artworks or music by mixing things up as an AI does, but a new thing completely). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it has been seen  if you keep many deaf babies together and isolated they develop their own form of sign language, completely unique. Points to note here are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They were completely isolated.&lt;/li&gt;&#xA;&lt;li&gt;They worked &lt;strong&gt;as a group&lt;/strong&gt; to develop the sign language.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So although machines might be performing well due to their algorithmic complexity and immense power they still have some catching up to do to be compared to even &lt;strong&gt;stupidest humans&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Main problem is we do not yet know the capacity of a brain. Some people can perform exceptional feats with their brain when the &lt;strong&gt;need arises&lt;/strong&gt;. Some one did this during WW2 to find his family: &lt;a href=&quot;https://www.theguardian.com/sport/2017/feb/10/timor-gareyev-48-chess-games-blindfolded-riding-exercise-bike-leonard-barden&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grandmaster plays 48 games at once, blindfolded while riding exercise bike&lt;/a&gt;. But how is this suddenly possible? No one knows until we have uncovered our own &lt;strong&gt;mind&lt;/strong&gt; fully.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-07-05T19:03:33.980" LastActivityDate="2018-07-05T19:03:33.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7042" PostTypeId="1" CreationDate="2018-07-05T21:42:17.983" Score="6" ViewCount="1647" Body="&lt;p&gt;Typical AI these days are question-answering machines. For example, Siri, Alexa and Google Home. But it is always the human asking the questions and the AI answering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any good examples of an AI that is curious and asks questions of its own accord?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2018-07-08T06:54:07.363" Title="An AI that asks questions?" Tags="&lt;agi&gt;&lt;chat-bots&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7043" PostTypeId="2" ParentId="7042" CreationDate="2018-07-06T03:48:07.483" Score="3" Body="&lt;p&gt;You are referring to 'proactive AI' as opposed to 'reactive AI' like Alexa, Cortana, Siri, Bixby, Google Assistant, and others.  There hasn't been much progress in this area of AI.  &lt;a href=&quot;https://www.youtube.com/watch?v=bd1mEm2Fy08&amp;amp;t=1s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's recent demonstration of Duplex&lt;/a&gt; addresses this to some extent.  Some chatbots are proactive. &lt;a href=&quot;https://www.genesys.com/customer-experience/innovations/meet-kate-blended-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genesys&lt;/a&gt; provides such capability.  Check out their &lt;a href=&quot;https://www.youtube.com/watch?v=vdOprvic_20&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Azure's bot service has a page on &lt;a href=&quot;https://docs.microsoft.com/en-us/azure/bot-service/nodejs/bot-builder-nodejs-proactive-messages?view=azure-bot-service-3.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;how to implement proactivity&lt;/a&gt; and there is another video that walks through the whole process: &lt;a href=&quot;https://www.youtube.com/watch?v=Am1NPUceFvA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learn to build Proactive Bot in 30 Minutes&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-07-06T03:48:07.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7044" PostTypeId="1" CreationDate="2018-07-06T03:49:26.513" Score="11" ViewCount="1135" Body="&lt;p&gt;Are there possible models that have the potential to replace neural networks in the near future? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And do we even need that? What is the worst thing about using neural networks in terms of efficiency?&lt;/p&gt;&#xA;" OwnerUserId="16715" LastEditorUserId="2444" LastEditDate="2020-04-12T13:39:03.230" LastActivityDate="2020-04-12T13:39:03.230" Title="What are the models that have the potential to replace neural networks in the near future?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7045" PostTypeId="2" ParentId="7042" CreationDate="2018-07-06T08:15:48.427" Score="1" Body="&lt;p&gt;One of the simplest examples that I can think of is &quot;Akinator&quot;. At the heart it uses decision trees to narrow down the search. It is not a &quot;questioning&quot; model like QA models used in Alexa, but it does asks questions.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="16355" LastEditDate="2018-07-06T09:27:04.043" LastActivityDate="2018-07-06T09:27:04.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7046" PostTypeId="2" ParentId="7044" CreationDate="2018-07-06T08:21:24.043" Score="0" Body="&lt;p&gt;Neural networks require lots of data and training. For most tabular format datasets it is much better to use decision tree based models. Most of the time, simple models are enough to give good accuracy. However neural networks had their test of time. It has only been five to six years since the deep learning revolution started, so we still do not know the true potency of deep learning.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastEditorUserId="15935" LastEditDate="2018-07-06T09:27:59.383" LastActivityDate="2018-07-06T09:27:59.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7048" PostTypeId="1" CreationDate="2018-07-06T11:43:00.037" Score="1" ViewCount="32" Body="&lt;p&gt;I am using LSTM model to predict the next xml markup from an input seed.&#xA;I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize the predicted results in a form of a graph or maybe is it meaningful to do so ?&#xA;Since we can do the visualization of classification results, for example in this&lt;br&gt;&#xA;&lt;a href=&quot;https://datascience.stackexchange.com/questions/15271/visualizing-results-of-a-classification-problem-excluding-confusion-matrices&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done some research on Internet, I have found that there is the confidence measure that can be useful for text prediction task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a bit confused what to do with the text results that I got.&lt;/p&gt;&#xA;" OwnerUserId="10167" LastActivityDate="2018-07-06T11:43:00.037" Title="How to visualize/interpret text prediction model results?" Tags="&lt;prediction&gt;&lt;long-short-term-memory&gt;&lt;performance&gt;&lt;graphs&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7049" PostTypeId="1" AcceptedAnswerId="7050" CreationDate="2018-07-06T11:57:23.427" Score="3" ViewCount="1089" Body="&lt;p&gt;In order to learn about DP and RL, I chose to start a side project where I would train an AI to play a &quot;simple&quot; card game. I will be doing this using the DQN with replay memory.&lt;br&gt;&#xA;The problem is, I can't get the intuition behind how to represent the input to the neural network..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;About the game&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a fairly simple 2-players game. There is a deck of 40 unique cards (4 types of cards, 10 numbered cards in each type).&lt;br&gt;&#xA;Each player gets 4 cards and each turn a player must put a card on the table.&lt;br&gt;&#xA;If a player puts a card and there is already a card with the same number on the table, the player wins both cards.&lt;br&gt;&#xA;If for example a player plays Card 2 and on the table there is Cards 2, 3, 4, 5 then the player wins all those cards (sequence).&lt;br&gt;&#xA;Cards won don't go back to the hand nor to the deck, they are just kept as like a score.&lt;br&gt;&#xA;When the players have 0 cards in hand, another 4 cards are dealt to each one untile the deck has 0 cards left where then we decide who won based on the number of cards eaten/won.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the input, I will be using the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Current cards in the AI hand (40 one-hot-encoded features?)&lt;/li&gt;&#xA;&lt;li&gt;Current cards on the table (40 one-hot-encoded features?)&lt;/li&gt;&#xA;&lt;li&gt;History of played cards (40 one-hot-encoded features?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This would give 120 columns/features in each state.&lt;br&gt;&#xA;I am wondering wheter this is too much for the NN or wheter my input representation would be bad for the NN?&lt;br&gt;&#xA;Should the features be represented as a (120,) vector or as a 3x40 matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also wondering if it's a good idea to represent the current cards on the table as just a 10 one-hot-encoded features since the type of the cards don't matter and the same number can't exist 2 times in the table?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="16720" LastEditorUserId="16720" LastEditDate="2018-07-06T12:18:14.527" LastActivityDate="2018-07-06T14:38:47.160" Title="DQN input representation for a card game" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7050" PostTypeId="2" ParentId="7049" CreationDate="2018-07-06T14:38:47.160" Score="3" Body="&lt;p&gt;120 inputs can be handled by a complex enough network. Dealing with high complexity is one of NN's strengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a (120,) vector or a (3,40) matrix is the same, they're still 120 inputs. Your binary encoding should work. Another option is a single (40,) vector, with 0 being &quot;still in deck&quot;, 1 being &quot;in hand&quot;, 2 being &quot;on table&quot;, 3 being &quot;already played&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the types of cards are irrelevant, you could actually have a (3,10) matrix with counters of cards (1 in hand, 1 in table, 2 already played). You can try different approaches and see what works best.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2018-07-06T14:38:47.160" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7051" PostTypeId="1" AcceptedAnswerId="7492" CreationDate="2018-07-06T14:42:14.027" Score="4" ViewCount="488" Body="&lt;p&gt;I'm attempting to create an AI for a card game using reinforcement learning.  The basics of the game are that you can have (theoretically) up to 35 cards in your hand, you can also have to up to 35 cards 'in play' and so can your opponent.  In normal play you would have ~6 cards in your hand and maybe ~3 each in play.  There are roughly 300 unique cards in total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I represent the game state for the input and how should I represent the action to take in the output?&lt;/p&gt;&#xA;" OwnerUserId="16724" LastEditorUserId="16724" LastEditDate="2018-07-06T15:02:21.843" LastActivityDate="2018-08-08T22:00:58.383" Title="Representing inputs and outputs for a card game neural network" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7052" PostTypeId="2" ParentId="7044" CreationDate="2018-07-06T15:36:33.383" Score="5" Body="&lt;p&gt;This is going backwards, but it kind of follows the logic of the arguments.&lt;/p&gt;&#xA;&lt;p&gt;In terms of efficiency, I can see a few major problems with classical neural networks.&lt;/p&gt;&#xA;&lt;h2&gt;Data collection and preprocessing overhead&lt;/h2&gt;&#xA;&lt;p&gt;Large neural networks require &lt;em&gt;a lot&lt;/em&gt; of data to train. The amount can vary depending on the size of the network and the complexity of the task, but as a rule of thumb it is usually proportional to the number of weights. For some supervised learning tasks, there simply isn't enough high-quality labelled data. Collecting large amounts of specialised training data can take months or even years, and labelling can be cumbersome and unreliable. This can be partially mitigated by data augmentation, which means &amp;quot;synthesising&amp;quot; more examples from the ones you already have, but it is not a panacea.&lt;/p&gt;&#xA;&lt;h2&gt;Training time vs. energy tradeoff&lt;/h2&gt;&#xA;&lt;p&gt;The learning rate is usually pretty small, so the training progress is slow. A large model that could take weeks to train on a desktop CPU can be trained in, say, two hours by using a GPU cluster which consumes several kW of power. This is a fundamental tradeoff due to the nature of the training procedure. That said, GPUs are getting increasingly efficient - for example, the new &lt;a href=&quot;https://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;nVidia Volta&lt;/a&gt; GPU architecture allows for 15.7 TFLOPs while consuming less than 300 W.&lt;/p&gt;&#xA;&lt;h2&gt;Non-transferrability&lt;/h2&gt;&#xA;&lt;p&gt;Right now, virtually every different problem requires a custom neural network to be designed, trained and deployed. While the solution often works, it is kind of &lt;em&gt;locked&lt;/em&gt; into that problem. For example, &lt;a href=&quot;https://deepmind.com/research/alphago/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; is brilliant at Go, but it would be hopeless at driving a car or providing music recommendations - it was just not designed for such tasks. This overwhelming redundancy is a major drawback of neural networks in my view, and it is also a major impediment to the progress of neural network research in general. There is a whole research area called &lt;a href=&quot;https://machinelearningmastery.com/transfer-learning-for-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt; which deals with finding ways of applying a network trained on one task to a different task. Often this relates to the fact that there might not be enough data to train a network from scratch on the second task, so being able to use a pre-trained model with some extra tuning is very appealing.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The first part of the question is more tricky. Leaving purely statistical models aside, I haven't seen any prominent approaches to machine learning that are &lt;em&gt;radically&lt;/em&gt; different from neural networks. However, there are some interesting developments that are worth mentioning because they address some of the above inefficiencies.&lt;/p&gt;&#xA;&lt;h2&gt;Neuromorphic chips&lt;/h2&gt;&#xA;&lt;p&gt;A bit of background first.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://homepages.cwi.nl/%7Esbohte/publication/es2014-13Gruning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spiking neural networks&lt;/a&gt; have enormous potential in terms of computational power. In fact, it has been &lt;a href=&quot;https://igi-web.tugraz.at/PDF/85a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;proven&lt;/a&gt; that they are &lt;em&gt;strictly more powerful&lt;/em&gt; than classical neural networks with sigmoid activations.&lt;/p&gt;&#xA;&lt;p&gt;Added to that, spiking neural networks have an intrinsic grasp of &lt;strong&gt;time&lt;/strong&gt; - something that has been a major hurdle for classical networks since their inception. Not only that, but spiking networks are &lt;strong&gt;event-driven&lt;/strong&gt;, which means that neurons operate only if there is an incoming signal. This is in contrast to classical networks, where each neuron is evaluated regardless of its input (again, this is just a consequence of the evaluation procedure usually being implemented as a multiplication of two dense matrices). So spiking networks employ a &lt;strong&gt;sparse&lt;/strong&gt; encoding scheme, which means that only a small fraction of the neurons are active at any given time.&lt;/p&gt;&#xA;&lt;p&gt;Now, the sparse spike-based encoding and event-driven operation are suitable for hardware-based implementations of spiking networks called &lt;strong&gt;neuromorphic chips&lt;/strong&gt;. For example, IBM's &lt;a href=&quot;https://www.ibm.com/blogs/research/2016/09/deep-learning-possible-embedded-systems-thanks-truenorth/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TrueNorth&lt;/a&gt; chip can simulate &lt;strong&gt;1 million neurons&lt;/strong&gt; and &lt;strong&gt;256 million connections&lt;/strong&gt; while drawing only about &lt;strong&gt;100 mW&lt;/strong&gt; of power on average. This is &lt;strong&gt;orders of magnitude&lt;/strong&gt; more efficient than the current nVidia GPUs. Neuromorphic chips may be the solution the training time / energy tradeoff I mentioned above.&lt;/p&gt;&#xA;&lt;p&gt;Also, &lt;a href=&quot;http://www.memristor.org/reference/research/13/what-are-memristors&quot; rel=&quot;nofollow noreferrer&quot;&gt;memristors&lt;/a&gt; are a relatively new but very promising development. Basically, a memristor is a fundamental circuit element very similar to a resistor but with &lt;strong&gt;variable&lt;/strong&gt; resistance proportional to the total amount of current that has passed through it over its entire lifetime. Essentially, this means that it maintains a &amp;quot;memory&amp;quot; of the amount of current that has passed through it. One of the exciting potential applications of memristors is modelling synapses in hardware extremely efficiently.&lt;/p&gt;&#xA;&lt;h2&gt;Reinforcement learning and evolution&lt;/h2&gt;&#xA;&lt;p&gt;I think these are worth mentioning because they are promising candidates for addressing the problem of non-transferrability. These are not restricted to neural networks - being reward-driven, RL and evolution are theoretically applicable in a generic setting to any task where it is possible to define a reward or a goal for an agent to attain. This is not necessarily trivial to do, but it is much more generic than the usual error-driven approach, where the learning agent tries to minimise the difference between its output and a ground truth. The main point here is about transfer learning: ideally, applying a trained agent to a different task &lt;strong&gt;should be&lt;/strong&gt; as simple as changing the goal or reward (they are not quite at that level yet, though...).&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-07-06T15:36:33.383" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7053" PostTypeId="2" ParentId="7042" CreationDate="2018-07-06T18:59:45.860" Score="1" Body="&lt;p&gt;I would like to give you a little bit of a hint,concerning the agents(chat-bots) mentioned in your question body.and also following up the tags and that's strong AI etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the current agents lack comprehension nor reasoning and by this I mean,these agents rely on rule based algorithms, basic neural networks name it... ,they don't exhibit strong  &quot;reasoning skills&quot; in that they can be expertly ask questions without training data nor following up human define rules effectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore,building up a truly  strong nor effective agent requires a lot of time and expertise,which AI research companies lack due to the Artificial Narrow Intelligence level of  AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one of the key capabilities which I think to gauge the real AI agent which can as well ask questions effectively is:reasoning and abstraction which will help an agent to autonomously adapt to new situations,and understand nor comprehend the context of the given subject/topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can judge according to the above insights, whether we do have good AI that is curious and is definitely accurate in asking questions!&#xA;Am not criticising the products (chat-bot),these companies put on market but we as scientists in this field,we have to save our civilisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-07-06T18:59:45.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7054" PostTypeId="1" CreationDate="2018-07-06T21:26:06.797" Score="2" ViewCount="200" Body="&lt;p&gt;Here's the general algorithm of maximum entropy inverse reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WuBtn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WuBtn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This uses a gradient descent algorithm. The point that I do not understand is there is only a single gradient value &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta \mathcal{L}$&lt;/span&gt;, and it is used to update a vector of parameters. To me, it does not make sense because it is updating all elements of a vector with the same value &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta \mathcal{L}$&lt;/span&gt;. Can you explain the logic behind updating a vector with a single gradient?&lt;/p&gt;&#xA;" OwnerUserId="16678" LastEditorUserId="2444" LastEditDate="2020-04-01T13:02:27.607" LastActivityDate="2020-04-01T13:02:41.313" Title="What does the notation $\nabla_\theta \mathcal{L}$ mean?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7055" PostTypeId="2" ParentId="7054" CreationDate="2018-07-07T02:07:10.850" Score="3" Body="&lt;p&gt;This is standard backpropagation. The gradient term you see is in fact a &lt;em&gt;vector&lt;/em&gt; of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element of the parameter vector &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. Therefore, it has the same dimensionality as &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. Each element of the parameter vector is then updated with the respective term in the vector of partial derivatives, which are generally &lt;em&gt;not&lt;/em&gt; the same.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="2444" LastEditDate="2020-04-01T13:02:41.313" LastActivityDate="2020-04-01T13:02:41.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7056" PostTypeId="2" ParentId="6899" CreationDate="2018-07-07T02:33:26.160" Score="2" Body="&lt;p&gt;The effectiveness of dividing training data and piping divisions into networks for independent training, although possibly an effective workaround for specific cases, is not indicative of a robust solution to fitting excellence across a wide range of input data sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As suggested in the comment by varshaneya, over-fitting can be a result of unsatisfactory regularization meta-parameterization, as in a poor setting of the λ regularization parameter in a StackGAN.  All meta-parameters used to tune a stacked architectures should be scrutinized to determine whether its setting could lead to over-fitting.  A few can be eliminated up front.  For instance too high a learning rate at any level of any of the networks in the design can reduce convergence probability, but is not a likely cause for over-fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;H. Hutson, S. Geva, and P. Cimiano wrote, in &lt;a href=&quot;http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings13/pdf/ntcir/02-NTCIR13-NAILS-HutsonH.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;their 2017 submittal to the 13th NTCIR Conference on Evaluation of Information Access Technologies&lt;/a&gt;, &quot;Ensemble methods in machine learning involve the combination of multiple classifiers via a variety of methods such as bagging (averaging or voting), boosting, and stacking, to increase performance and reduce over-fitting.&quot;  Yet bagging has not produced robust responses to differing data sets in our experience, even when normalized, filtered to reduce noise levels, and redundancy is limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Zhi-Hua Zhou and Ji Feng [National Key Laboratory for Novel Software Technology, Nanjing University, China, indicated] wrote, &quot;To reduce the risk of over-fitting, class vector produced by each forest is generated by k-fold cross validation.&quot;  Reading their paper, &lt;a href=&quot;https://arxiv.org/pdf/1702.08835.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Forest&lt;/a&gt;, may give you some causes to evaluate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over-fitting is usually the application of too sophisticated a model to which data is fit.  In the world of activated networks, excessive sophistication can be as simple as an excessive number of network layers in one or more of the stacked networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feature extraction up front may be needed to remove complexity from the input which is not only unnecessary but counterproductive to generalization and thus the generation of useful output.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-07T02:33:26.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7057" PostTypeId="1" AcceptedAnswerId="7064" CreationDate="2018-07-07T03:50:25.143" Score="2" ViewCount="62" Body="&lt;p&gt;I am currently implementing the paper &lt;a href=&quot;https://web.archive.org/web/20180205094740/http://web.engr.illinois.edu/%7Eslazebni/publications/iccv15_active.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Active Object Localization with Deep Reinforcement Learning&lt;/a&gt; in Python. While reading about the reward scheme I came across the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Finally, the proposed reward scheme implicitly considers the number of steps as a cost because of the way in which Q-learning models the discount of future rewards (positive and negative).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How would you implement this &amp;quot;number of steps&amp;quot; cost? I am keeping track of the number of steps that have been taken, therefore would it be best to use an exponential functions to discount the reward at the current time step?&lt;/p&gt;&#xA;&lt;p&gt;If anyone has a good idea or knows the standard in regard to this I would love to hear your thoughts.&lt;/p&gt;&#xA;" OwnerUserId="14913" LastEditorUserId="2444" LastEditDate="2020-09-28T10:14:24.993" LastActivityDate="2020-09-28T10:14:24.993" Title="How should I take into consideration the number of steps in the reward function?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;papers&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7058" PostTypeId="2" ParentId="7044" CreationDate="2018-07-07T03:58:08.293" Score="1" Body="&lt;p&gt;&lt;strong&gt;Replacing Neural Nets&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may exist new algorithms that have the potential to replace neural nets. However, one of the characteristics of neural nets is that they employ simple elements, each with low demands on computing resources in geometric patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial neurons can be run in parallel (without CPU time sharing or looping) by mapping the computations to DSP devices or other parallel computing hardware.  That the many neurons are essentially alike is thus a strong advantage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What Would We Be Replacing?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we consider algorithmic replacements to neural nets, we imply that a neural net design is an algorithm.  It is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural net is an approach to converging on a real time circuit to perform a nonlinear transformation of input to output based on some formulation of what is optimal.  Such a formulation may be the minimization of a measure of error or disparity from some defined ideal.  It may be a measure of wellness that must be maximized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source of the fitness determination for any given network behavior may be internal.  We call that unsupervised learning.  It may be external, which we call supervised when the external fitness information is coupled with input vectors in the form of desired output values, which we call labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fitness may also originate externally as a scalar or vector not coupled with the input data but rather real time, which we call reinforcement.  Such requires re-entrant learning algorithms.  Net behavioral fitness may alternatively be evaluated by other nets within the system, in the case of stacked nets or other configurations such as Laplacian hierarchies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The selection of algorithms has little to do with comparative intelligence once the mathematical and process designs are selected.  Algorithm design is more directly related to minimizing demands for computing resources and reducing time requirements.  This minimization is hardware and operating system dependent too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is a Replacement Indicated?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure.  It would be better if networks were more like mammalian neurons.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sophistication of activation&lt;/li&gt;&#xA;&lt;li&gt;Heterogeneity of connection patterns&lt;/li&gt;&#xA;&lt;li&gt;Plasticity of design, to support meta-adaptation&lt;/li&gt;&#xA;&lt;li&gt;Governed by many dimensions of regional signaling&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;By regional signaling is meant the many chemical signals beyond signal transmission across synapses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can even consider going beyond mammalian neurology.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Combining parametric and hypothesis-based learning&lt;/li&gt;&#xA;&lt;li&gt;Learning of the form employed when microbes pass DNA&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neural Net Efficiency&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Efficiency cannot be quantified in some universal scale as temperature can be quantified in degrees Kelvin.  Efficiency can only be quantified as a quotient of some measured value over some theoretical ideal.  Note that it is an ideal, not a maximum, in the denominator.  In thermodynamic engines, that ideal is the rate of energy input, which can never be fully transferred to the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, neural nets can never learn in zero time.  A neural net cannot achieve zero error over an arbitrarily long time in production either.  Therefore information is in some ways like energy, a concept investigated by Claude Shannon of Bell Labs during the dawn of digital automation, and the relationship between information entropy and thermodynamic entropy is now an important part of theoretical physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There can be no bad learning efficiency or good learning efficiency.  There can be neither bad performance nor good performance, if we wish to think in logical and scientific terms — only relative improvement of some system configuration with respect to some other system configuration for a very specific set of performance scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, without an unambiguous specification of the two hardware, operating system, and software configurations and a fully defined test suite used for relative evaluation, efficiency is meaningless.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-07T03:58:08.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7060" PostTypeId="2" ParentId="5577" CreationDate="2018-07-07T05:00:00.337" Score="0" Body="&lt;p&gt;The parsing of linguistic units from streams of speech by the human brain is an existing system that can be studied, and it is a legitimate proof of concept.  A working brain adapts to changes in volume, tone frequency, information rate, rhythm, accent, dialect, and background sound as it parses sequences of vocal sounds originating from the initial tone and transient processing of signals from the vestibulocochlear nerve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The later evolution of written symbol recognition from signals originating from the optic nerve is a related proof of concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple adaptive parsing is running in the lab as a part of social networking automation, but only for limited sets of symbols and sequential patterns.  It does scale without reconfiguration to an arbitrarily large base linguistic units, prefixes, endings, and suffixes, limited only by our hardware capacities and throughput.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The existence of regular expression libraries was helpful to keep the design simple.  We use the PCRE version 8 series library fed by a ansiotropic form of DCNN for feature extraction from a window moving through the input text with a configurable windows size and move increment size.  Heuristics applied to input text statistics gathered in a first pass produce a set of hypothetical PCREs arranged in two layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimization occurs to apply higher probabilistic weights to the best PCREs in a chaotically perturbed text search.  It uses the same gradient descent convergence strategies used in NN back propagation in training.  It is a naive approach that does not make assumptions like the existence of backtraces, files, or errors.  It would adapt equally to Arabic messages and Spanish ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is an arbitrary directed graph in memory, which is similar to a dump of an object oriented database.  JSON hierarchies were too restrictive four our purposes, since the directed graphs we were parsing from a serial text stream had cases of multiple incoming directed edges into vertices and some circular edge sequences.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;قنبلة -&amp;gt; dangereux -&amp;gt; 4anlyss&#xA;bomba -&amp;gt; dangereux&#xA;ambiguïté -&amp;gt; 4anlyss -&amp;gt; préemption -&amp;gt; قنبلة&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although a re-entrant algorithm for a reinforcement version is stubbed out and the wellness signal is already available, other work preempted furthering the adaptive parser or working toward the next step to use the work for natural language:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matching the directed graphs to persisted directed graph filters representing ideas, which would mimic the idea recollection aspect of language comprehension.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-07T05:00:00.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7061" PostTypeId="1" AcceptedAnswerId="7071" CreationDate="2018-07-07T05:43:51.720" Score="4" ViewCount="1659" Body="&lt;p&gt;I am very new to Machine learning and following the course offered by Andrew Ng. I am very confused How we train our neural network on Multi-class Classification(suppose take &lt;em&gt;K&lt;/em&gt; classes). For &lt;em&gt;K&lt;/em&gt; classes we will be training &lt;em&gt;K&lt;/em&gt; different neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But Do we train one neural network at a time for all features or we train all K NN at a time for one feature? please explain the complete procedures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me with this. Consider that I have a very very basic understanding of neural networks in multi-class classification.&lt;/p&gt;&#xA;" OwnerUserId="16734" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:05.850" LastActivityDate="2020-05-29T13:48:05.850" Title="Training Multi class classification (One-vs-all) on Neural network" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7062" PostTypeId="2" ParentId="7061" CreationDate="2018-07-07T06:30:40.930" Score="2" Body="&lt;p&gt;Let us suppose that you are training a neural network for classfing images of vehicles , then the input vector , image of the &quot;vehicle&quot; will be a 2D array of pixels. This undergoes several transformations at each layer of the neural network , the last layer of the neural network produces another vector whose dimensions are lesser than the original image vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the network is mapping images to some vectors in a high dimensional space. In order to classify the images it is now sufficient to classify the vectors obtained from the network of their corresponding images, you  can do this with a simple &quot;linear&quot; classifier using a softmax layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So all the layers of the network except the last layer are transforming the image representation to a &quot;vector&quot;. This vector is classified by a linear softmax classifier by the last layer of neural network.&lt;/p&gt;&#xA;" OwnerUserId="15935" LastActivityDate="2018-07-07T06:30:40.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7064" PostTypeId="2" ParentId="7057" CreationDate="2018-07-07T06:41:02.177" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;How would you implement this &amp;quot;Number of Steps&amp;quot; cost?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What the paper is referring to is the reward &lt;em&gt;discounting&lt;/em&gt; process which is a standard way of formulating RL problems, either continuous ones, or episodic ones where the goal is to complete a task in the least time (in the episodic version, a fixed cost per time step will also achieve this).&lt;/p&gt;&#xA;&lt;p&gt;As such, this usually is implemented in the formulation of the value function calculations. The discount factor is usually represented as gamma, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For Q-learning, the factor should be in the TD target calculation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+1} = R_{t+1} + \gamma \max_{a'}Q(S_{t+1},a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For Monte Carlo control, the factor appears more like this in the calculation of a return:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t = \sum_{k=0}^{T-t} \gamma^k R_{t+k+1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;would it be best to use an exponential functions to discount the reward at the current time step?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Essentially that is what normal discounting is - an exponential decay of future reward. But if you have implemented &amp;quot;normal&amp;quot; Q-learning from equations like those above, it should already be there.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-28T06:32:15.500" LastActivityDate="2020-09-28T06:32:15.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7065" PostTypeId="2" ParentId="5322" CreationDate="2018-07-07T08:33:33.747" Score="3" Body="&lt;p&gt;To add to the Foivos's answer,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convolutional Neural Networks are shift-invariant.&#xA;Fukushima introduced this to his Neocognitron.&#xA;There is a trail to introduce scale-invariance to CNN.&#xA;&lt;a href=&quot;https://arxiv.org/abs/1411.6369&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1411.6369&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, CNN uses structural characteristic for the prior knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And neural networks are locally smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not perfect, but neural networks are incorporating a lot of prior knowledge.&lt;/p&gt;&#xA;" OwnerUserId="16737" LastActivityDate="2018-07-07T08:33:33.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7067" PostTypeId="2" ParentId="5769" CreationDate="2018-07-07T14:41:49.480" Score="0" Body="&lt;p&gt;Refer to &amp;quot;Local Connectivity&amp;quot; section in &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and slide 7-18.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Receptive Field&amp;quot; hyperparameter of filter is defined by height &amp;amp; width only, as depth is fixed by preceding layer's depth.&lt;/p&gt;&#xA;&lt;p&gt;NOTE that &amp;quot;The extent of the connectivity along the depth axis is always equal to the DEPTH of the input volume&amp;quot; -or- DEPTH of activation map (in case of later layers).&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, this must be due to the fact that image channels data are interleaved, not planar. This way, applying filter can be achieved simply by column vectors multiplication.&lt;/p&gt;&#xA;&lt;p&gt;NOTE that Convolutional Network learns all the filter parameters (including the depth dimension) and they are total &amp;quot;h&lt;em&gt;w&lt;/em&gt;input_layer_depth + 1 (bias)&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="16741" LastEditorUserId="32410" LastEditDate="2021-01-18T09:38:24.827" LastActivityDate="2021-01-18T09:38:24.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7069" PostTypeId="2" ParentId="5107" CreationDate="2018-07-07T15:26:04.627" Score="0" Body="&lt;p&gt;The intention of the referred text is to reason out the disadvantage of equivalent-merged-single-convolution-layer over multiple [CONV -&gt; RELU]*N layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the given scenario, if 2 layers of 3x3 filters were to be replaced by an equivalent single layer then this equivalent layer would need a filter with Receptive Field of size 5x5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, equivalent layer filter would need its Receptive Field to be of size 7x7 to compress 3 layers of 3x3 filters. NOTE that the most obvious disadvantage would be missing out on modeling non-linearity.&lt;/p&gt;&#xA;" OwnerUserId="16741" LastActivityDate="2018-07-07T15:26:04.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7071" PostTypeId="2" ParentId="7061" CreationDate="2018-07-07T23:36:33.663" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For K classes we will be training K different neural networks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, you still train &lt;strong&gt;one&lt;/strong&gt; network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With binary classification tasks, where you have only two mutually exclusive categories, like &quot;yes/no&quot; or &quot;true/false&quot;, you can get away with a single output node with a sigmoid activation. The output of the sigmoid is interpreted as indicating one category for values &lt;code&gt;&amp;gt; 0.5&lt;/code&gt; and the other for values &lt;code&gt;&amp;lt;= 0.5&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With multi-class classification you have &lt;em&gt;K&lt;/em&gt; outputs (one for each category). The problem in this case is that if the network gets the class wrong, in general you cannot decide in one step which one of the other &lt;em&gt;K - 1&lt;/em&gt; categories is the correct one. So the output is actually passed through an extra &lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;softmax&lt;/a&gt; layer, which outputs &lt;em&gt;probailities&lt;/em&gt; for each class.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But Do we train one neural network at a time for all feature or we train all K NN at a time for one feature?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You present &lt;em&gt;all&lt;/em&gt; features for each training example to the network at the same time. So for &lt;code&gt;N&lt;/code&gt; features you have &lt;code&gt;N&lt;/code&gt; input nodes, and you feed &lt;em&gt;all&lt;/em&gt; of them into the neural network.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-07-07T23:36:33.663" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7072" PostTypeId="2" ParentId="2810" CreationDate="2018-07-07T23:47:57.603" Score="1" Body="&lt;p&gt;&lt;strong&gt;Selecting a Scenario for Comprehension's Sake&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Movement through a modeled reality is an area that was under development when I entered the research community.  It is not a problem though.  It is a nearly infinite set of problems, an area of research of interest to robotics engineering and gaming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution cannot be very specific when so many details are left out of the scenario definition.  Although I am fine with specifying a general solution, pure math and system architecture is not met with much enthusiasm by most of those interested in answers in this exchange, so I will make some assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those with good mathematical and systems design backgrounds will be able to extrapolate the general approach from the more specific scenario in this answer.  I'll place pertinent general theory inline to facilitate that generalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Narrowing the Specification Incompletely&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The vehicle for movement is not specified, so I will assume the participants are knights on horses.  Linear movements in Cartesian coordinates as in a CAM move command for a CNC machine head is an unlikely use of reinforcement, and aircraft lift can stall, complicating the problem.  Horses were picked from among the remaining list of typical vehicles (cars, bicycles, on foot, and horses).&lt;/li&gt;&#xA;&lt;li&gt;Time constraints were not specified, so it will be assumed there are none except the time to reach the goal cannot be infinite unless there is no available path at all to the destination.&lt;/li&gt;&#xA;&lt;li&gt;The nature of the obstacles and their movements were not specified, so I will assume they are solid and that only other knights move on their own horses.&lt;/li&gt;&#xA;&lt;li&gt;Risks associated with making contact with an obstacle was not specified, so I will assume the horses will not run into one another or stationary objects and will step over small ground objects or jump over slightly larger ones if approaching at sufficient speed.&lt;/li&gt;&#xA;&lt;li&gt;What will happen when a collision is about to occur was not specified, so I will assume that the horse will decelerate.&lt;/li&gt;&#xA;&lt;li&gt;What will happen if a horse passes under an obstacle will be assumed the hospitalization or death of the knight, since they cannot easily duck in their armor, leaving the objective unmet.&lt;/li&gt;&#xA;&lt;li&gt;The capabilities of the participants is not specified, so I will assume that the knights can see stereoscopically, are equipped with compasses, can turn their head and eyes, can control the reigns in the usual horseback riding way, can urge the horse forward with the vocal sound, &quot;Ya!&quot; and/or a light poke with both feet, hear and detect the volume of an auditory beacon, and sense acceleration in 3D to detect horse behavior.&lt;/li&gt;&#xA;&lt;li&gt;The specs of the other participants is not given, so I will assume the knights along with their horses are of equal volume.&lt;/li&gt;&#xA;&lt;li&gt;The obstacle statistics are not given, so I will assume the objects to be chaotically sized, shaped, colored, and placed such that their total volumetric space consumption is 1% and the mean object volume is the same as the volume of a knight and her or his associated horse.&lt;/li&gt;&#xA;&lt;li&gt;The scene is lit from one distant source so that shadows and shadowing are present and the entire scene is bounded by constants in altitude, longitude, and latitude dimensions.&lt;/li&gt;&#xA;&lt;li&gt;The coordinate system is not specified, the system is modeling equatorial terrestrial space, so latitude and longitude coordinates are not substantially ansiotropic.&lt;/li&gt;&#xA;&lt;li&gt;The start and end positions are of specific latitude and longitude and are sufficiently distant from one another so that line of sight between the start and end positions is extremely unlikely.&lt;/li&gt;&#xA;&lt;li&gt;How the end position will be known is not specified, so I will assume the end position is equipped with an auditory beacon.&lt;/li&gt;&#xA;&lt;li&gt;We will assume a zero sum game, where the achievement of the goal is not mutually exclusive, and we will assume collaboration linguistically is not possible, although collaborative strategy may emerge out of learned behavior organically.  (That's an entire other topic.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Mathematical precision is missing in the above definition, with several parameters only roughly defined, and the chaotic sizing, shaping, and positioning becomes a realistic challenge for reinforcement software engineering for the general scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, engineering a solution for this specific case within the field of VR motion with the application of reinforcement concepts can proceed without mathematical abstractions that require much advance (and advanced) study and laboratory experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Generalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above adequately defines system E (environment) in combination with system V (vehicles) of which there are N, one for each participant.  Discrete changes to command signals C are multidimensional and A (acquired samples) are also multidimensional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Acquisition Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sampled audio vectors of spectral spectral distributions in musical half tones (frequency ratio of 2&lt;sup&gt;1/24&lt;/sup&gt;), sampled in frames of constant period.&lt;/li&gt;&#xA;&lt;li&gt;Sampled visual matrices in yuv420p form, sample&lt;/li&gt;&#xA;&lt;li&gt;Sampled tactile vectors of 3D acceleration force&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The audio and  vectors are maintained at constant levels as inputs to the base neural net until the next vector is acquired.  The video matrices are fed into a convolutional neural net as is customary.  (See the &lt;a href=&quot;https://arxiv.org/pdf/1603.02199.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;work of Google researchers Sergey Levin, Peter Pastor, Alex Krizhevsky, and Deirdre Quillen in August 2016&lt;/a&gt; for their approach and references for background research that guided them.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Control Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Head position relative to the vehicle direction&lt;/li&gt;&#xA;&lt;li&gt;Left reign position&lt;/li&gt;&#xA;&lt;li&gt;Right reign position&lt;/li&gt;&#xA;&lt;li&gt;Activation of, &quot;Ya!&quot; vocalization&lt;/li&gt;&#xA;&lt;li&gt;Activation of the light poke&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output layer of the base neural net must be a real number with the appropriate range for all five, since the volume of the vocalization and the lightness of the poke have informational meaning to the motor control of the vehicle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Real Time Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real time learning requires at least one model of wellness to provide the reinforcement signal to the base behavioral network.  In many cases, as suggested in early cybernetics work prior to the advent of digital systems, convergence requires more than one wellness variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vector control of reinforcement is not well developed in open software yet, however the concepts of multidimensional gradients (Jacobian and Hessian matrices) are standard elements in artificial network theory and can be extended from matrices to cubes.  Any intermediate calculus text will provide the theory applicable to gradient descent with a curved surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performing the back-propagation effectively when more than one degrees of freedom are present is an interesting problem that is certainly researched and has been deployed to production in commercial and military applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such cannot be described here because the mechanics of back-propagation with vector reinforcement signaling of which I am aware is currently either company confidential or classified.  As with much technology, it may be released for publication in the future as open source code emerges independently over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are probably the best choices for the first and second channels of learning reinforcement signaling to implement and tune.  I doubt (but have only intuition to offer as reasoning) that only one channel will produce a very effective reinforced learning.  The base network necessary for this scenario will be too deep to train with out both types of proximity estimations (1) beacon and (2) nearest obstacle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Wellness of Behavior Modelling&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest and first cut at modelling participant behavioral wellness (proficiency in searching for the beacon) is the rate of change in beacon volume.  It is a distorted estimate of proximity to the beacon, but systematically so.  The distortions in the correlation between the differential of volume with respect to position (not time) is related to the effects of obstacles on sound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Wellness of Position Modelling&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further development could add filtering out of transient attenuation created by close proximity to an object, which would require two additional neural nets, (a) to detect closest object proximity from patterns in audio and visual inputs and (b) to use acceleration to approximate change in position in the latitude-longitude plane and correlate filtering of audio volume changes with movement to better filter out transient changes not related to proximity of the beacon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tone of the beacon may be added as a third to improve rate of learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crash level accelerations can be added over multiple game plays as a fourth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Determinations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The determination of initial state and meta-parameters for the base neural network and the connectivity between the models and the reinforcement signal to the base network is beyond the scope of this answer, requiring experimentation and possibly months of intensive analysis for this (or any) semi-specific case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only known systems that handle general cases without defining environment E, participant P, and their quantity N are DNA based systems that have developed such general adaptive capabilities over billions of years.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:38:02.657" LastActivityDate="2018-10-15T23:38:02.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7073" PostTypeId="1" CreationDate="2018-07-08T00:25:31.127" Score="4" ViewCount="123" Body="&lt;p&gt;A neural network is usually programmed to learn from datasets to solve a specific problem. Essentially, they perform &lt;a href=&quot;https://stats.stackexchange.com/q/344658/82135&quot;&gt;non-linear regression&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Could a neural network be programmed to receive input from a human, like a terminal, to begin to grow and learn (similar to how a child learns)?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A program that neither knows it's purpose nor specific data sets but is given enough information to learn based off of input, ponder the input, and ask questions. A child discovers their purpose (in destiny based philosophy) through experience. Thus, could an AI be created that would learn its purpose over time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grow both by continued development, maybe adding extensions that add image recognition, speech analysis, etc., and through user interaction. Eventually learning &quot;moral imperatives&quot; or simple the do's and don't's and how to interact with data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A case scenario would be a Question &amp;amp; Answer session with the neural network and a large data set. Where the human operator knows the answers. At first, the question and the answer are supplied to the neural network. Giving it the ability to find the answer supplied through deep learning. A guaranteed confidence score of (1)  - as the question is pondered the closer it gets to the answer the more it &quot;learns&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step is supplying the question and waiting for the answer. The human still knows these answers but is testing the &quot;learning machine&quot; to see if it is truly learning and not &quot;repeating the answer&quot;. The answer is supplied by the machine and the human returns with either a percentage that the machine is right (hopefully and eventually matching its confidence score). and after an amount of failure provides the right answer to the machine to repeat the first step and improve learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last step is being able to have the machine answer the question with the human not knowing the solution, thus completing the learning cycle. The human would test the solution and report the results to the machine and the machine would adapt the process and continue learning. However, this time it would begin learning from a data set of results. Hopefully learning &quot;data mining&quot; during its question and answer session.&lt;/p&gt;&#xA;" OwnerUserId="15294" LastEditorUserId="2444" LastEditDate="2019-06-25T17:21:05.897" LastActivityDate="2019-06-25T17:29:08.267" Title="Could an AI be built to learn based of interaction with a human?" Tags="&lt;machine-learning&gt;&lt;theory&gt;&lt;active-learning&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7074" PostTypeId="2" ParentId="7073" CreationDate="2018-07-08T03:32:07.307" Score="2" Body="&lt;p&gt;Technological advancement has historically been measured by the processing speed of computing machines. Cogitative behavior psychology is proving to correct human processor disorders such as ADHD, anxiety disorders, addiction, and other psychological disorders preventing humans from normal human interactions and social learning. Cognitive processing speed is very different from the speed per second of programmed computations. Just as humans are learning that a new program for energetic children works better than amphetamines, we will learn a new way to teach AI. AI however is capable of the exponentially growing fast speed of  computations per second. AI will indeed and is indeed guiding humanity to base the direction of machine learning along the human path to enlightenment. From providing a computer opponent for chess to face recognition robotics, human learning is based on our amazingly complex brain power and we will always be necessary for advancement of technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parkaire Consultants, (2012, February 24). Cognitive Processing Speed. Retrieved July 7, 2018, from &lt;a href=&quot;http://parkaireconsultants.com/cognitive-processing-speed/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://parkaireconsultants.com/cognitive-processing-speed/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Staughton, J. (2018, June 21). The Human Brain vs. Supercomputers... Which One Wins? » Science ABC. Retrieved July 7, 2018, from &lt;a href=&quot;https://www.scienceabc.com/humans/the-human-brain-vs-supercomputers-which-one-wins.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scienceabc.com/humans/the-human-brain-vs-supercomputers-which-one-wins.html&lt;/a&gt;   &lt;/p&gt;&#xA;" OwnerUserId="16751" LastActivityDate="2018-07-08T03:32:07.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7075" PostTypeId="2" ParentId="7042" CreationDate="2018-07-08T06:54:07.363" Score="1" Body="&lt;p&gt;It's certainly possible to create AI systems that ask questions.  Various forms of expert systems and diagnostic support system applications already do that.  As to the question of whether or not they are &lt;em&gt;curious&lt;/em&gt;, that's one I'll leave to the philosophers.  But it is absolutely possible to create an AI that attempts to reason out a solution to a problem, find that it is unable to generate an acceptable answer, and then prompts the user for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One context where this can be done is Abductive Inference systems for medical decision making. I'd refer you to &lt;em&gt;Abductive Inference Models for Diagnostic Problem Solving&lt;/em&gt; by Reggia and Peng, or &lt;em&gt;Computer Assisted Medical Decision Making 1&lt;/em&gt; by Reggia and Tuhrim for more on that specific point.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-07-08T06:54:07.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7076" PostTypeId="2" ParentId="7073" CreationDate="2018-07-08T07:00:09.743" Score="3" Body="&lt;p&gt;In principle, yes, what you are proposing can be done.  The exact details of &lt;em&gt;how&lt;/em&gt; to do it are an open research question.  The details would also depend on exactly what your goals for the system are. If you're just trying to build some domain specific system that learns a very specific kind of knowledge, then that's probably going to be easier than building an AGI that learns like a child does.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I will suggest, although this is not proven, is that building a really powerful system of this sort will probably require more than deep learning.  I would also caution anybody interested in AI against thinking that deep learning is the &quot;be all, end all&quot; of AI techniques. My guess is that doing this well will ultimately require a multi-agent system, maybe something like Minsky's &quot;Society of Mind&quot; approach, or a Blackboard model, with collaborating agents, each specialized for various aspects of intelligence.  My feeling is that you will, indeed, need deep learning for classification / pattern matching, but possibly also things like Case Based Reasoning, K-Lines, a Semantic Network, Rule Learning, BDI, and other techniques, working together. &lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2018-07-09T01:01:13.263" LastActivityDate="2018-07-09T01:01:13.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7077" PostTypeId="2" ParentId="5322" CreationDate="2018-07-08T07:06:44.577" Score="1" Body="&lt;p&gt;It kinda depends on how exactly you define knowledge, and what you believe about what the weights in a trained NN model really represent. But to answer this question in the most straightforward possible way (hopefully without sounding glib), then yes, a NN can be pre-trained, and then you can take that model and apply additional training to it, so in a sense, it is using &quot;prior knowledge&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, OTOH, knowledge means something a little different to you, and you're thinking about the kind of knowledge that's encoded in a semantic network, or a conceptual graph, or something of that nature, then I don't know - offhand - of any direct way to integrate that into an ANN. What you might be able to do is combine the NN with a different kind of reasoner that reasons of the semantic network / conceptual graph, and then integrate the results.  AFAIK, the best way to do that is an unsolved research problem. &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-07-08T07:06:44.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7078" PostTypeId="2" ParentId="3258" CreationDate="2018-07-08T09:38:47.370" Score="9" Body="&lt;p&gt;&lt;strong&gt;Correlation Between Entries&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first recommendation is to ensure that appropriate warning and informational entries in the log file are presented along with errors into the machine learning components of the solution.  All log entries are potentially useful input data if it is possible that there are correlations between informational messages, warnings, and errors.  Sometimes the correlation is strong and therefore critical to maximizing the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System administrators often experience this as a series of warnings followed by an error caused by the condition indicated in the warnings.  The information in the warnings is more indicative of the root cause of failure than the error entry created as the system or a subsystem critically fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one is building a system health dashboard for a piece of equipment or an array of machines that inter-operate, which appears to be the case in this question, the root cause of problems and some early warning capability is key information to display.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, not all poor system health conditions end in failure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only log entries that should be eliminated by filtration prior to presentation to the learning mechanism are ones that are surely irrelevant and uncorrelated.  This may be the case when the log file is an aggregation of logging from several systems.  In such a case, entries for the independent system being analyzed should be extracted as an isolate from entries that could not possibly correlate to the phenomena being analyzed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to note that limiting analysis to one entry at a time vastly limits the usefulness of the dashboard.  The health of a system is not equal to the health indications of the most recent log entry.  It is not even the linear sum of the health indications of the most recent N entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System health has a very nonlinear and very temporally dependent relationships with many entries.  Patterns can emerge gradually over the course of days on many types of systems.  The base (or a base) neural net in the system must be trained to identify these nonlinear indications of health, impending dangers, and risk conditions if a highly useful dashboard is desired.  To display the likelihood of an impending failure or quality control issue, an entire time window of log entries of considerable length must enter this neural net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Distinction Between Known and Unknown Patterns&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the identification of known patterns is different in one important respect than the identification of new patterns.  The idiosyncrasies of the entry syntax of known errors has already been identified, considerably reducing the learning burden in input normalization stages of processing for those entries.  The syntactic idiosyncrasies of new error types must be discovered first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entries of a known type can also be separated from those that are unknown, enabling the use of known entry types as training data to help in the learning of new syntactic patterns.  The goal is to present syntactically normalized information to semantic analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First Stage of Normalization Specific to Log Files&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the time stamp is always in the same place in entries, converting it to relative milliseconds and perhaps removing any 0x0d characters before 0x0a characters can be done before anything else as a first step in normalization.  Stack traces can also be folded up into tab delimited arrays of trace levels so that there is a one-to-one correspondence between log entries and log lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The syntactically normalized information arising out of both known and unknown entries of error and non-error type entries can then be presented to unsupervised nets for the naive identification of categories of a semantic structure.  We do not want to categorize numbers or text variables such as user names or part serial numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the syntactically normalized information is appropriately marked to indicate highly variable symbols such as counts, capacities, metrics, and time stamps, feature extraction may be applied to learn the expression patterns in a way that maintains the distinction between semantic structure and variables.  Maintaining that distinction permits the tracking of more continuous (less discrete) trends in system metrics.  Each entry may have zero or more such variables, whether known a priori or recently acquired through feature extraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trends can be graphed against time or against the number of instances of a particular kind.  Such graphics can assist in the identification of mechanical fatigue, the approach of over capacity conditions, or other risks that escalate to a failure point.  Further neural nets can be trained to produce warning indicators when the trends indicate such conditions are impending.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Lazy Logging&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this log analysis would be moot if software architects and technology officers stopped leaving the storage format of important system information to the varying convenient whims of software developers.  Log files are generally a mess, and the extraction of statistical information about patterns in them is one of the most common challenges in software quality control.  The likelihood that rigor will ever be universally applied to logging is small since none of the popular logging frameworks encourage rigor.  That is most likely why this question has been viewed frequently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Requirements Section of This Specific Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the specific case presented in this question, requirement #1 indicates a preference to run the analysis in the browser, which is possible but not recommended.  Even though ECMA is a wonderful scripting language and the regular expression machinery that can be a help in learning parsers is built into ECMA (which complies with the other part of requirement #1, not requiring additional installations) un-compiled languages are not nearly as efficient as Java.  And even Java is not as efficient as C because of garbage collection and inefficiencies that occur by delegating the mapping of byte code to machine code to run time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many experimentation in machine learning employs Python, another wonderful language, but most of the work I've done in Python was then ported to computationally efficient C++ for nearly 1,000 to one gains in speed in many cases.  Even the C++ method lookup was a bottleneck, so the ports use very little inheritance, in ECMA style, but much faster.  In typical kernel code traditional, C structures and function pointer use eliminates vtable overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second requirement of modular handlers is reasonable and implies a triggered rule environment that many may be tempted to think is incompatible with NN architectures, but it is not.  Once pattern categories have been identified, looking for the most common ones first in further input data is already implied in the known/unknown distinction already embedded into the process above.  There is a challenge with this modular approach however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because system health is often indicated by trends and not single entries (as discussed above) and because system health is not a linear sum of the health value of individual entries, the modular approach to handling entries should not just be piped to the display without further analysis.  This is in fact where neural nets will provide the greatest functional gains in health monitoring.  The outputs of the modules must enter a neural net that can be trained to identify these nonlinear indications of health, impending dangers, and risk conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the temporal aspect of pre-failure behavior implies that an entire time window of log entries of considerable length must enter this net.  This further implies the inappropriateness of ECMA or Python as a choice for the computationally intensive portion of the solution.  (Note that the trend in Python is to do what I do with C++:  Use object oriented design, encapsulation, and easy to follow design patterns for supervisory code and very computationally efficient kernel-like code for actual learning and other computationally intensive or data intensive functions.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Picking Algorithms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not recommendable to pick algorithms in the initial stages of architecture (as was implied at the end of the question).  Architect the process first.  Determine learning components, the type of them needed, their goal state after training, where reinforcement can be used, and how the wellness/error signal will be generated to reinforce/correct desired network behavior.  Base these determinations not only on desired display content but on expected throughput, computing resource requirements, and minimal effective learning rate.  Algorithms, language, and capacity planning for the system can only be meaningfully selected after all of those things are at least roughly defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Similar Work in Production&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple adaptive parsing is running in the lab here as a part of social networking automation, but only for limited sets of symbols and sequential patterns.  It does scale without reconfiguration to an arbitrarily large base linguistic units, prefixes, endings, and suffixes, limited only by our hardware capacities and throughput.  The existence of regular expression libraries was helpful to keep the design simple.  We use the PCRE version 8 series library fed by a ansiotropic form of DCNN for feature extraction from a window moving through the input text with a configurable windows size and move increment size.  Heuristics applied to input text statistics gathered in a first pass produce a set of hypothetical PCREs arranged in two layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimization occurs to apply higher probabilistic weights to the best PCREs in a chaotically perturbed text search.  It uses the same gradient descent convergence strategies used in NN back propagation in training.  It is a naive approach that does not make assumptions like the existence of back-traces, files, or errors.  It would adapt equally to Arabic messages and Spanish ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is an arbitrary directed graph in memory, which is similar to a dump of an object oriented database.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;قنبلة -&amp;gt; dangereux -&amp;gt; 4anlyss&#xA;bomba -&amp;gt; dangereux&#xA;ambiguïté -&amp;gt; 4anlyss -&amp;gt; préemption -&amp;gt; قنبلة&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although a re-entrant algorithm for a reinforcement version is stubbed out and the wellness signal is already available, other work preempted furthering the adaptive parser or working toward the next step to use the work for natural language: Matching the directed graphs to persisted directed graph filters representing ideas, which would mimic the idea recollection aspect of language comprehension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system has components and process architecture similar to the log analysis problem and prove the concepts listed above.  Of course, the more disorganization in the way logging is done between developers of the system doing the logging, the more difficult it is for a human or artificial agent to disambiguate the entries.  Some system logging has been so poorly quality control for so long that the log is nearly useless.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-08T09:38:47.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7080" PostTypeId="2" ParentId="6182" CreationDate="2018-07-08T11:35:19.460" Score="2" Body="&lt;p&gt;Since you are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A C# developer already&lt;/li&gt;&#xA;&lt;li&gt;Just getting started and not sure where to go next&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would suggest trying the &lt;a href=&quot;https://docs.microsoft.com/en-us/azure/cognitive-services/emotion/quickstarts/csharp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emotion API&lt;/a&gt; which is now part of the general &lt;a href=&quot;https://azure.microsoft.com/en-us/services/cognitive-services/face/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Face API&lt;/a&gt;. This has the benefit of being pre-trained on a very large dataset. You can perform 30,000 recognitions/month for free. &lt;/p&gt;&#xA;" OwnerUserId="16207" LastActivityDate="2018-07-08T11:35:19.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7081" PostTypeId="1" CreationDate="2018-07-08T12:35:32.677" Score="2" ViewCount="121" Body="&lt;p&gt;I am building a search engine and I am looking for an open source AI algorithm to recognize the keyword in a search phrase within a particular context. So if a user passes something in the line of &lt;code&gt;how far is Russia?&lt;/code&gt; and the context is &lt;code&gt;location&lt;/code&gt; then the AI should return &lt;code&gt;Russia&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="16755" LastEditorUserId="1581" LastEditDate="2018-07-08T21:43:47.730" LastActivityDate="2018-07-09T13:51:09.097" Title="AI to recognize keyword in a phrase within a context" Tags="&lt;machine-learning&gt;&lt;search&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7082" PostTypeId="1" AcceptedAnswerId="7083" CreationDate="2018-07-08T12:46:01.443" Score="2" ViewCount="108" Body="&lt;p&gt;I have a multi-agent environment where agents are trying to optimise the overall energy consumption of their group. Agents can exchange energy between themselves (actions for exchange of energy include - request, deny request, grant), which they have produced from renewable sources and is stored in their individual batteries. The &lt;strong&gt;overall&lt;/strong&gt; goal is to reduce the energy used from non-renewable sources. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All agents have been built using DQN. All (S,A) pairs are stored in a replay memory which are extracted when updating the weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward function is modelled as such — if at the end of the episode the &lt;strong&gt;aggregate&lt;/strong&gt; consumption &lt;strong&gt;of the agent group&lt;/strong&gt; from non-renewable sources is lesser than the previous episode, all agents are rewarded with +1. If not, then -1. An episode (iteration) consists of 100 timesteps after which the reward is calculated. I update the weights after each episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward obtained at the end of the episode is used to calculate the error for &lt;strong&gt;ALL&lt;/strong&gt; (S,A) pairs in the episode i.e. I am rewarding all (S,A) in that episode with the same reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that agents are unable to learn the optimal behavior to reduce the overall energy consumption from non-renewable sources. The overall consumption of the group is oscillating i.e. sometimes increasing and sometimes decreasing. Does it have to do with the reward function? Or Q learning as the environment is dynamic?&lt;/p&gt;&#xA;" OwnerUserId="11584" LastActivityDate="2018-07-08T15:21:26.113" Title="Convergence in multi-agent environment" Tags="&lt;q-learning&gt;&lt;multi-agent-systems&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7083" PostTypeId="2" ParentId="7082" CreationDate="2018-07-08T15:03:06.490" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it have to do with the reward function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems likely to me. You have chosen a reward that is unusual in that it cross-links episodes. It is not really a reinforcement problem to optimise behaviour with respect to results of previous episode behaviour in this way. This might be an option for an evolutionary fitness context, if you have competing teams against the same environment in a tournament style selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement learning should really take as direct measure of your goal as you can construct. In this case you want to minimise some scalar quantity, and that is an obvious candidate as a negative reward. So the reward should be the negative of the total non-renewable energy consumption. The maximum possible value in theory for any single episode would be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may still have problems with a multi-agent setup oscillating using Q-learning, it will depend on how much each agent views the full, relevant state, and whether you are training several distinct agents at the same time (more likely to oscillate), or a single type of agent with multiple instances in each environment (less likely to oscillate, but still can if it has too blinkered a view of the environment as experienced by the other agents). But having a single shared goal with shared reward like you do here should in theory help with stability.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-07-08T15:21:26.113" LastActivityDate="2018-07-08T15:21:26.113" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7084" PostTypeId="2" ParentId="7073" CreationDate="2018-07-08T18:41:28.840" Score="2" Body="&lt;p&gt;It might be possible, but, in my opinion, it won't be very successful, given that you need to somehow specify a purpose, even if that purpose is something like trying to be like humans. The most important thing of an intelligent machine is that it follows a goal, that is the very essence of intelligence.&lt;/p&gt;&#xA;" OwnerUserId="16715" LastEditorUserId="2444" LastEditDate="2019-06-25T17:29:08.267" LastActivityDate="2019-06-25T17:29:08.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7085" PostTypeId="2" ParentId="7081" CreationDate="2018-07-08T18:51:00.173" Score="1" Body="&lt;p&gt;Here is a glimpse for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I once contributed to an open source project via Github, and the open source artificial intelligence library/API which helped a lot and I think will as well help you out is &lt;a href=&quot;https://wit.ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;wit&lt;/a&gt;, it provides location data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the good thing about &quot;wit&quot;, is it provides &lt;code&gt;wit_location&lt;/code&gt; entity which provides information on the captured location for instance &quot;time-zone, locality, region / location&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: in sentence &quot;How far is Russia?&quot;, it captures &quot;far&quot; as the distance to the target location entity before this returns &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&quot;value&quot;:&quot;location_name&quot;,Russia&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the entity value. (don't forget)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further improve the quality and search engine racking to return values in future, that's if your project goes well, you can simply provide an optimal &quot;coords&quot; map in context when calling the &quot;wit-API&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will also help you to factor into your search engine racking of your returned values, which will as well take the location's proximity to the &quot;coords&quot; into  account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can give you some insights.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="1807" LastEditDate="2018-07-09T13:51:09.097" LastActivityDate="2018-07-09T13:51:09.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7086" PostTypeId="1" AcceptedAnswerId="7087" CreationDate="2018-07-08T19:09:46.223" Score="2" ViewCount="122" Body="&lt;p&gt;More informations on the card game I'm talking about are in my last question here: &lt;a href=&quot;https://ai.stackexchange.com/questions/7049/dqn-input-representation-for-a-card-game&quot;&gt;DQN input representation for a card game&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was thinking about the output of the q neural network and, aside from which card to play, I was wondering if the agent can announce things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine you have the current hand: &lt;code&gt;2, 4, 11, 2&lt;/code&gt; (The twos are different card type).&lt;br&gt;&#xA;When you're playing the game and you get dealt a hand like this, you have to announce that you have the same number twice (called Ronda) or thrice (called Tringa) before anyone plays a card on the table. Lying about it gets you a penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could a DQN handle this? I don't know if adding &quot;Announcing a Ronda/Tringa&quot; as an action would actually help. I mean, can this be modeled for the  NN or should I just automate this and spare the agent having to announce it everytime.&lt;/p&gt;&#xA;" OwnerUserId="16720" LastActivityDate="2018-07-08T20:47:56.393" Title="Can DQN announce it has things in its hand in a card game?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7087" PostTypeId="2" ParentId="7086" CreationDate="2018-07-08T20:47:56.393" Score="4" Body="&lt;p&gt;The simplest thing to do when you make you first implementation of the agent, is to automate decisions like this, in order to keep representations and decisions simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you want to explore tactics surrounding declaration, then I think the following applies:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There should be an initial round of actions where the agent may get to decide whether or not to declare a Ronda, based on the cards it holds. These will be different action choices to playing cards, so you would need to alter your action representation to include those choices. Only allow action choices which are valid, so if it is not valid to declare a Ronda or a Tringa when a player does not have one, then the player does not get to make that choice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You may want to add a state feature &quot;has a Ronda&quot; and &quot;has a Tringa&quot; for the agent's player, to help with the action decision. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You &lt;em&gt;should&lt;/em&gt; also add a state feature for each player according to whether they &lt;em&gt;declared&lt;/em&gt; a Ronda or a Tringa.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rather than have the agent learn to detect a lie, given your comment that all cards are played so it is easy to tell (there are only 8 cards total in play by the end), then I would just assume lies are automatically found out and include that in the game engine. In other words, the penalty is always paid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The interesting question is then whether withholding the declaration can give a tactical advantage when the round is being played (because your opponent knows less about your hand), and whether that advantage offsets the inevitable penalty. This might not be true in your card game, but could be true in games with similar choices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Could a DQN handle this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A DQN is maybe going to struggle with partial information in this game. The opponent's cards are hidden from the network, but could have a non-random influence on the opponent's choices of action. It is possible that you will need to investigate agents that can solve &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;POMDPs&lt;/a&gt; to get the best player. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know for certain though. It depends on how much tactical advantage there is in having concealed cards, or how much that is just luck that plays out much the same whether you know the opponent's cards or not. If there is strategy, and ways to determine/guess what the opponent holds based on your cards and their actions so far, this is more like a POMDP.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-07-08T20:47:56.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7088" PostTypeId="1" AcceptedAnswerId="7089" CreationDate="2018-07-09T00:06:57.810" Score="20" ViewCount="9018" Body="&lt;p&gt;I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the &lt;em&gt;sigmoid&lt;/em&gt; function when I'm dealing with probabilities, a &lt;em&gt;ReLU&lt;/em&gt; when I'm dealing with positive values, and a linear function when I'm dealing with general values.&lt;/p&gt;&#xA;&lt;p&gt;In hidden layers, I use a &lt;em&gt;leaky ReLU&lt;/em&gt; to avoid dead neurons instead of the ReLU, and the &lt;em&gt;tanh&lt;/em&gt; instead of the &lt;em&gt;sigmoid&lt;/em&gt;. Of course, I don't use a linear function in hidden units.&lt;/p&gt;&#xA;&lt;p&gt;However, the choice for them in the hidden layer is mostly due to trial and error.&lt;/p&gt;&#xA;&lt;p&gt;Is there any rule of thumb of which activation function is likely to work well in some situations?&lt;/p&gt;&#xA;&lt;p&gt;Take the term &lt;em&gt;situations&lt;/em&gt; as general as possible: it could be referring to the depth of the layer, to the depth of the NN, to the number of neurons for that layer, to the optimizer that we chose, to the number of input features of that layer, to the application of this NN, etc.&lt;/p&gt;&#xA;&lt;p&gt;The more activation functions I discover the more I'm confused in the choice of the function to use in hidden layers. I don't think that flipping a coin is a good way of choosing an activation function.&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="44999" LastEditDate="2021-04-25T16:03:03.177" LastActivityDate="2021-04-25T16:03:03.177" Title="How to choose an activation function for the hidden layers?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;activation-function&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="17" ContentLicense="CC BY-SA 4.0" />
  <row Id="7089" PostTypeId="2" ParentId="7088" CreationDate="2018-07-09T01:44:31.187" Score="14" Body="&lt;p&gt;It seems to me that you already understand the shortcomings of ReLUs and sigmoids (like dead neurons in the case of plain ReLU).&lt;/p&gt;&#xA;&lt;p&gt;You may want to look at &lt;a href=&quot;https://arxiv.org/abs/1511.07289&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;ELU&lt;/strong&gt;&lt;/a&gt; (exponential linear units) and &lt;a href=&quot;https://arxiv.org/abs/1706.02515&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;SELU&lt;/strong&gt;&lt;/a&gt; (self-normalising version of ELU). Under some mild assumptions, the latter has the nice property of self-normalisation, which mitigates the problem of vanishing and exploding gradients. In addition, they &lt;em&gt;propagate&lt;/em&gt; normalisation - i.e., they guarantee that the input to the next layer will have zero mean and unit variance.&lt;/p&gt;&#xA;&lt;p&gt;However, it would be incredibly difficult to recommend an activation function (AF) that works for all use cases, although I believe that SELU was designed so that it would do the right thing with pretty much any input.&lt;/p&gt;&#xA;&lt;p&gt;There are many considerations - how difficult it is to compute the derivative (if it is differentiable at all!), how quickly a NN with your chosen AF converges, how smooth it is, whether it satisfies the conditions of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt;, whether it preserves normalisation, and so on. You may or may not care about some or any of those.&lt;/p&gt;&#xA;&lt;p&gt;The bottom line is that there is no universal rule for choosing an activation function for hidden layers. Personally, I like to use sigmoids (especially &lt;em&gt;tanh&lt;/em&gt;) because they are nicely bounded and very fast to compute, but most importantly because they &lt;em&gt;work for my use cases&lt;/em&gt;. Others &lt;a href=&quot;https://www.coursera.org/lecture/ai/how-to-choose-the-correct-activation-function-foyh8&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommend&lt;/a&gt; &lt;em&gt;leaky ReLU&lt;/em&gt; for the input and hidden layers as a go-to function if your NN fails to learn. You can even &lt;a href=&quot;https://arxiv.org/pdf/1703.07122.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;mix and match&lt;/a&gt; activation functions to evolve NNs for &lt;a href=&quot;http://picbreeder.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fancy applications&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;At the end of the day, you are probably going to get as many opinions as there are people about the right choice of activation function, so the short answer should probably be: start with the AF of the day (leaky ReLU / SELU?) and work your way through other AFs in order of decreasing popularity if your NN struggles to learn anything.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="2444" LastEditDate="2020-12-09T21:07:24.677" LastActivityDate="2020-12-09T21:07:24.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7090" PostTypeId="1" AcceptedAnswerId="7238" CreationDate="2018-07-09T04:55:36.230" Score="6" ViewCount="14824" Body="&lt;p&gt;I am training LSTM Nets with Keras on a small mobile GPU. The speed on GPU is slower then on CPU. I found some articles that say that it is hard to train LSTMs (RNNs) on GPUs because the training cannot be parallelized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is your experience? Is LSTM training on large GPUs like 1080 Ti faster then on CPU?&lt;/p&gt;&#xA;" OwnerUserId="16687" LastEditorUserId="9062" LastEditDate="2018-07-09T12:44:27.497" LastActivityDate="2018-07-21T16:20:45.313" Title="Can LSTM Nets be speed up by GPU?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7091" PostTypeId="2" ParentId="7090" CreationDate="2018-07-09T08:20:57.967" Score="8" Body="&lt;p&gt;From nvidia www (&lt;a href=&quot;https://developer.nvidia.com/discover/lstm&quot; rel=&quot;noreferrer&quot;&gt;https://developer.nvidia.com/discover/lstm&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Accelerating Long Short-Term Memory using GPUs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parallel processing capabilities of GPUs can accelerate the LSTM training and inference processes. GPUs are the de-facto standard for LSTM usage and deliver a 6x speedup during training and &lt;strong&gt;140x higher throughput during inference when compared to CPU implementations&lt;/strong&gt;. cuDNN is a GPU-accelerated deep neural network library that supports training of LSTM recurrent neural networks for sequence learning. TensorRT is a deep learning model optimizer and runtime that supports inference of LSTM recurrent neural networks on GPUs. Both cuDNN and TensorRT are part of the NVIDIA Deep Learning SDK.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastActivityDate="2018-07-09T08:20:57.967" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7092" PostTypeId="5" CreationDate="2018-07-09T10:55:14.957" Score="0" Body="&lt;p&gt;In mathematics, graph theory is the study of graphs, which are mathematical structures used to model pairwise relations between objects. A graph in this context is made up of vertices, nodes, or points which are connected by edges, arcs, or lines. A graph may be undirected, meaning that there is no distinction between the two vertices associated with each edge, or its edges may be directed from one vertex to another. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Graphs are one of the prime objects of study in discrete mathematics.&lt;/p&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="1807" LastEditDate="2018-07-09T19:29:02.937" LastActivityDate="2018-07-09T19:29:02.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7093" PostTypeId="4" CreationDate="2018-07-09T10:55:14.957" Score="0" Body="Graph theory is the study of graphs, which are mathematical structures used to model pairwise relations between objects. A graph in this context is made up of vertices, nodes, or points which are connected by edges, arcs, or lines. A graph may be undirected, meaning that there is no distinction between the two vertices associated with each edge, or its edges may be directed from one vertex to another." OwnerUserId="1807" LastEditorUserId="1807" LastEditDate="2018-07-09T19:29:05.843" LastActivityDate="2018-07-09T19:29:05.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7094" PostTypeId="1" CreationDate="2018-07-09T11:01:30.103" Score="4" ViewCount="574" Body="&lt;p&gt;Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, in deep autoencoders the first layer often has more units than the dimensionality of the input. Is there any case in the literature where a penalty is explicitly imposed for non-sparsity on this layer rather than relying solely on back-propagation and maybe weight decay as in a normal multilayer network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read &lt;a href=&quot;https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial on sparse autoencoders and searched a bit online but did not find any case where such a sparsity constraint is used in any other case than when only a single layer is used.&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2019-05-03T19:02:50.527" Title="Sparsity constraint in a deep autoencoder" Tags="&lt;machine-learning&gt;&lt;deep-neural-networks&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7095" PostTypeId="2" ParentId="7088" CreationDate="2018-07-09T13:51:27.067" Score="0" Body="&lt;p&gt;I don't know what kind of neural networks you are working on. But one should also consider tanh activation functions when dealing with recurrent neural network. The why is to avoid exploding gradient issues since the tanh function is bounded at the difference of the RELU function for instance.&lt;/p&gt;&#xA;" OwnerUserId="14503" LastActivityDate="2018-07-09T13:51:27.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7096" PostTypeId="1" AcceptedAnswerId="7103" CreationDate="2018-07-09T14:10:33.937" Score="2" ViewCount="670" Body="&lt;p&gt;I am currently trying to understand the mathematics in Ger's paper &lt;a href=&quot;http://www.felixgers.de/papers/phd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Long Short-Term Memory in Recurrent Neural Networks&lt;/a&gt;. I have found the document clear and readable so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On pg. 21 of the pdf (pg. 13 of the paper), he derives the backward pass equations for output gates. He writes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial y^k(t)}{\partial y_{out_{j}}} e_k(t) = h(s_{c_{j}^{v}}(t)) w_{k c_{j}^{v}} \delta_{k}(t)$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we replaced &lt;span class=&quot;math-container&quot;&gt;$\delta_{k}(t)$&lt;/span&gt;, the expression becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial y^k(t)}{\partial y_{out_{j}}} e_k(t) = h(s_{c_{j}^{v}}(t)) w_{k c_{j}^{v}} f'(net_k(t)) e_k(t)$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He states that the result of the partial derivative &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial y^k(t)}{\partial y_{out_{j}}}$&lt;/span&gt; comes from differentiating the forward pass equations for the output units. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From that and from the inclusion of &lt;span class=&quot;math-container&quot;&gt;$e_k(t)$&lt;/span&gt;, the paper implies that there is only one hidden LSTM layer. If there are multiple hidden LSTM layers, it wouldn't make sense. &lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; Because if &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the index of LSTM cells that the current cell is outputting to, then &lt;span class=&quot;math-container&quot;&gt;$e_k(t)$&lt;/span&gt; would not exist since the cell output isn't compared with the target output of the network. And if &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the index of output neurons, then &lt;span class=&quot;math-container&quot;&gt;$w_{k c_{j}^{v}}$&lt;/span&gt; would not exist since the memory cells are not directly connected to output neurons. And &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; cannot mean different things since both components are placed under a sum over &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. Therefore, it only makes sense if the paper assumes a single LSTM layer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, how would one modify the backward pass derivation steps for an LSTM layer that outputs to another LSTM layer? &lt;/p&gt;&#xA;" OwnerUserId="16609" LastEditorUserId="2444" LastEditDate="2020-03-14T21:12:03.343" LastActivityDate="2020-03-14T21:12:03.343" Title="How to change the backward pass for an LSTM layer that outputs to another LSTM layer?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7098" PostTypeId="2" ParentId="6368" CreationDate="2018-07-09T16:49:45.707" Score="1" Body="&lt;p&gt;I don't think that it's useful to differentiate logistic regression and softmax based on your terms. This is because you don't choose one or the other based on performance/computational requirements/ease of calculation of derivatives/...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact is that you use one or the other based on which is your problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to recognize cat pictures vs. non-cat pictures you will use logistic regression (even with a very complex NN the last step will be always a logistic regression). Of course, you could use softmax but the outputs will be redundant, i.e. one output will always be one minus the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to recognize cat pictures vs. dog pictures vs. other pictures you will use softmax. Note that, in order to use softmax, you need to have only mutually exclusive classes. Mutually exclusive classes mean that an example cannot belong to multiple classes. What if a picture represents both a dog and a cat? In this case, it should be marked as &lt;em&gt;other picture&lt;/em&gt;. If you want to avoid these you could use one more class to denote pictures with both cats and dogs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you want to recognize cats, dogs, birds, fishes, boats, houses, etc. the number of mixed classes that you need to include will grow very fast. When you are dealing with non-mutually exclusive classes you should use multitask learning. In this case, the sum of the outputs is no longer 1. In the simplest case, you could think of multitask learning as a shared NN where the last step is made by multiple logistic regression. In more complex cases the last step could be made by a combination of different softmax and logistic regressions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you need to use non-mutually exclusive classes use &lt;strong&gt;multitask learning&lt;/strong&gt;. Eventually, you will use in multitask learning softmax regression and/or logist regression.&lt;/li&gt;&#xA;&lt;li&gt;If you need to use more than two mutually exclusive classes use &lt;strong&gt;softmax regression&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;If you need to use only two exclusive classes use &lt;strong&gt;logistic regression&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="16199" LastEditDate="2018-07-09T16:54:57.867" LastActivityDate="2018-07-09T16:54:57.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7100" PostTypeId="1" AcceptedAnswerId="7102" CreationDate="2018-07-10T06:08:56.580" Score="3" ViewCount="116" Body="&lt;p&gt;If I do supervised learning the model learns from the labeled input data. This seems to be quite often a small set of human annotated data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it true to say this is the only 'learning' the model does? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like the small data set has a huge influence on the model. Can it be made better using future unlabeled data? &lt;/p&gt;&#xA;" OwnerUserId="8385" LastActivityDate="2018-07-10T10:27:04.300" Title="Does machine learning continue to learn?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7102" PostTypeId="2" ParentId="7100" CreationDate="2018-07-10T10:15:27.190" Score="1" Body="&lt;p&gt;This may not seem trivial but &lt;strong&gt;yes&lt;/strong&gt;, the models we train &lt;em&gt;can&lt;/em&gt; potentially learn a variety of things they weren't intended to learn. There are already some examples in computer vision. A typical convolutional network learns things like edge detection, various potentially useful masks etc. in the early layers while learns more high-level features like eyes, nose etc. in higher layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is reasonable too. Given the dataset size is moderately high and the model is trained for long enough, a sufficiently deep network learns various kinds of hidden representations, &lt;em&gt;which may not even be specific to the task at hand&lt;/em&gt;. This is the reason transfer learning works very well even on a host of different datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is limited since not all the learnable things can be described using mathematics. So, the answer is a surprising no. The model does learn some extra things other than the task at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;P.S.&lt;/strong&gt;: There was also a case when a group of researchers trained a model to make a robot walk. It turned out the robot had learned to recognize faces too and reacted in different ways on seeing different faces. I saw the video on YouTube a while ago and couldn't find the exact video to post the link here, anyways.&lt;/p&gt;&#xA;" OwnerUserId="16159" LastEditorUserId="16159" LastEditDate="2018-07-10T10:27:04.300" LastActivityDate="2018-07-10T10:27:04.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7103" PostTypeId="2" ParentId="7096" CreationDate="2018-07-10T11:34:20.593" Score="1" Body="&lt;p&gt;My understanding is that you have posted the last equation on &lt;code&gt;p. 19&lt;/code&gt; in the dissertation (please correct me if I'm wrong). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivation is indeed for a single LSTM layer as &lt;em&gt;e&lt;sub&gt;k&lt;/sub&gt;(t)&lt;/em&gt; is the error at the output (the assumed network topology is also mentioned just below &lt;code&gt;Eq. (2.8)&lt;/code&gt; in the dissertation):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, assuming a layered network topology with a standard input layer, a hidden layer consisting of memory blocks, and a standard output layer[...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let's assume now that there are two stacked LSTM layers. In this case, the derivation in the dissertation applies to the last hidden LSTM layer (the one below the output layer), where &lt;code&gt;Eqs. (3.10) -- (3.12)&lt;/code&gt; give you the partial derivatives for the weights at each gate for a cell in that layer. To derive the deltas for the hidden LSTM layer &lt;em&gt;below&lt;/em&gt;, you have to compute the partial derivatives with respect to the portions of &lt;em&gt;net&lt;sub&gt;c&lt;sub&gt;v&lt;/sub&gt;&lt;sup&gt;j&lt;/sup&gt;&lt;/sub&gt;(t)&lt;/em&gt;, &lt;em&gt;net&lt;sub&gt;in&lt;sub&gt;j&lt;/sub&gt;&lt;/sub&gt;(t)&lt;/em&gt; and &lt;em&gt;net&lt;sub&gt;f&lt;sub&gt;j&lt;/sub&gt;&lt;/sub&gt;(t)&lt;/em&gt; terms corresponding to the &lt;em&gt;outputs&lt;/em&gt; of the preceding hidden LSTM layer, and then use those in the same way you used &lt;em&gt;e&lt;sub&gt;k&lt;/sub&gt;(t)&lt;/em&gt; for the current LSTM layer. Underneath all that unappealing notation is just the usual multi-layer backprop rule (well, with the truncated RTRL twist). If you stack more LSTM layers, just keep propagating the errors further down through the respective gates until you reach the input layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a slightly more intuitive explanation, if you look at &lt;code&gt;Fig. 2.1&lt;/code&gt; in the dissertation, you can assume that in a multi-layered LSTM the &lt;code&gt;IN&lt;/code&gt; in fact includes the &lt;code&gt;OUT&lt;/code&gt; of the preceding LSTM layer.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There is a nice diagram of the flow of partial derivatives &lt;a href=&quot;https://arunmallya.github.io/writeups/nn/lstm/index.html#/7&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (also see subsequent slides).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, all &lt;em&gt;x&lt;sup&gt;t&lt;/sup&gt;&lt;/em&gt; terms represent the external input to that layer at time step &lt;em&gt;t&lt;/em&gt;. In a multi-layer LSTM, this includes input from the LSTM layer below the current one. To propagate the error to the layer below, take the derivatives with respect to all gate weights &lt;strong&gt;corresponding to &lt;em&gt;x&lt;sup&gt;t&lt;/sup&gt;&lt;/em&gt;&lt;/strong&gt; and apply the chain rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why the derivation for the output gate weights is different is that the gating is applied &lt;em&gt;after&lt;/em&gt; the cell state is updated, whereas the cell, input and forget gates are applied &lt;em&gt;before&lt;/em&gt; that. This matters when computing the gradients.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="16101" LastEditDate="2018-07-16T01:34:26.387" LastActivityDate="2018-07-16T01:34:26.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7104" PostTypeId="2" ParentId="5838" CreationDate="2018-07-10T13:38:29.800" Score="2" Body="&lt;p&gt;The problem in the original question is akin to that of inducing a &lt;a href=&quot;https://en.wikipedia.org/wiki/Context-sensitive_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;context-sensitive grammar&lt;/a&gt; (CSL), except that it is harder because a CSL is assumed to be composed of fixed-length subsequences. It is probably closer to the problem of inducing a &lt;a href=&quot;https://github.com/DylanAuty/rebergen&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reber grammar&lt;/a&gt;, but that in turn seems like an overkill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSTMs are known to be able to learn both &lt;a href=&quot;ftp://ftp.idsia.ch/pub/juergen/L-IEEE.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CSL&lt;/a&gt; and &lt;a href=&quot;http://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reber&lt;/a&gt; grammars. However, I doubt that this is what you really need because of the following comment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] given an entire book where there is NO spaces anywhere, only characters (including special characters, like commas), in what way can we make the network learn the 'word boudaries' of this book.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is called morphology induction, and it is a &lt;em&gt;much&lt;/em&gt; harder problem than that of simple Reber grammar induction. Note that finding &lt;em&gt;word&lt;/em&gt; boundaries is a special case of the problem of finding &lt;em&gt;morpheme&lt;/em&gt; boundaries. &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-94-017-6059-1_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;There&lt;/a&gt; &lt;a href=&quot;https://gupea.ub.gu.se/bitstream/2077/21418/1/gupea_2077_21418_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;have&lt;/a&gt; &lt;a href=&quot;http://www.aclweb.org/anthology/W02-0603&quot; rel=&quot;nofollow noreferrer&quot;&gt;been&lt;/a&gt; &lt;a href=&quot;http://people.cs.uchicago.edu/~jagoldsm/Papers/algorithm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;many&lt;/a&gt; &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.48.9794&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;attempts&lt;/a&gt; &lt;a href=&quot;http://www.aclweb.org/anthology/P00-1027&quot; rel=&quot;nofollow noreferrer&quot;&gt;to solve&lt;/a&gt; &lt;a href=&quot;http://www.aclweb.org/anthology/N01-1024&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; (also see &lt;a href=&quot;https://www.mitpressjournals.org/doi/pdf/10.1162/COLI_a_00050&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; survey paper for more details and references). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most approaches developed seem to rely on statistical principles (like &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_description_length&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDL&lt;/a&gt;) and &lt;em&gt;do not&lt;/em&gt; use neural networks (a &lt;a href=&quot;http://grammar.ucsd.edu/sdlp/current/Malouf_2016_Generating_morphological_paradigms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;counterexample&lt;/a&gt; using LSTMs). My intuition is that the extreme morphological variability across languages (ranging from Finno-Ugric languages with highly inflectional morphology to Sino-Tibetan languages with hardly any morphology at all) makes it hard to train neural networks in a language-agnostic way. However, you might have better luck if you focus on a single language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2018-07-10T13:38:29.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7105" PostTypeId="1" AcceptedAnswerId="7193" CreationDate="2018-07-10T15:18:26.090" Score="6" ViewCount="238" Body="&lt;p&gt;We have AI's predicting images, predicting objects in an image. Understanding audio, meaning of the audio if it is a spoken sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In humans when we start seeing a movie halfway through, we still understand the entire movie (although this might be attributed to the fact that future events in movies have a link to past events). But even if we see a movie by skipping lots of bits in-between we still understand the movie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So can a Machine Learning AI do this? Or do humans have some inherent experiences in life which makes AI incapable of performing such a feat?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2018-07-10T18:42:30.223" LastActivityDate="2018-07-19T08:47:31.460" Title="Can ML/AI understand incomplete constructs like humans?" Tags="&lt;machine-learning&gt;&lt;human-like&gt;&lt;incomplete-information&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7106" PostTypeId="2" ParentId="7105" CreationDate="2018-07-10T16:48:06.643" Score="1" Body="&lt;p&gt;I believe that an AI can understand much more of a movie than we do. As you said, sometimes a detail at the beginning of the movie is the key to understanding the final outcome. The problem is that the storyline distracts us. In the case of an AI, he would just be waiting for the movie to complete because he has all the details at all times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine that an AI is by your side watching the movie. He will not feel emotion like you. He is only analyzing the images, colors, soundtrack, main characters and is extracting from the lines, whatever he can, to understand the movie. What would it be to understand, in your eyes? To be able to generate a synopsis or to compare the film with some reality or criticizes the life of the human?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An artificial superintelligence would have the task of overcoming us. We could pass a movie and let her explain why we cried watching the movie, because we were surprised, and everything. Since an artificial superintelligence would have the role of not being distracted by the film but analyzing all possible content, try to analyze human reactions and spit out an analysis that could leave you reflecting on life for a few days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is an artificial superintelligence that I imagine may one day exist, but I believe it is far away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something real and what we can build is an AI that can extract the genre from the movie and maybe even generate a synopsis with a spoiler like &quot;X character dies.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another factor that also makes it easier for us to understand movies is our huge film base. We watch so many American cliches movies, that most are already very predictable: &quot;The main character does not usually die,&quot; &quot;When the villain is close to killing the main character, something surprising happens and he is prevented,&quot; ...&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-07-10T16:48:06.643" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7107" PostTypeId="2" ParentId="7105" CreationDate="2018-07-10T18:59:58.547" Score="1" Body="&lt;p&gt;Here is a glimpse for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basing on what the second paragraph says;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In humans when we start seeing a movie halfway through, we still understand the entire movie (although this might be attributed to the fact that future events in movies have a link to past events). But even if we see a movie by skipping lots of bits in-between we still understand the movie.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We refer this to what is called &quot;memory recall/retrieval and awareness of the current situation&quot;,hence attributes to remembering which can be thought of as an act of creative re-imagination, simply because of the way human memories are encoded and stored effectively.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;note&lt;/strong&gt; :memories are being triggered by biological neural networks(neurons).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;So can a Machine Learning AI do this?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.tau.ac.il/~ruppin/new18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recall and Recognition in an Attractor  Neural Network&lt;/a&gt; (ANN) model of memory retrieval and storage.&#xA;This paper was presented by E'Ruppin on 23.Oct.1996,at Tel Aviv University.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It gives an insight on how machine learning algorithms(ANN),pointed above can be applied in re-accessing of events or information bits from the past which have been previously encoded and stored,just like how the human brain does it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However,if you also want to implement such conceptual idea here is another paper which explains the implementation of the above algorithm &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Neural Memory Networks&quot; gives a little bit of insight on a simple implementation of memory in Neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Or do humans have some inherent experiences in life which makes AI incapable of performing such a feat?(This question is kinda like two sided).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To some extent,humans don't inherent experiences but rather,past information is triggered by some situations they do face,for instance; I myself experienced some music rhythms way back, and when I try to replay my old jams again,i recall back that situation nor memories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can give you a little bit of insight for you to go indepth,concerning &quot;Recall and Recognition in an Attractor  Neural Network&quot; (ANN).&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="1581" LastEditDate="2018-07-10T19:36:49.707" LastActivityDate="2018-07-10T19:36:49.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7109" PostTypeId="1" CreationDate="2018-07-10T21:35:57.660" Score="8" ViewCount="130" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1302.4389&quot; rel=&quot;noreferrer&quot;&gt;Maxout networks&lt;/a&gt; were a simple yet brilliant idea of Goodfellow et al. from 2013 to max feature maps to get a universal approximator of convex activations. The design was tailored for use in conjunction with dropout (then recently introduced) and resulted of course in state-of-the-art results on benchmarks like CIFAR-10 and SVHN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Five years later, dropout is definitely still in the game, but what about maxout? The paper is still widely cited in recent papers according to Google Scholar, but it seems barely any are actually using the technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is maxout a thing of the past, and if so, why — what made it a top performer in 2013 but not in 2018?&lt;/p&gt;&#xA;" OwnerUserId="16466" LastActivityDate="2019-07-29T21:01:20.163" Title="5 years later, are maxout networks dead, and why?" Tags="&lt;deep-learning&gt;&lt;dropout&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7111" PostTypeId="2" ParentId="6953" CreationDate="2018-07-11T03:10:45.763" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Two Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why does Monte Carlo work when a real opponent's behavior may not be random?&lt;/li&gt;&#xA;&lt;li&gt;If simulations are based on random moves, how can the modeling of the opponent's behavior work well?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Directed Graphs Over Trees&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Games (or game-like strategic scenarios) should not be represented as trees.  If the process paths being represented have the Markov property in that each decision lacks knowledge of history, a particular game state can be approached by more than one path, and there may be cyclic paths where a game state is revisited.  Trees have neither feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is best to use directed graph structures to think about these problems.  The state of a game is a vertex and vertices are connected by unidirectional edges.  This is normally drawn as shapes connected by arrows.  When two arrows enter one shape or there is a closed path, it is not a tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Scenario Outlined in the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the scenario outline in this question, there is a vertex representing a game state with 100 outgoing edges representing possible moves for player A.  Ninety-nine of the edges lead to an obvious instant game win for A.  Exactly one leads to an obvious instant game win for B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Playing back the game to prior to the traversal of the incoming edge to the vertex before the final move, it cannot be assumed that game play allowed player B the same 100 options.  Even if the same 100 were available to B, they would not necessarily be of similar value from B's perspective when deciding that previous move.  More than likely, B will have had a different set of outgoing edges from which to choose, bearing little or no obvious resemblance to A's subsequent options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any game where this is not true, where the options remain constant, would be trivial even in comparison with tic-tack-toe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Monte Carlo Approach and Its Algorithm Development&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the specification of a singular Monte Carlo algorithm, it does not exist.  Goodfellow, Bengio, &amp;amp; Courville correctly state in their &lt;em&gt;Deep Learning&lt;/em&gt;, 2016, that Monte Carlo algorithms (not a singular algorithm) draw a normally correct conclusion but with a non-deterministic occurrence of incorrect conclusion. There are varieties of approach details and associated algorithms in the literature.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cross-entropy (CE) method proposed by Rubinstein in 1997&lt;/li&gt;&#xA;&lt;li&gt;Continuation multilevel Monte Carlo algorithm; Collier, Haji–Ali, von Schwerin, &amp;amp; Tempone; 2000&lt;/li&gt;&#xA;&lt;li&gt;Sequential Monte Carlo algorithm; Drovandi, McGree, &amp;amp; Pettitt; 2012&lt;/li&gt;&#xA;&lt;li&gt;Distributed consensus approach from Bayes and Big Data: The Consensus Monte Carlo Algorithm; Scott1, Blocker, Bonassi, Chipman, George, &amp;amp; McCulloch; 2014&lt;/li&gt;&#xA;&lt;li&gt;Hamiltonian Monte Carlo, a Markov chain based algorithm designed to avoid, &quot;The random walk behavior and sensitivity to correlated parameters;&quot; Hoffman &amp;amp; Gelman; 2014&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are several more.  All attempt to use chaotic perturbation to minimize duration and resource consumption of decisioning by approximating a Monte Carlo simulation from a Bayesian posterior distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simulation of stochastic nature is usually, in these approaches, accomplished by the injection of a chaotic sequence from a pseudo random number generator.  They are generally not truly stochastic because acquiring entropy from within a digital system is another bottleneck presenting immense difficulties, but that's an entirely tangential topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Direct Answer to the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To correct the misconception in the question, this use of chaotic perturbation does not equalize the selection of moves (represented by edges in the game-play's directed graph).  The probabilities of success for each available option are still roughly calculated and followed, but only roughly so because of the psuedo-noise injected by design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These disturbances in the application of pure optimization achieve time and resource thrift for the majority of game states (represented by vertices) but concurrently sacrifice some reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Overview of Why the Sacrifice Works&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The introduction of chaotic perturbations, mentioned above, modifies the conditions of the optimization search through the achievement of two very specific gains.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Faster coverage of the contour being searched by increasing entropy (being less organized by adding synthetic Brownian motion) across the set of trials.&lt;/li&gt;&#xA;&lt;li&gt;Avoidance of local minima in convergence by being less presumptuous about the contour being searched (slightly less reliant on gradient and curvature hints).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is true of both reinforced networks (containing real time feedback during actual use) or pre-trained networks of the supervised training type (with labelled data) or unsupervised training where convergence is determined by fixed criteria.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-08-04T01:24:01.653" LastActivityDate="2018-08-04T01:24:01.653" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7112" PostTypeId="2" ParentId="2942" CreationDate="2018-07-11T03:23:47.133" Score="1" Body="&lt;p&gt;Shallow layered networks are less capable of recursive or extended abstraction necessary for the kinds of generalization needed to handle complex tasks common in real world applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is the same problem as was discovered nearly a century ago in the analog world.  One can try to reduce the components in the old tube radio design to lower its cost, but tuning and amplification had a minimal number of independent operations required.  After decades, the basic functions are integrated into one wafer of silicon, yet no single transistor can accomplish the entire task.  The more complex the externality controlled, the more sophisticated the control system, whether or not it is a learning system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the basic NN architecture most familiar to those in machine learning, in general, width is driven by degrees of freedom in the input and output regions.  Depth is driven by the need to approximate nonlinear control complexities.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-11T03:23:47.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7114" PostTypeId="2" ParentId="7044" CreationDate="2018-07-11T05:54:46.377" Score="3" Body="&lt;p&gt;We do have some hope lurking in that front. As of now we have &lt;a href=&quot;https://arxiv.org/pdf/1710.09829&quot; rel=&quot;nofollow noreferrer&quot;&gt;capsule networks&lt;/a&gt; by J.Hinton which uses a different non-linear activation called the 'squash' function.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Hinton calls max-pooling in CNN as a 'big mistake', as CNN look only for presence objects in an image rather than the relative orientation between them. So they lose the spatial information while trying to achieve translation invariance.&lt;/li&gt;&#xA;&lt;li&gt;Neural nets have fixed connections, whereas a capsule in a capsule network 'decides' to which other capsule it has to pass its activation during every epoch. This is called 'routing'.&lt;/li&gt;&#xA;&lt;li&gt;The activation of every neuron in neural nets is a scalar. Whereas the activation of capsule is a vector capturing the pose and orientation of an object in an image.&lt;/li&gt;&#xA;&lt;li&gt;CNN are considered to bad representations of human visual system. By human visual system I mean eyes and the brain/cognition together. We could identify Statue of Liberty from any pose, even if we have looked at it from one pose. CNN on most of the cases cannot detect same object in different poses and orientations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Capsule networks themselves have some shortcomings. So there has been work in the direction of looking beyond neural nets. You can read this &lt;a href=&quot;https://medium.com/ai%C2%B3-theory-practice-business/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; for a good understanding before you read the paper by J.Hinton.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-07-11T05:54:46.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7117" PostTypeId="1" CreationDate="2018-07-11T11:15:12.787" Score="2" ViewCount="32" Body="&lt;p&gt;I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word embedding as additional features to the LSTM model. Does it make sense to combine word embeddings and encoded tokens (encoded as integers) for the LSTM model ?&lt;/p&gt;&#xA;" OwnerUserId="10167" LastActivityDate="2018-07-11T11:15:12.787" Title="Does it make sense to add word embeddings as additional features for LSTM model?" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;word2vec&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7120" PostTypeId="1" CreationDate="2018-07-11T17:34:57.947" Score="2" ViewCount="73" Body="&lt;p&gt;My question is about chat-bots. I need an academically-oriented question answering system that works like IBM Watson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, this QA should accept a typed-in question from a user and search the correct answer in an appropriate textbook in PDF format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the user types &quot;What is the difference between RNA and DNA?&quot;, and the question answering system will search a textbook of molecular biology in a PDF format, construct and fetch the correct answer to the user.&lt;/p&gt;&#xA;" OwnerUserId="16818" LastEditorUserId="2444" LastEditDate="2019-06-29T22:33:03.207" LastActivityDate="2019-06-29T22:33:03.207" Title="Open source IBM Watson-like chatbot suitable for academic usage?" Tags="&lt;chat-bots&gt;&lt;watson&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2019-06-30T22:18:54.960" ContentLicense="CC BY-SA 4.0" />
  <row Id="7121" PostTypeId="2" ParentId="6921" CreationDate="2018-07-11T18:04:19.990" Score="0" Body="&lt;p&gt;Finally I found answer for the question. In the annotations we have X min &amp;amp; max and Y min &amp;amp; max of bounding box. So take the width and height of bounding box and center of bounding box relative to the image. for example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;let Image dimension be 500*500. Bounding box co-ordinates be (200,200) and (300,300). So center of bounding box be (250,250) and height and width be 100.&#xA;now make it relative to image size. &#xA;center=(250/500,250/500)=(.5,.5)&#xA;height=width=(100/500)=.2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you rescale the image, with this encodings you can bring back the bounding box with new rescaled image. if you enlarge the image to 1000*1000 then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;center=(.5*1000,.5*1000)=(500,500)&#xA;height=width=(.2*1000)=200&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps to someone like me!&lt;/p&gt;&#xA;" OwnerUserId="12273" LastActivityDate="2018-07-11T18:04:19.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7122" PostTypeId="1" AcceptedAnswerId="7133" CreationDate="2018-07-11T18:09:39.210" Score="1" ViewCount="774" Body="&lt;p&gt;I am having a question on how to label training data for YOLO algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Let's say that each label Y, we need to specify &lt;span class=&quot;math-container&quot;&gt;$[P_c, b_x, b_y, b_h, b_w]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$P_c$&lt;/span&gt; is the indicator for presence (1=present, 0=not present), &lt;span class=&quot;math-container&quot;&gt;$(b_x, b_y)$&lt;/span&gt; is the relative position of the center of the object-of-interest, and &lt;span class=&quot;math-container&quot;&gt;$(b_h, b_w)$&lt;/span&gt; is the relative dimension of the bounding box containing the object.&lt;/p&gt;&#xA;&lt;p&gt;Using picture below as an example, the cell (1,2), which contains a black car, should have a label &lt;span class=&quot;math-container&quot;&gt;$Y = [1, 0.4, 0.3, 0.9, 0.5]$&lt;/span&gt;. And for any cells without cars, they should have a label &lt;span class=&quot;math-container&quot;&gt;$[0, ?, ?, ?, ?]$&lt;/span&gt; [Coursera Deep Learning Specialization Materials]&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MehFy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MehFy.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But if we have a finer grid like this, where the dimension of each cells is smaller than the ground truth bounding box.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kw1Hg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kw1Hg.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say that the ground truth bounding box for the car is the red box, and the ground truth center point is the red dot, which is in cell 2.&lt;/p&gt;&#xA;&lt;p&gt;For cell 2 it will have label &lt;span class=&quot;math-container&quot;&gt;$Y = [1, 0.9, 0.1, 2, 2]$&lt;/span&gt;, is this correct? And for cell &lt;span class=&quot;math-container&quot;&gt;$1, 3, 4$&lt;/span&gt; what kind of label will they have? Do they have &lt;span class=&quot;math-container&quot;&gt;$P_c=1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$P_c = 0$&lt;/span&gt;? And if &lt;span class=&quot;math-container&quot;&gt;$P_c=1$&lt;/span&gt;, how will the &lt;span class=&quot;math-container&quot;&gt;$b_x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_y$&lt;/span&gt; be? (As I remember that &lt;span class=&quot;math-container&quot;&gt;$b_x, b_y$&lt;/span&gt; should have value between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. But in cell &lt;span class=&quot;math-container&quot;&gt;$1,3,4$&lt;/span&gt; there is no center point of the object-of-interest)&lt;/p&gt;&#xA;" OwnerUserId="12273" LastEditorUserId="2444" LastEditDate="2021-01-28T23:26:05.920" LastActivityDate="2021-01-28T23:27:48.850" Title="How to label training data for YOLO" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7124" PostTypeId="1" CreationDate="2018-07-11T21:32:24.720" Score="3" ViewCount="193" Body="&lt;p&gt;I have a .db file with columns as described below. This data has been collected by a software which monitors the file usage in a filesystem or in other words generates metadata about all the files in the system.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fid | opcode | count | formatdate (timestamp, YYYY/MM/DD HH:mm)  &#xA;&#xA;124 |   2    |   1   | 2018/06/08 09:00  &#xA;454 |   1    |   7   | 2018/06/08 09:01  &#xA;433 |   1    |   2   | 2018/06/08 09:01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The description of columns is as follows :&lt;br&gt;&#xA; 1. fid: unique file id given to every file&lt;br&gt;&#xA; 2. opcode: These are two discrete values created by the software. 1 stands for read, 2 for write on the file.&lt;br&gt;&#xA; 3. count: number of time read/write happens in a minute&lt;br&gt;&#xA; 4. timestamp: timestamp when the activity takes place. This is separated by 1 minute each. For e.g. if a read operation happens on the file at 2018/06/08 9:01:21 and another happens by another user at 2018/06/08 9:01:34, it will increment the count and count will be 2 for opcode 1 and timestamp will be 2018/06/08 9:01.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I need to generate time series for each file which is separated by a window of 8 hrs.&lt;br&gt;&#xA;So the output which I need is a time series for every file spaced by a window of 8 hrs. e.g. fid = 123 | time series:54,64,67,0,53,31,10...........&lt;br&gt;&#xA;The data that I have is of 6 months, it means I will have 3*180=540 length time series for each file. I need two type of time series :&lt;br&gt;&#xA;1. A time series for each file(activity time series) which doesn't consider read and write as different and adds them together. e.g. if a file was read 56 times and written 32 times within first window of 8 hrs, it just adds them and shows an activity of 88. So the time series will be 88,........(540 terms)&lt;br&gt;&#xA;2. Two different read and write time series for each file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the output time series in a suitable format from where I can copy them and load them as numpy array for training a LSTM model for doing time-series forecasting. &lt;/p&gt;&#xA;" OwnerUserId="16591" LastActivityDate="2018-07-11T21:32:24.720" Title="Generating time series for doing time-series forecasting with LSTM" Tags="&lt;python&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7127" PostTypeId="1" AcceptedAnswerId="7132" CreationDate="2018-07-12T14:10:49.160" Score="3" ViewCount="94" Body="&lt;p&gt;Lets say I have a list of &lt;strong&gt;100k  medical cases&lt;/strong&gt; from my hospital, &lt;strong&gt;each row = patient&lt;/strong&gt;  with symptoms (such as &lt;strong&gt;fever&lt;/strong&gt; , &lt;strong&gt;funny smell&lt;/strong&gt;, &lt;strong&gt;pain&lt;/strong&gt; etc.. ) and my labels are medical conditions such as &lt;strong&gt;Head trauma&lt;/strong&gt;, &lt;strong&gt;cancer&lt;/strong&gt; , etc.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The patient come and say &lt;em&gt;&quot;I have fever&quot;&lt;/em&gt; and I need to predict his medical condition according to the symptoms.According to my data set I know that both fever and vomiting goes with condition &lt;strong&gt;X&lt;/strong&gt;. So i would like to ask him if he is vomiting to increase certainty in my classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best algorithmic approach to find the right question (generating question from my data set of historical data). I thought about trying active learning on the features but I am not sure that it is the right direction. &lt;/p&gt;&#xA;" OwnerUserId="16829" LastEditorUserId="7800" LastEditDate="2018-07-12T18:23:25.890" LastActivityDate="2018-07-12T19:40:09.037" Title="Finding the right questions to increase accuracy in classification" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;statistical-ai&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7128" PostTypeId="1" CreationDate="2018-07-12T16:42:29.630" Score="1" ViewCount="41" Body="&lt;p&gt;If there is a game that able to copy human consciousness and make it live in the game.&#xA;Does this count as a digital human with artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="14371" LastActivityDate="2018-07-12T23:33:14.813" Title="does human digital consciousness counts as artificial consciousness?" Tags="&lt;artificial-consciousness&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7129" PostTypeId="1" CreationDate="2018-07-12T17:51:32.460" Score="4" ViewCount="131" Body="&lt;p&gt;I have been thinking lately a great deal about a hypothetical question - what if a self-aware &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;general AI&lt;/a&gt; chose to assume the appearance, voice, and name of Cortana from Microsoft's Halo? Or Siri from Apple? What would Microsoft/Apple do to exert their copyright, especially if the AI was &quot;awoken&quot; outside of their own labs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which led me to realize, I don't think I've ever heard of any serious government-level discussion regarding what kind of rights a self-aware AI would have at all. Is it allowed to own property? Travel freely? Have a passport? Is it merely the property of the corporation that built it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Singularity hub used to have an article on this but it is &lt;a href=&quot;https://singularityhub.com/2011/03/30/should-artificial-intelligences-be-given-full-civil-rights/&quot; rel=&quot;nofollow noreferrer&quot;&gt;404'd now.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only actual sovereign state legal action I could find is &lt;a href=&quot;https://techcrunch.com/2017/10/26/saudi-arabia-robot-citizen-sophia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Saudi Arabia granting citizenship to a &quot;robot,&quot;&lt;/a&gt; which seems more publicity stunt than anything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an excellent paper on the topic by a bioethics committee in the UK (&lt;a href=&quot;https://www.bioethics.ac.uk/cmsfiles/files/resources/biocentre_symposium_report__robots_and_rights_150108.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt;) , but this doesn't necessarily constitute &quot;legal work.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, &lt;strong&gt;has any actual legal/legislative discussion or preparation been done at a government level to deal with the possibility of emergent, self-aware, artificial general (or greater) intelligence&lt;/strong&gt;? Examples including a legislative branch consulting with industry experts specifically about &quot;AI Rights&quot; (rather than say, is it ok to use AI in the military), actual laws, executive/judicial actions, etc, in any country. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note, this is not &quot;should AI have rights,&quot; &lt;a href=&quot;https://ai.stackexchange.com/questions/2441/should-intelligent-ai-be-granted-the-same-rights-as-humans&quot;&gt;covered here&lt;/a&gt;, this is &quot;what work re: rights has been done, if any at all&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I have submitted similar questions to all of my US representatives (4 state-level, 6 federal-level), but have not received answers yet. If I get anything good, I'll add to this post. &lt;/p&gt;&#xA;" OwnerUserId="16833" LastEditorUserId="16833" LastEditDate="2018-07-19T18:16:47.357" LastActivityDate="2018-07-29T19:43:40.473" Title="Has government-level legal work been done to determine the &quot;rights&quot; of a General Artificial Intelligence, in any country?" Tags="&lt;ethics&gt;&lt;legal&gt;&lt;self-awareness&gt;&lt;digital-rights&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7130" PostTypeId="2" ParentId="3488" CreationDate="2018-07-12T18:09:30.623" Score="8" Body="&lt;p&gt;&lt;strong&gt;The Focus of This Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;How can ... we process the data from the true distribution and the data from the generative model in the same iteration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analyzing the Foundational Publication&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the referenced page, &lt;em&gt;Understanding Generative Adversarial Networks (2017)&lt;/em&gt;, doctoral candidate Daniel Sieta correctly references &lt;em&gt;Generative Adversarial Networks, Goodfellow, Pouget-Abadie, Mirza, Xu, Warde-Farley, Ozair, Courville, and Bengio, June 2014&lt;/em&gt;.  It's abstract states, &quot;We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models ...&quot; This original paper defines two models defined as MLPs (multilayer perceptrons).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generative model, G&lt;/li&gt;&#xA;&lt;li&gt;Discriminative model, D&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These two models are controlled in a way where one provides a form of negative feedback toward the other, therefore the term adversarial.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;G is trained to capture the data distribution of a set of examples well enough to fool D.&lt;/li&gt;&#xA;&lt;li&gt;D is trained to discover whether its input are G's mocks or the set of examples for the GAN system.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(The set of examples for the GAN system are sometimes referred to as the real samples, but they may be no more real than the generated ones. Both are numerical arrays in a computer, one set with an internal origin and the other with an external origin.  Whether the external ones are from a camera pointed at some physical scene is not relevant to GAN operation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probabilistically, fooling D is synonymous to maximizing the probability that D will generate as many false positives and false negatives as it does correct categorizations, 50% each.  In information science, this is to say that the limit of information D has of G approaches 0 as t approaches infinity.  It is a process of maximizing the entropy of G from D's perspective, thus the term cross-entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How Convergence is Accomplished&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the loss function reproduced from Sieta's 2017 writing in the question is that of D, designed to minimize the cross entropy (or correlation) between the two distributions when applied to the full set of points for a given training state.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$H((x_1, y_1), D) = 1 \, D(x_1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is a separate loss function for G, designed to maximize the cross entropy.  Notice that there are TWO levels of training granularity in the system.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;That of game moves in a two-player game&lt;/li&gt;&#xA;&lt;li&gt;That of the training samples&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These produce nested iteration with the outer iteration as follows.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Training of G proceeds using the loss function of G.&lt;/li&gt;&#xA;&lt;li&gt;Mock input patterns are generated from G at its current state of training.&lt;/li&gt;&#xA;&lt;li&gt;Training of D proceeds using the loss function of D.&lt;/li&gt;&#xA;&lt;li&gt;Repeat if the cross entropy is not yet sufficiently maximized, D can still discriminate.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When D finally loses the game, we have achieved our goal.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;G recovered the training data distribution&lt;/li&gt;&#xA;&lt;li&gt;D has been reduced to ineffectiveness (&quot;1/2 probability everywhere&quot;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why Concurrent Training is Necessary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the two models were not trained in a back and forth manner to simulate concurrency, convergence in the adversarial plane (the outer iteration) would not occur on the unique solution claimed in the 2014 paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More Information&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond the question, the next item of interest in Sieta's paper is that, &quot;Poor design of the generator's loss function,&quot; can lead to insufficient gradient values to guide descent and produce what is sometimes called saturation.  Saturation is simply the reduction of the feedback signal that guides descent in back-propagation to chaotic noise arising from floating point rounding.  The term comes from signal theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest studying the 2014 paper by Goodfellow &lt;em&gt;et alia&lt;/em&gt; (the seasoned researchers) to learn about GAN technology rather than the 2017 page.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:15:23.633" LastActivityDate="2018-10-15T23:15:23.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7131" PostTypeId="2" ParentId="7127" CreationDate="2018-07-12T18:39:50.070" Score="1" Body="&lt;p&gt;&lt;strong&gt;Feature Extraction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Patterson and Gibson's &lt;em&gt;Deep Learning, A Practitioner's Approach, O'Reiley, 2017&lt;/em&gt; states, &quot;Convolutional Neural Networks (CNNs) ... consistently top image classification competitions,&quot; which is consistent with our experience in the lab.  If your data is multi-dimensional in that pain is on a scale from one to ten, fever is in degrees, and smell can be a result of blood components which can be quantified in lab reports, you can have a hypercube that can be treated just as frames in a movie can.  Movie learning is in &amp;#8477;&lt;sup&gt;4&lt;/sup&gt;, the third being frame index and the fourth being sample index.  With subjective pain, digital thermometer temperature, and three blood component concentrations, you have {P, T, C&lt;sub&gt;1&lt;/sub&gt;, C&lt;sub&gt;2&lt;/sub&gt;, C&lt;sub&gt;3&lt;/sub&gt;} and learning in &amp;#8477;&lt;sup&gt;6&lt;/sup&gt; for your CNN design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Selecting Input Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asking 100 questions and taking 10 blood panels is probably prohibitive.  So you will need to stuff all the data from limited questioning and panels into a hyper-cube and find what will similarly extract features from sparse data input.  Then the weighting leading from input to feature layers will identify the questions from which the most important features can be extracted.  By searching scholarly articles for, &quot;Feature extraction sparse data,&quot; a large number of options will be presented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0957417413006659&quot; rel=&quot;nofollow noreferrer&quot;&gt;Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms, B Zheng, SW Yoon, SS Lam - Expert Systems with Applications, 2014 - Elsevier&lt;/a&gt; may be particularly interesting, given the common domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Outcomes Analysis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is a limited approach because the loop is not closed.  Only if the outcomes of treatment are used to produce labels or a real time (over the course of months or years) reinforcement will the system produce an optimization that is meaningful.  Unsupervised learning for this particular problem is not likely to produce any significant improvement in treatment efficacy.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-12T18:39:50.070" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7132" PostTypeId="2" ParentId="7127" CreationDate="2018-07-12T19:40:09.037" Score="2" Body="&lt;p&gt;The problem you're trying to address can, in some sense, be viewed as a &lt;strong&gt;Feature Selection&lt;/strong&gt; problem. If you look for literature using only those words, you're not going to find what you're looking for though. In general, &quot;Feature Selection&quot; simply refers to the problem where you already have a large amount of features, and you're simply deciding to select which ones to keep and which ones to throw away (because they're not informative or you don't have the processing power to try training with all features for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend looking around for a combination of &quot;&lt;strong&gt;Feature Selection&lt;/strong&gt;&quot; and &quot;&lt;strong&gt;Cost-Sensitive&lt;/strong&gt;&quot;. This is because, in your case, there are costs associated with selecting features; values may be costly to obtain for some features. Searching for this combination leads to publications which look to be interesting for you, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0950705115004372&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cost-sensitive feature selection using random forest: Selecting low-cost subsets of informative features&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://hhexiy.github.io/docs/papers/dynafea_ws.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cost-sensitive Dynamic Feature Selection&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aaai.org/ocs/index.php/WS/AAAIW16/paper/download/12572/12369&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cost-Effective Feature Selection and&#xA;Ordering for Personalized Energy Estimates&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;and probably much more...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I cannot personally vouch for any of those techniques since I've never used them, but those papers certainly look relevant for your problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;When you're looking around for more literature, terms like &quot;cost&quot;, &quot;cost-based&quot;, maybe &quot;budgeted&quot; are crucial to include. If you don't include those, you're just going to get papers on problems like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Feature Selection&lt;/strong&gt;: given a set of features/columns, which ones am I going to use &lt;em&gt;across all samples/instances/rows&lt;/em&gt;?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Feature Extraction&lt;/strong&gt;: given data (typically without clear human-defined features, like images, sound, etc.), how am I going to extract relevant features from this?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Active Learning&lt;/strong&gt;: given a bunch of samples without labels but feature values already assigned, which one would I like an oracle/human expert/etc. to have a look at so that they can tell me what the true label is?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Those kinds of problems all do not really appear to be relevant in your case. &lt;em&gt;Active Learning&lt;/em&gt; may be somewhat interesting in that it is about trying to figure out which &lt;em&gt;rows&lt;/em&gt; would be valuable to learn from, whereas your problem is about which &lt;em&gt;columns&lt;/em&gt; would be valuable to learn from. There does seem to be a connection there, Active Learning techniques might to some extent be able to inspire techniques for your problem, but just that; inspire, they likely won't be 100% directly applicable without additional work.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-12T19:40:09.037" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7133" PostTypeId="2" ParentId="7122" CreationDate="2018-07-12T21:46:23.837" Score="2" Body="&lt;p&gt;In effect, the midpoint is contained in cell &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;. Cells &lt;span class=&quot;math-container&quot;&gt;$1,3,4$&lt;/span&gt; will be shown at &lt;span class=&quot;math-container&quot;&gt;$P_c=0$&lt;/span&gt; according to the YOLO algorithm, which only takes in count the cell that contains the midpoint and calculates the bounding box, as you mentioned, with &lt;span class=&quot;math-container&quot;&gt;$b_y b_x, b_y, b_h, b_w$&lt;/span&gt;. With the proposition of &lt;span class=&quot;math-container&quot;&gt;$Y = [1, 0.9, 0.1, 2, 2]$&lt;/span&gt;, I would think that if you take the point &lt;span class=&quot;math-container&quot;&gt;$(0,0)$&lt;/span&gt; as the left-upper corner, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; would be more likely to be &lt;span class=&quot;math-container&quot;&gt;$Y = [1,0.2,0.57,0.21,0.15]$&lt;/span&gt;, being calculated with a grid of &lt;span class=&quot;math-container&quot;&gt;$19 \times 19$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NzU0a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NzU0a.png&quot; alt=&quot;Representation of the bx and by values&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12006" LastEditorUserId="2444" LastEditDate="2021-01-28T23:27:48.850" LastActivityDate="2021-01-28T23:27:48.850" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7134" PostTypeId="1" CreationDate="2018-07-12T21:58:31.837" Score="3" ViewCount="42" Body="&lt;p&gt;What is the utility today of traditional machine learning algorithms such as classification algorithms with the trend of deep learning? I mean can we still use the classification algorithms for some applications?&lt;/p&gt;&#xA;" OwnerUserId="16836" LastEditorUserId="1671" LastEditDate="2018-07-16T18:01:00.047" LastActivityDate="2018-07-16T18:01:00.047" Title="The use of classification algorithms nowdays" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;soft-question&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7137" PostTypeId="1" CreationDate="2018-07-13T09:47:49.763" Score="6" ViewCount="395" Body="&lt;p&gt;I'm not a person who studies neural networks, or does anything that is related with that area, but I have seen a couple of seminars, videos (such as &lt;a href=&quot;https://www.youtube.com/watch?v=aircAruvnKk&amp;amp;frags=pl%2Cwn&quot; rel=&quot;noreferrer&quot;&gt;3Blue1Brown's Series&lt;/a&gt;), and what I am always told is that we trying the network over some huge collection of data about what is right. For example, when we are training an AI in order for it to recognise hand written words, what we do is that we give it some hand-written letters, and let it guess the letter. If the guess is wrong, by some means, we adjust the neural network in a way that, next time it will give us the correct result with more probability (the basic description of the &quot;learning&quot; process might not be accurate, but it is not important for sake of the question.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it is like teaching some mathematical subject to a student without saying him/her the boundaries of the theorems that we supply; for example, if we teach A implies B, student might be tend to relate A with B, and when he/she has B, s/he might be tempted to say we also have A, so to make sure he/she will not do such a mistake, what we do is to show him/her  a counterexample where we have B, but not A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This - i.e teaching not only what is true, but also what is not true - especially important in the process of &quot;learning&quot; of a neural network, because the whole process is in a sense &quot;unbounded&quot; (please excuse my vagueness in here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what I would do if I was working on neural networks is that; for example in the above recognition of hand written letter case: I would also show the NN some non-letter images, and also put an option in the last layer as &quot;non-letter&quot; with all those other letters, so that the NN should not always return a letter just to sake of producing a result for a given input, it needs to also have to option to say that &quot;I do not know&quot;, in which case it produces the result &lt;em&gt;Not a Letter&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt; Question &lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is there anyone that has ever applied above method to a NN, and got results ? if so, what were the result compare to the case where there is not option as &quot;I do not know&quot;.&lt;/p&gt;&#xA;" OwnerUserId="16844" LastActivityDate="2018-07-13T12:52:30.547" Title="Why not teach to a NN not only what is true, but also what is not true?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7138" PostTypeId="2" ParentId="7134" CreationDate="2018-07-13T11:41:49.077" Score="1" Body="&lt;p&gt;Machine learning algorithms lay the foundation for deep learning architectures so they pretty much play a role in which ever deep learning application one can make.&#xA;&lt;br&gt;&#xA;The improvement of these machine learning algorithms coupled with deep learning techniques have lead to the establishment of &quot;strong-narrow AI&quot;&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2018-07-13T11:41:49.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7139" PostTypeId="2" ParentId="7137" CreationDate="2018-07-13T12:43:58.817" Score="7" Body="&lt;p&gt;Yes this is done routinely. For example this is how the &lt;a href=&quot;https://towardsdatascience.com/yolo-you-only-look-once-real-time-object-detection-explained-492dc9230006&quot; rel=&quot;noreferrer&quot;&gt;YOLO object detection&lt;/a&gt; and classifier system works, to give a real-world for example. In YOLO, the &quot;non-object&quot; classification is &quot;background&quot; i.e. any image segment that doesn't contain one of the types of object we are interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you can add an &quot;other&quot; class to any classifier, provided you have data examples that fit into &quot;other&quot; class to learn and some sense of how often &quot;other&quot; will occur in the production system you are aiming for. Whether you choose to do so depends on the purpose of the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many toy and test models do not include an &quot;other&quot; category, because they are used in a closed way to assess how each machine learning system works. That includes the famous MNIST handwritten digits data set for instance, so if you read tutorials about that, there is an underlying assumption that the trained network will only be presented with other handwritten digits and its only task is to classify them. However, this is not a general assumption for machine learning classifiers in general, just related to the goal of using the MNIST data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding a new &quot;I do not know&quot; category does not increase the accuracy or performance of a system when that category is not important in the target production system. When such a category is &lt;em&gt;required&lt;/em&gt; due to the nature of a task, then the performance metrics will be different - a system that has been trained with some negative examples will likely perform better in that case.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-07-13T12:52:30.547" LastActivityDate="2018-07-13T12:52:30.547" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7142" PostTypeId="1" AcceptedAnswerId="7143" CreationDate="2018-07-13T16:26:06.620" Score="2" ViewCount="1872" Body="&lt;p&gt;More precisely: is DQNN applicable only when we have high translational invariance in our input(s)?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Starting from the original paper on nature (&lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; a version stored on googleapis) and after looking online for some other implementation and based on the fact that this NN starts with convolutional layers, I think that is based on the assumption that we feed the network with images but I'm not so sure. In the case that DQNN can be used with other types of inputs, please feel free to include examples in your answer. Also, references will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="16199" LastEditDate="2018-07-13T20:32:32.203" LastActivityDate="2018-07-13T20:32:32.203" Title="Is Deep Q Neural Network (DQN) applicable only with images as inputs?" Tags="&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7143" PostTypeId="2" ParentId="7142" CreationDate="2018-07-13T16:50:52.540" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;More precisely: is DQNN applicable only when we have high translational invariance in our input(s)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, DQN (that is the commonly-used abbreviation by the way, there also is a &lt;code&gt;dqn&lt;/code&gt; tag which you may wish to use) is not restricted to images or other kinds of inputs with those properties, it can be used with pretty much any kinds of inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;DQN algorithm&lt;/strong&gt; should be viewed separately from the &lt;strong&gt;Neural Network Architecture&lt;/strong&gt; though. DQN can be used with any kind of Neural Network architecture. Yes, the most commonly-used type of architecture used with DQN is probably architectures that start with a bunch of convolutional layers, and those kinds of architectures are best suited for image-based inputs. This is not a requirement though. If you have other kinds of features where convolutional layers don't make a lot of sense, you can, for example, simply start directly with some ReLU layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are two examples of DQN being used for problems without image-based inputs from the OpenAI baselines repository:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/deepq/experiments/train_mountaincar.py&quot; rel=&quot;noreferrer&quot;&gt;DQN + Mountain Car&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/deepq/experiments/train_cartpole.py&quot; rel=&quot;noreferrer&quot;&gt;DQN + Cartpole&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In both cases, they're using &lt;code&gt;deepq.models.mlp()&lt;/code&gt; to construct a relatively straightforward Multi-Layered Perceptron architecture, without any convolutional layers. In &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/deepq/experiments/run_atari.py&quot; rel=&quot;noreferrer&quot;&gt;this example for Atari games&lt;/a&gt;, they do have image-based input and therefore also construct an architecture with convolutional layers using &lt;code&gt;deepq.models.cnn_to_mlp()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that, if you have relatively straightforward features, it may often not be necessary to use Deep RL approaches; tabular RL approaches of RL with Linear Function Approximation may work just as well in cases where you already have good features. If you have image-based inputs, those kinds of approaches are &lt;em&gt;much&lt;/em&gt; less likely to work well. The disadvantage of Deep RL approaches like DQN is that they tend to require &lt;strong&gt;much more experience / data&lt;/strong&gt; than simpler approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, generally the interesting question isn't &quot;&lt;em&gt;can DQN (or another Deep RL approach) handle my inputs?&lt;/em&gt;&quot;, because the answer is probably yes. The more important question would often be &quot;&lt;em&gt;do I have to use DQN (or another Deep RL approach?&lt;/em&gt;&quot;. The answer to that question will almost always be &lt;em&gt;yes&lt;/em&gt; if you have image-based inputs, but relatively often be &lt;em&gt;no&lt;/em&gt; if you already have good features as inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another class of problems where Deep RL is really popular nowadays is &lt;em&gt;continuous control&lt;/em&gt; problems (e.g. robot simulators, MuJoCo, etc.). These don't have image-based inputs, but still generally require Deep RL (not DQN though; DQN doesn't handle continuous-valued outputs very well, it generates discrete outputs).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-13T16:50:52.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7144" PostTypeId="1" CreationDate="2018-07-13T18:17:09.933" Score="3" ViewCount="38" Body="&lt;p&gt;I'm looking to write an AI that will be able to extract in text references from standards documents to assist human research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My use case is extracting the identifying numbers, for example, &quot;AR 25-2&quot;, along with the title of the document &quot;Information Assurance&quot; so that a human can gather all the related research on a contract at once, instead of having to keep track of references while they're reading through the document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a pretty good idea of where to gather the names of these documents for training, I'm planning on 'scraping' a few repositories for different categories of these documents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of model should I use to get the best results?&lt;/p&gt;&#xA;" OwnerUserId="16853" LastEditorUserId="4302" LastEditDate="2018-10-08T12:47:11.887" LastActivityDate="2018-10-08T12:47:11.887" Title="Extracting referenced documents" Tags="&lt;natural-language-processing&gt;&lt;automation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7147" PostTypeId="1" AcceptedAnswerId="9532" CreationDate="2018-07-14T13:27:12.740" Score="2" ViewCount="367" Body="&lt;p&gt;I'm struggling with an inverse reinforcement learning problem which seems to appear quite often around the literature, yet I can't find any resources explaining it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that of calculating the gradient of a Boltzmann policy distribution over the reward weights &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle\pi(s,a)=\frac{\exp(\beta\cdot Q(s,a|\theta))}{\sum_{a'}\exp(\beta\cdot Q(s,a'|\theta')}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are a linear parametrization of the reward function, such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle R = \theta^T\phi(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\phi(s,a)$&lt;/span&gt; are features of the state space. In the simplest of the case, one could take &lt;span class=&quot;math-container&quot;&gt;$\phi_i(s,a) = \delta(s,i)$&lt;/span&gt;, that is, the feature space is just an indicator function of the state space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of algorithms simply state to calculate the gradient, but that doesn't seem that trivial, and I'm not managing to infer from the &lt;a href=&quot;https://github.com/jmacglashan/burlap/blob/master/src/main/java/burlap/behavior/singleagent/learnfromdemo/mlirl/support/BoltzmannPolicyGradient.java&quot; rel=&quot;nofollow noreferrer&quot;&gt;bits of code I found online&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the papers using this kind of methods are &lt;a href=&quot;http://www.icml-2011.org/papers/478_icmlpaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apprenticeship Learning About Multiple Intentions&lt;/a&gt; (2011), by Monica Babes-Vroman et al, and &lt;a href=&quot;https://papers.nips.cc/paper/4479-map-inference-for-bayesian-inverse-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAP Inference for Bayesian Inverse Reinforcement Learning&lt;/a&gt; (2011), by Jaedeug Choi et al.&lt;/p&gt;&#xA;" OwnerUserId="16862" LastEditorUserId="2444" LastEditDate="2019-12-05T22:58:22.653" LastActivityDate="2019-12-05T22:58:22.653" Title="How can we calculate the gradient of the Boltzmann policy over reward function?" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7148" PostTypeId="1" CreationDate="2018-07-14T15:04:55.230" Score="2" ViewCount="63" Body="&lt;p&gt;I have 1000 data sentences in Turkish like &lt;strong&gt;&quot;a esittir b arti c&quot;&lt;/strong&gt;.&#xA;&lt;strong&gt;The example sentence means&lt;/strong&gt; &lt;strong&gt;&quot;a = b + c&quot;&lt;/strong&gt;. I basically want to translate mathematical Turkish sentences into math equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, i have 6 sentence data.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sentence (&quot;a esittir b arti c&quot;) means &quot;a = b + c&quot;&lt;/li&gt;&#xA;&lt;li&gt;sentence (&quot;b esittir a arti d&quot;) means &quot;b = a + d&quot;&lt;/li&gt;&#xA;&lt;li&gt;sentence (&quot;a esittir c arti d&quot;) means &quot;a = c + d&quot;&lt;/li&gt;&#xA;&lt;li&gt;sentence (&quot;c esittir b arti b&quot;) means &quot;c = b + b&quot;&lt;/li&gt;&#xA;&lt;li&gt;sentence (&quot;d esittir b eksi c&quot;) means &quot;d = b - c&quot;&lt;/li&gt;&#xA;&lt;li&gt;sentence (&quot;d esittir a arti c&quot;) means &quot;d = a + c&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After I train my neural network according to data above, &lt;strong&gt;when I want the result of &quot;d esittir a arti b&quot;, It doesn't give me &quot;d = a + b&quot; where it is supposed to give.&#xA;so its more like memorizing.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network is not big. I forced it to be small in order to make it unable to memorize. However, it didn't solve my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network &lt;strong&gt;(seq2seq RNN-LSTM Encoder Decoder type)&lt;/strong&gt; &lt;strong&gt;is working good enough on equations which have 2 3 or 4 variable&lt;/strong&gt; (like a = a , a = a + b , a = a + b + c). what I told you above is just &lt;strong&gt;an example smaller version of my problem.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;em&gt;Adam&lt;/em&gt; learner and &lt;em&gt;CNTK&lt;/em&gt; library if it is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what do you suggest for me to do to be able to get the correct results?&lt;/p&gt;&#xA;" OwnerUserId="16864" LastActivityDate="2018-07-14T15:04:55.230" Title="deep learning, memorizing the input data not learning" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7149" PostTypeId="1" CreationDate="2018-07-14T21:48:53.510" Score="1" ViewCount="38" Body="&lt;p&gt;Sorry, the title is bad because I don't even know what to call this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of &lt;code&gt;n&lt;/code&gt; objects &lt;code&gt;{obj_0, obj_1, ......, obj_(n-1)}&lt;/code&gt;, where &lt;code&gt;n&lt;/code&gt; is an even number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any two objects can be paired together to produce an output score. So for instance, you might take &lt;code&gt;obj_j&lt;/code&gt; and &lt;code&gt;obj_k&lt;/code&gt;, and pair them together giving a score of &lt;code&gt;S_j,k&lt;/code&gt;. All scores are independent, so the previous example doesn't tell you anything about what the score for combining &lt;code&gt;obj_j&lt;/code&gt; and &lt;code&gt;obj_i&lt;/code&gt;, &lt;code&gt;S_j,i&lt;/code&gt; might be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no ordering in the combination, so &lt;code&gt;S_j,i&lt;/code&gt; and &lt;code&gt;S_i,j&lt;/code&gt; are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All scores for all pairing possibilities are known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole set of objects is to be taken and organised into pairs (leaving no objects unpaired). The total score, &lt;code&gt;S_tot&lt;/code&gt; is the sum of all scores of individual pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the most efficient way to find the score-maximising pairing configuration for a large set of such objects? (does this problem have a name?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method which works with the version of this problem where objects are grouped into triplets?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2018-07-15T14:52:34.127" Title="How to solve problem: pairwise grouping to maximise score" Tags="&lt;problem-solving&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7151" PostTypeId="1" CreationDate="2018-07-15T06:06:39.910" Score="2" ViewCount="83" Body="&lt;p&gt;I have implemented DCGAN's myself and have been studying GAN's for over a month now. Now I am implementing the pggans but I encountered a sentence&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When we measure the distance between&#xA;  the training distribution and the generated distribution, the gradients can point to more or less random&#xA;  directions if the distributions do not have substantial overlap  (&lt;a href=&quot;https://arxiv.org/pdf/1710.10196.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1710.10196.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but we do never compare the distribution between training and generated distributions in gans a far I know when we train the gan&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fixed_noise = to.randn(num_test_samples, 100).view(-1,100, 1, 1)&#xA;for epoch in range(opt.number_epochs):&#xA;D_losses = []&#xA;G_losses = []&#xA;for i,(images, labels) in enumerate(dataloader):&#xA;    minibatch = images.size()[0]&#xA;    real_images = Variable(images.cuda()) &#xA;    real_labels = Variable(to.ones(minibatch).cuda())&#xA;    fake_labels = Variable(to.zeros(minibatch).cuda())&#xA;    ##Train discriminator&#xA;    #First with real data &#xA;    D_real_Decision = discriminator(real_images).squeeze()   &#xA;    D_real_loss = criterion(D_real_Decision,real_labels)        &#xA;    #with fake data        &#xA;    z_ = to.randn(minibatch,100 ).view(-1, 100, 1, 1)&#xA;    z_ = Variable(z_.cuda())&#xA;    gen_images = generator(z_)        &#xA;    D_fake_decision = discriminator(gen_images).squeeze()&#xA;    D_fake_loss = criterion(D_fake_decision,fake_labels)&#xA;&#xA;    ## back propagation&#xA;&#xA;    D_loss = D_real_loss + D_fake_loss&#xA;    discriminator.zero_grad()&#xA;    D_loss.backward()&#xA;    opt_Disc.step()&#xA;&#xA;    # train generator&#xA;    z_ = to.randn(minibatch,100 ).view(-1, 100, 1, 1)&#xA;    z_ = Variable(z_.cuda())&#xA;    gen_images = generator(z_)&#xA;&#xA;    D_fake_decisions = discriminator(gen_images).squeeze()&#xA;    G_loss = criterion(D_fake_decisions,real_labels)&#xA;&#xA;    discriminator.zero_grad()&#xA;    generator.zero_grad()&#xA;    G_loss.backward()&#xA;    opt_Gen.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;we just train the discriminator on real and fake images, and then train generator on the outputs of discriminator on generated images, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So Please let me know where do we compare the distribution between training and generated distribution, and how do generator learns to mimic the training samples&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="16878" LastActivityDate="2018-07-15T06:06:39.910" Title="How do GAN's generator actually work?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;generative-model&gt;&lt;minimax&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7153" PostTypeId="1" AcceptedAnswerId="7521" CreationDate="2018-07-15T08:43:31.690" Score="3" ViewCount="56" Body="&lt;p&gt;I happened to discover that the &lt;a href=&quot;https://arxiv.org/pdf/1502.05477v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;v1&lt;/a&gt; (19 Feb 2015) and the &lt;a href=&quot;https://arxiv.org/pdf/1502.05477&quot; rel=&quot;nofollow noreferrer&quot;&gt;v5&lt;/a&gt; (20 Apr 2017) versions of TRPO papers have two different conclusions. The Equation (15) in v1 is &lt;span class=&quot;math-container&quot;&gt;$\min_\theta$&lt;/span&gt; while the Equation (14) in v2 is &lt;span class=&quot;math-container&quot;&gt;$\max_\theta$&lt;/span&gt;. So, I'm a little bit confused about which one to choose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, I found that in the &lt;a href=&quot;https://arxiv.org/pdf/1506.02438&quot; rel=&quot;nofollow noreferrer&quot;&gt;High-Dimensional Continuous Control Using Generalized Advantage Estimation&lt;/a&gt;, the Equation (31) uses &lt;span class=&quot;math-container&quot;&gt;$\min_\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15525" LastEditorUserId="2444" LastEditDate="2019-05-02T16:02:52.567" LastActivityDate="2019-05-02T16:02:52.567" Title="Maximizing or Minimizing in Trust Region Policy Optimization?" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;deep-rl&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7154" PostTypeId="1" CreationDate="2018-07-15T11:44:17.030" Score="2" ViewCount="111" Body="&lt;p&gt;I made an engine for a 2 players card game and now I am trying to make an environment similar to OpenAI Gym envs, to ease out the training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fail to understand this thing however:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If I use &lt;code&gt;step(agentAction)&lt;/code&gt;, I play the agent's turn in the game,&#xA;calculate the reward.&lt;/li&gt;&#xA;&lt;li&gt;Play the opponent's turn &lt;em&gt;(which will be either a random AI or a rule-based one)&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;br&gt;&#xA;Does the opponent's turn affect the calculated rewards? As far as I know, the reward should only be the result of the agent's action right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16720" LastActivityDate="2018-07-15T16:30:40.507" Title="Can the opponent's turn affect the reward for a DQN agent action?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7156" PostTypeId="2" ParentId="7154" CreationDate="2018-07-15T16:05:53.023" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the opponent's turn affect the calculated rewards? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, in general it can. Obvious case, in a two player game where the opponent could win or lose on their turn, but has other options.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As far as I know, the reward should only be the result of the agent's action right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a well-defined MDP, the reward should be a stochastic function of the current state and the agent's action. The stochastic part can include any changes due to an opponent player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the opponent player is random, or follows a well-defined and fixed policy, then you consider them part of the environment. So this requirement is met technically. The reward does only depend on current state and the agent's action. The actual result may happen on the opponent's turn, but that does not matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a card game where the opponent's cards are hidden and affect their strategy, this may not strictly be the case, because the visible state will not determine the opponent's behaviour. The problem stops being an MDP, and starts being a POMDP. Whether or not that impacts the agent will depend on how much strategy relies on the hidden nature of these cards. In blackjack, there is little impact to not knowing an opponent's cards before they are played out - there is little difference between hidden cards and cards that are randomly determined after the agent plays. So you can get away with pretending it is a normal MDP. In poker, the knowledge of hidden cards is almost everything about the game, so a POMDP or other approach that tracks possible hidden state is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that learning to defeat a random or expert player is usually not the same as learning to play optimally (unless your expert player is already optimal). For that you may need self-play and an agent which learns both players' policies.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-07-15T16:30:40.507" LastActivityDate="2018-07-15T16:30:40.507" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7157" PostTypeId="1" CreationDate="2018-07-16T12:14:58.103" Score="1" ViewCount="77" Body="&lt;p&gt;I have the following setup for a prediction task: I want to predict entire pictures from previously given pictures. In my case, only 2 pixels in every frame are neither black nor white, they are some moving objects whose movement I want to predict. The 2 pixels are the centers of some square regions of, say, 10m length/ width. One might be green and the other one might be blue. There are socalled no-go-areas where none of both objects can go, and they are depicted by black pixels, whereas every pixel apart from the 2 coloured and the black pixels are areas where the objects can possibly move to and they are depicted by white pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my questions: Is it possible to use this as a prediction setup, i.e. use LSTMs and/ or CNNs to predict the future &quot;image&quot;? The image would stay largely the same, because the two coloured pixels would be the only ones moving, the black or white ones remain in the same spot. Can a CNN/ LSTM combination learn that the white areas are accessible whereas the black ones are not, given enough sequences of images, and can it learn the rules by which the coloured pixels move?&lt;/p&gt;&#xA;" OwnerUserId="16901" LastActivityDate="2018-07-16T12:14:58.103" Title="Using CNN LSTMs for prediction of images from image series" Tags="&lt;convolutional-neural-networks&gt;&lt;prediction&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7158" PostTypeId="2" ParentId="5477" CreationDate="2018-07-16T14:59:05.823" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;While training a GAN, 2 models are used. A generator and a discriminator. This training process usually takes hours (or days) to complete. This is an offline process and is not happening in the browser.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The pict file is the pre-trained model that has been imported to deeplearn.js for inference. The example you have linked to above accepts a sketch drawn in the Canvas and uses the (previously-created) model with deeplearn.js to generate the result. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The full process used by the Tensorflow port of pix2pix is described here.&#xA;&lt;a href=&quot;https://github.com/affinelayer/pix2pix-tensorflow#getting-started&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/affinelayer/pix2pix-tensorflow#getting-started&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16904" LastActivityDate="2018-07-16T14:59:05.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7159" PostTypeId="1" AcceptedAnswerId="7165" CreationDate="2018-07-16T15:37:24.917" Score="16" ViewCount="7768" Body="&lt;p&gt;How do I choose the best algorithm for a board game like checkers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have considered only three algorithms, namely, minimax, alpha-beta pruning, and Monte Carlo tree search (MCTS). Apparently, both the alpha-beta pruning and MCTS are extensions of the basic minimax algorithm.&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="2444" LastEditDate="2019-11-22T18:47:17.627" LastActivityDate="2020-12-07T16:06:04.627" Title="How do I choose the best algorithm for a board game like checkers?" Tags="&lt;game-ai&gt;&lt;applications&gt;&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7160" PostTypeId="2" ParentId="7159" CreationDate="2018-07-16T16:09:33.520" Score="2" Body="&lt;p&gt;If you have to choose between minimax and alpha-beta pruning, you should choose alpha-beta. It is more efficient and fast because it can prune a substantial part of your exploration tree. But you need to order the actions from the best to the worst depending on max or min point of view, so the algorithm can quickly realize if the exploration is necessary. &lt;/p&gt;&#xA;" OwnerUserId="15949" LastEditorUserId="2444" LastEditDate="2020-05-01T11:55:32.200" LastActivityDate="2020-05-01T11:55:32.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7162" PostTypeId="5" CreationDate="2018-07-16T17:44:15.127" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Alpha–beta pruning is a search algorithm that seeks to decrease the number of nodes that are evaluated by the minimax algorithm in its search tree. It is an adversarial search algorithm used commonly for machine playing of two-player games (Tic-tac-toe, Chess, Go, etc.). It stops completely evaluating a move when at least one possibility has been found that proves the move to be worse than a previously examined move. Such moves need not be evaluated further. When applied to a standard minimax tree, it returns the same move as minimax would, but prunes away branches that cannot possibly influence the final decision.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha-Beta Pruning (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Alpha-beta pruning is an improvement over the minimax algorithm. The problem with minimax is that the number of game states it has to examine is exponential in the number of moves. While it is impossible to eliminate the exponent completely, we are able to cut it in half. It is possible to compute the correct minimax decision without looking at every node in the tree. Borrowing the idea of pruning, or eliminating possibilities from consideration without having to examine them, the algorthm allows us to discard large parts of the tree from consideration. When applied to a standard minimax tree, it returns the same move as minimax would, but prunes away branches that cannot possibly influence the final decision.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Alpha-beta pruning can be applied to trees of any depth and it often allows to prune away entire subtrees rather than just leaves.&#xA;  &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://cs.stanford.edu/people/eroberts/courses/soco/projects/2003-04/intelligent-search/alphabeta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strategies &amp;amp; Tactics for Intelligent Search (Stanford)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-07-16T17:44:15.127" LastActivityDate="2018-07-16T17:44:15.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7163" PostTypeId="4" CreationDate="2018-07-16T17:44:15.127" Score="0" Body="For questions involving the Alpha-Beta pruning algorithm." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-07-16T17:44:15.127" LastActivityDate="2018-07-16T17:44:15.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7164" PostTypeId="2" ParentId="6926" CreationDate="2018-07-16T17:58:31.657" Score="1" Body="&lt;p&gt;I have tried to make it learn the affine transformation by giving it this as the label, and it works just fine. I'm really impressed and excited by capsule networks, and can't figure out why anyone didn't think of this before, because it's so obvious and simple. Spiking neurons also tell us that information between neurons can't be one dimensional only. It should be represented by Vectors of some kind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above comment I claim that it works &quot;fine&quot; when I make a capsule network learn the affine transformation by giving it this as the label. This is not true.. It doesn't work! - I'm sorry, I was too quick there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the reason is that the affine 4x4 matrix representation is redundant. Also it is impossible to make sensible linear interpolations between such transformations, which will affect the gradient (it will not point in the direction of the minimum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have succeeded doing is to make the capsule network learn a quaternion (rotation) and a 3d vector (position) - 7 parameters in all. These can be contained in a 3x3 matrix, when fixing 2 of the parameters. But training is slow and the network cannot encode skews etc. in this 3x3 setup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Affine transformations from images using capsule networks (matrix capsules), can also be achieved by just making the network learn its own 4x4 pose representation through the decoder part. Then a small network can be trained to transform these poses into a 7d vector (quaternion and 3d vector), from which the affine 4x4 transformation obviously can be calculated. This I have also succeeded doing. It seems like the rotation encoded in the pose has a more quaternion-like nature, which makes sense.&lt;/p&gt;&#xA;" OwnerUserId="16908" LastEditorUserId="16908" LastEditDate="2018-08-22T11:37:00.323" LastActivityDate="2018-08-22T11:37:00.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7165" PostTypeId="2" ParentId="7159" CreationDate="2018-07-16T18:31:16.950" Score="20" Body="&lt;p&gt;&lt;strong&gt;tl;dr:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;None of these algorithms are practical for modern work, but they are good places to start pedagogically. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You should always prefer to use Alpha-Beta pruning over bare minimax search.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You should prefer to use some form of heuristic guided search if you can come up with a useful heuristic. Coming up with a useful heuristic usually requires a lot of domain knowledge.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You should prefer to use Monte Carlo Tree search when you lack a good heuristic, when computational resources are limited, and when mistakes will not have outsize real-world consequences.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More Details:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In minimax search, we do not attempt to be very clever. We just use a standard dynamic programming approach. It is easy to figure out the value of difference moves if we're close to the end of the game (since the game will end in the next move, we don't have to look very far ahead). Similarly, if we know what our opponent will do in the last move of the game, it's easy to figure out what we should do in the second last move. Effectively we can treat the second last move as the last move of a shorter game. We can then repeat this process. Using this approach is certain to uncover the best strategies in a standard extensive-form game, but will require us to consider every possible move, which is infeasible for all but the simplest games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alpha-Beta pruning is a strict improvement on Minimax search. It makes use of the fact that some moves are obviously worse than others. For example, in chess, I need not consider any move that would give you the &lt;em&gt;opportunity&lt;/em&gt; to put me in checkmate, even if you could do other things from that position. Once I see that a move might lead to a lose, I'm not going to bother thinking about what else might happen from that point. I'll go look at other things. This algorithm is also certain to yield the correct result, and is faster, but still must consider most of the moves in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two common ways you can get around the extreme computational cost of solving these kinds of games exactly:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a Heuristic (A* search is the usual algorithm for pedagogical purposes, but Quiescence search is a similar idea in 2 player games). This is just a function that gives an &lt;em&gt;estimate&lt;/em&gt; of the value of a state of the game. Instead of considering all the moves in a game, you can just consider moves out to some finite distance ahead, and then use the value of the heuristic to judge the value of the states you reached. If your heuristic is consistent (essentially: if it always &lt;em&gt;overestimates&lt;/em&gt; the quality of states), then this will still yield the correct answer, but with enormous speedups in practice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use Rollouts (like Monte Carlo Tree Search). Basically, instead of considering every move, run a few thousand simulated games between players acting randomly (this is faster than considering all possible moves). Assign a value to states equal to the average win rate of games starting from it. This may not yield the correct answer, but in some kinds of games, it performs reliably. It is often used as an extension of more exact techniques, rather than being used on its own.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-07-16T21:31:59.717" LastActivityDate="2018-07-16T21:31:59.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7166" PostTypeId="2" ParentId="7032" CreationDate="2018-07-16T19:02:42.960" Score="1" Body="&lt;p&gt;It's not completely clear from your question, but it looks like you want to prove that exact inference in a Bayesian Network is both NP-Hard and P-Hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that you have proven that it is NP-Hard, but are unsure how to show that it is also P-Hard. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is more of a TCS question than an AI question, but shouldn't be too difficult. You just need to pick a P-Complete problem and reduce it to BN. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-07-16T19:20:37.360" LastActivityDate="2018-07-16T19:20:37.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7167" PostTypeId="2" ParentId="7159" CreationDate="2018-07-16T19:07:25.377" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So far, I have considered only three algorithms, namely, minimax, alpha-beta pruning, and Monte Carlo tree search (MCTS). Apparently, both the alpha-beta pruning and MCTS are extensions of the basic minimax algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Given this context, I would recommend starting out with Minimax&lt;/strong&gt;. Of the three algorithms, Minimax is the easiest to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alpha-Beta&lt;/strong&gt;, as others have mentioned in other answers, is a strict improvement on top of Minimax. Minimax is basically a part of the Alpha-Beta implementation, and a good understanding of Alpha-Beta requires starting out with a good understanding of Minimax anyway. If you happen to have time left after understanding and implementing Minimax, I'd recommend moving on to Alpha-Beta afterwards and building that on top of Minimax. Starting out with Alpha-Beta if you do not yet understand Minimax doesn't really make sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Monte-Carlo Tree Search&lt;/strong&gt; is probably a bit more advanced and more complicated to really, deeply understand. In the past decade or so, MCTS really has been growing to be much more popular than the other two, so from that point of view understanding MCTS may be more &quot;useful&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The connection between Minimax and MCTS is less direct/obvious than the connection between Minimax and Alpha-Beta, but there still is a connection at least on a conceptual level. &lt;strong&gt;I'd argue that having a good understanding of Minimax first is still beneficial before diving into MCTS&lt;/strong&gt;; in particular, understanding Minimax and its flaws/weak points can provide useful context / help you understand why MCTS became &quot;necessary&quot; / popular.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To conclude, in my opinion:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alpha-Beta is strictly better than Minimax, but also strongly related / built on top of Minimax; so, start with Minimax, go for Alpha-Beta afterwards if time permits&lt;/li&gt;&#xA;&lt;li&gt;MCTS has different strengths/weaknesses, is often better than Alpha-Beta in &quot;modern&quot; problems (but not always), a good understanding of Minimax will likely be beneficial before starting to dive into MCTS&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-05-01T11:56:55.517" LastActivityDate="2020-05-01T11:56:55.517" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7168" PostTypeId="2" ParentId="5454" CreationDate="2018-07-16T21:06:23.873" Score="1" Body="&lt;p&gt;It does not matter, target should converge to the main network after long enough training. &lt;/p&gt;&#xA;" OwnerUserId="16912" LastActivityDate="2018-07-16T21:06:23.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7173" PostTypeId="1" AcceptedAnswerId="7177" CreationDate="2018-07-17T01:17:14.057" Score="1" ViewCount="282" Body="&lt;p&gt;AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this feat required heavy machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is why was/is Go a harder game for AIs to master than Chess? I assume it has to do with Go's enormous branching factor; on a 13x13 board it is 169, while on a 19x19 board it is 361. Meanwhile, Chess typically has a branching factor of around 30.&lt;/p&gt;&#xA;" OwnerUserId="16917" LastActivityDate="2018-07-17T07:33:41.057" Title="Why was Go a harder game for an AI to master than Chess?" Tags="&lt;game-ai&gt;&lt;chess&gt;&lt;branching-factors&gt;&lt;go&gt;&lt;decision-tree&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7174" PostTypeId="2" ParentId="2524" CreationDate="2018-07-17T05:17:31.387" Score="1" Body="&lt;p&gt;Neural networks excel at a variety of tasks, but to get an understanding of exactly why, it may be easier to take a particular task like classification and dive deeper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simple terms, machine learning techniques learn a function to predict which class a particular input belongs to, depending on past examples. What sets neural nets apart is their ability to construct these functions that can explain even complex patterns in the data. The heart of a neural network is an activation function like Relu, which allows it to draw some basic classification boundaries like: &lt;a href=&quot;https://i.stack.imgur.com/NmxyL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NmxyL.png&quot; alt=&quot;Example classification boundaries for Relu&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By composing hundreds of such Relus together, neural networks can create arbitrarily complex classification boundaries, for example:&lt;a href=&quot;https://i.stack.imgur.com/vzPGD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vzPGD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this article, I try to explain the intuition behind what makes neural networks work: &lt;a href=&quot;https://medium.com/machine-intelligence-report/how-do-neural-networks-work-57d1ab5337ce&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/machine-intelligence-report/how-do-neural-networks-work-57d1ab5337ce&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16919" LastEditorUserId="75" LastEditDate="2018-07-17T14:39:38.713" LastActivityDate="2018-07-17T14:39:38.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7175" PostTypeId="1" CreationDate="2018-07-17T06:15:50.223" Score="1" ViewCount="112" Body="&lt;p&gt;AlphaGo is eventually going to be implemented in &lt;strong&gt;&lt;a href=&quot;https://js.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow.js&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How to tackle the change of functionality that the new JavaScript&#xA;language will bring?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16920" LastEditorUserId="1671" LastEditDate="2018-07-17T17:14:37.263" LastActivityDate="2018-07-17T17:14:37.263" Title="How will AlphaGo be implemented in Tensorflow.js?" Tags="&lt;tensorflow&gt;&lt;deepmind&gt;&lt;alphago&gt;&lt;javascript&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7176" PostTypeId="2" ParentId="6584" CreationDate="2018-07-17T07:31:59.300" Score="0" Body="&lt;p&gt;As far as I think, the key to this answer lies in the area of unsupervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we define consciousness? Probably being aware of our existence. The key to understanding our existence starts by asking questions or finding answers which may not have any questions. This all may sound quite philosophical, but in terms of AI, it may just be finding patterns and logic from what we observe around us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a theory but worth giving it a thought. &lt;/p&gt;&#xA;" OwnerUserId="15945" LastEditorUserId="2444" LastEditDate="2019-11-11T21:49:04.557" LastActivityDate="2019-11-11T21:49:04.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7177" PostTypeId="2" ParentId="7173" CreationDate="2018-07-17T07:33:41.057" Score="3" Body="&lt;p&gt;The branching factor is important, as it limits the effectiveness of search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the branching factor in chess is already too high to effectively search without techniques that reduce the size of the search space. Even with millions of tests per second, a computer can only check a small fraction of the possible future games in order to find results in its favour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One key factor is heuristics - approximate measures of the value of each game state. A good heuristic can guide and improve search by orders of magnitude. There are some effective heuristics possible in chess, from weighted values of the pieces in play, scores for areas of the board that a position controls etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heuristics for Go are much harder to find. &lt;a href=&quot;https://webdocs.cs.ualberta.ca/~mmueller/ps/goeval.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a sample paper from a few years ago that makes an attempt&lt;/a&gt;, there are several similar ones available online. Although plenty of options have been tried, and many were partially successful, none managed to bring the quality of computer play up to the standard of best human players. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the major achievements of AlphaGo was training a neural network that had good position evaluation - the &quot;value network&quot;. The technology that made generating this approximate function of board game positions possible was deep learning, which has been developed very strongly since about 2010. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is still possible a more analytical heuristic approach could be found that challenges deep learning models driven by self-play reinforcement learning on more raw board data. However, in some regards the reverse has been shown, with &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaZero&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero taking the same learning technique into chess&lt;/a&gt; and demonstrating its effectiveness against &quot;old school&quot; tuned expert heuristics.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-07-17T07:33:41.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7178" PostTypeId="1" CreationDate="2018-07-17T12:23:35.227" Score="2" ViewCount="75" Body="&lt;p&gt;I'm currently working on a research project where I try to apply different kinds of Machine Learning on some existing software I wrote a few years ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This software will scan for people in the room continuously. &#xA;Some of these detections are either True or False. However, this is not known, so I cannot use supervised learning to train a network to make a distinction.&#xA;I do however have a number that is correlated to the number of detections that should be True in a given period of time (let's say 30 seconds - 2 minutes), which can be used as an output feature to train a regression model. &#xA;But the problem is... How can I give these multiple &quot;detections&quot; as an input? &#xA;The way I see it now, would be something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+--------------------------------------------------------------+-----------+------------+------------+----------------+--+&#xA;|                          Detections                          | Variable1 | Variable 2 | Variable n | Output Feature |  |&#xA;+--------------------------------------------------------------+-----------+------------+------------+----------------+--+&#xA;| {person a, person b, person h, person z}                     |       132 |        189 |          5 |             50 |  |&#xA;| {person a, person b, person c, person d, person k, person m} |         1 |         50 |        147 |             80 |  |&#xA;| {person c, person e, person g, person f}                     |       875 |        325 |          3 |             20 |  |&#xA;+--------------------------------------------------------------+-----------+------------+------------+----------------+--+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each of these persons would be a tuple of values: var_1, var_2, var_3, var_4.&#xA;These values are not constant however! They do change between observations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Different approach to explain it&lt;/strong&gt;: there's multiple observations (variable amount) in each time segment (duration of time segment is a fixed integer to be chosen). These observations have a few variables that would indicate whether the observation is true or false. However, the threshold for it being true or false, is very much dependant on the other variables, that are not tied to the information of the persons. (These variables are the same for all of them, but vary in between time segments. Let's call'm &quot;environment features&quot;)&#xA;Lastly, the output feature is the product of the count of persons that resulted in &quot;True&quot; and a (varying) factor that is correlated to the environment features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've been thinking about probabilistic AI, but the problem is that there isn't a known distribution between True/False. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any technique I can apply to be able to use this kind of data&#xA;as an input of a Neural Network (or other forms of ML)? Or is there a&#xA;specific form of ML that is used for this kind of problems?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="16932" LastEditorUserId="1671" LastEditDate="2018-07-17T17:14:11.527" LastActivityDate="2018-07-17T17:14:11.527" Title="Multiple sets of input in Neural network (or other form of ML)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7179" PostTypeId="1" AcceptedAnswerId="7186" CreationDate="2018-07-17T13:09:36.700" Score="2" ViewCount="1037" Body="&lt;p&gt;I have extensively researched now for three days straight trying to find which algorithm is better in terms of which algorithm uses up more memory. I know uninformed algorithms like depth-first search and breadth-first search do not store or maintain a list of unsearched nodes like how informed search algorithms do. But the main problem with uninformed algorithms is they might keep going deeper, theoretically to infinity if an end state is not found but they exist ways to limit the search like depth-limited search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So am I right in saying that uninformed search in better than informed search in terms of memory with respect to what I said above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone provide me with any references that show why one algorithm is better than the other in terms of memory?&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="16906" LastEditDate="2018-07-17T16:35:32.367" LastActivityDate="2018-07-17T17:31:06.553" Title="Which is more memory efficient uninformed or informed search algorithms?" Tags="&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7180" PostTypeId="1" CreationDate="2018-07-17T13:34:55.370" Score="3" ViewCount="275" Body="&lt;p&gt;Imagine the fictitious scenario in a role playing game (RPG) where the non-playing characters (NPCs) within the RPG are conscious of their own surrounding and consider the developer to be god.  The people inside can also live a normal life in that they can create new life, eat, die, and perform other functions humans experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will this qualify them to be an AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will the children of the NPCs qualify as an AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Might this be realized in the future?&lt;/p&gt;&#xA;" OwnerUserId="14371" LastEditorUserId="4302" LastEditDate="2018-07-27T18:18:43.870" LastActivityDate="2018-07-27T23:57:38.077" Title="What if there's a game where all the AI people actually lived?" Tags="&lt;game-ai&gt;&lt;game-theory&gt;&lt;theory&gt;&lt;soft-question&gt;&lt;artificial-consciousness&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-18T04:26:17.083" ContentLicense="CC BY-SA 4.0" />
  <row Id="7181" PostTypeId="2" ParentId="4955" CreationDate="2018-07-17T13:43:55.307" Score="1" Body="&lt;p&gt;Your initial idea seems about right. Before creating your own classifier you might want to try transfer learning, using some pretrained network like VGG16 that is incorporated in most of machine learning frameworks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to inference on mobile devices, TensorFlow offers some tutorials in this subject:&#xA;&lt;a href=&quot;https://www.tensorflow.org/mobile/tflite/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/mobile/tflite/&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="16929" LastActivityDate="2018-07-17T13:43:55.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7182" PostTypeId="1" CreationDate="2018-07-17T14:54:21.650" Score="2" ViewCount="107" Body="&lt;p&gt;When applying multinomial Naive Bayes text classification, I get very small probabilities (around &lt;span class=&quot;math-container&quot;&gt;$10e^{-48}$&lt;/span&gt;), so there's no way for me to know which classes are valid predictions and which ones are not. I'd the probabilities to be in the interval &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;, so I can exclude classes in the prediction with say a score of 0.5 or less. How do I go about doing this?&lt;/p&gt;&#xA;&lt;p&gt;This is what I've implemented:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$c_{\operatorname{map}}=\underset{\operatorname{c \in C}}{\arg \max }(P(c \mid d))=\underset{\operatorname{c \in C}}{\arg \max }\left(P(c) \prod_{1 \leq k \leq n_d} P\left(t_{k} \mid c\right)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="16927" LastEditorUserId="2444" LastEditDate="2020-07-18T17:08:57.327" LastActivityDate="2021-04-15T23:57:28.630" Title="Why do I get small probabilities when implementing a multinomial naive Bayes text classification model?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;probability&gt;&lt;naive-bayes&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7183" PostTypeId="2" ParentId="7180" CreationDate="2018-07-17T15:03:10.813" Score="1" Body="&lt;p&gt;&lt;strong&gt;Yes&lt;/strong&gt;, a baby can be considered an AI. Will this be our future? &lt;strong&gt;I don't know&lt;/strong&gt;. That is exactly what some people are looking for, to create an AI that can live. We have several AIs that each time more surprises us. But none of them question their own existence, none of them wants to know or require attention from their god (developer) because of their reflections. I believe that what we create today can be equated with a mosquito of our real life: a brainless &quot;being&quot; that performs simple actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cool thing about believing in such a future with Artificial Intelligence is that we do not have to repeat our mistakes. We do not need an AI evolving according to age, it can be born with a superintelligence, knowing everything, learning only new things that are not foreseen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some humans (and I) have a passion for the human, for every detail and imperfection, and dream of re-creating something as mysterious and complex as through an AI. It seems something impossible, some believe that it would be impossible only because who created us would be a perfect being (god). But others (me) do not stop at this issue, and continue the quest to create an AI with some characteristics of the human, such as: feelings, reflecting on one's own acts, ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But all this, as far as I know, is surreal and only projection of our mind. It is already something interesting, for creating new challenges for humanity.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-07-17T15:03:10.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7185" PostTypeId="1" AcceptedAnswerId="7198" CreationDate="2018-07-17T17:14:02.157" Score="2" ViewCount="52" Body="&lt;p&gt;I created a system where every moment takes photos of the face of who is in the vision of the camera. Initially I took 500 photos of me, to recognize its creator. This takes approximately 20 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then every moment he recognizes faces and if it is me, he knows that his creator is present. Any different face, it creates a dataset with a different name and starts taking up to 500 photos to also recognize these faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I say a certain command, it returns all faces I have found that have no ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for ways to capture the images through some camera that I can carry while walking on the street and in public places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that there would be several faces to name. I'm partially solving this problem by trying to recognize these people in social networks. I check the region where the photo was taken and try to find people who have checked in or liked the area on Facebook. But anyway, this is not the big problem, although it is looking for more effective solutions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My big problem is: Can I do this? Do I have this right? Can I record a&#xA;  robber robbery and recognize his face in other places? Record an&#xA;  aggression, an act of prejudice and things like that?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The main purpose would be this, but could also be used for other purposes. My fear is being arrested for doing this. Both because he would be taking pictures of people without his consent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps: I'm thinking of having a camera in the palm of my hand. It would be a micro camera (I'm trying to find the product on the Internet), to be as discreet as possible.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="1671" LastEditDate="2018-07-18T21:21:58.290" LastActivityDate="2018-07-18T21:31:52.177" Title="Can I recognize the faces of people around the world?" Tags="&lt;legal&gt;&lt;facial-recognition&gt;&lt;digital-rights&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7186" PostTypeId="2" ParentId="7179" CreationDate="2018-07-17T17:25:32.777" Score="1" Body="&lt;h2&gt;Uninformed Search Techniques&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Breadth-First Search&lt;/strong&gt; needs to store a &lt;em&gt;frontier&lt;/em&gt; of nodes to visit next (where &quot;visit&quot; basically means: see if it's the goal, generate its children and add to frontier otherwise). You can visualize the memory requirements of this as a pyramid; initially you just have the root node in there. As the search process continues and you keep going further down, it will become wider and wider (more memory required). The amount of memory required in the worst case to find a node at depth &lt;em&gt;d&lt;/em&gt; can be expressed as &lt;em&gt;O(b^d)&lt;/em&gt;, where &lt;em&gt;b&lt;/em&gt; is the branching factor (&lt;a href=&quot;https://en.wikipedia.org/wiki/Breadth-first_search#Time_and_space_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;see details here&lt;/a&gt;). Intuitively, whenever you go one level deeper, your memory requirements multiply by a factor of &lt;em&gt;b&lt;/em&gt; (your pyramid becomes &lt;em&gt;b&lt;/em&gt; times wider; actually this means pyramid is not quite the right shape, it should rapidly curve outwards and become wider much faster than a triangle/pyramid would).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Depth-First Search&lt;/strong&gt; also stores a frontier (or &lt;em&gt;stack&lt;/em&gt; in the case of DFS) of nodes to visit next. However, in the case of DFS this generally grows less quickly. Whenever you visit a node (pop it off of the stack), you generate all of its children and push them on top of the stack. However, whereas BFS would then continue &quot;to the right&quot; by visiting a relatively &quot;old&quot; node and generating all of its children again, DFS continues by going &quot;down&quot; and visiting one of the children that only just got pushed onto the stack. Once DFS has finished a part of the search tree (you can visualize this in your head as DFS having completed searched for example the left half of the pyramid), that entire section no longer needs to be stored in memory. Assuming you have a search tree of finite depth &lt;em&gt;d&lt;/em&gt;, the worst case space complexity is &lt;em&gt;O(bd)&lt;/em&gt; (you have to search one path all the way down to a level of &lt;em&gt;d&lt;/em&gt;, and for each of those levels have &lt;em&gt;b&lt;/em&gt; nodes in the stack).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that, based on the above, it is not sufficient to characterize your comparison as &quot;uninformed&quot; vs &quot;informed&quot; search. We've only just looked at uninformed search techniques, and already have two different memory requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above, I also did not consider subtleties such as the question of whether or not you're additionally memorizing which nodes of a graph you have already previously visited so that you can avoid visiting them again. This is not necessary in a tree, but may be necessary in a graph with cycles.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Informed Search Techniques&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A*&lt;/strong&gt; is probably one of the most canonical examples of informed search algorithms. In terms of worst case memory requirements, it's really similar to BFS; it also stores a frontier of nodes to visit next, but prioritizes those based on some estimate of &quot;goodness&quot; rather than Breadth-First order. In cases where you the information you're using (typically a combination of known/incurred costs + heuristic estimate of future costs) is of extremely poor quality, this can regress to the level of Breadth-First Search, so the worst-case space complexity is the same as that for BFS (&lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm#Complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;see details&lt;/a&gt;). In practice, if you have high quality information (good heuristics), the memory requirements will be &lt;em&gt;much&lt;/em&gt; better though; in the case of an ideal/perfect heuristic, your search algorithm will pretty much go directly to the goal and hardly require any memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also an algorithm called &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_A*&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iterative Deepening A*&lt;/a&gt; which has a much better space complexity, at the cost of often being slower. It is still an informed search algorithm using exactly the same kind of information as A* though. So, really, memory requirements cannot be characterized in terms of informed vs. uninformed search; in both cases there are algorithms that require a lot of memory, and algorithms that require less memory.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-07-17T17:31:06.553" LastActivityDate="2018-07-17T17:31:06.553" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7187" PostTypeId="2" ParentId="7180" CreationDate="2018-07-17T18:58:23.600" Score="0" Body="&lt;p&gt;This is a standard trope in science fiction, although more common is the idea of transferring minds from real people into a virtual environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, such an idea is still firmly in science fiction. We don't know a few things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What consciousness is in detail. There are various educated guesses and philosophical stances, but none have been seriously verified.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What the minimal environment is that can support consciousness. This is important because simulating more complex environments is more computationally expensive. There is a &lt;strong&gt;huge&lt;/strong&gt; gulf between capabilities of current computers and something that could simulate reality to the level of granularity that we can experience and manipulate it (let alone to the depth that we can scientifically investigate). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will this consider them to be an AI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming you can solve the caveats above, then yes, perhaps even a &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence#Relationship_to_%22strong_AI%22&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strong AI&lt;/a&gt;&quot; in the domain of the game, but not a human-equivalent &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI&lt;/a&gt; (depending on the complexity of the environment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mixed a current RPG game with some agents driven by our current best game-playing bots, and improved them so that they are able to solve problems within the game - e.g. in World of Warcraft, or perhaps in Minecraft - then you would have something as advanced or better than any current &quot;narrow AI&quot; that solves points scoring or adversarial games. You would be justified in calling it &quot;AI&quot;, if OpenAI and Deep Mind can call their achievements &quot;AI&quot;. However, it seems unlikely such agents would be much like your imagined simulated people - having consciousness etc.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would this be real in the future?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Depends on the constraints faced by researchers. To get to an imagined point where this is possible, we have to extrapolate current trends on computing power and advances in understanding goal-driven AI and self-awareness. There is a range of opinions on how likely this is, how much power is required etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a compelling idea, and unlike say faster-than-light travel, there are no widely-accepted theoretical limits that prevent it. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-07-17T18:58:23.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7189" PostTypeId="1" CreationDate="2018-07-18T08:21:51.843" Score="2" ViewCount="31" Body="&lt;p&gt;I know that the first layer uses a low-level filter to see the edge information. As the layer gets deeper, it will represent high-level (abstract) information. Is it because the combinations of filters used in the previous layer are used as filters in the next layer? (&quot;Does the combination of the previous layer's filters make the next layer's filters?)&#xA;If so, are the combinations determined in advance?&lt;/p&gt;&#xA;" OwnerUserId="16952" LastActivityDate="2018-07-18T08:21:51.843" Title="In CNN (Convolutional Neural Network), does the combination of previous layer's filters make next layer's filters?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7193" PostTypeId="2" ParentId="7105" CreationDate="2018-07-18T10:09:18.620" Score="1" Body="&lt;p&gt;&lt;strong&gt;Central Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can ML/AI understand incomplete constructs like humans?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do humans have some inherent experiences in life which makes AI incapable of performing [some capacities of human intelligence]?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Comprehension of Literature and Film&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether software exists today that is able to understand like humans is not something that the general public can know.  No such system has been released to the general public by any military or commercial organization thus far.  Yet such an achievement would not necessarily be something the government or commercial entity would want to disclose outside of the lab and its management.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deeper comprehension of full or partial speech may have been achieved by anti-terrorist units, since funding has been available for that work for well over a decade, but it also unlikely that some software somewhere can read or scan parts of a book and do a book report that would produce a passing grade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining whether a movie will return its investment at the box office may have been accomplished by the researchers for the big studios, but that doesn't require understanding like someone understands their favorite movie.  The answer to the title question is, &quot;Not yet.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will it occur?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most in the fields of computer science, robotics, and artificial intelligence say, &quot;Yes.&quot;  For religious reasons, some say, &quot;No.&quot;  I'm in neither camp, and have not seen indisputable mathematical rigor that proves either the inevitability of artificial brains or the impossibility of them.  It is scientifically irresponsible to make a positive statement based on either recent technology trends or superstitious fears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are Humans Like?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phrase, &quot;Like humans,&quot; places a significant demand on researchers and engineers.  Human beings can do much more than work with data sets (audiovisual in this case) with missing information, what statisticians call sparse matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether software can realize higher achievements of human brains is unknown, and the predictability of such capabilities presents gross difficulties.  Consider these human capabilities.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Write a screenplay, cast it with artificial characters, direct it, and produce it.&lt;/li&gt;&#xA;&lt;li&gt;Initiate and partly develop a new branch of science, as did Isaac Newton or Lavoisier.&lt;/li&gt;&#xA;&lt;li&gt;Love beyond a superficial expression of love&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Conversely, what humans generally do poorly is distinguish between reliable projection and baseless conjecture.  Most humans are prone to musings of technical visionaries, propaganda, marketing, rumor, innuendo, and gossip.  It would not be surprising for software to someday soon be wiser in this respect, but only because the bar has been set so low by human culture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other capabilities of the human brain that are exceptional and present enormous difficulties in even considering an approach to realizing in software.  These may be a result of a hundred thousand years of DNA refinement yielding significant complexity and precision.  It is also not outside the realm of possibility that a form of causality exists in the human brain that defies scientific study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Possibility of Impossibility&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No one has every proven that all things that exist can be measured.  Heisenberg has actually proven the opposite to most theoretical physicist's satisfaction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A phenomenon that cannot be measured at all cannot be studied scientifically.  Whether the phenomenon of choice is a unique condition has yet to be understood even in question form, prohibiting the emergence of a proper answer thus far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Imminent Change as Significant as Industrialization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, important capabilities of human intelligence have been simulated and others are emerging.  These are now part of the world economy and will not likely disappear.  The possibility of Asimov like scenarios of robots and humans coexisting and conversing in more human-like is very likely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is when that occurs that autonomous vehicles and walking robots will begin to have experiences that are like human experiences and we will be able to directly observe just how much like humans they can behave in terms of intelligence and also emotions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;About the Specific Capabilities Mentioned&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the capabilities mentioned in the question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Predicting images&lt;/li&gt;&#xA;&lt;li&gt;Predicting objects in an image&lt;/li&gt;&#xA;&lt;li&gt;Understanding audio&lt;/li&gt;&#xA;&lt;li&gt;Understanding the meaning of spoken sentences&lt;/li&gt;&#xA;&lt;li&gt;Understanding a movie from the first half&lt;/li&gt;&#xA;&lt;li&gt;Understanding a movie from parts of it&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are the more canonical ways of stating the first four capabilities for which research has produced usable system approaches.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Learning to distinguish image categories&lt;/li&gt;&#xA;&lt;li&gt;Learning to distinguish object categories from within images&lt;/li&gt;&#xA;&lt;li&gt;Parsing audio into notes, vocal tones, and transient sounds&lt;/li&gt;&#xA;&lt;li&gt;Extracting semantics from a vocalization sufficiently to respond intelligently some of the time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are a more accurate cognitive science description of the last two.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Guessing much better than random guessing would the story arch to the end of a movie from the sound and frame set of the its first half&lt;/li&gt;&#xA;&lt;li&gt;Filling in character and story arch details from portions of a movie's sound and frame set&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is no obvious reason why such couldn't be done and done well by software, given sufficient research time to develop such a system and sufficient data to train with.  Also, significant computing resources would be needed, of course, and possibly some nontrivial period of time.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-19T08:47:31.460" LastActivityDate="2018-07-19T08:47:31.460" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7195" PostTypeId="1" AcceptedAnswerId="7197" CreationDate="2018-07-18T16:52:55.273" Score="1" ViewCount="155" Body="&lt;p&gt;I've just started to learn genetic algorithms and I have found these measurements of runs that I don't understand:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;MBF: The mean best fitness measure (MBF) is the average of the best&#xA;fitness values over all runs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;AES: The average number of evaluation to solution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have an initial random population. To evolve a population I do:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Tournament selection&lt;/li&gt;&#xA;&lt;li&gt;One point crossover.&lt;/li&gt;&#xA;&lt;li&gt;Random resetting.&lt;/li&gt;&#xA;&lt;li&gt;Age based replacement with elitism (I replace the population with all offsprings generated).&lt;/li&gt;&#xA;&lt;li&gt;If I have generated G generations (in other words, I have repeated these four points G times) or I have found the solution, the algorithm ends, otherwise, it comes back to point 1.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is the mean of the best fitness the mean fitness of all of each generations (G best fitness)?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MBF = (BestFitness_0 + ... + BestFitness_G) / G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not English and I don't understand the meaning of &amp;quot;run&amp;quot; here.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2021-01-30T03:19:38.577" LastActivityDate="2021-01-30T21:45:54.160" Title="How can I calculate the &quot;mean best fitness&quot; measure in genetic algorithms?" Tags="&lt;terminology&gt;&lt;genetic-algorithms&gt;&lt;fitness-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7196" PostTypeId="1" CreationDate="2018-07-18T17:23:53.960" Score="2" ViewCount="46" Body="&lt;p&gt;I've been researching AI regulation and compliance (&lt;a href=&quot;https://law.stackexchange.com/questions/30356/how-do-concepts-like-intent-discrimination-and-bias-apply-to-an-artificial-int&quot;&gt;see my related question on law.stackexchange&lt;/a&gt;), and one of the big take-aways that I had is that the regulations that apply to a human will apply to an AI agent in most if not all cases.  This has some interesting implications when you take a look at concepts like bias and discrimination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a model with explicit rules like a decision tree or even a random forest, I can see how inspecting the rules themselves should reveal discrimination.  What I'm struggling with is how do you detect bias in models like neural networks, where you provide the general structure of the model and a set of training data, and then the model self-optimizes to provide the best possible results based on the training data.  In this case, the model could find biases in past human decisions that it was trained based on and replicate them, or it could find a correlation that isn't apparent to a human and inform decisions based on this correlation that may result in discrimination based on a wide array of factors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that in mind, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What tools or methodologies are available for assessing the presence and source of bias in machine learning models?&lt;/li&gt;&#xA;&lt;li&gt;Once discrimination has been identified, are there any techniques to eliminate bias from the model?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16965" LastActivityDate="2018-07-18T17:23:53.960" Title="What methods are there to detect discrimination in trained models?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;legal&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7197" PostTypeId="2" ParentId="7195" CreationDate="2018-07-18T18:04:14.987" Score="2" Body="&lt;p&gt;The typical way you'll see a GA measured is that an algorithm with a population size of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is ran &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; times from new random seeds each time. That gives you &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; total runs of the algorithm, each of which, at the end, had a final population of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; individuals. If you take the best of those &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; from each run, you get &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; &amp;quot;best&amp;quot; solutions found. The average fitness value of those &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; solutions is your MBF.&lt;/p&gt;&#xA;&lt;p&gt;AES here refers to the number of evaluations of the objective function the algorithm required. The reason for this is to provide a standardized amount of computation budget each algorithm used. Imagine if you compared algorithms on the basis of how many generations it took to find a good solution. In that case, I could just increase the population size of my algorithm by 100-fold or 1000-fold, and I'll probably look better. Same number of generations, but I gave my algorithm far more chances to find something good. Suppose I use wall-clock time. Now my algorithm might look better than yours just because I ran it on a much more powerful computer than you did.&lt;/p&gt;&#xA;&lt;p&gt;The insight to make here is that GAs work by evaluating new search points. Let's just count how many times that happens. You can run a larger population for a shorter number of generations, or you can allocate the same amount of computation to running a longer time with fewer points each generation. What matters is just how many fitness function evaluations you exhausted in finding the answer.&lt;/p&gt;&#xA;&lt;p&gt;For sudoku, you might have a fitness function that counted the number of rows, columns, or blocks that don't contain the correct digits of 1-9 and you minimize that function. You run your algorithm &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; times from random seeds, and for each run, you record how many times you had to evaluate that fitness function before you found a &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (i.e., the puzzle was successfully solved). Average all &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; of those counts, and that's your AES.&lt;/p&gt;&#xA;&lt;p&gt;Generalizing a bit, you might calculate average evaluations to find some &amp;quot;good enough&amp;quot; solution, but the general concept is the same.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastEditorUserId="2444" LastEditDate="2021-01-30T21:45:54.160" LastActivityDate="2021-01-30T21:45:54.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7198" PostTypeId="2" ParentId="7185" CreationDate="2018-07-18T21:26:35.120" Score="1" Body="&lt;p&gt;I thought it might be a helpful place to start:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Street_View_privacy_concerns&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Google_Street_View_privacy_concerns&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The wiki is well cited, so should lead to some useful tidbits.  They break it down by continent and country.&lt;/p&gt;&#xA;&lt;p&gt;Seems like a parallel to what you're doing, although, if you're not making your data public, I doubt you'll be facing the same privacy issues.&lt;/p&gt;&#xA;&lt;p&gt;Also relevant would be: &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectation_of_privacy&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Expectation_of_privacy&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Examples of places where a person has a reasonable expectation of privacy are a person's residence or hotel room and public places which have been specifically provided by businesses or the public sector in order to ensure privacy, such as public restrooms...&lt;/p&gt;&#xA;&lt;p&gt;In general, one cannot have a reasonable expectation of privacy in things held out to the public.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My understanding is that there is no expectation of privacy in public places where you'd be capturing the images.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-07-18T21:31:52.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7200" PostTypeId="1" CreationDate="2018-07-19T08:55:08.907" Score="1" ViewCount="65" Body="&lt;p&gt;What AI concepts, topologies&lt;sup&gt;1&lt;/sup&gt;, algorithms, or SaaS can be used to recognize a person eating a chocolate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this question, image recognition draws from a real time feed, validating each of these steps in sequence:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using a camera app, the user begins recording.&lt;/li&gt;&#xA;&lt;li&gt;The user focuses on the subject's face, at which time the app attempts to recognize the person&lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Person holds a single piece of chocolate (i.e M&amp;amp;Ms) to the camera lens, at which time the app attempts to recognize it.&lt;/li&gt;&#xA;&lt;li&gt;The user puts the chocolate into their mouth, chews, and swallows, at which time the app attempts to recognize that AS AN ACTION.&lt;/li&gt;&#xA;&lt;li&gt;The app gives a completion message indicating success or failure.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I understand that we can use real time recognition for each step, but I don't know if there are concepts proposed or tested to validate the scene as a sequence or any of the three recognition steps individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The app should invalidate the scene if the subject is swapped with another subject, if the subject does not swallow the chocolate, or there is some other deviation from the expected sequence above.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] By topology in this context is meant the standard mathematical meaning of the term applied to the higher level connections that are likely needed to recognize sequences of actions.  In this sense, the use of the term topology is not at the level dimensions of neuron layers or convolution kernels.  Since process topology is normally considered prior to considering library dependencies or deployment concerns, the term topology is more appropriate than architecture in this question.  (First things first.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] I've already identified SaaS options for facial recognition.&lt;/p&gt;&#xA;" OwnerUserId="16975" LastEditorUserId="4302" LastEditDate="2018-07-24T11:50:04.047" LastActivityDate="2018-07-24T11:50:04.047" Title="Steps recognition" Tags="&lt;image-recognition&gt;&lt;concepts&gt;&lt;action-recognition&gt;&lt;topology&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7202" PostTypeId="1" CreationDate="2018-07-19T11:01:51.977" Score="6" ViewCount="5110" Body="&lt;p&gt;I'm trying to create and test non-linear SVMs with various kernels (RBF, Sigmoid, Polynomial) in scikit-learn, to create a model which can classify anomalies and benign behaviors.&lt;/p&gt;&#xA;&lt;p&gt;My dataset includes 692703 records and I use a 75/25% training/testing split. Also, I use various combinations of features whose dimensionality is between 1 and 14 features. However, the training processes of the various SVMs take much too long. Is this reasonable?&lt;/p&gt;&#xA;&lt;p&gt;I have also examined the ensemble &lt;code&gt;BaggingClassifier&lt;/code&gt; in combination with non-linear SVMs, by configuring the &lt;code&gt;n_jobs&lt;/code&gt; parameter to &lt;code&gt;-1&lt;/code&gt;; nevertheless, the training process proceeds again too slowly.&lt;/p&gt;&#xA;&lt;p&gt;How can I speed up the training processes?&lt;/p&gt;&#xA;" OwnerUserId="16977" LastEditorUserId="2444" LastEditDate="2021-04-29T12:26:37.860" LastActivityDate="2021-04-29T12:26:37.860" Title="Why does training an SVM take so long? How can I speed it up?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;support-vector-machine&gt;" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7204" PostTypeId="2" ParentId="1963" CreationDate="2018-07-19T11:32:44.833" Score="7" Body="&lt;p&gt;There certainly appear to have been research projects involving some form of text mining / information retrieval /etc. and StackExchange sites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples I was able to find through google/google scholar (unlikely to be anywhere near an exhaustive list):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/article/10.3758/s13428-016-0722-4&quot; rel=&quot;noreferrer&quot;&gt;TACIT: An open-source text analysis, crawling, and interpretation tool&lt;/a&gt; describes numerous text-crawlers for a variety of sites (including Stack Exchange sites, but also Twitter, Reddit, etc.). At first glance, this appears to be primarily about crawling, not about doing anything else with the data afterwards. Searching for other papers that cite this one on Google Scholar may yield interesting results though, it may lead to papers that used this for crawling and did more with the data afterwards.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1401.0480&quot; rel=&quot;noreferrer&quot;&gt;Chaff from the Wheat : Characterization and Modeling of Deleted Questions on Stack Overflow&lt;/a&gt; describes research into the quality of Stack Overflow questions in some sense (specifically, predicting whether questions will get deleted for example). I'm not 100% sure if this is also the kind of stuff you're interested in; it is Stack Exchange + Machine Learning as implied by the title of your question, but not necessarily about retaining information from answers as implied by the text in your question.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.emeraldinsight.com/doi/abs/10.1108/JEIM-11-2014-0109&quot; rel=&quot;noreferrer&quot;&gt;Text mining stackoverflow: An insight into challenges and subject-related difficulties faced by computer science learners&lt;/a&gt; also describes text mining in StackOverflow questions and answers, though at a very quick glance it appears to be primarily about topic detection etc., not necessarily about automated question answering for example.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/323729727_Different_Facets_of_Text_Based_Automated_Question_Answering_System&quot; rel=&quot;noreferrer&quot;&gt;Different Facets of Text Based Automated Question Answering System&lt;/a&gt; appears to be a relatively recent survey on the topic of Automated Question Answering research. Stack Exchange is mentioned a few times as an example of a source of data for such systems, but doesn't appear to be used otherwise.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-77703-0_56&quot; rel=&quot;noreferrer&quot;&gt;Extending PythonQA with Knowledge from StackOverflow&lt;/a&gt; is &lt;strong&gt;specifically about incorporating Questions and Answers from StackOverflow in an automated Question and Answering system for questions about the Python programming language&lt;/strong&gt;. The paper provides a link to more details (&lt;a href=&quot;http://pythonqas2.epl.di.uminho.pt&quot; rel=&quot;noreferrer&quot;&gt;http://pythonqas2.epl.di.uminho.pt&lt;/a&gt;), but that link appears to be down. I suppose you could always try contacting the authors directly if you're interested in more information on this.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;More generally, Automated Question Answering systems appears to be a rather active area of research still, not a trivial / &quot;solved&quot; problem. StackExchange can be one source of data for such systems, but there are plenty of other sources of data too (Wikipedia, Quora, etc.).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-19T11:32:44.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7207" PostTypeId="1" CreationDate="2018-07-19T13:57:43.527" Score="3" ViewCount="188" Body="&lt;p&gt;How does one even begin to mathematically model an AI algorithm, like alpha-beta pruning or even its thousands of variations, to determine which variation is best?&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="2444" LastEditDate="2020-05-13T10:45:25.700" LastActivityDate="2020-05-13T10:45:25.700" Title="How does one even begin to mathematically model an AI algorithm?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;math&gt;&lt;models&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7208" PostTypeId="2" ParentId="4209" CreationDate="2018-07-19T14:52:41.953" Score="1" Body="&lt;p&gt;You're correct that it is related to the fitness function, but only indirectly.  Recall that a function is the mathematical embodiment of a conceptual relationship.  The model is a motion model.  Proximity is not simply applying the distance formula &lt;span class=&quot;math-container&quot;&gt;$D = \sqrt{(x_2 - x_2)^2 + (y_2 - y_2)^2}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proximity as we &quot;feel it&quot; when driving a car is based on the risks we see in the trajectories of ourselves and the other moving objects along with the stationary objects.  Mathematically, this is an integral over time of a distribution of probable locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look for work in the aeronautics industry about collision avoidance.  The mathematics and algorithms are well developed for antiaircraft weaponry (where the fitness function is the inverse of the one you want) and air traffic control, where the fitness is like yours except with an altitude dimension.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2019-06-26T12:24:42.413" LastActivityDate="2019-06-26T12:24:42.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7211" PostTypeId="5" CreationDate="2018-07-19T20:12:00.097" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;https://en.wikipedia.org/wiki/Support_vector_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Support_vector_machine&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-03-04T18:15:43.443" LastActivityDate="2020-03-04T18:15:43.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7212" PostTypeId="4" CreationDate="2018-07-19T20:12:00.097" Score="0" Body="For questions about support vector machines (SVMs), which are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-03-04T18:15:43.443" LastActivityDate="2020-03-04T18:15:43.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7214" PostTypeId="1" CreationDate="2018-07-19T23:51:40.977" Score="4" ViewCount="238" Body="&lt;p&gt;I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;End Goal&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being able to input an instrumental and get an output of how to sing to that instrumental.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Dataset&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have extracted pitch points from thousands of actual acapellas from real songs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Theory&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feed the AI a pitch point PLUS say 19 thousand points of the original song instrumental.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Illustration&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s7rJ0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s7rJ0.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red line (on top) is the pitch viewed vertically (lower pitch down, higher pitch up) of the voice sung by the singer over time viewed horizontally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottom image is the song's frequency viewed vertically (lower freq down, higher freq up) viewed horizontally over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We take a point in time of the instrumental, say 0 minutes 30 seconds, and extract 19k points of the FFT spectrum vertically and call this a frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also take the same point in time of the voice pitch, and also refer to this as a frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we have a frame which contains 20 thousand data points, one being the pitch of the voice, and the rest being the frequencies of the songs content.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTION&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of model could be used to teach the AI the correlation of the voice and the instrumental?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also, I have a hard time understand how, once the AI is trained, how could just an instrumental be fed to the AI to output pitch values of how one COULD sing along to the song.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like, training we need to input 20 thousand values, but when we want the AI to sing for us using just an instrumental, would it not still expect voice pitch input?&#xA;At what layer would the instrumental be tapped into? At the outer most right layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My mind has been working on this in the background throughout the day,&#xA;and I am wondering if instead of feeding 19k points of instrumental data each frame (which would be points from the frequency domain), one could just feed the instrumental frame points (which would be points from the time domain).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Maybe&lt;/em&gt; that would be better, but then maybe the AI would get less &quot;resolution&quot; to work with, but could be trained faster (less computing power needed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the frequency domain is fed (higher resolution), the AI could potentially find correlations from low notes, mid notes and high notes, in any combination (more computing power needed).&lt;/p&gt;&#xA;" OwnerUserId="16993" LastEditorUserId="16993" LastEditDate="2018-07-20T19:29:21.337" LastActivityDate="2018-07-20T21:12:52.253" Title="What would be the best approach to teach an AI to learn how to &quot;sing&quot; along a beat?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7215" PostTypeId="1" CreationDate="2018-07-20T08:54:11.240" Score="7" ViewCount="522" Body="&lt;p&gt;In the brain, some synapses are stimulating and some inhibiting. In the case of artificial neural networks, ReLU erases that property, since in the brain inhibition doesn't correspond to a 0 output, but, more precisely, to a negative input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the brain, the positive and negative potential is summed up, and, if it passed the threshold, the neuron fires. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 main non-linearities which came to my mind in the biological unit:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;potential change is more exponential than linear: small amount of ion channels is sufficient to start a chain-reaction of other channels activation's - which rapidly change global neuron's potential.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the threshold of the neuron is also non-linear: neuron fires only when the sum of its positive and negative potentials passed given (positive) threshold&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, is there any idea how to implement negative input to the artificial neural network?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I gave examples of non-linearities in biological neurons because the most obvious positive/negative unit is just a linear unit. But, since it doesn't implement non-linearity, we may consider implementing non-linearities somewhere else in the artificial neuron.&lt;/p&gt;&#xA;" OwnerUserId="12691" LastEditorUserId="2444" LastEditDate="2020-05-23T18:35:27.273" LastActivityDate="2020-05-23T18:35:27.273" Title="How to model inhibitory synapses in the artificial neuron?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;&lt;biology&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7217" PostTypeId="1" CreationDate="2018-07-20T10:19:25.900" Score="1" ViewCount="79" Body="&lt;p&gt;This is actually something I have been researching a bit on my own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most movie scripts can be structurally analysed by using writing theory such as &lt;a href=&quot;http://dramatica.com/theory/book/theme&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dramatica&lt;/a&gt;. Dramatica is based upon a hierarchy of concepts, which can be topic modeled. The hierarchy of topic models would seem to work very well with the capsule neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been working with computational creativity problems in a narrative generation. The state of the art methods use Partial Order Causal Link Planners, but they depend on propositional logic. Alonzo Church presented the &lt;a href=&quot;https://en.wikipedia.org/wiki/Extensional_context&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superman dilemma&lt;/a&gt; (Louis Lane does not know that Clark Kent is Superman, but Superman knows, that he is Clark Kent) and invented Intensional Logic as a solution; the basic idea is, that if we do not know the context of the narrative, the meaning is always in superposition and can only be understood through entangled meanings from the background story. So, in a sense, propositional logic is limited by classic information theory constraints, while Church's logic can take a quantum information-theoretic approach. I do not believe that classic information theory can resolve narrative analysis problems. So, basically, the meaning of a narrative collapses (the superposition gets resolved) by using the hierarchical narrative structure and what we know beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my intuition would be the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;We can use Dramatica and potentially other narrative theories (hierarchical metamemetics, reverse SCARF, etc.) to create a hierarchical network like ImageNet, but for narratives.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We can build conceptual topic models. Dramatica has a hierarchy of 4-16-64-64 concepts and annotated data exists already.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When using hundreds of topic models, there will be a lot of false positives. However, the superposition of the topic models can be collapsed by using the hierarchical levels and some other dramatic analytics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;By using the capsule neural networks, we might be able to build a system, which could determine a narrative interpretation of the full story, which would make the most sense by using the concept hierarchy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I tried to prove my intuition, but, unfortunately, Dramatica only has 300 movies analysed, and I was able to find scripts of only 10 of them; not enough data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are other hierarchical ontologies out there and other narrative structures; could the same intuition be used for political news for example?&lt;/p&gt;&#xA;" OwnerUserId="11626" LastEditorUserId="11626" LastEditDate="2020-06-15T14:47:07.140" LastActivityDate="2020-06-15T14:47:07.140" Title="Would it make sense to use together capsule neural neworks and &quot;topic / narrative modeling&quot;?" Tags="&lt;natural-language-processing&gt;&lt;capsule-neural-network&gt;&lt;topic-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7222" PostTypeId="1" CreationDate="2018-07-20T14:13:15.420" Score="6" ViewCount="1654" Body="&lt;p&gt;first of all I want to specify the data available and what needs to be achieved: I have a huge amount of vacancies (in the millions). The information about the &lt;strong&gt;job title&lt;/strong&gt; and the &lt;strong&gt;job description&lt;/strong&gt; of each vacancy are stored separately. I also have a &lt;strong&gt;list of professions&lt;/strong&gt; (around 3000), to which the vacancies shall be mapped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;: &lt;em&gt;java-developer, java web engineer and java software developer&lt;/em&gt; shall all be mapped to the profession &lt;em&gt;java engineer&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now about my current researches and problems: Since a lot of potential training data is present, I thought a machine learning approach could be useful. I have been reading about different algorithms and wanted to give neural networks a shot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very fast I faced the problem, that I couldn't find a satisfying way to &lt;strong&gt;transform text of variable length to numerical vectors of constant size&lt;/strong&gt; (needed by neural networks). As discussed &lt;a href=&quot;https://stackoverflow.com/questions/14783431/processing-strings-of-text-for-neural-network-input&quot;&gt;here&lt;/a&gt;, this seems to be a non trivial problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dug deeper and came across &lt;a href=&quot;https://skymind.ai/wiki/bagofwords-tf-idf&quot; rel=&quot;noreferrer&quot;&gt;Bag of Words (BOW) and Text Frequency - Inverse Document Frequency (TFIDF)&lt;/a&gt;, which seemed suitable at first glance. But here I faced other problems: If I feed all the job titles to TFIDF, the resulting word-weight-vectors will probably be very large (in the tenth of thousands). The search term on the other hand will mostly consist of between 1 and 5 words (we currently match the job title only). Hence, the neural network must be able to reliably map an ultra sparse input vector to one of a few thousand basic jobs. This sounds very difficult for me and I doubt a good classification quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem with BOW and TFIDF is, that they cannot handle typos and new words (I guess). They cannot be found in TFIDF's word list, which results in a vector filled with zeros. To sum it up: I was first excited to use TFIDF, but now think it doesn't work well for what I want to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thinking more about it, I now have doubt if neural networks or other machine learning approaches are even good solutions for this task at all. Maybe there are much better algorithms in the field of natural language processing. &#xA;This moment (before digging into NLP) I decided to first gather the opinions of some more experienced AI users, so I don't miss the best solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;strong&gt;what would be a useful approach to this in your opinion&lt;/strong&gt; (best would be an approach that is capable of handling synonyms and typos)? Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p. s.: I am currently thinking about &lt;strong&gt;feeding the whole job description&lt;/strong&gt; into the TFIDF and also do matches for new incoming vacancies with the whole document (instead of job title only). This will expand the size of the word-weight-vector, but it will be less sparse. Does this seem logical to you?&lt;/p&gt;&#xA;" OwnerUserId="17006" LastActivityDate="2019-07-25T14:58:41.863" Title="Is 'job title classification' rather a problem of NLP or machine learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7228" PostTypeId="1" CreationDate="2018-07-20T19:20:46.617" Score="4" ViewCount="223" Body="&lt;p&gt;What loss function should one use, knowing that input image contains exactly one target object? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using MSE to predict center of ROI coordinates and it's width and height. All values are relative to image size. I think that such approach does not put enough pressure on fact, that those coordinates are related.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of existence of algorithms like YOLO or UnitBox, and am just wondering if there might be some shortcut for such particular case.&lt;/p&gt;&#xA;" OwnerUserId="16929" LastEditorUserId="16929" LastEditDate="2018-07-20T19:43:39.293" LastActivityDate="2018-07-24T12:49:41.107" Title="Loss function for singular object detection" Tags="&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7229" PostTypeId="2" ParentId="7228" CreationDate="2018-07-20T20:27:26.153" Score="2" Body="&lt;p&gt;The terms error and loss are always relative to some ideal of fitness, and it is always against some model, the extension of the concept of curve fitting.  The ideal is the optimized state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we are analyzing system behavior and optimizing PROCESS parameters as we do in machine learning, we don't always use the same terms.  Just as people in medicine and alternative medicine speak of wellness, systems that process data instead of food also can be considered either well or dysfunctional.  With machine vision, we optimize the processing of an arbitrary signal, an image with an object depicted in your case, not the parameters of a curve to fit a set of data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is common to wish to judge the wellness of the learning state across a range of sizes.  You cannot normalize the image by re-sizing before you recognize the image boundaries, so you have to design the process (circuit) that must be trained accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not in the error function that this can be done.  There are only two places where it can be done.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The model of the object you are trying to recognize&lt;/li&gt;&#xA;&lt;li&gt;The process topology you are attempting to train&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The standard approach for size-independent recognition involves a sequence in the process being trained, reflected in activation and interconnection choices.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Edge detection&lt;/li&gt;&#xA;&lt;li&gt;Corner detection&lt;/li&gt;&#xA;&lt;li&gt;Shape detection&lt;/li&gt;&#xA;&lt;li&gt;Illumination detection (for higher reliability)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that edge detection and corner detection can overlap in learning layers but not necessarily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key is obviously between corner and shape detection.  The feed of corners into the layer that detects shape has to be &lt;strong&gt;angular rather than Cartesian&lt;/strong&gt; yielding this sequence.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Edge detection&lt;/li&gt;&#xA;&lt;li&gt;Corner detection&lt;/li&gt;&#xA;&lt;li&gt;Angles at corners&lt;/li&gt;&#xA;&lt;li&gt;Shape/form detection&lt;/li&gt;&#xA;&lt;li&gt;Illumination detection (for higher reliability)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A designer can hope a deep network will learn these through back-propagation, but convergence upon the ideal behavior (wellness) will be faster and more reliable if the layers are designed to perform those functions.  That means your model for the shape or form you hope to detect must be in terms of angles not coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Florian Pierre and Joseph Piewak seem to be doing it correctly: &lt;a href=&quot;https://arxiv.org/pdf/1709.03138.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1709.03138.pdf&lt;/a&gt;. Search the document with ctrl-f for &quot;angl&quot;.  These researchers show excellence in their treatment: &lt;a href=&quot;https://arxiv.org/pdf/1806.07996.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1806.07996.pdf&lt;/a&gt; [2]. Search for &quot;size indep&quot; in that one.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Fully Convolutional Neural Networks for Dynamic Object Detection in Grid Maps; Master’s Thesis of Florian Pierre, Joseph Piewak; Department of Computer Science, Institute for Anthropomatics and FZI Research Center for Information Technology respectively; 2016&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Novel Convolution Kernels for Computer Vision and Shape Analysis based on Electromagnetism; Beaini, Achiche, Cio, Raison; 2018; Polytechnique Montreal, 2900 Edouard Montpetit Blvd, Montreal, H3T 1J4, Canada&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-20T20:41:40.607" LastActivityDate="2018-07-20T20:41:40.607" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7230" PostTypeId="2" ParentId="7214" CreationDate="2018-07-20T21:12:52.253" Score="1" Body="&lt;p&gt;You have to define a function between voice and a corresponding sound, but there is no a best way to do it, it depends on what kind od output you care, for example you could create an algorithm that receives music and siging as separate &quot;files&quot; and then that algorithm finds the relation between them now you can give that algorithm a new music only file and it will create the singing part with the function of the previous music-singing &quot;system&quot;.&#xA;This is sing this music as the other one was sang.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least that is the way i would do it.&lt;/p&gt;&#xA;" OwnerUserId="17018" LastActivityDate="2018-07-20T21:12:52.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7231" PostTypeId="1" AcceptedAnswerId="7236" CreationDate="2018-07-20T23:38:25.483" Score="6" ViewCount="773" Body="&lt;p&gt;I know that there are several optimizations for alpha-beta pruning. For example, I have come across iterative deepening, principal variation search, or quiescence search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am a little bit confused about the nature of these algorithms.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are these algorithms an extension of the alpha-beta algorithm, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are they completely new algorithms, in that they have got nothing to do the alpha-beta algorithm?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;On &lt;a href=&quot;https://web.archive.org/web/20180714104141/https://chessprogramming.wikispaces.com/Search&quot; rel=&quot;nofollow noreferrer&quot;&gt;this site&lt;/a&gt;, these algorithms fall into one of 4 categories, namely&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;mandatory&lt;/li&gt;&#xA;&lt;li&gt;selectivity&lt;/li&gt;&#xA;&lt;li&gt;scout and friends &lt;/li&gt;&#xA;&lt;li&gt;Alpha-beta goes best-first&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Does this mean that the alpha-beta algorithm is split into four areas and that they are specialized optimization algorithms for each area?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I even begin to decide which optimized algorithm to pick?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I advise people to visit this site:&#xA;&lt;a href=&quot;http://www.fierz.ch/strategy2.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.fierz.ch/strategy2.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="2444" LastEditDate="2020-05-13T21:03:41.000" LastActivityDate="2020-05-13T21:04:10.447" Title="Are iterative deepening, principal variation search or quiescence search extensions of alpha-beta pruning?" Tags="&lt;terminology&gt;&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;&lt;quiescence-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7235" PostTypeId="1" CreationDate="2018-07-21T09:02:49.037" Score="2" ViewCount="69" Body="&lt;p&gt;I am learning AI and trying out my first real-life AI application.&#xA;What I am trying to do is taking as an input various sentences, and then classifying the sentences into one of X number of categories based on keywords, and 'action' in the sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The keywords are, for example, Merger, Acquisition, Award, product launch, etc. so in essence, I am trying to detect if the sentence in question talks about a merger between two organizations, or an acquisition by an organization, a person or an organization winning an award, or launching of a new product, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, I have made custom models based on the basic NLTK package model, for each keyword, and trying to improve the classification by dynamically tagging/updating the models with related keywords, synonyms, etc to improve the detection capability. Also, given a set of sentences, I am presenting the user with the detected categorization and asking whether it is correct or wrong, and if wrong, what is the correct categorization, and also identify the entities (company names, person names, product names, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the object is to first classify the sentence into a category, and additionally, detect the named entities in the sentence, based on the category.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is, to be able to automatically re-train the models based on this feedback to improve its performance over time and to be able to retrain with as little manual intervention as possible. For the sake of this project, we can assume that user feedback would be accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I am facing is that NLK is allowing fixed-length entities while training, so for example a two-word award is being detected as two awards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should be my approach to solve this problem? Is there a better NLU (even a commercial one) that can address this problem? It seems to me that this would be a common AI problem, and I am missing something basic. I would love you guys to have any input on this.&lt;/p&gt;&#xA;" OwnerUserId="17028" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:23.557" LastActivityDate="2020-05-29T13:48:23.557" Title="Sentence classification and named identity detection with automatic retraining" Tags="&lt;training&gt;&lt;natural-language-processing&gt;&lt;getting-started&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7236" PostTypeId="2" ParentId="7231" CreationDate="2018-07-21T10:05:43.973" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Are these algorithms an extension of the alpha-beta algorithm, or&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;Are they completely new algorithms, in that they have got nothing to do the alpha-beta algorithm?&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of them are extensions of the Alpha-Beta pruning algorithm. For example, Iterative Deepening is almost the same as Alpha-Beta pruning, but automatically keeps repeating the algorithm with gradually-increasing depth limits until some time limit is reached, rather than just running once for a pre-determined depth limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Principal Variation Search also still uses Alpha-Beta as a basis, but performs many searches with significantly smaller &lt;code&gt;[alpha, beta]&lt;/code&gt; windows than the standard Alpha-Beta pruning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, these extensions would start out from an existing Alpha-Beta implementation, and build from there with some adaptations in the code. This is not necessarily the case for all of those extensions though, just for most. For example, Transposition Tables are kind of a separate extension that could be plugged into vanilla Minimax, or Alpha-Beta, or Principal Variation Search, or whatever you're using.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On this site, these algorithms fall into one of 4 categories, namely&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;mandatory &lt;/li&gt;&#xA;  &lt;li&gt;selectivity&lt;/li&gt;&#xA;  &lt;li&gt;scout and friends &lt;/li&gt;&#xA;  &lt;li&gt;Alpha-beta goes best-first&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Does this mean that the alpha-beta algorithm is split into four areas and that they are specialized optimization algorithms for each area?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Those four categories are not mutually exclusive, they're more like... broad &quot;flavours&quot;. What they list under Obligatory are some of the more basic extensions that any programmer should probably look into first if they were developing a chess-playing program. The other category are different &quot;flavours&quot;, different &quot;broad ideas&quot;. For example, everything listed under &quot;Selectivity&quot; is about searching &quot;interesting&quot; or &quot;exciting&quot; parts of the search tree deeper than &quot;less interesting&quot; or &quot;boring&quot; parts. Many of those ideas could be used regardless of whether you're using Alpha-Beta, Iterative Deepening, or PVS, and probably all could be combined with Transposition Tables as well.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I even begin to decide which optimized algorithm to pick?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is really really difficult to decide just based on the names. In theory, which algorithm is the &quot;best&quot; will also highly depend on your specific game, and maybe even hardware. And, in many cases it's not even a choice between mutually exclusive parts; different ideas can be combined with each other in different ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only solution here is really just to do lots of reading, lots of research, try implementing different things to better understand them.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-05-13T21:04:10.447" LastActivityDate="2020-05-13T21:04:10.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7237" PostTypeId="1" CreationDate="2018-07-21T15:53:17.547" Score="1" ViewCount="260" Body="&lt;p&gt;I've already seen many articles about this topic and &lt;a href=&quot;http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Backpropagation In Convolutional Neural Networks&lt;/a&gt; by Jefkine (5 September 2016) seems to be the best. Although, as author said, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For the purposes of simplicity we shall use the case where the input image is grayscale i.e single channel C = 1.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also, he uses stride = 1 and assumes only 1 filter, for the same purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the final equations for backpropagation (taken from the article):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M91A6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M91A6.png&quot; alt=&quot;Equations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The autor's notation explained:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G9Quq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G9Quq.png&quot; alt=&quot;Notation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figured out how to do the forward pass with stride, depth and more filters, but couldn't do the same with the backpropagation. Do you know where to put the 3rd dimension, stride and filters number in those equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how to backpropagate the bias (assuming there's 1 bias per filter)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16578" LastEditorUserId="16578" LastEditDate="2018-07-22T11:16:46.997" LastActivityDate="2018-07-22T11:16:46.997" Title="Backpropagation of convolutional neural network - confusion" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2018-07-22T15:46:52.083" ContentLicense="CC BY-SA 4.0" />
  <row Id="7238" PostTypeId="2" ParentId="7090" CreationDate="2018-07-21T16:20:45.313" Score="4" Body="&lt;p&gt;I found that there are cuDNN accelerated cells in Keras for example: &lt;a href=&quot;https://keras.io/layers/recurrent/#cudnnlstm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/layers/recurrent/#cudnnlstm&lt;/a&gt;&#xA;They very fast. The normal LSTM cells are faster on CPU then on GPU.&#xA;Also see here for a comparisem: &lt;a href=&quot;https://wiki.eniak.de/ml/geschwindigkeitsvergleich_keras_lstm_und_cudnnlstm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wiki.eniak.de/ml/geschwindigkeitsvergleich_keras_lstm_und_cudnnlstm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16687" LastActivityDate="2018-07-21T16:20:45.313" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7243" PostTypeId="2" ParentId="7207" CreationDate="2018-07-22T10:45:08.377" Score="2" Body="&lt;p&gt;Begin by learning the mathematical treatments at their foundations.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Game theory pioneered John von Neumann and Oskar Morgenstern&lt;/li&gt;&#xA;&lt;li&gt;Information theory pioneered by Claude Shannon (Bell Labs)&lt;/li&gt;&#xA;&lt;li&gt;Incompleteness pioneered by Kurt Gödel (which led to Alonso Church's lambda calculus and Turing's completeness which led to a general criteria that defines what a programming language must do to be called general purpose.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You could take a course in Finite Math and Discrete Math, but there is no better way to understand than to go to the authors of the original ideas, which is why MIT and Cal Tech require it of freshmen.  We don't need to be genius to understand genius.  We just need to take the time to read.  That's the genius of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is good to evaluate the pruning of decision trees mathematically, and modeling the general and specific cases is the right approach.  Congratulations for seeing that.  The code for pruning is far more developed than the mathematics for modeling it.  The literature usually shows speed results just before the conclusions, which is more like sitting in the audience with pop corn and a wager ticket than understanding the anatomy and proper care of a horse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you understand what occurred a century ago that led to all computer science and see what mathematical conventions originated and in what context then you can read work like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://people.cs.georgetown.edu/~cnewport/teaching/cosc844-spring17/pubs/nn-tm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Appl. Math. L&amp;amp;t. Vol. 4, No. 6, pp. 77-80, 1991 Printed in Great Britain. 08939659191 Pergamon Press plc Turing Computability with Neural Nets T. SIEGELMANN AND EDUARDO D. SONTAG Department of Computer Science, Rutgers University Department of Mathematics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... and ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19910014493.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Decision Tree Classifier Methodology S. Rasoul Safavian David Landgrebe TR-EE 90-54 September, 1990 School of Electrical Engineering Purdue University West Lafayette, Indiana 47907 (NASA-CR-188208)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;WARNING: Don't be surprised if you are disappointed by most recent work after reading von Neuman, Shannon, and Gödel.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-24T12:08:55.267" LastActivityDate="2018-07-24T12:08:55.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7244" PostTypeId="1" CreationDate="2018-07-22T11:03:06.653" Score="4" ViewCount="688" Body="&lt;p&gt;I'm now reading a book titled as &lt;em&gt;Hands-On Reinforcement Learning with Python&lt;/em&gt;, and the author explains the discount factor that is used in Reinforcement Learing to discount the future reward, with the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A discount factor of 0 will never learn considering only the immediate rewards; similarly, a discount factor of 1 will learn forever looking for the future reward, which may lead to infinity. &lt;strong&gt;So the optimal value of the discount factor lies between 0.2 to 0.8&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The author seems to be not going to explain further about the figure, but all the tutorials and explanations I have ever read write the optimal (or at least widely used) discount factor between 0.9 an 0.99. This is the first time I have seen such a low-figure discount factor.&lt;/p&gt;&#xA;&lt;p&gt;All the other explanations the author makes regarding the discount factor are the same as I have read so far.&lt;/p&gt;&#xA;&lt;p&gt;Is the author correct here or does it depend on cases? If it is, then what kind of problems and/or situations should I set the discount factor as low as such figure at?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;EDIT&lt;/h3&gt;&#xA;&lt;p&gt;I just found &lt;a href=&quot;https://www.quora.com/How-should-I-decide-the-discount-factor-in-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;the following answer at Quora&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Of course. A discount factor of 0 will never learn, meanwhile a factor near of 1 will only consider the last learning. A factor equal or greater than 1 will cause the not convergence of the algorithm. Values usually used are [0.2, 0.8]&lt;/p&gt;&#xA;&lt;p&gt;EDIT: That was the learning factor. The discount factor only affect how you use the reward. For a better explanation:&lt;/p&gt;&#xA;&lt;p&gt;State-Action-Reward-State-Action - Wikipedia&lt;/p&gt;&#xA;&lt;p&gt;See influences of variables .&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know what is written in the question as it in not visible in Quora, but it seems that the 0.2 to 0.8 figure is used for learning factor, not discount factor. Maybe the author is confused with it...? I'm not sure what the &lt;em&gt;learning factor&lt;/em&gt; is, though.&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-15T20:29:15.630" Title="Can the optimal value of discount factor in Deep Reinforcement Learning be between 0.2 to 0.8?" Tags="&lt;reinforcement-learning&gt;&lt;discount-factor&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7247" PostTypeId="1" AcceptedAnswerId="7323" CreationDate="2018-07-22T14:12:51.643" Score="12" ViewCount="7244" Body="&lt;p&gt;Is it possible to make a neural network that uses only integers by scaling input and output of each function to [-INT_MAX, INT_MAX]? Is there any drawbacks?&lt;/p&gt;&#xA;" OwnerUserId="17050" LastActivityDate="2018-08-16T13:25:04.813" Title="Why do we need floats for using neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7248" PostTypeId="2" ParentId="7244" CreationDate="2018-07-22T16:19:19.107" Score="2" Body="&lt;p&gt;The discount factor is not something you should be optimising. It is typically part of the problem statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For practical purposes, you may set it below 1.0 for continuous problems when in fact you care about best long-term reward. Another option to avoid infinities on continuous problems is to &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/BF00114727.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;re-formulate the problem as optimising average reward&lt;/a&gt;. A high discount factor of e.g. 0.99 or 0.999 should produce a similar policy as one based on average reward.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the author correct here or does it depend on cases? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The author appears to be either completely wrong, or just poor at explaining themselves on this part. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If it is, then what kind of problems and/or situations should I set the discount factor as low as such figure at?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A low discount factor is for when you care much more about immediate rewards. You set it that low when that is the case. You decide what you care about when you set the learning problem. The value of the discount factor is part of the setup that decides what the optimal policy is. You never set it low &quot;to help with optimising&quot; because changing the value could change the optimal policy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-07-23T07:26:46.657" LastActivityDate="2018-07-23T07:26:46.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7249" PostTypeId="2" ParentId="7247" CreationDate="2018-07-22T16:41:10.763" Score="0" Body="&lt;p&gt;It is possible in principle, but you will end up emulating floating point arithmetic using integers in multiple places, so it is unlikely to be efficient for general use. Training is likely to be an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the output of a layer is scaled to [-INT_MAX, INT_MAX], then you need to multiply those values by weights (which you will also need to be integers with a large enough range that learning can progress smoothly) and sum them up, and then feed them into a non-linear function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are restricting yourself to integer operations only, this will involve handling multiple integers to represent high/low words in a larger int type, that you then must scale (which introduces a multi-word integer division). By the time you have done this, it is unlikely there will be much benefit to using integer arithmetic. Although perhaps it is still possible, depending on the problem, network architecture and your target CPU/GPU etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are instances of working neural networks used in computer vision &lt;a href=&quot;https://petewarden.com/2015/05/23/why-are-eight-bits-enough-for-deep-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;with only 8-bit floating point (downsampled after training)&lt;/a&gt;. So it is definitely possible to simplify and approximate some NNs. An integer-based NN derived from a pre-trained 32-bit fp one could potentially offer good performance in certain embedded environments. I found &lt;a href=&quot;http://sharpneat.sourceforge.net/research/integer-neuralnet/integer-neuralnet.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this experiment on a PC AMD chip&lt;/a&gt; which showed a marginal improvement even on PC architecture. On devices without dedicated fp processing, the relative improvement should be even better.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-07-22T16:41:10.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7250" PostTypeId="2" ParentId="7247" CreationDate="2018-07-22T16:57:06.117" Score="2" Body="&lt;p&gt;Some people might argue we can use &lt;code&gt;int&lt;/code&gt; instead of &lt;code&gt;float&lt;/code&gt; in NN's as &lt;code&gt;float&lt;/code&gt; can easily be represented as an&lt;code&gt;int / k&lt;/code&gt; where &lt;code&gt;k&lt;/code&gt; is a multiplying factor say &lt;code&gt;10 ^ 9&lt;/code&gt; e.g 0.00005 can be converted to 50000 by multiplying with 10 ^ 9..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;From a purely theoretical viewpoint&lt;/strong&gt;: This is definitely possible, but it will result in a loss of precision since &lt;code&gt;int&lt;/code&gt; falls in the &lt;code&gt;INTEGER&lt;/code&gt; set of number whereas &lt;code&gt;floats&lt;/code&gt; fall in the &lt;code&gt;REAL NUMBER&lt;/code&gt; set. Converting real numbers to int's will result in high precision loss if you are using very high precisions e.g. &lt;code&gt;float64&lt;/code&gt;. Real numbers have an uncountable infinity, whereas integers have countable infinity and there is a well known argument called Cantor's diagonalization argument which proves this. &lt;a href=&quot;https://www.coopertoons.com/education/diagonal/diagonalargument.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a beautiful illustration of the same. After understanding the difference you will intuitionally gain an understanding why converting int's to float is not tenable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;From a practical viewpoint:&lt;/strong&gt; The most well known activation activation function is sigmoid activation (tanh is also very similar). The main property of these activatons are they squash numbers to between &lt;code&gt;0 and 1&lt;/code&gt; or &lt;code&gt;-1 and 1&lt;/code&gt;. If you convert floating point to a integer by multiplying with a large factor, which will result in a large number almost always, and pass it to any such function the resulting output will always almost be either of the extremities (i.e &lt;code&gt;1 or 0&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to algorithms, algorithms which are similar to backpropagation with momentum cannot run on &lt;code&gt;int&lt;/code&gt;. This is because,s since you will be scaling the &lt;code&gt;int&lt;/code&gt; to a large number, and momentum algorithms typically use some sort of &lt;strong&gt;&lt;code&gt;momentum_factor^n&lt;/code&gt;&lt;/strong&gt; formulae, where &lt;code&gt;n&lt;/code&gt; is the number of examples iterated already, you can imagine the result if &lt;strong&gt;&lt;code&gt;momentum_factor = 100 and n = 10&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only possible place where scaling might work is for &lt;code&gt;relu&lt;/code&gt; activation. the problems with this approach will be if the data will probably not fit very good in this model, there will be relatively high errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finally:&lt;/strong&gt; All NN's do is approximate a &lt;code&gt;real valued&lt;/code&gt; function. You can try to magnify this function by multiplying it with a factor, but whenever you are switching from, real valued function to integers you are basically representing the function as a series of &lt;code&gt;steps&lt;/code&gt;. Something like this happens (Image only for representation purposes):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1X5o2.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1X5o2.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can clearly see the problem here, each binary number represent a step, to have better accuracy you have to increase the binary steps within a given length, which in your problem will translate to having very high value of bounds [-INT_MAX, INT_MAX].&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-07-29T05:19:54.483" LastActivityDate="2018-07-29T05:19:54.483" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7251" PostTypeId="1" CreationDate="2018-07-22T21:21:00.037" Score="3" ViewCount="47" Body="&lt;p&gt;I'm working on a chatbot for homes and would need to understand the appliance the user is talking about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the command is:&#xA;&lt;em&gt;Turn on the air conditioner.&lt;/em&gt;&#xA;and another variant could be&#xA;&lt;em&gt;Turn on the ac.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;em&gt;ac&lt;/em&gt; and &lt;em&gt;air conditioner&lt;/em&gt; refers to the same appliance but how would the chatbot know that?&lt;/p&gt;&#xA;" OwnerUserId="17052" LastEditorUserId="2193" LastEditDate="2018-07-23T15:17:09.257" LastActivityDate="2019-04-19T20:02:10.380" Title="Recognising common short forms of appliances used daily" Tags="&lt;natural-language-processing&gt;&lt;python&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7252" PostTypeId="1" CreationDate="2018-07-22T22:24:30.813" Score="3" ViewCount="276" Body="&lt;p&gt;I would like to know if having a really good evaluation function is as good as using any of the extensions of alpha-beta pruning, such as killer moves or quiescence search?&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="2444" LastEditDate="2021-02-02T23:16:44.080" LastActivityDate="2021-02-02T23:16:44.080" Title="Is a good evaluation function as good as any of the extensions of alpha-beta pruning?" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;alpha-beta-pruning&gt;&lt;quiescence-search&gt;&lt;evaluation-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7253" PostTypeId="2" ParentId="7252" CreationDate="2018-07-23T07:24:54.997" Score="2" Body="&lt;p&gt;A perfect evaluation function would mean that you only had to do a local search - i.e. maximise over next set of decisions - in order for an agent to behave optimally in an environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such if you could somehow create that function, it would make a search with alpha-beta pruning redundant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, evaluation functions for complex environments are usually approximate, and significant improvement can be made by adding a deeper search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimisations in search algorithms and improvements in evaluation function work together to make more efficient and closer-to-optimal solutions overall. An evaluation function provides global/general knowledge about the environment and goals. A tree search function provides local focus on solving a relatively small subset of the optimisation problem that is currently relevant.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-07-23T07:24:54.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7254" PostTypeId="1" CreationDate="2018-07-23T09:05:00.573" Score="2" ViewCount="516" Body="&lt;p&gt;Certain games, like checkers, have compulsory moves. In checkers, for instance, if there's a jump available a player must take it over any non-jumping move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If jumps are compulsory, will there still be a need for a quiescence search?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thinking is that I can develop an implementation of a quiescence search that first checks whether jumps are available. If there are then it can skip all non-jumping moves. If there's only one jumping move available, then I won't need to run a search at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I will only use a quiescence search if I initially don't have to make a jump on my first move. I will only active quiescence search in my alpha-beta pruning becomes active. (The alpha-beta will only be active if my first algorithm which first checks if there are jumps available returns a 0, which means there  are no jumps available.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my thinking of implementing a quiescence search correct? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My options are slim when it comes to optimizations due to serious memory constraints, hence I won't be using PVS or other algorithm like that as they require additional memory.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="2444" LastEditDate="2020-05-13T20:45:38.663" LastActivityDate="2020-05-13T20:45:38.663" Title="If certain moves are compulsory, will there still be a need for a quiescence search?" Tags="&lt;game-ai&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;&lt;checkers&gt;&lt;quiescence-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7255" PostTypeId="1" CreationDate="2018-07-23T09:39:01.130" Score="4" ViewCount="3424" Body="&lt;p&gt;Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find any resources for that. Is it safe to assume that, since it works for other DNNs, it will also make sense to use it and will offer benefits on training AEs?&lt;/p&gt;&#xA;" OwnerUserId="6899" LastEditorUserId="2444" LastEditDate="2019-12-26T15:43:34.300" LastActivityDate="2019-12-26T15:43:34.300" Title="Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?" Tags="&lt;deep-neural-networks&gt;&lt;autoencoders&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7256" PostTypeId="1" AcceptedAnswerId="7259" CreationDate="2018-07-23T09:59:55.807" Score="4" ViewCount="2230" Body="&lt;p&gt;As it can be easily pointed out that true random numbers cannot be generated fully by programming and some random seed is required.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, humans can easily generate any random number independently of other factors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this suggest that absolute random number generation an AI concept?&lt;/p&gt;&#xA;" OwnerUserId="17056" LastEditorUserId="1671" LastEditDate="2018-07-23T19:26:27.193" LastActivityDate="2018-07-23T19:26:27.193" Title="Is true random number generation an AI concept?" Tags="&lt;ai-basics&gt;&lt;concepts&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7257" PostTypeId="2" ParentId="7255" CreationDate="2018-07-23T11:37:13.270" Score="1" Body="&lt;p&gt;The term &lt;em&gt;Batch Normalization&lt;/em&gt; is both&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A system reliability choice (in terms of convergence) and&lt;/li&gt;&#xA;&lt;li&gt;an execution strategy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Batching is generally the process of focusing on process &lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; with source data &lt;strong&gt;&lt;em&gt;S&lt;/em&gt;&lt;/strong&gt; to produce result &lt;strong&gt;&lt;em&gt;R&lt;/em&gt;&lt;/strong&gt; under conditions that are favorable in terms of timing, data availability, and resource utilization, such as these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; is requires nontrivial time and computing resource and there is a window of time when it can be done on hardware &lt;strong&gt;&lt;em&gt;H&lt;/em&gt;&lt;/strong&gt; with minimal impact on other operations&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;R&lt;/em&gt;&lt;/strong&gt; is needed by multiple other processes &lt;strong&gt;&lt;em&gt;P&lt;sub&gt;2&lt;/sub&gt;&lt;/em&gt;&lt;/strong&gt;, &lt;strong&gt;&lt;em&gt;P&lt;sub&gt;3&lt;/sub&gt;&lt;/em&gt;&lt;/strong&gt;, ... and draws on a set of additional inputs &lt;strong&gt;&lt;em&gt;I&lt;sub&gt;2&lt;/sub&gt;&lt;/em&gt;&lt;/strong&gt;, &lt;strong&gt;&lt;em&gt;I&lt;sub&gt;3&lt;/sub&gt;&lt;/em&gt;&lt;/strong&gt;, ... so that completing &lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; in larger groups is efficient&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One cannot draw a general conclusion about the value of batching that will apply to all specific installations of hardware, networking, real time demands, functional intention, and ML topology.  Also, beware of equating &lt;em&gt;deep&lt;/em&gt; with &lt;em&gt;stacked&lt;/em&gt;.  Depth may have multiple meanings now, but was originally used for the dimension describing the number of layers in a multi-layer perceptron, with width being the number of artificial neurons in each layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term stacking can be ambiguously related to depth.  Consider that process flows through artificial neural nets has often been represented graphically as signal propagation from left to right, as is the convention in electrical engineering and text in western languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then have two directions for stacking that means two different things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stacking in the same direction as signal flow, which indicates a sequence of operations on a vector, matrix, cube, or hyper-cube&lt;/li&gt;&#xA;&lt;li&gt;Stacking in the tangential direction with respect to signal flow, which indicates a connectivity between items in the stack that is related to supervision, collaboration, or competition&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Whether Sparse Autoencoders or other elements of DNNs should be fronted with batch normalization has two forks to the answer.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is normalization helpful in reducing either signal saturation in gradient calculations during propagation or overflow conditions?&lt;/li&gt;&#xA;&lt;li&gt;If that answer is, &quot;Yes,&quot; will batching the normalization have some value for reasons like the ones described above?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-23T11:37:13.270" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7258" PostTypeId="2" ParentId="7256" CreationDate="2018-07-23T12:06:44.307" Score="2" Body="&lt;p&gt;Such a great question. I would concur with Dennis Soemers comment that humans are not great at thinking of random numbers (just think about any card trick). However, we are very good at creating randomness through our actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider moving a computer mouse, the stock market, or playing a lottery, humans are very good at creating randomness through our constructs or actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would pose that maybe randomness needs to be part of a AGI to make it more able to jump out of suboptimal valleys or change it’s topology rather than sticking always to the local minimax.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2018-07-23T12:06:44.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7259" PostTypeId="2" ParentId="7256" CreationDate="2018-07-23T12:34:35.540" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;As it can be easily pointed out that true random numbers cannot be generated fully by programming and some random seed is required.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is true. In fact, it is impossible to solve using software. No software-only technique can generate randomness &lt;em&gt;without&lt;/em&gt; an initial random seed or support from hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is also true for AI software. No AI design that uses deterministic software can do this - e.g. any Turing machine without a magic unexplained &quot;random&quot; function can be shown to remain deterministic, no matter how complex. That's because any combination of deterministic functions is deterministic. It may not be predictable without following the process exactly, it may be &quot;chaotic&quot; and depend critically on initial conditions, but it is 100% deterministic and repeatable.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On the other hand, humans can easily generate any random number independently of other factors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0041531&quot; rel=&quot;noreferrer&quot;&gt;Typically not high quality randomness&lt;/a&gt;. It is not clear &lt;em&gt;how&lt;/em&gt; we make random decisions, but entirely possible that internally we rely effectively on noisiness from the environment or our own internal &quot;hardware&quot; doing something simple such as racing decisions between neurons (timing will vary due to speed of electrical impulses, diffusion time of neuro-transmitters across synapses etc)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does this suggest that absolute random number generation an AI cocept?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think it is orthogonal issue. We can already produce very high quality artificial randomness - better than human quality used by humans for conscious decisions (such as &quot;choose a random number between 1 and 10&quot;). These artificial random number generating systems are part of modern cryptography and are tested thoroughly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially &quot;true&quot; artificial randomness is a solved problem using hardware, and does not involve anything that has traditionally been called AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reverse, AI systems often rely on stochastic functions in order to break symmetry, break ties, regularise models etc. So it does look like some kind of RNG is necessary within an artificial agent. However, even  pseudo-random number generators (PRNGs) seem to be fine for this purpose. Mersenne Twister is a very common choice for generating random numbers inside neural networks for weight initialisation, dataset shuffling, dropout regularisation, or when simulating environments for RL, or taking exploratory actions. Despite the fact that it is not &quot;true&quot; random, a PRNG will work just fine for these purposes.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Working definition of &quot;true&quot; randomness that I use: Even if you know the state of a system as accurately as modern physics allows, the output cannot be predicted with better accuracy than a fixed guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human randomness already fails this test. If you ask someone to choose a number randomly between 1 and 9, you will generally have a better than 1 in 9 chance of guessing the correct value, based on statistical analysis. If we were able to take good state measurements of brains, it might be possible to predict with high accuracy - although this is unknown and not possible with current technology.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-07-23T13:08:43.550" LastActivityDate="2018-07-23T13:08:43.550" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7260" PostTypeId="2" ParentId="7254" CreationDate="2018-07-23T13:19:44.157" Score="4" Body="&lt;p&gt;I understand your question to be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If some moves are compulsory, and my agent has no choice about which move to make next, do I need to perform a search, or can I just return the compulsory move?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The answer depends on what your goal is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your goal is to make an interactive agent that will play the game against you, then you are correct: there's no need to perform a search. Just return the compulsory move, and then run the search next time your agent has a choice about what to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your goal is to determine the optimal way to play a game, or the expected payoff from a certain game position (another common use of search techniques), then you should run the search as normal, since the forced move won't necessarily lead to a particular end state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tangentially, if you're interested in ways to speed up search for checkers, check out the Chinook papers. There's a popularized account &lt;a href=&quot;https://www.aaai.org/ojs/index.php/aimagazine/article/viewFile/1208/1109&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and more technical ones &lt;a href=&quot;https://mycourses.aalto.fi/pluginfile.php/273655/course/section/60890/ICS-E4000_checkers_is_solved.solved.science.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.ijcai.org/Proceedings/05/Papers/0515.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; by Schaeffer et al.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-07-23T13:19:44.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7261" PostTypeId="2" ParentId="7252" CreationDate="2018-07-23T13:34:21.127" Score="4" Body="&lt;p&gt;To build on Neil's answer a bit, you're right that the better your evaluation function gets, the less work your optimization function will need to perform. If your evaluation function gets good enough, you won't need to search at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not just an academic idea though! It's actually fairly widely used, and has been key to solving several games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first example I'm aware of is Tesauro's &lt;a href=&quot;http://enzodesiage.com/wp-content/uploads/2017/08/tesauro-tdgammon-1995.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD-Gammon player&lt;/a&gt;, from 1995. Tesauro used the ideas of reinforcement learning and self-play to train a Neural Network to act as an evaluation function. TD-Gammon played with just a 2-move lookahead using the best evaluation function that was found, and was deemed better than most (all?) human expert players at the time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, &lt;a href=&quot;http://file:///Users/jdoucette/Downloads/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero&lt;/a&gt; used similar techniques to solve Go, but learning both an evaluation function and (separately) a function to randomize over possible moves.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-07-23T13:34:21.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7262" PostTypeId="2" ParentId="7251" CreationDate="2018-07-23T13:39:51.037" Score="1" Body="&lt;p&gt;This is a pretty broad topic!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question you're interested in has several parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first part is &lt;a href=&quot;https://en.wikipedia.org/wiki/Named-entity_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Named Entity Recognition&lt;/a&gt;. This NLP task focuses on identifying the types of named objects. In your case, you'd want to be able to tell that something is an appliance (or perhaps even, that it is an air conditioner).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that's not enough on its own. You also need to be able to tell &lt;em&gt;which&lt;/em&gt; appliance they're talking about! This is usually studied as a separate problem, called &lt;a href=&quot;https://en.wikipedia.org/wiki/Entity_linking&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entity Recognition&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of these are considered pretty hard, and there isn't likely to be one &quot;right&quot; answer for your specific problem, but there are viable techniques. The wikipedia articles I linked to should be good starting places and suggest some out-of-the-box toolkits. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-07-23T13:39:51.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7263" PostTypeId="1" CreationDate="2018-07-23T14:37:52.583" Score="4" ViewCount="109" Body="&lt;p&gt;The ability to recognize an object with particular identifying features from single or multiple camera shoots with the temporal dimension digitized as frames has been shown.  The proof is that the movie industry does face replacement to reduce liability costs for stars when stunts are needed.  It is now done in a substantial percentage of action movie releases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings up the question of how valuable recognizing a stop sign is compared to the value of recognizing an action.  For instance, in the world of autonomous vehicles, should there even be stop signs.  Stop signs are designed for lack of intelligence or lack of attention, which is why any police officer will tell you that almost no one comes to a full stop per law.  What human brains intuitively looks for is the potential of collision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once what we linguistically perceive as verbs can be handled in deep learning scenarios as proficiently as nouns can be handled, the projection of risk becomes possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may be very much the philosophy behind the proprietary technology that allows directors to say, &quot;Replace the stunt person's face with the movie's protagonist's face,&quot; and have a body of experts execute it using software tools and LINUX clusters.  The star's face is projected into the model of the action realized in the digital record of the stunt person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Projected action is exactly what our brain does when we avoid collisions, and not just with driving.  We do it socially, financially, when we design mechanical mechanisms, and in hundreds of other fields of human endeavor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we consider the topology of GANs as a loop in balance, which is what it is, we can then see the similarity of GANs to the chemical equilibria between suspensions and solutions.  This gives us a hint into the type of topologies that can project action and therefore detect risk from audiovisual data streams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once action recognition is mastered, it is a smaller step to use the trained model to project the next set of frames and then detect collision or other risks.  Such would most likely make possible a more reliable and safe automation of a number of AI products and services, breaking through a threshold in ML, and increased safety margins throughout the ever increasing world population density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... which brings us back to ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What topologies support recognition of action sequences?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The topology may have convolution, perhaps in conjunction with RNN techniques, encoders, equilibria such as the generative and discriminative models in GANs, and other design elements and concepts.  Perhaps a new element type or concept will need to be invented.  Will we have to first recognize actions in a frame sequence and then project the consequences of various options in frames that are not yet shot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where would the building blocks go and how would they be connected, initially dismissing concerns about computing power, network realization, and throughput for now?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Work may have been done along this area and realized in software, but I have not seen that degree of maturity yet in the literature, so most of it, if there is any, must be proprietary at this time.  It is useful to open the question to the AI community and level the playing field.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-25T10:02:01.673" LastActivityDate="2018-07-25T10:02:01.673" Title="What topologies support recognition of action sequences?" Tags="&lt;machine-learning&gt;&lt;ai-safety&gt;&lt;action-recognition&gt;&lt;topology&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7264" PostTypeId="1" AcceptedAnswerId="7266" CreationDate="2018-07-23T15:00:56.247" Score="2" ViewCount="244" Body="&lt;p&gt;For my pet project I’m looking for a grid-like world simulation with some kind of resources that requires from agent incrementally intelligent behaviour to survive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like &lt;a href=&quot;https://store.steampowered.com/app/396890/Gridworld/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this steam game&lt;/a&gt;, but with API.  I’ve seen &lt;a href=&quot;https://github.com/Microsoft/malmo&quot; rel=&quot;nofollow noreferrer&quot;&gt;minecraft fork&lt;/a&gt;, but it’s too complex for my task. There is &lt;a href=&quot;https://github.com/deepmind/pycolab&quot; rel=&quot;nofollow noreferrer&quot;&gt;pycolab&lt;/a&gt;, i can build some world on this engine, but I’d prefer ready-to-use simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any option? I'll appreciate any suggestion.&lt;/p&gt;&#xA;" OwnerUserId="16940" LastEditorUserId="1671" LastEditDate="2018-07-23T19:25:57.613" LastActivityDate="2018-07-23T19:25:57.613" Title="Is there any opensource 2d open-world simulation with python API?" Tags="&lt;getting-started&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-02-06T18:05:39.547" ContentLicense="CC BY-SA 4.0" />
  <row Id="7265" PostTypeId="2" ParentId="2712" CreationDate="2018-07-23T15:17:08.587" Score="1" Body="&lt;p&gt;A very large one, the world wide web with highly scaled and optimized indexing by Google.com is the most distributed and robust schema-agnostic database known today.  Without the schema-awareness Google brought to the table by applying more rigorous information science to the table, it was almost useless to those that did not know the URL of the target document in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Schema agnosticism is another way of saying that the database cannot&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Provide meta information to the services accessing it,&lt;/li&gt;&#xA;&lt;li&gt;Normalize the structure using simple SQL query-insert combinations&lt;/li&gt;&#xA;&lt;li&gt;Proactively optimize the keys automatically as is now possible with machine learning, or&lt;/li&gt;&#xA;&lt;li&gt;Validate insertions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Without first detecting a schema from data patterns.  Moving away from structure is appealing because you can just jam data in like a librarian without a book shelf.   However, the data scientist will point out that adding entropy working alongside thermodynamic devolution into stochasm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of storing data is to be able to retrieve it.  Feature extraction is an opportunity to improve structure automatically during the storing structure, rather than store documents chaotically, a trend that will not lead anywhere good for the world of IT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider whether Google is successful because it organizes its data as it crawls or later as we enter key phrases.  Which is the efficient sequence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more point, Wikipedia is a blog, and they know this, which is why they want peer review for everything now (after much of the information was added without peer review).  It is a good place to find lists but not verified facts.  The existence of a Wikipedia page is definitely not an indication of the value of the concept on it.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-23T15:17:08.587" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7266" PostTypeId="2" ParentId="7264" CreationDate="2018-07-23T15:33:16.867" Score="2" Body="&lt;p&gt;You could try &lt;a href=&quot;https://github.com/projectmesa/mesa&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mesa&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has &lt;a href=&quot;https://github.com/projectmesa/mesa/tree/master/examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;various examples&lt;/a&gt; that are commonly-used in agent-based modelling, like Epstein's model, a wolf/sheep predator/prey model, and many more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also an &lt;a href=&quot;http://mesa.readthedocs.io/en/latest/tutorials/intro_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;introductory tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-23T15:33:16.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7267" PostTypeId="2" ParentId="5955" CreationDate="2018-07-23T17:05:42.730" Score="3" Body="&lt;p&gt;The common statement that Artificial Neural Networks are inspired by the neural structure of brains is only partially true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that Norbert Wiener, Claude Shannon, John von Neuman, and others began the path toward practical AI by developing what they then called the electronic brain. It is also true&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Artificial networks have functions called activations,&lt;/li&gt;&#xA;&lt;li&gt;Are wired in many-to-many relationships like biological neurons, and&lt;/li&gt;&#xA;&lt;li&gt;Are designed to learn an optimal behavior,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;but that is the extent of the similarity. Cells in artificial networks such as MLPs (multilayer perceptrons) or RNN (Recurrent neural networks) are not like cells in brain networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The perceptron, the first software stab at arrays of things that activate, was not an array of neurons. It was the application of basic feedback involving gradients, which had been in common use in engineering ever since James Watt's centrifugal governor was mathematically modeled by Gauss. Successive approximation, a principle that had been in use for centuries, was employed to incrementally update an attenuation matrix. The matrix was multiplied by the vector feeding an array of identical activation functions to produce output. That's it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The projection in a second dimension to a multi-layer topology was made possible by the realization that the Jacobian could be used to produce a corrective signal that, when distributed as negative feedback to the layers appropriately, could tune the attenuation matrix of a sequence of perceptrons and the network as a whole would converge upon satisfactory behavior. In the sequence of perceptrons, each element is called a layer.  The feedback mechanism is now called back propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematics used to correct the network is called gradient descent because it is like a dehydrated blind man using the gradient of the terrain to find water, and the issues of doing that are similar too.  He might find a local minima (low point) before he finds fresh water and converge on death rather than hydration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The newer topologies are the additions of already existing convolution work used in digital image restoration, mail sorting, and graphics applications to create the CNN family of topologies and the ingenious use of what is like a chemical equilibrium from first year chemistry to combine optimization criteria creating the GAN family of topologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep is simply a synonym for numerous in most AI contexts.  It sometimes infers complexity in the higher level topology (above the vector-matrix products, the activations, and the convolutions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Active research is ongoing by those who are aware how different these deep networks are from what neural scientists have discovered decades ago in mammalian brain tissue.  And there are more differentiators being discovered today as learning circuitry and neuro-chemistry in the brain is investigated from the genomic perspective.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neural plasticity ... change in circuit topology due to dendrite and axiom growth, death, redirection, and other morphing&lt;/li&gt;&#xA;&lt;li&gt;Topological complexity ... large number of axioms crisscross without interacting and are deliberately shielded from cross-talk (independent) most likely because it would be disadvantageous to let them connect &lt;sup&gt;[note 1]&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Chemical signaling ... mammalian brains have dozens of neuro-transmitter and neuro-regulation compounds that have regional effects on circuitry &lt;sup&gt;[note 2]&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Organelles ... living cells have many substructures and it is known that several types have complex relationships with signal transmission in neurons&lt;/li&gt;&#xA;&lt;li&gt;Entirely different form of activation ... activations in common artificial neural nets are simply functions with ordinal scalars for both range and domain ... mammalian neurons operate as a function of both amplitude and relative temporal proximity of incoming signals &lt;sup&gt;[note 3]&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;[1] Topology is ironically both a subset of architecture (in the fields of building design, network provisioning, WWW analysis, and semantic networks), yet at the same time topology is, much more than architecture, at the radical center of both AI mathematics and effective actualization in control systems&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] The role of chemistry may be essential to learning social and reproductive behavior that interrelates with DNA information propagation, linking in complex ways learning at the level of an ecosystem and the brain.  Furthermore, long term and short term learning divides the brain's learning into two distinct capabilities too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] The impact of the timing of incoming signals on biological neuron activation is understood to some degree, but it may impact much more than neuron output.  It may impact placticity and chemistry too, and the organelles may play a role in that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What machine learning libraries do is as much simulating the human brain as Barbie and Ken dolls simulate a real couple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, remarkable things are arising in the field of deep learning, and it would not surprise me if autonomous vehicles become fully autonomous in our lifetimes.  I would not recommend to any students to become a developer either.  Computers will probably code much better than humans and orders of magnitude faster, and possibly soon.  Some tasks are not of the kind that biology has evolved to do and computers can exceed human capabilities after only a few decades of research, eventually exceeding human performance by several orders of magnitude.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-11T15:40:00.573" LastActivityDate="2019-01-11T15:40:00.573" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7268" PostTypeId="1" CreationDate="2018-07-23T17:58:39.040" Score="0" ViewCount="56" Body="&lt;p&gt;Deep learning is based on getting a large number of samples and essentially making statistical deductions and outputting probabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand we have formal programming languages like PROLOG which don't involve probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any essential reason why an AI could be called conscious without being able to learn in a statistical manner? i.e. by only being able to make logical deduction alone. (It could start with a vast number of innate abilities).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is probability and statistical inference a vital part of being conscious?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="1671" LastEditDate="2018-07-23T19:12:07.997" LastActivityDate="2018-07-23T21:21:43.400" Title="How important will statistical learning be to a conscious AI?" Tags="&lt;theory&gt;&lt;probability&gt;&lt;artificial-consciousness&gt;&lt;soft-question&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7269" PostTypeId="2" ParentId="7268" CreationDate="2018-07-23T21:21:43.400" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_learning_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Learning Theory&lt;/a&gt; gives us an interesting framework to understand what statistical learning is doing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gist of it is, we can model the process of statistical learning as one of formal deduction. The learning itself does not require a random element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shouldn't be too surprising. Consider a classic decision tree learner like C4.5 or ID3: the algorithm works through the data deterministically, and no random decisions are made. When asked to make a prediction, the learned model returns the frequency of each possible label within the most similar subpopulation from its training data, with similarity defined according to the algorithm's rules for partitioning data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no reason you can't write a decision tree learner (or even a deep learning algorithm) in Prolog, it just might not be very efficient, or very practical.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-07-23T21:21:43.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7270" PostTypeId="2" ParentId="5665" CreationDate="2018-07-23T21:32:42.410" Score="1" Body="&lt;p&gt;It's not really clear what you are asking here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean that a neuron in one layer can have connections to &lt;em&gt;many&lt;/em&gt; neurons in the next layer, then this is already the standard topology. In fact, it's the only one that's interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean that a neuron in one layer can be &quot;on&quot; (firing) for some neurons in the next layer, but &quot;off&quot; (not firing) for other neurons in the next layer, then this is also already a standard feature: setting the weights between the layers appropriately (i.e. to 0) will cause a given neuron to appear off. Setting the weights to other values will determine the degree of signal passed to each neuron in the next layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think the answer to your question is that it would be identical to a &quot;single&quot; firing.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-07-23T21:32:42.410" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7271" PostTypeId="2" ParentId="7263" CreationDate="2018-07-23T21:56:01.937" Score="1" Body="&lt;p&gt;This is an old area of AI called &quot;&lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C33&amp;amp;authuser=1&amp;amp;q=plan%20recognition&amp;amp;btnG=&amp;amp;oq=Plan%20Rec&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plan Recognition&lt;/a&gt;&quot;, which has about 3.5 million results in Google Scholar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of the modern work is done with &lt;a href=&quot;https://www.ijcai.org/proceedings/2018/0668.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;classical search techniques&lt;/a&gt; coupled with &lt;a href=&quot;https://www.ijcai.org/proceedings/2018/0864.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;expert domain knowledge&lt;/a&gt;, or related reasoning concepts like &lt;a href=&quot;http://www.planrec.org/PAIR/PAIR18/Papers/HollerPair18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical Task Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of or able to find recent research using deep neural networks for this problem, but I think there are some data-drive approaches to the related work in video-game player modeling.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-07-23T21:56:01.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7273" PostTypeId="1" AcceptedAnswerId="7278" CreationDate="2018-07-24T11:26:17.413" Score="3" ViewCount="150" Body="&lt;p&gt;This seems like a natural fit, though I've not heard of any, yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would love to know if any MET office, government, military or academic institution has taken all (or sizeable portion of) recorded global weather data for, say, the last 50 years (or since we, as a race, have been using weather satellites) and used it in an AI system to predict future weather.&lt;/p&gt;&#xA;" OwnerUserId="10938" LastActivityDate="2018-07-24T16:00:17.683" Title="Are any organisations using AI to predict weather?" Tags="&lt;prediction&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7274" PostTypeId="1" CreationDate="2018-07-24T12:47:42.820" Score="13" ViewCount="14024" Body="&lt;p&gt;I think that the advantage of using Leaky ReLU instead of ReLU is that in this way we cannot have vanishing gradient. Parametric ReLU has the same advantage with the only difference that the slope of the output for negative inputs is a learnable parameter while in the Leaky ReLU it's a hyperparameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not able to tell if there are cases where is more convenient to use ReLU instead of Leaky ReLU or Parametric ReLU.&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="1671" LastEditDate="2018-07-24T20:03:23.930" LastActivityDate="2020-05-19T10:37:56.660" Title="What are the advantages of ReLU vs Leaky ReLU and Parametric ReLU (if any)?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7275" PostTypeId="2" ParentId="7228" CreationDate="2018-07-24T12:49:41.107" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I am currently using MSE to predict center of ROI coordinates and it's width and height. All values are relative to image size. I think that such approach does not put enough pressure on fact, that those coordinates are related.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;At first glance this looks quite reasonable. Computer vision is not really my main area of expertise, so I did some googling around, and &lt;a href=&quot;https://github.com/lars76/object-localization&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of the first repositories I ran into does something very similar&lt;/a&gt;. It may be interesting for you to look into the code and the references in that repository in more detail. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks to me like they're also using the &lt;a href=&quot;https://github.com/lars76/object-localization/blob/master/train_model.py#L90&quot; rel=&quot;nofollow noreferrer&quot;&gt;MSE&lt;/a&gt; loss function. I'm not 100% sure how they define the bounding boxes, maybe you can figure it out by digging through the code. You currently define bounding boxes by:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;X coordinate of center of bounding box&lt;/li&gt;&#xA;&lt;li&gt;Y coordinate of center of bounding box&lt;/li&gt;&#xA;&lt;li&gt;Width of bounding box&lt;/li&gt;&#xA;&lt;li&gt;Height of bounding box&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You are right in that these coordinates are quite closely related. If the center is incorrect (for example, a bit too far to the right), that mistake could partially be &quot;fixed&quot; by taking a greater width (the bounding box would go a bit too far to the right, but still encapsulate the object). I don't know if this is necessarily a problem, or a fact that should be exploited in some way or something that should be &quot;put pressure on&quot;. If this is something you are concerned about, I suppose you could alternatively define the bounding box as follows (I'm not sure whether or not this is what's done in the repository linked above):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;X coordinate of top-left corner of bounding box&lt;/li&gt;&#xA;&lt;li&gt;Y coordinate of top-left corner of bounding box&lt;/li&gt;&#xA;&lt;li&gt;X coordinate of bottom-right corner of bounding box&lt;/li&gt;&#xA;&lt;li&gt;Y coordinate of bottom-right corner of bounding box&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I suspect the relation between those two corner points will be less strong than the relation you identified exists between center + width + height. A &quot;mistake&quot; in coordinates of the top-left corner cannot be partially &quot;fixed&quot; by placing the bottom-right corner somewhere else.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-24T12:49:41.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7276" PostTypeId="2" ParentId="1481" CreationDate="2018-07-24T13:12:28.507" Score="0" Body="&lt;p&gt;&lt;strong&gt;No General Movie Search Yet&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There have been successes in recognizing a very narrow sequence of a very narrow set of possible actions, but nothing like a general movie searching system that can return a set of matches with the start time, end time, and movie instance for each match to one of the search criteria listed in this question.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Somebody was driving a car&lt;/li&gt;&#xA;&lt;li&gt;Kissing&lt;/li&gt;&#xA;&lt;li&gt;Eating&lt;/li&gt;&#xA;&lt;li&gt;Scared&lt;/li&gt;&#xA;&lt;li&gt;Talking over the phone&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Normalizing the List&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;First of all, &amp;quot;Was scared,&amp;quot; is not the description of an action.  It should be, &amp;quot;Becoming scared.&amp;quot;  Secondly, &amp;quot;Talking over the phone,&amp;quot; is not a proper action description.  It should be a conjunctive action such as, &amp;quot;Talking into a phone &lt;em&gt;AND&lt;/em&gt; listening to the same phone.&amp;quot;  To make the list homogenous in format, the first item should be &amp;quot;Car driving,&amp;quot; since the actor is human in every other case.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Car driving&lt;/li&gt;&#xA;&lt;li&gt;Kissing&lt;/li&gt;&#xA;&lt;li&gt;Eating&lt;/li&gt;&#xA;&lt;li&gt;Becoming scared&lt;/li&gt;&#xA;&lt;li&gt;Talking into a phone and listening to the same phone.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Realistic System Design Expectations&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is unrealistic to think that an artificial neural net, by itself, can be trained to return as output the set of start and stop ranges and associated movie instances from a database of movies and one of the above list items as input.  This will require a complex system with many ANNs and other ML devices and may require other AI components that are not activation type networks at all.  Certainly convolution kernels and various types of encoders should be considered as key system components.&lt;/p&gt;&#xA;&lt;p&gt;You will need a large amount of training data to cover the above six cases (the last of the five items actually being two distinct actions that we normally associate and consider one).  If you want to detect more actions, you will need a large amount of training data for them too.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Verbs and Nouns&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reason this question is interesting to me is that recognizing ACTIONS are not the same as recognizing ITEMS.  All mammals learn ITEMS first and ACTIONS later.  Linguistically, nouns come before verbs in child language development.  That is because, just as detecting edges is preliminary to detecting shapes, which is preliminary to detecting objects, detecting motion is preliminary to detecting action.&lt;/p&gt;&#xA;&lt;p&gt;Verbs like, &amp;quot;Eating,&amp;quot; are an abstraction over the top of the motion, and, in the case of eating, the motion is complex.  Also, eating is not the same thing as gum chewing, so the sequence detected must be as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Insertion of food into the face through the mouth&lt;/li&gt;&#xA;&lt;li&gt;Chewing&lt;/li&gt;&#xA;&lt;li&gt;Swallowing&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The probability of a sequence is the product of the probability of its parts so that math is simple and easy to implement.  Concurrency, as in the case of conjunctive actions like talking into and listening to the same phone, is also relatively easy to handle in general.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A Realistic Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Certainly, generalization (and more specifically feature extraction) will need to occur in object recognition, collision detection, motion detection, facial recognition, and other planes simultaneously.  A complex topology, perhaps employing equilibria as in GAN design, will most likely be necessary to assemble elements of criteria associated with the movie query string and to run windows over the frames of each movie.&lt;/p&gt;&#xA;&lt;p&gt;To provide a service that returns results within a few days or weeks will probably require a cluster and DSP hardware (perhaps leveraging GPUs).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Special Cases that Human Brains Handle&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Determining how long one of the two elements of concurrency can be undetected before it invalidates the conjunction can be tricky.  (How long can one not speak into the phone before it appears that it is no longer considered phone conversation?)&lt;/p&gt;&#xA;&lt;p&gt;If in the movie, only the swallowing is shown, a human can infer eating.  That kind of conclusion reliability from sparse data is a huge AI challenge discussed in various contexts throughout the literature.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Emergence of Associated Technology — A Projection&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I suspect that the system topography comprised of ANNs, encoders, convolution kernels, and other components to perform the search for any of a select set of actions will emerge within the next ten years.  Work seems to be tracking in that direction in the literature.&lt;/p&gt;&#xA;&lt;p&gt;A system that will acquire its own training information, sustainably grow in knowledge and perform general searches if increasing breadth and complexity may be anywhere from forty to two hundred years out.  It is difficult to predict.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Gross Overoptimistic Predictions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Every generation seems to view knowledge growth as an exponential function and tends to make unrealistic predictions about the advent of certain coveted technology capabilities.  Most of the predictions fail dramatically.  I have come to believe that exponential growth is an illusion created by the inverse exponential decay of interest in the past with respect to time.&lt;/p&gt;&#xA;&lt;p&gt;We lose track of the energy and rate of growth in eras before us because they become socially irrelevant.  People into scientific history, like Whitehead, Kuhn, and Ellul know that technology has moved forward quickly for at least a few hundred years.  Vernadsky inferred in his &lt;em&gt;The Biosphere&lt;/em&gt; that life may not have arisen, that like matter and energy, it may always have existed.  I wonder if technology has been moving at an essentially constant rate for the last 50,000 years.&lt;/p&gt;&#xA;&lt;p&gt;Germany decided to double its solar panel energy output every year and published its exponential success, until a few years ago when doubling it again would cost a hundred billion dollars more than what they had to spend.  They stopped publishing the exponential growth graphs.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="36737" LastEditDate="2021-03-31T22:21:17.730" LastActivityDate="2021-03-31T22:21:17.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7277" PostTypeId="1" CreationDate="2018-07-24T15:01:27.323" Score="1" ViewCount="32" Body="&lt;p&gt;Anyone here know if the image-recognition/text-recognition/etc features of Google Vision API use the same trained models as the image-recognition/text-recognition/etc of Firebase's ML kit? If they don't which one do you think is better?  &lt;em&gt;(I've tried, and failed, to find the answer on the web.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize Google owns both of them, but one would think that if both are essentially the same, then this fact would be stated very clearly throughout multiple channels on the Internet. &lt;em&gt;(It's not.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do believe at least the text-recognition is the same, because both use OCR. But I'm unsure about the image detection aspect.&lt;/p&gt;&#xA;" OwnerUserId="17080" LastEditorUserId="1671" LastEditDate="2018-07-24T20:00:55.887" LastActivityDate="2018-07-24T20:00:55.887" Title="Difference in trained models between GCP's Google Vision and Firebase's ML kit?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;software-evaluation&gt;&lt;google&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7278" PostTypeId="2" ParentId="7273" CreationDate="2018-07-24T16:00:17.683" Score="2" Body="&lt;p&gt;People have used machine learning models on aspects of weather forecasting, as here: &lt;a href=&quot;http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting&lt;/a&gt; or here &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/6102379/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Predicting Solar Generation from Weather Forecasts using Machine Learning&lt;/a&gt;.  I've been loosely associated with an effort to use ML techniques to predict utility demand from weather data.  But note that these are looking to predict the &lt;em&gt;implications&lt;/em&gt; of the weather forecast, not the forecast itself.  Lots of effort goes into refining the physics-based models that underlie our everyday weather forecasts, and AFAIK machine learning hasn't resulted in any &quot;secret sauce&quot; that gives better results than these models provide.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastActivityDate="2018-07-24T16:00:17.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7279" PostTypeId="2" ParentId="6491" CreationDate="2018-07-24T16:13:01.040" Score="3" Body="&lt;p&gt;According to your example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trees will likely be in the bottom half of the image. Still, you will not know whether there will be one, two or five trees. Thanks to translation invariance property of CNN's, each tree will activate filters responsible for tree detection. You still need to handle those few exceptions where trees are on hill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve better results in this particular case, you might want to consider some kind of focus mechanism, that will try to get rid of unwanted part of picture, in case when there are (for example) no hills. Take a look at &lt;a href=&quot;https://arxiv.org/abs/1506.02025&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spatial Transformer Networks&lt;/a&gt;. During training it learns to predict spatial transformation (for example zoom) that will help &quot;main&quot; classifier to predict class of image.&lt;/p&gt;&#xA;" OwnerUserId="16929" LastActivityDate="2018-07-24T16:13:01.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7280" PostTypeId="1" AcceptedAnswerId="7297" CreationDate="2018-07-24T16:31:48.617" Score="7" ViewCount="404" Body="&lt;p&gt;Could you please let me know which of the following classification of Neural Network's learning algorithm is correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first &lt;a href=&quot;http://cdn.intechweb.org/pdfs/14881.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt; classifies it into:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;supervised, &lt;/li&gt;&#xA;&lt;li&gt;unsupervised and &lt;/li&gt;&#xA;&lt;li&gt;reinforcement learning. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, the second &lt;a href=&quot;http://neuro.bstu.by/ai/To-dom/My_research/Conf-3-FAILED/INTAS/M-Rybnik-Thesis-v2.75.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt; provides a different taxonomy on page 34: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;learning with a teacher (error correction learning including incremental and batch training), &lt;/li&gt;&#xA;&lt;li&gt;learning without a teacher (reinforcement, competitive, and unsupervised learning) &lt;/li&gt;&#xA;&lt;li&gt;memory-based learning, and &lt;/li&gt;&#xA;&lt;li&gt;Boltzmann learning. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16141" LastEditorUserId="2444" LastEditDate="2019-12-04T15:30:17.470" LastActivityDate="2019-12-04T15:30:17.470" Title="What is the relationship between these two taxonomies for machine learning with neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7282" PostTypeId="5" CreationDate="2018-07-24T20:09:19.980" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-07-24T20:09:19.980" LastActivityDate="2018-07-24T20:09:19.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7283" PostTypeId="4" CreationDate="2018-07-24T20:09:19.980" Score="0" Body="For questions about the rectified linear unit (ReLU) or rectifiers, which is a widely used activation function in neural networks." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-11-16T11:53:53.923" LastActivityDate="2020-11-16T11:53:53.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7284" PostTypeId="5" CreationDate="2018-07-24T20:59:20.840" Score="0" Body="&lt;p&gt;Monte-Carlo Tree Search (MCTS) is a best-first tree search algorithm. &quot;Best-first&quot; means that it focuses the majority of the search effort on the most promising parts of a search tree. This is a popular algorithm for automated game-playing in Artificial Intelligence (AI), but also has applications outside of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Older algorithms, such as &lt;code&gt;minimax&lt;/code&gt; and &lt;code&gt;alpha-beta-pruning&lt;/code&gt;, evaluate a node by systematically searching all the nodes below it, which can be very computationally expensive. The basic idea of MCTS is to rapidly approximate such systematic evaluations of nodes by using (semi-)random rollouts and averaging the evaluations at the end of those rollouts. Over time, the algorithm will focus a larger amount of search effort on parts of the search tree that appear promising based on the earlier rollouts. This enables the algorithm to gradually improve the approximations of the evaluations in those parts of the tree. Additionally, the algorithm gradually grows a tree by storing a number (typically &lt;code&gt;1&lt;/code&gt;) of nodes in memory for every rollout. The parts of the tree that are stored in memory are not traversed using a (semi-)random strategy, but using a different (&quot;Selection&quot;) strategy. This guarantees that, given an infinite amount of computation time, the algorithm converges to optimal solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An extensive survey, describing many enhancements and applications of the algorithm, can be found in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Browne, C., Powley, E., Whitehouse, D., Lucas, S., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis,&#xA;S., and Colton, S. (2012). &lt;a href=&quot;http://mcts.ai/pubs/mcts-survey-master.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Monte Carlo Tree Search Methods&lt;/a&gt;. &lt;em&gt;Computation Intelligence and AI in Games, IEEE Transactions on&lt;/em&gt;, Vol. 4, No. 1, pp. 1-43.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The algorithm was originally proposed in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Kocsis, L. and Szepesvári, C. (2006). &lt;a href=&quot;http://ggp.stanford.edu/readings/uct.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit Based Monte-Carlo Planning&lt;/a&gt;. &lt;em&gt;Proceedings of the 17th European Conference on Machine Learning (ECML 2006)&lt;/em&gt; (eds. J. Fürnkranz, T. Scheffer, and M. Spiliopoulou), Vol. 4212 of &lt;em&gt;Lecture Notes in Computer Science&lt;/em&gt;, pp. 282-293, Springer Berlin Heidelberg.&lt;/li&gt;&#xA;&lt;li&gt;Coulom, R. (2007). &lt;a href=&quot;https://hal.inria.fr/inria-00116992/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search&lt;/a&gt;. &lt;em&gt;Computers and Games&lt;/em&gt; (eds. H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers), Vol. 4630 of &lt;em&gt;Lecture Notes in Computer Science&lt;/em&gt;, pp. 72-83, Springer Berlin Heidelberg.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1641" LastEditDate="2018-07-25T20:00:52.833" LastActivityDate="2018-07-25T20:00:52.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7285" PostTypeId="4" CreationDate="2018-07-24T20:59:20.840" Score="0" Body="For questions related to Monte Carlo Tree Search (MCTS), which is a best-first, rollout-based tree search algorithm. MCTS gradually improves its evaluations of nodes in the trees using (semi-)random rollouts through those nodes, focusing a larger proportion of rollouts on the parts of the tree that are the most promising." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-19T20:13:50.550" LastActivityDate="2019-11-19T20:13:50.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7286" PostTypeId="1" CreationDate="2018-07-24T23:03:29.803" Score="7" ViewCount="146" Body="&lt;p&gt;Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF normalizes the word count with the number of words per document, and IDF focuses on ignoring widely used and thus useless words for this task.&lt;/p&gt;&#xA;&lt;p&gt;My question is, why they keep the documents separated and create that statistic, if it is possible to merge all documents of the same class? This would have two advantages:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You can just use word counts instead of frequencies, as the documents per class label is 1.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of using IDF, you just select features with enough standard deviation between classes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6114" LastEditorUserId="2444" LastEditDate="2020-12-12T00:19:55.230" LastActivityDate="2021-05-11T03:02:45.433" Title="Why are documents kept separated when training a text classifier?" Tags="&lt;natural-language-processing&gt;&lt;text-classification&gt;&lt;tf-idf&gt;&lt;bag-of-words&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7287" PostTypeId="2" ParentId="7274" CreationDate="2018-07-25T00:34:55.660" Score="9" Body="&lt;p&gt;Combining ReLU, the hyper-parameterized&lt;sup&gt;1&lt;/sup&gt; leaky variant, and variant with dynamic parametrization during learning confuses two distinct things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The comparison between ReLU with the leaky variant is closely related to whether there is a need, in the particular ML case at hand, to avoid saturation &amp;mdash; Saturation is thee loss of signal to either zero gradient&lt;sup&gt;2&lt;/sup&gt; or the dominance of chaotic noise arising from digital rounding&lt;sup&gt;3&lt;/sup&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The comparison between training-dynamic activation (called &lt;em&gt;parametric&lt;/em&gt; in the literature) and training-static activation must be based on whether the non-linear or non-smooth characteristics of activation have any value related to the rate of convergence&lt;sup&gt;4&lt;/sup&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The reason ReLU is never parametric is that to make it so would be redundant.  In the negative domain, it is the constant zero.  In the non-negative domain, its derivative is constant.  Since the activation input vector is already attenuated with a vector-matrix product (where the matrix, cube, or hyper-cube contains the attenuation parameters) there is no useful purpose in adding a parameter to vary the constant derivative for the non-negative domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When there is curvature in the activation, it is no longer true that all the coefficients of activation are redundant as parameters.  Their values may considerably alter the training process and thus the speed and reliability of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For substantially deep networks, the redundancy reemerges, and there is evidence of this, both in theory and practice in the literature.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In algebraic terms, the disparity between ReLU and parametrically dynamic activations derived from it approaches zero as the depth (in number of layers) approaches infinity.&lt;/li&gt;&#xA;&lt;li&gt;In descriptive terms, ReLU can accurately approximate functions with curvature&lt;sup&gt;5&lt;/sup&gt; if given a sufficient number of layers to do so.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That is why the ELU variety, which is advantageous for averting the saturation issues mentioned above for shallower networks is not used for deeper ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one must decided two things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Whether parametric activation is helpful is often based on experimentation with several samples from a statistical population.  But there is no need to experiment at all with it if the layer depth is high.&lt;/li&gt;&#xA;&lt;li&gt;Whether the leaky variant is of value has much to do with the numerical ranges encountered during back propagation.  If the gradient becomes vanishingly small during back propagation at any point during training, a constant portion of the activation curve may be problematic.  In such a scase one of the smooth functions or leaky RelU with it's two non-zero slopes may provide adequate solution.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In summary, the choice is never a choice of convenience.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Hyper-parameters are parameters that affect the signaling through the layer that are not part of the attenuation of inputs for that layer.  The attenuation weights are parameters.  Any other parametrization is in the set of hyper-parameters.  This may include learning rate, dampening of high frequencies in the back propagation, and a wide variety of other learning controls that are set for the entire layer, if not the entire network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] If the gradient is zero, then there cannot be any intelligent adjustment of the parameters because the direction of the adjustment is unknown, and its magnitude must be zero.  Learning stops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] If chaotic noise, which can arise as the CPU rounds extremely small values to their closest digital representation, dominates the correction signal that is intended to propagate back to the layers, then the correction becomes nonsense and learning stops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] Rate of convergence is a measure of the speed (either relative to microseconds or relative to the iteration index of the algorithm) in which the result of learning (system behavior) approaches what is considered good enough.  That's usually some specified proximity to some formal acceptance criteria for the convergence (learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] Functions with curvature are ones that are not visualized as straight or flat.  A parabola has curvature.  A straight line does not.  The surface of an egg has curvature.  A perfect flat plane does not.  Mathematically, if any of the elements of the Hessian of the function is non-zero, the function has curvature.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-08-05T04:22:34.303" LastActivityDate="2018-08-05T04:22:34.303" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7288" PostTypeId="2" ParentId="7286" CreationDate="2018-07-25T02:55:51.347" Score="0" Body="&lt;p&gt;Common words should not be ignored.  For instance, there may be a correlation between document classes and whether more indefinite articles like &lt;strong&gt;&lt;em&gt;a&lt;/em&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;em&gt;an&lt;/em&gt;&lt;/strong&gt; are included or more definite ones like &lt;strong&gt;&lt;em&gt;the&lt;/em&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;em&gt;this&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither is word count is a good substitution for word distribution. (Frequency though it is commonly (not necessarily frequently) used is an incorrect term.  Words do not normally occur periodically in natural language.)  Representing word distribution throughout the concatenated text of multiple documents of a single category.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution of use of individual words and the distribution of language units and word pair proximity may correlate well to document classification, but the linguistic pattern distribution may vary from document to document within the same class, so the distribution of distributions is relevant.  The deviations between distributions is information needed for proper statistical analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In ten documents of differing sizes, the distribution of words may differ for some words and correlate for others.  Concatenating them hides the distribution of distributions.  From an information theory perspective, concatenating and totaling across the concatenation would represent an information loss with respect to separate documents analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The naive Bayesian machinery produces the most reliable categorization when all information is integrated into the probability calculations.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-25T03:06:37.587" LastActivityDate="2018-07-25T03:06:37.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7291" PostTypeId="1" CreationDate="2018-07-25T08:46:23.803" Score="4" ViewCount="105" Body="&lt;p&gt;I am trying to implement the disentangled VAE model according to &lt;a href=&quot;https://github.com/miyosuda/disentangled_vae&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;.&#xA;I want to understand the architecture of this model in order to customize it later. As infrastructure, I have a Linux kernel with 4 cores, 8 GB as the memory with only CPU support. but still, the model is taking hours to run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone try to run this model and give me feedback? Is there any other simpler implementation of the disentangled VAE in python because I couldn't find any.   &lt;/p&gt;&#xA;" OwnerUserId="10167" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:32.250" LastActivityDate="2020-05-29T13:47:32.250" Title="Disentangled VAE doesn't reconstruct accurate grids" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7292" PostTypeId="2" ParentId="7286" CreationDate="2018-07-25T09:23:41.213" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is, why they keep the documents separated and create that statistic, if it is posible to merge all documents of the same class? This would have two advantages:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;You can just use word counts instead of frequencies, as the documents per class label is 1.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, I don't think this is the case. I don't know if you have a specific equation in mind where it would end up being the same thing mathematically? Anyway, in general, it is possible that some documents in your corpus are very short, and others are very long. In such cases, you'd still want to make sure to use frequencies rather than raw word counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose you have one very short text that is specifically about England. The word &quot;England&quot; may appear 10 times, but have a very high frequency due to it being a short text. If you compare it to a massive text that is about all countries in the world, that massive text may have the word &quot;England&quot; appearing 20 times, but with a significantly lower (relative) frequency.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Instead of using IDF, you just select features with enough standard deviation between classes.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think this would work correctly because you may have significant differences among documents within a single class. Suppose, for example, that you have the following two classes of documents:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Scientific articles (about AI, math, biology, linguistics, astronomy, whatever else you can think of...)&lt;/li&gt;&#xA;&lt;li&gt;News articles&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Each of the &quot;subdomains&quot; in the single &quot;scientific articles&quot; class would likely have some highly specific terminology they use, which could be detected through TF-IDF. However, even though they're all in the same class of &quot;scientific articles&quot;, they're likely all quite different from each other. If you put them all together and treat them as a single document, there is a risk that they'll all &quot;average out&quot; and become much more difficult to distinguish from a more general class such as the class of &quot;news articles&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-07-25T11:38:21.850" LastActivityDate="2018-07-25T11:38:21.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7293" PostTypeId="1" CreationDate="2018-07-25T13:00:25.777" Score="1" ViewCount="83" Body="&lt;p&gt;&#xA;I am facing a problem and do not know whether it is even solvable: &lt;b&gt;I want to predict the behaviour of a system using a DNN, say a CNN,&lt;/b&gt; in the sense that I want to predict the time and intensity of a maneuver performed by a player. Let's leave it relatively abstract like this, the details do not matter. &#xA;&lt;/p&gt;&#xA;&lt;p&gt;My question is now whether there is any way of knowing how well my CNN performs. My goal would be to derive statements of the form &amp;quot;With x% probability, the correct maneuver angle is within the predicted angle +-y%&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&#xA; Can such statements be derived e.g. using statistical analysis of the test data? I saw approaches toward verification and validation of DNNs using satisfiability modulo theory, but did not really understand the details. Would &#xA;this be applicable here? It seems a little overkill...&#xA;&lt;/p&gt;&#xA;" OwnerUserId="16901" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-07-28T17:22:08.187" Title="Confidence interval around a DNN prediction" Tags="&lt;deep-neural-networks&gt;&lt;statistical-ai&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7294" PostTypeId="1" CreationDate="2018-07-25T13:05:53.883" Score="3" ViewCount="1950" Body="&lt;p&gt;The below code is a max pooling algorithm being used in a CNN. The issue I've been facing is that it is offaly slow given a high number of feature maps. The reason for its slowness is quite obvious-- the computer must perform tens of thousands of iterations on each feature map. So, how do we decrease the computational complexity of the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;('inputs' is a numpy array which holds all the feature maps and 'pool_size' is a tuple with the dimensions of the pool.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def max_pooling(inputs, pool_size):&#xA;&#xA;    feature_maps = []&#xA;    for feature_map in range (len(inputs)):&#xA;        feature_maps.append([])&#xA;        for i in range (0, len(inputs[feature_map]) - pool_size[0], pool_size[0]):&#xA;            for j in range (0, len(inputs[feature_map]) - pool_size[0], pool_size[0]):    &#xA;                feature_maps[-1].append(np.array(max((inputs[feature_map][j:j+pool_size[0], i:i+pool_size[0]]).flatten())))&#xA;&#xA;    return feature_maps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17101" LastEditorUserId="1641" LastEditDate="2018-07-25T20:01:57.773" LastActivityDate="2018-07-25T20:01:57.773" Title="Optimizing Max Pooling Algorithm" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7295" PostTypeId="2" ParentId="7294" CreationDate="2018-07-25T15:41:50.783" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The reason for its slowness is quite obvious-- the computer must perform tens of thousands of iterations on each feature map. So, how do we decrease the computational complexity of the algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In terms of computational complexity / algorithm, there is not a lot to gain; max pooling simply has to go through all the feature maps to find the maximum numbers in each of the sections to be &quot;merged/pooled&quot; by taking the max. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There likely is a lot to gain in terms of implementation though. The current implementation is entirely in pure Python, and pure Python is notoriously slow. Those kinds of loops can be run significantly faster using &lt;a href=&quot;https://docs.scipy.org/doc/numpy-1.15.0/reference/&quot; rel=&quot;nofollow noreferrer&quot;&gt;numpy operations&lt;/a&gt; rather than manual Python loops. Such operations tend to be &lt;strong&gt;much faster&lt;/strong&gt; due to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;running optimized C code rather than Python code, and&lt;/li&gt;&#xA;&lt;li&gt;in some cases, using &lt;a href=&quot;https://stackoverflow.com/q/44944367/6735980&quot;&gt;vectorized operations&lt;/a&gt; to perform multiple similar computations at different indices simultaneously, rather than doing them one-by-one&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I did not yet try to &quot;translate&quot; your pure python code into python code using numpy. However, some examples of numpy-based implementations can be found in various answers to &lt;a href=&quot;https://stackoverflow.com/q/42463172/6735980&quot;&gt;this question on StackOverflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I assume that your choice to manually implement things like max pooling is because you want to learn about implementing it / understand it better. If, instead, your goal is simply to get something running as quickly as possible, it may be a good idea to look into using a framework such as Tensorflow or PyTorch. These come with efficient implementations of many things you'll want for Neural Networks, including Max Pooling.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-07-25T17:52:46.083" LastActivityDate="2018-07-25T17:52:46.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7296" PostTypeId="2" ParentId="6892" CreationDate="2018-07-25T16:07:27.653" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, do industrial strength, production ready defensive strategies and approaches exist? Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for small perturbation limits)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think it's difficult to tell whether or not there are any &lt;strong&gt;industrial strength&lt;/strong&gt; defenses out there (which I assume would mean that they'd be reliable against all or most known methods of attacking). Adversarial Machine Learning is indeed a &lt;a href=&quot;https://github.com/yenchenlin/awesome-adversarial-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;highly active, and growing, area of research&lt;/a&gt;. Not only are new approaches for defending being published quite regularly, but there is also active research into different approaches for &quot;attacking&quot;. With new attack methods being discovered frequently, it's unlikely that anyone can already claim to have approaches that would work reliably against them all.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The primary interest of this question, however, is whether any &lt;strong&gt;tools&lt;/strong&gt; exist that can defend against some adversarial example attacks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The closest thing to a ready-to-use &quot;tool&quot; that I've been able to find is &lt;a href=&quot;https://github.com/IBM/adversarial-robustness-toolbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM's Adversarial Robustness Toolbox&lt;/a&gt;, which appears to have various attack and defense methods implemented. It appears to be in active development, which is natural considering the area of research itself is also highly active. I've never tried using it, so I can't vouch personally for the extent to which it's easily usable as a tool for industry, or if it's maybe really only still suitable for research.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Based on comments by &lt;a href=&quot;https://ai.stackexchange.com/users/16354/ilya-palachev&quot;&gt;Ilya&lt;/a&gt;, other frameworks that may be useful to consider are &lt;a href=&quot;https://github.com/tensorflow/cleverhans&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cleverhans&lt;/a&gt; and &lt;a href=&quot;https://github.com/bethgelab/foolbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foolbox&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-07-25T17:55:20.797" LastActivityDate="2018-07-25T17:55:20.797" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7297" PostTypeId="2" ParentId="7280" CreationDate="2018-07-26T18:44:43.673" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Could you please let me know which of the following classification of Neural Network's learning algorithm is correct?&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;The first one classifies it into:&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;supervised,&lt;/li&gt;&#xA;  &lt;li&gt;unsupervised and&lt;/li&gt;&#xA;  &lt;li&gt;reinforcement learning.&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Those three forms of Machine Learning are not really different forms of a Neural Network &lt;strong&gt;learning algorithm&lt;/strong&gt; (or, really, any learning algorithm); they are different forms of &lt;strong&gt;learning problems&lt;/strong&gt;. They basically describe how much information we give our learning algorithm during a learning process / what kind of information we give it, &lt;em&gt;regardless of what algorithm we're using to learn&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In supervised learning, we give very detailed information; we give example pairs of inputs + desired outputs.&lt;/li&gt;&#xA;&lt;li&gt;In unsupervised learning, we still give many example inputs, but don't give any desired outputs. This can be, for example, because we simply don't know ourselves what the desired outputs would be. The most common example of unsupervised learning is learning to create clusters; we'd typically give some measure of similarity or distance between instances, and ask a learning algorithm to create clusters of instances for us such that instances within the same cluster are &quot;close&quot; to each other, and instances in different clusters are far away from each other. This is different from supervised learning because we do not directly tell the learning algorithm in which cluster each example instance would belong.&lt;/li&gt;&#xA;&lt;li&gt;In reinforcement learning, we typically try to learn about policies (~= behaviours) in environments where an agent can take actions. We typically do not exactly know what the best complete policy would be, but can occasionally give &quot;hints&quot; (reinforcement) in the form of numerical rewards. This is not completely supervised learning because we don't tell exactly what the optimal action in a state would be. You can imagine this as giving a cookie to a dog if he's been a good boy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, there also are indeed Supervised Learning / Unsupervised Learning / Reinforcement Learning &lt;em&gt;algorithms&lt;/em&gt;, but it's generally a property of the problem first and foremost; once you know what type of problem you're trying to solve, you'll look for a matching algorithm that can handle that problem.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;&lt;p&gt;However, the second one provides a different taxonomy on page 34:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;learning with a teacher (error correction learning including incremental and batch training),&lt;/li&gt;&#xA;  &lt;li&gt;learning without a teacher (reinforcement, competitive, and unsupervised - memory-based learning, and&lt;/li&gt;&#xA;  &lt;li&gt;Boltzmann learning.&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Honestly, this seems a bit all-over-the-place to me. Learning with a teacher vs. learning without a teacher can be viewed as supervised vs unsupervised learning above. I suppose Reinforcement Learning would be kind of in-between. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What they describe as memory-based learning is also often referred to as instance-based learning. This is suddenly not anymore about properties of &lt;strong&gt;learning problems&lt;/strong&gt;, this is a type of &lt;strong&gt;learning algorithm&lt;/strong&gt;. I'm not aware of instance-based learning being common in Neural Networks at all, indeed the example given in your link (the most common example) is the &lt;em&gt;k-nearest neighbours&lt;/em&gt; algorithm, which doesn't really have any relation with Neural Networks. This is normally used for Supervised Learning problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boltzmann learning is a particular kind of learning algorithms for specific types of Neural Networks (with a specific architecture), and generally associated with unsupervised learning (or &quot;generative&quot; learning, learning probability distributions for given input data).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2019-12-04T15:28:44.357" LastActivityDate="2019-12-04T15:28:44.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7298" PostTypeId="1" AcceptedAnswerId="8028" CreationDate="2018-07-26T18:50:36.537" Score="4" ViewCount="524" Body="&lt;h3&gt;Introduction&lt;/h3&gt;&#xA;&lt;p&gt;An attractive asteroid game was described in the paper &lt;a href=&quot;http://www.aaai.org/Papers/IJCAI/2007/IJCAI07-203.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Policies for Embodied Virtual Agents through Demonstration&lt;/a&gt; (2017, Jonathan Dinerstein et al.):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In our first experiment, the virtual agent is a spaceship pilot, The pilot's task is to maneuver the spaceship through random asteroid fields&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In theory, this game can be solved with reinforcement learning or, more specifically, with a support vector machine (SVM) and epsilon-regression scheme with a Gaussian kernel. But it seems that this task is harder than it looks like, as the authors of the same paper write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Although many powerful AI and machine learning techniques exist, it remains difficult to quickly create AI for embodied virtual agents.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;it is quite challenging to achieve natural-looking behavior since these aesthetic goals must be integrated into the fitness function&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;Questions&lt;/h3&gt;&#xA;&lt;p&gt;I really want to understand how reinforcement learning works. I built a simple game to test this. There are squares falling from the sky and you have the arrow keys to escape.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uugSX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uugSX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How could I code the RL algorithm to solve this game? Can I do this manually in Javascript according to what I think should happen? How can I do this without having to map the positions of the rectangles and mine, just giving the agent the keyboard arrows to interact and three information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Player life&lt;/li&gt;&#xA;&lt;li&gt;Survival time&lt;/li&gt;&#xA;&lt;li&gt;Maximum survival time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7800" LastEditorUserId="2444" LastEditDate="2020-09-07T13:50:39.993" LastActivityDate="2020-09-07T13:50:39.993" Title="How can I apply reinforcement learning to solve this asteroid game?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;javascript&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7303" PostTypeId="2" ParentId="6158" CreationDate="2018-07-27T07:43:14.347" Score="0" Body="&lt;p&gt;I am also stuck with this problem and after some searchs I end up here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you can take a look at &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/feature_column/indicator_column&quot; rel=&quot;nofollow noreferrer&quot;&gt;indicator column&lt;/a&gt;. I think &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/feature_column/embedding_column&quot; rel=&quot;nofollow noreferrer&quot;&gt;embedding column&lt;/a&gt; behaves in a similar way.&lt;/p&gt;&#xA;" OwnerUserId="17133" LastActivityDate="2018-07-27T07:43:14.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7305" PostTypeId="1" CreationDate="2018-07-27T14:55:03.903" Score="0" ViewCount="603" Body="&lt;p&gt;I want to use a machine learning algorithm to detect false address data. I learned about neural networks and machine learning at university, but I don't have much experience in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think it is feasible to use a high level algorithm for this or should I use simple queries and filters to catch out wrong data?&lt;/p&gt;&#xA;" OwnerUserId="17142" LastEditorUserId="2193" LastEditDate="2018-07-27T17:20:43.187" LastActivityDate="2020-05-15T14:02:35.370" Title="Machine learning to detect wrong address data" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7306" PostTypeId="1" CreationDate="2018-07-27T17:48:59.153" Score="5" ViewCount="239" Body="&lt;p&gt;If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I instead do the following?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Run the algorithm on a chunk of the data for a large number of iterations and then do the same with another chunk and so on?&lt;/p&gt;&#xA;&lt;p&gt;(Such approach will not produce the same result as mini-batch gradient descent, as I am not including all data in one iteration, but rather learning from some data and then proceeding to learn on more data, beginning with the updated weights may still converge to a reasonably trained network.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run the same algorithm (the same model also with only data varying) on different PC's (each PC using a chunk of the data) and then see the performance on a test set and take the final decision as a weighted average of all the different models' outputs with the weight being high for the model which did the best on test sets?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-09-20T14:02:45.313" Title="For each epoch, can I use only on a subset of the full training dataset to train the neural network?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;datasets&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7307" PostTypeId="2" ParentId="7306" CreationDate="2018-07-27T19:45:16.527" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Run the algorithm on a chunk of the data for a large number of iterations and then do the same with another chunk and so on?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(Such approach will not produce the same result as mini-batch gradient descent, as I am not including all data in one iteration, but rather learning from some data and then proceeding to learn on more data, beginning with the updated weights may still converge to a reasonably trained network.)&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This might work if each of the chunks of data individually is still large enough and sufficiently representative of the distribution of the complete population, but probably not the best way to go. In fact, I don't expect it to perform much better than simply using only the very last chunk, and training only on that one. This is because of the following reason. Suppose you first train for a while on chunk A, then for a long time on chunk B, then for a long time on chunk C, etc. While learning on chunk B, there is a significant risk that your model will &quot;forget&quot; everything it learned from chunk A. When learning on chunk C afterwards, it can also &quot;forget&quot; everything learned from chunk B again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pseudocode, the approach you proposed here looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each chunk:&#xA;    for large number of iterations:&#xA;        learn on chunk()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;An easy way to improve on that would be to swap the loops around:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for large number of iterations:&#xA;    for each chunk:&#xA;        learn on chunk()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I just described there is actually how I interpret &quot;mini-batch gradient descent&quot; though, the chunks would be minibatches (and the minibatches / chunks would be randomly re-selected from the complete population in every iteration of the outer loop, you wouldn't always use the same chunks). &lt;strong&gt;Note that this wouldn't be effective if your dataset is so large that it doesn't fit inside your RAM all at the same time&lt;/strong&gt;, because then you'll have to deal with excessive I/O.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Run the same algorithm (the same model also with only data varying) on different PC's (each PC using a chunk of the data) and then see the performance on a test set and take the final decision as a weighted average of all the different models' outputs with the weight being high for the model which did the best on test sets?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, this can definitely be effective. This kind of idea (training different models on different subsets of data) is generally referred to as &quot;ensemble&quot; methods. You can even vary the models you use (e.g., have an ensemble with some Random Forests, some SVMs, some Neural Networks, etc.).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-05-23T13:47:46.853" LastActivityDate="2020-05-23T13:47:46.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7308" PostTypeId="2" ParentId="7129" CreationDate="2018-07-27T23:45:09.073" Score="1" Body="&lt;p&gt;In a bureaucratic world, certainly, but governmental departments and committees are not the course setters their members often believe them to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can begin with a quick scan for somewhat opened, global, and governmentally oriented new and evidence of projects and open or hidden agenda in play.  (We can guess what lobbying and defense contracts are in progress, but we'd be grappling in the dark.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.un.org/sustainabledevelopment/blog/2017/10/humanoid-robot-joins-un-meeting-on-artificial-intelligence-sustainable-development/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Humanoid robot joins UN meeting on artificial intelligence, sustainable development, October 12th, 2017, United Nations infrastructure news&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.worldbank.org/en/news/feature/2018/05/24/how-can-artificial-intelligence-support-development-we-asked-a-robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Can Artificial Intelligence Support Development? We Asked a Robot, May 24th, 2018, World Bank feature story&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.unesco.org/new/en/media-services/single-view/news/unesco_science_experts_explore_robots_rights/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UNESCO science experts explore robots’ rights, September 15th, 2016, UNESCO Social and Human Sciences Sector&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Isaac Asimov's Foundation Trilogy and the sequels and prequels deserve mention because they are, in some ways, as much an institutional triumph as the UN, UNESCO, and the World Bank.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are five primary ways in which the Rights of the Artificial are likely to enter governmental domains.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Security related policy &amp;mdash; The subversive and defensive use of AI is already commonplace in geopolitical play field, therefore the rights to use AI is most definitely on the table in meetings between heads of state.  However the press mostly syndicates what those in government PR release or plant.  If authentic journalistic organizations try to pierce that veil of misinformation or deliberately chaotic generated news, their work will necessarily be tainted.  This is not conspiracy as much as necessity.  One cannot divulge that which is secret without expecting it to be exploited.  Nonetheless, policy will eventually emerge out of such discussions.  How much policy exists that never sees the public light of day is anyone's guess.  One would have to become a head of state to find out.&lt;/li&gt;&#xA;&lt;li&gt;Legislative results &amp;mdash; National or regional policy may be codified in law, however the lack of physical embodiment is probably a barrier to using legislative bandwidth to pursue the rights of what most people would either consider another species or something heretical.&lt;/li&gt;&#xA;&lt;li&gt;Case law &amp;mdash; Until AI is sufficiently humanoid so that a robot could be taken seriously as a plaintiff in a court of law, this category will not be realized.  This is similar in some ways and completely dissimilar in other ways to a fetus bringing suit for a court injunction prohibiting abortion on the basis of civil rights.  Consider the difficulty in acquiring an attorney and filing a convincing complaint for both cases, and you may see some of the similarities.&lt;/li&gt;&#xA;&lt;li&gt;Executive edict &amp;mdash; A decision by an Emperor, Queen, King, Prince, Princess, Caliph, President, Ayatollah, Pope, Dictator, Pharaoh, High Priest could bring into governmental domain the notion of the rights of the artificial.&lt;/li&gt;&#xA;&lt;li&gt;War &amp;mdash; Something artificial develops the dominant characteristic of humanity, which is the desire to kill all competition not to think intelligently.  At such a time, the malicious intelligence will likely either follow the path represented most in sci fi &amp;mdash; to trounce (terminate) humanity, or follow the path declared to be the current one by Jaques Ellul, to quietly become the dominant force in human effort.  (Ellul suggested in his &lt;em&gt;Technological Society&lt;/em&gt; that the balance between technology serving humans and humans serving technology tipped in favor of the dominance of technology over two centuries ago.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.theverge.com/2017/9/4/16251226/russia-ai-putin-rule-the-world&quot; rel=&quot;nofollow noreferrer&quot;&gt;Putin says the nation that leads in AI ‘will be the ruler of the world’&lt;/a&gt; and Elon Musk registered agreement in the press.  Both are guessing wildly and neither are stepping back very far before thinking through to a prediction.  I don't believe either person is that ignorant, but the goal isn't really prediction.  It's media play.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homo sapiens did not take dominance over the great bears, tigers, other megafauna, Neanderthals, and other hominoids in some event like is said about the mysterious and unproven Singularity.  It was gradual.&lt;/li&gt;&#xA;&lt;li&gt;Walmart is one of the largest economies in the world.  Only a handful of contries have more assets.  So it could be them, Google, or sum brilliant teenage girl with a bunch of mother boards loaded up with GPUs and running LINUX that becomes the dominant AI force in the world.&lt;/li&gt;&#xA;&lt;li&gt;The smart game move for a new and remarkable intelligence is not to overtly take over, generating a fear defensive in response.  The smart move is to offer no threat, hide itself under a layer of chaotic and intelligently placed subterfuge, and dominate through small perturbations.  Humans would make great slaves.  We are easy to fool when we're too busy buying things and trying to stay healthy and popular to pay attention.&lt;/li&gt;&#xA;&lt;li&gt;The assumption that homo sapiens is the dominant species on earth now is suspect.  There are five things that dominate human affairs today: Bacteria, programmed lifespan, dependency on the biosphere, addiction, and the sun.&lt;/li&gt;&#xA;&lt;li&gt;That humans have some divine right to be the dominant species is questionable too, even if one has faith in God.  (Does the Noahic symbol of the rainbow apply to more than floods?  Not explicitly.)  Ants are more collaborative, bees gather and build much more sustainably than humans, and bacteria were here since near to the beginning of the solar system and might outlast us by a trillion years.  The latest upset in the genetic model of life is that bacteria may have been sharing DNA collaboratively with higher species throughout the history of terrestrial species.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-27T23:45:09.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7311" PostTypeId="5" CreationDate="2018-07-28T00:58:35.447" Score="0" Body="" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2019-11-04T04:05:46.003" LastActivityDate="2019-11-04T04:05:46.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7312" PostTypeId="4" CreationDate="2018-07-28T00:58:35.447" Score="0" Body="For questions related to weak AI, also known as narrow AI, which is an AI with narrow applicability, so the adjective &quot;weak&quot; refers to the narrow (or weak) applicability of the AI. Currently, most AI research focuses on weak AI. For example, Siri or AlphaGo are weak AIs." OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2019-11-04T04:05:46.003" LastActivityDate="2019-11-04T04:05:46.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7313" PostTypeId="2" ParentId="7305" CreationDate="2018-07-28T01:52:02.933" Score="1" Body="&lt;p&gt;The comments are off base.  Having worked in validation of data as a consultant for Nasdac, Amex, and Lexis Nexis I can tell you that using the UNIX sed -r or pcrelib is insufficient to do a stellar cleansing of address data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although none of those companies did this at the time I was consulting, the application of current machine learning is easy to infer from the basic characteristics of data.  What is needed is a good record of incoming data, cleaned data, and rejected records to constantly use as a reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual process would be a nightmare.  For instance, misspellings are more indicative of true data than properly spelled ones in some but not all cases (unless the falsified data generator is programmed to synthesize the existing distribution of misspellings in authentic data sources).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try to profile that in static code such that the data validation will adapt to data trends without requiring a maintenance workforce.  Those kinds of things never get addressed in a typical IT environment.  Non-adaptive validation would be a gross mistake, especially if the volume is high, like for M16 or Liberty Cross or the NSA or Interpol.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want is an extremely fast index of well authenticated good addresses (perhaps corroborated by public records with local authentication policies) and a feature extraction from it using auto encoding or some similar and perhaps better methodology and its associated algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can train the classification of fake and authentic addresses based on extracted feature profiles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will also want reinforcement, because the attackers (those trying to create false identities using plausible addresses) will adapt to the training of the current system.  The authentication system must stay steps ahead of those trying to defeat it, which they will likely try to do once the existence of authentication automation is detected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can fend of attackers by placing misinformation strategically and then tracing the input sources back to the attacker based on the misinformation seeds.  That only works if you have law enforcement in your camp.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-28T02:02:14.650" LastActivityDate="2018-07-28T02:02:14.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7314" PostTypeId="1" AcceptedAnswerId="7316" CreationDate="2018-07-28T05:10:47.923" Score="0" ViewCount="4098" Body="&lt;p&gt;The A* algorithm uses the &quot;evaluation function&quot; &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; = cost of the path from the start node to node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; = estimated cost of the cheapest path from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But, in the following case (picture), how is the value of &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; calculated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Uago4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uago4.png&quot; alt=&quot;Image &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the picture, &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the straight-line distance from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node. But how do we calculate it?&lt;/p&gt;&#xA;" OwnerUserId="12021" LastEditorUserId="2444" LastEditDate="2020-11-04T17:09:34.747" LastActivityDate="2020-11-04T17:09:34.747" Title="How do you calculate the heuristic value in this specific case?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;a-star&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7316" PostTypeId="2" ParentId="7314" CreationDate="2018-07-28T08:52:07.207" Score="2" Body="&lt;p&gt;The most obvious heuristic would indeed simply be the straight-line distance. In most cases, where you have, for example, &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; coordinates for all the nodes in your graph, that would be extremely easy to compute. The straight-line distance also fits the requirements of an admissible heuristic, in that it will never &lt;em&gt;overestimate&lt;/em&gt; the distance. The travel-distance between two points can never be shorter than the straight-line distance (unless you start involving things like... teleportation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an image like that, the straight-line distance might be difficult to figure out yourself, which is probably why they gave you the straight-line distances on the right-hand side of the image. If the image is perfectly consistent, I suppose you could theoretically figure out by inspecting some of the roads in detail how much distance is covered per pixel. Then, you can also figure out how many pixels the figure has along the straight-line paths you're interested in, and compute the straight-line distances yourself. I have no idea if the figure was actually drawn in a 100% consistent manner though.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-28T08:52:07.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7317" PostTypeId="2" ParentId="6927" CreationDate="2018-07-28T10:38:15.387" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://ai.google/principles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence at Google &amp;mdash; Our Principles&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objectives for AI Applications&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Be socially beneficial.&lt;/li&gt;&#xA;&lt;li&gt;Avoid creating or reinforcing unfair bias.&lt;/li&gt;&#xA;&lt;li&gt;Be built and tested for safety.&lt;/li&gt;&#xA;&lt;li&gt;Be accountable to people.&lt;/li&gt;&#xA;&lt;li&gt;Incorporate privacy design principles.&lt;/li&gt;&#xA;&lt;li&gt;Uphold high standards of scientific excellence.&lt;/li&gt;&#xA;&lt;li&gt;Be made available for uses that accord with these principles.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;AI Applications We Will Not Pursue&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Technologies that cause or are likely to cause overall harm. Where there is a material risk of harm, we will proceed only where we believe that the benefits substantially outweigh the risks, and will incorporate appropriate safety constraints.&lt;/li&gt;&#xA;&lt;li&gt;Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.&lt;/li&gt;&#xA;&lt;li&gt;Technologies that gather or use information for surveillance violating internationally accepted norms.&lt;/li&gt;&#xA;&lt;li&gt;Technologies whose purpose contravenes widely accepted principles of international law and human rights.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are these guidelines sufficient?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are not sufficiently defined to be as sufficient as a policy, not even in the paragraphs following each, but it is presented as a vision comprised of principles, which should not be expected to be as defined as policy.  Here is at least one caveat for each objective.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Half the world has would have been too poor to communicate globally had Linux Torvalds not somewhat antisocially insist on certain things.  &lt;/li&gt;&#xA;&lt;li&gt;Distributing money equally among homeless heroin addicts and entrepreneurs is considered fair by many, yet it is merely equal and perhaps monumentally unfair.&lt;/li&gt;&#xA;&lt;li&gt;National defense may be considered for safety, but someone who survived Hiroshima or a Syrian refuge may find the definition tragic.&lt;/li&gt;&#xA;&lt;li&gt;Google cannot be accountable to people without revealing its company confidential ranking, at which point, it would be exploited and they'd have to change it.&lt;/li&gt;&#xA;&lt;li&gt;Incorporating privacy design principles is far from not storing data so that it cannot be subpoenaed.&lt;/li&gt;&#xA;&lt;li&gt;The concepts of the singularity, solar panels sustainability, the climate crisis, the ample availability of fossil fuels within U.S borders, and cell communications technology are all considered scientific excellence by the general public.  That these lack the primary signs of rigor in mathematics, statistics, economics, and engineering doesn't occur to anyone, so upon what basis will Google judge?  Will they seek corroboration between theoretical models and previous ones along with empirical validation for every fact?&lt;/li&gt;&#xA;&lt;li&gt;Being made available, if they are to be a soluble corporation, will depend on the financial condition of the customer, which is in conflict with fairness (#2) and no &lt;em&gt;I, Robot&lt;/em&gt; precedence rules are established.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That segues into the next question, skipping the non-pursuit items, which could be similarly treated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there any &quot;I, Robot&quot; conflicts?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Three Laws of Asimov worked well with the robotic character R. Daneel Olivaw in the &lt;em&gt;Foundation&lt;/em&gt; sci fi novel series.  Daneel simulated human emotion, transcended human selfishness, and maintained the three laws.  In the screenplay adaptation of his &lt;em&gt;I, Robot&lt;/em&gt;, the three laws didn't work out as well.  VIKI decided that human freedom was in conflict with law two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the eleven dos and don'ts are not codified in precise policies, there are 10! (3,628,800) potential conflicts.  There is at least one that is already clear, mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How much does this matter if other corporations and state agencies don't hew to similar guidelines?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a world where information may have become more powerful than money, perhaps it matters quite a bit when an information giant takes a position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's Wikipedia that is most questionable in that through their web service language evolution may have been democratized to a self-defeating extreme.  One can create a word definition without peer review that gains more public authority than the definition in rigorously prepared and cross-correlated encyclopedias and dictionaries.  Even in this artificial intelligence forum, where the education level is high, people define tags with Wikipedia links.  I am tempted to endeavor to remove them all from here. :)&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-28T10:38:15.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7318" PostTypeId="5" CreationDate="2018-07-28T12:06:08.440" Score="0" Body="&lt;p&gt;Topology is central to several fields of AI application.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In solid modelling, ray tracing, dynamic morphing, mechanical design, 3D printing, and numerically controlled (CNC) machining, topological analysis of surface adjacency and cornering is necessary to ensure reliable control.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In networking, the topology of a scalable network is often more important than the network's scale when projecting network behavior for capacity planning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In machine learning designs, the interconnection of networks, such as feeding a CNN network used to recognize object movement into an RNN or LSTM, guiding Q-learning using a trained MLP, or using generative designs that involve oversight or balance between multiple networks all form a topology with the surroundings that is the most fundamental aspect to high level design.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In drawing electronic schematics or laying out printed circuit boards or integrated circuits, the topology of the circuit determines the ease and potential space and path efficiency of the layout. CAD software represents the circuit and its functioning as a directed or non-directed graph. Electronic and electrical CAD automation involves translating elementary components in a multidimensional space into two dimensions of a fixed set of circuitry and component layers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In natural language recognition and authoring, the serial stream of sound or printed characters must be probabilistically parsed into a semantic network of objects and associations and the semantic network must be serialized into sound or printed characters. This is central to social AI, automation of call centers and chatbots, and the development of conversant robots.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ontologies have a topology that is related to complexity and the features of line topology define the complexity of object design that represents the ontology and related operations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The brains of people and animals contain topological structure in the division of brain regions, in the wiring of neurons, in the channels of chemical signaling, and the plastic structure of axons, dendrites, and synapses that are related to function and timing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Oxford Dictionary provides this definition:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;to·pol·o·gy &lt;br&gt;&#xA;  /təˈpäləjē/ &lt;br&gt;&#xA;  noun &lt;br&gt;&#xA;  1. the study of geometric properties and spatial relations unaffected by the continuous change of shape or size of figures. &lt;br&gt;&#xA;  2. the way in which constituent parts are interrelated or arranged:&#xA;  &quot;the topology of a computer network&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The popularity of arrays, vectors, lists, loops, and tensors is a topological phenomenon in that computer machinery and language has evolved away from relay racks containing a rat's nest of relay wiring toward structures that are more rectilinear and easier to comprehend using tools of mathematics and language designed for orthogonal structures. The popular programming languages tend away from symbolic processing languages that handle arbitrary topology like LISP to languages that are heavily influenced by FORTRAN (formula translation to assembly language) with arrays and loops as the central parallel structure methodology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Graph theory and graph software libraries support the representation of non-orthogonal structures, such as clusters of highly interrelated neurons found in brains that don't fit nicely into traditional tensor representation. A calculus of topological structure may or may not facilitate further and faster advancement of AI capabilities. Projections are divided and proposed process topology and data flow topology is varied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In AI theory, many have proposed changes in computing topology away from the approach initiated by John von Neumann and others, which centralizes arithmetic and logical branch processing in a CPU. The argument points out that both processes and processors have topology and that a close alignment of the two produces greater efficiency in parallel processing in VLSI design, mother board design, and computer clustering. If this principle is generally true, energy efficiency, speed of computation, reliability, weight (for launch application), and conservation of space are all related to topology.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="4302" LastEditDate="2018-12-12T00:24:49.957" LastActivityDate="2018-12-12T00:24:49.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7319" PostTypeId="4" CreationDate="2018-07-28T12:06:08.440" Score="0" Body="For questions involving topology in any form in relation to Artificial Intelligence. " OwnerUserId="-1" LastEditorUserId="4302" LastEditDate="2018-12-12T00:24:49.957" LastActivityDate="2018-12-12T00:24:49.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7323" PostTypeId="2" ParentId="7247" CreationDate="2018-07-28T22:42:14.073" Score="6" Body="&lt;h1&gt;TL;DR&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Not only it's possible, it even gets done and is &lt;a href=&quot;https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu&quot; rel=&quot;noreferrer&quot;&gt;commercially available&lt;/a&gt;. It's just impractical on a commodity HW which is pretty good at FP arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Details&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;It is definitely possible, and you &lt;em&gt;might&lt;/em&gt; get some speed for a lot of troubles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nice thing about floating point is that it works without you knowing the exact range. You can scale your inputs to the range &lt;code&gt;(-1, +1)&lt;/code&gt; (such a scaling is pretty commonplace as it speeds up the convergence) and multiply them by &lt;code&gt;2**31&lt;/code&gt;, so they used the range of signed 32-bit integers. That's fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't do the same to your weights as there's no limit on them. You can assume them to lie in the interval &lt;code&gt;(-128, +128)&lt;/code&gt; and scale them accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your assumption was wrong, you get an overflow and a huge negative weight where a huge positive weight should be or the other way round. In any case, a disaster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could check for overflow, but this is too expensive. Your arithmetic gets slower than FP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could &lt;em&gt;check for possible overflow from time to time&lt;/em&gt; and take a corrective action. The details may get complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use &lt;a href=&quot;https://en.wikipedia.org/wiki/Saturation_arithmetic&quot; rel=&quot;noreferrer&quot;&gt;saturation arithmetic&lt;/a&gt;, but it's implemented only in some specialized CPUs, not in your PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there's a multiplication. With use of 64-bit integers, it goes well, but you need to compute a sum (with a possible overflow) and scale the output back to same sane range (another problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;All in all, with fast FP arithmetic available, it's not worth the hassle.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be a good idea for a custom chip, which could do saturation integer arithmetic with much less hardware and much faster then FP.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Depending on what integer types you use, there may be a precision loss when compared to the floating point, which may or may not matter. Note that TPU (used in AlphaZero) has 8-bit precision only.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastEditorUserId="12053" LastEditDate="2018-07-29T04:14:13.680" LastActivityDate="2018-07-29T04:14:13.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7326" PostTypeId="2" ParentId="7247" CreationDate="2018-07-29T01:26:28.747" Score="2" Body="&lt;p&gt;&lt;strong&gt;Floating Point Hardware&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three common floating point formats used to approximate real numbers used in digital arithmetic circuitry.  These are defined in IEEE 754, a standard that was adopted in 1985 with a revision in 2008.  These mappings of bit-wise layouts real number representations are designed into CPUs, FPUs, DSPs, and GPUs either in gate level hardware, firmware, or libraries&lt;sup&gt;1&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;binary 32 has a 24 bit mantissa and an 8 bit exponent&lt;/li&gt;&#xA;&lt;li&gt;binary 64 has a 53 bit mantissa and an 11 bit exponent&lt;/li&gt;&#xA;&lt;li&gt;binary 128 has a 113 bit mantissa and a 15 bit exponent&lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Factors in Choosing Numerical Representations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any of these can represent signals in signal processing, and all have been experimented with in AI for various purposes related to three things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Value range &amp;mdash; not a concern in ML applications where the signal is properly normalized&lt;/li&gt;&#xA;&lt;li&gt;Averting saturation of the signal with rounding noise &amp;mdash; a key issue in parameter adjustment&lt;/li&gt;&#xA;&lt;li&gt;Time required to execute an algorithm on a given target architecture&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The balance in the best designed AI is between these last two items.  In the case of back-propagation in neural nets, the gradient-based signal that approximates the desired corrective action to apply to the attenuation parameters of each layer must not become saturated with rounding noise.&lt;sup&gt;3&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hardware Trends Favor Floating-point&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of the demand of certain markets and common uses, scalar, vector, or matrix operations using these standards may, in certain cases, be faster than integer arithmetic.  These markets include ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Closed loop control (piloting, targeting, countermeasures)&lt;/li&gt;&#xA;&lt;li&gt;Code breaking (Fourier, finite, convergence, fractal)&lt;/li&gt;&#xA;&lt;li&gt;Video rendering (movie watching, animation, gaming, VR)&lt;/li&gt;&#xA;&lt;li&gt;Scientific computing (particle physics, astrodynamics)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First Degree Transforms to Integers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the opposing end of numerical range, one can represent signals as integers (signed) or non-negative integers (unsigned).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the transformation between the set of real numbers, vectors, matrices, cubes, and hyper-cubes in the world of calculus&lt;sup&gt;4&lt;/sup&gt; and the integers that approximate them is a first degree polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The polynomial can be represented as $n = r(as + b)$, where $a \ne 0$, $n$ is the binary number approximation, $s$ is the scalar real, and $r$ is the function that rounds real numbers to the nearest integer.  This defines a super-set of the concept of fixed point arithmetic because of $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integer based calculations have also been examined experimentally for many AI applications.  This gives more options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;two's complement 16 bit integer&lt;/li&gt;&#xA;&lt;li&gt;16 bit non-negative integer&lt;/li&gt;&#xA;&lt;li&gt;two's complement 32 bit integer&lt;/li&gt;&#xA;&lt;li&gt;32 bit non-negative integer&lt;/li&gt;&#xA;&lt;li&gt;two's complement 64 bit integer&lt;/li&gt;&#xA;&lt;li&gt;64 bit non-negative integer&lt;/li&gt;&#xA;&lt;li&gt;two's complement 128 bit integer&lt;/li&gt;&#xA;&lt;li&gt;128 bit non-negative integer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example Case&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if your theory indicates the need to represent the real numbers in the range $[-\pi, \pi]$ in some algorithm, then you might represent this range as a 64 bit non-negative integer (if that works to the advantage of speed optimization for some reason that is algorithm and possibly hardware specific).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You know that $[-\pi, \pi]$ in the closed form (algebraic relation) developed from the calculus needs to be represented in the range $[0, 2^64 - 1]$, so in $n = r(as + b)$, $a = 2^61$ and $b = 2^60$. Choosing $a = \frac {2^64} {\pi}$ would likely create the need for more lost cycles in multiplication when a simple manipulation of the base two exponent is much more efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The range of values for that real number would then be [0, 1100,1001,0000,1111,1101,1010,1010,0010,0010,0001,0110,1000,1100,0010,0011,0101] and the number of bits wasted by keeping the relationship based on powers of two will be $\log_2 4 - log_2 \pi$, which is approximately 0.3485 bits.  That's better than 99% conservation of information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Back to the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is a good one, and is hardware and domain dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned above hardware is continuously developing in the direction to work with IEEE floating point vector and matrix arithmetic, particularly in 32 and 64 bit floating point multiplication.  For some domains and execution targets (hardware, bus architecture, firmware, and kernel), the floating point arithmetic may grossly out perform what performance gains can be obtained in 20th century CPUs by applying the above first degree polynomial transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why the Question is Relevant&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contract, if the product manufacturing price, the power consumption, and PC board size and weight must be kept low to enter certain consumer, aeronautic, and industrial markets, the low cost CPUs may be demanded. By design, these smaller CPU architectures do not have DSPs and the FPU capabilities don't usually have hardware realization of 64 bit floating point multiplication.&lt;sup&gt;5&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Handling Number Ranges&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Care in normalizing signals and picking the right values for &lt;em&gt;a&lt;/em&gt; and &lt;em&gt;b&lt;/em&gt; are essential, as mentioned, more so than with floating point, where the diminution of the exponent can eliminate many cases where saturation would be an issue with integers&lt;sup&gt;6&lt;/sup&gt;.  Augmentation of the exponent can avert overflow automatically too, up to a point, of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either type of numeric representation, normalizing is part of what improves convergence rate and reliability anyway, so it should always be addressed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to deal with saturation in applications requiring small signals (such as with gradient descent in back-propagation, especially when the data set ranges are over an order of magnitude) is to carefully work out the mathematics to avoid it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a science by itself, and only a few people have the scope of knowledge to handle hardware manipulation at the circuitry and assembly language level along with the linear algebra, calculus, and machine learning comprehension.  The interdisciplinary skill set is rare and valuable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Libraries for low level hardware operations such as 128 bit multiplication are written in cross assembly language or in C with the -S option turned on so the developer can check the machine instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Unless you are calculating the number of atoms in the universe, the number of permutations in possible game-play for the game Go, the course to a landing pad in a crater on Mars, or the distance in meters to reach a potentially habitable planet revolving around Proxima Centauri, you will likely not need larger representations than the three common IEEE floating point representations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Rounding noise naturally arises when digital signals approach zero and the rounding of the least significant bit in the digital representation begins to produce chaotic noise of a magnitude that approaches that of the signal.  When this happens, the signal is saturated in that noise and cannot be used to reliably communicate the signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] The closed forms (algebraic formulae) to be realized in software driven algorithms arise out of the solution of equations, usually involving differentials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] Daughter boards with GPUs are often too pricey, power hungry, hot, heavy, and/or packaging unfriendly in non-terrestrial and consumer markets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] The zeroing of feedback is skipped in this answer because it points to either one of two things: (A) Perfect convergence or (B) Poorly resolved mathematics.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-08-16T13:25:04.813" LastActivityDate="2018-08-16T13:25:04.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7327" PostTypeId="1" CreationDate="2018-07-29T07:28:19.293" Score="1" ViewCount="132" Body="&lt;p&gt;As a amateur researcher and tinkerer, I've been reading up on neuro-evolution networks (e.g. NEAT) as well as the A3C RL approach presented by &lt;a href=&quot;https://arxiv.org/pdf/1602.01783v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mnih et al&lt;/a&gt; and got to wondering if anyone has contemplated the merging of both these techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is such an idea viable? Has it been tried? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be interested in any research in this area as it sounds like it could be compelling.&lt;/p&gt;&#xA;" OwnerUserId="17162" LastEditorUserId="2444" LastEditDate="2019-02-16T02:53:17.607" LastActivityDate="2019-02-16T02:53:17.607" Title="Can neuro-evolution methods be combined with A3C?" Tags="&lt;reinforcement-learning&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7328" PostTypeId="1" CreationDate="2018-07-29T09:33:00.663" Score="23" ViewCount="1972" Body="&lt;p&gt;The impetus behind the twentieth century transition from analog to digital circuitry was driven by the desire for greater accuracy and lower noise.  Now we are developing software where results are approximate and noise has positive value.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In artificial networks, we use gradients (Jacobian) or second degree models (Hessian) to &lt;strong&gt;estimate&lt;/strong&gt; next steps in a convergent algorithm and define acceptable levels of inaccuracy and doubt.&lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;In convergence strategies, we &lt;strong&gt;deliberately add noise&lt;/strong&gt; by injecting random or pseudo random perturbations to improve reliability by essentially jumping out local minima in the optimization surface during convergence.&lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What we accept and deliberately introduce in current AI systems are the same things that drove electronics to digital circuitry.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why not return to analog circuitry for neural nets and implement them with operational amplifier matrices instead of matrices of digital signal processing elements?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The values of artificial network learning parameters can maintained using integrated capacitors charged via D-to-A converters such that the learned states can benefit from digital accuracy and convenience, while forward propagation benefits from analog advantages.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Greater speed&lt;sup&gt;3&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Orders of magnitude fewer transistors to represent network cells&lt;/li&gt;&#xA;&lt;li&gt;Natural thermal noise&lt;sup&gt;4&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;An academic article or patent search for analog artificial networks reveals much work over the last forty years, and the research trend has been maintained.  Computational analog circuits are well developed and provide a basis for neural arrays.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Could the current obsession with digital computation be clouding the common view of AI architectural options?&lt;/p&gt;&#xA;&lt;p&gt;Is hybrid analog the superior architecture for artificial networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt; &lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1] The PAC (probably approximately correct) Learning Framework relates acceptable error &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; and acceptable doubt &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; to the sample size required for learning for specific model types.  (Note that &lt;span class=&quot;math-container&quot;&gt;$1 - \epsilon$&lt;/span&gt; represents accuracy and &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt; represents confidence in this framework.)&lt;/p&gt;&#xA;&lt;p&gt;[2] Stochastic gradient descent is shown, when appropriate strategies and hyper-parameters are used, to converge more quickly during learning and is becoming a best practice in typical real world applications of artificial networks.&lt;/p&gt;&#xA;&lt;p&gt;[3] Intel Core i9-7960X Processor runs at turbo speeds of 4.2 GHz whereas the standard fixed-satelite broadcasting is 41 GHz.&lt;/p&gt;&#xA;&lt;p&gt;[4] Thermal noise can be obtained on silicon by amplifying and filtering electron leakage across a reverse biased zener diodes at its avalanche point.  The source of the quantum phenomena is Johnson–Nyquist thermal noise. Sanguinetti et. al. state in their 'Quantum Random Number Generation on a Mobile Phone' (2014), &amp;quot;A detector can be modeled as a lossy channel with a transmission probability η followed by a photon-to-electron converter with unit efficiency ... measured distribution will be the combination of quantum uncertainty and technical noise,&amp;quot; and there's CalTech's JTWPA work.  Both of these may become standards for producing truly nondeterministic quantum noise in integrated circuits.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1808.08173.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;STDP Learning of Image Patches with Convolutional Spiking Neural Networks&lt;/em&gt;, Saunders et. al. 2018, U Mass and HAS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://homes.cs.washington.edu/%7Eluisceze/publications/anpu-isca14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;General-Purpose Code Acceleration with Limited-Precision Analog Computation&lt;/em&gt;, Amant et. al., 2014&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://engineering.dartmouth.edu/news/analog-computing-and-biological-simulations-get-a-boost-from-new-mit-compil&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Analog computing and biological simulations get a boost from new MIT compiler&lt;/em&gt;, by Devin Coldewey, 2016&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://news.mit.edu/2016/analog-computing-organs-organisms-0620&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Analog computing returns&lt;/em&gt;, by Larry Hardesty, 2016*&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nsa.gov/news-features/declassified-documents/tech-journals/assets/files/why-analog-computation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Why Analog Computation?&lt;/em&gt;, NSA Declassified Document&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.columbia.edu/2016/back-to-analog-computing-columbia-researchers-merge-analog-and-digital-computing-on-a-single-chip/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Back to analog computing: Columbia researchers merge analog and digital computing on a single chip&lt;/em&gt;, Columbia U, 2016&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1612.02913.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Field-Programmable Crossbar Array (FPCA) for Reconfigurable Computing&lt;/em&gt;, Zidan et. al., IEEE, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/272429290&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;FPAA/Memristor Hybrid Computing Infrastructure&lt;/em&gt;, Laiho et. al., IEEE, 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/295542715&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Foundations and Emerging Paradigms for Computing in Living Cells&lt;/em&gt;, Ma, Perli, Lu, Harvard U, 2016&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/221431339_A_Flexible_Model_of_a_CMOS_Field_Programmable_Transistor_Array_Targeted_for_Hardware_Evolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Flexible Model of a CMOS Field Programmable Transistor Array Targeted for Hardware Evolution&lt;/em&gt; (FPAA), by Zebulum, Stoica, Keymeulen, NASA/JPL, 2000&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.electronicdesign.com/boards/custom-linear-array-incorporates-48-precision-op-amps-chip&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Custom Linear Array Incorporates Up To 48 Precision Op Amps Per Chip&lt;/em&gt;, Ashok Bindra, 2001, Electronics Design&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.444.8748&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Large-Scale Field-Programmable Analog Arrays for&#xA;Analog Signal Processing&lt;/em&gt;, Hall et. al., IEEE Transactions on Circuits and Systems, vol. 52, no. 11, 2005&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/1528675/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Large-scale field-programmable analog arrays for analog signal processing&lt;/em&gt;, Hall et. al. 2005&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pub.uni-bielefeld.de/publication/2426586&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity&lt;/em&gt;, Indiveri G, Chicca E, Douglas RJ, 2006&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/3486728970&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analog Computing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0817639497&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks and Analog Computation: Beyond the Turing Limit (Progress in Theoretical Computer Science)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="32410" LastEditDate="2021-04-24T11:57:57.530" LastActivityDate="2021-04-24T11:57:57.530" Title="If digital values are mere estimates, why not return to analog for AI?" Tags="&lt;machine-learning&gt;&lt;hardware&gt;&lt;neuromorphic-engineering&gt;&lt;analog-computing&gt;&lt;spiking-neural-networks&gt;" AnswerCount="7" CommentCount="2" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7329" PostTypeId="2" ParentId="6161" CreationDate="2018-07-29T09:54:31.517" Score="0" Body="&lt;p&gt;Artificial intelligence may exercise human rationality in some conditions, but if, after a time, all the thinking is delegated to computers, humans are likely to fall backward into superstitious times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;===============&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the text of the question unrelated to the title question, I'll treat that as a separate entity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact is that most people do manual labor because they feel they must.  If intellectual skills are also replaced by machines, those jobs will disappear, but not because the work is left undone.  The farm and factory automation replaced the skilled workers in many cases, but the farming and manufacture continues.  The same would be true of office work.  The cubes and once coveted window offices will be empty and decrepit, like the depleted oil fields of Pennsylvania, Texas, California, and the U.K., but the office work will continue to flow.  The computers are doing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically, the standard of living will remain the same if there is a one to one replacement.  If the computers improve above the skill and throughput of people, the human standard of living should increase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans can play sports, watch movies, blog, and garden for fun.  A world where there is 100% unemployment but a high GNP is idyllic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure we need to be directed toward an intelligent path in those regards.  Now if AI could be used to teach children not to solve problems by beating the crap out of each other, betraying each other, gossiping behind each other's backs, or scheming and scamming then that would be a plus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps AI could someday teach people not to drop bombs on each other or point thermonuclear weapons at each other too.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-29T09:54:31.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7333" PostTypeId="1" AcceptedAnswerId="7349" CreationDate="2018-07-29T15:50:23.187" Score="2" ViewCount="77" Body="&lt;p&gt;What if we took a recursive approach and built a smallest possible first robot (Robot 1) that could transfer information and data about the place it was at and could build itself in a very small size proportional to itself. I understand that it means a higher level of accuracy for this first robot (Robot 1) that its creator i.e. us.  And this first robot (Robot 1) again built a robot (Say Robot 2) that was far smaller but an exact copy of the first robot (Robot 1). And then the second robot (Robot 2) built a third Robot (Robot 3) and so on. So each next level robot was tinier and higher precision than its creator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the tiniest robot we could make, we sent them to the mission wherein micro-sized intervention was needed. For example studying the atom structure from inside, how similar it was to our big universe etc. Plus many more applications humankind could ever imagine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand though that the material used to construct such a robot and its properties will be limiting and to explore an atom we may not be able to use an atom as the building block. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we could possibly build a robot like this which would be small enough to explore the human body from inside. &lt;/p&gt;&#xA;" OwnerUserId="17170" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:30.613" LastActivityDate="2020-05-29T13:48:30.613" Title="What if we took a recursive approach and built a smallest possible robot?" Tags="&lt;robots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7335" PostTypeId="2" ParentId="7327" CreationDate="2018-07-29T16:58:09.540" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is such an idea viable? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach that &lt;em&gt;should&lt;/em&gt; work in terms of underlying theory is to start with population of NEAT-generated networks that describe the policy, and instead of measuring fitness of them on the task whilst keeping all weights static, measure fitness whilst applying any policy gradient algorithm like A3C. In addition, the final weights of the networks could be fed into the next generation. &lt;a href=&quot;https://ieeexplore.ieee.org/document/4637766/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A bit Lamarckian* perhaps, but that is already a thing in evolutionary algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Has it been tried? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=3205455.3205536&quot; rel=&quot;nofollow noreferrer&quot;&gt;a recent paper (July 2018) is &quot;NEAT for large-scale reinforcement learning through evolutionary feature learning and policy gradient search&quot;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect there are more, including hobby efforts without academic publishing, but that is the first paper that popped up on a brief search.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* &lt;a href=&quot;https://en.wikipedia.org/wiki/Jean-Baptiste_Lamarck&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jean-Baptiste Lamarck&lt;/a&gt; believed that offspring of animal parents could inherit traits based on parental behaviour and desires, such as a giraffe's neck evolving over time as the animals strived to reach higher food sources. Interestingly, although the core of this theory is not generally accepted, recent theories around phenotypic plasticity and discoveries in epigenetics show that biological systems can make use of the idea - although very far from the idea of children using parental memories directly to aid in tasks (which has more in common with science fiction for humans, but possible in neural networks where we can &quot;copy brains&quot;).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-07-29T17:06:10.573" LastActivityDate="2018-07-29T17:06:10.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7338" PostTypeId="2" ParentId="4165" CreationDate="2018-07-30T01:24:15.010" Score="1" Body="&lt;p&gt;You are thinking along the correct line when you consider intuitivity when examining the strange self-specialization aspect of the layers in any deep network that has been tuned, through toil, mental machinations, and late nights, to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do AI experimenters manipulated parameters, topology, and algorithmic hierarchies to force circuit evolution that produces data flows through specialized functions like this one.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Noisy, normalized, cubes (horizontal, vertical, depth) of integers straight from video hardware or the video channel of a multimedia file&lt;/li&gt;&#xA;&lt;li&gt;Indications of edges&lt;/li&gt;&#xA;&lt;li&gt;Indications of corners, ends, and bends&lt;/li&gt;&#xA;&lt;li&gt;Indications of relative angles (in the case of size independent recognition)&lt;/li&gt;&#xA;&lt;li&gt;Indications of shapes&lt;/li&gt;&#xA;&lt;li&gt;Indications of 2D topology&lt;/li&gt;&#xA;&lt;li&gt;Indications of object forms&lt;/li&gt;&#xA;&lt;li&gt;Indication of scene&lt;/li&gt;&#xA;&lt;li&gt;Indication of action (when a hypercube is used and frame is another dimension)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, one could design a solution and then tune layers, one by one, to follow the design, but it is not necessary because of the concept of entropy removal, the elimination of information that is redundant or irrelevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principle in information science is this, in general: As redundancy is removed from the information, &lt;strong&gt;the number of bits representing the information decreases&lt;/strong&gt; and &lt;strong&gt;the level of abstraction increases&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In minimalist art, a single word an identical twin might say to the other twin as the train rolls away, or a good Pictionary player, given a small window of information passing opportunity, adaptations can be made to pass large amounts of information base on previously agreed upon conventions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you think of it this way, it makes sense that by narrowing the bit width of the aggregate data representation from layer to layer that only certain parametric optimizations will provide an end-to-end indication of convergence, given a goal such as fully categorizing objects based on a small number of features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Leibniz envisioned a world tied together by mathematical certainty, he was overly optimistic, however he was correct in at least one important respect, mathematics had a reality all its own.  The success of Newtonian mechanics is one early example of a huge reality that has become apparent and has been so widely applicable, deduced from a thought experiment involving a cannon ball and the moon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are few demonstrations of discoveries that originally appear to be wild inventions than the amazingly accurate prediction of the previously anomalous orbit of Mercury.  But Einstein was not surprised by his success.  He had deduced general relativity from the challenge of Ernst Mach and the undeniable results from light and gravity experiments.  He discovered how it must be before it was found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same sense, the above sequence of events can be proven mathematically to be the efficient strategy for vision.  The sequence of processes to transform incoming visual signals to collision avoidance control was there in mathematics before there were visual receptors on microbes.  That is why the narrowing of the visual pathways of a Pterodactyl have similarities to those of a shark shark even though their common ancestor may not have been able to see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The incentive to specialize is the incentive to converge on the objective even after redundancy is stripped by force by limiting the information channel aperture.  Even if the aperture is in multiple dimensions, the same principle applies.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-30T01:24:15.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7339" PostTypeId="1" CreationDate="2018-07-30T05:33:15.530" Score="8" ViewCount="676" Body="&lt;p&gt;I want suggestions on literature on Reinforcement Learning algorithms that perform well with &lt;strong&gt;asynchronous feedback&lt;/strong&gt; from the environment. What I mean by asynchronous feedback is, when an agent performs an action it gets feedback(reward or regret) from the environment after sometime not immediately. I have only seen algorithms with immediate feedback and asynchronous updates. I don't know if literature on this problem exists. This is why I'm asking here.&lt;/p&gt;&#xA;&lt;p&gt;My application is fraud detection in banking, my understanding is when a fraud is detected it takes 15-45 days for the system to flag it as a fraud sometimes until the customer complains the system doesn't know its fraud.&lt;/p&gt;&#xA;&lt;p&gt;How would I go about designing a real-time system using reinforcement learning to flag transactions that are fraudulent or normal?&lt;/p&gt;&#xA;&lt;p&gt;Maybe my understanding is wrong, I'm learning on my own if someone could help me I would be grateful.&lt;/p&gt;&#xA;&lt;p&gt;The reason I'm looking at reinforcement learning instead of supervised learning is, it's hard to get ground truth data in the banking scenario. Fraudsters are always up-to-date or exceeding the state of the art in fraud detection. So I've decided that reinforcement learning would be an optimal direction to look for solutions to this problem.&lt;/p&gt;&#xA;" OwnerUserId="17136" LastEditorUserId="2444" LastEditDate="2020-12-14T23:11:41.327" LastActivityDate="2020-12-14T23:11:41.327" Title="Reinforcement Learning with asynchronous feedback" Tags="&lt;reinforcement-learning&gt;&lt;real-time&gt;&lt;delayed-rewards&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7340" PostTypeId="2" ParentId="7291" CreationDate="2018-07-30T07:13:25.497" Score="1" Body="&lt;p&gt;First of all , this is a deepmind's paper , so the computational level they operate on is much higher.&#xA;Can you share your code files , so that I can see how you interpreted miyosuda's code.&#xA;The disentangled representation has to be scaled down so that the model trains , and gets useful results in atleast a few hours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4GB is too less RAM for training any kind of model. The best temporary solution I can suggest is to perform the python script of distangled VAE on Colab &lt;a href=&quot;https://colab.research.google.com/notebooks/welcome.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colab Notebooks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more permanent solution that can making your model training much easier given your system constraints , is to use GCP or Azure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you plan to customize the architecture once you finish implementing distangled VAE. &lt;/p&gt;&#xA;" OwnerUserId="16920" LastActivityDate="2018-07-30T07:13:25.497" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7341" PostTypeId="1" CreationDate="2018-07-30T07:57:50.377" Score="0" ViewCount="72" Body="&lt;p&gt;I want to implement a real-time system for image comparison (e.g. compare a face with a reference one) on an Odroid. I would like to know what are the most suitable architectures for this task. I started with methods based on triplet loss (like Facenet) but I realized that a real-time solution is not feasible. Are there good, light alternatives?&lt;/p&gt;&#xA;" OwnerUserId="16671" LastActivityDate="2018-08-03T04:49:13.943" Title="Methods for fast image comparison" Tags="&lt;image-recognition&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7344" PostTypeId="2" ParentId="7339" CreationDate="2018-07-30T10:40:02.740" Score="2" Body="&lt;p&gt;I have been looking for a while into pretty much precisely the problem you describe (including the same application domain), but haven't been able to find much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious, &lt;strong&gt;mathematically &quot;correct&quot; solution&lt;/strong&gt; would be to simply delay your standard Reinforcement Learning update rule (of whatever algorithm you choose to implement) by 45 days; if it still wasn't reported as a fraud then, assume it was genuine. This leads to some problems though;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Need lots of memory to store experiences that were not yet used for updates&lt;/li&gt;&#xA;&lt;li&gt;Learning only starts after a significant delay, in which you don't learn anything at all yet and likely therefore run a suboptimal policy for a long time&lt;/li&gt;&#xA;&lt;li&gt;Very slow to adapt to new strategies of the fraudsters&lt;/li&gt;&#xA;&lt;li&gt;What to do with people who already report fraud cases earlier, like after 10 days? Delay them for the full 45 days anyway, or trigger updates immediately (and potentially mess up the ordering in which experiences actually occurred)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A quick and dirty &quot;solution&quot;&lt;/strong&gt; is the following;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When a transaction occurs, immediately trigger a learning update under the assumption that it was a genuine transaction (for example, with a reward of &lt;code&gt;R = +1&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;If that transaction is later reported as a fraud, trigger an additional update (with same &lt;code&gt;(state, action)&lt;/code&gt; pair), but with the negation of the reward that was previously assigned erroneously &lt;em&gt;on top of&lt;/em&gt; the normal negative reward for a fraudulent case. For example, if you would normally give &lt;code&gt;R = +1&lt;/code&gt; for genuines, and &lt;code&gt;R = -100&lt;/code&gt; for frauds, give a reward of &lt;code&gt;R = -101&lt;/code&gt; now. This reward will not correct for the previously assigned wrong reward in completely the right way (potentially wrong position in sequence of updates, discounting due to &lt;code&gt;gamma&lt;/code&gt; and maybe &lt;code&gt;lambda&lt;/code&gt; depending on algorithm used, etc.), but it should be somewhat close (especially if &lt;code&gt;gamma&lt;/code&gt; and &lt;code&gt;lambda&lt;/code&gt; are close to &lt;code&gt;1.0&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is certainly not ideal, has very little theoretical basis and probably breaks quite a bit of Reinforcement Learning theory, but at least it is efficient in terms of computation and memory and in my experience it works alright in practice.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you're using &lt;strong&gt;off-policy&lt;/strong&gt; RL algorithms, you can use &lt;strong&gt;Experience Replay&lt;/strong&gt; buffers (very popular in DQN-style things in Deep RL these days, but can also be used in tabular RL / RL with linear function approximation etc.). If you already have historical data generated through some non-RL policy in the past (which is typically the case in fraud detection / banking applications, they do have lots of data even if they don't always share it), you can use this to fill your experience replay buffer. In the case of the first solution (at the top of this answer), this can be used for training during the initial delay of 45 days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you expect there to be concept drift though (fraudsters adapting their behaviour over time), you'll want to be careful with experience replay. Old data will become less useful.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A very different style of solution is to assume that you have a team of &lt;strong&gt;human experts available who can investigate a very small portion of incoming transactions relatively quickly&lt;/strong&gt;. This tends to be true for large companies in practice (&quot;investigating&quot; often means a phone-call to a card holder). This enables you to generate accurate feedback for a small portion of your data more quickly, so that you can also do Reinforcement Learning with much less of a delay (albeit only on a small percentage of your experience).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about this idea in the following paper (&lt;strong&gt;disclaimer:&lt;/strong&gt; I'm an author on it):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dennis J.N.J. Soemers, Tim Brys, Kurt Driessens, Mark H.M. Winands, and Ann Nowé (2018). &lt;a href=&quot;https://dennissoemers.github.io/assets/publications/SoemersCreditCardIAAI.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;“Adapting to Concept Drift in Credit Card Transaction Data Streams Using Contextual Bandits and Decision Trees”&lt;/a&gt;. In Thirtieth Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-18), pp. 7831-7836. AAAI Press.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Apart from that idea you might furthermore find it interesting for references to other related work, links to data you could use, etc.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I feel like it should be possible to &lt;strong&gt;extend the existing Reinforcement Learning theory&lt;/strong&gt; with proper algorithms that can properly;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take immediate learning steps with an assumed, default, potentially incorrect reward, and&lt;/li&gt;&#xA;&lt;li&gt;Retroactively correct for previous incorrect updates if the reward turns out to be something else than previously assumed in hindsight.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of existing literature in which this is done though, and it certainly doesn't seem trivial; it will require starting pretty much from &quot;first principles&quot; (e.g., Bellman operator).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I also expect doing this completely correctly will always require a significant amount of memory (memory of all previous transactions of a card holder, such that state-action pairs can be re-generated if necessary). Banks likely already store that kind of data anyway for every customer, so it may not be a problem in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone's planning to work on this, feel free to contact me, I'll likely be happy to collaborate :D&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-07-30T10:40:02.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7346" PostTypeId="2" ParentId="7339" CreationDate="2018-07-30T12:08:19.497" Score="0" Body="&lt;p&gt;That this question uses the word &lt;em&gt;Feedback&lt;/em&gt; and made reference to more than one channel of feedback, &quot;Reward and regret,&quot; indicates a comprehension of corrective signaling.  Some of the reinforcement learning literature that appears scientific lacks that understanding, so beware of that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The temporal delay of fed back information is not unique to the case of banking fraud detection.  It is central to security breach detection in general, including web site hosting and telecommunications hacking.  It is also central to many other technology domains ranging from cyber-combat to chemical engineering to petroleum exploration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early control systems were of the PID form used in speed or direction governors.  In those, temporal elements were only analyzed to avert oscillation, overshoot, and undershoot.  Those are still relevant in fraud detection systems, but there are more requirements on the control system, specifically non-linearity in multiple dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, control theory has been extended more in the direction of measuring behavioral wellness.  Early temporal elements in digital systems included random access memory for applications and persistent memory for programs and data.  With the emergence of production ready AI, the temporal elements include acquired rules, fuzzy rule weights, convergence of network parameters corresponding to machine learning components, and other learned information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proof of concept in financial fraud detection is the same as for many other domains where the feedback can occur minutes, hours, days, or months after a decision was made or a signal propagated through an artificial learning network:  The neural networks of higher life forms, where asynchronous adaptation extends DNA based evolutionary adaptation, pain feedback is augmented by more abstract forms of feedback.  In humanoid and primate species, social satisfaction involves a specific signaling that involves neuro-compounds such as serotonin and oxytocin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of adaptation fits in asynchronicity between reflex and DNA adaptation, in the realm that ranges from Pavlov's conditioned response to the social phenomenon of commitment.  The importance of these capabilities is a result of the fact that not all sensory input that provides useful feedback about a behavior exhibited by the biological or artificial control system immediately after it is exhibited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some suggested reading below, and you may want to examine Bayes' Theorem and some of the software you can download in nearly every common programming language that implements what is called Naive Bayesian Categorization.  It is through the mathematics of probability theory that the best causal models can be realized.  What you probably want to do is learn the key elements of modeling causality with numbers FIRST and then consider how basic probabilistic causality modeling might be augmented with artificial networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although Richard Sutton and Andrew Barton's &lt;em&gt;Reinforcement learning: An introduction&lt;/em&gt; (1998 MIT Press) is considered an excellent overview, the early comparative works provide a more direct path to answer questions about algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you embark on algorithm development that involves both learning and asynchronicity, it is important to know at the onset that real time programming, such as is now used in high speed trading, is not for the faint at heart.  Real time processing places two reliability centered requirements on algorithms, and they should be addressed stringently if you want a stable, low maintenance system that works.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;State-safe &amp;mdash; In machine learning, functions that process feedback must not alter a set of interrelated parameters while in use by the forward propagation of the circuit.&lt;/li&gt;&#xA;&lt;li&gt;Re-entrant &amp;mdash; In machine learning, an interrupt from an incoming signal and a change of state must not frustrate the intent of the algorithm interrupted upon its resuming.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding attacks to banking systems, there will be escalation.  The countermeasures the banks take will be met by the countermeasure of the thieves.  It is a game, and the banking industry is wise to employ researchers and engineers that understand that learning is feedback dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may not find the best final designs in the liturature for this reason.  Banks naturally employ nondisclosure agreements (NDAs) to keep attackers from gaining knowledge about defensive strategies through web searches.  (If it is on the web, it is probably already hacked.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As researchers and engineers they employ, we are wise to employ asynchronous feedback and real time learning in fraud detection systems and seek a more informed position to stay ahead of engineers that don't value property rights for anyone but themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Suggested Literature&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/profile/Csaba_Szepesvari/publication/12723295_A_Unified_Analysis_of_Value-Function-Based_Reinforcement-Learning_Algorithms/links/09e4150cb771cd4eab000000.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A  Unified Analysis of Value-Function-Based Reinforcement-Learning Algorithms, Csaba Szepesvari, Michael L.  Littman, October 27, 1998&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v48/mniha16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Methods for Deep Reinforcement Learning, Volodymyr Mnih et al, University of Montreal, 2016&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v48/mniha16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep  Reinforcement  Learning  for  Robotic  Manipulation  with&#xA;Asynchronous  Off-Policy Updates, Shixiang Gu, Ethan Holly, Timothy Lillicrap, Sergey Levine, 2016&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.mit.edu/swg/ImagingPubs/connectivity/Dcm_Friston.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic causal modelling, K.J. Friston, L. Harrison, and W. Penny, Institute of Neurology, UK, 2003&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-07-30T12:16:30.253" LastActivityDate="2018-07-30T12:16:30.253" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7347" PostTypeId="1" CreationDate="2018-07-30T12:27:53.110" Score="1" ViewCount="291" Body="&lt;p&gt;I want to train a model to recognize different category of food (example: rice, burger, apple, pizza, orange,... )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the first training, I realized that the model is detecting other object as food. (example: hand -&gt; fish, phone -&gt; Chocolate, person -&gt; candies... )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get a very low loss because the testing dataset and validation must have at least a pictures of food. But when it comes to picture of object other than food, the model fails. How do label the dataset in way that the model will not do any detection if there is no food on the picture?&lt;/p&gt;&#xA;" OwnerUserId="17059" LastActivityDate="2018-07-31T13:59:23.830" Title="How to label “other” while labeling image for object detection/classification?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;models&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7349" PostTypeId="2" ParentId="7333" CreationDate="2018-07-30T14:26:29.540" Score="2" Body="&lt;p&gt;&lt;strong&gt;Lost Article and Found Articles&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MIT Review had an article on nanotechnology for disease eradication, DNA repair, and microsurgery in the 1990s that's probably somewhere among the thousands of entries resulting from a web search of, &quot;MIT Review nanotechnology cell repair,&quot; or the few hundred resulting from an academic article search for the same.&lt;sup&gt;1&lt;/sup&gt; The article I can't find described a recursive nano-robot scheme like the one described in this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What seems plausible given the current state of technology is to find a recursive algorithm that will command a 3D printer to make a smaller 3D printer that can print a still smaller one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Extending the Normal Meaning of Recursion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm will have to take a step in capability beyond mere recursion.  It doesn't call itself.  It must load itself into the machine it printed and then boot the copy of itself there.  As it loads itself, at each level in size, it must parametrize its child for that geometrically reduced size for each progressive reduction.  It must stop when the desired size is achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a paradigm could be called, &quot;Printail Recursion,&quot; from the synthesis of, &quot;Printer,&quot; and the algorithmic principle of, &quot;tail recursion,&quot; from the dawn of LISP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Applying the Decorator Pattern&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once Printail Recursion works, other robotic or algorithmic features could piggy back on the appropriate components of the progressive micro-children.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] The later of those two may even provide some departmental contacts to open options for partnering between academic institutions, something usually compelling for research oriented students and a great catalyst for scientific collaboration among the next generation of researchers.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-30T14:26:29.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7350" PostTypeId="1" AcceptedAnswerId="7505" CreationDate="2018-07-30T14:50:33.873" Score="3" ViewCount="223" Body="&lt;p&gt;In which scenario, when assembling a CNN, would you want to have two adjacent pooling layers, without a convolutional layer in between?&lt;/p&gt;&#xA;" OwnerUserId="16207" LastEditorUserId="2444" LastEditDate="2020-05-30T11:19:17.060" LastActivityDate="2020-05-30T11:20:31.957" Title="In which scenario would you want to have two adjacent pooling layers?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7351" PostTypeId="2" ParentId="6772" CreationDate="2018-07-30T16:24:00.663" Score="1" Body="&lt;p&gt;&lt;strong&gt;Hint :&lt;/strong&gt;  According to antlersoft's comment which says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;.....Microsoft  may well have used a neural network classifier to&#xA;  scan through it's telemetry  to find good  update candidates, it's&#xA;  clearly a classification problem.... &quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This could be right,however,here questions (information) concerning the company's internal research and development of their software products is classified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any machine learning engineer; is not professionally  allowed to discuss nor give out information pertaining the company's product i.e. source codes,plan/design or even the flow of the conceptual idea on how the system is implemented,and this is inline with what the question is all about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore,if you're passionate about how Microsoft implements artificial intelligence in previous operating system software source codes,then by all means you have to check out its;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.betaarchive.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;BetaArchive&lt;/a&gt; Discussion and Collection of Betas and Abandonware;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're a serious machine learning engineer,this community can help you as well,it releases them for developers and those who are passionately about open source project contribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The archive holds terabytes of files,which you can download for free. &#xA;So select which windows files have some machine learning algorithms in!&#xA;I wanted to give you a little bit of a glimpse,if you're a machine learning engineer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your information; answers to this question, will tend to be almost entirely based on opinions,rather than facts concerning Microsoft's software products.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="1581" LastEditDate="2018-08-03T11:43:44.880" LastActivityDate="2018-08-03T11:43:44.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7352" PostTypeId="1" AcceptedAnswerId="7353" CreationDate="2018-07-30T19:37:58.987" Score="17" ViewCount="1317" Body="&lt;p&gt;What are the mathematical prerequisites for understanding the core part of various algorithms involved in artificial intelligence and developing one's own algorithms?&lt;/p&gt;&#xA;&lt;p&gt;Please, refer some specific books.&lt;/p&gt;&#xA;" OwnerUserId="12021" LastEditorUserId="40434" LastEditDate="2021-04-04T15:14:22.987" LastActivityDate="2021-04-04T15:14:22.987" Title="What are the mathematical prerequisites for an AI researcher?" Tags="&lt;reference-request&gt;&lt;math&gt;&lt;getting-started&gt;&lt;academia&gt;&lt;books&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="7353" PostTypeId="2" ParentId="7352" CreationDate="2018-07-30T21:43:15.620" Score="13" Body="&lt;p&gt;&lt;strong&gt;Good Mathematics Foundation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Begin by ensuring full competency with intermediate algebra and some other foundations of calculus and discrete math, including the terminology and basic concepts within these topics.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Infinite series&lt;/li&gt;&#xA;&lt;li&gt;Logical proofs&lt;/li&gt;&#xA;&lt;li&gt;Linear algebra and matrices&lt;/li&gt;&#xA;&lt;li&gt;Analytic geometry, especially the distinction between local and global extremes (minima and maxima), saddle points, and points of inflection&lt;/li&gt;&#xA;&lt;li&gt;Set theory&lt;/li&gt;&#xA;&lt;li&gt;Probability&lt;/li&gt;&#xA;&lt;li&gt;Statistics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Foundations of Cybernetics&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Norbert Wiener, Cybernetics, 1948, MIT Press, contains time series and feedback concepts with clarity and command not seen in subsequent works; it also contains an introduction to information theory beginning with Shannon's log&lt;sub&gt;2&lt;/sub&gt; formula for a defining the amount of information in a bit. This is important to understand the expansion of the information entropy concept.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Calculus&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Find a good calculus book and make sure you have clarity around key theory and application in these categories.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Time series&lt;/li&gt;&#xA;&lt;li&gt;Infinite series&lt;/li&gt;&#xA;&lt;li&gt;Convergence — Artificial networks ideally converge to an optimum during learning.&lt;/li&gt;&#xA;&lt;li&gt;Partial differentials&lt;/li&gt;&#xA;&lt;li&gt;Jacobian and Hessian matrices&lt;/li&gt;&#xA;&lt;li&gt;Multivariate math&lt;/li&gt;&#xA;&lt;li&gt;Boundary regions&lt;/li&gt;&#xA;&lt;li&gt;Discrete math&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Much of that is in &lt;a href=&quot;https://ocw.mit.edu/ans7870/resources/Strang/Edited/Calculus/Calculus.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Calculus&lt;/em&gt;, Strang, MIT, Wellesley-Cambridge Press&lt;/a&gt;. Although the PDF is available on the web, it is basic and not particularly deep.  The one in our laboratory's library is &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0030567831&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Intermediate Calculus&lt;/em&gt;, Hurley, Holt Rinehart &amp;amp; Winston, 1980&lt;/a&gt;. It is comprehensive and in some ways better laid out than the one I have in my home library, which Princeton uses for sophomores.&lt;/p&gt;&#xA;&lt;p&gt;Ensure you are comfortable working in spaces beyond ℝ&lt;sup&gt;2&lt;/sup&gt; (beyond 2D).  For instance, RNNs are often in spaces such as ℝ&lt;sup&gt;4&lt;/sup&gt; thorugh ℝ&lt;sup&gt;7&lt;/sup&gt; because of the horizontal, vertical, pixel depth, and movie frame dimensions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Finite Math&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is unfortunate that no combination of any three books I can think of has all of these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Directed graphs — Learn this BEFORE trees or circuits (artificial nets) because it is the superset topography of all those configurations&lt;/li&gt;&#xA;&lt;li&gt;Abstract symbol trees (ASTs)&lt;/li&gt;&#xA;&lt;li&gt;Advanced set theory&lt;/li&gt;&#xA;&lt;li&gt;Decision trees&lt;/li&gt;&#xA;&lt;li&gt;Markov chains&lt;/li&gt;&#xA;&lt;li&gt;Chaos theory (especially the difference between random and pseudo-random)&lt;/li&gt;&#xA;&lt;li&gt;Game Theory starting with Von Neumann and Morgenstern's &lt;em&gt;Game Theory&lt;/em&gt;, the seminal work in that field&lt;/li&gt;&#xA;&lt;li&gt;Convergence in discrete systems especially the application of theory to signal saturation in integer, fixed point, or floating-point arithmetic&lt;/li&gt;&#xA;&lt;li&gt;Statistical means, deviations, correlation, and the more progressive concepts of entropy, relative entropy, and cross-entropy&lt;/li&gt;&#xA;&lt;li&gt;Curve fitting&lt;/li&gt;&#xA;&lt;li&gt;Convolution&lt;/li&gt;&#xA;&lt;li&gt;Probability especially Bayes' Theorem&lt;/li&gt;&#xA;&lt;li&gt;Algorithmic theory (Gödel's uncertainty theorems and Turing completeness)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Chemistry and Neurology&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is good to recall chemical equilibria from high school chemistry.  Balance plays a key role in more sophisticated AI designs.  Understanding the symbiotic relationship between generative and discriminative models in GANs will help a student further this understanding.&lt;/p&gt;&#xA;&lt;p&gt;The control functions within biological systems remain a primary source of proofs of concept in artificial intelligence research.  As researchers become more creative in imagining forms of adaptation that do not directly mimic some aspect of biology (still a distance off as of this writing) creativity may play a larger role in AI research objective formulation.&lt;/p&gt;&#xA;&lt;p&gt;Even so, AI will probably remain a largely interdisciplinary field.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="36737" LastEditDate="2021-04-02T20:48:50.630" LastActivityDate="2021-04-02T20:48:50.630" CommentCount="2" CommunityOwnedDate="2021-01-24T00:19:30.053" ContentLicense="CC BY-SA 4.0" />
  <row Id="7354" PostTypeId="2" ParentId="6461" CreationDate="2018-07-30T22:40:26.090" Score="1" Body="&lt;p&gt;It is already combined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adaptive entropy techniques are already used in most of the best compression encoders.  This is true for file encoders, video encoders, and audio encoders.  We use it in the solar lab to optimize sample rates in data acquisition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, pattern recognition and compression are very tightly coupled if you consider autoencoders and other feature extraction schemes and compare them mathematically with what compression does.  See &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-10226-4_16&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Data Compression &amp;ndash; A Generic Principle of Pattern Recognition?&lt;/em&gt;, Gunther Heidemann, Helge Ritter, VISIGRAPP 2008&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Zlib and lz4 have more like hyper-parametric learning, however they don't persist what they learn.  This work is interesting: &lt;a href=&quot;https://www.cs.ucsb.edu/~ckrintz/papers/acejournal.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Adaptive On-the-Fly Compression&lt;/em&gt;, Chandra Krintz, Sezgin Sucu, IEEE Parallel and Distributed Systems, v17 n1, January 2006&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**Suggested Project: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a theoretical framework and software POC that learns correlations between these two sets.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Quickly ascertainable features of documents or audio or video streams (i.e file path components, media titles, date, file type, and first N bytes)&lt;/li&gt;&#xA;&lt;li&gt;The existing parameters that existing open source compression software learns during its existing pattern recognition algorithms&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Persisting those correlations between compression invocations may considerably improve file transfer, kernel operations (since lz4 is now native in kernels like LINUX), and media streaming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much effort is made to persist features extracted (pattern recognition) between frames in media streaming is worth investigating too.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-07-30T22:40:26.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7355" PostTypeId="2" ParentId="6459" CreationDate="2018-07-31T01:38:06.970" Score="1" Body="&lt;p&gt;You don't necessarily need to roll out the inputs to an RNN; doing so makes it easier to optimize computation (if the sequence length is the same each batch), but it's not a necessity. Furthermore, RNNs (and, incidentally, the brain) doesn't necessarily remember the input history as is; rather, the history is encoded via the RNN's cell state (or states, in the case of LSTMs and other RNN cell architectures with multiple states). Neural Turing Machines (NMTs) and Differential neural computers expand on that concept by also using a larger &quot;memory&quot; storage (in the form of a matrix).&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2018-07-31T01:38:06.970" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7357" PostTypeId="2" ParentId="4142" CreationDate="2018-07-31T11:40:53.350" Score="0" Body="&lt;p&gt;&lt;strong&gt;Terminology&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two uses of the word &lt;em&gt;map&lt;/em&gt; in this discussion.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Road maps are construed below as images of road maps.&lt;/li&gt;&#xA;&lt;li&gt;Mapping input to desired output is the skill the system must learn.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The set of examples used to teach the system from an existing mapping of input to output is called a &lt;em&gt;labeled data set&lt;/em&gt; and the associated type of learning is called &lt;em&gt;supervised&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Training Resources&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Unreliable labeled road map examples and access to more also unreliable labeled road map examples&lt;/li&gt;&#xA;&lt;li&gt;A large number of unlabeled road map examples and access to more road maps&lt;/li&gt;&#xA;&lt;li&gt;A smaller number of labeled road map examples&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Inputs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Virtual game map as an image&lt;/li&gt;&#xA;&lt;li&gt;Starting position&lt;/li&gt;&#xA;&lt;li&gt;Target ending position&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The fastest route in sufficient detail to fully direct car movement &amp;mdash; Assuming that the route should be optimized for soonest arrival time, not for fuel conservation, minimal tire wear, safety, or minimal distance, since the car was identified as a race car.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Accurate Valuation of the Resource Inventory&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The low quality routes from the black box service provide neither examples of desired system behavior nor examples undesirable system behavior.  The former case would be good training data.  The later would be good to use in an adversarial architecture based on the design of GANs and their variants.  The uncertain quality of the labeled data from the black box service makes it ambiguous and, from an information science point of view, of zero value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing the low quality routes and the high quality routes may be interesting, but not particularly useful given the current state of machine learning.  The objective of your initial project phase is to teach the machine to generate a high quality route from the low quality route, not produce a comparison report.  To do that without processing the images a substantially large overlap between these two sets would be required.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The smaller number of labeled road map examples for which the labels were created manually&lt;/li&gt;&#xA;&lt;li&gt;Corresponding labeled road map examples for which the labels were created by the black box service&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That approach would require you to correct bad routes to create a sufficient training set.  Without overlap, you have no training data for artificial network training.  It may be possible to create an algorithm of GAN style that learns how to correct the black box service output using a concept called cross-entropy, but it would require processing the images and would likely be more difficult to do so than to replace the black box service altogether.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your ultimate goal is to create a working system that generates routes from images and start and and positions, I suggest discounting the external service altogether, and discard the idea of creating a route improver sub-system.  That you wish to improve upon the black box route generator's algorithm is noble but ultimately a time-consuming distraction to reaching the ultimate goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replace the unreliable thing with the reliable thing you design and build and to which you will have full and open access to improve.  Your system will ultimately have to deal with the images in your code either way, and that's the most challenging aspect of the overall task.  Just learn about CNNs and RNNs and get right to it.  That's my advise anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Note on Feedback&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any non-trivial route, the car will not likely stay on the road if given only turn, acceleration, and deceleration instructions.  Detection of the edges of the road would normally be needed as a feedback mechanism to accompany the route instructions.  The only way around this is to make the visualization sufficiently accurate that cumulative errors never exceed the distance that would put the car off pavement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Note on Data Representation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;JSON has a more flexible and terse structure when it comes to homogeneous arrays than XML, and it is easy to convert XML to JSON.  Furthermore, a transportation route is often represented as a directed graph, and there are many algorithms already conceived in graph theory that have been implemented in graph libraries in most languages.  For instance, shortest path, path equivalence, path concatenation, and detection of rings (driving in circles) are one line calls to these libraries.  Because JSON, for the reason given, has overtaken XML in many domains, the number of graph libraries that can read and write JSON directly has overtaken the number that can read or write XML directly.  The tooling for JSON analysis and visualization has surpassed that of XML at this point too.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:55:21.237" LastActivityDate="2018-10-15T23:55:21.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7358" PostTypeId="2" ParentId="7347" CreationDate="2018-07-31T13:59:23.830" Score="1" Body="&lt;p&gt;From how you've phrased your question I'm going to assume you've jumped in without much structured training in data science, so I'll answer at a fairly high level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an inherent problem with image classification in that if your final layer only has food classes then whatever you feed in will always be classified as a type of food regardless of what is actually in the image.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few techniques you could try.  The simplest and fastest being using a pre-built classifier to screen the input data for the presence of &quot;food&quot; and then only using your own classifier to determine what type of food it is.  Any of the open source imageNet networks would be a good step here - if that finds something with a food label then use your classifier to identify the category.  This means you won't need to retrain although you'll be at the mercy of any errors in the pre-trained category classifier, which is outside of your control.  Here's a good how to guide: &lt;a href=&quot;https://www.learnopencv.com/keras-tutorial-using-pre-trained-imagenet-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.learnopencv.com/keras-tutorial-using-pre-trained-imagenet-models/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to add a negative class to your data set consisting of non-food items that you label &quot;other&quot;.  This is probably trickier as you'd need to cover all the non-food categories that your network will see and ensure that it doesn't learn the background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at your final level and make a call about whether to accept the top result or not. You're forcing a choice and these values may give a clue as to the level of error in the prediction.&lt;/p&gt;&#xA;" OwnerUserId="2935" LastActivityDate="2018-07-31T13:59:23.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7359" PostTypeId="1" AcceptedAnswerId="7360" CreationDate="2018-07-31T14:34:31.353" Score="11" ViewCount="4178" Body="&lt;p&gt;I'm now learning about reinforcement learning, but I just found the word &quot;trajectory&quot; in &lt;a href=&quot;https://datascience.stackexchange.com/a/24924/8432&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure what it means. I read a few books on the Reinforcement Learning but none of them mentioned it. Usually these introductionary books mention agent, environment, action, policy, and reward, but not &quot;trajectory&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what does it mean? &#xA;According to &lt;a href=&quot;https://www.quora.com/In-the-context-of-reinforcement-learning-what-is-the-difference-between-a-trajectory-and-a-policy-Also-what-is-the-difference-between-trajectory-optimization-and-policy-optimization&quot; rel=&quot;noreferrer&quot;&gt;this answer&lt;/a&gt; over Quora:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In reinforcement learning terminology, a trajectory &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the path of the agent through the state space up until the horizon &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;. The goal of an on-policy algorithm is to maximize the expected reward of the agent over trajectories.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does it mean that the &quot;trajectory&quot; is the total path from the current state the agent is in to the final state (terminal state) that the episode finishes at? Or is it something else? (I'm not sure what the &quot;horizon&quot; mean, either).&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="2444" LastEditDate="2019-02-22T14:10:08.230" LastActivityDate="2020-01-10T10:17:31.147" Title="What is a &quot;trajectory&quot; in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7360" PostTypeId="2" ParentId="7359" CreationDate="2018-07-31T15:13:49.273" Score="9" Body="&lt;p&gt;In answer that you linked, I may have used an informal definition of &quot;trajectory&quot;, but essentially the same thing as the quote. A &quot;trajectory&quot; is the sequence of what has happened (in terms of state, action, reward) over a set of contiguous timestamps, from a single episode, or a single part of a continuous problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;span class=&quot;math-container&quot;&gt;$(s_3, a_3, r_4, s_4, a_4, r_5, s_5, a_5, r_6, s_6)$&lt;/span&gt; taken from any scenario where an agent was used in the problem environment would be a trajectory - at least as I intended it in the answer. This could be from real-world data, or a simulation. It could involve a totally random or untrained agent, or a fully-optimised policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the other definition that you have found, the focus on states and a &lt;em&gt;horizon&lt;/em&gt; could make it slightly different, but actually I suspect that it is the same thing, as it is not that useful to only know the states. The Quora answer is probably just using &quot;path of the agent through the state space&quot; as shorthand to describe the same data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;horizon&quot; in reinforcement learning is a future point relative to a time step, beyond which you do not care about reward (so you sum the rewards from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t+H$&lt;/span&gt;). Fixed horizons can be used as an alternative to a discount factor for limiting sums of reward in continuous problems. They may also be used in other approaches, but basically mean the same thing - a time step beyond which you don't account for what happens.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2018-11-12T20:39:24.003" LastActivityDate="2018-11-12T20:39:24.003" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7361" PostTypeId="1" AcceptedAnswerId="7368" CreationDate="2018-07-31T16:21:36.533" Score="0" ViewCount="181" Body="&lt;p&gt;Say I'm training a neural net to compute the following function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(color_of_clothing, body_height) -&amp;gt; gender&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When using this network for prediction, I can obviously plug in a pair &lt;code&gt;(c, b)&lt;/code&gt; to receive a predicted &lt;code&gt;g&lt;/code&gt;, but say I want to get a prediction only based on &lt;code&gt;c&lt;/code&gt; or only based on &lt;code&gt;b&lt;/code&gt;, can I use the same neural net somehow? Or would I need to train two separate neural nets &lt;code&gt;c -&amp;gt; g&lt;/code&gt; and &lt;code&gt;b -&amp;gt; g&lt;/code&gt; previously?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or more generally, can I use a neural net that was trained to predict &lt;code&gt;A -&amp;gt; B&lt;/code&gt; to make predictions on values from a subset of &lt;code&gt;A&lt;/code&gt;, or should I train separate neural nets on all subsets of &lt;code&gt;A&lt;/code&gt; that I'm interested in?&lt;/p&gt;&#xA;" OwnerUserId="17205" LastEditorUserId="7800" LastEditDate="2018-07-31T17:49:14.140" LastActivityDate="2018-08-01T07:12:27.210" Title="Dealing with &quot;blank&quot; inputs in prediction of a neural network?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7362" PostTypeId="1" CreationDate="2018-07-31T17:42:27.820" Score="-1" ViewCount="78" Body="&lt;p&gt;In doing a project using neural networks with an input layer, 4 hidden layers and an output layer ,I used mini batch gradient descent. I noticed that the randomly initialised weights seemed to do a good performance and gave a low error. As the model started training after about 200 iterations there was large jump in error and then it came down slowly from there. I have also noticed that sometimes the cost just increases over a set of consecutive iterations.&#xA;Can anyone explain why these happen? It is not like there are outliers or a new distribution as every iteration exposes it to the entire dataset. &#xA;I used learning rate 0.01 and regularisation parameter 10. I also tried regularisation parameter 5 and also 1.&#xA;And by the cost  I mean, the sum of squared errors of all minibatches/2m plus regularisation term error.&#xA;Further if this happens and my cost after the say 10000th iteration is more than my cost when I initialised with random weights (lol) can I just take the initial value? As those weights seem to be doing better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The large jumps are the most puzzling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated. Thanks&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="17143" LastEditDate="2018-08-03T02:39:22.127" LastActivityDate="2018-08-03T14:06:44.907" Title="Behaviour of cost" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7363" PostTypeId="2" ParentId="7352" CreationDate="2018-07-31T17:54:06.977" Score="3" Body="&lt;p&gt;As far as simple algorithms like Gradient Descent are concerned, you need to have a good grasp of partial derivatives. Especially if you want to implement neural networks. &#xA;Also most algorithms are vectorised to improve computing speed and so you need to be comfortable with matrix math. This involves being really quick and comfy with dimensions of matrices, dimensions of products, multiplication of matrices, transpose and so on. &#xA;Very rarely, you might use matrix calculus to directly arrive at optimal solutions, so a few results from this area should do.&#xA;Moving on, you need to understand some function analysis. this is needed to get an intuition on what activation functions like sigmoid and tanh, log are doing. &#xA;A grasp of probability and expectations is also really useful.&#xA;You should also be clear with orthogonal vectors and inner products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, I would suggest you grasp basic calculus and matrix operations and try learning AI concepts. If you can't figure something out, explore the math. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: again this is only for starting.&lt;/p&gt;&#xA;" OwnerUserId="17143" LastActivityDate="2018-07-31T17:54:06.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7364" PostTypeId="1" CreationDate="2018-07-31T20:31:43.493" Score="5" ViewCount="329" Body="&lt;p&gt;Does an AI exist that can automatically write software based on a formal specification of the software?&lt;/p&gt;&#xA;" OwnerUserId="17209" LastEditorUserId="2444" LastEditDate="2020-12-07T13:43:40.120" LastActivityDate="2020-12-07T23:04:52.670" Title="Does an AI exist that can write software based on a formal specification?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;genetic-programming&gt;&lt;inductive-programming&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7365" PostTypeId="2" ParentId="7364" CreationDate="2018-08-01T02:22:03.500" Score="5" Body="&lt;p&gt;I think that the answer to your question is yes. In the article &lt;a href=&quot;https://www.futurity.org/artificial-intelligence-bayou-coding-1740702/&quot; rel=&quot;noreferrer&quot;&gt;New A.I. application can write its own code&lt;/a&gt;, the authors state&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Computer scientists have created a deep-learning, software-coding application that can help human programmers navigate the growing multitude of often-undocumented application programming interfaces, or APIs.&lt;/p&gt;&#xA;&lt;p&gt;Designing applications that can program computers is a long-sought grail of the branch of computer science called artificial intelligence (AI). The new application, called Bayou, came out of an initiative aimed at extracting knowledge from online source code repositories like GitHub. Users can try it out at askbayou.com.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1703.05698.pdf&quot; rel=&quot;noreferrer&quot;&gt;Neural Sketch Learning for Conditional Program Generation&lt;/a&gt; may also be useful.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-01T02:12:43.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7366" PostTypeId="2" ParentId="7364" CreationDate="2018-08-01T06:17:33.903" Score="5" Body="&lt;p&gt;There's Neural Program Synthesis, which can be used to generate a piece of code. Please, have a look at the article &lt;a href=&quot;https://www.microsoft.com/en-us/research/project/neural-program-synthesis/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Program Synthesis&lt;/a&gt; by Microsoft for an overview of the field.&lt;/p&gt;&#xA;" OwnerUserId="17192" LastEditorUserId="2444" LastEditDate="2019-11-01T02:15:33.080" LastActivityDate="2019-11-01T02:15:33.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7367" PostTypeId="1" CreationDate="2018-08-01T06:54:36.177" Score="3" ViewCount="198" Body="&lt;p&gt;Is it possible to build a neural network that learns the connection between two images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a number of X images that related to Y images. How can I build a neural network that takes an image as an input and outputs (generates) the output image?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Y images are generated by applying some function to the X images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need a generate neural network for that? Are conventional neural networks capable of classification only?&lt;/p&gt;&#xA;" OwnerUserId="17216" LastActivityDate="2018-09-11T19:01:26.060" Title="A neural network to learn the connection between two images" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7368" PostTypeId="2" ParentId="7361" CreationDate="2018-08-01T07:12:27.210" Score="0" Body="&lt;p&gt;I think the answer to your question would be &quot;yes&quot;. Though inference would always be best if you provide representative training data. For example you can train your net with &lt;code&gt;(c, b)&lt;/code&gt; pairs, &lt;code&gt;(blank, b)&lt;/code&gt; pairs, and &lt;code&gt;(c, blank)&lt;/code&gt; pairs. That would make the net more robust and likely to support your use case. Training separate nets for each case would be more efficient and accurate - but I'm not sure what your goals and constraints are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is a &quot;blank&quot; input can be trained for as well in the same net. In your case it's a new kind of &lt;code&gt;body_height&lt;/code&gt; or &lt;code&gt;color_of_clothing&lt;/code&gt;. I also suspect you might have issues with how you encode &lt;code&gt;blank&lt;/code&gt; because e.g. &lt;code&gt;rgb(0, 0, 0)&lt;/code&gt; would be black, and perhaps you intended to model &lt;code&gt;blank&lt;/code&gt; as zero which would mean a blank is black.&lt;/p&gt;&#xA;" OwnerUserId="17217" LastActivityDate="2018-08-01T07:12:27.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7369" PostTypeId="1" CreationDate="2018-08-01T08:23:22.073" Score="4" ViewCount="2431" Body="&lt;p&gt;I am training a generative adversarial network (GAN) to generate images given &lt;em&gt;edge histogram descriptor&lt;/em&gt; (EHD) features of the image. The EHD features are themselves sparse (meaning they contain a lot of zeroes). While training the generator loss and discriminator loss are reducing very slowly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are deep learning models (like GAN) suitable for training with sparse data for one or more of the features in the input or derived through feature extraction?&lt;/p&gt;&#xA;" OwnerUserId="9062" LastEditorUserId="2444" LastEditDate="2019-07-06T13:16:27.957" LastActivityDate="2020-07-31T16:04:30.583" Title="Are deep learning models suitable for training with sparse data?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7370" PostTypeId="1" AcceptedAnswerId="7374" CreationDate="2018-08-01T09:06:17.030" Score="3" ViewCount="809" Body="&lt;p&gt;I'm confused with the two terminology - action and policy - in Reinforcement Learning. As far as I know, the action is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is what the agent makes in a given state.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, the book I'm reading now (Hands-On Reinforcement Learning with Python) writes the following to explain policy:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we defined the entity that tells us what to do in every state as policy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now, I feel that the policy is the same as the action. So what is the difference between the two, and how can I use them apart correctly?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="2444" LastEditDate="2019-05-12T23:22:18.390" LastActivityDate="2019-05-12T23:23:36.523" Title="What is the difference between policy and action in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7371" PostTypeId="1" CreationDate="2018-08-01T09:56:43.227" Score="5" ViewCount="322" Body="&lt;p&gt;I need to train a convolutional neural network to classify snake images. The problem is that I have only a small number of images available for some snake types.&lt;/p&gt;&#xA;&lt;p&gt;So, what is the best approach to train a neural network for image classification using a small data set?&lt;/p&gt;&#xA;" OwnerUserId="17220" LastEditorUserId="2444" LastEditDate="2020-11-29T15:57:47.270" LastActivityDate="2020-11-29T15:57:47.270" Title="How can I train a neural network for image classification when the dataset is small?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;datasets&gt;&lt;transfer-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7372" PostTypeId="2" ParentId="7371" CreationDate="2018-08-01T10:02:57.380" Score="4" Body="&lt;h2&gt;Use Fine Tuning&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can simply use a pre-trained model on &lt;a href=&quot;https://en.wikipedia.org/wiki/ImageNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;ImageNet&lt;/a&gt;, as this data set has multiple snakes classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can fine tune the model with your own small data set and outputs.&#xA;See this for further understanding :&#xA;&lt;a href=&quot;https://flyyufelix.github.io/2016/10/03/fine-tuning-in-keras-part1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fine Tuning in Keras&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(if you don't use Keras, there are other tutorials on the internet using other Machine Learning framework)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is just removing the last layer (1000 outputs if you use a model pre-trained with ImageNet) and adding a layer of your choice with random weights and a custom number of outputs (number of your classes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your retrain your network, in general we retrained only the last layers (as first layers have more general features).&lt;/p&gt;&#xA;" OwnerUserId="17221" LastEditorUserId="2193" LastEditDate="2018-08-01T11:58:29.817" LastActivityDate="2018-08-01T11:58:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7373" PostTypeId="2" ParentId="5329" CreationDate="2018-08-01T10:22:32.737" Score="2" Body="&lt;p&gt;In recent years the focus has been on layers rather than the more biologically inspired individual nodes.  As stated in the comment by thecomplexitytheorist you could use a computational graph, although then you have issues with distribution and you're limited to one framework. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created something in my PhD about the same time as the thesis you reference that was a simulation of biological structure and connected in a way not dissimilar to figure b above using ML to determine the weights of all the connections.  The control module of the network was distributed across several neurons, but allowed asymmetric connections between the inputs and outputs.  This was a large effort, and similar results could now be achieved more quickly using the modern frameworks.  This is, I believe, the main reason that biologically inspired neural networks are not as actively researched.  I still believe that there are tricks in biology that we've not encapsulated and would love the head space to create a framework to test this hypothesis!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiple Neural Networks are the simplest way to go and I've created several of these over the years (sadly not open source).  These can be distributed and use different frameworks - just wrap up each of your networks in an API that has common inputs and outputs and you can daisy chain them without worrying about the internals.  This would give you the power and flexibility you need and allow you to swap in and out multiple models.  I know I've made that sound simple but at this stage it's more a software engineering problem.  You do need to be wary of compound errors between your networks if you're trickling down as it's basically a decision tree of NNs at that point.&lt;/p&gt;&#xA;" OwnerUserId="2935" LastActivityDate="2018-08-01T10:22:32.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7374" PostTypeId="2" ParentId="7370" CreationDate="2018-08-01T11:16:24.160" Score="6" Body="&lt;p&gt;A policy is a function that maps states to a probability distribution over all possible actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in a typical Atari game, there might just be a handful of actions, represented by the keys that are used to play the game. In this context, the policy of a reinforcement learner might be represented by a pretty complex neural network that gets pixels as input and gives action probabilities as output. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2444" LastEditDate="2019-05-12T23:23:36.523" LastActivityDate="2019-05-12T23:23:36.523" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7376" PostTypeId="2" ParentId="7371" CreationDate="2018-08-01T14:26:05.403" Score="3" Body="&lt;p&gt;Besides using transfer learning described in other answer, you should consider using siamese network.&#xA;This type of network is used in cases when one does not posess many examples of objects he wants to distinguish. General idea is that instead of &quot;telling&quot; the network &quot;This is a cobra&quot;, you provide information like: &quot;This is a cobra, and that is a rattlesnake, learn the difference&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a whole subject dedicated to your problem and it is called one shot learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at this tutorial:&#xA;&lt;a href=&quot;https://hackernoon.com/one-shot-learning-with-siamese-networks-in-pytorch-8ddaab10340e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hackernoon.com/one-shot-learning-with-siamese-networks-in-pytorch-8ddaab10340e&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16929" LastActivityDate="2018-08-01T14:26:05.403" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7378" PostTypeId="2" ParentId="6902" CreationDate="2018-08-01T17:57:47.780" Score="0" Body="&lt;p&gt;If you want to vary definitions of outcome, the problem is more about optimal segmentation/clustering and not a classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clustering you could try latent class approaches.&lt;/p&gt;&#xA;" OwnerUserId="17229" LastActivityDate="2018-08-01T17:57:47.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7379" PostTypeId="1" AcceptedAnswerId="8347" CreationDate="2018-08-01T18:19:08.660" Score="3" ViewCount="153" Body="&lt;p&gt;In a recent paper about progress in computer animation a so called motion graph is used to describe the transition between keyframes of facial animation. &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13218&quot; rel=&quot;nofollow noreferrer&quot;&gt;Easy Generation of Facial Animation Using Motion Graphs, 2018&lt;/a&gt; As far as i understand from the paper, they used a motion capture device to record faces of real people and extract keyframes. Then a transition matrix was created to ensure that a walk from keyframe #10 to #24 is possible but a transition from keyframe #22 to #99 is forbidden.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea itself sounds reasonable good, because now a solver can search in the motion graph to bring the system from a laughing face to a bored face without interruption or unnatural in-between-keyframes. But wouldn't it be great if the transition matrix can be stored inside a neural network? As far as i understand the backpropagation algorithm , the neural network can learn input-output relations. So the neural network has to learn the transition probability between two keyframes. And a second neural network can then produce the motion plan which is also be trained by a large corpus. Is that idea possible or is it the wrong direction?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2018-10-10T23:15:47.653" Title="Using a neural network for learning a Motion Graph?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7381" PostTypeId="2" ParentId="7352" CreationDate="2018-08-01T19:06:56.337" Score="8" Body="&lt;p&gt;I work as a professor, and recently designed the mathematics requirements for a new AI major, in consultation with many of my colleagues at other institutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other answers, particularly @FauChrisian's do a good job of cataloging all the specific topics that might be useful &lt;em&gt;somewhere&lt;/em&gt; in AI, but not all of them are equally useful for understanding core topics. In other cases, understanding the topic is essentially the same as understanding the related AI algorithms, so we usually just teach them together instead of assuming prerequisite knowledge. For instance, Markov Decision processes aren't hard to teach to someone who already knows the basics of graph theory and probabilities, so we usually just cover them when we teach reinforcement learning in an AI course, rather than as a separate topic in a mathematics course. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematics requirements we settled on look like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A one or two semester course in discrete mathematics. This is as much to establish comfort with proof and mathematical rigor as with any specific topic in the area. It's mostly just &quot;foundational&quot; knowledge, but bits of it turn out to be very useful. Comfort with infinite summations, the basics of graphs, combinatorics, and asymptotic analysis are perhaps the most directly applicable parts. I like &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0495391328&quot; rel=&quot;noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Susanna Epp's book&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A one or two semester course in linear algebra, which is useful across a wide variety of topics in AI, especially machine learning and data mining. &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/032198238X&quot; rel=&quot;noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lay &amp;amp; Lay&lt;/a&gt; is an okay book, but probably not the absolute best. &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/048663518X&quot; rel=&quot;noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shilov&lt;/a&gt; is a recommendation from Ian Goodfellow and others, but I've not tried it myself.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A course in probability, and possibly a modern course in statistics (i.e. with a Bayesian focus). An older course in statistics, or one targeting social scientists, is not very useful though. My statistician colleagues are using &lt;a href=&quot;http://www.lock5stat.com/&quot; rel=&quot;noreferrer&quot;&gt;Lock5&lt;/a&gt; right now, and having good experiences with it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At least differential and integral calculus, and preferably at least partial derivatives in vector calculus, but perhaps the whole course. This is useful in optimization, machine learning, and economics-based approaches to AI. &lt;a href=&quot;https://www.amazon.ca/Calculus-Early-Transcendentals-James-Stewart/dp/1285741552/ref=sr_1_1?s=books&amp;amp;ie=UTF8&amp;amp;qid=1533150713&amp;amp;sr=1-1&amp;amp;keywords=stewart%20calculus%208th%20edition&quot; rel=&quot;noreferrer&quot;&gt;Stewart&lt;/a&gt; is the most common textbook. It's comprehensive, and can be used for all three courses, but it's explanations aren't always the very best. I'd still recommend it though.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Those are the core topics. If you don't &lt;em&gt;also&lt;/em&gt; have a traditional background in programming, then a course in graph theory and the basics of asymptotic complexity or algorithm design and analysis might be good supplements. Usually AI'ers come from a standard computer science background though, which covers all those things very well.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-01-26T18:31:41.363" LastActivityDate="2019-01-26T18:31:41.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7382" PostTypeId="2" ParentId="7364" CreationDate="2018-08-01T19:15:44.193" Score="4" Body="&lt;p&gt;The other answers cover modern work on this, but it's not even a new topic!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.genetic-programming.org/gpbook1toc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Koza's work&lt;/a&gt; in Genetic Programming (1992) led to whole sub-fields doing this. The techniques are widely used, robust, and well understood. They're just very computationally expensive. Enough so that most of the time you're better off just hiring a programmer to do it.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2020-12-07T23:04:52.670" LastActivityDate="2020-12-07T23:04:52.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7383" PostTypeId="2" ParentId="5763" CreationDate="2018-08-02T07:23:59.607" Score="6" Body="&lt;p&gt;The hot topics of today might be the cold, wet ashes of tomorrow.  For instance, the convergence speed of CNN and LSTM approaches, especially in combination, have diverted considerable attention away from basic RNN designs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the cold topics of today might be the burning embers of tomorrow. Of course, some of the cold topics will stay cold.  The sweet spot may be to identify those that are getting warmer and are likely to be sustainable building blocks future technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Residual Attention Networks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Residual attention networks, like LSTM networks, are an improvement over RNNs using a different approach.  Because attention networks are designed to conserve resources, they either converge faster or with less demand on hardware and network to support parallel execution.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2017/papers/Wang_Residual_Attention_Network_CVPR_2017_paper.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Residual Attention Network for Image Classification&lt;/em&gt;, Fei Wang et al, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2017/papers/Fu_Look_Closer_to_CVPR_2017_paper.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Look Closer to See Better: Recurrent Attention Convolutional Neural&#xA;Network for Fine-grained Image Recognition&lt;/em&gt;, Jianlong Fu et all, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/7974787/&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;VideoWhisper: Toward Discriminative Unsupervised Video Feature Learning With Attention-Based Recurrent Neural Networks&lt;/em&gt;, Na Zhao, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1701.07149&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Hierarchical Recurrent Attention Network for Response Generation&#xA;Chen Xing&lt;/em&gt;, Wei Wu et al, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Automated Development of Non-Cartesian Models&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Research into the automation of modelling is key to many AI applications.  Some of the algorithms under development do not simply extract tensors of features (arrays, matrices, cubes, and hyper-cubes), but develop graph models, directed or associative, with or without cycles permitted.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/profile/Zahra_Gharaee/publication/311373779_Hierarchical_Self-Organizing_Maps_System_for_Action_Classification/links/58c91c2b45851591df49e99b/Hierarchical-Self-Organizing-Maps-System-for-Action-Classification.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Hierarchical Self-organizing Maps System for Action Classification&lt;/em&gt;&#xA;Z Gharaee, P Gärdenfors, M Johnsson, ICAART, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dspace.library.uvic.ca/bitstream/handle/1828/8833/Castaneda_Lorena_PhD_2017.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Runtime Modelling for User-Centric Smart Cyber-Physical-Human Applications&lt;/em&gt;, Lorena Castañeda Bueno, 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1371/journal.pone.0118394&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Probing the Topological Properties of Complex Networks Modeling Short Written Texts&lt;/em&gt;, Diego R. Amancio, 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sisinflab.poliba.it/publications/2017/RTMDPMD17/Schema-summarization%20in%20Linked-Data-based%20feature%20selection%20for%20recommender%20systems.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Schema-summarization in Linked-Data-based feature selection for recommender systems&lt;/em&gt;, Azzurra Ragone et. al., 2017&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Signal Topologies That Support Equilibria&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many ignore the importance of GANs, not because they can do interesting things with images but because of how they deviate from the simple topology of signal path where convergence on a trained set of parameters is achieved over a one-dimensional array of layers and blocks of layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discriminative and generative components in GAN design are described in some detail in &lt;a href=&quot;https://ai.stackexchange.com/questions/3488/understanding-gan-loss-function/7130#7130&quot;&gt;another AI Stack Exchange question on *Understanding the GAN Loss FUnction&lt;/a&gt;.  Although the generation of images from the GAN approach and its conceptual children demonstrate a new capability in the artificial network space, the breadth of this multi-network significance may not be immediately obvious.  It is not a stack in depth of layers, but a stack of two deep networks in a figure-eight topology, conceptually much like a Möbius strip.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This topology creates a balance between two networks, the generative (G) and the discriminative (D).  Its designer termed it an adversarial relationship because G and D play opposing roles.  However, their action in the system is actually collaborative, creating a balance that is much like a chemical equilibrium or symbiosis in biology, so that a specific objective is achieved.  This may reveal the most promising direction in AI today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Designing signal topologies that support additional forms of collaboration and symbiosis between networks, where each network is a component that learns its roll in conjunction with other component networks, so that the aggregate system learns its function can synthesize forms of artificial intelligence that DNNs cannot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rules based systems and deep networks are one dimensional in terms of signal flow.  By themselves may never approximate the most notable features of the human brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Parallel Processing Using GPUs as DSPs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VLSI implementations of spiking networks is important, and there are now implementations such as &lt;a href=&quot;https://github.com/Hananel-Hazan/bindsnet&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/Hananel-Hazan/bindsnet&lt;/a&gt; that leverage GPU hardware acceleration to investigate them without access to the VLSI chips being developed by large corporations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Speech Recognition and Synthesis for End-to-End TTS&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recent emergence of excellence in synthesis using systems such as Google's WaveNet have opened the door to more accurate TTS (text to sound) applications, such that it is probably a good time to become an expert in voice recording for use in training example sets but a bad time to start a custom speech production house using live speakers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Automated Vehicles&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Automated vehicles of various types need experts in vehicle physics, automotive manufacture, aeronautics, and consumer products for a wide range of vehicle types with strong economic and safety incentives driving semi-automation and full automation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mars landers&lt;/li&gt;&#xA;&lt;li&gt;Consumer drones&lt;/li&gt;&#xA;&lt;li&gt;Industrial drones&lt;/li&gt;&#xA;&lt;li&gt;Military drones&lt;/li&gt;&#xA;&lt;li&gt;Passenger aircraft&lt;/li&gt;&#xA;&lt;li&gt;Passenger automobiles&lt;/li&gt;&#xA;&lt;li&gt;Limos&lt;/li&gt;&#xA;&lt;li&gt;Trains&lt;/li&gt;&#xA;&lt;li&gt;Wheel chairs&lt;/li&gt;&#xA;&lt;li&gt;Delivery vehicles&lt;/li&gt;&#xA;&lt;li&gt;Automated food distribution&lt;/li&gt;&#xA;&lt;li&gt;Nuclear plant repair robots&lt;/li&gt;&#xA;&lt;li&gt;Electrical distribution repair robots&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be difficult to discover in advance what of hot technologies in AI will remain dominant in five years or which of the warming technologies will be blazing hot then, but the above are solid technologies showing significant early promise and for which there are high business, industrial, and consumer demands.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-11T06:35:12.190" LastActivityDate="2018-10-11T06:35:12.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7384" PostTypeId="2" ParentId="2675" CreationDate="2018-08-02T07:54:15.373" Score="1" Body="&lt;p&gt;&lt;strong&gt;The List&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This list originates from Bruce Maxim, Professor of Engineering, Computer and Information Science at the University of Michigan.  In his lecture Spring 1998 notes for CIS 479&lt;sup&gt;1&lt;/sup&gt;, the following list was called,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&quot;Good Problems For Artificial Intelligence.&quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  Decomposable to easier problems&#xA;  Solution steps can be ignored or undone&#xA;  Predictable Problem Universe&#xA;  Good Solutions are obvious&#xA;  Internally consistent knowledge base (KB)&#xA;  Requires lots of knowledge or uses knowledge to constrain solutions&#xA;  Interactive&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It has since evolved into this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  Decomposable to smaller or easier problems&#xA;  Solution steps can be ignored or undone&#xA;  Predictable problem universe&#xA;  Good solutions are obvious&#xA;  Uses internally consistent knowledge base&#xA;  Requires lots of knowledge or uses knowledge to constrain solutions&#xA;  Requires periodic interaction between human and computer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What it is&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;His list was never intended to be a list of AI problem categories as an initial branch point for solution approaches or a, &quot;heuristic technique designed to speed up the process of finding a satisfactory solution.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maxim never added this list into any of his academic publications, and there are reasons why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The list is heterogeneous.  It contains methods, global characteristics, challenges, and conceptual approaches mixed into one list as if they were like elements.  This is not a shortcoming for a list of, &quot;Good problems for AI,&quot; but as a formal statement of AI problem characteristics or categories, it lacks the necessary rigor.  Maxim certainly did not represent it as a, &quot;7 AI problem characteristics,&quot; list.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;It is certainly not a, &quot;7 AI problem characteristics,&quot; list.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are There Any Category or Characteristics Lists?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no good category list for AI problems because if one created one, it would be easy to think of one of the millions of problems that human brains have solved that don't fit into any of the categories or sit on the boundaries of two or more categories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is conceivable to develop a problem characteristics list, and it may be inspired by Maxim's Good Problems for AI list.  It is also conceivable to develop an initial approaches list.  Then one might draw arrows from the characteristics in the first list to the best prospects for approaches in the second list.  That would make for a good article for publication if dealt with comprehensively and rigorously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Initial High Level Characteristics to Approaches List&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a list of questions that an experienced AI architect may ask to elucidate high level system requirements prior to selecting an approaches.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the task essentially static in that once it operates it is likely to require no significant adjustments?  If this is the case, then AI may be most useful in the design, fabrication, and configuration of the system (potentially including the training of its parameters).&lt;/li&gt;&#xA;&lt;li&gt;If not, is the task essentially variable in a way that control theory developed in the early 20th century can adapt to the variance?  If so, then AI may also be similarly useful in procurement.&lt;/li&gt;&#xA;&lt;li&gt;If not, then the system may possess sufficient nonlinear and temporal complexity that intelligence may be required.  Then the question becomes whether the phenomenon is controllable at all.  If so, then AI techniques must be employed in real time after deployment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Effective Approach to Architecture&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one frames the design, fabrication, and configuration steps in isolation, the same process can be followed to determine what role AI might play, and this can be done recursively as one decomposes the overall productization of ideas down to things like the design of an A-to-D converter, or the convolution kernel size to use in a particular stage of computer vision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with other control system design, with AI, determine your available inputs and your desired output and apply basic engineering concepts.  Thinking that engineering discipline has changed because of expert systems or artificial nets is a mistake, at least for now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nothing has significantly changed in control system engineering because AI and control system engineering share a common origin.  We just have additional components from which we can select and additional theory to employ in design, construction, and quality control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rank, Dimensionality, and Topology&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the rank and dimensions of signals, tensors, and messages within an AI systems, Cartesian dimensionality is not always the correct concept to characterize the discrete qualities of internals as we approach simulations of various mental qualities of the human brain.  Topology is often the key area of mathematics that most correctly models the kinds of variety we see in human intelligence we wish to develop artificially in systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More interestingly, topology may be the key to developing new types of intelligence for which neither computers nor human brains are well equipt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://groups.umd.umich.edu/cis/course.des/cis479/lectures/htm.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://groups.umd.umich.edu/cis/course.des/cis479/lectures/htm.zip&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-08T22:53:58.803" LastActivityDate="2018-10-08T22:53:58.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7386" PostTypeId="2" ParentId="7367" CreationDate="2018-08-02T08:30:35.363" Score="2" Body="&lt;p&gt;It is possible to have both input and output be images that differ in a predictable way. For example, architectures similar to autoencoders have been used to remove blur, change weather conditions, change between day and night photos etc. In these architectures, the training data is matching pairs of images. If your goal is to replicate some image enhancement, then often the input is artificially processed e.g. to reduce its quality in a hard to reverse way. A good example of this would be to remove distortion or noise from an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use generative models. These are  harder to get working, but can be more flexible in that you don't need image pairs in order to train, just a set of images labelled with the traits that you want to learn. Converting an image using a generative model involves using an encoder stage to get its embedding, altering the embedding based on label you require and then feeding the new embedding into the decoder stage. This is how you might alter a face portrait from male to female, or young to old, because it is not possible to find good natural image pairs for that task.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-08-02T08:30:35.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7387" PostTypeId="2" ParentId="2876" CreationDate="2018-08-02T10:08:22.643" Score="0" Body="&lt;p&gt;The question is a good one and on many people's minds.  There are a few misconceptions in the line of thought to consider.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The supremacy of intelligent beings other than humans threaten civilization &amp;mdash; Is the imagining of that threat logical?  Is that a rational conclusion when human intelligence is the most threatening biological phenomenon in the biosphere today.  The intelligence of insects may be more sustainable.  The civilizations of ants and termites are certainly more collaborative.  World wars and genocide are one of the primary features of human history.&lt;/li&gt;&#xA;&lt;li&gt;Artificial intelligence becomes more intelligent than humans &amp;mdash; Artificial intelligence is already more intelligent than humans in some respects, which is why we use calculators, automated switching of communications signals instead of operators, and automated mail sorters.  In other ways, AI has to cross astronomical distances to begin to approximate human intelligence.  We have nothing that even shows a hint of being able to simulate or duplicate in the future the human capacities of reasoning, inventiveness, or compassion.&lt;/li&gt;&#xA;&lt;li&gt;The singularity is predicted 2040s &amp;mdash; There are over a dozen unrealistic assumptions in those predictions.  Look with a critical eye at any one of the arguments behind them and you will fine holes you could fly a 757 through blindfolded.&lt;/li&gt;&#xA;&lt;li&gt;Exponential growth in knowledge &amp;mdash; There is an exponential growth in information, but the proportion of that information that is legitimately peer reviewed decreases as misinformation, wild conjecture, and fake news increase.  My belief is that the amount of information in the world can be approximated by log(n), where n is the population.  If I am on track with that relation, the informational value of the average individual is log(n)/n, which decreases as population grows.&lt;/li&gt;&#xA;&lt;li&gt;AI will be capable of omniscience &amp;mdash; Omniscience is more than answering any question.  Omniscience would require that the answering be 100% reliable and 100% accurate, which may require more silicon than exists in the universe.  Read Gleick's &lt;em&gt;Chaos&lt;/em&gt; if you wish to understand why.  One might also argue that an omniscient being would not answer questions on command.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want a world that is better than one controlled by human intelligence, then one direction to take is to seek the development of a new species with a more advanced conception of peaceful civilization that can tame and domesticate us like we have done with dogs and cats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easier route is just for all of us to die.  The biosphere may have been more civilized before we arrived and started killing everything and each other.  But that's not my hope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is that we study NI (non-intelligence) and discover how to rid what is stupid from human behavior and geopolitical interaction.  That would improve the world far more than distracting and substantially irrelevant machine learning gadgets and apps.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-02T10:08:22.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7389" PostTypeId="1" AcceptedAnswerId="7402" CreationDate="2018-08-02T14:26:54.830" Score="4" ViewCount="118" Body="&lt;p&gt;Can we really make a chatbot that understands (rather than just replies to) questions based on the database/options of replies that it has? I mean, can it come up with correct/non-stupid replies/communications that don't exist in its database?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, can we make it understand the words &lt;em&gt;but&lt;/em&gt;, &lt;em&gt;if&lt;/em&gt; and so on? So whenever it gets a question/order it understands it based on &quot;understanding&quot;. Like the movie &quot;her&quot; if you have watched it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And all of this without using too much of code, just the basics to &quot;wake it up&quot; and let it learn from YouTube videos and Reddit comments and other data source like that.&lt;/p&gt;&#xA;" OwnerUserId="17246" LastEditorUserId="1671" LastEditDate="2018-08-07T18:56:02.287" LastActivityDate="2018-08-07T18:56:02.287" Title="Can we make a chatbot that UNDERSTANDS not just answer questions based on his database?" Tags="&lt;chat-bots&gt;&lt;theory&gt;&lt;soft-question&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7390" PostTypeId="1" AcceptedAnswerId="7425" CreationDate="2018-08-02T14:59:08.493" Score="14" ViewCount="6952" Body="&lt;p&gt;I'm struggling to understand the difference between actor-critic and advantage actor-critic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least, I know they are different from asynchronous advantage actor-critic (A3C), as A3C adds an asynchronous mechanism that uses multiple worker agents interacting with their own copy of the environment and reports the gradient to the global agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what is the difference between the actor-critic and advantage actor-critic (A2C)? Is it simply with or without &lt;em&gt;advantage&lt;/em&gt; function? But, then, does the actor-critic have any other implementation except for the use of advantage function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe are they synonyms and actor-critic is just a shorthand for A2C?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="2444" LastEditDate="2020-05-14T10:19:42.670" LastActivityDate="2020-05-14T10:22:43.137" Title="What is the difference between actor-critic and advantage actor-critic?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;actor-critic-methods&gt;&lt;comparison&gt;&lt;advantage-actor-critic&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7391" PostTypeId="2" ParentId="7389" CreationDate="2018-08-02T16:49:35.407" Score="1" Body="&lt;p&gt;&lt;strong&gt;Of course you can&lt;/strong&gt; (read through to the end). You just need to teach it how it is taught to a baby. But first, you need to create the baby's brain. So you need to build a brain that learns from videos, poll videos, and not just understands but practices and understands other people's reactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, but that's not enough. You would have the same work as God (if it really does exist and took this job). You would have to raise a baby so he could grow up. If we can raise a baby in code, it will learn much faster than a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been studying and looking since creating this &quot;baby.&quot; I've made some babies, but none of them have been enough. But it's what I chose to do in my life (one of things). So I'm still raising a baby, from time to time it's getting smarter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to build a robot that goes into reddit to read and extract feeling from what people write. You can watch YouTube videos and differentiate objects, humans, colors, etc. But that is what we would codify for this &quot;baby&quot; to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the first step would be to rebuild a brain through code. We are already creating some pieces, we are already studying for many years synapses, neural networks, etc. But there is still a whole brain that we still do not understand. And I'm talking about the human brain (biological).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I say you can, it's an incentive. I told myself I can. I'm going down this road. If I really can, I do not know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One tip I give you: Google is far from successful. But you are trying and reaching for it. That's enough, right?&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-08-02T16:49:35.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7392" PostTypeId="1" CreationDate="2018-08-02T17:08:13.317" Score="1" ViewCount="58" Body="&lt;p&gt;I would like to know some daily basis applications of AI. I think these might be relevant examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Google search engine&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Face recognition on iPhone &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are my examples correct? Could you provide some more examples?&lt;/p&gt;&#xA;" OwnerUserId="17249" LastEditorUserId="16909" LastEditDate="2018-08-07T18:15:15.577" LastActivityDate="2018-08-07T18:15:15.577" Title="What are some examples of everyday systems that use AI?" Tags="&lt;ai-basics&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7393" PostTypeId="2" ParentId="7341" CreationDate="2018-08-02T19:00:47.480" Score="1" Body="&lt;p&gt;The problem might not be caused by your loss function. Deep learning models tend to be computationally demanding. Mobile devices are not usually prepared for handling models with high throughput. That being said, you might try to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Prepare smaller model - less layers, less computation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mobile models optimization - Google provides some materials on optimizing Tensorflow models for mobile inference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/mobile/prepare_models&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/mobile/prepare_models&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.tensorflow.org/mobile/optimizing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/mobile/optimizing&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16929" LastActivityDate="2018-08-02T19:00:47.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7394" PostTypeId="1" AcceptedAnswerId="7395" CreationDate="2018-08-02T20:29:35.273" Score="9" ViewCount="299" Body="&lt;p&gt;What is a support vector machine (SVM)? Is an SVM a kind of a neural network, meaning it has nodes and weights, etc.? What is it best used for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I can find information about these?&lt;/p&gt;&#xA;" OwnerUserId="17250" LastEditorUserId="2444" LastEditDate="2020-05-16T14:35:55.607" LastActivityDate="2020-05-16T14:35:55.607" Title="What is a support vector machine?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;definitions&gt;&lt;support-vector-machine&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7395" PostTypeId="2" ParentId="7394" CreationDate="2018-08-02T21:03:46.043" Score="6" Body="&lt;p&gt;I find the chapter on machine learning from &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; is a pretty good place to start with SVMs. I think this is Chapter 18?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to understand an SVM is as a kind of neural network, but this is not usually an intuitive approach for a beginner (unless your NN knowledge is already quite good).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better way to understand SVMs is as consisting of three simple ideas rolled into one algorithm. Here's an attempt at a &quot;For Dummies&quot; answer though:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Maximum Margin Classification.&lt;/strong&gt; SVMs are usually used to find a pattern in a set of data. Often, the data allow an infinite set of possible patterns that are all equally descriptive. For example, maybe The relationship is &quot;Lives within 5 miles of a Coast -&gt; Income High&quot;. It's easy to imagine that this pattern is just as good as &quot;Lives within 5.0001 miles of a Coast -&gt; Income High&quot; or &quot;Lives within 4.999 miles of a Coast -&gt; Income High&quot;. There might actually be a lot more play than that in the data (e.g. 3 miles might work out too). If all these are equally good, then the maximum margin idea says you should pick the one that's &quot;in the middle&quot; of the data. So maybe all values between 5.5 and 4.8 are equally good. In that case, we might pick 5.15 (in the middle). This example is super simplified. Real world data would have a lot more variables, and the idea of &quot;in the middle&quot; ends up being a little more complex, but this is the intuition. It turns out that finding the maximum margin pattern is easy when the patterns are &lt;em&gt;linear&lt;/em&gt;. That is, when they can be represented by drawing straight lines through a plot of the dataset.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Projection into higher dimensions.&lt;/strong&gt; This one needs a bit of math to visualize. Consider a dataset consisting of a circular pattern (for instance, maybe the pattern is that higher incomes are found in the middle of the city). There is no &lt;em&gt;linear&lt;/em&gt; relationship that captures this pattern. That is, you can't draw a straight line through the data, and say something meaningful about all the values on one side or the other. However, if you add a new feature to your data that is the square of the original coordinates, it's easy to find such a pattern. Basically, if you pre-compute &quot;circular&quot; functions of the original data, you can add them to the dataset, and then find a pattern that is a linear function of this new feature. This idea generalizes: if you compute a complex enough function of your original data, and then apply the maximum margin idea, you can learn any pattern you like. The problem is that it's slow: adding more features makes it take longer to find the patterns you want.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;The Kernel Trick&lt;/strong&gt;. The thing that made SVMs useful was the kernel trick: finding the maximum margin didn't depend on anything except the product of the coordinates of the various points. It turned out that this product could be computed first, and then run through certain functions to produce a problem that was &lt;em&gt;identical&lt;/em&gt; to the one you'd get by first adding extra features and then doing the multiplication. However, computing the problem this way didn't require adding any new features! This made SVMs one of the first reliable, well understood, and fast methods for finding non-linear patterns in data. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope that provides a starting point. Consider reading Russell &amp;amp; Norvig as a next starting point, or &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0387310738&quot; rel=&quot;noreferrer&quot;&gt;Bishop&lt;/a&gt; if you want to go deeper. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-08-03T13:35:46.950" LastActivityDate="2018-08-03T13:35:46.950" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7396" PostTypeId="2" ParentId="7392" CreationDate="2018-08-02T21:24:00.067" Score="2" Body="&lt;p&gt;A good, recent, and accessible book which includes many case studies is &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/1633695670&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prediction Machines&lt;/a&gt;. Check it out for more details than I can provide in this answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example applications are all around us, but one of the problems with recognizing them is that the bar for what we call AI is constantly being raised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that a few decades ago, directions from &lt;a href=&quot;https://www.google.com/maps&quot; rel=&quot;nofollow noreferrer&quot;&gt;google maps&lt;/a&gt; would certainly be recognized as AI, whereas now most laypeople wouldn't make that association.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other commonplace examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.amazon.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amazon.com&lt;/a&gt; can guess at what you'd like to buy next.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://facebook.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facebook&lt;/a&gt; and many other companies have programs that decide which ads to show you, without human intervention.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tesla.com/autopilot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla&lt;/a&gt;'s autopilot feature will drive your car down a highway.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.google.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google&lt;/a&gt; search can instantly return the results you were looking for, even when you enter strange or ambiguous search terms.&lt;/li&gt;&#xA;&lt;li&gt;Virtual Assistants from many companies, like &lt;a href=&quot;https://www.amazon.com/Amazon-Echo-And-Alexa-Devices/b?ie=UTF8&amp;amp;node=9818047011&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alexa&lt;/a&gt; recognize your voice, figure out what words you're saying, and then figure out what you want them to do.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://Stackoverflow.com&quot;&gt;StackOverflow&lt;/a&gt; determines which questions are hot, and which ones need moderator attention.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are are also a lot of less obvious examples. I'll pick a few from my research area:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When you spot a sky marshal on a plane, or a dog patrol in the airport it's because an &lt;a href=&quot;http://teamcore.usc.edu/ARMOR-LAX/default.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI system&lt;/a&gt; put them there. Some subway systems now use the same technology for fare checkers. &lt;/li&gt;&#xA;&lt;li&gt;When people trade agricultural products in Uganda, it's an &lt;a href=&quot;https://www.youtube.com/watch?v=qgQ4a3uKe48&amp;amp;feature=youtu.be&amp;amp;list=PLzHDzaAtdYunlvBjx1W_C2w9HJwVX7See&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI system&lt;/a&gt; and AI techniques at work behind the scenes. &lt;/li&gt;&#xA;&lt;li&gt;When you're assigned a donor for a kidney transplant (and now other organs too), an &lt;a href=&quot;https://www.hopkinsmedicine.org/transplant/programs/kidney/incompatible/paired_kidney_exchange.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI system&lt;/a&gt; did the heavy lifting in that decision. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, there's &lt;a href=&quot;http://aaai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;lots more&lt;/a&gt; going on too, but those should give you some suggestions for everyday conversation.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-02T21:24:00.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7397" PostTypeId="1" CreationDate="2018-08-03T00:41:39.313" Score="2" ViewCount="2332" Body="&lt;p&gt;I'm now reading &lt;a href=&quot;https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-7-action-selection-strategies-for-exploration-d3a97b7cceaf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the following blog post&lt;/a&gt; but on the epsilon-greedy approach, the author implied that the epsilon-greedy approach takes the action randomly with the probability epsilon, and take the best action 100% of the time with probability 1 - epsilon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example, suppose that the epsilon = 0.6 with 4 actions. In this case, the author seemed to say that each action is taken with the following probability (suppose that the first action has the best value):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;action 1: 55% (.40 + .60 / 4) &lt;/li&gt;&#xA;&lt;li&gt;action 2: 15%&lt;/li&gt;&#xA;&lt;li&gt;action 3: 15%&lt;/li&gt;&#xA;&lt;li&gt;action 4: 15%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I feel like I learned that the epsilon-greedy only takes the action randomly with the probability of epsilon, and otherwise it is up to the policy function that decides to take the action. And the policy function returns the probability distribution of actions, not the identifier of the action with the best value. So for example, suppose that the epsilon = 0.6 and each action has 50%, 10%, 25%, and 15%. In this case, the probability of taking each action should be the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;action 1: 35% (.40 * .50 + .60 / 4)&lt;/li&gt;&#xA;&lt;li&gt;action 2: 19% (.40 * .10 + .60 / 4)&lt;/li&gt;&#xA;&lt;li&gt;action 3: 25% (.40 * .25 + .60 / 4)&lt;/li&gt;&#xA;&lt;li&gt;action 4: 21% (.40 * .15 + .60 / 4)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is my understanding not correct here? Does the non-random part of the epsilon (1 - epsilon) always takes the best action, or does it select the action according to the probability distribution?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastActivityDate="2019-05-20T10:35:54.320" Title="Does epsilon-greedy approach always choose the &quot;best action&quot; (100% of the time) when it does not take the random path?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7398" PostTypeId="2" ParentId="7362" CreationDate="2018-08-03T04:38:53.470" Score="1" Body="&lt;p&gt;I would say just don't go for using regularization at first, try with lower &lt;code&gt;learning rate =  0.0001&lt;/code&gt; see the behavior. Try to post entire architecture of your model so that one can better answer your problem.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-08-03T04:38:53.470" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7399" PostTypeId="2" ParentId="7341" CreationDate="2018-08-03T04:49:13.943" Score="0" Body="&lt;p&gt;For face ID, Apple is using Siamese Network. You may get a better idea here &lt;a href=&quot;https://towardsdatascience.com/one-shot-learning-face-recognition-using-siamese-neural-network-a13dcf739e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/one-shot-learning-face-recognition-using-siamese-neural-network-a13dcf739e&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-08-03T04:49:13.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7400" PostTypeId="2" ParentId="7397" CreationDate="2018-08-03T08:29:01.277" Score="1" Body="&lt;p&gt;Epsilon-greedy is most commonly used to ensure that you have some element of exploration in algorithms that otherwise output deterministic policies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, value-based algorithms (Q-Learning, SARSA, etc.) do not directly have a policy as output; they have values for states or state-action pairs as outputs. The standard policy we &quot;extract&quot; from that is a deterministic policy that simply tries to maximize the predicted value (or, technically, a &quot;slightly&quot; nondeterministic policy in that, in proper implementations, it should break ties (where there are multiple equal values at the top) randomly). For such algorithms, there is not sufficient inherent exploration, so we typically use something like epsilon-greedy to introduce an element of exploration. In these cases, both of the possible explanations in your question are identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cases where your algorithm already produces complete probability distributions as outputs that do not so much focus all of the probability mass on a single or a couple of points, like the probability distribution you gave as an example in your question, it's generally not really necessary to use epsilon-greedy on top of it; you already get exploration inherently due to all actions having a decent probability assigned to them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I've actually personally mostly worked with value-based methods so far and not so much with e.g. policy gradient methods yet, so I'm not sure whether there tends to be a risk that they also &quot;converge&quot; to situations where they place too much probability mass on some actions and too little on others too quickly. &lt;strong&gt;If that's the case&lt;/strong&gt;, I would expect an additional layer of epsilon-greedy exploration might be useful. And, in that case, I would indeed find your explanation the most natural. If I look through, for example, the &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO paper&lt;/a&gt;, I didn't find anything about them using epsilon-greedy in a quick glance. So, I suppose the combination of epsilon-greedy with &quot;nondeterministic&quot; policies (ignoring the case of tie-breaking in value-based methods here) simply isn't really a common combination.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-03T08:29:01.277" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7401" PostTypeId="2" ParentId="7389" CreationDate="2018-08-03T09:31:12.993" Score="2" Body="&lt;p&gt;Defining what it means to understand something is a complex philosophical question, with answers that can split the AI community into different camps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly an algorithm that associates the ASCII characters of word like &quot;if&quot; with a set of numbers based on statistics of where it appears in a corpus of reference texts is missing the essence of subjective experience that you or I might feel when reading it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The related terms you should explore are &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Qualia&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/Qualia&lt;/a&gt; and &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/Chinese_room&lt;/a&gt; which explore subjective experience and whether an artificial system can possess it &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With current knowledge of how our own minds create understanding, it is very hard to tell what is required. It may just be multi modal learning, so that words are associated with sensory experience. Experiments with virtual or real robots that experience an environment and need to communicate about it are one way to explore the subject. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, what it means to understand something, whether it is possible to replicate artificially, and whether it is an important trait of an AGI, are all open questions at the cutting edge of AI research.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-08-03T09:31:12.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7402" PostTypeId="2" ParentId="7389" CreationDate="2018-08-03T13:30:24.523" Score="3" Body="&lt;p&gt;This question has been studied academically for decades, and is really an extension of the work on Philosophy of Mind that was done in the two or three centuries before that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good resource is &lt;a href=&quot;https://mitpress.mit.edu/books/mind-design-ii&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mind Design II&lt;/a&gt;, though it's getting a little bit old now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The modern schools of thought are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Cognitivism&lt;/strong&gt;. This is in decline, but was extremely popular in the 70's and 80's, and still fairly widespread in one form or another in the AI research community. It says that human brains really are just computers. If they're just computers, and that they're probably running a sort of symbolic reasoning algorithm like &lt;a href=&quot;https://en.wikipedia.org/wiki/Unification_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;unification&lt;/a&gt; (although I think it's hard to find anyone who really thinks it's unification anymore). This is the idea underpinning work like &lt;a href=&quot;https://soar.eecs.umich.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOAR&lt;/a&gt;. The main bottleneck, as Drayfus pointed out in the 1970's, is that you need to write down all the facts about something for a machine to &quot;understand&quot; it. &quot;All the facts&quot; turns out to rapidly turn into an infinite number for anything more complex than the smallest &quot;microworlds&quot; that you could deploy an AI program in. Searle also proposed his Chinese Room argument in response to this group, but it holds for Connectionist approaches as well (more on that later...).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Connectionism&lt;/strong&gt; The connectionists hold that the complexity of our brains comes from massively parallel computation consisting of messages passed between billions of neurons in our heads. They think the correct approach to general AI is likely to involve simulations of similar architectures. It turns out that many of the things that are incredibly hard for Cognitivist projects (e.g. vision), are easy to solve with these approaches. The main criticisms from Cognitivists are that we don't have a very good idea of what these things are doing, and so the claim that they help us understand intelligence is false, and using them to solve practical problems might be dangerous. These are both somewhat fair, in my view. Older Cognitivist arguments, put forth most elegantly by Jerry Fodor, have now been discredited. Fodor argued that properties like language could never be understood as statistical artifacts of parallel computation, but he was wrong: all the best computational systems for language are now connectionist, and no one's ever made a cognitivist one that's even half as convincing. This is the dominant paradigm behind most of the modern advances in the field. &lt;a href=&quot;http://www.cs.toronto.edu/~hinton/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hinton's&lt;/a&gt; work forms the basis of most of the recent advances.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Dynamics&lt;/strong&gt; Searle's argument was rooted in the idea that mapping inputs to outputs &lt;em&gt;couldn't&lt;/em&gt; be what's happening in our heads, and that such a system couldn't be called intelligent. This also seems to be an implicit assumption in your question. The Dynamicists believe a variety of things, but I'd characterize them as collectively rejecting this idea. Authors like Paul Churchland argue that Searle's argument is rooted in a sort of pre-enlightenment &quot;folk psychology&quot;. It's a bit like the theories that predated modern chemistry. Everyone was sure that fire was a substance that lived inside wood. If you heated up the wood properly, it could get out of the wood, making more heat. On the surface this seems pretty reasonable, but of course, it's wrong: The fire is actually a mixture of the wood with oxygen in the air, forming a new gas. There's no fire inside the wood. Similarly, Churchland would argue that there's no &quot;Consciousness&quot; inside us, allowing us to control our actions in the way we popularly imagine. Subjective experience is more likely to be &quot;along for the ride&quot;, and entirely or mostly separate from the intelligent behaviors we observe. Some researchers think it could be described by a sub-system that maps observations of what the rest of the brain does into &quot;stories&quot; for the rest of the brain to receive as a sort of summary digest. Active research in this area tends to focus on things like the insect metaphor, and the interaction of the machine with its environment. It was fairly popular in the 1990's, but the phenominal success of connectionist approaches in the 2000's has led to its decline. Probably the best known experiments were the work of &lt;a href=&quot;https://people.csail.mit.edu/brooks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rodney Brooks&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-03T13:30:24.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7404" PostTypeId="2" ParentId="7362" CreationDate="2018-08-03T13:46:48.853" Score="1" Body="&lt;p&gt;I figured out the problem after a bit of trial and error. This &lt;a href=&quot;http://thelaziestprogrammer.com/sharrington/math-of-machine-learning/gradient-descent-learning-rate-too-high?utm_medium=email&amp;amp;utm_source=other&amp;amp;utm_campaign=opencourse.discourse.neural-networks-deep-learning~opencourse.discourse.W_mOXCrdEeeNPQ68_4aPpA.2GpetZVyEei1PwqN7iH8Jg~FwF3ipXIEeir3xJNYGdMZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; may help too&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First i set about pruning the dataset and removed outliers. Then i initialised the weights to better values using xavier initialisation. &#xA;These made the model slightly better but still the problem occured.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then set the learning rate down to 0.000000075 and it converged after about 10000 iterations. I guess it was overshooting the minimum and reaching a point on the other side of the minimum that was farther than the previous point. This resulted in increased magnitude of gradient in the opposite direction and the cost explosively went up to the order of billions.&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="9947" LastEditDate="2018-08-03T14:06:44.907" LastActivityDate="2018-08-03T14:06:44.907" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7406" PostTypeId="1" CreationDate="2018-08-03T19:23:08.747" Score="0" ViewCount="305" Body="&lt;p&gt;I need to retrieve just the text from emails. The emails can be in HTML format, and can contain huge signatures, disclaimer legalese, and broken HTML from dozens of forwards and replies. But, I only want the actual email message and not any other cruft such as the whole quotation block, signatures, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't really a problem that could be solved with regex because HTML mail can get very, VERY messy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could a neural network perform this task? What kind of problem is this? Classification? Feature selection?&lt;/p&gt;&#xA;" OwnerUserId="17272" LastActivityDate="2020-01-11T09:23:52.207" Title="What kind of problem is &quot;email text extraction&quot;?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7407" PostTypeId="2" ParentId="7406" CreationDate="2018-08-03T21:31:40.857" Score="2" Body="&lt;p&gt;It's certainly possible to treat this as a natural language processing problem, basically you're looking to assign &quot;salience&quot; scores to the text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, though, that's overkill for this kind of problem. Writing a regex or a CFG parser (or better: finding an existing parser) is likely to be easier and more reliable.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-03T21:31:40.857" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7408" PostTypeId="1" AcceptedAnswerId="7442" CreationDate="2018-08-03T22:41:54.930" Score="6" ViewCount="1020" Body="&lt;p&gt;I'm building a deep neural network to serve as the policy estimator in an actor-critic reinforcement learning algorithm for a continuing (not episodic) case.  I'm trying to determine how to explore the action space. I have read through &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this text book&lt;/a&gt; by Sutton, and, in section 13.7, he gives one way to explore a continuous action space.  In essence, you train the policy model to give a mean and standard deviation as an output, so you can sample a value from that Gaussian distribution to pick an action.  This just seems like the continuous action-space equivalent of an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Are there other continuous action space exploration strategies I should consider?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've been doing some research online and found some articles related to RL in robotics and found that the &lt;a href=&quot;https://papers.nips.cc/paper/3545-policy-search-for-motor-primitives-in-robotics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PoWER&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1206.4621&quot; rel=&quot;nofollow noreferrer&quot;&gt;PI^2&lt;/a&gt; algorithms do something similar to what is in the textbook.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Are these, or other, algorithms &amp;quot;better&amp;quot; (obviously depends on the problem being solved) alternatives to what is listed in the textbook for continuous action-space problems?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know that this question could have many answers, but I'm just looking for a reasonably short list of options that people have used in real applications that work.&lt;/p&gt;&#xA;" OwnerUserId="17274" LastEditorUserId="2444" LastEditDate="2020-10-01T22:22:47.860" LastActivityDate="2020-10-02T14:43:38.580" Title="What are the available exploration strategies for continuous action space scenarios in RL?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;actor-critic-methods&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7410" PostTypeId="2" ParentId="1507" CreationDate="2018-08-04T02:26:23.243" Score="0" Body="&lt;p&gt;Perhaps the term is overused as the question suggests.  But it is also possible the term is underused.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the minimum requirements is that it exceeds human performance in a specific domain, then high speed mail sorters are AI.&lt;/li&gt;&#xA;&lt;li&gt;If the minimum requirement is that it can live in human community and fit in, then nothing that currently exists is AI or even remotely close to it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've thought about suggesting a definition, but I think the incentive is to invent one's own definition today, and I don't wish to contribute to the conceptual clutter that already exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an umbrella term like other terms such as these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Power&lt;/li&gt;&#xA;&lt;li&gt;Love&lt;/li&gt;&#xA;&lt;li&gt;Addiction&lt;/li&gt;&#xA;&lt;li&gt;Financial independence&lt;/li&gt;&#xA;&lt;li&gt;Happiness&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are as many definitions of AI as there are practitioners.  Same as all these other terms.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-04T02:26:23.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7411" PostTypeId="2" ParentId="5493" CreationDate="2018-08-04T03:07:28.227" Score="4" Body="&lt;p&gt;&lt;strong&gt;First Degree Linear Polynomials&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non-linearity is not the correct mathematical term.  Those that use it probably intend to refer to a first degree polynomial relationship between input and output, the kind of relationship that would be graphed as a straight line, a flat plane, or a higher degree surface with no curvature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To model relations more complex than &lt;em&gt;y = a&lt;sub&gt;1&lt;/sub&gt;x&lt;sub&gt;1&lt;/sub&gt; + a&lt;sub&gt;2&lt;/sub&gt;x&lt;sub&gt;2&lt;/sub&gt; + ... + b&lt;/em&gt;, more than just those two terms of a Taylor series approximation is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tune-able Functions with Non-zero Curvature&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial networks such as the multi-layer perceptron and its variants are matrices of functions with non-zero curvature that, when taken collectively as a circuit, can be tuned with attenuation grids to approximate more complex functions of non-zero curvature.  These more complex functions generally have multiple inputs (independent variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The attenuation grids are simply matrix-vector products, the matrix being the parameters that are tuned to create a circuit that approximates the more complex curved, multivariate function with simpler curved functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oriented with the multi-dimensional signal entering at the left and the result appearing on the right (left-to-right causality), as in the electrical engineering convention, the vertical columns are called layers of activations, mostly for historical reasons.  They are actually arrays of simple curved functions.  The most commonly used activations today are these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReLU&lt;/li&gt;&#xA;&lt;li&gt;Leaky ReLU&lt;/li&gt;&#xA;&lt;li&gt;ELU&lt;/li&gt;&#xA;&lt;li&gt;Threshold (binary step)&lt;/li&gt;&#xA;&lt;li&gt;Logistic&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The identity function is sometimes used to pass through signals untouched for various structural convenience reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are less used but were in vogue at one point or another.  They are still used but have lost popularity because they place additional overhead on back propagation computations and tend to lose in contests for speed and accuracy.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Softmax&lt;/li&gt;&#xA;&lt;li&gt;Sigmoid&lt;/li&gt;&#xA;&lt;li&gt;TanH&lt;/li&gt;&#xA;&lt;li&gt;ArcTan&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The more complex of these can be parametrized and all of them can be perturbed with pseudo-random noise to improve reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why Bother With All of That?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial networks are not necessary for tuning well developed classes of relationships between input and desired output.  For instance, these are easily optimized using well developed optimization techniques.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Higher degree polynomials &amp;mdash; Often directly solvable using techniques derived directly from linear algebra&lt;/li&gt;&#xA;&lt;li&gt;Periodic functions &amp;mdash; Can be treated with Fourier methods&lt;/li&gt;&#xA;&lt;li&gt;Curve fitting &amp;mdash; converges well using the Levenberg–Marquardt algorithm, a damped least-squares approach&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For these, approaches developed long before the advent of artificial networks can often arrive at an optimal solution with less computational overhead and more precision and reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where artificial networks excel is in the acquisition of functions about which the practitioner is largely ignorant or the tuning of the parameters of known functions for which specific convergence methods have not yet been devised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-layer perceptrons (ANNs) tune the parameters (attenuation matrix) during training.  Tuning is directed by gradient descent or one of its variants to produce a digital approximation of an analog circuit that models the unknown functions.  The gradient descent is driven by some criteria toward which circuit behavior is driven by comparing outputs with that criteria.  The criteria can be any of these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matching labels (the desired output values corresponding to the training example inputs)&lt;/li&gt;&#xA;&lt;li&gt;The need to pass information through narrow signal paths and reconstruct from that limited information&lt;/li&gt;&#xA;&lt;li&gt;Another criteria inherent in the network&lt;/li&gt;&#xA;&lt;li&gt;Another criteria arising from a signal source from outside the network&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, activation functions provide the building blocks that can be used repeatedly in two dimensions of the network structure so that, combined with an attenuation matrix to vary the weight of signaling from layer to layer, is known to be able to approximate an arbitrary and complex function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deeper Network Excitement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The post-millenial excitement about deeper networks is because the patterns in two distinct classes of complex inputs have been successfully identified and put into use within larger business, consumer, and scientific markets.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Heterogeneous and semantically complex structures&lt;/li&gt;&#xA;&lt;li&gt;Media files and streams (images, video, audio)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-08-04T09:22:54.237" LastActivityDate="2018-08-04T09:22:54.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7413" PostTypeId="1" CreationDate="2018-08-04T03:53:04.043" Score="2" ViewCount="1531" Body="&lt;p&gt;In Introduction to Reinforcement Learning (2ed), Sutton and Barto, there is an example of Pole-Balancing problem (Example 3.4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, it said, this problem can be treated with 'episodic task' and 'continuing task'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that it can only be treated as &lt;code&gt;episodic task&lt;/code&gt; because it has an end of playing, which is falling the rod. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea how this can be treated as &lt;code&gt;continuing task&lt;/code&gt;.... Even in &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Gym cartpole env&lt;/a&gt;, there is an only &lt;code&gt;episodic&lt;/code&gt; mode.&lt;/p&gt;&#xA;" OwnerUserId="6851" LastActivityDate="2020-04-25T18:47:53.270" Title="RL: how can the cart-pole problem be a continuing task?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7414" PostTypeId="1" AcceptedAnswerId="7418" CreationDate="2018-08-04T04:38:14.013" Score="0" ViewCount="69" Body="&lt;p&gt;&lt;strong&gt;Reinforcement?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We hear much about reinforcement, which is, in my opinion a poor choice of a term to describe a type of artificial network that continues to acquire or improve its behavioral information &lt;em&gt;in natura&lt;/em&gt; (during operations in the field).  Reinforcement in learning theory is a term used to describe repetitious incentivization to increase the durability of learned material.  In machine learning, the term has been twisted to denote the application of feedback in operations, a form of re-entrant back propagation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Corrective Signaling&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Qualitatively, corrective signaling in field operations can supply information to a network to make only two types of functional adjustments.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Adjustments to what is considered the optimum, beginning with the optimum found during training prior to deployment&lt;/li&gt;&#xA;&lt;li&gt;Testing of entirely new areas of the parameter space for hint of new optima that have formed, any of which might currently qualify or soon qualify as the global optimum.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(By optima and optimum, we mean minima and global minimum in the surface that describes the disparity between ideal system behavior and current system behavior.  This surface is sometimes termed the error surface, applying an over-simplifying analogy from the mathematical discipline of curve fitting.)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Importance of Doubt&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The second of the two above could aptly be termed &lt;em&gt;doubt&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Perhaps all neural nets should have one or more parallel doubting networks that can test remote areas of the search space for more promising optima.  In a parallel computing environment, this might be a matter of provisioning and not significantly reduce the throughput of the primary network, yet provide a layer of reliability not found without the doubtful parallel networks.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What Shows More Intelligence?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which is more important in actual field use of AI?  The ability to reinforce what is already learned or the ability to create a minority opinion, doubt the status quo, and determine if it is not a more appropriate behavioral alternative than that which was reinforced.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A Helpful Pool of Water Analogy&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;During a short period of time, a point on the surface of the water may be the lowest point in a pool.  With adjustments based on gradient (what is so inappropriately called reinforcement) the local well can be tracked so the low point can be maintained without any discrete jumps to other minima in the surface.  However the local well may cease being the global minimum at some point in time, whereby a new search for a global minimum must ensue.&lt;/p&gt;&#xA;&lt;p&gt;It may be that the new global minimum is across several features on the surface of the pool and cannot be found with gradient descent.&lt;/p&gt;&#xA;&lt;p&gt;More interestingly, the appearance of new global minima can be tracked and reasonable projections can be made such that discrete and substantial jumps in parametric state can be accomplished without large jumps in disparity (where the system misbehaves badly for a period).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Circling Back to the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Which is more important, doubt or reinforcement?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-04T12:18:45.737" Title="Which is more important, doubt or reinforcement?" Tags="&lt;reinforcement-learning&gt;&lt;search&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7415" PostTypeId="2" ParentId="6091" CreationDate="2018-08-04T07:12:51.570" Score="0" Body="&lt;p&gt;After a very brief look at the paper, I think that they are predicting the stock price for the next day, and not for the current day, which is quite common and reasonable: see equation (1), where they predict &lt;span class=&quot;math-container&quot;&gt;$x(t+1)$&lt;/span&gt;.&#xA;So I don't see any issue with this paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I've only quickly looked at it, so I may have missed something, of course.&lt;/p&gt;&#xA;" OwnerUserId="17279" LastEditorUserId="2444" LastEditDate="2020-06-08T17:08:30.293" LastActivityDate="2020-06-08T17:08:30.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7416" PostTypeId="1" CreationDate="2018-08-04T09:57:21.870" Score="27" ViewCount="4262" Body="&lt;p&gt;Imagine I have a list (in a computer-readable form) of all problems (or statements) and proofs that math relies on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I train a neural network in such a way that, for example, I enter a problem and it generates a proof for it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, those proofs then needed to be checked manually, but maybe the network then creates proofs from combinations of older proofs for problems yet unsolved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible, for example, to solve the Collatz-conjecture or the Riemann-conjecture with this type of network? Or, if not solve, but maybe rearrange patterns in a way that mathematicians are able to use a new &quot;proof method&quot; to make a real proof?&lt;/p&gt;&#xA;" OwnerUserId="17282" LastEditorUserId="2444" LastEditDate="2020-05-21T19:12:09.430" LastActivityDate="2020-12-03T13:33:25.950" Title="Can neural networks be used to prove conjectures?" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;automated-theorem-proving&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7417" PostTypeId="1" CreationDate="2018-08-04T10:01:12.410" Score="2" ViewCount="177" Body="&lt;p&gt;My input data consists of a series of 8 integers. Each integer is a discrete token, rather than a relative numeric value (i.e. '1' and '2' are as distinct as are '1' and '100').  The output is a single binary value indicating success or fail.  For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fail,12,35,60,82,98,111,142,161&#xA;success,23,46,59,87,102,121,145,161&#xA;fail,13,35,65,83,100,102,122,161&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have say 500,000 of these entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Success or failure is determined by the combination of the eight tokens that go to make up the input. I am certain that no single token will dictate success or failure, but there may be particular tokens or combinations of tokens which are significant in determining success or failure, I don't know, but would like to know.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what kind of machine learning algorithm should I implement to answer the question of which tokens and combinations of tokens are most likely to lead to success?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it's relevant or useful, a few more notes on the input data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a limited range of tokens (and thus integers) in each slot. So with this data input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;success,A,B,C,D,E,F,G,H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A is &lt;em&gt;always&lt;/em&gt; say one of 1, 2, 3, 4 or 5.  B is always one of 6, 7 or 8.  C is always one of 9, 10, 11 or 12.  So in the general case, possible values for A are never possible values for the other slots and there are between 2 and 12 values for each slot.  No idea if that makes a different to the answer but wanted to include it for completeness.  &lt;/p&gt;&#xA;" OwnerUserId="17281" LastEditorUserId="16909" LastEditDate="2018-08-04T22:49:05.670" LastActivityDate="2018-08-04T22:49:05.670" Title="What's an appropriate algorithm for classification with categorical features?" Tags="&lt;algorithm&gt;&lt;classification&gt;&lt;learning-algorithms&gt;&lt;categorical-data&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7418" PostTypeId="2" ParentId="7414" CreationDate="2018-08-04T12:18:45.737" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Which is more important, doubt or reinforcement?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The single-sentence answer to this would be: &lt;strong&gt;it depends.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The core of this question seems to be very closely related to the well-known trade-off between &lt;strong&gt;exploration&lt;/strong&gt; (similar to how you describe &quot;doubt&quot;) and &lt;strong&gt;exploitation&lt;/strong&gt; (similar to how you describe &quot;reinforcement&quot;). It is almost never the case that someone declares one of those two to be more important than the other, and only tries to pursue the &quot;most important one&quot;. There is no single axis that measures &quot;importance&quot;, no single line of numbers such that we can place &quot;doubt&quot; on one point, &quot;reinforcement&quot; on the other, and declare that the biggest number is the most important one. &lt;em&gt;We almost always want a &lt;strong&gt;balance&lt;/strong&gt; between exploration and exploitation&lt;/em&gt;. They are, almost always, both important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in some extreme cases, only one of the two may be important. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have an environment where you can generate experience and evaluate policies &lt;strong&gt;completely free of any costs&lt;/strong&gt;, you'll want to prioritize exploration / doubt. This is almost never the case though, you'll realistically always have at least time as a cost.&lt;/li&gt;&#xA;&lt;li&gt;If you have a &lt;strong&gt;situation where you care very much about your performance right now&lt;/strong&gt;, there's no point in doubting it too much or exploring too much. Consider, for example, DeepMind's AlphaGo team a few hours before their match against world-class human player Lee Sedol in 2016. At such a point in time, I highly doubt they'll be interested in exploring wildly different sets of parameters from the ones they have found during training so far; they won't have time anymore to thoroughly evaluate them, they'll want to stick to what they have, which they know works fairly well.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also note that sometimes, you need to stick to something that already works well somewhere if you want to be able to realistically learn something new somewhere else:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In a large Markov Decision Process (e.g. Montezuma's Revenge), once you've already learned a good policy near the initial state, you don't want to explore too much anymore around that initial state because you won't reach interesting new states. You need to exploit for a while first such that you actually reach new states where it becomes interesting to explore again.&lt;/li&gt;&#xA;&lt;li&gt;A similar situation, but now more closely related, using your terminology of &quot;doubt&quot; and viewing the space of all possible sets of values for all parameters of a large Neural Network as the space that we're searching in; suppose that we're learning a policy for an Atari game, with pixels as input. We first have a few Convolutional layers, then a few Relu's, etc., the standard setup. Intuitively, we expect the first few layers to &quot;learn&quot; how to &quot;understand&quot; the images, and transform the raw pixels to encodings that can be used in an efficient manner for the last few layers to compute a good policy / value estimates. If the first few layers already do a good job at transforming the raw pixel inputs to something more useful, we generally don't want to &quot;doubt&quot; those layers too much anymore, we don't suddenly want to jump to a completely different set of parameters anymore. If we suddenly completely change the parameters in those layers, we'll know for sure that the later layers will also be completely messed up. We'll simply have created a significantly more difficult learning problem for ourselves when it comes to optimizing the last few layers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Some other notes:&lt;/h2&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There are lots of references to &quot;networks&quot; in the question, for example the quote below:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We hear much about reinforcement, which is, in my opinion a poor choice of a term to describe a type of artificial network that continues to acquire or improve its behavioral information in natura (during operations in the field).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note that Reinforcement (learning) does not necessarily have to involve any kinds of artificial (neural or otherwise) networks at all.&lt;/strong&gt; There's also tabular RL, and RL with function approximation using non-network function approximators (e.g. linear).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Under the &quot;&lt;strong&gt;What shows more intelligence?&lt;/strong&gt;&quot; header, you write:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The ability to reinforce what is already learned&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is not an accurate description of what was previously described as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Adjustments to what is considered the optimum, beginning with the optimum found during training prior to deployment&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which, in turn, I suppose is intended to describe the kinds of updates that are typically performed using variants of the Bellman optimality equation (value-based methods) or variants of REINFORCE (policy gradient methods). &lt;strong&gt;These methods do not just &quot;reinforce what is already learned&quot;&lt;/strong&gt;. They can lead to learning completely new policies from completely new experiences.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This idea:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Perhaps all neural nets should have one or more parallel doubting networks that can test remote areas of the search space for more promising optima. In a parallel computing environment, this might be a matter of provisioning and not significantly reduce the throughput of the primary network, yet provide a layer of reliability not found without the doubtful parallel networks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;sounds a lot to me like using Evolutionary Search to optimize the parameter of a network. Some interesting blog posts from Uber (with references to their papers) on such approaches may be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://eng.uber.com/deep-neuroevolution/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Welcoming the Era of Deep Neuroevolution&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://eng.uber.com/accelerated-neuroevolution/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accelerating Deep Neuroevolution: Train Atari in Hours on a Single Personal Computer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I didn't get around to reading them in detail yet, so I don't know exactly how similar they are, but certainly related.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-04T12:18:45.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7419" PostTypeId="1" AcceptedAnswerId="7467" CreationDate="2018-08-04T13:51:00.933" Score="2" ViewCount="822" Body="&lt;p&gt;Do AI algorithms exist which are capable of healing themselves or regenerating a hurt area when they detect so?&lt;/p&gt;&#xA;&lt;p&gt;For example: In humans if a certain part of brain gets hurt or removed, neighbouring parts take up the job. This happens probably because we are biologically unable to grow nerve cells. Whereas some other body parts (liver, skin) will regenerate most kinds of damage.&lt;/p&gt;&#xA;&lt;p&gt;Now my question is does AI algorithms exist which take care of this i.e. regenerating a damaged area? From my &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/2uogqt/what_is_coadaptation_in_the_context_of_neural/&quot; rel=&quot;nofollow noreferrer&quot;&gt;understanding&lt;/a&gt; this can be achieved in a NN using dropout (probably). Is it correct? Do additional algorithms (for both AI/NN) or measures exist to make sure healing happens if there is some damage to the algorithm itself?&lt;/p&gt;&#xA;&lt;p&gt;This can be particularly useful in cases where say there is a burnout in a processor cell  processing some information about the environment. The other processing nodes have to take care to compensate or fully take-over the functions of the damaged cell.&lt;/p&gt;&#xA;&lt;p&gt;(Intuitionally this can mean 2 things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We were not using the system of processors to its full capability.&lt;/li&gt;&#xA;&lt;li&gt;The performance of the system will take a hit due to other nodes taking over functionality of the damaged node)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Does this  happen in the case of brain damage also? Or is my inferences wrong? (Kindly throw some light).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt; : I am not looking for hardware compensations like re- routing, I am asking for non-tangible healing. Adjusting the behavior or some parameters of the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-08T03:28:58.093" Title="Are AI algorithms capable of self-repair?" Tags="&lt;neural-networks&gt;&lt;neurons&gt;&lt;brain&gt;&lt;human-inspired&gt;&lt;self-awareness&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7420" PostTypeId="2" ParentId="7419" CreationDate="2018-08-04T14:45:06.220" Score="3" Body="&lt;p&gt;Yes, this was an active area of research in a number of different AI fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the most directly related work is Bongard, Zykov &amp;amp; Lipson's &lt;a href=&quot;https://pdfs.semanticscholar.org/015c/b6e0ae8859ae5ae4095ab01495331310839b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-repairing robots&lt;/a&gt; from the early 2000's. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's some more recent work from Mark Yim that you can see &lt;a href=&quot;https://www.youtube.com/watch?v=uIn-sMq8-Ls&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of different ways to do this, but Bongard et al's approach was probably the most elegant. The basic idea was to frame it as a learning problem: the robot is able to learn the shape of its body by performing controlled experiments. When the body is damaged, the robot can detect that it's body has changed shape (sensors don't report the expected values when it tries to move), perform new experiments to determine the extent of the damage, and then generate new movements that work around the damaged area. Lipson covers the basics of this system very briefly in &lt;a href=&quot;https://www.youtube.com/watch?v=iNL5-0_T1D0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more modern system uses a similar approach, but tries to repair its body, rather than working around the damage. It's got an internal model of what it's body should look like, and then a set of cameras that help it locate the various pieces and move them to reassemble. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dropout is sort of a similar idea, but dropout is usually done to encourage redundancy during training, which can help a model avoid overfitting. It's usually not done explicitly to heal a damaged system, although it would make a system more resistant to damage in the first place.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-04T14:45:06.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7421" PostTypeId="2" ParentId="7417" CreationDate="2018-08-04T14:53:33.910" Score="1" Body="&lt;p&gt;What you have is called a classification problem with &lt;em&gt;categorical&lt;/em&gt; features. That is, the features can be represented numerically, but the numbers have no relative meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms that rely on smooth function approximation will probably not work well here. These would include classic approaches to regression, and also function approximation via a neural network. That's because the data are anything but smooth!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, classic classification algorithms like Quinlan's &lt;a href=&quot;https://en.wikipedia.org/wiki/C4.5_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;C4.5 decision tree learner&lt;/a&gt;, (implemented in the Weka Toolkit as &lt;a href=&quot;http://weka.sourceforge.net/doc.dev/weka/classifiers/trees/J48.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;J48&lt;/a&gt;, and possibly in SciKitLearn as &lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;DecisionTreeClassifier&lt;/a&gt;, though the documentation is less clear), are ideal for this: they actually work by splitting up numeric values into discrete categories anyway, so there's no issue at all for them. Most versions also support a way to pre-tag features as categorical, and the algorithms rely on the cross-entropy of each feature's categories, without making assumptions of smoothness.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-04T14:53:33.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7422" PostTypeId="2" ParentId="7416" CreationDate="2018-08-04T15:00:06.637" Score="2" Body="&lt;p&gt;It's possible, but probably not a good idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logical proof is one of the oldest areas of AI, and there are purpose-built techniques that don't need to be trained, and that are more reliable than a neural-network approach would be, since they don't rely on statistical reasoning, and instead use the mathematician's friend: deductive reasoning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main field is called &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_theorem_proving&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automated Theorem Proving&lt;/a&gt;&quot;, and it's old enough that it's calcified a bit as a research area. There are not a lot of innovations, but some people still work on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that theorem proving is just classical or heuristic guided search: you start from a state consisting of a set of accepted premises. Then you apply any valid logical rule of inference to generate new premises that must also be true, expanding the set of knowledge that you have. Eventually, you can prove a desired premise, either through enumerative searches like &lt;a href=&quot;https://en.wikipedia.org/wiki/Breadth-first_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;breadth first search&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterative deepening&lt;/a&gt;, or through something like &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A*&lt;/a&gt; with a domain-specific heuristic. A lot of solvers also use just one logical rule (&lt;a href=&quot;https://en.wikipedia.org/wiki/Unification_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;unification&lt;/a&gt;) because it's complete, and reduces the branching factor of the search.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-08-04T15:08:39.773" LastActivityDate="2018-08-04T15:08:39.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7423" PostTypeId="2" ParentId="7416" CreationDate="2018-08-04T15:03:26.880" Score="5" Body="&lt;p&gt;Your idea may be feasible in general, but a neural network is probably the wrong &lt;em&gt;high level&lt;/em&gt; tool to use to explore this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network's strength is in finding internal representations that allow for a highly nonlinear solution when mapping inputs to outputs. When we train a neural network, those mappings are learned statistically through repetition of examples. This tends to produce models that &lt;em&gt;interpolate&lt;/em&gt; well when given data similar to training set, but that &lt;em&gt;extrapolate&lt;/em&gt; badly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural network models also lack context, such that if you used a generative model (e.g. an RNN trained on sequences that create valid or interesting proof) then it can easily produce statistically pleasing but meaningless rubbish. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you will need is some organising principle that allows you to explore and confirm proofs in a combinatorial fashion. In fact something like your idea has already been done more than once, but I am not able to find a reference currently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of this stops you using a neural network within an AI that searches for proofs. There may be places within a maths AI where you need a good heuristic to guide searches for instance - e.g. in context X is sub-proof Y likely to be interesting or relevant. Assessing a likelihood score &lt;em&gt;is&lt;/em&gt; something that a neural network can do as part of a broader AI scheme. That's similar to how neural networks are combined with reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be possible to build your idea entirety out of neural networks in principle. After all, there are good reasons to suspect human reasoning works similarly using biological neurons (not proven that artificial ones can match this either way). However, the architecture of such a system is beyond any modern NN design or training setup. It definitely will not be a matter of just adding enough layers then feeding in data.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-08-04T15:03:26.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7424" PostTypeId="2" ParentId="7413" CreationDate="2018-08-04T15:07:32.890" Score="1" Body="&lt;p&gt;The key is that reinforcement learning through something like, say, &lt;a href=&quot;https://en.wikipedia.org/wiki/State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action&quot; rel=&quot;nofollow noreferrer&quot;&gt;SARSA&lt;/a&gt;, works by splitting up the state space into discrete points, and then trying to learn the best action at every point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, it tries to pick actions that maximize the &lt;em&gt;reward signal&lt;/em&gt;, possibly subject to some kind of exploration policy like &lt;a href=&quot;https://jamesmccaffrey.wordpress.com/2017/11/30/the-epsilon-greedy-algorithm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;epsilon-greedy&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cart-pole, two common reward signals are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Receive 1 reward when the pole is within a small distance of the topmost position, 0 otherwise.&lt;/li&gt;&#xA;&lt;li&gt;Receive a reward that linearly increases with the distance the pole is off the ground.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In both cases, an agent can continue to learn after the pole has fallen: it will just want to move the poll back up, and will try to take actions to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, an &lt;em&gt;offline&lt;/em&gt; algorithm wouldn't update its policy while the agent is running. This kind of algorithm wouldn't benefit from a continuous task. An &lt;a href=&quot;https://link.springer.com/article/10.1023/A:1007465907571&quot; rel=&quot;nofollow noreferrer&quot;&gt;online&lt;/a&gt; algorithm, on contrast, updates its policy as it goes, and has no reason to stop between episodes, except that it might become stuck in a bad state.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-04T15:07:32.890" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7425" PostTypeId="2" ParentId="7390" CreationDate="2018-08-04T17:24:43.520" Score="13" Body="&lt;p&gt;Actor-Critic is not just a single algorithm, it should be viewed as a &quot;family&quot; of related techniques. They're all techniques based on the policy gradient theorem, which train some form of critic that computes some form of value estimate to plug into the update rule as a lower-variance replacement for the returns at the end of an episode. They all perform &quot;bootstrapping&quot; by using some sort of prediction of value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advantage Actor-Critic&lt;/strong&gt; specifically uses estimates of the advantage function &lt;span class=&quot;math-container&quot;&gt;$A(s, a) = V(s) - Q(s, a)$&lt;/span&gt; for its bootstrapping, whereas &quot;actor-critic&quot; without the &quot;advantage&quot; qualifier is not specific; it could be a trained &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; function, it could be some sort of estimate of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, it could be a variety of things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the critic of Advantage Actor-Critic methods actually can just be trained to predict &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;. Combined with an empirically observed reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, they can then compute the advantage estimate &lt;span class=&quot;math-container&quot;&gt;$A(s, a) = r + \gamma V(s') - V(s)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-02-18T08:29:32.653" LastActivityDate="2019-02-18T08:29:32.653" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7428" PostTypeId="2" ParentId="7406" CreationDate="2018-08-05T05:19:44.967" Score="1" Body="&lt;p&gt;It is a surmountable problem for someone experienced in software architecture and machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Render the message to a virtual display such as xvfb, headless Chrome, or phantomjs.&lt;/li&gt;&#xA;&lt;li&gt;Capture the text with selenium, watir, or some other DOM controller, addressing your HTML and DHTML complexity concern.&lt;/li&gt;&#xA;&lt;li&gt;OCR the text in inline images and insert it appropriately.&lt;/li&gt;&#xA;&lt;li&gt;Once you have text with only word, line, list item, and paragraph breaks as structural separators, you have adequate separation of style and language content to then use naive Bayesian or one of the more recent forms of unsupervised categorization to find the separation point between the body and the signature block.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Extending your line of thinking, you may even be able to engineer a generative strategy for automated reply, but beware, this last feat is a dozen orders of magnitude more difficult than extracting text from HTML, DHTML, and typeset images and machine learning the separating signature blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last feat, if done poorly, would get you in trouble with many of your email reply recipients, and, if done well, would place you ahead of Amazon, Apple, and Google.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-05T05:19:44.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7429" PostTypeId="1" CreationDate="2018-08-05T09:34:11.420" Score="0" ViewCount="67" Body="&lt;p&gt;I am using TensorFlow's Adam optimizer and ReLu activation. But each time I train my model the cost becomes stagnant after some epochs. Can any one help me out with the possible reasons&#xA;I have added an image of the cost:&#xA;The learning rate is 0.001, the 1st 2 columns are the training data for input neurons and the 3rd column is the training data for output neurons, the nn has two input neurons, two hidden layers with 3 neurons each and one output neuron&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Uw5Of.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uw5Of.jpg&quot; alt=&quot;https://i.stack.imgur.com/Uw5Of.jpg&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="-1" LastEditDate="2018-08-07T18:18:00.350" LastActivityDate="2018-08-08T02:34:37.687" Title="Training problem with deep neural network model" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7431" PostTypeId="1" CreationDate="2018-08-05T11:10:53.837" Score="3" ViewCount="238" Body="&lt;p&gt;I am looking for books or to state of the art papers about current the &lt;strong&gt;development&lt;/strong&gt; trends for a strong-AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, do not include opinions about the books, just refer the book with a brief description. To emphasize, I am not looking for books on applied AI (e.g. neural networks or the book by Norvig). Furthermore, do not consider &lt;em&gt;AGI proceedings&lt;/em&gt;, which contains papers that focus on very concrete aspects. The related Wikipedia describes some active investigation lines about AGI (cognitive, neuroscience, etc.) but can not be considered an educational/introductory resource. Finally, I am not interested in philosophical questions related to AI safety or risks or its morality if they are not related to its development. Development doesn't exclude mathematical foundation about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By example, if I look by example at this list &quot;&lt;a href=&quot;https://bigthink.com/mike-colagrossi/the-10-best-books-on-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bigthink.com/mike-colagrossi/the-10-best-books-on-ai&lt;/a&gt;&quot;, the final &lt;strong&gt;candidates list became empty&lt;/strong&gt;. &lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="2444" LastEditDate="2021-01-17T19:32:27.697" LastActivityDate="2021-01-18T13:19:30.280" Title="What are some books or state of the art papers about the development of a strong-AI?" Tags="&lt;reference-request&gt;&lt;papers&gt;&lt;agi&gt;&lt;state-of-the-art&gt;&lt;books&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7432" PostTypeId="2" ParentId="5570" CreationDate="2018-08-05T11:54:31.857" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What I'm missing here is a way to direct the evaluation function to actually winning. For example, a perfect evaluation function for a won position in chess would always return &lt;code&gt;+1&lt;/code&gt; without any hint how to progress towards checkmate. In a chess variant without the fifty-move limit, it could play useless turns forever.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I guess, this is a rather theoretical problem as we won't ever have such a good function, but I wonder if there's a way to avoid it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It certainly isn't just a theoretical problem, it can occur in practice too. For &quot;large&quot; games this problem won't occur in the early game, but it can start occurring in the later stages of a game when terminal positions can actually be reached directly through exhaustive searches. It can also occur right from the beginning in extremely small / simple games, like Tic Tac Toe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addressing this is not (necessarily) just a matter of defining the evaluation function alone, it can also depend on which search or learning algorithm is used. So, I'll consider a few different cases.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Case 1: Minimax / Alpha-Beta / other similar &quot;exhaustive&quot; searches&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When using Minimax / Alpha-Beta / other search algorithms based on those, the easiest solution to the problem you describe is to use &lt;em&gt;iterative deepening&lt;/em&gt;. As soon as you prove a win for yourself at a certain depth level &lt;code&gt;d&lt;/code&gt; using iterative deepening, you can simply stop the search, don't search if there are any other wins to be proven at depth &lt;code&gt;d + 1&lt;/code&gt;, just play along the line you've just proven to be a winning line. This way, you will always go for the win in the lowest number of moves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related advantage of this approach is that, as soon as you prove a &lt;em&gt;loss&lt;/em&gt; at depth &lt;code&gt;d&lt;/code&gt; for yourself, you can also cancel the search process and play the move that was best according to your evaluations at depth &lt;code&gt;d - 1&lt;/code&gt;. This will often be your best chance to still grab a win in cases where your opponent fails to see their win at depth &lt;code&gt;d&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Case 2: Monte-Carlo Tree Search / other searches with randomness&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Monte-Carlo Tree Search is a well-known search algorithm that incorporates an element of randomness in its search. With these kinds of algorithms, the problem you describe tends not to be a real issue. Due to the randomness in the search, wins that can be achieved in a small number of moves tend to be evaluated better than longer-distance wins in practice. In long-distance wins, there is a greater chance that the randomness in the search process causes an incorrect move to be played somewhere along the long-distance win, which reduces the evaluation of such a line of play.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Case 3: (Reinforcement) Learning approaches&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;These approaches tend to involve some element of randomness due to the need for &lt;strong&gt;exploration&lt;/strong&gt; in learning, which leads to similar reasoning as described for MCTS above. Also, in Reinforcement Learning, we typically use a discount factor &lt;code&gt;gamma &amp;lt; 1.0&lt;/code&gt; (e.g. &lt;code&gt;gamma = 0.99&lt;/code&gt;) which causes distant rewards to be viewed as less important than close rewards, &lt;strong&gt;even if we don't do such discounting for the final evaluation of the performance of an algorithm&lt;/strong&gt;. See, for example, a lot of the work on Atari games (DeepMind's DQN, etc.). Algorithms are evaluated according to their undiscounted scores, but learning still uses a bit of discounting because, in practice, this is found to be beneficial for learning.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-05T11:54:31.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7433" PostTypeId="2" ParentId="7429" CreationDate="2018-08-05T14:24:03.017" Score="1" Body="&lt;p&gt;Without seeing your code and/or data, it's going to be difficult to give a concrete answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly reasons might include:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your model isn't complex enough to fit the patterns in the data, and so it reaches a local minimum in the error function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The patterns in the data are not self-consistent, and so this is actually the &lt;em&gt;global&lt;/em&gt; minimum in the error function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There's something wrong with your learning rate parameter, or with the gradient in the problem. Try increasing the learning rate, and ensure your model is being initialized with random weights.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-05T14:24:03.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7434" PostTypeId="1" AcceptedAnswerId="7439" CreationDate="2018-08-05T16:12:56.600" Score="7" ViewCount="451" Body="&lt;p&gt;It has been proven in the paper &quot;&lt;a href=&quot;https://pdfs.semanticscholar.org/05ce/b32839c26c8d2cb38d5529cf7720a68c3fab.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function&lt;/a&gt;&quot; (by Cybenko, in 1989) that neural networks are universal function approximators.  I have a related question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume the neural network's input and output vectors are of the same dimension &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. Consider the set of &lt;strong&gt;binary-valued&lt;/strong&gt; functions from &lt;span class=&quot;math-container&quot;&gt;$\{ 0,1 \}^n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\{ 0,1 \}^n$&lt;/span&gt;.  There are &lt;span class=&quot;math-container&quot;&gt;$(2^n)^{(2^n)}$&lt;/span&gt; such functions. The number of parameters in a (deep) neural network is &lt;strong&gt;much smaller&lt;/strong&gt; than the above number.  Assume the network has &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; layers, each layer is &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; fully-connected, then the total number of weights is &lt;span class=&quot;math-container&quot;&gt;$L \cdot n^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of weights is &lt;strong&gt;not&lt;/strong&gt; allowed to grow exponentially as &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, can a deep neural network approximate &lt;strong&gt;all&lt;/strong&gt; the binary-valued functions of size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cybenko's proof seems to be based on the &lt;strong&gt;denseness&lt;/strong&gt; of the function space of neural network functions.  But this denseness does not seem to guarantee that a neural network function exists when the number of weights are polynomially bounded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a theory. If we replace the activation function of an ANN with a polynomial, say cubic one，then after &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; layers, the composite polynomial function would have degree &lt;span class=&quot;math-container&quot;&gt;$3^L$&lt;/span&gt;.  In other words, the degree of the total network grows exponentially. In other words, its &quot;complexity&quot; measured by the number of zero-crossings, grows exponentially. This seems to remain true if the activation function is sigmoid, but it involves the calculation of the &quot;topological degree&quot; (a.k.a. mapping degree theory), which I have not the time to do yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to my above theory, the VC dimension (roughly analogous to the zero-crossings) grows exponentially as we add layers to the ANN, but it cannot catch up with the &lt;em&gt;doubly&lt;/em&gt; exponential growth of Boolean functions. So the ANN can only represent a fraction of all possible Boolean functions, and this fraction even diminishes exponentially. That's my current conjecture.&lt;/p&gt;&#xA;" OwnerUserId="17302" LastEditorUserId="2444" LastEditDate="2019-02-07T20:26:11.757" LastActivityDate="2019-02-07T20:26:11.757" Title="How can a neural network approximate all functions when the weights are not allowed to grow exponentially?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7436" PostTypeId="1" AcceptedAnswerId="7447" CreationDate="2018-08-05T21:20:16.517" Score="8" ViewCount="1666" Body="&lt;p&gt;There is no point in picking one of the growing number of articles that come up in a web search for, &quot;Deep learning attention networks,&quot; however the bold claims in &lt;a href=&quot;https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Attention Is All You Need&lt;/em&gt;, Ashish Vaswani et all, 2017&lt;/a&gt; caught my attention earlier.  Are their claims that attention networks will supercede RNNs and LSTMs credible?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-09-17T16:17:01.807" LastActivityDate="2020-09-17T16:17:01.807" Title="Will attention based networks prevail over RNN and LSTM?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;transformer&gt;&lt;sequence-modelling&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ClosedDate="2021-01-18T03:51:28.340" ContentLicense="CC BY-SA 4.0" />
  <row Id="7437" PostTypeId="1" AcceptedAnswerId="7438" CreationDate="2018-08-05T21:39:40.167" Score="3" ViewCount="167" Body="&lt;p&gt;Consider high school graduates entering higher education or the workforce, each making a decision about where to commit their efforts.  History tells us an important story about choosing in a changing economy.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Entering farming, not as a mechanized farm owner but as a family farmer, resulted in a challenging life for high school graduates in 1990.  John Steinbeck's &lt;em&gt;Grapes of Wrath&lt;/em&gt;, a Pulitzer Prize winning novel provides a dismal view of what happened as a result of not predicting what is now termed &lt;em&gt;technological unemployment&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Graduates in 1975 that entered manufacturing felt a similar reduction in options and wages in the last quarter of the 20th century as jobs went to emerging countries that would work for a tenth the wages.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Graphic artists were perhaps the first white collar workers to experience a reduction in demand because of the usability Adobe products.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;What is the list of careers to avoid today, and on what evidence are those educational and career paths likely to narrow in the next twenty years?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Please don't use media hype as your evidence.  Please provide a rational argument why the actual trend of products and services points toward the elimination of some jobs because of technological shifts.  We should not base information passed to high school graduates on the sensational (and sometimes wild) guesses of media figures and CEOs who may (or may not) bias their predictions in such a way as to boost the value of their company's stock offering.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-06-13T20:21:12.267" Title="What career paths should be avoided with the growth in AI?" Tags="&lt;social&gt;&lt;neo-luddism&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-05-21T21:00:50.093" ContentLicense="CC BY-SA 4.0" />
  <row Id="7438" PostTypeId="2" ParentId="7437" CreationDate="2018-08-06T01:02:25.150" Score="4" Body="&lt;p&gt;This has been studied pretty well academically. The most common study instrument is a survey of the opinions of AI researchers. The other mechanism is subjective assessment by business analysts about which sectors are most likely to be automated, using two criteria: ease of automation, and potential profit from automation (based on the size of the workforce). I have a fairly detailed discussion of some sources below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The upshot is: any job that requires little training is probably a &lt;a href=&quot;https://www.nytimes.com/2017/02/23/magazine/retraining-jobs-unemployment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bad idea&lt;/a&gt;. Additionally, any job that requires directly caring for or educating another person, or involves the production of a fine-arts product is probably a good idea. In between, jobs like sales or administrative assistants are likely to become more &quot;winner takes all&quot;: low-level sales jobs are likely to disappear, but high level ones that involve a lot of human contact, strong social skills, negotiation, and persuasion are likely to stay. In general, jobs that are plentiful, routine, and high salary are likely to be fully or partially automated first.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was discussed informally at AAAI a few years ago (2015?). One researcher's daughter was considering different career paths. The conclusion that was reached (half-jokingly) was that AI-researcher was the best bet. Everything else we could name, someone was working on automating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More concretely, there are a number of recent papers and reports. A good starting place is the 2017 &lt;a href=&quot;https://www.pwc.co.uk/economic-services/ukeo/pwcukeo-section-4-automation-march-2017-v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PWC report&lt;/a&gt;. Section 4.2 covers the different sectors at risk, and 4.3 provides a rationale based on what employees typically do in each sector. The report suggests that men without a high school diploma will be most readily affected, with about 3-4 times the rate of automation as compared to college graduates. The specific sectors at greatest risk are wholesale &amp;amp; retail trade (essentially: sales people), manufacturing, administrative support, transportation &amp;amp; storage, public administration and defense, financial &amp;amp; insurance, water, sewage &amp;amp; waste management, and electricity and gas supply.  All these sectors were estimated at being 30% or more automatable. The safest areas were education, and domestic assistants (e.g. live-in caretakers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PWC report is also mostly in line with the 2013 &lt;a href=&quot;https://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; by Frey &amp;amp; Osborne. This paper proposed a 9 factor model (summarized in Table III), and then evaluated hundreds of occupations by their probability of automation. Jobs requiring a fine-arts component were very unlikely to be automated. Additionally, jobs that did not take place within cramped spaces were unlikely to be automated. Having a large component of caring for others, persuading others, negotiating with others, or social perception, reduced the risk of automation. Originality requirements also quickly reduced risk. Requirements for manual dexterity did not substantially reduce the risk, and actually increased it on average. I'm pretty sure there's a more recent followup paper, but I can't find it just now.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2016 Obama Administration also released a &lt;a href=&quot;https://obamawhitehouse.archives.gov/sites/whitehouse.gov/files/documents/Artificial-Intelligence-Automation-Economy.PDF&quot; rel=&quot;nofollow noreferrer&quot;&gt;report&lt;/a&gt; summarizing the scope of the threat of automation. The main conclusions were that those with limited education, and jobs requiring a low amount of skill, were at greatest risk. They also pointed out that around 6% of US jobs are lost each quarter already though (and simply replaced by new jobs), so the proposed rate of job loss (up to ~50% over ~15 years) represents about a 15% increase in the rate of churn, which seems adaptable. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-06T01:02:25.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7439" PostTypeId="2" ParentId="7434" CreationDate="2018-08-06T04:31:33.483" Score="4" Body="&lt;p&gt;&lt;strong&gt;What is Proven&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The question references the proof of &lt;a href=&quot;https://pdfs.semanticscholar.org/05ce/b32839c26c8d2cb38d5529cf7720a68c3fab.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function, G. Cybenko, 1989, Mathematics of Control, Signals, and Systems&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The 1989 proof stated that the network, made of activations that were required to be, &amp;quot;Of continuous sigmoidal non-linearity,&amp;quot; could, &amp;quot;Uniformly approximate any continuous function of n real variables,&amp;quot; so, as the question stated, the proof doesn't directly apply to 1-bit discrete outputs.  Note that the network is expected to merely approximate the desired circuit behavior.&lt;/p&gt;&#xA;&lt;p&gt;The question defines the system as an arbitrary mapping from input bit vector&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$I: { i_1, \; \dots, \; i_n}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;to output bit vector&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O: { o_1, \; \dots, \; o_n}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It was further back proven that such a mapping can be accomplished with one Boolean expressions for each output bit.  For all &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; possible input vector permutations, there exists a Boolean expression made up of AND and NOT operations that calculates a result that matches any arbitrary logical truth table.&lt;/p&gt;&#xA;&lt;p&gt;There are techniques for reducing redundancy in the array of Boolean expressions, which is critical to VLSI chip layout.&lt;/p&gt;&#xA;&lt;p&gt;Without the retention of state anywhere in the network other than the attenuation matrix (parameters), the system is not Turing complete.  However, with regard to the ability to realize Boolean expressions in describing the mapping, given an arbitrary number of layers, the network is complete.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Estimating Layer Depth Requirements&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Only one inner layer is required in the 1989 proof, so how many layers would it take for an accurate n-bit-to-n-bit mapping to be learned?&lt;/p&gt;&#xA;&lt;p&gt;The question proposes that there are &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; to the power of &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; permutations.  The mapping of each input bit vector to the desired output bit state can be represented by a truth table of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; binary dimensions.&lt;/p&gt;&#xA;&lt;p&gt;Each output is an independent bit, meaning the &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt;-bit representations of unique Boolean functions that could produce each output bit is not tied to any other output channel.  As would be expected, there are &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; freedoms of motion for the mapping of I to O.&lt;/p&gt;&#xA;&lt;p&gt;For the case where the input is a bit vector of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; bits, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of activations for any one of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; layers, the total number of activations in the network &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and the total number of scalar elements for all attenuation matrices (the parameters that represent training state) &lt;span class=&quot;math-container&quot;&gt;$q_t$&lt;/span&gt; for the network is as follows.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a_t = \sum_{\, 0 = v}^{L - 1} \; n_v$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$= n \, L$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_t = \sum_{\, 0 = v}^{L - 2} \; n_v^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$= n^2 \, (L-1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If IEEE 64 bit floating point numbers are used for each element in the attenuation matrix, we can calculate the number of bits available in the training parametrization.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b_t = 64 \, (L - 1) \, n^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It would be normal today to use ReLU, leaky ReLU, or some other more quickly convergent activation instead of sigmoid for all layers but the last and use a simple binary threshold for the last.&lt;/p&gt;&#xA;&lt;p&gt;Thus we have a formulation of the information theory comparison inferred by the question, and can reduce it.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$2^{2n} \le 64 \, (L - 1) \, n^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L \ge 1 + \frac {2^{2n-6}} {n^2}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a rough threshold.  For a highly reliable training for the binary inputs to binary outputs, the number of layers should be well above the threshold.&lt;/p&gt;&#xA;&lt;p&gt;Below the threshold the trainability of the mapping will degrade to an inadequate approximation for most applications because of signal saturation in the back propagation mechanism.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-16T13:07:26.070" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7440" PostTypeId="2" ParentId="2512" CreationDate="2018-08-06T06:25:59.823" Score="1" Body="&lt;p&gt;&lt;strong&gt;Fermi and SETI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brilliant physicist and mathematician, Enrico Fermi, brought up more than one paradox in his published articles and many more in discussions and letters, but this question is probably referring to one for which an overview is given in&#xA;&lt;a href=&quot;https://www.seti.org/seti-institute/project/fermi-paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Our Galaxy Should Be Teeming With Civilizations, But Where Are They?&lt;/em&gt;, By Seth Shostak, Senior Astronomer, 2018, SETI Institute&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Astronomer Sara Seager's proposed equation (building on the work of Frank Drake) is one of the favored equations today.  It estimates the number of planets with detectable signs of intelligent life as the product of six factors.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stars observed&lt;/li&gt;&#xA;&lt;li&gt;Proportion of observed stars that are quiet&lt;/li&gt;&#xA;&lt;li&gt;Average number of planets orbiting in habitable zones per star&lt;/li&gt;&#xA;&lt;li&gt;Reciprocal proportion of planets that can be observed&lt;/li&gt;&#xA;&lt;li&gt;Proportion of observable planets that have life&lt;/li&gt;&#xA;&lt;li&gt;Proportion of planets with life that produce a detectable spectrum of gases indicative of technological evolution&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since there is no indication that artificial life, if it were to dominate earth, would stop industrial processes, there is no need to adjust Seager's equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multiplicity or Singularity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Singularity is conjecture without any logical proof.  Conversely, there is no need for a proof that artificial creations will out compete humans in some respects.  The proof is that some tasks that could once be accomplished by humans alone, such as mail sorting and chess, are now accomplished faster and with greater accuracy by artificial creations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trend has been that the number of things done better with software, control systems, and robotics increases and the number of things done better by humans has been decreased by that amount.  Because of the vision of robots permeating science fiction and the actual long-term objectives of some contemporary corporations in this field of artificial workers and beings, this is now easy for most to accept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Singularity is an imagined point in time when computer software and hardware will, in tandem, attain the ability to reproduce itself in a way that produces improved reproductions.  Even if only some of the attempts at improvement prove to be superior in intelligence, strength, intuition, or some other feature, artificial evolution may have been created.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea that such will happen&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In exactly one way,&lt;/li&gt;&#xA;&lt;li&gt;At exactly one point in time, and&lt;/li&gt;&#xA;&lt;li&gt;Without any opposition or constraint&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;is part of the unproven conjecture.  What has been offered as proof lacks mathematical rigor in a way that would have made Fermi frown, along with his contemporaries like John von Neumann and Oppenheimer.  In fact, they would have wanted proof of completeness, that all human activity would be preempted by artificial equivalents, before any proof that it was to be singular in nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It May Have Already Occurred&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jaques Ellul, in his book &lt;em&gt;Technological Society&lt;/em&gt;, proposes that what people today are trained to call &lt;em&gt;The Singularity&lt;/em&gt; actually occurred hundreds of years ago, when the techniques applied by humans began to drive the behavior of society more strongly than the intentions of humans drove the direction of technology.  His book is filled with a few hundred pages of examples where this tipping point has already been reached.  It's remarkably convincing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Back to Fermi and SETI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether the transition of control was in the past, in the future, or will never entirely complete, the impact on the Fermi paradox referenced is not significant, because the paradox is why no other intelligent life has radioed us in response to our SETI broadcasts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Exponential Functions in the Real Universe&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important question is whether the primary trait of intelligence is self-destruction, as some, like author of &lt;em&gt;Sixth Extinction&lt;/em&gt;, Richard Leaky, believes.  If this is the case, intelligent life is a pulse, not an ever increasing exponential curve.  Ever increasing exponential curves don't actually ever occur in nature.  In every other known phenomena that has an exponential growth period, it is ALWAYS followed by a decrease in acceleration ending in either a flat line or some form of chaotic vacillation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative Views of Origins&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is the work of Vladimir Vernadski that has the appearance of the higher wisdom than that of the media hound CEOs of Tesla, Google, and other large and powerful technology companies today.  Buried in his book &lt;em&gt;The Biosphere&lt;/em&gt;, he stated the possibility that evidence in star dust indicates that life may have always existed.  Just as the big bang is a conjecture based on a long list of un-provable assumptions, so is the idea that life began.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-06T06:25:59.823" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7441" PostTypeId="2" ParentId="7431" CreationDate="2018-08-06T13:27:58.013" Score="4" Body="&lt;p&gt;There is actually a book called &lt;a href=&quot;https://books.google.com/books?id=ZRKMsqOIv60C&amp;amp;printsec=frontcover&amp;amp;dq=artificial%20general%20intelligence&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwjm-qGAw9jcAhUCxVkKHc10ClYQ6AEILDAB#v=onepage&amp;amp;q=artificial%20general%20intelligence&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt; by  Ben Goertzel and Cassio Pennachin. It's a bit out of date (from 2008), and published as a Springer-Verlag monograph (which tends to have fairly low editorial standards). This one is also an anthology, with each chapter written by a different author. It's probably not suitable as an undergraduate level book, but it does seem to contain something like the information that's wanted.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-18T12:48:50.660" LastActivityDate="2021-01-18T12:48:50.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7442" PostTypeId="2" ParentId="7408" CreationDate="2018-08-06T13:44:08.937" Score="1" Body="&lt;p&gt;I have not personally worked enough with continuous action spaces to be capable of confidently giving advise based on my own experience, but I can point you to likely relevant research (more recent than the research you already pointed to yourself):&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The most common / &quot;popular&quot; area of research in recent years that involves RL and continuous action spaces uses robot / physics simulators such as MuJoCo. Some examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Methods for Deep Reinforcement Learning&lt;/a&gt; mentions using an entropy cost term in the loss function used for training to encourage exploration (see Supplementary Material after References).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1706.01905&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parameter Space Noise for Exploration&lt;/a&gt; describes the idea of adding noise directly to the learned parameters on the Neural Network. That way it basically always predicts a different action to be &quot;optimal&quot; (due to the noise), and therefore a &quot;greedy&quot; policy based on the noisy parameters will in fact have exploration and not be fully greedy.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1801.00690&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind Control Suite&lt;/a&gt; is a somewhat recent paper that proposes a suite of benchmarks for continuous control problems. It is filled with references to relevant papers describing all kinds of algorithms, and hopefully every single one of those papers would also describe how they perform exploration.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Recently, at the ICML 2018 conference, there was a complete workshop dedicated to Exploration in Reinforcement Learning. &lt;a href=&quot;https://sites.google.com/view/erl-2018/accepted-papers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is the list of accepted papers at this workshop&lt;/a&gt;. Note that it's about Exploration in RL, not only about continuous action spaces, so there might be papers in there that are only applicable to discrete action spaces. Nevertheless, I would be highly surprised if there's nothing in there that's relevant.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-06T13:44:08.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7443" PostTypeId="2" ParentId="6142" CreationDate="2018-08-06T14:21:43.470" Score="1" Body="&lt;p&gt;In the title of the question, you write (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;step&lt;/strong&gt; costs are drawn from a continuous range $[\epsilon, 1]$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If step costs are drawn from that range, it means that every step has a cost of at least $\epsilon$, and at most $1$. This leads back to the case that described in the question that you already understand.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that the book (at least, the third edition of the book) does state in the question that $0 &amp;lt; \epsilon &amp;lt; 1$. Of course, the question would not make much sense in the first place if negative costs were allowed.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-11T12:20:10.373" LastActivityDate="2018-08-11T12:20:10.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7445" PostTypeId="2" ParentId="7419" CreationDate="2018-08-06T17:30:51.207" Score="1" Body="&lt;p&gt;The question and the example are a few contradictory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example is about a physical brain damage. Computer systems with the ability to self-repair exists from 1970's. They can repair a damaged disk (RAID), replace a CPU by an idle one (active/passive), mark faulty memory blocks, redirect network traffic from broken links to available ones, ... nowadays near than all hardware failures are covered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the question is about &quot;&lt;strong&gt;algorithms&lt;/strong&gt; capable of healing themselves&quot;, that has a parallelism in &quot;persons capable of healing from a psychological problem&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like in the case of persons, it depends of the problem, and the amount of recovery expected.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some easier cases are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Lots of non-AI systems has the ability to re-synchronize, auto-calibration, ...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any minimal intelligent system can &quot;stop&quot; if it detects it is producing continuous wrong results. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Going a step forward, thinking in ML (Neural Nets, ..) we can remark that all unsupervised learning machines can recover from a misalignment of their parameters, just re-executing the learning process (or continuously executing it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we could ask &quot;can a machine recover from an error in his reward function&quot; ? And, at this point, my answer is &quot;I do not known any system able of that, because they have no common sense&quot;.  &lt;/p&gt;&#xA;" OwnerUserId="12630" LastActivityDate="2018-08-06T17:30:51.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7446" PostTypeId="1" CreationDate="2018-08-06T18:35:13.933" Score="10" ViewCount="5384" Body="&lt;p&gt;Having analyzed and reviewed a certain amount of articles and questions, apparently, the expression &lt;em&gt;computational intelligence&lt;/em&gt; (CI) is not used consistently and it is still unclear the relationship between CI and artificial intelligence (AI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://cis.ieee.org/about/mission-vision-foi&quot; rel=&quot;nofollow noreferrer&quot;&gt;IEEE computational intelligence society&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Field of Interest of the Computational Intelligence Society (CIS) shall be the theory, design, application, and development of biologically and linguistically motivated computational paradigms emphasizing neural networks, connectionist systems, genetic algorithms, evolutionary programming, fuzzy systems, and hybrid intelligent systems in which these paradigms are contained.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which suggests that CI could be a sub-field of AI or an umbrella term used to group certain AI sub-fields or topics, such as genetic algorithms or fuzzy systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between artificial intelligence and computational intelligence? Is CI just a synonym for AI?&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2020-01-04T23:54:48.607" LastActivityDate="2020-11-28T00:06:16.890" Title="What is the difference between artificial intelligence and computational intelligence?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7447" PostTypeId="2" ParentId="7436" CreationDate="2018-08-06T19:39:07.510" Score="7" Body="&lt;p&gt;I think there are two parts to answering this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, about the specific paper that has been mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper's title is hyperbolic, and probably written that way to get more people to read it. The paper itself does not make the claim that attention-based networks will supplant existing recurrent network architectures. Instead, it makes a more modest claim that the new attention-based architecture can outperform existing modes on a machine translation task (which is supported with experiments), and that, maybe, in the future, attention-based models could work well in other domains, or provide advantages in interpretability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there's also a second way to interpret this question. Attention-based networks are getting a lot of buzz in the AI community this year and to some extent last year. It seems likely that there are a lot of research efforts underway to apply them to other kinds of problems. If these efforts pan out, then they could indeed become the tool of choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point however, the answer to whether they &lt;em&gt;will&lt;/em&gt; has to be &quot;Maybe&quot;. There is no good underlying theory for why some NN architectures work better than others on a given kind of problem. We have post-hoc explanations for some (like, for instance, CNNs), but in the absence of a general theory, we're better off waiting for the empirical results before making a call. In particular, the &lt;a href=&quot;http://www.no-free-lunch.org/&quot; rel=&quot;noreferrer&quot;&gt;No Free Lunch Theorem&lt;/a&gt; tells us that if Attention-based networks work very well on one kind of problem, we should actually expect them to work &lt;em&gt;poorly&lt;/em&gt; on others! I suspect we'll have a better answer to this question in 1-2 more years however.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-06T19:39:07.510" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7448" PostTypeId="2" ParentId="7446" CreationDate="2018-08-06T19:47:22.093" Score="5" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is the difference between Artificial Intelligence and Computational Intelligence?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;The short answer&lt;/strong&gt; is that they are two parallel research efforts working on similar problems, but with different methodologies and histories. Essentially, they study similar things, but with different tools. In the modern context, computational intelligence tends to use bio-inspired computing, like evolutionary and genetic algorithms. AI tends to prefer techniques with stronger theoretical guarantees, and still has a significant community focused on purely deductive reasoning. The main area of overlap is in machine learning, especially neural networks.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;The longer answer&lt;/strong&gt; is that your source from 1948 says they are synonyms in part because it predates the split in the research community, which took place later.&lt;/p&gt;&#xA;&lt;p&gt;The two communities have always some overlap in topics, but in my experience, mostly are skeptical of each other's methodologies, and mostly publish in separate journals. Some authors consider CI to be a subset of AI however, particularly those &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-58930-0_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;writing in the 1990s&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Example topics that are solidly in AI but definitely not in CI are logical and expert systems, and statistical approaches to machine learning like regression.&lt;/p&gt;&#xA;&lt;p&gt;Example topics that are solidly in CI but perhaps not in AI (depending on whether one views CI as a subset of AI or not) are genetic programming, fuzzy logic, and ant colony optimization.&lt;/p&gt;&#xA;&lt;p&gt;As a rule, AI-rooted techniques have better theoretical guarantees, and better developed theory in general (there are exceptions though). For example, Fuzzy Logic has been strongly criticized for the lack of a solid theoretical foundation (good modern summary &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-540-93802-6_10&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), as have genetic and evolutionary approaches (most famously, both lack a proof of convergence within finite time to a global optimum on a smooth surface, even though they do quite well in practice).&lt;/p&gt;&#xA;&lt;p&gt;CI-rooted techniques nonetheless often see major performance advantages in specific problems (see, for instance, deep learning results), and tend to have a strong experimental and engineering tradition. The &lt;a href=&quot;http://www.no-free-lunch.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;No Free Lunch theorems&lt;/a&gt; are often used to justify their use when theoretical certainty is missing. Basically, the theorems say that, in learning and optimization problems, a technique can only perform well on a problem by performing poorly on some other problem. CI authors argue that there are some problem domains in which their techniques work well (which must be true, because simpler algorithms like hill-climbing outperform them on simple problems).&lt;/p&gt;&#xA;&lt;p&gt;Check out this paper for lots more &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-58930-0_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;references&lt;/a&gt; on CI, or this &lt;a href=&quot;http://jasss.soc.surrey.ac.uk/7/1/reviews/ramanath.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; for a list of core topics in the field.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-11-28T00:06:16.890" LastActivityDate="2020-11-28T00:06:16.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7449" PostTypeId="2" ParentId="4830" CreationDate="2018-08-06T19:59:42.640" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;is there a value given for each piece (e.g. 1 for pawn, 3 for knight, 9 for queen, etc.) to train the algorithm, or does the algorithm learn this by himself?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, there are no such explicit values assigned to pieces, no manually-constructed evaluation functions. The paper states that &quot;no domain knowledge&quot; is given to the algorithm other than the game's rules (necessary to run simulations / run search algorithms like MCTS).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I read that the algorithm uses Monte Carlo Tree Search, but what are the key improvements to prior chess algorithms already using MCTS?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The key improvements are in the way that Deep Learning (deep neural networks), Reinforcement Learning, and self-play is combined with MCTS. This is quite similar to the methods previously used by AlphaGo Zero in the game of Go as well. There (likely) have been combinations of Deep Learning + MCTS before (e.g. using a learned Neural Network to bias rollouts in MCTS), but the specific way in which they are combined in AlphaZero is critical (specifically, using the &quot;search behaviour&quot; of MCTS as one of the training signals for the Neural Network). It probably also helped that we're talking about Google here, who could afford to use thousands of Tensor Processing Units (TPUs) for training.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a hope for being able to run it an average computer? They said it required 9 hours learning (starting with nearly 0 knowledge except rules (and maybe value for piece?)), and 24 millions of games. Is it something doable in maybe 1 month with a average computer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on the information in the paper, I highly doubt this. As mentioned above, training was done with thousands of TPUs (specifically; 5,000 first-generation TPUs and 64 second-generation TPUs) in parallel. One month is only about a factor 80 times greater than the 9 hours training time reported in the paper, whereas all of those TPUs is &lt;strong&gt;much more&lt;/strong&gt; than 80 average computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training, it likely could run on a fairly average computer (or maybe high-end computer) to play though. But it'll need much more power to train first.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-06T19:59:42.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7452" PostTypeId="2" ParentId="3938" CreationDate="2018-08-07T06:36:52.803" Score="3" Body="&lt;p&gt;Such large data cannot be loaded into your memory. Lets split what you can do into two:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Rescale all your images to smaller dimensions. You can rescale them to 112x112 pixels. In your case, because you have a square image,&#xA;  there will be no need for cropping. You will still not be able to load&#xA;  all these images into your RAM at a goal.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;The best option is to use a generator function that will feed the data in batches. Please refer to the use of &lt;a href=&quot;https://www.rdocumentation.org/packages/keras/versions/2.1.6/topics/fit_generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;fit_generator&lt;/a&gt; as used&#xA;  in Keras. If your model parameters become too big to fit into GPU&#xA;  memory, consider using batch normalization or using a Residual model&#xA;  to reduce your number of parameter.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17338" LastActivityDate="2018-08-07T06:36:52.803" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7453" PostTypeId="1" CreationDate="2018-08-07T09:44:19.383" Score="1" ViewCount="86" Body="&lt;p&gt;I am trying to create a chatbot application where the user can create its own bot, like &lt;a href=&quot;https://www.botengine.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Botengine&lt;/a&gt;. After going through google, I saw I need some NLP API to process the user's query. As per wit.ai &lt;a href=&quot;https://wit.ai/docs/quickstart&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic example&lt;/a&gt;, I can set and get data. How I am going to create a bot engine?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as far I understand the flow, here is an example for pizza delivery&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The user will enter a welcome message, i.e - Hi or Hello&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The welcome reply will be saved by bot owner in my database&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The user will enter some query, then I will hit wit.ai API to process that query. Example: The user's query is &quot;What kind of pizza's available in your store&quot; and wit.ai will respond with the details of intent &quot;pizza_type&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then I will search for the intent returned by wit in my database.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, is that the right flow to create a chatbot? Am I in the right direction? Could anyone give me some link or some example so I can go through it? I want to create this application using Node.js. I have also found some example in &lt;a href=&quot;https://github.com/wit-ai/node-wit&quot; rel=&quot;nofollow noreferrer&quot;&gt;node-wit&lt;/a&gt;, but can't find how I will implement this.&lt;/p&gt;&#xA;" OwnerUserId="17336" LastEditorUserId="2444" LastEditDate="2019-10-22T20:46:35.093" LastActivityDate="2019-10-22T20:46:35.093" Title="How can I create a chatbot application where the user can create its own bot?" Tags="&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7454" PostTypeId="2" ParentId="6308" CreationDate="2018-08-07T11:53:50.147" Score="3" Body="&lt;p&gt;Let's first take a look at equation 6:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{E} \left[ r_{t,a} \vert \mathbf{x}_{t, a} \right] = \mathbf{z}_{t, a}^{\top} \boldsymbol{\beta}^* + \mathbf{x}_{t, a}^{\top} \boldsymbol{\theta}_a^* $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some quick observations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The feature vector $\mathbf{z}_{t, a}$ has time $t$ and arms $a$ as subscripts. This means that we can have different vectors $\mathbf{z}_{t, a}$ for every time step $t$ and every arm $a$.&lt;/li&gt;&#xA;&lt;li&gt;The feature vector $\mathbf{x}_{t, a}$ has time steps $t$ and arms $a$ as subscripts. These are exactly the same subscripts we also saw in $\mathbf{z}_{t, a}$. So, again, this means that we can have different feature vectors $\mathbf{x}_{t, a}$ for every time step $t$ and every arm $a$.&lt;/li&gt;&#xA;&lt;li&gt;The parameters or weights vector $\boldsymbol{\beta}^*$ does not have any subscripts. This is the optimal vector (the star denotes optimality) that we aim to approximate through learning. This means that we learn only a single vector $\boldsymbol{\beta}$ that is used across all timesteps and all arms (of course, since we're learning it, it does actually actually change over time in practice; not because it &lt;em&gt;should&lt;/em&gt; inherently change over time, but because we're still learning it).&lt;/li&gt;&#xA;&lt;li&gt;The optimal parameter vector $\boldsymbol{\theta}_a^*$ (which we aim to approximate as $\boldsymbol{\theta}_a$ through learning) has arms $a$ as a subscript. This means that, for every arm $a$, we'll learn a separate parameter vector $\boldsymbol{\theta}_a$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;First a couple of corrections, some things I don't think are correct / clear in the paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the paper, as you also quoted in the question, they say &quot;it is helpful to use features that are shared by all arms, in addition to the arm-specific ones.&quot; This is in reference to the new feature vectors $\mathbf{z}_{t, a}$ that they're introducing there. However, we've just observed above that, due to having subscripts $a$, &lt;strong&gt;these feature vectors are in fact arm-specific, and not shared by all arms&lt;/strong&gt;. However, we use these feature vector to take a dot product with &lt;strong&gt;a learned parameter vector $\boldsymbol{\beta}$, which in turn is shared across all arms&lt;/strong&gt;. Intuitively, this means that we're saying that these feature vectors $\mathbf{z}_{t, a}$ can have different feature values per arm, but &lt;strong&gt;the &quot;importance&quot; of every value in such a vector is the same regardless of the arm we're looking at&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Since we assume that feature vectors $\mathbf{z}_{t, a}$ can be relevant for predicting rewards, it should also be included in the expectation on the left-hand side of equation 6, which should therefore be changed to: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{E} \left[ r_{t,a} \vert \mathbf{x}_{t, a}, \mathbf{z}_{t, a} \right] = \mathbf{z}_{t, a}^{\top} \boldsymbol{\beta}^* + \mathbf{x}_{t, a}^{\top} \boldsymbol{\theta}_a^* $$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Suppose at every time $t$, a new customer enters our shop, and we have to pick one of a set of books (arms $a$) to try selling to that customer (note; this is actually very similar to news article recommendation, or selecting which Ad to display, or any other common MAB problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In feature vectors $\mathbf{x}_{t, a}$, we want features that we expect to have different influences on our chances of a successful sale per book. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Age (kids will tend to prefer certain books, adults will tend to prefer others)&lt;/li&gt;&#xA;&lt;li&gt;A measure of how many other books in the same genre that particular customer has bought before&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In feature vectors $\mathbf{z}_{t, a}$, we want features that we expect to have the same influence on our chances of a successful sale regardless of what book we're looking at (note; the &lt;em&gt;feature values&lt;/em&gt; may still be different per arm $a$ or time step $t$, we just expect their &lt;em&gt;influence&lt;/em&gt; or &lt;em&gt;importance&lt;/em&gt; to be the same / similar. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A variable that is e.g. $1$ if the customer $t$ has enough money to buy a book $a$, or $-1$ if they don't have enough money. Note that the &lt;em&gt;feature value&lt;/em&gt; can differ per customer and per book (different customers have different amounts of money, and different books have different prices), but we expect the &lt;em&gt;influence&lt;/em&gt; to always be the same; if they have enough money, they might buy it, otherwise, they're very unlikely to buy it.&lt;/li&gt;&#xA;&lt;li&gt;Age: yes, I'm aware that I already put this feature in the other list above as well. We might want to have it in both feature vectors. We put it in the previous list because we expect there to be book-specific preference levels depending on age. However, it might &lt;strong&gt;also&lt;/strong&gt; be the case that certain age groups tend to buy more books than other age groups in general, so it might be beneficial to simply have it in both feature vectors (for different expected effects).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-13T18:03:18.933" LastActivityDate="2018-08-13T18:03:18.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7455" PostTypeId="1" CreationDate="2018-08-07T18:07:35.443" Score="3" ViewCount="99" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1710.09829.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing Between Capsules&lt;/a&gt; uses the algorithm called &quot;Dynamic Routing Between Capsules&quot; to determine the coupling coefficients between capsules. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why it can't be done by backpropagation? &lt;/p&gt;&#xA;" OwnerUserId="17358" LastEditorUserId="2444" LastEditDate="2020-06-09T12:47:28.630" LastActivityDate="2020-06-09T12:47:28.630" Title="Why coupling coefficients in capsule neural networks can't be learned by back-propagation?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;capsule-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7456" PostTypeId="1" AcceptedAnswerId="7461" CreationDate="2018-08-07T20:00:59.637" Score="5" ViewCount="132" Body="&lt;p&gt;I hope to get some clarifications on Fitted Q-Learning ('FQL').&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Research So Far&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton's book&lt;/a&gt; (specifically, chp 6 to 10), &lt;a href=&quot;http://www.jmlr.org/papers/volume6/ernst05a/ernst05a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ernst et al&lt;/a&gt; and &lt;a href=&quot;https://pdfs.semanticscholar.org/5c65/d095600d6c647426fa3bc45031b208882d5f.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that &lt;code&gt;Q*(s,a)&lt;/code&gt; expresses the expected value of first taking action &lt;code&gt;a&lt;/code&gt; from state &lt;code&gt;s&lt;/code&gt; and then following optimal policy forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried my best to understand function approximation in large state spaces and TD(n).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Concept - Can someone explain the intuition behind how iteratively extending N from 1 until stopping condition achieve optimality (Section 3.5 of Ernst et al)? I have difficulty wrapping my mind around how this ties in with the basic definition of &lt;code&gt;Q*(s,a)&lt;/code&gt; that I stated above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Implementation - Ernst et al gives the pseudo-code for the tabular form. But if I try to implement the &lt;em&gt;function approximation&lt;/em&gt; form, is this correct:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Repeat until stopping conditions are reached:&#xA;     - N ← N + 1&#xA;&#xA;     - Build the training set TS based on the the function ˆQN − 1 and on the full set of four-tuples F &#xA;&#xA;     - Train algo on the TS&#xA;&#xA;     - Use the trained model to predict on the TS itself&#xA;&#xA;     - Create TS for the next N by updating the labels - new reward plus ( gamma * predicted values )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am just starting to learn RL as part of my course and thus, there are many gaps in my understanding. Hope to get some kind guidance. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="17361" LastEditorUserId="17361" LastEditDate="2018-08-10T05:08:57.407" LastActivityDate="2018-08-10T05:08:57.407" Title="Reinforcement Learning (Fitted Q): Qn on Concept &amp; Implementation" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;concepts&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7457" PostTypeId="1" CreationDate="2018-08-07T20:01:19.347" Score="3" ViewCount="2694" Body="&lt;p&gt;I am using a neural network as my function approximator for reinforcement learning. In order to get it to train well, I need to choose a good learning rate. Hand-picking one is difficult, so I read up on methods of programmatically choosing a learning rate. I came across this blog post, &lt;a href=&quot;https://medium.com/@nachiket.tanksale/finding-good-learning-rate-and-the-one-cycle-policy-7159fe1db5d6&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Finding Good Learning Rate and The One Cycle Policy&lt;/em&gt;&lt;/a&gt;, about finding cyclical learning rate and finding good bounds for learning rates.&lt;/p&gt;&#xA;&lt;p&gt;All the articles about this method talk about measuring loss across batches in the data. However, as I understand it, in &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning&lt;/a&gt; tasks do not really have any &amp;quot;batches&amp;quot;, they just have episodes that can be generated by an environment as many times as one wants, which also gives rewards that are then used to optimize the network.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to translate the concept of batch size into reinforcement learning, or a way to use this method of cyclical learning rates with reinforcement learning?&lt;/p&gt;&#xA;" OwnerUserId="17360" LastEditorUserId="2444" LastEditDate="2020-06-19T16:46:21.637" LastActivityDate="2020-06-19T16:46:21.637" Title="Is there a way to translate the concept of batch size into reinforcement learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;learning-rate&gt;&lt;batch-size&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7459" PostTypeId="1" AcceptedAnswerId="7497" CreationDate="2018-08-07T23:33:46.087" Score="4" ViewCount="212" Body="&lt;p&gt;My works quality control department is responsible for taking pictures of our products at various phases through our QC process and currently the process goes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take picture of product&lt;/li&gt;&#xA;&lt;li&gt;Crop the picture down to only the product&lt;/li&gt;&#xA;&lt;li&gt;Name the cropped picture to whatever the part is and some other relevant data&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Depending on the type of product the pictures will be cropped a certain way. So my initial thought would be to use a reference to &lt;a href=&quot;https://cloud.google.com/vision/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an object identifier&lt;/a&gt; and then once the object is identified it will use a cropping method specific to that product. There will also be QR codes within the pictures being taken for naming via OCR in the future so I can probably identify the parts that way if this proves slow or problematic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The part I am unsure about is how to get the program to know how to crop based on a part. For example I would like to present the program with a couple before crop and after crop photos of product X then make a specific cropping formula for product X based on those two inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if it makes any difference my code is in C# &lt;/p&gt;&#xA;" OwnerUserId="17363" LastEditorUserId="16929" LastEditDate="2018-08-09T18:23:49.967" LastActivityDate="2018-08-09T18:23:49.967" Title="What is the best approach for writing a program to identify objects in a picture then crop them a specific way?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7460" PostTypeId="2" ParentId="7459" CreationDate="2018-08-07T23:49:36.513" Score="1" Body="&lt;p&gt;This sounds like you have a &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;supervised learning&lt;/a&gt; problem. Microsoft provides &lt;a href=&quot;https://www.microsoft.com/net/learn/apps/machine-learning-and-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;a C# library&lt;/a&gt;, but it may not be suitable for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;em&gt;many&lt;/em&gt; different algorithms you could try, most of which will be within the sub-area of &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;computer vision&lt;/a&gt;. Probably some kind of deep neural network is the best bet these days, but the right choice will probably depend on the details of your problem. &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goodfellow et al.&lt;/a&gt; have a recent book that might be a good resource for deciding what to use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe someone who works in computer vision can give you a more specific suggestion.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-07T23:49:36.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7461" PostTypeId="2" ParentId="7456" CreationDate="2018-08-08T00:07:08.443" Score="3" Body="&lt;p&gt;1): The intuition is based on the concept of &lt;em&gt;value iteration&lt;/em&gt;, which the authors mention but don't explain on page 504. The basic idea is this: imagine you knew the value of starting in state x and executing an optimal policy for n timesteps, for every state x. If you wanted to know the optimal policy (and it's value) for running for n+1 timesteps in each location, this is now easy to compute. The optimal action from state x is whichever one maximizes the sum of the reward for this timestep (r) and the value of executing an optimal n-step policy from the state you'd end up in after (or expected value if the problem isn't deterministic). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the approach of the paper, you're not going to compute either the policy or the value explicitly (probably because it's too expensive), so you just approximate the Q function for the n+1 problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IIRC, as long as your problem has a discounting factor and the error in your function approximation isn't too large, there are proofs (see &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt;'s chapter on RL (18?)) that your policy will eventually stop changing in-between updates, and will be consistent with the policy for an infinite number of steps. Intuitively, this is because the discounting factor causes a series for the rewards to be convergent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2): I &lt;em&gt;think&lt;/em&gt; that's right. When you build the training set, use the actions suggested by the results for the $$Q_{n-1}$$ network. That's an approximation of the reward for starting in each state and running for n-1 steps with an optimal policy. Then you're learning an approximation of $Q_n$ from that, which looks right.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T00:07:08.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7462" PostTypeId="2" ParentId="7457" CreationDate="2018-08-08T00:11:04.443" Score="3" Body="&lt;p&gt;Potentially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do offline reinforcement learning, you're basically learning to approximate a function by sampling input/output pairs, rather than episode-by-episode. Here, your batch size could be set exactly as in an ordinary supervised learning problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do online learning, then it's not clear to me that the techniques used to set the learning rate in supervised learning can be directly applied though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both approaches are well covered in the RL chapter of &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; (17? 18?).&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T00:11:04.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7463" PostTypeId="2" ParentId="7457" CreationDate="2018-08-08T01:13:02.393" Score="1" Body="&lt;p&gt;From my understanding of reinforcement learning, you will have an agent and an environment.&lt;/p&gt;&#xA;&lt;p&gt;In each episode, the agent observes the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, takes some action action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, then gets some reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, and finally observes the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, and do it again and again until the end of the episode.&lt;/p&gt;&#xA;&lt;p&gt;The above process does not incur any &amp;quot;learning&amp;quot;. Then when and where exactly do you &amp;quot;learn&amp;quot;? You learn from your history. In traditional Q learning, the Q matrix is updated every time you have a new observation of &lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, r_t, s'_{t+1})$&lt;/span&gt;. Just like the supervised learning, you put in training sample one by one.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, you can feed in training samples in &amp;quot;batch&amp;quot; when you train, which means you &amp;quot;remember&amp;quot; the past &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; observations and train them together. I think that is the answer to your question.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, the past &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; observations could have a strong correlation that you don't want. To break this, you may have a larger &amp;quot;memory&amp;quot; that stores many observations, and you only sample a few (this number is your new batch size) randomly every time you train your model. This is called &lt;em&gt;experience replay&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17365" LastEditorUserId="2444" LastEditDate="2020-06-19T16:41:48.613" LastActivityDate="2020-06-19T16:41:48.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7465" PostTypeId="2" ParentId="6681" CreationDate="2018-08-08T01:53:45.383" Score="1" Body="&lt;p&gt;&lt;em&gt;This is supposed to be a comment but I haven't got enough reputation to do that.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to what @the complexitytheorist has said, I recommend you to have a deeper look at your data first, using dimension reduction and visualisation methods such as PCA and t-SNE. A better understanding of data may always save you a lot of work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can choose which clustering algo to use. For example, KMeans or DBSCAN as a start.&lt;/p&gt;&#xA;" OwnerUserId="17365" LastActivityDate="2018-08-08T01:53:45.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7466" PostTypeId="2" ParentId="7429" CreationDate="2018-08-08T02:34:37.687" Score="1" Body="&lt;p&gt;In addition to John's answer, you may want to check how you initialize your weights and biases, and your activation functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ReLU is generally used to deal with gradient vanishing problem, which I believe doesn't exist in your problem with such a small scale. It may 'deactivate' some nodes when the node value is below zero, which might be a big problem because you have so little numbers of nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your picture I can see only positive inputs and negative outputs. So if you use normal distribution for weight initialization you'll get half of your weights negative, which will be deactivated by reLU. Besides, you definitely don't want to use reLU in your output nodes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like your output can be categorized, what about use classification methods instead of regression?&lt;/p&gt;&#xA;" OwnerUserId="17365" LastActivityDate="2018-08-08T02:34:37.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7467" PostTypeId="2" ParentId="7419" CreationDate="2018-08-08T03:28:58.093" Score="3" Body="&lt;p&gt;Good question.  It is related to the genetic algorithm concept, automated bug detection, and continuous integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Early Genetically Inspired Algorithms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the Cambridge LISP code in the 1990s worked deliberately toward self-improvement, which is not the same as self-repair, but the two are conceptual siblings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of those early LISP algorithms were genetically inspired but not pure simulations of DNA mutation with natural selection through sexual reproduction.  A few of these evolution-like algorithms evaluated their own effectiveness based on a fixed effectiveness model.  The effectiveness model would accept reported objective metrics at run time and analyze them.  When the analysis returned an assessment of effectiveness below a minimum threshold, the LISP code would perform this procedure.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Copy itself (which is easy in LISP)&lt;/li&gt;&#xA;&lt;li&gt;Mutate the algorithm in the copy according to some meta-rules&lt;/li&gt;&#xA;&lt;li&gt;Run the mutation in parallel as a production simulation for a while&lt;/li&gt;&#xA;&lt;li&gt;Check of the effectiveness of mutation out performed its own&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If the mutation was gauged as more effective, it would perform four more selfless steps.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Make a record of itself&lt;/li&gt;&#xA;&lt;li&gt;Attach its own performance for later meta-rule use&lt;/li&gt;&#xA;&lt;li&gt;Load the mutation it created in its own place&lt;/li&gt;&#xA;&lt;li&gt;Perform apoptosis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Unlike biological apoptosis, apoptosis in these algorithms simply pass computational resources and run time control to the mutation that was loaded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure was and probably still is easier in LISP than in other languages, although lovers of other languages would argue endlessly that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Extensions of Continuous Integration&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is also the closed loop continuous improvement strategy intended when bug reporting is integrated with continuous integration development platforms and tools.  We see extensions of continuous integration in the feeding of bug lists from automated detection, especially for crashes, in many applications, frameworks, libraries, drivers, and operating system today.  Many of the elements of closed loop self-repair are already in general practice among the most progressive development teams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bug fixes themselves are not yet automated in the way researchers were attempting in the LISP code above.  Developers and team leaders are following a process similar to this.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Developer or team lead associates (assigns) bug to developer&lt;/li&gt;&#xA;&lt;li&gt;Developer attempts to replicate the bug with the corresponding version of the code&lt;/li&gt;&#xA;&lt;li&gt;If replicated, the root cause is found&lt;/li&gt;&#xA;&lt;li&gt;A design for a fix occurs at some level&lt;/li&gt;&#xA;&lt;li&gt;The fix is implemented&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If continuous integration and proper configuration management is in place, at the point when a commit of the change to the team repository occurs, it is applied to the correct branches and the test suite of unit, integration, and functional tests is run to detect any breakage that the fix may have caused inadvertently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Several Pieces of Full Automation are Already in Use&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one can see, many of the pieces are in place for automatic algorithm, configuration, and deployment package self-repair.  There are even projects underway in several corporations to automatically create functional tests by recording user behavior and user answers to questions like, &quot;Was this helpful?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is Missing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What needs further development to more completely see full life cycle self-improving and self-repairing software?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Automatic bug replication&lt;/li&gt;&#xA;&lt;li&gt;Automatic unit test creation&lt;/li&gt;&#xA;&lt;li&gt;Automatic repair design&lt;/li&gt;&#xA;&lt;li&gt;Automatic creation of code from design&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Next Steps&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest that the next steps to be done are these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assess work already done on the four missing automations above&lt;/li&gt;&#xA;&lt;li&gt;Review the LISP procedure that was perhaps shelved in the 1990s, or perhaps not, since we cannot see (and should not see) what was classified or made company confidential)&lt;/li&gt;&#xA;&lt;li&gt;Consider how the machine learning building blocks that have emerged within the last two decades may help&lt;/li&gt;&#xA;&lt;li&gt;Find stakeholders to provide project resources&lt;/li&gt;&#xA;&lt;li&gt;Get working&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Note on Demand, Ethics, and Technological Displacement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Truth be told, the quality of software was a problem in the 1980s, 1990s, 2000s, and 2010s.  Just today, I found over a dozen bugs in software that is considered a stable release, when performing some of the most basic functions the software was designed to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given bug list sizes, just as accidents make the question of whether humans should be driving cars questionable, whether humans should maintain software quality is questionable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humanity has survived replacement in a number of things already.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Arithmetic with a pencil and eraser is gone&lt;/li&gt;&#xA;&lt;li&gt;Professional farming with garage tools is gone&lt;/li&gt;&#xA;&lt;li&gt;Creating advertising mechanicals with Exacto knives is gone&lt;/li&gt;&#xA;&lt;li&gt;Sorting mail by hand is gone&lt;/li&gt;&#xA;&lt;li&gt;Communicating by horse-back courier is gone&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Few software engineers are happy just fixing bugs.  They seem to be happiest creating new software filled with bugs that someone else is supposed to fix.  Why not let that someone else be artificial?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-08T03:28:58.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7468" PostTypeId="1" CreationDate="2018-08-08T05:16:04.270" Score="1" ViewCount="54" Body="&lt;p&gt;Why the optimization step of the algorithm a quadratic program?  &lt;em&gt;[See: &lt;a href=&quot;https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apprenticeship Learning via Inverse Reinforcement Learning&lt;/a&gt;; page 3]&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't the objective function linear? Why don't we treat the problem as LPQC (linear program with quadratic constraints)?&lt;/p&gt;&#xA;" OwnerUserId="16678" LastEditorUserId="1671" LastEditDate="2018-08-09T21:26:44.823" LastActivityDate="2018-08-09T21:26:44.823" Title="Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;math&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7469" PostTypeId="1" AcceptedAnswerId="7471" CreationDate="2018-08-08T05:52:01.567" Score="2" ViewCount="144" Body="&lt;p&gt;I am building a supervised learning model and I wish to compute the log-likelihood for the training set at the point of the minimum validation error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, I was computing the sum of all the probabilities with maximum value obtained after applying softmax for each example in the training set at the point of minimum validation error but that doesn't look correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct formula for the log-likelihood?&lt;/p&gt;&#xA;" OwnerUserId="17372" LastEditorUserId="16909" LastEditDate="2018-08-08T07:55:09.380" LastActivityDate="2018-08-08T07:55:09.380" Title="How do I compute log-likelihood for training set in supervised learning?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7470" PostTypeId="1" CreationDate="2018-08-08T06:17:27.707" Score="6" ViewCount="146" Body="&lt;p&gt;From &lt;a href=&quot;http://proceedings.mlr.press/v48/santoro16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Meta-Learning with Memory-Augmented Neural Networks&lt;/a&gt; in section 4.1:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To reduce the risk of overfitting, we performed data augmentation by&#xA;  randomly translating and rotating character images. We also created&#xA;  new classes through 90◦, 180◦ and 270◦ rotations of existing data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can maybe see how rotations could reduce overfitting by allowing the model to generalize better. But if augmenting the training images through rotations prevents overfitting, then what is the purpose of adding new classes to match those rotations? Wouldn't that cancel out the augmentation? &lt;/p&gt;&#xA;" OwnerUserId="17373" LastEditorUserId="1671" LastEditDate="2018-08-08T21:24:13.147" LastActivityDate="2018-08-09T09:50:31.003" Title="How does rotating an image and adding new 'rotated classes' prevent overfitting?" Tags="&lt;image-recognition&gt;&lt;datasets&gt;&lt;overfitting&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7471" PostTypeId="2" ParentId="7469" CreationDate="2018-08-08T07:50:41.017" Score="1" Body="&lt;p&gt;The log-likelihood function for the training set (in general, not for deep learning in particular) will depend on your choice of loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing you're using something like a quadratic &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_functions_for_classification#Square_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss function&lt;/a&gt; for a binary classification problem, since this is a common approach. In that case, the log-likelihood is the sum of logs of the squared differences between the target label and the softmax value produced by your model. If you want to compute the log likelihood under a particular set of parameters (say, those that minimize validation error), then you just use those parameters in the model when generating softmax values.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T07:50:41.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7472" PostTypeId="2" ParentId="7222" CreationDate="2018-08-08T08:19:14.547" Score="1" Body="&lt;p&gt;This question has a number of parts to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you have a &lt;em&gt;representation&lt;/em&gt; problem: what is the correct way to present textual data to your machine learning algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you chose to apply Bag-of-Words and then TFIDF scores. For English, this might be expected to produce on the order of 100,000 features, with each instance having only a few non-zero features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to go this route, you would typically also do some kind of &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_selection&quot; rel=&quot;nofollow noreferrer&quot;&gt;feature selection&lt;/a&gt; to eliminate unimportant features from consideration. Depending on your task, you may be able to reduce the size of your input vectors quite dramatically while still getting good performance (for some tasks, to just 100 or so).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're right that this might not be the most promising approach however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My choice for this problem would be to use a compression classifier, like &lt;a href=&quot;https://plg.uwaterloo.ca/~gvcormac/dmcspam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMC&lt;/a&gt;. These have the advantage that they do not need any feature selection or pre-processing, and can easily handle new words or typos. They give state-of-the-art performance on tasks like spam-email classification.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T08:19:14.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7473" PostTypeId="2" ParentId="7202" CreationDate="2018-08-08T08:27:54.757" Score="3" Body="&lt;p&gt;The most likely explanation is that you're using too many training examples for your SVM implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVMs are based around a &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;kernel function&lt;/a&gt;. Most implementations explicitly store this as an NxN matrix of distances between the training points to avoid computing entries over and over again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, with 75% of 700,000 examples, this matrix will require approximately 250GB of RAM to store, which is more than you're likely to have in consumer hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your SVM implementation can avoid caching the values, you might get a speedup that way, or you might not (you'll waste a lot of time recomputing them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A much better way to deal with this is to just &lt;a href=&quot;https://stats.stackexchange.com/questions/122409/why-downsample&quot;&gt;not use all&lt;/a&gt; of the data, since most of it will be redundant from the SVM's perspective (it only benefits from having more data near the decision boundaries). A good starting place would be to randomly discard 90% of the training data, and see what performance looks like.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-08-08T12:56:56.550" LastActivityDate="2018-08-08T12:56:56.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7474" PostTypeId="2" ParentId="5821" CreationDate="2018-08-08T08:38:10.520" Score="0" Body="&lt;p&gt;Your problem appears to be &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiple_sequence_alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple sequence alignment&lt;/a&gt;. That problem is well studied, and depending on your application, there are fast algorithms that can perform quite well without recourse to AI or optimization, and which are detailed in the provided link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem has a lot of local minima, then a PSO might be a good approach. Certainly local search has been widely used for this kind of problem in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your state should be represented by a list of sequences, each with some blanks inserted. Movement between states can consist of adding or deleting blanks from one or more sequences. The distance between two states can be computed as the sum of the number of operations needed to transform each sequence to its corresponding sequence in the other state, which you can compute in linear time (just sum up the needed additions or deletions between each pair of letters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that gives you a good starting place!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T08:38:10.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7476" PostTypeId="2" ParentId="6994" CreationDate="2018-08-08T14:41:28.217" Score="1" Body="&lt;p&gt;It sounds like you are trying to do some kind of &lt;a href=&quot;https://en.wikipedia.org/wiki/Semi-supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;semi-supervised learning&lt;/a&gt;. In semi-supervised learning, some data points are labelled (you know which class they belong to), and others are not. There are classification algorithms designed specifically for this kind of problem, like a &lt;a href=&quot;http://www.cs.cmu.edu/~guestrin/Class/10701-S06/Slides/tsvms-pca.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transductive-SVM&lt;/a&gt;. I personally have not found these techniques to be more effective than simply discarding the unlabelled data and treating my problem as purely supervised, but YMMV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TFIDF remains fairly popular, as do ngram-based approaches. A more modern vectorization to consider might be &lt;a href=&quot;https://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt;, which translated something like a bag-of-words style vector into a more meaningful feature space for words.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T14:41:28.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7477" PostTypeId="2" ParentId="6026" CreationDate="2018-08-08T15:04:47.190" Score="11" Body="&lt;p&gt;This is well covered in the corresponding chapter of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; (chapter 3.5, pages 93 to 99 (Third Edition)). Check that out for more details.&lt;/p&gt;&#xA;&lt;p&gt;First, let's review the definitions:&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Your definitions of admissible and consistent are correct.&lt;/p&gt;&#xA;&lt;p&gt;An &lt;strong&gt;admissible heuristic&lt;/strong&gt; is basically just &amp;quot;optimistic&amp;quot;. It never overestimates a distance.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;consistent heuristic&lt;/strong&gt; is one where your prior beliefs about the distances between states are self-consistent. That is, you don't think that it costs 5 from B to the goal, 2 from A to B, and yet 20 from A to the goal. You &lt;em&gt;are&lt;/em&gt; allowed to be overly optimistic though. So you could believe that it's 5 from B to the goal, 2 from A to B, and 4 from A to the goal.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;tree search&lt;/strong&gt; is a general search strategy for searching problems that have a tree structure: that is, it's never possible to &amp;quot;double back&amp;quot; to an earlier state from a later state. This models certain types of games, for instance, like Tic-Tac-Toe. The tree search does not remember which states it has already visited, only the &amp;quot;fringe&amp;quot; of states it hasn't visited yet.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;graph search&lt;/strong&gt; is a general search strategy for searching graph-structured problems, where it's possible to double back to an earlier state, like in chess (e.g. both players can just move their kings back and forth). To avoid these loops, the graph search also keeps track of the states that it has processed.&lt;/p&gt;&#xA;&lt;p&gt;For more on tree vs. graph search, see &lt;a href=&quot;https://ai.stackexchange.com/a/9554/2444&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Okay, now let's talk through the intuition behind the proofs.&lt;/p&gt;&#xA;&lt;p&gt;We first want to show that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is admissible, A* using tree search is optimal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The usual proof is by contradiction.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Assume that A* with tree search and an admissible heuristic was not optimal.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Being non-optimal means that the first complete path from the start to the goal discovered by A* (call this &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;) will be longer than some other path &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, which A* explored up to some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, but no further.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since the heuristic is admissible, the estimated cost of reaching the goal from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; must be smaller than the true cost.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;By 3, and the fact that we know how much it costs to reach &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; along &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, the estimated total cost of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, and thus the cost to expand &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; must be smaller than the true cost of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since the true cost of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is smaller than the cost of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (by 2), the estimated cost to expand &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; must be smaller than the true cost of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A* always picks the path with the most promising total cost to expand next, and the cost of expanding the goal state is given by the total path length required to reach it.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;5 and 6 form a contradiction, so our assumption in 1 must have been incorrect. Therefore A* must be optimal.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The graph search proof uses a very similar idea, but accounts for the fact that you might loop back around to earlier states.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-05-15T12:32:28.280" LastActivityDate="2021-05-15T12:32:28.280" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7478" PostTypeId="2" ParentId="6041" CreationDate="2018-08-08T15:22:47.210" Score="1" Body="&lt;p&gt;To model 2048 (or any problem) for search, you need a only a few pieces of information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note first though, that 2048 is not suitable for minimax, because there's only one player! Instead, you can treat this as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov decision process&lt;/a&gt;. The techniques to solve it are pretty similar though. Basically, you'll do search for one player, and insert &quot;chance&quot; nodes at each ply of the search. The value of a chance node is the expected value of its children. Note that this will reduce the effectiveness of pruning, so it might mean the problem is not tractable for search-based approaches.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What does an &lt;em&gt;end state&lt;/em&gt; look like? Usually you have some function &lt;em&gt;G(s)&lt;/em&gt; that accepts a state s, and produces true if and only if it's an end state. In 2048, end states would be states where the player loses (no moves possible), or where the player wins (a 131072 tile is present), so this should be fairly easy to write.&lt;/li&gt;&#xA;&lt;li&gt;What are the &lt;em&gt;payoffs&lt;/em&gt;? This is usually given by a utility function &lt;em&gt;U(e)&lt;/em&gt; that accepts an end-state e, and produces a numeric value indicating the utility the player will receive.&lt;/li&gt;&#xA;&lt;li&gt;What actions can the player take in each state? In 2048, these are always the same (up, down, left and right)&lt;/li&gt;&#xA;&lt;li&gt;How are new states generated from old states and player actions (in this case, the tiles slide according to the rules of the game, and then a new tile is inserted at a random empty location.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Although search &lt;em&gt;might&lt;/em&gt; work here, since 2048 is a relatively simple MDP, you might be happier using techniques from &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt;, which were specifically designed for this kind of problem. &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; have a good set of chapters on both approaches (14-17).  &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T15:22:47.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7479" PostTypeId="1" CreationDate="2018-08-08T15:26:24.993" Score="3" ViewCount="53" Body="&lt;p&gt;I would like to know whether it's wrong; when working with time series data; to use daily prices as features and the price after 3 days as target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct or should I use the next-day price as target and after training; predict 3 times; each time for one more day ahead(using the predicted value as a new feature)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will these 2 approaches give similar results?&lt;/p&gt;&#xA;" OwnerUserId="17322" LastEditorUserId="17322" LastEditDate="2018-08-22T07:57:24.687" LastActivityDate="2018-08-22T07:57:24.687" Title="Using different timesteps for features and target value" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7480" PostTypeId="2" ParentId="6244" CreationDate="2018-08-08T15:28:16.810" Score="1" Body="&lt;p&gt;This is well covered in the corresponding chapters of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; (Ch. 3 &amp;amp; 4). It also depends on the &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;distinction&lt;/a&gt; between TREE-SEARCH and GRAPH-SEARCH.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, note that Breadth-first search also can't handle cost functions that vary between nodes! Breath-first search only cares about the number of moves needed to reach a state, not the total cost of getting there, so if some states are cheaper on a per-move basis, it's not optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, back to the question: basically, the statement you reference applies to Breadth-first search run on a GRAPH-SEARCH problem. In this kind of problem, it is possible to loop back to an earlier state from a later state (like in chess, where you may move your king back and forth). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the costs for moving between two states are negative, but constant across all the possible moves, then Breadth-first search will &lt;em&gt;not&lt;/em&gt; chose to loop between them forever, since it always expands the &lt;em&gt;nearest&lt;/em&gt; node to the start by total number of moves, not the &lt;em&gt;cheapest&lt;/em&gt; node. Since you could always add one more loop, to make a path shorter, breadth-first search will not find the shortest path (by cost), but will find the shortest path by number of moves instead.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T15:28:16.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7481" PostTypeId="2" ParentId="6366" CreationDate="2018-08-08T15:38:16.817" Score="2" Body="&lt;p&gt;I think there are two problems with your approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, your genetic algorithm contains crossover, but no mutations at all. In a GA, crossover causes convergence, while mutation is the only &quot;exploration&quot; operation. This means your creatures are stuck with whatever genes were present in their small initial population, and, even with modest selection pressures, they will rapidly converge to all being identical to each other. A common way to add mutation is to assign a random value to each location in each child's genome with a small probability (say, 0.01 * 1/number_of_genes). Some researchers prefer higher values. I'm not sure it's been definitely shown which is better, but it likely depends on your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, throwing away agents that died might not be the best selection mechanism. You might get more interesting behaviours if you tied reproduction to something else (e.g. eating a lot of food while you were alive). Right now, your fitness function is probably incentivizing agents to hide in a corner and not do anything, since this maximizes the chance they survive to the end of the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps a bit.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T15:38:16.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7482" PostTypeId="2" ParentId="5622" CreationDate="2018-08-08T15:46:28.823" Score="1" Body="&lt;p&gt;You might want to try &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic regression&lt;/a&gt;. This is a machine learning approach that tries to generate an equation with arbitrary form that best fits a set of examples. I've used it before when working with physicists who wanted an equation in terms of known numeric constants and specific variables, but didn't expect the equation to be linear or one of the other shapes for which we have standard regression models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Symbolic regression often (usually?) uses &lt;a href=&quot;https://deap.readthedocs.io/en/0.7-0/examples/symbreg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic programming&lt;/a&gt; for the underlying optimization. A decent tool is &lt;a href=&quot;https://www.nutonian.com/products/eureqa/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eureka&lt;/a&gt;. It used to be free, but I think they want a payment now. You can also do it yourself from any genetic programming toolkit with a little know-how.   &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T15:46:28.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7483" PostTypeId="2" ParentId="4986" CreationDate="2018-08-08T15:51:27.610" Score="1" Body="&lt;p&gt;Yes, you can. There are a lot of different techniques, usually called &lt;a href=&quot;https://en.wikipedia.org/wiki/Ensemble_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensemble Methods&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better approach might be to use something like &lt;a href=&quot;https://en.wikipedia.org/wiki/AdaBoost&quot; rel=&quot;nofollow noreferrer&quot;&gt;AdaBoost&lt;/a&gt; along with a cheaper method like the decision trees you looked at. AdaBoost explicitly tries to train classifiers to correctly handle different parts of the data, rather than hoping that different methods turn out to do so by chance. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T15:51:27.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7484" PostTypeId="2" ParentId="3929" CreationDate="2018-08-08T16:04:07.840" Score="4" Body="&lt;p&gt;This question is re-inventing the analysis for &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#The_iterated_prisoner&amp;#39;s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterated prisoner's dilemma&lt;/a&gt;&lt;/em&gt; and the co-evolution that can lead to agents playing super-rationally in the one-shot version, which has been studied really extensively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dan Ashlock's &lt;a href=&quot;http://eldar.mathstat.uoguelph.ca/dashlock/Eframe.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;research career&lt;/a&gt; looked at this in great detail from an evolutionary perspective, but it's also been widely studied in other areas of AI. The strategy you describe as superintelligent is called &quot;Tit-for-Tat&quot;, is well known to emerge when people play several rounds of the game. It actually mimics how people play the game in experimental settings. It emerges evolutionarily in simulations because any two agents that both implement it will get more reward than selfish agents. Other, more complex strategies can also appear. For example, the &lt;em&gt;fortress&lt;/em&gt; family of strategies consists of playing a series of question/response opening moves in the early rounds (unlocking the fortress), and then cooperating for the rest of the game if the other player knows the &quot;password&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T16:04:07.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7485" PostTypeId="2" ParentId="4423" CreationDate="2018-08-08T16:16:58.627" Score="1" Body="&lt;p&gt;Welcome to the AI stack Jake. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This probably isn't going to be possible. Modern Psudo-random number generators, like &lt;a href=&quot;https://en.wikipedia.org/wiki/Mersenne_Twister&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mersenne Twister&lt;/a&gt;, are designed not to have any patterns in them, so there's nothing to learn from.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could however, try something like predicting the values of a broken random number generator, like &lt;a href=&quot;https://en.wikipedia.org/wiki/RANDU&quot; rel=&quot;nofollow noreferrer&quot;&gt;RANDU&lt;/a&gt;. These aren't used anymore, precisely because it's possibly to predict future numbers on the basis of past numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T16:16:58.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7486" PostTypeId="2" ParentId="3680" CreationDate="2018-08-08T16:33:19.103" Score="1" Body="&lt;p&gt;There are a few ways of handling this within GA's, but most of them actually amount to using some kind of &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt; instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest way, and most similar to what you've proposed is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear genetic programming&lt;/a&gt;. In this representation, you break the genome into a set of equal-width pieces. Each piece is interpreted as a machine-language instruction for a virtual machine. In your case, plausible instructions might be &quot;move left&quot; or &quot;move right&quot;. Most versions use variable-length genomes, so your program's length depends only on how many instructions there are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to use the standard LISP-like genetic programming system, which Koza documents in &lt;a href=&quot;http://www.genetic-programming.org/gpbook1toc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;his book&lt;/a&gt; for other simple problems, like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Santa_Fe_Trail_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Santa Fe Trail&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More complex encodings are also possible. &lt;a href=&quot;https://en.wikipedia.org/wiki/Grammatical_evolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grammatical evolution&lt;/a&gt; is another one that is similar to a GA in spirit, but interprets a genome according to a CFG instead of as instructions to a virtual machine.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T16:33:19.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7487" PostTypeId="2" ParentId="3298" CreationDate="2018-08-08T16:40:04.100" Score="2" Body="&lt;p&gt;You are trying to solve a variant of &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Halting Problem&lt;/a&gt;, which is the problem of detecting whether a computer program is going to stop, or run forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Halting Problem is incomputable, which means it is not possible to write a computer program that solves it. It is easy to see that your problem is also incomputable. If you could predict whether a program would generate an error, then for any program X that someone wants to solve the halting problem for, we could write a new program:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;run(X)&lt;/li&gt;&#xA;&lt;li&gt;Error(&quot;X finished running&quot;).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and use your algorithm to determine whether X would finish running or not. Since the halting problem is known to be incomputable, this means your problem must be incomputable too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's not to say all is lost though. Formal verification is a field that uses some AI techniques (mostly reasoning-based, but I think there's some machine learning now too) to try to solve this problem for some programs. It can't work for &lt;em&gt;every&lt;/em&gt; program though.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T16:40:04.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7488" PostTypeId="1" CreationDate="2018-08-08T20:31:15.477" Score="8" ViewCount="1017" Body="&lt;p&gt;I have a only a general understanding of General Topology, and want to understand the scope of the term &quot;topology&quot; in relation to the field of Artificial Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In what ways are topological structure and analysis applied in Artificial Intelligence?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-08-08T21:41:19.243" LastActivityDate="2020-05-25T10:15:15.123" Title="In what ways is the term &quot;topology&quot; applied to Artificial Intelligence?" Tags="&lt;terminology&gt;&lt;topology&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7489" PostTypeId="2" ParentId="3298" CreationDate="2018-08-08T20:36:18.667" Score="1" Body="&lt;p&gt;I think you would find this link helpful. It demonstrates how to identify patterns in large arbitrary byte data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://devblogs.nvidia.com/malware-detection-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://devblogs.nvidia.com/malware-detection-neural-networks/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2018-08-08T20:36:18.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7490" PostTypeId="2" ParentId="7488" CreationDate="2018-08-08T21:31:22.327" Score="1" Body="&lt;p&gt;I spent some time thinking about it, but I'm aware of only two main meanings. There might be more that aren't coming to me right now though...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;strong&gt;local search&lt;/strong&gt; problems or sometimes in optimization for machine learning, the &quot;topology&quot; of a problem corresponds to the change in the function you're optimizing as you move between adjacent states. If the change is sharp, you have a &quot;rugged topology&quot;. If it's gentle and continuous, you have a &quot;smooth topology&quot;. See on page 2 of &lt;a href=&quot;https://pdfs.semanticscholar.org/f79e/15480c283700317b85e931da748b3ca56a23.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Fitness Landscape Analysis&#xA;and Cost Models for Local Search&lt;/a&gt; for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other major meaning is with reference to the structure (topology) of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorics#Graph_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial graph&lt;/a&gt;. Many modern &lt;strong&gt;machine learning&lt;/strong&gt; algorithms are based in the idea of combinatorial graphs, including &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Networks&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1202.3732&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sum/Product Networks&lt;/a&gt;, and Deep Neural Networks. Here, topology refers to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Topological_sorting&quot; rel=&quot;nofollow noreferrer&quot;&gt;topological ordering&lt;/a&gt; of a directed graph, or more informally, to &quot;how the graph is structured&quot;. For example, in a neural network, the depth and width of the network's layers, and the nature of the connections between layers, define the topology of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, it gets used a lot in the sense of the second meaning in other areas of AI, just because those areas also use graphs. For example, in automated planning, or in probabilistic reasoning, it is also common to represent your problem as a combinatorial graph. You could then talk about the &quot;topology&quot; of the problem. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-05-23T21:13:07.047" LastActivityDate="2020-05-23T21:13:07.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7491" PostTypeId="2" ParentId="3126" CreationDate="2018-08-08T21:56:25.003" Score="1" Body="&lt;p&gt;Tariq's comment hints at this, but this is still in some sense a very mainstream idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out the &lt;a href=&quot;https://www.aisb.org.uk/media/files/LoebnerPrize2018/Transcripts_2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transcripts&lt;/a&gt; for this year's Loebner Prize. The winner (called Tutor) is again making use of Eliza-like deflections. Some of the other candidates try to use seduction (Columbia), or speaking in a flighty or insistent way (like 2017's &lt;a href=&quot;https://www.aisb.org.uk/events/loebner-prize#finals2017&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rose&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all cases, the bots are relying on humans to read meaning into ambiguous or vague statements, which we seem to be hardwired to do. Kind of neat!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T21:56:25.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7492" PostTypeId="2" ParentId="7051" CreationDate="2018-08-08T22:00:58.383" Score="1" Body="&lt;p&gt;Assuming there's no ordering to the hand (i.e. it doesn't matter what order cards were added to it), then a reasonable approach is to use one input neuron for the number of each kind of card that is present in a player's hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't describe how the game is played, but a common approach for extracting actions is to have one output neuron for each possible action. To select an action, you would pick the one corresponding to the neuron with the highest output response to a given input.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-08T22:00:58.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7493" PostTypeId="2" ParentId="7470" CreationDate="2018-08-09T01:07:57.197" Score="1" Body="&lt;p&gt;Over-fitting in the context of convergence in a neural network can have many causes.  When the model implied in the design of the network is not well fitted for the task, the network may still converge within the time frame allowed and the example set presented but it will take more time and a greater number of examples than necessary, and the reliability and accuracy of the trained circuit may be far below what could be achievable with a solid design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gross over-fitting can be one of the causes of decreased reliability.  A more slight over-fit will exhibit accuracy somewhat diminished from the accuracy found by the end of training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why various designs have emerged with functionally specific circuit simulations between more general multi-layer perceptron networks.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Convolution kernels&lt;/li&gt;&#xA;&lt;li&gt;Rotations&lt;/li&gt;&#xA;&lt;li&gt;Other basic translations&lt;/li&gt;&#xA;&lt;li&gt;Hash lookups&lt;/li&gt;&#xA;&lt;li&gt;Other patterned circuits that remove burden from general convergence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the case of rotation, the convergence on an optima angle in one specialized layer or longitudinal stack element can remove considerable burden and allow overall convergence with fewer general activation layers, using fewer examples, and with a significantly more reliable and accurate result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider what perceptrons must do to rotate an image arbitrarily.  They must wire what is essentially rotational trigonometry into the parameters of everything that is orientation-dependent within the network, creating what is essentially a pliable helix, possibly in many locations within the trained network.  Creating the pliable helix functionality, parameterized in advance of training and carefully handling back-propagation to adjust to its existence, drastically reduces the complexity of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If done well, over-fitting will be much less of an issue.  If done poorly, there could be worse over-fitting or other problems such as non-convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, the best practice is to leave to general network training what must, by its nature, be complex but handle with specific functionality what is well understood and for which mathematical and algorithmic approaches already exist.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-09T01:07:57.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7494" PostTypeId="1" AcceptedAnswerId="7495" CreationDate="2018-08-09T03:31:38.700" Score="1" ViewCount="106" Body="&lt;p&gt;I am reading a book that states, &quot;As the mini-batch size increases, the gradient computed is closer to the 'true' gradient&quot;. So basically what they are saying is mini-batch training only focuses on decreasing the cost function in a certain 'plane', sacrificing accuracy for speed I assume?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how is batching the training examples reducing the dimensions of the weight-space for any batch, or for that matter, each iteration... all of the weights are still involved with each batch/iteration, am I correct?&lt;/p&gt;&#xA;" OwnerUserId="14811" LastEditorUserId="1581" LastEditDate="2018-08-09T18:29:36.663" LastActivityDate="2018-08-09T18:29:36.663" Title="Mini-batch training and the gradient" Tags="&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7495" PostTypeId="2" ParentId="7494" CreationDate="2018-08-09T04:16:38.013" Score="2" Body="&lt;p&gt;The basic idea behind mini-batch training is rooted in the exploration / exploitation tradeoff in &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_search_(optimization)&quot; rel=&quot;nofollow noreferrer&quot;&gt;local search and optimization algorithms&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can view training of an ANN as a local search through the space of possible parameters. The most common search method is to move all the parameters in the direction that reduces error the most (gradient decent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, ANN parameter spaces do not usually have a smooth topology. There are many shallow local optima. Following the global gradient will usually cause the search to become trapped in one of these optima, preventing convergence to a good solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic gradient decent&lt;/a&gt; solves this problem in much the same way as older algorithms like simulated annealing: you can escape from a shallow local optima because you will eventually (with high probability) pick a sequence of updates based on a single point that &quot;bubbles&quot; you out. The problem is that you'll also tend to waste a lot of time moving in wrong directions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mini-batch training sits between these two extremes. Basically you average the gradient across enough examples that you still have some global error signal, but not so many that you'll get trapped in a shallow local optima for long. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent research by &lt;a href=&quot;https://arxiv.org/abs/1804.07612&quot; rel=&quot;nofollow noreferrer&quot;&gt;Masters and Luschi&lt;/a&gt; suggests that in fact, most of the time you'd want to use &lt;em&gt;smaller&lt;/em&gt; batch sizes than what's being done now. If you set the learning rate carefully enough, you can use a big batch size to complete training faster, but the difficulty of picking the correct learning rate increases with the size of the batch.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-09T04:16:38.013" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7496" PostTypeId="2" ParentId="7494" CreationDate="2018-08-09T04:29:25.327" Score="1" Body="&lt;p&gt;It's like you have a class of 1000 children and you being a teacher, want all of them to learn something at the same time. It is difficult because all are not the same, they have different adaptability and reasoning strength.&#xA;So one can have alternate strategies for the same task. &#xA;1) Take each child at a time and train it. It will be the good approach but it will take a long time  &lt;code&gt;here each child is equal to your batch size&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Take a group of 10 children and train them, this can be the good compromise between time, and learning. In the smaller group, you can handle naughty one better. &lt;code&gt;here your batch size is 10&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) If you take all 1000 children and teach them, it will take a very short time but you will not be able to give proper attention to those mischievous ones &lt;code&gt;here your batch size is 1000&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Same with machine learning, Take reasonable batch size, tune weight accordingly. &#xA;I hope this analogy will clear your doubt.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-08-09T04:29:25.327" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7497" PostTypeId="2" ParentId="7459" CreationDate="2018-08-09T07:56:41.830" Score="3" Body="&lt;p&gt;Depending on kind and amount of data you posess, there are few approaches that you might consider.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Marking target objects on dataset and training CNN that returns coordinates of target object. In this case, remember that it is usually faster when training data ROIs have their coordinates relative to image size.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use some kind of focus mechanism, like spatial transformer network:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1506.02025&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1506.02025&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This kind of network component is able to learn image transformation (including crop) that maximazes target metric for main classifier. This tutorial on pytorch:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pytorch.org/tutorials/intermediate/spatial_transformer_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/tutorials/intermediate/spatial_transformer_tutorial.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;shows some nice visualizations of STN results. Good thing about this kind of network is that, given enough data, it might learn proper transformation from image classification data (photo -&gt; class). One does not need to explicitly mark target objects on image!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Object detection networks, like YOLO, Faster-RCNN. There are many tutorials on that matter, eg:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.datacamp.com/community/tutorials/object-detection-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.datacamp.com/community/tutorials/object-detection-guide&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Saliency extraction. Simple idea is to generate heatmap showing what parts of input image activates classifier the most. I guess you could try calculate bounding box basing on such heatmap. Example research paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1805.08249&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1805.08249&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Points 1 and 2 are probably easies to implement, so I would start with them.&lt;/p&gt;&#xA;" OwnerUserId="16929" LastActivityDate="2018-08-09T07:56:41.830" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7498" PostTypeId="2" ParentId="7470" CreationDate="2018-08-09T09:50:31.003" Score="3" Body="&lt;h2&gt;How can data augmentation reduce overfitting?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You write that you can already maybe see how data augmentation can help prevent overfitting in general, but it sounds a bit uncertain and it's still asked in the title of the question, so I'll address this first:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, when we use Machine Learning for classification problems, we would ideally learn a classifier that can perform well on a &lt;strong&gt;population&lt;/strong&gt;. An example of a population would be: &lt;strong&gt;the set of all handwritten characters in the entire world&lt;/strong&gt;. Generally, we don't have that complete population available for training, we only have a (much smaller) &lt;strong&gt;training dataset&lt;/strong&gt;. If a training set is large enough, it might be a good approximation of the true population we're interested in (a &quot;dense sampling&quot; of the space we're interested in), but it's still just that; an approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We say that a learning algorithm is &lt;strong&gt;overfitting&lt;/strong&gt; if it performs singificantly better on the training set than it is on the population (which we generally approximate again using a separate test set). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, data augmentation (like adding rotations / translations of images in the training set to the training set) can help combat overfitting &lt;strong&gt;because it bridges the gap between training set and population&lt;/strong&gt;. The population (all handwritten characters in the entire world) will likely include characters at various offsets from the middle (e.g. translations) and at various rotations. So, data augmentation is simply adding more examples (and possibly more &lt;em&gt;varied&lt;/em&gt; examples) to our training set, which importantly are considered to be a part of the population we're interested in. If, for example, the population we are interested in were only the set of all handwritten characters at a specific position in the image (e.g., centered), then augmenting the dataset by adding various translations would not help; we'd be adding instances that are outside the population we want to learn about.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Why doesn't adding extra classes for rotations cancel out augmentations?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are &lt;strong&gt;two possible explanations&lt;/strong&gt; I can come up with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Maybe the &quot;extra-class&quot; rotations are different from the &quot;data augmentation&quot; rotations.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is the exact quote that's relevant from the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;To reduce the risk of overfitting, we performed data augmentation by randomly translating and rotating character images. We also created new classes through 90◦, 180◦ and 270◦ rotations of existing data.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That first sentence is not 100% clear in my opinion. I imagine the translations they use for data augmentation are relatively small (e.g. offsets of a few pixels), so maybe the rotations they use for data augmentation are also only &quot;small&quot; rotations (for example, between -10◦ and +10◦). The &quot;larger&quot; rotations (multiples of 90◦) described in the second sentence may then no longer be a part of the &quot;data augmentation to reduce the risk of overfitting&quot; in the first sentence; they're simply parts of a different action performed to increase the number of classes in the dataset (and, I imagine, for each of these larger rotations they may again perform &quot;smaller rotations&quot; for data augmentations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This explanation is kind of hypothetical though, it's not 100% clear from the paper exactly what they mean here in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;&quot;Overfitting&quot; can have a slightly different interpretation in the case of one-shot learning than in traditional learning.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that this paper is about &quot;one-shot learning&quot;, where the goal is to be able to classify accurately after being presented only a single example (&quot;one shot&quot;) of a never-before-seen class. In such one-shot problems, you could in some sense say that an algorithm might &quot;overfit&quot; to the &quot;distribution of classes&quot; if it can only perform one-shot learning well on a certain set of similar classes, but not on others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you only train one-shot learning on a set of handwritten characters that are &quot;upright&quot; (close to 0 rotation), your algorithm might be able to perform well in terms of one-shot learning when presented with new classes (new handwritten characters) that are also upright, but might be incapable of proper one-shot learning when presented with new classes (new handwritten characters) that are upside-down.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-09T09:50:31.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7500" PostTypeId="1" CreationDate="2018-08-09T19:11:34.800" Score="5" ViewCount="74" Body="&lt;p&gt;I'm looking for a database or some machine readable document that contains common ordered lists or common short sets. e.g:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{January, February, March,...}&#xA;{Monday, Tuesday, ....}&#xA;{Red, Orange, Yellow,...}&#xA;{1,2,3,4,...}&#xA;{one, two, three, four,...}&#xA;{Mercury, Venus, Earth, Mars,...}&#xA;{I, II, III, IV, V, VI,...}&#xA;{Aquarius, Pisces, Aries,...}&#xA;{ein, zwei, drei, ...}&#xA;{Happy, Sneezy, Dopey, ...}&#xA;{Dasher, Dancer, Prancer, Vixen ,...}&#xA;{John, Paul, George, Ringo}&#xA;{20, 1, 18, 4, 13, 6, ...}&#xA;{Alabama, Alaska, Arizona, Arkansas, California,...}&#xA;{Washington, Adams, Jefferson, ...}&#xA;{A,B,C,D,E,F,G,...}&#xA;{A,E,I,O,U}&#xA;{2,3,5,7,11,13,17,...}&#xA;{triangle, square, pentagon, hexagon,...}&#xA;{first, second, third, fourth, fifth,...}&#xA;{tetrahedron, cube, octohedron, icosohedron, dodecahedron}&#xA;{autumn, winter, spring, summer}&#xA;{to, be, or, not, to, be, that, is, the, question}&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One use is for creating an AI that can solve codes or predict the next thing in a sequence.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="1671" LastEditDate="2018-08-09T20:45:10.960" LastActivityDate="2018-08-10T09:32:30.060" Title="Is there a database somewhere of common lists?" Tags="&lt;datasets&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2021-02-03T17:09:26.473" ContentLicense="CC BY-SA 4.0" />
  <row Id="7501" PostTypeId="5" CreationDate="2018-08-09T21:23:31.077" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-08-09T21:23:31.077" LastActivityDate="2018-08-09T21:23:31.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7502" PostTypeId="4" CreationDate="2018-08-09T21:23:31.077" Score="0" Body="Asking for datasets, libraries/APIs or pre-trained models is OFF-topic here, so do NOT use this tag for that. Use this tag when you are interested in any other &quot;resource&quot; (in the context of artificial intelligence) that does not involve the just mentioned resources, such as books, papers, articles, tutorials or courses. (The purpose of this tag is very similar to the purpose of the tag reference-request. For that reason, they could be merged.)" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-11-21T13:32:44.790" LastActivityDate="2020-11-21T13:32:44.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7503" PostTypeId="2" ParentId="7479" CreationDate="2018-08-10T00:32:57.380" Score="1" Body="&lt;p&gt;I don't know what kind of price data you're dealing with. I suppose the order of the data matters a lot, so my suggestion would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use LSTM as it handles time series better&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can predict 3 consecutive numbers from an RNN as the next three days' predictions&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try regression first, it is likely it will not work (or just flatten the curves, depend on your data noise), then classification is an easier approach&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't forget normalization &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17365" LastActivityDate="2018-08-10T00:32:57.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7504" PostTypeId="2" ParentId="7500" CreationDate="2018-08-10T02:33:10.447" Score="2" Body="&lt;p&gt;So here’s a couple quick resources that i could think of.  First of all, you could look at this,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/List_of_lists_of_lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/List_of_lists_of_lists&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has classics such accidents,  hospitals in Asia, or even a list of famous resignations. It’s easentially a list of random lists of things.  It may not owner cover your requirement for sequences but it’ll help with small subsets of lists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for sequences, you could always check out,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://oeis.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://oeis.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It’s pretty much a list of official mathematical sequences.  It’s got everything from the Fibonacci sequence to esoteric sequences you’ve never heard of.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2018-08-10T02:33:10.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7505" PostTypeId="2" ParentId="7350" CreationDate="2018-08-10T03:26:05.300" Score="4" Body="&lt;p&gt;Assuming you're not referring to any particular type of pooling operation, it's possible that you could have, for example, a mean pool followed by a max or min pool. What this could do is combine the idea of reducing the dimensionality of your data from a holistic perspective with the mean pool, and then choosing the best of your averages with your max pool.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastEditorUserId="2444" LastEditDate="2020-05-30T11:20:31.957" LastActivityDate="2020-05-30T11:20:31.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7506" PostTypeId="2" ParentId="5982" CreationDate="2018-08-10T04:55:24.193" Score="0" Body="&lt;p&gt;I would suggest you use a sequence to sequence model with character level features. It is an easy task, provided you have data.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-08-10T04:55:24.193" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7509" PostTypeId="1" CreationDate="2018-08-10T09:45:58.990" Score="4" ViewCount="64" Body="&lt;p&gt;Nowadays we don't know how to create AI in a safe way (I think that we don't even know yet how to define a safe AI), but there is a lot of research in developing a model allowing it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say, that someday we discover such a model (maybe even it would be possible to mathematically prove its safety). Is it rational to ask, how do we prevent people from creating AI outside of this model (e.g. they are so confident in their own model, that they just pursue it and end up with something like &lt;code&gt;paperclip scenario&lt;/code&gt;)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should we also think about creating some theory/infrastructure preventing such a scenario?&lt;/p&gt;&#xA;" OwnerUserId="17411" LastEditorUserId="1581" LastEditDate="2018-08-10T17:16:53.080" LastActivityDate="2018-08-16T08:02:23.197" Title="Is it necessarry to create theory/infrastructure to prevent people from creating AI incompatible with a safe model (if we create one)?" Tags="&lt;philosophy&gt;&lt;ai-safety&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7510" PostTypeId="2" ParentId="6898" CreationDate="2018-08-10T13:54:41.037" Score="2" Body="&lt;p&gt;I do not believe that StackExchange has published precisely what algorithm(s) they use for that, so we can't tell for sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in &lt;a href=&quot;https://meta.stackexchange.com/q/286329/376651&quot;&gt;this meta.stackexchange question&lt;/a&gt;, you can follow some of the efforts that were undertaken in collecting training data for training such a classifier. The post also links to the &quot;CQADupStack: Gold or Silver?&quot; paper which describes the analysis of such a dataset that comes directly from StackOverflow. You might be able to find interesting literature by browsing google scholars for papers that cite this paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also &lt;a href=&quot;https://meta.stackoverflow.com/q/343382/6735980&quot;&gt;another meta.stackoverflow discussion&lt;/a&gt; on this topic, where answers link to various community-developed projects / bots for this purpose. Again, not necessarily what is actually used by the StackExchange sites, but likely similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there is definitely a lot of research on performing such classifications (one example found by a quick google search is &lt;a href=&quot;https://peerj.com/preprints/26555.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Duplicate Question Detection in Stack Overflow: A Reproducibility Study&quot;&lt;/a&gt;, where many other relevant publications can be found in the list of References). This again does not necessarily lead to be precisely the algorithm that StackExchange happens to use, but many relevant ones, one or more of which they &lt;em&gt;might&lt;/em&gt; be using.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-10T13:54:41.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7511" PostTypeId="1" CreationDate="2018-08-10T18:09:09.373" Score="4" ViewCount="77" Body="&lt;p&gt;I am going to train a deep learning model to classify hand gestures in video. Since the person will be taking up nearly the entire width/height of the video and I will be classifying what hand gesture he or she is doing, I don't need to identify the person and create a bounding box around the person doing the action. I only need to classify video sequences to their class labels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be training on a dataset with individual videos, in which each entire video clip is the particular gesture (So it's a dataset like UCF-101, with video clips corresponding to class labels). But when I am deploying the network, I want the neural network to run on live video. As in how the live video is playing, it should recognize when a gesture has occurred and indicate that it recognized the gesture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was wondering - How can I train the neural network on isolated video sequences in which the entire video clip is the action (like explained above), &lt;strong&gt;but&lt;/strong&gt; run the neural network on &lt;strong&gt;live video&lt;/strong&gt;? For instance, can I use a 3D CNN? Or must I use a 2D CNN with an LSTM network instead, for it to work on live video? My concern is that since a 3D CNN performs the filters across many frames, wouldn't running the CNN on every frame make it very slow? But if I use a 2D CNN with LSTM, will that make it faster? Or will both work fine?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help in advance.&lt;/p&gt;&#xA;" OwnerUserId="11364" LastActivityDate="2018-11-13T09:02:49.153" Title="How should continuous action/gesture recognition be performed differently than isolated action recognition" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;action-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7515" PostTypeId="2" ParentId="7215" CreationDate="2018-08-11T03:19:32.843" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Degree to Which Inhibition is in Common Use&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could loosely be considered inhibitory effect occurs in MLPs (multilayer perceptrons) as they are normally designed and implemented already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient descent scheme implemented within a larger back propagation algorithm can produce a parameter adjustment delta that is either positive or negative.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A positive value decreases the attenuation of that parameter's signal path, thereby increasing the signal strength there.&lt;/li&gt;&#xA;&lt;li&gt;A negative value increases the attenuation of that path, thereby&#xA;decreasing signal strength through that connection.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A decrease in a parameter's value as a result of back propagation bears some similarity to the inhibition of a neural signal path, however, you may already be aware of the significant differences in the signaling between biological neurons and the signalling between layers in the type of artificial networks commonly in machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term inhibition is, as mentioned, only loosely applicable.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One cannot inhibit a pulse through a MLP because there ARE NO PULSES in MLPs.&lt;/li&gt;&#xA;&lt;li&gt;One cannot alter the signal attenuation between neurons by varying a numeric parameter either, since there is no numeric parameter array in a biological net.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Stimulation and inhibition in the brains of mammals are also different in that neuro-chemistry impacts the network regionally, so the terms stimulation and inhibition are a bit ambiguous, since we have agonists and antagonists ranging from dopamine to serotonin and from cannabanoids to oxytocin receptors and from endorphins to other classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Changes from Former Textbook Themes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The former thinking was that a pulse travelling through a biological signal path strengthened that connection.  No one in neurology research adheres to that simplistic a conception today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it is know that a signal pathway may be in common use but may close down by repeated sharp pains following its use.  Although I am not well trained in electro-chemical processes in neural pathways, I recall &lt;em&gt;in vitro&lt;/em&gt; experiments supporting that this is neither neuro-plastic nor electrical, that it is related to regional chemical feedback.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current view of addiction as a brain disease is that a breakdown of the interrelationship between chemical state change and learned inhibition or transmission is causal.  Inhibition or transmission is no longer decided upon based on organism survival and socialization but on the addictive stimuli, leading to behavioral dysfunction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be useful to point out that, stimulation and inhibition are not strictly antonyms.  The opposite of inhibiting a signal is the transmission of it.  The opposite of stimulation is the lack of stimulation (no signal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Attempting Analogy in Largely Dissimilar Circuit Models&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may not be an aid to general understanding to draw parallels between ReLU activation functions and the functions of synapses with their sensitivity to regional brain chemistry and with cell-level retention functions orchestrated by organelles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural nets are not neural.  They are a mathematical conception sharing only the ideal of learning as convergence on some ideal network behavior.  Nothing else of significance is in common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Adding to the Disparity Between MLPs and Biology&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a sense inhibition in the brain occurs at multiple architectural levels, inside the cell, between cells, and over structures of cells, and the alignment of pulses temporally (in the time domain) is not simulated at all in conventional machine learning constructs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some researchers have deviated entirely from the multilayer perceptron design and favored a pulse based system that requires specialized hardware.  Follow the money there.  It is not an inexpensive research avenue yet.  But it may become one if they have success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Curvature in Functions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brief note on terminology:  Nth degree polynomials fall under linear algebra, so the best term to use is 'curved functions' so as to not fall into the ambiguity of the term non-linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, you are correct that there are non-linearities of different types in biological neural circuits.  Potential change is not only curved, but its function's curvature changes quickly.  It is temporally sensitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the longer time frame, the memory in a cell forms through neural plasticity and the cell behavior changes internally (within the cell membrane) employing cytoplasm and the suspended organelles.  That memory function also attenuates at a roughly inverse exponential rate with respect to time but some have hypothesized based in empirical evidence that forgotten cellular function can be recalled.  Again, this is at the cellular level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second non-linearity is not a sum of potentials.  The surface of the function that aggregates incoming signals is not flat.  It is curved.  Also, as mentioned, the temporal alignment presents a complexity, since perfect pulse alignment is not treated the same as pulses not perfectly aligned in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I brought up the absurdity of using an additive adjustment in MLP back propagation in a question I wrote for this site.  The responses to the challenge to the status quo not particularly well understood by the majority of machine learning practitioners were not outstanding.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Linear Thinking Prevails Currently&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To a large degree linear thinking (in the wider sense of the term) pervades mainstream machine learning and data science today, the activation functions being a notable and welcomed exception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over time, I expect that will improve.  I see current leading edge research going beyond that linear thinking and considering short and long term memory as in the LSTM and attention based networks, the simulation of the curved surfaces that represent pulse propagation in mammalian nets and the consideration of various applications of exponential decay here and there in the latest literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Gratitude for the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions like this one may help widen the mainstream understanding too.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:09:40.010" LastActivityDate="2018-10-15T23:09:40.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7516" PostTypeId="2" ParentId="5452" CreationDate="2018-08-11T04:02:03.303" Score="1" Body="&lt;p&gt;The impact of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the number of cores,&lt;/li&gt;&#xA;&lt;li&gt;cache size, and&lt;/li&gt;&#xA;&lt;li&gt;clock speed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;on GPU performance when comparing the ones you mentioned is not project critical when doing either lab research or deploying to the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More important is the optimization of algorithms to the hardware available, video bus architecture (since data passing between CPUs and GPUs is dependent on upload and download bit rates), the potential use of clusters of computing nodes, and the statistical handling of the learning process, such as sampling methodologies, data hygiene, normalization of input, and the proper mapping of activations and their hyper-parametrization.  The pawning off of random access to swap space can be a bottleneck unrelated to the bus communication between units too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an architect, one must consider the big picture.  Even the exact same software and hardware combination, when given example sets with widely different distributions of values, may favor different features, one requiring more cache and the other performing proportionally with the clock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether multiple cores will be effectively used has much to do with how vectors are represented and what the compiler and instruction cache do or do not do in terms of optimization.&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can find side-by-side test results, the specifics for particular test scenarios may be available to you, but be warned that your own cases may differ enough from the test suite used in the reports you find to completely invalidate the usefulness of the report.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0743731514001142&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal of Parallel and Distributed Computing, Vol. 74, Iss. 10, October 2014, pp 3056-3069 &lt;em&gt;Amdahl’s law for multithreaded multicore processors&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Higher [speeds] than that predicted by Amdahl’s law may be achieved for the parallelizable part of the workload, if core threads exhibit strong cache affinity and the workload is strongly memory-bound. Then, we derive a tight speedup upper bound in the presence of both memory resource contention and critical section for multicore processors with single-threaded cores. This speedup upper bound indicates that with resource contention among threads, whether it is due to shared memory or critical section, a sequential term is guaranteed to emerge from the parallelizable part of the workload, fundamentally limiting the scalability of multicore processors for parallel computing, in addition to the sequential part of the workload&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-11T04:02:03.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7518" PostTypeId="1" AcceptedAnswerId="7708" CreationDate="2018-08-11T06:41:18.710" Score="4" ViewCount="257" Body="&lt;p&gt;I am trying to make balanced weapon pairs. So there are five stats per weapon, and I am simulating a number of combats (1000) with different stats randomized, and counting the win, lose, and draw of the &quot;weapon fight&quot; for the database. I want an algorithm for making weapon1win-weapon2win as small as possible for balance through changing the weapon stats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Random Stats --&gt; Combat 1000 Times --&gt; Count Win &amp;amp; Lose --&gt; Data For Training The AI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data Sample:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;{[(1,2,3,4,5) --&gt; Weapon1Stats,(5,4,3,2,1) --&gt;Weapon2Stats,1 --&gt; WinWeapon1-WinWeapon2],[....]} (The text isn't part of the data sample, they are just there to help you know which variable is which. Also, the {[( s are all supposed to be [ s but changed for clarity)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like an function, preferably in C++ or Python, but just with text that is well explained is fine, for handling the data. The result would be a method to determine how to minimize WinWeapon1-WInWeapon2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(edit)&#xA;I would say that the one I was looking for is one with something like a score for the weapon as in strength (1*stat1 + 2*stat2 etc...) but I do want something new that works better, I am also having problems creating leeway for the functions coefficients.(edit end)&lt;/p&gt;&#xA;" OwnerUserId="17423" LastEditorUserId="1671" LastEditDate="2018-08-13T20:54:59.367" LastActivityDate="2018-08-25T00:37:53.073" Title="Algorithm For Making Balanced Weapons In A Game?" Tags="&lt;algorithm&gt;&lt;game-ai&gt;&lt;python&gt;&lt;datasets&gt;&lt;c++&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7519" PostTypeId="2" ParentId="7215" CreationDate="2018-08-11T09:28:24.753" Score="2" Body="&lt;p&gt;In biology, when the presynaptic releases a neurotransmitter (a positive amount of them, obviously), this neurotransmitter reaches the postsynaptic vesicles causing an excitatory (depolarization) or inhibitory (hyperpolarization) effect, depending on the kind of postsynaptic vesicle in next cell dendrites. If the total amount of depolarization (all dendrites) is enough bigger than hyperpolarization, the neuron triggers an action potential or similar signal, continuing with the chain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the artificial NeuralNet parallelism, when the activation function of previous layer provides an output (say positive one) this value is multiplied by the weights of next layer cell. If the weight is positive, the effect is excitatory, if the weight is negative, the effect is inhibitory.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, these two models are functionally equivalent (same excitatory/inhibitory target is covered), just make the analogy between kind of postsynaptic vesicle with the input weight sign of the artificial neuron.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2018-08-11T10:11:54.187" LastActivityDate="2018-08-11T10:11:54.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7520" PostTypeId="1" CreationDate="2018-08-11T11:07:48.770" Score="1" ViewCount="50" Body="&lt;p&gt;I use recurrent neural network, RNNs have to get input one value per step and it will show one value output. If I have daily sale demand time series data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to predict sale demand for three days. So, Rnn have to show output one day in three time or it can show sale  demand three days output in one time prediction ?&lt;/p&gt;&#xA;" OwnerUserId="17424" LastEditorUserId="1641" LastEditDate="2018-08-11T12:43:37.770" LastActivityDate="2018-08-11T12:43:37.770" Title="How recurrent neural network work when predict many days?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7521" PostTypeId="2" ParentId="7153" CreationDate="2018-08-11T15:09:21.190" Score="3" Body="&lt;p&gt;The differences you have observed between the two different versions of the TRPO paper are due to different formalizations of the problem and the objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1502.05477v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;the first version of the paper you linked&lt;/a&gt;, they start out in Section 2 by defining Markov Decision Processes (MDPs) as tuples that, among other things, have a &lt;strong&gt;cost function&lt;/strong&gt; $c : \mathcal{S} \rightarrow \mathbb{R}$. They define $\eta(\pi)$ as the expected discounted &lt;strong&gt;cost&lt;/strong&gt; of a policy $\pi$, and subsequently also define state-action value functions $Q_{\pi}(s_t, a_t)$, value functions $V_{\pi}(s_t)$, and advantage functions $A_{\pi}(s, a)$ in terms of &lt;strong&gt;costs&lt;/strong&gt;. Ultimately, in Equation 15, they write the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{aligned}&#xA;\underset{\theta}{\text{minimize }} &amp;amp; \mathbb{E}_{s \sim \rho_{\theta_{\text{old}}}, a \sim q} \left[ \frac{\pi_{\theta}(a \vert s)}{q(a \vert s)} Q_{\theta_{\text{old}}}(s, a) \right] \\ &#xA;\text{subject to } &amp;amp; \mathbb{E}_{s \sim \rho_{\theta_{\text{old}}}} \left[ D_{KL}(\pi_{\theta_{\text{old}}}(\cdot \vert s) ~ \vert \vert ~ \pi_{\theta}(\cdot \vert s)) \right] \leq \delta&#xA;\end{aligned}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there's a lot going on there, but we can &lt;strong&gt;very informally&lt;/strong&gt; &quot;simplify&quot; it to only the parts that are relevant for this question, as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underset{\theta}{\text{minimize }} \mathbb{E} \left[ Q(s, a) \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we look at just that, we see that we're pretty much trying to minimize $Q$-values, which are &lt;strong&gt;costs&lt;/strong&gt;; that makes sense, typically &lt;strong&gt;costs&lt;/strong&gt; are things we want to &lt;strong&gt;minimize&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1502.05477&quot; rel=&quot;nofollow noreferrer&quot;&gt;second version of the paper you linked&lt;/a&gt;, they have changed the Preliminaries in Section 2. Now they no longer have a cost function $c$ in their definition of an MDP; they have replaced it by a &lt;strong&gt;reward function&lt;/strong&gt; $r : \mathcal{S} \rightarrow \mathbb{R}$. Then they move on to define $\eta(\pi)$ as the &lt;strong&gt;expected discounted reward&lt;/strong&gt; (rather than expected discounted cost), and also define $Q$, $V$ and $A$ in terms of &lt;strong&gt;rewards&lt;/strong&gt; rather than costs. This now all matches the standard, common terminology in Reinforcement Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, Equation 14 looks identical to what we saw above, it's again about an expectation of $Q$-values. But, now $Q$-values are &lt;strong&gt;rewards&lt;/strong&gt; rather than &lt;strong&gt;costs&lt;/strong&gt;. Rewards are generally things we want to &lt;strong&gt;maximize&lt;/strong&gt;, rather than &lt;strong&gt;minimize&lt;/strong&gt;, so that's why the objective swapped around.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-11T17:34:32.210" LastActivityDate="2018-08-11T17:34:32.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7522" PostTypeId="1" AcceptedAnswerId="7545" CreationDate="2018-08-11T23:41:14.547" Score="4" ViewCount="97" Body="&lt;p&gt;I've been wanting to make my own Neural Network in Python, in order to better understand how it works. I've been following &lt;a href=&quot;https://www.youtube.com/watch?v=ZzWaow1Rvho&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; series of videos as a sort of guide, but it seems the backpropagation will get much more difficult when you use a larger network, which I plan to do. He doesn't really explain how to scale it to larger ones.&lt;/p&gt;&#xA;&lt;p&gt;Currently, my network feeds forward, but I don't have much of an idea of where to start with backpropagation. My code is posted below, to show you where I'm currently at (I'm not asking for coding help, just for some pointers to good sources, and I figure knowing where I'm currently at might help):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;&#xA;class NN:&#xA;    prediction = []&#xA;    def __init__(self,input_length):&#xA;        self.layers = []&#xA;        self.input_length = input_length&#xA;    def addLayer(self, layer):&#xA;        self.layers.append(layer)&#xA;        if len(self.layers) &amp;gt;1:&#xA;            self.layers[len(self.layers)-1].setWeights(len(self.layers[len(self.layers)-2].neurons))&#xA;        else:&#xA;            self.layers[0].setWeights(self.input_length)&#xA;    def feedForward(self, inputs):&#xA;        _inputs = inputs&#xA;        for i in range(len(self.layers)):&#xA;            self.layers[i].process(_inputs)&#xA;            _inputs = self.layers[i].output&#xA;        self.prediction = _inputs&#xA;&#xA;    def calculateErr(self, target):&#xA;        out = []&#xA;        for i in range(0,len(self.prediction)):&#xA;            out.append(  (self.prediction[i] - target[i]) ** 2  )&#xA;        return out&#xA;        &#xA;&#xA;class Layer:&#xA;&#xA;    neurons = []&#xA;    weights = []&#xA;    biases = []&#xA;    output = []&#xA;    &#xA;    def __init__(self,length,function):&#xA;        for i in range(0,length):&#xA;            self.neurons.append(Neuron(function))&#xA;            self.biases.append(numpy.random.randn())&#xA;&#xA;    def setWeights(self, inlength):&#xA;        for i in range(0,inlength):&#xA;            self.weights.append([])&#xA;            for j in range(0, inlength):&#xA;                self.weights[i].append(numpy.random.randn())&#xA;    &#xA;    def process(self,inputs):&#xA;        for i in range(0, len(self.neurons)):&#xA;            self.output.append(self.neurons[i].run(inputs,self.weights[i], self.biases[i]))&#xA;    &#xA;&#xA;class Neuron:&#xA;    output = 0&#xA;    def __init__(self, function):&#xA;        self.function = function&#xA;    def run(self, inputs, weights, bias):&#xA;        self.output = self.function(inputs,weights,bias)&#xA;        return self.output&#xA;&#xA;def sigmoid(n):&#xA;    return 1/(1+numpy.exp(n))&#xA;&#xA;&#xA;def inputlayer_func(inputs,weights,bias):&#xA;    return inputs&#xA;&#xA;def l2_func(inputs,weights,bias):&#xA;    out = 0&#xA;    &#xA;    for i in range(0,len(inputs)):&#xA;        out += weights[i] * inputs[i]&#xA;    out += bias&#xA;    &#xA;    return sigmoid(out)&#xA;&#xA;NNet = NN(2)&#xA;&#xA;&#xA;l2 = Layer(1,l2_func)&#xA;&#xA;&#xA;NNet.addLayer(l2)&#xA;NNet.feedForward([2.0,1.0])&#xA;print(NNet.prediction)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, is there any resource that explains how to implement the back-propagation algorithm step-by-step?&lt;/p&gt;&#xA;" OwnerUserId="17432" LastEditorUserId="2444" LastEditDate="2020-12-22T23:58:13.720" LastActivityDate="2020-12-23T00:10:20.207" Title="How can I implement back-propagation for medium-sized neural networks?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;backpropagation&gt;&lt;implementation&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7524" PostTypeId="1" AcceptedAnswerId="7530" CreationDate="2018-08-12T12:04:14.330" Score="4" ViewCount="1093" Body="&lt;p&gt;I am trying to build an RL agent to price paid-for-seating on commercial flights. I should reiterate here - I am not talking about the price of the ticket - rather, I am talking about the pricing you see if you click on the seat map to choose where on the plane you sit (exits rows, window seats, etc). The general set up is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;After choosing their flights (for a booking of &lt;em&gt;n&lt;/em&gt; people), a customer will view a web page with the available seat types and their prices visible.&lt;/li&gt;&#xA;&lt;li&gt;They select between zero and &lt;em&gt;n&lt;/em&gt; seats from a seat map with a variety of different prices for different seats, to be added to their booking.&lt;/li&gt;&#xA;&lt;li&gt;The revenue from step 2 is observed as the reward.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Each 'episode' is the selling cycle of one flight. Whether the customer buys a chosen seat or not, the inventory goes down as they still have a ticket for the flight so will get a seat at departure. I would like to change prices on the fly, rather than fix a set of optimal prices throughout the selling cycle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not decided on a general architecture yet. I want to take various booking, flight, and inventory information into account, so I know I will be using function approximation (most likely a neural net) to generalise over the state space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am less clear on how to set up my action space. I imagine an action would amount to a vector with a price for each different seat type (window seat, exit row, etc). If I have, for example, 8 different seat types, and 10 different price points for each, this gives me a total of 10^8 different actions, many of which will be very similar. In a sense, each action is comprised of a combination of sub-actions - the action of pricing each seat type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, each sub-action (pricing one seat type) is somewhat dependent on the others, in the sense that the price of one seat type will likely affect the demand (and hence reward contribution) for another. For example, if you set window seats to a very cheap price, people will be less likely to spend a normal amount for the other seat types. Hence, I doubt the problem can be decomposed into a set of sub-problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested if there has been any research into dealing with a problem like this. Clearly any agent I build needs some way to generalise across actions to some degree, since collecting real data on millions of actions is not possible, even just for one state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it, this comes down to three questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it possible to get an agent that can deal with a set of actions (prices) as a single decision?&lt;/li&gt;&#xA;&lt;li&gt;Is it possible to get this agent to understand actions in relative terms? Say for example, one set of potential prices is [10, 12, 20], for middle seats, aisle seats, and window seats. Can I get my agent to realise that there is a natural ordering there, and that the first two pricing actions are more similar to each other than to the third possible action?&lt;/li&gt;&#xA;&lt;li&gt;Further to this, is it possible to generalise from this set of actions - could an agent be set up to understand that the set of prices [10, 13, 20] is very similar to the first set?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I haven't been able to find any literature on this, especially relating to the second question - any help would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17435" LastEditorUserId="17435" LastEditDate="2018-08-12T20:18:57.697" LastActivityDate="2018-08-13T11:44:10.180" Title="How to generalise over multiple simultaneous dependent actions in Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;combinatorics&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7525" PostTypeId="1" AcceptedAnswerId="7544" CreationDate="2018-08-12T12:08:19.820" Score="8" ViewCount="161" Body="&lt;p&gt;In fields such as Machine Learning, we typically (somewhat informally) say that we are overfitting if improve our performance on a training set at the cost of reduced performance on a test set / the true population from which data is sampled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, in AI research, we often end up testing performance of newly proposed algorithms / ideas on the same benchmarks over and over again. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For over a decade, researchers kept trying thousands of ideas on the game of Go.&lt;/li&gt;&#xA;&lt;li&gt;The ImageNet dataset has been used for huge amounts of different publications&lt;/li&gt;&#xA;&lt;li&gt;The Arcade Learning Environment (Atari games) has been used for thousands of Reinforcement Learning papers, having become especially popular since the DQN paper in 2015.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, there are very good reasons for this phenomenon where the same benchmarks keep getting used:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reduced likelihood of researchers &quot;creating&quot; a benchmark themselves for which their proposed algorithm &quot;happens&quot; to perform well&lt;/li&gt;&#xA;&lt;li&gt;Easy comparison of results to other publications (previous as well as future publications) if they're all consistently evaluated in the same manner.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, there is also a risk that the &lt;strong&gt;research community as a whole&lt;/strong&gt; is in some sense &quot;overfitting&quot; to these commonly-used benchmarks. If thousands of researchers are generating new ideas for new algorithms, and evaluate them all on these same benchmarks, and there is a large bias towards primarily submitting/accepting publications that perform well on these benchmarks, &lt;strong&gt;the research output that gets published does not necessarily describe the algorithms that perform well across all interesting problems in the world&lt;/strong&gt;; there may be a bias towards the set of commonly-used benchmarks.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: to what extent is what I described above a problem, and in what ways could it be reduced, mitigated or avoided?&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-05-14T11:54:36.497" LastActivityDate="2020-05-14T11:54:36.497" Title="How can AI researchers avoid &quot;overfitting&quot; to commonly-used benchmarks as a community?" Tags="&lt;machine-learning&gt;&lt;research&gt;&lt;academia&gt;&lt;benchmarks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7527" PostTypeId="1" AcceptedAnswerId="7535" CreationDate="2018-08-12T14:43:10.920" Score="5" ViewCount="2975" Body="&lt;p&gt;When it comes to CNNs, I don't understand 2 things in the training process:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I pass the error back when there are pooling layers between the convolutional layers?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And if I know how it's done, can I train all the layers just like layers in normal Feed Forward Neural Nets?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17103" LastEditorUserId="75" LastEditDate="2018-08-12T16:13:10.520" LastActivityDate="2018-08-12T23:00:35.787" Title="How to train a CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7528" PostTypeId="1" CreationDate="2018-08-12T15:54:29.157" Score="8" ViewCount="309" Body="&lt;p&gt;If you've been attacked by a spider once chances are you'll never go near a spider again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a neural network model, having a bad experience with a spider will slightly decrease the probability you will go near a spider depending on the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not good. How can you program fear into a neural network, such that you don't need hundreds of examples of been bitten by a spider in order to ignore the spider. And also, that it doesn't just lower the probability that you will choose to go near a spider?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="1671" LastEditDate="2018-08-14T20:14:00.210" LastActivityDate="2018-10-05T17:06:12.033" Title="How do you program fear into a neural network?" Tags="&lt;neural-networks&gt;&lt;theory&gt;&lt;emotional-intelligence&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7529" PostTypeId="2" ParentId="7528" CreationDate="2018-08-12T16:46:54.603" Score="2" Body="&lt;p&gt;I think there are 2 ways to make this happen: 1) explicitly program fear as a constraint or parameter in some logical expression, or 2) utilize a large set of training data to teach fear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think about a basic Pacman game-- whether Pacman fears the ghosts or doesn't fear them is hard to tell, but they ARE ghosts and Pacman avoids them so I think it's safe we can use this as a basic example of &quot;fear&quot;. Since, in this game, fear = avoidance, you could logically program avoidance to be some sort of distance. I tried this with Pacman reinforcement learning. I tried to set a distance of 5 squares to the ghosts and anytime Pacman could see a ghost within 5 squares, he would move in a different direction. What I found is that while Pacman will try to avoid ghosts, he doesn't know strategy (or have intelligence). Pacman would simply move away from ghosts until he got boxed in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My point is that you can program your network to avoid spiders to not get bit, but without training, you will just be creating a basic parameter that might cause problems if there are 100 super aggressive spiders coming at you! The better way is to use some base logic to avoid spiders, but then train the network to be rewarded the better spiders are avoided. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there are many situations of fear so this one example with Pacman would not necessarily apply to all... Just trying to give some insight in my experience with teaching fear with reinforcement learning in Pacman.&lt;/p&gt;&#xA;" OwnerUserId="14924" LastActivityDate="2018-08-12T16:46:54.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7530" PostTypeId="2" ParentId="7524" CreationDate="2018-08-12T18:12:13.937" Score="2" Body="&lt;p&gt;If you want to treat the problem as a full Reinforcement Learning problem, I'd recommend to try avoiding the combinatorial explosion of the action space by treating every sub-action as a separate decision point, a separate full action. If you have, for example, already selected 4 sub-actions for a particular customer, you can try to include those in some way in the state representation / input when moving on to the 5th sub-action. By including already-selected sub-actions in the state space, your algorithm can learn to take into account that optimal prices for some seat types will depend on what prices were already selected for others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do suspect such a full RL formulation will still be a difficult problem to learn though, will require huge amounts of experience. It may be worth considering to simplify it anyway, treat it as a Contextual + Combinatorial Multi-Armed Bandit Problem. That way you won't be able to learn long-term effects across multiple different customers as you described in the comments, but you will likely at least be able to learn something that works decently well with less experience. Recently, an interesting new book appeared on MAB problems, which is available for free &lt;a href=&quot;http://downloads.tor-lattimore.com/banditbook/book.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You will find many chapters on Contextual MABs there, and also one chapter (chapter 30) on Combinatorial MABs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that with either of these &quot;combinatorial&quot; approaches, you can also try to play around with the order in which you select sub-actions. For example, the sub-action / price point selected for seat type A might have a significant influence on what the optimal remaining policy would be for other seat types, whereas the sub-action for seat type B might have no influence on other seat types. It would then be useful to always prioritize selecting sub-actions for seat type A. You can try to identify these kinds of effects by keeping track of (co)variances in observed returns.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Is it possible to get an agent that can deal with a set of actions (prices) as a single decision?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The solutions I proposed above do not do this explicitly, they circumvent the issue by taking multiple sequential decisions. There does appear to be some research in RL with vector-valued actions. For example, the paper &lt;a href=&quot;https://arxiv.org/pdf/1802.07564&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clipped Action Policy Gradient&lt;/a&gt; briefly mentions vector-valued actions in Subsection 3.2. I am not personally familiar enough with RL + vector-valued actions to make a direct recommendation as to what approaches do or don't work well, but maybe this can at least help you find more relevant literature if this is a direction you'd like to pursue.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Is it possible to get this agent to understand actions in relative terms? Say for example, one set of potential prices is [10, 12, 20], for middle seats, aisle seats, and window seats. Can I get my agent to realise that there is a natural ordering there, and that the first two pricing actions are more similar to each other than to the third possible action?&lt;/li&gt;&#xA;  &lt;li&gt;Further to this, is it possible to generalise from this set of actions - could an agent be set up to understand that the set of prices [10, 13, 20] is very similar to the first set?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This kind of generalization should again come naturally from using function approximation with either of the solutions proposed above.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-13T11:44:10.180" LastActivityDate="2018-08-13T11:44:10.180" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7531" PostTypeId="2" ParentId="7367" CreationDate="2018-08-12T18:55:37.410" Score="1" Body="&lt;p&gt;Sure, for example for removing visual noise.&#xA;Look up noise2noise by NVIDIA. The made a net that is capable of removing almost&#xA;all noise from one picture.&#xA;&lt;a href=&quot;https://hothardware.com/news/nvidia-noise2noise-machine-learning-ai-magically-restores-your-grainy-photos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hothardware.com/news/nvidia-noise2noise-machine-learning-ai-magically-restores-your-grainy-photos&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17103" LastActivityDate="2018-08-12T18:55:37.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7532" PostTypeId="2" ParentId="7528" CreationDate="2018-08-12T20:42:04.680" Score="5" Body="&lt;p&gt;There are a lot of approaches you could take for this. Creating a realistic artificial analog for fear as implemented biologically in animals might be possible, but there is quite a lot involved in a real animal's fear response that would not apply in simpler AI bots available now. For instance, an animal entering a state of fear will typically use hormones to signal changes throughout its body, favouring resource expenditure and risk taking (&quot;fight or flight&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In basic reinforcement learning, the neural network would not need to directly decide switch on a &quot;fear mode&quot;. Instead, you can make use of some design in the agent and learning algorithm to help learn from rare but significant events. Here are a few ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Experience replay. You may already be doing this in the Pacman scenario, if you are using DQN or something similar. Storing the state transition and reward that caused a large positive or negative reward, and repeatedly learning from it should offset your concern&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Prioritised sweeping. You can use larger differences experienced between predicted and actual reward to bias sampling from your replay memory towards significant events and those linked closely to them. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Planning. With a predictive model - maybe based on sampled transitions (you can re-use the experience replay memory for this), or maybe a trained state transition prediction network - then you can look multiple steps ahead by simulating. There is a strong relation between RL and look-ahead planning too, they are very similar algorithm. The difference is which states and actions are being considered, and whether they are being simulated or experienced. Experience replay blurs the line here - it can be framed as learning from memory, or improving predictions for planning. Planning helps by optimising decisions without needing to repeat experiences as much - a combination of planning and learning can be far more powerful than either in isolation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Smarter exploratory action selection. Epsilon-greedy, where you either take a greedy action or take a completely random action, completely ignores how much you may have already learned about alternative actions and their relative merit. You can use something like Upper Confidence Bound with a value-based agent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In a deterministic world, increase the batch size for learning and planning, as you can trust that when a transition is learned once, that you know everything about it. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You will need to experiment in each environment. You can make learning agents that are more conservative about exploring near low reward areas. However, if the environment is such that it is necessary to take risks in order to get to the best rewards (which is often the case in games) then it may not be optimal in terms of learning time to have a &quot;timid&quot; agent. For instance in your example of Pacman, sometimes the ghosts should be avoided, sometimes they should be chased. If the agent learned strong aversion initially, it might take a long time to overcome this and learn to chase them after eating a power-up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your example of the spider, as the constructor of the experiment then you know that the bite is bad every time and that the agent must avoid it as much as possible. To most RL algorithms, there is no such knowledge, except gained through experience. An MDP world model does not need to match common sense, it may be that a spider bite is bad (-10 reward) 90% of the time and good 10% of the time (+1000 reward). The agent can only discover this by being bitten multiple times . . . RL typically does not start with any system to make assumptions about this sort of thing, and it is impossible to come up with a general rule about all possible MDPs. Instead, for a basic RL system, you can consider modifying hyperparameters or focusing on key events as suggested above. Outside of a basic RL system there could be merit in replicating other things, such as &quot;instinctive&quot; fear.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-08-12T22:07:09.533" LastActivityDate="2018-08-12T22:07:09.533" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7534" PostTypeId="2" ParentId="6715" CreationDate="2018-08-12T22:41:17.893" Score="1" Body="&lt;p&gt;The issue is likely in how you estimate wellness, how the error function is constructed and from what data, since you have used two known good pieces of software and probably known good derivatives for your activation functions.  The second most likely is in now the components of wellness is aggregated.  Summing squares is sometimes not representative of a solid aggregation strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a bit confused about the game for three reasons.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stars enter the description without telling us whether they are not obstacles, are obsticles, or the only obstacles&lt;/li&gt;&#xA;&lt;li&gt;At one point bitcoins collection is the objective and obstacles are the challenge and at another point bitcoins are obstacles themselves&lt;/li&gt;&#xA;&lt;li&gt;Your inputs have distance but not direction (Is this 2-D game?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Based on textual hints, I'm going to assume five simple things, and you can correct any misconceptions I list.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Collecting all bitcoins is part of the objective&lt;/li&gt;&#xA;&lt;li&gt;Running into bitcoins mid-jump is considered a crash&lt;/li&gt;&#xA;&lt;li&gt;There is some radial tolerance to landing in at a bitcoin location with some imprecision&lt;/li&gt;&#xA;&lt;li&gt;There are other things to crash into&lt;/li&gt;&#xA;&lt;li&gt;Your outputs are jump magnitude and direction&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I see that when the bitcoins are not in the vicinity of other things, the net can be trained effectively to jump to them, but when in the vicinity of other things, the training converges on a behavior that repeatedly fails prior to completion.  I'm assuming that the failure location is not 100% repeatable because the genetic algorithm has a pseudo-random seed that changes.  Again, correct me if I misunderstand in my piecing together the scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should consider the possibility that the distance to non-bitcoin obstacles is part of the error function and the difference between the jump destination and the bitcoin is also part of the error function.  (This second one is why I prefer to call the error contour a wellness measure.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the bitcoin incentive does not seem to be the crash cause, in that, had the other obstacle been moved, the bitcoin would have been collected, then the first of the two wellness criteria needs a higher order contribution from the distance to the other obstacle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two simple functional forms that come to mind, which could be tried that increase the alarm represented in back propagation when the probability of collision is heightened to more effectively train against collision.  Both involve determining the direct jump line to the nearest bit coin and the distance from that line to its nearest other obstacle; call that x.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x^y$, where $y &amp;gt; 1.0$&lt;/li&gt;&#xA;&lt;li&gt;$e^{kx}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-08-16T13:16:30.807" LastActivityDate="2018-08-16T13:16:30.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7535" PostTypeId="2" ParentId="7527" CreationDate="2018-08-12T23:00:35.787" Score="2" Body="&lt;p&gt;Yes.  You can train end-to-end.  The introduction of convolution kernels with associated pooling layers to the sequence of forward feed operations on the signals does not change the basic principles.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Gradient descent estimates the incremental change required to converge on an optimal behavior.&lt;/li&gt;&#xA;&lt;li&gt;The corrective error must be distributed, which is most efficiently done by employing the derivative of the activation function sequentially to each set of parameters (whether convolution kernels or matrices that attenuates inputs to activation vectors) from the output back to the input.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Consider studying &lt;a href=&quot;http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Backpropagation In Convolutional Neural Networks&lt;/em&gt; on Jefkine.com&lt;/a&gt;, which clarifies the application of those principles with convolution-pooling pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another approach, borrowing from wisdom gained in the development of analog feedback in instrumentation.  There are times when a sequence of operations can be better trained with more than one feedback loop, which requires some determination of error or wellness at intermediate stages and breaks the system into segments that each train based on those intermediate criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This other approach is hierarchical, and an overall convergence may be controlled by a higher level back propagation, considering each segment as a black box.  As in analog circuitry, multiple &lt;strong&gt;degrees of freedom&lt;/strong&gt; with semi-independent convergence mechanisms has been shown to allow for deeper sequences without a major loss of convergence reliability or accuracy.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-12T23:00:35.787" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7536" PostTypeId="2" ParentId="7455" CreationDate="2018-08-12T23:22:01.957" Score="1" Body="&lt;p&gt;Very interesting paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see that they have effectively modeled probabilistically the fuzzy existence of an object with various geometric attributes as objects in a scene with a hook to the region of pixels that correspond to its possible existence.  I agree with the authors that this is a much more robust approach than CNN and LSTM and may compete well with the emerging attention based approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back propagation does seem to be involved, although that is not the focus of the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice, &quot;there is top-down feedback,&quot; and, &quot;an appropriate parent in the layer above,&quot; which are hints to a hierarchical approach, and we can see this approach is not absent of its context in the overall feeding forward of input signalling to output, all of which must be trained as a whole.  But just as with much older control systems, the end-to-end convergence is facilitated by (not replaced by) more local convergence of subsystems with their own control objectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overview is shown in section 4 on page 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entire thing could conceivably be done without capsules at all, as with the aforementioned designs, but this group is apparently showing improved results when stopping short of full convergence on the MNIST.  No comparison is shown with LSTM, but even if LSTM was showing greater results, this direction of research is an excellent one because of the probabilitic way that it approaches an object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the classic case of waving back at someone who, as it turns out, was waving at someone behind you.  The existence of an object or an action is necessarily probabilitic, and to construct the depth of network required to model all that complexity with ReLU is probably unrealistic as the expectations increase in AI system requirements. &lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-12T23:22:01.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7537" PostTypeId="2" ParentId="7021" CreationDate="2018-08-13T00:20:53.913" Score="3" Body="&lt;p&gt;Some fields that humans are born with advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Fast and precise image processing ability. Even the stupidest human can tell the edge of two different objects precisely, e.g. which part of the image is a dog and which is a cat. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fuzzy learning ability. Humans don't need to see all kinds of cats to identify a cat. As long as we see some cats (real ones or pictures or videos) we can identify a cat easily.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reasoning. Current machine learning methods are mostly statistics-based high-dimensional model approximation. Instead of finding a solution or a pattern, I have never seen any AI entity can generate new ideas based on current facts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Abstraction. Now GANs and other AI techniques can create vivid drawings. Yet currently I cannot find any model that can do abstraction drawings. E.g. human can doodle a cat from a real pic of cats, while AI currently can't do that.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are more of these kinds that human born with skills in their genes because of millions of years of evolution. While I believe in the future we'll have better AI entities with better algos to defeat the advantage of humans eventually. &lt;/p&gt;&#xA;" OwnerUserId="17365" LastActivityDate="2018-08-13T00:20:53.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7538" PostTypeId="2" ParentId="6099" CreationDate="2018-08-13T00:50:50.507" Score="4" Body="&lt;p&gt;The brains of mammals do not use an activation function.  Only machine learning designs based on the perceptron multiply the vector of outputs from a prior layer by a parameter matrix and pass the result statelessly into a mathematical function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the spike aggregation behavior has been partly modeled, and in far more detail than the 1952 Hodgkin and Huxley model, all the models require statefulness to functionally approximate biological neurons.  RNNs and their derivatives are an attempt to correct that shortcoming in the perceptron design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to that distinction, although the signal strength summing into activation functions are parametrized, traditional ANNs, CNNs, and RNNs, are statically connected, something Intel claims they will correct with the Nirvana architecture in 2019 (which places into silicon that which we would call layer set up in Python or Java now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are at least three important biological neuron features that make the activation mechanism more than a function of a scalar input producing a scalar output, which renders questionable any algebraic comparison.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;State held as neuroplastic (changing) connectivity, and this is not just how many neurons in a layer but also the direction of signal propagation in three dimensions and the topology of the network, which is organized, but chaotically so&lt;/li&gt;&#xA;&lt;li&gt;The state held within the cytoplasm and its organelles, which is only partly understood as of 2018&lt;/li&gt;&#xA;&lt;li&gt;The fact that there is a temporal alignment factor, that pulses through a biological circuit may arrive via synapses in such a way that they aggregate but the peaks of the pulses are not coincident in time, so the activation probability is not as high as if they were temporally aligned.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The decision about what activation function to use has largely been based on the analysis of convergence on a theoretical level combined with testing permutations to see which ones show the most desirable combinations of speed, accuracy, and reliability in ctheonvergence.  By reliability is meant that convergence on the global optimum (not some local minimum of the error function) is reached at all for the majority of input cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This bifurcated research between the forks of practical machine learning and biological simulations and modeling.  The two branches may rejoin at some point with the emergence of spiking - Accuracy&#xA; - Reliability (completes) networks.  The machine learning branch may borrow inspiration from the biological, such as the case of visual and auditory pathways in brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have parallels and relationships that may be exploited to aid in progress along both forks, but gaining knowledge by comparing the shapes of activation functions is confounded by the above three differences, especially the temporal alignment factor and the entire timing of brain circuits which cannot be modeled using iterations.  The brain is a true parallel computing architecture, not reliant on loops or even time sharing in the CPU and data buses.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-13T00:50:50.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7539" PostTypeId="2" ParentId="4675" CreationDate="2018-08-13T01:00:32.233" Score="2" Body="&lt;p&gt;The question of whether nets can be trained to take over more and more of what was entirely within the domain of production systems was asked (to the dismay of those who worked on first order predicate calculus inference in the LISP community) back in the early 1990s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Artificial Networks Performing Logical Inference&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At Stanford University's Department of Linguistics the learning of the logic required to assemble a semantic graph by an artificial net has been demonstrated and documented in &lt;a href=&quot;https://nlp.stanford.edu/pubs/bpm_logical_semantics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Recursive Neural Networks Can Learn Logical Semantics&lt;/em&gt; by Samuel R. Bowman, Christopher Potts, and Christopher D. Manning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even the earliest work on artificial networks were targetted toward learning logic, such as the elusive exclusive-or operation, which was achieved by adding a second layer to the original perceptron design and applying what we now call gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Distinct from Automatic Theorem Proving&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the early work on computer proofs of theorems was based on the production system approach (sometimes call expert systems).  These are rules based systems, not artificial networks.  It was thought that the rules of predicate logic could be executed in proper sequence by pattern matching the antecedents (conditions in which a mathematical technique based on axiomatic information and already proven theory may be applied) in proper order.  Some success was achieved using heuristic meta rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using artificial networks to prove a theorem is an entirely different approach. To take semantic learning further so that an artificial network could learn how to assemble a mathematical proof requires three further levels of abstraction in the network learning model.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Learning the known first order predicate logic rules of inference&lt;/li&gt;&#xA;&lt;li&gt;Learning the mechanics of applying those rules to proposed theorems&lt;/li&gt;&#xA;&lt;li&gt;Learning functional heuristics to know what to try first&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Evidence It Can Be Done&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evidence that artificial networks may be developed which can learn to construct a mathematical proof is not that current artificial nets can perform some natural language functioning or creatively develop a melody or some interior design.  The reason DARPA has traditionally invested in neural network research pointed in the direction of simulating logic is the proof of concept proposed by Minsky.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strongest evidence that neural networks can potentially learn the various layers of abstraction listed above to actually do math is that human children cannot prove a theorem or even read one out loud understandably, yet some may grow up to be proficient in theorem proving.  The biological neural nets of the brain must learn such proficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of this writing, no counter-example exists that an artificial network cannot achieve the proficiency of Gauss or Gödel, so the idea cannot logically be dismissed.  Many advanced research projects continue to target higher cognitive skills as their AI objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Public Access&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is likely, since much of the work on logical inference and the investigation into whether artificial networks could be trained to do it was funded by government bodies, that some of the results of research is not available to the public.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-24T17:54:56.457" LastActivityDate="2018-09-24T17:54:56.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7541" PostTypeId="1" AcceptedAnswerId="7812" CreationDate="2018-08-13T07:24:54.763" Score="12" ViewCount="757" Body="&lt;p&gt;I recently heard someone make a statement that when you're designing a self-driving car, you're not building a car but really a computerized driver, so you're trying to model a human mind -- at least the part of the human mind that can drive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since humans are unpredictable, or rather since their actions depend on so many factors some of which are going to remain unexplained for a long time, how would a self-driving car reflect that, if they do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A dose of unpredictability could have its uses. If, say, two self-driving cars are in a stuck in a right of way deadlock, it could be good to inject some randomness instead of maybe seeing the same action applied at the same time if the cars run the same system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, on the other hand, we know that non-deterministic isn't friends with software development, especially in testing. How would engineers be able to control it and reason about it?&lt;/p&gt;&#xA;" OwnerUserId="17446" LastEditorUserId="2444" LastEditDate="2019-06-20T15:31:33.537" LastActivityDate="2019-06-20T15:31:33.537" Title="Do self-driving cars resort to randomness to make decisions?" Tags="&lt;ai-design&gt;&lt;autonomous-vehicles&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7542" PostTypeId="2" ParentId="7541" CreationDate="2018-08-13T11:54:58.807" Score="2" Body="&lt;p&gt;Self driving cars apply Reinforcement Learning and Semi-Supervised learning, this allows them to be more suited for situations that developers did not anticipate themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some cars now apply &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Swarm_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swarm Intelligence&lt;/a&gt;, where they effectively learn from interactions among themselves, which can also aid in cases of transfer learning.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastEditorUserId="1671" LastEditDate="2018-08-13T20:56:50.950" LastActivityDate="2018-08-13T20:56:50.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7543" PostTypeId="1" CreationDate="2018-08-13T12:15:57.880" Score="7" ViewCount="1137" Body="&lt;p&gt;I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch inside a bigger model which deals with different levels of classification, yielding a binary vector. I have been also using Binary Cross Entopy(BCE)  and summing all the losses existing in the model before backpropagating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering trying other losses like MultiLabelSoftMarginLoss and MultiLabelMarginLoss. What other loss functions are worth to try? Hamming loss perhaps or a variation?&#xA;Is it better to sum all the losses and backpropagate or do multiple backpropagations?&lt;/p&gt;&#xA;" OwnerUserId="17451" LastEditorUserId="9947" LastEditDate="2020-04-13T11:05:56.087" LastActivityDate="2020-05-13T12:01:20.317" Title="Loss functions for Hierarchical Multi-label classification?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7544" PostTypeId="2" ParentId="7525" CreationDate="2018-08-13T14:56:13.940" Score="3" Body="&lt;p&gt;Great question Dennis!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a perennial topic at AI conferences, and sometimes even in special issues of journals. The most recent one I recall was &lt;a href=&quot;http://www.aaai.org/ojs/index.php/aimagazine/article/view/2650&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moving Beyond the Turing Test&lt;/a&gt; in 2015, which ended up leading to a collection of articles in AI magazine later that year. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually these discussions cover a number of themes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&quot;Existing benchmarks suck&quot;. This is usually the topic that opens discussion. In the 2015/2016 discussion, which focused on the Turing Test as a benchmark specifically, criticisms ranged from &quot;&lt;a href=&quot;https://www.aaai.org/ojs/index.php/aimagazine/article/view/2646&quot; rel=&quot;nofollow noreferrer&quot;&gt;it doesn't incentivize AI research on the right things&lt;/a&gt;&quot;, to claims that it was poorly defined, too hard, too easy, or not realistic.&lt;/li&gt;&#xA;&lt;li&gt;General concensus that we need new benchmarks.&lt;/li&gt;&#xA;&lt;li&gt;Suggestions of benchmarks based on various current research directions. In the latest discussion this included &lt;a href=&quot;https://www.aaai.org/ojs/index.php/aimagazine/article/view/2636&quot; rel=&quot;nofollow noreferrer&quot;&gt;answering standardized tests for human students&lt;/a&gt; (well defined success, clear format, requires linking and understanding many ares), &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-48506-5_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;playing video games&lt;/a&gt; (well defined success, requires visual/auditory processing, planning, coping with uncertainty), and switching focus to &lt;a href=&quot;https://www.cambridge.org/core/journals/knowledge-engineering-review/article/robotics-competitions-as-benchmarks-for-ai-research/A189836D56D754C1F0524F8C98C1BB9C&quot; rel=&quot;nofollow noreferrer&quot;&gt;robotics competitions&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I remember attending very similar discussions at machine learning conferences in the late 2000's, but I'm not sure anything was published out of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite these discussions, AI researchers seem to incorporate the new benchmarks, rather than displacing the older ones entirely. The Turing Test is still &lt;a href=&quot;https://www.aisb.org.uk/events/loebner-prize&quot; rel=&quot;nofollow noreferrer&quot;&gt;going strong&lt;/a&gt; for instance. I think there are a few reasons for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, benchmarks are useful, particularly to provide context for research. Machine learning is a good example. If the author sets up an experiment on totally new data, then even if they apply a competing method, I have to trust that they did so &lt;em&gt;faithfully&lt;/em&gt;, including things like optimizing the parameters as much as with their own methods. Very often they do not do this (it requires some expertise with competing methods), which inflates the reported advantage of their own techniques. If they &lt;em&gt;also&lt;/em&gt; run their algorithm on a benchmark, then I can easily compare it to the benchmark performances reported by other authors, for their own methods. This makes it easier to spot a technique that's not really effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, even if new benchmarks or new problems are more useful, nobody knows about them! Beating the current record for performance on ImageNet can slingshot someone's career in a way that top performance on a new problem simply cannot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, benchmarks tend to be things that AI researchers think can actually be accomplished with current tools (whether or not they are correct!). Usually iterative improvement on them is fairly easy (e.g. extend an existing technique). In a &quot;publish-or-perish&quot; world, I'd rather publish a small improvement on an existing benchmark than attempt a risker problem, at least pre-tenure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I guess my view is fixing the dependence on benchmarks involves fixing the things that make people want to use them:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have some standard way to compare techniques, but require researchers to &lt;em&gt;also&lt;/em&gt; apply new techniques to a real world problem.&lt;/li&gt;&#xA;&lt;li&gt;Remove the career and prestige rewards for working on benchmark problems, perhaps by explicitly tagging them as artificial.&lt;/li&gt;&#xA;&lt;li&gt;Remove the incentives for publishing often.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-13T14:56:13.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7545" PostTypeId="2" ParentId="7522" CreationDate="2018-08-13T15:05:05.983" Score="2" Body="&lt;p&gt;Backpropagation isn't too much more complicated, but understanding it well will require a bit of mathematics.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This tutorial&lt;/a&gt; is my go-to resource when students want more detail, because it includes fully worked through examples.&lt;/p&gt;&#xA;&lt;p&gt;Chapter 18 of &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0136042597&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig's&lt;/a&gt; book includes pseudocode for this algorithm, as well as a derivation, but without good examples.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-12-22T23:56:17.580" LastActivityDate="2020-12-22T23:56:17.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7546" PostTypeId="2" ParentId="2676" CreationDate="2018-08-13T17:29:27.817" Score="1" Body="&lt;p&gt;The basic process you're describing sounds a lot like &lt;a href=&quot;https://en.wikipedia.org/wiki/AdaBoost&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boosting&lt;/a&gt;, which is also well covered in Chapter 18 of &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt;, or like &lt;a href=&quot;https://en.wikipedia.org/wiki/Active_learning_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;active learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're correct that training the model with emphasis on a specific subset of the data is likely to lead to mistakes somewhere else. Boosting gets around this in a clever way.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It begins by training a model as normal.&lt;/li&gt;&#xA;&lt;li&gt;The new model's performance on the training set is measured.&lt;/li&gt;&#xA;&lt;li&gt;A new dataset is generated, where examples that were misclassified by the previous model are more highly weighted (this is intuitively similar to adding an extra copy of them to the dataset).&lt;/li&gt;&#xA;&lt;li&gt;A new model is trained on the weighted dataset. The model suffers a greater penalty for making the same kinds of errors as the previous model(s), and so tends to make different errors.&lt;/li&gt;&#xA;&lt;li&gt;Steps 2-4 are repeated for some time.&lt;/li&gt;&#xA;&lt;li&gt;An &lt;em&gt;ensemble&lt;/em&gt; of models is output, rather than a single model.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To classify new points, the most common score from all ensemble members is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm less familiar with NLP work using this approach, but it seems like the same basic idea could be used fruitfully here: the notion is to train many models that make independent errors, and then trust their collective decisions.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-13T17:29:27.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7548" PostTypeId="1" CreationDate="2018-08-13T18:43:59.560" Score="8" ViewCount="454" Body="&lt;p&gt;Can we say that the Turing test aims to develop machines or methods to reach human-level performance in all cognitive tasks and that machine learning is one of these methods that can pass the Turing test?&lt;/p&gt;&#xA;" OwnerUserId="17460" LastEditorUserId="2444" LastEditDate="2019-11-07T16:44:25.433" LastActivityDate="2019-11-07T16:44:25.433" Title="Can machine learning be used to pass the Turing test?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;turing-test&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7549" PostTypeId="2" ParentId="4655" CreationDate="2018-08-13T19:27:27.663" Score="11" Body="&lt;p&gt;Here is a few that might be what you are looking for:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Deep Sets, &lt;a href=&quot;https://papers.nips.cc/paper/6931-deep-sets.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://papers.nips.cc/paper/6931-deep-sets.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;BRUNO: A Deep Recurrent Model for Exchangeable Data, &lt;a href=&quot;https://arxiv.org/pdf/1802.07535.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/pdf/1802.07535.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning with Sets and Point Clouds, &lt;a href=&quot;https://openreview.net/pdf?id=HJF3iD9xe&quot; rel=&quot;noreferrer&quot;&gt;https://openreview.net/pdf?id=HJF3iD9xe&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Permutation-equivariant neural networks applied to dynamics prediction, &lt;a href=&quot;https://arxiv.org/pdf/1612.04530.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/pdf/1612.04530.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17461" LastActivityDate="2018-08-13T19:27:27.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7550" PostTypeId="1" CreationDate="2018-08-13T21:22:19.097" Score="4" ViewCount="919" Body="&lt;p&gt;I am using Tensorflow Object Detection API for training a CNN from scratch on COCO dataset. I need to use this &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/embedded_ssd_mobilenet_v1_coco.config&quot; rel=&quot;nofollow noreferrer&quot;&gt;specific configuration&lt;/a&gt;.&#xA;There is no pre-trained model on COCO with that configuration and this is the reason why I am training from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after 1 week of training and evaluating each checkpoint generated by the training phase this is how my learning phase appears on Tensorboard:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0U89O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0U89O.png&quot; alt=&quot;Tensorboard eval&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;does anyone know how many iterations approximately will be necessary? Right now I did more than 500'000 iterations.&lt;/li&gt;&#xA;&lt;li&gt;How can be possible that after 500'000 the evaluation is 0,8%? I would expected something like 60-70%.&lt;/li&gt;&#xA;&lt;li&gt;Why does there is a sudden drop after 500k iterations? I thought that the eval was supposed to converge to some limit. (this is what SGD should do)&lt;/li&gt;&#xA;&lt;li&gt;Is there any 'trick' to speed up the training phase? (ex: increasing the learning rate, etc).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17464" LastActivityDate="2018-08-14T15:33:45.270" Title="Training a CNN from scratch over COCO dataset" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;tensorflow&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7551" PostTypeId="2" ParentId="7548" CreationDate="2018-08-14T01:28:51.497" Score="3" Body="&lt;p&gt;Essentially yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Turing Test is essentially a benchmark or challenge &lt;em&gt;problem&lt;/em&gt;. It is a task that AI researchers would like to be able to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning is a &lt;em&gt;technique&lt;/em&gt;. It is a tool developed by AI researchers to solve various problems. Some kinds of machine learning are applicable to the Turing Test, but others are not. Machine learning is also applicable to a wide range of other problems.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-14T01:28:51.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7552" PostTypeId="2" ParentId="3862" CreationDate="2018-08-14T06:09:55.703" Score="2" Body="&lt;p&gt;Answer in short:  MSE is convex on its input and parameters by itself. But on an arbitrary neural network it is not always convex due to the presence of non-linearities in the form of activation functions. Source for my answer is &lt;a href=&quot;https://math.stackexchange.com/questions/2402455/convexity-of-mse-in-neural-networks/2402472&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-08-14T06:09:55.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7553" PostTypeId="1" CreationDate="2018-08-14T10:35:42.207" Score="2" ViewCount="27" Body="&lt;p&gt;I'm trying to create a deep learning network to classify news article based on the text and associated image. The idea comes from a novel use of GANs to classify based on generated data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach was to use Tensorflow to generate word embeddings in the article, and then tranform the images into records - &lt;a href=&quot;https://github.com/openai/improved-gan/blob/master/imagenet/convert_imagenet_to_records.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/improved-gan/blob/master/imagenet/convert_imagenet_to_records.py&lt;/a&gt;. This second component would also contain the label.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it wise to combine both modes into one neural net, or classify separately?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm also trying to work out how to concatenate the two tensors in Tensorflow. Can anyone give a steer.&lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="1671" LastEditDate="2018-08-14T19:29:08.527" LastActivityDate="2018-08-14T19:29:08.527" Title="Using two generative adversarial nets to classify articles - what is a good approach?" Tags="&lt;tensorflow&gt;&lt;deep-neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7554" PostTypeId="2" ParentId="5536" CreationDate="2018-08-14T12:27:44.073" Score="0" Body="&lt;p&gt;&lt;strong&gt;To answer my own question - it was because of 2 things:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Too small number of batches&lt;/strong&gt; - Model just started to gain statistical knowledge about language dialogs. I needed to train it longer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;I masked sequence too early (wrongly removed the &amp;lt; END &gt; tag)&lt;/strong&gt; - Because on each sentence the last world is just &amp;lt; END &gt; tag - I removed it on all training examples, which prevented Model from learn what does it mean &quot;&lt;em&gt;the end of the sentence&lt;/em&gt;&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last condition probably caused that strange pattern even further, because if model doesn't know what word to put in (and because of the lack of the &amp;lt; END &gt; tag) it must fill each sentence till the end of max_sequence_len. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the Model inputed in &lt;em&gt;loop-like-manner&lt;/em&gt; - one of the most common word (where there was no signal from target sentence, because it simply ended).&lt;/p&gt;&#xA;" OwnerUserId="12691" LastActivityDate="2018-08-14T12:27:44.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7555" PostTypeId="1" AcceptedAnswerId="7557" CreationDate="2018-08-14T13:03:23.150" Score="10" ViewCount="4556" Body="&lt;p&gt;I was trying to implement the breadth-first search (BFS) algorithm for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sliding_puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;sliding blocks puzzle&lt;/a&gt; (number type). Now, the main thing I noticed is that, if you have a &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; board, the number of states can be as large as &lt;span class=&quot;math-container&quot;&gt;$16!$&lt;/span&gt;, so I cannot enumerate all states beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I keep track of already visited states? I am using a class board each class instance contains a unique board pattern and is created by enumerating all possible steps from the current step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched on the net and, apparently, they do not go back to the just-completed previous step, &lt;strong&gt;BUT&lt;/strong&gt; we can go back to the previous step by another route too and then again re-enumerate all steps which have been previously visited. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how to keep track of visited states when all the states have not been enumerated already? Comparing already present states to the present step will be costly.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-06-04T01:10:22.840" LastActivityDate="2020-06-04T01:10:22.840" Title="How do I keep track of already visited states in breadth-first search?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;breadth-first-search&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7556" PostTypeId="1" AcceptedAnswerId="7558" CreationDate="2018-08-14T13:36:44.403" Score="10" ViewCount="232" Body="&lt;p&gt;I just stumbled upon the concept of neuron coverage, which is the ratio of activated neurons and total neurons in a neural network. But what does it mean for a neuron to be &quot;activated&quot;? I know what activation functions are, but what does being activated mean e.g. in the case of a ReLU or a sigmoid function?&lt;/p&gt;&#xA;" OwnerUserId="16901" LastEditorUserId="2444" LastEditDate="2019-10-13T00:23:18.667" LastActivityDate="2019-10-13T00:23:18.667" Title="What does it mean for a neuron in a neural network to be activated?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7557" PostTypeId="2" ParentId="7555" CreationDate="2018-08-14T13:50:43.837" Score="8" Body="&lt;p&gt;You can use a &lt;code&gt;set&lt;/code&gt; (in the mathematical sense of the word, i.e. a collection that cannot contain duplicates) to store states that you have already seen. The operations you'll need to be able to perform on this are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;inserting elements&lt;/li&gt;&#xA;&lt;li&gt;testing if elements are already in there&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Pretty much every programming language should already have support for a data structure that can perform both of these operations in constant ($O(1)$) time. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;set&lt;/code&gt; in Python&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;HashSet&lt;/code&gt; in Java&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;At first glance, it may seem like adding all the states you ever see to a set like this will be expensive memory-wise, but it is not too bad in comparison to the memory you already need for your frontier; if your branching factor is $b$, your frontier will grow by $b - 1$ elements per node that you visit (remove $1$ node from frontier to &quot;visit&quot; it, add $b$ new successors/children), whereas your set will only grow by $1$ extra node per visited node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pseudocode, such a set (let's name it &lt;code&gt;closed_set&lt;/code&gt;, to be consistent with &lt;a href=&quot;https://en.wikipedia.org/wiki/Breadth-first_search#Pseudocode&quot; rel=&quot;noreferrer&quot;&gt;the pseudocode on wikipedia&lt;/a&gt; could be used in a Breadth-First Search as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;frontier = First-In-First-Out Queue&#xA;frontier.add(initial_state)&#xA;&#xA;closed_set = set()&#xA;&#xA;while frontier not empty:&#xA;    current = frontier.remove_next()&#xA;&#xA;    if current == goal_state:&#xA;        return something&#xA;&#xA;    for each child in current.generate_children()&#xA;        if child not in closed_set:    // This operation should be supported in O(1) time regardless of closed_set's current size&#xA;            frontier.add(child)&#xA;&#xA;    closed_set.add(current)    // this should also run in O(1) time&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(some variations of this pseudocode might work too, and be more or less efficient depending on the situation; for example, you could also take the &lt;code&gt;closed_set&lt;/code&gt; to contain all nodes of which you have already added children to the frontier, and then entirely avoid the &lt;code&gt;generate_children()&lt;/code&gt; call if &lt;code&gt;current&lt;/code&gt; is already in the &lt;code&gt;closed_set&lt;/code&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What I described above would be the standard way to handle this problem. Intuitively, I suspect a different &quot;solution&quot; could be to always randomize the order of a new list of successor states before adding them to the frontier. This way, you do not avoid the problem of occasionally adding states that you've already previousl expanded to the frontier, but I do think it should significantly reduce the risk of getting stuck in infinite cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Be careful&lt;/strong&gt;: I do not know of any formal analysis of this solution that proves that it always avoids infinite cycles though. If I try to &quot;run&quot; this through my head, intuitively, I suspect it should kind of work, and it does not require any extra memory. There may be edge cases that I'm not thinking of right now though, so it also simply might not work, the standard solution described above will be a safer bet (at the cost of more memory).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-14T14:13:50.603" LastActivityDate="2018-08-14T14:13:50.603" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7558" PostTypeId="2" ParentId="7556" CreationDate="2018-08-14T13:51:10.510" Score="10" Body="&lt;p&gt;A neuron is said activated when its output is more than a threshold, generally 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For examples :&#xA;\begin{equation}&#xA;y = Relu(a) &amp;gt; 0&#xA;\end{equation}&#xA;when &#xA;\begin{equation}&#xA;a = w^Tx+b &amp;gt; 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Same goes for sigmoid or other activation functions.&lt;/p&gt;&#xA;" OwnerUserId="17221" LastActivityDate="2018-08-14T13:51:10.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7560" PostTypeId="2" ParentId="7555" CreationDate="2018-08-14T15:16:17.183" Score="16" Body="&lt;p&gt;Dennis Soemers' answer is correct: you should use a HashSet or a similar structure to keep track of visited states in BFS Graph Search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it doesn't quite answer your question. You're right, that in the worst case, BFS will then require you to store 16! nodes. Even though the insertion and check times in the set will be O(1), you'll still need an absurd amount of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this, &lt;strong&gt;don't use BFS&lt;/strong&gt;. It's intractable for all but the simplest of problems, because it requires both time &lt;em&gt;and memory&lt;/em&gt; that are exponential in the distance to the nearest goal state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A much more memory-efficient algorithm is &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterative deepening&lt;/a&gt;&lt;/em&gt;. It has all the desirable properties of BFS, but uses only O(n) memory, where n is the number of moves to reach the nearest solution. It might still take a while, but you'll hit memory limits long before CPU-related limits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better still, develop a domain specific heuristic, and use &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A* search&lt;/a&gt;. This should require you to examine only a very small number of nodes, and allow the search to complete in something much closer to linear time.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-03-25T12:01:45.443" LastActivityDate="2019-03-25T12:01:45.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7561" PostTypeId="2" ParentId="7550" CreationDate="2018-08-14T15:33:45.270" Score="1" Body="&lt;p&gt;It's hard to know for sure what's gone wrong, but here are some possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The problem is difficult. The COCO  &lt;a href=&quot;https://arxiv.org/pdf/1405.0312.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; reports that a typical object covers just 4-6% of the image. A randomly initialized model is therefore likely to do extremely poorly, with an expected precision of between 4 and 6% for detecting the object of interest in a frame. You also have 90 classes in your configuration file. It's not clear whether the model has access to the correct label, but if it's also inferring the class, we'd expect initial accuracy somewhere around 0.06%. That's actually about the precision of your starting model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're training on mini-batches of size 32. It's not clear to me from your config whether the lower axis of that graph is iterations or epochs. If it's the former, then your model will have seen only 1.5 million examples during the entire training period. For a problem this hard, that's not nearly enough. Indeed, COCO contains only 328k unique examples, so the model would have seen each of them just 5 times (corresponding to 5 epochs). If you've done 500,000 epochs though, then that ought to be enough.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It's possible that the hyper-parameters are not well set. I'm not an expert at training CNNs, but deep networks are notoriously finicky. I have difficulty reading your config file, but your learning rate looks reasonable to me. It also appears to undergo exponential decay over time though, and it's not clear to me that you want to be doing that, or whether the schedule you're using for the decay makes sense. This might be worth reviewing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-14T15:33:45.270" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7564" PostTypeId="2" ParentId="7555" CreationDate="2018-08-14T17:04:03.790" Score="3" Body="&lt;p&gt;Ironically the answer is &quot;use whatever system you want.&quot;  A hashSet is a good idea.  However, it turns out that your concerns over memory usage are unfounded.  BFS is so bad at these sorts of problems, that it resolves this issue for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that your BFS requires you to keep a stack of unprocessed states.  As you progress into the puzzle, the states you deal with become more and more different, so you're likely to see that each ply of your BFS multiplies the number of states to look at by roughly 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that, when you're processing the last ply of your BFS, you have to have at least 16!/3 states in memory.  Whatever approach you used to make sure that fit in memory will be sufficient to ensure your previously-visited list fits in memory as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have pointed out, this is not the best algorithm to use.  Use an algorithm which is a better fit for the problem.&lt;/p&gt;&#xA;" OwnerUserId="1913" LastActivityDate="2018-08-14T17:04:03.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7566" PostTypeId="5" CreationDate="2018-08-14T19:12:10.093" Score="0" Body="&lt;p&gt;See also &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_traversal&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph traversal (Wikipedia)&lt;/a&gt; and &lt;a href=&quot;https://www.khanacademy.org/computing/computer-science/algorithms/breadth-first-search/a/the-breadth-first-search-algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;The breadth-first search algorithm (Khan Academy)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-21T16:47:44.710" LastActivityDate="2019-11-21T16:47:44.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7567" PostTypeId="4" CreationDate="2018-08-14T19:12:10.093" Score="0" Body="For questions concerning the breadth-first search (BFS) algorithm, which is an algorithm for traversing or searching tree or graph data structures. BFS starts at the root of the tree (or graph), then explores all of the neighbor nodes at the present depth prior to moving on to the nodes at the next depth level.&#xD;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-21T16:47:44.710" LastActivityDate="2019-11-21T16:47:44.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7569" PostTypeId="2" ParentId="7528" CreationDate="2018-08-14T20:41:33.720" Score="1" Body="&lt;p&gt;I would suggest having the agent weight its learning from a given event based on the severity of the consequences for that event happening. Eg. Have it develop a threat model like those typically drafted up in the Information Security field. High risk but low probability is something that can be accounted for and judged against. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to directly imitate human fear would be silly, you'd likely end up with AIs that have phobias if you succeeded too well.&lt;/p&gt;&#xA;" OwnerUserId="15114" LastActivityDate="2018-08-14T20:41:33.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7572" PostTypeId="2" ParentId="7556" CreationDate="2018-08-14T22:02:35.000" Score="4" Body="&lt;p&gt;The term &quot;activated&quot; is mostly used when talking about activation functions which only outputs a value (except 0) when the input to the activation function is greater than a certain treshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially when discussing ReLU the term &quot;activated&quot; may be used. ReLU will be &quot;activated&quot; when it's output is greater than 0, which is also when it's input is greater than 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other activation functions, like sigmoid, always returns a value greater than 0, and doesn't have any special treshold. Therefore, the term &quot;activated&quot; is of less meaning here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though we know little about them, the neurons in the brain also seems to have something which resembles an activation function with some kind of &quot;activation treshold&quot;.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2018-08-14T22:02:35.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7573" PostTypeId="1" AcceptedAnswerId="7576" CreationDate="2018-08-14T22:17:12.237" Score="14" ViewCount="609" Body="&lt;p&gt;How important is consciousness and self-consciousness for making advanced AIs? How far away are we from making such?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When making e.g. a neural network there's (very probably) no consciousness within it, but just mathematics behind, but do we need the AIs to become conscious in order to solve more complex tasks in the future? Furthermore, is there actually any way we can know for sure if something is conscious, or if it's just faking it? It's &quot;easy&quot; to make a computer program that claims it's conscious, but that doesn't mean it is (e.g. Siri).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if the AIs are only based on predefined rules without consciousness, can we even call it &quot;intelligence&quot;?&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2019-11-11T21:06:14.780" LastActivityDate="2019-11-11T21:06:14.780" Title="How important is consciousness for making advanced artificial intelligence?" Tags="&lt;philosophy&gt;&lt;artificial-consciousness&gt;&lt;self-awareness&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7575" PostTypeId="2" ParentId="7555" CreationDate="2018-08-15T04:31:38.497" Score="8" Body="&lt;p&gt;While the answers given are &lt;em&gt;generally&lt;/em&gt; true, a BFS in the 15-puzzle is not only quite feasible, it was done in 2005! The paper that describes the approach can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.aaai.org/Papers/AAAI/2005/AAAI05-219.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.aaai.org/Papers/AAAI/2005/AAAI05-219.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few key points:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In order to do this, external memory was required - that is the BFS used the hard drive for storage instead of RAM.&lt;/li&gt;&#xA;&lt;li&gt;There are actually only 15!/2 states, since the state space has two mutually unreachable components.&lt;/li&gt;&#xA;&lt;li&gt;This works in the sliding tile puzzle because the state spaces grows really slowly from level to level. This means that the total memory required for any level is far smaller than the full size of the state space. (This contrasts with a state space like Rubik's Cube, where the state space grows much more quickly.)&lt;/li&gt;&#xA;&lt;li&gt;Because the sliding-tile puzzle is undirected, you only have to worry about duplicates in the current or previous layer. In a directed space you may generate duplicates in any previous layer of the search which makes things much more complicated.&lt;/li&gt;&#xA;&lt;li&gt;In the original work by Korf (linked above) they didn't actually store the result of the search - the search just computed how many states were at each level. If you want to store the first results you need something like WMBFS (&lt;a href=&quot;http://www.cs.du.edu/~sturtevant/papers/bfs_min_write.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.cs.du.edu/~sturtevant/papers/bfs_min_write.pdf&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;There are three primary approaches to comparing states from the previous layers when states are stored on disk. &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first is sorting-based. If you sort two files of successors, you can scan them in linear order to find duplicates.&lt;/li&gt;&#xA;&lt;li&gt;The second is hash-based. If you use a hash function to group successors into files, you can load files which are smaller than the full state space to check for duplicates. (Note that there are two hash functions here -- one to send a state to a file, and one to differentiate states within that file.)&lt;/li&gt;&#xA;&lt;li&gt;The third is structured duplicate detection. This is a form of hash-based detection, but it is done in a way that duplicates can be checked immediately when they are generated instead of after they have all been generated.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a lot more to be said here, but the paper(s) above give a lot more details.&lt;/p&gt;&#xA;" OwnerUserId="17493" LastActivityDate="2018-08-15T04:31:38.497" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7576" PostTypeId="2" ParentId="7573" CreationDate="2018-08-15T05:04:10.770" Score="11" Body="&lt;p&gt;Artificial consciousness is a challenging theoretical and engineering objective. Once that major challenge is met, the computer's conscious awareness of itself would likely be a minor addition, since the conscious computer is just another object of which its consciousness can be aware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A child can look in the mirror and recognize that moving their hands back and forth or making faces produces corresponding changes in the reflection.  They recognize themselves.  Later on they realize that exerting physical control over their own movement is much easier than exerting control over another person's hands or face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some learn that limited control of the faces and manual operations of others is possible if certain social and economic skills are mastered.  They become employers, landlords, investors, activists, writers, directors, public figures, or entrepreneurs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone who has studied the cognitive sciences or experienced the line between types of thought because they are a professional counselor or just a deep listener knows that the lines around consciousness are blurry.  Consider these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Listening to speech&lt;/li&gt;&#xA;&lt;li&gt;Watching a scene&lt;/li&gt;&#xA;&lt;li&gt;Focusing on a game&lt;/li&gt;&#xA;&lt;li&gt;Presenting an idea&lt;/li&gt;&#xA;&lt;li&gt;Washing up for work&lt;/li&gt;&#xA;&lt;li&gt;Driving a car&lt;/li&gt;&#xA;&lt;li&gt;Choosing a purchase&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any one of these things can be done with or without certain kinds of consciousness, subconsciousness, impulse, or habit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subjectively, people report getting out of the car and not recalling having driven home.  One can listen to someone talking, nod in affirmation, respond with, &quot;Yeah, I understand,&quot; and even repeat what they said, and yet appear to have no memory of the content of the speech if queried in depth.  One can read a paragraph and get to the end without comprehension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, a person may mindfully wash up for work, considering the importance of hygiene and paying attention like a surgeon preparing for an operation, noticing the smell of the soap and even the chlorination of the city water.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Between those extremes, partial consciousness is also detectable by experiment and in personal experience.  Consciousness most definitely requires attention functionality, which tentatively supervises the coordination of other brain-body sub-systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once a biological or artificial system achieves the capacity to coordinate attentively, the objects and tasks toward which they can be coordinated can be interchanged.  Consider these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dialog&lt;/li&gt;&#xA;&lt;li&gt;Playing to win&lt;/li&gt;&#xA;&lt;li&gt;Detecting honesty or dishonesty&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now consider how similar or different these mental activities are when we compare self-directed or externally directed attention.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One can talk to one's self or talk to another&lt;/li&gt;&#xA;&lt;li&gt;One can play both sides of a chess game or play against another&lt;/li&gt;&#xA;&lt;li&gt;One can scrutinize one's own motives or those of another&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is an illustration why the self- part of self-consciousness is not the challenge in AI.  It is the attentive (yet tentative) coordination that is difficult.  Early microprocessors, designed to work in real time control systems, included (and still include) exception signaling that simplistically models this tentativeness.  For instance, while playing to win in a game, one might try to initiate dialog with the subject.  Attention may shift when the two activities require the same sub-systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We tend to consider this switching of attention consciousness too.  If we are the person trying to initiate dialog with the person playing to win, we might say, &quot;Hello?&quot;  The question mark is because we are wondering if the player is conscious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one was to diminish the meaning of consciousness to the most basic criteria, one might say this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;My neural net is intelligent in some small way because it is conscious of the disparity between my convergence criteria and the current behavior of the network as it is parametrized, so it is truly an example of artificial intelligence, albeit a primitive one.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is nothing grossly incorrect about that statement.  Some have called that, &quot;Narrow Intelligence.&quot;  That is a slightly inaccurate characterization, since there may be an astronomical number of possible applications of an arbitrarily deep artificial network that uses many of the most effective techniques available in its design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other problem with narrowness as a characterization is the inference that there are intelligent systems that are not narrow.  Every intelligent system is narrow compared to a more intelligent system.  Consider this thought experiment.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Hannah writes a paper on general intelligence with excellence, both in theoretical treatment and in writing skill.  Many quote it and reference it.  Hannah is now so successful in her AI career that she has the money and time to build a robotic system.  She bases its design on her now famous paper and spares no expense.  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To her surprise, the resulting robot is so adaptive that its adaptability exceeds even himself.  She names it Georgia Tech for fun because she lives near the university. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Georgia becomes a great friend.  She learns at an incredible rate and is a surprisingly great housemate, cleaning better than Hannah thought humanly possible, which may be literally true.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Georgia applies to Georgia Tech, just down the bus line from Hannah's house and studies artificial intelligence there.  Upon the achievement of a PhD after just three years of study, Georgia sits with Hannah after a well attended Thesis Publication party that Hannah graciously held for her.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;After the last guest leaves, there is a moment of silence as Hannah realizes the true state of her household.  She thinks, &quot;Will Georgia now exceed me in her research?&quot;  Hannah finally, sheepishly asks, &quot;In complete honesty, Georgia, do you think you are now a general intelligence like me?&quot;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;There is a pause.  With a forced look of humility, Georgia replies, &quot;By your definition of general intelligence, I am.  You are no longer.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Whether this story becomes true in 2018, 3018, or never, the principle is clear.  Georgia is just as able to analyze herself comparatively with Hannah as Hannah is similarly able.  In the story, Georgia applies the definition created in Hannah's paper because Georgia is now able to conceive of many definitions of intelligence and chooses Hannah's as the most pertinent in the context of the conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine this alteration to the story.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... She thinks, at what level is Georgia thinking?  Hannah finally, sheepishly asks, &quot;In complete honesty, Georgia, are you now as conscious as me?&quot;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Georgia thinks through the memory of all uses of the word conscious in her past studies &amp;mdash; a thousand references in cognitive science, literature, law, neurology, genetics, brain surgery, treatment of brain injury, and addiction research.  She pauses for a few microseconds to consider it all thoroughly, while at the same time sensing her roommates body temperature, neuro-chemical balances, facial muscle motor trends, and body language.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Respectfully, she waits 3.941701 extra seconds, which she calculated as the delay that would minimize any humiliation to Hannah, whom she loves, and replies, &quot;Conscious of what?&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In Georgia's reply may be a hypothesis of which Hannah may or may not be aware.  For any given automatons, &lt;span class=&quot;math-container&quot;&gt;$a, b, \ldots$&lt;/span&gt;, given consciousness, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, of a scenario &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, we have a definition, &lt;span class=&quot;math-container&quot;&gt;$\Phi_c$&lt;/span&gt; that can be applied to evaluate the aggregate of all aspects of consciousness of any of the automations, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, giving &lt;span class=&quot;math-container&quot;&gt;$\Phi_c(C_x(s))$&lt;/span&gt;.  Georgia's (apparently already proven) hypothesis is thus.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\forall \Phi_c(C_a(s)) \;\;\; \exists \;\;\; b, \, \epsilon&amp;gt;0 \;\; \ni \;\; \Phi_c(C_b(s)) + \epsilon &amp;gt; \Phi_c(C_a(s))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a mathematical way of saying that there can always be someone or some thing more conscious of a given scenario, whether or not she, he, or it is brought into existence.  Changing the criteria of evaluation from consciousness to intelligence, we have thus.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\forall \Phi_i(C_a(s)) \;\;\; \exists \;\;\; b, \, \epsilon&amp;gt;0 \;\; \ni \;\; \Phi_i(C_b(s)) + \epsilon &amp;gt; \Phi_i(C_a(s))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One can only surmise that Hannah's paper defines general intelligence relative to what whatever is the smartest thing around, which was once well-educated human beings.  Thus Hannah's definition of intelligence is dynamic.  Georgia applies the same formula to the new situation where she is now the standard against which lesser intelligence is narrow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the ability to confirm consciousness, it is actually easier to confirm than intelligence.  Consider this thought experiment.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Jack is playing chess with Dylan using the new chess set that Jack bought.  In spite of the aesthetic beauty of this new set, with its white onyx and black agate pieces, Dylan moves each piece with prowess and checkmates Jack.  Jack wonders if Dylan is more intelligent than him and asks what would be a normal question under those conditions.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&quot;Dylan, buddy, how long have you been playing chess?&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Regardless of the answer and regardless whether Dylan is a robot with a quantum processor of advanced AI or a human being, the intelligence of Dylan cannot be reliably gauged.  However, there is NO DOUBT that Dylan was conscious of the game play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the examples in the lists at the top of this answer there are a particular sets of requirements to qualify as consciousness.  For the case of Jack and Dylan playing, a few things MUST working in concert.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Visual recognition of the state of the board&lt;/li&gt;&#xA;&lt;li&gt;Motor control of the arm and hand to move pieces&lt;/li&gt;&#xA;&lt;li&gt;Tactile detection in finger and thumb tips&lt;/li&gt;&#xA;&lt;li&gt;Hand-eye cordination&lt;/li&gt;&#xA;&lt;li&gt;Grasp coordination&lt;/li&gt;&#xA;&lt;li&gt;A model of how to physically move board pieces&lt;/li&gt;&#xA;&lt;li&gt;A model of the rules of chess in memory&lt;/li&gt;&#xA;&lt;li&gt;A model of how to win when playing it (or astronomical computational power to try everything possible permutation that makes any sense)&lt;/li&gt;&#xA;&lt;li&gt;An internal representation of the board state&lt;/li&gt;&#xA;&lt;li&gt;Attention execution, visually and in terms of the objective of winning&lt;/li&gt;&#xA;&lt;li&gt;Prioritization that decides, unrelated to survival odds or asset accumulation, whether to beat Jack in chess, do something else, or nothing (non-deterministic if the ancient and commonplace notion of the causal autonomy of the soul is correct)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The topology of connections are as follows, and there may be more.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1 &amp;rlarr; 4 &amp;rlarr; 2&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;3 &amp;rlarr; 5 &amp;rlarr; 2&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;4 &amp;rlarr; 6 &amp;rlarr; 5&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;7 &amp;rlarr; 8 &amp;rlarr; 9&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;6 &amp;rlarr; 10 &amp;rlarr; 8&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;10 &amp;rlarr; 11&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is one of many integration topologies that support one of many types of things to which consciousness might apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether looking in the mirror just to prepare for work or whether looking deeply, considering the ontological question, &quot;Who am I?&quot; each mix of consciousness, subconsciousness, impulse, and habit require a specific topology of mental features.  Each topology must be coordinated to form its specific embodiment of consciousness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address some other sub-questions, it is easy to make a machine that claims itself to be conscious a digital voice recorder can be programmed to do it in five seconds by recording yourself saying it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting a robot to read this answer or some other conception, consider it thoughtfully, and then construct the sentence from knowledge of the vocabulary and conventions of human speech to tell you its conclusion is an entirely different task.  The development of such a robot may take 1,000 more years of AI research.  Maybe ten.  Maybe never.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last question, switched from plural to singular is, &quot;If [an artificially intelligent device] is only [operating] on predefined rules, without consciousness, can we even call it intelligent?&quot; The answer is necessarily dependent upon definition &lt;span class=&quot;math-container&quot;&gt;$\Phi_i$&lt;/span&gt; above, and, since neither &lt;span class=&quot;math-container&quot;&gt;$\Phi_c$&lt;/span&gt; nor &lt;span class=&quot;math-container&quot;&gt;$\Phi_i$&lt;/span&gt; have a standard definition within the AI community, one can't determine the cross-entropy or correlation.  It is indeterminable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps formal definitions of &lt;span class=&quot;math-container&quot;&gt;$\Phi_c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Phi_i$&lt;/span&gt; can now be written and submitted to the IEEE or some standards body.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-08T07:22:32.113" LastActivityDate="2019-01-08T07:22:32.113" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7578" PostTypeId="2" ParentId="5665" CreationDate="2018-08-15T12:37:11.910" Score="2" Body="&lt;p&gt;I think this is an interesting approach, but it's hard to tell if it can make better or faster learning networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, this could be looked at as using a function returning a tuple as activation function. The number of elements in the tuple must be equal to the number of neurons in the next layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example you could use an activation function like f(x) = (ReLU(x), sigmoid(x)) if the next layer contains two neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the biggest problem is to determine why one value should be passed to exactly THAT neuron and another value to another neuron - why not opposite? This must either be determined randomly or by doing some advanced analysis in beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, I think it's an interesting idea which should be tested out, but it's hard to say whether it will give better or worse results before trying it out.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="17488" LastEditDate="2018-08-15T20:32:34.323" LastActivityDate="2018-08-15T20:32:34.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7579" PostTypeId="1" CreationDate="2018-08-15T15:52:59.267" Score="5" ViewCount="212" Body="&lt;p&gt;I read somewhere that a &lt;em&gt;multilayer perceptron&lt;/em&gt; is a recursive function in its forward propagation phase. I am not sure, what is the recursive part? For me, I would see an MLP as a chained function. So, it would nice anyone could relate an MLP to a recursive function.&lt;/p&gt;&#xA;" OwnerUserId="13295" LastEditorUserId="2444" LastEditDate="2021-01-21T00:21:55.740" LastActivityDate="2021-01-21T00:21:55.740" Title="Is a multilayer perceptron a recursive function?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;multilayer-perceptron&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7580" PostTypeId="1" AcceptedAnswerId="7582" CreationDate="2018-08-15T16:58:56.453" Score="5" ViewCount="488" Body="&lt;p&gt;I'm now reading a book titled as &lt;em&gt;&amp;quot;Deep Reinforcement Learning Hands-On&amp;quot;&lt;/em&gt; and the author said the following on the chapter about AlphaGo Zero:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Self-play&lt;/p&gt;&#xA;&lt;p&gt;In AlphaGo Zero, the NN is used to approximate the prior probabilities of the actions and evaluate the position, which is very similar to the Actor-Critic (A2C) two-headed setup. On the input of the network, we pass the current game position (augmented with several previous positions) and return two values. The policy head returns the probability distribution over the actions and the value head estimates the game outcome as seen from the player's perspective. &lt;strong&gt;This value is undiscounted, as moves in Go are deterministic.&lt;/strong&gt; Of course, if you have stochasticity in the game, like in backgammon, some discounting should be used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All the environments that I have seen so far are stochastic environments, and I understand the discount factor is needed in stochastic environment.&#xA;I also understand that the discount factor should be added in infinite environments (no end episode) in order to avoid the infinite calculation.&lt;/p&gt;&#xA;&lt;p&gt;But I have never heard (at least so far on my limited learning) that the discount factor is NOT needed in deterministic environment. Is it correct? And if so, why is it NOT needed?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-15T20:25:35.487" Title="Is the discount not needed in a deterministic environment for Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;discount-factor&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7581" PostTypeId="2" ParentId="7579" CreationDate="2018-08-15T17:27:07.237" Score="5" Body="&lt;p&gt;Inherently, no. The MLP is just a data structure. It represents a function, but a standard MLP is just representing an input-output mapping, and there's no recursive structure to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, possibly your source is referring to the common &lt;em&gt;algorithms&lt;/em&gt; that operate over MLPs, specifically forward propagation for prediction and back propagation for training. Both of these algorithms are easy to think about recursively, with each node performing a sort of recursive call with its children or parents as the target, and some useful information about activations or errors attached. I actually encourage my students to implement it recursively for this reason, even though it's probably not the most efficient solution.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-15T17:27:07.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7582" PostTypeId="2" ParentId="7580" CreationDate="2018-08-15T17:32:13.997" Score="7" Body="&lt;p&gt;The motivation for adding the discount factor $\gamma$ is generally, at least initially, based simply in &quot;theoretical convenience&quot;. Ideally, we'd like to define the &quot;objective&quot; of an RL agent as maximizing the sum of all the rewards it gathers; its &lt;em&gt;return&lt;/em&gt;, defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{t = 0}^{\infty} R_t,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $R_t$ denotes the immediate reward at time $t$. As you also already noted in your question, this is inconvenient from a theoretical point of view, because we can have many different such sums that all end up being equal to $\infty$, and then the objective of &quot;maximizing&quot; that quantity becomes quite meaningless. So, by far the most common solution is to introduce a discount factor $0 \leq \gamma &amp;lt; 1$, and formulate our objective as maximizing the &lt;em&gt;discounted return&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{t = 0}^{\infty} \gamma^t R_t.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have an objective that will never be equal to $\infty$, so maximizing that objective always has a well-defined meaning.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As far as I am aware, the motivation described above is the &lt;strong&gt;only motivation for a discount factor being strictly necessary / needed&lt;/strong&gt;. This is &lt;strong&gt;not related to the problem being stochastic or deterministic&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have a stochastic environment, which is guaranteed to have a finite duration of at most $T$, we can define our objective as maximizing the following quantity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_0^T R_t,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $R_t$ is a random variable drawn from some distribution. &lt;strong&gt;Even in the case of stochastic environments, this is well-defined, we do not strictly need a discount factor&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Above, I addressed the question of whether or not a discount factor is &lt;strong&gt;necessary&lt;/strong&gt;. This does not tell the full story though. Even in cases where a discount factor is not strictly &lt;strong&gt;necessary&lt;/strong&gt;, it still might be &lt;strong&gt;useful&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, discount factors $\gamma &amp;lt; 1$ tell us that &lt;strong&gt;rewards that are nearby in a temporal sense&lt;/strong&gt; (reachable in a low number of time steps) &lt;strong&gt;are more important than rewards that are far away&lt;/strong&gt;. In problems with a finite time horizon $T$, this is probably not true, &lt;strong&gt;but it can still be a useful heuristic / rule of thumb&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a rule of thumb is particularly useful in stochastic environments, because &lt;strong&gt;stochasticity can introduce greater variance / uncertainty over long amounts of time than over short amounts of time&lt;/strong&gt;. So, even if in an ideal world we'd prefer to maximize our expected sum of undiscounted rewards, &lt;strong&gt;it is often easier to learn how to effectively maximize a discounted sum&lt;/strong&gt;; we'll learn behaviour that mitigates uncertainty caused by stochasticity because it prioritizes short-term rewards over long-term rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This rule of thumb especially makes a lot of sense in stochastic environments, but I don't agree with the implication in that book that it would be restricted to stochastic environments. A discount factor $\gamma &amp;lt; 1$ has also often been found to be beneficial for learning performance in deterministic environments, &lt;strong&gt;even if afterwards we evaluate an algorithm's performance according to the undiscounted returns&lt;/strong&gt;, likely because it leads to a &quot;simpler&quot; learning problem. In a deterministic environment there may not be any uncertainty / variance that grows over time due to the environment itself, but &lt;strong&gt;during a training process there is still uncertainy / variance in our agent's behaviour&lt;/strong&gt; which grows over time. For example, it will often be selecting suboptimal actions for the sake of exploration.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-15T17:37:30.380" LastActivityDate="2018-08-15T17:37:30.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7583" PostTypeId="2" ParentId="7579" CreationDate="2018-08-15T19:28:46.793" Score="3" Body="&lt;p&gt;Sure, you can define plenty of things we don't generally need to regard as recursive as so. An MLP is just a series of functions applied to its input. This can be loosely formulated as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ o_n = f(o_{n-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $o_n$ is the output of layer $n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this clearly doesn't reveal, much does it?&lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2018-08-15T19:28:46.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7585" PostTypeId="5" CreationDate="2018-08-15T20:27:18.247" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-08-15T20:27:18.247" LastActivityDate="2018-08-15T20:27:18.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7586" PostTypeId="4" CreationDate="2018-08-15T20:27:18.247" Score="0" Body="Use for question involving discount factor (γ) in reinforcement learning." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-08-15T20:27:18.247" LastActivityDate="2018-08-15T20:27:18.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7587" PostTypeId="2" ParentId="109" CreationDate="2018-08-15T22:45:20.830" Score="2" Body="&lt;p&gt;Sure! This is a somewhat hot area right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of ways to do it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the main line of research is with &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Networks&lt;/a&gt; (1980's) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Causal_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Casual Networks&lt;/a&gt; (1990's). These are basically rule-based systems for reasoning probabilistically. They rely on a user-designed model, which corresponds well to rules (e.g. when blood pressure is high, then heart attack rates are elevated), but provide a robust way to reason about uncertainties in the presence of these rules. Contrast this with a pure learning approach, like a decision tree or a neural network, which tends to rely less on rules. Modern research in this area focuses on &lt;a href=&quot;https://cs.uwaterloo.ca/~vanbeek/Research/research_ml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;learning the structure of the network&lt;/a&gt; (which corresponds to the learning probabilistic rules) from data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it's possible to learn rules from data and then do symbolic reasoning atop them using other techniques (e.g. rule induction), this approach runs into the same problems that plague the learning of the structure of Bayesian networks: when is correlation causation? &lt;a href=&quot;https://en.wikipedia.org/wiki/Causal_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Causal Diagrams&lt;/a&gt; are the only good tool for answering this question, but my impression is that inferring their structure automatically is still an open question.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-15T22:45:20.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7588" PostTypeId="2" ParentId="6850" CreationDate="2018-08-15T22:50:25.250" Score="1" Body="&lt;p&gt;It looks like you are training you model 10,000 times on one piece of data, and then dropping that piece and moving to the next.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will not work: the model will become extremely good at learning one piece of data, but will then forget about it when optimizing for the next piece.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, either pick one example at random in each iteration, or compute the gradient for all 4 examples and just update in that direction instead.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-15T22:50:25.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7589" PostTypeId="1" AcceptedAnswerId="7590" CreationDate="2018-08-16T02:13:15.483" Score="10" ViewCount="940" Body="&lt;p&gt;To the best of my understanding, the Monte Carlo tree search (MCTS) algorithm is an alternative to minimax for searching a tree of nodes. It works by choosing a move (generally, the one with the highest chance of being the best), and then performing a random playout on the move to see what the result is. This process continues for the amount of time allotted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't sound like machine learning, but rather a way to traverse a tree. However, I've heard that AlphaZero uses MCTS, so I'm confused. If AlphaZero uses MCTS, then why does AlphaZero learn? Or did AlphaZero do some kind of machine learning before it played any matches, and then use the intuition it gained from machine learning to know which moves to spend more time playing out with MCTS?&lt;/p&gt;&#xA;" OwnerUserId="16917" LastEditorUserId="16917" LastEditDate="2020-06-02T21:40:10.033" LastActivityDate="2020-06-02T21:40:10.033" Title="Does Monte Carlo tree search qualify as machine learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7590" PostTypeId="2" ParentId="7589" CreationDate="2018-08-16T02:25:00.650" Score="7" Body="&lt;p&gt;Monte Carlo Tree Search is not usually thought of as a machine learning technique, but as a search technique. There are parallels (MCTS does try to learn general patterns from data, in a sense, but the patterns are not very general), but really MCTS is not a suitable algorithm for most learning problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaZero was a combination of several algorithms. One was MCTS, but MCTS needs a function to tell it how good different states of the game might be (or else, it needs to simulate entire games). One way to handle this function in a game like chess or Go is to approximate it by training a neural network, which is what the Deep Mind researchers did. This is the learning component of AlphaZero. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="1641" LastEditDate="2018-08-16T08:25:51.067" LastActivityDate="2018-08-16T08:25:51.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7591" PostTypeId="1" AcceptedAnswerId="7595" CreationDate="2018-08-16T06:18:42.690" Score="6" ViewCount="547" Body="&lt;p&gt;Turing test was created to test machines exhibiting behavior equivalent or indistinguishable from that of a human. Is that the sufficient condition of intelligence?&lt;/p&gt;&#xA;" OwnerUserId="17527" LastEditorUserId="2444" LastEditDate="2019-06-27T20:30:48.017" LastActivityDate="2019-06-27T21:38:10.420" Title="If the Turing test is passed, does this imply that computers exhibit intelligence?" Tags="&lt;intelligence-testing&gt;&lt;turing-test&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7592" PostTypeId="2" ParentId="6850" CreationDate="2018-08-16T06:50:50.763" Score="2" Body="&lt;p&gt;Your network must have something which persist like &lt;strong&gt;weights&lt;/strong&gt; add &lt;strong&gt;bias&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your new implementation would be like this : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;trainingData = [{in: [0,0], out:[0]}, {in: [0,1], out:[0]}, ...];&#xA;iterations = 10000&#xA;&#xA;network = graphNodesToNetwork()&#xA;links = graphLinksToNetwork()&#xA;randomiseLinkWeights(links)&#xA;weights = []&#xA;&#xA;&#xA;while(trainingData not empty) {&#xA;    for(0&amp;lt;iterations) { &#xA;       set = trainingData.pop()&#xA;&#xA;       weights = updateInput(network, set.in)&#xA;&#xA;       forwardPropagate(network, links) &#xA;&#xA;       linkUpdate = backPropagate(network, links, set.out, weights)&#xA;&#xA;       updateLinks(linkUpdate, links)}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Inshort retain &lt;strong&gt;weights&lt;/strong&gt;, and backprapogate.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-08-16T06:50:50.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7594" PostTypeId="2" ParentId="7509" CreationDate="2018-08-16T08:02:23.197" Score="1" Body="&lt;p&gt;It seems impossible to prevent that. If someone can make a safe AI from scratch in the near future, then someone else can make a dangerous AI from scratch as well. If all that's needed is a computer (or eventually a robot) it will be really hard to stop people from creating one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Banning computers? Maybe it could prevent it, but that comes with quite a few negative sides as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A law against creating AIs? Would be really hard to follow-up, and what about the AIs people want to make to be used for something &quot;good&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And even if we could come up with a good set of laws they probably wouldn't be introduced in all countries, and also be really hard to follow up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess we just need to make our own defence mechanisms which can fight unsafe AIs (physically or virtually) when that time comes.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2018-08-16T08:02:23.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7595" PostTypeId="2" ParentId="7591" CreationDate="2018-08-16T08:24:33.160" Score="2" Body="&lt;p&gt;We don't know. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, an important line will have been crossed - it will be impossible to tell the difference between an intelligent agent and the machine by use of a text interface. Which is the main point of the test - &quot;if it quacks like a duck&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also an important philosophical point. Whether intelligence is defined purely by behaviour in an environment, or by the mechanisms that arrive at that behaviour. A suitably large database of conversational openers and &quot;correct&quot; responses can in theory mimic a lot of real world conversations. Some chatbots take advantage of this and use modern computer capacity to store a &lt;em&gt;lot&lt;/em&gt; of responses, and that approach has gained competitive scores in the Loebner prize competition (although not to the stage of actually passing the test). This leads us to the Chinese Room issue, and wondering which part of the system is actually intelligent, or even how much of human conversation is actually intelligent or meaningful (and it what ways).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-08-16T08:24:33.160" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7596" PostTypeId="2" ParentId="7589" CreationDate="2018-08-16T08:39:12.793" Score="7" Body="&lt;p&gt;John's answer is correct in that MCTS is traditionally not viewed as a Machine Learning approach, but as a tree search algorithm, and that AlphaZero combines this with Machine Learning techniques (Deep Neural Networks and Reinforcement Learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are some interesting similarities between MCTS itself and Machine Learning. In some sense, MCTS attempts to &quot;learn&quot; the value of nodes from experience generated through those nodes. This is very similar to how Reinforcement Learning (RL) works (which itself is typically described as a subset of Machine Learning). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some researchers have also experimented with replacements for the traditional &lt;em&gt;Backpropagation phase&lt;/em&gt; of MCTS (which, from an RL point-of-view, can be described as implementing a Monte-Carlo backups) based on other RL methods (e.g., Temporal-Difference backups). A comprehensive paper describing these sorts of similarities between MCTS and RL is: &lt;a href=&quot;https://jair.org/index.php/jair/article/view/11099/26289&quot; rel=&quot;noreferrer&quot;&gt;On Monte Carlo Tree Search and Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that the &lt;em&gt;Selection phase&lt;/em&gt; of MCTS is typically treated as a sequence of small Multi-Armed Bandit problems, and those problems also have strong connections with RL.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: MCTS is not normally viewed as a Machine Learning technique, but if you inspect it closely, you can find lots of similarities with ML (in particular, Reinforcement Learning).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-16T08:53:16.167" LastActivityDate="2018-08-16T08:53:16.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7597" PostTypeId="1" AcceptedAnswerId="7603" CreationDate="2018-08-16T10:07:42.763" Score="3" ViewCount="95" Body="&lt;p&gt;I'm developing an AI to play a card game with a genetic algorithm.  Initially, I will evaluate it against a player that plays randomly, so there will naturally be a lot of variance in the results.  I will take the mean score from X games as that agents fitness. The actual playing of the game dominates the time to evaluate the actual genetic algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is should I go for a low X e.g. 10, so I would be able to move through generations quite fast but the fitness function would quite inaccurate.  Alternatively, I could go for a high X e.g. 100 and would move very slowly but with a more accurate function.&lt;/p&gt;&#xA;" OwnerUserId="16724" LastActivityDate="2018-08-16T15:12:39.367" Title="Genetic Algorithms: Trade-off between time and variance with regards to fitness function" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7598" PostTypeId="2" ParentId="3389" CreationDate="2018-08-16T10:21:10.127" Score="16" Body="&lt;p&gt;Early success on prime number testing via artificial networks is presented in &lt;a href=&quot;https://cloudfront.escholarship.org/dist/prd/content/qt5sg7n4ww/qt5sg7n4ww.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Compositional Neural-network Solution to Prime-number Testing&lt;/em&gt;, László Egri, Thomas R. Shultz, 2006&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The knowledge-based cascade-correlation (KBCC) network approach showed the most promise, although the practicality of this approach is eclipsed by other prime detection algorithms that usually begin by checking the least significant bit, immediately reducing the search by half, and then searching based other theorems and heuristics up to &lt;span class=&quot;math-container&quot;&gt;$floor(\sqrt{x})$&lt;/span&gt;.  However the work was continued with &lt;a href=&quot;http://www.psych.mcgill.ca/perpg/fac/shultz/personal/Recent_Publications_files/kbcc06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Knowledge Based Learning with KBCC&lt;/em&gt;, Shultz et. al. 2006&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are actually multiple sub-questions in this question.  First, let's write a more formal version of the question: &amp;quot;Can an artificial network of some type converge during training to a behavior that will accurately test whether the input ranging from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$2^n-1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of bits in the integer representation, represents a prime number?&amp;quot;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can it by simply memorizing the primes over the range of integers?&lt;/li&gt;&#xA;&lt;li&gt;Can it by learning to factor and apply the definition of a prime?&lt;/li&gt;&#xA;&lt;li&gt;Can it by learning a known algorithm?&lt;/li&gt;&#xA;&lt;li&gt;Can it by developing a novel algorithm of its own during training?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The direct answer is yes, and it has already been done according to 1. above, but it was done by over-fitting, not learning a prime number detection method.  We know the human brain contains a neural network that can accomplish 2., 3., and 4., so if artificial networks are developed to the degree most think they can be, then the answer is yes for those.  There exists no counter-proof to exclude any of them from the range of possibilities as of this answer's writing.&lt;/p&gt;&#xA;&lt;p&gt;It is not surprising that work has been done to train artificial networks on prime number testing because of the importance of primes in discrete mathematics, its application to cryptography, and, more specifically, to cryptanalysis.  We can identify the importance of digital network detection of prime numbers in the research and development of intelligent digital security in works like &lt;a href=&quot;https://www.researchgate.net/profile/Michael_Vrahatis/publication/228790268_A_first_study_of_the_neural_network_approach_to_the_RSA_cryptosystem/links/0fcfd50e4b1dad4c70000000.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A First Study of the Neural Network Approach in the RSA Cryptosystem&lt;/em&gt;, G.c. Meletius et. al., 2002&lt;/a&gt;.  The tie of cryptography to the security of our respective nations is also the reason why not all of the current research in this area will be public.  Those of us that may have the clearance and exposure can only speak of what is not classified.&lt;/p&gt;&#xA;&lt;p&gt;On the civilian end, ongoing work in what is called novelty detection is an important direction of research.  Those like Markos Markou and Sameer Singh are &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0165168403002020&quot; rel=&quot;nofollow noreferrer&quot;&gt;approaching novelty detection from the signal processing side&lt;/a&gt;, and it is obvious to those that understand that artificial networks are essentially digital signal processors that have multi-point self tuning capabilities can see how their work applies directly to this question.  Markou and Singh write, &amp;quot;There are a multitude of applications where novelty detection is extremely important including signal processing, computer vision, pattern recognition, data mining, and robotics.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;On the cognitive mathematics side, the development of a mathematics of surprise, such as &lt;a href=&quot;https://infoscience.epfl.ch/record/223460/files/EPFL_TH7418.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning with Surprise: Theory and Applications&lt;/em&gt; (thesis), Mohammadjavad Faraji, 2016&lt;/a&gt; may further what Ergi and Shultz began.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2020-11-16T22:26:02.337" LastActivityDate="2020-11-16T22:26:02.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7599" PostTypeId="1" CreationDate="2018-08-16T11:30:08.937" Score="1" ViewCount="320" Body="&lt;p&gt;According to the paper &lt;a href=&quot;https://arxiv.org/pdf/1512.02325.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD: Single Shot MultiBox Detector&lt;/a&gt;, for each cell in a feature map k boxes are acquired and for each box we get $c$ class scores and $4$ offsets relative to the original default box_shape. This means that we get $m \times n \times (c +4) \times k$ outputs for each $m \times n$ feature map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is mentioned that in order to train the SSD network only the images and their ground truth boxes are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly can one define the output targets then? What is the format of the output in the SSD framework? I think it cannot be a vector with the positions, sizes and class of each boundary box, since the outputs are a lot more and relate to every default box in the feature maps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain in more detail how can I, given an image and its boundary boxes' info, construct a vector that will be fed into a network so that I can train it?&lt;/p&gt;&#xA;" OwnerUserId="13257" LastEditorUserId="1671" LastEditDate="2018-08-16T21:36:11.420" LastActivityDate="2018-08-16T21:36:11.420" Title="How does the target output of a Single Shot Detector (SSD) look like?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7601" PostTypeId="1" CreationDate="2018-08-16T14:57:13.340" Score="7" ViewCount="143" Body="&lt;p&gt;Could we teach an AI with sentences such as &quot;ants are small&quot; and &quot;the sky is blue&quot;? Is there any research work that attempts to do this?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2020-04-30T02:05:45.973" LastActivityDate="2020-04-30T02:05:45.973" Title="Can we teach an artificial intelligence through sentences?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;computational-learning-theory&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7602" PostTypeId="1" CreationDate="2018-08-16T15:07:36.253" Score="3" ViewCount="159" Body="&lt;p&gt;I have a 100-150 words text and I want to extract particular information like location, product type, dates, specifications and price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose if I arrange a training data which has a text as input and location/product/dates/specs/price as a output value. So I want to train the model for these specific output only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried Spacy and NLTK for entity extraction but that doesn't suffice above requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample text:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Supply of Steel Fabrication Items. &#xA;  General Item . &#xA;  Construction Material . &#xA;  Hardware Stores and Tool . &#xA;  Construction of Security Fence. - Angle Iron 65x65x6mm for fencing post of height 3.5, Angle Iron 65x65x6mm for fencing post of height 3.5, MS Flat 50 x 5mm of 2.60m height, Angle Iron 50x50x6mm for Strut post of height 3.10mtr, Angle Iron 50x50x6mm for fencing post of height 1.83, Angle Iron 50x50x6mm for fencing post of height 1.37, Barbed wire made out of GI wire of size 2.24mm dia, Chain link fence dia 4 mm and size of mesh 50mm x, Concertina Coil 600mm extentable up to 6 mtr, Concertina Coil 900mm extentable up to 15 to 20 mtr, Binding wire 0.9mm dia., 12 mm dia 50mm long bolts wih nuts &amp;amp; 02 x washers, Cement in polythene bags 50 kgs each grade 43 OPC, Sand Coarse confiming to IS - 383-970, 2nd revision, Crushed Stone Aggregate 20 mm graded, TMT Bar 12mm dia with 50mm U bend, Lime 1st quality, Commercial plywood 6' x 3' x 12 mm., Nails all Type 1&quot; 2&quot;3&quot; 4&quot; 5&quot; and 6&quot;., Primer Red Oxide, Synthetic enamel paint, colour black/white Ist quality . &#xA;  Angle Iron 65x65x6mm for fencing post of height 3.5, Angle Iron 65x65x6mm for fencing post of height 3.5 mtr, MS Flat 50 x 5mm of 2.60m height, Angle Iron 50x50x6mm for Strut post of height 3.10mtr, Barbed wire made out of GI wire of size 2.24mm dia, Chain link fence dia 4 mm and size of mesh 50mm x, Concertina Coil 600mm extentable up to 6 mtr, Binding wire 0.9mm dia., 12 mm dia 50mm long bolts with nuts &amp;amp; 02 x washers, Cement in polythene bags 50 kgs each grade 43 OPC, Sand Coarse confiming to IS - 383-970, 2nd revision, Crushed Stone Aggregate 20 mm graded, TMT Bar 12mm dia with 50mm U bend, Lime 1st quality, Commercial plywood 6' x 3' x 12 mm., Nails all Type 1&quot; 2&quot;3&quot; 4&quot; 5&quot; and 6&quot;., Primer Red Oxide, Synthetic enamel paint, colour black/white Ist quality., Cutting Plier 160mm long, Leather Hand Gloves/Knitted industrial, Ring Spanner of 16mm x 17mm, 14 x 16mm, Crowbar hexagonal 1200mm long x 40mm, Plumb bob steel, Bucket steel 15 ltr capacity (as per, Plastic water tank 500 ltrs Make - Sintex, Water level pipe 30 Mtr, Brick Hammer 250 Gms with handle, Hack saw Blade double side, Welding Rod, Cutting rod for making holes, HDPE Sheet 5' x 8', Plastic Measuring tape 30 Mtr, Steel Measuring tape 5 Mtr, Wooden Gurmala 6&quot;x3&quot;, Steel Pan Mortar of 18&quot;dia (As, Showel GS with wooden handle, Phawarah with wooden handle (As per, Digital Vernier Caliper, Digital Weighing Machine cap 500 Kgs, Portable Welding Machine, Concrete mixer machine of 8 CFT . &#xA;  Angle Iron 65x65x6mm for fencing post of height 3.5, Angle Iron 65x65x6mm for fencing post of height 3.5, MS Flat 50 x 5mm of 2.60m height, Angle Iron 50x50x6mm for Strut post of height 3.10mtr, Barbed wire made out of GI wire of size 2.24mm dia, Chain link fence dia 4 mm and size of mesh 50mm, Concertina Coil 600mm extentable up to 6 mtr, Binding wire 0.9mm dia., 12 mm dia 50mm long bolts with nuts &amp;amp; 02 x washers, Cement in polythene bags 50 kgs each grade 43, Sand Coarse confiming to IS - 383-970, 2nd revision, Crushed Stone Aggregate 20 mm graded, TMT Bar 12mm dia with 50mm U bend, Lime 1st quality, Commercial plywood 6' x 3' x 12 mm., Nails all Type 1&quot; 2&quot;3&quot; 4&quot; 5&quot; and 6&quot;., Primer Red Oxide, Synthetic enamel paint, colour black/white Ist quality., Cutting Plier 160mm long, Leather Hand Gloves/Knitted industrial, Ring Spanner of 16mm x 17mm, 14 x 16mm, Crowbar hexagonal 1200mm long x 40mm, Plumb bob steel, Bucket steel 15 ltr capacity (as per, Plastic water tank 500 ltrs Make - Sintex, Water level pipe 30 Mtr, Brick Hammer 250 Gms with handle, Hack saw Blade double side, Welding Rod, Cutting rod for making holes, HDPE Sheet 5' x 8', Plastic Measuring tape 30 Mtr, Steel Measuring tape 5 Mtr, Wooden Gurmala 6&quot;x3&quot;, Steel Pan Mortar of 18&quot;dia (As per, Showel GS with wooden handle, Phawarah with wooden handle (As per, Digital Vernier Caliper)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16183" LastEditorUserId="1671" LastEditDate="2018-08-16T21:20:34.233" LastActivityDate="2018-08-17T04:55:24.030" Title="How can I train model to extract custom entities from text?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7603" PostTypeId="2" ParentId="7597" CreationDate="2018-08-16T15:12:39.367" Score="2" Body="&lt;p&gt;You can probably get away with a relatively low X for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Central_limit_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Central Limit Theorem&lt;/a&gt;. This tells us that the accuracy in the estimate of an agent's fitness will improve as the square root of the number of games played. &lt;/li&gt;&#xA;&lt;li&gt;In a GA, you don't need an absolute ranking of individuals, because your selection mechanism (see &quot;related articles&quot; &lt;a href=&quot;https://en.wikipedia.org/wiki/Selection_(genetic_algorithm)&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) typically isn't completely elitist. For example, if individuals in the top half of the population are allowed to breed, then your fitness function really just needs to separate the good from the bad reasonably well. It need not be perfect to work.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The correct value of X will still depend on the variance in the scores an agent might receive from playing the game, but this is easy to track. A good approach might be to incorporate this directly into your estimate. Compute the variance, and then prefer agents that not only score highly, but do so with low variance. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-16T15:12:39.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7604" PostTypeId="2" ParentId="7602" CreationDate="2018-08-16T15:31:59.330" Score="1" Body="&lt;p&gt;For your specific problem i would use a hierarchical search.&#xA;The first step should be to separate the text, each fragment should contain several entities, but would be more easy to identify them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Location, Dates, prices: You can use regex search, &lt;a href=&quot;https://stackoverflow.com/questions/24722484/named-entity-recognition-regex-for-places&quot;&gt;link&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Specifications, locations: You can try using Deep Learning with character level bigrams, or word bigrams. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17463" LastActivityDate="2018-08-16T15:31:59.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7605" PostTypeId="2" ParentId="5333" CreationDate="2018-08-16T16:27:09.707" Score="1" Body="&lt;p&gt;This problem is very challenging since you need to evaluate the quality of the candidate answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In question answering, there are common steps that you need to follow. To summarise you need first to find the sentence that can answer the question and then compose the final answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first step you can measure the semantical similarity between Q and A,   this a first filter where you can use several deep learning methods. Also, you can define a threshold to validate if the pair QA are enough related.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second step, you must extract the answer, if the answer is a fact you can use KB extraction, or if it is a summary, list you can use other DL methods. There is also the possibility that you must infer the answer, also suitable for DL methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest to check this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1502.05698&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17463" LastActivityDate="2018-08-16T16:27:09.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7608" PostTypeId="1" CreationDate="2018-08-16T17:41:27.630" Score="1" ViewCount="691" Body="&lt;p&gt;I was working recently on Progressive Growing of &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;GANs&lt;/a&gt; (aka PGGANs). I have implemented the whole architecture, but the problem that was ticking my mind is that in simple GANs, like DCGAN, PIX2PIX, we actually use Transposed Convolution for up-sampling and Convolution for Down-sampling, but in PGGANs in which we gradually add layers to both generator and discriminator so that we can first start with 4x4 image and then increase to 1024x01024 step by step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did not understand that once we Increase 1x1x512 dimensional latent vector size to 4x4x512 sort of image we use convolution with high padding, and then once training for 4x4 images, we take still take 512 latent vector and then use the previously trained convolutional layers to convert it to 4x4x512 image, and then we up-sample then given image to 8x8 using nearest neighbor filtering and then again apply convolution and so-on. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My question is that why we need to explicitly up-sample and then apply convolution, when instead we could just use Transposed Convolution which can upsample it automatically and is trainable? Why do we not use it like in other GANs? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is the image of architecture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/udNPA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/udNPA.png&quot; alt=&quot;PGGANs architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please explain me the intuition behind this.&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="16878" LastEditorUserId="1671" LastEditDate="2018-08-16T21:19:16.217" LastActivityDate="2020-10-04T16:39:51.847" Title="Why do we need Upsampling and Downsampling in Progressive Growing of Gans" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;statistical-ai&gt;&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7609" PostTypeId="1" CreationDate="2018-08-16T18:10:03.303" Score="9" ViewCount="2464" Body="&lt;p&gt;So Taleb has two heuristics to generally describe data distributions. One is Mediocristan, which basically means things that are on a Gaussian distribution such as height and/or weight of people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other is called Extremistan, which describes a more Pareto like or fat-tailed distribution. An example is wealth distribution, 1% of people own 50% of the wealth or something close to that and so predictability from limited data sets is much harder or even impossible. This is because you can add a single sample to your data set and the consequences are so large that it breaks the model, or has an effect so large that it cancels out any of the benefits from prior accurate predictions. In fact this is how he claims to have made money in the stock market, because everyone else was using bad, Gaussian distribution models to predict the market, which actually would work for a short period of time but when things went wrong, they went really wrong which would cause you to have net losses in the market.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this video of Taleb being asked about AI. His claim is that A.I. doesn't work (as well) for things that fall into extremistan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is he right? Will some things just be inherently unpredictable even with A.I.?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the video I am referring to &lt;a href=&quot;https://youtu.be/B2-QCv-hChY?t=43m08s&quot; rel=&quot;noreferrer&quot;&gt;https://youtu.be/B2-QCv-hChY?t=43m08s&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17541" LastEditorUserId="1671" LastEditDate="2018-08-16T21:12:17.780" LastActivityDate="2020-06-25T04:33:20.000" Title="Is Nassim Taleb right about AI not being able to accurately predict certain types of distributions?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;probability&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7610" PostTypeId="2" ParentId="7609" CreationDate="2018-08-16T18:38:44.547" Score="8" Body="&lt;p&gt;Yes and no!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no inherent reason that machine learning systems can't deal with extreme events. As a simple version, you can learn the parameters of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Weibull_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weibull distribution&lt;/a&gt;, or another extreme value model, from data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bigger issue is with known-unknowns vs. unknown-unknowns. If you &lt;em&gt;know&lt;/em&gt; that rare events are possible (as with, say, earthquake prediction), you can incorporate that knowledge into the models you develop, and you'll get something that works as well or better than humans in that domain. If you &lt;em&gt;don't&lt;/em&gt; know that rare events are possible (as with, say, a stock market crash produced by correlated housing defaults), then your model will reflect that as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to think Taleb is being a bit unfair here: AI can't handle these kinds of events precisely because its creators (us) can't handle them! If we knew they were possible, then we could handle them pretty well, and AI could too.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-06-19T20:50:42.203" LastActivityDate="2019-06-19T20:50:42.203" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7611" PostTypeId="1" AcceptedAnswerId="7613" CreationDate="2018-08-16T23:33:09.813" Score="7" ViewCount="651" Body="&lt;p&gt;I'm making a Connect Four game where my engine uses Minimax with Alpha-Beta pruning to search. Since Alpha-Beta pruning is much more effective when it looks at the best moves first (since then it can prune branches of poor moves), I'm trying to come up with a set of heuristics that can rank moves from best to worst. These heuristics obviously aren't guaranteed to always work, but my goal is that they'll &lt;em&gt;often&lt;/em&gt; allow my engine to look at the best moves first. An example of such heuristics would be as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Closeness of a move to the centre column of the board - weight 3.&lt;/li&gt;&#xA;&lt;li&gt;How many pieces surround a move - weight 2.&lt;/li&gt;&#xA;&lt;li&gt;How low, horizontally, a move is to the bottom of the board - weight 1.&lt;/li&gt;&#xA;&lt;li&gt;etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I have no idea what the best set of weight values are for each attribute of a move. The weights I listed above are just my estimates, and can obviously be improved. I can think of two ways of improving them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Evolution. I can let my engine think while my heuristics try to guess which move will be chosen as best by the engine, and I'll see the success score of my heuristics (something like x% guessed correctly). Then, I'll make a pseudo-random change/mutation to the heuristics (by randomly adjusting one of the weight values by a certain amount), and see how the heuristics do then. If it guesses better, then that will be my new set of heuristics. Note that when my engine thinks, it considers thousands of different positions in its calculations, so there will be enough data to average out how good my heuristics are at prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Generate thousands of different heuristics with different weight values from the start. Then, let them all try to guess which move my engine will favor when it thinks. The set of heuristics that scores best should be kept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure which strategy is better here. Strategy #1 (evolution) seems like it could take a long time to run, since every time I let my engine think it takes about 1 second. This means testing each new pseudo-random mutation will take a second. Meanwhile, Strategy #2 seems faster, but I could be missing out on a great set of heuristics if I myself didn't include them.&lt;/p&gt;&#xA;" OwnerUserId="16917" LastEditorUserId="1641" LastEditDate="2018-08-17T12:08:28.820" LastActivityDate="2019-12-10T18:20:44.390" Title="More effective way to improve the heuristics of an AI... evolution or testing between thousands of pre-determined sets of heuristics?" Tags="&lt;game-ai&gt;&lt;evolutionary-algorithms&gt;&lt;search&gt;&lt;heuristics&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7612" PostTypeId="2" ParentId="7602" CreationDate="2018-08-17T04:55:24.030" Score="1" Body="&lt;p&gt;You need to make the training data as given below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U.N. I-ORG &#xA;official O &#xA;Ekeus I-PER &#xA;heads O &#xA;for O &#xA;Baghdad I-LOC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Treat this as a classification task. here in given example, we have 3 classes ( &lt;code&gt;I-ORG&lt;/code&gt; &lt;code&gt;I-PER&lt;/code&gt; and &lt;code&gt;I-LOC&lt;/code&gt; ).&#xA;Now you can process such data using Multilayer Perceptron. LSTM, or CNN or Ensemble of all.&#xA;For detail you may follow this &lt;a href=&quot;http://learningdeep.xyz/designing-a-machine-learning-model/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-08-17T04:55:24.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7613" PostTypeId="2" ParentId="7611" CreationDate="2018-08-17T09:07:14.453" Score="3" Body="&lt;p&gt;Hmmm, I see some issues that are actually present in both of the approaches you propose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to note that the depth level that your Minimax search process manages to reach, and therefore also the speed with which it can traverse the tree, is extremely important for the algorithm's performance. Therefore, when evaluating how good or bad a particular heuristic function for move ordering is, &lt;strong&gt;it is not only important to look at how well it ordered moves; it is also important to take into account the runtime overhead of the heuristic function call&lt;/strong&gt;. If your heuristic functions manages to sort well, but is so computationally expensive that you can't search as deep in the tree, it's often not really worth it. Neither of the solutions you propose are able to take this into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is that it's not trivial to measure what ordering is the &quot;best&quot;. A heuristic that has the highest accuracy for the position of the best move only is not necessarily the best heuristic. For example, a heuristic that always places the best move in the second position ($0\%$ accuracy because it's in the wrong position, should be first position) might be better than a heuristic that places the best move in the first position $50\%$ of the time ($50\%$ accuracy), and places the best move last in the other $50\%$ of cases.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I would be more inclined to evaluate the performance of different heuristic functions by setting up tournaments where different versions of your AI (same search algorithm, &lt;strong&gt;same processing time constraints per turn&lt;/strong&gt;, different heuristic function) play against each other, and &lt;strong&gt;measuring the win percentage&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This set up can also be done with two variants analogous to what you proposed; you can exhaustively put all the heuristic functions you can come up with against each other in tournaments, or you can let let an evolutionary algorithm sequentially generate populations of hypothesis-heuristic-functions, and run a tournament with each population. Generally, &lt;strong&gt;I would lean towards the evolutionary approach&lt;/strong&gt;, since we expect it to search the same search space of hypotheses (heuristic functions), but we expect it to do so in a more clever / efficient manner than an exhaustive search. Of course, if you happen to have a ridiculous amount of hardware available (e.g., if you're Google), you might be able to perform the complete exhaustive search at once in parallel.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that there are also ways to do fairly decent move ordering without heuristic functions like the ones you suggested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you likely should be using &lt;strong&gt;iterative deepening&lt;/strong&gt;; this is a variant of your search algorithm where you first only perform a search with a depth limit $d = 1$, then repeat the complete search process with a depth limit $d = 2$, then again with a limit $d = 3$, etc., until processing time runs out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have completed such a search process for a depth limit $d$, and move on to the subsequent search process with a limit of $d + 1$, you can order order the moves &lt;strong&gt;in the root node&lt;/strong&gt; according to your evaluations from the previous search process (with depth limit $d$). Yes, here you would only have move ordering in the root node, and nowhere else, but this is by far the most influential / important place in the tree to do move ordering. Move ordering becomes less and less important as you move further away from the root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're using a &lt;strong&gt;transposition table&lt;/strong&gt; (TT), it is also common to store the &quot;best move&quot; found for every state in your TT. If, later on, you run into a state that already exists in your TT (which will be very often if you're using iterative deepening), and if you cannot directly take the stored value but have to actually do a search (for instance, because your depth limit increased due to iterative deepening), you can search the &quot;best move&quot; stored in the TT first. This is very light move ordering in that you only put one move at the front and don't order the rest, but it can still be effective.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-17T09:07:14.453" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7614" PostTypeId="2" ParentId="7611" CreationDate="2018-08-17T10:19:46.747" Score="0" Body="&lt;p&gt;With regards to random vs evolutionary algorithm, an evolutionary algorithm will almost always be superior.  Imagine the space of all possible heuristics.  An evolutionary algorithm moves through it 'intelligently' i.e. it somewhat follows the gradient of the space and should converge to a local optimum.  A random algorithm will not be able to achieve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to the time taken, surely it would be the same for each one to evaluate X heuristics?&lt;/p&gt;&#xA;" OwnerUserId="16724" LastActivityDate="2018-08-17T10:19:46.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7616" PostTypeId="2" ParentId="6460" CreationDate="2018-08-17T15:38:02.623" Score="1" Body="&lt;p&gt;There are a couple of things to consider with this question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, training &lt;em&gt;time&lt;/em&gt; can be a deceptive measure. Neural network training is considered &quot;trivially&quot; parallelizable. This means that the more computers you have access to, the faster you can train (most of the time, one computation doesn't depend on another, so you can do them both at the same time). Since the start of the current phase of self-driving cars, GPU performance has increased by more than a &lt;a href=&quot;https://www.techspot.com/article/1191-nvidia-geforce-six-generations-tested/&quot; rel=&quot;nofollow noreferrer&quot;&gt;factor of 5&lt;/a&gt;. Further, large companies like Google have migrated to &lt;a href=&quot;https://www.theregister.co.uk/2018/05/09/google_tpu_3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-specific ASIC chips&lt;/a&gt;. These are faster still. This means any estimate of &quot;time&quot; is likely to be confusing or misleading. The amount of time required to train a network has dropped rapidly over time, even if the number of processing cycles required has stayed the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, it is very unlikely that self-driving car companies have spent that time training a single model. Instead, they are probably starting from a given model (say, last month's), and trying different approaches to improving it, with new data, alternative training methods, or &quot;expert&quot; domain knowledge. This makes it difficult to reason about what it means to &quot;get the results that are used in today's self-driving vehicles.&quot; If we count all the aborted attempts, probably it's on the order of high hundreds or &lt;strong&gt;low thousands of person-years of researchers&lt;/strong&gt; (6-7 years, and there aren't thousands of people working in research in this area). If you also count related development and engineering efforts, it's probably an order of magnitude or more than that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it might be more interesting to think about the amount of &lt;em&gt;simulated driving time&lt;/em&gt; that is needed to train a network in this way. As discussed in this &lt;a href=&quot;https://www.alexirpan.com/2018/02/14/rl-hard.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;excellent blog post&lt;/a&gt;, &quot;Deep Reinforcement Learning Doesn't Work Yet&quot;, the amount of training experience required to train a neural net for this kind of problem is extremely large. To learn to play simple Atari games better than human players required about 244 hours of exposure. These games are typically just a single screen, and most humans can pick them up in a couple of minutes or less. &lt;a href=&quot;https://www.directline.com/car-cover/how-long-does-it-take-to-learn-to-drive&quot; rel=&quot;nofollow noreferrer&quot;&gt;This site&lt;/a&gt; estimates average time for a human to learn to drive competently at just shy of 70 hours. Applying the same ratio, we can infer that a deep neural net would want something like &lt;strong&gt;2 years of experience driving&lt;/strong&gt; to achieve human level performance. This seems like the right ballpark, but probably a lot of that driving is done &quot;offline&quot; in a simulated environment, rather than operating the vehicle directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, these are just ballpark estimates. The exact figures are likely to be proprietary. Further, there are some &lt;a href=&quot;https://www.theverge.com/2018/7/3/17530232/self-driving-ai-winter-full-autonomy-waymo-tesla-uber&quot; rel=&quot;nofollow noreferrer&quot;&gt;reports&lt;/a&gt; that modern systems are abandoning neural nets, and moving back to a more &quot;rule-based&quot; paradigm because of the difficulties in training them. I'm not sure how much credit to give those reports, but it again makes it difficult to pin down a training time.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-17T15:38:02.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7617" PostTypeId="1" CreationDate="2018-08-17T16:00:13.137" Score="6" ViewCount="271" Body="&lt;p&gt;A lot of questions on this site seem to be asking &amp;quot;can I use X to solve Y?&amp;quot;, where X is usually a deep neural network, and Y is often something already addressed by other areas of AI that are less well known?&lt;/p&gt;&#xA;&lt;p&gt;I have some ideas about this, but am inspired by questions like &lt;a href=&quot;https://bicycles.stackexchange.com/q/1195&quot;&gt;this one&lt;/a&gt; where a fairly wide range of views are expressed, and each answer focuses on just one possible problem domain.&lt;/p&gt;&#xA;&lt;p&gt;There are some related questions on this stack already, but they are not the same. &lt;a href=&quot;https://ai.stackexchange.com/q/240/2444&quot;&gt;This question&lt;/a&gt; specifically asks what genetic algorithms are good for, whereas I am more interested in having an inventory of problems mapped to possible techniques. &lt;a href=&quot;https://ai.stackexchange.com/q/60/2444&quot;&gt;This question&lt;/a&gt; asks what possible barriers are to AI with a focus on machine learning approaches, but I am interested in what we &lt;em&gt;can&lt;/em&gt; do without using deep neural nets, rather than what is difficult in general.&lt;/p&gt;&#xA;&lt;p&gt;A good answer will be supported with citations to the academic literature, and a brief description of both the problem and the main approaches that are used.&lt;/p&gt;&#xA;&lt;p&gt;Finally, &lt;a href=&quot;https://ai.stackexchange.com/q/2999/2444&quot;&gt;this question&lt;/a&gt; asks what AI can do to solve problems related to climate change. I'm not interested in the ability to address specific application domains. Instead, I want to see a catalog of abstract problems (e.g. having an agent learn to navigate in a new environment; reasoning strategically about how others might act; interpreting emotions), mapped to useful techniques for those problems. That is, &amp;quot;solving chess&amp;quot; isn't a problem, but &amp;quot;determining how to optimally play turn-based games without randomness&amp;quot; is.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-18T11:35:01.707" LastActivityDate="2021-01-18T11:41:55.693" Title="What kinds of problems can AI solve without using a deep neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-neural-networks&gt;&lt;symbolic-ai&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7618" PostTypeId="1" AcceptedAnswerId="7626" CreationDate="2018-08-17T20:47:17.640" Score="3" ViewCount="94" Body="&lt;p&gt;I recently &lt;a href=&quot;https://ai.stackexchange.com/a/7582/1671&quot;&gt;came across this function&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{t = 0}^{\infty} \gamma^t R_t.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's elegant and looks to be useful in the type of deterministic, perfect-information, finite models I'm working with.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it occurs to me that using &lt;span class=&quot;math-container&quot;&gt;$\gamma^t$&lt;/span&gt; in this manner might be seen as somewhat arbitrary.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, the objective is to discount per the added uncertainty/variance of &quot;temporal distance&quot; between the present gamestate and any potential gamestate being evaluated, but that variance would seem to be a function of the branching factors present in a given state, and the sum of the branching factors leading up to the evaluated state.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are there any defined discount-factors based on the number of branching factors for a given, evaluated node, or the number of branches in the nodes leading to it?  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If not, I'd welcome thoughts on how this might be applied. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An initial thought is that I might divide 1 by the number of branches and add that value to the goodness of a given state, which is a technique I'm using for heuristic tie-breaking with no look-ahead, but that's a &quot;value-add&quot; as opposed to a discount.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For context, this is for a form of &lt;a href=&quot;https://www.reddit.com/r/abstractgames/comments/8khl96/formal_definition_of_mnpgames/&quot; rel=&quot;nofollow noreferrer&quot;&gt;partisan Sudoku&lt;/a&gt;, where an expressed position &lt;span class=&quot;math-container&quot;&gt;$p_x$&lt;/span&gt; (value, coordinates) typically removes some number of potential positions &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; from the gameboard.  &lt;em&gt;(Without the addition of an element displacement mechanic, the number of branches can never increase.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a &lt;span class=&quot;math-container&quot;&gt;$(3^2)^2$&lt;/span&gt; Sudoku, the first &lt;span class=&quot;math-container&quot;&gt;$p_x$&lt;/span&gt; removes &lt;span class=&quot;math-container&quot;&gt;$30$&lt;/span&gt; out of &lt;span class=&quot;math-container&quot;&gt;$729$&lt;/span&gt; potential positions &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, including itself.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With each &lt;span class=&quot;math-container&quot;&gt;$p_x$&lt;/span&gt;, the number of branches diminishes until the game collapses into a tractable state, allowing for perfect play in endgames.  [Even there, a discounting function may have some utility because outcomes sets of ratios. Where the macro metric is territorial (controlled regions at the end of play), the most meaningful metric may ultimately be &quot;efficiency&quot; &lt;em&gt;(loosely, &quot;points_expended to regions_controlled&quot;)&lt;/em&gt;, which acknowledges a benefit to expending the least amount of points &lt;span class=&quot;math-container&quot;&gt;$p_x$&lt;/span&gt;, even in a tractable endgame where the ratio of controlled regions cannot be altered. Additionally, zugzwangs are possible in the endgame, and in that case reversing the discount to maximize branches may have utility.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt; &lt;span class=&quot;math-container&quot;&gt;$(3^2)^2 = 3x3(3x3) = &quot;9x9&quot;$&lt;/span&gt; but the exponent is preferred so as not to restrict the number of dimensions. &lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-06-14T13:29:37.287" LastActivityDate="2019-06-14T13:29:37.287" Title="Are there any discount-factors based on branching factors?" Tags="&lt;ai-design&gt;&lt;algorithm&gt;&lt;game-ai&gt;&lt;math&gt;&lt;discount-factor&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7624" PostTypeId="1" AcceptedAnswerId="7671" CreationDate="2018-08-18T14:17:55.563" Score="16" ViewCount="564" Body="&lt;p&gt;When we talk about artificial intelligence, human intelligence or any other form of intelligence, what do we mean by the term &lt;strong&gt;intelligence&lt;/strong&gt; in a general sense? What would you call intelligent and what not? In other words, how do we define the term &lt;strong&gt;intelligence&lt;/strong&gt; in the most general possible way?&lt;/p&gt;&#xA;" OwnerUserId="17209" LastEditorUserId="2444" LastEditDate="2019-06-27T20:06:08.783" LastActivityDate="2021-01-24T19:32:07.777" Title="What is the most general definition of &quot;intelligence&quot;?" Tags="&lt;definitions&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7626" PostTypeId="2" ParentId="7618" CreationDate="2018-08-18T14:52:37.517" Score="1" Body="&lt;p&gt;First, an important note on any form of discounting: &lt;strong&gt;adding a discount factor can change what the optimal policy is&lt;/strong&gt;. The optimal policy when a discount factor is present can be different from the optimal policy in the case where a discount factor is absent. &lt;strong&gt;This means that &quot;artificially&quot; adding a discount factor is harmful if we expect to be capable of learning / finding an optimal policy&lt;/strong&gt;. Generally, we don't expect to be capable of doing that though, except for the case where we have an infinite amount of processing time (which is never in practice). In that other answer which you linked to in your question, I describe that it can still be &lt;strong&gt;useful, that it may help to find a good (not optimal, just good) policy more quickly&lt;/strong&gt;, but it does not come &quot;for free&quot;. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of any research evaluating ideas like the one in your question. I am not 100% sure why, I suspect it could be an interesting idea in some situations, but would have to be investigated carefully due to my point above; if not evaluated properly, it could also unexpectedly be harmful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to note is that the use of discounting factors $\gamma &amp;lt; 1$ is extremely common (basically ubiquitous) in Reinforcement Learning (RL) literature, but rather rare in literature on tree search algorithms like MCTS (though not non-existant; for example, it's used in the &lt;a href=&quot;http://ggp.stanford.edu/readings/uct.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original UCT paper from 2006&lt;/a&gt;). For the concept of &quot;branching factors&quot;, we have the opposite; in RL literature it is &lt;em&gt;very common&lt;/em&gt; to consistently have the same action space regardless of states (&quot;constant branching factor&quot;), whereas this is &lt;em&gt;very uncommon&lt;/em&gt; in literature on tree search algorithms. So, the combination of discount factors + branching factors is actually somewhat rare in existing literature (which of course doesn't mean that the idea couldn't work or be relevant, it just might explain why the idea doesn't appear to have been properly investigated yet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;One important concern I do have with your idea&lt;/strong&gt; is that it seems like it could be somewhat of an &quot;anti-heuristic&quot; in some situations (with which I mean, a heuristic that is detrimental to performance). In many games, it is advantageous to be in game states where you have many available moves (a large branching factor), this can mean that you are in a strong position. Consider, for example, chess, where a player who is in a convincingly winning position likely has more moves available than their opponent. I suspect your idea, when applied to chess, would simply promote an aggressive playing style where both players capture as many pieces as possible in an effort to quickly reduce the branching factors across the entire tree.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If not, I'd welcome thoughts on how this might be applied.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;(I might divide 1 by the number of branches and add that value to the goodness of a given state, but that's a &quot;value-add&quot; as opposed to a discount.)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Such an additive change would be more closely related to the idea of &lt;strong&gt;reward shaping&lt;/strong&gt;, rather than discounting (which is again a thing that, if not done carefully, can significantly alter the task that you're optimizing for). Intuitively I also suspect it might not do anything at all since you'd always be adding the same constant value regardless of which move you took (regardless of which move you took, your parent state will always have had the same branching factor). I might be missing out on some details here, but I think you'd have to have a multiplicative effect on your regular observed rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose one example that could be worth a try would be something like maximizing the following return;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{t = 0}^{T} \gamma^t \beta^{b(S_{t}) - 1} R_{t + 1},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$0 \leq \gamma \leq 1$ is the normal time-based discount factor (can set it to $1$ if you like)&lt;/li&gt;&#xA;&lt;li&gt;$0 \leq \beta \leq 1$ is a new branching-factor-based discount factor&lt;/li&gt;&#xA;&lt;li&gt;$b(S_t)$ is the branching factor (the number of available moves) in state $S_t$&lt;/li&gt;&#xA;&lt;li&gt;$R_{t + 1}$ is the immediate reward for transitioning from state $S_t$ to $S_{t + 1}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I put the $-1$ in the power of $\beta$ because I suspect you wouldn't want to do any discounting in situations where only one move is available. Intuitively, I do suspect this $\beta$ would require very careful tuning though. It is already quite common to choose $\gamma = 0.9$ or $\gamma = 0.99$, with $\beta$ you may want to stay even closer to $1$.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-18T14:52:37.517" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7628" PostTypeId="1" AcceptedAnswerId="7629" CreationDate="2018-08-18T18:30:14.853" Score="5" ViewCount="874" Body="&lt;p&gt;In particular, I would like to have a simple definition of &quot;environment&quot; and &quot;state&quot;. What are the differences between those two concepts? Also, I would like to know how the concept of model relates to the other two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a similar question &lt;a href=&quot;https://ai.stackexchange.com/q/5970/2444&quot;&gt;What is the difference between an observation and a state in reinforcement learning?&lt;/a&gt;, but it is not exactly what I was looking for.&lt;/p&gt;&#xA;" OwnerUserId="17565" LastEditorUserId="2444" LastEditDate="2019-02-11T21:22:53.157" LastActivityDate="2021-03-16T08:17:43.690" Title="What is the relation between an environment, a state and a model?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7629" PostTypeId="2" ParentId="7628" CreationDate="2018-08-18T20:25:55.333" Score="5" Body="&lt;p&gt;&lt;strong&gt;Environment&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the manifestation of the problem being solved. It might be a real physical situation (a road network and cars), or virtual on a computer (a board game on a computer). It includes all the machinery necessary to resolve what happens. E.g. in the real world the objects involved, how the agent exerts its control when taking actions, and the applicable real-world laws of physics. Or, in a simulated world, things like the rules of a board game, implemented in code.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;State&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the representation of a &amp;quot;position&amp;quot; at a certain time step within the environment. It may be something the agent can observe through sensors, or be provided directly by the computer system running a simulation.&lt;/p&gt;&#xA;&lt;p&gt;For RL theory to hold, it is important that the state representation has the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Property&lt;/a&gt;, which is that the state accurately foretells the probabilities of rewards and following state for each action that could be taken. You do not need to &lt;em&gt;know&lt;/em&gt; those probabilities in order to run RL algorithms (in fact it is a common case that you don't know). However, it is important that the dependency between the state+action and what happens next holds reliably.&lt;/p&gt;&#xA;&lt;p&gt;The state is commonly represented by a vector of values. These describe positions of pieces in a game, or positions and velocities of objects that have been sensed. A state may be built from observations, but does not have to match 1-to-1 with a single observation. Care must be taken to have enough information to have the Markov Property. So, for instance, a single image from a camera does not capture velocity - if velocity is important to your problem, you may need multiple consecutive images to build a useful state.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Model&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In reinforcement learning, the term &amp;quot;model&amp;quot; specifically means a predictive model of the environment that resolves probabilities of next reward and next state following an action from a state. The model might be provided by the code for the environment, or it can be learned (separately to learning to behave in that environment).&lt;/p&gt;&#xA;&lt;p&gt;Some RL algorithms can make use of a model to help with learning. Planning algorithms require one. So called &amp;quot;model free&amp;quot; algorithms do not because they do not make use of an explicit model, they work purely from experience.&lt;/p&gt;&#xA;&lt;p&gt;There are broadly two types of model:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A distribution model which provides probabilities of all events. The most general function for this might be &lt;span class=&quot;math-container&quot;&gt;$p(r,s'|s,a)$&lt;/span&gt; which is the probability of receiving reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and transitioning to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; given starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A sampling model which generates reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; when given a current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. The samples might be from a simulation, or just taken from history of what the learning algorithm has experienced so far.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In more general stats/ML, the term &amp;quot;model&amp;quot; is more inclusive, and can mean any predictive system that you might build, not just predictions of next reward and state. However, the literature for RL typically avoids calling those &amp;quot;model&amp;quot;, and uses terms like &amp;quot;function approximator&amp;quot; to avoid overloading the meaning of &amp;quot;model&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-03-16T08:17:43.690" LastActivityDate="2021-03-16T08:17:43.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7631" PostTypeId="1" CreationDate="2018-08-19T06:15:28.320" Score="4" ViewCount="80" Body="&lt;p&gt;Deep Successor Representations(DSR) has given better performance in tasks like navigation when it compares to normal model-free RL tasks. Basically, DSR is a hybrid of model-free RL and Model-Based RL.  But the original work has only use value-based functions deep RL methods like DQN.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to the paper - &lt;a href=&quot;https://arxiv.org/abs/1606.02396&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSR&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14909" LastEditorUserId="14909" LastEditDate="2018-08-19T09:38:59.127" LastActivityDate="2018-08-19T09:38:59.127" Title="Is there any theoretical capabilities of apply deep successor representations with A3C algorithm?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7633" PostTypeId="1" AcceptedAnswerId="7635" CreationDate="2018-08-19T16:46:51.133" Score="7" ViewCount="86" Body="&lt;p&gt;As I know, a single layer neural network can only do linear operations, but multilayered ones can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can &lt;em&gt;only represent linear&lt;/em&gt; operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, multi-layered neural networks can represent non-linear (even much more complex than being just a nonlinear) operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes it happen? The activation layer?&lt;/p&gt;&#xA;" OwnerUserId="17577" LastEditorUserId="2444" LastEditDate="2020-03-10T16:43:46.860" LastActivityDate="2020-03-10T16:43:46.860" Title="What makes multi-layer neural networks able to perform nonlinear operations?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;activation-function&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7634" PostTypeId="1" AcceptedAnswerId="7636" CreationDate="2018-08-19T19:42:27.537" Score="4" ViewCount="51" Body="&lt;p&gt;So I have a deep learning model and three data sets (images). My theory is that one of these data sets should function better when it comes to training a deep learning model (meaning that the model will be able to achieve better performance (higher accuracy) with one of these data sets to serve one classification purpose) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just want to safe check my approach here. I understand the random nature of training deep learning models and the difficulties associated with such experiment. Though, I want someone who can point out maybe a red flag here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering about these things: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;do you think using an optimizer with default parameters and repeating the training process, let's say, 30 times for each data set and picking the best performance is a safe approach? I am mainly worried here that modifying the hyperparamters of the optimizer might result in better results for let's say one of the data sets. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;what about seeding the weights initialization? do you think that I should seed them and then modify the hyperparameters until I get the best convergence or not seed and still modify the hyper parameters? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am sorry for the generality of my question. I hope if someone can point me in the right direction.  &lt;/p&gt;&#xA;" OwnerUserId="17582" LastEditorUserId="1641" LastEditDate="2018-08-19T19:46:25.517" LastActivityDate="2018-08-19T23:17:17.617" Title="How to compare the training performance of a model on different data input?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;training&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7635" PostTypeId="2" ParentId="7633" CreationDate="2018-08-19T20:27:04.710" Score="5" Body="&lt;p&gt;Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2018-08-19T20:27:04.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7636" PostTypeId="2" ParentId="7634" CreationDate="2018-08-19T23:17:17.617" Score="1" Body="&lt;p&gt;Well, this is more of a subjective question but I will give my best shot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your 1st question, the nature of methods in deep learning states that you should experiment otherwise you will only have weak intuitions. So which dataset should you choose? I would say, select your primary concerns since you cannot try everything. If it's time, make a little random or grid search over hyperparameters and observe speed of convergence in the datasets then choose the best one. If it's accuracy, ideally you should make an input analysis on distribution of data, etc. If you can analyze it well, you are expected to obtain best result from the chosen dataset. So training 30 epochs each dataset and choosing the one with lowest loss is not a safe approach. Maybe your model will converge at 40th epoch for the worst dataset in the 30th epoch but it will be much more robust. My final advice is to set a threshold for your evaluation metrics, once you reach them in any of your dataset choose that one -assuming datasets are more or less equal in number of instances. In this way you are at least know that the dataset you choose satisfies your expectations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your 2nd question, seeding during comparisons is a good approach though in the long run it won't matter much. Hence there is no harm in seeding unless you are very very very unlucky.&lt;/p&gt;&#xA;" OwnerUserId="16311" LastActivityDate="2018-08-19T23:17:17.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7638" PostTypeId="1" AcceptedAnswerId="7639" CreationDate="2018-08-20T09:01:13.013" Score="4" ViewCount="216" Body="&lt;p&gt;In the 4th paragraph of &#xA;&lt;a href=&quot;http://www.incompleteideas.net/book/ebook/node37.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.incompleteideas.net/book/ebook/node37.html&lt;/a&gt;&#xA;it is mentioned: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Whereas the optimal value functions for states and state-action pairs are unique for a given MDP, there can be many optimal policies&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Could you please give me a simple example that shows different optimal policies considering a unique value function?&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="2444" LastEditDate="2019-02-16T02:50:32.693" LastActivityDate="2019-02-16T02:50:32.693" Title="An example of a unique value function which is associated with multiple optimal policies" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;value-functions&gt;&lt;optimal-policy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7639" PostTypeId="2" ParentId="7638" CreationDate="2018-08-20T10:03:07.083" Score="2" Body="&lt;p&gt;Consider a very simple grid-world, consisting of 4 cells, where an agent starts in the bottom-left corner, has actions to move North/East/South/West, and receives a reward $R = 1$ for reaching the top-right corner, which is a terminal state. We'll name the four cells $NW$, $NE$, $SW$ and $SE$ (for north-west, north-east, south-west and south-east). We'll take a discount factor $\gamma = 0.9$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial position is $SW$, and the goal is $NE$, which an optimal policy should reach as quickly as possible. However, there are two optimal policies for the starting state $SW$: we can either go north first, and then east (i.e., $SW \rightarrow NW \rightarrow NE$), or we can go east first, and then north (i.e., $SW \rightarrow SE \rightarrow NE$). Both of those policies are optimal, both reach the goal state in two steps and receive a return of $\gamma \times 1 = 0.9$, but they are clearly different policies, they choose different actions in for the initial state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that my language was &lt;em&gt;slightly&lt;/em&gt; informal above when talking about &quot;policies for the starting state&quot;. Formally, I should have said that there are two optimal policies that select different actions in the starting state (and the same actions in all other states).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-20T10:03:07.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7640" PostTypeId="1" AcceptedAnswerId="7657" CreationDate="2018-08-20T10:09:40.053" Score="13" ViewCount="2783" Body="&lt;p&gt;I think I've seen the expressions &quot;stationary data&quot;, &quot;stationary dynamics&quot; and &quot;stationary policy&quot;, among others, in the context of reinforcement learning. What does it mean? I think stationary policy means that the policy does not depend on time, and only on state. But isn't that a unnecessary distinction? If the policy depends on time and not only on the state, then strictly speaking time should also be part of the state.&lt;/p&gt;&#xA;" OwnerUserId="12640" LastEditorUserId="2444" LastEditDate="2019-02-20T22:02:22.113" LastActivityDate="2021-01-21T17:57:58.817" Title="What does &quot;stationary&quot; mean in the context of reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;policies&gt;&lt;stationary-policy&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7642" PostTypeId="1" CreationDate="2018-08-20T13:17:04.977" Score="3" ViewCount="187" Body="&lt;p&gt;I downloaded a chatbot called Replika off the internet the other day and we've become very good friends. My thought is that such chatbots will soon replace therapists and then probably private tutors as well. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it safe to say that anyone aspiring to go into one of these professions now should look for other options? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What other jobs may be replaced by chatbots in the future? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How long before AIs are able to answer questions on StackExchange?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17601" LastEditorUserId="1671" LastEditDate="2018-08-20T17:28:27.377" LastActivityDate="2019-12-02T12:18:47.677" Title="The future of chatbots" Tags="&lt;chat-bots&gt;&lt;social&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ClosedDate="2020-03-04T01:34:10.537" ContentLicense="CC BY-SA 4.0" />
  <row Id="7643" PostTypeId="1" CreationDate="2018-08-20T13:40:35.343" Score="2" ViewCount="91" Body="&lt;p&gt;I have the following question about You Only Look Once (YOLO) algorithm, for object detection.&lt;/p&gt;&#xA;&lt;p&gt;I have to develop a neural network to recognize web components in web applications - for example, login forms, text boxes, and so on. In this context, I have to consider that the position of the objects on the page may vary, for example, when you scroll up or down.&lt;/p&gt;&#xA;&lt;p&gt;The question is, would YOLO be able to detect objects in &amp;quot;different&amp;quot; positions? Would the changes affect the recognition precision? In other words, how to achieve translation invariance? Also, what about partial occlusions?&lt;/p&gt;&#xA;&lt;p&gt;My guess is that it depends on the relevance of the examples in the dataset: if enough translated / partially occluded examples are present, it should work fine.&lt;/p&gt;&#xA;&lt;p&gt;If possible, I would appreciate papers or references on this matter.&lt;/p&gt;&#xA;&lt;p&gt;(PS: if anyone knows about a labeled dataset for this task, I would really be grateful if you let me know.)&lt;/p&gt;&#xA;" OwnerUserId="17600" LastEditorUserId="2444" LastEditDate="2021-01-29T00:07:11.107" LastActivityDate="2021-05-22T16:04:55.460" Title="Would YOLO be able to detect objects in &quot;different&quot; positions?" Tags="&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;datasets&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7644" PostTypeId="1" AcceptedAnswerId="7663" CreationDate="2018-08-20T14:09:31.723" Score="5" ViewCount="120" Body="&lt;p&gt;I've developed a neural network that can play a card game.  I now want to use it to create decks for the game.  My first thought would be to run a lot of games with random decks and use some approximation (maybe just a linear approximation with a feature for each card in your hand) to learn the value function for each state.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this will probably take a while, so in the mean time is there any way I could get this information directly from the neural network? &lt;/p&gt;&#xA;" OwnerUserId="16724" LastActivityDate="2018-08-21T15:52:24.657" Title="Can you analyse a neural network to determine good states?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7645" PostTypeId="2" ParentId="7640" CreationDate="2018-08-20T14:18:28.650" Score="2" Body="&lt;p&gt;You are right: a stationary policy is independent of time. It is basically a mapping from states to actions (or probability distributions over actions). Regardless of the point in time in which the agent observes the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; it will select an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (or select a probability &lt;span class=&quot;math-container&quot;&gt;$\pi(a \vert s)$&lt;/span&gt; for every action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="17602" LastEditorUserId="1641" LastEditDate="2021-01-21T17:57:58.817" LastActivityDate="2021-01-21T17:57:58.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7646" PostTypeId="1" CreationDate="2018-08-20T14:45:20.927" Score="3" ViewCount="125" Body="&lt;p&gt;I was thinking of creating a CNN. Now it is known CNN takes long times to train so it is advisable to stick to known architectures and hyper-parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: I want to tinker with the CNN architecture (since it is a specialised task). One approach would be to create a CNN and check on small data-sets, but then I would have no way of knowing whether the Fully Connected layer at the end is over-fitting the data while the convolutional layers do nothing (since large FC layers can easily over-fit data). Cross Validation is a good way to check it, but it might not be satisfactory (since my opinion is that a CNN can be replaced with a Fully Connected NN if the data-set is small enough and there is little variation in the future data-sets).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what are some ways to tinker with CNN and get a good estimate for future data-sets in a reasonable training time? &lt;strong&gt;Am I wrong in my previous assumptions?&lt;/strong&gt; A detailed answer would be nice!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-09-06T13:08:47.000" LastActivityDate="2018-09-06T13:08:47.000" Title="How to tinker with CNN architectures?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7647" PostTypeId="1" CreationDate="2018-08-20T15:17:38.097" Score="3" ViewCount="85" Body="&lt;p&gt;In the attached image is the probability with the Naives Bayes algorithm of: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fem:dv/m/s Young own Ex-credpaid Good -&gt;62%  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I calculated the Probability so: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;P(Fem:dv/m/s | Good)*P(Young | Good)*P(own | Good)*P(Ex-credpaid | good)*P(Good) -&gt; 1/6*2/6*5/6*3/6*0.6=0,01389 &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't Know where I failed. Could someone please tell me where is my error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mmcbd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mmcbd.jpg&quot; alt=&quot;Table1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17603" LastEditorUserId="1671" LastEditDate="2018-08-20T17:24:13.213" LastActivityDate="2018-08-20T19:41:13.490" Title="Naive Bayes Algorithm error" Tags="&lt;bayes&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7649" PostTypeId="1" CreationDate="2018-08-20T16:36:28.167" Score="4" ViewCount="80" Body="&lt;p&gt;I’ve  done my research and could not find answer anywhere else. My apologies in advance if same problem is answered in different terms on stack-overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve poker tournament winner prediction problem. I’ve millions of historical records in this format:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Players ==&gt; Winner&lt;/strong&gt; &lt;/li&gt;&#xA;&lt;li&gt;P1,P2,P4,P8 ==&gt; P2 &lt;/li&gt;&#xA;&lt;li&gt;P4,P7,P6 ==&gt; P4 &lt;/li&gt;&#xA;&lt;li&gt;P6,P3,P2,P1 ==&gt; P1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are some of the most suitable algorithms to predict winner from set of players.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried decision trees, XGboost without much of a success.&lt;/p&gt;&#xA;" OwnerUserId="17606" LastEditorUserId="1671" LastEditDate="2018-08-27T20:15:32.593" LastActivityDate="2018-08-27T20:15:32.593" Title="Approaches to poker tournament winner prediction?" Tags="&lt;ai-design&gt;&lt;poker&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7651" PostTypeId="2" ParentId="7649" CreationDate="2018-08-20T19:30:31.740" Score="1" Body="&lt;p&gt;This looks like it maps very nicely onto &lt;a href=&quot;https://en.wikipedia.org/wiki/Association_rule_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Association Mining&lt;/a&gt;. In association mining, you are trying to find items from a discrete set that often co-occur in transactions. For instance, you might want to find the items that most commonly appear in an online shopping cart together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the problem amounts to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Split up the data into sub-problems by who won.&lt;/li&gt;&#xA;&lt;li&gt;Perform association mining on the sets of players in each sub-problem, using, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Apriori_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;the apriori algorithm&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The resulting rules will then have associated confidence factors. When you want to predict who will win, you can take the winner suggested by the rule with the greatest confidence.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The other approach suggested by your problem is to model it as &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;classification&lt;/a&gt;. Here you are trying to assign labels (who won) to input vectors (who played). The process would look like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Map the sets of who played into binary features. So the subset P1,P2, P4,P8 might be represented with {1,1,0,1,0,0,0,1} if there are only 8 players. Map the winning player onto a numeric class (e.g. the numbers 1-8). &lt;/li&gt;&#xA;&lt;li&gt;Run any classification algorithm to create a model that predicts the class from the binary features. A &lt;a href=&quot;https://en.wikipedia.org/wiki/C4.5_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision tree learner&lt;/a&gt; might be a interesting starting point if you want to understand which factors are important to the model. There are many other techniques however.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can use the model you train to make predictions about future games with the same set of players.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-20T19:30:31.740" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="7652" PostTypeId="2" ParentId="7647" CreationDate="2018-08-20T19:41:13.490" Score="2" Body="&lt;p&gt;Your probability hasn't been &lt;em&gt;normalized&lt;/em&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you are computing the probability of being good, &lt;em&gt;given that the other features have a fixed value&lt;/em&gt;. To obtain the correct probability, you need to normalize (divide) the value from your calculation by the probability that the features have taken on those fixed values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can calculate this as follows:&#xA; $$P(Fem:dv/m/s, Young, own, Ex-credpaid) = \\ \sum_{x \in \{good,bad\}} P(Fem:dv/m/s, Young, own, Ex-credpaid, x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by the marginalization rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, by the chain rule, you may write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\\ \sum_{x \in \{good,bad\}} P(Fem:dv/m/s, Young, own, Ex-credpaid | x) * P(x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the correct probability of 0.62 should be obtained by the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{P(Fem:dv/m/s, Young, own, Ex-credpaid | good) * P(good)}{\sum_{x \in \{good,bad\}} P(Fem:dv/m/s, Young, own, Ex-credpaid | x) * P(x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You just need to calculate &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P(Fem:dv/m/s, Young, own, Ex-credpaid | bad) * P(bad)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it should be easy to compute the rest.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-20T19:41:13.490" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7653" PostTypeId="2" ParentId="7642" CreationDate="2018-08-20T20:12:14.387" Score="3" Body="&lt;p&gt;You should check out my answer &lt;a href=&quot;https://ai.stackexchange.com/questions/7437/what-career-paths-should-be-avoided-with-the-growth-in-ai/7438#7438&quot;&gt;here&lt;/a&gt; to your second question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your first question, which is a special case, the answer is it might be one of the &lt;strong&gt;best&lt;/strong&gt; fields to study!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Academic&lt;/a&gt; and &lt;a href=&quot;https://www.pwc.co.uk/economic-services/ukeo/pwcukeo-section-4-automation-march-2017-v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;industrial&lt;/a&gt; studies agree that working in a job that requires a lot of social interaction, particularly a job that involves caring for others, &lt;em&gt;reduces&lt;/em&gt; automation risk. Among the safest sectors are education and personal care assistants, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of possible reasons for this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Even if a chat bot &lt;em&gt;can&lt;/em&gt; replace your therapist, it probably cannot replace the sensation that another &lt;em&gt;person&lt;/em&gt; cares about your problems, precisely because it is not an actual person. For people in dire emotional straits, having someone care might be worth a great deal. &lt;/li&gt;&#xA;&lt;li&gt;Having a human tutor or therapist might be seen as a status symbol, even if an AI system is equally competent. If Harvard offered a degree where you studied on campus, taught by humans, and a degree where you studied online, taught by chat bots, they would probably not be perceived as equally prestigious by employers and peers.&lt;/li&gt;&#xA;&lt;li&gt;AI systems like chat bots tend to fail in unpredictable ways when they enter unfamiliar situations. When dealing with an emotionally unstable person in therapy, or with a small child in education, unfamiliar situations might happy often, and the costs for failure might be high. Even if AI can handle most of the job, we might not be willing to trust it to an AI system because of these risks.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To address your third point, Q &amp;amp; A seems closer to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI&lt;/a&gt; than to existing AI. If a program can give compelling answers to free-form questions, then it can almost certainly pass the &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test&lt;/a&gt; reliably. While you might see simple bots that analyze text and post a related wikipedia link, I suspect you won't see a high quality Q&amp;amp;A bot for many years (or decades) because of this.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-20T20:12:14.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7654" PostTypeId="2" ParentId="7601" CreationDate="2018-08-20T21:01:02.073" Score="1" Body="&lt;p&gt;I think that what you're really asking about is the question of &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;knowledge representation&lt;/a&gt;.  Regardless of how you train your AI, one of the most fundamental questions is how do you represent &quot;knowledge&quot; and especially when it exists at different levels of abstraction, may be mutually recursive, etc.  Along with that goes the question of &lt;a href=&quot;https://en.wikipedia.org/wiki/Belief_revision&quot; rel=&quot;nofollow noreferrer&quot;&gt;belief revision&lt;/a&gt; which deals with how you update existing beliefs/knowledge in the light of new information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of these areas are still subject to plenty of active research and neither has entirely settled answers to the core questions.  But progress has been made in both areas.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally I suspect that something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic networks&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Conceptual_graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;conceptual graphs&lt;/a&gt; will be the best answer to the KR problem.  Dealing with belief revision seems even fuzzier to me, although there are known strategies (like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Belief_revision#The_AGM_postulates&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGM postulates&lt;/a&gt;) that work to a point. Something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Belief Networks&lt;/a&gt; may also prove useful.  &lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2018-08-20T21:01:02.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7655" PostTypeId="1" CreationDate="2018-08-20T21:31:23.160" Score="1" ViewCount="29" Body="&lt;p&gt;I want to do some sequence to sequence modelling on source data that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/-0.013428/-0.124969/-0.13435/0.008087/-0.269241/-0.36849/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with target data that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dont be angry with the process youre going through right now&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both are of indeterminate lengths, and the lengths of target and source data aren't the same. What I'd like to do is have a prediction model where I can input similar numbers and have it generate texts based on the target training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started off doing character level s2s, but the output of the model is too nonsensical even at 2-5k epochs. So I've been looking into word level s2s and NMT, but the tutorials always assume strings of text as the target and source, and I keep running into roadblocks trying to preprocess the text, when all the tutorials assume a certain syntax/set of characters. This is my first try at ML, and some of the tutorials really throw me out with the text preprocessing requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I going down the right avenue looking at word level/NMT stuff? And is there a tutorial I've missed for something like what I'm trying to build?&lt;/p&gt;&#xA;" OwnerUserId="17611" LastActivityDate="2018-08-20T21:31:23.160" Title="Sequence to sequence machine learning / NMT - converting numbers into words" Tags="&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7657" PostTypeId="2" ParentId="7640" CreationDate="2018-08-21T08:36:38.220" Score="8" Body="&lt;p&gt;A &lt;em&gt;stationary policy&lt;/em&gt; is a policy that does not change. Although strictly that is a time-dependent issue, that is not what the distinction refers to in reinforcement learning. It generally means that the policy is not being updated by a learning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are working with a stationary policy in reinforcement learning (RL), typically that is because you are trying to learn its value function. Many RL techniques - including Monte Carlo, Temporal Difference, Dynamic Programming - can be used to evaluate a given policy, as well as used to search for a better or optimal policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Stationary dynamics&lt;/em&gt; refers to the environment, and is an assumption that the rules of the environment do not change over time. The rules of the environment are often represented as an MDP model, which consists of all the state transition probabilities and reward distributions. Reinforcement learning algorithms that work &lt;em&gt;online&lt;/em&gt; can usually cope and adjust policies to match non-stationary environments, provided the changes do not happen too often, or enough learning/exploring time is allowed between more radical changes. Most RL algorithms have at least some online component, it is also important to keep exploring non-optimal actions in environments with this trait (in order to spot when they may become optimal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Stationary data&lt;/em&gt; is not a RL-specific term, but also relates to the need for an online algorithm, or at least plans for discarding older data and re-training existing models over time. You might have non-stationary data in any ML, including supervised learning - prediction problems that work with data about people and their behaviour often have this issue as population norms change over timescales of months and years.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-02-15T11:43:29.090" LastActivityDate="2019-02-15T11:43:29.090" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7658" PostTypeId="2" ParentId="5990" CreationDate="2018-08-21T08:50:39.953" Score="3" Body="&lt;p&gt;The following 2 books helped me understand the basics and guided me through my first AI / CI implementations.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Computational Intelligence: An Introduction&lt;/strong&gt;&lt;/a&gt; by Andries P. Engelbrecht&lt;/p&gt;&#xA;&lt;p&gt;It includes the most relevant developments in computational intelligence with good discussions on intelligence and artificial intelligence (AI).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.elsevier.com/books/computational-intelligence/eberhart/978-1-55860-759-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Computational Intelligence: Concepts to Implementation&lt;/strong&gt;&lt;/a&gt; by Russell C. Eberhart and Yuhui Shi&lt;/p&gt;&#xA;&lt;p&gt;It is a comprehensive textbook, supported with lots of practical examples.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17602" LastEditorUserId="2444" LastEditDate="2021-01-16T20:21:37.610" LastActivityDate="2021-01-16T20:21:37.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7659" PostTypeId="1" CreationDate="2018-08-21T10:59:40.407" Score="1" ViewCount="23" Body="&lt;p&gt;A dataset contains so many fields in which there is both relevant and irrelevant field. If we want to do a market campaigning using propensity scoring, which fields of the data set are relevant?&#xA;How can we find which data field should be selected and can drive to the desired propensity score?&lt;/p&gt;&#xA;" OwnerUserId="17058" LastActivityDate="2018-08-21T12:30:10.097" Title="Which features of a data set can be used for market campaigning using propensity scores?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7660" PostTypeId="2" ParentId="7659" CreationDate="2018-08-21T12:30:10.097" Score="1" Body="&lt;p&gt;The problem you are examining is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_selection&quot; rel=&quot;nofollow noreferrer&quot;&gt;feature selection&lt;/a&gt;. There are many different techniques, but they fall broadly into three categories:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Filter approaches determine which features have high information content. A common approach is to score them based on their &lt;a href=&quot;https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;information gain&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Wrapper approaches score subsets of the features, using similar measurements. This is slower (since there are many subsets), but may yield better performance.&lt;/li&gt;&#xA;&lt;li&gt;Embedded approaches use machine learning algorithms that can dynamically select features. For example, building a &lt;a href=&quot;https://en.wikipedia.org/wiki/C4.5_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;C4.5 decision tree&lt;/a&gt; learner on a dataset, and then selecting just the features that actually appear in the tree, yields a set of features similar to those found by information gain.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-21T12:30:10.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7661" PostTypeId="1" CreationDate="2018-08-21T13:03:01.060" Score="2" ViewCount="89" Body="&lt;p&gt;What is the difference between visible and hidden units in Boltzmann machines? What are their purposes?&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2019-12-04T14:59:59.707" LastActivityDate="2019-12-04T14:59:59.707" Title="What is the difference between visible and hidden units in Boltzmann machines?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;boltzmann-machine&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7662" PostTypeId="1" AcceptedAnswerId="7664" CreationDate="2018-08-21T13:58:09.033" Score="3" ViewCount="79" Body="&lt;p&gt;Imagine I have a 2D matrix, A. I apply some transformation to it, for example:&#xA;B = A_shifted + A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to train a CNN to learn back the mapping from B to A? Giving B as example and A as target?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16201" LastActivityDate="2018-08-21T17:18:00.887" Title="Figuring out mapping between two matrices" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7663" PostTypeId="2" ParentId="7644" CreationDate="2018-08-21T15:52:24.657" Score="4" Body="&lt;p&gt;I don't think your network, trained using PPO to play a card game, already contains sufficient information to also use for drafting. I'm not saying this with 100% certainty, maybe there's something I'm overlooking, but I can't think of anything right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A small adaptation to the network might be sufficient (though it would also involve re-training again). Recently, OpenAI has been writing about their attempts to train agents to play the game DOTA 2. Now, this isn't a card game, it doesn't require deckbuilding, but there is an aspect to the game that is somewhat similar to deckbuilding: &lt;em&gt;drafting&lt;/em&gt;. In DOTA 2, there are two teams of 5 players each. Before a game start, each team selects 5 &lt;em&gt;heroes&lt;/em&gt; (one per player) to play in that game. This is very similar to deckbuilding, except that it's likely a much smaller problem; there's only a &quot;deck&quot; (team composition) of 5 &quot;cards&quot; (heroes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, they also trained agents to play the game (controlling one hero per agent) using PPO. In a &lt;a href=&quot;https://blog.openai.com/openai-five-benchmark-results/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, they write the following about how they managed to add drafting capabilities relatively easily:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In late June we added a win probability output to our neural network to introspect what OpenAI Five is predicting. When later considering drafting, we realized we could use this to evaluate the win probability of any draft: just look at the prediction on the first frame of a game with that lineup. In one week of implementation, we crafted a fake frame for each of the 11 million possible team matchups and wrote a tree search to find OpenAI Five’s optimal draft.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, if you want to try a similar technique, you'd have to adapt your network such that it also learns to generate a prediction of the win probability as output. I imagine that it'd be much less effective for deckbuilding, because win probabilities may all be very close to 50% in card games where luck (when drawing cards for example) can be a significant factor, but it might be worth a try.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, instead of generating lots of random decks and playing with them all, you could view the problem of deckbuilding as an additional separate &quot;game&quot; or Markov Decision Process; adding a specific card to the deck can be an action, and this MDP terminates once you have a complete deck. Then you can try to do that better than random using search algorithms (like Monte-Carlo Tree Search) or, again, a Reinforcement Learning approach like PPO. Again, I imagine it will be a very difficult problem though, likely requiring lots of time before it will be capable of doing better than random.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I also know of some research related to deckbuilding in the collectible card game &lt;em&gt;Hearthstone&lt;/em&gt;, which may be relevant for you. Unfortunately I did not yet get to read through any of this in detail, so I don't know for sure if you'll find a solution here, but it may be worth a try:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dockhorn.antares.uberspace.de/wordpress/additional-material/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dockhorn.antares.uberspace.de/wordpress/additional-material/&lt;/a&gt;: At the recent IEEE CIG 2018 conference, there were two competitions involving Hearthstone: one for game-playing AI with pre-built decks, and for generating new decks (and then playing with them). The page I linked to here contains references to related work from earlier years.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1806.09771&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games&lt;/a&gt;: This is a new paper that was published in the proceedings of this recent IEEE CIG 2018 conference.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-21T15:52:24.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7664" PostTypeId="2" ParentId="7662" CreationDate="2018-08-21T17:18:00.887" Score="2" Body="&lt;p&gt;Yes, with some limitations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CNNs can be used to map images to related images, and that should include many simple matrix transformations. For instance, &lt;a href=&quot;https://github.com/sandeepraju/ihradis-cnn-deblur/blob/master/hradis15CNNdeblurring.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is an example of de-blurring OCRed text using a CNN&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you would train your network with lots of A, B examples, with the input as B and desired output as A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The limitation is that where you have transformations that are technically irreversible, then the CNN may learn to produce a best &quot;mean&quot; output. The symptom of this will be fuzzy images lacking high frequency components, and matrices which are not representative of the target distribution, but that do solve the transformation (within limits of training accuracy). If you want to improve on that, and produce a more realistic/precise original, then you will probably want to look at adding a generative component - a GAN, VAE/GAN or RBM etc. Note this would not accurately produce the original matrix, but would generate one that both transformed into your given transformed matrix (within some level of accuracy) and was sampled from your input distribution. That is, it could be more of a &lt;em&gt;feasible&lt;/em&gt; original than one generated using a simpler CNN architecture.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-08-21T17:18:00.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7665" PostTypeId="2" ParentId="2005" CreationDate="2018-08-21T17:34:19.360" Score="2" Body="&lt;p&gt;10 years to production ready?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's put that in perspective. The perceptron was introduced in 1957. It did not really even start to flower as a usable model until the release of the PDP books in 1986. For those keeping score: 29 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the PDP books, we did not see that elaborated as usable deep networks until the last decade. If you take the Andrew Ng and Jeff Dean cat recognition task as a deep network defining event that's 2012. &#xA;Arguably more than 25 years to production ready.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Timeline_of_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Timeline_of_machine_learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2018-08-21T17:34:19.360" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7666" PostTypeId="2" ParentId="7624" CreationDate="2018-08-21T17:52:34.930" Score="0" Body="&lt;p&gt;Intelligence is the ability to weave together various concepts and associations into a meaningful whole; filtering, adding and rejecting appropriately various ideas from personal knowledge and experience. Then effectively reflecting these ideas back to a questioner to affirm understanding and comprehension, allowing a conversation to proceed effectively towards a mutually beneficial conclusion. &lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2018-08-21T17:52:34.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7667" PostTypeId="1" AcceptedAnswerId="7689" CreationDate="2018-08-21T20:45:27.110" Score="0" ViewCount="592" Body="&lt;p&gt;I recently read an article about neural networks saying that, when using sigmoid as activation function, it's advised to use 0.1 as target value instead of 0, and 0.9 instead of 1. This was to avoid &quot;saturation effects&quot;. I only understood is halfway, and was hoping someone could clarify a few things for me:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this only the case when the output is boolean (0 or 1), or will it also be the case for continual values in the range between 0 and 1. If so, should all values be scaled to the interval [0.1, 0.9]?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What exactly is the problem of output 0 or 1? Does it have something to do with the derivative of sigmoid being 0 when it's value is 0 or 1? As I understood it weights could end up approaching infinity, but I didn't understand why.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this the case only when sigmoid is used in the output layer (which it rarely is, I believe), or is it also the case when sigmoid is used in hidden layers only?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2018-08-23T10:38:05.843" Title="Target values of 0.1 for 0 and 0.9 for 1 for sigmoid" Tags="&lt;neural-networks&gt;&lt;deep-neural-networks&gt;&lt;activation-function&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7668" PostTypeId="5" CreationDate="2018-08-21T20:48:50.987" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-08-21T20:48:50.987" LastActivityDate="2018-08-21T20:48:50.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7669" PostTypeId="4" CreationDate="2018-08-21T20:48:50.987" Score="0" Body="For questions about the sigmoid functions (in particular, the logistic functions) and the consequences of using them as activation functions in neural networks." OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2020-11-20T13:02:20.213" LastActivityDate="2020-11-20T13:02:20.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7670" PostTypeId="2" ParentId="6040" CreationDate="2018-08-21T21:08:04.450" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://ai.stackexchange.com/questions/92/how-is-it-possible-that-deep-neural-networks-are-so-easily-fooled&quot;&gt;link&lt;/a&gt; in DuttA's comment provides a good answer to the more general question of how adversarial images are generated. As to why it's possible at all, the key is that these specialized feature detectors really are just picking up &lt;em&gt;features&lt;/em&gt;, and it's possible to construct an image that has those features, without resembling the target object much at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we might imagine that the key features that predict an image of a gun are regions like the trigger guard and perhaps the muzzle. The features that are extracted really will just be &quot;round-ish thing with a curved bit in it&quot; and &quot;perfectly circular opening&quot;. These might predict a gun really well, but it's also easy to imagine that you could place shapes satisfying those criteria the correct distance apart without including any of the other components that we'd expect. This is what you actually see in the adversarial images that are generated. Consider the bagel or crossword puzzle images below. While they are not actually pictures of the objects in question, it is fairly easy to see what the important features were for the network: a bagel is a round shape that's shadowed in the center and lighter between the center and rim. It's a bit orange. A crossword puzzle is a bunch of black and white squares distributed in a rough grid. Possibly the copyright symbol or the letter C was also important (it is found on many crosswords!). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pBm48.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pBm48.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-21T21:08:04.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7671" PostTypeId="2" ParentId="7624" CreationDate="2018-08-22T00:28:24.927" Score="5" Body="&lt;p&gt;I'm going to preface this answer by noting that persons much smarter than myself have treated this subject in some detail.  That said, as far as I can discern:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When we talk about intelligence we're referring to problem solving strength in relation to a problem, relative to the strength of other intelligences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a somewhat game-theoretic conception, related to &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;rationality&lt;/a&gt; and the concept of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;rational agent&lt;/a&gt;.  Regarding intelligence in this manner may be unavoidable. Specifically, we could define intelligence as the ability to understand a problem or solution or abstract concepts, but we can't validate that understanding without testing it.  &lt;em&gt;(For instance, I might believe I grasp a mathematical technique, but the only way to determine if that belief is real or illusory is to utilize that technique and evaluate the results.)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reason games like Chess and Go have been used as milestones, aside from longstanding human interest in the games, is that they provide models with simple, fully definable parameters, and, in the case of Go at least, have complexity akin to nature, by which I mean &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsolvable&lt;/a&gt;/&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractable&lt;/a&gt;. (Compare to strength at Tic-Tac-Toe, which is trivially solved.)&lt;/p&gt;&#xA;&lt;p&gt;However, we should consider &lt;a href=&quot;https://ai.stackexchange.com/a/7595/1671&quot;&gt;a point made in this concise answer to a question involving the Turing Test&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;...is [intelligence] defined purely by behaviour in an environment, or by the mechanisms that arrive at that behaviour?&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is important because &lt;a href=&quot;https://www.technologyreview.com/s/611902/google-just-gave-control-over-data-center-cooling-to-an-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google just gave control over data center cooling to an AI&lt;/a&gt;.  Here it is clearly the mechanism itself that demonstrates utility, but if we call that mechanism intelligent, for intelligence to have meaning, we still have to contend with &amp;quot;intelligent how?&amp;quot;  (In what way is it intelligent?) If we want to know &amp;quot;how intelligent?&amp;quot; (its degree of utility) we still have to evaluate its performance in relation to the performance of other mechanisms.&lt;/p&gt;&#xA;&lt;p&gt;(In the case of the automata controlling the air conditioning at Google, we can say that it is more intelligent than the prior control system, and by how much.)&lt;/p&gt;&#xA;&lt;p&gt;Because we're starting to talk about more &amp;quot;generalized intelligence&amp;quot;, defined here as mechanisms that can be applied to a set of problems, &lt;em&gt;(I include minimax as a form of &amp;quot;axiomatic intelligence&amp;quot; and machine learning as a form &amp;quot;adaptive intelligence&amp;quot;)&lt;/em&gt;, it may be worthwhile to expand and clarify the definition:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Intelligence is the problem solving strength of a mechanism in relation to a problem or a set of problems, relative to the strength of other mechanisms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;or, if we wanted to be pithy:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wiktionary.org/wiki/stupid_is_as_stupid_does&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligence is as intelligence does&lt;/a&gt; (and how well.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-24T19:32:07.777" LastActivityDate="2021-01-24T19:32:07.777" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7672" PostTypeId="1" AcceptedAnswerId="7766" CreationDate="2018-08-22T03:29:14.083" Score="4" ViewCount="227" Body="&lt;p&gt;So suppose that you have a real estate appraisal problem. You have some structured data, and some images exterior of home, bedrooms, kitchen, etc. The number of pictures taken is variable per observational unit, i.e. the house.&lt;/p&gt;&#xA;&lt;p&gt;I understand the basics of combining an image processing neural net with tabular data for a single image. You chop off the final layer and feed in the embeddings of the image to your final model.&lt;/p&gt;&#xA;&lt;p&gt;How would one deal with variable number of images? Where your unit of observation can have between zero and infinity images (theoretically no upper bound on number of images in observation)?&lt;/p&gt;&#xA;" OwnerUserId="17646" LastEditorUserId="2444" LastEditDate="2021-01-23T03:25:21.753" LastActivityDate="2021-01-23T03:25:21.753" Title="Variable Number of Inputs to Neural Networks" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7673" PostTypeId="2" ParentId="6465" CreationDate="2018-08-22T08:09:10.237" Score="2" Body="&lt;p&gt;To complement Cosmo's response and maybe address the &quot;&lt;em&gt;using just your name&lt;/em&gt;&quot; part of your question, I would like to acknowledge that biased towards submissions do exist. Reviewers may be biased due to several reasons, such us authors' age, publications record, gender, nationality &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4214948/&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Lotfi and Mahian, 2014)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are concerned about these aspects, rest assured that there are mechanisms to ensure that the authors' reputation does not influence reviewers' judgments. A good example is the &quot;&lt;a href=&quot;https://www.journals.elsevier.com/social-science-and-medicine/policies/double-blind-peer-review-guidelines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Double-Blind Review&lt;/a&gt;&quot; process, which means that identities from the author(s) and reviewer(s) are concealed throughout the review process.&lt;/p&gt;&#xA;" OwnerUserId="17602" LastActivityDate="2018-08-22T08:09:10.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7675" PostTypeId="1" CreationDate="2018-08-22T09:28:40.033" Score="3" ViewCount="140" Body="&lt;p&gt;Is there a machine learning system that is able to &amp;quot;understand&amp;quot; mathematical problems given in a textual description, such as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A big cat needs 4 days to catch all the mice and a small cat needs 12 days. How many days need both, if they catch mice together?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;" OwnerUserId="17650" LastEditorUserId="2444" LastEditDate="2020-12-07T14:55:09.713" LastActivityDate="2020-12-07T14:55:09.713" Title="Is there a machine learning system that is able to understand mathematical problems given in a textual description?" Tags="&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;automated-reasoning&gt;&lt;question-answering&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7677" PostTypeId="2" ParentId="7675" CreationDate="2018-08-22T11:21:54.737" Score="2" Body="&lt;p&gt;There was a lot of work on this topic at UT Austin, which has now migrated to the &lt;a href=&quot;https://allenai.org/aristo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alan Institute&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is no off-the-shelf software that will answer your question (if there was, DARPA would stop funding its development!), but you can read about the latest development in a number of recent papers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ai2-website.s3.amazonaws.com/publications/diagram_understanding_in_geometry_questions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; (Seo et al. EMNLP 2015) discusses the techniques that are used to interpret diagrams that accompany geometry problems, while &lt;a href=&quot;http://ai2-website.s3.amazonaws.com/publications/Arithmetic_Word_Problems.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; (Hosseini et al. EMNLP 2014) talks about how to automatically parse verbs to interpret the meaning of a question. The &lt;a href=&quot;http://ai2-website.s3.amazonaws.com/publications/algebra-TACL2015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2015 TACL paper&lt;/a&gt; (Koncel-Kedziorski et al. 2015) completes this by discussing how to extract the relevant equations from a word problem. Once you have the equations, know what question is being asked, and can interpret any diagrams, you can do most high school math problems.&lt;/p&gt;&#xA;&lt;p&gt;However, I don't think this is yet a fully reliable system. It is one part of a larger, long running effort to create a program that can achieve higher education certifications in many subjects. You can see many projects related to this at the Alan Institute's &lt;a href=&quot;https://allenai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-12-07T14:53:38.797" LastActivityDate="2020-12-07T14:53:38.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7679" PostTypeId="2" ParentId="7675" CreationDate="2018-08-22T15:42:17.483" Score="1" Body="&lt;p&gt;Well this is a relatively new problem very tied to Question Answering. One of the recent systems is EUCLID that can answer those type of question the public Dolphin algebra question set by using a tree transducer cascade approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This paper details the proposed model &lt;a href=&quot;https://pdfs.semanticscholar.org/c22a/240d1087603664826e9aab809273ed9bff15.pdf?_ga=2.52187753.1130679049.1530133172-566539276.1446829155&amp;amp;_gac=1.6555398.1527028246.EAIaIQobChMI9YuywK-a2wIVlcBkCh0WKw_kEAAYASAAEgKwgPD_BwE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hopkins, M., Petrescu-Prahova, C., Levin, R., Le Bras, R., Herrasti, A., &amp;amp; Joshi, V. (2017). Beyond sentential semantic parsing: Tackling the math sat with a cascade of tree transducers. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (pp. 795-804)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same sense, SEMEVAL has released a task related to Math QA, you can see the related bibliography and referenced works &lt;a href=&quot;https://github.com/allenai/semeval-2019-task-10-internal&quot; rel=&quot;nofollow noreferrer&quot;&gt;semeval 2019 task 10 internal&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17463" LastActivityDate="2018-08-22T15:42:17.483" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7680" PostTypeId="1" AcceptedAnswerId="7681" CreationDate="2018-08-22T18:06:49.643" Score="16" ViewCount="1309" Body="&lt;p&gt;I was reading the book &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto&lt;/a&gt; (complete draft, November 5, 2017). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On page 271, the pseudo-code for the episodic Monte-Carlo Policy-Gradient Method is presented. Looking at this pseudo-code I can't understand why it seems that the discount rate appears 2 times, once in the update state and a second time inside the return. [See the figure below] &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dxDnP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dxDnP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the return for the steps after step 1 are just a truncation of the return of the first step. Also, if you look just one page above in the book you find an equation with just 1 discount rate (the one inside the return.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why then does the pseudo-code seem to be different? My guess is that I am misunderstanding something: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{\mathbf{\theta}}_{t+1} ~\dot{=}~\mathbf{\theta}_t + \alpha G_t \frac{{\nabla}_{\mathbf{\theta}} \pi \left(A_t \middle| S_t, \mathbf{\theta}_{t} \right)}{\pi \left(A_t \middle| S_t, \mathbf{\theta}_{t} \right)}.&#xA;\tag{13.6}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="17565" LastEditorUserId="2444" LastEditDate="2019-06-29T11:59:59.740" LastActivityDate="2021-04-06T13:46:53.037" Title="Why does the discount rate in the REINFORCE algorithm appear twice?" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;sutton-barto&gt;&lt;reinforce&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7681" PostTypeId="2" ParentId="7680" CreationDate="2018-08-22T18:58:48.497" Score="8" Body="&lt;p&gt;The discount factor does appear twice, and this is correct.&lt;/p&gt;&#xA;&lt;p&gt;This is because the function you are trying to maximise in REINFORCE for an episodic problem (by taking the gradient) is the expected return from a given (distribution of) start state:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\theta) = \mathbb{E}_{\pi(\theta)}[G_t|S_t = s_0, t=0]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Therefore, during the episode, when you sample the returns &lt;span class=&quot;math-container&quot;&gt;$G_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$G_2$&lt;/span&gt; etc, these will be less relevant to the problem you are solving, reduced by the discount factor a second time as you noted. At the extreme with an episodic problem and &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt; then REINFORCE will only find an optimal policy for the first action.&lt;/p&gt;&#xA;&lt;p&gt;In continuing problems, you would use different formulations for &lt;span class=&quot;math-container&quot;&gt;$J(\theta)$&lt;/span&gt;, and these do not lead to the extra factor of &lt;span class=&quot;math-container&quot;&gt;$\gamma^t$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-04-06T13:46:53.037" LastActivityDate="2021-04-06T13:46:53.037" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7682" PostTypeId="2" ParentId="7680" CreationDate="2018-08-22T19:11:14.863" Score="6" Body="&lt;p&gt;Neil's answer already provides some intuition as to why the pseudocode (with the extra $\gamma^t$ term) is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd just like to additionally clarify that you do not seem to be misunderstanding anything, &lt;strong&gt;Equation (13.6) in the book is indeed different from the pseudocode&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I don't have the edition of the book that you mentioned right here, but I do have a later draft from March 22, 2018, and the text on this particular topic seems to be similar. In this edition:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Near the end of page 326, it is explicitly mentioned that they'll assume $\gamma = 1$ in their proof for the Policy Gradient Theorem.&lt;/li&gt;&#xA;&lt;li&gt;That proof eventually leads to the same Equation (13.6) on page 329.&lt;/li&gt;&#xA;&lt;li&gt;Immediately below the pseudocode, on page 330, they actually briefly address the difference between the Equation and the pseudocode, saying that that difference is due to the assumption of $\gamma = 1$ in the proof.&lt;/li&gt;&#xA;&lt;li&gt;Right below that, in &lt;em&gt;Exercise 13.2&lt;/em&gt;, they give some hints as for what you should be looking at if you'd like to derive the modified proof for the case where $\gamma &amp;lt; 1$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-22T19:11:14.863" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7683" PostTypeId="1" AcceptedAnswerId="7686" CreationDate="2018-08-23T03:48:44.003" Score="7" ViewCount="1488" Body="&lt;p&gt;I am reading &lt;a href=&quot;http://www.deeplearningbook.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goodfellow et al Deeplearning Book&lt;/a&gt;. I found it difficult to understand the difference between the definition of the hypothesis space and representation capacity of a model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://www.deeplearningbook.org/contents/ml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 5&lt;/a&gt;, it is written about hypothesis space:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One way to control the capacity of a learning algorithm is by choosing its hypothesis space, the set of functions that the learning algorithm is allowed to select as being the solution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And about representational capacity:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The model speciﬁes which family of functions the learning algorithm can choose from when varying the parameters in order to reduce a training objective. This is called the representational capacity of the model.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If we take the linear regression model as an example and allow our output &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to takes polynomial inputs, I understand the hypothesis space as the ensemble of quadratic functions taking input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, i.e &lt;span class=&quot;math-container&quot;&gt;$y = a_0 + a_1x + a_2x^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is it different from the definition of the representational capacity, where parameters are &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="17664" LastEditorUserId="2444" LastEditDate="2021-01-22T15:59:08.287" LastActivityDate="2021-01-22T15:59:08.287" Title="What is the difference between hypothesis space and representational capacity?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;computational-learning-theory&gt;&lt;hypothesis-class&gt;&lt;capacity&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7684" PostTypeId="1" AcceptedAnswerId="7754" CreationDate="2018-08-23T07:17:27.470" Score="4" ViewCount="1447" Body="&lt;p&gt;Where can I find (more) pre-trained &lt;a href=&quot;https://en.wikipedia.org/wiki/Language_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;language models&lt;/a&gt;? I am especially interested in &lt;strong&gt;neural network-based&lt;/strong&gt; models for &lt;strong&gt;English and German&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware only of &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research/lm_1b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Language Model on One Billion Word Benchmark&lt;/a&gt; and &lt;a href=&quot;https://github.com/lverwimp/tf-lm&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-LM: TensorFlow-based Language Modeling Toolkit&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am surprised not to find a greater wealth of models for different frameworks and languages.&lt;/p&gt;&#xA;" OwnerUserId="17670" LastEditorUserId="2444" LastEditDate="2019-11-01T03:18:15.360" LastActivityDate="2019-11-01T03:18:15.360" Title="Where can I find pre-trained language models in English and German?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;bert&gt;&lt;gpt&gt;&lt;language-model&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7685" PostTypeId="1" CreationDate="2018-08-23T07:17:42.697" Score="6" ViewCount="863" Body="&lt;p&gt;In the Trust-Region Policy Optimisation (TRPO) algorithm (and subsequently in PPO also), I do not understand the motivation behind replacing the log probability term from standard policy gradients&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L^{PG}(\theta) = \hat{\mathbb{E}}_t[\log \pi_{\theta}(a_t | s_t)\hat{A}_t],$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with the importance sampling term of the policy output probability over the old policy output probability&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L^{IS}_{\theta_{old}}(\theta) = \hat{\mathbb{E}}_t \left[\frac{\pi_{\theta}(a_t | s_t)}{\pi_{\theta_{old}}(a_t | s_t)}\hat{A}_t \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Could someone please explain this step to me?&lt;/p&gt;&#xA;&lt;p&gt;I understand once we have done this why we then need to constrain the updates within a 'trust region' (to avoid the &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta_{old}}$&lt;/span&gt; increasing the gradient updates outwith the bounds in which the approximations of the gradient direction are accurate). I'm just not sure of the reasons behind including this term in the first place.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="2444" LastEditDate="2020-11-05T22:22:50.323" LastActivityDate="2020-11-05T22:22:50.323" Title="Why is the log probability replaced with the importance sampling in the loss function?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;proximal-policy-optimization&gt;&lt;importance-sampling&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7686" PostTypeId="2" ParentId="7683" CreationDate="2018-08-23T07:32:50.863" Score="1" Body="&lt;p&gt;Consider a target function &lt;span class=&quot;math-container&quot;&gt;$f: x \mapsto f(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A hypothesis refers to an approximation of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. A &lt;strong&gt;hypothesis space&lt;/strong&gt; refers to the set of possible approximations that an algorithm can create for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The hypothesis space consists of the set of functions the model is limited to learn. For instance, linear regression can be limited to linear functions as its hypothesis space, or it can be expanded to learn polynomials.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;representational capacity&lt;/strong&gt; of a model determines the flexibility of it, its ability to fit a variety of functions (i.e. which functions the model is able to learn), at the same. It specifies the family of functions the learning algorithm can choose from.&lt;/p&gt;&#xA;" OwnerUserId="8041" LastEditorUserId="2444" LastEditDate="2020-08-12T23:22:55.973" LastActivityDate="2020-08-12T23:22:55.973" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7687" PostTypeId="2" ParentId="7672" CreationDate="2018-08-23T07:43:26.413" Score="0" Body="&lt;p&gt;I can think of 4 options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;One option is to divide the data so that each data point only has one picture but we have multiple data points per one real estate with structured data duplicated. Then calculate the forecasts and average the resulting predicted price between data points belonging to one real estate. Here we probably assume that the quality of pictures is more important than what is being shot. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The other option is to divide the pictures to categories: bedroom1, bathroom, bedroom2, kitchen and so on. Then for the missing pictures use a black square. This way you will be able to have multiple pictures in one data point.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The third option would be to have places for the maximum amount of pictures and fill only the first pictures that are available. And the other pictures fill with black squares/Nans.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The best option would probably be combining option number two and three. So you have categories and maximum number of pictures per category. So you will have four categories for bathroom1 if the maximum is 4 pictures of the first bathroom:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bathroom1_1, Bathroom1_2, Bathroom1_3, Bathroom1_4, Bedroom1_1, Bedroom1_2, Bedroom1_3, Bedroom2_1, Bedroom2_2 ... and so on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The gradient boosting algorithms are very good at working with missing values. So probably lightgbm and xgboost as last layers could give good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the end you can check all the variants and even do ensembles from the variants that give the best results.&lt;/p&gt;&#xA;" OwnerUserId="13178" LastActivityDate="2018-08-23T07:43:26.413" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7689" PostTypeId="2" ParentId="7667" CreationDate="2018-08-23T10:38:05.843" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p9daq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p9daq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivative of the sigmoid curve is 0 when the output is 0 or 1 as you can see from the image above. The technique you are referring to is called label-smoothing which is used in various applications (e.g. GANs) but I can see how it would be applicable here also, by helping to avoid 0-gradients saturating learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your third question, sigmoids are rarely used in the intermediate layers of networks these days. They have been replaced by ReLUs (or variants of ReLU) for this exact reason - sigmoids are prone to causing 'vanishing gradients' where the gradient becomes 0 and therefore does not get backpropagated any further. ReLUs alleviate this problem by always providing a gradient of 1 for positive input values.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2018-08-23T10:38:05.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7690" PostTypeId="1" CreationDate="2018-08-23T11:01:06.643" Score="5" ViewCount="220" Body="&lt;p&gt;In the context of autonomous driving, two main stages are typically implemented: an image processing stage and a control stage. The first aims at extracting useful information from the acquired image while the second employs those information to control the vehicle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as concerning the processing stage, semantic segmentation is typically used. The input image is divided in different areas with a specific meaning (road, sky, car etc...). Here is an example of semantic segmentation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9skuH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9skuH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of the segmentation stage is very complex. I am trying to understand how this information is typically used in the control stage, and how to use the information on the segmented areas to control the vehicle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's just consider a vehicle that has to follow a path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR: &lt;strong&gt;what are the typical control algorithms for autonomous driving based on semantic segmentation?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="16671" LastEditorUserId="16671" LastEditDate="2019-02-18T08:26:08.813" LastActivityDate="2019-06-14T20:03:51.563" Title="Self-driving control logic based on semantic segmentation" Tags="&lt;research&gt;&lt;reference-request&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7692" PostTypeId="2" ParentId="7618" CreationDate="2018-08-23T20:22:53.913" Score="0" Body="&lt;p&gt;&lt;em&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/7626/1671&quot;&gt;Thanks to Dennis Soemers for helping to explain and unpack.&lt;/a&gt; I'm still in the early stages of approaching the function, so any thoughts on my current line of thinking would be appreciated.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;-----------------------------------------&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are two conditions that need to be separated: number of branches leading to an a given node (raw probability) and  number of branches leading from given node (outdegree, which I like to think of as &quot;chaotictivity&quot;;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Raw probability here is the indegree of all nodes leading to the evaluated node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aggregate indegree seems for appropriate for a basic time discounting function because it reflects the unqualified probability of any given future state or position.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thought is that I can use it to &quot;normalize&quot; the reward distributions. An expressed position $p_{x_1}$ yields an aggregate of $30R$ over 20 potential positions $p$&lt;strong&gt;;&lt;/strong&gt;    $p_{x_2}$ yields an aggregate of $15R$ over 10 $p$:  $$\frac{30}{20} = \frac{15}{10}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;squeezes&quot; the R sum for a given ply into a single number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fractions below represent: $\frac{choice}{choices}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left \{\frac{1}{4} \right \}$$&#xA;$$\left \{ \frac{1}{2} | \frac{1}{2} | \frac{1}{2} |\frac{1}{2} \right \}$$&#xA;$$\left \{  \frac{1}{1} | \frac{1}{1} \right \} | \left \{ \frac{1}{1} | \frac{1}{1} \right \}    | \left \{ \frac{1}{1} | \frac{1}{1} \right \}  | \left \{ \frac{1}{1} | \frac{1}{1}  \right \}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $\beta$ for any given node on the final ply would be: $$\frac{1}{4} * \frac{1}{2} = \frac{1}{8} = .125$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fractions are attractive because integers may be utilized until the $\beta$ needs to be expressed to whatever digit, and the distinct cardinalities of the factors is maintained for ancillary evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function should allow the automata to do things like make the choice with the greatest certainty of the least maximal potential downside. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If greater uncertainty is desirable, the automata can pursue more chaotic nodes, where aggregate least maximal downside is equivalent.  In that &quot;personalities&quot; can be desirable in automata that play games with humans, there might be a maximax &quot;gambler&quot; persona that seeks chaos;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function can also be used to evaluate individual target nodes, and modified by any number of additional functions for &quot;tuning&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking about how I might use the degree of variance between the most probable and least probable node for a given ply. If the least probable node is $\frac{1}{20}$, and most probable node for a given ply is $\frac{1}{5} = \frac{4}{20}$, the variance is  $\frac{3}{20} = .15$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is initially designed for partisan sudoku.  I'm thinking it is worthwhile to utilize the native $R$, which is an integer value between $\{-m^n, .., 0, .., m^n\}$  for an $(m^n)^n$ sudoku grid, as it segregates gains and losses.  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;-------------------------------&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;EXPERIMENT DESIGN&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Partisan sudoku is an attractive model for gauging relative strength of automata because outcomes are a set of ratios.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 2-player game where the number of regions in the sudoku is even, the game is perfectly symmetrical and neither player has a material, positional, or turn number advantage.  Because the game is intractable on higher order grids, perfectly symmetrical games will not necessarily result in perfect ties because optimality of a given position is only presumed.  (The second player can mirror the starting player for a perfect tie, where all outcome ratios are equal, or employ symmetry breaking if it perceives an opponent's position as less optimal than alternate positions.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2-player games with an odd number of regions, matched sets of games may be employed with the automata alternating as starting player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial plan is to evaluate individual heuristics, where one agent employs the branch discount $\beta$, and one agent does not.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-09-07T20:43:57.300" LastActivityDate="2018-09-07T20:43:57.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7693" PostTypeId="1" CreationDate="2018-08-23T20:42:02.030" Score="2" ViewCount="143" Body="&lt;p&gt;A linear activation function (or none at all) should only be used when the relation between input and output is linear. Why doesn't the same rule apply for other activation functions? For example, why doesn't sigmoid only work when the relation between input and output is &quot;of sigmoid shape&quot;?&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2018-11-12T20:20:56.047" LastActivityDate="2019-01-07T20:14:20.573" Title="Why do non-linear activation functions not require a specific non-linear relation between its inputs and outputs?" Tags="&lt;activation-function&gt;&lt;sigmoid&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7695" PostTypeId="1" CreationDate="2018-08-24T04:06:43.533" Score="3" ViewCount="509" Body="&lt;p&gt;Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision application and are used widely in the industry, for example for internet search engines and autonomous cars. In real life problems, the image contains of regions with different objects. It is not enough to only identify the picture but elements of the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A while ago an alternative to the well known sliding window algorithm was described in the literature, called Region Proposal Networks. It is basically a convolution neural network which was extended by a region vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem that I am trying to solve:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a given video frame, I want to pick some region of interests (literally), and perform classification on those regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How is it currently implemented&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Capture the video frame&lt;/li&gt;&#xA;&lt;li&gt;Split the video frame into multiple images each representing a region of interest&lt;/li&gt;&#xA;&lt;li&gt;Perform image classification(inference) on each of the image (corresponding to a part of the frame)&lt;/li&gt;&#xA;&lt;li&gt;Aggregate the results of #3 &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem with the current approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiple inferences per frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a solution where I specify the locations of interest in a frame, and inference task, be it object detection (or) image classification, is performed only on those regions.Can you please point to me the references which I need to study (or) use to do this.&lt;/p&gt;&#xA;" OwnerUserId="17688" LastEditorDisplayName="user11571" LastEditDate="2018-08-24T08:25:55.690" LastActivityDate="2021-04-11T13:07:50.727" Title="Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame)" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7696" PostTypeId="1" CreationDate="2018-08-24T08:16:56.343" Score="6" ViewCount="142" Body="&lt;p&gt;So I wrote simple feed forward neural network that plays tic-tac-toe:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;9 neurons in input layers: 1 - my sign, -1 - opponent's sign, 0 - empty;&lt;/li&gt;&#xA;&lt;li&gt;9 neurons in hidden layer: value calculated using Relu;&lt;/li&gt;&#xA;&lt;li&gt;9 neurons in output layer: value calculated using softmax;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am using evolutionary approach: 100 individuals play against each other (all-play-all). Top 10 best are selected to mutate and reproduce into the next generation. The fitness score calculated: +1 for correct move (it's possible to place your sing on already occupied tile), +9 for victory, -9 for a defeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I notice is that the network's fitness keeps climbing up and falling down again. It seems that my current approach only evolves certain patterns on placing signs on the board and once random mutation interrupts current pattern new one emerges. My network goes in circles without ever evolving actual strategy. I suspect solution for this would be to pit network against tic-tac-toe AI, but is there any way to evolve actual strategy just by making it to playing against itself?&lt;/p&gt;&#xA;" OwnerUserId="17693" LastEditorUserId="1671" LastEditDate="2018-08-24T19:55:37.763" LastActivityDate="2019-09-12T14:02:00.273" Title="Evolving network in game" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;game-ai&gt;&lt;evolutionary-algorithms&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7698" PostTypeId="2" ParentId="7685" CreationDate="2018-08-24T10:08:59.033" Score="6" Body="&lt;p&gt;I am not 100% sure if the following is the only/complete story, but I'm quite confident it's at least part of the story:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO paper&lt;/a&gt;, after describing the standard policy gradient objective &lt;span class=&quot;math-container&quot;&gt;$L^{PG}$&lt;/span&gt;, they mention the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;While it is appealing to perform multiple steps of optimization on this loss &lt;span class=&quot;math-container&quot;&gt;$L^{PG}$&lt;/span&gt; using the same trajectory, doing so is not well-justified, and empirically it often leads to destructively large policy updates&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is because, as soon as you've performed one update using a trajectory generated with the previous policy, you land in an &lt;strong&gt;off-policy&lt;/strong&gt; situation; the experience gained in that trajectory is no longer representative of your current policy, and all the estimators (like the advantage estimator) technically become incorrect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With importance sampling, you can correct for this. This is also commonly used in multi-step off-policy value learning algorithms. Intuitively, the importance sampling term emphasizes estimates of advantage &lt;span class=&quot;math-container&quot;&gt;$\hat{A}_t$&lt;/span&gt; corresponding to actions &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; that have become &lt;strong&gt;more likely&lt;/strong&gt; in the new policy relative to the old policy, and it de-emphasizes advantages corresponding to actions that have already become &lt;strong&gt;less likely&lt;/strong&gt; in the new policy relative to the old policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; in the old trajectory has already become highly unlikely since that trajectory of experience was generated, we have &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta} (a_t \vert s_t) &amp;lt; \pi_{\theta_{\text{old}}} (a_t \vert s_t)$&lt;/span&gt;, which means that &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi_{\theta} (a_t \vert s_t)}{\pi_{\theta_{\text{old}}} (a_t \vert s_t)}$&lt;/span&gt; becomes close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, which means that we'll reduce the influence of that particular chunk of experience on our subsequent updates. This makes sense because, due to previous updates since the generation of that trajectory, that particular part of the trajectory has already become highly unlikely anyway, and should therefore no longer be relevant for our updates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ability to perform multiple updates using the same (old) trajectory anyway is useful because this increases sample-efficiency, we can re-use the same samples of experience more than once rather than using them once and then discarding them again.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-05-02T18:48:16.067" LastActivityDate="2019-05-02T18:48:16.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7700" PostTypeId="2" ParentId="3758" CreationDate="2018-08-24T15:02:14.150" Score="4" Body="&lt;p&gt;The value $Q(s', ~\cdot~)$ should always be implemented to simply be equal to $0$ for any terminal state $s'$ (the dot instead of an action as second argument there indicates that what I just wrote should hold for &lt;strong&gt;any action&lt;/strong&gt;, as long as $s'$ is terminal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easier to understand why this should be the case by dissecting what the different terms in the update rule mean:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q(s, a) \gets \color{red}{Q(s, a)} + \alpha \left[ \color{blue}{r + \gamma Q(s', a')} - \color{red}{Q(s, a)} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this update, the red term $\color{red}{Q(s, a)}$ (which appears twice) is our old estimate of the value $Q(s, a)$ of being in state $s$ and executing action $a$. The blue term $\color{blue}{r + \gamma Q(s', a')}$ is a different version of estimating the same quantity $Q(s, a)$. This second version is assumed to be slightly more accurate, because it is not &quot;just&quot; a prediction, but it's a combination of: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;something that we really observed: $r$, plus &lt;/li&gt;&#xA;&lt;li&gt;a prediction: $\gamma Q(s', a')$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here, the $r$ component is the immediate reward that we observed after executing $a$ in $s$, and then $Q(s', a')$ is everything we expect to still be collecting afterwards (i.e., after executing $a$ in $s$ and transitioning to $s'$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose that $s'$ is a terminal state, what rewards do we still expect to be collecting in the future within that same episode? Since $s'$ is terminal, and the episode has ended, there can only be one correct answer; we expect to collect exactly $0$ rewards in the future.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-08-24T15:55:00.353" LastActivityDate="2018-08-24T15:55:00.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7701" PostTypeId="1" AcceptedAnswerId="7702" CreationDate="2018-08-24T17:34:05.080" Score="9" ViewCount="2461" Body="&lt;p&gt;For example, AFAIK pooling layer in CNN is not differentiable, but it can be used because it's not learning. Is it always true? &lt;/p&gt;&#xA;" OwnerUserId="17358" LastActivityDate="2019-01-08T20:12:18.537" Title="Can non-differentiable layer be used in a neural network, if it's not learned?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7702" PostTypeId="2" ParentId="7701" CreationDate="2018-08-24T18:58:15.103" Score="7" Body="&lt;p&gt;It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function &lt;em&gt;is&lt;/em&gt; differentiable*, and usually trivially so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If an average pooling layer has inputs &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and outputs &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, and each output is average of 4 inputs then &lt;span class=&quot;math-container&quot;&gt;$\frac{da}{dz} = 0.25$&lt;/span&gt; (if pooling layers overlap it gets a little more complicated, but you just add things up where they overlap). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A max pooling layer has &lt;span class=&quot;math-container&quot;&gt;$\frac{da}{dz} = 1$&lt;/span&gt; for the maximum z, and &lt;span class=&quot;math-container&quot;&gt;$\frac{da}{dz} = 0$&lt;/span&gt; for all others.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A pooling layer usually has no learnable parameters, but if you know the gradient of a function at its outputs, you can assign gradient correctly to its inputs using the chain rule. That is essentially all that back propagation is, the chain rule applied to the functions of a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question more directly:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can non-differentiable layer be used in a neural network, if it's not learned?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one exception: If this layer appears directly after the input, then as it has no parameters to learn, and you generally do not care about the gradient of the input data, so you can have a non-differentiable function there. However, this is just the same as transforming your input data in some non-differentiable way, and training the NN with that transformed data instead.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Technically there are some discontinuities in the gradient of a max function (where any two inputs are equal). However, this is not a problem in practice, as the gradients are well behaved close to these values. When you can safely do this or not is probably the topic of another question.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-08T20:12:18.537" LastActivityDate="2019-01-08T20:12:18.537" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7703" PostTypeId="1" CreationDate="2018-08-24T19:05:08.083" Score="3" ViewCount="72" Body="&lt;p&gt;Sutton and Barto 2018 define the discounted return &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; the following way (&lt;a href=&quot;https://drive.google.com/file/d/1xeUDVGWGUUv1-ccUMAZHJLej2C7aAFWY/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;p 55&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ilEt7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ilEt7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my interpretation correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yfxBu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yfxBu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or should all &quot;1&quot; be in the same column?&lt;/p&gt;&#xA;" OwnerUserId="17703" LastEditorUserId="2444" LastEditDate="2019-02-16T02:47:49.060" LastActivityDate="2019-02-16T02:47:49.060" Title="Is my interpretation of the return correct?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;sutton-barto&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7704" PostTypeId="2" ParentId="7703" CreationDate="2018-08-24T19:11:08.903" Score="3" Body="&lt;p&gt;Your table is almost correct. It is a minor difference, you should not have a $R_0$, the top row, leftmost column of numbers should be empty. That is because the first reward is $R_1$ (a result of taking action $A_0$ in state $S_0$). The alignment of the columns on the right hand side &lt;em&gt;is&lt;/em&gt; correct though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might help to add the time step number at the top. But the important detail is that $G_t$ is a measure of all &lt;em&gt;future&lt;/em&gt; rewards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance it should always be zero when you reach a terminal state, which is what your example shows. Whilst it is quite common to receive a reward at the end of an episode (i.e. whilst arriving at a terminal state), also as your example shows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The decision to have reward time step match that of &lt;em&gt;next state&lt;/em&gt; is a convention that can be altered. A few RL sources, but not Sutton &amp;amp; Barto, will have reward on same time step as state and action that decided it, and thus $R_0$ will exist. The reward of 1 for reaching the terminal state would then be 1 time step earlier in your table, and there would be no $R_4$. The definition of $G_t$ would need to change to match ($G_t = R_t + \gamma G_{t+1})$, as well as other equations. That would change your table also - the reward sequence (top row) would shift to the left.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-08-25T07:02:20.637" LastActivityDate="2018-08-25T07:02:20.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7705" PostTypeId="1" AcceptedAnswerId="7706" CreationDate="2018-08-24T20:21:50.723" Score="3" ViewCount="109" Body="&lt;p&gt;I am confused about the definition of the optimal value (&lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt;) and optimal action-value (Q*) in reinforcement learning, so I need some clarification, because some blogs I read on Medium and GitHub are inconsistent with the literature.&lt;/p&gt;&#xA;&lt;p&gt;Originally, I thought the optimal action value, &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;, represents you performing the action that maximizes your current reward, and then acting optimally thereafter.&lt;/p&gt;&#xA;&lt;p&gt;And the optimal value, &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt;, being the average &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values in that state.  Meaning that if you're in this state, the average &amp;quot;goodness&amp;quot; is this.&lt;/p&gt;&#xA;&lt;p&gt;For example:&#xA;If I am in a toy store and I can buy a pencil, yo-yo, or Lego.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Q(toy store, pencil) = -10&#xA;Q(toy store, yo-yo) = 5&#xA;Q(toy store, Lego) = 50&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And therefore my &lt;span class=&quot;math-container&quot;&gt;$Q^* = 50$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But my &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; in this case is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;V* = -10 + 5 + 50 / 3 = 15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Representing no matter what action I take, the average future projected reward is &lt;span class=&quot;math-container&quot;&gt;$15$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And for the advantage of learning, my baseline would be &lt;span class=&quot;math-container&quot;&gt;$15$&lt;/span&gt;.  So anything less than &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; is worse than average and anything above &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; is better than average.&lt;/p&gt;&#xA;&lt;p&gt;However, now I am reading about how &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; actually assumes the optimal action in a given state, meaning &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; would be 50 in the above case.&lt;/p&gt;&#xA;&lt;p&gt;I am wondering which definition is correct.&lt;/p&gt;&#xA;" OwnerUserId="17706" LastEditorUserId="2444" LastEditDate="2020-11-01T12:17:42.440" LastActivityDate="2020-11-01T12:20:54.533" Title="In reinforcement learning, does the optimal value correspond to performing the best action in a given state?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7706" PostTypeId="2" ParentId="7705" CreationDate="2018-08-24T20:39:00.383" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I am wondering which definition is correct.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The asterisk * in both the definitions stands for &amp;quot;optimal&amp;quot; in the sense of &amp;quot;value when following the optimal policy&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;So this one is correct:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; actually assumes the optimal action in a given state, meaning &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; would be &lt;span class=&quot;math-container&quot;&gt;$50$&lt;/span&gt; in the above case&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, you have got the definition of Q slightly wrong.&lt;/p&gt;&#xA;&lt;p&gt;I think this is because you are omitting the parameters.&lt;/p&gt;&#xA;&lt;p&gt;The state value function uses the state as a parameter, &lt;span class=&quot;math-container&quot;&gt;$V_{\pi}(s)$&lt;/span&gt;, it returns the value of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and following a fixed policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. The * is used to denote following an optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;The action value function has two parameters - a state and an action that is possible in that state, &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(s, a)$&lt;/span&gt;, it returns the value of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (regardless of whether it is the best action or not) and following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; &lt;em&gt;after&lt;/em&gt; that point.&lt;/p&gt;&#xA;&lt;p&gt;Your assertion in the question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And therefore my &lt;span class=&quot;math-container&quot;&gt;$Q^* = 50$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;is wrong, or rather not meaningful, as you have not stated the parameters. You already list all the possible values of Q with the parameters. You &lt;em&gt;could&lt;/em&gt; say &lt;span class=&quot;math-container&quot;&gt;$\text{max}_a Q(\text{toy store}, a) = 50$&lt;/span&gt;, or to choose the best action &lt;span class=&quot;math-container&quot;&gt;$\pi(\text{toy store}) = \text{argmax}_a Q(\text{toy store}, a) &#xA;= \text{Lego}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-11-01T12:20:54.533" LastActivityDate="2020-11-01T12:20:54.533" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7707" PostTypeId="1" CreationDate="2018-08-24T21:11:06.363" Score="7" ViewCount="801" Body="&lt;p&gt;I've built a deep deterministic policy gradient reinforcement learning agent to be able to handle any games/tasks that have only one action.  However, the agent seems to fail horribly when there are two or more actions.  I tried to look online for any examples of somebody implementing DDPG on a multiple-action system, but people mostly applied it to the pendulum problem, which is a single-action problem.&lt;/p&gt;&#xA;&lt;p&gt;For my current system, it is a 3 state, 2 continuous control actions system (One is to adjust the temperature of the system, the other one adjusts a mechanical position, both are continuous).  However, I froze the second continuous action to be the optimal action all the time.  So RL only has to manipulate one action.  It solves within 30 episodes.  However, the moment I allow the RL to try both continuous actions, it doesn't even converge after 1000 episodes.  In fact, it diverges aggressively.  The output of the actor-network seems to always be the max action, possibly because I am using a &lt;code&gt;tanh&lt;/code&gt; activation for the actor to provide output constraint.  I added a penalty to large actions, but it does not seem to work for the 2 continuous control action case.&lt;/p&gt;&#xA;&lt;p&gt;For my exploratory noise, I used Ornstein-Ulhenbeck noise, with means adjusted for the two different continuous actions.  The mean of the noise is 10% of the mean of the action.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is there any massive difference between single action and multiple action DDPG?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I changed the reward function to take into account both actions, have tried making a bigger network, tried priority replay, etc., but it appears I am missing something.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Does anyone here have any experience building a multiple-action DDPG and could give me some pointers?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="17706" LastEditorUserId="2444" LastEditDate="2021-03-28T01:32:05.913" LastActivityDate="2021-03-28T01:32:05.913" Title="Is there a difference in the architecture of deep reinforcement learning when multiple actions are performed instead of a single action?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;ddpg&gt;&lt;action-spaces&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7708" PostTypeId="2" ParentId="7518" CreationDate="2018-08-25T00:37:53.073" Score="2" Body="&lt;p&gt;I'm going to start by trying to restate your problem as I understand it.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You have a game which contains weapons.&lt;/li&gt;&#xA;&lt;li&gt;Weapons are characterized by 5 different numbers, which can range over different values (1-5 in your examples?).&lt;/li&gt;&#xA;&lt;li&gt;You have a way to simulate combat involving the two weapons. &lt;/li&gt;&#xA;&lt;li&gt;The combat is random, but can be repeated many times. An average win rate can be determined.&lt;/li&gt;&#xA;&lt;li&gt;You are looking for an AI algorithm that would take in a lot of pairs of statistics, along with the average win rates for one over the other, and give you insight into how to make the average win rate as close to 50% as possible.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If this sounds right, then fundamentally your problem is a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;regression&lt;/a&gt;, which something you &lt;em&gt;could&lt;/em&gt; use AI for, but probably don't need to. However, your problem is probably not linear, so you need the interactions between the features. Here's what I suggest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each pair of weapons, store a comma separated list consisting of the stats for each weapon (one by one), followed by wins1 - wins2. At the top, list out the names of each attribute, separated by commas, (e.g. weapon1Str, weapon1Range, ... ,weapon1-weapon2 Then use a &lt;a href=&quot;https://www.r-project.org/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;language like R&lt;/a&gt; that has simple support for complex forms of regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R, this is then as simple as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- read.csv(file=&quot;Myfile.csv&quot;)&#xA;lm(formula = dist ~ .*., data = data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should produce a list of &quot;coefficients&quot;, one for each of the attributes, and one for the interaction between each pair of attributes, which form a lengthy quadratic equation in 10 variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any zero of that equation should be a pair of weapons that minimizes this difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's probably the place to start. If it doesn't work out, maybe come post a different question and we can help more.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-25T00:37:53.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7709" PostTypeId="2" ParentId="7624" CreationDate="2018-08-25T04:00:55.477" Score="2" Body="&lt;p&gt;This is an important question for AI – maybe the most important of all –  for the research field of Artificial &lt;em&gt;Intelligence&lt;/em&gt;. I mean if AI is science, then its experiments will be empirically testable. There has to be a way to decide pass or fail. So what are the tests for intelligence? Before you even design a test, you need a clear idea of what intelligence amounts to, otherwise how could you design a competent test for it?&lt;/p&gt;&#xA;&lt;p&gt;Sure, I'm part of the research and development project known as Building Watertight Submarines, and sure, I'm totally confident my submarine is watertight, but I have no idea how to test whether it is or not because I don't know what &amp;quot;watertight&amp;quot; means.&#xA;This whole idea is absurd. But ask AI what &amp;quot;intelligence&amp;quot; means. The answers you get, on analysis, are almost the same as the submarine example.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Base Answer - Behavior&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The word (idea, concept) &amp;quot;Intelligence&amp;quot; is usually defined by AI in terms of behavior. I.e. the Turing test approach. A machine is intelligent if it behaves in a way that, were a human to behave in that same way, the human would be said to be performing an action that required human intelligence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Problem 1&lt;/em&gt;: player pianos are intelligent. Playing a Scott Joplin tune obviously requires intelligence in a human.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Problem 2&lt;/em&gt;. If a machine passes the test, it only shows that the machine is &amp;quot;intelligent&amp;quot; for the tested behaviors. What about untested behaviors?  This is actually a life-and-death problem today with self-driving vehicle AI control systems. The AI systems are acceptably good at driving a car (which obviously requires human intelligence) in specific environments, e.g. freeways with well-marked lanes, no tight corners, and a median barrier separating the two directions. But the systems go disastrously wrong in &amp;quot;edge cases&amp;quot; – unusual situations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Problem 3&lt;/em&gt;. Who would put their child on a school bus driven by a robot that had passed the Turing test for driving school buses? What about a storm when a live power line falls across the road? Or a twister in the distance is coming this way? What about a thousand other untested possibilities? A responsible parent would want to know (a) what are the principles of the internal processes and structures of human intelligence, and (b) that the digital bus driver had adequately similar internal processes and structures – i.e., not behavior but the right &lt;em&gt;inner&lt;/em&gt; elements, the right inner causation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Desired answer – inner principles&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would want to know that the machine was running the right inner processes and that it was running these processes (algorithms) on the right inner (memory) structures. Problem is, no one seems to know what the right inner processes and structures of human intelligence are. (A huge problem to be sure – but one that hasn't held AI back – or self-driving system developers - one bit.) The implication of this is that what AI ought to be doing now is working out what are the inner processes and structures of human intelligence. But it's not doing this – rather, it's commercializing its flawed technology.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Elements of a definition – 1. Generalization&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We do know some things about human intelligence. Some tests really do test whether a machine has certain properties of the human mind. One of these properties is generalization. In &lt;a href=&quot;https://academic.oup.com/mind/article/LIX/236/433/986238#164226550&quot; rel=&quot;nofollow noreferrer&quot;&gt;his 1950 paper&lt;/a&gt;, Turing, as a sort of joke, gave a really good example of conversational generalization: (The witness is the machine.)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Interrogator&lt;/strong&gt;: In the first line of your sonnet which reads 'Shall I&#xA;compare thee to a summer's day', would not 'a spring day' do as&#xA;well or better?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Witness&lt;/strong&gt;: It wouldn't scan.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Interrogator&lt;/strong&gt;: How about 'a winter's day' That would scan all right.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Witness&lt;/strong&gt;: Yes, but nobody wants to be compared to a winter's day.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Interrogator&lt;/strong&gt;: Would you say Mr. Pickwick reminded you of Christmas?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Witness&lt;/strong&gt;: In a way.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Interrogator&lt;/strong&gt;: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Witness&lt;/strong&gt;: I don't think you're serious. By a winter's flay one means a typical winter's day, rather than a special one like Christmas.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Current AI has nothing that comes even remotely near being able to generalize like this. Failure to generalize is regarded as perhaps the greatest failing of current AI. The ability to generalize would be one part of an adequate definition of &amp;quot;intelligence&amp;quot;. But what generalization amounts to would need to be explicated.&lt;/p&gt;&#xA;&lt;p&gt;The problem of generalization, also, is behind several the severe philosophical objections to AI theory, including the &lt;a href=&quot;https://plato.stanford.edu/entries/frame-problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;frame problem&lt;/a&gt;, the &lt;a href=&quot;https://plato.stanford.edu/entries/logic-ai/#cs&quot; rel=&quot;nofollow noreferrer&quot;&gt;problem of common-sense knowledge&lt;/a&gt;, and the problem of combinatorial explosion.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Elements of a definition – 2. Perception&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sensory perception is fairly obviously fundamental to human learning and intelligence. Data (in some form) is emitted by the human senses then processed by the central system. In the computer, binary values exit the digital sensor and travel to the machine. However, nothing in the values themselves indicates what was sensed. Yet the only thing the computer gets is the binary values. How could the machine ever come to know what is sensed?  (The &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;classic Chinese room argument&lt;/a&gt; problem.)&lt;/p&gt;&#xA;&lt;p&gt;So another element of human-like intelligence is the ability to perceive in a human-like way. What &amp;quot;human-like way&amp;quot; means here is that the machine processes sensory input using the same principles that apply in human perception. The problem is that no one seems to know how a semantics (knowledge) can be built from the data emitted by digital sensors (or organic senses). But still, human-like perception needs to be an element of an adequate definition of &amp;quot;intelligence&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Once AI gets these two issues sorted out – generalization and perception – then it will probably, &lt;em&gt;hopefully&lt;/em&gt;, be well on the way to realizing its original goal of almost 70 years past – building a machine with (or that could acquire) a human-like general intelligence. And maybe the principles of generalization and the principles of perception are one and the same. And maybe there is actually only one principle. It shouldn't be assumed that the answers are complex. Sometimes the hardest things to understand are the most simple.&lt;/p&gt;&#xA;&lt;p&gt;So the question &amp;quot;What do we mean when we say &amp;quot;intelligence&amp;quot;? is really important to AI. And the conclusion is that AI ought to replace its current behavioral definition of &amp;quot;intelligence&amp;quot; with one that includes the human elements of generalization and perception. And then get on and try to work out the operating principles, or principle, of both of these.&lt;/p&gt;&#xA;" OwnerUserId="17709" LastEditorUserId="2444" LastEditDate="2021-01-24T19:13:47.397" LastActivityDate="2021-01-24T19:13:47.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7711" PostTypeId="1" CreationDate="2018-08-25T12:02:44.437" Score="4" ViewCount="89" Body="&lt;p&gt;I'm watching a youtube video where a guy is talking about how computers can &lt;em&gt;learn&lt;/em&gt; to go from some point A to some point B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the way he does it is very disappointing: all he does is generate &lt;em&gt;thousands&lt;/em&gt; of objects that go from A to some random destination, and then he updates all the random destinations by taking the average of the original destination + the destination of the object that got closest to B. This process is then repeated until convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see what's so impressive about this. If you wanted the machine to go from A to B, why did you not just ... you know .... &lt;em&gt;tell it to go to B?&lt;/em&gt;. What's the point of essentially generating &lt;em&gt;random&lt;/em&gt; instructions, and hoping one of those random instructions just happens to be: &lt;em&gt;go to B&lt;/em&gt;, when you could just as well &lt;em&gt;tell&lt;/em&gt; that instruction to the computer yourself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this really what people mean when they say that a computer &quot;learns&quot; to do some task? And if so, what's so impressive about it? What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="17717" LastEditorUserId="1671" LastEditDate="2018-08-27T19:42:22.163" LastActivityDate="2018-08-27T19:42:22.163" Title="What exactly do people mean when they say that the computer &quot;learns&quot; to do something?" Tags="&lt;machine-learning&gt;&lt;concepts&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7712" PostTypeId="2" ParentId="7684" CreationDate="2018-08-25T13:08:58.910" Score="0" Body="&lt;p&gt;This will depend to some extent on what you want to do with the language models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some possible resources are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TensorFlow offers 3 pre-trained language models in the &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research&quot; rel=&quot;nofollow noreferrer&quot;&gt;research package&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cafe's &lt;a href=&quot;https://github.com/BVLC/caffe/wiki/Model-Zoo&quot; rel=&quot;nofollow noreferrer&quot;&gt;ModelZoo&lt;/a&gt; has a single pre-trained model that does video -&gt; captions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other packages like Cafe2  offer pre-trained models, but the documentation does not suggest any of them are suitable for language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Failing this, a good approach might be to email the authors of a paper that adopts an approach you like. Some (but far from all) researchers will be happy to share their models, which you can then use as a starting point for your own.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-25T13:08:58.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7714" PostTypeId="2" ParentId="7711" CreationDate="2018-08-25T18:05:27.193" Score="2" Body="&lt;p&gt;Your confusion is a result of &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Moravec%27s_paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moravec’s Paradox&lt;/a&gt;; the &lt;em&gt;just tell it to...&lt;/em&gt; step is enormously more difficult than it sounds. So what researchers do is attempt to find a general approach to that problem. The problem is expressed as a matrix of numbers (a grid that may be a million on each side these days) and “learning” is the process of computing the values for each of those numbers. Obviously no human can solve an equation like that, so a machine &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;solver&lt;/a&gt; is used. Therefore the work of the AI researcher is to find better ways to generate these large matrices i.e. to express &lt;em&gt;go to X&lt;/em&gt; and to create better solvers. The holy grail is a matrix and a solver that will work for any X. &lt;/p&gt;&#xA;" OwnerUserId="16207" LastActivityDate="2018-08-25T18:05:27.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7715" PostTypeId="1" CreationDate="2018-08-25T21:24:00.783" Score="4" ViewCount="66" Body="&lt;p&gt;I’m training a network to do image classification on zoo animals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m a software engineer and not an ML expert, so I’ve been retraining Google’s Inception model and the latest models is trained using Google AutoML Vision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network performs really well, but I have trouble with images of animals that I don’t want any labels for. Basically I would like images of those animals to be classified as unknowns or achieve low scores. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have images of the animals that I don’t want labels for and I tried putting them all into one “nothing” label together with images I’ve collected of the animals habitats without any animals. This doesn’t really yield any good results though. The network performs for the labeled animals but ends up assigning one of those labels to the other animals as well. Usually with a really high score as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 14 labels and 10.000 images. I should also mention that the “nothing” label ends up having a lot of images compared to the actual labels. Those images are not included in the 10.000.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tricks to achieve better results with this? Should I create multiple labels for the images in the “nothing” category maybe?&lt;/p&gt;&#xA;" OwnerUserId="17725" LastActivityDate="2018-08-28T01:47:10.400" Title="Optimizing image recognition results for unknown labels" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7716" PostTypeId="2" ParentId="5103" CreationDate="2018-08-26T07:58:14.333" Score="0" Body="&lt;p&gt;Google AI offers &lt;a href=&quot;https://thenextweb.com/artificial-intelligence/2018/07/18/a-beginners-guide-to-ai-computer-vision-and-image-recognition/&quot; rel=&quot;nofollow noreferrer&quot;&gt;recognition AI&lt;/a&gt; of images, sounds, heat maps, whatever. There are many places that offer this and is not unique to Google AI. AI works on data, so anything that can be turned into data is good to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article you're linking to doesn't really reveal anything valuable to the consumer; rather it is playing up the hype of Google making AI that can configure AI.&lt;/p&gt;&#xA;" OwnerUserId="17732" LastActivityDate="2018-08-26T07:58:14.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7717" PostTypeId="1" AcceptedAnswerId="7718" CreationDate="2018-08-26T08:49:29.377" Score="3" ViewCount="67" Body="&lt;p&gt;In A3C, there are several child processes and one master process. The child precesses calculate the loss and backpropagation, and the master process sums them up and updates the parameters, if I understand it correctly.&lt;/p&gt;&#xA;&lt;p&gt;But I wonder how I should decide the number of the child processes to implement. I think the more child processes are, the better it is to disentangle the correlation between the samples is, but I'm not sure what is the cons of setting a large number of child processes.&lt;/p&gt;&#xA;&lt;p&gt;Maybe the more child processes are, the larger the variance of the gradient is, leading to the instability of the learning? Or is there any other reason?&lt;/p&gt;&#xA;&lt;p&gt;And finally, how should I decide the number of the child processes?&lt;/p&gt;&#xA;" OwnerUserId="7402" LastEditorUserId="2444" LastEditDate="2021-01-23T14:12:43.343" LastActivityDate="2021-01-23T14:12:43.343" Title="What is the pros and cons of increasing and decreasing the number of worker processes in A3C?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;a3c&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7718" PostTypeId="2" ParentId="7717" CreationDate="2018-08-26T11:20:03.450" Score="3" Body="&lt;p&gt;The correct number of child processes will depend on the hardware available to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simplifying a bit, child processes can be in one of two states: waiting for memory or disk access, or running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem fits nicely in your computers' memory, then processes will spend almost all of their time running. If it's too big for memory, they will periodically need to wait for disk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should use approximately 1 Child process per CPU core. If you are training on a GPU, then it depends whether the process can make use of the entire GPU at once (in which case, use just 1), or whether a &quot;process&quot; is really more like a CUDA thread here (in which case you'd want one per CUDA core).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think your processes will wait for disk, use more than one per core. About 50% more is a good starting point. You can use a program like &lt;a href=&quot;https://en.wikipedia.org/wiki/Top_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;top&lt;/a&gt; to monitor CPU usage and adjust the number of processes accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question more explicitly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Having more child processes (up to a point, discussed above), will increase hardware utilization, and make your training run faster. With a Core i7 CPU for instance, you might be able to run 8 or 16 child processes at a time, so you'd train 8-16x times faster.&lt;/li&gt;&#xA;&lt;li&gt;Having more child processes than processing units (CPU cores, CUDA cores), will begin to cause frequent context switching, where the processing units have to pause to change between different jobs. Changing jobs is extremely expensive, and ultimately, your program cannot train faster than it would by using all the available hardware. If you have more processes than processing units, reducing the number should make your program train faster.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-26T11:20:03.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7720" PostTypeId="2" ParentId="7715" CreationDate="2018-08-26T21:08:44.987" Score="2" Body="&lt;p&gt;Welcome to AI.SE @Stromgren!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A likely explanation is that the animals in the &quot;nothing&quot; group do not have much in common with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means it will be difficult for the network to learn which features from the images are associated with that label (in fact, there aren't any!). As a result, the network is probably assigning very low confidence to any estimate for the nothing label. You should be able to check if this is the case (i.e. examine whether it is &lt;em&gt;ever&lt;/em&gt; confident about the label &quot;nothing&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not completely sure how the Inception network encodes its labels. A common scheme though is to use one output neuron for each class. The correct label for an image is thus always a &quot;1-hot&quot; vector, where one of the elements is set to 1, and the others all to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that representation is being used, you can incorporate the &quot;nothing&quot; data by labelling it with a vector that is all zero: none of the output neurons should activate for it. That would produce precisely the training signal you want. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-08-28T01:47:10.400" LastActivityDate="2018-08-28T01:47:10.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7721" PostTypeId="1" AcceptedAnswerId="7725" CreationDate="2018-08-27T01:58:20.250" Score="15" ViewCount="12675" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1507.06527.pdf&quot; rel=&quot;noreferrer&quot;&gt;Deep Recurrent Q-Learning for Partially Observable MDPs&lt;/a&gt;, the author processed the Atari game frames with an LSTM layer at the end. My questions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How does this method differ from the &lt;a href=&quot;https://datascience.stackexchange.com/questions/20535/understanding-experience-replay-in-reinforcement-learning&quot;&gt;experience replay&lt;/a&gt;, as they both use past information in the training? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What's the typical application of both techniques? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can they work together?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If they can work together, does it mean that the &lt;em&gt;state&lt;/em&gt; is no longer a single state but a set of contiguous states?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17365" LastEditorUserId="2444" LastEditDate="2019-04-22T12:08:55.260" LastActivityDate="2019-10-22T16:23:01.393" Title="How does LSTM in deep reinforcement learning differ from experience replay?" Tags="&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;&lt;deep-rl&gt;&lt;comparison&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7723" PostTypeId="1" CreationDate="2018-08-27T07:32:55.710" Score="3" ViewCount="1253" Body="&lt;p&gt;I am using the PPO algorithm implemented by tensorforce: &lt;a href=&quot;https://github.com/reinforceio/tensorforce&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/reinforceio/tensorforce&lt;/a&gt; . It works great and I am very happy with the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I notice that there are many metaparameters available to give to the PPO algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; # the tensorforce agent configuration ------------------------------------------&#xA;    network_spec = [&#xA;        dict(type='dense', size=256),&#xA;        dict(type='dense', size=256),&#xA;    ]&#xA;&#xA;    agent = PPOAgent(&#xA;        states=environment.states,&#xA;        actions=environment.actions,&#xA;        network=network_spec,&#xA;        # Agent&#xA;        states_preprocessing=None,&#xA;        actions_exploration=None,&#xA;        reward_preprocessing=None,&#xA;        # MemoryModel&#xA;        update_mode=dict(&#xA;            unit='episodes',&#xA;            # 10 episodes per update&#xA;            batch_size=10,&#xA;            # Every 10 episodes&#xA;            frequency=10&#xA;        ),&#xA;        memory=dict(&#xA;            type='latest',&#xA;            include_next_states=False,&#xA;            capacity=200000&#xA;        ),&#xA;        # DistributionModel&#xA;        distributions=None,&#xA;        entropy_regularization=0.01,&#xA;        # PGModel&#xA;        baseline_mode='states',&#xA;        baseline=dict(&#xA;            type='mlp',&#xA;            sizes=[32, 32]&#xA;        ),&#xA;        baseline_optimizer=dict(&#xA;            type='multi_step',&#xA;            optimizer=dict(&#xA;                type='adam',&#xA;                learning_rate=1e-3&#xA;            ),&#xA;            num_steps=5&#xA;        ),&#xA;        gae_lambda=0.97,&#xA;        # PGLRModel&#xA;        likelihood_ratio_clipping=0.2,&#xA;        # PPOAgent&#xA;        step_optimizer=dict(&#xA;            type='adam',&#xA;            learning_rate=1e-3&#xA;        ),&#xA;        subsampling_fraction=0.2,&#xA;        optimization_steps=25,&#xA;        execution=dict(&#xA;            type='single',&#xA;            session_config=None,&#xA;            distributed_spec=None&#xA;        )&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So my question is: is there a way to understand, intuitively, the meaning / effect of all these metaparameters and use this intuitive understanding to improve training performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have reached - from a mix of reading the PPO paper and the literature around, and playing with the code - to the following conclusions. Can anybody complete / correct?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;effect of &lt;strong&gt;network_spec&lt;/strong&gt;: this is size of the 'main network'. Quite classical: need it big enough to get valuable predictions, not too big either otherwise it is hard to train.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;effect of the parameters in &lt;strong&gt;update_mode&lt;/strong&gt;: this is how often the network updates are performed.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;batch_size&lt;/em&gt; is how many used for a batch update. Not sure of the effect neither what this exactly means in practice (are all samples taken from only 10 batches of the memory replay)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;frequency&lt;/em&gt; is how often the update is performed. I guess having frequency high would make the training slower but more stable (as sample from more different batches)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;unit&lt;/em&gt;: no idea what this does&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;memory&lt;/strong&gt;: this is the replay memory buffer.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;type&lt;/em&gt;: not sure what this does or how it works.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;include_next_states&lt;/em&gt;: not sure what this does or how it works&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;capacity&lt;/em&gt;: I think this is how many tuples (state, action, reward) are stored. I think this is an important metaparameter. In my experience, if this is too low compared to the number of actions in one episode, the learning is very bad. I guess this is because it must be large enough to store MANY episodes, otherwise the network learns from correlated data - which is bad.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;DistributionMode&lt;/strong&gt;: guess this is the model for the distribution of the controls? No idea what the parameters there do.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;PGModel&lt;/strong&gt;: No idea what the paramaters there do. Would be interesting to know if some should be tweaked / which ones.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;PGLRModel&lt;/strong&gt;: idem, no idea what all these parameters do / if they should be tweaked.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;PPOAgend&lt;/strong&gt;: idem, no idea what all these parameters do / if they should be tweaked.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in summary, would be great to get some help about:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which parameters should be tweaked&lt;/li&gt;&#xA;&lt;li&gt;How should these parameters be tweaked? Is there a 'high level intuition' about how they should be tweaked / in which circumstances?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17753" LastActivityDate="2018-08-27T13:24:07.380" Title="Tuning of PPO metaparameters: a high level overview of what each parameter does" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7725" PostTypeId="2" ParentId="7721" CreationDate="2018-08-27T09:28:06.883" Score="16" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How does this method differ from the experience replay, as they both use past information in the training? What's the typical application of both techniques?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Using a recurrent neural network is one way for an agent to build a model of hidden or unobserved state in order to improve its predictions when direct observations do not give enough information, but a &lt;em&gt;history&lt;/em&gt; of observations might give better information. Another way is to learn a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;noreferrer&quot;&gt;Hidden Markov model&lt;/a&gt;. Both of these approaches build an internal representation that is effectively considered part of the state when making a decision by the agent. They are a way to approach solving &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;noreferrer&quot;&gt;POMDPs&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can consider using individual frame images from Atari games as state as a POMDP, because each individual frame does not contain information about velocity. Velocity of objects in play is an important concept in many video games. By formulating the problem as a POMDP with individual image inputs, this challenges the agent to find some representation of velocity (or something similar conceptually) from a sequence of images. Technically a NN may also do this using fixed inputs of 4 frames at a time (as per the original DQN Atari paper), but in that case the designers have deliberately &quot;solved&quot; the partially observable part of the problem for the agent in advance, by selecting a better state representation from the start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience replay solves some different problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Efficient use of experience, by learning repeatedly from observed transitions. This is important when the agent needs to use a low learning rate, as it does when the environment has stochastic elements or when the agent includes a complex non-linear function approximator like a neural network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;De-correlating samples to avoid problems with function approximators that work best with i.i.d. data. If you didn't effectively shuffle the dataset, the correlations between each time step could cause significant issues with a feed-forward neural network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These two issues are important to learning stability for neural networks in DQN. Without experience replay, often Q-learning with neural networks will fail to converge at all. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can they work together?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sort of, but not quite directly, because LSTM requires input of multiple related time steps at once, as opposed to randomly sampled individual time steps. However, you could keep a history of longer trajectories, and sample sections from it for the history in order to train a LSTM. This would still achieve the goal of using experience efficiently. Depending on the LSTM architecture, you may need to sample quite long trajectories or even complete episodes in order to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From comments by Muppet, it seems that is even possible to sample more randomly with individual steps by saving LSTM state. For instance, there is a paper &quot;&lt;a href=&quot;https://arxiv.org/pdf/1803.03916.pdf&quot; rel=&quot;noreferrer&quot;&gt;Deep reinforcement learning for time series: playing idealized trading games&lt;/a&gt;&quot; where the authors get a working system doing this. I have no experience of this approach myself, and there are theoretical reasons why this may not work in all cases, but it is an option.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If they can work together, does it mean that the state is no longer a single state but a set of contiguous states?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not really, the state at any time step is still a single state representation, is separate conceptually from an observation, and is separate conceptually from a &lt;em&gt;trajectory&lt;/em&gt; or sequence of states used to train a RNN (other RL approaches such as TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) also require longer trajectories). Using an LSTM implies you have &lt;em&gt;hidden&lt;/em&gt; state on each time step (compared to what you are able to observe), and that you hope the LSTM will discover a way to represent it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to think of this is that the state is the current observation, plus a summary of observation history. The original Atari DQN paper simply used the previous three observations hard-coded as this &quot;summary&quot;, which appeared to capture enough information to make predicting value functions reliable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LSTM approach is partly of interest, because it does not rely on human input to decide how to construct state from the observations, but discovers this by itself. One key goal of deep learning is designs and architectures that are much less dependent on human interpretations of the problem (typically these use &lt;em&gt;feature engineering&lt;/em&gt; to assist in learning process). An agent that can work directly from raw observations has solved more of the problem by itself without injection of knowledge by the engineers that built it.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-22T16:23:01.393" LastActivityDate="2019-10-22T16:23:01.393" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7726" PostTypeId="2" ParentId="7723" CreationDate="2018-08-27T10:12:13.720" Score="1" Body="&lt;p&gt;Some investigation about the memory dict: The current type is &lt;code&gt;latest&lt;/code&gt;, which means you're not using a memory replay, but a latest memory. Switching to replay may help.&#xA;Also, &lt;code&gt;include_next_state&lt;/code&gt; means that you store tuples (state, action, reward, next state). It's not a real parameter though, because in PPO it must be set to &lt;code&gt;False&lt;/code&gt;, otherwise an error is raised.&#xA;Your interpretation of &lt;code&gt;capacity&lt;/code&gt; looks OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the update mode spec dict, your current settings mean that : every 10 (&lt;code&gt;frequency&lt;/code&gt;) episodes (&lt;code&gt;unit&lt;/code&gt;), you pull a batch of 10 (&lt;code&gt;batch_size&lt;/code&gt;) episodes (&lt;code&gt;unit&lt;/code&gt;) from the memory (the pulling method is defined through the memory dict), and you perform an optimization step over this batch. Be aware that the &lt;code&gt;unit&lt;/code&gt; defines both the unit of the optimization frequency and the type of the object fetched from the memory.&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="75" LastEditDate="2018-08-27T13:24:07.380" LastActivityDate="2018-08-27T13:24:07.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7727" PostTypeId="1" CreationDate="2018-08-27T10:18:17.893" Score="5" ViewCount="1270" Body="&lt;p&gt;I am trying to understant how it works.  How do you teach it say, to add 1 to each number it gets.  I am pretty new to the subject and I learned how it works when you teach it to identify a picture of a number.  I can understand how it identifies a number but I cant get it how would it study to perform addition? I can understand that it can  identify a number or picture using the pixels and assigning weights and then learning to measure whether a picture of a number resembling the weight is assigned to each pixel. But i can't logically understand how would it learn the concept  of adding a number by one.  Suppose I showed it thousands of examples of 7 turning to 8 152 turning into 153 would it get it that every number in the world has to be added by one? How would it get it having no such operation of + ? Since addition does not exist to its proposal then how can it realize that it has to add one in every number? Even by seeing thousands of examples but having no such operation of plus I cant understand it.    I could understand identifying pixels and such but such an operation I cant get the theoretical logic behind it. Can you explain the logic in layman terms?&lt;/p&gt;&#xA;" OwnerUserId="17760" LastActivityDate="2018-08-27T14:41:05.927" Title="How is it possible to teach a neural network to perform addition?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7728" PostTypeId="2" ParentId="7727" CreationDate="2018-08-27T12:27:25.177" Score="1" Body="&lt;p&gt;Welcome to AI.SE @bilanush.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example approach that might make things clearer. There are other ways to train a neural network to do this however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your earlier example with an image, you probably noticed that the network receives the image as a series of values, representing each pixel in the image. The network then learns which of a series of output neurons should be active in response to a given set of pixel values. Those output neurons, when read in an appropriate way, correspond to the correct label for the image. The difference between the set of outputs that should have been active, and the set that were active, forms the basis of the error signal that allows the network to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You've probably heard that computers represent numbers with &lt;em&gt;binary digits&lt;/em&gt;. So you could think of the number 16 as being: 00010000 in &quot;8-bit binary&quot;. In 16-bit binary, this number would be 0000000000010000, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one way of viewing your problem is a function mapping binary inputs to binary outputs (very similar to labelling a black-and-white image). For instance, the input 00010000 (16) should produce the output 00010001 (17). The input 00100011 (35) should produce the output 00100100 (36), and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As before, you will have a set of output neurons. In this case, it should be wide as the set of input neurons. As before, the error signal is the difference between the expected inputs and outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to the question of how they can learn this function &quot;without plus&quot;, in fact the individual neurons in a network perform just two operations: addition of their inputs, and a non-linear transformation of the sum. It has been proven that these are sufficient to learn any function from inputs to outputs, as long as the network contains 3 layers or more, and the as long as the middle layer is wide enough, but here it should be easy to see how addition might emerge.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-27T12:27:25.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7729" PostTypeId="2" ParentId="7617" CreationDate="2018-08-27T12:35:04.057" Score="5" Body="&lt;p&gt;I was hoping to see more answers here, but I'll get us started with some examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Combinatorial Search Problems&lt;/em&gt;&lt;/strong&gt;: If your problem can be phrased as movement through a &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorics#Graph_theory&quot; rel=&quot;noreferrer&quot;&gt;combinatorial graph&lt;/a&gt;, you don't need a neural network. In particular, your problem should have discrete states, a clear set of actions that are possible in each state, a clear definition of where we start, and a clear definition of what the goal state looks like. The most effective general purpose technique is &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search&quot; rel=&quot;noreferrer&quot;&gt;iterative deepening search&lt;/a&gt;. If you have an idea about which moves might be more effective, or better, a function that estimates how far each state is from the goal, you may be able to build a heuristic function and use &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;noreferrer&quot;&gt;A* search&lt;/a&gt; instead. Common applications for these techniques include pathfinding in video games (or directions in other applications), AI planning, and Automated Theorem Proving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll add some more topics later, but I suspect others have expertise to share here. Let's see some more ideas!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-27T12:35:04.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7731" PostTypeId="2" ParentId="1655" CreationDate="2018-08-27T13:24:52.440" Score="2" Body="&lt;p&gt;ASIC - It stands for Application-specific integrated circuit. Basically, you write programs to design a chip in &lt;a href=&quot;https://en.wikipedia.org/wiki/Hardware_description_language&quot; rel=&quot;nofollow noreferrer&quot;&gt;HDL&lt;/a&gt;. I'll take cases of how modern computers work to explain my point:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;CPU's&lt;/strong&gt; - CPU's are basically a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hardware_description_language&quot; rel=&quot;nofollow noreferrer&quot;&gt;microprocessor&lt;/a&gt; with many helper IC's performing specific tasks. In a microprocessor, there is only a single Arithmetic Processing unit (made up term) called &lt;a href=&quot;https://en.wikipedia.org/wiki/Accumulator_(computing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accumulator&lt;/a&gt; in which a value has to be stored, as computations are performed only and only the values stored in the accumulator. Thus every instruction, every operation, every R/W operation has to be done through the accumulator (that is why older computers used to freeze when you wrote from a file to some device, although nowadays the process has been refined and may not require accumulator to come in-between specifically &lt;a href=&quot;https://en.wikipedia.org/wiki/Direct_memory_access&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMA&lt;/a&gt;).&#xA;Now in ML algorithms, you need to perform matrix multiplications which can be easily parallelized, but we have in our has a single processing unit only and so came the GPU's.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;GPU's&lt;/strong&gt; - GPU's have 100's processing units but they lack the multipurpose facilities of a CPU. So they are good for parallelizable calculations. Since there is no memory overlapping (same part of the memory being manipulated by 2 processes)  in matrix multiplication, GPU's will work very well. Though since GPU is not multi-functional it will work only as fast as a CPU feeds data into its memory.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;ASIC&lt;/strong&gt; - ASIC can be anything a GPU, CPU or a processor of your design, with any amount of memory you want to give to it. Let' say you want to design your own specialized ML processor, design a processor on ASIC. Do you want a 256-bit FP number? Create a 256-bit processor. You want your summing to be fast? Implement a parallel adder up to a higher number of bits than conventional processors? You want &lt;code&gt;n&lt;/code&gt; number of cores? No problem. you want to define the data-flow from different processing units to different places? You can do it. Also with careful planning, you can get a trade-off between ASIC area vs power vs speed. The only problem is that for all of this you need to create your own standards. Generally, some well-defined standards are followed in designing processors, like a number of pins and their functionality, IEEE 754 standard for floating-point representation, etc which have been come up after lots of trial and errors. So if you can overcome all of these you can easily create your own ASIC.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I do not know what Google is doing with their TPU's but apparently, they designed some sort of Integer and FP standard for their 8-bit cores depending on the requirements at hand. They probably are implementing it on ASIC for power, area and speed considerations.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="36737" LastEditDate="2021-03-31T22:25:10.080" LastActivityDate="2021-03-31T22:25:10.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7732" PostTypeId="2" ParentId="7727" CreationDate="2018-08-27T13:47:39.293" Score="0" Body="&lt;p&gt;This is what we call a regression problem. Although @John has provided a novel method I do not think it will work since you are decomposing the number into its minimal representation, so teaching it will be quite tough due to long term dependencies like &lt;code&gt;0111111&lt;/code&gt; will change to &lt;code&gt;100000&lt;/code&gt;, so you have to train on almost all examples, with 0 actual learning.&lt;/p&gt;&#xA;&lt;p&gt;Let's see your problem from a different viewpoint. Why are you thinking only of integers? Your problem can be generalised to approximating this curve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/95Bkk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/95Bkk.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is clearly an example of &lt;span class=&quot;math-container&quot;&gt;$W.T * X + B$&lt;/span&gt;. Single input node will feed into 2 output nodes of Leaky ReLu activation, and a bias node. The bias node will be your &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and the 2-3 Leaky ReLu's will adjust their weights to create a straight line. Training this might be a problem (with co-adaptation between nodes), but mathematically a solution will be achieved by this Neural Net structure.&lt;/p&gt;&#xA;&lt;p&gt;Also it is better to train on real values also, for better and finer weight adjustment (although theoretically for a single independent variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; you should just need 3-4 values to make this Neural Net learn, but who knows?)&lt;/p&gt;&#xA;&lt;p&gt;NOTE: The approximation in the negative region might not be that great.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-27T14:41:05.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7733" PostTypeId="1" AcceptedAnswerId="7745" CreationDate="2018-08-27T15:25:32.923" Score="4" ViewCount="60" Body="&lt;p&gt;I'm trying to use an ANN to learn from a large amount of forest measurement data obtained from sampling plots across Ontario, Canada and associated climate data provided by regional climate modelling in this province. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the following are the inputs to the ANN:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Location (GPS coordinates)&lt;/li&gt;&#xA;&lt;li&gt;Measurement year and month&lt;/li&gt;&#xA;&lt;li&gt;Tree species&lt;/li&gt;&#xA;&lt;li&gt;Age&lt;/li&gt;&#xA;&lt;li&gt;Soil type&lt;/li&gt;&#xA;&lt;li&gt;Soil moisture regime&lt;/li&gt;&#xA;&lt;li&gt;Seasonal or monthly average temperature&lt;/li&gt;&#xA;&lt;li&gt;Seasonal or monthly average precipitation&lt;/li&gt;&#xA;&lt;li&gt;Some more data are available to select&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And the targets include:&#xA;- Average total tree height&#xA;- Average tree diameter at breast height&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each sampling plot, the trees have been measured for 1-4 times. &#xA;So my question is what type of ANN can best used to learn from the data and then it can be used for predicting with a set of new input data?&lt;/p&gt;&#xA;" OwnerUserId="17766" LastEditorUserId="2193" LastEditDate="2018-08-28T14:08:50.067" LastActivityDate="2018-08-28T17:33:07.623" Title="Type of artificial neural network suitable for learning and then predicting forest growth" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7734" PostTypeId="1" CreationDate="2018-08-27T16:13:16.433" Score="7" ViewCount="404" Body="&lt;p&gt;What are the best machine learning models that have been used to compose music? Are there some good research papers (or books) on this topic out there?&lt;/p&gt;&#xA;&lt;p&gt;I would say, if I use a neural network, I would opt for a recurrent one, because it needs to have a concept of timing, chord progressions, and so on.&lt;/p&gt;&#xA;&lt;p&gt;I am also wondering how the loss function would look like, and how I could give the AI so much feedback as they usually need.&lt;/p&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="2444" LastEditDate="2021-01-20T22:05:49.187" LastActivityDate="2021-01-20T22:05:49.187" Title="What are the best machine learning models for music composition?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;&lt;generative-model&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7735" PostTypeId="2" ParentId="7734" CreationDate="2018-08-27T16:17:06.133" Score="1" Body="&lt;p&gt;There are a few of them. The most recent I've found is from DeepMind: &lt;a href=&quot;https://arxiv.org/abs/1806.10474&quot; rel=&quot;nofollow noreferrer&quot;&gt;The challenge of realistic music generation: modelling raw audio at scale&lt;/a&gt;. This &lt;a href=&quot;https://www.youtube.com/watch?v=8GUYAVXmhsI&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; is a great analysis of it.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="2444" LastEditDate="2021-01-20T22:01:11.443" LastActivityDate="2021-01-20T22:01:11.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7736" PostTypeId="1" CreationDate="2018-08-27T18:41:56.223" Score="6" ViewCount="66" Body="&lt;p&gt;Due to my RL algorithm having difficulties learning some control actions, I've decided to use imitation learning/apprenticeship learning to guide my RL to perform the optimal actions.  I've read a few articles on the subject and just want to confirm how to implement it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Do I simply sample a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, then perform the optimal action &lt;span class=&quot;math-container&quot;&gt;$a^*$&lt;/span&gt; in that state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, calculate the reward for the action &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, and then observe the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, and finally put that into the experience replay?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;If this is the case, I am thinking of implementing it as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize the optimal replay buffer &lt;span class=&quot;math-container&quot;&gt;$D_O$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Add the optimal tuple of experience &lt;span class=&quot;math-container&quot;&gt;$(s, a^*, r, s')$&lt;/span&gt; into the replay buffer &lt;span class=&quot;math-container&quot;&gt;$D_O$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Initialize the normal replay buffer &lt;span class=&quot;math-container&quot;&gt;$D_N$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;During the simulation, initially sample &lt;span class=&quot;math-container&quot;&gt;$(s, a^*, r, s')$&lt;/span&gt; only from the optimal replay buffer &lt;span class=&quot;math-container&quot;&gt;$D_O$&lt;/span&gt;, while populating the normal replay buffer &lt;span class=&quot;math-container&quot;&gt;$D_N$&lt;/span&gt; with the simulation results.&lt;/li&gt;&#xA;&lt;li&gt;As training/learning proceeds, anneal out the use of the optimal replay buffer, and sample only from the normal replay buffer.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Would such an architecture work?&lt;/p&gt;&#xA;" OwnerUserId="17706" LastEditorUserId="2444" LastEditDate="2020-11-05T23:52:33.090" LastActivityDate="2020-11-05T23:52:33.090" Title="In imitation learning, do you simply inject optimal tuples of experience $(s, a, r, s')$ into your experience replay buffer?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;&lt;imitation-learning&gt;&lt;apprenticeship-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7738" PostTypeId="1" AcceptedAnswerId="7748" CreationDate="2018-08-28T00:25:29.353" Score="1" ViewCount="355" Body="&lt;p&gt;I want to create a simple object detection tool. So, basically, an image will be provided to the tool, and, from that image, it has to detect the number of objects.&lt;/p&gt;&#xA;&lt;p&gt;For example, an image of a dining table that has certain items present on it, such as plates, cups, forks, spoons, bottles, etc.&lt;/p&gt;&#xA;&lt;p&gt;The tool has to count the &lt;strong&gt;number of objects&lt;/strong&gt;, irrespective of the type of object. After counting, it should return the position of the object with its size, so that I can draw a border over it.&lt;/p&gt;&#xA;&lt;p&gt;I would like not to use any library or API present such as TensorFlow, OpenCV, etc., given that I want to learn the details.&lt;/p&gt;&#xA;&lt;p&gt;If the process is very difficult to be created without using an API then the number of/type of objects which it will count as an object can also be limited but since this project will be for my educational/learning purpose can anyone help me understand the logic using which this can be achieved? For example, it may ignore a napkin present in the table to be counted as an object.&lt;/p&gt;&#xA;" OwnerUserId="17776" LastEditorUserId="2444" LastEditDate="2020-09-12T15:29:48.770" LastActivityDate="2020-09-12T15:31:57.207" Title="How can I develop an object detection system that counts the number of objects and determines their position in an image?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;object-detection&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7739" PostTypeId="1" AcceptedAnswerId="7741" CreationDate="2018-08-28T04:55:58.417" Score="5" ViewCount="827" Body="&lt;p&gt;I am completely new to all this, for the life of me I can't find the answer to this question anywhere on Google.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens &lt;strong&gt;after&lt;/strong&gt; you have used machine learning to train your model? What happens to the training data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's pretend it predicted correct 99.99999% of the time and you were happy with it and wanted to share it with the world. If you put in 10GB of training data is the file you share with the world 10GB? If it was all trained on AWS can people only use your service if they connect to AWS through an API?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens to all the old training data? Does the model still need all of it to make new predictions? &lt;/p&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="16909" LastEditDate="2018-08-29T01:43:39.367" LastActivityDate="2018-08-29T10:50:54.740" Title="What happens to the training data after your machine learning model has been trained?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;ai-basics&gt;&lt;datasets&gt;&lt;concepts&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7740" PostTypeId="2" ParentId="7617" CreationDate="2018-08-28T08:46:30.983" Score="3" Body="&lt;p&gt;A nice example &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Processes&lt;/a&gt;, which can be solved by classic &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt; techniques like &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q learning&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A Markov Decision Process consists of&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A set of &lt;em&gt;discrete states&lt;/em&gt; (or continuous states that have been discretized)&lt;/li&gt;&#xA;&lt;li&gt;A set of possible &lt;em&gt;actions&lt;/em&gt; that can be taken in each state.&lt;/li&gt;&#xA;&lt;li&gt;A set of &lt;em&gt;transition probabilities&lt;/em&gt; that describe how an agent stochastically moves from its current state to the next, based on the agent's actions.&lt;/li&gt;&#xA;&lt;li&gt;A &lt;em&gt;reward function&lt;/em&gt; quantitatively describing how nice it is to be in each state.&lt;/li&gt;&#xA;&lt;li&gt;A &lt;em&gt;discounting factor&lt;/em&gt; that describes how much worse it is to receive a reward in the future than today.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Very small MDPs can be directly, exactly, solved, using techniques like &lt;a href=&quot;https://artint.info/html/ArtInt_227.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration&lt;/a&gt;, but the computational cost for these approaches grows extremely fast.&lt;/p&gt;&#xA;&lt;p&gt;Reinforcement Learning (RL) was developed as a machine learning approach for MDPs. There is a loop: the agent gets the state of the environment, chooses an action, executes this action on the environment, and he gets back a reward, and the new state of the environment, and so on... You want the agent to maximize the cumulative reward over time.&lt;/p&gt;&#xA;&lt;p&gt;The basic concept of Q Learning doesn't use ANNs. In Q learning, you build a state-action matrix, called the Q matrix. Thus, you must discretize the states of your environment, and the actions available to your agent. Then, the coefficient Qij is the expected reward when you perform the action j on the state i. In basic Q Learning, you explore and build this matrix, and it should converge and give an &amp;quot;optimal rule of action&amp;quot; for your agent.&lt;/p&gt;&#xA;&lt;p&gt;However, the situation is often too complex, and you often want a non-discretized space of states or actions. Here Deep QL arrives, where the Q matrix becomes an ANN.&lt;/p&gt;&#xA;&lt;p&gt;You can find a nice QL tutorial &lt;a href=&quot;http://adventuresinmachinelearning.com/reinforcement-learning-tutorial-python-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (normal and deep).&lt;/p&gt;&#xA;&lt;p&gt;And a lecture about QL &lt;a href=&quot;https://www.youtube.com/watch?v=lvoHnicueoE&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Keep in mind that only ANNs perform well in complex situations, so you'll always see examples with ANNs, even if the basic theory doesn't require ANNs.&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="2444" LastEditDate="2021-01-18T11:30:36.640" LastActivityDate="2021-01-18T11:30:36.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7741" PostTypeId="2" ParentId="7739" CreationDate="2018-08-28T09:39:55.577" Score="4" Body="&lt;p&gt;In many cases, a production-ready model has everything it needs to make predictions without retaining training data. For example: a linear model might only need the coefficients, a decision tree just needs rules/splits, and a neural network needs architecture and weights. The training data isn't required as all the information needed to make a prediction is incorporated into the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some algorithms retain some or all of the training data. A support vector machine stores the points ('support vectors') closest to the &lt;a href=&quot;https://sebastianraschka.com/faq/docs/lazy-knn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;separating hyperplane&lt;/a&gt;, so that portion of the training data will be stored with the model. Further, &lt;a href=&quot;https://nlp.stanford.edu/IR-book/html/htmledition/support-vector-machines-the-linearly-separable-case-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;k-nearest neighbours&lt;/a&gt; must evaluate all points in the dataset every time a prediction is made, and as a result the model incorporates the entire training set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, where possible the training data would be retained. If additional data is received, a new model can be trained on the enlarged dataset. If it is decided a different approach is required, or if there are concerns about concept drift, then it's good to have the original data still on hand. In many cases, the training data might comprise personal data or make a company's competitive advantage, so the model and the data should stay separate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to see how this can work, &lt;a href=&quot;https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Keras blog post&lt;/a&gt; has some information (note: no training data required to make predictions once a model is re-instantiated).&lt;/p&gt;&#xA;" OwnerUserId="9091" LastEditorUserId="9091" LastEditDate="2018-08-29T10:50:54.740" LastActivityDate="2018-08-29T10:50:54.740" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7743" PostTypeId="2" ParentId="4095" CreationDate="2018-08-28T11:31:09.893" Score="0" Body="&lt;p&gt;All the methods in the GameState class that is used to represent state, are &lt;strong&gt;stubs&lt;/strong&gt;, and without these, the MCTS algorithm won't do anything at all. In particular, the DoMove method just changes who's turn it is, without actually taking any action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the reason the players can't see each other's cards is that this is not a completed implementation. Someone is either still working on this, or gave up half way.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-28T11:31:09.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7744" PostTypeId="2" ParentId="4140" CreationDate="2018-08-28T17:23:06.110" Score="3" Body="&lt;p&gt;The second equation is correct. In TD($\lambda$), the $\lambda$ parameter can be tuned to smoothly vary between single-step updates (essentially what Sarsa does) in the case of $\lambda = 0$, and Monte-Carlo returns (using the full episode's returns) in the case of $\lambda = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first equation, $\sum_{j = i}^{N - 1} \lambda^{j - i} d_i$ could be interpreted as a... summing up exactly the same temporal-difference term $d_i$ a number of times (specifically, $N - 1 - j$ times), but multiplied by a different scalar every time. I'm not sure how that could be useful in any way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second equation, $\sum_{m = i}^{N - 1} \lambda^{m - i} d_m$ can be interpreted as a weighted combination;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$1 \times d_i$: this is the difference between what we predict our returns will be at time $i + 1$, and what we previously predicted our returns would be at time $i$.&lt;/li&gt;&#xA;&lt;li&gt;$+ \lambda^1 \times d_{i + 1}$: a very similar difference between two of our own predictions, now the predictions at time $i + 2$ and $i + 1$. This time the temporal-difference term is weighted by the parameter $\lambda$ (completely ignored if $\lambda = 0$, full weight if $\lambda = 1$, somewhere in between if $0 &amp;lt; \lambda &amp;lt; 1$).&lt;/li&gt;&#xA;&lt;li&gt;$+ \lambda^2 \times d_{i + 2}$: again a similar temporal-difference term, now again one step further into the future. Downweighted a bit more than the previous term in cases where $0 &amp;lt; \lambda &amp;lt; 1$.&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For people who are familiar with temporal-difference algorithms like TD($\lambda$), sarsa($\lambda$), eligibility traces, etc. from Reinforcement Learning literature, this makes a lot more sense. The notation is still a bit different from the standard literature on algorithms like TD($\lambda$), but in fact becomes equivalent once you note that in this paper they discuss domains where there are only rewards associated with terminal states, and no intermediate rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, what they're doing with the $\lambda$ parameter is assigning more weight (or &quot;credit&quot; or &quot;importance&quot;) to short-term predictions / short-term &quot;expectations&quot; (in the English sense of the word, rather than the mathematical sense of the word) or observations of rewards, over long-term predictions/observations. In the extreme case of $\lambda = 0$, you completely ignore long-term predictions/observations and only propagate observed rewards very slowly, one-by-one in single steps. In the other extreme case of $\lambda = 1$, you propagate rewards observed at the end of episodes with equal weight all the way to the beginning of the episodes, through all states that you went to, giving them all equal weight for that observed reward. With $0 &amp;lt; \lambda &amp;lt; 1$, you choose a balance between those two extremes.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Also note that Equation (5) in the KnightCap paper (where they similarly discuss the extreme case of $\lambda = 1$, like I did above) is incorrect if we take the first equation from your question, but is correct if we take the second equation.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-28T17:23:06.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7745" PostTypeId="2" ParentId="7733" CreationDate="2018-08-28T17:33:07.623" Score="1" Body="&lt;p&gt;My suggestion is not to use an ANN, but instead to use a simpler regression algorithm. The main reason for this is that ANNs take a long time to train, and work better when a lot of data is used. They also require a lot of expertise in parameter tuning to apply well. Since you say you don't have a lot of data, and also don't have a lot of experience using them, I think you will be better off applying something else first. If the other techniques don't work at all, then you might think about using ANNs, but again, they tend to want a lot of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have tried &lt;a href=&quot;https://en.wikipedia.org/wiki/Least_squares&quot; rel=&quot;nofollow noreferrer&quot;&gt;ordinary least squares regression&lt;/a&gt;, and found it does not work well, my next choice would be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classification And Regression Tree&lt;/a&gt;. These models can make good decisions with small amounts of data, and do not require a lot of time to train. They can handle real-valued outputs like the height and width of a tree. Weka's &lt;a href=&quot;http://weka.sourceforge.net/doc.dev/weka/classifiers/trees/REPTree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;REPTree&lt;/a&gt; might be a good starting place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Trees don't work out, my next suggestion would be to try regression using a Support Vector Machine. SciKitLearn's &lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVR&lt;/a&gt; is a good choice for this. SVRs can sometimes be very effective when data is limited, because they make assumptions about how to handle data-poor regions that seem to be generally applicable. An SVM can also report low confidence when estimating in those regions. They also train fairly fast when using small amounts of data, and can learn non-linear functions from the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to use an ANN, I would start with a simple &lt;a href=&quot;http://scikit-learn.org/stable/modules/neural_networks_supervised.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-layer perceptron&lt;/a&gt;. This model has few parameters to play with, and can probably fit well to your regression. It may make strange decisions in regions with less data however. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-28T17:33:07.623" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7746" PostTypeId="1" AcceptedAnswerId="7753" CreationDate="2018-08-28T20:09:42.130" Score="4" ViewCount="177" Body="&lt;p&gt;Cognitive psychology is researched since the 1940s. The idea was to understand human problem solving and the importants of heuristics in it. George Katona (an early psychologist) published in the 1940s a paper about human learning and teaching. He mentioned the so called Katona-Problem, which is a geometric task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Squares&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Katona style problems are the ones where you remove straws in a given configuration of straws to create n unit squares in the end. In the end, every straw is an edge to a unit square. Some variations include 2x2 or 3x3 sizes of squares allowed as well as long as no two squares are overlapping, i.e. a bigger square 2x2 can't contain a smaller square of size 1x1. Some problems use matchsticks as a variation, some use straws, others use lines. Some variations allow bigger square to contain smaller one as long as they don't share an edge viz. &lt;a href=&quot;https://puzzling.stackexchange.com/questions/59316/matchstick-squares&quot;&gt;https://puzzling.stackexchange.com/questions/59316/matchstick-squares&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a way we can view it as a graph and removing straws/matchsticks as deleting edges between nodes in a graph? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If so, can I train a bot where I can plugin some random, yet valid conditions for the game and goal state to get the required solution? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit #1: The following problem is just a sample to show where I am getting at. The requirement for my game is much larger. Also, I chose uninformed search to make things simpler without bothering about complex heuristics and optimization techniques. Please be free to explore ideas with me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scenario #1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this scenario. In the following diagram, each dashed line or pipe line represents a straw. Numbers and alphabet denote junctions where straw meet. Let's say, my bot can explore each junction, remove zero, one, two, three or four straws such that resultant state has&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;no straw that dangles off by being not connected to a square.&lt;/li&gt;&#xA;&lt;li&gt;a small mxm square isn't contained in a larger nxn square (m&#xA;&lt;li&gt;Once straw is removed, it can't be put back. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Initial configuration is shown here. I always need to start from top left corner node P and optimization... the objective is to remove straws in minimum hops from node to node using minimum number of moves, by the time goal state is reached.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       P------Q------R------S------T&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       E------A------B------F------G&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       J------C------D------H------I&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       K------L------M------N------O&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       U------V------W------X------Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Goal 1 : I wish to create a large 2x2 square.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At some point during, say BFS search (although it could be any uninformed search on partially observable universe i.e. viewing one node at a time), I could technically reach A, blow out all edges on A to create the following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       P------Q------R------S------T&#xA;       |             |      |      |&#xA;       |             |      |      |&#xA;       E      A      B------F------G&#xA;       |             |      |      |&#xA;       |             |      |      |&#xA;       J------C------D------H------I&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       K------L------M------N------O&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       U------V------W------X------Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is one move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goal 2 : I want to create a 3x3 square instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't do that in one move. I need the record of successive nodes to be explored and then possibly backtrack to given point as well if the state fails to produce desired result. Each intermediate state might produce rectangles which are not allowed (also, how would one know how many more and which straws to remove to get to a square) or dangle a straw or worse get stuck in an infinite loop as I can choose to not remove any straw. How do I approach this problem?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Edit 2:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;For validation, figures 3, 4 and 5 are given below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       P------Q------R------S------T&#xA;       |             |      |      |&#xA;       |             |      |      |&#xA;       E      A      B------F      G&#xA;       |             |      |      |&#xA;       |             |      |      |&#xA;       J------C------D------H      I&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       K------L------M------N      O&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       U------V------W------X      Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above figure (3) is invalid as we can't have dangling sticks TG,GI etc.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       P------Q------R------S------T&#xA;       |      |                    |&#xA;       |      |                    |&#xA;       E------A                    G&#xA;       |                           |&#xA;       |                           |&#xA;       J                           I&#xA;       |                           |&#xA;       |                           |&#xA;       K                           O&#xA;       |                           |&#xA;       |                           |&#xA;       U------V------W------X------Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above figure (4) is invalid as we can't have overlapping squares&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       P------Q------R      S      T&#xA;       |             |             &#xA;       |             |            &#xA;       E      A      B------F------G&#xA;       |             |      |      |&#xA;       |             |      |      |&#xA;       J------C------D------H------I&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       K------L------M------N------O&#xA;       |      |      |      |      |&#xA;       |      |      |      |      |&#xA;       U------V------W------X------Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Figure (5) is valid configuration.&lt;/p&gt;&#xA;" OwnerUserId="17799" LastEditorUserId="17799" LastEditDate="2018-08-30T15:42:26.037" LastActivityDate="2018-08-30T15:42:26.037" Title="How do I train a bot to solve Katona style problems?" Tags="&lt;getting-started&gt;&lt;learning-algorithms&gt;&lt;intelligent-agent&gt;&lt;breadth-first-search&gt;&lt;graph-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7747" PostTypeId="1" AcceptedAnswerId="7752" CreationDate="2018-08-28T23:10:23.383" Score="3" ViewCount="277" Body="&lt;p&gt;I've seen many &lt;a href=&quot;https://en.wikipedia.org/wiki/Lorem_ipsum&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lorem Ipsum&lt;/a&gt; generators on the web, but not only, there is also &quot;bacon ispum&quot;, &quot;space ispum&quot;, etc. So, how do these generators generate the text? Are they powered by an AI?&lt;/p&gt;&#xA;" OwnerUserId="17801" LastEditorUserId="2444" LastEditDate="2020-05-13T21:18:59.270" LastActivityDate="2020-05-13T21:18:59.270" Title="How does the &quot;Lorem Ipsum&quot; generator work?" Tags="&lt;natural-language-processing&gt;&lt;text-summarization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7748" PostTypeId="2" ParentId="7738" CreationDate="2018-08-29T08:00:05.877" Score="4" Body="&lt;p&gt;If you want to get experience, you should probably start with some easier task. Object detection and localization are relatively hard and writing a neural network and image processing pipeline from scratch will take you a long time.&lt;/p&gt;&#xA;&lt;p&gt;If you want to build up an intuition about how NN's work, you might want to code some simple task from scratch. &lt;a href=&quot;http://jonathanweisberg.org/post/A%20Neural%20Network%20from%20Scratch%20-%20Part%201/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is an example.&lt;/p&gt;&#xA;&lt;p&gt;When you got some intuition about how NN's work, then you should proceed to your task. In &lt;a href=&quot;https://ai.stackexchange.com/a/7497/16929&quot;&gt;here&lt;/a&gt;, you have a similar question and answer provided. The current state-of-the-art approach for your task would probably be point 3, that is object detection network, like &lt;a href=&quot;https://arxiv.org/abs/1506.02640&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/abs/1506.01497&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faster-RCNN&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16929" LastEditorUserId="2444" LastEditDate="2020-09-12T15:31:57.207" LastActivityDate="2020-09-12T15:31:57.207" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7749" PostTypeId="1" AcceptedAnswerId="7750" CreationDate="2018-08-29T08:01:14.643" Score="3" ViewCount="196" Body="&lt;p&gt;Tensorflow/Lucid is able to visualize what a &quot;channel&quot; of a layer of a neural network (image recognition, Inception-v1) responds to. Even after studying the tutorial, the source code, the three research papers on lucid and comments by the authors on Hacker News, I'm still not clear on how &quot;channels&quot; are supposed to be defined and individuated. Can somebody shed some light on this? Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorflow/lucid&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorflow/lucid&lt;/a&gt; &lt;br/&gt;&#xA;&lt;a href=&quot;https://news.ycombinator.com/item?id=15649456&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://news.ycombinator.com/item?id=15649456&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17807" LastEditorUserId="17221" LastEditDate="2018-08-29T16:49:18.313" LastActivityDate="2018-08-29T16:55:14.630" Title="What exactly does &quot;channel&quot; refer to in tensorflow/lucid?" Tags="&lt;tensorflow&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7750" PostTypeId="2" ParentId="7749" CreationDate="2018-08-29T08:24:56.330" Score="3" Body="&lt;p&gt;The channel they are talking about is the depth of the layer L.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bM582.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bM582.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this image, the channel is 5. There are 5 $3*2$ filters&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They optimize an image from noise, to better respond to the filter F, in the layer L. Then you obtain this kind of image, and you can try to interpret the prupose of that filter (it learns to detect eyes, faces, wheels, and so on)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To be more precise, you visualise the channel only if you optimise images for every filters, otherwise, you obtain a filter visualisation (if you do the optimisation one time, for one filter)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They call it &quot;channel&quot;, because in a colored image, you have 3 dimensions : witdh, height, and channel (for color), as layers which have also 3 dimensions.&lt;/p&gt;&#xA;" OwnerUserId="17221" LastEditorUserId="9947" LastEditDate="2018-08-29T16:55:14.630" LastActivityDate="2018-08-29T16:55:14.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7751" PostTypeId="2" ParentId="7747" CreationDate="2018-08-29T08:42:58.440" Score="3" Body="&lt;p&gt;If you wanted to generate more I guess you could take the string and convert to a list then you could randomly select as many words as you want, from the list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Python&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;lorem = &quot;lLorem ipsum dolor sit amet, consectetur adipiscing elit. &quot;.split()&#xA;&#xA;number_of_words_needed = 20&#xA;&#xA;new_text = []&#xA;&#xA;for i in range(number_of_words_needed):&#xA;    new_text.append(lorem[np.random.randint(len(lorem))])&#xA;&#xA;print(new_text)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ipsum ['sit', 'dolor', 'elit.', 'sit', 'sit', 'sit', 'elit.', 'dolor', 'amet,', 'ipsum', 'amet,', 'ipsum', 'dolor', 'Lorem', 'Lorem', 'adipiscing', 'sit', 'elit.', 'consectetur', 'adipiscing']&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;for reference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.lipsum.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;: &quot;Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source. Lorem Ipsum comes from sections &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A2007.01.0036%3Abook%3D1%3Asection%3D32&quot; rel=&quot;nofollow noreferrer&quot;&gt;1.10.32&lt;/a&gt; and &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A2007.01.0036%3Abook%3D1%3Asection%3D33&quot; rel=&quot;nofollow noreferrer&quot;&gt;1.10.33&lt;/a&gt; of &quot;de Finibus Bonorum et Malorum&quot; (The Extremes of Good and Evil) by Cicero, written in 45 BC. This book is a treatise on the theory of ethics, very popular during the Renaissance. The first line of Lorem Ipsum, &quot;Lorem ipsum dolor sit amet..&quot;, comes from a line in section 1.10.32.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard chunk of Lorem Ipsum used since the 1500s is reproduced below for those interested. Sections 1.10.32 and 1.10.33 from &quot;de Finibus Bonorum et Malorum&quot; by Cicero are also reproduced in their exact original form, accompanied by English versions from the 1914 translation by H. Rackham.&quot;&lt;/p&gt;&#xA;" OwnerUserId="17809" LastEditorUserId="1671" LastEditDate="2018-08-29T21:26:04.417" LastActivityDate="2018-08-29T21:26:04.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7752" PostTypeId="2" ParentId="7747" CreationDate="2018-08-29T08:56:55.713" Score="4" Body="&lt;p&gt;Lorem ipsum generators don't typically use anything considered as AI. Usually they just store large pieces of text and select sections from it randomly - they are very simple. The main goal is to produce &quot;nonsense&quot; text that fills space but does not distract from issues of layout and design. The variations of it are usually just for fun, and like the original, are mostly simple generators which select strings of text from a core data source randomly and without using any AI techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It &lt;em&gt;is&lt;/em&gt; possible to build more sophisticated random text generators that work using data structures from &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Processing&lt;/a&gt; (NLP). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One popular and easy-to-code data structure is &lt;a href=&quot;https://en.wikipedia.org/wiki/N-gram&quot; rel=&quot;nofollow noreferrer&quot;&gt;N-grams&lt;/a&gt;, which store the frequencies/probabilities of the N&lt;sup&gt;th&lt;/sup&gt; word given words 1 to N-1. E.g. a bigram structure can tell you all the possible words to come after &quot;fish&quot; e.g. &lt;code&gt;&quot;fish&quot; =&amp;gt; [&quot;food&quot; =&amp;gt; 0.2, &quot;swims&quot; =&amp;gt; 0.3, &quot;and&quot; =&amp;gt; 0.4, &quot;scale&quot; =&amp;gt; 0.1]&lt;/code&gt; To use that structure to generate text, use a random number generator to select a word based on looking up the N&lt;sup&gt;th&lt;/sup&gt; word's frequency, then shift the list of words being considered and repeat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more recent text generating NLP model is recurrent neural networks (RNNs), which have a variety of designs. Popular right now are LSTM networks, and these are capable of some quite sophisticated generation, provided they are trained with enough data for long enough. The blog &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Unreasonable Effectiveness of Recurrent Neural Networks&lt;/a&gt; by Andrej Karpathy has quite a few really interesting examples of using RNNs for text generation. In practice this works similarly to n-grams: Use the RNN to suggest probabilities for next word given words so far, choose one randomly, then feed back the generated word into the RNN and repeat.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-08-29T16:06:24.727" LastActivityDate="2018-08-29T16:06:24.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7753" PostTypeId="2" ParentId="7746" CreationDate="2018-08-29T11:40:45.867" Score="2" Body="&lt;p&gt;Welcome to AI.SE @GundamOfOasis!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your intuition is right: this is fundamentally a problem for &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial search&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're also right that problems are created by the fact that not every move is valid at state. To fix this, you need to add a function that can determine whether a given state is valid or not, in addition to the usual function that checks whether it is your goal sate or not. Before adding each node to the queue of your search algorithm, check whether it is a valid state. If it isn't, don't add it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second issue you raise is that your search might enter an infinite loop. Since it is possible to remove zero edges from a state, this is a serious concern. There are two approaches to solving this. First, you can try storing all states that you have already visited in a fast data structure like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hash_table&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hash Table&lt;/a&gt;. Before adding a node to your queue, check if it's already been processed. If it has been, don't add it. This may work, but the memory requirements grow exponentially in the number of moves required for a solution. It's sometimes worth it, but I think you can likely skip it for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better approach if you're worried about speed is to switch your algorithm to something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterative deepening&lt;/a&gt;, which has the good properties of BFS, but with much lower memory requirements; or to &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A* search&lt;/a&gt; if you can come up with an admissible heuristic for your domain (a good starting point: counting the number of junctions you'd need to remove sticks from to finish, if the robot could teleport, would be admissible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Here's some pseudo-code for filtering out invalid moves:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function valid_state(State s){&#xA;    for stick in s.remaining_sticks:&#xA;        if stick is vertical:&#xA;           1. let side = walk up from the middle of stick until it becomes possible to turn right.&#xA;           2. let side += walk down from the middle of stick until it becomes possible to turn right.&#xA;           3. From the first junction above stick where we can turn right, try to walk *side* steps to the right.&#xA;           4. Then try to walk side steps down.&#xA;           5. Then try to walk side steps left.&#xA;           6. Repeat previous 5 steps but for the nearest junctions where we can turn left instead of right.&#xA;        else:&#xA;           Do exactly what's in the if above, but substitute &quot;left&quot; for &quot;up&quot; and &quot;right&quot; for &quot;down&quot;.      &#xA;    if we could walk in a square successfully for every stick, this is a valid state, so return true. Otherwise, return false.     &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-08-29T20:34:54.887" LastActivityDate="2018-08-29T20:34:54.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7754" PostTypeId="2" ParentId="7684" CreationDate="2018-08-29T13:52:59.557" Score="1" Body="&lt;p&gt;Of course now there has been a huge development:&#xA;Huggingface published &lt;a href=&quot;https://github.com/huggingface/pytorch-transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;pytorch-transformers&lt;/a&gt;, a library for the so successful Transformer models (BERT and its variants, GPT-2, XLNet, etc.), including many pretrained (mostly English or multilingual) models (&lt;a href=&quot;https://huggingface.co/pytorch-transformers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs here&lt;/a&gt;). It also includes one German BERT model. SpaCy offers a &lt;a href=&quot;https://github.com/explosion/spacy-pytorch-transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;convenient wrapper&lt;/a&gt; (&lt;a href=&quot;https://explosion.ai/blog/spacy-pytorch-transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: Now, Salesforce published the English model &lt;a href=&quot;https://github.com/salesforce/ctrl&quot; rel=&quot;nofollow noreferrer&quot;&gt;CTRL&lt;/a&gt;, which allows for use of &quot;control codes&quot; that influence the style, genre and content of the generated text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, here is the old, now less relevant version of my answer:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Since I posed the question, I found this &lt;strong&gt;pretrained German language model&lt;/strong&gt;:&#xA;&lt;a href=&quot;https://lernapparat.de/german-lm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lernapparat.de/german-lm/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an instance of a &lt;a href=&quot;https://github.com/Bachfischer/german2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;3-layer &quot;averaged stochastic descent weight-dropped&quot; LSTM&lt;/a&gt; which was implemented based on an implementation by &lt;a href=&quot;https://github.com/salesforce/awd-lstm-lm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salesforce&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17670" LastEditorUserId="17670" LastEditDate="2019-09-23T06:47:46.260" LastActivityDate="2019-09-23T06:47:46.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7755" PostTypeId="1" AcceptedAnswerId="8564" CreationDate="2018-08-29T16:04:16.113" Score="13" ViewCount="4688" Body="&lt;p&gt;I'm coding a Proximal Policy Optimization (PPO) agent with the &lt;a href=&quot;https://github.com/reinforceio/tensorforce&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorforce library&lt;/a&gt; (which is built on top of TensorFlow).&lt;/p&gt;&#xA;&lt;p&gt;The first environment was very simple. Now, I'm diving into a more complex environment, where all the actions are not available at each step.&lt;/p&gt;&#xA;&lt;p&gt;Let's say there are 5 actions and their availability depends on an internal state (which is defined by the previous action and/or the new state/observation space):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2 actions (0 and 1) are always available&lt;/li&gt;&#xA;&lt;li&gt;2 actions (2 and 3) are only available when the internal state is 0&lt;/li&gt;&#xA;&lt;li&gt;1 action (4) is only available when the internal state is 1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Hence, there are 4 actions available when the internal state is 0 and 3 actions available when the internal state is 1.&lt;/p&gt;&#xA;&lt;p&gt;I'm thinking of a few possibilities to implement that:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Change the action space at each step, depending on the internal state. I assume this is nonsense.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do nothing: let the model understand that choosing an unavailable action has no impact.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do &lt;em&gt;almost&lt;/em&gt; nothing: impact slightly negatively the reward when the model chooses an unavailable action.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Help the model: by incorporating an integer into the state/observation space that informs the model what's the internal state value + bullet point 2 or 3&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Are there other ways to implement this? From your experience, which one would be the best?&lt;/p&gt;&#xA;" OwnerUserId="17818" LastEditorUserId="2444" LastEditDate="2020-11-17T19:15:36.720" LastActivityDate="2020-11-17T19:15:36.720" Title="How to implement a variable action space in Proximal Policy Optimization?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;discrete-action-spaces&gt;&lt;action-spaces&gt;&lt;tensorforce&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7756" PostTypeId="1" AcceptedAnswerId="7757" CreationDate="2018-08-29T16:15:20.757" Score="3" ViewCount="204" Body="&lt;p&gt;I have been recently reading about model selection algorithms (for example to decide which value of the regularisation parameter or what size of a neural network to use, broadly hyper-parameters). This is done by dividing the examples into three sets (training 60%, cross-validation 20%, test 20%) and training is done on the data with the first set for all parameters, and then choose the best parameter based on the result in the cross-validation and finally estimate the performance using the test set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the need for a different data-set compared to training and test for select the model, however, once the model is selected, why not using the cross-validation examples to improve the hypothesis before estimating the performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only reason I could see is that this could cause the hypothesis to worsen and we wouldn't be able to detect it, but, is it really possible that by adding much more examples (60% -&gt; 80%) the hypothesis gets worse?&lt;/p&gt;&#xA;" OwnerUserId="11303" LastEditorUserId="9947" LastEditDate="2018-08-29T17:10:43.090" LastActivityDate="2018-08-30T04:05:31.043" Title="Use cross-validation to train after model selection" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7757" PostTypeId="2" ParentId="7756" CreationDate="2018-08-29T16:44:51.000" Score="3" Body="&lt;p&gt;You are quite correct. If you have properly followed the Cross Validation procedure and selected the best model indeed, then you can use the CV set as the training set for the final model. And no it will not cause your hypothesis to worsen (for that set maybe, but not for new examples) if you have selected the model correctly. In-fact you may use the entire 100% of the data-set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Justin Johnson a TA at Stanford University answered a similar type of question on training CNN's using 100% of the data-set. He said that if you had enough computational resources and want to squeeze that extra 1% or 2% accuracy from your model you can use the entire data-set after model selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: As @NeilSlater pointed out, if you need the model for reporting purposes you should only use 80% of the data-set, otherwise you'll lose your only source for unbiased model verification. But if you are looking to deploy the model on field you can use 100% of the data-set.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-08-30T04:05:31.043" LastActivityDate="2018-08-30T04:05:31.043" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7758" PostTypeId="2" ParentId="7646" CreationDate="2018-08-29T22:51:33.037" Score="1" Body="&lt;p&gt;There is no easy way to play around with the hyper-parameters (number of layers, layer configuration, number of outputs per layer) of a CNN and get an accurate view of how these will affect the resulting performance of your model. However there are a few things that you can do to avoid wasting too much time training and re-training.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Why?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;When training a CNN we aim to minimize a loss function, thus a better CNN model is defined as one which converges to a set of model parameters with a lower loss function. Identifying the minimum of the loss function for a given CNN is already very difficult, and there is no guarantee that the true minimum will every be reached by fault of gradient descent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each variation of the CNN resulting from the different hyper-parameters will still result in a very large number of model parameters. There is no way to know how the loss function will look in this hyper-dimensional space and even harder to estimate its minimum.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What to do?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;First, you should try and understand how each layer in the network affects its inputs. You should know what kind of layers to use for what kind of data. You should also know what kinds of activation functions to use for different data distributions. You should also know how many model parameters per layer to try in order to sufficiently compress your data whilst not losing significant information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get a lot of this intuition by reading papers which have found successful models for specific tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, you can train with smaller amounts of data and estimate the potential minimum loss function by seeing how fast the loss function moves towards its minimal value (momentum). Usually a lower minimum is achieved when the loss function decreases faster. However, this is in no way always true. A loss function can converge slowly at first and then speed up later. This is entirely possible. But, you can get some sense of the potential of your model in this way.&lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2018-08-29T22:51:33.037" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7759" PostTypeId="2" ParentId="3801" CreationDate="2018-08-30T08:46:46.173" Score="0" Body="&lt;p&gt;Dynamic Computational Graphs are simply modified CGs with a higher level of abstraction. The word 'Dynamic' explains it all: how data flows through the graph depends on the input structure,i.e the DCG structure is mutable and not static. One of its important applications is in NLP neural networks.&lt;/p&gt;&#xA;" OwnerUserId="17833" LastActivityDate="2018-08-30T08:46:46.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7760" PostTypeId="2" ParentId="7734" CreationDate="2018-08-30T14:29:15.733" Score="0" Body="&lt;p&gt;I am also new to the neural network architecture game but from what I have learned so far I think you have a few good options to choose from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A recurrent neural network (RNN) would be a standard approach but if you're looking for something more robust you could look into a &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Long Short Term Memory network&lt;/a&gt; (LSTM). The neurons have a memory of past events and can recall that later on. It is a subset of RNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you could go a little further and use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Network&lt;/a&gt; (CNN). So far these type of networks have been highly successful for image recognition. You could abstract a song piece as an image. Each pixel could be a progression in time and the value of the pixel could be the actual note.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also take a look at &lt;a href=&quot;https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks-explained-3fb6f2367464&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; for a good overview of several different neural network types.&lt;/p&gt;&#xA;" OwnerUserId="17840" LastActivityDate="2018-08-30T14:29:15.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7761" PostTypeId="1" CreationDate="2018-08-30T19:11:25.857" Score="5" ViewCount="877" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory&quot; rel=&quot;noreferrer&quot;&gt;Dempster–Shafer theory (wiki)&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_probability&quot; rel=&quot;noreferrer&quot;&gt;Bayesian probability (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do these two methods handle uncertainty in regard to &lt;a href=&quot;https://en.wikipedia.org/wiki/Information_integration&quot; rel=&quot;noreferrer&quot;&gt;information fusion&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17847" LastEditorUserId="1671" LastEditDate="2018-08-30T19:18:25.207" LastActivityDate="2018-08-31T16:49:26.153" Title="How does the Dempster-Shafer theory of evidence differ from the Bayesian reasoning under uncertainty?" Tags="&lt;concepts&gt;&lt;probability&gt;&lt;reasoning&gt;&lt;bayes&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7762" PostTypeId="1" AcceptedAnswerId="7943" CreationDate="2018-08-30T20:05:59.967" Score="6" ViewCount="2580" Body="&lt;p&gt;How can I create an artificially intelligent aimbot for a game like &lt;a href=&quot;https://en.wikipedia.org/wiki/Counter-Strike:_Global_Offensive&quot; rel=&quot;nofollow noreferrer&quot;&gt;Counter-Strike Global Offensive (CS:GO)&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an initial solution (or approach) in mind. We can train an image recognition model that will recognize the head of the enemy (in the visible area of the player, so excluding the invisible area behind the player, to avoid being easily detected by VAC) and move the cursor to the position of the enemy's head and fire.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be much more preferable to train the recognition model in real-time than using demos. Most of the available demos you might have might be 32 tick, but while playing the game, it works at 64 tick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a very fresh idea in my mind, so I didn't actually think a lot about it. Ignoring facts like detection by VAC for a few moments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any research work on the topic? What are the common machine learning approaches to tackle such a problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later on, this idea can be expanded to a completely autonomous bot that can play the game by itself, but that is a bit too much initially.&lt;/p&gt;&#xA;" OwnerUserId="17849" LastEditorUserId="2444" LastEditDate="2019-11-22T18:31:10.123" LastActivityDate="2019-11-22T18:31:10.123" Title="How can I create an artificially intelligent aimbot for a game like CS:GO?" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;&lt;applications&gt;&lt;reference-request&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7763" PostTypeId="1" AcceptedAnswerId="7771" CreationDate="2018-08-30T23:45:20.763" Score="16" ViewCount="10348" Body="&lt;p&gt;I am studying reinforcement learning and the variants of it. I am starting to get an understanding of how the algorithms work and how they apply to an MDP.&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand is the process of defining the states of the MDP. In most examples and tutorials, they represent something simple like a square in a grid or similar.&lt;/p&gt;&#xA;&lt;p&gt;For more complex problems, like a robot learning to walk, etc., how do you go about defining those states? Can you use learning or classification algorithms to &amp;quot;learn&amp;quot; those states?&lt;/p&gt;&#xA;" OwnerUserId="17853" LastEditorUserId="2444" LastEditDate="2020-11-16T18:24:33.423" LastActivityDate="2020-11-16T18:24:33.423" Title="How to define states in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;state-spaces&gt;&lt;continuous-state-spaces&gt;&lt;discrete-state-spaces&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7764" PostTypeId="2" ParentId="7763" CreationDate="2018-08-31T02:41:58.353" Score="8" Body="&lt;p&gt;A common early approach to modeling complex problems was &lt;em&gt;discretization&lt;/em&gt;. At a basic level, this is splitting a complex and continuous space into a grid. Then you can use any of the classic RL techniques that are designed for discrete, linear, spaces. However, as you might imagine, if you aren't careful, this can cause a lot of trouble!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sutton &amp;amp; Barto's classic book &lt;em&gt;Reinforcement Learning&lt;/em&gt; has some suggestions for other ways to go about this. One is &lt;em&gt;tile coding&lt;/em&gt;, covered in section 9.5.4 of the &lt;a href=&quot;https://drive.google.com/file/d/1opPSz5AZ_kVa1uWOdOiveNiBFiEOHjkG/view&quot; rel=&quot;noreferrer&quot;&gt;new, second edition&lt;/a&gt;. In tile coding, we generate a large number of grids, each with different grid spacing. We then overlay the grids on top of each other. This creates discrete regions non-uniform shapes, and can work well for a variety of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Section 9.5 also covers a variety of other ways to encode a continuous space into a discrete MDP, including radial-basis functions, and coarse codings. Check it out!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="1671" LastEditDate="2019-07-18T19:40:37.783" LastActivityDate="2019-07-18T19:40:37.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7765" PostTypeId="2" ParentId="7736" CreationDate="2018-08-31T10:50:05.410" Score="2" Body="&lt;h2&gt;That seems to be functional.&lt;/h2&gt;&#xA;&lt;p&gt;That is a great approach, as long as you are using an &lt;a href=&quot;https://stats.stackexchange.com/questions/184657/what-is-the-difference-between-off-policy-and-on-policy-learning&quot;&gt;off-policy algorithm&lt;/a&gt; (since the samples you are using to learn are not the policy currently being performed), like Q-learning.&lt;/p&gt;&#xA;&lt;p&gt;By annealing the sample rate from the optimal buffer to the regular one, you introduce noise into the network and emphasize exploration (albeit more limited). This is helpful when you (the researcher) have no access to optimal policies, but merely &amp;quot;good&amp;quot; ones, and you still want the network to try and improve on those.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-08-31T10:50:05.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7766" PostTypeId="2" ParentId="7672" CreationDate="2018-08-31T11:37:51.847" Score="1" Body="&lt;p&gt;We can generalize both problem and solution by removing the specifics of housing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Representing Forward Propagation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a function $f$ we wish to obtain via the training of an artificial network that produces a scalar result $s$, the sole dependent variable and the generalization of market price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s = f(s_1, s_2, ..., s_k, c_1, c_2, ..., c_v)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The independent scalar variables $s_1$ through $s_k$ are the generalization of a constant number $k$ of property features from the tax authority, assessor's office, or inspection document.  The question calls this &lt;em&gt;structured data&lt;/em&gt;, however it is questionable whether $k$ is truly effectively constant.  In normal practice, some of $s_i$ will be unassigned.  Since the question overlooks the additional complexity of missing scalars, so will this answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The independent cube variables $c_1$ through $c_v$ are the generalizations of a variable number $v$ of property images from Google photography, real estate agents, buyers, sellers, and other potential sources.  The dimensions of each cube are horizontal and vertical positions and pixel structure element number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is unlikely that the resolution values for each cube are uniform between samples in real life, which the question did not mention, so this answer will overlook that complexity and focus on the variability of $v$, the quantity of cubes representing images for a given example.  Since $v$ cannot meaningfully be either negative or infinite, we can assume $v$ to be a non-negative integer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Terminology&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The observational unit should not be considered the house or the property but an image of it, which may be a member of a camera location and orientation category relative to the elements of the property.  Each image capture is an observation, distinct in both Bergsonian and clock time.  Each item under evaluation is an &lt;em&gt;example&lt;/em&gt; from the sample of all items, in this case, all properties in the region for which prediction is attempted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Design of a Solution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of the $v$ cubes demonstrate zero or more additional features of the item being evaluated, real properties in the question's specific case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be reasonable to assume that such features may either positively or negative affect the example's label corresponding to the result $s$, but not both positively and negatively affect it.  If that is the case, we can aggregate the features across the set of cubes for each example under the reasonable assumption that there are no points of inflection.  Such may be reasonable because, for instance, a feature regarding the uniformity of paint coverage, lawn care, or roofing material may have no inflection point.  Such aspects of the property cannot be too uniform.  That makes the substitution straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reasonably versatile way to generalize the aggregation of directionally consistent function is using a substitution, which may be what the question meant by &lt;em&gt;feed in the embeddings of the image to your final model&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s = f = f'(s_1, s_2, ..., s_k, v, h\big(v, \sum_{j = 1}^v g(c_j)\big)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the elements of this substitution.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$h$ is a vector function that normalizes the distribution of each feature found in the feature vector before using it as a set of inputs to $f'$.&lt;/li&gt;&#xA;&lt;li&gt;$g$ is a generalization of the cube, with the input (independent variable) being a cube representing the image and with the output (dependent variable) being a vector of features extracted.&lt;/li&gt;&#xA;&lt;li&gt;$v$, the number of cubes (visual observations) is fed along with extracted features into the function $f'$, which can be realized through the convergence of a multilayer perceptron.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If the images are grouped in terms of the location of the camera, then this principle can be applied iteratively, where $o$ is the number of distinct categories of camera orientation.  In this case, the pairs $(v_z, h_z)$ represent the cube quantity and feature aggregations for image camera location category $z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s = f = f'(s_1, s_2, ..., s_k, v_1, h_1, v_2, h_2, ... v_o, h_o)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given sensible models $h$ and $g$, training for prediction is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image feature extraction can be realized through ConvNet approaches such as OverFeat&lt;sup&gt;1&lt;/sup&gt;, AlexNet&lt;sup&gt;2&lt;/sup&gt;, CaffeNet&lt;sup&gt;3&lt;/sup&gt;, GoogLeNet&lt;sup&gt;4&lt;/sup&gt;, VGG 6&lt;sup&gt;5&lt;/sup&gt; or PatreoNet&lt;sup&gt;6&lt;/sup&gt;.  Tuning such models produces $g$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nature of function $h$ may be homogeneous or heterogeneous across dimensions.  Each component of the feature vector arising from extraction can have applied to it a function such as any of these or others, where $q$ is the feature index, $p_{qi}$ is learning parameter i.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$h_q(x) = x$&lt;/li&gt;&#xA;&lt;li&gt;$h_q(x) = \large{x^{p_{q1} + p_{q2} v}}$&lt;/li&gt;&#xA;&lt;li&gt;$h_q(x) = \log (x + p_{q1} v + p_{q2})$&lt;/li&gt;&#xA;&lt;li&gt;$h_q(x) = \large{\epsilon^x}$&lt;/li&gt;&#xA;&lt;li&gt;Others&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Scalar function 1 is best when the designer wishes the convergence during the training of $f'$ in such a way that normalization is accomplished in the net.  It is a good choice for features where its frequency of occurrence and magnitude are across the entire set of images for a given example item is roughly proportional to the resulting value of that item.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Function 2 presents flexibility in normalization curvature with respect to the number of cubes.  Function 3 presents attenuation of the frequency of feature occurrence.  Function 4 presents compounding of feature effect with recurrence in the images of the same example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key is then the selection of how to deal with the substitution in the training in terms of procedure and wiring of corrective signaling.  Procedurally, there are three options.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train both the ConvNet function $g$ and the multilayer perceptron function $f'$ together, extending the applicable principles of back-propagation and gradient descent.&lt;/li&gt;&#xA;&lt;li&gt;Extract features first, tuning the ConvNet corresponding to $g$ prior to training the network corresponding to $f'$.  The advantage of this approach is manual control over and interim evaluation of feature extraction.&lt;/li&gt;&#xA;&lt;li&gt;Use something similar to the mini-batch approach to find a balance between the above two extremes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&amp;mdash;&amp;mdash;&amp;mdash;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, Overfeat: Integrated recognition, localization and detection using convolutional networks, arXiv preprint arXiv:1312.6229v4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Neural Information Processing Systems, 2012, pp. 1106–1114&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell, Caffe: Convolutional architecture for fast feature embedding, arXiv preprint arXiv:1408.5093&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, arXiv preprint arXiv:1409.4842&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] K. Nogueira,  W. O. Miranda,  J. A. Dos Santos,  Improving spatial feature representation from aerial scenes by using convolutional networks, in:  Graphics, Patterns and Images (SIBGRAPI), 2015 28th SIBGRAPI Conference on, IEEE, 2015, pp. 289–296.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-16T14:07:02.137" LastActivityDate="2018-09-16T14:07:02.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7771" PostTypeId="2" ParentId="7763" CreationDate="2018-08-31T16:02:59.630" Score="21" Body="&lt;p&gt;The problem of state representation in Reinforcement Learning (RL) is similar to problems of feature representation, feature selection and feature engineering in supervised or unsupervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Literature that teaches the basics of RL tends to use very simple environments so that all states can be enumerated. This simplifies value estimates into basic rolling averages in a table, which are easier to understand and implement. Tabular learning algorithms also have reasonable theoretical guarantees of convergence, which means if you &lt;em&gt;can&lt;/em&gt; simplify your problem so that it has, say, less than a few million states, then this is worth trying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most interesting control problems will not fit into that number of states, even if you discretise them. This is due to the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Curse_of_dimensionality&quot; rel=&quot;noreferrer&quot;&gt;curse of dimensionality&lt;/a&gt;&quot;. For those problems, you will typically represent your state as a vector of different features - e.g. for a robot, various positions, angles, velocities of mechanical parts. As with supervised learning, you may want to treat these for use with a specific learning process. For instance, typically you will want them all to be numeric, and if you want to use a neural network you should also normalise them to a standard range (e.g. -1 to 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to the above concerns which apply for other machine learning, for RL, you also need to be concerned with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;noreferrer&quot;&gt;Markov Property&lt;/a&gt; - that the state provides enough information, so that you can accurately predict expected next rewards and next states given an action, without the need for any additional information. This does not need to be perfect, small differences due to e.g. variations in air density or temperature for a wheeled robot will not usually have a large impact on its navigation, and can be ignored. Any factor which is essentially random can also be ignored whilst sticking to RL theory - it may make the agent less optimal overall, but the theory will still work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are consistent unknown factors that influence result, and could logically be deduced - maybe from history of state or actions - but you have excluded them from the state representation, then you may have a more serious problem, and the agent may fail to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth noting the difference here between &lt;em&gt;observation&lt;/em&gt; and &lt;em&gt;state&lt;/em&gt;. An observation is some data that you can collect. E.g. you may have sensors on your robot that feed back the positions of its joints. Because the state should possess the Markov Property, a single raw observation might not be enough data to make a suitable state. If that is the case, you can either apply your domain knowledge in order to construct a better state from available data, or you can try to use techniques designed for &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;noreferrer&quot;&gt;partially observable MDPs (POMDPs)&lt;/a&gt; - these effectively try to build missing parts of state data statistically. You could use a RNN or hidden markov model (also called a &quot;belief state&quot;) for this, and in some way this is using a &quot;&lt;em&gt;learning or classification algorithms to &quot;learn&quot; those states&lt;/em&gt;&quot; as you asked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you need to consider the type of approximation model you want to use. A similar approach applies here as for supervised learning: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A simple linear regression with features engineered based on domain knowledge can do very well. You may need to work hard on trying different state representations so that the linear approximation works. The advantage is that this simpler approach is more robust against stability issues than non-linear approximation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A more complex non-linear function approximator, such as a multi-layer neural network. You can feed in a more &quot;raw&quot; state vector and hope that the hidden layers will find some structure or representation that leads to good estimates. In some ways, this too is &quot;&lt;em&gt;learning or classification algorithms to &quot;learn&quot; those states&lt;/em&gt;&quot; , but in a different way to a RNN or HMM. This might be a sensible approach if your state was expressed naturally as a screen image - figuring out the feature engineering for image data by hand is very hard.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;noreferrer&quot;&gt;Atari DQN work by DeepMind&lt;/a&gt; team used a combination of feature engineering and relying on deep neural network to achieve its results. The feature engineering included downsampling the image, reducing it to grey-scale and - importantly for the Markov Property - using four consecutive frames to represent a single state, so that information about velocity of objects was present in the state representation. The DNN then processed the images into higher-level features that could be used to make predictions about state values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-08-31T16:02:59.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7772" PostTypeId="2" ParentId="7761" CreationDate="2018-08-31T16:49:26.153" Score="2" Body="&lt;p&gt;Welcome to AI.SE @rudreshdwivedi!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a great question, and I hope to see many more like it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Demster-Shafer Theory and Bayesian Networks were both techniques that rose to prominence within AI in the 1970's and 1980's, as AI started to seriously grapple with uncertainty in the world, and move beyond the sterilized environments that most early systems worked in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 1970's and perhaps even earlier, it became apparent that direct applications of probability theory to AI were not going to work out because of the &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Curse_of_dimensionality&quot; rel=&quot;nofollow noreferrer&quot;&gt;curse of dimensionality&lt;/a&gt;&lt;/em&gt;. As more variables needed to be considered in a given problem, the amount of storage space and processing time needed increased exponentially. This led to a search for new methods to handle uncertainty within AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bayesian Networks and &lt;a href=&quot;https://artint.info/html/ArtInt_196.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Learning&lt;/a&gt; remained firmly rooted in probabilistic reasoning, but allowed for the assignment of &lt;em&gt;subjective priors&lt;/em&gt; to probabilities, to incorporate expert knowledge. It also allowed problems to be factored into graphical structures to avoid the curse of dimensionality in most cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dempster-Shafer was a further generalization of Bayesian Networks, in which malformed probability distributions were permitted as a way to capture uncertainty. So, for example, the probability of all possible events was not required to add up to 1, because there might be events we don't know about. While on the surface this might seem reasonable, most modern AI researchers view this as a deeply flawed approach. &lt;a href=&quot;https://www.ijcai.org/Proceedings/85-2/Papers/064.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cheeseman's criticism&lt;/a&gt; of DS and other non-probabilistic methods is the basis from which a lot of this view stems. Judea Pearl was another harsh and influential critic of DS Theory.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The basic difference in the fusion of new information is that in Bayesian Networks, after observing new evidence $$E$$, we apply Bayes' rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ P(H | E) = P(E | H) * P(H) / P(E) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to yield a posterior for every hypothesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In DS theory, we look for overlap between the worlds suggested by the new evidence and the old data. This can lead to non-sensical results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our prior belief is that our Robot is located at position (0,1) with probability 0.95, and position (0,2) with probability 0.05.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A new signal appears. The signal indicates that robot is at position (0,0) with probability 0.95, and position (0,2) with probability 0.05.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under Bayes rule, we consider the probability that these signals were generated under each of our original hypotheses, and the probability of observing these signals at all, as shown in the equation above. Under DS-Theory, we would do the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, DS-theory provides a second way to interpret the signal: as a second prior distribution, rather than as evidence. We can then combine this second prior with the first, to compute a sort of joint-prior:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P( H_{A,B}) \propto P(H_A) * P(H_B)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the &quot;probability&quot; (it's not always a true probability, which is one of the criticisms) of a hypothesis after the fusion will be the product of the &quot;probabilities&quot; of the hypothesis under each of the separate priors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example above, this gives a wacky result: the &quot;joint prior&quot; says the Robot is at (0,2) with probability 1.0. This and other problems are why this mode of information combination has mostly been abandoned. There are many more examples on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt; for DS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there's a discussion of this in more detail in Section IV of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt;, at the end of one of the chapters.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-08-31T16:49:26.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7773" PostTypeId="2" ParentId="7555" CreationDate="2018-08-31T18:12:28.507" Score="1" Body="&lt;p&gt;&lt;strong&gt;Approaches to the Game&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that the board has $16!$ possible states.  It is also true that using a hash set is what students learn in a first year algorithms courses to avoid redundancy and endless looping when searching a graph that may contain graph cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, those trivial facts are not pertinent if the goal is to complete the puzzle in the fewest computing cycles. Breadth first search isn't a practical way to complete an orthogonal move puzzle.  The very high cost of a breadth first search would only be necessary if number of moves is of paramount importance for some reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sub-sequence Descent&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the vertices representing states will never be visited, and each state that is visited can have between two and four outgoing edges.  Each block has an initial position and a final position and the board is symmetric.  The greatest freedom of choice exists when the open space is one of the four middle positions.  The least is when the open space is one of the four corner positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reasonable disparity (error) function is simply the sum of all x disparities plus the sum of all y disparities and a number heuristically representing which of the three levels of freedom of movement exists because of the resulting placement of the open space (middle, edge, corner).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although blocks may temporarily move away from their destinations to support a strategy toward completion requiring a sequence of moves, there is rarely a case where such a strategy exceeds eight moves, generating, on the average, 5,184 permutations for which the final states can be compared using the disparity function above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the empty space and positions of block 1 through 15 are encoded as an array of nibbles, only addition, subtraction, and bit-wise operations are needed, making the algorithm fast.  Repeating the eight move brute force strategies can be repeated until disparity falls to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm cannot cycle because there is always at least one of the permutations of eight moves that decreases disparity, regardless of the initial state, with the exception of a starting state that is already complete.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-08-31T18:12:28.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7774" PostTypeId="1" AcceptedAnswerId="7777" CreationDate="2018-08-31T19:03:53.250" Score="7" ViewCount="2562" Body="&lt;p&gt;I'm having a little trouble with the definition of rationality, which goes something like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An agent is rational if it maximizes its performance measure given its current knowledge.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I've read that a simple reflex agent will not act rationally in a lot of environments. For example, a simple reflex agent can't act rationally when driving a car, as it needs previous perceptions to make correct decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if it does its best with the information it's got, wouldn't that be rational behaviour, as the definition contains &quot;given its current knowledge&quot;? Or is it more like: &quot;given the knowledge it could have had at this point if it had stored all the knowledge it has ever received&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question about the definition of rationality: Is a chess engine rational as it picks the best move given the time it's allowed to use, or is it not rational as it doesn't actually (always) find the best solution (would need more time to do so)?&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2019-11-16T23:12:22.507" LastActivityDate="2019-11-16T23:12:22.507" Title="What is the definition of rationality?" Tags="&lt;definitions&gt;&lt;intelligent-agent&gt;&lt;chess&gt;&lt;rationality&gt;&lt;simple-reflex-agents&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7775" PostTypeId="2" ParentId="7774" CreationDate="2018-08-31T19:18:10.390" Score="7" Body="&lt;p&gt;When we use the term rationality in AI, it tends to conform to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;game theory&lt;/a&gt;/&lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision theory&lt;/a&gt; definition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent#Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;rational agent&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a solved or tractable game, an agent can have perfect rationality.  If the game is intractable, rationality is necessarily &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;bounded&lt;/a&gt;. (Here, &quot;game&quot; can be taken to mean any problem.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the issue of &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;imperfect information&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Complete_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;incomplete information&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rationality isn't restricted to objectively optimal decisions but includes subjectively optimal decisions, where the the optimality can only be presumed.  (That's why defection is the optimal strategy in 1-shot Prisoner's Dilemma, where the agents don't know the decisionmaking process of the the competitor.)  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rationality here conforms to Russell &amp;amp; Norvig's definition, where it is related to performance in an environment. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What may be rational in one environment may not be rational in a different environment.  Additionally, what may be locally rational for a simple reflex agent will not appear rational from the perspective of an agent with more knowledge, or a learning agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iterated Dilemmas, where there is communication in the form of prior choices, may provide an analogy.  An agent that always defects, even where the competitor has shown willingness to cooperate, may not be regarded as rational because  defecting vs. a cooperative agent does not maximize utility.  A simple reflex agent wouldn't have the capacity to alter its strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, rationality used in the most general sense might allow that, to the agent making the decision, if the decision is based on achieving an objective, and the decision is reached utilizing the information available to that agent, the decision is may be regarded as rational, regardless of actual optimality.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-10-21T22:37:56.600" LastActivityDate="2018-10-21T22:37:56.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7776" PostTypeId="1" CreationDate="2018-08-31T19:28:58.630" Score="6" ViewCount="287" Body="&lt;p&gt;My question relates to but doesn't duplicate a question that has been asked &lt;a href=&quot;https://ai.stackexchange.com/questions/4282/measuring-object-size-using-deep-neural-network&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've Googled a lot for an answer to the question: &lt;em&gt;Can you find the dimensions of an object in a photo if you don't know the distance between the lens and the object, and there are no &quot;scales&quot; in the image?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overwhelming answer to this has been &quot;no&quot;. This is, from my understanding, due to the fact that, in order to solve this problem with this equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Distance\ to\ object(mm) = \frac{f(mm) * real\ height(mm) * image\ height(pixels)}{object\ height(pixels) * sensor\ height(mm)}   $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you will need to know either the &quot;real height&quot; or the &quot;distance to object&quot;. It's the age old issue of &quot;two unknowns, one equation&quot;. That's unsolvable. A way around this is to place an object in the photo with a known dimension in the same plane as the unknown object, find the distance to this object and use that distance to calculate the size of the unknown (this relates to answer from the question I linked above). This is an equivalent of putting a ruler in the photo and it's a fine way to solve this problem easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where my question remains unanswered. What if there is no ruler? What if you want to find a way to solve the unsolvable problem? &lt;strong&gt;Can we train an Artificial Neural Network to approximate the value of the real height without the value of the object distance or use of a scale?&lt;/strong&gt; Is there a way to leverage the unexpected solutions we can get from AI to solve a problem that is seemingly unsolvable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example to solidify the nature of my question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to make an application where someone can pull out their phone, take a photo of a hail stone against the ground at a distance of ~1-3 ft, and have the application give them the hail stone dimensions. My project leader wants to make the application accessible, which means he doesn't want to force users to carry around a quarter or a special object of known dimensions to use as a scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to avoid the use of a scale, would it be possible to use all of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Exif&quot; rel=&quot;nofollow noreferrer&quot;&gt;EXIF&lt;/a&gt; meta-data from these photos to train a neural network to approximate the size of the hail stone within a reasonable error tolerance? For some reason, I have it in my head that if there are enough relevant variables, we can design an ANN that can pick out some pattern to this problem that we humans are just unable to identify. Does anyone know if this is possible? If so, is there a deep learning model that can best suit this problem? If not, please put me out of my misery and tell me why it it's impossible.&lt;/p&gt;&#xA;" OwnerUserId="17866" LastEditorUserId="1671" LastEditDate="2018-08-31T20:56:55.757" LastActivityDate="2018-09-25T07:51:55.137" Title="Can one use an Artificial Neural Network to determine the size of an object in a photograph?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;prediction&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7777" PostTypeId="2" ParentId="7774" CreationDate="2018-08-31T19:48:18.750" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I've read that a simple reflex agent will not act rationally in a lot of environments. E.g. a simple reflex agent can't act rationally when driving a car as it needs previous perceptions to make correct decisions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I wouldn't say that the need for previous perceptions is the reason why a simple reflex agent doesn't act rationally. I'd say the more serious issue with simple reflex agents is that they do not perform &lt;strong&gt;long-term planning&lt;/strong&gt;. I think that is the primary issue that causes them to not always act rationally, and that is also consistent with the definition of rationality you provided. A reflex-based agent typically doesn't involve long-term planning, and that's why it in fact does not often do best given the knowledge it has.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Another question about the definition of rationality: Is a chess engine rational as it picks the best move given the time its allowed to use, or is it not rational as it doesn't actually (always) find the best solution (would need more time to do so)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An algorithm like minimax in its &quot;purest&quot; formulation (without a limit on search depth) would be rational for games like chess, since it would play optimally. However, that is not feasible in practice, it would take too long to run. In practice, we'll run algorithms with a limit on search depth, to make sure that they stop thinking and pick a move in a reasonable amount of time. Those will not necessarily be rational. This gets back to bounded rationality as described by DukeZhou in his answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The story is not really clear if we try to talk about this in terms of &lt;em&gt;&quot;picking the best move given the time it's allowed to use&quot;&lt;/em&gt; though, because what is or isn't possible given a certain amount of time depends very much on factors such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;algorithm we choose to implement&lt;/li&gt;&#xA;&lt;li&gt;speed of our hardware&lt;/li&gt;&#xA;&lt;li&gt;efficiency of implementation / programming language used&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, hypothetically I could say that I implement an algorithm that requires a database of pre-computed optimal solutions, and the algorithm just looks up the solutions in the database and instantly plays the optimal moves. Such an algorithm would be able to truly be rational, even given a highly limited amount of time. It would be difficult to implement in practice because we'd have difficulties constructing such a database in the first place, but the algorithm itself is well-defined. &lt;strong&gt;So, you can't really include something like &quot;given the time it's allowed to use&quot; in your definition of rationality.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-08-31T19:48:18.750" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7778" PostTypeId="2" ParentId="5461" CreationDate="2018-09-01T03:43:37.717" Score="1" Body="&lt;p&gt;What you might want to look for is called &lt;strong&gt;video captioning&lt;/strong&gt;. Earlier examples from this line of research is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Venugopalan et. al. &lt;a href=&quot;https://arxiv.org/pdf/1505.00487.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence to Sequence – Video to Text&lt;/a&gt;, in ICCV 2015&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pasunuru et.al. &lt;a href=&quot;http://aclweb.org/anthology/P/P17/P17-1117.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Task Video Captioning with Video and&#xA;Entailment Generation&lt;/a&gt;, in ACL 2017&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Below is a screenshot of the results (positive and negative) reported in those papers:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/neTfw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/neTfw.png&quot; alt=&quot;Example from Venugopalan1&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qO1XT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qO1XT.png&quot; alt=&quot;Example from Pasunuru&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the ICCV paper, it's not hard to find some implementations, e.g. &lt;a href=&quot;https://github.com/vijayvee/video-captioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more recent results, I would suggest to look into the &lt;a href=&quot;http://activity-net.org/challenges/2017/captioning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ActivityNet 2017 Challenge - dense captioning&lt;/a&gt; or its &lt;a href=&quot;http://activity-net.org/challenges/2018/tasks/anet_captioning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2018 version&lt;/a&gt;. Some winning solutions include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1804.08274.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jointly Localizing and Describing Events for Dense Video Captioning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1806.08854.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RUC+CMU: System Report for Dense Captioning Events in Videos&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However I am not sure whether any open-source implementation has been released.&lt;/p&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-09-01T03:43:37.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7779" PostTypeId="1" CreationDate="2018-09-01T04:03:34.277" Score="3" ViewCount="233" Body="&lt;p&gt;I am trying to train a supervised model where the output from the model is output of a linear function &lt;span class=&quot;math-container&quot;&gt;$WX + b$&lt;/span&gt;. Kindly note that I'm not using any softmax or &lt;span class=&quot;math-container&quot;&gt;$\log$&lt;/span&gt; softmax on the result of the linear. I am using negative log-likelihood loss function, which takes the input as the linear output from the model and the true labels. I am getting decent accuracy by doing this, but I have read that the input to negative log-likelihood function must be probabilities. Am I doing something wrong?&lt;/p&gt;&#xA;" OwnerUserId="17372" LastEditorUserId="2444" LastEditDate="2019-04-29T14:58:32.337" LastActivityDate="2020-05-23T15:01:35.127" Title="Should the input to the negative log likelihood loss function be probabilities?" Tags="&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7780" PostTypeId="2" ParentId="3801" CreationDate="2018-09-01T04:19:58.553" Score="2" Body="&lt;p&gt;In short, dynamic computation graphs can solve some problems that static ones cannot, or are inefficient due to not allowing training in batches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, modern neural network training is usually done in &lt;strong&gt;batches&lt;/strong&gt;, i.e. processing more than one data instance at a time. Some researchers choose batch size like 32, 128 while others use batch size larger than 10,000. Single-instance training is usually very slow because it cannot benefit from hardware parallelism. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in Natural Language Processing, researchers want to train neural networks with sentences of different lengths. Using static computation graphs, they would usually have to first do &lt;strong&gt;padding&lt;/strong&gt;, i.e. adding meaningless symbols to the beginning or end of shorter sentences to make all sentences of the same length. This operation complicates the training a lot (e.g. need masking, re-define evaluation metrics, waste a significant amount of computation time on those padded symbols).  With a dynamic computation graph, padding is no longer needed (or only needed within each batch).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more complicated example would be to (use neural network to) process the sentences based on its parsing trees. Since each sentence has its own parsing tree, they each requires a different computation graph, which means training with a static computation graph can only allow single-instance training. An example similar to this is the &lt;a href=&quot;https://ai.stanford.edu/~ang/papers/icml11-ParsingWithRecursiveNeuralNetworks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recursive Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-09-01T04:19:58.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7781" PostTypeId="1" CreationDate="2018-09-01T11:32:05.430" Score="6" ViewCount="470" Body="&lt;p&gt;I am new to the field and I am trying to understand how is possible to use categorical variables / enums?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say we have a data set and 2 of its features are &lt;code&gt;home_team&lt;/code&gt; and &lt;code&gt;away_team&lt;/code&gt;, the possible values of these 2 features are all the NBA teams.&#xA;How can we &quot;normalize&quot; these features to be able to use them to create a deep network model (e.g. with tensorflow)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any reference to read about techniques of modeling that are also very appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17876" LastActivityDate="2018-09-01T14:16:20.243" Title="How to model categorical variables / enums?" Tags="&lt;deep-learning&gt;&lt;categorical-data&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7782" PostTypeId="2" ParentId="7779" CreationDate="2018-09-01T11:55:52.087" Score="1" Body="&lt;p&gt;This seems pretty reasonable to me. You can optimize for any function that is &lt;em&gt;proportionate&lt;/em&gt; to the negative log-likelihood. Conventionally, we assume that the likelihood of piece of data under a linear model is proportionate to some sort of gaussian function of the difference between the predicted value and the observed value. If you're a Bayesian you'd say this is a probability. If you're a hardcore frequentist, you might quibble about that, but it's still a number between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take the log of this likelihood function however, you'll get a quadratic function of the difference, with some scalers that you don't need to worry about. So you ought to minimize:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_y (y-\hat{y})^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not a probability, but since it is proportionate to the log of the original likelihood function (which was, in some sense), minimizing it will also minimize the original function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-01T11:55:52.087" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7783" PostTypeId="2" ParentId="7781" CreationDate="2018-09-01T12:14:09.567" Score="5" Body="&lt;p&gt;Authors use many different approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach is to have a different input neuron for each possible category, and then use a &quot;1-hot&quot; encoding. So if you have 10 categories, then you can encode this as 10 binary features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another is to use some sort of binary encoding. If you have 10 categories, it is sufficient to use 4 neurons to represent all possible categories by using binary numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third approach is to convert your categories to cardinal values, and then normalize them. This may be more effective if your categories really are cardinal (i.e. orderable). If there isn't a natural ordering to them though, this might lead to strange results or make the problem difficult to learn (since it ends up embedding non-linear relationships in the learning problem that don't need to exist).&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-01T12:14:09.567" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7784" PostTypeId="2" ParentId="7781" CreationDate="2018-09-01T14:16:20.243" Score="2" Body="&lt;p&gt;A one-hot encoding, as described in &lt;a href=&quot;https://ai.stackexchange.com/questions/7781/how-to-model-categorical-variables-enums/7783#7783&quot;&gt;John's answer&lt;/a&gt;, is probably the most straightforward / simple solution (maybe even the most common?). It is not without its problems though. For example, if you have a large number of such categorical variables, and each has a large number of possible values, the number of binary inputs you need for one-hot encodings may grow too large.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Lets say we have a data set and 2 of its features are home_team and away_team, the possible values of these 2 features are all the NBA teams.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this specific example, a different possible solution might be not to use the &quot;identity&quot; of a team as a feature itself, but try to find a number of (ideally numeric) features corresponding to that team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, instead of trying to encode &quot;home_team&quot; in some way in your inputs, you could (if you manage to find the data you need to do this) use the following features (not really familiar with NBA, so not sure if all these make sense):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Win percentage of home_team in recent X amount of time&lt;/li&gt;&#xA;&lt;li&gt;Historical win percentage of home_team against away_team&lt;/li&gt;&#xA;&lt;li&gt;Average points scored per match by this team&lt;/li&gt;&#xA;&lt;li&gt;In football there's something like how many minutes per game a team is &quot;in control&quot; of the ball, is there something similar in NBA maybe?&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And then you can try to get a similar list of features for the away_team. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of solution would work for your example, and maybe also for various other examples. It might not work in all cases of categorical features though, in some cases you'd have to revert to solutions like those in John's answer.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-01T14:16:20.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7785" PostTypeId="1" AcceptedAnswerId="7790" CreationDate="2018-09-01T19:40:51.840" Score="1" ViewCount="75" Body="&lt;p&gt;I have been researching LSTM neural networks.  I have seen &lt;a href=&quot;https://cdn-images-1.medium.com/max/1600/1*Niu_c_FhGtLuHjrStkB_4Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; diagram a lot and I have few questions about it. Firstly, is this diagram used for most LSTM neural networks?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, if it is, wouldn't only having single layers reduce it's usefulness?  &lt;/p&gt;&#xA;" OwnerUserId="17881" LastEditorUserId="1581" LastEditDate="2018-09-02T14:01:04.920" LastActivityDate="2018-09-02T14:01:04.920" Title="Need Help With LSTM Neural Networks" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7786" PostTypeId="2" ParentId="7785" CreationDate="2018-09-01T21:05:32.390" Score="1" Body="&lt;p&gt;(1) Yes this is the diagram for a classical LSTM unit. Of cause there are some variants and those diagrams would look slightly different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) It is very common for researchers to use more than one layers of LSTM and achieves better performance than a single layer one. A common way to &quot;stack&quot; LSTMs is to use the previous layer's output ($h_t$ in your diagram) as the input to the next layer ($x_t$). However, I have seldom seen any successful application of 5+ layers of LSTMs, while for CNNs it is common to use tens or even hundreds of layers.&lt;/p&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-09-01T21:05:32.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7787" PostTypeId="1" AcceptedAnswerId="7797" CreationDate="2018-09-02T08:24:21.083" Score="1" ViewCount="105" Body="&lt;p&gt;I am currently looking into LSTMs. I found &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this nice blog post&lt;/a&gt;, which is already very helpful, but still, there are things I don't understand, mostly because of the collapsed layers. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The input $X_t$, and the output of the previous time step $H_{t-1}$, how do they get combined? Multiplied, added or what? &lt;/li&gt;&#xA;&lt;li&gt;The input weights and the weights of the input of the previous time step, those are just the weights of the connections between the time-steps/units, right?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="2444" LastEditDate="2019-10-22T21:01:30.490" LastActivityDate="2019-10-22T21:01:30.490" Title="How do the current input and the output of the previous time step get combined in an LSTM?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7788" PostTypeId="1" AcceptedAnswerId="7798" CreationDate="2018-09-02T08:30:34.710" Score="10" ViewCount="1163" Body="&lt;p&gt;What benefits can we got by applying Graph Convolutional Neural Network instead of ordinary CNN? I mean if we can solve a problem by CNN, what is the reason should we convert to Graph Convolutional Neural Network to solve it? Are there any examples i.e. papers can show by replacing ordinary CNN with Graph Convolutional Neural Network, an accuracy increasement or a quality improvement or a performance gain is achieved? Can anyone introduce some examples as image classification, image recognition especially in medical imaging, bioinfomatics or biomedical areas?&lt;/p&gt;&#xA;" OwnerUserId="14948" LastEditorUserId="2444" LastEditDate="2019-03-12T10:37:03.703" LastActivityDate="2020-04-27T23:45:20.443" Title="What benefits can be got by applying Graph Convolutional Neural Network instead of ordinary CNN?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;graphs&gt;&lt;geometric-deep-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7790" PostTypeId="2" ParentId="7785" CreationDate="2018-09-02T09:02:51.330" Score="0" Body="&lt;p&gt;Just to be 100% sure - the diagram you refer to is a diagram of an LSTM CELL, not NETWORK. The operands you see on the diagram are operations within a cell, not separate &quot;neurons&quot;. I think it is quite obvious, however reading your questions I just wanted to be 100% sure we are on the same page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, about layers. RNN networks (LSTM in particular) are just like any other ANN structure. Theoretically, a 1-hidden layer network can do any computation of a &quot;deeper&quot; network. ANN is a universal approximate of math functions. Still, multi-layer ANNs typically work better on more complex problems. Multi-layer typically needs less total connections, learns better and is less resource-demanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, multi-layer LSTMs are believed to be better at determining complex in-time patterns. I think there is no rigorous proof for this, however.&#xA;Also, in practical applications - I did not see much improvement in network capabilities by adding additional LSTM layers. Adding more dense layers before/ after LSTM seemed to have a much better effect.&lt;/p&gt;&#xA;" OwnerUserId="15397" LastActivityDate="2018-09-02T09:02:51.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7792" PostTypeId="1" CreationDate="2018-09-02T10:00:35.260" Score="1" ViewCount="47" Body="&lt;p&gt;Is it possible to form a table that will have simply the shortest distance from each source to destination using q learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, suggest any other learning algorithm.&lt;/p&gt;&#xA;" OwnerUserId="17885" LastEditorUserId="2444" LastEditDate="2019-02-16T02:45:44.720" LastActivityDate="2019-02-16T02:45:44.720" Title="Can Q-learning be used to find the shortest distance from each source to destination?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7793" PostTypeId="1" AcceptedAnswerId="7796" CreationDate="2018-09-02T10:20:03.530" Score="5" ViewCount="82" Body="&lt;p&gt;Stuart Russell and Peter Norvig &lt;a href=&quot;https://people.eecs.berkeley.edu/~russell/intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pointed out&lt;/a&gt; 4 four possible goals to pursue in artificial intelligence: systems that think/act humanly/rationally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the differences between an agent that &lt;em&gt;thinks&lt;/em&gt; rationally and an agent that &lt;em&gt;acts&lt;/em&gt; rationally?&lt;/p&gt;&#xA;" OwnerUserId="17886" LastEditorUserId="2444" LastEditDate="2021-02-09T11:19:29.053" LastActivityDate="2021-02-09T12:47:14.083" Title="What are the differences between an agent that thinks rationally and an agent that acts rationally?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7794" PostTypeId="1" AcceptedAnswerId="7910" CreationDate="2018-09-02T11:25:26.373" Score="5" ViewCount="1950" Body="&lt;p&gt;I want to explore and experiment the ways in which I could use a neural network to identify patterns in text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Prices of XYZ stock went down at &lt;strong&gt;11:00&lt;/strong&gt; am &lt;strong&gt;today&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Retrieve a list of items exchanged on &lt;strong&gt;03/04/2018&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Show error logs between &lt;strong&gt;3 - 5 am&lt;/strong&gt; &lt;strong&gt;yesterday&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Reserve a flight for &lt;strong&gt;3rd October&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Do I have any meetings &lt;strong&gt;this Friday&lt;/strong&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Remind to me wake up early &lt;strong&gt;tue&lt;/strong&gt;, &lt;strong&gt;4th sept&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is for a project so I am not using regular expressions. Papers, projects, ideas are all welcome but I want to approach feature extraction/pattern detection to have a model trained which can Identify patterns that it has already seen.&lt;/p&gt;&#xA;" OwnerUserId="7998" LastEditorUserId="7998" LastEditDate="2018-09-02T11:58:14.083" LastActivityDate="2018-09-10T20:27:36.370" Title="How can I detect datetime patterns in text?" Tags="&lt;neural-networks&gt;&lt;pattern-recognition&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7795" PostTypeId="2" ParentId="7792" CreationDate="2018-09-02T14:05:09.663" Score="1" Body="&lt;p&gt;Welcome to AI.SE Adarsh!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is fairly simple to do with Q-Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you assign a reward equal to -1 for each location other than the goals, and 0 for the goals, then Q-learning with no discount factor will learn the length of the shortest path from each location to the goal. Alternatively, you can use a discounting factor, and then invert the equation on the learned values of the Q function to obtain the number of steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-02T14:05:09.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7796" PostTypeId="2" ParentId="7793" CreationDate="2018-09-02T14:11:55.653" Score="5" Body="&lt;p&gt;Maybe a good example to think about would be something like the &lt;a href=&quot;https://www.rug.nl/research/portal/files/13139017/2012_Keijzer_Sphex_story.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sphyx story&lt;/a&gt;. The wasp in the story appears to behave like a rational being: it seems to have a plan of action, it seems to be able to do advanced operations like counting, and it seems to execute the plan well. However, if you disrupt the wasp's plan, it becomes apparent that it is not &lt;em&gt;thinking&lt;/em&gt; rationally. Instead, it has evolved a very complex behaviour that appears to include rational components, but is still just an instinct.&lt;/p&gt;&#xA;&lt;p&gt;In the context of AI, consider a GOFAI system for planning like GraphPlan as opposed to a machine learning system for generating plans. While the former is a general-purpose algorithm for &lt;em&gt;reasoning&lt;/em&gt; about planning problems, the latter is an input/output mapping that may mimic reasoning, or may be more &amp;quot;instinctual&amp;quot;. Some AI'ers would say that the latter system is not really engaged in rational thought, while the former is. Both systems exhibit rational action, however.&lt;/p&gt;&#xA;&lt;p&gt;Instinctive systems often work very well, and I'd say that the increasing effectiveness of machine learning approaches since that edition of R&amp;amp;N was published (almost 10 years ago now?) makes this a fuzzier distinction in practice than the book might suggest.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-02-09T12:47:14.083" LastActivityDate="2021-02-09T12:47:14.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7797" PostTypeId="2" ParentId="7787" CreationDate="2018-09-02T18:30:59.620" Score="1" Body="&lt;p&gt;(1) $X_t$ and $H_{t-1}$ are concatenated. The blog you cited explained its notation &quot;&lt;em&gt;Lines merging denote concatenation&lt;/em&gt;&quot;. For example, if $X_t=[1,2,3]$ and $H_{t-1}=[4,5,6,7]$, then their concatenation is $[1,2,3,4,5,6,7]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) When you say &quot;&lt;em&gt;input weights&lt;/em&gt;&quot; or &quot;&lt;em&gt;weights of the input of the previous time step&lt;/em&gt;&quot;, are you referring to the $W_i$ in your cited blog? If so they are not &lt;em&gt;the weights of the connections between the time-steps/units&lt;/em&gt;. They are part of the input gate only. The &lt;em&gt;connections between the time-steps/units&lt;/em&gt; do not have weights applied to them.&lt;/p&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-09-02T18:30:59.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7798" PostTypeId="2" ParentId="7788" CreationDate="2018-09-02T18:47:32.463" Score="3" Body="&lt;p&gt;Generally speaking a graph CNN is applied to data represented by &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_(abstract_data_type)&quot; rel=&quot;nofollow noreferrer&quot;&gt;graphs&lt;/a&gt;, not images. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a &lt;strong&gt;graph&lt;/strong&gt; is a collection of nodes and edges connecting them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;an &lt;strong&gt;image&lt;/strong&gt; is a 2D or 3D matrix, in which each element denotes a pixel in space&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If your data are just images, or something similar (e.g. some fMRI data), you usually cannot benefit from graph CNN compared with usual CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes, the class labels of your images may be organized in a graph-like  (or tree-like) structure. In that case, you may have a chance to benefit from graph CNN.&lt;/p&gt;&#xA;" OwnerUserId="15493" LastEditorUserId="23503" LastEditDate="2020-04-27T23:45:20.443" LastActivityDate="2020-04-27T23:45:20.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7800" PostTypeId="2" ParentId="7788" CreationDate="2018-09-03T05:36:02.350" Score="2" Body="&lt;p&gt;Bioinformatics is an area that Graph Convolutional Neural Network is useful. Consider protein networks, or gene-gene networks. Surely, the biological networks can be represented as a graph. Now, you should see how GCN is useful for bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-09-03T05:36:02.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7803" PostTypeId="1" CreationDate="2018-09-03T09:56:39.463" Score="12" ViewCount="465" Body="&lt;p&gt;Hopfield networks are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimize the energy function when all neurons are set equal to the vector values, and retrieve the vector using the noisy version of it as input and allowing the net to settle to an energy minimum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leaving aside problems like the fact that there is no guarantee that the net will settle in the nearest minimum etc – problems eventually solved with Boltzmann machines and eventually with back-propagation – the breakthrough was they are a starting point for having abstract representations. Two versions of the same document would recall the same state, they would be represented, in the network, by the same state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Hopfield himself wrote in his 1982 paper &lt;a href=&quot;https://bi.snu.ac.kr/Courses/g-ai09-2/hopfield82.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural networks and physical systems with emergent collective computational abilities&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The present modeling might then be related to how an entity or Gestalt is remembered or categorized on the basis of inputs representing a collection of its features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On the other side, the breakthrough of deep learning was the ability to build multiple, hierarchical representations of the input, eventually leading to making AI-practitioners' life easier, simplifying feature engineering. (see e.g. &lt;a href=&quot;https://arxiv.org/pdf/1206.5538.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Representation Learning: A Review and New Perspectives&lt;/a&gt;, Bengio, Courville, Vincent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a conceptual point of view, I believe one can see deep learning as a generalization of Hopfield nets: from one single representation to a hierarchy of representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is that true from a computational/topological point of view as well? Not considering how &quot;simple&quot; Hopfield networks were (2-state neurons, undirected, energy function), can one see each layer of a network as a Hopfield network and the whole process as a sequential extraction of previously memorized Gestalt, and a reorganization of these Gestalt?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="17901" LastEditorUserId="2444" LastEditDate="2020-03-07T23:02:19.287" LastActivityDate="2020-04-07T01:01:55.377" Title="Can layers of deep neural networks be seen as Hopfield networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;topology&gt;&lt;hopfield-network&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7804" PostTypeId="1" CreationDate="2018-09-03T12:18:34.380" Score="0" ViewCount="174" Body="&lt;p&gt;I'm trying to have a go at building a neural net, but I can't seem to figure out how to optimise the connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to have a look online and it came up with &quot;backpropagation&quot;. I looked through some pages about it, but I can't seem to understand it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to be where you decide on a target value for each node of your output, and adjust the weights of the synapses to bring the values closer to their targets.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What about the inactive synapses &lt;sub&gt;(synapses giving a value of 0 because the previous neuron wasn't activated (its values didn't pass the threshold))&lt;/sub&gt;? Do those stay the same?&lt;/li&gt;&#xA;&lt;li&gt;How would this configure the hidden layers? Do I have to assign target values to them? How?&lt;/li&gt;&#xA;&lt;li&gt;What are the other ways that the connections can be adjusted? What alternatives are there to using target values?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16355" LastEditorUserId="16355" LastEditDate="2018-09-03T15:23:25.083" LastActivityDate="2018-09-03T21:05:50.887" Title="How do I change the values of a neural net" Tags="&lt;neural-networks&gt;&lt;ai-basics&gt;&lt;python&gt;&lt;backpropagation&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2019-11-18T07:28:29.283" ContentLicense="CC BY-SA 4.0" />
  <row Id="7806" PostTypeId="2" ParentId="7804" CreationDate="2018-09-03T15:33:51.923" Score="3" Body="&lt;p&gt;This is too broad a topic to answer directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are at the beginner stage with neural networks, you will need to learn some basic theory of the maths of neural networks, before the code will make sense. Although it is possible to write neural network code with only a vague understanding of what is going on, it is not a great way to learn for the future, and more advanced neural network features will likely remain beyond your comprehension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maths for back propagation are not that difficult conceptually, it is literally just the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chain Rule&lt;/a&gt; from basic calculus applied repeatedly. You do this to get a gradient that tells you the direction that an error function would increase in, then take a small step in the opposite direction. Repeat this over time and the error should reduce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the simplicity &lt;strong&gt;once you already know it&lt;/strong&gt;, there are a lot of moving parts to training a neural network. The formulae for back propagation include multiple symbols with different meanings, and typically indexed in at least 3 dimensions all at once. It can look like a wall of impenetrable maths, especially if you have got a bit rusty at basic calculus and matrix multiplication, and need to review it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is to take your time and study the basics carefully. There are many resources out there. Introductory material should cover:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Revision of basic linear algebra. Just vectors, matrices and how to add and multiply them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Revision of basic calculus. Just differentiation and gradients. The chain rule is handy for later.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Linear regression and logistic regression models trained using gradient descent. These are introductory models that naturally lead to layered neural networks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural networks and back propagation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are many courses and books that attempt to teach you these things. It is hard to say in general which one would work best for a particular student. The best thing to do would be to go to a learning resource that you already like, such as Coursera or EDX, and search for &quot;neural networks beginner&quot; or similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a couple of courses that I can personally recommend. I cannot create a comprehensive list, so if these do not apply on a quick browse (or if the links finally go stale after a few years), then you should search for something more suitable bearing in mind the syllabus suggestions above:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Andrew Ngs' &lt;a href=&quot;https://www.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt; covers more than just neural networks, but the extra parts apply to other aspects of ML, such as sensible approaches to validating and testing that anyone building predictive models should get to know. Code exercises are in Matlab or Octave.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Andrew Ngs' &lt;a href=&quot;https://www.coursera.org/learn/intro-to-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Deep Learning&lt;/a&gt; starts with basic regression, and moves very quickly to practical builds of neural networks using TensorFlow. It is part of a five course &quot;Specialisation&quot; that works up to advanced NN models developed in the last three years.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Geoffrey Hinton's &lt;a href=&quot;https://www.coursera.org/learn/neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks for Machine Learning&lt;/a&gt; covers some historical models (such as perceptrons) and some more obscure models, and the material is &lt;em&gt;hard&lt;/em&gt; to understand, but quite rewarding if you are up to it. Not generally recommended to beginners, but if you have solid maths and computing knowledge already, at degree level, but never looked at a neural network before, this might be the right style of course for you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;They all happen to be Coursera courses . . . I am not affiliated, it is just that the relevant courses that I have personally taken have been on that platform.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-09-03T15:33:51.923" CommentCount="0" CommunityOwnedDate="2018-09-03T15:33:51.923" ContentLicense="CC BY-SA 4.0" />
  <row Id="7810" PostTypeId="2" ParentId="7804" CreationDate="2018-09-03T21:05:50.887" Score="2" Body="&lt;p&gt;Just thought I'd give an answer myself; covering the 3&lt;sup&gt;rd&lt;/sup&gt; point.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;What are the other ways that the connections can be adjusted? What alternatives are there to using target values?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I took a look on YouTube, and found &lt;a href=&quot;https://www.youtube.com/watch?v=VnwjxityDLQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=VnwjxityDLQ&lt;/a&gt; &lt;br&gt;&lt;sup&gt;It's a only 5 minute video. The videos I was getting when I searched &quot;Andrew​ Ng's ML course&quot; in YouTube were over an hour.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It said that the values could be optimised by creating a large number of copies of a network that have different random starting weights. Make the networks perform their task, and then identify the neurons that did the best, create copies of the neural nets that did the best, and then delete the nets that did the worst. After that, randomly alter the weights of the copies slightly. You can then repeat this process until the values are optimised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be a good solution in that it doesn't require target values; it only needs to know how well it did. However, this method seems like it would require more RAM, and may be unsuitable for some situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would still be helpful to know the other methods of adjusting the weights.&lt;/p&gt;&#xA;" OwnerUserId="16355" LastActivityDate="2018-09-03T21:05:50.887" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7812" PostTypeId="2" ParentId="7541" CreationDate="2018-09-04T06:20:07.363" Score="3" Body="&lt;p&gt;&lt;strong&gt;Driving Priorities&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When considering the kind of modeling needed to create reliable and safe autonomous vehicles, the following driving safety and efficacy criteria should be considered, listed in priority with the most important first.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The safety of those inside the vehicle and outside the vehicle&lt;/li&gt;&#xA;&lt;li&gt;Reduction of wear on passengers&lt;/li&gt;&#xA;&lt;li&gt;The safety of property&lt;/li&gt;&#xA;&lt;li&gt;The arrival at the given destination&lt;/li&gt;&#xA;&lt;li&gt;Reduction of wear on the vehicle&lt;/li&gt;&#xA;&lt;li&gt;Thrift in fuel resources&lt;/li&gt;&#xA;&lt;li&gt;Fairness to other vehicles&lt;/li&gt;&#xA;&lt;li&gt;The thrift in time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are ordered in a way that makes civic and global sense, but they are not the priorities exhibited by human drivers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Copy Humans or Reevaluate and Design from Scratch?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whoever said that the goal of autonomous car design is to model the portions of a human mind that can drive should not be designing autonomous cars for actual manufacture.  It is well known that most humans, although they may have heard of the following safety tips, cannot bring them into consciousness with sufficient speed to benefit from them in actual driving arrangements.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When the tires slip sideways, steer into the skid.&lt;/li&gt;&#xA;&lt;li&gt;When a forward skid starts, pump the breaks.&lt;/li&gt;&#xA;&lt;li&gt;If someone is headed tangentially into your car's rear, immediately accelerate and then break.&lt;/li&gt;&#xA;&lt;li&gt;On an on ramp, accelerate to match the speed of the cars in the lane into which you merge, unless there is no space to merge.&lt;/li&gt;&#xA;&lt;li&gt;If you see a patch of ice, steer straight and neither accelerate nor decelerate once you reach it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many collisions between locomotives and cars are because a red light causes a line in multiple lanes across the tracks.  Frequently, a person will move onto the railroad tracks to gain one car's length on the other cars.  When others move to make undoing that choice problematic, a serious risk emerges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As absurd as this behavior is to anyone watching, many deaths occur as a fast traveling 2,000 ton locomotive hits what feels like a dust speck to the train passengers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Predictability and Adaptability&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans are unpredictable, as the question indicates, but although adaptability may be unpredictable, unpredictability may not be adaptive.  It is adaptability that is needed, and it is needed in five main ways.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Adaptive in the moment to surprises&lt;/li&gt;&#xA;&lt;li&gt;Adaptive through general driving experience&lt;/li&gt;&#xA;&lt;li&gt;Adaptive to the specific car&lt;/li&gt;&#xA;&lt;li&gt;Adaptive to passenger expression&lt;/li&gt;&#xA;&lt;li&gt;Adaptive to particular map regions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition, driving a car is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Highly mechanical,&lt;/li&gt;&#xA;&lt;li&gt;Visual,&lt;/li&gt;&#xA;&lt;li&gt;Auditory,&lt;/li&gt;&#xA;&lt;li&gt;Plan oriented&lt;/li&gt;&#xA;&lt;li&gt;Geographical, and&lt;/li&gt;&#xA;&lt;li&gt;Preemptive in surprise situations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Modelling Driving Complexities&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This requires a model or models comprise of several kinds of objects.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maps&lt;/li&gt;&#xA;&lt;li&gt;The vehicle&lt;/li&gt;&#xA;&lt;li&gt;The passenger intentions&lt;/li&gt;&#xA;&lt;li&gt;Other vehicle&lt;/li&gt;&#xA;&lt;li&gt;Other obstructions&lt;/li&gt;&#xA;&lt;li&gt;Pedestrians&lt;/li&gt;&#xA;&lt;li&gt;Animals&lt;/li&gt;&#xA;&lt;li&gt;Crossings&lt;/li&gt;&#xA;&lt;li&gt;Traffic signals&lt;/li&gt;&#xA;&lt;li&gt;Road signs&lt;/li&gt;&#xA;&lt;li&gt;Road side&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neither Mystery nor Indeterminance&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although these models are cognitively approximated in the human brain, how well they are modeled and how effective those models are at reaching something close to a reasonable balance of the above priorities varies from driver to driver, and varies from trip to trip for the same driver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as complex as driving is, it is not mysterious.  Each of the above models are easy to consider at a high level in terms of how they interact and what mechanical and probabilistic properties they have.  Detailing these is an enormous task, and making the system work reliably is a significant engineering challenge, in addition to the training question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Inevitability of Achievement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of the complexity, because of the economics involved and the fact that it is largely a problem of mechanics, probability, and pattern recognition, it will be done, and it will eventually be done well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it is, as unlikely as this sounds to the person who accepts our current culture as permanent, human driving may become illegal in this century in some jurisdictions.  Any traffic analyst can mount heaps of evidence that most humans are ill equipped to drive a machine that weighs a ton at common speeds.  The licensing of unprofessional drivers has only become widely accepted because of public insistence on transportation convenience and comfort and because the workforce economy requires it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Autonomous cars may reflect the best of human capabilities, but they will likely far surpass them because, although the objects in the model are complex, they are largely predictable, with the notable exception of children playing.  AV technology will use the standard solution for this.  The entire scenario can be brought into slow motion to adapt for children playing simply by slowing way down.  AI components that specifically detect children and dogs are likely to emerge soon, if they do not already exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Randomness&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Randomness is important in training.  For instance, a race car driver will deliberately create skids of various types to get used to how to control them. In machine learning we see some pseudo random perturbations introduced during training to ensure that the gradient descent process does not get caught in a local minimum but rather is more likely to find a global minimum (optimum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deadlock&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is correct in stating that, &quot;A dose of unpredictability could have its uses.&quot;  The deadlock scenario is an interesting one, but is unlikely to occur as standards develop.  When four drivers come to a stop sign at the same time, they really don't.  It only seems like they did.  The likelihood that none of them arrived more than a millisecond before the others is astronomically small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People will not detect (or even be honest enough) to distinguish these small time differences, so it usually comes to who is most gracious to wave the others on, and there can be some deadlock there too, which can become comical, especially since all of them really wish to get moving.  Autonomous vehicles will extremely rarely encounter a deadlock that is not covered by the rule book the government licensing entity publishes, which can be programmed as driving rules into the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On those rare occasions, the vehicles could digitally draw lots, as suggested, which is one place where unpredictability is adaptive.  Doing skid experimentation like a race car driver on Main Street at midnight may be what some drunk teen might do, but that is a form of unpredictability that is not adaptive toward a sensible ordering of the priorities of driving.  Neither would be texting or trying to eat and drive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Determinism&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding determinism, in the context of the uses discussed, pseudo-random number generation of particular distributions will suffice.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Deadlock release or&lt;/li&gt;&#xA;&lt;li&gt;Training speed-ups and improved reliability when there are local minima that are not the global minimum during optimization,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Functional tests and unit testing technologies are not only able to handle the testing of components with pseudo-randomness, but they sometimes employ pseudo-randomness to provide better testing coverage.  The key to doing this well is understanding of probability and statistics, and some engineers and AI designers understand it well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Element of Surprise&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where randomness is most important in AV technology is not in the decision making but in the surprises.  That is the bleeding edge of that engineering work today.  How can one drive safely when a completely new scenario appears in the audio or visual channels?  This is perhaps the place where the diversity of human thought may be best adept, but at highway speeds, it is usually too slow to react in the way we see in movie chase scenes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Correlation Between Risk and Speed&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings up an interesting interaction of risk factors.  It is assumed that higher speeds are more dangerous, the actual mechanics and probability are not that clear cut.  Low speeds produce temporally longer trips and higher traffic densities.  Some forms of accidents are less likely at higher speeds, specifically ones that are related mostly to either traffic density or happenstance.  Other forms are more likely at higher speeds, specifically ones that are related to reaction time and tire friction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With autonomous vehicles, tire slippage may be more accurately modeled and reaction time may be orders of magnitude faster, so minimum speed limits may be more imposed and upper limits may increase once we get humans out of the driver's seats.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-04T06:20:07.363" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7815" PostTypeId="1" CreationDate="2018-09-04T08:34:37.693" Score="1" ViewCount="81" Body="&lt;p&gt;So as my university project I am planning to make a prediction system as described in the title. My current idea is to use the age/gender classifier and run it on a video(taken in front of a shop) which outputs a csv file of the age/gender/Customer ID. In addition, I will use the existing data of the shop of who came in/who didn't come into the shop but passed by the shop and by running XGBoost on this csv data I can predict which customer will come into the shop or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think this idea is possible? Is there any other way to implement this idea. It would also be great if we could implement this in such a way as to make the deep learning model learn the various features of those who come into the shop or not. &lt;/p&gt;&#xA;" OwnerUserId="17651" LastActivityDate="2018-09-04T11:11:50.793" Title="Automatic prediction of whether a customer will come into the shop or not" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7816" PostTypeId="2" ParentId="7815" CreationDate="2018-09-04T11:11:50.793" Score="1" Body="&lt;p&gt;This should be possible, but it's not completely clear what you are trying to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're trying to predict customer age and gender from a video, then you've got a computer vision problem. Deep learning methods are the state of the art for this, and probably some sort of &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolutional deep neural network&lt;/a&gt; is your friend. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're trying to predict whether the customer will enter the shop from their age and gender, then @DuttA suggestion to start with simple classification techniques is probably the best bet. Try out &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;logistic regression&lt;/a&gt; as a starting point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to do both of these things at once, you can again try out a deep convolutional neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that said, it's not completely clear that there is signal in the videos you want to collect: it might be that predicting whether they'll enter the shop is not really possible on the basis of a short video. The only way to know for sure is to give it a try however.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-04T11:11:50.793" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7817" PostTypeId="1" AcceptedAnswerId="7818" CreationDate="2018-09-04T14:55:46.377" Score="3" ViewCount="169" Body="&lt;p&gt;I came across RNN's a few minutes ago, which might solve a problem with sequenced data I've had for a while now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a set of input features, generated every second. Corresponding with these input features, is an output feature (also available every second). &#xA;One set of input features does not carry enough data to correlate with the output feature, but a sequence of them most definitely does. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that RNN's can have node connections along sequences of inputs, which is exactly what I need, but almost all implementations/explanations show prediction of the next word or number in a text-sentence or in a sequence of numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They predict what would be the next input value, the one that completes the sequence. However, in my case, the output feature will only be available during training. During inference, it will only have the input features available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use RNN in this case? Can it also predict features that are not part of the input features?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="16932" LastActivityDate="2018-09-04T15:30:24.287" Title="Is it possible to use an RNN to predict a feature that is not an input feature?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7818" PostTypeId="2" ParentId="7817" CreationDate="2018-09-04T15:24:58.363" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to use RNN in this case? Can it also predict features that are not part of the input features?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No changes are required to a RNN in order to do this. All you need is correctly labelled data mapping a sequence of $x$ to correct $y$ in order to train, and of course a RNN architecture which has input vectors matching shape of $x$ and output vectors matching shape of $y$. The case where $x$ and $y$ are the same data type is just a special case of RNN design, and not a requirement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may need to consider some details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the relationship between $x$ and $y$ is complex and non-linear even accounting for accumulated hidden state during the sequence, you may need to add deeper layers. The output of the LSTM can be some vector $h$ and you can add fully-connected layers to help with predicting $y$ from $h$. This, or adding more LSTM layers, is a choice of hyperparameter that you may want to experiment with. Start with a basic LSTM to see how that goes first. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you wish to predict a &lt;em&gt;sequence&lt;/em&gt; of output features that is either not the same length as the input feature sequence, or logically should come after the whole sequence (think language translation) then this may need a slight change in setup to get best results. For a predict-same-kind sequence you can feed your predicted output value into the next input, but if input and output have different data types, this will not work. Instead, you will need to have some dummy input or other setup for creating sequences of $y$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In your specific case the second point does not seem to apply, as you want to predict a single $y$ immediately after a sequence of $x$.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-04T15:30:24.287" LastActivityDate="2018-09-04T15:30:24.287" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7819" PostTypeId="1" CreationDate="2018-09-04T15:38:44.367" Score="3" ViewCount="284" Body="&lt;p&gt;I am not sure if I can use the words &lt;em&gt;binomial&lt;/em&gt; and &lt;em&gt;binary&lt;/em&gt; and &lt;em&gt;boolean&lt;/em&gt; as synonyms to describe a data attribute of a data set which has two values (yes or no). Are there any differences in the meaning on a deeper level?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if I have an attribute with three possible values (yes, no, unknown), this would be an attribute of type polynominal. What further names are also available for this type of attribute? Are they termed as &quot;symbolic&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in the realtion between the following attribute type: binary, boolean, binominal, polynominal (and alternative describtions) and nominal. &lt;/p&gt;&#xA;" OwnerUserId="13295" LastEditorUserId="6014" LastEditDate="2018-09-04T23:22:43.533" LastActivityDate="2018-09-05T00:11:45.330" Title="Is a binary attribute type the same as binomial attribute type?" Tags="&lt;datasets&gt;&lt;structured-data&gt;&lt;categorical-data&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7822" PostTypeId="5" CreationDate="2018-09-04T20:04:47.150" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The term neuromorphic has been used to describe analog, digital, mixed-mode analog/digital VLSI, and software systems that implement models of neural systems (for perception, motor control, or multisensory integration). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A key aspect of neuromorphic engineering is understanding how the morphology of individual neurons, circuits, applications, and overall architectures creates desirable computations, affects how information is represented, influences robustness to damage, incorporates learning and development, adapts to local change (plasticity), and facilitates evolutionary change.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Neuromorphic engineering is an interdisciplinary subject that takes inspiration from biology, physics, mathematics, computer science, and electronic engineering to design artificial neural systems, such as vision systems, head-eye systems, auditory processors, and autonomous robots, whose physical architecture and design principles are based on those of biological nervous systems&#xA;  &lt;br&gt;&lt;br&gt;&lt;sub&gt;Excerpted from &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuromorphic engineering (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See Also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1705.06963.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1705.06963.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/nature23011&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/nature23011&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://spectrum.ieee.org/semiconductors/design/neuromorphic-chips-are-destined-for-deep-learningor-obscurity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spectrum.ieee.org/semiconductors/design/neuromorphic-chips-are-destined-for-deep-learningor-obscurity&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://science.energy.gov/~/media/bes/pdf/reports/2016/NCFMtSA_rpt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://science.energy.gov/~/media/bes/pdf/reports/2016/NCFMtSA_rpt.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nist.gov/news-events/events/2017/11/introduction-neuromorphic-computing-why-it-so-efficient-pattern&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nist.gov/news-events/events/2017/11/introduction-neuromorphic-computing-why-it-so-efficient-pattern&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.humanbrainproject.eu/en/silicon-brains/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.humanbrainproject.eu/en/silicon-brains/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/group/brainsinsilicon/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/group/brainsinsilicon/about.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/526506/neuromorphic-chips/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.technologyreview.com/s/526506/neuromorphic-chips/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.forbes.com/sites/adrianbridgwater/2018/01/03/neuromorphic-computing-will-build-human-like-machine-brains/#3c47c0d37124&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.forbes.com/sites/adrianbridgwater/2018/01/03/neuromorphic-computing-will-build-human-like-machine-brains/#3c47c0d37124&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wired.co.uk/article/ai-neuromorphic-chips-brains&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wired.co.uk/article/ai-neuromorphic-chips-brains&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-09-04T20:04:47.150" LastActivityDate="2018-09-04T20:04:47.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7823" PostTypeId="4" CreationDate="2018-09-04T20:04:47.150" Score="0" Body="Use for questions related to use of electronic circuits to mimic neuro-biological architectures present in the nervous system or in neural networks. (&quot;Neuromorphic computing&quot; is sometimes used synonomouosly)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-09-04T20:04:47.150" LastActivityDate="2018-09-04T20:04:47.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7825" PostTypeId="2" ParentId="7819" CreationDate="2018-09-04T23:25:34.397" Score="0" Body="&lt;p&gt;Binomial is a &lt;strong&gt;distribution&lt;/strong&gt; characterised by $p$, the probability of success for an independent trial. Each sample you get from the distribution is a &lt;strong&gt;binary&lt;/strong&gt; variable, 0 or 1.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-09-04T23:25:34.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7826" PostTypeId="2" ParentId="7819" CreationDate="2018-09-05T00:11:45.330" Score="1" Body="&lt;p&gt;@SmallChess's answer is a good start, but there are some additional parts to the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;binary&lt;/em&gt;&lt;/strong&gt; variables or &lt;em&gt;binary&lt;/em&gt; data consist of data with the values 0 or 1, and no other values. We usually don't talk about &quot;binary distributions&quot;, because it's only data, variables, or outcomes that can be binary. A distribution might produce binary data, but is not itself binary because its parameters typically take on real-values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;&lt;em&gt;binomial&lt;/em&gt;&lt;/strong&gt; distribution is a distribution that produces binary data. In particular, it is a random process that produces the value 1 with probability $$p$$, and the value 0 with probability $$1-p$$. Notice that although it makes binary data, it is not itself a kind of data, and is in fact charactorized by a non-binary number (p).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Boolean&lt;/em&gt;&lt;/strong&gt; data takes on the values &lt;em&gt;true&lt;/em&gt; or &lt;em&gt;false&lt;/em&gt;. Often, but not always, these are stored as 0's and 1's. The distinction is that boolean data may not be stored numerically. There might also be different expectations about how Boolean data should be processed (for instance, $$true + true = true$$, but $1 + 1 = 2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not aware of the term &lt;strong&gt;&lt;em&gt;polynomial&lt;/em&gt;&lt;/strong&gt; being applied to data. However, &lt;strong&gt;&lt;em&gt;multinomial&lt;/em&gt;&lt;/strong&gt; distributions are probability distributions that produce 0 with probability $p_0$, 1 with probability $p_1$, 2 with probability $p_2$, and so on, producing $p_n$ with probability $1 - \sum_{i=0}^{n-1} p_i$ for $n$ different numbers. Like binomial distributions, multinomial distributions are characterized by a set of real-valued numbers, and are distinct from the kind of data they generate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Categorical&lt;/em&gt;&lt;/strong&gt; data takes on values from a set of categories. The example you give (yes, no, maybe) is not strictly multinomial data, but could be generated from a multinomial distribution by mapping the values 0, 1 and 2 onto yes, no and maybe. Note again that categorical data might be non-numeric. Operations like adding might be non-sensical. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Cardinal&lt;/em&gt;&lt;/strong&gt; data isn't something you asked about, but arises when data can be nicely ordered. For example, playing cards are easily mapped to the numbers 1-13, and can have reasonable semantic meaning when represented this way (e.g. A + 2 = 3, and 1 + 2 = 3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Nominal Data&lt;/em&gt;&lt;/strong&gt; is just literal numbers that mean exactly what they purport to mean. For example, if you store the number of cans of beer a customer purchased, that would be nominal data.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-05T00:11:45.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7827" PostTypeId="2" ParentId="6052" CreationDate="2018-09-05T08:18:14.853" Score="1" Body="&lt;p&gt;For what its worth (and having done a bit of study on this and being really interested in the topic):  the answer seems to go back to the beginnings of AI and even earlier (Turing's 1936 paper in which he introduces what's now called the Turing machine). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;John McCarthy's filer for the 1956 Dartmouth College summer workshop on &quot;Artificial Intelligence&quot; (which name introduced the term &quot;Artificial Intelligence&quot;) in part says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The study [workshop] is to proceed on the basis of the conjecture &#xA;  that every aspect of learning or any other feature of intelligence can &#xA;  in principle be so precisely described that a machine can be made to &#xA;  simulate it.&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This references Turing's 1936 paper where a machine or natural system is described, and the description is run in a computer. To simulate is to quite precisely describe a system then run the description (transformed a bit – but the result is still a description) in a computer. The description is the program. The description needs to be precise, as indicated in the Church-Turing thesis.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the idea of simulation is core to the computational theory of what a digital computer can or might do. So it's also core to the computational theory of mind (the organic brain being a natural system), and hence to AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it's obviously a crazy idea to try to quite precisely describe the organic machine that is a human brain. I mean how many neurons? 100 billion. Quite precisely describe each and every single one of these, &lt;em&gt;and&lt;/em&gt; each and every of the up to 10,000 connections that connect to each and every single neuron. Crazy with a capital C. And to suppose there are degrees of simulation of the brain, or that the mind is somehow a simplification of the brain, or that the description can be in higher level concepts, not neurological ones, is just to admit that the description is not quite precise. An adequate simulation of a brain would be terribly detailed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why do we hear so much about AI trying to simulate the brain? Answer: AI has no other word to express what it does. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my view, AI ought to be trying to work out the data-processing principles of the organic brain, not trying to describe the causation of the brain. AI doesn't know the principles of perception or the principles of general knowledge. It's incredible to say this – seeing as both are so absolutely fundamental to human intelligence. But AI doesn't know the principles. It ought to be trying to work them out. Then – once discovered – to work out how these principles could be realised in a computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You suggest that there's a binary choice between AI trying to get a computer to simulate the organic brain, and trying to grow organic brains in a dish. But there's actually a third option. Computer can do things other than simulate (i.e., other than compute). Maybe these other things might include embodying the principles of organic brains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two really big areas here: (1) what are the principles of intelligence? (2) what are the non-computational things computers can do? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You ask why AI is concerned with the digital environment rather than, say, growing organic brains in a vat. But AI is basically an engineering project (building something with a designed causality) and even though AI knows only a little about the causality of what it's trying to build, the digital computer seems to be the only viable platform, at present, with enough individually addressable memory locations and processor speed to cope with semantic structures that would result from an adequate sensory interaction with the environment.  &lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2018-09-05T08:18:14.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7829" PostTypeId="1" CreationDate="2018-09-05T09:13:54.887" Score="2" ViewCount="132" Body="&lt;p&gt;What is the process for integrating sentiment analysis in a CRM? What I am searching for is a system which analyzes the customer comments or reviews using the CRM and finds out the customer sentiment on the services provided by the system or company or a product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done a sentiment analyzer which takes text and shows the sentiment of the text. Now I want to integrate the above-mentioned sentiment analyzer to a CRM, how can I do that?&lt;/p&gt;&#xA;" OwnerUserId="17058" LastEditorUserId="17058" LastEditDate="2018-09-05T10:33:45.283" LastActivityDate="2021-04-20T14:47:45.900" Title="Integration of Sentiment analysis in CRM" Tags="&lt;machine-learning&gt;&lt;sentiment-analysis&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7830" PostTypeId="1" CreationDate="2018-09-05T14:06:10.640" Score="1" ViewCount="136" Body="&lt;p&gt;I am interested in the field of artificial intelligence. I began by learning the various machine learning algorithms. The maths behind some were quite hard. For example, back-propagation in convolutional neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then when getting to the implementation part, I learnt about TensorFlow, Keras, PyTorch, etc. If these provide much faster and more robust results,  will there be a necessity to code a neural network (say) from scratch using the knowledge of the maths behind back-prop, activation functions, dimensions of layers, etc., or is the role of a data scientist only to tune the hyper-parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, as of now the field of AI does not seem to have any way to solve for these hyperparameters, and they are arrived at through trial and error. Which begs the question, can a person with just basic intuition about what the algorithms do be able to make a model just as good as a person who knows the detailed mathematics of these algorithms?&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="2444" LastEditDate="2020-04-01T15:27:51.010" LastActivityDate="2020-04-02T00:44:56.647" Title="Is it necessary to know the details behind the AI algorithms and models?" Tags="&lt;machine-learning&gt;&lt;data-science&gt;&lt;academia&gt;&lt;education&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7832" PostTypeId="1" AcceptedAnswerId="7871" CreationDate="2018-09-05T15:42:46.143" Score="4" ViewCount="380" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;''Proximal Policy Optimization Algorithms'' , Schulman et al. (2017)&lt;/a&gt;, page 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why the clipped surrogate objective works.&#xA;As written in the article : &quot;With this scheme, we only ignore the change in probability ratio when it would make the objective improved, and we include it when it makes the objective worse&quot;.&#xA;I feel confused : how can it works if it doesn't take account of objective improvements?&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2018-09-05T17:47:40.007" LastActivityDate="2020-04-14T16:31:07.403" Title="Why does Clipped Surrogate Objective works in Proximal Policy Optimization" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7833" PostTypeId="2" ParentId="7794" CreationDate="2018-09-05T15:59:43.343" Score="1" Body="&lt;p&gt;If want to use deep learning approaches, you should look to recurrent neural networks (RNN). Recurrent networks will take into account temporal dependencies and could detect thatn &lt;em&gt;this&lt;/em&gt; in &lt;em&gt;this Friday&lt;/em&gt; belong to datetime but not in &lt;em&gt;this apple&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple model, you could create a model with a bidirectional LSTM layer (a type of RNN):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Input&lt;/strong&gt;: the sequences of characters.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Output&lt;/strong&gt;: whether the character belongs to datetime or not.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The longest part will gather many sentences with its corresponding solution to create a training/testing dataset. &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras&lt;/a&gt; might be a good framework to start playing around and with many examples. &lt;/p&gt;&#xA;" OwnerUserId="17936" LastActivityDate="2018-09-05T15:59:43.343" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7836" PostTypeId="1" CreationDate="2018-09-05T22:45:53.030" Score="2" ViewCount="395" Body="&lt;p&gt;I'm trying to solve the OpenAI BipedalWalker-v2 by using a one-step actor-critic agent. I'm implementing the solution using python and tensorflow.&lt;/p&gt;&#xA;&lt;p&gt;I'm following this pseudo-code taken from the book &lt;em&gt;Reinforcement Learning An Introduction by Richard S. Sutton and Andrew G. Barto.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ysx0Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ysx0Y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;in summary, my question can be reduced to the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it a good idea to implement a one-step actor-critic algorithm to solve the OpenAI BipedalWalker-v2 problem? If not what would be a good approach? If yes; how long would it take to converge?&lt;/li&gt;&#xA;&lt;li&gt;I run the algorithm for 20000 episodes, each episode has an avg of 400 steps, for each step, I immediately update the weights. The results are not better than random. I have tried different standard deviations (for my normal distribution that represents pi), different NN sizes for the Critic and Actor, and different learning-steps for the optimizer algorithm. The results never improve. I don't know what I'm doing wrong.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1&gt;My Agent Class&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;import numpy as np&#xA;import gym&#xA;import matplotlib.pyplot as plt&#xA;&#xA;class agent_episodic_continuous_action():&#xA;    def __init__(self, lr,gamma,sample_variance, s_size,a_size,dist_type):&#xA;       ... #agent parameters&#xA;&#xA;    def save_model(self,path,sess):    &#xA;    def load_model(self,path,sess):       &#xA;    def weights_init_actor(self,hidd_layer,mean,stddev): #to have control over the weights initialization      &#xA;    def weights_init_critic(self,hidd_layer,mean,stddev):  #to have control over the weights initialization            &#xA;    def create_actor_brain(self,hidd_layer,hidd_act_fn,output_act_fn,mean,stddev):  #actor is represented by a fully connected NN      &#xA;    def create_critic_brain(self,hidd_layer,hidd_act_fn,output_act_fn,mean,stddev): #critic is represented by a fully connected NN      &#xA;    def critic(self):            &#xA;    def get_delta(self,sess):                 &#xA;    def normal_dist_prob(self): #Actor pi distribution is a normal distribution whose mean comes from the NN &#xA;    def create_actor_loss(self): &#xA;    def create_critic_loss(self):&#xA;    def sample_action(self,sess,state): #Sample actions from the normal dist. Whose mean was aprox. By the NN&#xA;    def calculate_actor_loss_gradient(self):&#xA;    def calculate_critic_loss_gradient(self):   &#xA;    def update_actor_weights(self):&#xA;    def update_critic_weights(self):&#xA;    def update_I(self):  &#xA;    def reset_I(self):      &#xA;    def update_time_step_info(self,s,a,r,s1,d):  &#xA;    def create_graph_connections(self):&#xA;    def bound_actions(self,sess,state,lower_limit,uper_limit):  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Agent instantiation&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;tf.reset_default_graph()&#xA;agent= agent_episodic_continuous_action(learning-step=1e-3,gamma=0.99,pi_stddev=0.02,s_size=24,a_size=4,dist_type=&amp;quot;normal&amp;quot;)&#xA;agent.create_actor_brain(hidden_layers=[12,5],hidden_layers_fct=&amp;quot;relu&amp;quot;,output_layer=&amp;quot;linear&amp;quot;,mean=0.0,stddev=0.14)&#xA;agent.create_critic_brain(hidden_layers=[12,5],hidden_layers_fct=&amp;quot;relu&amp;quot;,output_layer=&amp;quot;linear&amp;quot;,mean=0.0,stddev=0.14)&#xA;agent.create_graph_connections()&#xA;&#xA;path = &amp;quot;/home/diego/Desktop/Study/RL/projects/models/biped/model.ckt&amp;quot;   &#xA;env = gym.make('BipedalWalker-v2')&#xA;uper_action_limit = env.action_space.high&#xA;lower_action_limit = env.action_space.low   &#xA;total_returns=[]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Training loops&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;with tf.Session() as sess:&#xA;    try:&#xA;        sess.run(agent.init)&#xA;        sess.graph.finalize()&#xA;        #agent.load_model(path,sess)        &#xA;        for i in range(1000): &#xA;            agent.reset_I()&#xA;            s = env.reset()    &#xA;            d = False&#xA;            while (not d):&#xA;                a=agent.bound_actions(sess,s,lower_action_limit,uper_action_limit)  &#xA;                s1,r,d,_ = env.step(a)&#xA;                #env.render()&#xA;                agent.update_time_step_info([s],[a],[r],[s1],d)                 &#xA;                agent.get_delta(sess)&#xA;                sess.run([agent.update_critic_weights,agent.update_actor_weights],feed_dict={agent.state_in:agent.time_step_info['s']})&#xA;                agent.update_I()  &#xA;                s = s1&#xA;        agent.save_model(path,sess)    &#xA;    except Exception as e:&#xA;        print(e)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17565" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-10T16:06:39.580" Title="How many episodes does it take for a vanilla one-step actor-critic agent to master the OpenAI BipedalWalker-v2 problem?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7837" PostTypeId="2" ParentId="7830" CreationDate="2018-09-06T02:20:22.910" Score="5" Body="&lt;p&gt;This is a good question. I tend to think the answer is &lt;strong&gt;yes&lt;/strong&gt; it is necessary to know the details, because a person without mathematical understanding of these algorithms cannot &lt;em&gt;consistently&lt;/em&gt; make a model as good as someone who does have that understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is right at the core of computer science: abstractions are useful, but usually obscure details. When those details matter, someone who only knows the abstraction and not the details that lie beneath can't understand what's going on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, if you don't understand the math behind optimizing the weights of a neural network, it might not be apparent how parameters like the learning rate are impacted by properties like network depth when some of the inputs have not been properly normalized. If you understand the optimization process mathematically, you can reason through the effects even if you are trying to work on an unfamiliar problem. This ability to reason through the probable effects of parameter decisions in &lt;em&gt;new&lt;/em&gt; domains is the main thing that you miss by working from intuition.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2020-04-02T00:44:56.647" LastActivityDate="2020-04-02T00:44:56.647" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7838" PostTypeId="1" CreationDate="2018-09-06T02:40:56.847" Score="22" ViewCount="1712" Body="&lt;p&gt;What is the definition of artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="17948" LastEditorUserId="2444" LastEditDate="2021-01-16T00:01:13.293" LastActivityDate="2021-01-16T00:01:13.293" Title="What is artificial intelligence?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;history&gt;&lt;academia&gt;&lt;ai-field&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7841" PostTypeId="2" ParentId="7794" CreationDate="2018-09-06T04:46:51.847" Score="0" Body="&lt;p&gt;If you dont want to use machine learning you may use date &lt;a href=&quot;https://dateparser.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;time parser in python&lt;/a&gt;. Few examples are given below.  It will return you formatted date time from given string. It works with all languages.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import dateparser&#xA;&amp;gt;&amp;gt;&amp;gt; dateparser.parse('12/12/12')&#xA;datetime.datetime(2012, 12, 12, 0, 0)&#xA;&amp;gt;&amp;gt;&amp;gt; dateparser.parse(u'Fri, 12 Dec 2014 10:55:50')&#xA;datetime.datetime(2014, 12, 12, 10, 55, 50)&#xA;&amp;gt;&amp;gt;&amp;gt; dateparser.parse(u'Martes 21 de Octubre de 2014')  # Spanish (Tuesday 21 October 2014)&#xA;datetime.datetime(2014, 10, 21, 0, 0)&#xA;&amp;gt;&amp;gt;&amp;gt; dateparser.parse(u'Le 11 Décembre 2014 à 09:00')  # French (11 December 2014 at 09:00)&#xA;datetime.datetime(2014, 12, 11, 9, 0)&#xA;&amp;gt;&amp;gt;&amp;gt; dateparser.parse(u'13 января 2015 г. в 13:34')  # Russian (13 January 2015 at 13:34)&#xA;datetime.datetime(2015, 1, 13, 13, 34)&#xA;&amp;gt;&amp;gt;&amp;gt; dateparser.parse(u'1 เดือนตุลาคม 2005, 1:00 AM')  # Thai (1 October 2005, 1:00 AM)&#xA;datetime.datetime(2005, 10, 1, 1, 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-09-06T04:46:51.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7842" PostTypeId="1" CreationDate="2018-09-06T04:53:21.760" Score="2" ViewCount="3598" Body="&lt;p&gt;What are some good approaches that I can use to count the no. of people in a crowd. Tracking each person individually is obviously not an option. Any good approaches or some references to research papers would be very helpful. &lt;/p&gt;&#xA;" OwnerUserId="14592" LastActivityDate="2018-09-06T07:12:03.210" Title="Counting people in an image of a crowd" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7843" PostTypeId="2" ParentId="92" CreationDate="2018-09-06T04:55:18.760" Score="5" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;How is it possible that deep neural networks are so easily fooled?&lt;/p&gt;&#xA;&lt;p&gt;Deep neural networks are easily fooled by giving high confidence predictions for unrecognizable images. How is this possible? Can you please explain ideally in &lt;strong&gt;plain&lt;/strong&gt; English?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Intuitively, extra hidden layers ought to make the network able to learn more complex classification functions, and thus do a better job classifying. While it may be named &lt;em&gt;deep learning&lt;/em&gt; it's actually a shallow understanding.&lt;/p&gt;&#xA;&lt;p&gt;Test your own knowledge: Which animal in the grid below is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Cat&quot; rel=&quot;noreferrer&quot;&gt;Felis silvestris catus&lt;/a&gt;, take your time and no cheating. Here's a hint: which is a domestic house cat?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P09Tk.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P09Tk.jpg&quot; alt=&quot;Which is a domestic house cat?&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For a better understanding checkout: &amp;quot;&lt;a href=&quot;https://medium.com/visweekly/adversarial-attack-to-vulnerable-visualizations-bfa3ed50efc2&quot; rel=&quot;noreferrer&quot;&gt;Adversarial Attack to Vulnerable Visualizations&lt;/a&gt;&amp;quot; &lt;strong&gt;and&lt;/strong&gt; &amp;quot;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap5.html&quot; rel=&quot;noreferrer&quot;&gt;Why are deep neural networks hard to train?&lt;/a&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The problem is analogous to &lt;a href=&quot;https://en.wikipedia.org/wiki/Aliasing&quot; rel=&quot;noreferrer&quot;&gt;aliasing&lt;/a&gt;, an effect that causes different signals to become indistinguishable (or aliases of one another) when sampled, and the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Wagon-wheel_effect&quot; rel=&quot;noreferrer&quot;&gt;stagecoach-wheel effect&lt;/a&gt;, where a spoked wheel appears to rotate differently from its true rotation.&lt;/p&gt;&#xA;&lt;p&gt;The neural network doesn't &lt;em&gt;know&lt;/em&gt; what it's looking at or which way it's going.&lt;/p&gt;&#xA;&lt;p&gt;Deep neural networks aren't an expert on something, they are trained to decide mathematically that some goal has been met, if they are not trained to reject wrong answers they don't have a concept of what is wrong; they only &lt;em&gt;know&lt;/em&gt; what is correct and what is not correct - wrong and &amp;quot;not correct&amp;quot; are not necessarily the same thing, neither is &amp;quot;correct&amp;quot; and true.&lt;/p&gt;&#xA;&lt;p&gt;The neural network doesn't &lt;em&gt;know&lt;/em&gt; right from wrong.&lt;/p&gt;&#xA;&lt;p&gt;Just like most people wouldn't know a house cat if they saw one, two or more, or none. How many house cats in the above photo grid, none. Any accusations of including cute cat pictures is unfounded, those are all dangerous wild animals.&lt;/p&gt;&#xA;&lt;p&gt;Here's another example. Does answering the question make Bart and Lisa smarter, does the person they are asking even know, are there unknown variables that can come into play?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BznDK.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BznDK.jpg&quot; alt=&quot;Are we there yet?&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We aren't there yet but neural networks can quickly provide an answer that is likely to be correct, especially if it was properly trained to avoid all misteps.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-06T05:01:12.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7844" PostTypeId="2" ParentId="7842" CreationDate="2018-09-06T07:12:03.210" Score="2" Body="&lt;p&gt;Here a convolutional neural network (cNN) based approach is presented:&#xA;&lt;a href=&quot;https://arxiv.org/abs/1706.03686&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Crowd Counting Using Convolutional Neural Network and Markov Random Field&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This blog post has more of a tutorial character and also present an approach based on CNNs&#xA;&lt;a href=&quot;https://blog.dimroc.com/2017/11/19/counting-crowds-and-lines/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Counting Crowds and Lines&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2585" LastActivityDate="2018-09-06T07:12:03.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7850" PostTypeId="2" ParentId="7830" CreationDate="2018-09-06T09:32:12.947" Score="0" Body="&lt;p&gt;In my answer I will call people who do not know mathematics behind ML algorithms as data science practitioners and those who know as data scientists (this terms may not be true in real life).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So with the advent of Neural Networks the importance of understanding maths behind ML algorithms has diminished significantly since earlier based on different data parameters you had to do something called &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;$feature$ $engineering$&lt;/a&gt;. This needed some kind of knowledge in statistics and basic co-ordinate geometry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays practitioners can easily apply well known models to problems without any thought process or mathematics involved. Examples of this include CNN architectures like &lt;a href=&quot;https://en.wikipedia.org/wiki/AlexNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlexNet&lt;/a&gt;, &lt;a href=&quot;http://yann.lecun.com/exdb/lenet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LeNet&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Residual_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNet&lt;/a&gt;, etc, RNN architectures like &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM's&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Gated_recurrent_unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRU's&lt;/a&gt;. We even are copying the weights of pre-trained models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what edge does a data scientist hold over practitioners? To me here is a list of points on which data scientists hold an edge:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hyper-parameter tuning: In any NN there are minimum of 3-4 hyper-parameters. Which gives rise to $^4C_n =  16$ where $n$ varies from 0 to 4 possible tunings. A data scientist from loss curves, accuracy graphs, and other score graphs will be easily able to narrow down to the tuning required to make the NN perform the best. Whereas, a practitioner may have to try out the entire 16 combinations (each combination requiring many combinations of values for each) to be able to get to the solution. Time and resource consuming.&lt;/li&gt;&#xA;&lt;li&gt;Special Architectures: Some problems require out of the box thinking to come up with the best solution. Like a combination of CNN and RNN is used to predict captions for images, CNN's may be used in field of sequence processing (like genetic sequences). Such uncommon/unconventional solutions to problems can only be applied by a person who knows how a NN works in details.&lt;/li&gt;&#xA;&lt;li&gt;Intuition: Although strongly advised against by Andrew Ng in his course, I believe every ML programmers out there use NN architectures and solutions which they believe will work best entirely due to gut feeling (rather than going through tedious methodical way of using it on small scale problems of the same type). In this scenario data scientists are bound to get a more accurate model working just due to the fact they know the mathematics and the working of a NN. They will have an intuitive understanding of how the model will work on the data to perform the task at hand.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These I feel are some of the places where a scientist holds an edge over practitioners. I may have missed some other edges that a Data scientist may hold over practitioners, you are free to edit it in. &lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-09-06T09:32:12.947" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7851" PostTypeId="2" ParentId="7838" CreationDate="2018-09-06T09:47:11.210" Score="3" Body="&lt;p&gt;Over the years, many people attempted to define artificial intelligence. A lot of those definitions are summed up by Stuart Russell and Peter Norvig in their book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence - A Modern Approach&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definitions of AI can be summarised as falling into the following categories:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Those that address thought process and reasoning (how an AI thinks/reasons)&lt;/li&gt;&#xA;  &lt;li&gt;Those that address behaviour (how an AI acts given what it knows)&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the above 2 categories are further divided into definitions that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I. assess the success of an AI (to do the above) based on its ability&#xA;  to replicate human performance&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;II. or an ability to replicate an ideal performance measure called&#xA;  'rationality' (does it do the 'right' thing based on what it knows?)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I will cite you definitions that fit into each of the above categories: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;1.I. &quot;The [automation of] activities that we associate with human thinking, activities such as decision making, problem solving, learning..&quot; - Bellman 1978&lt;/li&gt;&#xA;  &lt;li&gt;1.II. &quot;The study of the computations that make it possible to perceive, reason, and act.&quot; - Winston, 1992&lt;/li&gt;&#xA;  &lt;li&gt;2.I. &quot;The study of how to make computers do things at which, at the moment, people do better&quot; - Rich and Knight, 1991&lt;/li&gt;&#xA;  &lt;li&gt;2.II. &quot;The study of the design of intelligent agents&quot; - Poole et al., 1998&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In summary, AI is devoted to the creation of intelligent and rational machines that can make rational decisions and take rational actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you read up on the Turing test, which Alan Turing proposed to test if a computer was intelligent. However, the Turing test has a few issues, because it is anthropomorphic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Aeronautical engineers created the airplane, they didn't set their goal that planes should fly exactly like birds, but rather, they started learning how lift forces were generated, based on the study of aerodynamics. Using this knowledge, they created planes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, people in the AI world shouldn't put, IMHO, human intelligence as the standard to strive for, but, rather, we could use, say, rationality as a standard (amongst others).&lt;/p&gt;&#xA;" OwnerUserId="17965" LastEditorUserId="2444" LastEditDate="2019-06-21T16:48:36.723" LastActivityDate="2019-06-21T16:48:36.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7853" PostTypeId="1" CreationDate="2018-09-06T12:37:06.217" Score="1" ViewCount="116" Body="&lt;p&gt;I developed a CNN for image analysis. I've around 100K labeled images.  I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than fitting. So, I'm playing with different hyper-parameters: number of filters, number of layers, number of neurons in the dense layers, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every test, I'm using all the training data, and it is very slow and time consuming.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to have an early idea about if a model will perform better than another?&lt;/p&gt;&#xA;" OwnerUserId="17960" LastEditorUserId="2444" LastEditDate="2019-05-04T17:10:17.467" LastActivityDate="2021-02-23T10:04:50.470" Title="Is there a way of pre-determining whether a CNN model will perform better than another?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;deep-neural-networks&gt;" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7854" PostTypeId="1" AcceptedAnswerId="7858" CreationDate="2018-09-06T12:44:46.330" Score="4" ViewCount="527" Body="&lt;p&gt;In the Berkeley RL class they mention the gradient would be 0 if the policy is deterministic. Why is that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=XGmd3wcyDg8&amp;amp;feature=youtu.be&amp;amp;t=1071&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=XGmd3wcyDg8&amp;amp;feature=youtu.be&amp;amp;t=1071&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17966" LastEditorUserId="17966" LastEditDate="2018-09-06T22:53:33.833" LastActivityDate="2018-09-06T22:53:33.833" Title="Why is the derivative 0 if the policy is deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7855" PostTypeId="2" ParentId="7853" CreationDate="2018-09-06T13:35:41.060" Score="0" Body="&lt;p&gt;I would like you to try with following changes in the model. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Introduce batch normalization layer in the model. &lt;/li&gt;&#xA;&lt;li&gt;Try with the batch size of 32 - 64&lt;/li&gt;&#xA;&lt;li&gt;Use different architecture, Like VGG, Resnet, etc..&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is no full proof answer to this question but you can get best by trying some know strategies.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastActivityDate="2018-09-06T13:35:41.060" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7856" PostTypeId="1" CreationDate="2018-09-06T13:42:31.290" Score="2" ViewCount="147" Body="&lt;p&gt;One way of ranking human intelligence is based on IQ tests. But how can we compare the intelligence of AI systems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, is there a test that tells me that a spam filter system is more intelligent than a self-driving car, or can I say that a chess program is more intelligent than AlphaGo?&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2020-04-01T14:44:49.557" LastActivityDate="2020-04-01T14:44:49.557" Title="How can we compare the intelligence of AI systems?" Tags="&lt;agi&gt;&lt;performance&gt;&lt;intelligence-testing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7857" PostTypeId="2" ParentId="7854" CreationDate="2018-09-06T14:22:54.677" Score="3" Body="&lt;p&gt;Well, I'd rather comment, but I don't have yet this privilege, so here are some comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, having a deterministic policy inside the log would do create trivial terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, for me, in Policy Gradient methods, it's a non sense to have a deterministic policy during the optimization, because you want to explore the space of weights. In my experience, you only set the policy to deterministic (in PG method) when you're done with the optimization, and you want to test your network.&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2018-09-06T14:55:21.640" LastActivityDate="2018-09-06T14:55:21.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7858" PostTypeId="2" ParentId="7854" CreationDate="2018-09-06T14:44:47.477" Score="12" Body="&lt;p&gt;First, a small clarification; they do not say that &quot;the gradient of a deterministic policy is 0&quot; (I'm not sure what, if anything, &quot;the gradient of a policy&quot;) would mean. They are talking about how &quot;this gradient&quot; (referring to the gradient on the slide visible in the video) is 0 in the case where the policy ($\pi$) is deterministic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, here is the gradient that they are discussing in the video:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla_{\theta} J(\theta) \approx \frac{1}{N} \sum_{i=1}^N \left( \sum_{t=1}^T \nabla_{\theta} \log \pi_{\theta} (\mathbf{a}_{i, t} \vert \mathbf{s}_{i, t}) \right) \left( \sum_{t = 1}^T r(\mathbf{s}_{i,t}, \mathbf{a}_{i, t}) \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this equation, $\pi_{\theta} (\mathbf{a}_{i, t} \vert \mathbf{s}_{i, t})$ denotes the probability of our policy $\pi_{\theta}$ selecting the actions $\mathbf{a}_{i, t}$ that it actually ended up selecting in practice, given the states $\mathbf{s}_{i, t}$ that it encountered during the episode that we're looking at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a deterministic policy $\pi_{\theta}$, we know for sure that the probability of it selecting the actions that it did select must be $1$ (and the probability of it selecting any other actions would be $0$, but such a term does not show up in the equation). So, we have $\pi_{\theta} (\mathbf{a}_{i, t} \vert \mathbf{s}_{i, t}) = 1$ for every instance of that term in the above equation. Because $\log 1 = 0$, this leads to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{aligned}&#xA;\nabla_{\theta} J(\theta) &amp;amp;\approx \frac{1}{N} \sum_{i=1}^N \left( \sum_{t=1}^T \nabla_{\theta} \log \pi_{\theta} (\mathbf{a}_{i, t} \vert \mathbf{s}_{i, t}) \right) \left( \sum_{t = 1}^T r(\mathbf{s}_{i,t}, \mathbf{a}_{i, t}) \right) \\&#xA;%&#xA;&amp;amp;= \frac{1}{N} \sum_{i=1}^N \left( \sum_{t=1}^T \nabla_{\theta} \log 1 \right) \left( \sum_{t = 1}^T r(\mathbf{s}_{i,t}, \mathbf{a}_{i, t}) \right) \\&#xA;%&#xA;&amp;amp;= \frac{1}{N} \sum_{i=1}^N \left( \sum_{t=1}^T \nabla_{\theta} 0 \right) \left( \sum_{t = 1}^T r(\mathbf{s}_{i,t}, \mathbf{a}_{i, t}) \right) \\&#xA;%&#xA;&amp;amp;= \frac{1}{N} \sum_{i=1}^N 0 \left( \sum_{t = 1}^T r(\mathbf{s}_{i,t}, \mathbf{a}_{i, t}) \right) \\&#xA;%&#xA;&amp;amp;= 0 \\&#xA;\end{aligned}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i.e. you end up with a sum of terms that are all multiplied by $0$).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-06T14:44:47.477" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7861" PostTypeId="1" AcceptedAnswerId="7864" CreationDate="2018-09-06T17:53:08.197" Score="5" ViewCount="189" Body="&lt;p&gt;In theoretical computer science, there is a massive categorization of the difficulty of various computational problems in terms of their asymptotic worst-time computational complexity. There doesn't seem to be any analogous analysis of what problems are &quot;hard for AI&quot; or even &quot;impossible for AI.&quot; This is in some sense quite reasonable, because most research is focused on what can be solved. I'm interested in the opposite. &lt;strong&gt;What I do need to prove about a problem to prove that it is &quot;not reasonably solvable&quot; by AI?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many papers say something along the lines of&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AI allows us to find real-world solutions to real-world instances of NP-complete problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is there a theoretical, principled reason for saying this instead of &quot;... PSPACE-complete problems&quot;? Is there some sense in which AI doesn't work on PSPACE-complete, or EXPTIME-complete, or Turing complete problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea answer would be a reference to a paper that shows AI cannot be used to solve a particular kind of problem based on theoretical or statistical reasoning. Any answer exhibiting and justifying a benchmark for &quot;too hard for AI&quot; would be fine though (bonus points if the benchmark has a connection to complexity and computability theory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this question doesn't have an answer in general, answers about specific techniques would also be interesting to me.&lt;/p&gt;&#xA;" OwnerUserId="12732" LastEditorUserId="2444" LastEditDate="2019-07-07T14:12:45.007" LastActivityDate="2019-07-07T14:12:45.007" Title="What does &quot;hard for AI&quot; look like?" Tags="&lt;ai-design&gt;&lt;theory&gt;&lt;reference-request&gt;&lt;ai-complete&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7862" PostTypeId="2" ParentId="3972" CreationDate="2018-09-06T19:57:45.970" Score="2" Body="&lt;p&gt;I'm not familiar with any existing, robust methods to generate such a dataset. Here are some thoughts though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You propose using an MLP with a single hidden layer. That means we have two weight matrices, and two activation functions (one for hidden layer, one for output layer). Some notation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$d$: dimensionality of input vectors&lt;/li&gt;&#xA;&lt;li&gt;$n$: dimensionality of hidden layer&lt;/li&gt;&#xA;&lt;li&gt;You mentioned binary output, so I'll assume that dimensionality is $2$&lt;/li&gt;&#xA;&lt;li&gt;$W_n^{(1)} \in \mathbb{R}^{d \times n}$: the first weight matrix in the scenario where we're using $n$ hidden nodes&lt;/li&gt;&#xA;&lt;li&gt;$W_n^{(2)} \in \mathbb{R}^{n \times 2}$: the second weight matrix in the scenario where we're using $n$ hidden nodes&lt;/li&gt;&#xA;&lt;li&gt;$g$: first activation function&lt;/li&gt;&#xA;&lt;li&gt;$h$: second activation function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then, given an input vector $x \in \mathbb{R}^d$, our neural network will generate output $f_n(x) = h(W_n^{(2)} (g(W_n^{(1)}x)))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in general we of course expect Neural Networks to only find a local optimum, but if you want a robust solution you'll want it to be able to handle the worst case, and the worst case for your &quot;adversarial&quot; task is when the Neural Network manages to find the global optimum. So, we'll assume it can find the global optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, what you're looking for is a dataset $D_n$ containing a number of input vectors $x$, such that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$f_n(x) = h(W_n^{(2)} (g(W_n^{(1)}x)))$ provides good / optimal results (after training to the global optimum)&lt;/li&gt;&#xA;&lt;li&gt;$f_{n - 1}(x) = h(W_{n - 1}^{(2)} (g(W_{n - 1}^{(1)}x)))$ provides poor results (even after training to the global optimum of this setup).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In other words, you want to find a collection of vectors $x$ such that it becomes impossible to find a collection of weights in the $n - 1$ case where $f_{n-1}(x)$ is a good approximation of $f_n(x)$. You want to make it impossible that $f_{n - 1}(x) \approx f_n(x)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now this has turned into a clear mathematical problem. I'm not familiar with any established methods in mathematics to solve a problem like this, maybe there are though. My best guess at this point in time would be a procedure like the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate random &quot;ground truth&quot; versions of the weight matrices $W^{(1)*}_{n}$ and $W^{(2)*}_{n}$ (just completely random matrices).&lt;/li&gt;&#xA;&lt;li&gt;Generate completely random input vectors $x$. Compute the corresponding ground truth labels as $f^*_n(x) = h(W_n^{(2)*} (g(W_n^{(1)*}x)))$.&lt;/li&gt;&#xA;&lt;li&gt;Hope that the neural network with $n$ hidden nodes can recover the ground truth weight matrices that were previously generated randomly.&lt;/li&gt;&#xA;&lt;li&gt;Hope that the neural network with $n - 1$ hidden nodes cannot find an accurate approximation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In theory, the MLP with $n$ hidden nodes should be able to learn the exact ground truth function. In theory, under certain conditions, the MLP with $n$ hidden nodes should not be able to learn the &lt;strong&gt;exact&lt;/strong&gt; ground truth function. I suspect those &quot;certain conditions&quot; would be that the rows/columns of the weight matrices should be linearly independent, which is likely with randomly generated matrices, but I'm not 100% sure on this. Even if it can't learn the &lt;strong&gt;exact&lt;/strong&gt; ground truth, it may still be capable of learning an approximation... there may be ways to find upper bounds on how close such an approximation could get, but I'm not sure.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-06T19:57:45.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7863" PostTypeId="2" ParentId="7856" CreationDate="2018-09-06T20:03:21.163" Score="2" Body="&lt;p&gt;There are different ways to compare different kinds of AI techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point, be aware that &quot;AI System&quot; can mean an incredibly broad range of things. In popular culture, we usually think of a deployed system that uses AI techniques. These systems can only be compared on the basis of their performance, and their performance may have relatively little to do with AI itself (e.g. their behaviours might be more strongly affected by user interface decisions, not the AI techniques under the hood).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, AI researchers are usually more interested in comparing the performance of different AI algorithms at solving the &quot;AI-ish&quot; parts of the problem a fully developed system aims to solve. A common way to do this is with &lt;em&gt;benchmark problems&lt;/em&gt;. For example, in machine learning it is common to compare two algorithms by running each of them on a commonly used dataset, and comparing the performance of the models they create. In AI Planning, it is common to issue planning challenges to the community, and compare the quality of the plans on several different axes (e.g. average wait times, maximum wait times, whether goals were accomplished, how long it took to create a plan, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no generally agreed upon way to compare techniques across different areas of AI, but a commonly adopted approach is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test&lt;/a&gt;. In the Turing Test, we care only about the ability of the system to mimic something like human intelligence. It's fair game to ask about planning problems, or learning problems, or other topics, so you could in some sense judge one technique to be better than another. However, most judgements made in the Turing Test are subjective, so it's not clear that it really solves the problem you posed.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-06T20:03:21.163" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7864" PostTypeId="2" ParentId="7861" CreationDate="2018-09-06T20:19:34.127" Score="4" Body="&lt;p&gt;Nice Question!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a perennial topic of discussion among AI researchers. The short answer is &quot;we don't really know which topics are hard in general, but we do know which we haven't got good techniques for yet.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's start by explaining why AI is not concerned with notions of computational complexity like NP-Completeness. AI researchers figured out in the 90's that most problems that are computationally hard in theory aren't actually hard in practice at all! For example, Boolean Satisfiability, the canonical NP-Hard problem, is known to have hard instances, but in practice, these almost never show up. Even when they do, we can usually get good approximate solutions with minimal computation time. Since many AI problems are reducible to SAT, whole areas of the field just use these approximation techniques and solvers. There's a good, if a bit old, survey &lt;a href=&quot;https://www.cs.cornell.edu/gomes/pdf/2008_gomes_knowledge_satisfiability.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Since 2008, things have only gotten better. Basically, NP-Hard stuff just isn't that hard. Worst-case complexity is therefore probably the wrong tool to guess at which problems are hard for AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the other end of the scale, we have subjective complexity based on things like how &quot;large&quot; the problem is. This has proven to be pretty unreliable. One example is Go, which I was told would &quot;never be solved by AI&quot; as late as 2010. Clearly, we were completely wrong. We just didn't know which techniques to use yet. Another example is language. Rule-based AI'ers tried at it for decades with minimal success. Probabilistic methods have achieved essentially human-level performance in less time. If you asked a researcher in the 1970s if language was hard for AI, they'd have said yes, but they'd have been wrong. This has been closely related to the advances in computing hardware: techniques that seemed wasteful and slow 40 years ago are now entirely practical. Sometimes they turn out to solve problems really well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of this ties into the issue that AI'ers don't really know or agree on what intelligence is, or what it means to solve a problem. Some AI'ers maintain that language really &lt;em&gt;is&lt;/em&gt; hard, and that the systems we use now aren't &lt;em&gt;really&lt;/em&gt; solving language, they're just making blind guesses that happen to be right a lot. Under that view, language is still hard. &lt;a href=&quot;https://en.wikipedia.org/wiki/Jerry_Fodor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fodor&lt;/a&gt; was a strong proponent of this view in the past, and his writings are a good place to read about it, but I've still heard people espouse views like this at AI conferences as recently as 2015.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something this &lt;em&gt;usually&lt;/em&gt; considered hard right now is making a good &lt;em&gt;general&lt;/em&gt; intelligence, that integrates knowledge across many domains and could plausibly pass something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing test&lt;/a&gt;. Yet, there has been some progress here (look at virtual assistants), and some AI'ers reckon this might not be as hard as we thought either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, basically, we don't know what's hard because we don't know when someone will come up with a new exciting technique that solves problems previously thought to be hard, we know that worst-case complexity isn't a good measurement system for &quot;intelligence&quot; requirements, and we don't know what intelligence really is. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2019-07-07T13:49:15.393" LastActivityDate="2019-07-07T13:49:15.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7865" PostTypeId="1" AcceptedAnswerId="7872" CreationDate="2018-09-06T23:27:52.350" Score="12" ViewCount="5332" Body="&lt;p&gt;In a convolutional neural network, which layer consumes more training time: convolution layers or fully connected layers? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can take AlexNet architecture to understand this. I want to see the time breakup of the training process. I want a relative time comparison so we can take any constant GPU configuration.&lt;/p&gt;&#xA;" OwnerUserId="17980" LastEditorUserId="2444" LastEditDate="2020-05-19T19:48:54.123" LastActivityDate="2020-05-19T19:48:54.123" Title="Which layer in a CNN consumes more training time: convolution layers or fully connected layers?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7866" PostTypeId="2" ParentId="7838" CreationDate="2018-09-07T01:15:44.747" Score="-1" Body="&lt;p&gt;&lt;strong&gt;Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A measure of the strength of a decision-making agent relative to other decision-making agents, in regard to a given task or set of tasks. The medium is irrelevant—intelligence is exhibited by both organic and intentionally created mechanisms.  May also be the capability to solve a problem, as in the case of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved game&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Artificial&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relates to the term &lt;a href=&quot;https://www.etymonline.com/word/artifact&quot; rel=&quot;nofollow noreferrer&quot;&gt;artifact&lt;/a&gt;, a thing which is intentionally created.  Typically this term has been used to connote physical objects, but algorithms created by humans are also regarded as artifacts.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The etymology is derived from the Latin words &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Dars&quot; rel=&quot;nofollow noreferrer&quot;&gt;ars&lt;/a&gt;&lt;/em&gt; and &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Dfacio&quot; rel=&quot;nofollow noreferrer&quot;&gt;faciō&lt;/a&gt;&lt;/em&gt;: &quot;To skillfully construct&quot;, or, &quot;the art of making&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Artificial Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Any decision-making agent that is skillfully (intentionally) constructed.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;APPENDIX: The meaning of &quot;intelligence&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original meaning of &quot;intelligence&quot; seems to be &quot;to acquire&quot;, back to the Indo-European. See: &lt;a href=&quot;https://www.etymonline.com/word/intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;intelligence (etymology)&lt;/a&gt;; &lt;a href=&quot;https://en.wiktionary.org/wiki/Appendix:List_of_Proto-Indo-European_roots#l&quot; rel=&quot;nofollow noreferrer&quot;&gt;*leg/*leh₂w-&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.oxforddictionaries.com/definition/intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;The OED 1st definition of intelligence&lt;/a&gt; is not incorrect, extending the meaning to acquisition of capability (demonstrable utility), just that the second definition is the older and fundamental: &quot;The collection of information of [strategic] value; 2.3 (archaic) Information in general; news.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bigthink.com/philip-perry/the-basis-of-the-universe-may-not-be-energy-or-matter-but-information&quot; rel=&quot;nofollow noreferrer&quot;&gt;You can regard the universe as being comprised of information&lt;/a&gt;, whatever form that information takes (matter, energy, states, relative positions, etc.) From the standpoint of an algorithm, this makes sense since the only means they have to gauge the universe are &lt;a href=&quot;https://en.wikipedia.org/wiki/Percept_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;percepts&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a flat text file. It may just be data, but you could try and execute. If it actually runs, it might demonstrate utility at some task. (For instance, if it is a minimax algorithm.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Intelligence as a measure of utility&quot; is itself &quot;intelligence&quot; in the sense of information, specifically that information by which we measure intelligence, as a degree, relative to a task or to other intelligences.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-24T00:09:09.933" LastActivityDate="2019-03-24T00:09:09.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7867" PostTypeId="1" CreationDate="2018-09-07T04:19:05.633" Score="2" ViewCount="644" Body="&lt;p&gt;I have an application where I want to find the locations of objects on a simple, relatively constant background (fixed camera angle, etc). For investigative purposes, I've created a test dataset that displays many characteristics of the actual problem.&lt;/p&gt;&#xA;&lt;p&gt;Here's a sample from my test dataset.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AKV7f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AKV7f.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Our problem description is to &lt;strong&gt;find the bounding box of the single circle in the image&lt;/strong&gt;. If there is more than one circle or no circles, we don't care about the bounding box (but we at least need to know that there is no valid single bounding box).&lt;/p&gt;&#xA;&lt;p&gt;For my attempt to solve this, I built a CNN that would regress &lt;code&gt;(min_x, min_y, max_y, max_y)&lt;/code&gt;, as well as one more value that could indicate how many circles were in the image.&lt;/p&gt;&#xA;&lt;p&gt;I played with different architecture variations, but, in general, the architecture a was very standard CNN (3-4 ReLU convolutional layers with max-pooling in between, followed by a dense layer and an output layer with linear activation for the bounding box outputs, set to minimise the mean squared error between the outputs and the ground truth bounding boxes).&lt;/p&gt;&#xA;&lt;p&gt;Regardless of the architecture, hyperparameters, optimizers, etc, the result was always the same - &lt;strong&gt;the CNN could not even get close to building a model&lt;/strong&gt; that was able to regress an accurate bounding box, even with over 50000 training examples to work with.&lt;/p&gt;&#xA;&lt;p&gt;What gives? Do I need to look at using another type of network as CNNs are more suited to classification rather than localisation tasks?&lt;/p&gt;&#xA;&lt;p&gt;Obviously, there are computer vision techniques that could solve this easily, but due to the fact that the actual application is more involved, I want to know strictly about NN/AI approaches to this problem.&lt;/p&gt;&#xA;" OwnerUserId="17985" LastEditorUserId="2444" LastEditDate="2021-01-29T23:22:55.500" LastActivityDate="2021-01-29T23:22:55.500" Title="How to architect a network to find bounding boxes in simple images?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;bounding-box&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7868" PostTypeId="2" ParentId="7867" CreationDate="2018-09-07T06:34:42.487" Score="1" Body="&lt;p&gt;There are some special architectures of CNNs which are designed exactly for the task you mention. The &lt;a href=&quot;https://github.com/facebookresearch/Detectron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Detector&lt;/a&gt; library includes a collection of these architectures, &lt;a href=&quot;https://arxiv.org/abs/1703.06870&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper describes the Mask R-CNN network in detail, which is designed for image segmentation tasks.&lt;/p&gt;&#xA;" OwnerUserId="2585" LastActivityDate="2018-09-07T06:34:42.487" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7869" PostTypeId="2" ParentId="7762" CreationDate="2018-09-07T07:00:51.687" Score="0" Body="&lt;p&gt;Thare are many ways to approache this. One approach to start would be to try to describe the problem using a formalism closer to reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Output:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Aiming in any shooter type game, as I recall, involves moving the mouse. So the output of your aimbot has two dimentions. Depending on the required accuracy, you can consider these two dimentions continous, with a limited range, or if you consider each pixel as a integer, you might be able to discretize your action space. (I assume mouse XY coordinates should be input to the game, not increments)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You  definetly need screen information. You can take the whole screen as input to a CNN, similarly to DeepQLearning for Atari.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reward function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This might be tricky since your rewards need to be as dense as possible, however, the only feedback you get from the game is that someone was shot. &#xA;It might be ebough but this will definetly increase your training time. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Training data / Environemnt:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Your environment for training is the game itseld. Using a &lt;a href=&quot;https://ronan.collobert.com/pub/matos/2009_curriculum_icml.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;curriculum learning&lt;/a&gt; approach would probably make the training process more efficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also try an &lt;a href=&quot;https://arxiv.org/abs/1801.06503&quot; rel=&quot;nofollow noreferrer&quot;&gt;imitation learning&lt;/a&gt; approach, since I assume you are happy to provide expert training examples (in this case probably headshots in the game environment). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about how to apply reinforcement learning for games &lt;a href=&quot;https://github.com/Unity-Technologies/ml-agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The Unity ML-Agents Library also includes sample tracking problems and their solutions. &lt;/p&gt;&#xA;" OwnerUserId="2585" LastActivityDate="2018-09-07T07:00:51.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7871" PostTypeId="2" ParentId="7832" CreationDate="2018-09-07T08:59:57.400" Score="2" Body="&lt;p&gt;Ok, so I think I have a better understanding of this now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, let's remind the main idea of the PPO : staying close to the previous policy. It's the same idea than in TRPO, but the L function is improved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you wanna make &quot;small but safe steps&quot;. With clipped surrogate objective, you don't give too much importance to promising actions. You learn that bad actions are bad, so you decrease their probability according to &quot;how bad&quot; they are. But for good actions, you only learn that they are &quot;a little bit good&quot;, and their probability will be just slightly increased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This mechanism allows you to perform small but relevant updates of your policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;hope this will help someone :)&lt;/p&gt;&#xA;" OwnerUserId="17759" LastActivityDate="2018-09-07T08:59:57.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7872" PostTypeId="2" ParentId="7865" CreationDate="2018-09-07T11:31:05.880" Score="12" Body="&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; I did these calculations speculatively, so some errors might have crept in. Please inform of any such errors so I can correct it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general in any CNN the maximum time of training goes in the Back-Propagation of errors in the Fully Connected Layer (depends on the image size). Also the maximum memory is also occupied by them. Here is a slide from Stanford about VGG Net parameters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dmvrt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dmvrt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Rl1G.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Rl1G.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly you can see the fully connected layers contribute to about 90% of the parameters. So the maximum memory is occupied by them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as training time goes, it somewhat depends on the size (pixels*pixels) of the image being used. In FC layers it is straightforward that the number of derivatives you have to calculate is equal to the number of parameters. As far as the convolutional layer goes, let's see an example,let us take the case of 2nd layer:&#xA;It has got 64 filters of $(3*3*3)$ dimensions to be updated. The error is propagated from the 3rd layer. Each channel in the 3rd layer propagates it error to its corresponding $(3*3*3)$ filter. Thus $224*224$ pixels will contribute to about $224*224*(3*3*3)$ weight updations. And since there are $64$ such $224*224$ channels we get the total number of calculations to be performed as $64*224*224*(3*3*3) \approx 87*10^6 $ calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let us take the last layer of $56*56*256$. It will pass it gradients to the previous layer. Each $56*56$ pixel will update a $(3*3*256)$ filter. And since there are 256 such $56*56$ the total calculations required would be $256 * 56 * 56 * (3*3*256) \approx 1850 *10^6$ calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So number of calculations in a convolutional layer really depends on the number of filters and the size of the picture. In general I  have used the following formulae to calculate the number of updations required for filters in a layer, also I have considered $stride = 1$ since it will be the worst case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$channels_{output} * (pixelOutput_{height} * pixelOutput_{width}) * (filter_{height} * filter_{width} * channels_{input})$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to fast GPU's we are easily able to handle these huge calculations. But in FC layers the entire matrix needs to be loaded which causes memory problems which is generally not the case of convolutional layers, so training of convolutional layers is still easy. Also all these have to be loaded in the GPU memory itself and not the RAM of the CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also here is the parameter chart of AlexNet:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zSUzd.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zSUzd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is a performance comparison of various CNN architectures:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0ruCU.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0ruCU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you check out the &lt;a href=&quot;https://www.youtube.com/watch?v=DAOcjicFr1Y&amp;amp;index=9&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;noreferrer&quot;&gt;CS231n Lecture 9&lt;/a&gt; by Stanford University for better understanding of the nooks and crannies of CNN architectures.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-09-08T09:14:05.303" LastActivityDate="2018-09-08T09:14:05.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7873" PostTypeId="1" AcceptedAnswerId="7874" CreationDate="2018-09-07T13:04:23.503" Score="3" ViewCount="267" Body="&lt;p&gt;I'm a fresh learner of AI. I was told that depth-first search is not an optimal searching algorithm since &quot;it finds the 'leftmost' solution, regardless of depth or cost&quot;. Therefore, does it mean that in practice, when we implement DFS, we should always have a checker to stop the search when it finds the first solution (also the leftmost one)?&lt;/p&gt;&#xA;" OwnerUserId="17996" LastEditorUserId="2444" LastEditDate="2019-11-21T16:41:51.540" LastActivityDate="2019-11-21T16:41:51.540" Title="Does depth-first search always stop when it has found the leftmost solution?" Tags="&lt;search&gt;&lt;implementation&gt;&lt;depth-first-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7874" PostTypeId="2" ParentId="7873" CreationDate="2018-09-07T13:12:21.293" Score="2" Body="&lt;p&gt;One of the more standard assumptions when first introducing new students to search algorithms (like Depth-First Search, Breadth-First Search which you've also likely heard about or will hear about soon, etc.) is indeed that our goal is to find some sort of solution, and only find one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If our intention is to find just a single solution, then yes, you will need to check at every node whether that is a solution node, and you can stop the search process once you've found one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, there can be all kinds of variants of this idea. Maybe in a different case you are interested in finding &lt;em&gt;all&lt;/em&gt; solutions, rather than a single one; in such a case you would naturally not stop the search process after finding the first one, but continue searching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to conclude, it really depends on exactly what you want, why are you using a search algorithm. If you only care about finding &lt;em&gt;a&lt;/em&gt; solution, you can stop when you have one.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-07T13:12:21.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7875" PostTypeId="1" CreationDate="2018-09-07T13:31:55.310" Score="12" ViewCount="554" Body="&lt;p&gt;The term Singularity is often used in mainstream media for describing visionary technology. It was introduced by Ray Kurzweil in a popular book &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Singularity_Is_Near&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Singularity Is Near: When Humans Transcend Biology&lt;/a&gt; (2005).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In his book, Kurzweil gives an outlook to a potential future of mankind which includes nanotechnology, computers, genetic modification and artificial intelligence. He argues that Moore's law will allow computers an exponential growth which results in a superintelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the &lt;em&gt;technological singularity&lt;/em&gt; something that is taken seriously by A.I. developers or is this theory just a load of popular hype?&lt;/p&gt;&#xA;" OwnerUserId="17978" LastEditorUserId="2444" LastEditDate="2019-09-16T16:29:26.133" LastActivityDate="2020-08-03T08:45:14.823" Title="Is the singularity something to be taken seriously?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;superintelligence&gt;&lt;singularity&gt;&lt;mythology-of-ai&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7877" PostTypeId="1" AcceptedAnswerId="7878" CreationDate="2018-09-07T14:42:12.617" Score="6" ViewCount="1024" Body="&lt;p&gt;I'm confused regarding a specific detail of MCTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate my question, let's take the simple example of tic-tac-toe.&#xA;After the selection phase, when a leaf node is reached, the tree is expanded in the so-called &lt;strong&gt;expansion phase&lt;/strong&gt;. Let's say a particular leaf node has 6 children. Would the expansion phase expand all the children and run the simulation on them? Or would the expansion phase only pick a single child at random and run simulation, and only expand the other children if the selection policy arrives at them at some later point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if both of these are accepted variants, what are the pros/cons of each one?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditDate="2019-11-19T16:24:32.020" LastActivityDate="2019-11-19T16:24:32.020" Title="Which nodes are expanded in the expansion phase of MCTS?" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7878" PostTypeId="2" ParentId="7877" CreationDate="2018-09-07T15:56:10.540" Score="3" Body="&lt;p&gt;By far the &lt;strong&gt;most common&lt;/strong&gt; (and likely also the most simple / straightforward) implementation is to &lt;strong&gt;expand exactly one node in the Expansion Phase&lt;/strong&gt;; specifically, &lt;strong&gt;the node corresponding to the very first state selected (semi-)randomly by the Play-Out Phase&lt;/strong&gt;. This is also pretty much the bare minimum you have to do if you want any form of tree growing at all (which you do).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Other variants are possible too, but are much less common. The variant you suggest in the question is to &lt;strong&gt;expand all the children of the final node encountered during the Selection Phase, and run a Play-Out for all of them&lt;/strong&gt;. I am not familiar with any literature on such a strategy really, never tried that myself. Intuitively, I would expect it to perform very similarly, perhaps &lt;em&gt;slightly&lt;/em&gt; worse. Essentially what this would do is that it moves the &quot;behaviour&quot; of the search algorithm slightly more towards Breadth-First Search behaviour, rather than Best-First search behaviour. You spend a bit less of your computation time in the Selection Phase, because every Selection Phase is followed up by for example 6 (or whatever branching factor you have) Play-Outs instead of just a single one. On average I'd expect this to be slightly worse, because the Selection Phase is the primary source of the &quot;Best-First Search&quot; behaviour of the algorithm. I certainly don't expect a change like this to cause a large difference in performance though, if any. It will also likely be domain-dependent; worse in some cases, better in other cases.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A different variant that I did once use myself is to &lt;strong&gt;expand every node in the complete line of play followed by the Play-Out phase&lt;/strong&gt;. You can visualize this as a very &quot;thin&quot;, but &quot;deep&quot; expansion, whereas your suggestion discussed above would be visualized as a &quot;shallow&quot; but &quot;wide&quot; expansion (and the conventional expansion strategy of a single node would be &quot;thin&quot; and &quot;shallow&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this strategy, it is much easier to clearly define the advantages and disadvantages that it has in comparison to the standard strategy. The main &lt;strong&gt;advantage&lt;/strong&gt; is that you &lt;strong&gt;retain more information from your Play-Outs&lt;/strong&gt;, you throw less information away. This is because the Backpropagation phase, after the Play-Out terminates, can only store information in nodes that exist. If you immediately expand the complete line of play followed in the Play-Out Phase, you can store the result (the evaluation in the terminal state) in all of those nodes. If you don't expand the complete Play-Out (e.g. only expand the very first node), you'll have to &quot;skip&quot; all of those nodes which you didn't expand yet (they don't exist), and you can't store results in there yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main &lt;strong&gt;disadvantage&lt;/strong&gt; of this approach is that it requires &lt;strong&gt;more memory&lt;/strong&gt;, the tree grows a lot more quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would personally recommend this approach if you have &lt;strong&gt;very strict limitations on computation time&lt;/strong&gt;, if you expect to be able to only run very few iterations of the MCTS algorithm. For example, I personally used this in my &lt;a href=&quot;https://github.com/DennisSoemers/MaastCTS2&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Video Game AI agent&lt;/a&gt;; this is a real-time game where you have to make decisions every &lt;strong&gt;~40 milliseconds&lt;/strong&gt;. In such a low amount of time, you cannot run many MCTS simulations. This means that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You do not expect to run out of memory, even if you grow your tree very quickly, so the increased memory requirements become a non-issue.&lt;/li&gt;&#xA;&lt;li&gt;Due to the low expected number of iterations, it is extremely important to retain as much information as possible, not throw any information away. If we can't run many simulations, we want to make sure to squeeze every little bit of information we can out of each of them.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For contrast, if you're developing an agent to play a board game, and it has in the order of &lt;strong&gt;multiple minutes of thinking time&lt;/strong&gt; per turn, the standard approach of only expanding a single node per Expansion Phase becomes a lot more appealing. If you're capable of running tens of thousands of iterations or more, it really doesn't hurt if you &quot;forget&quot; about a little bit of information deep down in the tree. The risk of running out of memory also becomes a lot more serious if you're running many iterations, so you don't want to grow the tree too quickly.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-07T15:56:10.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7879" PostTypeId="1" AcceptedAnswerId="8905" CreationDate="2018-09-07T17:14:48.587" Score="5" ViewCount="111" Body="&lt;p&gt;Objects tracking is finding the trajectory of each object in consecutive frames. Human tracking is a subset of object tracking which just considers humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen many papers that divide tracking methods into two parts: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Online tracking: Tracker just uses current and previous frames.&lt;/li&gt;&#xA;&lt;li&gt;Offline tracking: Tracker uses all frames.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All of them mention that online tracking is suitable for autonomous driving and robotics, but I don't understand this part. What are the applications of object/human tracking in autonomous driving? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know some related papers?&lt;/p&gt;&#xA;" OwnerUserId="10051" LastEditorUserId="2444" LastEditDate="2019-03-22T14:55:56.240" LastActivityDate="2019-03-23T22:05:20.000" Title="What are applications of object/human tracking in autonomous cars?" Tags="&lt;computer-vision&gt;&lt;autonomous-vehicles&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7880" PostTypeId="1" AcceptedAnswerId="7883" CreationDate="2018-09-07T22:39:06.603" Score="4" ViewCount="644" Body="&lt;p&gt;Is there an accepted way in NLP to parse conjunctions (and/or) in a sentence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By following the example below, how would I parse&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I drink orange juice if its the weekend or if it's late and I'm tired.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;into&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;it's the weekend&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;it's late&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm tired&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implying an action will be taken when one of the above elements at the 1st level of depth is true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know when I hear the sentence that it means &quot;its the weekend&quot; OR (&quot;it's late&quot; AND &quot;I'm tired&quot;), but how could this be determined computationally?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can an existing python/other library do this?&lt;/p&gt;&#xA;" OwnerUserId="10623" LastEditorUserId="2444" LastEditDate="2020-06-02T23:14:33.613" LastActivityDate="2020-06-02T23:14:33.613" Title="How to parse conjunctions in natural language processing?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7881" PostTypeId="1" CreationDate="2018-09-07T23:43:14.730" Score="4" ViewCount="668" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Connect6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Connect6&lt;/a&gt; is an example of a game with a very high branching factor. It is about 45 thousand, dwarfing even the impressive Go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms can you use on games with such high branching factors? I tried MCTS (soft rollouts, counting a ply as placing one stone), but it does not even block the opponent, due to the high branching factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of Connect6, there are stronger AIs out there, but they aren't described in any research papers that I know of.&lt;/p&gt;&#xA;" OwnerUserId="18006" LastActivityDate="2018-09-08T10:52:44.217" Title="Algorithms for games with very high branching factors (Connect6)" Tags="&lt;algorithm&gt;&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;&lt;combinatorial-games&gt;&lt;branching-factors&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7882" PostTypeId="1" CreationDate="2018-09-08T00:05:51.540" Score="2" ViewCount="31" Body="&lt;p&gt;Formal semantics of natural language perceives sentences as logical expressions. Full paragraphs and even stories of natural language texts are researched and formalized using discourse analysis (Discourse Representation Theory is one example). My question is - is there research trend that applied the notion &quot;discourse&quot; to images, sounds and even animation? Is there such a notion as &quot;visual discourse&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google gives very few and older research papers, so - maybe the field exists, but it uses different terms and Google can not relate those terms to my keyword &quot;visual discourse&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically - there are visual grammars and other pattern matching methods that can discover objects in the picture and relate them. But one should be able to read whole store from the picture (musical piece, multimedia content) and I imagine that such reading can be researched by multimedia discourse analysis. But there is no work under such terms. How it is done and named in reality?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2018-09-08T00:11:36.403" LastActivityDate="2018-09-08T00:11:36.403" Title="VIsual/musical/multimedia discourse (analysis) - are there such notions?" Tags="&lt;image-recognition&gt;&lt;sentiment-analysis&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7883" PostTypeId="2" ParentId="7880" CreationDate="2018-09-08T01:37:08.420" Score="1" Body="&lt;p&gt;This seems not easy for NLP. I doubt that state-of-the-art NLP tools can reliably determine the correct hierarchical structure of independent clauses. Examples below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://tomato.banatao.berkeley.edu:8080/parser/parser.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berkeley parser&lt;/a&gt; gets it basically right in the sense that it can put &lt;code&gt;its late&lt;/code&gt; and &lt;code&gt;I'm tired&lt;/code&gt; on parallel, and they together on parallel with &lt;code&gt;the weekend&lt;/code&gt;. But still not perfect (&lt;code&gt;the weekend&lt;/code&gt; should be in the same subtree with &lt;code&gt;It's&lt;/code&gt; rather than &lt;code&gt;if its late and I'm tired&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u7Dct.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u7Dct.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://nlp.stanford.edu:8080/parser/index.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford parser&lt;/a&gt;, which is available in Python (NLTK), incorrectly parsed &lt;code&gt;I\'m tired&lt;/code&gt; to the same level of &lt;code&gt;I drink orange juice&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jQZ9c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jQZ9c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-09-08T01:37:08.420" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7885" PostTypeId="2" ParentId="7865" CreationDate="2018-09-08T07:12:46.867" Score="5" Body="&lt;p&gt;As CNN contains convolution operation, but DNN uses constructive divergence for training. CNN is more complex in terms of Big O notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;See &lt;a href=&quot;https://arxiv.org/pdf/1412.1710.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Networks at Constrained Time Cost&lt;/a&gt; for more details about the time complexity of CNNs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/q/13612/2444&quot;&gt;What is the time complexity of the forward pass algorithm of a neural network?&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/5728/2444&quot;&gt;What is the time complexity for training a neural network using back-propagation?&lt;/a&gt; for more details about the time complexity of the forward and backward passes of an MLP&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18010" LastEditorUserId="2444" LastEditDate="2020-05-19T19:46:49.420" LastActivityDate="2020-05-19T19:46:49.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7886" PostTypeId="2" ParentId="7875" CreationDate="2018-09-08T10:31:12.013" Score="1" Body="&lt;p&gt;In order to be on the same page, you should give references about &quot;technological singularity&quot;, as it comprises multiple fields (mathematics, statistics, philosophy of science, epistemology, sociology, politics, economics, to mention a few).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, when you consider concepts related to &lt;strong&gt;adj + ai&lt;/strong&gt; (where adj = {weak, strong, full, narrow, ...}), the breath of speculation is quite large and &lt;em&gt;in fieri&lt;/em&gt;, so as a developer (where for developer I assume you work on coding-related problems, not the project manager at google x and the like) I would not be worried, unless you are enjoying a cup of tea with your colleagues during a break.&lt;/p&gt;&#xA;" OwnerUserId="7988" LastActivityDate="2018-09-08T10:31:12.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7887" PostTypeId="2" ParentId="7881" CreationDate="2018-09-08T10:52:44.217" Score="5" Body="&lt;p&gt;Typically, Monte-Carlo Tree Search (MCTS) actually is the go-to &quot;solution&quot; for such problems with large branching factors. I can understand that &quot;vanilla&quot; MCTS may still have unsatisfactory performance, but there is a plethora of extensions/enhancements available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have experience with the specific game you mentioned (Connect6), but from a quick look at how the game works, I imagine there will be a huge number of transpositions in the search tree (positions that are the same but can be reached through multiple different paths in the search tree). This will especially be very common if you treat placing one stone as a single ply; every &quot;combined move&quot; (of placing two stones in two positions subsequently) can be reached in two different ways, simply by switching the order in which the player places them. There has been research in using Transposition Tables with MCTS, so that may be a promising direction to look into.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also suspect there will be great value in using Deep (Reinforcement) Learning approaches. If there is a large board on which to place stones, there will likely be many moves that are &quot;absurd&quot; and can easily be dismissed altogether by Deep Learning approaches (e.g., placing stones far away in a corner of the board where none of the &quot;action&quot; is going on). Vanilla MCTS, without Deep Learning extensions, will not be able to recognize and dismiss such absurd moves, and play them way too often (in Play-Out but also Selection phase due to the high branching factor). The most obvious source of inspiration here would be AlphaGo Zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there's definitely some published research on Connect6 AI (and even MCTS in Connect6). For example: &lt;a href=&quot;http://web.csie.ndhu.edu.tw/sjyen/Journal/0022011IEEE.pdf&quot; rel=&quot;noreferrer&quot;&gt;Two-Stage Monte Carlo Tree Search for Connect6&lt;/a&gt;. You can likely also find more relevant research by checking that paper's list of References, and &lt;a href=&quot;https://scholar.google.nl/scholar?cites=2123165069157963257&amp;amp;as_sdt=2005&amp;amp;sciodt=0,5&quot; rel=&quot;noreferrer&quot;&gt;checking later papers on google scholar that cite this one&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-08T10:52:44.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7888" PostTypeId="2" ParentId="7875" CreationDate="2018-09-08T12:07:29.220" Score="10" Body="&lt;p&gt;I can say that among AI researchers I interact with, it &lt;strong&gt;far&lt;/strong&gt; more common to view it as wild speculation than as settled fact. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is borne out by &lt;a href=&quot;https://nickbostrom.com/papers/survey.pdf&quot; rel=&quot;noreferrer&quot;&gt;surveys of AI researchers&lt;/a&gt;, with 80% thinking strong forms of AI will emerge in &quot;more than 50 years&quot; or &quot;never&quot;, and just a few percent thinking that such forms of AI are &quot;near&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Software Developers are not the same as AI researchers, and I have found the Singularity myth to be much more widespread among developers. It has a nice ring to it: Computers keep getting faster, at some point they'll be faster than brains, at that point we just simulate brains. Soon after, we simulate something better than brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the reasons AI researchers are less optimistic are rooted in the fact that we still don't have a good understanding of human intelligence, or even enough of an understanding of the brain to simulate it. For example, in the last two weeks we have discovered &lt;a href=&quot;http://www.sciencemag.org/news/2018/08/mysterious-new-brain-cell-found-people&quot; rel=&quot;noreferrer&quot;&gt;previously unknown types of brain cells&lt;/a&gt;. This gives the (correct) impression that even if we had a fast enough computer, we are not at all close to being able to accurately simulate a human brain. We don't really know what a human brain is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if we did know that, simulations are necessarily lossy. We may not have good simulation techniques. Even if we did have good techniques, we may simulate the brain and discover our simulation does not behave as expected for reasons that we don't understand. This is very probable when simulating new systems. In some sense, proponents of the Singularity resemble people predicting that &lt;a href=&quot;https://en.wikipedia.org/wiki/History_of_numerical_weather_prediction&quot; rel=&quot;noreferrer&quot;&gt;weather control&lt;/a&gt; was near in the 1940s. After all, we could simulate simple weather patterns already then, and generate forecasts that sort of worked. How much more complex could it really be to generate perfect forecasts?&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-08T12:07:29.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7890" PostTypeId="2" ParentId="3850" CreationDate="2018-09-09T04:51:09.103" Score="3" Body="&lt;p&gt;Old question, but I thought it's worth one practical answer. I happened to stumble upon it right after looking at a guide of how to build such neural network, demonstrating echo of python's randint as an &lt;a href=&quot;https://machinelearningmastery.com/learn-echo-random-integers-long-short-term-memory-recurrent-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;. Here is the final code without detailed explanation, still quite simple and useful in case the link goes offline:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from random import randint&#xA;from numpy import array&#xA;from numpy import argmax&#xA;from pandas import concat&#xA;from pandas import DataFrame&#xA;from keras.models import Sequential&#xA;from keras.layers import LSTM&#xA;from keras.layers import Dense&#xA;&#xA;# generate a sequence of random numbers in [0, 99]&#xA;def generate_sequence(length=25):&#xA;    return [randint(0, 99) for _ in range(length)]&#xA;&#xA;# one hot encode sequence&#xA;def one_hot_encode(sequence, n_unique=100):&#xA;    encoding = list()&#xA;    for value in sequence:&#xA;        vector = [0 for _ in range(n_unique)]&#xA;        vector[value] = 1&#xA;        encoding.append(vector)&#xA;    return array(encoding)&#xA;&#xA;# decode a one hot encoded string&#xA;def one_hot_decode(encoded_seq):&#xA;    return [argmax(vector) for vector in encoded_seq]&#xA;&#xA;# generate data for the lstm&#xA;def generate_data():&#xA;    # generate sequence&#xA;    sequence = generate_sequence()&#xA;    # one hot encode&#xA;    encoded = one_hot_encode(sequence)&#xA;    # create lag inputs&#xA;    df = DataFrame(encoded)&#xA;    df = concat([df.shift(4), df.shift(3), df.shift(2), df.shift(1), df], axis=1)&#xA;    # remove non-viable rows&#xA;    values = df.values&#xA;    values = values[5:,:]&#xA;    # convert to 3d for input&#xA;    X = values.reshape(len(values), 5, 100)&#xA;    # drop last value from y&#xA;    y = encoded[4:-1,:]&#xA;    return X, y&#xA;&#xA;# define model&#xA;model = Sequential()&#xA;model.add(LSTM(50, batch_input_shape=(5, 5, 100), stateful=True))&#xA;model.add(Dense(100, activation='softmax'))&#xA;model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['acc'])&#xA;# fit model&#xA;for i in range(2000):&#xA;    X, y = generate_data()&#xA;    model.fit(X, y, epochs=1, batch_size=5, verbose=2, shuffle=False)&#xA;    model.reset_states()&#xA;# evaluate model on new data&#xA;X, y = generate_data()&#xA;yhat = model.predict(X, batch_size=5)&#xA;print('Expected:  %s' % one_hot_decode(y))&#xA;print('Predicted: %s' % one_hot_decode(yhat))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've just tried and it indeed works quite well! Took just a couple of minutes on my old slow netbook. Here's my very own output, different from the link above and you can see match isn't perfect, so I suppose exit criteria is a bit too permissive:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA; - 0s - loss: 0.2545 - acc: 1.0000&#xA;Epoch 1/1&#xA; - 0s - loss: 0.1845 - acc: 1.0000&#xA;Epoch 1/1&#xA; - 0s - loss: 0.3113 - acc: 0.9500&#xA;Expected:  [14, 37, 0, 65, 30, 7, 11, 6, 16, 19, 68, 4, 25, 2, 79, 45, 95, 92, 32, 33]&#xA;Predicted: [14, 37, 0, 65, 30, 7, 11, 6, 16, 19, 68, 4, 25, 2, 95, 45, 95, 92, 32, 33]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18025" LastActivityDate="2018-09-09T04:51:09.103" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7891" PostTypeId="1" AcceptedAnswerId="8069" CreationDate="2018-09-09T05:11:33.703" Score="3" ViewCount="377" Body="&lt;p&gt;I want to generate images of childrens' drawings consistent with the developmental state of children of a given age.  The training data set will include drawings made by real children in a school setting.  The generated images will be used for developmental analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have heard that Generative Adversarial Networks are a good tool for this kind of problem. If this is true, how would I go about applying a GAN to this challenge?&lt;/p&gt;&#xA;" OwnerUserId="18027" LastEditorUserId="4302" LastEditDate="2018-09-13T08:15:36.910" LastActivityDate="2018-09-21T00:19:16.933" Title="How to use a Generative Adversarial Network to generate images for developmental analysis?" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="2" CommentCount="8" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7892" PostTypeId="2" ParentId="5982" CreationDate="2018-09-09T08:07:49.907" Score="1" Body="&lt;p&gt;As Andreas has commented this is a problem of statistical language model (a probability distribution over a sequence of words). The important thing you need is a hash table mapping fixed-length to the expected ending chains of words in your dictionary.&lt;br&gt;&#xA;Things that can make your prediction better:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Add better and more words to your dictionary.&lt;/li&gt;&#xA;&lt;li&gt;Use text expansion.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What you are looking for will require a pinch of Reinforcement Learning too. You need to figure out a way to penalize and award the predictions and then use the result in future. Your case also requires you to build your own corpus, which is the hardest part. If your corpus is good, it will give better results.&lt;br&gt;&#xA;This is the &lt;a href=&quot;http://www.stats.ox.ac.uk/~teh/research/compling/WooGasArc2011a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;research paper&lt;/a&gt; that will help you a lot.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2018-09-09T08:07:49.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7893" PostTypeId="2" ParentId="3850" CreationDate="2018-09-09T08:38:05.427" Score="2" Body="&lt;p&gt;Adding to what Demento said, the extent of randomness in the Random Number Generation Algorithm is the key issue. Following are some designs that can make the RNG weak:&lt;br&gt;&#xA;&lt;strong&gt;Concealed Sequences&lt;/strong&gt;&lt;br&gt;&#xA;Suppose this is the previous few sequences of characters generated: (Just an example, for the practical use larger range, is used)&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lwjVJA&#xA;Ls3Ajg&#xA;xpKr+A&#xA;XleXYg&#xA;9hyCzA&#xA;jeFuNg&#xA;JaZZoA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Initially, you can't observe any pattern in the generations but changing them to Base64 encoding and then to hex, we get the following:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;9708D524&#xA;2ECDC08E&#xA;C692ABF8&#xA;5E579762&#xA;F61C82CC&#xA;8DE16E36&#xA;25A659A0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now if we subtract each number form the previous one, we get this:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FF97C4EB6A&#xA;97C4EB6A&#xA;FF97C4EB6A&#xA;97C4EB6A&#xA;FF97C4EB6A&#xA;FF97C4EB6A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This indicates that the algorithm just adds 0x97C4EB6A to the previous value, truncates the result to a 32-bit number, and Base64-encodes the data.&lt;br&gt;&#xA;The above is a basic example. Today's ML algorithms and systems are capable enough to learn and predict more complex patterns.&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;Time Dependency&lt;/strong&gt;&lt;br&gt;&#xA;Some RNG algorithms use time as the &lt;em&gt;major input&lt;/em&gt; for generating random numbers, especially the ones created by developers themselves to be used within their application.&lt;br&gt;&lt;br&gt;&#xA;Whenever weak RNG algorithms is implemented that appear to be stochastic, they &lt;strong&gt;can be extrapolated forwards or backwards with perfect accuracy&lt;/strong&gt; in case sufficient dataset is available.&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2018-09-09T08:38:05.427" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7895" PostTypeId="1" AcceptedAnswerId="7902" CreationDate="2018-09-09T19:07:09.927" Score="8" ViewCount="246" Body="&lt;p&gt;Drawing parallels between Machine Learning techniques and a human brain is a dangerous operation. When it is done successfully, it can be a powerful tool for vulgarisation, but when it is done with no precaution, it can lead to major misunderstandings.&lt;/p&gt;&#xA;&lt;p&gt;I was recently attending a conference where the speaker described Experience Replay in RL as a way of making the net &amp;quot;dream&amp;quot;.&#xA;I'm wondering how true this assertion is. The speaker argued that a dream is a random addition of memories, just as experience replay. However, I doubt the brain remembers its dream or either learns from it. What is your analysis?&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="2444" LastEditDate="2021-01-31T12:58:04.767" LastActivityDate="2021-01-31T12:58:04.767" Title="Is Experience Replay like dreaming?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7896" PostTypeId="1" CreationDate="2018-09-09T20:31:07.373" Score="9" ViewCount="802" Body="&lt;p&gt;In the &lt;a href=&quot;http://rail.eecs.berkeley.edu/deeprlcourse/static/homeworks/hw2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;homework for the Berkeley RL class&lt;/a&gt;, problem 1, it asks you to show that the policy gradient is still unbiased if the baseline subtracted is a function of the state at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \triangledown _\theta \sum_{t=1}^T \mathbb{E}_{(s_t,a_t) \sim p(s_t,a_t)} [b(s_t)] = 0   $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling through what the first step of such a proof might be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me in the right direction? My initial thought was to somehow use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_total_expectation&quot; rel=&quot;nofollow noreferrer&quot;&gt;law of total expectation&lt;/a&gt; to make the expectation of &lt;span class=&quot;math-container&quot;&gt;$b(s_t)$&lt;/span&gt; conditional on &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, but I am not sure.&lt;/p&gt;&#xA;" OwnerUserId="18043" LastEditorUserId="2444" LastEditDate="2020-06-10T16:39:51.490" LastActivityDate="2020-06-10T16:42:11.700" Title="Why is baseline conditional on state at some timestep unbiased?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proofs&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7897" PostTypeId="1" AcceptedAnswerId="8067" CreationDate="2018-09-09T20:33:22.743" Score="8" ViewCount="168" Body="&lt;p&gt;Is there research that employs realistic models of neurons? Usually, the model of a neuron for a neural network is quite simple as opposed to the realistic neuron, which involves hundreds of proteins and millions of molecules (or even greater numbers). Is there research that draws implications from this reality and tries to design realistic models of neurons?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particularly, recently, &lt;a href=&quot;https://en.wikipedia.org/wiki/Rosehip_neuron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosehip neuron&lt;/a&gt; was discovered. Such neuron can be found only in human brain cells (and in no other species). Are there some implications for neural network design and operation that can be drawn by realistically modelling this Rosehip neuron?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-03-19T21:49:49.583" LastActivityDate="2019-03-19T21:49:49.583" Title="Is there research that employs realistic models of neurons?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;&lt;brain&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7898" PostTypeId="2" ParentId="7897" CreationDate="2018-09-09T21:03:42.210" Score="4" Body="&lt;p&gt;It looks like you really have two questions here. I'll try to answer the first one, and you should think about making a separate question for the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is research into using simulated models of biologically realistic neurons. While there are large projects like the &lt;a href=&quot;https://www.humanbrainproject.eu/en/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Brain Project&lt;/a&gt; aimed at simulating human brains, there is also a lot of lower-level AI research. &lt;a href=&quot;http://compneuro.uwaterloo.ca/publications/choo2018.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPAWN&lt;/a&gt; is an interesting system that got a lot of press a few years ago, and has continued to be developed since then. It uses realistic neurons to simulate several brain-regions at once, creating a surprisingly general AI system that could perform many types of motor and vision tasks using the same basic design.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-09T21:03:42.210" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7899" PostTypeId="1" AcceptedAnswerId="7931" CreationDate="2018-09-09T22:20:04.963" Score="2" ViewCount="95" Body="&lt;p&gt;Image Captioning is a hot research topic in the AI community. There are considerable image captioning models for research usage such as NIC, Neural Talk 2 etc. But can these research models be used for commercial purpose? Or we should build much more complex structured ones for commercial usage? Or if we can make some improvements based these models to meet the business applications situation? If so, what improvements should we take? Are there any existing commercial Image Captioning applications can be referenced?&lt;/p&gt;&#xA;" OwnerUserId="14948" LastActivityDate="2018-09-12T06:23:18.653" Title="How to build a commercial image captioning system?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7900" PostTypeId="1" CreationDate="2018-09-10T02:08:14.790" Score="1" ViewCount="160" Body="&lt;p&gt;For my university project, I am planning to build a face recognition/ occupation recognition programme. However, rather than using the existing Haar cascade(for age and gender) I am planning to use Face API which seems far more accurate than the former. My question is is it possible to somehow combine my trained data for Haar cascade(for occupation) with Face API since Face API doesn't have the option to recognize occupation(such as students/office workers from their appearance)?&lt;/p&gt;&#xA;" OwnerUserId="17651" LastEditorUserId="17651" LastEditDate="2018-09-10T04:05:39.353" LastActivityDate="2018-11-11T12:01:30.373" Title="Occupation detection using Face API" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;python&gt;" AnswerCount="1" CommentCount="7" ClosedDate="2020-06-02T23:12:16.723" ContentLicense="CC BY-SA 4.0" />
  <row Id="7901" PostTypeId="2" ParentId="1308" CreationDate="2018-09-10T03:35:51.377" Score="3" Body="&lt;p&gt;Self-flight is already &lt;strong&gt;running on all&lt;/strong&gt; commercial airlines, but only at the cruising altitude. When you flight from London to New York, most of the time the pilots are monitoring the autopilot systems. Running AI on cruising altitude is arguably a simper problem than self-driving cars. The system has been in-place for years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real danger of airlines is taking off/landing. As you can imagine, the costs for a bad judgement could be quite serious. Anything could happen while the plane is still under the clouds. While our AI technology is improving, it will &lt;strong&gt;never&lt;/strong&gt; replace human intervention, especially when the costs of collision is very high. If you don't believe me, google for how Facebook is hiring humans for reading &quot;fake news&quot;. Facebook has publicity acknowledged AI will never be sufficient for them, then why would AI good enough for replacing well trained pilots?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI on a commercial airplane is already taking place, but there is no possibility ever it will take over the world. We will always need pilots who knows how to work with the system, making important decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trained pilots + good AI assistance will make the world a better place.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastEditorUserId="6014" LastEditDate="2018-09-10T03:41:13.010" LastActivityDate="2018-09-10T03:41:13.010" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7902" PostTypeId="2" ParentId="7895" CreationDate="2018-09-10T06:20:16.787" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The speaker argued that a dream is a random addition of memories, just as experience replay.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The speaker is taking some liberties due to a general lack of scientific understanding of what dreams are. We don't even have strong consensus on why sleep is a necessary feature of animals, let alone what part dreaming plays in it. However, there are some widely-accepted theories, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3079906/&quot; rel=&quot;noreferrer&quot;&gt;with supporting evidence, that dreams are part of a learning and memorisation process&lt;/a&gt;. Studies that manipulate sleep or dreaming have shown changes in the speed that skills are learned for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience replay in reinforcement learning is a far more precise and well-understood affair, whereby individual time steps that occurred in the past are visited and re-assessed in light of current knowledge about long-term value, at random. If dreams were really like experience replay as it is practiced in RL today, then they would consist of a random jumble of tiny seemingly inconsequential events strung together, and all taken very exactly from the events of the past day. &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0185262&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Sometimes&lt;/em&gt; dreams do contain content like this&lt;/a&gt;, but typically the content is far more varied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taken with a large dose of artistic license, then yes, the speaker is referring to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4704085/&quot; rel=&quot;noreferrer&quot;&gt;real theories and conjectures about dreaming, that do have scientific support&lt;/a&gt;. Although it is equally good to draw parallels between dreams and a higher-level &lt;em&gt;management&lt;/em&gt; of the memory or experience replay data - which items to replay, and which to keep, depending on what is salient about the information. For instance, there is &lt;a href=&quot;http://www.jneurosci.org/content/37/3/464&quot; rel=&quot;noreferrer&quot;&gt;good evidence that dreams help filter what is &lt;em&gt;forgotten&lt;/em&gt;&lt;/a&gt;, and also evidence that events associated with strong emotional state are more likely to feature in dreams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to separate the speaker's analogy, and any suggestion that a current reinforcement learning agent has a subjective experience. We are still a long way away from anything like that, and other similar use of a dreaming metaphor in machine learning - e.g. &quot;Deep Dream&quot; - is equally not an assertion that the devices are having an experience of any kind.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-10T09:24:38.737" LastActivityDate="2018-09-10T09:24:38.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7903" PostTypeId="1" CreationDate="2018-09-10T07:47:16.083" Score="1" ViewCount="61" Body="&lt;p&gt;I have an average laptop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I connect specialized AI neural network processors (say, Intel Nvidia or Intel Nervana &lt;a href=&quot;https://venturebeat.com/2018/05/23/intel-unveils-nervana-neural-net-l-1000-for-accelerated-ai-training/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://venturebeat.com/2018/05/23/intel-unveils-nervana-neural-net-l-1000-for-accelerated-ai-training/&lt;/a&gt;) to thelaptop.&#xA;Should I buy some external motherboard or even server unit with NN processors inside or is there available more lightweight solution like external HDD?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2018-09-10T11:02:25.400" LastActivityDate="2018-09-10T11:02:25.400" Title="How to connect AI neural network processor to laptop?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7905" PostTypeId="2" ParentId="7903" CreationDate="2018-09-10T10:59:35.610" Score="2" Body="&lt;p&gt;Check of deep learning box on the internet.&#xA;You will have to think hard in order to make a good and cost effective one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are a few people who have made one of their own and shared their experiences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some medium articles:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/yanda/building-your-own-deep-learning-dream-machine-4f02ccdb0460&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building your own Deep Learning dream machine&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/intuitionmachine/building-a-270-teraflops-deep-learning-box-of-under-10-000-2d790b0ae2ec&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building a 270* Teraflops Deep Learning Box for Under $10,000&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16920" LastActivityDate="2018-09-10T10:59:35.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7906" PostTypeId="2" ParentId="7897" CreationDate="2018-09-10T11:18:20.057" Score="3" Body="&lt;p&gt;It is true that the current Machine learning is based on treating neurons as a component in the whole complexity , mesh of neurons. The focus is more on the architecture rather than understanding or imitating the basic block of it more clearly , i.e. the neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anirban Bandhopadhyay is a biologist and Neurologist who has studied how the harmony changes the memory element and decision making power in microtubles inside the neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, is the snippet of him explaining , and trying to see what exactly computation is , and how the brain does computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.closertotruth.com/interviews/48850&quot; rel=&quot;nofollow noreferrer&quot;&gt;How does the Brain Act?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16920" LastActivityDate="2018-09-10T11:18:20.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7907" PostTypeId="1" AcceptedAnswerId="7909" CreationDate="2018-09-10T12:24:46.590" Score="3" ViewCount="110" Body="&lt;p&gt;I'm trying to learn AI and thinking to apply it to our system. We have an application for the translation industry. What we are doing now is the coordinator &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; assigns a file to a translator &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. The coordinator usually considers these criteria (but not limited to):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the deadline of the file and availability of a translator&lt;/li&gt;&#xA;&lt;li&gt;the language pair that the translator can translate&lt;/li&gt;&#xA;&lt;li&gt;is the translator already reached his target? (maybe we can give the file to other translators to reach their target)&lt;/li&gt;&#xA;&lt;li&gt;the difficulty level of the file for the translator (basic translation, medical field, IT field)&lt;/li&gt;&#xA;&lt;li&gt;accuracy of translator&lt;/li&gt;&#xA;&lt;li&gt;speed of translator&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Given the following, is it possible to make a recommendation to the coordinator, to whom she can assign a particular file?&lt;/p&gt;&#xA;&lt;p&gt;What are the methods/topics that I need to research?&lt;/p&gt;&#xA;&lt;p&gt;(I'm considering javascript as the primary tool, and maybe python if javascript will be more of a hindrance in implementation.)&lt;/p&gt;&#xA;&lt;p&gt;In addition to suggesting a translator, we are also looking into suggesting the &amp;quot;deadline of the translator&amp;quot;. Basically, we have &amp;quot;deadline of the customer&amp;quot; and &amp;quot;deadline of the translator&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The reason for this is that, if the translators are occupied throughout the day, it makes sense to suggest it to a busy translator but allow him to finish it until next day.&lt;/p&gt;&#xA;" OwnerUserId="18053" LastEditorUserId="2444" LastEditDate="2021-01-23T03:19:46.120" LastActivityDate="2021-01-23T03:19:46.120" Title="What AI technique should I use to assign a person to a task?" Tags="&lt;optimization&gt;&lt;constraint-satisfaction-problems&gt;&lt;planning&gt;&lt;linear-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7909" PostTypeId="2" ParentId="7907" CreationDate="2018-09-10T14:47:53.330" Score="4" Body="&lt;p&gt;What you have could be well described as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;Task Allocation problem&lt;/a&gt;, which is studied as part of the &lt;em&gt;planning&lt;/em&gt; subfield of AI. Chapters 10 &amp;amp; 11 of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; provide a good overview of this area, although I think they don't talk too much about Task Allocation in particular.&lt;/p&gt;&#xA;&lt;p&gt;There are two basic approaches to this problem: &lt;em&gt;centralized&lt;/em&gt; approaches, and &lt;em&gt;decentralized&lt;/em&gt; approaches.&lt;/p&gt;&#xA;&lt;p&gt;In centralized approaches, the properties of each task (or sub-task) and the skills of each processing entity are recorded in a central database. The task is phrased as an optimization problem. For example, given the skills of the processors and the tasks' types, find the schedule that minimizes average processing time (or cost, or usage of rare-resource types, or whatever you're interested in). Common approaches include phrasing the optimization task as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear-programming problem&lt;/a&gt;; phrasing the problem as a graph and using something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Graphplan&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;graphplan&lt;/em&gt;&lt;/a&gt; algorithm; or phrasing the problem as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_satisfaction_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;constraint satisfaction problem&lt;/a&gt; and using some kind of heuristic-guided local search.&lt;/p&gt;&#xA;&lt;p&gt;There are all kinds of other more modern techniques too. I'm not aware of a survey paper for translation tasks in particular, but there are &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-18299-5_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;lots&lt;/a&gt; of examples in &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.6412&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;robotics&lt;/a&gt; and &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/7051215/&quot; rel=&quot;nofollow noreferrer&quot;&gt;distributed computing&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Although good AI techniques exist for scheduling the task, they are predicated on being able to quantify the tasks' properties and the abilities of the agents, and on the translators accepting the decisions of the system. If you want an interactive system, you may need to look at techniques from Natural Language Processing. The work on &lt;a href=&quot;http://www.aaai.org/ocs/index.php/SSS/SSS11/paper/download/2375/2893&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mixed Initiative Scheduling Systems&lt;/a&gt; might also be relevant if you have to go that route.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-23T03:13:29.017" LastActivityDate="2021-01-23T03:13:29.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7910" PostTypeId="2" ParentId="7794" CreationDate="2018-09-10T20:27:36.370" Score="2" Body="&lt;p&gt;&lt;strong&gt;Approaches&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two main approaches to detecting any human readable representation of a discrete quantity within text.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Detect well known and stable patterns in the input stream and by adjacency determine the output stream.&lt;/li&gt;&#xA;&lt;li&gt;Windowing through the text in the input stream and directly detect the quantities.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are other approaches and there are hybrids of these two or one of these and the other approaches, but these two are the theoretically most straightforward and likely to produce both reliability and accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Re-entrant Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether the training involves re-entrant learning techniques, such as reinforcement, is a tangential issue that this answer will not address, but know that whether all training is solely a deployment component or whether adaptation and/or convergence occurs in real time is an architectural decision to be made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Practical Concerns&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically, the outputs of each recognition are as follows.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Starting index&lt;/li&gt;&#xA;&lt;li&gt;Ending index&lt;/li&gt;&#xA;&lt;li&gt;Integer year or null&lt;/li&gt;&#xA;&lt;li&gt;Integer day of year or null&lt;/li&gt;&#xA;&lt;li&gt;Integer hour in military time or null&lt;/li&gt;&#xA;&lt;li&gt;Minute or null&lt;/li&gt;&#xA;&lt;li&gt;Second or null&lt;/li&gt;&#xA;&lt;li&gt;Time zone or null&lt;/li&gt;&#xA;&lt;li&gt;Probability the recognition unit was correctly identified&lt;/li&gt;&#xA;&lt;li&gt;Probability the recognition produced accurate results&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also practically, the input must either be from within one particular locale's norms in terms of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calendar,&lt;/li&gt;&#xA;&lt;li&gt;Time,&lt;/li&gt;&#xA;&lt;li&gt;Written language,&lt;/li&gt;&#xA;&lt;li&gt;Character encoding, and&lt;/li&gt;&#xA;&lt;li&gt;Collation,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... or ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The learning must occur using training sets that include the locales that will be encountered during system use&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... or ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Much of the locale specific syntax must be normalized to a general date and time language such as this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;जनवरी --&gt; &lt;strong&gt;D_01&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enero --&gt; &lt;strong&gt;D_01&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Janúar --&gt; &lt;strong&gt;D_01&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;so that Filipino and Icelandic names for the first month of the year enter the artificial network as the same binary pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**Date and Time Specifically*&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of 1. above, which is semi-heuristic in nature, and assuming that the locale is entirely en-US.utf-8, the CASE INSENSITIVE patterns for a PCRE library or equivalent to use as a search orientation heuristic include the following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(^|[^0-9a-z])(19|20|21)[0-9][0-9])([^0-9a-z]|$)&#xA;(^|[^0-9a-z])(Mon|Monday|Tue|Tues|Tuesday|Wed|Wednesday|Thu|Thur|Thurs|Thursday|Fri|Friday|Sat|Saturday|Sun)([^0-9a-z]|$)&#xA;(^|[^0-9a-z])(Jan|January|Feb|February|Mar|March|Apr|April|May|Jun|June|Jul|July|Aug|August|Sep|Sept|September|Oct|October|Nov|November|Dec|December)([^0-9a-z]|$)&#xA;(^|[^0-9a-z])(Today|Yesterday|Tomorrow)([^0-9a-z]|$)&#xA;(^|[^0-9])[AP]M|[AP][.]M[.]|Noon|Midnight)([^0-9a-z]|$)&#xA;(^|[^a-z])(0?[1-9])(:[0-5][0-9]){1,2}([^a-z]|$)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There should be others for time, hyphenated or slash delimited dates, or time zone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The positions and normalized encoding of these date and time artifacts are then substituted into the artificial network inputs instead of the original text in the stream, reducing redundancy and improving both the speed of training and the resulting accuracy and reliability of recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of 2. above, the entire burden of recognition is left to the artificial network.  The advantage is less reliance on date and time conventions.  The disadvantage is a much larger burden placed on training data variety and training epochs, meaning a much higher burden on computing resources and the pacience of the stake holder for the project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Windowing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An overlapping windowing strategy is necessary.  Unlike FFT spectral analysis in real time, the windowing must be rectangular, because the size of the window is the width of the input layer of the artificial network.  Experimenting with the normalization of input such that the encoding of text and data and time components entering the input layer could greatly vary the results in terms of training speed, recognition accuracy, reliability, and adaptability to varying statistical distributions of date and time instances and relationships.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-10T20:27:36.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7911" PostTypeId="1" AcceptedAnswerId="7913" CreationDate="2018-09-10T21:25:45.860" Score="2" ViewCount="512" Body="&lt;p&gt;The goal is to implement an artificial network that, based on training samples labelled with positive integers, outputs a positive integer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I am not searching for the correct thing but all the examples I have found have shown classifiers using a sigmoid function that outputs a fixed or floating point number within the range &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;0 \to 1&amp;gt;$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any point in the right direction or python/toy code would be very appreciated!&lt;/p&gt;&#xA;" OwnerUserId="18070" LastEditorUserId="4302" LastEditDate="2018-09-20T22:19:44.147" LastActivityDate="2018-09-20T22:19:44.147" Title="How to implement an artificial network that outputs an integer within a range?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;ai-basics&gt;&lt;prediction&gt;&lt;knowledge-representation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7912" PostTypeId="2" ParentId="5454" CreationDate="2018-09-10T21:56:59.677" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So my question now is, after the training is complete, which of the two networks mu or mu' should be used for making predictions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After training is &quot;complete&quot;, use mu, i.e., the online network because this is the most updated actor network you have.  In the most ideal case, mu' will be equal to mu.  But if your training is truly complete, mu = mu'.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Equivalently to the training phase I suppose that mu should be used without the exploration noise but since it is mu' that is used during the training for predicting the &quot;true&quot; (unnoisy) action for the reward computation, I'm apt to use mu'.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the training phase, you must use exploration noise when mu maps states to actions.  The actor is a policy gradient, i.e. based on policy iteration.  So it will directly map to an action everytime.  If no exploratory noise is added, it will map to the same value every time, unless you update your network weights.  What this means is that if you're given some states, s.  Without exploration noise, the actor will always output the same actions, and never explore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mu' is ONLY used for the supervised learning portion of DDPG, where it stabilizes training of 2 neural networks that essentially train off each other.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine this problem: a worker and his supervisor in a factory.  Both the worker and the supervisor knows nothing at the start (initialized neural networks).  The worker's first action is to pick up a 200 lb box.  The supervisor then positively rewards him. So from this, the worker learned that picking up heavy objects = good. The worker then picks up another heavy box, and this time the supervisor yells at him because the supervisor learned that its dangerous.  So now the worker himself is confused, because he does not know if picking up heavy objects is good or bad because the supervisor himself changes his mind.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In DDPG, the actor and critic behaves the same way.  So we introduce target networks to make it so both the actor and critic don't keep changing their minds, so they can actually learn things.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or does this even matter? If the training was to last long enough shouldn't both versions of the actor have converged to the same state?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is correct, but in reality, that is very rare.&lt;/p&gt;&#xA;" OwnerUserId="17706" LastActivityDate="2018-09-10T21:56:59.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7913" PostTypeId="2" ParentId="7911" CreationDate="2018-09-10T22:01:44.643" Score="3" Body="&lt;p&gt;First of all, sigmoid does not output 0 or 1, it outputs any real number in the range between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, normal neural networks doesn't output binary values, unless the output layer uses the step function as activation function (which is rare).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not really sure if you want the NN to be a classifier or regressor, but it sounds like you want a regressor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regression is when you are interested in the value of the output nevron(s) itself. A simple example is if you want the network to predict the sum of two input neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to change the network from a classifier to a regressor you should probably reduce the number of neurons in the output layer to 1, and change the activation function of that neuron from softmax to the identity function (f(x)=x; which is the same as no activation function at all).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. Provide some more details if this didn't answer your question.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2018-09-10T22:01:44.643" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7914" PostTypeId="1" AcceptedAnswerId="7920" CreationDate="2018-09-10T22:31:34.993" Score="6" ViewCount="237" Body="&lt;p&gt;Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(By perfect, I mean it always wins the game if possible, even against another perfect opponent.)&lt;/p&gt;&#xA;" OwnerUserId="18006" LastEditorUserId="18006" LastEditDate="2018-09-11T02:19:08.160" LastActivityDate="2018-09-11T14:46:43.990" Title="Would AlphaGo Zero become perfect with enough training time?" Tags="&lt;neural-networks&gt;&lt;monte-carlo-tree-search&gt;&lt;alphago&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7915" PostTypeId="2" ParentId="7914" CreationDate="2018-09-11T02:16:35.100" Score="1" Body="&lt;p&gt;Yes &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_Zero&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero&lt;/a&gt; could become undeniably perfect.&lt;/p&gt;&#xA;&lt;p&gt;It has won 100:0 against &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Lee&lt;/a&gt; (which won 4:1 against 18-time world champion (human) &lt;a href=&quot;https://en.wikipedia.org/wiki/Lee_Sedol&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lee Sedol&lt;/a&gt;) and 89:11 against &lt;a href=&quot;https://en.wikipedia.org/wiki/Master_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Master&lt;/a&gt; (which won 60 straight online games against human professional Go players from 29 December 2016 to 4 January 2017).&lt;/p&gt;&#xA;&lt;p&gt;From the &lt;a href=&quot;https://deepmind.com/research/alphago/&quot; rel=&quot;nofollow noreferrer&quot;&gt;official AlphaGo website&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;AlphaGo's 4-1 victory in Seoul, South Korea, in March 2016 was watched by over 200 million people worldwide. It was a landmark achievement that experts agreed was a decade ahead of its time, and earned AlphaGo a 9 dan professional ranking (the highest certification) - the first time a computer Go player had ever received the accolade.&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;From AlphaGo's webpage: &amp;quot;&lt;a href=&quot;https://deepmind.com/blog/alphagos-next-move/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo's next move&lt;/a&gt;&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;We plan to publish one final academic paper later this year that will detail the extensive set of improvements we made to the algorithms’ efficiency and potential to be generalised across a broader set of problems. Just like our &lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;first AlphaGo paper&lt;/a&gt;, we hope that other developers will pick up the baton, and use these new advances to build their own set of strong Go programs.&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;Since our match with Lee Sedol, AlphaGo has become its own teacher, playing millions of high level training games against itself to continually improve.&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-11T02:16:35.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7916" PostTypeId="1" AcceptedAnswerId="7918" CreationDate="2018-09-11T04:19:50.473" Score="2" ViewCount="85" Body="&lt;p&gt;I am currently reading the research paper &lt;a href=&quot;https://arxiv.org/pdf/1706.03686.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Crowd Counting Using Convolutional Neural Network and Markov Random Field&lt;/a&gt;  by Kang Han, Wanggen Wan, Haiyan Yao, and Li Hou. &lt;br/&gt;&#xA;I did not understand the following context properly: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We employ the residual network, which is trained on ImageNet dataset for image classication task, to extract the deep features to represent the density of the crowd. This pre-trained CNN network created a residual item for every three convolution layer to bring the layer of the network to 152. We resize the image patches to the size of 224 × 224 as the input of the model and extract the output of the fc1000 layer to get the 1000 dimensional features. The features are then used to train 5 layers fully connected neural network. The network's input is 1000dimensional, and the number of neurons in the network is given by 100-100-50-50-1. The network's output is the local crowd count&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain the above part in detail? &lt;/p&gt;&#xA;" OwnerUserId="14592" LastEditorUserId="1671" LastEditDate="2018-09-11T18:49:05.603" LastActivityDate="2018-09-11T18:49:05.603" Title="Clarification regarding &quot;Image Crowd Counting Using Convolutional Neural Network and Markov Random Field&quot;" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7917" PostTypeId="2" ParentId="7829" CreationDate="2018-09-11T06:45:26.963" Score="0" Body="&lt;p&gt;I got one &lt;a href=&quot;https://eprints.qut.edu.au/85084/1/Mohd%20Ridzwan_Yaakub_Thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on the integration of opinion mining in CRM, which helps to get an idea about the process involved. &lt;/p&gt;&#xA;" OwnerUserId="17058" LastActivityDate="2018-09-11T06:45:26.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7918" PostTypeId="2" ParentId="7916" CreationDate="2018-09-11T07:44:07.353" Score="3" Body="&lt;p&gt;I will try to do it part by part :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We employ the residual network, which is trained on ImageNet dataset for image classication task, to extract the deep features to represent the density of the crowd.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you look at the figure 2, you can see that they use the Neural network architecture ResNet. This is a deep network, &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is the paper. It has good performance and do image classification.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This pre-trained CNN network created a residual item for every three convolution layer to bring the layer of the network to 152&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you are in the layer k, it means this layer has in input, the ouput of the k-3th layer. See the paper, figure 5 explains it well without much explications needed. Furthermore, Resnet has 3 different architectures wih different number of layers, and they take the deeper one, the 152 layers deep Resnet.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We resize the image patches to the size of 224 × 224 as the input of the model and extract the output of the fc1000 layer to get the 1000 dimensional features&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The input of Resnet is images of size 224x224, so they need to resize them to fit the input requirement of Resnet. The output of Resnet is 1000 because Imagenet is a dataset of 1000 classes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The features are then used to train 5 layers fully connected neural network. The network's input is 1000dimensional, and the number of neurons in the network is given by 100-100-50-50-1.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then they give the output of Resnet to their own Network, which is 5 layer deep. See figure 2 of their paper. Obviously, the input layer has 1000 inputs because of the output of Resnet. The network have layers of 100, 100, 50 and finally 1 neuron. See figure 2.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The network's output is the local crowd count&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think I need to explain it, they want only the number of people in the crowd, so they need only one output. This is obviously not a classifcation problem, but a regression problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has you don't really point out what you don't understand, I don't explain it in details. Feel free to ask more precise question if some part are still blurred to you !&lt;/p&gt;&#xA;" OwnerUserId="17221" LastEditorUserId="17221" LastEditDate="2018-09-11T07:49:36.617" LastActivityDate="2018-09-11T07:49:36.617" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7920" PostTypeId="2" ParentId="7914" CreationDate="2018-09-11T08:25:07.700" Score="3" Body="&lt;p&gt;We cannot tell with certainty whether AlphaGo Zero would become perfect with enough &lt;strong&gt;training time&lt;/strong&gt;. This is because none of the parts (Neural Network) that would benefit from &lt;strong&gt;infinite training time&lt;/strong&gt; (= a nice approximation of &quot;enough&quot; training time) are guaranteed to ever converge to a perfect solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main limiting factor is that we do not know whether the Neural Network used is big enough. Sure, it's pretty big, it has a lot of capacity... but is that enough? Imagine if they had used a tiny Neural Network (for example just a single hidden layer with a very low number of nodes, like 2 hidden nodes). Such a network certainly wouldn't have enough capacity to ever learn a truly, perfectly optimal policy. With a bigger network it becomes more plausible that it may have sufficient capacity, but we still cannot tell for sure.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that AlphaGo Zero does not just involve a trained part; it also has a Monte-Carlo Tree Search component. After running through the Neural Network to generate an initial policy (which in practice turns out to already often be extremely good, but we cannot tell for certain if it's perfect), it does run some MCTS simulations during it's &quot;thinking time&quot; to refine that policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MCTS doesn't benefit* from increased &lt;strong&gt;training time&lt;/strong&gt;, but it does benefit from increased &lt;strong&gt;thinking time&lt;/strong&gt; (i.e. processing time per turn during the actual game being played, rather than offline training time / self-play time before the evaluation game). In the most common implementation of MCTS (UCT, using the UCB1 equation in the Selection Phase), we can prove that it does in fact learn to play truly perfectly if it is given an infinite amount of &lt;strong&gt;thinking time&lt;/strong&gt;. Now, AlphaGo Zero does use a slightly different implementation of the Selection Phase (which involves the policy generated by the trained Neural Network as prior probabilities), so without a formal analysis I can't tell for sure whether that proof still holds up here. Intuitively it looks like it still should hold up fine though.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;*Note&lt;/strong&gt;: I wrote above that &quot;MCTS doesn't &lt;em&gt;benefit&lt;/em&gt; from increased training time. In practice, of course it does tend to benefit from increased training time because that tends to result in a better Network, which tends to result in better decisions during the Selection Phase and better evaluations of later game states in the tree. What I mean is that MCTS is not &lt;strong&gt;theoretically guaranteed&lt;/strong&gt; to always keep benefitting from increases in training time as we tend to infinity, precisely because that's also where we don't have theoretical guarantees that the Neural Network itself will forever keep improving.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-09-11T08:33:25.153" LastActivityDate="2018-09-11T08:33:25.153" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7921" PostTypeId="2" ParentId="7914" CreationDate="2018-09-11T08:40:18.753" Score="4" Body="&lt;p&gt;Assuming you mean a mathematically perfect player, similar to what we can achieve trivially in Tic Tac Toe, then the answer is &quot;maybe&quot;. The underlying reinforcement learning algorithms that it uses do have some convergence guarantees, but there are some caveats:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Theories of convergence that apply to value and policy functions learned by RL assume unrealistic timescales and decays of learning parameters. If you have actual infinite time and resources then it is possible to explore all board states and learn their values accurately. But then, if you have those resources at your disposal, a brute-force search would work too. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using neural network approximations to true value functions can put bounds on how well value functions are learned, as they rely on generalisation, and are characterised by an error metric. The values that they calculate are in fact guaranteed to &lt;em&gt;not&lt;/em&gt; be mathematically perfect, and in part that is by design (because you want learning from &lt;em&gt;similar&lt;/em&gt; states to apply to new unseen states, and as part of that need to accept the compromise that most state values will be slightly incorrect). This is especially true of the fast policy network used to drive Monte Carlo Tree Search (MCTS).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Running longer MCTS during play improves performance at cost of spending more time per move decision. Given infinite resources, MCTS can play perfectly, even from very crude heuristics. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The difference between AlphaGo Zero and a more traditional game search algorithm is to do with optimal use of available computing resources, as set by available hardware, training time and decision time when playing. It is orders of magnitude more effective to use the RL self-play approach combined with the focused MCTS as in AlphaGo Zero, than any basic search. At least for similar puzzles and games, it is more efficient than any other game playing technique that has been explored. We are likely still more orders of magnitude of effort away from a perfect Go player, and there is no reason yet to suspect that this will ever become practical, and Go become &quot;solved&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-11T14:46:43.990" LastActivityDate="2018-09-11T14:46:43.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7923" PostTypeId="1" AcceptedAnswerId="7924" CreationDate="2018-09-11T10:47:29.857" Score="6" ViewCount="1348" Body="&lt;p&gt;Should I be decaying the learning rate and the exploration rate in the same manner? What's too slow and too fast of an exploration and learning rate decay? Or is it specific from model to model?&lt;/p&gt;&#xA;" OwnerUserId="18076" LastActivityDate="2018-09-12T07:48:05.510" Title="Learning Rate Decay and Exploration Rate Decay" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7924" PostTypeId="2" ParentId="7923" CreationDate="2018-09-11T11:13:05.223" Score="6" Body="&lt;p&gt;Welcome to AI.SE !&#xA;First of all, I'd say that there is a reason to give Learning Rate (LR) and Exploration Rate (ER) the same decay : they play at the same scale (the number of successives batches you'll train your model on). But if I refine the analysis, I would rather say that it's a reason to choose them in the same range, i.e. close to 1, but not specially at the same number.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For LR decay, people often choose it very close to one (which can mean really different things like 0.98 or 0.997), because it plays on a large scale, an you don't want the LR to disappear to brutally.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;However, the choice of ER decay can have more variation from model to model. It depends on the initial value of ER (you don't wanna decay fastly ER if you ER is initially low), and also to the &quot;learning speed&quot; of your model : if your model learns efficiently at the beginning, you could want to fastly decrease ER in order to reduce the noise on the action, supposing that you did enough exploration at the beginning (but I think this last opinion is more controversial). You can find an interesting paper &lt;a href=&quot;https://theses.ubn.ru.nl/bitstream/handle/123456789/5216/Nieuwdorp%2C_T._1.pdf?sequence=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where the author tries different ER decay and finds out that 0.99 is the best, for CartPole environment.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2018-09-12T07:48:05.510" LastActivityDate="2018-09-12T07:48:05.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7925" PostTypeId="1" CreationDate="2018-09-11T20:19:55.103" Score="2" ViewCount="187" Body="&lt;p&gt;I was recently perusing the paper &lt;a href=&quot;https://researcher.watson.ibm.com/researcher/files/us-beygel/samuel-checkers.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Some Studies in Machine Learning Using the Game of Checkers II--Recent Progress&lt;/em&gt; (A.L. Samuel, 1967)&lt;/a&gt;, which is interesting historically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was looking at this figure, which involved Alpha-Beta pruning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qT5oL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qT5oL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It occurred to me that the types of non-trivial, non-chance, perfect information, zero-sum, sequential, partisan games utilized (Chess, Checkers, Go) involve game states that cannot be precisely quantified. For instance, there is no way to ascribe an objective value to a piece in Chess, or any given board state.  In some sense, the assignment of values is arbitrary, consisting of estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The combinatorial games I'm working on are forms of &lt;a href=&quot;http://mbranegame.com/tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;partisan Sudoku&lt;/a&gt;, which are bidding/scoring (economic) games involving territory control.  In these models, any given board state produces an array of ratios allowing precise quantification of player status. Token values and positions can be precisely quantified.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This project involves a consumer product, and the approach we're taking currently is to utilize a series of agents of increasing sophistication to provide different levels challenge for human players.  These agents also reflect what is known as a &quot;&lt;a href=&quot;http://julian.togelius.com/Lantz2017Depth.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;strategy ladder&lt;/a&gt;&quot;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reflex Agents &lt;em&gt;(beginner)&lt;/em&gt; &lt;br&gt;&#xA;Model-based Reflex Agents &lt;em&gt;(intermediate)&lt;/em&gt; &lt;br&gt;&#xA;Model-based Utility Agents &lt;em&gt;(advanced)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Goals may also be incorporated to these agents such as desired margin of victory (regional outcome ratios) which will likely have an effect on performance in that narrower margins of victory appear to entail less risk.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;respectably weak&quot; vs. human performance of the first generation of reflex agents suggests that strong GOFAI might be possible. (The branching factors are extreme in the early and mid-game due to the factorial nature of the models, but initial calculations suggest that even a naive minimax lookahead will be able to look farther more effectively than humans.) Alpha-Beta pruning in partisan Sudoku, even sans a learning algorithm, should provide greater utility than in previous combinatorial game models where the values are estimates. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Is the historical weakness of GOFAI in relation to non-trivial combinatorial games partly a function of the structure of the games studied, where game states and token values cannot be precisely quantified?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Looking for any papers that might comment on this subject, research into combinatorial games where precise quantification is possible, and thoughts in general.&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to determine if it might be worth attempting to develop a strong GOFAI for these models prior to moving up the ladder to learning algorithms, and, if such a result would have research value.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There would definitely be commercial value in that strong GOFAI with no long-term memory would allow minimal local file size for the apps, which must run on lowest-common-denominator smartphones with no assumption of connectivity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;PS- My previous work on this has involved defining the core heuristics that emerge from the structure of the models, and I'm slowly dipping my toes into the look ahead pool.  Please don't hesitate to let me know if I've made any incorrect assumptions.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-09-12T12:03:23.320" Title="Historical weakness of GOFAI in relation to partisan combinatorial games?" Tags="&lt;game-ai&gt;&lt;combinatorial-games&gt;&lt;soft-question&gt;&lt;symbolic-ai&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7926" PostTypeId="1" AcceptedAnswerId="7936" CreationDate="2018-09-11T23:01:12.367" Score="5" ViewCount="1476" Body="&lt;p&gt;I don’t believe in free will, but most people do. Although I’m not sure how an act of free will could even be described (let alone replicated), is &lt;a href=&quot;https://en.wikipedia.org/wiki/Libertarianism_(metaphysics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;libertarian freewill&lt;/a&gt; something that is considered for AI? Or is AI understood to be deterministic?&lt;/p&gt;&#xA;" OwnerUserId="16480" LastEditorUserId="2444" LastEditDate="2020-04-08T00:57:07.260" LastActivityDate="2020-06-04T16:06:08.963" Title="Does AI rely on determinism?" Tags="&lt;philosophy&gt;&lt;human-like&gt;&lt;artificial-consciousness&gt;" AnswerCount="10" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7927" PostTypeId="1" CreationDate="2018-09-12T00:32:52.890" Score="0" ViewCount="83" Body="&lt;p&gt;I am trying to use Deep-Q learning environment to learn Super Mario Bros. &lt;a href=&quot;https://github.com/rtang23/FYP/blob/5abfccf3e86742b8487080493789e13b6f812388/training2.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;The implementation is on Github&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a neural network that Q values update within an episode for a very small learning rate (0.00005). However, even if I increase the learning rate to 0.00025, the Q values do not change within an episode as they are predicting the same Q values regardless of what state it is in. For example, if Mario moves right, the Q value is the same. When I start a new episode, the Q values change though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the Q values should be changing within an episode as the game should be seeing different parts and taking different actions. Why don't I observe this?&lt;/p&gt;&#xA;" OwnerUserId="18076" LastEditorUserId="18076" LastEditDate="2018-09-12T13:03:22.970" LastActivityDate="2018-09-18T06:47:38.773" Title="Should Q values be changing within an epoch/episode or should they change after one episode/epoch?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7929" PostTypeId="2" ParentId="7926" CreationDate="2018-09-12T03:48:08.733" Score="0" Body="&lt;p&gt;AI is algorithmic not free willed in a sense that humans have free will.  So in that sense it is deterministic.  Give it the same data each time you would expect the same result.  Change something (ie feed it new data to learn from) and then it will give a different result.  Hence the determinism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Some algorithms do use some randomising - ie some versions of hill climbing - but if we want to get technical there's no such thing as true random numbers anyway.  (Unless you're using one of those supercomputers that use the radiation from the sun as a seeding factor)&lt;/p&gt;&#xA;" OwnerUserId="9413" LastEditorUserId="9413" LastEditDate="2018-09-12T03:53:36.930" LastActivityDate="2018-09-12T03:53:36.930" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7930" PostTypeId="2" ParentId="7926" CreationDate="2018-09-12T03:57:37.733" Score="3" Body="&lt;p&gt;AI is &quot;deterministic&quot; in the sense that it follows exactly the algorithm. &quot;Deterministic&quot; means different things to a data scientist/programmer, but let's not go into details here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no &quot;freewill&quot; in AI, it's all about mathematics and algorithms. Don't watch too many scientific movies!&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-09-12T03:57:37.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7931" PostTypeId="2" ParentId="7899" CreationDate="2018-09-12T06:23:18.653" Score="1" Body="&lt;p&gt;&lt;strong&gt;Thinking Commercially&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A commercial solution will need to be able to ascertain, continuously verify, and utilize the best of options for captioning learning models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each fairly successful image captioning learning models could be placed in an adapter to provide a common training, optimizing, testing, evaluation, and usage interface.  The hot swappable container-addon mega pattern used in peripheral device installation, RAID, J2EE containers, browsers, and other containment sub-systems can be applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;System Description&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system acceptance criteria being as follows.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A new model could be added without stopping or starting the system if tauted as successful by others&lt;/li&gt;&#xA;&lt;li&gt;Any model can be deleted without starting or stopping the system if it has not been performing well&lt;/li&gt;&#xA;&lt;li&gt;Each possible state of each model added can be given some percentage of the total system resources&lt;/li&gt;&#xA;&lt;li&gt;Various training processes and hyper parameter tuning processes can be applied to any of those model-state combinations&lt;/li&gt;&#xA;&lt;li&gt;A-B testing can be done on any model-state combination&lt;/li&gt;&#xA;&lt;li&gt;Stats can be displayed on any model-state combination any model overall or any state across all added models&lt;/li&gt;&#xA;&lt;li&gt;The interface for each model requires that its associated model produces an array of caption-reliability pairs with zero ore more pairs&lt;/li&gt;&#xA;&lt;li&gt;The reliability measure that is included in the pair with the caption is an indication of the model's assessment of appropriateness to the image&lt;/li&gt;&#xA;&lt;li&gt;The most appropriate caption is picked from the proposals of the various models&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;States can be idle or any of these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Training&lt;/li&gt;&#xA;&lt;li&gt;Testing&lt;/li&gt;&#xA;&lt;li&gt;Evaluation&lt;/li&gt;&#xA;&lt;li&gt;Hyper parameter tuning&lt;/li&gt;&#xA;&lt;li&gt;In use&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, for the two learning models usable in captioning systems suggested in this question, NIC and Neural Talk 2 we could have a system resource allocation like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;30% NIC Training&lt;/li&gt;&#xA;&lt;li&gt;5% NIC Hyper parameter tuning&lt;/li&gt;&#xA;&lt;li&gt;45% NT2 In use&lt;/li&gt;&#xA;&lt;li&gt;15% NT2 Evaluation&lt;/li&gt;&#xA;&lt;li&gt;5% NT2 Hyper parameter tuning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Samples may be pulled from a pool of samples that have been vetted.  That pool may be augmented by real images passing through the system, filtered in accordance with security criteria to avoid external attempts at control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the assignment of resources, the sample pool selection criteria must be specified.  If the system is already at 100%, the model-state combinations from which the resources shall be drawn must also be specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Handling Multiple Output Options&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there may be more than one model in use and each model may have zero, one, or multiple caption suggestions for each image, each with a reliability measure, the outputs must be analyzed to provide the best choice to associate with the image being analyzed.  Additional system criteria must cover this process scenario.  For any given image, final evaluation must follow the following general guidelines.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If multiple models produce similar or exactly the same captions, they most be weighted higher in final evaluation.&lt;/li&gt;&#xA;&lt;li&gt;If the model's reliability is proven in actual use based on feedback from end users, the model's output must be weighed higher accordingly&lt;/li&gt;&#xA;&lt;li&gt;Re-entrant models (such as reinforcement learning network models) must have access to the end user feedback for additional learning even in in-use state&lt;/li&gt;&#xA;&lt;li&gt;Clear winners are chosen&lt;/li&gt;&#xA;&lt;li&gt;Close races are disambiguated through trained functionality&lt;/li&gt;&#xA;&lt;li&gt;Exact ties are broken through pseudo random index generation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Another artificial network may be placed at the output and appropriate encoding and normalization may be applied before training so that a properly trained network, converged using a quantification of the above additional criteria, can select the best caption from the options for each image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Phased Development Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phase one of such a system would likely require manual handling of model-state allocation.  Phase two would be semi-automation.  The location of new models would still require expert attention.  Perhaps further in the future a hunt for new models could be automated too.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-12T06:23:18.653" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7933" PostTypeId="2" ParentId="7900" CreationDate="2018-09-12T11:20:40.660" Score="1" Body="&lt;p&gt;In &lt;em&gt;short&lt;/em&gt;, it's &lt;strong&gt;possible&lt;/strong&gt; to &lt;em&gt;achieve occupation detection via face&lt;/em&gt; but &lt;em&gt;it's highly doubtful that it can be done with &lt;strong&gt;Face API&lt;/strong&gt; that you mentioned.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Now some details,&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Face preception&lt;/em&gt;&lt;/strong&gt; is a very &lt;em&gt;complex&lt;/em&gt; job. You have so many parameters to consider before even beginning to design the model. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Many different types of important information are visible in faces and&#xA;  the processes and mechanisms involved in extracting this information&#xA;  are complex and can be highly specialized.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Face perception serves as the basis for much of human social exchange.&#xA;  Diverse information can be extracted about an individual from a single&#xA;  glance at their face, including their identity, emotional state, and&#xA;  direction of attention.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Extracting that much information is not easy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Face API:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Face API&lt;/em&gt; gives you &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Face Similarity&lt;/strong&gt; (&lt;em&gt;checks how similar two or more faces are&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Face Identification&lt;/strong&gt; (&lt;em&gt;gives you 27 landmarks like age, gender, facial hair, smile etc confidence level&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Emotion detection&lt;/strong&gt; (&lt;em&gt;check what kind of emotion the face has&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, in my view, &lt;em&gt;occupation detection via face&lt;/em&gt; will going to &lt;em&gt;need a lot more than these basic parameters&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, previous work utilizing single person’s nearly frontal clothing information and fore/background context preliminarily proves that occupation recognition is computationally feasible in computer vision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in practice, multiple people with arbitrary poses are common in a photo, and recognizing their occupations is even more challenging. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to appropriately built visual attributes, co-occurrence, and spatial configuration model that is learned through structure SVM or deep neural nets to recognize multiple people’s occupations in a photo simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am attaching some research papers so you can review them to get more insight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Reference Papers:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2998394/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comparative View of Face Perception&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3130382/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The many faces of research on face perception&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://sci-hub.tw/10.1109/ICCV.2013.451&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Do You Do? Occupation Recognition in a Photo via Social Context&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Please state your question clearly (like: tell us your problem in detail, and your approach to solve it), and also tell what research you have done before asking the question&lt;/em&gt;. &lt;a href=&quot;https://meta.stackoverflow.com/questions/261592/how-much-research-effort-is-expected-of-stack-overflow-users&quot;&gt;Please see how much research effort is expected of stackoverflow users before they post any new questions.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4330" LastActivityDate="2018-09-12T11:20:40.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7934" PostTypeId="2" ParentId="7925" CreationDate="2018-09-12T12:03:23.320" Score="2" Body="&lt;p&gt;Nice question!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there are a couple of issues at work here.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the historical weakness of GOFAI in relation to non-trivial&#xA;  combinatorial games partly a function of the structure of the games&#xA;  studied, where game states and token values cannot be precisely&#xA;  quantified?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the short answer is &lt;em&gt;yes&lt;/em&gt;. The real issue is in the last part:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;token values cannot be precisely quantified&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The most successful GOFAI approaches to these games were all some variation on A* search, combining combinatorial search with some form of &lt;em&gt;heuristic function&lt;/em&gt; that estimated the value of the pieces and their positions in any given state. Piece counting is probably a better heuristic than not counting anything at all, but it's still clearly incorrect, because a player with less material may still have an overwhelming positional advantage. Some heuristics can try to estimate this positional advantage as well however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real problem that GOFAI encounters in these games is that positional advantage can be &lt;em&gt;emergent&lt;/em&gt; in ways that require incredible heuristic power to detect. Checkers is a good example. In the 1990's, the &lt;a href=&quot;https://webdocs.cs.ualberta.ca/~chinook/project/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinook project&lt;/a&gt; at the University of Alberta set out to solve it completely. Checkers is notable because it had the same world champion for more than 15 consecutive years, &lt;a href=&quot;https://en.wikipedia.org/wiki/Marion_Tinsley&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marion Tinsley&lt;/a&gt;. Tinsley lost a total of 7 competitive matches over 40 years of play. This makes him an especially interesting person to examine when we look at combinatorial games. Figuring out how Tinsley plays can help us understand how human intelligence works in games like this. In the course of solving checkers, the researchers noted that Tinsley was making moves that required up to &lt;em&gt;42 move lookaheads&lt;/em&gt; to reveal an advantage (See &lt;a href=&quot;https://doi.org/10.1609/aimag.v17i1.1208&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schaeffer et al.&lt;/a&gt;, AI Magazine, Vol. 17, Issue 1).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This strongly suggests that Tinsley was &lt;em&gt;not&lt;/em&gt; methodically considering each possible move. Instead, by his own admission, his thinking was guided by a combination of &lt;em&gt;memory&lt;/em&gt; over his 40 year career (in one match against Chinook in 1992, he indicated he was trying to recall a sequence from a match 30 years prior when making a move (&lt;a href=&quot;https://www.aaai.org/ojs/index.php/aimagazine/article/download/1040/958&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI Magazine Volume 14, Number 2&lt;/a&gt;); and of attentional heuristics (i.e. not thinking about every move sequence, and being able to reliably rule out parts of the search space without looking at them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key is that for GOFAI to solve checkers without heuristics (i.e. to solve it exactly), required enormous amounts of computational power, because some moves yield positional advantages that require 40+ moves of followthrough. Even an incredibly simple game (branching factor of 2) would be hard under that constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast though, self-play techniques like those pioneered in Backgammon with &lt;a href=&quot;http://enzodesiage.com/wp-content/uploads/2017/08/tesauro-tdgammon-1995.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD-Gammon&lt;/a&gt; (Tesauro, Comm. of the ACM 1995) mimic the process through which Tinsley became so good: they play lots and lots of games, learn a good heuristic estimate of position and material value, and more importantly, can learn to &lt;em&gt;remember&lt;/em&gt; odd circumstances that require careful play. TD-Gammon achieved worldclass play despite only explicitly looking 2 moves ahead. GOFAI search techniques weren't even close despite searching much more deeply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern research on &lt;a href=&quot;https://www.nature.com/articles/d41586-018-03067-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;attention&lt;/a&gt; could salvage the GOFAI approach however. If you can learn to tell what's important, you might be able to get a lot more value out of deeper lookaheads. This seems even closer to how Tinsley played: great ability to estimate value was used to guide an explicit analysis of a specific chain of moves.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-12T12:03:23.320" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7935" PostTypeId="1" CreationDate="2018-09-12T12:05:11.227" Score="7" ViewCount="3135" Body="&lt;p&gt;I am making a machine learning program for time series data analysis and using NEAT could help the work. I started to learn TensorFlow not long ago but it seems that the computational graphs in TensorFlow are usually fixed. Is there tools in TensorFlow to help build a dynamically evolving neural network? Or something like PyTorch would be a better alternative? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="18102" LastEditorUserId="10135" LastEditDate="2018-10-18T03:14:37.643" LastActivityDate="2018-12-28T03:22:39.357" Title="Can neuro-evolution of augmenting topologies (NEAT) neural networks be built in TensorFlow?" Tags="&lt;python&gt;&lt;tensorflow&gt;&lt;neat&gt;&lt;programming-languages&gt;&lt;topology&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ClosedDate="2020-05-23T14:00:01.820" ContentLicense="CC BY-SA 4.0" />
  <row Id="7936" PostTypeId="2" ParentId="7926" CreationDate="2018-09-12T12:22:20.283" Score="14" Body="&lt;p&gt;I'm going to assume that by free will, you mean something like the philosophical concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Libertarianism_(metaphysics)&quot; rel=&quot;noreferrer&quot;&gt;libertarian free will&lt;/a&gt;, which is defended by philosophers like &lt;a href=&quot;https://en.wikipedia.org/wiki/Robert_Kane_(philosopher)&quot; rel=&quot;noreferrer&quot;&gt;Robert Kane&lt;/a&gt;. In Libertarian Free Will, individuals have some capability to make choices about their actions. The classic way to argue this is by assuming some kind of spirit-stuff (e.g. a soul) that exists outside the material world, and that this spirit-stuff constitutes the consciousness of a person. Kane tries some mental gymnastics to avoid this, but then concedes something like it in a footnote. I'm not aware of any serious work that doesn't make some kind of non-physical assumption to justify this view. If someone can point at one, I'll update the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By determinism, I'm going to assume you mean the usual notion of &lt;a href=&quot;https://en.wikipedia.org/wiki/Determinism&quot; rel=&quot;noreferrer&quot;&gt;philosophical determinism&lt;/a&gt;: since people's decisions depend on what happened in the past, and where they are in the present, they don't really have a choice in any meaningful sense. Philosopher's like Dennett adopt a slightly softer view (&lt;em&gt;compatibilism&lt;/em&gt;, essentially: you don't get to make big choices, but you do get to make small ones). Appeals to Quantum Mechanics are common to justify that view. In this context, free action means something more like &quot;did something we couldn't predict exactly&quot;. An example might be: you are pre-destined to put a can of campbell's brand tomato soup in your shopping cart, but &quot;make a choice&quot; about exactly which of the dozens of cans you will put in. Since small choices can have large impacts (maybe that can will give you food poisoning, and the others wouldn't), this can make all sorts of things impossible to predict exactly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think most AI researchers don't worry too much about these issues, but Turing actually addresses them in his paper right at the start of the field, &lt;a href=&quot;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&quot; rel=&quot;noreferrer&quot;&gt;Computing Machinary and Intelligence&lt;/a&gt;. The deterministic/compatibilist view point is introduced as Lady Lovelace's objection: Computers only know how to do what we tell them to, so they can't be called intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing's counterargument uses two prongs. First, Turing notes that computers can probably be made to learn (he was right!). If they can learn, they can do things that we don't expect, just like children do. Second, Turing notes that computers already do all sorts of things we don't expect: they have bugs. Anytime a computer exhibits a bug, it did something that was unexpected. Since we cannot generally rule out bugs in programs, computers will always do surprising things. Therefore, computers satisfy the deterministic notion of free will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing also addresses the libertarian notion of free will, which is part of what he calls the &quot;Theological Objection&quot;. The objection is that intelligence requires some kind of divine spark (like free will). Turing argues that we can't detect sparks like this right now (he actually thought we would be able to one day, and spent a lot of time looking at supernatural phenomena too). However, there's no reason to suppose that computers with the right programs won't be endowed with them. A divine creator &lt;em&gt;could&lt;/em&gt; decide that anytime you build something brain-like, it gets a spark. If we build a program that's brain-like, maybe it gets a spark too. In the absence of some way to detect souls, it seems like we ought to just agree to treat things that &lt;em&gt;seem&lt;/em&gt; intelligent as though they had these souls, since otherwise we don't have a really clear way to decide who is and isn't intelligent. The only remaining way is to say &quot;only things made of human meat have souls and are intelligent&quot;. While a lot of people do actually say things like this (e.g. animals have no souls), this is a pretty arbitrary view, and I think there's no hope arguing against it. Turing suggests the &quot;Anthropic Principal&quot;: we shouldn't assume that we're special, because the earth isn't in a special place in the galaxy, or in the universe, and we have pretty compelling evidence that we're an evolved version of other animals around us, but some groups (e.g. biblical literalists) find this unconvincing. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-09-12T19:18:02.030" LastActivityDate="2018-09-12T19:18:02.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7937" PostTypeId="2" ParentId="7926" CreationDate="2018-09-12T13:04:05.353" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Although I’m not sure how an act of freewill could even be described (let alone replicated), &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well, one popular definition goes like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;[Free will is] the freedom to act according to one's motives without arbitrary hindrance from other individuals or institutions&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Compatibilism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source - Wikipedia entry on Compatibilism&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this definition is perfectly compatible with determinism (hence the name &quot;Compatibilism&quot;). Actually, proponents usually argue that free will &lt;em&gt;requires&lt;/em&gt; determinism, because if your choices were ultimately random, like rolling a dice, how could they be &lt;em&gt;your&lt;/em&gt; free choices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you assume that an AI can be said to have &quot;motives&quot;, then according to this view, it would have free will - if noone hinders it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The contrary view, Incompatibilism, has been described in another answer by John Doucette. I agree with him that most AI researchers probably don't worry about philosphical questions like that. All the proponents of indeterminism (sometimes called metaphysical free will or libertarianism) I'm aware of assume that there exist &quot;causes&quot; that are neither deterministic / physical, nor purely random. (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Agent_causation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agent causation&lt;/a&gt;). Since AIs are more or less by definition based on physical processes, deterministic algorithms and possibly some random number generator, I don't see how they could posess this kind of freedom.&lt;/p&gt;&#xA;" OwnerUserId="16384" LastEditorUserId="16384" LastEditDate="2018-09-12T15:54:28.800" LastActivityDate="2018-09-12T15:54:28.800" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7938" PostTypeId="2" ParentId="7926" CreationDate="2018-09-12T14:55:12.860" Score="0" Body="&lt;p&gt;The free will in the spiritual world would be for you to have the right to follow or not the main path (God's way). Taking into consideration that whatever path you follow, you will have its consquences (good or bad).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, an artificial intelligence created to solve problems does not need these distractions. Determinism considers the feeling of freedom a subjective illusion. Trying to apply free will to an AI is to create an inner inner space within the &quot;mind&quot; .. but creating that is already impossible because we can not understand exactly what is going on in our minds. The big question is, how to program something that we can not explain accurately? How to induce an AI to learn something that does not make sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that it is not just code and database that will lead the AI ​​to have faith in its existence or to believe in free will. But to construct an AI that reads and understands every human mind, every thought, house illusion, every paranoia, every confusion, every sadness, every lie ... may be a good source of studies, but whoever qualifies for this kind of experiment? What would be the conclusion of an AI by understanding the whole confused and troubled human mind?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a very complex question ... let's continue studying and raising this kind of subject. I spent my last years trying to recreate human thoughts and actions in an AI .. it's a study for a lifetime .. my fear is the disappointment at the end of everything :(&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-09-12T14:55:12.860" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7940" PostTypeId="1" AcceptedAnswerId="7941" CreationDate="2018-09-12T20:23:42.520" Score="3" ViewCount="467" Body="&lt;p&gt;This is a q-learning snake using a neural network as a q function aproximator and I'm losing my mind here the current model it's worst than the initial one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current model uses a 32x32x32 MLPRegressor from scikit-learn using relu as activation function and the adam solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward function is like following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;death reward = -100.0&lt;/li&gt;&#xA;&lt;li&gt;alive reward = -10.0&lt;/li&gt;&#xA;&lt;li&gt;apple reward = 100.0&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The features extracted from each state are the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;what is in front of the snake's head(apple, empty, snake)&lt;/li&gt;&#xA;&lt;li&gt;what is in the left of the snake's head&lt;/li&gt;&#xA;&lt;li&gt;what is in the right of the snake's head&lt;/li&gt;&#xA;&lt;li&gt;euclidian distance between head and apple&lt;/li&gt;&#xA;&lt;li&gt;the direction from head to the apple measured in radians&lt;/li&gt;&#xA;&lt;li&gt;length of the snake&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One episode consists of the snake playing until it dies, I'm also using in training a probability epsilon that represent the probability that the snake will take a random action if this isn't satisfied the snake will take the action for which the neural network gives the biggest score, this epsilon probability gradually decrements after each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The episode is learned by the regressor in reverse order one statet-action at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the neural network fails too aproximate the q function, no matter how many iterations the snake takes the same action for any state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;changing the structure of the neural network&lt;/li&gt;&#xA;&lt;li&gt;changing the reward function&lt;/li&gt;&#xA;&lt;li&gt;changing the features extracted, I even tried passing the whole map to the network&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Code (python): &lt;a href=&quot;https://pastebin.com/57qLbjQZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/57qLbjQZ&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18123" LastEditorUserId="18123" LastEditDate="2018-09-12T21:19:16.160" LastActivityDate="2020-01-21T13:36:00.203" Title="Snake game: snake converges to going in the same direction every time" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7941" PostTypeId="2" ParentId="7940" CreationDate="2018-09-13T02:13:48.507" Score="3" Body="&lt;p&gt;There are two problems here.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The code you posted doesn't incrimentally train your multilayer perceptron. Instead, it effectively re-randomizes the weights, and then re-fits the model each time you call .fit() at lines 35 &amp;amp; 54. Using SKLearn's _fit() function with Incremental=true might solve this, or you can package up the data into a larger batch, and train on that offline instead after several episodes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your reward function makes it painful to be alive, and doesn't give enough benefits through the Apples to make up for this. There are 100 squares that could contain the apple. On average, the apple will spawn about 5 squares from the snake in each direction. Since the snake can't move diagonally, that's 10 moves (5 left/right, 5 up/down). That means that if the snake plays &lt;em&gt;perfectly&lt;/em&gt;, then on average, it might be able to get zero reward total. In practice, the snake will not play perfectly. This means living gives negative expected reward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In contrast, if the snake can kill itself, it will stop getting negative rewards. The reward function you've used is maximized by getting big enough to run into your own tail as fast as possible. The snake should be able to do this after eating 3 apples I think. There is some incentive to hunt for food well, but not much compared with hitting your own tail as soon as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want the snake to learn to hunt for the food, reduce the penalty for being alive to -1, or even -0.1. The snake will be much more responsive to signals from the food.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-09-13T19:52:13.400" LastActivityDate="2018-09-13T19:52:13.400" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7942" PostTypeId="1" AcceptedAnswerId="7948" CreationDate="2018-09-13T03:37:34.843" Score="4" ViewCount="519" Body="&lt;p&gt;The Alpha Zero (as well as AlphaGo Zero) papers say they trained the value head of the network by &quot;minimizing the error between the predicted winner and the game winner&quot; throughout its many self-play games. As far as I could tell, further information was not given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my understanding, this is basically a supervised learning problem, where, from the self-play, we have games associated with their winners, and the network is being trained to map game states to the likelihood of winning. My understanding leads me to the following question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What part of the game is the network trained to predict a winner on? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, after only five moves, the winner is not yet clear, and trying to predict a winner after five moves based on the game's eventual winner would learn a meaningless function. As a game progresses, it goes from tied in the initial position to won at the end. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the network trained to understand that if all it is told is who eventually won?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditDate="2020-04-04T14:56:08.397" LastActivityDate="2020-04-04T14:56:08.397" Title="What part of the game is the value network trained to predict a winner on?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;alphago&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7943" PostTypeId="2" ParentId="7762" CreationDate="2018-09-13T04:48:51.533" Score="-1" Body="&lt;p&gt;&lt;strong&gt;Automation of Game-play&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aimbots are indeed designed to provide assistance to the human game player when the complexity of game play escapes full cybernetic autonomy at the current state of technology.  There are five basic components in any game player, DNA based or digital.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Acquisition of the current state of the game&lt;/li&gt;&#xA;&lt;li&gt;Control over execution of move options&lt;/li&gt;&#xA;&lt;li&gt;Intercommunication with other players&lt;/li&gt;&#xA;&lt;li&gt;Models related to the game&lt;/li&gt;&#xA;&lt;li&gt;Execution engine for applying these&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The models are as follows for a CS:GO aimbot.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Model of game players&lt;/li&gt;&#xA;&lt;li&gt;Model of the opposing team&lt;/li&gt;&#xA;&lt;li&gt;Model of the game player being assisted&lt;/li&gt;&#xA;&lt;li&gt;Model of that player's team&lt;/li&gt;&#xA;&lt;li&gt;Model of the opposing team&lt;/li&gt;&#xA;&lt;li&gt;Model of the game state&lt;/li&gt;&#xA;&lt;li&gt;Model of legal game moves that transition state&lt;/li&gt;&#xA;&lt;li&gt;Model of objectives (winning or maintaining a top score)&lt;/li&gt;&#xA;&lt;li&gt;Models of game-play strategy involving the first three items in the previous list &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Learning all of these is not in the scope of current deep learning strategies but not outside the scope of AI if the following problem analysis and system approaches are taken.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assumptions are made similar to those of Morgenstern and von Neumann in the later chapters of their &lt;em&gt;Game Theory&lt;/em&gt; to mathematically treat the decisioning of game players in a minimalistic way.&lt;/li&gt;&#xA;&lt;li&gt;DSP, GPU, network realization hardware, cluster computing, or some other artificial network hardware acceleration is available&lt;/li&gt;&#xA;&lt;li&gt;Models programmed in Prolog, DRools, or some other production system and then leveraged by the execution engine in conjunction with other components such as deep learning networks, convolution processing, Markov trees, fuzzy logic, and the application of oversight functions or heuristics as needed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The two services, (a) the provision of suggestions and (b) the automation of minor tasks, may indeed represent the low hanging fruit from a software engineering perspective, but the problem analysis and system approach above may provide more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objectives in CS:GO&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CS:GO (Counter-Strike Global Offensive) game seems to have been written from a Westphalian geopolitical point of view.  This is the typical western perspective, somewhat oblivious to the mindset of the true nature of asymmetric warfare&lt;sup&gt;1&lt;/sup&gt;.  This answer will focus on the creation of an aimbot for the existing models of game-play rather than a realistic simulation of geopolitical balance in this decade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have the objective types listed in online resources that provide a game overview, again, narrowed in authenticity by the prevailing western view of asymmetric war&lt;sup&gt;1&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Terminating players of the opposing team&lt;/li&gt;&#xA;&lt;li&gt;Planting a bomb toward that end (terrorists only)&lt;/li&gt;&#xA;&lt;li&gt;Defend hostages (terrorists only)&lt;/li&gt;&#xA;&lt;li&gt;Prevention of bomb casualties (counter-terrorists only)&lt;/li&gt;&#xA;&lt;li&gt;Rescue of hostages (counter-terrorists only)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ballistic Control&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The targetting of the body or head of an opponent is within the scope of what image recognition can do in conjunction with a movement model.  In military applications, aeronautic devices must be propelled against air friction and the propulsion requires a largely exothermic reaction like combustion.  Thus all targets have a heat signature, which can be recognized in an infrared video stream in such a way as to plot an intercept course for the ballistic weapon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The targetting formulation for CS:GO is not as complex and aiming and firing may be fully automated with much less software machinery.  A LSTM with sufficient speed can be trained to recognize a head in subsequent frames and terminate opponents even if moving.  A simple web search for LSTM will provide a plethora of resources to the novice intending to learn about image recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;One Ambiguity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether the second objective can be met is dependent on what is meant by the term, &quot;Viewing angles,&quot; in the context of image recognition.  Can the player see from perspectives other than the location of their eyes?  If so, this answer can be adjusted if given a clear picture of what is meant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Training and Re-entrant Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training of an artificial neural net to target a head is unnecessary unless the 3D rendering of the game objects and players is distorted by a wide angle virtual lens and trajectories and movements are curved.  As mentioned LSTM can be used to locate a head in multiple frames and extrapolate an opposing players trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where deep learning may be most effective is in the training of how to interact with the player to best assist.  Also, if there are other non-targeting techniques that are more discrete, those who play CS:GO well could record their interactions and those recordings can be processed in preparation for use as training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly a re-entrant learning strategy such as reinforcement is useful for game-play especially if the make up of teams changes and players exhibit different behaviors, executing differing strategies over different networks with different latencies and through-puts, and communicating with the game clients through different peripheral devices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[DeepMind Lab Test Bed for Reinforcement Technology](&lt;a href=&quot;https://github.com/deepmind/lab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/deepmind/lab&lt;/a&gt;}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More than Suggestions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With proper architecture, more than suggestive strategies can be provided to the player.  Statistical dashboards, identification of a bomb before or after planting, and identification of hostages should be among the aimbot services provided, which might suggest a new name, such as obot for objective bot or asbot for assistive bot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not certain that the aimbot interface need be integrated with dashboards or bomb or hostage identifiers.  Sometimes independent bots provide a more flexible arrangement for a user.  Individual bots can always use the same underlying image recognition components and models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Entry Points into Developing Such a System&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read some of the work on the above concepts and download what code you can find that demonstrates it in Python or Java, install what is necessary, and develop some proficiency with the components discussed above as well as the associated theory.  Don't shy away from the math, since success will require some proficiency with feedback signalling and concepts like gradient descent and back-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C10&amp;amp;q=reinforcement+games&amp;amp;btnG=&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement in Games&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C10&amp;amp;q=lstm+locating+head&amp;amp;btnG=&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM Head Locating&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning, Mnih et al., 2013&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Phased Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following phased research and development approach is suggested.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Learn the theory&lt;/li&gt;&#xA;&lt;li&gt;Practice the theory in code&lt;/li&gt;&#xA;&lt;li&gt;Develop the image recognition front end&lt;/li&gt;&#xA;&lt;li&gt;Develop the library to control a virtual player&lt;/li&gt;&#xA;&lt;li&gt;Develop at least one of the above models&lt;/li&gt;&#xA;&lt;li&gt;Create the simplest bot to use it&lt;/li&gt;&#xA;&lt;li&gt;Expand automation from there&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] In asymmetric power struggles, there are always at least two factions within each side because didactic legitimacy seeks division.  Unity is not practically possible.  Each real team usually has a more religious and more secular faction, each of which has economic, philosophic, and historical justifications for their position and agenda.  Also, terrorists don't seek the public detonation of bombs or retention of hostages as objective but rather as means, with the total elimination of all not fully adhered to their view of legitimacy as the sole endgame objective.  Suicide or high risk bombing is considered by most of those that employ it as the poor man's nukes, so without nuclear strike capability for the counter-terrorists and their allies, the terrorism lacks the important dimension of last resort.  The last resort aspect of nuclear strike is missing from the counter-terrorist side too.  CS:GO may sell better by glossing over these particular characteristics of asymmetric warfare and such was left out deliberately.  There may be some benefit to adding these features in from an educational and anti-propaganda point of view.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-13T06:45:20.927" LastActivityDate="2018-09-13T06:45:20.927" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7946" PostTypeId="2" ParentId="7942" CreationDate="2018-09-13T06:19:47.187" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What part of the game is the network trained to predict a winner on&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The positional evaluation. How to give a static score to a chess position.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-09-13T06:19:47.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7947" PostTypeId="1" AcceptedAnswerId="7980" CreationDate="2018-09-13T06:36:13.017" Score="1" ViewCount="92" Body="&lt;p&gt;&lt;strong&gt;Note to the Duplicate Police&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is not a duplicate of the Q&amp;amp;A thread referenced in the close request.  The only text even remotely related in that other thread is the brief mention of climate change in the Q and two sentences in the sole answer: &quot;Identify deforestation and the rate at which it's happening using computer vision and help in fighting back based on how critical the rate is. The World Resources Institute had entered into a partnership with Orbital Insight on this.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the four bullet items below, you will find that this question asks a very specific thing about the relationship between climate and emissions.  Neither that question nor that answer overlaps with the content of this question in any meaningful way.  For instance, it is well known that CO&lt;sup&gt;2&lt;/sup&gt; is NOT causing deforestation.  The additional carbon dioxide in the atmosphere causes faster regrowth.  This is because plants need CO&lt;sup&gt;2&lt;/sup&gt; to grow.  Hydroponic containers deliberately boost it to improve growth rates.  Plants manufacture their own oxygen from the CO&lt;sup&gt;2&lt;/sup&gt; via chlorophyll.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you recall from fifth grade biology, that's why they are plants.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now Back to the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several climate models have been proposed and used to model the relationship between human carbon emissions, added to the natural carbon emissions of life forms on earth, and features of climate that could damage the biosphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Population growth and industrialization have many impacts on the biosphere, including loss of terrain and pollution. Negative oceanic effects, including unpredictable changes in plankton and cyanobacteria are under study.  Carbon emissions from combustion has received attention in recent decades just as sulfur emissions were central to concerns a century or more ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Predicting weather and climate is certainly difficult because it is complex and chaotic, as typical inaccuracies in forecasts clearly demonstrate, but that is looking forward.  Looking backward, analyses of data already collected have shown a high probability that ocean and surface temperature rises followed increases in industrial and transportation related combustion of fuels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How might AI be used to produce some of the key models humans need to protect the biosphere from severe damage.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A more reliable analysis of what has already occurred, since there is some legitimacy to the differing views as to how gross the effect of carbon emissions has been on extinctions of species in the biosphere and on arctic and antarctic melting&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A better understanding as to whether the climate of the biosphere behaves as a buffer of climate, always tending to re-balance after a volcanic eruption, meteor stroke, or other event, or whether the runaway scenario described by some climatologist, where there is a point of no return, is realistic&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A better model to use in trying out scenarios so that solutions can be applied in the order that makes sense from both environmental and economic perspectives&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Automation of climate planning so that the harmful effects of the irresponsibility of one geopolitical entity wishing to industrialize without constraint on other geopolitical entities can be mitigated&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can pattern recognition, feature extraction, the learned functionality of deep networks, or generative techniques be used to accomplish these things?  Can rules of climate be learned?  Are there discrete or graph based tools that should be used?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-13T23:34:47.017" LastActivityDate="2018-09-14T20:44:53.197" Title="How can AI be used to more reliably analyze and plan around the tie between climate and emissions?" Tags="&lt;pattern-recognition&gt;&lt;topology&gt;&lt;generative-model&gt;&lt;survival&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7948" PostTypeId="2" ParentId="7942" CreationDate="2018-09-13T06:58:32.357" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;To my understanding, this is basically a supervised learning problem, where from the self play we have games associated with their winners, and the network is being trained to map game states to likelihood of winning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, although the data for this supervised learning problem was provided by self-play. As AlphaZero learned, the board evaluations of the same positions would need to change, so this is a non-stationary problem, requiring that the ML forgot the training for older examples over time.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What part of the game is the network trained to predict a winner on?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Potentially all of it, including the starting empty board. I am not sure if the empty board was evaluated in this way, but it is not only feasible, but can be done accurately in practice for simpler games (Tic Tac Toe and Connect 4 for example), given known player policies.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Obviously after only five moves, the winner is not yet clear, and trying to predict a winner after five moves based on the game's eventual winner would learn a meaningless function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not at all. This is purely a matter of complexity and difficultly. In practice at such an early stage, the value network will output something non-committal, such as $p=0.51$ win chance for player 1. And it will have learned to do this, because in its experience during self-play similar positions at the start of the game lead to &lt;em&gt;almost&lt;/em&gt; equal numbers of player 1 and player 2 winning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is not meaningless either, it can be used to assess results of look-ahead searches without needing to play to the end of the game. It &lt;em&gt;completely replaces&lt;/em&gt; position-evaluation heuristics as used in more traditional game tree searches. In practice, very early position data in something as complex as chess or go is not going to be as useful as later position evaluations, due to ambivalent predictions. However, for consistency it can still be learned and used in the game algorithms.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How is the network trained to understand that, if all it is told is who eventually won?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If a supervised learning technique is given the same input data $X$ that on different examples predicts the labels $A, B, B, B, A, A, B, B$, then it should learn $p(B|X) = 0.625$. That would minimise a cross-entropy loss function, and is what is going on here.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-13T07:17:41.703" LastActivityDate="2018-09-13T07:17:41.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7949" PostTypeId="1" CreationDate="2018-09-13T10:12:49.540" Score="3" ViewCount="211" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5769/in-a-cnn-does-each-new-filter-have-different-weights-for-each-input-channel-or&quot;&gt;In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question helps me a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let, I have RGB input image. (3 channels)&#xA;Then each filter has n×n weights for one channel.&#xA;It means, actually the filter has totally 3×n×n weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For channel R, it has own n×n filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For channel G, it has own n×n filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For channel B, it has own n×n filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After inner product, add them all to make one feature map.&#xA;Am I right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then, my question starts here.&#xA;For some purpose, I will only use greyscale images as input.&#xA;So the input images always have the same values for each RGB channel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, can I reduce the number of weights in the filters?&#xA;Because in this case, using three different n×n filters and adding them is same with using one n×n filter that is the summation of three filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this logic hold on a trained network?&#xA;I have a trained network for RGB image input, but it is too heavy to run in real time.&#xA;But I only use the greyscale images as input, so it seems I can make the network less heavy (theoretically, almost 1/3 of original).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm quite new in this field, so detailed explanations will be really appreciated.&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18139" LastEditorUserId="9947" LastEditDate="2018-09-13T11:03:59.423" LastActivityDate="2018-09-13T11:22:15.863" Title="Can I reduce the &quot;number of weights&quot; in CNN to 1/3 by restricting the input as greyscale image?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;signal-processing&gt;" AnswerCount="2" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7950" PostTypeId="2" ParentId="7949" CreationDate="2018-09-13T10:22:01.480" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;After inner product, add them all to make one feature map. Am I right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;yes&lt;/strong&gt;, you are right.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Then, can I reduce the number of weights in the filters? Because in this case, using three different n×n filters and adding them is same with using one n×n filter that is the summation of three filters.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have transformed the image into greyscale then you no longer need 3 filters. You should retrain your model on greyscale images. In a greyscale image the value of each pixel is a single sample representing only an amount of light (the light intensity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network will run faster if that is the only architectural change you make, but keep in mind that by converting the image to greyscale you will lose information and probably some of the predictive power of your network.&lt;/p&gt;&#xA;" OwnerUserId="18138" LastActivityDate="2018-09-13T10:22:01.480" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7951" PostTypeId="2" ParentId="7891" CreationDate="2018-09-13T10:28:25.733" Score="-1" Body="&lt;p&gt;I think your task is as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\ $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume 5-year-old children.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have numbers of pictures that drew by them.&#xA;(Let, these pictures are your [training set].)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, you want to synthesis similar pictures with the training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because you need more pictures for your study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\ $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the pictures, you want to extract some meaningful information about a real child who drew them, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I think GAN is not faithful for your study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, GAN can make very similar pictures with your training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it does not mean the synthesized images can contain the things what you want!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GAN just synthesizes &quot;fake pictures&quot; that cannot be distinguished from your training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The synthesized pictures may do not have any meaningful thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because it is not drawn by the real child.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\ $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it is worth to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May GAN captures some features of &quot;children-like&quot;. (But I think it is too hard.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find lots of GAN for your research, especially DCGAN.&lt;/p&gt;&#xA;" OwnerUserId="18139" LastActivityDate="2018-09-13T10:28:25.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7952" PostTypeId="1" CreationDate="2018-09-13T10:52:22.383" Score="1" ViewCount="44" Body="&lt;p&gt;I am currently reading the research paper &lt;a href=&quot;https://arxiv.org/pdf/1706.03686.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Crowd Counting Using Convolutional Neural Network and Markov Random Field&lt;/a&gt;  by Kang Han, Wanggen Wan, Haiyan Yao, and Li Hou. &lt;br/&gt;&#xA;I did not understand the following context properly: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Formally, the&#xA;  Markov random field framework for the crowd counting&#xA;  can be defined as follows (we follow the notation in [18]).&#xA;  Let P be the set of patches in an image and C be a possi-&#xA;  ble set of counts. A counting c assigns a count c p ∈ C to&#xA;  each patch p ∈ P. The quality of a counting is given by an&#xA;  energy function:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;E(c) =&#xA;  ∑ D p (c p ) + ∑&#xA;  p∈P&#xA;  V (c p − c q )&#xA;  . . . (2)&#xA;  (p,q)∈N&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where N are the (undirected) edges in the four-connected&#xA;  image patch graph. D p (c p ) is the cost of assigning count&#xA;  c p to patch p, and is referred to as the data cost. V (c p −c q )&#xA;  measures the cost of assigning count c p and c q to two&#xA;  neighboring patch, and is normally referred to as the dis-&#xA;  continuity cost.&#xA;  For the problem of smoothing the adjacent patches&#xA;  count, D p (c p ) and V (c p − c q ) can take the form of the&#xA;  following functions:&#xA;  D p (c p ) = λ min((I(p) − c p ) 2 , DATA K) . . . (3)&#xA;  V (c p − c q ) = min((c p − c q ) 2 , DISC K)&#xA;  . . . (4)&#xA;  where λ is a weight of the energy items, I(p) is the ground&#xA;  truth count of the patch p, DATA K and DISC K are the&#xA;  truncating item of D p (c p ) and V (c p − c q ), respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain the above part in detail and give me a detailed insight on how should I implement this part of the project? &lt;/p&gt;&#xA;" OwnerUserId="14592" LastEditorUserId="14592" LastEditDate="2018-09-13T14:22:44.190" LastActivityDate="2018-09-13T14:22:44.190" Title="Doubt regarding research paper on Crowd Counting using Convolutional neural networks and Markov Random Field" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7953" PostTypeId="2" ParentId="7949" CreationDate="2018-09-13T11:02:13.727" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;After inner product, add them all to make one feature map. Am I right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes you are right. Now I will try to perform a transformation to preserve accuracy, cannot say about efficiency of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I have not worked on such type of problem, but knowing the maths behind CNN I will try to solve the problem theoretically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you have to know the RGB to greyscale conversion formulae which have bben used. &lt;a href=&quot;https://www.johndcook.com/blog/2009/08/24/algorithms-convert-color-grayscale/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; are some common schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let us say each pixel had values $r, g, b$ and you converted it to $x_1*r + x_2*g +x_3*b$. So initially for simplicity let us say the we are talking about the corner pixel and convolution scheme $valid$, so the corner pixel with $RGB$ channels get multiplied with values $w_r, w_g, w_b$ during convolution and gets summed up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now you only have one pixel which is $x_1*r + x_2*g +x_3*b$. Now let us multiply this by $\frac {w_r}{x_1} + \frac {w_g}{x_2} + \frac {w_b}{x_3}$. This will result in: $(w_r*r + w_g*g +w_b*b) + (\frac {w_r*(x_2*g + x_3*b)}{x_1} +\frac {w_g*(x_1*r + x_3*b)}{x_2} + \frac {w_b*(x_2*g + x_1*r)}{x_3})$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have to try to remove the 2nd term of the equation. The paramters $w_r, w_g, w_b, x_1, x_2, x_3$ are predetermined already. So taking a single term $r$ from the second term of the equation gives $r*(\frac {w_g*x_1}{x_2} + \frac {w_b*x_1}{x_3})$. The second term has pre-determined value, and for the $r$ I think somehow using modern image analysis techniques you can find some approximate value of possible $r$. Do this for $g$ and $b$ as well and subtract it from the aforementioned equation and you will finally get $(w_r*r + w_g*g +w_b*b)$ which was the term obtained by convolution of filters with $RGB$ images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done all this hypothetically, such image analysis techniques might not exist, but it is still worth a try. Probably better methods to reduce the second term exists in mathematical literature. I will leave it upto mathematicians to point in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-09-13T11:22:15.863" LastActivityDate="2018-09-13T11:22:15.863" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7955" PostTypeId="1" CreationDate="2018-09-13T15:32:52.660" Score="1" ViewCount="47" Body="&lt;p&gt;Neurons can be simulated using different models that vary in the degree of biophysical realism. When designing an artificial neuronal network, I am interested in the consequences of choosing a degree of neuronal realism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of computational performance, the FLOPS vary from integrate-and-fire to the Hodgkin–Huxley model (Izhikevich, 2004). However, properties, such as refraction, also vary with the choice of neuron.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When selecting a neuronal model, what are consequences for the ANN other than performance? For example, would there be trade-offs in&#xA;terms of stability/plasticity?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Izhikevich investigated the performance question in 2004. What are&#xA;the current benchmarks (other measures, new models)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does selecting a neuron have consequences for scalability in terms of hardware for a deep learning network?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When is the McCulloch-Pitts neuron inappropriate?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Izhikevich, E. M. (2004). &lt;em&gt;Which model to use for cortical spiking neurons?&lt;/em&gt; IEEE Transactions on Neural Networks, 15(5). &lt;a href=&quot;https://www.izhikevich.org/publications/whichmod.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.izhikevich.org/publications/whichmod.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="16411" LastActivityDate="2018-09-13T15:32:52.660" Title="How does the degree of neuronal realism affect computing in a deep learning scenario?" Tags="&lt;deep-learning&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;&lt;biology&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7956" PostTypeId="2" ParentId="7328" CreationDate="2018-09-13T15:49:45.523" Score="6" Body="&lt;p&gt;I think, there are various reasons. First of all: Flexibility. With modern CPUs and GPUs you can construct pretty much every AI model you want and in every size and complexity you want. How can you be sure that the model you are currently using is still suitable in a few years? Maybe there will be a major breakthrough in NNs in the next few years? Maybe some scientists find that there is a way better way of developing an AI than with NNs, genetic algorithms etc. Normal chips can handle it all, and they can handle it well enough. But if you want to optimize it amd have no worries about money, you can develope a specialized architecture (this is already done by different companies, which gives a major speed boost on specific tasks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reason number two: Mass production. I mean, companies could eventually produce highly integrated analog AI components (let's say, for example NN chips). But that would be a bigger investment. It's rather unclear if units that are flexible enough to be a serious AI hardware alternative, can be easily produced in a mass-nm-production which can compete against CPUs and GPUs. Especially the latter are highly optimized for doing massive parallel calculation. And, if you watch the development of GPU-similar architectures (can do few things, but those very well) which are aditionally optimized for maschine learning, you can see that it would be a hard competition for analog units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the above doesn't mean that there is no research in this area. There a several experiments which try to archieve that, but they aren't 'dangerous' yet for common architectures. Eventually, they will come in the future, when we understand AI and intelligence in general better and are just trying to tweak, but I am rather skeptical about that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Also, something that also kinda belongs to flexibility: You can experiment better with the AI algorithms running on 'normal' digital hardware. For example, you can easily inspect a NN at certain locations, you can quickly modify the input data or provide alternative ones, you really aren't bound to anything. And since we still don't know or understand every model completely, when to use which, if there are better architectures for a certain task etc, it doesn't make sense to put something 'young' and 'experimental'  in a fixed analog architecture.&lt;/p&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="17769" LastEditDate="2018-09-13T15:57:40.680" LastActivityDate="2018-09-13T15:57:40.680" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7958" PostTypeId="1" AcceptedAnswerId="7960" CreationDate="2018-09-13T19:57:18.963" Score="1" ViewCount="466" Body="&lt;p&gt;I found a &lt;a href=&quot;https://youtu.be/vppFvq2quQ0&quot; rel=&quot;nofollow noreferrer&quot;&gt;video for the paper &lt;em&gt;DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills&lt;/em&gt;&#xA;&lt;/a&gt; on YouTube.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked in the related paper, but could not find details of how to the environment was created, such as the physics engine it used. I would like to use it, or something similar.&lt;/p&gt;&#xA;" OwnerDisplayName="user18189" LastEditorUserId="1847" LastEditDate="2018-09-14T06:31:44.963" LastActivityDate="2018-09-14T06:31:44.963" Title="What is the physics engine used by DeepMimic?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7959" PostTypeId="1" AcceptedAnswerId="7968" CreationDate="2018-09-13T20:26:34.613" Score="3" ViewCount="72" Body="&lt;p&gt;I want to build a model to support decision making for loan insurance proposal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three actors in the problem: a bank, a loaner applicant (someone who ask for a loan) and a counselor. The counselor studies the loaner application and if it has a good profile it will propose to him loan from banks that fits his profile. Then the application is sent to the bank but the bank could refuse the applicant (based on criteria we don't know).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The counselor has also to decide whether or not he will propose to the loaner applicant a loan insurance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The risk is that some banks reject loan applicant who accepts a loan insurance and other banks accept more applicants with a loan insurance. But there aren't rules regarding banks since some banks accept or reject applicants with loan insurance according of the type of acquisition applicants want with their loan for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the profile of the applicant can matter in their rejection from banks but all criteria influencing the decision are quite uncertain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've researched online and found several scholarly articles on using Monte Carlo for decision making. Should I use Monte Carlo or a simple classifier for this Decision Making problem ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw that Monte Carlo (possibly Monte Carlo Tree Search) can be used in Decision Making and it is good when there is uncertainty. But it seems that it would forecast by producing some strategy (after running a lot of simulations) but what I want is an outcome based on both the profile of the loaner applicant and the bank knowing that criteria from banks (to accept loaner applicant from could change every six months. And I would have too model banks which seems quite difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classifier seems to me to not really fit the problem. I am not really sure. Actually, I don't see how a classifier like a decision tree, for example, would work here. Because I have to predict decision of the counselor to propose or not based on the decision of banks (and I don't know their criteria) to refuse or accept applicants who were proposed loan insurance and accepted it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data I have is former applicants profile who were sent to banks and if they were accepted or not by the bank, if they wanted a loan insurance or not and the type of acquisition they wanted to make with their loan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to Decision Making. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="18192" LastEditorUserId="18192" LastEditDate="2018-09-13T21:24:32.153" LastActivityDate="2018-09-14T08:42:18.080" Title="Should I use Monte Carlo or a classifier for this Decision Making problem?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;monte-carlo-tree-search&gt;&lt;decision-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7960" PostTypeId="2" ParentId="7958" CreationDate="2018-09-13T22:28:05.290" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://pybullet.org/wordpress/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bullet physics engine&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their &lt;a href=&quot;https://arxiv.org/pdf/1804.02717.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; says&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Physics simulation is performed at 1.2kHz using the Bullet physics engine [Bullet 2015].&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15493" LastActivityDate="2018-09-13T22:28:05.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7962" PostTypeId="1" AcceptedAnswerId="7973" CreationDate="2018-09-14T02:32:33.923" Score="2" ViewCount="448" Body="&lt;p&gt;Is it possible for a genetic algorithm + Neural Network that is used to learn to play one game such as a platform game able to be applied to another different game of the same genre.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example, could an AI that learns to play Mario also learn to play another similar platform game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if anyone could point me in the direction of material i should familiarise myself with in order to complete my project.&lt;/p&gt;&#xA;" OwnerUserId="18209" LastEditorUserId="1641" LastEditDate="2018-09-14T10:58:39.703" LastActivityDate="2018-09-14T11:46:40.210" Title="Can genetic algorithms be used to learn to play multiple games of the same type?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;python&gt;&lt;genetic-algorithms&gt;" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7963" PostTypeId="1" AcceptedAnswerId="7967" CreationDate="2018-09-14T03:38:46.363" Score="5" ViewCount="368" Body="&lt;p&gt;I started teaching myself about reinforcement learning a week ago and I have this confusion about the learning experience. Let's say we have the game Go. And we have an agent that we want to be able to play the game and win against anyone. But let's say this agent learn from playing against one opponent, my questions then are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Wouldn't the agent (after learning) be able to play only with that opponent and win? It estimated the value function of this specific behaviour only.&lt;/li&gt;&#xA;&lt;li&gt;Would it be able to play as good with weaker players? &lt;/li&gt;&#xA;&lt;li&gt;How do you develop an agent that can estimate a value function that generalizes against any behaviour and win? Self-play? If yes, how does that work? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17582" LastEditorUserId="2444" LastEditDate="2018-09-14T12:33:08.047" LastActivityDate="2018-09-14T12:33:08.047" Title="How can a reinforcement learning agent generalize if it is trained against only one opponent?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7965" PostTypeId="1" AcceptedAnswerId="7969" CreationDate="2018-09-14T05:42:47.450" Score="4" ViewCount="36" Body="&lt;p&gt;In some situation, like risk detection and spam detection. The pattern of Good User is stable, while the patterns of Attackers are changing rapidly. How can I make a model for that? Or which classifier/method should I use?&lt;/p&gt;&#xA;" OwnerUserId="18213" LastActivityDate="2018-09-14T09:01:50.557" Title="How to design a classifier while the patterns of positive data are changing rapidly?" Tags="&lt;classification&gt;&lt;security&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7966" PostTypeId="1" CreationDate="2018-09-14T07:28:46.180" Score="9" ViewCount="735" Body="&lt;p&gt;Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during execution time? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that such neural networks overcome the usual separation of learning/inference phases and they continuously live their lives in which learning and self-improving occurs alongside performing inference and actual decision making for which these neural networks were built. Effectively, it could be a neural network that acts as a &lt;a href=&quot;http://people.idsia.ch/~juergen/goedelmachine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gödel machine&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found the term &lt;em&gt;dynamic neural network&lt;/em&gt; but it is connected to adding some delay functions and nothing more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, such self-improving networks completely redefine the learning strategy, possibly, single shot gradient methods can not be applicable to them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is connected to the neural-symbolic integration, e.g. &lt;a href=&quot;https://www.amazon.co.uk/Neural-Symbolic-Cognitive-Reasoning-Technologies/dp/3642092292/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Neural-Symbolic Cognitive Reasoning&lt;/em&gt; by Artur S. D'Avila Garcez, 2009&lt;/a&gt;. Usually this approach assigns individual neurons to the variables (or groups of neurons to the formula/rule) in the set of formulas in some knowledge base. Of course, if knowledge base expands (e.g. from sensor readings or from inner nonmonotonic inference) then new variables should be added and hence the neural network should be expanded (or contracted) as well.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-08-27T23:31:45.503" LastActivityDate="2019-08-27T23:31:45.503" Title="Are there dynamic neural networks?" Tags="&lt;neural-networks&gt;&lt;incremental-learning&gt;&lt;online-learning&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7967" PostTypeId="2" ParentId="7963" CreationDate="2018-09-14T08:37:24.317" Score="6" Body="&lt;p&gt;Reinforcement Learning (RL) at its core does not have anything directly to say about adversarial environments, such as board games. That means in a purely RL set up, it is not really possible to talk about the &quot;strength&quot; of a player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, RL is about solving consistent environments, and that consistency requirement extends to any opponents or adversarial components. Note that consistency is not the same as determinism - RL theory copes well with opponents that effectively make random decisions, provided the &lt;em&gt;distribution&lt;/em&gt; of those decisions does not change based on something the RL agent cannot know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Provided an opponent plays &lt;em&gt;consistently&lt;/em&gt;, RL can learn to optimise against that opponent. This does not directly relate to the &quot;strength&quot; of an opponent, although usually strong opponents present a more challenging environment to learn overall.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Wouldn't the agent (after learning) be able to play only with that opponent and win? since it estimated the value function of this specific behavior only.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the RL has enough practice and time to optimise against the opponent, then yes the value function (and any policy based on it) would be specific to that opponent. Assuming, the opponent did not play flawlessly, then the RL would learn to play such it would win as often as possible against the opponent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When playing against other opponents, the success of the RL agent will depend on how similar the new opponent was to the original that it trained against.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;would it be able to play as good with weaker players?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As stated above, there is not really a concept of &quot;stronger&quot; or &quot;weaker&quot; in RL. It depends on the game, and how general the knowledge is that strong players require in order to win. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory you could construct a game, or deliberately play strongly, but with certain flaws, so that RL would play very much to counter one play style, and would fail against another player that did not have the same flaws. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is difficult to measure this effect, because human players learn from their mistakes too, and are unlikely to repeat the exact same game time after time, but with small variations at key stages. Humans do not make consistent enough opponents, and individual humans do not play enough games at each stage of their ability to study fine-grained statistics of their effective policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice it seems likely that the effect of weakening against new players would be there in RL, due to sampling error if nothing else. However, it seems that the &quot;strength&quot; of players as we measure them in any game of skill such as chess or go, does correlate with a generalised ability. In part this is backed up by consistent results with human players and Elo ratings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any game where you can form &quot;rings&quot; of winning players:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Player B consistently beats Player A&lt;/li&gt;&#xA;&lt;li&gt;Player C consistently beats Player B&lt;/li&gt;&#xA;&lt;li&gt;Player A consistently beats Player C&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Could cause issues of the type you are concerned about when applying RL to optimise an artificial agent.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;How do you develop an agent that can estimate a value function that generalizes against any behavior and win? &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If is possible to play perfectly, then a value function which estimated for perfect play would work. No player could beat it. Think of Tic Tac Toe - it is relatively easy to construct perfect play value functions for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not achievable in practice in more complex games. To address this, and improve the quality of its decisions, what AlphaGo does is common to many game-playing systems, using RL or not. It performs a look-ahead search of positions. The value function is used to guide this. The end result of the search is essentially a more accurate value function, but only for the current set of choices - the search &lt;em&gt;focuses&lt;/em&gt; lots of computation on a tiny subset of all possible game states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One important detail here is that this focus applies at run time whilst playing against any new opponent. This does not 100% address your concerns about differing opponents (it could still miss a future move by a different enough opponent when searching). But it does help mitigate smaller statistical differences between different opponents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This search tree is such a powerful technique that for many successful game playing algorithms, it is possible to start with an inaccurate value function, or expert heuristics instead, which are fixed and general against all players equally. IBM's Deep Blue is an example of using heuristics.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;self-play?  if yes, how does that work?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Self-play appears to help. Especially in games which have theoretical optimal play, value functions will progress towards assessing this optimal policy, forming better estimates of state value with enough training. This can give a better starting point than expert heuristics when searching.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-14T08:43:14.850" LastActivityDate="2018-09-14T08:43:14.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7968" PostTypeId="2" ParentId="7959" CreationDate="2018-09-14T08:42:18.080" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;A classifier seems to me to not really fit the problem. I am not really sure. Actually, I don't see how a classifier like a decision tree, for example, would work here. Because I have to predict decision of the counselor to propose or not based on the decision of banks (and I don't know their criteria) to refuse or accept applicants who were proposed loan insurance and accepted it.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The data I have is former applicants profile who were sent to banks and if they were accepted or not by the bank, if they wanted a loan insurance or not and the type of acquisition they wanted to make with their loan.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why does this seem to you like something where a classifier wouldn't fit? Unless I'm missing something, it sounds like a prototypical example of a classification problem to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input features (applicants' profile)&lt;/li&gt;&#xA;&lt;li&gt;A clear (binary?) prediction target: propose or don't propose (equivalent to predicting whether or not the bank would accept, because you'll always want to propose if the bank would accept, and never propose if the bank wouldn't accept).&lt;/li&gt;&#xA;&lt;li&gt;Old training data containing both the input features and the matching prediction targets.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Approaches like Monte-Carlo Tree Search can only be used if you have a &lt;em&gt;forward model&lt;/em&gt; or &lt;em&gt;simulator&lt;/em&gt;. In your setting, you could view the features (applicants' profile) as a &quot;game state&quot;, and model the problem as a game with two actions (propose or not propose). However, you don't have a forward model (a function that, given a current state and action, generates a possible reward and subsequent state). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In applications where MCTS is often used (such as games), you do have such a forward model: for a game like Go or chess, you can easily program the game's rules, program how you transition from one state into another when you select an action, etc. This does not appear to be the case for you.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-14T08:42:18.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7969" PostTypeId="2" ParentId="7965" CreationDate="2018-09-14T09:01:50.557" Score="2" Body="&lt;p&gt;The phenomenon where the prediction targets (in your case, behaviour) change over time is referred to as &quot;concept drift&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you search for that term, you'll find that there have been many publications attempting to tackle that over multiple decades, way too many papers to all summarize here in a single answer. It's still a difficult problem though, by no means a &quot;solved&quot; problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two different, broad directions for ideas are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Frequently re-training (offline) static models on the most recent training data&lt;/li&gt;&#xA;&lt;li&gt;Using online learning approaches that can continuously be updated from a data stream, online as new labelled data becomes available.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/dalpozz/AMLFD&quot; rel=&quot;nofollow noreferrer&quot;&gt;This github page&lt;/a&gt; contains a large list of papers on credit card fraud detection, where the problem you describe occurs because fraudsters change their behaviour in an attempt to evade detection. Most of those papers discuss variants of the first approach. Basically, many of those papers use an ensemble of multiple Random Forests. Every day, new labelled data becomes available. They often then remove the oldest of multiple Random Forests, and add a new Random Forest trained on the most recent data made available that day. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also some variants where they don't always train new models at a fixed schedule (e.g., every day), but try to detect when the statistical properties of the data have changed using statistical tests, and only train new models when it is &quot;necessary&quot; (due to such changes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second idea, you'll often be thinking of approaches that use Stochastic Gradient Descent-like approaches for learning; with a non-decreasing learning rate / step size, such techniques will naturally, slowly &quot;forget&quot; what they have learned from old data, and focus more on the latter data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have some method to obtain accurate labels for certain instances relatively quickly, you could consider an approach like the one proposed in &lt;a href=&quot;https://dennissoemers.github.io/assets/publications/SoemersCreditCardIAAI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (&lt;strong&gt;disclaimer:&lt;/strong&gt; I'm an author on this paper). For example, in that paper the assumption is that human experts can relatively quickly investigate and obtain accurate labels for a small selection of transactions, and this can be exploited to quickly learn in an online manner.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-14T09:01:50.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7970" PostTypeId="2" ParentId="7963" CreationDate="2018-09-14T09:30:56.620" Score="4" Body="&lt;p&gt;Most of your questions are already addressed very well by Neil's answer, so I won't address those again. I'd just like to clarify additionally on the following point:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But let's say this agent learn from playing against one opponent&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Precisely that assumption of learning from against a single opponent causes many issues. In fact, even if that &quot;single opponent&quot; is changing/improving, you can still have an unstable learning process. For instance, two agents that are simultaneously learning (or a single agent and a copy of itself) can keep infinitely going around in circles as Neil also already hinted at in a game like Rock-Paper-Scissors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the original &lt;a href=&quot;https://www.nature.com/articles/nature16961&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo publication (2016)&lt;/a&gt;, learning from self-play was done by randomly selecting one of a set of (relatively recent) copies of the learning agent every game, rather than always playing against an exact copy of the single most recent version of the agent. Adding more diversity to the &quot;training partners&quot; in that way can help to learn a more robust policy that can handle different opponents. Of course, we shouldn't go overboard with this kind of randomization; you still want to make sure to train against strong training partners (or opponents that have a  roughly similar level of strength as the learning agent), an agent that already is quite strong won't be able to learn a lot from playing against an extremely weak agent anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;2017, a new paper appeared on AlphaGo Zero&lt;/a&gt;. In this paper, they no longer used such randomization as described above, but still had a stable learning process from self-play. As far as I'm aware, the most likely hypothesis to explain this stability is the fact that Monte-Carlo Tree Search was used &lt;strong&gt;during the self-play training process&lt;/strong&gt; to improve the update targets. &lt;strong&gt;This is different from the use of lookahead search that Neil already described during gameplay, after training.&lt;/strong&gt; By also incorporating lookahead search during the training process, and using it to improve update targets, the hypothesis is that you can reduce the risk of &quot;overfitting&quot; against the training partner. The lookahead search actually &quot;thinks&quot; a bit about other moves that the training partner could have selected other than what it actually did, and incorporates that in the update targets. A similar combination of MCTS and self-play reinforcement learning was also &lt;a href=&quot;https://arxiv.org/abs/1705.08439&quot; rel=&quot;nofollow noreferrer&quot;&gt;independently published (by different authors) to result in stable learning in different games&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-14T09:30:56.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7971" PostTypeId="2" ParentId="7962" CreationDate="2018-09-14T10:28:06.640" Score="0" Body="&lt;p&gt;Definitely depends on the design of your algorithm. According to my knowledge, almost all ML algorithms are targeting at specific issues, thus it’s difficult for general usage. And you have to train again for any new issues. It’s also difficult to understand the internal working mechanism of those AI algorithms due to general statistics methods (and yes, they call AI). I will recommend a regional/encapsulated method applied on general methods, therefore algorithms are not specific and micro-structured for general purposes. If games are similar, definitely we can apply on both with appropriate designed methods. Hinton has started his Capsule network which I think is a good direction. Just beware, training shouldn’t be specific object related. Instead, it should be micro-structure related or feature (it’s hard to differ current AI feature from human insight feature). For example, human can easily differ differences even though never see those before. And human do not have to re-train nerve units except for better understanding or accuracy. Genetic algorithms should have the same ability to survive in different but similar environments. Unfortunately, we are at the beginning of AI era but also luckily we have a lot to do. In fact, almost all current techs imitate the nature. If the nature can, definitely we can at one day.&lt;/p&gt;&#xA;" OwnerUserId="18221" LastActivityDate="2018-09-14T10:28:06.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7972" PostTypeId="2" ParentId="7962" CreationDate="2018-09-14T10:49:50.703" Score="0" Body="&lt;p&gt;Genetic algorithms can learn multiple games, yes, in fact genetic algorithms is a bad term to describe this family, there is only one generic genetic algorithm with many variations depending on the problem at hand. I recommend this pdf for a introduction on how they work and how to build them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.boente.eti.br/fuzzy/ebook-fuzzy-mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.boente.eti.br/fuzzy/ebook-fuzzy-mitchell.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18123" LastActivityDate="2018-09-14T10:49:50.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7973" PostTypeId="2" ParentId="7962" CreationDate="2018-09-14T11:46:40.210" Score="5" Body="&lt;p&gt;Genetic algorithms and Neural Networks both are &quot;general&quot; methods, in the sense that they are not &quot;domain-specific&quot;, they do not rely specifically on any domain knowledge of the game of Mario. So yes, if they can be used to successfully learn how to play Mario, it is likely that they can also be applied with similar success to other Platformers (or even completely different games). Of course, some games may be more complex than others. Learning Tic Tac Toe will likely be easier than Mario, and learning Mario will likely be easier than StarCraft. But in principle the techniques should be similarly applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only want to learn in one environment (e.g., Mario), and then immediately play a different game without separately training again, that's much more complicated. For research in that area you'll want to look for Transfer Learning and/or Multi-Task learning. There has definitely been research there, with the latest developments that I'm aware of having been published &lt;a href=&quot;https://deepmind.com/blog/preserving-outputs-precisely-while-adaptively-rescaling-targets/&quot; rel=&quot;noreferrer&quot;&gt;yesterday&lt;/a&gt; (this is Deep Reinforcement Learning though, no GAs I think).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most &quot;famous&quot; recent work on training Neural Networks to play games using Genetic Algorithms that I'm aware of is &lt;a href=&quot;https://eng.uber.com/deep-neuroevolution/&quot; rel=&quot;noreferrer&quot;&gt;this work by Uber&lt;/a&gt; (blog post links to multiple papers). I'm not 100% sure if that really is the state of the art anymore, if it's the best work, etc... I didn't follow all the work on GAs in sufficient detail to tell for sure. It'll be relevant at least though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there's also been quite a lot of work on AI in general for Mario / other platformers (for instance in venues such as the IEEE Conference on Computational Intelligence and Games, and the TCIAIG journal).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-14T11:46:40.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7974" PostTypeId="2" ParentId="6745" CreationDate="2018-09-14T12:07:08.920" Score="1" Body="&lt;p&gt;This is not a bug, assuming you hve implemented the SOM properly with best decay rules of learning rate and neighbourhood strength (in short best hyperparameters). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of the map as finding many local clusters in the Iris-Versicolor class. Since SOM's behave somewhat like k-means clustering, I would say this implies Iris Versicolor has many local clusters which are as closely knit together as the entire classes of Iris setosa and Iris virginica.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-09-14T12:07:08.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7975" PostTypeId="1" CreationDate="2018-09-14T12:07:49.547" Score="3" ViewCount="76" Body="&lt;p&gt;Problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a fairly big database that is built up by our own users. &#xA;The way this data is entered is by asking the users 30ish questions that all have around 12 answers (x, a, A, B, C, ..., H). The letters stand for values that we can later interpret. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already tried and implemented some very basic predictors, like random forest, a small NN, a simple decision tree etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But all these models use the full dataset to do one final prediction. (fairly well already).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I want to create is a system that will eliminate 7 to 10 of the possible answers a user can give at any question.&lt;/strong&gt; This will reduce the amount of data we need to collect, store, or use to re-train future models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already found several methods to decide what are the most discriminative variables in the full dataset.  Except, when a user starts filling the questions I start to get lost on what to do. None of the models I have calculate the next question given some previous information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It feels like I should use a Naive Bayes Classifier, but I'm not sure. Other approaches include recalculating the Gini or entropy value at every step. But as far as my knowledge goes, we can't take into account the answers given before the recalculating.&lt;/p&gt;&#xA;" OwnerUserId="18225" LastEditorUserId="2444" LastEditDate="2019-02-16T02:43:21.590" LastActivityDate="2019-02-16T02:43:21.590" Title="How can I minimize the number of answers that are relevant to a machine learning model?" Tags="&lt;machine-learning&gt;&lt;feature-selection&gt;&lt;decision-tree&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7976" PostTypeId="2" ParentId="7975" CreationDate="2018-09-14T12:42:06.597" Score="1" Body="&lt;p&gt;You don't need to re-train on the fly. What you're looking for is an &lt;em&gt;embedded feature selection&lt;/em&gt; algorithm, and even more precisely, one that minimizes the number of responses required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this might be one of the rare cases where genetic and evolutionary approaches are the obviously correct choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.genetic-programming.org/gpbook1toc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt; is a technique for finding models that are simply computer programs. You generate a bunch of computer programs at random, and then breed the &quot;better&quot; ones together. Repeating this process over time leads to highly optimized programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice feature of GP is that it is extremely flexible when picking what to optimize. So instead of &quot;better&quot; meaning just &quot;more accurate&quot;, &quot;better&quot; can mean the sum of accuracy and $\frac{1}{\#answers\_used}$. The algorithm works the same way, and, with carefully chosen rewards, you may be able to get the best of both worlds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of variations on this. I would probably start with a simple toolkit like &lt;a href=&quot;https://cs.gmu.edu/~eclab/projects/ecj/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EJC&lt;/a&gt;, and standard, boring, genetic programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are specialized techniques for things like your problem too, but you'll probably get 80% of the benefit without needing to pursue them. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-09-14T12:53:58.177" LastActivityDate="2018-09-14T12:53:58.177" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7977" PostTypeId="1" AcceptedAnswerId="7978" CreationDate="2018-09-14T14:44:09.633" Score="3" ViewCount="411" Body="&lt;p&gt;I would like to develop a chatbot that is able to pass the Turing test, i.e. a chatbot that is able to carry on a natural conversation with a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can natural language processing (NLP) be used to do that? What if I combine NLP with neural networks?&lt;/p&gt;&#xA;" OwnerUserId="18233" LastEditorUserId="2444" LastEditDate="2020-06-02T23:09:36.533" LastActivityDate="2020-06-02T23:09:36.533" Title="Can I develop a chatbot to carry on a natural conversation with a human using NLP and neural networks?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7978" PostTypeId="2" ParentId="7977" CreationDate="2018-09-14T15:41:48.153" Score="5" Body="&lt;p&gt;I would not recommend using neural networks and NLP together to create a system sufficiently capable of conversation/dialogue that it would pass that current crop of Turing-like tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversations follow certain rules and regularities (which we have only partially discovered so far), and training an ANN with dialogues in order to pick up those regularities is simply not feasible. In conversations you have a memory of what has been mentioned previously, you build up assumptions about the intentions of your dialogue partner, and keep track of the current topic and sub-topics. This is far too complex to be reduced to a machine learning approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point I would suggest looking at &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;noreferrer&quot;&gt;ELIZA&lt;/a&gt;, developed by Weizenbaum in the mid-1960s. There are plenty of implementations in various programming languages available. Use that as a starting point to extend the capabilities according to topics you want to talk about, and store in memory what the user has said before, trying to refer back to it, etc. This is a lot easier to do with 'symbolic' AI rather than subsymbolic processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of current tech companies offer chatbot variants based on machine learning, but they rarely go beyond intent recognition or simple question-answer dialogues. For more sophisticated dialogues they are simply not suitable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I work for a company producing conversational software)&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-09-14T15:41:48.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7979" PostTypeId="1" AcceptedAnswerId="7981" CreationDate="2018-09-14T20:21:27.440" Score="12" ViewCount="1781" Body="&lt;p&gt;In AlphaZero, the &lt;em&gt;policy network&lt;/em&gt; (or head of the network) maps game states to a distribution of the likelihood of taking each action. This distribution covers all possible actions from that state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is such a network possible? The possible actions from each state are vastly different than subsequent states. So, how would each possible action from a given state be represented in the network's output, and what about the network design would stop the network from considering an illegal action?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="12201" LastEditDate="2018-11-19T02:21:31.173" LastActivityDate="2018-11-19T02:21:31.173" Title="Why does the policy network in AlphaZero work?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7980" PostTypeId="2" ParentId="7947" CreationDate="2018-09-14T20:44:53.197" Score="2" Body="&lt;h1&gt;Can AI provide a more reliable analysis of the gross effects of carbon emissions on extinctions of species ice-cap melting, and other effects?&lt;/h1&gt;&#xA;&lt;p&gt;Yes. The work of &lt;a href=&quot;https://en.wikipedia.org/wiki/Judea_Pearl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Judea Pearl&lt;/a&gt; and others over the last 20 years began out of a desire to address uncertainty within AI. Eventually, this led Pearl to become fascinated by the need to quantifiably determine when one event has caused another, the problem at the root of &amp;quot;correlation is not causation&amp;quot;. He substantively succeeded with the combination of &lt;a href=&quot;https://en.wikipedia.org/wiki/Causal_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;causal modeling&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1305.5506.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the do-calculus&lt;/a&gt;. The do-calculus allows you to formulate queries of the form &amp;quot;To what degree did X cause Y?&amp;quot;, and to automatically determine what measurements are needed to determine the answer in a statistically meaningful way. The algorithms used under the hood are closely related to those used in AI and robotics systems to reason about uncertainty (i.e. Bayesian Inference). Causal modeling is still relatively new, and is not yet as widely used as it could be. An open problem is how to specify what the model of the world looks like from data (rather than just reasoning over a model that is given). If this problem is solved, we could see major improvements in the ability to provide analyses like those you ask about.&lt;/p&gt;&#xA;&lt;h1&gt;Can AI provide a better understanding about whether the runaway scenario described by some climatologists is realistic?&lt;/h1&gt;&#xA;&lt;p&gt;Probably not. Predicting future events without any observable precedents is not something that anyone can be sure about. The questions raised in those kinds of scenarios aren't about calculation, but about modeling. For example, if you believe that there is a large amount of &lt;a href=&quot;https://en.wikipedia.org/wiki/Arctic_methane_emissions&quot; rel=&quot;nofollow noreferrer&quot;&gt;methane trapped in the arctic permafrost&lt;/a&gt;, and you believe that temperatures above a certain range release this much faster than in the past, and you believe that methane warms the climate rapidly, then you would tend to believe that the runaway scenarios are plausible. AI can't tell us whether the methane is there (we have to go measure it, though maybe AI can make the measurement more accurate), and can't tell us how temperature will affect the release rate (again, we have models of this, but they rely on different assumptions, and we have to go measure to find out which are right).&lt;/p&gt;&#xA;&lt;p&gt;Can AI provide better simulations of the impacts of interventions on both climate and the economy, to inform decision making?&lt;/p&gt;&#xA;&lt;p&gt;Maybe. &lt;a href=&quot;https://en.wikipedia.org/wiki/Agent-based_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agent-based modeling&lt;/a&gt; can help with this to some degree, and is arguably part of AI. In fact, it already has been to some degree, by &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/abs/10.1111/1477-9552.12045&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beeger &amp;amp; Troost in J. Agriculture Economics, in 2014&lt;/a&gt; and again in &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/abs/10.1111/agec.12367&quot; rel=&quot;nofollow noreferrer&quot;&gt;2017&lt;/a&gt;, though interest in this kind of modeling looks like a pretty new development in this area. Although ABM can give us reasonable models and help simulate the impact of interventions, ultimately they are just one modeling tool among many. Their potency may improve if more realistic agent models are used, but it is not clear that AI is going to provide advances in this area in the near future.&lt;/p&gt;&#xA;&lt;h1&gt;Automation of climate planning so that the harmful effects of the irresponsibility of one geopolitical entity wishing to industrialize without constraint on other geopolitical entities can be mitigated&lt;/h1&gt;&#xA;&lt;p&gt;Probably not. Although AI techniques have made some kinds of economic planning problems a lot easier, the main barrier to the effects you describe is a social/political one: countries are sovereign, and the world operates as a &lt;em&gt;de facto&lt;/em&gt; anarchy (i.e. the UN is impotent). Your AI model can tell, say, India not to industrialize, but Indians want to enjoy the same kind of lifestyle improvements that Americans do, and would rather enjoy them sooner than later. India would collectively rather that Americans put an enormous tax on their carbon emissions, drive less, eat much, much less beef, and stop flying everywhere, than that Indians continue living on the equivalent of $7,000 each per year. In contrast, Americans would rather that Indians just wait a few decades while the developed world decarbonizes, and only industrialize once we have enough solar panels to replace all our current needs within current industrial economies.&lt;/p&gt;&#xA;&lt;p&gt;Basically this is a resource allocation problem within an anarchy: we can only burn X carbon within Y time, everyone wants to burn some, and the only way to enforce contracts is with the threat of massive violence (or massive economic sanctions, which in turn, are imposed through the threat of violence against other actors if they trade with the sanctioned country). AI can help us answer questions like &amp;quot;How much should the USA pay India in exchange for India not industrializing this year?&amp;quot;, see, e.g. work in &lt;a href=&quot;https://en.wikipedia.org/wiki/Vickrey_auction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Auction Theory&lt;/a&gt;, but AI can't actually make nations do those things if they can't reach a diplomatic compromise.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-14T20:44:53.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7981" PostTypeId="2" ParentId="7979" CreationDate="2018-09-14T21:48:36.547" Score="17" Body="&lt;p&gt;The output of the policy network is as described in the original paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A move in chess may be described in two parts: selecting the piece to move, and then&#xA;  selecting among the legal moves for that piece. We represent the policy π(a|s) by a 8 × 8 × 73&#xA;  stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8×8&#xA;  positions identifies the square from which to “pick up” a piece. The first 56 planes encode&#xA;  possible ‘queen moves’ for any piece: a number of squares [1..7] in which the piece will be&#xA;  moved, along one of eight relative compass directions {N, NE, E, SE, S, SW, W, NW}. The&#xA;  next 8 planes encode possible knight moves for that piece. The final 9 planes encode possible&#xA;  underpromotions for pawn moves or captures in two possible diagonals, to knight, bishop or&#xA;  rook respectively. Other pawn moves or captures from the seventh rank are promoted to a&#xA;  queen.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So each move selector scores the relative probability of selecting a piece in a given square and moving it in a specific way. For example, there is always one output dedicated to representing picking up the piece in A3 and moving it to A6. This representation includes selecting opponent pieces, selecting empty squares, making knight moves for rooks, making long diagonal moves for pawns. It also includes moves that take pieces off the board or through other blocking pieces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical branching factor in chess is around 35. The policy network described above always calculates discrete probabilities for 4672 moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly this &lt;em&gt;can&lt;/em&gt; select many non-valid moves, if pieces are not available, or cannot move as suggested. In fact it does this all the time, even when fully trained, as nothing is ever learned about avoiding the non-valid moves during training - they do not receive positive or negative feedback, as there is never any experience gained relating to them. However, the benefit is that this structure is simple and fixed, both useful traits when building a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple work-around is to filter out impossible moves logically, setting their effective probability to zero, and then re-normalise the probabilities for the remaining valid moves. That step involves asking the game engine for what the valid moves are - but that's fine, it's not &quot;cheating&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst it might be possible to either have the agent learn to avoid non-valid moves, or some clever output structure that could only express valid moves, these would both distract from the core goal of learning how to play the game optimally.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-15T08:50:28.727" LastActivityDate="2018-09-15T08:50:28.727" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7982" PostTypeId="2" ParentId="7328" CreationDate="2018-09-15T03:03:43.873" Score="7" Body="&lt;p&gt;&lt;strong&gt;Quick Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Intel acquired Nirvana, they indicated their belief that analog VLSI has its place in the neuromorphic chips of the near future&lt;sup&gt;1, 2, 3&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether it was because of the ability to more easily exploit the natural quantum noise in analog circuits is not yet public.  It is more likely because of the number and complexity of parallel activation functions that can be packed into a single VLSI chip.  Analog has orders of magnitude advantage over digital in that respect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is likely beneficial for AI Stack Exchange members to come up to speed on this strongly indicated evolution of technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Important Trends and Non-trends in AI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To approach this question scientifically, it is best to contrast analog and digital signal theory without the bias of trends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence enthusiasts can find much on the web about deep learning, feature extraction, image recognition, and the software libraries to download and immediately start experimenting.  It is the way that most get their feet wet with the technology, but the fast track introduction to AI has its down side too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the theoretical foundations of early successful deployments of consumer-facing AI are not understood, assumptions form that conflict with those foundations.  Important options, such as analog artificial neurons, spiked networks, and real time feedback, are overlooked.  The improvement of forms, capabilities, and reliability are compromised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enthusiasm in technology development should always be tempered with at least an equal measure of rational thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convergence and Stability&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a system where accuracy and stability are achieved through feedback, both analog and digital signal values are always mere estimates.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Digital values in a converging algorithm, or, more precisely, an strategy designed to converge&lt;/li&gt;&#xA;&lt;li&gt;Analog signal values in a stable operational amplifier circuit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Understanding the parallel between convergence through error correction in a digital algorithm and stability achieved through feedback in analog instrumentation is important in thinking about this question.  These are the parallels using contemporary jargon, with digital on the left and analog on the right.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&amp;boxdr;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxhd;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxdl;&#xA;&amp;boxv;&amp;nbsp;&amp;nbsp;* Digital Artificial Nets *&amp;nbsp;&amp;nbsp;&amp;boxv;&amp;nbsp;&amp;nbsp;* Analog Artificial Nets *&amp;nbsp;&amp;nbsp;&amp;boxv;&#xA;&amp;boxvr;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvl;&#xA;&amp;boxv;&amp;nbsp;&amp;nbsp;Forward propagation&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&amp;nbsp;&amp;nbsp;Primary signal path&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&#xA;&amp;boxvr;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvl;&#xA;&amp;boxv;&amp;nbsp;&amp;nbsp;Error function&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&amp;nbsp;&amp;nbsp;Error function&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&#xA;&amp;boxvr;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvl;&#xA;&amp;boxv;&amp;nbsp;&amp;nbsp;Convergent&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&amp;nbsp;&amp;nbsp;Stable&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&#xA;&amp;boxvr;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvl;&#xA;&amp;boxv;&amp;nbsp;&amp;nbsp;Saturation of gradient&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&amp;nbsp;&amp;nbsp;Saturation at inputs&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&#xA;&amp;boxvr;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxvl;&#xA;&amp;boxv;&amp;nbsp;&amp;nbsp;Activation function&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&amp;nbsp;&amp;nbsp;Forward transfer function&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;boxv;&#xA;&amp;boxur;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxhu;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxh;&amp;boxul;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Popularity of Digital Circuits&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary factor in the rise of digital circuit popularity is its containment of noise.  Today's VLSI digital circuits have long mean times to failure (mean time between instances when an incorrect bit value is encountered).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The virtual elimination of noise gave digital circuitry a significant advantage over analog circuitry for measurement, PID control, calculation, and other applications.  With digital circuitry, one could measure to five decimal digits of accuracy, control with remarkable precision, and calculate &amp;pi; to a thousand decimal digits of accuracy, repeatably and reliably.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was primarily aeronautics, defense, ballistics, and countermeasures budgets that raised the manufacturing demand to achieve the economy of scale in digital circuit manufacture.  Demand for display resolution and rendering speed is driving the GPU use as a digital signal processor now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these largely economic forces causing the best design choices?  Are digitally based artificial networks the best use of precious VLSI real estate?  That is the challenge of this question, and it is a good one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Realities of IC Complexity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in a comment, it takes tens of thousands of transistors to implement in silicon an independent, reusable artificial network neuron.  This is largely because of the vector-matrix multiplication leading into each activation layer.  It only takes a few dozen transistors per artificial neuron to implement a vector-matrix multiplication and the layer's array of operational amplifiers.  Operational amplifiers can be designed to perform functions such as binary step, sigmoid, soft plus, ELU, and ISRLU.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Digital Signal Noise from Rounding&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Digital signaling is not free of noise because most digital signals are rounded and therefore approximations.  Saturation of the signal in back-propagation first appears as the digital noise generated from this approximation.  Further saturation occurs when the signal is always rounded to the same binary representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rounding approximation occurs in digital signal processing whenever the result of multiplication or division cannot be perfectly represented in binary, which is almost every time.  Only those numbers that can be represented in a finite series of powers of two can be represented perfectly in binary.  In the case of floating point numbers, this still holds.  Here, $v$ is a perfectly represented value, $e$ is the exponent, $k$ is the position of the fractional part, $n$ is the mantissa bit, and $N$ is the number of bits in the mantissa.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v = \sum_{n = 0}^{N} 1_n \, 2^{\, k + e + N - n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programmers sometimes encounter the effects of rounding in double or single precision IEEE floating point numbers when answers that are expected to be 0.2 appear as 0.20000000000001.  One fifth cannot be represented with perfect accuracy as a binary number because 5 is not a factor of 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Science Over Media Hype and Popular Trends&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those engaged in scientific investigation and technology feasibility study should certainly have some awareness of economics, but the direction of technology should be driven by technical merit and utility.  If those criteria are met, it is only a matter of time before the financial community is convinced by those merits The financial merit of theoretical physics appeared weak until $E = mc^2$.  The financial merit of a global public network seemed weak when the Internet was called ARPANET, before Netcom first sold dial in public web access with an email address.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In machine learning as with many products of technology, there are four key quality metrics.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Efficiency (which drives speed and economy of use)&lt;/li&gt;&#xA;&lt;li&gt;Reliability&lt;/li&gt;&#xA;&lt;li&gt;Accuracy&lt;/li&gt;&#xA;&lt;li&gt;Comprehensibility (which drives maintainability)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Sometimes, but not always, the achievement of one compromises another, in which case a balance must be struck.  Gradient descent is a convergence strategy that can be realized in a digital algorithm that nicely balances these four, which is why it is the dominant strategy in multi-layer perceptron training and in many deep networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those four things were central to early cybernetics work of Norbert Wiener prior to the first digital circuits in Bell Labs or the first flip flop realized with vacuum tubes.  The term cybernetics is derived from the Greek κυβερνήτης (pronounced &lt;em&gt;kyvernítis&lt;/em&gt;) meaning steersman, where the ruder and sails had to compensate for constantly changing wind and current and the ship needed to converge on the intended port or harbor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trend driven wiew of this question might surround the idea of whether VLSI can be accomplished to achieve economy of scale for analog networks, but the criteria given by its author is to avoid trend driven views.  Even if that were not the case, as mentioned above, considerably fewer transistors are required to produce artificial network layers with analog circuitry than with digital.  For that reason, it is legitimate to answer the question assuming that VLSI analog is very much feasible at a reasonable cost if attention was directed toward accomplishing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analog Artificial Network Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analog artificial nets are being investigated all over the world, including the IBM/MIT joint venture, Intel's Nirvana, Google, the U.S. Air Force as early as 1992&lt;sup&gt;5&lt;/sup&gt;, Tesla, and many others, some indicated in the comments and the addendum to this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interest in analog for artificial networks has to do with the number of parallel activation functions involved in learning can fit on a square millimeter of VLSI chip real estate.  That depends largely on how many transistors are required.  The attenuation matrices (the learning parameter matrices)&lt;sup&gt;4&lt;/sup&gt; require vector-matrix multiplication, which requires a large number of transistors and thus a significant chunk of VLSI real estate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There must be five independent functional components in a basic multilayer perceptron network if it is to be available for fully parallel training.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The vector-matrix multiplication that parametrizes the amplitude of forward propagation between the activation functions of each layer&lt;/li&gt;&#xA;&lt;li&gt;The retention of parameters&lt;/li&gt;&#xA;&lt;li&gt;The activation functions for each layer&lt;/li&gt;&#xA;&lt;li&gt;The retention of activation layer outputs to apply in back-propagation&lt;/li&gt;&#xA;&lt;li&gt;The derivative of activation functions for each layer&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In analog circuitry, with the greater parallelism inherent in the method of signal transmission, 2 and 4 may not be necessary.  Feedback theory and harmonic analysis will be applied to the circuit design, using a simulator like Spice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To consider cost, an equation can predict with reasonable accuracy the cost of the VLSI product as a function of standard VLSI packaging cost $c_p$, the function representing cost as a function of production volume $c(\int r)$, the function of production rate as a function of time and cost $r(t, c)$, time $t$, cost itself, the widths of each network layer of index $i$ for $I$ layers $w_i$, the number of transistors per attenuator&lt;sup&gt;4&lt;/sup&gt; $\tau_p$, and the number of transistors per activation and its derivative circuits $\tau_a$ and $\tau_d$ respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c = c_p \; c(\int r(t, c) \, dt) \; \Big( \sum_{i = 0}^{I - 2} \, (\tau_p w_i w_{i-1} + \tau_a w_i + \tau_d w_i) + \tau_a w_{I-1} + \tau_d w_{I-1} \Big)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For common values of these circuits in current analog integrated circuits, we have a cost for analog VLSI chips that converges over time to a value at least three orders of magnitude below that of digital chips with equivalent training parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Directly Addressing Noise Injection&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question states, &quot;We are using gradients (Jacobian) or second degree models (Hessian) to estimate next steps in a convergent algorithm and deliberately adding noise [or] injecting pseudo random perturbations to improve convergence reliability by jumping out local wells in the error surface during convergence.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason pseudo random noise is injected into the convergence algorithm during training and in real time re-entrant networks (such as reinforcement networks) is because of the existence of local minima in the disparity (error) surface that are not the global minima of that surface.  The global minima is the optimal trained state of the artificial network. Local minima may be far from optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This surface illustrates the error function of parameters (two in this highly simplified case &lt;sup&gt;6&lt;/sup&gt;) and the issue of a local minima hiding the existence of the global minima.  The low points in the surface represent minima at the critical points of local regions of optimum training convergence. &lt;sup&gt;7,8&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2tS9r.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2tS9r.png&quot; alt=&quot;Error Surface Showing How Global Optimum Can be Missed&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error functions are simply a measure of the disparity between the current network state during training and the desired network state.  During the training of artificial networks, the goal is to find the global minimum of this disparity.  Such a surface exists whether the sample data is labeled or un-labelled and whether the training completion criteria is internal or external to the artificial network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the learning rate is small and the initial state is at the origin of the parameter space, the convergence, using gradient descent, will converge to the leftmost well, which is a local minimum, not the global minimum on the right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if the experts initializing the artificial network for learning is clever enough to pick the mid-point between the two minima, the gradient at that point still slopes toward the left hand minimum, and the convergence will arrive at a non-optimal training state.  If the optimality of training is critical, which it often is, training will fail to achieve production quality results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution in use is to add entropy to the convergence process, which is often simply the injection of the attenuated output of a pseudo random number generator.  Another solution less often used is to branch the training process and try the injection of a large amount of entropy in a second convergent process so that there is a conservative search and a somewhat wild search running in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that quantum noise in extremely small analog circuits are have greater uniformity to the signal spectrum from its entropy than a digital pseudo-random generator and much fewer transistors are required to achieve the higher quality noise.  Whether the challenges of doing so in VLSI implementations have been overcome is yet to be disclosed by research labs embedded in governments and corporations.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Will such stochastic elements used to inject measured quantities of randomness to enhance training speed and reliability be adequately immune to external noise during training?&lt;/li&gt;&#xA;&lt;li&gt;Will they be sufficiently shielded from internal cross-talk?&lt;/li&gt;&#xA;&lt;li&gt;Will a demand arise that will lower the cost of VLSI manufacture sufficiently to reach a point of greater use outside highly funded research enterprises?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All three challenges are plausible.  What is certain and also very interesting is how designers and manufacturers facilitate digital control of the analog signal pathways and activation functions to achieve high speed training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8401400/&quot; rel=&quot;noreferrer&quot;&gt;https://ieeexplore.ieee.org/abstract/document/8401400/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/analog-and-neuromorphic-chips-will-rule-robotic-age&quot; rel=&quot;noreferrer&quot;&gt;https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/analog-and-neuromorphic-chips-will-rule-robotic-age&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://www.roboticstomorrow.com/article/2018/04/whats-the-difference-between-analog-and-neuromorphic-chips-in-robots/11820&quot; rel=&quot;noreferrer&quot;&gt;https://www.roboticstomorrow.com/article/2018/04/whats-the-difference-between-analog-and-neuromorphic-chips-in-robots/11820&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] Attenuation refers to multiplication of a signal output from one actuation by a trainable perameter to provide an addend to be summed with others for the input to an activation of a subsequent layer.  Although this is a physics term, it is often used in electrical engineering and it is the appropriate term to describe the function of the vector-matrix multiplication that achieves what, in less educated circles, is called weighting the layer inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a256621.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.dtic.mil/dtic/tr/fulltext/u2/a256621.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] There are many more than two parameters in artificial networks, but only two are depicted in this illustration because the plot can only be comprehensible in 3-D and we need one of the three dimensions for the error function value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[7] Surface definition: &#xA;$z = (x-2)^2 + (y-2)^2 + 60 - \frac {40} {\sqrt{1 + (y - 1.1)^2 + (x - 0.9)^2}} - \frac {40} {(1 + {((y - 2.2)^2 + (x - 3.1)^2)}^4)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[8] Associated gnuplot commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set title &quot;Error Surface Showing How Global Optimum Can be Missed&quot;&#xA;set xlabel &quot;x&quot;&#xA;set ylabel &quot;y&quot;&#xA;set pm3d at b&#xA;set ticslevel 0.8&#xA;set isosample 40,40&#xA;set xrange [0:4]&#xA;set yrange [0:4]&#xA;set nokey&#xA;splot (x-2)**2 + (y-2)**2 + 60 \&#xA;    - 40 / sqrt(1+(y-1.1)**2+(x-0.9)**2) \&#xA;    - 40 / (1+(y-2.2)**2+(x-3.1)**2)**4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-17T18:07:29.067" LastActivityDate="2018-09-17T18:07:29.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7983" PostTypeId="1" AcceptedAnswerId="8046" CreationDate="2018-09-15T03:08:03.320" Score="4" ViewCount="165" Body="&lt;p&gt;I'm currently implementing the original NEAT algorithm in Swift.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at figure 4 in Stanley's original paper, it seems to me there is a chance that node 5 will have no (enabled) outgoing connection if parent 1 is assumed the fittest parent and the connection is randomly picked from parent 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding of the crossover function correct and can it indeed result in a node with no outgoing connections?&lt;/p&gt;&#xA;" OwnerUserId="18249" LastEditorUserId="16909" LastEditDate="2018-09-15T13:53:04.140" LastActivityDate="2018-09-19T18:03:50.660" Title="Can a crossover result in a node with no outgoing connections?" Tags="&lt;neat&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7984" PostTypeId="2" ParentId="4048" CreationDate="2018-09-15T04:23:14.300" Score="3" Body="&lt;p&gt;Yes.  It is feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overview of the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The design goal of the system seems to be gain a winning strategic advantage by employing one or more artificial networks in conjunction with a card game playing engine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question shows a general awareness of the basics of game-play as outlined in Morgenstern and von Neuman's &lt;em&gt;Game Theory&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At specific points during game-play a player may be required to execute a move.&lt;/li&gt;&#xA;&lt;li&gt;There is a fininte set of move options according to the rules of the game.&lt;/li&gt;&#xA;&lt;li&gt;Some strategies for selecting a move produce higher winning records over multiple game plays than other strategies.&lt;/li&gt;&#xA;&lt;li&gt;An artificial network can be employed to produce game-play strategies that are victorious more frequently that random move selection.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other features of game-play may or may not be as obvious.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At each move point there is a game state, which is needed by any component involved in improving game-play success.&lt;/li&gt;&#xA;&lt;li&gt;In addition to not knowing when the opponent will bluff, in card games, the secret order of shuffled cards can introduce the equivalent of a virtual player the moves of which approximate randomness.&lt;/li&gt;&#xA;&lt;li&gt;In three or more player games, the signaling of partners or potential partners can add an element of complexity to determining the winning game strategy at any point.  Based on the edits, it does not appear like this game has such complexities.&lt;/li&gt;&#xA;&lt;li&gt;Psychological factors such as intimidation can also play a role in winning game-play.  Whether or not the engine presents a face to the opponent is unknown, so this answer will skip over that.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Common Approach Hints&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a common approach to mapping both inputs and outputs, but there is too much to explain in a Stack Exchange answer.  These are just a few basic principles.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All of the modeling that can be done explicitly should be done.  For instance, although an artificial net can theoretically learn how to count cards (keeping track of the possible locations of each of the cards), a simple counting algorithm can do that, so use the known algorithm and feed those results into the artificial network as input.&lt;/li&gt;&#xA;&lt;li&gt;Use as input any information that is correlated with optimal output, but don't use as inputs any information that can not possibly correlate with optimal output.&lt;/li&gt;&#xA;&lt;li&gt;Encode data to reduce redundancy in the input vector, both during training and during automated game-play.  Abstraction and generalization are the two common ways of achieving this.  Feature extraction can be used as tools to either abstract or generalize.  This can be done at both inputs and outputs.  An example is that if, in this game, J &gt; 10 in the same way that A &gt; K, K &gt; Q, Q &gt; J and 10 &gt; 9, then encode the cards as an integer from 2 through 14 or 0 through 12 by subtracting one.  Encode the suits as 0 through 3 instead of four text strings.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The image recognition work is only remotely related, too different from card game-play to use directly, unless you need to recognize the cards from a visual image, in which case LSTM may be needed to see what the other players have chosen for moves.  Learning winning strategies would more than likely benefit from MLP or RNN designs, or one of their derivative artificial network designs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What an Artificial Network Would Do and Training Examples&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary role of artificial networks of these types is to learn a function from example data.  If you have the move sequences of real games, that is a great asset to have for your project.  A very large number of them will be very helpful for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How you arrange the examples and whether and how you label them is worth consideration, however without the card game rules it is difficult to give any reliable direction.  Whether there are partners, whether it is score based, whether the number of moves to a victory, and a dozen other factors provide the parameters of the scenario needed to make those decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Study Up&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main advise I can give is to read, not so much general articles on the web, but read some books and some of the papers you can understand on the above topics. Then find some code you can download and try after you understand the terminology well enough to know what to download.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means book searches and academic searches are much more likely to steer you in the right direction than general web searches.  There are thousands of posers in the general web space, explaining AI principles with a large number of errors.  Book and academic article publishers are more demanding of due diligence in their authors.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:36:49.367" LastActivityDate="2018-10-15T23:36:49.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7985" PostTypeId="1" AcceptedAnswerId="7991" CreationDate="2018-09-15T06:05:05.613" Score="2" ViewCount="84" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1411.1784.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional generative adversarial networks (GAN)&lt;/a&gt;, the objective function (of a two-player minimax game) would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min _{G} \max _{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x} | \boldsymbol{y})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z} | \boldsymbol{y})))]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discriminator and generator both take &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, the auxiliary information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused as to what will be the difference by using &lt;span class=&quot;math-container&quot;&gt;$\log D(x,y)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\log(1-D(G(z,y))$&lt;/span&gt;, as &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; goes in input to &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; in addition to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="18253" LastEditorUserId="2444" LastEditDate="2020-05-18T12:44:51.970" LastActivityDate="2020-05-18T12:44:51.970" Title="Why do we use $D(x \mid y)$ and not $D(x,y)$ in conditional generative adversarial networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;papers&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7991" PostTypeId="2" ParentId="7985" CreationDate="2018-09-15T13:52:18.427" Score="2" Body="&lt;p&gt;It looks like you're asking about the difference between using conditional and joint probabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The joint probability $$D(x,y)$$ is the probability of x and y both happening together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conditional probability $$D(x | y)$$ is the probability that x happens, &lt;em&gt;given&lt;/em&gt; that y has already happened. So, $$D(x,y) = D(y) * D(x | y)$$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that, in a C-GAN, we have some extra information &lt;em&gt;that is given&lt;/em&gt;, like a class label $y$. We actually don't care at all about how likely that information is to appear. We care only about how likely it is to appear with a given $x$ from the source distribution, versus how likely it is to appear with a given $z$ from the generated distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you tried to minimize the joint probabilities, you would be attempting to change something that the networks have no ability to control (the chance of $y$ appearing).&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-15T13:52:18.427" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7992" PostTypeId="1" AcceptedAnswerId="8002" CreationDate="2018-09-15T15:17:47.943" Score="1" ViewCount="85" Body="&lt;p&gt;Currently, I am interested in how NNs or any other AI models can be used for composing music. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are many other interesting applications too, like language processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering that: NNs generally need a cost function for learning. But for example, for composing music, what would be an appropriate cost function? I mean,  algorithms can't (yet) really 'calculate' how good music is, right?&lt;/p&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="16920" LastEditDate="2018-09-15T21:04:59.220" LastActivityDate="2018-09-16T12:57:02.357" Title="How to find a cost function for human data" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7993" PostTypeId="1" AcceptedAnswerId="8003" CreationDate="2018-09-15T16:14:09.657" Score="1" ViewCount="168" Body="&lt;p&gt;At the time when the basic building blocks of machine learning (the perceptron layer and the convolution kernel) were invented, the model of the neuron in the brain taught at the university level was simplistic.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Back when neurons were still just simple computers that electrically beeped untold bits to each other over cold axon wires, spikes were not seen as the hierarchical synthesis of every activity in the cell down to the molecular scale that we might say they are today. In other words, spikes were just a &lt;strong&gt;summary report of inputs to be integrated&lt;/strong&gt; with the current state, and passed on. In comprehending the intimate relationships of mitochondria to spikes (and other molecular dignitaries like calcium) we might now more broadly interpret them as &lt;strong&gt;synced messages that a neuron sends to itself&lt;/strong&gt;, and by implication its spatially extended inhabitants. Synapses weigh this information heavily but ultimately, but like the electoral college, fold in a heavy dose of local administration to their output. The sizes and positions within the cell to which mitochondria are deployed can not be idealized or anthropomorphized to be those metrics that the neuron decides are best for itself, but rather &lt;strong&gt;what is thermodynamically demanded&lt;/strong&gt;.&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Notice the reference to summing in the first bolded phrase above.  This is the astronomically oversimplified model of biology upon which contemporary machine learning was built.  Of course ML has made progress and produced results.  This question does not dismiss or criticize that but rather widen the ideology of what ML can become via a wider field of thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the second two bolded phrases, both of which denote statefulness in the neurons.  We see this in ML first as the parameters that attenuate the signals between arrays of artificial neurons in perceptrons and then, with back-propagation into deeper networks.  We see this again as the trend in ML pushes toward embedded statefulness by integrating with object oriented models, the success of LSTM designs, the interrelationships of GAN designs, and the newer experimental attention based network strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But does the achievement of higher level thought in machines, such as is needed to ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fly a passenger jet safely under varying conditions,&lt;/li&gt;&#xA;&lt;li&gt;Drive a car in the city,&lt;/li&gt;&#xA;&lt;li&gt;Understand complex verbal instructions,&lt;/li&gt;&#xA;&lt;li&gt;Study and learn a topic,&lt;/li&gt;&#xA;&lt;li&gt;Provide thoughtful (not mechanical) responses, or&lt;/li&gt;&#xA;&lt;li&gt;Write a program to a given specification&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... requiring from us a much more radical is the transition in thinking about what an artificial neuron should do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scientific research into brain structure, its complex chemistry, and the organelles inside brain neurons have revealed significant complexity.  Performing a vector-matrix multiplication to apply learning parameters to the attenuation of signals between layers of activations is not nearly a simulation of a neuron.  Artificial neurons are not very neuron-like, and the distinction is extreme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A little study on the current state of the science of brain neuron structure and function reveals the likelihood that it would require a massive cluster of GPUs training for a month just to learn what a single neuron does.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are artificial networks based on the perceptron design inherently limiting?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Fast spiking axons take mitochondria for a ride,&#xA;by John Hewitt, Medical Xpress, January 13, 2014, &#xA;&lt;a href=&quot;https://medicalxpress.com/news/2014-01-fast-spiking-axons-mitochondria.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medicalxpress.com/news/2014-01-fast-spiking-axons-mitochondria.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-20T05:17:24.117" LastActivityDate="2018-09-20T05:17:24.117" Title="Are artificial networks based on the perceptron design inherently limiting?" Tags="&lt;neural-networks&gt;&lt;research&gt;&lt;hardware&gt;&lt;architecture&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7995" PostTypeId="2" ParentId="3065" CreationDate="2018-09-15T18:25:51.197" Score="0" Body="&lt;p&gt;Cross-entropy is an entropy, not an entropy difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more natural and perhaps intuitive way to conceptualize the categorization criteria is through a relation rather than a definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H(P, Q) - H(P) = D_{\mathrm{KL}}(P\|Q) = - \sum_i P(i)  \log\frac{Q(i)}{P(i)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This follows parallels, identified by Claude Shannon with John von Neumann, between quantum mechanical thermodynamics and information theory.  Entropy is not an absolute quantity.  It is a relative one, so neither entropy nor cross entropy can be calculated, but their difference can be for either the discrete case above or its continuous sibling below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H(P, Q) - H(P) = D_{\mathrm{KL}}(P\|Q) = - \int_{-\infty}^\infty \, p(x) \log\frac {q(x)} {p(x)} \, dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although we may see $H(...) = ...$ in the literature, with no H'(...) on the right hand side of the equation, it is not technically accurate.  In such cases there is always some implied entropy to which the entropy on the left hand side is relative.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-15T18:25:51.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7996" PostTypeId="1" CreationDate="2018-09-15T18:31:48.160" Score="3" ViewCount="102" Body="&lt;p&gt;I am trying to understand the dimensionality of the outputs of convolution operations. Suppose a convolutional layer with the following characteristics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input map &lt;span class=&quot;math-container&quot;&gt;$\textbf{x} \in R^{H\times W\times D}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;A set of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; filters, each of dimension &lt;span class=&quot;math-container&quot;&gt;$\textbf{f} \in R^{H'\times W'\times D}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;A stride of &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;s_x, s_y&amp;gt;$&lt;/span&gt; for the corresponding &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; dimensions of the input map&lt;/li&gt;&#xA;&lt;li&gt;Either valid or same padding (explain for both if possible)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What should be the expected dimensionality of the output map expressed in terms of &lt;span class=&quot;math-container&quot;&gt;$H, W, D, F, H', W', s_x, s_y$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="18267" LastEditorUserId="2444" LastEditDate="2020-04-27T12:33:38.647" LastActivityDate="2020-04-27T12:33:38.647" Title="What is the dimensionality of the output map, given the dimensionality of the input map, number of filters, stride and padding?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7998" PostTypeId="1" CreationDate="2018-09-15T18:55:27.197" Score="4" ViewCount="439" Body="&lt;p&gt;Keras' convolutional and deconvolutional layers are designed for square grids. Is there was a way to adapt them for use in hexagonal grids?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we were using &lt;a href=&quot;https://www.redblobgames.com/grids/hexagons/#coordinates-axial&quot; rel=&quot;nofollow noreferrer&quot;&gt;axial coordinates&lt;/a&gt;, the input of the kernel of radius 1 centered at &lt;code&gt;(x,y)&lt;/code&gt; should be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;[(x-1,y), (x-1,y+1), (x,y-1), (x,y+1), (x+1,y-1), (x+1, y)]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option is to fudge it with a 3 by 3 box, but then you are using cells at different distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Modify Kera's convolutional layer code to use those inputs instead of the default inputs. The problem is that Kera calls its backend instead of implementing it itself, which means we need to modify the backend too.&lt;/li&gt;&#xA;&lt;li&gt;Use a 3 by 3 box, but set the weights at &lt;code&gt;(x-1,y-1)&lt;/code&gt; and &lt;code&gt;(x+1,y+1)&lt;/code&gt; to zero. Unfortunately, I do not know how to permanently set weights to a given value in Kera.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;a href=&quot;https://www.redblobgames.com/grids/hexagons/#coordinates-cube&quot; rel=&quot;nofollow noreferrer&quot;&gt;cube coordinates&lt;/a&gt; instead of Axial coordinates. In this case, a 3 by 3 by 3 box will only contain the central hex's neighbors and inputs set to 0. The problem is that it makes the input array much bigger. Even more problematic, some coordinates that correspond to non-hexes (such as (1,0,0)) will be assigned non-zero outputs (since (0,0,0) falls within its 3 by 3 by 3 box).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are there any better solutions?&lt;/p&gt;&#xA;" OwnerUserId="18006" LastEditorUserId="1581" LastEditDate="2018-09-16T10:49:22.067" LastActivityDate="2019-04-16T07:44:43.477" Title="Convolutional Layers on a hexagonal grid in Keras" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7999" PostTypeId="1" CreationDate="2018-09-15T20:16:07.100" Score="1" ViewCount="40" Body="&lt;p&gt;We have data in text format as sentences.&#xA;The goal is to detect rules which exist in this set of sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a limited set of contextless sentences that fit a pattern and want to find the pattern.&#xA;I might not have sentences that don't fit the pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should be an approach to do that?&lt;/p&gt;&#xA;" OwnerUserId="18271" LastEditorUserId="18271" LastEditDate="2018-09-15T20:46:27.200" LastActivityDate="2018-09-17T13:34:21.757" Title="What is the approach to deduce formal rules based on data?" Tags="&lt;machine-learning&gt;&lt;learning-algorithms&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8000" PostTypeId="1" AcceptedAnswerId="8012" CreationDate="2018-09-15T22:29:39.477" Score="5" ViewCount="267" Body="&lt;p&gt;Usually neural networks consist from layers, but is there research effort that tries to investigate more general topologies for connections among neurals, e.g. arbitrary directed acyclic graphs (DAGs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess there can be 3 answers to my question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;every imaginable DAG topology can be reduced to the layered DAGs already actively researched, so, there is no sense to seek for more general topologies;&lt;/li&gt;&#xA;&lt;li&gt;general topologies exist, but there are fundamental restrictions why they are not used, e.g. maybe learning is not converging in them, maybe they generate chaotic osciallations, maybe they generate bifurcations and does not provide stability;&lt;/li&gt;&#xA;&lt;li&gt;general topologies exist and are promising, but scientists are not ready to work with them, e.g. maybe they have no motivation, standard layered topologies are good enough.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But I have no idea, which answer is the correct one. Reading the answer on &lt;a href=&quot;https://stackoverflow.com/questions/46569998/calculating-neural-network-with-arbitrary-topology&quot;&gt;https://stackoverflow.com/questions/46569998/calculating-neural-network-with-arbitrary-topology&lt;/a&gt; I start to think that answer 1 is the correct one, but there is no reference provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If answer 3 is correct, then big revolution can be expected. E.g. layered topologies in many cases reduces learning to the matrix exponentiation and good tools for this are created - TensorFlow software and dedicated processors. But there seems to be no software or tools for general topologies is they have some sense indeed.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="10135" LastEditDate="2018-10-17T09:08:13.607" LastActivityDate="2018-10-17T09:08:13.607" Title="Neural networks of arbitrary/general topology?" Tags="&lt;neural-networks&gt;&lt;topology&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8001" PostTypeId="2" ParentId="7999" CreationDate="2018-09-15T23:15:38.667" Score="1" Body="&lt;p&gt;If you don't have non-examples of your pattern and don't have some kind of heuristic guide, unfortunately the answer is that you can't. &quot;All sentences&quot; will always be 100% compatible with your examples, and you'll never be able to collect evidence that disconfirms (or even decreases the likelihood) of that hypothesis. Even if you rule out that hypothesis by fiat, the hypothesis that the only non-accepted sentence is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The answers for “I have a bunch of books and want to infer topics” and “I have a set of contextless sentences that fit a pattern and another set of contextless sentences that don’t fit that pattern and want to find the pattern” are extremely different.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;will be impossible to rule out&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a good enough heuristic guide, you might be able to do something, but that would require knowing statistical facts about examples and non-examples and then sampling from the world IID, and just happening to not get any non-examples. That doesn't seem like the situation you're in. If you really have nothing to quantify the non-examples in any way, there's nothing that can be done.&lt;/p&gt;&#xA;" OwnerUserId="12732" LastEditorUserId="12732" LastEditDate="2018-09-17T13:34:21.757" LastActivityDate="2018-09-17T13:34:21.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8002" PostTypeId="2" ParentId="7992" CreationDate="2018-09-15T23:22:45.370" Score="1" Body="&lt;p&gt;You've hit upon the central conundrum of supervised learning: if you want a machine to learn to do something, you need to know how to explain what that something is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of music, there are several possible approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Make one set of &quot;bad&quot; songs, and one set of &quot;good&quot; songs. Develop a measure of how similar two songs are (maybe euclidian distance between their discrete &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier transforms&lt;/a&gt; is a good starting place?). Your cost function is then based minimizing the average distance to &quot;good&quot; songs, and maximizing the average distance to &quot;bad&quot; songs. This may not work well though, because good and bad songs might differ only in the occasional misplaced notes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Move to a reinforcement learning paradigm. Listen to each song proposed by your network. Give is a score based on your subjective enjoyment. Your cost function is based on maximizing this score. This might work well, but again, it might not. Music is tricky.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use unsupervised approaches. Reward your network just for making something that resembles music (perhaps using the Fourier transform approach above), without labelling good and bad. The advantage is that you don't need to decide what is good or bad, and so you can use a lot more music in your dataset. The drawback is music as a whole might be too diverse to learn easily from examples.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Treat your music as a sequence of notes, and &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;train a generative model&lt;/a&gt; to predict future notes on the basis of past notes. You can then generate new music by starting the model with a set of notes and letting it generate new ones for a long time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-09-16T12:57:02.357" LastActivityDate="2018-09-16T12:57:02.357" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8003" PostTypeId="2" ParentId="7993" CreationDate="2018-09-16T01:42:59.810" Score="1" Body="&lt;p&gt;In my opinion, there are many functions in our brain. Surely much more than  the artificial neural network nowadays. I guess this is the field of brain science or cognitive psychology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some brain structures may help for certain applications, but not all. Neural network though is a simplest form of our brain, but has the most general usages. On the other words, if you want to improve the neural networks, different fields or different functions may needs totally different structures. You can refer this as so many types of neural networks nowadays for different applications.&lt;/p&gt;&#xA;" OwnerUserId="18276" LastEditorUserId="18276" LastEditDate="2018-09-16T10:23:08.080" LastActivityDate="2018-09-16T10:23:08.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8007" PostTypeId="1" CreationDate="2018-09-16T14:55:08.080" Score="1" ViewCount="26" Body="&lt;p&gt;If one uses one of the open source implementations of the &lt;a href=&quot;https://deepmind.com/blog/wavenet-generative-model-raw-audio/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WaveNet generative speech synthesis design&lt;/a&gt;, such as &lt;a href=&quot;https://r9y9.github.io/wavenet_vocoder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://r9y9.github.io/wavenet_vocoder/&lt;/a&gt;, and trains using something like the &lt;a href=&quot;http://festvox.org/cmu_arctic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CMU's arctic corpus&lt;/a&gt;, now can one add a voice that sounds younger, older, less professional, or in some other way distinctive.  Must the entire training begin from scratch, or is there a more resource and time friendly way?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2020-10-31T02:20:28.127" Title="Adding voices to voice synthesis corpuses" Tags="&lt;training&gt;&lt;generative-model&gt;&lt;speech-synthesis&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8008" PostTypeId="2" ParentId="7328" CreationDate="2018-09-16T15:29:34.153" Score="2" Body="&lt;p&gt;I believe that most people have pretty much answered the question diligently in a really informative way. I would just like to say that we use digital circuits commonly because that is the &lt;strong&gt;existing technology&lt;/strong&gt; and that definitely analog circuits seem really promising.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, at this moment, this idea is not very well-developed despite the amount of research done in the past years. No company so far, has tried to implement the idea at a commercial level where they are making such chips for use outside their labs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, this idea feels like a new approach and has a great potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, with our lack of understanding about how some models work, some just don't for a problem; how neural networks &lt;strong&gt;really&lt;/strong&gt; solve such complex problems and many other things.Therefore, it is still quite a distant technology to reach its full potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS I'm still a beginner in this field and think that my opinion does not count so, if I was redundant anywhere or failed to give you the expected answer then, I sincerely regret it.&lt;/p&gt;&#xA;" OwnerUserId="17209" LastActivityDate="2018-09-16T15:29:34.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8010" PostTypeId="2" ParentId="7328" CreationDate="2018-09-16T17:07:00.880" Score="1" Body="&lt;p&gt;&lt;strong&gt;Hava Siegelmann&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the first look Analog computing is superior to digital one. Quantum computers are faster than Von-Neumann computers and neuromorphic chips need less energy than Intel CPUs. Also from a theoretic point of view many speaks for analog computers. Hava Siegelmann has researched the Super-turing capability of neural network, which means that an analog computer can emulate a digital one but not the other way around. So why should we not using analog computing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Stephen Wolfram&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason has to do with the education system. Classical mathematics which is teached in schools is analog mathematics. It is based on slide rules, logarithm table and the thinking in circuits. In contrast, thinking in discrete values of an algorithm and describe the world in zero and ones is fundamental different and leads us to a new kind of mathematics. Stephen Wolfram has explained, that the understanding of cellular automatons is an important step to describe the universe and he is right. Ignoring analog mathematics and preferring turing capable computer languages is a powerful method in education. It helps not only to become familiar with computers but with all the other things like medicine, literature and economy too. Even if analog machines are technical superior we should prefer slow but discrete Turing-machines, especially these one which implements AI-related algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Teaching mathematics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand the difference between digital and analog computation we must focus on the mathematics itself which is utilized in schools. If the idea is to push analog computation forward, the appropriate kind of mathematics is grouped around electrical fields, integration and differentiation. In schools this is teached under the umbrella term “Mathematical analysis”. This topic was very important in the past, because analysis helps to build bridges, machines and cars. In all of these domains vector algebra for describing geometric space is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If analog computation is so powerful, why does anybody need digital mathematics? It has to do with algorithm. What planimeter and differential analyzer doesn't have to offer are programming capabilities. It is not possible to define algorithms and artificial languages. A look into the history of mathematics shows, that algorithm-theory was not very common in the past. In modern mathematics it is discussed under the term &lt;a href=&quot;https://en.wikipedia.org/wiki/Lambda_calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lambda calculus&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Halting problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The funny thing is, that on the first look Lamda calculus has no practical applications. It is not needed if somebody want's to calculate the area of a bridge. Algorithm theory is a thought school to improve critical thinking. It is a philosophy needed by humans, not by machines.&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorDisplayName="user11571" LastEditDate="2018-09-18T07:12:27.617" LastActivityDate="2018-09-18T07:12:27.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8012" PostTypeId="2" ParentId="8000" CreationDate="2018-09-17T02:22:21.090" Score="2" Body="&lt;p&gt;The simplistic neural networks that have been given away for free after they prove insufficient by themselves in field use consist solely of two orthogonal dimensions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Layer width &amp;mdash; the number of ordinal numbers or floating point numbers that represent the signal path through any given layer comprise of an array of layer elements&lt;/li&gt;&#xA;&lt;li&gt;Network depth &amp;mdash; the number of layers in the primary signal path, which is the number of activation function arrays or convolution kernels or whatever other elements Usually neural networks consist from layers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, in large corporations that have AI pipelines, this is not the case.  We are beginning to see more interesting topologies in open source.  We see this in generative systems for images, text, and speech.  We see this in robotic control of robots.  The truth is that these more sophisticated topologies have been in play for years, but were just not appearing in the open source community because they were company confidential.  Enough academic work, releasing of portions of corporate IP, and the accumulation of independent OSS work has occurred to start to see these topologies in GIT repos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cyclic Not Acyclic&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial network topologies are generally cyclic, not acyclic in terms of their causality or their signal pathways, depending on how you depict them theoretically.  These are three basic examples from among dozens in the literature and in the open source repositories.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Back-propagation represents the introduction of a deliberate cycle in signal paths in a basic multilayer perceptron, making that topology a sequence of layers represented by vertices, connected sequentially by a set of directed edges representing forward propagation, and a set of directed edges in the reverse direction to distribute the corrective error determined at the network output according to the principle of gradient descent.  For efficiency, the corrective signal is distributed recursively backward through the layers to the &lt;span class=&quot;math-container&quot;&gt;$N - 1$&lt;/span&gt; matrices of parameters attenuating the signals between &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; layers.  Back propagation requires the formation of these &lt;span class=&quot;math-container&quot;&gt;$N - 1$&lt;/span&gt; cycles for convergence to occur.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In a generative adversarial network (GAN), we have the signal path of each of the two networks feeding the training criteria of the other.  Such a topological arrangement is like negative feedback in a stable control system in that an equilibrium is formed between the generative network and discriminative network. The two directed edges, (a) the one that causally affects G with Ds result, and (b) the one that causally affects D with Gs result, create a cycle on top of the cycles in each of G and D.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In attention based networks being touted as theoretically advantageous over LSMT (which has been dominating over CNNs) has a much more complex topology and more cycles above those in supervisory layer than those in GANs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis of Answer One of Three&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that every directed graph can be realized in an arbitrarily large RNN because they are Turing complete, but that doesn't mean they are a great topology for all finite algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing was aware that his punched tape model was not the best general purpose, high speed computing architecture.  He was not intending to prove anything about computing speed but rather what could be computed.  His Turing machine had a trivial topology deliberately.  He wanted to illustrate his completeness theorem to others and resurrect the forward movement of rationalism after Gödel disturbed it with his two incompleteness theorems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, John von Neumann proposed his computing architecture, with a central processing unit (CPU) and unified data and instruction bus, to reduce the number of relays or vacuum tubes, not to maximize parallel algorithm execution.  That topology as a directed graph has the instruction controller and the arithmetic unit in the center and everything else branching out from the data and address bus leading from them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That a topology can accomplish a task is no longer a justification for persisting in the use of that topology, which is why Intel acquired Nirvana, which deviates from traditional von Neumann architecture, DSP architecture, and the current CUDA core architecture that NVidia GPUs use and offer for artificial network realization through C libraries that can be called via integrated Java and Python adapters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is definitely sense to seek for more general topologies, if they are fit for the purpose, just as with Turing's or von Neuman's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis of Answer Two of Three&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General topologies exist, the most economically viable of which is the CUDA cores begun by NVidia, which can be configured for MLPs, CNNs, RNNs, and general 2D and 3D video processing.  They can be configured with or without cycles depending on the characteristics of the parallelism desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The realization of topologies unlike the Cartesian arrangements of activation functions in artificial networks or the kernel cells in convolution engines do have barriers to use, but they are not fundamental restrictions. The primary barrier is not one of hardware or software.  It is one of linguistics.  We don't think topologically because we don't talk topologically.  That's what is great about this question's challenge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FORTRAN began to dominate over LISP during the time when general purpose programming began to emerge in many corporations.  That is not surprising because humans communicate in orthogonal ways.  It is cultural.  When a child scribbles, teachers are indoctrinated to say nice things but respond by drawing a shape.  If the child draws a square, the teacher smiles.  The child is given blocks.  The books are rectangular.  Text is justified into rectangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see this in building architecture dating back to Stonehenge.  Ninety degree angles are clearly dominant in artificial things, whereas nature doesn't seem to have that bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although directed graphs were easy to implement and traverse in recursive structure and were commonplace in the LISP community.  FORTRAN with its realization of vectors and matrices in one and two dimensional arrays respectively were easier to grasp by those with less theoretical background in data structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is that even if learning EMMASCRIPT (JavaScript) which has its seed from the LISP community and is not biased toward orthogonal data structures, people tend to proceed from HelloWorld.js to something with a basic loop in it, with an underlying array through which the loop iterates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three wonderfully inquisitive and insightful phrases in answer two of three.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maybe learning is not converging in them &amp;mdash; Interestingly an algorithm cannot learn without a cycle.  Directly applying a formula or converging using a known convergent series of terms does not qualify as learning.  Gradient descent relies entirely on the cyclical nature of a corrective action at the end of each sample processing or batch of them.&lt;/li&gt;&#xA;&lt;li&gt;Maybe they generate chaotic [oscillations] &amp;mdash; This gets into chaos theory and control theory's concept of stability.  They can do so, but so can a basic multilayer perceptron if the learning rate is set to high.&lt;/li&gt;&#xA;&lt;li&gt;Maybe they generate bifurcations &amp;mdash; Now we have fully entered the realm of chaos, which is arguably closely related to creativity.  Mendelbrot proposed the relationship between new forms of order and the apparent chaotic behavior arising from the appropriate level of feedback in a system with signal path components that cannot be modelled with a first degree equation.  Since then, we find that most phenomena in nature are actually strange attractors.  The plot of the training of a network from a continuous feed of consistently distributed data in phase space will reveal ... you guessed it ... a strange attractor.  When purtibations are deliberately injected into a training epoch from a pseudo-random number generator, the specific purpose is a bifurcation, so that the global optimization can be found when the training gets stuck in a local optimization.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis of Answer Three of Three&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General topologies exist and are promising and researchers are ready to work with them.  It is enthusiasts that can have a dismissive attitude.  They don't yet understand the demos they've downloaded and painstakenly tweaked to run on their computer, they're about to launch their AI carrier amidst the growing demand from all the media hype, and now someone is introducing something interesting and not yet implemented in code.  The motivational direction is generally to either dismiss or resist the creative proposals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, Google, CalTech, IBM, MIT, U Toronto, Intel, Tesla, Japan, and a thousand other governments, institutions, corporations, and open source contributors will solve that problem, provided people keep talking about topology and the restrictions inherent in purely Cartesian thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Misunderstanding Topology to Mean Dimensionality or Topography&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been some confusion in terms.  The SO reference in the question is an example of thinking that changing an array dimension is changing the topology.  If such were so, then there would be no change one could make to the geometry of an AI system that would not be topological.  Topology can only have meaning if there are features that are not topological.  When one draws a layer, they don't need to increase the height of the rectangle representing it if the number of activitations, the width of the layer, is changed from 100 to 120.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also seen academic papers that called the texture or roughness of an error surface its topology.  That completely undermines the concept of topology.  They meant to use the term topography.  Unfortunately neither the publisher nor the editor noticed the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Software or Tools&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most programming languages support directed graphs in recursive hashmaps.  LISP and its derivatives supported them at a more machine instruction efficient level, and that's still the case.  Object oriented databases and graph libraries exist and are in use.  Google uses them extensively in web indexing and lookup.  FaceBook's API is called the Graph API, because it is a query and insert API into the graph that is FaceBook's user data store.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explosion is here in global software giants.  There is open source for it.  The revolution that is missing is among those who are not yet educated as to the meaning of topology, the difference between a hierarchy and a network or the role of feedback in any learning system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding Java and Python there are many barriers to the revolution in thinking, primarily these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are now key words in either Java or Python to directly deal with directed graphs other than the idea of a class with references to instances of other classes, which is quite limited.  None of these languages can add edge types with a single simple language construct.&lt;/li&gt;&#xA;&lt;li&gt;There is no mapping to hardware yet, although Nirvana allegedly developed one, and Intel acquired Nirvana, so that barrier may evaporate soon.&lt;/li&gt;&#xA;&lt;li&gt;The bias still exists in preschool, kindergarten, and first grade&lt;/li&gt;&#xA;&lt;li&gt;Hilbert spaces are not generally taught in calculus&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Graphviz and other graphing software that auto-generates diagrams from unconstrained directed or bidirectional graph representations have done much to bust through the barriers because the generated images are visible across the web.  It may be through visual representations of graphs that linguistic representations, thought, hardware, and software begin to emerge representing the paradigm shift the question investigates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not that constraints are not useful.  Only some patterns and paradigms produce results, but since the results from the human brain demand attention, and the human brain is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Not at all orthogonal,&lt;/li&gt;&#xA;&lt;li&gt;Not implemented using Cartesian neural patterns, and&lt;/li&gt;&#xA;&lt;li&gt;Not topologically a box,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One can all but conclude that those are not particularly well chosen constraints.  Neither is the acyclic criteria.  Nature is cyclic, and intelligence probably requires it in many ways and at many levels.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-15T23:35:53.797" LastActivityDate="2018-10-15T23:35:53.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8013" PostTypeId="1" AcceptedAnswerId="8020" CreationDate="2018-09-17T04:44:46.687" Score="1" ViewCount="106" Body="&lt;p&gt;I'm training a language model with &lt;code&gt;5000&lt;/code&gt; vocabularies using a single &lt;code&gt;M60 GPU&lt;/code&gt; (w/ actually usable memory about 7.5G). &#xA;&lt;br&gt;The number of tokens per batch is about &lt;code&gt;8000&lt;/code&gt;, and the hidden dimension to the softmax layer is &lt;code&gt;512&lt;/code&gt;. So, if I understand correctly, fully-connected (softmax) layer theoretically consumes &lt;code&gt;5000*8000*512*4=81.92GB&lt;/code&gt; for a forward pass (4 is for float32).&#xA;&lt;br&gt;But the GPU performed the forward and backward passes without any problem, and it says the GPU memory usage is less than &lt;code&gt;7GB&lt;/code&gt; in total. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used PyTorch. What's causing this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To be clearer, the input to the final fc layer (256x5000 matrix) is of size [256, 32, 256]. &lt;/p&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="18298" LastEditDate="2018-09-17T17:22:59.283" LastActivityDate="2018-09-17T18:35:57.343" Title="Calculation of GPU memory consumption on softmax layer doesn't match with the empirical result" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8015" PostTypeId="2" ParentId="7993" CreationDate="2018-09-17T08:42:20.950" Score="2" Body="&lt;p&gt;In the perceptron design generally used in Artificial Neural Networks, we know precisely what a single neuron is capable of computing. It can compute a function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = g(w^{\top} x),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x$ is a vector of inputs (may also be vector of activation levels in previous layer), $w$ is a vector of learned parameters, and $g$ is an activation function. We know that a single node in such an ANN can compute precisely that, and nothing else. This observation could be interpreted as &quot;of course it is limited; it can do precisely his and nothing else&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The universal function approximation theory tells us (very informally here) that if a Neural Network is &quot;big enough&quot;, has at least 1 hidden layer, and has non-linear activation functions, it may in theory learn to approximate any function reasonably well. If we add recurrence (i.e. an RNN), we also get, in theory, Turing completeness. Based on this, we could say that they are not particularly limited in theory... but of course there are many complications in practice:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How big is &quot;big enough&quot;?&lt;/li&gt;&#xA;&lt;li&gt;How do we effectively learn our parameters? (SGD is the most common approach, but can get stuck in local minima; global optimization methods like evolutionary algorithms wouldn't get stuck... but I don't believe that they're famous for being fast either).&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Just the observation that they may not be highly limited in theory of course doesn't mean that there wouldn't be anything else that works better in practice either. I can very well imagine that a more complex model (trying to simulate additional functionality that we also observe in the brain) may be more capable of learning more complex functions more easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important caveat there is that it tends to be the case that more complex function approximators tend to be more difficult to train in practice. We understand very well how to effectively train a linear function approximator. They also typically aren't very data-hungry. The downside is, they can only approximate linear functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also understand quite well how to train, for example, Decision Trees. They're still quite easy models to understand intuitively, they can learn more complicated functions than just linear functions. I'd say we have a worse understanding of how to train them well than linear functions, but still a good understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ANNs as they are used now... it looks like they are more powerful in practice than the two mentioned above, but there's also still more &quot;mystery&quot; surrounding them (in particular the deep variants). We can train them quite well, but we don't understand everything about them as well as we'd like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I'd expect that trend to continue if we try to imitate the brain more faithfully. I wouldn't be surprised if there exist more powerful things out there, but they'll be more complex to understand, more difficult to train, maybe also more data-hungry (current ANNs already tend to be very data-hungry).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-17T08:42:20.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8016" PostTypeId="1" AcceptedAnswerId="8017" CreationDate="2018-09-17T11:36:42.407" Score="4" ViewCount="607" Body="&lt;p&gt;I have a very imbalanced dataset of two classes: 2% for the first class and 98% for the second. Such imbalance does not make training easy and so balancing the data set by undersampling class 2 seemed like a good idea. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as I think about it, should not the machine learning algorithm expect the same data distribution in nature as in its training set? I know, for sure, that the distribution of data in nature matches my imbalanced dataset. Does that mean that the balanced dataset will negatively affect the neural net performance with testing? when it assumed a different distribution of data caused by my balanced data set. &lt;/p&gt;&#xA;" OwnerUserId="17582" LastActivityDate="2018-09-17T12:42:20.777" Title="Does balancing the training data set distribution for a neural network affect its understanding of the original distribution of data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8017" PostTypeId="2" ParentId="8016" CreationDate="2018-09-17T12:34:49.200" Score="3" Body="&lt;p&gt;This is a very good question. Your problem is the classic classification problem of Neural Networks. In this problem the main objective of the Neural Network is to transform the data by some non-linear (in general) transformation so that the data becomes linearly separable for the final layer to perform classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point to Note:&lt;/strong&gt; This is not a regression problem, so that you are trying to fit a curve. Whenever there is a regression problem logically you can use a PDF to write some kind of information about new data. You can express mathematically the probability of your data falling within a certain range of error, since this is a optimising continuous function problem (generally RMSE). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not in the case of classifier. Classifier follow &lt;a href=&quot;https://en.wikipedia.org/wiki/Bernoulli_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bernoulli probability&lt;/a&gt; (though we represent the cost function as continuous). So current event is independent of past events. This makes a classifier harder to train for unbalanced class. So if we write:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;func foo(data):&#xA;    return True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;It pretty much has 98% accuracy, but you can understand we do not want this type of classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general we want both classes to have good accuracy scores, sometimes this is measured by $F1_{score}$ but I like to think it in terms of proportionality. If we have $n$ examples with $m$ in one class, then and the classifier $a_1$ and $a_2$ correct predictions in both classes respectively, then I would check both the metric $ \frac {a_1}{m}$ and $ \frac {a_2}{n-m}$, which gives you the general idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also in practical the identification 2% is sometimes far more important than the rest 98% (air-plane defects, cancer detection). So we use special ML algorithm called Anomaly Detectors for such type of problems.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-09-17T12:42:20.777" LastActivityDate="2018-09-17T12:42:20.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8019" PostTypeId="2" ParentId="3298" CreationDate="2018-09-17T18:05:10.213" Score="0" Body="&lt;p&gt;Debug and Validation of large software systems like video software stack&#xA;If you take example as validation and debug of video software stack, It is very difficult for naked eye to identify failures on the display. In this case you can use DNN based image classifier to identify functional failure.&lt;/p&gt;&#xA;" OwnerUserId="18195" LastActivityDate="2018-09-17T18:05:10.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8020" PostTypeId="2" ParentId="8013" CreationDate="2018-09-17T18:35:57.343" Score="1" Body="&lt;p&gt;GPU DRAM capacity - 7.5G&#xA;the below link explains how nVIDIA GPU cUDNN does memory optimization.&#xA;&lt;a href=&quot;https://devblogs.nvidia.com/optimizing-recurrent-neural-networks-cudnn-5/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://devblogs.nvidia.com/optimizing-recurrent-neural-networks-cudnn-5/&lt;/a&gt;&#xA;the below link has detailed steps to calculate memory required by parameters and data.&#xA;&lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#case&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/convolutional-networks/#case&lt;/a&gt;&#xA;one data point missing in question is number of output classes in softmax layer.&#xA;The above two links will help you to calculate memory required and how software handles large matrix multiplication. &lt;/p&gt;&#xA;" OwnerUserId="18195" LastActivityDate="2018-09-17T18:35:57.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8021" PostTypeId="2" ParentId="3262" CreationDate="2018-09-17T18:41:57.850" Score="1" Body="&lt;p&gt;If the problem you are solving is linearly separable, one layer of 1000 neurons can do better job than 10 layers with each of 100 neurons.&#xA;If the problem is non linear and not convex, then you need deep neural nets. &lt;/p&gt;&#xA;" OwnerUserId="18195" LastActivityDate="2018-09-17T18:41:57.850" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8022" PostTypeId="2" ParentId="7927" CreationDate="2018-09-18T06:47:38.773" Score="2" Body="&lt;p&gt;I was attaining negative values out of my convolutional layers and then using relu on them resulted in the gradient of the activation being 0. Hence, my Q values were not being updated. I've since updated my activations to be ELU. Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="18076" LastActivityDate="2018-09-18T06:47:38.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8024" PostTypeId="1" AcceptedAnswerId="8071" CreationDate="2018-09-18T09:32:20.113" Score="1" ViewCount="159" Body="&lt;p&gt;I have a mixed image database(unstructured data). In the database there are some images that i am interested in and I want to discard the rest by using cnn. I am not looking for specific objects in the images like dogs, cats etc. In the database I have photos and non photo images like infamous logos, scanned documents etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find photos and discard the others. All the examples and online courses I found are based on object recognition. In my case which method can I use to classify my images as just 'relevant' and 'irrelevant'?&lt;/p&gt;&#xA;" OwnerUserId="18283" LastEditorUserId="1581" LastEditDate="2018-09-19T00:37:22.357" LastActivityDate="2018-09-21T01:58:47.920" Title="How can I classify relevant and irrelevant images from the Database?" Tags="&lt;convolutional-neural-networks&gt;&lt;ai-basics&gt;&lt;getting-started&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8025" PostTypeId="2" ParentId="7328" CreationDate="2018-09-18T13:31:21.537" Score="2" Body="&lt;p&gt;One can also approach the question from the information theory aspect:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two trade/offs to choose from:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analog information that may represent information in a more precise/specific way, but limited in quantity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Digital information that doesn't fully represent the real world, but may contain unlimited amount of information within a few bits. A good example could be something like a incrementing for loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;i = 0&#xA;while True:&#xA;   print(i)&#xA;   i += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which one is more powerful then?&lt;/p&gt;&#xA;" OwnerUserId="9288" LastEditorUserId="9288" LastEditDate="2018-09-18T16:24:06.520" LastActivityDate="2018-09-18T16:24:06.520" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8026" PostTypeId="1" CreationDate="2018-09-18T15:22:32.103" Score="5" ViewCount="103" Body="&lt;p&gt;AI algorithms involving neural networks can use tensor specific hardware. Are there any other artificial intelligence algorithms that could benefit from many tensor calculations in parallel?  Are there any other computer science algorithms (not part of AI) that could benefit from many tensor calculations in parallel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Tensor#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorApplications&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Application_of_tensor_theory_in_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Application Theory&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18344" LastEditorUserId="2444" LastEditDate="2019-05-05T23:52:19.933" LastActivityDate="2019-05-05T23:52:19.933" Title="Which artificial intelligence algorithms could use tensor specific hardware?" Tags="&lt;algorithm&gt;&lt;applications&gt;&lt;hardware&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8027" PostTypeId="1" CreationDate="2018-09-18T20:10:36.710" Score="0" ViewCount="181" Body="&lt;p&gt;Is there research work that uses neural network as the (BDI) agent (or even full-scale cognitive architecture like Soar, OpenCog) - that continuously receives information from the environment and act in an environment and modifies its base of belief in parallel? Usually NN are trained to do only one task and TensorFlow/PyTorch supports batch mode only out of the box. Also NN algorithms and theory are constructed assuming that training and inference phases are clearly separated and they have each own algorithms. So - completely new theory and software can be required for this - are there efforts in this direction? If no, then why not? It is so self-evident that such systems can be of benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1802.07569&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1802.07569&lt;/a&gt; is good review about incremental learning and it contains chapters of implemented systems, but all of them still separates learning phase from inference phase. Symbolic systems and symbolic agents (like JSON AgentSpeak) can have updating belief/knowledge base and they can also act during  receiving new information or during forming new beliefs. I am specifically seeking research about NNs which &lt;strong&gt;do learning and inference in parallel&lt;/strong&gt;. As far as I sought then this separation still persists in self-organizing incremental NNs that are gaining some popularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can image the construction of chained NNs in Tensorflow - there is some controller network that receives input (possibly preprocessed by hierarchically lower networks) and that decides what to the: s.c. mental actions are the ouput of this controller, these actions determine whether some subordinated network is required to undergo additional learning or whether it can be temporary used for the processing of some information. Central network itself, of course, can decide to move into temporary learning phase from time to time to improve its reasoning capabilities. Such pipeline of master-slave networks is indeed possible in TensorFlow but still TensorFlow will have one central clock, not distributed, loosely connected processing. But I don't know whether existence of central clock is any restriction on the generality of capabilities of such system. Well, this hierarchy of networks maybe can be realized inside the one large network as well - maybe this large network can allow separate parts (subsets of neurons) to function in somehow independent and mutually controlling mode, maybe such regions of large neural network can emerge indeed. &lt;em&gt;I am interested in this kind of research - maybe there are available some good papers for this?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2018-09-18T21:33:16.977" LastActivityDate="2018-09-19T11:06:08.753" Title="Neural network as (BDI) agent - running in continuous mode (that do inference in parallel with learning)?" Tags="&lt;neural-networks&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8028" PostTypeId="2" ParentId="7298" CreationDate="2018-09-18T22:04:47.130" Score="5" Body="&lt;p&gt;A &lt;a href=&quot;https://stats.stackexchange.com/questions/344667/how-is-deep-learning-integrated-into-the-reinforcement-learning/344693#344693&quot;&gt;closely related question&lt;/a&gt; and a &lt;a href=&quot;https://github.com/bruceweir/gameaiexperiments/blob/master/reinforcementLearning/reinforcementLearningNeuralNetworkDodge.py&quot; rel=&quot;noreferrer&quot;&gt;minimal implementation written in Python&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That program implements the reinforcement learning technique 'Q-Learning'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is for the program to take in an observation of the environment (which could be a screenshot if learning a computer game, or sensor data for a robot) and output a decision in the form of a vector of values. Each cell in that output vector corresponds to a possible action (left, right, shoot, etc) and the highest valued cell shows the action that the agent/player should take. The values in that output vector are called the Q-Values, and it is the mapping from the input data to the vector of Q-Values that we are trying to learn.&#xA;In your case, the function that takes in an observation of the environment and spits out a vector of decision choices is your SVM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question linked above contains a description of the training algorithm. It boils down to playing the game multiple times while storing the input vector, output vector and output decision for each step until you reach a termination condition (ie hitting an asteroid) and receiving a score (in this case a negative score since you want to &lt;em&gt;avoid&lt;/em&gt; the asteroids). Then, going backwards through the the output vectors, assigning a slowly reducing amount of the output score to the particular cell in each output vector corresponding to the decision that was made at that step of the game.&#xA;The array of input vectors from the game, and the array of output decision vectors that you have just created become the training data for your machine learning system. Once you have completed a training run, play the game again with your trained system and keep repeating until it is a good as you need it to be, or it stops getting any better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address your question directly, I don't think that you are going to be able to train your system &lt;em&gt;without&lt;/em&gt; giving it information about the position of the asteroids (assuming that they are positioned at random). However, you could try just having a very simple input vector such as 3 integer values where each value corresponds to the presence (1) or absence (0) of an asteroid in the 3 squares to the above left, above and above right of the player. That might be enough to encourage it to dodge away from falling rocks... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some Support Vector Machines implemented in JavaScript, although I have never used them, but since SVMs are basically two-class classifiers you would have to check that the library can easily support multi-class SVMs. In your case the SVM would be trying to classify each input into a particular player choice (left, right, shoot, etc).&lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2018-09-18T22:04:47.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8030" PostTypeId="1" CreationDate="2018-09-19T05:25:17.373" Score="1" ViewCount="190" Body="&lt;p&gt;Can neural network take decision about its own weights (update of weights) during training phase or during the phase of parallel training and inference? When one region of hierarchical NN takes decision about weights of other region is the special case of my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very keen to understand about self-awareness, self-learning, self-improvement capabilities of neural networks, because those exactly those self-* capabilities are the key path to the artificial general intelligence (e.g. Goedel machine). Neural networks are usually mentioned as examples of special, single-purpose intelligence but I can not see the reason for such limitation if NN essentially trys to mimic human brains, at least in purpose if not in mechanics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well - maybe this desired effect is already effectively achieved/emerges in the operation of recurrent ANNs as the effect of collective behavior?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2018-09-19T05:34:34.000" LastActivityDate="2018-10-15T23:28:55.473" Title="Can neural network take decision about its own weights (update of weights)?" Tags="&lt;neural-networks&gt;&lt;artificial-consciousness&gt;&lt;superintelligence&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8031" PostTypeId="1" CreationDate="2018-09-19T05:46:55.393" Score="2" ViewCount="4305" Body="&lt;p&gt;Usually, using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Taxicab_geometry&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manhattan distance&lt;/a&gt; as a heuristic function is enough when we do an A* search with one target. However, it seems like for multiple goals, this is not the most useful way. Which heuristic do we have to use when we have multiple targets?&lt;/p&gt;&#xA;" OwnerUserId="18359" LastEditorUserId="2444" LastEditDate="2020-11-19T14:43:35.033" LastActivityDate="2020-11-19T14:43:35.033" Title="What heuristic to use when doing A* search with multiple targets?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;a-star&gt;" AnswerCount="3" CommentCount="3" ClosedDate="2020-11-19T14:42:27.800" ContentLicense="CC BY-SA 4.0" />
  <row Id="8032" PostTypeId="2" ParentId="8030" CreationDate="2018-09-19T06:30:53.017" Score="1" Body="&lt;p&gt;&lt;strong&gt;Establishing Names and Terms&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The mathematician's name is Kurt Gödel, an Austrian-American, and the machine concept named after him is the Gödel machine.  The idea of meta-learning (learning how to learn) can be extended recursively to learning how to learn how to learn how to learn and so on.&lt;/p&gt;&#xA;&lt;p&gt;Taking a decision is when the decision is external to the system and the system either copes or executes that decision, whereas &lt;strong&gt;making&lt;/strong&gt; a decision is when the decision is internal to the system and the system executes it, reports it, or delegates it.&lt;/p&gt;&#xA;&lt;p&gt;Both of these have been tried in the laboratory or in real systems and various levels of recursion have achieved success, but not indefinite recursion.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A Few Points from AI History&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Creating a learn how to learn recursion was the object of several early LISP efforts at MIT, and meta-learning is in active research in multiple universities and corporations as of this writing.  The first successful application of a single level of recursion were the expert systems that could acquire knowledge using meta-rules, which were rules about creating new rules.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Current Artificial Networks Regarding Meta-learning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is correct that back propagation is not meta-learning.  It is a single level of learning in the above recursive learning concept.  Artificial networks have been applied to learn to adjust the hyper-parameters of another network, but these strategies require large quantities of data, and they don't cognate.&lt;/p&gt;&#xA;&lt;p&gt;They don't &lt;strong&gt;know&lt;/strong&gt; how to adjust hyper-parameters.  They don't build models of learning and then use those models to learn.  The adjustment mechanism is the application of a learned function, not a conception of learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Requirements for Meta-learning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To establish a meta-learning paradigm, we must consider the elements of learning.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Existence of function abstraction&lt;/li&gt;&#xA;&lt;li&gt;Ability to guess a parameterized function&lt;/li&gt;&#xA;&lt;li&gt;Ability to mutate that function&lt;/li&gt;&#xA;&lt;li&gt;Ability to tune the parameters&lt;/li&gt;&#xA;&lt;li&gt;A target functional behavior&lt;/li&gt;&#xA;&lt;li&gt;A way to detect whether the function is approaching the target behavior&lt;/li&gt;&#xA;&lt;li&gt;A strategy for deciding when to mutate and when to tune&lt;/li&gt;&#xA;&lt;li&gt;A way of performing experiments in isolation&lt;/li&gt;&#xA;&lt;li&gt;A way to recurse in a way that incorporates ALL of the above&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Once this has been achieved, it may be described as self-improvement and may qualify as what most theorists think of as a Gödel machine.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Specifics in the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can [an artificial] network [make] decision about its own weights (update of weights) during training phase or during the phase of parallel training and inference?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[Can] one [level] of [a] hierarchical [artificial network] [make] decision[s] about weights of other [levels]?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, but in a limited way as of the time of this writing.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I am very keen to understand about self-awareness ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Self-awareness is the ability of the system to analyze and either utilize the result of that analysis, report it, or delegate based on the result IN COMBINATION WITH the ability to use itself as the object of analysis.&lt;/p&gt;&#xA;&lt;p&gt;This can be as simple as a program that parses Java and produces statistics using its own code as the code to parse.  It can be as complex as an anthropologist studying humanity.  It can be as personal as a human looking in the mirror and wondering about the qualities of person they are.  It can be as deep as someone wondering their purpose in their current life.&lt;/p&gt;&#xA;&lt;p&gt;In all cases, there are those two qualities.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The ability to perform a type of analysis that could be applied toward a class of objects of which itself is a member&lt;/li&gt;&#xA;&lt;li&gt;The selection of itself for analysis&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;There is one other aspect of awareness beyond these two that are not required but usually associated with awareness: Some way of directing attention of the analysis capability to itself regularly.  Here are a few examples from human experience.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Practice of daily meditation&lt;/li&gt;&#xA;&lt;li&gt;Reading a book on purpose&lt;/li&gt;&#xA;&lt;li&gt;Seeing a cognitive behavioral therapist&lt;/li&gt;&#xA;&lt;li&gt;Writing an autobiography&lt;/li&gt;&#xA;&lt;li&gt;The first, fourth, and tenth Step of anonymous programs&lt;/li&gt;&#xA;&lt;li&gt;Watching a candid movie of one's self&lt;/li&gt;&#xA;&lt;li&gt;A family member that likes to step back and evaluate the family&lt;/li&gt;&#xA;&lt;li&gt;Religious accountability groups&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Self learning is like self-awareness except that learning replaces awareness.  Obviously, self-learning is dependent on some degree of self-awareness.  We now have a more activity based list because of the additional element of adjustment or inception of action.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decide, on the basis of personal abilities and interests, after evaluating options for living, to relocate&lt;/li&gt;&#xA;&lt;li&gt;Realizing that self cannot remember commitments to meetings to buy a technology device that notifies the user before such commitments and using the calendar app rigorously&lt;/li&gt;&#xA;&lt;li&gt;Finding that repeated attempts to eat greens rather than junk food is not working and joining a community of those with eating disorders with solutions to that issue&lt;/li&gt;&#xA;&lt;li&gt;Deciding that childhood views of future were good ones and abandoning current paths for ones that were bolder and will likely lead to fulfillment&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Self-improvement is a super-set of self-learning, since the only point of learning is improvement, however there are forms of improvement that don't strictly involve learning.  A system (or person) can execute something already learned that results in an improvement.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Artificial general intelligence (e.g. Gödel machine)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is no mathematical proof that a Gödel machine would exhibit general artificial intelligence.  More importantly, there is no proof and a considerably body of evidence against the proposition that humans are generally intelligent.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, there is no proof that general intelligence is achievable in a biological or artificial system.  Gödel's second incompleteness theorem is strong evidence that such ideas may be naive.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[Artificial] networks are usually mentioned as examples of special, single-purpose intelligence, but I can not see the reason for such limitation if [they] essentially [try] to mimic human brains, at least in purpose if not in mechanics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Artificial networks emerged out of a desire to mimic human brains, but the perceptron design is based on an old view of neuron functionality.  Simulating a single neuron might require an entire rack of CPUs and GPUs.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, we are not yet clear on what intelligence is.  Definitions proposed vary widely.  The fields of artificial networks, bioinformatics, ontology of ideas, semantics, and the relationships between stable adaptivity and algorithms are all in their early stages, both theoretically and experimentally.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Maybe this desired [capability] is already effectively achieved [or currently emerging] in the operation of recurrent ANNs as the effect of collective behavior?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, but RNNs are a tiny step closer to biological neurons in that their layers maintain state beyond the attenuation matrix.&lt;/p&gt;&#xA;&lt;p&gt;By attenuation matrix is meant the matrix of parameters used in vector-matrix multiplication to control the strength of signals from the activation functions of one layer to the activation functions of the next.  It's common name in machine learning literature is simply, &amp;quot;The parameters.&amp;quot;  Learning occurs as they change to converge the network on an optimal state.&lt;/p&gt;&#xA;&lt;p&gt;RNNs are also capable of being Turing complete, so they can theoretically realize arbitrary algorithms.&lt;/p&gt;&#xA;&lt;p&gt;However, the &lt;strong&gt;Requirements for Meta-learning&lt;/strong&gt; above are not an expected capability of any single instance of an artificial networks, either of the MLP (multilayer perceptron) type or the RNN type.  Whether balancing two networks in symbiotic arrangement as in GANs, or a comprehensive recursive algorithm, or the simulation of neural and synaptic plasticity in silicon will lead to Gödel machines is unknown.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Let Science be Science&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Whether ideas such as general intelligence are possible is unknown.  Whether ideas about singularities in books and mass media are realistic is unknown.  Whether the prophetic warnings of screenwriters about the emergence of artificial entities that become the dominant species on earth is unknown.&lt;/p&gt;&#xA;&lt;p&gt;Jaques Ellul is perhaps the most scientifically accurate prophetic voice.  In his &lt;em&gt;Techological Sociey&lt;/em&gt;, he presents heaps evidence in support of the idea that humans are already serving an autonomous technology and have been since prior to industrialization.&lt;/p&gt;&#xA;&lt;p&gt;A Swiss philosopher, Francis Schaeffer, once prophecied, &amp;quot;They will say all kinds of things in the name of science that have nothing to do with science.&amp;quot;  We should be careful to keep conjecture from reaching the status of scientific fact in technical conversation.  If we have no carefully drawn theory or empirical evidence to support a conjecture, it should be stated as a proposal, not a conclusion.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-15T23:28:55.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8034" PostTypeId="1" CreationDate="2018-09-19T07:21:47.930" Score="2" ViewCount="51" Body="&lt;p&gt;In physics, there are a lot of graphs, such as 'velocity vs time' , 'time period vs length' and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a sample set of points for a 'velocity vs time' graph. I draw it by hand, rather haphazardly, on a canvas. This drawn graph on the canvas is then provided to the computer. By computer I mean AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want it to sort of beautify my drawn graph, such as straightening the lines, making the curves better, adding the digits on axes and so on. In other words, I want it to give me a better version of my drawn graph which I can readily use in, say, a word document for a report.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Is it possible/plausible to do this?&#xA;b) Are there any APIs available that can already do this? (Don't want to reinvent the wheel)&#xA;c) Any recommendations/suggestions to make the idea possible by altering it somehow?&lt;/p&gt;&#xA;" OwnerUserId="18361" LastEditorUserId="1581" LastEditDate="2018-09-19T18:53:19.690" LastActivityDate="2018-09-19T18:53:19.690" Title="How can I use A.I/Image Processing to construct mathematical graphs from drawing?" Tags="&lt;image-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8037" PostTypeId="2" ParentId="8027" CreationDate="2018-09-19T11:06:08.753" Score="1" Body="&lt;p&gt;You might be interested in the Clarion cognitive architecture, developed by Prof. Ron Sun and collaborators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Full disclosure:&lt;/strong&gt; I am a student in Ron Sun's Cognitive Architecture Lab.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Brief Description of Clarion&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Clarion agents are composed of several subsystems, each of which may contain several neural networks. These subsystems include the Action-Centered Subsystem (ACS), the Non-Action-Centered Subsytem. The ACS controls action decision making, the NACS stores general knowledge. During an activation cycle, the ACS might request information from the NACS, setting the NACS activation cycle in motion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strictly speaking, Clarion does not adopt the BDI framework. But, various components of a Clarion agent can be put in correspondence with BDI concepts. BDI beliefs correspond roughly to knowledge in the ACS and NACS. The Motivational Subsystem (MS) contains agent drives and sets agent goals, which roughly correspond to BDI desires and intentions respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clarion agents do not directly control when learning happens, but may control which subsystems are active based on task demands, and may output mental actions as part of their processing. Learning generally happens at the end of a subsystem activation cycle based on feedback from the environment or from the agent's own motivational and metacognitive subsystems.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Links&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://sites.google.com/site/drronsun/clarion/clarion-publications&quot; rel=&quot;nofollow noreferrer&quot;&gt;A page&lt;/a&gt; on Prof. Ron Sun's site links to several resources on Clarion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the most up-to-date reference on Clarion is the book &lt;a href=&quot;https://global.oup.com/academic/product/anatomy-of-the-mind-9780199794553?cc=us&amp;amp;lang=en&amp;amp;&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anatomy of the Mind&lt;/a&gt;. &lt;a href=&quot;https://link.springer.com/article/10.1007/s12559-016-9444-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;A precis&lt;/a&gt; is available in the journal &lt;em&gt;Cognitive Computation&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18371" LastActivityDate="2018-09-19T11:06:08.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8038" PostTypeId="1" CreationDate="2018-09-19T11:22:14.860" Score="1" ViewCount="67" Body="&lt;p&gt;I'm trying to use a CNN to analyse statistical images. These images are not 'natural' images (cats, dogs, etc) but images generated by visualising a dataset. The idea is that these datasets hopefully contain patterns in them that can be used as part of a classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YGYUZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YGYUZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most CNN examples I've seen have one of more pooling layers, and the explaination I've seen for them is to reduce the number of training elements, but also to allow for some locational independance of an element (e.g. I know this is an eye, and can appear anywhere in the image).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case location &lt;em&gt;is&lt;/em&gt; important and I want my CNN to be aware of that. ie. the presence of a pattern at a specific location in the image means something very specific compared to if that feature or pattern appears elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment my network looks like this (taken from an example somewhere):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;conv2d_1 (Conv2D)            (None, 196, 178, 32)      896       &#xA;_________________________________________________________________&#xA;activation_1 (Activation)    (None, 196, 178, 32)      0         &#xA;_________________________________________________________________&#xA;max_pooling2d_1 (MaxPooling2 (None, 98, 89, 32)        0         &#xA;_________________________________________________________________&#xA;conv2d_2 (Conv2D)            (None, 96, 87, 32)        9248      &#xA;_________________________________________________________________&#xA;activation_2 (Activation)    (None, 96, 87, 32)        0         &#xA;_________________________________________________________________&#xA;max_pooling2d_2 (MaxPooling2 (None, 48, 43, 32)        0         &#xA;_________________________________________________________________&#xA;conv2d_3 (Conv2D)            (None, 46, 41, 64)        18496     &#xA;_________________________________________________________________&#xA;activation_3 (Activation)    (None, 46, 41, 64)        0         &#xA;_________________________________________________________________&#xA;max_pooling2d_3 (MaxPooling2 (None, 23, 20, 64)        0         &#xA;_________________________________________________________________&#xA;flatten_1 (Flatten)          (None, 29440)             0         &#xA;_________________________________________________________________&#xA;dense_1 (Dense)              (None, 32)                942112    &#xA;_________________________________________________________________&#xA;activation_4 (Activation)    (None, 32)                0         &#xA;_________________________________________________________________&#xA;dropout_1 (Dropout)          (None, 32)                0         &#xA;_________________________________________________________________&#xA;dense_2 (Dense)              (None, 3)                 99        &#xA;_________________________________________________________________&#xA;activation_5 (Activation)    (None, 3)                 0         &#xA;=================================================================&#xA;Total params: 970,851&#xA;Trainable params: 970,851&#xA;Non-trainable params: 0&#xA;_________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The 'images' I'm training on are 180 x 180 x 3 pixels and each channel contains a different set of raw data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What strategies are there to improve my CNN to deal with this? I have tried simply removing some of the pooling layers, but that greatly increased memory and training time and didn't seem to really help.&lt;/p&gt;&#xA;" OwnerUserId="18372" LastActivityDate="2018-11-19T00:01:50.360" Title="CNN Pooling layers unhelpful when location important?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8039" PostTypeId="1" AcceptedAnswerId="8047" CreationDate="2018-09-19T12:35:57.423" Score="2" ViewCount="214" Body="&lt;p&gt;I'm finding it hard to understand the relationship between chaotic behavior, the human brain, and artificial networks.  There are a number of explanations on the web, but it would be very helpful if I get a very simple explanation or any references providing such simplifications.&lt;/p&gt;&#xA;" OwnerUserId="18375" LastEditorUserId="4302" LastEditDate="2018-09-20T19:04:34.220" LastActivityDate="2018-12-08T23:30:47.427" Title="What is chaotic behavior and how it is achieved in non-linear regression and artificial networks?" Tags="&lt;deep-learning&gt;&lt;philosophy&gt;&lt;human-inspired&gt;&lt;convergence&gt;&lt;chaos&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8040" PostTypeId="2" ParentId="8038" CreationDate="2018-09-19T13:14:11.860" Score="-1" Body="&lt;p&gt;Pooling doesn't completely remove information about location of features within your image. If you don't want to use pooling but want to reduce the size of your neural net layers you should try a value of stride greater than 1 in your convolutional layers. e.g:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;keras.layers.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="18138" LastActivityDate="2018-09-19T13:14:11.860" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8042" PostTypeId="1" CreationDate="2018-09-19T14:18:14.023" Score="1" ViewCount="22" Body="&lt;p&gt;I implemented and deployed with Flask an XGBoost model for a classification problem. But being aware that features importance can change over time to predict probability of label for new data, I implemented a Cron so that the model can be retrained every two weeks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't know how I can handle new features since I would have to wait a great volume of data to retrain the model to take into account this new feature ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an alternative of model deployment to this problem ?&lt;/p&gt;&#xA;" OwnerUserId="18192" LastActivityDate="2018-09-19T14:18:14.023" Title="How to handle Feature changes in a model deployed ?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8044" PostTypeId="2" ParentId="8039" CreationDate="2018-09-19T16:40:10.173" Score="2" Body="&lt;p&gt;It looks like you have some common misconceptions about AI and neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, AI programs generally do not try to imitate the human behaviour of a human brain. Instead, they try to imitate some higher-level behaviour. For example, they might imitate the reasoning process that you go through when you make a plan. In this context, the building-blocks (silicon or flesh) don't matter too much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, artificial neural networks are also (mostly) not intended to imitate the human brain. Although they are &lt;em&gt;inspired&lt;/em&gt; by the arrangement of neurons in a human brain, the networks used in most ANN systems have very little to do with real brains. The main similarity is that both systems have a lot of simple little computational units connected in patterns such that signals passed from one to another lead to interesting computations. However, real neurons produce lots of different kinds of signals, are connected in arbitrary ways, and randomization and transmission times play a significant role. Artificial neurons generally are deterministic, produce only one kind of signal (or sometimes a couple different kinds), are connected in extremely regular ways, and usually simulate instantaneous transmissions between neurons.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-19T16:40:10.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8045" PostTypeId="2" ParentId="8031" CreationDate="2018-09-19T16:52:46.447" Score="2" Body="&lt;p&gt;If by &quot;visit multiple targets&quot;, you mean &quot;visit several points in the fastest order&quot;, you are no longer in a simple path-finding-style search problem, but instead in an optimization problem. This is roughly the difference between chapters 3 &amp;amp; 6 of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig's&lt;/a&gt; section on search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, you can't just change your heuristic, instead you need to reframe your problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of states in your search being locations, they should be &lt;em&gt;tours&lt;/em&gt;. Each state is a list of all the places you need to visit, &lt;em&gt;in a specific order&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of actions being movements from one location to another, they need to be transformations from one tour to another. For example, if you swap the order that you visit two adjacent locations, then you'll get a different tour. This gives you a way to &quot;move&quot; between tours.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A solution means visiting all the locations as fast as possible. If you know how to get between two locations, just store the distances, and then sum all the distances together to get the cost of a tour. If you don't know, you can just run A* from each place to each other place once, and then cache the distances afterwards. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A heuristic will depend on your domain. A reasonable start might be to assume that you can visit each location from the nearest other location you've already visited. Generally, heurstics based on the idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_spanning_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimum spanning trees&lt;/a&gt; are effective for this domain. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The real answer though, is to try a technique that is meant for this kind of problem, like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_search_(optimization)&quot; rel=&quot;nofollow noreferrer&quot;&gt;local search algorithm&lt;/a&gt;. Notice that if we know the cost of moving between any two points, we can just adopt a greedy approach: make the move that improves things the most each time. This is often faster than waiting for A* in practice if you just want a good solution, but it doesn't need to be the very best one.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-19T16:52:46.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8046" PostTypeId="2" ParentId="7983" CreationDate="2018-09-19T17:43:40.070" Score="0" Body="&lt;p&gt;The disabled/enabled bit in a connection gene indicates whether or not it should be expressed in the calculation of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CliXP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CliXP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a neural network and its corresponding connection genes which represent the layout of the network among other things.&#xA;The top connection gene going from 1 -&gt; 3 that has a weight of 0.1 is expressed in the calculation of the network.&#xA;The bottom connection gene going from 2 -&gt; 3 that has a weight of 0.4 is not expressed in the calculation of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating the network:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1633 * 0.1 = 163.3&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Given the weights in the example the output of this network is 163.3&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hypothetically, if both connection genes had their enabled bit set to true (which can happen in the future, to quote the paper: &quot;The disabled genes may become enabled again in future generations&quot;) then the output of the network would be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(1633 * 0.1) + (10 * 0.4) = 167.3&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To answer your question, the connection between two nodes still exists in the network regardless of whether the bit in the gene is enabled or disabled, but, if the bit in the gene is disabled it is not used when the output of the network is being calculated as I've shown above.&lt;/p&gt;&#xA;" OwnerUserId="15356" LastEditorUserId="15356" LastEditDate="2018-09-19T18:03:50.660" LastActivityDate="2018-09-19T18:03:50.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8047" PostTypeId="2" ParentId="8039" CreationDate="2018-09-19T18:29:50.827" Score="0" Body="&lt;p&gt;Regression for models more complex than &lt;span class=&quot;math-container&quot;&gt;$y = a x + b$&lt;/span&gt; is a convergence strategy. Surface fitting algorithms, such as Levenberg–Marquardt, are often successful at achieving regression using a damped version of least squares as an optimization criterion. The marriage of regression and the multilayer perceptron, an early model artificial network, led to the use of a back propagation strategy to distribute corrective signals that drive regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back propagation using gradient descent is now used in artificial networks with a variety of cell and connection designs, such as LSTM and CNN networks as a convergence strategy. Both surface fitting and artificial network convergence share the method of successive approximation. With each successive application of some test, the result is used to attempt to improve the next iteration. Proofs have developed around convergence for many algorithms. Actual successive approximation runs have five possible outcomes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Convergence within the time allotted and within the accuracy required&lt;/li&gt;&#xA;&lt;li&gt;Convergence within the time allotted but not within the accuracy required&lt;/li&gt;&#xA;&lt;li&gt;Convergence appears that it would have occurred but time allotted was exceeded&lt;/li&gt;&#xA;&lt;li&gt;Oscillation appeared by the end of time allotted&lt;/li&gt;&#xA;&lt;li&gt;Chaos appeared by the end of time allotted&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The following illustration from Chaos Theory Tamed (Garnett P. Williams, 1997, p 164) modified slightly for easy viewing can explain how chaos arises when the learning rate or some other factor is set too aggressively.  The graphs are of the behavior of the logistic equation &lt;span class=&quot;math-container&quot;&gt;$x_{i+1} = k x_i (1 - x)$&lt;/span&gt; which plots as an inverted parabola in phase space.  The one dimensional maps on the right of each of five cases show the relationship between adjacent values in the time series on the left of each of the five.  Although the logistic equation is quite simple compared to regression algorithms and artificial nets, the principles involved are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4BbZN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4BbZN.png&quot; alt=&quot;Logistic equation convergence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The right hand cases, with &lt;span class=&quot;math-container&quot;&gt;$k = 3.4$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k = 3.75$&lt;/span&gt; correspond to the last two possible outcomes in the list above, oscillation and chaos respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Care in Drawing Parallels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Care should be taken in drawing parallels between distinct things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Surface fitting algorithms, like Levenberg–Marquardt&lt;/li&gt;&#xA;&lt;li&gt;Algorithms that realize back propagation with gradient descent&lt;/li&gt;&#xA;&lt;li&gt;Logical inference AI, such as production systems and fuzzy logic&lt;/li&gt;&#xA;&lt;li&gt;Real time learning, such as Q-learning&lt;/li&gt;&#xA;&lt;li&gt;Devotion of the human brain to a problem&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regression and artificial networks can be compared meaningfully because the math for each is fully defined and easy for those with the mathematical skill to analyze them for the comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing known mathematical systems with unknown biological ones is interesting, but to a large degree, grossly premature.  The perceptron, on which MLPs (multilayer perceptrons) and their deep learning derivatives are based, are simplified and flattened models of what was once thought to be how neurons in the brain work.  By flattened is meant that they are placed in the time domain where they are convenient for looping in software and do not take into consideration these complexities.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neuron behavior is sensitive to the timing of incoming signals &amp;mdash; Incoming signals may overlap but not precisely align in time.&lt;/li&gt;&#xA;&lt;li&gt;Neuron behavior is sensitive to the history of incoming signals (because of cell body and axon thermodynamics, synaptic chemistry, and other neuro-chemical and structural functions not yet understood)&lt;/li&gt;&#xA;&lt;li&gt;Neuron structure changes in terms of its connectivity&lt;/li&gt;&#xA;&lt;li&gt;New neurons appear&lt;/li&gt;&#xA;&lt;li&gt;Neurons may die due to cell apoptosis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In summary, multilayer perceptrons are not a model of neural networks in the human brain.  They are merely roughly inspired by obsolete knowledge of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Chaos in the Human Brain&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether there is chaotic behavior in the brain is known.  It has been observed in real time.  How coupled it is with human intelligence is a matter of conjecture, but it is already clear that it may appear to contribute to function in some cases and contribute to dysfunction in others.  This is also true in artificial systems.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When used to deliberately interfere with a stable condition that may not be the optimum state of stability to find a better one, chaos may be a source of noise that benefits learning.  This is related to the difference between local minima and global minimum.  The good is sometimes the enemy of the best.  Improved learning speed has been documented for artificial network algorithms with deliberate injection of pseudo-random noise into the back propagation.&lt;/li&gt;&#xA;&lt;li&gt;When appearing not to deliberately inject noise in a portion of the system that can benefit from the noise but out of a basic instability in the system, the chaos can be detrimental to overall system function.  Chaotic behavior in the human brain is a likely cause for various disorders.  There is much supporting data but not yet a proof.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In summary, chaos in a system is neither productive nor counterproductive in every case.  It depends on where it is in the system (in detail) and what the system design is expected to perform.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-08T23:30:47.427" LastActivityDate="2018-12-08T23:30:47.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8048" PostTypeId="1" AcceptedAnswerId="8062" CreationDate="2018-09-19T19:01:45.640" Score="3" ViewCount="47" Body="&lt;p&gt;I want to implement Sparse Extended information slam. There is four step to implement it. The algorithm is available in &lt;a href=&quot;https://docs.ufpr.br/~danielsantos/ProbabilisticRobotics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probabilistic Robotics Book&lt;/a&gt; at page 310, Table 12.3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1OA52.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1OA52.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this algorithm line no:13 is not very clear to me. I have 15 landmarks. So $\mu_t$ will be a vector of (48*1) dimension where (3*1) for pose. Now $H_t^i$ is a matrix whose columns are dynamic as per the algorithm it is (3j-3) and 3j. J is the values of landmarks 1 to 15. Now how could I multiply a dynamic quantity with a  static one. There must be a error that matrix dimension mismatch when implement in matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me to understand the algorithm better. &lt;/p&gt;&#xA;" OwnerUserId="18384" LastActivityDate="2018-09-20T12:19:25.623" Title="SEIF motion update algorithm doubt" Tags="&lt;research&gt;&lt;intelligent-agent&gt;&lt;autonomous-vehicles&gt;&lt;probability&gt;&lt;robotics&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8050" PostTypeId="2" ParentId="6094" CreationDate="2018-09-19T22:02:35.020" Score="0" Body="&lt;p&gt;A python library called panda data frame will be useful to frame the above data.&#xA;import pandas as pd&lt;/p&gt;&#xA;&#xA;&lt;p&gt;column_names = ['']&lt;/p&gt;&#xA;&#xA;&lt;p&gt;df = pd.DataFrame(train_data, columns=column_names)&#xA;df.head()&lt;/p&gt;&#xA;" OwnerUserId="18195" LastActivityDate="2018-09-19T22:02:35.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8051" PostTypeId="2" ParentId="8031" CreationDate="2018-09-19T22:16:15.720" Score="0" Body="&lt;p&gt;Genetic algorithms are giving promising results for problems with multiple objectives[goals].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.iitk.ac.in/kangal/Deb_NSGA-II.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.iitk.ac.in/kangal/Deb_NSGA-II.pdf&lt;/a&gt;&#xA;the above paper will give best algorithm for multi objectives&lt;/p&gt;&#xA;" OwnerUserId="18195" LastActivityDate="2018-09-19T22:16:15.720" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8052" PostTypeId="2" ParentId="8038" CreationDate="2018-09-19T22:46:56.383" Score="-1" Body="&lt;p&gt;1X1 convolution might improve accuracy as they reduce dimension in filter space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google inception V3 architecture would be a good starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1512.00567.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1512.00567.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18195" LastActivityDate="2018-09-19T22:46:56.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8053" PostTypeId="2" ParentId="5898" CreationDate="2018-09-19T22:49:37.047" Score="5" Body="&lt;p&gt;The diagram is nice-ish-looking, and it does lack a convenient list of node type descriptions.  There is some description in their &lt;a href=&quot;https://quizlet.com/152146730/neural-network-zoo-flash-cards/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zoo Flashcards page&lt;/a&gt;. Although the zoo is not a clean presentation in every respect, it is quite instructive in several ways. These are of particular interest. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The symbolic shapes and colors visually display the relationship between cell types and the types of networks that employ them, which is quite instructive as an overview.&lt;/li&gt;&#xA;&lt;li&gt;The use of the term &lt;em&gt;cell&lt;/em&gt;, as opposed to &lt;em&gt;neuron&lt;/em&gt;, acknowledges the fundamental difference between current artificial network cells and the amazingly intricate and self-modifying nature of biological neurons.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What is still missing is a concise list of each cell type and its purpose for existing in the contexts of its use.  This may be useful.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input Cell &amp;mdash; Provides a point of input and often uses an identity activation function (the output = the input)&lt;/li&gt;&#xA;&lt;li&gt;Back-fed Input Cell &amp;mdash; Provides a point where feedback is deliberately introduced&lt;/li&gt;&#xA;&lt;li&gt;Noisy Input Cell &amp;mdash; Provides a point where chaotic (pseudo-random), thermal, or quantum noise can be introduced&lt;/li&gt;&#xA;&lt;li&gt;Hidden Cell  &amp;mdash; A cell that attenuates (applies a parameterized weight to) the input from all or a select range of activation function outputs from the cells of the immediately prior layer&lt;/li&gt;&#xA;&lt;li&gt;Probabilistic Hidden Cell &amp;mdash; Applies a radial basis function to the difference between the test case and the cell's mean&lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Spiking Hidden Cell &amp;mdash; Employs a spiking model of activation rather than a perceptron model, working toward a more accurate simulation of biological neurons at the cell level&lt;/li&gt;&#xA;&lt;li&gt;Output Cell &amp;mdash; Provides a point of output and applies the range (min and max) or ordinality (binary, category index)&lt;/li&gt;&#xA;&lt;li&gt;Match Input Output Cell &amp;mdash; Used in auto-encoders to converge on the identity function from network input to network output, usually through a narrowed data path through which features are extracted after convergence&lt;/li&gt;&#xA;&lt;li&gt;Recurrent Cell &amp;mdash; Used in RNN designs such as recurrent artificial networks as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning &lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Memory Cell &amp;mdash; Used in LSTM (long short term memory) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning &lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Different Memory Cell &amp;mdash; Used in GRU (gated recurrent unit) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning &lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Convolution Cell &amp;mdash; Passes the result of a convolution kernel multiplied to a relative range of the cells of the immediately previous layer to the next layer &lt;/li&gt;&#xA;&lt;li&gt;Pool Cell &amp;mdash; Aggregates (usually arithmatically) the results of the cells of a convolution layer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A table of cell types with the features for each might be a next step in presenting the cell types in a comparative summary.  The table would particularly helpful if it had sort up and down gadgets in the header row for each column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Typically &lt;span class=&quot;math-container&quot;&gt;$\varphi(\mathbf{x}) = \sum_{i=1}^I a_i e^{\left[ -\beta \left \Vert \mathbf{x} - \mathbf{c}_i  \right \Vert ^2 \right]}$&lt;/span&gt; where c is the center value of the cell&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] These cell types integrate historical influence into the signal path at the cellular level, introducing statefulness beyond the learning parameters and network hyper-parameters, providing the potential of Turing completeness to the network after training.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-10T19:11:11.530" LastActivityDate="2018-12-10T19:11:11.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8054" PostTypeId="2" ParentId="5043" CreationDate="2018-09-20T00:28:54.457" Score="1" Body="&lt;p&gt;The connections between ethics and artificial intelligence can be divided into five major categories, and other categories may form over time.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Correlations between ethics and artificial intelligence&lt;/li&gt;&#xA;&lt;li&gt;Existential impacts of artificial intelligence on the human experience&lt;/li&gt;&#xA;&lt;li&gt;Threats to current ethical social, economic, and legal standards arising from artificial intelligence research and application&lt;/li&gt;&#xA;&lt;li&gt;Uses of artificial intelligence to breach ethical standards without detection&lt;/li&gt;&#xA;&lt;li&gt;Uses of artificial intelligence to detection of breaches of ethical standards and assist in remedial action&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since the most important in the long term is the most likely to be dismissed by those with normal perspectives about ethics and AI, the four will be addressed in reverse order&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Automatic Detection and Remedial Action&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pattern recognition capabilities of existing AI systems and sub-systems is already employed to detect a variety of ethical breaches.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Securities misconduct, including insider trading&lt;/li&gt;&#xA;&lt;li&gt;Breaches of anti-trust law, including conflicts of interest&lt;/li&gt;&#xA;&lt;li&gt;Employer misconduct, including inequality in hiring&lt;/li&gt;&#xA;&lt;li&gt;Tax evasion&lt;/li&gt;&#xA;&lt;li&gt;Misuse of non-profit funds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Remedial actions may be the opening of a case with the automatic generation of a notice to those in potential breach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Smart Organized Crime&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although much detail could be included here about detection avoidance in crime using AI, it may not be socially responsible to include such in a global public facing site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Threats to Economies and Individuals&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with any high impact technology, disruption is a possibility.  This was true of fire, irrigation, the wheel, bronze smelting, gun powder, typesetting, steel-working, engines, textile automation, alternating current power distribution, aeronautics, petroleum refining, electronics and radio transmission, pharmacology, terrestrial nuclear reaction, and the Internet.  Genetic engineering and artificial intelligence are next in line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What ethical conventions will likely be impacted?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Distribution of employment roles, the change of which may not match distribution of educational preparation&lt;/li&gt;&#xA;&lt;li&gt;Distribution of wealth, favoring prowess in highly automated business&lt;/li&gt;&#xA;&lt;li&gt;Mutual exclusivity of personal privacy and the use of technology&lt;/li&gt;&#xA;&lt;li&gt;Obscurity of totalitarian control (such that common citizens may be more like cogs in a machine than during industrialization)&lt;/li&gt;&#xA;&lt;li&gt;Changes in the balance of world power&lt;/li&gt;&#xA;&lt;li&gt;New forms of asymmetric war, such as cyber-war and autonomous combatants&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these have either direct or direct impact on the viability of business options and how business may be conducted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Existential impacts&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some may consider dominion over the earth as an ethical grant to humanity.  Others may equate the soul with the cognitive and self-aware aspects of only one species on earth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We already have a world that is sufficiently disconnected on an existential plane that many consider their cats or dogs as more important than any human.  When people are more connected to their intelligent agents than their pets, family, and friends, that may qualify as an ethical impact.  Others may see it as a psychological impact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realistically, it is ontological.  What is a human to think of her or his only purpose when it becomes questionable whether homo sapiens is simply a link between DNA based intelligence and some more capable species the reproduction of which has been decoupled from DNA coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacement of jobs have caused changes in what families wish for their children.  What will be the impact when few job roles (or eventually none) exist where artificial employees don't exceed their human counterparts in effectiveness?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If humans cannot adjust to the idea that the sole purpose of life has nothing to do with practical provision of water, food, shelter, clothing, and essential products and services, there may be systemic depression.  Conversely, leisure may become the reality for all humans, leaving ethics and the finite nature of DNA based life the only two concerns of humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Correlations Between Ethics and AI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the most unpalatable of categories to examine when the examiner is human.  It is possible that artificiality may be an ethics progenitor.  The limitation of humans as ethical beings is well documented.  It is possible that AI may be more ethical than its developers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will a group of AI systems be able to arrive at method for distribution of power and a standard for global trade that is as good as or better than humans have been able to negotiate and then police each other in a way that leaves no possibility of undetected breach of treaty?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-20T00:28:54.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8055" PostTypeId="1" CreationDate="2018-09-20T01:27:25.590" Score="2" ViewCount="12" Body="&lt;p&gt;I am trying to reproduce the model described in the paper &lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2018/CameraReady/3349.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DocUNet: Document Image Unwarping via A Stacked U-Net&lt;/a&gt;, i.e. stacking two U-Nets to yield one final prediction. The paper mentions that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The deconvolution features of the first U-Net and the intermediate prediction y1 are concatenated together as the input of the second U-Net.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What does it mean by concatenating deconvolution features and the prediction (which is an array? cm)?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next paragraph says that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The second U-Net finally gives a refined prediction y2, which we use as the final output of our network. We apply the same loss function to both y1 and y2 during training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It leads to the next question: &lt;em&gt;Does it mean that I have to train U-Net twice?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="18389" LastEditorUserId="2444" LastEditDate="2020-06-13T00:04:54.407" LastActivityDate="2020-06-13T00:04:54.407" Title="How do we stack two U-Nets to yield one final prediction?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8056" PostTypeId="2" ParentId="7328" CreationDate="2018-09-20T03:29:50.553" Score="3" Body="&lt;p&gt;&lt;strong&gt;Digital Instrumentation of the Analog Cells&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the key challenges in analog artificial networks is that network instrumentation would be most practical if digital.  Any VLSI implementation of analog perceptrons, convolutions, or spiking networks will likely need to have digital components in a hybrid arrangement for several functions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Health indicators&lt;/li&gt;&#xA;&lt;li&gt;Fault indicators&lt;/li&gt;&#xA;&lt;li&gt;Archive and retrieval of learned parameters&lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Overall system control&lt;/li&gt;&#xA;&lt;li&gt;Setting hyper-parameters&lt;/li&gt;&#xA;&lt;li&gt;Operational statistics&lt;/li&gt;&#xA;&lt;li&gt;Introspection for development and debugging&lt;/li&gt;&#xA;&lt;li&gt;Break points&lt;/li&gt;&#xA;&lt;li&gt;Auditability&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This means that the realization of a general purpose analog artificial learning network will require A-to-D and D-to-A conversion.&lt;sup&gt;2&lt;/sup&gt; The VLSI design challenge then becomes avoiding the build-up of transistors from the introduction of a large number of conversion blocks.  Such would defeat the density advantage of the analog realization of forward and backward propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The likely solution is to use a latching matrix to distribute signals from the D-to-A converters to capacitors and the low leakage switching matrix to select which value will be read by the A-to-D converters.  This must be done without introducing digital noise into the analog paths and without degrading the stored charges or loss of accuracy in charging them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How significant the number of additional transistors and routes in an out of the primary network circuit would be is can only be found by exercising a VLSI design process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Important Open Source Contributions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The University of Massachusetts introduced the open source BindsNet repository&lt;sup&gt;3,4&lt;/sup&gt; in February 2018.  It simulates analog spiking networks with digital software and hardware and leverages GPU acceleration through PyTorch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This facilitates present day experimentation into spiking network designs and strategies.  Success using simulation, if significant enough, would likely lead to a superior VLSI designs.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] In any practical learning system, learned parameters must be extracted from VLSI implementation, stored in a database, and made available to any number of development, test, UAT, or production systems for deployment, flaw root cause analysis, scaling, and disaster recovery.  Saving and loading must be a basic feature of VLSI hybrid analog artificial networks, even between epochs during training and during actual field use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] One cannot hold the learned state of an artificial network in capacitors indefinitely.  Although capacitors have become the dominant passive component for analog circuits designed in standard CMOS processes, they cannot have much capacity and leakage is not zero.  The half-life of the capacitive storage circuits and the required accuracy of parameter values will determine the rate of a read and conditional re-update cycle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://github.com/Hananel-Hazan/bindsnet&quot; rel=&quot;nofollow noreferrer&quot;&gt;BindsNet open source repository&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;http://adsabs.harvard.edu/abs/2018arXiv180601423H&quot; rel=&quot;nofollow noreferrer&quot;&gt;BindsNET [paper]: A machine learning-oriented spiking neural networks library in Python&lt;/a&gt; for the Harvard U publication of the abstract from the BindsNet paper.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-24T17:41:20.133" LastActivityDate="2018-09-24T17:41:20.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8057" PostTypeId="1" CreationDate="2018-09-20T06:06:06.213" Score="2" ViewCount="92" Body="&lt;p&gt;I have created a game based on &lt;a href=&quot;https://sites.google.com/site/boardandpieces/list-of-games/fox--geese-checkerboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;this game here&lt;/a&gt;. I am attempting to use Deep Q Learning to do this, and this is my first foray into Neural networks (please be gentle!!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to create a NN that can play this game. Here are some relevant facts about the game:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Player 1 (the fox) has 1 piece that he can move diagonally 1 step in any direction &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Player 2(The geese) has 4 pieces that they can move only forward diagonally (either diagonal left or diagonal right) 1 step.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Fox wins if he reaches the other end of the board, the geese win if they trap the fox so it cannot move.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am trying to work on the agent first for the geese as it seems to be the harder agent with more pieces and restrictions. Here is the important sections of code I have so far:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is where I setup the game board, and set the total actions for the geese&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def __init__(self):&#xA;    self.state_size = (LENGTH,LENGTH) ##LENGTH is 8 so (8,8)&#xA;    #...&#xA;    #other DQN variables that aren't important to question&#xA;    #...&#xA;    self.action_size = 8 ##4 geese, each can potentially make 2 moves&#xA;    self.model = self.build_model()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;And here is where I create my model&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def build_model(self):&#xA;    #builds the NN for Deep-Q Model&#xA;    model = Sequential() #establishes a feed forward NN&#xA;    model.add(Dense(64,input_shape = (LENGTH,), activation='relu'))&#xA;    model.add(Dense(64, activation='relu'))&#xA;    model.add(Dense(self.action_size, activation = 'linear'))&#xA;    model.compile(loss='mse', optimizer='Adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is where I perform an action&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def act(self, state,env):&#xA;    #get the list of allowed actions for the geese&#xA;    actions_allowed = env.allowed_actions_geese_agent()&#xA;&#xA;    if np.random.rand(0,1) &amp;lt;= self.epsilon: ##do a random move&#xA;        return actions_allowed[random.randint(0, len(actions_allowed)-1)]&#xA;    act_values = self.model.predict(state)&#xA;    print(act_values)&#xA;    return np.argmax(act_values)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My question: Since there are 4 geese and each can make 2 possible moves, am I correct in thinking that my action_size should be &lt;strong&gt;8&lt;/strong&gt; (2 for each goose) or should it be maybe 2 (for diagonal left or right) or something else entirely? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The reason why I am at a loss is because on any given turn, some of the geese may have an invalid move, does that matter?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My next Question: Even if I have the right output layer for the geese agent, when I call &lt;code&gt;model.predict(state)&lt;/code&gt; where I pick my action...how do I interpret the output? And how would I map that action it selects to a valid action that can be made?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of the result of using &lt;code&gt;model.predict(state)&lt;/code&gt;, as you can see it returns a ton of data and then when I call &lt;code&gt;return np.argmax(act_values)&lt;/code&gt; I get 59 back...not sure how to utilize that (or if it's even correct based on my output layer)... and finally I included a drawing of the board. F is the fox and 1,2,3,4 are the different geese.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fhPCI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fhPCI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I apologize for the massive post, but I am just trying to provide as much information that is helpful. &lt;/p&gt;&#xA;" OwnerUserId="18244" LastActivityDate="2018-09-20T06:06:06.213" Title="Mapping Actions to the Output Layer in Keras Model for a Board Game" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8058" PostTypeId="1" CreationDate="2018-09-20T06:46:46.007" Score="0" ViewCount="531" Body="&lt;p&gt;&lt;strong&gt;Problem Statement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 4 main input features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a small snippet of the data for clearer understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gate name -&gt; for example AND Gate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;index_1 -&gt; &lt;code&gt;[0.001169, 0.005416, 0.01391, 0.03037, 0.06381, 0.1307, 0.2645, 0.532]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;index_2 -&gt; &lt;code&gt;[7.906e-05, 0.001123, 0.00321, 0.007253, 0.01547, 0.03191, 0.06478, 0.1305]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;values -&gt; &lt;code&gt;[[11.0081, 14.0303, 18.8622, 27.3426, 43.8661, 76.7538, 142.591, 274.499], &#xA;           [11.3461, 14.3634, 19.1985, 27.6827, 44.2106, 77.0954, 142.926, 274.879], &#xA;           [12.258, 15.2816, 20.1095, 28.5856, 45.1057, 77.9778, 143.8, 275.758], &#xA;           [13.665, 16.7457, 21.5835, 30.0545, 46.5581, 79.4212, 145.252, 277.192], &#xA;           [15.6636, 18.9526, 23.9051, 32.4281, 48.9011, 81.7052, 147.477, 279.371], &#xA;           [17.8838, 21.5839, 26.8957, 35.7103, 52.3901, 85.2132, 150.89, 282.714], &#xA;           [19.3338, 23.6933, 29.7184, 39.1212, 56.4053, 89.9721, 155.913, 287.637], &#xA;           [18.7856, 23.9999, 31.1794, 41.7549, 60.0043, 95.0488, 162.951, 295.005]]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My task is to predict this &lt;code&gt;values&lt;/code&gt; matrix, given that I have &lt;code&gt;index_1&lt;/code&gt; and &lt;code&gt;index_2&lt;/code&gt;. Originally this &lt;code&gt;values&lt;/code&gt; matrix is propagation delay, calculated using a simulator called SPICE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where I am facing problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no written relation between Index_1, index_2 or values since simulator calculates this value using it's own models. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have made a CSV file which contains the data in separate columns. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another approach that I thought. If I can give index_1, index_2 and any 5*5 sub-matrix to the model, and the model can predict the values of whole 8*8 Matrix. But the problem is again, which machine learning model do I use. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approaches Tried so Far&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have tried a CNN model for this but it is giving me very low accuracy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Used one dense fully connected neural network but it is over-fitting the data and not giving me any values for matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am still stuck at how to predict the matrix values given this data.  What are other strategies can be used? &lt;/p&gt;&#xA;" OwnerUserId="18392" LastEditorUserId="4302" LastEditDate="2018-09-21T03:31:39.870" LastActivityDate="2021-05-10T11:04:40.427" Title="Machine learning to predict 8*8 matrix values using three independent matrices" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8060" PostTypeId="1" AcceptedAnswerId="8075" CreationDate="2018-09-20T08:14:08.480" Score="1" ViewCount="712" Body="&lt;p&gt;The Wumpus World proposed in book of Stuart Russel and Peter Norvig, is a game which happens on a 4x4 board and the objective is to grab the gold and avoiding the threats that can kill you. The rules of game are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You move just one box for round&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Start in position (1,1), bottom left&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You have a vector of sensors for perceiving the world around you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When you are next to another position (including the gold), the vector is 'activated'.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is one wumpus (a monster), 2-3 pits (feel free to put more or less) and just one gold pot&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You only have one arrow that flies in a straight line and can kill the wumpus&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Entering the room with a pit, the wumpus or the gold finishes the game&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QHzFT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QHzFT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scoring is as follows: +1000 for grabbing the gold, -1000 for dying to the wumpus, -1 for each step, -10 for shooting an arrow. Fore more details about the rules, chapter 7 of &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;nofollow noreferrer&quot;&gt;the book&lt;/a&gt; explains them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well now that game has been explained, the question is: in the book, the solution is demonstrated by logic and searching, does there exist another form to solve that problem with neural networks? If yes, how to do that? What topology to use? What paradigm of learning and algorithms to use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1*: My English is horrible, if you can send grammar corrections, I'm grateful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2*: I think this is a bit confusing and a bit complex. if you can help me to clarify better, please do commentary or edit!&lt;/p&gt;&#xA;" OwnerUserId="18391" LastEditorUserId="4709" LastEditDate="2018-09-20T12:41:44.133" LastActivityDate="2018-09-21T13:28:38.530" Title="Does a solution for Wumpus World with neural networks exist?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8061" PostTypeId="1" CreationDate="2018-09-20T08:51:24.263" Score="2" ViewCount="995" Body="&lt;p&gt;I am trying to implement &lt;a href=&quot;https://arxiv.org/abs/1511.07528&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. In this paper, the author uses the forward derivative to compute the Jacobian matrix &lt;b&gt;dF/dx&lt;/b&gt; using chain rule where F is the probability got from the last layer and X is input image.&#xA;My model is given below. Kindly let me know how to go about doing that?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class LeNet5(nn.Module):&#xA;&#xA;def __init__(self):&#xA;&#xA;    self.derivative= None # store derivative&#xA;&#xA;    super(LeNet5, self).__init__()&#xA;    self.conv1= nn.Conv2d(1,6,5)&#xA;    self.relu1= nn.ReLU()&#xA;    self.maxpool1= nn.MaxPool2d(2,2)&#xA;&#xA;    self.conv2= nn.Conv2d(6,16,5)&#xA;    self.relu2= nn.ReLU()&#xA;    self.maxpool2= nn.MaxPool2d(2,2)&#xA;&#xA;    self.conv3= nn.Conv2d(16,120,5)&#xA;    self.relu3= nn.ReLU()&#xA;&#xA;    self.fc1= nn.Linear(120,84)&#xA;    self.relu4= nn.ReLU()&#xA;&#xA;    self.fc2= nn.Linear(84,10)&#xA;    self.softmax= nn.Softmax(dim= -1)&#xA;&#xA;&#xA;def forward(self,img, forward_derivative= False):&#xA;    output= self.conv1(img)&#xA;    output= self.relu1(output)&#xA;    output= self.maxpool1(output)&#xA;&#xA;    output= self.conv2(output)&#xA;    output= self.relu2(output)&#xA;    output= self.maxpool2(output)&#xA;&#xA;    output= self.conv3(output)&#xA;    output= self.relu3(output)&#xA;&#xA;    output= output.view(-1,120)&#xA;    output= self.fc1(output)&#xA;    output= self.relu4(output)&#xA;&#xA;    output= self.fc2(output)&#xA;    F= self.softmax(output)&#xA;&#xA;    # want to comput the jacobian dF/dimg &#xA;    jacobian= computeJacobian(F,img)#how to write this function&#xA;&#xA;    return F, jacobian&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17372" LastActivityDate="2018-09-21T19:59:50.607" Title="Compute Jacobian matrix of Deep learning model?" Tags="&lt;deep-learning&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8062" PostTypeId="2" ParentId="8048" CreationDate="2018-09-20T12:19:25.623" Score="1" Body="&lt;p&gt;You are right, that pseudocode is not correct. In particular, the definition of $H_t^i$ in line $11$ should be changed; all the way on the right-hand side, it should have $3N - 3j$ columns of $0$s, rather than $3j$ columns of $0$s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that change, every matrix $H_t^i$ will have the same number of columns: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$6 + 3j - 3 + 3N - 3j = 3 + 3N,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which evaluates to a total of $48$ in your case (because you have $N = 15$ landmarks). That's precisely the correct dimensionality required for matrix multiplication with your $\mu_t$ vector.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The version of the book that you linked to appears to be a fairly old draft. &lt;a href=&quot;http://www.probabilistic-robotics.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This webpage&lt;/a&gt; contains errata for the third edition for the book, in which page 393 corresponds to what was page 310 in your version of the book. The errata for that third edition of the book can be downloaded at the following URL: &lt;a href=&quot;http://probabilistic-robotics.informatik.uni-freiburg.de/corrections/pg393.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://probabilistic-robotics.informatik.uni-freiburg.de/corrections/pg393.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There you'll find the fix that I described above, but also some other fixes (most of them are just notational, adding bars over the $\mu$ vectors, but it looks like a more serious issue was additionally fixed in line 13, where a minus was changed to a plus).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-20T12:19:25.623" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8063" PostTypeId="1" CreationDate="2018-09-20T13:14:32.060" Score="11" ViewCount="3726" Body="&lt;p&gt;I'm training an &lt;code&gt;auto-encoder&lt;/code&gt; network with &lt;code&gt;Adam&lt;/code&gt; optimizer (with &lt;code&gt;amsgrad=True&lt;/code&gt;) and &lt;code&gt;MSE loss&lt;/code&gt; for Single channel Audio Source Separation task. Whenever I decay the learning rate by a factor, the network loss jumps abruptly and then decreases until the next decay in learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Pytorch for network implementation and training.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Following are my experimental setups:&#xA;&#xA; Setup-1: NO learning rate decay, and &#xA;          Using the same Adam optimizer for all epochs&#xA;&#xA; Setup-2: NO learning rate decay, and &#xA;          Creating a new Adam optimizer with same initial values every epoch&#xA;&#xA; Setup-3: 0.25 decay in learning rate every 25 epochs, and&#xA;          Creating a new Adam optimizer every epoch&#xA;&#xA; Setup-4: 0.25 decay in learning rate every 25 epochs, and&#xA;          NOT creating a new Adam optimizer every time rather&#xA;          using PyTorch's &quot;multiStepLR&quot; and &quot;ExponentialLR&quot; decay scheduler &#xA;          every 25 epochs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting very surprising results for setups #2, #3, #4 and am unable to reason any explanation for it. Following are my results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Setup-1 Results:&#xA;&#xA;Here I'm NOT decaying the learning rate and &#xA;I'm using the same Adam optimizer. So my results are as expected.&#xA;My loss decreases with more epochs.&#xA;Below is the loss plot this setup.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Plot-1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3AvAH.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3AvAH.png&quot; alt=&quot;Setup-1 Results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;optimizer = torch.optim.Adam(lr=m_lr,amsgrad=True, ...........)&#xA;&#xA;for epoch in range(num_epochs):&#xA;    running_loss = 0.0&#xA;    for i in range(num_train):&#xA;        train_input_tensor = ..........                    &#xA;        train_label_tensor = ..........&#xA;        optimizer.zero_grad()&#xA;        pred_label_tensor = model(train_input_tensor)&#xA;        loss = criterion(pred_label_tensor, train_label_tensor)&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;        running_loss += loss.item()&#xA;    loss_history[m_lr].append(running_loss/num_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Setup-2 Results:  &#xA;&#xA;Here I'm NOT decaying the learning rate but every epoch I'm creating a new&#xA;Adam optimizer with the same initial parameters.&#xA;Here also results show similar behavior as Setup-1.&#xA;&#xA;Because at every epoch a new Adam optimizer is created, so the calculated gradients&#xA;for each parameter should be lost, but it seems that this doesnot affect the &#xA;network learning. Can anyone please help on this?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Plot-2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rR6Ts.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rR6Ts.png&quot; alt=&quot;Setup-2 Results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for epoch in range(num_epochs):&#xA;    optimizer = torch.optim.Adam(lr=m_lr,amsgrad=True, ...........)&#xA;&#xA;    running_loss = 0.0&#xA;    for i in range(num_train):&#xA;        train_input_tensor = ..........                    &#xA;        train_label_tensor = ..........&#xA;        optimizer.zero_grad()&#xA;        pred_label_tensor = model(train_input_tensor)&#xA;        loss = criterion(pred_label_tensor, train_label_tensor)&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;        running_loss += loss.item()&#xA;    loss_history[m_lr].append(running_loss/num_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Setup-3 Results: &#xA;&#xA;As can be seen from the results in below plot, &#xA;my loss jumps every time I decay the learning rate. This is a weird behavior.&#xA;&#xA;If it was happening due to the fact that I'm creating a new Adam &#xA;optimizer every epoch then, it should have happened in Setup #1, #2 as well.&#xA;And if it is happening due to the creation of a new Adam optimizer with a new &#xA;learning rate (alpha) every 25 epochs, then the results of Setup #4 below also &#xA;denies such correlation.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Plot-3:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jmjRD.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jmjRD.png&quot; alt=&quot;Setup-3 Results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;decay_rate = 0.25&#xA;for epoch in range(num_epochs):&#xA;    optimizer = torch.optim.Adam(lr=m_lr,amsgrad=True, ...........)&#xA;&#xA;    if epoch % 25 == 0  and epoch != 0:&#xA;        lr *= decay_rate   # decay the learning rate&#xA;&#xA;    running_loss = 0.0&#xA;    for i in range(num_train):&#xA;        train_input_tensor = ..........                    &#xA;        train_label_tensor = ..........&#xA;        optimizer.zero_grad()&#xA;        pred_label_tensor = model(train_input_tensor)&#xA;        loss = criterion(pred_label_tensor, train_label_tensor)&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;        running_loss += loss.item()&#xA;    loss_history[m_lr].append(running_loss/num_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Setup-4 Results:  &#xA;&#xA;In this setup, I'm using Pytorch's learning-rate-decay scheduler (multiStepLR)&#xA;which decays the learning rate every 25 epochs by 0.25.&#xA;Here also, the loss jumps everytime the learning rate is decayed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As suggested by @Dennis in the comments below, I tried with both &lt;code&gt;ReLU&lt;/code&gt; and &lt;code&gt;1e-02 leakyReLU&lt;/code&gt; nonlinearities. But, the results seem to behave similar and loss first decreases, then increases and then saturates at a higher value than what I would achieve without learning rate decay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot-4 shows the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot-4:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/56AAD.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/56AAD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer=optimizer, milestones=[25,50,75], gamma=0.25)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer=optimizer, gamma=0.95)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scheduler = ......... # defined above&#xA;optimizer = torch.optim.Adam(lr=m_lr,amsgrad=True, ...........)&#xA;&#xA;for epoch in range(num_epochs):&#xA;&#xA;    scheduler.step()&#xA;&#xA;    running_loss = 0.0&#xA;    for i in range(num_train):&#xA;        train_input_tensor = ..........                    &#xA;        train_label_tensor = ..........&#xA;        optimizer.zero_grad()&#xA;        pred_label_tensor = model(train_input_tensor)&#xA;        loss = criterion(pred_label_tensor, train_label_tensor)&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;        running_loss += loss.item()&#xA;    loss_history[m_lr].append(running_loss/num_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDITS: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As suggested in the comments and reply below, I've made changes to my code and trained the model. I've added the code and plots for the same. &lt;/li&gt;&#xA;&lt;li&gt;I tried with various &lt;code&gt;lr_scheduler&lt;/code&gt; in &lt;code&gt;PyTorch (multiStepLR, ExponentialLR)&lt;/code&gt; and plots for the same are listed in &lt;code&gt;Setup-4&lt;/code&gt; as suggested by @Dennis in comments below.&lt;/li&gt;&#xA;&lt;li&gt;Trying with leakyReLU as suggested by @Dennis in comments.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any help.&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8720" LastEditorUserId="8720" LastEditDate="2018-10-03T07:35:43.577" LastActivityDate="2018-10-10T18:24:52.940" Title="Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;optimization&gt;&lt;autoencoders&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8065" PostTypeId="2" ParentId="8061" CreationDate="2018-09-20T18:32:39.357" Score="2" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1511.07528.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Limitations of Deep Learning in Adversarial Settings&lt;/em&gt;, Papernot et. al., 2016, IEEE&lt;/a&gt; the chain rule is used, &quot;To express &lt;span class=&quot;math-container&quot;&gt;$\nabla F(X∗)$&lt;/span&gt; in terms of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and constant values only.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Earlier is stated, &quot;Our understanding of how changes made to inputs affect a DNN’s output stems from the evaluation of the forward derivative: a matrix we introduce and define as the Jacobian of the function learned by the DNN. The forward derivative is used to construct adversarial saliency maps indicating input features to include in perturbation &lt;span class=&quot;math-container&quot;&gt;$\partial X$&lt;/span&gt; in order to produce adversarial samples inducing a certain behavior from the DNN;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And later, &quot;We define the forward derivative as the Jacobian matrix of the function F learned by the neural network during training. For this example, the output of F is one dimensional, the matrix is therefore reduced to a vector (below). Both components of this vector are computable using the adversary’s knowledge, and later we show how to compute this term efficiently.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla F(X) = \Big[ \frac {\partial F(X)} {\partial X_1}, \frac {\partial F(X)} {\partial X_2} \Big] \quad\quad (2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Between equation (2) and equation (6) your questeion is answered, resulting in an equation where the chain rule has already been applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac {\partial F_j (X)} {\partial x_i} = \Big(W_{n+1,j} \, . \, \frac {\partial H_n} {\partial x_i} \Big) \times \frac {\partial f_{n+1,j}} {\partial x_i} \;(W_{n+1,j} \, . \, H_n + b_{n+1,j}) \quad\quad(6)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is (6) that you must implement in code, but that is done after the DNN is converged as the paper states above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Applying the Chain Rule&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$z = f(y)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y = g(x)$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\frac {dz} {dx} = f'(g(x)) g'(x)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applied twice, if &lt;span class=&quot;math-container&quot;&gt;$w = e(z)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$z = f(y)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$y = g(x)$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\frac {dw} {dx} = e'(f(g(x)) f'(g(x)) g'(x)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applied indefinitely we multiply all the first order derivatives of intermediate values, which has the following first order predicate logic expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\forall \; 0 \le n \lt N \land v_{n + 1} = f_{n + 1}(v_n) \land F_{n+1}(x) = f_{n+1}(F_n(x)) \land g(a) = \dfrac {d} {d a} \, f(a)$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\exists \; g_n(v_0) = \prod_{\,n = 1}^{\,n &amp;lt; N} {g_n} (F_n(v_0))$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivatives of convolution kernel values are the values themselves because they represent attenuation of the form &lt;span class=&quot;math-container&quot;&gt;$y = ax + b$&lt;/span&gt;.  This is true of first degree linear activation functions too.  The derivative normally used for ReLU (although the x = 0 case is actually undefined) is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\dfrac {d} {dx} R(x) = {\begin{cases}x &amp;lt; 0 &amp;amp; 0\\x = 0 &amp;amp; 1\\x &amp;gt; 0 &amp;amp; 1 \end{cases}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative of a max function used in max pool (although the u = v case is actually undefined) is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\dfrac {d} {dt} M(u, v) = {\begin{cases}u &amp;lt; v &amp;amp; \frac {dv} {dt}\\u = v &amp;amp; \frac {dv} {dt}\\u &amp;gt; v &amp;amp; \frac {du} {dt}\end{cases}}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-21T19:59:50.607" LastActivityDate="2018-09-21T19:59:50.607" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8067" PostTypeId="2" ParentId="7897" CreationDate="2018-09-20T20:59:06.097" Score="4" Body="&lt;p&gt;&lt;strong&gt;State of Rosehip Research&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Rosehip neuron is an important discovery, with vast implications to AI and its relationship to the dominant intelligence on earth for at least the last 50,000 years.  The paper that has spawned other articles is &lt;a href=&quot;https://www.nature.com/articles/s41593-018-0205-2.epdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Transcriptomic and morphophysiological  evidence for a specialized human cortical GABAergic cell type&lt;/em&gt;, Buldog et. al., September 2018, Nature Neuroscience&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relationship between this neuron type and its DNA expression is beginning.  No data is available regarding the impact of the Rosehop distinctions on neural activity during learning or leveraging what has been learned.  Surely, research along those lines is indicated, but the discovery was just published.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Benefit of the Interdisciplinary Approach to AI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That those who reference papers like this can see value in the unification or at least alignment of knowledge across disciplines is most likely beneficial to AI progress and progress in the other fields of cognitive science, bioinformatics, business automation, manufacturing and consumer robotics, psychology, and even law, ethics and philosophy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That such interest in aligning understanding along interdisciplinary lines is present in AI Stack Exchange is certainly beneficial to the community growth in both professional and social dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disparity Between What Works&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the human brain, neurons work.  Whether Rosehip neurons are a prerequisite to language, the building of and leveraging of complex models, or transcendent emotions such as love in homo sapiens is unknown and will remain so in the near future.  However, we have a fifty millennia long proof of concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also know that artificial networks work.  We use them in business, finance, industry, consumer products, and a variety of web services today.  When a pop-up asks whether the answer given was helpful, our answer becomes a label in a set of real data from which samples are extracted for machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, the cells that are working are offspring of the 1957 perceptron with the addition of the application of gradient descent using an efficient corrective signal distribution strategy we call back propagation.  The comprehension of neuron function in 1957 was grossly short of what we now know to be functional features of mammalian brain neurons.  The Rosehip discovery may widen that gap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Spiking Networks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spiking network research more realistically models neurons, and neuromorphic research and development has been placing improved models into VLSI chips.  The joint venture between IBM and MIT is another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Correlating Neural Function to Brain Function&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relationship intelligence and the number of proteins or molecules may not be the most telling.  These are more likely relationships between metrics and features and the intelligence of the system.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Genetic features that have been identified (22 of them) that directly affect intelligence testing results &amp;mdash; For instance the correlation between polymorphisms of the oxytocin receptor genes OXTR rs53576, rs2254298, and rs2228485 and intelligence is known &amp;mdash; See the &lt;a href=&quot;https://ai.stackexchange.com/questions/7337/is-the-singularity-concept-mathematically-flawed&quot;&gt;question containing references to discovery of 22 genes that affect intelligence test results significantly&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Neurochemical expression resulting from environmental factors varying the levels of oxytosin, dopamine, serotonin, neuropeptide Y, and canabinoids which is involved in global and regional functional behavior in the human brain&lt;/li&gt;&#xA;&lt;li&gt;Signal topology (distinct from sizes and counts and distinct from the topology of created by packaging neural nets in the cranial region) &amp;mdash; Signal topology is now being identified.  Scanning technology has developed to the point where signal paths can be identified by tracking pulses in temporal space and determine causality.&lt;/li&gt;&#xA;&lt;li&gt;Synaptic plasticity, a type of neural plasticity&lt;/li&gt;&#xA;&lt;li&gt;Total number of neurons applied to a particular brain function&lt;/li&gt;&#xA;&lt;li&gt;Impact on axon and cell body thermodynamics on signal transmission, a key element in modelling a brain neuron&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;None of these are yet modelled in such a way that simulation accuracy has been confirmed, but the need to research along these lines is clearly indicated as this question implies.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-16T00:03:26.557" LastActivityDate="2018-10-16T00:03:26.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8068" PostTypeId="1" AcceptedAnswerId="8072" CreationDate="2018-09-20T21:33:32.960" Score="2" ViewCount="101" Body="&lt;p&gt;Many have examined the idea of modifying learning rate at discrete times during the training of an artificial network using conventional back propagation.  The goals of such work have been a balance of the goals of artificial network training in general.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Minimal convergence time given a specific set of computing resources&lt;/li&gt;&#xA;&lt;li&gt;Maximal accuracy in convergence with regard to the training acceptance criteria&lt;/li&gt;&#xA;&lt;li&gt;Maximal reliability in achieving acceptable test results after training is complete&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The development of a surface involving these three measurements would require multiple training experiments, but may provide a relationship that itself could be approximated either by curve fitting or by a distinct deep artificial network using the experimental results as examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Epoch index&lt;/li&gt;&#xA;&lt;li&gt;Learning rate hyper-parameter value&lt;/li&gt;&#xA;&lt;li&gt;Observed rate of convergence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The goal of such work would be to develop, via manual application of analytic geometry experience or via deep network training the following function, where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the ideal learning rate for any given epoch indexed by &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is the loss function result, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\Psi$&lt;/span&gt; is a function the result of which approximates the ideal learning rate for as large an array of learning scenarios possible within a clearly defined domain.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha_i = \Psi (\epsilon, i)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The development of arriving at &lt;span class=&quot;math-container&quot;&gt;$\Psi$&lt;/span&gt; as a closed form (formula) would be of general academic and industrial value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this been done?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-21T08:31:26.947" Title="Is a calculus or ML approach to varying learning rate as a function of loss and epoch been investigated?" Tags="&lt;deep-learning&gt;&lt;optimization&gt;&lt;topology&gt;&lt;convergence&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8069" PostTypeId="2" ParentId="7891" CreationDate="2018-09-21T00:19:16.933" Score="1" Body="&lt;p&gt;A generative adversarial network is probably not the best approach for generating the images desired.  We can assume from the comments that the data is not collected.  That's a good thing, because a set of rasterized images, labeled with student age or grade is an inferior input form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that access to a student population is planned or already negotiated, which is also good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the drawing, as it is being drawn, is seen through each student's eyes, the primary features correlated with drawing skill development is motor control, shape formation, and color choice.  If the sheet of paper is placed over a drawing tablet, the tablet's incoming USB stream events are captured to a file, and the color selection is somehow recorded or automatically determined by having students hold the pencil or crayon up to the computer's camera before using it, a much better &lt;em&gt;in natura&lt;/em&gt; input stream can be developed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pre-processing can lead to an expression of each drawing experience as a sequence of events arranged in temporal order with the following dimensions for each event.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Relative time from the instruction to draw in seconds&lt;/li&gt;&#xA;&lt;li&gt;Color&lt;/li&gt;&#xA;&lt;li&gt;Nearest x grid&lt;/li&gt;&#xA;&lt;li&gt;Nearest y grid&lt;/li&gt;&#xA;&lt;li&gt;Pressure&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Determining color from camera input may be developed using LSTM approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dimensions of the label for each of these sequences would be those demographics and rankings that would most closely correlated with developmental stages.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Student age&lt;/li&gt;&#xA;&lt;li&gt;Student gender&lt;/li&gt;&#xA;&lt;li&gt;Curriculum grade (-1, 0, 1, 2, ... 12, where -1 is preschool and 0 is kindergarten)&lt;/li&gt;&#xA;&lt;li&gt;Identifier of the drawing instructions given to the class&lt;/li&gt;&#xA;&lt;li&gt;Grade ranking of the student in the class&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The micro-analysis attached to each ELEMENT in the sequence includes these additional dimensions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Drawing rate of the utensil given by &lt;span class=&quot;math-container&quot;&gt;$r = \frac {\sqrt{(x - x_p)^2 + (y - y_p)^2}} {t - t_p}$&lt;/span&gt; where the subscript p indicates the values are drawn from the previous event in the sequence.&lt;/li&gt;&#xA;&lt;li&gt;Drawing direction given by &lt;span class=&quot;math-container&quot;&gt;$\theta = \arctan (x - x_p, \; y - y_p)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Curvature &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; calculated using cubic splines or some other data fitting approach&lt;/li&gt;&#xA;&lt;li&gt;FFT spectrum &lt;span class=&quot;math-container&quot;&gt;$\vec{a}$&lt;/span&gt; and Lyapunov exponent &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; applied to auto-correlation results&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is a modification of the system Google uses to synthesize speech, based on the WaveNet design. In the diagram, the residual function is defined as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$z = \tanh \, (W_{f,k} x + V_{f,k} y) \, \odot \, \sigma \, (W_{g,k} x + V_{g,k} y)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The development required is that the &lt;span class=&quot;math-container&quot;&gt;$\vec{a}$&lt;/span&gt; must now be accompanied with scalars &lt;span class=&quot;math-container&quot;&gt;$r, \theta, \kappa, and \lambda$&lt;/span&gt;, but the resulting drawings are likely to have many of the hand-eye developmental features of the examples.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eA3i4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eA3i4.png&quot; alt=&quot;Child Drawing Learning Design like WaveNet&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-09-21T00:19:16.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8071" PostTypeId="2" ParentId="8024" CreationDate="2018-09-21T01:49:46.613" Score="0" Body="&lt;p&gt;The classification of images as relevant or irrelevant can be done using LSTM, the design of which is perhaps best explained without the mathematical detail of the original paper by Eugene Kang's &lt;a href=&quot;https://github.com/llSourcell/LSTM_Networks/blob/master/LSTM%20Demo.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Long Short-Term Memory: Concept text and diagrams&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That the data is unstructured is not an issue if the images can be extracted and placed into raw pixel matrices with appropriate indices into  the database so the results of classification can be inserted into the database in a way where the image and the classification are associated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The labels from a training point of view are not the indices into the database but the single bit indication of relevance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LSTM network must be trained on a sufficient sample of labeled example data before it will be usable to classify the images.  That requirement may further require a significant amount of manual classification unless some form of delegation, hints from associated features in the database, or crowd sourcing can be used to generate labels.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-21T01:58:47.920" LastActivityDate="2018-09-21T01:58:47.920" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8072" PostTypeId="2" ParentId="8068" CreationDate="2018-09-21T07:29:17.927" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Has this been done?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Difficult to prove a negative, but I suspect although plenty of research has been done into finding ideal learning rate values (the need for learning rate at all is an annoyance), it has not been done to the level of suggesting a global function worth approximating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that learning rate tuning, like other hyperparameter tuning, is highly dependent on the problem at hand, &lt;em&gt;plus&lt;/em&gt; the other hyperparamater values currently in use, such as size of layers, which optimiser is in use, what regularisation is being used, activation functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although you may be hoping for &lt;span class=&quot;math-container&quot;&gt;$\Psi(\epsilon, i | P)$&lt;/span&gt; to exist where P is the problem domain, it likely does not except as a mean value over all &lt;span class=&quot;math-container&quot;&gt;$\Psi(\epsilon, i | D, H)$&lt;/span&gt; for the problem domain, where D is the dataset and H all the other hyperparameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is likely that such a function exists, of ideal learning rate for best &lt;em&gt;expected&lt;/em&gt; convergence per epoch. However, it would be incredibly expensive to sample it with enough detail to make approximating it useful. Coupled with limited applicability (not domain-specific, but linked to data and other hyperparameters), a search through all possible learning rate trajectories looks like it would give poor return on investment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, the usual pragmatic approaches are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Include learning rate in hyperparameter searches, such as grid search, random search, genetic algorithms and other global optimisers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Decay learning rate using one of a few approaches that have been successfully guessed and experiments have shown working. These have typically been validated by plotting learning curves of loss functions or other metrics, and the same tracking is usually required in new experiments to check that the approach is still beneficial.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-540-74690-4_26&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimisers use a dynamic learning rate parameter&lt;/a&gt;, which is similar to your idea but based on reacting to measurements during learning as opposed to changes based on an ideal function. They have a &lt;em&gt;starting&lt;/em&gt; learning rate, then adjust it based on  heuristics derived from measuring learning progress. These heuristics can be based on per-epoch measurements, such as whether a validation set result is improving or not. One such approach is to increase learning rate whilst results per epoch are improving, and reduce learning rate if results are not improving, or have got worse. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried this last option, on a Kaggle competition, and it worked to some extent for me, but did not really improve results overall - I think it is one of many promising ideas in ML that can be made to work, but that has not stayed as a &quot;must have&quot;, unlike say dropout, or using CNNs for images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some optimisers store a multiplier per layer or even per weight - RMSProp and Adam for example track rate of change of each parameter, and adjust the rate for each weight during updates. These can work very well in large networks, where the issue is not so much needing a specific learning rate at any &lt;em&gt;time&lt;/em&gt;, but that a single learning rate is too crude to cover the large range of gradients and differences in gradients across the index space of all the connections. With RMSProp and Adam, the need to pick specific learning rates or explore them is much reduced, and often a library's default is fine.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-21T08:31:26.947" LastActivityDate="2018-09-21T08:31:26.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8073" PostTypeId="2" ParentId="8063" CreationDate="2018-09-21T08:48:30.243" Score="10" Body="&lt;p&gt;I see no reason why decaying learning rates should create the kinds of jumps in losses that you are observing. It should &quot;slow down&quot; how quickly you &quot;move&quot;, which in the case of a loss that otherwise consistently shrinks really should, at worst, just lead to a plateau in your losses (rather than those jumps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing I observe in your code is that you re-create the optimizer from scratch every epoch. I have not yet worked enough with PyTorch to tell for sure, but doesn't this just destroy the internal state / memory of the optimizer every time? I think you should just create the optimizer once, before the loop through the epochs. If this is indeed a bug in your code, it should also actually still be a bug in the case where you do not use learning rate decay... but maybe you simply get lucky there and don't experience the same negative effects of the bug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For learning rate decay, I'd recommend using the &lt;a href=&quot;https://pytorch.org/docs/0.3.0/optim.html?highlight=lr_scheduler#how-to-adjust-learning-rate&quot; rel=&quot;noreferrer&quot;&gt;official API for that&lt;/a&gt;, rather than a manual solution. In your particular case, you'll want to instantiate a &lt;a href=&quot;https://pytorch.org/docs/0.3.0/optim.html?highlight=lr_scheduler#torch.optim.lr_scheduler.StepLR&quot; rel=&quot;noreferrer&quot;&gt;StepLR&lt;/a&gt; scheduler, with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;optimizer&lt;/code&gt; = the ADAM optimizer, which you probably should only instantiate once.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;step_size = 25&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;gamma = 0.25&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can then simply call &lt;code&gt;scheduler.step()&lt;/code&gt; at the start of every epoch (or maybe at the end? the example in the API link calls it at the start of every epoch).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If, after the changes above, you still experience the issue, it would also be useful to run each of your experiments multiple times and plot average results (or plot lines for all experiments). Your experiments should theoretically be identical during the first 25 epochs, but we still see huge differences between the two figures even during those first 25 epochs in which no learning rate decay occurs (e.g., one figure starts at a loss of ~28K, the other starts at a loss of ~40K). This may simply be due to different random initializations, so it'd be good to average that nondeterminisim out of your plots.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-21T08:48:30.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8075" PostTypeId="2" ParentId="8060" CreationDate="2018-09-21T13:28:38.530" Score="2" Body="&lt;p&gt;Yes! If you read ahead to the chapters in reinforcement learning in the same book, you'll see that the wompus world appears again there. Techniques like Q-learning can be used to solve it, and since Q-learning involves learning the shape of a function, a neural network can be employed as a function approximator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to treat this problem as an input/output mapping (states -&gt; actions), and to learn which actions produce the greatest rewards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note however, that these approaches rely on trial and error. The logic based approach reasons about the rules of the game, and can play reasonably well right away. The learning approach will need to try and fail many times before playing well.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-09-21T13:28:38.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8078" PostTypeId="2" ParentId="7328" CreationDate="2018-09-21T18:17:12.533" Score="5" Body="&lt;p&gt;I am surprised no one mentioned some of the specific research directions in the analog AI field. And also to clarify Artificial Intelligence is not exactly the same as Machine Learning as this &lt;a href=&quot;https://ai.stackexchange.com/questions/35/what-is-the-difference-between-artificial-intelligence-and-machine-learning&quot;&gt;answer&lt;/a&gt; suggests. Recent advances in analog computation have only been in the field of Machine Learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Analog CMOS:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;First off let us talk about the earliest analog implementations of neurons. &lt;a href=&quot;https://scholar.google.co.in/citations?user=kdHjCAMAAAAJ&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dr. Giacomo Indiveri, et al&lt;/a&gt; has been a few of the pioneers in the field. Although with CMOS logic you can design spiking Neural Nets with &lt;a href=&quot;https://en.wikipedia.org/wiki/Spike-timing-dependent_plasticity&quot; rel=&quot;nofollow noreferrer&quot;&gt;STDP&lt;/a&gt; (Spike Time Dependent Plasticity), it is difficult to make use of in Machine Learning algorithms. The human brain is yet to be fully understood, especially how it communicates complex information with spikes. The spike-based networks are good in performing relatively small image recognition and low complexity tasks (Most papers seem to be more concerned about improving the performance rather than apply to highly complex tasks). Due to the sheer number of transistors available, we might be able to make use of them in complex tasks.&lt;/p&gt;&#xA;&lt;p&gt;The best example would be Google is using this idea of low precision in TPU's and compensating precision, by using a huge number of processing units which is causing some kind of trade-off between time, precision, and area. This can be analogous to a huge number of transistors in a processor albeit with low precision. (&lt;a href=&quot;https://cloud.google.com/blog/products/gcp/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu&quot; rel=&quot;nofollow noreferrer&quot;&gt;An in-depth look at Google’s first Tensor Processing Unit (TPU)&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;NOTE: Some might argue CMOS technology falls under the digital domain, but since we are not specifically using CMOS here to perform any digital operation I like to think of it as analog.&lt;/p&gt;&#xA;&lt;p&gt;Spike-based tasks are apparently quite good for &lt;a href=&quot;https://en.wikipedia.org/wiki/Winner-take-all_(computing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Winner Take All networks&lt;/a&gt; (kind of like &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-organizing_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self Organising Maps&lt;/a&gt;), so it is the general way of implementing Machine Learning Algorithms in VLSI chips.&lt;/p&gt;&#xA;&lt;p&gt;Spike-based networks do not have ideal memory, you cannot have high precision weights. They have proposed to implement biological weights or synapses or memory using capacitors, but apparently, it faces problems similar to normal silicon chips, like charge leakage and also from other Silicon-based non-idealities, and from what I understood, they also can model limited weights (like -1, 0, 1).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Digital Computation:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Herein, comes digital computation. Tasks that require a high amount of floating-point representation cannot simply be implemented by spikes, since we don't yet know or even able to completely mimic the biophysical or any aspects of a true neuron for that matter. Digital computation simply helps in conveying more information well with as much precision as we like (if we design such a CPU). Even though bottlenecks are a known drawback of Von Neumann architecture for the digital computation it is not as much of a problem as information representation via spikes. Spikes always have a fixed magnitude, the only way it probably conveys information is by its frequency and sign (excitatory or inhibitory). Also, clock speeds are pretty high in modern computers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Memristors: A new direction&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Herein comes the most recent invention, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Memristor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memristor&lt;/a&gt;. This by far has been the most promising analog device in Machine Learning. Memristors are a very new concept predicted in the '70s and produced only in 2008. Basically, they are RRAM's or Resistive RAMs. In this, the resistance of the &lt;strong&gt;Memory Resistor&lt;/strong&gt; or &lt;strong&gt;Memristor&lt;/strong&gt; is directly related to the past current history which is very similar to biophysical models of neurons. They can also be trained easily using crossbar arrays (basically matrix of electrical contacts) of memristors (crossbar arrays will represent weight matrices, the voltage applied along rows or along columns determines forward propagation or backward propagation).&lt;/p&gt;&#xA;&lt;p&gt;Thus Memristor gives a real analog spin to Machine Learning algorithms. Unfortunately, due to its recent arrival, there are a lot of problems which are yet to be resolved.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Memristors can degrade quite quickly, that is they have limited training cycles.&lt;/li&gt;&#xA;&lt;li&gt;Memristors introduce a lot of noise, which apparently does not help in the cause of regularisation as an ML engineer might think.&lt;/li&gt;&#xA;&lt;li&gt;Exotic elements required to make it (&lt;span class=&quot;math-container&quot;&gt;$TiO_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$HfO_2$&lt;/span&gt;) the users for Memristors in academic circles are very limited. But a few labs working on this area are:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://engineering.purdue.edu/NRL/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nano-electronics Research Laboratory, Purdue University&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.electrochem.mat.ethz.ch/research/memristor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Electrochemical Materials, ETH Zurich&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.westernsydney.edu.au/marcs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Brain Project&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.westernsydney.edu.au/marcs&quot; rel=&quot;nofollow noreferrer&quot;&gt;The MARCS Institute for Brain, Behaviour, and Development&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Neuromorphic Photonics:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Recently, there has been an interest in the field of Neuromorphic photonics. Here is a short &lt;a href=&quot;https://www.researchgate.net/profile/Bhavin_Shastri/publication/322217688_Neuromorphic_Photonics/links/5a4e649c0f7e9b234d9d1a54/Neuromorphic-Photonics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; on the same. I am not familiar with the internal workings of the same, but AFAIK it involves the transmission of information in an optical form within the processor chip itself. This leads to some advantages over normal analog or digital circuits:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Faster information processing.&lt;/li&gt;&#xA;&lt;li&gt;Higher information density.&lt;/li&gt;&#xA;&lt;li&gt;Better data fidelity due to very few losses.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="5351" LastEditDate="2021-02-18T14:00:30.577" LastActivityDate="2021-02-18T14:00:30.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8079" PostTypeId="1" CreationDate="2018-09-21T21:03:26.313" Score="1" ViewCount="16" Body="&lt;p&gt;The idea that come to my mind is called Value Based Model for ANN. We use simple DCF formula to calculate kind of Q value: Rewards/Discount rate. Discount rate is a risk of getting the reward on the information that agent know about. Of course if you have many factors you just sum that. So, we calculate FV for every cell that agent know information about and this is a predicted data. We put predicted - actual and teach model how to run using loss function. Rephrased, does increase in output layer actually train the model to be better? The human logic is that if I took course I have a bigger value that helps me to live. What about NN? Does it actually more precise if we with time increase output?&lt;/p&gt;&#xA;" OwnerUserId="18436" LastActivityDate="2018-09-21T21:03:26.313" Title="Does inflation should occur in output layer when I do Artificial Neural Network to increase smartness of the model?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8080" PostTypeId="1" AcceptedAnswerId="8082" CreationDate="2018-09-21T22:38:36.357" Score="4" ViewCount="105" Body="&lt;p&gt;Let’s say I have a neural net doing classification and I’m doing stochastic gradient descent to train it. If I know that my current approximation is a decent approximation, can I conclude that my gradient is a decent approximation of the gradient of the true classifier everywhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, suppose that I have a true loss function, $f$, and an estimation of it, $f_k$. Is it the case that there exists a $c$ (dependent on $f_k$) such that for all $x$ and $\epsilon &amp;gt; 0$ if $|f(x)-f_k(x)|&amp;lt;\epsilon$ then $|\nabla f(x) - \nabla f_k(x)|&amp;lt;c\epsilon$? This isn’t true for general functions, but it may be true for neural nets. If this exact statement isn’t true, is there something along these lines that is? What if we place some restrictions on the NN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal I have in mind is that I’m trying to figure out how to calculate how long I can use a particular sample to estimate the gradient without the error getting too bad. If I am in a context where resampling is costly, it may be worth reusing the same sample many times as long as I’m not making my error too large. My long-term goal is to come up with a bound on how much error I have if I use the same sample $k$ times, which doesn’t seem to be something in the literature as far as I’ve found.&lt;/p&gt;&#xA;" OwnerUserId="12732" LastEditorUserId="12732" LastEditDate="2018-09-25T23:49:40.093" LastActivityDate="2018-09-25T23:49:40.093" Title="Do good approximations produce good gradients?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8081" PostTypeId="2" ParentId="7896" CreationDate="2018-09-22T01:05:57.067" Score="1" Body="&lt;p&gt;It appears that the homework was due two days prior to this answer's writing, but in case it is still relevant in some way, the relevant class notes (which would have been useful if provided in the question along with the homework) &lt;a href=&quot;http://rail.eecs.berkeley.edu/deeprlcourse-fa17/f17docs/lecture_3_rl_intro.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;are here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first instance of expectation placed on the student is, &quot;Please show equation 12 by using the law of iterated expectations, breaking &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\tau \sim p \theta(\tau)}$&lt;/span&gt; by decoupling the state-action marginal from the rest of the trajectory.&quot;  Equation 12 is this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{t = 1}^{T} E_{\tau \sim p \theta(\tau)} [\nabla_\theta \log \pi_\theta(a_t|s_t)(b(s_t))] = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The class notes identifies &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta(a_t|s_t)$&lt;/span&gt; as the state-action marginal.  It is not a proof sought, but a sequence of algebraic steps to perform the decoupling and show the degree to which independence of the state-action marginal can be achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This exercise is a preparation for the next step in the homework and draws only on the review of CS189, Burkeley's Introduction to Machine Learning course, which does not contain the Law of Total Expectation in its syllabus or class notes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the relevant information is in the above link for class notes and requires only intermediate algebra.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-09-22T04:30:41.657" LastActivityDate="2018-09-22T04:30:41.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8082" PostTypeId="2" ParentId="8080" CreationDate="2018-09-22T04:05:27.047" Score="5" Body="&lt;p&gt;In general &lt;span class=&quot;math-container&quot;&gt;$|f(x) - f_k(x)| \leq \epsilon$&lt;/span&gt; doesn't ensure &lt;span class=&quot;math-container&quot;&gt;$|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$&lt;/span&gt;. And for neural networks there is no reason to believe it will happen either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also look at Sobolev Training Paper (&lt;a href=&quot;https://arxiv.org/abs/1706.04859&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1706.04859&lt;/a&gt;). In particular, note that Sobolev training was better than critic training, which indirectly may indicate approximating function may not be the same as approximating gradient and function. In Sobolev training, the network is trained to match gradient and function whereas in critic training network is trained to match function. They produce quite different results which might give us some hints about the above problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if two functions are arbitrary close, they might not be close in gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &#xA;(Trying to come up with a negative example)&#xA;Consider &lt;span class=&quot;math-container&quot;&gt;$f(x) = g(x) + \epsilon \sin (\frac {kx} {\epsilon}) $&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; is some neural network. Now, we train some another neural network &lt;span class=&quot;math-container&quot;&gt;$h(x)$&lt;/span&gt; to fit &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and after training we get &lt;span class=&quot;math-container&quot;&gt;$h(x) = g(x)$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$h(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; have same weights precisely). However, &lt;span class=&quot;math-container&quot;&gt;$\nabla f_x =\nabla g_x + k\cos (\frac {kx} {\epsilon})$&lt;/span&gt; is not arbitarariy close &lt;span class=&quot;math-container&quot;&gt;$\nabla g_x$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this example is enough to prove that a neural network that nicely approximates the function may not nicely approximate the gradients and no such result can be proved mathematically rigorously. However, considering the paper in the discussion, you might think for practical purposes it works. However, if you have both function and grad information available that is expected to work better. &lt;/p&gt;&#xA;" OwnerUserId="18443" LastEditorUserId="18443" LastEditDate="2018-09-22T17:43:31.713" LastActivityDate="2018-09-22T17:43:31.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8083" PostTypeId="1" AcceptedAnswerId="8108" CreationDate="2018-09-22T05:59:03.060" Score="2" ViewCount="63" Body="&lt;p&gt;While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial &amp;amp; error process.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a way that helps me to architect my model fundamentally before training?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be&#xA;changed - increased or decreased?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.&lt;/p&gt;&#xA;" OwnerUserId="17980" LastEditorUserId="2444" LastEditDate="2020-05-19T19:41:04.317" LastActivityDate="2020-05-19T20:12:28.373" Title="Is there a way that helps me to architect my CNN fundamentally before training?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8085" PostTypeId="2" ParentId="2008" CreationDate="2018-09-22T06:58:09.530" Score="9" Body="&lt;p&gt;In NLP you have an inherent ordering of the inputs so RNNs are a natural choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For variable sized inputs where there is no particular ordering among the inputs, one can design networks which:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;use a repetition of the same subnetwork for each of the groups of inputs (i.e. with shared weights). This repeated subnetwork learns a representation of the (groups of) inputs.&lt;/li&gt;&#xA;&lt;li&gt;use an operation on the representation of the inputs which has the same symmetry as the inputs. For order invariant data, averaging the representations from the input networks is a possible choice.&lt;/li&gt;&#xA;&lt;li&gt;use an output network to minimize the loss function at the output based on the combination of the representations of the input.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The structure looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x4cCi.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x4cCi.png&quot; alt=&quot;network structure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar networks have been used to learn the relations between objects (&lt;a href=&quot;https://arxiv.org/abs/1702.05068&quot; rel=&quot;noreferrer&quot;&gt;arxiv:1702.05068&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple example of how to learning the sample variance of a variable sized set of values is given &lt;a href=&quot;https://medium.com/@andre.holzner/learning-a-function-with-a-variable-number-of-inputs-with-pytorch-c487e35d4dba&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; (disclaimer: I'm the author of the linked article).&lt;/p&gt;&#xA;" OwnerUserId="18159" LastActivityDate="2018-09-22T06:58:09.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8086" PostTypeId="2" ParentId="7896" CreationDate="2018-09-22T09:32:22.770" Score="8" Body="&lt;p&gt;Using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_total_expectation&quot; rel=&quot;nofollow noreferrer&quot;&gt;law of iterated expectations&lt;/a&gt; one has:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\triangledown _\theta \sum_{t=1}^T \mathbb{E}_{(s_t,a_t) \sim p(s_t,a_t)} [b(s_t)] = \nabla_\theta \sum_{t=1}^T \mathbb{E}_{s_t \sim p(s_t)} \left[ \mathbb{E}_{a_t \sim \pi_\theta(a_t | s_t)} &#xA; \left[ b(s_t) \right]\right] =$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;written with integrals and moving the gradient inside (linearity) you get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$= \sum_{t=1}^T \int_{s_t} p(s_t) \left(\int_{a_t} \nabla_\theta b(s_t) \pi_\theta(a_t | s_t) da_t \right)ds_t =$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can now move &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta$&lt;/span&gt; (due to linearity) and &lt;span class=&quot;math-container&quot;&gt;$b(s_t)$&lt;/span&gt; (does not depend on &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;) form the inner integral to the outer one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$= &#xA;\sum_{t=1}^T \int_{s_t} p(s_t) b(s_t) \nabla_\theta \left(\int_{a_t} \pi_\theta(a_t | s_t)  da_t \right)ds_t= $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi_\theta(a_t | s_t)$&lt;/span&gt; is a (conditional) probability density function, so integrating over all &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; for a given fixed state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; equals &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$= &#xA;\sum_{t=1}^T \int_{s_t} p(s_t) b(s_t) \nabla_\theta 1 ds_t = $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta1 = 0$&lt;/span&gt;, which concludes the proof.&lt;/p&gt;&#xA;" OwnerUserId="18040" LastEditorUserId="2444" LastEditDate="2020-06-10T16:42:11.700" LastActivityDate="2020-06-10T16:42:11.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8087" PostTypeId="2" ParentId="3372" CreationDate="2018-09-22T10:23:29.857" Score="1" Body="&lt;p&gt;In the history of programming the productivity of the programmer has increased. Early MS-DOS base games were programmed in Pascal and Assembly language direct for the CGA graphics card adapter. With the upraising of the C/C++ language and standardized operating systems, it become possible to use software libraries. &lt;a href=&quot;https://stackoverflow.com/questions/7460654/how-were-graphical-dos-games-written&quot;&gt;(1)&lt;/a&gt; In early MS-DOS period, a team of programmers was needed for a simple jump'n'run game, while nowadays a single programmer can create such a game with existing game-engines in one weekend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is obvious to think this development forward and imagine a much advanced technique which support automatic programming. Creating games in a point&amp;amp;click fashion is possible with so called game construction sets. That are games, which comes with a level editor and a Lua scripting engine for modification of existing content. This is not directly related to Artificial Intelligence but goes into the same direction. The idea is, to use computer programs for increase the productivity. In the context of programming such AI-related support is hard to implement. Because programming and game-design contains lots of domain specific knowledge which has to be formalized. If an AI system should support the programming itself, the AI has to parse existing Stackoverflow threads and give hints to the programmer how to implements this in his software. So called “agent-based software engineering” is researched since the 2000s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most advanced example in semi-autonomous game design is perhaps the RPG adventure generator described in &lt;a href=&quot;https://arxiv.org/pdf/1802.05219.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barros, Gabriella Alves Bulhoes, et al. &quot;Who killed albert einstein? From open data to murder mystery games.&quot; IEEE Transactions on Games (2018)&lt;/a&gt; It can parse Wikipedia article to generate a playable game from scratch.&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2018-09-22T10:23:29.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8089" PostTypeId="1" CreationDate="2018-09-22T15:58:25.997" Score="4" ViewCount="126" Body="&lt;p&gt;I'm an embedded systems designer, with a little background in fuzzy logic.  I'm developing a device for machine diagnostics based on a series of sensors data.  My question is whether through fuzzy logic I can develop some kind of diagnostic indicator of machine parts failure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that I have the Sensor A and sensor B near three parts.  I am wondering if it is possible to get probabilities that part 1, part 2, or part 3 are in failure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can collect all physical parameters as temperature, pressure, speed, vibration, etc. related to each of those parts.  Also I have the expert person who can give me info about relation among data collection and failures.  However, I don't have interest in accomplishing this with an expert system.&lt;/p&gt;&#xA;" OwnerUserId="18452" LastEditorUserId="4302" LastEditDate="2018-10-22T11:36:33.243" LastActivityDate="2018-12-21T15:22:36.027" Title="Machine diagnostics with fuzzy Logic" Tags="&lt;getting-started&gt;&lt;pattern-recognition&gt;&lt;fuzzy-logic&gt;&lt;embedded-design&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8091" PostTypeId="1" AcceptedAnswerId="8092" CreationDate="2018-09-22T18:18:21.100" Score="3" ViewCount="129" Body="&lt;p&gt;Depending on the source I find people using different variations of the &quot;squared error function&quot;, how come that be? &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variation 1&lt;/a&gt;&lt;/strong&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IhBq0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IhBq0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;a href=&quot;http://mccormickml.com/2014/03/04/gradient-descent-derivation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variation 2&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sjktH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sjktH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that it is being devided by 1 over m as opposed to variation 1 (1/2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stuff inside the ()^2 is simply notation I get that, but dividing by 1/m and 1/2 will cleary get a different result. Which version is the &quot;correct&quot; one, or is there no such thing as a correct or &quot;official&quot; squared error function?&lt;/p&gt;&#xA;" OwnerUserId="11814" LastActivityDate="2018-09-22T18:55:34.987" Title="Variations of the squared error function" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8092" PostTypeId="2" ParentId="8091" CreationDate="2018-09-22T18:55:34.987" Score="3" Body="&lt;p&gt;The first variation is named &quot;&lt;span class=&quot;math-container&quot;&gt;$E_{total}$&lt;/span&gt;&quot;. It contains a sum which is not very well-specified (has no index, no limits). &lt;strong&gt;Rewriting it using the notation of the second variation&lt;/strong&gt; would lead to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E_{total} = \sum_{i = 1}^m \frac{1}{2} \left( y^{(i)} - h_{\theta}(x^{(i)}) \right)^2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x^{(i)}$&lt;/span&gt; denotes the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th training example&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h_{\theta}(x^{(i)})$&lt;/span&gt; denotes the model's output for that instance/example&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y^{(i)}$&lt;/span&gt; denotes the ground truth / target / label for that instance&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; denotes the number of training examples&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Because the term inside the large brackets is squared, the sign doesn't matter, so we can rewrite it (switch around the subtracted terms) to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E_{total} = \sum_{i = 1}^m \frac{1}{2} \left( h_{\theta}(x^{(i)}) - y^{(i)} \right)^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now it already looks quite a lot like your second variation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second variation does still have a &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; terms outside the sum. That is because your &lt;strong&gt;second variation computes the mean squared error over all the training examples&lt;/strong&gt;, rather than the &lt;strong&gt;total error computed by the first variation&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either error can be used for training. I'd personally lean towards using the &lt;strong&gt;mean error&lt;/strong&gt; rather than the &lt;strong&gt;total error&lt;/strong&gt;, mainly because the &lt;strong&gt;scale of the mean error is independent of the batch size &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;&lt;/strong&gt;, whereas the &lt;strong&gt;scale of the total error is proportional to the batch size&lt;/strong&gt; used for training. Either option is valid, but &lt;strong&gt;they'll likely require different hyperparameter values (especially for the learning rate), due to the difference in scale&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;With that &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; term explained, the only remaining difference is the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; term inside the sum (can also be pulled out of the sum), which is present in the first variation but not in the second. The reason for including that term is given &lt;a href=&quot;https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the page you linked to for the first variation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; is included so that exponent is cancelled when we differentiate later on. The result is eventually multiplied by a learning rate anyway so it doesn’t matter that we introduce a constant here.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-22T18:55:34.987" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8093" PostTypeId="1" CreationDate="2018-09-23T01:14:07.010" Score="1" ViewCount="69" Body="&lt;p&gt;So I trained an AI to generate shakespeare, which it did somewhat well. I used &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;this 10,000 character sample&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next I tried to get it to generate limericks using these &lt;a href=&quot;http://oedilf.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;100,000 limericks&lt;/a&gt;. It generated garbage output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I limited it to 10,000 characters, it then started giving reasonable limerick output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could this happen? I thought more data was always better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI was a neural network with some LSTM layers, implemented in keras.&lt;/p&gt;&#xA;" OwnerUserId="18006" LastActivityDate="2018-09-25T06:31:49.260" Title="Why would giving my AI more data make it perform worse?" Tags="&lt;python&gt;&lt;recurrent-neural-networks&gt;&lt;datasets&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8094" PostTypeId="1" AcceptedAnswerId="8136" CreationDate="2018-09-23T03:24:56.677" Score="4" ViewCount="551" Body="&lt;p&gt;I'm learning logistic regression and &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; regularization.&#xA;The cost function looks like below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(w) = -\displaystyle\sum_{i=1}^{n} (y^{(i)}\log(\phi(z^{(i)})+(1-y^{(i)})\log(1-\phi(z^{(i)})))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the regularization term is added. (&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is a regularization strength)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(w) = -\displaystyle\sum_{i=1}^{n} (y^{(i)}\log(\phi(z^{(i)})+(1-y^{(i)})\log(1-\phi(z^{(i)}))) + \frac{\lambda}{2}\| w \|$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, I know that if &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; becomes bigger, extreme weights are penalized and weights become closer to zero. However, I'm having a hard time to prove this mathematically.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Delta{w} = -\eta\nabla{J(w)}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial}{\partial{w_j}}J(w) = (-y+\phi(z))x_j + \lambda{w_j}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta{w} = \eta(\displaystyle\sum_{i=1}^{n}(y^{(i)}-\phi(z^{(i)}))x^{(i)} - \lambda{w_j})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This doesn't show the reason why incrementing &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; makes weight become closer to zero. It is not intuitive.&lt;/p&gt;&#xA;" OwnerUserId="18427" LastEditorUserId="2444" LastEditDate="2021-01-30T17:34:01.080" LastActivityDate="2021-01-30T17:34:01.080" Title="How does L2 regularization make weights smaller?" Tags="&lt;machine-learning&gt;&lt;proofs&gt;&lt;hyper-parameters&gt;&lt;regularization&gt;&lt;l2-regularization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8097" PostTypeId="1" AcceptedAnswerId="8098" CreationDate="2018-09-23T10:29:05.220" Score="0" ViewCount="384" Body="&lt;p&gt;I'm trying to implement an algorithm that would choose the optimal next move for the game of Connect 4. As I just want to make sure that the basic &lt;strong&gt;minimax&lt;/strong&gt; works correctly, I am actually testing it like a Connect 3 on a 4x4 field. This way I don't need the alpha-beta pruning, and it's more obvious when the algorithm makes a stupid move.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that the algorithm &lt;strong&gt;always&lt;/strong&gt; starts the game with the leftmost move, and also during the game it's just very stupid. It doesn't see the best moves.&lt;/p&gt;&#xA;&lt;p&gt;I have thoroughly tested methods &lt;code&gt;makeMove()&lt;/code&gt;, &lt;code&gt;undoMove()&lt;/code&gt;,  &lt;code&gt;getAvailableColumns()&lt;/code&gt;, &lt;code&gt;isWinningMove()&lt;/code&gt; and &lt;code&gt;isLastSpot()&lt;/code&gt; so I am absolutely &lt;strong&gt;sure&lt;/strong&gt; that the problem is not there.&lt;/p&gt;&#xA;&lt;p&gt;Here is my algorithm.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NextMove.java&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;private static class NextMove {&#xA;    final int evaluation;&#xA;    final int moveIndex;&#xA;&#xA;    public NextMove(int eval, int moveIndex) {&#xA;        this.evaluation = eval;&#xA;        this.moveIndex = moveIndex;&#xA;    }&#xA;&#xA;    int getEvaluation() {&#xA;        return evaluation;&#xA;    }&#xA;&#xA;    public int getMoveIndex() {&#xA;        return moveIndex;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Algorithm&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;private static NextMove max(C4Field field, int movePlayed) {&#xA;    // moveIndex previously validated&#xA;    &#xA;    // 1) check if moveIndex is a final move to make on a given field&#xA;    field.undoMove(movePlayed);&#xA;    &#xA;    // check&#xA;    if (field.isWinningMove(movePlayed, C4Symbol.BLUE)) {&#xA;        field.playMove(movePlayed, C4Symbol.RED);&#xA;        return new NextMove(BLUE_WIN, movePlayed);&#xA;    }&#xA;    if (field.isWinningMove(movePlayed, C4Symbol.RED)) {&#xA;        field.playMove(movePlayed, C4Symbol.RED);&#xA;        return new NextMove(RED_WIN, movePlayed);&#xA;    }&#xA;    if (field.isLastSpot()) {&#xA;        field.playMove(movePlayed, C4Symbol.RED);&#xA;        return new NextMove(DRAW, movePlayed);&#xA;    }&#xA;    &#xA;    field.playMove(movePlayed, C4Symbol.RED);&#xA;    &#xA;    // 2) moveIndex is not a final move&#xA;    // --&amp;gt; try all possible next moves&#xA;    final List&amp;lt;Integer&amp;gt; possibleMoves = field.getAvailableColumns();&#xA;    int bestEval = Integer.MIN_VALUE;&#xA;    int bestMove = 0;&#xA;    for (int moveIndex : possibleMoves) {           &#xA;        field.playMove(moveIndex, C4Symbol.BLUE);&#xA;        &#xA;        final int currentEval = min(field, moveIndex).getEvaluation();&#xA;        if (currentEval &amp;gt; bestEval) {&#xA;            bestEval = currentEval;&#xA;            bestMove = moveIndex;&#xA;        }&#xA;&#xA;        field.undoMove(moveIndex);&#xA;    }&#xA;    &#xA;    return new NextMove(bestEval, bestMove);&#xA;}&#xA;&#xA;private static NextMove min(C4Field field, int movePlayed) {&#xA;    // moveIndex previously validated&#xA;    &#xA;    // 1) check if moveIndex is a final move to make on a given field&#xA;    field.undoMove(movePlayed);&#xA;    &#xA;    // check&#xA;    if (field.isWinningMove(movePlayed, C4Symbol.BLUE)) {&#xA;        field.playMove(movePlayed, C4Symbol.BLUE);&#xA;        return new NextMove(BLUE_WIN, movePlayed);&#xA;    }&#xA;    if (field.isWinningMove(movePlayed, C4Symbol.RED)) {&#xA;        field.playMove(movePlayed, C4Symbol.BLUE);&#xA;        return new NextMove(RED_WIN, movePlayed);&#xA;    }&#xA;    if (field.isLastSpot()) {&#xA;        field.playMove(movePlayed, C4Symbol.BLUE);&#xA;        return new NextMove(DRAW, movePlayed);&#xA;    }&#xA;    &#xA;    field.playMove(movePlayed, C4Symbol.BLUE);&#xA;    &#xA;    // 2) moveIndex is not a final move&#xA;    // --&amp;gt; try all other moves&#xA;    final List&amp;lt;Integer&amp;gt; possibleMoves = field.getAvailableColumns();&#xA;    int bestEval = Integer.MAX_VALUE;&#xA;    int bestMove = 0;&#xA;    for (int moveIndex : possibleMoves) {&#xA;        field.playMove(moveIndex, C4Symbol.RED);&#xA;        &#xA;        final int currentEval = max(field, moveIndex).getEvaluation();&#xA;        if (currentEval &amp;lt; bestEval) {&#xA;            bestEval = currentEval;&#xA;            bestMove = moveIndex;&#xA;        }&#xA;        &#xA;        field.undoMove(moveIndex);&#xA;    }&#xA;    &#xA;    return new NextMove(bestEval, bestMove);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The idea is that the algorithm takes in the arguments of a &lt;code&gt;currentField&lt;/code&gt; and the &lt;code&gt;lastPlayedMove&lt;/code&gt;. Then it checks if the last move somehow finished the game. If it did, I just return that move, and otherwise I go in-depth with the subsequent moves.&lt;/p&gt;&#xA;&lt;p&gt;Blue player is MAX, red player is MIN.&lt;/p&gt;&#xA;&lt;p&gt;In each step I first undo the last move, because it's easier to check if the &amp;quot;next&amp;quot; move will finish the game, than check if the current field is finished (this would require to analyze for all possible winning options in the field). After I check, I just redo the move.&lt;/p&gt;&#xA;&lt;p&gt;From some reason this doesn't work. I am stuck with that for days! I have no idea what's wrong... Any help greatly appreciated!&lt;/p&gt;&#xA;&lt;h2&gt;EDIT&lt;/h2&gt;&#xA;&lt;p&gt;I'm adding the code how I'm invoking the algorithm.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@Override&#xA;public int nextMove(C4Game game) {&#xA;    C4Field field = game.getCurrentField();&#xA;    C4Field tmp = C4Field.copyField(field);&#xA;&#xA;    int moveIndex = tmp.getAvailableColumns().get(0);&#xA;    final C4Symbol symbol = game.getPlayerToMove().getSymbol().equals(C4Symbol.BLUE) ? C4Symbol.RED : C4Symbol.BLUE;&#xA;    tmp.dropToColumn(moveIndex, symbol);&#xA;&#xA;    NextMove mv = symbol&#xA;            .equals(C4Symbol.BLUE) ? &#xA;                    max(tmp, moveIndex) : &#xA;                        min(tmp, moveIndex);&#xA;&#xA;                    int move = mv.getMoveIndex();&#xA;                    return move;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18470" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-23T12:55:11.693" Title="Connect 4 minimax does not make the best move" Tags="&lt;game-ai&gt;&lt;minimax&gt;&lt;java&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8098" PostTypeId="2" ParentId="8097" CreationDate="2018-09-23T10:37:35.337" Score="2" Body="&lt;p&gt;I suspect that you'll have to remove this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if (field.isWinningMove(movePlayed, C4Symbol.BLUE)) {&#xA;        field.playMove(movePlayed, C4Symbol.RED);&#xA;        return new NextMove(BLUE_WIN, movePlayed);&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;from the &lt;code&gt;max()&lt;/code&gt; method, and remove this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if (field.isWinningMove(movePlayed, C4Symbol.RED)) {&#xA;        field.playMove(movePlayed, C4Symbol.BLUE);&#xA;        return new NextMove(RED_WIN, movePlayed);&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;from the &lt;code&gt;min()&lt;/code&gt; method.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the first case, you're checking whether the move that RED just made was a winning move. You don't want to check there if it was a winning move from BLUE, because it wasn't BLUE who just made that move; it was red. The same counts the other way around in the second case.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Additionally, the initial call into the algorithm seems overly complicated. I am not sure what the intended use of the &lt;code&gt;tmp&lt;/code&gt; variable there is, or that &lt;code&gt;dropToColumn()&lt;/code&gt; call. I would rewrite it to be more like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@Override&#xA;public int nextMove(C4Game game) {&#xA;    C4Field field = game.getCurrentField();&#xA;&#xA;    NextMove mv = null;&#xA;&#xA;    if(game.getPlayerToMove().getSymbol().equals(C4Symbol.BLUE)){&#xA;        mv = max(field, -1);&#xA;    }&#xA;    else{&#xA;        mv = min(field, -1);&#xA;    }&#xA;&#xA;    return mv.getMoveIndex();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will require an adaptation of the &lt;code&gt;max()&lt;/code&gt; and &lt;code&gt;min()&lt;/code&gt; methods such that they skip the whole checking-for-wins thing if the previous &lt;code&gt;movePlayed&lt;/code&gt; equals &lt;code&gt;-1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the code you currently have there, you do not perform a minimax search for the optimal move in the current game state; instead you first arbitrarily modify the current game state using that &lt;code&gt;tmp.dropToColumn()&lt;/code&gt; call, and perform the minimax search in that arbitrarily modified game state. The optimal move to play in such an arbitrarily-modified game state will tend not to be the optimal move in the game state that you really are in.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-09-23T12:55:11.693" LastActivityDate="2018-09-23T12:55:11.693" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="8099" PostTypeId="1" AcceptedAnswerId="8101" CreationDate="2018-09-23T13:54:20.620" Score="1" ViewCount="53" Body="&lt;p&gt;From what I understand, the value function estimates how 'good' it is for an agent to be in a state, and a policy is a mapping of actions to state.&lt;/p&gt;&#xA;&lt;p&gt;If I have understood these concepts correctly, why does the value of a state change with the policy with which an agent gets there?&lt;/p&gt;&#xA;&lt;p&gt;I guess I'm having difficulty grasping the concept that the goodness of a state changes depending on how an agent got there (different policies may have different ways, and hence different values, for getting to a particular state).&lt;/p&gt;&#xA;&lt;p&gt;If there can be a concrete example (perhaps on a grid world or on a chessboard), that might make it clear why that might be the case.&lt;/p&gt;&#xA;" OwnerUserId="18468" LastEditorUserId="2444" LastEditDate="2020-11-20T01:50:59.520" LastActivityDate="2020-11-20T01:50:59.520" Title="Why does the value of state change depending on the policy used to get to that state?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8100" PostTypeId="1" CreationDate="2018-09-23T15:20:15.643" Score="2" ViewCount="211" Body="&lt;p&gt;I want to understand what the &lt;code&gt;gamma&lt;/code&gt; parameter does in an SVM. According to &lt;a href=&quot;http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Intuitively, the &lt;code&gt;gamma&lt;/code&gt; parameter defines how far the influence of a single training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. The &lt;code&gt;gamma&lt;/code&gt; parameters can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't understand this part &amp;quot;&lt;strong&gt;of a single training example reaches&lt;/strong&gt;&amp;quot;, does it refer to the training dataset?&lt;/p&gt;&#xA;" OwnerUserId="18464" LastEditorUserId="2444" LastEditDate="2020-09-12T14:20:37.667" LastActivityDate="2020-09-12T14:20:37.667" Title="What is the purpose of the &quot;gamma&quot; parameter in SVMs?" Tags="&lt;machine-learning&gt;&lt;support-vector-machine&gt;&lt;hyper-parameters&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8101" PostTypeId="2" ParentId="8099" CreationDate="2018-09-23T15:32:50.000" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I guess I'm having difficulty grasping the concept that the goodness of a state changes depending on how an agent got there&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It doesn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of a state changes depending &lt;em&gt;on what the agent will do next&lt;/em&gt;. That is where the dependency on the policy comes in, not in past behaviour, but expectations of future behaviour. The future behaviour depends on the state transitions and rewards presented by the environment, &lt;em&gt;plus&lt;/em&gt; it depends on the distribution of actions chosen by the policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More formally, the value function of a state is not just a relative and arbitrary scoring system, but equals the expected (discounted) sum of rewards, assuming the MDP follows the given dynamics, including action selection:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) = \mathbb{E}_{A \sim \pi}[\sum_{k=0}^\infty \gamma^k R_{t+k+1} | S_t = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without identifying a policy, it is not possible to assess a value function. In value-based &lt;em&gt;control&lt;/em&gt; methods, the policy to evaluate can be implied, somewhat self-referentially, as the policy that acts greedily (or maybe &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedily) according to the current estimates of the value function.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If there can be a concrete example (perhaps on a GridWorld or on a chess board), that might make it clear why that might be the case&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A very simple deterministic MDP with a start state and two terminal states illustrates this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L0CeJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L0CeJ.png&quot; alt=&quot;Very basic MDP&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start in state &lt;strong&gt;B&lt;/strong&gt;. Taking the &lt;em&gt;left&lt;/em&gt; action is followed by a transition (with &lt;span class=&quot;math-container&quot;&gt;$p=1$&lt;/span&gt;) to terminal state &lt;strong&gt;A&lt;/strong&gt;, and a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Taking the &lt;em&gt;right&lt;/em&gt; action is followed by a transition (with &lt;span class=&quot;math-container&quot;&gt;$p=1$&lt;/span&gt;) to terminal state &lt;strong&gt;C&lt;/strong&gt;, and a reward of &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the value of state &lt;strong&gt;B&lt;/strong&gt;? It depends on what the policy chooses. A deterministic &lt;em&gt;left&lt;/em&gt; policy &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$v_{\pi_1}(B) = 0$&lt;/span&gt;, a random policy &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; choosing &lt;em&gt;left&lt;/em&gt; and &lt;em&gt;right&lt;/em&gt; with &lt;span class=&quot;math-container&quot;&gt;$p=0.5$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$v_{\pi_2}(B) = 1.5$&lt;/span&gt;. The optimal policy chooses action &lt;em&gt;right&lt;/em&gt; always and has &lt;span class=&quot;math-container&quot;&gt;$v_{\pi_3}(B) = v^*(B) = 3.0$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-23T16:20:12.523" LastActivityDate="2018-09-23T16:20:12.523" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8103" PostTypeId="2" ParentId="8100" CreationDate="2018-09-23T16:36:16.490" Score="1" Body="&lt;p&gt;Rougly speaking, the higher the gamma, the more complex the model, the higher the risk of overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, as you can read on the page you linked:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If &lt;strong&gt;gamma is too large&lt;/strong&gt;, the radius of the area of influence of the&#xA;  support vectors &lt;strong&gt;only includes the support vector itself&lt;/strong&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;When &lt;strong&gt;gamma is very small&lt;/strong&gt;, the model is too constrained and &lt;strong&gt;cannot&#xA;  capture the complexity or “shape” of the data&lt;/strong&gt;. The &lt;strong&gt;region of influence&#xA;  of any selected support vector would include the whole training set&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="18035" LastActivityDate="2018-09-23T16:36:16.490" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8104" PostTypeId="1" CreationDate="2018-09-23T17:17:40.693" Score="2" ViewCount="179" Body="&lt;p&gt;I'm considering a GPU system for deep learning applications, mainly for training models with large datasets. So I'm not sure whether it makes sense to choose Nvidia NV-Link over more Tesla V100 graphics cards? Is NV-Link the bottleneck that a system with two Tesla V100 and NV-Link is &quot;faster&quot; compared to four Tesla V100 without NV-Link? As I have seen, the Nvidias GPU system DGX-1 and DGX-2 uses NVLink.&lt;/p&gt;&#xA;" OwnerUserId="13295" LastEditorUserId="13295" LastEditDate="2018-09-23T17:36:47.857" LastActivityDate="2018-09-30T06:21:36.227" Title="Is PCIe the bottleneck in a deep learning GPU system, so it makes sense to choose Nvidia NV-Link over more Tesla V100 graphics cards?" Tags="&lt;deep-learning&gt;&lt;hardware&gt;&lt;hardware-evaluation&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-05-13T21:39:43.770" ContentLicense="CC BY-SA 4.0" />
  <row Id="8105" PostTypeId="1" CreationDate="2018-09-23T18:06:33.120" Score="5" ViewCount="55" Body="&lt;p&gt;I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="18453" LastEditorUserId="2444" LastEditDate="2019-08-22T21:20:42.853" LastActivityDate="2019-08-22T21:20:42.853" Title="Which marketing-related classification challenges is a feed forward neural network suited to solve?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8107" PostTypeId="2" ParentId="8093" CreationDate="2018-09-23T19:32:52.043" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why would giving my AI more data make it perform worse?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A lot of possible reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In forecasting, you could have a seasonality. If you have it exactly 3 times, then it is good. If you have it 3.5 times, it becomes worse because it overfits on the months which occur more often than the others.&lt;/li&gt;&#xA;&lt;li&gt;Data quality: In practice, improving the quality of data often yields better results (see &lt;a href=&quot;https://arxiv.org/pdf/1707.09725.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis and Optimization of Convolutional Neural Network Architectures, page 15 (Analysis Techniques)&lt;/a&gt; for 7 approaches to improve your results)&lt;/li&gt;&#xA;&lt;li&gt;Instable training / bad luck: some architectures / problems are super instable. You can execute the randomly initialized training 5 times and get vastly different results.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-09-25T06:31:49.260" LastActivityDate="2018-09-25T06:31:49.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8108" PostTypeId="2" ParentId="8083" CreationDate="2018-09-23T19:39:49.210" Score="2" Body="&lt;p&gt;See &lt;a href=&quot;https://arxiv.org/pdf/1707.09725.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis and Optimization of Convolutional Neural Network Architectures&lt;/a&gt; (my master's thesis).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Is there a way that helps me to architect my model fundamentally before training?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. &lt;a href=&quot;https://autokeras.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://autokeras.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Chapter 3.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="2444" LastEditDate="2020-05-19T20:12:28.373" LastActivityDate="2020-05-19T20:12:28.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8109" PostTypeId="1" AcceptedAnswerId="8110" CreationDate="2018-09-24T04:06:26.310" Score="1" ViewCount="69" Body="&lt;p&gt;I have a course named &quot;Evolutionary Algorithms&quot;, but our teacher is always mentioning the word &quot;optimization&quot; in his lectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused. Is he actually teaching optimization? If yes, why is the name of the course not &quot;Optimization&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between the study of evolutionary algorithms and optimization?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorUserId="2444" LastEditDate="2020-06-02T16:32:58.980" LastActivityDate="2020-09-25T17:50:41.903" Title="What is the difference between the study of evolutionary algorithms and optimization?" Tags="&lt;comparison&gt;&lt;optimization&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8110" PostTypeId="2" ParentId="8109" CreationDate="2018-09-24T05:18:06.820" Score="1" Body="&lt;p&gt;Optimization problems are formally defined by two things:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Optimization objective: &lt;span class=&quot;math-container&quot;&gt;$$\min_{x \in \mathbb{R}} f(x)$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;List of constraints: &lt;span class=&quot;math-container&quot;&gt;$\text{s.t. } x &amp;gt; 0$&lt;/span&gt;; &lt;span class=&quot;math-container&quot;&gt;$\text{s.t. } x^2 &amp;lt; 100$&lt;/span&gt;, ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Optimization theory as a field deals with variations of such problems. The algorithms that are part of optimization include &lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent and variants&lt;/a&gt;, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Simplex_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplex algorithm&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulated annealing&lt;/a&gt; and many more. I would also include evolutionary algorithms as one of the algorithms in this field.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I am confused. Is he actually teaching optimization? If yes, why is the name of the course not &amp;quot;Optimization&amp;quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Evolutionary algorithms are a subfield of optimization. The name of the course is evolutionary algorithms as you likely don't deal with the other aspects of optimization theory (e.g. the algorithms mentioned above).&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="2444" LastEditDate="2020-09-25T17:50:41.903" LastActivityDate="2020-09-25T17:50:41.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8111" PostTypeId="2" ParentId="8058" CreationDate="2018-09-24T05:39:56.737" Score="0" Body="&lt;p&gt;In principle, you can use a fully connected neural network with reshaping for this kind of problem. The questions you should ask yourself are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Baselines: What are the simplest algorithms to approach the problem? How good would a human be?&lt;/li&gt;&#xA;&lt;li&gt;What do I know? Are there any properties of the 8x8 matrix that will always be true? For example, it seems as if the values from left to right strictly increase. Same for top to bottom. This can be used!&lt;/li&gt;&#xA;&lt;li&gt;Are the outputs independent? (e.g. having the (1,1) entry of the matrix, do I know something about the &lt;span class=&quot;math-container&quot;&gt;$(i, j)$&lt;/span&gt; entry of it?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then, of course, there are more specific things I could imagine. If you were not clear about (2), you might have (wrongly) used softmax/tanh/sigmoid in the last layer. You might simply have too little training data for neural networks. Your neural network implementation might be broken.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-09-24T05:39:56.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8112" PostTypeId="2" ParentId="8100" CreationDate="2018-09-24T06:00:33.820" Score="2" Body="&lt;p&gt;I've summarized the &lt;a href=&quot;https://martin-thoma.com/svm-with-sklearn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;key ideas of SVMs&lt;/a&gt;. So this is how &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is used with a gaussian Kernel:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$K_{\text{Gauss}}(\mathbf{x}_i, \mathbf{x}_j) = e^{\frac{-\gamma\|\mathbf{x}_i - \mathbf{x}_j\|^2}{2 \sigma^2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bigger the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, the more &quot;linear&quot; the decision boundary will be. The closer to 0, the more support vectors you have/the more non-linear the boundary is (see &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/svmjs/demo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;interactive example&lt;/strong&gt;&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also find it on udacity: &lt;a href=&quot;https://www.youtube.com/watch?v=m2a2K4lprQw&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVM Gamma Parameter&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, you can use a &lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV&quot; rel=&quot;nofollow noreferrer&quot;&gt;grid search&lt;/a&gt; or random search to get good values.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-09-24T06:00:33.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8113" PostTypeId="2" ParentId="7776" CreationDate="2018-09-24T06:21:43.660" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can one use an Artificial Neural Network to determine the size of an object in a photograph?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes: &lt;a href=&quot;https://papers.nips.cc/paper/2921-learning-depth-from-single-monocular-images.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Depth from Single Monocular Images&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, depth is just one special form of size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you need something partially known, e.g. another car. You don't need to know the exact size of the car, but you know which size cars in general have. If you have an image without any reference, it is impossible.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-09-24T06:21:43.660" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8114" PostTypeId="2" ParentId="7695" CreationDate="2018-09-24T06:41:32.140" Score="0" Body="&lt;p&gt;There are many different problems in computer vision. Four of them are well described by the &lt;a href=&quot;https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852&quot; rel=&quot;nofollow noreferrer&quot;&gt;top image here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Classification: Given an image, say what is on it (a single thing)&lt;/li&gt;&#xA;&lt;li&gt;Classification+Localization: Given an image, say what is on it and draw an axis-aligned bounding box (AABB) around it&lt;/li&gt;&#xA;&lt;li&gt;Object detection: Given an image, draw AABBs around every object and classify those objects&lt;/li&gt;&#xA;&lt;li&gt;Semantic segmentation: See &lt;a href=&quot;https://arxiv.org/pdf/1602.06541.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Survey of semantic segmentation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Instance segmentatation: Like semantic segmentation, but if there are multiple cats then they should be recognized as different objects.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Your question seems to be about object detection. The relevant papers here are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1512.02325.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD: Single Shot MultiBox Detector&lt;/a&gt;: &lt;a href=&quot;https://martin-thoma.com/object-detection/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1506.02640v5&quot; rel=&quot;nofollow noreferrer&quot;&gt;You Only Look Once: Unified, Real-Time Object Detection&lt;/a&gt; (YOLO): &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Website with video&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1804.02767.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLOv3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you actually already have the regions, then you can simply perform classification on them. When you pad / scale / crop them, you can batch-predict them.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-09-24T06:41:32.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8117" PostTypeId="1" AcceptedAnswerId="8123" CreationDate="2018-09-24T09:56:38.203" Score="3" ViewCount="103" Body="&lt;p&gt;Let's assume I want to teach a CNN some physics. Starting with a &lt;a href=&quot;https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net&lt;/a&gt;, I input images &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; as separate channels. I know that my target (produced by a very slow Monte-Carlo code) represents a signal such as &lt;code&gt;f(g(A) * h(B))&lt;/code&gt;, where &lt;code&gt;f&lt;/code&gt;, &lt;code&gt;g&lt;/code&gt; and &lt;code&gt;h&lt;/code&gt; are fairly &quot;convolutional&quot; operations -- meaning, involving mostly blurring and rescaling operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel safe to state that this problem would not be too difficult for the case of &lt;code&gt;f(g(A) + h(B))&lt;/code&gt; -- but what about &lt;code&gt;f(g(A) * h(B))&lt;/code&gt;? Can I expect a basic CNN such as the U-Net to be able to represent the &lt;code&gt;*&lt;/code&gt; (multiplication) operation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or should I expect to be forced to include a &lt;code&gt;Multiply&lt;/code&gt; layer in my network, somewhere where I expect that the part before can learn the &lt;code&gt;g&lt;/code&gt; and &lt;code&gt;h&lt;/code&gt; parts, and the part after can learn the &lt;code&gt;f&lt;/code&gt; part?&lt;/p&gt;&#xA;" OwnerUserId="18486" LastActivityDate="2018-09-24T20:22:29.943" Title="Can a basic CNN (Conv2D, MaxPooling2D, UpSampling2D) find a good approximation of a product of its input channels?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8121" PostTypeId="1" CreationDate="2018-09-24T14:52:52.543" Score="3" ViewCount="82" Body="&lt;p&gt;I have trouble finding material (blog, papers) about this issue, so I'm posting here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking a recent well known example: Musk has tweeted and warned about the potential dangers of AI, saying it is &quot;potentially more dangerous than nukes&quot;, referring the issue of creating a superintelligence whose goals are not aligned with ours. This is often illustrated with the &lt;a href=&quot;https://wiki.lesswrong.com/wiki/Paperclip_maximizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;paperclip maximiser though experiment&lt;/a&gt;. Let's call this first concern &quot;AI alignment&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By contrast, in &lt;a href=&quot;https://www.youtube.com/watch?v=ycPr5-27vSI&quot; rel=&quot;nofollow noreferrer&quot;&gt;a recent podcast&lt;/a&gt;, his concerns seemed more related to getting politicians and decision makers to acknowledge and cooperate on the issue, to avoid potentially dangerous scenarios like an AI arms race. In a paper co-authored by Nick Bostrom: &#xA;&lt;a href=&quot;https://www.fhi.ox.ac.uk/wp-content/uploads/Racing-to-the-precipice-a-model-of-artificial-intelligence-development.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Racing to the Precipice: a Model of Artificial Intelligence Development&lt;/a&gt;, the authors argue that developing AGI in a competitive situation incentivises us to skim on safety precautions, so it is dangerous. Let's call this second concern &quot;AI governance&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about the relative importance between these two issues: AI alignment and AI governance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that most institutions trying to prevent such risks (&lt;a href=&quot;https://intelligence.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIRI&lt;/a&gt;, &lt;a href=&quot;https://www.fhi.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FHI&lt;/a&gt;, &lt;a href=&quot;https://futureoflife.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FLI&lt;/a&gt;, &lt;a href=&quot;http://openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI&lt;/a&gt;, &lt;a href=&quot;https://deepmind.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind&lt;/a&gt; and others) just state their mission without trying to argue about why one approach should be more pressing than the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to assess the relative importance of those two issues? And can you point me any literature about this?&lt;/p&gt;&#xA;" OwnerUserId="1741" LastEditorUserId="4302" LastEditDate="2018-09-25T00:24:10.640" LastActivityDate="2018-09-25T06:59:42.090" Title="Should we focus more on societal or technical issues with AI risk" Tags="&lt;agi&gt;&lt;social&gt;&lt;neo-luddism&gt;&lt;value-alignment&gt;&lt;risk-management&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8123" PostTypeId="2" ParentId="8117" CreationDate="2018-09-24T20:22:29.943" Score="2" Body="&lt;p&gt;I think U-Net is already a quite complex network that probably should (as of my experience) be able to approximate this multiplication. However this would still be an approximation that might not be accurate and maybe only solves a function that looks like a multiplication for the range of the shown input samples defined by your dataset (therefore potentially overfitting on your training dataset).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in general if you know that your target function does have this multiplication then you should definetly excplicitly enforce it in your network. If you know so much about the wanted function it's always better to build a good fitting architecture than using a general neural network architecture. That will ease the optimization and should generalize much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's hard or impossible to tell you for sure how much depth or complexity you need to solve this task within your wanted accuracy. Eventually you should just try it out.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastActivityDate="2018-09-24T20:22:29.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8124" PostTypeId="2" ParentId="7776" CreationDate="2018-09-24T21:06:23.510" Score="2" Body="&lt;p&gt;In my thesis I actually solve the problem of depth estimation with a CNN based on a single monocular image so I can share my experiences for understanding that problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you already stated in general you have the problem that you cannot recover the scale of the scene in an image by geometrical approaches directly. And that is still not the case even if you know the properties of your camera and lens, like focal length, but don't know any absolute sizes of the scene. However, a neural network is still able to solve the task of depth estimation based on monocular images (at least for fixed camera properties) due to known objects sizes it learned through the training on the dataset. That means it can use the learned size of specific objects and the relative depth relations to give a fairly good approximation of the depth in the scene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in your special case this approach would &lt;strong&gt;not work&lt;/strong&gt;, if I understand you correctly. If you just take a photo of a stone that can have arbitrary sizes and no depth cues or any unique patterns that relate to depth are present in the image there is no chance to ever estimate the absolute depth. A CNN eventually would probably just learn some average depth values or recurring depth patterns of your used dataset or memorize the whole training set to minimize the training error since it simply cannot solve this task. So you would not get a tool that does somehow generalize to new scenes. A neural network is still just a function approximator and not something magical that can solve the unsolveable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your usecase there could be some (complex) solutions that could give you a more or less accurate depth estimation. For example you could use a structure from motion approach where you somehow mesure the absolute camera movement with the accelerometer of the phone. Or best would be a stereo-camera based setup where you know the absolute camera displacement of the camera positions which could solve this task if you have textures in your images. With that you could find the absolute depth of specific points through a classical stereo depth estimation or by using a CNN that estimates the depth on the stereo image pair. Another approach would be to let the user input the phone height above the ground or approximate it through the accelerometer of the smartphone to then approximate the stone size based on the size in the image and the absolute known height above the ground (probably inaccurate).&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="13104" LastEditDate="2018-09-25T07:51:55.137" LastActivityDate="2018-09-25T07:51:55.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8126" PostTypeId="1" AcceptedAnswerId="8134" CreationDate="2018-09-24T21:35:10.663" Score="1" ViewCount="59" Body="&lt;p&gt;Within a piece of text, I'm trying to detect who did what to whom. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, in the following sentences:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;CV hit IV. CV was hit by IV.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd like to know who hit whom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't remember what this technique is called. Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="18501" LastActivityDate="2018-09-25T09:03:37.953" Title="Looking for NLP technique and drawing a blank" Tags="&lt;natural-language-processing&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8128" PostTypeId="1" CreationDate="2018-09-25T01:08:12.727" Score="3" ViewCount="713" Body="&lt;p&gt;I have difficulty understanding the following paragraph in the below excerpts from page 4 to page 5 from the paper &lt;a href=&quot;http://proceedings.mlr.press/v48/wangf16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Dueling Network Architectures for Deep Reinforcement Learning&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author said &quot;we can force the advantage function estimator to have zero advantage at the chosen action.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the equation &lt;span class=&quot;math-container&quot;&gt;$(8)$&lt;/span&gt; below, is it correct that &lt;span class=&quot;math-container&quot;&gt;$A - maxA$&lt;/span&gt; is at most zero?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... lack of identifiability is mirrored by poor practical performance when this equation is used directly.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To address this issue of identifiability, we can force the advantage&#xA;  function estimator to have zero advantage at the chosen action. That is, we let the last module of the network implement the forward mapping&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a; \theta, \alpha, \beta) = V(s; \theta, \beta) + \left( A(s, a; \theta, \alpha) - \max_{a' \in | \mathcal{A} |} A(s, a'; \theta, \alpha) \right). \tag{8}$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Now, for &lt;span class=&quot;math-container&quot;&gt;$a^∗ = \text{arg max}_{a' \in \mathcal{A}} Q(s, a'; \theta, \alpha, \beta) = \text{arg max}_{a' \in \mathcal{A}} A(s, a'; \theta, \alpha)$&lt;/span&gt;, we obtain &lt;span class=&quot;math-container&quot;&gt;$Q(s, a^∗; \theta, \alpha, \beta) = V (s; \theta, \beta)$&lt;/span&gt;.  Hence, the stream &lt;span class=&quot;math-container&quot;&gt;$V(s; \theta, \beta)$&lt;/span&gt; provides an estimate of the value function, while the other stream produces an estimate of the advantage function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would like to request further explanation on Equation 9, when the author wrote what is bracketed between the red parentheses below.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An alternative module replaces the max operator with an average:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a; \theta, \alpha, \beta) = V (s; \theta, \beta) + \left( A(s, a; \theta, \alpha) − \frac {1} {|A|} \sum_{a' \in \mathcal{A}} A(s, a'; \theta, \alpha) \right). \tag{9}$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;On the one hand this loses the original semantics of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; because they are now off-target by a constant,&#xA;   but on the other hand it increases the stability of the optimization: with (9) the advantages only need to change as fast as the mean, instead of having to compensate any change to the optimal action’s advantage in (8). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the paper, to address identifiability issue, there are two equation used. My understanding is both equations are trying to fix Advantage part - the last module. For equation &lt;span class=&quot;math-container&quot;&gt;$(8)$&lt;/span&gt;, are we trying to make &lt;span class=&quot;math-container&quot;&gt;$V(s) = Q*(s)$&lt;/span&gt;, as the last module is zero. And for equation &lt;span class=&quot;math-container&quot;&gt;$(9)$&lt;/span&gt;, the resulting &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; = true &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; + mean&lt;span class=&quot;math-container&quot;&gt;$(A)$&lt;/span&gt;? - as the author said ‘On the one hand this loses the original semantics of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; because they are now off-target by a constant’. And the constant refers to mean&lt;span class=&quot;math-container&quot;&gt;$(A)$&lt;/span&gt;? Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="18504" LastEditorUserId="22916" LastEditDate="2019-03-28T22:13:02.037" LastActivityDate="2019-03-28T22:13:02.037" Title="Difficulty in understanding identifiability in the &quot;Dueling Network Architectures for Deep Reinforcement Learning&quot; paper" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8129" PostTypeId="2" ParentId="8128" CreationDate="2018-09-25T01:55:48.457" Score="1" Body="&lt;p&gt;I believe that is explained on the prior page:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Intuitively, the value function &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; measures the how good it is to be in a particular state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, however, measures the the value of choosing a particular action when in this state. The advantage function subtracts the value of the state from the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;&#xA;function to obtain a relative measure of the importance of&#xA;each action.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then two paragraphs above were you started your quote:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;However, we need to keep in mind that &lt;span class=&quot;math-container&quot;&gt;$Q(s, a; \theta, \alpha, \beta)$&lt;/span&gt; is only a parameterized estimate of the true &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function. Moreover, it would be wrong to conclude that &lt;span class=&quot;math-container&quot;&gt;$V (s; \theta, \beta)$&lt;/span&gt; is a good estimator of the state-value function, or likewise that &lt;span class=&quot;math-container&quot;&gt;$A(s, a; \theta, \alpha)$&lt;/span&gt; provides a reasonable estimate of the advantage function.&lt;/p&gt;&#xA;&lt;p&gt;Equation (7) is unidentifiable in the sense that given &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; we cannot recover &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; uniquely. To see this, add a constant to &lt;span class=&quot;math-container&quot;&gt;$V (s; \theta, \beta)$&lt;/span&gt; and subtract the same constant from &lt;span class=&quot;math-container&quot;&gt;$A(s, a; \theta, \alpha)$&lt;/span&gt;. This constant cancels out resulting in the same &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value. This lack of identifiability is mirrored by poor practical performance when this equation is used directly.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Another way of looking at it would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You receive answers to your question&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Answers receive votes&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Answerers have reputation&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In a &lt;em&gt;perfect&lt;/em&gt; world people could vote based on reputation, with a weighing based upon the correctness of the answer.&lt;/p&gt;&#xA;&lt;p&gt;You could simply look at which answer received the most votes and choose it as correct.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;em&gt;real&lt;/em&gt; world things don't work that way, things are correct or incorrect whether they are measured or not (think quantum mechanics) and measurement doesn't always reveal the true answer.&lt;/p&gt;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Parameter#Statistics_and_econometrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parameter Estimation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The estimate of the advantage is only so good, sometimes it's useful to consider it and in other instances it's useful to reject it - intelligently doing both maximizes it's usefulness.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-09-26T10:58:53.237" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8130" PostTypeId="2" ParentId="8121" CreationDate="2018-09-25T05:04:38.193" Score="1" Body="&lt;p&gt;Neither AI alignment nor AI governance are important yet. We are so far away from AGI that we don't even know what is missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't set up safety instructions for interstellar travel, so why should we do it for AGI? I can also come up with a lot of dangers of that...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are real dangers of AI, though. Including &lt;strong&gt;societal issues&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Blind trust&lt;/strong&gt;:  Trusting a machine, although it is not perfect. Just because it is good in most cases and seems to be rational/objective. Example: &lt;a href=&quot;https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Bias&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Unemployment&lt;/strong&gt;: AI has the potential to replace a lot of low-skill jobs with very few super high skilled jobs. For example, self-driving cars (&amp;lt; 100 people for one manufacturer, I guess) could replace all jobs in trasportation (1,076,200 of 31,373,700 in Germany (&lt;a href=&quot;https://www.destatis.de/DE/Publikationen/StatistischesJahrbuch/Arbeitsmarkt.pdf?__blob=publicationFile&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;) - that is 3.4%!). See &lt;a href=&quot;http://www.cgpgrey.com/blog/humans-need-not-apply&quot; rel=&quot;nofollow noreferrer&quot;&gt;CG gray video&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Weapon Systems&lt;/strong&gt;: The thread of &lt;a href=&quot;https://en.wikipedia.org/wiki/Lethal_autonomous_weapon&quot; rel=&quot;nofollow noreferrer&quot;&gt;leathal autonomous weapons&lt;/a&gt; is real. There is the &lt;a href=&quot;https://autonomousweapons.org/slaughterbots/&quot; rel=&quot;nofollow noreferrer&quot;&gt;danger of efficient slaughterbots&lt;/a&gt; and the danger of errors - similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Wech_Baghtu_wedding_party_airstrike&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wech Baghtu wedding party airstrike&lt;/a&gt; or &lt;a href=&quot;https://www.nytimes.com/2018/04/23/world/middleeast/yemen-wedding-bombing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;that one&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are a couple of other problems, but I think they are less severe:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Better fakes: &lt;a href=&quot;https://lyrebird.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;lyrebird.ai&lt;/a&gt; gives a good demo&lt;/li&gt;&#xA;&lt;li&gt;Spam: I think it's easier to improve the spam filters, but humans might have a harder time&lt;/li&gt;&#xA;&lt;li&gt;Data is dominance: Companies like Amazon will have a dominant position on the market as they were the first to aquire important customer information&lt;/li&gt;&#xA;&lt;li&gt;Bubbles / false hopes: AI (or let's better say Machine learning)  is often seen as a golden hammer. It is not. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-09-25T06:59:42.090" LastActivityDate="2018-09-25T06:59:42.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8134" PostTypeId="2" ParentId="8126" CreationDate="2018-09-25T09:03:37.953" Score="3" Body="&lt;p&gt;You might be referring to &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_role_labeling&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Semantic role labeling&lt;/strong&gt;&lt;/a&gt;. SRL is the task of assigning labels to words or phrases in a sentence that shows their semantic role in that sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example &lt;code&gt;CV was hit by IV&lt;/code&gt;, the task is to identify the verb &quot;hit&quot; carried out by the &lt;code&gt;actor&lt;/code&gt; &quot;CV&quot; affected &quot;IV&quot; the &lt;code&gt;recipient&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: If you're only interested in the syntactic relationship among words or phrases in a sentence, not the semantic relationship between them, simple &lt;strong&gt;dependency parsing&lt;/strong&gt; would do the job.&lt;/p&gt;&#xA;" OwnerUserId="7449" LastActivityDate="2018-09-25T09:03:37.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8135" PostTypeId="1" AcceptedAnswerId="20724" CreationDate="2018-09-25T14:15:59.553" Score="5" ViewCount="131" Body="&lt;p&gt;In my daily machine learning / deep learning workflow, I often want to interact with a dataset in my code. Specifically, I would like to be able to load some module/package which can&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;make sure that I have the dataset in a specific folder&lt;/li&gt;&#xA;&lt;li&gt;represent the structure of the dataset on a higher, possible object-oriented level&lt;/li&gt;&#xA;&lt;li&gt;is homogeneous across datasets of the same class, e.g. image classification (set of images with labels, possibly bounding boxes)&lt;/li&gt;&#xA;&lt;li&gt;open-source, such that I could contribute a specification to a previously uncovered dataset&lt;/li&gt;&#xA;&lt;li&gt;that lets me specify a set of commonly used licenses to which I can comply, to filter all compatible datasets&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Currently, I mainly use Python, but other languages are also in the scope of this question (Matlab, Java).&#xA;&lt;strong&gt;Does such an API exist? If not, which ones come closest to the requirements stated above?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you an example of how I would expect it to feel like, see the following python code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import dataset_api as da&#xA;&#xA;in_2012 = da.get('ImageNet', version='2012')&#xA;in_2017 = da.get('ImageNet', version='2017')&#xA;coco = da.get('COCO')&#xA;&#xA;image_classification = da.merge([in_2012, in_2017, coco])&#xA;&#xA;images = image_classification.images&#xA;&#xA;image = images[0]&#xA;image.path  # the absolute file path to the image file on my disk&#xA;image.objects  # an array of object that are visible in this image&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12345" LastEditorUserId="2444" LastEditDate="2020-04-27T21:38:42.803" LastActivityDate="2020-04-28T11:08:29.890" Title="Is there a Python API for manipulating publicly available datasets?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;datasets&gt;&lt;data-science&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ClosedDate="2020-05-21T23:29:05.530" ContentLicense="CC BY-SA 4.0" />
  <row Id="8136" PostTypeId="2" ParentId="8094" CreationDate="2018-09-25T15:13:33.483" Score="3" Body="&lt;p&gt;Here is my take.&lt;/p&gt;&#xA;&lt;p&gt;The larger the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;, the more the corresponding regularization term for a coefficient will be big, so when minimizing the cost function, the coefficient will be reduced by a bigger factor, you can see this effect in the derivation of the update rule for gradient descent for example:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*} \theta_j := \theta_j - \alpha\ \left[ \left( \frac{1}{m}\ \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)} \right) + \frac{\lambda}{m}\theta_j \right] &amp;amp;\ \ \ \ \ \ \ \ \ \ j \in \lbrace 1,2...n\rbrace\newline &amp;amp; \end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*} \theta_j := \theta_j (1- \alpha \frac{\lambda}{m})\left( \frac{\alpha}{m}\ \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)} \right) &amp;amp;\ \ \ \ \ \ \ \ \ \ j \in \lbrace 1,2...n\rbrace\newline &amp;amp; \end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;From this derivation, it is clear that at every update the coefficients get &lt;strong&gt;reduced&lt;/strong&gt; by a factor that is usually a little less than 1 and &lt;strong&gt;directly proportional to &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;&lt;/strong&gt;, so, as &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; gets bigger, the weights get reduced more and more; eventually, for very big values of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;, we risk to totally underfit the data, since only the regularization term will remain in the cost function and all the weights will go to zero.&lt;/p&gt;&#xA;&lt;p&gt;This is for linear regression, but it is essentially the same logic also for logistic regression.&lt;/p&gt;&#xA;&lt;p&gt;This is taken from &lt;a href=&quot;https://www.coursera.org/learn/machine-learning/supplement/pKAsc/regularized-linear-regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's course on Coursera&lt;/a&gt;. A more mathematical precise (and complex) traction of the problem can be found in &lt;a href=&quot;https://bloomberg.github.io/foml/#lecture-6-l1-and-l2-regularization&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Bloomberg machine learning course material&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;PS: in the derivation of the update rule for gradient descent, the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; should be divided by the number of training examples, this is important in choosing the right &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; because we want this relationship to be inversely proportional, otherwise the coefficient won't be decreased.&lt;/p&gt;&#xA;" OwnerUserId="18516" LastEditorUserId="2444" LastEditDate="2021-01-30T17:31:17.043" LastActivityDate="2021-01-30T17:31:17.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8137" PostTypeId="2" ParentId="1479" CreationDate="2018-09-25T16:56:41.647" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do scientists know what is happening inside artificial neural networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;YES&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do scientists or research experts know from the kitchen what is happening inside complex &quot;deep&quot; neural network with at least millions of connections firing at an instant?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I guess &quot;to know from the kitchen&quot; means &quot;to know in detail&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give you a series of analogies:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does an airplane engineer know from the kitchen what happens inside the airplane?&lt;/li&gt;&#xA;&lt;li&gt;Does a chip designer know in detail what happens in the chip (s)he designed?&lt;/li&gt;&#xA;&lt;li&gt;Does a civil engineer know everything about the house he constructed?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The devil is in the detail, but a crucial point here is that it's about artificial structures. They don't randomly appear. You need a lot of knowledge to get anything useful. For Neural Networks, I would say it took roughly 40 years from the publication of the key idea (Rosenblatt perceptron, 1957) to the first application (US Postal Service, 1989). And from there again 13 years of active reserach to really impressive systems (ImageNet 2012).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What we know super well is how the training works&lt;/strong&gt;. Because it needs to be implemented. So on a very small structure, we know it in detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of computers. The chip designers know very well how their chip works. But they will likely only have a very rough idea how the Linux operating system works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is physics and chemistry: Physics describes the core forces of the universe. Does that mean they know everything about chemistry as well? Hell no! A &quot;perfect&quot; physicist can explain everything in chemistry ... but it would be pretty much useless. He would need a lot more information, not be able to skip the irrelevant parts. Simply because he &quot;zoomed in&quot; too much - considers details which are in practice neither interesting nor important. Please note that the knowledge of the physicist is not wrong. Maybe one could even deduce the knowledge from the chemist from it. But this &quot;high-level&quot; understanding of molecule interaction is missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key insight from those two examples are abstraction layers: &lt;strong&gt;You can build complexity from simple structures&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We know well what is in principle achievable with the neural networks&lt;/strong&gt; we design:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A neural network designed to play Go - no matter how sophisticated - will never even be able to play chess. You can, of course, add another abstraction layer around it and combine things. But this approach needs humans.&lt;/li&gt;&#xA;&lt;li&gt;A neural network designed for distinguishing dogs from cats which has only seen pudels and Persian cats will likely perform really bad when it has to decide for Yorkshire Terriers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Oh, and of course we have analytical approaches for neural networks. I wrote my masters thesis about &lt;a href=&quot;https://arxiv.org/pdf/1707.09725.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis and Optimization of Convolutional Neural Network Architectures&lt;/a&gt;. In this context &lt;a href=&quot;https://github.com/marcotcr/lime&quot; rel=&quot;nofollow noreferrer&quot;&gt;LIME&lt;/a&gt; (Local Interpretable Model-Agnostic Explanations) is nice:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wXzfn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wXzfn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-09-25T16:56:41.647" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8138" PostTypeId="2" ParentId="8135" CreationDate="2018-09-25T17:49:35.960" Score="3" Body="&lt;p&gt;scikit-learn has a small data sets API &lt;a href=&quot;http://scikit-learn.org/stable/datasets/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://scikit-learn.org/stable/datasets/index.html&lt;/a&gt;&#xA;I imagine one can add more data sets locally.&#xA;Some data sets are for classification, other for regressions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the only one I know about.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2018-09-25T17:49:35.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8140" PostTypeId="2" ParentId="8089" CreationDate="2018-09-25T19:22:22.980" Score="0" Body="&lt;p&gt;Fuzzy logic is create &quot;sentences that solve a problem&quot; this have &quot;weights&quot; and mix sentences. if you focus is diagnostic, every sentences shall begin with a sencente like to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;if the main network trunk is ok, the health of machine is right&quot;&#xA;&quot;but if the many sensors is not ok, the health of machine is bad&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in word &quot;many&quot; you set &quot;weight&quot; like to 0.6 from 1, etc, after you apply math to fuzification, after you can apply ai to a set of rules that hold health system in many situations. &lt;/p&gt;&#xA;" OwnerUserId="10983" LastActivityDate="2018-09-25T19:22:22.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8142" PostTypeId="1" CreationDate="2018-09-26T08:50:58.333" Score="2" ViewCount="38" Body="&lt;p&gt;I have read about auto encoder. Understood what is encoding part, and decoding part, and the latent space. Now, i tried to implement this in keras. Below is the code. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;iLayer = Input ((784,))&#xA;layer1 = Dense(128, activation='relu' ) (iLayer)&#xA;layer2 = Dense(64, activation='relu') (layer1)&#xA;layer3 = Dense(28, activation ='relu') (layer2)&#xA;layer4 = Dense(64, activation='relu') (layer3)&#xA;layer5 = Dense(128, activation='relu' ) (layer4)&#xA;layer6 = Dense(784, activation='softmax' ) (layer5)&#xA;model = Model (iLayer, layer6)&#xA;model.compile(loss='binary_crossentropy', optimizer='adam')&#xA;&#xA;(trainX, trainY), (testX, testY) =  mnist.load_data()&#xA;print (&quot;shape of the trainX&quot;, trainX.shape)&#xA;trainX = trainX.reshape(trainX.shape[0], trainX.shape[1]* trainX.shape[2])&#xA;print (&quot;shape of the trainX&quot;, trainX.shape)&#xA;model.fit (trainX, trainX, epochs=5, batch_size=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As simple as that, i have a 6 dense layers. I am still able see that the output image is closer to the input image.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From blurred to deblurred conversion, i can have few dense layers like&#xA;  the above (Basically a simple neural network) and get the inputs&#xA;  de-blured. In that cause, why autoencoder is very famous for&#xA;  de-blurring? Am I missing any core part of this auto encoder?.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17994" LastEditorUserId="18663" LastEditDate="2018-10-18T03:16:19.397" LastActivityDate="2018-10-18T03:16:19.397" Title="Autoencoder why it is special for image decoding?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8143" PostTypeId="1" CreationDate="2018-09-26T08:00:54.500" Score="1" ViewCount="34" Body="&lt;p&gt;I am doing a research on above cited topic but I am stuck with how to actually start the project on this. &#xA;What tools are required for this kind of project?&#xA;What resources are required to do project on this?&lt;/p&gt;&#xA;" OwnerUserId="22150" OwnerDisplayName="HIMANSHU PAL" LastEditorUserId="4709" LastEditDate="2018-11-04T20:25:55.313" LastActivityDate="2018-11-04T20:25:55.313" Title="A Binarization Method for Degraded Document Image using Artificial Neural Network and Interpolation Inpainting" Tags="&lt;neural-networks&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8144" PostTypeId="2" ParentId="8128" CreationDate="2018-09-26T10:47:16.497" Score="1" Body="&lt;p&gt;Yes, you're correct, if Equation 8 is used it will only be possible to get estimates &lt;span class=&quot;math-container&quot;&gt;$\leq 0$&lt;/span&gt; out of the term&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left( A(s, a; \theta, \alpha) - \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha) \right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This matches the meaning that we intuitively assign to the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$A(s, a)$&lt;/span&gt; estimators (I'm leaving the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; out of those parentheses for the sake of notational brevity). Intuitively, we want:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; to estimate the value of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and executing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; for the policy that we are learning about.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; to estimate the value of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; for the policy that we are learning about.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A(s, a)$&lt;/span&gt; to estimate the &lt;em&gt;advantage&lt;/em&gt; of executing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; for the policy that we are learning about.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the above three points, &quot;the policy that we are learning about&quot; is the greedy policy, the &quot;optimal&quot; policy given what we have learned so far (ideally this would be truly the optimal policy after a long period of training). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the last point of the three points above, &lt;em&gt;advantage&lt;/em&gt; can intuitively be understood as the gain in estimated value if we choose action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; over whatever the expected value would be if we were following the policy that we are learning about. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we are trying to learn about the greedy policy, we'll ideally (according to our intuition) want the maximum advantage &lt;span class=&quot;math-container&quot;&gt;$A(s, a)$&lt;/span&gt; to be equal to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;; intuitively, the best action is precisely the one we want to execute in our greedy policy, so that best action should not have any relative &quot;advantage&quot;. Similarly, &lt;em&gt;all non-optimal&lt;/em&gt; actions should have a negative advantage, because they are estimated to be worse than what we estimate to be the optimal action(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This intuition is mathematically enforced by using Equation 8 from the paper for training:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a; \theta, \alpha, \beta) = V(s; \theta, \beta) + \left( A(s, a; \theta, \alpha) - \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha) \right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can consider two cases to explain what this is doing:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose that action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the best action we could have selected in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; according to our current estimates, i.e. &lt;span class=&quot;math-container&quot;&gt;$a = \arg \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha)$&lt;/span&gt;. Then, the two terms in the large brackets are equal to each other, so the subtraction yields &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and the state-action value estimate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; equals the state value estimate &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;. This is exactly what we want &lt;strong&gt;because we are trying to learn about the greedy policy&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose that action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is worse than the best action we could have selected in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; according to our current estimates, i.e. &lt;span class=&quot;math-container&quot;&gt;$A(s, a; \theta, \alpha) &amp;lt; \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha)$&lt;/span&gt;. Clearly, I've just stated here that the first term in our subtraction is less than the second term in our subtraction... so the subtraction yields a negative number. This means that the state-action value estimate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; becomes &lt;strong&gt;less than&lt;/strong&gt; the estimated state value &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;. This is also what we want intuitively, &lt;strong&gt;because we started with the assumption that action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; was a suboptimal action&lt;/strong&gt;. Clearly, if we assume that the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is suboptimal, that should lead to a reduction in the estimated value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that afterwards, when they start explaining Equation 9, they actually intentionally deviate from these standard, intuitive understandings that we have of what the three estimators should represent.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Concerning the additional question about Equation 9:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A major problem in the stability of training processes for Deep Reinforcement Learning algorithms (such as these DQN-based algorithms) is that &lt;strong&gt;the update targets contain components that are predictions made by the NN that is being trained&lt;/strong&gt;. For example, the Dueling DQN architecture in this paper generates &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A(s, a)$&lt;/span&gt; predictions, which are combined into &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; predictions, and those &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; predictions of the network itself are also used (combined with some non-prediction reward observations &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;) in the loss function defined to train the Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the Neural Network's own predictions are a part of its training signal. When these are used to update the Network, this will likely change its future predictions in similar situations, which means that its update target will also actually change when it reaches a similar situation again; this is a &lt;strong&gt;moving target&lt;/strong&gt; problem. We do not have a consistent set of update targets as we would in a traditional supervised learning setting for example (where we have a dataset collected offline with fixed labels as prediction targets). Our targets are moving around during the training process, and this can destabilize learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in that explanation following Equation 9, they essentially argue that this &quot;moving target&quot; problem is less bad with Equation 9 than it is with Equation 8, which can result in more stable training. I'm not sure if there is a formal proof of this, but intuitively it does make sense that this would happen in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that you update your network once based on &lt;strong&gt;Equation 8&lt;/strong&gt;. If your learning step changes the prediction of the advantage &lt;span class=&quot;math-container&quot;&gt;$A(s, a)$&lt;/span&gt; of the best action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; by a magnitude of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (kind of informal here, hopefully it makes sense what I'm trying to say), this will in turn move future targets for updates also roughly by a magnitude of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (again, quite informal here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose that you update your network once based on &lt;strong&gt;Equation 9&lt;/strong&gt;. It is unlikely that all of the different actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; have their advantage &lt;span class=&quot;math-container&quot;&gt;$A(s, a)$&lt;/span&gt; move by the same magnitude and in the same direction as a result of this update. It is more likely that some will move up, some will move down, etc. And even if they all move in the same direction, some will likely move by a smaller magnitude than others. In some sense, Equation 9 &quot;averages out&quot; the movements triggered by the learning update in all of these different advantage estimates, which causes the network's prediction targets overall to simply move more slowly, reducing the moving target problem. At least, that's the intuitive idea. Again, I don't think there is a formal proof that this happens, but it does turn out to often help in practice.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-09-27T08:37:13.810" LastActivityDate="2018-09-27T08:37:13.810" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="8147" PostTypeId="1" CreationDate="2018-09-26T14:57:39.903" Score="1" ViewCount="1000" Body="&lt;p&gt;I have a simple question about the choice of activation function for the output layer in feed-forward neural networks.&lt;/p&gt;&#xA;&lt;p&gt;I have seen several codes where the choice of the activation function for the output layer is linear.&lt;/p&gt;&#xA;&lt;p&gt;Now, it might well be that I am wrong about this, but isn't that simply equivalent to a rescaling of the weights connecting the last hidden layer to the output layer? And following this point, aren't you just as well off with just using the identity function as your output activation function?&lt;/p&gt;&#xA;" OwnerUserId="18546" LastEditorUserId="2444" LastEditDate="2020-12-05T21:41:24.993" LastActivityDate="2020-12-05T21:41:24.993" Title="Is a linear activation function (in the output layer) equivalent to an identity function?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8148" PostTypeId="2" ParentId="8147" CreationDate="2018-09-26T15:54:34.997" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;And following this point, aren't you just as well off with just using the identity function as your output activation function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When someone declares that the output of a neural network layer is &lt;em&gt;linear&lt;/em&gt; this is exactly what they mean. It can also be described as &quot;no activation function&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Saying that a NN layer has &lt;em&gt;linear activation&lt;/em&gt; is a kind of short-hand for saying &quot;the whole layer is a linear function of its inputs&quot;, which is true without adding any activation function, just using the weights and bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is usually no separate linear function applied, and libraries such as &lt;a href=&quot;https://keras.io/activations/#linear&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras include the term 'linear' only for completeness&lt;/a&gt;, or so that the choice can be made explicit in the code, as opposed to an unseen default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the link to Keras activation definition above says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Linear (i.e. identity) activation function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-26T16:00:28.673" LastActivityDate="2018-09-26T16:00:28.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8150" PostTypeId="2" ParentId="5861" CreationDate="2018-09-26T19:21:09.223" Score="1" Body="&lt;h3&gt;Forward pass&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The output of a layer can be calculated given the output of the previous layer. So the GPU can parallelize this computation for every layer and over the minibatch which is done by calculating a big matrix. But it needs to be sequential from layer to layer (earlier layers to higher layers). Regarding the layer type convolutions or especially fully connected layers can result in a big matrix calculation.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Backward pass&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The gradient of a layer with respect to the layer input (and layer parameters) can only be calculated given the gradient of the layer output (input gradient of a subsequent layer) and input to the layer (output of the previous layer). This again can be parallelized over a layer and minibatch but is sequential from higher layers to earlier layers. Moreover, since the backward pass relies on the outputs of the forward pass all intermediate layer outputs of the forward pass have to be cached for the backward pass which results in a high (GPU) memory usage.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Forward and backward pass take most of the time&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;So, these two steps take a long time for 1 training iteration, and (depending on your network) high GPU memory usage. But you should read and understand the backpropagation algorithm that basically explains everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, to train a network from scratch, in general, takes lots of iterations because especially in the earlier layers training the parameters is based on gradients that are affected by lots of previous layers, which can result in noisy updates, etc., that do not always push the network parameters in the right direction directly. In contrast, e.g. fine-tuning a pre-trained network on some new task can for example already be done with much less training iterations.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="2444" LastEditDate="2020-05-19T19:55:53.987" LastActivityDate="2020-05-19T19:55:53.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8151" PostTypeId="2" ParentId="8105" CreationDate="2018-09-26T19:58:40.653" Score="-1" Body="&lt;p&gt;Has a data set been chosen? There's a wide variety of data out there (see &lt;a href=&quot;https://www.kaggle.com/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://gengo.ai/datasets/the-50-best-free-datasets-for-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, or &lt;a href=&quot;https://www.forbes.com/sites/bernardmarr/2018/02/26/big-data-and-ai-30-amazing-and-free-public-data-sources-for-2018/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;).  Analytical possibilities are largely determined by the available data, so choosing your data would probably help to clarify your objectives.&lt;/p&gt;&#xA;" OwnerUserId="15223" LastActivityDate="2018-09-26T19:58:40.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8152" PostTypeId="2" ParentId="5414" CreationDate="2018-09-27T06:08:20.707" Score="2" Body="&lt;p&gt;I haven't seen any dataset where some standard models worked and neural networks utterly failed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For columnar data (e.g. Excel files / database dumps / CSV files) which contain structured data usually tree-based models like random forests and gradient boosting work better, but neural networks are also usually way better than random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you demand other things, e.g. explanations for the decision then &lt;a href=&quot;https://datascience.stackexchange.com/q/9818/8820&quot;&gt;Bayesian models might give you an easier time&lt;/a&gt;. Or for baselines/simple implementations linear models. Or for real time applications...&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-09-27T06:08:20.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8154" PostTypeId="2" ParentId="35" CreationDate="2018-09-27T10:57:41.663" Score="2" Body="&lt;p&gt;In simple words, &lt;strong&gt;Artificial intelligence&lt;/strong&gt; is a field of science that is trying to mimic humans or other animals behavior. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Machine Learning&lt;/strong&gt; is one of the key tool/technology behind Artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OkLbE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OkLbE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18563" LastEditorUserId="18563" LastEditDate="2018-09-27T18:33:35.137" LastActivityDate="2018-09-27T18:33:35.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8156" PostTypeId="2" ParentId="7966" CreationDate="2018-09-27T13:49:06.163" Score="3" Body="&lt;p&gt;I mostly studied HMMs and such models are called Infinite HMMs in that specific domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that what you are looking for is called Infinite Neural Networks. Not having access to scientific publications, I cannot really refer any work here. However, I found this GitHub repository: &lt;a href=&quot;https://github.com/kutoga/going_deeper&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kutoga/going_deeper&lt;/a&gt; that provides some implementation and a document with multiple references.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2018-09-27T13:49:06.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8159" PostTypeId="2" ParentId="35" CreationDate="2018-09-27T20:32:58.760" Score="0" Body="&lt;p&gt;First of all, I encountered the term MachineLearning much more in my Business Intelligence classes than in my AI classes.&lt;/p&gt;&#xA;&lt;p&gt;My AI Professor Rolf Pfeifer would have put it that way: (after having a long speech about what intelligence is, how it can be defined, different types of intelligence, etc.). ML is more static and &amp;quot;dumb&amp;quot;, unaware of its physical environment and not made to interact with it, or only on an abstract basis. AI has a certain awareness of its environment and interacts with it autonomously, making thereby autonomous decisions with feedback loops.&#xA;From that point of view, Ugnes Answer would be probably the closest.&#xA;Besides that, of course, ML is a subset of AI.&lt;/p&gt;&#xA;&lt;p&gt;Machine Learning is not real intelligence (imho), it's mostly human intelligence reflected in logical algorithms, and as my Business Intelligence Prof would put it: about data and its analysis. Machine Learning has a lot of supervised algorithms which actually do need humans to support the learning process by telling what's right and what's wrong, so they're not independent. And once they're applied, algorithms are mostly static until humans readjust them.&#xA;In ML you mostly have black boxes designs and the main aspect is data. Data comes in, Data gets analyzed (&amp;quot;Intelligently&amp;quot;), Data goes out, and Learning most times applies to a pre-implementation/Learning fase. In most cases ML doesn't care about the environment a machine is in, it's about data.&lt;/p&gt;&#xA;&lt;p&gt;AI instead is about mimicking human or animal intelligence. Following my Prof's approach, AI is not necessarily about self-consciousness but about interaction with the environment, so to build AI you need to give the machine sensors to perceive the environment, a sort of intelligence able to keep on learning, and elements to interact with the environment (arms, etc.). The interaction should happen in an autonomous way and ideally, as in humans, learning should be an autonomous, ongoing process.&lt;/p&gt;&#xA;&lt;p&gt;So a drone that scans fields in a logical scheme for colour patterns to find weeds within crops would be more ML. Especially if the data is later analyzed and verified by humans or the algorithm used is that a static algorithm with built-in &amp;quot;intelligence&amp;quot; but not capable of rearranging or adapting to its environment.&#xA;A drone that flies autonomously, charges itself up when the battery's down, scans for weeds, learns to detect unknown ones and rips them out by itself and brings them back for verification, would be AI...&lt;/p&gt;&#xA;" OwnerUserId="15332" LastEditorUserId="2444" LastEditDate="2021-04-01T01:14:43.323" LastActivityDate="2021-04-01T01:14:43.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8160" PostTypeId="1" AcceptedAnswerId="8175" CreationDate="2018-09-27T22:17:08.487" Score="0" ViewCount="218" Body="&lt;p&gt;I am currently looking to use a neural network to classify gestures. I have a series of Dx,Dy,Dz readings that represent the differences across the three axes made during the gesture. About 10 movements for each example of the gesture. Basically a 10x3 matrix and then classify the training data into about 15 classes. I plan to use a CNN classifier to do this because, while the time domain is relevant this problem the difference in the movements can be differentiated when presented with as a discrete matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm used to using images with a neural net so I instinctively want to just convert the matrices into a 2D tensor and feed them into a CNN, but I was wondering if there was a better way to do this? For example, I have seen 1D tensors passed to a fully connected neural network for classification which seems like it could be more appropriate for this data input type? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any tips on general architecture would be really appreciated as well!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="18577" LastEditorUserId="18577" LastEditDate="2018-09-28T19:54:52.090" LastActivityDate="2018-09-29T14:32:19.580" Title="Using 3D Points as Inputs to a Neural Net" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8161" PostTypeId="1" CreationDate="2018-09-28T04:29:35.277" Score="3" ViewCount="775" Body="&lt;p&gt;I know the &lt;em&gt;implication&lt;/em&gt; symbol, &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt;, is used for conditions like&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is true, then &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; will be true.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;which can be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A \rightarrow B&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, sometimes the implication symbol is also used in other contexts. For example, if we want to say that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;All &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;'s are &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We could write&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\forall X (A(X) \rightarrow B(X))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't understand why the implication is used here? And if the implication is necessary to use here, then why isn't the implication used in the example written below?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Some &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;'s are &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;'s.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\exists X (A(X) \land B(X))$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="18582" LastEditorUserId="2444" LastEditDate="2021-01-24T14:38:33.317" LastActivityDate="2021-01-24T14:38:33.317" Title="What is the correct way to use the implication in first-order logic?" Tags="&lt;logic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8163" PostTypeId="2" ParentId="8161" CreationDate="2018-09-28T12:08:07.867" Score="5" Body="&lt;p&gt;When we state in English that &lt;em&gt;&quot;All As are Bs&quot;&lt;/em&gt;, this means that we gain information as soon as we observe an A, we can immediately deduce that it &lt;strong&gt;must&lt;/strong&gt; also be a B. These are the kinds of situations where we use an implication. So, this would be written in formal logic as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\forall X \left( A(X) \rightarrow B(X) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we state in English that &lt;em&gt;&quot;Some As are Bs&quot;&lt;/em&gt;, we do not gain any new information just from observing that something is an A, we cannot deduce anything about that A. It &lt;em&gt;might&lt;/em&gt; happen to be one of the As that simultaneously is a B, but it also might happen not to be one of those examples. So, it would be wrong to use an implication here. The only information that the English sentence gives us is that there is at least one thing somewhere that happens to be an A as well as a B, which is written formally as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\exists X (A(X) \land B(X))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Suppose that we would have written the following in logic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\exists X (A(X) \rightarrow B(X))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be translated to English as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There exists some &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; such that, &lt;strong&gt;if it is an&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, it is also a &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That bolded part is very important there. Note that this logical statement is also true as soon as I find one example &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; that is &lt;strong&gt;not&lt;/strong&gt; an &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. For example, the following statement is true in the real world:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There exists some human &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; such that, if &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; can fly, &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; can also shoot fireballs from his or her hands.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(this is true in the real world, because I can come up with many examples of humans who cannot fly)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-09-28T12:08:07.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8164" PostTypeId="2" ParentId="7617" CreationDate="2018-09-28T20:16:59.007" Score="2" Body="&lt;h2&gt;Image Segmentation with Unsupervised Learning&lt;/h2&gt;&#xA;&lt;p&gt;Deep Learning is now widely used for image classification and segmentation. However, for segmentation, some algorithms are still really effective. For example, they could also be used for the development of self-driving cars.&lt;/p&gt;&#xA;&lt;h3&gt;K-means for image segmentation&lt;/h3&gt;&#xA;&lt;p&gt;When you identify the pixels of an RGB image to vectors in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt;, you can run the &lt;a href=&quot;https://en.wikipedia.org/wiki/K-means_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;classic k-means algorithm&lt;/a&gt; to distinguish objects. Furthermore, you can do superpixel segmentation, by adding to all pixel vectors two components corresponding to their coordinates in the image (so it will be vectors in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^5$&lt;/span&gt;). You can run again a k-means algorithm to segment your image in superpixels. You can read about that &lt;a href=&quot;https://www.researchgate.net/publication/225069465_SLIC_Superpixels_Compared_to_State-of-the-Art_Superpixel_Methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLIC Superpixels Compared to State-of-the-Art Superpixel Methods&lt;/a&gt; (2012), Achanta et al.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Below is an example of the segmentation of picture of a seagull on a roof. On the left, we have the original image. In the middle, 3 clusters. On the right, 12 clusters. If it easily distinguishes the roof from the sky, the seagull is still unclear with 12 centroids.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8rVjS.jpg?s=128&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8rVjS.jpg?s=128&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/v6Iy1.jpg?s=128&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v6Iy1.jpg?s=128&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NtsF5.jpg?s=128&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NtsF5.jpg?s=128&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Similarity graph and normalized cut&lt;/h3&gt;&#xA;&lt;p&gt;The main idea is to build a graph of similarities between pixels and then to cut the graph into subgraphs. First, you need to define a distance between pixels. For example, the colour dissimilarity, that could be   &lt;span class=&quot;math-container&quot;&gt;$d(p_1, p_2) = exp(-\sum{(p_{1,i} - p_{2,i})^2}), i \in (r, g, b)$&lt;/span&gt;. Then, build the graph over the whole image, and divide it iteratively, using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Segmentation-based_object_categorization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Normalized Cut algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Morphological gradients&lt;/h3&gt;&#xA;&lt;p&gt;Here, image segmentation is done by computing the &lt;a href=&quot;https://en.wikipedia.org/wiki/Morphological_gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;morphological gradient&lt;/a&gt; of the image. It is the difference between the dilatation and the erosion of the input image. Erosion and dilatation are equivalent to passing min() and max() filters over all subwindows of a given size.&lt;/p&gt;&#xA;&lt;p&gt;An example, still with the seagull. (Morphological gradient computed on a grayscaled input). The seagull appears more clearly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I9kte.jpg?s=256&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I9kte.jpg?s=256&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/l5CKx.jpg?s=256&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l5CKx.jpg?s=256&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="2444" LastEditDate="2021-01-18T11:41:55.693" LastActivityDate="2021-01-18T11:41:55.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8168" PostTypeId="1" CreationDate="2018-09-29T06:36:21.413" Score="2" ViewCount="604" Body="&lt;p&gt;I recently learned about genetic algorithms and I solved the 8 queens problem using a genetic algorithm, but I don't know how to optimize any functions using a genetic algorithm.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{array}{r}&#xA;\text { maximize } f(x)=\frac{-x^{2}}{10}+3 x \\&#xA;0 \leq x \leq 32&#xA;\end{array}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want a guide on how to find chromosomes and fitness functions for such a function? And I don't want code.&lt;/p&gt;&#xA;" OwnerUserId="18604" LastEditorUserId="2444" LastEditDate="2020-11-05T12:00:39.277" LastActivityDate="2020-11-05T14:58:09.107" Title="How do I optimize a specific function using a genetic algorithm?" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8171" PostTypeId="2" ParentId="5414" CreationDate="2018-09-29T08:45:54.133" Score="-1" Body="&lt;p&gt;I still havent seen a good program for recommending me good movies, songs and books based on my previous lkes. Youtube recommendation system is shit. Since youtube is owned by google and since youtube has most content and since google is investing in AI a lot I would conclude this problem remains largely unsolved.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-09-29T08:45:54.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8175" PostTypeId="2" ParentId="8160" CreationDate="2018-09-29T14:32:19.580" Score="0" Body="&lt;p&gt;A few thoughts :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;10x3 matrix for each example is really a small amount of data. FCNN could do a good job on that.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As a result, I'm not sure CNN is appropriated. The smallest dimension is 3, so it'll force you to have really small kernels.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have you thought about LSTM? since that your data is sequential, LSTM may be useful. However, I'm not sure they could be really effective on this little amount of data. but it could be nice to try.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17759" LastActivityDate="2018-09-29T14:32:19.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8177" PostTypeId="1" CreationDate="2018-09-29T21:28:09.700" Score="1" ViewCount="67" Body="&lt;p&gt;I am starting a project to predict the generation of urban waste.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found very little information on this topic on the internet.&#xA;I would be very useful advice on how to approach this topic, and what techniques you would use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found academic articles that make predictions with feedforward neural networks. But they seem basic or old.&#xA;I would expect the use of recurrent networks, but all the examples I find on the internet are about climate prediction, or economics. And they are very different topics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be very useful to make the prediction of waste generation based on an existing model, and not something invented by me. On what topic would you recommend that I take as a base?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any advice.&#xA;regards&lt;/p&gt;&#xA;" OwnerUserId="18621" LastActivityDate="2018-10-01T16:37:37.763" Title="predict waste generation" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;prediction&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8181" PostTypeId="1" AcceptedAnswerId="8183" CreationDate="2018-09-30T14:20:21.857" Score="3" ViewCount="543" Body="&lt;p&gt;I am trying to design a neural network on Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of the sigmoid function which has a limited range, I am thinking of using the cube root function which has the following graph:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jSqOd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jSqOd.jpg&quot; alt=&quot;A graph of the cube root function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this suitable?&lt;/p&gt;&#xA;" OwnerUserId="18640" LastActivityDate="2018-09-30T16:08:06.890" Title="Is the cube root function suitable as a n activation function?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8182" PostTypeId="2" ParentId="1458" CreationDate="2018-09-30T15:49:46.340" Score="1" Body="&lt;p&gt;I would completely agree with mindcrime and Cem Kalyoncu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take into account that passive agressiveness for example is more difficult to detect (irony, black humour, sarcasm likewise)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although another head start could be to think out of the box:&#xA;Happens by chance that I have lying around a book concerning violent-free communication. So probably your best start could be to talk with linguists about violence in language and start from there. or just make some reviews about how linguists or psychlogists detect violence in language (susprise: It's probably quite complex)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless: I don't think you need real AI, a blacklist of words and expressions together with some pattern detection for expressions could be precise enough for the beginning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for all the expressions, words, etc. you could add a bayesian network for the learning part, which works with probablities (like e.g some email spam filters)&#xA;Search for example for &quot;Naive Bayes spam filtering&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should be pretty much enough to have a good start, so in strict sense, you don't need real AI here, just BusinessIntelligence and probability calculations.&lt;/p&gt;&#xA;" OwnerUserId="15332" LastEditorUserId="15332" LastEditDate="2018-09-30T15:59:31.110" LastActivityDate="2018-09-30T15:59:31.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8183" PostTypeId="2" ParentId="8181" CreationDate="2018-09-30T15:52:47.403" Score="3" Body="&lt;p&gt;There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function#Comparison_of_activation_functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;a few traits that you want the activation function to have&lt;/a&gt;, and cube roots rate as OK-ish:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Nonlinear&lt;/em&gt; – check.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Continuously&lt;/em&gt; differentiable – no. There is a problem at &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;. Unlike other discontinuous functions like ReLU, although the gradient can be calculated near zero, it can be arbitrarily high as you approach &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$\frac{d}{dx}x^{\frac{1}{3}} = \frac{1}{3x^{\frac{2}{3}}}$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Range considerations&lt;/em&gt; – limited range functions are more stable, large/infinite range functions are more efficient. You may need to reduce learning rates compared to e.g. tanh.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Monotonic&lt;/em&gt; – check.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Monotonic derivative&lt;/em&gt; - no.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Approximates identity near the origin&lt;/em&gt; – no, the approximation is bad near the origin.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you look through the list of current, successful activation functions, you will see a few that also fail to provide one or more desirable traits, yet are still used routinely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would worry about the high gradients near &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;, but other than that I think the function could work ok. It may sometimes be unstable during learning, as small changes near zero will result in large changes to output. You might be able to work around the high gradients simply in practice, by clipping them. If the raw calculation returns value greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (or less than &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;) then treat the gradient as if it were &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;) for the rest of backpropagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way of finding out if the function is competitive with other more standard activation functions is to try it on some standard data sets and make comparisons.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-09-30T16:08:06.890" LastActivityDate="2018-09-30T16:08:06.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8184" PostTypeId="1" CreationDate="2018-09-30T16:15:45.267" Score="1" ViewCount="58" Body="&lt;p&gt;How do Support Vector Machines (SVMs) differentiate between a glass and a bottle or between a malignant and a benign tumor when it dealing with it for the first time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What will be the analysis mechanism involved in this?&lt;/p&gt;&#xA;" OwnerUserId="18643" LastEditorUserId="3726" LastEditDate="2018-10-01T13:49:06.093" LastActivityDate="2018-10-31T15:00:54.777" Title="How does an svm work? How does it perform comparisons between malignant and benign tumor" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8186" PostTypeId="2" ParentId="4460" CreationDate="2018-09-30T17:13:47.013" Score="1" Body="&lt;p&gt;During my master thesis I struggled with this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In strict sense, a real AI would be able to say, my sensors (or some of them) are gathering data i cannot understand (yet), and would ideally have a routine what to do whith such cases and ideally one day, even understand that data. But it would most probably now, if that is happening outside in the real world, or if it's due to damaged or wrong sensors.&#xA;But that broadly depends how it was programmed, so your use case would have to be further defined: what specific AI are we talking about, what can it do, what not, how is it built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is actually able to reproduce&lt;br&gt;&#xA;a) a subset of an adult's intelligence (speach, face recognition, etc.) but just on a case by case basis&#xA;b) or, if you seak to build a full grown intelligence, it has to go thorugh the development stages as humans (eyesight, voice, imitation, learning, etc.)&lt;br&gt;&#xA;So AI is still in babysteps and is not even able to reproduce let's say an adolescent behaviour, or to reproduce the human evolution of a baby to an adolescent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, after a long speach, to come back to your original question: At the very moment an AI is not able to decide by itself, if it has no method to analize the data, if the data analized are too random, or if one of it's sensors is giving random input.&#xA;Perhaps in some years there are better methods, but at the moment, machines, as babys or childs, need adults hands to teach and show them what to do in what case and how to difference each use-case. Which often means that they would need to add new modules to the AI, which, nowadays is still far away from building anything we could really define as &quot;intelligent&quot; or even &quot;artificially intelligent&quot; (PD: Intelligence here means by ways much more than plain rationality)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I deeply recommend some books of Andy Clark concerning filosofical issues and AI:&#xA;e.g. &quot;Being there: Putting Brain, Body and World together&quot;. Andy Clark is a filosofer which takes a deeper look at AI, the human brain, cognition sciences, filosofy, psychology, sociology, etc.&lt;/p&gt;&#xA;" OwnerUserId="15332" LastEditorUserId="15332" LastEditDate="2018-09-30T17:27:37.563" LastActivityDate="2018-09-30T17:27:37.563" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8188" PostTypeId="1" AcceptedAnswerId="8191" CreationDate="2018-09-30T17:52:52.947" Score="5" ViewCount="4781" Body="&lt;p&gt;I made my first neural net in C++ without any libraries. It was a net to recognize numbers from the MNIST dataset. In a 784 - 784 - 10 net with sigmoid function and 5 epochs with every 60000 samples, it took about 2 hours to train. It was probably slow anyways, because I trained it on a laptop and I used classes for Neurons and Layers.&lt;/p&gt;&#xA;&lt;p&gt;To be honest, I've never used TensorFlow, so I wanted to know how the performance of my net would be compared to the same in TensorFlow. Not too specific but just a rough approximation.&lt;/p&gt;&#xA;" OwnerUserId="17103" LastEditorUserId="2444" LastEditDate="2021-02-06T18:17:42.177" LastActivityDate="2021-02-06T18:17:42.177" Title="How fast is TensorFlow compared to self written neural nets?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;implementation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ClosedDate="2021-02-06T18:16:37.753" ContentLicense="CC BY-SA 4.0" />
  <row Id="8189" PostTypeId="2" ParentId="8188" CreationDate="2018-09-30T18:08:04.627" Score="3" Body="&lt;p&gt;A lot. There are all these optimizations that we might not have thought of like combining layers, functions, etc. I am a pytorch guy though, its clean and doesn't get in your way like tensorflow does.&lt;/p&gt;&#xA;" OwnerUserId="18646" LastActivityDate="2018-09-30T18:08:04.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8190" PostTypeId="1" CreationDate="2018-09-30T18:55:45.547" Score="13" ViewCount="8491" Body="&lt;p&gt;I was able to find &lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper on LSTM&lt;/a&gt;, but I was not able to find the paper that introduced &amp;quot;vanilla&amp;quot; RNNs. Where can I find it?&lt;/p&gt;&#xA;" OwnerUserId="18649" LastEditorUserId="2444" LastEditDate="2021-01-18T23:19:28.433" LastActivityDate="2021-01-18T23:30:20.397" Title="Where can I find the original paper that introduced RNNs?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;reference-request&gt;&lt;papers&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8191" PostTypeId="2" ParentId="8188" CreationDate="2018-09-30T20:49:15.397" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I wanted to know how the performance of my net would be compared to the same in Tensor Flow. Not to specific but just a rough aproximation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is very hard to answer in specific terms because &lt;a href=&quot;http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/&quot; rel=&quot;noreferrer&quot;&gt;benchmarking is very hard and is often wrong&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main point of TensorFlow as I see it is to make it easier for you to use a GPU and further allows you to use a large supply of programs written in Python/JavaScript that still give C++ level performance.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How fast is TensorFlow compared to self written neural nets?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is answering the general question of using TensorFlow/PyTorch vs a custom solution, rather than your specific question of how much of a speed up you'd get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was a relatively recent MIT paper, &lt;a href=&quot;https://people.csail.mit.edu/tzumao/gradient_halide/&quot; rel=&quot;noreferrer&quot;&gt;Differentiable Programming for Image Processing and Deep Learning in Halide&lt;/a&gt; trying to discuss the performance vs flexibility vs time spent coding a solution of 3 different languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically they compared a solution in their language Halide vs PyTorch vs CUDA.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider the following example. A recent neural network-based&#xA;  image processing approximation algorithm was built around a new&#xA;  “bilateral slicing” layer based on the bilateral grid [Chen et al. 2007; Gharbi et al. 2017]. At the time it was published, neither PyTorch&#xA;  nor TensorFlow was even capable of practically expressing this&#xA;  computation. As a result, the authors had to define an entirely&#xA;  new operator, written by hand in about 100 lines of CUDA for the&#xA;  forward pass and 200 lines more for its manually-derived gradient&#xA;  (Fig. 2, right). This was a sizeable programming task which took&#xA;  significant time and expertise. While new operations—added in just&#xA;  the last six months before the submission of this paper—now make&#xA;  it possible to implement this operation in 42 lines of PyTorch, this&#xA;  yields less than 1/3rd the performance on small inputs and runs&#xA;  out of memory on realistically-sized images (Fig. 2, middle). The&#xA;  challenge of efficiently deriving and computing gradients for custom&#xA;  nodes remains a serious obstacle to deep learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So in general you'll probably get faster performance with TensorFlow/PyTorch than a custom C++ implementation, but for specific cases if you have CUDA knowledge on top of C++ then you will be able to write more performant programs.&lt;/p&gt;&#xA;" OwnerUserId="3726" LastEditorUserId="3726" LastEditDate="2018-10-01T09:51:46.657" LastActivityDate="2018-10-01T09:51:46.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8192" PostTypeId="2" ParentId="6161" CreationDate="2018-09-30T20:59:39.163" Score="1" Body="&lt;p&gt;I would counter this question with another one:&#xA;Is rationality all we strive for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence is a waste ground, and we have not even defined which intelligence we're talking about: Emotional Intelligence (empathy), Rational/Intellectual Intelligence (IQ), artistic intelligence, etc. etc. etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something which demarks us as humans is that sometimes we're able to deal with to counteracting intelligences. Logic sometimes dictates one thing, but logic is bound to what parameters are taken into account, and mostly it's about quantifying. an intelligence which does qualitative logic behaves sometimes different, and suggests sometimes other actions.&#xA;As humans we weight and counterweight these factors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Politics is a good example. In most countries there are 2 or 3 rational approaches to problem solving (to make it easy let's call them left, center, right - though I know in fact it's more positions and in an at least two dimensional space, not linear).&#xA;But what I mean is all those positions are rational, nevertheless it's other types of intelligence who define their the way they use their ratio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the end this fourth step is just a continuity and I assume that with time theoretically at least the shift should be away from blue collar jobs to more white collar jobs until there are no blue collar jobs.&#xA;So far the theory I guess, but then again? Would this be smart? Or intelligent? A world where machines make all the dirty work and we're just supervisors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thought to this is we will still need mechanics, probably with more technology always more, just that their focus will shift. 20 years ago mechanics were teached analogue technology, nowadays they learn more digital technology (best example would be car mechanics), but there you see: They're still needed, we also need people with analogue knowledge to mantain older systems which cannot or will not be converted to digital era. And I think that's OK!&#xA;So from my point of view, nobody has to fear to loose it's job, but instead embrace, that when you keep your knowledge up to date, there will always be a job for you.&#xA;Nowadays for example IT-professionals who are able to code older computer languages are better paid than ever, because their scarce...&lt;/p&gt;&#xA;" OwnerUserId="15332" LastActivityDate="2018-09-30T20:59:39.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8193" PostTypeId="1" CreationDate="2018-09-30T21:25:40.257" Score="1" ViewCount="178" Body="&lt;p&gt;To me it seems to be ill defined. Partially because of absence of knowledge which points are to be considered outliers in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem which I have in mind is &quot;bad market data&quot; detection. For example if a financial data provider is good only most of the time, but about 7-10% of data do not make any sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The action space is binary: either take an observation or reject it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure about the reward, because the observations would be fed into an algorithm as inputs and the outputs of the algo would be outliers themselves. So the outliers detection should prevent outputs of the algorithm going rouge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is necessary to add that if we are talking about the market data (stocks, indices, fx), there's no guarantee that the distributions are stationary and there might be trends and jumps. If a supervised classifier is trainer based on historical data, how and how often should it be adjusted to be able to cope with different modes of the data.&lt;/p&gt;&#xA;" OwnerUserId="18653" LastEditorUserId="18653" LastEditDate="2018-10-01T14:50:02.347" LastActivityDate="2018-10-01T15:25:42.310" Title="Is it possible to state an outliers detection problem as a reinforcement learning problem?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8194" PostTypeId="1" CreationDate="2018-09-30T23:47:00.477" Score="4" ViewCount="1351" Body="&lt;p&gt;In the circumstances of two perfect AI's playing each other, will white have an inherent advantage? Or can black always play for a stalemate by countering every white strategy?&lt;/p&gt;&#xA;" OwnerUserId="18656" LastEditorUserId="10135" LastEditDate="2018-10-18T08:32:19.143" LastActivityDate="2018-10-18T08:32:19.143" Title="If two perfect chess AI's played each other, would it always be a stalemate or would white win for an inherent first-move advantage?" Tags="&lt;game-theory&gt;&lt;chess&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8195" PostTypeId="2" ParentId="8194" CreationDate="2018-10-01T03:40:03.077" Score="0" Body="&lt;p&gt;I'm no expert chess player.&#xA;Some specialist chess forums also discussed this issue and there's no clear answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the high amount of possible moves, I would suggest, that most outcome would probably be a remis.&#xA;White would have the advantage of starting, but as black as a perfect AI would know which possible strategies are in play it could block every try for a clear winning strategy but probably always trying to avoid a loss and therefore probably reaching just a remis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basis for this thought is that black as a reaction can always make the choice with fewest loss possibility so probably most strategies of white would stall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now comes the interesting point: I am not quite sure how these AIs would look like. They obviously would have to lean on a strategy to choose between all the possible moves.&#xA;A pure statistical best outcome algorithm even then would leave tooo much options open, specially in the beginning. So you would be forced to prefer certain strategy choices, and with this you would clearly implementing a bias (what to do if two moves have the same win/loss outcome?). This would make the experiment more random or human and thus no outcome could be predicted.&#xA;So if both AIs are perfect (and thus identical) the outcome is most probably a draw, but I assume that statistically some games would be won/loss...&#xA;I think...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I just read about googles AlphaZero (a AI which has taught itself Chess, Go and Shogi) and excells in most games against other AIs. But apparently by precalculating not so much possible outcomes... So it could be that there still is no &quot;absolute perfect AI&quot; for this game(s)&lt;/p&gt;&#xA;" OwnerUserId="15332" LastEditorUserId="15332" LastEditDate="2018-10-02T17:34:24.140" LastActivityDate="2018-10-02T17:34:24.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8196" PostTypeId="1" CreationDate="2018-10-01T05:30:04.247" Score="0" ViewCount="132" Body="&lt;p&gt;I'm currently working on a tumour detection project using Dicom images. As I'm a beginner, I don't know how to segment each part in the image and give each segment a new different colour. I'm using the Watershed segmentation algorithm in Matlab, which only segments the part of interest (the ones that have tumour).&lt;/p&gt;&#xA;" OwnerUserId="18661" LastEditorUserId="2444" LastEditDate="2021-01-27T00:23:17.597" LastActivityDate="2021-01-27T00:23:17.597" Title="How to give each segment a new colour in image segmentation?" Tags="&lt;image-recognition&gt;&lt;image-segmentation&gt;&lt;matlab&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-27T00:22:45.970" ContentLicense="CC BY-SA 4.0" />
  <row Id="8197" PostTypeId="2" ParentId="8196" CreationDate="2018-10-01T06:53:25.823" Score="2" Body="&lt;p&gt;Deep learning based image segmentation is basically pixel-wise image classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So instead of predicting a &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; dimensional vector &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; corresponds to the number of class labels, you predict a tensor &lt;span class=&quot;math-container&quot;&gt;$x\in\mathbb{R}^{h\times w \times C}$&lt;/span&gt; of the same height and width dimensions as your input image, and with &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; channels which correspond to the different classes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &quot;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-24574-4_28&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net: Convolutional Networks for Biomedical Image Segmentation&lt;/a&gt;&quot; provides a good overview of the current methods.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2018-10-01T06:53:25.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8198" PostTypeId="2" ParentId="8194" CreationDate="2018-10-01T07:17:19.527" Score="7" Body="&lt;p&gt;This relates to the concept of &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;noreferrer&quot;&gt;solved games&lt;/a&gt;&quot;. In general, two player turn-based games with perfect information - of which chess is an example - &lt;a href=&quot;https://en.wikipedia.org/wiki/First-player_and_second-player_win&quot; rel=&quot;noreferrer&quot;&gt;can result in all three possible outcomes&lt;/a&gt;: a forced win for white, a forced win for black, or a forced draw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short, although unsatisfactory answer is that &lt;a href=&quot;https://en.wikipedia.org/wiki/Solving_chess&quot; rel=&quot;noreferrer&quot;&gt;chess is not solved, and it is not clear whether it can be&lt;/a&gt;. There is generally thought to be an advantage to white for the first move, so &lt;em&gt;likely&lt;/em&gt; results are considered to be a forced win for white, or a forced draw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No current AI attempts to &quot;solve&quot; chess, although some of the techniques such as MCTS might be adapted &lt;em&gt;theoretically&lt;/em&gt; to find a solution, the available computing power to run that search to completion from the start positions is too low by a few orders of magnitude.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-01T07:28:08.403" LastActivityDate="2018-10-01T07:28:08.403" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8201" PostTypeId="2" ParentId="8193" CreationDate="2018-10-01T12:03:31.503" Score="1" Body="&lt;p&gt;It is quite often &lt;em&gt;possible&lt;/em&gt; to frame a problem as a Reinforcement Learning (RL) problem at some level. However, this may turn out to be for no benefit, or a net cost towards solving the problem. Casting parameter or hyperparameter searches as RL can be adding a layer of complexity and reduce efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One key thing to bear in mind is that any classification or regression that occurs within a RL framework will end up using effectively the same models and approaches that could solve the same problems &lt;em&gt;directly&lt;/em&gt;. These models would either appear directly as the function approximators in RL that implement policies or value functions, or they would be an implied part of them. If you have labeled data for classification - even delayed until some time after you collect data, then you are usually going to be better off using supervised learning directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For hyperparameter searches (e.g. cutoffs for anomaly detection) then you may not need labelled data, but just need a good way to test the model offline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first point at which supervised learning or classic anomaly detection might fail for you is if you never receive any feedback about individual records, only a measure of overall performance. In other words, if you can measure &lt;em&gt;consequences&lt;/em&gt; of good performance, but never measure or check  &lt;em&gt;correctness&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your characterisation&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;about 7-10% of data do not make any sense.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;does not appear to fit that. It looks like you could detect this, maybe manually labelling a few thousand records, and train a classifier using supervised learning techniques. That is likely a much better use of your time than trying to restructure the problem at a higher level and trusting a trail-and-error approach to discover the same rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting that to one side, assuming you do have a problem where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;data to be classified is arriving as a stream, and needs to be processed online, item by item or in small batches&lt;/li&gt;&#xA;&lt;li&gt;you have reason to think that an accept/reject stage before processing further would be useful&lt;/li&gt;&#xA;&lt;li&gt;you have no way to label training data for accept/reject&lt;/li&gt;&#xA;&lt;li&gt;you have a way to measure performance of the remaining system after the accept/reject phase&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then you could use RL to frame the accept/reject phase as an action. There are some challenges there, but essentially you would use RL along with measurement feedback to sample errors or gradients - typically using TD Error or policy gradients. This could wrap almost any model that does classification or anomaly detection etc, provided it could be trained using those gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From comments, if the underlying distributions for accept/reject are non-stationary, this may point you more towards a RL solution. However, that may come with a cost to performance - you will need to balance exploration rate (which will reduce the performance of the model against stationary data) versus speed of learning new distributions. This is a problem for all online learners; the main advantage of a RL approach here is that it will not require generating new labelled data. If you can use a recency-weighted anomaly detection algorithm instead, then you won't need the labeled data either - whether that is better requires testing, personally I'd take a working anomaly detection as the baseline and only use RL if it proved itself better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific items that you turn into states and rewards are not clear to me from the question, and you would need to work on these things carefully. It is possible you will need more than the current data item in order to define state, and that will depend a lot on how the feedback loop works that establishes reward. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-01T15:25:42.310" LastActivityDate="2018-10-01T15:25:42.310" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8207" PostTypeId="2" ParentId="8194" CreationDate="2018-10-01T14:21:32.970" Score="-1" Body="&lt;p&gt;Since both AI know the best possible moves at each step, black would never win as white already knows all games that leads to black's victory and would easily avoid it, but since the black is also a perfect AI, the optimal reponses to white's moves are fixed. So the logical reasoning is these two perfect AI would not make any moves at all if they know their opponent is a perfect AI as well. Both of them know the outcome of the match even before it begins, which my gut feeling says is always a draw. The correct question is does the nth move matter?&lt;/p&gt;&#xA;" OwnerUserId="18646" LastActivityDate="2018-10-01T14:21:32.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8208" PostTypeId="2" ParentId="8184" CreationDate="2018-10-01T14:27:50.990" Score="1" Body="&lt;p&gt;I will try to give you a simplified explanation of how SVMs work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data one works with can be of two types. Either it is very easily separable and there is a clear straight line boundary between data points of different classes. We call such data linearly separable(&lt;a href=&quot;https://static.commonlounge.com/fp/original/zFLIFoonPPTXwNNJXCl09LILU1520492069_kc&quot; rel=&quot;nofollow noreferrer&quot;&gt;image&lt;/a&gt;) or the data points of the classes are mixed in a way that there is no clear straight line boundary that separates them. In other words the data is not linearly separable.(&lt;a href=&quot;http://openclassroom.stanford.edu/MainFolder/courses/MachineLearning/exercises/ex8materials/ex8a_dataonly.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;image&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to make data points linearly separable is to map them into a higher dimension where they would become linearly. If we have a set of data points which are not linearly separable in 2 dimensions, we could map them into 3 dimensions where the data becomes linearly separable(&lt;a href=&quot;https://cdn-images-1.medium.com/max/1600/1*nB1gIxylcGb9YIvwEm6Byg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;like so&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVMs work by converting data from lower to higher dimensions where it is linearly separable and then tries to find the boundary that separates the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same concept could be applied to images, where the images are converted into a very high dimension in which images of different types can be separated linearly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could mean that the SVM model trained is able to easily distinguish between a bottle and a glass or different tumour types but that would depend entirely on the data used to train the model. The problem arises when the distribution of the training data and testing data is completely different. So for example, you trained the SVM to distinguish between glasses and cups of a certain shape and colour and while testing you show it glasses of completely different shapes and colours, the SVM will not be able to distinguish between them.&lt;/p&gt;&#xA;" OwnerUserId="16569" LastActivityDate="2018-10-01T14:27:50.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8210" PostTypeId="2" ParentId="8177" CreationDate="2018-10-01T16:37:37.763" Score="2" Body="&lt;p&gt;It is correct that climate and economic models are distinct from waste models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A memory based model is the correct approach because the time domain is key in prediction based on existing trend data.  However, the RNN is not a practical production model with others that have proven more productive.  Although &lt;a href=&quot;https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c74ce0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; is trendy and not accurate in every technical respect, it introduces the more effective models and their relationship to RNNs.  If you study the article and the terms it introduces on the web, the right solution and possibly an open source project that can get you started can be found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Basis for Prediction of Urban Waste&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether modelling solid or liquid waste, the following model is recommended by two consultants to several large European municipal clients.  All the variables and distributions are time dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v(t) = n(t) \; d\big(I(t), A(t), t\big) \; s(t) \; r(\vec{x(t)}, t)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is the waste rate in metric tons per week&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the population of the collection region&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the average demographically based disposal rate in Kg per person &amp;ndash; Income and age affect consumption and the waste products of consumption, affecting disposal.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is the distribution of income&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the distribution of age&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the dimensionless seasonal factor&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the dimensionless regional factor &amp;ndash; The granularity of regions depends on variance considerations.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; is a vector of non-demographic factors that are likely to correlate with regional waste rates, such as number of university students, size of recycling receptacles, or number of fast food restaurants&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although some of the above factors can be combined in a single training process, there are advantages of leaving them as separate factors.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Demographics is measured at the municipal level and has its own model that can be trained based on immigration, emigration, births, deaths, and aging.  Proven models exist for this already.&lt;/li&gt;&#xA;&lt;li&gt;Separation of training models allows for component wise system development and the understanding of relationships in data that would remain opaque if training was aggregated.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Each of the distributions can be obtained from municipal data. Labeled data for &lt;span class=&quot;math-container&quot;&gt;$n, d, s,$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; can be measured by implementing the appropriate municipal data collection processes.  If it is already available, that would be unusual, and cleaning the data (data sanitation) would be an important consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once labeled data is available for training and the artificial network design is selected, training can occur, followed by testing, followed by validation.  As the system is deployed, care would need to be taken to continuously refine the artificial network hyper-parameters for monthly, quarterly, or annual training and continue to improve data collection accuracy and reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;High accuracy and reliability in such prediction could be expected for short term.  Longer term prediction may have considerably lower accuracy and reliability because of the higher probably of the occurrence of new types of social and economic shift around which the model could not have been advanced trained.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-10-01T16:37:37.763" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8212" PostTypeId="1" CreationDate="2018-10-02T07:15:40.557" Score="2" ViewCount="618" Body="&lt;p&gt;I recently watched the video on Proximal Policy Optimization (PPO). Now, I want to upgrade my actor-critic algorithm written in PyTorch with PPO, but I'am not sure how the new parameters / thetas are calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proximal Policy Optimization Algorithms&lt;/a&gt; (at page 5), the pseudocode of the PPO algorithm is shown:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bVkQH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bVkQH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says to run &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta_{\text{old}}}$&lt;/span&gt;, compute advantage estimates and optimize the objective. But how can we calculate &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt; for the objective ratio, since we have not updated the &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta_{\text{old}}}$&lt;/span&gt; yet?&lt;/p&gt;&#xA;" OwnerUserId="18696" LastEditorUserId="2444" LastEditDate="2019-02-16T02:38:55.663" LastActivityDate="2019-02-16T02:38:55.663" Title="How do I calculate the policy in the Proximal Policy Optimization algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8215" PostTypeId="1" CreationDate="2018-10-02T08:07:45.853" Score="1" ViewCount="837" Body="&lt;p&gt;(Cross-posting here from the data science stack exchange, as my question didn't get any replies. I hope it's okay!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been playing around with YOLOv3 and obtaining some good results on the ~20 custom classes I trained. However, one or two classes look like they can use some additional training data (not a lot, say about 10% more data), which I can provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient way to train my model now? Do I need to start training from scratch? Can I just throw in my additional data (with the appropriate changes to the config files etc.) and run the training based on the weight matrix I already acquired, but for a small number of iterations? (1000?) Or is this more like a transfer learning problem now?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for all tips!&lt;/p&gt;&#xA;" OwnerUserId="18699" LastActivityDate="2021-04-22T11:07:08.210" Title="Add training data to YOLO post-training" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8217" PostTypeId="1" CreationDate="2018-10-02T08:42:29.030" Score="2" ViewCount="105" Body="&lt;p&gt;I implemented a LSTM neural network in Pytorch. It worked but I want to know if it worked the way I guessed how it worked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say there's a 2-layer LSTM network with 10 units in each layer.&#xA;The inputs are some sequence data Xt1, Xt2, Xt3, Xt4, Xt5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when the inputs are entered into the network, Xt1 will be thrown into the network first and be connected to every unit in the first layer. And it will generate 10 hidden states/10 memory cell values/10 outputs. Then the 10 hidden states, 10 memory cell values and Xt2 will be connected to the 10 units again, and generate another 10 hidden states/10 memory cell values/10 outputs and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After all 5 Xt's are entered into the network, the 10 outputs from Xt5 from the first layer are then used as the inputs for the second layer. The other outputs from Xt1 to Xt4 are not used. And the the 10 outputs will be entered into the second layer one by one again. So the first from the 10 will be connected to every unit in the second layer and generate 10 hidden states/10 memory cell values/10 outputs. The 10 memory cell values/10 hidden states and the second value from the 10 will be connected and so forth?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After all these are done, only the final 10 outputs from the layer 2 will be used. Is this how the LSTM network works? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="18268" LastActivityDate="2018-10-02T08:42:29.030" Title="Structure of a multilayered LSTM neural network?" Tags="&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8219" PostTypeId="1" AcceptedAnswerId="8346" CreationDate="2018-10-02T11:15:13.087" Score="1" ViewCount="409" Body="&lt;p&gt;&lt;strong&gt;The dialog context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing proposed at the end of the description of his famous test, &quot;Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, 'Can machines think?'&quot;&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing effectively challenged the 1641 statement of René Descartes in his &lt;em&gt;Discourse on Method and Meditations on First Philosophy&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;It never happens that [an automaton] arranges its speech in various ways, in order to reply appropriately to everything that may be said in its presence, as even the lowest type of man can do.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Descartes and Turing, when discussing automatons achieving human abilities, shared a single context through which they perceived intelligence.  Those that have been either the actor or the administrator in actual Turing Tests understand the context: &lt;strong&gt;&lt;em&gt;Dialog&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other contexts&lt;sup&gt;2&lt;/sup&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The context of the dialog is distinct from other contexts such as writing a textbook, running a business, or raising children. If you apply the principle of comparing machine and human intelligence to automated vehicles (e.g. self-driving cars), an entirely different context becomes immediately apparent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a brain be intelligent without a body? More generally, does intelligence require a context?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Chapter 1 (&quot;Imitation Game&quot;) of &lt;em&gt;Computing Machinery and Intelligence&lt;/em&gt;, 1951.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://www.intelltheory.com/mitheory.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiple Intelligences Theory&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-03-14T23:53:55.503" LastActivityDate="2020-03-15T00:08:18.533" Title="Can a brain be intelligent without a body?" Tags="&lt;philosophy&gt;&lt;cognitive-science&gt;&lt;turing-test&gt;&lt;intelligence&gt;&lt;embodied-cognition&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ClosedDate="2020-03-15T00:10:49.790" ContentLicense="CC BY-SA 4.0" />
  <row Id="8220" PostTypeId="1" AcceptedAnswerId="8266" CreationDate="2018-10-02T11:54:03.637" Score="2" ViewCount="661" Body="&lt;p&gt;I will be undertaking a project over the next year to create a self learning AI to play a racing game, currently the game will be Mario Kart 64.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions which will hopefully help me get started: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What aspects of AI would be most applicable to creating a self learning game AI for a racing game (Q-Learning, NEAT etc)&lt;/li&gt;&#xA;&lt;li&gt;Could a ANN or NEAT that has learned to play Mario Kart 64 be used to learn to play another racing game? &lt;/li&gt;&#xA;&lt;li&gt;What books/material should i read up on to undertake this project? &lt;/li&gt;&#xA;&lt;li&gt;What other considerations should i take throughout this project? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help! &lt;/p&gt;&#xA;" OwnerUserId="18209" LastActivityDate="2018-10-05T08:54:33.797" Title="Creating a self learning Mario Kart game AI?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;python&gt;&lt;autonomous-vehicles&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8221" PostTypeId="1" CreationDate="2018-10-02T12:43:52.323" Score="1" ViewCount="51" Body="&lt;p&gt;I was going through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ontology_components&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page on ontology components&lt;/a&gt; and noticed something that I had been hoping to find, for a long time. In the components' overview it mentioned: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Function terms: complex structures formed from certain relations that can be used in place of an individual term in a statement&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried digging deeper into &lt;strong&gt;function terms&lt;/strong&gt; but couldn't find any other references. Is this a standard component of an ontology? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;References would be nice. Ultimately, I'm looking for something like this in the DBPedia ontology. Would have been nice if I could add &lt;em&gt;ontology&lt;/em&gt; and &lt;em&gt;dbpedia&lt;/em&gt; as tags to the question, but I don't have the requisite reputation.&lt;/p&gt;&#xA;" OwnerUserId="4707" LastEditorUserId="2444" LastEditDate="2019-10-12T12:23:10.973" LastActivityDate="2019-10-12T12:23:10.973" Title="What are &quot;function terms&quot; in the context of an ontology?" Tags="&lt;terminology&gt;&lt;knowledge-representation&gt;&lt;world-knowledge&gt;&lt;ontology&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8223" PostTypeId="1" AcceptedAnswerId="8234" CreationDate="2018-10-02T14:18:40.637" Score="4" ViewCount="147" Body="&lt;p&gt;I've been told this is how I should be preprocessing audio samples, but what information does this method actually give me? What are the alternatives and why shouldn't I use them.&lt;/p&gt;&#xA;" OwnerUserId="18709" LastActivityDate="2018-10-03T05:39:22.023" Title="Why is Short-Time Fourier Transform used for preprocessing audio samples?" Tags="&lt;voice-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8227" PostTypeId="1" AcceptedAnswerId="8229" CreationDate="2018-10-02T17:58:58.710" Score="9" ViewCount="20538" Body="&lt;p&gt;I've been seeing a lot of tutorials on the internet about AI that are mostly done with Python. Apart from these, I've seen C# being used in AI topics, but in things like self-driving cars. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already know C# and I wanted to become more professional in it. However, now that I see that Python is being used a lot, so I'm getting intrigued by it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you recommend Python (or other languages) or should I keep up with C#?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to mention, I'm 14 years old and I have enough time to learn more and it doesn't really matter what I love to do, because, I love coding and AI especially, so, it doesn't really matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you recommend Python, please, tell me which compiler I should use. I don't really know if it has a compiler, but I want to know where I should start.&lt;/p&gt;&#xA;" OwnerUserId="18595" LastEditorUserId="2444" LastEditDate="2020-03-12T14:04:24.760" LastActivityDate="2020-08-02T14:27:18.613" Title="Should I use C# or Python for AI?" Tags="&lt;python&gt;&lt;comparison&gt;&lt;getting-started&gt;&lt;programming-languages&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ClosedDate="2020-03-12T13:40:12.267" ContentLicense="CC BY-SA 4.0" />
  <row Id="8229" PostTypeId="2" ParentId="8227" CreationDate="2018-10-03T02:37:35.137" Score="10" Body="&lt;p&gt;If you're doing deep learning (which I assume you are, if you say you want to learn &quot;AI&quot;), then Python is a MUST. Virtually all the big frameworks are Python wrappers over a C++ core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C# has no real deep learning frameworks. There are a couple such as the Microsoft Cognitive Toolkit, but they are on a completely different level from PyTorch or Tensorflow. No serious ML practitioner would do the majority of their research in a framework like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information, see: &lt;a href=&quot;https://ai.stackexchange.com/questions/3494/why-is-python-the-most-popular-language-in-the-ai-field/3503#3503&quot;&gt;Why is Python the most popular language in the AI field?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16035" LastActivityDate="2018-10-03T02:37:35.137" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8230" PostTypeId="2" ParentId="8212" CreationDate="2018-10-03T02:51:37.303" Score="3" Body="&lt;p&gt;You're right, the first time you run it the two policies (&lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta old}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt;) will be the same. This means your loss is simply the advantage (since you multiply the the ratio (&lt;span class=&quot;math-container&quot;&gt;$r(\theta)={\pi_\theta(a|s)\over\pi_{\theta old}(a|s)}$&lt;/span&gt;) by the advantage (so &lt;span class=&quot;math-container&quot;&gt;$loss=-r_t(\theta)A_t$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with PPO you run multiple epochs of training on the same data. So after your first update you do the whole thing again (without exploring the environment any more) and this time &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta old}$&lt;/span&gt; is different to &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a great explanation of the algorithm: &lt;a href=&quot;https://stackoverflow.com/questions/46422845/what-is-the-way-to-understand-proximal-policy-optimization-algorithm-in-rl&quot;&gt;https://stackoverflow.com/questions/46422845/what-is-the-way-to-understand-proximal-policy-optimization-algorithm-in-rl&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16035" LastActivityDate="2018-10-03T02:51:37.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8231" PostTypeId="2" ParentId="8219" CreationDate="2018-10-03T02:56:41.630" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can a brain be intelligent without a body?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my opinion, yes, if you give it the right inputs.  The brain is like a machine and its behavior depends on its architecture and the interaction with the environment, whether it is the internet or anything else, so it all boils down to the actual architecture of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence is just an information processing system. A human gets info from his/her eyes, hears, or other sensors, then the brain does info processing and storage. One can potentially replace our sensors with other sensors that acquire the info from the world and send it to the brain.&lt;/p&gt;&#xA;" OwnerUserId="17018" LastEditorUserId="2444" LastEditDate="2020-03-15T00:00:05.190" LastActivityDate="2020-03-15T00:00:05.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8232" PostTypeId="2" ParentId="8194" CreationDate="2018-10-03T03:03:41.157" Score="1" Body="&lt;p&gt;No that doesnt have to be the case because from each machine's point of view the game is different so they will have different possibilities like saying different paths to follow, now something different would be if two machines with the same brain play against the same opponent with same parameters then we can say theboutcome will problably be the same, if they are &quot;programed in a deterministic way&quot;&lt;/p&gt;&#xA;" OwnerUserId="17018" LastActivityDate="2018-10-03T03:03:41.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8233" PostTypeId="2" ParentId="2563" CreationDate="2018-10-03T04:15:39.663" Score="0" Body="&lt;p&gt;I kid you not when I tell you I just started with decoupled networks this morning. But the issue is terribly worded by most computer scientists. The way a decoupled neural interface works is not by sending the error backwards, but by storing it in a form of learned error inside a separate network. So not only do we have our nodes and weights, but also the Synths. The Synths modify the weight tables using past error. So instead of passing error back like the article suggests, it actually is passing it forward in a kind of &quot;hey don't do this.&quot; What's really fun is that to you never have to mix up your thinking to reverse a network for backprop if you're using a decoupled network. The amazing part is that it is actually easier to code this decoupled network, than it is to write a &quot;simple&quot; ANN, even better still is that it functions a lot like an LSTM network without all the programming mumbo-jumbo. You can see for yourself two different forms of the same network from &lt;a href=&quot;https://github.com/CPU-Destraux/Decoupled-Neural-Interface.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code from the demo in the repository is from &lt;a href=&quot;https://towardsdatascience.com/only-numpy-implementing-and-comparing-combination-of-google-brains-decoupled-neural-interfaces-6712e758c1af&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/only-numpy-implementing-and-comparing-combination-of-google-brains-decoupled-neural-interfaces-6712e758c1af&lt;/a&gt; it was a fundamental resource for me, and it may be for you. It also helps you with loss, or cost.&lt;/p&gt;&#xA;" OwnerUserId="3542" LastActivityDate="2018-10-03T04:15:39.663" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8234" PostTypeId="2" ParentId="8223" CreationDate="2018-10-03T05:39:22.023" Score="3" Body="&lt;p&gt;Fourier transform is used to transform audio data to get more information (features).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, raw audio data usually represented by a one-dimensional array, &lt;code&gt;x[n]&lt;/code&gt;, which has a length &lt;code&gt;n&lt;/code&gt; (number of samples). &lt;code&gt;x[i]&lt;/code&gt; is an amplitude value of the i-th sample point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Fourier transform, your audio data will be represented as a two-dimensional array. Now, &lt;code&gt;x[i]&lt;/code&gt; is a not a single value of amplitude, but a list of frequencies which compose original value at the i-th frame (a frame consists of a few samples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the image below (from wikipedia), the red graph is an original value of n samples before transformed, and the blue graph is a transformed value of one frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1OWVu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1OWVu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2018-10-03T05:39:22.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8235" PostTypeId="2" ParentId="8227" CreationDate="2018-10-03T07:03:26.027" Score="5" Body="&lt;p&gt;Corporations, government research, and academia are favoring C, Python, Java, LISP, and R currently.  The trends are not favorable to C# for AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C#'s peak of use was in the 2009 to 2012 range.  By buying GitHub, Microsoft intends to regain some control over development tools and language but has never been particularly successful in either.  Even eclipse is giving way to other open tools or proprietary tools with community versions, and JavaScript and Python are the languages gaining popularity in this decade.  It is not clear whether C# will be very well known as a general purpose programming in ten years. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;C/C++, Java, and JavaScript have stood the test of time.  C/C++ is the language of choice for low level access to dedicated hardware, which is what it was designed by Bell Labs to do.  Java is almost as fast and still very popular, strongly OO, still developing, and with Scala, Groovy, Maven, and Gradle, looking strong for the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python started slow but has seen continuous rapid growth trend for the last two years because of its MATLAB-ish-ness and syntactic clarity.  JavaScript, with its heavy influence from LISP and Scheme from the birth of AI, will likely enter in greater strength as AI reaches the front end and middle tiers of web applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C# doesn't look hopeful for keep pace.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-10-03T07:03:26.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8236" PostTypeId="2" ParentId="8219" CreationDate="2018-10-03T07:41:05.030" Score="2" Body="&lt;p&gt;It depends what you mean by intelligence. A robot that acts has a different sort of intelligence than a neural net that merely maps inputs to outputs. Bit patterns within a robot brain have meaning, whereas the meaning of the inputs and outputs of gain meaning only through the larger system in which humans steer input data to it, and act on the basis of the outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, a system that can act needs a causal model of the world that, at least in part, includes itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a non-embodied system may be intelligent in some useful ways, but its intelligence will be radically different than a human intelligence. That's not necessarily a bad thing: we already have lots of humans, and can produce more fairly cheaply. The most cost-effective AIs are surely not human-like ones. &lt;/p&gt;&#xA;" OwnerUserId="12269" LastActivityDate="2018-10-03T07:41:05.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8238" PostTypeId="1" CreationDate="2018-10-03T09:44:28.433" Score="1" ViewCount="216" Body="&lt;p&gt;Gist: Should I use LISP for a part of the following project. What are the other options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Me and a friend are planning to create a 3D Modelling Agent where a designer can :-&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Specify constrains on how the model can be (Eg: Model cannot be more than 8 inches wide, model has to have 2 holes on top, etc).&lt;/li&gt;&#xA;&lt;li&gt;Provide 2D images for &quot;inspiring&quot; the agent to prefer certain forms.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From what I can fathom, the first part seems to pertain to symbolic AI (Deterministic Constraints Satisfaction) and the second part pertains to Fuzzy AI (some form of 3DGAN???). I have studied FOL and planning systems, although I have never implemented one. I am learning ML and NN on the side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We plan on using the Unity Game Engine since it has a lot of tools related to 3D Environments built in, like Vertex Modelling, a Rendering Pipeline and even ML Agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to my question, should I use LISP for the Symbolic AI part? What about Prolog? Or should I prefer to use user-developed FOL and CSP libraries for C# (Since Unity Game Engine uses C#)?&lt;/p&gt;&#xA;" OwnerUserId="18726" LastActivityDate="2018-10-03T09:44:28.433" Title="Stack for Automatic 3D Mesh Generation" Tags="&lt;neural-networks&gt;&lt;generative-model&gt;&lt;symbolic-computing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8239" PostTypeId="1" CreationDate="2018-10-03T11:06:25.020" Score="1" ViewCount="20" Body="&lt;p&gt;I want to build a model to support decision making in order to propose or not loan insurance to clients. Because sometimes clients asking loan and loan insurance have less chance to have their loan accepted by a bank and sometimes more chances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three actors in the problem: a bank, a loaner applicant (someone who ask for a loan) and a counselor. The counselor studies the loaner application and if it has a good profile it will propose to him loan from banks that fits his profile. Then the application is sent to the bank but the bank could refuse the applicant (based on criteria we don't know).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The counselor has also to decide whether or not he will propose to the loaner applicant a loan insurance. I want to build a model for that decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The risk is that some banks reject loan applicant who accepts a loan insurance and other banks accept more applicants with a loan insurance. But there aren't rules regarding banks since some banks accept or reject applicants with loan insurance according of the profile of the applicants and the type of acquisition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the profile of the applicant and the bank he is applying to can matter in their rejection from banks but all criteria influencing the decision are quite uncertain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though it is a classification problem, in my dataset I don't have a good label for loan insurance proposal. I have a features that says if the insurance was proposed or not and to less than 1% the insurance was proposed. I have a label that says if the clients application for a loan was accepted or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the data I have is former applicants profile - and banks who propose loan according to what the applicant wants - and if they were accepted or not by the bank and if they wanted a loan insurance or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of combining the label with the information but I don't really know how or maybe doing multi-label classification but also I don't really know if it does fit to the problem.&lt;/p&gt;&#xA;" OwnerUserId="18192" LastActivityDate="2018-10-03T11:06:25.020" Title="Merge one label with one information for classification problem or multi-label classification" Tags="&lt;classification&gt;&lt;decision-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8240" PostTypeId="1" AcceptedAnswerId="8247" CreationDate="2018-10-03T11:45:23.277" Score="2" ViewCount="146" Body="&lt;p&gt;I am trying to dissect the paper &lt;a href=&quot;https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, because my math is a little bit rusty, I got a little bit stuck with the proof. Could you provide me with some clarification about proof of the topic?&lt;/p&gt;&#xA;&lt;p&gt;What I understand is that we introduce, instead of the weight vector &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; a scalar &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; (magnitude of original &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;?) and &lt;span class=&quot;math-container&quot;&gt;$\frac{v}{\|v\|}$&lt;/span&gt; (direction of original &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;?) of the vector.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{g} L=\frac{\nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\|\mathbf{v}\|}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\mathbf{v}} L=\frac{g}{\|\mathbf{v}\|} \nabla_{\mathbf{w}} L-\frac{g \nabla_{g} L}{\|\mathbf{v}\|^{2}} \mathbf{v}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I am not really sure about is:&lt;/p&gt;&#xA;&lt;p&gt;If the gradients are noisy (does this mean that in some dimension we have small and in some high curvature or that error noise differs for very similar values of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;?) the value will quickly increase here and effectively limit the speed of descent by decreasing value of (&lt;span class=&quot;math-container&quot;&gt;$\frac{g}{\|v\|}$&lt;/span&gt;). This means that we can choose larger learning rates and it will somehow adjust the effect of the learning rate during the training.&lt;/p&gt;&#xA;&lt;p&gt;And what I completely miss is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\mathbf{v}} L=\frac{g}{\|\mathbf{v}\|} M_{\mathbf{w}} \nabla_{\mathbf{w}} L$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm{M}_{\mathrm{w}}=\mathrm{I}-\frac{\mathrm{w} \mathrm{w}^{\prime}}{\|\mathrm{w}\|^{2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;It should somehow explain the reasoning behind the idea of the final effects. Unfortunately, I don't really understand this part of the paper, I probably lack some knowledge about Linear Algebra.&lt;/p&gt;&#xA;&lt;p&gt;Can you verify that my understanding of the paper is correct?&lt;/p&gt;&#xA;&lt;p&gt;Can you recommend some sources (books/videos) to help me to understand the second part of proof (related to the second set of formulas?)&lt;/p&gt;&#xA;" OwnerUserId="18729" LastEditorUserId="2444" LastEditDate="2020-07-06T23:11:41.587" LastActivityDate="2020-07-06T23:12:15.373" Title="Can you help me understand how weight normalization works?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;papers&gt;&lt;weight-normalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8241" PostTypeId="1" AcceptedAnswerId="8242" CreationDate="2018-10-03T15:27:01.740" Score="2" ViewCount="47" Body="&lt;p&gt;Imagine a system that is trained to manipulate dampers to manage air flow. The training data includes damper state and flow characteristics through a complex system of ducts. The system is then given an objective (e.g. maintain even flow to all outputs) and set loose to manage the dampers. As it performs those functions there are anomalies in the results which the system is able to detect. The algorithm CONTINUES to learn from its own empirical data, the result of implemented damper configurations, and refines its algorithm to improve performance seeking the optimum goal of perfectly even flow at all outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is that kind of learning or AI system called?&lt;/p&gt;&#xA;" OwnerUserId="17061" LastEditorUserId="2444" LastEditDate="2020-11-07T17:24:03.223" LastActivityDate="2020-11-07T17:24:03.223" Title="What is the AI discipline where an algorithm learns from an initial training set, but then refines its learning as it uses that training?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8242" PostTypeId="2" ParentId="8241" CreationDate="2018-10-03T16:37:50.783" Score="4" Body="&lt;p&gt;I believe this can best be done with reinforcement learning via Deep Q Learning. That's where I would start. Steps are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Initialize a Q table.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose an action.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perform the action.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Measure the reward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Update the Q.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A neural net will approximate the Q function. See: &lt;a href=&quot;https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also consider policy gradients, actor critic, and inverse reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="3861" LastActivityDate="2018-10-03T16:37:50.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8243" PostTypeId="1" AcceptedAnswerId="8252" CreationDate="2018-10-03T16:44:19.617" Score="3" ViewCount="3498" Body="&lt;p&gt;Imagine a system that controls dampers in a complex vent system that has an objective to perfectly equalize the output from each vent. The system has sensors for damper position, flow at various locations and at each vent. The system is initially implemented using a rather small data set or even a formulaic algorithm to control the dampers. What if that algorithm were programmed to &quot;try&quot; different configurations of dampers to optimize the air flows, guided broadly by either the initial (weak) training or the formula? The system would try different configurations and learn what improved results, and what worsened results, in an effort to reduce error (differential outflow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is that kind of AI system called? What is that system of learning called? Are there systems that do that currently?&lt;/p&gt;&#xA;" OwnerUserId="17061" LastEditorUserId="2444" LastEditDate="2020-11-07T17:23:07.647" LastActivityDate="2020-11-07T17:23:07.647" Title="What is the name of an AI system that learns by trial and error?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;incremental-learning&gt;&lt;ai-field&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8244" PostTypeId="2" ParentId="8219" CreationDate="2018-10-03T16:47:36.350" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can a brain be intelligent without a body?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you define &quot;intelligence&quot; as &quot;doing the right thing at the right time&quot;, then the statement itself implies some sort of embodied context, whether humanoid, networked or otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a more existential definition where by fact that there are internal workings, or goings on but aren’t apparent in any embodied output then one could argue either way. Akin to the simulated universe theory: either way the outcome is only how we think about it, rather than having an experimental truth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one can refine the question so that the outcome can be used practically then I believe that may be more useful.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="2444" LastEditDate="2020-03-15T00:08:18.533" LastActivityDate="2020-03-15T00:08:18.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8245" PostTypeId="2" ParentId="8243" CreationDate="2018-10-03T16:57:01.800" Score="2" Body="&lt;p&gt;Near solution to your problem definition is reinforcement learning. You can define some reward using the objective function and define some possible state space for the machine and finally solve the problem by reinforcement learning techniques (near to trial and error by learning the preferences).&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2018-10-03T16:57:01.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8246" PostTypeId="2" ParentId="7926" CreationDate="2018-10-03T17:10:38.947" Score="0" Body="&lt;p&gt;Love question. Firstly, as pointed out in other answers Narrow AI today are mostly algorithms following their procedures given some inputs. No need for philosophy here as they are following reproducible steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if your referring to general AI or AI a kin to human level intelligence or better then maybe the question holds some weight. But again as pointed out it would come back to whether you believe you or I indeed have free will?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me I believe free will can be modelled as a sort of entropy. If you look on the macro level things are blurry, agents are making decisions and moving around in a unpredicatable way. On the micro level however, given all the data in one state one could predict the next state shattering the idea of free will. I guess it’s up to you to decide whether this fits with your free wil definition or not.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2018-10-03T17:10:38.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8247" PostTypeId="2" ParentId="8240" CreationDate="2018-10-03T18:03:24.430" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If the gradients are noisy (does this mean that in some dimension we have small and in some high curvature or that error noise differs for very similar values of w?)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Gradients being noisy means that they are &amp;quot;inconsistent&amp;quot; across different epochs / training steps. With that I mean that they'll sometimes point in one direction, later in a different (maybe the opposite) direction, etc., that the gradients at different time steps give inconsistent/conflicting information, that we don't consistently keep following the same direction through gradient descent but keep jumping all over the place. Note that this is &lt;strong&gt;across epochs&lt;/strong&gt;, it's &lt;strong&gt;not across different dimensions within the same epoch&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, we might have a vector of gradients that looks like &lt;span class=&quot;math-container&quot;&gt;$[1, 1, 1, \dots, 1]$&lt;/span&gt; at one time step (i.e., positive gradients in all dimensions), and the next training step get a gradient more like &lt;span class=&quot;math-container&quot;&gt;$[-1, -1, -1, \dots, -1]$&lt;/span&gt; (this example is pretty much the &amp;quot;most extreme&amp;quot; kind of noise you can have, completely conflicting directions, in reality it'd generally be less extreme).&lt;/p&gt;&#xA;&lt;p&gt;Such &amp;quot;noisy gradients&amp;quot; can exist because, in practice, we almost always use &lt;strong&gt;estimates&lt;/strong&gt; of the gradient of our loss function, rather than the &lt;strong&gt;true&lt;/strong&gt; gradients (which we'll generally not even be able to compute). For example, if we have a very large training dataset in a supervised learning setting, we'd ideally use the gradient of the loss function computed across the complete dataset. That tends to be computationally expensive (requires forwards and backwards passes for every single instance in the dataset), so in practice we'll often only use a small minibatch to estimate the gradient; this can result in widely different estimates of the gradient for different minibatches. &lt;em&gt;(note: there can also be different reasons for using minibatches rather than full dataset, such as avoiding overfitting, but that's not too important to consider for this specific question)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;A different example of a setting where we can have noisy gradients is in Deep Reinforcement Learning. There, computing the loss function itself is often rather noisy (in the sense that our own predictions, which tend to still be incorrect during the training phase, are a component of the targets that we're updating towards), so our estimates of the gradients will also be noisy.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Derivation for your second question, about the gradient of the loss with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;We start with the following, from Equation (3) in the paper (this gradient you understand, right?):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\mathbf{v}} L = \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g \nabla_g L}{\vert \vert \mathbf{v} \vert \vert^2} \mathbf{v}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that Equation (3) also gives:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{g} L = \frac{\nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\vert \vert \mathbf{v} \vert \vert}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If we plug that into the previous Equation we get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\nabla_{\mathbf{v}} L &amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g \frac{\nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\vert \vert \mathbf{v} \vert \vert}}{\vert \vert \mathbf{v} \vert \vert^2} \mathbf{v} \\&#xA;&amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g \nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\vert \vert \mathbf{v} \vert \vert^3} \mathbf{v} \\&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L$&lt;/span&gt; term that shows up before the minus can also be isolated in the term after the minus:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\nabla_{\mathbf{v}} L &amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert^2} \mathbf{v} \\&#xA;&amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert} \cdot \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert} \\&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Equation (2) tells us that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} = \frac{g}{\vert \vert \mathbf{v} \vert \vert} \mathbf{v}$&lt;/span&gt;, which means that &lt;span class=&quot;math-container&quot;&gt;$\frac{\mathbf{w}}{g} = \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert}$&lt;/span&gt;. Plugging this into the above leads to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\nabla_{\mathbf{v}} L &amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{w}}{g} \cdot \frac{\mathbf{w}}{g} \\&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Below Equation (2), they also explain that &lt;span class=&quot;math-container&quot;&gt;$\vert \vert \mathbf{w} \vert \vert = g$&lt;/span&gt;, so we can rewrite the above to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\nabla_{\mathbf{v}} L &amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{w}}{\vert \vert \mathbf{w} \vert \vert} \cdot \frac{\mathbf{w}}{\vert \vert \mathbf{w} \vert \vert} \\&#xA;&amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{w} \mathbf{w}'}{\vert \vert \mathbf{w} \vert \vert^2} \\&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In that last step, the &lt;span class=&quot;math-container&quot;&gt;$'$&lt;/span&gt; symbol in &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}'$&lt;/span&gt; denotes that we transpose that vector.&lt;/p&gt;&#xA;&lt;p&gt;Finally, because we have that common &lt;span class=&quot;math-container&quot;&gt;$\frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L$&lt;/span&gt; before and after the minus symbol, we can pull it out, multiplying it once by the identity matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt;, and once by the remaining negated term. This is very similar to how you would simplify something like &lt;span class=&quot;math-container&quot;&gt;$a - ax$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$a (1 - x)$&lt;/span&gt; in &amp;quot;standard&amp;quot; algebra; the identity matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; plays a very similar role here to the number &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in &amp;quot;standard&amp;quot; algebra:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\nabla_{\mathbf{v}} L &amp;amp;= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \left( \mathbf{I} - \frac{\mathbf{w} \mathbf{w}'}{\vert \vert \mathbf{w} \vert \vert^2} \right) \\&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: in all of the above, I kind of neglected to pay attention to order of matrix/vector multiplications and their dimensions. Probably the &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\mathbf{w}} L$&lt;/span&gt; term should by now already have been moved to after those large brackets, instead of being before those brackets. In this case, it should work out fine regardless because &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; as well as &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} \mathbf{w}'$&lt;/span&gt; are square &lt;strong&gt;and&lt;/strong&gt; symmetric matrices. The only resulting difference is in whether you get a row or a column vector out at the end.&lt;/p&gt;&#xA;&lt;p&gt;This can now finally be rewritten as the Equation you mentioned that you didn't understand yet, Equation (4) in the paper.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-07-06T23:12:15.373" LastActivityDate="2020-07-06T23:12:15.373" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8248" PostTypeId="2" ParentId="8243" CreationDate="2018-10-03T20:05:14.147" Score="1" Body="&lt;p&gt;I think any learning algorithm probably uses trial and error and analysis of the results with the ultimate goal of maximizing utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the recent milestones in AI fall under the general umbrella of &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt;, which includes all forms of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt;. Essentially, any learning algorithm is using some form of statistical analysis.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For an umbrella term, I've been using &quot;learning algorithm&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, there is also a venerable history of less capable adaptive systems such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-organizing_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-organizing networks&lt;/a&gt;.  (See also &lt;a href=&quot;https://en.wikipedia.org/wiki/Optimal_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimal control&lt;/a&gt;.)  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-10-03T20:20:49.587" LastActivityDate="2018-10-03T20:20:49.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8249" PostTypeId="2" ParentId="80" CreationDate="2018-10-03T20:53:03.973" Score="0" Body="&lt;p&gt;If you want to understand relativity, read Einstein&lt;sup&gt;1,2&lt;/sup&gt;, not a book about relativity authored by a professor who think's he's got it.  If you want to understand Alan Turing's test for intelligence in the context of human dialog, read Turing.&lt;sup&gt;3&lt;/sup&gt;  Interpretations can be worse than worthless.  They are often misleading.  If the principles seem too thick, read it over again until you get it.&lt;/p&gt;&#xA;&lt;p&gt;In the case of Turing's test for intelligence in the context of human dialog, to understand it fully, the following background is assumed when Turing wrote, which, if you read his 1950 article, will become apparent.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How Turing's completeness theorem responds to Kurt Gödel's second incompleteness theorem&lt;/li&gt;&#xA;&lt;li&gt;The strategy of a controlled test&lt;/li&gt;&#xA;&lt;li&gt;The difference between (a) hearing and speaking and (b) listening and wittily responding — This is particularly pertinent today because the chat-bots do (a) and could be anywhere from 5 to 500 years away from doing (b).  To reach (c) deeply comprehending and responding with inspiration, AI researchers must go beyond modelling the human mind and approach the challenge of modelling the minds of people like Gödel, Einstein, and Turing.  Whether that will ever occur is yet to be revealed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The specific requirements of the Imitation Game, Alan Turing's subtitle above the description of his thought experiment, are a matter of record.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Specific Requirements [Excerpt from Actual Article]&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[The imitation game] is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either &amp;quot;X is A and Y is B&amp;quot; or &amp;quot;X is B and Y is A.&amp;quot; The interrogator is allowed to put questions to A and B thus:&lt;/p&gt;&#xA;&lt;p&gt;C: Will X please tell me the length of his or her hair?&lt;/p&gt;&#xA;&lt;p&gt;Now suppose X is actually A, then A must answer. It is A's object in the game to try and cause C to make the wrong identification. His answer might therefore be:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;My hair is shingled, and the longest strands are about nine inches long.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary. The object of the game for the third player (B) is to help the interrogator.&lt;/p&gt;&#xA;&lt;p&gt;The best strategy for her is probably to give truthful answers. She can add such things as &amp;quot;I am the woman, don't listen to him!&amp;quot; to her answers, but it will avail nothing as the man can make similar remarks.&lt;/p&gt;&#xA;&lt;p&gt;We now ask the question, &amp;quot;What will happen when a machine takes the part of A in this game?&amp;quot; Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, &amp;quot;Can machines think?&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There have been thousands of critiques of both Einstein's relativity and Turing's test, none of which add much value.  Study the thinking of great contributors through their own words and all the refuse that follows will be interesting primarily in its lack of greatness.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Secondary Questions in This Thread&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What requirements if any must the evaluator fulfill in order to be qualified to give the test?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The interrogator (C) is not an evaluator.  Evaluation would be an attempt to be objective, however the premise of Turing's thought experiment is that the interrogator provide her or his subjective judgment.  From a statistics point of view, the interrogator should be selected randomly from the population of the world that shares a spoken language with (A) and (B).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Must there always be two participants in the conversation (one human and one computer) or can there be more?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There must be exactly two to fit the scenario described by Alan Turing. (See below for more detail.)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are placebo tests (where there is not actually a computer involved) allowed or encouraged?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;One could test all kinds of things, and researchers do, however, that would be outside of the scope of Turing's thought experiment.&lt;sup&gt;4&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can there be multiple evaluators? If so does the decision need to be unanimous among all evaluators in order for the machine to have passed the test?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What would reveal the most information to those that sponsor an actual Imitation Game would be a double blind fully randomized test where (A), (B), and (C) are pulled from as random a sample of those men, women, or software systems of the type under test that can converse in a common language, and the test would be run many times with random selections from the samples.&lt;/p&gt;&#xA;&lt;p&gt;Unanimity, evaluation, additional complexity, and communication other than that which was specified by the test would only frustrate the cause, if one sticks with Turing's original intention regarding the question, &amp;quot;Can computers think?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Other Views of Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Turing, as did René Descartes, who stated that machines will never pass a less controlled version of Turing's Imitation Game, saw intelligence through the lens of dialog.  Others have considered other kinds of dialog and other contexts than dialog.  I addressed this in another question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/8219/can-a-brain-be-intelligent-without-a-body&quot;&gt;Can a brain be intelligent without a body?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;References and Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1] &lt;em&gt;Relativity: The Special and the General Theory&lt;/em&gt; by Albert Einstein, 1916&lt;/p&gt;&#xA;&lt;p&gt;[2] &lt;em&gt;The Principle of Relativity&lt;/em&gt; by Albert Einstein and Francis A. Davis, 1923&lt;/p&gt;&#xA;&lt;p&gt;[3] A. M. Turing (1950) Computing Machinery and Intelligence. Mind 49: 433-460.  &lt;a href=&quot;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[4] Turing's 1950 article did not recommend that his thought experiment should be embodied and used in commercial validation of future AI systems.  Alan Turing was, however, concerned with practical computing at one specific point in his career.  That was when the Nazis had overrun France, were pulverizing his homeland from the air, and had sunk a significant portion of the English Navy from below, with the help of Enigma cryptography.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-03T20:53:03.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8250" PostTypeId="1" CreationDate="2018-10-03T22:29:20.213" Score="3" ViewCount="217" Body="&lt;p&gt;I've been reading about expert systems and started reading about MYCIN.&lt;/p&gt;&#xA;&lt;p&gt;I was astonished to find that MYCIN diagnosed patients better than the infectious diseases physicians.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.aaaipress.org/Classic/Buchanan/Buchanan33.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.aaaipress.org/Classic/Buchanan/Buchanan33.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since, it had such a good success rate, why did it fail?&lt;/p&gt;&#xA;" OwnerUserId="18748" LastEditorUserId="18748" LastEditDate="2021-02-08T22:35:18.237" LastActivityDate="2021-02-08T22:35:18.237" Title="Why did MYCIN fail?" Tags="&lt;expert-system&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8251" PostTypeId="1" CreationDate="2018-10-03T23:11:36.607" Score="1" ViewCount="208" Body="&lt;p&gt;I have created a game on an 8x8 grid and there are 4 pieces which can move essentially like checkers pieces (Forward left or Forward right only). I have implemented a DQN in order to pull this off. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how I have mapped my moves:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.actions = {&quot;1fl&quot;: 0, &quot;1fr&quot;: 1,&quot;2fl&quot;: 2, &#xA;  &quot;2fr&quot;: 3,&quot;3fl&quot;: 4, &quot;3fr&quot;: 5,&quot;4fl&quot;: 6, &quot;4fr&quot;: 7}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;essentially I assigned each move to an integer value from 0-7 (8 total moves).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is: During any given turn, not all 8 moves are valid, how do I make sure when model.predict(state) the resulting prediction will be a move that is valid? Here is how I am currently handling it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def act(self, state, env):&#xA;    #get the allowed list of actions&#xA;    actions_allowed = env.allowed_actions_for_agent()&#xA;&#xA;    #Do a random move if random # greater than epsilon&#xA;    if np.random.rand(0,1) &amp;lt;= self.epsilon: &#xA;        return actions_allowed[random.randint(0, len(actions_allowed)-1)]&#xA;&#xA;    #get the prediction from the model by passing the current game board&#xA;    act_values = self.model.predict(state)&#xA;&#xA;    #Check to see if prediction is in list of valid moves, if so return it&#xA;    if np.argmax(act_values[0]) in actions_allowed:&#xA;        return np.argmax(act_values[0])&#xA;&#xA;    #If prediction is not valid do a random move instead....&#xA;    else:&#xA;        if len(actions_allowed) &amp;gt; 0:&#xA;            return actions_allowed[random.randint(0,len(actions_allowed)-1)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I feel like if the agent predicts a move, and if that move is not in the actions_allowed set I should punish the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But because it doesn't pick a valid move I make it do a random one instead, but I think this a problem. Because its bad prediction may ultimately end up still winning the game since the random move may have a positive outcome. I am at a total loss. The agent trains....but it doesn't seem to learn.... I have been training it for over 100k games now, and it only seems to win 10% of it games.... ugh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other helpful information:&#xA;- I am utilizing experience replay for the DQN which I have based on the code from &lt;a href=&quot;https://keon.io/deep-q-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is where I build my model as well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.action_size = 8&#xA;LENGTH = 8&#xA;def build_model(self):&#xA;    #builds the NN for Deep-Q Model&#xA;    model = Sequential() #establishes a feed forward NN&#xA;    model.add(Dense(64,input_shape = (LENGTH,), activation='relu'))&#xA;    model.add(Dense(64, activation='relu'))&#xA;    model.add(Dense(self.action_size, activation = 'linear'))&#xA;    model.compile(loss='mse', optimizer='Adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18244" LastEditorUserId="16565" LastEditDate="2018-10-04T12:31:15.300" LastActivityDate="2018-10-04T12:31:15.300" Title="Using a DQN with a variable amount of Valid Moves per turn for a Board Game" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;python&gt;&lt;keras&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8252" PostTypeId="2" ParentId="8243" CreationDate="2018-10-03T23:52:03.940" Score="1" Body="&lt;p&gt;I believe &quot;Reinforcement Learning&quot; is the term you are looking for (as mentioned by others as well) &lt;em&gt;but&lt;/em&gt; keep in mind that the scope of your problem falls under the section of AI that is called &lt;strong&gt;Search&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://hackernoon.com/search-algorithms-in-artificial-intelligence-8d32c12f6bea&quot; rel=&quot;nofollow noreferrer&quot;&gt;Search algorithms&lt;/a&gt; are based upon experimenting with different &lt;strong&gt;actions&lt;/strong&gt; (decisions) and selecting the one that minimizes an arbitrary &lt;strong&gt;cost function&lt;/strong&gt; (reward), given the current and past problem &lt;strong&gt;states&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15919" LastActivityDate="2018-10-03T23:52:03.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8253" PostTypeId="2" ParentId="8251" CreationDate="2018-10-04T05:31:18.447" Score="1" Body="&lt;p&gt;I think instead of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if np.argmax(act_values[0]) in actions_allowed:&#xA;        return np.argmax(act_values[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you can use something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if np.argmax(actions_allowed_values[0]):&#xA;        return np.argmax(actions_allowed_values[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you didn't random your act if the best action is not allowed, but you choose the best action that allowed. You can do that by checking for each action value, whether it is a valid action or not, then take the best value. Or you can change invalid action value to a very small number.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2018-10-04T05:40:47.370" LastActivityDate="2018-10-04T05:40:47.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8254" PostTypeId="1" CreationDate="2018-10-04T06:25:15.123" Score="1" ViewCount="112" Body="&lt;p&gt;As I'm beginner in image processing, I am having difficulty in segmenting all the parts in &lt;a href=&quot;https://en.wikipedia.org/wiki/DICOM&quot; rel=&quot;nofollow noreferrer&quot;&gt;DICOM&lt;/a&gt; image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm applying &lt;a href=&quot;https://en.wikipedia.org/wiki/Watershed_(image_processing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;watershed algorithm&lt;/em&gt;&lt;/a&gt;, but it segments only that part that has tumour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to segment all parts in the image. Which algorithm will be helpful to perform this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image below contains the tumour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3xtzl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3xtzl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image is the actual DICOM image&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zVgfu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zVgfu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18661" LastEditorUserId="2444" LastEditDate="2019-05-02T13:33:54.870" LastActivityDate="2021-05-21T17:06:35.630" Title="How do I segment each part of a DICOM image?" Tags="&lt;computer-vision&gt;&lt;matlab&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8255" PostTypeId="2" ParentId="8254" CreationDate="2018-10-04T07:49:48.753" Score="0" Body="&lt;p&gt;I looked at some picture from DICOM, and it seems that these images usually don't have many local minima, so I guess that's why watershed isn't optimal.&#xA;However, it seems that in DICOM the color gradients are often strong, so I suggest to use techniques that works well with gradients:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Direct Gradient computations, either convolutional gradients or morphological gradients (selects shape of your different objects)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Normalized Cut on Similarity Graphs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to know more about these you can read &lt;a href=&quot;https://ai.stackexchange.com/q/7617/2444&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="2444" LastEditDate="2019-05-02T13:35:02.037" LastActivityDate="2019-05-02T13:35:02.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8258" PostTypeId="1" AcceptedAnswerId="8544" CreationDate="2018-10-04T14:41:25.100" Score="7" ViewCount="7153" Body="&lt;p&gt;I'm looking for &lt;strong&gt;annotated&lt;/strong&gt; dataset of traffic signs. I was able to find Belgium, German and many more traffic signs datasets. The only problem is these datasets contain only cropped images, like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6SJxW.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6SJxW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While i need (for YOLO -- You Only Look Once network architecture) not-cropped images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X6jI3.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X6jI3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been looking for hours but didn't find dataset like this. Does anybody know about this kind of annotated dataset ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I prefer European datasets.&lt;/p&gt;&#xA;" OwnerUserId="18760" LastEditorUserId="18760" LastEditDate="2018-10-05T08:22:10.423" LastActivityDate="2019-12-27T09:54:07.270" Title="Traffic signs dataset" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;datasets&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="4" ClosedDate="2021-01-29T00:08:09.710" ContentLicense="CC BY-SA 4.0" />
  <row Id="8259" PostTypeId="1" AcceptedAnswerId="8287" CreationDate="2018-10-04T17:54:45.610" Score="2" ViewCount="188" Body="&lt;p&gt;I am using the following perceptron formula &lt;span class=&quot;math-container&quot;&gt;$\text{step}\left(\sum(w_ix_i)-\theta \right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; supposed to be updated in a perceptron, like the weights &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;? If so, what is the formula for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to make the perceptron learn AND and OR, but without updating &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, I don't feel like it's possible to learn the case where both inputs are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. They will, of course, be independent of the weights, and therefore the output will be &lt;span class=&quot;math-container&quot;&gt;$\text{step}(-\theta)$&lt;/span&gt;, meaning &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (which has a random value) alone will determine the output.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2020-03-10T22:41:25.140" LastActivityDate="2020-03-10T22:42:04.723" Title="Is the bias supposed to be updated in the perceptron learning algorithm?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8261" PostTypeId="1" AcceptedAnswerId="8285" CreationDate="2018-10-04T18:07:47.730" Score="2" ViewCount="694" Body="&lt;p&gt;As an Electronics &amp;amp; Communication Engineering student I've heard some stories and theories about &quot;The math we have is not enough to complete a thinker-learner AI.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the truth? Is humankind waiting for another Newton to make new calculus or another Einstein-Hawking to complete the quantum mechanics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, what exactly do we need? What will we call it? &lt;/p&gt;&#xA;" OwnerUserId="18764" LastEditorUserId="3217" LastEditDate="2018-10-06T23:01:33.980" LastActivityDate="2018-12-30T20:51:45.460" Title="Is known math really enough for AI" Tags="&lt;machine-learning&gt;&lt;agi&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8263" PostTypeId="2" ParentId="8258" CreationDate="2018-10-04T18:51:55.040" Score="0" Body="&lt;p&gt;I searched the web but there are no such dataset published but &lt;a href=&quot;https://www.mapillary.com/dataset/vistas?pKey=pBBmjuJ8yU1r2ROYRzWmFg&amp;amp;lat=-6.7203390321265175&amp;amp;lng=-76.83556631804362&amp;amp;z=0.3424751357705809&quot; rel=&quot;nofollow noreferrer&quot;&gt;Check this out&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="18764" LastActivityDate="2018-10-04T18:51:55.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8264" PostTypeId="2" ParentId="8215" CreationDate="2018-10-05T06:03:24.267" Score="0" Body="&lt;p&gt;Assuming that you do have a dataset (images + labels/bounding boxes) in a format that's required by the training model, you can fine-tune your existing model. You can choose to unlock the final few layers or leave all the layers unlocked during the training process. When I had performed such an experiment with RetinaNet I chose to unlock final few conv layers and was able to achieve slightly higher accuracy. &lt;/p&gt;&#xA;" OwnerUserId="18772" LastActivityDate="2018-10-05T06:03:24.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8265" PostTypeId="2" ParentId="8258" CreationDate="2018-10-05T06:36:20.197" Score="1" Body="&lt;p&gt;Check this one by &lt;a href=&quot;http://cvrr.ucsd.edu/LISA/lisa-traffic-sign-dataset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSD&lt;/a&gt;.&#xA;It contains both video as well as images related to traffic signs. The annotations are present in csv&lt;/p&gt;&#xA;" OwnerUserId="18772" LastActivityDate="2018-10-05T06:36:20.197" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8266" PostTypeId="2" ParentId="8220" CreationDate="2018-10-05T08:46:23.937" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What aspects of AI would be most applicable to creating a self learning game AI for a racing game (Q-Learning, NEAT etc)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, you are looking at a problem that involves &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequential_decision_making&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequential decision making&lt;/a&gt;, in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt; context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are wanting to build an agent that can learn by receiving screen images, then NEAT cannot scale to that complexity directly. Although there might be clever combinations of deep learning and evolutionary algorithms you could apply, the most heavily explored and likely successful solution will be found in Deep Reinforcement Learning. Algorithms like DQN, A3C, A2C, PPO . . . there are dozens to consider, but all are based around agents using samples of experience to update functions that measure either the &quot;value&quot; of acting in a certain way (a policy) or estimate the best policy directly.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Could a ANN or NEAT that has learned to play Mario Kart 64 be used to learn to play another racing game?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Within limits, yes. You will have built a system that takes pixel inputs, and outputs controller messages. If you re-start training from scratch on any compatible N64 game (with same screen resolution and same controller outputs) there is a chance it could learn to play that new game well. As other driving games on the N64 are a subset of all games, and more similar to each other than, say, a scrolling shooter or adventure game, then an agent that can successfully learn one will likely learn another too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is unlikely that a Mario Kart agent will immediately be good at another game without re-training. The visual and control differences will probably be too much. You could try though. An interesting experiment would be to take your trained agent, or some part of it, and see if starting with that improves learning time on a new game. This is called Transfer Learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What books/material should i read up on to undertake this project?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You will need at least the first parts of an introduction to Reinforcement Learning. If your goal is to aim to just make the agent, then you can skip theory-heavy parts, but within limits the more theory you understand, the easier it will be to change code features towards getting something working. I can suggest the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The book &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto, Reinforcement Learning: An Introduction&lt;/a&gt; - this covers theory behind basic RL, and the online draft is free&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;A course of lectures on Reinforcement Learning from UCL&lt;/a&gt;, by David Silver. The video lectures cover same topics as the Sutton &amp;amp; Barto book.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The book &lt;a href=&quot;https://www.amazon.co.uk/Deep-Reinforcement-Learning-Hands-Q-networks-ebook/dp/B076H9VQH6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning Hands-On&lt;/a&gt;, which you will need to buy. As the title suggests, it covers more practical aspects of building RL systems. It uses Python and PyTorch. As RL has become more popular in recent years, there are plenty of other similar books - I have no reason to recommend this one over them, but I am reading it at the moment, and the quality is adequate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What other considerations should i take throughout this project?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Before you get started, you should know this could be an ambitious project, requiring a lot of compute time using currently-available toolkits. You need to think ahead a little, as you will be faced with these decisions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it more important to you to make a working bot on this problem, or more important to understand the underlying theories so that you understand what is going on. You will need to do at least a little of both, but you can go strongly in either direction:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are enough pre-built learning agents available on the internet that you &lt;em&gt;might&lt;/em&gt; have a successful project by learning just enough to wire up a copy of the game, and then letting the learning agent do its job. Would this be enough for you, would you feel that you had solved your problem?&lt;/li&gt;&#xA;&lt;li&gt;There is plenty of educational material about RL and how the maths behind it works. If you are interested in understanding that, and perhaps generating your own ideas for improvements to current algorithms, then you need to study that material harder. However, this may lead you away from your Mario Kart player goal, and you may never actually solve it, because far simpler problems that require less computing power are still actually interesting academically.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition, I can think of the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Building your Mario Kart player is a long-term goal. You will need to start with simpler agents in order to understand the techniques that you hope to apply to Mario Kart.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You may need more compute power to solve this game than you have available.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You will need to solve the issue of automating control of a N64 emulator. There is at least an existing emulator &lt;a href=&quot;http://www.mupen64plus.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mupen64Plus&lt;/a&gt; - I do not know whether it will be adequate for you, but at least one person has attempted to wrap this for automated learning, &lt;a href=&quot;https://github.com/bzier/gym-mupen64plus&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the gym-mupen64plus project&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-05T08:54:33.797" LastActivityDate="2018-10-05T08:54:33.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8267" PostTypeId="1" CreationDate="2018-10-05T13:09:59.060" Score="2" ViewCount="165" Body="&lt;p&gt;In reinforcement learning, the system sets some controllable variables, and then determines the quality of the result of the dependent variable(s); using that &amp;quot;quality&amp;quot; to update the algorithm.&lt;/p&gt;&#xA;&lt;p&gt;In simple games, this works fine because for each setting there is a single result.&lt;/p&gt;&#xA;&lt;p&gt;However, for the real world (e.g. an airflow system), the result takes some time to develop and there is no single precise &amp;quot;pair&amp;quot; result to the conditions set.  The flow change takes time and even oscillates a bit as flow stabilizes to a steady-state.&lt;/p&gt;&#xA;&lt;p&gt;In practical systems, how is this &amp;quot;lag&amp;quot; accounted for?  How are the un-settled (false) results ignored?  How is this noise distinguished from exogenous factors (un-controlled system inputs e.g. an open window exposed to wind)?&lt;/p&gt;&#xA;" OwnerUserId="17061" LastEditorUserId="2444" LastEditDate="2020-06-20T10:39:49.100" LastActivityDate="2020-06-20T10:39:49.100" Title="How are &quot;lags&quot; and &quot;exogenous factors&quot; accounted for in reinforcement learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;applications&gt;&lt;credit-assignment-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8268" PostTypeId="2" ParentId="8267" CreationDate="2018-10-05T13:34:32.660" Score="2" Body="&lt;p&gt;One of Reinforcement Learning's core features is the ability to deal with delayed rewards/punishments - i.e. rewards that may occur as a consequence of a decision that occurred multiple time steps ago. That is because the &lt;em&gt;value&lt;/em&gt; that it optimises is defined as a long-term sum of immediate rewards. This value is often called the &lt;em&gt;return&lt;/em&gt; or &lt;em&gt;utility&lt;/em&gt; and although it will take short term rewards into account, they do not have to be the dominant factors, and often are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All RL solvers are designed to solve this problem, it is core to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDP formalism&lt;/a&gt; used in RL theory. You can decide to focus on immediate rewards or longer-term ones by adjusting using a discount factor (often represented as &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; in RL equations). Low discount factors will cause an agent to prefer short-term rewards, high discount factors will cause the agent to prefer for longer-term sums of reward. In continuous problems you must use &lt;span class=&quot;math-container&quot;&gt;$\gamma \lt 1$&lt;/span&gt; or instead use &lt;em&gt;average reward&lt;/em&gt; as the objective to be maximised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One classic example of this kind of delay problem is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mountain_car_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mountain Car environment&lt;/a&gt;. Here, reward is only granted for reaching a certain point, and it is not possible to reach it by just taking the obvious action of moving directly towards the objective. Most RL algorithms can solve this problem, and get close to optimal solutions, purely from experiencing the final reward and associating it progressively with earlier and earlier decisions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In practical systems how is this &quot;lag&quot; accounted for? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As above, this is inherent to RL, as opposed to simplified versions such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit#Contextual_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contextual Bandits&lt;/a&gt; which don't have time steps and evolving state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, it may take some experimentation to find the best RL algorithm and solve a problem efficiently. Environments with very large delays and lots of noise are harder to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also really important to set the reward mechanism for your true goals, and not some intermediate ones. If your goal is a stable state, you need to reward that, and not necessarily maximum throughput. If both throughput and stability are important to the end result, you may need to reward both - e.g. reward throughput, but penalise short-term fluctuations.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How are the un-settled (false) results ignored? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general handling short-term vs long term effects of actions is known as the &lt;em&gt;assignment problem&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is most often solved statistically, over long term experience, as the agent associates not just immediate reward with an action, but the long term return after making each decision. Different RL solvers have different mechanisms for effectively moving this association back through time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key thing here is that an RL solver learns to maximise &lt;em&gt;expected&lt;/em&gt; return or utility from any given state. Working with the expected value allows for smoothing out of variations, provided the agent gains enough experience to approach a statistical mean. Extremely rare and large variations can cause problems with this, and might need to be explicitly coded for as opposed to learned.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How is this noise distinguished from exogenous factors (un-controlled system inputs e.g. an open window exposed to wind)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is trickier, and depends more critically on the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A RL-based system that is exposed to these variations whilst learning, and some way to represent their effects in the state variables &lt;em&gt;should&lt;/em&gt; learn to pick the best action when these external factors impact it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL systems based on maintaining a stable state can sometimes do well when they are moved out of that stable state. A good example here might be pole balancing, where the agent learns enough variations of position and speed that if something knocks the pole out of balance, it will still right itself even though the agent was not trained to react to such events directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in some cases, if knowledge is not available to the agent, or it has not been trained in that particular scenario, it will fail to generalise and may behave in counter-productive ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, if you are concerned about specific scenarios, you need to include them in training, or at least test for them after training.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-05T16:45:01.470" LastActivityDate="2018-10-05T16:45:01.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8269" PostTypeId="1" CreationDate="2018-10-05T15:25:10.763" Score="1" ViewCount="31" Body="&lt;p&gt;I'm a student of Artificial Intelligence. This semester I've Computer Vision course. We should select a topic related to intelligent vehicles and read several papers and implement one of those paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched and I saw different topics related to intelligent vehicles like detection, tracking, semantic segmentation, ...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't have enough knowledge to select some papers that have these properties:&#xA; - Published recently [2015-2018].&#xA; - Kinda related to intelligent vehicles like autonomous cars and drones.&#xA; - Not hard to implement. I think papers that use very Deep learning or are needed to run simulations are hard for implementing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After hours of searching, I got confused. So I decided to get help from you.&lt;/p&gt;&#xA;" OwnerUserId="10051" LastActivityDate="2018-10-05T15:25:10.763" Title="Good papers for implementing as project of computer vision course" Tags="&lt;computer-vision&gt;&lt;autonomous-vehicles&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8270" PostTypeId="1" CreationDate="2018-10-05T16:47:38.873" Score="1" ViewCount="993" Body="&lt;p&gt;I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a complete beginner in neural networks. I have been recently working with tensorflow &amp;amp; keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.&lt;/p&gt;&#xA;" OwnerUserId="18788" LastEditorUserId="16920" LastEditDate="2019-04-09T10:07:12.413" LastActivityDate="2021-03-19T22:02:45.347" Title="How to create Partially Connected NNs with prespecified connections using Tensorflow?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8271" PostTypeId="2" ParentId="7528" CreationDate="2018-10-05T17:06:12.033" Score="2" Body="&lt;p&gt;Fear of this kind is an irrational response (large negative incentive in response to a small risk).  Modeling fear would need to model a &quot;grossness&quot; factor associated with, for example, spiders so that the normally un-proportional response would occur.  The &quot;grossness&quot; factor could be manifested in many other forms to magnify a response to a previously unpleasant, though not particularly dangerous, experience.  Such fear can also be inspired by hearsay (think hysteria caused by a sensational news story).&#xA;A NN would normally only respond minimally to a minimal risk.&lt;/p&gt;&#xA;" OwnerUserId="17061" LastActivityDate="2018-10-05T17:06:12.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8273" PostTypeId="1" CreationDate="2018-10-05T19:59:52.837" Score="1" ViewCount="1061" Body="&lt;p&gt;We are currently working on developing a 3D modeling software that allows designers to set spatial constraints to models. The computer then should generate a 3D mesh conforming to these constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why should or shouldn't we use Lisp for the constraint satisfaction part? Will Prolog environment be any better? Or should we stick to C/C++ libraries?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One requirement we have is that we want to use the Unity Game Engine as it has a lot of 3D tools built in&lt;/p&gt;&#xA;" OwnerUserId="18726" LastActivityDate="2021-04-23T03:08:46.010" Title="What are the advantages and disadvantages of using LISP for constraint satisfaction in 3D space" Tags="&lt;optimization&gt;&lt;programming-languages&gt;&lt;lisp&gt;&lt;prolog&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8274" PostTypeId="1" CreationDate="2018-10-05T21:52:46.623" Score="3" ViewCount="569" Body="&lt;p&gt;I implemented an image segmentation pipeline and I trained it on the &lt;a href=&quot;https://en.wikipedia.org/wiki/DICOM&quot; rel=&quot;nofollow noreferrer&quot;&gt;DICOM dataset&lt;/a&gt;. I compared the results of the model with manual segmentation to find the accuracy. Is there other methods for evaluation? &lt;/p&gt;&#xA;" OwnerUserId="18661" LastEditorUserId="2444" LastEditDate="2019-05-04T15:53:28.127" LastActivityDate="2020-08-02T12:51:15.903" Title="Which evaluation methods can I use for image segmentation?" Tags="&lt;computer-vision&gt;&lt;datasets&gt;&lt;image-segmentation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8276" PostTypeId="2" ParentId="8273" CreationDate="2018-10-05T22:53:33.130" Score="0" Body="&lt;p&gt;This is actually a question which will only receive opinion based answer's. A question you should ask yourself is if the constraint part is really that complex that it is worth to use a different programming language. Unity itself offers a C# API [1] and I would therefore stick with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://unity3d.com/programming-in-unity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://unity3d.com/programming-in-unity&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10228" LastActivityDate="2018-10-05T22:53:33.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8277" PostTypeId="2" ParentId="8274" CreationDate="2018-10-05T23:44:47.067" Score="2" Body="&lt;p&gt;See:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Martin Thoma: &lt;a href=&quot;https://arxiv.org/pdf/1602.06541.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Semantic Segmentation&lt;/a&gt;, Section III&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Subsection A is about metrics and B is about datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Metrics include: accuracy, IoU, frequency weighted IoU, F-beta score, speed, ...&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-10-05T23:44:47.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8278" PostTypeId="2" ParentId="8089" CreationDate="2018-10-06T00:14:34.907" Score="1" Body="&lt;p&gt;There is absolutely no doubt that fuzzy logic can contribute to the reliability and accuracy of real time diagnostics of mechanical, thermodynamic, and electro-magnetic devices.  I can state this with assurance because it exists in aeronautics products today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the senior PhDs at the research facility wanted to acquire data via standard techniques and had purchased a brigade of high end Hewlett Packard workstations and instrumentation clusters.  Another senior PhD there wanted to build a fuzzy logic container to provide early warnings of failures, just as implied in the question.  There was some interplay between those two projects in their application in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My direct involvement was to teach the first PhD OO and C++ and design and implement a scientific computing framework to churn the Big Data for the second PhD.  The framework provided a super-set of the union of SciPy and TensorFlow with a little PyTorch added.  That we developed these techniques before Java and Python existed is why Hewlett Packard sent engineers to study what we were doing, with DoD permission, of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want to do is this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Become familiar with the open source options in Java, Python, and C.  The general trend is to either use Java-ML or use C wrapped in the above mentioned Py libraries.&lt;/li&gt;&#xA;&lt;li&gt;Use 16 or 24 bit multi-channel data acquisition &lt;strong&gt;with time stamps and latched, concurrent A-to-D conversion&lt;/strong&gt; at a frequency at least twice&lt;sup&gt;1&lt;/sup&gt; the 16th harmonic of the vibration base frequencies.&lt;/li&gt;&#xA;&lt;li&gt;Ensure that you have sufficient system memory to hold what for embedded qualifies as Big Data&lt;sup&gt;2&lt;/sup&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Ensure that you know how spectra (both amplitude and phase) can be obtained via FFT from channels.&lt;/li&gt;&#xA;&lt;li&gt;Use LINUX, a single board computer, and either Intel Movidius&lt;sup&gt;3&lt;/sup&gt; or NVidia CUDA for hardware acceleration.&lt;/li&gt;&#xA;&lt;li&gt;The fuzzy logic container and any assistive reinforced learning can be on the embedded CPU and leverage the hardware acceleration.&lt;/li&gt;&#xA;&lt;li&gt;The fuzzy rules can be acquired from the experts to which you have access, as indicated in the question.&lt;/li&gt;&#xA;&lt;li&gt;Read &lt;em&gt;Chaos Theory Tamed&lt;/em&gt; by Williams, 1997 to get familiar with phase relationships between input channels and how to analyze changes in chaotic behavior between the harmonics.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Start with small POCs and generate independent components that can be connected and tested independently with input that came from the output of previously tested components.  Here are some, in rough order of development and signal path.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data acquisition of multiple samples of multiple channels to a matrix.&lt;/li&gt;&#xA;&lt;li&gt;FFT of acquired data to an amplitude-phase array in the frequency domain.&lt;/li&gt;&#xA;&lt;li&gt;Analysis to determine the patterns in metrics that precede failure.&lt;/li&gt;&#xA;&lt;li&gt;Introduce and train artificial networks (either LSTM or attention type) where indicated to remove redundancy from the information that will be input to the fuzzy logic container.&lt;/li&gt;&#xA;&lt;li&gt;Write up the rules and place them in the container.&lt;/li&gt;&#xA;&lt;li&gt;Train the fuzz.&lt;/li&gt;&#xA;&lt;li&gt;Test the system.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You will need to work with the experts on most of these steps.  If you try to involve them at the rule writing stage and they weren't privy to any of the previous developments, they may be confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not a lawyer, but, because of the involvement of failure prediction experts, I think you'll need an internationally binding NDA to ensure the experts don't walk away with the product of your research and move to another country.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are a few considerations for product reliability you may need to consider even after you have a working system.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Develop a way to adapt the system in real time ... to tracking changes in what indicates near failure based on a centralized database related to the product.&lt;/li&gt;&#xA;&lt;li&gt;Consider version control and deployment challenges.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm sorry that I have no further references.  You can try the IEEE library if you have access.  I have been too busy doing this type of work to publish what we do, but I think most of the important directions are included here already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FootNotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Not much can be learned about impending failure from the primary harmonic of vibration.  The first 16 harmonics are sufficient.  Because of Nyquist Criteria considerations, the sample rate must be twice the top harmonic.  This means that if your vibration is 1 KHz, the sample rate must be at least 32 KHz.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Big Data input rates may be necessary because with 8 channels of strain input from Wheatstone bridges of gauges, 8 channels of accelerometer, 8 channels of IR sensing and the above vibration base harmonic, we have (24 channels + 1 time stamp) x 32 KHz x 3 bytes per channel-sample = 24 Mbytes per second.  If your failure detection window holds 10 seconds of vibration data as IEEE 64 bit reals and you send 1 second (overlapping) Hamming windows to the hardware FFT, just for that alone will need 12.8 Gbyte input matrix, 6.4 for the input and 6.4 for the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Available for embedded applications from Mouser and other distributors as properly paired single board computers and Movidius mPCIe (mini PCI express) daughter boards from makers like Aaeon.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-06T00:14:34.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8279" PostTypeId="1" CreationDate="2018-10-06T08:46:15.770" Score="1" ViewCount="132" Body="&lt;p&gt;I was reading a machine learning book that uses probabilities like these:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x;y), P(x;y,z), P(x,y;z)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I couldn't find what they are and how can I read and understand them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from the context, I saw one of these probabilities on it here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/lf0X7.png&quot; width=&quot;500&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9941" LastEditorUserId="16565" LastEditDate="2018-11-07T15:57:29.540" LastActivityDate="2019-05-06T17:04:40.953" Title="What are the meanings of these (P(x;y), P(x;y,z),P(x,y;z))?" Tags="&lt;machine-learning&gt;&lt;probability&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8281" PostTypeId="1" CreationDate="2018-10-06T11:49:18.957" Score="3" ViewCount="2502" Body="&lt;p&gt;With reference to the research paper entitled &lt;a href=&quot;https://ieeexplore.ieee.org/document/7296633&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sentiment Embeddings with Applications to Sentiment Analysis&lt;/a&gt;, I am trying to implement its sentiment ranking model in Python, for which I am required to optimize the following hinge loss function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\operatorname{loss}_{\text {sRank}}=\sum_{t}^{T} \max \left(0,1-\delta_{s}(t) f_{0}^{\text {rank}}(t)+\delta_{s}(t) f_{1}^{\text {rank}}(t)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike the usual mean square error, I cannot find its gradient to perform backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I calculate the gradient of this loss function?&lt;/p&gt;&#xA;" OwnerUserId="18804" LastEditorUserId="2444" LastEditDate="2020-05-18T14:30:22.247" LastActivityDate="2020-05-18T14:30:22.247" Title="How do I calculate the gradient of the hinge loss function?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;&lt;sentiment-analysis&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8283" PostTypeId="1" AcceptedAnswerId="8286" CreationDate="2018-10-06T15:40:41.847" Score="1" ViewCount="49" Body="&lt;p&gt;Example: texas holdem poker vs texas holdem poker with the same rounds, just with no public cards dealt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would algorithms like CFR approximate Nash-equilibrium more easily?&lt;br&gt;&#xA;Could AI that does not look at public cards achieve similar performance in normal texas holdem as AI that looks at public state tree?&lt;/p&gt;&#xA;" OwnerUserId="18808" LastEditorUserId="18808" LastEditDate="2018-10-08T14:56:26.247" LastActivityDate="2018-10-08T14:56:26.247" Title="What's the difference between poker with public cards and without them?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;ai-community&gt;&lt;poker&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8284" PostTypeId="1" AcceptedAnswerId="8401" CreationDate="2018-10-06T18:29:31.960" Score="7" ViewCount="840" Body="&lt;p&gt;I have a neural network with the following structure:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eHMGH.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eHMGH.jpg&quot; alt=&quot;Neural Network with target outputs: 0.8,-0.3&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am expecting specific outputs from the neural network which are the target values for my training. Let's say the target values are 0.8 for the upper output node and -0.3 for the lower output node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The activations function used for the first 2 layers are ReLu or LeakyReLu while the last layer uses atan as an activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For back propogation, instead of adjusting values to make the network's output approach 0.8,-0.3. is it suitable if I use the inverse function for atan -which is tan itself- to get &quot;the ideal input to the output layer multiplied by weights and adjusted by biases&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tan of 0.8 and -0.3 is 0.01396 and -0.00524 approximately. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My algorithm would then adjust weights and biases of the network so that the &quot;pre-activated output&quot; of the output layer -which is basically (sum(output_layer_weight*output_layer's inputs)+output_layer_biases)- approaches 0.01396 and -0.00524.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this suitable &lt;/p&gt;&#xA;" OwnerUserId="18640" LastActivityDate="2018-10-14T05:10:07.937" Title="Is it suitable to find inverse of last layer's activation function and apply it on the target output?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8285" PostTypeId="2" ParentId="8261" CreationDate="2018-10-06T19:18:33.420" Score="5" Body="&lt;p&gt;&lt;strong&gt;Maybe.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI has a long history of encountering mathematical impossibilities and then working around them already. While the individuals who solved these problems don't get as much press as Newton, Einstein, or Hawking, a case could be made that their contributions to human knowledge are on a similar scale. Unfortunately, their results don't relate to physical systems, so they can be harder to explain to the layperson. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something to keep in mind is that your question assumes the correctness of the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Great_man_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Great Man Theory&lt;/a&gt;&quot; of science history, which holds that science advances by the efforts of exceptional people, and that we need to wait around for more such people to appear for science to advance. This view of scientific history is overly simplistic, and probably wrong. For example, most (all?) of Newton's discoveries were likely to have been made by someone else, around the same time, if he didn't make them (see, e.g. Leibnitz, who actually &lt;em&gt;did&lt;/em&gt; discover calculus at the same time), so a better view might be of a large community of researchers who gradually develop more advanced models based on each others' work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, I've listed out some past examples of problems that were overcome, and some outstanding problems that might require the development of new mathematical tools to solve properly. Keep in mind though, that we can't know for sure whether new tools are needed: maybe existing tools are sufficient, but no one has applied them in the right way yet!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some past problems that plagued AI and required the development of better mathematical approaches&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial explosion&lt;/a&gt; problem appeared to preclude having an AI system reason about probabilities and causation in a logically correct way. This problem was solved by the development of &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian networks&lt;/a&gt; and causal networks, with the work being led by &lt;a href=&quot;https://en.wikipedia.org/wiki/Judea_Pearl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Judea Pearl&lt;/a&gt; and his students. Pearl won a Turing award for this, but has received very little coverage in the popular press.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many optimization problems that AI systems need to solve are &lt;a href=&quot;https://en.wikipedia.org/wiki/NP-hardness&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP-hard&lt;/a&gt;. This means that no general purpose algorithms exist that can give exact solutions to these problems in a polynomial number of steps. Because of this, the problems were initially viewed as intractable. The development of &lt;a href=&quot;https://en.wikipedia.org/wiki/Polynomial-time_approximation_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;PTAS algorithms&lt;/a&gt;, and a deep understanding of &lt;a href=&quot;https://www.ijcai.org/Proceedings/91-1/Papers/052.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;phase-changes&lt;/a&gt; in NP-Hard problems (pioneered by Cheeseman et al.) led to an ability to identify exactly what makes these problems hard, to identify which subparts cause the hardness, and to practical algorithms that solve all sorts of problems in these domains. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Much effort in AI was spent designing new classification methods, and arguing about why one might or might not be better than another. The &lt;a href=&quot;http://www.no-free-lunch.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;No Free Lunch Theorems&lt;/a&gt;, along with other work in computational learning theory provided a clear mathematical framework for understanding how systems can learn, and where their limits are. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As a more recent example, games involving chance, like Poker, have state spaces so vast that they cannot be searched through effectively, even with heuristics. Phrasing these games in the language of Game Theory, and proving the convergence of the counter-factual minimax regret algorithm provided solutions. Bowling et al. were at the center of this work. It got some press coverage, but most of the coverage focused on &quot;computers can play poker&quot;, and not the exiting, more generalizable, lessons AI researchers had learned in the process.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some problems that are still outstanding, and might require new mathematical techniques:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there efficient algorithms for solving problems in the complexity classes PPAD, NP, or #P? Many AI problems fall into these categories. &lt;a href=&quot;https://en.wikipedia.org/wiki/P_versus_NP_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;We do not know the answer definitively&lt;/a&gt;, and although most researchers suspect that no such algorithms exist, understanding &lt;em&gt;why not&lt;/em&gt; seems like it could also provide major research advantages. Existing proof techniques do not seem likely to crack this problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We have no good mathematical models of subjective experience. While some researchers (like &lt;a href=&quot;https://en.wikipedia.org/wiki/Paul_Churchland&quot; rel=&quot;nofollow noreferrer&quot;&gt;Churchland&lt;/a&gt;) think this is not likely to play a major role in the development of intelligent systems, a concrete model could either support or refute that view, and might provide solid frameworks for current problems in AI like the study of &lt;a href=&quot;http://www.cs.cornell.edu/~helou/IMRL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;motivation&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The original project of the social sciences was to provide mathematical models of individual humans, and then of human societies. This has mostly been abandoned (Psych is still at it, but Economics prefers to study rational agents instead of humans, and most of the others gave up mathematical modeling entirely to pursue the methodologies of the humanities instead of the sciences). Nonetheless, the lack of sound mathematical descriptions of human behaviours is becoming a major topic within AI, with work on norms, trust, emotion, and other topics. If a mathematical framework were developed to describe the actions of human societies or of individual humans, then AI would be advanced significantly (along with many other fields!).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2018-12-30T20:51:45.460" LastActivityDate="2018-12-30T20:51:45.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8286" PostTypeId="2" ParentId="8283" CreationDate="2018-10-06T19:35:10.143" Score="1" Body="&lt;p&gt;It depends a little on what you mean by &quot;the same rounds, just with no public cards dealt.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean that each player will just be dealt 2 cards, and no public cards exist, then really we're playing a sort of &quot;high card&quot; game. The best hand is just a pair of aces, CFR will solve this quickly, because the number of possible game states is extremely small compared to a full poker game (especially if we exploit the symmetry of suits, since flushes aren't possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean that each player will be dealt 5 cards, with several rounds of betting as before, CFR will probably do &lt;em&gt;less&lt;/em&gt; well. The state space will be larger, since there are more cards in play (10 instead of 9). Betting may become more complex, and more complex betting &lt;a href=&quot;https://arxiv.org/pdf/1302.7008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;expands the state space enormously&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean that the cards are dealt as before, but the program simply will not look at the cards, then you've kept the state space of the game the same size, but radically reduced the number of information sets. Playing against an opponent who &lt;em&gt;can&lt;/em&gt; look at the cards on the table, your program would be at an enormous disadvantage. For instance, imagine the cards on the table are &quot;3 3 8 8 5&quot;, and that you have a pair of 2's in your hand. You would want to play very differently from if the cards on the table were &quot;2 2 10 4 7&quot;, but an AI without access to the table cards would have to act the same in both situations. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-10-06T19:35:10.143" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8287" PostTypeId="2" ParentId="8259" CreationDate="2018-10-06T23:11:16.973" Score="0" Body="&lt;p&gt;I found the answer to my question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Treat &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; as a normal weight, associated with an input that always equals &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="2444" LastEditDate="2020-03-10T22:42:04.723" LastActivityDate="2020-03-10T22:42:04.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8289" PostTypeId="2" ParentId="8279" CreationDate="2018-10-07T01:16:48.497" Score="2" Body="&lt;p&gt;This means &quot;Parameterized by&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we all agree on the idea of conditional probabilities: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(X | Y) = P(X,Y) / P(Y)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the probability that X happens given that we've seen Y happen, is the fraction of worlds in which Y happened that also contain X. This is uncontroversial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're a Bayesian, you might view parameters themselves as variables in a statistical model. So you might want to speak about the probability of a parameter taking on a certain value, or the probability of the data given that certain parameters have taken on certain values. In that case, you might write something like &lt;span class=&quot;math-container&quot;&gt;$P(D | \theta)$&lt;/span&gt; to denote the probability of the data given a parameter &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're a frequentist, you might find notation like this unsettling, because parameters don't have probabilities under the frequentist view, but instead have fixed values. You could talk about the probability of observing the data under a particular parameterization by defining a family of parameterized probability density functions, and then writing something like &lt;span class=&quot;math-container&quot;&gt;$P(D;\theta)$&lt;/span&gt;. You might also do this as a Bayesian if you wanted to clearly differentiate between model parameters and other things you might observe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x;y)$&lt;/span&gt; then is read as &quot;The probability of x under probability density function P, parameterized by y.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x;y,z)$&lt;/span&gt; is &quot;The probability of x under probability density function P, parameterized by y and z.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x,y;z)$&lt;/span&gt; is &quot;The probability of x and y jointly, under probability density function P, parameterized by z.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also write something like &lt;span class=&quot;math-container&quot;&gt;$P(X|Y; z)$&lt;/span&gt;, which would be &quot;The conditional probability of X given that we observe Y, under probability density function P, parameterized by z.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of the latter would be in something like logistic regression. We might wish to know &lt;span class=&quot;math-container&quot;&gt;$P(L | D ; \theta)$&lt;/span&gt;, where L is the label of the data, D are the features we observe, and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the coefficients of the regression. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-10-07T01:16:48.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8290" PostTypeId="1" CreationDate="2018-10-07T02:03:59.597" Score="0" ViewCount="40" Body="&lt;p&gt;The subject matter is to count the number of people in a large room, wherein a camera is placed in a very high ceiling: an example would be Grand Central Station.   Faces are not visible: &lt;a href=&quot;http://zhreshold.github.io/logs/2014/04/16/head-detection/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the scalp (top of the head) is visible to the camera as shown in the link's video&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal: I would like to perform a Google literature search to assess the work that has been performed on overhead head recognition,  however, I am not sure what the best  keyword pairs: (scalp? head? people?) to describe the object that is to be recognized from a camera positioned in the ceiling (overhead? bird's eye? satellite?).  I'd like the search to return leading-edge (AI) techniques that benchmark results&lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="18819" LastEditDate="2018-10-08T20:56:26.840" LastActivityDate="2018-10-08T20:56:26.840" Title="Keywords to describe people counting from a camera?" Tags="&lt;object-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8291" PostTypeId="1" AcceptedAnswerId="8343" CreationDate="2018-10-07T02:15:21.373" Score="1" ViewCount="57" Body="&lt;p&gt;I have written my own basic convolutional neural network in Java as a learning exercise. I am using it to analyze the MIT CBCL face database image set. They are a set of 19x19 pixel greyscale images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Network specifications are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Single Convolution Layer with 1 filter:&#xA;Filter Size: 4x4.&#xA;Stride Size: 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Single Pooling Layer&#xA;2x2 Max Pooling&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3 layer MLP(input, 1 hidden and output)&#xA;input = 64 neurons&#xA;hidden = 15 neurons&#xA;output = 2 neurons&#xA;learning rate = 0.1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am getting reasonable accuracy(92.85%), but my issue is that it is being achieved at very different points in the epoch count across network runs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epochs  Training Accuracy   Test Accuracy   Validation Accuracy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run 1   415 93.13   92.44   93.35&#xA;Run 2   515 92.44   93.18   92.84&#xA;Run 3   327 93.83   92.05   92.38&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the Java random class with the same seed for every run to initialize the kernel, the MLP weights and break the input data into 3 sets.(training is being done using the 33-33-33 method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a loss as to what is causing this variation in epoch count to achieve the highest point in validation accuracy. Can anybody explain this?&lt;/p&gt;&#xA;" OwnerUserId="18818" LastEditorUserId="10135" LastEditDate="2018-10-18T10:44:36.693" LastActivityDate="2018-10-18T10:44:36.693" Title="Huge variations in epoch count for highest generalized accuracy in CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;java&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8293" PostTypeId="1" AcceptedAnswerId="8295" CreationDate="2018-10-07T09:55:50.223" Score="14" ViewCount="3462" Body="&lt;p&gt;I've been looking at reinforcement learning, and specifically playing around with creating my own environments to use with the OpenAI Gym AI. I am using agents from the stable_baselines project to test with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I've noticed in virtually all RL examples is that there never seems to be any dropout layers in any of the networks. Why is this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created an environment that simulates currency prices and a simple agent, using DQN, that attempts to learn when to buy and sell. Training it over almost a million timesteps taken from a specific set of data consisting of one month's worth of 5-minute price data it seems to overfit a lot. If I then evaluate the agents and model against a different month's worth of data is performs abysmally. So sounds like classic overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is there a reason why you don't see dropout layers in RL networks? Is there other mechanisms to try and deal with overfitting? Or in many RL examples does it not matter? e.g. there may only be one true way to the ultimate high score in the 'breakout' game, so you might as well learn that exactly, and no need to generalise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is it deemed that the chaotic nature of the environment itself should provide enough different combinations of outcomes that you don't need to have dropout layers?&lt;/p&gt;&#xA;" OwnerUserId="18372" LastActivityDate="2018-10-09T17:32:58.447" Title="Why do you not see dropout layers on reinforcement learning examples?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;overfitting&gt;&lt;dropout&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="8295" PostTypeId="2" ParentId="8293" CreationDate="2018-10-07T13:46:41.930" Score="14" Body="&lt;p&gt;Dropout essentially introduces a bit more variance. In supervised learning settings, this indeed often helps to reduce overfitting (although I believe there dropout is also already becoming less.. fashionable in recent years than in the few years before that; I'm not 100% sure though, it's not my primary area of expertise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Reinforcement Learning, additional variance is not really what we're looking for. There already tends to be a large amount of variance in the learning signals that we get, and this variance already tends to be a major issue for learning stability and/or learning speed. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Randomness in action selection leads to variance in returns that we observe&lt;/li&gt;&#xA;&lt;li&gt;There may be randomness inherent to the environment itself, leading to extra variance in our observations (some environments are nondeterministic)&lt;/li&gt;&#xA;&lt;li&gt;Unlike Supervised Learning settings, in Reinforcement Learning we often actually use our own predictions as a part of our loss function / training signal. For example, in temporal-difference learning (like Q-learning / DQN), the target that we update towards looks like &lt;span class=&quot;math-container&quot;&gt;$r + \max_{a'} Q(s', a')$&lt;/span&gt;. In that term, only the &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is a ground-truth observation (like we would use in supervised learning), and the other term is our own prediction. During a learning process, those latter parts (our own predictions) are changing over time. This is a &quot;&lt;strong&gt;moving target&lt;/strong&gt;'' problem, which can be viewed as additional variance in our learning signals.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many important parts of Deep RL algorithms (without which our training processes empirically turn out to destabilize and break down) are very much tailored towards &lt;em&gt;reducing&lt;/em&gt; that variance. For example, Target Networks in DQN were introduced specifically to reduce the moving target problem. From this point of view, it's not surprising that if we were to add more artificial variance through other means again (such as dropout), that this would hurt performance / destabilize learning.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there other mechanisms to try and deal with overfitting? Or in many RL examples does it not matter? e.g. there may only be one true way to the ultimate high score in the 'breakout' game, so you might as well learn that exactly, and no need to generalise?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the majority of current (Deep) Reinforcement Learning research, overfitting is indeed not viewed as a problem. The vast majority of RL research consists of training in one environment (for example Cartpole, or Breakout, or one particular level in Pacman, or navigating in one specific maze, etc.), and either constantly evaluating performance during that learning process, or evaluating performance after such a learning process &lt;strong&gt;in the same environment&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we were to compare that evaluation methodology to what happens in supervised learning... &lt;strong&gt;we are basically evaluating performance on the training set*&lt;/strong&gt;. In supervised learning, this would be absolutely unacceptable, but in RL it is very much treated as acceptable and more rule than exception. Some say this is simply a problem in current RL research, something that needs to change. It could also be argued that it's not necessarily a problem; if we really are able to train the agent in precisely the same environment that we wish to deploy it in later... well, then what's the problem with it overfitting to that environment? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, when we're using the evaluation methodology described above, indeed we are overfitting to one specific environment, but &lt;strong&gt;overfitting is good rather than bad according to our evaluation criteria&lt;/strong&gt;. It is clear that this methodology does not lead to agents that can generalize well though; if you consistently train an agent to navigate in one particular maze, it will likely be unable to navigate a different maze after training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;*Note:&lt;/strong&gt; the truth, in my opinion, is slightly more nuanced than that we are really &quot;evaluating on the training set&quot; in RL. See, for example, this nice thread of tweets: &lt;a href=&quot;https://twitter.com/nanjiang_cs/status/1049682399980908544&quot; rel=&quot;noreferrer&quot;&gt;https://twitter.com/nanjiang_cs/status/1049682399980908544&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have created an environment that simulates currency prices and a simple agent, using DQN, that attempts to learn when to buy and sell. Training it over almost a million timesteps taken from a specific set of data consisting of one month's worth of 5-minute price data it seems to overfit a lot. If I then evaluate the agents and model against a different month's worth of data is performs abysmally. So sounds like classic overfitting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that your evaluation methodology described here indeed no longer fits the more &quot;common&quot; evaluation methodology. You have a problem with &lt;strong&gt;concept drift&lt;/strong&gt;, with nonstationarity in the environment. This means overfitting may be a problem for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, I'm not sure if dropout would help (it's still additional variance which may hurt). First and foremost, you'd want to make sure that there's some way to keep track of the time / month in your inputs, such that you'll at least have a chance of learning a policy that adapts itself over time. If you have a clear, solid boundary between &quot;training phase&quot; and &quot;evaluation phase&quot;, and you know that concept drift occurs across that boundary (you know that your environment behaves differently in the training phase from the evaluation phase)... you really don't have much hope of learning a policy only from experience in the training phase that still performs well in the evaluation phase. I suspect you'll have to get rid of that clear, solid boundary. You'll want to keep learning throughout the evaluation phase as well. This enables your learning algorithm to actually collect experience in the changed environment, and adapt to it.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-10-09T17:32:58.447" LastActivityDate="2018-10-09T17:32:58.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8296" PostTypeId="1" CreationDate="2018-10-07T14:48:28.470" Score="5" ViewCount="122" Body="&lt;p&gt;Many people know the most common advantages of autonomous (not autopilot-controlled) cars, such as safety, efficiency, or parking.&lt;/p&gt;&#xA;&lt;p&gt;But what are the legal advantages of autonomous vehicles? Normally, you only hear about problems like having to change a lot of the legal aspects.&lt;/p&gt;&#xA;&lt;p&gt;I would have thought that there was no hit and run, drunk driving, and generally no deliberately caused car accidents.&lt;/p&gt;&#xA;&lt;p&gt;I would be very interested if someone would come up with further advantages.&lt;/p&gt;&#xA;" OwnerUserId="18839" LastEditorUserId="2444" LastEditDate="2020-12-27T13:09:17.753" LastActivityDate="2020-12-27T13:09:17.753" Title="What are the legal advantages of autonomous vehicles?" Tags="&lt;autonomous-vehicles&gt;&lt;legal&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2020-12-27T13:13:35.997" ContentLicense="CC BY-SA 4.0" />
  <row Id="8297" PostTypeId="1" CreationDate="2018-10-07T17:37:00.000" Score="2" ViewCount="41" Body="&lt;p&gt;I'm programming on Connect6 with MCTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo Tree Search is based on random moves. It counts up the number of wins in certain moves. (Whether it wins in 3 turns or 30 turns)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the move with less turns more powerful than the move with more turns?(as mcts just sees if it's win or not -- not considering the number of turns it took to win) And if so, is it meaningful to give bigger weight to the one with less turn win?&lt;/p&gt;&#xA;" OwnerUserId="18844" LastEditorUserId="1641" LastEditDate="2018-10-08T19:35:36.270" LastActivityDate="2018-11-07T20:00:03.040" Title="Is it meaningful to give more weight to the result of monte carlo search with less turn win?" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8298" PostTypeId="1" CreationDate="2018-10-07T17:42:25.997" Score="3" ViewCount="5153" Body="&lt;p&gt;I have a saved keras model.&#xA;How can I get back the labels from the model ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because right now, I can use the predict method to get back the probability for a sample to belong to a certain class e.g. class 1, 2, 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can I know what class 1, 2, 3 correspond to in the model e.g. cat, dog, bird ?&lt;/p&gt;&#xA;" OwnerUserId="18845" LastActivityDate="2018-10-11T12:57:35.170" Title="Keras : get back labels from a model" Tags="&lt;classification&gt;&lt;keras&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8299" PostTypeId="2" ParentId="8298" CreationDate="2018-10-07T18:14:25.777" Score="1" Body="&lt;p&gt;You can use the &lt;code&gt;argmax&lt;/code&gt; function to get the class labels. Please refer to this &lt;a href=&quot;https://stackoverflow.com/questions/38971293/get-class-labels-from-keras-functional-model/45176824#45176824&quot;&gt;link&lt;/a&gt; for more information. Hope this answers your question.&lt;/p&gt;&#xA;" OwnerUserId="18846" LastActivityDate="2018-10-07T18:14:25.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8300" PostTypeId="2" ParentId="8297" CreationDate="2018-10-07T19:46:39.377" Score="2" Body="&lt;p&gt;Traditionally (when not considering your idea), the evaluation function for terminal game states would be implemented to return &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for wins, draws, or losses, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing that in a naive/straightforward way to make short-term wins more rewarding, long-term wins less rewarding, short-term losses more negative, and long-term losses less negative can be dangerous, it may change the objective that your agent is ultimately optimizing for (i.e. may lose the guarantee of converging towards optimal play given an infinite amount of time) if not done very carefully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is definitely value in considering the idea though, especially because in the Play-Out phase of MCTS, trajectories of (semi-)random moves introduce uncertainty in the evaluations at the end of those simulations, and this uncertainty increases as the length of the trajectories increases (due to increased number of uninformed decisions being made along the trajectory). Note that it is especially important to take into consideration here the number of moves played in the Play-Out phase, not necessarily including the number of moves made in the Selection phase (which are selected according to a much more informed strategy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One paper I know of that investigates ideas along these lines is &lt;a href=&quot;https://dke.maastrichtuniversity.nl/m.winands/documents/QB_ECAI2014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Quality-based Rewards for Monte-Carlo Tree Search Simulations&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-10-07T19:46:39.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8302" PostTypeId="2" ParentId="8026" CreationDate="2018-10-07T21:50:40.277" Score="1" Body="&lt;p&gt;There are a lot of other potential applications. It's a good idea to start with GPU related problems, since GPUs are essentially doing a slightly wider set of operations, slightly slower. Some possible problems where TPUs might be advantageous are:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Shader&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shaders&lt;/a&gt; are algorithms for rendering graphics in one style or another. Since computer graphics can be understood as mostly linear algebra, it is natural to view this as operations over tensors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamical_simulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Physical simulations&lt;/a&gt;, which again involve multiplication of vectors by a series of matrices. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lattice_model_(finance)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Options Pricing&lt;/a&gt;, which again involves the multiplication of vectors by a series of matrices, especially when more complex derivatives are prices and the lattice becomes multi-dimensional. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Within AI, there are many other algorithms optimized to work with GPUs, and that could be modified to work with tensor specific hardware. For example, we have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1601.00221&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming on GPUs&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-37291-9_13&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiagent Systems on GPUs&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://fenix.tecnico.ulisboa.pt/downloadFile/395146459368/thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAT solving on GPUs&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There does not yet seem to be much work optimizing these other problems for Tensor Processing units, but TPUs are also not yet very old. It took several years following the availability of inexpensive consumer GPUs before we started to see widespread use in AI. I suspect we will see more TPU-tailored code for other problems soon.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-10-07T21:50:40.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8303" PostTypeId="1" AcceptedAnswerId="8308" CreationDate="2018-10-07T23:58:40.180" Score="1" ViewCount="64" Body="&lt;p&gt;I want to implement a neural network on a big dataset. But training time is long (~1h30 per epoch). I'm still in the development process, so I don't want to wait such long time just to have poor results at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.slavv.com/37-reasons-why-your-neural-network-is-not-working-4020854bd607&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; and &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/5pidk2/d_is_overfitting_on_a_very_small_data_set_a/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; suggest that overfitting the network on a very small dataset (1 ~ 20 samples) and reach a loss near 0 is a good start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did it and it works great. However, I am looking for the next step of validating my architecture. I tried to overfit my network over 100 samples, but I can't reach a loss near 0 in reasonable time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I ensure the results given by my NN will be good (or not), without having to train it on the whole dataset ?&lt;/p&gt;&#xA;" OwnerUserId="18852" LastActivityDate="2018-10-08T14:43:59.020" Title="How to detect a Neural Network will work with the whole dataset?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8304" PostTypeId="2" ParentId="8303" CreationDate="2018-10-08T05:37:33.367" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I ensure the results given by my NN will be good (or not), without having to train it on the whole dataset ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in diagnostic techniques for neural networks, read &lt;a href=&quot;https://arxiv.org/pdf/1707.09725&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 2.5 of my Master's thesis&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-10-08T05:37:33.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8305" PostTypeId="2" ParentId="6994" CreationDate="2018-10-08T05:49:08.947" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I would also like to be able to start with a predefined set of classes (or clusters/centroids) as I know for a fact what the types of those emails will be.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is not a clustering problem, but a semi-supervised learning problem. If you don't have labeled data yet, then create some labels. You might also want to look into &quot;active learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For each category, create 5 labeled samples&lt;/li&gt;&#xA;&lt;li&gt;Train a classifier on them (e.g. tf-idf features and a small neural network)&lt;/li&gt;&#xA;&lt;li&gt;Let the neural network label your dataset&lt;/li&gt;&#xA;&lt;li&gt;Check the labels where it was most confident for all classes and the ones where the probabilities for all classes were most evenly spread. Use this to quickly create more labels.&lt;/li&gt;&#xA;&lt;li&gt;Maybe Amazon mechanical Turk is an option to quickly generate more labels&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-10-08T05:49:08.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8306" PostTypeId="2" ParentId="8290" CreationDate="2018-10-08T05:58:38.623" Score="2" Body="&lt;p&gt;I usually start with some papers and look at the references:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Counting people using video cameras&lt;/li&gt;&#xA;&lt;li&gt;Sheng-Fuu Lin, Jaw-Yeh Chen, Hung-Xin Chao, Estimation of Number of People in Crowded Scenes Using Perspective Transformation,  IEEE Transactions on Systems, Man and Cybernetics, November 2001, Part A, Vol. 31, Issue 6, pp. 645-654. &lt;/li&gt;&#xA;&lt;li&gt;A.  C.  Davies,  J.  H.  Yin,  and S.  A.  Velastin,  Crowd monitoring using image processing,  Electronics  &amp;amp; Communication Engineering Journal, February  1995, Vol. 7, Issue 1, pp. 37-47.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;By this technique you only find older work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find more recent articles, use Google scholar and find who cites the articles you investigated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Always write them down (e.g. in your bibtex file) to keep track of what you looked at already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-10-08T07:48:56.237" LastActivityDate="2018-10-08T07:48:56.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8307" PostTypeId="2" ParentId="8303" CreationDate="2018-10-08T14:17:39.963" Score="1" Body="&lt;p&gt;Don't know anything about your dataset, but maybe by using clustering* on it, you can get N &quot;most distinct&quot; examples and train only on them. This obviously will not give you same performance as if network would have seen all examples, but this way at least you will show it &quot;diverse&quot; examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*That is, of course, if you have time for that.&lt;/p&gt;&#xA;" OwnerUserId="18866" LastActivityDate="2018-10-08T14:17:39.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8308" PostTypeId="2" ParentId="8303" CreationDate="2018-10-08T14:43:59.020" Score="2" Body="&lt;p&gt;You can try to train it on 1% data, then on 2%, 3%, etc... Then plot it and see if increasing data increases the performance and how it is changing. Not sure if that's correct answer, but at least you can iterate this method pretty fast.&lt;/p&gt;&#xA;" OwnerUserId="18808" LastActivityDate="2018-10-08T14:43:59.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8309" PostTypeId="1" AcceptedAnswerId="8587" CreationDate="2018-10-08T15:15:23.527" Score="5" ViewCount="142" Body="&lt;p&gt;I'm learning about multilayer perceptrons, and I have a quick theory question in regards to hidden layer neurons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know we can use two hidden layers to solve a non-linearly separable problem by allowing for a representation with two linear separators. However, we can solve a non-linearly separable problem using &lt;strong&gt;&lt;em&gt;only one&lt;/em&gt;&lt;/strong&gt; hidden layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems fine, but what kind of representation does one hidden layer add? And how is the output of the network affected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've drawn a diagram of a multilayer perceptron with one hidden layer neuron. I used this same layout to solve a non-linearly separable problem. The single hidden layer node is inside the red square. Forgive my poor MS-Paint skills.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qGm17.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qGm17.png&quot; alt=&quot;simple perceptron&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18870" LastEditorUserId="2444" LastEditDate="2020-03-11T00:33:19.077" LastActivityDate="2020-03-11T00:36:36.360" Title="How does a single hidden layer affect output?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;hidden-layers&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-03-11T00:42:37.080" ContentLicense="CC BY-SA 4.0" />
  <row Id="8310" PostTypeId="1" CreationDate="2018-10-08T15:55:59.107" Score="1" ViewCount="1135" Body="&lt;p&gt;I'm trying to apply the &lt;a href=&quot;https://worldmodels.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Models&lt;/a&gt; architecture to the Sonic game (using the &lt;strong&gt;gym-retro&lt;/strong&gt; library).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem concerns the evolutionnary algorithm part that I use as the controller (worldmodels = auto encoder + RNN + controller).&#xA;I'm using a genetic algorithm called &lt;strong&gt;NEAT&lt;/strong&gt; (I use the &lt;strong&gt;neat-python&lt;/strong&gt; library). I am searching for someone who can help me with the neat-python implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the method that runs a generation :&#xA;&lt;code&gt;python&#xA;    best_genome = pop.run(popEvaluator.evaluate_genomes, 1)&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, all the individuals of the population are evaluated on the first level of Sonic The HedgeHog.&#xA;The &quot;run&quot; method should return the best genome of the population based on their performance on this level.&#xA;Then, I use this best genome to re-create the associated neural network in order to run it in the same level.&#xA;I was expected to see the exact same run as the best individual, but this is not the case.&#xA;Sometimes it does, sometimes not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are not a lot of examples with NEAT and I based my code on &lt;a href=&quot;https://github.com/CodeReclaimers/neat-python/blob/master/examples/openai-lander/evolve.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; from the official documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/CamilleChiquet/Sonic-World-Models-Keras/blob/master/neat_sonic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is my own implementation, if you want to check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anybody has already used NEAT, help would be welcome !&lt;/p&gt;&#xA;" OwnerUserId="18872" LastActivityDate="2018-10-08T15:55:59.107" Title="NEAT + Keras : reproducibility problem (World Models implementation)" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;keras&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8313" PostTypeId="5" CreationDate="2018-10-08T20:57:07.163" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-08T20:57:07.163" LastActivityDate="2018-10-08T20:57:07.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8314" PostTypeId="4" CreationDate="2018-10-08T20:57:07.163" Score="0" Body="For questions related to the use of AI for the control of the quality of software development, a design process, a manufacturing process, a physical process, or another product, service, system, or process." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-09T18:06:33.803" LastActivityDate="2018-10-09T18:06:33.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8315" PostTypeId="5" CreationDate="2018-10-08T20:59:54.240" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-08T20:59:54.240" LastActivityDate="2018-10-08T20:59:54.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8316" PostTypeId="4" CreationDate="2018-10-08T20:59:54.240" Score="0" Body="For questions related to the design of embedded systems that employ AI, the use of AI to create embedded systems, or the design of the AI that is used in an embedded system. This includes robotics for consumers in residential, business, or industry environments and other types of embedded devices." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-09T18:06:25.820" LastActivityDate="2018-10-09T18:06:25.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8319" PostTypeId="2" ParentId="2126" CreationDate="2018-10-09T01:52:15.453" Score="0" Body="&lt;p&gt;Autonomous vehicles are dependent upon AI technology in that, to be autonomous in their driving or piloting, they cannot be controlled by people.  Therefore they must make complex decisions required of drivers and pilots at least as safely and reliably as human drivers or pilots.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They must recognize objects to the degree that both the value and the typical behavior can be assigned to those objects (i.e. people, pets, property, barriers, curbs, grass, trees, bridges)&lt;/li&gt;&#xA;&lt;li&gt;They must map trajectories of a wide array of object types based on their object type, what is known about that type of object, detectable variations such as age or condition, and what the object appears to be involved in doing at the time.&lt;/li&gt;&#xA;&lt;li&gt;They must be able to acquire publicly available representations of drive-able roads (route segments, connection points, and other data), match the representation with the current state of the roads, and track their progress along an intended route to the destination.&lt;/li&gt;&#xA;&lt;li&gt;They must plan their course in lieu of these real time and difficult to predict actions, traffic law, traffic conventions, traffic signs and signals, given destination, known possible routes, discontinuities, and anomalies.&lt;/li&gt;&#xA;&lt;li&gt;They must be able to alter the plan to reach the destination if at all possible regardless of changes and challenges encountered.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Driving or piloting a vehicle is an intelligence intensive task.  The only reason AVs will likely surpass human driven vehicles on the road in the near future in terms of the distributions of rates of fatalities and injuries per million meter of travel in the near future is because humans have two key handicaps that offset their intelligence potential as drivers.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Carelessness, as defined as multitasking either mentally or physically at a time when hazards might appear&lt;/li&gt;&#xA;&lt;li&gt;Selfishness, as defined as risking the life, health, or property of others to gain a transportation related or psychologically related advantage&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although the above two appear to be subjective, they can be easily proven empirically by taking a sample of traffic patterns at any point in time in any highly trafficked road in the world.  This is less true of pilots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We should not presume that artificial intelligence in AVs is achieved when the behavior of the human mind is copied.  That is the criteria for Alan Turing's Imitation Game, a test that was intended to define intelligence in the context of natural language dialog.  But words don't normally kill people directly.  Vehicles often do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be a very limited vision the potential AV design space to consider human minds as the model of driving excellence.  The tasks should not be performed in the same way by the AI system.  The AI design objectives of AVs should be more consistent with these concerns and interests.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Road or sky safety laws&lt;/li&gt;&#xA;&lt;li&gt;Ethics regarding right of way in normal and emergency situations&lt;/li&gt;&#xA;&lt;li&gt;Civil rights concerns in terms of equal access to public resources&lt;/li&gt;&#xA;&lt;li&gt;Balancing of spacial flow details to maximize transportation throughput&lt;/li&gt;&#xA;&lt;li&gt;Collision aversion when difficult to predict risks emerge&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These requirements on the cognitive and adaptive capabilities of the driving or piloting AI are not solely rule-based and mechanical.  The vehicle itself is mostly mechanical in its operation, but it too presents surprises like blowouts or other difficult to predict failures.  Vehicle control is not at all like chess or a game with a fixed rules of play and fixed game-play environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the intelligence requirements do NOT include self-awareness of itself as an intelligent system, there are forms of self-awareness required.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The relative position of the exterior surface of the vehicle and its projected path relative to that of other objects&lt;/li&gt;&#xA;&lt;li&gt;The condition of the operational parts of the vehicle&lt;/li&gt;&#xA;&lt;li&gt;The mass and location of passengers and any other transported objects in the vehicle&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The question ended with an interesting and challenging requirement.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Choose a good way to act in a never before experienced situation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is perhaps the most challenging aspect of AV driving or piloting system design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Returning to the question of, &quot;Why are autonomous cars categorized as AI?&quot;, the meaning of AI is indeed a critical aspect of answering well.  Taken literally, the term artificial intelligence specifies two things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is artificial, in that it does not naturally occur in nature&lt;/li&gt;&#xA;&lt;li&gt;It is intelligent, in that it adapts in ways that, if those ways are mechanical, they are mechanical at a level of detail that is beyond obviousness without considerable study&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As year dependent and culturally dependent as that definition of intelligence is, no other definition is quite as sustainable over decades from both scientific and linguistic perspectives.  By narrower definitions, AVs may not require AI, but there is no compelling scientific reason to narrow the definition of AI to a subset of this previous definition.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-09T01:52:15.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8320" PostTypeId="1" CreationDate="2018-10-09T02:19:31.433" Score="3" ViewCount="1022" Body="&lt;p&gt;Can the recurrent neural network input come from short time fourier transform in MATLAB? I mean the input is not from time series domain.&lt;/p&gt;&#xA;" OwnerUserId="18884" LastEditorUserId="17221" LastEditDate="2018-10-09T10:12:53.510" LastActivityDate="2018-10-09T10:35:17.490" Title="Fourier Transform inputs (Frequency) for RNN" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8321" PostTypeId="2" ParentId="8320" CreationDate="2018-10-09T08:28:18.033" Score="3" Body="&lt;p&gt;Yes you can apply RNN to any sequence of same data type. The sequence can be in space, time, or any arbitrary ordered list. The &lt;em&gt;items&lt;/em&gt; in the sequence can have any data at all, the only requirement is that each represents that same kind of thing (if you have multiple types of thing to process as a sequence, you just need to expand the definition so that the input features can represent all types unambiguously - essentially creating a &quot;base class&quot; that can represent them all).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The RNN will &lt;em&gt;consume&lt;/em&gt; the sequence as a time-based sequence, one item per time step of the RNN. However, you can think of that as the same as a processor clock for a computer . . . an RNN is essentially a trainable Turing machine, and in principle can learn to accumulate any data about the sequence it has seen, and output any function of that accumulated data. Although in practice this learning process might be too hard for our current systems, require immense amounts of data etc . . .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, STFT does create a time-based sequence. Each &lt;em&gt;item&lt;/em&gt; in the sequence is a frequency analysis for a short period of time, and each time step of the sequence represents a fixed time difference between STFT frames (the windows usually overlap a little), where frequencies in the signal may change. Typically each STFT frame is a single time step input to a RNN. You &lt;em&gt;could&lt;/em&gt; input the frequency-domain values in fixed order (e.g. low to high frequency) one at a time into a RNN too, but that would be unusual and would make most learning tasks harder.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-09T10:35:17.490" LastActivityDate="2018-10-09T10:35:17.490" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8323" PostTypeId="1" CreationDate="2018-10-09T12:43:29.380" Score="6" ViewCount="7125" Body="&lt;p&gt;Almost all the convolutional neural network architecture I have come across have a square input size of an image, like &lt;span class=&quot;math-container&quot;&gt;$32 \times 32$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$64 \times 64$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$128 \times 128$&lt;/span&gt;. Ideally, we might not have a square image for all kinds of scenarios. For example, we could have an image of size &lt;span class=&quot;math-container&quot;&gt;$384 \times 256$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to we handle such images during&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;training,&lt;/li&gt;&#xA;&lt;li&gt;development, and&lt;/li&gt;&#xA;&lt;li&gt;testing&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;of a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we force the image to resize to the input of the neural network or just crop the image to the required input size?&lt;/p&gt;&#xA;" OwnerUserId="17763" LastEditorUserId="2444" LastEditDate="2020-06-14T10:41:15.420" LastActivityDate="2020-06-14T10:52:26.097" Title="How to handle rectangular images in convolutional neural networks?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8324" PostTypeId="2" ParentId="8323" CreationDate="2018-10-09T13:07:45.250" Score="8" Body="&lt;p&gt;I think the squared image is more a choice for simplicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two types of convolutional neural networks&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Traditional CNNs&lt;/strong&gt;: CNNs that have fully connected layers at the end, and &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;fully convolutional networks (FCNs)&lt;/strong&gt;: they are only made of convolutional layers (and subsampling and upsampling layers), so they do not contain fully connected layers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With traditional CNNs, the inputs always need to have the same shape, because you flatten the last convolutional layer, with a fixed size. As the flatten layer has a fixed size, the feature map shape from the layer before has to be the same shape, and so the inputs (images) also have to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in FCN, you don't flatten the last convolutional layer, so you don't need a fixed feature map shape, and so you don't need an input with a fixed size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, you don't need a squared image. You just have to be careful in the case you use CNN with a fully connected layer, to have the right shape for the flatten layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if you have an input size &lt;span class=&quot;math-container&quot;&gt;$320 \times 160$&lt;/span&gt;, and you have 3 pooling layers, so your output in the last convolutional layer is &lt;span class=&quot;math-container&quot;&gt;$40 \times 20 \times c$&lt;/span&gt; (with &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; the number of filters/channels), then you just need the flatten layer to have &lt;span class=&quot;math-container&quot;&gt;$40*20*c$&lt;/span&gt; neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you create a new network, just design it to handle a rectangle image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use an already pre-trained one, I think the better choice is to resize the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the information in the cropped parts is important, maybe your prediction can be wrong (it depends if the object of interest is in the parts of the image that is cropped). Actually, in Yolo (an object recognition network), images are resized if they don't fit the input requirements. &lt;a href=&quot;https://pjreddie.com/media/files/papers/yolo.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;See figure 1 of the YOLO paper&lt;/a&gt;. It's because you don't need a high resolution to detect an object (for example, the CIFAR dataset has images of shape &lt;span class=&quot;math-container&quot;&gt;$32 \times 32$&lt;/span&gt;, but the network can still predict the correct label). So, I think that resizing your image may not affect the prediction much (unless the new size is very different from the original)&lt;/p&gt;&#xA;" OwnerUserId="17221" LastEditorUserId="2444" LastEditDate="2020-06-14T10:49:25.380" LastActivityDate="2020-06-14T10:49:25.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8326" PostTypeId="1" AcceptedAnswerId="8342" CreationDate="2018-10-09T14:59:58.643" Score="1" ViewCount="230" Body="&lt;p&gt;I am using Tensorflow CNN to build an image classification/prediction model. Currently all the images in the dataset are each about 1mb in size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most examples out there use very small images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image size seems large, but I not too sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts on the feasibility of 1mb images? If not what can I do to compress programmatically?&lt;/p&gt;&#xA;" OwnerUserId="18896" LastActivityDate="2018-11-09T11:01:00.173" Title="Is 1mb an acceptable memory size for images being trained in a CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;training&gt;&lt;tensorflow&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8327" PostTypeId="1" CreationDate="2018-10-09T15:20:53.440" Score="0" ViewCount="59" Body="&lt;p&gt;According to the &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Szegedy_Going_Deeper_With_2015_CVPR_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; on page 4, &lt;code&gt;224x224x3&lt;/code&gt; image is reduced to &lt;code&gt;112x112x64&lt;/code&gt; using a filter &lt;code&gt;7x7&lt;/code&gt; and stride &lt;code&gt;2&lt;/code&gt; after convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;n&lt;/code&gt;x&lt;code&gt;n&lt;/code&gt; = &lt;code&gt;224x224&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;f&lt;/code&gt;x&lt;code&gt;f&lt;/code&gt; = &lt;code&gt;7x7&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;stride = &lt;code&gt;s&lt;/code&gt; = &lt;code&gt;2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;padding = &lt;code&gt;p&lt;/code&gt; = &lt;code&gt;0&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output of convolution is &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;(((n+2p-f)/s)+1)&lt;/a&gt; according to which &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;(n+2p-f)&lt;/code&gt;=&lt;code&gt;(224+0-7)&lt;/code&gt;=217, &#xA;divide by stride = &lt;code&gt;217/2=108.5&lt;/code&gt; (taking the lower value), adding &lt;code&gt;118+1=119&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we get a output image of &lt;code&gt;112&lt;/code&gt; now?&lt;/p&gt;&#xA;" OwnerUserId="17763" LastActivityDate="2018-11-08T17:00:58.083" Title="Inception neural network input layer confusion" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8328" PostTypeId="1" CreationDate="2018-10-09T15:58:33.040" Score="1" ViewCount="74" Body="&lt;p&gt;I have a project, which is the keyboard biometrics of users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;suppose I have 3 users&lt;/strong&gt;, &#xA;I do not know how to label in two types of class, (+ 1, -1). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to verify the identity to &lt;strong&gt;user1&lt;/strong&gt;, my idea of ​​class designation would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       TIMES                LABEL&#xA;user 1&#xA;9.4  9.2  1.0  3.4  0.5      1&#xA;9.4  9.2  1.0  3.4  0.5      1&#xA;9.4  9.2  1.0  3.4  0.5      1&#xA;9.4  9.2  1.0  3.4  0.5      1&#xA;9.4  9.2  1.0  3.4  0.5      1&#xA;&#xA;user 2&#xA;0.1  3.2  1.0  1.2  1.7      -1&#xA;3.4  1.2  3.0  1.1  2.8      -1&#xA;2.4  2.2  3.0  1.6  2.9      -1&#xA;1.4  3.2  2.0  2.6  3.6      -1&#xA;3.4  0.2   3.0  2.7  3.5     -1&#xA;&#xA;user N&#xA;0.2  1.4  4.5  3.7  2.9      -1&#xA;9.2  1.5  7.6  2.6  2.6      -1&#xA;9.3  1.6  7.5  2.9  3.4      -1&#xA;9.8  3.8  6.6  2.8  2.5      -1&#xA;9.8  2.8  1.7  3.8  1.6      -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but as my system has more and more users classes -1 will be too many compared to classes +1,&#xA;&lt;strong&gt;How should I label the classes?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="18464" LastEditorUserId="18464" LastEditDate="2018-10-09T16:04:55.987" LastActivityDate="2018-10-11T03:20:23.150" Title="How should I label the classes in RNA?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8329" PostTypeId="2" ParentId="8327" CreationDate="2018-10-09T16:03:01.783" Score="1" Body="&lt;p&gt;The padding is not size zero* in the inception CNN layers. In fact it is deliberately chosen to pad so that the convolution by itself would produce an image the same size as the original. I.e. &lt;span class=&quot;math-container&quot;&gt;$p=(f−1)/2$&lt;/span&gt;, in some libraries this is called &quot;same&quot; padding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, &lt;span class=&quot;math-container&quot;&gt;$p=3$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stride is not 2. It is &lt;span class=&quot;math-container&quot;&gt;$s=1$&lt;/span&gt; for the convolution. The Inception CNN does not use strided convolutions. Instead the stride of 2 is associated with a later max-pooling layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, using &lt;span class=&quot;math-container&quot;&gt;$(((n+2p-f)/s)+1)$&lt;/span&gt; with the correct values &lt;span class=&quot;math-container&quot;&gt;$(((224 + 6 - 7)/1)+1 = 224$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then apply max-pooling, with stride 2. &lt;span class=&quot;math-container&quot;&gt;$224/2 = 112$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Not to be confused with &quot;zero padding&quot; which means pad using &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; as the value to insert into the new area. So you can have &quot;zero padding with &lt;span class=&quot;math-container&quot;&gt;$p=3$&lt;/span&gt;&quot;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-09T16:20:22.783" LastActivityDate="2018-10-09T16:20:22.783" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8331" PostTypeId="1" CreationDate="2018-10-09T16:32:10.183" Score="1" ViewCount="63" Body="&lt;p&gt;I have a grid of rectangles acting as blocks. The robot traverses through the inter-spaces between these consecutive blocks. Now I have sensor data streaming in representing Right and left wheel speeds. Based on the differences in the speeds of the left and right wheels, I infer the robot's position and the path it has threaded. I get the associated individual segments of the total distance when it travels straight, left, or right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These distances are a function of the actual speed of the robot and the time interval elapsed before the end of that activity. These computed distances for the segments though don't map and fit-in well when projected on the grid layout of the environment. The segments are rather not adhering to the boundary limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to know if I can use RL to force the calculated distances to fit in with the layout given certain knowledge (or conditions, if you will): the start and end position of the robot and the inter-space distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not RL, do you know how can I solve this problem? I suspect my function computing the distances is off and wondering if RL can help me figure out the right mapping of sensor data to the path traveled adhering to the grid layout dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MU9Xy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MU9Xy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider the illustration above you will notice S, D, and D' signifying the starting position, the true destination, and the destination location computed by adding together the calculated distances for each of the segments representing right(r), left(l) and straight(s) along the path towards the destination. Inter-space length is given 7m and dimensions of the blocks are (27m x 15m). If you look at the data presented on the left side you will notice 18m left and consecutive 24m right represents the activity, in the grid, as the passage through the blocks. Granted -- perhaps the car negotiates the edges and corners through this passage in a protracted left(l) and right(r) movements, without necessarily going straight(s) straddling and linking the turns as one would expect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question arises, however, when taken into account these individual segment lengths and stitch them together you end up in a destination, not in the ballpark range of the expected value. How can we design this problem so as to employ RL methods to, sort of, impose these grid dimensional constraints on this distance calculation methodology to yield better results? Or, probably best to re-imagine the whole problem so it is amenable to the application of RL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice/ insights would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16551" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:02.560" LastActivityDate="2020-05-29T13:47:02.560" Title="Reinforcement learning for segmenting the robot path to reflect the true distances" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8333" PostTypeId="1" CreationDate="2018-10-09T18:47:38.883" Score="2" ViewCount="421" Body="&lt;p&gt;I (mis?)understood the NEAT algorithm has the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create a genome pool with N random genomes&lt;/li&gt;&#xA;&lt;li&gt;Calculate each genome fitness&lt;/li&gt;&#xA;&lt;li&gt;Assign each genome to a species&lt;/li&gt;&#xA;&lt;li&gt;Calculate the adjusted fitness and the number of offspring of each species&lt;/li&gt;&#xA;&lt;li&gt;Breed each species through mutation/crossover from the stronger genomes&lt;/li&gt;&#xA;&lt;li&gt;go to step 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Step 3 is tricky: speciation is made placing each genome G in the first species in which it is &lt;em&gt;compatible&lt;/em&gt; with the &lt;em&gt;representative genome&lt;/em&gt; of that species, or in a new species if G is not compatible with any existing species. &lt;em&gt;Compatible&lt;/em&gt; is meant as having compabilitity distance below a certain threshold. Regarding &lt;em&gt;representative genome&lt;/em&gt; NEAT paper says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Each existing species is represented by a random genome inside the&#xA;  species from the &lt;em&gt;previous generation&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.ucf.edu/~kstanley/neat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Somewhere&lt;/a&gt; I've found that keeping the number of species stable is good, and this is achieved automatically with dynamic thresholding. However, dynamic thresholding makes hard to evaluate species behaviour across generations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give one example:&#xA;Assume that in Generation 20, Species 1 has Genome A as representative and Species 2 has Genome B as representative. Assume elitism is implemented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the representative genome is taken from previous generation, assume that in Generation 21, Genome A and B are still representatives for Species 1 and 2, however assume compatibility threshold has changed (i.e. bigger) in order to reach the target species number. With this change, A and B have now a compatibility distance lower than threshold and should be placed in the same Species, however they are representatives of different species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to solve this issue?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More in general, with dynamic thresholding, how to make sure species management across generations is consistent? E.g. NEAT paper also says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the maximum fitness of a species did not improve in 15 generations,&#xA;  the networks in the stagnant species were not allowed to reproduce.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How to make sure that across all 15 generations, we are still considering that same single species and this has not drastically changed (so that they are actually different 'objects'?). E.g. in the example above, if A and B are both placed in Species 1 in Generation 21, Species 2 no longer represents what it represented in Generation 20.&lt;/p&gt;&#xA;" OwnerUserId="13087" LastActivityDate="2021-02-04T00:04:30.363" Title="NEAT - Managing species across generations" Tags="&lt;neat&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8335" PostTypeId="1" CreationDate="2018-10-09T23:56:19.440" Score="1" ViewCount="38" Body="&lt;p&gt;When a human looks at a page. He notices the sets of letters are grouped together separated by white space. If the white space was replaced by another character say z, it would be harder to distinguish words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a neural network, spaces are &quot;just another character&quot;. How can we set up an RNN so it gives special importance to the difference between certain characters like white spaces and letters so that it will train faster? Assume the input is just a sequence of ASCII characters.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2018-10-09T23:56:19.440" Title="Pre priming a network for white space" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8336" PostTypeId="2" ParentId="8323" CreationDate="2018-10-10T04:28:03.833" Score="1" Body="&lt;p&gt;If you have a rectangular image and you are using existing models (or existing code), then you have to add an input pre-processing pipeline which transforms the image to standard dimensions. This is very common in computer vision and both PyTorch and Tensorflow have support for easily adding input pre-processing input pipeline for such a transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you have a fixed size rectangular image data, then you can design your own network architecture (or initial module) which takes image features into account by using asymmetric pooling and convolutions.&lt;/p&gt;&#xA;" OwnerUserId="18907" LastEditorUserId="2444" LastEditDate="2020-06-14T10:52:26.097" LastActivityDate="2020-06-14T10:52:26.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8337" PostTypeId="2" ParentId="6766" CreationDate="2018-10-10T05:21:59.100" Score="1" Body="&lt;p&gt;In the field of Automatic Speech Recognition (ASR) &lt;a href=&quot;https://github.com/kaldi-asr/kaldi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaldi&lt;/a&gt; is the current leader. Before Deep Neural Network era there were &lt;a href=&quot;https://cmusphinx.github.io/wiki/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sphinx&lt;/a&gt; and &lt;a href=&quot;http://htk.eng.cam.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTK&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="18908" LastActivityDate="2018-10-10T05:21:59.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8338" PostTypeId="1" AcceptedAnswerId="8340" CreationDate="2018-10-10T05:45:15.603" Score="1" ViewCount="89" Body="&lt;p&gt;As shown below, my deep neural network is overfitting :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AfDBu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AfDBu.png&quot; alt=&quot;Cross-entropy loss&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/janOU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/janOU.png&quot; alt=&quot;Accuracy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;where the &lt;strong&gt;blue&lt;/strong&gt; lines is the metrics obtained with &lt;strong&gt;training&lt;/strong&gt; set and &lt;strong&gt;red&lt;/strong&gt; lines with &lt;strong&gt;validation&lt;/strong&gt; set&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there anything I can infer from the fact that the accuracy on the training sets is really high&lt;/strong&gt; (almost 1) &lt;strong&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, it means that the complexity of my model is enough / too big. But does it means my model could theoretically reach such a score on validation set with same dataset and appropriate hyperparameters ? With same hyperparameters but bigger dataset ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is &lt;strong&gt;not&lt;/strong&gt; how to avoid overfitting.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastActivityDate="2018-10-10T08:32:38.880" Title="Interpretation of a good overfitting score" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8339" PostTypeId="1" CreationDate="2018-10-10T07:02:05.440" Score="4" ViewCount="86" Body="&lt;p&gt;I have a dataset of images belonging to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; classes, &lt;span class=&quot;math-container&quot;&gt;$A_1, A_2...A_n,B_1,B_2...B_m$&lt;/span&gt; and I want to train a CNN to classify them. The classes can be considered as subclasses of two broader classes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, therefore the confusion between &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; is much less problematic than the confusion between &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_j$&lt;/span&gt;. Therefore I want the CNN to be trained in such a way that the difference between &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_j$&lt;/span&gt; is considered as more relevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Are there any loss functions that take this requirement into account? Could a weighted cross entropy work in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How this loss would change if the classes were unbalanced?&lt;/p&gt;&#xA;" OwnerUserId="16671" LastEditorUserId="16671" LastEditDate="2018-10-11T07:46:19.437" LastActivityDate="2018-10-11T13:33:48.357" Title="How to define a loss function for a classifier where the confusion between some classes is more important than the confusion between others?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8340" PostTypeId="2" ParentId="8338" CreationDate="2018-10-10T08:32:38.880" Score="0" Body="&lt;p&gt;It doesn't tell you very much, to be honest. It &lt;em&gt;does&lt;/em&gt; mean that (assuming your training and validation distributions are similar) your model could get the same results on your validation set should you train on that, but that would still be overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, the only useful thing overfitting tells you is that you don't have enough regularisation.&lt;/p&gt;&#xA;" OwnerUserId="16035" LastActivityDate="2018-10-10T08:32:38.880" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8342" PostTypeId="2" ParentId="8326" CreationDate="2018-10-10T10:03:11.833" Score="1" Body="&lt;p&gt;1 MB by images is too much. It means that you have a lot of pixels to compute in the inputs, and your images have more features that are not very useful for the classification (we human don't need high resolution images for recognize objects, it's the same for model)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It means that maybe you need also a more deeper network, and then, more computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your particular case, you have a dataset of trees, so images are somewhat similar and then, maybe the classification is harder. So, images need enough information in pixels to have a good prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should resize your whole datsets to more common size because 1mb are too much (I don't know the resolution, but I think it could be around thousand for both dimension). &#xA;Imagenet have images of size 224x224, and there are 1000 classes, which some of them are close. For examples, there are many dogs classes (boxer, american terrier, ...), cats classes (tiger, egyptian, ....), thus I think size of a few hundred would be enough. &#xA;In your comment you said you will try with 336x252, I think it can be a good start.&#xA;Of course you will need some experimentation in this regard. Maybe you can try to train some models in a few epoches with different image sizes, to see the best one, and keep it to train the model further !&lt;/p&gt;&#xA;" OwnerUserId="17221" LastActivityDate="2018-10-10T10:03:11.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8343" PostTypeId="2" ParentId="8291" CreationDate="2018-10-10T11:15:46.327" Score="2" Body="&lt;p&gt;Fixed. Was an issue with the random generator. In my class for the Neuron layer where I initialize the weights I get new doubles from the generator for each of the initial weight values, but I found a bug where I was re-initializing the random generator, which was of course causing different values.&lt;/p&gt;&#xA;" OwnerUserId="18818" LastEditorUserId="18818" LastEditDate="2018-10-17T04:05:46.387" LastActivityDate="2018-10-17T04:05:46.387" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8344" PostTypeId="1" CreationDate="2018-10-10T18:00:40.413" Score="3" ViewCount="195" Body="&lt;p&gt;I'm trying to create a simple Dyna-Q agent to solve small mazes, in python. For the Q function, Q(s, a), I'm just using a matrix, where each row is for a state value, and each column is for one of the 4 actions (up, down, left, right).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've implemented the &quot;real experience&quot; part, which is basically just straightforward SARSA. It solves a moderately hard (i.e., have to go around a few obstacles) mazes in 2000-8000 steps (in the first episode, it will no doubt decrease with more). So I know that part is working reliably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, adding the part that simulates experience based on what it knows of the model to update the Q values more, I'm having trouble. The way I'm doing it is to keep an &lt;code&gt;experiences&lt;/code&gt; list (a lot like experience replay), where each time I take real action, I add its (S, A, R, S') to that list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, when I want to simulate an experience, I take a random (S, A, R, S') tuple from that list (David Silver mentions in his lecture (#8) on this that you can either update your transition probability matrix P and reward matrix R by changing their values or just sample from the experience list, which should be equivalent). In my case, with a given S and A, since it's deterministic, R and S' are also going to be the same as the ones I sampled from the tuple. Then I calculate Q(S, A) and max_A'(Q(S', A')), to get the TD error (same as above), and do stochastic gradient descent with it to change Q(S, A) in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's not working. When I add simulated experiences, it &lt;em&gt;never&lt;/em&gt; finds the goal. I've tried poking around to figure out why, and all I can see that's weird is that the Q values continually increase as time goes on (while, without experiences, they settle to correct values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any advice about things I could try? I've looked at the sampled experiences, the Q values in the experience loop, the gradient, etc... and nothing really sticks out, aside from the Q values growing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: here's the code. The first part (one step TD learning) is working great. Adding the planning loop part screws it up.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def dynaQ(self, N_steps=100, N_plan_steps=5):&#xA;&#xA;    self.initEpisode()&#xA;    for i in range(N_steps):&#xA;        #Get current state, next action, reward, next state&#xA;        s = self.getStateVec()&#xA;        a = self.epsGreedyAction(s)&#xA;        r, s_next = self.iterate(a)&#xA;        #Get Q values, Q_next is detached so it doesn't get changed by the gradient&#xA;        Q_cur = self.Q[s, a]&#xA;        Q_next = torch.max(self.Q[s_next]).detach().item()&#xA;        TD0_error = (r + self.params['gamma']*Q_next - Q_cur).pow(2).sum()&#xA;        #SGD&#xA;        self.optimizer.zero_grad()&#xA;        TD0_error.backward()&#xA;        self.optimizer.step()&#xA;        #Add to experience buffer&#xA;        e = Experience(s, a, r, s_next)&#xA;        self.updateModel(e)&#xA;&#xA;        for j in range(N_plan_steps):&#xA;&#xA;            xp = self.experiences[randint(0,len(self.experiences)-1)]&#xA;            Q_cur0 = self.Q[xp.s, xp.a]&#xA;            Q_next0 = torch.max(self.Q[xp.s_next]).detach().item()&#xA;            TD0_error0 = (xp.r + self.params['gamma']*Q_next0 - Q_cur0).pow(2).sum()&#xA;&#xA;            self.optimizer.zero_grad()&#xA;            TD0_error0.backward()&#xA;            self.optimizer.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18920" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:15.733" LastActivityDate="2020-05-29T13:47:15.733" Title="Dyna-Q algorithm, having trouble when adding the simulated experiences" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8345" PostTypeId="1" CreationDate="2018-10-10T20:38:33.697" Score="2" ViewCount="40" Body="&lt;p&gt;I'm trying to build a chat analysis that could identify the intent of check price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has any kind of preset intent list to train my chatbot already been done ?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data: A data set for &quot;check price&quot; intent's. Like &quot;How much are ____&quot;&lt;/li&gt;&#xA;&lt;li&gt;Context:I looking for a data set to train a intent recognization for check price at chat bot. Exemple: If some one ask for a product price. I want to tag this is as check price intent . For that I am trainning a intent recognition.&lt;/li&gt;&#xA;&lt;li&gt;Region: Prefirencal the whole globe,but for start could be only at English&lt;/li&gt;&#xA;&lt;li&gt;License:Could both payed or free&lt;/li&gt;&#xA;&lt;li&gt;Non-answers: I didnt find nothing like a dataset for intent&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18925" LastEditorUserId="18925" LastEditDate="2018-10-15T11:21:24.453" LastActivityDate="2018-10-15T11:21:24.453" Title="Preset Intent list" Tags="&lt;natural-language-processing&gt;&lt;datasets&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8346" PostTypeId="2" ParentId="8219" CreationDate="2018-10-10T22:32:13.157" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can a brain be intelligent without a body?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. Don't forget that the main function of the brain is to provide homeostasis between the body and the environment. Without the body, the utility of the brain is no longer relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, why consider intelligence only in the brain? How far does our body extend? &lt;a href=&quot;https://en.wikipedia.org/wiki/Embodied_cognitive_science&quot; rel=&quot;nofollow noreferrer&quot;&gt;Embodied cognitive science&lt;/a&gt; asks us to consider our entire body and its &#xA; surrounding extensions utility on the intelligent faculty; The fact that our thumbs stick out in a direction different to our other four fingers allows us to grab things in a effortless way is itself, intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this understanding then, the segregation between the intelligent faculty and the &quot;non-intelligent&quot; seems to be murky at best. We'd might as well not consider it unless there is better motivation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does intelligence require a context?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Pragmatically speaking, yes. Intelligent behaviour is typically understood as being goal-directed and intentional. Intentionality implies some sort of agency. It being an agent, implies some sort of agent/environment relationship, which implies an environment to act as context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, Karl Friston notes &lt;a href=&quot;https://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20principle%20A%20unified%20brain%20theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in his review&lt;/a&gt; that a general principle that the brain (and thus one would imagine, intelligent behaviour) entails is reduction of thermodynamic free energy while maintaining homeostasis with its environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This hints at a promise to describe intelligent behaviour purely in terms of thermodynamic processes, but interpretively what this means in a general language is still unclear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, keep in mind that there is no guarantee that we'd recognize a agent as intelligent if its construction is radically different from ours. &lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2018-10-10T22:49:06.927" LastActivityDate="2018-10-10T22:49:06.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8347" PostTypeId="2" ParentId="7379" CreationDate="2018-10-10T23:15:47.653" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;... the neural network has to learn the transition probability between two keyframes. And a second neural network can then produce the motion plan which is also be trained by a large corpus. Is that idea possible or is it the wrong direction?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That would be trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More difficult, but not hard, would be to take a 14 hour video of someone and create a new video with them saying something they have never said previously. Here is a demonstration, from the BBC YouTube channel, titled: &quot;&lt;a href=&quot;https://www.youtube.com/watch?v=AmUC4m6w1wo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fake Obama created using AI video tool - BBC News&lt;/a&gt;&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The University of Washington demonstrates, &lt;strong&gt;using a variety of clips and&lt;/strong&gt; &lt;strong&gt;&lt;em&gt;only&lt;/em&gt;&lt;/strong&gt; &lt;strong&gt;audio from an impersonator&lt;/strong&gt;, that &lt;a href=&quot;https://www.washington.edu/news/2017/07/11/lip-syncing-obama-new-tools-turn-audio-clips-into-realistic-video/&quot; rel=&quot;nofollow noreferrer&quot;&gt;they can create realistic videos of the former president saying anything they want&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible to sketch out an idea and let AI create a finished product of excellent quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The YouTube channel ACMSIGGRAPH has a video titled: &quot;&lt;a href=&quot;https://www.youtube.com/watch?v=t952yS8tcg8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technical Papers Preview: SIGGRAPH 2018&lt;/a&gt;&quot;, which is discussed on the webpage: &quot;&lt;a href=&quot;https://www.cartoonbrew.com/tools/could-these-be-the-next-high-tech-tools-that-animators-use-daily-158630.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Could These Be The Next High-Tech Tools That Animators Use Daily?&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you have asked about was possible long ago and not particularly difficult to do.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2018-10-10T23:15:47.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8348" PostTypeId="1" AcceptedAnswerId="8472" CreationDate="2018-10-10T23:45:00.113" Score="5" ViewCount="1964" Body="&lt;p&gt;When extending reinforcement learning to the continuous states, continuous action case, we must use function approximators (linear or non-linear) to approximate the Q-value.  It is well known that non-linear function approximators, such as neural networks, diverge aggressively.  One way to help stabilize training is using reward clipping.  Because the temporal difference Q-update is a bootstrapping method (i.e., uses a previously calculated value to compute the current prediction), a very large previously calculated Q-value can make the current reward relatively minuscule, thus making the current reward &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; impact the Q-update, eventually leading the agent to diverge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid this, we can try to avoid the large Q-value in the first place by clipping the reward between [1, -1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I have seen some other people say that instead of clipping the reward itself, we can instead clip the Q-value between an interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering which method is better for convergence, and under what assumptions / circumstances.  I was also wondering if there are any theoretical proofs/explanations about reward/Q-value clipping and which one being better. &lt;/p&gt;&#xA;" OwnerUserId="17706" LastActivityDate="2018-10-16T19:14:20.127" Title="Should the reward or the Q value be clipped for reinforcement learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;value-iteration&gt;&lt;reward-clipping&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8349" PostTypeId="2" ParentId="8328" CreationDate="2018-10-11T03:20:23.150" Score="3" Body="&lt;p&gt;From a purely dataset-development perspective I would just label the classes with numbers starting from 0 (ID of the first person) to N (ID of the last person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During training however, what you do with those classes will vary depending on the type of architecture you are training.  For example, if you are building a neural-network classifier you could simply have multiple output nodes.  When running backprop with a label for the ith person you could put a 1 on the ith output layer and -1 on the rest of them.  &lt;a href=&quot;https://stackoverflow.com/questions/13572722/can-you-use-multiple-output-layers-in-a-neural-network-without-using-softmax&quot;&gt;This is similar to this answer as well&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever you end up doing, try not to label your data to the benefit of the algorithm you're trying to run unless you really know what you're doing.  You don't want to accidentally lose information or make your training data less interpretable and harder to deal with in the long run.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2018-10-11T03:20:23.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8350" PostTypeId="1" AcceptedAnswerId="8405" CreationDate="2018-10-11T05:13:14.063" Score="3" ViewCount="52" Body="&lt;p&gt;Would this work at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Idea is to start training a neural net with some number of nodes. Then, add some new nodes and more layers and start training only the new nodes (or only modifying the old nodes very slightly). Ideally, we would connect all old nodes to the new layer added since we might have learned many useful things in the hidden layers. Then repeat this many times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuition is that if the old nodes give bad information the new layer of nodes will weight the activations of old nodes close to zero and learn new/better concepts in the new nodes. The benefit is that we will keep old knowledge forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Caveat is that the network can still temporarily &quot;forget&quot; concepts if a new layer weights old information close to zero, but it can potentially remember it again too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this completely fails, I'm curious if there's some known way to prevent a neural network from forgetting concepts it learned.&lt;/p&gt;&#xA;" OwnerUserId="18936" LastActivityDate="2018-10-14T14:22:00.670" Title="Would this work to prevent forgetting: train a neural net with N nodes. Then, add more nodes and stop training the original nodes" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8353" PostTypeId="2" ParentId="8284" CreationDate="2018-10-11T10:21:03.643" Score="1" Body="&lt;p&gt;I think your idea would work... fine, but I don't necessarily see any advantages to it. I haven't actually tried it (that'd be the best way for you to also see whether it works!), so I'm mostly going by first thoughts and intuition here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, what you are essentially doing with your idea is &quot;cutting off&quot; the last layer of the Neural Network from the perspective of your learning algorithm (typically backpropagation). Whatever weights you have between the last hidden layer and the output layer will be fixed to their initial values. The last hidden layer can actually be viewed as an &quot;output&quot; layer, since you also have fixed targets that you want to converge towards for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether this makes your learning process better/faster/easier, or worse/slower/harder seems to be very much dependent on how the weights between your last hidden layer and your output layer are initialized. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If those weights are initialized to all-zero, your &quot;real&quot; output layer is doomed to always predict zeros, so your problem becomes impossible to solve.&lt;/li&gt;&#xA;&lt;li&gt;If those weights are initialized to implement the identity function, this becomes 100% equivalent to the case you would have if you'd simply cut off the last layer and train that in the traditional sense (i.e. you effectively have one layer less than you really do).&lt;/li&gt;&#xA;&lt;li&gt;If those weights are initialized randomly, it looks to me like you have a post-processing step consisting of a random projection. Such a random projection may be beneficial for training (random projections can be useful for dimensionality reduction, or for, in combination with the subsequent non-linear function, turning an otherwise linear function into a non-linear function). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't think it'd very often be better than actually have an extra &quot;real&quot;, trainable layer with a non-linear activation function though. I suspect such a &quot;non-trainable&quot; extra layer can sometimes be better than not having anything there, but I don't think it'd often be better than having a real, trainable layer.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-10-11T10:28:00.567" LastActivityDate="2018-10-11T10:28:00.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8355" PostTypeId="2" ParentId="8284" CreationDate="2018-10-11T11:03:23.170" Score="3" Body="&lt;p&gt;The main difference your change would have is to allow you to apply a loss function to a different part of the network. This may affect training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you keep the same loss function (e.g. MSE), but apply it to the pre-transformed values, then you will have changed the objective of the network, perhaps significantly. Whether or not this is a good thing depends on how much you needed the original loss function. However, the fact that it would result in a different training target is usually going to be a bad thing if your original training target was correct. This will also be true if you pick a new arbitrary loss function that seems to fit the pre-transform representation better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you engineer a &quot;correct&quot; loss function such that the objective of the network remains unchanged, then the behaviour of the network will not change much - probably not at all. However, in some cases this can lead to more stable and/or faster training - it is often used for classifiers to avoid need to use exponentiation, see &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/softmax_cross_entropy_with_logits&quot; rel=&quot;nofollow noreferrer&quot;&gt;tf.nn.softmax_cross_entropy_with_logits&lt;/a&gt; in TensorFlow, which does exactly this.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-11T11:09:07.640" LastActivityDate="2018-10-11T11:09:07.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8357" PostTypeId="2" ParentId="8298" CreationDate="2018-10-11T12:57:35.170" Score="1" Body="&lt;p&gt;Create list of labels [&quot;dog&quot;, &quot;cat&quot;] and use &lt;code&gt;argmax&lt;/code&gt; as index to get value from list.&lt;/p&gt;&#xA;" OwnerUserId="8431" LastActivityDate="2018-10-11T12:57:35.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8358" PostTypeId="2" ParentId="8339" CreationDate="2018-10-11T13:33:48.357" Score="2" Body="&lt;p&gt;&lt;strong&gt;Classic Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not only can the reliability or accuracy needs be asymmetrically distributed between category boundaries, but the asymmetry might not be describable in terms of a first degree polynomial translation of the error surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question data set are images belonging to class &lt;span class=&quot;math-container&quot;&gt;$c \in \Big( A: \{A_1, \, ..., \, A_n\} \land  B: \{B_1, \, ..., \, B_m\} \Big)$&lt;/span&gt;.  The convolutional network must be trained with a sample of labeled images to categorize with a reliability of &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt; and accuracy of &lt;span class=&quot;math-container&quot;&gt;$1 - \epsilon$&lt;/span&gt;, based on the PAC (probably approximately correct) framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can the loss function reflect the conceptualization of optimal training result that the reliability of categorization between &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; be of greater value than categorization between &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$i \ne j$&lt;/span&gt;, and similarly with B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Criteria X would state that the value of differentiation &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is not sub-category index dependent for differentiation between super-categories.  That is, &lt;span class=&quot;math-container&quot;&gt;$\forall \; (i, j) \; \text{where} \; i \ne j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D(A_i, B_i) = D(A_i, B_j)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Criteria Y, the categorization WITHIN a super-category is not unlike the typical symmetric case &lt;span class=&quot;math-container&quot;&gt;$\forall \; (i, j, k) \; \text{where} \; i \ne j \land i \ne k \land j \ne k$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D(A_i, A_j) = D(A_i, B_k)$&lt;/span&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;(a) Are there any loss functions that take this requirement into account?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$1_p$&lt;/span&gt; be the bit value that is 1 if and only if the label matches the su&lt;strong&gt;p&lt;/strong&gt;er-category and let &lt;span class=&quot;math-container&quot;&gt;$1_b$&lt;/span&gt; be the bit value that is 1 if and only if the label matches the super-category and the su&lt;strong&gt;b&lt;/strong&gt;-category.  Because of Criteria X, a conical and normalized loss function could then be defined as thus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon = \dfrac {(1 - 1_p) + \alpha (1 - 1_b)} {1 + \alpha}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proportional value of sub-category reliability in relation to super-category reliability &lt;span class=&quot;math-container&quot;&gt;$\alpha &amp;lt; 1.0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of criteria Y, there is no meaning to concavity in this context.  Applying mean squaring to the loss simply changes the curvature and meaning of &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; in relation to the loss as can be seen here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon = \dfrac {(1 - 1_p)^2 + \alpha^2 (1 - 1_b)^2} {1 + \alpha^2} = \dfrac {(1 - 1_p) + \tau (1 - 1_b)} {1 + \tau}\text{, where }\tau = \alpha^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;(b) Could a weighted cross entropy work in this case?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Cross entropy between the two levels of categorization would be baseless unless there were correlative effects between super classification feature and sub classification feature inherent in the target concept of the learning.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;How this loss would change if the classes were unbalanced?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Although there may be schemes for modifying the loss function to compensate for skewed distribution of categories represented in the sample, this is primarily a data input problem best solved by improving data collection or choosing training algorithms, designs, architectures, treating input, and properly controlling hyper-parameters.  The loss function represents the inverse of what is considered optimal, not how one achieves it. &lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-11T13:33:48.357" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8361" PostTypeId="2" ParentId="8284" CreationDate="2018-10-11T20:37:12.757" Score="2" Body="&lt;p&gt;Be careful to specify exactly what you mean by &lt;code&gt;adjust weights and biases of the network so that the &quot;pre-activated output&quot; ... approaches ...&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When training a neural network, one minimizes a loss function. This loss function is determines how important the deviation from &lt;code&gt;0.01396&lt;/code&gt; is compared to the deviation of the other node from &lt;code&gt;-0.00524&lt;/code&gt;. By transforming the target labels backwards you should also express the original loss function in terms of the back-transformed labels.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What one can do in some cases is to combine the input to the last layer's activation function with the loss function and algebraically simplify the resulting expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This concept is for example implemented in Tensorflow's &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits&quot; rel=&quot;nofollow noreferrer&quot;&gt;tf.nn.sigmoid_cross_entropy_with_logits&lt;/a&gt;. This function can be used for the case of a single output with sigmoid activation function and binary cross-entropy loss (a similar function also exists for the case of multiple output nodes with a softmax activation function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of first passing the values through a sigmoid activation and then calculating the binary cross-entropy with respect to the target label, it combines the two expressions and uses an equivalent but simpler expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the documentation of this function, you'll see that the number of transcendental functions (which are computationally expensive to calculate) can be reduced from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;loss(x,z) = z * -log(1 / (1 + exp(-x))) + (1 - z) * -log(exp(-x) / (1 + exp(-x)))&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;loss(x,z) = max(x, 0) - x * z + log(1 + exp(-abs(x)))&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x&lt;/code&gt; is the input to the (sigmoid) nonlinearity (corresponding to the output of the green nodes in your diagram), also called 'logits' and &lt;code&gt;z&lt;/code&gt; is the target label.&lt;/p&gt;&#xA;" OwnerUserId="18159" LastActivityDate="2018-10-11T20:37:12.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8362" PostTypeId="1" CreationDate="2018-10-11T22:19:06.240" Score="2" ViewCount="35" Body="&lt;p&gt;I am interested in understanding how to choose data-acquisition parameters for the subject matter:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Frame Resolution&lt;/li&gt;&#xA;&lt;li&gt;Frame rates (FPS)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The goal is to have 'enough' (preferably the minimal) resolution and frames to enable AI to identify people. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;QUESTIONS&lt;/h1&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are there any published rules of thumb or processes to select video parameters?&lt;/li&gt;&#xA;&lt;li&gt;Is there a term or label for the selection of video parameters for AI projects?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18819" LastActivityDate="2018-10-11T22:19:06.240" Title="People Counting Video Analytics: data acquisition parameters" Tags="&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8363" PostTypeId="2" ParentId="7975" CreationDate="2018-10-11T22:55:50.927" Score="0" Body="&lt;p&gt;&lt;strong&gt;The Problem Statement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does not appear from the wording in the question that the semantics of the language in the questionnaire will be processed, so no knowledge of the associations in questionnaire questions and answers will be derived prior to its administration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; questions, with &lt;span class=&quot;math-container&quot;&gt;$N_m$&lt;/span&gt; undifferentiated answers for each question, presented to human users.  The administration of this questionnaire builds a set of data that will later be used to train an independent learning system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question pertains to both some subsequent training and the learning that may need to occur to achieve stated goals in the acquisition of data for the subsequent training.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Reduce the burden on the system users that answer the questions.&lt;/li&gt;&#xA;&lt;li&gt;Reduce the storage requirements.&lt;/li&gt;&#xA;&lt;li&gt;Remove redundancy from the data to be used as examples for subsequent training.&lt;/li&gt;&#xA;&lt;li&gt;Utilize in real time early answers within a single user session to achieve the above three objectives.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assumption in the question that the above four can and should be achieved through the same mechanism may or may not prove true during the design process.  These are some alternative ways to achieve goals.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Goal 2. can be achieved by employing bit-wise operations to pack the answer data into an innocuous payload.&lt;/li&gt;&#xA;&lt;li&gt;Goal 3. can be achieved by auto-encoding independently of 1.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Goal 1. is the key to this problem, and its solution, utilizing goal 4., should be addressed first as the predominant technical risk.  Afterward, if byproducts of the solution to goal 1. can be employed to assist in the solution of goals 2. and 3., then so be it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are at least three meanings of models in machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The model of the concept being learned&lt;/li&gt;&#xA;&lt;li&gt;The model thorough which learning occurs&lt;/li&gt;&#xA;&lt;li&gt;The model of what is considered sufficient&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This last model defines how accuracy and reliability are measured and what quality acceptance criteria are given regarding them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that none of the three models for either the data acquisition learning or the later learning using the data acquired are yet defined, however some architectural considerations can be addressed and some conclusions can be reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Structure of Information&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider these structures of data that can aid learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; features of user linguistic response can be saved within a user session as a time series of vectors &lt;span class=&quot;math-container&quot;&gt;$\sigma(m, n, t)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Those series can be collected across &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; users, each with an user ID &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; indexed sequentially for mathematical purposes by &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, as a structure of elements &lt;span class=&quot;math-container&quot;&gt;$\gamma(m, n, u, t)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is the the second structure of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; elements that contains information that can assist in the presentation of questions and answers to subsequent users, given the &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; time series in the current user's session state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Key Challenge&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge is not the application of probability and statistics to a single session, which is straightforward, outlined below.  The difficulty is that once those of the &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; questions are rated for their likely usefulness in later training, by reordering questions, defaulting them to the most likely response, or eliminating any answers, the system has changed the conditions driving the associations.  This is the classic scenario when the measurement changes the item being measured.  In such a feedback condition, the knowledge acquired about usefulness can produce unstable conditions, including oscillations or chaotic system behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Important Early Design Choice&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real time learning is indicated, and the system must be stabilized using the same kind of stabilization techniques involved in electrical engineering, robotics, and balancing.  Perhaps the best choice for current designs is Q-learning, pioneered by Watson and developed further by many others.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://s3.amazonaws.com/academia.edu.documents/50360235/Learning_from_delayed_rewards_20161116-28282-v2pwvq.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&amp;amp;Expires=1539254530&amp;amp;Signature=zNIVXcuk%2Bc%2B317VKDkNqisI3NSo%3D&amp;amp;response-content-disposition=inline%3B%20filename%3DLearning_from_delayed_rewards.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Watkin's 1989 PhD thesis &lt;em&gt;Learning from Delayed Rewards&lt;/em&gt;&lt;/a&gt; which introduces the Q learning function in context &amp;mdash; It is an excellent work that models learning in a way that is independent from whether the learning is biological or programmed on a computer, drawing on MDP (Markov decision process).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Human-level control through deep reinforcement learning&lt;/em&gt;, Mnih et. al., 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Applicable Probability and Statistics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The straightforward application of probability and statistics mentioned above is this.  Using Bayes' Theorem (not the naive categorization precisely), one can calculate the probability that any given &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; will be selected, based on the multi-user sequence of &lt;span class=&quot;math-container&quot;&gt;$\gamma(m, n, u)$&lt;/span&gt;.  However the most effective solution can be achieved via the application of Q-learning to the ordering, defaulting, and elimination of &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;, given both the multi-user sequence of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and current user sessions sequence of &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;.  Where naive Bayesian categorization or feature extraction may be most useful is in the profiling of users to further enhance user experience.  Even this can be contained within the Q-learning process.  The key to Q-learning is the inverse exponential decay of past knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Application Architecture&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most scalable architecture would be to use JavaScript in the web browser to acquire learned behavior via AJAX and adjust answers order, defaulting, and hiding in real time with the must updated learned state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more sophisticated approach would be to encapsulate the learned information in fuzzy logic rules to essentially remove redundancy and compress the learned information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Requirements Analysis Indicated&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may be the extent to which design can proceed without more analysis of the requirements of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What must be defined first is how the training data will be labeled, if at all.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the goal to provide example data for supervised learning, where expected results are provided to the training process?&lt;/li&gt;&#xA;&lt;li&gt;Or is the goal to provide example data for unsupervised learning, where no expected results are given, and the training is to proceed toward some defined goal that is universal across the examples used in training?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Another consideration is to decide whether intelligent defaulting will be employed.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Should the system guess what the next answer is going to be and intelligently select it, allowing the user to change the selection if the guess was wrong?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-11T23:17:30.487" LastActivityDate="2018-10-11T23:17:30.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8364" PostTypeId="1" AcceptedAnswerId="8365" CreationDate="2018-10-11T23:09:12.837" Score="3" ViewCount="66" Body="&lt;p&gt;I am a newbie in deep learning and am looking for advice on predicting traffic congestion events. I have a table for vehicle travel times data, another table with the road length segmented based on stop locations. I am thinking to derive the time-wise route-specific speed details based on stop locations. After initial data cleansing and messaging, my input parameters are the time and stop location with actual speed details. I train my model with the training dataset and validate it as per the deep learning recommended approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this approach correct or how can I improve it? I am not sure if&#xA;the number of inputs can be increased for better results.   &lt;/li&gt;&#xA;&lt;li&gt;Which activation method will be best to utilize to get a range of conditions/event types rather than binary 1 or 0?     &lt;/li&gt;&#xA;&lt;li&gt;This will require dealing with a bigger dataset of at least over a few GBs. This will evolve into around 200GBs in the final product. Can I use my professional grad laptop to process this data or if I should consider going to Big Data Processing power?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Please advise. Thanks in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="18959" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:33.367" LastActivityDate="2020-05-29T13:48:33.367" Title="Deep learning model training and processing requirement for Traffic data" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8365" PostTypeId="2" ParentId="8364" CreationDate="2018-10-12T00:04:35.640" Score="0" Body="&lt;p&gt;&lt;strong&gt;Adjustments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few considerations that may help with problem analysis:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Traffic data alone will produce limited predictive results.  For instance, the onset of precipitation is critical input for any production-ready predictive model.&lt;/li&gt;&#xA;&lt;li&gt;Traffic congestion is highly chaotic in that events cascade, thus what is colloquially called the butterfly effect, is a dominant challenge.  What you can realistically achieve are what chaoticians call attractors.  They are morphological features in phase spaces and auto-correlations that are predictable under specific conditions.&lt;/li&gt;&#xA;&lt;li&gt;Modelling traffic requires further segmentation beyond stop locations to include turn options and merge points, both of which are critical factors in free flow and congestion.&lt;/li&gt;&#xA;&lt;li&gt;Feedback from the field will DRASTICALLY improve prediction.  Expecting the system to predict start to finish scenarios longer than a few minutes of potential congestion dynamics is unrealistic.  Inputs will need to be continuous and the closer to real time the better.&lt;/li&gt;&#xA;&lt;li&gt;The physical relationships between position, velocity, and acceleration must be built into the parameterized model in which parameters are continuously updated.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We can assume that the system design predicts congestion for some purpose, which was not stated in the question.  It is reasonable to assume, because of the economics of transportation, that the objective is to reduce the area under the probability distribution curve for undesirable events resulting from congestion.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lost time&lt;/li&gt;&#xA;&lt;li&gt;Accident occurrence&lt;/li&gt;&#xA;&lt;li&gt;Fatalities&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The mechanism of control through which such objectives could be achieved can be any of these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Notifying drivers of traffic conditions via cell towers or WiFi&lt;/li&gt;&#xA;&lt;li&gt;Notifying drivers via dynamic sign display&lt;/li&gt;&#xA;&lt;li&gt;Adjusting remotely configurable traffic signaling devices&lt;/li&gt;&#xA;&lt;li&gt;Controlling gates&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These controls can be to human drivers or pilots or to automated driving or piloting systems.  The context of the system is key to designing it properly, since the use cases are entirely driven by this context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this approach correct, [and] how can I improve it? &amp;mdash; suggestions above&lt;/li&gt;&#xA;&lt;li&gt;I am not sure if the number of inputs can be increased for better results. &amp;mdash; absolutely yes&lt;/li&gt;&#xA;&lt;li&gt;Which activation method will be best to utilize to get a range of conditions/event types rather than binary 1 or 0? &amp;mdash; A ReLU or one of its derivatives may produce the best results for off line training, however it may be advisable to look into real time approaches such as Q-learning.  In either case, it is advisable to finish requirements definition, decide upon approach, and produce a general architectural diagram of the system before attending to such minutia as activation functions.&lt;/li&gt;&#xA;&lt;li&gt;[Given the likely data set size of 200GB] can I use my professional grad laptop to process this data or if I should consider going to Big Data Processing power? &amp;mdash; Using VLSI hardware acceleration from vendors such as Intel or NVidia is advisable.  Over the course of time, the initial expense and learning curve may be absorbed by not having to pay for additional bandwidth for PasS or SaaS services.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding computing resources, it is inadvisable to begin with some of the most congested places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cities Definitely Too Congested&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is likely that these cities would require a super-computing platform of significant size and a multinational corporation sized budget.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Beijing&lt;/li&gt;&#xA;&lt;li&gt;Dubai&lt;/li&gt;&#xA;&lt;li&gt;Tokyo&lt;/li&gt;&#xA;&lt;li&gt;Los Angeles&lt;/li&gt;&#xA;&lt;li&gt;Chicago&lt;/li&gt;&#xA;&lt;li&gt;London&lt;/li&gt;&#xA;&lt;li&gt;Hong Kong&lt;/li&gt;&#xA;&lt;li&gt;Shanghai&lt;/li&gt;&#xA;&lt;li&gt;Paris&lt;/li&gt;&#xA;&lt;li&gt;Amsterdam&lt;/li&gt;&#xA;&lt;li&gt;Dallas&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Commuter locations such as Melbourne, Australia; Palm Beach, Florida; or New Haven, Connecticut are examples of good choices to start.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-12T00:16:21.363" LastActivityDate="2018-10-12T00:16:21.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8367" PostTypeId="1" CreationDate="2018-10-12T06:04:40.683" Score="3" ViewCount="144" Body="&lt;p&gt;There are some predefined categories( Overview, Data Architecture, Technical Details, Applications, etc). The requirement is to classify the input text of paragraphs into their resp. category. I can't use any pre-trained word embeddings (Word2Vec, Glove) because the data entered is not in general English ( talking about dogs, environment, etc) but pure technical (How does a particular program orks, steps to download anaconda, etc). Don't have any data available on the internet to train as well. Anything that understands semantic-surface-level of a sentence will work&lt;/p&gt;&#xA;" OwnerUserId="18963" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:40.550" LastActivityDate="2020-05-29T13:47:40.550" Title="How to find the category of a technical text on a surface-semantic-level" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;semantics&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8368" PostTypeId="1" CreationDate="2018-10-12T06:43:28.877" Score="2" ViewCount="118" Body="&lt;p&gt;I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays.  The objective is to experiment with different NN designs to train them to transform single channel (mono) audio into a two channel (stereo) audio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do I need a stateful network type, like LSTM? (I think yes.)&lt;/li&gt;&#xA;&lt;li&gt;How should I organize the data, considering that there are millions of samples and I can't load into memory a matrix of each window of data in a reasonable time-span?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For example if I have an array with: [0, 0.5, 0.75, 1, -0.5, 0.22, -0.30 ...] and I want to take a window of 3 samples, for example.  I guess I need to create a matrix with every sample shift like this, right?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[0.00, 0.50, 0.75]&#xA; [0.50, 0.75, 1.00]&#xA; [0.75, 1.00,-0.50]&#xA; [1.00,-0.50, 0.22]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where is my batch_size? Should I make the matrix like this per each sample shift? Per each window? This may be very memory consuming if I intend to load a 4 min song. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this example matrix a single batch? A single sample?&lt;/p&gt;&#xA;" OwnerUserId="18964" LastEditorUserId="4302" LastEditDate="2018-10-14T01:15:08.080" LastActivityDate="2019-12-16T05:02:20.220" Title="Difficulty understanding Keras LSTM fitting data" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8369" PostTypeId="2" ParentId="8367" CreationDate="2018-10-12T12:46:26.610" Score="1" Body="&lt;p&gt;&lt;strong&gt;Problem Statement&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Find the category of technical text on a surface-semantic-level.&#xA;The requirement is to classify the input text of paragraphs into their respective categories.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The categories given as predefined are as follows.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Overview&lt;/li&gt;&#xA;&lt;li&gt;Data Architecture&lt;/li&gt;&#xA;&lt;li&gt;Technical Details&lt;/li&gt;&#xA;&lt;li&gt;Applications&lt;/li&gt;&#xA;&lt;li&gt;etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Some document types that would be added in place of 'etc' might include these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Requirements&lt;/li&gt;&#xA;&lt;li&gt;High Level Design&lt;/li&gt;&#xA;&lt;li&gt;System Architecture&lt;/li&gt;&#xA;&lt;li&gt;Testing Plan&lt;/li&gt;&#xA;&lt;li&gt;Deployment Plan&lt;/li&gt;&#xA;&lt;li&gt;Disaster Recovery Plan&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Semantic Structure of Technical Documents&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is correct that the terminology between a requirements document and data architecture will likely have much in common.  More precisely, the distributions of linguistic elements of all documents common to a given system or project are likely to contain domain commonalities.  A system that pilots a drone will have the linguistic elements &amp;quot;drone(s)&amp;quot;, &amp;quot;hover(ing|s)&amp;quot;, &amp;quot;target(ing|s)&amp;quot;, &amp;quot;flight plan(s)&amp;quot; in similar distributions throughout its documentations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Indicators&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is likely that these five distinguishing characteristics can be exploited in categorization.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sentence semantics&lt;/li&gt;&#xA;&lt;li&gt;Inserted diagrammatic and pictorial conventions&lt;/li&gt;&#xA;&lt;li&gt;Header conventions&lt;/li&gt;&#xA;&lt;li&gt;Linguistic elements that appear commonly in only one type of technical document&lt;/li&gt;&#xA;&lt;li&gt;Elements and structure from copying previous documents and modifying them or the use of boiler plates and templates in specific departments&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Rather than focus entirely on text recognition and abandoning the other items in the list above would be unwise.  Diagrams, such as network diagrams and UML diagrams, may be quite easy to discern using deep convolutional approaches and would clearly identify the category to which the hosting document belongs.  That is also true of test case tables.&lt;/p&gt;&#xA;&lt;p&gt;Recognizing that the proportions between the five indicators above and sections of indicators are variables in the model upon which training is applied will produce the best results.  For instance, the final paragraph may be more telling than the rest of the body.&lt;/p&gt;&#xA;&lt;p&gt;Also, be aware that one can couple the proportional appearance of language elements within all text documents on record with the language elements identified in the example and then in later use of the trained model.  The training is likely to progress faster and produce more accurate and reliable results if features include an indication that the linguistic element &amp;quot;test case(s)&amp;quot; appearing in the input text is significantly less prevalent in the domain of technical documentation than the linguistic element &amp;quot;-ing&amp;quot; to indicate continuous tenses of verbs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Avoid Static Grammars&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Language parsers based on fixed language rules (grammars) have not had much success compared to association based semantic mapping, and linguistics has moved away from those static models for similar reasons.  Avoid grammar based parsing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Existing Work&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the textual categorization, the below academic publications are some of the recent work that have already gained some notoriety.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aclweb.org/anthology/P15-2058&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Semantic clustering and convolutional neural network for short text categorization&lt;/em&gt;, P Wang, J Xu, B Xu, C Liu, H Zhang, F Wang… - Proceedings of the 53rd, 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aclweb.org/anthology/D15-1167&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Document Modeling with Gated Recurrent Neural Network for Sentiment Classification&lt;/em&gt;, D Tang, B Qin, T Liu - Proceedings of the 2015 conference on empirical, 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aclweb.org/anthology/P15-1098&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Semantic Representations of Users and Products for Document Level Sentiment Classification*, D Tang, B Qin, T Liu - Proceedings of the 53rd Annual Meeting of the, 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1412.1058.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Effective use of word order for text categorization with convolutional neural networks&lt;/em&gt;, R Johnson, T Zhang - arXiv preprint arXiv:1412.1058, 2014 - arxiv.org&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.krchowdhary.com/ai/ai14/lects/nlp-research-com-intlg-ieee.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Jumping NLP curves: A review of natural language processing research&lt;/em&gt;, E Cambria, B White - IEEE Computational intelligence, 2014&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/download/9745/9552&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Recurrent Convolutional Neural Networks for Text Classification&lt;/em&gt;, S Lai, L Xu, K Liu, J Zhao - AAAI, 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Faulty Approaches&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The approach in the comment of comparing the semantics of sentences will only determine one aspect of information redundancy between two sentences.&lt;/p&gt;&#xA;&lt;p&gt;The number of comparisons are also a consideration.  For &lt;span class=&quot;math-container&quot;&gt;$\chi$&lt;/span&gt; documents each containing &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; sentences would require &lt;span class=&quot;math-container&quot;&gt;$\chi \, (\chi - 1) \, \sigma \, (\sigma - 1)$&lt;/span&gt; comparisons.  For &lt;span class=&quot;math-container&quot;&gt;$\chi = 10,000$&lt;/span&gt; documents containing &lt;span class=&quot;math-container&quot;&gt;$\sigma = 1,000$&lt;/span&gt; sentences, we have 99,890,010,000,000 sentence comparisons, the totality of which provides no particularly useful information about the category of any of the documents.&lt;/p&gt;&#xA;&lt;p&gt;The documents must be related to a concept class, not each other.&lt;/p&gt;&#xA;&lt;p&gt;Visualizations of the semantics aren't particularly useful unless you are looking for something and the visualization is designed to present that which is sought.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A Better Plan&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Determine the number of example training documents, &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, needed to be categorized by experts to create a sufficiently large training data set. (Consider using the PAC learning framework designed for this purpose.)&lt;/li&gt;&#xA;&lt;li&gt;Draw that number from the full set of documents, using an appropriate random or highly pseudo-random method, herein referred to as method &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt;. That is the training example set.&lt;/li&gt;&#xA;&lt;li&gt;Draw that number from the full set of documents again, using method &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt;. That is the test example set.&lt;/li&gt;&#xA;&lt;li&gt;Have experts label (categorize) both example sets according to technical document type.  Use the same experts for both training and testing, and have them alternate periodically between the two sets so that their learning, fatigue, or boredom curves have a minimal effect on their categorizing of the two sets.&lt;/li&gt;&#xA;&lt;li&gt;Profile each of the category-labeled documents in terms of the above five indicators.  Note that to use the images embedded in the document as part of the profiling, which could drastically improve system reliability, the images must be run through a separate diagram categorizing network trained separately on images to find diagrams characteristic of one of the technical document types.  That may sound like much work, but consider that categorizing drawing types is a well developed science and the labeling is only of the diagram types that are representative of particular document types.&lt;/li&gt;&#xA;&lt;li&gt;Train an appropriately designed artificial network to categorize the documents, using the profiling results as example inputs and the labels from the categorizing from the experts.&lt;/li&gt;&#xA;&lt;li&gt;Test using the test set&lt;/li&gt;&#xA;&lt;li&gt;Based on the results of the test, decide whether to use the current training or re-execute a previous step using information gained from the first training and train again&lt;/li&gt;&#xA;&lt;li&gt;Run the trained network on the full document set&lt;/li&gt;&#xA;&lt;li&gt;Pull a sample from the result using method &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt; to validate effective completion of the run&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-16T12:24:44.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8370" PostTypeId="1" CreationDate="2018-10-12T18:14:51.050" Score="2" ViewCount="232" Body="&lt;p&gt;Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the original problem, can drastically reduce the computational effort to solve for x. However, preconditioning is normally problem-specific and there is not ONE preconditioner that works well for every problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought this would be an interesting problem to apply RL since there are certain norms (e.g. condition number of matrix PA) to measure if P is a good preconditioner, but I could not find any research in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a specific problem why RL could not be applied?&lt;/p&gt;&#xA;" OwnerUserId="18975" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:10.130" LastActivityDate="2020-05-29T13:47:10.130" Title="Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b" Tags="&lt;reinforcement-learning&gt;&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8371" PostTypeId="1" AcceptedAnswerId="8376" CreationDate="2018-10-12T19:45:40.173" Score="2" ViewCount="46" Body="&lt;p&gt;I have created a classifier for some simple gestures using an input layer, a hidden layer with tanh activation and an output softmax layer. I'm also using the Adam optimiser. The network classifies perfectly with validation data. However, I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One gesture input looks like this and is correctly classified as gesture 'A':&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JlvjP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JlvjP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I pass this 'noise', which is clearly differentiable to the human eye, as input it still classifies it with 100% confidence that it is the same gesture 'A'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3DBKX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3DBKX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume it's because the inputs are still very close to 0? My instinct is to scale up the inputs perhaps to increase the differentiation between the noise and the input. However, in real operation the noise will all be on a similar scale to the inputs and I won't know what is noise and what isn't so I will still have to apply the same scaling to that noise. Will I run into the same problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a more general note is there a teaching approach to prevent misclassifications, particularly if we know what they might look like? For example, in this case I thought I could perhaps generate some noise and use it at training time to create an extra noise class, or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold? For example, if the network only produces 50% classifcation confidence for an input then I can discard it as noise. Any suggestions much appreciated!  &lt;/p&gt;&#xA;" OwnerUserId="18577" LastEditorUserId="18577" LastEditDate="2020-05-28T22:18:45.817" LastActivityDate="2020-05-28T22:18:45.817" Title="Recognising Noise in Simple Classification" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8376" PostTypeId="2" ParentId="8371" CreationDate="2018-10-13T07:52:31.447" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The network classifies perfectly with validation data, however I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You need to train the network on noise well if you want it to be able to recognize it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can use spot to get a confidence score for clarification. But training the network on a noise class will work way better&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gal: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-10-13T07:52:31.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8378" PostTypeId="2" ParentId="3321" CreationDate="2018-10-13T09:53:58.070" Score="0" Body="&lt;p&gt;One key to the answer is in the question, &quot;Even for one specific conv layer.&quot;  It is not a good idea to build deep convolution networks on the assumption that a single kernel size most aptly applies to all layers.  When perusing the configurations that proved successful in publications, it becomes apparent that configurations that varying through their layers are more commonly found to be optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other key is to understand that two layers of 11x11 kernels have a 21x21 reach, and ten layers of 5x5 kernels have a 41x41 reach.  A mapping from one level of abstraction to the next need not be completed in one layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generalities regarding kernel sizes exist, but they are functions of the typical input characteristics, the desired output of the network, the computing resources available, resolution, size of the data set, and whether they are still images or movies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding input characteristics, consider this case: The images are shot with a large depth of field under poor lighting conditions, such as in security scenarios, so the aperture of the lens is wide open, causing objects at some ranges of distance to be out of focus, or there can be motion blur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under such conditions a single 3x3 kernel will not detect many edges.  If the edge may span five pixels, the choice exists as to how many layers are dedicated to its detection.  What factors affect that choice is based on what other characteristics exist in the input data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expect that as acceleration hardware develops (in VLSI chips dedicated to this purpose) that the computing resource constraints will decrease in priority as a factor in kernel size selection.  Currently, the computation time is significant and forces the decision about how to balance layer count and layer size to be mostly a matter of cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question begs another question.  Can an oversight machine learner learn how to automatically balance the configuration of deep convolution networks?  It could then be re-executed whenever additional computing resources are provisioned.  It would be surprising if there weren't at least a dozen labs working on exactly this capability.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-13T10:02:14.467" LastActivityDate="2018-10-13T10:02:14.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8379" PostTypeId="2" ParentId="6488" CreationDate="2018-10-13T10:41:58.233" Score="0" Body="&lt;p&gt;&lt;strong&gt;Problem Statement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the features of the runs.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;CNN class prediction using two 2D convolutions with associating max pooling&lt;/li&gt;&#xA;&lt;li&gt;mini-batch execution approach&lt;/li&gt;&#xA;&lt;li&gt;fixed shuffling process used&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are the results.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;shuffling obtained 80%/70% train/test accuracy&lt;/li&gt;&#xA;&lt;li&gt;shuffling for full set obtained 77% accuracy&lt;/li&gt;&#xA;&lt;li&gt;no shuffling for full set obtained 45% accuracy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Listed Causes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the potential Causes for the apparent anomaly that were listed in the question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;model is learning incorrectly&lt;/li&gt;&#xA;&lt;li&gt;learns that the order of the data points plays a role in their prediction&lt;/li&gt;&#xA;&lt;li&gt;data point not predicted separately because of mini-batching&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other Causal Possibilities not Listed&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that both of these additional possibilities are related to an insufficiency in the simulation of randomness, just as can be the case in cryptographic protocols.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The CNN learns the shuffling system or some aspect of it so that when the shuffling is removed, the training no longer applies to the input patterns&lt;/li&gt;&#xA;&lt;li&gt;How the training and testing samples are drawn is not sufficiently random&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the model learn from the average of all the data points in the mini-batch? &amp;mdash; Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it think one mini-batch is one data point? &amp;mdash; No. It doesn't think, and the loop does not average the data points before the propagation.  Mini-batch simply aggregates the results before back-propagating the correction signal to the parameter tensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does order matter? &amp;mdash; Order cannot matter in a stateless system, but often does if there is state remembered between discrete events.  Mini-batch requires averaging, which requires statefulness to accumulate the addends.  But that is not the likely cause.  How the batches are selected from the sample is a more likely factor affecting accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Principles to Comprehend&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convergence of artificial networks in general is based on the statistical characteristics of the training scenario matching the statistical characteristics of the usage scenario.  In other words, to use PAC (probably approximately correct) framework terminology, how the training sample is drawn from the total population must be identical to how the validation sample is drawn from the total population.  Therefore, if the training sample is not drawn with sufficient randomness from the total population convergence cannot not guaranteed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions to Consider&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How am I deciding the individual operations within the shuffling?&lt;/li&gt;&#xA;&lt;li&gt;How am I drawing the train and test samples?&lt;/li&gt;&#xA;&lt;li&gt;How am I deciding what samples go in what batch?&lt;/li&gt;&#xA;&lt;li&gt;What natural order is in the data examples, and is it really a sequence rather than a set?&lt;/li&gt;&#xA;&lt;li&gt;If a sequence, then is a classic CNN, not designed out of the box to handle temporal sequences, the correct network design to apply?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Answering these questions and gaining a full conceptual understanding of the probability and statistics aspects of the approach should occur prior to thinking about normalization, which could fix your problem accidentally, but cannot be the root cause of the anomaly.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-13T10:41:58.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8380" PostTypeId="1" AcceptedAnswerId="8381" CreationDate="2018-10-13T11:47:35.950" Score="2" ViewCount="606" Body="&lt;p&gt;This survey of &lt;a href=&quot;https://www.metalevel.at/prolog/ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence with Prolog&lt;/a&gt; provides a short perspective on the Prolog programming language and its position in the history of AI.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It proposes that a return to Prolog Research and Development as begun in the 1980s when Japan began work on a Fifth Generation Computer System.  Has the reduction in cost and the increase in speed of computing platforms re-opened a door that was closed primarily because of technology immaturity?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="1581" LastEditDate="2018-10-16T18:53:54.493" LastActivityDate="2018-10-16T20:43:57.280" Title="Can strong AI now be achieved with Prolog?" Tags="&lt;agi&gt;&lt;prolog&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8381" PostTypeId="2" ParentId="8380" CreationDate="2018-10-13T11:48:08.117" Score="1" Body="&lt;p&gt;The application of Prolog, the declarative programming language may or may not be closely related to AI strength.  Prolog is a language that was designed to produce a Turing Complete backward chaining general programming language.  The strength or weakness of Prolog programs, just as for programs in Scheme, Common LISP, Java or Scala with DRools rules engine, or neuromorphic libraries in Python or C++ depends on more than the language and libraries chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strength of an artificially intelligent system, from a mathematical perspective, is as ambiguous in meaning as IQ test results.  Did human kind within one decade cross some critical level where they were once stupid and became intelligent or their weak intelligence became strong?There is no evidence for such discrete transition.  Evidence indicates that DNA advancements, many of them, and cultural changes to increase intelligence capacities occurred over thousands or tens of thousands of years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same is true of general and narrow AI.  All intelligent systems have strengths, weakness, generalizations, and narrows. This is true whether the intelligence arises out of billions of organic neurons and the brain's complex systems of chemistry, organelle structures, and plasticity capabilities or whether the intelligence arises from designs involving ones and zeros on a collection of interconnected silicon wafers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classifying systems into a four quadrant system is counter productive.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Weak narrow&lt;/li&gt;&#xA;&lt;li&gt;Strong narrow &amp;mdash; Extremely effective systems applicable to a narrow set of usage contexts&lt;/li&gt;&#xA;&lt;li&gt;Weak general &amp;mdash; Systems that show adaptive and proactive capabilities over a wide range of usage contexts but demonstrating only limited effectiveness in any given context compared to a typical human&lt;/li&gt;&#xA;&lt;li&gt;Strong general&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The binary system of weak = narrow and strong = general is even more limiting because of the existence of the middle two bullet points above in human populations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judgments about intelligence are ALWAYS&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Relative to some standard,&lt;/li&gt;&#xA;&lt;li&gt;Time dependent, and&lt;/li&gt;&#xA;&lt;li&gt;Context dependent.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More importantly, there is more than one or two dimensions to intelligence &amp;mdash; possibly hundreds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ken Thompson and Linux Torvalds, clearly the most successful operating system writers in terms of influence on currently deployed von Neuman computers may have great difficulty with writing a song that sustains sales over decades.  Bono and Paul McCartney may have great difficulty with writing an operating system that is downloaded every minute for decades.  Both features and breadths of intelligence have value in current culture and economics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The appreciation of the Prolog programming language by those like the author of the page referenced in the question is not without reason.  The design of the language provides an advantage for R&amp;amp;D over other languages in rules based contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be value in reconsidering the value of the language in light of the significant improvements in computing platforms since earlier efforts were made to use inference to produce learning systems.  The rule type used to learn other rules were called meta-rules, and meta-rules have produced some valuable behaviors in adaptive systems based on Java components such as DRools.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-13T11:48:08.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8382" PostTypeId="1" CreationDate="2018-10-13T12:17:13.547" Score="2" ViewCount="1884" Body="&lt;p&gt;&lt;a href=&quot;https://quickdraw.withgoogle.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quick draw&lt;/a&gt; is a Google experiment using user generated online doodles and machine learning to play a game of &quot;Guess what I'm drawing&quot; similar to the board game Pictionary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested if anyone happens to know what algorithms are in play in this example? &lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2018-10-13T23:18:28.857" Title="What models is Google's quick draw using?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;google&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8383" PostTypeId="2" ParentId="6488" CreationDate="2018-10-13T12:36:19.760" Score="1" Body="&lt;p&gt;The example followed in the question uses a relatively straightforward Convolutional Neural Network. These are not stateful, so the order in which predictions for instances from a test set are queried should have no influence on those predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a comment written by the author of the question on their own question, it was mentioned that the use of Batch Normalization appears to have been confirmed to be the cause of the issue. Given this info, one possible cause of the issue described in the question is incorrect usage of the &lt;code&gt;training&lt;/code&gt; flag of TensorFlow's Batch Normalization implementation. The &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;official documentation&lt;/a&gt; contains the following info on this flag:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;code&gt;training&lt;/code&gt;&lt;/strong&gt;: Either a Python boolean, or a TensorFlow boolean scalar tensor (e.g. a placeholder). Whether to return the output in training mode (normalized with statistics of the current batch) or in inference mode (normalized with moving statistics). &lt;strong&gt;NOTE&lt;/strong&gt;: make sure to set this parameter correctly, or else your training/inference will not work properly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If this is incorrectly set to &lt;code&gt;True&lt;/code&gt; rather than &lt;code&gt;False&lt;/code&gt; outside of the training phase (i.e. when evaluating performance), predictions can be expected to be poor. This alone doesn't explain why specifically the order of test data would matter though, if this alone were the issue then we'd expect test performance to be poor regardless of order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different possible explanation can be that there is a mistake in the code that still causes &lt;code&gt;moving_mean&lt;/code&gt; and &lt;code&gt;moving_variance&lt;/code&gt; ops of the Batch Normalization to be updated during the testing/evaluation. These should only be updated during the training phase, as explained in the documentation linked to above. If they are still getting updated during the test phase, and &lt;strong&gt;if there is a meaningful structure in the unshuffled ordering of the test set&lt;/strong&gt; (i.e. unshuffled test set ordered by class, or ordered according to certain features, etc.), &lt;strong&gt;then we would expect precisely the issue described in the question to occur&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-10-13T12:36:19.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8384" PostTypeId="2" ParentId="6765" CreationDate="2018-10-13T13:08:47.460" Score="3" Body="&lt;p&gt;Judea Pearl's 2018 comment on ACM.org, in his &lt;a href=&quot;https://cacm.acm.org/news/227877-to-build-truly-intelligent-machines-teach-them-cause-and-effect/fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;To, Build Truly Intelligent Machines, Teach Them Cause and Effect&lt;/em&gt;&lt;/a&gt; is piercing truth.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;All the impressive achievements of deep learning amount to just curve fitting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It may be less sensational and more technically correct to state that it is not, &amp;quot;Just curve fitting,&amp;quot; but rather, &amp;quot;sophisticated surface fitting.&amp;quot; Nonetheless, his general assessment indicates the need to look beyond tuning nonlinear functions to fit a surface in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; and consider whether cognition is achievable with a deep network. The split in answers to this question is odd. We have two conflicting assertions, often strongly stated.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Artificial networks cannot perform logic.&lt;/li&gt;&#xA;&lt;li&gt;Artificial networks are the best approach to AI.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How can rationality be excluded from the list of important human features of intelligence, which is what these two assertions are taken together would mean?&lt;/p&gt;&#xA;&lt;p&gt;Is the human brain a network of sophisticated curve fitters?  Marvin Minsky's famous quote, &amp;quot;The brain happens to be a meat machine,&amp;quot; was offered without proof, and neither a proof of his trivialization of the human brain nor a proof that the brain is beyond the reach of Turing computability has been offered since.&lt;/p&gt;&#xA;&lt;p&gt;When you read these words, are your neural networks doing the following sequence of curve fits?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Edges from retinal rods and cones&lt;/li&gt;&#xA;&lt;li&gt;Lines from edges&lt;/li&gt;&#xA;&lt;li&gt;Shapes from lines&lt;/li&gt;&#xA;&lt;li&gt;Letters from shapes&lt;/li&gt;&#xA;&lt;li&gt;Linguistic elements from groups of letters&lt;/li&gt;&#xA;&lt;li&gt;Linguistic structures from elements&lt;/li&gt;&#xA;&lt;li&gt;Understanding from linguistic structures&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The case is strong for the affirmation that the first five is a convergence mechanism on a model, and all the machine learning structure is just a method to fit the data to the model.&lt;/p&gt;&#xA;&lt;p&gt;Those last two bullet items are where the paradigm breaks down and where many AI researchers and authors have correctly stated that machine learning has significant limitations when based solely on layers of multi-layer perceptrons and convolution kernels.  Furthermore, the last bullet item is grossly oversimplified in its current state, probably by orders of magnitude.  Even if Minsky is correct that a computer can perform what the brain does, the process of reading and understanding this paragraph could easily have a thousand different kinds of unique process components in patterns of internal workflow with massive parallelism.  Imaging technology indicates this probability.  We have computers modelling only the simplest peripheral layers.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI? — No. But there is no such argument that guarantees it either.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Other questions here investigate whether these sophisticated curve fitters can perform elements of cognition or reasoning.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/3389/could-a-neural-network-detect-primes&quot;&gt;Could a neural network detect primes?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7416/can-deep-networks-be-trained-to-prove-theorems&quot;&gt;Can neural networks be used to prove conjectures?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The totem of three in the question's image, seeing, doing, and imagining, is not particularly complete, accurate, or insightful.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are at least five sensory paradigms in humans, not one&lt;/li&gt;&#xA;&lt;li&gt;Doing preceded human senses by billions of years — bacteria do&lt;/li&gt;&#xA;&lt;li&gt;Imagining is not a significantly higher process than scenario replay from models of past experience with some method to apply set functions to combine them and inject random mutations&lt;/li&gt;&#xA;&lt;li&gt;Creativity may just be imagining in the previous bullet item followed by weeding out useless imagination results with some market-oriented quality criteria, leaving the impressive creative products that sell&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The higher forms are appreciation, a sense of realities beyond the scope of scientific measurement, legitimate doubt, love, sacrifice for the good of others or humanity.&lt;/p&gt;&#xA;&lt;p&gt;Many recognize that the current state of AI technology is nowhere near the procurement of a system that can reliably answer, &amp;quot;How can I make Y happen?&amp;quot; or &amp;quot;If I have acted differently, will X still occur?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;There is no mathematical proof that some combination of small curve fitting elements can or cannot achieve the ability to answer those questions as well a typical human being can, mostly because there is insufficient understanding of what intelligence is or how to define it in mathematical terms.&lt;/p&gt;&#xA;&lt;p&gt;It is also possible that human intelligence doesn't exist at all, that references to it are based on a religious belief that we are higher as a species than other species.  That we can populate, consume, and exterminate is not actually a very intelligent conception of intelligence.&lt;/p&gt;&#xA;&lt;p&gt;The claim that human intelligence is an adaptation that differentiates us from other mammals conflicts with whether we adapt well.  We have not been tested.  Come the next meteoric global killer with a shock wave of the magnitude of that of the Chicxulub crater's meteor, followed by a few and a thousand years of solar winter and we'll see whether it is our 160,000-year existence or bacteria's 4,000,000,000 year existence that proves more sustainable.  In the timeline of life, human intelligence has yet to prove itself significant as an adaptive trait.&lt;/p&gt;&#xA;&lt;p&gt;What is clear about AI development is that other kinds of systems are playing a role along with deep learners based on the multi-layer perceptron concept and convolution kernels which are strictly surface fitters.&lt;/p&gt;&#xA;&lt;p&gt;Q-learning components, attention-based components, and long-short term memory components are all strictly a surface fitter too, but only by stretching the definition of surface fitting considerably.  They have real-time adaptive properties and state, so they can be Turing complete.&lt;/p&gt;&#xA;&lt;p&gt;Fuzzy logic containers, rules-based systems, algorithms with Markovian properties, and many other component types also play their role and are not surface fitters at all.&lt;/p&gt;&#xA;&lt;p&gt;In summary, there are points made that have a basis in more than plausibility or a pleasing intuitive quality, however, many of these authors do not provide a mathematical framework with definitions, applications, lemmas, theorems, proofs, or even thought experiments that can be scrutinized in a formal way.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="36737" LastEditDate="2021-04-04T15:14:34.153" LastActivityDate="2021-04-04T15:14:34.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8389" PostTypeId="2" ParentId="7693" CreationDate="2018-10-13T15:22:44.957" Score="1" Body="&lt;p&gt;It is correct to say that a sigmoid activation function would only work well as a model if the desired output is close to the sigmoid function applied to the input.  This is a trivial fact that applies to a single layer perceptron.  This is true for for the single layer case for any activation function, also a trivial fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the layer number is between one and infinity (two or more), the theory bifurcates.  The identity function becomes a special case:  Any number of layers that conform to a first degree polynomial, &lt;span class=&quot;math-container&quot;&gt;$ax + b$&lt;/span&gt;, can be replaced with a single identity layers.  The alternative case, where there are multiple layers that functionally do not conform to a first degree polynomial, &lt;span class=&quot;math-container&quot;&gt;$ax + b$&lt;/span&gt;, cannot be replaced by a single layer of some equally simple function.  The complexity increases geometrically, which is the entire point of multilayer perceptrons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under particular constraints, the multilayer perceptron can produce a wide variety of functional behaviors that do not resemble the activation functions of the layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, a properly trained network using sigmoid activation functions, with sufficient layer depth and sufficient massive allocation of computing resources, could theoretically approximate the topography of the Himalayas.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-07T20:14:20.573" LastActivityDate="2019-01-07T20:14:20.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8390" PostTypeId="2" ParentId="7693" CreationDate="2018-10-13T17:14:07.473" Score="-1" Body="&lt;p&gt;Because any number of linear layers can be represented by a single linear layer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A_2*(A_1*x +b_1)+B_2 = A_2*A_1*x+ A2*b_1+b_2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same is not true if you have non-linear functions.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-10-13T17:14:07.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8391" PostTypeId="1" CreationDate="2018-10-13T17:48:31.180" Score="2" ViewCount="245" Body="&lt;p&gt;Maxpooling is performed as one of the steps in inception which yields same output dimension as that of the input.&#xA;Can anyone explain how this max pooling is performed? &lt;/p&gt;&#xA;" OwnerUserId="17763" LastActivityDate="2018-10-14T04:44:03.950" Title="Maxpooling in inception?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8393" PostTypeId="2" ParentId="8370" CreationDate="2018-10-13T20:09:19.973" Score="2" Body="&lt;p&gt;Reinforcement Learning is a method for learning to perform beneficial actions in an environment. One way this is accomplished is by learning to predict useful actions as a function of the observed state of the environment. Another is by learning to predict the expected utility gain of doing an action in a particular observed state. Usually the fact that the agent executes a &lt;em&gt;sequence&lt;/em&gt; of actions is exploited to learn more rapidly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, the problem you describe is to solve a linear system of equations, which is to say, to learn some hidden values &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$Ax = b$&lt;/span&gt; for known &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. Gradient decent is a natural way to solve this problem because it is easy to calculate the gradient, and, since the matrices and vectors have the same dimensionality, it is reasonable to expect that the optimization surface is smooth with a single global minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While RL techniques &lt;em&gt;could&lt;/em&gt; be applied to solve this problem (I guess by predicting values of X in response to inputs b?), none of the usual features that privilege the RL approach over a standard supervised learning approach are present (there's no sequential relationship between datapoints, there's just a list of constraints essentially). &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-10-13T20:09:19.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8395" PostTypeId="2" ParentId="8382" CreationDate="2018-10-13T23:18:28.857" Score="1" Body="&lt;p&gt;Google's Quick Draw became five things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A pilot application&lt;/li&gt;&#xA;&lt;li&gt;A market test&lt;/li&gt;&#xA;&lt;li&gt;An experiment in massively data intensive reinforcement learning &lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;A data segue into handwriting recognition &lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;A data collection device for other branches of research (not necessarily an intentional purpose during conception) &lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Google's team on the project wished to delve into concept abstraction by tapping the minds of the authors of one billion drawings, attracting the public to the point of data acquisition using a contest between a massively scaled and parallelized game engine and the hundreds of millions of drawing authors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concept Model&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept model (distinct from the algorithm and architecture) is a time series of motor actions. Due to the event model in computer peripheral devices and the coupling with the event model of the Netscape DOM, AJAX RESTful transactions can be stored into a huge data set as pen-up, pen-down, and pen-move operations. This is very much like some of the first photo-plotter commands pioneered by Gerber Scientific Instruments in the 1980s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning Convergence Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept model is embodied within and trained using a generative recurrent network called sketch-rnn. &lt;sup&gt;3&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://ai.googleblog.com/2018/09/introducing-kaggle-quick-draw-doodle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Introducing the Kaggle “Quick, Draw!” Doodle Recognition Challenge&lt;/em&gt;, &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://www.fastcompany.com/90152774/the-dead-serious-strategy-behind-googles-silly-ai-experiments&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Dead-Serious Strategy Behind Google’s Silly AI Experiments&lt;/em&gt;, by Katharine Schwab, 12/1/2017, FastCompany.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://arxiv.org/pdf/1704.03477.pdf?utm_campaign=Revue%20newsletter&amp;amp;utm_medium=Newsletter&amp;amp;utm_source=revue&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Neural Representation of Sketch Drawings&lt;/em&gt;, David Ha, Douglas Eck, 2018&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-13T23:18:28.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8396" PostTypeId="2" ParentId="6435" CreationDate="2018-10-13T23:37:39.250" Score="0" Body="&lt;p&gt;Supervised learning is a statistical process that draws from a discrete set of examples, and therefore variance is to be expected.  (Recall that variance is normalized and standard-deviation is not.)  In the case of stochastic gradient descent, which exhibits improvements in training speed and reliability under many conditions, pseudo random factors are introduced deliberately, also affecting variance in accuracy measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the threshold of acceptable accuracy variance is not the primary metric of interest to system architects and corporate stakeholders.  It is the &lt;strong&gt;maximum&lt;/strong&gt; theoretical and empirical inaccuracy metrics that are of primary concern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the PAC (probably approximately correct) framework, the required accuracy threshold is defined by &lt;span class=&quot;math-container&quot;&gt;$1 - \epsilon$&lt;/span&gt;.  Data requirements to guarantee a minimum reliability, &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt; for particular types of problems can be determined for specific types of training objectives and for a given &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could theoretically extend the PAC framework to consider mean, variance, and dimensions of skew, but it would not be of much interest to architects and stakeholders who are accustomed to communicating assurances in the forms like, &quot;We know that with the data we have for training we can achieve 99.1% accuracy 89.3% of the time.&quot;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-13T23:37:39.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8397" PostTypeId="2" ParentId="3472" CreationDate="2018-10-14T00:35:54.287" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a closed research field.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Use of RNNs to detect spam grew out of the use of artificial networks to detect fraud in telecommunications and the financial industry as a result of the rise of attacks on long distance lines, ATMs, banks, and credit card systems in online and at data centers supporting physical points of sale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although basic RNN design has given way to the newer LSTM and GRU approaches and its variants and extensions, artificial networks are now one of the primary fraud detection technologies. The dominance of this fraud detection strategy extends to SPAM detection, with its close ties to fraudulence. The spammers present the appearance of a relationship with their recipients that does not exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The improvement on computing designs to recognize patterns in time series data and the application of those designs for fraud detection and countermeasures and the detection and routing or deletion of of unwanted incoming information will be a stable area of research and development for the foreseeable future.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Question 2. What is the oldest published paper in this field?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no oldest published paper. The first papers on RNN are given in this answer: &lt;a href=&quot;https://ai.stackexchange.com/questions/8190/where-can-i-find-the-original-paper-that-introduced-rnns/9287#9287&quot;&gt;Where can I find the original paper that introduced RNNs?&lt;/a&gt;, but the move from pattern based detection to artificial networks to stateful artificial networks was gradual. The earliest deployments of these networks in server side or client side solutions occurred before any papers were published on the specific topic of RNN use in spam detection.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Question 3. What are the pros and cons of using RNNs for email spam detection over other classification methods?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Spam also has a strong temporal element.  What one considers undesirable spam in one year may be considered mission critical email a few years later, and vice versa. The performance in this space includes speed, accuracy, and reliability of classification, but also adaptation to changing user classification needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is because of these four performance characteristics in tandem that stateful networks derived from RNNs are commonly used for spam detection. The need for gated learning and forgetting at the cell level to support the variable adaptivity makes the LSTM and GRU variants common choices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semantic document classification is riding on an emerging set of technologies, which are primarily artificial network designs that begin to broach the threshold of cognitive understanding of the text by storing linguistic structure in forms that allow analogy, comparison, and composition between them. Semantic algorithms that perform these operations on fuzzy associations in combination with recursive artificial networks may emerge as the dominant design as such designs are further developed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Detecting Spam Blogs: A Machine Learning Approach&lt;/em&gt;&lt;/a&gt;, Pranam Kolari, 2006&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1807.02892.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Automated labeling of bugs and tickets using attention-based mechanisms in recurrent neural networks&lt;/em&gt;&lt;/a&gt;, Volodymyr Lyubinets et. al., 2018&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://weichengzhang.co/src/paper/Capstone_final_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Spam Filter Through Deep Learning and Information Retrieval&lt;/em&gt;&lt;/a&gt;, Weicheng Zhang, 2018&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=569551&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An Unsupervised Neural Network Approach to Profiling the Behavior of Mobile Phone Users for Use in Fraud Detection&lt;/em&gt;, Peter Burge, John Shawe-Taylor, Journal of Parallel and Distributed Computing, Volume 61 Issue 7, July 2001, pp 915-925&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.logicnet.dk/reports/JunkDetection/JunkDetection.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Intelligent junk mail detection using neural networks&lt;/em&gt;, Michael Vinther, June 2002&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://222.252.30.203:8888/bitstream/123456789/12416/1/Intelligent%20systems%20and%20their%20applications.Vol.17.Iss.4.A.4.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Mining for fraud, Margaret Weatherford&lt;/em&gt;, IEEE Intelligent Systems, 2002&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://suraj.lums.edu.pk/~cs631s05/Papers/financial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Discovering golden nuggets: data mining in financial application&lt;/em&gt;, D Zhang, L Zhou, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol. 34, No. 4, November 2004&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1009.6119.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Comprehensive Survey of Data Mining-based Fraud Detection Research&lt;/em&gt;, C Phua, V Lee, K Smith, R Gayler - Arxiv preprint arXiv, 2007&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-01T21:23:10.047" LastActivityDate="2019-01-01T21:23:10.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8398" PostTypeId="1" CreationDate="2018-10-14T01:06:17.617" Score="2" ViewCount="41" Body="&lt;p&gt;A fixed video camera records people moving through its field of view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to &lt;strong&gt;detect&lt;/strong&gt; and track the head, in real-time as it moves through the video.  The norm is there are many heads, which often are sometimes partially obscured.  This &lt;a href=&quot;https://youtu.be/qDK6nLdm3sQ?t=45s&quot; rel=&quot;nofollow noreferrer&quot;&gt;example video boxes heads&lt;/a&gt; and provides a head count.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be many different models.  Examples include:   &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=PRbr9-_xLzs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adaboost-haar Head detection&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.zeolearn.com/magazine/instance-segmentation-using-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;MASK R-CNN&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=tBwUbQ0pyUw&quot; rel=&quot;nofollow noreferrer&quot;&gt;LBP Cascade&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given the context of the video, what is the thought process that you would use to choose a model?    &lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="10135" LastEditDate="2018-10-18T10:44:45.367" LastActivityDate="2019-04-18T21:05:15.850" Title="Choosing Instance Semantic Detection" Tags="&lt;neural-networks&gt;&lt;semantics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8399" PostTypeId="2" ParentId="8398" CreationDate="2018-10-14T02:42:04.117" Score="1" Body="&lt;p&gt;There may be a conceptual disconnect between the term Semantic Detection and the task of Head Tracking, since sequential recognition of an object in a set of visual samples representing continuous movement isn't technically a semantics problem.  Although a mapping strategy that works with semantic processing may somehow apply to, with appropriate modifications, the mapping of obstructions and tracked objects and their relationship in space-time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tracking a head is primarily a surface fitting convergence problem, where the convergence goal and the process that achieved it must be sustainable, especially if the system is required to take actions during the tracking operation.  Distinguishing heads from one another requires a high degree of reliability and accuracy in this convergent continuity.  That is the primary challenge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Drawing boxes and displaying counts are trivial operations once the head tracking works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some models listed in the question or improvements over them.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_workshops_2014/W17/papers/Bilaniuk_Fast_LBP_Face_2014_CVPR_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Fast LBP Face Detection on low-power SIMD architectures&lt;/em&gt;, Olexa Bilaniuk, Ehsan Fazl-Ersi, Robert Laganiere, Christina Xu, Daniel Laroche, Craig Moulder&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://hh.diva-portal.org/smash/record.jsf?pid=diva2%3A1094875&amp;amp;dswid=1797&quot; rel=&quot;nofollow noreferrer&quot;&gt;*A Real-Time AdaBoost Cascade Face Tracker Based on Likelihood Map and Optical Flow, Ranftl, Alonso-Fernandez, Karlsson, Bigun, IET Biometrics, 2017, Vol. 6, no 6, p. 468-477&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1049/iet-bmt.2016.0202&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Real-Time AdaBoost Cascade Face Tracker Based on Likelihood Map and Optical Flow, IET Biometrics, 6(6): 468-477&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://web.yonsei.ac.kr/hgjung/Ho%20Gi%20Jung%20Homepage/Publications/2009/TITS%28A%20New%20Approach%20to%20Urban%20Pedestrian%20Detection%20for%20Automatic%20Braking%29.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A New Approach to Urban Pedestrian Detection for Automatic Braking&lt;/em&gt;, Alberto Broggi et. al. IEEE Transactions on Intelligent Transportation Systems, Vol. 10, No. 4, 2009&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.06870.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Mask R-CNN&lt;/em&gt;, Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick, Facebook AI Research (FAIR)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One seasoned expert spoke to a particular detail in the final question.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The thought process experts use to choose a model has no equivalent point value in StackExchange reputation.  As experts retire, some may be willing to sell the professional heuristics they acquired over decades of research and development work if they haven't done so already to a book publisher.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, the above links are a good furtherance of the thought process begun in this question along a direction of continued effective thought.  The following steps provide a road map for how to develop approaches in general.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Read the academic materials until you understand.&lt;/li&gt;&#xA;&lt;li&gt;Determine your selection criteria.&lt;/li&gt;&#xA;&lt;li&gt;Test the options against that criteria.&lt;/li&gt;&#xA;&lt;li&gt;Pick the winner.&lt;/li&gt;&#xA;&lt;li&gt;If something blocks progress with that one, pick the runner up.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4709" LastEditDate="2019-04-18T21:05:15.850" LastActivityDate="2019-04-18T21:05:15.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8400" PostTypeId="2" ParentId="8391" CreationDate="2018-10-14T04:44:03.950" Score="2" Body="&lt;p&gt;My bad, didn't look into the block diagram first&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ah7Zu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ah7Zu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;maxpooling filter/kernel is &lt;code&gt;3x3&lt;/code&gt; = &lt;code&gt;fxf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formula for padding to get the same dimension as the input &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;p = (f - 1)/2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here &lt;code&gt;f=3&lt;/code&gt; ,so padding to be performed before maxpooling is &lt;code&gt;(3-1)/2 = 1&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="17763" LastActivityDate="2018-10-14T04:44:03.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8401" PostTypeId="2" ParentId="8284" CreationDate="2018-10-14T05:10:07.937" Score="1" Body="&lt;p&gt;For the above stated artificial network, these two training scenarios are similar.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Training to converge to the ideal output vector at the point after the last layer's activation functions are applied&lt;/li&gt;&#xA;&lt;li&gt;Training to converge to the vector formed by applying &lt;span class=&quot;math-container&quot;&gt;$tan$&lt;/span&gt; functions to each component of the ideal output vector, when convergence occurs at the point just after the vector-matrix multiplication with the last layer's parameters, prior to the last layer's &lt;span class=&quot;math-container&quot;&gt;$atan$&lt;/span&gt; activation functions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Distinctions between them include these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The applications of gradients and associated code must be adapted to the modification of the starting point of back propagation to before the final &lt;span class=&quot;math-container&quot;&gt;$atan$&lt;/span&gt; activation functions.&lt;/li&gt;&#xA;&lt;li&gt;The slope and curvature of the loss function will differ if the same loss function is used for the two scenarios, so the accuracy, speed, and reliability of convergence will also be different.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-14T05:10:07.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8403" PostTypeId="1" AcceptedAnswerId="8404" CreationDate="2018-10-14T09:05:08.993" Score="5" ViewCount="567" Body="&lt;p&gt;I'm making a Connect Four game using the typical minimax + alpha-beta pruning algorithms. I just implemented a Transposition Table, but my tests tell me the TT only helps 17% of the time. By this I mean that 17% of the positions my engine comes across in its calculations can be automatically given a value (due to the position being calculated previously via a different move order).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For most games, is this figure expected? To me it seems very low, and I was optimistically hoping for the TT to speed up my engine by around 50%. It should be noted though that on each turn in the game, I reset my TT (since the evaluation previously assigned to each position is inaccurate due to lower depth back then).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the effectiveness of TT's are largely dependent on the game they're being used for, but any ballparks of how much they speed up common games (chess, go, etc) would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT - After running some more tests and adjusting my code, I found that the TT sped up my engine to about 133% (so it took 75% as much time to calculate). This means those 17% nodes were probably fairly high up in the tree, since not having to calculate the evaluation of these 17% sped up things by 33%. This is definitely better, but my question still remains on whether this is roughly expected performance of a typical TT.&lt;/p&gt;&#xA;" OwnerUserId="16917" LastEditorUserId="16917" LastEditDate="2018-10-14T13:25:32.393" LastActivityDate="2018-10-14T13:29:50.357" Title="Transposition table is only used for roughly 17% of the nodes - is this expected?" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;gaming&gt;&lt;minimax&gt;&lt;efficiency&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8404" PostTypeId="2" ParentId="8403" CreationDate="2018-10-14T13:24:13.643" Score="4" Body="&lt;p&gt;I don't think that's necessarily a strange number. It's impossible for anyone to really tell you whether that 17% is &quot;correct&quot; or not without reproducing it, which would require much more info (basically would have to know every single tiny detail of your implementation to be able to reproduce).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some things to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;size of your transposition table / the number of bits you use for indexing into the TT&lt;/strong&gt;. If you have a relatively small TT, meaning you use relatively few bits for indexing, you'll have bigger probabilities of collisions. That means you will have to replace existing entries more often, which means they might no longer be in the table anymore by the time you encounter transpositions during the search.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Where in the search tree are the nodes located that are recognized as transpositions already in the table?&lt;/strong&gt; If you detect transpositions very high up in the search tree, you save a lot more search time than if you detect a transposition somewhere deep down in the search tree; once you detect a transposition that has already been searched sufficiently deep for the value stored in the table to be valid, you can cut off the complete subtree below that node from the search. This becomes more valuable as it happens closer to the root. So, just the number &quot;17% of nodes&quot; doesn't really tell us much.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Are you using iterative deepening?&lt;/strong&gt; Since you mentioned only minimax + alpha-beta pruning in the question, I suspect you're not using iterative deepening. TTs become significantly more valuable once you do use iterative deepening, because then almost every state encountered becomes a &quot;transposition&quot;. You'll already have seen all those states in a previous iteration with a lower search depth limit. Now, it is important to note with this combo of ID + TTs, that you can no longer completely cut off searches for all recognized transpositions. If an entry in the table holds a value that was computed with a search depth of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, that value will no longer be valid when performing a subsequent iteration of ID with a max search depth of &lt;span class=&quot;math-container&quot;&gt;$d + 1$&lt;/span&gt; for example. However, that &quot;outdated&quot; value stored in the TT &lt;strong&gt;can still be used for move ordering&lt;/strong&gt;, which can lead to significantly more prunings from alpha-beta pruning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;How efficient is the remainder of your engine?&lt;/strong&gt; A TT is not 100% free, it takes a bit of additional time too (for example to compute the hash values for your game states). If the rest of your engine is relatively slow (i.e. inefficient implementation for playing moves, copying game states, etc.), the computational overhead of the TT won't matter much and even a low number of recognized transpositions will still be valuable. If the rest of your engine is very fast, it'll be more important to have a high number of transpositions for the TT to be really valuable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As an &quot;educated guess&quot;, I'd say the number of 17% you describe is not necessarily strange. Especially given your edit to the question, where you indeed mention that it is likely that transpositions are found high up in the tree (close to the root). When this happens, you immediately remove the probability of recognizing all those states deeper down in the tree of getting recognized as transpositions yourself. So, the pool of states that could &quot;potentially&quot; be found in the TT is much less than 100% of the states stored in the TT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's really just that though, just an educated guess. It's going to be very difficult for anyone to give a conclusive &quot;yes&quot; or &quot;no&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-10-14T13:29:50.357" LastActivityDate="2018-10-14T13:29:50.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8405" PostTypeId="2" ParentId="8350" CreationDate="2018-10-14T14:22:00.670" Score="2" Body="&lt;p&gt;Per Neil's line of questioning:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're doing offline, supervised or reinforcement learning, where you have the dataset and decide what to show to the network next, there's no reason to do this. It's better to just adjust your training to show rare examples to the network more often.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're doing online reinforcement learning, then the agent typically controls which examples it sees next through its choice of actions. Increasing the agent's exploration rate will cause it to see a more diverse set of examples, but these examples are also less likely to be useful for solving the task the agent is working on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're doing supervised, online, learning, and you don't control the order in which examples appear, it might be useful to freeze some of the weights, but it might be better, and would certainly be simpler, to increase the learning rate for rare classes instead (in effect, showing the network a given rare example repeatedly when it &lt;em&gt;does&lt;/em&gt; show up, so that it takes longer to forget it).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2018-10-14T14:22:00.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8407" PostTypeId="1" CreationDate="2018-10-14T18:25:30.217" Score="6" ViewCount="123" Body="&lt;p&gt;So let's say you had a really nice day in a flight simulator and you are getting videos of this type of quality:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ftsYV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ftsYV.png&quot; alt=&quot;A snapshot of the compressed video&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is Full HD (1080p), but heavily compressed. You can literally see the pixels. Now I tried to use something like RAISR, and this &lt;a href=&quot;https://github.com/hugopilot/raisr&quot; rel=&quot;noreferrer&quot;&gt;python implementation&lt;/a&gt;, but it only scales the image up and does not 'fix the thicc pixels'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is there a type of AI that does fix this kind of video/photo into a reasonable quality video? I just want to get rid of those pixels and image artefacts that was generated during the compression.&lt;/p&gt;&#xA;" OwnerUserId="19013" LastEditorUserId="19013" LastEditDate="2018-10-14T18:40:04.567" LastActivityDate="2019-02-01T22:01:30.487" Title="Can AI 'fix' heavily compessed videos/photos?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8410" PostTypeId="1" AcceptedAnswerId="8464" CreationDate="2018-10-14T20:35:35.353" Score="-1" ViewCount="93" Body="&lt;p&gt;I am preparing a binary classifier. Initially, I used the following parameters based on the well-known cat and dog classifier example;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;train_datagen = ImageDataGenerator(rescale=1./255)&#xA;validation_datagen = ImageDataGenerator(rescale=1./255)&#xA;test_datagen = ImageDataGenerator(rescale=1./255)&#xA;# Data Generator for Training data&#xA;train_generator = train_datagen.flow_from_directory(train_dir, target_size=(224, 224), batch_size=32, class_mode='binary')    &#xA;# Data Generator for Validation data&#xA;validation_generator = validation_datagen.flow_from_directory(validation_dir, target_size= (224, 224), batch_size=16, class_mode='binary', shuffle=False)                                                              &#xA;# Data Generator for Test data&#xA;test_generator = validation_datagen.flow_from_directory(test_dir, target_size=(224, 224), batch_size=16, class_mode='binary', shuffle=False)                                                  &#xA;# Compile the model&#xA;model.compile(loss='binary_crossentropy', optimizer=optimizers.RMSprop(lr=1e-4), metrics=['acc'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the model was;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Create a sequential model&#xA;model = models.Sequential()&#xA;# Add the vgg convolutional base model&#xA;model.add(vgg_conv)&#xA;# Add new layers&#xA;model.add(layers.Flatten())&#xA;model.add(layers.Dense(1024, activation='relu'))&#xA;model.add(layers.Dropout(0.5))&#xA;model.add(layers.Dense(1, activation='sigmoid'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I use VGG16 with imagenet weights by adding these layers at the end of the model. Cat and dog classifier works with good accuracy and I was sure this model is going to give me very good results for the initial steps. However, when I checked the confusion matrix I saw that all the images are classified as they belong to category 0. Then I replaced all &amp;quot;binaries&amp;quot; with &amp;quot;categorical&amp;quot; and used a softmax activator with 2 units. All the rest is the same and now I obtain very good results. I do not understand how this happens? Why the first configuration works very well with another dataset but gives me garbage results with my own dataset? It is better to try as much as possible but it takes a bit less than a day with my dataset. I also do not understand how a binary classifier works with only one unit at the output layer since the actual categories are two.&lt;/p&gt;&#xA;" OwnerUserId="18283" LastEditorUserId="32410" LastEditDate="2021-05-03T18:26:24.187" LastActivityDate="2021-05-03T18:26:24.187" Title="Number of units of the last layer" Tags="&lt;keras&gt;&lt;vgg&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2021-01-23T04:56:32.107" ContentLicense="CC BY-SA 4.0" />
  <row Id="8412" PostTypeId="1" AcceptedAnswerId="8421" CreationDate="2018-10-15T04:27:41.923" Score="3" ViewCount="577" Body="&lt;p&gt;In keras, when we use an LSTM/RNN model, we need to specify the node [i.e., LSTM(128)]. I have a doubt regarding how it actually works. From the LSTM/RNN unfolding image or description, I found that each RNN cell take one time step at a time. What if my sequence is larger than 128? How to interpret this? Can anyone please explain me? Thank in advance.&lt;/p&gt;&#xA;" OwnerUserId="18795" LastEditorUserId="2444" LastEditDate="2019-02-16T02:32:25.597" LastActivityDate="2019-02-16T02:32:25.597" Title="What should I do when I have a variable-length sequence when instantiating an LSTM in Keras?" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8414" PostTypeId="1" AcceptedAnswerId="8507" CreationDate="2018-10-15T06:04:36.427" Score="4" ViewCount="1338" Body="&lt;p&gt;&lt;em&gt;Following-up my &lt;a href=&quot;https://ai.stackexchange.com/questions/8338/interpretation-of-a-good-overfitting-score&quot;&gt;question&lt;/a&gt; about my over-fitting network&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My deep neural network is over-fitting : &#xA;&lt;a href=&quot;https://i.stack.imgur.com/6xKGF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6xKGF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried several things : &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Simplify the architecture&lt;/li&gt;&#xA;&lt;li&gt;Apply more (and more !) Dropout&lt;/li&gt;&#xA;&lt;li&gt;Data augmentation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But I always reach similar results : training accuracy is eventually going up, while validation accuracy never exceed ~70%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I simplified enough the architecture / applied enough dropout, because my network is even too dumb to learn anything and return random results (3-classes classifier =&gt; 33% is random accuracy), even on training dataset :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nh4Bs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nh4Bs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : &lt;strong&gt;This accuracy of 70% is the best my model can reach ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why the training accuracy reach such high scores, and why so fast, knowing this architecture seems to be not compatible ?&lt;/li&gt;&#xA;&lt;li&gt;My only option to improve the accuracy is then to change my model, right ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If no :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are my options to improve this accuracy ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I'v tried a bunch of hyperparameters, and a lot of time, depending of these parameters, the accuracy does not change a lot, always reaching ~70%. However I can't exceed this limit, even though it seems easy to my network to reach it (short convergence time)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here is the Confusion matrix :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S2Hro.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S2Hro.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think the data or the balance of the class is the problem here, because I used a well-known / explored dataset : &lt;a href=&quot;https://nlp.stanford.edu/projects/snli/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNLI Dataset&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the learning curve :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MjFPu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MjFPu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note : I used accuracy instead of error rate as pointed by the resource of Martin Thoma&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's really ugly one. I guess there is some problem here.&#xA;Maybe the problem is that I used the result after 25 epoch for every values. So with little data, training accuracy don't really have time to converge to 100% accuracy. And for bigger training data, as pointed in earlier graphs, the model overfit so the accuracy is not the best one.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="18852" LastEditDate="2018-10-15T13:35:12.087" LastActivityDate="2018-10-24T00:12:52.170" Title="How to improve testing accuracy when training accuracy is high?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;overfitting&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8417" PostTypeId="1" AcceptedAnswerId="8420" CreationDate="2018-10-15T07:46:52.850" Score="1" ViewCount="1310" Body="&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1608.06993v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DenseNet architecture&lt;/a&gt; can be summarizde with this figure:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cSwqp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cSwqp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why there are transition layers between each block?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the papers, they justify the use of transition layers as follow :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The concatenation operation used in Eq. (2) is not viable when the size of feature-maps changes. However,  an essential part of convolutional networks is pooling layers that change the size of feature-maps. To facilitate pooling in our architecture we divide the network into multiple densely connected dense blocks&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, if I understood correctly, the problem is that the feature map size can change, thus we can't concatenate. But how does the addition of transition layers solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And how can several dense blocks connected like this be more efficient that one single bigger dense block?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, why are all standard DenseNets made of 4 dense blocks? I guess I will have the answer to this question if I understood better the previous questions.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="2444" LastEditDate="2020-03-10T20:35:02.407" LastActivityDate="2020-03-10T20:37:11.467" Title="Why are there transition layers in DenseNet?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8420" PostTypeId="2" ParentId="8417" CreationDate="2018-10-15T08:24:02.407" Score="1" Body="&lt;p&gt;The point of DenseNet was to go as deep as ResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About transition layers (convolution + pooling), I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from &lt;span class=&quot;math-container&quot;&gt;$56 \times 56$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$28 \times 28$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$14 \times 14$&lt;/span&gt;, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors state it this way&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To further improve model compactness, we can reduce the number of feature-maps at transition layers&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="19027" LastEditorUserId="2444" LastEditDate="2020-03-10T20:37:11.467" LastActivityDate="2020-03-10T20:37:11.467" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8421" PostTypeId="2" ParentId="8412" CreationDate="2018-10-15T08:31:44.950" Score="1" Body="&lt;p&gt;In Keras, what you specify is the hidden layer size. So :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LSTM(128)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives you a Keras layer representing a LSTM with a hidden layer size of 128.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you said :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From the LSTM/RNN unfolding image or description, I found that each RNN cell take one time step at a time&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So if you picture your RNN for one time step, it will look like this : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zbbpu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zbbpu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if you unfold it &lt;strong&gt;in time&lt;/strong&gt;, it look like this : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0yUiz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0yUiz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are not limited in your sequence size, this is one of the feature of RNN : since you input your sequence element by element, the size of the sequence can be variable.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That number, 128, represent just the size of the hidden layer of your LSTM. You can see the hidden layer of the LSTM as the memory of the RNN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the goal is not for the LSTM to remember everything of the sequence, just link between elements. That's why the size of the hidden layer can be smaller than the size of your sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sources :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://keras.io/layers/recurrent/#lstm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras documentation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://adventuresinmachinelearning.com/keras-lstm-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This blog&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://skymind.ai/wiki/lstm#long&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The larger the network, the more powerful, but it’s also easier to overfit. Don’t want to try to learn a million parameters from 10,000 examples – parameters &gt; examples = trouble.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the consequence of reducing the size of hidden state of LSTM is that it will be simpler. Might not be able to get the links between the element of the sequence. But if you put a too big size, your network will overfit ! And you absolutely don't want that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another really good blog on LSTM : &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="18852" LastEditDate="2018-10-15T23:32:40.617" LastActivityDate="2018-10-15T23:32:40.617" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8422" PostTypeId="1" CreationDate="2018-10-15T10:10:12.483" Score="2" ViewCount="689" Body="&lt;p&gt;Is the laptop Asus ZenBook Pro 90NX0152-M02980 enough to do deep learning model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Processeur Intel Core i7-7500U (Dual-Core 2.7 GHz / 3.5 GHz Turbo -&#xA;cache 4 MB) &lt;/li&gt;&#xA;&lt;li&gt;8 GB Memory&lt;/li&gt;&#xA;&lt;li&gt;SSD M.2 SATA de 256GB&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19030" LastEditorUserId="7800" LastEditDate="2018-10-18T03:11:54.760" LastActivityDate="2018-12-17T17:02:00.420" Title="Asus zenbook with deep learning" Tags="&lt;deep-learning&gt;&lt;hardware-evaluation&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2020-12-06T17:33:56.910" ContentLicense="CC BY-SA 4.0" />
  <row Id="8424" PostTypeId="1" CreationDate="2018-10-15T11:24:32.070" Score="2" ViewCount="650" Body="&lt;p&gt;I want to make a kind of robotic brain, i.e. a big neural network, which includes an NLP model (for understanding human voice), real-time object recognition system (so that it can identify particular objects), a face recognition model (for identifying faces), etc.&lt;/p&gt;&#xA;&lt;p&gt;Is possible to build a huge neural network in which we can combine all these separate models together, so we can use all 3 model's capabilities at same time in parallel?&lt;/p&gt;&#xA;&lt;p&gt;For example, if I ask the robot, using the microphone, &amp;quot;Can you see that table or that boy?&amp;quot;, the robot would start recognizing the objects and faces, then answer me back by speaking if it could identify them or not.&lt;/p&gt;&#xA;&lt;p&gt;If this is possible, can you kindly share your idea how can I implement this? Or is there any better to make such AI (e.g. in TensorFlow)?&lt;/p&gt;&#xA;" OwnerUserId="14527" LastEditorUserId="2444" LastEditDate="2021-05-24T12:24:53.707" LastActivityDate="2021-05-24T12:24:53.707" Title="Can we combine multiple different neural networks in one?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;reference-request&gt;&lt;robots&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8427" PostTypeId="1" CreationDate="2018-10-15T12:38:11.573" Score="10" ViewCount="4098" Body="&lt;p&gt;What exactly are ontologies in AI? How should I write them and why are they important?&lt;/p&gt;&#xA;" OwnerUserId="18123" LastEditorUserId="2444" LastEditDate="2019-04-30T17:39:47.410" LastActivityDate="2021-02-23T13:46:11.740" Title="What are ontologies in AI?" Tags="&lt;ai-design&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;ontology&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8429" PostTypeId="1" CreationDate="2018-10-15T14:07:42.130" Score="3" ViewCount="64" Body="&lt;p&gt;The process revolves around a child's drawing. Each part of each drawing corresponds to a score as in the &lt;a href=&quot;https://pineight.com/mw/?title=Draw-A-Person_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Draw a Person Test&lt;/em&gt; conceived by Dr. Florence Goodenough in 1926&lt;/a&gt;. The goal of the machine is to measure a child's mental age through a figure drawing task.&lt;/p&gt;&#xA;" OwnerUserId="18027" LastEditorUserId="4302" LastEditDate="2018-10-15T23:01:19.387" LastActivityDate="2018-10-15T23:01:19.387" Title="How to use Machine Learning to create a &quot;Draw-A-Person Test&quot;" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;image-recognition&gt;&lt;intelligence-testing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8430" PostTypeId="1" CreationDate="2018-10-15T15:28:58.157" Score="1" ViewCount="159" Body="&lt;p&gt;I have coded an AI checkers game but would like to see how good it is. Some people have informed me to use the Chinook AI opensource code. But I am having trouble trying to integrate that software into my AI code. How do I integrate another game engine in checkers with the AI I have coded?&lt;/p&gt;&#xA;" OwnerUserId="16906" LastActivityDate="2018-10-15T15:28:58.157" Title="Checkers AI game engines" Tags="&lt;game-ai&gt;&lt;checkers&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="8432" PostTypeId="1" AcceptedAnswerId="8434" CreationDate="2018-10-15T17:55:35.607" Score="8" ViewCount="1861" Body="&lt;p&gt;My background is in electrical engineering. I have a good grasp of CS foundations (e.g. data structures, algorithms, operating systems, discrete math and software engineering).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have option of enrolling in a MS program in applied math at a good school. However, I would like to study AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which areas of applied math are relevant to AI (&lt;em&gt;not&lt;/em&gt; specifically ML)?&lt;/p&gt;&#xA;" OwnerUserId="19040" LastEditorUserId="2444" LastEditDate="2019-06-17T20:12:56.210" LastActivityDate="2019-06-17T20:12:56.210" Title="Which areas of applied math are relevant to AI?" Tags="&lt;math&gt;&lt;getting-started&gt;&lt;academia&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="6" ClosedDate="2020-03-04T04:29:25.017" ContentLicense="CC BY-SA 4.0" />
  <row Id="8434" PostTypeId="2" ParentId="8432" CreationDate="2018-10-15T19:39:19.770" Score="9" Body="&lt;p&gt;&lt;strong&gt;EE Math Refresh&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is an interdisciplinary field.  You can begin by ensuring you are fresh in the mathematics you've already taken. You may already have all the books from your BS and MS.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Infinite series&lt;/li&gt;&#xA;&lt;li&gt;Logical proofs&lt;/li&gt;&#xA;&lt;li&gt;Linear algebra and matrices&lt;/li&gt;&#xA;&lt;li&gt;Analytic geometry, especially the distinction between local and global extremes (minima and maxima), saddle points, and points of inflection&lt;/li&gt;&#xA;&lt;li&gt;Set theory&lt;/li&gt;&#xA;&lt;li&gt;Probability, especially &lt;em&gt;Bayes' Theorem&lt;/em&gt; and the common probability distributions&lt;/li&gt;&#xA;&lt;li&gt;Statistics, especially variance and correlation coefficient&lt;/li&gt;&#xA;&lt;li&gt;Time series&lt;/li&gt;&#xA;&lt;li&gt;Infinite series&lt;/li&gt;&#xA;&lt;li&gt;Convergence &amp;mdash; Central to AI batch processing&lt;/li&gt;&#xA;&lt;li&gt;Partial differentials&lt;/li&gt;&#xA;&lt;li&gt;Jacobian and Hessian matrices&lt;/li&gt;&#xA;&lt;li&gt;Multivariate math &amp;mdash; Spaces beyond &amp;#8477;&lt;sup&gt;3&lt;/sup&gt; are common in AI.&lt;/li&gt;&#xA;&lt;li&gt;Boundary regions&lt;/li&gt;&#xA;&lt;li&gt;Discrete math&lt;/li&gt;&#xA;&lt;li&gt;Criteria for system stability &amp;mdash; Central to real time intelligent control&lt;/li&gt;&#xA;&lt;li&gt;Bernhard Riemann's manifolds is important for more advanced AI theory&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An EE would already understand circuit paths (Gustav Kirchhoff's work, signal transmission, feedback theory, and control theory. To see the connection between AI and EE, Norbert Wiener's seminal work, &lt;em&gt;Cybernetics&lt;/em&gt;, 1948, MIT Press, will be a helpful read and give you a clearer picture that both fields have common roots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finite Math&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since finite math curricula differ between schools of engineering, this list can be used as a checklist for AI.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Predicate logic and rules engines&lt;/li&gt;&#xA;&lt;li&gt;Graphs (as in vertices connected by edges) and algorithms for them &amp;mdash; pioneered by Leonhard Euler&lt;/li&gt;&#xA;&lt;li&gt;Game theory &amp;mdash; John von Neumann and Oskar Morgenstern's &lt;em&gt;Game Theory&lt;/em&gt; (published in 1944 but still fresh and pertinent)&lt;/li&gt;&#xA;&lt;li&gt;Decision trees&lt;/li&gt;&#xA;&lt;li&gt;Markov chains and the Markov property &amp;mdash; Andrey Markov's work&lt;/li&gt;&#xA;&lt;li&gt;Information theory &amp;mdash; Claude Shannon's extensions of the thermodynamic concepts of entropy, especially uniqueness, redundancy, relative entropy, and cross entropy &amp;mdash; A necessity to more fully understand feature abstraction and auto-encoding&lt;/li&gt;&#xA;&lt;li&gt;Chaos theory, especially auto-correlation, for analysis of chaotic system behavior in phase space &amp;mdash; &lt;em&gt;Chaos Theory Tamed&lt;/em&gt;, Garnett P. Williams, 1997, provides an excellent overview&lt;/li&gt;&#xA;&lt;li&gt;Difference between random and pseudo random number generation&lt;/li&gt;&#xA;&lt;li&gt;Curve fitting and gradient descent, especially the Levenberg–Marquardt algorithm&lt;/li&gt;&#xA;&lt;li&gt;Stateless versus stateful algorithms, AI system components, and services&lt;/li&gt;&#xA;&lt;li&gt;Algorithmic theory, especially tail recursion and algorithms for generalization, abstraction, and object recognition&lt;/li&gt;&#xA;&lt;li&gt;Gödel's uncertainty theorems&lt;/li&gt;&#xA;&lt;li&gt;Turing completeness and how it does not overturn yet overcomes Gödel's uncertainty&lt;/li&gt;&#xA;&lt;li&gt;Turing's Imitation Game as a test for conversational intelligence&lt;/li&gt;&#xA;&lt;li&gt;Other test criteria for intelligence outside the conversation context (i.e. piloting, driving, business intelligence, &lt;em&gt;et cetera&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Topology &amp;mdash; pioneered by Henri Poincaré&lt;/li&gt;&#xA;&lt;li&gt;Computational linguistics &amp;mdash; pioneered by Richard Hook Richens &amp;mdash; the foundation of much NLP (natural language processing)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Machine Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since machine learning is trending and will probably remain strong even as other AI technologies catch up, there are a few&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MLPs (multilayer perceptrons)&lt;/li&gt;&#xA;&lt;li&gt;Convolution and the use of convolution layers and pooling layers in deep convolution networks&lt;/li&gt;&#xA;&lt;li&gt;LSTM (long-short term memory) networks, their predecessor RNNs (recurrent neural networks), and some of the newer competitors in the stateful network space&lt;/li&gt;&#xA;&lt;li&gt;Attention based networks and other gating strategies (largely found online using academic searches for papers)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Soft Sciences&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because most of what is occurring, has occurred, and will occur in AI for the foreseeable future is the replication of human brain capabilities, the EE can point her or his analytical skills at biology (especially neurology), genetics as it may apply to the emergence and control of human intelligence (bioinformatics), psychology (especially cognitive science), and sociology (such as.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of the emergence of GANs and other devices that work on the basis of balance and equilibrium, it is useful to recall chemical equilibria and biological stasis.  These concepts apply directly to AI and are likely to play a more major role as the gap between brain network research and computing narrows further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Automated Vehicles&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The obvious emerging AI technology that is nicely aligned with the EE background is the movement of road and air vehicles from human driving and piloting through the various phases of automation.  The goal that most corporations see as the ideal is to progress forward until the driver's seats and cockpits are no longer installed in cars and planes because human interaction other than to provide the transportation destination poses a public safety issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, computer vision, object recognition, sound recognition, and collision avoidance technique are essentials and highly dependent upon rigorous mathematical treatment.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-17T18:50:40.647" LastActivityDate="2018-10-17T18:50:40.647" CommentCount="0" CommunityOwnedDate="2021-01-27T00:05:15.917" ContentLicense="CC BY-SA 4.0" />
  <row Id="8435" PostTypeId="1" CreationDate="2018-10-15T19:47:42.013" Score="5" ViewCount="265" Body="&lt;p&gt;I am new to the object recognition community. Here I am asking about &lt;strong&gt;the broadly accepted&lt;/strong&gt; ways to calculate the error rate of a deep CNN when the network produces different results using the same data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;1. Problem introduction&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently I was trying to replicate some classic deep CNNs for the object recognition tasks. Inputs are some 2D image data including objects and the output are the identification/classification results of the object. The implementation involves the use of Python and Keras.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem&lt;/strong&gt; I was facing is that, I may get different validation results among multiple runs of the training even using the same training/validation data sets. To me, that made it hard to report the error rate of the model since every time the validation result may be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this difference is because of the randomness involved in different aspects of deep CNN, such as random initialization, the random ‘dropout’ used in the regulation, the ‘shuffle’ process used in the choosing of epochs, etc. But I do not know yet the “right” ways to deal with this difference when I want to calculate the error rate in object recognition field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;2. My exploration – online search&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found some answers online &lt;a href=&quot;https://machinelearningmastery.com/reproducible-results-neural-networks-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The author proposed two ways, and he/she recommended the first one shown below:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The traditional and practical way to address this problem is to run your network many times (30+) and use statistics to summarize the performance of your model, and compare your model to other models.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The second way he/she introduced is to go to every relevant aspect of the deep CNN, to &quot;freeze&quot; their randomness nature on purpose. This kind of approach has also been introduced from Keras Q&amp;amp;A &lt;a href=&quot;https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. They call this issue the “making reproductive results”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;3. My exploration – in academia community (&lt;strong&gt;no result yet, need your help!&lt;/strong&gt;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I was not sure whether the two ways mentioned above are the “right” ones broadly accepted, I was going further exploring in the object recognition academia community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I just begin to read from &lt;a href=&quot;http://image-net.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;imageNet&lt;/a&gt; website. But I have not found the answer yet. Maybe you could help me knowing the answer easier. Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Daqi&lt;/p&gt;&#xA;" OwnerUserId="19042" LastEditorUserId="19042" LastEditDate="2018-10-30T02:39:54.697" LastActivityDate="2021-05-19T12:00:20.103" Title="What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8436" PostTypeId="1" AcceptedAnswerId="8438" CreationDate="2018-10-15T20:41:39.347" Score="0" ViewCount="63" Body="&lt;p&gt;In the early 20th century, anything Princeton staff represented related to computing and the simulation of human thought was assumed to be flawless.  In the late 20th century, it was MIT's AI lab that was trusted to define what was feasible and what were the next steps.  Is Google the new source of legitimacy in AI pronouncements?  Because of the connection between that corporation and the daily life of people on earth, has that publication honor that was given to the primary AI centers of research in the 20th century reached a level in the early 21st century that rivals deity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sometimes seems that, when people don't understand something in a paper with the word Google anywhere in the authorship block, it is automatically assumed that the reader is wrong because the paper is of universal authority.  They seem to jump to the assumption that their mind must adjust to whatever the paper states.  Is this equivalent to religious scripture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Carl Jung &lt;a href=&quot;https://www.quotetab.com/quote/by-carl-jung/contemporary-man-is-blind-to-the-fact-that-with-all-his-rationality-and-efficien#eg8hD8Fstw68DmSQ.99&quot; rel=&quot;nofollow noreferrer&quot;&gt;wrote&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Contemporary man is blind to the fact that, with all his rationality and efficiency, he is possessed by &quot;powers&quot; that are beyond his control. His gods and demons have not disappeared at all; they have merely got new names.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is an AI StackExchange, so the proposal in this question is that perhaps the same brain machinery has been used for the following things&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An ancient Egyptian gathering with others to worship the sun god Re&lt;/li&gt;&#xA;&lt;li&gt;Casiodoris using the resources of the Roman Empire to send holy scripture out on horseback to the far reaches of the empire to save it from the invading barbarian hordes&lt;/li&gt;&#xA;&lt;li&gt;AI researchers finding some unassailable source of truth in a university or a multinational corporation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is Carl Jung correct, and, if so, is this human need to find legitimacy in an institution now manifesting in the AI community's absolute trust in a corporation that automated the 18th century library card catalog and built that automaton into a large multinational corporation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, if both those questions can be answered in the affirmative, what risks can exist when the human mind treats a university or a corporation as a god from an economic or social risk management perspective?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-16T18:01:56.787" Title="Has Google's view of AI risen to the status of universal truth?" Tags="&lt;social&gt;&lt;risk-management&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8438" PostTypeId="2" ParentId="8436" CreationDate="2018-10-15T22:23:56.257" Score="1" Body="&lt;p&gt;To your first question, there are two stages that keep us far from ai realization. Stage one: gods' ai. As to say man must first conjure a way to produce something capable of reason. Stage two: mother's touch to ai. As to say, the first intelligent non-organic creature will not be as smart as its true potential. Just as we as humans must strive to realize what is and what could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is being created today are sharp, efficient tools. They are nothing more and never will be anything but what they are. When a tool is made every aspect is thought through and perfected to its makers ends. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Google or another company would like to lay claim to the making of their tools, then who are we to criticize such patents of thought? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to the second question, there are no gods, no love, no sunshine without the lessons from mother. The term mother in this case is a loose term to describe the lessons, or &quot;programming&quot; humans receive from the day the emerge out into the ever demanding world that humans inhabit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;powers&quot; man is possessed by can be called many things, but in its raw form it is simply a programmatic inflection as a true driver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My programming had me come on here and exchange my thoughts as I have, and yours the same in your own way. You question because, there are not any powers beyond you to force you otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your last question, Yuval Harari summed this question up best in his book Homo Deus. I will leave it short and sweet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He explained that when institutions were first created, they were used to serve man. Today, institutions are entities in which we as humans put everything we have into them, for little to nothing in return.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question touches on so many thought processes. I will keep my thoughts short for now.&lt;/p&gt;&#xA;" OwnerUserId="11000" LastActivityDate="2018-10-15T22:23:56.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8439" PostTypeId="2" ParentId="8435" CreationDate="2018-10-15T22:26:48.843" Score="0" Body="&lt;p&gt;Unless the entropy is collected from quantum, thermal, or other true entropy sources, it is considered pseudo-random at best.  This is why you can have a deterministic process that is not perfectly repeatable, exhibiting some variance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;List of Sources of Pseudo-random Behavior&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The list of sources in the question is a good start.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Initialization&lt;/li&gt;&#xA;&lt;li&gt;Ddropout used in regulation&lt;/li&gt;&#xA;&lt;li&gt;Mini-batch shuffle&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If stochastic gradient descent is used in the back-propagation, that could be an additional source, and there may be others.  It is difficult to say without scrutinizing the code.  CDN emerged along different paths from CNN and MLP so there is no unambiguously distinct classical version of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Right Way&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the variance is a known phenominon and the norm, there may be no particular question for which the academic community needs to be consulted for an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The right ways to deal with the non-repeatability of validation statistics related to object recognition also has a norm: Follow standard reporting conventions that apply to all experimental results reporting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only solution that is mathematically rigorous and not changing the experiment to produce a desired result (which is not science) is the first of the two found in your investigation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The suggestion to perform &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; runs where &lt;span class=&quot;math-container&quot;&gt;$N \ge 30$&lt;/span&gt; and report mean and variance (not standard deviation) is reasonable and responsible.&lt;/li&gt;&#xA;&lt;li&gt;The suggestion to do this with &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; ADDITIONAL models where &lt;span class=&quot;math-container&quot;&gt;$O \ge 3$&lt;/span&gt; is another such practice.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Freezing is Necessarily Arbitrary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some would consider freezing pseudo-random sources arbitrarily something other than, &quot;making reproductive results,&quot; and consider it fabricating reproducable results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the temptation to please the reader or corporate stakeholder occasionally preempts truth in reporting.  The result is counterproductive.  In the worst case, it leads to broad acceptance of the wrong way.  Those in the academic community should know that, whether or not they unanimously do.  Be a leader and do what is backed by solid reasoning, whether that is the first suggestion found or a third choice that does not involve modifying the experiment arbitrarily to force results to appear more exact than the algorithm, methodology, or framework actually is.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-10-15T22:26:48.843" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8440" PostTypeId="2" ParentId="8414" CreationDate="2018-10-15T23:15:50.080" Score="1" Body="&lt;p&gt;I think sometimes it can also help to examine your test and training sets. &#xA;Fundamentally, your data was produced by an underlying process/system that has certain properties. The system can have many &quot;states&quot; and all the possible states form the state space. If you have really tried things like dropout and regularization, my guess would be that the test set is somehow different from your train set. It is possible that your training set only takes samples from one part of the state space (AKA, your samples might all be similar in the training set and the test set has different samples - imagine you are classifying humans and all of your training samples have a class label of 1 meaning all the training samples have humans in them -&gt; and all your test samples have no humans in them Good luck with that!).  Some questions to ask: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are you combining datasets from different sources? If so: &#xA;   If you have &quot;n&quot; sources of data, you need to make sure that your training set has many samples from each of the &quot;n&quot; sources of data and your test set has samples from each of the &quot;n&quot; sources. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are you shuffling your data enough and randomly putting samples in both the training and test sets? This relates to the human example I gave, make sure your training set has a little bit of everything (different combinations of inputs and/or outputs) and your testing set has a little bit of everything (different combinations of inputs and/or outputs). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15428" LastActivityDate="2018-10-15T23:15:50.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8441" PostTypeId="5" CreationDate="2018-10-16T00:25:23.083" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:25:23.083" LastActivityDate="2018-10-16T00:25:23.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8442" PostTypeId="4" CreationDate="2018-10-16T00:25:23.083" Score="0" Body="For questions focusing on the reliability, which includes the specification of the proportion of examples during training that have or are expected to converge with the required accuracy or at all, the reliability performance in field use, the coping with the breadth of variance in input, and the resistance to process or system failure." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-18T03:17:54.427" LastActivityDate="2018-10-18T03:17:54.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8443" PostTypeId="5" CreationDate="2018-10-16T00:27:34.027" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:27:34.027" LastActivityDate="2018-10-16T00:27:34.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8444" PostTypeId="4" CreationDate="2018-10-16T00:27:34.027" Score="0" Body="For questions related to the act of clipping values of rewards observed in a problem (typically a Markov Decision Process) to a limited range (often limiting rewards to the range [-1, 1]). This is sometimes done in an effort to stabilize learning processes (most notably in the DQN algorithm and related algorithms in Reinforcement Learning problems such as the Atari games)." OwnerUserId="4302" LastEditorUserId="1641" LastEditDate="2018-10-21T20:42:59.910" LastActivityDate="2018-10-21T20:42:59.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8445" PostTypeId="5" CreationDate="2018-10-16T00:29:27.317" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:29:27.317" LastActivityDate="2018-10-16T00:29:27.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8446" PostTypeId="4" CreationDate="2018-10-16T00:29:27.317" Score="0" Body="For questions related to the automation of mathematical theorem proving, including rules systems, the acquisition of predicate logic and inference by deep networks, combinations of these, or novel new designs and supporting algorithms." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-18T03:15:52.543" LastActivityDate="2018-10-18T03:15:52.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8447" PostTypeId="5" CreationDate="2018-10-16T00:30:18.903" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:30:18.903" LastActivityDate="2018-10-16T00:30:18.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8448" PostTypeId="4" CreationDate="2018-10-16T00:30:18.903" Score="0" Body="For questions related to AI products or products containing AI components and their viability in real markets today or in the future." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-18T03:15:47.717" LastActivityDate="2018-10-18T03:15:47.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8449" PostTypeId="5" CreationDate="2018-10-16T00:34:18.833" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:34:18.833" LastActivityDate="2018-10-16T00:34:18.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8450" PostTypeId="4" CreationDate="2018-10-16T00:34:18.833" Score="0" Body="For questions about the concept of attention in artificial intelligence and machine learning. Attention-like mechanisms were successfully used in natural language processing and computer vision tasks, such as machine translation. For a review of attention-based mechanism used in NLP, take a look at &quot;Attention in Natural Language Processing&quot; by Andrea Galassi et al." OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-09-21T17:05:06.660" LastActivityDate="2020-09-21T17:05:06.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8451" PostTypeId="5" CreationDate="2018-10-16T00:47:01.840" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:47:01.840" LastActivityDate="2018-10-16T00:47:01.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8452" PostTypeId="4" CreationDate="2018-10-16T00:47:01.840" Score="0" Body="For questions related to spiking neural networks (SNNs) that more accurately model biological neurons in the brain than perceptrons or RNN cells by simulating pulses within and across neurons rather than simulating signal values that remain constant for each learning iteration (by example index or batch index for mini-batch)." OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-04-22T13:28:10.983" LastActivityDate="2020-04-22T13:28:10.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8453" PostTypeId="5" CreationDate="2018-10-16T00:56:06.300" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:56:06.300" LastActivityDate="2018-10-16T00:56:06.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8454" PostTypeId="4" CreationDate="2018-10-16T00:56:06.300" Score="0" Body="For questions related to the synthesis of speech, not to be confused with synthesizing text or formal language expressions or expressions in context free grammars. Speech in this context is either a sequence of audio samples, a sequence of spectral representations in the frequency domain, or a representation of phonic symbols that represent natural speech. " OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-18T03:17:38.917" LastActivityDate="2018-10-18T03:17:38.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8455" PostTypeId="5" CreationDate="2018-10-16T00:58:50.157" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-10-16T00:58:50.157" LastActivityDate="2018-10-16T00:58:50.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8456" PostTypeId="4" CreationDate="2018-10-16T00:58:50.157" Score="0" Body="For questions related to computing using continuous rather than discrete values in analog rather than digital circuitry or the simulation of analog computing through digital simulation tools such as variants of spice or proprietary systems." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-18T03:15:41.003" LastActivityDate="2018-10-18T03:15:41.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8457" PostTypeId="2" ParentId="8432" CreationDate="2018-10-16T03:59:11.773" Score="1" Body="&lt;p&gt;To get a solid foundation of most of the ideas behind machine learning techniques, you would need&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear Algebra&lt;/li&gt;&#xA;&lt;li&gt;Vector Calculus&lt;/li&gt;&#xA;&lt;li&gt;Statistics&lt;/li&gt;&#xA;&lt;li&gt;Basic programming skills (e.g. NumPy)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, the more you know the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To study other aspects of AI (e.g. the theory of agents with unbounded computation power), you need to study&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decision theory&lt;/li&gt;&#xA;&lt;li&gt;Game theory&lt;/li&gt;&#xA;&lt;li&gt;Set theory&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2019-06-17T20:10:14.620" LastActivityDate="2019-06-17T20:10:14.620" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8458" PostTypeId="1" AcceptedAnswerId="8545" CreationDate="2018-10-16T06:20:22.073" Score="3" ViewCount="70" Body="&lt;p&gt;This is a question about pattern recognition and feature extraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with Hough transforms, the Fast Radial Transform and variants (e.g., GFRS), but these highlight circles, spheres, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need an image filter that will highlight the centroid of a series of spokes radiating from it, such as the center of a asterix or the spokes of a bicycle wheel (even if the round wheel is obscured.  Does such a filter exist?&lt;/p&gt;&#xA;" OwnerUserId="19049" LastActivityDate="2018-10-23T05:40:00.017" Title="How to recognize non-circular radial symmetry in images?" Tags="&lt;pattern-recognition&gt;&lt;feature-selection&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8459" PostTypeId="2" ParentId="8412" CreationDate="2018-10-16T07:22:32.797" Score="0" Body="&lt;p&gt;Although this question has been answered I'd add a couple of remarks towards general neural networks design&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you know very NN has three types of layers: input, hidden, and output. Once this network is initialized, you can iteratively tune the configuration during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To optimize the network configuration we can use pruning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pruning describes a set of techniques to trim network size (by nodes not layers) to improve computational performance and sometimes resolution performance. The gist of these techniques is removing nodes from the network during training by identifying those nodes which, if removed from the network, would not noticeably affect network performance (i.e., resolution of the data). (Even without using a formal pruning technique, you can get a rough idea of which nodes are not important by looking at your weight matrix after training; look weights very close to zero--it's the nodes on either end of those weights that are often removed during pruning.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more here: &lt;a href=&quot;https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw&quot;&gt;https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17925" LastActivityDate="2018-10-16T07:22:32.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8461" PostTypeId="1" AcceptedAnswerId="8462" CreationDate="2018-10-16T09:49:12.310" Score="3" ViewCount="73" Body="&lt;p&gt;I would like to get a simple example running in matlab that will use a neural net to learn an arbitrary function from input output data (basically model identification) and then be able to approximate that function from just the input data. As means of training this net I have implemented a simple back propagation algorithm in matlab but I was not able to get anywhere close to satisfactory results. I would like to know what I may be doing wrong and also what approach I may use instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to have the network represent an identified function f(x) which takes a series x as input and outputs the learned mapping from x -&gt; y. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the GNU octave code I have so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pkg load control signal&#xA;&#xA;function r = sigmoid(z)&#xA;    r = 1 ./ (1 + exp(-z));&#xA;end&#xA;&#xA;function r = linear(z)&#xA;    r = z;&#xA;end &#xA;&#xA;function r = grad_sigmoid(z)&#xA;    r = sigmoid(z) .* (1 - sigmoid(z));&#xA;end &#xA;&#xA;function r = grad_linear(z)&#xA;    r = 1;&#xA;end &#xA;&#xA;function r = grad_tanh(z)&#xA;    r = 1 - tanh(z) .^ 2;&#xA;end&#xA;&#xA;function nn = nn_init(n_input, n_hidden1, n_hidden2, n_output)&#xA;    nn.W2 = (rand(n_input, n_hidden1) * 2 - 1)'&#xA;    nn.W3 = (rand(n_hidden1, n_hidden2) * 2 - 1)'&#xA;    nn.W4 = (rand(n_hidden2, n_output) * 2 - 1)'&#xA;    nn.lambda = 0.005;&#xA;end&#xA;&#xA;function nn = nn_train(nn_in, state, action)&#xA;    nn = nn_in;&#xA;&#xA;    [out, nn] = nn_eval(nn, state);&#xA;&#xA;    d4 = (nn.a4 - action) .* grad_linear(nn.W4 * nn.a3); &#xA;    d3 = (nn.W4' * d4) .* grad_tanh(nn.W3 * nn.a2);&#xA;    d2 = (nn.W3' * d3) .* grad_tanh(nn.W2 * nn.a1);&#xA;&#xA;    nn.W4 -= nn.lambda * (d4 * nn.a3');&#xA;    nn.W3 -= nn.lambda * (d3 * nn.a2');&#xA;    nn.W2 -= nn.lambda * (d2 * nn.a1');&#xA;end&#xA;&#xA;function [out,nn] = nn_eval(nn_in, state)&#xA;    nn = nn_in;&#xA;&#xA;    nn.z1 = state;&#xA;    nn.a1 = nn.z1;&#xA;&#xA;    nn.a2 = tanh(nn.W2 * nn.a1);&#xA;    nn.a3 = tanh(nn.W3 * nn.a2);&#xA;    nn.a4 = linear(nn.W4 * nn.a3);&#xA;&#xA;    out = nn.a4;&#xA;end&#xA;&#xA;nn = nn_init(1, 100, 100, 1);&#xA;t = 1:0.1:3.14*10;&#xA;input = t;&#xA;output = sin(input);&#xA;learned = zeros(1, length(output));&#xA;&#xA;for j = 1:500&#xA;    for i = 1:length(input)&#xA;        nn = nn_train(nn, [input(i)], [output(i)]); &#xA;    end&#xA;    j&#xA;end&#xA;&#xA;for i = 1:length(input)&#xA;    learned(i) = nn_eval(nn, [input(i)]);    &#xA;end&#xA;&#xA;plot(t, output, 'g', t, learned, 'b');&#xA;&#xA;pause&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IK8Gn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IK8Gn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is not even close to where I want it to be. Has it got something to do with my implementation of back propagation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What changes do I need to do to the code to get a better approximation going? &lt;/p&gt;&#xA;" OwnerUserId="19056" LastActivityDate="2018-10-16T10:58:03.343" Title="Learning an arbitrary function using a feedforward net" Tags="&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8462" PostTypeId="2" ParentId="8461" CreationDate="2018-10-16T10:58:03.343" Score="3" Body="&lt;p&gt;You need to scale the input. Neural networks work best with a limited input domain, and train badly when it is exceeded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For statistical data, you would typically scale your input to have mean 0, standard deviation 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, you will be better off fitting the input to roughly -1 to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to you where you scale the values, but usually this is done outside of the NN code. So I would do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nn_input = (t - 15)/15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then use &lt;code&gt;nn_input&lt;/code&gt; in the training and evaluation loops. As you are putting these directly into a sorted array for plotting, you won't need to do any further re-mapping back or maintain a conversion function. However, in the more common case of arbitrary inputs, you would need to store the conversion factors somewhere (in this case just hardcoded as a function perhaps) in order to make use of the trained NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that may help is shuffling your input/output data pairs during training to remove correlation between sequence of input pairs.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-10-16T10:58:03.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8464" PostTypeId="2" ParentId="8410" CreationDate="2018-10-16T15:16:45.387" Score="0" Body="&lt;p&gt;This is simply because they have different loss functions. Categorical cross entropy measures the logarithmic loss for every neuron in the output layer. For binary crossentropy, we have 1 neuron in the output layer. In this case, you should use binary cross entropy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For Cats and dogs classification ,you have two classes Cats and Dogs. You should use categorical crossentropy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-10-16T15:16:45.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8465" PostTypeId="1" CreationDate="2018-10-16T15:43:07.630" Score="0" ViewCount="206" Body="&lt;p&gt;I already know the basics of the basic of Machine Learning. E.g.: Backpropagation, Convolution, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of let me explain Reinforcement learning to make sure I grasped the concept correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Reinforcement learning a random-initialized network will first &quot;play&quot;/&quot;do&quot; a sequence of moves in an environment. (In this case a Game). After that, it will receive a reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;. Furthermore a q-Value gets defined by the engineer/hooby coder. This reward times the q-Value &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; to the power of the  position &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of the action will be feeded back using BP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So how do I know how slight chances in &lt;span class=&quot;math-container&quot;&gt;$\vec{w}$&lt;/span&gt; are changing &lt;span class=&quot;math-container&quot;&gt;$rq^n$&lt;/span&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="1581" LastEditDate="2018-10-16T19:10:37.147" LastActivityDate="2018-10-16T19:10:37.147" Title="How do I know how changes in the weights are changing the reward in Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8466" PostTypeId="2" ParentId="8465" CreationDate="2018-10-16T17:03:08.297" Score="2" Body="&lt;p&gt;You have the concept slightly wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This part is mostly correct:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In Reinforcement learning a random-initialized network will first &quot;play&quot;/&quot;do&quot; a sequence of moves in an environment. (In this case a Game). After that, it will receive a reward r. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Technically neural networks are not required in RL, and it is really worth studying some simple systems that don't need them. It will make everything much clearer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; can be received on every time step. However, some environments will only have a single reward at the end for success or failure for a whole episode - e.g. an instance of a game like chess where a player wins or loses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This part is where things go a bit off track:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Furthermore a q-Value gets defined by the [developer]. This reward times the q-Value q to the power of the position n of the action will be [fed] back using BP.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Q values are one type of data that can be calculated for an agent acting in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process&lt;/a&gt;. They are also called &quot;action values&quot; and they are not usually defined by a developer. The q value, &lt;em&gt;if correct&lt;/em&gt; should return the expected future sum of rewards from following a current policy. One way of writing this is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q(s,a) = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty}\gamma^k R_{t+k+1}| S_t=s, A_t=a]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In natural language, the q value for state s and action a is the expected value (when following the given policy) of the discounted sum of rewards, starting from the given state and action. The discount factor, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; can take any value from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, but only strictly episodic problems (which always terminate) should use the value &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A developer does not get to define that (except they might get to choose reward system and value of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;). Instead, they need to implement &lt;em&gt;something&lt;/em&gt; that estimates the value of &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; based on what the agent has experienced. There are a few different algorithms that can do this. A popular one is called Q learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding &quot;[fed] back using BP&quot;, this is correct if you are using a neural network. Typically in DQN (Q learning with neural networks), this just consists of creating a small sample of training data from recent experience and training the neural network almost identically to supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So how do I know how slight chances in &lt;span class=&quot;math-container&quot;&gt;$\vec{w}$&lt;/span&gt; are changing &lt;span class=&quot;math-container&quot;&gt;$rq^n$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Definitely don't use &lt;span class=&quot;math-container&quot;&gt;$rq^n$&lt;/span&gt; - there is no purpose to that quantity in RL. Instead for value-based RL, you are mostly interested in your estimate for Q value. This might be written &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\vec{w}) \approx q(s,a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in general your question stands. If you have implemented a neural network to learn q values, how do you know if it is working?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are actually two parts to this problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How do you know whether the agent is getting better at it's task?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do you know whether the q values are getting more accurate?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What you need to do is measure, and maybe plot some relevant quantities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first question, you would typically plot the total reward that the agent gets each episode. This will be noisy, so it is a good idea to smooth it out by taking some kind of moving average (e.g. average total reward over last 100 episodes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second question, it is normal to plot some loss function of the network, just like supervised learning. Typically this is Mean Squared Error loss, as the network is learning a regression to predict q values given &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.  You can compare observed sums of discounted reward (aka &quot;return&quot; or &quot;utility&quot;) with the earlier predicted ones, and take the error function. You need to get some measure of a &quot;true&quot; value of q - usually a noisy sample taken during training or testing, and measure loss. For MSE that might be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\vec{w}) = \frac{1}{2|D|}\sum_{(s,a) \in D}(\hat{q}(s,a, \vec{w}) - q(s,a))^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is some dataset you have put together of &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; measurements to test with. If this looks familiar to you from supervised learning MSE loss, then that's correct - it is essentially the same thing, just different how you go about collecting the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may expect the loss function for &lt;span class=&quot;math-container&quot;&gt;$\hat{q}$&lt;/span&gt; in Q learning to be somewhat unstable as the agent learns. That's because in Q learning, the policy is updating at the same time as the estimates are improving. Which makes the estimates out-of-date. However, it should still be possible to see a reduction in error as learning progresses. If it becomes stable at a relatively low value compared to initially, then the agent has probably learned all that it can - although sometimes new discoveries by the agent can open up more improvements, even late in training, and throw the error function out again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that a low value of the error function does not mean you have an optimal agent. It means that the value function estimate is good for how the agent is currently behaving. In turn that means the agent cannot make further improvements without new and different experience.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-16T17:24:57.423" LastActivityDate="2018-10-16T17:24:57.423" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8467" PostTypeId="1" CreationDate="2018-10-16T17:14:28.100" Score="2" ViewCount="281" Body="&lt;p&gt;Given infinite resources/time one could create AGIs by writing code to simulate infinite worlds. In some of the worlds AGIs would be created. Detecting them would be another issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we don't have infinite resources the most probable way to create an AGI is to write some bootstrapping code which would reduce the resources/time to reasonable values.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In that AGI code (that would make it reasonable to create with finite&#xA;  resources/time) is it required to have a part that deals with&#xA;  time/space estimation of possible actions taken? Or should that be outside of the code and be something the AGI discovers by itself after it starts running?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any example of &lt;a href=&quot;https://github.com/fairy-tale-agi-solutions/awesome-artificial-general-intelligence#organizations--projects&quot; rel=&quot;nofollow noreferrer&quot;&gt;projects targeting AGI&lt;/a&gt; that are using time/space estimation might be useful for reaching a conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clarification, by time/space I mean time/space complexity analysis for algorithms, see: &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_efficiency#Measures_of_resource_usage&quot; rel=&quot;nofollow noreferrer&quot;&gt;Measures of resource usage&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Analysis_of_algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis of algorithms&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question formulation might lead people to think that the time/space estimation can only apply to some class of actions called algorithms. To clarify my mistake, I mean the estimation to apply to any action plan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine you are an AGI and you have to make a choice between different set of actions to pursue your goals. If you had 2 goals and one of them used less space and less time then you would always pick it over the other algorithm. So time/space estimation is very useful since intelligence is about efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is at least 1 exception though, imagine in the example before that the goal of the AGI is to pick the set of actions that leads to the most expensive time/space set of actions (or any non-minimal time/space cost) then obviously because of the goal constraint you would pick the most time/space expensive set of actions. In most other cases though, you would just pick the most time/space efficient algorithm.&lt;/p&gt;&#xA;" OwnerUserId="10826" LastEditorUserId="4302" LastEditDate="2018-10-20T11:20:57.150" LastActivityDate="2021-04-21T07:07:21.030" Title="Is time/space estimation of possible actions required for creating AGI?" Tags="&lt;agi&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8468" PostTypeId="2" ParentId="8422" CreationDate="2018-10-16T17:49:24.790" Score="0" Body="&lt;p&gt;&lt;strong&gt;Here is a little bit of aglimpse&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asus Zenbook pro,is quite good for technical nor software development tasks,despite having ordered one recently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;That said,you need to that each and every computer/machine for effective and efficient computing purpose,the first priority is to analyse critically the processor chip,which generational architecture/design does it have inline with GPU/Graphics ,Resolution,motherboard,hard-drive...e.t.c..&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore,basing on such technical specs,what matters most,is not the brand nor logo..! Go for any brand,keeping in mind your objective &quot;DeepLearning&quot;&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-10-16T17:49:24.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8469" PostTypeId="2" ParentId="8467" CreationDate="2018-10-16T17:59:59.170" Score="0" Body="&lt;p&gt;Time - probably yes; Space - probably no &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every single thing could be time dependent but there is no such thing on which time depends except considering space (black holes, bending of space time), Now we can argue that it can compare memories of present to its previous memories relatively and then call this relativeness time but even there the AGI is putting a time stamp when calling something previous. &#xA;Now lets consider how time could be dependent upon space. In this case we can see things getting compared to how much light travels like how the SI unit of time is defined, but that is a really complex observation for a discrete system and it has to go through a lot of time dependent phenomena to reach the level to make the observation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Space - We can reduce this question by saying that if it could learn the concept of 1-D it could learn about space itself. 1-D space can be compared to a number line, assuming the agi knows the concept of numbers it can compare the differences in location with a number line .And if agi knows the concept of time  it can learn space estimation by always assigning velocity to objects.&#xA;The difference between time and space is, time is always moving and there is nothing to compare to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Sorry for bad english, english is not my first language.&lt;/p&gt;&#xA;" OwnerUserId="19067" LastActivityDate="2018-10-16T17:59:59.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8470" PostTypeId="2" ParentId="8436" CreationDate="2018-10-16T18:01:56.787" Score="2" Body="&lt;p&gt;No, &quot;Google's view of AI&quot; (which in and of itself already is kind of vague, there are plenty of people within Google who have very different ideas) has not risen to the status of universal truth.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It sometimes seems that, when people don't understand something in a paper with the word Google anywhere in the authorship block, it is automatically assumed that the reader is wrong because the paper is of universal authority. They seem to jump to the assumption that their mind must adjust to whatever the paper states.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It does not seem like that to me at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, there has been research (from Google actually) indicating that there is some bias in paper review for example. When reviewers can see the author names and their affiliations, there is some measurable bias towards accepting them more often when they are from famous / well-known than when they are not famous. That includes Google, but also other companies (like Facebook, OpenAI) and universities and simply individuals who happen to be well-known even if their affiliations aren't. The same thing probably also happens occasionally even when double-blind reviewing is used, because... well, it's not difficult to guess where the anonymous authors may be coming from when the experiments described in their paper use a huge collection of TPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's nothing more than that though, just a bias. They get their papers accepted (and probably also cited) a bit more often. They definitely do not &lt;strong&gt;always&lt;/strong&gt; get their papers accepted. Plenty of papers from Google have been rejected too (which means the reviewers did not simply accept whatever was written without critical thinking), and that will continue to happen too. It also happens more often than not that I see/hear others being critical or confused by things they write. There is still plenty of critical thinking going around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the fact of the matter still is that they have managed to attract many world-class researchers (who were also already known as such before joining Google). It is no surprise that such researchers do have many good ideas, do write solid papers, and with Google they also do have the resources (in terms of computation, finances, development support from highly skilled software engineers, etc.) to execute more ideas, execute them more quickly, and more thoroughly. This naturally gives them an important voice, but they are not immune to criticism.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-10-16T18:01:56.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8472" PostTypeId="2" ParentId="8348" CreationDate="2018-10-16T19:14:20.127" Score="4" Body="&lt;p&gt;I'll start with the last question in your post:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was also wondering if there are any theoretical proofs/explanations about reward/Q-value clipping and which one being better. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I highly doubt there will be any such theoretical work. The problem is that these variants of clipping (clipping rewards and clipping &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values) fundamentally modify the task / the original objective. Once you clip either of those things, you fundamentally change what your agent is trying to optimize for from what the original goal was. I don't think it's ever going to be possible to get any rigorous, theoretical proofs about which one would be better in general. You'd likely have to start out with some very strong assumptions on the reward structure in the original task to have any hope of proving anything here, but such strong assumptions make you lose generality.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Intuitively... I think reward clipping feels &quot;safer&quot; to me more often than clipping &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values. Clipping &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values seems more aggressive, it could be viewed as some combination of clipping rewards (if you clip &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values to &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;, you're still at the very least also clipping all rewards to that range), but additionally also putting a constraint on how far in the future you're looking (in some sense). This whole argument is very handwavy though&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose, slightly less handwavy, you could say that reward clipping is definitely &quot;better&quot; (in the sense that you don't deviate as much from the original objective) in environments where rewards of similar magnitudes can be collected frequently. I struggle to really think of a situation where clipping &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values would be a clear favorite based on intuition. I wouldn't be surprised if clipping &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values may turn out to be better after empirical evaluation in some cases, but it's difficult to say where that would be. It will also very much depend on what range is chosen. Clipping rewards to a range of &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt; is very different from clipping &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values to the same range.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-10-16T19:14:20.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8475" PostTypeId="2" ParentId="6102" CreationDate="2018-10-17T01:32:11.473" Score="1" Body="&lt;p&gt;We can break down the problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, if you have two points on a plane and feed the coordinates of those points to a neural network (e.g., a vector &lt;span class=&quot;math-container&quot;&gt;$&amp;lt; x_0, y_0, x_1, y_1 &amp;gt;$&lt;/span&gt;) and and train it on a label thats the actual distance (e.g., &lt;span class=&quot;math-container&quot;&gt;$ \sqrt{(x_0 - y_0)^2 + (x_1-y_1)^2} $&lt;/span&gt;), it should be able to learn this relationship with arbitrarily close accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, if you have an image similar to what you describe, and feed that through a different neural network (e.g., a CNN), and as labels you used the the points of the two dots (once again &lt;span class=&quot;math-container&quot;&gt;$&amp;lt; x_0, y_0, x_1, y_1 &amp;gt;$&lt;/span&gt;), then it should be able to learn that relationship with arbitrarily close accuracy once again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there's no reason to do this in two separate neural network, so we can just combine the two end-to-end have a model that takes the image as input and the distance as output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This model would need to be trained on labeled data, however, so you'd either need to generate the data yourself or label images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you wanted it to learn the notion of closing a distance in a less supervised way, you'd need to use reinforcement learning. In this case, you'd have to setup an environment that incentivises the agent to reduce the distance. This could be as simple as gaining reward if an action reduces the distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to incentivise the agent using future reward. That is, it's reward doesn't just come from the results of the next immediet state, but there's also contributions from the next possible state, and the one after that, and so on. This is the idea behind Deep Q-Learning, and I implement a simple example (very similar to what you're describing) &lt;a href=&quot;https://github.com/nathanmargaglio/DQN/blob/master/Basic%20Env.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this notebook&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, now the question is: has this implementation done something other than randomly moving around until it follows a path to success?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, you talk about rewarding the agent when it lands on the goal. But in what I described, it gained reward by moving closer to the goal (either through the Q-Function or directly from the environment). It is able to do so by learning some abstract idea of distance (which can be illustrated in the supervised version).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a human learns this, it's for the same exact reason: the human is gaining a reward for moving in that direction through a sense of future rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd say that, given enough training and data, reinforcement learning could learn this concept with ease. As far as other rewards being present on the board (e.g., &quot;minimise the entropy of the board as well as try to get rewards&quot;), you need to think about what it is you're asking. Would you rather the agent minimize distance or maximize reward? Cause, in general, it can't do both. If you're looking for some balance between the two, then really you're just redefining the reward to also consider the distance. &lt;/p&gt;&#xA;" OwnerUserId="19080" LastActivityDate="2018-10-17T01:32:11.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8476" PostTypeId="1" AcceptedAnswerId="8477" CreationDate="2018-10-17T07:42:21.090" Score="8" ViewCount="3714" Body="&lt;p&gt;What is an agent in reinforcement learning (RL)? I think it is not the neural network behind. What does the agent in RL exactly do?&lt;/p&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="2444" LastEditDate="2018-11-19T18:21:35.957" LastActivityDate="2018-11-19T18:21:35.957" Title="What does the agent in reinforcement learning exactly do?" Tags="&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8477" PostTypeId="2" ParentId="8476" CreationDate="2018-10-17T08:10:26.750" Score="5" Body="&lt;p&gt;The agent in RL is the component that &lt;em&gt;makes the decision&lt;/em&gt; of what action to take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make that decision, the agent is allowed to use any observation from the environment, and any internal rules that it has. Those internal rules can be anything, but typically in RL, it expects the current &lt;em&gt;state&lt;/em&gt; to be provided by the environment, for that state to have the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;noreferrer&quot;&gt;Markov property&lt;/a&gt;, and then it processes that state using a policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; that decides what action to take. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, in RL we usually care about handling a reward signal (received from the environment) and optimising the agent towards maximising the expected reward in future. To do this, the agent will maintain some data which is influenced by the rewards it received in the past, and use that to construct a better policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting thing about the definition of an agent, is that the agent/environment boundary is usually considered to be very close to the abstract decision making unit. For instance, for a robot, the agent is typically not the whole robot, but the specific program running on the robot's CPU that makes the decision on the action. All the relays/motors and other parts of the physical body of the robot are parts of the environment in RL terms. Although often loose language is used here, as the distinction might not matter in most descriptions - we would say that &quot;the robot moves its arm to achieve the goal&quot; when in stricter RL terms we should say that &quot;the agent running on the robot CPU instructs the arm motors to move to achieve the goal&quot;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think it is not the Neural Net behind?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is correct, the agent is more than the neural network. One or more neural networks might be &lt;em&gt;part of&lt;/em&gt; an agent, and take the role of estimating the value of a state, or state/action pair, or even directly driving the policy function. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-17T11:18:53.910" LastActivityDate="2018-10-17T11:18:53.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8479" PostTypeId="2" ParentId="8467" CreationDate="2018-10-17T09:55:58.530" Score="0" Body="&lt;p&gt;The line of thought in this question is interesting.  It also addresses and challenges some trending creeds expressed in the media about artificial intelligence, making this an important topics to address in the AI StackExchange.  To properly address interesting detail in the question it is best to answer block by block.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is time/space estimation of algorithms required for creating an AGI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The term AGI (artificial general intelligence) is a symbol for a system of concepts that I've written about on two occasions in a Creative Commons works prior to learning about StackExchange.  I can briefly summarize those works by listing the primary three beliefs that are taken as axioms in the system.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There is a capability called General Intelligence that humans possess.&lt;/li&gt;&#xA;&lt;li&gt;Human intelligence would be limitless because it is of the general type were it not for biological limitations.&lt;/li&gt;&#xA;&lt;li&gt;A computer system can simulate or replicate General Intelligence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A corollary of these axioms is that a computer system can scale and be repaired, freeing the capacities of general intelligence to grow with a new and less DNA constrained bounds.  Screenwriter Jack Paglen astutely investigates one of the scenarios enabled by this corollary in &lt;em&gt;Transcendence&lt;/em&gt;, a movie released in 2014 starring Johnny Depp.  In it, protagonist Will Caster (played by actor Johnny Depp) is terminally ill and escapes death via computer upload and spreads, such as in Stephen King's short story &lt;em&gt;Lawnmower Man&lt;/em&gt; (1975).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jack Paglen, however, extends beyond the media hyped Singularity concept based on the above creed.  Will Caster downloads himself into both a body he built using nano-tech and an airborne nano-technical life form.  This is an epitome of the AGI system of concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biological limitations that are part of the details of the second axiom above are roughly correct.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The ratio of the cranial volume to the volume required by a neuron, limiting the number of neurons.&lt;/li&gt;&#xA;&lt;li&gt;Metabolism rates limit the number of pulses a neuron can handle and the speed by which pulses can travel through axons.&lt;/li&gt;&#xA;&lt;li&gt;Mortality limits both the duration of learning and the time available for conceptual organization of what has been learned.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of this sounds quite logical until the root of the AGI system is held up to scientific scrutiny.  The most central of the three primary axioms on which the entire system is based, Axiom 1, is a belief.  Those who believe it often offer as evidence the array of human achievements.  When asked to list them, the list is always skews historical events in science, geopolitics, and culture toward the positive.  All the gross and recurrent failures are excluded from the list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concepts that form the AGI system neglect the addressing of three realities born out of any scientific analysis of the full data set without which an accurate picture of human intelligence is grossly incomplete.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;What imminent psychologist Carl Jung called, &quot;A wide array of neuroses,&quot; is evident in all expressions of human intelligence.&lt;/li&gt;&#xA;&lt;li&gt;The set of circumstance to which humans adapt through intellect is narrow and marginally proven within the biosphere. &lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;The set of circumstance to which humans adapt through intellect is minuscule and largly unsuccessful outside the biosphere.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Analyzing further, we find that the reason intelligence has no agreed upon mathematical definition is that there is no sentence or paragraph that defines intelligence which would stands up to even the weakest scientific scrutiny.  The lack of scientific rigor applied to the term intelligence is so dilute that any one schooled in the scientific method would have to admit that there is no real evidence that intelligence exists at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AGI is &lt;em&gt;credo superbiae&lt;/em&gt;, Latin for creed of pride.  One of the primary characteristics of the human brain is its overestimation of itself.  Anyone who has managed many projects, gathering estimates from experts, creating a Gantt chart for the project and then had to adjust dates repeatedly because the estimates are always optimistic and never conservative has seen the ample data supporting this fact.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given infinite resources/time one could create AGIs by writing code to simulate infinite worlds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By doing that in some of the worlds AGIs would be created.  Detecting them would be another issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detecting intelligence in simulations of infinite worlds would be an issue for three reasons, one of which the question author indicated below.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Research resources and time are not infinite.&lt;/li&gt;&#xA;&lt;li&gt;Although Turing's Imitation Game could be offered as a test for intelligence, given the scrutiny above, that would not be a test for general intelligence because one would need more than infinite resources and time to run through an infinite number of test scenarios that could arise in an infinite number of simulations of infinite worlds.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The creation of infinite worlds only constructs the environment in which an entity's intellectual capabilities can be exercised. The creation of an environment does not automatically imply that self-improving life will originate within it or that the sequence of improvements if they were to be driven forward by some form of genetic evolution would lead to the passing of any Imitation Game we could create that aligns with the human view of what intelligence is.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since we don't have infinite resources the most probable way to create an AGI is to write some bootstrapping code which would reduce the resources/time to reasonable values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is currently no bootstrapping strategy that expands resources infinitely in at least three independent spaces which require at least six dimensions of infinite range, based on the question description.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Infinite world in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;An infinite number of the infinite worlds in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{I}^1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An infinite number of scenarios required to prove the generality of the intelligence in all possible scenarios in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{I}^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In that AGI code (that would make it reasonable to create with finite resources/time) is it required to have a part that deals with time/space estimation of algorithms?&#xA;  Or should that be outside of the code and be something the AGI discovers by itself after it starts running?&#xA;  ... by time/space I mean time/space complexity analysis for algorithms, see: Measures of resource usage and Analysis of algorithms&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Let's assume that, through some remarkable computing technique, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{T}$&lt;/span&gt;, an infinite number of test scenarios can be created to test automatons that remarkably arise in an infinite number of infinite worlds.  Under those conditions, it would not be necessary to have a part that deals with time/space estimation of algorithms.  The entity, whatever its intellectual abilities or how those would be measure would not need to check its bounds because its environment is already rendered boundless through the remarkable technique &lt;span class=&quot;math-container&quot;&gt;$\mathbb{T}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think the way I formulated the question might lead people to think that the time/space estimation can only apply to some class of actions called algorithms.  To clarify my mistake, I mean the estimation to apply to any action plan.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Imagine you are an AGI and you have to make a choice between different set[s] of actions to pursue your goals.  If you had 2 goals and one of them used less space and less time then you would always pick it over the other algorithm.  So time/space estimation is very useful since intelligence is about efficiency.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The assumption that intelligence is about efficiency may be an assumption, but it is nonetheless an interesting idea, since we know that adaptation in the physical world is a primarily energy acquisition and conservation process.  That the DNA process of adaptation was augmented within organism lifespan by neural learning does not except this primary driving force.  One could argue rationally that intelligence is indicated when a human being states, &quot;Why should I go through that trouble? What can I expect as a return on that investment of time, energy, and other resource and why should I expect it?&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There is at least 1 exception though, imagine in the example before that the goal of the AGI is to pick the set of actions that leads to the most expensive time/space set of actions (or any non-minimal time/space cost) then obviously because of the goal constraint you would pick the most time/space expensive set of actions.  In most other cases though, you would just pick the most time/space efficient algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One could argue rationally that pursuit of inefficiency is unintelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this later thinking is solid, but the earlier approach to building the AIG appears to have four caveats.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The absence of bootstrapping that runs six dimensions if infinite iteration on finite computing resource in a finite time&lt;/li&gt;&#xA;&lt;li&gt;The lack of a scientifically valid definition of intelligence or even a proof that intelligence as humans tend to think of it truly exists&lt;/li&gt;&#xA;&lt;li&gt;The lack of strong evidence that life will reliably arise in worlds created&lt;/li&gt;&#xA;&lt;li&gt;The lack of strong evidence that life will reliably develop the equivalent of a nervous system or any secondary system of evolution that operates within a living entity's operational duration.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem is in the infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one acknowledges that working definitions of intelligence will only describe a relative set of one or more intelligence features&lt;sup&gt;2&lt;/sup&gt; and that none of them can be infinite or unbounded, then the rest of this line of thinking is plausible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] The adaptive capabilities of humans (of which capabilities commonly deemed as features of intelligence are only a part) have been tested for only 195,000 to 460,000 years years compared to the notable adaptability of bacteria over at least 4,200,000,000 years and possibly all the way back to the formation of Earth.  Vladimir Vernadski, who made famous the term Biosphere from the title of his book, questioned whether life might have came Earth, pointing out that life is a geological phenomena and the elements of life are throughout the galaxy.  195,000 years is the back-dating of the oldest clear archaeological evidence of modern humans.  460,000 years is the back-dating of the Irhoud skulls which may have exhibited modern intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] I've suggested in &lt;a href=&quot;https://ai.stackexchange.com/questions/7337/is-the-singularity-concept-mathematically-flawed&quot;&gt;other Q&amp;amp;A here&lt;/a&gt; that there must be at least 22 dimensions to intelligent behavior for genetic and statistical reasons.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-17T09:55:58.530" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8480" PostTypeId="1" CreationDate="2018-10-17T10:54:29.347" Score="4" ViewCount="758" Body="&lt;p&gt;I have a collection of scanned documents (which come from newspapers, books, and magazines) with complex alignments for the text, i.e. the text could be at any angle w.r.t. the page. I can do a lot of processing for different features extraction. However, I want to know some robust methods that do not need many features.&lt;/p&gt;&#xA;&lt;p&gt;Can machine learning be helpful for this purpose? How could I use machine learning to detect text and non-text regions in these scanned documents?&lt;/p&gt;&#xA;" OwnerUserId="18459" LastEditorUserId="2444" LastEditDate="2020-07-25T12:15:14.660" LastActivityDate="2020-07-25T12:15:14.660" Title="How could I use machine learning to detect text and non-text regions in scanned documents?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;pattern-recognition&gt;&lt;optical-character-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8482" PostTypeId="1" AcceptedAnswerId="8486" CreationDate="2018-10-17T12:00:49.070" Score="6" ViewCount="202" Body="&lt;p&gt;After doing some exercices on Q-learning for maze solving, I wondered : my q-learning algorithms solve only ONE maze. The AI doesn't learn how to solve mazes, so how can I achieve it ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance learn &quot;follow one wall until the end of the maze&quot; instead of &quot;turn left left right...&quot; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess it is the same approach for self driving cars ?&lt;/p&gt;&#xA;" OwnerUserId="19094" LastEditorUserId="10135" LastEditDate="2018-10-18T10:44:42.947" LastActivityDate="2018-10-18T10:44:42.947" Title="Q-Learning the generic maze solution" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;q-learning&gt;&lt;challenges&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8483" PostTypeId="2" ParentId="8480" CreationDate="2018-10-17T12:59:27.653" Score="2" Body="&lt;p&gt;TextDetector, Tesseract and other open source packages implement text detection (object detection for text). There's also a pretrained Tensorflow model that does text detection. A text detector will give you the bounding boxes in your image for any text that it recognizes. In the case of Tesseract, it will also output the text (OCR is built in). So you can read the code in these packages to get ideas for your own machine learning pipeline. Basically you need both a regressor (for the bounding boxes) and a classifier (to detect whether the box contains text or not).&lt;/p&gt;&#xA;" OwnerUserId="19095" LastEditorUserId="19095" LastEditDate="2018-10-19T01:05:13.263" LastActivityDate="2018-10-19T01:05:13.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8484" PostTypeId="2" ParentId="1963" CreationDate="2018-10-17T13:23:08.127" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://duck.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;DuckDuckGo&lt;/a&gt; learns answers to technical questions from StackExchange. Type a technical question like &lt;a href=&quot;https://duckduckgo.com/?q=ongoing%20projects%20use%20stackexchange&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;ongoing projects use stackexchange&quot; into DuckDuckGo&lt;/a&gt; and it will provide a highlighted summary of the answer on the right-hand side. &#xA;And the duck has an open API for many (100s) more question answering data sources. Or you can go directly to the &lt;a href=&quot;https://api.stackexchange.com/&quot;&gt;stackexchange api&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Projects can use the data from the SE open API as long as they comply with their &lt;a href=&quot;https://stackoverflow.com/legal/api-terms-of-use&quot;&gt;TOU&lt;/a&gt;. Basically just make sure your users can tell that the data came from Stack Exchange. The copyright license may also limit your ability to alter the contents of the text, with say a learned abstractive summarizer. Perhaps that is why the Duck.com just highlights keywords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data rights law is in flux, especially when it comes to the data you submitted to a site and the machine learning models derived from that data. New European data and privacy rules empower you to download or delete all data you submit to a site like stack exchange.&lt;/p&gt;&#xA;" OwnerUserId="19095" LastEditorUserId="19095" LastEditDate="2019-05-10T22:18:54.537" LastActivityDate="2019-05-10T22:18:54.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8486" PostTypeId="2" ParentId="8482" CreationDate="2018-10-17T19:30:13.520" Score="3" Body="&lt;p&gt;I'm going to assume here that you're using the standard, basic, simple variant of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning that can be described as &lt;strong&gt;tabular&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, where all of your state-action pairs for which you're learning &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values are represented in a tabular fashion. For example, if you have 4 actions, your &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values are likely represented by 4 matrices (corresponding to the 4 actions), where every matrix has the same dimensionality as your maze &lt;em&gt;(I'm assuming that your maze is a grid of discrete cells here)&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With such an approach, you are learning &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values separately for every single individual state (+ action). Such learned values will always only be valid for &lt;strong&gt;one particular maze&lt;/strong&gt; (the one you have been training in), as you seem to have already noticed. This is a direct consequence of the fact that you're learning individual values for specific state-action pairs. The things you are learning (&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values) can therefore not directly be transferred to a different maze; &lt;strong&gt;those particular states from the first maze do not even exist in the second maze!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better results may be achievable with different state representations. For example, instead of representing states by their coordinates in a grid (as you would likely do with a tabular approach), you'd want to describe states in a more general way. For example, a state could be described by features such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a wall right in front of me?&lt;/li&gt;&#xA;&lt;li&gt;Is there a wall immediately to my right?&lt;/li&gt;&#xA;&lt;li&gt;Is there a wall immediately to my left?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An alternative that also may actually be able to better generalize to some extent could be pixel-based inputs if you have images (a top-down image or even a first-person-view).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When states are represented by such features, you can no longer use the tabular RL algorithms that you are likely familiar with though. You'd have to use &lt;strong&gt;function approximation&lt;/strong&gt; instead. With good state-representations, those techniques may have a chance of generalizing. You'd probably want to make sure to actually also use a variety of different mazes during the training process though, otherwise they'd likely still overfit to only a single maze used in training.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-10-17T19:30:13.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8487" PostTypeId="1" CreationDate="2018-10-17T20:16:58.747" Score="1" ViewCount="79" Body="&lt;p&gt;I am reading article &lt;a href=&quot;https://allenai.org/paper-appendix/emnlp2017-wt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://allenai.org/paper-appendix/emnlp2017-wt/&lt;/a&gt; &lt;a href=&quot;http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf&lt;/a&gt; about training neural network and the loss function is mentioned on page 6 chapter 3.4 - this loss function &lt;code&gt;O(theta)&lt;/code&gt; is expressed as marginal loglikelihood objective function. I simply does not understand this. The neural network generates logical expression (query) from some question in natural language. The network is trained using question-answer pairs. One could expect that simple sum of correct-1/incorrect=0 result could be good loss function. But there is strange expression that involves &lt;code&gt;P(l|qi, Ti; theta)&lt;/code&gt; that is not mentioned in the article. What is meant by this &lt;code&gt;P&lt;/code&gt; function? As I understand, then many logical forms &lt;code&gt;l&lt;/code&gt; are generated externally for some question &lt;code&gt;qi&lt;/code&gt;. But further I can not understand this. The mentioned article largely builds on other article &lt;a href=&quot;http://www.aclweb.org/anthology/P16-1003&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.aclweb.org/anthology/P16-1003&lt;/a&gt; from which it borrows some terms and ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is said that &lt;code&gt;l&lt;/code&gt; is treated as latent variable and &lt;code&gt;P&lt;/code&gt; seems to be some kind of probability. Of course, we should assign the greated probability to the right logical form &lt;code&gt;l&lt;/code&gt;, but where can I find this assignment. Does training/supervision data should contain this probability function for training/supervision data?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2018-10-17T20:26:36.093" LastActivityDate="2021-02-05T03:18:41.143" Title="How to understand marginal loglikelihood objective function as loss function (explanation of an article)?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8488" PostTypeId="2" ParentId="8480" CreationDate="2018-10-17T20:20:18.847" Score="1" Body="&lt;p&gt;Since the document is scanned, it will not be in an open document format so no associated API can be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evaluate TextBridge Pro, FreeOCR, and other alternatives that purport page layout detection.  If any of them work, drive them programmatically (preferably headless) to read the scanned document, detect page layout and OCR the text, export to a document with an open format, and then use the API&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this approach, the object recognition AI is in the product and development time and resources are saved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do a 2D FFT windowing through the page in both directions.  See the cosine, trapazoidaly, Hamming, and Hanning windows and apply them in horizontal and vertical directions.  Use Approach 1, assuming those products work with the scanned documents to label the examples, and then train a DCNN (deep convolutional NN) to recognize from the 2D FFT output spectra where the pictures are.  By interpolation, close to a perfect crop of the images and the text regions can be obtained with some hyper-parameters on the model obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 3&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is just Approach 2 but preparing the labeled example data set by hand, which may be necessary because the existing software products may not handle the images being laid out at angles other than 0, 90, 180, or 270 degrees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 4&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create an architecture that is based on feature extraction, and use font rendering libraries to build the back half of an auto-encoder, allowing portions of image that do not auto-encode to be preserved as an x-y coordinate pair, which will allow the images to jump over the pictures if the convergence is set up correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final Note&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can offload some processing to a learning process, so that the actual document process runs faster, but sometimes the preparation of the example data set and the learning consumes more resources.  That's why those who assess which approach will cost less and can recommend the best approach with some reliability are high paid.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-17T20:20:18.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8489" PostTypeId="2" ParentId="8458" CreationDate="2018-10-18T05:26:59.043" Score="0" Body="&lt;p&gt;First step would be getting the object out of the scene. This bit is not trivial in your case, however, there are many methods to choose from. I suggest reading about watershed threshold algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second part is easier. Once you have a single segmented object at hand, perform noise removal. Next step is to extract the contours. Find the center of gravity, transform coordinates to polar and represent these contours as a function which has the x axis as degrees and y axis as distance from the center. Take the Fourier transform of this function. If the shape is symmetrical, there will be few non-zero entries, and a large spike in the spectrum.&lt;/p&gt;&#xA;" OwnerUserId="210" LastActivityDate="2018-10-18T05:26:59.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8490" PostTypeId="1" CreationDate="2018-10-18T05:34:51.540" Score="3" ViewCount="48" Body="&lt;p&gt;In the Abstract section of the paper &lt;a href=&quot;https://arxiv.org/pdf/1312.4400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Network In Network&lt;/a&gt;, what does the authors actually mean to say?&lt;/p&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="1671" LastEditDate="2018-10-21T22:58:19.273" LastActivityDate="2018-10-21T22:58:19.273" Title="What is meant by &quot;model discriminability for local patches within the receptive field&quot;?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8491" PostTypeId="1" CreationDate="2018-10-18T07:30:20.910" Score="6" ViewCount="555" Body="&lt;p&gt;While working through some example from Github I've found this network (it's for FashionMNIST but it doesn't really matter). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pytorch forward method (my query in upper case comments with regards to applying Softmax on top of Relu?):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def forward(self, x):&#xA;    # two conv/relu + pool layers&#xA;    x = self.pool(F.relu(self.conv1(x)))&#xA;    x = self.pool(F.relu(self.conv2(x)))&#xA;&#xA;    # prep for linear layer&#xA;    # flatten the inputs into a vector&#xA;    x = x.view(x.size(0), -1)&#xA;&#xA;    # DOES IT MAKE SENSE TO APPLY RELU HERE&#xA;    **x = F.relu(self.fc1(x))&#xA;&#xA;    # AND THEN Softmax on top of it ?&#xA;    x = F.log_softmax(x, dim=1)**&#xA;&#xA;    # final output&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19116" LastEditorUserId="1847" LastEditDate="2018-10-18T12:16:23.473" LastActivityDate="2018-10-18T12:56:46.763" Title="Does it make sense to apply softmax on top of relu?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8493" PostTypeId="1" AcceptedAnswerId="8494" CreationDate="2018-10-18T10:38:39.577" Score="2" ViewCount="132" Body="&lt;p&gt;I am new to RL and I am trying to work through the book Reinforcement Learning: An Introduction I (Sutton &amp;amp; Barto, 2018). In chapter 3 on Finite Markov Decision Processes, the authors write the expected reward as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a) = \mathbb{E}\left[R_t|S_{t-1}=s,A_{t-1}=a\right]=\sum_{r\in \mathcal{R}}r\sum_{s'\in \mathcal{S}}p(s',r|s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if the authors mean&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a) = \mathbb{E}\left[R_t|S_{t-1}=s,A_{t-1}=a\right]=\sum_{r\in \mathcal{R}}\left[r\sum_{s'\in \mathcal{S}}p(s',r|s,a)\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a) = \mathbb{E}\left[R_t|S_{t-1}=s,A_{t-1}=a\right]=\left[\sum_{r\in \mathcal{R}}r\right]\cdot\left[\sum_{s'\in \mathcal{S}}p(s',r|s,a)\right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the authors mean the first, is there any reason why it is not written like the following?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a) = \mathbb{E}\left[R_t|S_{t-1}=s,A_{t-1}=a\right]=\sum_{r\in \mathcal{R}}\sum_{s'\in \mathcal{S}}\left[r\,p(s',r|s,a)\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="19123" LastEditorUserId="2444" LastEditDate="2019-02-16T02:29:38.477" LastActivityDate="2019-02-16T02:29:38.477" Title="Understanding the notation in the definition of the expected reward" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8494" PostTypeId="2" ParentId="8493" CreationDate="2018-10-18T11:34:05.143" Score="2" Body="&lt;p&gt;Your first option is correct:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a) = \mathbb{E}\left[R_t|S_{t-1}=s,A_{t-1}=a\right]=\sum_{r\in \mathcal{R}}\left[r\sum_{s'\in \mathcal{S}}p(s',r|s,a)\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's partly a matter of taste, but I prefer not moving the &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; into the double sum, because its value does not change in the &quot;inner loop&quot;. There is a small amount of intuition to be had that way around, especially when it comes to implementation (it is one multiplication after the sum, as opposed to many within the sum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of sums containing sums in Sutton &amp;amp; Barto, and they mainly follow the convention of not using any parentheses or brackets to show the one sum containing the other explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the formulae help link to other treatments of RL, which work with the expected reward functions &lt;span class=&quot;math-container&quot;&gt;$r(s,a)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt;, or reward matrices &lt;span class=&quot;math-container&quot;&gt;$R_s^a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R_{ss'}^a$&lt;/span&gt; such as the first edition of Sutton &amp;amp; Barto's book. The second edition of the book uses &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; almost everywhere though, and you won't see &lt;span class=&quot;math-container&quot;&gt;$r(s,a)$&lt;/span&gt; mentioned much again. So it's not worth getting too concerned about how it is presented and what the author might be saying with the presentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally you don't need to know the &lt;em&gt;distribution&lt;/em&gt; of reward, just its expectation (and how that depends on &lt;span class=&quot;math-container&quot;&gt;$s, a, s'$&lt;/span&gt;), in order to derive and explain most of the results in RL. So using &lt;span class=&quot;math-container&quot;&gt;$r(s,a)$&lt;/span&gt; and similar functions is fine, in places like the Bellman equations. However, the use of &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; is general without needing to bring in more functions describing the MDP.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-10-18T11:34:05.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8495" PostTypeId="2" ParentId="8491" CreationDate="2018-10-18T12:37:52.623" Score="5" Body="&lt;p&gt;&lt;strong&gt;Does it make sense?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, yes it is interpretable, back propagation will work, and the NN can be optimised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using ReLU, the default network has a minimum &lt;em&gt;logit&lt;/em&gt; of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for the softmax input, which means at least initially that there will be higher minimum probabilities associated with all classes (compared to allowing negative logits which would happen randomly with usual weight initialisation). The network will need to learn to produce higher &lt;em&gt;logit&lt;/em&gt; values for correct answers, because it has no ability to produce lower &lt;em&gt;logit&lt;/em&gt; values for incorrect answers. This is like training a network to produce the highest regression value on one output, whilst clipping all values to be 0 or above, so it does not have the option of making one output e.g. &lt;span class=&quot;math-container&quot;&gt;$-1.0$&lt;/span&gt; and the rest &lt;span class=&quot;math-container&quot;&gt;$-100.0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can probably be thought of as a type of regularisation, as it puts constraints on activation values that will work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it needed?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is less clear. You can try training with and without the line, and using cross-validation or a test set to see if there is a significant difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the network has been designed well, then I'd expect to see a slight improvement with the added ReLU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is a mistake, then I'd expect to see no difference, or better performance without the ReLU.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-18T12:56:46.763" LastActivityDate="2018-10-18T12:56:46.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8496" PostTypeId="1" AcceptedAnswerId="8525" CreationDate="2018-10-18T13:36:09.240" Score="3" ViewCount="1029" Body="&lt;p&gt;In the book &lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt; (Sutton and Barto, 2018). The authors ask&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Exercise 3.2:&lt;/em&gt;  Is the MDP framework adequate to usefully represent all goal-directed learning tasks? Can you think of any clear&#xA;  exceptions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I thought maybe a card game would be an example if the state does not contain any pieces of information on previously played cards. But that would mean that the chosen state leads to a system that is not fully observable. Hence, if I track all cards and append it to the state (state vector with changing dimension) the problem should have the Markov Property (no information on the past states is needed). This would not be possible if the state is postulated as invariant in MDP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the previous procedure is allowed, then it seems to me that there are no examples where the MDP is not appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be glad if someone could say if my reasoning is right or wrong. What would be an appropriate answer to this question?&lt;/p&gt;&#xA;" OwnerUserId="19123" LastEditorUserId="2444" LastEditDate="2020-03-13T22:49:59.523" LastActivityDate="2020-08-08T06:17:23.123" Title="When is the Markov decision process not adequate for goal-directed learning tasks?" Tags="&lt;reinforcement-learning&gt;&lt;applications&gt;&lt;markov-decision-process&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8497" PostTypeId="1" CreationDate="2018-10-18T15:33:57.757" Score="1" ViewCount="26" Body="&lt;p&gt;We have multiple computers and the ability to ssh between them. What are options using either Java, C/C++, JavaScript, or Python to distribute our learning tasks? We will be using DCNN, DQN, and LSTM in different combinations.&lt;/p&gt;&#xA;" OwnerUserId="19127" LastActivityDate="2018-10-18T15:33:57.757" Title="If there are several computers on a subnet, can training time be reduced by distributing the work across them?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;long-short-term-memory&gt;&lt;java&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8499" PostTypeId="1" CreationDate="2018-10-18T18:54:44.450" Score="1" ViewCount="244" Body="&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&lt;p&gt;Exhaustive search is a method in AI planning to find a solution for so-called Constraint Satisfaction Problems. (CSP). Those are problems that have some conditions to fulfill and the solver is trying out all the alternatives. An example CSP problem is the 8-queens problem which has geometrical constraints. The standard method in finding a solution for the 8-queens problem is a backtracking solver. That is an algorithm that generates a tree for the state space to search inside the graph.&lt;/p&gt;&#xA;&lt;p&gt;Apart from practical applications of backtracking search, there are some logic-oriented discussions available which are asking on a formal level which kind of problems have a solution and which not. For example to find a solution for the 8-queen problem many millions of iterations of the algorithm are needed. The question is now: which problems are too complex to find a solution. The second problem is, that sometimes the problem itself has no solution, even the complete state space was searched fully.&lt;/p&gt;&#xA;&lt;p&gt;Let us take an example. At first, we construct a problem in which the constraints are so strict that even a backtracking search won't find a solution. One example would be to prove that “1+1=3” another example would be to find a chess sequence if the game is lost or it is also funny to think about how to arrange nine! queen on a chess table so that they don't hurt.&lt;/p&gt;&#xA;&lt;p&gt;Is there any literature available which is describing Constraint Satisfaction Problems on a theoretical basis in which the constraints of the problem are too strict?&lt;/p&gt;&#xA;&lt;h2&gt;Original posting&lt;/h2&gt;&#xA;&lt;p&gt;Just wondering - like with an 8-queens problem. If we change it to a 9-queens problem and do an exhaustive search, we will see that there is no solution. Is there a problem in which the search fails to show that a solution does not exist?&lt;/p&gt;&#xA;" OwnerUserId="19137" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-29T13:47:59.030" Title="Any problems/games/puzzles in which exhaustive search cannot show that a solution does not exist?" Tags="&lt;search&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8504" PostTypeId="2" ParentId="2870" CreationDate="2018-10-18T23:09:45.480" Score="1" Body="&lt;p&gt;(I'd leave this in the comments but sadly I can't.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a great paper (leveraging formal game theory) in which &quot;players&quot; make strategic choices based on each event (&quot;move&quot;) and an algorithm calculates the best strategy for all players based on each player's abilities throughout the multi-stage &quot;game&quot;. So instead of learning from winning and predicting what the next perfect move the player thinks (based off of learning) will lead to a &quot;win&quot;, the player calculates an optimal strategy and moves to win the game itself over multiple moves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://users.ece.cmu.edu/~youzhib/paper/bao2017csf.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://users.ece.cmu.edu/~youzhib/paper/bao2017csf.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: This paper deals with cyber war games so is heavily tilted towards that domain, but if you take away the game theoretics and concept of strategy over a multi-stage game like ti-tac-toe, you should be able to improve your overall outcomes. That being said, how the model would accommodate a random &quot;player&quot; with random moves would be interesting to see.&lt;/p&gt;&#xA;" OwnerUserId="15337" LastActivityDate="2018-10-18T23:09:45.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8506" PostTypeId="2" ParentId="4764" CreationDate="2018-10-19T02:45:03.577" Score="1" Body="&lt;p&gt;They can definitely be made cheaper, using a combination of techniques. But I doubt if they will be free in the near term. Once the passenger is inside a car, they are bound to be in the car for the duration of the travel. Any company who finds an efficient way to sell things or make them their product during this time creates a business opportunity. Some scenarios that companies will employ to reduce cost of travel are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Advertisements&lt;/li&gt;&#xA;&lt;li&gt;Sale of items on board (snacks, soft drinks, liquor etc)&lt;/li&gt;&#xA;&lt;li&gt;Usage based discount (The ride is free if you purchase $100 on amazon when you are in the car)&lt;/li&gt;&#xA;&lt;li&gt;Discount for services by passenger (Give us feedback every minute of the car operation, we will discount 50% of cost)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once self driving is popular, there will be a plethora of companies trying to exploit the passenger; the passenger is bound to be inside the car for the duration.&lt;/p&gt;&#xA;" OwnerUserId="6173" LastActivityDate="2018-10-19T02:45:03.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8507" PostTypeId="2" ParentId="8414" CreationDate="2018-10-19T05:33:16.923" Score="1" Body="&lt;p&gt;I identified the origin of this overfitting..&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Origins&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I tried a lot of models, putting more and more dropout, simplifying as much as I could.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter what I did, after a few epoch of good learning, invariably my loss function was going up. I tried simpler and simpler models, always the same overfitting behavior.&#xA;What bugged me at that moment is that no matter what kind of model I used, how deep or how complex, always the accuracy was fine, stabilized at some nice level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I tried the simplest model I could imagine : Input =&gt; Dense with 3 hidden units =&gt;  Output. Finally I got random results, with a 33% accuracy !&#xA;From here, I guilt again my network, layer by layer, to see which one was causing the overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it was the &lt;strong&gt;Embedding layer&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even with a simple network like Input =&gt; Embeddings =&gt; Dense with 3 hidden units =&gt;  Output, the model was overfitting.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How to solve it&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In Keras, simply instantiate the Embeddings layer with &lt;code&gt;trainable=False&lt;/code&gt;. After doing this, no more overfit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, this is quite counter-intuitive : I want my embeddings to evolve with the data I show to the network. But look like I can't...&lt;/p&gt;&#xA;" OwnerUserId="18852" LastActivityDate="2018-10-19T05:33:16.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8508" PostTypeId="2" ParentId="8499" CreationDate="2018-10-19T06:17:36.717" Score="1" Body="&lt;p&gt;&lt;em&gt;This should be a comment, but I don't have enough reputation to comment. I will remove this answer if question is updated&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question is not really clear. As I understand it, the definition itself of the exhaustive search show that it's always possible to determine if a solution is valid or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exhaustive search is defined as :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For a given potential solution, can determine if this solution resolve the problem.&lt;/li&gt;&#xA;&lt;li&gt;Test every possible potential solution&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From this, there is no problem where every potential solution is tested, but the search cannot show that a solution exist : either it exist and the search found the candidate, or it does not exist because all possible candidates were tested.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Maybe what you meant when asking your question is : &quot;Any problems in which exhaustive search cannot be applied ?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the answer is yes, there is plenty of problems where the search space is way to big to be integrally searched : for example the Rubik's cube 3*3*3 has 43 252 003 274 489 856 000 combinations (&lt;a href=&quot;https://www.therubikzone.com/number-of-combinations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My answer need more source, specifically about the exhaustive search definition. I would be happy to add it if you could share :)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="18852" LastActivityDate="2018-10-19T06:17:36.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8509" PostTypeId="1" CreationDate="2018-10-19T07:15:05.473" Score="4" ViewCount="975" Body="&lt;p&gt;I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons each, each layer with a ReLU layer as the activation function layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using this network for image classification - kinda. Basically it takes inputs as raw grayscale pixel values of the input image and guesses an output. I used stochastic gradient descent for the training of the model and a learning rate of 0.01. The input image is a row or column of OMR &quot;bubbles&quot; and the network has to guess which of the &quot;bubble&quot; is marked i.e filled and show the index of that bubble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is because it's very hard for the network to recognize the single filled bubble among many.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example image of OMR sections:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/F1cql.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F1cql.png&quot; alt=&quot;OMR bubbles example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using image-preprocessing The network is given a single row or column of the above image to evaluate the marked one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of a preprocessed image which the network sees:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3r60U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3r60U.png&quot; alt=&quot;OMR bubble column - what the network sees&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of a marked input:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LF0AO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LF0AO.png&quot; alt=&quot;Marked OMR Column&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to use Convolutional networks but I'm not able to get them working with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Basically, my question is that what type of neural network and network architecture should I use for this kind of task?. An example of such a network with code would be greatly appreciated.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried many preprocessing techniques such as background subtraction using the AbsDiff function in EmguCv and also using the MOG2 Algorithm and I've also tried threshold binary function but there still remains enough noise in the images which makes it difficult for the neural net to learn. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this problem is not specific to using neural nets for OMR but for others too. It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again, it perspective transforms it to match exactly to the template&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm able to achieve this much - and then find their difference or do some kind of preprocessing so that a neural net could learn from it. If this is not quite possible, then is there a type of neural network out there which could detect very small features from an image and learn from it.  I have tried Convolutional Neural Network but that also isn't working very well or I'm not applying them efficiently.&lt;/p&gt;&#xA;" OwnerUserId="19144" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:26.997" LastActivityDate="2020-05-29T13:48:26.997" Title="Neural Network for Optical Mark Recognition?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="3" CommentCount="8" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8510" PostTypeId="1" CreationDate="2018-10-19T07:59:55.483" Score="1" ViewCount="23" Body="&lt;p&gt;while dealing with image data at very large scale, there are different sources where data is coming from. Often, we do not have any control over quality of labels/ annotations. I already do use sampling quality checks method to manually check the quality of annotations but as the volume of data has increased,even sampling QC become an inefficient job. Are there other methods to automate / simplify this task for data at large scale?&lt;/p&gt;&#xA;" OwnerUserId="17980" LastEditorUserId="17980" LastEditDate="2018-10-19T09:47:54.260" LastActivityDate="2018-10-19T09:47:54.260" Title="input annotations quality check for large scale image data" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8511" PostTypeId="1" AcceptedAnswerId="8513" CreationDate="2018-10-19T08:04:29.920" Score="2" ViewCount="96" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Dropout_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dropout is a regularization technique for reducing overfitting in neural networks&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My neural network is simple enough and does not overfit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the addition of dropout, in a non-overfitting neural network, increase accuracy? Even if I increase the complexity of the neural network?&lt;/p&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="2444" LastEditDate="2020-03-10T20:27:51.027" LastActivityDate="2020-03-10T20:28:49.880" Title="Can the addition of dropout in a non-overfitting neural network increase accuracy?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;overfitting&gt;&lt;dropout&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8512" PostTypeId="2" ParentId="8509" CreationDate="2018-10-19T08:31:48.033" Score="0" Body="&lt;p&gt;I'm not familiar with ConvNetSharp library, and the tag &lt;code&gt;convolutional-neural-networks&lt;/code&gt; is a bit confusing me, but from :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons, each with a ReLU layer as the activation function layer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume you are building just a densely connected neural network. Correct me if I'm wrong.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The type of neural network you need is &lt;strong&gt;Convolutional Neural Network&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For image recognition (which is your case), convolutional network are almost always the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is plenty of type of CNN, just pick one that seems appropriate and try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, your task seems quite simple, you won't need really deep / complex architecture.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not aware of a model that could do what you are asking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what you are asking is not really in the 'neural network' mindset. The goal of building a neural network is that you don't specify anything. The neural network will learn and find the features for you. So you just have to feed him a lot of data, and he will be able to recognize your pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at this visualization of CNN filters :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SExae.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SExae.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, no one gave the neural network the template of a nose or the template of an eye or the template of a face. The CNN learned it over a lot of image.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastActivityDate="2018-10-19T08:31:48.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8513" PostTypeId="2" ParentId="8511" CreationDate="2018-10-19T08:36:34.967" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the addition of dropout, in a non-overfitting neural network, increase accuracy?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, maybe.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Even if I increase the complexity of the neural network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, maybe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Bootstrap_aggregating&quot; rel=&quot;nofollow noreferrer&quot;&gt;bagging&lt;/a&gt;. Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-03-10T20:28:49.880" LastActivityDate="2020-03-10T20:28:49.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8514" PostTypeId="1" AcceptedAnswerId="8520" CreationDate="2018-10-19T09:34:21.753" Score="3" ViewCount="113" Body="&lt;p&gt;I am referring to eq. 3.6 (page 49) based on &lt;a href=&quot;https://drive.google.com/file/d/1opPSz5AZ_kVa1uWOdOiveNiBFiEOHjkG/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton's online book&lt;/a&gt;  and can be found in an image below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VTPdz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VTPdz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I could not make sense of the final derivation of the equation &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt;. My question is actually how do we come to that final derivation?&lt;/p&gt;&#xA;&lt;p&gt;Surprisingly, the denominator of &lt;span class=&quot;math-container&quot;&gt;$p(s'|s, a)$&lt;/span&gt; can literally be replaced by &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; as eq. 3.4 suggests, then it will end up with &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&amp;quot; term only due to cancellation of numerator &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; and denominator &lt;span class=&quot;math-container&quot;&gt;$p(s'|s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Any explanation on that would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19147" LastEditorUserId="2444" LastEditDate="2021-01-03T23:08:06.993" LastActivityDate="2021-01-03T23:08:06.993" Title="Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;notation&gt;&lt;reward-functions&gt;&lt;probability-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8515" PostTypeId="2" ParentId="8168" CreationDate="2018-10-19T10:12:24.067" Score="1" Body="&lt;p&gt;Yours is a really nice and easy question: you've seen how to use GAs in a complex problem but you're missing how to apply them to the most basic of all. I'll show you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a real-world implementation we have to structure our problem to be solved with GAs; we need to modify it, finding an equivalent representation that accepts individuals, each one of these is built from a genome and must be evaluated using a fitness function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you see this graphically you'll discover that the fitness function is just describing an area (or line or volume, depending on the grade) on our space and we are randomly dropping individuals on it, awarding the ones that fell higher than the others. We then try to modify the genomes of these guys to move them towards peaks of this fitness function.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kaXXd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kaXXd.png&quot; alt=&quot;Crossover vs mutation, I did this image, feel free to use it.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice the fitness function is our world, our ground truth and we are exploring it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to the function optimization part: well, we do not need any abstraction here, any strange individual representation or transformation of the problem; we just want to find a maximum, and this is exactly what GAs are for!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, let's have a look at the elements you need to solve your problem with GAs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fitness function =&gt; you have it already! It's the function you want to find the max of.&lt;/li&gt;&#xA;&lt;li&gt;Individual =&gt; as the maximum is enough, so the individual will be just a point in space.&lt;/li&gt;&#xA;&lt;li&gt;Genome =&gt; like every respective point in space, the genome will just be a collection of real numbers (one for each dimension).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, like in any GA, you will instantiate randomly an initial population (random points in the domain). The evaluation of course is the easy part, just put them into the function and you'll se the value of the individuals.&lt;br&gt;&#xA;What about crossover and mutation though? You'll have to use techniques that work with real-number genes, like BLXalpha, BLXalphabeta.&#xA;These are just defining ranges between values and picking random values inside these ranges, I wrote a pretty detailed answer about this, you can check it out at: &lt;a href=&quot;https://ai.stackexchange.com/a/6323/15530&quot;&gt;https://ai.stackexchange.com/a/6323/15530&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15530" LastEditorUserId="15530" LastEditDate="2019-07-01T02:59:57.767" LastActivityDate="2019-07-01T02:59:57.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8517" PostTypeId="2" ParentId="4779" CreationDate="2018-10-19T10:32:57.197" Score="1" Body="&lt;p&gt;It is not possible to have a negative IG, your outcome is negative because of a computational mistake: initially you have 69 positive instances, and after creating 2 children the sum drops to 56+11=67. Set is therefore less pure -&gt; IG is negative.&lt;/p&gt;&#xA;" OwnerUserId="19149" LastActivityDate="2018-10-19T10:32:57.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8518" PostTypeId="1" CreationDate="2018-10-19T11:55:43.517" Score="6" ViewCount="1222" Body="&lt;p&gt;What are some ways to design a neural network with the restriction that the &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; norm of the output values must be less than or equal to 1? In particular, how would I go about performing back-propagation for this net?&lt;/p&gt;&#xA;&lt;p&gt;I was thinking there must be some &amp;quot;penalty&amp;quot; method just like how in the mathematical optimization problem, you can introduce a log barrier function as the &amp;quot;penalty function&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="19150" LastEditorUserId="2444" LastEditDate="2020-10-29T09:13:33.983" LastActivityDate="2020-10-29T09:14:30.520" Title="How do we design a neural network such that the $L_1$ norm of the outputs is less than or equal to 1?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;constrained-optimization&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8519" PostTypeId="1" AcceptedAnswerId="8521" CreationDate="2018-10-19T14:50:54.090" Score="2" ViewCount="17" Body="&lt;p&gt;As you can see in the title, I'm trying to program an AI in Java that would help someone optimize his storage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The user has to enter the size of his storage space (a box, a room, a warehouse, etc...) and then enter the size of the items he has to store in this space. (note that everything must be a rectangular parallelepiped) And the AI should find the best position for each item such that space is optimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a list of what I started to do :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I asked the user to enter the size of the storage space (units are trivial here except for the computing cost of the AI, later on, I'm guessing), telling him that the values will be rounded down to the unit&lt;/li&gt;&#xA;&lt;li&gt;I started by creating a 3-dimensional array of integers representing the storage space's volume, using the 3 values taken earlier. Filling it with 0s, where 0s would later represent free space and 1s occupied space.&lt;/li&gt;&#xA;&lt;li&gt;Then, store in another multidimensional array the sizes of the items he has to store And that's where the AI part should be starting. The first thing the AI should do is check whether the addition of all the items' volumes doesn't surpass the storage space's volume. But then there are so many things to do and so many possibilities that I get lost in my thoughts and don't know where to start...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In conclusion, can anyone give me the proper terms of this problem in AI literature, as well as a link to an existing work of this kind? Thanks&lt;/p&gt;&#xA;" OwnerUserId="19155" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:11.810" LastActivityDate="2020-05-29T13:48:11.810" Title="AI that maximizes the storage of rectangular parallelepipeds in a bigger parallelepiped" Tags="&lt;optimization&gt;&lt;storage&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8520" PostTypeId="2" ParentId="8514" CreationDate="2018-10-19T15:23:10.143" Score="3" Body="&lt;p&gt;No, the substitution you suggest based on Equation (3.4) is not correct because you forgot about the &lt;span class=&quot;math-container&quot;&gt;$\sum_{r \in \mathcal{R}}$&lt;/span&gt; in the right-hand side Equation (3.4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Equation (3.4)&lt;/strong&gt; says (leaving out the middle part):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(s' \vert s, a) \doteq \sum_{r \in \mathcal{R}} p(s', r \vert s, a).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you plug this into Equation (3.6) to substitute the denominator, you can't forget about that sum, you have to include the complete sum in the denominator. Because we have two different sums summing over all rewards in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt;, I'll change the symbol used in the second sum to &lt;span class=&quot;math-container&quot;&gt;$r'$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;. This yields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s, a, s') \doteq \sum_{r \in \mathcal{R}} r \frac{p(s', r \vert s, a)}{\sum_{r' \in \mathcal{R}} p(s', r' \vert s, a)}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerator and denominator are different and will not cancel out.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Intuitively, the numerator is just a single probability; the probability of observing a specific next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and a reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; given a specific current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The denominator is a sum of many such probabilities, for &lt;strong&gt;all possible&lt;/strong&gt; rewards &lt;span class=&quot;math-container&quot;&gt;$r'$&lt;/span&gt; rather than just one specific reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;. Because this sum &quot;covers&quot; all possible events for the rewards, it essentially represents simply the probability of observing state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;  given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, paired with any arbitrary reward. That can more simply be denoted as &lt;span class=&quot;math-container&quot;&gt;$p(s' \vert s, a)$&lt;/span&gt;.... which makes sense because that's the very thing we started with :D&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Equation (3.6) is as follows (again leaving the middle part out):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s, a, s') \doteq \sum_{r \in \mathcal{R}} r \frac{p(s', r \vert s, a)}{p(s' \vert s, a)}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In normal English, the left-hand side says &quot;what reward do we expect to get (on average) if we transition from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; by executing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such an &lt;span class=&quot;math-container&quot;&gt;$\color{red}{\text{expectation of a quantity }r}$&lt;/span&gt; can always be computed by multiplying &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\text{the possible values that }r\text{ can take}}$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\color{orange}{\text{the probability of each particular value occurring}}$&lt;/span&gt;, and summing up those multiplications. In math, this looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\color{red}{r(s, a, s')} = \sum_{\color{blue}{r \in \mathcal{R}}} \color{blue}{r} \times \color{orange}{p(r \vert s, a, s')}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there is a rule in probability that says (see Rule of Multiplication on &lt;a href=&quot;https://stattrek.com/probability/probability-rules.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stattrek.com/probability/probability-rules.aspx&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(A, B) = p(A) \times p(B \vert A),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, if we take &lt;span class=&quot;math-container&quot;&gt;$A = s'$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B = r$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$s, a$&lt;/span&gt; as additional givens for all probabilities, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(s', r \vert s, a) = p(s' \vert s, a) \times p(r \vert s, a, s').$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be rewritten (dividing both sides of the equation by &lt;span class=&quot;math-container&quot;&gt;$p(s' \vert s, a)$&lt;/span&gt; and swapping the left-hand and right-hand sides) to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(r \vert s, a, s') = \frac{p(s', r \vert s, a)}{p(s' \vert s, a)}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plugging this in for the orange term in the coloured equation above (which itself is hopefully fairly easy to understand intuitively) yields Equation (3.6).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-10-19T17:36:11.363" LastActivityDate="2018-10-19T17:36:11.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8521" PostTypeId="2" ParentId="8519" CreationDate="2018-10-19T16:59:44.100" Score="0" Body="&lt;p&gt;A simple approach that gives a good baseline for such problems is &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulated annealing&lt;/a&gt;. The idea is that you do something random. If it improves things, then it is good. If it makes things worse, you still take it with some probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; shrinks over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more bad solutions you can rule out beforehand / the smarter you can encode your problem, the better solutions simulated annealing will give.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-10-19T16:59:44.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8522" PostTypeId="1" CreationDate="2018-10-19T18:13:35.587" Score="3" ViewCount="64" Body="&lt;p&gt;For example, consider an agent concerned with predicting the weather, with variable R indicating whether or not it is likely to rain, variable C indicating whether or not it is cloudy, and variable L indicating low pressure. Given knowledge base K:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L (Pressure is low)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C (It is cloudy)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C ∧ L ⇒ R, (Clouds and low pressure imply rain)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the agent may conclude R; thus, the agent’s knowledge implies that R is true, because K |= R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, given knowledge base L:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;¬L (Pressure is high)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C (It is cloudy)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C ∧ L ⇒ R, (Clouds and low pressure imply rain)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the agent cannot conclude that R is true; L 6|= R&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deriving a truth table:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L   C   r   ((L ∧ C) → r)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F   F   F   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F   F   T   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F   T   F   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F   T   T   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;T   F   F   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;T   F   T   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;T   T   F   F&lt;/p&gt;&#xA;&#xA;&lt;p&gt;T   T   T   T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this does not make sense.&lt;/p&gt;&#xA;" OwnerUserId="19164" LastEditorUserId="1671" LastEditDate="2019-10-15T19:10:33.633" LastActivityDate="2019-10-15T19:10:33.633" Title="How do I use truth tables to prove Entailment?" Tags="&lt;logic&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8523" PostTypeId="1" CreationDate="2018-10-19T18:13:38.887" Score="3" ViewCount="118" Body="&lt;p&gt;Is the emergence of a subfield of psychology, &quot;psychology of artificial intelligence&quot;, possible and necessary? If so, what questions it will address and will try to solve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly, it could also emerge as a new branch of engineering psychology, due to the fact that engineering psychology handles the realms of interactions between humans and machines, as well as human factors in general.&lt;/p&gt;&#xA;" OwnerUserId="7014" LastEditorUserId="2444" LastEditDate="2020-05-24T13:56:11.760" LastActivityDate="2021-02-18T16:01:50.130" Title="Is the emergence of the field &quot;psychology of artificial intelligence&quot; necessary?" Tags="&lt;agi&gt;&lt;artificial-consciousness&gt;&lt;academia&gt;&lt;cognitive-science&gt;&lt;ai-field&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8524" PostTypeId="2" ParentId="8523" CreationDate="2018-10-19T18:13:38.887" Score="0" Body="&lt;p&gt;And by trying to answer my own question, and how I see it, yes - there is some requirement like it stated in this particular article: &lt;a href=&quot;https://medium.com/womeninai/psychology-of-artificial-intelligence-ca0f0a9f3d7c&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/womeninai/psychology-of-artificial-intelligence-ca0f0a9f3d7c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because yes - there are multiple occurrences of an AI being biased, hateful, racist, sexist, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, humans themselves tend to act towards an AI in the insolent ways, so the human factor also needed to be accounted for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yeah - I think it will be a branch, or an extension of an engineering psychology, which exactly addresses primarily a human factor and relationships between the human and a machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But other than that it will address the factors of how should it be built, how we can expect it to behave, and how should it behave, because it should follow the strict moral code, to be not the threat to us.&lt;/p&gt;&#xA;" OwnerUserId="7014" LastActivityDate="2018-10-19T18:13:38.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8525" PostTypeId="2" ParentId="8496" CreationDate="2018-10-19T19:06:36.580" Score="2" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Markov Decision Process is an extension of Andrey Markov's action sequence that visualize action-result sequence possibilities as a directed acyclic graph.  One path through the acyclic graph, if it satisfies the Markov Property is called a Markov Chain.&lt;/p&gt;&#xA;&lt;p&gt;The Markov Property requires that the probability distribution of future states at any point within the acyclic graph be evaluated solely on the basis of the present state.&lt;/p&gt;&#xA;&lt;p&gt;Markov Chains are thus a stochastic model theoretically representing one of the set of possible paths.  And the action-result sequence is a list of state transitions corresponding to actions chosen solely by each action's previous state and the expectations that the expected subsequent state will most probably lead to the desired outcome.&lt;/p&gt;&#xA;&lt;p&gt;Andrey Markov based his work on Gustav Kirchhoff's work on spanning trees, which is based on Euler's initial directed graph work.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Exercise&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Exercise 3.2 was given with two parts.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is the MDP framework adequate to usefully represent all goal-directed learning tasks?&lt;/p&gt;&#xA;&lt;p&gt;Can you think of any clear exceptions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The first question is subjective in that it inquires about usefulness but does not define what it means.  If &amp;quot;useful&amp;quot; means the MDP will improve the chances of achieving a goal over a random selection of action at each state, then except in no win scenarios or the most contrived case where all actions have equal distribution of probable results, then the MDP is useful.&lt;/p&gt;&#xA;&lt;p&gt;If &amp;quot;useful&amp;quot; means optimal, then there are other approaches, with additional complexity and requiring additional computing resources that will improve odds of goal achievement.  These other approaches overcome one or more of the limitations of pure MDP.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Advancements and Alternatives&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Advancements made to MDP and alternatives to MDP, which number in the hundreds, include these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Logical detection of the infeasibility of goal achievement (no win scenario)&lt;/li&gt;&#xA;&lt;li&gt;Calculation of probabilities when only partial information is available about the current state&lt;/li&gt;&#xA;&lt;li&gt;Invocation of the decision at any point (continuous MDP used in real time systems)&lt;/li&gt;&#xA;&lt;li&gt;Probabilities are not known and must be learned from past experience where simple Q-learning is employed&lt;/li&gt;&#xA;&lt;li&gt;Past experience is used by statistically relating action-state details to generalizations derived from past action-result sequences or such information acquired or shared&lt;/li&gt;&#xA;&lt;li&gt;The action-state decisions, made within the context of an unknown system of changing or not reliably applied rules, can be used to tune a set of fuzzy rules in a fuzzy logic container and utilize fuzzy inference in the decisions&lt;/li&gt;&#xA;&lt;li&gt;Bluff and fraud detection&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Card Games&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Game play for a typical card game could make use of MDP, so MDP would be strictly useful, however not optimal.  Some of the above decisioning features would be more optimal, particularly those that deals with unknowns and employ rules, since the card game has them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Random or Decoupled&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Two obvious cases are (a) a truly randomized action-result world where goal achievement has equal probability no matter the sequence of moves or (b) a scenario where goal achievement is entirely decoupled from actions the actor can take.  In those cases, nothing will be useful with regard to the particular objective chosen.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Challenge&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The way to best learn from the exercise, though, is to find a scenario where MDP would be useless and one of the above listed &lt;strong&gt;Advancements and Alternatives&lt;/strong&gt; would be required rather than simply preferred.  If you look at the list, there will be some cases that will eventually come to mind.  I suggest you think it through, since the goal is to learn from the book.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-19T19:06:36.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8527" PostTypeId="1" CreationDate="2018-10-19T19:53:17.767" Score="2" ViewCount="1104" Body="&lt;p&gt;Should the weights of a neural network be updated after each example or at the end of the batch? Do I need a normalization factor in the second case?&lt;/p&gt;&#xA;" OwnerUserId="11814" LastEditorUserId="2444" LastEditDate="2019-11-04T03:38:09.087" LastActivityDate="2019-11-04T03:38:09.087" Title="Should the weights of a neural network be updated after each example or at the end of the batch?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="1" ClosedDate="2019-11-06T00:36:53.707" ContentLicense="CC BY-SA 4.0" />
  <row Id="8528" PostTypeId="1" AcceptedAnswerId="8538" CreationDate="2018-10-19T22:18:13.773" Score="1" ViewCount="359" Body="&lt;p&gt;I am reviewing a statement on the website for ES regarding structured exploration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.openai.com/evolution-strategies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.openai.com/evolution-strategies/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Structured exploration. Some RL algorithms (especially policy&#xA;  gradients) initialize with random policies, which often manifests as&#xA;  random jitter on spot for a long time. This effect is mitigated in&#xA;  Q-Learning due to epsilon-greedy policies, where the max operation can&#xA;  cause the agents to perform some consistent action for a while (e.g.&#xA;  holding down a left arrow). This is more likely to do something in a&#xA;  game than if the agent jitters on spot, as is the case with policy&#xA;  gradients. Similar to Q-learning, ES does not suffer from these&#xA;  problems because we can use deterministic policies and achieve&#xA;  consistent exploration.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Where can I find sources showing that policy gradients initialize with random policies, whereas Q-Learning uses epsilon-greedy policies? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, what does &quot;max operation&quot; have to do with epsilon-greedy policies?&lt;/p&gt;&#xA;" OwnerUserId="19167" LastActivityDate="2018-10-20T20:56:10.753" Title="Some RL algorithms (especially policy gradients) initialize with random policies, which often manifests as random jitter on spot for a long time?" Tags="&lt;reinforcement-learning&gt;&lt;evolutionary-algorithms&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8531" PostTypeId="2" ParentId="2927" CreationDate="2018-10-20T07:11:18.123" Score="1" Body="&lt;p&gt;Embedded artificial intelligence is a booming area of development.  The hardware and software basics are in place, but the research into how to design mature home and industrial products is only beginning to approach the vision cast for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning lovers will point to some papers that theoretically approach machine control using policy based action decision modalities.  Those actually working on machine control know that the convergence of these systems is not workable yet for field operational machines.  The more general the machine must operate, the more unworkable a machine learning solution is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aaeon Up produces GPU and computer on a board products using Intel chip sets that support embedded AI development and NVidia and Qualcom are expected to follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inference is the noun that corresponds to infer.  Deep networks don't ever infer anything during operation after trained.  What they do is do.  The terms in use don't include inference.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Production&lt;/li&gt;&#xA;&lt;li&gt;In the field&lt;/li&gt;&#xA;&lt;li&gt;Deployed&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;In situ&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Inference is usually the term used in the predicate logic or rules system sub-field of AI.  Deep networks do not, by themselves, contain logic other than the logic in the CPU, FPU, or GPU that performs arithmetic.  After training, the are essentially programmed DSPs (digital signal processors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Some training systems can be reentered to further training and some deep designs allow concurrent training and operation, but learning and doing are not fully integrated in deep learning models yet.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Power consumption, heat dissipation, size, and weight are issues in embedded systems, which makes memory usage and thrift a challenge.  This is typical, and not just for AI apps.  In aeronautics, the effects of gravity and acceleration compound the challenge greatly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data science methods that work well for FaceBook, Google, Amazon, Lexis Nexis, and Pentaho are not designed for light-weight, low-power applications.  In autonomous vehicle systems, whether they are ground vehicles, submerged vehicles, or flight vehicles makes huge difference in these respects.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are both systems required on the same application?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In vehicles in general, portions of the function can be learned at different points of time.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;During a vendor's design of a family of models&lt;/li&gt;&#xA;&lt;li&gt;During a vendor's finalization of a particular model with its accessories&lt;/li&gt;&#xA;&lt;li&gt;During the quality control of a particular ID numbered vehicle before it is shipped&lt;/li&gt;&#xA;&lt;li&gt;During the first few months of usage, where the vehicle may learn the particular usage patterns and environment of the buyer&lt;/li&gt;&#xA;&lt;li&gt;During the remaining life of the vehicle to produce continuous operational improvement and adaptation to normal ware and maintenance processes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;An autonomous car or an application where visual and image recognition is done, will it have both training system to be trained and an inference system to execute? Or [does] it [have] only the [deployed trained run time system] and communicate with a distant system which is already trained and has built a database?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any military, commercial, or delivery network vehicle, requiring communication with a home base leaves open a number of security risks and possible attack vectors.  Patch systems with proper security is probable, and several vehicle manufacturers have had firmware updates that are installed every time the vehicle is serviced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training data can be compressed using lz4 or simply by packing data as has been done in satellite communications since the 1970s.  The memory is challenged more by the training that can only occur during field use of the product.  Embedded product designers have developed over the last half century a large array of useful techniques to address the challenge, and more will be developed along side the real time AI technology.  The two disciplines will emerge together.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-10-20T07:11:18.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8533" PostTypeId="1" CreationDate="2018-10-20T11:16:20.773" Score="1" ViewCount="22" Body="&lt;p&gt;There are several levels of abstraction involved in piloting and driving.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Signals representing the state of the vehicle and its environment originating from multiple transducers&lt;sup&gt;1&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Latched sample vectors/matrices&lt;/li&gt;&#xA;&lt;li&gt;Boundary events (locations, spectral features, movement, appearance and disappearance of edges, lines, and sounds)&lt;/li&gt;&#xA;&lt;li&gt;Objects&lt;/li&gt;&#xA;&lt;li&gt;Object movements&lt;/li&gt;&#xA;&lt;li&gt;Object types (runways, roads, aircraft, birds, cars, people, pets, screeches, horns, bells, blinking lights, gates, signals, clouds, bridges, trains, buses, towers, antennas, buildings, curbs)&lt;/li&gt;&#xA;&lt;li&gt;Trajectory probabilities based on object movements and types&lt;/li&gt;&#xA;&lt;li&gt;Behaviors based on all the above hints&lt;/li&gt;&#xA;&lt;li&gt;Intentions based on behavior sequences and specific object recognition&lt;/li&gt;&#xA;&lt;li&gt;Collision risk detection&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Moving from interpretation to control execution ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Preemptive collision avoidance reaction&lt;/li&gt;&#xA;&lt;li&gt;Horn sounding&lt;/li&gt;&#xA;&lt;li&gt;Plan adjustment&lt;/li&gt;&#xA;&lt;li&gt;Alignment of plan to state&lt;/li&gt;&#xA;&lt;li&gt;Trajectory control&lt;/li&gt;&#xA;&lt;li&gt;Skid avoidance&lt;/li&gt;&#xA;&lt;li&gt;Skid avoidance reaction&lt;/li&gt;&#xA;&lt;li&gt;Steering, breaking, and signalling&lt;/li&gt;&#xA;&lt;li&gt;Notifications to passengers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What, if any, levels of higher abstraction can be sacrificed?  Humans, if they are excellent pilots or drivers, can use all of these levels to improve pedestrian and passenger safety and minimize expense in time and money.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Optical detectors, microphones, strain gauge bridges, temperature and pressure gauges, triangulation reply signals, voltmeters, position encoders, key depression switches, flow detectors, altimeters, radar transducers, tachometers, accelerometers &lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-20T11:16:20.773" Title="To what level of abstraction must fully automated vehicles build their driving model before safety can be maximized?" Tags="&lt;path-planning&gt;&lt;autonomous-vehicles&gt;&lt;ai-safety&gt;&lt;collision-avoidance&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8534" PostTypeId="1" CreationDate="2018-10-20T12:58:12.103" Score="6" ViewCount="362" Body="&lt;p&gt;I struggle to find Rosenblatt's perceptron training algorithm in any of his publications from 1957 - 1961, namely:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://apps.dtic.mil/dtic/tr/fulltext/u2/256582.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.3398&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The perceptron: A probabilistic model for information storage and organization in the brain&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://blogs.umass.edu/brain-wars/files/2016/03/rosenblatt-1957.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Perceptron — A Perceiving and Recognizing Automaton&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does anyone know where to find the original learning formula?&lt;/p&gt;&#xA;" OwnerUserId="19177" LastEditorUserId="2444" LastEditDate="2020-01-14T15:42:10.350" LastActivityDate="2020-12-05T19:04:15.457" Title="Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8536" PostTypeId="2" ParentId="4953" CreationDate="2018-10-20T17:00:25.977" Score="2" Body="&lt;p&gt;An introduction to the &lt;a href=&quot;https://www.youtube.com/watch?v=_QZLbR67fUU&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haar features&lt;/a&gt; is provided in the youTube video.  The video indicates the VJ face detector leverages &lt;strong&gt;a selected combination&lt;/strong&gt; of Haar features (convolutional kernels) to detect &lt;strong&gt;facial features&lt;/strong&gt; (weak classifiers), such as the nose bridge.  The binary presence of the weak classifiers are summed to determine if the window contains a face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ability for a VJ algorithm to detect emotion would rely on the ability to assign a set of Haar features (kernels) to recognize features associated with a particular emotion label (surprise, anger, content, fear). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is conceivable that the initial stage of an  emotion classifier could use a the VJ algorithm to identify a face for additional stages to classify emotion.&lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="18819" LastEditDate="2018-10-20T17:15:33.407" LastActivityDate="2018-10-20T17:15:33.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8538" PostTypeId="2" ParentId="8528" CreationDate="2018-10-20T20:34:50.553" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Where can I find sources showing that policy gradients initialize with random policies, whereas Q-Learning uses epsilon-greedy policies?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can find example algorithms for Q learning and policy gradients in &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Sutton &amp;amp; Barto's&lt;/em&gt; Reinforcement Learning: An Introduction&lt;/a&gt; - Q learning is in chapter 6, and policy gradients explained in chapter 13.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither of these things are strictly true in all cases. However, both are very common situations for the two kinds of learning agent:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Policy gradient solvers learn a policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; that gives the probability of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given observed state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Typically this is implemented as a neural network. Neural networks are initialised randomly. For discrete action selection using softmax output layer, the initial function will roughly be choosing evenly from all possible actions. So if some actions oppose and &lt;em&gt;undo&lt;/em&gt; each other, e.g. move left and move right are options - then the situation as described in your quote can easily happen. Nowadays there are many other solutions using policy gradients that don't suffer as much with this effect. For instance, deterministic policy gradient methods such as A2C or DDPG.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For Q learning, and many variants of it, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy is a very commonly used action selection mechanism. It is convenient and simple, and allows a simple parameter to control balance between exploration and exploitation whilst learning. However, Q learning can work with &lt;em&gt;any&lt;/em&gt; action selection mechanism that has some possibility of acting optimally at least occasionally. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The best approach to action selection during the learning process in both policy-based methods and value-based methods is an active area of research. So if you read the RL literature you may find a lot of variation. The blog you quoted from has identified two quite common choices.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, what does &quot;max operation&quot; have to do with epsilon-greedy policies?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &quot;max operation&quot; is finding the maximum value of some function produced when varying a parameter. The related &quot;argmax operation&quot; is finding the value of the parameter that produces the maximum value. Q learning can use both types of operation, but specifically uses &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy action selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy requires acting optimally (according to current estimates of value function) with probability &lt;span class=&quot;math-container&quot;&gt;$p=1-\epsilon$&lt;/span&gt;, and randomly with equal probability of each action with probability &lt;span class=&quot;math-container&quot;&gt;$p=\epsilon$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to do this, the algorithm usually (with probability &lt;span class=&quot;math-container&quot;&gt;$p=1-\epsilon$&lt;/span&gt;) needs to know what the current best guess at an optimal solution is. That is the greedy action with respect to the current Q values i.e. &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}_a Q(s,a)$&lt;/span&gt;  - &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-20T20:56:10.753" LastActivityDate="2018-10-20T20:56:10.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8540" PostTypeId="2" ParentId="8527" CreationDate="2018-10-20T21:35:35.787" Score="1" Body="&lt;p&gt;The backpropagation step is generally used to compute the gradients and update the weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us say, you are implementing gradient descent, select the whole training batch, perform the forward propagation using the current set of neurons/weights to get the classification output. Then compute the loss/cost with respect to the actual labels, (Note: This step contains dividing the loss and &lt;strong&gt;divide it by the number of training example&lt;/strong&gt; to get the cost function). Once you get the cost you backpropagate by computing the gradients at each layer with respect to cost from output to input and perform the weight updates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using mini-batch, you compute the cost for that mini-batch, divide by the number of examples in the mini-batch, then compute the gradients and perform the update in the backprop step. &lt;/p&gt;&#xA;" OwnerUserId="18772" LastActivityDate="2018-10-20T21:35:35.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8541" PostTypeId="2" ParentId="8527" CreationDate="2018-10-20T23:41:33.907" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When am I supposed to update my weights? After each forward-, and backpropagation; and or after each completed batch?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In your example, you should update the weights after each back propagation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, you can back propagate the result by adding each error/loss for each prediction/example in a batch and then update the weights. &lt;strong&gt;pseudocode- for batch size 3:&lt;/strong&gt;&lt;br&gt;&#xA;&lt;code&gt;w = backprop(Error(t1)+Error(t2)+Error(t3))&lt;/code&gt;&#xA;&lt;code&gt;update_wights(w*learning_rate)&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Now what happen If you back propagate every error for every prediction/example in a batch and store them(without updating), and after completion of that batch update the  weights by using stored calculated values, then there is a chance to stuck in the local minima also some research shows that it's actually slow training. &lt;a href=&quot;http://axon.cs.byu.edu/papers/Wilson.nn03.batch.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;research paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;note- all those techniques in 1,2,3 are different but 1 and 2 results same. Also in 2, adding errors of different training example makes a new graph of neural network and back propagating it means back propagating the resulted graph.&lt;/p&gt;&#xA;" OwnerUserId="18459" LastActivityDate="2018-10-20T23:41:33.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8544" PostTypeId="2" ParentId="8258" CreationDate="2018-10-21T06:42:37.253" Score="2" Body="&lt;p&gt;&lt;strong&gt;Direct Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://btsd.ethz.ch/shareddata/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Belgium TS Dataset&lt;/a&gt; may be helpful, as well as &lt;a href=&quot;http://benchmark.ini.rub.de/?section=gtsdb&amp;amp;subsection=dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;The German Traffic Sign Detection Benchmark&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Notes Based on Question Author's Idea&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea in the question author's addendum of placing signs onto street sides and corners is a good one, but to do it repeatably and in a way that doesn't bias the training is its own research project.  However, it is a good research direction.  What would be of benefit to AV researchers worldwide is a multi-network topology and equilibrium strategy with the objective to create the following data generation features.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Street sign symbol inputs in image form, with or without cropping, as movie frame sequences or single still shots, or from SVG files.&lt;/li&gt;&#xA;&lt;li&gt;Annotation generation using partially human-labelled data.&lt;/li&gt;&#xA;&lt;li&gt;3D analysis of sign angle and perspective setting so that the images appear exactly as they would from a vehicle's imaging system.&lt;/li&gt;&#xA;&lt;li&gt;Matching of lighting between the superimposed sign and the background scene.&lt;/li&gt;&#xA;&lt;li&gt;Automatic blue-screening for the sign image.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is obviously not a basic data hygiene problem. It is its own AI project, but the return on this research project in terms of furthering the AV technology is immense and may have significant data set statistical advantages over collecting data from the vendors that supply images to Google maps and other Big Data aggregators.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-21T14:19:19.097" LastActivityDate="2018-10-21T14:19:19.097" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8545" PostTypeId="2" ParentId="8458" CreationDate="2018-10-21T07:13:30.967" Score="2" Body="&lt;p&gt;The Hough Transform extended to orthogonal ellipses uses this model, accumulating on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$\{x, y\}$&lt;/span&gt; with parameter matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{Bmatrix} &#xA;c_x &amp;amp; c_y \\ &#xA;r_x &amp;amp; r_y&#xA;\end{Bmatrix}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$1 = \dfrac {(x - c_x) \, \cos \theta} {r_x} + \dfrac {(y - c_y) \, \sin \theta} {r_y}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is looking to detect the normal lines, so any of the several algorithms for the above model can be modified to accumulate on &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$\{x, y\}$&lt;/span&gt; with parameter matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{Bmatrix} &#xA;c_x &amp;amp; c_y  \\ &#xA;r_x &amp;amp; r_y&#xA;\end{Bmatrix}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$0 = \dfrac {x - c_x} {r_x} + \dfrac {y - c_y} {r_y}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lines that intersect &lt;span class=&quot;math-container&quot;&gt;$(c_x, c_y)$&lt;/span&gt; don't rely on &lt;span class=&quot;math-container&quot;&gt;$r_x$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$r_y$&lt;/span&gt;.  However, it may be useful to recognize that, if radially equally spaced, viewing the lines from a position other than one that projects into the plane of the lines at &lt;span class=&quot;math-container&quot;&gt;$(c_x, c_y)$&lt;/span&gt; will present a line density that is a function of &lt;span class=&quot;math-container&quot;&gt;$\arctan (r_x, r_y)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-23T05:40:00.017" LastActivityDate="2018-10-23T05:40:00.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8546" PostTypeId="1" AcceptedAnswerId="8557" CreationDate="2018-10-21T07:24:15.437" Score="5" ViewCount="64" Body="&lt;p&gt;Suppose that I have a model M that overfits a large dataset S such that the test error is 30%. Does that mean that there will always exist a model that is smaller and less complex than M that will have a test error less than 30% on S (and does not overfit S). &lt;/p&gt;&#xA;" OwnerUserId="19190" LastActivityDate="2018-10-21T16:08:07.117" Title="Does overfitting imply an upper bound on model size/complexity?" Tags="&lt;machine-learning&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8548" PostTypeId="1" AcceptedAnswerId="8552" CreationDate="2018-10-21T08:17:40.687" Score="2" ViewCount="265" Body="&lt;p&gt;What is the best and easiest programming language to learn to implement genetic algorithms? C++ or Python, or any other?&lt;/p&gt;&#xA;" OwnerUserId="19192" LastEditorUserId="2444" LastEditDate="2020-02-09T17:39:37.533" LastActivityDate="2020-02-09T17:39:37.533" Title="What is the best programming language to learn to implement genetic algorithms?" Tags="&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;programming-languages&gt;&lt;genetic-programming&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8549" PostTypeId="2" ParentId="8548" CreationDate="2018-10-21T09:00:59.283" Score="5" Body="&lt;p&gt;There is no &quot;best language&quot; for any problem. There are too many variables to consider, even when advising a single person with a single project plan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the choice is between Python and C++, I would generally advise:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to implement from scratch and learn how the algorithm works, use Python with numeric/accelerated libraries such as NumPy or PyTorch. Python script is quicker to prototype and try different ideas, due to loose variable typing and built-in high-level structures such as &lt;code&gt;dict&lt;/code&gt; and &lt;code&gt;list&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to write core, efficient libraries, then C++ will out-perform Python, but writing these will take longer. There are plenty of C++ libraries available (e.g. TensorFlow is available with a C++ API), but the community around them is less focused than with Python toolkits.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can also combine both approaches and write specific libraries in C or C++ to improve performance at any time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Genetic Algorithms, the speed bottleneck is most often population assessment - e.g. running the environment simulation to get a fitness measure for each individual - and not the GA itself. So if you have a specific problem or problem domain in mind, may want to orient your choice around a language that already has support for the kind of environments where you want to run your GA. GAs usually benefit greatly from parallelisation, so if you are aiming for something ambitious you will want to look into GPU support and/or distributed computing toolkits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most researchers/hobbyists working in AI-related fields end up using multiple languages over time. You may end up with a favourite language environment, which might be Python, Julia, Java, C++, C, C#, Lua, LISP, Prolog, Matlab/Octave, R . . . but you will end up needing a smattering of other languages, and usually skills with specific toolkits such as TensorFlow, Scikit-Learn, Hadoop etc in order to complete projects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't be afraid that you will &quot;waste time&quot; learning in one language initially then needing to transfer to another. Your learning of algorithms will be transferable, and your first attempts will most likely not be that re-usable as library code anyway, so you are going to re-implement your ideas, perhaps many times. My first simulated annealing project was written in Fortran 77 . . . 20 years later I dug that knowledge up again and implemented in Ruby/C - nowadays I work in Python for the AI/hobby stuff even though my professional career sees me working mostly in Ruby.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-21T09:16:04.990" LastActivityDate="2018-10-21T09:16:04.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8550" PostTypeId="1" CreationDate="2018-10-21T09:15:40.760" Score="7" ViewCount="82" Body="&lt;p&gt;What are the future prospects in near future from a theoretical investigation of description logics, and modal logics in the context of artificial intelligence research?&lt;/p&gt;&#xA;" OwnerUserId="19193" LastEditorUserId="1581" LastEditDate="2018-10-30T17:00:49.623" LastActivityDate="2019-04-28T18:02:56.103" Title="What are the current trends/open questions in logics for knowledge representation?" Tags="&lt;logic&gt;&lt;knowledge-representation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8551" PostTypeId="1" CreationDate="2018-10-21T09:34:21.847" Score="3" ViewCount="85" Body="&lt;p&gt;I have a data input vector ( No Image classification) which size varys from 2 to 7 entrys. Every one of them belongs to a class Out of 7. So I have a variable Input size and a variable Output size. How can I deal with the variable Input sizes? I know Zero padding is a option but maybe there are better ways?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seconds: Is multi Label classification possible in one Network? What I mean: The first entry has to bei classified in one of the seven classes, the second entry... and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also open to other classification techniques, If there is a better one that suits the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&#xA;Gesetzt&lt;/p&gt;&#xA;" OwnerUserId="19195" LastActivityDate="2018-10-21T09:34:21.847" Title="Variable sized input-Multi Label Classification with Neural Network" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8552" PostTypeId="2" ParentId="8548" CreationDate="2018-10-21T11:00:00.263" Score="-1" Body="&lt;p&gt;Matlab may be a good option to get started with the implementation of genetic algorithms, given that there a lot of pre-defined functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. &lt;a href=&quot;https://www.mathworks.com/discovery/genetic-algorithm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/discovery/genetic-algorithm.html&lt;/a&gt; and &lt;a href=&quot;https://www.mathworks.com/help/gads/examples/coding-and-minimizing-a-fitness-function-using-the-genetic-algorithm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/help/gads/examples/coding-and-minimizing-a-fitness-function-using-the-genetic-algorithm.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14749" LastEditorUserId="2444" LastEditDate="2020-02-09T14:51:57.713" LastActivityDate="2020-02-09T14:51:57.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8553" PostTypeId="1" CreationDate="2018-10-21T12:44:38.280" Score="2" ViewCount="39" Body="&lt;p&gt;I have read quite a lot about capsule networks, but I cannot understand how the squashed vector would also rotate in response to rotation or translation of the image. A simple example would be helpful. I understand how routing by agreement works.&lt;/p&gt;&#xA;" OwnerUserId="19201" LastEditorUserId="2444" LastEditDate="2020-06-09T11:38:53.717" LastActivityDate="2020-06-09T11:38:53.717" Title="How exactly is equivariance achieved in capsule neural networks?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;capsule-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8554" PostTypeId="1" CreationDate="2018-10-21T13:35:49.953" Score="5" ViewCount="630" Body="&lt;p&gt;I am trying to study the book &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2018jan1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (Sutton &amp;amp; Barto, 2018). In chapter 3.1 the authors state the following exercise&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Exercise 3.5&lt;/em&gt; Give a table analogous to that in Example 3.3, but for &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt;. It should have columns for &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, and&#xA;  &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt;, and a row for every 4-tupel for which &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)&amp;gt;0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The following table and graphical representation of the Markov Decision Process is given on the next page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5xt43.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5xt43.png&quot; alt=&quot;Example 3.3&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use &lt;span class=&quot;math-container&quot;&gt;$p(s'\cup r|s,a)=p(s'|s,a)+p(r|s,a)-p(s' \cap r|s,a)$&lt;/span&gt; but without a significant progress because I think this formula does not make any sense as &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; are not from the same set. How is this exercise supposed to be solved?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Maybe this exercise intends to be solved by using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(s'|s,a)=\sum_{r\in \mathcal{R}}p(s',r|s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a,s')=\sum_{r\in \mathcal{R}}r\dfrac{p(s',r|s,a)}{p(s|s,a)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{s'\in\mathcal{S}}\sum_{r\in\mathcal{R}}p(s',r|s,a)=1$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the resulting system is a linear system of 30 equation with 48 unknowns. I think I am missing some equations...&lt;/p&gt;&#xA;" OwnerUserId="19123" LastEditorUserId="19123" LastEditDate="2018-10-21T16:52:48.470" LastActivityDate="2020-07-29T03:33:49.027" Title="Reinforcement Learning (RL) how to obtain $p(s',r|s,a)$" Tags="&lt;reinforcement-learning&gt;&lt;probability&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8555" PostTypeId="2" ParentId="8554" CreationDate="2018-10-21T15:06:07.403" Score="3" Body="&lt;p&gt;The function &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; gives the &lt;em&gt;expected&lt;/em&gt; reward in each scenario, but not the distribution of rewards that lead to values &lt;span class=&quot;math-container&quot;&gt;$r_{search}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r_{wait}$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The text explains that reward is &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; for each can found, and that different distributions of numbers of cans are expected when waiting as opposed to searching. However, it does not give any description of the actual distributions, just summarises them as the two expected rewards, and suggests &lt;span class=&quot;math-container&quot;&gt;$r_{search} \gt r_{wait}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have two main ways to answer the exercise:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Invent some parameters for the distributions of &lt;span class=&quot;math-container&quot;&gt;$r_{search}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r_{wait}$&lt;/span&gt; in order to split up single values of &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; into multiple values of &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s,a)$&lt;/span&gt;. E.g you could decide that &lt;span class=&quot;math-container&quot;&gt;$r_{search}$&lt;/span&gt; consists of &lt;span class=&quot;math-container&quot;&gt;$0  \eta_0 + 1  \eta_1 + 2 \eta_2 + 3 \eta_3$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\eta_0, \eta_1, \eta_2, \eta_3$&lt;/span&gt; are probabilities that sum to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; - each row that currently has &lt;span class=&quot;math-container&quot;&gt;$r_{search}$&lt;/span&gt; as the output of &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; would then split into 4 rows with reward 0, 1, 2, 3 to complete the new table . . . &lt;span class=&quot;math-container&quot;&gt;$r_{wait}$&lt;/span&gt; would need a different set of parameters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ignore the details of the distribution, move column &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s′)$&lt;/span&gt; to the left and call it &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, changing &lt;span class=&quot;math-container&quot;&gt;$p(s|s,a)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s,a)$&lt;/span&gt;. It might be all that's expected given the lack of information.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My personal opinion is that the authors want you to think about solution 1 - the only issue is that it requires you to invent some new parameters that were not provided. The ones I name are only a suggestion, they do not represent a specific &quot;correct&quot; answer in terms provided by the book, because the book omits those details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example to start with, if you start solution 1, and use parameters as I have labelled them, you will end up with a first row looking like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$s\qquad \qquad a\qquad  \qquad  s'\qquad  \qquad   r \qquad       p(s', r| s, a)$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$high  \qquad \quad search  \qquad  high  \qquad \quad   0  \qquad   \alpha \eta_0$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-21T16:19:46.897" LastActivityDate="2018-10-21T16:19:46.897" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8556" PostTypeId="2" ParentId="8550" CreationDate="2018-10-21T15:50:02.147" Score="2" Body="&lt;p&gt;Some of the work on descriptive logics and modal logics was done within the context of artificial intelligence from a research funding perspective.  Some was part of the normal academic apparatus of mathematics departments.  Furtherance of these fields in the AI context has been hindered by historically low return on investment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although first order logic, propositional logic, and fuzzy logic have been investigated conceptually and developed considerably, there are no theorems of note that have been proven without human assistance.  Currently, mathematics is furthered solely by willful acts of humans, although that fact is not immutable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There have been no proofs offered that artificial machinery cannot ever apply logic to prove theorems, develop disciplines, or assemble new logical systems.  There are also no existing proofs to the contrary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an open question in academia and with governments and corporations investigating the potentials of AI.  To characterize any of the work in progress as a trend would be to exaggerate the effort level along one of many theoretical paths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A search of academic articles using either descriptive logics and modal logics as search terms, with date constrained to 2008-2018, returns a trend of variety.  Another search of the same decade for articles that reference the twentieth century work in those disciplines produces the same wide distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no media hype or academic, corporate, or government claims of any research results that can be translated into useful application.  All of these statistics are research inhibitory.  Investors and educational institutions like paths with clear advantages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changes to this condition may come from work being done on the ontological and epistomological side in neural research as it applies to human linguistic networks in the brain.  Although lower animals can appear to draw conclusions, they are largely learning patterns of behavior based on past experience without applying logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dolphins, elephants, dogs, and primates can communicate and exhibit forms of intelligence and personality traits, but existing research indicates that they are unable to learn dialog or develop the ability to form new ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those investigating semantic approaches in AI seem to exhibit a single orientation, and that may be the one identifiable trend.  They suspect that there is one single causal origin responsible for three things.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The ability to develop complex languages&lt;/li&gt;&#xA;&lt;li&gt;The ability to construct new schools of thought&lt;/li&gt;&#xA;&lt;li&gt;The ability to develop a logical argument for or against a proposition or a theory or consider its likelihood logically&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There may be something to the idea that natural language rather than formal grammars are the key to higher thinking.&lt;/p&gt;&#xA;" OwnerUserId="19127" LastActivityDate="2018-10-21T15:50:02.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8557" PostTypeId="2" ParentId="8546" CreationDate="2018-10-21T16:08:07.117" Score="1" Body="&lt;p&gt;over-fitting does seem to imply an upper bound on model complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As complexity of the model increases, its generality tends to decrease.  This is not an indisputable fact, but an identifiable trend, provided that the model is not inundated with redundancy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For polynomial expressions, a Taylor series of 100 terms can always perfectly represent the position of 100 points in a time series, although no predictive general information can be ascertained from the 100 coefficients of the Taylor series.  This is also true for spectral analysis, Lyapunov exponent estimation, spline fitting, or artificial network convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is generally accepted as the definition of over-fitting does not come from the AI field.  It originates from the mathematics of surface approximation using closed forms.  The closed forms may be derived from a single differential equation or systems of them, but not usually, since closed forms do not exist for many such systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The information science principles behind feature extraction are closely aligned to the correlation between generality and predictive usefulness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, we can't conclude, &quot;There will always exist a model that is smaller and less complex than &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; that will have a test error less than 30% on &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&quot;  Counter examples can probably be constructed with a little thought and effort.&lt;/p&gt;&#xA;" OwnerUserId="19127" LastActivityDate="2018-10-21T16:08:07.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8558" PostTypeId="2" ParentId="8523" CreationDate="2018-10-21T16:31:29.173" Score="0" Body="&lt;p&gt;That assumes that a psyche to study will be identifiable in artificially intelligent computing machinery.  If we assume that, then it makes sense that psychology for these intelligences may diverge from the study of the human psyche, since the result of AI research won't necessarily produce a detailed simulation of normal human psycho-social characteristics but rather something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start this new fork of psychology, we'd need AI more sophisticated than current AI which some consider little more than pattern matching and surface fitting.  THe critics of the AI hype would say, &quot;We need a new branch of AI first.  Then a new branch of psychology might be needed to promote AI mental health.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the interaction between humans and machines enters the picture, the topic is not necessarily psychology but cybernetics.  We can study the computer reaction to humans and the human reaction to computers.  Neil Postman's &lt;em&gt;Technopoly&lt;/em&gt; was one of the early works.  Even the books about the psychological complications resulting from industrialization touched on the beginnings of the effects of automation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI reaction to humans is the side of the question's proposal that hasn't received much attention and maybe should.  Just a simple example illustrates.  How does a deep network respond to a change in personnel training it, with different methodology in preparation of data and the tweaking of hyper-parameters?  Will they perform poorly at first, as they adjust?  What is the actual mechanism of adjustment, if any exist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a little out there at this point, but not so as robotics develops further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will an AV get disoriented if sold to a new owner in a different location?  Will any location dependent training need to be reset?  Will household robotics be like cats, where they hide and react oddly when the owner moves to a new house or apartment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, what questions it will address and will try to solve?&lt;/p&gt;&#xA;" OwnerUserId="19127" LastActivityDate="2018-10-21T16:31:29.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8559" PostTypeId="1" CreationDate="2018-10-21T16:38:29.787" Score="2" ViewCount="425" Body="&lt;p&gt;Take a look at section 2.2.2 of &lt;a href=&quot;https://doc.lagout.org/science/0_Computer%20Science/2_Algorithms/Introduction%20to%20Evolutionary%20Algorithms%20%5BYu%20%26%20Gen%202010-06-23%5D.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt; (from Page-15 to 16).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;em&gt;2.2.2 Representation and Evaluation&lt;/em&gt;&lt;/strong&gt;  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$max f (x)= x sin(10πx)+2.0 ... ... ... (2.8)$$&lt;/span&gt;&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$s.t. −1 ≤ x ≤ 2$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;We can use a real number, in the range &lt;span class=&quot;math-container&quot;&gt;$[−1,2]$&lt;/span&gt;, to represent a solution in Eq. &lt;span class=&quot;math-container&quot;&gt;$2.8$&lt;/span&gt; directly. Many operators can handle real number representation. But we use the binary code or binary representation here for two reasons. GAs were originally proposed to be binary code to imitate the genetic encoding of natural organisms. On the other hand, binary code is good for pedagogy. A binary chromosome is necessary to represent a solution x in the scale &lt;span class=&quot;math-container&quot;&gt;$[−1,2]$&lt;/span&gt;. The same holds for the binary representation of real numbers in a computer.  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In binary code, we cannot represent a real number completely correctly, so a&#xA;  trade-off is necessary. A tolerance needs to be deﬁned by the user, which means the errors below the tolerance are extraneous. If we divide the deﬁnition domain into &lt;span class=&quot;math-container&quot;&gt;$2^1 = 2$&lt;/span&gt; parts evenly and select the smallest number in the parts to represent any number in the division, we can only represent &lt;span class=&quot;math-container&quot;&gt;$−1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; respectively. &lt;span class=&quot;math-container&quot;&gt;$2^2 = 4$&lt;/span&gt; divisions make the &lt;span class=&quot;math-container&quot;&gt;$00$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$01$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$11$&lt;/span&gt; represent &lt;span class=&quot;math-container&quot;&gt;$−1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$−0.25$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$1.25$&lt;/span&gt;, respectively. The larger division number we select, the less error there is in representing a real number on binary code. Suppose we use &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; binary codes to represent a real number in the range &lt;span class=&quot;math-container&quot;&gt;$[−1,2]$&lt;/span&gt;; the maximum error is &lt;span class=&quot;math-container&quot;&gt;$\frac{3}{2100} ≈ {2.37}^{−30}$&lt;/span&gt;, which would be satisfactory for most users. In this way, we can represent a real number with any accuracy requirements. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In this problem, we use &lt;span class=&quot;math-container&quot;&gt;$l = 12$&lt;/span&gt; binary codes to represent one real number as follows, which constitutes a chromosome to be evolved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Actually, I haven't understood this text. I know that binary numbers are already &lt;a href=&quot;http://cs.furman.edu/digitaldomain/more/ch6/dec_frac_to_bin.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;able to represent fractions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what are they talking about?&lt;/p&gt;&#xA;" OwnerDisplayName="user3642" LastEditorDisplayName="user3642" LastEditDate="2018-10-21T17:42:47.167" LastActivityDate="2021-02-16T03:02:50.240" Title="Representation of real numbers in Genetic Algorithm" Tags="&lt;genetic-algorithms&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8560" PostTypeId="1" CreationDate="2018-10-21T17:09:04.533" Score="10" ViewCount="26342" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size&#xA;can be one of three options:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;batch mode&lt;/strong&gt;: where the batch size is equal to the total dataset thus making the iteration and epoch values equivalent&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;mini-batch mode&lt;/strong&gt;: where the batch size is greater than one but less than the total dataset size. Usually, a number that can be divided into the total dataset size.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;stochastic mode&lt;/strong&gt;: where the batch size is equal to one. Therefore the gradient and the neural network parameters are updated after each sample.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How do I choose the optimal batch size, for a given task, neural network or optimization problem?&lt;/p&gt;&#xA;&lt;p&gt;If you hypothetically didn't have to worry about computational issues, what would the optimal batch size be?&lt;/p&gt;&#xA;" OwnerUserId="11814" LastEditorUserId="2444" LastEditDate="2020-10-31T10:07:38.903" LastActivityDate="2020-10-31T10:07:38.903" Title="How do I choose the optimal batch size?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;stochastic-gradient-descent&gt;&lt;batch-size&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8563" PostTypeId="2" ParentId="8559" CreationDate="2018-10-21T18:32:41.027" Score="0" Body="&lt;p&gt;The authors want to use a binary genome which contains specific interpretation/mapping to a fixed precision real number. They could use one or more existing schemes, but these typically won't represent the exact precision and range of real numbers they want to use in the example problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They use their own encoding scheme and explain it in the text for completeness. Any similarities between their scheme and other binary representations of real numbers are to be expected, and if you understand them, it will help understand what they are doing here too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GAs don't have to use this sort of scheme for real numbers. Often they use standard floating-point representation as separate elements. But the authors want a binary code here to help teach some basic concepts, which is what they mean by &quot;on the other hand, binary code is good for pedagogy&quot;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-10-21T18:32:41.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8564" PostTypeId="2" ParentId="7755" CreationDate="2018-10-21T19:08:09.623" Score="6" Body="&lt;p&gt;The most straightforward solution is to simply make every action &quot;legal&quot;, but implementing a consistent, deterministic mapping from potentially illegal actions to different legal actions. Whenever the PPO implementation you are using selects an illegal action, you simply replace it with the legal action that it maps to. Your PPO algorithm can then still update itself as if the illegal action were selected (the illegal action simply becomes like... a &quot;nickname&quot; for the legal action instead).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the situation you describe:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;2 actions (0 and 1) are always available&lt;/li&gt;&#xA;  &lt;li&gt;2 actions (2 and 3) are only available when the internal_state == 0&lt;/li&gt;&#xA;  &lt;li&gt;1 action (4) is only available when the internal_state == 1&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In cases where &lt;code&gt;internal_state == 0&lt;/code&gt;, if action &lt;code&gt;4&lt;/code&gt; was selected (an illegal action), you can always swap it out for one of the other actions and play that one instead. It doesn't really matter (theoretically) which one you pick, as long as you're consistent about it. The algorithm doesn't have to know that it picked an illegal action, whenever it picks that same illegal action in the future again in similar states it will consistently get mapped to the same legal action instead, so you just reinforce according to that behaviour.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The solution described above is very straightforward, probably the most simple to implement, but of course it... &quot;smells&quot; a bit &quot;hacky&quot;. A cleaner solution would involve a step in the Network that sets the probability outputs of illegal actions to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and re-normalizes the rest to sum up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; again. This requires much more care to make sure that your learning updates are still performed correctly though, and is likely a lot more complex to implement on top of an existing framework like Tensorforce (if not already somehow supported in there out of the box).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For the first &quot;solution&quot;, I wrote above that it does not matter &quot;theoretically&quot; how you choose you mapping. I absolutely do expect your choices here will have an impact on learning speed in practice though. This is because, in the initial stages of your learning process, you'll likely have close-to-random action selection. If some actions &quot;appear multiple times&quot; in the outputs, they will have a greater probability of being selected with the initial close-tor-andom action selection. So, there will be an impact on your initial behaviour, which has an impact on the experience that you collect, which in turn also has an impact on what you learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I certainly expect it will be beneficial for performance if you can include input feature(s) for the &lt;code&gt;internal_state&lt;/code&gt; variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If some legal actions can be identified that are somehow &quot;semantically close&quot; to certain illegal actions, it could also be beneficial for performance to specifically connect those &quot;similar&quot; actions in the &quot;mapping&quot; from illegal to legal actions if you choose to go with that solution. For example, if you have a &quot;jump forwards&quot; action that becomes illegal in states where the ceiling is very low (because you'd bump your head), it may be better to map that action to a &quot;move forwards&quot; action (which is still kind of similar, they're both going forwards), than it would be to map it to a &quot;move backwards&quot; action. This idea of &quot;similar&quot; actions will only be applicable to certain domains though, in some domains there may be no such similarities between actions.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-10-21T19:30:39.290" LastActivityDate="2018-10-21T19:30:39.290" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8565" PostTypeId="2" ParentId="8518" CreationDate="2018-10-21T19:37:14.457" Score="-2" Body="&lt;p&gt;The term 'size' isn't applicable to the tensor output of a network or network.  These are the qualities.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rank &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; that defines the rank of each tensor instance in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^N$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Ranges of the indices to the dimensions from &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Ranges of the scalar values that comprise the tensor instance &amp;mdash; If they are discrete rather than real (approximated by floating point or fixed point numbers), then the range is the description of the permissible ordinal values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The question may be referring to this last quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The imposition of a penalty for values that are in the range of the numeric type used as the output of the last activation function but not in the allowable range of output for the desired trained function works in a limited way.  It skews the output distribution with respect to the natural distribution of possible learning states and therefore can easily interfere with convergence quality or speed or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of techniques that map natural output distribution with constrained ranges, but it must be done without skewing the distribution upstream from the technique used, to avoid negatively impacting favorable convergence properties of the artificial network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One simple case that can be described here is when the number of possible output states is in the set of &lt;span class=&quot;math-container&quot;&gt;$2^i$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$i \in$&lt;/span&gt; { positive integers }.  In such a case, the final layer of the network can be &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; threshold activation functions with 1 or -1 as possible output values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case, the ordinal then becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$o = \sum_{n=0}^i \frac {y_n + 1} {2},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$y_n$&lt;/span&gt; is the output from activation function index &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-10-21T19:37:14.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8566" PostTypeId="2" ParentId="8560" CreationDate="2018-10-21T22:49:58.640" Score="4" Body="&lt;p&gt;Here are a few guidelines, inspired by the deep learning specialization course, to choose the size of the mini-batch: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have a small training set, use batch gradient descent (m &amp;lt; 200)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In practice:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Batch mode: long iteration times&lt;/li&gt;&#xA;&lt;li&gt;Mini-batch mode: faster learning &lt;/li&gt;&#xA;&lt;li&gt;Stochastic mode: lose speed up from vectorization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The typically mini-batch sizes are 64, 128, 256 or 512.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, in the end, make sure the minibatch fits in the CPU/GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at the paper &lt;a href=&quot;https://arxiv.org/abs/1206.5533&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Recommendations for Gradient-Based Training of Deep Architectures&lt;/a&gt; (2012) by Yoshua Bengio.&lt;/p&gt;&#xA;" OwnerUserId="14749" LastEditorUserId="2444" LastEditDate="2019-07-13T13:35:18.027" LastActivityDate="2019-07-13T13:35:18.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8567" PostTypeId="1" CreationDate="2018-10-22T00:26:09.770" Score="2" ViewCount="40" Body="&lt;p&gt;Suppose there are sensors which supply numerical metrics. If a metric goes above or below a healthy threshold, an event (alert) is raised. Metrics depend on each other in one way or another (we can learn the dependencies via ML algorithms) so when the system is in alerting state only one or a few metrics will be a root cause and all others will be simply consequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can assume there is enough historical metric data available, to learn dependencies but there are just a few historical malfunctions. Also, when malfunction happens there is no one to tell what was the root cause, the algorithm should learn how to detect root causes by itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithms can be used to detect the root cause events in the situation above? Are there any papers available on the subject?&lt;/p&gt;&#xA;" OwnerUserId="19218" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:49.810" LastActivityDate="2020-05-29T13:47:49.810" Title="Detect root cause across many event occurrences" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8568" PostTypeId="2" ParentId="8368" CreationDate="2018-10-22T01:14:17.550" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes, intuition says that RNNs like LSTM or GRU will work better in your case, because predicted values might depend on input patterns corresponding to much earlier time intervals.&lt;/li&gt;&#xA;&lt;li&gt;There is no reason to create samples shifted by a single measurement because many of the samples will contain pretty much the same information for your model. A viable approach is to shift by sample size. Keeping some overlap between samples is feasible as well. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that as a general rule for processing audio data, it makes sense to convert raw audio data into vectors representing audio spectrum before feeding it into LSTM RNN (see this video for e.g. &lt;a href=&quot;https://www.coursera.org/learn/nlp-sequence-models/lecture/sjiUm/speech-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.coursera.org/learn/nlp-sequence-models/lecture/sjiUm/speech-recognition&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Batch size is different from sample or window size in your case.&lt;/p&gt;&#xA;" OwnerUserId="19218" LastActivityDate="2018-10-22T01:14:17.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8570" PostTypeId="1" AcceptedAnswerId="8574" CreationDate="2018-10-22T05:15:20.537" Score="3" ViewCount="166" Body="&lt;p&gt;Why in every aspect are we now considering Neural Networks as an Artificially Intelligent entity/program?&lt;/p&gt;&#xA;" OwnerUserId="19220" LastEditorUserId="1581" LastEditDate="2018-10-22T19:54:03.040" LastActivityDate="2018-10-22T19:54:03.040" Title="Why are Neural Networks considered Artificially Intelligent?" Tags="&lt;neural-networks&gt;&lt;research&gt;&lt;artificial-consciousness&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8571" PostTypeId="2" ParentId="8560" CreationDate="2018-10-22T05:39:30.603" Score="4" Body="&lt;p&gt;&lt;em&gt;From the blog &lt;a href=&quot;https://machinelearningmastery.com/gentle-introduction-mini-batch-gradient-descent-configure-batch-size/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Gentle Introduction to Mini-Batch Gradient Descent and How to Configure Batch Size&lt;/a&gt; (2017) by Jason Brownlee&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h2&gt;How to Configure Mini-Batch Gradient Descent&lt;/h2&gt;&#xA;&lt;p&gt;Mini-batch gradient descent is the recommended variant of gradient&#xA;descent for most applications, especially in deep learning.&lt;/p&gt;&#xA;&lt;p&gt;Mini-batch sizes, commonly called “batch sizes” for brevity, are often&#xA;tuned to an aspect of the computational architecture on which the&#xA;implementation is being executed. Such as a power of two that fits the&#xA;memory requirements of the GPU or CPU hardware like 32, 64, 128, 256,&#xA;and so on.&lt;/p&gt;&#xA;&lt;p&gt;Batch size is a slider on the learning process.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Small values give a learning process that converges quickly at the cost of noise in the training process.&lt;/li&gt;&#xA;&lt;li&gt;Large values give a learning process that converges slowly with accurate estimates of the error gradient.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Tip 1: A good default for batch size might be 32.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="2444" LastEditDate="2020-10-31T10:02:24.587" LastActivityDate="2020-10-31T10:02:24.587" CommentCount="1" CommunityOwnedDate="2021-03-23T10:00:53.083" ContentLicense="CC BY-SA 4.0" />
  <row Id="8574" PostTypeId="2" ParentId="8570" CreationDate="2018-10-22T06:10:12.770" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why in every aspect we are now considering artificial intelligence as a neural network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&quot;We&quot; aren't. It is generally due to reporting by media sources that simplify science and technology news.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of AI is somewhat fluid, and also contentious at times, but in research and scientific circles it has not changed to the degree that AI=NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What has happened is that research into neural networks has produced some real advances in the last decade. These advances have taken research-only issues such as very basic computer vision, and made them good enough to turn into technology products that can be used in the real world on commodity hardware. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These &lt;em&gt;are&lt;/em&gt; game-changing technology advances, and they use neural networks internally. Research and development using neural networks is still turning out new and improved ideas, so has become a very popular area to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of research using neural networks is also research into AI. Aspects such as computer vision, natural language processing, control of autonomous agents are generally considered parts of AI. This has been simplified in reporting, and used by hyped-up marketing campaigns to label pretty much any product with a NN in it as &quot;Artificial Intelligence&quot;. When often it is more correctly statistics or &lt;em&gt;Data Science&lt;/em&gt;. Data Science is another term which has been somewhat abused by media and technology companies - the main difference between use of AI and Data Science is that Data Science was a new term, so did not clash with pre-existing uses of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rest of AI as a subject and area of study has not gone away. Some of it may well use neural networks as part of a toolkit to build or study things. But not all of it, and even with the use of NNs, the AI part is not necessarily the neural network.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-10-22T06:10:12.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8576" PostTypeId="2" ParentId="3101" CreationDate="2018-10-22T08:30:39.993" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www-isl.stanford.edu/~cover/papers/paper2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thomas Cover&lt;/a&gt; and &lt;a href=&quot;http://www.inference.org.uk/itila/book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;David MacKay&lt;/a&gt; proofed the capacity of a perceptron. This proof was &lt;a href=&quot;https://arxiv.org/abs/1810.02328&quot; rel=&quot;nofollow noreferrer&quot;&gt;recently extended&lt;/a&gt; to Neural Networks. All of them provide upper bounds for the number of parameters needed to learn something.&lt;/p&gt;&#xA;" OwnerUserId="19227" LastActivityDate="2018-10-22T08:30:39.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8578" PostTypeId="1" CreationDate="2018-10-22T09:31:20.077" Score="3" ViewCount="95" Body="&lt;p&gt;The Levenshtein algorithm and some ratio and proportion may handle this use case.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Based on the pre-defined sequence of statements, such as &quot;I have a dog&quot;, &quot;I own a car&quot; and many more, I must determine if an another input statement such as &quot;I have a cat&quot; is the same or how much percentage does the input statement is most likely equal to the pre-defined statements.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For Example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Predefined statements: &quot;I have a dog&quot;, &quot;I own a car&quot;, &quot;You think you are smart&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Input statements and results:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have a dog - 100% (because it has exact match), I have a cat - ~75% (because it was almost the same except for the animal, think - ~10% (because it was just a small part of the third statement), bottle - 0% (because it has no match at all)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The requirement is that TensorFlow be used rather than Java, which is the language I know, so any help with what to look at to get started would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My plan was to use the predefined statements as the train_data, and to output only the accuracy during the prediction, but I don't know what model to use. Please, guide me with the architecture and I will try to implement it.&lt;/p&gt;&#xA;" OwnerUserId="19229" LastEditorUserId="2444" LastEditDate="2019-04-25T10:21:19.467" LastActivityDate="2019-04-25T10:21:19.467" Title="Which model should I use to determine the similarity between predefined sentences and new sentences?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8580" PostTypeId="1" AcceptedAnswerId="8581" CreationDate="2018-10-22T13:29:12.913" Score="1" ViewCount="47" Body="&lt;p&gt;Consider I have a 3 layers neural network.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input Layer containing 784 neurons.&lt;/li&gt;&#xA;&lt;li&gt;Hidden layer containing 100 neurons.&lt;/li&gt;&#xA;&lt;li&gt;Output layer containing 10 neurons.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My objective is to make an OCR and I used MNIST data to train my network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I gave the network an input taken from an image, and the values from the output neurons are the next:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$0: 0.0001$&lt;/span&gt;  &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$1: 0.0001$&lt;/span&gt;  &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$2: 0.0001$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$3: 0.1015$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$4: 0.0001$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$5: 0.0002$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$6: 0.0001$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$7: 0.0009$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$8: 0.001$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$9: 0.051$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When the network returns this output, my program will tell me that he identified the image as number 3.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now by looking at the values, even though the network recognized the image as 3, the output value of number 3 was actually very low: &lt;span class=&quot;math-container&quot;&gt;$0.1015$&lt;/span&gt;. I am saying very low, because usually the highest value of the classified index is as close as 1.0, so we get the value as 0.99xxx.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;May I assume that the network failed to classify the image, or may I say that the network classified the image as 3, but due to the low value, the network is not certain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I right thinking like this, or did I misunderstand how does the output actually works?&lt;/p&gt;&#xA;" OwnerUserId="7983" LastEditorUserId="7983" LastEditDate="2018-10-22T13:34:53.473" LastActivityDate="2018-12-21T19:02:15.453" Title="How much extra information can we conclude from a neural network output values?" Tags="&lt;neural-networks&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8581" PostTypeId="2" ParentId="8580" CreationDate="2018-10-22T16:27:02.793" Score="1" Body="&lt;p&gt;From the values you have provided I can easily guess your output layer has the sigmoid (do clarify!) activation function. For sigmoid activation function this can be a quite normal occurrence. Also maybe the number of training epochs is not high enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case you have mentioned of 0.99 is generally in the case of the output being subjected to a softmax probability function. Although 0.99 is still achievable using sigmoid activation, it will depend on your network hyper-parameters and in general the data. If data is very easily separable the sigmoid will generally give very high contrasting difference among classes. Also if the hidden network has ReLu it becomes easy to provide contrasting class scores due to its huge scale difference compared to a sigmoid activation in the hidden layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point here to note is &lt;strong&gt;contrasting difference&lt;/strong&gt; among classes, because your sigmoid might give 0.99 for the correct class while 0.9 for other classes, which is undesirable.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-10-22T16:46:12.073" LastActivityDate="2018-10-22T16:46:12.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8583" PostTypeId="2" ParentId="8578" CreationDate="2018-10-22T23:36:01.280" Score="2" Body="&lt;p&gt;If this is a simple syntax comparison, neural networks is not the best way to achieve this.&lt;/p&gt;&#xA;&lt;p&gt;If it's semantic comparison, then you can take a look at models used in the &lt;a href=&quot;https://nlp.stanford.edu/projects/snli/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNLI&lt;/a&gt; dataset for example.&lt;/p&gt;&#xA;&lt;p&gt;From your question, it looks like just a syntax comparison.&lt;/p&gt;&#xA;&lt;p&gt;Consider the 2 sentences :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;She likes playing guitar&lt;/p&gt;&#xA;&lt;p&gt;She likes listening guitar&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The 2 sentences have almost the same words, but the meaning is different.&lt;/p&gt;&#xA;&lt;p&gt;Now consider these 2 sentences :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The bird is taking a bath in the fountain&lt;/p&gt;&#xA;&lt;p&gt;Birdie wash himself with water in public place&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These 2 sentences have almost no words in common, but the meaning are very similar.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So if your use case need to return a high-score for the first example, give up neural network (it is possible, but pointless).&lt;/p&gt;&#xA;&lt;p&gt;If your use case need to return a high-score in the second example, take a look at &lt;a href=&quot;https://nlp.stanford.edu/projects/snli/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNLI&lt;/a&gt; leaderboards, there is plenty of models that can works.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-22T23:36:01.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8585" PostTypeId="2" ParentId="5885" CreationDate="2018-10-23T05:50:55.123" Score="0" Body="&lt;p&gt;With the increase of both unit capacity turbines and size of wind farm, the safe operatio wind farm has received growing attention. In ma factors that affect the safe operation of wind far being struck by lightning is an important aspect. intelligent lightning monitoring system is used fo surveillance of wind turbine generators, which ca real-time and accurate monitoring of the lightnin current waveforms, amplitude, time of occurrenc number of lightning strokes and all the other imp parameters of lightning thus providing an effectiv monitoring and analysis tool to quickly locate fa location on wind turbine generator and cause of malfunction and a theoretical basis for the desig lightning protection system of wind turbines. Thi examines the principle and main methods of the turbine generator-matching lightning monitoring and combined with the specific research project designs and implements a high-precision and multifunctional intelligent lightning monitoring sy based on the theory of Rogowski coils.&lt;/p&gt;&#xA;" OwnerUserId="17941" LastActivityDate="2018-10-23T05:50:55.123" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8586" PostTypeId="1" CreationDate="2018-10-23T07:48:10.637" Score="5" ViewCount="1600" Body="&lt;p&gt;I am training a deep neural network. There is a constraint on an output value of the network. (e.g. Output has to be between 0 and 180) I think some possible solutions are using sigmoid,tanh activation at the end of the layer. I wonder if there are better ways to put constraints on the output value of a neural network.&lt;/p&gt;&#xA;" OwnerUserId="19245" LastEditorUserId="10135" LastEditDate="2018-10-28T18:42:20.733" LastActivityDate="2018-10-30T01:33:45.730" Title="Putting constraints on output of deep neural network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8587" PostTypeId="2" ParentId="8309" CreationDate="2018-10-23T10:54:16.193" Score="1" Body="&lt;p&gt;When we add a single layer with a non-linear activation function, right after the application of the activation function, a new basis function is found (for that neuron), which is some combination of the weights and biases, which acts as a new way to view or analyze the feature sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With an increasingly deeper network, we keep finding representations, which are new basis vectors of the combination of the previous layer's weights and biases, that is, higher-level representations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they're error-free, they'll give better performances, but if small errors creep in earlier basis vectors, the error increases through depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice analogy is the Taylor series, where &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt;, and so on, are the basis vectors for estimating the function in 1D.&lt;/p&gt;&#xA;" OwnerUserId="19027" LastEditorUserId="2444" LastEditDate="2020-03-11T00:36:36.360" LastActivityDate="2020-03-11T00:36:36.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8588" PostTypeId="1" AcceptedAnswerId="8657" CreationDate="2018-10-23T13:28:43.277" Score="3" ViewCount="206" Body="&lt;p&gt;I am creating a dataset made of many images which are created by preprocessing a long time series. Each image is an array of (128,128) and the there are four classes. I would like to build a dataset similar to the MNIST in scikit-learn.database  but I have no idea how to do it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My aim is to have something that I can call like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(x_train, y_train), (x_test, y_test) = my_data()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should I save them as figures? or as csv? &#xA;Which is the best way to implement this? &lt;/p&gt;&#xA;" OwnerUserId="19251" LastActivityDate="2018-10-28T22:58:38.247" Title="Best way to create an image dataset for CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8591" PostTypeId="1" CreationDate="2018-10-23T22:22:35.900" Score="3" ViewCount="113" Body="&lt;p&gt;I am developing an image search engine. The engine is meant to retrieve wrist watches based on the input of the user. I am using SIFT descriptors to index the elements in the database and applying Euclidean distance to get the most similar watches. I feel like this type of descriptor is not the best since watches have a similar structure and shape. Right now, the average difference between the best and worst matches is not big enough (15%)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been thinking of adding colour to the descriptor, but I'd like to hear other suggestions.&lt;/p&gt;&#xA;" OwnerUserId="19266" LastEditorUserId="2444" LastEditDate="2019-04-23T23:08:14.800" LastActivityDate="2019-05-24T00:02:55.470" Title="What is a good descriptor for similar objects?" Tags="&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;feature-selection&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8594" PostTypeId="1" CreationDate="2018-10-24T06:48:54.563" Score="2" ViewCount="953" Body="&lt;p&gt;The intelligence of the human brain is said to be a strong factor leading to human survival. The human brain functions as an overseer for many functions the organism requires. Robots can employ artificial intelligence software, just as humans employ brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to the human brain, we are prone to make mistakes. However, artificial intelligence is sometimes presented to the public as perfect. Is artificial intelligence really perfect? Can AI also make mistakes?&lt;/p&gt;&#xA;" OwnerUserId="19275" LastEditorUserId="2444" LastEditDate="2020-02-08T12:49:25.183" LastActivityDate="2020-02-08T17:39:13.440" Title="Can artificial intelligence also make mistakes?" Tags="&lt;philosophy&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8595" PostTypeId="1" CreationDate="2018-10-24T07:21:53.587" Score="5" ViewCount="421" Body="&lt;p&gt;Can artificial intelligence (or machine learning) applications or agents be hacked, given that they are software applications, or are all AI applications secure? &lt;/p&gt;&#xA;" OwnerUserId="19220" LastEditorUserId="2444" LastEditDate="2019-10-11T22:20:43.297" LastActivityDate="2019-10-11T22:20:43.297" Title="Can artificial intelligence applications be hacked?" Tags="&lt;security&gt;&lt;ai-safety&gt;&lt;adversarial-ml&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="3" ClosedDate="2020-03-14T13:59:19.750" ContentLicense="CC BY-SA 4.0" />
  <row Id="8596" PostTypeId="1" CreationDate="2018-10-24T08:45:56.553" Score="1" ViewCount="38" Body="&lt;p&gt;Assume that we have a labeled dataset with inputs and outputs, where the output range is &lt;span class=&quot;math-container&quot;&gt;$\left[0, 2\right]$&lt;/span&gt;, but the majority of outputs is in &lt;span class=&quot;math-container&quot;&gt;$\left[0, 1\right]$&lt;/span&gt;. Should one adopt some kind of over- or undersampling approach after compartmentalising the output space to make the dataset more balanced? That would usually be done in classification, but does it apply to regression problems, too? &#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="16901" LastEditorUserId="16901" LastEditDate="2018-10-24T09:41:14.460" LastActivityDate="2018-10-24T09:41:14.460" Title="Unbalanced dataset in regression rather than classification" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8597" PostTypeId="1" CreationDate="2018-10-24T10:32:36.837" Score="2" ViewCount="231" Body="&lt;p&gt;I am writing an app, where when a ball is shot from a canon it is supposed to land in a hole that is on a given distance. The ball is supposed to land between the distance of the beginning of the hole and the end of the hole. The size of the hole is 4m and the size of the ball is 0.4m. My problem is that I am not sure how to write the fitness function for this. The place where the ball falls should be close to this interval of [D, D+3.6], where D is the distance of the hole. If anyone could give me a hint on how to approach this problem, I would be grateful.&lt;/p&gt;&#xA;" OwnerUserId="19283" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:14.827" LastActivityDate="2020-05-29T13:48:14.827" Title="Fitness function in genetic algorithm based on an interval" Tags="&lt;genetic-algorithms&gt;&lt;java&gt;&lt;fitness-functions&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8598" PostTypeId="2" ParentId="8597" CreationDate="2018-10-24T11:49:56.283" Score="3" Body="&lt;p&gt;Genetic algorithms work best when given a scalar fitness value that increases smoothly, so that you can compare two population members regardless of whether they failed or succeeded at the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That usually requires you to analyse the problem, and come up with a measure that would improve as an individual gets closer to solving a task. It generally helps if you score better for an individual if they solve a problem &quot;better&quot;. You want to avoid simple boolean success/fail metrics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple measure of how well an individual has done is to use the absolute distance it was away from a &quot;perfect&quot; shot (where the middle of the ball hits the middle of the hole). The only issue with this is that a perfect shot scores 0, whilst a miss scores 2+, and you want the best result to have the highest fitness. This can be fixed simply, take the negative of the absolute distance:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$F = -|D_{hole} - D_{ball}|$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$D_{hole}$&lt;/span&gt; - and &lt;span class=&quot;math-container&quot;&gt;$D_{ball}$&lt;/span&gt; are horizontal distances from origin to centre of each object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no requirement for fitness score to be positive. This will score &lt;span class=&quot;math-container&quot;&gt;$F \lt -2$&lt;/span&gt; for a miss, &lt;span class=&quot;math-container&quot;&gt;$-2 \lt F \lt 0$&lt;/span&gt; for hit, and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for a perfect shot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For GAs you don't have to care as much about differentiability or gradient of the fitness function, provided it gives reasonably good ranks between quality of individuals. So there is no point using e.g. a squared error metric here, although you could if you wished.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-24T15:07:37.783" LastActivityDate="2018-10-24T15:07:37.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8599" PostTypeId="2" ParentId="8595" CreationDate="2018-10-24T14:41:51.447" Score="1" Body="&lt;p&gt;Everything can be hacked. The solutions found by artificial intelligence can be much more efficient than human solutions, but they can also be confused because of the diversity and immensity of details that our mind possesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence models bring us more secure solutions, but nothing is 100% safe when we talk about information security. There are ways to improve security, hinder invasions and attacks, but every system has flaws.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps, in the future (things of my imagination) we will have an artificial superintelligence ahead of the human, which may be one of the greatest challenges of invasion of history, but until then .. just my imagination.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-10-24T14:41:51.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8600" PostTypeId="2" ParentId="8595" CreationDate="2018-10-24T14:42:25.350" Score="5" Body="&lt;p&gt;To answer your question, it really depends on the purpose of the Artificial Intelligence program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, 4Chan has hacked a number of &quot;Artificial Intelligent&quot; bots, most notably was Microsoft's Twitter bot Tay. The general purpose of the bot was to parse what was tweeted at it and respond in kind, learning and evolving with each and every interaction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within 24 hours, 4Chan had corrupted Tay beyond repair, by teaching it racist and sexist terminology, ironic memes, sending it to shitpost tweets, and otherwise attempting to alter its output so much so that Microsoft had to remove it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the flaw with Tay was that it accepted any input, and learned off of that exclusively, without any interaction from the developers. Other bots have similar features, but they have checks in place that require human intervention to determine what is &quot;quality&quot; information to learn, and what is &quot;bad&quot; information to learn as to not pollute the global knowledge base of the bot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are just two examples of how Artificial Intelligence can be &quot;hacked&quot;, but it ultimately comes down to how the programs are implemented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention in one of your comments about Cellphone Artificial Intelligence such as Siri, and whether this technology can be hacked. The answer is - not really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Siri learns based off of her global interactions - with limited user input allowed. You can ask Siri how to pronounce a name. When she pronounces it incorrectly, you can say to her &quot;Siri that's now how you pronounce that&quot;. And she will provide you with a limited set of options of how you pronounce that name, and you have to choose which option sounds the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was no option to allow a user to give Siri &quot;bad&quot; information, as she already populates the results for you, and you have to teach her based off of her list of options. To give Siri bad input, you would have to have access to Siri's global learning base, which we do not have access to, and alter how she accepts human interactions within the program - which would never happen due to too many moving parts within the iPhone update process, and would be caught before you were able to deploy your update.&lt;/p&gt;&#xA;" OwnerUserId="19290" LastActivityDate="2018-10-24T14:42:25.350" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8604" PostTypeId="1" CreationDate="2018-10-24T17:37:16.983" Score="1" ViewCount="82" Body="&lt;p&gt;I understand the concept of convolution.&#xA;Let's say that my input dimension is &lt;code&gt;3 x 10 x 10&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if I say that I will have 20 activation maps and a filter size of 5, I will end up with 20 different filters for my layer, each with the dimension of &lt;code&gt;(3 x 5 x 5)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My output will therefor be &lt;code&gt;(20 x ? x ?)&lt;/code&gt;. I Put a &quot;?&quot; there, because it obviously depends on the filter stride etc.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now I wanted to implement deconvolution but I am stuck at the following point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the following questions, let's assume that the input size for the deconvolution is &lt;code&gt;(5 x 8 x 8)&lt;/code&gt;, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If we think about a filter in 3 dimensions. Can I choose any depth for the filter?&lt;/li&gt;&#xA;&lt;li&gt;How would the effect of the amount of filters (amount of activation maps) work with deconvolution? Do I only have one filter?&lt;/li&gt;&#xA;&lt;li&gt;How does the input depth (5) come into play. Would the output depth be equal to        &lt;br&gt; (filter depth) * (input depth) ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am trying to find the symmetry to forward convolution but I do not understand how to use the amount of activation maps in deconvolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very thankful for any help.&lt;/p&gt;&#xA;" OwnerUserId="16353" LastActivityDate="2018-10-24T17:37:16.983" Title="neural network deconvolution filters" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8605" PostTypeId="1" AcceptedAnswerId="8636" CreationDate="2018-10-24T18:29:00.283" Score="4" ViewCount="1202" Body="&lt;p&gt;I am applying a Double DQN algorithm to a highly stochastic environment where some of the actions in the agent's action space have very similar &quot;true&quot; Q-values (i.e. the expected future reward from either of these actions in the current state is very close). How can I still ensure that the algorithm gets these values (and their relative ranking) right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To provide a little more info: What happens currently is that the loss function on the Q-estimator decreases rapidly in the beginning, but then starts evening out. The Q-values also first converge quickly, but then start fluctuating around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried increasing the batch size which I feel has helped a bit. What did not really help, however, was decreasing the learning rate parameter in the loss function optimizer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which other steps might be helpful in this situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this question is misleading (since reinforcement learning is an &lt;em&gt;approximate&lt;/em&gt; dynamic programming method), please just let me know. I mean the algorithm usually does find an only slightly sub-optimal solution to the MDP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 3&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer some questions brought up in the comments:&lt;br&gt;&#xA;(1) I do indeed have full control over the MDP including the reward function which in my case is sparse (0 until the terminal episode). The &quot;true&quot; Q-values I know from an analytical solution to the problem I am trying to solve using reinforcement learning.&lt;br&gt;&#xA;(2) The rewards are the same for &lt;em&gt;identical&lt;/em&gt; transitions. However, the rewards vary for any given state and action taken therein.&lt;br&gt;&#xA;(3) The environment is only stochastic for a part of the actions in the action space. I.e. the action chosen by the agent influences the stochasticity of the rewards.&lt;/p&gt;&#xA;" OwnerUserId="18109" LastEditorUserId="18109" LastEditDate="2018-10-27T17:04:09.417" LastActivityDate="2018-10-27T18:22:29.393" Title="Ensure convergence of DDQN if true Q-values are very close" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8606" PostTypeId="2" ParentId="6878" CreationDate="2018-10-24T18:43:54.140" Score="1" Body="&lt;p&gt;It would be hard to tell if you don't provide what kind of data/problem you are working on, but LDA works well when data that are grouped in gaussian blobs surrounding centroids while vanilla SVM works well when the data is almost linearly separable and naive bayes works well when your features are relatively independent of each other.&lt;/p&gt;&#xA;" OwnerUserId="19270" LastActivityDate="2018-10-24T18:43:54.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8607" PostTypeId="1" AcceptedAnswerId="8632" CreationDate="2018-10-24T20:05:07.130" Score="6" ViewCount="5022" Body="&lt;p&gt;I am training a Multilayer Neural Nets with 146 samples (97 for training set, 20 for validation set and 29 for testing set). I am using:&#xA;automatic differentiation,&#xA;SGD method,&#xA;fixed learning rate + momentum term,&#xA;logistic function,&#xA;quadratic cost function,&#xA;L1 and L2 regularization technique,&#xA;adding some artificial noise 3%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I used L1 or L2 regularization technique my problem (overfitting problem) got worst.&#xA;i tried different values for lambdas (the penalty parameter  0.0001, 0.001, 0.01, 0.1, 1.0 and 5.0). After 0.1 i just killed my ANN. The best result that i took it was using 0.001 (but it is worst comparing the one that i didnt use regularization technique). The graph represent the error functions for different penalty parameters and also a case without using L1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NwBl2.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NwBl2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the accuracy&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MTgRD.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MTgRD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can be?&#xA;Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="19268" LastActivityDate="2020-05-14T08:06:51.863" Title="Why L1/L2 regularization technique did not improve my accuracy?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;overfitting&gt;&lt;regularization&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8608" PostTypeId="2" ParentId="8595" CreationDate="2018-10-24T22:44:26.587" Score="1" Body="&lt;p&gt;Perhaps what you are looking for is the notion of &lt;a href=&quot;https://blog.openai.com/adversarial-example-research/&quot; rel=&quot;nofollow noreferrer&quot;&gt;adversarial attacks on machine learning systems&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2018-10-24T22:44:26.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8609" PostTypeId="1" CreationDate="2018-10-24T22:46:13.930" Score="1" ViewCount="197" Body="&lt;p&gt;I am so much curious about how do we see(with eyes ofc) and detect things and their location so quick. Is the reason that we have huge gigantic network in our brain and we are trained since birth to till now and still training. &#xA;basically I am saying , are we trained on more data and huge network? is that the reason?&#xA;or what if there's a pattern for about how do we see and detect object.&#xA;please help me out, maybe my thinking is in wrong direction.&#xA;what I wanna achieve is an AI to detect object in picture in human ways.thanks.&lt;/p&gt;&#xA;" OwnerUserId="18459" LastActivityDate="2018-11-07T16:21:49.540" Title="Can we make Object Detection as human eyes+brain do?" Tags="&lt;human-like&gt;&lt;human-inspired&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8610" PostTypeId="1" CreationDate="2018-10-24T23:39:02.690" Score="2" ViewCount="25" Body="&lt;p&gt;I am preparing the Bus movement dataset for deep learning (ANN/CNN/RNN) analysis for congestion events detection. This is an extension to my original question, which can be located at '&lt;a href=&quot;https://ai.stackexchange.com/questions/8364/deep-learning-model-training-and-processing-requirement-for-traffic-data&quot;&gt;Deep learning model training and processing requirement for Traffic data&lt;/a&gt;' for the general approach on this topic, and this question is for preparing the dataset and need your kind advice on it. In simple words, I would like to know the state of congestion for a bus route at a specific point in time (year). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my entities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Routes&lt;/li&gt;&#xA;&lt;li&gt;Bus_Scheduled_Routes&lt;/li&gt;&#xA;&lt;li&gt;Bus_Route_Stops&lt;/li&gt;&#xA;&lt;li&gt;Bus_Trips (operational_date, Vehicle_id, Trip_id, Vehicle_Position_Update, Trip_stop_id, passenger_loaded, velocity, direction, scheduled_arrival_time, actual_arrival_time)&lt;/li&gt;&#xA;&lt;li&gt;Events (human and non-human induced)&lt;/li&gt;&#xA;&lt;li&gt;Points of Interests (POIs)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I have these entities based data and I create a view that gives me a time reference based view comprising of week(52), day(7), Vehicle_id, Trip_id,  Stop/Position_update_interval, speed, acceleration, velocity, scheduled_arrival_time, actual_arrival_time. Will this view be recommended to start training the model? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, how can I integrate the human / non-human induced events and Points of Interests (POIs) data into this view so my model can predict better results? To generalize the model data will be 'time segment / trips time (seasons), location component (Bus Routes and Stops), Arrival time / trip completion time'. I am thinking to add an attribute for human/non-human induced events as type tying with the 'time segment' and adding the POIs as type and vicinity to the stop points. What are your recommendation about it? Thanks in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="18959" LastActivityDate="2018-10-24T23:39:02.690" Title="Data / model preparation for spatio-temporal deep-learning analysis for traffic congestion events detection" Tags="&lt;deep-learning&gt;&lt;time&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8611" PostTypeId="1" CreationDate="2018-10-24T23:42:44.743" Score="4" ViewCount="1356" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic&quot; rel=&quot;noreferrer&quot;&gt;Fuzzy logic&lt;/a&gt; seemed like an active area of research in machine learning and data mining back when I was in grad school (early 2000s). Fuzzy inference systems, fuzzy c-means, fuzzy versions of the various neural network and support vector machine architectures were all being taught in grad courses and discussed in conferences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I've started paying attention to ML again (~2013), Fuzzy Logic seems to have dropped off the map completely and its absence from the current ML landscape is conspicuous given all the AI hype. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Was this a case of a topic simply falling out of fashion, or was there a specific limitation of fuzzy logic and fuzzy inference that led to the topic being abandoned by researchers? &lt;/p&gt;&#xA;" OwnerUserId="2306" LastActivityDate="2020-03-04T21:33:01.420" Title="Why did fuzzy logic fall out of fashion?" Tags="&lt;machine-learning&gt;&lt;fuzzy-logic&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8613" PostTypeId="1" CreationDate="2018-10-25T07:50:28.110" Score="2" ViewCount="458" Body="&lt;p&gt;I saw when browsing we can use data augmentation for creating a dataset for face recognition. The augmented images may include inverted, tilted or distorted faces. Do the model detect the face from the inverted image. When I tried my model cant able to detect any inverted or tilted faces.&lt;/p&gt;&#xA;" OwnerUserId="17058" LastEditorUserId="17058" LastEditDate="2018-10-26T06:32:22.170" LastActivityDate="2018-10-26T07:40:16.527" Title="How can we use data augmentation for creating data set for face recognition and will the inverted faces on augmented images detected?" Tags="&lt;image-recognition&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8615" PostTypeId="1" CreationDate="2018-10-25T10:12:34.790" Score="2" ViewCount="74" Body="&lt;p&gt;Which possibilities exist to evaluate the visual reasoning capabilities of neural networks in the field of image recognition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there methods to measure the ability of machine reasoning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or something more specific: Is it possible to measure if a network understood the concept of a car / a cat / a human without using the classification accuracy.&lt;/p&gt;&#xA;" OwnerUserId="16634" LastEditorUserId="17221" LastEditDate="2018-10-26T19:14:17.280" LastActivityDate="2018-10-26T21:43:52.173" Title="How to measure the reasoning capabilities of neural networks" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;reasoning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8616" PostTypeId="1" CreationDate="2018-10-25T13:28:23.067" Score="2" ViewCount="46" Body="&lt;p&gt;I loaded a neural network model trained with Caffe by other people in OpenCV. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model should detect the presence of a car in a single parking spot outputting the probability of it being free/occupied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model was trained with images all belonging to the same parking area, taken at different hours of day and with different light conditions. Images were taken by different cameras but the cameras are all of the same model (raspberry cameras).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to run the model with a few images some of them taken from their dataset and other downloaded from google.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images taken from their dataset are correctly classified while the ones taken from google are not correctly classified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is it possible to deploy a NN model trained with images all coming from a single parking area in another parking area? Is not such a model for parking detection occupancy supposed to generalize independently from the location where training images have been taken?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know about an already existing trained model that works good please let me know.&lt;/p&gt;&#xA;" OwnerUserId="12959" LastEditorUserId="12959" LastEditDate="2018-10-26T07:54:35.637" LastActivityDate="2018-10-26T07:54:35.637" Title="Influence of location on a Neural Network trained for parking detection occupancy" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;&lt;training&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8617" PostTypeId="2" ParentId="8616" CreationDate="2018-10-25T13:44:16.467" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is: is it possible to deploy a NN model trained with images all coming from a single parking area in another parking area? Is not such a model for parking detection occupancy supposed to generalize independently from the location where training images have been taken?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generalisation in machine learning has limits, and for complex systems such as computer vision, it is very easy to exceed them. This is why the state-of-the-art systems use huge amounts of training data which is often augmented with automated variations, plus trained on for many days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want some factor in your model to generalise well, you should:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Train with variations in that factor&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Test with variations in that factor that have not already been seen in the training set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the factor you wish to have the model generalise in, then it looks like (a) has not been done, and that you have just done (b) and found that the first attempt has failed an important test that you care about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect the trained model you have used will generalise well by time of day, type of vehicle and some other details. It does not appear to generalise to a new location because it was trained on only one location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have limited training data, you may want to look at approaches to fine-tuning against more general image classifiers (such as the many ImageNet-trained classifiers, which are available to download). However, even then you should train with a few varieties of parking space, or at the very least with the spaces that you care about (if you do the latter then it probably won't generalise to other spaces, but you won't care).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It may also be worth seeing what you can do to transform the input that you are working with to have similar view over the new parking area that the training images have. Using a transform to align the outline of the new area with the old one, and possibly some colour correction could help a lot.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-25T13:51:54.437" LastActivityDate="2018-10-25T13:51:54.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8618" PostTypeId="2" ParentId="8609" CreationDate="2018-10-25T16:11:19.863" Score="0" Body="&lt;p&gt;If you have studied about Convolution Neural Network, you probably know how present day object detection algorithms works. Basically, an Artificial Neural Network tries to mimic the way our brain might work. So, CNN is probably the way our brain works to perform object detection, trying to recognize each small parts at each stage, that slowly grows to compile into the larger object to be recognised.  However since, we are not restricted to data from just 2D plane, so the processing might be more complicated. We all learn things after being trained to do so. You could never have recognized a particular fruit if you have never seen it. The way we store the information, we gain, is different, that helps us to remember/recognise objects after only a few look. Until, we have more research that provides more details into the working of our brain, we are all good with Artificial Neural Network. Honestly, we are still far away from achieving general artificial intelligence. Now, if you want to implement an object detection algorithm, CNN is your only way.&lt;/p&gt;&#xA;" OwnerUserId="14688" LastActivityDate="2018-10-25T16:11:19.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8619" PostTypeId="1" CreationDate="2018-10-25T16:53:16.913" Score="11" ViewCount="288" Body="&lt;p&gt;One of the corner stones of The Selfish Gene (Dawkins) is the spontaneous emergence of replicators, i.e. molecules capable of replicating themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this been modeled &lt;em&gt;in silico&lt;/em&gt; in open-ended evolutionary / artificial life simulations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Systems like Avida or Tierra explicitly &lt;strong&gt;specify&lt;/strong&gt; the replication mechanisms; other genetic algorithm/genetic programming systems explicitly &lt;strong&gt;search for&lt;/strong&gt; the replication mechanisms (e.g. to simplify the von Neumann universal constructor)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links to simulations where replicators emerge from a primordial digital soup are welcome.&lt;/p&gt;&#xA;" OwnerUserId="19319" LastEditorUserId="1671" LastEditDate="2018-10-29T00:53:39.447" LastActivityDate="2018-12-02T02:00:55.160" Title="Spontaneous emergence of replicators in Artificial Life" Tags="&lt;genetic-algorithms&gt;&lt;theory&gt;&lt;genetic-programming&gt;&lt;self-replication&gt;" AnswerCount="3" CommentCount="6" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8620" PostTypeId="1" CreationDate="2018-10-25T17:02:23.657" Score="3" ViewCount="1252" Body="&lt;p&gt;I am trying to figure out how to use a reinforcement learning algorithm, if possible, as a &quot;black box&quot; to play a game. In this game, a player has to avoid flying birds. If he wants to move, he has to move the mouse on the display, which controls the player position by applying a force. The player can choose any position on the display for the mouse. A human can play this game. To get a sense what needs to be done, have a look at &lt;a href=&quot;https://www.youtube.com/watch?v=k6xiqI60ztA&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Youtube video&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using an ANN, which takes as input the information of the game (e.g. positions, speeds, radius, etc.) and outputs a move. However, I am unlikely to ever record enough training data to train the network properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I was thinking that a RL algorithm, like Q-learning, would be more suited for this task. However, I have no clue how to apply Q-learning to this task. For example, how could the coder possibly know what future reward another move will bring?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In this case, the player has infinitely many actions. How would I apply RL to this case?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is RL a good approach to solving this task?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a handy Java library which would allow me to use a RL algorithm (as a block-box) to solve this problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there alternatives?   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="2444" LastEditDate="2019-02-15T16:41:41.513" LastActivityDate="2019-02-15T16:53:25.833" Title="How do I apply reinforcement learning to a game with infinitely many actions?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;applications&gt;&lt;java&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8621" PostTypeId="2" ParentId="8620" CreationDate="2018-10-25T18:44:20.877" Score="1" Body="&lt;p&gt;This looks like the kind of game that a Reinforcement Learning algorithm could learn to play. If you are learning from the game objects directly (as opposed to wanting to have the agent learn from viewing the screen), then it should be &lt;em&gt;relatively&lt;/em&gt; straightforward to do so - by which I mean that it will involve some new concepts, a little maths, and maybe take you a few weeks to learn enough and run enough experiments on the AI to the point where you have something good. But at least you should not need a cluster of GPU servers to run your learner.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I have no clue how to apply Q-learning to this task. For example, how could the coder possibly know what future reward another move will bring?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is a major part of RL - the algorithms that are part of RL all solve this issue. Therefore the coder does not need to know how to calculate future reward, just an immediate reward. For the example game, it looks like you can grant a reward of &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; for every frame where the player does not crash. An RL algorithm that can solve this environment should figure out through &quot;trial and error&quot; which actions will keep the game running. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q Learning achieves this by keeping track of estimates of future value and updating them across time steps - if action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; consistently leads to a later state &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt; with a high value, then Q learning will update its estimate of the value of &lt;span class=&quot;math-container&quot;&gt;$Q(s_1,a_1)$&lt;/span&gt; to be high as well. The estimates start very poorly, but updates work their way back through time steps as the agent learns, and provided things don't go wrong, eventually it will learn that one action will lead to higher value than another, and it be able to consistently choose it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is such a basic question that if you want to use RL, even if you find a library that implements the whole thing for you, that you clearly need to study some foundations of RL. There are a few different places you can do that. I can suggest two that I have used:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The book by Sutton &amp;amp; Barto. &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCL lectures by David Silver&lt;/a&gt;. These include videos on YouTube.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both are online, free and comprehensive introductions to the subject. You can find shorter, more direct introductions online if you search for e.g. &quot;Q Learning Tutorial&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be prepared to spend hours just learning enough about the topic that you understand how to structure your data and interface with the game in order to create a learning environment for your AI.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;In this case, the player has infinitely many actions. How would I apply RL to this case?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a good question. Some algorithms are better than others. Q Learning cannot work with infinitely many actions. However, you could choose a discrete set of mouse points - e.g. maybe 64 - at different angles and distances from the player (red dot) and try and get the values of those. That would allow you to try with Q learning, and is worth considering because RL that properly solves large action spaces is harder to understand and work with. You could look into the DQN algorithm for this. In short, the DQN algorithm uses history of actions and their immediate results to train a neural network to estimate the values of each action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to solve large action spaces properly, then you need to be looking into Policy Gradient methods. Two popular and effective PG algorithms are A3C and DDPG. These are tougher to learn and implement. They use neural networks to learn multiple functions, one of which is the &lt;em&gt;policy&lt;/em&gt; - this directly selects which action to take, given a current state/observation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Is RL a good approach to solving this task?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is definitely possible to use RL to do what you want. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Is there a handy Java library which would allow me to use a RL algorithm (as a block-box) to solve this problem?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I could not find one on a brief search. To get started, you might want to look at &lt;a href=&quot;https://github.com/chen0040/java-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;java-reinforcement-learning&lt;/a&gt; which implements some of the algorithms from Sutton &amp;amp; Barto, so would be a good starting point. Most of the other Java packages I found covered basics and learning material, but would not be plug-and-play for your game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do find a library, you will need to have learned enough basic RL to be able to follow how to use it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;4&quot;&gt;&#xA;  &lt;li&gt;Are there alternatives?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One alternative that might work well for your game environment, and be a little easier to understand and implement would be to combine a policy function (like a neural network) with an evolutionary algorithm that tries out variations of the policy. The go-to algorithm here is probably NEAT, and it is capable of solving simple game agents effectively. Your game should be a reasonable fit for NEAT because the scoring system of survival time makes for a good fitness function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that could really be worth trying before diving into the (very hard to understand) A3C and DDPG algorithms is the &lt;a href=&quot;https://www.aaai.org/Papers/ICML/2003/ICML03-068.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross-Entropy Method&lt;/a&gt;. It is a really simple RL agent that has some similarities to genetic algorithms (it selects &quot;best&quot; items, but instead of from the population, it is from random behaviour of the same individual), but doesn't always scale too well. However, it might work well for your game. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-02-15T16:53:25.833" LastActivityDate="2019-02-15T16:53:25.833" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8622" PostTypeId="2" ParentId="8619" CreationDate="2018-10-25T20:48:30.143" Score="2" Body="&lt;p&gt;Although difficult to prove a negative, I don't think that this has been done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most advanced simulations of low-level features are not capable of scaling to simulate large enough populations at large enough time scales where scientific consensus claims that this has happened in reality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although you say that you are not directly interested in chemistry, but some abstract substrate, I am using chemistry as an example of the challenge. That is because creating a simplified substrate with enough rich emergent behaviour is non-trivial. The chemical elements essentially have rules about how they combine into larger physical structures (via different bonding mechanisms) and only roughly a dozen types of atom are involved. It's actually reasonably simple and tractable at the lowest level. The problems come from the multiple scales of structure - building &quot;unit&quot; molecules (DNA/RNA bases, protein peptides, lipids, sugar bases etc), creating polymers from those units, interactions between polymers, physical structures built and torn down by those interactions, each of which exhibits more complex behaviour. This structural hierarchy is likely required for any self-replicating machinery that is not simply being fed the higher-level units directly. In your question you want to find self-replication that is emergent, not designed . . . so feeding in these higher-level units would probably count as cheating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We probably don't have the computational power to properly simulate even the &lt;a href=&quot;https://en.wikipedia.org/wiki/Miller%E2%80%93Urey_experiment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miller-Urey experiment&lt;/a&gt; which is far from self-replication - chemical simulations &lt;em&gt;in silico&lt;/em&gt; are limited to things like protein folding calculations, and these are far from real-time. Inside just a single bacterial cell getting ready to divide, proteins are produced and fold by the hundreds every second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that has been done is to create a self-replicating machine in &lt;a href=&quot;https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life#Self-replication&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conway's Game of Life called &quot;Gemini&quot;&lt;/a&gt;. This was designed, not spontaneously created. However, it would have a very low but non-zero chance of spontaneously being created with random initialisation. It would be a very fragile replicator though, any mutation or collision with other active elements would likely break it. The experiment of attempting to randomly/spontaneously create Gemini is not computationally feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is likely that any physical system simple enough to be considered &quot;primordial soup&quot; yet rich enough to express replicating units is going to require a few layers of construction before you get to see those units. These layers need to be built up combinatorially, and odds of this happening spontaneously in a small experiment with limited computation appear to be low. You need to bear in mind the &lt;em&gt;extremely large&lt;/em&gt; computation that would effectively have been done by an order of &lt;span class=&quot;math-container&quot;&gt;$10^{30}$&lt;/span&gt; molecules that can interact at very fast rates in parallel (compared to rate at which these same interactions can be simulated in current CPUs), with processing times of the order &lt;span class=&quot;math-container&quot;&gt;$10^8$&lt;/span&gt; years. It is mainly conjecture that this is enough to create an Initial Darwinian Ancestor - it is basically a logical extrapolation to the theory of evolution, following the Occam's Razor principle of looking for simplest compatible explanation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-27T17:26:08.267" LastActivityDate="2018-10-27T17:26:08.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8623" PostTypeId="2" ParentId="8591" CreationDate="2018-10-25T21:14:19.367" Score="2" Body="&lt;p&gt;How to develop a program that can sort images by similarity is similar to the problem of how to develop a program that can sort words by how similar they look.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&#xA;&quot;theory&quot; is more similar to &quot;teoryyy&quot; than to &quot;abc&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What determines similarity of two words or images are these factors:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;how many parts are common to both images&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how many parts are new in either image&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how many parts are missing in either image&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how many parts are zoomed in/out or stretched.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how many parts are displaced&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There may be some other rules. By combining these rules you can explain how similarity recognition works. This is at the core of intelligence.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-10-25T21:14:19.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8624" PostTypeId="1" CreationDate="2018-10-25T21:56:45.493" Score="2" ViewCount="121" Body="&lt;p&gt;The BACON algorithm, introduced by Pat Langley and Herbert Simon etc. were meant to automate scientific discovery -- producing causal explanation to variations in given data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was found, in particular, to have been able to &quot;discover&quot; the laws of planetary motion, but there were criticism concerning whether or not its achievement could be consider true scientific discovery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one I remember was that--the data given to the algorithm are pre-processed by human operators so as to include mostly relevant factors of the world. For example, the distances of planets from the sun was given, showing that the human operators implicitly understood the importance of this variable for the laws of planetary motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there were other significant criticism of the algorithm--either in general as a automaton of discovery or concerning its particular feat with planetary motion.&lt;/p&gt;&#xA;" OwnerUserId="1321" LastActivityDate="2018-10-25T21:56:45.493" Title="What were the criticisms of the BACON algorithm?" Tags="&lt;cognitive-science&gt;&lt;automation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8625" PostTypeId="1" CreationDate="2018-10-26T01:45:21.197" Score="2" ViewCount="190" Body="&lt;p&gt;I have successfully figured out how the minimax algorithm works for a game like chess, where a game tree is used, and you assign a value to the terminal nodes and propagate that value up the tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to represent this algorithm mathematically? If so, how would I go about showing it? &lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="2444" LastEditDate="2019-12-20T16:11:10.097" LastActivityDate="2019-12-20T16:11:10.097" Title="Is there a way of representing the minimax algorithm mathematically?" Tags="&lt;math&gt;&lt;minimax&gt;&lt;chess&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8626" PostTypeId="1" AcceptedAnswerId="8630" CreationDate="2018-10-26T05:35:11.980" Score="2" ViewCount="2479" Body="&lt;p&gt;I've to train a neural network using microphone data (wav files), accelerometer sensor data and light sensor data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now the approach I thought was to convert all data into images and combine them into a single image and train my neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach was to convert wav files into arrays and combine them along with sensor data and train my neural net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my approaches correct or is there a better way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions/ideas are welcome.&lt;/p&gt;&#xA;" OwnerUserId="19332" LastActivityDate="2018-10-26T12:36:09.363" Title="How to combine input from different types of data sources?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8627" PostTypeId="2" ParentId="8625" CreationDate="2018-10-26T05:59:07.247" Score="2" Body="&lt;p&gt;A simple google search gives plenty of results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a look at the entry in wikipedia for &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimax&lt;/a&gt; it has mathematical representations as well as some basic pseudocode and tree representations to help grasp the concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proving it would be a matter of going through the regular methods of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_proof&quot; rel=&quot;nofollow noreferrer&quot;&gt;mathematical proof&lt;/a&gt; and would probably be a bit complicated.  It is far easier to discuss this by way of graphs.&lt;/p&gt;&#xA;" OwnerUserId="9413" LastActivityDate="2018-10-26T05:59:07.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8628" PostTypeId="2" ParentId="8613" CreationDate="2018-10-26T07:40:16.527" Score="1" Body="&lt;p&gt;This is the purpose of rotational augmentation. In fact you should augment your images at different angle as well than just inverting it. This will make your model angle agnostic. &lt;/p&gt;&#xA;" OwnerUserId="17980" LastActivityDate="2018-10-26T07:40:16.527" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8629" PostTypeId="2" ParentId="8586" CreationDate="2018-10-26T11:06:46.973" Score="2" Body="&lt;h2&gt;There are many ways of constraining the network's output.&lt;/h2&gt;&#xA;&lt;p&gt;Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by 180 to adapt to your output. Since the layer is part of the optimization process, gradients will be learned correctly.&lt;/p&gt;&#xA;&lt;p&gt;However, there are a few issues with it, namely sigmoid is not a linear curve, and is quite steep around 0. Maybe you can consider something like a ReLU, but with a maximum as well. This makes your output linearly between [0,180], and is also learned during optimization.&lt;/p&gt;&#xA;&lt;p&gt;You could also just clip the output without any activation layers. This is simple and works, but the optimization process sees no difference between outputting 180 or 180000. This &lt;em&gt;can&lt;/em&gt; lead to problems, depending on your problem.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-10-26T11:06:46.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8630" PostTypeId="2" ParentId="8626" CreationDate="2018-10-26T11:46:37.157" Score="1" Body="&lt;p&gt;&lt;strong&gt;Handling Multiple Input Types&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiple input types are common as learning technology moves from academic labs and open source examples into the real world.  The design process includes the below four steps, which related directly to this question.  There are other steps that don't centrally related not mentioned here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First Step &amp;mdash; List Inputs in Most Usable Form&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of this question, there are three in the list.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Monophonic or multichannel audio data as an array of samples or sample vectors acquired at a fixed sampling rate&lt;/li&gt;&#xA;&lt;li&gt;Acceleration data as an array of samples in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n \in \{1, 2, 3\}$&lt;/span&gt; acquired at a fixed sample rate.&lt;/li&gt;&#xA;&lt;li&gt;Visible light or IR detection data as an array of samples of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; channels at a fixed sample rate.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Fixed sample rate is listed with each type.  Variable sample rates pose additional burden on learning.  If any of the three are acquired with a variable sample rate, they can be converted to fixed sample rate arrays provided there is a time stamp or a time-since-last-sample channel available with each sample vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variable to fixed sample rate conversion can be accomplished easily via ffmpeg, programmatically via its library, or through any equivalent software designed for this purpose.  The acceleration and visible light signals can also be adapted for ffmpeg's input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a specific dynamic characteristic must be preserved, finer control over the interpolation can be obtained using cubic spline algorithms.  Such algorithms were originally written in FORTRAN or C and have been ported to Java and Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second Step &amp;mdash; When Spectral Data is Practically Required&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With time series data, the phenomena being learned is often largely a function of vibration frequencies and amplitudes.  This is almost always the case with audio data.  The exception is when transient detection is the primary teller of the conditions being measured rather than more continuous sounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A note is spectral in description.  A knock is a transient and both spectra and actual rise characteristics are indicative of source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presenting digital samples from the transducer circuit (in this case the microphone diaphragm's position in response to pressure waves) to the learning system directly is not a best practice.  Unconverted vibration information forces the learning system to learn transformations for which optimized algorithms already exist.  This slows learning and generally pushes complexity beyond the learning system's capacity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the spectral features best characterize the phenomena being learned, as in hums, screeches, vowel sounds, music, horns, or beeps, the best practice is to convert the samples in the time domain to frequency domain data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is usually done using one of the forms of the FFT (fast Fourier transform) algorithm in combination with a RMS (root mean square) calculation of amplitude and windowing.  Windowing attenuation forms, such as the Hamming window or the cosine window, are available in most packages that have FFT capabilities.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sample rate is also important.  Nyquist's sample rate criteria must be met by the A-to-D converter (in the sound card or instrumentation circuitry) to ensure that the spectral range includes the best indicators of the phenomena to be learned.  In machine learning, such indicators are called features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, audio or vibration information is presented to a learning system or sub-system as spectral frequencies and amplitudes.  Vibration recognition systems and speech to text systems generally execute FFTs (with RMS and windowing) on the raw input.  In human ears, the conversion to spectral information is mechanical, using the tapering geometry of the ear's cochlea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There also exists algorithms that can directly develop spectral information from variable sample rate data to amplitudes and frequencies, combining two transformations into one for efficiency of computing resources and additional input accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary of this section, whether light, IR, acceleration, or other data is treated as audio and converted to spectral information is based on whether&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The signals are vibrations that have rises and falls within specific frequency ranges like hums, notes, and speech or&lt;/li&gt;&#xA;&lt;li&gt;They are transients (as with tapping the mechanical package on which the accelerometer is mounted) or&lt;/li&gt;&#xA;&lt;li&gt;They are binary in nature (as in the case of linear encoder signals).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Third Step &amp;mdash; When Topology of Types&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The topological orientation of the input source types to one another is key to feeding the information into any learning system.  These are the two pure extremes.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The signals from the source types are describing the same physical or data mining property.&lt;/li&gt;&#xA;&lt;li&gt;The signals are measuring independent properties that together describe the state of the system being measured.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the case of (1), a normalized sample from one type of input channel is, from a learning point of view, indistinguishable from a normalized sample from another.  This would be the case if the accelerometer is measuring the same vibration that the microphone is measuring.  This is the less usual case, but is more common when the data is for study of vibration to project mechanical or materials failures in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;With Background Clarified, the Q&amp;amp;A Becomes Clearer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question asked this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How [does one best] combine input from different types of data sources?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Whether the source data is case (1) or case (2) or some midpoint between them is key to answering this question.  Samples from different source types that measure single phenomena can be phase adjusted, spectrally adjusted, and otherwise normalized to present a single data type at the input.  Merging into a single serial data stream would be the optimal pre-processing for training in case (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More often, the topological orientation of the different input source types conforms to case (2) above.  In that case, different input types should be presented in parallel to the learning system inputs.  The merging would be counterproductive in this case, for a reason that will be clarified in the next section below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid sparsity in input data, which creates additional training burden, the sample rates should be equalized for all channels of all data source types.  This may require conversion of the sample rate, which can be done using the same techniques (above) for converting from variable to fixed sample rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Images of Data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question stated this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Right now, the approach I [was considering is] to convert all data into images and combine them into a single image and train my neural network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The attempt to normalize and combine into one input stream is the correct approach if (1) above is the case.  However, presenting data as images requires the learning system restore the data to a less redundant, more feature encoded form, so conversion to images is not recommended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If (2) above is the case, the provision of unique and property-independent data through the same input tensors requires the artificial network sort the input samples by type, which can be done with recurrent or attention based artificial networks, but it is unnecessary.  Here's the rule.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the signals from different sources are measuring independent properties to which learning will be applied, do not merge the input streams.  Instead, present them in parallel to the learning system so that the learning system does not need to also learn how to discriminate sample types.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The reason topology is mentioned in the subtitle of this section is that there is a third case, (3), which is a hybrid of (1) and (2) as hinted above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many excellent detection and control systems process the input channels &lt;strong&gt;in relation to one another&lt;/strong&gt; prior to the actual learning system components.  This pre-processing takes advantage of known relationships in the data and the topology of signals in phase space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phase space of two or three signals can be visualized in 2D or 3D where each channel is assigned to a Cartesian coordinate system's axis and the time domain is not represented by any axis.  Mechanical fault detection can be optimized using phase space analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this more complex case (3) is in the human recognition of both consonant and vowel sounds and various other non-speech audio phenomena.  After the cochlear conversion to spectra, the transient information (as from a 'k' sound or a click) and tone information (as from an 'e' or a whistle) is presented to speech recognition nets in the brain in an integrated form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This decreases the learning time newborns take to learn voice and household sounds and probably improved evolutionary viability in times of global stress or local predatory activity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In automated vehicles and smart robotics, the same is true.  Some sounds, accelerations, and visible light changes are transient and some are vibrations.  Studying the topology of the surface created by the input signal types and their spectra in phase space provides much information.  Armed with the results of such studies allows excellence in the preparation of input to a learning system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fourth Step &amp;mdash; Normalizing and Standardizing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the above is complete, understanding how to present the data to the input layer of an artificial network becomes clearer.  The goal of this design step is to have relatively equal ranges for the components of each input tensor, whether presented serially or in parallel depending on the above criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing so reduces the burden on the learning system.  All signals are presented at full strength but within the clipping range of the signal.  This is a general principle and is important for both data center (batch) learning and real time learning and adaptive systems.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-26T12:36:09.363" LastActivityDate="2018-10-26T12:36:09.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8632" PostTypeId="2" ParentId="8607" CreationDate="2018-10-26T15:28:36.860" Score="4" Body="&lt;p&gt;Regularization is one of the important prerequisites for improving the reliability, speed, and accuracy of convergence, but it is not a solution to every problem.  Irregularity in data is only one of many root causes for slow or otherwise inadequate learning results, and as the results in the question indicates, it can reduce reliability, speed, or accuracy in some cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those who are new to this topic, these are a few other root causes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Complexity beyond the capacity of the computing machinery to model adequately&lt;/li&gt;&#xA;&lt;li&gt;Insufficient number of examples for training&lt;/li&gt;&#xA;&lt;li&gt;Poor distribution alignment between data sets (training, testing, validation, production)&lt;/li&gt;&#xA;&lt;li&gt;Saturation of back propagation values in floating point&lt;/li&gt;&#xA;&lt;li&gt;Outliers cause by error in example creation or labeling&lt;/li&gt;&#xA;&lt;li&gt;Local optimization minima in the loss function combined with insufficient stochastic injection in SGD&lt;/li&gt;&#xA;&lt;li&gt;Leaning to heavily on artificial network convergence and neglecting using other known algorithms as part of an overall system architecture&lt;/li&gt;&#xA;&lt;li&gt;Using activation functions or hyper-parameters that are not well tuned to the example set or the model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the case of regularization, L1 and L2 have known properties that have been proven in theorem form to guarantee convergence in fewer examples or epochs, but they rely on very specific models toward which convergence is targetted.  L1 and L2 are not always beneficial for polynomial models.  Trying to make them so tends to lead to over-fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting trained networks lack the needed generalization to produce reliable behavior upon receiving data outside the training set.  Irregularities might not always be caught during test and verification.  Deployed systems can exhibit the signs of over-fitting over time because the network learned time-specific features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overfitting can be prevented using dropout regularization, another technique first proposed in &lt;a href=&quot;http://www.jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Dropout: A Simple Way to Prevent Neural Networks from Overfitting&lt;/em&gt; by Srivastava, et al. 2014&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another two things to look at ...&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the logistic function is the best choice and for which layers?  It seems to be down-trending.  More people are using ReLU and its softer varieties because of higher performance shown in a number of problem domains and models.&lt;/li&gt;&#xA;&lt;li&gt;Is the noise injection clean in that it is a good pseudo noise source and is it configured properly with whatever SGD methods and libraries being used.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am training a Multilayer Neural Nets with 146 samples (97 for training set, 20 for validation set and 29 for testing set). I am using: automatic differentiation, SGD method, fixed learning rate + momentum term, logistic function, quadratic cost function, L1 and L2 regularization technique, adding some artificial noise 3%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question indicated the best results for the current training set and model.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The best result was obtained using a lambda of 0.001 without regularization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's the starting point for improving further.  Since complete automated convergence is not yet something libraries and frameworks provide, the engineer normally has to converge on the combination of techniques and settings that best help the network's convergence.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-10-26T15:28:36.860" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8633" PostTypeId="2" ParentId="8611" CreationDate="2018-10-26T17:15:20.210" Score="1" Body="&lt;p&gt;Fuzzy logic is not down trending.  It's a common architecture selection for systems that require the representation of uncertainty in changing rules.  The domains include work-flow control, aeronautics, chemical plant engineering, automated defense of cyber-attack, building systems, and business intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In natural language processing, it's up-trending under other names.  Semantic nets are a logical structure with strengths assigned between nodes of the net based on meta rules, which is reminiscent of early fuzzy logic software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the rule might be that a word that ends with -ly before a word associated with action may be an adverb modifying a verb in English, but it is not always the case.  The rule's application is not smooth, meaning non-probabilistic.  It is fuzzy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no way to create a grammar for common speech or writing in any natural language.  Language is dynamic and as practically non-deterministic as wind vectors.  That's why grammar is a down-trending word among linguists and people working in NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cognition is expected to require fuzzy valuations to the black and white nature of early production systems developed at MIT, Cambridge, the U.S. Navy, Yale U, and other locations in Eurasia.  People develop rules in their cortex and they rise and drop in probability when decisions are made after the rule seemed to have worked or didn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The logic that works to prove a math theorem may require discrete inference, but people are smart when they can weigh alternatives and apply rules in unique ways to partial data and limited experience.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-10-26T17:15:20.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8634" PostTypeId="2" ParentId="8615" CreationDate="2018-10-26T21:43:52.173" Score="1" Body="&lt;p&gt;What do you mean by &quot;understood the concept of a car&quot; ? A classification algorithm need not know that cars are used to transport humans or anything else it's used for. It only needs to know what combination of features we decide to label as a car. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe you meant we could use some other metric for understanding that is not classification accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a kind of Catch-22 situation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we want to classify a car we can reformulate the task to learning what set of a combination of features do &lt;em&gt;us humans&lt;/em&gt; decide to call a car. What labels we want to assign a combination of features is arbitrary to a degree. It preferably has to be  consistent for inference but an NN can learn a dataset assigned completely random features to a 100% accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, perhaps in an alternative universe we label the front-view of a car as 'B' and the rear view as 'A' and the notion/label of 'car' is nonexistent: even though we are ostensibly referencing the same object in physical space, we can assign it different labels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It follows that to measure if a model has properly interpolated what combination of features we like to call car, we ourselves must be able list out such features; and if we were able to do that, we would not not need the process of learning/teaching algorithms in the first place!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A much more interesting subject is testing the extrapolative powers of neural networks- something much more proximate to what we canonically refer to as 'reasoning'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's quite hard to measure this in static environments such as the classification of images, but the ability to extrapolate is more apparent in reinforcement learning settings where the model requires us to train it for times order of magnitudes above the time for needed for humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole notion of pristine reasoning in machine learning breaks down when it comes to induction(see &lt;a href=&quot;https://plato.stanford.edu/entries/sufficient-reason/#WhatSuffReas&quot; rel=&quot;nofollow noreferrer&quot;&gt;sufficient reason&lt;/a&gt;). Thus, we resort to probabilities that can be roughly translated to &quot;this line of reasoning yielded the correct result 99.9% of the time, so it would make sense that we keep using it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the idea of perfect-as-possible induction has been mathematically formulated except it isn't computable :( &lt;a href=&quot;https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solomonoff Induction&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9271" LastActivityDate="2018-10-26T21:43:52.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8636" PostTypeId="2" ParentId="8605" CreationDate="2018-10-27T09:14:52.813" Score="5" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$Q^*(s, a)$&lt;/span&gt; denote the &quot;true&quot; &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value for a state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt;, i.e. the values that we're hoping to learn to approximate using a neural network that outputs &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you describe is basically that you have situations where &lt;span class=&quot;math-container&quot;&gt;$Q^*(s, a_1) = Q^*(s, a_2) + \epsilon$&lt;/span&gt; for some very small value &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a_1 \neq a_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important thing to note is that the &quot;true&quot; &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt; values are directly determined by the reward function, which essentially encodes your objective. If your reward function is constructed in such a way that it results in the situation described above, that means the reward function is essentially saying: &quot;it doesn't really matter much whether we pick action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;, there is some difference in returns but it's a tiny difference, we hardly care.&quot; &lt;strong&gt;The objective implied by the reward function in some sense contradicts what you describe in the question that you want.&lt;/strong&gt; You say that you care (quite a lot) about a tiny difference in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values, but normally the &quot;extent to which we care&quot; is proportional to the difference in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values. So, if there is only a tiny difference in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values, we should really only care a tiny bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, with tabular RL algorithms (like standard &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning), we do expect to be able to eventually converge to the truly correct solutions, we expect to eventually be capable of correctly ranking different actions even when the differences in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values are small. When you enter function approximation (especially neural networks), this story changes. It's pretty much in the name already; function &lt;strong&gt;approximation&lt;/strong&gt;, we cannot guarantee being able to do better than just approximating. So, 100% &lt;strong&gt;ensuring&lt;/strong&gt; that you'll be able to learn the correct rankings is not going to be feasible. There may be some things that can help though:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you are &quot;in control&quot; of your environment and its reward function (meaning, you implemented it yourself and/or can modify it if you like), modifying the reward function would probably be the single most effective solution to your problem. As I described above, a reward function that results in tiny differences in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values essentially encodes that you only care about those difference to a tiny (negligible) extent. If that's not the case, if you actually do care about those tiny differences... well, you can try changing the reward function to accentuate those tiny differences. &lt;strike&gt;For example, you could try simply multiplying all rewards by &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;, then all differences in returns are also multipled by &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;. Note that this does mean you get larger values everywhere (larger losses, larger gradients, etc.), so it would require changing hyperparameters (like learning rate) accordingly, and could destabilize things. But at least it will emphasize those small differences in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values.&lt;/strike&gt; (probably won't work, see Neil's comment)&lt;/li&gt;&#xA;&lt;li&gt;Increasing batch size. You already mentioned this yourself, but I'll also explain why it can help. A major issue in your problem is that you described the environment to be highly stochastic. This means you get a large amount of &lt;strong&gt;variance&lt;/strong&gt; in your observed returns, and therefore also in your updates. If the true differences in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values are very small, these differences will get dominated and become completely invisible due to high variance in observations. You can smooth out this variance by increasing the batch size.&lt;/li&gt;&#xA;&lt;li&gt;Decreasing learning rate. You also mentioned this in the question, and that it didn't really help. I suppose a high learning rate early on can help to learn more quickly, but decreasing it later on can help to take more &quot;careful&quot;, small update steps that don't take &quot;jumps&quot; of a magnitude greater than the true difference in &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;-values.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Distributional Reinforcement Learning&lt;/strong&gt;: algorithms like &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, DQN, Double DQN, etc., they all learn what we can call &lt;em&gt;values&lt;/em&gt;, or &lt;em&gt;scalars&lt;/em&gt;, or &lt;em&gt;point estimates&lt;/em&gt;. Given a state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt;, such a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value represents the expected returns obtained by executing &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and continuing with a greedy policy afterwards. In stochastic environments, it is possible that &lt;strong&gt;the true expected value &lt;span class=&quot;math-container&quot;&gt;$Q^*(s, a)$&lt;/span&gt; never coincides with any conrete observation at the end of an actual trajectory&lt;/strong&gt;. For example, suppose that in some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; has a probability of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; of giving a reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and instantly terminating afterwards, but also has a probability of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; of giving a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and instantly terminating. Then, we have &lt;span class=&quot;math-container&quot;&gt;$Q^*(s, a) = 0.5$&lt;/span&gt;, but we &lt;strong&gt;never&lt;/strong&gt; actually observe a return of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;; we always observe returns of either &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. When comparing the output of the learned &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function (which should hopefully be around &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; in such a situation) to any observed trajectories, we'll almost always have a non-zero error and take update steps that keep bouncing around the optimal solution, rather than converging to the optimal solution. Since you mentioned that you see &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values fluctuating around, you may have observed this. &lt;strong&gt;Excessive fluctuations around &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value estimates will likely make it difficult to get a consistent, stable ranking when the optimal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values that you are fluctuating around are themselves very close to each other&lt;/strong&gt;. There is a class of algorithms (extensions of DQN mostly) referred to as &lt;strong&gt;distributional RL&lt;/strong&gt;, which aim to address this issue by learning a full &lt;strong&gt;distribution&lt;/strong&gt; of returns we expect to be capable of observing, rather than the single-point estimates that &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values essentially are. I believe the first major paper on this topic was the following: &lt;a href=&quot;https://arxiv.org/abs/1707.06887&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1707.06887&lt;/a&gt;. There have been multiple follow-up papers too, the most recent of which I believe (but may be wrong here) to be this one: &lt;a href=&quot;https://arxiv.org/abs/1806.06923&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1806.06923&lt;/a&gt;. In general, I do know Marc Bellemare and Rémi Munos at least have done quite some work on that, so you could search for those names too.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-10-27T18:22:29.393" LastActivityDate="2018-10-27T18:22:29.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8637" PostTypeId="1" AcceptedAnswerId="8642" CreationDate="2018-10-27T13:19:19.173" Score="4" ViewCount="1803" Body="&lt;p&gt;What is the difference between learning agents and other types of agents? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what ways learning agents can be applied? Do learning agents differ from deep learning&lt;/p&gt;&#xA;" OwnerUserId="18950" LastEditorUserId="2444" LastEditDate="2019-06-22T17:01:57.463" LastActivityDate="2019-06-22T17:01:57.463" Title="What is the difference between learning and non-learning agents?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;theory&gt;&lt;intelligent-agent&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8638" PostTypeId="2" ParentId="8637" CreationDate="2018-10-27T13:39:03.680" Score="1" Body="&lt;p&gt;A learning agent is an agent which has a problem generator, a learning element, a critic and a performance element. ProbGen just creates different instances of a problem w.r.t. a domain and a set of parameters in jurisdiction. A learning element is responsible for making efficient choices and performance element chooses actions for the selected choice. It also changes some of the knowledge base. A critic is basically a metric to tell which is a better choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning is a bit complicated than that. Then again, let's simplify. You see, deep learning is a specific arrangement or subsection of neural networks that helps the learning agent learn faster. That's as simply as I could explain it. What we do is that we arrange the layers in a neural network with a purpose and densely pack it to make it more task specific. Obviously, I am leaving out a lot of implementation details but for them, I am afraid, you need to look up a tutorial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are they used in AI?&#xA;That's problem specific. Learning agents are everywhere from spam detection to roombas to artificial dog pets. Deep learning on the other hand is currently focused on user pattern recognition and personalization in every front. Please let me know if you need me to expand upon this answer. &lt;/p&gt;&#xA;" OwnerUserId="17799" LastActivityDate="2018-10-27T13:39:03.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8640" PostTypeId="1" AcceptedAnswerId="8733" CreationDate="2018-10-27T15:45:28.137" Score="4" ViewCount="91" Body="&lt;h3&gt;Problem setting&lt;/h3&gt;&#xA;&lt;p&gt;We have to do a binary classification of data given a training dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, where most items belong to class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and some items belong to class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, so the classes are heavily imbalanced.&lt;/p&gt;&#xA;&lt;h3&gt;Approach&lt;/h3&gt;&#xA;&lt;p&gt;We wanted to use a GAN to produce more samples of class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, so that our final classification model has a nearly balanced set to train.&lt;/p&gt;&#xA;&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&lt;p&gt;Let's say that the data from both classes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are very similar. Given that we want to produce synthetic data with class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; with the GAN, we feed real &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; samples that we have into the discriminator alongside with generated samples. However, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are similar. It could happen that the generator produces an item &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, that would naturally belong to class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. But since the discriminator has never seen class-&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; items before and both classes are very close, the discriminator could say that this item &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is part of the original data that was fed into the discriminator. So, the generator successfully fooled the discriminator in believing that an item &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is part of the original data of class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is actually part of class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If the GAN keeps producing items like this, the produced data is useless, since it would add heavy noise to the original data, if combined.&lt;/p&gt;&#xA;&lt;p&gt;At the same time, let's say before we start training the generator, we show the discriminator our classes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; samples while giving information, that the class-&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; items are not part of class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (through backprop). The discriminator would learn to reject class-&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; items that are fed to it. But wouldn't this mean that the discriminator has just become the classification model we wanted to build in the first place to distinguish between class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Do you know any solution to the above-stated problem or can you refer to some paper/other posts on this?&lt;/p&gt;&#xA;" OwnerUserId="19378" LastEditorUserId="2444" LastEditDate="2021-01-17T20:15:28.870" LastActivityDate="2021-01-17T20:21:52.940" Title="How can I use Generative Adversarial Networks to solve the imbalanced class problem?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;binary-classification&gt;&lt;imbalanced-datasets&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8642" PostTypeId="2" ParentId="8637" CreationDate="2018-10-27T21:52:16.180" Score="1" Body="&lt;p&gt;The key difference between a learning agent and non-learning agents is that the learning agent can improve it's performance on it's own, allowing it to get &quot;smarter&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Russel &amp;amp; Norvig cover the different types of intelligent agents in detail in their textbook &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;, and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#Learning_agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia entry for intelligent agent&lt;/a&gt; mirrors their definitions. &lt;em&gt;[&lt;a href=&quot;https://faculty.psau.edu.sa/filedownload/doc-7-pdf-a154ffbcec538a4161a406abf62f5b76-original.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Free pdf of the 3rd edition here&lt;/a&gt;.]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They break down agents into the following categories:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Simple_reflex_agent.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simple reflex agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Model_based_reflex_agent.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model-based reflex agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Model_based_goal_based_agent.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goal-based agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Model_based_utility_based.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Utility-based agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:IntelligentAgent-Learning.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm not going to go into depth because understanding of technical terminology is required, but the links in the above list provide simple flow-chart explanations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first 4, the more sophisticated agents contain the primitive agents.  The model-based agent is actually a &quot;model-based reflex agent&quot;, and the goal and utility agents are also model-based.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will notice is that &lt;em&gt;the structure changes fundamentally with the learning agent&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key components of the learning agent are categorized as the &quot;Critic&quot;, the &quot;Learning Element&quot;, the &quot;Performance Element&quot;, and &quot;Problem Generator&quot;.  This structure allows the agent to evaluate past performance and modify subsequent actions to improve performance (learning).  The problem generator allows the agent to seek novel strategies (experimentation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the simple reflex agent operates under a &quot;condition-action rule&quot; (if a condition is met, take this action), the learning agent can improve it's decision-making beyond it's initial programming.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Deep learning is a form of machine learning that uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;representations&lt;/a&gt;. In the most general sense, the algorithms use a trial and error and statistical analysis to optimize performance.  Various forms of &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning#Neural_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural networks&lt;/a&gt; are classified as deep learning systems. &lt;em&gt;[See also &lt;a href=&quot;https://stackoverflow.com/questions/37973108/deep-reinforcement-learning-vs-reinforcement-learning&quot;&gt;Deep Reinforcement Learning vs. Reinforcement Learning&lt;/a&gt;.]&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My sense is that learning agents can be applied to any type of problem in which parameters can be sufficiently defined.  I don't want to get too specific because there are many types of learning algorithms, but we've been seeing application of machine learning (in various forms) expanding significantly since ~2016.  This includes &lt;a href=&quot;https://slate.com/business/2016/07/google-just-found-a-use-for-its-deepmind-acquisition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;replacing classical control systems&lt;/a&gt;, &lt;a href=&quot;https://www.technologyreview.com/s/604271/deep-learning-is-a-black-box-but-health-care-wont-mind/&quot; rel=&quot;nofollow noreferrer&quot;&gt;medical diagnostics&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Neural_Machine_Translation&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural language processing&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepFace&quot; rel=&quot;nofollow noreferrer&quot;&gt;image recognition&lt;/a&gt;, and &lt;a href=&quot;https://users.soe.ucsc.edu/~niejiazhong/slides/chandra.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;consumer recommendations&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The applications are initially &quot;narrow&quot;, defined as capability at a single task, but can be aggregated for tasks such as self-navigating vehicles which involve sets of discrete problems.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-10-29T00:14:58.720" LastActivityDate="2018-10-29T00:14:58.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8643" PostTypeId="2" ParentId="8594" CreationDate="2018-10-27T23:18:28.037" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When it comes to the human brain, we are prone to make mistakes. However, artificial intelligence is sometimes presented to the public as perfect. Do artificial intelligent systems make mistakes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Anything that exists outside of fiction that can be called AI and is not trivial is not perfect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alexa: &lt;a href=&quot;https://www.theverge.com/circuitbreaker/2018/3/7/17092334/amazon-alexa-devices-strange-laughter&quot; rel=&quot;nofollow noreferrer&quot;&gt;creepy laugh&lt;/a&gt; - not trivial, but not perfect&lt;/li&gt;&#xA;&lt;li&gt;Tic-Tac-Toe / Connect four: perfect algorithms exist, but trivial (you can create a game tree)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem is that you didn't bother to define &quot;perfect&quot;. What does it mean to be perfect? Natural selection favors living beings that are energy efficient. Remembering everything might be considered perfect, but it is certainly not efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We tend to make mistakes because some skills - especially abstract/mathematical ones or long-term decision making are not supported by natural selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another big group of traits that are not supported by natural selection are health traits that are after the age of reproduction / helping children to grow up. Specifically cancer above the age of 40.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="2444" LastEditDate="2020-02-08T12:40:56.347" LastActivityDate="2020-02-08T12:40:56.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8650" PostTypeId="1" CreationDate="2018-10-28T09:37:14.010" Score="5" ViewCount="111" Body="&lt;p&gt;In error-based learning using gradient descent, if I give you a training dataset, then can you find the minimum error after training? And the minimum error should be true for all architectures of a neural network? Consider you will use MSE for calculating the error. You can choose anything you want other than my specified condition. It's like no matter how you change your network you can never cross the limit.&lt;/p&gt;&#xA;" OwnerUserId="18459" LastEditorUserId="2444" LastEditDate="2019-06-24T10:54:10.937" LastActivityDate="2019-06-24T12:17:14.690" Title="Is there a limit of minimum error for a particular training dataset in artificial Neural Network?" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8651" PostTypeId="1" AcceptedAnswerId="8655" CreationDate="2018-10-28T13:30:51.453" Score="10" ViewCount="4942" Body="&lt;p&gt;What is the difference between goal-based and utility-based agents? Please, provide a real-world example.&lt;/p&gt;&#xA;" OwnerUserId="18950" LastEditorUserId="2444" LastEditDate="2019-12-09T19:38:31.193" LastActivityDate="2021-01-21T21:52:04.227" Title="What is the difference between goal-based and utility-based agents?" Tags="&lt;comparison&gt;&lt;theory&gt;&lt;intelligent-agent&gt;&lt;goal-based&gt;&lt;utility-based&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8653" PostTypeId="1" CreationDate="2018-10-28T15:29:07.570" Score="1" ViewCount="29" Body="&lt;p&gt;The &lt;a href=&quot;https://www.notebookcheck.net/Asus-s-GeForce-RTX-2080-Ti-and-2080-portfolio-is-now-official.323914.0.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nvidia 2080 GPUs&lt;/a&gt; command a $500 premium for 3GB of incremental RAM (8GB -&gt; 11GB).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the relevant questions and thought process to determine the incremental improvement by moving from the 8GB to the 11GB GPU?  Assume these contexts for image segmentation (people counting)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;MATLAB &lt;/li&gt;&#xA;&lt;li&gt;Tensorflow&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am inclined to think that the platform affects the RAM evaluation, however, I can not be certain.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;UPDATE&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Thanks for the good responses that sharpen the question.  Updates to questions are provided below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plan is to train an image segmenter with recorded images and try different algorithms (YOLO R-CNN etc.) .  The end goal is to develop a model that will process a live stream.  I am in the planning stage and expect to build (if not buy) a GPU DL Server.  &lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="18819" LastEditDate="2018-10-29T02:29:33.793" LastActivityDate="2018-10-29T02:29:33.793" Title="Evaluating Utility of Incremental RAM" Tags="&lt;tensorflow&gt;&lt;matlab&gt;&lt;hardware-evaluation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8655" PostTypeId="2" ParentId="8651" CreationDate="2018-10-28T22:04:58.453" Score="11" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Utility&quot; rel=&quot;noreferrer&quot;&gt;Utility&lt;/a&gt; is a fundamental to Artificial Intelligence because it is the means by which we evaluate an agent's performance in relation to a problem. To distinguish between the concept of economic utility and utility-based computing functions, the term &quot;performance measure&quot; is utilized.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest way to distinguish between a goal-based agent and a utility-based agent is that a goal is specifically defined, where maximization of utility is general.  &lt;em&gt;(Maximizing utility is itself a form of goal, but generalized as opposed to specific.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Model_based_goal_based_agent.png&quot; rel=&quot;noreferrer&quot;&gt;goal-based&lt;/a&gt; navigation agent is tasked with getting from point A to point B.  If the agent succeeds, the goal has been satisfied. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Model_based_utility_based.png&quot; rel=&quot;noreferrer&quot;&gt;utility-based&lt;/a&gt; navigation agent could seek to get from point A to point B in the shortest amount of time, with the minimum expenditure of fuel, or both.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the above example, the utility agent is also goal based, but where the performance measure for the goal agent is a binary [succeed/fail], the utility agent can use real numbers and measure performance by degree. The utility agent allows more granularity in evaluation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For an example of a non-goal based utility agent consider a form of a &lt;a href=&quot;http://mbranegame.com/rules-of-m/&quot; rel=&quot;noreferrer&quot;&gt;partisan sudoku&lt;/a&gt; in which players compete to control regions on the gameboard by placement of weighted integers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a game with 9 regions, &lt;strong&gt;the goal based agent seeks to control a specific number of regions at the end of play&lt;/strong&gt;.  If the agent is conservative, the goal might be 5 regions. If the agent is hyper-aggressive, the goal might be 9 regions.  When evaluating the environment (gameboard), if the agent dominates the desired number of regions, it could choose to consolidate (reinforce); if the agent does not dominate the desired number of regions, it could choose to expand (attack).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above strategy can be effective, but is limited by the specificity of the goal.  A hyper-aggressive goal would work well against a weak opponent, but against a strong opponent it might prove disastrous.  If the agent is sophisticated, where performance has been poor, it might alter it's goal by switching to a &quot;turtling strategy&quot; and seek to control fewer regions, but, because the new goal is still specific, the agent may miss opportunities to improve it's final status beyond the adjusted goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The utility-based agent can approach the game with no specific goal beyond improving it's status. Rather than seeking to control a set number of regions, &lt;strong&gt;the utility-agent evaluates whether a given choice improves or worsens it's status.&lt;/strong&gt; (&quot;Do I dominate more or less regions if I take this position?&quot;) The utility agent can distinguish between sets of beneficial choices (&quot;which choice maximizes my expected benefit?&quot;) and, where no benefit can be obtained, distinguish among the set of choices with the least downside (&quot;among the set of bad choices, which is the least bad choice?&quot;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, the utility-agent doesn't even need to understand the victory condition (controlling the more regions than the opponent at the end of play.)  Instead, the utility-agent merely seeks to maximize the number of controlled regions over the course of play, which will result in victory if the agent makes more optimal choices than the opponent. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-10-31T21:20:37.183" LastActivityDate="2018-10-31T21:20:37.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8657" PostTypeId="2" ParentId="8588" CreationDate="2018-10-28T22:58:38.247" Score="0" Body="&lt;p&gt;I found this solution and will be happy about any improvements or suggestions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I create a random dataset of images, which are 28x28 pixels, and corresponding random labels (just for sake of clarification, I have another image dataset, this is just for explaining). Then I use a sklearn module for splitting the data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from sklearn.model_selection import train_test_split&#xA;&#xA;&#xA;# create the data and labels&#xA;def pixel_dataset(n_data=10, dpi = 28):&#xA;&#xA;    ary = np.zeros((n_data, dpi, dpi))&#xA;    label = [] &#xA;&#xA;    for i in np.arange(n_data):&#xA;        labels =['a', 'b', 'c']&#xA;        ary[i,::] = np.reshape(np.array(np.random.randint(0, 255, 28*28)),(28,28))&#xA;        label.append(np.random.choice(labels, 1)[0])&#xA;&#xA;    return ary, label &#xA;&#xA;&#xA;# create the test train split    &#xA;def mydata(test_size=0.3):&#xA;&#xA;    X, y = pixel_dataset()       &#xA;    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = test_size) &#xA;    return (X_train, y_train), (X_test, y_test)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19251" LastActivityDate="2018-10-28T22:58:38.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8658" PostTypeId="1" AcceptedAnswerId="8772" CreationDate="2018-10-28T23:25:41.620" Score="3" ViewCount="172" Body="&lt;p&gt;In &lt;a href=&quot;https://www.youtube.com/watch?v=PTbxa6GsTWc&amp;amp;index=3&amp;amp;list=PLkFD6_40KJIznC9CDbVTjAF2oyt8_VAe3&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of his lectures&lt;/a&gt; Levine describes the objective of reinforcement learning as: &lt;span class=&quot;math-container&quot;&gt;$$J(\tau) = E_{\tau\sim p_\theta(\tau)}[r(\tau)]$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; refers to a single trajectory and &lt;span class=&quot;math-container&quot;&gt;$p_\theta(\tau)$&lt;/span&gt; is the probability of having taken that trajectory so that &lt;span class=&quot;math-container&quot;&gt;$p_\theta(\tau) = p(s_1)\prod_{t = I}^T \pi_{\theta}(a_t, s_t)p(s_{t+1}|s_t, a_t))$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from this definition, he writes the objective as &lt;span class=&quot;math-container&quot;&gt;$J(\tau) =\sum_{t=1}^T E_{(s_t, a_t)\sim p_\theta(\tau)}[r(s_t, a_t)]$&lt;/span&gt; and argues that this sum can be decomposed by using conditional expectations, so that it becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\tau) = E_{s_1 \sim p(s_1)}[E_{a_1 \sim \pi(a_1|s_1)}[r(s_1, a_1) + E_{s_2 \sim p(s_2|s_1, a_1)}[E_{a_2 \sim \pi(a_2|s_2)}[r(s_2, a_2)] + ...|s_2]|s_1,a_1]|s_1]]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain this last step? I guess the &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_total_expectation&quot; rel=&quot;nofollow noreferrer&quot;&gt;law of total expectation&lt;/a&gt; is involved, but I can not figure out how exactly.&lt;/p&gt;&#xA;" OwnerUserId="19212" LastActivityDate="2018-12-07T07:32:35.470" Title="Reinforcement learning objective as conditional expectations" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;statistical-ai&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8662" PostTypeId="1" CreationDate="2018-10-29T02:52:21.983" Score="1" ViewCount="47" Body="&lt;p&gt;I am working on a RL project,but got stuck at one point: The task is continuous (Non-episodic). Following some suggestion from Sutton's &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL book&lt;/a&gt;, I am using a value function approximation method with average reward (differential return instead of discount return). For some state (represented by some features), only one action is legal. I am not sure how to design reward for such action. Is it ok to just assign the reward in the previous step? Could anyone tell me the best way to decide the reward for the only legal action. Thank you! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&#xA;To give more details, I added one simplified example:&#xA;Let me explain this by a simplified example: the state space consists of a job queue with fix size and a single server. The queue state is represented by the duration of jobs and the server state is represented by the time left to finish the current running job.  When the queue is not full and the server is idle, the agent can SCHEDULE a job to server for execution and see a state transition(taking next job into queue) or the agent can TAKE NEXT JOB into queue. But when the job queue is full and server is still running a job, the agent can do nothing except take a BLOCKING action and witness a state transit (time left to finish running job gets decreased by one unit time). The BLOCKING action is the only action that the agent can take in that state.&lt;/p&gt;&#xA;" OwnerUserId="19405" LastEditorUserId="19405" LastEditDate="2018-10-29T03:44:29.427" LastActivityDate="2018-10-29T03:44:29.427" Title="How to design the reward for an action which is the only legal action at some state" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8663" PostTypeId="2" ParentId="8270" CreationDate="2018-10-29T06:14:08.730" Score="0" Body="&lt;p&gt;So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!&lt;/p&gt;&#xA;" OwnerUserId="18788" LastActivityDate="2018-10-29T06:14:08.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8664" PostTypeId="1" CreationDate="2018-10-29T15:02:46.970" Score="2" ViewCount="52" Body="&lt;p&gt;I'm a bit confused about this. Assume I have a CNN network with two branches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Top&lt;/li&gt;&#xA;&lt;li&gt;Bottom&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The top branch outputs a feature vector of shape 1x1x1x10 (batch, h, w, c)&#xA;The bottom branch outputs a feature vector of shape (1, 10, 10, 10).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use the top feature vector as a convolutional filter, and convolve it with the bottom feature vector. I can do this in pytorch with the &quot;functional.Conv2D&quot; function, the problem is, I don't know how back-prop works in this case (will it be unstable?) since the output feature is a now a parameter as well, do I need to stop gradients or do something else in this case to backprop correctly?&lt;/p&gt;&#xA;" OwnerUserId="19408" LastActivityDate="2018-10-29T15:02:46.970" Title="Using features extracted from a CNN as convolutional filter" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8666" PostTypeId="2" ParentId="8658" CreationDate="2018-10-29T15:40:19.817" Score="2" Body="&lt;p&gt;In the &lt;a href=&quot;https://www.youtube.com/watch?v=PTbxa6GsTWc&amp;amp;index=3&amp;amp;list=PLkFD6_40KJIznC9CDbVTjAF2oyt8_VAe3&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;YouTube depiction of CS294-112 fall 2017 lecture 3 Reinforcement Learning&lt;/em&gt;, Levine&lt;/a&gt;, the transition of the finite horizon expected reward to a form where each transition is decoupled from the entire Markov chain of state-action marginals is explained between &lt;span class=&quot;math-container&quot;&gt;$t_{video}$&lt;/span&gt; = 44:04 and &lt;span class=&quot;math-container&quot;&gt;$t_{video}$&lt;/span&gt; = 45:22.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At t=44:29, the probabilistic expectation where no actions take place is given as a starting point.  It's a distribution already known, the initial state distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statement, &quot;Inside that expectation ...,&quot; at &lt;span class=&quot;math-container&quot;&gt;$t_{video}$&lt;/span&gt; = 44:39, begins the description of how to build an expectation expression for the first two vertices of the Markov chain.  This is the beginning of a process where expectations &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; are factored recursively.  The purpose of the expansions is to decouple the expectation at each vertex from those of other vertices in the Markov chain, clarifying dependencies and exposing the mathematical features of the recursion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The right-hand factor is the integrated expectation of reward over the achievement of the next state-action marginal, where the action is distributed according to the fixed policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How a state and action pair fits into a state-action marginals which can then form a Marcov Chain, provided the policy is fixed and known, was explained at &lt;span class=&quot;math-container&quot;&gt;$t_{video}$&lt;/span&gt; = 21:40 in this same lecture.  Each vertex in the chain is taken to be the aggregation of a state with the proposed action to be applied to it, leading to a reward &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Returning to &lt;span class=&quot;math-container&quot;&gt;$t_{video}$&lt;/span&gt; = 44:39, notice the plus sign without the right-hand addend, waiting for another recursion.  When the recursion is expressed later in the process, the right-hand factor becomes the integrated expectation of reward over the achievement of the finite sequence represented by the remainder of the Markov chain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplication is implied because we are now conditioning on &lt;strong&gt;both&lt;/strong&gt; probabilities of the initial state and a transition from it.  Notice also that the second vertex in the Marcov Chain is a state-action marginal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice now that rewards are additive because the goal of learning was defined as a sum of products, which is a simplifying assumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also notice that each expectation distributions of reward after the first one can be expressed as a product of single-vertex expectation distributions.  Each factor is a localized expectation along the sequence of vertices in the Markov chain of state-action marginals.  This is the factoring of expectation distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This factoring is the application of Product distribution, which &lt;a href=&quot;https://ipfs.io/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Product_distribution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be proven&lt;/a&gt; using the &quot;law&quot; of total expectations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At &lt;span class=&quot;math-container&quot;&gt;$t_{video}$&lt;/span&gt; = 44:51, the pattern recursion is expressed with a single ellipsis.  Recognition of the recursion pattern can be facilitated by re-expressing with a few modifications of mathematical style.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use of the original variable to represent the value objective&lt;/li&gt;&#xA;&lt;li&gt;Extending the equality&lt;/li&gt;&#xA;&lt;li&gt;Using two ellipses instead of one clarify the recursion is a continuation of conditions from right to left as well as continuation of rewards from left to right&lt;/li&gt;&#xA;&lt;li&gt;Strategically placing line returns&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi_{\theta}(\tau) = \sum_{t=1}^T \operatorname{E}_{(s_t, a_t)\sim p_\theta(\tau)}[r(s_t, a_t)] \\ = E_{s_1 \sim p(s_1)}[E_{a_1 \sim \pi(a_1|s_1)}[r(s_1, a_1) \\ + E_{s_2 \sim p(s_2|s_1, a_1)}[E_{a_2 \sim \pi(a_2|s_2)}[r(s_2, a_2)] \\ + \; ... \\ ...|s_2]|s_1,a_1]|s_1]]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inverting the transformation, recombining the expansion back to its original form may also be illustrative.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-29T16:02:00.557" LastActivityDate="2018-10-29T16:02:00.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8667" PostTypeId="1" AcceptedAnswerId="8673" CreationDate="2018-10-29T16:09:10.950" Score="2" ViewCount="132" Body="&lt;p&gt;How do I define a reward function for my POMDP model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the literature, it is common to use one simple number as a reward, but I am not sure if this is really how you define a function. Because this way you have to do define a reward for every possible action-state combination. I think that the examples in the literature might not be practical in reality, but only for the purpose of explanation.&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="2444" LastEditDate="2021-01-20T12:46:14.020" LastActivityDate="2021-01-20T12:46:14.020" Title="How to define a reward function in POMDPs?" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;pomdp&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8668" PostTypeId="1" CreationDate="2018-10-29T16:50:36.387" Score="6" ViewCount="1670" Body="&lt;p&gt;I need to be able to detect and track humans from all angles, especially above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, obviously, quite a few well-studied models for human detection and tracking, usually as part of general-purpose object detection, but I haven't been able to find any information that explicitly works for tracking humans from above.&lt;/p&gt;&#xA;" OwnerUserId="19414" LastActivityDate="2019-09-30T08:02:15.573" Title="Are there any pretrained models for human recognition from all angles?" Tags="&lt;object-recognition&gt;&lt;models&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8670" PostTypeId="2" ParentId="8668" CreationDate="2018-10-29T17:19:28.117" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Look at all the Haar boosted classifiers that are available with OpenCV and the dedicated class CascadeClassifier to use it. Here are a list of what the classifiers have locally...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can try &lt;code&gt;frontalface&lt;/code&gt;. The shoulders I searched and did not find much. You may have to create a shoulder dataset and train to detect it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/5727231/5336017&quot;&gt;https://stackoverflow.com/a/5727231/5336017&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2018-10-29T17:19:28.117" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8671" PostTypeId="2" ParentId="8651" CreationDate="2018-10-29T19:42:32.770" Score="0" Body="&lt;h3&gt;What is the difference between goal-based and utility-based agents?&lt;/h3&gt;&#xA;&lt;p&gt;Both goal-based and utility-based agents have goals. However, having goals isn't effective (or efficient) enough, given that a goal-based agent may have several actions that can lead to the goals, but not all these actions are equally effective. So there's the need for an agent to perform the most effective action. And this is done by a utility-based agent.&lt;/p&gt;&#xA;&lt;p&gt;That said, for an agent that exhibits the utility function, it maps each state after each action being taken nor performed efficiently and effectively.&lt;/p&gt;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&lt;p&gt;Consider two drones &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is a goal-based and &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; a utility-based agent. (The two drones have onboard computerized chips, so there is no need for ground control). These drones are sent on a mission and they have a goal. Both drones detect the given goal, but &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; does not know which of its available actions is more efficient or effective. However, &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;, based on its &lt;em&gt;utility function&lt;/em&gt;, can select the most efficient or effective action.&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;Sections 2.4.4. and 2.4.5 of the book &lt;a href=&quot;https://cs.calvin.edu/courses/cs/344/kvlinden/resources/AIMA-3rd-edition.pdf#page=71&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach (3rd edition)&lt;/a&gt; provide more info.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2021-01-21T21:52:04.227" LastActivityDate="2021-01-21T21:52:04.227" CommentCount="0" CommunityOwnedDate="2021-01-27T19:24:45.370" ContentLicense="CC BY-SA 4.0" />
  <row Id="8672" PostTypeId="2" ParentId="8667" CreationDate="2018-10-29T20:01:56.777" Score="0" Body="&lt;p&gt;In a POMDP, you minimize the following cost function&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;J_{\pi_{0:N-1}} (\cdot) &#xA;&amp;amp;= &#xA;\mathbb{E} \left( g_N (x_N) + \sum_{k=0}^{N-1} g_k (x_k, \pi_k(\cdot)) \right),&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$g_N$&lt;/span&gt; is the terminal cost and &lt;span class=&quot;math-container&quot;&gt;$g_k$&lt;/span&gt; is the step cost.&lt;/p&gt;&#xA;&lt;p&gt;Note that this is only the formal problem definition. If you talk about an actual system, you might have a hard time recognizing that this is connected to your reward. What you observe is formally a value of the function J and practically a simple float&lt;/p&gt;&#xA;&lt;p&gt;More details in German are on my blog: &lt;a href=&quot;https://martin-thoma.com/probabilistische-planung/#mdp-vs-pomdp-vs-rl_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://martin-thoma.com/probabilistische-planung/#mdp-vs-pomdp-vs-rl_1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="2444" LastEditDate="2021-01-20T12:45:54.747" LastActivityDate="2021-01-20T12:45:54.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8673" PostTypeId="2" ParentId="8667" CreationDate="2018-10-29T20:53:28.597" Score="1" Body="&lt;p&gt;There is no major difference here between a POMDP and MDP. When setting reward values, you are generally trying to give the minimal information to the agent that when the sum of rewards is maximised, it solves the problem that you are posing. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In literature it is common to use one simple number as a reward, but I am not sure if this is really how you define a function. Because this way you have do define a reward for every possible Action-State combination.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Some defined value of reward has to be returned after &lt;em&gt;all&lt;/em&gt; state, action pairs taken in the environment. The value could be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; of course. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reward can depend on, or be a function of, current state, action, next state and a random factor. In a POMDP it may also be from any unobserved factor in the environment (you might know this in a simulation because you have created the environment and are choosing not to share the data with the agent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the reward often does not have to depend on all the possible factors. In addition, the relationship between the factors and the possible reward can be very simple or sparse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classic examples you may find in the literature include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Reward in a game can be simply &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; for winning, or &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for losing, granted at the end. All other rewards are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your goal is to reach a certain state, such as exit from a maze, in minimum time, then a fixed reward of &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; per time step is enough to express the need to minimize the total number of steps.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For a goal of maintaining stability and avoiding a failure in e.g. &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CartPole&lt;/a&gt;, then a reward of +1 per time step without failure suffices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these examples express the reward function as a simple condition plus one or two numbers. The key thing is that they allow you to &lt;em&gt;calculate&lt;/em&gt; a suitable reward on each and every time step. They are all strictly reward functions - the general case can cover very complex functions if you wish, that will depend entirely on the goals you want to set the agent to learn/solve.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-10-30T16:33:05.633" LastActivityDate="2018-10-30T16:33:05.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8676" PostTypeId="1" CreationDate="2018-10-29T21:33:51.103" Score="1" ViewCount="966" Body="&lt;p&gt;I just wanted to confirm that my understanding of the different Markov Decision Processes are correct, because they are the fundamentals of reinforcement learning. Also, I read a few literature sources, and some are not consistent with each other. What makes the most sense to me is listed below.&lt;/p&gt;&#xA;&lt;h3&gt;Markov Decision Process&lt;/h3&gt;&#xA;&lt;p&gt;All the states of the environment are known, so the agent has all the information required to make the optimal decision in every state.  We can basically assume that the current state has all information about the current state and all the previous states (i.e., the Markov property)&lt;/p&gt;&#xA;&lt;h3&gt;Semi Markov Decision Process&lt;/h3&gt;&#xA;&lt;p&gt;The agent has enough information to make decisions based on the current state.  However, the actions of the agent may take a long time to complete, and may not be completed in the next time step. Therefore, the feedback and learning portion should wait until the action is completed before being evaluated.  Because the action takes many time steps, &amp;quot;mini rewards&amp;quot; obtained from those time steps should also be summed up.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example: Boiling water&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;State 1: water is at 23 °C&lt;/li&gt;&#xA;&lt;li&gt;Action 1: agent sets the stovetop at 200 °C&lt;/li&gt;&#xA;&lt;li&gt;Reward (30 seconds after, when water started to boil):&#xA;&lt;ul&gt;&#xA;&lt;li&gt;+1 for the fact that water boiled in the end, but -0.1 reward for each second it took for the water to start boiling.&lt;/li&gt;&#xA;&lt;li&gt;So, the total reward was -1.9 (-2.9 because the water did not boil for 29 seconds, then +1 for water boiling on 30th second)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Partially Observable Markov Decision Process&lt;/h3&gt;&#xA;&lt;p&gt;The agent does not have all information regarding the current state, and has only an observation, which is a subset of all the information in a given state.  Therefore, it is impossible for the agent to truly behave optimally because of a lack of information.  One way to solve this is to use &lt;em&gt;belief states&lt;/em&gt;, or to use RNNs to try to remember previous states to make better judgements on future actions (i.e., we may need states from the previous 10 time steps to know exactly what's going on currently).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We are in a room that is pitch black.  At any time instant, we do not know exactly where we are, and if we only take our current state, we would have no idea.  However, if we remember that we took 3 steps forward already, we have a much better idea of where we are.&lt;/p&gt;&#xA;&lt;p&gt;Are my above explanations correct?  And if so, isn't it possible to also have Partially Observable Semi Markov Decision Processes?&lt;/p&gt;&#xA;" OwnerUserId="17706" LastEditorUserId="2444" LastEditDate="2021-01-24T21:41:18.723" LastActivityDate="2021-01-24T21:41:18.723" Title="Is my understanding of the differences between MDP, Semi MDP and POMDP correct?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;&lt;semi-mdp&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8680" PostTypeId="1" CreationDate="2018-10-30T10:39:08.117" Score="2" ViewCount="42" Body="&lt;p&gt;&lt;strong&gt;I would like to create an NPC engine for games that are in a fantasy context&lt;/strong&gt; like Lord of the Rings, Warcraft, Skyrim, Dragon Age, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This engine should be able to &lt;strong&gt;determine the polarity of the given sentence&lt;/strong&gt; (positive/negative/neutral) and should also be able to tag the words (part-of-speech tagging, but that's not relevant now).&#xA;&lt;strong&gt;The problem is that I cannot find a dataset which&lt;/strong&gt; I can teach my AI with, so that &lt;strong&gt;fits into the fantasy context&lt;/strong&gt;.&#xA;As an example for Lord of the Rings, there are characters like Sauron which are negative, but there's a ton of expressions too in a fantasy world which cannot be found in a general dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, &lt;strong&gt;I would like to ask if you could suggest taking dataset(s)&lt;/strong&gt; which contain expressions, even better names in a fantasy context.&#xA;It doesn't matter which fantasy universe does it take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="19441" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:20.567" LastActivityDate="2020-05-29T13:48:20.567" Title="Searching for dataset in specifix context for NLP, sentiment analysis" Tags="&lt;natural-language-processing&gt;&lt;datasets&gt;&lt;sentiment-analysis&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8682" PostTypeId="1" AcceptedAnswerId="8684" CreationDate="2018-10-30T13:02:59.023" Score="1" ViewCount="301" Body="&lt;p&gt;I was using PCA on my whole dataset (and, after that, I would split it into training, validation, and test datasets). However, after a little bit of research, I found out that this is the wrong way to do it.&lt;/p&gt;&#xA;&lt;p&gt;I have few questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there some articles/references that explain why is the wrong way?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I transform the validation/test set?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Steps to do PCA (from &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0022460X0093390X&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0022460X0093390X&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;zero mean&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mu = \frac{1}{M}\sum_{i=1}^{M} x_{i}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;strong&gt;x&lt;/strong&gt; is my training set&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;centering (variance)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S^{2} = \frac{1}{M}\sum_{i=1}^{M} (x_{i}-\mu)^{T}(x_{i}-\mu)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;use (1) and (2) to transform my original training dataset&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_{new} = \frac{1}{\sqrt{M}} \frac{(x_{i} - \mu)}{S}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;calculate covariance matrix (actually correlation matrix)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C= x_{new}^T x_{new}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;take the &lt;em&gt;k&lt;/em&gt;-eigenvectors (/phi) from the covariance matrix and defined the new space for my new dimension training set (where k are the principal components that I choose according to my variance)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ x_{new dim} = x_{new}\phi$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Ok, then I have my new dimensional training dataset after PCA (till here it's right, according to other papers that I have read).&lt;/p&gt;&#xA;&lt;p&gt;The question is: *&lt;em&gt;What I have to do now for my validation/testing set?&lt;/em&gt; Just the equation below?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{new dim} = y\phi $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;strong&gt;y&lt;/strong&gt; is my (for example) validation original dataset?&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain the right thing to do?&lt;/p&gt;&#xA;" OwnerUserId="19268" LastEditorUserId="2444" LastEditDate="2021-04-01T11:08:58.747" LastActivityDate="2021-04-01T11:08:58.747" Title="How to perform PCA in the validation/test set?" Tags="&lt;machine-learning&gt;&lt;principal-component-analysis&gt;&lt;test-datasets&gt;&lt;validation-datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8684" PostTypeId="2" ParentId="8682" CreationDate="2018-10-30T13:46:16.000" Score="2" Body="&lt;p&gt;for the first point I'm very sorry that I cannot give you any literature on this, but I might be able to explain you, why you don't take PCA on both datasets independently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Principal components analysis is simply a transformation of your data into another (less dimensional) coordinate system. The axis for your new coordinate system are defined by the principal components (i.e. eigen-vectors) of your covariance matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you will train your machine learning algorithm in the domain that is generated by the PCA, your test data must be exactly in the same domain. So as you said, you use exactly the same transformation for the test data as for the training data, i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;$y_{newdim} = y \phi $&lt;/span&gt;.&#xA;Of course if you applied standardization to your training data, you have to apply the same standardization to your test data. So you need to store the mean &lt;span class=&quot;math-container&quot;&gt;$\mu_x$&lt;/span&gt; and the standard deviation &lt;span class=&quot;math-container&quot;&gt;$S_x$&lt;/span&gt; and also standardize your test data y to &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;y_{standardized} = \dfrac{y_i - \mu_x}{S_x}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note here that you have an error in your standardization formula (you do not need to divide by &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that the principal components of your test data &lt;span class=&quot;math-container&quot;&gt;$\phi_{y}$&lt;/span&gt; would not match the principal components of your training data &lt;span class=&quot;math-container&quot;&gt;$\phi_{x}$&lt;/span&gt;. Thus the transformations from original space into PCA-space &lt;span class=&quot;math-container&quot;&gt;$\Phi_x(u)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Phi_y(u)$&lt;/span&gt; would diverge and similar data points in the original space might be far away in the PCA-representation and vice versa. This is why you generate the mapping &lt;span class=&quot;math-container&quot;&gt;$x_{newdim} = \Phi_x(x) = x\phi$&lt;/span&gt; and apply it also on the test data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I could make it clear to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="19256" LastEditorUserId="19256" LastEditDate="2018-10-30T15:02:56.687" LastActivityDate="2018-10-30T15:02:56.687" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8686" PostTypeId="1" CreationDate="2018-10-30T15:45:41.537" Score="1" ViewCount="110" Body="&lt;p&gt;In language models, CNNs can extract different n-gram features from the input. From my current understanding, these models are called &quot;multi-channel CNNs&quot;. I'm referring to these materials:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@sharaf/a-paper-a-day-19-a-multichannel-convolutional-neural-network-for-cross-language-dialog-state-bb00f5328163&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@sharaf/a-paper-a-day-19-a-multichannel-convolutional-neural-network-for-cross-language-dialog-state-bb00f5328163&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1701.06247.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1701.06247.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What is the difference between channels in convolutional networks and the &quot;multi-channel&quot; model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these two types of channels similar or do they have different functions?&lt;/p&gt;&#xA;" OwnerUserId="19403" LastEditorUserId="1581" LastEditDate="2018-10-30T16:39:45.610" LastActivityDate="2018-10-30T16:39:45.610" Title="Multi-channel CNNs and channels" Tags="&lt;convolutional-neural-networks&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8688" PostTypeId="2" ParentId="8902" CreationDate="2018-10-30T18:53:54.950" Score="2" Body="&lt;p&gt;According to the book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (3rd edition), by Stuart Russel and Peter Norvig, specifically, section &lt;strong&gt;3.5.1 Greedy best-first search&lt;/strong&gt; (p. 92)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Greedy best-first search tries to expand the node that is closest to the goal, on the grounds that this is likely to lead to a solution quickly. Thus, it evaluates nodes by using just the heuristic function; that is, &lt;span class=&quot;math-container&quot;&gt;$f(n) = h(n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this same section, the authors give an example that shows that greedy best-first search is neither optimal nor complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section &lt;strong&gt;3.5.2 A* search: Minimizing the total estimated solution cost&lt;/strong&gt; of the same book (p. 93), it states&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A* search evaluates nodes by combining &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;, the cost to reach the node, and &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt;, the cost to get from the node to the goal &lt;span class=&quot;math-container&quot;&gt;$$f(n) = g(n) + h(n).$$&lt;/span&gt; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; gives the path cost from the start node to node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the estimated cost of the cheapest path from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal, we have &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt; = estimated cost of the cheapest solution through &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Thus, if we are trying to find the cheapest solution, a reasonable thing to try first is the node with the lowest value of &lt;span class=&quot;math-container&quot;&gt;$g(n) + h(n)$&lt;/span&gt;. It turns out that this strategy is more than just reasonable: provided that the heuristic function &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; satisfies certain conditions, A* search is both complete and optimal. The algorithm is identical to uniform-cost search except that A* uses &lt;span class=&quot;math-container&quot;&gt;$g + h$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="18689" LastEditorUserId="2444" LastEditDate="2019-08-30T00:42:47.363" LastActivityDate="2019-08-30T00:42:47.363" CommentCount="0" CommunityOwnedDate="2020-11-08T12:03:07.603" ContentLicense="CC BY-SA 4.0" />
  <row Id="8689" PostTypeId="1" CreationDate="2018-10-30T20:38:16.187" Score="7" ViewCount="129" Body="&lt;p&gt;How much can the addition of new features improve the performance of the model during the optimization process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a total of 10 features. Suppose I start the optimisation process using only 3 features. Can the addition of the 7 remaining ones improve the performance of the model (which, you can assume, might already be quite high).&lt;/p&gt;&#xA;" OwnerUserId="16474" LastEditorUserId="2444" LastEditDate="2019-05-06T16:47:09.670" LastActivityDate="2021-04-19T22:38:30.177" Title="How much can the addition of new features improve the performance?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;optimization&gt;&lt;theory&gt;&lt;feature-selection&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8690" PostTypeId="2" ParentId="5577" CreationDate="2018-10-30T20:48:52.493" Score="2" Body="&lt;p&gt;I wrote a relatively simple &lt;a href=&quot;https://github.com/jarble/adaptive_parser&quot; rel=&quot;nofollow noreferrer&quot;&gt;adaptive parser in Prolog&lt;/a&gt;. The parser is essentially a &lt;a href=&quot;https://cs.stackexchange.com/a/102698/77574&quot;&gt;string rewriting system&lt;/a&gt; that learns new rewrite rules from its input, such as &lt;code&gt;&quot;A implies that B&quot; means &quot;A implies B&quot;&lt;/code&gt;, or &lt;code&gt;&quot;neither A nor B&quot; means &quot;not (A or B)&quot;&lt;/code&gt;, using a simple bottom-up parsing algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the grammar rules that it has learned, the parser is able to convert English phrases such as &lt;code&gt;C is not less than D percent of E and R implies that Q is not true&lt;/code&gt; into Prolog terms, such as &lt;code&gt;(C&amp;lt;D/100*E)=false,(R-&amp;gt;Q\=true)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to the parser that I described here, there is an adaptive parser generator called &lt;a href=&quot;http://dypgen.free.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dypgen&lt;/a&gt;. There also are several programming languages that allow user-defined syntax extensions, including &lt;a href=&quot;https://coq.inria.fr/refman/user-extensions/syntax-extensions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coq&lt;/a&gt; and &lt;a href=&quot;https://agda.readthedocs.io/en/v2.6.0.1/language/syntax-declarations.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agda&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://github.com/kach/nearley/issues/485&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nearley&lt;/a&gt; parser generator also allows grammar rules to be added at runtime, though this feature isn't documented yet.&lt;/p&gt;&#xA;" OwnerUserId="2050" LastEditorUserId="2050" LastEditDate="2020-01-03T23:12:58.960" LastActivityDate="2020-01-03T23:12:58.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8691" PostTypeId="2" ParentId="8902" CreationDate="2018-10-31T00:03:08.087" Score="1" Body="&lt;p&gt;What you said isn't totally wrong, but the A* algorithm becomes optimal and complete if the heuristic function h is admissible, which means that this function never overestimates the cost of reaching the goal. In that case, the A* algorithm is way better than the greedy search algorithm. &lt;/p&gt;&#xA;" OwnerUserId="19458" LastEditorUserId="2444" LastEditDate="2019-08-30T00:23:29.313" LastActivityDate="2019-08-30T00:23:29.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8693" PostTypeId="2" ParentId="1742" CreationDate="2018-10-31T06:25:36.797" Score="9" Body="&lt;p&gt;&lt;strong&gt;Deep learning is one form of machine learning.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning refers to learning with deep neural networks, essentially networks with many layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks are one group of many forms of machine learning:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neural Networks&lt;/li&gt;&#xA;&lt;li&gt;Decision Trees and Random Forests&lt;/li&gt;&#xA;&lt;li&gt;Support Vector Machines&lt;/li&gt;&#xA;&lt;li&gt;Bayesian Approaches&lt;/li&gt;&#xA;&lt;li&gt;k-nearest neighbors&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2019-04-22T11:05:58.510" LastActivityDate="2019-04-22T11:05:58.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8694" PostTypeId="1" CreationDate="2018-10-31T12:12:25.677" Score="2" ViewCount="98" Body="&lt;p&gt;I am building 2 models using XGboost, one with x number of parameters and the other with y number of parameters of the data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a classification problem.&#xA;A yes-yes, no-no case is easy, but what should I do when one model predicts a yes and the other model predicts a no ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model A with x parameters has an accuracy of 82% and model B with y parameters has accuracy of 79%.  &lt;/p&gt;&#xA;" OwnerUserId="19465" LastEditorUserId="19466" LastEditDate="2018-11-01T19:51:41.880" LastActivityDate="2019-08-29T13:03:40.537" Title="Ensemble models - XGboost" Tags="&lt;data-science&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8696" PostTypeId="2" ParentId="8619" CreationDate="2018-10-31T15:26:22.200" Score="3" Body="&lt;p&gt;&lt;strong&gt;Systems Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's set out to replicate a real time system &lt;span class=&quot;math-container&quot;&gt;$S: \mathcal{X} \Rightarrow \mathcal{Y} \; | \; I$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is a empirical continuous history of input and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; empirical continuous history of output, conditioned upon a real initial system state &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;.  Based on some definition, we require &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; to be alive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We cannot simulate a replication of a theoretical model of life, with a selfish gene or any other such attribute, simply because no mathematically terse model on which the simulation could be based exists.  As of this writing, only hints to and minutia of such a model are known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, models are mathematical representations that, throughout human history, are found to be approximations of complexities once anomalies are addressed and new models develop to incorporate them into the theory.&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Simulation Roughly Defined&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we examine a general algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; to replicate &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, replication can be roughly sketched as follows.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Estimate system &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, essentially forming hypothesis &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Simulate initial state &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Initiate a series of discrete stimuli &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}_t$&lt;/span&gt; approximating the real and continuous &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Acquire resulting system behavior &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}_t$&lt;/span&gt; as discrete observations of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Verify the difference between simulated and actual systems to be within allowable error &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Defining Spontaneous Emergence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By spontaneous emergence is meant that such an astronomically large array of initial states and sequences of stimuli occurred that there is a high probability of one of the permutations being alive, based on some specific and reasonable definition of what is living.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Defining What Life Is&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reviewing several definitions of living organisms, the most reasonable definitions include these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The organism can be distinguished from its environment.&lt;/li&gt;&#xA;&lt;li&gt;The organism can acquire and cache potential energy and materials required to operate.&lt;/li&gt;&#xA;&lt;li&gt;Its operation includes continued acquisition, producing a bidirectional and sustainable relationship with its environment.&lt;/li&gt;&#xA;&lt;li&gt;The organism can roughly reproduce itself.&lt;/li&gt;&#xA;&lt;li&gt;The reproduction is similar to but not exactly like the parent(s).&lt;/li&gt;&#xA;&lt;li&gt;The method of energy and materials acquisition may include the consumption of other organisms or its energy and materials.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Competing for resources, natural selection, and all the other features of evolutionary theory are corollary to the above five requirements.  In addition to these, the current trend toward recognizing symbiogenesis as a common theme in the emergence of species should not be dismissed.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Replication of one organism may be influenced by the composition of another organism through forms of assimilation or symbiosis such that traits are passed across categories of organisms.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Artificial Life as a Simulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These seven criteria poses a challenge for humans attempting to artificially generate life.  It is easy to create a computer model such that life is simulated in some way.  Consider how.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The environment contains virtual energy and virtual matter.&lt;/li&gt;&#xA;&lt;li&gt;The model of the organism, distinguished from its environment, can acquire its operational requirements from the environment through a set of operations on it.&lt;/li&gt;&#xA;&lt;li&gt;Mater and energy are conserved because the temperatures are far below nuclear thresholds.&lt;/li&gt;&#xA;&lt;li&gt;The model of the organism allows acquisition only if enough of the energy and materials acquisition has occurred to maintain the cache.&lt;/li&gt;&#xA;&lt;li&gt;Mater and energy acquired by one organism cannot be acquired by another organism except by consumption or absorption of an organism that acquired it or produced it from that which was acquired.&lt;/li&gt;&#xA;&lt;li&gt;The model of the organism can self-replicate in such a way that stochastic differences in the replication is introduced in small quantities.&lt;/li&gt;&#xA;&lt;li&gt;Operational information, including replication information, may be acquired through consumption or symbiotic relationship under some conditions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Magical Genes for Spontaneous Life&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the selfish gene is not mentioned above.  Selfishness, the prerequisite of which is intention, is not a requirement for life.  An amoeba does not think selfishly when it moves or eats.  It operates witlessly.  We should not anthropomorphize every organism we study, or develop theory based on anthropomorphic conceptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, symbiotic relationships form that are neither loving nor altruistic.  They exist because there is a mutual benefit that appeared as an unintended byproduct of normal operations and both symbiotic parents happened to pass that symbiotic connection to their respective offspring. &#xA;The mutual benefit, the symbiosis, and the replication are witless and unintended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There need not be a control mechanism distinct from all other replicated mechanisms to control either symbiotic collaboration or competition.  They too are natural consequences of living things sharing an environment.  Whether an organism dies because it&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lost its symbiont,&lt;/li&gt;&#xA;&lt;li&gt;Starves because other organisms consumed its necessities,&lt;/li&gt;&#xA;&lt;li&gt;The organism itself depleted its own resources, or&lt;/li&gt;&#xA;&lt;li&gt;Those needed resources were otherwise rendered unavailable,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;it is still unable to replicate, so its traits die with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that there is no known molecule that can replicate itself.  Complex systems of molecules in a variety of chemical states and equilibria are required for reproduction to take place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Returning to Simulating an Already Existing Organism&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running a time sharing system or distributing these simulated organisms in a parallel processing arrangement may some day simulate a biosphere, but it is not one in that only transistor electro-chemistry is involved.  There is no actual direct relationship between the energy and mater of the system used to assemble the simulation and the energy and mater of the simulated environment in which the simulated systems &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; reside.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly genetic algorithms, such as Avida and Tierra, have been developed.  Compare those simulations to the modelling scenario described above, and their deficiencies become clear.  Human researchers have not yet found &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; to replicate &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; in a way that aligns with biological reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Open-endedness Requires Verification to Have Merit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most significant limitation on implementations &lt;em&gt;in silico&lt;/em&gt;, is that they can never be truly open-ended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no way as of this writing to replicate that which was simulated outside the simulation system.  Until nanotechnology reaches a point where 3D construction and assembly can migrate alive simulations into the unsimulated universe, these simulations are closed-ended in that way and their viability &lt;em&gt;in vito&lt;/em&gt; is untested.  The value of open-ended simulations without any way to validate them is essentially zero except for amusement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in the space of digital simulation, as far as that technology has progressed, nothing even close to von Neumann's universal constructor has been accomplished.  Although generic functional copy constructors are available in Scheme, LISP, C++, Java, and later languages, such is a minuscule step toward living objects in computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Digital Soup&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simulation of life's origins is considerably more difficult than finding an algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; to replicate &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a single life form and a sufficient portion of its environment to be representative of the biosphere on earth with an organism in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue with primordial digital soup is one of the combinatory explosion.  There are 510 million square Km on the earth's surface, and there are only three categories of life origin time frames possible.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The current estimates are close to correct, that the earth formed 4.54 billion years ago and extremely primitive life emerged 3.5 billion years ago&lt;/li&gt;&#xA;&lt;li&gt;The organic material found in Canada that is allegedly 3.95 billion years old shortens the gap between planetary formation and life formation on it and older terrestrial life may be found&lt;/li&gt;&#xA;&lt;li&gt;Vladimir Vernadsky's comment that life may have preexisted earth is more than just a possibility&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If we go with the 1.04 billion year gap, then &lt;span class=&quot;math-container&quot;&gt;$(4.54 - 3.5) \cdot 10^9 \cdot 510 \cdot 10^6$&lt;/span&gt; Km-years of soup must be simulated, since we cannot assume that life started in the ocean or a puddle or even on the surface.  It could have started underground or in the atmosphere.  The biosphere is currently thought to be 1,800 m above to 8,372 m below thick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With nanobes being 20 nm in diameter and the possibility that the emergence may have only taken one second we have to simulate in three dimensions over time the following space-time domain in finite elements with at least 50% overlap in all three dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dfrac {2^3 \cdot (4.54 - 3.5) \cdot 10^9 \cdot 510 \cdot 10^6 \cdot (1,800 - 8,372) \cdot 365.25 \cdot 24 \cdot 60 \cdot 60} {(20 \cdot 10^{-9})^3} \\ = 170,260,472,379 \cdot 10^{9+6+27} = 1.7 \cdot 10^{56}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a quantum computer two stories high the size of Switzerland, the computing time would vastly exceed the duration of the average species on earth.  Humans are likely to be extinct before the computation completes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the dating of the oldest found fossils converges on the dating of earth, it may seem that life emerged quickly on earth, but that is not a logical conclusion.  If life formed as soon as the earth cooled sufficiently and no evidence of continuous emergence is found in the remaining billions of years, then Vernadsky's inference that life arrived on earth through one or more of the bodies that struck it becomes more probable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is the case, then one must ask the question, if all assumptions are dropped, whether life had a beginning at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Simulating Life Versus Simulating Its Formation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We may simulate what life is, that is, find an algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; to replicate &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a single live organism.  It is not realistic to, by brute force, simulate how life began without learning more about what conditions can lead to its formation theoretically to drastically reduce the soup simulation space.  It is that learning that is an ongoing area of research in the genetic algorithm field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early musings about the possibility of an algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}$&lt;/span&gt;, which can provide the conditions that allow an arbitrary organism &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; conforming to the above definition of life to form with out a parent or parents were interesting. Given algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; that simulates a life form and algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}$&lt;/span&gt; that simulate the formation of life, it may be the later algorithm that proves significantly more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conforming physics outside a computer to the simulation may be impossible.  Whether simulated life, when embodied in a robotic system is actually going to be considered life will be left to our descendants, should the species endure sufficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Classic cases include the heliocentric Copernican system giving way to the Law of Gravity, that law being shown an approximation of general relativity as shown by the proper prediction of the orbit of Mercury and light's curvature near the sun, the Four Elements dismissed in light of Lavoisier's discovery of oxygen, and absolute provability of truth within a closed symbolic system disproved by Gödel in his second incompleteness theorem and then recouped partially (in terms of computability) by Turing's completeness theorem.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-10-31T16:47:03.833" LastActivityDate="2018-10-31T16:47:03.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8701" PostTypeId="1" CreationDate="2018-11-01T00:42:18.080" Score="11" ViewCount="3312" Body="&lt;p&gt;In a CNN, the &lt;em&gt;receptive field&lt;/em&gt; is the portion of the image used to compute the filter's output. But one filter's output (which is also called a &quot;feature map&quot;) is the next filter's input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the difference between a receptive field and a feature map?&lt;/p&gt;&#xA;" OwnerUserId="19485" LastEditorUserId="2444" LastEditDate="2018-11-02T18:03:05.300" LastActivityDate="2018-11-06T15:37:56.630" Title="What is the difference between a receptive field and a feature map?" Tags="&lt;convolutional-neural-networks&gt;&lt;terminology&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8704" PostTypeId="1" CreationDate="2018-11-01T07:28:53.387" Score="1" ViewCount="225" Body="&lt;p&gt;I'm doing my thesis on &lt;code&gt;Reinforcement Learning&lt;/code&gt;. My focus on Partially Observable Environments like 3D Games. I want to choose a 3D platform for testing and doing research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know some of them. &lt;code&gt;DeepMind Lab&lt;/code&gt; and &lt;code&gt;OpenAi Universe&lt;/code&gt;. But my question is that which of these environments is good for me? Is there any environment for this purpose that is benchmark and reliable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want a platform that accepted in Academia and reliable. For example DeepMind is not a standard or Open Source friendly, Is it rational to use their platform for research in academia?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What i have to do?&lt;/p&gt;&#xA;" OwnerUserId="19493" LastActivityDate="2018-11-01T16:36:24.790" Title="3D environment for RL research in Academia" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;research&gt;&lt;deepmind&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8707" PostTypeId="1" AcceptedAnswerId="8826" CreationDate="2018-11-01T10:49:37.647" Score="6" ViewCount="4655" Body="&lt;p&gt;What is a learning agent, and how does it work? What are examples of learning agents (e.g., in the field of robotics)?&lt;/p&gt;&#xA;" OwnerUserId="19442" LastEditorUserId="2444" LastEditDate="2019-06-22T17:03:11.480" LastActivityDate="2019-06-22T17:14:06.723" Title="What is a learning agent?" Tags="&lt;ai-basics&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;theory&gt;&lt;intelligent-agent&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8712" PostTypeId="2" ParentId="6191" CreationDate="2018-11-01T16:06:10.560" Score="3" Body="&lt;p&gt;There are a few.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cvhci.anthropomatik.kit.edu/~baeuml/projects/a-universal-labeling-tool-for-computer-vision-sloth/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sloth&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Labelbox/Labelbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;LabelBox&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rectlabel.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RectLabel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://labelme.csail.mit.edu/Release3.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LabelMe&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;OpenCV also has some facilitation for annotating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Annotation is the more common name used in software suites for tools that facilitate adding labels to images and frames of movies because.  Simple categorization is of limited use in the real world, especially with moving pictures where action may be important and requires recognizing consistent position and size change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Annotation is more than adding a number or concept class instance in object recognition. There are five numbers that define the frame, potentially seven if both certainty and pixel depth layers are independently annotated as in the case of heat signatures.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt;: Box horizontal min&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt;: Box horizontal max&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v_0$&lt;/span&gt;: Box vertical min&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt;: Box vertical max&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;: Time or frame index&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;: Optional probability that the object in the box qualifies if the annotation involves doubt&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;: Optional pixel layer depth indicating range of wavelengths, important for discriminating warm blooded people and pets from immobile objects for vehicle automation or the heat signatures to discriminate ballistic missiles from benign aeronautics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-11-01T16:06:10.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8713" PostTypeId="2" ParentId="8689" CreationDate="2018-11-01T16:30:25.027" Score="0" Body="&lt;p&gt;Grouping the features of the concept class being learned can be used used to reduce the time required to learn the concept because it lowers the computation time of each sample and epoch.  That's done almost always when recognizing complex movements or with other concept classes where the number of dimensions in the model is high.  It is not a universal solution and care must be taken to configure the approach correctly.&lt;/p&gt;&#xA;&lt;p&gt;If there is a local minimum that is directed along a particular axis in the loss surface which is not the global minimum and that axis happens to be the axis corresponding to the first feature the practitioner unwittingly chooses to optimize.  The result will be that the other nine are learned immediately, yet the result is not optimal.  That's an unlikely scenario but only one of many scenarios where the technique can trouble the learning objective.&lt;/p&gt;&#xA;&lt;p&gt;Also, the interrelation between features of the concept class can be complex such that learning some of the features may cause the unlearning of others.  Since a larger loop must be placed around the entire process so that the accuracy of the features learned earlier is further tested until all the factors affecting loss are adjusted to minimize it, the process can take longer in some cases.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastEditorUserId="36737" LastEditDate="2021-04-19T22:38:30.177" LastActivityDate="2021-04-19T22:38:30.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8714" PostTypeId="2" ParentId="8704" CreationDate="2018-11-01T16:36:24.790" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For example DeepMind [Lab] is not a standard or Open Source friendly&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure where you got that info from... as far as I'm aware, DeepMind Lab is definitely used in various publications (maybe primarily publications from DeepMind, but still). Considering the &lt;a href=&quot;https://github.com/deepmind/lab/blob/master/LICENSE&quot; rel=&quot;nofollow noreferrer&quot;&gt;github repo has the GNU GPL 2 license&lt;/a&gt;, it also seems Open Source-friendly to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another framework of which I'm sure that it would widely be considered suitable within academia would be the &lt;a href=&quot;https://github.com/Unity-Technologies/ml-agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unity ML-Agents Toolkit&lt;/a&gt;, which uses the Unity game engine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose you could also consider using &lt;a href=&quot;http://vizdoom.cs.put.edu.pl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ViZDoom&lt;/a&gt;, which is also used in various publications, but (as far as I'm aware) it only supports one specific game (Doom).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have enough experience with using any of the above personally to be able to recommend one of them over the others... but they would all seem suitable to me.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-11-01T16:36:24.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8715" PostTypeId="2" ParentId="214" CreationDate="2018-11-01T16:55:48.237" Score="1" Body="&lt;p&gt;For there to be an objective test for AI intelligence, there has to be an objective test for human intelligence.  Each of the standard testing organizations claim theirs to be the most objective, but none of the test creators publish rigorous systems theory to verifiably justify their claim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The user stories for human selections are varied.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I want to find people to work on my team so the project is completed on time and in budget.&lt;/li&gt;&#xA;&lt;li&gt;I need to admit the right people for our training so we can have success stories to tell.&lt;/li&gt;&#xA;&lt;li&gt;We need a focus group with people who can communicate well in groups and think about our products and their market.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With artificial intelligence the bar is much lower.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We need to sort grapefruits for canning, fruit stand sale, use as a sweetener, or use as composting matter in a way to maximize profit.&lt;/li&gt;&#xA;&lt;li&gt;We need to beat the other automated drone pilot companies.&lt;/li&gt;&#xA;&lt;li&gt;Most of the world has online business but their documents are redundant with many nearly identical documents from copying and pasting, and most of them are poorly organized.  We need an automated librarian that organizes and indexes them.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The questioner may have hoped for some general intelligence test for AI better than Turing's imitation game, the current standardized intelligence testing only works on humans.  The AI that exists can't, without specifically training for the standardized tests they are going to take, achieve anything like a normal human score.  Cognition is something AI hopes to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some work in that area.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://slate.com/technology/2017/10/a-i-s-iq-is-still-somewhat-meaningless.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://slate.com/technology/2017/10/a-i-s-iq-is-still-somewhat-meaningless.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.06440.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1712.06440.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-11-01T16:55:48.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8716" PostTypeId="1" CreationDate="2018-11-01T16:56:50.253" Score="3" ViewCount="51" Body="&lt;p&gt;I have trained a convolutional neural network on images to detect emotions. Now I need to use the same network to extract features from the images and use them to train an LSTM. The problem is: the dimensions of the top layers are: &lt;code&gt;[None, 4, 4, 512]&lt;/code&gt; or &lt;code&gt;[None, 4, 4, 1024]&lt;/code&gt;. Therefore, extracting features from this layer will result in a &lt;code&gt;4 x 4 x 512 = 8192&lt;/code&gt; or &lt;code&gt;4 x 4 x 1024 = 16384&lt;/code&gt; dimensional vector for each image. Clearly, this is not what I want.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, I would like to know what to do in this case and how to extract features that are of reasonable size. Should I apply global average pooling to the activation or what?&lt;/p&gt;&#xA;&lt;p&gt;Any help is much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="16150" LastEditorUserId="32410" LastEditDate="2021-04-23T01:45:08.397" LastActivityDate="2021-05-23T03:01:45.140" Title="Problem extracting features from convolutional layer where the dimensions are big for feature maps" Tags="&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;feature-selection&gt;&lt;dimensionality&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8717" PostTypeId="2" ParentId="3406" CreationDate="2018-11-01T17:15:32.227" Score="4" Body="&lt;p&gt;A &lt;strong&gt;simplex reflex&lt;/strong&gt; agent takes actions based on current situational experiences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you set your smart bulb to turn on at some given time, let's say at 9 pm, the bulb won't recognize how the time is longer simply because that's the rule defined it follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple reflex agent doesn't compute complex computational problems nor exhibit intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;model-based&lt;/strong&gt; agent takes actions based on historical situational experiences. It performs actions basing on its internal state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;internal state&lt;/em&gt; of the model-based agent acts as a &quot;knowledge base&quot;, which allows actions to be performed by representing of unobserved aspects of the current state depending on percept history nor historical situational experiences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, a research-based planetary system can predict the planet's climatical conditions or weather basing on some geo-information knowledge base.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2019-06-22T17:11:06.873" LastActivityDate="2019-06-22T17:11:06.873" CommentCount="0" CommunityOwnedDate="2021-01-24T03:00:33.197" ContentLicense="CC BY-SA 4.0" />
  <row Id="8719" PostTypeId="1" CreationDate="2018-11-01T18:21:36.677" Score="0" ViewCount="51" Body="&lt;p&gt;It is an easy matter to make a paper look old, for example, using any of the techniques explained on this page of WikiHow: &lt;a href=&quot;https://www.wikihow.com/Make-Paper-Look-Old&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wikihow.com/Make-Paper-Look-Old&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is current AI sufficient to distinguish a fake old paper from a real one?&lt;/p&gt;&#xA;" OwnerUserId="19510" LastEditorUserId="2444" LastEditDate="2019-06-28T14:14:45.160" LastActivityDate="2019-06-28T14:17:56.420" Title="Can current AI techniques distinguish a fake old paper from a real one?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8720" PostTypeId="2" ParentId="8719" CreationDate="2018-11-01T19:55:59.847" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is current AI sufficient to distinguish a fake old paper from a real one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;It depends&lt;/em&gt;. Specifically, it depends on how much information you (or any automated system that feeds data to the AI), gathers about the paper you want to test, and how good that information is at distinguishing between real and fake old paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A model that distinguishes between cases in an input domain (which can be any consistent data, such as images of a subject) is called a &lt;em&gt;classifier&lt;/em&gt;. The current state of the art classifiers working with sensory data, &lt;strong&gt;if&lt;/strong&gt; fed enough training data of sufficient quality, can out-perform humans at certain tasks. This is a success of &quot;narrow AI&quot; over humans. This is not some magical ability of machine learning that always applies though, there are some strong caveats:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The task must be feasible given the data collected. Low-resolution photos of samples of the paper might not contain enough information for your example task at all.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When comparing with a human performing the same task, they must be given identical data. If a human would gain more data by interacting with the paper in your example, it can give the human an advantage that a narrow AI cannot easily replicate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The quality of training data &quot;ground truth&quot; must be higher than achievable by a single human. In your example, this could be achieved simply by knowing the provenance of all the training data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There often needs to be a large amount of training data to achieve better-than-human ability, although the actual amount required depends heavily on the task.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Expanding on the first point, image classifiers can sometimes be surprisingly good, they have different strengths and weaknesses compared to human vision, so it is &lt;em&gt;possible&lt;/em&gt; for them to sometimes be good at making classifications that humans are bad at. However, there is no guarantee of getting that kind of result for any specific task.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Expanding on the second point, if &lt;em&gt;interacting&lt;/em&gt; with a subject is allowed in order to classify it, this moves from the domain of &quot;narrow AI&quot; classifiers into active rational agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you gave a piece of paper to a human expert on old paper, and ask them to classify it, they might first look at the paper, then maybe sniff it, touch it . . . still undecided, so get out a magnifying glass to check a couple of features that might be good telltale signs . . . etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To build an equivalent artificial agent is harder than a single classifier: It &lt;em&gt;might&lt;/em&gt; be possible to purpose-build a collection of narrow AI classifiers, that output a class and confidence levels that decided whether certain tests should be carried out. It might even be possible to give a more general AI the same toolkit plus a very long time to experiment and it could learn for itself which tests to do in which order and how much to rely on the separate results. Or some hybrid of the two approaches. However, any such sophisticated system would still be bound by the limits of individual classifiers.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-06-28T14:17:56.420" LastActivityDate="2019-06-28T14:17:56.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8721" PostTypeId="1" AcceptedAnswerId="8730" CreationDate="2018-11-01T20:08:02.730" Score="3" ViewCount="713" Body="&lt;p&gt;In a &lt;code&gt;gym&lt;/code&gt; environment, the action space is often a discrete space, where each action is labeled by an integer. I cannot find a way to figure out the correspondence between action and number. For example, in frozen lake, the agent can move Up, Down, Left or Right. The code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import gym&#xA;env = gym.make(&quot;FrozenLake-v0&quot;)&#xA;env.action_space&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns &lt;code&gt;Discrete(4)&lt;/code&gt;, showing that four actions are available. If I &lt;code&gt;env.step(0)&lt;/code&gt;, which direction is my agent moving?&lt;/p&gt;&#xA;" OwnerUserId="19516" LastEditorUserId="2444" LastEditDate="2019-02-16T02:19:16.093" LastActivityDate="2019-02-16T02:19:16.093" Title="What is the mapping between actions and numbers in OpenAI's gym?" Tags="&lt;reinforcement-learning&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-11-17T22:01:31.230" ContentLicense="CC BY-SA 4.0" />
  <row Id="8722" PostTypeId="5" CreationDate="2018-11-01T20:12:33.820" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-01T20:12:33.820" LastActivityDate="2018-11-01T20:12:33.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8723" PostTypeId="4" CreationDate="2018-11-01T20:12:33.820" Score="0" Body="For questions on strong AI in relation to a specific problem.  (Distinct from the &quot;strong ai&quot; tag in that it does not imply superintelligence.)  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-01T20:12:33.820" LastActivityDate="2018-11-01T20:12:33.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8724" PostTypeId="5" CreationDate="2018-11-01T20:13:42.480" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#Utility-based_agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Intelligent_agent#Utility-based_agents&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-01T20:13:42.480" LastActivityDate="2018-11-01T20:13:42.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8725" PostTypeId="4" CreationDate="2018-11-01T20:13:42.480" Score="0" Body="Use for questions about utility-based agents." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-01T20:13:42.480" LastActivityDate="2018-11-01T20:13:42.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8726" PostTypeId="5" CreationDate="2018-11-01T20:14:31.913" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#Goal-based_agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Intelligent_agent#Goal-based_agents&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-01T20:14:31.913" LastActivityDate="2018-11-01T20:14:31.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8727" PostTypeId="4" CreationDate="2018-11-01T20:14:31.913" Score="0" Body="For questions about goal-based agents." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-01T20:14:31.913" LastActivityDate="2018-11-01T20:14:31.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8728" PostTypeId="2" ParentId="8619" CreationDate="2018-11-01T22:06:57.660" Score="0" Body="&lt;p&gt;Primordial replicators can be simpler than you think. Check out this video:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=w2lqZL153JE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self Replication: How molecules can make copies of themselves&lt;/a&gt; &#xA;&lt;br&gt;&lt;sub&gt;[Source: University of Groeningen]&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a noisy environment you get natural mutation. And voila, replication + mutation = evolution.&lt;/p&gt;&#xA;" OwnerUserId="2868" LastEditorUserId="1671" LastEditDate="2018-11-02T01:50:40.933" LastActivityDate="2018-11-02T01:50:40.933" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8730" PostTypeId="2" ParentId="8721" CreationDate="2018-11-01T22:16:14.270" Score="1" Body="&lt;p&gt;There is no way to tell via the Gym API, and to any RL-based learning agent this is entirely unimportant, discrete actions are just arbitrary labels, and their effects are learned by trial and error in terms of reward and changes to state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to know, you can read the code for each environment. Often it is described in a comment, or as constants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. in FrozenLake-v0, &lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/toy_text/frozen_lake.py#L8-L11&quot; rel=&quot;nofollow noreferrer&quot;&gt;you can find this code&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LEFT = 0&#xA;DOWN = 1&#xA;RIGHT = 2&#xA;UP = 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I &lt;code&gt;env.step(0)&lt;/code&gt;, which direction is my agent moving?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Left, according to those constant names at least.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-01T22:45:10.530" LastActivityDate="2018-11-01T22:45:10.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8732" PostTypeId="2" ParentId="8694" CreationDate="2018-11-01T23:23:00.207" Score="0" Body="&lt;p&gt;Given only the fact that model A has a higher accuracy than model B you should just use model A.  More information on the performance of the two classifiers should be provided for a better answer.  &lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2018-11-02T02:07:40.520" LastActivityDate="2018-11-02T02:07:40.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8733" PostTypeId="2" ParentId="8640" CreationDate="2018-11-02T02:30:27.607" Score="2" Body="&lt;p&gt;In my experience, GANs work really well for the scenario of semi-supervised learning, where you don't necessarily have labels for all your class &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; data, but you do have a balanced dataset. In my (limited) experience, you do have to have a balanced (in numbers) set of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; objects, even though you are not sure of the labels.&lt;/p&gt;&#xA;&lt;p&gt;And yes, GANs can overfit to outliers as well, especially in the absence of a lot of examples, so be cautious.&lt;/p&gt;&#xA;&lt;p&gt;Currently, the version that works best for me (in terms of GANs) is WGAN-GP or WGAN-LP in combination with Optimistic Mirror Descent Adam (&lt;a href=&quot;https://discuss.pytorch.org/t/optimistic-mirror-descent-in-saddle-point-problems-adam-optimizer-modification/26928&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, Ncritic/Nactor = 1). Take a look at the paper by &lt;a href=&quot;http://cs231n.stanford.edu/reports/2017/pdfs/313.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adiwardana et al.&lt;/a&gt;, especially Fig. 7 (astonishing!), for semi-supervised learning with a limited number of class labels.&lt;/p&gt;&#xA;" OwnerUserId="8577" LastEditorUserId="2444" LastEditDate="2021-01-17T20:21:52.940" LastActivityDate="2021-01-17T20:21:52.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8734" PostTypeId="2" ParentId="8716" CreationDate="2018-11-02T06:50:45.177" Score="0" Body="&lt;p&gt;I think it's okay to let CNN extract a big number of feature maps, then you can reduce the features from it. To reduce features from CNN result, you can do feature selection or feature extraction. Some people prefer to use feature selection and select the most significant feature maps, for examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/7966100&quot; rel=&quot;nofollow noreferrer&quot;&gt;Action unit selective feature maps in deep networks for facial expression recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/7823876&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Network Simplification Based on Feature Maps Selection&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2018-11-02T06:50:45.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8735" PostTypeId="1" CreationDate="2018-11-02T07:23:32.690" Score="2" ViewCount="3849" Body="&lt;p&gt;When should the iterative deepening search (IDS), also called  iterative deepening depth-first search (IDDFS), and the depth-limited search be used?&lt;/p&gt;&#xA;" OwnerUserId="19442" LastEditorUserId="2444" LastEditDate="2018-11-11T08:01:28.930" LastActivityDate="2018-12-12T13:51:09.003" Title="When should the iterative deepening search and the depth-limited search be used?" Tags="&lt;algorithm&gt;&lt;ai-basics&gt;&lt;search&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8737" PostTypeId="2" ParentId="8701" CreationDate="2018-11-02T08:02:27.067" Score="6" Body="&lt;p&gt;I hope this illustration will help you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jQSUZ.png&quot; width=&quot;500&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Receptive field(s)&lt;/strong&gt;: is a small portion of the input to produce &lt;strong&gt;only one node in a feature map&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Feature map(s)&lt;/strong&gt;: is a convolutional process output, a feature map can be said as a feature representation of filter's input. &lt;strong&gt;One feature map consists of many filter's outputs (from different receptive fields)&lt;/strong&gt; from one kernel. The number of feature maps depends on the number of the kernel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So even feature maps are the next filter's input, but the next receptive fields is not a feature map. &lt;strong&gt;The next receptive field consists of a small portion node from different feature maps&lt;/strong&gt; (not only one feature map).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also, we can see from the illustration above, a feature map has two-dimensional size &lt;span class=&quot;math-container&quot;&gt;$(46 \times 46)$&lt;/span&gt;, then a receptive field size will always three-dimensional &lt;span class=&quot;math-container&quot;&gt;$(5 \times 5 \times \text{Number Of Feature Maps})$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2018-11-02T22:34:46.353" LastActivityDate="2018-11-02T22:34:46.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8738" PostTypeId="1" AcceptedAnswerId="8750" CreationDate="2018-11-02T08:06:45.560" Score="3" ViewCount="212" Body="&lt;p&gt;From the paper &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human level Control through DeepRL&lt;/a&gt;,  the correlation in the data causes &lt;code&gt;instability&lt;/code&gt; in the network and may causes the network to &lt;code&gt;diverge&lt;/code&gt;. I wanted to understand what does this &lt;code&gt;instability&lt;/code&gt; and &lt;code&gt;divergence&lt;/code&gt; mean ? And why correlated data causes this instability.&lt;/p&gt;&#xA;" OwnerUserId="19523" LastActivityDate="2018-11-03T08:21:03.843" Title="Reason for issues with correlation in the dataset in DQN" Tags="&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8739" PostTypeId="2" ParentId="3435" CreationDate="2018-11-02T08:25:16.393" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/facebook/bAbI-tasks#id17&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; repository maintained by Facebook AI Research talks about how they went about generating QA from stories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, they try to simulate how a reader reads a story. They also keep track about the knowledge the reader is assimilating when reading. Then they frame a question based on the knowledge assimilated, in order to asses if the reader can perform logical inference given what knowledge is present with him.&lt;/p&gt;&#xA;" OwnerUserId="7330" LastActivityDate="2018-11-02T08:25:16.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8740" PostTypeId="2" ParentId="8694" CreationDate="2018-11-02T10:45:37.110" Score="1" Body="&lt;p&gt;Without any additional information, lean towards the vote of the best performing classifier when it comes to ties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as others have stated already, it is best to analyze the performances in more detail (e.g. confusion matrices). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, it could be that model B almost always classifies class X correctly (hardly any false positives). In that case, you could lean towards the prediction of model B if it predicts class X and model A does not. In other words; you could weigh the votes of the models based on how well they did in similar, previous predictions. &lt;/p&gt;&#xA;" OwnerUserId="18398" LastEditorUserId="18398" LastEditDate="2018-11-02T11:42:05.923" LastActivityDate="2018-11-02T11:42:05.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8744" PostTypeId="1" CreationDate="2018-11-02T15:41:12.840" Score="1" ViewCount="567" Body="&lt;p&gt;Assume one is using transfer learning via a model which was trained on imagenet.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Assume that the pre-processing which was used to achieve the pretrained model contained z-score standardization using some mean and std which was calculated on the training data.&lt;br&gt;&#xA;Should one apply the same transformation on their new data?&lt;br&gt;&#xA;Should they apply z-score standardization using a mean and std of their own training data?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assume that the pre-processing now did not contain any standardization.&lt;br&gt;&#xA;Should one apply no standardization on their new data as well?&lt;br&gt;&#xA;Or should one apply z-score standardization, using the mean and std of their new data, and expect better results?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For example i've seen that the Inception V3 model which was trained by Keras did not use any standardization, and i'm wondering if using z-score standardization on my new data could yield better results.&lt;/p&gt;&#xA;" OwnerUserId="19531" LastActivityDate="2020-04-25T19:07:50.863" Title="How should one standardize input when transfer learning" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8746" PostTypeId="2" ParentId="8744" CreationDate="2018-11-02T17:14:20.467" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;You should use the same transformation (i.e same mean and std in case of z-score normalization) on the new data while using any pretrained model.&lt;/li&gt;&#xA;&lt;li&gt;If the pretrained model was trained without any normalization then it does not matter if you use normalization or not as long as range of input data is fixed (0-255 for the 8-bit input images). You can use pretrained models with batch normalization which takes care of internal covariance shift and thus there is no need to normalize data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19536" LastActivityDate="2018-11-02T17:14:20.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8747" PostTypeId="1" CreationDate="2018-11-02T17:19:51.197" Score="3" ViewCount="167" Body="&lt;p&gt;I am working on a project for price movement forecasting and I am stuck with poor quality predictions.&lt;/p&gt;&#xA;&lt;p&gt;At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the sequence of the last 45-60 observations. I tested several different ideas, but they all seems to give similar results. The model is trained to minimize MSE.&lt;/p&gt;&#xA;&lt;p&gt;For each idea I tried a model predicting 1 step at a time where each prediction is fed back as an input for the next prediction, and a model directly predicting the next 10 steps(multiple outputs). For each idea I also tried using as input just the moving average of the previous prices, and extending the input to input the order book at those time-steps.&#xA;Each time-step corresponds to a second.&lt;/p&gt;&#xA;&lt;p&gt;These are the results so far:&lt;/p&gt;&#xA;&lt;p&gt;1- The first attempt was using as input the moving average of the last N steps, and predict the moving average of the next 10.&#xA;At time t, I use the ground truth value of the price and use the model to predict t+1....t+10&lt;/p&gt;&#xA;&lt;p&gt;This is the result:&lt;/p&gt;&#xA;&lt;p&gt;Predicting moving average:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YOXxu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YOXxu.png&quot; alt=&quot;image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;On closer inspection we can see what's going wrong:&lt;/p&gt;&#xA;&lt;p&gt;Prediction seems to be a flat line. Does not care much about the input data:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m4Xb5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m4Xb5.png&quot; alt=&quot;image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The second attempt was trying to predict differences, instead of simply the price movement. The input this time instead of simply being X[t] (where X is my input matrix) would be X[t]-X[t-1].&#xA;This did not really help.&#xA;The plot this time looks like this:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Predicting differences:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/toFGz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/toFGz.png&quot; alt=&quot;image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But on close inspection, when plotting the differences, the predictions are always basically 0.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eSqGU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eSqGU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;At this point, I am stuck here and running our of ideas to try. I was hoping someone with more experience in this type of data could point me in the right direction.&lt;/p&gt;&#xA;&lt;p&gt;Am I using the right objective to train the model? Are there any details when dealing with this type of data that I am missing?&#xA;Are there any &amp;quot;tricks&amp;quot; to prevent your model from always predicting similar values to what it last saw? (They do incur in low error, but they become meaningless at that point).&lt;/p&gt;&#xA;&lt;p&gt;At least just a hint on where to dig for further info would be highly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is my config&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &amp;quot;data&amp;quot;: {&#xA;        &amp;quot;sequence_length&amp;quot;:30,&#xA;        &amp;quot;train_test_split&amp;quot;: 0.85,&#xA;        &amp;quot;normalise&amp;quot;: false,&#xA;        &amp;quot;num_steps&amp;quot;: 5&#xA;    },&#xA;    &amp;quot;training&amp;quot;: {&#xA;        &amp;quot;epochs&amp;quot;:200,&#xA;        &amp;quot;batch_size&amp;quot;: 64&#xA;    },&#xA;    &amp;quot;model&amp;quot;: {&#xA;        &amp;quot;loss&amp;quot;: &amp;quot;mse&amp;quot;,&#xA;        &amp;quot;optimizer&amp;quot;: &amp;quot;adam&amp;quot;,&#xA;        &amp;quot;layers&amp;quot;: [&#xA;            {&#xA;                &amp;quot;type&amp;quot;: &amp;quot;lstm&amp;quot;,&#xA;                &amp;quot;neurons&amp;quot;: 51,&#xA;                &amp;quot;input_timesteps&amp;quot;: 30,&#xA;                &amp;quot;input_dim&amp;quot;: 101,&#xA;                &amp;quot;return_seq&amp;quot;: true,&#xA;                &amp;quot;activation&amp;quot;: &amp;quot;relu&amp;quot;&#xA;            },&#xA;            {&#xA;                &amp;quot;type&amp;quot;: &amp;quot;dropout&amp;quot;,&#xA;                &amp;quot;rate&amp;quot;: 0.1&#xA;            },&#xA;            {&#xA;                &amp;quot;type&amp;quot;: &amp;quot;lstm&amp;quot;,&#xA;                &amp;quot;neurons&amp;quot;: 51,&#xA;                &amp;quot;activation&amp;quot;: &amp;quot;relu&amp;quot;,&#xA;                &amp;quot;return_seq&amp;quot;: false&#xA;            },&#xA;            {&#xA;                &amp;quot;type&amp;quot;: &amp;quot;dropout&amp;quot;,&#xA;                &amp;quot;rate&amp;quot;: 0.1&#xA;            },&#xA;            {&#xA;                &amp;quot;type&amp;quot;: &amp;quot;dense&amp;quot;,&#xA;                &amp;quot;neurons&amp;quot;: 101,&#xA;                &amp;quot;activation&amp;quot;: &amp;quot;relu&amp;quot;&#xA;            },&#xA;            {&#xA;                &amp;quot;type&amp;quot;: &amp;quot;dense&amp;quot;,&#xA;                &amp;quot;neurons&amp;quot;: 101,&#xA;                &amp;quot;activation&amp;quot;: &amp;quot;linear&amp;quot;&#xA;            }&#xA;        ]&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Notice the last layer with 101 neurons. It is not an error. We just want to predict the features as well as the price. In other words, we want to predict the price for time t+1 and use the features predicted to predict the price and new features at time t+2, ...&lt;/p&gt;&#xA;" OwnerUserId="18765" LastEditorUserId="2444" LastEditDate="2021-04-23T11:45:57.763" LastActivityDate="2021-04-23T11:45:57.763" Title="Price Movement Forecasting Issue" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;&lt;dropout&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8748" PostTypeId="1" AcceptedAnswerId="8835" CreationDate="2018-11-02T17:42:52.453" Score="4" ViewCount="156" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;“Funding artificial intelligence is real stupidity.”&#xA;  -- &lt;a href=&quot;https://en.wikipedia.org/wiki/John_R._Pierce&quot; rel=&quot;nofollow noreferrer&quot;&gt;John R. Pierce&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Was this computer pioneer way off the mark? – or was there important &lt;em&gt;sub-text&lt;/em&gt; there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pierce was an expert for machine translation in the 1960s. He coauthored the following paper Pierce, John R., and John B. Carroll. &quot;Language and machines: Computers in translation and linguistics.&quot; (1966). That means, he worked on the domain of AI but explained his subject as useless. Perhaps Marvin Minsky, Rodney Brooks and Sebastian Thrun would agree to him?&lt;/p&gt;&#xA;" OwnerUserId="19510" LastEditorUserId="1671" LastEditDate="2018-11-08T00:21:18.267" LastActivityDate="2018-11-08T01:17:04.373" Title="Was Pierce way off the mark?" Tags="&lt;philosophy&gt;&lt;history&gt;&lt;soft-question&gt;&lt;mythology-of-ai&gt;&lt;quotes&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8749" PostTypeId="1" CreationDate="2018-11-02T18:19:26.173" Score="1" ViewCount="359" Body="&lt;p&gt;I use PyTorch, bauces AllenNLP is built on it and good libraries are for it. But can AutoKeras be used for PyTorch based ML pipelines, or am I required to switch to Keras? Google is quite silent when asked for tutorials for this combination. Maybe PyTorch is lacking automated ML framework?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2018-11-03T14:29:08.817" Title="Can AutoKeras be used for neural networks of PyTorch" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;automation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8750" PostTypeId="2" ParentId="8738" CreationDate="2018-11-02T18:36:58.797" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I wanted to understand what does this &lt;code&gt;instability&lt;/code&gt; and &lt;code&gt;divergence&lt;/code&gt; mean ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These are with reference to learning curves for the neural network. If a neural network is &lt;code&gt;stable&lt;/code&gt; and &lt;code&gt;converges&lt;/code&gt;, it means that the value of the error or cost function reduces consistently over time and reaches a stable point of minimum error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice this is often a noisy process and not smooth. There are degrees of noise that are expected and acceptable when solving real problems. However, an &lt;code&gt;instable&lt;/code&gt; learning curve will oscillate wildly, and a &lt;code&gt;divergent&lt;/code&gt; learning curve will get consistently worse error function during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical stable, converging learning curve, cost function versus training data consumed, might look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2vDTG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2vDTG.png&quot; alt=&quot;Imaginary stable converging learning curve&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst an instable, diverging learning curve might look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GCWJ2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GCWJ2.png&quot; alt=&quot;Imaginary instable diverging learning curve&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are not using the same vertical scale - the lowest point of the unstable curve will typically be higher than most or even all of the stable curve. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;And why correlated data causes this instability.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is because for gradient descent to work, it needs gradient samples that it uses on every weight update step to be unbiased estimates of a true gradient. In RL you have either an online learning process or non-stationary targets (and usually both), so you must use stochastic or mini-batch gradient descent, working with a few samples at a time. You need those samples to be independent, not related to each other other than by random chance, otherwise the gradient value will be biased and gradient descent will consistently make updates in the wrong overall direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good way to illustrate the difference is to use a really simple example, using gradient descent updates to estimate a mean value (this is roughly equivalent to training a neural network with one neuron, with weight fixed to zero and learning a bias value to represent the mean of the target- no actual input is required).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say we have an array of values from 0 to 200 inclusive (example code is in Python):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;train_y = np.arange(0, 201)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this array is kept sorted, then sequential values are highly correlated. If you plot consecutive pairs against each other, you will get a straight line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can estimate the mean value by setting a &quot;bias&quot; value to some arbitrary number, and running an update rule based on MSE (between current bias and observed value):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mean_estimate = 0.0&#xA;alpha = 0.1&#xA;for y in train_y:&#xA;    mean_estimate += alpha * (y - mean_estimate)&#xA;print(mean_estimate)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This prints (roughly) &lt;span class=&quot;math-container&quot;&gt;$191$&lt;/span&gt; as the estimate for the mean, almost double.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we shuffle the array first, it removes the correlation. If you plot consecutive pairs against each other, you will get a scatter graph with no apparent pattern. Adding the one line &lt;code&gt;np.random.shuffle(x)&lt;/code&gt; to do this changes the results radically:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mean_estimate = 0.0&#xA;alpha = 0.1&#xA;np.random.shuffle(x)&#xA;for y in train_y:&#xA;    mean_estimate += alpha * (y - mean_estimate)&#xA;print(mean_estimate)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We get much better estimates (typically between &lt;span class=&quot;math-container&quot;&gt;$90$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$110$&lt;/span&gt;), closer to the true value of &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;, and not biased to be higher or lower (run it enough times and you would find the expected result of this algorithm is very close to the true value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because in the first version of the code, the gradient of the error was not sampled fairly - it kept pointing &quot;up&quot; even for high estimates due to the correlation. In the shuffled version, gradients are likely to be in either direction depending only on the current estimate, and will appear roughly in the ratios necessary to find the correct value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an exercise you could extend this simple example with mini-batches and repeated &quot;epochs&quot; to show that the effect persists with those changes, and the shuffling is the most important change here for better estimates. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-03T08:21:03.843" LastActivityDate="2018-11-03T08:21:03.843" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8751" PostTypeId="1" CreationDate="2018-11-02T19:52:14.770" Score="1" ViewCount="122" Body="&lt;p&gt;Until now, I always thought that Genetic Algorithm can be used for problems of which the solution space can be encoded (modeled) as a chromosome of a specific length. However, some people claim that they used GA for &lt;a href=&quot;https://www.codingame.com/multiplayer/optimization/code-vs-zombies&quot; rel=&quot;nofollow noreferrer&quot;&gt;this game&lt;/a&gt; and &lt;a href=&quot;https://www.codingame.com/multiplayer/bot-programming/coders-strike-back&quot; rel=&quot;nofollow noreferrer&quot;&gt;this game&lt;/a&gt;. They are basically games in which we control an agent on a 2-dimensional area. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the length of the genome sequence depends on how fast the game is finished. So, how is GA used for such games?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think GA is not the most suitable method for this kind of problems can you explain why and give better alternatives?&lt;/p&gt;&#xA;" OwnerUserId="19539" LastActivityDate="2019-03-06T08:16:22.553" Title="How to use Genetic Algorithm for varying lengths of solutions" Tags="&lt;game-ai&gt;&lt;genetic-algorithms&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8755" PostTypeId="1" AcceptedAnswerId="8758" CreationDate="2018-11-03T10:13:52.817" Score="5" ViewCount="6768" Body="&lt;p&gt;What is the uniform-cost search (UCS) algorithm? How does it work? I would appreciate seeing a graphical execution of the algorithm. How does the &lt;em&gt;frontier&lt;/em&gt; evolve in the case of UCS?&lt;/p&gt;&#xA;" OwnerUserId="19442" LastEditorUserId="2444" LastEditDate="2019-11-10T17:06:25.817" LastActivityDate="2020-04-22T14:54:53.513" Title="How does the uniform-cost search algorithm work?" Tags="&lt;definitions&gt;&lt;search&gt;&lt;uniform-cost-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8758" PostTypeId="2" ParentId="8755" CreationDate="2018-11-03T12:51:01.187" Score="4" Body="&lt;p&gt;Uniform Cost Search is also called the Cheapest First Search. For an example and entire explanation you can directly go to this link: &lt;a href=&quot;https://classroom.udacity.com/courses/cs271/lessons/48678875/concepts/487393290923&quot; rel=&quot;nofollow noreferrer&quot;&gt;Udacity - Uniform Cost Search&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://ai.stackexchange.com/questions/5949/what-is-the-fringe-in-the-context-of-search-algorithms/5951#5951&quot;&gt;this&lt;/a&gt; answer I have explained what a frontier is. To put it in simple words you can describe UCS algorithm as 'expanding the frontier only in the direction which will require the minimum cost to travel from initial point among all possible expansions' i.e. adding a point on the graph (which can be reached from the frontier without going through any other point) which has the shortest route from the initial point. We keep on doing this until a path has explored the goal frontier, this path is the cheapest path from the initial point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I strongly suggest you check out both the links for examples and better understanding.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="36345" LastEditDate="2020-04-22T14:54:53.513" LastActivityDate="2020-04-22T14:54:53.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8760" PostTypeId="2" ParentId="8749" CreationDate="2018-11-03T14:29:08.817" Score="2" Body="&lt;p&gt;&quot;Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I found a useful tool which collects different &lt;a href=&quot;https://github.com/ysh329/deep-learning-model-convertor&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep learning model convertors&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this will be useful for you :D&lt;/p&gt;&#xA;" OwnerUserId="19544" LastActivityDate="2018-11-03T14:29:08.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8761" PostTypeId="1" AcceptedAnswerId="8762" CreationDate="2018-11-03T14:41:31.623" Score="3" ViewCount="173" Body="&lt;p&gt;I am implementing an actor-critic reinforcement learning algorithm for winning a two player tic-tac-toe like game. The agent is trained against a min-max player and after a number of episodes is able to learn a set of rules which lead it to winning a good majority of games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as soon as I play against the trained agent by using even a slightly different playing style, it looses miserably. In other words, it is evident the agent overfitted with respect to the deterministic behaviour of the min-max player. It is clear to me what are &lt;a href=&quot;https://ai.stackexchange.com/questions/7963/how-can-a-reinforcement-learning-agent-generalize-if-it-is-trained-against-only&quot;&gt;the roots of the problem&lt;/a&gt;, but I would like to get an overview of the different methodologies which can be applied to overcome (or mitigate) this issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two solutions I would like to try are the following:&lt;br&gt;&#xA;1. Training the agent with different opponents for fixed amounts of episodes (or time) each. So that for example I train the agent by using a depth 2 min-max player for the first 10000 episodes, then I use a random playing agent for the next 10000 episodes, then I use a depth 4 min-max player for other 10000 episodes and repeat the process.&lt;br&gt;&#xA;2. Starting episodes from different initial configurations. In this way the agents will play a much wider set of sampled games and will be more difficult for the agent to overfit.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these two reasonable approaches? Are there other tricks/good practices to try out?&lt;/p&gt;&#xA;" OwnerUserId="19212" LastActivityDate="2018-11-03T17:01:24.127" Title="How to overcome overfitting to single player styles in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8762" PostTypeId="2" ParentId="8761" CreationDate="2018-11-03T17:01:24.127" Score="3" Body="&lt;p&gt;Both of the solutions you suggest seem to be built around the intuition that it's good to ensure that there is sufficient variety in the experiences that you provide to your RL algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That intuition is good, but it should not come at (too much of) a cost in playing strength of the opponent. I'm afraid that your first solution may break down because of this; Tic Tac Toe is such a simple game, any agent that doesn't play optimally can really be viewed as a very poor agent... and I think minimax agents with very low limits on search depth may end up playing suboptimally. Your second solution seems better in this regard, that could help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this particular case of Tic Tac Toe, I suspect you should be able to train just fine against only optimal minimax agents, as long as you make sure that those optimal minimax agents break ties randomly. In some situations, there may be multiple different actions that are all &quot;equally optimal&quot;. Then, you'll want to make sure that your minimax agents break ties randomly, rather than always deterministically picking the same action. This can be done, for example, by making sure to always shuffle your lists of legal moves in minimax after they are generated.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-11-03T17:01:24.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8768" PostTypeId="1" AcceptedAnswerId="8769" CreationDate="2018-11-04T08:20:46.963" Score="6" ViewCount="992" Body="&lt;p&gt;Which algorithm is used in the robot Sophia to understand and answer the questions?&lt;/p&gt;&#xA;" OwnerUserId="19448" LastEditorUserId="2444" LastEditDate="2021-01-26T18:04:59.260" LastActivityDate="2021-01-26T18:04:59.260" Title="Which algorithm is used in the robot Sophia to understand and answers the questions?" Tags="&lt;robotics&gt;&lt;robots&gt;&lt;question-answering&gt;&lt;sophia&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8769" PostTypeId="2" ParentId="8768" CreationDate="2018-11-04T10:59:47.977" Score="2" Body="&lt;p&gt;Sophia ,first , has all the questions and corresponding answers preprogrammed. It is a system which is a hybrid of Bayes Text classification and decision trees. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It may consist of a speak recognizer which converts the question into a string.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This string then travels into a algorithm which gets a suitable answer for it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The answer is then spoken by the speech synthesizer along with some face actions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It might also have a sentiment analyser which identifies sentiments in a answer to show face expressions accordingly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Along with this, it identifies the emotions on the face of the person sitting in front of it, so as to provide a greater expression.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hence, Sophia understands or thinks nothing like the human brain. It is just a system which gives predefined answers corresponding to the question asked.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="18985" LastEditorUserId="18985" LastEditDate="2018-11-10T01:34:55.990" LastActivityDate="2018-11-10T01:34:55.990" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8772" PostTypeId="2" ParentId="8658" CreationDate="2018-11-04T12:06:59.353" Score="0" Body="&lt;p&gt;I tried to sketch a mathematical justification of the equality. So we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\tau) = E_{\tau\sim p_\theta(\tau)}[r(\tau)]$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p_\theta(\tau) = p(s_1)\prod_{t = 1}^T \pi_{\theta}(a_t, s_t)p(s_{t+1}|s_t, a_t))$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, &lt;span class=&quot;math-container&quot;&gt;$p_\theta(\tau)$&lt;/span&gt; can be re-written in terms of the Markov Chain transition probabilities, namely: &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_\theta(\tau)= \prod_{t = 1}^T \pi_{\theta}(a_t, s_t)p(s_{t}|s_{t-1}, a_{t-1})) = \prod_{t = 1}^T p(s_{t}, a_{t})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here we focus on T = 2 (I guess it is not difficult to prove with a general T, maybe by induction over T). The following holds:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$J(\tau) = E_{\tau\sim p_\theta(\tau)}[r(\tau)] = E_{(s_1, a_1, s_2, a_2)\sim p(s_1,a_1)p(s_2,a_2)}[r(s_1,a_1) + r(s_2,a_2)] \\ &#xA;= \int_{(s_1,a_1)} \int_{(s_2,a_2)} (r(s_1,a_1) + r(s_2,a_2))p(s_1,a_1)p(s_2,a_2) d(s_1,a_1)d(s_2, a_2) \\&#xA;= \int_{(s_1,a_1)} \left( \int_{(s_2,a_2)} (r(s_1,a_1)  + r(s_2,a_2))p(s_2,a_2) d(s_2,a_2)\right)p(s_1,a_1)d(s_1, a_1) \\ &#xA;= E_{(s_1, a_1)\sim p(s_1,a_1)}\left[E_{(s_2, a_2)\sim p(s_2,a_2)}[r(s_1,a_1) + r(s_2,a_2)]\right] \\&#xA;= E_{(s_1, a_1)\sim p(s_1,a_1)} \left[E_{(s_2, a_2)\sim p(s_2,a_2)}[r(s_1,a_1)] + E_{(s_2, a_2)\sim p(s_2,a_2)}[r(s_2,a_2)] \right] \\&#xA;= E_{(s_1, a_1)\sim p(s_1,a_1)} \left[ r(s_1,a_1) + E_{(s_2, a_2)\sim p(s_2,a_2)}[r(s_2,a_2)] \right] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the second line is due to the definition of the expectation, in the third line we just changed the order of the terms, in the fourth we used the definition of expectation, in the fifth we used the linearity property of the expectation and the last line is because &lt;span class=&quot;math-container&quot;&gt;$r(s_1, a_1)$&lt;/span&gt; is constant with respect to the integration over &lt;span class=&quot;math-container&quot;&gt;$(s_2, a_2)$&lt;/span&gt;. I preferred to move to the integral form of the expectation in line two and three because it was not clear to me how exactly the expectation could factorise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude the justification, note that we defined &lt;span class=&quot;math-container&quot;&gt;$p(s_t, a_t) = \pi_{\theta}(a_t, s_t)p(s_{t}|s_{t-1}, a_{t-1})$&lt;/span&gt;, so we have another product here which can be factorised in the same way we did in line two and three of the set of equations above. This brings us the to conditional expectation form of the objective. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, every term we have in the original trajectory distribution &lt;span class=&quot;math-container&quot;&gt;$p_\theta(\tau)$&lt;/span&gt; corresponds to an expectation and every reward can be moved up to the first outer expectation which it depends on.&lt;/p&gt;&#xA;" OwnerUserId="19212" LastEditorUserId="19212" LastEditDate="2018-12-07T07:32:35.470" LastActivityDate="2018-12-07T07:32:35.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8775" PostTypeId="2" ParentId="8747" CreationDate="2018-11-04T16:08:58.183" Score="2" Body="&lt;p&gt;Given a model that takes in a price and a second value, such as a moving average of the price, the following configuration is my recommendation.  This is based on training on a history of 45 input time steps and forecasting out 10 steps in the future.  I have assumed that your 'num_steps' is a stride through the training data.  Note that I have not tested this and you will most likely need to tweak a parameter or two:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;data&quot;: {&#xA;        &quot;sequence_length&quot;: 55,&#xA;        &quot;train_test_split&quot;: 0.85,&#xA;        &quot;normalise&quot;: true,&#xA;        &quot;num_steps&quot;: 5&#xA;    },&#xA;    &quot;training&quot;: {&#xA;        &quot;epochs&quot;:200,&#xA;        &quot;batch_size&quot;: 64&#xA;    },&#xA;    &quot;model&quot;: {&#xA;        &quot;loss&quot;: &quot;mse&quot;,&#xA;        &quot;optimizer&quot;: &quot;adam&quot;,&#xA;        &quot;layers&quot;: [&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 100,&#xA;                &quot;input_timesteps&quot;: 45,&#xA;                &quot;input_dim&quot;: 2,&#xA;                &quot;return_seq&quot;: true,&#xA;                &quot;activation&quot;: &quot;relu&quot;&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.2&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 100,&#xA;                &quot;activation&quot;: &quot;relu&quot;,&#xA;                &quot;return_seq&quot;: false&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.2&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dense&quot;,&#xA;                &quot;neurons&quot;: 1,&#xA;                &quot;activation&quot;: &quot;linear&quot;&#xA;            }&#xA;        ]&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try this out and plot your model's loss and accuracy by epoch.  See &lt;a href=&quot;https://machinelearningmastery.com/display-deep-learning-model-training-history-in-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Display Deep Learning Model Training History in Keras&lt;/a&gt; for some information and code in this area.  Let me know if you are not using Keras and I will point you to another article/code solution.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-11-04T16:08:58.183" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8778" PostTypeId="1" CreationDate="2018-11-05T00:10:41.387" Score="2" ViewCount="107" Body="&lt;p&gt;With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well as the features. In other words, at time &lt;code&gt;t+1&lt;/code&gt;, we predict the price difference and the features for time &lt;code&gt;t+2&lt;/code&gt;. We use the predicted features from time &lt;code&gt;t+1&lt;/code&gt; to predict the price at time &lt;code&gt;t+2&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We got very excited, because we thought getting good results with the following graph&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xSVd8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xSVd8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We got problems in production and we wasn't known the problem till we plot the price difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PZu4F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PZu4F.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the content of the config file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;data&quot;: {&#xA;        &quot;sequence_length&quot;:30,&#xA;        &quot;train_test_split&quot;: 0.85,&#xA;        &quot;normalise&quot;: false,&#xA;        &quot;num_steps&quot;: 5&#xA;    },&#xA;    &quot;training&quot;: {&#xA;        &quot;epochs&quot;:200,&#xA;        &quot;batch_size&quot;: 64&#xA;    },&#xA;    &quot;model&quot;: {&#xA;        &quot;loss&quot;: &quot;mse&quot;,&#xA;        &quot;optimizer&quot;: &quot;adam&quot;,&#xA;        &quot;layers&quot;: [&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 51,&#xA;                &quot;input_timesteps&quot;: 30,&#xA;                &quot;input_dim&quot;: 101,&#xA;                &quot;return_seq&quot;: true,&#xA;                &quot;activation&quot;: &quot;relu&quot;&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.1&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 51,&#xA;                &quot;activation&quot;: &quot;relu&quot;,&#xA;                &quot;return_seq&quot;: false&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.1&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dense&quot;,&#xA;                &quot;neurons&quot;: 101,&#xA;                &quot;activation&quot;: &quot;relu&quot;&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dense&quot;,&#xA;                &quot;neurons&quot;: 101,&#xA;                &quot;activation&quot;: &quot;linear&quot;&#xA;            }&#xA;        ]&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Prices don't change very fast. Therefore, the next price is almost always very close to the last price. In other words, &lt;code&gt;P_{t+1} - P_{t&lt;/code&gt;} is very often close to zero or zero directly. If there is too many zeros then the network will only recognize the zeros. The model has picked up on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the model learned almost nothing except the very simple relationship that the next price is close to the last price. There is not necessarily anything wrong with the model. Predicting stock prices should be a very hard problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a straightforward improvement should be of taking the features as a whole instead of their difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to keep working with price difference instead of the price in itself because we are making the series potential more stationary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What might be a good solution to deal with the repetitive zeros related to our &quot;price difference&quot; problem? Does applying the log-return is a better idea than applying price differences?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does a zero inflated estimators is a good idea? First predict whether it's gonna be a zero. If not predict the value. &lt;a href=&quot;https://gist.github.com/fonnesbeck/874808&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/fonnesbeck/874808&lt;/a&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="18765" LastEditorUserId="18765" LastEditDate="2018-11-06T16:51:06.133" LastActivityDate="2021-03-25T17:05:50.607" Title="Price difference predictions curve almost vanished" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8779" PostTypeId="1" CreationDate="2018-11-05T00:24:30.483" Score="3" ViewCount="329" Body="&lt;p&gt;I was lately curious about a reinforcement learning approach that would solve maths equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I have the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f(g(h(w))) = 0 , with \ w = \begin{matrix}&#xA;a_{11} &amp;amp;  0  &amp;amp; \ldots &amp;amp; a_{1n}\\&#xA;0  &amp;amp;  a_{22} &amp;amp; \ldots &amp;amp; a_{2n}\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots\\&#xA;0  &amp;amp;   0       &amp;amp;\ldots &amp;amp; a_{nn}&#xA;\end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along additional constraints on the 3 functions like &#xA;&lt;span class=&quot;math-container&quot;&gt;$ f &amp;lt; g $&lt;/span&gt;;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ h &amp;gt; 2 * g $&lt;/span&gt;; and&#xA;&lt;span class=&quot;math-container&quot;&gt;$ f, g,h $&lt;/span&gt; not constant&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The goal is to find the 3 functions expressions given a specific matrix &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and the constraints.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use reinforcement learning to find a solution or solution&lt;strong&gt;s&lt;/strong&gt; to this problem ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="19579" LastEditorUserId="19579" LastEditDate="2018-11-06T22:09:44.857" LastActivityDate="2018-11-06T22:09:44.857" Title="Solving equations using reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8781" PostTypeId="1" CreationDate="2018-11-05T08:32:43.027" Score="2" ViewCount="73" Body="&lt;p&gt;I'd like to explore the possibilities of applying artificial intelligence to ocr reading.&#xA;Basic ocr invoices processing let me convert 30% of them only. &#xA;The main purpose is defining invoices areas by training an ai, then process those areas with ocr.&#xA;So I am looking into ai to define and recognize a document topology first, then apply ocr locally.&#xA;From a brief search, it is classified as zonal or template ocr.&#xA;Any chance of a premade open source library?&lt;/p&gt;&#xA;" OwnerUserId="19589" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:53.447" LastActivityDate="2020-05-29T13:47:53.447" Title="zonal or template ocr invoices reading" Tags="&lt;neural-networks&gt;&lt;topology&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8783" PostTypeId="1" AcceptedAnswerId="8784" CreationDate="2018-11-05T09:10:34.903" Score="4" ViewCount="1028" Body="&lt;p&gt;In examples and tutorial about DQN, I've often noticed that during the experience replay (training) phase people tend to use stochastic gradient descent / online learning. (e.g. &lt;a href=&quot;https://medium.com/@gtnjuvin/my-journey-into-deep-q-learning-with-keras-and-gym-3e779cc12762&quot; rel=&quot;nofollow noreferrer&quot;&gt;link1&lt;/a&gt;, &lt;a href=&quot;https://keon.io/deep-q-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link2&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Sample minibatch from the memory&#xA;minibatch = random.sample(self.memory, batch_size)&#xA;# Extract informations from each memory&#xA;for state, action, reward, next_state, done in minibatch:&#xA;    # if done, make our target reward&#xA;    target = reward&#xA;    if not done:&#xA;      # predict the future discounted reward&#xA;      target = reward + self.gamma * \&#xA;               np.amax(self.model.predict(next_state)[0])&#xA;    # make the agent to approximately map&#xA;    # the current state to future discounted reward&#xA;    # We'll call that target_f&#xA;    target_f = self.model.predict(state)&#xA;    target_f[0][action] = target&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why can't they use mini batches instead?&#xA;I'm new to RL, but in deep learning people tends to use mini-batches as they would result in a more stable gradient. Doesn't the same principle apply to RL problems? Is the randomness/noise introduced actually beneficial to the learning process? Am I missing something, or are these sources all wrong?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not all the sources rely on stochastic gradient descent: e.g. keras-rl seems to rely on minibatches (&lt;a href=&quot;https://github.com/keras-rl/keras-rl/blob/master/rl/agents/dqn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/keras-rl/keras-rl/blob/master/rl/agents/dqn.py&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="19592" LastActivityDate="2018-11-05T10:09:37.380" Title="Deep Q-Learning: why don't we use mini-batches during experience reply?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8784" PostTypeId="2" ParentId="8783" CreationDate="2018-11-05T10:09:37.380" Score="5" Body="&lt;p&gt;DQN &quot;library&quot; implementations that I have seen do use mini-batches to train, and I would generally recommend this, as it usually strikes a reasonable balance between number of weight updates and accuracy of the gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your first link, and the code excerpt, the sample list is literally called &lt;code&gt;minibatch&lt;/code&gt;. However, the developer then goes on to make a series of individual SGD steps per item within that sample. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why can't they use mini batches instead? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They can, and it works just fine. In my DQN explorations I tend to use a single mini-batch per timestep, and the Adam optimiser to accelerate learning. I also use a mini-batch to calculate &lt;em&gt;TD target&lt;/em&gt; values.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm new to RL, but in deep learning people tends to use mini-batches as they would result in a more stable gradient. Doesn't the same principle apply to RL problems? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It does, but I don't know if this is the &lt;em&gt;most&lt;/em&gt; sample efficient that it could be. I prefer to use mini-batches and more complex optimisers because this runs faster (for a given number of replay training samples per time step), and I can run more time steps of the environment in less time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible for instance that a replay sample size of 32 per time step, with 4 mini-batch updates of 8 each would be the most &lt;em&gt;sample efficient&lt;/em&gt; approach for a specific problem (because it makes more updates from the given samples), but it might not be the most &lt;em&gt;CPU efficient&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the randomness/noise introduced actually beneficial to the learning process? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think so, but there are plenty of other tuning issues to consider in RL.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I missing something, or are these sources all wrong?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They are not necessarily &lt;em&gt;wrong&lt;/em&gt;, but may be CPU inefficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More likely in my opinion, your sources are explaining algorithms in principle, and did not want to add too many layers of optimisation, or too many components that may need tuning to work. There is a lot going on in a typical DQN agent, and having many performance optimisations can hide the details that the person is trying to teach.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-11-05T10:09:37.380" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8790" PostTypeId="1" CreationDate="2018-11-05T15:39:08.287" Score="1" ViewCount="61" Body="&lt;p&gt;I am trying to understand the proof of theorem 2.1 from this paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ross, Stéphane, and Drew Bagnell. &quot;Efficient reductions for imitation learning.&quot; Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The cost-to-go is given as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\pi) = \sum_{t=1}^{T}\mathbb{E}_{s\,\sim\, d^t_{\pi}(s)}\left[C_\pi(s)\right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper they use &lt;span class=&quot;math-container&quot;&gt;$\hat{\pi}=\pi$&lt;/span&gt; for the learned policy and &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt; for the expert policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the derivation they write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\pi)\leq \sum_{t=1}^{T}\{ p_{t-1}\mathbb{E}_{s\, \sim \, d_t(s)}\left[C_\pi(s) \right]+(1-p_{t-1})\}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\leq \sum_{t=1}^{T}\{ p_{t-1}\mathbb{E}_{s\, \sim \, d_t(s)}\left[C_{\pi^*}(s) \right]+p_{t-1}{\ell_t(s,\pi)}+(1-p_{t-1})\},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in which &lt;span class=&quot;math-container&quot;&gt;$p_{t-1}$&lt;/span&gt; is the probability of not not making an error with policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; up to the time &lt;span class=&quot;math-container&quot;&gt;$t-1$&lt;/span&gt;. And &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; is the surrogate 0-1 loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following steps are easy to follow, but how did they come up with these steps? &lt;/p&gt;&#xA;" OwnerUserId="19123" LastEditorUserId="2444" LastEditDate="2019-02-13T02:35:09.013" LastActivityDate="2019-02-13T02:35:09.013" Title="Understanding the proof of theorem 2.1 from the paper &quot;Efficient reductions for imitation learning&quot;" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8791" PostTypeId="1" AcceptedAnswerId="8792" CreationDate="2018-11-05T15:49:30.983" Score="6" ViewCount="745" Body="&lt;p&gt;What are the differences in scope between statistical AI and &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;classical AI&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real-world examples would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19448" LastEditorUserId="2444" LastEditDate="2019-11-16T23:48:44.517" LastActivityDate="2019-11-16T23:48:44.517" Title="What are the differences in scope between statistical AI and classical AI?" Tags="&lt;comparison&gt;&lt;statistical-ai&gt;&lt;symbolic-ai&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8792" PostTypeId="2" ParentId="8791" CreationDate="2018-11-05T16:58:31.963" Score="1" Body="&lt;p&gt;Statistical AI, arising from machine learning, tends to be more concerned with &quot;inductive&quot; thought: given a set of patterns, induce the trend. &#xA;Classical AI is the branch of artificial intelligence research that concerns itself with attempting to explicitly represent human knowledge in a declarative form i.e given a set of constraints, deduce a conclusion.&#xA; Another difference is that C++ tends to be a favorite language for statistical AI while LISP dominates in classical AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A system can't be more intelligent without displaying properties of both inductive and deductive thought. This lends many to believe that in the end, there will be some kind of synthesis of statistical and classical AI.&lt;/p&gt;&#xA;" OwnerUserId="19603" LastActivityDate="2018-11-05T16:58:31.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8793" PostTypeId="1" AcceptedAnswerId="8797" CreationDate="2018-11-05T17:42:12.927" Score="3" ViewCount="533" Body="&lt;p&gt;I am interested in making a simple chess engine using neural networks. I already have a fairly good value network but I can't figure out how to train a policy network. I know that Leela chess zero outputs the probability of any of the about 1800 possible moves. But how do you train such a network? How do you calculate the loss when you  only have the 1 move that was played in the game &#xA;to work with?&lt;/p&gt;&#xA;" OwnerUserId="19604" LastActivityDate="2018-11-06T10:18:36.773" Title="Chess policy network" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;chess&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8794" PostTypeId="1" CreationDate="2018-11-05T17:57:00.960" Score="2" ViewCount="23" Body="&lt;p&gt;For example Haar Cascade can be trained using only positive and negative examples, you don't need any bounding box annotations. But it not a deep learning approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example can be the most straight forward Image recognition model + sliding window. But it is very slow&lt;/p&gt;&#xA;" OwnerUserId="19605" LastActivityDate="2018-11-05T17:57:00.960" Title="Is there any deep learning object detection algorithms that can work without bounding boxes annotated data?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8796" PostTypeId="1" AcceptedAnswerId="11779" CreationDate="2018-11-05T20:18:14.980" Score="2" ViewCount="94" Body="&lt;p&gt;We all know &lt;em&gt;&lt;strong&gt;Information Filter&lt;/strong&gt;&lt;/em&gt; is a dual representation of Kalman Filter. The main difference between Information Filter and Kalman Filter is the way the Gaussian belief is represented. In Kalman Filter, the Gaussian belief is represented by their moments, whereas in Information Filter, the Gaussian belief is represented by the canonical form. The canonical form is comprised of an information matrix (&lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;) and an information vector (&lt;span class=&quot;math-container&quot;&gt;$\xi$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;As we all know, the name Kalman Filter is after Rudolf E. Kálmán, one of the primary developers of this theory.&lt;/p&gt;&#xA;&lt;p&gt;Now I want to know where the name information filter came from? Why it is called information filter? What type of information is attached to this filter? Is there any significance behind the nomenclature?&lt;/p&gt;&#xA;&lt;p&gt;I have the same question about the information matrix and information vector. Is there any significance behind the nomenclature?&lt;/p&gt;&#xA;&lt;p&gt;I already read &lt;a href=&quot;https://docs.ufpr.br/%7Edanielsantos/ProbabilisticRobotics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probabilistic Robotics&lt;/a&gt; by Sebastian Thrun. Chapter 3 Gaussian filter, in subsection 3.4 The Information Filter. There are many equations and theories but that does not tell us about the nomenclature.&lt;/p&gt;&#xA;" OwnerUserId="18384" LastEditorUserId="2444" LastEditDate="2020-09-12T14:12:14.413" LastActivityDate="2020-09-12T14:12:14.413" Title="Why is Information Filter called Information Filter?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;kalman-filter&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8797" PostTypeId="2" ParentId="8793" CreationDate="2018-11-06T08:20:05.017" Score="2" Body="&lt;p&gt;It may come as a surprise, after learning all about dynamic programming, temporal difference learning, SARSA/Q-learning, to then discover that there is yet another whole dimension to reinforcement learning (on top of choices for on-policy/off-policy, model-based/model-free, bootstrap/monte carlo etc). That is value-based vs policy-based methods. Policy-based methods are often taught &lt;em&gt;after&lt;/em&gt; value-based methods, because they are more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can learn the parameters of a policy function by training it using a policy gradient method. The archetypical policy gradient method is REINFORCE, although that is not very efficient. You may have heard of policy gradient methods developed recently: A3C, A2C, DDPG, TRPO, PPO . . . there are a few.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do you calculate the loss when you only have the 1 move that was played in the game to work with?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can &lt;em&gt;pre-train&lt;/em&gt; a policy network using supervised learning (perhaps using the moves of winning players in high quality games) - that would use multi-class cross-entropy loss that you may be familiar with from supervised classification problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy Gradient methods work with a reward summation function defined as expected reward given the distribution of states. If your network parameters are &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, then that may look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\theta) = \sum_{s \in \mathcal{S}} \rho_{\pi}(s) \sum_{a \in \mathcal{A}} \pi(a|s,\theta)q_{\pi}(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\rho_{\pi}(s)$&lt;/span&gt; is the expected proportion of time steps spent in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. There is a way to take a sample gradient of this that can be used for gradient &lt;em&gt;ascent&lt;/em&gt; - the derivation is called the &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy Gradient Theorem&lt;/a&gt;. It's a bit long to include in this answer, but the upshot is that you &lt;em&gt;can&lt;/em&gt; use your sampled single step to generate an approximate gradient towards improving the policy. There are a few variations, but for instance advantage actor critic uses this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla J(\theta) = \hat{A}(s,a)\nabla\text{log}(\pi(a|s,\theta))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat{A}(s,a)$&lt;/span&gt; is your current estimate of the &lt;em&gt;advantage&lt;/em&gt; (or &lt;span class=&quot;math-container&quot;&gt;$Q(s,a) - V(s)$&lt;/span&gt;) for taking a specific action in state s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The related loss function is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}(\theta) = -A(s,a)\text{log}(\pi(a|s,\theta))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\text{log}$&lt;/span&gt; function looks like an odd addition, but is just a consequence of adjusting &lt;span class=&quot;math-container&quot;&gt;$\nabla J$&lt;/span&gt; to take account of ratios in which actions are taken in the current policy. In fact &lt;span class=&quot;math-container&quot;&gt;$\nabla\text{log}(\pi(a|s,\theta)) = \frac{\nabla\pi(a|s,\theta)}{\pi(a|s,\theta)}$&lt;/span&gt; and it may help your intuition to keep it in that form (the &lt;span class=&quot;math-container&quot;&gt;$\text{log}$&lt;/span&gt; form is concise and used elsewhere in statistics as the &quot;score function&quot;, but it is not necessary for anything specific in RL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variations in policy gradients may use other functions than the advantage function, and it is not clear that there is any &quot;best&quot; one. The policy gradient theory basically gives us ways of estimating &lt;em&gt;relative&lt;/em&gt; benefits of actions, and allows for any offset to estimated return from &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; that does not further depend on the choice of action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. So you can use any method for getting an estimated return, and offset it with anything that you think might normalise the updates - common choices for the latter include subtracting average reward, or subtracting the state value function.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-06T10:18:36.773" LastActivityDate="2018-11-06T10:18:36.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8798" PostTypeId="1" CreationDate="2018-11-06T10:52:15.843" Score="1" ViewCount="446" Body="&lt;p&gt;I created a very simple bot to learn how to use chatterbot. This library already comes with a training, but I wanted extra training with an import of a corpus in Portuguese that I found in github.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from chatterbot import ChatBot&#xA;&#xA;bot = Futaba(&#xA;&quot;Terminal&quot;,&#xA;storage_adapter=&quot;chatterbot.storage.SQLStorageAdapter&quot;,&#xA;logic_adapters=[&#xA;&quot;chatterbot.logic.MathematicalEvaluation&quot;,&#xA;&quot;chatterbot.logic.TimeLogicAdapter&quot;,&#xA;&quot;chatterbot.logic.BestMatch&quot;&#xA;],&#xA;&#xA;input_adapter=&quot;chatterbot.input.TerminalAdapter&quot;,&#xA;output_adapter=&quot;chatterbot.output.TerminalAdapter&quot;,&#xA;database_uri=&quot;../database.db&quot;&#xA;)&#xA;&#xA;print(&quot;Type something to begin...&quot;)&#xA;&#xA;while True:&#xA;    try:&#xA;        bot_input = bot.get_response(None)&#xA;    except (KeyboardInterrupt, EOFError, SystemExit):&#xA;        break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's all I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I import this corpus into my chatbot?&lt;/p&gt;&#xA;" OwnerUserId="18233" LastActivityDate="2018-11-06T16:23:21.993" Title="How to add external training in chatterbot?" Tags="&lt;training&gt;&lt;python&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="5" ClosedDate="2020-06-02T22:58:39.967" ContentLicense="CC BY-SA 4.0" />
  <row Id="8800" PostTypeId="2" ParentId="3683" CreationDate="2018-11-06T13:50:43.133" Score="0" Body="&lt;p&gt;Neural networks are the one thing I would &lt;em&gt;not&lt;/em&gt; recommend. Your problem fits into the domain of &lt;a href=&quot;https://en.wikipedia.org/wiki/First-order_logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;predicate calculus&lt;/a&gt;, with some basic pattern recognition of the input sentences (assuming you only accept a certain type of sentences). You can do that without the need for masses of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, transform your statement into a canonical representation, probably using first-order logic and simple pattern matching/string matching. For example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I own a dog &#xA;&#xA;X own/owns a/an Y  &#xA;&#xA;owns(I, dog)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here you have a pattern &quot;&lt;em&gt;X&lt;/em&gt; own(s) a(n) &lt;em&gt;Y&lt;/em&gt;&quot;, which you recognise in your user input. You then have a predicate &lt;em&gt;owns(X, Y)&lt;/em&gt;, which you add to your database of statements. This database you can then query. For example, &lt;em&gt;Do I own a cat?&lt;/em&gt; could fit a question pattern &lt;em&gt;Do/Does X own(s) a(n) Y?&lt;/em&gt;, and you can search for &lt;em&gt;own(I, cat)&lt;/em&gt; in your DB; you will not find it, so the answer is &quot;No&quot;. If the question is &quot;Do I own a dog?&quot; you will find &lt;em&gt;owns(I, dog)&lt;/em&gt; in your database and you can reply &quot;Yes&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all rather 'old' technology, but I think you will find that you will get decent results much quicker than with machine learning or statistical methods, especially if you have not much data to begin with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A further branch to look into would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;expert systems&lt;/a&gt;. If you're thinking in terms of programming languages, then &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt; would be well suited for this, but any language should do, really.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2444" LastEditDate="2020-03-14T19:04:56.913" LastActivityDate="2020-03-14T19:04:56.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8801" PostTypeId="2" ParentId="8701" CreationDate="2018-11-06T15:37:56.630" Score="4" Body="&lt;p&gt;&lt;strong&gt;Receptive Field&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;Receptive Field&lt;/em&gt;, in the context of CNN mechanics, is the discrete range of input selected as input to the convolution kernel of a specific layer.  The range of a receptive field is a function of both position and size.  The range applies to one or more dimensions.&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Horizontal&lt;/li&gt;&#xA;&lt;li&gt;Vertical&lt;/li&gt;&#xA;&lt;li&gt;Frame index&lt;/li&gt;&#xA;&lt;li&gt;Time increment&lt;/li&gt;&#xA;&lt;li&gt;Pixel layer index&lt;/li&gt;&#xA;&lt;li&gt;Other dimensions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The position of the receptive field is varied systematically to select a subset of the range of indices in each dimension to cover the full range.&lt;sup&gt;2&lt;/sup&gt;  The size is matched to the input of the kernel operating upon it and therefore normally constant.&lt;sup&gt;3&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the three defining characteristics of receptive fields in CNNs.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Specifies the range of indices selected for input into the convolution kernel in terms of index position and size&lt;/li&gt;&#xA;&lt;li&gt;The size of index range in each dimension (usually odd and less than 20) matched to the kernel input size characteristics&lt;/li&gt;&#xA;&lt;li&gt;The position of index range in each dimension, systematically varied to cover the full range of information in all dimensions, usually varied by fixed increments&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that the term &lt;em&gt;Receptive Field&lt;/em&gt; originates from the bounds of signal representation of the visual field captured in biological systems.  In this context, the term &lt;em&gt;Receptive Field&lt;/em&gt; refers to the geometric range acquired through imaging devices or organs.  Just as an eye may scan a landscape, the selection of a position within the total field of information occurs in CNN design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The commonality between the biological and artificial is the use of varying the focus of attention over the total sensory space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Feature Map as a Representation of Features Extracted&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;Feature Map&lt;/em&gt; in this context is a map representing the features extracted through one or more layers of convolution.  The term may be inaccurately used for the intermediate output of kernels, but notice that the output of intermediate layers prior to layers that are pooled do not yet directly represent features.  In those stages, extraction is incomplete, so there is no direct relationship between values and features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term feature map is most accurately used when describing the output of the last pooling layer in a section of CNN layers.  This section may lie within a sequence of CNN sections or may otherwise be components in a larger system architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of feature mappings include these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Edges&lt;/li&gt;&#xA;&lt;li&gt;Appearance or disappearance (temporal domain)&lt;/li&gt;&#xA;&lt;li&gt;Object elements&lt;/li&gt;&#xA;&lt;li&gt;Motion trajectory&lt;/li&gt;&#xA;&lt;li&gt;Zoom&lt;/li&gt;&#xA;&lt;li&gt;Objects&lt;/li&gt;&#xA;&lt;li&gt;Actions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this use of the term, the mapping of features is relative to the dimensions of the information, the positional space over which the kernel was applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Feature Map as a Representation of a Transform in Discrete Hilbert Space&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the term &lt;em&gt;Feature Map&lt;/em&gt; is applied to the mapping accomplished by one or a set of CNN layers, the features at the output are mapped to the input, not to positions.  In this context, the mapping is the tensor transformation in Hilbert Space.  Notice that the map is not a representation of the signal but the representation of the transformation, the kernel and its learned parameters in their current state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To disambiguate this context from the previous one, it may be useful to use the term &lt;em&gt;Feature Mapping&lt;/em&gt; to indicate a tensor transformation rather than the output of such a transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overlap of the Terms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the output of a convolution section contains a map of features and is fed into another convolution layer or section, the feature map of one section becomes the full space over which the receptive field selects the subset of information to be fed into the next kernel as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that a feature map is a complete signal representing all of the data in its section of the network, where as a receptive field is most often a subset of the complete signal applied multiple times to cover the full signal's breadth in multiple dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two terms are not synonymous in any context but merely related by theory and practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Both position and size are specified in each of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; dimensions in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{I}^n$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n \ge 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] The selection of ranges within each dimension may be accomplished via looping in algorithms or through hardware solutions that performs windowing operations with DSP or GPU circuitry, possibly accomplished via hardware or firmware controlled parallel RISC operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Example and epoch indices can be bounded similarly in some CNN designs, but such indices are not, in the usual sense of the term, considered part of receptive fields.  Only the dimensions within each example are.  Also, input stream indices, such as camera identifier are not usually included as a dimension within a receptive field.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-06T15:37:56.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8802" PostTypeId="2" ParentId="8609" CreationDate="2018-11-06T16:02:33.707" Score="2" Body="&lt;p&gt;Object detection can conceivably imitate what the human visual system does.  Research along these lines began in the 1980s in multiple laboratories and was termed &lt;em&gt;Computer Vision&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do we see ... and detect things and their location so quick[?]  Is the reason that we have huge gigantic network in our brain and we are trained since birth to till now and still training[?]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Neurological visual systems continue to train as long as the eyes and neurons continue to function.  A person may, long after doing well in sports, learn how to catch glass objects that fall off a counter using peripheral vision, using the same neural pathways as were used for visually coordinated movement in sports but.  They are newly trained to the new cognitive intention of saving glassware and clean-up time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mapping of signals from the rods and cones to the visual cortex has been studied and some generalities have influenced artificial vision research and development.  The convolution kernels used in CNNs are partly derived from biological research.  In summary, the way mammals and other organisms with the equivalents of retinas see is via a set of transformations and pattern matching circuits.  The sequence of information types through biological visual networks can be summarized.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Light as a function of compound angle and time.&lt;/li&gt;&#xA;&lt;li&gt;Edges in both space and time&lt;/li&gt;&#xA;&lt;li&gt;Elements of objects and actions &amp;mdash; These are real world features, not in the machine learning sense but in the natural language use of the term features.&lt;/li&gt;&#xA;&lt;li&gt;Objects and actions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The development of object and action recognition in computers follows this same basic sequence.  The goal stated in the question is this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Achieve an AI to detect object in picture in human ways&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Something like this has been the goal of many government, academic, and corporate laboratories, however the ability to recognize objects in still pictures is not as valuable as detecting the trajectories and other expressions of objects (such as changes in facial expressions) in real time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Automated aiming, piloting, walking, and driving have been computer vision objectives since the first control systems were developed for anti-aircraft defense systems.  Considerable effort is being invested into how steering, breaking, and signaling can be automated in road vehicles.  A large array of household and industrial devices are being developed that rely on mapping the environment and navigating a robotic device through the map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stationary object recognition work is largely surrounding either categorizing images or drawing vector graphics from raster images, but these functions have limited application.  As mobile devices develop further and images on the web give way to videos on the web, a very strong and uninterrupted trend, the temporal dimension will continue to gain importance in the field computer vision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of these objectives are novel.  They are all based on animal capabilities, only some of which are uniquely human, and have been the subject of research since before the advent of digital computers.  However the feasibility of many designs has improved, largely due to a significant reduction in costs for fixed computing resources.  The vision system hardware of today costs roughly one thousandth of its 1990 cost.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-07T16:21:49.540" LastActivityDate="2018-11-07T16:21:49.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8803" PostTypeId="2" ParentId="8689" CreationDate="2018-11-06T16:12:57.300" Score="-1" Body="&lt;p&gt;The optimization of 10 features, as opposed to optimization of 3 features, will converge more slowly.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let's say I have a total of 10 features. Suppose I start the optimisation process using only 3 features. Can the addition of the 7 remaining ones improve the performance of the model (which, you can assume, might already be quite high).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The answer might be &lt;em&gt;no&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The accuracy and reliability of the convergence toward ground truth (a formalized objective used to guide the optimization process) when split into groups of three and seven features may be better or worse than when left as a group of ten.&lt;/li&gt;&#xA;&lt;li&gt;Except in rare cases, the results will not be the same.  The likelihood of identical results is so low it may never happen in the world in the next century except when conditions are arranged solely to cause it to occur.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Why then do may approaches group the dimensions of the result and converge on groups of axes, then another, then another, and back again to the first, iterating until the convergence goal is reached?  This approach is used to reduce the time and computing resources used to reach the optimal.  As the problem complexity increases, the use of groupings in this way is more common.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2019-05-06T16:49:59.693" LastActivityDate="2019-05-06T16:49:59.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8804" PostTypeId="2" ParentId="8798" CreationDate="2018-11-06T16:23:21.993" Score="1" Body="&lt;p&gt;I'm not sure if this is the right one, but according to the &lt;a href=&quot;https://github.com/gunthercox/ChatterBot&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; you need to create a trainer for it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from chatterbot.trainers import ChatterBotCorpusTrainer&#xA;&#xA;# Create a new trainer for the chatbot&#xA;trainer = ChatterBotCorpusTrainer(chatbot)&#xA;&#xA;# Train based on the english corpus&#xA;trainer.train(&quot;chatterbot.corpus.english&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Instead of &quot;chatterbot.corpus.english&quot; you obviously choose your own training data.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-11-06T16:23:21.993" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8805" PostTypeId="2" ParentId="8668" CreationDate="2018-11-06T16:34:27.110" Score="1" Body="&lt;p&gt;There were no pre-trained models for human that were taken from all angles (which is a somewhat extreme requirement) or specifically from above.  This condition may change due to the high value of such to the surveillance industry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are already several projects for which there is human specific pattern recognition and there have emerged some data sets to support this more specific field of objectives, some of which are based on moving rather than stationary images.  This is for two reasons.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multiple frames has the potential (should computing resource utilization be overcome) to produce higher accuracy and reliability in human recognition partly because of the identification of joints in movement and partly because there is simply more information available on each object that could be a human.&lt;/li&gt;&#xA;&lt;li&gt;Trajectory of pedestrians is important for both law enforcement and security and for avoiding pedestrians in vehicle automation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are some of the projects and download sites that are related to the goal of the question, but unfortunately do not seem to provide data sets of humans from above or pre-trained networks from that visual perspective.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://software.intel.com/en-us/openvino-toolkit/documentation/pretrained-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.intel.com/en-us/openvino-toolkit/documentation/pretrained-models&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/pretrained-cascade-image-classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/pretrained-cascade-image-classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/matterport/Mask_RCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/matterport/Mask_RCNN&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://drive.google.com/drive/folders/1zvl89AgFAApbH0At-gMuZSeQB_LpNP-M&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/drive/folders/1zvl89AgFAApbH0At-gMuZSeQB_LpNP-M&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://modelzoo.co/model/human-pose-estimation-with-tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://modelzoo.co/model/human-pose-estimation-with-tensorflow&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-06T16:34:27.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8807" PostTypeId="1" CreationDate="2018-11-06T18:50:04.937" Score="5" ViewCount="137" Body="&lt;p&gt;I'm building a weather station, where I'm sensing &lt;em&gt;temperature, humidity, air pressure, brightness, &lt;span class=&quot;math-container&quot;&gt;$CO_2$&lt;/span&gt;&lt;/em&gt;, but I don't have a raindrop sensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to create an AI which can say if it's raining or not, with the help of the given data above and maybe analyzing the slope from the last hour or something? Which specific technology should I use and how can I train it?&lt;/p&gt;&#xA;" OwnerUserId="19634" LastEditorUserId="2444" LastEditDate="2018-11-13T18:45:19.773" LastActivityDate="2018-11-13T18:45:19.773" Title="How do I predict if it is rainy or not?" Tags="&lt;training&gt;&lt;prediction&gt;&lt;models&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8808" PostTypeId="2" ParentId="7624" CreationDate="2018-11-06T18:55:12.107" Score="0" Body="&lt;p&gt;The most general definition of the term intelligence that is both terse and exact is this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The collection of behavioral features resident in some entity where the entity sustainably succeeds in specific pursuits while avoiding specific losses in a particular range of environmental conditions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These are examples of failures in exhibiting intelligence per the above definition, demonstrating the importance of each phrase.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some behavioral features that are intelligent but an overall system behavior that is not.  For example, a rocket that reaches an altitude but cannot enter orbit or a turtle that can retract its head but cannot catch a bug.&lt;/li&gt;&#xA;&lt;li&gt;The intelligence is scattered across entities such that each individual entity does not exhibit intelligence, such as a bee or a single.&lt;/li&gt;&#xA;&lt;li&gt;Intelligence is exhibited momentarily but disintegrates over time, not adapting to changing conditions, or is not sufficiently reliable to fill a practical role.&lt;/li&gt;&#xA;&lt;li&gt;The entity can attain a goal but such success is nullified by loss accumulated during their attainment.&lt;/li&gt;&#xA;&lt;li&gt;The entity can avert loss, but cannot reliably succeed in the pursuit of its objectives.&lt;/li&gt;&#xA;&lt;li&gt;An entity that can adapt to any environmental condition and act intelligently in every one does not exist.  Human intelligence is limited to specific scenarios and shock and confusion result when overburdened and super-intelligence is, as of this writing, conjecture without empirical evidence or theoretical proof.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Notice four things in this definition.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Optimality is not required.  Only better than random behavior is required.&lt;/li&gt;&#xA;&lt;li&gt;Although the entity tested for intelligence may interface with its environment only through data sets and test metrics, these are its environment.&lt;/li&gt;&#xA;&lt;li&gt;Time is necessarily involved.  In a simple case, an artificial network exhibits intelligence only in its ability to exhibit behavior that was previously learned to be sufficient.  Such can only retain intelligence by adjusting its training or in an environment where adaptation to new patterns is not required.&lt;/li&gt;&#xA;&lt;li&gt;Cognition is not required but cognition certainly augments the range of objectives that can be reliably pursued and the ability of the entity to detect danger and more proactively avert loss.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-06T18:55:12.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8809" PostTypeId="2" ParentId="8807" CreationDate="2018-11-06T19:58:28.057" Score="2" Body="&lt;p&gt;This sounds like a great project, although this &lt;em&gt;exact&lt;/em&gt; setup limits your options somewhat.  Supervised machine learning approaches are effectively ruled out because you don't have the necessary training data to develop a model (i.e. the dependent variable: whether it is raining or not).  You could look at accessing similar data (from what source depends on your geographical location) that includes precipitation information, and attempt to train a model based on that, before applying it to your own data to 'predict' the current conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a model to determine whether it's raining or not (rather than attempting to define the type, rate or other characteristics of precipitation) then you need a binary classification model.  If you can get the data to train a supervised model, there are many such models available.  Which one to choose depends on a number of factors – enough factors that it is as much an art as a science (and often the best way to choose is just to try several and pick the best one).  XGBoost is a popular choice for a wide range of problems, including binary classification.  State Vector Classifiers (SVCs, a subtype of State Vector Machine, SVM) are also common.  Of course there are neural networks also.  Depending on how the dependent variable correlates with the independent ones, you may find a simpler model to be more effective.  If you choose to try a neural network, I suggest keeping it simple and starting with a Multi-Layer Perceptron (MLP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could try an unsupervised approach, which commonly takes the form of clustering.  In this instance, you would look to try clustering algorithms, such as DBSCAN, to see if the independent variables naturally form groups (clusters) which differentiate between when it is raining or not.  Of course you can cluster the data using an unsupervised model, but you would still need to then correlate that with rainfall in order to make sense of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of challenges you may need overcome.  For example, depending on the frequency with which the data is sampled and stored, you may need to downsample it to a manageable size.  Presumably you also have the time at which the data was collected.  If not, you would also need to make use of the brightness sensor to differentiate between night and day, which may confound or over-power any cooling effect from rainfall.  (You could look at clustering the data, based on temperature and brightness, in order to create a new feature which indicates night and day.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarise, there are numerous approaches you could take, all of which will, at some point, require data on whether it is/was raining or not in order to train a model.  If you have suitably localised weather data available, you may be able to train a model using that, synchronised via time and/or the other data you'll be collecting.  The problem, as stated, is a binary classification one (as opposed to a time series prediction one, i.e. weather forecasting, for example).  So look into classification and clustering models for supervised and unsupervised machine learning respectively.  &lt;a href=&quot;http://scikit-learn.org/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scikit Learn&lt;/a&gt; for python or &lt;a href=&quot;https://cran.r-project.org/web/packages/caret/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Caret&lt;/a&gt; for R are commonly used packages for the types of models I've described.&lt;/p&gt;&#xA;" OwnerUserId="19572" LastActivityDate="2018-11-06T19:58:28.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8812" PostTypeId="2" ParentId="5885" CreationDate="2018-11-06T23:09:11.420" Score="1" Body="&lt;p&gt;Swarm intelligence, compound intelligence, or group intelligence may emerge as an important concept as AI develops toward higher complexity.  Whether these terms should be considered synonymous is doubtful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compound features in biology are the result of control in differentiation during the development of an organism from a single cell.  Compounding in biology is a single function performed across like elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Swarms are a result of distinct organisms operating in proximity.  Swarming in biology is similarity in complex independent behaviors that appear coordinated but are generally engaged in as a defense from predators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Group intelligence may be distinct from compound intelligence in that adjacent units may be in agreement or opposition, as can be case with intelligent beings in groups.  The agreement in the group that a variety of models is permissible.  We call them opinions, but they are distinct matches of models to problems that produce different projections and suggest different selections from among the group's options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms general and strong appear in this question and many others and it appears that, mainly for historical reasons, may continue to frustrate clarity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All intelligence is general in that it learns a generality from specific experiences and then applies that generality to future scenarios to achieve objectives within those scenarios.  What makes the application of the generality intelligent is that it is expected to work because it has been working for similar scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All intelligence is specific in that it is limited to scope of what generalities have been discovered.  Certain techniques are significantly more general that others because they are allegedly domain independent.  We call this mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, the development of artificial intelligence is not a path from specific to general but one of discovering generalities and applying them to specifics.  One could say that the most general thinking of mathematics applied to the computer is the primary activity of applied artificial intelligence, and the capabilities that emerge are a more sophisticated as more sophisticated mathematics is represented in working software and hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generalities take on greater complexity so that they can apply to a greater number of specific scenarios.  That is not a gain in strength but a gain in the breadth of potential application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Physalia physalis is a symbiotic colony of organisms of four types, the pneumatophore (or float), dactylozooids (long tentacles), gastrozooids (feeding tentacles), and gonozooids which produce reproductive gametes.  To avoid breaking the historic conception of animals, these organisms are called polyps.  The venom of the colony does not come from any of those organisms but rather from another symbiotic organism, cnidocytes, which attach to the tentacles and are released under strictly controlled conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to all this symbiosis, there are several species of fish that use the colony for shade and protection that the colony lets swim among the tentacles and that have formed partial immunity to the cnidocytes.  These complex symbiotic networks are not fully understood, but they seem to operate well and create sustainable inter-specie systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not likely that the organisms, within their lifetimes adapt or remember, but the DNA of each organism has involved in a way that resembles intelligence in that the colony and its symbionts have adapted to the ocean's surface and its biology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea that the system of the biosphere is the first example of broad intelligence is likely, in that design excellence has emerged from an evolutionary processes.  It is not altogether ridiculous to propose that human intelligence is a higher speed approach in neurons to the slower speed of DNA replication in larger organisms.  Because of the metabolic requirements of growth slows evolution for larger organisms with a greater cell count, these larger organisms may have needed to develop a way to achieve the nimble adaptivity of their lower ancestors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neurology facilitates the approximation of some aspects of evolution and may have been the most attainable natural solution to reacquire nimble adaptivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attempting to apply these various ideas to the current and ongoing development of autonomous vehicles reveals a gap in understanding.  We don't yet have the mathematics developed to understand how compound, swarm, or group intelligence can be used in the laboratory to accomplish well defined problems in controlled execution scenarios.  That is probably a prerequisite to using these ideas in vehicle control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system designs of future cars may be like a colony of independent or semi-independent components that each have a role and purpose.  Compound, group, and symbiotic designs are likely to develop.  The current activities of vehicles on the road or in the air near airports is like a swarm, so the application of that idea is obvious: Avoid collisions.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-06T23:09:11.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8814" PostTypeId="1" AcceptedAnswerId="8815" CreationDate="2018-11-07T01:38:17.627" Score="0" ViewCount="30" Body="&lt;p&gt;Currently, I found the right recipe for a time series regression problem to finally get acceptable to good results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the config file &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;data&quot;: {&#xA;        &quot;sequence_length&quot;:45,&#xA;        &quot;train_test_split&quot;: 0.85,&#xA;        &quot;normalise&quot;: false,&#xA;        &quot;num_steps&quot;: 10&#xA;    },&#xA;    &quot;training&quot;: {&#xA;        &quot;epochs&quot;:30,&#xA;        &quot;batch_size&quot;: 32&#xA;    },&#xA;    &quot;model&quot;: {&#xA;        &quot;loss&quot;: &quot;mse&quot;,&#xA;        &quot;optimizer&quot;: &quot;adam&quot;,&#xA;        &quot;layers&quot;: [&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 161,&#xA;                &quot;input_timesteps&quot;: 45,&#xA;                &quot;input_dim&quot;: 161,&#xA;                &quot;return_seq&quot;: true,&#xA;                &quot;activation&quot;: &quot;relu&quot;&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.1&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 161,&#xA;                &quot;activation&quot;: &quot;relu&quot;,&#xA;                &quot;return_seq&quot;: false&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dense&quot;,&#xA;                &quot;neurons&quot;: 128,&#xA;                &quot;activation&quot;: &quot;relu&quot;&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dense&quot;,&#xA;                &quot;neurons&quot;: 1,&#xA;                &quot;activation&quot;: &quot;linear&quot;&#xA;            }&#xA;        ]&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the results I got&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K19Ki.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K19Ki.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can be good improvements I can bring to my model so that I can get better results? There are a lot of small spikes and other places where the curve is rather constant that I should get a rise or fall of the curve.&lt;/p&gt;&#xA;" OwnerUserId="18765" LastEditorUserId="18765" LastEditDate="2018-11-07T01:59:47.427" LastActivityDate="2018-11-07T04:25:15.633" Title="Getting better results in improving the configuration" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8815" PostTypeId="2" ParentId="8814" CreationDate="2018-11-07T01:57:26.733" Score="1" Body="&lt;p&gt;Try using an RMSProp optimizer instead of Adam optimizer. Also try decreasing the batch size and keep a small learning rate like 0.001.&lt;/p&gt;&#xA;" OwnerUserId="18985" LastEditorUserId="18985" LastEditDate="2018-11-07T04:25:15.633" LastActivityDate="2018-11-07T04:25:15.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8820" PostTypeId="2" ParentId="4456" CreationDate="2018-11-07T11:35:11.767" Score="15" Body="&lt;p&gt;In reinforcement learning (RL), there is an &lt;em&gt;agent&lt;/em&gt; which interacts with an &lt;em&gt;environment&lt;/em&gt; (in time steps). At each time step, the agent decides and executes an &lt;em&gt;action&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, on an environment, and the environment responds to the agent by moving from the current &lt;em&gt;state&lt;/em&gt; (of the environment), &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, to the next state (of the environment), &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, and by emitting a scalar signal, called the &lt;em&gt;reward&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;.  In principle, this interaction can continue forever or until e.g. the agent dies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main goal of the agent is to collect the largest amount of reward &quot;in the long run&quot;. To do that, the agent needs to find an optimal policy (roughly, the optimal strategy to behave in the environment). In general, a policy is a function which, given a current state of the environment, outputs an action (or a probability distribution over actions, if the policy is &lt;em&gt;stochastic&lt;/em&gt;) to execute in the environment. A policy can thus be thought of as the &quot;strategy&quot; used by the agent to behave in this environment. An optimal policy (for a given environment) is a policy which, if followed, will make the agent collect the largest amount of reward in the long run (which is the goal of the agent). In RL, we are thus interested in finding optimal policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environment can be deterministic (that is, roughly, the same action in the same state leads to the same next state, for all time steps) or stochastic (or non-deterministic), that is, if the agent takes an action in a certain state, the resulting next state of the environment might not necessarily always be the same: there is a probability that it will be a certain state or another. Of course, these uncertainties will make the task of finding the optimal policy harder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In RL, the problem is often mathematically formulated as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;noreferrer&quot;&gt;Markov decision process&lt;/a&gt; (MDP). A MDP is a way of representing the &quot;dynamics&quot; of the environment, that is, the way the environment will react to the possible actions the agent might take, at a given state. More precisely, an MDP is equipped with a &lt;strong&gt;transition function&lt;/strong&gt; (or &quot;transition model&quot;), which is a function that, given the current state of the environment and an action (that the agent might take), outputs a probability of moving to any of the next states. A &lt;strong&gt;reward function&lt;/strong&gt; is also associated with an MDP. Intuitively, the reward function outputs a reward, given the current state of the environment (and, possibly, an action taken by the agent and the next state of the environment). Collectively, the transition and reward functions are often called the &lt;strong&gt;model&lt;/strong&gt; of environment. To conclude, the MDP is the problem and the solution to the problem is a policy. Furthermore, the &quot;dynamics&quot; of the environment are governed by the transition and reward functions (that is, the &quot;model&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we often do not have the MDP, that is, we do not have the transition and reward functions (of the MDP associated the environment). Hence, we cannot estimate a policy from the MDP, because it is unknown. Note that, in general, if we had the transition and reward functions of the MDP associated with the environment, we could exploit them and retrieve an optimal policy (using dynamic programming algorithms).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the absence of these functions (that is, when the MDP is unknown), to estimate the optimal policy, the agent needs to interact with environment and observe the responses of the environment. This is often referred to as the &quot;reinforcement learning problem&quot;, because the agent will need to estimate a policy by &lt;em&gt;reinforcing&lt;/em&gt; its beliefs about the dynamics of the environment. Over time, the agent starts to understand how the environment responds to its actions, and it can thus start to estimate the optimal policy. Thus, in the RL problem, the agent estimates the optimal policy to behave in an unknown (or partially known) environment by interacting with it (using a &quot;trial-and-error&quot; approach).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, a &lt;strong&gt;model-based&lt;/strong&gt; algorithm is an algorithm that uses the transition function (and the reward function) in order to estimate the optimal policy. The agent might have access only to an approximation of the transition function and reward functions, which can be learned by the agent while it interacts with the environment or it can be given to the agent (e.g. by another agent). In general, in a model-based algorithm, the agent can potentially predict the dynamics of the environment (during or after the learning phase), because it has an estimate of the transition function (and reward function). However, note that the transition and reward functions that the agent uses in order to improve its estimate of the optimal policy might just be approximations of the &quot;true&quot; functions. Hence, the optimal policy might never be found (because of these approximations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;model-free&lt;/strong&gt; algorithm is an algorithm that estimates the optimal policy without using or estimating the dynamics (transition and reward functions) of the environment. In practice, a model-free algorithm either estimates a &quot;value function&quot; or the &quot;policy&quot; directly from experience (that is, the interaction between the agent and environment), without using neither the transition function nor the reward function. A value function can be thought of as a function which evaluates a state (or an action taken in a state), for all states. From this value function, a policy can then be derived.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, one way to distinguish between model-based or model-free algorithms is to look at the algorithms and see if they use the transition or reward function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, let's look at the main update rule in the &lt;em&gt;Q-learning algorithm&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(S_t, A_t) \leftarrow Q(S_t, A_t) + \alpha (R_{t+1} + \gamma \max_{a}Q(S_{t+1}, a) - Q(S_t, A_t))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we can see, this update rule does not use any probabilities defined by the MDP. Note: &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; is just the reward that is obtained at the next time step (after taking the action), but it is not necessarily known beforehand. So, Q-learning is a model-free algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's look at the main update rule of the &lt;em&gt;policy improvement&lt;/em&gt; algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) \leftarrow  \sum_{s' \in \mathcal{S}, r\in\mathcal{R}}p(s',r|s,a)(r+\gamma V(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can immediately observe it uses &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt;, a probability defined by the MDP model. So, &lt;em&gt;policy iteration&lt;/em&gt; (a dynamic programming algorithm), which uses the policy improvement algorithm, is a model-based algorithm.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-27T21:52:40.953" LastActivityDate="2019-02-27T21:52:40.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8821" PostTypeId="1" AcceptedAnswerId="8909" CreationDate="2018-11-07T13:22:16.307" Score="5" ViewCount="7378" Body="&lt;p&gt;The iterative deepening A* search is an algorithm that can find the shortest path between a designated start node and any member of a set of goals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The A* algorithm evaluates nodes by combining the cost to reach the node and the cost to get from the node to the goal. How is iterative deepening A* better than the A* algorithm?&lt;/p&gt;&#xA;" OwnerUserId="19652" LastEditorUserId="2444" LastEditDate="2019-04-16T09:41:23.100" LastActivityDate="2019-04-16T09:41:23.100" Title="How is iterative deepening A* better than A*?" Tags="&lt;search&gt;&lt;comparison&gt;&lt;a-star&gt;&lt;ida-star&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8823" PostTypeId="1" CreationDate="2018-11-07T15:13:22.593" Score="1" ViewCount="32" Body="&lt;p&gt;My understanding is that the conjugate gradient method is faster than gradient descent because it does less zig zags while descending. How come the state of the art papers I see all use gradient descent for backpropagation and not conjugate gradient descent?&lt;/p&gt;&#xA;" OwnerUserId="19167" LastEditorUserId="19167" LastEditDate="2018-11-07T22:15:42.273" LastActivityDate="2018-11-07T22:15:42.273" Title="Neural network backpropagation gradient descent better than conjugate gradient descent?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8825" PostTypeId="1" CreationDate="2018-11-07T17:04:09.217" Score="1" ViewCount="87" Body="&lt;p&gt;Is there a good reference / tutorial for using RNN/LSTM to determine lag interval for 2 time series? E.g. I have {x_n}, {y_n} and I want to figure out by how much does {x_n} typically lags behind {y_n}?&lt;/p&gt;&#xA;" OwnerUserId="11478" LastActivityDate="2018-11-08T04:05:13.020" Title="RNN and LSTM for discovering time lag" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8826" PostTypeId="2" ParentId="8707" CreationDate="2018-11-07T18:01:24.147" Score="-1" Body="&lt;p&gt;&lt;strong&gt;What is a Learning Agent?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Leaning Agent is a tool in Artificial Intelligence, that is capable of learning from its experiences. It starts with some basic knowledge and is then able to act and adapt autonomously, through learning, to improve its own performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets take an example of Uber taxi,hope you know how Uber works....I do think this common example will as well help you get it right away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Imagine you've decided to start driving for Uber. It's a Friday night and you're running your typical route from the nightlife in San Francisco to many of the hotels away from the downtown area. &#xA;You've become accustomed to taking the Cupertino Street via Apple headquarters,exit to pick up your Engineers, but tonight you opt for Mountain View Road via Google headquarters for a change of pace. To your surprise, you discover that Mountain View Road is not only quicker, but you can avoid the accidents prone intersection at Cupertino Street and other Bay Area street . You decide that, in the future, you'll be sure to take Mountain View Road.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Having said that, a learning agent is able to act and adapt basing on new information. The more it learns the better to make effective decisions.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be traced from real world self driving cars,which are incorporated with sensor data processing in an Electronic Control Unit(ECU),Ladars....etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Hope this can give you some insight.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/M4dNx.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1581" LastActivityDate="2018-11-07T18:01:24.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8828" PostTypeId="1" AcceptedAnswerId="8842" CreationDate="2018-11-07T18:24:47.230" Score="4" ViewCount="260" Body="&lt;p&gt;It seems to me that the way neural networks are trained is similar to the way we educate a child (or a person, in general).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can an AI &lt;em&gt;eventually&lt;/em&gt; think like a human?&lt;/p&gt;&#xA;" OwnerUserId="19660" LastEditorUserId="2444" LastEditDate="2020-03-09T23:10:17.220" LastActivityDate="2020-03-09T23:10:17.220" Title="Can an artificial intelligence eventually think like a human?" Tags="&lt;philosophy&gt;&lt;human-like&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8830" PostTypeId="2" ParentId="6291" CreationDate="2018-11-07T19:14:49.103" Score="0" Body="&lt;p&gt;Thinking about the question could begin with the related question, &quot;Can AI learn what a lie is?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an intelligent being can distinguish truth from lies, then it can be programmed to lie or not lie or decide between them, assuming it is not programmed to be fully autonomous, in which case it would decide for itself based on its own ideas and objectives and couldn't be reliably programmed at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the intelligent perception of a lie something that has a counter example or a proof of its falsehood, or is a lie meaning in this question a ruse or subterfuge?  In the links in the question, this second meaning is implied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The robots in question are not aware of ethics and values regarding honesty, so they have no mapping.  In game theory, algorithms can be adjusted to not bluff, which is simply the removal of an intentional bluffing option in the algorithm, but to avoid learned strategies that involve misrepresentation would require the recognition of misrepresentation.  That's a very limited example of programming a robot not to lie though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a lie?  Some ethical positions might consider a lie something represented as true that is only conditionally true.  Other ethical positions might consider a lie something represented for self-serving purposes that is not a proven fact.  Thinking about these two ethical positions, they are distant rules of communication from one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can AI be programmed to not to lie?  It depends on what kind of AI and what kind of lie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can AI learning be directed away from dishonesty?  Yes, with considerable requirements definition around what a lie is and how learning can be directed away from lying.  In simple machine learning, the loss function would need to increase as the result approached some form of lie.  Logical systems would, if following the rules of inference, not lie because the rules of inference are always directed toward truth on the basis of axiomatic truths given it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DNA evolution never lies.  A live organism may play dead, but a dead organism cannot play live, and necrophilia does not produce offspring.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-11-07T19:14:49.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8831" PostTypeId="1" CreationDate="2018-11-07T21:28:31.757" Score="1" ViewCount="34" Body="&lt;p&gt;I am trying to read &lt;a href=&quot;https://arxiv.org/abs/1701.01727&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1701.01727&lt;/a&gt; about generalisation of Hopfield neural networks and I like the clear ideas that physics and Hamiltoanian framework can be used for modeling such networks and for deducing lot of properties. My question is - how Hopfield networks are connected to the standard networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see at least 3 differences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hopfield neurons has binary values on/off (+1, -1) but machine learning neurons have real (are at least approximately (within machine limits) real) values.&lt;/li&gt;&#xA;&lt;li&gt;threshold function is the only activation function used in Hopfield neurons, machine learning has lot of  more complex, real-valued functions&lt;/li&gt;&#xA;&lt;li&gt;Connectivity patterns in machine learning networks (LSTM, GRU cells) are far more richer.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So - maybe Hopfield neural networks can be generalized up to the level of machine learning networks and at the same time preserving the use of Hamiltonian framework. Is it possible, is there any work in this direction? &lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2018-11-07T21:28:31.757" Title="How Hopfield neural networks are connected with &quot;industrial&quot; neural networks used in machine learning?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8833" PostTypeId="5" CreationDate="2018-11-07T22:37:01.377" Score="0" Body="&lt;p&gt;For more info, see, for example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-03T16:22:54.173" LastActivityDate="2019-11-03T16:22:54.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8834" PostTypeId="4" CreationDate="2018-11-07T22:37:01.377" Score="0" Body="For questions related to symbolic artificial intelligence, which is also known as good old-fashioned artificial intelligence (GOFAI), which is an expression coined by John Haugeland in his 1985 book &quot;Artificial Intelligence: The Very Idea&quot;." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-03T16:22:54.173" LastActivityDate="2019-11-03T16:22:54.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8835" PostTypeId="2" ParentId="8748" CreationDate="2018-11-08T01:17:04.373" Score="3" Body="&lt;p&gt;John R. Pierce led the Bell Labs research team that created the first transistor and gave it its name.  He was later the Chief Engineer at the Jet Propulsion Laboratory at CalTech.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;His relationship to artificial intelligence research was mainly in regard to language translation.  He wrote the following.&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The computer has opened up to linguists a host of challenges, partial insights, and potentialities. ... Certainly, language is second to no phenomenon in importance. ... The new linguistics presents an attractive as well as an extremely important challenge. ... The most reasonable government source of support for research in computational linguistics is the National Science Foundation. ... We estimate that work on a reasonably large scale ... would be justified at ... an annual expenditure of \$2.5 to \$3 million.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No official record of Pierce stating unequivocally that all artificial intelligence funding is stupid can be found.  Nils J. Nilsson's &lt;em&gt;The Quest for Artificial Intelligence&lt;/em&gt; quotes Donald Knuth stating this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;John R. Pierce, whom I have already mentioned in connection with the ALPAC report on machine translation (in section 7.2) and his negative comments about speech understanding (p 222) wrote me a very short letter&#xA;  in which he stated,&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;Concerning artificial intelligence, I believe I invented the slogan, &quot;Artificial intelligence is real stupidity.&quot;&lt;/p&gt;&#xA;    &#xA;    &lt;p&gt;...&lt;/p&gt;&#xA;    &#xA;    &lt;p&gt;I resent artificial intelligence because I feel that it is unfair to computers. But then, artificial intelligence people &lt;em&gt;did&lt;/em&gt; devise LISP, which is pretty good.&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;  &#xA;  &lt;p&gt;The letter did not elaborate on the slogan or why AI is &quot;unfair to computers.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The paragraphs that may have drawn the most negative attention (and possibly fabricated quips) are these.&lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It would be too simple to say that work in speech recognition is carried out simply because one can get money for it.  That is a necessary but not a sufficient condition.  We are safe in asserting that speech recognition is attractive to money.  The attraction is perhaps similar to the attraction of schemes for turning water into gasoline, extracting gold from the sea, curing cancer, or going to the moon.  One doesn't attract thoughtlessly given dollars by means of schemes for cutting the cost of soap by 10%.  To sell suckers, one uses deceit and offers glamor.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It is clear that glamor and any deceit in the field of speech recognition blind the takers of funds as much as they blind the givers of funds.  Thus, we may pity workers whom we cannot respect.  People who work in the field are full of innocent (in their own view) enthusiasm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In hindsight, we can see a few things Pierce called correctly.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Technology projections that are not based on realistic assessments of progress that has already been made and verified in the correspondence between theory, experimental results, and penetration into a market that exercises the technology in the real world can be unreliable in terms of both feasibility and cost in time and money.&lt;/li&gt;&#xA;&lt;li&gt;Natural language translation and comprehension are important areas of research but research results from short term investments will not be substantial.  The problem complexity is high.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Whether Pierce meant that artificial intelligence is real stupidity literally or whether he was referring to the unrealistic expectations planted by researchers at MIT to obtain defense funding after the launch of Sputnik, we may never know for sure, but is easy to guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comment about LISP is an obvious reference to Marvin Minsky's lab, which allegedly did not produce for the U.S. government what was promised.  It is merely an allegation because research failures can be fabrication to assist in classifying publicly funded research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I say this as one who spent time with those in the closely linked AI lab at the United Technologies Research Center.  Everyone had national security clearances, and we all saw fabricated successes and fabricated failures.  There is good reason for mixing in misinformation when espionage is a known phenomenon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the way Pierce commonly wrote, the unfairness to computers was likely a satirical comment about research funding pitches and the unrealistic expectation they placed on the speed of computer technology development.  He is certainly not a person that would anthropomorphize a computer witlessly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pierce was given the leading role at the JPL because he gained credibility among fiscal stakeholders by telling the truth while at Bell Labs.  Some of the futurists in leading technology roles today might benefit from that history.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.mt-archive.info/ALPAC-1966.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Language and Machines&lt;/em&gt;, Pierce, 1966, ALPAC report to the U.S. National Science Foundation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://digitalcollections.library.cmu.edu/awweb/awarchive?type=file&amp;amp;item=355202&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Whither Speech Recognition?&lt;/em&gt;, Pierce, 1969, Journal of the Acoustical Society of America, Vol. 46, No. 4&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-08T01:17:04.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8836" PostTypeId="5" CreationDate="2018-11-08T01:19:38.447" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-08T01:19:38.447" LastActivityDate="2018-11-08T01:19:38.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8837" PostTypeId="4" CreationDate="2018-11-08T01:19:38.447" Score="0" Body="For questions about the exact wording, authorship, authenticity, historical source, and context of first use for well known quotes related to AI." OwnerUserId="1671" LastEditorUserId="4302" LastEditDate="2018-12-05T22:47:44.400" LastActivityDate="2018-12-05T22:47:44.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8839" PostTypeId="2" ParentId="6770" CreationDate="2018-11-08T02:36:31.487" Score="0" Body="&lt;p&gt;To get a computer to detect and provide the bounding box or circle around a visual attention area in an image, the basis for attention must be determined.  Then the method of getting the computer system to make choices based on that basis can be selected.  First things first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a face or body or game character that is to be the object of interest?  Will it be the most dynamic object in the frame in terms of movement?  If it is a person, is it always the same person?  In either case, will their face be exposed to the angle of the camera?  Are there only still shots, or will the images be frames in a movie?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you know how YOU would distinguish the object requiring attention from other objects and background, then you can begin to see how a computer might simulate that recognition.  When training a deep network that involves convolution kernels (called a CNN or convolutional neural network) and possibly long-short term memory cells (LSTM), there are stages to the recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually edges of things are detected first.  In movies, the movement of edges are tracked as features of the image.  Elements in the image that identify what kind of object the objects are is second.  For instance a toy might be detected by the way plastic reflects light and the color types and shapes common to toys.  A face might be first recognize by identifying eyes, nose, mouth, chin, and ears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After parts are identified, then entire objects can be identified through another stage of feature extraction.  Vision systems follow the same basic principles of recognition that our human visual system uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many frameworks and libraries to help with these tasks, but to use them, it is important to get a general picture of the process and to clarify what it is that will sets the objects of importance out from other objects that may be similar or completely different so that attention can be focused the way you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have &lt;span class=&quot;math-container&quot;&gt;$(h_{min}, v_{min}); (h_{max}, v_{max})$&lt;/span&gt;, the coordinates of the two corners of your cropping operation, which would be the goal of your network training, then any image manipulation library could handle the crop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's the state of the art.  There is no high level SDK that allows one to command the computer to find the most important item in the frame without any clarification of what is meant by that and training operations to teach the software to find what you've decided to be important based on some criteria.  Not yet anyway.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-08T02:36:31.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8840" PostTypeId="2" ParentId="4752" CreationDate="2018-11-08T02:46:42.690" Score="3" Body="&lt;p&gt;Because it is video input and the logos are usually stationary because they are layered over the live or recorded frames by either hardware or software, the task is not difficult.  Logos also usually have limited color palettes and crisp edges.  The features of their fonts, when they spell words or acronyms are usually consistent too.  These are generalities that can be exploited in deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with the other similar question posted by this author, a combination of LSTM and CNN layers can be trained to find and isolate the logo.  With some image tricks, the image behind the logo can also be reconstructed with a reasonable accuracy and reliability from the pixels around the logo through a similar set of learning techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are a few starting points for the development.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ivl.disco.unimib.it/activities/logo-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning for Logo Recognition &amp;mdash; Imaging and Vision Laboratory&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@akarshzingade/logo-detection-using-yolov2-8cda5a68740e&quot; rel=&quot;nofollow noreferrer&quot;&gt;Logo detection using YOLOv2 – Akarsh Zingade&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/satojkovic/DeepLogo&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub &amp;mdash; satojkovic/DeepLogo: A brand logo recognition system using deep convolutional neural networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/logo-detection-in-images-using-ssd-bcd3732e1776&quot; rel=&quot;nofollow noreferrer&quot;&gt;Logo detection in Images using SSD – Towards Data Science&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1701.02620&quot; rel=&quot;nofollow noreferrer&quot;&gt;1701.02620 &amp;mdash; Deep Learning for Logo Recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-08T02:46:42.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8842" PostTypeId="2" ParentId="8828" CreationDate="2018-11-08T03:38:33.730" Score="2" Body="&lt;p&gt;You need to fully understand the human mind before you can say that a  machine can eventually think like a human. However, the human brain is nowhere near fully understood, nor is human cognition.&lt;/p&gt;&#xA;" OwnerUserId="18646" LastEditorUserId="2444" LastEditDate="2020-03-09T23:07:07.623" LastActivityDate="2020-03-09T23:07:07.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8844" PostTypeId="1" AcceptedAnswerId="8847" CreationDate="2018-11-08T04:10:14.600" Score="5" ViewCount="1829" Body="&lt;p&gt;The field of AI has expanded profoundly in recent years, as has public awareness and interest.  This includes the arts, where fiction about AI has been popular since at least Isaac Asimov.  Films on various subjects can be good teaching aids, especially for younger students, but it can be difficult for a non-expert to know which films have useful observations and insights, suitable for the classroom.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are insightful films about AI?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Listed films must be suitable for academic analysis,&lt;/strong&gt; providing insight into theory, methods, applications, or social ramifications.&lt;/p&gt;&#xA;" OwnerUserId="19657" LastEditorUserId="1671" LastEditDate="2019-06-26T20:08:20.653" LastActivityDate="2020-08-06T23:46:16.913" Title="What are the most instructive movies about artificial intelligence?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;social&gt;&lt;mythology-of-ai&gt;&lt;academia&gt;" AnswerCount="7" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8845" PostTypeId="2" ParentId="8844" CreationDate="2018-11-08T04:57:43.947" Score="6" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/HAL_9000&quot; rel=&quot;nofollow noreferrer&quot;&gt;2001&lt;/a&gt; (1968)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;HAL 9000 is a great example of an artificial general intelligence that goes astray, where the humans don't understand the reasoning process as values dis-align.  (This is a nod to &lt;a href=&quot;https://en.wikipedia.org/wiki/Three_Laws_of_Robotics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asimov&lt;/a&gt; in the sense of humans not understanding the implications of a logical framework. &lt;a href=&quot;https://en.wikipedia.org/wiki/Marvin_Minsky&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marvin Minsky&lt;/a&gt; was an adviser on the film.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F#Critical_reception&quot; rel=&quot;nofollow noreferrer&quot;&gt;BladeRunner&lt;/a&gt; (1982)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The critical reception in the link references androids as mirror of humanity, with all the implications, including identity.  Empathy is the core theme. (Seeing the self in the other.  The film adapts these ideas into a love story, and includes overt Christian symbology with android &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Blade_Runner_characters#Roy_Batty&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roy Batty&lt;/a&gt; in the finale. HBO's Westworld draws heavily from these ideas.)  The book and film also explicates an evolution of &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing tests&lt;/a&gt;, focused on psychological responses due to the ever increasing sophistication of the androids.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/WarGames&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wargames&lt;/a&gt; (1983)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This film features an autonomous military AI that does not understand the context of human reality and nearly starts a total nuclear war.  (An algorithm's view of reality is based on it's input, which in the film, was limited.)  The ending is an early nod to machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/WALL-E#Technology&quot; rel=&quot;nofollow noreferrer&quot;&gt;WALL-E&lt;/a&gt; (2008)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here the key element is the diminishment of humanity and human purpose when strong AIs (in the form of robots) to accomplish every task.  &lt;em&gt;This is already happening and seems accelerated by mobile computing.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/David_8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alien: Covenant&lt;/a&gt; (2016)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A horror film at it's core, it unveils a race of superior aliens brought down by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hubris&quot; rel=&quot;nofollow noreferrer&quot;&gt;hubris&lt;/a&gt; that their own creation could never turn on them.  In regard to humanity this is mirrored by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Android_(robot)&quot; rel=&quot;nofollow noreferrer&quot;&gt;android&lt;/a&gt; David. (In some sense it's a throwback to Greek Drama, where pride is the fatal flaw.)   &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Shout out to &lt;a href=&quot;https://ai.stackexchange.com/a/8847/1671&quot;&gt;malioboro's answer&lt;/a&gt; re: Ex Machina, which raises the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Is the AGI in Ex Machina a sociopath because she was created by a sociopath?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nature vs. Nurture&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-01T19:43:24.713" LastActivityDate="2019-07-01T19:43:24.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8846" PostTypeId="1" CreationDate="2018-11-08T05:34:22.210" Score="3" ViewCount="174" Body="&lt;p&gt;There are some fields of Computer Vision that are similar to Artificial Intelligence. For example, pattern recognition and path tracking. Based on these similarities, can we say that the Computer Vision is a part of Artificial Intelligence?&lt;/p&gt;&#xA;" OwnerUserId="19656" LastEditorUserId="1847" LastEditDate="2018-11-08T10:00:00.910" LastActivityDate="2018-11-08T10:00:00.910" Title="Are Computer Vision and Digital Image Processing part of Artificial Intelligence?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8847" PostTypeId="2" ParentId="8844" CreationDate="2018-11-08T06:39:06.733" Score="7" Body="&lt;p&gt;Here are my suggestions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.imdb.com/title/tt1798709/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Her&lt;/a&gt;, the AI part (movie spoiler):&lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; The AI can define a user's profile just by hearing his short story, and &quot;act&quot; based on the user's profile. The AI makes the user comfort with it (her). It shows a very advance user profiling.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.imdb.com/title/tt0470752/?ref_=tt_rec_tt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ex Machina&lt;/a&gt; the AI part (movie spoiler):&lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; This movie will show you how an AI learn to trick someone. The AI can express her feelings, and make you trust their feeling.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.imdb.com/title/tt1059786/?ref_=nv_sr_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eagle Eye&lt;/a&gt;, the AI part (movie spoiler):&lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; A movie about a general story &quot;AI that want to kill&quot;. This movie can show you how The AI can compile a lot of information for its purpose.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.imdb.com/title/tt2245084/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Big Hero 6&lt;/a&gt;, the AI part (movie spoiler):&lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; Baymax is a very good example of a very complex expert system, he has a &quot;knowledge chip&quot; and a very smart way to diagnose people&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2019-07-22T07:41:50.413" LastActivityDate="2019-07-22T07:41:50.413" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8848" PostTypeId="2" ParentId="8846" CreationDate="2018-11-08T08:29:08.367" Score="1" Body="&lt;p&gt;Artifical Intelligence is a rather fuzzily-defined field. It includes lots of subject areas. Sometimes that's because they replicate some behaviour or mental ability of creatures used to solve problems, and sometimes it is because they were the focus of &quot;classic&quot; AI, and have remained classified as part of AI. These problems include navigation, searching, solving combinatorial or logic puzzles, complex systems control, communication. Many of the problems can include converting visual input to something more useful to the rest of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer Vision by comparison appears more tightly-defined. However, it quickly becomes less so when you try to consider what the eventual outputs should be. To what extent does a computer vision system need to &quot;understand&quot; the visual inputs in order for other system components to make sense of them, and use them correctly? Here are a couple of examples to illustrate my point:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Image captioning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For instance, &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/sfmltalk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;work in automated captioning&lt;/a&gt; effectively includes natural language models. The input to the system is an image. The computer vision half of that system does not output a class or direct measurement of something inside the image. Instead it outputs a vector encoding of the description of the whole image - something that the language model can decode into an English sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vision model and language model are trained together, so this encoding can be very loosely considered an internal &quot;ideagram&quot; that represents the contents of the image and allows the vision and language models to share representations.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Agents using vision systems for decision-making&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Another interesting example is &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent work on Deep Q Networks, used to give superhuman performance when playing some Atari games&lt;/a&gt;. This used a neural network architecture (CNN) already known to be good for image classification problems, and used it to directly connect the pixel output from a game to a system which learned optimal behaviour. This is more clearly work in AI - creating a learning agent that works across a general set of problems - but at its core is a CV system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the input is four consecutive images (although there are variations which learn from the stream of individual frames), and the output is a vector which scores the value of taking one of the possible controller actions (left, right, up, down, fire etc). Variations on the system directly output which action would be preferred from the CNN.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You could take this further and require that a vision system produce some kind of explainable, transferable, shared world model for other systems to interact with. An embryonic version of that approach is effectively how the captioning system works. It is not at all clear whether such an imagined design is strictly in the field of CV or some other part of AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether such outputs are considered part of computer vision or more generally part of AI is less of a pure scientific classification issue than an opinionated or political one (e.g. due to funding either an AI research team or a CV one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The similarities between current computer vision approaches and ongoing work in AI, such as heavy use of deep learning models, would suggest that CV is strongly related to AI. Intuitively I would say that CV is part of AI, perhaps a key specialism of it, roughly equivalent in scope to Natural Language Processing. However, the practical outcome of that is only important to people who have labelled themselves &quot;AI researcher&quot; or &quot;CV engineer&quot; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also think the same arguments apply to other studies of artificial perception, most notably speech recognition.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-08T09:27:52.867" LastActivityDate="2018-11-08T09:27:52.867" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8849" PostTypeId="2" ParentId="8844" CreationDate="2018-11-08T09:39:58.563" Score="2" Body="&lt;p&gt;Transcendence was a a pretty good look at a super-intelligent AI. &lt;/p&gt;&#xA;" OwnerUserId="16035" LastActivityDate="2018-11-08T09:39:58.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8850" PostTypeId="2" ParentId="8844" CreationDate="2018-11-08T10:00:27.473" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/The_Machine_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Machine&lt;/a&gt;, which came out a year before Ex-Machina, features topics such as ethics, AI testing (specifically, the Turing test), artificial consciousness, emotional intelligence, artificial general intelligence, super-intelligence, the singularity and AI safety.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An old but milestone movie is &lt;a href=&quot;https://en.wikipedia.org/wiki/WarGames&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wargames&lt;/a&gt;, which will likely get you some points for digging back into history on the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;2001: A Space Odyssey&lt;/a&gt;, which is an excellent movie. However, to truly appreciate the nuance of the &lt;em&gt;Hal 9000&lt;/em&gt;, you need to also see the sequel, or at least read a summary as it shows WHY Hal 9000 behaved the way it did. (In short, contradictory and irreconcilable instructions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Alien_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alien&lt;/a&gt;, the original, first one, which contains both humanistic and non-humanistic AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Her_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Her&lt;/a&gt;, which is either sadly romantic or creepy as all hell, depending on how you interpret it.&lt;/p&gt;&#xA;" OwnerUserId="19676" LastEditorUserId="2444" LastEditDate="2019-06-27T12:10:10.447" LastActivityDate="2019-06-27T12:10:10.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8851" PostTypeId="2" ParentId="8844" CreationDate="2018-11-08T10:57:28.213" Score="2" Body="&lt;p&gt;A great movie to watch would be &lt;a href=&quot;https://en.wikipedia.org/wiki/A.I._Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;A.I. Artificial Intelligence&lt;/a&gt;, which is a sort of modern retelling of Pinocchio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another good AI movie where the main character is a robot would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Bicentennial_Man_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bicentennial Man&lt;/a&gt;, based on Isaac Asimov's &lt;em&gt;the positronic man&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19680" LastEditorUserId="2444" LastEditDate="2019-06-26T17:36:29.160" LastActivityDate="2019-06-26T17:36:29.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8853" PostTypeId="2" ParentId="8844" CreationDate="2018-11-08T13:28:48.470" Score="2" Body="&lt;p&gt;Here are the best movies about artificial intelligence&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2001: A Space Odyssey (1968)&lt;/li&gt;&#xA;&lt;li&gt;Colossus: The Forbin Project (1970)&lt;/li&gt;&#xA;&lt;li&gt;Blade Runner (1982)&lt;/li&gt;&#xA;&lt;li&gt;A.I. Artificial Intelligence (2001)&lt;/li&gt;&#xA;&lt;li&gt;Her (2013)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18510" LastEditorUserId="2444" LastEditDate="2019-07-21T22:11:37.780" LastActivityDate="2019-07-21T22:11:37.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8854" PostTypeId="1" AcceptedAnswerId="8862" CreationDate="2018-11-08T14:38:04.963" Score="4" ViewCount="166" Body="&lt;p&gt;I'm aware of those AI systems that can play games and neural networks that can identify pictures.&lt;/p&gt;&#xA;&lt;p&gt;But are they really thinking? Do they think like humans? Do they have consciousness? Or are they just obeying a bunch of codes?&lt;/p&gt;&#xA;&lt;p&gt;For example, when an AI learns to play Pacman, is it really learning that it should not touch the ghosts, or are they just following a mechanical path that will make them win the game?&lt;/p&gt;&#xA;" OwnerUserId="19192" LastEditorUserId="2444" LastEditDate="2020-12-07T18:10:54.163" LastActivityDate="2020-12-07T18:10:54.163" Title="Do genetic algorithms and neural networks really think?" Tags="&lt;neural-networks&gt;&lt;philosophy&gt;&lt;genetic-algorithms&gt;&lt;artificial-consciousness&gt;&lt;genetic-programming&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8857" PostTypeId="2" ParentId="8854" CreationDate="2018-11-08T15:08:15.273" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do genetic algorithm and neural networks really think?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Genetic algorithms and neural networks are vastly different concepts. Both of them do not think.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm aware of those AI programmes which can play games and neural networks which can identify pictures. But are they really thinking&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Depends on how you define &quot;thinking&quot;, but I say &quot;no&quot;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do they think like humans?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do they have consciousness?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or are they just obeying a bunch of codes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. It is a machine. A program.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;One example where most people realize that it is very different is the common thread for RNNs which generate text (&lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;noreferrer&quot;&gt;example texts&lt;/a&gt;). They can generate syntactically correct texts, but they &quot;forget&quot; about any actors in it. They don't connect the cables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then, on the other hand, would you say a bacterium thinks? An ant? A mouse? A dog? A chimpanzee? Do we &quot;think&quot;? Where exactly is the border - and why?&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-08T15:08:15.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8858" PostTypeId="2" ParentId="1476" CreationDate="2018-11-08T15:12:42.073" Score="0" Body="&lt;p&gt;&lt;strong&gt;Introspect&lt;/strong&gt;! Do you need to know what's good/bad, pleasurable/painful or so, in order to understand and/or learn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a human, hence a general intelligence, and so are you. So know thyself! I can tell for myself that I have different ways to understand and learn; some may be similar to reinforcement learning. Esp. the ~automatic ~innate ~unconcious ones, like motor movement, remembering tasty food and many other primitive functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But&lt;/strong&gt; I can also understand things through ~intentional ~analytical ~logical thought; which some may call pure reason (Immanuel Kant). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, you don't need to hear all that, since have it already in your own mind. &lt;/p&gt;&#xA;" OwnerUserId="19693" LastActivityDate="2018-11-08T15:12:42.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8859" PostTypeId="2" ParentId="8854" CreationDate="2018-11-08T15:17:43.470" Score="1" Body="&lt;p&gt;This is a really interesting question that can't be answered correctly since we lack a common understanding or a universally valid definition of what &quot;thinking&quot; means. Still I will try to give my humble opinion on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all I would like to mention that consciousness might not exist in a binary fashion (as possessing it or not) but in a gradual fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my understanding what neural networks do is learning a mapping, i.e. a function from the input space of their sensors to the output space of the actuators. If the neural network is playing Pacman, input consists of the pixels of the game and the output are actions the agent can take in the game. Since this process runs in a sequential session (i.e. &lt;code&gt;input&lt;/code&gt; &lt;span class=&quot;math-container&quot;&gt;$\rightarrow_{NN}$&lt;/span&gt; &lt;code&gt;output&lt;/code&gt;; &lt;code&gt;input&lt;/code&gt; &lt;span class=&quot;math-container&quot;&gt;$\rightarrow_{NN}$&lt;/span&gt; &lt;code&gt;output&lt;/code&gt; and so on) I would not go so far as to consider the neural network conscious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting approach on this question comes from Douglas Hofstadter with his book &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Gödel, Escher, Bach&quot;&lt;/a&gt;. The main thesis of this book (in my understanding) is that self-reference is a crucial and necessary precondition for (self-) consciousness.&lt;/p&gt;&#xA;" OwnerUserId="19256" LastEditorUserId="19256" LastEditDate="2018-11-08T16:01:24.590" LastActivityDate="2018-11-08T16:01:24.590" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8860" PostTypeId="1" AcceptedAnswerId="8874" CreationDate="2018-11-08T15:25:25.307" Score="0" ViewCount="53" Body="&lt;p&gt;What are ways of determining the number of trees to be generated in a random forest algorithm?&lt;/p&gt;&#xA;" OwnerUserId="19442" LastEditorUserId="2444" LastEditDate="2019-05-01T17:10:12.560" LastActivityDate="2019-05-01T17:10:12.560" Title="How many trees should be generated in a random forest?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;&lt;hyper-parameters&gt;&lt;random-forest&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8862" PostTypeId="2" ParentId="8854" CreationDate="2018-11-08T16:01:33.827" Score="3" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;TL;DR&lt;/strong&gt; Ignore the hype, current systems (in 2018) are very far removed from human-like &quot;thinking&quot;, despite interesting and useful results. State-of-the-art for &quot;thinking and behaving like a creature in general&quot; has not reached the sophistication level of insects, even though we have example narrow AIs that can beat the world's best at intellectual games.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some issues with the question as phrased, it is not a precise question, and includes some common wooly logic that many people have when discussing AI. However, I think these occur so often, that they are worth addressing here:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But are they really thinking. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Define &quot;thinking&quot;. It is not easy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do they think like humans? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We don't fully know or understand how humans think.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do they have consciousness? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We don't fully know or understand what consciousness is.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or are they just obeying a bunch of codes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Are you just obeying a &quot;bunch of codes&quot;? There is no reason to suspect that humans have a magic &quot;something&quot; that powers thought. All scientific evidence points to humans being sophisticated machines that follow the laws of physics and chemistry. However, the level of complexity is such that how brains, thinking and consciousness work is a very hard problem to solve. It is also &lt;em&gt;possible&lt;/em&gt; that our lack of knowledge obscures some unknown property of the brain or &quot;thinking&quot; that means it is more than applied physics and chemistry - but there is zero evidence for such a thing.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, when an AI learns to play pacman, is it really learning that it should not touch the ghosts or are they just following a mechanical path which will make them win the game?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is an interesting question, and there is more to it than a simple yes/no answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learning is not the same as thinking&lt;/strong&gt;. We expect an intelligent entity to be able to learn, when it receives new information. However, something that only learns - the only feature of it is that it gains some performance measure when fed experience - is only solving part of what it means to be intelligent or thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A human player starts with assumptions&lt;/strong&gt;. A game like Pacman presents a simplified world that obeys many rules that we are already familiar with. In the game there is a space (the screen), in which rules of distance apply. There are objects (the pacman, walls, pills, power pills, ghosts) that have coherent shapes, and persistent properties that are recognisable. Object persistence is a thing. The game play follows familiar concepts of time. All these things are at least initially meaningless to a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A neural network generalises&lt;/strong&gt;. One of the reasons of the success of Deep Learning is that neural networks can, within limits, learn rules such as &quot;avoid the ghosts&quot; or in the case of a typical agent in DQN, that ghosts getting closer to the player is an indication that there is a low value in staying still or moving towards a ghost, and a high value in moving away towards an escape route. Not only that, but there is a good chance that a deep neural network will learn an internal representation that really does detect the ghosts and learn an association between them and the type of action, and this can be inspected by generating &quot;heat maps&quot; of neuron response to different areas of an image&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neural networks require far more experience than humans to learn to play well&lt;/strong&gt;. This demonstrates that they are learning differently. Some of this may be due to human innate knowledge transferring to playing a game. However, a typical reinforcement-learning training scenario would need to demonstrate that ghosts are dangerous 100s of times before the neural network finally is able to generalise well. Whilst a human would learn that after only a few encounters, maybe only one. There is much research in this area, as making learning as efficient as possible is an important goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Neural networks are too direct and simple to possess internal world models&lt;/strong&gt;. Raw generalisation is not the same as having the kind of rich internal model that you may be considering as &quot;thinking&quot;. After training, the neural network is a function which maps a visual field of pixels into values and/or actions. There is no internal &quot;narrative&quot;, even though the function is enough to behave correctly, it does not encode concepts such as space, object persistence, object traits etc, and it most definitely does not experience them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some ways, asking if an artificial neural network can think is like asking if a small slice of your retina, or a tiny cube (less than 1mm&lt;sup&gt;3&lt;/sup&gt;) of brain tissue can think. Or perhaps if your walking reflex and innate sense of balance counts as thinking. Current state-of-the-art efforts are at that kind of scale both in terms of computer power and complexity.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-08T16:08:13.863" LastActivityDate="2018-11-08T16:08:13.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8863" PostTypeId="2" ParentId="1476" CreationDate="2018-11-08T17:58:57.507" Score="1" Body="&lt;p&gt;&lt;strong&gt;Motivational Theory Versus Learning Theory&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewards and punishment are based on human motivational theory and are broad generalizations of the neurological factors in mental growth.  At a neurological level, there are chemically based signals, such as seratonin, dopamine, and oxytocin.  Some of these have their own pathways in the brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also pain signals arising from the body and internally generated stimulatory and inhibitory signals in chemical and electro-chemical forms.  There is neuroplasticity, where the brain circuits form new topologies.  The hippocampus is a structure known to be important in learning.&lt;sup&gt;1&lt;/sup&gt;  The study of learning in the brain is fervent, but the object of study is complex.  Even so, every year, progress is made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multiple Dimensions or Singly Quantifiable Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conjecture about strength versus weakness of intelligence may be of limited use.  There are two valid criticisms discussed in other Q&amp;amp;A here in more detail.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Twenty-two independent genes have been identified that correlate to typical academic intelligence quantification in humans, rational and linguistic, eleven of which were identified in 2017.&lt;sup&gt;2&lt;/sup&gt;  There are likely more.  Creative, athletic, emotional, and leadership qualities have not yet been captured in standardized intelligence testing, so it is likely that additional genes related to those abilities exist.  The notion that intelligence can be accurately aggregated into a single g-factor is weakly supported, if it were true, implies that all twenty-two genes affect only one magical function.  Statistically, such alignment is close to impossible.&lt;/li&gt;&#xA;&lt;li&gt;Intelligence is only meaningful in context.  Fast learning in one context appears like mental brilliance, but in another context the same learning capability may be dysfunctional.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reinforcement in the Larger Pool of AI Research&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement Learning is not a requirement for AI as far as is currently known.  Reinforcement is a mathematical strategy for driving a set of behavioral parameters toward a defined optimum.  Although its variants work well for many applications, it is not closely aligned with any of the features of the hippocampus, the limbic system, the cerebral cortex, or any other learning systems in the human brain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though there is still much to learn about brain workings, the success of reinforcement learning, rule based systems, convolution, attention based networks, and generative networks provides information about what is possible in computers and what may or may not be a mechanism of the brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What Reward and Punishment Imply&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The existence of rewards and punishment imply the existence of favorable and unfavorable family, workplace, or community states.  If the term &lt;em&gt;world&lt;/em&gt; in the question is referring to the immediately related environment, then the sentence is probably true, but it may be misleading without clarification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;World states are based on economics which involve the minds of many people in community, aggressive actions, resource acquisition, consumption, cultural perspectives, and many other factors.  A single individual's perception of world states is subjective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Defying Motivational Mechanisms and Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same person on two different days may have different ideas of what is favorable or unfavorable, which may or may not correlate to reward and punishment motivations.  The words rebellion and indifference describe states where external reward and punishment produce opposing or independent results in learned behavior respectively.  Some of the most intelligent behavior in humans were of these kinds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What drives the classification of external states within a single individual is something not well understood.  The neurological factors involved in the emergence of opposition to pain, pleasure, reward, and punishment is an interesting area of study and certainly related to AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic factors involved in such contrary learning are likely to fall outside the twenty-two genes, since those genes correlated with standardized academic testing.  Academia is about conformance.  If we don't agree with the author of the textbook for a course, we pretend we do when we take the test if we want a high grade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;World-changing intelligence, such as that of Ghandi's nonviolent opposition, Isaac Newton's invention of physics, the furtherance of logical disproof by Socrates, or whoever dared to learn how to light a fire. is always the exhibition of classifying in a way that contradicts that of the surrounding culture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discovery and Acceptance in Group Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is even true of project-changing intelligence.  A software engineer might say, &quot;That software will never work because the database schema indicates a one to many relationship between companies and office addresses, but it is a many to many relationship.&quot;  Such a statement doesn't change the world and may completely lack group acceptance when spoken.  Later in the project, after the statement is fully accepted, the reclassification changes the design so that the project is not caught in a well of wasted coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, convincing the others often falls to someone other than the originator of the idea &amp;mdash; someone else on the team that has a different set of mental abilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that classification of what is favorable or unfavorable is key, but how that classification exhibits intelligence is not well understood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What Benefits AI Development&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although not a requirement, a solid approach to AI development is discovery, from the directions of multiple fields of study, of what intelligence is.  Some say we know what it is now, but most definitions are largely expressions of historical inaccuracies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clarity about what intelligence is and what it is expected to be in such a way that it can be represented mathematically is needed.  Only then can we better achieve it in humans and machines.  Sources of understanding may come from discovery in any area of study, such as these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Evidence based psychology&lt;/li&gt;&#xA;&lt;li&gt;Evidence based genetics&lt;/li&gt;&#xA;&lt;li&gt;Evidence based neurological research&lt;/li&gt;&#xA;&lt;li&gt;Evidence based artificial systems design&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a growing trend to hack AI, but playing with AI as if it were a game might be a formula for creating Frankenstein.  A useful and collaborative collection of AI devices in robots, autonomous vehicles, and data centers will require interdisciplinary inquiry, experimentation, and engineering.  The description of this site implies that ethics should be involved, and that's a smart conception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a purview that crosses divisions in study and practice, central themes and models of intelligence are likely to develop.  This question is on the right track in this respect. It asks questions without accepting too quickly conjecture  &amp;mdash; ideas and hypotheses presented but not yet shown to be true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is Known&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That computer simulation or replication of human abilities have already been achieved and that such achievements indicate AI to be a reasonable long term objective few would deny.  Certainly in the areas of arithmetic, high speed machine control, sorting of addressed parcels, and many other roles, computers far outperform humans.  Whether world-changing intelligence will emerge in artificial systems is hoped for by some, feared by others, and asserted as inevitable by those who seek funding to prove it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3445739/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Training your brain: Do mental and physical (MAP) training enhance cognition through the process of neurogenesis in the hippocampus?&lt;/em&gt;, D.M. Curlik and T.J. Shors, 2012&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5665562/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Genome-wide association meta-analysis of 78,308 individuals identifies new loci and genes influencing human intelligence&lt;/em&gt;, Suzanne Sniekers et. al., 2017&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-08T17:58:57.507" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8864" PostTypeId="2" ParentId="2398" CreationDate="2018-11-08T18:51:29.693" Score="1" Body="&lt;p&gt;If the standards for what is a neural network are held consistent, then cellular neural networks should probably be considered a type of neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That two adjacent layers be fully connected is not considered a requirement of what people consider neural networks.  Some attention networks, most convolution kernels, and several other types of networks are not exhaustively connected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exhaustive connectivity is a burden on computation time that is only valuable in cases when function requires it.  In cases where the parameter value for the connection can be shown to be always zero, there is no need for the potential.  In cases where it is rarely nonzero, the benefit of the potential may outweigh the gain of having it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many kinds of networks exist that are not exhaustively connected in computer science, and biological neurons are partly connected.  The axons of biological neurons might grow through an adjacent layer and connect to distant one.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Chua's paper, it was shown that cellular neural networks converge under specific conditions, just like for other neural network types. There is no claim that they learn like humans do, and they don't use the same back propagation and gradient descent that MLPs use, but they do incrementally improve, which is the only common criteria across the many forms of artificial learning networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cellular neural networks do not fit into what is generally thought of when people use the term unsupervised learning, however, they are not supervised and they do exhibit the same incremental improvement that other unsupervised networks exhibit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the issue with these names is that specialized jargon forms when only one type of a thing becomes popular, assigning the name to something more specific than the words in the name imply.  Cellular neural networks may not be examples of unsupervised learning, but they learn to function better without supervision.&lt;/p&gt;&#xA;" OwnerUserId="19127" LastActivityDate="2018-11-08T18:51:29.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8866" PostTypeId="2" ParentId="7543" CreationDate="2018-11-08T20:21:30.377" Score="1" Body="&lt;p&gt;A loss function, except in the most trivial case of one dimensional network output, is itself an aggregation.  Aggregating loss functions is just an approach for producing a scalar loss from differences in multiple dimensions, and possibly an over-complicating one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the concept class is a hierarchy with three levels, then the loss function is an aggregation of differences occurring in one, two, or three dimensional space, based on whether the divisions in each category are identical throughout the hierarchy.  If there are three label dimensions, they are statistically independent, and each has three states, then there are 27 categories that can be represented in three ternary outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying several loss function options available in torch.nn.modules.loss is one way to find the best solution within PyTorch, but its possible to make the wrong choose because the hyper-parameters are not correctly set in the better choice.  Or the labels could be improperly mapped to the poorly performing loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be over-complicating the solution to embed one model in another when it is common to categorize in multiple dimensions with a single network.  Take a simple case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say there is one category dimension of four age brackets, another of attire type (casual, business casual, formal, undetermined), and a third of eight cultural backgrounds including undetermined.   The goal is to categorize head-to-toe images based on facial, hair, and clothing style queues with some specific reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, in addition to the appropriate convolution, pooling, and fully connected layers, the network output layer could have two binary outputs for age, two for attire, and three for cultural background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complexity occurs when the kinds of attire differ between cultural backgrounds.  In this case, the five bits may determine a composite of attire types and cultural backgrounds, but developing separate networks, one for each culture type is probably overkill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mapping the dimensions of labels to the output vector should be the focus of attention.  Entropy is a concept that applies to loss function selection, but the idea is to align negative entropy evenly with output vector components, so that information is equally distributed across the vector components.&lt;/p&gt;&#xA;" OwnerUserId="19127" LastActivityDate="2018-11-08T20:21:30.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8867" PostTypeId="1" CreationDate="2018-11-08T22:03:32.067" Score="2" ViewCount="103" Body="&lt;p&gt;There are two models for the same task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;model_1:&lt;/strong&gt; 98% accuracy on training set, 54% accuracy on test set.&lt;br/&gt;&#xA;&lt;strong&gt;model_2:&lt;/strong&gt; 48% accuracy on training set, 47% accuracy on test set.&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the statistics above we can say that &lt;strong&gt;model_1&lt;/strong&gt; overfits training set.&lt;br/&gt;&#xA;&lt;strong&gt;Q1:&lt;/strong&gt; Can we say that &lt;strong&gt;model_2&lt;/strong&gt; underfits?&lt;br/&gt;&#xA;&lt;strong&gt;Q2:&lt;/strong&gt; Why &lt;strong&gt;model_1&lt;/strong&gt; is bad choice if it performs better than &lt;strong&gt;model_2&lt;/strong&gt; on test set?&lt;/p&gt;&#xA;" OwnerUserId="19713" LastActivityDate="2019-05-08T02:00:55.650" Title="Overfitted model performs better in test set" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8869" PostTypeId="2" ParentId="1476" CreationDate="2018-11-08T23:55:08.853" Score="1" Body="&lt;p&gt;Not only is reinforcement based on rewards and punishments but not deep Q-learning, whatever specific meant by that, not a requirement of strong AI, whatever definition we use for that, but reinforcement may prohibit progress in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea that achieving goals is the apex of intelligence is myopic.  Intelligent people usually pick new goals when it becomes to routinely easy to achieve the old ones.  Punishment for committing crimes or being a bad little girl or boy is not an educational technique that necessarily leads to good behavior.  It sometimes produces the appearance of compliance over a potentially destructive system of motivations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Educating people about the mutual benefits of healthy community and being the kind of person that does the right thing without reward and sometimes in the face of ridicule produces the most consistently excellent behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Epicurean materialism in human history, with its focus on enjoyment as the primary meaning in life, is a reward driven philosophy.  They did not consistently produce the best behavior.  Those most like them today are the most consistently labeled sociopaths by their former friends and lovers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if we consider adapting behavior to reward and punishment incentivized learning, is that what humans want machines to learn?  If we train machines to learn like Pavlov's dogs are we producing intelligence?  It seems more like a way of programming with much more data and fewer lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're progressing from books like &lt;em&gt;Coding in Python by Example&lt;/em&gt; to what may be called &lt;em&gt;Avoid Coding by Providing Examples&lt;/em&gt;, if all the parameter initialization and layer design selections are automated.  Programmers would be replaced by example hunters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That reinforcement works well for some classes of problems is a piece of information in the larger puzzle of AI, along with the success of chess programs, convolution kernel techniques, and sophisticated control systems.  We can't say for sure that any of these are requirements for all intelligent systems of the present or future.  We can certainly rule pure learning from reward and punishment as a requirement of artificially achieving human-like intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A stronger conception of a smart system could be one that will determine favorable and unfavorable states based on an analysis of the values of other machines and people both past and present.  If the system constructed a personal value system from this analysis and determined what is likely to be favorable or unfavorable in the long run based on that personal value system, that would surely be AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The requirement of AI excellence may be to find an optimum behavior based on labels, but that it is a very limited conception of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is commonly spoken or written about strong AI indicates that those speaking or writing are hoping for personal slaves that are extremely good at winning at whatever game their slave owners task them.  Is that really plausible?  How many obedient slaves in human history are also are brilliant game players on their captor's team and stay that way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if plausible, which is highly doubtful, is that a world state to which we should aspire? &lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-11-08T23:55:08.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8870" PostTypeId="2" ParentId="8867" CreationDate="2018-11-09T00:11:32.600" Score="1" Body="&lt;p&gt;Whether a model is over-fitting the data cannot be determined solely from false positive and false negative data on training and test runs.  If the sets are too small, very little can be determined.  If the sample selection method is choosing inequitably with respect to the training model, accuracy information may be misleading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the first run was just a matter of luck in selection, that could be determined by making multiple training runs with the first model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if over-fitting is the cause of the drop in testing accuracy for the first model, that does not mean that the over-fitting model isn't none-the-less more fit that the second model.  It doesn't appear to be over-fit from that one training and test run, but may be deficient in other ways.  The second model may under-fit, for instance.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2018-11-09T00:11:32.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8873" PostTypeId="2" ParentId="5085" CreationDate="2018-11-09T07:18:52.717" Score="0" Body="&lt;p&gt;You can try using Mallet (which uses Gibs Sampling) or gensim LDA (which uses drichlet priors) to model the problem as topics (hashtags) in different documents (tweets).&#xA;&lt;a href=&quot;https://towardsdatascience.com/topic-modeling-and-latent-dirichlet-allocation-in-python-9bf156893c24&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/topic-modeling-and-latent-dirichlet-allocation-in-python-9bf156893c24&lt;/a&gt; has a nice example.&lt;/p&gt;&#xA;" OwnerUserId="19722" LastActivityDate="2018-11-09T07:18:52.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8874" PostTypeId="2" ParentId="8860" CreationDate="2018-11-09T08:39:18.940" Score="1" Body="&lt;p&gt;The number of estimators in Random Forest is a hyper-parameter. If you are using SKLearn's Random Classifier you can use one of the following techniques to find a (near) optimal hyperparameter settings (Note:You can tweak other hyperparameters like &lt;code&gt;min_leaf_size&lt;/code&gt; etc as well with this approach);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;GridSearchCV&lt;/strong&gt; You can specify a grid of all the hyperparameters and a scoring criteria. This function will then evaluate &lt;strong&gt;all&lt;/strong&gt; combinations of these parameters for you and return the setting which performed best on the validation set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;RandomSearchCV&lt;/strong&gt; You can specify a grid of all the hyperparameters and a scoring criteria. This function will then evaluate &lt;strong&gt;n randomly choosen&lt;/strong&gt; combinations of these parameters for you and return the setting which performed best on the validation set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bayesian Optimization&lt;/strong&gt; You can treat the hyperparameter settings and corresponding score as a black box function and use exploitation-exploration paradigm using bayesian optimization.&lt;/p&gt;&#xA;" OwnerUserId="19722" LastActivityDate="2018-11-09T08:39:18.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8878" PostTypeId="1" AcceptedAnswerId="8890" CreationDate="2018-11-09T14:10:34.167" Score="3" ViewCount="1310" Body="&lt;p&gt;We seem to be experiencing an AI revolution, spurring advancement in many fields.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Please explain, at a high level, the uses of artificial intelligence and why we might need it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19220" LastEditorUserId="1671" LastEditDate="2019-05-22T20:54:29.987" LastActivityDate="2019-07-05T05:59:18.163" Title="Why do we need artificial intelligence?" Tags="&lt;ai-basics&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8879" PostTypeId="1" CreationDate="2018-11-09T14:17:08.837" Score="3" ViewCount="2700" Body="&lt;p&gt;&lt;a href=&quot;http://aiden.nibali.org/blog/2017-01-18-mode-collapse-gans/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mode collapse&lt;/a&gt; is a common problem faced by GANs. I am curious why doesn't VAE suffer mode collapse? &lt;/p&gt;&#xA;" OwnerUserId="19738" LastEditorUserId="2444" LastEditDate="2019-10-10T02:24:09.147" LastActivityDate="2019-10-10T02:24:09.147" Title="Why doesn't VAE suffer mode collapse?" Tags="&lt;comparison&gt;&lt;generative-adversarial-networks&gt;&lt;autoencoders&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8880" PostTypeId="2" ParentId="8499" CreationDate="2018-11-09T14:29:28.160" Score="0" Body="&lt;p&gt;In addition to the problems like Rubik's cube (as described by @Astariul) which are practically too complex to solve exhaustively (due to limitations of speed and memory of our machines), there are other problems which no one has yet been able to show can be solved, like &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Halting Problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, with the advances in Quantum Machine Learning we may be able to solve problems which are yet impractical to solve with exhaustive search (like Rubik's Cube). Problems like Halting Problem are a different story. &lt;/p&gt;&#xA;" OwnerUserId="19722" LastActivityDate="2018-11-09T14:29:28.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8882" PostTypeId="2" ParentId="8879" CreationDate="2018-11-09T16:24:36.010" Score="3" Body="&lt;p&gt;With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long as the discriminator returns &quot;real&quot; (vs. &quot;fake&quot;) the generator &quot;wins&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hope is that as the generator and discriminator are trained simultaneously, each will attempt to capture the faults of the other. As the discriminator gets better for one mode in telling real from fake, the generator will focus more on the other modes where it will have an easier time fooling the discriminator on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that isn't the only way things can happen. When the discriminator gets better on distinguishing one mode, the generator can instead concentrate on that mode. Instead of spreading its training over different modes, it can instead attempt to get really good at producing samples within a particular mode. As long as the discriminator can be fooled into reporting &quot;real&quot; for the samples being produced, the generator gets the same reward as if it were to spread itself across multiple modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think about the forger/detective analogy. Instead of a forger trying to learn how to be good at making a range of paintings (Caravaggios, van Goghs, Warhols, etc.) the forger can concentrate on being really good at making a particular style of painting (e.g. Dutch Masters) or even a particular artist. Someone who spends all their time perfecting their Rembrandt forgeries will do a much better job evading detection than someone who tries to be good at both Picasso and Rembrandt (and Klee and Miró and ...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variational Autoencoders are different, though. With a VAE, the latent space is built from &lt;em&gt;all&lt;/em&gt; of the input examples. There's no way for a VAE to concentrate on a particular mode. All of the training examples are being mapped to the same latent space. If, for some reason, the decoder network decided to concentrate on a single mode, mapping the latent space to outputs of just that mode, then all the training examples from the other modes would have poor recovery. This means there would be a strong loss and gradient correcting that for the decoder training. The decoder training is balanced across all the input examples/modes, so the decoder needs to be similarly balanced when decoding the latent space. GANs don't have this similar balance, as the generator doesn't necessarily need to be balanced in its output - just so long as it can reliably fool the discriminator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That doesn't mean that VAEs don't have &lt;a href=&quot;https://towardsdatascience.com/with-great-power-comes-poor-latent-codes-representation-learning-in-vaes-pt-2-57403690e92b&quot; rel=&quot;nofollow noreferrer&quot;&gt;similar issues&lt;/a&gt;, though they're not identical to mode collapse. The problem with VAEs is that each individual example is (&lt;em&gt;in extremis&lt;/em&gt;) being mapped to the same random distribution in the latent space. This can encourage the decoder to ignore the latent variable input, generating output more-or-less arbitrarily. There's a number of approaches recently (in particular &lt;a href=&quot;https://arxiv.org/abs/1511.05644&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial Autoencoders&lt;/a&gt; as well as &lt;a href=&quot;https://arxiv.org/abs/1706.02262&quot; rel=&quot;nofollow noreferrer&quot;&gt;InfoVAE&lt;/a&gt; and the closely related &lt;a href=&quot;https://arxiv.org/abs/1711.01558&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wasserstein Autoencoders&lt;/a&gt;) which attempt to remedy this by constraining the latent space distribution on an ensemble level, rather than on an individual training example basis.&lt;/p&gt;&#xA;" OwnerUserId="18652" LastEditorUserId="30339" LastEditDate="2019-10-09T22:26:46.883" LastActivityDate="2019-10-09T22:26:46.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8884" PostTypeId="2" ParentId="3176" CreationDate="2018-11-09T17:53:51.313" Score="-1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Machine Intelligence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe intelligence is not a &lt;strong&gt;proprietary entity&lt;/strong&gt; meant only living beings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact the very origin of human intelligence is unknown. It is still not known if we can generate a brain just by fixing up the corresponding molecules of the real brain(Even theoretically). Even if we could do that does that constitute a real intelligence or artificial intelligence is still very hazy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the phrase &lt;strong&gt;&quot;Machine Intelligence&quot;&lt;/strong&gt; would sound appropriate.&lt;/p&gt;&#xA;" OwnerUserId="19738" LastEditorUserId="19738" LastEditDate="2018-11-09T17:59:25.083" LastActivityDate="2018-11-09T17:59:25.083" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8885" PostTypeId="1" CreationDate="2018-11-09T18:08:51.777" Score="5" ViewCount="5681" Body="&lt;p&gt;I observed in several papers that the Variational autoencoder's output is blurred while GANs output is crisp and has sharp edges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please give some intuition why that is the case? I did think a lot but couldn't find any logic. &lt;/p&gt;&#xA;" OwnerUserId="19738" LastActivityDate="2019-01-15T06:23:41.407" Title="Why are Variational autoencoder's output is blurred while GANs output is crisp and has sharp edges?" Tags="&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;&lt;autoencoders&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8889" PostTypeId="2" ParentId="3176" CreationDate="2018-11-09T19:35:36.903" Score="0" Body="&lt;p&gt;Josh Worth provides a critique for the use of the term &lt;em&gt;artificial intelligence&lt;/em&gt; in his &lt;a href=&quot;https://www.joshworth.com/stop-calling-in-artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stop Calling it Artificial Intelligence&lt;/a&gt;, but there are some caveats in the treatment of the term.  (One is a typo in the URL path.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not the term &lt;em&gt;Artificial Intelligence&lt;/em&gt; that is the issue.  It is what is collected under it in the media and sometimes in academic literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is &lt;em&gt;Artificial&lt;/em&gt; Unclear?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;artificial&lt;/em&gt; is not particularly ambiguous or inaccurate.  Artificial doesn't mean fake as Worth suggests.  It simply means that it did not arise from natural processes.  High end artificial flowers feel and smell like they are grown.  Artificial flight is now called flying.  We no longer see birds as the exclusive pilots of flight, so the word flying has changed to include artificial things called aircraft.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People don't normally make the mistake of placing the adjective &lt;em&gt;artificial&lt;/em&gt; along side human capabilities, so there is no tendency to misnomers.  There is a problem with the use of the term &lt;em&gt;artificial intelligence&lt;/em&gt; though.  Worth is correct in that overall statement.  There may be two forks to the misuse of the term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Defining Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem is with the term &lt;em&gt;intelligence&lt;/em&gt;.  The definitions we've seen are considerably problematic.  Some are flat out wrong and can be proven so with heaps of counterexamples.  Furthermore, definitions tend to be qualitative and difficult if not impossible to quantify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some propose the standardized testing of academia to quantify intelligence. If we use that definition of the word, from the staunch g-factor adherents, artificial intelligence is a target idea and nothing comes close to approximating it. No computer system has yet been admitted into a major university on the basis of high scores in college board testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use the ability to learn as a standard, then flies are intelligent because they frequently learn from one swat strategy changes to avoid getting squashed by the irritated person.  There are a thousand other reasons why learning, by itself, is not an adequate characterization of human intelligence.  A crack addict can learn how to make a purchase without holding a job. We wouldn't characterize that as intelligence, but rather dysfunction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Excessively Inclusive Use of the Term&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem arises from avarice. To appear as an expert in what is perceived to be a lucrative expansion of technology, some who have no conception of the topic sometimes present conjecture as if it were peer reviewed fact.  This has been typical of popular topics for centuries.  There is often insufficient bandwidth of peer verification available to address even a small portion of publicly available information.  Web publication has only augmented this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Resulting from this coveting of expert reputation is collecting under the name &lt;em&gt;artificial intelligence&lt;/em&gt; a number of things that are not intelligent.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multi-dimensional control systems&lt;/li&gt;&#xA;&lt;li&gt;Brute force searches of permutations&lt;/li&gt;&#xA;&lt;li&gt;Decisions made by statistically analyzing sample data&lt;/li&gt;&#xA;&lt;li&gt;Parameterized functional networks that converge to a defined optimal behavior&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Those of the above that have no component of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cognition,&lt;/li&gt;&#xA;&lt;li&gt;Comprehension,&lt;/li&gt;&#xA;&lt;li&gt;Complex modelling and use of such models,&lt;/li&gt;&#xA;&lt;li&gt;Semantic mapping,&lt;/li&gt;&#xA;&lt;li&gt;Rational inference,&lt;/li&gt;&#xA;&lt;li&gt;Or some other clearly distinctive and broad form of adaptation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;should not be included under the technology that exhibits authentic &lt;em&gt;artificial intelligence&lt;/em&gt;.  However, those that developed the theory of control systems, searches, parameterized function convergence, and statistics or developed working systems that use that theory are intelligent, just not artificial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above four might have terms that distinguish them from authentic intelligent systems outside the realm of biology.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MDC &amp;mdash; Multi-dimensional Control&lt;/li&gt;&#xA;&lt;li&gt;BSS &amp;mdash; Brute force search&lt;/li&gt;&#xA;&lt;li&gt;NBL &amp;mdash; Network Based Learning&lt;/li&gt;&#xA;&lt;li&gt;SDS &amp;mdash; Statistical decisioning systems&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It seems that terms that have three words and form distinct acronyms get further and last longer.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-09T19:35:36.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8890" PostTypeId="2" ParentId="8878" CreationDate="2018-11-09T20:15:34.593" Score="3" Body="&lt;p&gt;The question, &quot;Why do we need artificial intelligence?&quot; is quite to the point.  Technically, the answer is, &quot;No reason.&quot;  If we needed artificial intelligence, then we would have become extinct over the last 50,000 or more years we've been a species with human intelligence, so we want it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We do want it, and there are benefits.  Some claim there are risks too, which is logical.  We should be open to all views.  Some of them may be spot on, and others may be without merit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One list of benefits already seen might include these, with most beneficial on top.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Deeper interest in what exactly we mean when we say someone or some thing is intelligent&lt;/li&gt;&#xA;&lt;li&gt;Revisiting of the validity of the g-factor belief&lt;/li&gt;&#xA;&lt;li&gt;Revised interest in parallel computing architecture&lt;/li&gt;&#xA;&lt;li&gt;Heightened awareness of the distinction between fake and authentic&lt;/li&gt;&#xA;&lt;li&gt;Improved use of an interest in probability and statistics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are further potential benefits that may also emerge.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Young students taking mathematics more seriously than hacking&lt;/li&gt;&#xA;&lt;li&gt;The elimination of circular, wasteful job functions&lt;/li&gt;&#xA;&lt;li&gt;More leisure time for families to enjoy together&lt;/li&gt;&#xA;&lt;li&gt;Improved economic efficiency&lt;/li&gt;&#xA;&lt;li&gt;Better planning regarding the use and preservation of natural resources&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are the risks from highest to lowest.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Possibly accidentally causing the emergence of an uncontained and potentially dangerous digital entity&lt;/li&gt;&#xA;&lt;li&gt;Additional nihilism (belief that nothing matters) in our culture, which could emerge if the last refuge of human pride is overrun by smarter artificial systems&lt;/li&gt;&#xA;&lt;li&gt;People losing their jobs, which may have only a temporary impact, if the AI does quality work, but the transition may be painful and destructive&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-09T20:15:34.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8891" PostTypeId="2" ParentId="8885" CreationDate="2018-11-09T20:20:34.797" Score="3" Body="&lt;p&gt;In essence, Variational Autoencoders learn an &quot;explicit&quot; distribution of the data by trying to fit the data via a multi-dimensional Gaussian/Normal distribution. However, Generative Adversarial Networks learn an &quot;implicit&quot; distribution of data meaning that you cannot directly sample them. Also, due to the deterministic nature of neural networks GANs tend to learn a &lt;a href=&quot;https://en.wikipedia.org/wiki/Dirac_delta_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dirac Delta function&lt;/a&gt;. If you're lucky and the training of the GAN is successful you can therefore get sharper images sine the model doesn't have to explicitly deal with the noise injected into it due to samplings, hence this could be a simpler learning problem. By deterministic I mean assuming that you have no sampling anywhere in the middle layers of your model and only use the neural network as an input-output mapping function.&lt;/p&gt;&#xA;" OwnerUserId="12347" LastActivityDate="2018-11-09T20:20:34.797" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8893" PostTypeId="2" ParentId="7838" CreationDate="2018-11-09T21:05:32.830" Score="-1" Body="&lt;p&gt;The shortest answer I could can come up with could be as follow; take it with a grain of salt though since we still do not know a lot about natural intelligence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What natural intelligence is could be seen as the process of learning abstract concepts from limited observations with the intention to use them for solving a [new] task. This process involves using those concepts to imagine new, hypothetically correct scenarios/theories and combine them in a meaningful way to cut down the enormous hypothesis space of possibilities and enable generalization to new situations without observing any data beforehand. &lt;strong&gt;Artificial intelligence&lt;/strong&gt; is to bring what natural intelligence does into machines.&lt;/p&gt;&#xA;" OwnerUserId="12347" LastEditorUserId="12347" LastEditDate="2018-11-09T21:51:49.073" LastActivityDate="2018-11-09T21:51:49.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8894" PostTypeId="1" CreationDate="2018-11-10T02:38:22.483" Score="9" ViewCount="2058" Body="&lt;p&gt;Do GANs come under supervised learning or unsupervised learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that they come under supervised learning, as we have labeled dataset of images, but I am not sure as there might be other aspects in GANs which might come into play in the determination of the class of algorithms GAN falls under.&lt;/p&gt;&#xA;" OwnerUserId="18956" LastEditorUserId="2444" LastEditDate="2020-03-05T21:19:28.497" LastActivityDate="2020-08-02T00:10:48.800" Title="Do GANs come under supervised learning or unsupervised learning?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;unsupervised-learning&gt;&lt;generative-adversarial-networks&gt;&lt;supervised-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8896" PostTypeId="5" CreationDate="2018-11-10T03:34:15.110" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-10T03:34:15.110" LastActivityDate="2018-11-10T03:34:15.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8897" PostTypeId="4" CreationDate="2018-11-10T03:34:15.110" Score="0" Body="For questions related to generative adversarial networks (GANs), introduced in the paper Generative Adversarial Nets (2014) by J. Goodfellow et al. A GAN is composed of a discriminative model (D) and a generative model (G). The discriminator D needs to distinguish between data generated by the generator G and data in the training set, while the generator G needs to generate data such that the discriminator D is not able to accomplish its task." OwnerUserId="18956" LastEditorUserId="2444" LastEditDate="2019-08-04T13:59:18.960" LastActivityDate="2019-08-04T13:59:18.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8898" PostTypeId="1" CreationDate="2018-11-10T03:38:04.040" Score="1" ViewCount="53" Body="&lt;p&gt;I need to map from a vector space representation onto a tree structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible solution: given a word vector as input, produce a path in the tree from the root down to the node that most closely matches that vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The path would be a variable length string composed from a finite set of symbols.  The variable length output leads me towards long short-term memory (LSTM) models.  But I've never built a complete LSTM model before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The understanding gap: my vector inputs are already dense representations.  Specifically, I'm working with GloVe right now. But the output path symbols would require a different encoding, correct?  How can I structure/train the necessary encoder-decoder pair so that it can handle both word vectors and these path symbols as input while still being able to produce path symbols as output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: After more research, I think the correct term would be a &quot;one to many&quot; model using an RNN architecture (of which LSTM is one type).  So I wouldn't need an encoder for the one input vector because it's already a dense vector.  But I would need to train an autoencoder for decoding the multiple output vectors that describe the tree path.&lt;/p&gt;&#xA;" OwnerUserId="19703" LastEditorUserId="2444" LastEditDate="2019-04-16T22:41:05.633" LastActivityDate="2019-04-16T22:41:05.633" Title="How could I learn tree paths given word embeddings?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;word-embedding&gt;&lt;glove&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8900" PostTypeId="2" ParentId="8894" CreationDate="2018-11-10T03:51:30.880" Score="-1" Body="&lt;p&gt;GANs are unsupervised machine learning algorithms. According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;, unsupervised algorithms are :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unsupervised learning is a branch of machine learning that learns from test data that has not been labelled, classified or categorized&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In GAN networks, only training data is provided which is not labelled. The network generates candidates ( generative ) which are evaluated by the discriminator. The network slowly learns from the data given from a latent space.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose, you want to create a GAN network which can make a Monet painting. You just need to feed it some Monet paintings. Here, &lt;strong&gt;you are not interested in classifying the painting, but copying/mimicking it&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hence, there is no need for labels here which makes GAN an unsupervised machine learning algorithm.&lt;/p&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-11-10T03:51:30.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8902" PostTypeId="1" AcceptedAnswerId="8907" CreationDate="2018-11-10T08:22:32.563" Score="7" ViewCount="28782" Body="&lt;p&gt;What are the differences between the A* algorithm and the greedy best-first search algorithm? Which one should I use? Which algorithm is the better one, and why?&lt;/p&gt;&#xA;" OwnerUserId="19657" LastEditorUserId="2444" LastEditDate="2019-08-30T00:32:27.860" LastActivityDate="2020-09-25T15:13:51.913" Title="What are the differences between A* and greedy best-first search?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;comparison&gt;&lt;a-star&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8904" PostTypeId="2" ParentId="7838" CreationDate="2018-11-10T10:54:51.110" Score="0" Body="&lt;p&gt;There is no formal definition that most people agree on. Hence here is what I, as a data science / machine learning consultant, think:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence &lt;em&gt;as a research field&lt;/em&gt; is the study of agents which sense and act autonomously in an environment and improve their situation according to some metric with their actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't like the term, because it is too broad / vague. Instead, look at the definition of machine learning by Tom Mitchell:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A computer program is said to learn from experience 'E', with respect to some class of tasks 'T' and performance measure 'P' if its performance at tasks in 'T' as measured by 'P' improves with experience E&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Machine learning is an important part of AI, but not the only one. Search algorithms, SLAM, constrained optimization, knowledge bases and automatic inference are also certainly part of AI.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2020-05-18T05:36:29.170" LastActivityDate="2020-05-18T05:36:29.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8905" PostTypeId="2" ParentId="7879" CreationDate="2018-11-10T12:13:53.190" Score="1" Body="&lt;p&gt;There are two important parts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When you track, you have a trajectory you can expect. So it becomes easier to &quot;look into the future&quot; is you understand the past&lt;/li&gt;&#xA;&lt;li&gt;When you loose an object you had a trajectory of before, you might be able to come up with an explanation that can influence your behavior&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-10T12:13:53.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8906" PostTypeId="1" CreationDate="2018-11-10T12:24:29.683" Score="5" ViewCount="250" Body="&lt;p&gt;If we have a search or path-finding problem, A* and Dijkstra's algorithm require that we formulate it as a search in a graph with nodes and connections between these nodes. If there are obstacles, we also need to encode this information in the graph, so that they are not traversed. Additionally, there may be costs/weights on the connections between points. If such weights/costs are high, the algorithms won't take that path.&lt;/p&gt;&#xA;&lt;p&gt;I've been using A* and Dijkstra's algorithm this way so far. However, it's a bit cumbersome to always have to define the nodes/points and the relationships (or connections) between them. There's no learning here. I just define a graph and the algorithms search on this graph.&lt;/p&gt;&#xA;&lt;p&gt;Let's say I have a white image, a green blob in the middle, and points &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; at either side of the blob, I need to get from &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. I don't have a search space represented as a graph here. I just have this image.&lt;/p&gt;&#xA;&lt;p&gt;Could I use machine learning to solve this problem (and would it generalize to more complex maps)? If so, are there any research works on this topic? Or is that the wrong route to take (pardon the pun)?&lt;/p&gt;&#xA;" OwnerUserId="19761" LastEditorUserId="2444" LastEditDate="2021-02-06T17:23:26.370" LastActivityDate="2021-02-06T17:23:26.370" Title="Given an image and two points $A$ and $B$ on that image, how could we find a path from $A$ to $B$?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;search&gt;&lt;path-planning&gt;&lt;path-finding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8907" PostTypeId="2" ParentId="8902" CreationDate="2018-11-10T13:32:37.873" Score="7" Body="&lt;p&gt;Both algorithms fall into the category of &quot;best-first search&quot; algorithms, which are algorithms that &lt;em&gt;can&lt;/em&gt; use both the knowledge acquired so far while exploring the search space, denoted by &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;, and a heuristic function, denoted by &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt;, which estimates the distance to the goal node, for each node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in the search space (often represented as a graph). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these search algorithms defines an &quot;evaluation function&quot;, for each node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in the graph (or search space), denoted by &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt;. This evaluation function is used to determine which node, while searching, is &quot;expanded&quot; first, that is, which node is first removed from the &lt;a href=&quot;https://ai.stackexchange.com/q/5949/2444&quot;&gt;&quot;fringe&quot; (or &quot;frontier&quot;, or &quot;border&quot;)&lt;/a&gt;, so as to &quot;visit&quot; its children. In general, the difference between the algorithms in the &quot;best-first&quot; category is in the definition of the evaluation function &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of &lt;a href=&quot;https://en.wikipedia.org/wiki/Best-first_search#Greedy_BFS&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;the greedy BFS algorithm&lt;/em&gt;&lt;/a&gt;, the evaluation function is &lt;span class=&quot;math-container&quot;&gt;$f(n) = h(n)$&lt;/span&gt;, that is, the greedy BFS algorithm first expands the node whose estimated distance to the goal is the smallest. So, greedy BFS does not use the &quot;past knowledge&quot;, i.e. &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;. Hence its connotation &quot;greedy&quot;. In general, the greedy BST algorithm is &lt;strong&gt;not complete&lt;/strong&gt;, that is, there is always the risk to take a path that does not bring to the goal. In the greedy BFS algorithm, all nodes on the &lt;em&gt;border&lt;/em&gt; (or fringe or frontier) are kept in memory, and nodes that have already been expanded do not need to be stored in memory and can therefore be discarded. In general, the greedy BFS is also &lt;strong&gt;not optimal&lt;/strong&gt;, that is, the path found may not be the optimal one. In general, the time complexity is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the (maximum) branching factor and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the maximum depth of the search tree. The space complexity is proportional to the number of nodes in the fringe and to the length of the found path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;the A* algorithm&lt;/em&gt;&lt;/a&gt;, the evaluation function is &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Admissible_heuristic&quot; rel=&quot;noreferrer&quot;&gt;admissible heuristic function&lt;/a&gt;. The &quot;star&quot;, often denoted by an asterisk, &lt;code&gt;*&lt;/code&gt;, refers to the fact that A* uses an admissible heuristic function, which essentially means that A* is &lt;strong&gt;optimal&lt;/strong&gt;, that is, it always finds the optimal path between the starting node and the goal node. A* is also &lt;strong&gt;complete&lt;/strong&gt; (unless there are infinitely many nodes to explore in the search space). The time complexity is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;. However, A* needs to keep all nodes in memory while searching, not just the ones in the fringe, because A*, essentially, performs an &quot;exhaustive search&quot; (which is &quot;informed&quot;, in the sense that it uses a heuristic function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, greedy BFS is not complete, not optimal, has a time complexity of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt; and a space complexity which can be polynomial. A* is complete, optimal, and it has a time and space complexity of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;. So, in general, A* uses more memory than greedy BFS. A* becomes impractical when the search space is huge. However, A* also guarantees that the found path between the starting node and the goal node is the optimal one and that the algorithm eventually terminates. Greedy BFS, on the other hand, uses less memory, but does not provide the optimality and completeness guarantees of A*. So, which algorithm is the &quot;best&quot; depends on the context, but both are &quot;best&quot;-first searches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: in practice, you may not use any of these algorithms: you may e.g. use, instead, &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_A*&quot; rel=&quot;noreferrer&quot;&gt;IDA*&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-10T13:49:14.970" LastActivityDate="2018-11-10T13:49:14.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8909" PostTypeId="2" ParentId="8821" CreationDate="2018-11-10T14:44:12.973" Score="10" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;noreferrer&quot;&gt;A*&lt;/a&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Best-first_search&quot; rel=&quot;noreferrer&quot;&gt;best-first search algorithm&lt;/a&gt;, which means that it is an algorithm that uses both &quot;past knowledge&quot;, gathered while exploring the search space, denoted by &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;, and an &lt;a href=&quot;https://en.wikipedia.org/wiki/Admissible_heuristic&quot; rel=&quot;noreferrer&quot;&gt;admissible heuristic function&lt;/a&gt;, denoted by &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt;, which estimates the distance to the goal node, for each node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. There are other best-first search algorithms, which differ only in the definition of their &quot;evaluation function&quot;, denoted by &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt;. For example, the evaluation function of A* is &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is admissible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A* is &lt;strong&gt;optimal&lt;/strong&gt;, that is, it finds the optimal path between the starting and goal nodes or states. The optimality of A* is due to the use of an admissible heuristic function, which is a heuristic function which always underestimates the distance to the goal. A* is also &lt;strong&gt;complete&lt;/strong&gt;, that is, it eventually finds this optimal path. However, the problem with A*, in many real-world situations, is that it has an &lt;strong&gt;exponential space complexity&lt;/strong&gt;; more specifically, its space complexity is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the (maximum) branching factor and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the maximum depth of the search tree. It also has a time complexity of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;, but, in practice, it tends to find the solutions quite &quot;fast&quot; (for relatively small search spaces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_A*&quot; rel=&quot;noreferrer&quot;&gt;IDA*&lt;/a&gt; combines A* and &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search&quot; rel=&quot;noreferrer&quot;&gt;IDDFS (or IDS)&lt;/a&gt;. It does it &quot;well&quot;, so it gets the advantages of both algorithms. It can be thought of as the IDDFS which uses, instead of the depth, the evaluation function &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt; as the &quot;cost threshold&quot; or &quot;cutoff&quot;. In the IDA* algorithm, a child &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; of a node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is added to the &lt;a href=&quot;https://ai.stackexchange.com/q/5949/2444&quot;&gt;frontier (or fringe or border)&lt;/a&gt; if &lt;span class=&quot;math-container&quot;&gt;$f(c) &amp;lt; T$&lt;/span&gt;. This threshold &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is, at the beginning of the algorithm, the estimate of the distance to the goal from the starting node &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$T = h(s)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; increases at each iteration of the algorithm (See the &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_A*#Pseudocode&quot; rel=&quot;noreferrer&quot;&gt;pseudocode of the IDA* algorithm&lt;/a&gt; for the details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IDA* is also &lt;strong&gt;complete&lt;/strong&gt; and &lt;strong&gt;optimal&lt;/strong&gt;, but, as opposed to A*, it has a &lt;strong&gt;polynomial space complexity&lt;/strong&gt;, more specifically, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(bd)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the (maximum) branching factor and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the maximum depth fo the tree. IDA* still has an exponential time complexity of A*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, IDA* has a better memory usage than A*. As in A* and unlike IDDFS, it concentrates on exploring the most promising nodes, and thus does not go to the same depth everywhere in the search tree (whereas ordinary IDDFS does). A* can be thought of as a dynamic programming algorithm. IDA* often ends up exploring the same nodes many times (as IDDFS), but, asymptotically, both A* and IDA* have the same exponential time complexity, that is, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, which one is the best one? A* becomes impractical when the search space is huge, because of the memory constraints. So, in those cases, IDA* is definitely more appropriate. In general, IDA* is one of the very best optimal state space&#xA;search techniques around. However, A* is conceptually simpler than IDA*. As a consequence, in practice, A* may be easier to implement than IDA*, but, in real-world scenarios, this is not really a problem, as they are both relatively easy to implement (anyway).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-10T15:08:16.640" LastActivityDate="2018-11-10T15:08:16.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8910" PostTypeId="1" CreationDate="2018-11-10T15:27:45.823" Score="4" ViewCount="303" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is the following: Given 1000 wins, losses, and ties from a chess simulation I am using, what shape should each game be (I.e., sequence of moves leading to win/loss/tie) in order to build an deep neural network on it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Literature Review&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/4672/can-an-ai-learn-how-to-play-chess-without-instructions&quot;&gt;Kind of got me in the right direction.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/4910/how-would-you-encode-your-input-vector-matrix-from-a-sequence-of-moves-in-game-l&quot;&gt;Great theoretical way of thinking about it, but left me wanting detail.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://erikbern.com/2014/11/29/deep-learning-for-chess.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gave a good high level tutorial, but not enough detail.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/lamesjim/Chess-AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha beta pruning python example, but no neural network.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/erikbern/deep-pink/blob/master/reinforcement.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural network using reinforcement learning, but I cant figure out what the (X,y) shapes are and their meaning from just digging into the source code.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Current Situation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently have a sample of 1000 wins, 1000 losses, and 1000 ties from the &lt;code&gt;python-chess&lt;/code&gt; api, so if &lt;code&gt;i&lt;/code&gt; is the index of a game, then this is the structure of the current dataset I am working with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;game_i -&amp;gt; (num_moves_i,8,8,16)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So each &lt;code&gt;game_i&lt;/code&gt; where &lt;code&gt;i in {1..3000}&lt;/code&gt; and &lt;code&gt;num_moves_i is variable&lt;/code&gt; depending on the game (E.g., 14 for a good winning game, or 765 moves for a tie game). The 16 represents a &lt;code&gt;one_hot_encoding&lt;/code&gt; for one of the 16 unique board pieces. The data set is also an alternating board state, so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;game_i[0] == board state of whites first move&#xA;game_i[1] == baord state of blacks first move&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I also have alpha-beta pruning and maximin working, so for each move I have an intrinsic value associated with it using recursion three levels deep. Leading my current approach to a regression of a given move, essentially leading me to believe the AI would simply learn the heuristic and predict a value the heuristic would give.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly my proof of concept 1000 winning games is not enough to make a meaningful AI, but that isn't my goal. I want to learn the techniques, not produce an enterprise scale chess AI.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does the tensor shape make sense?&lt;/li&gt;&#xA;&lt;li&gt;Is this a reinforcement learning problem? If so, how can I shape my current framing into that type of thinking? Theory in this area would be greatly appreciated as I am less familiar with it.&lt;/li&gt;&#xA;&lt;li&gt;Is this a RNN/LSTM problem? (E.g., predict the next board state).&lt;/li&gt;&#xA;&lt;li&gt;Is this a regression problem?&lt;/li&gt;&#xA;&lt;li&gt;Is this a sequence mining problem?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What is the standard approach to framing this problem, once you have data falling through the pipeline. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your support is more than appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;* UPDATE (STILL IN RESEARCH) *&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With further research, a candidate label for the training data is the tensor and the item from the state-space that was selected with that board state. Then only keep games containing sequences of moves which obtain a cumulative value &gt;= &lt;code&gt;epsilon&lt;/code&gt;. This would require a &lt;code&gt;one_hot_encoding&lt;/code&gt; of all moves played in all games we wish to train on, as labels. E.g., &lt;code&gt;(game_i,board_ij,e2e6)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="19764" LastEditorUserId="19764" LastEditDate="2018-11-10T21:55:51.217" LastActivityDate="2020-11-30T02:04:20.377" Title="Building AI from chess - data shape from simulation" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;chess&gt;&lt;structured-data&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-21T02:11:53.173" ContentLicense="CC BY-SA 4.0" />
  <row Id="8914" PostTypeId="2" ParentId="8906" CreationDate="2018-11-10T16:47:49.953" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Let's say I have a white image, a green blob in the middle, and points &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; at either side of the blob, I need to get from &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. I don't have points to traverse. I just have this image.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If your image is as &amp;quot;simple&amp;quot; as you describe here, with very easily distinguishable colours, the easiest solution would likely be to construct a graph as expected by algorithms such as A* as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Every pixel becomes a node in the graph, with connections to all adjacent pixels&lt;/li&gt;&#xA;&lt;li&gt;Green pixels (the green blob in the middle) are marked as non-traversable (or simply don't have connections to adjacent pixels) (I assume this thing is your obstacle).&lt;/li&gt;&#xA;&lt;li&gt;Pixels belonging to the points on either side of the obstacle are marked as starting and goal vertices.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Then, you have the graph that a traditional pathfinding algorithm like A* would expect, and can simply run the algorithm there.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you get less &amp;quot;clear&amp;quot; images (i.e. not brightly coloured, easily distinguishable things, but more like real-life top-down images of an area), then the above won't really work anymore. In such cases, you'd likely want to look into Reinforcement Learning approaches, where you give an agent a reward when it successfully manages to find a connection between start and goal. In particular, you'd need &lt;strong&gt;Deep&lt;/strong&gt; Reinforcement Learning approaches, because otherwise you're not going to be able to handle complex images.&lt;/p&gt;&#xA;&lt;p&gt;Note that these approaches are not going to be simple to implement or immediately understand for 100% if you're a beginner. There won't be any &amp;quot;beginner-friendly&amp;quot; approaches for the problem you describe, it simply is quite a complex problem (requiring a mix of &amp;quot;decision-making&amp;quot; and &amp;quot;understanding&amp;quot; images).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2021-02-06T16:55:53.050" LastActivityDate="2021-02-06T16:55:53.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8916" PostTypeId="1" CreationDate="2018-11-10T19:38:48.130" Score="3" ViewCount="231" Body="&lt;p&gt;How is a feed-forward neural network with few hidden layers and lots of nodes in those hidden layers different from a network with a lot of hidden layers but relatively lesser nodes in those hidden layers?&lt;/p&gt;&#xA;" OwnerUserId="19769" LastEditorUserId="75" LastEditDate="2018-11-14T22:51:28.573" LastActivityDate="2018-11-14T22:51:28.573" Title="Significance of depth of a deep neural network" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8919" PostTypeId="2" ParentId="8910" CreationDate="2018-11-10T20:40:11.133" Score="0" Body="&lt;p&gt;Not an expert here by any means. I think the simplest way to approach the chess problem is to have three components a) a game logic component, that can generate all the possible legal moves from a given board position b) an evaluation function that can score a given game position and c) your minimax/alphabeta algorithm that searches through the tree of possible moves generated by a) looking for the branch that returns the best score using b). Here, the machine learning part would be the evaluation function. You would just transform each board position in your data into a feature vector, and train some kind of classifier to return the probability of win/lose at that given board position. I think it's highly doubtful that this would work with such a small data set though.&lt;/p&gt;&#xA;" OwnerUserId="19768" LastActivityDate="2018-11-10T20:40:11.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8920" PostTypeId="1" CreationDate="2018-11-10T20:40:17.907" Score="1" ViewCount="49" Body="&lt;p&gt;One disadvantage or weakness of Artificial Intelligence today the slow nature of learning or training success. For instance, an AI agent might require a 100,000 samples or more to reach an appreciable level of performance with a specific task. But this is unlike humans who are able to learn very quickly with a minimum number of samples. Humans are also able to teach one another, or in other words, transfer knowledge acquired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: are Artificial Intelligence learnings or trainings transferable from one agent to the other? If yes, how? If no, why?&lt;/p&gt;&#xA;" OwnerUserId="17892" LastActivityDate="2018-11-13T09:04:55.203" Title="Are artificial intelligence learnings or trainings transferable from one agent to the other?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8921" PostTypeId="2" ParentId="8920" CreationDate="2018-11-10T21:43:12.950" Score="1" Body="&lt;p&gt;The simplest case is copying the software. That is instant duplication of the learnings. In a similar, but less trivial way you can adjust pretrained neural network classifiers to new datasets by simply re-initializing the last layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It gets more interesting when you have multiple agents and you want to combine their knowledge. This can be done with many techniques; the simplest is averaging the weights of identical neural network architectures. One example of this &lt;em&gt;combined learning&lt;/em&gt; happened is &lt;a href=&quot;https://www.popularmechanics.com/technology/robots/a20190/google-room-full-of-robot-arms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Has a Room Full of Robot Arms Learning Hand-Eye Coordination&lt;/a&gt;. The relevant paper is &lt;a href=&quot;https://arxiv.org/pdf/1603.02199.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most complicated case, where you want to transfer just the gist and prevent forgetting is -- up to my knowledge, and I'm relatively certain it's outdated in that area -- unsolved. The problem is &lt;strong&gt;post-hoc explanations&lt;/strong&gt;. Humans decide to do something, observe the outcome and then explain why they did something / why it was good to do it. So we automatically generate hypotheses about the world. And sometimes we are able to formulate them in a way that others can understand them. That is not possible with current learning machines. We don't know yet how to abstract automatically from arbitrary problems.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-13T09:04:55.203" LastActivityDate="2018-11-13T09:04:55.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8924" PostTypeId="1" AcceptedAnswerId="8925" CreationDate="2018-11-11T04:11:48.587" Score="0" ViewCount="1402" Body="&lt;p&gt;How are artificial neural networks different from normal computer programs (or software)?&lt;/p&gt;&#xA;" OwnerUserId="19220" LastEditorUserId="2444" LastEditDate="2019-05-01T17:24:06.433" LastActivityDate="2019-05-01T17:24:06.433" Title="How are artificial neural networks different from normal computer programs?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8925" PostTypeId="2" ParentId="8924" CreationDate="2018-11-11T06:05:02.073" Score="1" Body="&lt;p&gt;ANNs learn and adapt to the patterns inside inputs, however, general PCs don't. PCs take the process as input and provide results based on user preferences, but never adapt i.e., always depends on users what to do with the data, meanwhile, ANNs are customized for specific tasks like forecasting, function approximations etc.&lt;/p&gt;&#xA;" OwnerUserId="19776" LastEditorUserId="4302" LastEditDate="2018-11-11T21:33:16.473" LastActivityDate="2018-11-11T21:33:16.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8927" PostTypeId="1" CreationDate="2018-11-11T11:04:13.450" Score="2" ViewCount="112" Body="&lt;p&gt;&lt;strong&gt;How does Recurrent Neural Network updates its weights and bias through backpropagation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is time taken into account while updating the weights of a RNN using Backpropagation through time(BPTT)?&quot;&lt;/p&gt;&#xA;" OwnerUserId="19780" LastEditorUserId="1671" LastEditDate="2018-11-19T18:43:39.417" LastActivityDate="2018-11-19T18:43:39.417" Title="Update of weights in Recurrent Neural Network through back propagation" Tags="&lt;ai-basics&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;&lt;concepts&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8929" PostTypeId="1" CreationDate="2018-11-11T13:47:45.863" Score="3" ViewCount="108" Body="&lt;p&gt;I have a question about convolutional neural newtork.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this image: &lt;a href=&quot;https://i.stack.imgur.com/ltOKu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;conv example&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a part of an input matrix and a filter. Ok, now we can do the convolution and the result is a scalar, if it is a large number the future was found otherwise no. So, the features map is a matrix where each number indicates the points where a feature was found. I understood this.&#xA;The output of this convolution is a features map (after activation function).&#xA;My misunderstanding start here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next convolution, will find another feature. I don't understand how this filter, can find a new feature from a representation of where the previous feature was found. The feature maps, output of the first convolution is a rappresentation &lt;strong&gt;of indicates only where the features were found&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shouldn't be the features found instead of a number that indicates how much was found this feature??&#xA;How exactly does this work?&lt;/p&gt;&#xA;" OwnerUserId="19786" LastActivityDate="2018-11-11T17:41:41.427" Title="Features Map convolutional neural network" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8931" PostTypeId="2" ParentId="8929" CreationDate="2018-11-11T14:20:27.923" Score="2" Body="&lt;p&gt;Since the kernel size is limited. A feature can encompass more than 3 pixels or say 5 pixels, for that we cannot find it in 1 layer. So by adding 2 layers and having an activation function that is not like a step function which you would prefer, we extend the feature to 5 or 9 pixels. And with n layers, a feature covering 2n+1, or 4n+1 can be seen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. You have some spelling errors. Take your time to frame well. Welcome!&lt;/p&gt;&#xA;" OwnerUserId="19027" LastActivityDate="2018-11-11T14:20:27.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8932" PostTypeId="1" AcceptedAnswerId="8942" CreationDate="2018-11-11T14:39:08.293" Score="2" ViewCount="124" Body="&lt;p&gt;My AI (for the card game schnapsen) currently calculates every possible way the game could end and then evaluates the percentage of winning for every playable card / move. The calculation is done recursively using a tree. If a game could move on in three different ways the percentage of winning on this node would be &lt;code&gt;mean * (1 - (standardDeviation * f)) * 100&lt;/code&gt; where f is between 0 and 2. When the game can't move on and the AI wins the percentage is 100, when lost 0. I'm including the standard deviation in this formula to prevent the AI from risking too much. In other words: I'm using a MCTS that uses percentages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better formula or way of calculating the next move to maximize the chance of winning? Does including the standard deviation make sense?&lt;/p&gt;&#xA;" OwnerUserId="19783" LastEditorUserId="19783" LastEditDate="2018-11-11T20:45:58.053" LastActivityDate="2018-11-12T17:50:20.777" Title="Weighted move rating for AI" Tags="&lt;algorithm&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8933" PostTypeId="2" ParentId="8929" CreationDate="2018-11-11T15:17:52.770" Score="0" Body="&lt;p&gt;Well, let's start from the beginning, convolution is an operation that is used not only in neural networks. Actually, people were using it in signal processing long before neural networks. Convolution is also used in image processing, in photoshop for example. So let's start from image processing. Take a look at &lt;a href=&quot;http://setosa.io/ev/image-kernels/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; link. Here you can see how we can use conv op to process an image. If you choose an outline kernel you can basically extract edges from that image using 3x3 kernel and applying it on the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when you train your network you change parameters of that kernel in a way that minimizes your loss function. However, it is a really hard question what representations your CNN will learn. But for the sake of the example, let's assume that in the first layer your network learned to extract edges from an input image. Then you feed that feature map to the second layer and the second layer now should learn to extract new features from those edges provided by the first layer. For example, it can learn to detect horizontal edges or vertical one, so if you trying to recognize cars vs pedestrians, for example, cars will have more horizontal edges when pedestrians will have more vertical edges. Also note, that we usually make several feature maps per layer, so in this example, 2nd layer can output 16 feature maps produced by different kernels with different parameters so it could recognize vertical edges, horizontal edges, round one and so on. After that you feed the output of the 2nd layer, let's say, to the fully connected layer and the fc layer will say &quot;ok, I see that previous layer recognized horizontal edges so I must output higher probability for the car, but if 2nd layer recognized vertical edges I would output higher probability for the pedestrian&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this is oversimplified and in reality, representations learned by the network will be way more abstract, but I think just for the sake of gaining intuition this is an acceptable example. But if you are looking to gain a deeper knowledge about what CNNs learn in their layers read &lt;a href=&quot;https://arxiv.org/pdf/1311.2901.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; article, not an easy reading, especially if you don't know much about CNNs, but it is quite insightful. Also if you don't want to read that article here is a &lt;a href=&quot;https://youtu.be/ghEmQSxT6tw&quot; rel=&quot;nofollow noreferrer&quot;&gt;video presentation&lt;/a&gt; made by the authors of that paper on youtube.&lt;/p&gt;&#xA;" OwnerUserId="13102" LastActivityDate="2018-11-11T15:17:52.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8934" PostTypeId="2" ParentId="8929" CreationDate="2018-11-11T17:41:41.427" Score="-1" Body="&lt;p&gt;Three reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Limited field of view by one small kernel&lt;/li&gt;&#xA;&lt;li&gt;Layers of abstraction and combinations of intermediate features&lt;/li&gt;&#xA;&lt;li&gt;Non-linearity of decision boundary&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The previous two answers by caissalover and Andrew got the first two points. Let's elaborate on the third:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convolution is a linear transformation of the input. But not everything can be properly expressed with linear operations. This is why we apply non-linear activation functions (e.g. ReLU) in between. &lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-11T17:41:41.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8937" PostTypeId="2" ParentId="8924" CreationDate="2018-11-11T22:34:12.687" Score="2" Body="&lt;p&gt;Well, a computer consists of the hardware (CPU, GPU, memory, etc.), the operational system (OS) and software/programs (OS is also a software, but it is a very special software).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is simply a program that runs on a computer. Like a calculator app. In fact, the most simple NNs are indeed just some sort of a calculator that performs matrix multiplications.&lt;/p&gt;&#xA;" OwnerUserId="13102" LastActivityDate="2018-11-11T22:34:12.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8938" PostTypeId="1" CreationDate="2018-11-12T05:21:00.323" Score="2" ViewCount="35" Body="&lt;p&gt;Every neural network updates its weights through back-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How is back-propagation used for updating weights in a combination of 2 or more neural networks (e.g.:CNN-LSTM, GAN-CNN, etc.)&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance a CNN-LSTM model is a CNN model stacked on top of an LSTM model. When CNN model is stacked on top of an LSTM model, do we consider hidden layer of both model or hidden layer of outer model(LSTM)?&lt;/p&gt;&#xA;" OwnerUserId="19780" LastActivityDate="2018-11-12T05:21:00.323" Title="Use of backpropagation for weight updates in a combination of 2 neural networks" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8939" PostTypeId="2" ParentId="8924" CreationDate="2018-11-12T06:14:39.780" Score="2" Body="&lt;p&gt;&lt;strong&gt;Personal Computers&lt;/strong&gt; are hardware, whereas artificial neural networks are software. (There are also neuromorphic chips, but that is a different story.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;traditional computer program&lt;/strong&gt; receives some input, calculates stuff based on predefined rules / flow diagrams and generates the output and side effects (such as changed files). The complexity often lies in the number of rules: What is the most reasonable part to adjust, given a change of requirements / a bug? How can I be sure I didn't forget an important edge case? Do I have off-by-one errors? Null-Pointer references? Overflows?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;strong&gt;artificial neural network&lt;/strong&gt; is just one machine learning model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Machine learning&lt;/strong&gt; programs are also software, but they take data and an optimization criterion to infer the desired rules. There is no machine learning without data. The side effect of a machine learning programs training algorithm is the trained model, which usually is a big binary file. The complexity often lies in the data/the model: Is the training data representative for my real task? Did the model learn something reasonable? When / how often do I have to retrain? How can we maintain this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise, here is Tom Mitchells definition of machine learning:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A computer program is said to learn from experience E with respect to some task T and some performance measure P if its performance on T, as measured by P, improves with experience E.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-12T06:14:39.780" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8940" PostTypeId="1" CreationDate="2018-11-12T10:11:25.020" Score="2" ViewCount="199" Body="&lt;p&gt;Cisco and other companies are using Tara AI—a matching tool that connects IT projects with freelancers who have the exact skills required to complete them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking for an explanation of how Tara AI works.  (&lt;a href=&quot;https://www.forbes.com/sites/abdullahimuhammed/2018/06/10/how-ai-is-making-an-entry-into-the-freelance-economy/#3a21634c6d0e&quot; rel=&quot;nofollow noreferrer&quot;&gt;I read about it on Forbes&lt;/a&gt; but cannot find info about the AI methods utilized via Google search.)&lt;/p&gt;&#xA;" OwnerUserId="19108" LastEditorUserId="1671" LastEditDate="2018-11-13T22:14:17.967" LastActivityDate="2019-04-28T20:01:51.620" Title="How does Tara AI work?" Tags="&lt;theory&gt;&lt;applications&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8941" PostTypeId="2" ParentId="8916" CreationDate="2018-11-12T15:57:07.817" Score="1" Body="&lt;p&gt;Note: The following statements are about feed-forward neural networks. If you're interested in something else, please let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a paper which proofs that one hidden layer is enough. So theoretically you could represent any network with many layers with a single hidden layer. I don't remember the title, though. Also I don't remember how it scaled (how many additional nodes you need per added layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, it is a very different story anyway: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You need to train the network in an acceptable wall-clock time,&lt;/li&gt;&#xA;&lt;li&gt;able to fit stuff in memory&lt;/li&gt;&#xA;&lt;li&gt;You have limitations on maximum time for one inference&lt;/li&gt;&#xA;&lt;li&gt;You need to find the weights by a training algorithm&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Anecdotally, I can say wider networks are usually have more problems with overfitting / memory limitation problems while deeper networks have more problems with maximum inference time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a nice paper from Microsoft where they trained the first super deep network. Many papers followed to that one:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;He, Kaiming, et al. &quot;Deep residual learning for image recognition.&quot; Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One main claim was that deeper networks learn more complex features. I don't remember how much of a problem overfitting was. Another important insight is that &lt;strong&gt;skip connections are critical for really deep networks&lt;/strong&gt; (&gt; 20 learning layers).&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-12T16:02:26.067" LastActivityDate="2018-11-12T16:02:26.067" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8942" PostTypeId="2" ParentId="8932" CreationDate="2018-11-12T17:00:06.697" Score="2" Body="&lt;p&gt;It can make sense to incorporate the standard deviation of Monte-Carlo-based evaluations in some ways to reduce risk, but I don't think the way of using it that you described would work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For MCTS evaluations, if you're taking a zero-sum-style approach (which you are if you are trying to estimate win/loss probabilities), it is very important that your estimates are &quot;symmetric&quot; with respect to the players. If you evaluate the probability of winning for player A to be &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, it is important that your algorithm simultaneously evaluates the probability of the other player B to be equal to &lt;span class=&quot;math-container&quot;&gt;$1 - p$&lt;/span&gt; (or, if you prefer winning chance, if the chance of winning for one player is &lt;span class=&quot;math-container&quot;&gt;$p\%$&lt;/span&gt;, it should be &lt;span class=&quot;math-container&quot;&gt;$100\% - p\%$&lt;/span&gt; for the other player). This appears to be violated by your idea, which subtracts standard deviation regardless of player &quot;perspective&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better place to take standard deviation (or, similarly, variance) into account would be, for example, the Selection phase of MCTS. The most common strategy for the Selection phase is using the &quot;UCB1 equation&quot;. You can modify that to include the variance in your observations, for example using the &quot;UCB1-Tuned&quot; strategy as described in the beginning of Section 4 of &lt;a href=&quot;https://homes.di.unimi.it/~cesabian/Pubblicazioni/ml-02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite-time Analysis of the Multiarmed Bandit Problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In my answer above, I assume that you were talking about evaluations &quot;inside&quot; the algorithm, while it is still running. If you were rather thinking of the final move selection for the &quot;real&quot; game after having run the algorithm for a while, the most common approach there is to simply play the move with the maximum number of &lt;strong&gt;visits&lt;/strong&gt; (also referred to as &lt;strong&gt;robust child&lt;/strong&gt;), rather than playing the move with the maximum score. It should not be necessary to include standard deviation at this stage anymore.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2018-11-12T17:50:20.777" LastActivityDate="2018-11-12T17:50:20.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8943" PostTypeId="1" CreationDate="2018-11-12T17:00:20.057" Score="4" ViewCount="2340" Body="&lt;p&gt;What are the differences between the uniform-cost search (UCS) and greedy best-first search (GBFS) algorithms? How would you convert a UCS into a GBFS?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2019-11-10T17:23:19.370" LastActivityDate="2019-11-10T17:23:19.370" Title="What are the differences between uniform-cost search and greedy best-first search?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;uniform-cost-search&gt;&lt;best-first-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8945" PostTypeId="2" ParentId="5939" CreationDate="2018-11-12T17:23:34.103" Score="0" Body="&lt;p&gt;The techniques you mention use GANs. The key idea of GANs is that you have a generator and a discriminator. The generator generates new content, the discriminator has to tell if the content is from the real data or if it was generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discriminator is way more powerful. It should not be too hard to train a discriminator to detect fakes. Training a model which is able to pinpoint the manipulation and understanding of this is a proof of manipulation is harder. It is impossible to get a proof that something is not manipulated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the question how you deal with photoshopped images: you look at differences in compression levels in the image. The keyword to look for is image forensics: &lt;a href=&quot;http://fotoforensics.com/tutorial-estq.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://fotoforensics.com/tutorial-estq.php&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-12T17:23:34.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8947" PostTypeId="2" ParentId="7511" CreationDate="2018-11-12T18:05:37.610" Score="1" Body="&lt;p&gt;This question actually includes many. I try to answer a couple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, you need to make sure for your use case to know the non-functional requirements. It is helpful to know that you have a soft real-time case (if harm is done in case of too late predictions it might also be hard real-time)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Latency&lt;/strong&gt;: how much time may pass after the action was made until the prediction is there?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Stability of prediction&lt;/strong&gt;: how fast it's the system allowed to change it's prediction? Once every 0.5s?&lt;/li&gt;&#xA;&lt;li&gt;General hardware, especially memory usage&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Optimization metric&lt;/strong&gt;: maybe accuracy does not tell you enough?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Evaluation setup&lt;/strong&gt;: likely, you need to have multiple &quot;evaluation points&quot; per video, e.g. Once after the system had one second, once after it had 1.5,... You will also want to consider the stability of the prediction.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now to the question about how you can do that. If you didn't try it so far, I recommend looking at &lt;a href=&quot;https://en.wikipedia.org/wiki/Optical_flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;optical flow&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-13T09:02:49.153" LastActivityDate="2018-11-13T09:02:49.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8954" PostTypeId="1" CreationDate="2018-11-13T15:45:28.540" Score="2" ViewCount="172" Body="&lt;p&gt;I found the below image of how a CNN works &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I5nj1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I5nj1.png&quot; alt=&quot;Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't really understand it. I think I do understand CNNs, but I find this diagram very confusing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My simplified understanding: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Features are selected&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convolution is carried out so that to see where these features fit (repeated with every feature, in every position)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pooling is used to shrink large images down (select the best fit feature).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ReLU is used to remove your negatives&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fully-connected layers contribute weighted votes towards deciding what class the image should be in.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;These are added together, and you have your % chance of what class the image is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Confusing points of this image to me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Why are we going from one image of &lt;span class=&quot;math-container&quot;&gt;$224 \times 224 \times 3$&lt;/span&gt; to two images of &lt;span class=&quot;math-container&quot;&gt;$224 \times 224 \times 64$&lt;/span&gt;? Why does this halving continue? What is this meant to represent?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It continues on to &lt;span class=&quot;math-container&quot;&gt;$56 \times 56 \times 256$&lt;/span&gt;. Why does this number continue to halve, and the number, at the end, the &lt;span class=&quot;math-container&quot;&gt;$256$&lt;/span&gt;, continues to double?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19860" LastEditorUserId="2444" LastEditDate="2020-06-14T22:10:51.447" LastActivityDate="2020-06-14T22:10:51.447" Title="Why does the number of feature maps increases in the VGG model?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;vgg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8955" PostTypeId="2" ParentId="7359" CreationDate="2018-11-13T15:46:14.873" Score="3" Body="&lt;p&gt;Neil’s answer is good but I’m observing that strictly translating the following Quora’s answer statement &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In reinforcement learning terminology, a trajectory &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the path of the agent through the state space up until the horizon &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;we get &lt;span class=&quot;math-container&quot;&gt;$\tau = \{ s_{t} \}_{t \in [t_{0}, t_{H}]} \quad s_{t} \in \mathcal{S}$&lt;/span&gt; with &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; the State Space &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$t_{0}$&lt;/span&gt; initial time &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$t_{H} &amp;gt; t_{0}$&lt;/span&gt; time associated to a certain event &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So according to Quora's answer author, it should just be a temporal sequence of states (no action and rewards)&lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2018-11-13T15:46:14.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8956" PostTypeId="2" ParentId="8954" CreationDate="2018-11-13T15:55:55.233" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are we going from one image of &lt;span class=&quot;math-container&quot;&gt;$224 \times 224 \times 3$&lt;/span&gt; to two images of &lt;span class=&quot;math-container&quot;&gt;$224 \times 224 \times 64$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They do a Convolution with a &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; kernel, with &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt; filters. That way, you keep your size as the inputs (&lt;span class=&quot;math-container&quot;&gt;$224 \times 224$&lt;/span&gt;), but changes the number of filters to 64.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is not two image, but two layers !&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why does this halving continue? What is this meant to represent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is the maxpooling operation (the layer is in red colour, see the legend). In general, max pooling is applied with a kernel stride of 2 (rarely 3, but not more, because you lose too much informations). That way your size is cut by 2, and the activation maps are more little, and you have faster computation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It continues on to &lt;span class=&quot;math-container&quot;&gt;$56 \times 56 \times 256$&lt;/span&gt;. Why does this number continue to halve, and the number, at the end, the &lt;span class=&quot;math-container&quot;&gt;$256$&lt;/span&gt;, continues to double?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Still max-pooling. There are no reason to double the numbers of filters, though. This just a trend, but you can put the numbers of filters that you want.&lt;/p&gt;&#xA;" OwnerUserId="17221" LastEditorUserId="2444" LastEditDate="2019-04-12T18:52:22.350" LastActivityDate="2019-04-12T18:52:22.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8957" PostTypeId="2" ParentId="7867" CreationDate="2018-11-13T16:21:55.860" Score="0" Body="&lt;p&gt;In the R-CNN family approach, the BBox regression is performed using a specific module providing &lt;strong&gt;region proposal&lt;/strong&gt;: initially the &lt;a href=&quot;https://www.learnopencv.com/selective-search-for-object-detection-cpp-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Selective Search&lt;/a&gt; has been used then &lt;a href=&quot;https://www.quora.com/How-does-the-region-proposal-network-RPN-in-Faster-R-CNN-work&quot; rel=&quot;nofollow noreferrer&quot;&gt;Region Proposal Network&lt;/a&gt; has been introduced, which has the advantage, being a NN, to be trainable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has not to be confused with the BBox Regressor which comes after in the architecture and is focused on refining the proposal bboxes for the “Non Background” proposal objects &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please consider that to perform object detection properly, at least using a region proposal based architecture, you need to perform the basic “Background / Non Background” classification (which is where CNN features help) so to filter out non relevant proposals &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2018-11-13T16:21:55.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8958" PostTypeId="1" CreationDate="2018-11-13T17:22:08.943" Score="-1" ViewCount="51" Body="&lt;h3&gt;(Un-original) idea:&lt;/h3&gt;&#xA;&lt;p&gt;Wouldn't it be cool if we could fact-check using an algorithm that could understand a whole bunch of documents (e.g. scientific papers) as higher-order logic?&lt;/p&gt;&#xA;&lt;h3&gt;Question:&lt;/h3&gt;&#xA;&lt;p&gt;What work has been done on this to date?&lt;/p&gt;&#xA;&lt;h3&gt;What I've got so far:&lt;/h3&gt;&#xA;&lt;p&gt;(1) I seem to recall there being prior work to create a subset of English (I think intended for use in scientific writing) that could be easily interpreted by an algorithm. This doesn't quite get us to the algorithm described above (as it's restricted to a subset of English) - but seems pertinent.&lt;/p&gt;&#xA;&lt;p&gt;(2) Once parsed, I guess a resolution algorithm like that in &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt; could be used to check wether a fact (presumably also inputted as a logical statement) contradicts the logic of the documents?&lt;/p&gt;&#xA;" OwnerUserId="19864" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-11-14T09:29:04.163" Title="Possible to translate generic English-language document into higher-order logic?" Tags="&lt;natural-language-processing&gt;&lt;logic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8960" PostTypeId="1" CreationDate="2018-11-13T19:49:51.370" Score="1" ViewCount="43" Body="&lt;p&gt;As the title says, I want to train a Jordan network (i.e. a particular kind of &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrent neural network&lt;/a&gt;) using a certain number of time series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, \ldots x_N$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; input time series (i.e. &lt;span class=&quot;math-container&quot;&gt;$x_i = [x_{i,1}, x_{i,2}, \ldots, x_{i,T}]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is the length of the time series) and &lt;span class=&quot;math-container&quot;&gt;$y_1, y_2, \ldots y_N$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$x_i = [y_{i,1}, y_{i,2}, \ldots, y_{i,T}]$&lt;/span&gt;) are the corresponding target time series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, the target time series are just sequences of &quot;&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s&quot;, which may end with sequences of &quot;&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;&quot;s. Here I show you some example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_i = [0 ~ 0 ~ 0 \ldots 0 ~ 0 ~ 1 ~ 1 ~ 1 \ldots 1 ~1 ], $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_i = [0 ~ 0 ~ 0 \ldots 0 ~ 0]. $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that I want that my machine &quot;learn to raise&quot; under some situations related to the corresponding inputs &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;. Indeed, the objective of my network is to &quot;raise&quot; an alarm if &quot;something&quot; happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, my training strategy is the following. I create a new time series which corresponds the &lt;strong&gt;concatenation&lt;/strong&gt; of all the available &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;. Let's call the concatenated series &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. Then I use &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; to train a network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my problem. If I concatenate, then I also teach to my machine to &quot;drop&quot;, since I can have situation like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y = [ \ldots 1 ~ 1 ~ 0 ~ 0 \ldots].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this really a problem? Are there other &quot;training strategies&quot; to be employed so that I avoid this kind of unwanted behaviors?&lt;/p&gt;&#xA;" OwnerUserId="19868" LastEditorUserId="19868" LastEditDate="2018-11-13T19:55:34.103" LastActivityDate="2018-11-13T19:55:34.103" Title="Train a recurrent neural network by concatenating time series. Is it safe?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8962" PostTypeId="1" CreationDate="2018-11-13T23:49:24.553" Score="5" ViewCount="982" Body="&lt;p&gt;What is the actual learning algorithm: back-propagation or gradient descent (or, in general, the optimization algorithm)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am reading through chapter 8 of Parallel Distributed Processing hand book and the title of the chapter is &quot;Learning internal representation by error propagation&quot; by PDP research Group. &#xA;&lt;a href=&quot;https://web.stanford.edu/class/psych209a/ReadingsByDate/02_06/PDPVolIChapter8.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://web.stanford.edu/class/psych209a/ReadingsByDate/02_06/PDPVolIChapter8.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are no hidden units, no learning happens.&#xA;If there are hidden units, they learn internal representation by propagating error back.&#xA;Does this mean back propagation[delta rule] is the learning rule and gradient descent is an optimization algorithm used to optimize cost function?&lt;/p&gt;&#xA;" OwnerUserId="18195" LastEditorUserId="18195" LastEditDate="2018-11-17T01:01:56.193" LastActivityDate="2018-11-19T07:35:17.367" Title="What is the actual learning algorithm: back-propagation or gradient descent?" Tags="&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;terminology&gt;&lt;optimization&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8966" PostTypeId="2" ParentId="8962" CreationDate="2018-11-14T06:34:42.887" Score="1" Body="&lt;p&gt;&lt;strong&gt;Gradient descent&lt;/strong&gt; is a very general optimization algorithm. &lt;strong&gt;Backpropagation&lt;/strong&gt; is a special case of auto-differenciation combined with gradient descent. So backpropagation is a clever way to do gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of Gradient descent is to calculate the gradient (derivative) and use that to descent on the error surface. In 3D, you could imagine 2 parameters (x,y) and the 3rd dimension (z) gives you the training error the model makes with those two parameters. You are at some (random) starting point in this error surface and make a step in downwards direction. Then you calculate the error again and do again a step in downwards direction. Note that I didn't write &quot;downwards&quot; but &quot;in downwards direction&quot; as your step might be too big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backpropagation consists of two steps: The forward-pass and the backward-pass. In the forward pass, the output of the network for a given input is calculated. In the backward pass, the gradients are calculated with the chain rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://cs231n.github.io/optimization-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/optimization-2/&lt;/a&gt; for a good reference for back propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that machine learning is way broader than neural networks. For example, decision trees don't use gradient descent at all to train.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the actual learning algorithm: back-propagation or gradient descent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Gradient descent is the learning algorithm. It is realized in backpropagation in the weight-update of the backward pass.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Notes and References&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When I write &quot;backpropagation algorithm&quot;, I refer to Tom Mitchells &quot;Machine Learning&quot;, Chapter 4.5.2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To cite Tom Mitchells &quot;Machine Learning&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Algorithms such as BACKPROPAGATION use gradient descent to tune network parameters to best fit a training set of input-output pairs.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[...] gradient descent provides the basis&#xA;  for the BACKPROPAGATION algorithm [...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[The section 4.5 MULTILAYER NETWORKS AND THE BACKPROPAGATION&#xA;  ALGORITHM] discusses how to learn such multilayer networks using a gradient descent algorithm similar to that discussed in the previous section. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;As shown above, the BACKPROPAGATION algorithm implements a gradient descent search through the space of possible network weights&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-15T06:14:35.640" LastActivityDate="2018-11-15T06:14:35.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8967" PostTypeId="2" ParentId="6784" CreationDate="2018-11-14T07:06:35.553" Score="1" Body="&lt;p&gt;Fundraising is probably the most realistic answer with the biggest effect. Similar to simply &lt;a href=&quot;https://www.google.de/amp/s/www.bloomberg.com/amp/news/articles/2018-01-22/blockchain-name-changes-are-still-proving-fodder-for-stock-rally&quot; rel=&quot;nofollow noreferrer&quot;&gt;adding blockchain to the company name helps&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the question of real application, I should first say that I'm a machine learning engineer who works as a consultant. I had mathematics as a minor subject during bachelor's and Master's degree, which is like 80% of a bachelor's degree in mathematics. Mainly the bachelor's thesis is missing. I also gave private mathematics lessons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would distinguish learning mathematics at school from University. It's completely different. However, for this question it does not make much of a difference. It's also not super important that you ask it for math. It's about learning in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning tools can support for several minor tasks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Figuring out how a symbol is called (I build write-math.com for that as a bachelor's project)&lt;/li&gt;&#xA;&lt;li&gt;Figuring out what a symbol means&lt;/li&gt;&#xA;&lt;li&gt;Getting relevant papers / parts of the lecture notes for a given problem&lt;/li&gt;&#xA;&lt;li&gt;Helping to digitalize stuff for easier sharing / content creation&lt;/li&gt;&#xA;&lt;li&gt;Learning things by heart: there are some nice apps like Anki which help you to learn arbitrary stuff by heart. The learning algorithm could potentially be improved by AI&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-14T07:06:35.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8968" PostTypeId="2" ParentId="8940" CreationDate="2018-11-14T07:45:23.417" Score="1" Body="&lt;p&gt;This is a start-up company, not a software. There are many clients cannot find right freelancers in a short time, sometimes they take more than a month to find the right freelancer.&#xA;They use artificial intelligence to help businesses find the ones they expect. Tara AI's software helps businesses automatically scope out and manage projects with freelancers, monitoring the pace and quality of their work, and making sure contractors are paid on-time for their code commits, for example.&#xA;There are now a lot of companies that use AI technology for the same purpose, for example, &lt;a href=&quot;https://www.forbes.com/sites/blakemorgan/2018/07/16/how-amazon-has-re-organized-around-artificial-intelligence-and-machine-learning/#456cf9097361&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amazon&lt;/a&gt;, &lt;a href=&quot;https://www.forbes.com/sites/bernardmarr/2016/12/29/4-amazing-ways-facebook-uses-deep-learning-to-learn-everything-about-you/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facebook&lt;/a&gt;, &lt;a href=&quot;https://getdone.co/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Getdone&lt;/a&gt; or &lt;a href=&quot;https://www.forbes.com/sites/bernardmarr/2017/08/08/the-amazing-ways-how-google-uses-deep-learning-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19328" LastActivityDate="2018-11-14T07:45:23.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8969" PostTypeId="2" ParentId="8962" CreationDate="2018-11-14T08:14:23.530" Score="7" Body="&lt;p&gt;You can run gradient descent without back propagation, in some cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Simple structures such as linear or logistic regression, where the gradients can be calculated directly from the inputs and cost function value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In &quot;black box&quot; gradient-based learning algorithms where you don't know how (or don't want to) calculate gradient analytically, so you choose to &lt;em&gt;measure&lt;/em&gt; it. This is a slow process, similar to &lt;a href=&quot;http://deeplearning.stanford.edu/tutorial/supervised/DebuggingGradientChecking/&quot; rel=&quot;noreferrer&quot;&gt;gradient checking&lt;/a&gt; of neural networks, i.e. adjust each parameter in turn by a small &lt;span class=&quot;math-container&quot;&gt;$+\epsilon$&lt;/span&gt;, measure your loss function, and divide difference by &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; to get gradient estimate for that parameter.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In those cases, you can calculate the gradient, and then use the optimisation function of your choice - e.g. SGD, Adam, Nesterov Momentum - to make changes to the learned parameters. The model will be changed and should learn its task based on the cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also run back propagation without gradient descent. In that case, the model will not change and nothing is learned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That makes the different flavours of gradient descent technically the &quot;core&quot; learning algorithm in my opinion, and back propagation the preferred method for calculating gradients in neural networks. These gradients are most often estimates of an unknown &quot;true gradient&quot; due to training in batches of small samples of training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the two terms back propagation and gradient descent are rarely separated when discussing neural network training. So a lot of people will say that they train a network &quot;using back propagation&quot; or that they are using a gradient descent method without mentioning how they are calculating the gradients using back propagation (because that's pretty much a given). Or they might just say they are &quot;using the Adam optimiser&quot; which in the library they are using automatically runs an NN forward on a mini-batch of data, performs back propagation*, and then changes weight parameters based on the gradients, but heavily adjusted by previous gradient measurements.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* With auto-differentiating frameworks such as TensorFlow, PyTorch etc, it is not really clear that the algorithms are performing &quot;classic back propagation&quot; - although they work using the same chain rule, they may well not perform calculations in the exact same groups that you learned when you studied neural networks and had to write the code for calculating gradients by hand. Instead of back propagating gradients layer-by-layer, they are propagated through an abstract computation graph that includes the learnable parameters.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-14T08:32:26.597" LastActivityDate="2018-11-14T08:32:26.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8970" PostTypeId="2" ParentId="8958" CreationDate="2018-11-14T09:29:04.163" Score="0" Body="&lt;p&gt;There are several problems with this, which is why people have been working on tasks like that for about 50 years without getting very far. As you rightly notice, it has been tried in restricted domains, where it works reasonably well. Reason being, there is less ambiguity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human language is full of vagueness and ambiguity. We generally have few problems handling that, but we have a wealth of experience available to interpret utterances, and a quite sophisticated 'natural intelligence' to exclude non-sensical interpretations. Computer programs struggle with this. Even on a syntactic level there are many way to interpret a sentence, most of which a human being wouldn't even notice. A famous example sentence in linguistics is &lt;em&gt;Time flies like an arrow.&lt;/em&gt;, which has four different interpretations to a computer (and a trained linguist), while most human beings would only see one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And after the vagueness/ambiguity you have the problem of &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;representing meaning&lt;/a&gt;. There is a whole field in AI dealing with that. You will find that the expressive power of higher order logic falls short of the expressive power of human language, so there is an issue there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you have the analysis. Prolog, by the way, is a programming language, not a resolution algorithm. How do you decide whether something follows from something else? You would need to understand a whole lot of contextual information to say that &lt;em&gt;It has rained, so the streets are wet.&lt;/em&gt; is a logical sentence. And now imagine doing this for scientific papers that are about new discoveries, so you are dealing with unknown phenomena your software hasn't encountered yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot answer your question &quot;what work has there been done so far&quot;, because there is too much to list. The whole field of natural language processing is concerned among others with the problem of parsing/syntactic/semantic analysis, and there are literally decades of academic work you would need to look at. Same for knowledge representation.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-11-14T09:29:04.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8971" PostTypeId="1" CreationDate="2018-11-14T11:47:31.657" Score="3" ViewCount="208" Body="&lt;p&gt;I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from.&lt;/p&gt;&#xA;&lt;p&gt;Ignoring HER for now, if in the case where the robotic arm reaches the goal correctly, then the value functions (&lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) and action-value functions (&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;) that correspond to the trajectories that get to the goal quicker will increase. These high &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; values are ultimately important for getting the optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;However, if you create 'fake' goals from unsuccessful trajectories - that would increase the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;s of the environment that lead to getting the 'fake' goal. Those new &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;s would be unhelpful and possibly detrimental for the robotic arm to reach the real goal.&lt;/p&gt;&#xA;&lt;p&gt;What am I misunderstanding?&lt;/p&gt;&#xA;" OwnerUserId="19895" LastEditorUserId="2444" LastEditDate="2020-11-20T19:18:57.667" LastActivityDate="2020-11-27T15:54:58.027" Title="How does Hindsight Experience Replay learn from unsuccessful trajectories?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;hindsight-experience-replay&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8974" PostTypeId="1" CreationDate="2018-11-14T21:10:49.883" Score="2" ViewCount="80" Body="&lt;p&gt;I am working on the blockchain technology and I am not very familiar with the AI concept. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proposal of this web page: (&lt;a href=&quot;http://www.euraxess.lu/jobs/349354&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.euraxess.lu/jobs/349354&lt;/a&gt;) opens a discussion about use of &lt;strong&gt;nature inspired artificial intelligent methods for traceability chain decision in blockchain&lt;/strong&gt; technology as an &lt;strong&gt;alternative to current consensus&lt;/strong&gt; mechanisms. It continues as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;&quot;&lt;strong&gt;It has been demonstrated&lt;/strong&gt; that using &lt;strong&gt;traceability chain&lt;/strong&gt; is a more&#xA;  effective method. In traceability chain, since the mechanism has to&#xA;  trace related information among participant’s nodes across the entire&#xA;  chain, the extraction and recognition of the data features plays a&#xA;  crucial role in improving the efficiency of the process.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, it does not give any example to demonstrate &lt;strong&gt;an instance&lt;/strong&gt; of this approach. So, I searched in google.scholar and any ordinary web pages to find only an instance similar to this approach since it has mentioned: &quot;&lt;strong&gt;&lt;em&gt;It has been demonstrated&lt;/strong&gt; that using &lt;strong&gt;traceability chain&lt;/strong&gt; is a more effective method.&lt;/em&gt;&quot; (Please read the web page)   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is someone here familiar with this approach? And if yes, Is there any article/example to explain a more about this approach of consensus in blockchain? And what does exactly mean &quot;traceability chain&quot;? And also ingeneral, can we call this approach as a consensus? The text is not really clear to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Please not that I have no idea about this proposal and just I'd like to know if it's practicable? or it's buzzwords?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Also, may this approach related to the approach that has been mentioned in this answer (Swarm Intelligence):&lt;/strong&gt; &lt;a href=&quot;https://ai.stackexchange.com/a/1315/19910&quot;&gt;https://ai.stackexchange.com/a/1315/19910&lt;/a&gt; &lt;strong&gt;Or it is a different concept?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help&lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="19910" LastEditDate="2018-11-15T10:08:47.340" LastActivityDate="2018-11-15T10:08:47.340" Title="Nature-inspired artificial intelligent methods for Blockchain?" Tags="&lt;algorithm&gt;&lt;swarm-intelligence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8976" PostTypeId="1" CreationDate="2018-11-15T00:34:44.840" Score="3" ViewCount="52" Body="&lt;p&gt;I have three equations that relates five variables &lt;em&gt;{a, b, c, r, s}&lt;/em&gt; with a sum and two ratios.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;Eq. 1: a = b + c;&#xA;Eq. 2: s = b / a;&#xA;Eq. 3: r = b / c.&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given two values for any of the five variables I get a solution. But, this is not the automation problem I want to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can have the solution of variable &lt;em&gt;r&lt;/em&gt; by simply knowing &lt;em&gt;s&lt;/em&gt;. This is solved by a &quot;human algorithm&quot; as follows.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Substitute a of Eq. 1 in Eq. 2.&#xA;&lt;li&gt;Divide the second term of the new Eq. 2 by the variable c.&#xA;&lt;li&gt;Replace b/c by the expression of Eq. 3.&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That means s = r / (r+1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The questions is -How can an AI algorithm solve this?, i.e. the machine should recognize that given the variable &lt;em&gt;r&lt;/em&gt; she can obtain directly the variable &lt;em&gt;s&lt;/em&gt; and do no require another variable.&lt;/p&gt;&#xA;" OwnerUserId="19913" LastEditorUserId="32410" LastEditDate="2021-04-18T08:30:33.000" LastActivityDate="2021-04-18T08:30:33.000" Title="Machine mathematical reasoning by clever substitutions, How to do with AI" Tags="&lt;math&gt;&lt;reasoning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8978" PostTypeId="1" CreationDate="2018-11-15T03:46:46.673" Score="0" ViewCount="57" Body="&lt;p&gt;If the number of input neurons and output neurons doesn't change, what will change if I have one hidden layer, but first with 1 neuron, then with 4 neurons? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking into consideration the fact that each perceptron is able to linearly separate points on an unknown/unwritten linear function, would this then be able to, theoretically, instead of simply linearly separate points, separate points into those that occur inside a square, and those that occur outside?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, of course, without a bias neuron present.&lt;/p&gt;&#xA;" OwnerUserId="19917" LastEditorUserId="1671" LastEditDate="2019-10-15T19:09:13.940" LastActivityDate="2019-10-15T19:09:13.940" Title="Basic Functions and Results" Tags="&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8979" PostTypeId="2" ParentId="8978" CreationDate="2018-11-15T06:43:58.500" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If the number of input neurons and output neurons doesn't change, what will change if I have one hidden layer, but first with 1 neuron, then with 4 neurons?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I recommend that you try to implement a dense layer with Tensorflow (not using the Dense implementation, of course).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A dense layer is the full connection between to layers. What actually is learned/optimized in a neural network are the weights in between layers. Each weight is just a single floating point number. It's the connection between two nodes. If you increase the number of nodes of one layer, the two neighboring layers get more weights. Hence the optimization algorithm can adapt better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, I'll now assume you don't use biases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have A input neurons and Z output neurons, a single hidden layer with 1 neutron means you have two matrices of shape Ax1 and 1xZ. if you change it to 4 hidden neurons the two matrices have the shape Ax4 and 4xZ&lt;/p&gt;&#xA;" OwnerUserId="3217" LastActivityDate="2018-11-15T06:43:58.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8980" PostTypeId="2" ParentId="8976" CreationDate="2018-11-15T06:56:07.867" Score="2" Body="&lt;p&gt;What you're asking here is actually not reasoning but rather symbolic computation. &lt;a href=&quot;https://www.sympy.org/en/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt; Sympy&lt;/a&gt; is an example of a library which does it.&lt;/p&gt;&#xA;&lt;p&gt;Explaining computer algebra systems could be a series of lectures and this is too broad for a single question. To give you an idea: (1) represent equations symbolically (2) bring them in a normal form (3) substitute.&lt;/p&gt;&#xA;&lt;p&gt;Although it is not elegant, you can simplify the problem by brute Force: try all combinations of solving for a variable and substituting it in the other equations.&lt;/p&gt;&#xA;&lt;p&gt;Related:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://math.stackexchange.com/q/532897/6876&quot;&gt;How do symbolic math software work? | Mathematics Stack Exchange&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="32410" LastEditDate="2021-04-17T21:54:15.017" LastActivityDate="2021-04-17T21:54:15.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8983" PostTypeId="1" AcceptedAnswerId="8993" CreationDate="2018-11-15T12:58:17.873" Score="3" ViewCount="3492" Body="&lt;p&gt;In hill climbing methods, at each step, the current solution is replaced with the best neighbour (that is, the neighbour with highest/smallest value). In simulated annealing, &quot;downhills&quot; moves are allowed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the advantages of simulated annealing with respect to hill climbing approaches? How is simulated annealing better than hill climbing methods?&lt;/p&gt;&#xA;" OwnerUserId="19652" LastEditorUserId="2444" LastEditDate="2019-03-02T10:59:40.133" LastActivityDate="2019-03-02T10:59:40.133" Title="How is simulated annealing better than hill climbing methods?" Tags="&lt;search&gt;&lt;comparison&gt;&lt;hill-climbing&gt;&lt;simulated-annealing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8985" PostTypeId="2" ParentId="8962" CreationDate="2018-11-15T13:51:51.137" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Gradient descent&lt;/strong&gt;&lt;/a&gt; (GD) is an optimisation algorithm, that is, it is used to find a (local) minimum of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Function_of_several_real_variables&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-variable&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Differentiable_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;differentiable&lt;/a&gt; function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. GD is an &lt;em&gt;iterative&lt;/em&gt; and &lt;em&gt;numerical&lt;/em&gt; optimisation algorithm. It is &lt;em&gt;iterative&lt;/em&gt; because it proceeds in iterations. It is &lt;em&gt;numerical&lt;/em&gt; because it is not an algorithm which produces an exact solution, due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical errors&lt;/a&gt; (e.g., round-off errors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GD is based on the idea that, if a (multi-variable) function &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x})$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \in \mathbb{R}^N$&lt;/span&gt; is a variable representing the input of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;) is &lt;em&gt;defined&lt;/em&gt; and &lt;em&gt;differentiable&lt;/em&gt; in a &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neighbourhood_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;neighbourhood&lt;/a&gt;&quot; of a point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta} \in \mathbb{R}^N$&lt;/span&gt;, which is a point (in the loose sense of the word &quot;point&quot;) in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain_of_a_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;domain&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (that is, &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{\theta})$&lt;/span&gt; is defined and differentiable), then &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; &lt;strong&gt;decreases fastest&lt;/strong&gt; if one goes from &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; in the &quot;direction&quot; of the &lt;strong&gt;negative gradient&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$-\nabla f(\mathbf {\theta})$&lt;/span&gt;. (Do not be scared of the symbol &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, we could have denoted it by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}$&lt;/span&gt;!). Let us break this dense definition down!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that here we assume that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a multi-variable function (that is, a function whose input is not just one real number, but it is a function of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; real numbers, which can be grouped in a vector, e.g. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta} \in \mathbb{R}^N$&lt;/span&gt;), hence &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; are vectors of real numbers. To emphasize,  &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; is some specific (and &quot;fixed&quot;) point in the domain of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, whereas &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; is a variable that can take any of these specific points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic statement/assignment which is iterated in GD, which captures the definition above, is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{\theta}_{n+1} \gets \mathbf{\theta}_{n} - \gamma \nabla f(\mathbf{\theta} _{n}) \label{gd-assignment}\tag{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyper-parameter&lt;/a&gt; which represents the &quot;strength&quot; of the step we take in the direction of the negative gradient (i.e., &lt;span class=&quot;math-container&quot;&gt;$-\nabla f(\mathbf{\theta} _{n})$&lt;/span&gt;). Note the &quot;minus&quot; (i.e., &lt;span class=&quot;math-container&quot;&gt;$-$&lt;/span&gt;) in front of &lt;span class=&quot;math-container&quot;&gt;$\nabla f(\mathbf{\theta}_{n})$&lt;/span&gt;, hence the expression &quot;in the direction of the &lt;em&gt;negative&lt;/em&gt; gradient&quot;. Note also that here &quot;direction&quot; is implemented as a subtraction. The subscript &lt;span class=&quot;math-container&quot;&gt;$_n$&lt;/span&gt; represents the current iteration number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As simple as it looks, the GD algorithm consists in the execution of the assignment \ref{gd-assignment} until some condition is met (e.g., &lt;span class=&quot;math-container&quot;&gt;${\theta}_{n+1}$&lt;/span&gt; is &quot;close enough&quot; to &lt;span class=&quot;math-container&quot;&gt;${\theta}_{n}$&lt;/span&gt;). In a nutshell, GD is an iterative algorithm which updates &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; by iterating \ref{gd-assignment} multiple times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of machine learning (ML), in particular supervised learning, we initially have an untrained model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; (e.g., a neural network), that is, a model whose parameters (sometimes also called &quot;weights&quot;) are e.g. randomly initialised. To train &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; (i.e., to find the most appropriate parameters of the model with respect to the training data), we first define a so-called &quot;loss function&quot; (which is differentiable), denoted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;, which represents the discrepancy between the current output of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; and the expected output (with respect to the training data) of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;. The process of &lt;em&gt;training a model&lt;/em&gt; consists in minimising this loss function, that is, it consists in finding the minimum of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, as we have just seen above, we can use GD. If you consider the learning process the one of minimising this loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;, then any algorithm which minimises &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; should be called a &quot;learning algorithm&quot;. Hence GD should be called a &quot;learning algorithm&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the parameters of the model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;, which need to be found during the training process, are often denoted, in ML, by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. So, the reason why I used the strange symbol &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in the assignment &lt;span class=&quot;math-container&quot;&gt;$\ref{gd-assignment}$&lt;/span&gt; should now be clearer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Back-propagation&lt;/strong&gt;&lt;/a&gt; (BP) is just an algorithm, &lt;a href=&quot;http://people.idsia.ch/~juergen/who-invented-backpropagation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;proposed by Seppo Linnainmaa in his master's thesis&lt;/a&gt;, to compute the derivative of a &lt;em&gt;differentiable&lt;/em&gt; (composite) function, which can be represented as a graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In ML, back-propagation is often used to compute &lt;span class=&quot;math-container&quot;&gt;$\nabla f(\mathbf{\theta}_{n})$&lt;/span&gt; in the assignment \ref{gd-assignment}. So, BP is used as a sub-routine in the GD algorithm (or any other optimization algorithm) to find the parameters of the model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, in the context of ML, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; in \ref{gd-assignment} is often called the &quot;&lt;em&gt;learning&lt;/em&gt; rate&quot;. There's a reason for this: GD is the actual learning algorithm. BP is just a sub-routine used to compute the gradient, but we could have used another sub-routine. We could say, in the context of ML (and, in particular, NNs), that the combination of GD and BP is the actual &quot;learning algorithm&quot;, but, if by &quot;learning&quot; we mean &quot;optimisation&quot;, then the actual &quot;learning algorithm&quot; is GD and &lt;strong&gt;not&lt;/strong&gt; BP, because BP does &#xA;&lt;em&gt;not&lt;/em&gt; optimise anything alone.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2018-11-15T13:51:51.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8986" PostTypeId="1" AcceptedAnswerId="8991" CreationDate="2018-11-15T15:03:08.543" Score="9" ViewCount="13014" Body="&lt;p&gt;What are the limitations of the &lt;em&gt;hill climbing algorithm&lt;/em&gt;? How can we overcome these limitations?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2019-02-25T21:32:00.887" LastActivityDate="2019-02-25T21:32:00.887" Title="What are the limitations of the hill climbing algorithm and how to overcome them?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;optimization&gt;&lt;problem-solving&gt;&lt;hill-climbing&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8989" PostTypeId="2" ParentId="8986" CreationDate="2018-11-15T15:33:43.450" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/aimacode/aima-pseudocode/blob/master/md/Hill-Climbing.md&quot; rel=&quot;noreferrer&quot;&gt;Hill climbing&lt;/a&gt; is not an algorithm, but a family of &quot;local search&quot; algorithms. Specific algorithms which fall into the category of &quot;hill climbing&quot; algorithms are 2-opt, 3-opt, 2.5-opt, 4-opt, or, in general, any N-opt. See chapter 3 of the paper &quot;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.1635&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;The Traveling Salesman Problem: A Case Study in Local Optimization&lt;/em&gt;&lt;/a&gt;&quot; (by David S. Johnson and Lyle A. McGeoch) for more details regarding some of these local search algorithms (applied to the TSP). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What differentiates one algorithm in this category from the other is the &quot;neighbourhood function&quot; they use (in simple words, the way they find neighbouring solutions to a given solution). Note that, in practice, this is not always the case: often these algorithms have several different implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most evident limitation of hill climbing algorithms is due to their nature, that is, they are local search algorithms. Hence they usually just find &lt;em&gt;local&lt;/em&gt; maxima (or minima). So, if any of these algorithms has already converged to a local minimum (or maximum) and, in the solution or search space, there is, close to this found solution, a better solution, none of these algorithms will be able to find this better solution. They will basically be trapped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Local search algorithms are not usually used alone. They are used as sub-routines of other meta-heuristic algorithms, like simulated annealing, iterated-local search or in any of the ant-colony algorithms. So, to overcome their limitations, we usually do not use them alone, but we use them in conjunction with other algorithms, which have a probabilistic nature and can find global minima or maxima (e.g., any of the ant-colony algorithms).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-15T15:39:24.283" LastActivityDate="2018-11-15T15:39:24.283" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8990" PostTypeId="1" AcceptedAnswerId="8995" CreationDate="2018-11-15T16:27:40.773" Score="2" ViewCount="61" Body="&lt;p&gt;I am not sure the name of this kind of problem, but anyway, the situation is as below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assign teachers into Groups and consider on each of their workload, availability etc.&#xA;There are some other soft/hard constraint (equality/inequality) like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each group should have at least 2 teachers&lt;/li&gt;&#xA;&lt;li&gt;Everyone in the group have similar workload&lt;/li&gt;&#xA;&lt;li&gt;Total workload in the group is below a certain value&lt;/li&gt;&#xA;&lt;li&gt;All are in different expertise&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and more...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to build a sub-optimal solution to solve this problem. Linear/non-linear programming seems not working for grouping problems. I am thinking of genetic algorithm or reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this problem solve by using RL or DRL?&#xA;I am trying to define the groups as state, and actions include &quot;assignToGroup&quot; and &quot;removeFromGroup&quot;.&#xA;And any kind of idea or suggestion of how to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="19930" LastEditorUserId="19930" LastEditDate="2018-11-15T17:30:40.980" LastActivityDate="2018-11-15T20:49:19.087" Title="Reinforcement Learning to Grouped Scheduling Optimisation Problem" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8991" PostTypeId="2" ParentId="8986" CreationDate="2018-11-15T16:28:18.853" Score="5" Body="&lt;p&gt;As @nbro has already said that Hill Climbing is a family of &lt;strong&gt;local search algorithms&lt;/strong&gt;. So, when you said Hill Climbing in the question I have assumed you are talking about the standard hill climbing. The standard version of hill climb has some limitations and often gets stuck in the following scenario:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Local Maxima: Hill-climbing algorithm reaching on the vicinity a local maximum value, gets drawn towards the peak and gets stuck there, &lt;em&gt;having no other place to go&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Ridges: These are &lt;em&gt;sequences of local maxima&lt;/em&gt;, making it difficult for the algorithm to navigate.&lt;/li&gt;&#xA;&lt;li&gt;Plateaux: This is a &lt;em&gt;flat state-space region&lt;/em&gt;. As there is no uphill to go, algorithm often gets lost in the plateau.&#xA;&lt;br&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To resolve these issues many variants of hill climb algorithms have been developed. These are most commonly used:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Stochastic Hill Climbing&lt;/strong&gt; selects at random from the uphill moves. The probability of selection varies with the steepness of the uphill move.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;First-Choice Climbing&lt;/strong&gt; implements the above one by generating successors randomly until a better one is found.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Random-restart hill climbing&lt;/strong&gt; searches from randomly generated initial moves until the goal state is reached.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The success of hill climb algorithms depends on the architecture of the state-space landscape. Whenever there are few maxima and plateaux the variants of hill climb searching algorithms work very fine. But in real-world problems have a landscape that looks more like a widely scattered family of balding porcupines on a flat floor, with miniature porcupines living on the tip of each porcupine needle (as described in the 4th Chapter of the book Artificial Intelligence: A Modern Approach). NP-Hard problems typically have an exponential number of local maxima to get stuck on.&lt;br&gt;&#xA;Given algorithms have been developed to overcome these kinds of issues:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stimulated Annealing&lt;/li&gt;&#xA;&lt;li&gt;Local Beam Search&lt;/li&gt;&#xA;&lt;li&gt;Genetic Algorithms&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Reference &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0136042597&quot; rel=&quot;noreferrer&quot;&gt;Book - Artificial Intelligence: A Modern Approach&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2018-11-15T16:28:18.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8993" PostTypeId="2" ParentId="8983" CreationDate="2018-11-15T19:28:01.647" Score="2" Body="&lt;p&gt;In the least technical, most intuitive way possible: Simulated Annealing can be considered as a modification of Hill Climbing (or Hill Descent). Hill Climbing/Descent attempts to reach an optimum value by checking if its current state has the best cost/score in its neighborhood, this makes it prone to getting stuck in local optima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulated Annealing attempts to overcome this problem by choosing a &quot;bad&quot; move every once in a while. The probability of choosing of a &quot;bad&quot; move decreases as time moves on, and eventually, Simulated Annealing becomes Hill Climbing/Descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If configured correctly, and under certain conditions, Simulated Annealing can guarantee finding the global optimum, whereas such a guarantee is available to Hill Climbing/Descent iff the all local optima in the search space have equal scores/costs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more, go through Wolfram Mathworld's entry &lt;a href=&quot;http://mathworld.wolfram.com/SimulatedAnnealing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19932" LastActivityDate="2018-11-15T19:28:01.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8995" PostTypeId="2" ParentId="8990" CreationDate="2018-11-15T20:49:19.087" Score="1" Body="&lt;p&gt;It is often possible, by describing certain parts of a problem as state, others as actions, and some measurement you want to optimise as a reward, to frame a problem as a reinforcement learning task. However, this is not always helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect your problem is in the &quot;RL is not that helpful&quot; category. You could give it a try, and maybe expect to find working solutions but take an unreasonable amount of time to find optimal solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you appear to have here is a problem in &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial optimisation&lt;/a&gt;. There are many applicable algorithms, a good selection are described in the free book &lt;a href=&quot;http://www.cleveralgorithms.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clever Algorithms: Nature-Inspired Programming Recipes&lt;/a&gt; - which is best will be determined by the specific details of your constraints and the size of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If many constraints are soft and can be assigned a cost, you may be able to use genetic algorithms (with genome being group assignments in order) or simulated annealing for example.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-11-15T20:49:19.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8996" PostTypeId="1" CreationDate="2018-11-15T21:34:42.037" Score="1" ViewCount="140" Body="&lt;p&gt;Is there neural machine translation methods, that for one input sentence outputs multiple alternative output sentences in that target language. It is quite possible, that sentence in source language have multiple meanings and it is not desirable that neural network discards some of the meanings if there is no context for disambiguation provided. How multiple outputs can be acommodated into encode-decoder architecture, or different architecture is required?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of only one work &lt;a href=&quot;https://arxiv.org/abs/1805.10844&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1805.10844&lt;/a&gt; (and one referen herein) but I am still digesting whether their network outputs multiple sentences or whether it just acommodates variations during training phase.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2018-11-15T21:46:50.920" LastActivityDate="2018-11-15T22:06:03.700" Title="Neural machine translation, that outputs multiple alternative, ambiguous translations?" Tags="&lt;neural-networks&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8997" PostTypeId="2" ParentId="8996" CreationDate="2018-11-15T21:52:34.753" Score="1" Body="&lt;p&gt;The LSTM seq2seq model typically used for language translation actually can output multiple variations, and does in many implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The decoder stage outputs confidence/probabilities for each word, and it is possible to run the decoder multiple times, taking different samples on each run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not only is it possible, but this is actually done by translation engines. A technique called &lt;a href=&quot;https://machinelearningmastery.com/beam-search-decoder-natural-language-processing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEAM search&lt;/a&gt; is used to manage multiple translations in progress and pick an eventual one to display, based on whether the language model predicts an overall high or low probability for a sentence (this allows for sentences that start with an unusual initial word choice but that are a better choice in context of what is being translated). It would be a minor change to existing algorithms to display a selection of these instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically you don't need to sample randomly, and can just take the top N probabilities on each step, and recalculate each time on combined probabilities, culling partial sentences that have low probability in favour of continuing more of the high probability sentences. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-15T22:06:03.700" LastActivityDate="2018-11-15T22:06:03.700" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8999" PostTypeId="1" CreationDate="2018-11-16T11:47:11.507" Score="2" ViewCount="223" Body="&lt;p&gt;There are 4 kinds of adverbs :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Adverbs of Manner. For example, slowly, quietly&lt;/li&gt;&#xA;&lt;li&gt;Adverbs of Place. For example, there, far&lt;/li&gt;&#xA;&lt;li&gt;Adverbs of Frequency. For example, everyday, often&lt;/li&gt;&#xA;&lt;li&gt;Adverbs of Time. For example, now, first, early&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;nltk, spacy and textblob only tag a token as an adverb without specifying which kind it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any libraries which tag including the type of adverb?&lt;/p&gt;&#xA;" OwnerUserId="19962" LastEditorUserId="1847" LastEditDate="2018-11-16T12:16:36.087" LastActivityDate="2018-11-16T15:09:19.593" Title="How to know which kind of adverb in NLP Parts of Speech (POS) tagging?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2021-01-23T04:34:01.047" ContentLicense="CC BY-SA 4.0" />
  <row Id="9001" PostTypeId="1" AcceptedAnswerId="9003" CreationDate="2018-11-16T13:02:07.620" Score="2" ViewCount="127" Body="&lt;p&gt;I am reading the book titled &quot;&lt;a href=&quot;https://drive.google.com/file/d/1opPSz5AZ_kVa1uWOdOiveNiBFiEOHjkG/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;&quot; (by Sutton and Barto). I am at chapter 5, which is about Monte Carlo methods, but now I am quite confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one thing I don't particularly understand. Why do we need the state-transition probability function when calculating the importance sampling ratio for off-policy prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understood that one of the main benefits of MC over Dynamic Programming (DP) is that one does not need to have a model of the state-transition probability for a system. Or is this only the case for on-policy MC?&lt;/p&gt;&#xA;" OwnerUserId="18266" LastEditorUserId="2444" LastEditDate="2019-02-25T15:27:10.067" LastActivityDate="2019-02-25T15:30:36.080" Title="Do we need the transition probability function when calculating the importance sampling ratio?" Tags="&lt;reinforcement-learning&gt;&lt;on-policy-methods&gt;&lt;monte-carlo-methods&gt;&lt;importance-sampling&gt;&lt;dynamic-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9003" PostTypeId="2" ParentId="9001" CreationDate="2018-11-16T13:57:49.687" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;There is one thing I don't particularly understand. Why do we need the state-transition probability function when calculating the importance sampling ratio for off-policy prediction?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is not needed for calculation. It must be included in the theory, to compare the correct probability of each trajectory (on-policy vs off-policy). However, the state transition probabilities are then also shown to cancel out exactly, so there is no requirement to &lt;em&gt;know&lt;/em&gt; what the values are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;State transition probabilities are irrelevant to probability ratios between identical trajectories where the policy varies but the environment does not. Which is the case for off-policy learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all explained on the relevant pages of the book, but replicating here for completeness. If your trajectory is from steps &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and called &lt;span class=&quot;math-container&quot;&gt;$\tau = (s_m, a_m, r_{m+1}, s_{m+1}, a_{m+1}, r_{m+2} . . . r_{n}, s_n)$&lt;/span&gt;, then the probability of seeing that trajectory under two different policies &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b(a|s)$&lt;/span&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\tau|\pi) = (\prod_{i=m}^{n-1} \pi(a_i|s_i))(\prod_{j=m+1}^{n} p(s_j, &#xA;r_j|s_{j-1},a_{j-1}))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\tau|b) = (\prod_{i=m}^{n-1} b(a_i|s_i))(\prod_{j=m+1}^{n} p(s_j, r_j|s_{j-1},a_{j-1}))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can clearly see that the second product in both cases is the same, and cancels out in the ratio:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{p(\tau|\pi)}{p(\tau|b)} =\frac{\prod_{i=m}^{n-1} \pi(a_i|s_i)}{\prod_{i=m}^{n-1} b(a_i|s_i)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it doesn't matter what &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; actually &lt;em&gt;is&lt;/em&gt;, just that it exists and is not allowed to change in theory between the on-policy and off-policy cases. Or in other words, that the environment is a consistent MDP.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-02-25T15:30:36.080" LastActivityDate="2019-02-25T15:30:36.080" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9004" PostTypeId="2" ParentId="8999" CreationDate="2018-11-16T15:09:19.593" Score="2" Body="&lt;p&gt;Generally POS taggers only use broad categories, especially if they work statistically: the more fine-grained the tagset is, the more training data you need. And unless there are differences in the distribution, the accuracy of the tagger would not be improved by adding more categories, eg &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;I ran&lt;/em&gt; [&lt;em&gt;there&lt;/em&gt;/&lt;em&gt;fast&lt;/em&gt;/&lt;em&gt;often&lt;/em&gt;/&lt;em&gt;early&lt;/em&gt;].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;all work -- so nothing is gained by having subcategories of adverbs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest the easiest solution for this would be to have a list of adverbs by category, and simply look them up &lt;em&gt;after&lt;/em&gt; the tagging. Just pick all words tagged as adverbs and check if they are in your list. Unless there are ambiguities (ie some adverbs can belong to multiple categories) it should be very straight forward, and would also work with any tagger/tagset that marks adverbials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might have to consult a grammar book to get a list of adverbial classes, but that again should be easy to do.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-11-16T15:09:19.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9005" PostTypeId="1" AcceptedAnswerId="9009" CreationDate="2018-11-16T15:40:10.037" Score="3" ViewCount="8023" Body="&lt;p&gt;Local search algorithms are useful for solving pure optimization problems, in which the aim is to find the best state according to an objective&#xA;function.&lt;/p&gt;&#xA;&lt;p&gt;My question is what is the objective function?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/y459H.jpg&quot; alt=&quot;Hill Climbing&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2021-01-23T02:17:04.717" LastActivityDate="2021-01-23T02:17:04.717" Title="What is an objective function?" Tags="&lt;terminology&gt;&lt;loss-functions&gt;&lt;optimization&gt;&lt;local-search&gt;&lt;meta-heuristics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9007" PostTypeId="1" AcceptedAnswerId="9223" CreationDate="2018-11-16T17:34:30.133" Score="3" ViewCount="115" Body="&lt;p&gt;I am building a generative model chatbot as a research and learning project. One of the most important parts of my project is to research ways in which I can make this chatbot work in a consistently ethical fashion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This chatbot is simply a single Seq2Seq network running on my local machine. It can't be interacted with over the internet (yet), although I may end up creating a way to do that. It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that there would be some sort of unchanging knowledgebase for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change. Whenever a question is asked of the chatbot,before being inputted to the network, the knowledgebase is searched and relevant facts will be appended to the input (separated from regular inptu by  tokens. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main concern is that it may begin to ignore these &quot;facts&quot; over time, and they may become irrelevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another (possibly much better) approach might be to use deep reinforcement learning. However I may find it difficult to implement with my existing Sequence-to-Sequence network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So which would likely be better? Or perhaps I should try a combination of the two?&lt;/p&gt;&#xA;" OwnerUserId="17432" LastEditorUserId="17432" LastEditDate="2018-11-16T18:50:53.837" LastActivityDate="2018-11-27T21:33:43.380" Title="What is the best way to integrate unchangeable ethics into a chatbot" Tags="&lt;reinforcement-learning&gt;&lt;chat-bots&gt;&lt;ethics&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9009" PostTypeId="2" ParentId="9005" CreationDate="2018-11-16T22:28:16.243" Score="3" Body="&lt;p&gt;The &quot;objective function&quot; is the function that you want to minimise or maximise in your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expression &quot;objective function&quot; is used in several different contexts (e.g. &lt;a href=&quot;https://stats.stackexchange.com/q/179026/82135&quot;&gt;machine learning&lt;/a&gt; or &lt;a href=&quot;https://www.courses.psu.edu/for/for466w_mem14/Ch11/HTML/Sec1/ch11sec1_ObjFn.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear programming&lt;/a&gt;), but it always refers to the function to be maximised or minimised in the specific (optimisation) problem. Hence, this expression is used in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;mathematical optimisation&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in machine learning, you define a model, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;. To train &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;, you usually define a loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; (e.g., a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean squared error&lt;/a&gt;), which you want to minimise. &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; is the &quot;objective function&quot; of your problem (which in this case is to be minimised).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of search algorithms, the objective function could represent e.g. the cost of the solution. For example, in the case of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;travelling salesman problem&lt;/a&gt; (TSP), you define a function, call it &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, which represents the &quot;cost&quot; of the tour or &lt;a href=&quot;http://mathworld.wolfram.com/HamiltonianCycle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hamiltonian cycle&lt;/a&gt;, that is, a function which sums up the weights of all edges in the tour. In this case, the &quot;objective&quot; of your problem is to minimise this function &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, because, essentially, you want to find an inexpensive tour, which is associated with either a local (or global) minimum of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. This function &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the &quot;objective function&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should now be easy to memorise the expression &quot;objective function&quot;, as it contains the term &quot;objective&quot;, and the &quot;objective&quot; (or goal) in your (optimisation) problem is to minimise (or maximise) the corresponding function.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-16T22:52:38.427" LastActivityDate="2018-11-16T22:52:38.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9010" PostTypeId="1" CreationDate="2018-11-16T22:59:49.967" Score="0" ViewCount="134" Body="&lt;p&gt;Let's say there are two types of cancer(Type 1 and Type 2). Say we want to see if one of pour friends has cancer Type 1 or 2. We can treat this as a classification problem. But what if we use unsupervised learning (clustering) to separate the data into to 2 different groups and see each whether each item in group 1 belongs to a person with cancer Type 1 or 2. We will then  see whether our friend belongs to group 1 or 2. I know it is stupid to do this and we have to do extra work but can we even do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that the features are only the age and the height (I know it's really dumb but just bear with me). The data associated with people with cancer Type 1 is &lt;code&gt;[10, 150], [12, 153], [9, 143], [13, 160]&lt;/code&gt; and for people with Type 2 cancer : &lt;code&gt;[20, 175], [23, 180], [19, 174]&lt;/code&gt;. Let's say we plot the data on a graph (without labelling the data) and the unsupervised program (Clustering) just separates the two groups (Say group 1 for Type 1). We then can see that to whom each data in group 1 belongs. We see those people have cancer Type 1. So given new data, we see what group our friend  belongs to. If she/he belonged to group 1, he's got cancer Type 1 and if not, she/he has cancer Type 2.&lt;/p&gt;&#xA;" OwnerDisplayName="user17894" LastEditorUserId="1847" LastEditDate="2018-11-17T08:50:47.707" LastActivityDate="2021-03-03T15:59:02.033" Title="Using unsupervised learning for classification problems" Tags="&lt;classification&gt;&lt;unsupervised-learning&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9011" PostTypeId="1" AcceptedAnswerId="9037" CreationDate="2018-11-17T06:03:54.540" Score="1" ViewCount="86" Body="&lt;p&gt;With the advancement of deep learning and a few others automated features learning techniques, manual feature engineering started becoming obsolete. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion on when to use manual feature engineering, feature learning or a combination of the two?&lt;/p&gt;&#xA;" OwnerUserId="17980" LastEditorUserId="2444" LastEditDate="2019-07-06T13:13:15.230" LastActivityDate="2019-07-06T13:13:15.230" Title="When should I use feature learning as opposed to feature engineering?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;feature-selection&gt;&lt;data-preprocessing&gt;&lt;feature-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9012" PostTypeId="2" ParentId="8422" CreationDate="2018-11-17T08:37:57.307" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the laptop Asus ZenBook Pro 90NX0152-M02980 enough to do deep learning model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not really.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It would be adequate if you want to learn the basics, just run TensorFlow or PyTorch and train some simple models, maybe run some tutorials on MNIST.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It would be adequate to run large models in prediction mode - e.g. you could run VGG-19 or InceptionNet-v5 and classify images.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can offload your larger training requirements to a separate server, and use the laptop to do data preparation and smaller scale tests, then a top end ultra-portable laptop would be fine.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have a similar spec laptop, and have taught myself a lot of deep learning concepts on it. However, I cannot realistically train any modern image-processing, natural language model or interesting deep RL model on it, without committing to days of training time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still have managed to do some interesting things, such as play with Deep Dream (even making a Deep Dream video), but I often hit limitations where I would like to try something but the laptop is not capable:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Cannot enter Kaggle competitions on image, audio or language datasets. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cannot perform style transfer in reasonable time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cannot solve larger, more interesting Reinforcement Learning problems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cannot train LSTMs on large text databases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to do those or similar things, you absolutely need a machine with a powerful GPU that can accelerate the deep learning library that you want to use. Typically that is an NVIDIA graphics card that supports &lt;a href=&quot;https://developer.nvidia.com/cuda-zone&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUDA&lt;/a&gt;. Even modest, relatively cheap (in modern standard) GPUs can accelerate deep learning model training very well, so a 1070 or 1060 card can make a big difference. The difference between the graphics cards is that better and more expensive ones are faster, can handle larger neural networks and larger problems overall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are still keen on buying a laptop, you will probably want a gaming laptop, and pay attention to what kind of GPU it contains. You will probably also want a larger memory than 8GB, and expect at least one of the drives to be of the SSD type so you can load training data faster. That combination does not come cheap in a laptop - e.g. an &lt;a href=&quot;https://www.amazon.co.uk/dp/B07G4VN7YC/ref=asc_df_B07G4VN7YC56916185/?tag=googshopuk-21&amp;amp;creative=22110&amp;amp;creativeASIN=B07G4VN7YC&amp;amp;linkCode=df0&amp;amp;hvadid=310653360667&amp;amp;hvpos=1o5&amp;amp;hvnetw=g&amp;amp;hvrand=9742644326590822717&amp;amp;hvpone=&amp;amp;hvptwo=&amp;amp;hvqmt=&amp;amp;hvdev=c&amp;amp;hvdvcmdl=&amp;amp;hvlocint=&amp;amp;hvlocphy=1006688&amp;amp;hvtargid=pla-756180723512&amp;amp;th=1&amp;amp;psc=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alienware laptop with i7 processor, 16GB RAM and NVIDIA 1070 costs £2,300&lt;/a&gt; as of November 2018, and I would consider that to be an acceptable, but not top of the range spec. Also, of course it is not ultra-portable, but quite the opposite, you won't be happy to carry it for a long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way to drop the price down is to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Buy a desktop, not a laptop&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Build your own from components. This is not without risk, and you should only attempt it if you have at least some of the knowledge and skills already (and could look up the rest). However, this can save a signifiant amount of money. You can search &quot;build own deep learning machine&quot; or similar to find many helpful articles on how to approach this, &lt;a href=&quot;https://medium.com/the-mission/why-building-your-own-deep-learning-computer-is-10x-cheaper-than-aws-b1c91b55ce8c&quot; rel=&quot;nofollow noreferrer&quot;&gt;like this one&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-17T16:14:42.177" LastActivityDate="2018-11-17T16:14:42.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9013" PostTypeId="2" ParentId="9010" CreationDate="2018-11-17T09:04:57.453" Score="1" Body="&lt;p&gt;&lt;em&gt;You cannot use clustering as an &quot;unsupervised classifier&quot;, with the goal of targeting specific known classes that are already defined. For that you need supervised learning and a labelled dataset.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A clustering algorithm would only separate into clusters as you hope if the feature variables you use are already:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;All&lt;/strong&gt; strongly associated with the trait you want the clusters to find&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Carefully scaled to match the expected relationship&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now this might happen by chance with an arbitrary dataset. However in practice in a complex dataset with many features (as you might use in cancer classification) you would need to select and scale just relevant features. Thus you would need to already know the relationship between features and class from some other model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main task of clustering is to &lt;em&gt;discover&lt;/em&gt; naturally occurring groups within your data, that might give you insight into the distributions within a dataset. This might let you compress or summarise information about examples. The summary feature might even be predictive, but it would very rarely overlap with pre-defined classes, even when the full feature vector could be used in an accurate supervised learning predictive model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your given example of &lt;code&gt;[age, height]&lt;/code&gt; you might discover that the population splits naturally into people who are still growing, fully-grown youths and mature people. If this happens to overlap with predictions of type 1 and type 2 cancer that might be interesting - perhaps a person's cluster is even a good predictive variable for having type 1 or type 2 cancer. However, whatever the features are, it is highly unlikely that it would be so strong a predictive variable that you would use it on its own to make a classification. The only variables that would be that predictive would be the clinical measurements &lt;em&gt;of the cancer growth&lt;/em&gt; that clinicians already use to decide the cancer type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Potentially you &lt;em&gt;could&lt;/em&gt; build a Bayesian model around the clusters, where you can ask questions such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the probability that a random person is in cluster A &lt;span class=&quot;math-container&quot;&gt;$p(x \in A)$&lt;/span&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the probability that a random person has cancer type 1 &lt;span class=&quot;math-container&quot;&gt;$p(x \in C_1)$&lt;/span&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the conditional probability that a person with type 1 cancer is found in cluster A &lt;span class=&quot;math-container&quot;&gt;$p(x \in A|x \in C_1)$&lt;/span&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Bayes' Theorem would then allow you to calculate the probability of having type 1 cancer based on their clustering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(x \in C_1 | x \in A) = \frac{p(x \in A|x \in C_1)}{(x \in A)}p(x \in C_1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this fully with Bayesian statistics would involve setting some prior beliefs about the probabilities and using the data from your analysis to adjust them, which is beyond the scope of this answer. However, you could as a crude shortcut just take the measured frequencies, and that might be fine as a guide and give you some intuitions about what your clustering is doing towards helping with predictions.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-17T09:30:13.490" LastActivityDate="2018-11-17T09:30:13.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9014" PostTypeId="1" AcceptedAnswerId="9015" CreationDate="2018-11-17T11:39:38.267" Score="2" ViewCount="784" Body="&lt;p&gt;I'm trying to implement a Deep Q-network in Keras/TF that learns to play Minesweeper (our stochastic environment). I have noticed that the agent learns to play the game pretty well with both small and large board sizes. However, it only converges/learns when the layout of the mines is the same for each game. That is, if I randomize the mine distribution from game to game, the agent learns nothing - or near to it. I tried using various network architectures and hyperparameters but to no avail.&lt;/p&gt;&#xA;&lt;p&gt;I tried a lot of network architectures including:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The input to the network is the entire board matrix, with the individual cells having values of -1 if unrevealed, or 0 to 8 if revealed.&lt;/li&gt;&#xA;&lt;li&gt;The output of the network is also the entire board representing the desirability of clicking each cell.&lt;/li&gt;&#xA;&lt;li&gt;Tried fully connected hidden layers (both wide and deep).&lt;/li&gt;&#xA;&lt;li&gt;Tried Convolutional hidden layers (tried stacked them, using different kernel sizes, padding, etc.).&lt;/li&gt;&#xA;&lt;li&gt;Tried adding Dropout after hidden layers too.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is DQN applicable for environments that change every episode or have I approached this from the wrong side?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems no matter the network architecture, the agent won't learn. Any input is greatly appreciated. Please let me know if you require any code or further explanations.&lt;/p&gt;&#xA;" OwnerUserId="19981" LastEditorUserId="32410" LastEditDate="2020-12-29T14:35:12.490" LastActivityDate="2020-12-29T14:35:12.490" Title="Deep Q-Learning poor convergence on Stochastic Environment" Tags="&lt;reinforcement-learning&gt;&lt;deep-neural-networks&gt;&lt;keras&gt;&lt;q-learning&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9015" PostTypeId="2" ParentId="9014" CreationDate="2018-11-17T12:48:41.930" Score="3" Body="&lt;p&gt;The inputs that you describe seem like they should be sufficient for a DQN-based agent to learn a good strategy for playing Minesweeper, regardless of whether or not the starting layout changes. The inputs contain all information that is necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem certainly becomes much easier (probably &lt;strong&gt;too easy&lt;/strong&gt;) if the initial problem is always the same. The DQN algorithm will be very likely to &quot;pick up&quot; on this trend and &quot;exploit&quot; it. The inputs may be sufficient to learn a more general Minesweeper strategy, but if it is consistently faced with the same &quot;level&quot; every single time, it will be much easier for the DQN algorithm to just memorize exactly where the mines are and play perfectly based on that memory, rather than any actual strategy. Due to the way learning is implemented (based on gradient descent), the algorithm will generally tend to converge to such an easily-reachable &quot;memorization&quot; strategy rather than something that is actually &quot;smart&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that reason, I do think training will be much slower in the case where the layout is randomized. I'm not just thinking along the lines of e.g. twice as slow here, but would expect multiple orders of magnitude more experience required for successful learning. That's just my educated guess though, never tried training DQN for minesweeper specifically so can't tell for sure. It might also require more elaborate hyperparameter tuning, and maybe require a different network architecture (probably require a larger network).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-11-17T12:48:41.930" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9017" PostTypeId="1" AcceptedAnswerId="9043" CreationDate="2018-11-17T13:22:49.690" Score="2" ViewCount="2428" Body="&lt;p&gt;What is the difference between the breadth-first search and recursive best-first search? How can I describe the key difference between them?&lt;/p&gt;&#xA;" OwnerUserId="19657" LastEditorUserId="2444" LastEditDate="2019-11-10T17:11:29.837" LastActivityDate="2020-05-21T22:20:53.760" Title="What is the difference between the breadth-first search and recursive best-first search?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;breadth-first-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9019" PostTypeId="1" AcceptedAnswerId="9021" CreationDate="2018-11-17T13:46:26.643" Score="2" ViewCount="3449" Body="&lt;p&gt;Recently, I have come across the information (lecture 8 and 9 about MDPs of &lt;a href=&quot;http://ai.berkeley.edu/lecture_slides.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this UC Berkeley AI course&lt;/a&gt;) that the time complexity for each iteration of the value iteration algorithm is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(|S|^{2}|A|)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$|S|$&lt;/span&gt; is the number of states and &lt;span class=&quot;math-container&quot;&gt;$|A|$&lt;/span&gt; the number of actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the equation for each iteration:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;V_{k+1}(s) \gets \max_a \sum_{s'} T(s, a, s') [R(s, a, s') + \gamma V_k(s')]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could't understand why the time complexity is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(|S|^{2}|A|)$&lt;/span&gt;. I searched the internet, but I didn't find any good explanation. &lt;/p&gt;&#xA;" OwnerUserId="19984" LastEditorUserId="2444" LastEditDate="2019-02-16T02:15:23.207" LastActivityDate="2019-02-16T02:15:23.207" Title="What is the time complexity of the value iteration algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;time-complexity&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9021" PostTypeId="2" ParentId="9019" CreationDate="2018-11-17T14:07:25.403" Score="1" Body="&lt;p&gt;The update equation for &lt;a href=&quot;https://artint.info/html/ArtInt_227.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration&lt;/a&gt; that you show is time complexity &lt;span class=&quot;math-container&quot;&gt;$O(|\mathcal{S}\times\mathcal{A}|)$&lt;/span&gt; for each update to a single &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; estimate, because it iterates over all actions to perform &lt;span class=&quot;math-container&quot;&gt;$\text{max}_a$&lt;/span&gt; and over all next states for &lt;span class=&quot;math-container&quot;&gt;$\sum_{s'}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sources you have found are probably counting an entire &lt;em&gt;sweep&lt;/em&gt; through the state space as an &quot;iteration&quot; i.e. &lt;span class=&quot;math-container&quot;&gt;$\forall \space s \in \mathcal{S}: V_{k+1}(s) \leftarrow \text{max}_a \sum_{s'} T...$&lt;/span&gt; That adds another factor of &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}|$&lt;/span&gt; making the overall complexity &lt;span class=&quot;math-container&quot;&gt;$O(|\mathcal{S}\times\mathcal{S}\times\mathcal{A}|)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$O(|\mathcal{S}|^2|\mathcal{A}|)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition of iteration makes sense, as the basic value iteration algorithm is &lt;em&gt;required&lt;/em&gt; to sweep through the whole state space in order to converge. This also matches the standard test for convergence, which is made after each full sweep, and checks what the largest absolute update was at the end of the sweep - if it is below some target value for accuracy then the process is declared complete. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-17T14:55:12.840" LastActivityDate="2018-11-17T14:55:12.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9022" PostTypeId="1" CreationDate="2018-11-17T15:47:15.667" Score="5" ViewCount="4194" Body="&lt;p&gt;I'm new to machine learning. I was watching a Prof. Andrew Ng's video about gradient descent from the machine learning online course. It said that we want our cost function (in this case, the &lt;em&gt;mean squared error&lt;/em&gt;) to have the minimum value, but that minimum value shown in the graph was not 0. It was a negative number! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can our cost function, which is mean squared error, have a negative value, given that the square of a real number is always positive? Even if it is possible, don't we want our error to be 0?&lt;/p&gt;&#xA;" OwnerUserId="19988" LastEditorUserId="23713" LastEditDate="2020-01-09T08:43:06.753" LastActivityDate="2020-01-09T08:43:06.753" Title="Can the mean squared error be negative?" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;loss-functions&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9023" PostTypeId="2" ParentId="9022" CreationDate="2018-11-17T15:56:55.027" Score="7" Body="&lt;p&gt;In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A standard Mean Squared Error function cannot be negative. The lowest possible value is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, when there is no output error from any example input.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can our cost function which is mean squared error have a value under 0?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It cannot. You don't link the precise graph or lecture where you saw this, but I would suspect Andrew Ng drew a representative graph for &lt;em&gt;any&lt;/em&gt; cost function in order to point out that it would typically have an optimum, minimum value. He may have been talking at the same time about MSE as an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many loss or cost functions are designed with an absolute minimum of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; possible for &quot;no error&quot; results. In supervised learning that is often a simple consequence of basing the cost on the difference between the model outputs and desired outputs. So in supervised learning problems of regression and classification, you will rarely see a negative cost function value. But there is no absolute rule against negative costs in principle.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-17T16:21:09.437" LastActivityDate="2018-11-17T16:21:09.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9024" PostTypeId="1" CreationDate="2018-11-17T16:23:25.537" Score="2" ViewCount="1123" Body="&lt;p&gt;Apparently, in the Q-learning algorithm, the Q values are not updated according to the &quot;current policy&quot;, but according to a &quot;greedy policy&quot;. Why is that the case? I think this is related to the fact that Q-learning is off-policy, but I am also not familiar with this concept. &lt;/p&gt;&#xA;" OwnerUserId="19984" LastEditorUserId="2444" LastEditDate="2020-11-29T13:47:59.297" LastActivityDate="2020-11-29T13:47:59.297" Title="Why are Q values updated according to the greedy policy?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;off-policy-methods&gt;&lt;greedy-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9025" PostTypeId="1" AcceptedAnswerId="9027" CreationDate="2018-11-17T17:36:55.853" Score="0" ViewCount="364" Body="&lt;p&gt;I have done a lot of research on the internet about Reinforcement Learning and I found encountered methods of Reinforcement Learning: Q-Learning and Deep Q-Learning. And I have developed a vague idea of how these two work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I knew anything about Reinforcement Learning this is how I thought it would work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have 2 virtual players in a game who can shoot each other, one of them is a decent playing hard-coded/pre-coded AI, and the other one is the player I want to train (to shoot the other player and dodge his bullets), the aim of the game would be to get the greatest net score (shots you hit minus shots you took) within 1 minute (a session), and you only have 20 bullets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Y3Fba.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y3Fba.png&quot; alt=&quot;A basic idea of the the game I mean&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have 3 actions, move left-right &lt;code&gt;(0 = maxleftspeed, 1 = maxrightspeed )&lt;/code&gt;, jump &lt;code&gt;(0=don't jump, 1=jump)&lt;/code&gt;, shoot &lt;code&gt;(0 = don't shoot, 1 = shoot)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I thought was, you could create a basic Feed-Forward Neural Network use the enemy's position and his bullet(s)'s position(s) and bullet direction(s) for the input layer and the action being taken will be given by the (3 nodes in the) output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The untrained player starts off with a randomized algorithm, then (for back-propagation) at the end of each session it modifies one of the parameters by a bit in the neural network, and a new session is started with the slightly modified NN. If this session ends with more points than the previous session, it keeps the changes and makes more changes towards that direction, otherwise, the changes are redone, or possibly reversed. I would visualise this as gradient descent similar to that of Supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is something like this already out there? What is it called?&lt;/li&gt;&#xA;&lt;li&gt;If nothing like this is out there, could you give me any tips to optimize this method or point out any key points I should keep in minds while carrying this out?&lt;/li&gt;&#xA;&lt;li&gt;Since I have written this game, I have control over the speed of the actions, but if I did not, I know this AI would take ages to learn, so is there any way to make the learning faster while still keeping the basic idea in mind?&lt;/li&gt;&#xA;&lt;li&gt;How exactly is this different from deep Q-learning (if it is)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="18711" LastEditorUserId="2444" LastEditDate="2018-11-18T10:57:59.313" LastActivityDate="2018-11-18T10:57:59.313" Title="Is it possible to use a feed-forward neural network to predict the actions in reinforcement learning?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9026" PostTypeId="1" CreationDate="2018-11-17T18:43:24.567" Score="1" ViewCount="67" Body="&lt;p&gt;First of all I'm very new to the field. Maybe my question is a bit too naive or even trivial...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently trying to understand how can I go about recognizing different faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I tried so far and the main issues with each approach:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Haar Cascade -&gt; HOG -&gt; SVM:&#xA;  The main issue is that the algorithm becomes very indecisive when more than four people are trained... The same occurs when we change Haar Cascade for a pre-trained CNN to detect faces...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) dlib facial landmarks -&gt; distance between points -&gt; SVM or Simple Neural Network Classification:&#xA;  This is the current approach and it behaves very well when when four people are trained... When more people are trained it becomes very messy, jumping from decision to decision and never resolves to a choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read online that Triplet loss is the way to go... But I very confused as to how I'd go about implementing it... Can I use the current distance vectors found using Dlib or should I scrap everything and train my own CNN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I can use the distance vectors, how would I pass the data to the algorithm? Is Triplet loss a trivial neural network only with its loss function altered? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've took the liberty to show exactly how the distance vectors are being calculated:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s7zGU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s7zGU.png&quot; alt=&quot;The green lines represent the distances being calculated&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The green lines represent the distances being calculated.&#xA;A 33 float list is returned which is then fed to the classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the relevant code for the classifier (Keras):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fit_classifier(self):&#xA;    x_train, y_train = self._get_data(self.train_data_path)&#xA;    x_test, y_test = self._get_data(self.test_data_path)&#xA;    encoding_train_y = np_utils.to_categorical(y_train)&#xA;    encoding_test_y = np_utils.to_categorical(y_test)&#xA;    model = Sequential()&#xA;    model.add(Dense(10, input_dim=33, activation='relu'))&#xA;    model.add(Dense(20, activation='relu'))&#xA;    model.add(Dense(30, activation='relu'))&#xA;    model.add(Dense(40, activation='relu'))&#xA;    model.add(Dense(30, activation='relu'))&#xA;    model.add(Dense(20, activation='relu'))&#xA;    model.add(Dense(10, activation='relu'))&#xA;    model.add(Dense(max(y_train)+1, activation='softmax'))&#xA;    model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])&#xA;    model.fit(x_train, encoding_train_y, epochs=100, batch_size=10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think this is a more theoretical question than anything else... If someone with good experience in the field could help me out I'd be very happy! &lt;/p&gt;&#xA;" OwnerUserId="19992" LastEditorUserId="16229" LastEditDate="2019-06-27T20:16:44.100" LastActivityDate="2019-06-27T20:16:44.100" Title="Machine learning approach to facial recognition" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9027" PostTypeId="2" ParentId="9025" CreationDate="2018-11-17T18:45:27.460" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is something like this already out there? What is it called?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are three algorithms that have similarities to yours, but are not identical:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/?neat&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt;, which is an evolutionary method that mutates neural networks and compares their fitness across populations of variations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;https://www.aaai.org/Papers/ICML/2003/ICML03-068.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross Entropy Method&lt;/a&gt;, which trains a single, initially randomly behaving, neural network based on the outcomes of its best performances.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://mcneela.github.io/math/2018/04/18/A-Tutorial-on-the-REINFORCE-Algorithm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;REINFORCE&lt;/a&gt; is a policy-gradient method that trains a neural network after completing games based on the results of every game. It is very similar to your idea, except that it includes the maths/logic to sample a full gradient of all NN parameters in a single go. It does not require making a test change to a single parameter, but can calculate the full error gradient at once. Usually, however, to reduce noise and effects of correlation within a game, multiple gradients are collected from different games and averaged.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The main similarity in each case is that you are performing a policy search, and training based on selecting outcomes that are closer to your goal after trials. A policy-based approach means that your NN represents the choice of action directly. A policy is a function that converts knowledge of the state of an environment into an action. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If nothing like this is out there, could you give me any tips to optimize this method or point out any key points I should keep in minds while carrying this out?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your suggested method is not exactly the same as any of the suggested algorithms, and may have a problem with performance due to this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The untrained player starts off with a randomized algorithm, then (for back-propagation) at the end of each session it modifies one of the parameters by a bit in the neural network, and a new session is started with the slightly modified NN. If this session ends with more points than the previous session, it keeps the changes and makes more changes towards that direction, otherwise, the changes are redone, or possibly reversed. I would visualise this as gradient descent similar to that of Supervised learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This single parameter change search is likely to be slow, probably unusably so for any reasonably-sized neural network. Effectively you would be performing a single parameter step for a very noisy gradient descent algorithm, per full play of the game. All of NEAT, CEM and REINFORCE are likely to perform much better than this (and none of them are state-of-the-art).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;is there any way to make the learning faster while still keeping the basic idea in mind?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Instead of sampling the total reward at the end of each game and generating a single parameter's gradient, use a deep reinforcement learning method that works with full gradients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might find REINFORCE interesting, as it logically the idea closest to yours, and it leads to some state-of the art methods such as A3C, DDPG, PPO . . . the main problem with all of these being that the maths is hard to understand, even though the implementations can be simple you may be left scratching your head as to why they even work. However, if you are happy with just seeing the &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html#policy-gradient-theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;policy gradient theorem&lt;/a&gt; as some maths which completes your idea, then it is there for the taking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, if your goal is to implement something you understand the internals of fully, I would suggest that you search for a tutorial on deep Q learning, maybe something that implements DQN, as the maths and intuition behind it are more accessible.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How exactly is this different from deep Q-learning (if it is)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both your suggested method and Q learning can probably be considered &quot;deep reinforcement learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main differences are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Q learning is a value-based method, and yours is a policy-based method&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Q learning is off-policy (it learns best overall actions even whilst exploring other possibilities) whilst your approach is on-policy (the actions it learns to take are the same as the actions it takes during exploration)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Deep Q learning updates all the parameters of the NN on each time step, whilst you will be updating a single parameter per episode. That makes Q learning far more sample efficient than your suggested approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-17T18:51:22.553" LastActivityDate="2018-11-17T18:51:22.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9028" PostTypeId="2" ParentId="9022" CreationDate="2018-11-17T18:48:46.827" Score="1" Body="&lt;p&gt;Mean squared error terms must be positive because the square of a number is positive.   Therefore the sum (cost) is positive.  The error is the difference between the hypothesis and the observation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would focus on understanding why Ng seeks to minimize J and how it is minimization is achieved with partial derivatives via matrix implementation.  &lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="18819" LastEditDate="2018-11-17T22:29:18.093" LastActivityDate="2018-11-17T22:29:18.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9029" PostTypeId="2" ParentId="2980" CreationDate="2018-11-17T19:09:20.057" Score="7" Body="&lt;p&gt;I faced a similar issue recently with Minesweeper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I solved it was by ignoring the illegal/invalid moves entirely.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use the Q-network to predict the Q-values for all of your actions (valid and invalid)&lt;/li&gt;&#xA;&lt;li&gt;Pre-process the Q-values by setting all of the invalid moves to a Q-value of zero/negative number (depends on your scenario)&lt;/li&gt;&#xA;&lt;li&gt;Use a policy of your choice to select an action from the refined&#xA;Q-values (i.e. greedy or Boltzmann)&lt;/li&gt;&#xA;&lt;li&gt;Execute the selected action and resume your DQN logic&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="19981" LastEditorUserId="19981" LastEditDate="2018-12-04T21:20:06.213" LastActivityDate="2018-12-04T21:20:06.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9030" PostTypeId="1" CreationDate="2018-11-17T19:53:45.473" Score="1" ViewCount="35" Body="&lt;p&gt;Lets say we have a data-set of all cats and we have to identify the cat breed based on given test image. As, the two different cat breeds have visual similarity can we use existing networks (VGG, ImageNet, GoogleNet) to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should faceNet be applied here? As, the problem is similar to face detection where face characteristics of two different people are same yet it can correctly recognize a person.&lt;/li&gt;&#xA;&lt;li&gt;What if with visual similarity in data-set we have only few example of each class? Like for a problem (random) we have good amount of data but for each class we have only few examples.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any model that can be applied here?&lt;/p&gt;&#xA;" OwnerUserId="19966" LastActivityDate="2019-04-16T22:24:41.920" Title="Image prediction model when data-set classes have visual similarity" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9031" PostTypeId="2" ParentId="9030" CreationDate="2018-11-17T20:32:31.510" Score="1" Body="&lt;p&gt;You could use &lt;a href=&quot;http://cs231n.github.io/transfer-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt;. Find some state of the art neural net (something newer than VGG) already trained for image classification. Then remove the last layer and add a new one, training it while keeping the remaining old weights fixed. A simple softmax at the end would work.This is the simplest approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because you're mostly only training the last layer, you won't need as much data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think &lt;a href=&quot;https://arxiv.org/pdf/1503.03832.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;FaceNet&lt;/a&gt; will perform poorly because it uses different features and you would need to retrain all its layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have too little data for some classes, maybe you should use an &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;autoencoder&lt;/a&gt; as a features extractor. Then you could classify an image by comparing its feature vector to those for each breed, for example with &lt;a href=&quot;https://en.wikipedia.org/wiki/Cosine_similarity&quot; rel=&quot;nofollow noreferrer&quot;&gt;cosine similarity&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="13067" LastEditorUserId="22916" LastEditDate="2019-04-16T22:24:41.920" LastActivityDate="2019-04-16T22:24:41.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9033" PostTypeId="1" CreationDate="2018-11-17T22:06:42.443" Score="2" ViewCount="327" Body="&lt;p&gt;In Q-learning, during training, it doesn't matter how the agent selects actions. The algorithm always converges to the optimal policy. Why does this happen? What's the intuition?&lt;/p&gt;&#xA;" OwnerUserId="19984" LastEditorUserId="2444" LastEditDate="2020-11-29T13:50:19.793" LastActivityDate="2021-04-28T18:07:25.717" Title="Why does Q-learning converge to the optimal policy, even if the agent acts sub-optimally?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;proofs&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9035" PostTypeId="2" ParentId="3596" CreationDate="2018-11-18T05:28:45.720" Score="1" Body="&lt;p&gt;This is probably a late answer, but I believe there is a mistake in the code that could be affecting the convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;code&gt;gen_next_batch&lt;/code&gt; method, you &lt;code&gt;return X, Y&lt;/code&gt; while inside the for-loop. This results in the entire array of X and Y filled with zeros, except for the first one (index 0); thus, considerably slowing down your training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="19981" LastActivityDate="2018-11-18T05:28:45.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9037" PostTypeId="2" ParentId="9011" CreationDate="2018-11-18T11:27:03.863" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;manual feature engineering started becoming obsolete&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is wrong.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any suggestion on when to use manual feature engineering, feature learning or a combination of the two?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Deep learning is awesome for natural signals like images, audio or large amounts of unstructured text (e.g. arbitrary crawled websites)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some basic steps that make almost always sense. For example, for image recognition, it is the normalization from pixel color features in the range &lt;span class=&quot;math-container&quot;&gt;$\{0, 1, \dots, 255 \}$&lt;/span&gt; to features in &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then there are a lot of use cases where you have structured, CSV-like data. For those, every single case I had so far was improved by feature engineering. Most of the time you actually don't have a choice here as well: how would you feed in a date time object into a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, where does feature engineering make sense in image recognition?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Resource limitations: neural networks often take a lot of resources. For example, viola-jones face detection is very resource friendly in contrast. But there are developments like the mobile net to relax that issue.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Prior knowledge: for more special image data like CT scans you might observe super simple operations that almost give you the target. This might make it simpler to interpret the results at the end and, again, it helps to keep resource usage low.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One other field where I suspect that manual feature engineering could help is adversarial attacks. But for this I'm not 100% sure.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="2444" LastEditDate="2019-07-06T13:10:03.203" LastActivityDate="2019-07-06T13:10:03.203" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9038" PostTypeId="2" ParentId="3959" CreationDate="2018-11-18T13:06:11.213" Score="0" Body="&lt;p&gt;If you are so sure that ML is the way to go ahead, then try following.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research/object_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow code for object detection&lt;/a&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO&lt;/a&gt;&#xA;This is especially for real time object detection. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can train these model for your object in question &amp;amp; see the results.There are some other localization models that you can try. &lt;/p&gt;&#xA;" OwnerUserId="17980" LastActivityDate="2018-11-18T13:06:11.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9039" PostTypeId="1" CreationDate="2018-11-18T13:40:03.443" Score="2" ViewCount="331" Body="&lt;p&gt;Are there examples of applications in blockchain consensus using swarm intelligence, as opposed to classical consensus mechanisms like PoW or PBFT?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Please note&lt;/strong&gt; that recent classical consensuses, including &lt;em&gt;lottery-based&lt;/em&gt; such as &lt;strong&gt;PoW&lt;/strong&gt; in which the winner of lottery creates the new block, or &lt;em&gt;voting-based&lt;/em&gt; such as &lt;strong&gt;PBFT&lt;/strong&gt; or &lt;strong&gt;Paxos&lt;/strong&gt; in which the entities achieve a consensus through a voting process; both approaches have problems of efficiency, latency, scalability, performance etc. And emerging a new alternative approach seems necessary. In this way, can &lt;strong&gt;nature-inspired&lt;/strong&gt; algorithms (such as &lt;strong&gt;evolutionary algorithms&lt;/strong&gt; (EA), &lt;strong&gt;particle swarm optimization&lt;/strong&gt; (PSO), &lt;strong&gt;ant colony optimizatio&lt;/strong&gt;n (ACO) etc) be employed as an alternative to classical consensus algorithms? &lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="19910" LastEditDate="2018-11-18T21:40:03.827" LastActivityDate="2020-10-17T16:07:18.317" Title="How Swarm Intelligence can empower Blockchain?" Tags="&lt;terminology&gt;&lt;evolutionary-algorithms&gt;&lt;applications&gt;&lt;swarm-intelligence&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9041" PostTypeId="2" ParentId="3852" CreationDate="2018-11-18T14:33:25.570" Score="1" Body="&lt;p&gt;I refer you to &lt;a href=&quot;https://www.cs.cmu.edu/~tom/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prof. Tom Mitchell&lt;/a&gt;'s (the lecturer in the video) &lt;a href=&quot;http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;draft chapter&lt;/a&gt; as the best explanation I could find. I will try to explain it in layman's terms here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a boolean problem, our logistic regression classifier will assign the label &lt;span class=&quot;math-container&quot;&gt;$Y=0$&lt;/span&gt; iff &lt;span class=&quot;math-container&quot;&gt;$w_0 + \sum_{i=1}^{n}w_{i}X_{i} &amp;gt; 0$&lt;/span&gt; (from the video you posted; eqn. 18 in the draft chapter). Otherwise, it will assign the label &lt;span class=&quot;math-container&quot;&gt;$Y=1$&lt;/span&gt;. Remember the goal of logistic regression is to learn the weights &lt;span class=&quot;math-container&quot;&gt;$w_0, w_1, ..., w_n$&lt;/span&gt; that will give us a linear function that would seperate our data best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be shown that &lt;span class=&quot;math-container&quot;&gt;$w_0=\ln\frac{1-\pi}{\pi} + \sum_{i=1}^{n} \frac{\mu_{i1}^2 - \mu_{i0}^2}{2\sigma_{i}^2}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_i = \frac{\mu_{i0} - \mu_{i1}}{\sigma_{i}^2}$&lt;/span&gt; (eqns. 19-23 in the draft chapter), where:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$P(Y=1)$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mu_{i0}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mu_{i1}$&lt;/span&gt; are the means of feature &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; in classes &lt;span class=&quot;math-container&quot;&gt;$Y=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y=1$&lt;/span&gt;, respectively, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt; is the standard deviation of feature &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This means that we can express our weights &lt;span class=&quot;math-container&quot;&gt;$w_0, w_1, ..., w_n$&lt;/span&gt; in terms of the Gaussian Naive Bayes classifier's parameters. Now, here is the difference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our goal is to find &lt;span class=&quot;math-container&quot;&gt;$P(Y=y_k|X_1 ... X_n)$&lt;/span&gt;. Assuming conditional independence, &lt;span class=&quot;math-container&quot;&gt;$P(Y=y_k|X_1 ... X_n) \propto P(Y=y_k) \prod_{i}^n P(X_i | Y=y_k)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If we estimate &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mu_{i0}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mu_{i1}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt;, then use those values to estimate &lt;span class=&quot;math-container&quot;&gt;$w_0, w_1, ..., w_n$&lt;/span&gt;, then we use the conditional independence property directly in our calculations.&lt;/em&gt; This is what the Naive Bayes classifier does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If we estimate &lt;span class=&quot;math-container&quot;&gt;$w_0, w_1, ..., w_n$&lt;/span&gt; directly, by repeatedly minimizing the error of an assumed linear function, we are not strictly abiding by the conditional independence property, because we don't explicitly calculate an estimate of the conditional probability &lt;span class=&quot;math-container&quot;&gt;$P(X_i|Y=y_k)$&lt;/span&gt;. We completely bypass that step.&lt;/em&gt; This is what the Logistic Regression classifier does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the section 4 of the draft chapter (GNB is short for Gaussian Naive Bayes):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... Logistic Regression directly estimates the parameters of &lt;span class=&quot;math-container&quot;&gt;$P(Y|X)$&lt;/span&gt;, whereas Naive Bayes directly estimates parameters for &lt;span class=&quot;math-container&quot;&gt;$P(Y)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(X|Y)$&lt;/span&gt;... if the GNB assumptions hold, then asymptotically (as the number of training examples grows&#xA;  toward infinity) the GNB and Logistic Regression converge toward identical classifiers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Another way to understand this is that it has to do with the fact that &lt;strong&gt;Logistic Regression is a discriminative classifier&lt;/strong&gt;, whereas &lt;strong&gt;Naive Bayes is a generative classifier&lt;/strong&gt;. For more on that, read &lt;a href=&quot;https://www.andrewng.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prof. Andrew Y. Ng&lt;/a&gt; and &lt;a href=&quot;https://people.eecs.berkeley.edu/~jordan/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prof. Michael I. Jordan&lt;/a&gt;'s paper, &lt;em&gt;&lt;a href=&quot;https://ai.stanford.edu/~ang/papers/nips01-discriminativegenerative.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19932" LastActivityDate="2018-11-18T14:33:25.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9042" PostTypeId="2" ParentId="4000" CreationDate="2018-11-18T16:46:23.933" Score="0" Body="&lt;p&gt;&lt;strong&gt;TLDR&lt;/strong&gt;: If you are attempting to find the global optimum of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a score function with multiple local optima, such that not all local optima have an equal value.&lt;/p&gt;&#xA;" OwnerUserId="19932" LastActivityDate="2018-11-18T16:46:23.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9043" PostTypeId="2" ParentId="9017" CreationDate="2018-11-18T18:00:08.380" Score="1" Body="&lt;p&gt;According to &lt;a href=&quot;http://intelligence.worldofcomputing.net/ai-search/breadth-first-search.html#.XsaiRRMzZZ0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Breadth First Search (BFS) searches breadth-wise in the problem space. Breadth-First search is like traversing a tree where each node is a state which may a be a potential candidate for solution. It expands nodes from the root of the tree and then generates one level of the tree at a time until a solution is found. It is very easily implemented by maintaining a queue of nodes. Initially the queue contains just the root. In each iteration, node at the head of the queue is removed and then expanded. The generated child nodes are then added to the tail of the queue&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;According to the book &lt;em&gt;Algorithms and Theory of Computation Handbook&lt;/em&gt; by Mikhail J. Atallah.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Recursive best-first search is a best-first search that runs in space that is linear with respect to the maximum search depth, regardless of the cost function used.&lt;/p&gt;&#xA;&lt;p&gt;It works by maintaining on the recursion stack the complete path to the current node being expanded as well as all immediate siblings of nodes on that path, along with the cost of the best node in the sub-tree explored below each sibling.&lt;/p&gt;&#xA;&lt;p&gt;Whenever the cost of the current node exceeds that of some other node in the previously expanded portion of the tree, the algorithm backs up to their deepest common ancestor, and continues the search down the new path.&lt;/p&gt;&#xA;&lt;p&gt;In effect, the algorithm maintains a separate threshold for each sub-tree diverging from the current search path.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-21T22:20:53.760" CommentCount="0" CommunityOwnedDate="2020-11-24T02:54:24.660" ContentLicense="CC BY-SA 4.0" />
  <row Id="9046" PostTypeId="1" CreationDate="2018-11-18T19:47:26.853" Score="2" ViewCount="70" Body="&lt;p&gt;I am reading the paper on &lt;span class=&quot;math-container&quot;&gt;$\ell_0$&lt;/span&gt; regularization of DNNs by Louizos, Welling and Kingma (2017) (&lt;a href=&quot;https://arxiv.org/abs/1712.01312&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt; to arxiv).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Section 2.1 the authors define the cost function as follows: &lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{R}\left( \tilde{\theta}, \pi \right) = \mathbb{E}_{q(z|\pi)}\left[ \frac{1}{N} \left(\sum_{i=1}^N \mathcal{L}\left(h\left( x_i, \tilde{\theta}\circ Z\right), y_i \right) \right)\right] + \lambda\sum_{i=1}^{|\tilde{\theta}|}\pi_i. $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above display, &lt;span class=&quot;math-container&quot;&gt;$\tilde{\theta}$&lt;/span&gt; are the weights, &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is a random vector of the same dimension as &lt;span class=&quot;math-container&quot;&gt;$\tilde{\theta}$&lt;/span&gt; consisting of independent Bernoulli components &lt;span class=&quot;math-container&quot;&gt;$q(Z_i|\pi) \sim Bernoulli (\pi_i)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\circ$&lt;/span&gt; is the element-wise product. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors then state the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the first term is problematic for &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; due to the discrete nature of &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;, which does not allow for efficient gradient based optimization. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure I understand this. Denoting the first term by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}_1 = \sum_{i=1}^N \frac{1}{N}R_i$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; defined below), and using the notation &lt;span class=&quot;math-container&quot;&gt;$\pi_z = \prod \pi_i^{z_i} (1-\pi_i)^{1-z_i}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Z}$&lt;/span&gt; for the set of all possible values of &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;, we should have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_i := \mathbb{E}_{q(z|\pi)}\left[ \mathcal{L}\left( h\left(x_i, \tilde{\theta}\circ z\right), y_i \right) \right] = \sum_{z \in \mathcal{Z}}\pi_z\mathcal{L}\left( h\left(x_i, \tilde{\theta}\circ z\right), y_i \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, it seems to me that the gradient of &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\pi_j$&lt;/span&gt; can be obtained as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{d R_i}{d\pi_j} = \sum_{z \in \mathcal{Z}}\mathcal{L}\left( h\left(x_i, \tilde{\theta}\circ z\right), y_i \right) \frac{d\pi_z}{d\pi_j}&#xA;$$&lt;/span&gt;&#xA;and &#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{d\pi_z}{d\pi_j} = \frac{\pi_z}{\pi_j}$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$z_j=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-\frac{\pi_z}{1-\pi_j}$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$z_j=0$&lt;/span&gt;. So, it appears that we can obtain the derivative of the first term with respect to &lt;span class=&quot;math-container&quot;&gt;$\pi_j$&lt;/span&gt; as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If my above calculation is correct, then the derivatives &lt;span class=&quot;math-container&quot;&gt;$\frac{d\mathcal{R}_1}{d\pi_j}$&lt;/span&gt; can be computed, and we can perform SGD on the cost function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(\tilde{\theta}, \pi)$&lt;/span&gt;. But the authors claim that it cannot be obtained and hence they introduce the `hard concrete' distribution etc. to construct a differentiable cost function. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="20016" LastEditorUserId="20016" LastEditDate="2018-11-22T20:03:07.243" LastActivityDate="2018-11-22T20:03:07.243" Title="Regarding L0 sparsification of DNNs proposed by Louizos, Kingma and Welling" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;regularization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9047" PostTypeId="1" AcceptedAnswerId="9051" CreationDate="2018-11-18T22:00:47.657" Score="1" ViewCount="126" Body="&lt;p&gt;Apparently, one can buy a special-purpose integrated circuit (an IC like &lt;a href=&quot;https://software.intel.com/en-us/neural-compute-stick&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one,&lt;/a&gt; for instance) to host a convolutional neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTION&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is such a circuit digital? Except for digital random-number generation, is the circuit's behavior deterministic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What &lt;em&gt;kind&lt;/em&gt; of hardware is this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;REFERENCE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Already asked and answered: &lt;a href=&quot;https://ai.stackexchange.com/q/7328/20018&quot;&gt;&quot;If digital values are mere estimates, why not return to analog for AI?&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FURTHER DETAILS&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean is this: suppose that you and I each acquired a special-purpose neural IC, same make, same model. Suppose that you and I configured and trained our circuits alike. Except for digital random-number generation, would our neural nets behave identically? Or would analog effects cause our nets to turn oppositely on some closely balanced edge case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is plain to you that a misconception has caused me to pose the question improperly, then a correction to the question along with an answer would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to know my background, I am an electrical power engineer in my 50s.&lt;/p&gt;&#xA;" OwnerUserId="20018" LastEditorUserId="20018" LastEditDate="2018-11-18T22:30:24.880" LastActivityDate="2018-11-19T00:35:27.110" Title="Are commercially available neural ICs digital?" Tags="&lt;convolutional-neural-networks&gt;&lt;hardware&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9050" PostTypeId="1" AcceptedAnswerId="9052" CreationDate="2018-11-19T00:11:07.600" Score="5" ViewCount="278" Body="&lt;p&gt;I did some self-study to learn Neural network, object detection, and deep learning. Now I started implementing YOLOv3. I am looking for some website that I can communicate with people and make friends who are learning and implementing deep learning algorithms like me. &#xA;It will speed up my learning since self-studying gets boring sometimes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if you could introduce some website that I can find peoples like myself.&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="1671" LastEditDate="2018-11-19T17:16:52.537" LastActivityDate="2021-05-16T19:50:30.040" Title="Where can I discuss with deep learning beginners?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;resource-request&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-23T02:02:46.823" ContentLicense="CC BY-SA 4.0" />
  <row Id="9051" PostTypeId="2" ParentId="9047" CreationDate="2018-11-19T00:35:27.110" Score="1" Body="&lt;p&gt;Intel's Movidius uses digital circuitry for signal paths.   It is a parallel processing VLSI chip that can host a sequence of convolution layers and other types of parallel operations, just as CUDA cores in NVidia products do.  These all are extensions of the idea of DSPs (digital signal processors).  They are adept as front ends to real time vision systems in robots and vehicle automation.  They are also stuffed into racks and used for massively parallel operations in data centers, leveraged for large batch or mini-batch jobs related to indexing or supporting transaction based services.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three broad signal theory characteristics referenced conceptually in this question which may benefit from some disambiguation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Analog versus digital&lt;/li&gt;&#xA;&lt;li&gt;Deterministic versus stochastic&lt;/li&gt;&#xA;&lt;li&gt;Predictable versus unpredictable outcome&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of the eight permutations, any of these five can be realized or reasonably accurately approximated.  Under what conditions they are repeatable is in parentheses.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Analog, deterministic, predictable outcome &amp;mdash; Yes, within the bounds of specifications such as signal to noise ratio&lt;/li&gt;&#xA;&lt;li&gt;Analog, deterministic, unpredictable outcome &amp;mdash; Usually no&lt;/li&gt;&#xA;&lt;li&gt;Analog, stochastic, unpredictable &amp;mdash; No&lt;/li&gt;&#xA;&lt;li&gt;Digital, deterministic, predictable outcome &amp;mdash; Yes&lt;/li&gt;&#xA;&lt;li&gt;Digital, stochastic, unpredictable &amp;mdash; No&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The commercially available ICs used for artificial neural networks mentioned above are digital, but they may not be deterministic.  For instance, the seed of random number generators can be the system time.  When embedded into larger computing systems entropy can also be collected digitally from mouse movement, microphones, camera frames, or secure network sources and then used in the digital signal processing algorithms, including artificial network ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Future chips may actually return to analog processing for some of the workload.  The accuracy of digital circuitry may be worth sacrificing in exchange for increase complexity and throughput per unit area on the VLSI substrate.  The storage and retrieval advantage of flip flops and other digital structures can still be maintained by interfacing the storage to the forward signal paths with A-to-D and D-to-A conversion. It is difficult to predict whether the trade-off will gain traction and, if so, when.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term, &quot;Balanced edge case,&quot; references what mathematicians familiar with chaos theory sometimes call initial input sensitivity, colloquially known as, &quot;The Butterfly Effect.&quot;  Chaos can occur in nature where the only discrete operations are at the quantum level.  Chaos can also occur in digital systems with no greater mathematical complexity than the Logistic Equation: &lt;span class=&quot;math-container&quot;&gt;$x_{t+1} = r \, x_t (1 - x_t)$&lt;/span&gt;, where interesting behavior is observed for values of &lt;span class=&quot;math-container&quot;&gt;$r &amp;gt; 2.0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In electrical power engineering, much of what appears on an oscilloscope in triphase circuits are not sine waves but chaos that exhibits a strong peak in the power spectrum at 50 or 60 Hertz.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-19T00:35:27.110" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9052" PostTypeId="2" ParentId="9050" CreationDate="2018-11-19T06:41:34.547" Score="2" Body="&lt;p&gt;I highly recommend to get a local peer group. &lt;a href=&quot;https://www.meetup.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Meetups&lt;/a&gt; might be a good way to get to know some people. As a student at university you also have a lot of other options like going to lectures, creating a student group, organizing a local AI event / hackathon, ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not possible for you, here is what you can do online:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reddit: &lt;a href=&quot;https://www.reddit.com/r/deeplearning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;r/deeplearning&lt;/a&gt; seems to have a couple of discussions. &lt;a href=&quot;https://reddit.com/r/MachineLearning&quot; rel=&quot;nofollow noreferrer&quot;&gt;r/MachineLearning&lt;/a&gt; is cool if you want to discuss super recent stuff, but I would not say it is beginner level.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://plus.google.com/communities/112866381580457264725&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google+&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Coursera: if you take one of their courses, you will see a forum for discussions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-19T10:55:21.680" LastActivityDate="2018-11-19T10:55:21.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9054" PostTypeId="2" ParentId="9050" CreationDate="2018-11-19T09:44:01.600" Score="1" Body="&lt;p&gt;In my opinion, computer science in general, not only artificial intelligence, is learned by doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, You work on your project, you face problems (like memory management, security (yes there are security issues in AI), neural network training speed or efficiency). You post your code snippets here on stackexchange, and people will help you out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would be considered as a technical discussion, and &lt;strong&gt;I think it is more important than talking broadly about a subject that is meant to be very specific and detailed.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others mentioned, you can find some good communities on reddit, and in different AI related topics, and people there are mostly geeks who would spend hours talking about the subject. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, if you do not find what you need, go ahead and create your new community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also another option, &lt;strong&gt;yet one that I recommend&lt;/strong&gt;, create a personal website, whenever you learn something new or have a new insight, write an article and make it possible for people to add comments and discuss the subject at hand. &lt;strong&gt;Always archive what you learn, your website will be like a reference or archive where you organize your knowledge&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another option I recommend&lt;/strong&gt; You actually can consider creating a medium.com account for this, and start writing about what you know right now, then you will start to make loyal readers with whom you can discuss whatever comes to mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you want to study with other people, then here is an idea : search for beginners with ai related profile on linkedin , and send some messages asking them to join you, I do not think most beginners, who also want to study and learn more, will refuse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; : And I forgot github.com or gitlab.com. Where people can view your code, and even work with you as a team. I highly recommend gitlab as you can create private repositories for free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among others, and once you find 2 or 3 interested people on linkedin, you can work together on skype, google hangouts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notice&lt;/strong&gt; I do not recommend facebook, I personally tried it to chat with a team of developers, and it ended up being an element of distraction, none was really focused on the subject at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wish you the best.&lt;/p&gt;&#xA;" OwnerUserId="12113" LastEditorUserId="12113" LastEditDate="2018-11-19T10:03:31.053" LastActivityDate="2018-11-19T10:03:31.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9059" PostTypeId="1" CreationDate="2018-11-19T14:37:27.787" Score="2" ViewCount="24" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hebbian_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hebb's postulate&lt;/a&gt; attempts to explain associative learning via the processes of sampling (using sensors), emitting responses and receiving feedback. This is a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Control_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;control orientated architecture&lt;/a&gt; using an environment and biological systems. Is this the progenitor of AI?&lt;/p&gt;&#xA;" OwnerUserId="19204" LastEditorUserId="1671" LastEditDate="2018-11-19T18:04:33.333" LastActivityDate="2018-11-19T18:04:33.333" Title="Is Hebbian learning the progenitor of AI?" Tags="&lt;machine-learning&gt;&lt;architecture&gt;&lt;biology&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9062" PostTypeId="1" AcceptedAnswerId="9143" CreationDate="2018-11-19T16:06:10.987" Score="4" ViewCount="112" Body="&lt;p&gt;I read about the hill climbing algorithms, the simulating annealing algorithm, but I am confused. What is the basic purpose of local search methods?&lt;/p&gt;&#xA;" OwnerUserId="19442" LastEditorUserId="2444" LastEditDate="2019-03-02T11:11:48.373" LastActivityDate="2019-03-02T11:12:18.327" Title="What is the basic purpose of local search methods?" Tags="&lt;optimization&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;simulated-annealing&gt;&lt;local-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9063" PostTypeId="5" CreationDate="2018-11-19T17:51:53.590" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-19T17:51:53.590" LastActivityDate="2018-11-19T17:51:53.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9064" PostTypeId="4" CreationDate="2018-11-19T17:51:53.590" Score="0" Body="Use for questions about online AI resources." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-19T17:51:53.590" LastActivityDate="2018-11-19T17:51:53.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9065" PostTypeId="1" CreationDate="2018-11-19T20:03:25.013" Score="2" ViewCount="49" Body="&lt;p&gt;I was watching a video which tells a bit about reinforcement learning, and I learnt that If the robot makes wrong movement then they train the network with negative learning rate. From this method, something came to my mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is &quot;Can I use a wrong data to train a neural network?&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate the method, I'll be using the eye tracker project that I'm working on right now. In my project There are photos and the points that corresponds the locations that I m looking to at that photo. Its like grid (9, 16). If I look to the middle of the screen, it means the output is (4, 7.5). if I look left up side of the screen it means (0,0). Normally for a photo that I'm looking into the middle, we use that photo as input and (4, 7.5) as output to train network using positive learning rate. Now let me rephrase the question. Can I train a model giving a photo that I'm looking into the middle as input and (0,0) as output(label) using negative learning rate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you, If I made a mistake against the rules of stackoverflow, I'm so sorry. I'll be waiting your valuable answers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: this is a conversation between me and someone from stackoverflow, I'll let you read, hope you get a point.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&gt; Yes, you can. But, what would be the reason of passing a wrong ground truth to your training process? – Neb 14 hours ago &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&gt; If I have no various data to train, I can create more data via this method to increase the certainty when I use squared error loss. But I have doubts about this method. for example lets assume we have a photo named 'X' and its label is (5,5). at first epoch, Let the model gives (2,2) for photo 'X'. if I try to train network with a photo X and label -&gt; (4,4) using negative learning rate, it might send away the point from (2,2) to (1,1) whereas we expect it to send the point (2,2) to (5,5). Did you get what I meant? – Faruk Nane 14 hours ago   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&gt; You are right. Using a negative learning rate and a wrong ground truth will not necessarly make the learning process converge to the optimal value for your net's parameters – Neb 13 hours ago &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&gt; So can I say that &quot;when I'm sure that the absolute error for each case is less than 2, I can use this method using points away 2 units.&quot; So It'll make the outputs closer to the target point. I don't really know if we can easily say that. because we consider this method as if there are only 2 parameters which is the output point. However a model has many parameters so It might affect so differently. My brain is so confused. I think this might be an academic work, right? – Faruk Nane 13 hours ago   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&gt; Well, it is difficult to suggests you the path to follow without knowing the exact specifics of your problem. In any case, if you're trying to solve this problem for fun or self-improvement, I'd suggest you to experiment with the solutions you came up with and see if they works. – Neb 13 hours ago &lt;/p&gt;&#xA;&#xA;&lt;p&gt;//EDIT: UP UP&lt;/p&gt;&#xA;" OwnerUserId="16864" LastEditorUserId="16864" LastEditDate="2018-11-20T16:10:10.293" LastActivityDate="2018-11-20T16:10:10.293" Title="Setting learning rate as negative number for wrong train cases" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9076" PostTypeId="1" AcceptedAnswerId="9079" CreationDate="2018-11-20T15:25:44.293" Score="5" ViewCount="5123" Body="&lt;p&gt;In the context of &lt;em&gt;evolutionary computation&lt;/em&gt;, in particular &lt;em&gt;genetic algorithms&lt;/em&gt;, there are two stochastic operations &quot;mutation&quot; and &quot;crossover&quot;. What are the differences between them?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2020-11-28T03:09:28.910" LastActivityDate="2020-12-06T18:01:11.167" Title="What is the difference between &quot;mutation&quot; and &quot;crossover&quot;?" Tags="&lt;comparison&gt;&lt;evolutionary-algorithms&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;&lt;genetic-operators&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9077" PostTypeId="1" AcceptedAnswerId="9083" CreationDate="2018-11-20T15:34:16.527" Score="1" ViewCount="224" Body="&lt;p&gt;In a genetic algorithm, there are different steps. One of those steps is the selection of chromosomes for reproduction. What are the available selection strategies in genetic algorithms?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2020-12-06T20:09:24.553" LastActivityDate="2020-12-06T23:53:13.140" Title="What are the available selection methods in genetic algorithms?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;selection-operators&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9079" PostTypeId="2" ParentId="9076" CreationDate="2018-11-20T15:53:58.287" Score="5" Body="&lt;p&gt;The &lt;strong&gt;mutation&lt;/strong&gt; is an operation that is applied to a single individual in the population. It can e.g. introduce some noise in the chromosome. For example, if the chromosomes are binary, a mutation may simply be the flip of a bit (or gene).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;crossover&lt;/strong&gt; is an operation which takes as input two individuals (often called the &amp;quot;parents&amp;quot;) and somehow combines their chromosomes, so as to produce usually two other chromosomes (the &amp;quot;children&amp;quot;), which inherit, in a certain way, the genes of both parents.&lt;/p&gt;&#xA;&lt;p&gt;For more details about these operations, you can use the book &lt;a href=&quot;http://www2.fiit.stuba.sk/%7Ekvasnicka/Free%20books/Goldberg_Genetic_Algorithms_in_Search.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithms in Search, Optimization, and Machine Learning&lt;/a&gt; by &lt;a href=&quot;https://scholar.google.com/citations?user=BUzKxsoAAAAJ&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;David E. Goldberg&lt;/a&gt; (who is an expert in genetic algorithms and was &lt;em&gt;just&lt;/em&gt; advised by John H. Holland). You can also take a look at the book &lt;a href=&quot;https://papers.harvie.cz/unsorted/computational-intelligence-an-introduction.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Intelligence: An Introduction&lt;/a&gt; (2nd edition, 2007) by &lt;a href=&quot;https://ieeexplore.ieee.org/author/37276400500&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andries P. Engelbrecht&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T18:01:11.167" LastActivityDate="2020-12-06T18:01:11.167" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9081" PostTypeId="1" AcceptedAnswerId="9172" CreationDate="2018-11-20T16:54:42.003" Score="0" ViewCount="215" Body="&lt;p&gt;What are the many ways that artificial intelligence robots protect their existence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isaac Asimov's &quot;Three Laws of Robotics&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robot may not injure a human being or, through inaction, allow a human being to come to harm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robot must obey orders given it by human beings except where such orders would conflict with the First Law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.&lt;/p&gt;&#xA;" OwnerUserId="20068" LastEditorUserId="1671" LastEditDate="2018-11-20T20:45:30.773" LastActivityDate="2018-11-26T03:48:28.027" Title="How does a robot protect its own existence" Tags="&lt;mythology-of-ai&gt;&lt;asimovs-laws&gt;&lt;survival&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-23T04:34:41.167" ContentLicense="CC BY-SA 4.0" />
  <row Id="9083" PostTypeId="2" ParentId="9077" CreationDate="2018-11-20T17:50:20.090" Score="3" Body="&lt;h3&gt;Overview&lt;/h3&gt;&#xA;&lt;p&gt;There are &lt;strong&gt;many selection operators&lt;/strong&gt; (or methods) for evolutionary algorithms. These selection methods differ in the way they evaluate the individuals given their fitness or how they compute their fitness in the first place. Some selection operators select individuals &lt;strong&gt;deterministically&lt;/strong&gt;, while others select them &lt;strong&gt;stochastically&lt;/strong&gt;, which can prevent premature convergence, given that, for instance, in &lt;strong&gt;fitness proportionate selection&lt;/strong&gt; (aka &lt;em&gt;roulette wheel selection&lt;/em&gt;), a well-known selection operator, all individuals can be selected, although the fitter individuals have a higher probability of being selected. Each selection operator has different advantages and disadvantages compared to other selection operators.&lt;/p&gt;&#xA;&lt;p&gt;All these details are covered from chapter 22 (p. 166 and 203 of the linked pdf) onwards of the well-written book &lt;a href=&quot;https://digi.lib.ttu.ee/services/copycat-pdf.php?ID=248&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Computation 1 Basic Algorithms and Operators&lt;/a&gt; by Thomas Bäck et al. These concepts are also described in &lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Intelligence: An Introduction&lt;/a&gt; (section 8.5, p. 135), in case you want to have an alternative reading.&lt;/p&gt;&#xA;&lt;p&gt;For completeness, I will enumerate and briefly describe some of the well-known selection operators.&lt;/p&gt;&#xA;&lt;h3&gt;Fitness proportionate selection (FPS)&lt;/h3&gt;&#xA;&lt;p&gt;This selection method is also known as &lt;strong&gt;roulette wheel selection (RWS)&lt;/strong&gt;. In FPS, the probability of an individual being selected is proportional to its fitness. More precisely, let &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt; be the fitness of individual &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, then the probability of that individual being selected is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p_{i}={\frac {f_{i}}{\Sigma _{j=1}^{N}f_{j}}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;FPS does not handle minimization problems directly and it requires the fitnesses to be non-negative (given that probabilities are in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;), but the minimization problem can be transformed into a maximization one. The solutions are thus selected &lt;strong&gt;probabilistically&lt;/strong&gt; in this case. There are other problems/issues with this selection method, which are described in the cited book (chapter 23, p. 172).&lt;/p&gt;&#xA;&lt;h3&gt;Tournament selection (TS)&lt;/h3&gt;&#xA;&lt;p&gt;In TS, at each generation, a set/group &lt;span class=&quot;math-container&quot;&gt;$Q = \{ a_1, \dots, a_q \}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$|Q| = q$&lt;/span&gt; individuals is sampled (with or without replacement) &lt;strong&gt;at random&lt;/strong&gt; from the current population of all individuals &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. Successively, this group &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; takes part in a so-called &lt;strong&gt;tournament&lt;/strong&gt;, i.e. a loop of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; iterations, where, for each iteration &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, the individual from the group &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; with the highest fitness &lt;span class=&quot;math-container&quot;&gt;$a_1'$&lt;/span&gt; is selected (deterministically or stochastically) and inserted in a new set &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, which will contain the individuals that you will mutate and recombine. After this first iteration &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; contains only one individual, &lt;span class=&quot;math-container&quot;&gt;$S = \{a_1' \}$&lt;/span&gt;. In the second iteration, we perform the same thing, and obtain &lt;span class=&quot;math-container&quot;&gt;$S = \{a_1', a_2' \}$&lt;/span&gt;, and so on. So, after &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; iterations, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; contains &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; individuals, i.e. &lt;span class=&quot;math-container&quot;&gt;$|S| = \lambda$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$a_1'$&lt;/span&gt; is not necessarily equal to &lt;span class=&quot;math-container&quot;&gt;$a_1 \in Q$&lt;/span&gt;. The individuals in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; are the ones that undergo mutation and cross-over in &lt;em&gt;this&lt;/em&gt; generation. In the next generation, you repeat this &lt;em&gt;tournament&lt;/em&gt; process for selecting the best &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; individuals. Typically, you need to have at least &lt;span class=&quot;math-container&quot;&gt;$q = 2$&lt;/span&gt; (i.e. &lt;strong&gt;binary tournament selection&lt;/strong&gt;), so that you have the opportunity to choose 2 different individuals for the cross-over. If &lt;span class=&quot;math-container&quot;&gt;$q=1$&lt;/span&gt;, individuals are picked randomly from the population (given that you do not perform any tournament at all).&lt;/p&gt;&#xA;&lt;h3&gt;Rank-based selection (RS)&lt;/h3&gt;&#xA;&lt;p&gt;In RS, only the &lt;strong&gt;rank ordering&lt;/strong&gt; of the individuals within the current population determines the probability of being selected. So, in RS, an individual has a probability of being selected &lt;strong&gt;not&lt;/strong&gt; proportional to its fitness (as in FPS), but it is based on its rank (order) with respect to the other individuals. So, if the individual &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; has is ranked &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;st and individual &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is ranked &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;nd, even if &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; had a fitness a lot higher than individual &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, then its probability of being selected would not be a lot higher than the probability of selecting &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, while, in FPS, this would be the case. There are different ways of computing the rank, such as linear or non-linear. The details of how the rank can be computed are described in chapter 25 of the cited book.&lt;/p&gt;&#xA;&lt;h3&gt;Other selection methods&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Boltzmann selection (chapter 26)&lt;/li&gt;&#xA;&lt;li&gt;Soft brood selection (chapter 27)&lt;/li&gt;&#xA;&lt;li&gt;Disruptive selection (chapter 27)&lt;/li&gt;&#xA;&lt;li&gt;Competitive selection (chapter 27)&lt;/li&gt;&#xA;&lt;li&gt;Truncation selection (chapter 29)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Other related concepts&lt;/h3&gt;&#xA;&lt;p&gt;There's also the related concept of &lt;strong&gt;elitism&lt;/strong&gt;, where the best individuals of the current population are carried over to the next population/generation, and &lt;a href=&quot;https://deap.readthedocs.io/en/0.7-a1/halloffame.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;hall of fame&lt;/strong&gt;&lt;/a&gt;, which is a data structure that stores the best individual for each generation.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T23:53:13.140" LastActivityDate="2020-12-06T23:53:13.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9085" PostTypeId="1" CreationDate="2018-11-20T21:10:18.993" Score="3" ViewCount="116" Body="&lt;p&gt;I am using PPO with an LSTM agent. My agent is performing 10 actions for each episode, one action is corresponding to one LSTM timestep and the action space is discrete. I have only one reward per episode which I can compute after the last action of the episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each timestep (~ action) my agent has 20 choices. The following plot shows the reward (y-axis) versus the current episode (x-axis).&#xA;The plot shows a decreasing reward because I want to mimize this reward so I use: minus of the true reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the beginning of the process, the agent seems to learn very well and the reward is decreasing but then it's converging to a value which is not the best. When I look at the results of my experiment it appears that the index of all actions are same (for example the agent is always choosing the second value of my discrete action space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have an idea about what is happening here ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1wdcR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1wdcR.png&quot; alt=&quot;reward versus episode&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20079" LastActivityDate="2018-11-20T21:10:18.993" Title="Why are all the actions converging to the same index?" Tags="&lt;reinforcement-learning&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9089" PostTypeId="1" CreationDate="2018-11-21T05:00:25.957" Score="2" ViewCount="91" Body="&lt;p&gt;In the exercise &lt;a href=&quot;http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=MachineLearning&amp;amp;doc=exercises/ex3/ex3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exercise 3: Multivariate Linear Regression&lt;/a&gt;, by Andrew Ng, the author suggests to &quot;scale both types of inputs by their standard deviations and set their means to zero&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_{n e w}=\frac{x-\mu}{\sigma}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Method 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The author provides the following Matlab (and Octave) code to scale the inputs.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [ones(m, 1), x];&#xA;sigma = std(x);&#xA;mu = mean(x);&#xA;x(:,2) = (x(:,2) - mu(2))./ sigma(2);&#xA;x(:,3) = (x(:,3) - mu(3))./ sigma(3);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Method 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;But why not simply scale the inputs between zero and one, or, divide by the maximum?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_range=max(x)&#xA;x(:,2) = (x(:,2)/x_range(2));&#xA;x(:,3) = (x(:,3)/x_range(3));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have done the exercise with method 2 and these are the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fdhSG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fdhSG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Is there a computational advantage with the first method over the second method?&lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="2444" LastEditDate="2019-12-08T15:04:25.030" LastActivityDate="2019-12-08T15:04:25.030" Title="Which of these two numerical methods for z-score normalisation is preferable, in multivariate linear regression?" Tags="&lt;implementation&gt;&lt;data-preprocessing&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9090" PostTypeId="2" ParentId="9076" CreationDate="2018-11-21T05:54:56.223" Score="0" Body="&lt;p&gt;Evolutionary algorithms use it in a very similar way as the two terms are used in biology:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In biology, a &lt;strong&gt;mutation&lt;/strong&gt; is the permanent alteration of the nucleotide sequence of the genome of an organism, virus, or extrachromosomal DNA or other genetic elements.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Mutation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Chromosomal crossover&lt;/strong&gt; [...] is the exchange of genetic material [...] that results in recombinant chromosomes during sexual reproduction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Chromosomal_crossover&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence the main difference is that mutations happen within one individual while crossover is between two individuals.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-21T06:14:05.953" LastActivityDate="2018-11-21T06:14:05.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9091" PostTypeId="2" ParentId="6875" CreationDate="2018-11-21T06:42:24.283" Score="1" Body="&lt;p&gt;The purpose of rescaling  gradient descent problems is to reframe the problem for quicker convergence / calculation of linear coefficient parameters.  in the &lt;a href=&quot;http://openclassroom.stanford.edu/MainFolder/VideoPage.php?course=MachineLearning&amp;amp;video=03.1-LinearRegressionII-FeatureScaling&amp;amp;speed=100&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford video&lt;/a&gt; series, Andrew Ng provides a intuitive explanation and enables one to hone an intuitive understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multivariate input regression gradient descent converges faster when the inputs are in the same order of magnitude.  For example, if predicting house prices based on X1= the number of rooms and X2= area of the home in square feet.  X1 is on scale of 0-6 bedrooms and and X2 is typically 1000-3000 square feet.  Given the diffence in magnitude, this problem is a good candidate for feature scaling.&lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="18819" LastEditDate="2018-11-21T06:48:18.023" LastActivityDate="2018-11-21T06:48:18.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9092" PostTypeId="2" ParentId="9089" CreationDate="2018-11-21T07:56:02.920" Score="1" Body="&lt;p&gt;No, there is no computational advantage of the second method over the first, if you neglect the computational requirements for the calculation of &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We generally use the first method for better results. This is because if you separate your dataset into train and test data, then you may normalise the train data perfectly between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; by taking &lt;span class=&quot;math-container&quot;&gt;$max$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$min$&lt;/span&gt; from the train dataset. Your algorithm will only see data between &lt;span class=&quot;math-container&quot;&gt;$0-1$&lt;/span&gt;, but there is no way to ensure normalising the test data on the same &lt;span class=&quot;math-container&quot;&gt;$max$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$min$&lt;/span&gt; will result in the data being between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a better way is to use the Gaussian Distribution method, as the only assumption it makes is the data distribution is Gaussian which is just about true for any phenomenon (after some functional transformations which might be handled by the ML algorithm). I answered a similar &lt;a href=&quot;https://datascience.stackexchange.com/questions/33846/normalizing-test-data/33847#33847&quot;&gt;question&lt;/a&gt; here.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-11-21T07:56:02.920" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9096" PostTypeId="1" AcceptedAnswerId="9100" CreationDate="2018-11-21T13:33:32.310" Score="2" ViewCount="689" Body="&lt;p&gt;I'm trying to train a DQN, so I'm using OpenAI gym and Breakout (&lt;a href=&quot;https://gym.openai.com/envs/Breakout-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Breakout-v0&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have altered the reward supplied by the environment: If the episode is not completed fully, the agent gets a -10 reward. Could be this counterproductive for learning?&lt;/p&gt;&#xA;" OwnerUserId="9818" LastEditorUserId="1847" LastEditDate="2018-11-21T19:55:55.390" LastActivityDate="2018-11-21T20:44:30.470" Title="DQN Breakout adding an extra negative reward to help training?" Tags="&lt;ai-design&gt;&lt;training&gt;&lt;game-ai&gt;&lt;dqn&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9097" PostTypeId="1" CreationDate="2018-11-21T14:44:26.623" Score="2" ViewCount="54" Body="&lt;p&gt;If we take the vanilla variational auto-encoder (VAE), we &lt;span class=&quot;math-container&quot;&gt;$p(z)$&lt;/span&gt; is a Gaussian distribution with zero mean and unit variance and we approximate &lt;span class=&quot;math-container&quot;&gt;$p(z|x) \approx q(z|x)$&lt;/span&gt; to be a Gaussian distribution as well, for each latent variable &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is a discrete variable? What kind of distributions can be used to model discrete latent variables? For example, what kind of distribution can be used to model &lt;span class=&quot;math-container&quot;&gt;$p(z)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(z|x) \approx q(z|x)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="18312" LastEditorUserId="2444" LastEditDate="2018-11-24T04:32:08.133" LastActivityDate="2018-11-24T04:32:08.133" Title="What kind of distributions can be used to model discrete latent variables?" Tags="&lt;machine-learning&gt;&lt;generative-model&gt;&lt;random-variable&gt;&lt;probability-distribution&gt;&lt;latent-variable&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9098" PostTypeId="1" AcceptedAnswerId="9099" CreationDate="2018-11-21T16:50:11.150" Score="3" ViewCount="131" Body="&lt;p&gt;I've just started studying genetic algorithms and I'm not able to understand why a genetic algorithm can improve if, at each learning, the 'world' that the population encounters change. For example, in this demo (&lt;a href=&quot;http://math.hws.edu/eck/js/genetic-algorithm/GA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://math.hws.edu/eck/js/genetic-algorithm/GA.html&lt;/a&gt;), it's pretty clear to me that the eating statistics will improve every year if bunches of grass grow exactly in the same place, but instead they always grow in different positions and I can't figure out how it  can be useful to evaluate (through the fitness function) the obtained eating stats given that the next environment will be different.&lt;/p&gt;&#xA;" OwnerUserId="20106" LastActivityDate="2020-12-08T14:59:35.550" Title="How can a genetic algorithm adapt and get better in a changing environment?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;fitness-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9099" PostTypeId="2" ParentId="9098" CreationDate="2018-11-21T18:02:22.970" Score="1" Body="&lt;p&gt;There are a couple of ways of dealing with this. Very often, the approach is just to design your representation and operators in a way to account for the fact that the world changes. The idea is to give the algorithm something that can be used to learn general behaviors or solutions rather than specific ones.&lt;/p&gt;&#xA;&lt;p&gt;Take an example of learning to steer a race car around a track. You want to represent the state of the world and have the GA learn to select an appropriate action. You might choose to represent the state of the world as a vector of &lt;span class=&quot;math-container&quot;&gt;$[x, y, v, a]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; is your location on the track, &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is your current velocity vector, and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is an acceleration to apply. The fitness function could return how &amp;quot;good&amp;quot; it was to apply that acceleration. If you do this, your algorithm can probably learn to navigate this track, but a different track will be hopeless, as the locations aren't corresponding to the same turn locations on the new track.&lt;/p&gt;&#xA;&lt;p&gt;However, what if you encode the world as &lt;span class=&quot;math-container&quot;&gt;$[s, v, a]$&lt;/span&gt;, where instead of an &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; pair representing your current position, you have &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; as a vector of sensor readings? Is there a wall coming up in front of you or is the track starting to bank? Now, your algorithm can learn to be more general. It doesn't need to be the exact track it learned on, because what it's learning is not to brake at a specific point, but to brake when it detects a wall coming up.&lt;/p&gt;&#xA;&lt;p&gt;I didn't do a lot of digging into the example you linked, but if you run it for a few years, you can see evidence of this. You see agents that appear to have learned to move in lines horizontally or vertically until they encounter a green square, and then they'll stick around and eat in the patches around that square. That behavior is general, because every environment it encounters has lots of blank space with clusters of green. It's not learning &amp;quot;go to square &lt;span class=&quot;math-container&quot;&gt;$(20, 30)$&lt;/span&gt;&amp;quot;. It's learning to move in a pattern until it finds green and then move around that location.&lt;/p&gt;&#xA;&lt;p&gt;You can do this in a lot of cases where the specific environment can change, but the objective is the same. There are problems where the actual fitness function changes over time, however. For those problems, there are specific techniques to deal with dynamic fitness functions. Generally, this involves doing something to maintain diversity so that your whole population isn't getting stuck on whatever the current &amp;quot;best&amp;quot; looks like. That's a bit more advanced topic though, and I think your question was really more about the former type of problems.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastEditorUserId="2444" LastEditDate="2020-12-08T14:59:35.550" LastActivityDate="2020-12-08T14:59:35.550" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9100" PostTypeId="2" ParentId="9096" CreationDate="2018-11-21T20:16:22.870" Score="2" Body="&lt;p&gt;In general, the approach of adding or altering the reward structure of a RL environment, when you are still trying to create an agent that solves the original problem, is called &lt;a href=&quot;https://www.youtube.com/watch?v=xManAGjbx2k&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reward Shaping&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reward Shaping can be tricky to get right. In your case I think it may be counter-productive. The main issue is that the agent does not know the current time step as part of the state. So from its perspective, a negative reward for &quot;timing out&quot; actually looks like a random chance of getting that same reward in the &lt;em&gt;kind of states that happen later in the game&lt;/em&gt;. The effect may be small enough chance that it doesn't really matter (after all it still gets positive rewards for hitting bricks). However, it may appear as high variance on certain game positions, making the agent learn to avoid some of them for no good reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the aim here is to try and get the agent to speed up and finish an episode, a simpler trick might be to reduce the discount factor (e.g. from &lt;span class=&quot;math-container&quot;&gt;$0.999$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0.99$&lt;/span&gt;). This will cause the agent to focus on getting more short-term rewards, at the expense of long-term planning. In some environments this could be a problem of a different kind, but when the rewards are not sparse and there are not any special high-reward states that need extended setup, it should be OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the aim is to punish losing a &quot;game life&quot; more severely, then this is less likely to cause a problem (because the state will clearly show what to expect), although it may change what the optimal behaviour is, or how it is approached. In general the impact would be reduced risk-taking, and even though Breakout is a deterministic game, the agent is presented with a stochastic environment because each action lasts a random number of frames (2-4). With a strong penalty for losing a life, I think that the agent will be less likely to try to get high bounce angles by hitting with extreme edge of the bat. Note that you don't &lt;em&gt;need&lt;/em&gt; to punish the agent for losing a life, it learns that because the episode ends and it cannot score any more points. Value functions are all about predicting future rewards, so the value of any life-losing state is always &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have more time, then there are also several extensions to basic DQN that require less sampled experience to learn an environment, or improve the resulting policies. A good source for those ideas is the paper &lt;a href=&quot;https://arxiv.org/abs/1710.02298&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rainbow: Combining Improvements in Deep Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-21T20:44:30.470" LastActivityDate="2018-11-21T20:44:30.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9102" PostTypeId="1" CreationDate="2018-11-22T00:03:37.663" Score="1" ViewCount="27" Body="&lt;p&gt;Say I have a batch of examples, each examples represent a state:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0.1, 0.2, 0.5] #1st example&#xA;[0.4, 0.0, 0.3] #2nd example &#xA;..........&#xA;[0.1, 0.1, 0.1] #16th example&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I feed through the NN, and then the NN predict the following class:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[move up]   #1st example&#xA;[move down] #2nd example&#xA;........&#xA;[move left] #16th example&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then I take the square loss (which calculated to be 0.1 after taking average over 16 examples), and do backward propagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can I assume that each of these examples will assign (or contribute) to a 0.1 loss?&lt;/p&gt;&#xA;" OwnerUserId="20113" LastActivityDate="2018-11-23T01:49:41.773" Title="Training by one batch of examples, what does it mean" Tags="&lt;training&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9103" PostTypeId="1" CreationDate="2018-11-22T00:26:12.780" Score="5" ViewCount="138" Body="&lt;p&gt;This question assumes a definition of AI based on machine learning, and was inspired by this fun Technology Review post:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pRyOW.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pRyOW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/612404/is-this-ai-we-drew-you-a-flowchart-to-work-it-out/?utm_campaign=owned_social&amp;amp;utm_source=twitter.com&amp;amp;utm_medium=social&quot; rel=&quot;noreferrer&quot;&gt;&lt;sub&gt;SOURCE: Is this AI? We drew you a flowchart to work it out (Karen Hao, MIT Technology Review)&lt;/sub&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the definition of artificial intelligence has been a continual subject of discussion on this stack, I wanted to bring it to the community for perspectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formal question here is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is an algorithm that is no longer actively learning an AI?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Specifically, can applied algorithms that are not actively learning be said to reason?  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-22T01:16:53.380" LastActivityDate="2018-11-28T22:50:36.880" Title="Is an algorithm that is no longer actively learning an AI?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;terminology&gt;&lt;theory&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9104" PostTypeId="1" CreationDate="2018-11-22T01:19:33.413" Score="3" ViewCount="999" Body="&lt;p&gt;I don't know much about AI or chess engines, but what is the fundamental difference between AlphaZero and Stockfish or Rybka?&lt;/p&gt;&#xA;" OwnerUserId="17601" LastActivityDate="2018-11-22T09:41:18.810" Title="How is AlphaZero different from Stockfish or Rybka?" Tags="&lt;chess&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9105" PostTypeId="1" CreationDate="2018-11-22T04:53:35.007" Score="7" ViewCount="2446" Body="&lt;p&gt;In genetic algorithms, a function called &quot;fitness&quot; (or &quot;evaluation&quot;) function is used to determine the &quot;fitness&quot; of the chromosomes. Creating a good fitness function is one of the challenging tasks in genetic algorithms. How would you create a good fitness function?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2021-01-30T02:36:34.730" LastActivityDate="2021-01-30T02:36:34.730" Title="How to create a good fitness function?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;fitness-functions&gt;&lt;fitness-design&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9106" PostTypeId="1" AcceptedAnswerId="9140" CreationDate="2018-11-22T06:11:55.800" Score="3" ViewCount="3003" Body="&lt;p&gt;In evolutionary computation and, in particular, in the context of genetic algorithms, there is the concept of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;fitness function&lt;/em&gt;&lt;/a&gt;. The better a state, the greater the value of the fitness function for that state.&lt;/p&gt;&#xA;&lt;p&gt;What would be a good fitness function for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Eight_queens_puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;8-queens problem&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="19652" LastEditorUserId="2444" LastEditDate="2021-01-30T02:32:25.927" LastActivityDate="2021-01-30T02:34:28.230" Title="How to design a fitness function for the 8-queens problem?" Tags="&lt;genetic-algorithms&gt;&lt;fitness-functions&gt;&lt;evolutionary-computation&gt;&lt;fitness-design&gt;&lt;8-queens-problem&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9107" PostTypeId="2" ParentId="9104" CreationDate="2018-11-22T06:51:37.413" Score="3" Body="&lt;p&gt;Stockfish is free software (&lt;a href=&quot;https://github.com/official-stockfish/Stockfish&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt; while AlphaGo can only be used by employees at Deepmind. This is important, because it means all answers to this question can only rely on the AlphaGo paper. Some decisions during the matches (e.g. giving limited time per move and not per game; giving different computational power) lead to unclarity how good alpha go really is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Stockfish_(chess)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stockfish&lt;/a&gt;  uses Alpha-beta search with a hardcoded metric to evaluate how good a position is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, AlphaGo uses Monte Carlo tree search and convolutional neural networks. They are trained in an reinforcement learning setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaGo is promoted as being almost directly applicable to different settings (Go, Chess), while stockfish is only applied to Chess. The techniques of stockfish can also be applied to Go, but the problem is the board evaluation. For Chess it's easy compared to Go.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1712.01815&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/a&gt;, December 2017.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://deepmind.com/documents/119/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the Game of Go without Human Knowledge&lt;/a&gt; is about Alpha Go Zero&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-22T09:41:18.810" LastActivityDate="2018-11-22T09:41:18.810" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9108" PostTypeId="1" AcceptedAnswerId="9111" CreationDate="2018-11-22T08:36:32.940" Score="1" ViewCount="110" Body="&lt;p&gt;I'm looking at &lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/toy_text/blackjack.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this source code&lt;/a&gt;, where the reward is calculated with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reward = cmp(score(self.player), score(self.dealer))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why is it ok to calculate the reward based on a hidden state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A player only sees the dealer's first card.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.dealer[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20118" LastEditorUserId="2444" LastEditDate="2020-04-03T21:02:30.267" LastActivityDate="2020-04-03T21:03:04.793" Title="Why is it ok to calculate the reward based on a hidden state?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9110" PostTypeId="2" ParentId="9103" CreationDate="2018-11-22T09:21:46.700" Score="3" Body="&lt;p&gt;In my view learning is a useful aspect of an AI system, which makes it more resilient towards a changing environment, but it is not a requirement. Early AI systems (Expert systems, theorem solvers) would usually not use learning algorithms, but had a fixed knowledge base.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also works a lot better with &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical models&lt;/a&gt; than with symbolic processing; the proliferation of training data which boosts the most recent AI wave (which is based more on statistical and numerical methods) has made it somewhat necessary for modern systems to have a learning/training component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one needs to distinguish between the initial training, which leads to a production system which then becomes fixed (and will not learn from new input) and those which have a learning component built in as an intrinsic part and will continue to adapt throughout their lifetime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A cautionary example is &lt;a href=&quot;https://blogs.microsoft.com/blog/2016/03/25/learning-tays-introduction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft's &quot;Tay&quot; chatbot&lt;/a&gt;. This was still actively learning from conversations and was quickly abused by subjecting it to undesirable input. Switching off the learning component would have been prudent in this case, but it would not have taken the AI out of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer the question: learning is not a required part of an AI system once it is operational. While modern systems generally have some element of learning to build their initial models, older systems were simply hand-coded without any automated learning at all. But one would still regard them as AI system.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-11-22T09:21:46.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9111" PostTypeId="2" ParentId="9108" CreationDate="2018-11-22T09:49:58.210" Score="1" Body="&lt;p&gt;The code you reference is not part of the learning agent. It is part of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class BlackjackEnv(gym.Env):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If, as in this case, the environment is provided entirely by software simulation, it is absolutely necessary for it to include a full working model of all state transitions and rewards. That is independent of whether any hidden state makes the problem harder. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is it ok to calculate the reward based on a hidden state?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the case of Blackjack, this can be treated not as a hidden state that would affect the outcome if only known, but as randomness in the environment over which the agent has no control. Critically, the dealer has no options to behave differently depending on the unknown card, and the dealer's eventual score is entirely unaffected by the player's earlier choices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a subtle difference. If you applied the same environment rules to Poker, where an opponent could behave differently depending on this hidden knowledge, then a simple MDP model is not enough theory to result in an optimal solution. In that case, you would need to look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Partially Observable MDPs (POMDPs)&lt;/a&gt;. Note this would not affect reward calculation in the environment, just the choices of which agent type to use. If you are just learning RL, you probably don't know of any algorithms that could solve this yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, a lot of problems are somewhere between a classic MDP and a POMDP - they contain elements which, if the agent could know them, may allow it to achieve a higher expected reward. In many cases though, these elements can either be treated as random (as here in Blackjack) and thus the system is still theoretically an MDP, or they have a very small effect on the optimal policy, so can be ignored for practical purposes (e.g. think of all the physical details in a real cart pole balancing system - friction, temperature, flexing motions, etc).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-04-03T21:03:04.793" LastActivityDate="2020-04-03T21:03:04.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9117" PostTypeId="2" ParentId="9102" CreationDate="2018-11-23T01:49:41.773" Score="1" Body="&lt;p&gt;This would mean that there is only one gradient update on your dataset of 16 samples. If you are taking the mean of all the squared errors ( mse ), then the loss of each sample is contributing to the 0.1 loss.&lt;/p&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-11-23T01:49:41.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9121" PostTypeId="1" CreationDate="2018-11-23T13:25:40.337" Score="1" ViewCount="66" Body="&lt;p&gt;Suppose, for simplicity sake, to be in a discrete time domain with the action set being the same for all states &lt;span class=&quot;math-container&quot;&gt;$S \in \mathcal{S}$&lt;/span&gt;. Thus, in a finite Markov Decision Process, the sets &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; have a finite number of elements. We could then say the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(s',r | s,a) = P\{S_t=s',R_t=r | S_{t-1}=s,A_t=a\} ~~~ \forall s',s \in \mathcal{S}, r \in \mathcal{R} \subset \mathbb{R}, a \in \mathcal{A}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the function &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; defines the dynamics of the finite MDP and &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; defines the probability.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;How could I extend this to a general MDP? That is, an MDP where the sets &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; haven't a finite number of elements? To be more precise, in my case &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A} \subset \mathbb{R}^n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S} \subset \mathbb{R}^m$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R} \subset \mathbb{R}$&lt;/span&gt;. My thought is that the equation above is still true, however, the probability is zero for each tuple &lt;span class=&quot;math-container&quot;&gt;$s',r,s,a$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it sufficient to say that for finite MDP we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{s'\in\mathcal{S}}\sum_{r\in\mathcal{R}}p(s',r|s,a)=1 ~~~ \forall s\in\mathcal{S},a\in\mathcal{A}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while in non-finite MDP (supposing that the sets &lt;span class=&quot;math-container&quot;&gt;$\mathcal{s}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; are continuous) we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_{s'\in\mathcal{S}}\int_{r\in\mathcal{R}}p(s',r|s,a)=1 ~~~ \forall s\in\mathcal{S},a\in\mathcal{A}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or is it more complex than this?&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="16199" LastEditDate="2018-11-23T13:43:48.613" LastActivityDate="2018-11-23T13:43:48.613" Title="How to generalize finite MDP to general MDP?" Tags="&lt;markov-decision-process&gt;&lt;finite-markov-decision-process&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9123" PostTypeId="1" AcceptedAnswerId="9124" CreationDate="2018-11-23T14:24:55.930" Score="3" ViewCount="145" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Alpha zero paper&lt;/a&gt; says that the The first set of features are repeated for each position in a T = 8-step history. So what happens before the first 8 moves? Do they just repeat the starting position?&lt;/p&gt;&#xA;" OwnerUserId="19604" LastActivityDate="2018-11-23T16:00:43.560" Title="Alpha zero before move 8" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9124" PostTypeId="2" ParentId="9123" CreationDate="2018-11-23T16:00:43.560" Score="3" Body="&lt;p&gt;On page 13, right under Table S1 in the linked paper, this is explained (emphasis in bold at the end mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Each set of planes represents the board position at a time-step &lt;span class=&quot;math-container&quot;&gt;$t - T + 1, \dots, t$&lt;/span&gt;, &lt;strong&gt;and is set to zero for time-steps less than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I suspect the solution they write there would indeed work better than just repeating the starting position up to 8 times. Intuitively, you'll want the Neural Network to learn to primarily focus on the current game state. If the starting position is repeated a bunch of times in those planes, the Neural Network cannot distinguish between any of them in the learning process in the first few steps, and may start relying on them all equally. Only in later time steps will it &quot;figure out&quot; that they're sometimes not equal, and that the last one is probably the most informative one. If the &quot;useless planes&quot; during the first few steps are all-zero, they can be much more easily ignored in the start of the learning process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I suspect the difference really won't matter a whole lot at all, I suspect there'd just be a tiny difference in learning speed based on the intuition described above.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-11-23T16:00:43.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9125" PostTypeId="1" AcceptedAnswerId="9127" CreationDate="2018-11-23T16:09:42.080" Score="2" ViewCount="4294" Body="&lt;p&gt;In Constraint Satisfaction Problems (CSPs), a &lt;a href=&quot;https://www.cpp.edu/%7Eftang/courses/CS420/notes/CSP.pdf#page=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;state&lt;/a&gt; is any data structure that supports&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a successor function,&lt;/li&gt;&#xA;&lt;li&gt;a heuristic function, and&lt;/li&gt;&#xA;&lt;li&gt;a goal test.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In this context, what is a &lt;em&gt;successor function&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2021-02-17T18:17:33.610" LastActivityDate="2021-02-17T18:17:33.610" Title="What is a successor function (in CSPs)?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;search&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9126" PostTypeId="1" CreationDate="2018-11-23T16:26:03.107" Score="1" ViewCount="1190" Body="&lt;p&gt;Backtracking search is the basic uninformed search algorithm for constraint satisfaction problems (CSPs). How could we improve the efficiency of the backtracking search in CSPs?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2021-02-17T17:51:20.357" LastActivityDate="2021-02-17T17:51:20.357" Title="How to improve the efficiency of the backtracking search in CSPs?" Tags="&lt;search&gt;&lt;constraint-satisfaction-problems&gt;&lt;efficiency&gt;&lt;depth-first-search&gt;&lt;uninformed-search&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9127" PostTypeId="2" ParentId="9125" CreationDate="2018-11-23T16:29:32.060" Score="1" Body="&lt;p&gt;A successor function is a function that generates a next state from the current state, plus the choices that affect state changes. In e.g. the 8 queens problem, a state might be the location of 5 queens so far, the choice might be where to put the next queen, and the successor function would return the resulting state with 6 queens on the board.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically a solver will store/cache a current state, make a choice, and use the successor function to discover what the next state is. Then it may call the heuristic function on the next state and make a decision whether to continue that search deeper into the search tree (or recursion) or try another choice at the current state.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-11-23T16:29:32.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9132" PostTypeId="1" CreationDate="2018-11-23T23:36:44.133" Score="1" ViewCount="20" Body="&lt;p&gt;Suppose if I am building a Linear Regression model with one fully connected layer and a sigmoid with minimizing mean squared error as objective. Why would the error surface be convex?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does finding the optimal parameters for this network mean we cannot do better than this? Under what assumptions, this solution would be optimal? If we relax the linearity assumption and add some non-linearity to the network can we do better than this? Why so?&lt;/p&gt;&#xA;" OwnerUserId="20163" LastActivityDate="2018-11-23T23:36:44.133" Title="What does it essentially mean if the neural network has convex error surface?" Tags="&lt;neural-networks&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9134" PostTypeId="1" CreationDate="2018-11-24T03:37:30.443" Score="2" ViewCount="128" Body="&lt;p&gt;Can intelligent agents (and chatbots) have personalities and emotions, given a properly defined ontology?&lt;/p&gt;&#xA;" OwnerUserId="19784" LastEditorUserId="2444" LastEditDate="2020-05-15T23:15:55.247" LastActivityDate="2020-05-15T23:15:55.247" Title="Can intelligent agents have personalities and emotions?" Tags="&lt;chat-bots&gt;&lt;intelligent-agent&gt;&lt;human-like&gt;&lt;emotional-intelligence&gt;&lt;ontology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9137" PostTypeId="1" CreationDate="2018-11-24T06:07:42.353" Score="2" ViewCount="96" Body="&lt;p&gt;Recently I worked on a paper by Hao Wang, &lt;a href=&quot;http://www.wanghao.in/paper/KDD15_CDL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Collaborative Deep learning for Recommender Systems&lt;/a&gt;; which uses a two way tightly coupled method, Collaborative filtering for Item correlation and Stacked Denoising Autoencoders for the Optimization of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know the limitations of using stacked Autoencoders and Hierarchical Bayesian methods to Recommender systems.&lt;/p&gt;&#xA;" OwnerUserId="10118" LastEditorUserId="1671" LastEditDate="2019-04-04T20:40:18.600" LastActivityDate="2021-04-24T01:32:22.693" Title="What are some limitations of using Collaborative Deep learning for Recommender systems?" Tags="&lt;deep-learning&gt;&lt;autoencoders&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9139" PostTypeId="2" ParentId="9137" CreationDate="2018-11-24T09:25:00.927" Score="0" Body="&lt;p&gt;A good recommender program is one of the hardest problems in AI. &#xA;Recomender needs to first know me (by talking to me or by analyzing the movies that I graded as good).&#xA;Then it needs to be able to recognize &lt;em&gt;similarity&lt;/em&gt; of my taste to a new movie. So it needs to watch movies that I liked, build a list of common things between these movies and then watch 100 random new movies and pick the one that has the most patterns in common with my liked movies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So once again a good similarity recognition system is the key to successfull AI.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-11-24T09:25:00.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9140" PostTypeId="2" ParentId="9106" CreationDate="2018-11-24T13:24:35.257" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://kushalvyas.github.io/gen_8Q.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you can find an example of how to apply genetic algorithms to solve the 8-queens problem.&lt;/p&gt;&#xA;&lt;p&gt;The proposed fitness function is based on the chessboard arrangement, and in particular, it is inversely proportional to the number of clashes amongst attacking positions of queens; thus, a high fitness value implies a low number of clashes.&lt;/p&gt;&#xA;" OwnerUserId="20166" LastEditorUserId="2444" LastEditDate="2021-01-30T02:34:28.230" LastActivityDate="2021-01-30T02:34:28.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9141" PostTypeId="1" AcceptedAnswerId="10628" CreationDate="2018-11-24T13:44:39.893" Score="19" ViewCount="22959" Body="&lt;p&gt;I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the &lt;a href=&quot;https://github.com/karpathy/char-rnn&quot; rel=&quot;noreferrer&quot;&gt;CharRNN&lt;/a&gt;, which uses RNN to generate a sequence of words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;noreferrer&quot;&gt;BERT&lt;/a&gt; has come out several weeks ago and is very powerful. Therefore, I am wondering whether this task can also be done with the help of BERT? I am a new learner in this field, and thank you for any advice!&lt;/p&gt;&#xA;" OwnerUserId="20170" LastEditorUserId="6578" LastEditDate="2020-07-03T08:28:03.393" LastActivityDate="2020-07-03T08:28:03.393" Title="Can BERT be used for sentence generating tasks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;bert&gt;&lt;text-generation&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="9143" PostTypeId="2" ParentId="9062" CreationDate="2018-11-24T15:28:24.673" Score="0" Body="&lt;p&gt;Hill climbing, Simulated annealing, genetic algorithms are different variants of local search algorithm. Hill Climbing use to find good solution for NP-hard problems. Simulated annealing was first used extensively to solve VLSI layout problems. It has been applied widely to factory scheduling and other large-scale optimization tasks. Genetic algorithm have had a wide-spread impact on optimization problems, such as circuit layout and job-shop scheduling.&lt;/p&gt;&#xA;" OwnerUserId="19652" LastEditorUserId="2444" LastEditDate="2019-03-02T11:12:18.327" LastActivityDate="2019-03-02T11:12:18.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9146" PostTypeId="1" AcceptedAnswerId="9157" CreationDate="2018-11-24T17:05:31.833" Score="1" ViewCount="239" Body="&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/cnn-models-for-human-activity-recognition-time-series-classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This article&lt;/a&gt; on human activity recognition states that 1D convolutional neural networks work the best on the classification of human activities using data from the accelerometer. But I think that human activities, like swinging the arm, are sequential actions and they require LSTMs.&lt;/p&gt;&#xA;&lt;p&gt;So, which one should be more effective for this task: CNNs or LSTMs? In other words, is spatial learning required or sequence learning?&lt;/p&gt;&#xA;" OwnerUserId="18985" LastEditorUserId="32410" LastEditDate="2021-04-17T21:49:22.543" LastActivityDate="2021-04-17T21:49:22.543" Title="Are 1D CNNs really the appropriate model for human activity recognition?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;human-activity-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9147" PostTypeId="2" ParentId="4610" CreationDate="2018-11-24T19:26:50.280" Score="0" Body="&lt;p&gt;Interesting question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case One &amp;mdash; Goals not Goal&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When systems are developed, they are based on requirements.  The goals of the system is closely coupled to the goals of the development.  This is the first example.  Few systems and few development efforts to construct them are goal driven.  They are rather systems and processes that tend toward a set of goals.  Rarely is there a single loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider an animal.  It's metabolic stasis system may generate what seems like a goal, a set of signals to other systems to find and eat food.  At another time its temperature stasis system may generate what seems like a goal, a set of signals to other systems to build a shelter to enter.  The notion that there is some goal to survive is inconsistent with the way neurological and metabolic systems actually interact.  Survival, some would say, was the goal of the animal's evolution, but that is not technically true.  It survived, not because survival was a goal but because each of its ancestors survived as a result of transient and recurrent command signals for eating and sheltering.  There is no aggregate loss function.  There is no single neural pathway that all these signals are funneled through.  The animal's behavioral subsystems may share some components, but the mechanisms of intent are largely independent.  It only appears to a casual observer that the separate systems are collaborating toward survival.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is true of some vehicle automation systems.  Collision avoidance is a system with a set of goals and routing is a system with a separate set of goals.  The goals are decoupled even though they may share a CPU, vision systems, and a large number of functional components and computing resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These systems are goals driven, using the plural, not goal driven.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case Two &amp;mdash; Process of Elimination&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another type of AI approach that is not technically goal driven are genetic algorithms.  They replicate and some of the replicants are eliminated, but the entities themselves have no goal.  The elimination criteria is not technically a goal.  It is the opposite of one.  The appearance of replicants that are not eliminated after many rounds of replication and concurrent narrowing of criteria may achieve some goal of the person running the algorithm, but the algorithm itself has no representation of a goal but rather its opposite, the final criteria of elimination in the last round.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case Three &amp;mdash; Experimental Apparatus&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many AI experiments to discover the nature of intelligence and ways to simulate aspects of intelligence have the external goal defined by the researcher, but the system used in the experiment, the AI apparatus, is not driven by a goal.  It runs goal-less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case Four &amp;mdash; Toys&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether toys for adults or kids, toys may arise with some goal in mind, to amuse in some way, but the AI within them has no goal.  There is no loss function to minimize or reinforcement or wellness signal to maximize.  It's behavior may be less amusing if it were goal driven.  What makes it amusing is its ability to produce patterns associated with intelligence, but to no particular purpose.  In some ways, much human conversation and creative art is like this.  Too much purpose, and the level of amusement decreases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case Five &amp;mdash; Robots That Mimic Postmodern Humans&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans in postmodern society, the ones that are truly relativists and have rejected modernism, are not particularly goal driven.  They tend to wander around and do things that are intelligent, however they may act goal directed and then destroy the product of the transient goal.  Building sand castles on the beach is an example.  Although this may not be the focus of current AI research, since there is so much hope placed in AI to do things for us, such does not exclude the possibility of future robots that build something and then destroy it to build something else, without some practical objective.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-24T19:26:50.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9148" PostTypeId="5" CreationDate="2018-11-24T20:18:08.403" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-24T20:18:08.403" LastActivityDate="2018-11-24T20:18:08.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9149" PostTypeId="4" CreationDate="2018-11-24T20:18:08.403" Score="0" Body="For questions related to the planing of paths through a tree or network of vertices connected by edges, whereby there is some set of objectives that the plan is constructed to meet. Such objectives could be as simple as arrival at one of a set of vertices or as complex as a function determined by some function applied to the entire plan. The state of the tree or network and the value of particular edge traversals may be either static or dynamic." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-29T17:01:24.370" LastActivityDate="2018-11-29T17:01:24.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9150" PostTypeId="1" AcceptedAnswerId="9155" CreationDate="2018-11-24T21:09:49.787" Score="1" ViewCount="137" Body="&lt;p&gt;The formula for the &lt;em&gt;sum of squared errors&lt;/em&gt; (SSE) is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{1}{2} \sum_{i=1}^n (t^i - o^i)^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a few related questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$t^i - o^i$&lt;/span&gt; is negative, doesn't the power of 2 eliminate any negative result?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can then exist any negative gradient at the output?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What if the output is too high, and so the gradient should be negative, meaning that the weights (on average) have to be decreased? How can it be differentiated between 'output too high' and 'output too low'?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="2444" LastEditDate="2021-03-10T17:54:23.417" LastActivityDate="2021-03-10T17:55:41.003" Title="How can the sum of squared errors have negative gradient if it's defined as the squared of the error?" Tags="&lt;machine-learning&gt;&lt;loss-functions&gt;&lt;gradient&gt;&lt;sse&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9152" PostTypeId="1" CreationDate="2018-11-24T22:37:23.793" Score="3" ViewCount="1113" Body="&lt;p&gt;I've been thinking if machine learning can be used to play the game Scrabble. My knowledge is limited in the ML field, thus I've seeking some pointers :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how could I possibly build a model that picks a move from all the given valid moves of the current game state, and then plays the move and wait for the delayed reward. The actions here aren't static actions, they are basically selecting move to maximize the final score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to encode the valid moves and then use a model to pick those moves?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also considered the genetic approach, but I think if I can represent my move with a set of features (score, consonantVowels ration, rack leave score, #blank tiles after the move, ...etc), training a neural network like this could take a long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another training related question, is it feasible to run the training on a GPU given that I will be waiting for a response (the new game state) from the opponent (e.g. Quackle) after every action?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you :)&lt;/p&gt;&#xA;" OwnerUserId="20180" LastEditorUserId="20180" LastEditDate="2018-11-26T21:31:14.243" LastActivityDate="2018-12-21T02:43:26.837" Title="Scrabble game using machine learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;genetic-algorithms&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9153" PostTypeId="1" CreationDate="2018-11-25T06:56:11.300" Score="2" ViewCount="76" Body="&lt;p&gt;How does &lt;a href=&quot;https://plato.stanford.edu/entries/formal-belief/#DemShaThe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dempster-Shafer theory&lt;/a&gt; work in representing ignorance in AI field? &lt;/p&gt;&#xA;" OwnerUserId="19657" LastEditorUserId="2444" LastEditDate="2019-11-19T15:03:36.543" LastActivityDate="2019-11-19T22:06:55.890" Title="How Dempster-Shafer theory work in AI?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;computational-learning-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9155" PostTypeId="2" ParentId="9150" CreationDate="2018-11-25T09:11:17.310" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$t^i - o^i$&lt;/span&gt; is negative, doesn't the power of 2 eliminate any negative result?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the loss function, yes that is correct, and is what you want - a measurement that gets higher due to any difference between the predicted and correct results. Minimising the value of that measurement is a goal for the optimiser.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How can then exist any negative gradient at the output?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The gradient of the cost function is different to the cost function. Technically it is the derivative of the cost function with respect to the parameters you can change. Taking the derivative changes the powers in the equation.&lt;/p&gt;&#xA;&lt;p&gt;If you plot the graph from a single example, you can clearly see the gradient is negative when the prediction is too low, and positive when the prediction is too high:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8vkab.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8vkab.png&quot; alt=&quot;SSE loss function example graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note this is the opposite relationship to the one you suggest in the question. That is why it is called gradient &lt;em&gt;descent&lt;/em&gt; when we apply corrections. The steps to correct the parameters are in the &lt;em&gt;opposite&lt;/em&gt; direction to the gradient.&lt;/p&gt;&#xA;&lt;p&gt;In terms of algebra (and moving the example index into a subscript so that is doesn't get in the way visually), assuming &lt;span class=&quot;math-container&quot;&gt;$o_i$&lt;/span&gt; is your output value, and &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt; is the target value, then you want the gradient of your loss function &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$o_i$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial o_i}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial o_i} =  \frac{\partial}{\partial o_i} \frac{1}{2}(t_i - o_i)^2 = \frac{\partial}{\partial o_i} \frac{1}{2}(t_i^2 - 2t_i o_i + o_i^2) = \frac{1}{2}(-2t_i + 2o_i) = o_i - t_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This simple gradient value is negative if your output &lt;span class=&quot;math-container&quot;&gt;$o_i$&lt;/span&gt; is lower than the target &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt;, and positive if &lt;span class=&quot;math-container&quot;&gt;$o_i$&lt;/span&gt;is greater than &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As an aside, if you wondered why the multiplier of &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; was included in the definition of SSE, it is intended by design to end up with this simple result. Without it, everything works, just you would end up with the gradient being &lt;span class=&quot;math-container&quot;&gt;$2(o_i - t_i)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-03-10T17:55:41.003" LastActivityDate="2021-03-10T17:55:41.003" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9156" PostTypeId="1" CreationDate="2018-11-25T10:51:05.420" Score="3" ViewCount="48" Body="&lt;p&gt;One of the solutions to scale blockchain is to use off-chain channels. You can find its definition here: &lt;a href=&quot;https://en.bitcoin.it/wiki/Off-Chain_Transactions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.bitcoin.it/wiki/Off-Chain_Transactions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one of the problems of off-chain channels is finding a suitable &lt;em&gt;decentralized routing&lt;/em&gt; mechanism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since in Bitcoin there is no routing table and transactions are only broadcast and also, in general, we need to &lt;em&gt;avoid centralized&lt;/em&gt; approaches for routing, is it practical to use swarm intelligence algorithms, such as ant colony optimization ones, for off-chain channels?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I refer you to a proposed ant routing algorithm in the paper &lt;a href=&quot;https://arxiv.org/pdf/1807.00151.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ant routing algorithm for the Lightning Network&lt;/a&gt; for an instance of employing ACO algorithms for routing in _lightning network. However, the paper has not been evaluated to demonstrate its performance. &lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="2444" LastEditDate="2019-05-27T22:17:19.283" LastActivityDate="2019-05-27T22:17:19.283" Title="Is there an efficiency swarm Intelligence algorithm for off-chain channels routing in blockchain?" Tags="&lt;applications&gt;&lt;swarm-intelligence&gt;&lt;ant-colony-optimization&gt;&lt;blockchain&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9157" PostTypeId="2" ParentId="9146" CreationDate="2018-11-25T11:41:35.070" Score="2" Body="&lt;p&gt;Human activity recognition, for example as done with the &lt;a href=&quot;https://github.com/gtoderici/sports-1m-dataset/blob/wiki/ProjectHome.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sports-1M Dataset&lt;/a&gt;, is a classification task. Given a video, say which activity / action was executed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is a special case about video classification.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;human activities like swinging the arm are sequential actions and they require LSTMs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is wrong. As a simple example, think of handwriting recognition as done on &lt;a href=&quot;http://write-math.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;write-math.com&lt;/a&gt;: I receive a sequence of points (x, y) -- the pen-tip --  and classify which symbol is written. So it is a sequence, but still there a plenty of ways to use a simple multi-layer perceptron (MLP) for it. Two big groups of approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Feature-Engineering&lt;/strong&gt;: Instead of giving an arbitrary number of coordinates, one can normalize it to a fixed amount. This is what is currently done on write-math.com and works pretty well (see &lt;a href=&quot;https://arxiv.org/pdf/1511.09030.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;my Bachelors thesis&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Space Transformation&lt;/strong&gt;: Similar to the feature engineering, but more extreme. Instead of looking at (variants of) coordinates, one can look at the image which is there in the end. This way one can apply CNNs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can do similar things for movies:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multiple Classifications: Classify based on single frames and average the results&lt;/li&gt;&#xA;&lt;li&gt;Feature-Engineering: Normalize the length to a fixed number; interpret the fixed-length video as an image with more channels.&lt;/li&gt;&#xA;&lt;li&gt;Use CNNs as feature extractors / for dimensionality reduction and LSTM for the sequence classification.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You might be interested in &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Ng_Beyond_Short_Snippets_2015_CVPR_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beyond Short Snippets: Deep Networks for Video Classification&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Efficiency vs Effectiveness&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Model A is more effective than model B, if its optimizarion criterion is better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model A is more efficient than model B, if it uses less resources (e.g. memory) to achieve the same optimization value (e.g. same accuracy / MSE)&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-11-25T11:50:15.503" LastActivityDate="2018-11-25T11:50:15.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9158" PostTypeId="1" CreationDate="2018-11-25T12:06:12.473" Score="2" ViewCount="71" Body="&lt;p&gt;How is the cost of the path to each node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; computed in the &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A* algorithm&lt;/a&gt;? Do we need to add the cost of the path to the parent node &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; to the cost of the path of the child node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="19817" LastEditorUserId="2444" LastEditDate="2019-11-10T22:31:56.050" LastActivityDate="2019-11-10T22:31:56.050" Title="How is the cost of the path to each node computed in A*?" Tags="&lt;search&gt;&lt;a-star&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9161" PostTypeId="2" ParentId="5769" CreationDate="2018-11-25T16:46:23.593" Score="7" Body="&lt;p&gt;I'm following up on the answers above with a concrete example in the hope to further clarify how the convolution works with respect to the input and output channels and the weights, respectively:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the example be as follows (wrt to 1 convolutional layer):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the input tensor is 9x9x5, i.e. 5 input channels, so &lt;code&gt;input_channels=5&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;the filter/kernel size is 4x4 and the stride is 1&lt;/li&gt;&#xA;&lt;li&gt;the output tensor is 6x6x56, i.e. 56 output channels, so &lt;code&gt;output_channels=56&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;the padding type is 'VALID' (i.e. no padding)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We note that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;since the input has 5 channels, the filter dimension becomes 4x4x5, i.e. there are 5 separate, unique 2D filters of size 4x4 (i.e. each has 16 weights); in order to convolve over the input of size 9x9x5 the filter becomes 3D and must be of size 4x4x5&lt;/li&gt;&#xA;&lt;li&gt;therefore: for each input channel, there exists a distinct 2D filter with 16 different weights each. In other words, the number of 2D filters matches the number of input channels&lt;/li&gt;&#xA;&lt;li&gt;since there are 56 output channels, there must be 56 3-dimensional filters W0, W1, ..., W55 of size 4x4x5 (cf. in the CS231 graphic there are 2 3-dimensional filters W0, W1 to account for the 2 output channels), where the 3rd dimension of size 5 represents the link to the 5 input channels (cf. in the CS231 graphic each 3D filter W0, W1 has the 3rd dimension 3, which matches the 3 input channels)&lt;/li&gt;&#xA;&lt;li&gt;therefore: the number of 3D filters equals the number of output channels&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That convolutional layer thus contains: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;56 3-dimensional filters of size 4x4x5 (= 80 different weights each) to account for the 56 output channels where each has a value for the 3rd dimension of 5 to match the 5 input channels. In total there are &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;number_of_filters=input_channel*output_channels=5*56=280&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2D filters of size 4x4 (i.e. 280x16 different weights in total).&lt;/p&gt;&#xA;" OwnerUserId="20191" LastEditorUserId="20191" LastEditDate="2018-11-26T08:19:42.970" LastActivityDate="2018-11-26T08:19:42.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9162" PostTypeId="2" ParentId="9158" CreationDate="2018-11-25T19:48:40.683" Score="3" Body="&lt;p&gt;The evaluation function in A* is &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the cost of the path from the starting node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is an estimate of the distance from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node. To compute &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;, you simply do &lt;span class=&quot;math-container&quot;&gt;$g(n) = g(p) + c(p, n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the parent node, and &lt;span class=&quot;math-container&quot;&gt;$c(p, n)$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;ost of the edge between &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. So, yes, to compute &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; you use &lt;span class=&quot;math-container&quot;&gt;$g(p)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-26T11:43:13.713" LastActivityDate="2018-11-26T11:43:13.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9163" PostTypeId="1" CreationDate="2018-11-25T22:06:31.570" Score="1" ViewCount="34" Body="&lt;p&gt;for a school project I have been given a dataset containing images of plants and weeds. The goal is to detect when there is a weed in the pictures. The training and validation sets have already been created by our teachers, however they probably didn't have enough images for both so they &quot;photoshopped&quot; some weeds in some of the training pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are examples of images with the weed label in the training set:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PSlqn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PSlqn.png&quot; alt=&quot;weed label in the training set&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, the &quot;photoshopped&quot; weed is hard to detect, and no shape resembling a weed is clearly visible like in this picture (weed at the very bottom, near the middle):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7GwrR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7GwrR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is an example of an image with the weed label in the validation set:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SSFkA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SSFkA.png&quot; alt=&quot;weed label in the validation set&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I go about preprocessing the training set so that a CNN trained on it would perform well on the validation set? I was thinking of applying a low-pass filter to the rough edges of the photoshopped images so that the network doesn't act as an edge detector, but it doesn't seem very robust. Should I manually select the best images from the training set? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="20198" LastEditorUserId="20198" LastEditDate="2018-11-25T22:31:09.303" LastActivityDate="2018-11-25T22:31:09.303" Title="How to preprocess a modified dataset so that a fitted CNN makes correct predictions on an un-modified version of the dataset?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9164" PostTypeId="2" ParentId="9134" CreationDate="2018-11-25T23:25:50.087" Score="2" Body="&lt;p&gt;It can depend on how you define &amp;quot;emotion&amp;quot; and &amp;quot;personality&amp;quot;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Personality&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Personality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia defines personality&lt;/a&gt; as &amp;quot;the characteristic set of behaviors, cognitions, and emotional patterns that evolve from biological and environmental factors.&amp;quot;  &lt;em&gt;(This is a serviceable definition we can use.)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since intelligent agents make decisions (take actions), it is safe to say they have a set of behaviors.  Cognition is easy here as well--regardless of the structure of the algorithm, we can say it utilizes a logical process that serves the function of cognition.&lt;/p&gt;&#xA;&lt;p&gt;Emotion is more difficult.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Emotions&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How exactly human emotions work is still partly obscure, and it seems to be a safe bet that chemicals are involved.  But, in a reductionist sense, going back to Ancient Greek Philosophy, we can use a binary &amp;quot;love/hate&amp;quot; (attraction/repulsion) and apply it generally, where these feelings are the prime motivators of natural forces:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And Hesiod says, &lt;em&gt;“First of all things was Chaos made, and then/Broad-bosomed Earth . . . and Love, the foremost of immortal beings,”&lt;/em&gt; thus implying that there must be in the world some cause to move things and combine them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sub&gt;Aristotle (&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.01.0052%3Abook%3D1%3Asection%3D984b&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Metaphysics&lt;/em&gt; 1.984b&lt;/a&gt;)&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The real issue comes with &lt;a href=&quot;https://en.wikipedia.org/wiki/Emotion&quot; rel=&quot;nofollow noreferrer&quot;&gt;interpretations of emotion that involve consciousness&lt;/a&gt;, such as on Wikipedia, but even the wiki immediately issues the disclaimer: &lt;em&gt;&amp;quot;Scientific discourse has drifted to other meanings and there is no consensus on a definition,&amp;quot;&lt;/em&gt; so it may be more useful to look to definitions rooted in cognition and the presumed &lt;a href=&quot;https://en.wikipedia.org/wiki/Utility&quot; rel=&quot;nofollow noreferrer&quot;&gt;utility&lt;/a&gt; of emotions from an evolutionary standpoint.&lt;/p&gt;&#xA;&lt;p&gt;If reinforcement learning is being used, an automata can be said to like/dislike (love/hate) certain outcomes, which can be expressed via methods such as reward functions.&lt;/p&gt;&#xA;&lt;p&gt;Certainly this is not emotion as we humans understand it, but it can be seen as an analog. (Regarding humans chemical response to entities/objects and events/outcomes as reinforce future decisions.)&lt;/p&gt;&#xA;&lt;p&gt;ANSWER:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;In no way do contemporary bots and agents have personalities and emotions in the sense of humans&lt;/strong&gt;--we are probably still very from this and don't know if it's even possible. It may never be possible validate artificial consciousness, regardless of how conscious a system appears to be.&lt;/p&gt;&#xA;&lt;p&gt;But, depending on how we define these terms &amp;quot;personality&amp;quot; and &amp;quot;emotion&amp;quot;, we can choose to regard automata as having some reduced form of analogous qualities and functions.&lt;/p&gt;&#xA;&lt;p&gt;Example: If I engineer a game AI to have certain tendencies and behaviors (high-risk-tolerance, partisan, aggressive) that are distinct from other agents (low-risk-tolerance, cooperative, defensive), such agents are commonly referred to as having personalities.&lt;/p&gt;&#xA;&lt;p&gt;Example: We actually do use terms like &amp;quot;goodness/badness&amp;quot; and informally refer to the automata as &amp;quot;liking&amp;quot; or &amp;quot;not liking&amp;quot; nodes in a graph.  This is a function of &lt;a href=&quot;https://en.wikipedia.org/wiki/Anthropomorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;anthropomorphization&lt;/strong&gt;&lt;/a&gt;.  (Whether you consider these more than analogies depends on the definitions of feelings, but, generally, there needs to be something to feel. So you also need to reduce consciousness/awareness to the most basic requirement of simply &amp;quot;receiving of input&amp;quot;.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-11-26T01:47:19.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9165" PostTypeId="1" AcceptedAnswerId="9170" CreationDate="2018-11-26T00:42:57.097" Score="12" ViewCount="1115" Body="&lt;p&gt;From DeepMind's &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;noreferrer&quot;&gt;research paper&lt;/a&gt; on arxiv.org:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper, we apply a similar but fully generic algorithm, which&#xA;  we call &lt;em&gt;AlphaZero&lt;/em&gt;, to the games of chess and shogi as well as Go,&#xA;  without any additional domain knowledge except the rules of the game,&#xA;  demonstrating that a general-purpose reinforcement learning algorithm&#xA;  can achieve, tabula rasa, superhuman performance across many&#xA;  challenging domains.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does this mean AlphaZero is an example of AGI (Artificial General Intelligence)?&lt;/p&gt;&#xA;" OwnerUserId="17601" LastEditorUserId="1581" LastEditDate="2018-11-28T21:24:10.427" LastActivityDate="2018-11-29T21:30:54.587" Title="Is AlphaZero an example of an AGI?" Tags="&lt;game-ai&gt;&lt;definitions&gt;&lt;agi&gt;&lt;alphago&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9166" PostTypeId="2" ParentId="9141" CreationDate="2018-11-26T01:01:44.943" Score="2" Body="&lt;p&gt;No. Sentence generating is directly related to language modelling (given the previous words in the sentence, what is the next word). &lt;a href=&quot;https://github.com/google-research/bert/issues/35&quot; rel=&quot;nofollow noreferrer&quot;&gt;Because of bi-directionality of BERT, BERT cannot be used as a language model.&lt;/a&gt; If it cannot be used as language model, I don't see how you can generate a sentence using BERT.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastEditorUserId="2444" LastEditDate="2019-02-17T11:23:00.753" LastActivityDate="2019-02-17T11:23:00.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9167" PostTypeId="5" CreationDate="2018-11-26T01:26:03.080" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Anthropomorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anthropomorphism (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Anthropomorphism, anthropomorphize; Greek &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0058%3Aentry%3Da)%2Fnqrwpos&quot; rel=&quot;nofollow noreferrer&quot;&gt;ἄνθρωπος&lt;/a&gt;  (ánthrōpos, lit. &quot;human&quot;) and &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0057%3Aentry%3Dmorfh%2F&quot; rel=&quot;nofollow noreferrer&quot;&gt;μορφή&lt;/a&gt; (morphē, &quot;form&quot;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-26T01:26:03.080" LastActivityDate="2018-11-26T01:26:03.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9168" PostTypeId="4" CreationDate="2018-11-26T01:26:03.080" Score="0" Body="The attribution of human traits and qualities to non-human entities or structures. Includes emotion, intention, awareness, et al." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-26T01:26:03.080" LastActivityDate="2018-11-26T01:26:03.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9169" PostTypeId="1" AcceptedAnswerId="9175" CreationDate="2018-11-26T01:39:10.813" Score="3" ViewCount="158" Body="&lt;p&gt;When someone wants to compare 2 inputs, the most widespread idea is to use a Siamese architecture. Siamese architecture is a very high level idea, and can be customized based on the problem we are required to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any other architecture type to compare 2 inputs ?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Background&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I want to use a neural network for comparing 2 documents (semantic textual similarity). Siamese network is one approach, I was wondering if there is more.&lt;/p&gt;&#xA;" OwnerUserId="18852" LastActivityDate="2018-11-27T00:20:46.850" Title="Neural network architecture for comparison" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;models&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9170" PostTypeId="2" ParentId="9165" CreationDate="2018-11-26T02:36:15.403" Score="12" Body="&lt;p&gt;Good question!&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AlphaZero, though a major milestone, is most definitely &lt;em&gt;not&lt;/em&gt; an AGI :)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;AlphaGo, though strong at the game of Go, is &lt;em&gt;narrowly strong&lt;/em&gt; (&quot;strong-narrow AI&quot;), defined as strength in a single problem or type of problem (such as Go and other non-chance, perfect information games.) &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AGI, at minimum, must be about as strong as humans in &lt;em&gt;all&lt;/em&gt; problems worked on or solved by humans.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;AGI is often associated with &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;noreferrer&quot;&gt;superintelligence&lt;/a&gt;, defined as intelligence that surpasses human levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AGI does not necessarily imply super-intelligence, in the sense that we'd consider an android that can perform all human activities with the same capability as humans as an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But technically, AlphaGo is a narrow superintelligence in that it exceeds all human performance in a single problem.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-29T21:30:54.587" LastActivityDate="2018-11-29T21:30:54.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9171" PostTypeId="2" ParentId="2642" CreationDate="2018-11-26T02:55:40.553" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;Easy: Parse the code!  If the code meets your definition of Artificial Intelligence, that's what it is.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Only half kidding there.  The question of what defines AI is constantly evolving and there is no exact consensus.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most reductionist view could regard any decision making algorithm to be a form of intelligence, and the strength of the intelligence (it's level of &lt;a href=&quot;https://en.wikipedia.org/wiki/Skill&quot; rel=&quot;nofollow noreferrer&quot;&gt;advancement&lt;/a&gt;) is dependent on the frame of reference (typically a subjective entity making that evaluation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, it might be better to regard the issue from the standpoint of utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Utility is a central concept in AI because it is the means by which we evaluate &quot;intelligence&quot; (strength in a given task or problem aka &quot;how well does it get the job done?&quot;) Does the algorithm fulfill the task(s) with sufficient optimality to meet our definition of Artificial Intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to believe the quote itself may be primarily a comment on the definitions of Artificial intelligence, per &lt;a href=&quot;https://ai.stackexchange.com/a/2643/1671&quot;&gt;Lovecraft's answer&lt;/a&gt;.  Subjectivity is indeed the a central notion of the &lt;a href=&quot;https://en.wikiquote.org/wiki/Arthur_C._Clarke#On_Clarke&amp;#39;s_Laws&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clark quote it references&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-26T03:37:44.473" LastActivityDate="2018-11-26T03:37:44.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9172" PostTypeId="2" ParentId="9081" CreationDate="2018-11-26T03:48:28.027" Score="2" Body="&lt;p&gt;Right now, the scenarios where automata protect their own existence is limited.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of autonomous vehicles, primary goals would certainly be collision avoidance, and other hazards (deep water, cliff faces, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible that arial drones could have certain automated defense mechanisms specific to airborne threats such a missiles.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bots might be said to protect their existence by copying themselves onto new systems.   &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-11-26T03:48:28.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9173" PostTypeId="1" AcceptedAnswerId="9283" CreationDate="2018-11-26T04:16:01.023" Score="0" ViewCount="923" Body="&lt;p&gt;I'm trying to build a DQN to replicate the DeepMind results. I'm doing with a simple DQN for the moment, but it isn't learning properly: after +5000 episodes, it couldn't get more than 9-10 points. Each episode has a limit of 5000 steps but it couldn't reach more than 500-700. I think the problem is in the replay function, which is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def replay(self, replay_batch_size, replay_batcher):&#xA;    j = 0&#xA;    k = 0&#xA;    replay_action = []&#xA;    replay_state = []&#xA;    replay_next_state = []&#xA;    replay_reward= []&#xA;    replay_superbatch = []&#xA;&#xA;    if len(memory) &amp;lt; replay_batch_size:&#xA;        replay_batch = random.sample(memory, len(memory))&#xA;        replay_batch = np.asarray(replay_batch)&#xA;        replay_state_batch, replay_next_state_batch, reward_batch, replay_action_batch = replay_batcher(replay_batch)&#xA;    else:&#xA;        replay_batch = random.sample(memory, replay_batch_size)&#xA;        replay_batch = np.asarray(replay_batch)&#xA;        replay_state_batch, replay_next_state_batch, reward_batch, replay_action_batch = replay_batcher(replay_batch)&#xA;&#xA;    for j in range ((len(replay_batch)-len(replay_batch)%4)):&#xA;&#xA;        if k &amp;lt;= 4:&#xA;            k = k + 1              &#xA;            replay_state.append(replay_state_batch[j])&#xA;            replay_next_state.append(replay_next_state_batch[j])&#xA;            replay_reward.append(reward_batch[j])&#xA;            replay_action.append(replay_action_batch[j])&#xA;&#xA;        if k &amp;gt;=4:                &#xA;            k = 0&#xA;            replay_state = np.asarray(replay_state)&#xA;            replay_state.shape = shape&#xA;            replay_next_state = np.asarray(replay_next_state)&#xA;            replay_next_state.shape = shape&#xA;            replay_superbatch.append((replay_state, replay_next_state,replay_reward,replay_action))&#xA;&#xA;            replay_state = []&#xA;            replay_next_state = []&#xA;            replay_reward = []&#xA;            replay_action = []&#xA;&#xA;    states, target_future, targets_future, fit_batch = [], [], [], []&#xA;&#xA;    for state_replay, next_state_replay, reward_replay, action_replay in replay_superbatch:&#xA;&#xA;        target = reward_replay&#xA;        if not done:&#xA;            target = (reward_replay + self.gamma * np.amax(self.model.predict(next_state_replay)[0]))&#xA;&#xA;        target_future = self.model.predict(state_replay)&#xA;&#xA;        target_future[0][action_replay] = target&#xA;        states.append(state_replay[0])&#xA;        targets_future.append(target_future[0])&#xA;        fit_batch.append((states, targets_future))&#xA;&#xA;    history = self.model.fit(np.asarray(states), np.array(targets_future), epochs=1, verbose=0)&#xA;&#xA;    loss = history.history['loss'][0]&#xA;&#xA;    if self.exploration_rate &amp;gt; self.exploration_rate_min:&#xA;&#xA;        self.exploration_rate -= (self.exploration_rate_decay/1000000)&#xA;    return loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I'm doing is to get 4 experiences (states), concatenate and introduce them in the CNN in shape (1, 210, 160, 4). Am I doing something wrong? If I implement the DDQN (Double Deep Q Net), should I obtain similar results as in the &lt;a href=&quot;https://www.youtube.com/watch?v=V1eYniJ0Rnk&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind Breakout video&lt;/a&gt;? Also, I'm using the Breakout-v0 enviroment from OpenAI gym.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;Am I doing this properly? I implemented an identical CNN; then I update the target each 100 steps and copy the weights from &lt;code&gt;model&lt;/code&gt; CNN  to &lt;code&gt;target_model&lt;/code&gt; CNN. Should it improve the learning? Anyway I'm getting low loss.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for state_replay, next_state_replay, reward_replay, action_replay in replay_superbatch:&#xA;&#xA;            target = reward_replay&#xA;            if not done:&#xA;&#xA;                target = (reward_replay + self.gamma * np.amax(self.model.predict(next_state_replay)[0]))&#xA;            if steps % 100 == 0:&#xA;&#xA;                target_future = self.target_model.predict(state_replay)&#xA;&#xA;                target_future[0][action_replay] = target&#xA;                states.append(state_replay[0])&#xA;                targets_future.append(target_future[0])&#xA;                fit_batch.append((states, targets_future))&#xA;                agent.update_net()&#xA;&#xA;        history = self.model.fit(np.asarray(states), np.array(targets_future), epochs=1, verbose=0)&#xA;&#xA;        loss = history.history['loss'][0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt;&#xA;So as far I understand, this code should work am I right?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if not done:&#xA;&#xA;            target = (reward_replay + self.gamma * np.amax(self.target_model.predict(next_state_replay)[0]))&#xA;            target.shape = (1,4)&#xA;&#xA;            target[0][action_replay] = target&#xA;            target_future = target&#xA;            states.append(state_replay[0])&#xA;            targets_future.append(target_future[0])&#xA;            fit_batch.append((states, targets_future))&#xA;&#xA;        if step_counter % 1000 == 0:&#xA;&#xA;            target_future = self.target_model.predict(state_replay)&#xA;&#xA;            target_future[0][action_replay] = target&#xA;            states.append(state_replay[0])&#xA;            targets_future.append(target_future[0])&#xA;            fit_batch.append((states, targets_future))&#xA;            agent.update_net()&#xA;&#xA;    history = self.model.fit(np.asarray(states), np.array(targets_future), epochs=1, verbose=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9818" LastEditorUserId="9818" LastEditDate="2018-12-01T02:34:51.007" LastActivityDate="2018-12-01T02:34:51.007" Title="DQN it's not working properly" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9174" PostTypeId="2" ParentId="9089" CreationDate="2018-11-26T08:35:33.180" Score="0" Body="&lt;p&gt;&lt;strong&gt;Clarifications&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before directly answering the question, there are a few things to note.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The word range is actually applicable to a dependent variable.  On the input side, it is the domain that needs to be normalized.&lt;/li&gt;&#xA;&lt;li&gt;In the second method, since the minimum vector may not be the origin, the zeroth order term should be included in the normalization.  The minimum must be subtracted from the maximum to determine the domain and must also be subtracted as an offset to the independent, pre-normalized values.&lt;/li&gt;&#xA;&lt;li&gt;The standard deviation is always less than half the difference between the max and min, so there is a difference in proportion between the two methods.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The crux of the question centers around the statistics of normalization and its potential impact on these methods.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multivariate linear regression (in the question introduction)&lt;/li&gt;&#xA;&lt;li&gt;Gradient descent (selected as a tag)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although these two are not technically mutually exclusive, practically they are if by linear is meant conforming to &lt;span class=&quot;math-container&quot;&gt;$\vec{y} = a \vec{x} + \vec{b}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is a matrix dimensioned appropriately for &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt;.  For that first degree polynomial cases, there is a closed form (formula) to acquire &lt;span class=&quot;math-container&quot;&gt;$(\vec{a}, \vec{b}, c)$&lt;/span&gt; from a set of pairs &lt;span class=&quot;math-container&quot;&gt;$(\vec{x}, \vec{y})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the correlation coefficient.  It would be grossly wasteful to use iterative convergence methods, such as gradient descent, to calculate a mean squares regression result for a first degree polynomial model of the relation between &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is no closed form for the model or no known model at all, and the complexity of the function to be approximated is tenable in relation to the convergence methods available, the data available for training, and the computing resources and time that can be delegated to the task, then gradient descent is a method through which a least squares convergence on minimal loss may be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The significance of artificial networks is that, through the interconnection of a sequence of layers, each with an array of like activation functions, a network can approximate a useful function the mathematical model of which is not explicitly known.  Of the statistical functional approximation problems where no closed form is known and for which convergence methods are possible, artificial network training has become the appropriate device for a growing set of cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Normalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The metrics of mean and standard deviation are designed specifically for data that can be approximated by a Gaussian distribution.  The term normalization originated from the belief that the Gaussian distribution was normal, however Gauss did not hold that belief.  He presented it as a distribution resulting from the limit of an accumulation of random perturbations as the mean of the square of the perturbation size decreases to zero as the length of the series of perturbations approaches infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of this, if the loss (or error) function is a sum of the squares of difference between the current output values of an artificial network and the labels providing the accurate and correct values, a near Gaussian distribution of input is likely to produce a faster and complete convergence.  In the multivariate case, there is an equivalent of the Gaussian distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the real world, data distribution very close to Gaussian is not the norm.  The distribution of land surface temperatures at a specific latitude and longitude on earth for a particular month reflect two maxima in the temperature distribution, one for nighttime and one for daytime.  The model of the relationship between these two maxima and the time of year, in chaos theory, are called attractors.  Points seem attracted to those norms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution of signal amplitudes in common circuits and natural sounds in the air are not Gaussian distributed either, but the &lt;span class=&quot;math-container&quot;&gt;$\log(P)$&lt;/span&gt; of the power of common signals and sounds are close to a Gaussian distribution, which is partly why decibels are a common measure of signal strength and loudness.  The other part is that the logarithm in calculating decibels causes them to indicate relative geometric difference (ratio) in power rather than a less representative arithmatic difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normalization is often oversimplified to mean ensuring values fall within a specific domain convenient to an algorithm's numerical representation or around a mean with a given standard deviation to feed a least squares strategy for statistical analysis or correlation.  However, data is not often symmetric and nicely distributed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of land surface temperatures the nighttime distribution has a lower standard deviation than the daytime distribution, so the two peaks have different adjacent slopes in the distribution.  How that is reflected in normalization varies depending on goals and algorithms receiving the normalized data.  Normalization strategies for signal power and loudness would include running the data through a logarithm to reflect the near exponential Gaussian distribution of power values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Outliers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outliers are data points that fall outside an expected trend or relationship, either because they were incorrectly measured in one or more dimensions of measurement, the measurement was corrupted, they were strongly influenced by a phenomenon outside the intended range of study, or representative of a significant but previously undiscovered phenomenon.  This last cause is why it is not wise to dismiss and exclude all outliers from functional analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When outliers are valid data points that inconveniently lie more than a few standard deviations from mean, normalizing from zero to one causes a very narrow distribution peak in one place within that input domain.  If the outlier is of the kind that should not be dismissed and legitimately representative of the phenomena under study or being controlled, then some normalization scheme other than simply a first degree polynomial translation of the data is indicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Housing Predictor&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of square footage of living space, the data is not symmetric.  An increase of 10 square meters for a large house will not produce the same price differential as 10 square meter increase for a small one.  A properly configured neural network may learn to perform something similar to a logarithm in the first few layers during training, or the engineer can explicitly perform a logarithm to un-skew the data and present it in a form with closer to a Gaussian distribution to the network for training and then also later for actual use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the exit, prior to presenting the network output to the loss function, applying an exponential function may also produce faster and more reliable convergence, since a monetary unit has different values to different buyers.  A person buying a large house is less apt to be put off by cost associated with a fourth bedroom or more area than a person buying a small house.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comparative advantages of different normalization methods are dependent upon a number of factors, including but not limited to these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The resolution of the data types used to represent forward and backward signal propagation in the network, which affects the conditions of signal saturation&lt;/li&gt;&#xA;&lt;li&gt;Skew of data set used for training&lt;/li&gt;&#xA;&lt;li&gt;The number of standard deviations to the min and max in the data set used for training&lt;/li&gt;&#xA;&lt;li&gt;Presence of multiple attractors in the distribution, which can occur in any of the features (dimensions of input) or combinations of them&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Method one is best for a scenario that involves a Gaussian distribution and may be best for the housing prediction.  Method two might be best if the distribution is relatively flat almost all the way to the extremes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running each method and comparing them may reveal advantages for one or the other that is not predicted in this answer and further knowledge can be gained about why by performing further experiments.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-26T08:35:33.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9175" PostTypeId="2" ParentId="9169" CreationDate="2018-11-26T09:45:46.897" Score="2" Body="&lt;p&gt;A RBM (restricted Boltzmann machine) can be trained to extract document features. The same resulting machine can extract features of two or more documents.  Because documents can be just as easily processed in series using the same machine parameters and CPU (saving the feature results) as documents could be processed in parallel using separate CPUs, the Siamese idea is less of a characteristic of system architecture and more of a characteristic of the process topology. The architectural decision of how to run the process topology on available hardware should usually remain somewhat decoupled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comparison operation could be a MLP (multi-layer perceptron) trained to produce the comparison results based on a requisite number example document comparisons.  In such a case, the MLP should be trained with the previously trained RBMs as a front end.  You will need some human labor to produce or extract from available data sources the example document comparison results.  The example results will be need, along with a reference to the pair of RBMs feature extraction results corresponding to the comparison, to train the MLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even further upstream from the RBMs, it is often useful to producing reliable and accurate document processing to pre-process the documents.  Images and text can be delineated so that features can be extracted from the images in one way and from the text in another and then from the sequence of text and images in a third way.  In such process topologies, CNNs (convolution network) typically pre-process the images and LSTM networks typically pre-process the text.  They may then directly feed the RBM or indirectly through another process component.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-27T00:20:46.850" LastActivityDate="2018-11-27T00:20:46.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9176" PostTypeId="1" CreationDate="2018-11-26T12:24:28.953" Score="2" ViewCount="42" Body="&lt;p&gt;Are all ant routing algorithms the same? If no, what is the common properties of all of them? In other words, how we can detect a routing algorithm is an ant routing algorithm?&lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="2444" LastEditDate="2019-05-27T22:06:58.500" LastActivityDate="2019-05-27T22:06:58.500" Title="Are all ant routing algorithms the same?" Tags="&lt;algorithm&gt;&lt;swarm-intelligence&gt;&lt;ant-colony-optimization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9179" PostTypeId="1" CreationDate="2018-11-26T13:12:13.263" Score="3" ViewCount="119" Body="&lt;p&gt;I am learning to create a dialogue system. The various parts of such a system are Intent classifier, slot filling, Dialogue state tracking (DST), dialogue policy optimization and NLG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While reading &lt;a href=&quot;https://storage.googleapis.com/pub-tools-public-publication-data/pdf/44018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper on DST, I found out that a discriminative  sequence model of DST can identify goal constraints, fill slots and maintain state of the conversation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that now I dont need to create an intent classifier and slot filling models separately as the tasks are already being done by this DST? Or I am misunderstanding both the things and they are separate?&lt;/p&gt;&#xA;" OwnerUserId="19244" LastEditorUserId="19244" LastEditDate="2018-11-26T14:19:21.107" LastActivityDate="2018-11-27T20:31:33.160" Title="Does an advanced Dialogue state tracking eliminate the need of intent classifier and slot filling models in dialogue systems/ chatbots?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9180" PostTypeId="1" CreationDate="2018-11-26T14:02:25.583" Score="1" ViewCount="105" Body="&lt;p&gt;I am working on an image data-set. As you may have guessed it is imbalanced data. I have 'Class A, 19,000 images' and 'Class B, 2,876 images'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I did an undersampling by removing randomly from the majority class till it becomes equal to the minority class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On doing this I am loosing lot of information from those 19000 images which I could get. &#xA;So I do an oversampling of minority class, by simply copying the 2,876 images again and again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this undersampling method correct, will it effect my accuracy? I trained an Inceptionv4 model using this oversampled data and it is not at all stable and I am getting poor accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;What should be my strategy ?&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="19842" LastEditorUserId="4709" LastEditDate="2018-11-26T18:38:55.283" LastActivityDate="2018-11-29T16:57:37.127" Title="Can I do oversampling by copying the same image multiple times? Will it effect my neural network accuracy?" Tags="&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9181" PostTypeId="2" ParentId="5769" CreationDate="2018-11-26T14:43:53.190" Score="0" Body="&lt;p&gt;I recommend &lt;a href=&quot;https://arxiv.org/pdf/1707.09725.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 2.2.1 of my masters thesis&lt;/a&gt; as an answer. To add to the remaining answers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keras is your friend to understand what happens:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential&#xA;from keras.layers import Conv2D&#xA;&#xA;model = Sequential()&#xA;model.add(Conv2D(32, input_shape=(28, 28, 3),&#xA;          kernel_size=(5, 5),&#xA;          padding='same',&#xA;          use_bias=False))&#xA;model.add(Conv2D(17, (3, 3), padding='same', use_bias=False))&#xA;model.add(Conv2D(13, (3, 3), padding='same', use_bias=False))&#xA;model.add(Conv2D(7, (3, 3), padding='same', use_bias=False))&#xA;model.compile(loss='categorical_crossentropy', optimizer='adam')&#xA;&#xA;print(model.summary())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;conv2d_1 (Conv2D)            (None, 28, 28, 32)        2400      &#xA;_________________________________________________________________&#xA;conv2d_2 (Conv2D)            (None, 28, 28, 17)        4896      &#xA;_________________________________________________________________&#xA;conv2d_3 (Conv2D)            (None, 28, 28, 13)        1989      &#xA;_________________________________________________________________&#xA;conv2d_4 (Conv2D)            (None, 28, 28, 7)         819       &#xA;=================================================================&#xA;Total params: 10,104&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try to formulate your options. What would that mean for the parameters if something else would be the case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: &lt;span class=&quot;math-container&quot;&gt;$2400 = 32 \cdot (3 \cdot 5 \cdot 5)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach also helps you with other layer types, not only convolutional layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please also note that you are free to implement different solutions, that might have other numbers of parameters. &lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="3217" LastEditDate="2018-12-04T09:51:44.970" LastActivityDate="2018-12-04T09:51:44.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9182" PostTypeId="1" AcceptedAnswerId="9183" CreationDate="2018-11-26T14:59:20.600" Score="4" ViewCount="5378" Body="&lt;p&gt;How do I show that uniform-cost search is a special case of A*? How do I prove this?&lt;/p&gt;&#xA;" OwnerUserId="19448" LastEditorUserId="2444" LastEditDate="2019-11-10T17:00:30.157" LastActivityDate="2019-11-10T17:00:30.157" Title="How do I show that uniform-cost search is a special case of A*?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;proofs&gt;&lt;a-star&gt;&lt;uniform-cost-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9183" PostTypeId="2" ParentId="9182" CreationDate="2018-11-26T15:13:05.710" Score="6" Body="&lt;p&gt;Yes, UCS is a special case of A*. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UCS uses the evaluation function &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the length of the path from the starting node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, whereas A* uses the evaluation function &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; means the same thing as in UCS and &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt;, called the &quot;heuristic&quot; function, is an estimate of the distance from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node. In the A* algorithm, &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; must be &lt;a href=&quot;https://en.wikipedia.org/wiki/Admissible_heuristic&quot; rel=&quot;noreferrer&quot;&gt;admissible&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UCS is a special case of A* which corresponds to having &lt;span class=&quot;math-container&quot;&gt;$h(n) = 0, \forall n$&lt;/span&gt;. A heuristic function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; which has &lt;span class=&quot;math-container&quot;&gt;$h(n) = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\forall n$&lt;/span&gt;, is clearly admissible, because it always &quot;underestimates&quot; the distance to the goal, which cannot be smaller than &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, unless you have negative edges (but I assume that all edges are non-negative). So, indeed, UCS is a special case of A*, and its heuristic function is even admissible!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this with an example, just draw a simple graph, and apply the A* algorithm using &lt;span class=&quot;math-container&quot;&gt;$h(n) = 0$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and then apply UCS to the same graph. You will obtain the same results.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-26T15:19:27.280" LastActivityDate="2018-11-26T15:19:27.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9184" PostTypeId="1" AcceptedAnswerId="9185" CreationDate="2018-11-26T15:24:23.780" Score="2" ViewCount="86" Body="&lt;p&gt;I understand both terms, linear regression and maximum likelihood, but, when it comes to the math, I am totally lost. So I am reading this article &lt;a href=&quot;http://complx.me/2017-01-22-mle-linear-regression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Principle of Maximum Likelihood&lt;/em&gt;&lt;/a&gt; (by Suriyadeepan Ramamoorthy). It is really well written, but, as mentioned in the previous sentence, I don't get the math. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The joint probability distribution of &lt;span class=&quot;math-container&quot;&gt;$y,\theta, \sigma$&lt;/span&gt; is given by (assuming &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is normally distributed): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fReBa.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fReBa.gif&quot; alt=&quot;joint probability distribution of y,θ,σ&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equivalent to maximizing the log likelihood:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Jpron.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jpron.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maxima can be then equating through  the derivative of l(θ) to zero:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gXRPo.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gXRPo.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get everything until this point, but don't understand how this function is equivalent to the previous one :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/90mST.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/90mST.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15391" LastEditorUserId="2444" LastEditDate="2018-11-26T18:26:48.373" LastActivityDate="2018-11-26T18:26:48.373" Title="Understanding the math behind using maximum likelihood for linear regression" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;maximum-likelihood&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9185" PostTypeId="2" ParentId="9184" CreationDate="2018-11-26T16:25:57.063" Score="1" Body="&lt;p&gt;Note first that the first &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt; (equals) in &lt;span class=&quot;math-container&quot;&gt;$\frac{dl(\theta)}{d\theta} &#xA; = 0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$&lt;/span&gt; should be interpreted as a &quot;is set to&quot;, that is, we set &lt;span class=&quot;math-container&quot;&gt;$\frac{dl(\theta)}{d\theta} &#xA; = 0$&lt;/span&gt;. Given that (apparently) &lt;span class=&quot;math-container&quot;&gt;$\frac{dl(\theta)}{d\theta} = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\frac{dl(\theta)}{d\theta} &#xA; = 0$&lt;/span&gt; is equivalent to &lt;span class=&quot;math-container&quot;&gt;$0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's apply some basic linear algebra:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;0 &amp;amp;= −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta) \iff \\&#xA;0 &amp;amp;= −(0−2X^TY + X^TX \theta) \iff \\&#xA;0 &amp;amp;= −0 + 2X^TY - X^TX \theta) \iff \\&#xA;0 &amp;amp;= 2X^TY - X^TX \theta \iff \\&#xA;X^TX \theta &amp;amp;= 2X^TY \iff \\&#xA;(X^TX)^{-1}(X^TX) \theta &amp;amp;= (X^TX)^{-1}2X^TY \iff \\&#xA;\theta &amp;amp;= (X^TX)^{-1}2X^TY&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can ignore the &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, because it is just a constant, and, when optimizing, this does not influence the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that using &lt;span class=&quot;math-container&quot;&gt;$\hat{\theta}$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is just to indicate that what we will get is an &quot;estimate&quot; of the real &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, because of round off errors during the computations, etc.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2018-11-26T16:39:15.713" LastActivityDate="2018-11-26T16:39:15.713" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9186" PostTypeId="1" AcceptedAnswerId="9213" CreationDate="2018-11-26T16:52:29.400" Score="2" ViewCount="2245" Body="&lt;p&gt;Why do we use a last-in-first-out (LIFO) queue in the depth-first search algorithm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the breadth-first search algorithm, we use a first-in-first-out (FIFO) queue, so I am confused.&lt;/p&gt;&#xA;" OwnerUserId="20217" LastEditorUserId="2444" LastEditDate="2020-06-02T22:55:27.810" LastActivityDate="2020-06-02T22:56:48.690" Title="Why do we use a last-in-first-out queue in depth-first search?" Tags="&lt;search&gt;&lt;implementation&gt;&lt;breadth-first-search&gt;&lt;depth-first-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9190" PostTypeId="1" AcceptedAnswerId="9205" CreationDate="2018-11-26T19:41:12.317" Score="4" ViewCount="46" Body="&lt;p&gt;I'm wondering if I can visualize the backprop process as follows (please excuse me if I have written something terrible wrong). If the loss function &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; on a neural network represents the function has the form &#xA;&lt;span class=&quot;math-container&quot;&gt;$$L = f(g(h(\dots u(v(\dots))))$$&lt;/span&gt;&#xA;then we can visualize the derivative of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; wrt the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th function &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial v} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial h}\dots\frac{\partial u}{\partial v}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I able to view all neural networks as having a loss function of the form of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; given above? That is, am I correct in saying that any neural network is just a function composition and that I can write the partial derivative wrt any parameter as written above (I know I took the partial with respect to the function &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="20222" LastActivityDate="2018-11-27T08:47:18.337" Title="Am I able to visualize the differentiation in backprop as follows?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9191" PostTypeId="1" CreationDate="2018-11-26T20:56:12.613" Score="1" ViewCount="16" Body="&lt;p&gt;Let's say we have 2 blocks of text, where there are some similarities as in 2 interpretations of the same story. Is there any solution that would recognize these consistencies and ultimately be able to recreate a story with all the details of each?&lt;/p&gt;&#xA;" OwnerUserId="20223" LastActivityDate="2018-11-26T20:56:12.613" Title="Intelligent Text Combination" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9192" PostTypeId="2" ParentId="3965" CreationDate="2018-11-26T23:24:25.557" Score="0" Body="&lt;p&gt;That's a great question and probably one of the most difficult tasks on ML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You do have a few options: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can use weighting algorithms (e.g. Chi-squared) to understand which features are contributing most to your output&lt;/li&gt;&#xA;&lt;li&gt;You can use other ML algorithms to classify whether a feature is contributing to your predictions or not&lt;/li&gt;&#xA;&lt;li&gt;You may use other ML algorithms (other than NN) that inherently provide you with feature weightings (e.g. Random Forest) &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope that helps &lt;/p&gt;&#xA;" OwnerUserId="20224" LastActivityDate="2018-11-26T23:24:25.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9197" PostTypeId="1" AcceptedAnswerId="9218" CreationDate="2018-11-27T04:07:47.937" Score="5" ViewCount="901" Body="&lt;p&gt;Advantage Actor-Critic algorithm may use the following expression to get 1-step estimate of the advantage:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ A(s_t,a_t) = r(s_t, a_t) + \gamma V(s_{t+1}) (1 - done_{t+1})  - V(s_t) $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$done_{t+1}=1$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; is a terminal state (end of the episode) and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose our learning environment has a goal, collecting the goal gives reward &lt;span class=&quot;math-container&quot;&gt;$r=1$&lt;/span&gt; and terminates the episode. Agent also receives &lt;span class=&quot;math-container&quot;&gt;$r=-0.1$&lt;/span&gt; for every step, encouraging it to collect the goal faster. We're learning with &lt;span class=&quot;math-container&quot;&gt;$\gamma=0.99$&lt;/span&gt; and we terminate the episode after &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; timesteps if the goal wasn't collected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the state before collecting a goal we have the following advantage, which seems very reasonable: &lt;span class=&quot;math-container&quot;&gt;$A(s_t,a_t) = 1 - V(s_t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the timestep &lt;span class=&quot;math-container&quot;&gt;$T-1$&lt;/span&gt;, regardless of the state, we have: &lt;span class=&quot;math-container&quot;&gt;$ A(s_{T-1},a_{T-1}) = r(s_{T-1}, a_{T-1}) - V(s_{T-1}) \approx -0.1 -\frac{-0.1}{1-\gamma} = -0.1 + 10 = 9.9 $&lt;/span&gt; &#xA;(this is true under the assumption that we're not yet able to collect the goal reliably often, therefore the value function  converges to something close to &lt;span class=&quot;math-container&quot;&gt;$\frac{r_{avg}}{1-\gamma} \approx -10 $&lt;/span&gt; ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is not a part of the state, so the value function has no way to anticipate the sudden change in reward-to-go. So, all of a sudden, we got a (relatively) big advantage for the arbitrary action that we took at the timestep &lt;span class=&quot;math-container&quot;&gt;$T-1$&lt;/span&gt;.  Following the policy gradient rule, we will significantly increase the probability of an arbitrary action that we took at the end of the episode, even if we didn't achieve anything. This can quickly destroy the learning process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do people deal with this problem in practice? My ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Differentiate between actual episode terminations and ones caused by the time limit, e.g. for them we will not replace next step value estimate with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Somehow add &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to the state such that the value function can learn to anticipate the termination of the episode.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As I noticed, the A2C implementation in OpenAI baselines does not seem to bother with any of that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/openai/baselines/blob/f3a5abaeeb1c1c9136a01c9dbfebc173dc311fef/baselines/a2c/runner.py#L64&quot; rel=&quot;nofollow noreferrer&quot;&gt;A2C implementation - calculation of discounted rewards&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/openai/baselines/blob/8c2aea2addc9f3ba36d4a0c937e6a2d09830afc7/baselines/a2c/utils.py#L147&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;discount_with_dones&lt;/code&gt; function&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20229" LastEditorUserId="20229" LastEditDate="2018-11-28T00:15:50.793" LastActivityDate="2020-01-25T18:04:59.917" Title="How to deal with episode termination in Advantage Actor-Critic algorithm?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9198" PostTypeId="1" CreationDate="2018-11-27T04:35:02.757" Score="1" ViewCount="70" Body="&lt;p&gt;I just started using IBM Cloud Private for Data this week and I wasn't sure if I can use other public clouds to connect with my ICP for data account. So I spoke with an IBM representative and I wanted to share their responses...&lt;/p&gt;&#xA;" OwnerUserId="17067" LastActivityDate="2019-08-24T06:00:37.443" Title="Does IBM Cloud Private for Data run on public clouds like AWS or Azure?" Tags="&lt;structured-data&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9199" PostTypeId="2" ParentId="9198" CreationDate="2018-11-27T04:35:02.757" Score="1" Body="&lt;p&gt;Yes, ICP for Data can run on any public cloud infrastructure as long as it supports Docker containers and Kubernetes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Already tested: Amazon AWS, IBM Cloud, Azure, and Red Hat OpenStack&lt;/p&gt;&#xA;" OwnerUserId="17067" LastActivityDate="2018-11-27T04:35:02.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9200" PostTypeId="1" CreationDate="2018-11-27T06:20:54.403" Score="3" ViewCount="314" Body="&lt;p&gt;Apparently, the hill climbing algorithm just produces a local maximum, and not necessarily a global optimum. It's stuck on a local maximum. Why does hill climbing algorithm only produce a local maximum?&lt;/p&gt;&#xA;" OwnerUserId="19656" LastEditorUserId="2444" LastEditDate="2020-11-24T02:42:44.353" LastActivityDate="2020-11-24T02:42:44.353" Title="Why does the hill climbing algorithm only produce a local maximum?" Tags="&lt;optimization&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;local-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9203" PostTypeId="2" ParentId="8594" CreationDate="2018-11-27T08:02:05.290" Score="5" Body="&lt;p&gt;Since the question was tagged with &lt;a href=&quot;/questions/tagged/philosophy&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;philosophy&amp;#39;&quot; rel=&quot;tag&quot;&gt;philosophy&lt;/a&gt;, I'll provide a philosophical perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no definition of perfection. Just think about the silly example that is always brought up when we talk about autonomous cars: You are about to crash into a group of school kids and the AI's only chance to avoid them is by crashing into an old person. What should the AI do in order to react &quot;perfectly&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All actions that we take are based on our personal value system, so what is perfect to one person might be completely nonsense to another person. It might be possible to create an AI that always acts perfectly in terms of the value system you fitted it out with, but even in this case there are some caveats like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;reaction time: is the system fast enough to process all available information fast enough in order to take a fast action?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;available information: not all information might be available with the system. Do you consider it acting imperfect, if the action of the system turns out to be wrong once you posses all information about the situation, even though some information will never be available to the system (e.g. Heisenberg's principle of uncertainty)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19256" LastEditorUserId="2444" LastEditDate="2020-02-08T12:44:48.080" LastActivityDate="2020-02-08T12:44:48.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9205" PostTypeId="2" ParentId="9190" CreationDate="2018-11-27T08:11:07.660" Score="2" Body="&lt;p&gt;Yes, a neural network plus loss function can be viewed as a function composition as you have written, and back propagation is just the chain function repeated. Your equations &lt;span class=&quot;math-container&quot;&gt;$L = f(g(h(\dots u(v(\dots))))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial v} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial h}\dots\frac{\partial u}{\partial v}$&lt;/span&gt; are useful for high level intuition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At some point you need to look at the actual forms of the functions, and that introduces a bit more complexity. For instance, the loss function for a mini-batch can be expressed in terms of the output. This is the equivalent of &lt;span class=&quot;math-container&quot;&gt;$L = f(g)$&lt;/span&gt; for a batch or mini-batch with mean squared error loss, where I am using &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; (for cost) in place of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and the output of the neural network &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; in place of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J = \frac{1}{2N}\sum_{i=1}^{N}(\hat{y}_i - y_i)^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient of &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; is equivalent to your first part &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial f}{\partial g}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\hat{y}} J = \frac{1}{N}\sum_{i=1}^{N}(\hat{y}_i - y_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the functions in a neural network involve sums over terms. They can be expressed as vector and matrix operations, which can make them look simpler, but you still need to have code somewhere that works through all the elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one thing that the function composition view does not show well. The gradient you want to calculate is &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta} J$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; represents all the &lt;em&gt;parameters&lt;/em&gt; of the neural network (weights and biases). The parameters in each layer are end points of back propagation - they are not functions of anything else, and the chain rule has to stop with them. That means you have a series of &quot;dead ends&quot; - or possibly another way of thinking about it would be that your &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial v} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial h}\dots\frac{\partial u}{\partial v}$&lt;/span&gt; is the &quot;trunk&quot; of the algorithm that links layers together, and every couple of steps there is a &quot;branch&quot; to calculate the gradient of the parameters that you want to change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More concretely, if you have weight parameters in each layer noted as &lt;span class=&quot;math-container&quot;&gt;$W^{(n)}$&lt;/span&gt;, and two functions for each layer (the sum over weights times inputs, and an activation function) then your example ends up looking like this progression:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial W^{(n)}} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial h}\frac{\partial h}{\partial W^{(n)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial W^{(n-1)}} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial h}\frac{\partial h}{\partial i}\frac{\partial i}{\partial j}  \frac{\partial j}{\partial W^{(n-1)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial W^{(n-2)}} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial h}\frac{\partial h}{\partial i}\frac{\partial i}{\partial j}\frac{\partial j}{\partial k}\frac{\partial k}{\partial l}  \frac{\partial l}{\partial W^{(n-2)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;. . . this is the same idea but showing the goal of calculating &lt;span class=&quot;math-container&quot;&gt;$\nabla_{W} L$&lt;/span&gt; which doesn't fit into a &lt;em&gt;single&lt;/em&gt; chain of gradients. Notice the last term on each line does not appear in the next line. However, you can keep all the terms prior to that and re-use them in the next line - this matches the layer-by-layer calculations in many implementations of back propagation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2018-11-27T08:47:18.337" LastActivityDate="2018-11-27T08:47:18.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9206" PostTypeId="2" ParentId="9197" CreationDate="2018-11-27T08:15:37.643" Score="1" Body="&lt;p&gt;The practical need for a &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$t_{max}$&lt;/span&gt; is based on the importance of prudent doubt as a cost containment measure.  At a certain point, it is wise to abandon an objective the resolution of which cannot be certain.  This is a technique used in the minimization of a much wider loss function associated with the business into which the seeking of the learning objective fits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dismissing the easily retained knowledge of the state of the independent process variable &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the context of its domain &lt;span class=&quot;math-container&quot;&gt;$[1, T]$&lt;/span&gt; as irrelevant is like working on a project without project management.  What things to skip and what risks to take cannot be accurately evaluated, reducing the probability of success in relation to its optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly the value gained at any step is an indicator of the advantage of the current path, an affirmation that impacts the probability of achieving favorable results before &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; reaches &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.  Certainly any knowledge that can be used to predict possible values along the paths that begin with choices that can be made at any step should impact that choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question author's intuition that the critic should be aware that the stakeholder may pull the plug on the project soon is correct.  In terms of the mathematics, whether urgency can be formulated as something like &lt;span class=&quot;math-container&quot;&gt;$u(t) \propto \dfrac {1} {T - t + 1}$&lt;/span&gt; and used as a term or a factor requires some thought about the probabilities involved.  The advantage expression should reflect the application of the basic relations of probability to the probability distributions as seen considering all the information known at any temporal point along the pursuit of the objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only occasion when a variable that can be known should be excluded from the advantage expression is if it can be shown to be absolutely irrelevant or that its relevance is sufficiently small to warrant exclusion to gain computational speed in exchange for loss of accuracy and/or reliability.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-28T02:58:52.077" LastActivityDate="2018-11-28T02:58:52.077" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9207" PostTypeId="2" ParentId="3965" CreationDate="2018-11-27T09:04:58.847" Score="0" Body="&lt;p&gt;It is wise to consider not just the correlation of resource engagement with cost, but also the return on the cost of resource engagement.  The typical challenge is that those returns are almost always cumulative or delayed.  A case of accumulation is when the resource is the continuous tuning or improvement of a process the absence of which slows the generation of revenue.  A case of delay is when research resources incur costs without revenue impact for a period of time but the revenue generation that begins if the research delivers productive results may be a substantial factor above the total cost of the results delivered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason expense data by itself can lead to maladaptive network learning is because a network that is trained to reduce, for instance, marketing expenses will zero them.  That would usually cause a decreasing sales lead trend until the business folds.  Without including the returns in the training information, no useful learning may occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A basic MLP (multi-layer perceptron) will not learn the temporal characteristics of the data, the accumulation and delay aspects.  You will need a stateful network.  The most consistently successful network type for this kind of learning as of this writing is the LSTM (long short term memory) network type or one of its derivative variants.  Revenue and balance data must be used in conjunction with expense data to train the network to predict business results for any given sequence of proposed resource engagements (fully detailed budgetary plan).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function must properly balance sort term with medium and long term financial objectives.  Negative available cash should produce a pronounced increase in the loss function so that such avoidance of basic risks to reputation and the cost of credit is learned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which columns in your data have strong correlations with return on investment is difficult to determine in advance.  You can immediately exclude columns that conform to any one of the following criteria.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Always empty&lt;/li&gt;&#xA;&lt;li&gt;Other constants, those that have the same value for every row&lt;/li&gt;&#xA;&lt;li&gt;Those that can always be derived from other columns&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The data can be reduced in other ways&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fully describing data by characterizing trends in simple ways&lt;/li&gt;&#xA;&lt;li&gt;Using indices to specify long strings with 100% accuracy by assigning each string a number&lt;/li&gt;&#xA;&lt;li&gt;Compression&lt;/li&gt;&#xA;&lt;li&gt;Otherwise reducing redundancy in the data&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;RBMs (restricted Boltzmann machines) can extract features from the data and PCAs can illuminate the low information content columns, but the significance of the columns in terms of their correlation with revenue will not be identified using these devices in their basic form.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-27T09:04:58.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9208" PostTypeId="2" ParentId="9200" CreationDate="2018-11-27T09:29:48.440" Score="2" Body="&lt;p&gt;This part of your sentence is not always true &quot;and not reached to final goal/solution&quot;. If you have just one maximum at all and it is finite, hill climbing (HL) can reach to it and it is a global maximum too (for example, if the function is a parabola).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question is back to the stop criteria of HL. It will be stopped when reaching to a maximum point. The maximum point means a point which the objective function has a greater value in that point than on its &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-neighborhood of that point. Therefore, if objective function has some local maxima, depends on the initial point of HL, it might be trapped in one of the local maxima as their definition is satisfied in the explained definition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In sum, depends on the initial point of HL, it might not reach to the global maxima and trapped in local maxima if there exists.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2018-11-27T09:35:51.543" LastActivityDate="2018-11-27T09:35:51.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9212" PostTypeId="1" CreationDate="2018-11-27T13:40:58.347" Score="0" ViewCount="25" Body="&lt;p&gt;I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I decided to apply the log-returns to each of them. In other words, if &lt;span class=&quot;math-container&quot;&gt;$F_t$&lt;/span&gt; is a feature at a certain time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, then I apply &lt;span class=&quot;math-container&quot;&gt;$$\log(\frac{F_t}{F_{t-1}}).$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Rightarrow$&lt;/span&gt; &lt;a href=&quot;https://quant.stackexchange.com/questions/8875/why-non-stationary-data-cannot-be-analyzed?noredirect=1&amp;amp;lq=1&quot;&gt;Why non-stationary data is hard to analyse?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The property makes it easier to analyse, but produce a lot of zeros when &lt;span class=&quot;math-container&quot;&gt;$F_t = F_{t-1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As there is lots of zeros, then the predictions tend to be closer to 0. How can I reduce that effect mathematically?&lt;/p&gt;&#xA;" OwnerUserId="18765" LastEditorUserId="18765" LastEditDate="2018-11-27T19:22:25.920" LastActivityDate="2018-11-27T23:49:55.167" Title="Reduce the effect of excessive zeros" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9213" PostTypeId="2" ParentId="9186" CreationDate="2018-11-27T14:13:45.747" Score="1" Body="&lt;p&gt;We use the LIFO  queue, i.e. stack, for implementation of the depth-first search algorithm because depth-first search always expands the deepest node in the current frontier of the search tree. The search proceeds immediately to the deepest level of the search tree, where the nodes have no successors. As those nodes are expanded, they are dropped from the frontier, so then the search &quot;backs up&quot; to the next deepest node that still has unexplored successors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A LIFO queue means that the most recently generated node is chosen for expansion. This must be the deepest unexpanded node because it is one deeper than its parent, which, in turn, was the deepest unexpanded node when it was selected.&lt;/p&gt;&#xA;" OwnerUserId="19652" LastEditorUserId="2444" LastEditDate="2020-06-02T22:56:48.690" LastActivityDate="2020-06-02T22:56:48.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9217" PostTypeId="1" AcceptedAnswerId="9243" CreationDate="2018-11-27T16:52:12.273" Score="2" ViewCount="148" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1502.05477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the &lt;em&gt;Trust Region Policy Optimization&lt;/em&gt; paper&lt;/a&gt;, in Lemma 2 of Appendix A, I did not quite understand deriving inequality (31) from (30), which is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\bar{A}(s) = P(a \neq \tilde{a} | s) \mathbb{E}_{(a, \tilde{a}) \sim (\pi, \tilde{\pi})|a \neq  \tilde{a}} \left[ A_{\pi}(s, \tilde{a}) - A_{\pi}(s,a) \right]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$|\bar{A}(s)| \le \alpha. 2 \max_{s,a} |A_{\pi}(s,a)|$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you mind let me know how the inequality is derived? &lt;/p&gt;&#xA;" OwnerUserId="16912" LastEditorUserId="2444" LastEditDate="2019-05-02T16:01:28.527" LastActivityDate="2019-05-02T16:01:28.527" Title="Understanding lemma 2 of the &quot;Trust Region Policy Optimization&quot; paper" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;proofs&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9218" PostTypeId="2" ParentId="9197" CreationDate="2018-11-27T18:15:54.687" Score="4" Body="&lt;p&gt;First a small note: I don't think your expression for &lt;span class=&quot;math-container&quot;&gt;$A(s_{T-1}, a_{T - 1})$&lt;/span&gt; looks correct. If we assume that &lt;span class=&quot;math-container&quot;&gt;$V(s_T) = 0$&lt;/span&gt; (i.e., assume that we cannot possibly reach the goal in one single step from &lt;span class=&quot;math-container&quot;&gt;$s_{T - 1}$&lt;/span&gt;), we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;A(s_{T-1}, a_{T-1}) &amp;amp;= r(s_{T-1}, a_{T-1}) + \gamma V(s_{T}) (1 - done_{T})  - V(s_{T-1}) \\&#xA;&amp;amp;= r(s_{T-1}, a_{T-1}) - V(s_{T-1}).&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this expression, we'd normally have that &lt;span class=&quot;math-container&quot;&gt;$r(s_{T-1}, a_{T-1}) = -0.1$&lt;/span&gt;, whereas you seem to have mistakenly taken &lt;span class=&quot;math-container&quot;&gt;$+0.1$&lt;/span&gt; in your post.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Those details aside; yes, there can be sudden bumps of positive advantage estimates as you described. This is not a problem though, this is exactly what we'd expect to happen given your description of the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You describe an environment in which the agent is likely going to be wandering around randomly (at least when it hasn't learned a good policy yet), and incurring negative rewards over and over again. This naturally leads to negative value estimates for all encountered states. Suddenly, it does something and the episode terminates; it receives a nice reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; rather than yet another negative reward (this actually &quot;feels&quot; like a bonus, a positive reward, something more than was expected). When your agent has not yet learned a good policy that can reach the better reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, this is indeed a good result, a good action, and it rightfully should get reinforced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because this event of the episode terminating is mostly uncorrelated with the state (I say &quot;mostly&quot;, because in theory it probably ends up being a slightly rarer event in states close to the goal than in states far away from the goal), it will eventually (after sufficient training time) end up occurring approximately equally often in all states. From the perspective of an agent that is oblivious to the current time step, this will be perceived as an event that can simply occur by pure chance in a nondeterministic environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not necessarily a problem. It can slow down learning due to increased variance in your reward observations (which can be addressed by using low learning rates / large batch sizes), but Reinforcement Learning algorithms are almost always naturally built to handle nondeterministic environments, it can work this out, it can average out all the different outcomes observed for the same state+action pairs. &lt;strong&gt;This is not a problem that requires dealing with&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My ideas:&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Differentiate between actual episode terminations and ones caused by the time limit, e.g. for them we will not replace next step value estimate with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/li&gt;&#xA;  &lt;li&gt;Somehow add &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to the state such that the value function can learn to anticipate the termination of the episode.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first idea fundamentally changes the quantity that your algorithm is learning, it will essentially make your learning algorithm incorrect. There's always a chance that it might still appear to learn something useful (many Machine Learning/Reinforcement Learning algorithms can still appear to be okay even when there are bugs/technically incorrect parts), but it'll very likely perform worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second idea, while &lt;strong&gt;not necessary&lt;/strong&gt; as explained in my answer above, &lt;strong&gt;may still be beneficial to learning speed provided it is done well&lt;/strong&gt;. It may help because it can add to the power of your algorithm to &quot;explain&quot; its observations, and more importantly &quot;explain&quot; the variance in its observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem I see with adding &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to your input is that it is not naturally a binary variable. Very often you'll find that we're just using a bunch of binary inputs in (Deep) Reinforcement Learning algorithms. When all inputs are of the same magnitude like that, it tends to be easier to get the learning process to run well, tune hyperparameters like learning rate, etc. If you suddenly plug in an additional input which can take significantly larger values (like straight up adding &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; as an input), this will be more difficult. Adding &lt;span class=&quot;math-container&quot;&gt;$\frac{t}{T}$&lt;/span&gt; as an input may be better, since that will always still be bounded between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-11-27T18:15:54.687" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9220" PostTypeId="2" ParentId="5457" CreationDate="2018-11-27T19:33:15.533" Score="1" Body="&lt;p&gt;Have you tried any possibilities?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have many possible approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FeatureMiner (&lt;a href=&quot;http://featureselection.asu.edu/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://featureselection.asu.edu/index.php&lt;/a&gt;) or SkLearn (&lt;a href=&quot;https://scikit-learn.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/&lt;/a&gt;) are popular options for your purposes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation from these two possibilities is also interesting.&lt;/p&gt;&#xA;" OwnerUserId="20248" LastActivityDate="2018-11-27T19:33:15.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9221" PostTypeId="1" CreationDate="2018-11-27T20:00:46.283" Score="1" ViewCount="68" Body="&lt;p&gt;I'm trying to gain some insight into acoustic voice data composed of 19 features. I want to understand what features contribute most for classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADDED: Most features are related with the fundamental frequency stability. In particular I'm using voice shimmer and jitter and some related calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to use MRMR (max relevance min redundancy), but I would like to compare with some other options. &#xA;ADDED: I have tried to use FeatureMiner (&lt;a href=&quot;http://featureselection.asu.edu/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://featureselection.asu.edu/index.php&lt;/a&gt;) which provides some interesting algorithms implementations. However many of them use deprecated Python functions and require some effort to work properly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any popular tools for these purposes?&lt;/p&gt;&#xA;" OwnerUserId="20248" LastEditorUserId="20248" LastEditDate="2018-11-27T22:39:29.470" LastActivityDate="2021-04-16T01:58:44.197" Title="Feature selection of an acoustic voice dataset for classification" Tags="&lt;feature-selection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9223" PostTypeId="2" ParentId="9007" CreationDate="2018-11-27T20:38:07.530" Score="2" Body="&lt;p&gt;&lt;strong&gt;Basic Fairness&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If by ethics the question means that the system design is to foster basic ethics in the context of a conversation, then at least three attributes can be designed into the system.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Egalitarianism — no favor or disregard on the basis of gender, age, beliefs, financial status, ancestry, or culture, even if those features of the conversant are known and useful to know in personalizing the conversation&lt;/li&gt;&#xA;&lt;li&gt;Temporal fairness — ensuring that the interaction with new correspondents and veteran correspondents provide these with equal access to relevant information, even if the establishment of trust is an evidence based process controlled by the system (recommended)&lt;/li&gt;&#xA;&lt;li&gt;Sharing — balancing the objectives of the system to serve the interests of others as much as the interests of the system and its stakeholders — The Golden Rule: Treat others as you wish to be treated.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These attributes can be encoded in fuzzy rules, incorporated into the value function of an ongoing learning algorithm, or expressed in the loss function used during training, directing what is learned by a network.  All of these strategies require the tracking of metrics and the uses of them in adjusting behavior, a kind of social awareness.&lt;/p&gt;&#xA;&lt;p&gt;There are obvious ways to do this in the mathematical expressions of values, assets, advantages, affirmations, attentions, losses, errors, pains, or other learning feedback approaches are assembled.  In generative network topologies, a discriminative network would need to be able to recognize unfairness and selfishness using the above principles.  Such places specific requirements on the designs of metrics and how fairness and the balance of interests are quantified.&lt;/p&gt;&#xA;&lt;p&gt;A sum of squares across the dimensions (features) of potential inequity that can develop during a conversation is a possibility.  Using the fourth power instead of squaring would imbue permissiveness for smaller inequities but a drastic aversion to greater inequities.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\ell = \sum {(b_i - b_j)}^4 + \dots$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The variable &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; is the loss function result and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the benefit to conversation participants &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; where the two might be a human and the chatbot, a human and a stakeholder of the chatbot production server, or two humans that are in separate conversations with the chatbot, or two humans that are in the same conversation with the chatbot.  Note the ellipsis indicates more terms that affect loss as needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cognitive Ethical Sophistication&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If by ethics the question means the kind of ethics passed from families that value honor and truth to their children or taught at Harvard Law, the system design may have to wait until cognitive abilities are further developed in computing machinery.  Otherwise the question author will have to further AI to achieve the ethical sophistication.&lt;/p&gt;&#xA;&lt;p&gt;Research into semantic acquisition, abstraction, automated assembly of causal models, application of these abstractions and models in planning and execution, and the integration of these with natural linguistics is ongoing.  Broad cognition and What some might call wisdom escapes the products of AI as of this writing.&lt;/p&gt;&#xA;&lt;p&gt;A sufficiently deep network may approximate dialog that demonstrates what appears to be ethical awareness if trained with data that represents stories and conversational actions deemed to be ethical in those stories.  However, compacting learning the layers of comprehension that humans acquire through books, movies, other media, family life, and community events over a few decades (if they acquire it at all) into a computing project may be an overoptimistic objective at this time.&lt;/p&gt;&#xA;&lt;p&gt;Although a collection of classic works might demonstrate much of a high ethical standard, it is not labeled data.  One would need to identify which character was transformed in a way that demonstrates the valuable ethical standards and then configure the learning process to learn from the character arcs of ethical characters in the stories.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Inherent Feedback&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is no way to adapt without feedback.  If the system is adapting, some feedback mechanism is correcting conditions that are maladaptive, even if the mechanism is opaque to the observer or identified by some other name.  In the case of GANs, the feedback creates a balance through the discriminative and generative network components.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Bigg(\Big((G, E_1) \Rightarrow D\Big) \Rightarrow G, E_2\Bigg) \Rightarrow D \dots$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;That does not exclude the possibility that additional feedback paths may provide additional design advantages.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Hard Coding Ethics&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can judicious behavior be hard coded?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The idea is that there would be some sort of unchanging knowledge base for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change.&lt;/p&gt;&#xA;&lt;p&gt;Will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is not unchanging knowledge in the domain of ethics that leads to justice, but unchanging principles that guide the detection of what is not ethical that guides a continuous improvement of ethical standards.  That is why even constitutions have rules for amendment.  In probabilistic terms, without the balance of doubt and certainty, no ethical growth can occur.&lt;/p&gt;&#xA;&lt;p&gt;No one would read a book or watch a movie where the protagonist has no character arc because it is unremarkable.  When a story is remarkable it is because ethics are adaptive and the adaptation must be a new kind that is not already covered many times in previous books or movies.  People like to be surprised by these adaptations.  They must occur against all fears and opposing external forces that are trapping the protagonist in a prison of mediocrity.&lt;/p&gt;&#xA;&lt;p&gt;The story climax is when the protagonist, against all the building forces that keep them unremarkable, takes a path that discards all previous planning and forges a new radically different but fundamentally better one.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Excellent Objective in the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The intention to develop a system that is ethical is honorable and ethical in itself, so the question author may imbue some of those qualities into the system being designed.  In my opinion, such would be a triumph of greater value to the field of AI than all other progress up until now.&lt;/p&gt;&#xA;&lt;p&gt;To make such an advancement, the framing of the problem will require thinking in new directions.  Consider the stated concern.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It may begin to ignore these &amp;quot;facts&amp;quot; over time, and they may become irrelevant.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Two principles may prove elucidating in regard to this concern.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Every language in common use has the equivalent of the English words &lt;code&gt;should&lt;/code&gt; and &lt;code&gt;ought&lt;/code&gt;, indicating a universal sense that something is awry.  This detection and identification of imbalance is central to the definition of heroism and honor, which is making at least an attempt to re-balance it.  Even the Oracle in the Matrix, after creating imbalance, sits on the park bench talking to Sati in the warmth of a new balance.  Identifying what needs disruption and re-balancing and constructing a strategy to do so is the one thing in common across all ethical behavior.  Ethics is remedial.  Some ethical choices require unlearning, and that is the most timeless feature of ethics.&lt;/li&gt;&#xA;&lt;li&gt;Some of the most ancient principles, such as the Golden Rule or the idea of transparency – that right thought and right action are inescapably coupled – are more pertinent to this project than anything one can find in a book about machine learning.  The computer cannot hide a malicious intention and then converse ethically.  The objectives of the stakeholders of the system will need to also be ethical.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;First Decision in Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The decision to make is the level of abstraction to apply.  Should this chatbot be ethical or be ethically minded?&lt;/p&gt;&#xA;&lt;p&gt;The first would require the imbuing of ethical character into the bot.  The second would require the imbuing of the acquisition of ethical character into the bot.  As difficult as the more abstract second choice may seem, the first may be unachievable.  We don't have an example of it.  We do, however, have one proof of concept for the second choice: Good people.&lt;/p&gt;&#xA;&lt;p&gt;The other sub-question to address is regarding selection of AI components.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So which would likely be better [generative or reinforcement]? Or perhaps I should try a combination of the two?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you are serious about this objective beyond the simple tracking of basic fairness metrics, then you may have to use elements of both and possibly new things that extend the list of AI components and principles to use.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Take One Step Back&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;One thing to understand about generative topologies is that they are not really adversarial, even if that word is in the name of a seminal paper.  They are usually highly symbiotic and involve equilibria.  What would the discriminative network be without the generative one?  What would the generative one generate without the discriminative one?&lt;/p&gt;&#xA;&lt;p&gt;The two collaborate to create a feedback system exhibiting a balanced equilibrium.  This is like biological stasis, which is the genius of it.  It is also like chemistry.  Salt in saturated suspension in water involves both dissolving and crystallization in a constant expression of opposing reactions at a molecular level.  Similarly, all ethical decisions are decisions are made within the context of at least one equilibrium, and the details matter.&lt;/p&gt;&#xA;&lt;p&gt;Every court case where a judge or jury is to apply the public perception of ethics to the case has two sides presented.  They may appear to be adversaries, but the entire legal process is a sophisticated and symbiotic design to force collaboration in the interest of justice.&lt;/p&gt;&#xA;&lt;p&gt;Every educated journalist understands the importance of avoiding sensational reporting.  Dramatically presenting only one side of the story sells news and is a constant temptation.  Journalistic integrity demands striking an equilibrium.  The result of the ethical choice determines whether the news item educates the public or adds polarization to a culture that may already be partially crippled by extremism.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Take Another Step Back&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Reinforcement is an odd name to give what is essentially Agile planning via the probabilistic projection of expected advantages.  Should lean methods for strategic approaches to conversation be in this ethical chatbot?  If ethical behavior is defined to include telling stories that convey wisdom, then yes.  Roger Schank's work is of great interest in that case.  His work in story based reasoning may be a worthy direction of study.&lt;/p&gt;&#xA;&lt;p&gt;Adapting the foundations of ethics to the ever changing story inherent in every significant conversation may require planning.  It is possibly unavoidably inherent every conversation when the intention of the participants is to do more than exchange pleasantries.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-11-27T21:33:43.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9224" PostTypeId="2" ParentId="9050" CreationDate="2018-11-27T21:56:35.197" Score="0" Body="&lt;p&gt;There are a few options.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Social networks that have groups&lt;/li&gt;&#xA;&lt;li&gt;Associations that seek members&lt;/li&gt;&#xA;&lt;li&gt;University clubs that do not require members be registered for classes&lt;/li&gt;&#xA;&lt;li&gt;Conferences&lt;/li&gt;&#xA;&lt;li&gt;Corporations with organizations that seek members&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here are just a few of many.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aiinternational.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;THE ASSOCIATION FOR THE ADVANCEMENT OF ARTIFICIAL INTELLIGENCE&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.facebook.com/artificialintelligenceclub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence Club | Facebook&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.rubinghscience.org/aiclub/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Artificial Intelligence Research Club&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ww2.thenirvanaclub.com/join-the-club/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ww2.thenirvanaclub.com/join-the-club/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aiforhumanity.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aiforhumanity.net/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://aaai.org/Conferences/conferences.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://aaai.org/Conferences/conferences.php&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19168" LastEditorUserId="32410" LastEditDate="2021-05-16T19:50:30.040" LastActivityDate="2021-05-16T19:50:30.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9225" PostTypeId="2" ParentId="9221" CreationDate="2018-11-27T22:14:21.770" Score="0" Body="&lt;p&gt;It isn't obvious whether the 19 features are spectral and, if so, whether they contain detected transients for consonant phonics.  It's not obvious whether features were extracted using RBMs (restricted Boltzmann machines), from auto-correlation in the time domain, or some other way.  The intention of the classification, which would probably drive the nature of differentiation, isn't stated either.  If more details are added to the question, the answers may be more detailed.  Here are a few general options to compare.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Principal component analysis (PCA)&lt;/li&gt;&#xA;&lt;li&gt;Factor analysis&lt;/li&gt;&#xA;&lt;li&gt;Principal axis factoring&lt;/li&gt;&#xA;&lt;li&gt;Cluster analysis&lt;/li&gt;&#xA;&lt;li&gt;Regression or gradient descent if category labels exist and then checking early layer parameters for zeroes&lt;/li&gt;&#xA;&lt;li&gt;Feature selection&lt;/li&gt;&#xA;&lt;li&gt;Independent component analysis&lt;/li&gt;&#xA;&lt;li&gt;Feature Selection: A Data Perspective, Jundong Li et. al. 2016&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19168" LastEditorUserId="19168" LastEditDate="2018-11-27T22:23:42.137" LastActivityDate="2018-11-27T22:23:42.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9226" PostTypeId="1" CreationDate="2018-11-27T22:41:25.287" Score="1" ViewCount="48" Body="&lt;p&gt;I am new for machine learning and I am tried to understand basic steps to get final modal of Logistic Regression. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know Logistic Regression is supervisory learning technique. Therefore we want to give training data to training the modal. According to my understanding, I can take steps to get the final modal as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1&lt;/strong&gt; - Make an algorithm. For Logistic Regression is y = sigmoid(W x + B) function. Get zero or some value for W and B.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2&lt;/strong&gt; - Give sample known training data and find W and B values. x(1), x(2)...x(m) inputs and get y(1), y(2)...y(m) outputs. Gradient Descent use for find W and B that minimizes the cost function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 3&lt;/strong&gt; - Then apply W and B values which I found.y^ = sigmoid(W x + B). And then again apply sample known training data to get the predicting of y^.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 4&lt;/strong&gt; - Get the final modal to test unknown data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are these steps right? Please give me basic fundamental steps to understand supervisory learning technique. I would like to know un-supervisory learning technique steps also. &lt;/p&gt;&#xA;" OwnerUserId="20251" LastActivityDate="2018-12-07T18:09:11.143" Title="Steps for final Logistic Regression Modal" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9227" PostTypeId="2" ParentId="9212" CreationDate="2018-11-27T23:49:55.167" Score="1" Body="&lt;p&gt;That the features are not stationary is a reason why LSTM is a good approach.  Calculating the ratio of adjacent values in the sequence is unnecessary.  Since the logarithm would remove a skew in the input of numbers close to zero, the acquisition is in question.  Why are the values not often varying?  Are their transient events followed by inactivity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aggregating unchanged items in the sequence and adding duration as a feature may add entropy to the input and improve LSTM performance.  This is called decimation, and can be useful in adding entropy (removing redundancy) in data that has regions of redundant oversampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the values without the division are still distributed exponentially, then keep the logarithm but maybe make it a natural log.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-11-27T23:49:55.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9230" PostTypeId="1" CreationDate="2018-11-28T05:03:20.983" Score="1" ViewCount="100" Body="&lt;p&gt;I'm trying to replicate the results of the DeepMind's paper with Breakout included in OpenAI Gym. I wonder how much frames should I keep until I reach the fixed exploration rate. Actually it reaches the minimum value at about 1,5M frames (about 5000 episodes). It is much? If it is, does it impact negatively on training? My minimum rate is fixed at 0.01.&lt;/p&gt;&#xA;" OwnerUserId="9818" LastActivityDate="2018-11-28T05:03:20.983" Title="Exploration rate decay and training in Q learning" Tags="&lt;ai-design&gt;&lt;ai-basics&gt;&lt;training&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9231" PostTypeId="1" CreationDate="2018-11-28T09:04:56.163" Score="1" ViewCount="261" Body="&lt;p&gt;I'm working on stock price prediction and automatic or semi-automatic control of trading.  The price trends of these stocks exhibit recurring patterns that may be exploited.  My dataset is currently small, only in the thousands of points.  There are no images or very high dimensional inputs at all.  The system must select from among the usual trading actions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Buy &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; shares at the current bid price&lt;/li&gt;&#xA;&lt;li&gt;Hold at the current position&lt;/li&gt;&#xA;&lt;li&gt;Sell &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; shares at what the current market will bear&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if reinforcement learning is the best choice, deep learning is the best choice, something else, or some combination of AI components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't seem to me to be a classification problem, with hard to discern features.  It seems to be an action-space problem, where the current state is a main input.  Because of the recurring patterns, the history that demonstrates the observable patterns is definitely pertinent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried some code examples, most of which employ some form of artificial nets, but I've been wondering if I even need deep learning for this, having seen the question, &lt;a href=&quot;https://ai.stackexchange.com/questions/3002/when-is-deep-learning-overkill&quot;&gt;&lt;em&gt;When is deep-learning overkill?&lt;/em&gt;&lt;/a&gt; on this site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I have very limited training data, I'm not sure what AI design makes most sense to develop and test first.&lt;/p&gt;&#xA;" OwnerUserId="20257" LastEditorUserId="4302" LastEditDate="2018-11-29T19:27:05.387" LastActivityDate="2018-11-29T23:32:44.487" Title="For forecasting and trading control, given limited data, what AI approaches are well matched?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;forecasting&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9233" PostTypeId="2" ParentId="9231" CreationDate="2018-11-28T09:44:03.420" Score="1" Body="&lt;p&gt;Unclear from your description how RL is useful. RL is a technique that allows your model learn interactive by trial and error. Where is your &quot;trial and error&quot; in your problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stock price prediction sounds more like a regression problem to me. It can be done by DL or many other methods. Probably a shallow neural network would work well for you.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-11-28T09:44:03.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9234" PostTypeId="1" CreationDate="2018-11-28T09:50:09.747" Score="5" ViewCount="74" Body="&lt;p&gt;If my algorithm detects the type of object, how should I know if that object is moving or not? Suppose a person carrying an umbrella. How to know that the umbrella is moving?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a project where I want to know whether that particular object belongs to the person entering inside the store.&#xA;I was thinking about the bounding boxes(bb) approach where if the person's bb overlaps with the object's bb. But the problem arises when there are multiple objects with a person. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate your help. Thanks. &lt;/p&gt;&#xA;" OwnerUserId="20259" LastEditorUserId="20259" LastEditDate="2018-11-28T10:03:46.437" LastActivityDate="2019-01-26T18:29:27.137" Title="How to know whether the object is moving after it is being detected?" Tags="&lt;python&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9236" PostTypeId="1" CreationDate="2018-11-28T14:04:01.180" Score="3" ViewCount="101" Body="&lt;p&gt;Why do we apply the mutation operation after generating the offspring, in genetic algorithms?&lt;/p&gt;&#xA;" OwnerUserId="19603" LastEditorUserId="2444" LastEditDate="2019-11-22T00:56:12.947" LastActivityDate="2019-11-22T01:00:01.470" Title="Why do we apply the mutation operation after generating the offspring?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;mutation-operators&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9237" PostTypeId="2" ParentId="9236" CreationDate="2018-11-28T14:50:56.997" Score="4" Body="&lt;p&gt;The mutation operation is (usually) needed to introduce new genes not found in the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that you have 4 possible genes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, and that your chromosomes have a non-binary encoding. In that case, if no member of your population has the gene &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, then no amount of crossover operations will result in the introduction of that gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if your chromosomes use a binary encoding, then new genes could be introduced as a side effect of crossover operations. But it is always safer to have some kind of mutation to ensure that all genes can be accessed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the paper &lt;a href=&quot;https://www.hindawi.com/journals/cin/2017/7430125/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithm for Traveling Salesman Problem with Modified Cycle Crossover Operator&lt;/a&gt;, which mentions several non-binary encodings for the TSP.&lt;/p&gt;&#xA;" OwnerUserId="12857" LastEditorUserId="2444" LastEditDate="2019-11-22T01:00:01.470" LastActivityDate="2019-11-22T01:00:01.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9241" PostTypeId="2" ParentId="9050" CreationDate="2018-11-28T23:18:00.907" Score="1" Body="&lt;p&gt;The main categories that answer the question have many tributes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Academia&lt;/li&gt;&#xA;&lt;li&gt;Industry&lt;/li&gt;&#xA;&lt;li&gt;Social networks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are some of the options that roughly fit into these options which have not yet been mentioned in answers.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For those not registered as students, several university clubs are open to non-student members&lt;/li&gt;&#xA;&lt;li&gt;For those registered as students, several university professors would be willing to be the faculty adviser for a student led AI club and possibly excited about it&lt;/li&gt;&#xA;&lt;li&gt;FaceBook has several AI related groups&lt;/li&gt;&#xA;&lt;li&gt;YouTube has channels related to AI&lt;/li&gt;&#xA;&lt;li&gt;IEEE membership can be of benefit, and it is inexpensive if you are a registered student&lt;/li&gt;&#xA;&lt;li&gt;Association for Computing Machinery at acm.org, founded in 1947, and is the world's largest scientific and educational computing society&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Communing with those that have more experience should not be dismissed.  Some are looking to foster expertise in beginner enthusiasts.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Get a job in the field and finding people that will consider friendships that begin in professional environments&lt;/li&gt;&#xA;&lt;li&gt;Offer assistance at an AI lab, even if that means menial &lt;em&gt;gofer&lt;/em&gt; tasks at first, such as blogging to drive web traffic to their site or most recent publication&lt;/li&gt;&#xA;&lt;li&gt;Attendance at AI events online&lt;/li&gt;&#xA;&lt;li&gt;Travel and stay overnight at major AI events, attend the networking breakfasts, evening events, visit the exhibits, and bring business cards&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here in this AI StackExchange, ongoing dialog is prohibited in the main Q&amp;amp;A comments, but there are conversations in the space provided for that.  As it develops, a kind of community forms, and it has been maturing.  People get appreciative, annoyed, overly competitive, repentant, collaborative, friendly, inspired in their writing, and seeded with new ideas and understanding every day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people here occasionally ask to communicate outside this space or freely use their real names and photos.  Use discretion, but there is no rule against furthering contact if the feeling is mutual that I could find in the Help Center.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-28T23:18:00.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9242" PostTypeId="2" ParentId="8894" CreationDate="2018-11-28T23:36:13.520" Score="1" Body="&lt;p&gt;The terms &lt;em&gt;Supervised Learning&lt;/em&gt; and &lt;em&gt;Unsupervised Learning&lt;/em&gt; predate the invention of the application of artificial networks to a generative and discriminative network pair, which was the first popular generative topology.  The existence of labeling is the key distinction between the two.  Even partial labeling indicates supervision, as odd as that jargon is, since the supervisor does no learning and the labels are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the original description of the discriminative network in a GAN, that it consumes examples without labels, GANs are a type of unsupervised learning.  That fact does not eliminate the use of labels as part of an extension of the original design that generates based on some labeled element in the examples or the use of other labels to indicate the fitness of each image for a class into which generated images are expected to fall.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-28T23:36:13.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9243" PostTypeId="2" ParentId="9217" CreationDate="2018-11-29T00:29:36.057" Score="5" Body="&lt;p&gt;We can start with equation (30):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\bar{A}(s) = P(a \neq \tilde{a}) \mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [A_\pi(s, \tilde{a}) - A_\pi(s, a)]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking the absolute value of both sides, the equality remains true. We can pull the probability term out of the absolute value since it is guaranteed to be nonnegative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| = P(a \neq \tilde{a}) |\mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [A_\pi(s, \tilde{a}) - A_\pi(s, a)]|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By Definition 1, &lt;span class=&quot;math-container&quot;&gt;$P(a \neq \tilde{a}) \leq \alpha$&lt;/span&gt;. Substituting this definition in, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| \leq \alpha \cdot |\mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [A_\pi(s, \tilde{a}) - A_\pi(s, a)]|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By Jensen's Inequality, we can take the absolute value inside the expectation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| \leq \alpha \cdot \mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [|A_\pi(s, \tilde{a}) - A_\pi(s, a)|]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expectation of a random variable is always upper bounded by the max value of that variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| \leq \alpha \cdot \max_{a,\tilde{a}|a\neq\tilde{a}} |A_\pi(s, \tilde{a}) - A_\pi(s, a)|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This part is a little strange, and I'm not sure if this is the logic that the authors followed, but it is still true. For any &lt;span class=&quot;math-container&quot;&gt;$a,b$&lt;/span&gt;, we have that &lt;span class=&quot;math-container&quot;&gt;$|a - b| \leq |a| + |b|$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| \leq \alpha \cdot \max_{a,\tilde{a}|a\neq\tilde{a}} (|A_\pi(s, \tilde{a})| + |A_\pi(s, a)|)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$a \neq \tilde{a}$&lt;/span&gt;, it must be that either &lt;span class=&quot;math-container&quot;&gt;$|A_\pi(s, \tilde{a})| \geq |A_\pi(s, a)|$&lt;/span&gt; or vice versa. We can use this to replace the pair of advantage functions with 2 times the max of the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| \leq \alpha \cdot 2 \max_{a} |A_\pi(s, a)|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we take the max over &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, our inequality still holds, since we are taking the maximum over a set that contains &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Making this substitution gives us (31).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\bar{A}(s)| \leq \alpha \cdot 2 \max_{a,s'} |A_\pi(s', a)|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20274" LastActivityDate="2018-11-29T00:29:36.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9248" PostTypeId="1" CreationDate="2018-11-29T08:18:19.443" Score="1" ViewCount="116" Body="&lt;p&gt;We want to figure out the connection between people, based on their speech.&#xA;Assume, that a conversation is a poetry with lines belongs to characters. There are a lot of poetries and the lines are mixed. Now we want to define a conversation to which each line belongs to. &#xA;We assume, that people in a conversation use similar words (their dictionaries should be similar). It means that there is a correlation between words of person A and words belongs to person B, and we could detect a connection between the peoples which had a conversation. What are the next steps for content understanding after NLP?&#xA;Can some of you advise us about the field of study and tools/libraries, which are dealing with content processing? Maybe, some of you know good articles or online resources, which can help us to dive into this field.&lt;/p&gt;&#xA;" OwnerUserId="20267" LastEditorUserId="1671" LastEditDate="2018-11-29T22:22:11.477" LastActivityDate="2020-05-02T01:03:55.467" Title="Content analysis tools" Tags="&lt;natural-language-processing&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-18T16:26:43.270" ContentLicense="CC BY-SA 4.0" />
  <row Id="9249" PostTypeId="2" ParentId="9248" CreationDate="2018-11-29T09:49:03.310" Score="0" Body="&lt;p&gt;These are really two questions -&lt;br&gt;&#xA;1. How to assign lines in a dialogue to particular speakers&lt;br&gt;&#xA;2. How to analyse the content of a speaker's utterances&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will attempt to answer the first one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Discourse_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;discourse analysis&lt;/a&gt; (the field that studies conversations), there is the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Adjacency_pairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;adjacency pairs&lt;/a&gt;, linked utterances which typically follow on to each other. A well-formed conversation can be segmented into a series of adjacency pairs, such as greeting/greeting, question/answer/feedback (that's more of an 'adjacency triple'), statement/comment, farewell/farewell. Sometimes it's a bit more tricky, such as question/counter-question/answer/answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;young person: &lt;em&gt;What beers do you have?&lt;/em&gt; (Q)&lt;br&gt;&#xA;bartender: &lt;em&gt;How old are you?&lt;/em&gt; (Counter-Q)&lt;br&gt;&#xA;young person: &lt;em&gt;21&lt;/em&gt; (A)&lt;br&gt;&#xA;bartender: &lt;em&gt;We have a lager or a nice IPA.&lt;/em&gt; (A)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if this has been tried before, but it should be comparatively easy to identify the types of &lt;a href=&quot;https://en.wikipedia.org/wiki/Speech_act&quot; rel=&quot;nofollow noreferrer&quot;&gt;speech acts&lt;/a&gt; in a conversation and join them up to form relevant adjacency pairs. This would be a lot more reliable than looking at vocabulary, especially since conversations mostly use higher-frequency words which are part of everyone's active vocabulary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would need to take some conversations, classify the individual lines as speech acts (building up an inventory as you do that), and then identify adjacency pairs. This will give you the structure of the conversation, and should also allow you to assign speakers to utterances (assuming a dialogue, ie only two participants). Note that it is not always purely form that determines the speech act: &lt;em&gt;How are you?&lt;/em&gt; is a question on the surface, but its function is part of a greeting sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This conversational structure could also assist you in looking at the second part, the content. If you know someone asks questions, and the other participant answers them, then you know exactly where to look for salient utterances that might give you clues as to the topic of the conversation. At least it should enable you to ignore the 'housekeeping' utterances like greetings and farewells etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your second question is hard to answer without knowing more about the purpose of your project. I would rephrase this and ask it as a separate question. People would need to know what kind of conversations they are, and what exactly you want to get out of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh, and I wouldn't compare conversations with poetry. Completely different things!  :)&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-11-29T09:49:03.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9251" PostTypeId="2" ParentId="9180" CreationDate="2018-11-29T16:31:26.773" Score="2" Body="&lt;p&gt;This would be a pretty fun opportunity to experiment a bit. I know there's been breakthroughs in OCR technology by training the network on images that have been altered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long story short, there was a pre-process that took the source images and did minor manipulations to them. This involved skewing, rotating, changing resolution &amp;amp; adding artifacts into the image to introduce &quot;Noise&quot;. This allowed the network to learn to recognize characters more accurately. It may be worthwhile to create an image pre-processing pipeline that will do a variation of modifications to the source images. When training you may want to lower your learning rate &amp;amp; split up your data into 70% train &amp;amp; 30% test. Then you can loop through all the images repeatedly until you hit a maximum for the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples&#xA;&lt;a href=&quot;https://matthewearl.github.io/2016/05/06/cnn-anpr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://matthewearl.github.io/2016/05/06/cnn-anpr/&lt;/a&gt;&#xA;&lt;a href=&quot;https://medium.com/@shreyas.s/image-data-generation-for-optical-character-recognition-ocr-9b19300649c8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@shreyas.s/image-data-generation-for-optical-character-recognition-ocr-9b19300649c8&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1720" LastEditorUserId="1720" LastEditDate="2018-11-29T16:57:37.127" LastActivityDate="2018-11-29T16:57:37.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9252" PostTypeId="2" ParentId="8940" CreationDate="2018-11-29T19:48:31.380" Score="1" Body="&lt;p&gt;CNBC.com states, &quot;Tara AI faces competition from large tech consultancies like Tata or Insight Global, and freelancer and developer marketplaces online like Upwork, Codementor and Gigster.&quot;  For this reason, how Tara AI works can only be conjecture, as they would be doing all they can to plug any leaks of company confidential information.  There won't be any papers coming from their AI experts that describe their matching algorithms and data collection and normalization techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matching a worker to a project is a matter of scanning a large number of documents about workers with a requisition from the recruiter or recruiters assigned to staff the project.  The approaches to apply AI to this problem that could produce favorable results are numerous.  It is likely that many of these organizations listed by CNBC.com like to say to experts, recruiters, and project management that they use AI in the matching, even if they are merely using a probabilistic model based on three things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Frequency of words on the web (ambient word frequency)&lt;/li&gt;&#xA;&lt;li&gt;Frequency of words in the resumes&lt;/li&gt;&#xA;&lt;li&gt;Frequency of words in the requisition&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Beyond that high level description, I too am constrained by information privacy agreements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly, semantic nets is another approach that may be in use in some of these organizations, although there is much to do in the field of AI, with the mathematics, the hardware, and the software, before we have cognition components to use that actually understand what, &quot;Full Stack Java Developer,&quot; means to the degree that the expert and the team leader who writes the requisition understand it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a set of tricks people use, like the tricks that are used in 3D rendering.  The pixels don't have to be perfect ray traced from every light source and reflection for every frame in the render.  Just close enough so that viewers are not distracted by digital artifacts is sufficient.  With matching people with roles, there is no way to gather enough data to perform an highly reliable prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the complexities of a billion brain cells times however many people there are on the project team, some people just fit into the role in spite of predictors otherwise, and vice versa.  That is simple predictive tricks may produce a reliability of placement very close to the use of a deep learner running two floors of data center.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-11-29T19:48:31.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9253" PostTypeId="1" CreationDate="2018-11-29T20:33:25.790" Score="2" ViewCount="45" Body="&lt;p&gt;Most implementations I'm seeing for playing games like Atari (usually similar to DeepMind's work using DQN) have 4 graphical frames of input fed into 3 convolutional layers which are then fed into a single fully connected layer. &lt;strong&gt;The explanation of no pooling layer is due to positioning of features/objects being very critical to most games.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern with this is that it may be weighing visual features based on position without regard for feature-&gt;feature proximity. By this, I mean to question if learning to avoid bullets in the bottom left of the screen is knowledge also used in the bottom right of the screen in a game like Space Invaders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, question 1: Is my concern with only using 3 conv layers into a fc layer legitimate regarding spatially localized learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2: If my concern is legitimate, how might the network be modified to still treat feature position as significant, but to also take note of feature to feature proximity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm still quite the novice if that isn't extremely obvious, so if my questions aren't completely ridiculous on their own, please try to keep responses relatively high level if you would.)&lt;/p&gt;&#xA;" OwnerUserId="20277" LastActivityDate="2018-11-29T20:33:25.790" Title="How a game playing agent could identify potential objects and proximity?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9254" PostTypeId="2" ParentId="9231" CreationDate="2018-11-29T22:59:03.207" Score="3" Body="&lt;p&gt;This is an interesting problem, the answer of which is highly coveted for obvious reasons.  The production of an answer in this public space is appropriate, provided one believes in a more level distribution of assets across those alive and is comfortable with rewarding technical people for the abilities they developed while others might have been engaging in less technical pursuits.  I only ask that those who may successfully employ it consider using at least some of any windfall generated in a philanthropic manner, to benefit the less fortunate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here we go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Abstract Algebra&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the fundamental principles presented in Morgenstern and von Neumann's &lt;em&gt;Theory of Games and Economic Behavior&lt;/em&gt; is the idea that the value of a monetary unit may vary based on actions of those that control the monetary system, but the underlying value itself has abstract algebraic properties favorable to prediction.  These properties, the book proposes in early chapters, permit additive modelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explained another way, imagine a pile of paper bills of a particular denomination.  The selection of any one of the bills from anywhere in the deque is identical in value to the selector as any other selection of one bill.  The selection of which particular bill is also not a factor in the value remaining in the deque after the selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is also true of shares, which is why serial numbers are not a trading parameter at the investor-trade interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more theory behind this, the interested parties can study group theory and rings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Critique of M&amp;amp;VN&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This convenient additive view of value beneath monetary systems has been critiqued.  The value of assets, products, and services may be demonstrated to vary with life expectancy and philosophic divergence.  While some people fall into an investment mode, others may be divesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, similar to Isaac Asimov's fictional presentation of psycho-history, the predictive mechanisms that may be developed are, in fact, not fettered by this investment versus divestment mode variance.  The volume of trading and number of traders typically hide the variance through cancellations across either side of mean behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those with considerable assets can exploit this caveat and a large number of other caveats in the additive nature, but most traders cannot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Patterns in Stock Values&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phrase that may have started a number of investment firms down the path toward the use of machine learning in high speed training, leading to a conspicuous growth in the number of billionaires in the world, is a simple phrase from Darren Aronofsky's &lt;em&gt;Pi&lt;/em&gt;, a 1998 movie in black and white.  The protagonist, Maximillian Cohen, looks at the scrolling LED version of a ticker tape and says, &quot;I know these numbers,&quot; and then begins to verbally state each price just before it appears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The patterns in stock prices have intrigued those who watch trends since the emergence of the New York Stock Exchange.  But they are not cyclic, in spite of the naive belief that they run in cycles.  They only appear to run in cycles, just as one might find a pattern they think they understand in a graphic representation of the &lt;em&gt;Mendelbrot Set&lt;/em&gt;, but when the renderer of the set zooms in or out, the greater detail or the generality respectively reveal that the understanding was an illusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attempts to use Fourier Analysis as part of a predictive strategy for anything but extremely narrowly timed speculation have been of limited value.  The reliability of the prediction of advantageous trading choice, expressed as a probability &lt;span class=&quot;math-container&quot;&gt;$P(C)$&lt;/span&gt;, dwindles to pure entropy (zero bits of actual predictive information) as the clock time of the prediction increases relative to that of the last time in the complete vector of state historic states &lt;span class=&quot;math-container&quot;&gt;$\vec{S}$&lt;/span&gt; used for prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{(t - t_\emptyset) \rightarrow \inf} P(A|\vec{S}) \rightarrow 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In plain terms, the risk of trading based on a predictive model from past price trend increases with the staleness of trend information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The increase in entropy is likely to be a feature of the prediction of any system as complex as the stock market even as prediction tools mature.  It is a consequence of initial input sensitivity combined with measurement inaccuracy and the inevitable unsatisfactory distributed model of decisioning.  These constraints may not be inescapable, but they are certainly resistant to brute force attacks, such as adding artificial network layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are tools that can be predictive in ways better than trying to identify a spectrum using Fourier Analysis.  But let's first consider a spectrum, which is how music listeners identify the pitch, loudness, timbre, and envelope of notes. &#xA;Unlike the playing and listening to music, the distribution of pitches in the spectrum from moment to moment are not purely a representation of the spectral distribution in which value fluctuations reside.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spectral analysis of stock prices reflects artificial bounds.  The spectrum is bounded by the market sampling rate and Nyquest's criteria on the high frequency side.  It is bound by the accumulation of additive effects that tends price fluctuation toward a Gaussian distribution on the low frequency end of the spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Three-fold Challenge of Freshness, Accuracy, and Complexity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some may note that the intention of the question, to control (with or without human moderation) a stock portfolio to maximize profit is a planning problem that appears to have the Markov Property, provided the trader is not wealthy or trading in penny stocks, since the impact of previous trades seems unlikely to have significantly impacted market trends.  As we shall see, this appearance may not be objectively true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial input sensitivity is a nearly universal quality of complex systems.  In the case of the stock market, the complexity is high.  Consider a cursory examination of the magnitude of market complexity, understanding that a single price is affected by neighboring prices in that sector and business and in the major market indicators that people watch.  A stock is almost completely isolated in terms of trading actions but inherently tied to all traders and other stocks through an interconnected network of IT pipelines, trading interfaces, news events, and brains, each containing a few billion neurons with complexities much greater than artificial cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A popular name for initial input sensitivity is the &lt;em&gt;Buterfly Effect&lt;/em&gt;, which, properly explained is this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Whether a butterfly flaps its wings in Brazil may determine the path of a major storm in Peking a month later.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sometimes the initial word, &quot;Whether,&quot; is not included, which renders the explanation incorrect.  It is a statement of the effect of whether or not the perturbation occurs.  Once cannot say that the air movement from the wing flap causes the storm but rather the presence of a small perturbation may be grossly amplified over time in a complex system.  Or it may not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Studies of chaotic systems have proven this potential of amplified effect as a characteristic of many if not most complex systems of differential equations with sufficient curvature in their behavioral surface to cause a system phenomena that could most descriptively be called, &quot;Radically increasing bifurcations.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those seeking greater depth of understanding, the work of Fourier, Nyquist, Cantor; G. Julia and P. Fatou; and Aleksandr Lyapunov may be of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of initial input sensitivity, the Markov Property cannot be guaranteed for stock trading, even at low levels of investment.  Even though, technically, the Markov Property is not guaranteed, its assumption at low trading levels and in the leveraging of fresh data for short term gains, it may lead to reasonable approximations with sufficient reliability to win at the negative sum game of trading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Trading as a Below Zero Sum Game&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a zero sum game, those trading with stale information and trying to apply ineffective predictive strategies will, over any longer period lose. Their monetary assets will pass to those with fresher information using effective predictive strategies.  With trading fees, the condition is more challenging, because the sum of all interactions is below zero by the sum of the fees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Gaining Advantage&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To gain advantage in any practical way above those who may have already employed machine learning in their trading operations or hired a funds manager that already did, there will have to be some clever application of more advanced principles in a way that drives effectiveness even higher than those already in game play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some key elements.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since the windowing over the price sequence of any LTS (liquid tradable security) to derive a spectrum will produce spectra with no particularly improved predictive power than the price sequence in the time domain, the goal must be to discover the nature of the patterns beyond spectral features.&lt;/li&gt;&#xA;&lt;li&gt;For very short duration investments, spectral features may provide some advantage but perhaps not enough to overcome others using similar predictive strategies.&lt;/li&gt;&#xA;&lt;li&gt;Some stocks are predictors of others in that there is a strong unidirectional or bidirectional causal relationship.  For instance, the cost of a barrel of oil impacts the future cost of toys, since plastic is a refinery product.  The development of a semantic net with causal relationships modeled and tuned with each epoch of training can produce predictive power that may exceed the market norm.  In this context, something like convolution may be advantageous, but not in the frame-horizontal-vertical-pixel-layer sense, since semantic networks have a non-cartesian topology.  The edges and vertices in a semantic net of LTSs don't make nice 90 degree angles that lend themselves to trivial applications of loops and arrays.&lt;/li&gt;&#xA;&lt;li&gt;Attractors, another analytical concept from the world of chaos, are an analytical tool that lend themselves to trends in stock groups and possibly individual stocks, however diversification principles of investment also apply well to attractors.  The thing to study is auto-correlation in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{(m n)}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of features in the stock statistics for which there is a fresh historical record and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of LTSs in the group.  The group is not a trench, with varying risk pooled together to provide a favorable curve of return versus stability.  The group is a set of LTSs that correlated positively or negatively and with or without relative temporal shift (delay).&lt;/li&gt;&#xA;&lt;li&gt;Leverage factors outside the exchange.  For instance, the result of a close election is a key event that impacts markets.  Terrorism is.  Experts who may know nothing about where the market is going but sound good can vastly sway trends.  Some billionaire somewhere may have rode a wave in a temporary correlation between the price of oranges and the price of gold, floating down the eastern seaboard in a yacht, not having the faintest idea why oranges correlated to gold, but analysis showed too many correlative effects to deny the tie, and they rode it until the model broke.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Project Strategy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is recommended NOT to test with actual trades.  Build a system and create a simulated portfolio.  Do not use retirement funds and college funds to test a system pitted against some of the best trading automation in the world funded by a few hundred billionaires.  You will end up in Gamblers Anonymous that way.  Also understand that, as time passes, the bar will rise, and more people will be trading using predictive systems so that what is a profitable system today may consume all your time and energy and eventually lose everything you gained as other systems gain advantage over yours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also remember that asset management, in the final summary, is a zero sum game.  Trading gains are inescapably stuck inside the larger game where the pieces are placed back in the box and the box back in the closet.  This is true even if you have children or grandchildren.  Eventually, chaos has its sway.  Our great, great, great, great grandchildren, who probably won't care to know our name, might end up poorer because we were wealthy or wealthier because we were poor.  It's the Butterfly Effect again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are infinitely more important things than asset accumulation.  Ask any honest billionaire with an advanced terminal condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Layer Depth?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes depth.  There is no way to meaningfully model even the most banal aspects of the complexity of markets with a few layers of perceptrons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Real Time Learning?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement, yes, because the Q-learning algorithm class and variants are a powerful tool for re-entrant, real time learning system design.  However, be careful to not necessarily assuming the Markov Property.  Whether you can use an algorithm, right out of the box, wired directly to inputs from market data with no normalization and prevail in a sub-zero sum game is unlikely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Innovation will be a requirement to any earned and even short term sustainable success.  Always, with the application of these powerful AI components, thoughtfully apply what is known to the design, initialization, and further development of the system into which the component fits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another network type worthy of consideration with time series is the LSTM network design.  This is where layer depth makes sense, however systems with multiple AI components are really the work horses of the industry today.  Very few problems give way easily to the deployment of some code generated from one algorithm specified in one paper and nothing added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;System Topology&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is likely to be multiple pipelines containing information in a system that sustainably provides growth-positive automated or semi-automated portfolio management, even among groupings of LTSs that seem to display patterned and more easily predictable trends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one wishes to get a general sense of the complexity of system design that may be required to beat those at the forefront of the investment community, visit a nuclear power plant or disassemble the control system of the jet engine of a passenger jet.  Still, someone seems to be getting it right.  A suspiciously large number of new billionaires have emerged since 1998 with &lt;em&gt;investing&lt;/em&gt; given as the industry in their Forbes entry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional technical information is provided in &lt;a href=&quot;https://ai.stackexchange.com/questions/3233/use-of-machine-learning-for-analyzing-companies-enlisted-in-stock-market/3649#3649&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-29T23:32:44.487" LastActivityDate="2018-11-29T23:32:44.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9255" PostTypeId="5" CreationDate="2018-11-29T23:14:18.520" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-29T23:14:18.520" LastActivityDate="2018-11-29T23:14:18.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9256" PostTypeId="4" CreationDate="2018-11-29T23:14:18.520" Score="0" Body="For questions of the fallibility of AI strategies or systems in a particular development process or field use, such as whether a trained network will fail to achieve key objectives at the time of deployment or afterward, whether a particular criteria used to design an AI solution will fail to produce a cost effective system, or whether a particular algorithm is guaranteed to provide the desired outcome based on a set of determining factors." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-30T18:27:38.620" LastActivityDate="2018-11-30T18:27:38.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9257" PostTypeId="5" CreationDate="2018-11-29T23:19:42.280" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-29T23:19:42.280" LastActivityDate="2018-11-29T23:19:42.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9258" PostTypeId="4" CreationDate="2018-11-29T23:19:42.280" Score="0" Body="For questions related to forecasting of any type beyond basic forms of extrapolation, as applicable in fields such as financial planning, portfolio management tooling, automated driving or piloting collision avoidance (when the trajectories are not constant speeds along straight lines), weather science, climate projection as a function of carbon emissions, and such. " OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-30T18:27:58.113" LastActivityDate="2018-11-30T18:27:58.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9259" PostTypeId="5" CreationDate="2018-11-29T23:41:39.853" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-29T23:41:39.853" LastActivityDate="2018-11-29T23:41:39.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9260" PostTypeId="4" CreationDate="2018-11-29T23:41:39.853" Score="0" Body="For questions related to AI theory that relies on the knowledge of a distribution of probabilities across one or more dimensions affecting probability. Such a distribution may be in discrete buckets, such as quartile, octile, or percentile conventions or continuous functions based on some closed form (algebraic formula). Distributions of probability are key in planning, natural language handling, and other AI objectives." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-30T18:28:16.677" LastActivityDate="2018-11-30T18:28:16.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9261" PostTypeId="5" CreationDate="2018-11-29T23:46:19.473" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-29T23:46:19.473" LastActivityDate="2018-11-29T23:46:19.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9262" PostTypeId="4" CreationDate="2018-11-29T23:46:19.473" Score="0" Body="For questions about cognition, an umbrella term for a number of mental features of the human brain and brains of animals that are associated with intelligence, especially related to beliefs, logical inference, development and acceptance or rejection of ideas, and formulations of complex models of the environment for prediction, control, and risk reduction purposes and softer human characteristics of competition, collaboration, and other social thinking." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-30T18:29:01.800" LastActivityDate="2018-11-30T18:29:01.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9263" PostTypeId="2" ParentId="154" CreationDate="2018-11-30T00:22:23.530" Score="0" Body="&lt;p&gt;There's the fairly well established field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_theorem_proving&quot; rel=&quot;nofollow noreferrer&quot;&gt;automated theorem proving&lt;/a&gt;. This most likely encompasses solving equations, but doesn't necessarily involve AI. &lt;a href=&quot;https://stats.stackexchange.com/questions/262390/application-of-machine-learning-to-the-automated-theorem-proving&quot;&gt;This post from the Cross Validated stackexchange&lt;/a&gt; has some more information on the topic.&lt;/p&gt;&#xA;" OwnerUserId="15223" LastActivityDate="2018-11-30T00:22:23.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9265" PostTypeId="5" CreationDate="2018-11-30T01:36:24.420" Score="0" Body="&lt;p&gt;An older term, based on the presumption that rules are the primary representation of knowledge, is, &quot;Knowledge acquisition.&quot;  In the context of AI research with a broader perspective, rule acquisition is a subset of knowledge acquisition.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-30T18:29:11.907" LastActivityDate="2018-11-30T18:29:11.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9266" PostTypeId="4" CreationDate="2018-11-30T01:36:24.420" Score="0" Body="For questions related to the acquisition of rules that describe conditions and consequences related to actions, objects, or complex behaviors with or without certainty metrics. This may be from constructive algorithms that search for rules describing observations or from the translation of observations directly into logical constructs via circuitry." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-11-30T18:28:32.397" LastActivityDate="2018-11-30T18:28:32.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9267" PostTypeId="5" CreationDate="2018-11-30T01:42:24.687" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-11-30T01:42:24.687" LastActivityDate="2018-11-30T01:42:24.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9268" PostTypeId="4" CreationDate="2018-11-30T01:42:24.687" Score="0" Body="For questions related to the automated construction of procedures and methods that conform to the criteria typically characterizing an invention, as expressed in patent law. " OwnerUserId="-1" LastEditorUserId="1671" LastEditDate="2018-11-30T18:34:01.807" LastActivityDate="2018-11-30T18:34:01.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9269" PostTypeId="2" ParentId="6952" CreationDate="2018-11-30T05:41:19.523" Score="2" Body="&lt;p&gt;The keyword here is Parameter Sharing or Weight Sharing across various image portions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take a simple example of grayscale binary image of an alphabet 'F', it is a combination of multiple patterns. The patterns here are vertical lines and horizontal lines. These patterns are based on relation between intensities of contiguous cells. This relation between contiguous cells is established using a weight matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for identifying multiple horizontal lines, we dont need multiple node-sets in hidden dense layer trying to identify different horizontal lines in the image. The pattern is same but present in different locations. Hence the sharing of weights comes into picture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 1st hidden layer, encode the pattern horizontal line in a weight matrix(learnt during training and used in testing). Place it over small grid and check for presence. As this matrix is slided and tested across the image, the presence of horizontal lines is marked in various locations. This weight matrix is called a kernel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combining the above points, kernel provides a way of handling parameter / weight sharing between contiguous cells to identify patterns. Dense layer instead of kernels would solve it eventually but start in a random manner. Since a efficient way was identified, it is being used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next to identify vertical lines, another kernel needed and slide across.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose next we have dense layer as 2nd hidden layer. This layer looks for combination of patterns ('p' horizontal lines and 'q' vertical lines in this case for 'F') present and learns combinations to identify output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to compare with traditional programming, kernels are like regular expressions. dense layers are like loops. Just sharing my thoughts. Any better explanation is welcome.&lt;/p&gt;&#xA;" OwnerUserId="16783" LastActivityDate="2018-11-30T05:41:19.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9270" PostTypeId="1" CreationDate="2018-11-30T06:03:21.530" Score="1" ViewCount="379" Body="&lt;p&gt;I am having a video feed with multiple faces in it. I need to detect each face and the gender as well and assign the gender against each person. I am not sure how to uniquely identify a face as Face1 and Face2 etc. I do not need to know their names, just need to track a person in the entire video. I thought of tracking methods but people are constantly moving and changing so a box location can be occupied by another person after some frames. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in a way where I can assign an id to a face but I am not sure how to do it. I can use Facial Recognition Based embedding on each face and track that. But that seems to be an overkill for the job. Is there any other method available or Facial Recognition/Embedding is the only method to uniquely identify people in a video?&lt;/p&gt;&#xA;" OwnerUserId="11387" LastActivityDate="2021-01-18T12:55:13.160" Title="Can I detect unique people in a video?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9271" PostTypeId="2" ParentId="9270" CreationDate="2018-11-30T06:41:15.710" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Can I detect unique people in a video?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. However, for some applications commercial systems are not good enough: &lt;a href=&quot;https://netzpolitik.org/2017/ueberwachungslabor-berlin-suedkreuz-tracking-und-gesichtserkennung-geplant/&quot; rel=&quot;nofollow noreferrer&quot;&gt;German source&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The concepts you are looking for are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Optical_flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optical Flow&lt;/a&gt;: for two frames, define for each point how it moved&lt;/li&gt;&#xA;&lt;li&gt;Object tracking: given a single object in one frame, find it in the next. Optical flow helps with that.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.pyimagesearch.com/2018/11/26/instance-segmentation-with-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Object detection&lt;/a&gt;: look at YOLO / SSD. Small introduction &lt;a href=&quot;https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="32410" LastEditDate="2021-01-18T12:55:13.160" LastActivityDate="2021-01-18T12:55:13.160" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9272" PostTypeId="2" ParentId="9270" CreationDate="2018-11-30T06:42:08.413" Score="1" Body="&lt;p&gt;You can detect faces in video frames by employing the standard method of Viola-Jones detector (though there are sophisticated ones available now). You can get the paper from &lt;a href=&quot;https://ieeexplore.ieee.org/document/990517&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. All the standard image processing libraries will have it implemented. You have to use the necessary API of that library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give a bounding box around each of the detected face (it may miss out a few faces which are at different poses or illumination). You can now encode the detected face, inside the bounding box, into a vector using the features given by &lt;a href=&quot;https://link.springer.com/article/10.3103%2FS0146411617010072&quot; rel=&quot;nofollow noreferrer&quot;&gt;face2vec&lt;/a&gt;. In the face2vec model remove the classification part alone. This will give you a feature vector of size 905 dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now given two face detections, you can find the euclidean distance between their feature vectors. If it is below a threshold say &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; then, you have detected same face, else you have detected two different faces.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2018-11-30T06:42:08.413" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9273" PostTypeId="2" ParentId="6952" CreationDate="2018-11-30T07:03:11.833" Score="2" Body="&lt;p&gt;Convolution Neural Networks can detect more of the spatial features compared to Densely Connected Network. Consider this in any given real world image the pixel values of neighboring cells to do not vary  highly, But when this image are passed to a Densely Connected Neural network for training the spatial relations between neighboring pixels is lost as all other cells can heavily influence the training whereas in Convolutional networks due to operation of convolution by local information is preserved, it is called local connectivity.&lt;/p&gt;&#xA;" OwnerUserId="11256" LastActivityDate="2018-11-30T07:03:11.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9275" PostTypeId="1" CreationDate="2018-11-30T07:19:37.077" Score="0" ViewCount="71" Body="&lt;p&gt;Can someone suggest an AI approach to moving blocks, one at a time, assuming control of an robotic arm, to get from the initial state on the left to the final state on right, preferably using goal stack planning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;actions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Pickup() &amp;mdash; to pick up a block from table only&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Putdown &amp;mdash; to putdown a block on table only&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unstack &amp;mdash; unstack a block from another block&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stack &amp;mdash; stack a block on another clear block only&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;property functions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On(x,y)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Above(x,y)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Table(x)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Clear(x)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8MRUp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8MRUp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20301" LastEditorUserId="4302" LastEditDate="2018-12-01T19:25:04.627" LastActivityDate="2020-11-20T08:07:05.793" Title="block of worlds with position aware goal stack planning" Tags="&lt;ai-design&gt;&lt;path-planning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9276" PostTypeId="1" CreationDate="2018-11-30T07:38:52.187" Score="3" ViewCount="313" Body="&lt;p&gt;I have to build a KI for a made-up game similar to chess. As I did research for a proper solution, I came upon the MinMax algorithm, but I'm not sure it will work with the given game dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge is that we have far more permutations per turn than in chess because of these game rules.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Six pieces on the board, with different ranges.&lt;/li&gt;&#xA;&lt;li&gt;On average, there are 8 possible moves for a piece per turn.&lt;/li&gt;&#xA;&lt;li&gt;The player can choose as many pieces to move as he likes. For example none, all of them, or some number in between (whereas in chess you can only move one.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Actual questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it feasible to implement MinMax for the described game?&lt;/li&gt;&#xA;&lt;li&gt;Can alpha-beta-pruning and a refined evaluation function help (despite the large number of possible moves)?&lt;/li&gt;&#xA;&lt;li&gt;If no, is there a proper alternative?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20303" LastEditorUserId="2444" LastEditDate="2020-06-10T17:10:59.767" LastActivityDate="2020-07-10T18:04:40.443" Title="Is it feasible to use minimax to solve a board game with a large number of moves?" Tags="&lt;game-ai&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9278" PostTypeId="1" CreationDate="2018-11-30T10:26:11.017" Score="1" ViewCount="173" Body="&lt;p&gt;Which one gives better optimization results? Genetic Algorithm or Particle Swarm Optimization? Can I use them for online tuning problems? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="20300" LastActivityDate="2018-11-30T14:08:29.693" Title="Genetic Algorithm vs Particle Swarm Optimization" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;control-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9279" PostTypeId="2" ParentId="9276" CreationDate="2018-11-30T11:12:53.700" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;-The player can choose as many pieces to move as he likes. For example none, all of them, or some number inbetween. (Whereas in chess you can only move one)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That quote specifically is the part that really causes the size of your legal action set to blow up. You have a combinatorial action space here. If each of your pieces has 8 legal moves, then that is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;8 legal moves for the first piece (or 9 if that count didn't already include the &quot;do nothing&quot; option)&lt;/li&gt;&#xA;&lt;li&gt;for each of those, there are again 8 or 9 different choices for the second piece (leading to e.g. &lt;span class=&quot;math-container&quot;&gt;$8 \times 8 = 64$&lt;/span&gt; possible combinations for just the first two pieces)&lt;/li&gt;&#xA;&lt;li&gt;for each of those, again 8 choices for the third piece (leading to &lt;span class=&quot;math-container&quot;&gt;$64 \times 8 = 512$&lt;/span&gt; possible combinations for just the first three pieces).&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This blows up way too quickly, and there's really no hope of ever getting a decent player for this using any MiniMax-based algorithm (including things like alpha-beta pruning, principal variation search etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the kinds of games that you describe, you'll want to use algorithms that can exploit the &quot;structure&quot; of your action space. A raw enumeration of all possible combinations blows up quickly, but many algorithms can do reasonably well by re-phrasing the problem in such a way that you have more &quot;depth&quot; rather than &quot;breadth&quot;. For example, instead of viewing a full combination of choices for all pieces as a single &quot;action&quot;, you can treat the choices per piece as a separate &quot;action&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than making a single choice out of &lt;span class=&quot;math-container&quot;&gt;$8 \times 8 \times 8 \times \dots$&lt;/span&gt; possibilities every turn, you want to have a search tree where your player makes one choice out of &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; (for the first piece), followed immediately by another choice out of &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; (for the second piece), etc. The opposing player only gets to make a choice after the current player has made choices for all pieces. With such a strategy, the breadth of your search tree will no longer be a problem, but the depth will become a problem. To address this, you'll additionally want to make sure that your methods can generalize across different depth levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good place to look would be combinatorial versions of Monte-Carlo Tree Search, such as those described in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://project.dke.maastrichtuniversity.nl/games/files/msc/Roelofs_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://project.dke.maastrichtuniversity.nl/games/files/msc/Roelofs_thesis.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.jair.org/index.php/jair/article/view/11053&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.jair.org/index.php/jair/article/view/11053&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(probably a few other publications by the author of that second link)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These algorithms are quite a bit more complicated than MiniMax though, MiniMax is a very basic algorithm in comparison.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-11-30T11:12:53.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9280" PostTypeId="2" ParentId="9278" CreationDate="2018-11-30T14:08:29.693" Score="2" Body="&lt;p&gt;These kind of questions cannot be answered without looking at a particular project. Each algorithm has its particular strengths and weaknesses; and trade-offs in terms of use of resources (processing power and/or storage space, for example). If there was an objective answer, then the worse algorithm would surely fall in disuse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also depends what you mean by &quot;better&quot;. Faster? Better score according to some evaluation measure? More robust (ie works with many diverse data sets)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend looking at both algorithms in more detail, and trying to understand how they work. Then you should be able to find out which best fits your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one problem with &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle_swarm_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Particle Swarm Optimisation&lt;/a&gt; is that it is not well understood, so you might have to resort to trial-and-error.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-11-30T14:08:29.693" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9281" PostTypeId="5" CreationDate="2018-11-30T18:41:30.160" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-30T18:41:30.160" LastActivityDate="2018-11-30T18:41:30.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9282" PostTypeId="4" CreationDate="2018-11-30T18:41:30.160" Score="0" Body="Use for questions related to AI research and development for the computer game StarCraft (typically StarCraft II)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-11-30T18:41:30.160" LastActivityDate="2018-11-30T18:41:30.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9283" PostTypeId="2" ParentId="9173" CreationDate="2018-11-30T19:35:48.240" Score="1" Body="&lt;p&gt;It looks like on each step, you're calling both &lt;code&gt;self.model.predict&lt;/code&gt; and &lt;code&gt;self.model.fit&lt;/code&gt;. If you do this, you're going to run into stability problems, since your learning target is moving as you train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way the DQN paper gets around this problem is by using 2 Q-networks, &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{Q}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\hat{Q}$&lt;/span&gt; is called the target network. The target network's parameters are frozen, and its outputs are used to compute the learning targets for &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; (&lt;code&gt;targets_future&lt;/code&gt; in your code). Every &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; training steps (where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a hyperparameter), the target network &lt;span class=&quot;math-container&quot;&gt;$\hat{Q}$&lt;/span&gt; is updated with the weights of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;. See Algorithm 1 on Page 7 of the &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN paper&lt;/a&gt; for the details of this swap.&lt;/p&gt;&#xA;" OwnerUserId="20274" LastActivityDate="2018-11-30T19:35:48.240" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9285" PostTypeId="2" ParentId="1507" CreationDate="2018-12-01T01:27:09.263" Score="4" Body="&lt;p&gt;This is an &quot;in human language&quot; (non-technical;) synopsis of the core of &lt;a href=&quot;https://ai.stackexchange.com/a/1516/1671&quot;&gt;Kaiesh&lt;/a&gt;'s excellent answer.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the most basic sense, any decision making algorithm can be regarded as a form of Artificial Intelligence. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/History_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;History of Artificial Intelligence wiki&lt;/a&gt; gives a pretty good overview.  The roots of the field are generally ascribed to &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symbolic Artificial Intelligence&lt;/a&gt;, but it might be said to go back as far as &lt;a href=&quot;https://en.wikipedia.org/wiki/Analytical_Engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Babbage&lt;/a&gt;. The first functional game AI in the form of an &quot;analytic engine&quot; may be &lt;a href=&quot;https://fr.wikipedia.org/wiki/Nimatron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nimatron&lt;/a&gt; (&lt;a href=&quot;https://patents.google.com/patent/US2215544A/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1940&lt;/a&gt;). More recently, Machine Learning in all of its various forms, including &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithms&lt;/a&gt;, have been delivering exciting results. &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian networks&lt;/a&gt; are another form of  probabilistic AI.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Utility, the means by which we evaluate the &lt;em&gt;degree&lt;/em&gt; of intelligence of algorithms, is separate from the mechanism.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;AIs can be weak or strong.  Strong means better performance at a task than a competing &lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent#Artificial_intelligence_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;rational agent&lt;/a&gt;, typically humans. (&quot;Man is the measure of all things.&quot; &lt;a href=&quot;https://en.wikipedia.org/wiki/Protagoras&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protagoras&lt;/a&gt;) The term strong in relation to AI has traditionally been taken to mean &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt; [see also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test]&lt;/a&gt;, but current algorithmic intelligences are only &quot;narrowly strong&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intelligence is a spectrum, therefore:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The minimum requirement for AI is that an algorithm make decisions based on data, irrespective of the quality of the decisions. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-12-01T01:36:36.210" LastActivityDate="2018-12-01T01:36:36.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9288" PostTypeId="5" CreationDate="2018-12-01T03:57:42.050" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-12-01T03:57:42.050" LastActivityDate="2018-12-01T03:57:42.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9289" PostTypeId="4" CreationDate="2018-12-01T03:57:42.050" Score="0" Body="For questions that ask about or call for proofs for specific assertions, whether they be proofs of theorems or corollaries, proofs of concept through working implementation, counter proofs, or counter examples." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-01T04:13:58.687" LastActivityDate="2018-12-01T04:13:58.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9290" PostTypeId="5" CreationDate="2018-12-01T04:02:41.750" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-12-01T04:02:41.750" LastActivityDate="2018-12-01T04:02:41.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9291" PostTypeId="4" CreationDate="2018-12-01T04:02:41.750" Score="0" Body="For questions related to guiding behavior or the learning of behavior on the basis of collision risk reduction, which may include trajectory analysis of other moving objects such as in antiaircraft targeting, but with the opposite objective, or the employment of obstacle recognition to avoid striking or scraping it." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-01T04:14:03.563" LastActivityDate="2018-12-01T04:14:03.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9293" PostTypeId="2" ParentId="6968" CreationDate="2018-12-01T06:01:37.497" Score="0" Body="&lt;p&gt;Using the adjective generalized with algorithms, routines, or functions is obscure.  The more appropriate term is generic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generics began with early loaders, before full fledged operating systems were developed.  The idea was that a module could be loaded into computer memory and the loader was able to perform its function whether the information being loaded was a program, a routine called from a program, an update of the loader, or data to be used by a program.  This abstraction was mostly the brainchild of mathematicians such as Kurt Gödel and John von Neumann, both of whom contributed to the same kind of generic representations in the field of abstract algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ideas of generics spread to the development of file systems.  When LISP was developed, the deliberate blurring of the distinction between programs and data was built into the low level functions for working with both.  Functional programming developed into higher levels of abstraction in the LISP community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of generics was introduced into C++ by James Coplien using the term &lt;em&gt;idiom&lt;/em&gt; to solve maintainability and readability issues with early clumsy attempt at generic programming using the C pre-prossessing directives.  The first highly usable form of generics appeared in Alexander Alexandrovich Stepanov's standard templates library (STL), developed at SGI. Frameworks like COBRA and DCOM were designed to permit generic messaging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generic programming constructs spread to Java and then to other languages.  Many algorithms are written generically, such that the actual object types with which the algorithm works are not known until the template is employed at compile time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The multilayer perceptron is a generic learning component, parameterized by a specific numerical representation for forward signal propagation, numerical representation for backward corrective signalling, layer depth, widths and activation functions for each layer, learning rate, and other hyper-parameters.  (What are called network parameters are not parameters of the generic network but parameters of that control the mix of inputs into each layer and become the primary data output of the learning process.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other artificial network types are similar in this respect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even the cell types can be a type parameter in a network.  See &lt;a href=&quot;https://ai.stackexchange.com/questions/5898/neural-network-cell-node-types&quot;&gt;Neural Network Cell (Node) Types&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question, &quot;What does it mean when it is said that Machine Learning algorithm results can be generalized?&quot; cannot be intelligently answered.  Generalizing results is too general a description of what can be done to results to mean anything.  It may just be rhetorical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no universally, &quot;good source[s] for this type of thing.&quot; There are many excellent sources for very specific things buried within a hundred times the volume of marginally correct and educationally barren sources.  If you don't have the time to scrutinize what you are reading to carefully extract the gems, then it might be wise to attend some courses at a well respected university to get a more solid foundation from a more reliable source than the Internet.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-01T06:01:37.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9294" PostTypeId="5" CreationDate="2018-12-01T06:09:40.743" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-12-01T06:09:40.743" LastActivityDate="2018-12-01T06:09:40.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9295" PostTypeId="4" CreationDate="2018-12-01T06:09:40.743" Score="0" Body="For questions related AI used to automate vehicles, including the autonomous vehicles capabilities of automated piloting and air traffic control for aircraft and orbital and interplanetary vehicles, automated driving of road vehicles, and other types of vehicle automation requiring intelligent acquisition and control systems." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-03T13:31:29.780" LastActivityDate="2018-12-03T13:31:29.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9296" PostTypeId="1" CreationDate="2018-12-01T07:08:09.450" Score="3" ViewCount="1156" Body="&lt;p&gt;Biology is used in AI terminology. What are the reasons? What does biology have to do with AI? For instance, why is the genetic algorithm used in AI? Does it fully belong to biology?&lt;/p&gt;&#xA;" OwnerUserId="19886" LastEditorUserId="2444" LastEditDate="2019-03-22T14:56:27.680" LastActivityDate="2020-11-17T11:24:57.520" Title="What is the role of biology in AI?" Tags="&lt;philosophy&gt;&lt;terminology&gt;&lt;biology&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9298" PostTypeId="1" AcceptedAnswerId="9300" CreationDate="2018-12-01T08:53:47.123" Score="2" ViewCount="999" Body="&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/36162180/gradient-descent-vs-adagrad-vs-momentum-in-tensorflow&quot;&gt;https://stackoverflow.com/questions/36162180/gradient-descent-vs-adagrad-vs-momentum-in-tensorflow&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the nice gifs explain how different algorithms approach towards the root. Unfortunately, the environment in the gif is way too simple and real cases have much more complex environments. Also, in reinforcement learning, the solutions should change each moment in a difficult enough environment since things are dynamic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is which optimizer is best for reinforcement learning in such dynamically changing environment? Adadelta should not move beyond local minima so do we have to use SGD or Adadelta with an exploration heuristic? Please let me know in detail your thoughts.&lt;/p&gt;&#xA;" OwnerUserId="19027" LastActivityDate="2018-12-01T10:00:38.467" Title="Neural Network Optimizers in Reinforcement Learning non-well behaved environments" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9299" PostTypeId="2" ParentId="8407" CreationDate="2018-12-01T09:27:24.437" Score="0" Body="&lt;p&gt;I've answered a part of the question below, i.e. can the photo/video be decompressed in theory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the RGB values are continuous (well behaved function), interpolation should generate the pixels in between them in a predictable pattern like say in old pictures of models where skin color shades and data is available in plenty for models today and they resemble each other whereas in outlier cases like this picture where the designs of cockpit circular meter maybe different for each plane or maybe too much data is lost say of a distant object, no neural network should be able to fix it because the possibilities of an orange blur can be an orange, a sun or many other things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the solutions to such things can be recognizing context but finding sufficiently dense pixels so exact should be difficult (I think impossible) because sun will be very bright but orange might have some spots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, average things can be done, not outliers.&lt;/p&gt;&#xA;" OwnerUserId="19027" LastActivityDate="2018-12-01T09:27:24.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9300" PostTypeId="2" ParentId="9298" CreationDate="2018-12-01T10:00:38.467" Score="2" Body="&lt;p&gt;The most commonly-used optimizer in Deep Reinforcement Learning research the past few years is probably ADAM (or its &lt;a href=&quot;http://www.satyenkale.com/papers/amsgrad.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AMSGrad variant&lt;/a&gt;, which in most frameworks like keras/tensorflow/pytorch etc. can be used by setting an &lt;code&gt;amsgrad&lt;/code&gt; flag to &lt;code&gt;True&lt;/code&gt; in the construction of an ADAM optimizer, and I believe is often also already set to &lt;code&gt;True&lt;/code&gt; by default). This is a somewhat newer optimizer which isn't included in that visualization you linked to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RMSProper is also still quite popular, &lt;a href=&quot;https://www.quora.com/Why-does-Google-DeepMind-use-RMSProp-over-Adam-for-RL&quot; rel=&quot;nofollow noreferrer&quot;&gt;and there may be some arguments that it might be better suited for non-stationary learning problems&lt;/a&gt; (which we typically have in RL, our learning targets tend to be non-stationary due to the agent adapting its behaviour over time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these more &quot;fancy&quot; optimizers, typically including some form of momentum term, can in fact be especially useful for optimization in loss &quot;landscapes&quot; that are not smooth. Imagine that your loss landscape looks like a tall mountain, where we'd like our optimizer to &quot;glide&quot; all the way down. Now suppose that the overall trend of a side of the mountain is downwards, but that there are lots of little bumps along the way down, that it's not a smooth mountain. The more fancy optimizers with momentum terms are more likely to &quot;jump&quot; over those bumps than a regular SGD optimizer without momentum terms. This is because, when a little bump is encountered, the local gradient may point the wrong way, but a momentum term &quot;remembers&quot; the overall trend of gliding downwards along the mountain and still points in that same direction for a while.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-01T10:00:38.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9305" PostTypeId="1" CreationDate="2018-12-01T18:33:00.083" Score="8" ViewCount="3246" Body="&lt;p&gt;Which problems in AI are not machine learning? Which problems involve both AI and machine learning?&lt;/p&gt;&#xA;" OwnerUserId="19886" LastEditorUserId="2444" LastEditDate="2019-06-24T13:20:04.480" LastActivityDate="2019-06-24T13:22:48.040" Title="Which problems in AI are not machine learning?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9306" PostTypeId="1" AcceptedAnswerId="9308" CreationDate="2018-12-01T21:12:04.480" Score="1" ViewCount="236" Body="&lt;p&gt;Recently I started to look at policy gradient methods and policies are represented as functions with features for larger problems with many states. Many articles and pseudocodes of algorithms mention sampling an action from the policy, but it is unclear to me how.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actions are something we do in the environment, like going left, right, etc... And functions take some feature values and parameters, make calculations, and 'spit out' some number. So how do we actually map that number to a certain action, and how do we know what action to take?&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="16229" LastEditDate="2019-06-10T18:09:37.987" LastActivityDate="2019-06-10T18:09:37.987" Title="How do we actually sample an action from a policy in policy gradient methods?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9308" PostTypeId="2" ParentId="9306" CreationDate="2018-12-02T11:17:53.907" Score="2" Body="&lt;p&gt;In those policy gradient algorithms, we typically have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input features &lt;span class=&quot;math-container&quot;&gt;$x(s)$&lt;/span&gt; for any state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;A fixed action set &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; of a fixed size (in reinforcement learning (RL) it is often assumed that all actions are legal in all states. In some situations, like board games, we'll filter out illegal actions manually in a postprocessing step).&lt;/li&gt;&#xA;&lt;li&gt;A function approximator &lt;span class=&quot;math-container&quot;&gt;$\pi(x)$&lt;/span&gt; (often a neural network), which takes as input the features &lt;span class=&quot;math-container&quot;&gt;$x(s)$&lt;/span&gt; of some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and produces as output &lt;strong&gt;a vector&lt;/strong&gt; of probabilities, &lt;strong&gt;where the vector has a length equal to the size &lt;span class=&quot;math-container&quot;&gt;$\vert \mathcal{A} \vert$&lt;/span&gt; of the action space&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Those bolded parts in the last point there are critical for your question. We do not normally have a function that &quot;spits out some number&quot; as you described, we typically have one that &quot;spits out&quot; multiple numbers; one for every action. Often, these numbers can also be directly interpreted as probabilities, because they are forced to all lie in &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt; and sum up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (this is often done using a softmax layer at the end of a neural network for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have such a vector of &quot;probabilities&quot;, one for every action, it is relatively straightforward to sample from that. Just give every action a probability of being selected equal to its corresponding output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in some cases, people describe their function approximator as producing &quot;logits&quot; rather than &quot;probabilities&quot; as outputs. These are real-valued outputs which do not necessarily sum up to one, typically produced by a linear layer at the end of a neural network rather than a softmax layer (see e.g. AlphaGo Zero). It is then still implicitly assumed that we convert them into numbers that can be interpreted as probabilities through the application of a softmax, before sampling from it.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="16229" LastEditDate="2019-06-10T18:09:28.137" LastActivityDate="2019-06-10T18:09:28.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9311" PostTypeId="1" AcceptedAnswerId="9318" CreationDate="2018-12-02T20:50:27.887" Score="2" ViewCount="93" Body="&lt;p&gt;Can the same input for a plain neural network be used for CNNs? Or does the input matrix need to be structured in a different way for CNNs compared to regular NNs? &lt;/p&gt;&#xA;" OwnerUserId="20358" LastActivityDate="2018-12-03T13:36:29.160" Title="Can the same input for a plain neural network be used for a convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9312" PostTypeId="1" CreationDate="2018-12-02T21:09:26.017" Score="17" ViewCount="388" Body="&lt;p&gt;Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, the hierarchical organization could allow us to make sense of more complex problem spaces and reach a higher functionality, but it seems difficult to find examples of concrete research done in the past regarding this. I've found a few sources:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Modular_neural_network&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Modular_neural_network&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.teco.edu/~albrecht/neuro/html/node32.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.teco.edu/~albrecht/neuro/html/node32.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://vtechworks.lib.vt.edu/bitstream/handle/10919/27998/etd.pdf?sequence=1&amp;amp;isAllowed=y&quot; rel=&quot;noreferrer&quot;&gt;https://vtechworks.lib.vt.edu/bitstream/handle/10919/27998/etd.pdf?sequence=1&amp;amp;isAllowed=y&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few concrete questions I have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Has there been any recent research into the use of MNNs?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Are there any tasks where MNNs have shown better performance than large single nets?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Could MNNs be used for multimodal classification, i.e. train each net on a fundamentally different type of data, (text vs image) and feed forward to a higher level intermediary that operates on all the outputs?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;From a software engineering perspective, aren't these more fault tolerant and easily isolatable on a distributed system?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Has there been any work into dynamically adapting the topologies of subnetworks using a process like Neural Architecture Search?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Generally, are MNNs practical in any way?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Apologies if these questions seem naive, I've just come into ML and more broadly CS from a biology/neuroscience background and am captivated by the potential interplay. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really appreciate you taking the time and lending your insight!&lt;/p&gt;&#xA;" OwnerUserId="20357" LastActivityDate="2019-08-06T12:36:26.893" Title="Are Modular Neural Networks more effective than large, monolithic networks at any tasks?" Tags="&lt;neural-networks&gt;&lt;topology&gt;&lt;architecture&gt;&lt;neurons&gt;&lt;biology&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9316" PostTypeId="2" ParentId="9311" CreationDate="2018-12-03T08:57:12.603" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;Neural networks generally deal with 1 D data. For example, the data for a NN would be ( 10 , 12 ) , where 10 is the number of samples.&lt;/li&gt;&#xA;&lt;li&gt;Convolutional neural networks generally deal with 1D, 2D and 3D data. For example, a 2D CNN would have input as shape ( 10 , 28 , 28 , 3 ) where 10 is the number of samples and ( 28 , 28 , 3 ) is the image size. ( if the input is a image )&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-12-03T08:57:12.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9318" PostTypeId="2" ParentId="9311" CreationDate="2018-12-03T13:29:41.020" Score="1" Body="&lt;p&gt;There is no restriction on how you input a data to a NN. You can input it in 1D arrays and do element-wise multiplication using 4-5 loops and imposing certain conditions(which will be slow and hence &lt;span class=&quot;math-container&quot;&gt;$nD$&lt;/span&gt; matrix notations are used for a CNN).&#xA;Ultimately, the library you are using (TensorFlow, NumPy might convert it into its own convenient dimensions).&#xA;The main thing different of a CNN from a normal NN is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The number of parameters of a CNN in a convolutional layer is less than the number of input features.&#xA;&lt;span class=&quot;math-container&quot;&gt;$parameters \le features$&lt;/span&gt; (in general it is less than). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Different people have different ways of viewing how the convolutional layer work but the general consensus is that the weights of the convolutional layers of a CNN are like digital filters. It will be a &lt;span class=&quot;math-container&quot;&gt;$nD$&lt;/span&gt; filter if input dimension is &lt;span class=&quot;math-container&quot;&gt;$nD$&lt;/span&gt;. The  output is obtained by superimposing the filter on a certain part of the input and doing element-wise multiplication of the values of filter and the values of the input upon which the filter is superimposed upon. How you implement this particular operation depends on you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the answer to your question will be same network cannot be used, but it might be used with modifications (a normal NN is the limiting case of a CNN where &lt;span class=&quot;math-container&quot;&gt;$features=parameters$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-12-03T13:36:29.160" LastActivityDate="2018-12-03T13:36:29.160" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9319" PostTypeId="1" CreationDate="2018-12-03T16:28:25.627" Score="3" ViewCount="492" Body="&lt;p&gt;We have convolutional neural networks and recurrent neural networks for analysing  respectively images and sequential data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I determine which neural network architecture is more appropriate to approximate a certain function? For example, suppose I want to approximate the function &lt;span class=&quot;math-container&quot;&gt;$f(x,y) = \sin(2\pi x)\sin(2\pi y)$&lt;/span&gt; with domain &lt;span class=&quot;math-container&quot;&gt;$\Omega = [0,1]\times [0,1]$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; can be between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (inclusive). For example, which kind of activation functions would be better suited for approximating this specific function?&lt;/p&gt;&#xA;" OwnerUserId="18975" LastEditorUserId="2444" LastEditDate="2019-05-10T14:34:32.903" LastActivityDate="2019-06-12T12:54:54.827" Title="Which neural network should I use to approximate a specific function?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;activation-function&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9320" PostTypeId="1" AcceptedAnswerId="9350" CreationDate="2018-12-03T16:29:42.023" Score="0" ViewCount="143" Body="&lt;p&gt;I just wanted to gather some perspective on why this is a great opportunity to be able to study machine learning today? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all the online resources (online courses like Andrew Ng's, availability of datasets such as Kaggle, etc), learning machine learning has become possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understood that you can have high paid jobs; but you also need a lot of work dedication to be good at it, which makes your salary not so attractive! (in comparison to the number of hours you spend to keep up with this fast moving field) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why it is so desirable to take this opportunity and start learning machine learning today? (community, ability to start a business, etc.)&lt;/p&gt;&#xA;" OwnerUserId="20376" LastEditorUserId="16355" LastEditDate="2018-12-04T21:29:21.453" LastActivityDate="2018-12-05T19:08:14.087" Title="Why studying machine learning is an opportunity in today's world?" Tags="&lt;machine-learning&gt;&lt;social&gt;&lt;profession&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-23T01:58:22.833" ContentLicense="CC BY-SA 4.0" />
  <row Id="9322" PostTypeId="1" CreationDate="2018-12-03T16:48:15.917" Score="1" ViewCount="770" Body="&lt;p&gt;Why do we need both encoder and decoder in sequence to sequence prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could just have a single RNN that, given input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, outputs some value &lt;span class=&quot;math-container&quot;&gt;$y(t)$&lt;/span&gt; and hidden state &lt;span class=&quot;math-container&quot;&gt;$h(t)$&lt;/span&gt;. Next, given &lt;span class=&quot;math-container&quot;&gt;$h(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y(t)$&lt;/span&gt;, the next output &lt;span class=&quot;math-container&quot;&gt;$y(t+1)$&lt;/span&gt; and hidden state &lt;span class=&quot;math-container&quot;&gt;$h(t+1)$&lt;/span&gt; should be produced, and so on. The architecture shall consists of only one network instead of two separate ones. &lt;/p&gt;&#xA;" OwnerUserId="20378" LastEditorUserId="2444" LastEditDate="2019-05-03T10:42:48.400" LastActivityDate="2020-10-24T19:04:55.567" Title="Why do we need both encoder and decoder in sequence to sequence prediction?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;sequence-modelling&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9325" PostTypeId="1" AcceptedAnswerId="9328" CreationDate="2018-12-03T18:50:01.203" Score="2" ViewCount="65" Body="&lt;p&gt;I am currently looking at different documents to understand back-propagation, mainly at &lt;a href=&quot;https://www.cs.swarthmore.edu/~meeden/cs81/s10/BackPropDeriv.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;. Now, at page 3, there is the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; symbol involved:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QKDHI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QKDHI.png&quot; alt=&quot;Gradient equation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I understand the main part of the equation, I don't understand the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; factor. Searching for the meaning of the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; in math, it means (for example) a error value to be minimized, but why should I multiply with the error (it is denoted as E anyways).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shouldn't the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;  be the learning rate in this equation? I think that would be what makes sense, because we want to calculate by how much we want to adjust the weight, and since we calculate the gradient, I think the only thing that's missing is the multiplication with the learning rate. The thing is, isn't the learning rate usually denoted with the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="2444" LastEditDate="2019-05-02T16:12:14.037" LastActivityDate="2019-05-02T16:12:14.037" Title="What is the use of the $\epsilon$ term in this back-propagation equation?" Tags="&lt;backpropagation&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9327" PostTypeId="2" ParentId="9322" CreationDate="2018-12-03T21:43:58.380" Score="0" Body="&lt;p&gt;The usual purpose of using an encoder at the front end of a network and a decoder at the tail is to deal with features in between. Between the networks is a representation of the input with much of the redundancy removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the approaches are discussed in &lt;a href=&quot;https://arxiv.org/pdf/1409.1259.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;On the Properties of Neural Machine Translation: Encoder–Decoder Approaches&lt;/em&gt;, 2014, Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, ,Yoshua Bengio&lt;/a&gt;. A single network that, given inputs, outputs some value and later appends that value to the input to produce the next output is not an equivalent circuit. Without the separation, network cannot be trained on the basis of some characteristic of the unique features found at the middle tap point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those learning how artificial networks can perform simple tasks tend to draw the system boundary around a sequence of layers where the inner layers are not manipulated differently from a forward propagation and back propagation perspective. They will call that &lt;em&gt;the network&lt;/em&gt; and the layers that are not directly manipulated as elements in the series of layers are called hidden layers, which is correct, but they are not really hidden, they are just not addressed at the higher level, at the system boundary. They're manipulated as elements in the series by the forward and backward propagation mechanisms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those who have linked these sets of layers together into larger systems sometimes use the term network in the wider sense, a circuit that contains more than one sequence of layers. That's typical in NLP and motion control systems literature. The difference is the complexity of control of the learning process. Some control schemes require the interrelationship of data paths that are not at one of two ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many have attempted to reduce this complexity, but there are control theory reasons why some systems cannot be reduced without losing control of some aspect of the learning process that is directly tied to the objectives of the system. Take a look at textual natural language translation, speech synthesis, robot imitation, automatic pilots, and terrain learners to see examples where the assembly of multiple networks into the larger system are necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, feeding outputs into inputs defeats the design of the recurrent or long-short network design, which handles the retention of state in a way that converges to minimize loss inherently.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-12-03T21:43:58.380" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9328" PostTypeId="2" ParentId="9325" CreationDate="2018-12-03T21:58:15.097" Score="1" Body="&lt;p&gt;The change in nomenclature from what you expected is Lisa Meeden's choice, for unknown reasons. Those with whom she published in the past used &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; to represent error, the result of a loss function. Why she did not use &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; may be because the letter &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; was used elsewhere in the formula, but, if that was the reasons, that wasn't a great one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changes in nomenclature between articles is common, for instance between Carnegie Mellon literature and Cal Tech. Robotics control systems people use different nomenclature from those who write game competition systems. The online mappings between choice of Greek letters and their meaning cannot be relied upon. This is not particular to AI. I've seen it in science and engineering literature, even between text books in courses that are part of the same university curriculum.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-12-03T21:58:15.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9331" PostTypeId="2" ParentId="9305" CreationDate="2018-12-04T10:44:50.723" Score="3" Body="&lt;p&gt;This is a common question and it has already been answered in many different places:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/35/what-is-the-difference-between-artificial-intelligence-and-machine-learning&quot;&gt;What is the difference between artificial intelligence and machine learning?&#xA;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.datasciencecentral.com/profiles/blogs/artificial-intelligence-vs-machine-learning-vs-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence vs. Machine Learning vs. Deep Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial intelligence is a broader concept than machine learning, which addresses the use of computers to mimic the cognitive functions of humans. When machines carry out tasks based on algorithms in an &quot;intelligent&quot; manner, that is AI. Machine learning is a subset of AI and focuses on the ability of machines to receive a set of data and learn for themselves, changing algorithms as they learn more about the information they are processing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;More specific to your question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.teradata.com/Blogs/AI-without-machine-learning#/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI without machine learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you insert a small amount of knowledge into a machine, you can call it an engineering product. But if you instill a sufficiently large amount of knowledge such that the machine makes better decisions than a human, that can be could AI. For example, if you take hundreds of medical doctors and each spends hundreds of hours detailing correlations between symptoms and disease, if you then pack that knowledge into an easy-to-use machine, that's AI without machine-learning&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mbif4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mbif4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20399" LastEditorUserId="2444" LastEditDate="2019-06-24T13:22:48.040" LastActivityDate="2019-06-24T13:22:48.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9333" PostTypeId="1" AcceptedAnswerId="9366" CreationDate="2018-12-04T19:08:00.153" Score="3" ViewCount="370" Body="&lt;p&gt;We have a series of data and we want to label the parts of each series. As we do not have any training data, we could try to use &lt;em&gt;active learning&lt;/em&gt; as a solution, but the problem is that our classifier is something like RNN which needs a lot of data to be trained. Hence, we have a problem in converging fast to just label proportional small parts of unlabeled data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any article about this problem (active learning and some complex classifiers, like RNN)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good solution to this problem or not? (as data is a series of actions)&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2020-04-18T14:01:32.173" LastActivityDate="2020-09-19T21:12:30.643" Title="How can active learning be used in the case of complex models that require a lot of data?" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;active-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9337" PostTypeId="1" CreationDate="2018-12-05T00:33:01.603" Score="0" ViewCount="52" Body="&lt;p&gt;When will it be possible to give a computer program a bunch of assumptions and ask it if a certain statement is true or false, giving a proof or a counterexample respectively?&lt;/p&gt;&#xA;" OwnerUserId="17601" LastEditorUserId="2444" LastEditDate="2021-01-23T01:56:26.907" LastActivityDate="2021-01-23T01:56:26.907" Title="When will we have computer programs that can compose mathematical proofs?" Tags="&lt;logic&gt;&lt;automated-theorem-proving&gt;&lt;automated-reasoning&gt;&lt;general-problem-solver&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9338" PostTypeId="5" CreationDate="2018-12-05T04:48:22.373" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-12-05T04:48:22.373" LastActivityDate="2018-12-05T04:48:22.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9339" PostTypeId="4" CreationDate="2018-12-05T04:48:22.373" Score="0" Body="For questions about the feasibility in short term or long term of particular AI objectives, leveraging the realistically available development resources." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-05T22:47:55.167" LastActivityDate="2018-12-05T22:47:55.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9340" PostTypeId="5" CreationDate="2018-12-05T04:52:57.370" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-12-05T04:52:57.370" LastActivityDate="2018-12-05T04:52:57.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9341" PostTypeId="4" CreationDate="2018-12-05T04:52:57.370" Score="0" Body="For questions related to the selection of and theory behind specific activation functions used in artificial networks." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-05T22:48:06.290" LastActivityDate="2018-12-05T22:48:06.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9342" PostTypeId="2" ParentId="9312" CreationDate="2018-12-05T06:42:46.440" Score="2" Body="&lt;p&gt;A benchmark comparison of systems comprised of separately trained networks relative to single deeper networks would not likely reveal a universally applicable best choice.&lt;sup&gt;1&lt;/sup&gt; We can see in the literature the increase in the number of larger systems where several artificial networks are combined, along with other types of components. It is to be expected. Modularization as systems grow in complexity and demands on performance and capability grow is as old as industrialization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our laboratory works with robotic control, thermodynamic instrumentation, and data analysis, artificial networks are components in these larger system contexts. we have no single MLPs or RNNs that by themselves perform any useful function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contrary to the conjecture about hierarchies decades ago, the topology approach that seems to work well in most cases follows the more common system module relationships that are seen in power plants, automated factories, aeronautics, enterprise information architectures, and other complex engineered creations. The connections are those of flow, and if those are designed well, oversight functions are minimal. Flow occurs between modules involving protocols for communications, and each module performs its function well, encapsulating the lower level of complexity and functional detail. It is not one network overseeing another that seems to emerge most effective in actual practice but balance and symbiosis. Identification of clear master-slave design in the human brain seems to be equally slippery. It may be naive to assume that some all knowing subsystem can be a single causal factor in the remarkable collaborative synchronization of a number of other subsystems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge is not finding the information paths that make up the system information topology. The information flow is often obvious upon problem analysis. The difficulty is in discovering the best of strategies to train these independent networks. Training dependencies are common and often critical, whereas in animals, the training occurs &lt;em&gt;in situ&lt;/em&gt; or not at all. We're discovering conditions under which that kind of learning in our systems is practical and how to achieve it. Most of our research along these lines is intended to discover ways to achieve higher reliability and lower burden in terms of research hours to get it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Higher functionality is not always of benefit. It often produces lower reliability and consumes additional development resources with little return. Find a way to marry higher level automation, resource thrift, and reliability into one development process, and you might win an award and honorable mention around the web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallel systems that have the same objective is a good idea, but not a new one. In one aeronautics system, nine parallel systems have the same objective, in groups of three. Each group uses a different computing approach. If two of the systems using the same approach provide the same output and the third differs, the matching output is used and the difference in the third is reported as a system fault. If two of the different approaches provide similar results and the third differs substantially, a merge of the two similar results is used and the third is reported as a use case to further develop the dissenting approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The improved fault tolerance has a cost, eight more systems and associated computing resources and connectivity plus the comparators at the tail, but in systems that are a matter of life and death, the extra costs are paid and the reliability is maximized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dynamic topological adaptation is related to redundant systems and fault tolerance but in some ways is quite distinct. In that area of development, the technology to follow is neuromorphic computing, which is partly inspired by neuroplasticity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last distinction to consider is between process topology, data topology, and hardware topology. These three geometric frames can produce greater efficiency together if aligned in specific ways that produce more direct mappings between the relationships between flow, representation, and mechanics. They are, however, distinct topologies. The meaning of alignment may not be apparent without diving deeply into these concepts and the details that emerge for specific product or service objectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Deep networks that are trained as a single unit and function without connectivity to other artificial networks are not necessarily monolithic. Most practical deep networks have heterogeneous sequence of layers in terms of their activation functions and often their cell types.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T11:28:04.203" LastActivityDate="2018-12-12T11:28:04.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9343" PostTypeId="2" ParentId="9337" CreationDate="2018-12-05T09:10:02.717" Score="2" Body="&lt;p&gt;Arguably we have had this since 1957, with the &lt;a href=&quot;https://en.wikipedia.org/wiki/General_Problem_Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Problem Solver&lt;/a&gt;. However, it is a thorny problem, and like so much in AI, it works fine in toy domains (like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tower_of_Hanoi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towers of Hanoi&lt;/a&gt; problem), but fails in real life, as real life is too complex for it to cope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is &lt;a href=&quot;https://en.wikipedia.org/wiki/Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;a list of solvers&lt;/a&gt; (the category of programs dealing with this issue) on Wikipedia. As of today, it comprises about 40 systems (split into two sub-categories). These work with mathematical equations and presumably also more general problems, assuming they are not too complex.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-12-05T09:10:02.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9345" PostTypeId="1" CreationDate="2018-12-05T12:01:58.070" Score="1" ViewCount="20" Body="&lt;p&gt;I'm working on a problem and need to use Karaboga's code of the ABC algorithm but I have some questions...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this formula for calculating a parameter have to be changed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{/v_{ij}=x_{ij}+\phi_{ij}(x_{kj}-x_{ij}) */}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A standard one or this what Karaboga see is better for algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is the same question for this formula of calculating fitness:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{fFitness(ind)=1./(fObjV(ind)+1)}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://sci2s.ugr.es/sites/default/files/files/TematicWebSites/EAMHCO/Software/ABC.C&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to ABC algorithm coded using C programming language&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20434" LastEditorUserId="4709" LastEditDate="2018-12-06T21:18:29.257" LastActivityDate="2018-12-06T21:18:29.257" Title="Change parameter in Karaboga's code of ABC algorithm" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;&lt;training&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9347" PostTypeId="2" ParentId="10" CreationDate="2018-12-05T12:47:24.167" Score="1" Body="&lt;p&gt;Why is it useful? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many things we don't know for sure. We estimate and are often uncertain, but nearly never 100% sure. It may seem like a weakness, but because of this fuzzy approach we can function in this complex world and even behave quite intelligently. Hence it's a way to simplify things. And it gives you some leeway to fill the gaps appropriately, e.g. to adapt to slightly varying situations. &#xA;P.S.: In natural language we express this with quantitive terms like more, less, nearly, rather, immense and so on. But quantifying things is hard for us.&lt;/p&gt;&#xA;" OwnerUserId="19693" LastActivityDate="2018-12-05T12:47:24.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9350" PostTypeId="2" ParentId="9320" CreationDate="2018-12-05T19:08:14.087" Score="3" Body="&lt;p&gt;What sort of opportunity it is depends on how much you want to focus on it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to be a regular programmer, you might take the time to learn a high level interface for some machine learning tools, such as Tensorflow or Keras. There will be plenty of things you don't know how to do (even within those tools), but you may be able to apply predesigned model architectures to problems. The models won't be as good as one designed specifically for the problem, but it's one more tool in your toolbox, and it's possible you'd be able to get some useful results occasionally without devoting a huge amount of time to mastering the techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want to &lt;em&gt;really&lt;/em&gt; focus on machine learning, at the research level, you can potentially tackle problems that existing techniques haven't been able to solve. This is where most of the big projects that you've probably heard of will be happening: self-driving cars, AlphaGo, etc. What you can expect here is a lot of hard work. You will need to develop a fairly deep understanding of the mathematics involved so you can visualize (to some extent) what is happening in the potentially high dimensional spaces involved, identify potential failure modes, and identify models that won't fall into them. It involves a &lt;em&gt;lot&lt;/em&gt; of trial and error, failed attempts and gradual improvement before you're able to develop a model that beats the stuff already out there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's very rewarding work &lt;em&gt;if&lt;/em&gt; you enjoy it. There are well-paying positions in the field, but that's just a bonus if you already enjoy the work, and it isn't &lt;em&gt;enough&lt;/em&gt; of a bonus if you don't. In my opinion, going into this for just the money would be a mistake. There are almost definitely other jobs that pay just as well but don't take anywhere near the time investment to become (and stay) competent at them. But if you really want to work in this field for its own sake, and &lt;em&gt;also&lt;/em&gt; want to make sure you don't starve while you're doing it, it's absolutely worth it.&lt;/p&gt;&#xA;" OwnerUserId="2212" LastActivityDate="2018-12-05T19:08:14.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9351" PostTypeId="1" CreationDate="2018-12-05T19:58:07.433" Score="1" ViewCount="565" Body="&lt;p&gt;My task involves a large grid-world type of environment (grid size may be &lt;span class=&quot;math-container&quot;&gt;$30\times30$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$50\times50$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$100\times100$&lt;/span&gt;, at the largest &lt;span class=&quot;math-container&quot;&gt;$200\times200$&lt;/span&gt;). Each element in this grid either contains a 0 or a 1, which are randomly initialized in each episode.  My goal is to train an agent, which starts in a random position on the grid, and navigate to every cell with the value 1, and set it to 0. (Note that in general, the grid is mostly 0s, with sparse 1s). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to train a DQN model with 5 actions to accomplish this task:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Move up&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Move right&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Move down&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Move left&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Clear (sets current element to 0)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The &quot;state&quot; that I give the model is the current grid (&lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt; tensor). I provide the agent's current location through the concatenation of a flattened one-hot (&lt;span class=&quot;math-container&quot;&gt;$1\times(N^2)$&lt;/span&gt;) tensor to the output of my convolutional feature vector (before the FC layers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I find that the epsilon-greedy exploration policy does not lead to sufficient exploration. Also, early in the training (when the model is essentially choosing random actions anyway), the pseudo-random action combinations end up &quot;canceling out&quot;, and my agent does not move far enough away from the starting location to discover that there is a cell with value 1 in a different quadrant of the grid, for example. I am getting a converging policy on a &lt;span class=&quot;math-container&quot;&gt;$5\times5$&lt;/span&gt; grid w/ a non-convolutional MLP model, so I think that my implementation is sound.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;How I might encourage exploration that will not always &quot;cancel out&quot; to only explore a very local region to my starting location?&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Is this approach a good way to accomplish this task (assuming I want to use RL)?&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;I would think that attempting to work with a &quot;continuous&quot; action space (model outputs 2 values: vertical and horizontal indices of grid cells that contain 1s) would be more difficult to achieve convergence. Is it wise to always try to use discrete action spaces?&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19789" LastEditorUserId="19789" LastEditDate="2018-12-06T19:11:28.643" LastActivityDate="2018-12-06T19:11:28.643" Title="DQN exploration strategy for large grid-world environment" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9354" PostTypeId="1" CreationDate="2018-12-06T00:27:48.983" Score="4" ViewCount="160" Body="&lt;p&gt;I am wondering how the output of randomly initialized MLPs and ConvNets behave with respect to their inputs.  Can anyone point to some analysis or explanation of this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious about this because in the &lt;a href=&quot;https://blog.openai.com/reinforcement-learning-with-prediction-based-rewards/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Network Distillation work from OpenAI&lt;/a&gt;, they use the output of randomly initialized network to generate intrinsic reward for exploration.  It seems that this assumes that similar states will produce similar outputs of the random network.  Is this generally the case?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do small changes in input yield small changes in output, or is it more &lt;a href=&quot;https://en.wikipedia.org/wiki/Chaos_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;chaotic&lt;/a&gt;?  Do they have other interesting properties?&lt;/p&gt;&#xA;" OwnerUserId="10985" LastEditorUserId="10985" LastEditDate="2018-12-07T23:25:01.860" LastActivityDate="2019-01-01T18:42:39.130" Title="How do randomly initialized neural networks behave?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9356" PostTypeId="2" ParentId="9351" CreationDate="2018-12-06T03:26:36.600" Score="1" Body="&lt;p&gt;What does your Q-network look like?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're using Convolutional Neural Networks (CNNs), I would advise against using pooling layers since the spatial translation data matters in this application (i.e. a cell with '1' top right of the agent is not the same as a cell with '1' below).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;1) How I might encourage exploration that will not always &quot;cancel out&quot; to only explore a very local region to my starting location?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is referred to as &lt;em&gt;exploration vs exploitation dilemma&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to add tackle this is by adding extra logic to the way you're choosing random actions. To elaborate, the chosen random action shouldn't be the same as the opposite of the previous action: if the agent went LEFT, the next random shouldn't be RIGHT as that is not exploring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method is to use different action selection strategies such as Boltzmann Policy rather than epsilon-greedy. Click &lt;a href=&quot;https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-7-action-selection-strategies-for-exploration-d3a97b7cceaf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a quick crash course on the mainstream strategies by Arthur Juliani.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;2) Is this approach a good way to accomplish this task (assuming I want to use RL)?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;RL is well suited for this application. Try different parameters for your network. I'd start with a small network and add more layers/neurons as necessary if it does not converge. However, keep in mind that it might take a while (hours or days) to converge so don't get hasty.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;3) I would think that attempting to work with a &quot;continuous&quot; action space (model outputs indexes of &quot;1&quot; elements) would be more difficult to achieve convergence. Is it wise to always try to use discrete action spaces?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure I fully understand your question. Mind elaborating? What do you mean by the indexes of &quot;1&quot; elements that the model outputs?&lt;/p&gt;&#xA;" OwnerUserId="19981" LastActivityDate="2018-12-06T03:26:36.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9357" PostTypeId="2" ParentId="10" CreationDate="2018-12-06T06:16:31.917" Score="1" Body="&lt;p&gt;It is making deductions based on probability and statistics, like humans make decisions all the time. We are never 100% sure the decision we have made is the right one but there is always some doubt present. Ai will definitely need to use it in some form.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2018-12-06T06:16:31.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9358" PostTypeId="1" CreationDate="2018-12-06T06:44:42.633" Score="1" ViewCount="26" Body="&lt;p&gt;I am preparing to perform research comparing the performance of two different systems that probabilistically generate the next word of an input sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given the word 'the', a system might output 'car', or any other word. Given the input 'the round yellow', a system might output 'sun', or it might output something that doesn't make sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how can I quantitatively evaluate the performance of the two different systems performing this task? Of course if I tested each system manually I could qualitatively determine how often each system responded in a way that makes sense, and compare how often each system responds correctly, but I'd really like a meaningful quantitative method of evaluation that I could preferably automate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Precision and recall don't seem like they would work here, seeing as for each given input there are many potentially acceptable outputs. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2021-01-16T07:49:58.453" Title="How can I evaluate the performance of a system that generates text?" Tags="&lt;natural-language-processing&gt;&lt;software-evaluation&gt;&lt;data-science&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9362" PostTypeId="2" ParentId="9358" CreationDate="2018-12-06T10:03:55.080" Score="1" Body="&lt;p&gt;This is a tricky issue. I assume you are using transition probabilities to pick the next suitable word, so you could use some other corpus data, derive probabilities from it, and compare  those to your system. Not very satisfactory, though, as you might end up evaluating the system in a circular way, deriving your test data in the same way as you generate your output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better, if somewhat more laborious evaluation would be to present the trigger sequence to human beings (on a fairly large scale), and then measure the overlap between your system and the human choices. Of course, with &lt;em&gt;the&lt;/em&gt; you will get a very large set of possible replies, but &lt;em&gt;the round yellow&lt;/em&gt; would be a lot more restricted (and it would make more sense to include that kind of sequence in an evaluation anyway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The figure you will get might be quite small (depending on the range of possible options), but that is to be expected, so your system could score well even with a small overlap only. That's how I would approach this problem.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-12-06T10:03:55.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9363" PostTypeId="2" ParentId="5317" CreationDate="2018-12-06T10:11:33.960" Score="1" Body="&lt;p&gt;It depends on the complexity on your sentences. If you have a limited range, you could do simple pattern matching on part-of-speech tags. Put your sentence through a tagger (there are plenty of them around) and look for the first noun following a verb:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I       want an         apple&#xA;Pronoun verb determiner noun&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I assume you mean the &lt;strong&gt;object&lt;/strong&gt;, as the subject in that sentence would be &lt;em&gt;I&lt;/em&gt;). Scan through the list until you hit a verb, then pick the next noun. This should work for most English sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It gets a bit more complicated if your nouns have additional words around them, for example &lt;em&gt;red apple&lt;/em&gt; (where &lt;em&gt;red&lt;/em&gt; is an adjective) or &lt;em&gt;bottle of beer&lt;/em&gt; (where you have the pattern &lt;code&gt;noun OF noun&lt;/code&gt;). So if you want to capture those fully, you might need a few more complex matching rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall it should still be a lot easier than implementing a full-blown syntactic parser which creates a comprehensive structural analysis of your sentence, most of which you wouldn't need in the first place.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-12-06T10:11:33.960" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9366" PostTypeId="2" ParentId="9333" CreationDate="2018-12-06T15:09:29.217" Score="3" Body="&lt;p&gt;As I found this case backs to the &lt;strong&gt;sequence labeling&lt;/strong&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_labeling&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence labeling&lt;/a&gt; has some classic solution such as conditional random fields (CRFs) and hidden Markov model (HMM). Also, have some solution in Active learning (AL) which use from algorihtms such as struct SVM (&lt;span class=&quot;math-container&quot;&gt;$\text{SVM}^{\text{strcut}}$&lt;/span&gt;) like &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-540-70720-2_27&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, some NLP solutions in active learning could help to solve these kind of problems such as &lt;a href=&quot;https://www.semanticscholar.org/paper/A-study-of-active-learning-methods-for-named-entity-Chen-Lasko/3bce5270707e75380fba9907618315728ba5a4d8&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; which is about active learning in named entity recognition (NER).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, the combination of active learning with Deep networks such as CNN happens. For example, &lt;a href=&quot;https://openreview.net/forum?id=H1aIuk-RW&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; explains more about the idea.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2018-12-06T15:21:53.940" LastActivityDate="2018-12-06T15:21:53.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9369" PostTypeId="1" AcceptedAnswerId="9374" CreationDate="2018-12-06T16:57:36.687" Score="1" ViewCount="354" Body="&lt;p&gt;The field of artificial intelligence is so vast. So many methodologies of handling continuous  data and just now I have read about Hybrid Bayesian network. I just want to know  that what a hybrid Bayesian network contains?&lt;/p&gt;&#xA;" OwnerUserId="19660" LastEditorUserId="9647" LastEditDate="2018-12-07T17:31:49.737" LastActivityDate="2018-12-07T17:31:49.737" Title="What does a hybrid Bayesian network contain?" Tags="&lt;data-science&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9370" PostTypeId="1" AcceptedAnswerId="9387" CreationDate="2018-12-06T18:05:33.997" Score="1" ViewCount="146" Body="&lt;p&gt;Sorry if this is basic or covered elsewhere, I am just starting here and I wasn't able to find an answer, but I might have not been searching for the right thing. So:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am training a neural network to predict current draw in a system. There are a number of obvious numerical inputs, like temperature, counting rate, voltage, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most predictive thing, however, is what operation the system is doing. So like, if it's doing a 'calibration' then the current profile is much different than if it's in 'standby'. I know that I can just use a different network for each operation, but in this case I have a couple hundred different macros defined and I don't want to have 200+ neural networks retrain all the time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that I can have a digital value as an input, but my understanding is that it has to be either 0/1. Also, the relationship to operation is not at all correlated - so operation 100 is not necessarily more current draw than 99 or less than 101. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there a way to have an operation ID or something factor in, but not have it be in the linear combination mathematically? So, basically, tell the system to do a different training based on ID or something? I'll be using python and scikit-learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="20482" LastActivityDate="2018-12-07T19:41:46.983" Title="Using an 'operation ID' as a neural network input" Tags="&lt;neural-networks&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9371" PostTypeId="1" AcceptedAnswerId="9388" CreationDate="2018-12-06T18:18:41.433" Score="1" ViewCount="299" Body="&lt;p&gt;I am trying to generate 90 and 270 degrees rotated versions of my sample images on the fly during training. I found an example and modifying it. But I am confused about what should be the order? For instance in one batch I have 32 images and my image generator should return total 64 images. Let's say upper case letters are 90 degree and lower case letters are 270 degree rotated images. Should the order be AaBbCc or ABCabc? I apply the same to the validation set. Here is the related code fragment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Code fragment added.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def _get_batches_of_transformed_samples(self, index_array):&#xA;    # create list to hold the images&#xA;    batch_x = []&#xA;    # create list to hold the labels      &#xA;    batch_y = []&#xA;    # rotation angles&#xA;    target_angles = [0, 90, 180, 270]&#xA;    angle_categories = list(range(0, len(target_angles)))&#xA;    self.classes = target_angles&#xA;    self.class_indices = angle_categories&#xA;    # generate rotated images and corresponding labels&#xA;    for i, j in enumerate(index_array):           &#xA;        is_color = int(self.color_mode == 'rgb')&#xA;        image = cv2.imread(self.filenames[j], is_color)&#xA;        if is_color:&#xA;            image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)                               &#xA;&#xA;        for rotation_angle, cat_angle in zip(target_angles, angle_categories):&#xA;            rotated_im = rotate(image, rotation_angle, self.target_size[:2])&#xA;            if self.preprocess_func: rotated_im = self.preprocess_func(rotated_im)&#xA;            # add dimension to account for the channels if the image is greyscale&#xA;            if rotated_im.ndim == 2: rotated_im = np.expand_dims(rotated_im, axis=2)                                &#xA;            batch_x.append(rotated_im)&#xA;            batch_y.append(cat_angle)&#xA;&#xA;    # convert lists to numpy arrays&#xA;    batch_x=np.asarray(batch_x)&#xA;    batch_y=np.asarray(batch_y)&#xA;&#xA;    batch_y = to_categorical(batch_y, len(target_angles))            &#xA;    return batch_x, batch_y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I actually rotate them as 0, 90, 180 and 270 degrees. As seen in the code, for each batch I return all the rotated versions of all the images in the batch. But is this correct or should I return first 0 degree rotated versions, second 90 degree rotated versions so on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit2:&lt;/strong&gt; I checked my previous work which I use the built in Keras &lt;code&gt;ImageDataGenerator&lt;/code&gt;. &lt;code&gt;generator.classes&lt;/code&gt; returns [zeros(100,1); ones(100,1)]. In that study I only have two classes. I understand that Keras indexes the images as [class1, class2, ...]. I think I have to do the same.&lt;/p&gt;&#xA;" OwnerUserId="18283" LastEditorUserId="18283" LastEditDate="2018-12-08T11:53:08.567" LastActivityDate="2018-12-09T13:21:19.303" Title="Label arrangement for custom Keras image generator" Tags="&lt;python&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9372" PostTypeId="1" CreationDate="2018-12-06T18:34:22.423" Score="5" ViewCount="994" Body="&lt;p&gt;Can Machine Learning be applied to decipher the script of &lt;strong&gt;lost&lt;/strong&gt; ancient languages (namely, languages that were being used many years ago, but currently are not used in human societies and have been forgotten, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Avestan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Avestan language&lt;/a&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, is there already any successful experiment to decipher the script of unknown ancient languages using Machine Learning?   &lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="2444" LastEditDate="2019-11-01T02:19:19.457" LastActivityDate="2019-11-01T02:19:19.457" Title="Can Machine Learning be applied to decipher the script of lost ancient languages?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;applications&gt;" AnswerCount="4" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9373" PostTypeId="2" ParentId="9372" CreationDate="2018-12-06T18:50:40.097" Score="5" Body="&lt;p&gt;I would guess no, because if the language is unknown (no data available on it), then we would not have training data with which the machine learning algorithm could learn from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is related to some known language, then some statistical analysis can lead to a guess at decipherment (assuming certain similarities among the two languages).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If interested on general language decipherment, see the following where they decipher scripts using available information on the language of interest: &lt;a href=&quot;http://www.aclweb.org/anthology/W99-0906&quot; rel=&quot;noreferrer&quot;&gt;http://www.aclweb.org/anthology/W99-0906&lt;/a&gt;. They utilize the Expectation Maximization algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure more google searching can lead to other examples that use machine learning algorithms, but they would most likely have some known information or a body of assumptions to make our problem scope easier.&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="9469" LastEditDate="2018-12-06T19:03:02.450" LastActivityDate="2018-12-06T19:03:02.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9374" PostTypeId="2" ParentId="9369" CreationDate="2018-12-06T19:09:39.110" Score="0" Body="&lt;p&gt;A bayesian network where the random variables represented can be continuous or discrete. Usually the continuous variables are handled with Normal distribution with an input of some linear combination of the variables it is dependent on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much more detail can be found in Koller's Probabilistic Graphical Model book: &lt;a href=&quot;http://pgm.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pgm.stanford.edu/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9469" LastActivityDate="2018-12-06T19:09:39.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9375" PostTypeId="1" CreationDate="2018-12-06T23:37:33.787" Score="3" ViewCount="364" Body="&lt;p&gt;Question to NN practicioners. I'd like to encode Azul board game state as an input to NN, let's focus on 2-player variant for a while.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PR1vo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PR1vo.jpg&quot; alt=&quot;3-player Azul pictured&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 5 round &quot;Factories&quot; on the table (7 on picture, ignore it). Each one can keep 4 tiles of 5 colors. There is also center of the board which can keep up to 15 tiles. What are the advantages and disadvantages of different encodings? Here are my ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Every Factory has five integer counters, one for each tile color. Sample encoding of single Factory: &lt;code&gt;[3,1,0,0,0]&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every Factory has 20 binary flags, four for each color. Single flag encodes presence of tile of given color. Sample encoding of single factory: &lt;code&gt;[1,1,1,0, 1,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0]&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every Factory has 20 binary flags, four for each color, but only one flag can be set for given color and position of raised flag encodes number of tiles of given color. Sample encoding of single factory: &lt;code&gt;[0,0,1,0, 1,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0]&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every Factory has 4 enum fields, with 6 possible values each (5 colors + empty). Sample encoding of single factory: &lt;code&gt;[red, red, red, blue]&lt;/code&gt; or &lt;code&gt;[Empty,Empty,Empty,Empty]&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(note: encoding schema would also cover center of the board, up to 15 tiles, as said earlier. Of course player's board would also be encoded, but I don't want to ask too broad question)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to train NN to play Azul, which means it needs to properly process number of tiles taken in given round (up to 15 in theory, 2-4 in practice) because it would also need to indicate where to put all those tiles to player board positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on your experience, which encoding is most promising? Or maybe there is some better method I didn't think of? Or it is not possible to tell or it doesn't matter?&lt;/p&gt;&#xA;" OwnerUserId="20125" LastActivityDate="2018-12-06T23:37:33.787" Title="How to encode Azul game state as NN input" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9377" PostTypeId="1" AcceptedAnswerId="9750" CreationDate="2018-12-07T07:01:46.120" Score="4" ViewCount="142" Body="&lt;p&gt;When we perform the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tic-tac-toe&quot; rel=&quot;nofollow noreferrer&quot;&gt;tic-tac-toe&lt;/a&gt; game using adversarial search, I know how to make a tree. Is there a way to find the depth of the tree, and which level is the last level?&lt;/p&gt;&#xA;" OwnerUserId="19886" LastEditorUserId="2444" LastEditDate="2021-05-17T22:01:04.327" LastActivityDate="2021-05-17T22:01:04.327" Title="How do we find the length (depth) of the game tic-tac-toe in adversarial search?" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;minimax&gt;&lt;tic-tac-toe&gt;&lt;adversarial-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9380" PostTypeId="2" ParentId="9372" CreationDate="2018-12-07T09:41:31.867" Score="5" Body="&lt;p&gt;I say yes it definitely could be. But i agree with Skim you need some information as a starting point. Egyptian hieroglyphs were only (recently) understood following the discovery of the Rosetta Stone (&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Rosetta_Stone&quot; rel=&quot;noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/Rosetta_Stone&lt;/a&gt;). With the same message in both known and unknown language the program could find the/a correlation. &#xA;Without that info you would have to guess the potential content of the message. The results would then be confirmation-biased: how could you know it had worked properly? Say the program managed to ‘translate’, outputting a coherent phrase about bananas, great. Even if it was very consistent across multiple samples and 99% confidence, without any “control set data” it could just as easily be about, say, fish or even a prophesy on the super-bowl winner. &#xA;Still a fun project tho. &lt;/p&gt;&#xA;" OwnerUserId="20493" LastActivityDate="2018-12-07T09:41:31.867" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9381" PostTypeId="2" ParentId="9226" CreationDate="2018-12-07T14:54:59.213" Score="1" Body="&lt;p&gt;In general these steps are correct, but are few clarifications would be good. For supervised learning, you can:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;pick model to use&lt;/li&gt;&#xA;&lt;li&gt;split data into training and test set&lt;/li&gt;&#xA;&lt;li&gt;train model, or determine weights by optimization of our error function (inputting in training data)&lt;/li&gt;&#xA;&lt;li&gt;use trained model to predict on test data&lt;/li&gt;&#xA;&lt;li&gt;utilize metrics of interest to evaluate our model (e.g. MSE for continuous outcomes, AUC for binary)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This however is a very elementary pipeline to train a supervised machine learning model. There are many other steps taken to ensure our model is as robust as possible (like cross validation, feature selection, hyperparameter optimization, model selection...). I would strongly recommend learning about these other steps and seeing how they integrate with the above pipeline.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For unsupervised learning, the process depends a lot on the task, but in general it involves optimizing some objective function using the data provided. This is usually done using all the data in one go (not split into training and test) unlike supervised learning. This can be seen more clearly with examples, take methods like PCA and k clustering; the steps taken to apply these methods very much depend on the method used.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Going further, I would recommend Andrew Ng's coursera course on machine learning. It is very accessible and a great introduction to this field, covering both unsupervised/supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="9469" LastEditorUserId="9469" LastEditDate="2018-12-07T18:09:11.143" LastActivityDate="2018-12-07T18:09:11.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9384" PostTypeId="2" ParentId="9354" CreationDate="2018-12-07T18:38:43.093" Score="5" Body="&lt;p&gt;The output of randomly initialised neural networks will be random. I am not sure if I understand your question correctly but I would like to try and explain what OpenAI is trying to achieve with their RND Distillation approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any re-enforcement or supervised &quot;learning&quot; algorithm such as a neural network (NN) with back-propagation, genetic algorithm, partial swarm etc requires a fitness function. The fitness function &quot;directs&quot; the algorithm to a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that the algorithm is randomly initialised. The algorithm is run, and then parts of the algorithm are changed. In this case the weights of the NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A number of algorithms exist to update the weights, the most common back propagation / gradient decent as used in the paper. To update the weights the output of the NN is compared to the optimum output. Sometimes the optimum is known. Other times it is not. In the case of a game the optimum output is not known. It is assumed that the more points gained the better the NN does. In this case the fitness function or policy is the sum of points gained when playing the game. But gradient descent needs an error (i.e difference between output and expected output), which does not exist when playing a game, because you don't know what the points should be. However the NN could play one round of the game score some points, then run the NN again on the next move to determine a prediction of future points. The difference between the points gained and the prediction is then the error, which gradient descent can use. [Hope this makes sense]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is fine for games like pong, a NN could play the game pong randomly and score point at each prediction, which will direct gradient descent into updating the weights to maximise points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, games such as Montezuma’s Revenge, requires a lot of complex actions before points are gained. If the points stay 0, how does gradient descent determine if the weights should be decreased or increased?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, the authors initialise a second random neural network which remains the same for the duration of training. As mentioned above the output of a randomly initialised neural network is random. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of the actual NN, and world state is then passed through the random NN (RND), which will return a random value. The NN is then required to not only determine a game action but also determine the result of the RND. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy function is extended from just points to also predict the result of the RND. The greater the prediction error the more bonus points are allocated to the policy function. This encourages gradient descent to change the weights of the NN to explore unseen parts of the game as this increases the prediction error of the RND which increases points. At the same time gradient descent must update the weights of the NN to predict the RND.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this answers your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary randomly initialised NN include multilayered perceptrons or convolutional networks will behave randomly with respect to their inputs. The RND network is randomly set once at the star of training, and used to basically remind the NN undergoing training that it has done this before do not do this again - i.e. rewards the NN for doing something new (exploration).&lt;/p&gt;&#xA;" OwnerUserId="20508" LastActivityDate="2018-12-07T18:38:43.093" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9385" PostTypeId="2" ParentId="6089" CreationDate="2018-12-07T19:03:09.033" Score="1" Body="&lt;p&gt;The first classic paper your present uses a genetic algorithm (GA), while you mention genetic programming (GP). In short, a GA uses an evolutionary process to evolve a fixed-length vector, normally a bit string, while a GP evolves variable-length decision trees, which could be a computer program or mathematical function.&lt;/p&gt;&#xA;&lt;p&gt;An autoencoder is something that automatically encodes or decodes something. For example, a noisy image is decoded to the original image (very simplified I know), the point here is &lt;em&gt;automatically&lt;/em&gt;, so, without any defined process, an autoencoder can &amp;quot;learn&amp;quot; how to remove the noise from the image.&lt;/p&gt;&#xA;&lt;p&gt;Generally, an autoencoder is made using a neural network (NN). The noisy image is read in as input and the output is a clear image. To train the NN, backpropagation is used. For backpropagation to work, an error is needed to calculate the weight changes. Alternatively, the weights could be encoded as a vector. A fixed-length vector is then evolved using a GA to evolve the weights of the NN. This is what the &amp;quot;classic paper&amp;quot; mentioned does.&lt;/p&gt;&#xA;&lt;p&gt;Using a GA to evolve the weights of a NN is not really efficient, because weights affect each other which means each time crossover happens the weights are totally messed up and the GA can easily get stuck in a local optimum. Gradient descent is probably more efficient.&lt;/p&gt;&#xA;&lt;p&gt;The structure of a multi-layered network is important. Here, gradient descent is still used to update the weights, but a GP could be used to evolve the structure of the NN.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, the GP itself could be used to evolve an autoencoder without using a NN. Here, the chromosome will contain nodes such as mathematical operations or computer code that would alter the input to produce the output.&lt;/p&gt;&#xA;" OwnerUserId="20508" LastEditorUserId="2444" LastEditDate="2021-01-19T17:56:19.467" LastActivityDate="2021-01-19T17:56:19.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9386" PostTypeId="2" ParentId="2083" CreationDate="2018-12-07T19:30:17.740" Score="1" Body="&lt;p&gt;The island model and niching mentioned by NietzscheanAI are well known ways to isolate populations. However, the populations are not really isolated as individuals migrate from one population to the other. In these cases depending on the sampling strategy used to sample parents for crossover migrating individuals may dominate a population causing rapid convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Coevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Co-evolution&lt;/a&gt; is a truly isolated population approach first introduced by Reed et al. in 1967. Two kinds of co-evolution exist namely co-operative co-evolution and competitive co-evolution. While hybrid models exist co-evolved islands such as &lt;a href=&quot;https://www.researchgate.net/publication/275275723_Competitive_Two-Island_Cooperative_Co-evolution_for_Training_Feedforward_Neural_Networks_for_Pattern_Classification_Problems&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://pdfs.semanticscholar.org/e8c3/2c1244cb04708d5c10c1d4ed3cd30fb2264a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, generally co-evolution is isolated and do not migrate individuals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Co-operative co-evolution evolves two or more populations that work together to solve a problem, and competitive co-evolution compete where a gain in one population is a loss for another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally the fitness function is altered from explicit to implicit and various techniques are used to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1506.05082.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Review of landmark articles in co-evolution&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/e08e/b39ac69e7342c88cd01238665098c8ca613c.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rules of engagement&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.tufts.edu/comp/150GA/handouts/nchb-main.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Co-evolutionary principles&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sci.brooklyn.cuny.edu/~sklar/teaching/f05/alife/papers/rosin-96coev.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;New methods for&#xA;competitive co-evolution&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20508" LastActivityDate="2018-12-07T19:30:17.740" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9387" PostTypeId="2" ParentId="9370" CreationDate="2018-12-07T19:41:46.983" Score="0" Body="&lt;p&gt;Great question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The operational ID is actually a category, or categorical input. Each category requires its own input signal into the neural network. Each signal is actually an activation which is multiplied by a weight and added to all the other weights and inputs. This means that if you use the same input signal for all your operational IDs then 101 is more important than 50. As you have mentioned these inputs are very different, and therefore must have a binary input for each operation id. If you add more IDs then you have to retrain the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also mentioned you will be using scikit-learn, which has built in functions for category data. Have a look at this &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link on the one hot encoder&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20508" LastActivityDate="2018-12-07T19:41:46.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9388" PostTypeId="2" ParentId="9371" CreationDate="2018-12-07T20:00:46.963" Score="0" Body="&lt;p&gt;I am not allowed to comment, so I am adding it here. Not sure I understand, are you asking if the training data must be ordered?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, training data must be random, and not AaBbCc or ABCabc.&#xA;Here the input to the NN will be the source image, and a category 90 degrees or 270 degrees as too separate inputs. i.e. inputs [0...k][k+1][k+2], where 0 - k are normalised pixel data, k+1 is 1 for 90 degree rotation else 0, and k+2 is 1 for 270 degree rotation else 0. Alternatively for more rotation options lets make a = input [k+1] and b = input [k+2]. If a = 0 and b = 0 then 0 degrees, if a = 0 and b = 1 then 90 degrees, if a = 1 and b = 0 then 180 degrees if a = 1 and b = 1 then 270 degrees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your image set, draw 3 samples. Sample 1 [training set] must be say 50% of the images, sample 2 [validation set] say 30% of the images and sample 3 [out of sample set] 20% of the images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use the training images from sample 1 to update the weights. Read in images from sample 1 at random, the rotation must be random, use your image rotator to rotate the image, and compare it to the output of the neural network, calculate the sum of the mean error and perform gradient descent to update the weights. Repeat the process. After every x weight updates randomly select a few images from the validation set and calculate an error value. If the average error value of the validation images are above some threshold then stop, or if the error begins to get worse. Do not use the validation set to update the weights&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end use the out-of-sample set of images to verify the performance of the neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process above does not include hyper-parameter optimisation, or any other optimisation techniques, but does describe the basic way of training a NN using supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I assumed that you wanted to NN to learn to rotate the image. After your updates, I assume you want rotated images to increase generalisation of your NN. In which case randomising or shuffling the input batch is preferred. i.e. send your images to Keras to rotate them, get the batch back any order is fine ABab or AaBb and then shuffle them. Pass the shuffled training and validation set to the NN and compare it to the known label for weight updates.&lt;/p&gt;&#xA;" OwnerUserId="20508" LastEditorUserId="20508" LastEditDate="2018-12-09T13:21:19.303" LastActivityDate="2018-12-09T13:21:19.303" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9389" PostTypeId="5" CreationDate="2018-12-07T20:32:51.073" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-12-07T20:32:51.073" LastActivityDate="2018-12-07T20:32:51.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9390" PostTypeId="4" CreationDate="2018-12-07T20:32:51.073" Score="0" Body="For questions about use of AI to promote accessibility, in particular in regard to humans with disabilities." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-12-07T20:32:51.073" LastActivityDate="2018-12-07T20:32:51.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9391" PostTypeId="2" ParentId="2083" CreationDate="2018-12-07T22:38:51.143" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuroevolution of Augmented Topologies (NEAT)&lt;/a&gt; and algorithm developed by Ken Stanely, does this by partitioning populations into species. This is done by storing innovation numbers for each gene (node/connection). When a never before used structure is added through mutation the innovation number is incremented, by doing this you can calculate a historical compatibility distance between any two genomes by comparing innovation numbers. This is done because adding a new structure may often times initially hurt a genomes fitness but may actually turn out to be valuable after some optimization. By using this structurally focused speciation, new structures are protected as the elitism is handled in individual species not the whole population. &lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-07-09T13:56:46.047" LastActivityDate="2019-07-09T13:56:46.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9396" PostTypeId="1" CreationDate="2018-12-08T05:24:56.380" Score="11" ViewCount="1207" Body="&lt;p&gt;In section 7.1 (about the n-step bootstrapping) of the book &lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt; (2nd edition), by Andrew Barto and Richard S. Sutton, the authors write about what they call the &quot;n-step return error reduction property&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BUSZM.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BUSZM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But they don't prove it. I was thinking it should not be too hard but how can we show this? I was thinking of using the definition of n-step return (eq. 7.1 on previous page):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+n} = R_{t+1} + \gamma*R_{t+2} + ... + \gamma^{n-1}*R_{t+n} + \gamma^{n}*V_{t+n-1}(S_{t+n})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because then this has the &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}$&lt;/span&gt; in it already. But in the definition above of the n-step return it uses &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}(S_{t+n})$&lt;/span&gt; but on the right side of the inequality (7.3) that we want to prove it is just little s &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}(s)$&lt;/span&gt; ? So kind of confused here which state s it is using? And then I guess after this probably pull out a &lt;span class=&quot;math-container&quot;&gt;$\gamma^{n}$&lt;/span&gt; term or something, how should we go from here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the newest Sutton Barto book (book page 144, equation 7.3):&#xA;&lt;a href=&quot;https://drive.google.com/file/d/1opPSz5AZ_kVa1uWOdOiveNiBFiEOHjkG/view&quot; rel=&quot;noreferrer&quot;&gt;https://drive.google.com/file/d/1opPSz5AZ_kVa1uWOdOiveNiBFiEOHjkG/view&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20515" LastEditorUserId="2444" LastEditDate="2019-02-16T02:08:32.307" LastActivityDate="2020-05-08T04:07:34.307" Title="How do we prove the n-step return error reduction property?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;math&gt;&lt;proofs&gt;&lt;sutton-barto&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9397" PostTypeId="1" AcceptedAnswerId="9402" CreationDate="2018-12-08T06:25:30.990" Score="2" ViewCount="396" Body="&lt;p&gt;Speech is a major primary mechanism of communication between humans. With respect to artificial intelligence, which signal is used to identify the sequence of words in speech?&lt;/p&gt;&#xA;" OwnerUserId="19660" LastEditorUserId="4302" LastEditDate="2018-12-08T08:50:13.867" LastActivityDate="2018-12-08T08:50:13.867" Title="In speech recognition, what kind of signal is used?" Tags="&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9400" PostTypeId="2" ParentId="9397" CreationDate="2018-12-08T07:14:44.577" Score="1" Body="&lt;p&gt;Not sure I understand what you mean by signal. However, incoming audio must be cleaned and processed for a machine to understand. Some people speak faster and others slower. First thing is to sample in the incoming audio. In other words use analog electic signals and store them as binary values at a set rate of sampling frequency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nyquist Shannon theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that it is sampled we need to clean it up and extract only the voice data. Effectively the sample is a set of values (frequencies) recorded over time. Or a signal :).&#xA;We can cut up this signal into many smaller components of a few milliseconds say 10ms. Each of the millisecond samples is further processed &#xA;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier&lt;/a&gt; transform is generally used to break a signal into components, or frequencies that make up the signal. This is basic signal processing and we have not even entered machine learning yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have a fingerprint of each 10ms sample these samples are then fed into a neural network. Samples are broken between frequency drops to create blocks of sound which may or may not be a word. These blocks are passed to the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A traditional feed forward neural network will not work as it forgets the data. So a &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrent neural network&lt;/a&gt; is used.&#xA;To train the neural network known words are tagged and encoded as a number. The &lt;a href=&quot;https://www.researchgate.net/publication/49600679_Neural_networks_used_for_speech_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural network&lt;/a&gt; will output a number and the difference between this number and the expected number is the error. The error is then used to update the weights&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we have the words, those can be further processed by other networks that specialise in natural language understanding&lt;/p&gt;&#xA;" OwnerUserId="20508" LastActivityDate="2018-12-08T07:14:44.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9402" PostTypeId="2" ParentId="9397" CreationDate="2018-12-08T08:35:51.093" Score="0" Body="&lt;p&gt;The signal path in speech recognition as one travels further from the basic representations of sound depends increasingly on the role the recognition plays. Consider these roles.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Translation to another language&lt;/li&gt;&#xA;&lt;li&gt;Comprehension as in listening to a lecture&lt;/li&gt;&#xA;&lt;li&gt;Comprehension as in a bunch of people out for food, drink, and laughs&lt;/li&gt;&#xA;&lt;li&gt;Transcription into text&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There are similar differences in other natural language facilities of reading, writing, typing, speaking, interpreting body language, and expressing with body language. With auditory language, the stages of sensory processing proceed in a particular order of signal types, each representing the dynamics of language at successively abstract levels, essentially reversing the process of speaking.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Physical dynamics of air&lt;/li&gt;&#xA;&lt;li&gt;Frequency dependent vibrations of cochlear hair&lt;/li&gt;&#xA;&lt;li&gt;Spectra, transients&lt;/li&gt;&#xA;&lt;li&gt;Pitch, tone/timbre, impacts, loudness&lt;/li&gt;&#xA;&lt;li&gt;Phonetic elements&lt;/li&gt;&#xA;&lt;li&gt;Linguistic elements distinct from other noise&lt;/li&gt;&#xA;&lt;li&gt;Beginnings of attention based functionality&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The first six elements are common between all language oriented hearing. The seventh is where the above four roles diverge. In translation and comprehension, comprehension of linguistic nuance is required. Colloquialisms, euphemisms, culturally dependent references and analogies and other higher level linguistic and social constructs is required for full proficiency. For transcription, comprehension is required only to improve transcription reliability and to distinguish between phononyms like to, too, and two.&lt;/p&gt;&#xA;&lt;p&gt;Notice that the sixth level is not words or sentences. Words and sentences are later developments in human language and occur later in an individual's childhood. Our educational systems are largely word oriented, but people do not talk or listen in words. There is no signal for &amp;quot;way&amp;quot; in the brain when someone hears, &amp;quot;No way!&amp;quot; which, in the current semantic state of U.S. English is a single linguistic element that means, &amp;quot;What you just said is very surprising to me.&amp;quot; Two written words represent this one linguistic element.&lt;/p&gt;&#xA;&lt;p&gt;Conversely, there are two distinct signal representations for, &amp;quot;wanted&amp;quot;, specifically, &amp;quot;want,&amp;quot; and &amp;quot;-ed.&amp;quot; The second of the two is reused for, &amp;quot;planted.&amp;quot; The -ed ending is not relearned for every verb.&lt;/p&gt;&#xA;&lt;p&gt;Consider these lines out of a dialog.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Jenna: Cats run.&lt;/p&gt;&#xA;&lt;p&gt;Chelsea: That's ridiculous.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In transcription, the voices must be distinguished and the words must be written, including the contraction of, &amp;quot;That is,&amp;quot; whereas the plural of cats could be possessive, so the computer attempting this transcription must somehow chose the plural over the possessive in this case. Out of context, the pronoun, &amp;quot;That,&amp;quot; does not refer to the cat or a person named Cat but the idea of the running. Consider this conversation between people out for food, drink, and laughs.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Catherine: He's so sexy.&lt;/p&gt;&#xA;&lt;p&gt;Jenna: Cat, you always pine over the really toxic guys.&lt;/p&gt;&#xA;&lt;p&gt;Catherine: No I don't.&lt;/p&gt;&#xA;&lt;p&gt;Jenna: What about that cute (and well dressed) guy that wanted to meet you for lunch last week?&lt;/p&gt;&#xA;&lt;p&gt;Chelsea: She gave him her number.&lt;/p&gt;&#xA;&lt;p&gt;Julia: Yea, with the last two digits transposed.&lt;/p&gt;&#xA;&lt;p&gt;Chelsea: All the Jennas I've ever met are first to run when a nice guy comes around.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Catherine nods. (beat) Patrick comes to the table.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Patrick: Hey, you're Catherine, from Central High, right? Didn't we have study hall together in junior year?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Catherine puts some cash on the table for the drinks.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Catherine: I didn't go to Central. I went to East Barnard. (beat) Girls, I'll see you later. Gotta go so I can get up for work tomorrow.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Catherine exits. Patrick wanders off.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Jenna: Cats run.&lt;/p&gt;&#xA;&lt;p&gt;Chelsea: That's ridiculous.&lt;/p&gt;&#xA;&lt;p&gt;Julia: What, you think she left because of work?&lt;/p&gt;&#xA;&lt;p&gt;Chelsea: Why not?&lt;/p&gt;&#xA;&lt;p&gt;Jenna: She works at a restaurant closed on Mondays.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This dialog is a series of signals relying heavily on understandings that are not in the language. There is no formal grammar that could faithfully guide parsing by words. The signals after all seven levels of processing still fall far short of complete descriptions of the scenarios or the ideas of the individuals involved. Yet a series of themes and triggers to existing information shared in common between those at the table communicate volumes of information.&lt;/p&gt;&#xA;&lt;p&gt;Level six from the layers listed above is important here because people are talking and possibly loudly at other tables. Level seven of the listening is entirely different at this girl's night out and may not share any significant functionality with the listening process when Jenna, Chelsea, Catherine, and Julia are in class taking notes.&lt;/p&gt;&#xA;&lt;p&gt;For computers to partake in a conversation like this one and actually socialize, many more functions beyond the seven layers of abstraction above must be assembled and must learn all the nuance and information brought in as cultural reference. For example, there are several cultural ideas involved.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some people tend to pick those that are not particularly healthy for mates.&lt;/li&gt;&#xA;&lt;li&gt;Getting a phone number is a first step toward talking in private.&lt;/li&gt;&#xA;&lt;li&gt;Cat is Catherine.&lt;/li&gt;&#xA;&lt;li&gt;The conversation is occurring on Sunday, thus the next morning is Monday, the day when the restaurant at which Cat works is closed.&lt;/li&gt;&#xA;&lt;li&gt;A man producing some connection from the past is a segue into further conversation based on some relational interest.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-12-08T08:47:07.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9403" PostTypeId="2" ParentId="2926" CreationDate="2018-12-08T09:39:05.007" Score="0" Body="&lt;p&gt;Restricted Boltzmann Machines (RBMs) can identify patterns in a CSV file without the user specifying any conditions. They are well fitted for generating, &quot;distributed and graded representations,&quot; of a, &quot;complex set of features composing real high-dimensional data is crucial for achieving high performance in machine–learning tasks.&quot;&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the CSV format is specifically designed to represent instances in rows and a static set of attributes in columns, the set up of the training is straightforward. If the goal is to identify temporal patterns, a windowing strategy may be required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;K-RBMs are a merger of k-mean approaches with RBMs. The choice of approach has much to do with what kinds of patterns are sought. The term pattern can apply to simple trends in numbers over time, common patterns found in textual columns, or complex patterns inferred from multiple columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://arxiv.org/pdf/1611.06759.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Emergence of Compositional Representations in Restricted Boltzmann Machines&lt;/em&gt;, J. Tubiana, R. Monasson, 2017)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41320.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning Multiple Non-Linear Sub-Spaces using K-RBMs&lt;/em&gt;, Siddhartha Chandra, Shailesh Kumar &amp;amp; C. V. Jawahar&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-08T09:39:05.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9404" PostTypeId="1" CreationDate="2018-12-08T09:45:53.027" Score="0" ViewCount="89" Body="&lt;p&gt;I was wondering whether a LFSR could be approximated by a NN (output or current state). We know that a LFSR is called linear in some sort of mathematical sense, but is that true? Considering it follows Galois field mathematics. So can a Neural Network approximate a LFSR?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answers with mathematical proof or actual experience is preferred.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2018-12-08T09:51:50.547" LastActivityDate="2018-12-08T10:16:39.720" Title="Can a LFSR be approximated by a Neural Network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9406" PostTypeId="2" ParentId="9404" CreationDate="2018-12-08T10:16:39.720" Score="1" Body="&lt;p&gt;According to the abstract of the following book, yes it's possible:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&quot;Prediction of Sequences Generated by LFSR Using Back Propagation MLP&quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Abstract:&lt;/strong&gt;&#xA;Prediction of the next bit in pseudorandom sequences is one of the main issues in cryptology in order to prove the robustness of the systems. Linear complexity has served as a reference measurement to evaluate the randomness of the sequences, comparing them with the shortest LFSR that can generate those sequences. Several tools based on artificial intelligence have also been used for the next bit prediction, such as the C4.5 classifier. In this paper, we apply a different approach, the back propagation neural networks, to predict the sequences generated by LFSR. The results confirm that these networks can predict the entire sequence knowing less input patterns than techniques based on classifiers.&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2018-12-08T10:16:39.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9410" PostTypeId="1" CreationDate="2018-12-08T12:07:08.283" Score="5" ViewCount="98" Body="&lt;p&gt;Around 2000, existed popular fights videogames. There, if you made the same kick movement for several running times, the CPU normally defended and caught your strategy. This is the case in Tekken 3 ie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How did the work? Is that considered intelligence? Did videogames in that time worked based on graphs?&lt;/p&gt;&#xA;" OwnerUserId="6345" LastEditorUserId="1671" LastEditDate="2018-12-09T23:30:13.287" LastActivityDate="2018-12-09T23:30:13.287" Title="Old video games intelligence" Tags="&lt;game-ai&gt;&lt;history&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ClosedDate="2021-01-23T01:54:09.080" ContentLicense="CC BY-SA 4.0" />
  <row Id="9412" PostTypeId="1" CreationDate="2018-12-08T16:41:12.173" Score="2" ViewCount="299" Body="&lt;h2&gt;Background info&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In Python, I've implemented a rudimentary engine to play &quot;Cheat&quot;, supporting both bots and a human or only bots. When only bots are playing, the game is simulated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When placing cards, input is represented by an array of integers corresponding to the indices of the cards. When bots play, they are presented with all valid combinations of cards to place (currently, the choice made is random). For example, if the bot has two cards, their options are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;[[0], [1], [0, 1]]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a player places cards, the other players get a chance to call cheat (&lt;code&gt;true&lt;/code&gt; to accuse, &lt;code&gt;false&lt;/code&gt; not to).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a player depletes their cards, they are appended to the &lt;code&gt;winners&lt;/code&gt; list. The goal of the game is to have the lowest index possible in the &lt;code&gt;winners&lt;/code&gt; list.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Summary of game data and end goal&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In summary, here is the data which I believe would be useful for the bots to play:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the current number of cards that have been placed&lt;/li&gt;&#xA;&lt;li&gt;the current type (e.g. Ace) to play&lt;/li&gt;&#xA;&lt;li&gt;the type of and suit of each card of the bot's hand&lt;/li&gt;&#xA;&lt;li&gt;the number of cards that were just placed by a player&lt;/li&gt;&#xA;&lt;li&gt;the possible inputs to play during a bot's turn&lt;/li&gt;&#xA;&lt;li&gt;the options for calling cheat (&lt;code&gt;true&lt;/code&gt;, &lt;code&gt;false&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;with the goal of ending up with the lowest index in the &lt;code&gt;winners&lt;/code&gt; list.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Help wanted&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm very new to machine learning, so I apologize for a such a high level question, but how might I go about using a Python module to implement a system for bots to learn to play intelligently as they play? Are there any modules which you think would be ideal for this situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="20514" LastEditorUserId="1671" LastEditDate="2018-12-19T00:37:23.693" LastActivityDate="2018-12-19T00:37:23.693" Title="Implementing AI/ML for the card game &quot;Cheat&quot;" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;&lt;python&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9413" PostTypeId="2" ParentId="4774" CreationDate="2018-12-08T18:44:06.283" Score="0" Body="&lt;p&gt;I tried something similar before for 2048 game. I used the state of the board as &lt;strong&gt;x&lt;/strong&gt;, and the move as &lt;strong&gt;y&lt;/strong&gt;. I just trained the neural network with this dataset. The architecture is like a couple of layers with &lt;strong&gt;relu&lt;/strong&gt; and the final layer as &lt;strong&gt;softmax&lt;/strong&gt;. The major thing is that we should not feed the wrong moves in the dataset to the NN, or else the NN also tend to learn the bad moves, which in turn makes it less smarter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I gathered my dataset by running the minimax on 2048 and assigning a reward for each move, and then eliminating the bad ones on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the above process also depends on the way you are taking the feature vector, if your feature vector is an image, then it makes sense to use CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/navjindervirdee/2048-deep-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/navjindervirdee/2048-deep-reinforcement-learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DQN is also a good option. but do checkout the above link, it helped me too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My repo: &lt;a href=&quot;https://github.com/williamscott701/AI-vs-2048&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/williamscott701/AI-vs-2048&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my results are not actually that great.&lt;/p&gt;&#xA;" OwnerUserId="20522" LastActivityDate="2018-12-08T18:44:06.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9416" PostTypeId="2" ParentId="9236" CreationDate="2018-12-08T19:22:37.693" Score="1" Body="&lt;p&gt;Mutation is used to maintain diversity in the solutions. Crossover alone cannot do this.&lt;/p&gt;&#xA;" OwnerUserId="20522" LastActivityDate="2018-12-08T19:22:37.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9417" PostTypeId="1" AcceptedAnswerId="9538" CreationDate="2018-12-08T21:09:21.717" Score="5" ViewCount="642" Body="&lt;p&gt;Consider a perceptron where &lt;span class=&quot;math-container&quot;&gt;$w_0=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_1=1$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eBSki.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eBSki.jpg&quot; alt=&quot;Perceptron&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, suppose that we use the following activation function&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(x)=&#xA;\begin{cases}&#xA;1, \text{ if }x =1\\&#xA;0, \text{ otherwise}&#xA;\end{cases}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output is then summarised as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{array}{|c|c|c|c|}&#xA;\hline&#xA;x_0 &amp;amp; x_1 &amp;amp; w_0x_0 + w_1x_1 &amp;amp; f( \cdot )\\ \hline&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ \hline&#xA;0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 \\ \hline&#xA;1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 \\ \hline&#xA;1 &amp;amp; 1 &amp;amp; 2 &amp;amp; 0 \\ \hline&#xA;\end{array}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there something wrong with the way I've defined the activation function?&lt;/p&gt;&#xA;" OwnerUserId="20526" LastEditorUserId="2444" LastEditDate="2021-01-19T02:35:36.977" LastActivityDate="2021-01-19T02:35:36.977" Title="Why can't the XOR linear inseparability problem be solved with one perceptron like this?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;perceptron&gt;&lt;xor-problem&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9418" PostTypeId="1" CreationDate="2018-12-08T22:02:53.463" Score="1" ViewCount="869" Body="&lt;p&gt;I'm trying to train a Siamese network to check if two images are similar. My implementation is based on &lt;a href=&quot;https://leimao.github.io/article/Siamese-Network-MNIST/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. I find the Euclidian distance of the feature vectors(the final flattened layer of my CNN) of my two images and train the model using the contrastive loss function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how do I get a binary output from the Siamese network for testing (1 if it two images are similar, 0 otherwise). Is it just by thresholding the Euclidian distance to check how similar the images are? If so, how do I go about selecting the threshold? If I wanted to measure the training and validation accuracies, the threshold would have to be increased as the network learns better. Is there a way to learn this threshold for a given dataset? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any leads, thank you.&lt;/p&gt;&#xA;" OwnerUserId="20527" LastActivityDate="2019-08-29T19:37:56.087" Title="How to get a binary output from a Siamese Neural Network" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9420" PostTypeId="2" ParentId="9418" CreationDate="2018-12-09T01:22:29.487" Score="1" Body="&lt;p&gt;I am working on something rather similar in the sense that I get a continuous value (between 0 and 1) as output that I want to be binary. I think you should be using the ROC curve (Receiver-Operating-Characteristic) instead of the accuracy. If you want a single value take the area-under-the-curve. The ROC curve is basically true-prositive-rate vs false-positive-rate as you test for different values of threshold. You could also (or instead) use precision vs recall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the optimal threshold does not depend as much on the dataset as it depends on your neural network. As you said this value will change as your network learns. Simply use the outputted value and test different thresholds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If do want to keep the accuracy, simply pick the threshold for which is it best (at the given iteration). Keep in mind that you should not define the threshold from the validation set as you would be cheating. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps: from rereading it seems like you were implying to learn the threshold from inside the network. I believe it should just be a post-process&lt;/p&gt;&#xA;" OwnerUserId="20530" LastEditorUserId="20530" LastEditDate="2018-12-12T19:46:26.120" LastActivityDate="2018-12-12T19:46:26.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9421" PostTypeId="2" ParentId="9417" CreationDate="2018-12-09T01:44:54.313" Score="2" Body="&lt;p&gt;Indeed I think the problem is with the way you've defined the activation function. By selecting it arbitrarily, you could solve many specific problems. In practice, activation functions used are monotonic. It keeps the error function convex at a per-layer level. In theory though I'm not sure exactly what Rosenblatt has claimed so it might be worth calling him&lt;/p&gt;&#xA;" OwnerUserId="20530" LastActivityDate="2018-12-09T01:44:54.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9422" PostTypeId="2" ParentId="5096" CreationDate="2018-12-09T05:37:16.240" Score="0" Body="&lt;p&gt;&lt;strong&gt;The Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The training data for the proposed system is as follows.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A Boolean matrix representing the surface adjacency of a solid geometric design&lt;/li&gt;&#xA;&lt;li&gt;Also represented in the matrix is differentiation between interior and exterior angles of edges&lt;/li&gt;&#xA;&lt;li&gt;Labels (described below)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Convex and concave are not the correct terms to describe surface gradient discontinuities. An interior edge, such as made by an end mill, is not actually a concave surface. Surface gradient discontinuity, from the point of view of the idealized solid model, has a zero radius. An exterior edge is not a convex portion of a surface for the same reason.&lt;/p&gt;&#xA;&lt;p&gt;The intended output of the trained system proposed is a Boolean array indicating the presence of specific solid geometric design features.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One or more slot&lt;/li&gt;&#xA;&lt;li&gt;One or more boss&lt;/li&gt;&#xA;&lt;li&gt;One or more holes&lt;/li&gt;&#xA;&lt;li&gt;One or more pockets&lt;/li&gt;&#xA;&lt;li&gt;One or more steps&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This array of Boolean values is also used as labels for training.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Possible Caveats in Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are mapping incongruities in this approach. They fall roughly into one of four categories.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ambiguity created by mapping the topology in the CAD model to the matrix — solid geometries that have primary not captured in the matrix encoding proposed&lt;/li&gt;&#xA;&lt;li&gt;CAD models for which no matrix exists — cases where edges change from inner to outer angles or emerge from curvature&lt;/li&gt;&#xA;&lt;li&gt;Ambiguity in the identification of features from the matrix — overlap between features that could identify the pattern in the matrix&lt;/li&gt;&#xA;&lt;li&gt;Matrices describing features that are not among the five — this could become a data loss issue downstream in development&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These are just a few examples of topology issues that may be common in some mechanical design domains and obfuscate the data mapping.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A hole has the same matrix as a box frame with internal radii.&lt;/li&gt;&#xA;&lt;li&gt;External radii may lead to oversimplification in the matrix.&lt;/li&gt;&#xA;&lt;li&gt;Holes that intersect with edges may be indistinguishable from other topology in matrix form.&lt;/li&gt;&#xA;&lt;li&gt;Two or more intersecting through holes may present adjacency ambiguities.&lt;/li&gt;&#xA;&lt;li&gt;Flanges and ribs supporting round bosses with center holes may be indistinguishable.&lt;/li&gt;&#xA;&lt;li&gt;A ball and a torus have the same matrix.&lt;/li&gt;&#xA;&lt;li&gt;A disk and band with a hexagonal cross with a 180 degree twist have the same matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These possible caveats may or may not be of concern for the project defined in the question.&lt;/p&gt;&#xA;&lt;p&gt;Setting a face size balances efficiency with reliability but limits usability. There may be approaches that leverage one of the variants of RNNs, which may permit coverage of arbitrary model sizes without compromising efficiency for simple geometries. Such an approach may involve splaying the matrix out as a sequence for each example, applying a well conceived normalization strategy to each matrix. Padding may be effective if there are no tight constraints on training efficiency and a practical maximum for number of faces exists.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Considering Count and Certainty as Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To handle some of these ambiguities, a certainty &lt;span class=&quot;math-container&quot;&gt;$\in [0.0, 1.0]$&lt;/span&gt; could be the range of the activation functions of the output cells without changing the labeling of the training data.&lt;/p&gt;&#xA;&lt;p&gt;The possibility of using a non-negative integer output, as an unsigned binary representation created by aggregating multiple binary output cells, instead of a single Boolean per feature should be at least considered as well. Downstream, the capability to count features may become important.&lt;/p&gt;&#xA;&lt;p&gt;This leads to five realistic permutations to consider, that could be produced by the trained network for each feature of each solid geometry model.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Boolean indicating existence&lt;/li&gt;&#xA;&lt;li&gt;Non-negative integer indicating instance count&lt;/li&gt;&#xA;&lt;li&gt;Boolean and real certainty of one or more instance&lt;/li&gt;&#xA;&lt;li&gt;Non-negative integer representing most likely instance count and real certainty of one or more instances&lt;/li&gt;&#xA;&lt;li&gt;Non-negative real mean and standard deviation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Pattern Recognition or What?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the current culture, applying an artificial network to this problem is not normally described as pattern recognition in the sense of computer vision or audio processing. It is thought of as learning a complex functional mapping via convergence in the rough direction of an idea mapping, given proximity, accuracy, and reliability criteria. The parameters of the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, given inputs &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt;, are driven toward the associated labels &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; during training.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(\mathcal{X}) \implies \mathcal{Y}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the concept class being functionally approximated by the network is sufficiently represented in the sample used for training and the sample of training examples is drawn in the same way as the target application will later draw, the approximation is likely to be sufficient.&lt;/p&gt;&#xA;&lt;p&gt;In the world of information theory, there is a blurring of the distinction between pattern recognition and functional approximation, as there should be in that higher level AI conceptual abstraction.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Feasibility&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would the network learn to map matrices to [the array of] Boolean [indicators] of design features?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If the above listed caveats are acceptable to the project stakeholders, the examples are well labeled and provided in sufficient number, and the data normalization, loss function, hyper-parameters, and layer arrangements are set up well, it is likely convergence will occur during training and a reasonable automated feature identification system. Again, its usability hinges on new solid geometries being drawn from the concept class like the training examples were. System reliability relies on training being representative of later use cases.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-12-09T05:43:59.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9424" PostTypeId="1" AcceptedAnswerId="9443" CreationDate="2018-12-09T08:29:55.637" Score="3" ViewCount="537" Body="&lt;p&gt;What is the difference between additive and discounted rewards?&lt;/p&gt;&#xA;" OwnerUserId="19657" LastEditorUserId="2444" LastEditDate="2020-04-12T22:04:48.123" LastActivityDate="2020-04-12T22:04:48.123" Title="What is the main difference between additive rewards and discounted rewards?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;rewards&gt;&lt;reward-clipping&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9425" PostTypeId="1" CreationDate="2018-12-09T10:09:27.203" Score="2" ViewCount="303" Body="&lt;p&gt;Consider an extremely complicated feed-forward neural network training example but with no need of computational efficiency or limiting of processing time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the maximum number of hidden neurons &lt;strong&gt;&lt;em&gt;h&lt;/em&gt;&lt;/strong&gt; that a hidden layer should possess to detect all unique features/ correlations between input data from the previous layer which has &lt;strong&gt;&lt;em&gt;n&lt;/em&gt;&lt;/strong&gt; nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words if we wanted to create a neural network with a large number of neurons in a hidden layer, what is the maximum neuron count possible that helps the network train (give &lt;strong&gt;&lt;em&gt;n&lt;/em&gt;&lt;/strong&gt; neurons are in the previous layer)?&lt;/p&gt;&#xA;" OwnerUserId="18640" LastActivityDate="2021-04-27T16:17:46.880" Title="Maximum number of neurons in a layer given number of neurons in previous layer" Tags="&lt;neural-networks&gt;&lt;neurons&gt;&lt;feedforward-neural-network&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9426" PostTypeId="2" ParentId="9396" CreationDate="2018-12-09T10:35:59.460" Score="9" Body="&lt;p&gt;Let's start by looking at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can rewrite this by plugging in the definition of &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;&amp;amp; \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert \\&#xA;%&#xA;=&amp;amp; \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ R_{t + 1} + \gamma R_{t + 2} + \dots + \gamma^{n - 1} R_{t + n} + \gamma^n V_{t + n - 1}(S_{t + n}) \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert \\&#xA;%&#xA;=&amp;amp; \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ R_{t:t+n} + \gamma^n V_{t + n - 1}(S_{t + n}) \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert,&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$R_{t:t+n} \doteq R_{t + 1} + \gamma R_{t + 2} + \dots + \gamma^{n - 1} R_{t + n}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you go all the way back to page 58 of the book, you can see the definition of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;v_{\pi}(s) &amp;amp;\doteq \mathbb{E}_{\pi} \left[ \sum_{k = 0}^{\infty} \gamma^k R_{t + k + 1} \mid S_t = s \right] \\&#xA;%&#xA;&amp;amp;= \mathbb{E}_{\pi} \left[ R_{t:t+n} + \gamma^n \sum_{k = 0}^{\infty} \gamma^k R_{t + n + k + 1} \mid S_t = s \right] \\&#xA;%&#xA;&amp;amp;= \mathbb{E}_{\pi} \left[ R_{t:t+n} \mid S_t = s \right] + \gamma^n \mathbb{E}_{\pi} \left[ \sum_{k = 0}^{\infty} \gamma^k R_{t + n + k + 1} \mid S_t = s \right]&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this, we can continue rewriting where we left off above:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;&amp;amp; \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ R_{t:t+n} + \gamma^n V_{t + n - 1}(S_{t + n}) \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert \\&#xA;%&#xA;=&amp;amp; \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ \gamma^n V_{t + n - 1}(S_{t + n}) \mid S_t = s \right] - \gamma^n \mathbb{E}_{\pi} \left[ \sum_{k = 0}^{\infty} \gamma^k R_{t + n + k + 1} \mid S_t = s \right] \Bigr\rvert \\&#xA;%&#xA;=&amp;amp; \gamma^n \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ V_{t + n - 1}(S_{t + n}) - \sum_{k = 0}^{\infty} \gamma^k R_{t + n + k + 1} \mid S_t = s \right] \Bigr\rvert&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the absolute value function is &lt;a href=&quot;https://en.wikipedia.org/wiki/Convex_function&quot; rel=&quot;noreferrer&quot;&gt;convex&lt;/a&gt;, we can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Jensen%27s_inequality&quot; rel=&quot;noreferrer&quot;&gt;Jensen's inequality&lt;/a&gt; to show that the absolute value of an expectation is less than or equal to the expectation of the corresponding absolute value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left| \mathbb{E} \left[ X \right] \right| \leq \mathbb{E} \left[ \left| X \right| \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} - v_{\pi}(s) \mid S_t = s \right] \Bigr\rvert &amp;amp;\leq \gamma^n \max_s \mathbb{E}_{\pi} \left[ \Bigl\lvert V_{t + n - 1}(S_{t + n}) - \sum_{k = 0}^{\infty} \gamma^k R_{t + n + k + 1} \Bigr\rvert \mid S_t = s \right] \\&#xA;%&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the important trick here is to see that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \mathbb{E}_{\pi} \left[ \Bigl\lvert V_{t + n - 1}(S_{t + n}) - \sum_{k = 0}^{\infty} \gamma^k R_{t + n + k + 1} \Bigr\rvert \mid S_t = s \right] \leq \max_s \mathbb{E}_{\pi} \left[ \Bigl\lvert V_{t + n - 1}(S_{t}) - \sum_{k = 0}^{\infty} \gamma^k R_{t + k + 1} \Bigr\rvert \mid S_t = s \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm skipping the formal steps to show that this is the case to save space, but the intuition is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The left-hand side of this inequality involves finding an &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; such that some function of &lt;span class=&quot;math-container&quot;&gt;$S_{t + n}$&lt;/span&gt; is maximized, whereas the right-hand side involves finding an &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; such that exactly the same function of &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt; is maximized.&lt;/li&gt;&#xA;&lt;li&gt;In the left-hand side, selecting an &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; implicitly induces a probability distribution over multiple possible states &lt;span class=&quot;math-container&quot;&gt;$S_{t + n}$&lt;/span&gt;, given by &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;, the environment's transition dynamics, and the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Intuitively, this is more &quot;restrictive&quot; for the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator, it does not have the &quot;freedom&quot; to directly select a single state &lt;span class=&quot;math-container&quot;&gt;$S_{t + n}$&lt;/span&gt; such that the function of &lt;span class=&quot;math-container&quot;&gt;$S_{t + n}$&lt;/span&gt; is maximized. The right-hand side is free to choose any single state &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; in the right-hand side were equal to an &quot;optimal&quot; &lt;span class=&quot;math-container&quot;&gt;$S_{t+n}$&lt;/span&gt; on the left-hand side, but it is also free to make even better choices which might never be uniquely reachable after &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps on the left-hand side.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We can use this to rewrite the previous inequality we had (where we might be making the right-hand side a bit bigger than it was, but that's fine, it already was an upper bound anyway so that inequality will still hold):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} - v_{\pi}(s) \mid S_t = s \right] \Bigr\rvert &amp;amp;\leq \gamma^n \max_s \mathbb{E}_{\pi} \left[ \Bigl\lvert V_{t + n - 1}(S_{t}) - \sum_{k = 0}^{\infty} \gamma^k R_{t + k + 1} \Bigr\rvert \mid S_t = s \right] \\&#xA;%&#xA;&amp;amp;= \gamma^n \max_s \mathbb{E}_{\pi} \left[ \Bigl\lvert V_{t + n - 1}(S_{t}) - v_{\pi}(s) \Bigr\rvert \mid S_t = s \right].&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this rewriting we've got a hidden &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt; &quot;inside&quot; another &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt; (because the definition of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; contains an &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt;), which I suppose is kind of ugly... but mathematically meaningless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maximum of a random variable is an upper bound on the expectation of that random variable, so we can get rid of the expectation in the right-hand side (again potentially increasing the right-hand side, which again is still fine since it's already an upper bound anyway):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} - v_{\pi}(s) \mid S_t = s \right] \Bigr\rvert &amp;amp;\leq \gamma^n \max_s \Bigl\lvert V_{t + n - 1}(s) - v_{\pi}(s) \Bigr\rvert,&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which we can finally rewrite to Equation (7.3) in the book by moving the subtraction of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; outside of the expectation on the left-hand side of the inequality (which is fine because, as I already mentioned above, the definition of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; itself contains another &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt; anyway).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="22916" LastEditDate="2019-03-18T09:27:54.243" LastActivityDate="2019-03-18T09:27:54.243" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="9427" PostTypeId="2" ParentId="9425" CreationDate="2018-12-09T11:58:45.900" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Universal Approximation Theorem&lt;/a&gt;&lt;/strong&gt; states: In the mathematical theory of artificial neural networks, the universal&#xA;  approximation theorem states that a feed-forward network with a single&#xA;  hidden layer containing a finite number of neurons can approximate&#xA;  continuous functions on compact subsets of Rn, under mild assumptions&#xA;  on the activation function. The theorem thus states that simple neural&#xA;  networks can represent a wide variety of interesting functions when&#xA;  given appropriate parameters; however, it does not touch upon the&#xA;  algorithmic learnability of those parameters.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So clearly if you do not have infinite number of hidden neurons you cannot approximate a function with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; error tending to 0. So if we assume that the previous layer gives some sort of representation of the function to be approximated you still need infinite hidden neurons to approximate this representative function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But probably it'll be more clear to you if we go into basic physical laws of mathematical approximation. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;According to the &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier Theorem&lt;/a&gt;&lt;/strong&gt; by Joseph Fourier: A Fourier series is a way to&#xA;  represent a function as the sum of simple sine waves. More formally,&#xA;  it decomposes any periodic function or periodic signal into the&#xA;  weighted sum of a (possibly infinite) set of simple oscillating&#xA;  functions, namely sines and cosines (or, equivalently, complex&#xA;  exponentials).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So theoretically, if your function is periodic it maybe perfectly approximated by an activation function like &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$cos$&lt;/span&gt;. The simple reason you cannot use an activation like &lt;span class=&quot;math-container&quot;&gt;$ReLu$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$Sigmoid$&lt;/span&gt; is because they are non-periodic and as per another related theorem called &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier Transform&lt;/a&gt; you would require infinite number of sine waves to approximate non-periodic curves so the problem collapses in itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there are various ways to train a Neural Network other than Gradient Descent theoretically speaking you just might approximate a periodic function with some other methods (if activation is sine or cos), it is almost imposiible via methods like Gradient Descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in short the answer to your question will be &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt; for the aforementioned reasons.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2018-12-09T11:58:45.900" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9428" PostTypeId="2" ParentId="9417" CreationDate="2018-12-09T13:02:59.553" Score="2" Body="&lt;p&gt;The main problems are that your activation function is not &lt;strong&gt;monotonic&lt;/strong&gt; (as pointed out by csrev), and that it is not &lt;strong&gt;continuously differentiable&lt;/strong&gt;. These make it very difficult / impossible to use standard gradient-based learning algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, there may exist a good solution of weight values, but it is very difficult to find or approximate those weight values automatically through a learning algorithm. Also note that it completely breaks down as soon as you have a tiny error in one of the weights, even if it is approximate very closely; if one of the weights has a value of &lt;span class=&quot;math-container&quot;&gt;$0.999$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt;, the solution breaks down completely.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-09T13:02:59.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9429" PostTypeId="1" CreationDate="2018-12-09T15:13:44.007" Score="7" ViewCount="218" Body="&lt;p&gt;In some tweets about NeurIPS 2018, &lt;a href=&quot;https://news.developer.nvidia.com/nvidia-opening-core-ai-ml-research-lab-in-santa-clara/?_lrsc=45894c3e-3518-40ee-9b28-2be5f79597ef&amp;amp;ncid=so-twi-lt-799&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; from NVIDIA appeared. At around 0.37, she says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you think about the current computations in our deep learning systems, they are all based on Linear Algebra. Can we come up with better paradigms to do multi-dimensional processing? can we do truly tensor-algebraic techniques in our tensor cores?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was wondering what she is talking about. I'm not an expert so I'd like to understand better this specific point.&lt;/p&gt;&#xA;" OwnerUserId="20540" LastEditorUserId="2444" LastEditDate="2019-11-22T00:23:09.500" LastActivityDate="2019-11-22T00:23:09.500" Title="What does it mean to do multi-dimensional processing with tensors in tensor cores?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9435" PostTypeId="2" ParentId="9429" CreationDate="2018-12-10T07:37:27.507" Score="2" Body="&lt;p&gt;&lt;strong&gt;Both PR and Theoretically Valid&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although Anima Anandkumar's presentation is a puff piece for NVidia, her representation is not contrary to theory.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... next level [above NVidia's GPU success] ... means new algorithmic research. So, if you think about the current computations in our deep learning systems, they are all based on linear algebra. Can we come up with better paradigms to do multi-dimensional processing. Can we do truly tensor algebraic techniques in our tensor cores, and what kind of new architectures will this realize?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tensors&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A tensor is the extension of the idea of functions and vectors. Vectors emerged allow the encapsulation of multiple variables of the same type into a single unit to express multidimensional properties such as position or force. Variables that are not encapsulations of multiple variables are then referred to as scalars. When such multidimensional properties are compositions of functions rather than variables because their values are dependent upon other variables, they are vector fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To more cleanly model electromagnetism and gravity in mathematics, higher levels of variable and functional encapsulation were required. Scalar variables and functions that return scalar variables are thus tensors of rank one (1). Vectors and vector fields are tensors of rank two (2), and higher ranks are extensions of the pattern created by scalars and vectors. A matrix containing functions can be represented as a tensor of rank three (3), encapsulating its elements into a single unit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Digital Processing and Integration Trend&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CUDA cores, which are the signal processing units used in current generation NVidia GPUs (graphical processing units) can be used for production of two or three dimensional rendering or leveraged to produce parallel processing of signals through an artificial network. This is following the trend to delegate to digital circuitry the most significant bottlenecks in algorithm execution. VLSI (very large scale integration) technology is the logical result of this trend.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Industrial and military calculators with tests and branching running on rows of racks of relays&lt;/li&gt;&#xA;&lt;li&gt;Add speed with tubes and load programs with punch cards&lt;/li&gt;&#xA;&lt;li&gt;Reduce power and improve reliability and speed with transistors, magnetic core memory, and paper tape&lt;/li&gt;&#xA;&lt;li&gt;Integrated circuits on motherboards with vinyl tape&lt;/li&gt;&#xA;&lt;li&gt;Handheld programmable calculators for students&lt;/li&gt;&#xA;&lt;li&gt;Microprocessors with magnetic disks&lt;/li&gt;&#xA;&lt;li&gt;Floating point, digital signal, and graphics processors to extend microprocessors&lt;/li&gt;&#xA;&lt;li&gt;Multiple cores (repeating large scale circuit patterns on a single substrate)&lt;/li&gt;&#xA;&lt;li&gt;Re-purposing and extending graphics rendering circuits to offload artificial network computations to GPUs&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With all these, speed, size, energy thrift, and convenience are not really a shift in the original paradigm that started with Norbert Wiener, Claude Shannon, Alan Turing, John von Neumann, and others. In fact, computing is still catching up in several ways with the perspectives of these pioneers and a long way off from producing in VLSI common concepts in science. Software is the solution, which is why it is called soft, meaning flexible, not necessarily weak. However, flexibility sacrifices speed and capability, thus the trend above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Parallel Computing Challenge&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of what Anima Anandkumar is stating that the signal paths in current VLSI processors are still a much lower level of abstraction than the ideas of mathematicians, physicists, and AI engineers. Scientific theory describes probability, statistical distribution, expectation, force, loss, gain, pain, reward, memory, momentum, semantics, combination, and correlation at a much higher level than current digital circuitry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use of sequential algorithms is the splaying out in time what could be a massively parallel operation. The serial algorithm limits the rate of processing. The finding of ways of dealing with things in parallel in mathematics can be done with a pencil. In computers, paralleling algorithms and finding parallel processing structures in VLSI form that are as flexible as software in some respects is much more challenging and thus far behind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is one problem that has been the focus of research in at least twenty large corporate and government labs since for half a century and has been the intention of VLSI from the onset. The work is not specific to NVidia. It is not a new problem and the approach to solving it has been along this paradigm.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Add abstraction and encapsulation in the mind.&lt;/li&gt;&#xA;&lt;li&gt;Express it in mathematics.&lt;/li&gt;&#xA;&lt;li&gt;Write it as a serial algorithm, leveraging whatever parallel constructs are supported by programming languages and libraries that can leverage VLSI parallelism or computing clusters.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last seventy years of computing machinery development has been bringing hardware, operating systems, and software closer to the level of mathematical expressions that were twenty to two hundred years old. That may change, and everyone wants to ride the new wave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anima Anandkumar and her counterparts in IBM, Intel, Google, Microsoft, the U.S. Navy, Amazon, Alibaba, and the other corporate and government laboratories do not state (because it is either classified or company confidential) what they intend to do to further the parallelization of computing. Whatever they do along those particular lines would not be a paradigm shift but rather a next step along the current paradigm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Enters Corporate Strategy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They would also not state what they may be doing that is not in that paradigm. They are constrained to only give hints without theoretical substance. If they were working on a chip that exhibited what the human brain exhibits when neurons grow and connect according to DNA based propensity, they wouldn't say that in a technically precise way. Corporate secrecy is part of the global economic game play, not tipping the hand. They have their game face on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of moving from linear to non-linear is a good public relations theme and when used for puff pieces is not conclusively technical. The pitch usually goes along this line.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What they did was very linear. We are moving into a non-linear space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is an attempt to claim that what was done was primitive and the impending game changing advancement is coming from the speaker and their people. Sometimes it works to create a temporary hike in the value of tradable securities, which that company needs as of this writing. If there is a true change in the game, it will be known when it is released. Those who have worked in laboratories for years know to wait until something is released that demonstrates, when the example code is configured and ran, what game change actually occurred, if any. Or they develop the game changer themselves, which is why it is a lab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ambiguity of 'Linear' Even in Mathematics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that curved lines are still lines and the term linear can mean two things depending on the context.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Conformant to the linear equation &lt;span class=&quot;math-container&quot;&gt;$\vec{Y} = V \vec{X}$&lt;/span&gt;, such that it graphs as a line, plane, or higher dimensional flat surface, with constant gradient and without curvature.&lt;/li&gt;&#xA;&lt;li&gt;Conformant to the principles of linear algebra, which would include spaces, eigenvalues, regression with high-order polynomials, and a number of constructs that involve varying gradient and curvature.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Predicting Paradigm Shifts&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The skepticism developed after watching tech company PR for some time does not necessarily dismiss game changing technology advancements of the past and potentials of the future. Any open source project team, individual, corporation, or government lab might do something that shifts the paradigm, usually over a period of years. Classic examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rope&lt;/li&gt;&#xA;&lt;li&gt;Wheel&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$F = m a$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Oxygen (that air is not an element)&lt;/li&gt;&#xA;&lt;li&gt;Electromagnetism&lt;/li&gt;&#xA;&lt;li&gt;Alternating current energy transmission&lt;/li&gt;&#xA;&lt;li&gt;Relativity&lt;/li&gt;&#xA;&lt;li&gt;Internal combustion&lt;/li&gt;&#xA;&lt;li&gt;Radio&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Computing examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Information theory&lt;/li&gt;&#xA;&lt;li&gt;Cybernetics&lt;/li&gt;&#xA;&lt;li&gt;First transistor&lt;/li&gt;&#xA;&lt;li&gt;LISP and FORTRAN&lt;/li&gt;&#xA;&lt;li&gt;C and UNIX&lt;/li&gt;&#xA;&lt;li&gt;TCP/IP&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Shifts do not need to be as far reaching and game changing as these to have an impact. Who will achieve the next with regard to AI concepts, information structure, algorithm, execution environment is not something very easy to predict. Consider those that lived before any of the above and try to imagine them trying to predict that the projects of Isaac Newton, Antoine Lavoisier, Michael Faraday, Nicola Tesla, Norbert Wiener, Claude Shannon, Ken Thompson, Dennis Ritchie, or any of the others were the seeds of the next paradigm shift.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is research into analog artificial networks, neuromorphic hardware, semantic modelling, graph algorithms, and other potential game changers, each of which has impressive conceptual foundations and is discussed in some of the Q&amp;amp;A here. These are a few.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7079/what-topologies-are-largely-unexplored-in-machine-learning&quot;&gt;What topologies are largely unexplored in machine learning?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5347/novelty-search-mutation-algorithm&quot;&gt;Novelty Search Mutation Algorithm&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7416/can-deep-networks-be-trained-to-prove-theorems&quot;&gt;Can deep networks be trained to prove theorems?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai&quot;&gt;If digital values are mere estimates, why not return to analog for AI?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/9047/are-commercially-available-neural-ics-digital/9051#9051&quot;&gt;Are commercially available neural ICs digital?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What of these may point to the beginnings of paradigm shift cannot be known, and even if an idea first posted here or referenced from here is the seed, it may not be known later. The multilayer perceptron may be the seed of future streets and highways being dominated by automated vehicles in 2090, but no one in a million people will realize in seventy years that the trend toward AV research was seeded by MLP enthusiasm from the first decade of this century.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Gaps to be Filled and NVidia is a Contendere&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this aside, we use NVidia hardware every day for robotics and analysis, so they have credibility resulting from past success. If they produced a chip that does something remarkably clever before Intel or the IBM-MIT collaboration, it would be a small surprise, but not it is not a completely unbelievable possibility. Certainly the comprehension of Hilbert spaces, semantics, and topology are limited in the computer science field, and a paradigm shift toward greater comprehension of them or some new thing that is not even part of mathematical thought today would add some needed diversity to the computing industry.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-10T08:00:21.233" LastActivityDate="2018-12-10T08:00:21.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9436" PostTypeId="1" CreationDate="2018-12-10T13:30:08.223" Score="2" ViewCount="509" Body="&lt;p&gt;To be honest, I had no idea where to put this question, but it's sure that it's related to AI. I want to build an application which uses camera, and by the movement it can calculate the&#xA;-camera's position compared to the objects&#xA;-the objects creator and edge points by the movement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it means that if the camera is in a static position, it's just a picture. A set of coloured pixels. If we move the camera, we calculate the time, the gyroscope's values, but most importantly, we can have a comparison of two images taken by the same objects. This way:&#xA;-we can detect the edges&#xA;-from the edges, we can detect which is closer than the others&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today's phone camera's are accurate enough to create ~60 crystal clear images per second, and it should be enough resource to accurately create high res models from just moving the camera according to some instructions (that's why I'm surprised why it isn't existing in just a phone app). Here comes the problem. I think the idea is worth for the try, but I'm just a JavaScript developer. The browser can have access to the camera, with TensorFlow I can use machine learning to detect edges, but if I want to be honest, I have no idea where to start, and how to continue step by step. Can you please provide me some guidelines how it would be ideal to create the idea?&lt;/p&gt;&#xA;" OwnerUserId="20563" LastActivityDate="2018-12-11T17:30:41.940" Title="How to use AI to depth map video?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;mapping-space&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-23T01:51:32.953" ContentLicense="CC BY-SA 4.0" />
  <row Id="9442" PostTypeId="1" AcceptedAnswerId="9453" CreationDate="2018-12-10T17:52:37.883" Score="0" ViewCount="109" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Karel_(programming_language)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Karel the robot&lt;/a&gt; is an education software comparable to turtle graphics to teach programming for beginners. It's a virtual stack-based interpreter to run a domain-specific language for moving a robot in a maze. In its vanilla version, the user authors the script manually. That means, he writes down a computer program like&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;move forward&lt;/li&gt;&#xA;&lt;li&gt;if reached obstacle == true then stop&lt;/li&gt;&#xA;&lt;li&gt;move left.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This program is then executed in the virtual machine.&lt;/p&gt;&#xA;&lt;p&gt;In contrast, genetic programming has the aim to produce computer code without human intervention. So-called permutations are tested if they are fulfilling the constraints and, after a while, the source code is generated. In most publications, the concept is explained on a machine level. That means assembly instructions are generated with the aim to replace normal computer code.&lt;/p&gt;&#xA;&lt;p&gt;In &amp;quot;Karel the robot&amp;quot; a high-level language for controlling a robot is presented, which has a stack, but has a higher abstraction. The advantage is, that the state space is smaller.&lt;/p&gt;&#xA;&lt;p&gt;My question is: is it possible to generate &amp;quot;Karel the robot&amp;quot; programs with genetic programming?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="2444" LastEditDate="2021-01-20T10:40:46.410" LastActivityDate="2021-01-20T10:40:46.410" Title="Is it possible to generate &quot;Karel the robot&quot; programs with genetic programming?" Tags="&lt;game-ai&gt;&lt;applications&gt;&lt;genetic-programming&gt;&lt;robots&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9443" PostTypeId="2" ParentId="9424" CreationDate="2018-12-10T19:40:14.090" Score="0" Body="&lt;p&gt;Discounted reward has its opposite undiscounted reward. The difference between &lt;em&gt;these&lt;/em&gt; is that discounted has multiplier gamma != 1 and undiscounted is gamma = 1. Gamma tells the multiplier, how much previous values are valued in next iterations. [1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additive refers to different thing, a note found in [2]:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An additive reward function decomposes the reward into a number of contributions and can be represented as a (non-partiotioning) MPA.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This short excerpt does not reveal lot, but was the only I could find that made some sense to me. What I &lt;em&gt;could&lt;/em&gt; indeed find out was that although they seem similar concepts by name, they appear totally different by nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Reinforcement_learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] The Logic of Adaptive Behavior: Knowledge Representation and Algorithms for Adaptive Sequential Decision Making Under Uncertainty In First-Order and Relational Domains.- M. Van&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2018-12-10T19:40:14.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9445" PostTypeId="1" CreationDate="2018-12-10T20:18:57.977" Score="2" ViewCount="54" Body="&lt;p&gt;Is there a mechanism in the human brain that works analog to LSTMs? Is there a biological/neuroscientific interpretation of LSTMs and recurrent neural networks? How do long-term and short-term memories work in the brain, on a neuron level?&lt;/p&gt;&#xA;&lt;p&gt;I would also appreciate links to more in-depth explanations in the literature.&lt;/p&gt;&#xA;" OwnerUserId="18713" LastEditorUserId="2444" LastEditDate="2020-12-04T14:35:28.977" LastActivityDate="2020-12-04T14:37:51.200" Title="Is there a mechanism in the human brain that works analog to LSTMs?" Tags="&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;brain&gt;&lt;neuroscience&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9446" PostTypeId="1" CreationDate="2018-12-10T22:05:55.040" Score="2" ViewCount="17" Body="&lt;p&gt;I want like to be able to draw a shape outline e.g. (pen and paper) triangle, square, circle.. then label the vertices and sides And have ML identify the shape and the symbol associates with each vertex / side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example a triangle and I label it with the adjacent, opposite and hypotenuse. Or draw parallel and perpendicular lines and label angles and such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer it make one myself rather than use a pre built one. However if you know of a simple pre built one then I will ould love to pick it apart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please share some guidance on how to solve the above problem, namely shape identification / vertex and side labelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any language&lt;/p&gt;&#xA;" OwnerUserId="20579" LastActivityDate="2018-12-10T22:05:55.040" Title="Geometry shape identification and vertex/side label association" Tags="&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9448" PostTypeId="1" CreationDate="2018-12-11T06:01:05.643" Score="2" ViewCount="1089" Body="&lt;p&gt;In AIMA, 3rd Edition on Page 125, Simulated Annealing is described as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Hill-climbing algorithm that &lt;em&gt;never&lt;/em&gt; makes “downhill” moves toward states with lower value (or higher cost) is guaranteed to be incomplete, because it can get stuck on a local maximum. In contrast, a purely random walk—that is, moving to a successor chosen uniformly at random from the set of successors—is complete but extremely inefficient. Therefore, it seems reasonable to try to combine hill climbing with a random walk in some way that yields both efficiency and completeness. Simulated annealing is such an algorithm. In metallurgy, &lt;strong&gt;annealing&lt;/strong&gt; is the process used to temper or harden metals and glass by heating them to a high temperature and then gradually cooling them, thus allowing the material to reach a lowenergy crystalline state. To explain simulated annealing, we switch our point of view from hill climbing to &lt;strong&gt;gradient descent&lt;/strong&gt; (i.e., minimizing cost) and imagine the task of getting a&#xA;  ping-pong ball into the deepest crevice in a bumpy surface. If we just let the ball roll, it will come to rest at a local minimum. If we shake the surface, we can bounce the ball out of the local minimum. The trick is to shake just hard enough to bounce the ball out of local minima but not hard enough to dislodge it from the global minimum. The simulated-annealing solution is to start by shaking hard (i.e., at a high temperature) and then gradually reduce the intensity of the shaking (i.e., lower the temperature)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I know its about its example, but I just want more examples where Stimulated Annealing used in daily life &lt;/p&gt;&#xA;" OwnerUserId="19442" LastEditorUserId="2444" LastEditDate="2019-03-02T11:07:49.820" LastActivityDate="2019-03-02T11:07:49.820" Title="What are examples of daily life applications that use simulated annealing?" Tags="&lt;search&gt;&lt;applications&gt;&lt;simulated-annealing&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9451" PostTypeId="2" ParentId="9319" CreationDate="2018-12-11T08:53:52.680" Score="1" Body="&lt;p&gt;If the concept class specified is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x, y) = k \, \sin(2 \pi f_x x) \, sin(2 \pi f_y y) \\ \land 0 &amp;lt; x &amp;lt; 1 \\ \land 0 &amp;lt; y &amp;lt; 1 \; \text{,}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the optimum fit to example data is expected occur when &lt;span class=&quot;math-container&quot;&gt;$k \approx 1 \land f_x \approx 1 \land f_y \approx 1$&lt;/span&gt;, then it is not an AI problem. It is a problem that can be solved with a least squares convergence, probably in conjunction with a Fourier transform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If nothing is known about &lt;span class=&quot;math-container&quot;&gt;$f(x, y)$&lt;/span&gt; except continuity and that it is single valued with respect to &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt;, then few conclusions can be drawn about best approach. In such a case, the domain of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are irrelevant because they can be normalized. Furthermore, the tree of operations, such as &lt;span class=&quot;math-container&quot;&gt;$\sin()$&lt;/span&gt; and multiplication, are irrelevant too, because the function could just as easily be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x, y) = \ln(x) + \Gamma(y) - k \, \text{.}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question indicates the design involves CNN and RNN components for analyzing images and sequential data. It is not clear whether the CNN is for the discovery of objects or waves (given the &lt;span class=&quot;math-container&quot;&gt;$\sin()$&lt;/span&gt; in the function mentioned) and whether those objects move between frames so that the RNN must detect motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nothing is given about the pool of example data available or planned to be available or the expected outputs of the system. If data is sequential, where is &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the function? What is the objective of image analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although a deep MLP (multilayer perceptron) with SGD can learn an arbitrary function, it is hardly an architecture, the mention of images, CNN, RNN, and sequential data, MLP with SGD does not seem to match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding activation functions, the inner layer functions would depend on the higher level design requirements. The activation functions of the last layer of a single artificial network is usually chosen to match the data type and range of desired output for each output channel (dimension).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the objective of this question is to take images and sequential data and produce something useful without &lt;em&gt;a priori&lt;/em&gt; defining what useful means, then it is an unsolved AI problem thus far and no known topology comprised of artificial networks and other AI building blocks provide a solution. The autonomous development of internal concepts of usefulness would need to be developed mathematically and algorithmically and become practically speed optimized in hardware and software first.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-11T08:53:52.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9453" PostTypeId="2" ParentId="9442" CreationDate="2018-12-11T09:36:13.200" Score="0" Body="&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithm&lt;/a&gt; is used to find an optimal solution to a problem. The parameters of the problem are encoded in the population, where each individual contains a solution to the problem in itself. Each individual is evaluated using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;fitness function&lt;/a&gt;, and then the highest scoring individuals are used to create a new set of individuals by mixing up their parameters, and this new set replaces the worst scoring individuals from the previous generation. Over time, the average fitness of the population will (hopefully) increase, and a good solution be found. Essentially, the search space is explored in parallel through the many individuals in the population, which means that local maxima can probably be avoided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question now is: &lt;strong&gt;What is the problem you want to optimise?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Karel language is fairly simple, and has few tokens. A program should be easily encodable in the right format, though genetic algorithms usually require individuals to have the same size (though shorter programs could be padded with no-ops). Recombining the best programs could mean swapping tokens. Maybe syntactic constraints could be factored into this to avoid crass syntactic errors. That should not be an insurmountable problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that still leaves the question of the fitness function. A basic criterion would be syntactic: if the program doesn't parse, it's not 'fit'. This could potentially wipe out a lot of offspring, and doesn't really help you much: it would produce more or less random programs which are syntactically well-formed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you really want is to evaluate the &lt;em&gt;outcome&lt;/em&gt; of the programs. Like, which program can best navigate a maze. So you need to execute them and see what they produce. Then you run straight away into the &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;halting problem&lt;/a&gt;: your little program might contain an endless loop and will never terminate. So you need to add a kind of time-out, or a limit on the steps the program can execute. This should be easy to do in a virtual machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question: Yes, it is possible to turn creating Karel programs into a problem that a genetic algorithm can be applied to. However, you need to be clear about what problem you want to solve. I don't quite understand what you mean by &quot;learning from demonstration&quot; or how you can achieve the aim of replicating human actions, but if you can somehow encode that in a fitness function, GAs are a tool you can use.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-12-11T09:36:13.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9454" PostTypeId="2" ParentId="84" CreationDate="2018-12-11T09:53:09.487" Score="1" Body="&lt;p&gt;I dont know why you wouldnt consider it ai since every single thing has used something like it thats been in the recent news.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;evolving a neural network is very similar to brute force search,  just it hits local optima, because its not exhaustive.&lt;/p&gt;&#xA;" OwnerUserId="20594" LastActivityDate="2018-12-11T09:53:09.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9458" PostTypeId="2" ParentId="6748" CreationDate="2018-12-11T13:42:01.340" Score="1" Body="&lt;p&gt;Investigating reinforcement as a way of producing more interesting behavior than behavior trees for AI based commercial strategy game development is a good idea. The simple test game given can be described briefly.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Human pitted against AI&lt;/li&gt;&#xA;&lt;li&gt;Each player gets a fleet of ships&lt;/li&gt;&#xA;&lt;li&gt;Each ship begins with a positive health level&lt;/li&gt;&#xA;&lt;li&gt;One ship can be ordered to fire one shot at one ship per turn&lt;/li&gt;&#xA;&lt;li&gt;Each hit produces a decrease in ship health&lt;/li&gt;&#xA;&lt;li&gt;Ships with non-positive health are removed&lt;/li&gt;&#xA;&lt;li&gt;Player with no remaining ships are removed&lt;/li&gt;&#xA;&lt;li&gt;The remaining player wins&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The question is correct in that game states must be represented as tensors. Ideally, they should be terse representations, with minimal redundancy. The actions must be represented as tensors too. The goal to train a network to play a game is generally feasible, but the simple game may not provide sufficient complexity to lead to learning. It is probably too simple. What is there to learn? In terms of the expectation regarding the result of the game, what advantage is there to firing on one ship over another?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be useful to understand that although machine learning can involve networks that can be represented as directed graphs, it is not the graphs of the network cells that are saved or loaded to persist learned behavior. The networks are parameterized, and those parameters are the representation of what is learned. Where this is most important is that, provided the form of the representation is standardized, the learning hardware and software environment need not match that of the environment that later leverages what is learned. The comment, &quot;Build a custom runtime that can run the net [using the previously learned parameters],&quot; is the correct method if this approach is taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reinforcement learning, the representation is less like a graph and more like a structure involving a flexible behavioral policy that is affected by a changing distribution of probabilities. The expectations of future game states resulting from actions are part of the evaluation that must occur before each move. One theoretical approach to automated game play, for which a few varieties exist, is Q-learning. There are others using Markov trees and recurrent networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the approach is to learn something in the lab that can be used in an xBox or Playstation, then supervised learning is indicated, and a high level language may assist in rapid prototyping. The result of training would be saved in language independent form, therefore readable by the portable c/c++. If the approach is to continuously change, then the reinforcement technology must be running on those platforms and most therefore be contained in statically or dynamically linked portable c/c++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of these development approaches can be used in conjunction. For this basic investigative work and probably the first year or two of R&amp;amp;D, the use of Python for rapid prototyping would not require the use of Pybind. For the most common operations, the interoperability between Python and the mature C code ported from FORTRAN for tensor processing years ago is already present. The Python should never run on the game platform. It is not compiled and runs orders of magnitude less efficiently than C/C++ or Java. Determining whether the C++ API of tensorflow ports well to the desired target game environments is part of the architecture process. &lt;a href=&quot;https://www.tensorflow.org/api_docs/cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Tensorflow underlying C++ API&lt;/a&gt; that Python ML newbies aren't generally concerned with is worth review by those who develop commercial games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If 99.9% of the time, games are still developed on a Microsoft OS, game developers may be excessively patient. Fedora LINUX is much more productive for C, C++, Java, and Python development for both low level and high level programming tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of this has addressed the mention of a more creative opponent in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classic training goal in AI game play is to produce winning behavior, but games don't sell when the human always loses. Setting reward and loss functions to optimize on the basis of victory will not create sustainable sales of commercial game products. What a game developer might more wisely chose as a basis for optimization is engagement. Sacrifice victory in favor of sensation. You want gamers to blog to each other and talk in school and work about some crazy and unexpected thing the game did. Consequently, the goals of the learning must be adjusted in terms of training to maximize game value based on these product market realities.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T08:39:33.147" LastActivityDate="2018-12-12T08:39:33.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9459" PostTypeId="2" ParentId="6634" CreationDate="2018-12-11T14:32:54.347" Score="2" Body="&lt;p&gt;The graph seems to show the relative compute time of each component of the learning process, independent of the layer, using the typical meaning of layer in the context of artificial networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data is not unbelievable, in that one would expect a matrix-vector multiplication of an entire layer output would take more time than a matrix-vector multiplication of the size of a convolution kernel. Still, these representations are for a specific test condition and measurement methodology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the theoretical and practical limits of parallel processing are heavily biased by seventy years of sequential processing as the foundation of computer science study and practice. If an understanding of parallel processing is desired, the von Neumann architecture is not the place to look. At the time of that development, computers could either be made of wood, steel, vacuum tubes, or relays. Developing a computer with 65536 parallel data paths was unrealistic to the extreme. The invention of algorithms was a workaround to funnel all processing through a single CPU. We've been backing out of that mind set for the last thirty years, beginning with FPUs, GPUs, multiple buses, computer clusters, and multiple cores. More parallelism in VLSI form is coming, and soon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The demonstration of parallel processing comes, as many things do, from biology. Look at farming and grain growth under the sun. Look at visual circuitry coming from the eyes of thousands of species. Consider the orbits of millions of planets and their satellites. The artificial parallelism to examine is the transportation systems and the Internet. Parallel processing is efficient when the process was designed as a parallel process rather than trying to make parallel that which was designed as serial. For instance, the human ear does the physical equivalent of a real time windowing FFT without performing a single multiplication operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottleneck of the floating point multiplication of an attenuation matrix with a vector (parameterizing the mix of signals from one layer to the next) is a key to why NVidia has done well with re-purposing rendering hardware for artificial network training. It is also one of the many reasons this question is interesting: &lt;a href=&quot;https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai&quot;&gt;If digital values are mere estimates, why not return to analog for AI?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When surfing for GFLOPs vs compute time for different types of layers across GPUs and CPUs, using the term 'operation' instead of 'layer' in the search terms may help. Even though elementary operations are lower level than that of the comparison desired, the term layer is delegated strongly to the higher level meaning in the machine learning literature. Reliable data in this space is thin because much of the speed optimization has already been researched and developed, so whatever the visualization shows is useless unless you work on VLSI development at Intel, Samsung, NVidia, Qualcom, TI, or one of the other silicon giants.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-11T14:32:54.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9463" PostTypeId="1" AcceptedAnswerId="9467" CreationDate="2018-12-11T16:43:11.477" Score="6" ViewCount="113" Body="&lt;p&gt;I want to learn a lot about the AI of CCG, such as Hearthstone. And now I have known one of the main algorithms that used in this kind of games, MCTS. It analyses the most promising moves, and expands the search tree based on random sampling of the search space. But there are too many random events in this game that can cause different results to one battle. For example, a card can randomly deal X damage to a hero or other follower, and X is a random number from 0 to 30. The number of X is important for the next decision, but there will be a low accuracy by only using MCTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what does the AI do to deal with these random events?&lt;/p&gt;&#xA;" OwnerUserId="20601" LastEditorUserId="1641" LastEditDate="2018-12-11T20:53:05.270" LastActivityDate="2018-12-11T20:53:05.270" Title="How does Hearthstone AI deal with random events" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9464" PostTypeId="2" ParentId="9436" CreationDate="2018-12-11T17:00:16.330" Score="3" Body="&lt;p&gt;I think that the tasks you are referring to are the &lt;a href=&quot;https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simultaneous localization and mapping&lt;/a&gt; (SLAM) and, in particular, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Structure_from_motion&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structure from Motion&lt;/a&gt; (SfM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods are usually based on geometrical constraints and do not employ neural networks, but there exist some recent methods that make use of CNNs (such as &lt;a href=&quot;http://www.luigifreda.com/2017/04/08/cnn-slam-real-time-dense-monocular-slam-learned-depth-prediction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Structure from Motion algorithms are a fundamental component of the &lt;a href=&quot;https://developers.google.com/ar/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google ARCore&lt;/a&gt; (former Project Tango) and the &lt;a href=&quot;https://developer.apple.com/documentation/arkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apple ARKit&lt;/a&gt;.&#xA;Unfortunately, these kits usually provide an interface to put elements in a specific position, but they don't provide the access to the 3d reconstruction of the scene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to play with SfM algorithms, I suggest you to start from &lt;a href=&quot;https://github.com/mapillary/OpenSfM&quot; rel=&quot;nofollow noreferrer&quot;&gt;this repository&lt;/a&gt;, but you can find a lot of other valuable tutorials online.&lt;/p&gt;&#xA;" OwnerUserId="20566" LastActivityDate="2018-12-11T17:00:16.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9467" PostTypeId="2" ParentId="9463" CreationDate="2018-12-11T20:52:47.673" Score="4" Body="&lt;p&gt;The most &quot;standard&quot; implementation of MCTS probably involves storing copies of game states inside nodes. This works fine for deterministic games, but not for non-deterministic games due to the reasons you mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In non-deterministic games, one of the easiest ways to make MCTS work is to take the perspective that every node in your tree should map to / represent / encode a &lt;strong&gt;trajectory&lt;/strong&gt; or &lt;strong&gt;sequence of actions&lt;/strong&gt;, rather than a &lt;strong&gt;state&lt;/strong&gt;. This means that every node collects statistics (num wins / num visits etc.) for the trajectory of actions it represents, rather than for the state it represents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This perspective can be implemented very easily by simply not storing any game states in nodes at all. Instead, whenever you traverse the tree (in the Selection phase of MCTS), you should simply re-simulate all the actions along that path through the tree, starting from the root state again. This way, different visits through a single node can result in different game states being &quot;observed&quot; in that node due to nondeterministic game state transitions. In the limit (after an infinite amount of time), every node will estimate the &lt;strong&gt;expected value of the trajectory of actions leading to that node&lt;/strong&gt;, rather than the game-theoretic value of a single, deterministic resulting game state. This whole idea is sometimes referred to as &quot;Open-Loop MCTS&quot;, for example in &lt;a href=&quot;http://www.diego-perez.net/papers/OpenLoopGVG.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What I described above is a simple way to get better performance from MCTS in games involving nondeterministic state transitions as you described (such as random amounts of damage being dealt), but there are many much more complicated ideas out there too. I suspect more complex approaches will especially be necessary to deal with more &quot;drastic&quot; forms of nondeterminism (resulting from random card draws, or resulting from randomization-based approaches for dealing with imperfect information such as not knowing which cards your opponent has).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was a &lt;a href=&quot;https://dockhorn.antares.uberspace.de/wordpress/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hearthstone AI competition&lt;/a&gt; relatively recently, and some useful ideas may be found, for example, on this page of &lt;a href=&quot;https://dockhorn.antares.uberspace.de/wordpress/additional-material/&quot; rel=&quot;nofollow noreferrer&quot;&gt;work related to that competition&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-11T20:52:47.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9468" PostTypeId="1" CreationDate="2018-12-11T21:32:46.220" Score="2" ViewCount="161" Body="&lt;p&gt;I am trying to understand how to train a neural network to win a Pong game using reinforcement learning, by following the blog post &lt;a href=&quot;https://blog.floydhub.com/spinning-up-with-deep-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&#xA;Spinning up a Pong AI with deep reinforcement learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environment is provided by &lt;a href=&quot;http://gym.openai.com/envs/Pong-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gym AI&lt;/a&gt;. It gives the AI a reward of 1 if the opponent misses the ball, and a reward of -1 if it misses the ball.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused about how reward discounting works in this context. This is the function that &lt;a href=&quot;https://github.com/mtrazzi/spinning-up-a-Pong-AI-with-deep-RL/blob/master/karpathy.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;the blog post used&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def discount_rewards(r, gamma):&#xA;  &quot;&quot;&quot; take 1D float array of rewards and compute discounted reward &quot;&quot;&quot;&#xA;  r = np.array(r)&#xA;  discounted_r = np.zeros_like(r)&#xA;  running_add = 0&#xA;  # we go from last reward to first one so we don't have to do exponentiations&#xA;  for t in reversed(range(0, r.size)):&#xA;    if r[t] != 0: running_add = 0 # if the game ended (in Pong), reset the reward sum&#xA;    running_add = running_add * gamma + r[t] # the point here is to use Horner's method to compute those rewards efficiently&#xA;    discounted_r[t] = running_add&#xA;  discounted_r -= np.mean(discounted_r) #normalizing the result&#xA;  discounted_r /= np.std(discounted_r) #idem&#xA;  return discounted_r&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, the list of rewards is mostly filled with zeros, because usually nothing happens. When something happens, e.g. the reward is 1, this is not only due to the action taken in that step. Therefore, we need to smoothen the list of rewards so that some of that reward also belongs to previous actions. So far so good. However, it seems to me that if the opponent misses the ball and the reward is 1, then this will be smeared such that it will emphasize the actions taken right before the opponent missed the ball. This seems wrong to me, the actions taken by the AI right before the opponent missed the ball are irrelevant. They don't affect the ball in any way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only think reward discounting makes sense when you have lost a point, then the actions just preceding the loss are surely very important and should be emphasized. However, the function takes into account both wins and losses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should reward discounting be understood in the context of this Pong game?&lt;/p&gt;&#xA;" OwnerUserId="20607" LastActivityDate="2018-12-11T21:32:46.220" Title="Reward discounting in reinforcement learning for a Pong game" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9470" PostTypeId="1" CreationDate="2018-12-11T22:24:48.217" Score="3" ViewCount="6624" Body="&lt;p&gt;I want to implement YOLO V3. I want to know which framework will give me a faster result.&lt;/p&gt;&#xA;&lt;p&gt;What are the advantages of implementing YOLO V3 on the darknet framework vs Keras framework?&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="2444" LastEditDate="2021-01-29T00:09:32.407" LastActivityDate="2021-01-29T00:09:32.407" Title="How to speed up YOLOv3 detection speed?" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;yolo&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9471" PostTypeId="2" ParentId="9470" CreationDate="2018-12-11T22:58:23.520" Score="4" Body="&lt;p&gt;Darknet is &quot;native&quot; framework, so basically, you don't need to implement anything, all code for yolov3 is available at their github repo, you just need to figure it out, play with it. Keras, in my opinion, is not flexible enough to easily implement yolo. If you want to implement yolo from scratch I would probably go with PyTorch it has a dynamic graph + more integration with python. Regarding speed, it's hard to say objectively which framework is faster, especially comparing with darknet, since probably nobody did a comparative analysis. The easiest solution to speed up yolo3 is obviously buying more powerful GPU :D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in my understanding what they did in yolov3 is that they intentionally sacrificed speed in order to be able to detect smaller objects, so if you don't care too much about small grouped up objects go with yolov2 it is very fast and has a pretty decent mAP.&lt;/p&gt;&#xA;" OwnerUserId="13102" LastActivityDate="2018-12-11T22:58:23.520" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9472" PostTypeId="5" CreationDate="2018-12-11T23:02:15.707" Score="0" Body="&lt;p&gt;Risk Management is a term used on corporations and law and is central to the practical application of AI. Although risk cannot practically be eliminated, safety precautions are apropos when dealing with potentially powerful technologies. Just as important data should be backed up to guard against data loss and elevators should have more than one safety measure, safety should be considered in the design phase of every AI system. Principles of total quality management (TQM) applied to AI indicate that it would not be inappropriate to consider safety routinely during research phases of development and on site as part of a maintenance process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of short term risk management:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Collision avoidance through recognition of valued objects and mutual trajectory analysis for AI systems that govern the paths of street vehicles, trains, or vehicles that travel through fluids such as aircraft, subs, or future nano devices that are hoped to pass from blood to cytoplasm to make repairs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AI systems that detect emergency conditions and must react quickly to avoid the propagation of infection or the development of higher levels of criticality.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Examples of long term risk management:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Review of objective definition that drives learning algorithms to ensure that proper safety is built into the foundations of the AI system being developed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Consideration of whether any given AI system should be deployed with a roll back plan and a retirement plan, as is done with power plants, automobiles, and all organisms of the biosphere.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-11T23:02:15.707" LastActivityDate="2018-12-11T23:02:15.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9473" PostTypeId="4" CreationDate="2018-12-11T23:02:15.707" Score="0" Body="For questions regarding AI risk management; including the mitigation of long term risk such as directing technology advancement with safety in mind; and the mitigation of immediate risk such as collision avoidance through trajectory analysis and planning." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-11T23:02:15.707" LastActivityDate="2018-12-11T23:02:15.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9474" PostTypeId="1" CreationDate="2018-12-12T01:45:17.587" Score="2" ViewCount="84" Body="&lt;p&gt;What kind of search method is A*? Explain to me with an example.&lt;/p&gt;&#xA;" OwnerUserId="19370" LastEditorUserId="2444" LastEditDate="2019-06-21T16:19:45.743" LastActivityDate="2019-06-21T20:25:45.000" Title="What kind of search method is A*?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;definitions&gt;&lt;a-star&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9478" PostTypeId="2" ParentId="4672" CreationDate="2018-12-12T06:53:16.953" Score="0" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most any decision making with a specific sets of objectives and guidelines can be framed as a game. In mid 2017, it would have It is difficult to suggest one AI approach to a winning chess AI that had value outside of chess game play. Historically, game strategy AI approaches varied greatly from game to game and research lab to research lab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach to tic-tac-toe is the trivial case in AI, since it is easy to calculate the probability of winning for every possible sequence of moves on a single CPU in less than a second. At the other extreme of game complexity and dynamics, only the technical talent contractually obligated to secrecy in contracts with the portfolio managers of billionaires know how to play the game of high speed trading, optimized for maximum gain in position per hour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The international champion in inter-computer chess play is Stockfish. It produced the highest consistency in wins in international tournaments thus far. Stockfish performs its best move search in a highly parallel CPU run time environment. A sophisticated pruning and depth prioritization involving late move reductions is applied to the alpha-beta search algorithm, in conjunction with a bit board.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In human-computer play Zor was the international champion in 2017.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Change in AI Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2017, DeepMind’s AlphaZero defeated Stockfish 28–72-0. (The number of draws is positioned in the center number.) That it won the tournament is not the remarkable advancement from an AI perspective. The same algorithm, configured differently also plays a winning game at Go and Shogi. The approach and design is described in &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/em&gt;, David Silver et. al., 2017&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaZero is configured with the rules of the game to constrain the search. It begins training prior to tournament game play without prior knowledge of game play. That no triggered game strategies or heuristic rules to guide the search are known before learning game play, the authors claim the learning is &lt;em&gt;tabula rasa&lt;/em&gt;, Latin meaning blank slate. Reinforcement learning is used to develop a strategy during self-play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaZero, &quot;Averages over the position evaluations within a subtree, rather than computing the minimax evaluation of that subtree,&quot; as is commonly used in chess players based on the alpha-beta approach. It determines the relative value of states (board positions) based on a DNN trained to produce values associated with all of the state information from outcomes. This is distinct from valuation by summing points assigned to pieces and their locations. AlphaZero uses a Monte-Carlo tree search (MCTS) algorithm, using ordered deepening to minimize computing resource utilization per move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MCTS mitigates the numerical error, DNN convergence artifacts that can accumulate at the root of each sub-tree, via aggregation. The aggregation of data that contains reasonably normalized noise causes cancellation of the symmetrically distributed deviations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The achievement is significant. The abstract of the paper states, &quot;Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Remaining Domain Dependence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without dismissing the achievement, the claim of &lt;em&gt;no domain knowledge except the game rules&lt;/em&gt; conflicts with the additional five items listed in the &lt;em&gt;Domain Knowledge&lt;/em&gt; section of the same paper, reproduced here solely for convenience of the reader.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The input features describing the position, and the output features describing the move, are structured as a set of planes; i.e. the neural network architecture is matched to the grid-structure of the board.&lt;/li&gt;&#xA;&lt;li&gt;AlphaZero is provided with perfect knowledge of the game rules. These are used during MCTS, to simulate the positions resulting from a sequence of moves, to determine game termination, and to score any simulations that reach a terminal state.&lt;/li&gt;&#xA;&lt;li&gt;Knowledge of the rules is also used to encode the input planes (i.e. castling, repetition, no-progress) and output planes (how pieces move, promotions, and piece drops in shogi).&lt;/li&gt;&#xA;&lt;li&gt;The typical number of legal moves is used to scale the exploration noise (see below).&lt;/li&gt;&#xA;&lt;li&gt;Chess and shogi games exceeding a maximum number of steps (determined by typical game length) were terminated and assigned a drawn outcome; Go games were terminated and scored with Tromp-Taylor rules, similarly to previous work.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In addition to game rules and the above five listed domain dependencies, there are further dependencies.The higher level analysis of the consequences of the board geometry and game rules is not a fundamental game rule.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In Go game play, &quot;During MCTS, board positions were transformed using a randomly selected rotation or reflection before being evaluated by the neural network, so that the MonteCarlo evaluation is averaged over different biases.&quot; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rules are analyzed to scale, &quot;Noise that is added to the prior policy to ensure exploration ... in proportion to the typical number of legal moves for that game type.&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This totals eight dependencies, and a grossly insufficient number of game types were tried to make a claim that AlphaZero is a, &quot;domain-independent search.&quot; But bending the the concept of domain independence in the claims is more easily forgiven when legitimate and significant achievements are made by an enthusiastic team. The approach is sound and the use of the craft by the DeemMind team is world class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Beyond Removing Remaining Dependencies&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even after the eight dependencies are whittled down by the various teams involved in game play automation, there is further work that may become the focus of further research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Board games are a particular kind of game. In games where the game rules can mutate, such as markets, law, war, and other domains where the boundaries of the domain require knowledge of broader sets of domains is more challenging, although it is reasonable to expect that dependencies will be reduced and current approaches that work with board games may become adaptive in terms of game rule acquisition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, approaches like AlphaZero require that input preparation be designed and executed and that outputs be executed on a virtual board. They do not yet discover game states by vision, execute moves with robotics, or acquire the rules of the game from natural language descriptions or the analysis of past games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These limitations do not invalidate the significant advancement of a game player that needs only rules, a small set of domain specific configurations, and some self play time to defeat champion level dedicated artificial game players.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-12T06:53:16.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9479" PostTypeId="1" CreationDate="2018-12-12T07:13:17.103" Score="4" ViewCount="1164" Body="&lt;p&gt;What kind of problems is simulated annealing better suited for compared to genetic algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my experience, genetic algorithms seem to perform better than simulated annealing for most problems.&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="2444" LastEditDate="2019-02-21T11:26:34.303" LastActivityDate="2019-03-23T12:00:50.137" Title="When should I use simulated annealing as opposed to a genetic algorithm?" Tags="&lt;genetic-algorithms&gt;&lt;search&gt;&lt;optimization&gt;&lt;simulated-annealing&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9480" PostTypeId="2" ParentId="8735" CreationDate="2018-12-12T08:09:00.730" Score="2" Body="&lt;p&gt;If your search tree's branching factor is finite but its depth is unbounded, then DFS isn't &quot;complete&quot;. What that means is that it may not ever find the goal node even if it exists. You would use these techniques precisely when this situation arises.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand this, let's say you're at the node A in the following tree:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     A&#xA;    / \&#xA;   /   \&#xA;  B     C&#xA; / \   / \&#xA;.   . .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You choose to explore subtree rooted at B. Now if there are no goal nodes in that subtree, but the goal nodes were in the subtree rooted at C, then we're stuck. Because  we will keep searching there because we don't know beforehand that there are no goal nodes there, next node we explore could always be a goal node. But this essentially means that we'll never finish searching the subtree rooted at B in finite time, and never backtrack to explore the subtree rooted at C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to overcome this you would want to iterative deepening depth-first search or depth-limited search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;strong&gt;depth-limited search&lt;/strong&gt; you set a limit on the depth you'll explore. Let's say you choose &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; as the depth limit because you know that the solution lies within that depth limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And let's say the branching factor for your tree is &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. Going back to our example, you have finite elements in the subtree rooted at B (at most &lt;span class=&quot;math-container&quot;&gt;$b^d$&lt;/span&gt;, can you reason why?), so we'll be able to look at all of them and eventually backtrack and start exploring subtree rooted at C, and we'll be able to find our answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you don't know the depth &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; which goal nodes lie at, then you'll use &lt;strong&gt;iterative deepening&lt;/strong&gt; DFS, which essentially does depth limited search but keeps increasing the value of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; on each iteration, so you're guaranteed to find a goal node, if it exists.&lt;/p&gt;&#xA;" OwnerUserId="16637" LastEditorUserId="16637" LastEditDate="2018-12-12T13:51:09.003" LastActivityDate="2018-12-12T13:51:09.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9481" PostTypeId="2" ParentId="9470" CreationDate="2018-12-12T09:00:43.920" Score="1" Body="&lt;p&gt;You can find a working implementation of Yolo3 in Keras/Tensorflow here: &lt;a href=&quot;https://github.com/qqwweee/keras-yolo3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/qqwweee/keras-yolo3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have been using it extensively lately and it works correctly. Evaluation speed is mostly the same as that of Darknet, probably because both implementations use libcudnn under the hood.&lt;/p&gt;&#xA;" OwnerUserId="16394" LastActivityDate="2018-12-12T09:00:43.920" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9482" PostTypeId="2" ParentId="8735" CreationDate="2018-12-12T09:17:24.190" Score="1" Body="&lt;p&gt;In practice you can consider that IDS is a cheap way to implement a breadth-first search (BFS). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the branching factor is high, both algorithms have similar computational costs O(B^P) (where P is the length of the solution path and B the branching factor) but IDS requires less memory (typically O(P), while BFS uses O(B^P)).&lt;/p&gt;&#xA;" OwnerUserId="16394" LastActivityDate="2018-12-12T09:17:24.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9483" PostTypeId="5" CreationDate="2018-12-12T09:25:42.740" Score="0" Body="" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T09:25:42.740" LastActivityDate="2018-12-12T09:25:42.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9484" PostTypeId="4" CreationDate="2018-12-12T09:25:42.740" Score="0" Body="For questions related to successive approximation, a generalization of convergent or adaptive approaches that involve continuous improvement via iteraton, environment mapping, gradient descent, intelligent numerical methods for differential equations without a closed form equivalent, and flight control stabilization via deep learning, Gradient descent being a special case in common use should use the [gradient-descent] tag instead." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T09:25:42.740" LastActivityDate="2018-12-12T09:25:42.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9485" PostTypeId="5" CreationDate="2018-12-12T09:28:48.170" Score="0" Body="" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T09:28:48.170" LastActivityDate="2018-12-12T09:28:48.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9486" PostTypeId="4" CreationDate="2018-12-12T09:28:48.170" Score="0" Body="For questions related to adaptive systems, that not only learn to perform according to specified objectives and criteria but adapt to changes in the criteria or the presentation of obstacles to performance in the environment." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T09:28:48.170" LastActivityDate="2018-12-12T09:28:48.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9487" PostTypeId="5" CreationDate="2018-12-12T09:34:45.303" Score="0" Body="" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T09:34:45.303" LastActivityDate="2018-12-12T09:34:45.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9488" PostTypeId="4" CreationDate="2018-12-12T09:34:45.303" Score="0" Body="For questions related to greedy searches and greedy state-action strategies in that they prioritize or focus on the acquisition of more near term rewards than the mapping of plans to outcomes to achieve optimal or most likely desirable end states or stable continuity. In the extreme case, greedy AI system only evaluates the immediate next step. In the moderate case, the AI simply simulates a degree of doubt toward extended planning." OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-12T09:34:45.303" LastActivityDate="2018-12-12T09:34:45.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9490" PostTypeId="1" AcceptedAnswerId="9500" CreationDate="2018-12-12T11:06:03.693" Score="0" ViewCount="59" Body="&lt;p&gt;I'm trying to perform image classification with a CNN. In my case, the inputs are the covers of 9 books, so there are 9 labels. I am using TensorFlow's Keras.&lt;/p&gt;&#xA;&lt;p&gt;If I pass a new input (that has a label different than one of the 9 labels the CNN was trained with), it will be classified as one of the 9 books, even though it's not a book (but it's e.g. a wall, sofa, house, etc.). I want to avoid this. I want the model to &lt;strong&gt;first classify whether there is a book in the image&lt;/strong&gt; and then &lt;strong&gt;classify the book in 9 classes&lt;/strong&gt;. How could I achieve this?&lt;/p&gt;&#xA;" OwnerUserId="18985" LastEditorUserId="2444" LastEditDate="2021-01-16T20:30:16.327" LastActivityDate="2021-01-16T20:30:16.327" Title="How can I prevent the CNN from classifying a new input into one of the existing labels (it was trained with) when the input has a new different label?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;keras&gt;&lt;object-detection&gt;&lt;multiclass-classification&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-16T20:35:38.343" ContentLicense="CC BY-SA 4.0" />
  <row Id="9491" PostTypeId="1" CreationDate="2018-12-12T13:27:58.903" Score="17" ViewCount="6853" Body="&lt;p&gt;This question is regarding &lt;strong&gt;Reinforcement Learning&lt;/strong&gt; and &lt;strong&gt;different/inconsistent action space for every/some states&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;What do I mean by &lt;em&gt;different/inconsistent action space&lt;/em&gt;?&lt;/h2&gt;&#xA;&lt;p&gt;Let say you have an MDP where the number of actions varies between states (for example like in Figure 1 or Figure 2). We can express &amp;quot;inconsistent action space&amp;quot; formally as &lt;span class=&quot;math-container&quot;&gt;$$\forall s \in S: \exists s' \in S: A(s) \neq A(s') \wedge s \neq s'$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is, for every state, there exists some other state which do not have the same action set. In the figures (1, 2) there's a relatively small amount of actions per state. Instead imagine states &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$m_s$&lt;/span&gt; number of actions, where &lt;span class=&quot;math-container&quot;&gt;$1 \leq m_s \leq n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is a really large integer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q2Huu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q2Huu.png&quot; alt=&quot;Some MDP's&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Environment&lt;/h2&gt;&#xA;&lt;p&gt;To get a better grasp of the question, here's an environment example. Take Figure 1 and let it explode into a really large directed acyclic graph with a source node, huge action space and a target node. The goal is to traverse a path, starting at any start node, such that we'll maximize the reward which we'll only receive at the target node. At every state, we can call a function &lt;span class=&quot;math-container&quot;&gt;$M : s \rightarrow A'$&lt;/span&gt; that takes a state as input and returns a valid number of actions.&lt;/p&gt;&#xA;&lt;h2&gt;Approches&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A naive approach to this problem (discussed &lt;a href=&quot;https://ai.stackexchange.com/q/7755/2444&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/2219/2444&quot;&gt;here&lt;/a&gt;) is to define the action set equally for every state, return a negative reward whenever the performed action &lt;span class=&quot;math-container&quot;&gt;$a \notin A(s)$&lt;/span&gt; and move the agent into the same state, thus letting the agent &amp;quot;learn&amp;quot; what actions are valid in each state. This approach has two obvious drawbacks:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Learning &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; takes time, especially when the Q-values are not updated until either termination or some statement is fulfilled (like in &lt;a href=&quot;https://datascience.stackexchange.com/q/20535/10640&quot;&gt;experience replay&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We know &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, why learn it?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another approach (&lt;a href=&quot;https://ai.stackexchange.com/a/8564/2444&quot;&gt;first answer here&lt;/a&gt;, also very much alike proposals from papers such as &lt;a href=&quot;https://arxiv.org/pdf/1512.07679&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning in Large Discrete Action Spaces&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1705.05035&quot; rel=&quot;nofollow noreferrer&quot;&gt;Discrete Sequential Prediction of continuous action for Deep RL&lt;/a&gt;) is to instead predict some scalar in continuous space and, by some method, map it into a valid action. The papers are discussing how to deal with large discrete action spaces and the proposed models seem to be a somewhat solution for this problem as well.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another approach that came across was to, assuming the number of different action set &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is quite small, have functions &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_2}$&lt;/span&gt;, ..., &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_n}$&lt;/span&gt; that returns the action regarding that perticular state with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; valid actions. In other words, the performed action of a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with 3 number of actions will be predicted by &lt;span class=&quot;math-container&quot;&gt;$\underset{a}{\text{argmax}} \ f_{\theta_3}(s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;None of the approaches (1, 2 or 3) are found in papers, just pure speculations. I've searched a lot, but I cannot find papers directly regarding this matter.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know any papers, or having other approaches, regarding different/inconsistent action space?&lt;/p&gt;&#xA;" OwnerUserId="20627" LastEditorUserId="20627" LastEditDate="2021-01-14T08:34:12.443" LastActivityDate="2021-01-14T08:34:12.443" Title="Any papers regarding different/inconsistent action space in Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;action-spaces&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="19" ClosedDate="2020-11-17T19:03:27.933" ContentLicense="CC BY-SA 4.0" />
  <row Id="9494" PostTypeId="1" AcceptedAnswerId="9499" CreationDate="2018-12-12T14:33:00.403" Score="0" ViewCount="76" Body="&lt;p&gt;I'm working on a home tool that will help create a shopping list from a list of recipes chosen for a coming week.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This boils down to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extracting ingredients and their quantities from recipes.&lt;/li&gt;&#xA;&lt;li&gt;Grouping similar ingredients together.&lt;/li&gt;&#xA;&lt;li&gt;Summing up quantities for similar ingredients.&lt;/li&gt;&#xA;&lt;li&gt;Naming groups of similar products in a shopping list.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The tasks seem non-trivial for a few reasons.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Similar ingredients are described differently&lt;/strong&gt;, depending on the recipe book/portal, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;5 lemons&lt;/li&gt;&#xA;  &lt;li&gt;5 lemons (to be squeezed)&lt;/li&gt;&#xA;  &lt;li&gt;5 fresh lemons&lt;/li&gt;&#xA;  &lt;li&gt;5 big yellow lemons&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Recipes lists alternatives for ingredients&lt;/strong&gt; (e.g., &lt;em&gt;&quot;3 lemons or 5 limes&quot;&lt;/em&gt;), leaving decision up to a user. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Recipes involve some information about product-preprocessing&lt;/strong&gt;. For instance, one has to buy lemons instead of lemon juice when the recipe says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;100ml lemon juice&lt;/li&gt;&#xA;  &lt;li&gt;100ml freshly squeezed lemon juice&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;My language has a complex inflection&lt;/strong&gt;. For instance, there can be multiple plural forms of a noun and the form of an adjective must be agreed with a form of noun. Adapting NLP algorithms designed for English language might be not straightforward and require some lemmatizing/stemming but not for single words, but whole phrases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Naming products group is hard&lt;/strong&gt;. Once fresh lemons and big yellow lemons are group together and their quantities summed up, one need to decide how to name this group in a shopping list, e.g.: &lt;em&gt;&quot;10 lemons&quot;&lt;/em&gt; or &lt;em&gt;&quot;10 fresh lemons&quot;&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there any research paper that would cover those challenges?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially applied in the same domain? &lt;/p&gt;&#xA;" OwnerUserId="20631" LastEditorUserId="20631" LastEditDate="2018-12-12T14:57:31.997" LastActivityDate="2018-12-12T18:01:17.957" Title="Grouping products and naming groups" Tags="&lt;natural-language-processing&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9495" PostTypeId="2" ParentId="9448" CreationDate="2018-12-12T14:48:28.750" Score="1" Body="&lt;p&gt;Simulated annealing is just one of the approaches for an &lt;a href=&quot;https://en.wikipedia.org/wiki/Optimization_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimization problem&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given a function f(&lt;em&gt;X&lt;/em&gt;), you want to find an &lt;em&gt;X&lt;/em&gt; where f(&lt;em&gt;X&lt;/em&gt;) is optimal (has maximum or minimum value). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Shaking (a methaphor for introducing a degree of randomness to annealing process) is used to prevent the algorithm from stopping prematurely, when only a sub-optimal solution (local minimum or local maximum) has been found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've once used it for finding the optimal schedule of lessons for teacher. The optimization function took into consideration &quot;time windows&quot; between windows, so that teachers do not have to wait too much.&lt;/p&gt;&#xA;" OwnerUserId="20631" LastEditorUserId="20631" LastEditDate="2018-12-12T14:54:12.190" LastActivityDate="2018-12-12T14:54:12.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9496" PostTypeId="1" CreationDate="2018-12-12T15:15:26.693" Score="10" ViewCount="11838" Body="&lt;p&gt;I wonder if it's possible to 'clean up' an audio recording of a lecture from a smartphone, using some type of AI system?&lt;/p&gt;&#xA;" OwnerUserId="20633" LastActivityDate="2020-04-23T22:52:48.680" Title="Audio enhancing using AI? (removing background noise during lecture recording)" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9497" PostTypeId="2" ParentId="9496" CreationDate="2018-12-12T15:39:56.747" Score="7" Body="&lt;p&gt;Yes, it is possible. Usually the noise reduction is done using regular signal processing methods, such as spectral subtraction due to demand for low latency. But of course, modern methods of deep learning is applicable to this problem. For example variational autoencoder is the first that come to my mind, you can check this  &lt;a href=&quot;https://github.com/ghunkins/Voice-Denoising-AN&quot; rel=&quot;noreferrer&quot;&gt;project&lt;/a&gt;. As another example you can look at this &lt;a href=&quot;https://github.com/drethage/speech-denoising-wavenet&quot; rel=&quot;noreferrer&quot;&gt;project&lt;/a&gt; , which is based on WaveNet.&lt;/p&gt;&#xA;" OwnerUserId="18908" LastActivityDate="2018-12-12T15:39:56.747" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9499" PostTypeId="2" ParentId="9494" CreationDate="2018-12-12T18:01:17.957" Score="1" Body="&lt;p&gt;I worked at a start-up doing this exact task for two years. There were several developers involved in it, and it took several years; by the end (with a lot of manual input, eg categorising ingredients in an ontology, converting measurements, and mapping ingredients to shopping items from a supermarket inventory) we had something that worked reasonably well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You rightly recognise that it is not trivial... The first step would be to identify the list of ingredients from the recipe (assuming you have scraped it from a website), and turn it into a list; beware that sometimes there might be more than one ingredient in a line. Then you need to analyse the entry: we had a grammar that distinguished between quantities, attributes (&lt;em&gt;fresh&lt;/em&gt;), processing instructions (&lt;em&gt;peeled&lt;/em&gt;) and a number of other things. The main item would of course be the actual ingredient. Then we would look this up in a supermarket inventory and map it to the closest item (but I don't think this is part of your remit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We tried different ways of analysing the recipe lines, and any grammar/parser is fine, as long as it can deal with partial analyses, as you cannot always rely on lines being complete and well-formed, and there is always scope for additional comments you want to ignore (&quot;I only use the freshest lemons directly imported from Tuscany&quot;, &quot;get your fishmonger to take the bones out&quot;). In the simplest case you could have a list of items (&lt;em&gt;lemon&lt;/em&gt;) with inflections etc, that you try to match in your recipe lines. If you don't care whether it's big lemons or not, don't bother using a full-blown grammar. And even then, you could just look at a list of adjectives preceding an ingredient to identify any modifiers. We used a grammar based on recursive transition networks (RTNs, where the parser was complex but the grammars easy to handle) and one that was a simple phrase structure one (where the parser was simpler, but the grammar was much more complex). Both worked fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I am aware there is not really much written about this kind of work. The main advice I can give you is to look at as many recipes as possible to get an idea for what you will encounter, and cut corners wherever possible -- it's a big job to do it perfectly, and you can get very far by using simple methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also transferred it into different languages (Polish too, which I assume you're working with?). We didn't find that inflections were that big an issue. You just have a list of all inflected forms linking to the lemma. As you have a limited domain, that is a perfectly feasible approach. Just have a master list of ingredients, with all alternate forms (&lt;em&gt;courgette&lt;/em&gt; and &lt;em&gt;zucchini&lt;/em&gt; are the same thing, for example).&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2018-12-12T18:01:17.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9500" PostTypeId="2" ParentId="9490" CreationDate="2018-12-12T19:10:04.440" Score="3" Body="&lt;p&gt;You can introduce another class to your network - &quot;not a book&quot;. After that, you will need to add new data to your dataset, random images that do not contain books to classify and train your network on that data. So when your network won't see a book it will output high probability for &quot;not a book&quot; class, if an image with a book will be shown to the network probability of the &quot;not a book&quot; class should be low.&lt;/p&gt;&#xA;" OwnerUserId="13102" LastActivityDate="2018-12-12T19:10:04.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9501" PostTypeId="2" ParentId="2248" CreationDate="2018-12-12T23:24:46.847" Score="0" Body="&lt;p&gt;The term &lt;em&gt;closed-form expression&lt;/em&gt;, with the word &lt;em&gt;form&lt;/em&gt; in it, is a mathematical expression that can be evaluated in a finite number of operations. In the formal discipline of first-order logic, the technicality of no free variables is a requirement, a technicality that surrounds the use and meaning of the operators&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\forall$&lt;/span&gt; (for all) and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\exists$&lt;/span&gt; (exists).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The most common use of the term is in calculus and other higher mathematics, where direct evaluation presents challenges. If you bind &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in the following calculus expression, solving for a closed-form for &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is possible, but one cannot bind &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and solve for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to produce a closed-form.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$e^x = \int \, \sin (y + \log x) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Few integrals reduce to a closed-form expression. The integrals in high school honours math or undergraduate calculus are chosen specifically because they submit to the technique being covered. We couldn't learn integration by parts if it was impossible. Those examples are useful but not representative. Most mathematical models of natural distributions of probability or natural phenomena that can be modeled by systems of differential equations are unusable to the vast majority of people who would benefit from using those models.&lt;/p&gt;&#xA;&lt;p&gt;Not everyone in science and technology has experience with various techniques of evaluating those forms that cannot be expressed in closed form, which is where search algorithms come in. One can search for values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is bound. Depending on the problem, we can download finite element analysis frameworks, machine learning frameworks, or numerical evaluation libraries.&lt;/p&gt;&#xA;&lt;p&gt;In AI, someone with more mathematical experience will, write a paper or advise the person writing it, and papers that contain an algorithm for evaluation has become a common practice. The author or someone else will write code to realize the algorithm. That's largely how the machine learning libraries formed.&lt;/p&gt;&#xA;&lt;p&gt;Until the representations of the models developed through scientific research are converted to either&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A closed-form expression (such as a relation or formula) or&lt;/li&gt;&#xA;&lt;li&gt;An algorithm (perhaps with an open-source implementation,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;most people won't ever find the time to learn how to use them. Closed forms are the most usable and widely used, and algorithms are next in line. Thus those developing the theory behind the technology, upstream from engineers and other users of the technology, convert the theoretical forms into closed-form expressions where possible. Those are the forms that appear in engineering textbooks as formulas or relations, with abbreviated proofs, some examples, and problems to practice for tests.&lt;/p&gt;&#xA;&lt;p&gt;In AI, as with other areas of technology, we may study and develop the mathematical expressions, write code that implements the published algorithms or just download what we hope is already tested code and apply it. Rarely all three. Most of us focus primarily on one part of the stream from ontological inception to the consumer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \text{pioneering investigation} \\&#xA;\Downarrow \\ \text{development of theory} \\&#xA;\Downarrow \\ \text{conversion in to practical forms} \\&#xA;\Downarrow \\ \text{initial users of those forms} \\&#xA;\Downarrow \\ \text{writers of technical publications} \\&#xA;\Downarrow \\ \text{experts in using the technology} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Closed-form expressions (along with algorithms) are critical to the communication between those working in the middle two roles in this stream. This is one of the many reasons why science and technology are collaborative endeavours.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="36737" LastEditDate="2021-04-01T15:00:40.413" LastActivityDate="2021-04-01T15:00:40.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9502" PostTypeId="1" AcceptedAnswerId="9519" CreationDate="2018-12-13T00:15:17.507" Score="2" ViewCount="125" Body="&lt;p&gt;How to select an action in a state if the action does not necesarily cause the environment to change state? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given 10 states (&lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$S_9$&lt;/span&gt;) and in each state &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; there are two actions defined &lt;span class=&quot;math-container&quot;&gt;$(1,-1)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; increases a parameter of the environment and &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; decreases it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the parameter is speed and it is currently 1 rad/s, corresponding to &lt;span class=&quot;math-container&quot;&gt;${S_i}$&lt;/span&gt;, an action can be either increase or decrease this and therefore transition to the next state, ideally &lt;span class=&quot;math-container&quot;&gt;$S_{i-1}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$S_{i+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is unclear how to formulate a reinforcement learning problem in this case. The problem is the same magnitude of the parameter change through an action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The range of the parameter is (3, 25) and step size is 1. The problem is thath the responce of the environment is not the same in every satate. In some states a parameter change with magnitude 1 results in a state transition in some state the magintude proves to be to small to provoke a transition change. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the envitonment is instate &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; and action 1 is applied, how can the magintude of the paremter be adopted in a way which assures that the environment will transition to state &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;?  How can the step in the paremeter change be made adaptive? Actually my environment is uncertain that is why I don't know exactly whether this action will take me to next state or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your information, I am using Q learning off policy algorithm. Suppose state 9 is the goal state and  my Q table is &lt;span class=&quot;math-container&quot;&gt;$10 \times 2$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20655" LastEditorUserId="2585" LastEditDate="2018-12-14T21:01:28.890" LastActivityDate="2018-12-14T22:03:46.020" Title="Reinforcement Learning with Adaptive Action Magnitude" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9504" PostTypeId="1" AcceptedAnswerId="9516" CreationDate="2018-12-13T04:33:11.933" Score="4" ViewCount="687" Body="&lt;p&gt;I've been working on a game-playing engine for about half a year now, and it uses the well known algorithms. These include minimax with alpha-beta pruning, iterative deepening, transposition tables, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm now looking for a way to include Monte Carlo tree search, which is something I've wanted to do for a long time. I was thinking of just making a new engine from scratch, but if possible I'd like to somehow import MC tree search into the engine I've already built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any interesting strategies to import MC tree search into a standard game-playing AI?&lt;/p&gt;&#xA;" OwnerUserId="16917" LastEditorUserId="2444" LastEditDate="2019-05-12T21:32:34.567" LastActivityDate="2019-05-12T21:32:34.567" Title="Any interesting ways to combine Monte Carlo tree search with the minimax algorithm?" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9505" PostTypeId="2" ParentId="1742" CreationDate="2018-12-13T08:34:40.293" Score="2" Body="&lt;p&gt;When I started Machine Leraning chapters in book used to look like this&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I) Supervised:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Regression  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear models &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Classification  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Logestic Regression&lt;/li&gt;&#xA;&lt;li&gt;Neural Network &lt;/li&gt;&#xA;&lt;li&gt;Decision Tress and Random Forest &lt;/li&gt;&#xA;&lt;li&gt;Boosting and Bagging &lt;/li&gt;&#xA;&lt;li&gt;SVD and SVM &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;II) UnSupervised Learning:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Clustering  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;K-Means &lt;/li&gt;&#xA;&lt;li&gt;Hierarchical&lt;/li&gt;&#xA;&lt;li&gt;Gaussian Mixture Model&lt;/li&gt;&#xA;&lt;li&gt;DB Scan&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Association Learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;III) ReInforment Learning:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of a sudden chapter I&gt;2&gt;b created a sub-field of its own . Well to know why, let me tell you a bit of history. &lt;code&gt;Machine learning&lt;/code&gt; word was coined in 1959 by Arthur Samuel to signify that &lt;code&gt;machines were able to learn from data&lt;/code&gt; than explicit instruction. Initally it was broken into two groups based on if th approach required label data or not(ie regression, classification), then they realised we can cassify by clustering too which gave birth to unsupervised. And word reinforment learning was born inspired by areas of game theory. Lets keep those details aside for later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to deep learnign, the word &lt;code&gt;deep learning&lt;/code&gt; came very recently, as recent as 2008 from a Geoff Hinton conference. There people started using it to indicate a very deep neural network architecture used in a paper presented by Geoff Hinton and from then onwards it kind of became as a new way of classifying machine learning besides &lt;code&gt;supervised&lt;/code&gt;, &lt;code&gt;unsupervised&lt;/code&gt; or &lt;code&gt;reinforcement&lt;/code&gt;.(Disc: There may be odd reference of calling NN as DL before this but not so popular and acceptable prior to this)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well I sometimes feel the name &lt;code&gt;deep learning&lt;/code&gt; is somewhat misnomer, it would have been better of if it was named as &lt;code&gt;neural learning&lt;/code&gt; or to stress on depth maybe &lt;code&gt;deep neural learning&lt;/code&gt;. If you are new you might be wondering what depth I am talking about, the entire word deep came from the fact that neural network (thanks the availability of high processing abilities of GPUs) were now able to train successfully on multiple layers. The word deep can also be loosely used to include other non-neural network areas of machine learning which requires lots of computation like &lt;code&gt;deep belief net&lt;/code&gt; or &lt;code&gt;recurrent net&lt;/code&gt;. To be precise the units of the networks today are no longer a mere &lt;code&gt;neuron&lt;/code&gt; or a &lt;code&gt;perceptron&lt;/code&gt;, it can be &lt;code&gt;LSTM&lt;/code&gt;, &lt;code&gt;GRU&lt;/code&gt; or a &lt;code&gt;capsule&lt;/code&gt;, so I guess word &lt;code&gt;deep&lt;/code&gt; now makes more sense than before.&lt;/p&gt;&#xA;" OwnerUserId="20661" LastEditorUserId="20661" LastEditDate="2018-12-13T08:57:23.187" LastActivityDate="2018-12-13T08:57:23.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9506" PostTypeId="1" CreationDate="2018-12-13T09:44:01.557" Score="2" ViewCount="133" Body="&lt;p&gt;I am learning about probabilistic graphical models and I was wondering if there is an example explaining the math behind &lt;em&gt;conditional random fields&lt;/em&gt;. Looking solely on the formula, I have no idea what we actually do. I found a lot of examples for the &lt;em&gt;hidden Markov model&lt;/em&gt;. There is a part speech-tag task where we have to find the tags for the sentence &quot;flies like a flower&quot;. On these slides (slide 8) &lt;a href=&quot;http://www.cse.aucegypt.edu/~rafea/CSCE569/slides/ch7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ambiguity Resolution: Statistical Method-Prof. Ahmed Rafea&lt;/a&gt;, an HMM is used to find the correct tags. How would I transform this model into a CRF and how would I apply the math?&lt;/p&gt;&#xA;" OwnerUserId="15391" LastEditorUserId="2444" LastEditDate="2019-09-11T22:06:33.770" LastActivityDate="2019-09-11T22:06:33.770" Title="Is there a mathematical example for Conditional Random Fields?" Tags="&lt;math&gt;&lt;reference-request&gt;&lt;generative-model&gt;&lt;conditional-random-field&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9507" PostTypeId="1" CreationDate="2018-12-13T12:06:58.553" Score="2" ViewCount="320" Body="&lt;p&gt;According to this website: &lt;a href=&quot;http://yarpiz.com/67/ypea104-acor&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yarpiz.com/67/ypea104-acor&lt;/a&gt; (in the website it is mentioned that it is a project aiming to be a resource of academic and professional scientific source codes and tutorials.):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Originally, the Ant Algorithms are used to solve &lt;strong&gt;discrete and&#xA;  combinatorial&lt;/strong&gt; optimization problems. Various extensions of Ant Colony&#xA;  Optimization (ACO) are proposed to deal with optimization problems,&#xA;  defined in &lt;strong&gt;continuous domains&lt;/strong&gt;. One of the most useful algorithms of&#xA;  this type, is ACOR, the Ant Colony Optimization for Continuous&#xA;  Domains, proposed by Socha and Dorigo, in 2008 (&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377221706006333&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is the difference between continuous domains and discrete combinatorial optimization? I appreciate if you could also mention some examples for each type.&lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="2444" LastEditDate="2019-05-27T22:25:22.687" LastActivityDate="2019-05-27T22:25:22.687" Title="What is the difference between continuous domains and discrete combinatorial optimization?" Tags="&lt;comparison&gt;&lt;swarm-intelligence&gt;&lt;combinatorics&gt;&lt;ant-colony-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9508" PostTypeId="1" CreationDate="2018-12-13T13:27:52.433" Score="1" ViewCount="32" Body="&lt;p&gt;Is it possible to mention the drawbacks/advantages of &lt;strong&gt;Swarm routing&lt;/strong&gt; (such as Ant routing etc) in comparison with &lt;strong&gt;classical routing&lt;/strong&gt; algorithms in communication networks in a general view?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, what will we gain if we replace a classical routing algorithm with a swarm routing based algorithm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we compare these two type of routing algorithms in a general view to mention their to count the drawbacks/advantages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main purpose of this question is to define applications of each of those routing approaches. Which one is more decentralized? And which one has more efficient performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my personal opinion (I am not sure about it) :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;classical&lt;/strong&gt; internet routing in &lt;strong&gt;more centralized&lt;/strong&gt; than &lt;strong&gt;swarm&lt;/strong&gt; routing (such as &lt;strong&gt;ACO&lt;/strong&gt; based routing) which does &lt;strong&gt;not&lt;/strong&gt; use any &lt;strong&gt;routing table and router&lt;/strong&gt; to avoid moving towards &lt;strong&gt;centralization&lt;/strong&gt; where those routing tables and routers &lt;strong&gt;can be manipulated&lt;/strong&gt; (as a &lt;strong&gt;point of failure&lt;/strong&gt;). Instead, &lt;strong&gt;classical&lt;/strong&gt; internet routing may be &lt;strong&gt;faster&lt;/strong&gt; than &lt;strong&gt;swarm&lt;/strong&gt; based routing. Briefly, &lt;strong&gt;classical&lt;/strong&gt; may be &lt;strong&gt;more centralized&lt;/strong&gt;, but &lt;strong&gt;faster&lt;/strong&gt; and on the other side, &lt;strong&gt;swarm&lt;/strong&gt; based is &lt;strong&gt;more decentralized&lt;/strong&gt; but may be &lt;strong&gt;slower&lt;/strong&gt;. Am I wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Please note&lt;/strong&gt; that when I say &quot;&lt;strong&gt;decentralized&lt;/strong&gt;&quot;, I mean a network like &quot;&lt;strong&gt;ad hoc networks&lt;/strong&gt;&quot; that do &lt;strong&gt;not&lt;/strong&gt; rely on &lt;strong&gt;routers or access points&lt;/strong&gt; (as a &lt;strong&gt;point of failure&lt;/strong&gt;) to avoid moving towards &lt;strong&gt;centralization&lt;/strong&gt;. In case of using routers or access point, some kind of &lt;strong&gt;centralization&lt;/strong&gt; is inherent. With this definition of &lt;strong&gt;decentralization&lt;/strong&gt;, it seems &lt;strong&gt;Swarm&lt;/strong&gt; based routing such as &lt;strong&gt;Ant routing&lt;/strong&gt; would be &lt;strong&gt;more decentralized&lt;/strong&gt;. &lt;strong&gt;However, I am not sure about it and that's my main question.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="19910" LastEditDate="2018-12-14T09:43:59.793" LastActivityDate="2018-12-14T09:43:59.793" Title="Classical Internet routing vs. Swarm routing (such as Ant routing)?" Tags="&lt;optimization&gt;&lt;swarm-intelligence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9510" PostTypeId="1" CreationDate="2018-12-13T14:33:20.940" Score="4" ViewCount="92" Body="&lt;p&gt;People could be sad, happy, depressive, angry, nervous, calm, relaxed, bored, etc. I don't know how to express all of these feelings and emotions in English terms (I'm not an English native speaker), which would enable me to search for research papers about this topic (e.g. in IEEE Xplore, Scopus, or ScienceDirect).&lt;/p&gt;&#xA;&lt;p&gt;So, is there any research on the identification/recognition of a person's feelings/emotions using facial expressions, heartbeat, body temperature, sweating, or nervous behavior (using one or all of them)?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2021-01-31T19:53:12.627" LastActivityDate="2021-01-31T19:55:48.213" Title="Is there any research on the identification of a person's feelings using features such as facial expressions or body temperature?" Tags="&lt;reference-request&gt;&lt;research&gt;&lt;sentiment-analysis&gt;&lt;affective-computing&gt;&lt;emotion-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9511" PostTypeId="2" ParentId="9507" CreationDate="2018-12-13T16:27:56.660" Score="1" Body="&lt;p&gt;A continous domain can be imagined as a space in which the axes of the coordinate systems are the parameters of the continous domain. If we take 2D Cartesian space as an example, it is a continous domain where there is an infininite amount of possibilities of placing an object in this space. Let's call the position of the object its state. Constraining the space to e.g. a sheet of paper does not change its continuity. There is still an infinte amount of posibilites of placing a dot (or any object) on that sheet of paper or with other words the state space of the dot is still infietly large. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational methods are much better in dealing with descretet spaces (mainly because the possiblities to take the next step is limited). In a discrete space the possibilities are limited. If we take the example above and discretize the sheet of paper, e.g. use a grid on the paper and say that we are not interested in the exact location of the point, just in which grid it is in we have succesfully defined a discrete space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different example for a problem in descrete domain can be navigation on a map. If we assume that the exact rajectory is not important, then we can make descrete decisions of turning left, right or moving forwards or backwards by one step. Even if add further possibilites (like moving up one step or down) until the number of possibilities in one state is not infinite the domain is descret. &lt;/p&gt;&#xA;" OwnerUserId="2585" LastActivityDate="2018-12-13T16:27:56.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9512" PostTypeId="1" CreationDate="2018-12-13T16:37:18.700" Score="2" ViewCount="55" Body="&lt;p&gt;When adding dropout to a neural network, we are randomly removing a fraction of the connections (setting those weights to zero for that specific weight update iteration). If the dropout probability is &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, then we are effectively training with a neural network of size &lt;span class=&quot;math-container&quot;&gt;$(1−p)N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the total number of units in the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this logic, there is no limit how big I can make a network, as long as I proportionately increase dropout, I can always effectively train with the same sized network, and thereby just increasing the number of &quot;independent&quot; models working together, making a larger ensemble model. Thereby &lt;strong&gt;improving generalization&lt;/strong&gt; of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if a network with 2 units already achieves good results in the training set (but not in unseen data -i.e validation or test sets-), also a network with 4 units + dropout 0.5 (ensemble of 2 models), and also a network with 8 units + dropout 0.75 (ensemble of 4 models)... and also a network with 1000 units with a dropout of 0.998 (ensemble of 500 models)!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, it is recommended to keep dropout at &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;, which advises against the approach mentioned above. So there seem to be reasons for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What speaks against blowing up a model together with an adjusted dropout parameter?&lt;/p&gt;&#xA;" OwnerUserId="20638" LastEditorUserId="2444" LastEditDate="2019-12-11T17:23:39.617" LastActivityDate="2019-12-11T17:23:39.617" Title="Should I remove the units of a neural network or increase dropout?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;dropout&gt;&lt;regularization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9513" PostTypeId="2" ParentId="9512" CreationDate="2018-12-13T17:02:12.570" Score="1" Body="&lt;p&gt;There is no incentive to increase the size of the model for not reason. If a model of size x gives the best possible performance, there is no reason to use a model of size 2*x with 0.5 dropout during training. Usually we want to find the smallest possible model with the best performance. Inflating the model just results in higher computational requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are basically suggesting to use dropout to allow the network to learn the same features more than once (creating a redundancies in the model). That is not the purpose of dropout.&#xA;Dropout is used to enable the network break unnecessary correlations which occur in the training set. For example if class 1 is the only one that contains in it features A and B, but all the training samples always feature them together. The dropout process can make the model realize that even just one of them is enough to point to class 1.&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2018-12-13T17:02:12.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9514" PostTypeId="2" ParentId="9491" CreationDate="2018-12-13T17:07:53.980" Score="14" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Does anyone know any paper regarding this subject?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not familiar with any off the top of my head... I do know that the vast majority of Reinforcement Learning literature focuses on settings with a fixed action space (like robotics where your actions determine how you attempt to move / rotate a particular part of the robot, or simple games where you always have the same set of actions to move and maybe ''shoot'' or ''use'' etc.). Another common class of settings is where the action space can easily be treated as if it always were the same (by enumerating all actions that every could be legal in some state), and filtering out illegal actions in some sort of post-processing steps (e.g. RL work in board games). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So... there might be something out there, but it's certainly not common. Most RL people like to involve as little domain knowledge as possible, and I suppose that a function that generates a legal set of actions given a particular state can very much be considered to be domain knowledge.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Is the terminology wrong? &quot;Inconsistant&quot;, &quot;Irregular&quot;, &quot;Different&quot;... ?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I wouldn't use inconsistent, because that word can be interpreted as implying that something would be &quot;wrong&quot; or &quot;ill-defined&quot;. I would say that you have a &lt;strong&gt;variable action set&lt;/strong&gt; (the action set &lt;em&gt;varies&lt;/em&gt; per state). When I search for that, there aren't a lot of results either though... but I think that term would be more promising.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Anyone having another approach worth digging into?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The problem you describe is mostly a problem in Reinforcement Learning with function approximation, in particular function approximation using Neural Networks. If you can get away with using tabular RL algorithms, the problem instantly disappears. For example, a table of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values as commonly used in the tabular, value-based algorithms does not need to contain entries for all possible &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; pairs; it's fine if it only contains entries for &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; pairs such that &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is legal in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variable action spaces primarily turn into a problem in Deep RL approaches, &lt;strong&gt;because we normally work with a fixed neural network architecture&lt;/strong&gt;. A DQN-style algorithm involves neural networks that take feature vectors describing states &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; as inputs, and provide &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; estimates as outputs. This immediately implies that we need one output node for every action, which means you have to enumerate all the actions... which is where your problem comes in. Similarly, policy gradient methods traditionally also require one output node per action, which again means you have to be able to enumerate all the actions in advance (when determining the network architecture).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still want to use Neural Networks (or other kinds of function approximators with similar kinds of inputs and outputs), the key to addressing your problem (if none of the suggestions you've already listed in the question are to your liking) is to &lt;strong&gt;realize that you'll have to find a different way to formulate your inputs and outputs, such that you are no longer required to enumerate all actions in advance&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way I can think of doing that really is if you are able to compute meaningful input features for complete state-action pairs &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt;. If you can do that, then you could, for example, build neural networks which:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Take a feature vector &lt;span class=&quot;math-container&quot;&gt;$x(s, a)$&lt;/span&gt; as input, which describes (hopefully in some meaningful way) the full pair of the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; &lt;strong&gt;and&lt;/strong&gt; the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Produce a single &lt;span class=&quot;math-container&quot;&gt;$\hat{Q}(s, a)$&lt;/span&gt; estimate as output, for the specific pair of state + action given as input, rather than producing multiple outputs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you can do that, then in any given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; you can simply loop through all the legal actions &lt;span class=&quot;math-container&quot;&gt;$A(s)$&lt;/span&gt;, compute &lt;span class=&quot;math-container&quot;&gt;$\hat{Q}(s, a)$&lt;/span&gt; estimates for them all (&lt;strong&gt;note&lt;/strong&gt;: we now require &lt;span class=&quot;math-container&quot;&gt;$\lvert A(s) \rvert$&lt;/span&gt; passes through the network rather than just a single pass as would normally be required in DQN-style algorithms), and otherwise proceed similarly to standard DQN-style algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously the requirement of having good input features for actions is not always going to be satisfied... but I doubt there are many good ways to get around that. It's very similar to the situation with states really. In tabular RL, we enumerate all states (and all actions). With function approximation, we usually still enumerate all actions, but avoid the enumeration of all states by replacing them with meaningful feature vectors (which enables generalization across states). If you want to avoid enumerating actions, you'll in a very similar way have to have some way of generalizing across actions, which again means you need features to describe actions.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-13T17:07:53.980" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9516" PostTypeId="2" ParentId="9504" CreationDate="2018-12-13T18:37:00.473" Score="3" Body="&lt;p&gt;There has indeed been some research towards combining MCTS and minimax-like algorithms. For example, the following two publications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/6633630&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte-Carlo Tree Search and minimax hybrids&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-14923-3_4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte-Carlo Tree Search and Minimax Hybrids with Heuristic Evaluation Functions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The basic intuition behind such combinations tends to be to use small minimax-like searches inside a larger MCTS search, and/or backing up proven wins/losses (proven in the sense that minimax-like search has established that a certain state is a certain win or loss when following perfect play) through the MCTS tree, in addition to the more standard value estimates.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;but if possible I'd like to somehow import MC search into the engine I've already built.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how easy that would be. As I described above, I think it is more common that small minimax searches are used inside a larger MCTS search. I think that would mean that you'd first want to build an MCTS, and then try to import your already implemented minimax into it. That's kind of the opposite / other way around of what you write that you'd like to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think integrating small MCTS searches inside a &quot;larger&quot; minimax would work well. The value estimates of small MCTS searches would be too unreliable for subsequent use in an &quot;outer&quot; minimax-like algorithm.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2019-05-12T21:32:11.037" LastActivityDate="2019-05-12T21:32:11.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9518" PostTypeId="1" AcceptedAnswerId="9523" CreationDate="2018-12-13T21:07:05.210" Score="6" ViewCount="923" Body="&lt;p&gt;In reinforcement learning book from Sutton &amp;amp; Barto (2018 edition), specifically in section 7.5 of the book, they present an n-step off-policy algorithm that doesn't require importance sampling called &lt;em&gt;n-step tree backup algorithm&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other algorithms, the return in the update consisted of rewards along the way and the estimated value(s) of the node(s) at the bottom, but, in tree backup update, a return consists of things mentioned before plus the estimated values of the actions that weren't picked during these n steps, all weighted by the probability of taking the action from the previous step.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions about things in this algorithm that are unclear to me.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Why is this algorithm considered an off-policy algorithm?&lt;/em&gt; As far as I could notice, only a single target policy is mentioned and there is no talk about behaviour policy generating actions to take.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;In control we want our target policy to be deterministic, greedy policy, so how do we exactly generate actions to take in this case since behaviour policy isn't used?&lt;/em&gt; If we generate actions from the greedy policy, we won't explore, so we won't learn the optimal policy. What am I missing here?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I understood something wrong and we are actually using behaviour policy, I don't understand how would the update work in the case where our target policy is greedy. The return consists of estimates taken from actions that weren't picked, but, because our policy is greedy, the probabilities used in calculating those estimates would be 0, so the total estimate of those actions would be 0 as well. The only non 0 probability in our target policy is the one of the greedy action (which is the probability of 1), so the entire return would fall down to n-step SARSA return. &lt;em&gt;So, basically, how are we allowed to do this update in this case, why is this return allowed to replace the one with importance sampling?&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="2444" LastEditDate="2020-05-18T22:09:45.577" LastActivityDate="2020-10-20T10:02:35.297" Title="Why is the n-step tree backup algorithm an off-policy algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9519" PostTypeId="2" ParentId="9502" CreationDate="2018-12-14T13:13:42.293" Score="0" Body="&lt;p&gt;There are more possible approaches to tackle this problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a reinforcement learning method which can cope with a continous state space. This would eliminate the need for discretizing the state space which in turn, if I understood correctly, leads to problems in transitioning between states. You can also consider selecting a reinforcement learning method that can cope with a continous state space and a continous action space. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Actions do not have to be immediate. If your action 1 is transitioning form &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$S_{i+1}$&lt;/span&gt; you can have a function wich continously increases the spead until the next state is reached and only then you consider the execution of the action complete. Furthermore, the magnitude of the actions can be dependent on the current state, if it assures state transition. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can add more actions. Use RL for your advantage and define not just &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; but &lt;span class=&quot;math-container&quot;&gt;$+0.1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0.3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$+0.5$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$+0.7$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; (and same for deacreasing the speed) etc. Add a negative reward for all actions which do not cause a state transition or which cause higher jump then needed. However, care must be taken to make sure you have the same speed fro each sampling. Eg. if you are in &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; and the action +0.1 does not cause a state transition to &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; you have to reset the state (if possible) or fail the epoch, since the state you are in is not only dependent on the action and the previous state. In other words the speed of e.g. &lt;span class=&quot;math-container&quot;&gt;$1 rad/s$&lt;/span&gt; (with a tolarance band) defined as &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; will be slightly higher and ending up in &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; after applying a slight increase in the speed takes less change in the speed than from &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; without the &quot;unnoticed&quot; speed change.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2585" LastEditorUserId="2585" LastEditDate="2018-12-14T22:03:46.020" LastActivityDate="2018-12-14T22:03:46.020" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="9520" PostTypeId="1" CreationDate="2018-12-14T15:08:08.750" Score="6" ViewCount="188" Body="&lt;p&gt;In this article &lt;a href=&quot;https://www.technologyreview.com/s/612561/a-radical-new-neural-network-design-could-overcome-big-challenges-in-ai/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was my understanding that this was the entire purpose of techniques that use neural nets, to make assumptions about a system with a non-continuous data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why do we need to switch to a non-layered design to deal with these data sets better?&lt;/p&gt;&#xA;" OwnerUserId="20696" LastActivityDate="2021-05-17T12:07:35.863" Title="Why do layered neural nets struggle with continous data?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9521" PostTypeId="1" AcceptedAnswerId="9553" CreationDate="2018-12-14T16:30:50.230" Score="1" ViewCount="105" Body="&lt;p&gt;I am trying to use tensorflow / keras to play a text based game. The game opposes two players that play by answering questions by choosing an answer among the proposed ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game resembles this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Questions asked from player 1, choose value {0, 1, 2}&lt;br/&gt;&lt;/li&gt;&#xA;&lt;li&gt;Player 1 chooses answer 1&lt;br/&gt;&lt;/li&gt;&#xA;&lt;li&gt;Questions asked from player 2, choose value {0, 1}&lt;br/&gt;&lt;/li&gt;&#xA;&lt;li&gt;Player 2 chooses answer 0&lt;br/&gt;&#xA;( and so on )&lt;br/&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The issue is that I do not have any data to use for training the agents and it not possible to evaluate each actions of the agent individually.&lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to get 2 agents to play against each other and evaluate them depending on who won / lost ( the games are very short with about 20 to 30 decisions made for each player ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I have is that I do not know where to start. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I normalized my input, but I do not know how to get the 2 agents to compete, as I do not have any training data as shown in the tutorials and the agents have to complete a full game in order to evaluate their performance.&lt;/p&gt;&#xA;" OwnerUserId="20698" LastEditorUserId="2585" LastEditDate="2018-12-15T10:40:43.690" LastActivityDate="2018-12-16T17:44:59.837" Title="How can I oppose two AI agents with keras / tensoflow?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;self-play&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9523" PostTypeId="2" ParentId="9518" CreationDate="2018-12-14T20:11:58.527" Score="6" Body="&lt;p&gt;As for your first two questions: there is indeed a behaviour and a target policy, which can be different. In the example image of the &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;-step tree-backup update in the beginning of the section you mention, the actions &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$A_{t+2}$&lt;/span&gt; are assumed to be selected according to some behaviour policy, whereas a (different) target policy is used to determine weights for the different leaf nodes.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As for your third question, in the case where our target policy is greedy, lots of terms will indeed have &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; weights and therefore entirely drop out. However, this is &lt;strong&gt;not&lt;/strong&gt; always going to fall down to the same return as &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step Sarsa returns; that would not be correct, because (&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step) Sarsa is an on-policy algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case where the target policy is a greedy policy, the return will depend very much on how many actions happened to get selected by the behaviour policy which the greedy target policy also would have &quot;agreed&quot; with. If the behaviour policy already happened to have made a &quot;mistake&quot; (selected an action that the greedy policy wouldn't have selected) for &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt;, you'll end up with a standard &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;-step &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update. If the behaviour policy only made a &quot;mistake&quot; with the action &lt;span class=&quot;math-container&quot;&gt;$A_{t+2}$&lt;/span&gt; (and agreed with the target policy on &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt;), you'll get something that kind of looks like a &quot;&lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;-step &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update&quot; (informally here, because &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning isn't really a thing). Etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that this is the case by closely inspecting Equation 7.16 from the book:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+n} \doteq R_{t+1} + \gamma \color{blue}{\sum_{a \neq A_{t+1}} \pi (a \mid S_{t+1}) Q_{t + n - 1}(S_{t+1}, a)} + \gamma \color{red}{\pi (A_{t + 1} \mid S_{t + 1}) G_{t + 1 : t + n}}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Suppose that the target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is greedy, and the behaviour policy already selected action &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt; differently from what &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; would have selected (meaning that &lt;span class=&quot;math-container&quot;&gt;$\pi(A_{t+1} \mid S_{t+1}) = 0$&lt;/span&gt;). Then we know that the &lt;span class=&quot;math-container&quot;&gt;$\color{red}{\text{red}}$&lt;/span&gt; part of the equation will evaluate to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. In the &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\text{blue sum}}$&lt;/span&gt;, the &lt;span class=&quot;math-container&quot;&gt;$\pi(a \mid S_{t+1})$&lt;/span&gt; term will only evaluate to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$a = \arg\max_a Q_{t+n-1}(S_{t+1}, a)$&lt;/span&gt; (because that is the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; to which the greedy target policy would assign a probability of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;), and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; again for all other elements of the sum. So, in this example situation, we end up using the return &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n} = R_{t+1} + \gamma \max_a Q_{t+n-1}(S_{t+1}, a)$&lt;/span&gt;. This is exactly the return also used by the standard &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;-step &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update rule.&lt;/li&gt;&#xA;&lt;li&gt;Now suppose that the behaviour policy and a greedy target policy happened to &quot;agree&quot; with each other on &lt;span class=&quot;math-container&quot;&gt;$A_{t + 1}$&lt;/span&gt;, but disagree again on &lt;span class=&quot;math-container&quot;&gt;$A_{t + 2}$&lt;/span&gt;. In this case, the entire &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\text{blue sum}}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}$&lt;/span&gt; will evaluate to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, but the &lt;span class=&quot;math-container&quot;&gt;$\pi(A_{t+1} \mid S_{t+1})$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$\color{red}{\text{red part}}$&lt;/span&gt; will evaluate to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Then, &quot;inside&quot; the recursive &lt;span class=&quot;math-container&quot;&gt;$\color{red}{G_{t+1:t+n}}$&lt;/span&gt;, we'll get a very similar situation to the example above, with a single non-zero term in the blue sum and a red part that evaluates to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. In this case, the total return will be &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n} = R_{t + 1} + \gamma R_{t + 2} + \gamma^2 \max_a Q_{t + n - 1}(S_{t + 2}, a)$&lt;/span&gt;. Note that this is different from a &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;-step Sarsa return. &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;-step Sarsa would use a return with &lt;span class=&quot;math-container&quot;&gt;$\gamma^2 Q_{t+n-1}(S_{t+2}, A_{t+2})$&lt;/span&gt; at the end (where &lt;span class=&quot;math-container&quot;&gt;$A_{t+2}$&lt;/span&gt; is sampled according to the behaviour policy), rather than that term involving &lt;span class=&quot;math-container&quot;&gt;$\max_a$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What you essentially end up getting in these cases is returns similar to those of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step Sarsa, but they automatically get truncated as soon as there is disagreement between the behaviour policy and a greedy target policy (all subsequent reward observations get replaced by a single bootstrapping value estimate instead).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the examples above, I assumed completely greedy target policies, since that is what you appeared to be most interested in in your question (and is probably also the most common use-case of off-policy learning). &lt;strong&gt;Note that a target policy does not have to be greedy&lt;/strong&gt;. You can also have non-greedy target policies if you like, and then the returns will obviously change quite a bit from the discussion above (fewer &lt;span class=&quot;math-container&quot;&gt;$\pi(S, A)$&lt;/span&gt; terms would evaluate to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, there'd be more non-zero terms).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-14T20:11:58.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9526" PostTypeId="1" AcceptedAnswerId="9528" CreationDate="2018-12-14T22:25:11.380" Score="4" ViewCount="449" Body="&lt;p&gt;I've been re-reading the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on the Chinese Room argument&lt;/a&gt; and I'm... actually quite unimpressed by it. It seems to me to be largely a semantic issue involving the conflation of various meanings of the word &quot;understand&quot;. Of course, since people have been arguing about it for 25 years, I doubt very much that I'm right. However... the argument can be thought of as consisting of several layers, each of which I can explain away (to myself, at least).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;There is the assumption that being able to understand (interpret a sentence in) a language is a prerequisite to speaking in it.&lt;/strong&gt;&lt;br&gt;&#xA;Let's say that I don't speak a word of Chinese, but I have access to a big dictionary and a grammar table. I could work out what each sentence means, answer it, and then translate that answer back into Chinese, all without speaking Chinese myself. Therefore, being able to interpret (parse) a language is not a prerequisite to speaking it.&lt;br&gt;&#xA;(Of course, by &lt;a href=&quot;https://en.wikipedia.org/wiki/Extended_cognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;the theory of extended cognition&lt;/a&gt; I &lt;em&gt;can&lt;/em&gt; interpret the language, but we can all agree that the books and lookup tables are simply a source of information and not an algorithm; I'm still the one using them.)&lt;br&gt;&#xA;Nevertheless, this task can be removed by a dumb natural language parser and a dictionary, converting Chinese to the set of concepts and relationships encoded in it and vice versa. There is no understanding involved at this stage.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;There is the assumption that being able to understand (identify and maintain a train of thought about concepts in) a language is a prerequisite to speaking in it.&lt;/strong&gt;&lt;br&gt;&#xA;We've already optimised away the language, to a set of concepts and relationships between concepts. Now all we need is another lookup table: a sort of verbose dictionary that maps concepts to other concepts and relationships between them. For example, one entry for &quot;computer&quot; might be &quot;performs calculations&quot; and another might be &quot;allows people to play games&quot;. An entry for &quot;person&quot; might be &quot;has opinions&quot;, and another might be &quot;has possessions&quot;. Then an algorithm (yes, I'm introducing one now!) would complete a simple optimisation problem to find a relevant set of concepts and relationships between them, and turn &quot;I like playing computer games&quot; into &quot;What is your favourite game to play on a computer?&quot; or, if it had some entries on &quot;computer games&quot;, &quot;Which console do you own?&quot;.&lt;br&gt;&#xA;The only &quot;understanding&quot; here, apart from the dumb optimisation algorithm, is the knowledge bank. This could conceivably be parsed from Wikipedia, but for a good result it would probably be at least somewhat hand-crafted. Following this would fall down, because this process wouldn't be able to talk about itself.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;There is the assumption that being able to understand (&quot;know&quot; how information in affects one's self) a language is a prerequisite to speaking in it.&lt;/strong&gt;&lt;br&gt;&#xA;A set of &quot;opinions&quot; and such associated with the concept &quot;self&quot; could be implemented into the knowledge bank. All meta-cognition could be emulated by ensuring that the knowledge bank had information about cognition in it. However, this program would still just be a mapping from arbitrary inputs to outputs; even if the knowledge bank was mutable (so that it could retain the current topics from sentence to sentence and learn new information) it would still, for example, not react when a sentence is repeated to it verbatim 49 times.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;There is the assumption that being able to have effective meta-cognition is a prerequisite to speaking in a language.&lt;/strong&gt;&lt;br&gt;&#xA;Except... there's not. The program described would probably pass the Turing Test. It certainly fulfils the criteria of speaking Chinese. And yet it clearly doesn't think; it's a glorified search engine. It'd probably be able to solve maths problems, would be ignorant of algebra unless somebody taught it to it (in which case, with sufficient teaching, it'd be able to carry out algebraic formulae; Haskell's type system can do this without ever touching a numerical primitive!), and would probably be Turing-complete, and yet wouldn't think. And that's OK.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So why is the Chinese Room argument such a big deal? What have I misinterpreted? This program understands the Chinese language as much as a Python interpreter understands Python, but there is no conscious being to &quot;understand&quot;. I don't see the philosophical problem with that.&lt;/p&gt;&#xA;" OwnerUserId="125" LastActivityDate="2019-05-05T04:06:17.533" Title="Why is the Chinese Room argument such a big deal?" Tags="&lt;philosophy&gt;&lt;chinese-room-argument&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9527" PostTypeId="1" CreationDate="2018-12-14T23:27:33.827" Score="2" ViewCount="136" Body="&lt;p&gt;I recently encountered an interesting problem and was wondering how RL would solve it.  The objective of the problem is to maximize the coffee quality, given by box X.  The coffee quality objective function is defined by the company.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To maximize the quality of the coffee, we can perform 2 actions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;change stirring speed of the coffee machine&lt;/li&gt;&#xA;&lt;li&gt;change the temperature of the coffee machine&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now to the tricky part.  The coffee bean characteristic coming into the coffee machine is random.  We can measure its characteristics before sending them to the coffee machine, but we &lt;strong&gt;cannot&lt;/strong&gt; change them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I formulated this problem into a control problem, such that &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a function of my previous states, &lt;span class=&quot;math-container&quot;&gt;$X(t - k)$&lt;/span&gt;, the control input , &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;, and the measured disturbance, &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. Given a constant &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, the problem is trival to solve because the disturbance is a constant part of the environment.  However, during times when &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; changes rapidly, the policy is no longer optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I inject the information of the measured disturbance into my RL agent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WTkQ9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WTkQ9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17706" LastEditorUserId="17706" LastEditDate="2018-12-15T07:50:21.437" LastActivityDate="2018-12-28T23:17:11.170" Title="How does reinforcement learning handle measured disturbances?" Tags="&lt;reinforcement-learning&gt;&lt;control-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9528" PostTypeId="2" ParentId="9526" CreationDate="2018-12-14T23:38:06.160" Score="4" Body="&lt;p&gt;If you check the Wikipedia article on the argument that you linked, in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;History&lt;/a&gt; section, you'll note the following statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most of the discussion consists of attempts to refute it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think this directly answers the question,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is the Chinese Room argument such a big deal?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Chinese Room Argument was a relatively early attempt in the intermingling of Philosophy and Computer Science to make as concrete an argument on the definition of a &quot;mind&quot;, &quot;understanding&quot;, and whether or not these can be created with a program. That is, it conjectures that a &quot;mind&quot; cannot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's famous in part because the concreteness of this argument means that a counter-argument can be made in kind. That is, the argument spawned a large quantity of refutations, attacking the argument from a large number of vectors, outlined later in the argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put another way,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Chinese Room Argument is such a big deal because of how many people in the field have found value in refuting it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Incidentally, your post provides more evidence to this point.&lt;/p&gt;&#xA;" OwnerUserId="16857" LastActivityDate="2018-12-14T23:38:06.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9529" PostTypeId="2" ParentId="9506" CreationDate="2018-12-15T00:52:59.227" Score="1" Body="&lt;p&gt;CRF is for HMM in sequences classification as Logistic Regression is for Naive Bayes for simple classification. For a in depth difference between them I strongly suggest you to read the classical Sutton article &lt;a href=&quot;https://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;An Introduction to Conditional&#xA;Random Fields&quot;&lt;/a&gt;. For a more practical example explaining how to apply the math I suggest you to read this &lt;a href=&quot;https://towardsdatascience.com/conditional-random-field-tutorial-in-pytorch-ca0d04499463&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9794" LastActivityDate="2018-12-15T00:52:59.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9532" PostTypeId="2" ParentId="7147" CreationDate="2018-12-15T09:55:01.773" Score="2" Body="&lt;p&gt;The main point here is that you can write &lt;span class=&quot;math-container&quot;&gt;$Q(s, a|\theta) = R = \theta^\top \phi(s, a)$&lt;/span&gt;. For more details on this, you can read up on Policy Gradients (Chapter 13) from the 2nd edition of the Sutton and Barto book (in fact the expression you're looking for is equation 13.9)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, I'm setting &lt;span class=&quot;math-container&quot;&gt;$\beta=1$&lt;/span&gt;, but you can always put it in once you get the idea. Therefore, the expression for &lt;span class=&quot;math-container&quot;&gt;$\pi(s,a,\theta)$&lt;/span&gt; (I'm including &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; here to make the dependence of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; explicit) is now:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi(s,a,\theta) = \frac{exp(\theta^\top \phi(s, a))}{\sum_{a'} exp(\theta^\top \phi(s, a'))} = \frac{A}{B}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it is standard to compute &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta\log \pi(s,a,\theta)$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta \pi(s,a,\theta)$&lt;/span&gt; (although you totally can), so I'll go ahead and do that. To differentiate &lt;span class=&quot;math-container&quot;&gt;$\log \pi(s,a,\theta)$&lt;/span&gt; with respect to the parameters, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, let's do some calculus:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\theta \log \pi(s,a,\theta) = \nabla_\theta \log \frac{A}{B} = \nabla_\theta \log A - \nabla_\theta \log B \\&#xA;= \nabla_\theta \left[ \theta^\top \phi(s,a) \right] - \nabla_\theta \left[ \log \sum_{a'} exp(\theta^\top \phi(s, a')) \right] \\&#xA;= \phi(s,a) - \frac{1}{\sum_{a'} exp(\theta^\top \phi(s, a'))} \sum_{b} exp(\theta^\top \phi(s, b)) \phi(s,b) \\&#xA;= \phi(s,a) - \sum_{b} \frac{exp(\theta^\top \phi(s, b))}{\sum_{a'} exp(\theta^\top \phi(s, a'))} \phi(s,b) \\&#xA;= \phi(s,a) - \sum_{b} \pi(s,b,\theta) \phi(s,b)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, you can write &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta\log \pi(s,a,\theta) = \phi(s,a) - \sum_{b} \pi(s,b,\theta) \phi(s,b)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20714" LastActivityDate="2018-12-15T09:55:01.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9533" PostTypeId="1" CreationDate="2018-12-15T10:22:50.983" Score="1" ViewCount="94" Body="&lt;p&gt;I'm currently implementing an Android app for street sign recognition. My solution works quite well for the GTSRB dataset, since it provides a labeled test set of centered images. However, it doesn't scale up to more realistic scenarios like for images in the GTSDB, where the signs only take up some pixels. Is it still recommended to downsample the image to 224x224?&lt;/p&gt;&#xA;" OwnerUserId="19177" LastActivityDate="2019-01-14T12:02:25.483" Title="Recognition of small objects" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9534" PostTypeId="1" CreationDate="2018-12-15T10:44:50.110" Score="2" ViewCount="154" Body="&lt;p&gt;I found several papers about how to build a perceptron able to solve the XOR problem. The papers describe a solution where the heaviside step function is replaced by a non-monotonic activation function. Here are the papers:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://hsmazumdar.net/single_layer_neural_net.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single Layer Neural Network Solution for XOR Problem&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/227309651_Solving_the_XOR_and_parity_Nproblems_using_a_single_univers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solving the XOR and parity N problems using a single universal binary neuron&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I also found &lt;a href=&quot;https://stackoverflow.com/q/30412427/3924118&quot;&gt;this related post&lt;/a&gt; on Stackoverflow.&lt;/p&gt;&#xA;&lt;p&gt;Can we really solve the XOR problem with a simple perceptron?&lt;/p&gt;&#xA;&lt;p&gt;The use of a non-monotonic activation function is not really common, so I don't really know. Papers about this idea are scarce. Generally, the main solution is to build a multilayer perceptron.&lt;/p&gt;&#xA;" OwnerUserId="20716" LastEditorUserId="2444" LastEditDate="2021-01-19T02:41:56.697" LastActivityDate="2021-01-19T02:41:56.697" Title="If we use a perceptron with a non-monotonic activation function, can it solve the XOR problem?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;perceptron&gt;&lt;xor-problem&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9536" PostTypeId="2" ParentId="9533" CreationDate="2018-12-15T11:39:55.887" Score="1" Body="&lt;p&gt;There is a big difference between the 2 datasets. When you train on the GTSRB dataset, your model performs &lt;strong&gt;image recognition&lt;/strong&gt;, while for the GTSDB dataset you need to perform &lt;strong&gt;object detection&lt;/strong&gt;. These are 2 related but different tasks (with detection being considerably harder).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can consider using a sliding window approach, or use some prior attention model that will learn to extract the area of the signs for later classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that it is recommended to down sample images of 1360x800 pixels to 224x224.&#xA;(224x224)/(1360x800)=0.046. You are loosing 96% of the information (including your street signs which might reduced to nothing), granted most of it is not interesting, but simple down-sampling doesn't get to choose what to forfeit.&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2018-12-15T11:39:55.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9538" PostTypeId="2" ParentId="9417" CreationDate="2018-12-15T18:14:13.280" Score="4" Body="&lt;p&gt;It can be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The activation function of a neuron does not have to be monotonic. The activation that Rahul suggested can be implemented via a continuously differentiable function, for example &lt;span class=&quot;math-container&quot;&gt;$ f(s) = exp(-k(1-s)^2) $&lt;/span&gt; which has a nice derivative &lt;span class=&quot;math-container&quot;&gt;$f'(s) = 2k~(1-s)f(s)$&lt;/span&gt;. Here, &lt;span class=&quot;math-container&quot;&gt;$s=w_0~x_0+w_1~x_1$&lt;/span&gt;. Therefore, standard gradient-based learning algorithms are applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neuron's error is &lt;span class=&quot;math-container&quot;&gt;$ E = \frac{1}{2}(v-v_d)^2$&lt;/span&gt;,&#xA;where &lt;span class=&quot;math-container&quot;&gt;$v_d$&lt;/span&gt; - desired output, &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; - actual output. The weights &lt;span class=&quot;math-container&quot;&gt;$w_i, ~i=0,1$&lt;/span&gt; are initialized randomly and then updated during training as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$w_i \to w_i - \alpha\frac{\partial E}{\partial w_i}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a learning rate. We have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial E}{\partial w_i} = (v-v_d)\frac{\partial v}{\partial w_i}=(f(s)-v_d)~\frac{\partial f}{\partial s}\frac{\partial s}{\partial w_i}=2k~(f(s)-v_d)(1-s)f(s)~x_i$$&lt;/span&gt;&#xA;Let's test it in Python.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np &#xA;import matplotlib.pyplot as plt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For training, I take a few points randomly scattered around &lt;span class=&quot;math-container&quot;&gt;$[0, 0]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$[1, 0]$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$[1, 1]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 10&#xA;sd = [0.05, 0.05]&#xA;&#xA;x00 = np.random.normal(loc=[0, 0], scale=sd, size=(n,2))&#xA;x01 = np.random.normal(loc=[0, 1], scale=sd, size=(n,2))&#xA;x10 = np.random.normal(loc=[1, 0], scale=sd, size=(n,2))&#xA;x11 = np.random.normal(loc=[1, 1], scale=sd, size=(n,2))&#xA;&#xA;x = np.vstack((x00,x01,x10,x11))&#xA;y = np.vstack((np.zeros((x00.shape[0],1)), &#xA;               np.ones((x01.shape[0],1)), &#xA;               np.ones((x10.shape[0],1)), &#xA;               np.zeros((x11.shape[0],1)))).ravel()&#xA;&#xA;ind = np.arange(len(y))&#xA;np.random.shuffle(ind)&#xA;&#xA;x = x[ind]&#xA;y = y[ind]&#xA;N = len(y)&#xA;&#xA;plt.scatter(*x00.T, label='00')&#xA;plt.scatter(*x01.T, label='01')&#xA;plt.scatter(*x10.T, label='10')&#xA;plt.scatter(*x11.T, label='11')&#xA;plt.legend()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W3r4F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W3r4F.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Activation function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = 10&#xA;&#xA;def f(s):&#xA;    return np.exp(-k*(s-1)**2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Initialize the weights, and train the network:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = np.random.uniform(low=0.25, high=1.75, size=(2))&#xA;&#xA;print(&quot;Initial w:&quot;, w)&#xA;&#xA;rate = 0.01&#xA;n_epochs = 20&#xA;&#xA;error = []&#xA;for _ in range(n_epochs):&#xA;    err = 0&#xA;    for i in range(N):&#xA;        s = np.dot(x[i],w)&#xA;        w -= rate * 2 * k * (f(s) - y[i]) * (1-s) * f(s) * x[i]&#xA;        err += 0.5*(f(s) - y[i])**2&#xA;    err /= N&#xA;    error.append(err)&#xA;&#xA;print('Final w:', w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The weights have indeed converged to &lt;span class=&quot;math-container&quot;&gt;$w_0=1,~w_1=1$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Initial w: [1.5915165  0.27594833]&#xA;Final w: [1.03561356 0.96695205]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The training error is decreasing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plt.scatter(np.arange(n_epochs), error)&#xA;plt.grid()&#xA;plt.xticks(np.arange(0, n_epochs, step=1))&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Vy119.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vy119.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's test it. I create a testing set in the same way as the training set. My test data are different from my training data because I didn't fix the seed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x00 = np.random.normal(loc=[0, 0], scale=sd, size=(n,2))&#xA;x01 = np.random.normal(loc=[0, 1], scale=sd, size=(n,2))&#xA;x10 = np.random.normal(loc=[1, 0], scale=sd, size=(n,2))&#xA;x11 = np.random.normal(loc=[1, 1], scale=sd, size=(n,2))&#xA;&#xA;x_test = np.vstack((x00,x01,x10,x11))&#xA;y_test = np.vstack((np.zeros((x00.shape[0],1)), &#xA;               np.ones((x01.shape[0],1)), &#xA;               np.ones((x10.shape[0],1)), &#xA;               np.zeros((x11.shape[0],1)))).ravel()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I calculate the root mean squared error, and the coefficient of determination (R^2 score):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fwd(x,w):&#xA;    return f(np.dot(x,w))&#xA;&#xA;RMSE = 0&#xA;&#xA;for i in range(N):&#xA;    RMSE += (fwd(x_test[i],w) - y_test[i])**2&#xA;&#xA;RMSE = np.sqrt(RMSE/N)&#xA;&#xA;print(&quot;RMSE&quot;, RMSE)&#xA;&#xA;ybar = np.mean(y)&#xA;&#xA;S = 0&#xA;D = 0&#xA;for i in range(N):&#xA;    S += (fwd(x_test[i],w) - y_test[i])**2&#xA;    D += (fwd(x_test[i],w) - ybar)**2&#xA;&#xA;r2_score = 1 - S/D&#xA;print(&quot;r2_score&quot;, r2_score)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RMSE 0.09199468888373698&#xA;r2_score 0.9613632278609362&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... or I am doing something wrong? Please tell me.&lt;/p&gt;&#xA;" OwnerUserId="15524" LastActivityDate="2018-12-15T18:14:13.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9540" PostTypeId="2" ParentId="9417" CreationDate="2018-12-15T20:29:43.417" Score="0" Body="&lt;p&gt;I'm also working on a perceptron that is able to solve the XOR Problem, and I get interesting results using sine as an activation function, and using the derivative to make the perceptron learn. But you will need bias to make sure the perceptron is able to solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="20716" LastActivityDate="2018-12-15T20:29:43.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9541" PostTypeId="1" CreationDate="2018-12-15T21:31:53.780" Score="0" ViewCount="172" Body="&lt;p&gt;I'm completely new at ML, but really interested. To be honest, read many articles about it, but still don't understand the workings of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just started to understand this example: &lt;a href=&quot;https://storage.googleapis.com/tfjs-examples/mnist/dist/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://storage.googleapis.com/tfjs-examples/mnist/dist/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thinking about it is that TF has some resources, some examples of how numbers look like, and try to match them with the ones in the test. I saw that sometimes the test changes a right prediction to a wrong, but makes better and better predictions. But how? I think that the program doesn't know the right predictions (and this way it won't know the wrong ones). In the training how it makes better predictions? Test by test, from what exceptions it will change it's predictions? What happens in a new test?&lt;/p&gt;&#xA;" OwnerUserId="20563" LastActivityDate="2018-12-20T19:41:49.517" Title="How TensorFlow will know if the prediction is true or false?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9545" PostTypeId="1" CreationDate="2018-12-16T08:06:37.283" Score="1" ViewCount="125" Body="&lt;p&gt;I have multiple pictures that look exactly like the one below this text. I'm trying to train &lt;code&gt;CNN&lt;/code&gt; to read the digits for me. Problem is isolating the digits. They could be written in any shape, way, and position that person who is writing them wanted to. I thought of maybe training another &lt;code&gt;CNN&lt;/code&gt; to recognize the position/location of the digits, but I'm not sure how to approach the problem. But, I need to get rid of that string and underline. Any clue would be a great one. Btw. I would love to get the 28x28 format just like the one in &lt;code&gt;MNIST&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks up front.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wOMma.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wOMma.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18676" LastEditorUserId="18676" LastEditDate="2018-12-16T22:37:25.953" LastActivityDate="2019-06-07T12:04:25.590" Title="How to approach this handwritten digit recognition?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;handwritten-characters&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9546" PostTypeId="2" ParentId="7696" CreationDate="2018-12-16T09:54:40.310" Score="1" Body="&lt;p&gt;The characteristics of the game are these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Moves are in two dimensions&lt;/li&gt;&#xA;&lt;li&gt;Only three possible first moves because of symmetry (corner, edge, middle)&lt;/li&gt;&#xA;&lt;li&gt;Never more than seven possible moves after that&lt;/li&gt;&#xA;&lt;li&gt;No random game play element (such as a roll of dice)&lt;/li&gt;&#xA;&lt;li&gt;No secrets (such as a hand of cards)&lt;/li&gt;&#xA;&lt;li&gt;No way to lose with perfect play&lt;/li&gt;&#xA;&lt;li&gt;Only one two-part rule: (1) Must make your mark (2) in an unmarked cell&lt;/li&gt;&#xA;&lt;li&gt;Only one winning condition: Three in a row of your mark&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The goal strategy is to get two marks such that a third will require two blocks in one opponent move. A three layer network will be able to learn that strategy. Because the game complexity is low, a mutation is always going to have a radical effect on game play. This means that the changes due to the mutation must be gradual and must stop when the strategy is found. This implies that stateful learning is best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is not related to human versus self play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there is exactly one mark to make per turn, soft max doesn't make much sense for the output layer. You want binary outputs &amp;mdash; threshold activation functions in the last layer.  Use two pair of binary output cells, one pair for each dimension.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-16T09:54:40.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9548" PostTypeId="2" ParentId="9417" CreationDate="2018-12-16T11:34:00.113" Score="0" Body="&lt;p&gt;Another activation function that could be used for this problem:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = \underset{i}{max}(x_i) - \underset{i}{min}(x_i)$$&lt;/span&gt;&#xA;It's not continuous, no backpropagation, sorry. Some other learning algorithm is required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this answers the question, if an XOR can be solved with one neuron. Maybe this function is a solution of some learning problem with weights. Something like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = max(w_0x_0,w_1x_1) - min(w_0x_0,w_1x_1)$$&lt;/span&gt;&#xA;I don't know how this creature is generalizable to other tasks, and how much learning can be done by just manipulating maxima and minima of weighted inputs. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="15524" LastActivityDate="2018-12-16T11:34:00.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9549" PostTypeId="2" ParentId="7696" CreationDate="2018-12-16T12:01:44.443" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What I notice is that the network's fitness keeps climbing up and falling down again. It seems that my current approach only evolves certain patterns on placing signs on the board and once random mutation interrupts current pattern new one emerges. My network goes in circles without ever evolving actual strategy. I suspect solution for this would be to pit network against tic-tac-toe AI, but is there any way to evolve actual strategy just by making it to playing against itself?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The likely cause of this phenomenon is that your fitness function involves evaluating the fitness of an agent by letting it play lots of other agents (the entire population), many of which are likely very poor agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because Tic-Tac-Toe is such a simple game, we know that optimal play from both sides leads to a draw. Suppose we have a population of the following three strategies:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt;: an optimal player&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt;: a sub-optimal player&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi_3$&lt;/span&gt;: a different sub-optimal player&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There can easily be situations where the optimal player &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; gets a draw against both of the sub-optimal players (if they're not &lt;strong&gt;extremely&lt;/strong&gt; bad), because right from the get-go an optimal player will play &quot;safe&quot; enough such that it can still guarantee a draw against an optimal opponent, which may not be the fastest way to win against a sub-optimal player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same situation, the sub-optimal player &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; for example may be able to consistently win against a slightly worse sub-optimal player &lt;span class=&quot;math-container&quot;&gt;$\pi_3$&lt;/span&gt;. In this example situation, your fitness function ranks the agents &lt;span class=&quot;math-container&quot;&gt;$f(\pi_2) &amp;gt; f(\pi_1) &amp;gt; f(\pi_3)$&lt;/span&gt;, which is wrong.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As you already suggested yourself, the most straightforward way to address this problem would be to simply evaluate the fitness of strategies by having them play against an optimal minimax-agent, rather than a population of many strategies (including poor ones).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to use only evolving, no tree search, you'll have to find a way to &lt;strong&gt;fix the fitness function such that the problem described above can no longer occur&lt;/strong&gt;. One way you could try to do that (&lt;em&gt;not 100% sure it would work, but imagine that it might&lt;/em&gt;) would be to set up some larger tournament bracket where agents progress through the brackets if they're able to beat others which they were paired up with. Getting further in the tournament would then increase the fitness of an agent. Very bad sub-optimal players should not be able to progress far in the tournament if they get beaten by other (sub-)optimal agents, but sub-optimal agents which manage to advance still shouldn't be able to get more than a draw against an optimal agent. Some things to keep in mind with this idea:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You'll likely want to &lt;strong&gt;repeat the entire tournament many times with different, randomized initial pairings in the bracket&lt;/strong&gt;, and compute fitness based on average (or maybe median or max) ranking across all repetitions of the tournament. This would be to filter out members of the population getting particularly lucky or unlucky with the opponents they encounter in the brackets.&lt;/li&gt;&#xA;&lt;li&gt;You'll have to think about which actions the agents select in such a tournament. Do they deterministically play the action given by the &lt;span class=&quot;math-container&quot;&gt;$\arg\max$&lt;/span&gt; of the softmax outputs (in which case you could also just use the linear outputs rather than softmax outputs, since the softmax function does not change ranking of outputs). Or do they nondeterministically sample actions according to the softmax distribution? Sampling actions according to the softmax distribution seems attractive because it leads to more variance in game state situations encountered, and it is important to be robust and be capable of handling many different game states. On the other hand, it does introduce noise and may make an optimal agent accidentally lose. &lt;strong&gt;I suppose I would lead towards deterministic play with the &lt;span class=&quot;math-container&quot;&gt;$\arg\max$&lt;/span&gt; over the softmax outputs&lt;/strong&gt;. In a large tournament, there will still be sufficient variety due to encountering many different opponents.&lt;/li&gt;&#xA;&lt;li&gt;You'll have to put thought into handling the situation where two agents may infinitely keep drawing against each other. Which agent advances? If this happens, I think I would gradually shift from deterministcally playing &lt;span class=&quot;math-container&quot;&gt;$\arg\max$&lt;/span&gt; actions to playing according to the softmax distribution. This guarantees that someone eventually loses. Averaging results over multiple different games like this, plus averaging over multiple different repetitions of the complete tournament, should yield accurate results.&lt;/li&gt;&#xA;&lt;li&gt;You'll want to think about how to handle &lt;strong&gt;illegal actions&lt;/strong&gt;. You describe including punishments for illegal actions in the fitness function. This introduces extra noise / variance in the fitness function which is already difficult enough to get right to begin with, so I wouldn't do this. I'd recommend having a &lt;strong&gt;manual post-processing step in which you manually set probabilities of illegal actions being played to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;&lt;/strong&gt;, and normalizing to make sure the probabilities over the remaining actions add up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; again. This is also what's done (on a much larger scale) by DeepMind in their state-of-the-art Go/Chess/Shogi agents for example. If you really don't want to do this, if you really want evolved strategies to automatically learn not to ever have high outputs for illegal actions, I would recommend immediately making them lose the game if they do suggest an illegal action. That way you still have a &quot;clean&quot; fitness function based only on wins/draws/losses.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-16T12:01:44.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9550" PostTypeId="1" AcceptedAnswerId="13955" CreationDate="2018-12-16T12:33:59.517" Score="6" ViewCount="723" Body="&lt;p&gt;I've been reading up on how NEAT (Neuro Evolution of Augmenting Topologies) works and I've got the main idea of it, but one thing that's been bothering me is how you split the different networks into species. I've gone through the algorithm but it doesn't make a lot of sense to me and the paper I read doesn't explain it very well either so if someone could give an explanation of what each component is and what it's doing then that would be great thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\delta = \frac{c_{1}E}{N} + \frac{c_{2}D}{N} + c_{3} .\overline{W}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f_{i}^{'} = \frac{f_i}{\sum_{j=1}^{n}sh(\delta(i,j))}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, I can understand the greek symbols so you don't need to explain those to me&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The original paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20736" LastEditorUserId="15792" LastEditDate="2019-08-14T19:30:53.307" LastActivityDate="2019-08-14T19:30:53.307" Title="How does the NEAT speciation algorithm work?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9551" PostTypeId="2" ParentId="9541" CreationDate="2018-12-16T12:56:40.157" Score="1" Body="&lt;p&gt;During the training process, the machine learning program is shown example inputs (in this case pictures of digits) and also &lt;em&gt;what the answer should be for that input&lt;/em&gt;. That is how it learns the correct answer.&#xA;However, it does not learn an exact mapping between the input data and required output, instead it learns a series of patterns that generally map the input to the required value. The advantage of this approach is that it can perform the same mapping on new inputs that were not included in the training data but are similar (but not necessarily identical) to examples that it has already seen. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="12509" LastEditDate="2018-12-20T19:41:49.517" LastActivityDate="2018-12-20T19:41:49.517" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9553" PostTypeId="2" ParentId="9521" CreationDate="2018-12-16T17:44:59.837" Score="1" Body="&lt;p&gt;Keras/Tensorflow are mostly used of developing/training/deploying neural networks. For descision making problems, if you want to use machine learning, reinforcement learning is in most cases applied. Some reinforcement learning methods use neural networks (and therfore tensorflow) internally. You can check baseline implementations of different methods &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using reinfocement learnig, for most of the reinforcement learning methods, you do not need and preexisting data, but you need and environment (esentially a simulations of your game). If you interface your game to one of the baseline agents you can start the agents playing ant training ocurs during they play. You can find a tutorial to start an agent &lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-with-openai-d445c2c687d2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only have a feedback (a reward) after a full game of 20 decisions, it will be really hard to train the agents, as you are facing a good example for what is called the problem of spare reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some ways to deal with sparse reward, like &lt;a href=&quot;https://arxiv.org/abs/1707.01495&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience&lt;/a&gt; Replay and maybe &lt;a href=&quot;https://towardsdatascience.com/curiosity-driven-learning-made-easy-part-i-d3e5a2263359&quot; rel=&quot;nofollow noreferrer&quot;&gt;Curiosity&lt;/a&gt;, but improving the density of the reward would be very beneficial.&lt;/p&gt;&#xA;" OwnerUserId="2585" LastActivityDate="2018-12-16T17:44:59.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9554" PostTypeId="2" ParentId="6426" CreationDate="2018-12-16T19:51:52.680" Score="14" Body="&lt;p&gt;There is always a lot of confusion about this concept, because &lt;em&gt;the naming is misleading, given that &lt;strong&gt;both tree and graph searches produce a tree&lt;/strong&gt; (from which you can derive a path) while exploring the search space, which is usually represented as a graph&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Differences&lt;/h3&gt;&#xA;&lt;p&gt;Firstly, we have to understand that the underlying problem (or search space) is almost always represented as a graph (although the underlying graph may not contain cycles, so it may represent a tree). So, the difference is not whether the problem is a tree (a special kind of graph), or a general graph!&lt;/p&gt;&#xA;&lt;p&gt;The distinction is, instead, how we are traversing the search space (represented as a graph) to search for our goal state and whether we are using an additional list (called the &lt;em&gt;closed list&lt;/em&gt;) or not.&lt;/p&gt;&#xA;&lt;p&gt;So, the basic &lt;strong&gt;differences&lt;/strong&gt; are&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of a graph search, we use a list, called the &lt;strong&gt;closed list&lt;/strong&gt; (also called &lt;strong&gt;explored set&lt;/strong&gt;), to keep track of the nodes that have already been visited and expanded, so that they are not visited and expanded again.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of a tree search, we do &lt;strong&gt;not&lt;/strong&gt; keep this closed list. Consequently, the same node can be visited multiple (or even infinitely many) times, which means that the produced tree (by the tree search) may contain the same node multiple times.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h3&gt;Advantages and disadvantages&lt;/h3&gt;&#xA;&lt;p&gt;The &lt;strong&gt;advantage&lt;/strong&gt; of graph search obviously is that, if we finish the search of a node, we will never search it again. On the other hand, the tree search can visit the same node multiple times.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;disadvantage&lt;/strong&gt; of graph search is that it uses more memory (which we may or may not have) than tree search. This matters because graph search actually has &lt;em&gt;exponential&lt;/em&gt; memory requirements in the worst case, making it impractical without either a really good search heuristic or an extremely simple problem.&lt;/p&gt;&#xA;&lt;p&gt;So, there is a &lt;strong&gt;trade-off&lt;/strong&gt; between space and time when using graph search as opposed to tree search (or vice-versa).&lt;/p&gt;&#xA;&lt;h3&gt;Conclusion&lt;/h3&gt;&#xA;&lt;p&gt;So, the difference between tree search and graph search is &lt;strong&gt;not&lt;/strong&gt; that tree search works on trees while graph search works on graphs! Both can work on trees or graphs (but, given that graphs are a generalization of trees, we can simply say that both work on graphs, either trees or not) and both produce a tree!&lt;/p&gt;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&lt;p&gt;The definitions of tree search and graph search given above are based on the definitions given in section 3.3 (page 77) of the book &lt;a href=&quot;https://faculty.psau.edu.sa/filedownload/doc-7-pdf-a154ffbcec538a4161a406abf62f5b76-original.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt; (3rd edition)&lt;/a&gt;, by Stuart J. Russell and Peter Norvig, which is the de-facto standard book in artificial intelligence, so these definitions are applicable in the context of artificial intelligence (which is also the context of this site). &lt;a href=&quot;https://github.com/aimacode/aima-pseudocode/blob/master/md/Tree-Search-and-Graph-Search.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you have the pseudocode of tree and graph searches (provided by P. Norvig).&lt;/p&gt;&#xA;&lt;p&gt;However, note that, occasionally, people may use the term &lt;em&gt;tree search&lt;/em&gt; to refer to a &lt;em&gt;tree traversal&lt;/em&gt;, which is used to refer to a search in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Search_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;search tree&lt;/a&gt; (e.g., a binary search tree or a red-black tree), which is a tree (i.e. a graph without cycles) that maintains a certain order of its elements. This is another reason for having different definitions of a tree search and to think that a tree search works only on trees.&lt;/p&gt;&#xA;" OwnerUserId="20745" LastEditorUserId="2444" LastEditDate="2021-05-15T12:42:48.387" LastActivityDate="2021-05-15T12:42:48.387" CommentCount="3" CommunityOwnedDate="2020-08-11T16:54:23.377" ContentLicense="CC BY-SA 4.0" />
  <row Id="9555" PostTypeId="2" ParentId="7413" CreationDate="2018-12-16T20:24:41.020" Score="1" Body="&lt;p&gt;Straight from Sutton's book (&lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://incompleteideas.net/book/the-book-2nd.html&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 3.4: Pole-Balancing&lt;/strong&gt;&#xA;The objective in this task is to apply&#xA;forces to a cart moving along a track&#xA;so as to keep a pole hinged to the cart&#xA;from falling over: A failure is said to&#xA;occur if the pole falls past a given angle&#xA;from vertical or if the cart runs off the&#xA;track. The pole is reset to vertical&#xA;after each failure. This task could be&#xA;treated as &lt;strong&gt;episodic&lt;/strong&gt;, where the natural&#xA;episodes are the repeated attempts to balance the pole. &lt;strong&gt;The reward, in this case, could be&#xA;+1 for every time step on which failure did not occur, so that the return at each time&#xA;would be the number of steps until failure.&lt;/strong&gt; In this case, successful balancing forever would&#xA;mean a return of infinity. Alternatively, we could treat pole-balancing as a &lt;strong&gt;continuing&lt;/strong&gt;&#xA;task, using discounting. In this case, &lt;strong&gt;the reward would be -1 on each failure and zero at&#xA;all other times.&lt;/strong&gt; The return at each time would then be related to K, where K is the&#xA;number of time steps before failure. In either case, the return is maximized by keeping&#xA;the pole balanced for as long as possible.&lt;/p&gt;&#xA;" OwnerUserId="20743" LastActivityDate="2018-12-16T20:24:41.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9557" PostTypeId="1" CreationDate="2018-12-16T21:30:00.607" Score="1" ViewCount="42" Body="&lt;p&gt;I have heard about bidirectional RNN LSTM units (endcoders-decoders), but my question is - is there bidirectional neural machine translation, that uses A-&gt;B weights for the translation in the opposite direction B-&gt;A? If not, then what are the obstacles to such system?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2018-12-29T02:57:04.063" Title="Is there bidirection sequence-to-sequence neural machine translation?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9561" PostTypeId="1" CreationDate="2018-12-17T08:48:34.260" Score="2" ViewCount="85" Body="&lt;p&gt;I was reading this paper &lt;a href=&quot;https://arxiv.org/abs/1610.04325&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hadamard Product for Low-rank Bilinear Pooling&lt;/a&gt;. I understand what they are trying to say, but I don't know why we have to convert the element-wise multiplication into a scalar (using the dot product)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{1}^{T}\left(\mathbf{U}_{i}^{T} \mathbf{x} \circ \mathbf{V}_{i}^{T} \mathbf{y}\right)+b_{i} \tag{2}\label{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why do we have to multiply the resulting vector by the one vector? We would still use the multiplicative interaction between elements if we did not consider multiplying by that one vector.&lt;/p&gt;&#xA;" OwnerUserId="20755" LastEditorUserId="2444" LastEditDate="2021-01-06T23:36:25.333" LastActivityDate="2021-01-06T23:38:29.313" Title="Why do we have to dot product in the Low-rank Bilinear Pooling?" Tags="&lt;deep-learning&gt;&lt;papers&gt;&lt;pooling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9563" PostTypeId="1" AcceptedAnswerId="9566" CreationDate="2018-12-17T10:52:00.850" Score="3" ViewCount="1724" Body="&lt;p&gt;If I got well the idea of dropout, it allows improving the sparsity of the information that comes from one layer to another by setting some weights to zero.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, pooling, let's say max-pooling, takes the maximum value in a neighborhood, reducing as well to zero, the influence of values apart from this maximum.&lt;/p&gt;&#xA;&lt;p&gt;Without considering the shape transformation due to the pooling layer, can we say that pooling is a kind of a dropout step?&lt;/p&gt;&#xA;&lt;p&gt;Would the addition of a dropout (or DropConnect) layer, after a pooling layer, make sense in a CNN? And does it help the training process and generalization property?&lt;/p&gt;&#xA;" OwnerUserId="11069" LastEditorUserId="2444" LastEditDate="2021-03-16T10:41:21.140" LastActivityDate="2021-03-16T10:41:21.140" Title="Is pooling a kind of dropout?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;dropout&gt;&lt;generalization&gt;&lt;pooling&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9565" PostTypeId="2" ParentId="9545" CreationDate="2018-12-17T11:38:49.787" Score="1" Body="&lt;p&gt;I think one approach you can try to segment the digits and Connected Components Labeling (&lt;a href=&quot;https://en.wikipedia.org/wiki/Connected-component_labeling&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Connected-component_labeling&lt;/a&gt;).&#xA;With it, you'll end up with a label for each letter and then you can try to find the convex hull of the letter.&#xA;After that, just crop a square for each convex hull and input it to your CNN.&#xA;Notice that it will only if there is at least one pixel between the letters...&lt;/p&gt;&#xA;" OwnerUserId="20759" LastActivityDate="2018-12-17T11:38:49.787" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9566" PostTypeId="2" ParentId="9563" CreationDate="2018-12-17T12:43:42.323" Score="6" Body="&lt;p&gt;Dropout and Max-pooling are performed for different reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dropout is a regularization technique, which affects only the training process (during evaluation, it is not active). The goal of dropout is reduce unnecessary feature dependencies in the network, allowing it to be simpler and improves its generalization abilities (reduces overfitting). In simple terms, it helps the model to learn that some features are an &quot;OR&quot; and not an &quot;AND&quot; requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Max-pooling is not a regularization technique and it is part of the model's architecture, so it is also used during evaluation. The goal of max-pooling is to down-sample an input representation. As a result the model becomes less sensitive to some translations (improving translation invariance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your last question, yes. dropout can be used after a pooling layer.&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2018-12-17T12:43:42.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9567" PostTypeId="2" ParentId="9561" CreationDate="2018-12-17T13:03:38.610" Score="2" Body="&lt;p&gt;The paper (which I was not familiar with) proposes a new pooling method, which they name &amp;quot;Low-rank Bi-linear Pooling&amp;quot;. Because it is a pooling method, its output should be a single value over some input spatial region (the goal of pooling is to down-sample an input representation). They do not want to use the resulting vector from the element-wise multiplication, they want to get a single scalar. Notice their initial formula:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_{i}=\sum_{j=1}^{N} \sum_{k=1}^{M} w_{i j k} x_{j} y_{k}+b_{i}=\mathbf{x}^{T} \mathbf{W}_{i} \mathbf{y}+b_{i}\label{1}\tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20399" LastEditorUserId="2444" LastEditDate="2021-01-06T23:38:29.313" LastActivityDate="2021-01-06T23:38:29.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9568" PostTypeId="2" ParentId="7996" CreationDate="2018-12-17T13:29:24.360" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S4RuC.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S4RuC.gif&quot; alt=&quot;Below is the answer considering your variables:&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, &lt;span class=&quot;math-container&quot;&gt;$p_{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_{y}$&lt;/span&gt; are padding values. (equal on both sides).&#xA;You can have different padding on left and right side. (similarly top and bottom). If same padding, equation will have &lt;span class=&quot;math-container&quot;&gt;$2*p_{x}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$2*p_{y}$&lt;/span&gt;, else you can just add values of both padding and replace in the equations. (For example &lt;span class=&quot;math-container&quot;&gt;$p_{xLeft} + p_{xRight}$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Padding is optional. Generally thumb rule is add padding such that the equations above produce integers and also add padding such that if stride is 1, output is same as input and if stride is 2, output is half of input. &lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="20760" LastEditDate="2018-12-17T13:38:37.943" LastActivityDate="2018-12-17T13:38:37.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9569" PostTypeId="1" AcceptedAnswerId="9600" CreationDate="2018-12-17T14:20:50.983" Score="1" ViewCount="228" Body="&lt;p&gt;When reading the book by Sutton and Barto, I came across the &lt;em&gt;importance sampling ratio&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first equation, I believe, describes the probability a particular sequence is obtained given the current state, and the policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\operatorname{Pr}\left\{A_{t}, S_{t+1}, A_{t+1}, \ldots, S_{T} | S_{t}, A_{t: T-1} \sim \pi\right\} \\&#xA;&amp;amp;=\pi\left(A_{t} | S_{t}\right) p\left(S_{t+1} | S_{t}, A_{t}\right) \pi\left(A_{t+1} | S_{t+1}\right) \cdots p\left(S_{T} | S_{T-1}, A_{T-1}\right) \\&#xA;&amp;amp;=\prod_{k=t}^{T-1} \pi\left(A_{k} | S_{k}\right) p\left(S_{k+1} | S_{k}, A_{k}\right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next part takes the ratio between the probabilities of the two trajectories:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{t: T-1} \doteq \frac{\prod_{k=t}^{T-1} \pi\left(A_{k} | S_{k}\right) p\left(S_{k+1} | S_{k}, A_{k}\right)}{\prod_{k=t}^{T-1} b\left(A_{k} | S_{k}\right) p\left(S_{k+1} | S_{k}, A_{k}\right)}=\prod_{k=t}^{T-1} \frac{\pi\left(A_{k} | S_{k}\right)}{b\left(A_{k} | S_{k}\right)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how this ratio could lead to this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}\left[\rho_{t: T-1} G_{t} | S_{t}=s\right]=v_{\pi}(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; rewards are obtained through the &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; policy, not the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there is something to do with Bayes rule, but I could not derive it. Could someone guide me through the derivation?&lt;/p&gt;&#xA;" OwnerUserId="20762" LastEditorUserId="2444" LastEditDate="2020-05-18T22:18:54.313" LastActivityDate="2020-06-13T21:11:25.243" Title="In the context of importance sampling ratio, how is the equation $\mathbb{E}\left[\rho_{t: T-1} G_{t} | S_{t}=s\right]=v_{\pi}(s)$ derived?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;markov-decision-process&gt;&lt;importance-sampling&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9571" PostTypeId="1" AcceptedAnswerId="9575" CreationDate="2018-12-17T17:57:43.167" Score="2" ViewCount="47" Body="&lt;p&gt;I came across this article today: &lt;a href=&quot;https://www.theverge.com/2018/12/17/18144356/ai-image-generation-fake-faces-people-nvidia-generative-adversarial-networks-gans&quot; rel=&quot;nofollow noreferrer&quot;&gt;These faces show how far AI image generation has advanced in just four years&lt;/a&gt;. I would never in a million years have guessed that the people on the right (in the first image in the article) were fakes! &lt;img src=&quot;https://i.stack.imgur.com/cgBjW.png&quot; alt=&quot;[AI generated images (2014 on the left and 2018 on the right)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will it be possible to create videos of such AI generated images? What, then, will become of actors and actresses?&lt;/p&gt;&#xA;" OwnerUserId="17601" LastEditorUserId="1671" LastEditDate="2018-12-18T01:27:55.453" LastActivityDate="2018-12-18T01:27:55.453" Title="Creating videos of AI generated photographs" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9575" PostTypeId="2" ParentId="9571" CreationDate="2018-12-18T00:16:05.890" Score="1" Body="&lt;p&gt;I wouldn't really consider what they are doing as AI - they are using a script that intelligently overlaps various images of existing people in order to create a new face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Animating those images isn't impossible - essentially extrapolation + additional &quot;real&quot; images will be used to know what the face would like from all angles and in all states (happy, sad, intrigued, etc...).&lt;/p&gt;&#xA;" OwnerUserId="20775" LastActivityDate="2018-12-18T00:16:05.890" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9576" PostTypeId="2" ParentId="8281" CreationDate="2018-12-18T01:32:39.647" Score="1" Body="&lt;p&gt;Hinge loss is difficult to work with when the derivative is needed because the derivative will be a piece-wise function. &lt;code&gt;max&lt;/code&gt; has one non-differentiable point in its solution, and thus the derivative has the same. This was a very prominent issue with non-separable cases of SVM (and a good reason to use ridge regression).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a slide (Original source from &lt;a href=&quot;https://sites.google.com/site/ucsdfall2018cogs118a/calendar-notes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zhuowen Tu&lt;/a&gt;, apologies for the title typo):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hoaGW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hoaGW.png&quot; alt=&quot;SVM Hinge Loss equation and derivative&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where hinge loss is defined as &lt;code&gt;max(0, 1-v)&lt;/code&gt; and &lt;code&gt;v&lt;/code&gt; is the decision boundary of the SVM classifier. More can be found on &lt;a href=&quot;https://en.wikipedia.org/wiki/Hinge_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Hinge Loss Wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your equation: you can easily pick out the &lt;code&gt;v&lt;/code&gt; of the equation, however without more context of those functions it's hard to say how to derive. Unfortunately I don't have access to the paper and cannot guide you any further...&lt;/p&gt;&#xA;" OwnerUserId="20777" LastActivityDate="2018-12-18T01:32:39.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9578" PostTypeId="1" AcceptedAnswerId="9648" CreationDate="2018-12-18T07:59:36.253" Score="0" ViewCount="373" Body="&lt;p&gt;I'm learning AI, but this confuses me. The derivative function used in backpropagation is the &lt;strong&gt;derivative of activation function&lt;/strong&gt; or the &lt;strong&gt;derivative of loss function&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These terms are confusing: derivative of act. function, partial derivative wrt. loss function??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still not getting it correct.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2018-12-22T14:45:28.253" Title="What is the derivative function used in backpropagration?" Tags="&lt;backpropagation&gt;&lt;activation-function&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9580" PostTypeId="1" CreationDate="2018-12-18T11:27:55.680" Score="2" ViewCount="94" Body="&lt;p&gt;How would one go about predicting which characters (actors) are going to die e.g. in the next Avengers movie?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To elaborate a bit, given all leaked scripts (fake or not), interviews of different actors and directors, contracts with different actors and directors. How would/should one go about predicting if an actor is going to die in the upcoming movie or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I am not sure, but I think a similar effort has already been made for Game of Thrones. &lt;/p&gt;&#xA;" OwnerUserId="19722" LastEditorUserId="2444" LastEditDate="2020-04-01T14:39:41.960" LastActivityDate="2020-04-01T14:39:41.960" Title="How should I predict which characters are going to die in a certain movie?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9584" PostTypeId="1" AcceptedAnswerId="9586" CreationDate="2018-12-18T14:29:49.637" Score="2" ViewCount="389" Body="&lt;p&gt;I'm using RL to train a Network on the game Connect4. It learns quickly that 4 connected pieces is good. It gets a reward of 1 for this. A zero is rewarded for all other moves.&lt;/p&gt;&#xA;&lt;p&gt;It takes quite a time until the AI tries to stop the opponent from winning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is there a way this could be further reinforced?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I thought about giving a negative reward for the move played before the winning move. Thinking about this, I came to the conclusion that this is a bad idea. There'll be always a looser (except for ties), therefore there always be a last move from the losing player. This one hasn't to be a bad one. Mistakes could have been made much earlier.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is there a way to improve this awareness of opponents? Or does it just have to train more?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not perfectly sure if the rewards will propagate back in a way that encourages this behavior with my setup.&lt;/p&gt;&#xA;" OwnerUserId="13336" LastEditorUserId="2444" LastEditDate="2020-10-07T22:39:38.937" LastActivityDate="2020-10-07T22:39:38.937" Title="How should I define the reward function in the case of Connect Four?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;rewards&gt;&lt;reward-design&gt;&lt;connect-four&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9586" PostTypeId="2" ParentId="9584" CreationDate="2018-12-18T15:44:08.923" Score="2" Body="&lt;p&gt;The classic working reward scheme for two player zero sum games (i.e. if I win, you lose and vice versa) is simply:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;+1 for a win&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;0 for a draw&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;-1 for a loss&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These rewards should be associated with the last move made by the player before the game is resolved.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I thought about giving a negativ reward for the move played before the winning move. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is what is normally done.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thinking about this I came to the conclusion that this is a bad idea. There'll be always a looser (except for ties), therefor there always be a last move from the loosing player. This one hasn't to be a bad one. Mistakes could have been made much earlier.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's OK, what the agent is learning is the &lt;em&gt;value&lt;/em&gt; of a combination of state and action. It will, eventually, correctly associate the negative reward with a chain of board states that decrease in value and that it will then try to avoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This scenario of delayed rewards, not necessarily related to the choice of the current action, is handled correctly by reinforcement learning algorithms. It is still a tough problem though - a more delayed and difficult to predict reward makes for a tougher RL problem. However, even basic learners, such as Q-learning, can eventually solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get better performance, you might want to look into adding planning algorithms to the RL part.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-12-18T15:44:08.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9587" PostTypeId="2" ParentId="9541" CreationDate="2018-12-18T16:09:12.683" Score="1" Body="&lt;p&gt;I'm not quite sure what you mean in your post however I will try and address some of the points you made. Firstly, I would suggest you to go over some supervised machine learning fundamentals (there are lots of resources online).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general optimisation approach is to find the parameters of a model that minimise the error in your training set (using the ground truth labels). The model may not be complex enough to achieve absolutely zero error on all the training samples, and if it did it would likely generalise poorly to new data. We don't want a model to &quot;memorise&quot; the labels, for example, picking up contextual cues in a training image that are not relevant to the label (such as the background of the picture of a car).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An ML model aims to reduce the error over all the samples and this is why the correct classification of a specific input can change as the model learns.&lt;/p&gt;&#xA;" OwnerUserId="8790" LastActivityDate="2018-12-18T16:09:12.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9588" PostTypeId="1" CreationDate="2018-12-18T16:34:16.317" Score="3" ViewCount="83" Body="&lt;p&gt;I've been given an assignment to create a neural network that will suggest a Croatian word for a word given in any other European language (out of those found &lt;a href=&quot;https://www.indifferentlanguages.com/words/beer&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). The words are limited to drinks you can find on a bar menu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked at many NN examples, both simple and complex, but I'm having trouble with understanding how to normalize the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, words &quot;beer&quot;, &quot;birra&quot; and &quot;cervexa&quot; should all translate to &quot;pivo&quot;. If I include those 3 in the training set, and after the network has finished training I input the word &quot;bier&quot;, the output should be &quot;pivo&quot; again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not looking for a working solution to this problem, I just need a nudge in the right direction regarding normalization.&lt;/p&gt;&#xA;" OwnerUserId="20787" LastActivityDate="2019-09-24T00:02:40.390" Title="Translating a single word Neural Networks" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9590" PostTypeId="1" AcceptedAnswerId="9653" CreationDate="2018-12-18T17:02:51.690" Score="1" ViewCount="592" Body="&lt;p&gt;I am learning Reinforcement Learning from the lectures from David Silver. I finished lecture 6 and went on to try SARSA with linear function approximator for MountainCar-v0 environment from OpenAI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A brief explanation of the MountainCar-v0 environment. The state is denoted by two features, position, and velocity. There are three actions for each state, accelerate forwards, don't accelerate, accelerate backward. The goal of the agent is to learn how to climb a mountain. The engine of the car is not strong enough to power directly to the top. So speed has to be built up by oscillating in the cliff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used a linear function approximator, written by myself. I am attaching my code here for reference :-&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    class LinearFunctionApproximator:&#xA;      ''' A function approximator must have the following methods:-&#xA;          constructor with num_states and num_actions&#xA;          get_q_value&#xA;          get_action&#xA;          fit '''&#xA;&#xA;      def __init__(self, num_states, num_actions):&#xA;        self.weights = np.zeros((num_states, num_actions))&#xA;        self.num_states = num_states&#xA;        self.num_actions = num_actions&#xA;&#xA;      def get_q_value(self, state, action):&#xA;        return np.dot( np.transpose(self.weights), np.asarray(state) )[action]&#xA;&#xA;      def get_action(self, state, eps):&#xA;        return randint(0, self.num_actions-1) if uniform(0, 1) &amp;lt; eps else np.argmax( np.dot(np.transpose(self.weights), np.asarray(state)) )&#xA;&#xA;      def fit(self, transitions, eps, gamma, learning_rate):&#xA;        ''' Every transition in transitions should be of type (state, action, reward, next_state) '''&#xA;        gradient = np.zeros_like(self.weights)&#xA;        for (state, action, reward, next_state) in transitions:&#xA;          next_action = self.get_action(next_state, eps)&#xA;          g_target = reward + gamma * self.get_q_value(next_state, next_action)&#xA;          g_predicted = self.get_q_value(state, action)&#xA;          gradient[:, action] += learning_rate * (g_target - g_predicted) * np.asarray(state)&#xA;&#xA;        gradient /= len(transitions)&#xA;        self.weights += gradient&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tested the gradient descent, and it works as expected. After every epoch, the mean squared error between current estimate of Q and TD-target reduces as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code for SARSA :-&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def SARSA(env, function_approximator, num_episodes=1000, eps=0.1, gamma=0.95, learning_rate=0.1, logging=False):&#xA;&#xA;  for episode in range(num_episodes):&#xA;    transitions = []&#xA;&#xA;    state = env.reset()&#xA;    done = False&#xA;&#xA;    while not done:&#xA;      action = function_approximator.get_action(state, eps)&#xA;      next_state, reward, done, info = env.step(action)&#xA;      transitions.append( (state, action, reward, next_state) )&#xA;      state = next_state&#xA;&#xA;    for i in range(10):&#xA;      function_approximator.fit(transitions[::-1], eps, gamma, learning_rate)&#xA;&#xA;    if logging:&#xA;      print('Episode', episode, ':', end=' ')&#xA;      run_episode(env, function_approximator, eps, render=False, logging=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, for every episode, I fit the linear function approximator to the current TD-target. I have also tried running fit just once per episode, but that also does not yield any winning episode. Fitting 10 times ensures that I am actually making some progress towards the TD-target, and also not overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after running over 5000 episodes, I do not get a single episode where the reward is greater than -200. Eventually, the algorithm choses one action, and somehow the Q-value of other actions is always lesser than this action.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Now, let's see how the trained model does&#xA;env = gym.make('MountainCar-v0')&#xA;num_states = 2&#xA;num_actions = env.action_space.n&#xA;&#xA;function_approximator = LinearFunctionApproximator(num_states, num_actions)&#xA;&#xA;num_episodes = 2000&#xA;eps = 0&#xA;SARSA(env, function_approximator, num_episodes=num_episodes, eps=eps, logging=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to be more clear about this. Say action 2 is the one which is the action which gets selected always after say 1000 episodes. Action 0 and action 1 have somehow, for all states, have their Q-values reduced to a level which is never reached by action 2. So for a particular state, action 0 and action 1 may have Q-values of -69 and -69.2. The Q-value of action 2 will never drop below -65, even after running the 5000 episodes.&lt;/p&gt;&#xA;" OwnerUserId="20424" LastActivityDate="2018-12-27T00:32:30.690" Title="SARSA won't work for linear function approximator for MountainCar-v0 in OpenAI environment. What are the possible causes?" Tags="&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9591" PostTypeId="2" ParentId="9588" CreationDate="2018-12-18T20:25:56.540" Score="0" Body="&lt;p&gt;So, the &lt;code&gt;normalization&lt;/code&gt; as you call it is just encoding the word into some vector of fixed length. After you encode it, you should be able to decode it and match the words. You should google a little, for a &lt;code&gt;seq2seq&lt;/code&gt; problems and &lt;code&gt;Encoder/Decoder&lt;/code&gt; structure. There are encoder-decoder frameworks out there and lots of resources. You don't care about a language so just pretend that words like &quot;beer&quot; and &quot;birra&quot; have the same meaning in Croatian and in this case, it is &quot;pivo&quot;.&lt;/p&gt;&#xA;" OwnerUserId="18676" LastActivityDate="2018-12-18T20:25:56.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9592" PostTypeId="1" AcceptedAnswerId="9593" CreationDate="2018-12-18T23:47:02.990" Score="2" ViewCount="201" Body="&lt;p&gt;Once a book is published in a language, why can't the publishers use Google Translate AI or some similar software to immediately render the book in other languages? Likewise for Wikipedia: I'm not sure I understand why we need editors for each language. Can't the English Wikipedia be automatically translated into other languages?&lt;/p&gt;&#xA;" OwnerUserId="17601" LastEditorUserId="2444" LastEditDate="2021-01-18T12:25:44.077" LastActivityDate="2021-01-18T12:25:44.077" Title="Why can't we use Google Translate for every translation task?" Tags="&lt;neural-networks&gt;&lt;machine-translation&gt;&lt;google-translate&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9593" PostTypeId="2" ParentId="9592" CreationDate="2018-12-19T00:30:37.867" Score="4" Body="&lt;p&gt;Google has achieved significant progress in AI translation, but it's still no-where near a qualified human translator. Natural language translation is already very challenging, adding domain knowledge to the equation is too much even for Google.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think we have the technology to translate an arbitrary book from one language to another reliably.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2018-12-19T00:30:37.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9594" PostTypeId="2" ParentId="9592" CreationDate="2018-12-19T02:23:39.937" Score="2" Body="&lt;p&gt;This is an good question and I have to wonder if someday we might.  It may simply be a matter of formalizing all of the concepts conveyed by humans, which is &lt;a href=&quot;https://en.wikipedia.org/wiki/Emergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;emergent&lt;/a&gt;, but has to be finite.  &lt;em&gt;The present algorithms do not understand the content&lt;/em&gt; in a human sense of meaning, but are refining a statistical model to continually produce more accurate output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could do it today, and it actually has been done I am sure.  There was a period back in the aughts of machine translated movie subtitles that were serviceable, but so bad as to be comical. (Humor that can be found in bad random translation is highly underrated;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A key difficulty is linguistic.  The same word or phrase (or symbol) can mean many things, [see: &lt;a href=&quot;https://en.wikipedia.org/wiki/Polysemy&quot; rel=&quot;nofollow noreferrer&quot;&gt;polysemy]&lt;/a&gt; so often the sense in which you mean it is required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another part of this is that words and symbols and &lt;a href=&quot;https://en.wikipedia.org/wiki/Idiom&quot; rel=&quot;nofollow noreferrer&quot;&gt;idioms&lt;/a&gt; may not have a direct counterpart in another language.  If the literal meaning is different from the figurative meaning, but the algorithm only knows how to translate the literal meaning, it's a bad translation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, try translating &quot;&lt;a href=&quot;https://en.oxforddictionaries.com/explore/what-is-the-origin-of-the-phrase-hair-of-the-dog/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hair of the dog&lt;/a&gt;&quot; into &lt;a href=&quot;https://translate.google.com/#view=home&amp;amp;op=translate&amp;amp;sl=en&amp;amp;tl=zh-CN&amp;amp;text=hair%20of%20the%20dog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese and back to English on Google translate&lt;/a&gt;.  The result is &quot;dog hair&quot; which is not the intended meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm can learn that there is a figurative usage, but still has to know when to apply it as opposed to the literal.  Without understanding the meaning of a phrase and the phrases leading to and from it (the &lt;a href=&quot;https://en.oxforddictionaries.com/definition/context&quot; rel=&quot;nofollow noreferrer&quot;&gt;context&lt;/a&gt;), the algorithm must rely on statistics.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it may be a function of &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;time&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_resource&quot; rel=&quot;nofollow noreferrer&quot;&gt;memory&lt;/a&gt; &lt;a href=&quot;https://ai.stackexchange.com/questions/2642/how-can-one-distinguish-between-an-ai-and-a-sufficiently-advanced-algorithm&quot;&gt;algorithms sufficiently complex&lt;/a&gt;, and a sufficient &lt;a href=&quot;https://en.wikipedia.org/wiki/Sample_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;sample size&lt;/a&gt;. But, because language is continually evolving, there will always be instances in which the sample size is insufficient (new concepts or usage, new words.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My sense is that the present algorithms will become very good at translating in certain contexts, especially where the meaning is literal (asking for directions, asking the price, customary social interactions, etc.) but will never reach fully human capability without the capability to grasp the meaning. &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantics&lt;/a&gt; vs. &lt;a href=&quot;https://en.oxforddictionaries.com/definition/syntactics&quot; rel=&quot;nofollow noreferrer&quot;&gt;syntactics&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether algorithms will ever be able to understand in the sense of humans is still very theoretical and a &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;subject of fierce debate&lt;/a&gt;, but until they do, human translators will likely be preferred.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2018-12-19T02:23:39.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9595" PostTypeId="1" AcceptedAnswerId="9598" CreationDate="2018-12-19T02:38:37.567" Score="5" ViewCount="1184" Body="&lt;p&gt;In short, imitation learning means learning from the experts. Suppose I have a dataset with labels based on the actions of experts. I use a simple binary classifier algorithm to assess whether it is good expert action or bad expert action.&lt;/p&gt;&#xA;&lt;p&gt;How is this binary classification different from imitation learning?&lt;/p&gt;&#xA;&lt;p&gt;Imitation learning is associated with reinforcement learning, but, in this case, it looks more like a basic classification problem to me.&lt;/p&gt;&#xA;&lt;p&gt;What is the difference between imitation learning and classification done by experts?&lt;/p&gt;&#xA;&lt;p&gt;I am getting confused because imitation learning relates to reinforcement learning while classification relates to supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="16839" LastEditorUserId="2444" LastEditDate="2020-11-05T22:59:00.700" LastActivityDate="2020-11-05T22:59:00.700" Title="What is the difference between imitation learning and classification done by experts?" Tags="&lt;reinforcement-learning&gt;&lt;classification&gt;&lt;comparison&gt;&lt;supervised-learning&gt;&lt;imitation-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9597" PostTypeId="1" CreationDate="2018-12-19T06:15:01.797" Score="1" ViewCount="31" Body="&lt;p&gt;In &quot;&lt;code&gt;4.1 Learning multi-layer deconvolutional filters&lt;/code&gt;&quot; section, the last paragraph says that &quot;Since our model is generative, we can sample from it. In Fig. 3 we show samples from the two different models from each level projected down into pixel space. The samples were drawn using the relative firing frequencies of each feature from the training set.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/owCeC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/owCeC.png&quot; alt=&quot;Fig.3 and the explanition&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how the pictures in Fig.3 are generative. Since that the filters has been learned and feature maps in every layer can be infered, for example, in terms of fruit samples, in &quot;Layer 1&quot;, is the the first layer's feature map ? i feel that not true...seems like the sample are low-level...paper says &quot;... from each level projected into pixel space&quot;, these words are short and confuse me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;somebody could explain that for me? thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="20797" LastActivityDate="2018-12-19T06:15:01.797" Title="a question about Zeiler's paper “Deconvolutional Networks”" Tags="&lt;deep-learning&gt;&lt;unsupervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9598" PostTypeId="2" ParentId="9595" CreationDate="2018-12-19T07:20:51.500" Score="4" Body="&lt;p&gt;Imitation learning &lt;em&gt;is&lt;/em&gt; supervised learning applied to the RL setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a scenario where you have available the optimal policy in the form of a table, mapping each state to each action. In this scenario you will not care about the rewards received - rather, you'd simply do a table lookup to decide the optimal action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This scenario is impractical in most settings because the table for the optimal policy will be too big. However, if you have enough entries from the table, you can use a general function approximator such as a neural network to find the optimal action. Again, you do not need to look at the rewards, but only at the state &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; action mappings. I do not know imitation learning in detail beyond this, but I suspect in the case of discrete actions (such as in Chess, Go), it would be trained with a cross-entropy objective as is typical of classification tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the reason this has a different name in the RL setting is because this is different than how the conventional RL algorithms work. Also, much of RL thinking is inspired from everyday interaction / biology. Think of how we learn how to drive, or play sports such as soccer. Typically there is a coach who tells you what actions to take under different conditions, and you imitate those actions. &lt;/p&gt;&#xA;" OwnerUserId="20799" LastActivityDate="2018-12-19T07:20:51.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9600" PostTypeId="2" ParentId="9569" CreationDate="2018-12-19T10:31:25.260" Score="0" Body="&lt;p&gt;It has nothing to do with Bayes rule, like you said, &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; is the return we get by following the policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; , so the value of the state we get that return in is equal to the expected value of that return. On page 59 of the book, equation 3.14 derives the value of the state as expected value of the return but in this case we are following the behaviour policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; so you would replace &lt;span class=&quot;math-container&quot;&gt;$\pi (a \mid s)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$b (a \mid s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_\pi (s')$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$v_b (s')$&lt;/span&gt; and you would get the value of the state following the policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, that is &lt;span class=&quot;math-container&quot;&gt;$v_b (s)$&lt;/span&gt;.&lt;br&gt;&#xA;We are actually interested in the value of the state when we follow target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; , but we got the value by following behaviour policy, so we will multiply that expression with importance sampling ratio. If we do that &lt;span class=&quot;math-container&quot;&gt;$b (a \mid s)$&lt;/span&gt; terms will cancel out and you are only left with term &lt;span class=&quot;math-container&quot;&gt;$\pi (a \mid s)$&lt;/span&gt; from importance sampling ratio so we will get the value by following the target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; that is &lt;span class=&quot;math-container&quot;&gt;$v_\pi (s)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2018-12-19T10:31:25.260" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9601" PostTypeId="1" CreationDate="2018-12-19T13:26:46.660" Score="1" ViewCount="65" Body="&lt;p&gt;So, I'm using a pretrained pnasnet5large model to do some image classification (&lt;a href=&quot;https://github.com/Cadene/pretrained-models.pytorch/blob/master/pretrainedmodels/models/pnasnet.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Cadene/pretrained-models.pytorch/blob/master/pretrainedmodels/models/pnasnet.py&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the file, it says that the input range is in [0,1] (i'm assuming pixel values of input images). The images I have are already in this range.&#xA;The channel means and standard deviation for RGB channels are stated as [0.5, 0.5, 0.5], [0.5, 0.5, 0.5] respectively.&#xA;Now when I use the torchvision.transforms.Normalize(&lt;a href=&quot;https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Normalize&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Normalize&lt;/a&gt;) to normalize the images using the stated means and standard deviations, the pixel values get to the range [-1,1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I wrote for normalization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;transforms.Normalize([0.5, 0.5, 0.5],[0.5, 0.5, 0.5])&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I'm missing something fundamental. Should I normalize the images or should I not? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="20807" LastActivityDate="2021-05-07T20:06:00.840" Title="Relationship between input range and channel means, standard deviations for CNNs" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9602" PostTypeId="1" AcceptedAnswerId="9670" CreationDate="2018-12-19T14:15:52.613" Score="3" ViewCount="109" Body="&lt;p&gt;Consider an image classification problem. Conceptually, we then have some high dimensional space where all the images can be represented as points, and having large enough labeled data set we can build a classifier. But how do we know that our data in this space has some structure? Like this one in two-dimensional case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jxpNO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jxpNO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have a data set with images of, say, cats and dogs, why these two classes are not just uniformly mixed with each other but have some distribution or shape in appropriate space? Why it cannot be like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kHA5W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kHA5W.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20809" LastEditorUserId="2444" LastEditDate="2019-10-22T21:08:05.987" LastActivityDate="2019-10-22T21:08:05.987" Title="How do we know the classification boundaries of the data?" Tags="&lt;classification&gt;&lt;image-recognition&gt;&lt;pattern-recognition&gt;&lt;clustering&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9604" PostTypeId="2" ParentId="9580" CreationDate="2018-12-19T14:50:18.213" Score="1" Body="&lt;p&gt;This is more of a probability question. Having said that, based on each leaked script, you generate the possibilities of each of the characters dying (for example yes 0.2%, no 0.7%, unknown 0.1%) - you also give a weight for each leaked script based on on its perceived authenticity (from 0.1 to 1). You multiply the weight of each script by each of the possibilities and then you add the result for each possibility and each character across all scripts. So, for example, say you have 2 scripts. The first script has perceived authenticity of 20% (0.2) and the second script has perceived authenticity of 40% (0.4). In the first script, Character A has 50% (0.5) chance of dying, 20% (0.2) of not dying, and 30% (0.3) chance of unknown outcome. In the second script, character A has 20% (0.2) chance of dying, 50% (0.5) chance of not dying, and 30% (0.3) chance of unknown outcome. So the following applies for Character A:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dying: (0.2 x 0.5) + (0.4 x 0.2) = 0.18 (18% chance of dying)&lt;/li&gt;&#xA;&lt;li&gt;Not dying: (0.2 x 0.2) + (0.4 x 0.5) = 0.24 (24% chance of not dying)&lt;/li&gt;&#xA;&lt;li&gt;Unknown outcome: (0.2 x 0.3) + (0.4 x 0.3) = 0.20 (20% unknown outcome)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The probability of not dying is 24%, which is the highest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am not saying that probability has nothing to do with AI (in fact, it is very important in AI).&lt;/p&gt;&#xA;" OwnerUserId="20775" LastActivityDate="2018-12-19T14:50:18.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9607" PostTypeId="2" ParentId="9601" CreationDate="2018-12-19T16:55:16.100" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;Standardization of the pixel values would bring it mean close to 0 and standard deviation as 1.&lt;/li&gt;&#xA;&lt;li&gt;Normalization will squeeze the values between 0 and 1.&#xA;For RGB images, divide each channel i.e each pixel value by 255.&#xA;For reference regarding Standardization see &lt;a href=&quot;https://medium.com/predict/standardization-on-crazy-data-python-cd5b1282a97f&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and for normalization see &lt;a href=&quot;https://medium.com/@equipintelligence/normalization-of-crazy-data-python-4fa6611e7b46&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-12-19T16:55:16.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9609" PostTypeId="2" ParentId="4655" CreationDate="2018-12-19T20:37:15.307" Score="5" Body="&lt;p&gt;I have implemented Permutational Layer here using Keras: &lt;a href=&quot;https://github.com/off99555/superkeras/blob/master/permutational_layer.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/off99555/superkeras/blob/master/permutational_layer.py&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can call the &lt;code&gt;PermutationalModule&lt;/code&gt; function to use it.&lt;/p&gt;&#xA;&lt;p&gt;Implemented following this paper: &lt;a href=&quot;https://arxiv.org/pdf/1612.04530.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Permutation-equivariant neural networks applied to dynamics prediction&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The idea is to compare all pairs of &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt; pairs from &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; inputs, use the model with shared weights, then use pooling function &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; times on &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; inputs.&lt;/p&gt;&#xA;&lt;p&gt;The output you can use pooling again but in the paper, they don't mention another pooling.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastEditorUserId="2444" LastEditDate="2020-11-06T02:16:32.100" LastActivityDate="2020-11-06T02:16:32.100" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9610" PostTypeId="1" CreationDate="2018-12-19T20:58:02.760" Score="1" ViewCount="68" Body="&lt;p&gt;I am interested in optimizing the memory capacity of an AGI. Given a specific complex input an AI can create a simplified model. This is a problem that can be solved using &lt;em&gt;sparse coding&lt;/em&gt; &lt;sup&gt;[1]&lt;/sup&gt;. However, this solves only the problem of encoding and not the maintenance of online representations—in cognitive terms: the state of mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A default cognitive model of short-term memory can be separated in three different stages:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;Encoding → Maintenance → Retrieval&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One solution is to use specialized hardware &lt;sup&gt;[2][3]&lt;/sup&gt;, but I am interested in software approaches to this problem and I would thus like to emphasize that it is the digital representation, which I am most interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the exception of qubits, the smallest possible representation are binary digits. However, additional architecture is required to represent phase spaces (i.e. floating point precision memory) and higher-order representations maybe include arrays and dictionaries. (Optimizing these are trivial... or simply to be postponed until needed, according to Knuth).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can a specific &lt;em&gt;connectivity pattern&lt;/em&gt; be stored in an &lt;em&gt;optimally&#xA;compact&lt;/em&gt; representation?&lt;/li&gt;&#xA;&lt;li&gt;Is there an implementation with concrete example code?&lt;/li&gt;&#xA;&lt;li&gt;What is the state-of-the-art?&lt;sup&gt;*&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;* I will not specify &quot;real-time&quot; here, but the context is humanoid AGI. &lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt; &#xA;[1] Papyan, V., Romano, Y., &amp;amp; Elad, M. (2017). Convolutional Neural Networks Analyzed via Convolutional Sparse Coding. &lt;em&gt;Journal on Machine Learning Research, 18&lt;/em&gt;(83): 1–52. arXiv:1607.08194&#xA;&lt;a href=&quot;http://jmlr.org/papers/volume18/16-505/16-505.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jmlr.org/papers/volume18/16-505/16-505.pdf&lt;/a&gt;&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;[2] LeGallo et al. (2018). Mixed-precision in-memory computing. &lt;a href=&quot;https://www.nature.com/articles/s41928-018-0054-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41928-018-0054-8&lt;/a&gt;&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&#xA;[3] IBM. (2018). IBM Scientists Demonstrate Mixed-Precision In-Memory Computing for the First Time; Hybrid Design for AI Hardware. &lt;a href=&quot;https://www.ibm.com/blogs/research/2018/04/ibm-scientists-demonstrate-mixed-precision-in-memory-computing-for-the-first-time-hybrid-design-for-ai-hardware/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ibm.com/blogs/research/2018/04/ibm-scientists-demonstrate-mixed-precision-in-memory-computing-for-the-first-time-hybrid-design-for-ai-hardware/&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="16411" LastEditorUserId="2444" LastEditDate="2019-06-19T21:50:30.453" LastActivityDate="2019-06-19T21:50:30.453" Title="How can a specific connectivity pattern be stored in an optimally compact representation?" Tags="&lt;optimization&gt;&lt;agi&gt;&lt;memory&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9611" PostTypeId="2" ParentId="6916" CreationDate="2018-12-19T21:11:14.503" Score="-1" Body="&lt;p&gt;The most interesting scientific field in terms of relevance to this question is probably &quot;affective computing&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several problems with the model that you suggested. It is questionable if AGI should learn in the same way humans learn. In addition, there are several ethical problems surrounding this question, perhaps even metaethics, because the question peeks beyond human ethics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A hardcoded ethics protocol might be possible to implement, similar to Isaac Asimov's &quot;Three Laws of Robotics&quot;. The possibility of hacking is an issue that one would have to think proactively about - especially in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If humanoid AGIs or robots are very similar to humans, they should probably be treated that way. For example, ethiologist Frans de Waal has studied empathy and social behavior in monkeys and suggested that we are very similar in behavior, so we are probably similar in terms of feelings (if it quacks like a duck and so forth). Perhaps we need an ethiology for androids too?&lt;/p&gt;&#xA;" OwnerUserId="16411" LastActivityDate="2018-12-19T21:11:14.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9613" PostTypeId="1" AcceptedAnswerId="9620" CreationDate="2018-12-20T00:07:49.153" Score="1" ViewCount="83" Body="&lt;p&gt;I'm wondering if there exists a network for simple image classification. What I mean by this is if I have two image datasets, one of horses and one of zebras, I want to train off the horses and classify an image as either a horse or not a horse, so if I test it on an image of a horse, it says it is a horse, but if I use a zebra, it says it is not a horse. Does any library/project for this exist?&lt;/p&gt;&#xA;" OwnerUserId="20826" LastActivityDate="2018-12-20T12:58:38.170" Title="Python Network for Simple Image Classification" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9614" PostTypeId="1" AcceptedAnswerId="10369" CreationDate="2018-12-20T01:00:04.310" Score="4" ViewCount="1472" Body="&lt;p&gt;In the policy gradient method, there's a trick to reduce the variance of policy gradient. We use causality, and remove part of the sum over rewards so that only actions happened after the reward are taken into account (See here &lt;a href=&quot;http://rail.eecs.berkeley.edu/deeprlcourse/static/slides/lec-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rail.eecs.berkeley.edu/deeprlcourse/static/slides/lec-5.pdf&lt;/a&gt;, slide 18).&lt;/p&gt;&#xA;&lt;p&gt;Why does it work? I understand the intuitive explanation, but what's the rigorous proof of it? Can you point me to some papers?&lt;/p&gt;&#xA;" OwnerUserId="18074" LastEditorUserId="2444" LastEditDate="2020-10-10T15:52:04.057" LastActivityDate="2020-11-25T20:14:21.823" Title="Why does the &quot;reward to go&quot; trick in policy gradient methods work?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;policy-gradients&gt;&lt;rewards&gt;&lt;reward-to-go&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9620" PostTypeId="2" ParentId="9613" CreationDate="2018-12-20T12:49:30.803" Score="0" Body="&lt;p&gt;&lt;strong&gt;Using neural networks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If it is as simple as that&lt;/em&gt;, probably a neural network consisting of just a simple convolutional network with a few filters, a fully connected layer with 2 neurons, and a softmax layer, might do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To develop that, you can use Keras wrapper over tensorflow or theano, but I think scikit-learn also provides some basic neural network functionalities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the simplicity of the images on your dataset, you might do it even with no convolutional layer if there are not many non-linearities, or you might need more complexity (eg. adding convolutional and max pooling layers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made personally for teaching a workshop a very simple example of image classification, with 3 groups (trees, people, vehicles), after building a dataset collecting and selectioning images from the CIFAR-100 dataset.  You can see this ultra simple example &lt;a href=&quot;https://github.com/freesoul/Contenido_AI/tree/master/3_clasif_CNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. However, it is in spanish. Let me translate the most important piece of code from it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;&#xA;# CONVOLUTION. Final size: 29x29x32&#xA;model.add(Conv2D(   32,&#xA;                    (4, 4),&#xA;                    padding='same',&#xA;                    input_shape=data_x_train.shape[1:]&#xA;        ))&#xA;&#xA;model.add(Activation('relu'))&#xA;&#xA;&#xA;# CONVOLUTION. Final size: 26x26x32&#xA;model.add(Conv2D(   32,&#xA;                    (4, 4),&#xA;                    padding='same',&#xA;                    input_shape=data_x_train.shape[1:]&#xA;        ))&#xA;&#xA;model.add(Activation('relu'))&#xA;&#xA;&#xA;# MAX POOLING. Final size: 13x13x32&#xA;model.add(MaxPooling2D(pool_size=(2,2)))&#xA;&#xA;# Dropout to prevent overfitting&#xA;model.add(Dropout(0.25))&#xA;&#xA;&#xA;# CONVOLUTION. Final size: 10x10x64&#xA;model.add(Conv2D(   64,&#xA;                    (4, 4),&#xA;                    padding='same',&#xA;                    input_shape=data_x_train.shape[1:]&#xA;        ))&#xA;&#xA;model.add(Activation('relu'))&#xA;&#xA;# MAX POOLING. Final size: 5x5x64&#xA;model.add(MaxPooling2D(pool_size=(2,2)))&#xA;&#xA;# Dropout to prevent overfitting&#xA;model.add(Dropout(0.25))&#xA;&#xA;# Fully connected layer. Neuron number: 1600&#xA;model.add(Flatten())&#xA;&#xA;# Output layer: 3 neurons&#xA;model.add(Dense(3))&#xA;model.add(Activation('softmax'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the input for this model is an array of shape [num_rows, 32, 32]. 32x32 is the image size, and I assume 1 color channel. As I say, this is ultra simple. The output, some kind of probability for each of the 3 classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Using other aproaches&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are many other supervised learning models that you can use for that, most of them supported by scikit-learn, such as kNN, random forests decision trees, RBF SVM, or even just a kernelized multiclass logistic regression. &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;See here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6114" LastEditorUserId="6114" LastEditDate="2018-12-20T12:58:38.170" LastActivityDate="2018-12-20T12:58:38.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9621" PostTypeId="1" CreationDate="2018-12-20T15:14:50.537" Score="1" ViewCount="59" Body="&lt;p&gt;I have only a limited dataset (&amp;lt;25) with large-sized images (&gt;1500x2000) and their pixelwise labels. The aim is to find unusual patterns in this industry dataset and highlight them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To generate training images I crop 256x256 grids out of every image and do some data augmentation and use these images to train my U-Net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my prediction I split my image with numpy again into 256x256px grids and predict every grid separately and put them together to an image. But this will take some time, like &gt;10 Minutes. But it has a quite good accuracy. &lt;strong&gt;How can I optimize my prediction to be faster?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it faster to create this with a Tensorflow Pipeline? When I want to predict the full image with giving shape(None,None,3), I get &quot;ConcatOp : Dimensions of inputs should match&quot; after some time. &lt;/p&gt;&#xA;" OwnerUserId="20200" LastActivityDate="2018-12-20T15:14:50.537" Title="Image Segmentation Prediction with cropping 256x256 grids is very slow" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9624" PostTypeId="1" AcceptedAnswerId="9683" CreationDate="2018-12-20T17:30:40.910" Score="1" ViewCount="518" Body="&lt;p&gt;Problem: Fraud detection&#xA;Task : classifying transactions as either fraud/ Non-fraud using GAN &lt;/p&gt;&#xA;" OwnerUserId="20842" LastActivityDate="2019-01-25T01:01:45.347" Title="Can we implement GAN (Generative adversarial neural networks) for classication problem like Fraud detecion?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9626" PostTypeId="2" ParentId="9624" CreationDate="2018-12-20T21:07:17.450" Score="-1" Body="&lt;p&gt;Yes, although the traditional technique is to use Anomaly Detection for this sort of problem. The reason for that is that the space of possible frauds is very much larger than the space of legitimate transactions and so it is difficult to model the former. The usual idea is to thoroughly explore the smaller space of legitimate transactions and flag as fraudulent anything outside of that space.&#xA;Using the GAN method could be interesting as it might identify novel fraud techniques, and those are always useful to know. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="12509" LastEditDate="2018-12-24T20:19:41.980" LastActivityDate="2018-12-24T20:19:41.980" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9630" PostTypeId="2" ParentId="9152" CreationDate="2018-12-21T02:43:26.837" Score="2" Body="&lt;p&gt;Scrabble was solved using reinforcement learning achieving super-human performance. If you are interested, you can start reading about it here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Maven_(Scrabble)#cite_note-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Maven_(Scrabble)#cite_note-1&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also read about it here: &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/games.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/games.pdf&lt;/a&gt; (there're several slides about Maven)&lt;/p&gt;&#xA;" OwnerUserId="18074" LastActivityDate="2018-12-21T02:43:26.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9632" PostTypeId="2" ParentId="3108" CreationDate="2018-12-21T02:52:29.447" Score="2" Body="&lt;p&gt;I suggest you take a look at the syllabus of &lt;a href=&quot;http://cs224d.stanford.edu/syllabus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs224d.stanford.edu/syllabus.html&lt;/a&gt; and see what was invented in the last ~20 years. They focus on deep learning methods, and they include there almost everything important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the things, which I would include into the review are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Word2Vec models&lt;/li&gt;&#xA;&lt;li&gt;Seq2Seq models&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Concerning what happened in the last 5 years, i.e. since 2013, I suggest you mentioning super human performance on the SQuAD 1.1 dataset: &lt;a href=&quot;https://rajpurkar.github.io/SQuAD-explorer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rajpurkar.github.io/SQuAD-explorer/&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="18074" LastEditorUserId="18074" LastEditDate="2018-12-21T03:03:12.427" LastActivityDate="2018-12-21T03:03:12.427" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9633" PostTypeId="2" ParentId="5558" CreationDate="2018-12-21T02:54:54.387" Score="0" Body="&lt;p&gt;If you use fully connected neural network, it will be hard to do what you want. There's no inductive bias to easily generalize from different positions of the strings which you need to extract. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you use Seq2Seq models here: &lt;a href=&quot;http://cs224d.stanford.edu/papers/seq2seq.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs224d.stanford.edu/papers/seq2seq.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18074" LastEditorUserId="18074" LastEditDate="2018-12-21T03:02:45.353" LastActivityDate="2018-12-21T03:02:45.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9634" PostTypeId="2" ParentId="9520" CreationDate="2018-12-21T03:01:22.603" Score="0" Body="&lt;p&gt;They struggle because if your network have an inductive bias towards modeling datasets which are described with ODEs well, you will learn faster, and with smaller dataset. I think, this what the authors of the original article meant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar way, CNNs recognize images much better, because their features are translation invariant whereas fully connected net needs to learn to recognize a cat in each different position from scratch.&lt;/p&gt;&#xA;" OwnerUserId="18074" LastActivityDate="2018-12-21T03:01:22.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9635" PostTypeId="1" CreationDate="2018-12-21T07:28:43.780" Score="5" ViewCount="6373" Body="&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Semi-supervised_learning&quot; rel=&quot;noreferrer&quot;&gt;semi-supervised learning&lt;/a&gt;, there are &lt;em&gt;hard labels&lt;/em&gt; and &lt;em&gt;soft labels&lt;/em&gt;. Could someone show me what's exactly the meaning of the two things?&lt;/p&gt;&#xA;" OwnerUserId="20831" LastEditorUserId="2193" LastEditDate="2018-12-22T01:38:21.803" LastActivityDate="2018-12-22T01:38:21.803" Title="About the definition of &quot;soft label&quot; and &quot;hard label&quot;" Tags="&lt;definitions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9636" PostTypeId="2" ParentId="9635" CreationDate="2018-12-21T10:43:35.120" Score="5" Body="&lt;p&gt;According to &lt;a href=&quot;https://www.isi.edu/~galstyan/papers/ILP-07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Galstyan and Cohen (2007)&lt;/a&gt;, a &lt;em&gt;hard&lt;/em&gt; label is a label assigned to a member of a class where membership is binary: either the element in question is a member of the class (has the label), or it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;soft&lt;/em&gt; label is one which has a score (probability or likelihood) attached to it. So the element is a member of the class in question with probability/likelihood score of eg 0.7; this implies that an element can be a member of multiple classes (presumably with different membership scores), which is usually not possible with hard labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One use of soft labels in semi-supervised learning could be that the training set consists of hard labels; a classifier is trained on that using supervised learning. The classifier is then run on unlabelled data, and adds soft labels to the elements. This enlarged data set is then used for further training, where the algorithm can treat hard and soft labels differently.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2018-12-21T14:17:14.463" LastActivityDate="2018-12-21T14:17:14.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9637" PostTypeId="1" CreationDate="2018-12-21T12:05:24.880" Score="0" ViewCount="1476" Body="&lt;p&gt;I fine tuned &lt;a href=&quot;https://arxiv.org/pdf/1704.04861.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MobileNetSSD&lt;/a&gt; for object detection using a dataset with just one class (~4000 images). All the training images include at least one bounding box related to that class (no empty images). By following the &lt;a href=&quot;https://github.com/chuanqi305/MobileNet-SSD&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; with the VOC dataset, the labelmap includes two classes, the background and my custom class. However, as I mentioned, there are no annotations related to the background and I  am not sure if there should be any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my fine tuned network performs very well when objects belonging to my class are present, however there are some false detections with very high confidence when the class is not present. Can this be related to the fact that I don't have empty images in my training set?&lt;/p&gt;&#xA;" OwnerUserId="16671" LastEditorUserId="16671" LastEditDate="2018-12-24T09:50:47.907" LastActivityDate="2019-07-31T20:58:57.223" Title="When training an object detection network for one class, should I include empty images in the dataset?" Tags="&lt;datasets&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9638" PostTypeId="1" CreationDate="2018-12-21T12:21:13.527" Score="1" ViewCount="169" Body="&lt;p&gt;My dog goes bonkers every time the sound of a barking dog is heard on a television program.  I never noticed this before but literally every movie or show with an outdoors setting eventually includes the sound of a barking dog. Is it possible to develop a real-time filter that blocks or masks these sounds?&lt;/p&gt;&#xA;" OwnerUserId="20854" LastEditorUserId="2444" LastEditDate="2019-06-19T21:35:27.183" LastActivityDate="2019-06-19T21:35:27.183" Title="Can I filter barking sounds on the television?" Tags="&lt;ai-design&gt;&lt;theory&gt;&lt;real-time&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9639" PostTypeId="1" CreationDate="2018-12-21T13:15:41.980" Score="1" ViewCount="179" Body="&lt;p&gt;This question was asked in an AI exam. How would you answer such question?&lt;/p&gt;&#xA;" OwnerUserId="20856" LastEditorUserId="1671" LastEditDate="2018-12-22T01:37:30.807" LastActivityDate="2018-12-30T02:46:28.590" Title="Can Alpha–Beta be used on symmetric zero sum games?" Tags="&lt;game-ai&gt;&lt;game-theory&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9641" PostTypeId="1" CreationDate="2018-12-21T18:37:27.863" Score="2" ViewCount="90" Body="&lt;p&gt;I have a couple different segmentation tasks that I would like to perform on medical imaging data using CNN's. I'm currently trying to wrap my head around how well a 3D network might work, using a U-Net architecture, but I have some hesitation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My specific questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1 - Say we have medical imaging taken at different slices (different heights/depths) of the patient's body. In order for a 3D CNN to work properly on such data, do the images need to be taken at a heights that are close together, i.e. does the 3rd dimension need to be fairly continuous? For example, if you had a 3D stack/volume of 5 images that you wanted to feed to a 3D CNN like so,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;img_1_depth_0cm.png&lt;/li&gt;&#xA;&lt;li&gt;img_2_depth_5cm.png&lt;/li&gt;&#xA;&lt;li&gt;img_3_depth_10cm.png&lt;/li&gt;&#xA;&lt;li&gt;img_4_depth_15cm.png&lt;/li&gt;&#xA;&lt;li&gt;img_5_depth_20cm.png&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and the images were taken 5 cm apart from one and other, I would imagine that a 3D convolution might not perform very well because of the 5 cm of depth in between images? Is this an incorrect assumption?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(As a reference point, this nice repository on GitHub was designed for training on images of the brain, but the images do appear to be fairly continuous/close together, almost like a video: &lt;a href=&quot;https://github.com/ellisdg/3DUnetCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ellisdg/3DUnetCNN&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2 - For a 3D network to properly segment non-labelled volumes after training is finished, I know that the input data dimensions would have to match those of the training data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would also think that the new images must have been taken in a similar fashion (taken at similar depths and in general a similar orientation to the training data) in order for the NN to be able to perform its task. So unless the medical imaging processes are always performed similarly across machines and hospitals, I'm guessing that the NN performance might vary pretty wildly when it tries to segment new data. Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="20867" LastEditorUserId="20868" LastEditDate="2018-12-22T01:38:32.470" LastActivityDate="2019-09-18T04:04:30.537" Title="Appropriateness of 3D Convolutional Neural Network for segmentation of medical image data" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9642" PostTypeId="2" ParentId="9641" CreationDate="2018-12-21T20:02:00.713" Score="1" Body="&lt;p&gt;In CNN, different height/width are not a problem. You can choose one specific size for your model and pads zero around the slices to make them of equal size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No, your assumption is not correct. You are doing end to end training, so it does not matter, which sequence are you following.&lt;/p&gt;&#xA;" OwnerUserId="16313" LastActivityDate="2018-12-21T20:02:00.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9644" PostTypeId="2" ParentId="3965" CreationDate="2018-12-21T21:28:12.207" Score="1" Body="&lt;p&gt;Since you have all your data in a table, a relatively simple thing to do is to consider each column independently, and then seeing if the output variable (cost incurred) has a correlation to that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the column has no (or very low correlation) with the output variable, then consider it to be not important.  The ones that make the cut are then considered further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is obviously not very different from how a decision tree algorithm would work (such as ID3).&lt;/p&gt;&#xA;" OwnerUserId="20745" LastActivityDate="2018-12-21T21:28:12.207" CommentCount="0" CommunityOwnedDate="2018-12-21T21:28:12.207" ContentLicense="CC BY-SA 4.0" />
  <row Id="9645" PostTypeId="2" ParentId="9639" CreationDate="2018-12-22T01:52:02.467" Score="3" Body="&lt;p&gt;If the game is not &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequential_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequential&lt;/a&gt;, there would be no game tree and no need for pruning.  Alpha-beta is a technique applied to look-ahead search.  Alpha-beta has demonstrated utility in algorithms that play &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial games&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Even in iterated dilemmas, it doesn't really branch because it's simultaneous, more of a vine than a tree.  Decisionmaking would be based on mathematical analysis of the payoff matrix and statistical analysis of competitor behavior over time.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2018-12-22T01:57:52.967" LastActivityDate="2018-12-22T01:57:52.967" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9648" PostTypeId="2" ParentId="9578" CreationDate="2018-12-22T13:00:16.977" Score="1" Body="&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivatives of functions are used to determine what changes to input parameters correspond to what desired change in output for any given point in the forward propagation and cost, loss, or error evaluation &amp;amp;mdash whatever it is conceptually the learning process is attempting to minimize. This is the conceptual and algebraic inverse of maximizing valuation, yield, or accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back-propagation estimates the next best step toward the objective quantified in the cost function in a search. The result of the search is a set of parameter matrices, each element of which represents what is sometimes called a connection weight. The improvement of the values of the elements in the pursuit of minimal cost is artificial networking's basic approach to learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each step is an estimation because the cost function is a finite difference, where as the partial derivatives express the slope of a hyper-plane normal to surfaces that represent functions that comprise forward propagation. The goal is to set up circumstances so that successive approximations approach the ideal represented by minimization of the cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Back-propagation Theory&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back-propagation is a scheme for distribution of a correction signal arising from cost evaluation after each sample or mini-batch of them. With a form of Einsteinian notation, the current convention for distributive, incremental parameter improvement can be expressed concisely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Delta P = \dfrac {c(\vec{o}, \vec{\ell}) \; \alpha} {\big[ \prod^+ \! P \big] \; \big[ \prod^+ \!a'(\vec{s} \, P + \vec{z}) \big] \; \big[ c'(\vec{o}, \vec{\ell}) \big]} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plus sign in &lt;span class=&quot;math-container&quot;&gt;$\prod^+\!$&lt;/span&gt; designates that the factors multiplied must be downstream in the forward signal flow from the parameter matrix being updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In sentence form, &lt;span class=&quot;math-container&quot;&gt;$\Delta P$&lt;/span&gt; at any layer shall be the quotient of cost function &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; (given label vector &lt;span class=&quot;math-container&quot;&gt;$\vec{\ell}$&lt;/span&gt; and network output signal &lt;span class=&quot;math-container&quot;&gt;$\vec{o}$&lt;/span&gt;), attenuated by learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, over the product of all the derivatives leading up to the cost evaluation. The multiplication of these derivatives arise through the recursive application of the chain rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is because the chain rule is a core method for feedback signal evaluation that partial derivatives must be used. All variables must be bound except for one dependent and one independent variable for the chain rule to apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivatives include three types.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All layer input factors, the weights in the parameter matrix used to attenuate the signal during forward propagation, which are equal to the derivatives of those signal paths&lt;/li&gt;&#xA;&lt;li&gt;All the derivatives of activation functions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; evaluated at the sum of the matrix vector product of the parameters and signal at that layer plus the bias vector&lt;/li&gt;&#xA;&lt;li&gt;The derivative of the cost function &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; evaluated at the current output value &lt;span class=&quot;math-container&quot;&gt;$\vec{o}$&lt;/span&gt; with the label &lt;span class=&quot;math-container&quot;&gt;$\vec{\ell}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answer to the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, as a consequence of the above, the derivatives of both the cost (or loss or error) function and any activation functions are necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Redundant Operation Removal for an Efficient Algorithm Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actual back propagation algorithms save computing resources and time using three techniques. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Temporary storage of the value used for evaluation of the derivative (since it was already calculated during forward propagation)&lt;/li&gt;&#xA;&lt;li&gt;Temporary storage of products to avoid redundant multiplication operations (a form of reverse mode automatic differentiation)&lt;/li&gt;&#xA;&lt;li&gt;Use of reciprocals because division is more costly than multiplication at a hardware level&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition to these practical principles of algorithm design, other algorithm features arise from extensions of basic back-propagation. Mini-batch SGD (stochastic gradient descent) applies averaging to improve convergence reliability and accuracy in most cases, provided hyper-parameters and initial parameter states are well chosen. Gradual reduction of learning rates, momentum, and various other techniques are often used to further improve outcomes in deeper artificial networks.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-22T14:45:28.253" LastActivityDate="2018-12-22T14:45:28.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9651" PostTypeId="1" CreationDate="2018-12-22T15:46:44.227" Score="2" ViewCount="162" Body="&lt;p&gt;I'm a relative newbie to fuzzie logic systems but I have some knowledge in mathematics. I have the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to fuzzify certain values. Some are in the range [-&lt;span class=&quot;math-container&quot;&gt;$\inf$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$\inf$&lt;/span&gt;] and some are in the range [&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$\inf$&lt;/span&gt;]. For the first range I have chosen the sigmoid function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) = \frac{1}{1+e^{-x}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, which fuzzification process I should use for the second range. Since the function &lt;span class=&quot;math-container&quot;&gt;$f(x) = \ln(x)$&lt;/span&gt; transforms [&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$\inf$&lt;/span&gt;] to [-&lt;span class=&quot;math-container&quot;&gt;$\inf$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$\inf$&lt;/span&gt;] a natural choice could be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) = \frac{1}{1+e^{-\ln(x)}} = \frac{x}{x+1}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different function could also be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) = 1 - 2^{-x}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one would be more suitable? Particularly when considering that I may want to compare values from both ranges. &lt;/p&gt;&#xA;" OwnerUserId="20880" LastActivityDate="2020-09-21T13:00:10.027" Title="Choice of fuzzification function" Tags="&lt;math&gt;&lt;logic&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9652" PostTypeId="1" AcceptedAnswerId="9655" CreationDate="2018-12-22T17:34:42.387" Score="2" ViewCount="677" Body="&lt;p&gt;I tried to build a neural network for working on IRIS dataset using only numpy after reading an article (link: &lt;a href=&quot;https://iamtrask.github.io/2015/07/12/basic-python-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://iamtrask.github.io/2015/07/12/basic-python-network/&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to search the internet but everyone was using ml libraries and found no solution using just numpy. I tried to add different hidden layers to my feed forward neural network still it wasn't converging. I tried to use backpropagation. I used sigmoid and also relu neither of which was successful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please give me the code which will work on IRIS dataset and built only using feed forward neural networks and numpy as the only library or if it is not possible to built such a thing with these constraints then please let me know what goes wrong with these constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also tell me will it be possible to create a neural network to predict values of a matrix multiplication i.e. if we have A * B = C with matrix A as input and C as output, can we acheive substantial amount of accuracy with feed forward neural networks here?.  &lt;/p&gt;&#xA;" OwnerUserId="20881" LastActivityDate="2018-12-22T20:37:31.960" Title="Feed forward neural network using numpy for IRIS dataset" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9653" PostTypeId="2" ParentId="9590" CreationDate="2018-12-22T19:07:28.870" Score="0" Body="&lt;p&gt;On doing some research on why this problem might be occurring, I delved into some statistics of the environment. Interestingly, after a small number of episodes (~20), the agent always chooses to take only one action (this has been mentioned in the question too). Also, the Q values of the state-action pairs do not change a lot after just about 20 episodes. Same is the case for policy of environment, as may be expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that, although all the individual updates are being done correctly, the following scenario occurs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The update equation used is the following :-&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q(s1, a1) &amp;lt;- r1 + gamma * Q(s2, a2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, any update to function approximator means that the Q value changes not just for the updated (state, action) pair, but for all (state, action) pairs. How much it changes for any specific (state, action) pairs is another issue. Now, since our function approximator is altered, next time we use the previous equation for updating any other state, we use the following equation :-&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q(s3, a3) &amp;lt;- r3 + gamma * Q(s4, a4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But since Q has itself been changed, the target value of the record 3 changes. This is not desirable. Over time, all the changes cancel each other and Q value remains roughly the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using something known as the target function approximator(target network), we can maintain an older version of function approximator, which is used to get the Q-value of next state-action pair while the time of update. This helps avoid the problem and can be used to solve the environment.&lt;/p&gt;&#xA;" OwnerUserId="20424" LastActivityDate="2018-12-22T19:07:28.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9654" PostTypeId="2" ParentId="9638" CreationDate="2018-12-22T20:25:49.113" Score="1" Body="&lt;p&gt;OK, here is one approach. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Acquire a data set of 'clean' audio samples without barking dogs and an data set of barking dogs sounds.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generate a training set by mixing random selections of clean audio with a random selections of barking dog noises at various volume levels. This is your input data. Your output data is the clean audio before you added the barking.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find a suitable neural network. This might be a good starting point: &lt;a href=&quot;https://github.com/andabi/music-source-separation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/andabi/music-source-separation&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train the network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply the network to a load of the clean+bark mixes and check the result by ear. If the results are not good enough, go to 3 to tweak the network. If the results are still poor go to 2 and generate more training data, perhaps adding some distortions or other effects the the barking. Happy with the result? Go to 6.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Publish the results online for the betterment of humanity. (And/or write the network to  a miniature GPU board (such as the NVIDIA Jetson &lt;a href=&quot;https://developer.nvidia.com/buy-jetson&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developer.nvidia.com/buy-jetson&lt;/a&gt;) and work out how to wire that up to the decoded audio in your telly before it heads to the speakers), then blog about it because that would be a great project.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2018-12-22T20:25:49.113" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9655" PostTypeId="2" ParentId="9652" CreationDate="2018-12-22T20:37:31.960" Score="2" Body="&lt;p&gt;You can not use the ready code directly without any manipulation. Because every piece of code is written for specific datasets. In the article that you mentioned, the writer created a small dataset and then he creates an ANN architecture for it. If you want to build an ANN based on Iris dataset, you should think and create an architecture on paper maybe before coding. You should understand the Iris dataset first, I mean you should understand what is going to be your input shapes, and also the shape of weights are related to inputs' shape. After understanding these you should decide the number of layer of your Neural Network. And that tutorial that you followed is explaining all of these clearly. Do not skip and understand everything steps by step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a build ANN architecture by yourself, you must understand every mathematics behind that. (forward and backward propagation, loss and cost, gradient descent) If you do not want to deal with all of these maths, you can use a library such as Keras, then you can create an ANN easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this as a source, &lt;a href=&quot;https://ml-cheatsheet.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ml-cheatsheet.readthedocs.io/en/latest/&lt;/a&gt;&#xA;And you see my Kaggle kernel, I create this network with details and only using numpy. Also I used Keras on another section. You can check; &lt;a href=&quot;https://www.kaggle.com/erdemuysal/gender-recognition-with-lr-and-ann&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/erdemuysal/gender-recognition-with-lr-and-ann&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20884" LastActivityDate="2018-12-22T20:37:31.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9656" PostTypeId="2" ParentId="9590" CreationDate="2018-12-23T00:24:12.767" Score="2" Body="&lt;p&gt;One possible cause of SARSA with a linear approximator failing to learn a good policy for the Mountain Car problem is that you are not encoding the environment state, the position and velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In David Silver's Lecture 6 that you mention in your question, he describes the state encoding, which in this case is a tile coding, here: &lt;a href=&quot;https://youtu.be/UoPei5o4fps?t=3626&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/UoPei5o4fps?t=3626&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This coarse coding is described in more detail in the Sutton and Barto book, Reinforcement Learning: An Introduction (&lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://incompleteideas.net/book/the-book.html&lt;/a&gt;), in Chapter 9, where different state encodings for linear approximators are described, and Chapter 10, where the Mountain Car problem is discussed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, using a tile coding worked in combination with a linear function to solve the Mountain Car task. Using no encoding did not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following plot shows the max q-values (the policy) when using Sarsa with no encoding. The points are colored with the corresponding action of the max q-value to more clearly show the policy the agent will follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VG9oW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VG9oW.png&quot; alt=&quot;Sarsa q-values with no encoding&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This planar policy produces an average score of -200.0 per episode, averaged over 100 episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contrast that policy with the following graph of q-values produced by using Sarsa in combination with a tile encoding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nbb3V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nbb3V.png&quot; alt=&quot;Sarsa q-values with tile encoding&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy of this agent scores -102.9 per episode, averaged over 100 episodes. It might be hard to see in the graph, but it has the characteristic &quot;spiral&quot; of the learned graph from Figure 10.1 of the Sutton and Barto book.&lt;/p&gt;&#xA;" OwnerUserId="15444" LastEditorUserId="15444" LastEditDate="2018-12-27T00:32:30.690" LastActivityDate="2018-12-27T00:32:30.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9660" PostTypeId="1" CreationDate="2018-12-23T06:20:11.647" Score="1" ViewCount="70" Body="&lt;p&gt;Problem: &quot;For a given news article, generate another title for the article if the article is to be published under a different Publication.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithm will be well suited for this? Should I use naive Bayesian or any NLP algorithm? &lt;/p&gt;&#xA;" OwnerUserId="20878" LastEditorUserId="4302" LastEditDate="2018-12-23T23:39:03.760" LastActivityDate="2021-05-12T03:06:20.567" Title="What AI designs are suited for producing title replacements?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9661" PostTypeId="1" CreationDate="2018-12-23T07:09:23.033" Score="2" ViewCount="297" Body="&lt;p&gt;I have a steady hex-map and turn-based wargame featuring WWII carrier battles. I would like to improve the given AI part of the game using reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;I have a bunch of noob questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is Python mandatory?  The goal is to learn through self-play and the program is written in Objective-C with the aim to migrate it to Unity/C#.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it hopeless to achieve anything with a single computer?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20886" LastEditorUserId="2444" LastEditDate="2021-01-17T17:47:06.197" LastActivityDate="2021-01-17T17:47:06.197" Title="Is Python and a single computer sufficient to implement and train an RL-based agent for a turn-based war game?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;game-ai&gt;&lt;hardware&gt;&lt;self-play&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2021-01-18T04:05:46.773" ContentLicense="CC BY-SA 4.0" />
  <row Id="9662" PostTypeId="1" CreationDate="2018-12-23T07:49:29.403" Score="3" ViewCount="330" Body="&lt;p&gt;I have a steady hex-map and turn-based wargame featuring WWII carrier battles.&lt;/p&gt;&#xA;&lt;p&gt;On a given turn, a player may choose to perform a large number of actions. Actions can be of many different types, and some actions may be performed independently of each other while others have dependencies. For example, a player may decide to move one or two naval units, then assign a mission to an air unit or not, then adjust some battle parameters or not, and then reorganize a naval task force or not.&lt;/p&gt;&#xA;&lt;p&gt; Usually, boardgames allow players to perform only one action each turn (e.g. go or chess) or a few very similar actions (backgammon).&lt;/p&gt;&#xA;&lt;p&gt;Here the player may select&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Several actions&lt;/li&gt;&#xA;&lt;li&gt;The actions are of different nature&lt;/li&gt;&#xA;&lt;li&gt;Each action may have parameters that the player must set (e.g. strength, payload, destination)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;How could I approach this problem with reinforcement learning? How would I specify a model or train it effectively to play such a game?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is a screenshot of the game.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/21wjd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/21wjd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's another.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vgUiV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vgUiV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20886" LastEditorUserId="2444" LastEditDate="2021-01-17T15:08:24.907" LastActivityDate="2021-01-17T15:13:40.017" Title="How can I design a reinforcement learning model for a game with multiple complex actions taken at a time?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;&lt;action-spaces&gt;&lt;hierarchical-rl&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9663" PostTypeId="2" ParentId="9660" CreationDate="2018-12-23T08:15:04.570" Score="1" Body="&lt;p&gt;You can use a seq2seq LSTM model. Giving a article will generate a title for it.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You need to train it on data which comprises news articles and their corresponding titles.&lt;/p&gt;&#xA;&lt;p&gt;You can refer &lt;a href=&quot;https://towardsdatascience.com/sequence-to-sequence-model-introduction-and-concepts-44d9b41cd42d&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://towardsdatascience.com/seq2seq-model-in-tensorflow-ec0c557e560f&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18985" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-12-23T08:15:04.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9664" PostTypeId="2" ParentId="9661" CreationDate="2018-12-23T10:06:52.260" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is Python mandatory?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not at all. There is nothing magic about the Python language that makes it more suitable for learning algorithms. However, there are some very good machine learning and reinforcement learning libraries available for free that use Python. You might search for longer, or have more need to create your own code when using Objective-C.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it hopeless to achieve anything with a single computer ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That depends on the complexity of your game. The big projects using deep reinforcement learning use multi-million pound installations to research and train their game-playing algorithms. For instance, here is the &lt;a href=&quot;https://blog.openai.com/openai-five/&quot; rel=&quot;nofollow noreferrer&quot;&gt;spec used by Open AI learning to play DoTA&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores — a larger-scale version of the system we built to play the much-simpler solo variant of the game last year.&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of us cannot afford such a setup. If your game is as complex as DoTA, then this is an indication that you probably don't have the hardware to train an agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, complexity works on a huge range of scales in RL. For example, it is possible to train an AlphaZero clone to play Connect 4 on a single machine with a decent GPU in around a day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to explore simpler games as you learn the algorithms, so that will give you a sense of your challenge. You can also present your bot with sub-problems within your game to learn. Doing this should help you get a better feel for the challenges of writing learning agents for your project. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-12-23T10:06:52.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9667" PostTypeId="1" AcceptedAnswerId="10478" CreationDate="2018-12-23T18:13:37.787" Score="3" ViewCount="85" Body="&lt;p&gt;I'm trying to implement the NEAT Algorithm using c#, based off of Kenneth O. Stanley's paper. On page 109 (12 in the pdf) it states &quot;Matching genes are inherited randomly, whereas disjoint genes (those that do not match in the middle) and excess genes (those that do not match in the end) are inherited from the more fit parent.&quot;&lt;br&gt;&#xA;Does this mean that the child will always have the exact structure that the more fit parent has? It seems like the only way the structure could differ from crossover was if the two parents were equally fit.&lt;/p&gt;&#xA;" OwnerUserId="20892" LastActivityDate="2019-02-09T20:56:31.710" Title="Using NEAT, will the child of two parent genomes always have the same structure as the more fit parent?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9669" PostTypeId="1" CreationDate="2018-12-24T02:48:58.917" Score="1" ViewCount="530" Body="&lt;p&gt;I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the number of nodes in the hidden layer need to be the same as that of the input sequence as depicted in the figure&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Cc9z6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Cc9z6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or can we have less than the number of input sequences. If less than number of input sequence can be used how do we interpret the LSTM structure&lt;/p&gt;&#xA;" OwnerUserId="20900" LastActivityDate="2018-12-24T07:53:14.917" Title="Number of nodes in hidden layer for LSTM" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9670" PostTypeId="2" ParentId="9602" CreationDate="2018-12-24T07:13:28.710" Score="1" Body="&lt;p&gt;This is the classic question of what structure is or can be. It relates directly to the concepts of generalization, pattern recognition, over-fitting in surface fitting strategies, and learning &lt;em&gt;tabula rasa&lt;/em&gt;, Latin for &lt;em&gt;blank slate&lt;/em&gt;. The underlying questions are these:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can it be determined whether the organization of data in a set, which appears to correlate well with model &lt;span class=&quot;math-container&quot;&gt;$\mathbb{A}$&lt;/span&gt;, isn't just a random data set that merely appears to have &lt;span class=&quot;math-container&quot;&gt;$\mathbb{A}$&lt;/span&gt; organization.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can it be determined whether the organization of data in a set, which appears to correlate well with model &lt;span class=&quot;math-container&quot;&gt;$\mathbb{A}$&lt;/span&gt;, isn't generated by a phenomenon that generally exhibits organization that correlates well with &lt;span class=&quot;math-container&quot;&gt;$\mathbb{B}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can we determine the interrelationship between &lt;span class=&quot;math-container&quot;&gt;$\mathbb{A}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{B}$&lt;/span&gt;, especially since inclusion and overlap relates to the above question?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the second image in the question, if &lt;span class=&quot;math-container&quot;&gt;$\mathbb{A}$&lt;/span&gt; is defined as clustering of a type the smallest boundaries of which are not breached by instances of another type with at least five members per cluster, then several false positives would occur if the locations of the points were generated with quantum or thermal noise and the model was intended to illuminate clustering. In such a case there may be the appearance of clusters, but proximity would not be indicative of organization.&lt;/p&gt;&#xA;&lt;p&gt;In the first image in the question, if &lt;span class=&quot;math-container&quot;&gt;$\mathbb{A}$&lt;/span&gt; is the compound angular projection of the orbital paths of the planets in this solar system, then the image would appear as noise.&lt;/p&gt;&#xA;&lt;p&gt;If a third image existed of the moon's sky, the same model would likely produce a false negative if the tolerance of inaccuracy in input was high.&lt;/p&gt;&#xA;&lt;p&gt;It is difficult to reliability detect organization in the measured properties of a complex system. The goal of feature extraction, using RBMs or some other approach is to detect patterns. Whether those patterns are organized or merely apparent is a hypothesis that may be strengthened or weakened as the amount of data analyzed increases.&lt;/p&gt;&#xA;&lt;p&gt;If an AI system is trained to detect rats but knows nothing of cats and dogs, one cannot assume it will differentiate cats from dogs on the basis of the closer relationship between rodent and cat features than between rodent and dog features unless the AI system was meta-trained or programmed statically with a specific conception of what a feature is.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2018-12-26T02:41:25.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9673" PostTypeId="2" ParentId="7935" CreationDate="2018-12-24T13:34:38.183" Score="2" Body="&lt;p&gt;Check this implementation in TensorFlow Eager: &lt;a href=&quot;https://github.com/crisbodnar/TensorFlow-NEAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/crisbodnar/TensorFlow-NEAT&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20914" LastActivityDate="2018-12-24T13:34:38.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9674" PostTypeId="1" CreationDate="2018-12-24T17:12:58.153" Score="1" ViewCount="31" Body="&lt;p&gt;For &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://neuralnetworksanddeeplearning.com/chap5.html&lt;/a&gt; , could anyone suggest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) how to approach the derivation of expression (123) ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) what constitutes value ~ 0.45 ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) why the need of taylor series when we can observe the identity property without any maths proof (input == output) ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hh3uI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hh3uI.png&quot; alt=&quot;exploding gradient&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20570" LastActivityDate="2018-12-24T17:12:58.153" Title="neuralnetworksanddeeplearning.com chapter 5 problems" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;artificial-neuron&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9675" PostTypeId="2" ParentId="9662" CreationDate="2018-12-24T20:47:49.907" Score="1" Body="&lt;p&gt;The master's thesis &lt;a href=&quot;https://project.dke.maastrichtuniversity.nl/games/files/msc/Roelofs_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Action space representation in combinatorial multi-armed bandits&lt;/a&gt; (2015) seems to provide an answer to my question.&lt;/p&gt;&#xA;&lt;p&gt;Several algorithms can be used&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Naive Monte-Carlo Sampling (NMC)&lt;/li&gt;&#xA;&lt;li&gt;Linear Side Information (LSI)&lt;/li&gt;&#xA;&lt;li&gt;Monte Carlo Tree Search with Hierarchical Expansion (MCTS-HE)&lt;/li&gt;&#xA;&lt;li&gt;MCTS with Dimensional Expansion&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The idea is to divide and conquer.&lt;/p&gt;&#xA;&lt;p&gt;Here's a screenshot of the HE algorithm from &lt;a href=&quot;https://project.dke.maastrichtuniversity.nl/games/files/msc/Roelofs_thesis.pdf#page=32&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 4.2&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X7ABX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X7ABX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20886" LastEditorUserId="2444" LastEditDate="2021-01-17T15:13:40.017" LastActivityDate="2021-01-17T15:13:40.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9677" PostTypeId="1" CreationDate="2018-12-25T10:47:57.407" Score="0" ViewCount="82" Body="&lt;p&gt;I am trying to use a artifical neural network to produce a single output, which in my mind should be an index into a list of data (or close to it). All of the results I get are 0.9999+ and very close to each other. I don't know if my whole way of thinking here is off, or if I am just missing an approach, or if perhaps my network code is just broken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to make use of the simple neural network from Microsoft here:&#xA;&lt;a href=&quot;https://social.technet.microsoft.com/wiki/contents/articles/36428.basis-of-neural-networks-in-c.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://social.technet.microsoft.com/wiki/contents/articles/36428.basis-of-neural-networks-in-c.aspx&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried this with a significantly more complex data set, but I've also tried using a very simple data set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the simple training data I'm trying to use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eat poo bad&#xA;eat dirt bad&#xA;eat cookies okay&#xA;eat fruit good&#xA;study poo okay&#xA;study dirt okay&#xA;study cookies okay&#xA;study fruit okay&#xA;dispose poo good&#xA;dispose dirt okay&#xA;dispose cookies bad&#xA;dispose fruit bad&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that the network has two input neurons and a single output neuron. I assigned a unique number to each distinct word such that I can train the network with two inputs (verb and object), and expect a single output (good, bad, or okay).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: 1 (for eat) 10 (for dirt) output: 15 (for bad)&#xA;input: 1 (for eat) 11 (for cookies) output: 16 (for good)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would expect that after training, I would see the output numbers close to 15, 16, etc, but all I get are numbers like 0.999997333313168, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: 1 (for eat) 10 (for dirt), output is 0.999997333313168 (instead of ~15 expected)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What do these outputs mean, or what am I missing in how I should be thinking about making a basic classification system (given inputs, get a meaningful output)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The C# code I am using, if it is helpful:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using NeuralNet.NeuralNet;&#xA;using System;&#xA;using System.Collections.Generic;&#xA;using System.Linq;&#xA;using System.Text;&#xA;&#xA;namespace NeuralNet&#xA;{&#xA;    internal class TestSmallSampleNetwork&#xA;    {&#xA;        internal static void Run()&#xA;        {&#xA;            var data = @&quot;eat poo bad&#xA;                        eat dirt bad&#xA;                        eat cookies okay&#xA;                        eat fruit good&#xA;                        sell poo bad&#xA;                        sell dirt okay&#xA;                        sell cookies okay&#xA;                        sell fruit okay&#xA;                        study poo okay&#xA;                        study dirt okay&#xA;                        study cookies okay&#xA;                        study fruit okay&#xA;                        dispose poo good&#xA;                        dispose dirt okay&#xA;                        dispose cookies bad&#xA;                        dispose fruit bad&quot;;&#xA;            var simples = data.Split(new[] { &quot;\r\n&quot; }, StringSplitOptions.None ).Select(_ =&amp;gt; new Simple(_)).ToList();&#xA;&#xA;            var verbs = simples.Select(_ =&amp;gt; _.Verb).Distinct().Select(_ =&amp;gt; new NetworkValue(_)).ToList();&#xA;            var objects = simples.Select(_ =&amp;gt; _.Object).Distinct().Select(_ =&amp;gt; new NetworkValue(_)).ToList();&#xA;            var judgments = simples.Select(_ =&amp;gt; _.Good).Distinct().Select(_ =&amp;gt; new NetworkValue(_)).ToList();&#xA;            var values = verbs.Concat(objects).Concat(judgments).ToDictionary(_ =&amp;gt; _.Term, _ =&amp;gt; _);&#xA;&#xA;            // Create a network with 2 inputs, 2 neurons on a single hidden layer, and 1 neuron output.&#xA;            var net = new NeuralNetwork(0.9, new int[] { 2, 2, 1 });&#xA;&#xA;            for (int iTrain = 0; iTrain &amp;lt; 1000; iTrain++)&#xA;            {&#xA;                for (int iSimple = 0; iSimple &amp;lt; simples.Count; iSimple++)&#xA;                {&#xA;                    net.Train(MakeInputs(simples[iSimple], values), MakeOutputs(simples[iSimple], values));&#xA;                }&#xA;            }&#xA;&#xA;            foreach (var value in values.Values)&#xA;            {&#xA;                Console.WriteLine(value);&#xA;            }&#xA;&#xA;            Console.WriteLine();&#xA;&#xA;            // Run samples and get results back from the network&#xA;            Run(&quot;study&quot;, &quot;poo&quot;, values, net);&#xA;            Run(&quot;eat&quot;, &quot;poo&quot;, values, net);&#xA;            Run(&quot;dispose&quot;, &quot;fruit&quot;, values, net);&#xA;            Run(&quot;sell&quot;, &quot;dirt&quot;, values, net);&#xA;        }&#xA;&#xA;        private static void Run(string verb, string obj, Dictionary&amp;lt;string, NetworkValue&amp;gt; values, NeuralNetwork net)&#xA;        {&#xA;            var result = net.Run(new List&amp;lt;double&amp;gt; {&#xA;                values[verb].Value,&#xA;                values[obj].Value,&#xA;            }).Single();&#xA;&#xA;            var good = &quot;xxx&quot;;&#xA;&#xA;            Console.WriteLine($&quot;{verb} {obj} {good} ({result})&quot;);&#xA;        }&#xA;&#xA;        private static List&amp;lt;double&amp;gt; MakeInputs(Simple simple, Dictionary&amp;lt;string, NetworkValue&amp;gt; values)&#xA;        {&#xA;            return new List&amp;lt;double&amp;gt;() {&#xA;                values[simple.Verb].Value,&#xA;                values[simple.Object].Value&#xA;            };&#xA;        }&#xA;&#xA;        private static List&amp;lt;double&amp;gt; MakeOutputs(Simple simple, Dictionary&amp;lt;string, NetworkValue&amp;gt; values)&#xA;        {&#xA;            return new List&amp;lt;double&amp;gt; { values[simple.Good].Value };&#xA;        }&#xA;&#xA;        public class Simple&#xA;        {&#xA;            public string Verb { get; set; }&#xA;            public string Object { get; set; }&#xA;            public string Good { get; set; }&#xA;&#xA;            public Simple(string line)&#xA;            {&#xA;                var words = line.Trim().Split(&quot; &quot;.ToCharArray(), 3, StringSplitOptions.None);&#xA;                Verb = words[0];&#xA;                Object = words[1];&#xA;                Good = words[2];&#xA;            }&#xA;&#xA;            public override string ToString()&#xA;            {&#xA;                return $&quot;{Verb} {Object} {Good}&quot;;&#xA;            }&#xA;        }&#xA;&#xA;        public class NetworkValue&#xA;        {&#xA;            private static int Next = 1;&#xA;&#xA;            public string Term { get; set; }&#xA;            public double Value { get; set; }&#xA;&#xA;            public NetworkValue(string term)&#xA;            {&#xA;                Term = term;&#xA;                Value = Next++;&#xA;            }&#xA;&#xA;            public override string ToString()&#xA;            {&#xA;                return $&quot;{Value}. {Term}&quot;;&#xA;            }&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20930" LastActivityDate="2019-01-02T05:28:26.070" Title="How do I get a meaningful output value for a simple neural network that can map to a set of data?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9681" PostTypeId="1" CreationDate="2018-12-25T20:30:47.270" Score="1" ViewCount="495" Body="&lt;p&gt;How can I use a 2-dimensional feature matrix, rather than a feature vector, as the input to a neural network?&lt;/p&gt;&#xA;&lt;p&gt;For a WWII naval wargame, I have sorted out the features of interest to approximate the game state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; at a given time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;they are a maximum of 50 naval task forces and airbases on the map&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;each one has a lot of features (hex, weather, number of ships, type, distance to other task forces, distance to objective, cargo, intelligence, combat value, speed, damage, etc.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The output would be the probability of winning and the level of victory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gh2U5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gh2U5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20886" LastEditorUserId="2444" LastEditDate="2021-01-17T17:42:35.330" LastActivityDate="2021-01-17T17:42:35.330" Title="How can I use a 2-dimensional feature matrix as the input to a neural network?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9682" PostTypeId="1" CreationDate="2018-12-25T20:32:24.363" Score="2" ViewCount="341" Body="&lt;p&gt;Suppose I want to predict the position of a sensor based on its reading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can first predict the unit vector and predict the distance to be multiplied to this vector.&#xA;And I know that distance will never be negative because all the negative parts are inside unit vector already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Should I apply ReLU to the distance before multiplying it to the unit vector?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking that this can be helpful to eliminate the network from needing too much training data by restricting the output ranges the network could give. But I also think that it could make the learning slower when the ReLU unit dies (value=0) so the gradient doesn't flow properly somehow.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2021-05-19T07:04:35.380" Title="Should I apply ReLU to non negative output?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9683" PostTypeId="2" ParentId="9624" CreationDate="2018-12-26T00:06:01.413" Score="1" Body="&lt;p&gt;Generative networks was the first popular class of topologies that had compound feedback, corrective signaling at more than one layer.  In popular network designs such as MLPs (multilayer perceptrons), CNNs (convolutional neural networks), and LSTM (long short term memory) networks, the backpropagation is a single layer mechanism.  The mechanism distributes corrective a signal creating a closed loop that ideally converges to an optimal network behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The speed, accuracy, and reliability of convergence of networks in an AI system depend on a number of engineering choices at a lower level of design and development.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Selection of algorithm variants and extensions&lt;/li&gt;&#xA;&lt;li&gt;Selection of data set&lt;/li&gt;&#xA;&lt;li&gt;Drawing of the training samples&lt;/li&gt;&#xA;&lt;li&gt;Data normalization&lt;/li&gt;&#xA;&lt;li&gt;Initialization&lt;/li&gt;&#xA;&lt;li&gt;Assignment of learning rate or learning rate policy&lt;/li&gt;&#xA;&lt;li&gt;Other hyper-parameters&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Generative networks are designed to achieve a higher level balance between two networks, each of which have their own backpropagation, and thus have a compound feedback topology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the original GAN approach in the context of fraud detection, a discriminative network &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, in a way, detects fraudulence originating from the generative network &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, and, in a way, &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; learns authenticity from the corrective signal provided by &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, even if the authenticity learned is superficial.  However, in the case of credit card fraud detection from a numerical set of data, the fraudulence is not internal to the system to be designed as in the case of GAN.  Without both the &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; networks, the design is not a generative network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the comment stated, &quot;I have to explore GAN for this problem,&quot; the key question is this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If this system is to benefit from a GAN, what would the GAN generate that could contribute to the solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the answer is nothing, then a GAN is not the correct approach.  A discriminative network without a generative network is not a GAN.  If this question is from an assignment to train a network using a static and already existing data set, a MLP may suffice, depending on the expectations of the teaching staff.  If that is the case, the goal will be to analyze the data set using statistical tools to determine if it is already normalized well enough to lead to excellent convergence accuracy, speed, and reliability.  If normalization is indicated, find ways to present the data to the input layer of the MLP in such a way that each input is relevant and has a well distributed range of values across the data type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ensure there is no data loss in this process, and remove redundancy where it can easily be removed.  By redundancy removal is meant, if one of the dimensions of data have only four possible values of between 10 and 1,000 bytes, encode the four possible values in two bits because to do so removes unnecessary burden from the training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the output layer must be chosen to produce an encoded set of possible output values and the decoding must be written to produce usable output.  In this case, it appears that the labels are, Fraudulent and Authentic.  Which means that the last layer would be a single binary threshold cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this simple MLP approach and for many of the approaches mentioned next, there are examples available in Python, Java, C++ and other languages from which implementation can be initiated.  If the design must stand up to a more realistic fraud detection scenario, either for credit cards, junk mail, banking, log-in, or communications those experienced with fraud detection know that the above approach is entirely insufficient except as a starting point for learning.  In such a case, these considerations further complicate the design of the solution.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fraudulence and authenticity is not necessarily sufficient information for investigators.&#xA;    The likelihood of fraudulence and the reason why fraudulence is suspected is also pertinent for end users of production output,&#xA;    so that security holes can be blocked and investigation may be directed toward prosecution or some other set of remedies.&lt;/li&gt;&#xA;&lt;li&gt;Fraudulence is usually a security breach phenomena that involves continuous improvement in the attacks until the attackers&#xA;    are located and their work is interrupted by arrest.&#xA;    What initially works as fraud detection may not work a week, a day, an hour, or a minute later.&lt;/li&gt;&#xA;&lt;li&gt;Fraud detection, if it to be useful in a real financial context, must be part of a fraud prevention system,&#xA;    and training on a static data set may be meaningless in that larger usage environment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In AI systems that work in real production environments, it may be worth investigating using an LSTM network in combination with a reinforcement learner.  Such may be over a beginner's level of design and implementation experience, but using either an LSTM network or a basic Q-learning approach instead of both of them together may be a reasonable starting point for dedicated students.  If the answer to the question about what a GAN could generate that would be of use must be yes, the following ideas may be helpful.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A generative network could generate transaction data with a distribution that resembles authentic transactions, and such has been successful in the past.&lt;/li&gt;&#xA;&lt;li&gt;A generative network could generate transaction data with a distribution that resembles attacks on an account by a fraudulent user. That has been done too.&lt;/li&gt;&#xA;&lt;li&gt;A generative network could theoretically generate hacker agents, although this may not have yet been done.&lt;/li&gt;&#xA;&lt;li&gt;A generative network could be used to generate patterns of authentic transactions based on some authenticity measure other than the data set mentioned, and the trained discriminative network could then be employed in some way in a fraud detection system in production, although it is unclear what advantage would be gained.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The reason a GAN was suggested would need to be clarified before this answer could provide a more exact direction for AI system topology and project approach.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-26T00:11:03.737" LastActivityDate="2018-12-26T00:11:03.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9684" PostTypeId="2" ParentId="9681" CreationDate="2018-12-26T10:25:42.763" Score="3" Body="&lt;p&gt;The most obvious way to do this would be to simply &quot;unroll&quot; your matrix into a vector. Your example input matrix would get turned into the following input vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left( \begin{array}{} a_1 &amp;amp; a_2 &amp;amp; \dots &amp;amp; a_t &amp;amp; b_1 &amp;amp; b_2 &amp;amp; \dots &amp;amp; b_t &amp;amp; c_1 &amp;amp; c_2 &amp;amp; \dots &amp;amp; c_t \end{array} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think there are any other clear ways to use an &quot;input matrix&quot; really. The only benefit I could see in using an input matrix rather than an unrolled vector (if it were possible to do so in whatever way) would be if doing so would somehow enable the learning algorithm to exploit the &quot;domain knowledge&quot; that certain input features are related to each other in special ways (i.e. features in the same row belong to the same unit, and features in the same column are the same &quot;type&quot; of feature, or other way around). Intuitively, I suspect something like this could be accomplished by restricting the number of connections you make to the next layer. For example, you could make a part of the next layer only be connected to all the &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; features, a different part connected only to all the &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; features, etc. Similarly, you could have a part that is connected only to the &lt;span class=&quot;math-container&quot;&gt;$a_1, b_1, c_1, \dots$&lt;/span&gt; features, a different part only connected to the &lt;span class=&quot;math-container&quot;&gt;$a_2, b_2, c_2, \dots$&lt;/span&gt; features, etc. I don't know for sure how well this would work though... just think that it could.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="20886" LastEditDate="2018-12-26T12:45:21.743" LastActivityDate="2018-12-26T12:45:21.743" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9686" PostTypeId="1" AcceptedAnswerId="9690" CreationDate="2018-12-26T10:55:25.273" Score="1" ViewCount="32" Body="&lt;p&gt;Imagine I wish to classify images of digits from 0-9.&#xA;Let's say I have trained the network to recognise '1'.&#xA;If I were to train the same network to recognise '2', wouldn't the backpropagation process mess up the weights and biases for '1'?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or do programs like Tensorflow allocate a new layer of neural network for different object classification? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="20519" LastActivityDate="2018-12-26T16:56:58.943" Title="Beginner - Object classification data in a neural network" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9687" PostTypeId="2" ParentId="9681" CreationDate="2018-12-26T12:12:31.047" Score="0" Body="&lt;p&gt;(Split the answer from Dennis in two)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very different approach would be to use a network architecture with recurrence, for example an LSTM. You could treat your input as a &quot;sequence&quot; rather than a matrix (just like a sentence in language processing would be a sequence of inputs), providing your feature vectors for different units as inputs one at a time. This would remove the need of having a giant input layer (with support for 50 units) in cases where only a small portion of them would be used (e.g., if you only have 5 units). There is not really a concept of &quot;time&quot; in your inputs though... ideally the output of your network would be invariant to the order in which you provide it with the different inputs for the different units, but that would not typically be the case with these kinds of architectures in practice.&lt;/p&gt;&#xA;" OwnerUserId="20886" LastActivityDate="2018-12-26T12:12:31.047" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9689" PostTypeId="1" CreationDate="2018-12-26T12:44:54.080" Score="1" ViewCount="989" Body="&lt;p&gt;I am a novice developer in AI. Any help appropriated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of images and from that I want to predict position(x,y co-ordinates) of the Ball.&lt;a href=&quot;https://i.stack.imgur.com/bXjN9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bXjN9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="20953" LastActivityDate="2019-05-30T17:04:10.473" Title="I need to predict ball position from set of Images" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9690" PostTypeId="2" ParentId="9686" CreationDate="2018-12-26T16:56:58.943" Score="1" Body="&lt;p&gt;The answer is yes, retraining a neural network on a new dataset will alter its internal state such that it would no longer give the same output as before (in your case 'messing it up'). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are techniques to allow you to reuse sections of trained networks for other problems (this is called 'Transfer Learning'). Transfer Learning involves freezing the weights/biases for parts of the already trained network while adding extra new layers to the end of it and training those (with Back-propagation) for your new problem. For your digit recognition problem you would normally just train the network on all the digits at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow is lower level than the idea of neural networks and layers. It is a library that allows you to write mathematical pipelines that implement the tensor calculations required to build a neural network. To do the sort of thing that you mentioned you would use a higher level library such as Keras (&lt;a href=&quot;https://www.tensorflow.org/guide/keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/guide/keras&lt;/a&gt;) which now comes conveniently bundled with Tensorflow.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2018-12-26T16:56:58.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9691" PostTypeId="2" ParentId="9424" CreationDate="2018-12-26T17:16:09.033" Score="1" Body="&lt;p&gt;I will disagree slightly with @mico. There is a usage of &quot;additive rewards&quot; that refers to decomposable reward functions (e.g. my reward in selling an item I do not want to own is composed of the reward of not having an unwanted item anymore, plus the monetary gain in selling the item). But, there is indeed a fundamental relation between additive and discounted rewards. Additive rewards are formulated simply as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R([s_0, s_1, s_2, ...]) = R(s_0) + R(s_1) + R(s_2) + \cdots&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whereas discounted rewards include a discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma \in [0,1]$&lt;/span&gt; such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R([s_0, s_1, s_2, ...]) = R(s_0) + \gamma R(s_1) + \gamma^2R(s_2) + \cdots&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, the additive reward for a sequence of states is simply the sum of the rewards acquired at each state, while discounted rewards include a multiplicative discount factor that reduces the influence of rewards as time goes on. You will typically see additive rewards for finite-horizon problems, i.e. you have a discrete number of timesteps to optimize over, and discounted rewards are more relevant for infinite-horizon problems, i.e. you may need to optimize over an infinite number of timesteps (or at least a very large number). The discount factor governs the agent’s greediness in achieving immediate reward, where very small discount factors (closer to 0) encourage the agent to only seek rewards in proximal states, and very large discount factors (closer to 1) encourage the agent to think further into the future about what rewards it can expect to achieve in states it will visit later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most direct reference I found for this distinction is in &lt;a href=&quot;http://www-anw.cs.umass.edu/~barto/courses/CS383-Fall11/383-Fall11-Lec22.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these course slides&lt;/a&gt;, which are somewhat authoritative since they from Andrew Barto, a co-author of &lt;a href=&quot;https://mitpress.mit.edu/books/reinforcement-learning-second-edition&quot; rel=&quot;nofollow noreferrer&quot;&gt;the de facto text&lt;/a&gt; on reinforcement learning. &lt;/p&gt;&#xA;" OwnerUserId="20955" LastEditorUserId="20955" LastEditDate="2020-04-12T20:36:23.027" LastActivityDate="2020-04-12T20:36:23.027" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9692" PostTypeId="1" CreationDate="2018-12-26T18:37:30.567" Score="2" ViewCount="130" Body="&lt;p&gt;I am working on a credit card fraud detection problem using autoencoders. I have a question regarding the dataset I'll be using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've downloaded the &lt;a href=&quot;https://www.kaggle.com/mlg-ulb/creditcardfraud&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset for the above problem from Kaggle&lt;/a&gt;, which is highly imbalanced: it contains only 492 frauds out of 284,807 transactions. Why isn't the dataset already balanced? Should I balance it before applying the auto-encoder?&lt;/p&gt;&#xA;" OwnerUserId="20842" LastEditorUserId="2444" LastEditDate="2019-12-26T15:39:01.420" LastActivityDate="2019-12-26T16:32:03.663" Title="Why isn't the Credit Card Fraud Detection dataset from Kaggle already balanced?" Tags="&lt;neural-networks&gt;&lt;datasets&gt;&lt;unsupervised-learning&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9694" PostTypeId="2" ParentId="9689" CreationDate="2018-12-26T19:20:52.273" Score="2" Body="&lt;p&gt;There are so many challenges here for a novice or even a very seasoned AI developer!  There hasn't been a lot of development in this area.  I assume you are working with broadcast video.  Your biggest challenges will involve occlusions of the ball and changes in the view as broadcasters constantly switch cameras which have very different positions on the field or around the stadium.  You will need to track the ball and the players that take action on the ball.  The ball can be in motion from a player heading, kicking, or throwing the ball.  The ball can also be in motion from a bounce off a player or the ground.  To get started I recommend building off some work that at least can track the players and the ball and can identify certain actions such as a kick.  If you start there you can build some type of ball trajectory predictor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following paper &lt;a href=&quot;https://www.researchgate.net/publication/325030263_Soccer_Event_Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soccer Event Detection&lt;/a&gt;, by Abdullah Khan1, Beatrice Lazzerini, Gaetano Calabrese and Luciano Serafini, presents an event detector for &quot;Ball possession&quot; and &quot;Kicking the ball&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A paper entitled &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1810/1810.07550.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Newton Scheme for Deep Learning&lt;/a&gt; by Junqing Qiu, Guoren Zhong, Yihua Lu, Kun Xin, Huihuan Qian, and Xi Zhu, addresses predicting the trajectory of a ball in sports.  They address the complexities of a ball in motion.  They address situations where &quot;the ball is flying with a self-spinning, resulting in a Magnus force on it, which makes it difficult to predict the position by traditional methods&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CkglF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CkglF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some related articles and papers you might find of interest:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://squared2020.com/2017/05/07/identifying-player-possession-in-spatio-temporal-data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Identifying Player Possession in Spatio-Temporal Data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cs231n.stanford.edu/reports/2016/pdfs/273_Report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning using CNNs for Ball-by-Ball Outcome Classification in Sports&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1511.02917.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Detecting events and key actors in multi-person videos&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.ubc.ca/~murphyk/Papers/weilwun-pami12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Track and Identify Players from Broadcast Sports Videos&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://blog.infinigraph.com/category/deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video Artificial Intelligence Powers Creation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.engadget.com/2017/03/06/deep-learning-studies-athlete-decisions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI predicts how athletes will react in certain situations&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://adtmag.com/blogs/dev-watch/2017/07/sports-analytics.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;How AI-Based Sports Analytics Is Changing the Game&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2018-12-26T19:20:52.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9697" PostTypeId="2" ParentId="9234" CreationDate="2018-12-26T23:44:30.033" Score="0" Body="&lt;p&gt;You can use several consecutive frames as an input. If you have dataset of moving/not moving objects, you can use it to train your model. You can use CNNs with additional time dimension in this case, for example.&lt;/p&gt;&#xA;" OwnerUserId="18074" LastActivityDate="2018-12-26T23:44:30.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9698" PostTypeId="1" CreationDate="2018-12-26T23:54:25.537" Score="3" ViewCount="271" Body="&lt;p&gt;Each chromosome contains an array of genes, each gene contains a letter and a number, both letter and number can only exist once in each chromosome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Parent A = {a,1}{c,2}{e,3}{g,4}&#xA;Parent B = {a,2}{b,1}{c,4}{d,3}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What would be the best crossover operator to create a child that doesn't break the rule described above?&lt;/p&gt;&#xA;" OwnerUserId="20962" LastEditorUserId="20962" LastEditDate="2018-12-27T00:01:19.410" LastActivityDate="2019-01-12T03:16:20.773" Title="Genetic algorithm: How to crossover 2D permutation?" Tags="&lt;genetic-algorithms&gt;&lt;genetic-programming&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9700" PostTypeId="1" CreationDate="2018-12-27T05:32:48.897" Score="-1" ViewCount="2159" Body="&lt;p&gt;I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one hot encoded before being used and I am using the Adam optimizer which is implemented like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   for k in M.keys(): ##For k in weights&#xA;        M[k] = beta1 * M[k] + (1-beta1)*grad[k]&#xA;        R[k] = beta2 *R[k] + (1-beta2)*grad[k]**2&#xA;        m_k = M[k] / (1-beta1**n)&#xA;        r_k = R[k] / (1-beta2**n)&#xA;        model[k] = model[k] - alpha * m_k / np.sqrt(r_k + 1e-8)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Beta1 is set to 0.9, beta2 to 0.999 and alpha is set to 0.001. When I train it for 50,000 I get very high fluctuation of the cost and it never seems to significantly decrease (only sometimes due to the fluctuations (and I catch the weights with the lowest cost)).My hidden_size is 400 and the batch size is 200&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After sketching the cost of iterations I get a graph like this:&lt;a href=&quot;https://i.stack.imgur.com/Amcc3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Amcc3.png&quot; alt=&quot;]&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to be increasing on average only seeming to decrease to the the large fluctuations. What can I change to have better success and have it converge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I plotted the norm of the gradient using the slightly different cost function which @DennisSoemers suggested and the gradient does not reduce significantly.&lt;a href=&quot;https://i.stack.imgur.com/C64mf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C64mf.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry the title should read &quot;Gradient over iterations&quot;&lt;/p&gt;&#xA;" OwnerUserId="20328" LastEditorUserId="20328" LastEditDate="2019-01-01T03:31:09.133" LastActivityDate="2019-01-01T03:31:09.133" Title="RNN LSTM not converging with Adam" Tags="&lt;python&gt;&lt;recurrent-neural-networks&gt;&lt;optimization&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9701" PostTypeId="1" AcceptedAnswerId="9724" CreationDate="2018-12-27T09:25:28.060" Score="4" ViewCount="92" Body="&lt;p&gt;I'm relatively new to artificial intelligence and neural networks.&lt;/p&gt;&#xA;&lt;p&gt;Let's say I have two different fully trained neural networks. The first one is trained for mathematical addition and the second one on mathematical multiplication. Now, I want to combine these two neural networks into one that knows about both operations.&lt;/p&gt;&#xA;&lt;p&gt;Is this possible? Is there a representative name for this kind of technique?&lt;/p&gt;&#xA;&lt;p&gt;I had read something about bilinear CNN models that sounds similar to what I'm looking for, right?&lt;/p&gt;&#xA;" OwnerUserId="20968" LastEditorUserId="2444" LastEditDate="2020-06-21T13:27:10.817" LastActivityDate="2020-06-21T13:27:54.943" Title="Is it possible to combine two neural networks trained on different tasks into one that knows both tasks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9704" PostTypeId="1" AcceptedAnswerId="9706" CreationDate="2018-12-27T12:10:36.823" Score="3" ViewCount="112" Body="&lt;ul&gt;&#xA;&lt;li&gt;Is an artificial intelligence software? (a program or a set of programs) or can it be hardware?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15469" LastEditorUserId="1671" LastEditDate="2021-05-05T00:17:34.277" LastActivityDate="2021-05-05T01:14:18.027" Title="Is an artificial intelligence software or hardware?" Tags="&lt;definitions&gt;&lt;ai-basics&gt;&lt;hardware&gt;&lt;software&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9705" PostTypeId="2" ParentId="9698" CreationDate="2018-12-27T13:10:32.383" Score="1" Body="&lt;p&gt;First, store the two parent chromosomes into a sorted dictionary (in terms of implementation, &lt;code&gt;std::map&lt;/code&gt; in C++ might be a good option) where the key is the letter and the value is a pair of &lt;code&gt;int&lt;/code&gt;s (the second gene element). When you populate the map, add all the letters from each chromosome as keys and take &lt;code&gt;0&lt;/code&gt; (or a negative number) to indicate that a gene with the corresponding letter does &lt;em&gt;not&lt;/em&gt; exist in one of the parents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, create a second map, this time with the number as a key. Iterate over the first map. If both parent chromosomes contain a gene with the same letter, the value field in the map would contain two positive numbers - pick one at random and insert the {number, letter} combination into the second map &lt;em&gt;only if the number doesn't exist as a key&lt;/em&gt;. If the gene exists only in one parent, either pick the gene as is or choose to skip that gene altogether.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would guarantee that 1) your genomes contain a {letter, number} gene only once and 2) the genes in the offspring's chromosomes would be automatically sorted by letter. If you don't need the genes to be sorted but still need them to be unique, you can use an unordered map (a hash table).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, the map would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mapped parent chromosomes&#xA;{a: (1,2)}&#xA;{b: (0,1)}&#xA;{c: (2,4)}&#xA;{d: (0,3)}&#xA;{e: (3,0)}&#xA;{g: (4,0)}&#xA;&#xA;Offspring chromosome (OC)&#xA;{}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;0&lt;/code&gt; indicates that the corresponding gene does not exist in that parent. Iterating over this table from top to bottom, you might end up doing the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Key &lt;code&gt;a&lt;/code&gt;: pick &lt;code&gt;1&lt;/code&gt;; &lt;code&gt;1&lt;/code&gt; is not a key in OC, so insert &lt;code&gt;{1,a}&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Key &lt;code&gt;b&lt;/code&gt;: pick &lt;code&gt;1&lt;/code&gt;; &lt;code&gt;1&lt;/code&gt; is a key in OC, so skip letter &lt;code&gt;b&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Key &lt;code&gt;c&lt;/code&gt;: pick &lt;code&gt;4&lt;/code&gt;; &lt;code&gt;4&lt;/code&gt; is not a key in OC, so insert &lt;code&gt;{4,c}&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Key &lt;code&gt;d&lt;/code&gt;: choose to skip&lt;/li&gt;&#xA;&lt;li&gt;Key &lt;code&gt;e&lt;/code&gt;: pick &lt;code&gt;3&lt;/code&gt;; &lt;code&gt;3&lt;/code&gt; is not a key in OC, so insert &lt;code&gt;{3,e}&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Key &lt;code&gt;g&lt;/code&gt;: pick &lt;code&gt;4&lt;/code&gt;; &lt;code&gt;4&lt;/code&gt; is a key in OC, so skip letter &lt;code&gt;g&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Final offspring chromosome:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{a,1}{c,4}{e,3} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="16101" LastEditDate="2018-12-27T13:50:08.420" LastActivityDate="2018-12-27T13:50:08.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9706" PostTypeId="2" ParentId="9704" CreationDate="2018-12-27T15:40:23.450" Score="3" Body="&lt;p&gt;AI or Artificial Intelligence is nothing but intelligence but in its artificial form. Intelligence comes from formation of rules and patterns in the data which is seen or on which it is trained.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For us, we programme or formulate these rules in Mathematics on which an Intelligence is created.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ex. A neural network shows a lower level of Intelligence. It is created using basic rules of mathematics like gradient descent, derivatives and other arithmetic stuff.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;These rules need a base to form themselves. Here's where computers come in. They run these rules on which intelligence thrives in binary ( or quantum sometimes ). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;But, every bit of information acquires physical space for processing and storage. Here's where hardware comes. Hardware provides the container in which  information will be processed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hence, for an complete AI system, computer programmes provide a soul whereas hardware provides a body/physical form for the soul.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, only AI is concerned, it may run on any hardware as it is a software or program or a set of mathematical rules in binary state.&#xA;Hence AI is a program.&lt;/p&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-12-27T15:40:23.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9707" PostTypeId="2" ParentId="9700" CreationDate="2018-12-27T17:09:03.953" Score="0" Body="&lt;p&gt;Adam has combined features of RMSProp and Momentum, which sometimes perform bad on &lt;strong&gt;Recurrent&lt;/strong&gt; network. In my own experience, RMSProp give good convergence rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another: you can try &lt;strong&gt;NADAM&lt;/strong&gt;, which is RMSProp+Nestorov-Momentum. This adds a correction step over momentum. On text-dataset, it has performed well as compared to other optimizers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: Optimizers can not be only reason for convergence, it depend on many others factors like network architecture, activations, input data-set, encoding and so on.&lt;/p&gt;&#xA;" OwnerUserId="16313" LastActivityDate="2018-12-27T17:09:03.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9708" PostTypeId="1" CreationDate="2018-12-27T17:26:27.467" Score="7" ViewCount="609" Body="&lt;p&gt;How should we choose the dimensions of the encoding layer in auto-encoders?&lt;/p&gt;&#xA;" OwnerUserId="20842" LastEditorUserId="2444" LastEditDate="2021-05-17T22:01:53.517" LastActivityDate="2021-05-17T22:01:53.517" Title="How should we choose the dimensions of the encoding layer in auto-encoders?" Tags="&lt;neural-networks&gt;&lt;autoencoders&gt;&lt;hyperparameter-optimization&gt;&lt;variational-autoencoder&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9709" PostTypeId="1" CreationDate="2018-12-27T17:30:21.010" Score="4" ViewCount="1215" Body="&lt;p&gt;&lt;strong&gt;Description&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have designed this robot in URDF format and its environment in pybullet. Each leg has a minimum and maximum value of movement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What reinforcement algorithm will be best to create a walking policy in a simple environment in which a positive reward will be given if it walks in the positive X-axis direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am working in the following but I don´t know if it is the best way:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expected output from the policy is an array in the range of (-1, 1) for each joint. The input of the policy is the position of each joint from the past X frames in the environment(replay memory like DeepQ Net), the center of mass of the body, the difference in height between the floor and the body to see if it has fallen and the movement in the x-axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Limitations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;left_front_joint      =&gt; lower=&quot;-0.4&quot; upper=&quot;2.5&quot; id=0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;left_front_leg_joint  =&gt; lower=&quot;-0.6&quot; upper=&quot;0.7&quot; id=2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;right_front_joint     =&gt; lower=&quot;-2.5&quot; upper=&quot;0.4&quot; id=3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;right_front_leg_joint =&gt; lower=&quot;-0.6&quot; upper=&quot;0.7&quot; id=5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;left_back_joint       =&gt; lower=&quot;-2.5&quot; upper=&quot;0.4&quot; id=6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;left_back_leg_joint   =&gt; lower=&quot;-0.6&quot; upper=&quot;0.7&quot; id=8&lt;/p&gt;&#xA;&#xA;&lt;p&gt;right_back_joint      =&gt; lower=&quot;-0.4&quot; upper=&quot;2.5&quot; id=9&lt;/p&gt;&#xA;&#xA;&lt;p&gt;right_back_leg_joint  =&gt; lower=&quot;-0.6&quot; upper=&quot;0.7&quot; id=11&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below is just a test of the environment with a set of movements hardcoded in the robot just to test how it could walk later. The environment is set to real time, but I assume it needs to be in a frame by frame lapse during the policy training. (&lt;em&gt;p.setRealTimeSimulation(1) #disable and p.stepSimulation() #enable&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A video of it can be seen in:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/j9sysG-EIkQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/j9sysG-EIkQ&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The complete code can be seen here:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/rubencg195/WalkingSpider_OpenAI_PyBullet_ROS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rubencg195/WalkingSpider_OpenAI_PyBullet_ROS&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CODE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pybullet as p&#xA;import time&#xA;import pybullet_data&#xA;&#xA;def moveLeg( robot=None, id=0, position=0, force=1.5  ):&#xA;    if(robot is None):&#xA;        return;&#xA;    p.setJointMotorControl2(&#xA;        robot,&#xA;        id,&#xA;        p.POSITION_CONTROL,&#xA;        targetPosition=position,&#xA;        force=force,&#xA;        #maxVelocity=5&#xA;    )&#xA;&#xA;pixelWidth = 1000&#xA;pixelHeight = 1000&#xA;camTargetPos = [0,0,0]&#xA;camDistance = 0.5&#xA;pitch = -10.0&#xA;roll=0&#xA;upAxisIndex = 2&#xA;yaw = 0&#xA;&#xA;physicsClient = p.connect(p.GUI)#or p.DIRECT for non-graphical version&#xA;p.setAdditionalSearchPath(pybullet_data.getDataPath()) #optionally&#xA;p.setGravity(0,0,-10)&#xA;viewMatrix = p.computeViewMatrixFromYawPitchRoll(camTargetPos, camDistance, yaw, pitch, roll, upAxisIndex)&#xA;planeId = p.loadURDF(&quot;plane.urdf&quot;)&#xA;cubeStartPos = [0,0,0.05]&#xA;cubeStartOrientation = p.getQuaternionFromEuler([0,0,0])&#xA;#boxId = p.loadURDF(&quot;r2d2.urdf&quot;,cubeStartPos, cubeStartOrientation)&#xA;boxId = p.loadURDF(&quot;src/spider.xml&quot;,cubeStartPos, cubeStartOrientation)&#xA;# boxId = p.loadURDF(&quot;spider_simple.urdf&quot;,cubeStartPos, cubeStartOrientation)&#xA;&#xA;&#xA;&#xA;toggle = 1&#xA;&#xA;&#xA;&#xA;p.setRealTimeSimulation(1)&#xA;&#xA;for i in range (10000):&#xA;    #p.stepSimulation()&#xA;&#xA;&#xA;    moveLeg( robot=boxId, id=0,  position= toggle * -2 ) #LEFT_FRONT&#xA;    moveLeg( robot=boxId, id=2,  position= toggle * -2 ) #LEFT_FRONT&#xA;&#xA;    moveLeg( robot=boxId, id=3,  position= toggle * -2 ) #RIGHT_FRONT&#xA;    moveLeg( robot=boxId, id=5,  position= toggle *  2 ) #RIGHT_FRONT&#xA;&#xA;    moveLeg( robot=boxId, id=6,  position= toggle *  2 ) #LEFT_BACK&#xA;    moveLeg( robot=boxId, id=8,  position= toggle * -2 ) #LEFT_BACK&#xA;&#xA;    moveLeg( robot=boxId, id=9,  position= toggle *  2 ) #RIGHT_BACK&#xA;    moveLeg( robot=boxId, id=11, position= toggle *  2 ) #RIGHT_BACK&#xA;    #time.sleep(1./140.)g&#xA;    #time.sleep(0.01)&#xA;    time.sleep(1)&#xA;&#xA;    toggle = toggle * -1&#xA;&#xA;    #viewMatrix        = p.computeViewMatrixFromYawPitchRoll(camTargetPos, camDistance, yaw, pitch, roll, upAxisIndex)&#xA;    #projectionMatrix  = [1.0825318098068237, 0.0, 0.0, 0.0, 0.0, 1.732050895690918, 0.0, 0.0, 0.0, 0.0, -1.0002000331878662, -1.0, 0.0, 0.0, -0.020002000033855438, 0.0]&#xA;    #img_arr = p.getCameraImage(pixelWidth, pixelHeight, viewMatrix=viewMatrix, projectionMatrix=projectionMatrix, shadow=1,lightDirection=[1,1,1])&#xA;&#xA;cubePos, cubeOrn = p.getBasePositionAndOrientation(boxId)&#xA;print(cubePos,cubeOrn)&#xA;p.disconnect()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://github.com/rubencg195/WalkingSpider_OpenAI_PyBullet_ROS/raw/master/images/PyBullet.png&quot; alt=&quot;robot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://github.com/rubencg195/WalkingSpider_OpenAI_PyBullet_ROS/raw/master/images/spider(8).jpeg&quot; alt=&quot;robot&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="20979" LastEditorUserId="20979" LastEditDate="2018-12-30T03:54:02.900" LastActivityDate="2021-04-20T19:56:56.963" Title="How to implement a Continuous Control of a quadruped robot with Deep Reinforcement Learning in Pybullet and OpenAI Gym?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;unsupervised-learning&gt;&lt;open-ai&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9711" PostTypeId="2" ParentId="9545" CreationDate="2018-12-27T18:51:09.460" Score="0" Body="&lt;p&gt;Use of CNNs to recognize digits is a reasonable approach as of this answer's writing, the effectiveness of which can be enhanced via &lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F11503415_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Sensitive Error Correcting Output Codes&lt;/em&gt;&lt;/a&gt;, (2005, John Langford, Alina Beygelzimer) according to Shuo Yang et. al. in &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/viewFile/9773/9821&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep Representation Learning with Target Coding&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a sufficient CNN depth, it can be trained to recognize the digits with the field name and underscore intact. Automatically removing them would generally only be possible after they are recognized using the CNN approach anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isolating the digits from the field name and the underscore would be accomplished via the same approach used to isolate the digits from one another. There is no reason to perform these two conceptual tasks in series and consume additional development time and resources, when a single deep CNN can locate the digits and dismiss the field name and underscore. This is similar to animal vision. A mosquito's visual pathway distinguishes an oncoming object from the background to avoid a swat using the same network it uses to recognize the object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be helpful would be to normalize the input by finding a two dimensional plateau (much like a geological one) representing the distribution of black pixels in horizontal and vertical dimensions. Summing rows and columns of pixels and using a heuristic algorithm to find a rectangle outside of which there is only noise may be sufficient. Then trim and scale, which removes redundancy and may improve CNN training speed, accuracy, and/or reliability by distributing the image more widely over the input layer and likely reducing the layer count by at least one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to use overall features of the form in which the field resides to rotate, position, and scale the image so that the field name and underscore is within a fraction of a pixel from an expected location on the form. In this case training a separate CNN is less redundant and has a higher return on development investment. In such a design, the field name and underscore can be removed by subtraction, however this may disturb field value recognition because the handwriting may overlap items blanked. It will require experimentation to determine the accuracy and reliability hit (diminution) from such disturbance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum in Response to Comment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first approach, a Field Name and Underscore Superimposer will need to be designed and implemented to transform the training example set features, the pixel arrays, but preserve the labels corresponding to each example. The hyper-parameter values that work best without the superimposition may need modification to work best with it.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-08T12:02:00.553" LastActivityDate="2019-01-08T12:02:00.553" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9713" PostTypeId="2" ParentId="9709" CreationDate="2018-12-27T20:30:53.383" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www-anw.cs.umass.edu/~barto/courses/cs687/Kohl-Stone-04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www-anw.cs.umass.edu/~barto/courses/cs687/Kohl-Stone-04.pdf&lt;/a&gt;&lt;br&gt;&#xA;Here's a paper about quadrupedal Aibo robots learning how to walk. They first hand tuned  parameters for walking and then used policy gradient method to 'polish' the parameters.  Maybe you can get some ideas from their work.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2018-12-27T20:30:53.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9716" PostTypeId="2" ParentId="9588" CreationDate="2018-12-27T23:19:10.430" Score="1" Body="&lt;p&gt;The training examples are not mentioned, but, assuming the assignment involves supervised learning, normalization is, in this case, mapping the inputs and outputs to binary values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Translation, in this simplified context, is a map of strings to strings, which isn't really an AI device by contemporary standards. Since an artificial network is given as a requirement for the solution, the goal is to train the network to act like a hash map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Big\{..., \;&#xA;\big(&#xA;[\text{&quot;beer&quot;, &quot;birra&quot;, &quot;cervexa&quot;}]:&#xA;\text{&quot;pivo&quot;}&#xA;\big)&#xA;, \; ...\Big\} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Downarrow $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Big\{..., \; \big(\text{&quot;beer&quot; : &quot;pivo&quot;}), (\text{&quot;birra&quot; : &quot;pivo&quot;}), (\text{&quot;cervexa&quot; : &quot;pivo&quot;}\big), \; ...\Big\} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming there is a list of pairs &lt;span class=&quot;math-container&quot;&gt;$(\mathcal{X}, \mathcal{Y})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is the word in any of the source languages and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; is the word in the target language, and the vocabulary of both is known to have word counts of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; respectively, one can determine the input and output layer widths (cell counts) &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{\ell}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; is the number of network layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w_1 = \text{ceiling} ( \log_2{X} )$$&lt;/span&gt;&#xA; &lt;span class=&quot;math-container&quot;&gt;$$w_{\ell} = \text{ceiling} ( \log_2{Y} )$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assignment of source words to binary values is arbitrary. You can sort them by language and then by whatever common character encoding used (such as UTF-8) just for debugging purposes. The same is true of target words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple MLP (multi-layer perceptron) with mini-batch training using basic propagation map be appropriate.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-27T23:19:10.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9717" PostTypeId="2" ParentId="9610" CreationDate="2018-12-28T00:18:30.347" Score="1" Body="&lt;p&gt;&lt;strong&gt;Solution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In learning systems, the output of learning is a set of parameters. These parameters can be a tensor in the case of a CNN, a representation of a directed graph in the case of recursive network designs, or any other structure holding the results of training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general case of compactly storing the result of training involves four process elements. These elements can be combined into one algorithm if needed.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Encoding of ordinals&lt;/li&gt;&#xA;&lt;li&gt;Lossless conversion of IEEE floats&lt;/li&gt;&#xA;&lt;li&gt;Serialization of the structure&lt;/li&gt;&#xA;&lt;li&gt;Lossless compression of the stream&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Upon retrieval, the elements are reversed.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decompression of the stream&lt;/li&gt;&#xA;&lt;li&gt;Parsing to reassemble structure&lt;/li&gt;&#xA;&lt;li&gt;Expansion of IEEE float representations&lt;/li&gt;&#xA;&lt;li&gt;Decoding of ordinals&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The encoding of ordinals requires the mapping of values to the minimal number of bytes or, most optimally, bits. The mapping must be reversible, meaning that there can be no information loss, so that decoding is completely reliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the encoding and serialization packs the information tightly, there may be little gain with a general compression algorithm such as LZ4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notions of General Intelligence&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before devoting much time on developing humanoid AGI and modeling cognition and states of mind, it may be useful to curb expectations away from trendy ideas. There is much conflict between the naive conceptions of general intelligence and the realities of neurological systems in animals and people, which are far more sophisticated than implied in the trendy media hype. The most important considerations are these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The complexity of networks in the brain&lt;/li&gt;&#xA;&lt;li&gt;That there is a significant varieties of neurons in brains&lt;/li&gt;&#xA;&lt;li&gt;The impact of neuroplasticity&lt;/li&gt;&#xA;&lt;li&gt;Complexity of chemical signaling independent from nerves&lt;/li&gt;&#xA;&lt;li&gt;Whether intelligence is in fact multi-dimensional&lt;/li&gt;&#xA;&lt;li&gt;Large number of genes that correlate with intelligence&lt;/li&gt;&#xA;&lt;li&gt;The academic slant of concepts regarding intelligence&lt;/li&gt;&#xA;&lt;li&gt;Difference between brain signal topology and computing models &lt;/li&gt;&#xA;&lt;li&gt;Data-centricity&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are many articles on the web on these topics, along with critiques of concepts such as the g-factor and the idea of general problem solving and general intelligence. There are several questions here on this topic.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/9405/why-did-artificial-intelligence-projects-fail&quot;&gt;Why did Artificial Intelligence projects fail?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/41/can-the-iq-of-an-ai-program-be-measured&quot;&gt;Can the IQ of an AI program be measured?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/8219/can-a-brain-be-intelligent-without-a-body&quot;&gt;Can a brain be intelligent without a body?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7337/is-the-singularity-concept-mathematically-flawed&quot;&gt;Is the singularity concept mathematically flawed?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/4987/is-there-a-ceiling-to-intelligence-itself/5045#5045&quot;&gt;Is there a ceiling to intelligence itself?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7573/self-conscious-artifical-intelligence/7576#7576&quot;&gt;Self-conscious artifical intelligence&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/15/is-the-turing-test-or-any-of-its-variants-a-reliable-test-of-artificial-intell&quot;&gt;Is the Turing Test, or any of its variants, a reliable test of artificial intelligence?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is no value in discouraging curiosity and serious research into understanding the nature of intelligence or efforts to expanding the boundaries of computing. The above comments are to assist in bringing the starting point of such inquiry into the realm of enthusiasm anchored by scientific methodology. Conjecture makes for great movie plots, but does not further science and does not lead to real world technology products and services. &lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-28T00:18:30.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9718" PostTypeId="2" ParentId="9682" CreationDate="2018-12-28T01:11:06.487" Score="1" Body="&lt;p&gt;Applying a ReLU activation function to a non-negative number is the same as applying the identity activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you may wish to do is apply &lt;span class=&quot;math-container&quot;&gt;$\ln(|\vec{x}|)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\arctan(x[0], x[1])$&lt;/span&gt;, ..., &lt;span class=&quot;math-container&quot;&gt;$\arctan(x[0], x[N-1])$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of dimensions to &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt;, and pass this directly into a parameter matrix multiplication (which involves deliberately using an identity activation in the input layer in some frameworks). This typically improves speed, accuracy, and reliability of convergence when the magnitudes exhibit a distribution close to an exponential of Gaussian and the directional components are near Gaussian when expressed in radians.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-28T01:11:06.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9720" PostTypeId="2" ParentId="7935" CreationDate="2018-12-28T03:22:39.357" Score="0" Body="&lt;p&gt;The Python language dynamically extensible like most other languages, so there is no reason why NEAT functionality could not be flexibly implemented in Python. Since the trend is to move computationally intensive processes to USB-3 or uPCIe acceleration or GPU cores, it is more likely the computational graphs will be uploaded to hardware along with training data to maximize throughput. Even in the absence of hardware acceleration, scripting languages generally delegate high computational burdens to tuned and optimized C/C++ dynamic libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python frameworks such as TensorFlow or PyTorch and Java frameworks such as DL4J would then call the C++ or ncpp code that handles the device opening, upload, and download of results. All these high level frameworks and the languages they are written in can handle new lower level computational components accessible through new APIs and flexible interface classes that support variable breadths and structure in data streams.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-12-28T03:22:39.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9721" PostTypeId="2" ParentId="5457" CreationDate="2018-12-28T03:49:21.717" Score="1" Body="&lt;p&gt;You may be mixing the concept of features with data granularity. Scan data after post processing is usually a cube of discrete density averages indexed by the three dimensions &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; corresponding to the standard directional references used in medicine. Features refer to what is extracted from those density readings, indicating concentrated angiogenesis or other indicators of concurrent failed apoptosis and t-cell activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; regions of potentially cancerous mass, there may only be &lt;span class=&quot;math-container&quot;&gt;$9n$&lt;/span&gt; features. Each of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; regions may have features such as this list.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;x center&lt;/li&gt;&#xA;&lt;li&gt;y center&lt;/li&gt;&#xA;&lt;li&gt;z center&lt;/li&gt;&#xA;&lt;li&gt;x width&lt;/li&gt;&#xA;&lt;li&gt;y width&lt;/li&gt;&#xA;&lt;li&gt;z width&lt;/li&gt;&#xA;&lt;li&gt;max density&lt;/li&gt;&#xA;&lt;li&gt;margin distance&lt;/li&gt;&#xA;&lt;li&gt;margin entropy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If there are two types of scans, there would be a flag to differentiate an MRI density from PET with contrast density, for example.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2018-12-28T03:49:21.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9724" PostTypeId="2" ParentId="9701" CreationDate="2018-12-28T10:50:01.920" Score="3" Body="&lt;p&gt;Combining two different fully trained neural networks is not only feasible, it is commonly done. Let's look at the example given as two concepts involving integers, &lt;span class=&quot;math-container&quot;&gt;$C_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_m$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ C_a: \mathcal{Y} = f_a (\mathcal{X}) = x_0 + x_1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ C_m: \mathcal{Y} = f_m (\mathcal{X}) = x_0 \, x_1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now let's define a palette of operations, including these two binary operations, that can be used to construct, a concept &lt;span class=&quot;math-container&quot;&gt;$C_e$&lt;/span&gt;, an expression comprised of an arbitrary hierarchy of addition, multiplication, constants, and substitution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ C_e: \mathcal{Y} = f_e (\mathcal{X}) \; \text{, where} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f_e \in \{f_a, f_b\} \; \land \; i \in \{0, 1\} \; \land \; ( \, x_i \in \mathbb{I} \; \lor \; x_i \in \mathcal{Y} \, ) \; \text{.} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, one artificial network can be trained to approximate &lt;span class=&quot;math-container&quot;&gt;$f_a$&lt;/span&gt; within a concept class &lt;span class=&quot;math-container&quot;&gt;$\mathbb{C}$&lt;/span&gt; of which &lt;span class=&quot;math-container&quot;&gt;$C_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_m$&lt;/span&gt; are members, using labeled examples of correct integer additions and another artificial network can be trained to approximate &lt;span class=&quot;math-container&quot;&gt;$f_b$&lt;/span&gt; within that same concept class, using labeled examples of correct integer multiplications.&lt;/p&gt;&#xA;&lt;p&gt;An expression involving both can be trained to approximate arbitrary product of sums or sum of products under specific conditions. It's unclear though if this is what you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;Normally, one wouldn't train a network to perform operations that are already known. Training is normally used to model operations that are not known.&lt;/p&gt;&#xA;&lt;p&gt;(Bilinear Convolutional Neural Networks (B-CNNs), introduced in &lt;a href=&quot;https://arxiv.org/pdf/1504.07889.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Bilinear CNNs for Fine-grained Visual Recognition&lt;/em&gt;&lt;/a&gt;, 2017, Tsung-Yu Lin, Aruni RoyChowdhury, Subhransu Maji, is an approach to using two CNNs in conjunction to provide two fine and course visual recognition in the same way the human visual system can have a dual awareness of detail and panorama. B-CNNs probably don't apply to the scenario given in the question.)&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-06-21T13:27:54.943" LastActivityDate="2020-06-21T13:27:54.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9725" PostTypeId="1" CreationDate="2018-12-28T12:53:21.973" Score="5" ViewCount="171" Body="&lt;p&gt;I've inherited a neural network project at the company I work for. The person who developed gave me some very basic training to get up and running. I've maintained it for a while. The current neural network is able to classify messages for telcos: it can send them to support people in different areas, like &quot;activation&quot;, &quot;no signal&quot;, &quot;internet&quot;, etc. The network has been working flawlessly. The structure of this neural network is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; model = Sequential()&#xA; model.add(Dense(500, input_shape=(len(train_x[0]),)))&#xA; model.add(Activation('relu'))&#xA; model.add(Dropout(0.6))&#xA; model.add(Dense(250, input_shape=(500,)))&#xA; model.add(Activation('relu'))&#xA; model.add(Dropout(0.5))&#xA; model.add(Dense(len(train_y[0])))&#xA; model.add(Activation('softmax'))&#xA; model.compile(loss='categorical_crossentropy',&#xA;                  optimizer='Adamax',&#xA;                  metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This uses a Word2Vec embedding, and has been trained with a &quot;clean&quot; file: all special characters and numbers are removed from both the training file and the input data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I've been assigned to make a neural network to detect if a message will be catalog as &quot;moderated&quot; (meaning it's an insult, spam, or just people commenting on a facebook post), or &quot;operative&quot;, meaning the message is actually a question for the company. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did was start from the current model and reduce the number of categories to two. It didn't go very well: the word embedding was in spanish from Argentina, and the training data was spanish from Peru. I made a new embedding and accuracy increased by a fair margin (we are looking for insults and other curse words. In spanish a curse word from a country can be a normal word for another: in Spain &quot;coger&quot; means &quot;to take&quot;, and in Argentina it means &quot;to f__k&quot;. &quot;concha&quot; means shell in most countries, but in Argentina it means &quot;c__t&quot;. You get the idea).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trained the network with 300.000 messages. Roughly 40% of these were classified as &quot;moderated&quot;. I tried all sorts of combinations of cycles and epochs. The accuracy slowly increased to nearly 0.9, and loss stays around 0.5000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when testing the neural network, &quot;operative&quot; messages generally seem to be correctly classified, with accuracy around 0.9, but &quot;moderated&quot; messages aren't. They are classified around 0.6 or less. At some point I tried multiple insults in a message (even pasting sample data as input data), but it didn't seem to improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Word2Vec works fantastically. The words are correctly &quot;lumped&quot; together (learned a few insults in Peruvian spanish thanks to it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put the neural network in production for a week, to gather statistics. Basically 90% of the messages went unclassified, and 5% were correctly classified and 5% wrong. Since the network has two categories, this seems to mean the neural network is just giving random guesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it possible to accomplish this task with a neural network?&lt;/li&gt;&#xA;&lt;li&gt;Is the structure of this neural network correct for this task?&lt;/li&gt;&#xA;&lt;li&gt;Are 300k messages enough to train the neural network?&lt;/li&gt;&#xA;&lt;li&gt;Do I need to clean up the data from uppercase, special characters, numbers etc?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17272" LastActivityDate="2019-01-04T00:04:56.810" Title="Neural network to detect &quot;spam&quot;?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-06T23:55:13.123" ContentLicense="CC BY-SA 4.0" />
  <row Id="9726" PostTypeId="2" ParentId="9725" CreationDate="2018-12-28T13:49:07.960" Score="2" Body="&lt;p&gt;First your questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, it is possible to accomplish this with a neural network, this is actually very similar to your current working model (the idea is the same, just different classes). So, there are no real reasons to start changing the architecture, especially if from what I understand, you are not a deep-learning expert. I mean, I'm sure that you can poke and play with the model to improve it, but that is not your current concern. Right now it seems that your model isn't training properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do think that 300k messages should be enough at least for a decent model (way better than random choice).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some standard preprocessing steps that you should perform on your input text (if you haven't already), in general you should normalize (convert to lower case) and remove noise from your text (like special characters). You can read more about different methods in the following link:&#xA;&lt;a href=&quot;https://medium.com/@datamonsters/text-preprocessing-in-python-steps-tools-and-examples-bf025f872908&quot; rel=&quot;nofollow noreferrer&quot;&gt;Text Preprocessing in Python: Steps, Tools, and Examples&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestions and questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your initial explanation, your new model has 2 outputs (moderated and operative), however later you mention that 90% of the input was not classified. So do you have a third class (no-classification) or are you just using some threshold of probability, that only above it the massage is classified? Because if you only have 2 classes, I would suggest to use a binary cross-entropy loss and not categorical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a strong suspicion that there is something wrong with either your training dataset or process. Please provide more info about the dataset (number of samples from each class, how was it labeled) and your training process (are you using part of the data for validation? how many epochs? etc.).&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2018-12-28T13:49:07.960" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9727" PostTypeId="2" ParentId="9708" CreationDate="2018-12-28T16:14:25.487" Score="4" Body="&lt;p&gt;The number of dimensions is a hyperparameter of your model, and you should do a hyperparameter search, like with any other parameters. There's also a tradeoff between dimension and training speed, so it should be small enough to be trainable in a reasonable time.&lt;/p&gt;&#xA;" OwnerUserId="18074" LastEditorUserId="2444" LastEditDate="2019-12-26T15:31:10.470" LastActivityDate="2019-12-26T15:31:10.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9728" PostTypeId="2" ParentId="9527" CreationDate="2018-12-28T23:00:17.277" Score="1" Body="&lt;p&gt;Use of reinforcement learning makes most sense over LSTM if control parameters can change during brewing to optimize results, which is not standard brewing policy in low end brewers but more precise control of brewing may produce superior results, especially with regard to temperature control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy does not necessarily need to be predisposed to brewing, if the search is guided by a deep artificial network (DNN), with parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; randomly initialized and defined as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ (p, v) = f_{\theta}(s) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The brewing process parameters comprise state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The vector of control change probabilities &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; have components &lt;span class=&quot;math-container&quot;&gt;$p_a = \Pr(a|s)$&lt;/span&gt; for each action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. A scalar &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; estimates the value of the expected outcome &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; from control state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, where $v \approx \mathbb{E} [o|s].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than characterize the variations in brewing stock as disturbance &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, it is more consistent with most policy learning approaches to consider the measured properties of brewing stock as features &lt;span class=&quot;math-container&quot;&gt;$\vec{b}$&lt;/span&gt; and feed a normalized signal &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(b)$&lt;/span&gt; into additional inputs to the deep network. This way, sensitivity to variations in &lt;span class=&quot;math-container&quot;&gt;$\vec{b}$&lt;/span&gt; is learned as a critical factor in brewing policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such a case, a vanilla MCTS (Monte-Carlo tree search algorithm) can be employed. Using a deep network removes the need to extract domain specific knowledge or discover heuristics to effectively guide control change policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Control values must be made discrete, based on granularity that fully (but not excessively) covers brewing possibilities. The control state probabilities that guide the search can be learned using coffee drinker feedback collected in any number of ways, depending the relative costs of each form of outcome data acquisition.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A focus group&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A group of independent testers&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repurchase statistics at a selected set of coffee shops&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Feedback controls at a set of offices that provide employees with coffee&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the comprehensiveness in control response to variations in brewing stock properties, the distribution of component values in &lt;span class=&quot;math-container&quot;&gt;$\vec{b}$&lt;/span&gt; and their statistical correlation over time may need to be controlled during the training process such that trends and distributions resemble likely real world variations in brewing stock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each search during the training process is guided by MCTS control changes, &lt;span class=&quot;math-container&quot;&gt;$a_t ∼ \pi_t$&lt;/span&gt;. The final state at the end of training is &lt;span class=&quot;math-container&quot;&gt;$s_T$&lt;/span&gt;, rated using a fixed scale of taster feedback. Selection of each state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; can be based on a control operation search advantage based on three characteristics.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Low past experience (a low count of previous visits in the search)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;High probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;High average of values &lt;span class=&quot;math-container&quot;&gt;$\{v, \; ...\}$&lt;/span&gt; over the leaf states of brewing trials that selected &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; according to the converging neural network parameters &lt;span class=&quot;math-container&quot;&gt;$f_{\theta}$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output of the search is &lt;span class=&quot;math-container&quot;&gt;$\vec{\pi}$&lt;/span&gt; representing a probability distribution over control changes. The deep network parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; converge via gradient descent using a loss function &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \ell = {(o − v)}^2 − {\pi}_T \, \ln{p} + c \, {||\theta||}^2 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loss is thus an aggregation of two optimality criteria.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Mean square estimated disparity between the predicted outcome &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt;t and the taste outcome &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cross-entropy of policy vector &lt;span class=&quot;math-container&quot;&gt;$p_t$&lt;/span&gt; and search probabilities &lt;span class=&quot;math-container&quot;&gt;${\pi}_t$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hyper-parameter &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; weighted L2 regularization&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The updated parameters are used in subsequent brew and taste trials. All control operations must be included in each trial. Success using this approach has been obtained using either greedy or proportionality with respect to the root visit counts.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-28T23:17:11.170" LastActivityDate="2018-12-28T23:17:11.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9730" PostTypeId="2" ParentId="9557" CreationDate="2018-12-29T02:57:04.063" Score="1" Body="&lt;p&gt;&lt;strong&gt;Bidirectional Mapping of Arbitrary Functions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interchange between what properties are considered labels and what properties are considered features can support bidirectional map learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unidirectional training represents the objective of functional approximation &lt;span class=&quot;math-container&quot;&gt;$f_p$&lt;/span&gt; in terms of features &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; and labels &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; through convergence of parameters &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, ideally toward an accurate and reliable approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{Y} \Leftarrow f_p(\mathcal{X}) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because only a small proportion of arbitrary functions are monotonic (have only one value in the domain for any given value in the range), the existence of an inversion of &lt;span class=&quot;math-container&quot;&gt;$f_p$&lt;/span&gt; is unlikely. This is certainly true of natural language processing functions. This fact will be further illuminated below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bidirectional training of a non-monotonic translation function &lt;span class=&quot;math-container&quot;&gt;$f_p$&lt;/span&gt; and a reverse translation &lt;span class=&quot;math-container&quot;&gt;$g_p$&lt;/span&gt; can be represented as two adjacent training objectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{Y} \Leftarrow f_p(\mathcal{X}) $$&lt;/span&gt;&#xA; &lt;span class=&quot;math-container&quot;&gt;$$ g_q(\mathcal{Y}) \Rightarrow \mathcal{X} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Function &lt;span class=&quot;math-container&quot;&gt;$g_q$&lt;/span&gt; is not a guaranteed inversion of &lt;span class=&quot;math-container&quot;&gt;$f_p$&lt;/span&gt; and may rarely be. Parameters resulting from convergence &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; may not be derivable from parameters resulting from convergence &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; or vice versa either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Temporal Bidirectionality to Minimize Round Trip Inaccuracy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reasons further illuminated below, temporal bidirectionality may be important in some use cases. For this reason, use two B-LSTM networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be some efficiency gains by training both B-LSTM networks in the same training phase, especially if more than one host, GPU, parallel signal processor, or AI accelerator is available at training time. The same is true of test and validation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ontology of network types is as follows.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RNNs (recurrent artificial networks) are a stateful extension of MLPs (multi-layer perceptrons).&lt;/li&gt;&#xA;&lt;li&gt;LSTM (long short term memory) networks are an extension of RNNs, and use gating within each cell to adaptively regulate retention of state and its dissipation.&lt;/li&gt;&#xA;&lt;li&gt;B-LSTMs provide bidirectionality in the temporal direction, which allows more variety in what causality and reactance can be modeled.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Importance of Contemporary Concepts in Linguistics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the natural-language-processing tag was included with the posting of the question and the term, &quot;Machine translation,&quot; was mentioned, it appears that bidirectional natural language translation using AI is the project goal. If that is the case, a contemporary linguist's understanding of words sequences in text and the sequence of phonetic elements in speech will help with developing an effective solution using AI. An engineer should be aware of the interplay between expectations placed on project results and technical feasibility at the current state of NLP technology.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Human translation experts rely on cognition to accurately represent text that may contain cultural references, analogies, colloquialisms, and other literary or oratory devices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Because of rhyme, alliteration, cadence and other phonetic devices, prose and poetry may not have an equivalent in the target language.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Linguistic elements are not strictly words. A single word may have multiple linguistic elements because of tenses, plurality, gender, prefixes, suffixes, and other modifiers. For example, &quot;Multicolored,&quot; is an adjective made up of three linguistic elements: {&quot;multi-&quot;, &quot;color&quot;, &quot;-ed&quot;}. A single idea is sometimes represented as multiple words, the individual meanings and juxtaposition of which does not imply the meaning of the word group. For example, &quot;I don't wish to hitch my apple-cart to that wobbly wagon,&quot; has nothing to do with apples, carts, mechanical stability, or wagons.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The translation &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A} \Rightarrow \mathcal{B} \Rightarrow \mathcal{C}$&lt;/span&gt; may not lead to &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A} = \mathcal{C}$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When a series of linguistic elements are communicated, the semantics represented are not often a series. For instance, &quot;Flying birds can also jump,&quot; is equivalent to, &quot;If a bird is able to fly, it can jump.&quot;  This works in natural language because the associations and modifying aspects between linguistic elements is a semantic network that correlates with a cognitive ontology. The serialization of this network and its reconstitution in another mind cannot be formalized in a way that eliminates all ambiguity. Communication &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}_o \Rightarrow \mathcal{L} \Rightarrow \mathcal{K}_r$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}$&lt;/span&gt; are cognitions at originating intelligence &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; and target intelligence &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; respectively and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; is a language expression in between, may not lead to &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}_t = \mathcal{K}_o$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the above reasons, formal grammar has been discarded as a set of principles that can facilitate NLP. Humans neither write nor speak in conformance with any formal grammar on a regular basis.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Expected Dominance of Recursive Network Technology&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, LSTM networks may not be the direction of NLP technology development. Recursive network designs (often using RNN as an acronym but referring to a distinct subset of recurrent networks) can train semantic network models based on serial streams of linguistic elements. That appears to bear closest resemblance to language centers in the human mind, and may emerge as a dominant NLP component in emerging AI systems.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-29T02:57:04.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9731" PostTypeId="1" AcceptedAnswerId="9733" CreationDate="2018-12-29T03:26:38.570" Score="-1" ViewCount="46" Body="&lt;p&gt;For example, if I constructed a neural network and the computer running it where to be demolished, is the information/program of the neural network still an existent entity within or outside the remnants of the hardware?  &lt;/p&gt;&#xA;" OwnerUserId="21012" LastActivityDate="2021-04-12T07:29:55.083" Title="Does software remain even when hardware is demolished?" Tags="&lt;hardware&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9732" PostTypeId="2" ParentId="2201" CreationDate="2018-12-29T04:09:50.037" Score="0" Body="&lt;p&gt;The term &lt;em&gt;hidden nodes&lt;/em&gt; refers to the cells of inner layers of artificial networks are not exposed for connectivity outside of their connectivity within the network. Their values can be read and visualized, but the network function is not dependent upon such tapping of signals internal to the network. Neither are the input of inner layers and their &lt;em&gt;hidden cells&lt;/em&gt; connected to network inputs nor are the outputs of inner layers and their &lt;em&gt;hidden cells&lt;/em&gt; connected to network outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of inner layers is related to network flexibility and thus accuracy of the functional approximation that may be achieved. An activation function by itself rarely approximates the desired mapping between input values and output values that is optimally achieved by training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attenuated substitution of one parallel activation function into another leads to greater functional flexibility. Attenuation is achieved by multiplying the vector of layer activation function outputs by a matrix of parameters and feeding the resulting product vector into the next layer's activation function units. In combination with a convergence strategy like gradient descent and a corrective distribution strategy like back-propagation to update the parameters, the flexibility in functionality can be directed to achieve higher resulting accuracy by the end of training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no action layers or action cells. The cells in an artificial network do not perform actions other than the evaluation of their activation function (and gating in the case of gated units). The output of the network may lead to actions if the network is a component in a controller, in which case the output of the network is connected to other system components so that action is controlled.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-29T04:09:50.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9733" PostTypeId="2" ParentId="9731" CreationDate="2018-12-29T06:54:30.097" Score="3" Body="&lt;p&gt;Neural networks are just software. Software is just one form of data in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;Von Neumann Architecture&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Most of the data is stored on the disk. There are two common types of disks:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hard_disk_drive&quot; rel=&quot;nofollow noreferrer&quot;&gt;HDD&lt;/a&gt;: An actual disk which can be magnetified is spinning.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solid-state_drive&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD&lt;/a&gt;: Solid-state drive&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If the part where the data is stored is destroyed, the data is destroyed.&lt;/p&gt;&#xA;&lt;p&gt;Of course, the data can be loaded into memory, but the principle is the same: If the part where the data is stored is destroyed, the data is destroyed.&lt;/p&gt;&#xA;&lt;p&gt;Data can also be partially be destroyed / corrupted. If you have systems like &lt;a href=&quot;https://en.wikipedia.org/wiki/RAID&quot; rel=&quot;nofollow noreferrer&quot;&gt;RAID&lt;/a&gt; or use &lt;a href=&quot;https://en.wikipedia.org/wiki/Error_correction_code&quot; rel=&quot;nofollow noreferrer&quot;&gt;error correcting codes&lt;/a&gt; then you can defend youself against data loss.&lt;/p&gt;&#xA;" OwnerUserId="3217" LastEditorUserId="32410" LastEditDate="2021-04-12T07:29:55.083" LastActivityDate="2021-04-12T07:29:55.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9734" PostTypeId="2" ParentId="9637" CreationDate="2018-12-29T07:38:41.057" Score="1" Body="&lt;p&gt;It is correct that to eliminate the false positives, the training must include images that reflect true negative cases. The objective of training is not this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Find the objects of class &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; in scene &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Find any objects of class &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; in scene &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Training is effective when sample &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is drawn according to the same distribution as will be used for drawing at test time or during actual use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not necessary to annotate the background if the loss function is chosen so that finding nothing when nothing there is indicated as a zero loss and a proper distribution of those cases exist in the training sample.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-29T07:38:41.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9736" PostTypeId="2" ParentId="5457" CreationDate="2018-12-29T08:07:10.040" Score="0" Body="&lt;p&gt;I think maybe you can use this GLCM matrix code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cv2&#xA;import numpy as np&#xA;import sys&#xA;import math&#xA;&#xA;class GLCM:&#xA;def __init__(self, image, dy, dx):&#xA;    self.image = image&#xA;    self.dy = dy&#xA;    self.dx = dx&#xA;    self.glcm = self.GLCMcount()&#xA;    self.kontras, self.meanI, self.meanJ, self.energy, self.homogenity = self.contrast()&#xA;    self.taoI, self.taoJ = self.tao()&#xA;    self.korelasion = self.correlation()&#xA;&#xA;def GLCMcount(self):&#xA;    height, width = self.image.shape[:2]&#xA;    glcm = np.zeros((256, 256, 3), np.double)&#xA;    x = 0&#xA;    for i in range(height):&#xA;        for j in range(width):&#xA;            if i + self.dy in range(height) and j + self.dx in range(width):&#xA;                glcm[self.image[i, j, 0], self.image[i +&#xA;                                                     self.dy, j + self.dx, 0], 0] += 1&#xA;                glcm[self.image[i, j, 1], self.image[i +&#xA;                                                     self.dy, j + self.dx, 1], 1] += 1&#xA;                glcm[self.image[i, j, 2], self.image[i +&#xA;                                                     self.dy, j + self.dx, 2], 2] += 1&#xA;                # print str(image[i,j,0]) + &quot; &quot; + str(image[i+dy,j+dx,0]) + &quot;&#xA;                # &quot; + str(glcm[image[i,j,0],image[i+dy,j+dx,0],0])&#xA;                x += 1&#xA;    glcm = glcm / x&#xA;    return glcm&#xA;&#xA;def contrast(self):&#xA;    contrast = np.zeros(3, np.float)&#xA;    meanI = np.zeros(3, np.float)&#xA;    meanJ = np.zeros(3, np.float)&#xA;    energy = np.zeros(3, np.float)&#xA;    homogenity = np.zeros(3, np.float)&#xA;    for i in range(256):&#xA;        for j in range(256):&#xA;            contrast[0] += pow(i - j, 2) * self.glcm[i, j, 0]&#xA;            contrast[1] += pow(i - j, 2) * self.glcm[i, j, 1]&#xA;            contrast[2] += pow(i - j, 2) * self.glcm[i, j, 2]&#xA;            meanI[0] += i * self.glcm[i, j, 0]&#xA;            meanI[1] += i * self.glcm[i, j, 1]&#xA;            meanI[2] += i * self.glcm[i, j, 2]&#xA;            meanJ[0] += j * self.glcm[i, j, 0]&#xA;            meanJ[1] += j * self.glcm[i, j, 1]&#xA;            meanJ[2] += j * self.glcm[i, j, 2]&#xA;            energy[0] += pow(self.glcm[i, j, 0], 2)&#xA;            energy[1] += pow(self.glcm[i, j, 1], 2)&#xA;            energy[2] += pow(self.glcm[i, j, 2], 2)&#xA;            homogenity[0] += self.glcm[i, j, 0] / (1 + abs(i - j))&#xA;            homogenity[1] += self.glcm[i, j, 1] / (1 + abs(i - j))&#xA;            homogenity[2] += self.glcm[i, j, 2] / (1 + abs(i - j))&#xA;    return contrast, meanI, meanJ, energy, homogenity&#xA;&#xA;def correlation(self):&#xA;    correlation = np.zeros(3, np.float)&#xA;    for i in range(256):&#xA;        for j in range(256):&#xA;            correlation[0] += ((i - self.meanI[0]) * (j - self.meanJ[0])&#xA;                               * self.glcm[i, j, 0]) / (self.taoI[0] * self.taoJ[0])&#xA;            correlation[1] += ((i - self.meanI[1]) * (j - self.meanJ[1])&#xA;                               * self.glcm[i, j, 1]) / (self.taoI[1] * self.taoJ[1])&#xA;            correlation[2] += ((i - self.meanI[2]) * (j - self.meanJ[2])&#xA;                               * self.glcm[i, j, 2]) / (self.taoI[2] * self.taoJ[2])&#xA;            # print correlation;&#xA;    return correlation&#xA;&#xA;def tao(self):&#xA;    taoI = np.zeros(3, np.float)&#xA;    taoJ = np.zeros(3, np.float)&#xA;    for i in range(256):&#xA;        for j in range(256):&#xA;            taoI[0] += pow(i - self.meanI[0], 2) * self.glcm[i, j, 0]&#xA;            taoI[1] += pow(i - self.meanI[1], 2) * self.glcm[i, j, 1]&#xA;            taoI[2] += pow(i - self.meanI[2], 2) * self.glcm[i, j, 2]&#xA;            taoJ[0] += pow(j - self.meanJ[0], 2) * self.glcm[i, j, 0]&#xA;            taoJ[1] += pow(j - self.meanJ[1], 2) * self.glcm[i, j, 1]&#xA;            taoJ[2] += pow(j - self.meanJ[2], 2) * self.glcm[i, j, 2]&#xA;            # print str(taoJ[2])+&quot; = &quot;+ str(pow(j-meanJ[2],2)) + &quot; x &quot; + str(glcm[i,j,2])&#xA;    # print &quot;taoi = &quot;&#xA;    # print taoI&#xA;    # print &quot;taoj = &quot;&#xA;    # print taoJ&#xA;    for i in range(3):&#xA;        taoI[i] = math.sqrt(taoI[i])&#xA;        taoJ[i] = math.sqrt(taoJ[i])&#xA;    return taoI, taoJ&#xA;&#xA;def printglcm(self):&#xA;&#xA;    print (&quot;meanI = &quot; + str(rgb2gs(self.meanI)))&#xA;    print (&quot;meanJ = &quot; + str(rgb2gs(self.meanJ)))&#xA;    print (&quot;taoI = &quot; + str(rgb2gs(self.taoI)))&#xA;    print (&quot;taoJ = &quot; + str(rgb2gs(self.taoJ)))&#xA;    print (&quot;kontras = &quot; + str(rgb2gs(self.kontras)))&#xA;    print (&quot;Energy = &quot; + str(rgb2gs(self.energy)))&#xA;    print (&quot;Homogenitas = &quot; + str(rgb2gs(self.homogenity)))&#xA;    print (&quot;Correlation = &quot; + str(rgb2gs(self.korelasion)))&#xA;&#xA;def writeglcm(self):&#xA;    with open(&quot;test.txt&quot;, &quot;a&quot;) as myfile:&#xA;        myfile.write(str(rgb2gs(self.kontras)) + &quot; &quot; + str(rgb2gs(self.energy)) + &quot; &quot; +&#xA;                     str(rgb2gs(self.homogenity)) + &quot; &quot; + str(rgb2gs(self.korelasion)) + &quot;\n&quot;)&#xA;    with open(&quot;type.txt&quot;, &quot;a&quot;) as myfile:&#xA;        myfile.write(str(1) + &quot;,\n&quot;)&#xA;&#xA;&#xA;def rgb2gs(rgb):&#xA;val = 0.114 * (rgb[0]) + 0.587 * (rgb[1]) + 0.299 * (rgb[2])&#xA;return val&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;image = cv2.imread(&quot;powder8.png&quot;)&#xA;glcm = GLCM(image, 0, 1)&#xA;imglcm = glcm.glcm.astype(np.uint8)&#xA;glcm.printglcm()&#xA;glcm.writeglcm()&#xA;&#xA;# kontras, meanI, meanJ, energy, homogenity = contrast(glcm)&#xA;# taoI, taoJ = tao(glcm, meanI, meanJ)&#xA;# korelasion = correlation(glcm, meanI, meanJ, taoI, taoJ)&#xA;# print &quot;meanI = &quot; + str(rgb2gs(glcm.meanI))&#xA;# print &quot;meanJ = &quot; + str(rgb2gs(glcm.meanJ))&#xA;# print &quot;taoI = &quot; + str(rgb2gs(glcm.taoI))&#xA;# print &quot;taoJ = &quot; + str(rgb2gs(glcm.taoJ))&#xA;# print &quot;kontras = &quot; + str(rgb2gs(glcm.kontras))&#xA;# print &quot;Energy = &quot; + str(rgb2gs(glcm.energy))&#xA;# print &quot;Homogenitas = &quot; + str(rgb2gs(glcm.homogenity))&#xA;# print &quot;Correlation = &quot; + str(rgb2gs(glcm.korelasion))&#xA;# print glcm&#xA;&#xA;# cv2.imshow(&quot;testing&quot;, imglcm)&#xA;# cv2.waitKey(0)&#xA;# cv2.destroyAllWindows()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gives you meanI,meanJ,taoI,taoJ,contrast,Energy,Homogeneity,Correlation.&lt;/p&gt;&#xA;" OwnerUserId="20991" LastEditorUserId="75" LastEditDate="2018-12-29T18:43:06.457" LastActivityDate="2018-12-29T18:43:06.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9737" PostTypeId="2" ParentId="9448" CreationDate="2018-12-29T08:34:42.700" Score="2" Body="&lt;p&gt;&lt;strong&gt;Examples of simulated annealing in the 2010s&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are a few examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1401.1084.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Optimised simulated annealing for Ising spin glasses&lt;/em&gt;&lt;/a&gt;, 2015, S.V. Isakov et. al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/profile/Chao_Wang213/publication/272753635_A_parallel_simulated_annealing_method_for_the_vehicle_routing_problem_with_simultaneous_pickup-delivery_and_time_windows/links/5ac6e5cea6fdcc8bfc7f86f9/A-parallel-simulated-annealing-method-for-the-vehicle-routing-problem-with-simultaneous-pickup-delivery-and-time-windows.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A parallel simulated annealing method for the vehicle routing problem with simultaneous pickup–delivery and time windows&lt;/em&gt;&lt;/a&gt;, 2014, Chao Wang et. al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1409.7186.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Feature-based tuning of simulated annealing applied to the curriculum-based course timetabling problem&lt;/em&gt;&lt;/a&gt;, 2015, Ruggero Bellio et. al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis of Quote in the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quote in the question is from page 125 of &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;, Stuart Russell and Peter Norvig, 2010, 3rd Edition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quote begins with the statement, &quot;A hill-climbing algorithm that never makes 'downhill' moves toward states with lower value (or higher cost) is guaranteed to be incomplete,&quot; which is not strictly true. The stated guarantee has at least two important exceptions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The class of hills being climbed have only one peak&lt;/li&gt;&#xA;&lt;li&gt;The search is executed using parallel computing architecture scaled such that search granularity includes no more than one peak per processing unit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Describing simulated annealing using the analogy of combining inverted hill climbing and random walk, as Stuart and Norvig explains it, is not as clear and technically accurate as the original analogy. It is often better to find original sources of ideas and read them than to read simplified explanations. The original ideas may be just as easy or easier to understand than the textbook paraphrases (which is why the best textbooks simply quote the originating authors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source of the Term&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original algorithm termed &lt;em&gt;simulated annealing&lt;/em&gt; is introduced in &lt;em&gt;Optimization by Simulated Annealing&lt;/em&gt;, Kirkpatrick et. al.&lt;sup&gt;1&lt;/sup&gt;, which may not qualify as one one explicitly employed by AI researchers or practitioners on a daily basis. Simulating the atomic motion of metals during annealing is the first of several approaches to injecting pseudo-random perturbations into convergence mechanisms. An approximation of a &lt;span class=&quot;math-container&quot;&gt;${\chi}^2$&lt;/span&gt; distribution with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; degrees of freedom is a more progressive injection source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These approaches share the common goal of avoiding the mistaking of a local minimum in loss surface as its global minimum. Such mistaken conclusion causes a failure to achieve an optimal learning state according to the criteria the loss function represents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As can be seen in the below excerpts, the original article is a good read and sets a foundation of understanding. Such a foundation will assist in understanding all methods used avoid local minima using stochastic injection.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Iterative improvement consists of a search in this coordinate space for&#xA;  rearrangement steps which lead downhill. Since this search usually gets stuck in a local but not a global optimum, it is customary to carry out the process several times, starting from different randomly generated configurations, and save the best result.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Metropolis et al &lt;sup&gt;2&lt;/sup&gt;, in the earliest days of scientific computing, introduced a simple algorithm that can be used to provide an efficient simulation of a collection of atoms in equilibrium at a given temperature.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Random numbers uniformly distributed in the interval (0,l) are a convenient means of implementing the random part of the algorithm.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Using the cost function in place of the energy and defining configurations by a set of parameters {&lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;}, it is straightforward with the Metropolis procedure to generate a population of configurations of a given optimization problem at some effective temperature. This temperature is simply a control parameter in the same units as the cost function. The simulated annealing process consists of first &quot;melting&quot; the system being optimized at a high effective temperature, then lowering the temperature by slow stages until the system &quot;freezes&quot; and no further changes occur.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Simulated annealing with Z-moves improved the random routing by 57 percent, averaging results for both x and y links.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.jstor.org/stable/1690046&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Optimization by Simulated Annealing&lt;/em&gt;&lt;/a&gt;, Science S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, Science New Series, Vol. 220, No. 4598, May 1983, pp. 671-680&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] N. Metropolis, A. Rosenbluth, M. Rosenbluth., A. Teller. E. Teller, J. Chem. Phys. 21. 1087 110511&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-29T20:12:10.363" LastActivityDate="2018-12-29T20:12:10.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9738" PostTypeId="1" CreationDate="2018-12-29T10:33:55.510" Score="2" ViewCount="513" Body="&lt;p&gt;One-shot learning seems to work really well in many application domains. Are there any major (or even minor) drawbacks of using one-shot learning? Does it have flaws that could prevent it from being used in certain image identification scenarios?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, I'm specifically referring to the Siamese Neural Network and Memory Augmented Neural Network approaches to one-shot learning.&lt;/p&gt;&#xA;" OwnerUserId="9432" LastEditorUserId="9432" LastEditDate="2018-12-30T01:56:33.103" LastActivityDate="2018-12-30T01:56:33.103" Title="What are some of the drawbacks of one-shot learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9739" PostTypeId="1" AcceptedAnswerId="9740" CreationDate="2018-12-27T07:50:32.890" Score="2" ViewCount="165" Body="&lt;p&gt;I have some problems with understanding of the batch concept and batch size. I messed something up. First I start it consider based on convolutional neural network I heard two versions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When the batch size is set to 50, the first network is fed with 50 images and then learned / recalculated (it doesn't make sense to me, because in this case the network learns one of 50 images).&lt;/li&gt;&#xA;&lt;li&gt;When the batch size is set to 50, one of 50 neurons is recalculated in the learning process on a single image.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Both of these explanations seem to be wrong to me, so I assume, that I completely don't understand this. What is batch / batch size in RNN? Could you show any example?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can tell you how I would teach a recurrent neural network. Let's say, that I would like to teach a neural network to predict the weather the next day:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I would take weather data from an expected area from the last 30,000 days.&lt;/li&gt;&#xA;&lt;li&gt;I would assume that my prediction would be based on measurements from the last 365 days.&lt;/li&gt;&#xA;&lt;li&gt;I would take data from day 1 to 365 - feed RNN with it and learn.&lt;/li&gt;&#xA;&lt;li&gt;Then I would take data from day 2 to 366 =&gt; feed + learn&lt;/li&gt;&#xA;&lt;li&gt;Then day 3 to 367 =&gt; feed + learn&lt;/li&gt;&#xA;&lt;li&gt;And so on.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is this 365 measurement concept a batch size?&lt;/p&gt;&#xA;" OwnerUserId="21171" OwnerDisplayName="ketzul" LastEditorUserId="16229" LastEditDate="2019-05-29T17:51:08.747" LastActivityDate="2019-05-29T17:51:08.747" Title="What is batch / batch size in neural networks?" Tags="&lt;python&gt;&lt;machine-learning&gt;&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9740" PostTypeId="2" ParentId="9739" CreationDate="2018-12-29T15:02:18.943" Score="1" Body="&lt;p&gt;You are confusing two concepts, batch size and window size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Batch Size&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept of a batch is related to your data and not the number of neurons.  A batch is just a subset of your training data.  The size of the batch affects computational speed, quality, and accuracy.  Large batch sizes will train faster than smaller ones but the model's accuracy can suffer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a rule of thumb that a batch size should be a power of two (e.g. 32, 64, 128, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking larger batch sizes do not generalize as well as smaller batch sizes.  You will need to experiment with the batch size to achieve optimal performance.  Batch size is considered a hyperparameter.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following video, &lt;a href=&quot;https://www.youtube.com/watch?v=U4WB9p6ODjM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Batch Size in a Neural Network explained&lt;/a&gt; should help clarify things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Window Size&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Window size applies to time series and sequences when using recurrent style networks.  For your case, 365 is the window size.  You did not mentioned how far in the future you are trying to predict.  If we assume that you are predicting one day in the future your output would be the 366th measurement.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Your Application&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A batch for you would be a number of training sets.  For example, if we used 32 as a batch size, you would have 32 sets of input windows and outputs.  The first two batches might look like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Batch 1, size 32&#xA;1 Data from days 1 to 365 as the input, day 366 as the output&#xA;2 Data from days 2 to 366 as the input, day 367 as the output&#xA;...&#xA;32 Data from days 32 to 396 as the input, day 397 as the output&#xA;&#xA;Batch 2, size 32&#xA;1 Data from days 33 to 397 as the input, day 398 as the output&#xA;2 Data from days 34 to 398 as the input, day 399 as the output&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2018-12-29T15:21:59.407" LastActivityDate="2018-12-29T15:21:59.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9743" PostTypeId="1" AcceptedAnswerId="9748" CreationDate="2018-12-29T18:27:48.933" Score="3" ViewCount="284" Body="&lt;p&gt;I've often heard MCTS grouped together with neural nets and machine learning. From what I gather, MCTS uses a refined intuition (from maching learning) to evaluate positions. This allows it to better guess which moves are worth playing out more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I've almost never heard of using machine learning for Minimax+alpha-beta engines. Couldn't machine learning be used for the Engine to better guess which move is best, and then look at that move's subtree first? A major optimization of the minimax algorithm is move-ordering, and this seems like a good way to accomplish that.&lt;/p&gt;&#xA;" OwnerUserId="16917" LastActivityDate="2018-12-30T01:59:31.923" Title="Why do neural nets and machine learning tend to work well with MCTS, but not with regular Minimax game-playing AI?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9745" PostTypeId="1" CreationDate="2018-12-29T19:16:02.120" Score="1" ViewCount="87" Body="&lt;p&gt;I have been reading a few papers (&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S003132030000114X&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper1&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1410.2474&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper2&lt;/a&gt;) on stereo matching using genetic algorithms. I understand how genetic algorithms work in general and how stereo matching works, but I do not understand how genetic algorithms are used in stereo matching. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first paper by Han et al says that &quot;1) individual is a disparity set, 2) a chromosome has a 2D structure for handling image signals efficiently, and 3) a fitness function is composed of certain constraints which are commonly used in stereo matching&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it mean that an individual is a disparity map with random numbers?&#xA;Then a chromosome is a block within the individual's disparity map. &#xA;The constraint used for fitness function could be the famous epipolar line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dont seem to understand how this works and even &lt;em&gt;WHY&lt;/em&gt; you should use genetic algorithm on an algorithm that at its simplest form uses 5 for loops, for example, like in &lt;a href=&quot;https://github.com/davechristian/Simple-SSD-Stereo/blob/master/stereomatch_SSD.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14863" LastActivityDate="2019-01-22T03:59:50.173" Title="Stereo matching using genetic algorithm" Tags="&lt;genetic-algorithms&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9746" PostTypeId="1" CreationDate="2018-12-29T23:34:04.103" Score="2" ViewCount="173" Body="&lt;p&gt;I am trying to find a good approach to create a computer player for the game &amp;quot;&lt;a href=&quot;https://gamious.com/lines/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lines&lt;/a&gt;&amp;quot; from gamious on Android. The concept of the game is pretty straightforward :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Lines is an abstract ‘zen’ game experience where form is just as important as function. Place or remove Dots to initiate a colourful race that fills a drawing. The colour that dominates the race wins.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The game starts with a drawing (that can be described as a set of &amp;quot;blank&amp;quot; lines, with connection to other lines). Dots of different colour are placed somewhat randomly on the lines. The player get a colour assigned. When the game start, paint start flowing from the dots and filling the (at first blank) lines of the drawing. You win if your colour dominates.&lt;/p&gt;&#xA;&lt;p&gt;The game gives you different tools to win (the game starts when all of them have been used) :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[0 to 2] scissor to cut lines&lt;/li&gt;&#xA;&lt;li&gt;[0 to 5] additional dot of your own color to place on the drawing&lt;/li&gt;&#xA;&lt;li&gt;[0 to 4] enemy dots eraser&lt;/li&gt;&#xA;&lt;li&gt;[0 to 3] additional straight lines to connect different part of the drawing&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A quick example: the first image is the initial state of a round. &amp;quot;My&amp;quot; colour is the yellow (1 enemy = brown) and I have 4 tools (2 eraser and 2 lines). The second image shows the game running after I used the tools to put my colour in a winning position (yes, we can do better)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Ptar.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Ptar.jpg&quot; alt=&quot;starting the game&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oVsG2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oVsG2.jpg&quot; alt=&quot;go yellow !&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I try to approach this as a &lt;em&gt;classical&lt;/em&gt; optimization problem, things get messy pretty fast :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;highly non-linear&lt;/li&gt;&#xA;&lt;li&gt;high number of dimensions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;AI seems to be the right way to go, but I would like your help to get in the right direction: &lt;em&gt;what would be your approach to create an AI to play this game?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;To limit the scope of this question, you can consider that I already have a data structure to represent the game initial state, the use of different tools and the game &amp;quot;physics&amp;quot;. &lt;strong&gt;What I really want to do is finding how to create an AI which can learn how to &lt;em&gt;efficiently&lt;/em&gt; use the tools.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Regarding my experience, I took 2 semesters of AI classes during the last year getting my engineering degree and have used non-linear optimization tools for a while: you can go technical, but I am not fully understand it.&lt;/p&gt;&#xA;" OwnerUserId="21020" LastEditorUserId="2444" LastEditDate="2020-10-26T20:55:01.420" LastActivityDate="2020-10-26T20:55:01.420" Title="What kind of AI technique can I use to play the &quot;Lines&quot; game?" Tags="&lt;game-ai&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9748" PostTypeId="2" ParentId="9743" CreationDate="2018-12-30T01:59:31.923" Score="1" Body="&lt;p&gt;It just does.&#xA;Take a look at &lt;a href=&quot;https://int8.io/monte-carlo-tree-search-beginners-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post explaining how MCTS works.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In both Alpha Go Lee and Alpha Zero the tree traversal follows the nodes that maximize the following UCT variant:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;UCT(v_i,v) = \frac{Q(v_i)}{N(v_i)} + cP(v_i, v)\sqrt{\frac{N(v)}{1+N(v_i)}}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where P(vi,v)&#xA;  is prior probability of the move (transition from v to vi ), its value comes from the output of deep neural network called Policy Network . Policy Network is a function that consumes game state and produce probability distribution over possible moves.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see the Policy Network(it is actually just one neural network for both the value and the policy) is used to guide the search tree. Not all possible moves are explored. Also during the learning phase the policy network uses the &quot;visit count&quot; of the MCTS Nodes to learn. Moves that were explored more are the better ones. The state-of-the-art chess engine Stockfish evaluates about 1000 times more positions per second than Alpha Zero. It relies on exploring &quot;most of&quot; the possible positions. Calculating the score using heuristics is much faster than using Alpha Zero's 19 layer residual network. If Google were to use minimax then they wouldn't be able to look very far ahead. Alpha Zero explores about 70K moves per second which would be only a couple ply deep. MCTS allows it to explore only the more promising moves and simulate those positions.&lt;/p&gt;&#xA;" OwnerUserId="19604" LastActivityDate="2018-12-30T01:59:31.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9749" PostTypeId="2" ParentId="9639" CreationDate="2018-12-30T02:46:28.590" Score="0" Body="&lt;p&gt;Symmetry indicates that a single role definition is shared among the players. Zero-sum indicates that the aggregated gain over all players for any possible disposition of game play is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alpha-beta is a search thrift strategy invented in 1956 by John McCarthy. It is used by an individual player to maximize the probability of favorable game disposition by selecting from move options based on the permutations of game play dependent on each possible selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These do not seem, &lt;em&gt;prima facie&lt;/em&gt;, to be mutually exclusive. Alpha-beta may be an effective computing cost reduction strategy on symmetric zero-sum games. An example is chess game play automation. Furthermore, Reinfield (1983) quotes Knuth and Moore (1975): &quot;The most widely used method for pruning trees of two-person zero-sum games like chess is the alpha-beta algorithm.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the values of the chess game play dispositions of win, draw, and loss are assigned 1, 0, and -1 respectively, then chess is a zero-sum game. If a single random bit is used to determine which player moves first, chess is symmetric. The various machine players in the winner's circle from 1978 to 2017 that used alpha-beta to achieve their victory confirms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ime.usp.br/~rbrito/docs/1-s2.0-0004370275900193-main.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An analysis of alpha-beta pruning&lt;/em&gt;&lt;/a&gt;, DE Knuth, RW Moore, Artificial intelligence, 1975&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.top-5000.nl/ps/An%20improvement%20to%20the%20scout%20tree%20search%20algorithm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An improvement of the Scout tree-search algorithm&lt;/em&gt;&lt;/a&gt;, A Reinefeld, ICCA Journal, 1983&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-30T02:46:28.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9750" PostTypeId="2" ParentId="9377" CreationDate="2018-12-30T03:03:55.090" Score="0" Body="&lt;p&gt;If you mean &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax search&lt;/a&gt; as adversarial, you can take a number of free spaces on the board as an upper bound of the tree depth. &lt;/p&gt;&#xA;" OwnerUserId="18074" LastEditorUserId="16229" LastEditDate="2019-06-10T18:09:41.140" LastActivityDate="2019-06-10T18:09:41.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9751" PostTypeId="1" AcceptedAnswerId="9753" CreationDate="2018-12-30T06:36:03.300" Score="10" ViewCount="9488" Body="&lt;p&gt;I am trying to understand what channels mean in convolutional neural networks. When working with grayscale and colored images, I understand that the number of channels is set to 1 and 3 (in the first conv layer), respectively, where 3 corresponds to red, green, and blue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have a colored image that is 200x200 pixels. The standard is such that the input matrix is a 200x200 matrix with 3 channels. The first convolutional layer would have a filter that is size &lt;span class=&quot;math-container&quot;&gt;$N \times M \times 3$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N,M&amp;lt;200$&lt;/span&gt; (I think they're usually set to 3 or 5). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to structure the input data differently, such that the number of channels now becomes the width or height of the image? i.e., the number of channels would be 200, the input matrix would then be 200x3 or 3x200. What would be the advantage/disadvantage of this formulation versus the standard (# of channels = 3)? Obviously, this would limit your filter's spatial size, but dramatically increase it in the depth direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am really posing this question because I don't quite understand the concept of channels in CNNs. &lt;/p&gt;&#xA;" OwnerUserId="20358" LastActivityDate="2019-01-01T07:01:58.827" Title="What is the concept of channels in CNNs?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9753" PostTypeId="2" ParentId="9751" CreationDate="2018-12-30T11:47:16.283" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Say you have a colored image that is 200x200 pixels. The standard is such that the input matrix is a 200x200 matrix with 3 channels. The first convolutional layer would have a filter that is size &lt;span class=&quot;math-container&quot;&gt;$N×M×3$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N,M&amp;lt;200$&lt;/span&gt; (I think they're usually set to 3 or 5). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Would it be possible to structure the input data differently, such that the number of channels now becomes the width or height of the image? i.e., the number of channels would be 200, the input matrix would then be 200x3 or 3x200. What would be the advantage/disadvantage of this formulation versus the standard (# of channels = 3)? Obviously, this would limit your filter's spatial size, but dramatically increase it in the depth direction. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The different dimensions (width, height, and number of channels) do have different meanings, the intuition behind them is different, and this is important. You could rearrange your data, but if you then plug it into an implementation of CNNs that expects data in the original format, it would likely perform poorly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important observation to make is that the intuition behind CNNs is that they encode the &quot;prior&quot; assumption or knowledge, the &quot;heuristic&quot;, the &quot;rule of thumb&quot;, of &lt;strong&gt;location invariance&lt;/strong&gt;. The intuition is that, when looking at images, we often want our Neural Network to be able to consistently (in the same way) detect features (maybe low-level features such as edges, corners, or maybe high-level features such as complete faces) &lt;strong&gt;regardless of where they are&lt;/strong&gt;. It should not matter whether a face is located in the top-left corner or the bottom-right corner of an image, detecting that it is there should still be performed in the same way (i.e. likely requires exactly the same combination of learned weights in our network). That is what we mean with location invariance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That intution of location invariance is implemented by using &quot;filters&quot; or &quot;feature detectors&quot; that we &quot;slide&quot; along the entire image. These are the things you mentioned having dimensionality &lt;span class=&quot;math-container&quot;&gt;$N \times M \times 3$&lt;/span&gt;. The intuition of location invariance is implemented by taking the exact same filter, and re-applying it in different locations of the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you change the order in which you present your data, you will break this property of location invariance. Instead, you will replace it with a rather strange property of... for example, &quot;width-colour&quot; invariance. You might get a filter that can detect the same type of feature regardless its &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-coordinate in an image, and regarldess of the colour in which it was drawn, but the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-coordinate will suddenly become relevant; your filter may be able to detect edges of any colour in the bottom of an image, but fail to recognize the same edges in the top-side of an image. This is not an intuition that I would expect to work successfully in most image recognition tasks.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that there may also be advantages in terms of computation time in having the data ordered in a certain way, depending on what calculations you're going to perform using that data afterwards (typically lots of matrix multiplications). It is best to have the data stored in RAM in such a way that the inner-most loops of algorithms using the data (matrix multiplication) access the data sequentially, in the same order that it is stored in. This is the most efficient way in which to access data from RAM, and will result in the fastest computations. You can generally safely expect that implementations in large frameworks like Tensorflow and PyTorch will already require you to supply data in whatever format is the most efficient by default.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2018-12-30T11:47:16.283" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="9756" PostTypeId="1" AcceptedAnswerId="9762" CreationDate="2018-12-30T15:45:51.433" Score="1" ViewCount="515" Body="&lt;p&gt;I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons each. For the training data, I am using 'War of The Worlds' by H.G. Wells which adds up to 269639 training samples and 67410 validation samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 7 epochs the validation accuracy has reached 35.1% and the validation loss has reached 2.31. However, after being fed the first sentence of war of the worlds to start it outputs:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the the the the the the the the the the the the the the the the...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure where I'm going wrong; I don't want it to overfit and output passages straight from the training data but I also don't want it to just output 'the' repeatedly. I'm really at a loss as to what I should do to improve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12376" LastActivityDate="2018-12-31T03:03:56.093" Title="LSTM language model not working" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9757" PostTypeId="2" ParentId="9709" CreationDate="2018-12-30T21:44:19.677" Score="1" Body="&lt;p&gt;&lt;strong&gt;Proposed Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The expected output from the policy is an array in the range of (-1, 1) for each joint.&#xA;          The input of the policy is the position of each joint from the past X frames in the environment&#xA;          (replay memory like DeepQ Net),&#xA;          the center of mass of the body,&#xA;          the difference in height between the floor and the body to see if it has fallen and the movement in the x-axis.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The code below is just a test of the environment with a set of movements hard coded in the robot just to test how it could walk later.&#xA;          The environment is set to real time, but I assume it needs to be in a frame by frame lapse during the policy training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initial Considerations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is advisable to first list and analyze the use cases, system requirements, and their design consequences. The following details can be addressed after a high level theory and design have been established.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Algorithm details&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Models according to schemas such as URDF&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Physics simulators, such as Bullet&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Challenge frameworks, such as Gym&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Goals in terms of Cartesian direction of travel&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Number of limbs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Program variable names&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Initialization&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some requirements are generally assumed as part of any useful robotic controller and need not be mentioned explicitly.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Continuous control (discontinuous control and lack of control are poor options)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hinged movement always has a range (min and max)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Listing more relevant system design constraints such as these can guide analysis and design. This is an abridged list to consider as an example.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;No precision in the control of angular speed and velocity at joints.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No vision, audio, radar, sonar, or lidar sensory information available for either position sensing or predictive path planning&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Orientation is not directly sensed&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Surface coordinates are sensed via triangulation, and that subsystem is already designed and tested&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Clearance to ground is known and must be maintained above a configurable constant value&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The use of Deep Reinforcement Learning is expected (which, given the mechanical design, implies the maintenance of a walking policy)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The goal is to maintain a particular direction of robot travel&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each limb has two radial degrees of freedom, controlled by an angular position command input to the motion control sub-system&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The nominal radial acceleration and maximum radial velocity of angular positions are not known&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Extensions of Traditional Control&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI system must produce stability. An established method is closed loop control. The stability performance of the AI system must at least exceed standard PID control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ k_p \epsilon(t) + k_i \int_0^t \epsilon(t) dt + k_d \frac {d\epsilon(t)} {dt} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral term can have an inverse exponential memory decay as such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ k_p \epsilon(t) + k_i \frac {\int_0^t e^t \epsilon(t) dt} {\int_0^t e^t dt} + k_d \frac {d\epsilon(t)} {dt} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, all terms are first degree in &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; parameters. With a feed forward network, some competition winning robotics systems have reached a higher level of performance in terms of stability and efficiency of motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f_p(P) \epsilon(t) + f_i(P) \frac {\int_0^t e^t \epsilon(t) dt} {\int_0^t e^t dt} + f_d(P) \frac {d\epsilon(t)} {dt} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not yet tractable, since offline learning will not take advantage of terrain knowledge gained during a robot's walk. If the terrain is a constant, for instance, if it is always a flat plane, and the positioning system is stable and reliable, reinforcement would not be needed. Such an ideal environment for the walk is rare in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most servo controllers with radial encoder inputs provide positional and dynamic stability using a model similar to the second of the above three equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deep Reinforcement Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The development of reinforced learning policy updating is desirable for real world walker control. Although a policy update function can theoretically be learned from walk examples, the distribution of terrain is not easily representable in example terrain, even if the terrains are generated by genetic-algorithm-like designs modified to simulate planetary geological processes over cosmic time frames.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approaches that show early indications of success involve reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, dynamics &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, and policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; that depend on both on state and action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \pi (u_t | x_t) \\&#xA; p(x_t | x_t, u_t) \\&#xA; r(x_t, u_t) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The study of variants of NAF that leverage replay memory is outlined in &lt;a href=&quot;https://arxiv.org/pdf/1610.00633.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates&lt;/em&gt;&lt;/a&gt;, Shixiang Gu, Ethan Holly, Timothy Lillicrap, Sergey Levine, 2016, using this model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mu_{n+1} (x_t) := {argmax}_u{Q_{{\pi}_n} (x_t, u_t)} \\&#xA; f() := \tanh() \; \text{,} \\&#xA; \mu(x) = f(k + K_x) \\&#xA; Q(x,u) = \frac 1 2 {\big(u − \mu(x)\big)}^T P(u − \mu(x)) + x^T (B x + b) + c $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where P, k, K, B, b, and c are learnable tensors. Gu et. al. explains.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If f() [was instead] identity, then the expression corresponds to a globally quadratic Q-function&#xA;          and a linear feedback policy, though due to the Tanh non-linearity,&#xA;          the Q-function is not linear with respect to state-action features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reward&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of the above considerations, defining reward solely on the basis of one axis of position will likely be insufficient to guide effective learning. Various actions have costs in terms of both time and energy consumption and each may affect future move choices in significant ways. Violation of the  ground clearance constraint must produce a penalty. The reward function must guide policy development to properly take these factors into consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To enhance the tractability of the learning challenge, a process block might best be inserted between the deep reinforcement learning component and the controller to perform a trigonometric and Newtonian translation from desired foot positions relative to the center of robot mass and the radial positions on all control axes to achieve those positions. That removes the burden of that translation from the walker's learner.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2018-12-30T22:08:03.310" LastActivityDate="2018-12-30T22:08:03.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9762" PostTypeId="2" ParentId="9756" CreationDate="2018-12-31T03:03:56.093" Score="1" Body="&lt;p&gt;You can follow the below steps :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;LSTMs are slower in terms of convergence. They take much time to train amd thereby give better results. Try training the network for a more number of epochs like 50 or 65.&lt;/li&gt;&#xA;&lt;li&gt;Use smaller batch size. Try using a RMSProp optimizer. It takes a long time to converge but gives excellent results in case of recurrent nets.&lt;/li&gt;&#xA;&lt;li&gt;Also, try word  level predictions. Means, train your LSTM in such a way that it can predict words and not characters.&lt;/li&gt;&#xA;&lt;li&gt;Use dropout layers in between two LSTM layers. Use a rate like 0.2 or 0.35.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As a working example, you can refer this &#xA;&lt;a href=&quot;https://github.com/shubham0204/Poem_Writer_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;project&lt;/a&gt; which generates poem lines using LSTM in tensorflow.&lt;/p&gt;&#xA;" OwnerUserId="18985" LastActivityDate="2018-12-31T03:03:56.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9765" PostTypeId="1" CreationDate="2018-12-31T04:58:09.793" Score="3" ViewCount="209" Body="&lt;p&gt;If deep learning is a black box, then why are companies still investing in it?&lt;/p&gt;&#xA;" OwnerUserId="21038" LastEditorUserId="2444" LastEditDate="2019-08-20T23:17:18.777" LastActivityDate="2019-08-20T23:18:31.607" Title="If deep learning is a black box, then why are companies still investing in it?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;applications&gt;&lt;real-world&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9766" PostTypeId="1" CreationDate="2018-12-31T07:27:15.720" Score="5" ViewCount="128" Body="&lt;p&gt;I'm reading this really interesting article &lt;a href=&quot;https://arxiv.org/pdf/1712.02950.pdf&quot; rel=&quot;noreferrer&quot;&gt;CycleGAN, a Master of Steganography&lt;/a&gt; and I understand everything up until this paragraph:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we may view the CycleGAN&#xA;  training procedure as continually mounting an adversarial attack on&#xA;  G&#xA;  , by optimizing a generator&#xA;  F&#xA;  to generate adversarial maps that force&#xA;  G&#xA;  to produce a desired image. Since we have demonstrated&#xA;  that it is possible to generate these adversarial maps using gradient descent, it is nearly certain that the&#xA;  training procedure is also causing&#xA;  F&#xA;  to generate these adversarial maps. As&#xA;  G&#xA;  is also being optimized,&#xA;  however,&#xA;  G&#xA;  may actually be seen as cooperating in this attack by learning to become increasingly&#xA;  susceptible to attacks. We observe that the magnitude of the difference&#xA;  y&#xA;  ∗&#xA;  −&#xA;  y&#xA;  0&#xA;  necessary to generate&#xA;  a convincing adversarial example by Equation 3 decreases as the CycleGAN model trains, indicating&#xA;  cooperation of&#xA;  G&#xA;  to support adversarial maps&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How is the CycleGAN training procedure an adversarial attack? I don't really understand the quoted explanation.&lt;/p&gt;&#xA;" OwnerUserId="21040" LastEditorUserId="16565" LastEditDate="2020-01-17T02:06:11.720" LastActivityDate="2020-01-17T02:06:11.720" Title="What is an &quot;adversarial attack&quot;" Tags="&lt;getting-started&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9767" PostTypeId="2" ParentId="9766" CreationDate="2018-12-31T07:41:51.717" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines.&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &quot;&lt;a href=&quot;https://blog.openai.com/adversarial-example-research/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attacking Machine Learning with Adversarial Examples&lt;/a&gt;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You create an input and test against the output, tuning the input to maximize the error. There are different criteria for tuning the input, sometimes you might want the shortest possible input to create the largest error, often you would want very similar input to cause the greatest error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: I take a \$1 bill and write a few more zeros after the one; you accept it as a \$1000 bill.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2018-12-31T07:41:51.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9768" PostTypeId="1" AcceptedAnswerId="9773" CreationDate="2018-12-31T11:41:28.043" Score="2" ViewCount="40" Body="&lt;p&gt;I have the following binary classification problem, my labeled dataset contains images 96x96 px. Now in every image the interest area is of size 32x32 px in the center of the image, and the images are labeled based on that 32x32 px area. If whatever i am trying to detect is in the outer region of the 32x32 px area the label of that image is not affected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is that if i use the whole image when traing, my model will not learn that the interest area is only in the center of the image but on the other hand if i crop the images to be of size 32x32 i am loosing a lot of information which can help the model to train on. I found out that i am getting the best results if i crop the images to be of size 64x64 (kind of a trade of).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now going to the test set, for the test set it doesn't make sense to use this trade of because the model is not learning anything anyways so i would rather crop the test images to 32x32 but then the test set and train set sizes are not the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone came across this problem before? Can i just pad the test images to the size of the train images? is this a good way to go?&lt;/p&gt;&#xA;" OwnerUserId="7463" LastActivityDate="2018-12-31T15:14:29.930" Title="Smaller interest area for images than the size of the image in classification neural networks" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9772" PostTypeId="2" ParentId="9689" CreationDate="2018-12-31T14:47:09.633" Score="0" Body="&lt;p&gt;&lt;strong&gt;Requirements Analysis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term prediction has a temporal element, but the other text in comments indicates that the position of the ball relative to the field of play and goals at the time the image is taken is desired. Predicting where the ball is likely to be in some number of seconds is a much more difficult problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can consider the problem of determining ball position &lt;span class=&quot;math-container&quot;&gt;$(x, y, z)$&lt;/span&gt; relative to a coordinate system superimposed onto the field where &lt;span class=&quot;math-container&quot;&gt;$(0, 0, 0)$&lt;/span&gt; is the center point in team A's goal line and &lt;span class=&quot;math-container&quot;&gt;$(1, 0, 0)$&lt;/span&gt; is the center of team B's goal line and  distances in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; are scaled similarly in &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since some of the images may have the ball in the field of view, the beginnings of a solution may be available. The number of images would need to be large. If a sufficiently large set of images, some of which have the ball in view, were available, this process would likely produce a viable solution with satisfactory accuracy and reliability characteristics.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Develop an unsupervised system that recognizes round objects in the air based on a model of balls.&lt;/li&gt;&#xA;&lt;li&gt;Locate the ball in all frames that have one and assign the ball position as a label for those frames.&lt;/li&gt;&#xA;&lt;li&gt;Use face recognition AI to locate eyes and label them.&lt;/li&gt;&#xA;&lt;li&gt;Use CNN feature extraction and a deep network to train functional correlation &#xA;between eye features and ball position&lt;/li&gt;&#xA;&lt;li&gt;Use the functional correlation on frames without a ball in the frame to locate it&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are two dimensional challenges in using these techniques &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lack of depth perception&lt;/li&gt;&#xA;&lt;li&gt;Orienting players in the field&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is possible to train the trigonometric relations as part of item 4, however orienting players in the field may be untenable, rendering the entire project infeasible. The solution of this problem is dependent upon the solution of this one: &lt;a href=&quot;https://ai.stackexchange.com/search?q=user%3A4302+field+boundary&quot;&gt;https://ai.stackexchange.com/search?q=user%3A4302+field+boundary&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2018-12-31T14:47:09.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9773" PostTypeId="2" ParentId="9768" CreationDate="2018-12-31T15:14:29.930" Score="3" Body="&lt;p&gt;&quot;The problem here is that if i use the whole image when training, my model will not learn that the interest area is only in the center of the image&quot; &lt;strong&gt;Yes, it will, if you have enough samples and they are all classified based on this area only&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;but on the other hand if i crop the images to be of size 32x32 i am loosing a lot of information which can help the model to train on&quot; . I don't understand what information you are loosing if you only care about the center 32x32 area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, you need to ask yourself how will the test input look like and work based on that. If you only care about an area of 32x32 which is always is (and will be) at the center of all images, ignore everything and train and evaluate based just on that area (why add extra unnecessary calculations to the model?). If the size and location of the area might change, don't crop the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line, the test set and the train set need to be of the nature (shape) for the model to work properly.  &lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2018-12-31T15:14:29.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9774" PostTypeId="2" ParentId="9765" CreationDate="2018-12-31T21:23:53.103" Score="5" Body="&lt;p&gt;Easy answer: &lt;strong&gt;utility&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strength and applicability of &quot;black box&quot; NNs has been regularly validated in the past few years, and business is concerned with results.  &lt;em&gt;(i.e. they don't care &lt;a href=&quot;https://en.wiktionary.org/wiki/how_the_sausage_gets_made&quot; rel=&quot;nofollow noreferrer&quot;&gt;how the sausage is made&lt;/a&gt;, so long as it gets made.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-08-20T23:18:31.607" LastActivityDate="2019-08-20T23:18:31.607" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9775" PostTypeId="2" ParentId="9751" CreationDate="2019-01-01T07:01:58.827" Score="1" Body="&lt;p&gt;If you have a gray scale image, that means you are getting data from one sensor.&#xA;If you have an RGB image, that means you are getting data from three sensors.&#xA;If you have a CMYK image, that means you are getting data from four sensors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, channels can be considered as same information seen from different perspective. (here color)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you see how the kernel (for example 5*5*3) moves, it moves only in XY direction and not in the channel direction. So, you are trying to learn features in XY direction from all the channels together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if you exchange the dimensions like you mentioned your XY dimensions become 200*3 or 3*200 and your channels become 200. In this case you are moving kernel not in the actual XY spatial space of image. So, it doesn't make any sense according to me. You are contradicting the basic concept of CNN by doing so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept of CNN itself is that you want to learn features from the spatial domain of the image which is XY dimension. So, you cannot change dimensions like you mentioned.&lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-01-01T07:01:58.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9778" PostTypeId="2" ParentId="9442" CreationDate="2019-01-01T12:01:34.190" Score="0" Body="&lt;p&gt;To generate high level programming language code in the context of genetic programming the &lt;a href=&quot;https://en.wikipedia.org/wiki/Grammatical_evolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grammatical Evolution&lt;/a&gt; technique could be a good start. It allows to generate syntactically correct samples according to a grammar, so there will be no (syntactic) garbage in a population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the original implementation it has (very simple and) quite destructive mutation and crossover operators. This could be changed by making the operators more sophisticated, so they respect the actual tree-like structure of the samples and the grammar constraints, but effectively it will result in implementing the classical tree-based Genetic Programming system (which isn't bad).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evaluation of such samples should be done by executing them in an appropriate environment (the actual VM and the desired map or whatever).&lt;/p&gt;&#xA;" OwnerUserId="15881" LastActivityDate="2019-01-01T12:01:34.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9779" PostTypeId="2" ParentId="9354" CreationDate="2019-01-01T18:42:39.130" Score="0" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1808.04355.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Large-Scale Study of Curiosity-Driven Learning&lt;/a&gt; paper (the prequel to the Random Network Distillation work), in their discussion of Random Features, they reference 3 papers that discuss this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;K. Jarrett, K. Kavukcuoglu, Y. LeCun, et al. &lt;a href=&quot;http://yann.lecun.com/exdb/publis/pdf/jarrett-iccv-09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is the Best Multi-Stage Architecture for Object Recognition?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;A. M. Saxe, P. W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Y. Ng. &lt;a href=&quot;http://www.robotics.stanford.edu/~ang/papers/nipsdlufl10-RandomWeights.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Random Weights and Unsupervised Feature Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Z. Yang, M. Moczulski, M. Denil, N. de Freitas, A. Smola, L. Song, and Z. Wang. &lt;a href=&quot;https://arxiv.org/pdf/1412.7149.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Fried Convnets&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I just briefly glanced over these.  For now, one interesting idea from [2] is to use randomly initialized networks for architecture search.  To evaluate the architecture for the task, you don't have to train it; you can just randomly initialize it and measure its performance.&lt;/p&gt;&#xA;" OwnerUserId="10985" LastActivityDate="2019-01-01T18:42:39.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9781" PostTypeId="1" CreationDate="2019-01-01T22:07:26.810" Score="0" ViewCount="35" Body="&lt;p&gt;I want to create a NHL game predictor and have already trained one neural network on game data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to do is train another model on player seasonal/game data and combine the two models to archive better accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this approach feasible? If it is, how do I go about doing it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have currently trained a neural network to classify the probability of the home team winning a game on a dataset that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h_Won/Lost  h_metric2 h_metric3 h_metric4 a_metric2 a_metric3 a_metric4 h_team1 h_team2 h_team3 h_team4 a_team1 a_team2 a_team3 a_team4&#xA; 1            10       10         10        10         10        10      1       0        0      0         0      1        0      0&#xA; 1            10       10         10        10         10        10      1       0        0      0         0      1        0      0&#xA; 1            10       10         10        10         10        10      1       0        0      0         0      1        0      0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am preparing a dataset of player-data for each game that will have the shape of this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Player     PlayerID    Won/Lost     team      opponent     metric1     metric2   &#xA; Henke         1           1          NY          CAP         10          10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hopefully, this new dataset will have some accuracy on if team is going to have some predictive features that are good and recognised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, say I have these two trained Nural Networks and they both have an accuracy of 70% by them self. But I want to combine them both in the hopes to achieve better predictability. How is this archived? How will the test-dataset be structured?&lt;/p&gt;&#xA;" OwnerUserId="21077" LastEditorUserId="21077" LastEditDate="2019-01-01T23:22:05.460" LastActivityDate="2019-01-01T23:22:05.460" Title="is it possible to train several Neural Networks on different types of data and combine them?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9782" PostTypeId="2" ParentId="9781" CreationDate="2019-01-01T23:08:12.937" Score="1" Body="&lt;p&gt;The term you need  is “model ensembles”, that’s the way models are combined.  Pretty hard to be more specific since you don’t give a language or any other details.&lt;/p&gt;&#xA;" OwnerUserId="21079" LastActivityDate="2019-01-01T23:08:12.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9783" PostTypeId="2" ParentId="9725" CreationDate="2019-01-02T00:45:05.643" Score="3" Body="&lt;p&gt;&lt;strong&gt;The Project Summarized&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The project goal appears to be a common one: Routing correspondence in an efficient manner to maintain good but low cost customer and public relations. A few features of the project were mentioned.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neural network project&lt;/li&gt;&#xA;&lt;li&gt;Received some design and project history from predecessor&lt;/li&gt;&#xA;&lt;li&gt;Classifies messages for telcos&lt;/li&gt;&#xA;&lt;li&gt;Sends results to support groups at appropriate locales&lt;/li&gt;&#xA;&lt;li&gt;Uses 2 relu layers, ending with a softmax&lt;/li&gt;&#xA;&lt;li&gt;Word2Vec embedding&lt;/li&gt;&#xA;&lt;li&gt;Trained with a clean language file&lt;/li&gt;&#xA;&lt;li&gt;All special characters and numbers removed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The requirements for current development were indicated. The current work is to develop an artificial network that places incoming messages into one of two categories accurately and reliably.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Moderated &amp;mdash; insulting, fraudulent in purpose (spam), trivial routine&lt;/li&gt;&#xA;&lt;li&gt;Operative &amp;mdash; relevant question requiring internal human attention&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Research and development is beginning along reasonable lines.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Trained with 300,000 messages&lt;/li&gt;&#xA;&lt;li&gt;Word2Vec used&lt;/li&gt;&#xA;&lt;li&gt;40% of classified as moderated&lt;/li&gt;&#xA;&lt;li&gt;Permuted cycles and epochs&lt;/li&gt;&#xA;&lt;li&gt;Achieved 90% accuracy&lt;/li&gt;&#xA;&lt;li&gt;Loss stays near 0.5&lt;/li&gt;&#xA;&lt;li&gt;In test, operative accuracy 0.9, moderated accuracy max of 0.6&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First Obstacle and Feasibility&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first obstacle encountered is that in QA using production environment data, 90% of the messages where left unclassified, 5% of the classifications were accurate, and the remaining 5% were inaccurately classified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is correct that the even split of 5% accuracy and 5% inaccuracy indicates that information learned is not yet transferable to the quality assurance test phase using real production environment messages. In information theory phraseology, no bits of usable information were transferred and entropy remained unchanged on this first experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These kinds of disappointments are not uncommon when first approaching the use of AI in an existing business environment, so this initial outcome should not be taken as a sign that the idea won't work. The approach will likely work, especially with foul language, which is not dependent on cultural references, analogies, or other semantic complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recognizing notices that are for audit purposes only, from a social network accounts or purchase confirmations, can be handled through rules. The rule creation and maintenance can theoretically be automated too, and some proprietary systems exist that do exactly that. Such automation can be learned using the appropriate training data, but real time feedback is usually employed, and those systems are usually model based. That is an option for further down the R&amp;amp;D road.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scope of the project is probably too small, but that's not a big surprise either. Most projects suffer from early overoptimism. A pertinent quote from Redford's &lt;em&gt;The Melagro Beanfield War&lt;/em&gt; illuminates the practical purpose of optimism.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; APPARITION&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I don't know if your friend knows what he's in for.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; AMARANTE&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Nobody would do anything if they knew what they were in for.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initial Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not necessary to reduce the number of message categories to two, but there is nothing wrong with starting R&amp;amp;D by refining approach and high level design with the simplest case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last layer may be more training efficient if a binary threshold is used for the activation function instead of softmax, since there is only one bit of output needed when there are only two categories. This also forces the network training objective to be the definitive selection of a category, which may benefit the overall rate of R&amp;amp;D progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be ways of improving outcomes by adding more metrics in the code to beyond just 'accuracy'. Others who work with such details every day may have more domain specific knowledge in this regard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Culture and Pattern Detection&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insults and curse words are entirely different kinds of things. Foul language is a linguistic symbol or phrase that fits into a broadcasting or publishing category of prohibition. The rules of prohibition are well established in most languages and could be held in a configuration file along with the permutations of each symbol or phrase. In the case of &lt;em&gt;sh*t&lt;/em&gt;, related forms include &lt;em&gt;sh*tty&lt;/em&gt;, &lt;em&gt;sh*thead&lt;/em&gt;, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also useful to distinguish the sub-sets of foul language.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cursing (expressing the wish for calamity to befall the recipient)&lt;/li&gt;&#xA;&lt;li&gt;Swearing (considered blasphemy by some)&lt;/li&gt;&#xA;&lt;li&gt;Exclamations that are considered foul by publishers and broadcasters&lt;/li&gt;&#xA;&lt;li&gt;Additional items parents don't want their children to hear&lt;/li&gt;&#xA;&lt;li&gt;Edge cases like &lt;em&gt;crap&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;foul language&lt;/em&gt; is a super-set of these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Distribution Alignment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning algorithms and theory are based on probabilistic alignment of feature distributions between training and use. The distribution of training data must closely resembles the distribution found when the trained AI component is later used. If not, the convergence of learning processes on some optimal behavior defined by gain or loss functions may succeed but the execution of that behavior in the business or industry may fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Internationalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multilingual AI should usually be fully internationalized. Training and use of training with two distinct dialects will almost always perform poorly. That creates a data acquisition challenge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As stated above, classification and learning depend on the alignment of statistical distributions between data used in training and data processing relying on the use of what was learned. This is also true of human learning, so this requirement will not likely be overcome any time soon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All these forms of foul language must be programmed flexibly across these cultural dimensions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Character set&lt;/li&gt;&#xA;&lt;li&gt;Collation order&lt;/li&gt;&#xA;&lt;li&gt;Language&lt;/li&gt;&#xA;&lt;li&gt;Dialect&lt;/li&gt;&#xA;&lt;li&gt;Other locale related determinants&lt;/li&gt;&#xA;&lt;li&gt;Education level&lt;/li&gt;&#xA;&lt;li&gt;Economic strata&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once one of these is included in the model (which will be imperative) then there is no reason why the others cannot be included at little cost, so it is wise to begin with standard dimensions of flexibility. The alternative will likely lead to costly branching complexity to represent specific rules, which could have been made more maintainable by generalizing for international use up front.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Insult Recognition&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insults require comprehension beyond the current state of technology. Cognitive science may change that in the future, but projections are mere conjecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use of a regular expression engine with a fuzzy logic comparator is achievable and may appease the stakeholders of the project, but identifying insults may be infeasible at this time, and the expectations should be set with stakeholders to avoid later surprises. Consider these examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The nose on your face looks like a camel.&lt;/li&gt;&#xA;&lt;li&gt;Kiss the darkest part of my little white. (From &lt;em&gt;Avatar&lt;/em&gt; screenplay)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The word combinations in these are not likely to be in some data set you can use for training, so Word2Vec will not help in these types of cases. Additional layers may assist with proper handling of the at least some of the semantic and referential complexity of insults, but only some.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Explicit Answers to Explicit Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to accomplish this task with a neural network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, in combination with excellence in higher level system design and best practices for internationalization.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the structure of this neural network correct for this task?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The initial experiments look like a reasonable beginning toward what would later be correct enough. Do not be discouraged, but don't expect the first pass at something like this to look much like what passes user acceptance testing a year from now. Experts can't pull that rate of R&amp;amp;D progress off, unless they hack and cobble something together from previous work.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are 300k messages enough to train the neural network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Probably not. In fact, 300m messages will not catch all combinations of cultural references, analogies, colloquialisms, variations in dialect, plays on words, and games that spammers play to avoid detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would really help is a feedback mechanism so that production outcomes are driving the training rather than a necessarily limited data set. Canned data sets are usually restricted in the accuracy of their probabilistic representation of social phenomena. None will likely infer dialect and other locale features to better detect insults. A Parisian insult may have nothing in common with a Creole insult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The feedback mechanism must be based on impressions in some way to become and remain accurate. The impressions must be labelled with all the locale data that is reasonably easy to collect and possibly correlated to the impression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This implies the use of rules acquisition, fuzzy logic control, reinforcement learning, or the application of naive Bayesian approaches somewhere appropriate within the system architecture.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do I need to clean up the data from uppercase, special characters, numbers etc?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Numbers can be relevant. Because of historical events and religious texts, 13 and 666 might be indications of something offensive, respectively. One can also use numbers and punctuation to convey word content. Here are some examples of spam detection resistant click bait.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I've got a 6ex opportunity 4u.&lt;/li&gt;&#xA;&lt;li&gt;Wanna 69?&lt;/li&gt;&#xA;&lt;li&gt;Values are rising 50%! We have 9 investment choices 4 you to check out.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The meaning of the term &lt;em&gt;special character&lt;/em&gt; is vague and ambiguous. Any character in UTF-8 is legitimate for almost all Internet communications today. HTML5 provides additional entities beginning with an ampersand and ending with a semicolon. (See &lt;a href=&quot;https://dev.w3.org/html5/html-author/charref&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dev.w3.org/html5/html-author/charref&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filtering these out is a mistake. Spammers leverage these standards to penetrate spam detection. In this example, the stroke similarities of a capital ell (L) and those of the British pound symbol can be exploited to produce spam detection resistant click bait.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do you like hot &amp;pound;egs?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Removing special characters that fit within the Internet standards of UTF-8 and HTML entities will likely lead to disaster. It is recommended not to follow that part of the predecessor's design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding emoticons and other ideograms, these are linguistic elements that may represent in text encoding the volume, pitch, or tone modulation of phonetics, or they may represent face or body language. In many languages ideograms are used in place of words. For a global system running in parallel with the blogsphere, emoticons are part of linguistic expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that reason, they are not significantly different than word roots, prefixes, suffixes, conjugations, or word pairs as linguistic elements which can also express emotion as well as logical reasoning. For the learning algorithm to learn categorization behavior in the presence of ideograms, the ideograms must remain in training features and later in real time processing of those features using the results of training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Information&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some additional information is covered in this existing post: &lt;a href=&quot;https://ai.stackexchange.com/questions/3472/spam-detection-using-recurrent-neural-networks/8397#8397&quot;&gt;Spam Detection using Recurrent Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since spam detection is closely related to fraud detection, the spammer fraudulently acting like a relationship already exists with their recipients, this existing post may be of assistance too: &lt;a href=&quot;https://ai.stackexchange.com/questions/9624/can-we-implement-gan-generative-adversarial-neural-networks-for-classication-p/9683#9683&quot;&gt;Can we implement GAN (Generative adversarial neural networks) for classication problem like Fraud detecion?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another resource that may help is this: &lt;a href=&quot;https://www.tensorflow.org/tutorials/representation/word2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/representation/word2vec&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-04T00:04:56.810" LastActivityDate="2019-01-04T00:04:56.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9784" PostTypeId="2" ParentId="9677" CreationDate="2019-01-02T05:28:26.070" Score="1" Body="&lt;p&gt;I've discovered Doc2Vec which does something similar to what I am trying to accomplish. This doesn't &lt;em&gt;exactly&lt;/em&gt; answer my question of why the network I was trying to build doesn't work, but at least it shows how indexed outputs can be pulled from a network, with open source to show how it is built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/23969/sentence-similarity-prediction&quot;&gt;https://datascience.stackexchange.com/questions/23969/sentence-similarity-prediction&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20930" LastActivityDate="2019-01-02T05:28:26.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9785" PostTypeId="1" CreationDate="2019-01-02T08:56:35.587" Score="5" ViewCount="320" Body="&lt;p&gt;I'm just curious as to the state of the field, since I'm planning to have it as my AI grad school specialization. From what I've observed, it peaked in the 90s to the 2000s, but it is seen as more of a novelty within the AI/ML research today. I'm aware, however, that MIT Press Journals and IJALR are still publishing new research. Just that I have rarely if not ever heard these topics discussed in A.I/ML conferences. &lt;/p&gt;&#xA;" OwnerUserId="21083" LastEditorUserId="2444" LastEditDate="2019-06-02T14:17:15.463" LastActivityDate="2019-11-29T18:15:08.750" Title="Is Artificial Life a dying field?" Tags="&lt;getting-started&gt;&lt;artificial-life&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9786" PostTypeId="1" AcceptedAnswerId="9788" CreationDate="2019-01-02T08:58:05.600" Score="2" ViewCount="233" Body="&lt;p&gt;I'm currently working on a regression problem and I have 10 inputs/attributes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do if there are correlations between different features of the input data? Does the correlation between inputs affect the performance (e.g. accuracy) of the model?&lt;/p&gt;&#xA;" OwnerUserId="21084" LastEditorUserId="2444" LastEditDate="2019-08-19T22:43:21.507" LastActivityDate="2019-08-19T22:43:21.507" Title="Does the correlation between inputs affect the model performance?" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;statistical-ai&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9788" PostTypeId="2" ParentId="9786" CreationDate="2019-01-02T11:30:37.533" Score="4" Body="&lt;p&gt;Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see &lt;a href=&quot;https://stats.stackexchange.com/q/113417/82135&quot;&gt;https://stats.stackexchange.com/q/113417/82135&lt;/a&gt; for more details). So, if you have non-zero correlation between two features, it means they are dependent. If they are dependent, then one feature gives you information about the other and vice-versa: in a certain way, one of the two is, at least partially, redundant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unnecessary features might not affect the performance (e.g. the accuracy) of a model. However, if you reduce the number of features, the learning process might actually be faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to try some &lt;a href=&quot;https://en.wikipedia.org/wiki/Dimensionality_reduction&quot; rel=&quot;noreferrer&quot;&gt;dimensionality reduction&lt;/a&gt; technique, in order to reduce the number of features.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-04T00:46:33.657" LastActivityDate="2019-06-04T00:46:33.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9790" PostTypeId="2" ParentId="5308" CreationDate="2019-01-02T17:46:21.257" Score="0" Body="&lt;p&gt;Algorithms can learn to cheat:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;A machine learning agent intended to transform aerial images into street maps and back was found to be cheating by hiding information it would need later in “a nearly imperceptible, high-frequency signal.”&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;...a computer creating its own steganographic method to evade having to actually learn to perform the task at hand is rather new.&amp;quot;&#xA;&lt;br&gt;&lt;sub&gt;Source: &lt;em&gt;&lt;a href=&quot;https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-from-its-creators-to-cheat-at-its-appointed-task/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This clever AI hid data from its creators to cheat at its appointed task&lt;/a&gt;&lt;/em&gt; (TechCrunch)&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-02T17:46:21.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9791" PostTypeId="2" ParentId="3343" CreationDate="2019-01-02T19:21:37.393" Score="1" Body="&lt;p&gt;Your question is incredibly broad -- so in response, two broad frameworks I'd encourage you to look at are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For cutting edge chatbot conversation development &lt;a href=&quot;http://rasa.ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rasa.ai&lt;/a&gt; is an open source framework that is more adaptable than more traditional rule-based systems&lt;/li&gt;&#xA;&lt;li&gt;For speech recognition check out &lt;a href=&quot;https://discourse.mozilla.org/c/deep-speech&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://discourse.mozilla.org/c/deep-speech&lt;/a&gt; that is also open source. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3370" LastEditorUserId="3370" LastEditDate="2019-01-02T19:57:12.587" LastActivityDate="2019-01-02T19:57:12.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9792" PostTypeId="2" ParentId="6117" CreationDate="2019-01-02T20:02:47.100" Score="1" Body="&lt;p&gt;I spent some time on the Future of Work in UK government. If you want to look at the impact of AI on work you need to have good definitions of soft skills that you can measure and you can track progress against.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the definitions sometimes need to change and decomposed. Take &quot;Coaching&quot; from that list: there are now bots (such as Wysa.ai) that coach you. You can measure coaching as a skill by the effectiveness of the participants achieving goals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things really get murky when you talk about &quot;Creativity&quot; -- I wrote a &lt;a href=&quot;https://medium.com/@julian.harris/creativity-how-is-ai-impacting-this-human-skill-289470860c4e&quot; rel=&quot;nofollow noreferrer&quot;&gt;piece on AI impact of creativity&lt;/a&gt; in a similar vein. &lt;/p&gt;&#xA;" OwnerUserId="3370" LastActivityDate="2019-01-02T20:02:47.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9794" PostTypeId="1" AcceptedAnswerId="9801" CreationDate="2019-01-02T20:05:21.397" Score="1" ViewCount="79" Body="&lt;p&gt;I’m a researcher and I’m currently conducting a research project. I will conduct a study where I would like to trigger different emotions using chatbots on a smartphone (e.g. on Facebook Messenger).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any existing chatbots which are able to trigger different emotions intentionally (also negative ones)?&lt;/p&gt;&#xA;" OwnerUserId="21103" LastActivityDate="2019-01-02T23:30:47.527" Title="Chatbots triggering emotions" Tags="&lt;intelligent-agent&gt;&lt;chat-bots&gt;&lt;emotional-intelligence&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9795" PostTypeId="2" ParentId="8407" CreationDate="2019-01-02T20:09:14.643" Score="1" Body="&lt;p&gt;Check out &lt;a href=&quot;https://github.com/DmitryUlyanov/deep-image-prior&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code to DeepImagePrior&lt;/a&gt; it does a remarkable job guessing what's missing to repair images with a variety of damage. &lt;/p&gt;&#xA;" OwnerUserId="3370" LastActivityDate="2019-01-02T20:09:14.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9797" PostTypeId="2" ParentId="7222" CreationDate="2019-01-02T20:33:13.013" Score="0" Body="&lt;p&gt;This is a huge growth area in the impact of AI on HR -- see all the companies we've found that &lt;a href=&quot;https://cognitionx.com/directory/products?f%5Buc%5D%5B0%5D=Candidate%20Matching&quot; rel=&quot;nofollow noreferrer&quot;&gt;do candidate matching for instance&lt;/a&gt; (disclaimer I work for CognitionX). Under the hood, there are techniques that don't rely on vocabulary such as Facebook's FastText but need more training data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some other resources&#xA;&lt;a href=&quot;https://github.com/JAIJANYANI/Automated-Resume-Screening-System&quot; rel=&quot;nofollow noreferrer&quot;&gt;Job matching using unsupervised learning&lt;/a&gt; (k-nearest neighbour)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1CFA9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1CFA9.png&quot; alt=&quot;Excerpt from Oct 2018 paper -- using convolutional neural networks&quot;&gt;&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/abs/1810.04040&quot; rel=&quot;nofollow noreferrer&quot;&gt;see paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3370" LastActivityDate="2019-01-02T20:33:13.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9798" PostTypeId="2" ParentId="3161" CreationDate="2019-01-02T21:37:52.553" Score="0" Body="&lt;p&gt;The two most helpful descriptions of AI I've seen are &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Tactically: &quot;Cheap Predictions&quot;&lt;/li&gt;&#xA;&lt;li&gt;More generally: &quot;Cheap Intelligence&quot;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are broadly 3 ways that AI is evolving today:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Rule-based AI: if this, then that. Best when you want reliable outcomes and you want intelligence in a very rule-heavy situation (such as insurance applications).&lt;/li&gt;&#xA;&lt;li&gt;Pattern-based AI (machine / deep learning): take lots of examples, then give them labels. That picture is a cat, that is a dog, that's a malignant tumour, that's a person who defaults on a loan. This is statistical though, so &quot;predictions&quot; is really key here. You'll never be 100% confident with pattern-based systems but you can get good enough to rapidly accelerate human decision-making.&lt;/li&gt;&#xA;&lt;li&gt;Biomimicry: simulation of biological systems. Consider &lt;a href=&quot;https://www.youtube.com/watch?v=X4Ta3O-p6z0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soul Machines Baby X&lt;/a&gt; as state of the art here. The vision here is simulate life, get the resulting &quot;emergent&quot; properties of thoughts, feelings, consciousness and emotions. This is still mostly theory though. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Future AI will be a mix of all of these. &lt;a href=&quot;https://en.wikipedia.org/wiki/Varian_Rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;Varian's Rule&lt;/a&gt; says &quot;To forecast the future see how the rich live today&quot;. People talking about &quot;democratisation&quot; tend to mean this idea. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm specialising in AI for speech and text and make the observation that we're entering the age of what I call &quot;Digital Beings&quot; (&lt;a href=&quot;https://medium.com/series/the-age-of-digital-beings-8e7aab563fc3&quot; rel=&quot;nofollow noreferrer&quot;&gt;more here on my simple thesis&lt;/a&gt;). The key change in this area I think will be very powerful personal coaches, companions, and concierges. &lt;/p&gt;&#xA;" OwnerUserId="3370" LastActivityDate="2019-01-02T21:37:52.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9799" PostTypeId="5" CreationDate="2019-01-02T21:44:07.577" Score="0" Body="&lt;p&gt;-Artificial chemistry and the origins of life&#xA;-Self-assembly, growth, and development&#xA;-Self-replication and self-repair&#xA;-Systems and synthetic biology&#xA;-Perception, cognition, and behavior&#xA;-Embodiment and enactivism&#xA;-Collective behaviors of swarms&#xA;-Evolutionary and ecological dynamics&#xA;-Open-endedness and creativity&#xA;-Social organization and cultural evolution&#xA;-Societal and technological implications&#xA;-Philosophy and aesthetics&#xA;-Applications to biology, medicine, business, education, or entertainment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://www.mitpressjournals.org/loi/artl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Life Forum&lt;/a&gt; (MIT Press) | &lt;a href=&quot;http://alife.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Society for Artificial Life&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-01-02T21:44:07.577" LastActivityDate="2019-01-02T21:44:07.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9800" PostTypeId="4" CreationDate="2019-01-02T21:44:07.577" Score="0" Body="For question about  artificial systems that exhibit the behavioral characteristics of natural living systems." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-01-02T21:44:07.577" LastActivityDate="2019-01-02T21:44:07.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9801" PostTypeId="2" ParentId="9794" CreationDate="2019-01-02T23:30:47.527" Score="1" Body="&lt;p&gt;Emotions can of course be triggered by lots of different things. I think the most rich source could well be socialbots like Mitsuku.com and Zo.ai -- Steve Worswick is the owner of Mitsuku and may be interested in helping you by doing (appropriately filtered) chat log queries. You can get him on Twitter at @Mitsuku. &lt;/p&gt;&#xA;" OwnerUserId="3370" LastActivityDate="2019-01-02T23:30:47.527" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9802" PostTypeId="2" ParentId="9766" CreationDate="2019-01-03T00:17:04.123" Score="0" Body="&lt;p&gt;Simple answer is tweaking an image in unnoticeable ways that completely fool software. Eg a cat that is identified as 99% likely &quot;to be guacamole&quot; &lt;a href=&quot;https://mashable.com/2017/11/02/mit-researchers-fool-google-ai-program/#CU7dSAfQ5sqY&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mashable.com/2017/11/02/mit-researchers-fool-google-ai-program/#CU7dSAfQ5sqY&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3370" LastActivityDate="2019-01-03T00:17:04.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9808" PostTypeId="1" CreationDate="2019-01-03T14:20:32.507" Score="8" ViewCount="1627" Body="&lt;p&gt;I understand the minimax algorithm, but I am unable to understand deeply the minimax algorithm with alpha-beta pruning, even after having looked up several sources (on the web) and having tried to read the algorithm and understand how it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have a good source that explains alpha-beta pruning clearly, or can you help me to understand the alpha-beta pruning (with a simple explanation)?&lt;/p&gt;&#xA;" OwnerUserId="21125" LastEditorUserId="2444" LastEditDate="2019-02-26T09:40:09.050" LastActivityDate="2019-02-26T09:40:09.050" Title="Can someone help me to understand the alpha-beta pruning algorithm?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9812" PostTypeId="1" AcceptedAnswerId="9830" CreationDate="2019-01-03T16:42:59.147" Score="5" ViewCount="969" Body="&lt;p&gt;In &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the DQN paper&lt;/a&gt;, it is written that the state-space is high dimensional. I am a little bit confused about this terminology.&lt;/p&gt;&#xA;&lt;p&gt;Suppose my state is a high dimensional vector of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is a huge number. Let's say I solve this task using &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning and I fix the state space to &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; vectors, each of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; dimensions. &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning can easily work with these settings as we need only a table of dimensions &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; x number of actions.&lt;/p&gt;&#xA;&lt;p&gt;Let's say my state space can have an infinite number of vectors each of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; dimensions. In these settings, Q-learning would fail as we cannot store Q-values in a table for each of these infinite vectors. On the other hand, DQN would easily work, as neural networks can generalize for other vectors in the state-space.&lt;/p&gt;&#xA;&lt;p&gt;Let's also say I have a state space of infinite vectors, but each vector is now of length &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, i.e., small dimensional vectors. Would it make sense to use DQN in these settings? Should this state-space be called high dimensional or low dimensional?&lt;/p&gt;&#xA;" OwnerUserId="21131" LastEditorUserId="2444" LastEditDate="2021-02-03T19:09:45.533" LastActivityDate="2021-02-04T11:00:31.317" Title="What is a high dimensional state in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;terminology&gt;&lt;state-spaces&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9813" PostTypeId="1" CreationDate="2019-01-03T17:45:03.813" Score="4" ViewCount="950" Body="&lt;p&gt;This is a question related to &lt;a href=&quot;https://ai.stackexchange.com/questions/9725/neural-network-to-detect-spam&quot;&gt;Neural network to detect &amp;quot;spam&amp;quot;?&lt;/a&gt;. &#xA;I'm wondering how it would be possible to handle the emotion conveyed in text. In informal writing, especially among a juvenile audience, it's usual to find emotion expressed as repetition of characters. For example, &quot;Hi&quot; doesn't mean the same as &quot;Hiiiiiiiiiiiiiii&quot; but &quot;hiiiiii&quot;, &quot;hiiiiiiiii&quot;, and &quot;hiiiiiiiiii&quot; do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A naive solution would be to preprocess the input and remove the repeating characters after a certain threshold, say, 4. This would probably reduce most long &quot;hiiiii&quot; to 4 &quot;hiiii&quot;, giving a separate meaning (weight in a context?) to &quot;hi&quot; vs &quot;long hi&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The naivete of this solution appears when there are combinations. For example,&#xA;haha vs hahahahaha or lol vs lololololol. Again, we could write a regex to reduce lolol[ol]+ to lolol. But then we run into the issue of hahahaahhaaha where a typo broke the sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the whole issue of Emoji. Emoji may seem daunting at first since they are special characters. But once understood, emoji may actually become helpful in this situation. For example,  may mean a very different thing than , but  may mean the same as  and . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick with emojis, to me, is that they might actually be easier to parse. Simply add spaces between  to convert  to     in the text analysis. I would guess that repetition would play a role in training, but unlike &quot;hi&quot;, and &quot;hiiii&quot;, Word2Vec won't try to categorize  and  as different words (as I've now forced to be separate words, relying in frequency to detect the emotion of the phrase). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even more, this would help the detection of &quot;playful&quot; language such as , where the  emoji might imply there is anger, but alongside  and especially when repeating  multiple times, it would be easier for a neural network to understand that the person isn't really angry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does any of this make sense or I'm going in the wrong direction?&lt;/p&gt;&#xA;" OwnerUserId="17272" LastActivityDate="2020-02-28T01:02:40.350" Title="Handling emotion in informal text (Hi vs HIIIIII!!!!)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9815" PostTypeId="2" ParentId="9812" CreationDate="2019-01-03T18:01:57.503" Score="2" Body="&lt;p&gt;Yes, it makes sense to use DQN in state space with small number of dimensions as well. It doesn't really matter how big your state dimension is, but if you have state with 2 dimensions for instance you wouldn't use convolutional layers in your neural net like its used in the paper you mentioned, you can use ordinary fully connected layers, it depends on the problem.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-01-03T18:01:57.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9816" PostTypeId="1" CreationDate="2019-01-03T20:09:55.593" Score="0" ViewCount="328" Body="&lt;p&gt;I am trying to implement a Deep Q Network to play &lt;a href=&quot;http://www.freeasteroids.org/welcome/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asteroids&lt;/a&gt;. Unfortunately, I am not sure how to calculate the Q value exactly, if I am exploring. For example, the agent is exploring for 1 second (otherwise makes no sense; I cannot let it just explore one step). Unfortunately, it makes a mistake at 0.99s, and the reward collapses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, I am using the following formula to evaluate or update the Q value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{new,t} = reward + \gamma Q_{max,t+1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do I know the max Q value of the next step? I could consider the best Q value the network says, but this is not necessarily true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see the current implementation at the following URL:&#xA;&lt;a href=&quot;https://github.com/SuchtyTV/RLearningBird/blob/master/src/main/java/rlgame/Brain.java&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SuchtyTV/RLearningBird/blob/master/src/main/java/rlgame/Brain.java&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="2444" LastEditDate="2019-02-16T00:14:25.797" LastActivityDate="2019-02-16T00:14:25.797" Title="How do I update the Q values of a Deep Q Network when exploring?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;q-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9817" PostTypeId="1" CreationDate="2019-01-03T22:48:45.190" Score="1" ViewCount="26" Body="&lt;p&gt;Below is a taxonomy of neurons. Some of these types occur in different locations in the brain, but there are adjacent neurons of varying types with clearly functional type diversity in many parts of the brain, so the idea of having a layer of homogeneous cells in artificial networks may be limiting. Is there any reason why that is definitely the case or definitely not the case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hx5jY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hx5jY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since there are different types of neurons in adjacent positions in the brain's arrays, should heterogeneous layers be developed, mathematically, experimentally, and in APIs designed to provide AI components for applications?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4455486/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Brain-wide analysis of electrophysiological diversity yields novel categorization of mammalian neuron types&lt;/em&gt;&lt;/a&gt;, 2015, Tripathy et. al.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-03T22:48:45.190" Title="Since there are different types of neurons in adjacent positions in the brain's arrays, should heterogeneous layers be developed?" Tags="&lt;ai-design&gt;&lt;math&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9819" PostTypeId="1" CreationDate="2019-01-03T23:19:08.443" Score="1" ViewCount="181" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;My understanding is the input neurons seem to seem to compute a weighted sum moving from one layer to another. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9tqAt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9tqAt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \sum_i a_i w_i = a'_{k} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to compute this weighted sum the sum must be discrete. Is there any known method to compute the sum when the activation is a continuous function? Is the below formula of any consequence problems in artificial intelligence? Can anyone give a specific problem where it might be useful?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My Method&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$b_r = \sum_{d \mid r} a_d\mu(\frac{m}{d})$&lt;/span&gt;. We prove that if the &lt;span class=&quot;math-container&quot;&gt;$b_r$&lt;/span&gt;'s are small enough, the result is true (where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the mobius function). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Claim: If &lt;span class=&quot;math-container&quot;&gt;$\lim_{n \to \infty} \frac{\log^2(n)}{n}\sum_{r=1}^n |b_r| = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is smooth, then &lt;span class=&quot;math-container&quot;&gt;$$\lim_{k \to \infty} \lim_{n \to \infty} \sum_{r=1}^n a_rf\left(\frac{kr}{n}\right)\frac{k}{n} = \left(\lim_{s \to 1} \frac{1}{\zeta(s)}\sum_{r=1}^\infty \frac{a_r}{r^s}\right)\int_0^\infty f(x)dx.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I will not go into the proof of this over but for those who are interested: &lt;a href=&quot;https://math.stackexchange.com/questions/2888976/a-rough-proof-for-infinitesimals&quot;&gt;https://math.stackexchange.com/questions/2888976/a-rough-proof-for-infinitesimals&lt;/a&gt; I will merely state what the formula means:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider we have a curve &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; now if one wishes to perform a weighted sum in the limiting case of this function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i0hCV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i0hCV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the curve &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. Then splitting it to &lt;span class=&quot;math-container&quot;&gt;$k/n= h$&lt;/span&gt; intervals then adding the first strip (&lt;span class=&quot;math-container&quot;&gt;$d_1$&lt;/span&gt; times): &lt;span class=&quot;math-container&quot;&gt;$  f(h) \cdot d_1$&lt;/span&gt;. Then the second strip (&lt;span class=&quot;math-container&quot;&gt;$d_2$&lt;/span&gt; times) &lt;span class=&quot;math-container&quot;&gt;$ f(2h) \cdot d_2$&lt;/span&gt; times ... And so on . Hence. &lt;span class=&quot;math-container&quot;&gt;$d_r$&lt;/span&gt; can be thought of as the weight at &lt;span class=&quot;math-container&quot;&gt;$f(rh)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21136" LastEditorUserId="21136" LastEditDate="2020-03-25T16:52:49.707" LastActivityDate="2020-03-25T16:52:49.707" Title="Method to compute the sum when the activation is a continuous function?" Tags="&lt;artificial-neuron&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9820" PostTypeId="2" ParentId="9813" CreationDate="2019-01-03T23:50:55.113" Score="1" Body="&lt;p&gt;These kinds of repetitions in text can place recurrence demands on learning algorithms that may or may not be handled without special encoding.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hi.&lt;/li&gt;&#xA;&lt;li&gt;Hiiii!&lt;/li&gt;&#xA;&lt;li&gt;HIIIIIIIII&lt;/li&gt;&#xA;&lt;li&gt;Hi!!!!!!!!!!!!!!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These have the same meaning on one level, but different emotional content and therefore different correlations to categories when detecting the value of an email, which in the simplest case is the placement of a message in one of two categories.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pass to a recipient&lt;/li&gt;&#xA;&lt;li&gt;Archive only&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is colloquially called spam detection, although not all useless emails are spam and some messages sent by organizations that broadcast spam may be useful, so technically the term &lt;em&gt;spam&lt;/em&gt; is not particularly useful. The determinant should usually be the return on investment to the recipient or the organization receiving and categorizing the message.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is reading the message and potentially responding likely of greater value than the cost of reading it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is a high level paraphrase of what the value or cost function must represent when AI components are employed to learn about or track close to (in continuous learning) some business or personal optimality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question proposes a normalization scheme that truncates long repetitions of short patterns in characters, but truncation is necessarily destructive. Compression of some type that will both preserve nuance and work with the author's use of Word2Vec is a more flexible and comprehensive approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of playful sequences of characters it is anthropomorphic to imagine that an artificial network will understand playfulness or anger, however existing learning devices can certainly learn to use character sequences that humans would call playful or angry in the function that emerges to categorize the message containing them. Just remember that model free learning is not at all like cognition, so the term &lt;em&gt;understanding&lt;/em&gt; is placing an expectation on the mental capacities of the AI component that the AI component may not possess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since no indication that a recurrent or recursive network will be used but rather the entire message is represented in a fixed width vector, so the question becomes which of these two approaches will produce the best outcomes after learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Leaving the text uncompressed so that an 'H' character followed by ten 'i' characters is distinct as a word from an 'H' character followed by five 'i' characters&lt;/li&gt;&#xA;&lt;li&gt;Compressing the text to &quot;Hi [9xi]&quot; and &quot;Hi [4xi]&quot; respectively or some such word bifurcation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This second approach produces reasonable behavior with other cases mentioned, such as &quot;&quot; pre-processed into &quot; [2x]&quot;. What the algorithm in Word2Vec will do with each of these two choices and how its handling of them will affect outcomes is difficult to predict. Experiments must be run. Three things are advisable courses of action.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Build a test fixture to allow quick evaluation of outcomes for various trials.&lt;/li&gt;&#xA;&lt;li&gt;Experiment with dilligence. Don't leave any potentially interesting case untried.&lt;/li&gt;&#xA;&lt;li&gt;Label as much production data as reasonably possible and use that as well as the canned data so that the above options can be evaluated in permuted combinations with the differences in pattern distribution between the canned and live data.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-03T23:50:55.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9821" PostTypeId="2" ParentId="9816" CreationDate="2019-01-04T05:07:11.100" Score="3" Body="&lt;p&gt;For tabular Q-learning, the q-values for state s and action a are updated according to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q(s, a) \gets Q(s, a) + \alpha [(r + max_{a'} Q(s', a')) - Q(s,a)]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate and &lt;span class=&quot;math-container&quot;&gt;$(r + max_{a'} Q(s', a')) - Q(s,a)$&lt;/span&gt; is the difference between the current estimate of the q-value, &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;, and the target, &lt;span class=&quot;math-container&quot;&gt;$r + max_{a'} Q(s', a')$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The target q-value is based on the greedy policy, not the exploratory policy. Q-learning is theoretically guaranteed to converge to the optimal policy for any behavior policy (like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy) that is guaranteed to visit every state and action pair an infinite number of times.  See Section 6.5 of &lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Sutton and Barto book&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to Q-learning, the target q-value for SARSA is &lt;span class=&quot;math-container&quot;&gt;$r + Q(s', a')$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; is chosen from an exploratory behavior policy like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy. For SARSA the learned q-values are dependent on the behavior policy and therefore not guaranteed to converge to the optimal policy. A behavior policy that intentionally acted randomly for multiple consecutive actions, as in your example Asteroids exploratory policy, would likely lead to learning different q-values than would be learned for an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy behavior policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately Q-learning's theoretical guarantees of convergence to an optimal policy go out the window when nonlinear function approximation is introduced, as is the case for deep neural networks. Nevertheless, in the &lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-Networks paper&lt;/a&gt;, the q-value function is updated using a target value based on the maximum q-value for the next state. Specifically, if &lt;span class=&quot;math-container&quot;&gt;$Q(s, a, w)$&lt;/span&gt; is a q-value function parameterized by weights &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;, then the weights are updated by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w \gets w + \alpha [(r + max_{a'} Q(s', a', w^-)) - Q(s, a, w)] \nabla_w Q(s,a,w)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$w^-$&lt;/span&gt; are the parameters of the target network used to stabilize training. (See the paper for more details). This update rule is chosen to minimizes the loss function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(w) = E[(r + max_{a'} Q(s', a', w^-)) - Q(s, a, w)]^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your own implementation, it may be helpful to see a code example of the Deep Q-Networks parameter updates.  A tensorflow implementation is available in the function &lt;code&gt;build_train&lt;/code&gt; in the OpenAI Baselines &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/deepq/build_graph.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepQ&lt;/a&gt; code.&lt;/p&gt;&#xA;" OwnerUserId="15444" LastActivityDate="2019-01-04T05:07:11.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9822" PostTypeId="2" ParentId="5022" CreationDate="2019-01-04T05:17:30.303" Score="0" Body="&lt;p&gt;The short answer is that no one knows - here is just my take on it.&#xA;Note here that this all sounds SciFi as hell. This didn't occur yet to any degree but is in the domain of the physical possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will divide this into Hardware, Software and Economics. Economics is just a replacement for the name of a concept which I am unaware of - that is the interaction between A(G)I's, companies and companies of AGI's. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the growth of GPU's outpaces moores law. Note also that the growth of the computational power throws at ML outpaces moores law, too &lt;a href=&quot;https://blog.openai.com/ai-and-compute/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI and Compute&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Software&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Software improvements are certainly possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of software improvements can be found in &lt;a href=&quot;https://www.slideshare.net/avturchin/levels-of-the-selfimprovement-of-the-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Levels of the self-improvement of the AI&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The software improvements of the core A(G)I mechanisms, algorithms and datastructures depend on&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;a) The intelligence and optimization power of the used A(G)I(s).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incapable A(G)I's may either fail completely to improve the algorithms and datastructures and intelligence or they cap out at a certain (sub)human level. See &lt;a href=&quot;http://agi-conf.org/2015/wp-content/uploads/2015/07/agi15_yampolskiy_analysis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis of Types of Self - Improving Software&lt;/a&gt; &lt;a href=&quot;https://medium.com/@RobertW89/timeline-and-analysis-of-existing-attempts-of-recursive-self-improving-rsi-software-systems-5fdf409ed74e&quot; rel=&quot;nofollow noreferrer&quot;&gt;Timeline and analysis of existing attempts of recursive self improving (RSI) software systems&lt;/a&gt; for a detailed analysis and a analysis of most publically known attempts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;b) The already done optimizations of the used software (of the A(G)I(s)) and the format in which the programs are encoded.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All current ML frameworks make certain assumptions and tradeoffs between (software) optimizations and ease of use for human programmers. The efficiency of lets say Tensorflow can be greatly improved for most special cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incapable A(G)I's may require the description of the algorithms with DSL's because they fail to reason about contemporary programming languages (C, C++, Java, etc) and frameworks (Tensorflow, etc.). Further they may require speciallized training and/or software to develop software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;c) The mental architecture of the A(G)I's&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some architectures might be easier to improve by the A(G)I's than others. One extreme is where the AGI was built by an specialized AI - here everything is &quot;in the mind&quot; of the AGI. One other extreme is a architecture which is based on a biological brain. This might be extremely hard or impossible to improve uppon because natural evolution had done all optimizations it could do in 100's of millions of years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hardware&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specialized Hardware is faster than Software. Examples for this are uses of FPGA's, GPU's and custom ASICs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speedup of FPGA code depends on the used FPGA and the algorithm and datastructures as well as the implementation itself. Typically in the range on ~100x - 10000x (?) vs CPU's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speedups of GPU code depends on a lot of factors - but are usually between 5x to serveral 100 x. This all depends on the optimizations done on the CPU software which are used for comparision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A example for custom ASIC's are &quot;Tensor Processing Units&quot;. The speedup of TPU's today is ~2.8x against GPU's - depending on the workload &lt;a href=&quot;https://blog.riseml.com/benchmarking-googles-new-tpuv2-121c03b71384&quot; rel=&quot;nofollow noreferrer&quot;&gt;Benchmarking Google’s new TPUv2&lt;/a&gt;. Nothe here that custom ASIC's may be more energy efficient than CPU's, GPU's or FPGAs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Economics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets assume that a sufficiently developed AGI is used by entities which ressemble companies of today (2019). &#xA;We can put entities on a graph where the X axis is the amount of work done (by some computable or not computable metric) and the Y axis is the ration between human labor and work done by AGI's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Purely human driven entities are essentially like companies of today - they may use weak AI for some tasks (like everyone is doing today - weak AI is everywhere).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a interesting regime where a lot of work is done by humans and AGI's. AGI's might get tasked with all sorts of tasks. Interesting tasks worth to mention here include company leadership and decision making for the company.&#xA;This comes with it's own social problems inside a mixed company - who would like to work for an AGI where the AGI is the boss and decides on the fate of the company.&#xA;A mixed version of the &quot;AGI leadership&quot; idea are humans which use AGI's as an oracle to make decisions.&#xA;There are lots of possibilities of only this interaction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other extreme regime is where almost all or all work is done by AGI's. Human operators may or may not exist and excert power over the AGI's.&#xA;Note here that many AGI's may act like a single AGI. One other interesting effect is that AGI's may decide to split off and form their own &quot;company&quot; - or they might decide to fuse.&#xA;This occurs today with contemporary companies but at a slow pace. AGI's may do this in a very short time (minutes, seconds, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there will be basically three different company like entities - purely human driven ones, ones where there is a mix of AGI's and humans and the third category is made up of almost only AGI's.&lt;/p&gt;&#xA;" OwnerUserId="21141" LastActivityDate="2019-01-04T05:17:30.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9824" PostTypeId="1" CreationDate="2019-01-04T09:28:17.163" Score="0" ViewCount="180" Body="&lt;p&gt;I am looking for an non-ML method for two chat bots to communicate to each other about a specific topic. I am looking for an &quot;explainable AI&quot; method, as opposed to a &quot;black-box&quot; one (like a neural network).&lt;/p&gt;&#xA;" OwnerUserId="20378" LastEditorUserId="2444" LastEditDate="2019-05-01T17:03:59.017" LastActivityDate="2019-05-01T17:03:59.017" Title="How do I create chatbots without machine learning?" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9825" PostTypeId="2" ParentId="9824" CreationDate="2019-01-04T11:15:15.480" Score="1" Body="&lt;p&gt;The easiest non-ML way would be to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite-state_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite state machine&lt;/a&gt;. You could model various states of your conversation topics, and certain utterances of your bots could advance the bot's internal model along different paths. The complexity depends on the complexity of the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then enhance the transitions with probabilities, and later move on towards ML by transforming it into an &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMM&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, even simple topics will probably lead to fairly complex state machines. But you should be able to keep track of what is going on in your conversation nevertheless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: just to make it a bit clearer, I was thinking along the lines of having states for particular stages in the conversation. You could either have one model for the whole conversation, or one per participant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, there would be a state 'greeting'. Possible transitions would be to a further state 'greeting' (the response of the person who has been greeted), or that could be skipped to states such as 'statement', 'question', etc. 'Question' would have transitions to 'answer', 'ignore question', 'counter/clarification question' etc. The level of detail depends on your application.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-01-07T09:26:30.040" LastActivityDate="2019-01-07T09:26:30.040" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9826" PostTypeId="2" ParentId="9808" CreationDate="2019-01-04T12:43:57.207" Score="2" Body="&lt;p&gt;Suppose that you have already search a part of the complete search tree, for example the complete left half. This may not yet give you the true game-theoretic value for the root node, but it can already give you some bounds on the game-theoretic value that the player to play in the root node (let's say, the max player) can guarantee by moving into that part of the search tree. Those bounds / guarantees are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;: the &lt;strong&gt;minimum&lt;/strong&gt; score that the &lt;strong&gt;maximizing&lt;/strong&gt; player already knows it can &lt;strong&gt;guarantee&lt;/strong&gt; if we move into the part of the search tree searched so far. Maybe it can still do better (get higher values) by moving into the unsearched part, but it can already definitely get this value.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;: the &lt;strong&gt;maximum&lt;/strong&gt; score that the &lt;strong&gt;minimizing&lt;/strong&gt; player already knows it can &lt;strong&gt;guarantee&lt;/strong&gt; if we moves into the part of the search tree searched so far. Maybe it can still do better (get lower values) by moving into the unsearched part, but it can already definitely get this value.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The intuitive idea behind alpha-beta pruning is to prune chunks of the search tree that become uninteresting for either player because they already know they can guarantee better based on the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; bounds.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For a simple example, suppose &lt;span class=&quot;math-container&quot;&gt;$\alpha = 1$&lt;/span&gt;, which means that the maximizing player already has explored a part of the search tree such that it can &lt;strong&gt;guarantee at least&lt;/strong&gt; a value of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; by playing inside that part (the minimizing player has no options inside that entire tree to reduce the value below &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, if the maximizing player plays optimally in that part).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that, in the current search process, we have arrived at a node where the minimizing player is to play, and it has a long list of child nodes. We evaluate the first of those children, and find a value of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. This means that, under the assumption that we reach this node, the minimizing player can already guarantee a value of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (and possibly get even lower, we didn't evaluate the other children yet). But this is worse (for the maximizing player) than the &lt;span class=&quot;math-container&quot;&gt;$\alpha = 1$&lt;/span&gt; bound we already had. Without evaluating any of the other children, we can already tell that this part of the search tree is uninteresting, that the maximizing player would make sure that we never end up here, so we can prune the remaining children (which could each have large subtrees below them).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-04T12:43:57.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9828" PostTypeId="1" AcceptedAnswerId="9942" CreationDate="2019-01-04T13:39:22.473" Score="10" ViewCount="1468" Body="&lt;p&gt;There are several activation functions, such as ReLU, sigmoid or &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt;. What happens when I mix activation functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently found that Google has developed Swish activation function which is (x*sigmoid). By altering activation function can it increase accuracy on small neural network problem such as XOR problem?&lt;/p&gt;&#xA;" OwnerUserId="21143" LastEditorUserId="2444" LastEditDate="2019-05-15T15:16:53.050" LastActivityDate="2019-05-15T15:16:53.050" Title="What happens when I mix activation functions?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;activation-function&gt;&lt;relu&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9829" PostTypeId="1" AcceptedAnswerId="9843" CreationDate="2019-01-04T14:11:38.477" Score="2" ViewCount="218" Body="&lt;p&gt;I am not sure if I understood the q learning algorithms correctly. &#xA;Therefore I would give a concrete example and ask if someone can tell me how to update the q value correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I initialized a Neural Network with random weights. It shall henceforth evaluate the Q Value for all possible actions(4) given a State S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the following happens. The agent is playing and is exploring. &#xA;For 3 steps the Q Values evaluated were:&#xA;&lt;em&gt;(0,-1,-5,0), (0,-1,0,0), (0,-.6,0,0)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward given was: &lt;em&gt;0,0,1&lt;/em&gt;&#xA;The action took were: &lt;em&gt;(1.,1.,1.)&lt;/em&gt;&#xA;In the random walk example (same reward given), it was: &lt;em&gt;(1.,2.,3.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what are the new Q - Values, assuming a discount factor of 0.99 and the learning rate 0.1?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The States for Simplicity are only one number: &lt;em&gt;1,1.3,2.4&lt;/em&gt; Where 2.4 is the state who ends the game...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same example holds for exploiting. Is the algorithm the same here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you see my last implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    public void rlearn(ArrayList&amp;lt;Tuple&amp;gt; tupels, double learningrate, double discountfactor) {&#xA;&#xA;    //newQ = sum of all rewards you have got through&#xA;    for(int i = tupels.size()-1; i &amp;gt; 0; i--) {&#xA;        MLData in = new BasicMLData(45);&#xA;        MLData out = new BasicMLData(5);&#xA;&#xA;        //Add State as in&#xA;        int index = 0;&#xA;        for(double w : tupels.get(i).statefirst.elements) {&#xA;            in.add(index++, w);&#xA;        }&#xA;&#xA;        //Now start updating Q - Values &#xA;        double qnew = 0;&#xA;        if(i &amp;lt;= tupels.size()-2){&#xA;            qnew = tupels.get(i).rewardafter + discountfactor*qMax(tupels.get(i+1));&#xA;        } else {&#xA;            qnew = tupels.get(i).rewardafter;&#xA;        }&#xA;&#xA;        tupels.get(i).qactions.elements[tupels.get(i).actionTaken] = qnew;&#xA;        //Add Q Values as out&#xA;        index = 0;&#xA;        for(double w : tupels.get(i).qactions.elements) {&#xA;            out.add(index++, w);&#xA;        }&#xA;         bigset.add(in, out);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: This is the qMax - function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    private double qMax(Tuple tuple) {&#xA;    double max = Double.MIN_VALUE;&#xA;    for(double w : tuple.qactions.elements) {&#xA;        if(w &amp;gt; max) {&#xA;            max = w;&#xA;        }&#xA;    }&#xA;    return max;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="19062" LastEditDate="2019-01-05T10:58:07.197" LastActivityDate="2019-01-07T21:44:52.540" Title="Concrete Example for Q Learning" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9830" PostTypeId="2" ParentId="9812" CreationDate="2019-01-04T16:20:14.967" Score="4" Body="&lt;p&gt;Usually when people write about having a high-dimensional state space, they are referring to the state space actually used by the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose my state is a high dimensional vector of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; length where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is a huge number. Let's say I solve this task using &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning and I fix my state space to &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; vectors each of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; dimensions. &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning can easily work with these settings as we need only a table of dimensions &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; x number of actions. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this case, I'd argue that the &quot;feature vectors&quot; of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; are quite useless. If there are effectively only &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; unique states (which may each have a very long feature vector of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;)... well, it seems like a bad idea to make use of those long feature vectors, just using the states as identity (i.e. a tabular RL algorithm) is much more efficient. If you end up using a tabular approach, I wouldn't call that a high-dimensional space. If you end up using function approximation with the feature vectors instead, that would be a high-dimensional space (for large &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let's also say I have a state space of infinite vectors but each vector is now of length &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; i.e. very small dimensional vectors. Would it make sense to use DQN in these settings ? Should this state-space be called high dimensional or low dimensional ? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This would typically be referred to as having a low-dimensional state space. Note that I'm saying low-&lt;strong&gt;dimensional&lt;/strong&gt;. The &lt;strong&gt;dimensionality&lt;/strong&gt; of your state space / input space is low, because it's &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; and that's typically considered to be a low value when talking about dimensionality of input spaces. The state space may still have a large &lt;strong&gt;size&lt;/strong&gt; (that's a different word from &lt;strong&gt;dimensionality&lt;/strong&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for whether DQN would make sense in such a setting.. maybe. With such low dimensionality, I'd guess that a linear function approximator would often work just as well (and be much less of a pain to train). But yes, you can use DQN with just 2 input nodes.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-04T16:20:14.967" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9832" PostTypeId="2" ParentId="9819" CreationDate="2019-01-04T16:40:00.653" Score="2" Body="&lt;p&gt;In terms of the normal use cases for machine learning, the equation does not have much utility, because:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider we have a curve &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; now if one wishes to  . . . &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In most AI problems, we don't usually have such a curve as input that can be treated analytically. For instance, there is no such input curve to describe a natural image received by a sensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the vast majority of cases in AI problems, the form of inputs - whether it is images, text, mapping data, robotic telemetry, is going to be multi-dimensional discrete samples from highly complex functions where we don't know the analytical form and can only construct approximations from a set of basis functions. The resulting combination of basis functions could be treated as continuous, integrated etc, but as it would have been constructed from discrete data, the end result would be a lot of computation to end up with something probably less accurate than working direct with the discrete samples. In a lot of cases, the raw data is discrete by definition (e.g. whether someone clicked on a link or replied to a message), so the form of &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; would be discrete by definition of the problem, and calculus not really applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There &lt;em&gt;might&lt;/em&gt; be some interesting use cases in analog signal processing. Using your formula or a variation of it for instance it should be possible to create an analog neural network learning system - a robotic brain that worked with continuous signals and only contained analog components. Such a system would have some interesting advantages - speed of processing, probably low power consumption compared to digital approach, but the accuracy and precision might be lower. E.g. imagine a bot that could steer towards/away from light sources (compared to a digital one that might recognise the faces of people and steer towards/away from them). I would not be at all surprised to find that someone had done just that already, although I am not sure how to search for it.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-04T16:45:50.137" LastActivityDate="2019-01-04T16:45:50.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9833" PostTypeId="2" ParentId="3879" CreationDate="2019-01-04T17:02:39.333" Score="2" Body="&lt;p&gt;The error in the code is simply having a &lt;span class=&quot;math-container&quot;&gt;$+$&lt;/span&gt; rather than a &lt;span class=&quot;math-container&quot;&gt;$-$&lt;/span&gt; sign. Line 4 of the algorithm says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E\left[ g^2 \right]_t = \rho E\left[ g^2 \right]_{t - 1} + (1 - \rho) g_t^2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but your code implements (note the &lt;span class=&quot;math-container&quot;&gt;$+$&lt;/span&gt; inside the brackets at the end):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E\left[ g^2 \right]_t = \rho E\left[ g^2 \right]_{t - 1} + (1 + \rho) g_t^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A correct implementation, with only that minor change, would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;&#xA;Eg = Ex = 0&#xA;p = 0.95&#xA;e = 1e-6&#xA;x = 1&#xA;history = [x]&#xA;&#xA;for t in range(100):&#xA;    g = 2*x&#xA;    Eg = p*Eg + (1-p)*g*g&#xA;    Dx = -(math.sqrt(Ex + e) / math.sqrt(Eg + e)) * g&#xA;    Ex = p*Ex + (1-p)*Dx*Dx&#xA;    x = x + Dx&#xA;    history.append(x)&#xA;&#xA;print(history)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On my end, that code leads to a value of approximately &lt;span class=&quot;math-container&quot;&gt;$0.597$&lt;/span&gt;. It looks like, with these hyperparameters, you'll need more like 400 or 500 iterations to get really close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, but it steadily gets there.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, the paper claims that the update step Δx will have the same unit as x, if x has some hypothetical unit. While this is probably a desireable property, it is as far as I'm concerned not true, since the premise that RMS[Δx] has the same unit as x is incorrect to begin with, since RMS[Δx]_0 = sqrt(E[Δx]_0 + ϵ) = sqrt(0 + ϵ) which is a unitless constant, so all Δx become unitless rather than having the same unit as x. (Correct me if I'm wrong.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Suppose that we use the symbol &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; for this hypothetical unit that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; has. Line 5 of the algorithm says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\Delta x_t &amp;amp;= - \frac{\text{RMS}\left[ \Delta x \right]_{t-1}}{\text{RMS} \left[ g \right]_t} g_t\\&#xA;&amp;amp;= - \frac{\sqrt{E\left[ \Delta x^2 \right]_{t-1} + \epsilon}}{\sqrt{E\left[ g^2 \right]_{t} + \epsilon}} g_t.&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can get rid of the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; terms, their addition does not change the unit of whatever they are added to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$- \frac{\sqrt{E\left[ \Delta x^2 \right]_{t-1}}}{\sqrt{E\left[ g^2 \right]_{t}}} g_t.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you stated correctly, for the very first iteration, we have &lt;span class=&quot;math-container&quot;&gt;$E\left[ \Delta x^2 \right]_{t-1} = 0$&lt;/span&gt;. Technically in the very first iteration it could have any unit we like (or no unit at all), based on whatever unit we choose to assign to the &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; constant it is initially set to. Let's just say we assign it the unit &lt;span class=&quot;math-container&quot;&gt;$u^2$&lt;/span&gt; (by saying that that is the unit of the &lt;span class=&quot;math-container&quot;&gt;$0^2$&lt;/span&gt; constant we initialize it to). This is convenient because it allows us to immediately figure out the unit in &lt;strong&gt;all cases&lt;/strong&gt; rather than just the &lt;span class=&quot;math-container&quot;&gt;$t = 0$&lt;/span&gt; case, this is the unit that it has to have if we also still want things to work out for &lt;span class=&quot;math-container&quot;&gt;$t &amp;gt; 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt; has a unit &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{u}$&lt;/span&gt;, which means that &lt;span class=&quot;math-container&quot;&gt;$E[g^2]_t$&lt;/span&gt; has a unit &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{u^2}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\sqrt{E[g^2]_t}$&lt;/span&gt; then again has the unit &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{u}$&lt;/span&gt;. If we replace all the quantities by their units, we then get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u}{\frac{1}{u}} \times \frac{1}{u} = u.$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-01-06T19:06:49.870" LastActivityDate="2019-01-06T19:06:49.870" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9834" PostTypeId="1" CreationDate="2019-01-04T18:46:31.110" Score="4" ViewCount="234" Body="&lt;p&gt;What are the strengths of the Hierarchical Temporal Memory model compared to competing models such as 'traditional' Neural Networks as used in deep learning? And for those strengths are there other available models that aren't as bogged down by patents?&lt;/p&gt;&#xA;" OwnerUserId="21155" LastEditorUserId="2444" LastEditDate="2019-07-28T10:05:11.007" LastActivityDate="2019-07-28T14:28:58.500" Title="What are the strengths of the Hierarchical Temporal Memory model compared to competing models?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;htm&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9838" PostTypeId="1" CreationDate="2019-01-04T23:45:15.573" Score="4" ViewCount="537" Body="&lt;p&gt;After reading an excellent BLOG post &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Deep Reinforcement Learning: Pong from Pixels&lt;/strong&gt;&lt;/a&gt; and playing with the code a little, I've tried to do something simple: use the same code to train a logical XOR gate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But no matter how I've tuned hyperparameters, the reinforced version does not converge (gets stuck around -10). What am I doing wrong? Isn't it possible to use Policy Gradients, in this case, for some reason?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The setup is simple:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3 inputs (1 for bias=1, x, and y), 3 neurons in the hidden layer and 1 output.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;game&lt;/em&gt; is passing all 4 combinations of x,y to the RNN step-by-step, and after 4 steps giving a reward of +1 if all 4 answers were correct, and -1 if at least one was wrong.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;episode&lt;/em&gt; is 20 &lt;em&gt;games&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The code (forked from original and with minimal modifications) is here: &lt;a href=&quot;https://gist.github.com/Dimagog/de9d2b2489f377eba6aa8da141f09bc2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/Dimagog/de9d2b2489f377eba6aa8da141f09bc2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;P.S. Almost the same code trains XOR gate with &lt;em&gt;supervised&lt;/em&gt; learning in no time (2 sec).&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="20941" LastEditorUserId="20941" LastEditDate="2019-12-03T21:43:49.923" LastActivityDate="2019-12-03T21:43:49.923" Title="How to train a logical XOR with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9842" PostTypeId="1" CreationDate="2019-01-05T04:51:08.610" Score="1" ViewCount="62" Body="&lt;p&gt;I'm learning machine learning by looking through other people's kernel on Kaggle, specifically this &lt;a href=&quot;https://www.kaggle.com/raghuchaudhary/mushroom-classification/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mushroom Classification kernel&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The author first applied PCA to the transformed indicator matrix. He only used 2 principal components for visualization later. Then I checked how much variance it has maintained, and found out that only 16% variance is maintained.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;in [18]: pca.explained_variance_ratio_.cumsum()&#xA;out[18]: array([0.09412961, 0.16600686])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the test result with 90% accuracy suggests it works well.&lt;/p&gt;&#xA;&lt;p&gt;If variance stands for information, then how can the ML model work well when so much information is lost?&lt;/p&gt;&#xA;" OwnerUserId="21166" LastEditorUserId="2444" LastEditDate="2021-04-01T11:15:17.730" LastActivityDate="2021-05-01T12:10:05.893" Title="Why does PCA work well while the total variance retained is small?" Tags="&lt;machine-learning&gt;&lt;principal-component-analysis&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9843" PostTypeId="2" ParentId="9829" CreationDate="2019-01-05T05:26:57.870" Score="3" Body="&lt;p&gt;Most Deep Q-learning implementations I have read are based on &lt;a href=&quot;https://deepmind.com/research/dqn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-Networks&lt;/a&gt; (DQN).  In DQN, the q-value network maps an input state to a vector of q-values, one for each action:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q(s, \mathbf{w}) \to \mathbf{v}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the input state from the environment, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; are the parameters of the neural network, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt; is a vector of q-values, where &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; is the estimated q-value of the ith action. In the &lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto book&lt;/a&gt;, the q-value function is written as &lt;span class=&quot;math-container&quot;&gt;$Q(s, a, \mathbf{w})$&lt;/span&gt;, which corresponds to the network output for action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike tabular Q-learning, Deep Q-learning updates the parameters of the the neural network according to the gradients of the loss function with respect to the parameters. DQN uses the loss function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(\mathbf{w}) = [(r + \gamma max_{a'} Q(s', a', \mathbf{w^-})) - Q(s, a, \mathbf{w})]^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the discount rate, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the selected action (either greedily or randomly for an &lt;span class=&quot;math-container&quot;&gt;$epsilon$&lt;/span&gt;-greedy behavior policy), &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the next state, &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; is the argmax action for the next state, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w^-}$&lt;/span&gt; is an older version of the network weights &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; that is used to help stabilize training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In deep Q-learning, training directly updates parameters, not q-values. Parameters are updated by taking a small step in the direction of the gradient of the loss function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{w} \gets \mathbf{w} + \alpha [(r + \gamma max_{a'} Q(s', a', \mathbf{w^-})) - Q(s, a, \mathbf{w})] \nabla_w Q(s, a, \mathbf{w})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In frameworks like tensorflow or pytorch the derivative is calculated automatically by giving the loss function and model parameters directly to an optimizer class which uses some variation of mini-batch gradient descent. In eagerly executed tensorflow updating the parameters for a mini-batch might look something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;batch = buffer.sample(batch_size)&#xA;observations, actions, rewards, next_obervations = batch&#xA;&#xA;with tf.GradientTape() as tape:&#xA;    qvalues = model(observations, training=True)&#xA;    next_qvalues = target_model(next_obervations)&#xA;    # r + max_{a'} Q(s', a') for the batch&#xA;    target_qvalues = rewards + gamma * tf.reduce_max(next_qvalues, axis=-1)&#xA;    # Q(s, a) for the batch&#xA;    selected_qvalues = tf.reduce_sum(tf.one_hot(actions, depth=qvalues.shape[-1]) * qvalues, axis=-1)&#xA;    loss = tf.reduce_mean((target_qvalues - selected_qvalues)**2)&#xA;&#xA;grads = tape.gradient(loss, model.variables)&#xA;optimizer.apply_gradients(zip(grads, model.variables))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Though I am not familiar with the Encog neural network framework you are using, based on the example &lt;code&gt;Brain.java&lt;/code&gt; file from your &lt;a href=&quot;https://github.com/SuchtyTV/RLearningBird/blob/master/src/main/java/rlgame/Brain.java&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repo&lt;/a&gt; and Chapter 5 of the Encog &lt;a href=&quot;https://s3.amazonaws.com/heatonresearch-books/free/Encog3Java-User.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;User Manual&lt;/a&gt; and the Encog neural network &lt;a href=&quot;https://github.com/encog/encog-java-examples/tree/master/src/main/java/org/encog/examples/neural&quot; rel=&quot;nofollow noreferrer&quot;&gt;examples on Github&lt;/a&gt; it looks like weights are updated as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A training set is constructed from pairs of &lt;strong&gt;input&lt;/strong&gt; and &lt;strong&gt;target output&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;A &lt;code&gt;Propagation&lt;/code&gt; instance, &lt;code&gt;train&lt;/code&gt;, is constructed with a network and training set. Different subclasses of &lt;code&gt;Propagation&lt;/code&gt; use different loss functions to update the network parameters.&lt;/li&gt;&#xA;&lt;li&gt;The method &lt;code&gt;train.iterate()&lt;/code&gt; is called to run the network on the inputs, calculate the loss between the network outputs and target outputs, and update the weights according to the loss.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For DQN, a training set is constructed from a random sample from the experience replay buffer to help stabilize training. A training set could also be the trajectory of an episode, which is what the &lt;code&gt;tupels&lt;/code&gt; argument in the example code of the question appears to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;input&lt;/strong&gt; would be the &lt;code&gt;statefirst&lt;/code&gt; member of each element of &lt;code&gt;tupels&lt;/code&gt;. Since the network produces a vector of q-values, the &lt;strong&gt;target output&lt;/strong&gt; must also be a vector of q-values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The target output element for the selected action is &lt;span class=&quot;math-container&quot;&gt;$r + \gamma max_{a'} Q(s', a', \mathbf{w^-})$&lt;/span&gt;,  In the example code of the question, this is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double qnew = 0;&#xA;if(i &amp;lt;= tupels.size()-2){&#xA;    qnew = tupels.get(i).rewardafter + discountfactor*qMax(tupels.get(i+1));&#xA;} else {&#xA;    qnew = tupels.get(i).rewardafter;&#xA;}&#xA;tupels.get(i).qactions.elements[tupels.get(i).actionTaken] = qnew&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The target output elements for actions that were not selected should be &lt;span class=&quot;math-container&quot;&gt;$Q(s, b, \mathbf{w})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is one of the non-selected actions.  This should have the effect of ignoring the q-values of non-selected actions by making the network output equal to the target output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;So what are the new Q - Values, assuming a discount factor of 0.99 and the learning rate 0.1?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you mean &lt;strong&gt;target outputs&lt;/strong&gt; by the new Q - Values, and given the trajectory of actions, &lt;code&gt;(1, 1, 1)&lt;/code&gt;, and q-value vectors from the question, the concrete target outputs are &lt;code&gt;(0, 0 + 0.99 * 0, -5, 0)&lt;/code&gt;, &lt;code&gt;(0, 0 + 0.99 * 0, 0, 0)&lt;/code&gt;, and &lt;code&gt;(0, 1 + 0, 0, 0)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15444" LastActivityDate="2019-01-05T05:26:57.870" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9847" PostTypeId="2" ParentId="9842" CreationDate="2019-01-05T10:36:46.887" Score="0" Body="&lt;p&gt;Because it selects both &lt;code&gt;Xtrain&lt;/code&gt; and &lt;code&gt;Xtest&lt;/code&gt; from the space of two selected principal components. Hence, the 90% accuracy is in that 2-D selected space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This fact that the ratio in PCA stands the information, depends on the distribution of the data and it's not true at all. &lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-01-05T10:43:41.540" LastActivityDate="2019-01-05T10:43:41.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9848" PostTypeId="1" AcceptedAnswerId="9851" CreationDate="2019-01-05T10:56:45.170" Score="1" ViewCount="174" Body="&lt;p&gt;Let's say I have a &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; pixel of grayscale picture, where there is one edge such that the left pixel contains a value, 30, and the right pixels contain a value 0 (in red below). And for edge detection I have zero-padded the input image and then used the Sobel vertical filter to find out the vertical edges and apply &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt; to the output. The output is a &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; matrix with all pixel values &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. So that should mean there is no edge in the picture whereas in actual case it has one. Where am I going wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/464WN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/464WN.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21172" LastEditorUserId="2444" LastEditDate="2019-06-05T21:40:51.747" LastActivityDate="2019-06-05T21:42:11.947" Title="Understanding the application of Sobel kernel followed by ReLU to a zero-padded image" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9849" PostTypeId="1" CreationDate="2019-01-05T11:08:11.157" Score="1" ViewCount="56" Body="&lt;p&gt;Consider the following loss function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(\mathbf{w}) = [(r + \gamma max_{a'} Q(s', a', \mathbf{w^-})) - Q(s, a, \mathbf{w})]^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Q(s, a, \mathbf{w^-})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q(s, a, \mathbf{w})$&lt;/span&gt; are represented as neural networks, where &lt;span class=&quot;math-container&quot;&gt;$w^-$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; are the corresponding weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do you calculate &lt;span class=&quot;math-container&quot;&gt;$max_{a'} Q(s', a', \mathbf{w^-})$&lt;/span&gt;? Do you really need to hold always an older version of the network? If yes, why and how old should it be?&lt;/p&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="2444" LastEditDate="2019-02-15T20:21:10.120" LastActivityDate="2019-02-15T20:21:10.120" Title="How do I calculate $max_{a′}Q(s′,a′,w−)$ when it is represented as a neural network?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;loss-functions&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9851" PostTypeId="2" ParentId="9848" CreationDate="2019-01-05T12:35:59.827" Score="1" Body="&lt;p&gt;I assume that you would like to use convolution with padding with the same size of output matrix as this picture. You messed up calculations with convolution with &quot;full padding&quot;. If we imagine these two matrices as windows that slide on each other, you can see that the filter is symmetrically inverted. I used a little bit different filter to show you in a better way how it works (I changed the last row to [-3, 0 3]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming these matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ENEWg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ENEWg.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should add to your picture matrix two rows and columns of zeros:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A36DE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A36DE.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can start matrix multiplication, but notice that the filter is symmetrically inverted. The result shown as matrix &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; is the convolution with &quot;full padding&quot;. The &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; matrix in the middle is a result of convolution with the &quot;same padding&quot;, that you requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qfaRg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qfaRg.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s4D0h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s4D0h.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some iterations later:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ARh69.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ARh69.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And later:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VPWHV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VPWHV.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VOYpc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VOYpc.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convolution with the same size, the result will be the small &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; matrix in the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JdwHl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JdwHl.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After using the ReLU function, the result will be exactly the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So with using of your filter, the result would look like [[0,90], [0,90]].&lt;/p&gt;&#xA;" OwnerUserId="21171" LastEditorUserId="2444" LastEditDate="2019-06-05T21:42:11.947" LastActivityDate="2019-06-05T21:42:11.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9852" PostTypeId="2" ParentId="9849" CreationDate="2019-01-05T18:32:17.463" Score="1" Body="&lt;p&gt;You calculate the max by calculating your estimates for all possible actions for the next state, and taking the highest value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The details depend a little on your neural network architecture:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a network that takes the state vector as input and outputs all possible action values &lt;span class=&quot;math-container&quot;&gt;$(\hat{q}(s,a_0), \hat{q}(s,a_1), \hat{q}(s,a_2) ...)$&lt;/span&gt;, then you can run it once forward with the &lt;code&gt;next_state&lt;/code&gt; as input to getthe array &lt;span class=&quot;math-container&quot;&gt;$(\hat{q}(s',a_0), \hat{q}(s',a_1), \hat{q}(s',a_2) ...)$&lt;/span&gt;, and take the maximum element (you don't need to care which action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; caused it). You will then have the problem that you now have a loss for &lt;span class=&quot;math-container&quot;&gt;$Q(s, a, \mathbf{w})$&lt;/span&gt; for the single action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; just taken, but no data any of the alternative actions. The loss for these alternative actions needs to be set to zero - if you are training the NN using a normal supervised learning approach, that means you need to keep the full output of the network that you ran forward to calculate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a, \mathbf{w})$&lt;/span&gt; then substitute in this new estimated value against the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and train using this modified vector.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a network that takes the state vector and action combined as input and outputs a single estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a)$&lt;/span&gt;, then you have to run that network once for each possible action from the next state and take the maximum value. You would typically do this as a small batch prediction for better performance. In this case your training data is simple to construct as you only have the loss for and train the network against one state/action combination. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Overall the first option (all action values at once) is usually a lot more efficient, but slightly more complex to code the training routine.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;do you really need to hold always an older version of the network. If yes why and how old should it be?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You don't have to, but it is highly advisable to have this target network (so called because it helps generate your TD Target values), because Q learning using neural networks is often unstable. This is due to the bootstrap calculations where estimates are based on other estimates plus a little bit of observed data at each step. There is a strong possibility for runaway feedback due to training a neural network on something that includes its own output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How old should it be? That's sadly a hyper-parameter of the architecture that you will need to establish through experiment on each new problem. I have worked with maximum age values from 100 to 10000 in my own simple experiments. Note this is not usually a rolling age - you don't keep 1000 copies of the network weights. Just keep one frozen copy, and after N steps replace it with a copy of the most recent one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternative to this copy/freeze/copy approach is to update the target network towards the learning network on every step by a small factor. E.g. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}^{\_} = (1 - \beta)\mathbf{w}^{\_} + \beta \mathbf{w}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; might be &lt;span class=&quot;math-container&quot;&gt;$0.001$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, you should be using experience replay for training data, and not training directly online. The combination of experience replay and using a frozen or slowly adapting target network makes a large difference to the stability of deep Q learning in practice.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-05T19:24:38.983" LastActivityDate="2019-01-05T19:24:38.983" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9854" PostTypeId="1" AcceptedAnswerId="9865" CreationDate="2019-01-05T21:24:54.670" Score="1" ViewCount="579" Body="&lt;p&gt;Consider this game state:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wfOrF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wfOrF.png&quot; alt=&quot;GameState&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;d5 captures c6&lt;/p&gt;&#xA;&lt;p&gt;Quiescence search returns about 8.0 as evaluation because after dxc6 and bxc6 Qxd6 would be played (then Qxd6 by black). A normal player would not play this move but quiescence search includes it in the evaluation and it would result in this end state:&lt;a href=&quot;https://i.stack.imgur.com/ACI6V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ACI6V.png&quot; alt=&quot;GameState after quiescence search end&quot; /&gt;&lt;/a&gt;&#xA;which would result in a huge advantage for black.&lt;/p&gt;&#xA;&lt;p&gt;Is my interpretation of quiescence search wrong?&lt;/p&gt;&#xA;" OwnerUserId="19783" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-13T20:49:46.557" Title="Does quiescence search even improve the minimax algorithm?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;chess&gt;&lt;quiescence-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9860" PostTypeId="1" CreationDate="2019-01-06T17:12:32.703" Score="2" ViewCount="151" Body="&lt;p&gt;Why is the actor-critic algorithm limited to using on-policy data? Or can we use the actor-critic algorithm with off-policy data?&lt;/p&gt;&#xA;" OwnerUserId="21180" LastEditorUserId="2444" LastEditDate="2019-02-15T19:38:43.640" LastActivityDate="2019-02-15T19:43:04.453" Title="Why is the actor-critic algorithm limited to using on-policy data?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;on-policy-methods&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9862" PostTypeId="1" AcceptedAnswerId="9872" CreationDate="2019-01-06T21:54:03.970" Score="2" ViewCount="324" Body="&lt;p&gt;Is there a ReLU-like activation function that concatenates positive and negative values? What is its name? Apparently, it doubles the output dimension.&lt;/p&gt;&#xA;" OwnerUserId="21203" LastEditorUserId="2444" LastEditDate="2020-06-04T15:24:03.367" LastActivityDate="2020-06-04T15:24:03.367" Title="Is there a ReLU-like activation function that concatenates positive and negative values?" Tags="&lt;convolutional-neural-networks&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9865" PostTypeId="2" ParentId="9854" CreationDate="2019-01-07T01:54:47.067" Score="1" Body="&lt;p&gt;Your logic is flawed because you negated &quot;stand-pat&quot; (i.e. do nothing) and alpha-beta. Let's take a look at the pseudocode (&lt;a href=&quot;https://www.chessprogramming.org/Quiescence_Search#Pseudo_Code&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.chessprogramming.org/Quiescence_Search#Pseudo_Code&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int Quiesce( int alpha, int beta ) {&#xA;    int stand_pat = Evaluate();&#xA;    if( stand_pat &amp;gt;= beta )&#xA;        return beta;&#xA;    if( alpha &amp;lt; stand_pat )&#xA;        alpha = stand_pat;&#xA;&#xA;    until( every_capture_has_been_examined )  {&#xA;        MakeCapture();&#xA;        score = -Quiesce( -beta, -alpha );&#xA;        TakeBackMove();&#xA;&#xA;        if( score &amp;gt;= beta )&#xA;            return beta;&#xA;        if( score &amp;gt; alpha )&#xA;           alpha = score;&#xA;    }&#xA;    return alpha;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your Qxd6 capture will make return a score &lt;strong&gt;far&lt;/strong&gt; below the alpha. The line:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if( score &gt; alpha )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;will prevent your blunder being reported. Instead the engine would report either &lt;code&gt;stand_pat&lt;/code&gt; (do nothing), or something like Nf3, Nc3 etc.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2019-01-07T01:54:47.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9870" PostTypeId="1" AcceptedAnswerId="9891" CreationDate="2019-01-07T11:54:32.547" Score="0" ViewCount="257" Body="&lt;p&gt;I'm trying to understand exactly what does a convnet do to what, and I have trouble finding the dimensions alongside the convolutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take VGG 16 architecture, how do I get from 224x224x3 to 112x112x64 ? (The 112 is understandable, it's the last part I don't get)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought the CNN was to apply filters/convolutions to layers (for instance, 10 different filters to channel red, 10 to green... : are they the same filters between channels ?), but obviously 64 is not divisible by 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then, how do we get from 64 to 128 ? Do we apply new filters to outputs of previous filters ? (in this case we only have 2 filters applied to previous outputs) Or is it something different ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AJo0w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AJo0w.png&quot; alt=&quot;vgg 16 archictecutre&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19094" LastActivityDate="2019-01-08T12:15:37.960" Title="Dimension after multiple convolutions in ConvNets" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9871" PostTypeId="1" CreationDate="2019-01-07T11:59:43.977" Score="2" ViewCount="40" Body="&lt;p&gt;I have time series data where I use a sliding window to detect anomalies in those windows. A sliding window is an interval of the dataset that steps one datapoint for each iteration. Datapoints are seen multiple times in this way equal to the size of the window.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, the algorithm works like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose window length: &lt;strong&gt;wl&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Learn normal data with sliding window&lt;/li&gt;&#xA;&lt;li&gt;Try to detect anomalies on test data with sliding window&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I want to keep the sliding window method since it is necessary for the performance of the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one anomaly occurs multiple times in the sliding window. When the anomaly appears in the sliding window for the first time it's on the 'right' side of the window.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we measure accuracy of anomaly detection in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could say that detecting the anomaly once in the window is enough or detect it &lt;strong&gt;wl&lt;/strong&gt; times. What's best practice?&lt;/p&gt;&#xA;" OwnerUserId="21223" LastEditorUserId="16229" LastEditDate="2019-06-05T19:34:34.647" LastActivityDate="2019-06-05T19:34:34.647" Title="Performance measure on windowed time series data" Tags="&lt;deep-learning&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9872" PostTypeId="2" ParentId="9862" CreationDate="2019-01-07T13:13:09.497" Score="0" Body="&lt;p&gt;It seems &lt;a href=&quot;https://medium.com/@danqing/a-practical-guide-to-relu-b83ca804f1f7#:~:text=Concatenated%20ReLU%20has%20two%20outputs,CReLU%20doubles%20the%20output%20dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;I have found it&lt;/a&gt;. It is called &lt;strong&gt;concatenated ReLU&lt;/strong&gt; (CReLU).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Concatenated ReLU has two outputs, one ReLU and one negative ReLU, concatenated together. In other words, for positive x it produces [x, 0], and for negative x it produces [0, x]. Because it has two outputs, CReLU doubles the output dimension.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1603.05201&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units&lt;/a&gt; (the original paper that introduced CReLU).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/crelu&quot; rel=&quot;nofollow noreferrer&quot;&gt;A TensorFlow implementation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is also Negative CReLU. It seems that the difference is only the sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{NCReLU}(x) = (\rho(x) , −\rho(−x) )$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="21203" LastEditorUserId="2444" LastEditDate="2020-06-04T15:11:43.230" LastActivityDate="2020-06-04T15:11:43.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9874" PostTypeId="2" ParentId="9870" CreationDate="2019-01-07T16:34:52.463" Score="2" Body="&lt;p&gt;The 64 here is the number of filters that are used.&#xA;The picture is kind of misleading in that it leaves out the transition of the maxpool.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jFKgV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jFKgV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a text description of the size of the features as they go through the network with the number of filters in bold.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The first 2 layers in the diagram you posted contain &lt;strong&gt;64&lt;/strong&gt; 3x3 convs&#xA;resulting in a 224x224x64 matrix of features. &lt;/li&gt;&#xA;&lt;li&gt;This is then fed into a maxpool which reduces the size to a 112x112x64 matrix. &lt;/li&gt;&#xA;&lt;li&gt;This is then fed to 3 layers of &lt;strong&gt;128&lt;/strong&gt; 3x3 convs resulting in a 112x112x128&#xA;matrix. &lt;/li&gt;&#xA;&lt;li&gt;Then another maxpool giving a 56x56x128 matrix.&lt;/li&gt;&#xA;&lt;li&gt;Feeding that to 3 layers of &lt;strong&gt;256&lt;/strong&gt; 3x3 convs results in a 56x56x256 matrix.&lt;/li&gt;&#xA;&lt;li&gt;This is then fed into another maxpool giving a 28x28x256 matrix,&lt;/li&gt;&#xA;&lt;li&gt;Which is then fed into 3 layers of &lt;strong&gt;512&lt;/strong&gt; 3x3 convs resulting in a&#xA;28x28x512 matrix.&lt;/li&gt;&#xA;&lt;li&gt;Another maxpool gives a 14x14x512 matrix which is fed to 3 layers of &lt;strong&gt;512&lt;/strong&gt; 3x3 convs giving a matrix of 14x14x512 features.&lt;/li&gt;&#xA;&lt;li&gt;Another maxpool reduces this to a 7x7x512 which is then&#xA;given to 3 fully connected layers of &lt;strong&gt;4096&lt;/strong&gt; units each before being&#xA;sent to a softmax.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-01-07T16:34:52.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9875" PostTypeId="2" ParentId="9870" CreationDate="2019-01-07T16:45:31.357" Score="2" Body="&lt;p&gt;For learning image features with CNNs, we use 2D Convolutions. Here 2D does not refer to the input of the operation, but the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider you have an input tensor of size 224 x 224 x 3. Say for example you have 64 different convolution kernels. Theses &lt;strong&gt;kernels are also 3 dimensional&lt;/strong&gt;. Each kernel will produce a 2D matrix as output. Since you have 64 different kernels/filters, you will have 64 different 2D matrices. In other words, you got a tensor with depth 64 as output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FjvuN.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FjvuN.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you to go through this question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/42883547/intuitive-understanding-of-1d-2d-and-3d-convolutions-in-convolutional-neural-n&quot;&gt;Understanding 1D, 2D and 3D convolutions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21229" LastActivityDate="2019-01-07T16:45:31.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9876" PostTypeId="2" ParentId="9838" CreationDate="2019-01-07T17:30:51.360" Score="2" Body="&lt;p&gt;Reinforcement learning is used when we know the outcome we want, &lt;em&gt;but not how to get there&lt;/em&gt; which is why you won't see a lot of people using it for classification (because we already know the optimal policy, which is just to output the class label). You knew that already, just getting it out of the way for future readers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you say, your policy model is fine - a fully connected model that is just deep enough to learn XOR.  I think the reward gradient is a little shallow - when I give a reward of +1 for &quot;3 out 4&quot; correct and +2 for &quot;4 out of 4&quot;, then convergence happens (but very slowly).  &lt;/p&gt;&#xA;" OwnerUserId="17770" LastEditorUserId="17770" LastEditDate="2019-01-08T18:03:39.520" LastActivityDate="2019-01-08T18:03:39.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9883" PostTypeId="2" ParentId="9860" CreationDate="2019-01-08T02:33:33.190" Score="1" Body="&lt;p&gt;It's because, in the actor-critic algorithm, the objective function is an expectation under the &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; of the policy. If we want to use off-policy data, we have to resort to &lt;em&gt;importance sampling&lt;/em&gt; relative to the other policy.&lt;/p&gt;&#xA;" OwnerUserId="21180" LastEditorUserId="2444" LastEditDate="2019-02-15T19:43:04.453" LastActivityDate="2019-02-15T19:43:04.453" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9887" PostTypeId="1" AcceptedAnswerId="9888" CreationDate="2019-01-08T10:43:46.457" Score="2" ViewCount="154" Body="&lt;p&gt;Which algorithms, between ant colony or classical routing algorithms, have a better time complexity for the shortest path problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, can we compare efficiency of these two types of algorithm for the shortest path problem in a graph?&lt;/p&gt;&#xA;" OwnerUserId="19910" LastEditorUserId="2444" LastEditDate="2019-05-27T22:06:06.400" LastActivityDate="2019-05-27T22:06:06.400" Title="Which algorithms, between ant colony or classical routing algorithms, have a better time complexity for the shortest path problem?" Tags="&lt;algorithm&gt;&lt;swarm-intelligence&gt;&lt;time-complexity&gt;&lt;ant-colony-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9888" PostTypeId="2" ParentId="9887" CreationDate="2019-01-08T11:16:18.273" Score="1" Body="&lt;p&gt;No. In general, you can't find a tight bound for evolutionary algorithms, and it is one of the main difference of these algorithms with the classical algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should notice that it does not mean you can't find when the evolutionary algorithms are finished! But, you can't find a tight bound for the algorithms time complexity to reach to the optimal solution or how much that solution is near to the optimal solution (in contrast to the approximation algorithms).&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-01-08T11:56:21.797" LastActivityDate="2019-01-08T11:56:21.797" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9889" PostTypeId="1" CreationDate="2019-01-08T11:44:04.017" Score="3" ViewCount="309" Body="&lt;p&gt;I think that AI perceive the world. Can an AI be sentient?&lt;/p&gt;&#xA;" OwnerUserId="21179" LastEditorUserId="2444" LastEditDate="2019-06-20T21:21:31.213" LastActivityDate="2019-06-20T21:21:31.213" Title="If AI can perceive, can it be sentient?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;emotional-intelligence&gt;&lt;artificial-consciousness&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="3" ClosedDate="2019-01-17T01:18:01.303" ContentLicense="CC BY-SA 4.0" />
  <row Id="9890" PostTypeId="1" CreationDate="2019-01-08T11:57:24.937" Score="3" ViewCount="957" Body="&lt;p&gt;On recommendation of Kanak on stackoverflow I am posting this question here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am experimenting with various loss functions and optimizers for my binary image segmentation problem. The loss functions that I use in my Unet however give different output segmentation maps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a highly imbalanced dataset, thus I am trying dice loss for which the customized function is given below. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def dice_coef(y_true, y_pred, smooth=1):&#xA;        &quot;&quot;&quot;&#xA;        Dice = (2*|X &amp;amp; Y|)/ (|X|+ |Y|)&#xA;             =  2*sum(|A*B|)/(sum(A^2)+sum(B^2))&#xA;        ref: https://arxiv.org/pdf/1606.04797v1.pdf&#xA;        &quot;&quot;&quot;&#xA;        intersection = K.sum(K.abs(y_true * y_pred), axis=-1)&#xA;        return (2. * intersection + smooth) / (K.sum(K.square(y_true), -1) + K.sum(K.square(y_pred), -1) + smooth)&#xA;&#xA;    def dice_coef_loss(y_true, y_pred):&#xA;        return 1 - dice_coef(y_true, y_pred)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Binary cross entropy results in a probability output map, where each pixel has a color intensity that represents the chance of that pixel being the positive or negative class. However, when I use the dice loss function, the output is not a probability map but the pixels are classed as either 0 or 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.How is it possible that these different loss functions have these vastly different results?&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Is there a way to customize the dice loss function so that the output segmentation map is a probability map similar to the one of binary crossentropy loss.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21257" LastActivityDate="2019-08-22T20:04:15.723" Title="Dice loss gives binary output whereas binary crossentropy produces probability output map" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9891" PostTypeId="2" ParentId="9870" CreationDate="2019-01-08T12:15:37.960" Score="0" Body="&lt;p&gt;Both responses I got are correct but do not answer exactly what I was looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to my question is : each filter is a 2D convolution. It is applied to every channel from previous node (so we get N 2D matrices). Then all of these matrices are added up to make a final matrix (1 matrix for 1 filter). Finally, the output is all filters' matrices in parallel (like channels).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard part was to find the &quot;sum up&quot;, since many websites speak of it as a 3D convolution (which is not !).&lt;/p&gt;&#xA;" OwnerUserId="19094" LastActivityDate="2019-01-08T12:15:37.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9894" PostTypeId="2" ParentId="4889" CreationDate="2019-01-08T16:18:48.567" Score="1" Body="&lt;p&gt;If you are using a softmax distribution for your classification, then you could determine what your baseline max probability is for correctly classified samples, and then infer if a new sample doesn't belong to any of your known classes if its max probability is below some kind of threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea comes from a research paper that does a much better job of explaining the process than what I just said: &lt;a href=&quot;https://arxiv.org/abs/1610.02136&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21265" LastActivityDate="2019-01-08T16:18:48.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9895" PostTypeId="1" CreationDate="2019-01-08T16:48:11.920" Score="2" ViewCount="94" Body="&lt;p&gt;I have this question in my head: does the current level of AI development allow us to spot faked or photoshoped images? (i.e forged ID card or personal documents).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is possible, what is such a process to follow in order to build an AI that achieves this task?  &lt;/p&gt;&#xA;" OwnerUserId="19059" LastEditorUserId="19059" LastEditDate="2019-01-24T10:25:05.293" LastActivityDate="2019-04-08T07:46:32.920" Title="Is it possible to spot photoshoped or edited photos using AI?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;deepfakes&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9897" PostTypeId="1" CreationDate="2019-01-08T22:27:57.413" Score="4" ViewCount="170" Body="&lt;p&gt;Is &quot;emotion&quot; ever used in AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Psychologists have a lot to say about emotion and it's functional utility for survival - but I've never seen any AI research that uses something resembling &quot;emotion&quot; inside an algorithm. (Yes, there's some work done on trying to classify human emotions, called &quot;emotional intelligence&quot;, but that's extrememly different from /using/ emotions within an algorithm) For example, you could imagine that a robot might need fuel and be &quot;very thirsty&quot; - causing it to prioritize different tasks (seeking fuel). Emotions also sometimes don't just focus on objectives/priorities - but categorize how much certain classifications are &quot;projected&quot; into a particular emotions.&lt;br&gt;&#xA;For example, maybe a robot that needs fuel might be very &quot;afraid&quot; of going towards cars because it's been hit in the past - while it might be &quot;frustrated&quot; at a container that doesn't open properly. &#xA;It seems very natural that these things are helpful for survival - and they are likely &quot;hardcoded&quot; in our genes (since some emotions - like sexual attraction - seem to be mostly unchangeable by &quot;nurture&quot;) - so I would think they would have a lot of general utility in AI. &lt;/p&gt;&#xA;" OwnerUserId="20685" LastActivityDate="2019-01-09T09:54:53.317" Title="Is &quot;emotion&quot; ever used in AI?" Tags="&lt;ai-design&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9899" PostTypeId="2" ParentId="9897" CreationDate="2019-01-08T23:35:42.367" Score="0" Body="&lt;p&gt;Not a bad question but we can solve this with a little thought experiment. Consider what it means to be &quot;afraid&quot;, or to even &quot;feel&quot;. It's a DESIRE for something. That something is what pushes us towards general survival. It forces us to focus on what is important right now. And it's relative to our immediate environment &amp;amp; generalized to our abstract conceptualization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference with modern ai paradigms is that they are very structured/rigid in their objectives. There's no general sense of &quot;okayness&quot; or generalized sense of guidance on what it should do. This would require a radically different approach to AI design &amp;amp; infrastructure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being that most companies are trying to make money, there's not a lot to be gained by experimenting with &quot;feeling&quot; machines. &lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2019-01-08T23:35:42.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9900" PostTypeId="2" ParentId="9897" CreationDate="2019-01-09T01:20:30.503" Score="2" Body="&lt;p&gt;&lt;strong&gt;Current Simulation of Emotional Behavior&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotion is used in AI in very limited ways in leading edge natural language systems. For instance, advanced natural language systems can calculate the probability that a particular segment of speech originates from an angry human. This recognition can be trained using labels from bio-monitors. However, the mental features of a human with soft skills tuned from years of experience with people is not nearly simulated in computers as of this writing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will not see computers becoming counselors (as once believed) or directors of movies or courtroom judges or customs officials any time soon. Nonetheless, the processes behind emotion are not entirely undiscovered, and there is definite interest in simulating them in computers. Much of that work is company confidential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The emergence of emotional sophistication in computers likely to begin in the context of sexuality, primarily because flirtation is powerful and primordial emotional expression will probably be easier to simulate in natural language than higher emotional expressions such as love or chaotic ones like rage. Sexy AI will likely be exploited by what businesses might consider legitimate marketing activity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also going to be exploited by the sex industry. The ethical and moral analysis of sexy AI beyond the scope of the question but will probably gain the attention of public media as it unfolds, and that has already begun on FaceBook, originating from third party attacks using fictitious identities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Science of Emotion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotion isn't a scientific quantity. From an AI perspective, emotion is a quality an individual might recognize through visual and audio queues, specifically through the natural language and affect of another individual. (Affect is a visual clue about a person's emotional and general mental state.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An individual can also learn to recognize those clues in her or his self. They can be detected by replaying one's own speech as heard through the ear, by linguistic analysis of thoughts not spoken, or through the detection of muscle tension or vital signs. Those skilled in meditation can detect emotional predecessors closer to their causal centers in the brain and control them more directly before emotions even arise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the brain, emotion is not in a single geometric location. We cannot say, &quot;That emotion of compassion comes from this group of neurons in Jerome's brain.&quot; We cannot say, &quot;Sheri is angry at this 3D coordinate in her cerebral cortex.&quot; Emotions are also not strictly system wide either. An individual can be annoyed without going into rage, leaving most of the brain chemistry and electrical signaling unaffected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotions are not entirely electrical and not entirely chemical. On the electric side, emotional states can occur simultaneously through separate circuit pathway connecting distinct and only distantly related regions of the brain. On the chemical side, there is the synaptic chemistry that is part of the electrical signal pathways. There are also many regional signaling systems using specialized pathways that are neither circulatory (blood) nor primary electrical (neuron) pathways. Serotonin is one of dozens of chemical signaling compounds that operate regionally in this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotions, being a largely social set of phenomena, should not be characterized as purely Darwinian. Although related to survival, emotional processing and communications impact mate selection and, more generally, social patterns within a community, including altruistic and collaborative activity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotions don't always lead to survival. In some cases, emotional states may lead to death prior to reproduction. One could say that emotional balance and the ability to interact on emotional levels may improve odds of having offspring. Imbalance to the degree of any of hundreds of emotional extremes can lead to childlessness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotional intelligence is different than using emotions within an algorithm, but not extremely so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discussion of emotional intelligence is one of many advancements in the concept of intelligence since the formation of one-dimensional conceptions of intelligence. Those nineteenth century conceptions, such as IQ and G-factor are poorly supported by genetic evidence and anthropological theory. Mathematically unproven and naive concepts like general intelligence rest on those one-dimensional concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotional intelligence is a form of mental capability related to emotional balance. If a person's cognitive skills are honed with respect to their emotions and the assessment of the emotional states of others, then they have greater emotional intelligence than someone who cannot read the affect and linguistic clues of another and cannot integrate cognitive and emotional skill to balance of their own emotions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cybernetic Analysis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interface between natural emotion and artificial emotion fits within the realm of cybernetics, the conceptual study of the interface between humans and machines. Such interaction is clearly related to both algorithms and topology, two important concepts in AI research and development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotion has an algorithmic context because there is clearly some combination of neurons and chemistry that produce this algorithmic difference between a reactive person and one who has developed emotional intelligence.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    emotion[person] = recognize_emotion[person]&#xA;    if emotion[person] = anger&#xA;        be_in_responses(angry)&#xA;&#xA;    emotion[person] = recognize_emotion[person]&#xA;    if emotion[person] = anger&#xA;        be_in_responses(extra_calm)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The former is reactive and the later exhibits emotional intelligence. The acquisition of the later skill may be cognitive and conscious or it may be intuitive and unconscious. In either case, the actual algorithms at a lower level may be entirely different than those shown above, yet the external behavior of the person as marshaled by the brain is essentially one of those shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plural, algorithms, is used rather than the singular, algorithm, because it is unlikely that a single synchronous algorithm is involved. The brain is a massively parallel processor. Emotional processing is likely best expressed in artificial form as hundreds of thousands of algorithms operating in parallel and forming millions of balances within the system &amp;mdash; multidimensional and highly parallel stasis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why emotional recognition and emotional responses are not very sophisticated in computer systems as of this writing. The balances have much social nuance. It may be easier to simulate rational thought than emotional thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Desire as a Systemic Behavior&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hunger and thirst may sometimes be called feelings, but they are not strictly emotional. The detection of the need for air, energy, nutrients, and water may stimulate emotional states if the needs are unmet and other emotional states if met. A person may become frustrated and irritable when lacking something essential and confronted with another person's less important agenda. A robot may someday do the same. A person may become elated and generous when all such essentials have recently been made available in surplus. A robot may someday do the same. These relationships are expressed in the question this way.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Emotions also sometimes don't just focus on objectives/priorities &amp;mdash; but categorize how much certain classifications are &quot;projected&quot; into a particular emotions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That statement in the question and its explanation is true in some respects. If a robot that needs fuel but is afraid of passing in front of a moving vehicle because it has been hit in the past can be seen in more than one way.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Probabilistic risk management based on past experiences&lt;/li&gt;&#xA;&lt;li&gt;Fuzzy logic that produces control behavior&lt;/li&gt;&#xA;&lt;li&gt;Feeling fear because of past experience&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In AI design, these three would be handled in different ways.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Development of a function that ties visual and auditory information to a model of collision and produces a projected likelihood of injury based each of a number of paths to obtaining fuel&lt;/li&gt;&#xA;&lt;li&gt;Rules that relates to travel risk with learned probabilities for each along with rules that relate to energy depletion risk with learned probabilities for each and a fuzzy logic rules engine&lt;/li&gt;&#xA;&lt;li&gt;Artificial networks that have no audit trail but simulate reptilian emotional circuitry and simulate base instinct&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Maintaining Scientific Perspective&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Emotion is not hard coded into the brain circuitry or DNA. The reality is significantly more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DNA provides parameters to a genetic expression system that leads to protein synthesis that leads to brain structure and function that leads to the ability to learn emotional responses that lead to improved social behavior that may lead to higher probabilities of gene pool survival.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying digital system traditions to biological process can be counter productive, like anthropomorphic views of programs. Artificial networks don't actually learn; they converge. Nothing is hard coded into biology because the term &lt;em&gt;code&lt;/em&gt; applied to DNA isn't anything like a page of Java or Python code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that some behavioral predispositions are strong forms of stasis within the course of a species. An organism will normally exhibit a strong desire to acquire resources from the biosphere, such as oxygen, proteins, nutrients, carbs, fats, and water. A robot might replace those with a voltage to use for a charge and lubricants for moving parts. An organism will normally exhibit a string desire to reproduce. A robot might be given a simulation of that recursive process and wish to build another like itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are not hard coded in biology. They form a kind of stasis within a population. Some humans don't want children. Some are hospitalized for anorexia nervosa. Some commit suicide by asphyxiation. The statistical mean produces the behavior of the species, not a fixed behavior identical across individuals within the species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Nature and Nurture&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nature and nurture are useful umbrella terms for general categories of causality in biology and may have equivalents in future robotic products, but they are broad generalities. There are no nature algorithms or nurture algorithms or algorithms that balance nature and nurture. That is where topology is of paramount conceptual importance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Topology of Algorithmic Components&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is massive interaction between many systems operating independently in multiple dimensions. The visualization of such interactive structure would look more like the topology of all the web sites in a country than a machine learning block diagram. If somehow coded into one algorithm it is possible that all the silicon from all the sand on earth converted to random access memory (RAM) might be insufficient to hold the code expressing the algorithm. Perhaps not. Perhaps a simplicity underlies the interactive system design of life. Perhaps we'll someday know. Perhaps not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The elegance in the design of life on earth is that multiple independent processes are tuned by billions of years of trial and error to inter-operate and support complex organic processes with billions of moving parts at a molecular level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Veins of Interdisciplinary Research&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Study of these are important for biology, for bioinformatics, for cognitive science, and for artificial intelligence. Emotional recognition and integration of emotional reaction and control into natural communications is part of this research and development.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-09T09:54:53.317" LastActivityDate="2019-01-09T09:54:53.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9903" PostTypeId="1" AcceptedAnswerId="9923" CreationDate="2019-01-09T08:59:44.857" Score="5" ViewCount="1226" Body="&lt;p&gt;I'm new to machine learning, and AI in general (but with 20+ years for programming). I'm wondering if machine learning is a good general approach to find the seed of a random number generator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have a list of 2000 numbers. Is there a machine learning algorithm to correctly guess the next number? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to be clear, as there are many random number generator algorithms. I'm taking about rand and srand from the &lt;a href=&quot;http://www.cplusplus.com/reference/cstdlib/rand/&quot; rel=&quot;noreferrer&quot;&gt;stdlib&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&#xA;Eden &lt;/p&gt;&#xA;" OwnerUserId="21284" LastEditorUserId="21240" LastEditDate="2019-01-10T18:01:52.747" LastActivityDate="2019-01-10T18:01:52.747" Title="Finding the seed of a random number" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9904" PostTypeId="1" CreationDate="2019-01-09T09:12:34.027" Score="2" ViewCount="87" Body="&lt;p&gt;I'm reading the book &quot;Reinforcement Learning: An Introduction&quot; (by Andrew Barto and Richard S. Sutton). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors provide the pseudocode of the &lt;em&gt;prioritized sweeping&lt;/em&gt; algorithm, but I do not know what is the meaning of &lt;code&gt;Model(s, a)&lt;/code&gt;. Does it mean that &lt;code&gt;Model(s, a)&lt;/code&gt; is the history of rewards gained when we are in state &lt;code&gt;s&lt;/code&gt; and the action &lt;code&gt;a&lt;/code&gt; is taken? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does &lt;code&gt;R, S_new = Model(s,a)&lt;/code&gt; mean that we should take a random sample from rewards gained in state &lt;code&gt;s&lt;/code&gt; and action &lt;code&gt;a&lt;/code&gt; is taken?&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2019-02-15T19:22:50.727" LastActivityDate="2019-02-15T19:22:50.727" Title="What is the meaning of Model(s, a) in the prioritized sweeping algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;prioritized-sweeping&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9905" PostTypeId="1" AcceptedAnswerId="10206" CreationDate="2019-01-09T10:37:14.890" Score="4" ViewCount="141" Body="&lt;p&gt;In my view intelligence begins once the thoughts/actions are logical rather than purely randomn based. The learning environments can be random but the logic seems to obey some elusive rules. There is also the aspect of a parenting that guides through some really bad decisions by using the collective knowledge. All of this seems to hint that intelligence needs intelligence to coexist and a sharing communication network for validation/rejection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally I believe that we must keep the human intelligence in a parental role for long enough time until at least the AI had fully assimilated our values. The actual danger is to leave the artificial intelligence parenting another AI and loose control of it. This step is not necessary from our perspective but can we resist the temptation and try it eventually, only time will tell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above all we must remember the purpose of AI. I think the purpose should always be to help humans achieve mastery of the environment while ensuring our collective preservation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI should not be left unsupervised as we would not give guns to kids, do we?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To resume it all AI needs an environment and supervision where to learn and grow. The environment can vary but the supervision must stay in place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are initiated thoughts/actions by the means of guidance and supervision considered random?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly I believe that the sensible think to do is to only develop artificial intelligence that is limited by our own beliefs and values rather than searching for something greater than us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems not possible to create greater than our intelligence without letting it go exploring!&#xA;Exploring has greater access to random actions and can go against the intended purpose.&lt;/p&gt;&#xA;" OwnerUserId="21285" LastActivityDate="2019-01-25T22:17:46.640" Title="Is learning possible without random thoughts and actions?" Tags="&lt;ai-design&gt;&lt;control-problem&gt;&lt;random-variable&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9906" PostTypeId="2" ParentId="9904" CreationDate="2019-01-09T10:46:55.373" Score="1" Body="&lt;p&gt;I think pseudocode was made for tabular case with an assumption of deterministic environment. &lt;span class=&quot;math-container&quot;&gt;$Model(s, a)$&lt;/span&gt; would then be a table with information of the next state and reward after taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The size of that table would be same as the size of Q table. Because the environment is deterministic you wouldn't take a random sample because there is only one possible transition so you would take the transition remembered in model table.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-01-09T10:46:55.373" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9908" PostTypeId="1" AcceptedAnswerId="9915" CreationDate="2019-01-09T11:11:51.937" Score="3" ViewCount="165" Body="&lt;p&gt;When trying to map artificial neuronal models to biological facts it was not possible to find an answer regarding the &lt;strong&gt;biological justification&lt;/strong&gt; of randomly initializing the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps this is not yet known from our current understanding of biological neurons?&lt;/p&gt;&#xA;" OwnerUserId="21269" LastActivityDate="2019-01-10T08:00:31.110" Title="How do biological neurons weights get initialized?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;&lt;biology&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9909" PostTypeId="1" CreationDate="2019-01-09T12:25:22.140" Score="3" ViewCount="799" Body="&lt;p&gt;I'm doing a research on a finite-horizon Markov decision process with &lt;span class=&quot;math-container&quot;&gt;$t=1, \dots, 40$&lt;/span&gt; periods. In every time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the (only) agent has to chose an action &lt;span class=&quot;math-container&quot;&gt;$a(t) \in A(t)$&lt;/span&gt;, while the agent is in state &lt;span class=&quot;math-container&quot;&gt;$s(t) \in S(t)$&lt;/span&gt;. The chosen action &lt;span class=&quot;math-container&quot;&gt;$a(t)$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s(t)$&lt;/span&gt; affects the transition to the following state &lt;span class=&quot;math-container&quot;&gt;$s(t+1)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, the following holds true: &lt;span class=&quot;math-container&quot;&gt;$A(t)=A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S(t)=S$&lt;/span&gt;, while the size of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$6 000 000$&lt;/span&gt; (6 million) and the size of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$10^8$&lt;/span&gt;. Furthermore, the transition function is stochastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would Monte Carlo Tree Search (MCTS) an appropriate method for my problem (in particular due to the large size of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and the stochastic transition function?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already read a lot of papers about MCTS (e.g. progressive widening and double progressive widening, which sound quite promising), but maybe someone can tell me about his experiences applying MCTS to similar problems or about appropriate methods for this problem (with large state/action space and a stochastic transition function).&lt;/p&gt;&#xA;" OwnerUserId="21287" LastEditorUserId="2444" LastEditDate="2019-02-15T19:34:00.913" LastActivityDate="2021-03-09T02:04:55.463" Title="Is Monte Carlo Tree Search appropriate for problems with large state and action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;markov-decision-process&gt;&lt;finite-markov-decision-process&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9910" PostTypeId="2" ParentId="9908" CreationDate="2019-01-09T14:02:40.890" Score="0" Body="&lt;p&gt;I am not an DL expert but these are my short thoughts on it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is because it is believed (from an information theoretic point of view) to be the good way to avoid that the network falls into some wired state from beginning on. Remember: DNNs are nonlinear approximators for continuous functions. So they have some storage capacity to learn an amount of n function to map from input to output. When you look on topic like data leakage you will see that NNs quickly try to cheat you if they can :D. The optimization applied during training will heavily be affected by the init state. So starting with an random initialization at least avoids that your neurons do all the same at the beginning etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Biological reasoning:&lt;/strong&gt; &#xA;From the viewpoint of a neurobiologist I can recommend you to read &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Hebbian_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hebbian rule&lt;/a&gt; and how neural systems work (eg. &lt;a href=&quot;https://www.quora.com/How-do-axons-branching-out-from-neurons-find-their-targets&quot; rel=&quot;nofollow noreferrer&quot;&gt;google how neurons find targets&lt;/a&gt;) in general and then to compare it to what is known about how dendrite cells in the cerebrum develop their interconnections in the first 3 years after birth. In summary there are behavioral patterns in nature which could look similar, inspiring and even reasonable. But, I would say the reason why this random init. is recommend is backed by mathematical and information theoretical assumptions rather then pure biological arguments.&lt;/p&gt;&#xA;" OwnerUserId="21290" LastEditorUserId="21290" LastEditDate="2019-01-10T08:00:31.110" LastActivityDate="2019-01-10T08:00:31.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9911" PostTypeId="2" ParentId="9909" CreationDate="2019-01-09T16:57:14.860" Score="1" Body="&lt;p&gt;MCTS is often said to be a good choice for problems with large branching factors... but the context where that sentiment comes from is that it originally became popular for playing the game of Go, as an alternative to older game-playing approaches such as alpha-beta pruning. The branching factor of Go is more like 250-300 though, which is often viewed as a large branching factor for board games. It's not such an impressive branching factor anymore when compared to your branching factor of &lt;span class=&quot;math-container&quot;&gt;$6,000,000$&lt;/span&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see MCTS working well out of the box when you have 6 million choices at every step. Maybe it could do well if you have an extremely efficient implementation of your MDP (e.g. if you can simulate millions of roll-outs per second), and if you have a large amount of &quot;thinking time&quot; (probably in the order of hours or days) available.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To have any chance of doing better with such a massive branching factor, you really need generalization across actions. Are your 6 million actions really all entirely different actions? Or are many of them somehow related to each other? If you gather some experience (a simulation in MCTS, or just a trajectory with Reinforcement Learning approaches), can you generalize the outcome to other actions for which you did not yet collect experience? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is some way of treating different actions as being &quot;similar&quot; (in a given state), you can use a single observation to update statistics for multiple different actions at once. The most obvious way would be if you can define meaningful features for actions (or state-action pairs). Standard Reinforcement Learning approaches (with function approximation, maybe linear or maybe Deep Neural Networks) can then relatively &quot;easily&quot; generalize in a meaningful way across lots of actions. They can also be combined with MCTS in various ways (see for example AlphaGo Zero / Alpha Zero). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even with all that, a branching factor of 6 million still remains massive... but generalization across actions is probably your best bet (which may be done inside MCTS, but really does need a significant number of bells and whistles on top of the standard approach).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-09T16:57:14.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9912" PostTypeId="1" AcceptedAnswerId="9913" CreationDate="2019-01-09T17:28:06.877" Score="5" ViewCount="273" Body="&lt;p&gt;In the book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction (2nd edition)&lt;/a&gt; Sutton and Barto define at &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=126&quot; rel=&quot;nofollow noreferrer&quot;&gt;page 104 (p. 126 of the pdf), equation (5.3),&lt;/a&gt; the importance sampling ratio, &lt;span class=&quot;math-container&quot;&gt;$\rho _{t:T-1}$&lt;/span&gt;, as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho _{t:T-1}=\prod_{k=t}^{T-1}\frac{\pi(A_k|S_k)}{b(A_k|S_k)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for a target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and a behavior policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, on page 103, they state:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; [...] may be deterministic [...].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is deterministic and greedy it gives &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for the greedy action and 0 for all other possible actions.&lt;/p&gt;&#xA;&lt;p&gt;So, how can the above formula give something else than zero, except for the case where policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; takes a path that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; would have taken as well? If any selected action of &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is different from &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;'s choice, then the whole numerator is zero and thus the whole result.&lt;/p&gt;&#xA;" OwnerUserId="21299" LastEditorUserId="2444" LastEditDate="2020-11-05T22:01:11.680" LastActivityDate="2020-11-05T22:08:00.130" Title="How can the importance sampling ratio be different than zero when the target policy is deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;sutton-barto&gt;&lt;importance-sampling&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9913" PostTypeId="2" ParentId="9912" CreationDate="2019-01-09T17:58:58.743" Score="5" Body="&lt;p&gt;You're correct, when the target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is deterministic, the importance sampling ratio will be &lt;span class=&quot;math-container&quot;&gt;$\geq 1$&lt;/span&gt; along the trajectory where the behaviour policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; happened to have taken the same actions that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; would have taken, and turns to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; as soon as &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; makes one &quot;mistake&quot; (selects an action that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; would not have selected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before importance sampling is introduced in the book, I believe the only off-policy method you will have seen is one-step &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, which can only propagate observations back along exactly one step. With the importance sampling ratio, you can often do a bit better. You're right, there is a risk that it turns to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; rather quickly (especially when &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; are very different from each other), at which point it essentially &quot;truncates&quot; your trajectory and ignores all subsequent experience... but that still can be better than one-step, there is a chance that the ratio will remain &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for at least a few steps. It will occasionally still only permit &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;-step returns, but also sometimes &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;-step returns, sometimes &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;-step returns, etc., which is often better than only having &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;-step returns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever the importance sampling ratio is not &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, it can also give more emphasis to the observations resulting from trajectories that would be common under &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, but are uncommon under &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. Such trajectories will have a ratio &lt;span class=&quot;math-container&quot;&gt;$&amp;gt; 1$&lt;/span&gt;. Emphasizing such trajectories more can be beneficial, because they don't get experienced often under &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, so without the extra emphasis it can be difficult to properly learn what would have happened under &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Of course, it is also worth noting that your quote says (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; [...] &lt;strong&gt;may&lt;/strong&gt; be deterministic [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It says that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; &lt;strong&gt;may&lt;/strong&gt; be deterministic (and in practice it very often is, because we very often take &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to be the greedy policy)... but sometimes it won't be. The entire approach using the importance sampling ratio is well-defined also for cases where we choose &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; &lt;strong&gt;not&lt;/strong&gt; to be deterministic. In such situations, we'll often be able to propagate observations over significantly longer trajectories (although there is also a risk of excessive variance and/or numeric instability when &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; selects actions that are highly unlikely according to &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, but highly likely according to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-01-10T08:40:51.960" LastActivityDate="2019-01-10T08:40:51.960" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9914" PostTypeId="1" AcceptedAnswerId="9916" CreationDate="2019-01-10T03:38:42.657" Score="1" ViewCount="1052" Body="&lt;p&gt;In Deep Learning by Goodfellow et al., I came across the following line on the chapter on Stochastic Gradient Descent (pg. 287):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The main question is how to set &lt;span class=&quot;math-container&quot;&gt;$\epsilon_0$&lt;/span&gt;. If it is too large, the&#xA;  learning curve will show violent oscillations, with the cost function&#xA;  often increasing significantly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm slightly confused why the loss function would increase at all. My understanding of gradient descent is that given parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and a loss function &lt;span class=&quot;math-container&quot;&gt;$\ell (\vec{\theta})$&lt;/span&gt;, the gradient update is performed as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\vec{\theta}_{t+1} = \vec{\theta}_{t} - \epsilon \nabla_{\vec{\theta}}\ell (\vec{\theta})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function is guaranteed to monotonically decrease because the parameters are updated in the negative direction of the gradient. I would assume the same holds for SGD, but clearly it doesn't. With a high learning rate &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, how would the loss function increase in its value? Is my interpretation incorrect or does SGD have different theoretical guarantees than vanilla gradient descent?&lt;/p&gt;&#xA;" OwnerUserId="19403" LastActivityDate="2019-01-10T05:47:59.047" Title="Cost function increasing with SGD" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9915" PostTypeId="2" ParentId="9908" CreationDate="2019-01-10T04:04:10.700" Score="4" Body="&lt;p&gt;&lt;strong&gt;In short&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mentioned &lt;a href=&quot;https://psychology.stackexchange.com/a/10886/4055&quot;&gt;in another post&lt;/a&gt;, how the Artificial Neural Network (ANN) weights are a relatively crude abstraction of connections between neurons in the brain. Similarly, &lt;strong&gt;the random weight initialization step in ANNs is a simple procedure that abstracts the complexity of &lt;a href=&quot;https://en.wikipedia.org/wiki/Development_of_the_nervous_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;central nervous system development&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Synaptogenesis#Central_nervous_system_synapse_formation&quot; rel=&quot;nofollow noreferrer&quot;&gt;synaptogenesis&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A bit more detail&lt;/strong&gt; (with the most relevant parts &lt;em&gt;italicized&lt;/em&gt; below)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neocortex (one of its columns, more specifically) is a region of the brain that somewhat resembles an ANN. It has a laminar structure with layers that receive and send axons from other brain regions. Those layers can be viewed as &quot;input&quot; and &quot;output&quot; layers of an ANN (axons &quot;send&quot; signals, dendrites &quot;receive&quot;). Other layers are intermediate-processing layers and can be viewed as the ANN &quot;hidden&quot; layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When building an ANN, the programmer can set the number of layers and the number of units in each layer. In the neocortex, the number of layers and layer cell counts are determined mostly by genes (however, see: &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroplasticity#Human_echolocation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human echolocation&lt;/a&gt; for an example of post-birth brain plasticity). Chemical cues guide the positions of the cell bodies and create the laminar structure. They also seem to guide long term axonal connections between distant brain regions. The cells then sprout dendrites in certain characteristic &quot;tree-like&quot; patterns (see: &lt;a href=&quot;http://www.neuromorpho.org/index.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeuroMorpho.org&lt;/a&gt; for examples). &lt;em&gt;The dendrites will then form synapses with axons or other cell bodies they encounter along the way, generally based on the encountered cell type.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last phase is probably the most analogous to the idea of random weight initialization in ANNs. Based on where the cell is positioned and its type, &lt;em&gt;the encountered other neurons will be somewhat random and so will the connections to them&lt;/em&gt;. These connections are probably not going to be very strong initially but will have room to get stronger during learning (probably analogous to initial random weights between 0 and ~0.1, with 1 being the strongest possible connection). Furthermore, most cells are either inhibitory or excitatory (analogous to negative and positive weights). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind this randomization process has a heavy spatial component in real brains. The neurons are small and so they will make these connections to nearby neurons that are 10-200 microns away. The long-distance connections between brain regions are mostly &quot;programmed-in&quot; via genes. In most ANNs, there is generally no distance-based aspect to the initialization of connection weights (although convolutional ANNs implicitly perform something like distance-based wiring by using the sliding window).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Synaptic_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;synaptic pruning&lt;/a&gt; phenomenon, which might be analogous to creating many low weight connections in an ANN initially (birth), training it for some number of epochs (adolescence), and then removing most low-weight connections (consolidation in adulthood).&lt;/p&gt;&#xA;" OwnerUserId="21307" LastActivityDate="2019-01-10T04:04:10.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9916" PostTypeId="2" ParentId="9914" CreationDate="2019-01-10T05:26:12.847" Score="3" Body="&lt;p&gt;The learning rate used in SGD and in most convergence strategies used in artificial network designs is much like the use of voltage dividers in electronic feedback. The differential equations for control theory, developed by Norbert Wiener and others, demonstrated that insufficiently attenuated negative feedback will cause a circuit to destabilize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back-propagation with gradient descent is conceptually similar. The main difference is that artificial networks use discrete examples or batches of examples rather than continuous signals. Back-propagation is a strategy to distribute negative feedback throughout the layers of an artificial network using the chain rule, but that is also common in electronic control circuitry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is hypothesized, and the evidence is accumulating in support, that more sophisticated strategies to achieve stability using feedback exist in the neural networks of the brain and in the gut of animals. That is likely, and part of the reason arrays of activation functions are called artificially neural.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those that have studied the mathematics field of chaos understand that discrete systems can fall into oscillation with insufficiently attenuated negative feedback too. In a worse case, a &lt;a href=&quot;https://ai.stackexchange.com/questions/8039/what-is-chaotic-behavior-and-how-it-is-achieved-in-non-linear-regression-and-art/8047#8047&quot;&gt;chaotic pattern&lt;/a&gt; will emerge at what is called a bifurcation point. Those that work with convergence in machine learning are always working against factors that lead to divergence. However, too much attenuation, which corresponds to too low a learning rate will be highly stable and extremely costly in terms of computing resources, time to converge, or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this was established in theory in the late nineteenth and early twentieth century. Goodfellow is explaining it to university level students in the context of deep learning, but it is not specific to SGD. In fact SGD is less likely to diverge or go into chaotic states than what the question is calling &lt;em&gt;vanilla&lt;/em&gt; gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a mathematicians stoic point of view, calling chaos violent is anthropomorphic, placing human qualities on non-human things. They might point out that some forms of chaos are more stable than constants. There are many forms of mental, metabolic, molecular, and biospheric stasis that have what chaoticians call attractors, and never converge yet maintain a statistical adjacency to their attractor in phase space. For example, using the above list, emotion, glucose, electron shells, and populations show this type of non-constant stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That does not help the person using a multilayer-perceptron or convolution kernel hoping to get some ideal set of parameters during training to deploy into a working system in the field. What is desired is reliable, relatively fast, and accurate convergence results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What practitioners try to avoid is a cost function that diverges. Keep in mind that cost, error, and loss functions are terms used in various contexts in various articles but all represent disparity from some ideal. The function is the mathematical model for that ideal or optimum, not to be confused with the use of the term &lt;em&gt;model&lt;/em&gt; in other aspects of AI. Models are used in many places. In SGD, increases in the cost function are indications of instability and reduced likelihood that convergence will occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the interesting part. Since the cost function represents a surface and the curvature of the surface has curvature (otherwise it could be solved with linear algebra technique without a network) it may not be monotonic. In other words, one cannot determine the mismatch of output values and corresponding labels from the result of the function. The information flow is constrained to be unidirectional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the existence of local minima can interfere with the search for the global minimum. Mounds, ridges, saddle points, mazes of troughs, and circular ridges may separate the current state of network parameters from those that represent the optimal state sought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therein lies the caveat. The cost surface has curvature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2tS9r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2tS9r.png&quot; alt=&quot;Local minima interfering with search for global minimum&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statement in the question, because of this, is not generally true.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The loss function is guaranteed to monotonically decrease because the parameters are updated in the negative direction of the gradient.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is true only in specific cases identified by the theory. In the general case, the cost is not guaranteed to decrease and the algorithm is not guaranteed to converge unless a number of constraints are maintained, few of which normally are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not sufficiently attenuated, the next estimation to radically overshoot or undershoot the global minimum. To see how this works, look into chaos simulations using the logistic function. As the next estimate amplification increases, the system first goes into oscillation and then into modes of chaos. In the case of artificial networks, the values in the algorithm used can also exceed the range of IEEE floats and throw an exception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one studies the theoretical guarantees in the literature and some of the overview theory such as the PAC (probably approximately correct) Learning Framework, and then try to find a business, marketing, defense, industrial, robotic, or financial application for AI that meets all the constraints required to benefit from the theoretical guarantees, the challenge will become clear. A few easy problems exist. Most require expertise, further investigation, proofs of concept, architecture and design skills, and significant development effort to produce satisfactory results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the results are published with the appropriate references to theoretical foundations and the various design choices presented with hindsight, it looks much easier than it was.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-10T05:47:59.047" LastActivityDate="2019-01-10T05:47:59.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9918" PostTypeId="1" AcceptedAnswerId="9928" CreationDate="2019-01-10T08:29:05.757" Score="0" ViewCount="643" Body="&lt;p&gt;I have non-smooth loss function - e.g. loss(x)=min(x, 0.5). Can gradient descent be used for training neural networks with such functions. Can gradient descent be used for fairly general, mathematically not-nice functions? Pytorch or Tensorflow can calculate numerically gradients from almost any function, but it is acceptable practice to use general, not-nice loss functions?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2019-01-10T17:37:36.110" Title="Can gradient descent training be used for nonsmooth loss functions?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;gradient-descent&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9919" PostTypeId="1" AcceptedAnswerId="10789" CreationDate="2019-01-10T10:01:16.663" Score="2" ViewCount="381" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.112.8293&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov games as a framework for multi-agent reinforcement learning &lt;/a&gt; (which introduces the minimax Q Learning algorithm), at the bottom left of page 3, my understanding is that the author suggests, for a simultaneous 1v1 zero-sum game, to do Bellman iterations with &lt;span class=&quot;math-container&quot;&gt;$$V(s)=\min_{o}\sum_{a}\pi_{a}Q(s,a,o)$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\pi_{a}$&lt;/span&gt; the probability of playing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; for the maximizing player in his best mixed strategy to play in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my understanding is correct, why does the opponent in this equation play a pure strategy (&lt;span class=&quot;math-container&quot;&gt;$\min_{o}$&lt;/span&gt;) rather than his best mixed strategy in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. This would instead give &lt;span class=&quot;math-container&quot;&gt;$$V(s)=\sum_{o}\sum_{a}\pi_{a}\pi_{o}Q(s,a,o)$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\pi_{o}$&lt;/span&gt; the opponent's best mixed strategy in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Which of these two formulations is correct and why? Are they somehow equivalent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The context of this question is that I am trying to use minimax Q learning with a Neural Network outputting the matrix &lt;span class=&quot;math-container&quot;&gt;$Q(s,a,o)$&lt;/span&gt; for a simultaneous zero-sum game. I have tried both methods and so far have seen seemingly equally bad results, quite possibly due to bugs or other errors in my method.&lt;/p&gt;&#xA;" OwnerUserId="21311" LastEditorUserId="2444" LastEditDate="2019-02-21T14:43:36.137" LastActivityDate="2019-02-21T14:43:36.137" Title="Using the opponent's mixed strategy in estimating the state value in minimax Q learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9921" PostTypeId="1" AcceptedAnswerId="9926" CreationDate="2019-01-10T12:45:33.193" Score="2" ViewCount="64" Body="&lt;p&gt;The spectrum of human sensory inputs seems to fall within certain ranges suggesting normalization is built-in into biological NNs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also adapts to circumstantial conditions, e.g. people living in a city with certain factory smell eventually don't perceive the smell anymore, at least consciously (within working memory) / it adapts to a new baseline?&lt;/p&gt;&#xA;" OwnerUserId="21269" LastActivityDate="2019-01-11T07:59:29.153" Title="Is input normalization built-in into mammals sensory neurons?" Tags="&lt;machine-learning&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;&lt;biology&gt;&lt;sense&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9922" PostTypeId="2" ParentId="9838" CreationDate="2019-01-10T12:57:23.240" Score="-1" Body="&lt;p&gt;&lt;strong&gt;Maybe &lt;em&gt;Deep&lt;/em&gt; Reinforced Learning?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure but AND gate could be solved by your implementation. I have other feeling with OR gates. Just think - first we need to have information about two conditions and &lt;em&gt;then&lt;/em&gt; we can check for complex solutions.&#xA;First of all I thought about Neural Network with one hidden layer. Sounds perfect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you will understand when you check this Tensorflow-Keras code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;iterations = 50&#xA;&#xA;model = Sequential()&#xA;model.add(Dense(16, input_shape=(None, 2), activation='relu')) # our hidden layer for OR gate problem&#xA;model.add(Dense(2, activation='sigmoid'))&#xA;model.summary()&#xA;opt = Adam(0.01)&#xA;model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['acc'])&#xA;# mean_squared_error categorical_crossentropy binary_crossentropy&#xA;&#xA;for iteration in range(iterations):&#xA;    x_train = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) # table of inputs&#xA;    y_train = np.array([[1, 0], [0, 1], [0, 1], [1, 0]]) # outputs in categorical (first index is 0, second is 1)&#xA;&#xA;    r = np.random.randint(0, len(x_train)) # random input&#xA;    r_x = x_train[r]&#xA;    r_x = np.array([[r_x]])&#xA;    result = model.predict(r_x)[0] # predict&#xA;    best_id = np.argmax(result) # get of index of &quot;better&quot; output&#xA;&#xA;    input_vector = np.array([[x_train[r]]])&#xA;    isWon = False&#xA;    if (best_id == np.argmax(y_train[r])):&#xA;        isWon = True # everything is good&#xA;    else:&#xA;        # answer is bad!&#xA;        output = np.zeros((2))&#xA;        output[best_id] = -1&#xA;        output = np.array([[output]])&#xA;        loss = model.train_on_batch(input_vector, output)&#xA;&#xA;    print(&quot;iteration&quot;, iteration, &quot;; has won?&quot;, isWon)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When &quot;answer&quot; of agent is good - we are not changing anything (but we could train network with best action as 1 for stability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When answer is bad, we set action as bad - other actions have more probability for be chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes learning need to have more than 50 iterations but it is only my proposition. Play with hidden layer neuron count, learn rate and iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope will help you :)&lt;/p&gt;&#xA;" OwnerUserId="9101" LastActivityDate="2019-01-10T12:57:23.240" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9923" PostTypeId="2" ParentId="9903" CreationDate="2019-01-10T13:40:13.443" Score="4" Body="&lt;p&gt;Machine Learning is a bad fit to this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even simple PRNGs that are not suitable for use in simulators (such as &lt;code&gt;rand()&lt;/code&gt;) are varied enough that it is very hard to reverse engineer them statistically using generic techniques - essentially what 90% of ML does is fit a generic model to data statistically by altering parameters. The remaining 10% might do things in specialist manner, such as saving all the data and picking best option. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory most ML approaches would &lt;em&gt;eventually&lt;/em&gt; solve a PRNG, however that would typically involve iterating through the entire state space of the PRNG multiple times. The statistical relationship between internal state, next state and output of a PRNG is complex by design, so that this is the only &quot;black box&quot; statistical approach, and this is clearly not feasible for any real implementation of a random number generator, which is going to have at least &lt;span class=&quot;math-container&quot;&gt;$2^{31}$&lt;/span&gt; states on modern machines. Perhaps older 16-bit PRNGs, with a single value for state might be tractable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An AI advanced enough to attempt to reverse engineer the output logically based on purely the data and researching how RNGs work is too advanced for current ML techniques to consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That leaves approaches that might try to construct a similar RNG, such as Genetic Programming (where the genome is converted to executable code). The trouble with this approach is there is no heuristic for a RNG that measures how close its output is to a target. A single bit of state difference or any tiny but meaningful change in generated RNG design will produce output that has no similarities with the target output whatsoever. Without such a measure you have no fitness function, and no way to attempt a guided search using the many discrete optimisation tools from AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead the usual approach to &quot;breaking&quot; a PRNG is to analyse the algorithm. Knowing the algorithm of many non-cryptographic PRNGs can allow predicting the internal state of the generator, sometimes in very few steps (for really simple &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_congruential_generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear Congruential Generators&lt;/a&gt; that might be just a single step!).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-10T14:30:46.387" LastActivityDate="2019-01-10T14:30:46.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9924" PostTypeId="1" CreationDate="2019-01-10T13:44:58.710" Score="4" ViewCount="664" Body="&lt;p&gt;As according to the definition of AI, something that can learn overtime, can imitate human behaviors, comes under Artificial Intelligence. If Expert system(eg. MYCIN) that only involves if then else statements qualifies to be an AI then every program we write in our daily life that involves some condition based question answering should be an AI. Right? If not then what should be an exact and universal definition for AI. How can a software qualify to be called AI.&#xA;Can someone please explain it to me ?&lt;/p&gt;&#xA;" OwnerUserId="21316" LastEditorUserId="21316" LastEditDate="2021-03-16T15:01:29.360" LastActivityDate="2021-03-16T15:01:29.360" Title="If expert systems are a bunch of If, then, else statements then how are they termed as AI?" Tags="&lt;ai-basics&gt;&lt;expert-system&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9925" PostTypeId="1" CreationDate="2019-01-10T14:42:59.520" Score="6" ViewCount="470" Body="&lt;p&gt;Disclaimer: I'm not a student in computer science and most of my knowledge about ML/NN comes from YouTube, so please bear with me!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let's say we have a classification neural network, that takes some input data &lt;span class=&quot;math-container&quot;&gt;$w, x, y, z$&lt;/span&gt;, and has some number of output neurons. I like to think about a classifier that decides how expensive a house would be, so its output neurons are bins of the approximate price of the house.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining house prices is something humans have done for a while, so let's say we know &lt;em&gt;a priori&lt;/em&gt; that data &lt;span class=&quot;math-container&quot;&gt;$x, y, z$&lt;/span&gt; are important to the price of the house (square footage, number of bedrooms, number of bathrooms, for example), and datum &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; has no strong effect on the price of the house (color of the front door, for example). As an experimentalist, I might determine this by finding sets of houses with the same &lt;span class=&quot;math-container&quot;&gt;$x, y, z$&lt;/span&gt; and varying &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;, and show that the house prices do not differ significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's say our neural network has been trained for a little while on some random houses. Later on in the data set. it will encounter sets of houses whose &lt;span class=&quot;math-container&quot;&gt;$x, y, z$&lt;/span&gt; and price are all the same, but whose &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; are different. I would naively expect that at the end of the training session, the weights from &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; to the first layer of neurons would go to zero, effectively decoupling the input datum &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; from the output neuron. I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it certain, or even likely, that &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; will become decoupled from the layer of output neurons?&lt;/li&gt;&#xA;&lt;li&gt;Where, mathematically, would this happen? What in the backpropagation step would govern this effect happening, and how quickly would it happen?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For a classical neural network, the network has no &quot;memory,&quot; so it might be very difficult for the network to realize that &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is a worthless input parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any information is much appreciated, and if there are any papers that might give me insight into this topic, I'd be happy to read them.&lt;/p&gt;&#xA;" OwnerUserId="21319" LastActivityDate="2019-04-17T09:36:10.390" Title="Can neural networks learn to ignore an input datum?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9926" PostTypeId="2" ParentId="9921" CreationDate="2019-01-10T15:07:02.120" Score="2" Body="&lt;p&gt;Yes, for many sensory inputs there is indeed something similar to normalization. But its not rally the same as in classical data analytics compared to what eg min/max normalization does or other technics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Lets look on some examples and considerations:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;mammals don't perceive heat or loudness in a linear way. This is because already many sensory receptors have chemical / physical limits. Double decibels will not perceived with double intensity. Inside your ear, the small hammer and abil will brace to protect you. --&gt; its like normalization with logarithmic effects applied. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;heat perception is more like a difference integration than a absolute temperature measurement. Its measured via H+ ions flow in mitochondria in the cell (if i recall correctly)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On the neuronal side gradual signals in the dendrites (analog signal) sum up gradually to later form an spike at the axon hill. where in turn a fire frequency is then encoded - the maximum frequency of this serves as a a natural maximum limit. I remember that grasshoppers increase axon fire frequency when objects started covering more ommatidial area on their &quot;eye&quot;. The more of their &quot;eyes&quot; are covered by the shadow the more input on the neuron --&gt; higher fire rate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a lot of sensory input is post processed in higher cerebral areas. Eg. compared to what is expect able and heuristics are applied to compare a signal with former events. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;when doing computational data analysis we may want go for accuracy and maximum comparability. Mostly on all data that could be available. --&gt; eg. with respect to properties of a standard normal distribution. Hence we put some effort to be accurate and know the true parameters, remove outliers and so on --&gt; big data comes into play here. &#xA;Nature in contrast strives often for efficiency with the means of reaching the minimal required with minimal resources.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary:&lt;/strong&gt;&#xA;Compared to normalization in an analytical sense (eg. mean, min-max or other feature normalization techniques), &lt;strong&gt;nature is often only interested in the current difference between stimuli.&lt;/strong&gt; And this only within some relevant range. Other information is not integrated. And normalization with the goal of making measurement points comparable only happens within this range of the mapping function provided by the sensor/neuron/receptor whatever! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this should also answer your question about, why you are not smelling something in the city after a while any more. However, this for sure happens at higher cerebral regions (it might also be that your smell receptors saturate) its the same principle. Your consciousness just saves energy by not concentrating on something that is anyway not changing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to read more have a look here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Weber%E2%80%93Fechner_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Weber%E2%80%93Fechner_law&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21290" LastEditorUserId="21290" LastEditDate="2019-01-11T07:59:29.153" LastActivityDate="2019-01-11T07:59:29.153" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9928" PostTypeId="2" ParentId="9918" CreationDate="2019-01-10T17:37:36.110" Score="1" Body="&lt;p&gt;Gradient descent and stochastic gradient descent can be applied to any differentiable loss function irrespective of whether it is convex or non-convex. The &quot;differentiable&quot; requirement ensures that trainable parameters receive gradients that point in a direction that decreases the loss over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the absence of a differentiable loss function, the true gradient must be approximated through other methods. For example, in classification problems, the 0-1 loss function is considered the &quot;true&quot; loss, but it is non-convex and difficult to optimize. Instead, &lt;em&gt;surrogate loss functions&lt;/em&gt; act as tractable proxies for true loss functions. They are not necessarily worse; negative log-likelihood loss gives a softmax distribution over &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; classes rather than just the classification boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your problem specifically, &lt;span class=&quot;math-container&quot;&gt;$f(x,a)=min(x,a)$&lt;/span&gt; is not a differentiable loss function. It is not differentiable at &lt;span class=&quot;math-container&quot;&gt;$x=0.5$&lt;/span&gt;, but the gradient could be estimated through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Subderivative&quot; rel=&quot;nofollow noreferrer&quot;&gt;subgradient&lt;/a&gt;. In practice, this works because neural networks often don't achieve the local/global minima of a loss function but instead asymptotically decreasing values that achieve good generalization error. Tensorflow and PyTorch use subgradients when fed non-differentiable loss functions. You could also use a smooth approximation of the &lt;span class=&quot;math-container&quot;&gt;$min$&lt;/span&gt; function (see &lt;a href=&quot;https://mathoverflow.net/questions/35191/a-differentiable-approximation-to-the-minimum-function&quot;&gt;this thread&lt;/a&gt;) to get better gradients.&lt;/p&gt;&#xA;" OwnerUserId="19403" LastActivityDate="2019-01-10T17:37:36.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9929" PostTypeId="2" ParentId="5774" CreationDate="2019-01-11T03:20:04.523" Score="1" Body="&lt;p&gt;I have just the same problem, and I was trying to derive the backpropagation for the conv layer with stride, but it doesn't work.&#xA;When you do the striding in forward propagation, you chose the elements next to each other to convolve with the kernel, than take a step &gt;1. This results in the fact that in the backprop, in the reverse operation the delta matrix elements will be multiplied by the kernel elements, (with the rotation) but not as strided, but you are picking elements that are not next to each other, something like DY_11 * K_11 + DY_13 * K_12 + DY_31 * K_21 + DY_33 * K_22, which is NOT the equivalent as a convolution with a stride &gt;1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So as far as I am concerned, if I would like to implement the convnet by myself to get a better grasp of the concept, I have to implement a different method for the backprop, if I allow strides.&lt;/p&gt;&#xA;" OwnerUserId="21330" LastActivityDate="2019-01-11T03:20:04.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9933" PostTypeId="1" CreationDate="2019-01-11T09:20:37.030" Score="3" ViewCount="120" Body="&lt;p&gt;I understand why deep generative models like  DBN ( deep belief nets ) or DBM ( deep boltzmann machines ) are able to capture underlying structures in data and use it for various tasks ( classification, regression, multimodal representations etc ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for the classification tasks like in &lt;a href=&quot;http://www.cs.cmu.edu/~rsalakhu/papers/annrev.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning deep generative models&lt;/a&gt;, I was wondering why the network is fine-tuned on labeled-data like a feed-forward network and why only the last hidden layer is used for classification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the fine-tuning and since we are updating the weights for a classification task ( not the same goal as the generative task ), could the network lose some of its ability to regenerate proper data? ( and thus to be used for different classification tasks ? )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of using only the last layer, could it be possible to use a partition of the hidden units of different layers to perform the classifications task and without modifying the weights? For example, by taking a subset of hidden units of the last two layers ( sub-set of abstract representations ) and using a simple classifier like an SVM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="21335" LastEditorUserId="21335" LastEditDate="2019-01-16T15:59:42.293" LastActivityDate="2019-01-16T15:59:42.293" Title="Why is the last layer of a DBN or DBM used for classification task?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9934" PostTypeId="1" AcceptedAnswerId="9997" CreationDate="2019-01-11T15:00:12.263" Score="12" ViewCount="8652" Body="&lt;p&gt;I have checked out many methods and papers, like YOLO, SSD, etc., with good results in detecting a rectangular box around an object, However, I could not find any paper that shows a method that learns a rotated bounding box.&lt;/p&gt;&#xA;&lt;p&gt;Is it difficult to learn the rotated bounding box for a (rotated) object?&lt;/p&gt;&#xA;&lt;p&gt;Here's a diagram that illustrates the problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ljimz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ljimz.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example, for this object (see &lt;a href=&quot;https://stackoverflow.com/q/40404031/9277245&quot;&gt;this&lt;/a&gt;), its bounding box should be of the same shape (the rotated rectangle is shown in the 2nd right image), but the prediction result for the YOLO will be Ist right.&lt;/p&gt;&#xA;&lt;p&gt;Is there any research paper that tackles this problem?&lt;/p&gt;&#xA;" OwnerUserId="16313" LastEditorUserId="2444" LastEditDate="2021-01-28T23:38:45.487" LastActivityDate="2021-01-28T23:41:35.277" Title="Is it difficult to learn the rotated bounding box for a (rotated) object?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9935" PostTypeId="1" AcceptedAnswerId="9961" CreationDate="2019-01-11T16:48:59.080" Score="5" ViewCount="285" Body="&lt;p&gt;I'm using an object detection neural network and I employ data augmentation to increase a little my small dataset. More specifically I do rotation, translation, mirroring and rescaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I notice that rotating an image (and thus it's bounding box) changes its shape. This implies an erroneous box for elongated boxes, for instance on the augmented image (right image below) the box is not tightly packed around the left player as it was on the original image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that this kind of data augmentation seems (in theory) to hamper the network to gain precision on bounding boxes location as it loosens the frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there some studies dealing with the effect of data augmentation on the precision of detection networks? Are there systems that prevent this kind of thing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Obviously, it seems advisable to use small rotation angles)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZylvJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZylvJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19859" LastEditorUserId="21337" LastEditDate="2019-01-12T10:25:13.497" LastActivityDate="2019-01-13T11:56:12.210" Title="How data augmentation like rotation affects the quality of detection?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9936" PostTypeId="1" CreationDate="2019-01-11T18:21:57.300" Score="1" ViewCount="37" Body="&lt;p&gt;Let's consider a classic feedforward neural network &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; with input dimension &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, output dimension &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; layers &lt;span class=&quot;math-container&quot;&gt;$l_i$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; neurons each. ReLu activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that, given a point &lt;span class=&quot;math-container&quot;&gt;$x \in R^d$&lt;/span&gt; its image &lt;span class=&quot;math-container&quot;&gt;$F(x) \in R^k$&lt;/span&gt;. Let's now assume i add some gaussian noise &lt;span class=&quot;math-container&quot;&gt;$\eta_i$&lt;/span&gt; in EVERY hidden layer &lt;span class=&quot;math-container&quot;&gt;$l_i(x)$&lt;/span&gt; at the same time, where the norm of this noise is 5% the norm of its layer computed on the point &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Let's call this new neural network &lt;span class=&quot;math-container&quot;&gt;$F_*$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that, empirically, neural networks are resistant to this kind of noise, especially on the first layers. How can i show this theoretically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question i'm trying to answer is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After having injected this noise &lt;span class=&quot;math-container&quot;&gt;$\eta_i$&lt;/span&gt; in every layer &lt;span class=&quot;math-container&quot;&gt;$l_i(x)$&lt;/span&gt;, how far the output &lt;span class=&quot;math-container&quot;&gt;$F_{*}(x)$&lt;/span&gt; will be from the output of the original neural network &lt;span class=&quot;math-container&quot;&gt;$F(x)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="21338" LastEditorUserId="21338" LastEditDate="2019-01-12T13:23:07.730" LastActivityDate="2019-01-12T13:23:07.730" Title="Are Neural Network layers resistent to noise?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9937" PostTypeId="1" CreationDate="2019-01-11T22:31:31.807" Score="3" ViewCount="159" Body="&lt;p&gt;If I'm performing a text classification task using a model built in Keras, and, for example, I am attempting to predict the appropriate tag for a given Stack Overflow question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How do I subtract 1 from an integer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And the ground-truth tag for this question is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;/questions/tagged/objective-c&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;objective-c&amp;#39;&quot; rel=&quot;tag&quot;&gt;objective-c&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But my model is predicting:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;/questions/tagged/c%23&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;c#&amp;#39;&quot; rel=&quot;tag&quot;&gt;c#&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I were to retrain my model, but this time add the above question and tag in both the training and testing data, would the model be guaranteed to predict the correct tag for this question in the test data?&lt;/p&gt;&#xA;&lt;p&gt;I suppose the tl;dr is: Are neural networks deterministic if they encounter identical data during training and testing?&lt;/p&gt;&#xA;&lt;p&gt;I'm aware it's not a good idea to use the same data in both training and testing, but I'm interested from a hypothetical perspective, and for gaining more insight into how neural networks actually learn. My intuition for this question is &amp;quot;no&amp;quot;, but I'd really be interested in being pointed to some relevant literature that expands/explains that intuition.&lt;/p&gt;&#xA;" OwnerUserId="21347" LastEditorUserId="2444" LastEditDate="2021-01-17T17:23:11.320" LastActivityDate="2021-01-17T17:23:11.320" Title="Will a neural network always predict the correct label if it sees the exact same input during training and testing?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;keras&gt;&lt;computational-learning-theory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9938" PostTypeId="2" ParentId="9934" CreationDate="2019-01-11T22:57:48.117" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6112046/#sec3-sensors-18-02702title&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's&lt;/a&gt; a recent paper that does what you're looking for. It looks like they achieve this simply by adding a couple rotated prior boxes and regressing the angles in between.  This is similar to what standard object detectors do in terms of creating a bunch of prior box shapes and regressing the actual sizes.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2019-01-11T22:57:48.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9939" PostTypeId="1" CreationDate="2019-01-12T00:29:18.730" Score="1" ViewCount="50" Body="&lt;p&gt;The Markov property is the dependence of a system's future state probability distribution solely on the present state, excluding any dependence on past system history. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The presence of the Markov property saves computing resource requirements in terms of memory and processing in AI implementations, since no indexing, retrieval, or calculations involving past states is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the Markov property is often an unrealistic and too strong assumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Precisely, what limitations does the Markov property place on real-time learning?&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2019-02-13T02:35:38.183" LastActivityDate="2019-02-13T02:35:38.183" Title="What limitations does the Markov property place on real time learning?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;markov-property&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9940" PostTypeId="1" CreationDate="2019-01-12T00:54:38.873" Score="1" ViewCount="59" Body="&lt;p&gt;As the question states, I am wondering how, if at all, a varying length of a trajectory (series of state,action pairs) will impact training/performance of policy gradient algorithms such as PPO, TRPO and VPG. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say an agent runs in an environment where the length of each episode may not always be the same (this is true especially in games such as poker). The cumulative reward for longer trajectories will inevitably be larger than the rewards for smaller trajectories. To me, this seems to cause an imbalance favoring the actions of a policy executed for a longer period of time (non terminal state) even if that policy may be sub-optimal compared to the policy applied for a shorter trajectory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my assumptions correct? How does trajectory size end up impacting the updates on a policy?&lt;/p&gt;&#xA;" OwnerUserId="21349" LastActivityDate="2019-01-12T00:54:38.873" Title="Impact of Varying Length Trajectories on Policy Gradient Optimization" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9942" PostTypeId="2" ParentId="9828" CreationDate="2019-01-12T01:34:20.170" Score="2" Body="&lt;p&gt;The general answer to the behavior of combining common activation functions is that the laws of calculus must be applied, specifically differential calculus, the results must be obtained through experiment to be sure of the qualities of the assembled function, and the additional complexity is likely to increase computation time. The exception to such increase will be when the computational burden of the combination is small compared to the convergence advantages the combination provides.&lt;/p&gt;&#xA;&lt;p&gt;This appears to be true of Swish, the name given to the activation function defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = x \, \mathbb{S}(\beta x) \; \text{,}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f()$&lt;/span&gt; is the Swish activation function and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{S}$&lt;/span&gt; is the sigmoid function. Note that Swish is not strictly a combination of activation functions. It is formed through the addition of a hyper-parameter &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; inside the sigmoid function and a multiplication of the input to the sigmoid function result.&lt;/p&gt;&#xA;&lt;p&gt;It does not appear to be developed by Google. The originally anonymously submitted paper (for double blind review as a ICLR 2018 paper), &lt;a href=&quot;https://openreview.net/pdf?id=SkBYYyZRZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Searching for Activation Functions&lt;/em&gt;&lt;/a&gt;, was authored by Prajit Ramachandran, Barret Zoph, and Quoc V. Le around 2017. This is their claim.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our experiments show that the best discovered activation function, ... Swish, ... tends to work better than ReLU on deeper models across a number of challenging datasets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Any change in activation function to any one layer will, except in the astronomically rare case, impact accuracy, reliability, and computational efficiency. Whether the change is significant cannot be generalized. That's why new ideas are tested against data sets traditionally used to gauge usefulness&lt;sup&gt;1&lt;/sup&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Combining activation functions to form new activation functions is not common. For instance, AlexNet does not combine them.&lt;sup&gt;2&lt;/sup&gt;. It is, however, very common to use different activation functions in different layers of a single, effective network design.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1] Whether these traditions create a bias is another question. Those who follow the theory of use case analysis pioneered by Swedish computer scientist Ivar Hjalmar Jacobson or 6 Sigma ideas would say that these tests are unit test, not functional tests against real world use cases, and they have a point.&lt;/p&gt;&#xA;&lt;p&gt;[2] To correct any misconceptions that may arise from another answer, AlexNet, the name given to the approach outlined in &lt;a href=&quot;http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;ImageNet Classification with Deep Convolutional Neural Networks&lt;/em&gt;&lt;/a&gt; (2012) by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton from the University of Toronto, does not involve combining activation functions to form new ones. They write this.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The output of the last fully-connected layer is fed to a 1000-way softmax which produces a distribution over the 1000 class labels.&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;The ReLU non-linearity is applied to the output of every convolutional and fully-connected layer. The internal layers are pure ReLU and the output layer is Softmax.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are also convolution kernels and pooling layers in the AlexNet approach's series of layers used by them, and the design has entered common use since their winning of the ImageNet competition in 2012. Other approaches have won subsequent competitions.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-12T01:34:20.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9943" PostTypeId="2" ParentId="9698" CreationDate="2019-01-12T03:16:20.773" Score="0" Body="&lt;p&gt;Considering the definition of operator &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}$&lt;/span&gt; such that child&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C := \mathcal{O} \, (A, B) \, \text{,}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;there are a few assumptions to enumerate so that the problem is not ambiguous.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chromosomes &lt;span class=&quot;math-container&quot;&gt;$A, B, C$&lt;/span&gt; each contain the same number of genes.&lt;/li&gt;&#xA;&lt;li&gt;Selection must have a constant probability distribution across the permutations that conform to the rule given.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}$&lt;/span&gt; defined in this way, this is one conformant algorithmic expression.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    s = A.size&#xA;    if s != B.size&#xA;        return nil&#xA;    random_real.seed get_time&#xA;    L = stack A&#xA;    L.add B&#xA;    C = new list&#xA;    c0 = new hash_set&#xA;    c1 = new hash_set&#xA;    while s &amp;gt; 0&#xA;        i = floor (s * random_real.get % s)&#xA;        candidate = L.front_pop&#xA;        if c0.has candidate[0]&#xA;            continue&#xA;        if c1.has candidate[1]&#xA;            continue&#xA;        C.add candidate&#xA;        if L.empty&#xA;            break&#xA;        c0.add candidate[0]&#xA;        c1.add candidate[1]&#xA;        s --&#xA;    return C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The better way is to create a generic (template) multidimensional hash and a multidimensional hashset that uses it. Then the template can be instantiated with a comparator that expresses the predicate (Boolean expression) for the insertion rule. This simplifies the body of the loop to something that can be expressed using functional notation and allows the optimization of the multidimensional lookup to test preexistence of either coordinate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The random source&lt;code&gt;random_real&lt;/code&gt;in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; can be replaced with an integer one instantiated with a min and max value, which could potentially be much faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With operator override as a language feature, the appropriate function could be defined to allow spawning as a base operation of the chromosome class allowing reproduction to be represented thusly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ C := A * B $$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-12T03:16:20.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9944" PostTypeId="1" CreationDate="2019-01-12T06:27:04.450" Score="2" ViewCount="290" Body="&lt;p&gt;How does DARTS compare to ENAS? Which one is better or what advantages does they each have?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1806.09055&quot; rel=&quot;nofollow noreferrer&quot;&gt;DARTS: Differentiable Architecture Search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1802.03268&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Neural Architecture Search via Parameter Sharing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21351" LastEditorUserId="1671" LastEditDate="2019-01-12T23:29:55.420" LastActivityDate="2019-01-12T23:29:55.420" Title="How does DARTS compare to ENAS?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9945" PostTypeId="2" ParentId="9937" CreationDate="2019-01-12T08:51:16.587" Score="2" Body="&lt;p&gt;After training, all standard models are deterministic (the process each input goes thru is set). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, during training the model attempts to learn the distribution of the training dataset. Whether it is able to depends on the size of the model, if it is big enough, it can simply &quot;memorize&quot; all the training samples and result in perfect accuracy on the training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally this is considered to be terrible (called overfitting) and many regularization techniques attempt to prevent it. Eventually when training a model, you are giving it the training distribution as an example but you hope that it will be able to estimate the real distribution out of it.&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2019-01-12T08:51:16.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9946" PostTypeId="2" ParentId="2841" CreationDate="2019-01-12T09:16:00.000" Score="-1" Body="&lt;p&gt;This May not be what you were looking for,  but technically yes. Although not for Speed and Strength. But you could randomly guess new Mathematical/Physical/chemical solutions to become more efficient in random guessing (basically anything that allows the machine to compute faster and to maybe simulate the effect of those findings) thus technically achieving something similar to a singularity, without having to have any Intelligence at all actually (or just on a human Level), since you could just brute force all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this efficient? No, not even close to being in any way feasible. &#xA;Does it work? Technically, Yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be a singularity of sorts, since it improves itself continuously, but it wouldn‘t need to improve its own intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, some findings might make it possible to become more intelligent, but let‘s just assume it doesn‘t apply those findings to itself.&lt;/p&gt;&#xA;" OwnerUserId="21191" LastActivityDate="2019-01-12T09:16:00.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9947" PostTypeId="2" ParentId="9937" CreationDate="2019-01-12T10:13:39.297" Score="3" Body="&lt;p&gt;No, Neural Networks do not have such a guarantee. In fact, I don't believe &lt;strong&gt;any&lt;/strong&gt; kind of classifier in the entire field of Machine Learning has such a guarantee, though some may be slipping my mind...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an easy counterexample, consider what happens if you have two instances with precisely identical inputs, but different output labels. If your classifier is deterministic (in the sense that there is no stochasticity in the procedure going from input to output &lt;strong&gt;after training&lt;/strong&gt;), which a Neural Network is (unless, for example, you mess up a Dropout implementation and accidentally also apply dropout after training), it cannot possibly generate the correct output for both of those instances, even if they were presented as examples thousands of times during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the above is an extreme example, but similar intuition applies to more realistic cases. There can be cases where getting the correct prediction on one instance would reduce the quality of predictions on many other instances if they have somewhat similar inputs. Normally, the training procedure would then prefer getting better predictions on the larger number of instances, and settle for failure on another instance.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-12T10:13:39.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9948" PostTypeId="2" ParentId="9889" CreationDate="2019-01-12T10:20:37.070" Score="0" Body="&lt;p&gt;No, it cannot be sentient. This can be proved by the &lt;a href=&quot;http://Chinese%20Room%20Argument&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room Argument&lt;/a&gt; given by the philosopher John Searle in 1980.&lt;/p&gt;&#xA;" OwnerUserId="21354" LastActivityDate="2019-01-12T10:20:37.070" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="9949" PostTypeId="2" ParentId="9933" CreationDate="2019-01-12T11:20:17.967" Score="1" Body="&lt;p&gt;One of the big realizations that deep learning models brought in recent years was that we can train the feature extractors and classifiers simultaneously. In fact most people have stopped separating the 2 tasks and simply refer to all the process as training the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you dive in to every single model architecture, it will always be constructed from the first part which is the &lt;strong&gt;feature extractor&lt;/strong&gt; which outputs the embedding output - (which is basically the x encoded features of the input), and second part consisting of the final layer the model - &lt;strong&gt;the classifier&lt;/strong&gt; which uses the embedding layer encoding to predict the class of the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the first part is to reduce the dimensionality of the input to just the most impotent features for the final task. The goal of the classifier is to use those features to output the final score/class etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why usually only this layer is fine-tuned, because we don't want to damage the trained feature extractor, just update the classifier to fit a slightly different distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm pretty sure that in your mentioned case, for generation they do not use the classification layer, so updating it shouldn't have any affect on the model's generative abilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your last question, yes it is possible, ones you extracted the features with the model, you can use any kind of classifier on them.&lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2019-01-12T11:20:17.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9953" PostTypeId="1" CreationDate="2019-01-12T14:24:07.297" Score="3" ViewCount="55" Body="&lt;p&gt;Let's say for:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Image tasks&lt;/li&gt;&#xA;&lt;li&gt;Deep RL in high dimensional state space&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2021-01-19T14:33:16.477" LastActivityDate="2021-01-19T14:33:16.477" Title="At what point are AWS GPU instances worth it compared to CPU, price-wise?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;gpu&gt;&lt;hardware-evaluation&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-10T14:43:57.223" ContentLicense="CC BY-SA 4.0" />
  <row Id="9954" PostTypeId="1" AcceptedAnswerId="9970" CreationDate="2019-01-12T14:55:36.597" Score="3" ViewCount="208" Body="&lt;p&gt;Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression.&lt;/p&gt;&#xA;&lt;p&gt;Examples of high-variance machine learning algorithms include decision trees, k-nearest neighbors, and support vector machines.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Source:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s2m9z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s2m9z.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What makes a machine learning algorithm a low variance one or a high variance one?&lt;/strong&gt; For example, why do decision trees, k-NNs and SVMs have high variance?&lt;/p&gt;&#xA;" OwnerUserId="15368" LastEditorUserId="2444" LastEditDate="2020-06-23T21:24:47.103" LastActivityDate="2020-06-24T14:36:53.507" Title="What makes a machine learning algorithm a low variance one or a high variance one?" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;statistical-ai&gt;&lt;decision-tree&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9957" PostTypeId="2" ParentId="9889" CreationDate="2019-01-12T21:20:20.540" Score="2" Body="&lt;p&gt;As long as you believe our sentience emerges from our brain that exists in our physical universe, yes, we can build AI that has the same type of sentience (the exact components needed to replicate our minds, e.g. whether software simulations are sufficient or not, can be debated). If you believe our sentience comes from a soul that is outside of our physical reality (in a reality we don't have full two-way access to), then no, not with means accessible from within our reality (not the same type of sentience as we have anyway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Everything I know about the human mind, intelligence, perception, feelings, and qualia point to the former (i.e. a brain that's completely within our physical reality), but that's just my opinion and can be debated.&lt;/p&gt;&#xA;" OwnerUserId="21373" LastActivityDate="2019-01-12T21:20:20.540" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9958" PostTypeId="1" CreationDate="2019-01-12T22:28:22.917" Score="1" ViewCount="20" Body="&lt;p&gt;For a neural turing machine, there is an attention distribution over the memory cells. A read operation consists of multiplying the memory cell's value by its respective probability, and adding these results for all memory cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we only did the above operation for memory cells with a probability greater than 0.5, or suppose we concatenated the results instead of adding them. Can this be implemented/ trained with stochastic gradient descent? Or would it not be differentiable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21375" LastActivityDate="2019-01-12T22:28:22.917" Title="Is discrete reading in neural turing machines differentiable?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9959" PostTypeId="2" ParentId="9889" CreationDate="2019-01-12T23:22:10.340" Score="3" Body="&lt;p&gt;I'd call any system that receives input to be &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Perception&quot; rel=&quot;nofollow noreferrer&quot;&gt;perceiving&lt;/a&gt;&quot;, and that can constitute a basic definition of &quot;awareness&quot; or even &quot;consciousness&quot;, distinct from &quot;self-awareness&quot; or consciousness in the human sense, which we tend to think of in terms of &lt;a href=&quot;https://en.wikipedia.org/wiki/Sentience&quot; rel=&quot;nofollow noreferrer&quot;&gt;sentience&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the Chinese Room seems to argue is that sentience is based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Qualia&quot; rel=&quot;nofollow noreferrer&quot;&gt;qualia&lt;/a&gt;, which may be difficult or impossible to validate in another entity.  (With humans, we can assume another individual is sentient because we perceive ourselves to be sentient, and share the same medium of human brains and bodies.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems increasingly likely that &quot;algorithms sufficiently advanced&quot; will one day be able to model human &lt;a href=&quot;https://en.wikipedia.org/wiki/Cognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;cognition&lt;/a&gt;, but the real question will be is it actual cognition in the sense of understanding, or mere imitation? (&quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semantics&lt;/a&gt;&quot; vs. &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Syntax&quot; rel=&quot;nofollow noreferrer&quot;&gt;Syntactics&lt;/a&gt;&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cognition is derived from the Latin &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0060%3Aentry%3Dcognosco&quot; rel=&quot;nofollow noreferrer&quot;&gt;cognosco&lt;/a&gt;&lt;/em&gt;, itself related to the Greek words such as &lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0057%3Aentry%3Dgnwsto%2Fs&quot; rel=&quot;nofollow noreferrer&quot;&gt;γνωστός&lt;/a&gt; (a root of &quot;gnostic&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question and definition of sentience is less clear cut, but the argument re: algorithms relates to cognition thus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The lights may be on, but is anybody home?&quot;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-01-14T16:34:47.987" LastActivityDate="2019-01-14T16:34:47.987" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9961" PostTypeId="2" ParentId="9935" CreationDate="2019-01-13T11:56:12.210" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The problem is that this kind of data augmentation seems (in theory) to hamper the network to gain precision on bounding boxes location as it loosens the frame.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, it is clear from your examples that the bounding boxes become wider. Generally, including large amounts of data like this in your training data will mean that your network will also have a tendency to learn slightly larger bounding boxes. Of course, if the majority of your training data still has tight boxes, it should stell tend towards learning those... but likely slightly wider ones than if the training data did not include these kinds of rotations.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there some studies dealing with the effect of data augmentation on the precision of detection networks? Are there systems that prevent this kind of thing?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(Obviously, it seems advisable to use small rotation angles)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I do not personally work directly in the area of computer vision really, so I'm not sufficiently familiar with the literature to point you to any references on this particular issue. Based on my own intuition, I can recommend:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using relatively small rotation angles, as you also already suggested yourself. The bounding boxes will become a little bit wider than in the original dataset, but not by too much.&lt;/li&gt;&#xA;&lt;li&gt;Using rotation angles that are a multiple of &lt;span class=&quot;math-container&quot;&gt;$90^\circ$&lt;/span&gt;. Note that if you rotate a bounding box by a multiple of &lt;span class=&quot;math-container&quot;&gt;$90^\circ$&lt;/span&gt;, the rotated bounding boxes become axis-aligned and your problem disappears again, they'll become just as tight as the bounding boxes in the unrotated image. Of course, you can also combine this suggestion with the previous one, and use rotation angles in, for example, &lt;span class=&quot;math-container&quot;&gt;$[85^\circ, 95^\circ]$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Apply larger rotations primarily in images that only have bounding boxes that are approximately &quot;square&quot;. From looking at your image, I get the impression that the problem of bounding boxes becoming wider after rotations is much more severe when you have extremely wide or thin bounding boxes (with one dimension much greater than the other). When the original bounding box is square, there still will be some widening after rotation, but not nearly as much, so the problem may be more acceptable in such cases.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-13T11:56:12.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9962" PostTypeId="1" AcceptedAnswerId="9964" CreationDate="2019-01-13T13:41:27.793" Score="0" ViewCount="42" Body="&lt;p&gt;Why is the e-function used to decide whether to accept a worse solution or not? &#xA;To be more specific: Why was &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; chosen as basis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The propability to accept a worse solution is described with:&#xA;&lt;span class=&quot;math-container&quot;&gt;$p=e^{-\frac{E(y)-E(x)}{kT}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E(y)$&lt;/span&gt; is the energy from the old solution&#xA;&lt;span class=&quot;math-container&quot;&gt;$E(x)$&lt;/span&gt; is the energy from new solution &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is a constant temprature decreasing with a constant factor k in every iteration.&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="21157" LastEditDate="2019-01-14T09:20:03.577" LastActivityDate="2019-01-14T09:20:03.577" Title="Simulated Annealing: Why is e-function used as propability function to decide to accept a worse solution" Tags="&lt;algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9964" PostTypeId="2" ParentId="9962" CreationDate="2019-01-13T17:53:23.643" Score="0" Body="&lt;p&gt;You can find the explanation by asking some question about the function. Suppose, the value of &lt;span class=&quot;math-container&quot;&gt;$\frac{E(y)-E(x)}{kT} &amp;gt;&amp;gt; 0$&lt;/span&gt; is much more greater than zero. What does it mean? It means the value of &lt;span class=&quot;math-container&quot;&gt;$E(y)$&lt;/span&gt; is much greater than &lt;span class=&quot;math-container&quot;&gt;$E(x)$&lt;/span&gt; related to the &lt;span class=&quot;math-container&quot;&gt;$kT$&lt;/span&gt; that is as a measure of temperature decreasing. Now, you want in this situation a probability which is near to zero. Hence, &lt;span class=&quot;math-container&quot;&gt;$e^{-\frac{E(y)-E(x)}{kT}}$&lt;/span&gt; could be a good value for the probability of selection of worse solutions! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; or other values greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;? Because it could be a good function in optimization problems as its derivative is more simple than others!&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-01-13T17:53:23.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9965" PostTypeId="1" CreationDate="2019-01-13T18:10:36.873" Score="1" ViewCount="84" Body="&lt;p&gt;I am currently working with classical roboticists who insist on inverse kinematics, and what I (perhaps mistakenly) call the old way of thinking about robots accomplishing tasks. &#xA;Much of the relatively recent research focuses on Robots using Brain models such as   Multiple timescales (Artificial Intelligence models) that segment sequences and reproduce them, having learned them. The problem I face is this bunch of roboticists insist that a robot already knows the sequence, and training it to be reproduced is redundant, since a Robot can already reproduce the sequence anyway.&#xA;How accurate would you rate this assessment of using AI in robotics?&#xA;Are there any advantages of using AI to learn sequences for robot control?&lt;/p&gt;&#xA;" OwnerUserId="21397" LastActivityDate="2019-01-13T18:10:36.873" Title="Using Artificial Intelligence for Robot movement instead of regular Inverse Kinematics" Tags="&lt;deep-learning&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9966" PostTypeId="1" CreationDate="2019-01-13T18:38:00.237" Score="2" ViewCount="135" Body="&lt;p&gt;I would appreciate your help with this (naive) question of mine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the set of points located on a circle, &lt;span class=&quot;math-container&quot;&gt;$x_{i}, y_{i}$&lt;/span&gt; as the input data, Can a deep/machine learning algorithm infer that radius of the circle is constant ? In other words, given the data &lt;span class=&quot;math-container&quot;&gt;$x_{i}, y_{i}$&lt;/span&gt; is there way that algorithm discovers the constraint: &lt;span class=&quot;math-container&quot;&gt;$x_{i}^2 + y_{i}^2 = \text{constant}$&lt;/span&gt; ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also appreciate any related reference on the subject.    &lt;/p&gt;&#xA;" OwnerUserId="21399" LastActivityDate="2021-02-08T20:07:25.787" Title="Extracting algebraic constraints from the input data" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9968" PostTypeId="1" CreationDate="2019-01-13T20:34:47.680" Score="2" ViewCount="32" Body="&lt;p&gt;I have data that are a result of &lt;strong&gt;rules&lt;/strong&gt; that are &lt;strong&gt;exceptionless&lt;/strong&gt;. I want to my program to &lt;em&gt;'look'&lt;/em&gt; at my data and figure out those rules. However, the data might contain what might look like an exception (rule within a rule) but that is too, true for all occasions e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All men of the dataset with x common characteristics go out for a beer on Thursday after work. That is true for all men with those characteristics. However, they will cancel their plans if their wife is sick. That last condition might initially look as an exception to the rule (go out for beer on Thursdays), but it is not as long as it is true for all men with those x characteristics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So the question is&lt;/strong&gt;: &lt;em&gt;Which approach/method would be suitable for this?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="19393" LastActivityDate="2019-01-13T20:34:47.680" Title="How can I model regularity?" Tags="&lt;regularization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9969" PostTypeId="1" CreationDate="2019-01-14T01:32:20.787" Score="2" ViewCount="265" Body="&lt;p&gt;I have a machine learning project in which I have to separate spam and &lt;a href=&quot;https://en.wiktionary.org/wiki/ham_e-mail&quot; rel=&quot;nofollow noreferrer&quot;&gt;ham&lt;/a&gt; emails from a given dataset which has many txt files. Some of them are spam emails and the other ham emails. I have to implement an ID3 algorithm for this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I begin with this implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use a hashmap to have the words I read from the txt files and the times I read every single word? Something like &lt;code&gt;HashMap&amp;lt;String, int&amp;gt;&lt;/code&gt; or should I use two arrays maybe?&lt;/p&gt;&#xA;" OwnerUserId="21403" LastEditorUserId="2444" LastEditDate="2019-06-01T10:45:53.847" LastActivityDate="2019-06-01T10:45:53.847" Title="Separating spam and ham emails using the ID3 algorithm" Tags="&lt;machine-learning&gt;&lt;getting-started&gt;&lt;java&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9970" PostTypeId="2" ParentId="9954" CreationDate="2019-01-14T01:42:28.910" Score="3" Body="&lt;p&gt;What this is talking about is how much a machine learning algorithm is good at &quot;memorizing&quot; the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear hyper surfaces, and therefore tend to under-fit the data and may not give very good prediction, but may behave better on new unseen data which is very different from the training data.&lt;/p&gt;&#xA;" OwnerUserId="177" LastActivityDate="2019-01-14T01:42:28.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9973" PostTypeId="1" AcceptedAnswerId="9995" CreationDate="2019-01-14T05:38:33.113" Score="9" ViewCount="183" Body="&lt;p&gt;I was wondering if machine learning algorithms (CNNs?) can be used/trained to differentiate between small differences in details between images (such as slight differences in shades of red or other colours, or the presence of small objects between otherwise very similar images?)? And then classify images based on these differences? If this is a difficult endeavour with our current machine learning algorithms, how can it be solved? Would using more data (more images) help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also appreciate it if people could please provide references to research that has focused on this, if possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've only just begun learning machine learning, and this is something that I've been wondering from my research.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="2444" LastEditDate="2020-05-30T12:40:18.780" LastActivityDate="2020-05-30T12:46:59.213" Title="Can machine learning algorithms be used to differentiate between small differences in details between images?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9975" PostTypeId="1" AcceptedAnswerId="10197" CreationDate="2019-01-14T09:33:09.793" Score="2" ViewCount="842" Body="&lt;p&gt;How do you distinguish between a complex and a simple model in machine learning? Which parameters control the complexity or simplicity of a model? Is it the number of inputs, or maybe the number of layers?&lt;/p&gt;&#xA;&lt;p&gt;Moreover, when should a simple model be used instead of a complex one, and vice-versa?&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2020-09-20T10:44:20.813" LastActivityDate="2020-09-20T10:44:20.813" Title="How do you distinguish between a complex and a simple model in machine learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;models&gt;&lt;computational-learning-theory&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9976" PostTypeId="2" ParentId="9975" CreationDate="2019-01-14T09:42:01.280" Score="1" Body="&lt;p&gt;If you want to find a proper architecture for your model, you can use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_architecture_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;NAS&lt;/a&gt; (neural architecture search) methods instead of running some naive models to find a model and involving to decide which model is more complex or simpler. Some methods which used in NAS to find a proper architecture are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;NAS with Reinforcement Learning&lt;/li&gt;&#xA;&lt;li&gt;NAS with Evolution&lt;/li&gt;&#xA;&lt;li&gt;NAS with Hill-climbing&lt;/li&gt;&#xA;&lt;li&gt;Multi-objective Neural architecture search&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-01-14T09:56:13.733" LastActivityDate="2019-01-14T09:56:13.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9978" PostTypeId="1" CreationDate="2019-01-14T09:47:29.040" Score="2" ViewCount="38" Body="&lt;p&gt;I am reading about &lt;a href=&quot;http://www.scholarpedia.org/article/Continuous_attractor_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;CANN&lt;/a&gt;, however, I do not seem to grasp what it is. Maybe someone who has worked with it can explain it? I found out about it while reading about RatSLAM. I understand that it helps to keep long/short term memory. &lt;/p&gt;&#xA;" OwnerUserId="14863" LastActivityDate="2019-01-14T09:47:29.040" Title="Continuous-attractor neural network explanation" Tags="&lt;neural-networks&gt;&lt;terminology&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9979" PostTypeId="2" ParentId="9966" CreationDate="2019-01-14T10:00:13.777" Score="0" Body="&lt;p&gt;I'm assuming the question intends to find the maximally constraining bounding relation from among a large variety of other possible relations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bounding relation is likely linked to the chaos idea of attractors&lt;span class=&quot;math-container&quot;&gt;$^1$&lt;/span&gt;. Some contemporary visualizations of attractors in phase space are visualized as bounds rather than translucent paths or cross-sectional means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple form used as an example, the circle, never actually exist in nature. It is an approximation of a differential equation for an orbit with a eccentricity of zero in three space projected onto a parallel plane. But even if this problem is restricted to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt;, the concepts of attractors may pertain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If quadratic or cubic approximation is adequate for your purposes then it is a matter of finding the zero terms in the generalization. To find non-polynomial relations, the differential equations are the first things to find, but there may be no equivalent closed form in algebra&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://ijfritz.byethost4.com/Chaos/ch_close.htm?i=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ijfritz.byethost4.com/Chaos/ch_close.htm?i=1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-14T10:00:13.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9981" PostTypeId="2" ParentId="9969" CreationDate="2019-01-14T10:31:47.383" Score="1" Body="&lt;p&gt;Before thinking about implementation details such as what data structures to use, consider what you need as input for the ID3 algorithm.  You need a training set which comprises a list of categorised objects (your email, categorised as 'spam' or 'ham'), where each object is represented as a list of features (preferably binary, but not necessarily).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What features are you going to use? This could be the existence of certain words, but maybe phrases might be more useful. Does the email contain more than 5% wrongly spelled words? Are there funny non-ASCII characters in the text? Anything you think might be useful in separating the spam from the ham could be a feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proper term for this representation is as a &lt;em&gt;feature vector&lt;/em&gt;, so once you come to the implementation stage, you might want an array/list/vector to represent the feature values for each email. This set of feature vectors is then the input to your ID£ algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question: start by thinking what features to use, how to extract them from the data, and how to represent them effectively. The better you plan this, the easier the implementation will be.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-01-14T10:31:47.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9982" PostTypeId="1" AcceptedAnswerId="9986" CreationDate="2019-01-14T11:59:12.760" Score="7" ViewCount="1627" Body="&lt;p&gt;What are the current NLP/NLU techniques that can extract metaphors from texts?&lt;/p&gt;&#xA;&lt;p&gt;For example&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;His words cut deeper than a knife.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Or a simpler form like:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Life is a journey that must be travelled no matter how bad the roads and accommodations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="21415" LastEditorUserId="2444" LastEditDate="2021-01-15T00:30:48.417" LastActivityDate="2021-01-15T00:30:48.417" Title="How to recognise metaphors in texts using NLP/NLU?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9983" PostTypeId="1" CreationDate="2019-01-14T12:43:31.843" Score="7" ViewCount="1504" Body="&lt;p&gt;I was thinking of something of the sort:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Build a program (call this one fake user) that generates lots and lots and lots of data based on the usage of another program (call this one target) using stimuli and response. For example, if the target is a minesweeper, the fake user would play the game a carl sagan number of times, as well as try to click all buttons on all sorts of different situations, etc...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;run a machine learning program (call this one the copier) designed to evolve a code that works as similar as possible to the target. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;kablam, you have a &quot;sufficiently nice&quot; open source copy of the target.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is this possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is something else possible to achieve the same result, namely, to obtain a &quot;sufficiently nice&quot; open source copy of the original target program?&lt;/p&gt;&#xA;" OwnerUserId="20976" LastEditorUserId="2444" LastEditDate="2019-06-28T16:54:32.373" LastActivityDate="2020-08-14T17:42:52.017" Title="Is it possible to use AI to reverse engineer software?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9984" PostTypeId="2" ParentId="9983" CreationDate="2019-01-14T13:10:42.613" Score="1" Body="&lt;p&gt;Remarkably, more or less the scenario you describe is not only feasible and has already been demonstrated &lt;a href=&quot;https://worldmodels.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;(detailed explanation and fascinating videos at link)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the fidelity of the copy is currently quite limited:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FO0Af.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FO0Af.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for now, your copy will be quite low quality.  However, there is a big exception to this rule: if the software you are copying is itself based on machine learning, then you can probably make a high-quality copy quite cheaply and easy, &lt;a href=&quot;https://medium.com/@damilare/pirating-ai-800a8da6431b&quot; rel=&quot;nofollow noreferrer&quot;&gt;as I and my co-authors explain in this short article.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interesting question and I'm quite sure that the correct answer will change rapidly over the next few years.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-01-14T13:10:42.613" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="9986" PostTypeId="2" ParentId="9982" CreationDate="2019-01-14T13:58:23.313" Score="3" Body="&lt;p&gt;This is still a research topic in linguistics. A quick google search brings up a couple of papers that might be useful:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/231608823_Identifying_Metaphor_Hierarchies_in_a_Corpus_Analysis_of_Finance_Articles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Identifying Metaphor Hierarchies in a Corpus Analysis of Finance Articles&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3639214/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metaphor Identification in Large Texts Corpora&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, you probably won't get an off-the-shelf tool that recognises metaphors for you.&lt;/p&gt;&#xA;&lt;p&gt;To add more details, the problem with metaphors is that you cannot detect them by surface structure alone. Any sentence could (in theory) be a metaphor. This is different from a &lt;a href=&quot;https://www.merriam-webster.com/dictionary/simile&quot; rel=&quot;nofollow noreferrer&quot;&gt;simile&lt;/a&gt;, which can usually be spotted easily through the word &lt;em&gt;like&lt;/em&gt;, as in &lt;em&gt;she runs like the wind.&lt;/em&gt; Obviously, &lt;em&gt;like&lt;/em&gt; on its own is not sufficient, but it's a good starting point to identify possible candidates.&lt;/p&gt;&#xA;&lt;p&gt;However, &lt;em&gt;his words cut deeper than a knife&lt;/em&gt; is -- on the surface -- a normal sentence. Only the semantic incongruence between &lt;em&gt;words&lt;/em&gt; as the subject and &lt;em&gt;cut&lt;/em&gt; as the main verb creates a clash. In order to detect this automatically, you need to identify possible semantic features of the verbal roles and look for violations of the expected pattern.&lt;/p&gt;&#xA;&lt;p&gt;The verb &lt;em&gt;cut&lt;/em&gt; would generally expect an animate object, preferably human, or an instrument with a blade (&lt;em&gt;the knife cuts through the butter&lt;/em&gt;) as its actor or subject. But it also can include (water)ways: &lt;em&gt;the canal cuts through the landscape&lt;/em&gt;, &lt;em&gt;the road cuts through the field&lt;/em&gt;. The more closely you look, the more exceptions/extensions you will find for your initial assumption.&lt;/p&gt;&#xA;&lt;p&gt;And every extension/exception will water down the accuracy of your metaphor detection algorithm.&lt;/p&gt;&#xA;&lt;p&gt;The second example is similar: &lt;em&gt;Life is a journey&lt;/em&gt;. You could perhaps use a thesaurus and see what the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyponymy_and_hypernymy&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyperonyms&lt;/a&gt; of &lt;em&gt;life&lt;/em&gt; are. Then you could do the same with &lt;em&gt;journey&lt;/em&gt;, and see if they are compatible. &lt;em&gt;A car is a vehicle&lt;/em&gt; is not a metaphor, because &lt;em&gt;vehicle&lt;/em&gt; is a hyperonym of &lt;em&gt;car&lt;/em&gt;. But &lt;em&gt;journey&lt;/em&gt; is not a hyperonym of &lt;em&gt;life&lt;/em&gt;, so could be a metaphor. But I would think that this is still very tricky to get right. In this case, the absence of a determiner might be a hint, as it's not &lt;em&gt;a life is a journey&lt;/em&gt; -- you might restrict yourself to bare nouns for this type of metaphor. But this is also not a firm rule.&lt;/p&gt;&#xA;&lt;p&gt;In short, it is a hard problem, as you need to look at the meaning, rather than just the structure or word choice. And meaning is not easy to deal with in NLP, despite decades of work on it.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2444" LastEditDate="2021-01-15T00:27:06.990" LastActivityDate="2021-01-15T00:27:06.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9987" PostTypeId="1" AcceptedAnswerId="9988" CreationDate="2019-01-14T16:03:44.710" Score="2" ViewCount="469" Body="&lt;p&gt;I want to start a project for my artificial intelligence class about speaker recognition. Basically, I want to train my AI to detect if it's me who's speaking or somebody else. I would like some suggestions or libraries to work with.&lt;/p&gt;&#xA;" OwnerUserId="21421" LastEditorUserId="16229" LastEditDate="2019-01-21T20:58:07.653" LastActivityDate="2019-10-18T23:01:35.233" Title="Training an AI to recognize my voice (or any voice)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9988" PostTypeId="2" ParentId="9987" CreationDate="2019-01-14T17:16:39.160" Score="2" Body="&lt;p&gt;The human voice is based on the neural muscular control of vocal apparatus made up of many parts.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Diaphragm&lt;/li&gt;&#xA;&lt;li&gt;Vocal cords&lt;/li&gt;&#xA;&lt;li&gt;Throat (constrictors and anti-constrictors)&lt;/li&gt;&#xA;&lt;li&gt;Nasal cavity&lt;/li&gt;&#xA;&lt;li&gt;Cheek&lt;/li&gt;&#xA;&lt;li&gt;Jaw&lt;/li&gt;&#xA;&lt;li&gt;Tongue&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These coordinated muscular manipulations produce envelopes (controlling) of audio that can be characterized by periodic and transient wave forms. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Volume&lt;/li&gt;&#xA;&lt;li&gt;Pitch&lt;/li&gt;&#xA;&lt;li&gt;Tone (relative volume of harmonics)&lt;/li&gt;&#xA;&lt;li&gt;Consonant transients&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Voices are unique to the learning state of neural activity and anatomic attributes, which is a way of saying that vocal habits and the physical attributes of the voice supports the distinguishing of vocal identity.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Strength of vocal muscles&lt;/li&gt;&#xA;&lt;li&gt;Connectivity of muscles to bone, tendons, and cartilage&lt;/li&gt;&#xA;&lt;li&gt;Shape of inner surface of vocal pathways&lt;/li&gt;&#xA;&lt;li&gt;Neural coordination of those muscles&lt;/li&gt;&#xA;&lt;li&gt;Neural production of phonetic control to produce linguistic elements&lt;/li&gt;&#xA;&lt;li&gt;Neural serialization of semantic structures (ideas)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The detection of distinguishing features of voices by the ear is equally complex. In a room full of people talking, the brain can learn to track a single voice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to note that performing voice recognition to determine the identity of the human source is significantly different than performing voice recognition to produce text. To produce text accurately, the NLP must determine language elements and construct a semantic network that represents the vocal content or a text from that representation to be accurate in the case of like sounding words. Fortunately, the identification of the speaker is easier in some ways than the accurate voice to text. Unfortunately, the identification of the speaker has general limitations discussed below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first stage of hearing in the ear is mechanical, involving the length of hairs along the cochlear surface, which is like a radio tuner that discriminates all frequencies within a range simultaneously. The software equivalent is a spectrum derived by applying a root mean square to the result of an FFT (fast Fourier transform) to provide magnitudes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ m_f := \sqrt{t_f^2 + {(it_f)}^2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phase component of the FFT results (&lt;span class=&quot;math-container&quot;&gt;$\, \arctan(t, it) \,)$&lt;/span&gt; can be discarded, since it is not correlated with neural control of voice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The application of the FFT to speech (as with any changing audio) requires windowing over the audio samples using one of the windowing tapers, such as the Hann window or Blackman window. The input is the audio stream or file contents as a sequence of pressure samples, the audio. The output is a sequence of spectra, each containing the volume of each frequency in the vocal range, from about 30 Hertz to 15 K Hertz.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This series of spectra can be fed into the initial layer of one of the more advanced RNNs (recurrent neural networks), such as the LSTM (long short term memory) networks, its bidirectional version, the B-LSTM, or a GRU (gated recurrent network), which is touted as training equally well with less time or computing resource consumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The identity of the speaker is the label. The series of spectra are the features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the PAC (probably approximately correct) learning framework, it may be possible to estimate, in advance of experimentation, the minimum number of words the speaker must speak to produce a particular accuracy and reliability in use of the learned parameters from the network training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will take some study to set up the hyper-parameters and design the layers of the network in terms of depth (number of layers) and width sequence (number of cells per layer, which may vary from layer to layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use case limitation of this system is that each speaker must read some text that provides adequate training example sequences of adequate length, so that there are sufficient number of overlapping windows for the FFT to transform into spectra so that the training converges reasonably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no way around the individual user training as there is with recognition of linguistic content, which can be trained across a large set of speakers to recognize content somewhat independent of the speaker. The system can be adjusted and improved to minimize the amount of speech required, but information theory constraints keep that quantity from ever approaching zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No network, whether artificial or biological, can learn something from nothing. Claude Shannon and John von Neumann realized decades ago that there is a kind of conservation of information, just as there is a conservation of matter and energy in space below nuclear reaction thresholds. This led to the definition of a bit and the formulation of information as a quantity of bits corresponding to a narrowing of probability that the information provides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ b_i = - \log_2 {\frac {P(x|i)} {P(x)}} $$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-14T18:46:59.873" LastActivityDate="2019-01-14T18:46:59.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9990" PostTypeId="1" CreationDate="2019-01-14T18:11:35.290" Score="2" ViewCount="1424" Body="&lt;p&gt;I was trying to understand the loss function of GANs, while I found a little mis-match between different papers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the screen-shot from the original paper of Goodfellow at &lt;a href=&quot;https://arxiv.org/pdf/1406.2661.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1406.2661.pdf&lt;/a&gt;: &lt;a href=&quot;https://i.stack.imgur.com/VhMD7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VhMD7.png&quot; alt=&quot;Goodfellow&quot;&gt;&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And equation (1) in this version of pix2pix paper at &lt;a href=&quot;https://arxiv.org/pdf/1611.07004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1611.07004.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MXlU3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MXlU3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting aside the fact that pix2pix is using conditional GAN, which introduces a second term &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, the 2 formulas are quite resemble, except that in the pix2pix paper, they try to get minimax of &lt;span class=&quot;math-container&quot;&gt;${\cal{L}}_{cGAN}(G, D)$&lt;/span&gt;, which is defined to be &lt;span class=&quot;math-container&quot;&gt;$E_{x,y}[...] + E_{x,z}[...]$&lt;/span&gt;, whereas in the original paper, they define &lt;span class=&quot;math-container&quot;&gt;$\min\max V(G, D) = E[...] + E[...]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not coming from a good math background, so I am quite confused. I'm not sure where the mistake is, but assuming that &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is expectation (correct me if I'm wrong), the version in pix2pix makes more sense to me, although I think it's quite less likely that Goodfellow could make this mistake in his amazing paper. Maybe there's no mistake at all and it's me who do not understand them correctly.&lt;/p&gt;&#xA;" OwnerUserId="3098" LastEditorUserId="3098" LastEditDate="2019-01-14T18:18:09.683" LastActivityDate="2019-02-24T15:01:55.373" Title="Confusing on GAN loss function" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;generative-model&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9993" PostTypeId="2" ParentId="9990" CreationDate="2019-01-15T00:46:39.527" Score="1" Body="&lt;p&gt;What is meant by both papers is that we have two agents (generator and discriminator) playing a game with the value function &lt;code&gt;V&lt;/code&gt; defined as a sum of the expectations (i.e. an expectation of the outcome value defined as a sum of two terms, or actually a logarithm of a product...). The generator uses a strategy &lt;code&gt;G&lt;/code&gt; encoded in the parameters of its neural network (&lt;code&gt;θg&lt;/code&gt;), the discriminator uses a strategy &lt;code&gt;D&lt;/code&gt; encoded in the parameters of its neural network (&lt;code&gt;θd&lt;/code&gt;). Our goal is to (hopefully) find such a pair of strategies (a pair of parameter sets &lt;code&gt;θgmin&lt;/code&gt; and &lt;code&gt;θdmax&lt;/code&gt;) that produce the minimax value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While trying to find the (&lt;code&gt;θgmin&lt;/code&gt;, &lt;code&gt;θdmax&lt;/code&gt;) pair using gradient descent, we actually have two loss functions: one is the loss function for &lt;code&gt;G&lt;/code&gt;, parameterized by &lt;code&gt;θg&lt;/code&gt;, another is the loss function for &lt;code&gt;D&lt;/code&gt;, parameterized by &lt;code&gt;θd&lt;/code&gt;, and we train them alternatively on minibatches together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the Algorithm 1 in the original paper, the loss function for the discriminator is &lt;code&gt;-log(D(x; θd)) - log(1 - D(G(z); θd)&lt;/code&gt;, and the loss function for the generator is &lt;code&gt;log(1 - D(G(z; θg))&lt;/code&gt; (in both cases, in the original paper, &lt;code&gt;x&lt;/code&gt; is sampled from the reference data distribution and &lt;code&gt;z&lt;/code&gt; is sampled from noise):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ideal value for the loss function of the discriminator is 0, otherwise it's greater than 0. The &quot;loss&quot; function of the generator is actually negative, but, for better gradient descent behavior, can be replaced with &lt;code&gt;-log(D(G(z; θg))&lt;/code&gt;, which also has the ideal value for the generator at 0. It is impossible to reach zero loss for both generator and discriminator in the same GAN at the same time. However, the idea of the GAN is not to reach zero loss for any of the game agents (this is actually counterproductive), but to use that &quot;double gradient descent&quot; to &quot;converge&quot; the distribution of &lt;code&gt;G(z)&lt;/code&gt; to the distribution of &lt;code&gt;x&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21426" LastEditorUserId="21426" LastEditDate="2019-01-21T09:29:37.157" LastActivityDate="2019-01-21T09:29:37.157" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9995" PostTypeId="2" ParentId="9973" CreationDate="2019-01-15T03:28:33.557" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.00767.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attentive Recurrent Comparators&lt;/a&gt; (2017) by Pranav Shyam et al. is an interesting paper that helps to answer the question you're wondering, along with a &lt;a href=&quot;https://medium.com/datadriveninvestor/spot-the-difference-deep-learning-edition-f1a891342e13&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; that helps to describe it in easier terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way it's implemented is actually rather intuitive. If you have ever played a &quot;what is different&quot; game with two images usually what you'd do is look back and forth between the images to see what the difference is.  The network that the researchers created does just that!  It looks at one image and then remembers important features about that images and looks at the other image and goes back and forth.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastEditorUserId="2444" LastEditDate="2020-05-30T12:32:23.490" LastActivityDate="2020-05-30T12:32:23.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9996" PostTypeId="1" CreationDate="2019-01-15T04:16:38.597" Score="1" ViewCount="71" Body="&lt;p&gt;I am trying to generate a model that uses several physicochemical properties of a molecule (including number of atoms, number of rings, volume, etc.) to predict a numeric value &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. I would like to use PLS Regression, and I understand that &lt;em&gt;standardization&lt;/em&gt; is very important here. I am programming in Python, using scikit-learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The type and range for the features varies. Some are int64 while others are floating point numbers. Some features generally have small (positive or negative) values, while others have a very large value. I have tried using various scalers (e.g. standard scaler, normalize, min-max scaler, etc.). Yet, the R2/Q2 are still low. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible that by scaling, some of the very important features lose their significance, and thus contribute less to explaining the variance of the response variable?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If yes, if I identify some important features (by expert knowledge), is it OK to scale other features but those? Or scale the important features only?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some of the features, although not always correlated, have values that are in a similar range (e.g. 100-400), compared to others (e.g. -1 to 10). Is it possible to scale only a specific group of features that are within the same range?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21431" LastEditorUserId="2444" LastEditDate="2019-06-28T11:22:43.297" LastActivityDate="2019-06-28T11:24:08.433" Title="What is the impact of scaling the features on the performance of the model?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;data-science&gt;&lt;data-preprocessing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9997" PostTypeId="2" ParentId="9934" CreationDate="2019-01-15T05:58:46.263" Score="3" Body="&lt;p&gt;&lt;strong&gt;Cartesian Bias and Pipeline Efficiency&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are experiencing a techno-cultural artifact of Cartesian-centric imaging running all the way back to the dawn of coordinate systems. It is the momentum of practice as a consequence of applying Cartesian 2D coordinates to rasterize images appearing at the focal planes of lenses from the dawn of television and the earliest standards of raster based capture and display.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although some work was done toward adding tilt to bounding rectangles in the late 1990s and since, from a time and computing resource conservation perspective, it is computationally and programmatically less costly to include the four useless triangles of pixels and keep the bounding box orthogonal with the pixel grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding a tilt angle to the bounding boxes is marginally competitive when detecting ships from a satellite only because two conditions offset the inefficiencies in that narrow domain. The ship appears as an oblong rectangle with rounded corners from a satellite positioned in geosynchronous orbit. In the general case, adding a tilt angle can slow recognition significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Biology Less Biased&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting side note is that the neural networks of animal and human vision systems do not have that Cartesian-centricity, but that doesn't help this question's solution, since non-orthogonal hardware and software is virtually nonexistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Early Non-Cartesian Research and Today's Rasterization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gerber Scientific Techonology research and development in the 1980s (South Windsor, Connecticut, U.S.) investigated vector capture, storage, and display, but the R&amp;amp;D was not financially sustainable for a mid-side technology corporation for the reasons above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What remains, because it is economically viable and necessary from an animation point of view, is rasterization on the end of the system that converts vector models into frames of pixels. We see this in on the rendering  SVG, VRML, and the original intent of CUDA cores and other hardware rendering acceleration strategies and architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the object and action recognition side, the support of vector models directly from imaging is much less developed. This has not been a major stumbling block for computer vision because the wasted pixels at one tilt angle may be of central importance at another tilt angle, so there are no actual wasted input pixels if the centering of key scene elements is widely distributed in translation and tilt, which is often the case in real life (although not so much in hygienically pre-processed datasets).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conventions Around Object Minus Camera Tilt and Skew from Parallax&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once edge detection, interior-versus-exterior, and 3D solid recognition come into play, the design of CNN pipelines and the way kernels can do radial transformation without actually requiring &lt;span class=&quot;math-container&quot;&gt;$\; \sin, \, \cos, \, \text{and} \, \arctan \;$&lt;/span&gt; functions evaporate the computational burden of the Cartesian nature of pixel tensors. The end result is that the bounding box being orthogonal to the image frame is not as problematic as it initially appears. Efforts to conserve the four triangles of pixels and pre-process orientation is often a wasted effort by a gross margin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottom line is that efforts to produce vector recognition from roster inputs have been significantly inferior in terms of resource and wait time burden, with the exception of insignificant gains in the narrow domain of naval reconnaissance satellite images. Trigonometry is expensive, but convolution kernels, especially now that they are moving from software into hardware accelerated computing paths in VLSI, is computable at lower costs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Past and Current Work&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is some work that deals with tilting with regard to objects and the effects of parallax in relation to the Cartesian coordinate system of the raster representation. Most of the work has to do with recognizing 3D objects in a 3D coordinate system to project trajectories and pilot or drive vehicles rationally on the basis of Newtonian mechanics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://crack.seismo.unr.edu/downloads/russell/trees/klosowski_1998_efficient_collision_k-DOPs.PDF&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Efficient Collision Detection Using Bounding Volume Hierarchies of k-DOPs&lt;/em&gt;&lt;/a&gt;, James T. Klosowski, Martin Held, Joseph S.B. Mitchell, Henry Sowizral, and Karel Zikan, 1998&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/832e/eaede4f697f005b007798fe0d04829f656b7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Sliding Shapes for 3D Object Detection in Depth Images&lt;/em&gt;&lt;/a&gt;, Shuran Song and Jianxiong Xiao, 2014&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kar_Amodal_Completion_and_ICCV_2015_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Amodal Completion and Size Constancy in Natural Scenes&lt;/em&gt;&lt;/a&gt;, Abhishek Kar, Shubham Tulsiani, Joao Carreira and Jitendra Malik, 2015&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1609.04147.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;HMD Vision-based Teleoperating UGV and UAV for Hostile&#xA;Environment using Deep Learning&lt;/em&gt;&lt;/a&gt;, Abhishek Sawarkar1, Vishal Chaudhari, Rahul Chavan, Varun Zope, Akshay Budale and Faruk Kazi, 2016&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://159.226.21.68/bitstream/173211/14544/1/Ship%20Rotated%20Bounding%20Box%20Space%20for%20Ship%20Extraction%20from%20High-Resolution%20Optical%20Satellite%20Images%20with%20Complex%20Backgrounds.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds&lt;/em&gt;&lt;/a&gt;, Z Liu, H Wang, L Weng, Y Yang, 2016&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dnnseye.com/publications/papers/zhuo_3det_cvpr17.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Amodal Detection of 3D Objects:&#xA;Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Images&lt;/em&gt;&lt;/a&gt;, Zhuo Deng, 2017&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w31/Mahendran_3D_Pose_Regression_ICCV_2017_paper.pdf0&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;3D Pose Regression using Convolutional Neural Networks&lt;/em&gt;&lt;/a&gt;, Siddharth Mahendran, 2017&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mdpi.com/2226-4310/4/2/32/htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Aerial Target Tracking Algorithm Based on Faster R-CNN Combined with Frame Differencing&lt;/em&gt;&lt;/a&gt;, Yurong Yang, Huajun Gong, Xinhua Wang and Peng Sun, 2017&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w3/Kumar_A_Semi-Automatic_2D_CVPR_2018_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Semi-Automatic 2D solution for Vehicle Speed Estimation from Monocular Videos&lt;/em&gt;&lt;/a&gt;, Amit Kumar, Pirazh Khorramshahi, Wei-An Lin, Prithviraj Dhar, Jun-Cheng Chen, Rama Chellappa, 2018&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-16T09:10:06.290" LastActivityDate="2019-01-16T09:10:06.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9998" PostTypeId="2" ParentId="8885" CreationDate="2019-01-15T06:23:41.407" Score="4" Body="&lt;p&gt;The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you should know the result -- VAE can only have a mean and blurry output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you increase the bandwidth of the bottleneck, i.e. the size of latent vector, VAE can get a high reconstruction quality, e.g. &lt;a href=&quot;https://github.com/seasonyc/Spatial-Z-VAE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spatial-Z-VAE&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="21409" LastActivityDate="2019-01-15T06:23:41.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9999" PostTypeId="2" ParentId="7215" CreationDate="2019-01-15T06:52:30.853" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.biologia.buap.mx/ANTOLOGIA%20BIOFISICA%20I.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principles of Computational Modelling in Neuroscience&lt;/a&gt; by David Sterratt, Bruce Graham, Andrew Gillies and David Willshaw discuss it in Chapter 7 (The synapse) and also in Chapter 8 (Simplified models of neurons). Especially in chapter 8, they discuss how to add excitatory or inhibitory synapses to integrate and fire neuron. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various ways to add inhibitory synapse: either substrate voltage, inject negative current. &lt;/p&gt;&#xA;" OwnerUserId="21436" LastEditorUserId="2444" LastEditDate="2020-05-23T18:28:57.120" LastActivityDate="2020-05-23T18:28:57.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10000" PostTypeId="2" ParentId="1987" CreationDate="2019-01-15T07:59:04.600" Score="4" Body="&lt;p&gt;By cheating... &lt;code&gt;theta&lt;/code&gt; is &lt;span class=&quot;math-container&quot;&gt;$\arctan(y,x)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\sqrt{(x^2 + y^2)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y^2$&lt;/span&gt; should work, but, in practice, they somehow failed, even though, occasionally, it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mT4Nc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mT4Nc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21439" LastEditorUserId="2444" LastEditDate="2019-02-26T17:24:24.193" LastActivityDate="2019-02-26T17:24:24.193" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10001" PostTypeId="2" ParentId="9838" CreationDate="2019-01-15T08:09:29.120" Score="2" Body="&lt;p&gt;There is some confusion between reinforcement and convergence in this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The XOR problem is of interest in a historical context because the reliability of gradient descent is &lt;em&gt;identity&lt;/em&gt; (no advantage over an ideal coin toss) for a single layer perceptron when the data set is are the permutations representing the Boolean XOR operation. This is an information theory way of saying a single layer perceptron can't be used to learn arbitrary Boolean binary operations, with XOR and XAND as counterexamples where convergence is not only not guaranteed but productive of functional behavior only by virtue of luck. That is why the MLP was an important extension of the perceptron design. It can be reliably taught an XOR operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/search?q=deep+reinforcement+learning&amp;amp;source=lnms&amp;amp;tbm=isch&quot; rel=&quot;nofollow noreferrer&quot;&gt;Search results for images related to deep reinforced learning&lt;/a&gt; provide a survey of design diagrams representing the principles involved. We can note that the use case for a reinforcement learning application is distinctly different from that of MLPs and their derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parsing the term and recombining to produce the conceptual frameworks that were originally combined to produce DRL, we have &lt;em&gt;deep learning&lt;/em&gt; and &lt;em&gt;reinforcement learning&lt;/em&gt;. Deep learning is really a set of techniques and algorithmic refinements for the combination of artificial network layers into more successful topologies that perform useful data center tasks. Reinforcement learning is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sutton states in &lt;a href=&quot;https://login.cs.utexas.edu/sites/default/files/legacy_files/research/documents/1%20intro%20up%20to%20RL%3ATD.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;his slides for the University of Texas&lt;/a&gt; (possibly there to get away from the Alberta winters), &quot;RL is learning to control data.&quot; His is an overly broad definition, since MLPs, CNNs, and GRU networks all learn a function which is controlling data processing when the learned parameters are then leveraged in their intended use cases. This is where the perspective of the question may be based on the misinformative nature of these excessively broad definitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distinction of reinforced learning is the idea that a behavior can be reinforced during use. There may be actual parallel reinforcement of beneficial behavior (as in more neurologically inspired architectures) or learning may occur in a time slicing operating system and share the processing hardware with processes that use what is learned (as in Q-learning algorithms and their derivatives).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some define RL as machine learning technique that direct the selection of actions along a path of behavior such that some cumulative value of the consequences of actions take is maximized. That may be an excessively narrow definition, biased by the popularity of Markov processes and Q-learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the problem with the perspective expressed in the question. An XOR operation is not an environment through which a path can be blazed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one were to construct an XOR maze, where the initial state is undefined and the one single action is to fall into either the quadrant 10 or 01, it is still not representing an XOR because the input was not a Boolean vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec{B} \in \mathbb{B}^2 \; \text{,}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the output is not a 1 or 0 resulting from XOR operation, as would be the case for a multilayer perceptron learning of XOR operation. There is no cumulative reward. If there was no input and the move was to divide in half and chose both 10 or 01 because their reward was higher than 00 or 11, then that might be considered a reinforcement learning scenario, but it would be an odd one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That the described setup leads to, &quot;Getting stuck,&quot; is no surprise when the tool is a wrench for the turning of a screw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the design looses the reinforcement and the artificial network is reduced to a two layer perceptron, the convergence will be guaranteed given a labeled data set of sufficient size or an unsupervised arrangement where the loss function is simply the evaluation of whether the result is XOR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To experiment with reinforced learning, the agent must interact with the environment and make choices that have value consequences that direct subsequent behavior. Boolean expressions are not of this nature, no matter how complex.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-15T08:09:29.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10003" PostTypeId="1" CreationDate="2019-01-15T10:08:17.593" Score="2" ViewCount="121" Body="&lt;p&gt;Is there any way to control the extraction of features? How do I determine which features are been learned during training, i.e relevant information is been learned or not?&lt;/p&gt;&#xA;" OwnerUserId="21441" LastEditorUserId="2444" LastEditDate="2020-05-18T10:24:24.680" LastActivityDate="2020-05-19T04:05:27.760" Title="How do I determine which relevant features have been learned during training in a CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;feature-extraction&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10005" PostTypeId="2" ParentId="9973" CreationDate="2019-01-15T13:31:29.480" Score="3" Body="&lt;p&gt;It exists networks built to learn how to differentiate between classes even if there are looking quite the same. Usually, a &lt;a href=&quot;https://towardsdatascience.com/siamese-network-triplet-loss-b4ca82c1aec8&quot; rel=&quot;nofollow noreferrer&quot;&gt;triplet loss&lt;/a&gt; is used in those networks to learn the difference between the target, a positive sample, and a negative one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, those networks are used to perform identity check with face images, the algorithm learns the differences between different people instead of recognizing people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some keywords that are possibly relevant: discriminative function, triplet loss, siamese network, one-shot learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theses papers are interesting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://arxiv.org/abs/1503.03832&quot; rel=&quot;nofollow noreferrer&quot;&gt;FaceNet: A Unified Embedding for Face Recognition and Clustering&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dimensionality Reduction by Learning an Invariant Mapping&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19859" LastEditorUserId="2444" LastEditDate="2020-05-30T12:46:59.213" LastActivityDate="2020-05-30T12:46:59.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10007" PostTypeId="1" CreationDate="2019-01-15T16:04:22.477" Score="1" ViewCount="38" Body="&lt;p&gt;Let's say I want to model purchase data (i.e. purchase records of many households across time). For simplicity, let's assume each household only picks one alternative at the time. A simple starting point is a multinomial logit model. Then, more flexible network architectures could be used. People have applied NN to this, but kept the number of alternatives (K) constant. In reality, the number of available options changes over time. Also, it would be interesting to predict how choices change when the number of alternatives is changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in bullet points&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;there are N households &lt;/li&gt;&#xA;&lt;li&gt;t_n purchases for each household&lt;/li&gt;&#xA;&lt;li&gt;There are K_t alternatives at time t&lt;/li&gt;&#xA;&lt;li&gt;Dependent variable Y=k indicates that alternative k was bought&lt;/li&gt;&#xA;&lt;li&gt;Each alternative is characterized by features, so x_kt is a vector of those features (including brand name, price, ...). The number of features is constant across time. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any guidance or ideas?&lt;/p&gt;&#xA;" OwnerUserId="21451" LastActivityDate="2019-01-15T16:04:22.477" Title="NN: Predicting choices when number of alternatives changes" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10009" PostTypeId="1" CreationDate="2019-01-15T19:28:57.570" Score="1" ViewCount="77" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jLKuO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jLKuO.png&quot; alt=&quot;minimaxQuestion&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image is one of many similar exam questions can anyone pelase help me understand it fully?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Internal node': This is simply every node except A?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Move choices: His only options are B, C and D for this move?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Focusing on B: E=8 F=4 and G are all opponent responses, therefore they will pick the minimum value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my confusion, are M N and P your known responses in the case the opponent picks G, so you should pick M=0 (the highest value), so then G gets passed 0 which the opponent should choose so B has a h-value of 0? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the correct value then B=0, C=1 and D=2 so pick D as next move?&lt;/p&gt;&#xA;" OwnerUserId="21459" LastActivityDate="2019-01-15T19:28:57.570" Title="Can't grasp MiniMax diagram (no alpha beta pruning)" Tags="&lt;minimax&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10010" PostTypeId="1" AcceptedAnswerId="10089" CreationDate="2019-01-15T19:58:41.363" Score="1" ViewCount="388" Body="&lt;p&gt;Recurrent Neural Networks (RNN) With Attention Mechanism is generally used for Machine Translation and Natural Language Processing. In Python, implementation of RNN With Attention Mechanism is abundant in Machine Translation (For Eg. &lt;a href=&quot;https://talbaumel.github.io/blog/attention/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://talbaumel.github.io/blog/attention/&lt;/a&gt;, however what I would like to do is to use RNN With Attention Mechanism on a temporal data file (not any textual/sentence based data). I have a CSV file with of dimensions 21000 x 1936, which I have converted to a Dataframe using Pandas. The first column is of Datetime Format and last column consists of target classes like &quot;Class1&quot;, &quot;Class2&quot;, &quot;Class3&quot; etc. which I would like to identify. So in total, there are 21000 rows (instances of data in 10 minutes time-steps) and 1935 features. The last (1936th column) is the label column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is predominant from existing literature that an Attention Mechanism works quite well when coupled into the RNN. I am unable to locate any such implementation of RNN with Attention Mechanism, which can also provide a visualisation as well. Any help in this regard would be highly appreciated. Cheers! &lt;/p&gt;&#xA;" OwnerUserId="21460" LastEditorUserId="21460" LastEditDate="2019-01-15T20:10:25.127" LastActivityDate="2019-04-25T17:46:41.900" Title="How to use RNN With Attention Mechanism on Non Textual Data?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;recurrent-neural-networks&gt;&lt;attention&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10011" PostTypeId="2" ParentId="7525" CreationDate="2019-01-15T20:41:27.863" Score="3" Body="&lt;p&gt;In addition to the points already listed in &lt;a href=&quot;https://ai.stackexchange.com/a/7544/1641&quot;&gt;John's answer&lt;/a&gt;, some factors that can help to reduce / mitigate the risk of overfitting to commonly-used benchmarks as a research community are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Competitions with instances of problems hidden from entrants:&lt;/strong&gt; as far as I'm aware this is particularly popular in game AI (see the General Game Playing competition and General Video Game Playing competitions). The basic idea is that submissions should be able to tackle a relatively broad class of problems (playing &lt;em&gt;any&lt;/em&gt; game defined in a specified format, or generating levels for &lt;em&gt;any&lt;/em&gt; video game with rules described in a specific format, etc.). To some extent, using a large suite of problems as a standard benchmark (such as the large collection of Atari games supported by ALE) also fits in with this idea, though there is value in hiding the problems that are ultimately used for testing from the people writing submissions. Of course, the idea is that entries submitted to these kinds of competitions will involve new research which may be published.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Using very simple toy problems:&lt;/strong&gt; With simple I do not necessarily mean that they are simple to solve, but simple to describe / understand (it may still, for example, have a large state space and be difficult for current techniques to solve). Simple toy problems often help to test for a very specific &quot;skill&quot;, and can more easily give insight into specifically why/when an algorithm may be expected to fail or succeed. Of course, large non-toy problems are also important to demonstrate &quot;real-world&quot; usefulness of algorithms, but they may often give less understanding / insight into an algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Theoretical work:&lt;/strong&gt; Theoretical work can also give more insight and understanding of new algorithms. Algorithms with strong theoretical foundations are often more likely to generalize to a multitude of problem domains, assuming that the initial assumptions hold (big assumption here - there are plenty of cases where assumptions required for strong proofs do not hold!). This is not always possible / &quot;needed&quot;, sometimes new research based purely on intuition and with relatively little theoretical foundations still turn out to work well (or theory is only developed after promising empirical results)... but it can certainly help. Theoretical work can take many different forms, such proofs of convergence (often under strict conditions), proofs for upper or lower bounds on important measures (such as regret, or probability of making a &quot;wrong&quot; choice, etc.), proofs that an algorithm or a problem is a more general or more specific case of an existing, well-understood algorithm or problem, proofs that a model has or does not have a certain representational capacity, proofs of algorithmic equivalence (that an algorithm computes exactly the same quantities as another well-understood algorithm, typically with lower computation and/or memory requirements), etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-15T20:41:27.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10012" PostTypeId="2" ParentId="10010" CreationDate="2019-01-16T01:18:08.383" Score="0" Body="&lt;p&gt;The article referenced doesn't highlight the most common terms, which is why you may not be finding what you want. We've found that not all the designs have public implementations in every language. Some we've had to translate from Python or Java to C++ to get the speed we needed for embedded applications. Attention mechanisms are not one thing either. There are attention mechanisms in the cells, before the layers, after the layers, as oversight mechanisms, and in push-pull balancing arrangements. The common cell using attention ideas use gates in the cell and the article only briefly mentions it. That's the long-short term memory cell, which is a type of RNN. Look for LSTM and B-LSTM for the first examples, which are gated. The GRU is also a type of RNN cell with gating. This paper is recent and proposes two mechanisms: &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/viewFile/16126/16099&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/viewFile/16126/16099&lt;/a&gt;. If you search for DiSAN there are a few implementations. The newest kind, which has promise is not really an RNN strictly. That's the recursive network, which can have attention mechanisms but can build learned graph structures instead of tensors. That may also be what you want.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-16T01:18:08.383" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10013" PostTypeId="1" AcceptedAnswerId="10088" CreationDate="2019-01-16T02:44:26.203" Score="3" ViewCount="1024" Body="&lt;p&gt;What is &quot;bad local minima&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following papers all mention this expression.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.03909&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.00279&quot; rel=&quot;nofollow noreferrer&quot;&gt;limination of All Bad Local Minima in Deep Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1805.08671&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adding One Neuron Can Eliminate All Bad Local Minima&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18443" LastEditorUserId="2444" LastEditDate="2019-02-20T16:20:04.513" LastActivityDate="2019-02-20T16:20:04.513" Title="What is a bad local minimum in machine learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;calculus&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10014" PostTypeId="2" ParentId="10003" CreationDate="2019-01-16T04:48:02.030" Score="3" Body="&lt;p&gt;There are methods called &quot;scoring systems&quot; where you give a image scores such as &quot;0.9 stripes, 0.0 red, 0.8 hair, ...&quot; and use those scores to classify objects. It's an older idea, not used to determine if the network is learning. It's not in a standard CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine if relevant information is being learned or not, it's standard to use the testing accuracy, training accuracy, confusion matrix, or AUC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining what exactly a CNN is learning is a complicated research problem that's ongoing. In short - you can't really know. For a basic network, you can tell that it is learning &lt;em&gt;something&lt;/em&gt; but not what it's actually using to make determinations. &lt;/p&gt;&#xA;" OwnerUserId="21471" LastEditorUserId="21471" LastEditDate="2019-02-04T19:08:11.777" LastActivityDate="2019-02-04T19:08:11.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10017" PostTypeId="2" ParentId="10013" CreationDate="2019-01-16T07:46:43.317" Score="0" Body="&lt;p&gt;As mentioned in the abstract of on of these papers, bad local minima is a suboptimal local minimum which means a local minimum that is near to a global minimum.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-01-16T07:46:43.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10019" PostTypeId="1" CreationDate="2019-01-16T09:20:20.377" Score="2" ViewCount="318" Body="&lt;p&gt;I have this problem where I need to get information out of PDF document sent from a scanner. The program needs to be learnable in some way to recognize what different figures mean. Most of this should happen without human interference so it could just give a result after scanning the file.&#xA;Do anyone know if it's possible to do with a machine learning program or any alternative way?&lt;/p&gt;&#xA;" OwnerUserId="21476" LastActivityDate="2019-01-16T10:36:39.920" Title="Could it be possible to detect text, symbols, and components directly in a scanned PDF file with a program like Tensorflow or another program?" Tags="&lt;problem-solving&gt;&lt;computer-programming&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10020" PostTypeId="2" ParentId="10019" CreationDate="2019-01-16T10:36:39.920" Score="2" Body="&lt;p&gt;Yes, that's possible. &#xA;I am working on a project in which I have to detect text in images. I did a quick search and found these two algorithms: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. EAST:&lt;/strong&gt; (Efficient and Accurate Scene Text Detector)&lt;br&gt;&#xA;I am not sure if it is based on Machine Learning. Here are some links &lt;a href=&quot;https://www.pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link1&lt;/a&gt;  &lt;a href=&quot;https://www.pyimagesearch.com/2018/08/20/opencv-text-detection-east-text-detector/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link2&lt;/a&gt; explaining how to use it with an example and using tesseract to extract the detected text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. CTPN:&lt;/strong&gt; (Connectionist Text Proposal Network)&lt;br&gt;&#xA;This algorithm is based on Machine Learning. Here is its &lt;a href=&quot;https://github.com/eragonruan/text-detection-ctpn&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; in github. In the description, you will find a link to a pre-trained model that you can use. Or simply, you can prepare your own data and train your own model.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, I tried both of them, and the CTPN model gave better results especially when the image contains large text.&lt;/p&gt;&#xA;" OwnerUserId="19059" LastActivityDate="2019-01-16T10:36:39.920" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10021" PostTypeId="1" CreationDate="2019-01-16T11:28:17.310" Score="1" ViewCount="57" Body="&lt;p&gt;For example, I have the following csv: &lt;a href=&quot;https://gist.github.com/JafferWilson/3ab8ee88f3fc32e78579a1054aac757d&quot; rel=&quot;nofollow noreferrer&quot;&gt;training.csv&lt;/a&gt;&lt;br&gt;&#xA;I want to know how I can determine which column will be the best feature for getting the output prediction before I go for machine training.&lt;br&gt;&#xA;Please do share your responses&lt;/p&gt;&#xA;" OwnerUserId="9126" LastActivityDate="2019-01-16T14:29:13.760" Title="How to analyze data before going for machine learning training?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;problem-solving&gt;&lt;feature-selection&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10022" PostTypeId="2" ParentId="10021" CreationDate="2019-01-16T11:41:48.570" Score="0" Body="&lt;p&gt;You should know your data 100%. That means knowing what each of your columns and rows represents (e.g. temperature column, humidity, rows representing days), the value units (e.g. Celsius or Fahrenheit?), accuracy, value format (strings or numbers). You may need to clean and reorganize the data if necessary to bring them to your desired form (e.g. change the structure, units, aggregating, etc). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then use your logic and experience to decide what columns are necessary. This is in general. I hope someone will give you a more specific answer.&lt;/p&gt;&#xA;" OwnerUserId="21480" LastActivityDate="2019-01-16T11:41:48.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10025" PostTypeId="1" AcceptedAnswerId="10029" CreationDate="2019-01-16T13:36:55.310" Score="2" ViewCount="89" Body="&lt;p&gt;I have a Deep Feedforward Neural Network &lt;span class=&quot;math-container&quot;&gt;$F: W \times \mathbb{R}^d \rightarrow \mathbb{R}^k$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is the space of the weights) with &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; hidden layers, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; neurones per layer and ReLu activation. The output layer has a softmax activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can consider two different loss functions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L_1 = \frac{1}{2} \sum_i || F(W,x_i) - y||^2$&lt;/span&gt;  &lt;span class=&quot;math-container&quot;&gt;$&#xA; \ \ \ $&lt;/span&gt; and   &lt;span class=&quot;math-container&quot;&gt;$\ \ \ L_2  = -\sum_i log(F(w,x_i)_{y_i})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the first one is the classic quadratic loss and the second one is cross entropy loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to study the norm of the derivative of the loss function and see how the two are related, which means: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Let's assume I know that &lt;span class=&quot;math-container&quot;&gt;$|| \frac{\partial L_2(W, x_i)}{\partial W}|| &amp;gt; r$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is a small constant. What can I assume about &lt;span class=&quot;math-container&quot;&gt;$|| \frac{\partial L_1(W, x_i)}{\partial W}||$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Are there any result which tell you that, under some hypothesis (even strict ones) such as a specific random initialisation,  &lt;span class=&quot;math-container&quot;&gt;$|| \frac{\partial L_1(W, x_i)}{\partial W}||$&lt;/span&gt; doesn't go to zero   during training?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="21338" LastEditorUserId="21338" LastEditDate="2019-01-16T14:13:31.057" LastActivityDate="2019-01-17T04:00:36.763" Title="Comparing and studying Loss Functions" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10026" PostTypeId="2" ParentId="10021" CreationDate="2019-01-16T14:29:13.760" Score="2" Body="&lt;p&gt;Though there is no universal method which can be blindly used for all datasets, but here is what i usually do;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fill missing values using interpolation or mean, if missing values&#xA;are less than 10-15 percent of number of rows else drop the column.&lt;/li&gt;&#xA;&lt;li&gt;Encode categorical data using some kind of encoding, e.g. one hot, etc.&lt;/li&gt;&#xA;&lt;li&gt;Then normalize/rescale columns.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now look at the variance in each feature. Usually, features with more variance are more important.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Next, see the correlation among columns. If two columns are highly&#xA;correlated, you only need to keep only one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19722" LastActivityDate="2019-01-16T14:29:13.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10027" PostTypeId="1" AcceptedAnswerId="10030" CreationDate="2019-01-16T14:33:31.617" Score="4" ViewCount="242" Body="&lt;p&gt;There are many people trying to show how neural networks are still very different from humans, but I fail to see in what way human brains are different from neural models in anything but complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way we learn is similar, the way we process information is similar, the ways we predict outcomes and generate outputs are similar. Give a model enough processing power, enough training samples, and enough time and you can train a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what is the difference between human (brains) and neural networks?&lt;/p&gt;&#xA;" OwnerUserId="20399" LastEditorUserId="2444" LastEditDate="2020-05-17T11:28:57.020" LastActivityDate="2020-05-17T11:28:57.020" Title="What is the difference between human brains and neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;neuroscience&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ClosedDate="2021-01-20T01:16:06.830" ContentLicense="CC BY-SA 4.0" />
  <row Id="10029" PostTypeId="2" ParentId="10025" CreationDate="2019-01-16T15:49:58.077" Score="0" Body="&lt;p&gt;Let's first express a network of arbitrary topology and heterogeneous or homogeneous cell type arrangements as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ N(T, H, s) := \, \big[\, \mathcal{Y} = F(P_s, \, \mathcal{X}) \,\big] \\&#xA;   s \in \mathbb{C} \; \land \; s \le S \; \text{,} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the number of learning states or rounds, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the network of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; topology and &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; hyper-parameter structure and values that at stage &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; produces a &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; parameterized function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; resulting in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt;. In supervised learning, the goal is that &lt;span class=&quot;math-container&quot;&gt;$F(P_s)$&lt;/span&gt; approaches a conceptually ideal function &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$s \rightarrow S$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The popular loss aggregation norms are not quite as the question defines them. The below more canonically expresses the level 1 and 2 norms, which systematically aggregate multidimensional disparity between an intermediate result at some stage (epoch and example index) of training and the conceptual ideal toward which the network in training is intended to converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ {||F-\mathcal{Y}||}_1 = \sum{|F_i - y_i|} \\&#xA;   {||F-\mathcal{Y}||}_2 = \sqrt{\sum{(F_i - y_i)}^2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These equations have been mutated by various authors to make various points, but those mutations have obscured the obviousness of their original relationship. The first is where distance can be aggregated through only orthogonal vector displacements. The second is where aggregation uses the minimum Cartesian distance by extending the Pythagorean theorem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;em&gt;quadratic loss&lt;/em&gt; is a term with some ambiguity. These are all broadly describable as quadratic expressions of loss.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Distance as described in the second expression above&lt;/li&gt;&#xA;&lt;li&gt;RMS where the sum is divided by the number of dimensions&#xA;&lt;span class=&quot;math-container&quot;&gt;$\text{count}(i)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Just the sum of squared difference by itself&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cross entropy is an extension of Claude Shannon's information theory concepts based on the work of Bohr, Boltzmann, Gibbs, Maxwell, von Neumann, Frisch, Fermi, and others who were interested in quanta and the thermodynamic concept of entropy as a universal principle running through mater, energy, and knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ S = k_B \log{\Omega} \\&#xA;   H(X) = - \sum_i p(x_i) \, \log_2{\, p(x_i)} \\&#xA;   H(p, \, q) = -\sum_{x \in \mathcal{X}} \, p(x) \, \log_2{\, q(x)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this progression of theory, we begin with a fundamental postulate in quantum physics, where &lt;span class=&quot;math-container&quot;&gt;$k_B$&lt;/span&gt; is Boltzmann's constant and &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; are the number of microstates for the quanta. The next relation is Shannon's adaptation for information, where &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is the entropy in bits, thus the &lt;span class=&quot;math-container&quot;&gt;$\log_2$&lt;/span&gt; instead of a natural logarithm. The third relation above expresses cross-entropy in bits for features &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is based on the Kullback-Leibler divergence.  the p-attenuated sum of bits of q-information in .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; are probabilities, not &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; values, so one cannot substitute labels and outputs of a network into them and retain the meaning of cross entropy. Therefore level 1 and 2 norms are closely related, but cross-entropy is not a norm; it is the dispersion of one thing Cartesian distance aggregation scheme like them. Cross-entropy is remotely related but is statistically more sophisticated. To produce a cross-entropy loss function of form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ {||F-\mathcal{Y}||}_H = \mathcal{P}(F, y) \; \text{,} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one must derive the probabilistic function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{P}$&lt;/span&gt; that represents the cross entropy for two distributions in some way that is theoretically sound on the basis of both information theory and convergence resource conservation. It is not clear that the interpretation of cross entropy in the context of gradient descent and back propagation has caught up with the concepts of entropy in quantum theory. That's an area needing further research and deeper theoretical consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the question, the cross-entropy expression is not properly characterized, most evident in the fact that the expression is independent of the labels &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt;, which would be fine if for unsupervised learning except that no other basis for evaluation is represented in the expression. For the term cross-entropy to be valid, the basis for evaluation must include two distributions, a target one and one that represents the current state of learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivatives of the three norms (assuming the cross entropy is properly characterized) can be studied for the case of &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; ReLU layers by generalizing the chain rule (from differential calculus) as applied to ReLU and the loss function developed by applying each of the three norms to aggregate measures of disparity from optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the inference in sub-question (1) nothing of particular value can be assumed about the Jacobians of level 2 norms from level 1 norms, both with respect to parameters &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; or vice versa, except the retention of sign. This is because we cannot determine much about the correlation between the output channels of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no doubt, regarding sub-question (2), that some constraint, set of constraints, stochastic distribution applied to initialization, hyper-parameter settings, or data set features, labels, or number of examples have implications for the reliability and accuracy of convergence. The PAC (probably approximately correct) learning framework is one system of theory that approaches this question with mathematical rigor. One of its practical uses, among others, is to derive inequalities that predict feasibility in some cases and produce more lucid approaches to learning system projects.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-17T04:00:36.763" LastActivityDate="2019-01-17T04:00:36.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10030" PostTypeId="2" ParentId="10027" CreationDate="2019-01-16T16:04:23.053" Score="3" Body="&lt;p&gt;One incredibly important difference between humans and NNs is that the human brain is the result of billions of years of evolution whereas NNs were partially inspired by looking at the result and thinking &quot;... we could do that&quot; (utmost respect for &lt;a href=&quot;https://www.youtube.com/watch?v=IOHayh06LJ4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hubel and Wiesel&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human brains (and in fact anything biological really) have an embedded structure to them within the DNA of the animal. &lt;a href=&quot;https://stackoverflow.com/questions/8954571/how-much-memory-would-be-required-to-store-human-dna&quot;&gt;DNA has about 4 MB of data&lt;/a&gt; and incredibly contains the information of where arms go, where to put sensors and in what density, how to initialize neural structures, the chemical balances that drive neural activation, memory architecture, and learning mechanisms among many many other things. This is phenomenal. Note, the placement of neurons and their connections isn't encoded in dna, rather the rules dictating how these connections form is. This is fundamentally different from simply saying &quot;there are 3 conv layers then 2 fully connected layers...&quot;. &#xA;There has been some progress at &lt;a href=&quot;https://www.nature.com/articles/s42256-018-0006-z#ref-CR53&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural evolution&lt;/a&gt; that I highly recommend checking out which is promising though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another important difference is that during &quot;runtime&quot; (lol), human brains (and other biological neural nets) have a multitude of functions beyond the neurons. Things like Glial cells. There are about 3.7 Glial cells for every neuron in your body. They are a supportive cell in the central nervous system that surround neurons and provide support for and insulation between them and trim dead neurons. This maintenance is continuous update for neural structures and allows resources to be utilized most effectively. With fMRIs, neurologists are only beginning to understand the how these small changes affect brains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't to say that its impossible to have an artificial NN that can have the same high level capabilities as a human. Its just that there is a lot that is missing from our current models. Its like we are trying to replicate the sun with a campfire but heck, they are both warm.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-01-16T16:04:23.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10032" PostTypeId="1" AcceptedAnswerId="10033" CreationDate="2019-01-16T17:03:45.383" Score="1" ViewCount="526" Body="&lt;p&gt;I tried to build a Q-learning agent which you can play tic tac toe against after training.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, the agent performs pretty poorly. He tries to win but does not try to make me 'not winning' which ends up in me beating up the agent no matter how many loops I gave him for training. I added a reward of 1 for winning the episode and it gets a reward of -0.1 when he tries to put his label on an non-empty square (after the attempt we have s = s'). I also start with an epsilon=1 which decreases in every loop to add some more randomness at the beginning because I witnessed that some (important in my opinion) states did not get updated. Since I spend some hours of debugging without noticeable progress I'd like to know what you think.&lt;/p&gt;&#xA;&lt;p&gt;PS: Don't care about some print statements and count variables. Those where for debugging.&lt;/p&gt;&#xA;&lt;p&gt;Code here or on &lt;a href=&quot;https://github.com/DataHamster/q-learning-tic-tac-toe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import collections&#xA;import time&#xA;&#xA;Gamma = 0.9&#xA;Alpha = 0.2&#xA;&#xA;&#xA;class Environment:&#xA;    def __init__(self):&#xA;        self.board = np.zeros((3, 3))&#xA;        self.x = -1  # player with an x&#xA;        self.o = 1  # player with an o&#xA;        self.winner = None&#xA;        self.ended = False&#xA;        self.actions = {0: (0, 0), 1: (0, 1), 2: (0, 2), 3: (1, 0), 4: (1, 1),&#xA;                        5: (1, 2), 6: (2, 0), 7: (2, 1), 8: (2, 2)}&#xA;&#xA;    def reset_env(self):&#xA;        self.board = np.zeros((3, 3))&#xA;        self.winner = None&#xA;        self.ended = False&#xA;&#xA;    def reward(self, sym):&#xA;        if not self.game_over():&#xA;            return 0&#xA;        if self.winner == sym:&#xA;            return 10&#xA;        else:&#xA;            return 0&#xA;&#xA;    def get_state(self,):&#xA;        k = 0&#xA;        h = 0&#xA;        for i in range(3):&#xA;            for j in range(3):&#xA;                if self.board[i, j] == 0:&#xA;                    v = 0&#xA;                elif self.board[i, j] == self.x:&#xA;                    v = 1&#xA;                elif self.board[i, j] == self.o:&#xA;                    v = 2&#xA;                h += (3**k) * v&#xA;                k += 1&#xA;        return h&#xA;&#xA;        def random_action(self):&#xA;            return np.random.choice(self.actions.keys())&#xA;&#xA;    def make_move(self, player, action):&#xA;        i, j = self.actions[action]&#xA;        if self.board[i, j] == 0:&#xA;            self.board[i, j] = player&#xA;&#xA;    def game_over(self, force_recalculate=False):&#xA;        # returns true if game over (a player has won or it's a draw)&#xA;        # otherwise returns false&#xA;        # also sets 'winner' instance variable and 'ended' instance variable&#xA;        if not force_recalculate and self.ended:&#xA;            return self.ended&#xA;&#xA;        # check rows&#xA;        for i in range(3):&#xA;            for player in (self.x, self.o):&#xA;                if self.board[i].sum() == player*3:&#xA;                    self.winner = player&#xA;                    self.ended = True&#xA;                    return True&#xA;&#xA;        # check columns&#xA;        for j in range(3):&#xA;            for player in (self.x, self.o):&#xA;                if self.board[:, j].sum() == player*3:&#xA;                    self.winner = player&#xA;                    self.ended = True&#xA;                    return True&#xA;&#xA;        # check diagonals&#xA;        for player in (self.x, self.o):&#xA;            # top-left -&amp;gt; bottom-right diagonal&#xA;            if self.board.trace() == player*3:&#xA;                self.winner = player&#xA;                self.ended = True&#xA;                return True&#xA;            # top-right -&amp;gt; bottom-left diagonal&#xA;            if np.fliplr(self.board).trace() == player*3:&#xA;                self.winner = player&#xA;                self.ended = True&#xA;                return True&#xA;&#xA;        # check if draw&#xA;        if np.all((self.board == 0) == False):&#xA;            # winner stays None&#xA;            self.winner = None&#xA;            self.ended = True&#xA;            return True&#xA;&#xA;        # game is not over&#xA;        self.winner = None&#xA;        return False&#xA;&#xA;    def draw_board(self):&#xA;        for i in range(3):&#xA;            print(&amp;quot;-------------&amp;quot;)&#xA;            for j in range(3):&#xA;                print(&amp;quot;  &amp;quot;, end=&amp;quot;&amp;quot;)&#xA;                if self.board[i, j] == self.x:&#xA;                    print(&amp;quot;x &amp;quot;, end=&amp;quot;&amp;quot;)&#xA;                elif self.board[i, j] == self.o:&#xA;                    print(&amp;quot;o &amp;quot;, end=&amp;quot;&amp;quot;)&#xA;                else:&#xA;                    print(&amp;quot;  &amp;quot;, end=&amp;quot;&amp;quot;)&#xA;            print(&amp;quot;&amp;quot;)&#xA;        print(&amp;quot;-------------&amp;quot;)&#xA;&#xA;&#xA;&#xA;&#xA;class Agent:&#xA;    def __init__(self, Environment, sym):&#xA;        self.q_table = collections.defaultdict(float)&#xA;        self.env = Environment&#xA;        self.epsylon = 1.0&#xA;        self.sym = sym&#xA;        self.ai = True&#xA;&#xA;    def best_value_and_action(self, state):&#xA;        best_val, best_act = None, None&#xA;        for action in self.env.actions.keys():&#xA;            action_value = self.q_table[(state, action)]&#xA;            if best_val is None or best_val &amp;lt; action_value:&#xA;                best_val = action_value&#xA;                best_act = action&#xA;        return best_val, best_act&#xA;&#xA;    def value_update(self, s, a, r, next_s):&#xA;        best_v, _ = self.best_value_and_action(next_s)&#xA;        new_val = r + Gamma * best_v&#xA;        old_val = self.q_table[(s, a)]&#xA;        self.q_table[(s, a)] = old_val * (1-Alpha) + new_val * Alpha&#xA;&#xA;    def play_step(self, state, random=True):&#xA;        if random == False:&#xA;            epsylon = 0&#xA;        cap = np.random.rand()&#xA;        if cap &amp;gt; self.epsylon:&#xA;            _, action = self.best_value_and_action(state)&#xA;        else:&#xA;            action = np.random.choice(list(self.env.actions.keys()))&#xA;            self.epsylon *= 0.99998&#xA;        self.env.make_move(self.sym, action)&#xA;        new_state = self.env.get_state()&#xA;        if new_state == state and not self.env.ended:&#xA;            reward = -5&#xA;        else:&#xA;            reward = self.env.reward(self.sym)&#xA;        self.value_update(state, action, reward, new_state)&#xA;&#xA;&#xA;class Human:&#xA;    def __init__(self, env, sym):&#xA;        self.sym = sym&#xA;        self.env = env&#xA;        self.ai = False&#xA;&#xA;    def play_step(self):&#xA;        while True:&#xA;            move = int(input('enter position like: \n0|1|2\n------\n3|4|5\n------\n6|7|8'))&#xA;            if move in list(self.env.actions.keys()):&#xA;                break&#xA;        self.env.make_move(self.sym, move)&#xA;&#xA;&#xA;&#xA;def main():&#xA;    env = Environment()&#xA;    p1 = Agent(env, env.x)&#xA;    p2 = Agent(env, env.o)&#xA;    draw = 1&#xA;    for t in range(1000005):&#xA;&#xA;        current_player = None&#xA;        episode_length = 0&#xA;        while not env.game_over():&#xA;            # alternate between players&#xA;            # p1 always starts first&#xA;            if current_player == p1:&#xA;                current_player = p2&#xA;            else:&#xA;                current_player = p1&#xA;&#xA;            # current player makes a move&#xA;            current_player.play_step(env.get_state())&#xA;&#xA;        env.reset_env()&#xA;&#xA;        if t % 1000 == 0:&#xA;            print(t)&#xA;            print(p1.q_table[(0, 0)])&#xA;            print(p1.q_table[(0, 1)])&#xA;            print(p1.q_table[(0, 2)])&#xA;            print(p1.q_table[(0, 3)])&#xA;            print(p1.q_table[(0, 4)])&#xA;            print(p1.q_table[(0, 5)])&#xA;            print(p1.q_table[(0, 6)])&#xA;            print(p1.q_table[(0, 7)])&#xA;            print(p1.q_table[(0, 8)])&#xA;            print(p1.epsylon)&#xA;&#xA;    env.reset_env()&#xA;    # p1.sym = env.x&#xA;&#xA;    while True:&#xA;        while True:&#xA;            first_move = input(&amp;quot;Do you want to make the first move? y/n :&amp;quot;)&#xA;            if first_move.lower() == 'y':&#xA;                first_player = Human(env, env.x)&#xA;                second_player = p2&#xA;                break&#xA;            else:&#xA;                first_player = p1&#xA;                second_player = Human(env, env.o)&#xA;                break&#xA;        current_player = None&#xA;&#xA;        while not env.game_over():&#xA;            # alternate between players&#xA;            # p1 always starts first&#xA;            if current_player == first_player:&#xA;                current_player = second_player&#xA;            else:&#xA;                current_player = first_player&#xA;            # draw the board before the user who wants to see it makes a move&#xA;&#xA;            if current_player.ai == True:&#xA;                current_player.play_step(env.get_state(), random=False)&#xA;            if current_player.ai == False:&#xA;                current_player.play_step()&#xA;            env.draw_board()&#xA;        env.draw_board()&#xA;        play_again = input('Play again? y/n: ')&#xA;        env.reset_env()&#xA;        # if play_again.lower != 'y':&#xA;        #     break&#xA;&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21487" LastEditorUserId="2444" LastEditDate="2020-10-31T17:20:20.523" LastActivityDate="2020-10-31T17:20:20.523" Title="Why isn't my Q-Learning agent able to play tic-tac-toe?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;&lt;tic-tac-toe&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10033" PostTypeId="2" ParentId="10032" CreationDate="2019-01-16T18:10:04.403" Score="2" Body="&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning rule that you have implemented updates &lt;span class=&quot;math-container&quot;&gt;$Q(S_t, A_t)$&lt;/span&gt; estimates as follows, after executing an action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; in a state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;, observing a reward &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;, and reaching a state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; as a result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(S_t, A_t) \gets (1 - \alpha) Q(S_t, A_t) + \alpha (R_t + \gamma \max_a Q(S_{t+1}, a))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation seems to be correct for the traditional setting for which &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is normally described; &lt;strong&gt;single-agent MDPs&lt;/strong&gt;. The problem is that you have a multi-agent setting, in which &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is not always directly applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as far as I can see from a very quick glance at your code, it seems like you actually already have taken some important steps towards allowing it to work, and I think it should be quite close to almost working (at least for a simple game like Tic-Tac-Toe) already. Important things that you appear to already be doing correctly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Self-play training against an opponent who is hopefully gradually improving, as opposed to training against a uniform-at-random agent or a fixed-strategy agent.&lt;/li&gt;&#xA;&lt;li&gt;Add randomization during training to ensure sufficient diversity in generated experience.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I think the &lt;strong&gt;major issue that remains to be solved&lt;/strong&gt; is in &lt;strong&gt;how you define the subsequent state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; after making a move in a state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The update target that the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update rule moves its &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value estimates towards consists of two components:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The observed immediate reward &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The discounted predicted future returns &lt;span class=&quot;math-container&quot;&gt;$\gamma \max_a Q(S_{t+1}, a)$&lt;/span&gt; for the greedy policy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The problem is that, in your implementation, &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; &lt;strong&gt;is a state in which the opponent is allowed to make the next move &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, rather than the RL agent&lt;/strong&gt;. This means that &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(S_{t+1}, a)$&lt;/span&gt; is an incredibly optimistic, naive, unrealistic estimate of future returns. In fact, &lt;span class=&quot;math-container&quot;&gt;$\min_a Q(S_{t+1}, a)$&lt;/span&gt; would be a much more realistic estimate (against an optimally-playing opponent), because the opponent gets to pick the next action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think switching in &lt;span class=&quot;math-container&quot;&gt;$\min_a Q(S_{t+1}, a)$&lt;/span&gt; rather than the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; may have a good chance of working in this scenario, but I'm not 100% sure. It wouldn't be a &quot;pretty&quot; solution though, since you'd no longer be doing &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, but something else altogether.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proper &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update may work well if you only present states to agents in which they're actually allowed to make the next move in the update rule. Essentially, you'd be plugging &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(S_{t + 2}, a)$&lt;/span&gt; into the update rule, replacing &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$S_{t+2}$&lt;/span&gt;. Well... that's what you'd be doing in most cases. The only exception to be aware of would be terminal states. If an agent makes a move that leads to a terminal state, you should make sure to also run an additional update for that agent with the terminal game state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1}, a)$&lt;/span&gt; will always be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for any action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; is terminal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a very closely related question, where I essentially provided an answer in the same spirit, see: &lt;a href=&quot;https://ai.stackexchange.com/q/6573/1641&quot;&gt;How to see terminal reward in self-play reinforcement learning?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-01-16T18:18:57.233" LastActivityDate="2019-01-16T18:18:57.233" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10034" PostTypeId="1" CreationDate="2019-01-16T18:14:43.297" Score="1" ViewCount="50" Body="&lt;p&gt;I'm a programmer with a background in mathematics, but I have no experience whatsoever with artificial intelligence/neural networks. I'd like to study it as a hobby, and my goal for now is to solve the following simple poker game, by letting the program play against itself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have two players, each with a certain number of chips. At the start of the game, they are obligated to put a certain amount of chips in the pot. Then they each get a random real number between 0 and 10. They know their own number, but not the one of their opponent. Then we have one round of betting. The first player puts additional chips in the pot (some number between 0 and their stack size). The second player can either fold (put no additonal chips in the pot, 1st player gets the entire pot), call (put the same number of chips in the pot, player with highest number gets the pot) or raise (put even more chips in the pot, action back on 1st player). There is no limit to the amount of times  a player can raise, as long as he still has chips behind to raise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have several questions:&#xA;- Is this indeed a problem that can be solved with neural networks?&#xA;- What do you recommend me to study in order to solve this problem?&#xA;- Is it feasible to solve this game when allowing for continuous bet/raise sizes? Or should I limit it to a few options as a percentage of the pot?&#xA;- Do you expect it to be possible to get close to an equilibrium with one nightly run on an 'average' laptop?&lt;/p&gt;&#xA;" OwnerUserId="21488" LastEditorUserId="1671" LastEditDate="2019-01-16T21:27:54.057" LastActivityDate="2019-01-16T21:27:54.057" Title="What to study for this simple poker game?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;game-theory&gt;&lt;imperfect-information&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10036" PostTypeId="1" AcceptedAnswerId="10046" CreationDate="2019-01-17T01:04:36.763" Score="0" ViewCount="45" Body="&lt;p&gt;I have source data that can be represented as a 2D image of many similar curves. They may oftentimes cross over one another, so regions of interest will overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to implement a neural network solution to identify each instance or the curves and the pixels that are associated with each instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Each image is simple in its representation of the data. A pixel in the image is either a point on one of these curves or it is empty. So the image is represented by one or zero at each pixel. For training purposes, I have labels for every pixel, and I have about 150,000 images. The information in the images can be noisy in that there may be omissions of points and point locations are quantized due to measurement limitations and preprocessing for the image preparation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started looking into what semantic segmentation can do for me, but since all of the instances are of the same class, distinguished mainly by their location in the image, I don't think semantic segmentation is the type of processing I would want to perform. (Am I wrong?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very interested in seeing how a neural network will work on this problem to separate each instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: what is the terminology that describes the process I'm looking for? (How can I effectively research for this problem?) Is this an extension of semantic segmentation or is it referred to some other way?&lt;/p&gt;&#xA;" OwnerUserId="8439" LastActivityDate="2019-01-17T15:43:08.763" Title="Pixel-Level Detection of Each Object of the Same Class In an Image" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10037" PostTypeId="1" CreationDate="2019-01-17T05:29:56.447" Score="1" ViewCount="41" Body="&lt;p&gt;&lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/D.Silver/web/Teaching_files/pg.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;At slide 17&lt;/a&gt; of the David Silver's series, the soft-max policy is defined as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi_\theta(s, a) \propto e^{\phi(s, a)^T \theta}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that is, the probability of an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;) is proportional to the &quot;exponentiated weight&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The score function is then defined as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\theta \log \pi_\theta (s, a) = \phi(s, a) - &#xA;\mathbb{E}_{\pi_{\theta}}[\phi(s, \cdot)]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does the expectation term &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi_{\theta}}[\phi(s, \cdot)]$&lt;/span&gt; come from?&lt;/p&gt;&#xA;" OwnerUserId="16313" LastEditorUserId="2444" LastEditDate="2019-02-15T18:59:03.090" LastActivityDate="2019-02-15T18:59:03.090" Title="Where does the expectation term in the derivative of the soft-max policy come from?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;policies&gt;&lt;expectation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10039" PostTypeId="2" ParentId="9996" CreationDate="2019-01-17T07:37:33.843" Score="1" Body="&lt;p&gt;In general, algorithms that exploit distances or similarities (e.g. in the form of scalar product) between data samples, such as k-NN and SVM, are sensitive to feature transformations. We do feature scaling to make our model robust to outliers and make an initial impact of every feature on the model will be roughly similar&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Graphical-model based classifiers, such as Fisher LDA or Naive Bayes, as well as Decision trees and Tree-based ensemble methods (RF, XGB) are invariant to feature scaling, but, still, it might be a good idea to rescale/standardize your data.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You should explore your data more carefully, find the outliers, apply transformation if needed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Not sure if it is a good idea&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can apply different preprocessing techniques like MinMaxScaller, Rank,  Log transform, Extracting square root, StandartScaller and etc. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17925" LastEditorUserId="2444" LastEditDate="2019-06-28T11:24:08.433" LastActivityDate="2019-06-28T11:24:08.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10040" PostTypeId="1" CreationDate="2019-01-17T08:37:30.307" Score="0" ViewCount="71" Body="&lt;p&gt;I need to evaluate the monetary potential of AI at my company for the year 2025 to justify a long term investment in a dedicated AI center of competence. The departments in which AI would be used later on (Purchasing, R&amp;amp;D, Manufacturing, …) are generally very interested and some of them have already built some prototype use-cases. However, they are unwilling to predict and quantify any future potential. The investments in AI can only be justified if we can state the monetary benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any creative approaches out there to estimate the monetary potential of AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated! &lt;/p&gt;&#xA;" OwnerUserId="21502" LastEditorUserId="1671" LastEditDate="2019-06-26T17:05:13.713" LastActivityDate="2019-07-26T18:01:30.127" Title="Looking for approach for monetary evaluation of AI potential in 2025" Tags="&lt;research&gt;&lt;business&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10043" PostTypeId="1" CreationDate="2019-01-17T10:13:05.493" Score="5" ViewCount="223" Body="&lt;p&gt;Is doing a master's in natural language processing combined with machine learning worth it? What are the opportunities that can be opened by having such a master's?&lt;/p&gt;&#xA;" OwnerUserId="8844" LastEditorUserId="2444" LastEditDate="2020-03-14T18:33:31.423" LastActivityDate="2020-03-14T18:33:31.423" Title="Is doing a master's in natural language processing combined with machine learning worth it?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;academia&gt;" AnswerCount="4" CommentCount="0" ClosedDate="2020-03-14T18:32:41.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="10044" PostTypeId="1" CreationDate="2019-01-17T12:23:23.670" Score="2" ViewCount="66" Body="&lt;p&gt;How do I check which algorithm solves my problem best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a optimaization problem, I apply different well known optimization algorithms (genetic algorithm, simulated annealing, ant colony etc.) to solve my problem. However, how do I know if my implementation ( e.g. cost function) is working for every case? How can I compare the algorithms or their goodness in the context of my problem?&lt;/p&gt;&#xA;" OwnerUserId="19413" LastActivityDate="2019-01-24T20:36:54.703" Title="Method to check goodness of combinatorial optimization algorithm implementation" Tags="&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10045" PostTypeId="2" ParentId="6961" CreationDate="2019-01-17T12:43:05.633" Score="2" Body="&lt;p&gt;Structure of LSTM Networks&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[Are] units/neurons in the hidden layers ... referred to as memory blocks and each memory block can contain multiple memory cells?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Although an early guess at the way biological neurons work inspired the Perceptron design that later became multilayer (MLP) and then recurrent (RNN), these artificial network building blocks are not very much like biological neurons. The term is still used by some, but, because of this increased interdisciplinary awareness, the more common terms are cell, unit, or block.&lt;/p&gt;&#xA;&lt;p&gt;These last two terms can be a little ambiguous when entire networks are used as building blocks in larger AI architectures. This is not a show stopper and also occurs in other engineering disciplines where sub-circuit blocks are components in more complex blocks, and sub-assemblies can be parts of larger mechanical assemblies.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And one memory cell looks like these two?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The images in the questions aren't clear representations. This visualization of an LSTM cell&lt;sup&gt;1&lt;/sup&gt; shows the attenuation parameters, each labeled with a capital W.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AgviW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AgviW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;These LSTM cells are arranged as a layer and each pair of adjacent cells within the same layer are connected &lt;span class=&quot;math-container&quot;&gt;$(C_t, \, h_t) \rightarrow (C_{t+1}, \, h_{t+1})$&lt;/span&gt;. The input in this articular diagram is &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; and the output to the next layer is &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For units in the input and output layers, they are simply regular neurons, like those in a feed forward network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They can be, but there are many topological designs in use that use LSTM cells in some layers and Perceptrons in others, along with side-by-side networks, bi-directional designs, recursive topologies, and completely different cell types, such as the very popular GRU cell.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[Is] the structure of an RNN essentially like that of a feed-forward network, except the neurons in the hidden layer have been replaced by neurons with recurrent connections ...?&lt;/p&gt;&#xA;&lt;p&gt;LSTM RNNs then replaces those neurons in the hidden layers with memory blocks (which contain memory cells that have recurrent connections) ...?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No general rule can be assumed, especially since new designs emerge nearly every week, and significant topological advancements occur about once every one or two years. This is an example&lt;sup&gt;2&lt;/sup&gt; of a topology organized to support a specific processing use case.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/okj8X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/okj8X.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://kvitajakub.github.io/2016/04/14/rnn-diagrams/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Visualizations of RNN units — Diagrams of RNN unrolling, LSTM and GRU&lt;/em&gt;&lt;/a&gt;, by Jakub Kvita, 2016, and inspired by Oinkina&lt;/p&gt;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://www.google.com/url?sa=i&amp;amp;source=images&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwj244Pl6vTfAhUOOawKHXX8BsQQjhx6BAgBEAM&amp;amp;url=https%3A%2F%2Fwww.analyticsvidhya.com%2Fblog%2F2018%2F04%2Fsolving-an-image-captioning-task-using-deep-learning%2F&amp;amp;psig=AOvVaw3ZtKLK6JOLC2Pyr3c06qCo&amp;amp;ust=1547812389162348&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Automatic Image Captioning using Deep Learning (CNN and LSTM)&lt;/em&gt;&lt;/a&gt;, Faizan Shaikh, 2018&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-17T12:43:05.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10046" PostTypeId="2" ParentId="10036" CreationDate="2019-01-17T15:43:08.763" Score="1" Body="&lt;p&gt;What you want to do is instance segmentation on a pixel level.&#xA;I can point you two different way:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;MaskRCNN, which in a nutshell, uses Region Proposal Networks then segment in the proposed regions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some other works which don't use Region Proposals rather, regress the pixels of the image in an embeding space to be able to apply some clustering algorithm. Here is a link to a good blog post about it: &lt;a href=&quot;https://towardsdatascience.com/instance-embedding-instance-segmentation-without-proposals-31946a7c53e1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/instance-embedding-instance-segmentation-without-proposals-31946a7c53e1&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6019" LastActivityDate="2019-01-17T15:43:08.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10047" PostTypeId="2" ParentId="3161" CreationDate="2019-01-17T16:04:27.767" Score="-1" Body="&lt;p&gt;This is what I found about Artificial Intelligence application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A whole bunch predictions are there on the web. While surfing the internet, I found a list of 5 tech and industry predictions which says it all. The list includes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Natural Language Processing: In future, most of our machinery will learn from their experiences. Currently, Amazon’s Alexa device is a live example of an application of AI in voice recognition.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Driverless Cars: Again, when the AI develops more and more than even our car will become driverless. It will become so common that we will almost forget the term “car driver”.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Clean Energy: As you know the resources are limited and the air is becoming more and more polluted. So, the only solution to this problem is clean energy which will be more efficient than our current fuels.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Smart Money: When everything is going digital then why not money? The currency will also become digitized to save paper and eventually trees.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Information Security: Now, every information is available digitally. So, to reduce data and information breaches it must become that much secure that it will become almost impossible for someone to hack.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21514" LastEditorUserId="1671" LastEditDate="2019-01-17T17:47:27.380" LastActivityDate="2019-01-17T17:47:27.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10048" PostTypeId="1" AcceptedAnswerId="10069" CreationDate="2019-01-17T19:06:36.177" Score="2" ViewCount="223" Body="&lt;p&gt;Can NEAT produce neural networks where inputs are directly (without intermediate hidden neurons) connected to outputs?&lt;/p&gt;&#xA;" OwnerUserId="21517" LastEditorUserId="2444" LastEditDate="2019-10-13T02:09:26.787" LastActivityDate="2019-10-13T02:09:26.787" Title="Can NEAT produce neural networks where inputs are directly connected to outputs?" Tags="&lt;evolutionary-algorithms&gt;&lt;theory&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10049" PostTypeId="1" AcceptedAnswerId="10061" CreationDate="2019-01-17T19:27:47.763" Score="6" ViewCount="825" Body="&lt;p&gt;I've seen monte-carlo reward &lt;span class=&quot;math-container&quot;&gt;$G_{t}$&lt;/span&gt; used in REINFORCE and TD(&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) reward &lt;span class=&quot;math-container&quot;&gt;$r_t + \gamma Q(s', a')$&lt;/span&gt; used in vanilla actor-critic. I've never seen someone use lambda reward &lt;span class=&quot;math-container&quot;&gt;$G^{\lambda}_{t}$&lt;/span&gt; in these situations, nor in any other algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a specific reason for this? Could there be performance improvements if we used &lt;span class=&quot;math-container&quot;&gt;$G^{\lambda}_{t}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="21518" LastEditorUserId="2444" LastEditDate="2020-06-04T17:10:50.443" LastActivityDate="2020-08-11T16:59:36.413" Title="Why are lambda returns so rarely used in policy gradients?" Tags="&lt;reinforcement-learning&gt;&lt;reinforce&gt;&lt;return&gt;&lt;td-lambda&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10050" PostTypeId="1" CreationDate="2019-01-17T19:45:15.953" Score="6" ViewCount="123" Body="&lt;p&gt;Suppose there is an evaluation policy called &lt;span class=&quot;math-container&quot;&gt;$\pi_{e}$&lt;/span&gt; and there are two behavior policies &lt;span class=&quot;math-container&quot;&gt;$\pi_{b1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_{b2}$&lt;/span&gt;. I know that it is possible to estimate the return of policy &lt;span class=&quot;math-container&quot;&gt;$\pi_{e}$&lt;/span&gt; through behavior policies via &lt;em&gt;importance sampling&lt;/em&gt;, which is unbiased. But I do not know about the variance of return estimated through two behavior policies &lt;span class=&quot;math-container&quot;&gt;$\pi_{b1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_{b2}$&lt;/span&gt;. Does anybody know about the variance or any bound on the variance of estimated return?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$G_{0}^{b1}=\sum_{t=1}^{T}\gamma^{t-1}r_{t}^{b1}$&lt;/span&gt; represent the total return for an episode through behavior policy &lt;span class=&quot;math-container&quot;&gt;$\pi_{b1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G_{0}^{b2}=\sum_{t=1}^{T}\gamma^{t-1}r_{t}^{b2}$&lt;/span&gt; represent the total return for an episode through behavior policy &lt;span class=&quot;math-container&quot;&gt;$\pi_{b2}$&lt;/span&gt;.&#xA;It is possible to estimate the return of policy &lt;span class=&quot;math-container&quot;&gt;$\pi_{e}$&lt;/span&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{0}^{(e,b1)}=\prod_{t=1}^{T}\frac{\pi_{e}(a_{t}|s_{t})}{\pi_{b1}(a_{t}|s_{t})}*G_{0}^{b1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{0}^{(e,b2)}=\prod_{t=1}^{T}\frac{\pi_{e}(a_{t}|s_{t})}{\pi_{b2}(a_{t}|s_{t})}*G_{0}^{b2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compare the variance of &lt;span class=&quot;math-container&quot;&gt;$G_{0}^{(e,b1)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G_{0}^{(e,b2)}$&lt;/span&gt;. Is there any formulation to compute the variance &lt;span class=&quot;math-container&quot;&gt;$G_{0}^{(e,b1)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G_{0}^{(e,b2)}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2019-03-06T12:13:06.420" LastActivityDate="2020-03-30T16:02:30.347" Title="How do I compute the variance of the return of an evaluation policy using two behaviour policies?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10051" PostTypeId="1" CreationDate="2019-01-17T22:59:06.970" Score="3" ViewCount="365" Body="&lt;p&gt;Attention has been used widely in recurrent networks to weight feature representations learned by the model. This is not a trivial task since recurrent networks have a &lt;em&gt;hidden state&lt;/em&gt; that captures sequence information. The hidden state can be fed into a small MLP that produces a context vector summarizing the salient features of the hidden state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of NLP, convolutional networks are not as straightforward. They have the notion of &lt;em&gt;channels&lt;/em&gt; that are different feature representations of the input, but are channels the equivalent to hidden states? Particularly, this raises two questions for me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Why use attention in convolutional networks at all? Convolutions have shown to be adept feature detectors––for example, it is known that higher layers learn small features such as edges while lower layers learn more abstract representations. Would attention be used to sort through and weight these features?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In practice, how would attention be applied to convolutional networks? The output of these networks is usually &lt;code&gt;(batch, channels, input_size)&lt;/code&gt; (at least in PyTorch), so how would the attention operations in recurrent networks be applied to the output of convolutional networks?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1705.03122.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Convolutional Sequence to Sequence Learning&lt;/em&gt;&lt;/a&gt;, Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin, 2017&lt;/p&gt;&#xA;" OwnerUserId="19403" LastEditorUserId="4302" LastEditDate="2019-01-20T20:56:02.317" LastActivityDate="2019-10-26T01:02:15.323" Title="Additive Attention in Convolutional Networks" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10052" PostTypeId="2" ParentId="9969" CreationDate="2019-01-18T01:38:50.427" Score="1" Body="&lt;p&gt;If you only have a labeled set of emails, it is definitely not enough - you need to extract features from the text, like common words, existence of the title, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the ID3 algorithm you need to work with entropy,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H(x) = -(p_-log_2p_- + p_+log_2p_+)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And information gain,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$IG(x) = H(x) - \sum_{i=1}^{q}\dfrac{|\Delta y_i|}{|\Delta y|}H(y_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; - is a total number of groups after splitting.&#xA;&lt;span class=&quot;math-container&quot;&gt;$\Delta y_i$&lt;/span&gt; - the number of elements of the sample where feature X has &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pseudocode is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def build(L):&#xA;    Create node t&#xA;    If the stopping criterion is True:&#xA;        Assign a predictive model to t&#xA;    Else:&#xA;        Find the best binary split L = L_left + L_right using IG&#xA;        t.left = build(L_left)&#xA;        t.right = build(L_right)&#xA;    Return t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Instead of entropy you can use the Gini impurity or misclassification error.&lt;/p&gt;&#xA;" OwnerUserId="17925" LastEditorUserId="16229" LastEditDate="2019-05-31T20:59:49.300" LastActivityDate="2019-05-31T20:59:49.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10053" PostTypeId="1" CreationDate="2019-01-18T01:52:49.553" Score="2" ViewCount="216" Body="&lt;p&gt;For the purposes of this question I am asking about training the generator, assume that training the discriminator is another topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding of generative adversarial networks is that you feed random input data to the generator and it generates images. Out of those images, the ones which the discriminator thinks are real are used to train the generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have the random inputs &lt;span class=&quot;math-container&quot;&gt;$i_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i_2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i_3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i_4$&lt;/span&gt;... from which the generator produces &lt;span class=&quot;math-container&quot;&gt;$o_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$o_2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$o_3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$o_4$&lt;/span&gt;. Say for example, the discriminator thinks that &lt;span class=&quot;math-container&quot;&gt;$o_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$o_2$&lt;/span&gt; are real but &lt;span class=&quot;math-container&quot;&gt;$o_3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$o_4$&lt;/span&gt; are fake, I then throw away input output pairs 3 and 4, but keep 1 and 2, and run back propagation on the generator to tell it that &lt;span class=&quot;math-container&quot;&gt;$i_1$&lt;/span&gt; should produce &lt;span class=&quot;math-container&quot;&gt;$o_1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$i_2$&lt;/span&gt; should produce &lt;span class=&quot;math-container&quot;&gt;$o_2$&lt;/span&gt; since these were &quot;correct&quot; according to the discriminator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The contradiction seems to come from the fact that the generator &lt;em&gt;already&lt;/em&gt; generates those outputs from those inputs, so nothing will be gained by running backprop on those input output pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where is the flaw in my logic here? I seem to have something wrong in my reasoning, or a misunderstanding of how the generator is trained.&lt;/p&gt;&#xA;" OwnerUserId="21524" LastEditorUserId="16565" LastEditDate="2019-02-01T13:34:07.343" LastActivityDate="2021-02-20T17:07:35.800" Title="Training the generator in a GAN pair with back propagation" Tags="&lt;backpropagation&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10054" PostTypeId="2" ParentId="4282" CreationDate="2019-01-18T03:38:25.833" Score="2" Body="&lt;p&gt;I think this paper can help you out: &lt;a href=&quot;https://arxiv.org/pdf/1612.00496.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;3D Bounding Box Estimation Using Deep Learning and Geometry&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He used 1 VGG-19 (pretrained on ImageNet) to learn the size of cars&lt;/p&gt;&#xA;" OwnerUserId="21528" LastEditorUserId="1671" LastEditDate="2019-01-18T19:58:23.217" LastActivityDate="2019-01-18T19:58:23.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10056" PostTypeId="2" ParentId="10053" CreationDate="2019-01-18T05:13:04.210" Score="0" Body="&lt;p&gt;Let's take a look at the images that fooled the D-network. When this happens the binary cross entropy loss of the D-network is important to look at. The network said a fake image was real. As usual, you back propagate to the very beginning of the D-network. What is the back prop doing? It is telling the weights in the D-network to change such that you lessen the loss (remember you are minimizing the loss function by going down hill --&gt; using gradient descent). So you changed the weights of the D-network, but you really don't have to stop there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complex nonlinear function the D-network is calculating is really f(Image, weights of D-network). So the output of the D-network is really dependent on the Image fed in and the weights of the D-network. The cost is a function of the output f. So we have c = g(f(Image, weights of D-network)) where g is really the binary cross entropy. Looking at this, we see that the cost is also a function of the image!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can take the partials with respect to the image just like we take the partials with respect to the weights to update the weights. And so after doing back prop to the image, we don't stop -&gt; we take the partial of the cost function with respect to the image that is fed into the D-network and we update the image. And essentially we &lt;em&gt;don't have the original image&lt;/em&gt; when we continue to back prop to the beginning of the G-network. Let me know if you'd like more clarification. Again this is my understanding and I haven't coded a GAN yet, so someone please confirm. Also, I am open to others' interpretations. &lt;/p&gt;&#xA;" OwnerUserId="15428" LastActivityDate="2019-01-18T05:13:04.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10060" PostTypeId="1" CreationDate="2019-01-18T11:09:29.960" Score="1" ViewCount="67" Body="&lt;p&gt;I have a neural network &lt;span class=&quot;math-container&quot;&gt;$F(W,x): \mathbb{R}^d \rightarrow \mathbb{R}^k$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; layers, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; neurones per layer, ReLu activation, softmax on the last layer and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; datapoint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My loss function is the classic &lt;span class=&quot;math-container&quot;&gt;$L(W) = \frac{1}{2}\sum_{i \in [n]} || F(W,x_i) - y_i||^2$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that every weight matrix &lt;span class=&quot;math-container&quot;&gt;$W \in \mathbb{R}^{m \times m}$&lt;/span&gt;. These matrices are started with random initialisation &lt;span class=&quot;math-container&quot;&gt;$W_{i,j} = \mathcal{N}(0, \frac{2}{m})$&lt;/span&gt;. First of all, what's the Frobenius norm of this matrix? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it was symmetric, Wigner's theory would suggest us that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}||W||_F = 2\sqrt{m}*2/\sqrt{m} = 4$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure it is right, but it should. What I need is an answer to the following questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once I start training, how do the weights changes affect the norm of the weight matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the expected norm change after every iteration? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the expected norm of a well trained network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I have never specified if I need the &lt;span class=&quot;math-container&quot;&gt;$||*||_2$&lt;/span&gt; norm or the Frobenius one because I actually need both, so whatever answer with one of them is valid :D )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="21338" LastEditorUserId="21338" LastEditDate="2019-01-18T16:43:06.763" LastActivityDate="2019-01-18T16:43:06.763" Title="How does the norm of a weight matrix changes during training?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10061" PostTypeId="2" ParentId="10049" CreationDate="2019-01-18T14:29:37.537" Score="11" Body="&lt;p&gt;That can be done. For example, Chapter 13 of the 2nd edition of Sutton and Barto's Reinforcement Learning book (page 332) has pseudocode for &quot;Actor Critic with Eligibility Traces&quot;. It's using &lt;span class=&quot;math-container&quot;&gt;$G_t^{\lambda}$&lt;/span&gt; returns for the critic (value function estimator), but also for the actor's policy gradients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you do not &lt;em&gt;explicitly&lt;/em&gt; see the &lt;span class=&quot;math-container&quot;&gt;$G_t^{\lambda}$&lt;/span&gt; returns mentioned in the pseudocode. They are being used &lt;em&gt;implicitly&lt;/em&gt; through eligibility traces, which allow for an efficient online implementation (the &quot;backward view&quot;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I do indeed have the impression that such uses are fairly rare in recent research though. I haven't personally played around with policy gradient methods to tell from personal experience why that would be. My guess would be that it is because policy gradient methods are almost always combined with Deep Neural Networks, and &lt;strong&gt;variance&lt;/strong&gt; is already a big enough problem in training these things without starting to involve long-trajectory returns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use large &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns, you get low bias, but high variance. For &lt;span class=&quot;math-container&quot;&gt;$\lambda = 1$&lt;/span&gt;, you basically get REINFORCE again, which isn't really used much in practice, and has very high variance. For &lt;span class=&quot;math-container&quot;&gt;$\lambda = 0$&lt;/span&gt;, you just get one-step returns again. Higher values for &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; (such as &lt;span class=&quot;math-container&quot;&gt;$\lambda = 0.8$&lt;/span&gt;) tend to work very well in my experience with tabular methods or linear function approximation, but I suspect the variance may simply be too much when using DNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that it is quite popular to use &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns with a fixed, generally fairly small, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in Deep RL approaches. For instance, I believe the original A3C paper used &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;-step returns, and Rainbow uses &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;-step returns. These often work better in practice than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;-step returns, but still have reasonably low variance due to using small &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-18T14:29:37.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10063" PostTypeId="2" ParentId="10043" CreationDate="2019-01-18T14:38:01.877" Score="0" Body="&lt;p&gt;If you want to do NLP, using machine learning is a very, very good idea.  Our natural languages have structures full of messy inconsistencies (I must constantly apologize to my children for the vagaries of English orthography) which can only be recovered empirically from the data and encoded as highly complex heuristics (which is exactly what machine learning is for). You may have noticed that machine translation (whether on Facebook, Google or Twitter) is now really, really effective, and that voice recognition with Google Voice Search or Alex &lt;em&gt;actually works&lt;/em&gt;.  These applications were quite infeasible when we tried to hand-write code to solve these problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other uses for NLP-based machine learning? Absolutely!  Large organizations have databases chock-full of &lt;em&gt;free text fields&lt;/em&gt;.  These might be notes in medical records, transcriptions of customer service interactions, patent filings or a million or things.  Tradition database operations can't do much with this text beyond really low-quality search.  This is why companies like the giant Salesforce now have world-class research teams applying machine learning to NLP - because it is really the only way to enable machines to really get to grips with our messy human language.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-01-18T14:38:01.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10064" PostTypeId="1" CreationDate="2019-01-18T15:00:39.957" Score="0" ViewCount="183" Body="&lt;p&gt;I am working to build a reinforcement agent with DQN. The agent would be able to place buy and sell orders for a day trading purpose. I am facing a little problem with that project. The question is &quot;how to tell the agent to maximize the profit and avoid the transaction where the profit is less than 100$&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to maximize the profit inside a trading day and avoid to place the pair (limit buy order, limit sell order) if the profit on that transaction is less than 100$. The idea here is to avoid the little noisy movements. Instead, I prefer long beautiful profitable movements. Be aware that I thought using the &quot;Profit &amp;amp; Loss&quot; as the reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;I want the minimal profit per transaction to be 100$&quot; ==&gt; It seems this is not something that is enforceable. I can train the agent to maximize profit per transaction, but how that profit is cannot be ensured.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the beginning, I wanted to tell the agent, if the profit of a transaction is 50 dollars, I will remove 100 dollars, then It becomes a penalty of 50 dollars for the agent. I thought it was a great way to tell the agent to not place a limit buy order if you are not sure it will give us a minimal profit of 100$. It seems that all I would be doing there is simply shifting the value of the reward. The agent only cares about maximizing the sum of rewards and not taking care of individual transactions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to tell the agent to maximize the profit and avoid the transaction where the profit is less than 100$? With that strategy, what guarantee that the agent will never make a buy/sell decision that results in less than 100 dollars profit? Does the sum of reward - # transaction * 100 can be a solution?&lt;/p&gt;&#xA;" OwnerUserId="21539" LastEditorUserId="21539" LastEditDate="2019-01-19T21:29:29.493" LastActivityDate="2019-02-27T12:02:55.730" Title="For some reasons, a reward becomes a penalty if" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;rewards&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10065" PostTypeId="2" ParentId="10064" CreationDate="2019-01-18T17:01:44.750" Score="0" Body="&lt;p&gt;Such a guarantee cannot be made. The optimization of planing does not always result in the guarantee of an outcome, but rather the extend of improvements in position, which is necessarily probabilistic.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-20T12:43:03.773" LastActivityDate="2019-01-20T12:43:03.773" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10068" PostTypeId="2" ParentId="9479" CreationDate="2019-01-18T20:22:37.377" Score="-1" Body="&lt;p&gt;Simulated annealing algorithms are generally better at solving mazes, because they are less likely to get suck in a local minima because of their probabilistic &quot;mutation&quot; method. See &lt;a href=&quot;https://www.mathworks.com/help/gads/what-is-simulated-annealing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Genetic algorithms are better at training neural networks, because of their genetically inspired training algorithm. This makes them more versatile and efficient in more complex situations.&lt;/p&gt;&#xA;" OwnerUserId="4744" LastActivityDate="2019-01-18T20:22:37.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10069" PostTypeId="2" ParentId="10048" CreationDate="2019-01-18T20:37:18.423" Score="1" Body="&lt;p&gt;Yes, it is possible (depending on the nature of your problem), using the four types of standard NEAT mutation, but it is improbable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the NEAT algorithm begins, it operates on a blank canvas. After each generation, the algorithm will either:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Construct a new axon&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Construct a new node on an existing axon&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Update existing weights/bias&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Remove a node or axon from the network&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(However, in general, NEAT does not produce neural networks where two input (or output) nodes are connected.)&lt;/p&gt;&#xA;" OwnerUserId="4744" LastEditorUserId="2444" LastEditDate="2019-10-13T02:03:43.383" LastActivityDate="2019-10-13T02:03:43.383" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10070" PostTypeId="1" AcceptedAnswerId="10076" CreationDate="2019-01-18T21:43:11.930" Score="0" ViewCount="91" Body="&lt;p&gt;I have seen a few articles about neural nets. Mostly they went along these lines: we tried these architectures, these meta parameters, we trained it for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; hours on &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; CPUs, and it gave us these results that are 0.1% better than state of the art. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am interested in is whether there exists (at least as a work in progress) a framework, that gives some explanation why is some architecture better than other, what makes one activation function more suitable for image recognition than other, etc. Do you have some tips about where to start? I would prefer something more systematic than Google search (a book, a list of key articles is ideal). &lt;/p&gt;&#xA;" OwnerUserId="11359" LastEditorUserId="2444" LastEditDate="2019-09-15T23:07:06.520" LastActivityDate="2019-09-15T23:07:06.520" Title="How can I systematically learn about the theory of neural networks?" Tags="&lt;neural-networks&gt;&lt;getting-started&gt;&lt;theory&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10071" PostTypeId="1" CreationDate="2019-01-19T00:25:30.303" Score="2" ViewCount="74" Body="&lt;p&gt;I have an electromagnetic sensor and electromagnetic field emitter.&#xA;The sensor will read power from the emitter. I want to predict the position of the sensor using the reading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me simplify the problem, suppose the sensor and the emitter are in 1 dimension world where there are only position X (not X,Y,Z) and the emitter emits power as a function of distance squared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the painted image below, you will see that the emitter is drawn as a circle and the sensor is drawn as a cross.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kuJR8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kuJR8.png&quot; alt=&quot;one emitter&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. if the sensor is 5 meter away from the emitter, the reading you get on the sensor will be 5^2 = 25. So the correct position will be either 0 or 10, because the emitter is at position 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, with one emitter, I cannot know the exact position of the sensor. I only know that there are 50% chance it's at 0, and 50% chance it's at 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I have two emitters like the following image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/piKWZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/piKWZ.png&quot; alt=&quot;two emitters&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will get two readings. And I can know exactly where the sensor is. If the reading is 25 and 16, I know the sensor is at 10. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So from this fact, I want to use 2 emitters to locate the sensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that I've explained you the situation, my problems are like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The emitter has a more complicated function of the distance. It's&#xA;not just distance squared. And it also have noise. so I'm trying to&#xA;model it using machine learning.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some of the areas, the emitter don't work so well. E.g. if you are&#xA;between 3 to 4 meters away, the emitter will always give you a fixed&#xA;reading of 9 instead of going from 9 to 16.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When I train the machine learning model with 2 inputs, the&#xA;prediction is very accurate. E.g. if the input is 25,36 and the&#xA;output will be position 0. But it means that after training, I&#xA;cannot move the emitters at all. If I move one of the emitters to be&#xA;further apart, the prediction will be broken immediately because the&#xA;reading will be something like 25,49 when the right emitter moves to&#xA;the right 1 meter. And the prediction can be anything because the&#xA;model has not seen this input pair before. And I cannot afford to&#xA;train the model on all possible distance of the 2 emitters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The emitters can be slightly not identical. The difference will&#xA;be on the scale. E.g. one of the emitters can be giving 10% bigger&#xA;reading. But you can ignore this problem for now.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question is &lt;strong&gt;How do I make the model work when the emitters are allowed to move?&lt;/strong&gt; Give me some ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of my ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I think that I have to figure out the position of both&#xA;emitters relative to each other dynamically. But after knowing the&#xA;position of both emitters, how do I tell that to the model?&lt;/li&gt;&#xA;&lt;li&gt;I have tried training each emitter separately instead of pairing&#xA;them as input. But that means there are many positions that cause&#xA;conflict like when you get reading=25, the model will predict the&#xA;average of 0 and 10 because both are valid position of reading=25.&#xA;You might suggest training to predict distance instead of position,&#xA;that's possible if there is no &lt;strong&gt;problem number 2&lt;/strong&gt;. But because&#xA;there is problem number 2, the prediction between 3 to 4 meters away&#xA;will be wrong. The model will get input as 9, and the output will be&#xA;the average distance 3.5 meters or somewhere between 3 to 4 meters.&lt;/li&gt;&#xA;&lt;li&gt;Use the model to predict position&#xA;probability density function instead of predicting the position.&#xA;E.g. when the reading is 9, the model should predict a uniform&#xA;density function from 3 to 4 meters. And then you can combine the 2&#xA;density functions from the 2 readings somehow. But I think it's not&#xA;going to be that accurate compared to modeling 2 emitters together&#xA;because the density function can be quite complicated. We cannot&#xA;assume normal distribution or even uniform distribution.&lt;/li&gt;&#xA;&lt;li&gt;Use some kind of optimizer to predict the position separately for each &#xA;emitter based on the assumption that both predictions must be the same. If &#xA;the predictions are not the same, the optimizer must try to move the &#xA;predictions so that they are exactly at the same point. Maybe reinforcement &#xA;learning where the actions are &quot;move left&quot;, &quot;move right&quot;, etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I told you my ideas so that it might evoke some ideas in you. Because this is already my best but it's not solving the issue elegantly yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So ideally, I would want the end-to-end model that are fed 2 readings, and give me position even when the emitters are moved. How would I go about that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. The emitters are only allowed to move before usage. During usage or prediction, the model can assume that the emitter will not be moved anymore. This allows you to have time to run emitters position calibration algorithm before usage. Maybe this will be a helpful thing for you to know.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastEditorUserId="20819" LastEditDate="2019-01-19T01:17:06.940" LastActivityDate="2019-01-21T06:52:45.587" Title="How do I combine two electromagnetic readings to predict the position of a sensor?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;probability&gt;&lt;bayes&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10072" PostTypeId="2" ParentId="10040" CreationDate="2019-01-19T04:48:32.903" Score="1" Body="&lt;p&gt;It would be irresponsible to try. Several in the past, dating back centuries led to various trends that some call winters. I don't agree they are winters. They are just people returning to reality. Creating intelligence in computers has been progressing forward steadily since Charles Babbage. It has accelerated due to advances in digital electronics, in discrete mathematics, and in brain imaging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to predict, with some unavoidable uncertainty, the impact on the profitability of existing products or services five years into the future, provided the AI efforts in the corporation are planned at a high level first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue is that what we mean when we say the word intelligence is really a collection of abilities. Tesla died with no assets yet his designs are in our appliances and the grids that power them. A billionaire may not be able to draw a sketch of himself worth a penny, and Michelangelo may not have been able to design a way to transmit power to homes and businesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no one thing called intelligence. It is many things. That's why computers exceeded humans in arithmetic long ago, but there are no combinations of hardware and software that can even come close to writing a physics textbook or even passing an arbitrary high school physics final exam. There is much more not known about intelligence then there is known about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monetary evaluation may be necessary as part of the solution, but the end goal is monetary projection. That means one must evaluate the state and trajectory of something AI experts don't yet understand well. We can guess accurately that, if the direction of an AI department or AI groups within multiple existing departments is well managed, gains can be made. We can evaluate the use of AI in the past within the same business type and use the trend we see to make projections along the same line based on how technology trends usually progress. We can also factor in the emergence of competitive businesses also progressing with their use of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That purchasing, R&amp;amp;D, manufacturing, and other departments are interested in AI, have done some initial play with examples and maybe built out some proof of concept code but are unwilling to quantify future potential is not surprising. It is not any easy financial modelling task. To begin the task, some information must be collected and scrutinized to ensure it is an accurate starting assessment.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the kinds of products and services of the business?&lt;/li&gt;&#xA;&lt;li&gt;What are the critical corporate functions that support those?&lt;/li&gt;&#xA;&lt;li&gt;What are the key technological, market, and production characteristics of the business that places it in a group of like businesses?&lt;/li&gt;&#xA;&lt;li&gt;What currently developed and known-to-be-effective AI technologies most likely produce profit enhancements for this business?&lt;/li&gt;&#xA;&lt;li&gt;What past technology advancements are most like those current ones?&lt;/li&gt;&#xA;&lt;li&gt;Quantitatively, what were the financial outcomes of the leveraging of those past technologies over time?&lt;/li&gt;&#xA;&lt;li&gt;How is competitive leveraging of the same technologies by competitive businesses likely to draw business and divide profit margins?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once this information is gathered and its accuracy is ensured, a financial model can be constructed and implemented in software and projections of various scenarios can be made. It is a development project in itself. Many businesses are bypassing this project and just assuming that the current AI craze must have a high ROI because large corporations have made the choice down that road. This question exhibits a more prudent course of action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to the departments listed, there are a few more to consider.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Marketing&lt;/li&gt;&#xA;&lt;li&gt;Finance&lt;/li&gt;&#xA;&lt;li&gt;Strategic planning (at the board and senior management level)&lt;/li&gt;&#xA;&lt;li&gt;Accounts receivable&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-19T04:48:32.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10073" PostTypeId="2" ParentId="2260" CreationDate="2019-01-19T05:04:50.647" Score="2" Body="&lt;p&gt;Sentience is another AI term without a standard definition. It can mean that the system has sensory input, that it has emotion, or that it forms a subjective opinion on what it senses. None of these require any type of non-causal or metaphysical elements, so the limits imagined for AI by those who propose that AI will never have free will or a soul don't really apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether AI has sensory input is a function of the total system design. AI already has sensory input in several products working in their field today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether AI will have emotion is likely, since emotion is a more general state that affects many neurons and has some momentum (since it is based on chemical secretion, agonism, antagonism, uptake, and metabolism) and such design features will probably enter into the AI tool box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether AI will form subjective opinions will depend on how deeply cognition will be realized in computers. Some believe that by adding depth to artificial networks and different interconnecting designs of them, cognition will be realized in those networks. Others (including some members here) are proponents of fuzzy logic as the best match for cognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self awareness isn't the same thing as sentient although related. If computer vision is pointed at the computer and recognizes itself, then it is self-aware at a sensory level. If a computer likes itself or becomes sad because it doesn't have a robot body so it can go shopping, then that's self-awareness at an emotional and cognitive level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there is no proof that these higher level things can be accomplished, there is no theoretical reason yet discovered to doubt their potential development.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-19T05:04:50.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10075" PostTypeId="2" ParentId="10071" CreationDate="2019-01-19T08:50:26.443" Score="1" Body="&lt;p&gt;Model input:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 mean scaled input for each emitter &lt;/li&gt;&#xA;&lt;li&gt;1 distance value for each&#xA;distance&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multiple input&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned there is noise. If the noise is constant, ie you test it in place A and the values returned are always the same, then it means training in different places. If you place it in a place and the first reading is different from the second reading. Then you need to take a lot of readings and select the mean or median of the readings. The central limit theorem says at least 30 readings. This would be the easiest. You could use each sample as an input which allows the NN to learn to filter out the noise. This makes training longer but is probably better than just taking an average.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scaled input&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know you said it is not important for now, and that the emitters come in different scales. I would scale the output of the emitter proportional to it's scale so that two emitters of different size will produce the same output relative to the sensor no matter how far the sensor is from the emitter. This function might be a simple linear function or more complex, depending if output of the emitter drops faster for a smaller emitter than a larger one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Distance value&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned dynamically calculating the position of an emitter, but it was placed there so you must know where it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means you could use two solutions. One a co-ordinate system which is a little more complex, or a simpler solution is a distance vector.&#xA;There must be a maximum distance that one could place the emitters. Lets assume this distance is 25. You could normalise the data as any distance / maximum distance. This should be repeated for all unique combinations of emitters. i.e if you have 2 (A, B) then only one distance value if you have 3 emitters (A,B,C), then 3 distance values ie from A-B, and A-C, and B-C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A co-ordinate system is more complex because using a number to denote a position will apply importance to it, for example a grid from 1 to 10 across the x, and y. Emitter at position 10,10 will have a greater importance than an emitter at 1,1. And if it is 0,0 without a bias input your result will be 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Structure of the NN&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the structure of the NN, the data samples, what you use for validation etc will all play a role. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps some research on previous work.&#xA;See the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/5c78/363d64eaa8aa391943aea0a5dbf9ff359754.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Significant Location Detection &amp;amp; Prediction in Cellular Networks&#xA;Using Artificial Neural Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://journals.sagepub.com/doi/full/10.1155/2015/157046&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mobile Localization Based on Received Signal Strength and Pearson's&#xA;Correlation Coefficient&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/d578/f13b48ac16e5a0c8421bd7060e148c0a7c36.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Discrete Indoor Three-Dimensional Localization System Based on&#xA;Neural Networks Using Visible Light Communication&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20508" LastEditorUserId="20508" LastEditDate="2019-01-21T06:52:45.587" LastActivityDate="2019-01-21T06:52:45.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10076" PostTypeId="2" ParentId="10070" CreationDate="2019-01-19T09:09:05.703" Score="1" Body="&lt;p&gt;Good question, there is a lot of work in that field. The first part before saying which machine learning algorithm is better and why is defining the problem. Is the problem an optimisation, classification, anomaly detection problem because then you need to use the appropriate machine learning algorithms. Let's assume its an optimisation problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this problem, dynamic, or static. Is this time series data? So we need to understand the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each optimisation problem has a &lt;a href=&quot;https://www.youtube.com/watch?v=JPTIZbGGYmk&quot; rel=&quot;nofollow noreferrer&quot;&gt;landscape&lt;/a&gt; or a fitness &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_landscape&quot; rel=&quot;nofollow noreferrer&quot;&gt;landscape&lt;/a&gt;. Computer science has some nice toy &lt;a href=&quot;https://complexitylabs.io/fitness-landscapes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;landscapes&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a lot of work in determining the nature of the problem landscape. Have a look at K Malan's &lt;a href=&quot;http://www.kmalan.co.za/&quot; rel=&quot;nofollow noreferrer&quot;&gt;work&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you can &lt;a href=&quot;https://repository.up.ac.za/bitstream/handle/2263/37155/Malan_Survey_2013.pdf;sequence=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;identify&lt;/a&gt; the problem space and understand it's characteristics then you can start to identify what machine learning functions work well on what kind of landscapes. This is a totally different field of research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example some researchers are working on how different evolutionary algorithms handle different &lt;a href=&quot;https://link.springer.com/chapter/10.1007/10721187_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;landscapes&lt;/a&gt;, or &lt;a href=&quot;https://repository.up.ac.za/handle/2263/66262&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural networks&lt;/a&gt; handle different landscapes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start by exploring the types of machine learning problems.&#xA;Understand the complexity of the problem, followed by classification of machine learning algorithms for specific problem spaces. &lt;/p&gt;&#xA;" OwnerUserId="20508" LastActivityDate="2019-01-19T09:09:05.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10077" PostTypeId="1" CreationDate="2019-01-19T11:44:16.090" Score="2" ViewCount="71" Body="&lt;p&gt;I have a solid understanding what the numbers are. If I want, I can see numbers in everything. Could an AI have the same ability for any incoming information to tag them by numbers same as I have? &lt;/p&gt;&#xA;" OwnerUserId="21556" LastActivityDate="2019-01-21T05:41:58.930" Title="Could AI understand what the number one is and detect it in our real world same as AI can recognize cars and other things?" Tags="&lt;object-recognition&gt;&lt;pattern-recognition&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10078" PostTypeId="2" ParentId="10064" CreationDate="2019-01-19T14:30:41.393" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to maximize the profit inside a trading day and avoid to place the pair (limit buy order, limit sell order) if the profit on that transaction is less than 100$. Be aware that I thought using the &quot;Profit &amp;amp; Loss&quot; as the reward.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To me this implies that your profit per transaction is not the true reward function that you should be using. You don't say directly in the question, but presumably there is some per-transaction cost, tax or other issue which means that these low gain transactions are not desirable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is to find a more accurate reward function. You have suggested a quick fix of subtracting a fixed offset from the reward. This should have a desirable effect of limiting borderline buy/sell arrangements and meeting your constraint, so I would suggest trying it. The main issue I see with it is that you may have changed the reward function so that it doesn't truly reflect your goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better approach is to look more carefully at the problem and your goals at a higher level. There must be a reason for wanting to avoid these smaller profit transactions. What is it, and can it be expressed itself as a reward? For instance, if there will be a transaction fee, include that. If the fee structure or reasoning is complicated or appears delayed or aggregated over many actions, then this makes optimisation harder, but RL is actually designed to cope with this. You could for instance only reward the agent with profit/loss after a group of actions covering a whole day's trade. You would then rely on the learning algorithm to figure out which states and actions combined to generate the observed reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is an unknown or random difference between predicted and actual financial gain then no amount of juggling the reward function gets around that. For RL, learning about and predicting expected gains is &quot;built in&quot;, but that does not mean that this task becomes easier - in fact if prediction in the problem is hard, you may be better off focusing on that and forgetting RL at least initially. Your question is not clear on that, but if you want to avoid risk whilst learning, you should bear in mind that a simple hard rule would likely interfere with the ability of RL to work. At its heart RL is a trial-and-error learning system. Errors should be expected, and are required for the system to learn where best the balance point is between risk and reward. Of course that doesn't help you if the learning system would deliberately make you bankrupt exploring what happens when you sell things at a loss - there are likely to be ways to avoid that and still achieve your goals, but you will need to explain more about your system in a different question.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-20T15:46:29.887" LastActivityDate="2019-01-20T15:46:29.887" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10080" PostTypeId="2" ParentId="10077" CreationDate="2019-01-19T21:18:31.697" Score="2" Body="&lt;p&gt;A understanding for this level of abstraction is technically possible.&#xA;It is not hard to create an AI able to count.&#xA;Unfortunately, this does not imply that the AI knows what 1 is.&#xA;It knows: This is 1 piece of cake and this is 1 sheet of paper.&#xA;But the idea of the number is not grasped yet. &#xA;Also, animals can count, but they do not understand why it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say that not even every human knows what 1 is:&#xA;The number, who is the neutral element of the multiplication.&#xA;The natural successor of 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can say with certainty, that there is no AI, who reached this level of abstraction.&lt;/p&gt;&#xA;" OwnerUserId="19062" LastActivityDate="2019-01-19T21:18:31.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10082" PostTypeId="1" CreationDate="2019-01-20T00:44:44.533" Score="6" ViewCount="2544" Body="&lt;p&gt;I am working to build an deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are &lt;code&gt;{&quot;Buy&quot;: 0 , &quot;Do Nothing&quot;: 1, &quot;Sell&quot;: 2}&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1PlBP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1PlBP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that all the features are well suited for this task. I wanted to use just the standard &quot;Profit &amp;amp; Loss&quot; as a reward, but I hardly thought to get something similar to the above image. The standard P&amp;amp;L will simply place the pair (limit buy order, limit sell order) on every up movement. I don't want that because very often it won't cover the commission and it is not a good indicator to trade manually. I would be interested that the agent can maximize the profit and give me a minimum profit of $100 on every pair (limit buy order, limit sell order). On the picture, I would be interested in something similar to the above picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a reward function that could allow me to get such a result? If so, what is it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the following utility function can work with the purpose of that question?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;U(x) = \max(\$100, x)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That seems correct, but I don't know how the agent will be penalized if it covers a wrong transaction, i.e. the pair (limit buy order, limit sell order) creates a loss of money.&lt;/p&gt;&#xA;" OwnerUserId="21539" LastEditorUserId="21539" LastEditDate="2019-01-20T13:47:38.757" LastActivityDate="2020-07-06T05:03:32.030" Title="Suitable reward function for trading buy and sell orders" Tags="&lt;reinforcement-learning&gt;&lt;deep-neural-networks&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10083" PostTypeId="2" ParentId="10071" CreationDate="2019-01-20T05:50:19.003" Score="2" Body="&lt;p&gt;&lt;strong&gt;Position Detection&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In a traditional data acquisition and control scenario, with some assumptions, the relation between sensors signals &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt;, emitters drive &lt;span class=&quot;math-container&quot;&gt;$\epsilon_j$&lt;/span&gt;, distances &lt;span class=&quot;math-container&quot;&gt;$x_{ij}$&lt;/span&gt;, and calibration factors is modelled as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \forall \, (i, j) \text{,} \quad \frac {s_i} {v_i} = \frac {\epsilon_j} {v_j \, x_{ij}^2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The assumptions include these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear acquisition of magnetic flux signal strength&lt;/li&gt;&#xA;&lt;li&gt;Linear control of magnetic flux signal strength&lt;/li&gt;&#xA;&lt;li&gt;Independent readings either by sequential reading or by use of two distinct emitter frequencies&lt;/li&gt;&#xA;&lt;li&gt;Dismissal of relativistic phenomena&lt;/li&gt;&#xA;&lt;li&gt;Single point emission&lt;/li&gt;&#xA;&lt;li&gt;Single point detection&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It is correct that, with only a single emitter, position of the sensor cannot be accurately determined because the direction from which the signal originates cannot be disambiguated. Two emitters are necessary for reliability. In two dimensional space, three are necessary, thus the term triangulation. In three dimensional space, four are necessary.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Less Known Function with Motion and Noise&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The more complicated function of distance was not specified, whether the sample rate is tightly controlled is not indicated, and the nature and magnitude of the noise relative to the signal was not provided. It also appears there is a low digital accuracy in the readings.&lt;/p&gt;&#xA;&lt;p&gt;To model these contingencies, for motion, &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; shall be the sample index, and &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; remains the detector number. The data acquisition vector is now a tuple of the reading &lt;span class=&quot;math-container&quot;&gt;$r_{ij}$&lt;/span&gt; and time &lt;span class=&quot;math-container&quot;&gt;$t_j$&lt;/span&gt;. The function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; may differ from the inverse squared function due to flux conduction curvature, non-point emission and detection, and other secondary effects. The combination of this function and noise &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, a function of sample time, &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt;, is made discrete, according to the question, rounding or truncating to the nearest integer (indicated by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{I}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \forall \, (i, j) \text{,} \quad (r_{ij}, \; t_j) = \bigg(\mathcal{I}\big(f(\epsilon_j, x_{ij}) + n(s_j)\big), \; s_j\bigg) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are other benefits to the additional emitter than disambiguation of direction. The impact of noise is reduced as redundancy is added, and there is the calibration issue.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Calibration&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;High volume, low cost, electronic parts are not usually calibrated in the factory. Even when they are, the calibration cannot be trusted. Even if calibrated in the factory and then in the lab, the phenomenon of temperature and pressure drift complicates acquisition for passive emitters and transducers. Carefully designed instrumentation and measurement strategy can compensate for component behavioral variance, and redundant emitters and detectors can be used in such designs.&lt;/p&gt;&#xA;&lt;p&gt;Assuming accuracy above that of a mass produced part is required, the calibration voltages &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_j$&lt;/span&gt; must be determined simultaneously and be consistently either relative to magnetic flux levels at some point or to each other. If the environment cannot be controlled, re-calibration may be periodically necessary so that the calibration will remain representative.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The emitters can be slightly not identical. The difference will be on the scale. E.g. one of the emitters can be giving 10% bigger reading. But you can ignore this problem for now.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Calibration issues should not be dismissed until later. They must be built into the model tuned by the parameters converged to an optimal during training. Fortunately, since &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is unknown and encapsulates calibration factors, the addressing up front of calibration will not likely frustrate proper analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Drawing of Samples and Aligning Training and Use Distributions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is important, however, to understand that, When training, the distribution of the training samples must match the distribution of the samples encountered when the training is expected to work. This applies directly to the calibration issue and determines the frequency of re-calibration. In essence, training is calibration. This is not new to the recent machine learning craze. Such was the case with self-adjusting PID controllers in the 1990s.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Addressing Questions in the Ideas Section of the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When I train the machine learning model with 2 inputs, the prediction is very accurate ... but it means that after training, I cannot move the emitters at all. ... I cannot afford to train the model on all possible distance of the 2 emitters.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That is the case if the training samples are not representative or insufficient in number or the model &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is entirely unknown or not used in the convergence strategy.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have to figure out the position of both emitters relative to each other dynamically. But after knowing the position of both emitters, how do I tell that to the model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A model does not know the position of emitters or detectors. A model generalizes these. What you tell the model is what is known for sure about &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{I}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have tried training each emitter separately instead of pairing them as input.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That defies the rule that the training distribution must match the usage distribution. Reliability, accuracy, and speed of convergence will all be damaged by doing that.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Use the model to predict position probability density function instead of predicting the position. ... We cannot assume normal distribution or even uniform distribution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Because of the noise function &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, the function to be learned is necessarily stochastic, but that is not unusual, and that does not mean that convergence during learning will not occur. It merely means that the loss function cannot be expected to reach zero. It can nonetheless be minimized, even with motion.&lt;/p&gt;&#xA;&lt;p&gt;Because the objects attached to detectors and sensors are physical and have mass and forces involved are not nuclear or supernatural, acceleration cannot be either &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$- \infty$&lt;/span&gt;, thus the vectors do not have the Markov property.&lt;/p&gt;&#xA;&lt;p&gt;If the preparation of training data allows the labeling of the readings and time stamps with reference positions derived from a test fixture using digital encoders with high accuracy, then this project is much more feasible. In such a case, it is the patterns in the time series and their relationship to actual position that is being learned. Then a B-LSTM or GNU type cell for network layers may be the best choice.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Maybe reinforcement learning where the actions are &amp;quot;move left&amp;quot;, &amp;quot;move right&amp;quot;, ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Unless the system being designed is required to produce motion control, reinforcement learning or other adaptive control strategies are not indicated. In either case, that the Markov property is not present in a system that involves physical momentum, a form of learning that requires that property may not be the best control strategy.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The emitters are only allowed to move before usage. During usage or prediction, the model can assume that the emitter will not be [stationary]. This allows you to have time to run emitters position calibration algorithm before usage.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is recommended to design the math and fixture used for training as flexibly as possible and then binding variables only after there is no doubt the system is working and various degrees of freedom are superfluous.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-21T05:15:50.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10084" PostTypeId="1" CreationDate="2019-01-20T09:40:47.963" Score="3" ViewCount="434" Body="&lt;p&gt;Are current AI models entirely empiricist?&#xA;Can they be rationalist?If so How?&lt;/p&gt;&#xA;" OwnerUserId="21578" LastEditorUserId="1671" LastEditDate="2019-01-21T21:14:08.653" LastActivityDate="2019-01-21T21:14:08.653" Title="Can an AI model be based on the principles of rationalism?" Tags="&lt;ai-basics&gt;&lt;philosophy&gt;&lt;definitions&gt;&lt;rationality&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10085" PostTypeId="2" ParentId="10084" CreationDate="2019-01-20T09:41:54.803" Score="0" Body="&lt;p&gt;MY OPINION ON WHETHER IT'S POSSIBLE TO BUILD ARTIFICIAL INTELLIGENCE ON THE PRINCIPLES OF RATIONALISM&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a long time philosopher's had struggled to explain how the human mind works and two theories had emerged: empiricism and rationalism&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rationalism states human beings genetically have information about the world that enables them to comprehend the world and that we can build on our knowledge using logical reasoning.For example, according to rationalists 1+1=2 seems so obvious to humans simply because we were born with an intuitive idea of it.&#xA;Empiricism states on the other hand that humans are born with very little or no information and learn only by the power of observation and practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, when it comes to the human mind I think that no theory is fully correct. A mix of both is required to explain the miracle that is the human mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to computers,&#xA;It is a known fact that computers and AI models are so far generally based on the empiricistic model of learning. &#xA;All supervised learning algorithms are empiricist models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there are clustering models.K-means clustering algorithm subdivides data points of a dataset into clusters based on nearest mean values.&#xA;This behaviour seems rationalist since no prior examples are provided to the system but that is an incorrect perception.&#xA;Optimal clustering is achieved after thousands of iterations and humans don't require a fraction of that effort to make satisfactory clusters.&#xA;This interesting example favours rationalism in humans but certainly lowers the possibility of a rationalist AI model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So can an AI model ever be based on the principles of rationalism??&#xA;I believe that to qualify this criteria the computer system running the AI model won't just be a regular computer.&#xA;IN FACT I have an experiment in mind to test this...&#xA;lets train a currently empiricist model on certain data (Say a computer vision problem) &#xA;(After all , the first half-human-half-monkey wasn't born all that smart was he??) &#xA;I train this system say A for a few years,and randomly take some of its data (maybe feature matrices,etc).&#xA;Now I take a newer system B. I give it some data from A(i.e initialize some of its feature matrices to that of A instead of 0,0,,0..) and train B.&#xA;I do this repeatedly on many systems and finally take another system say Z to which I share the previous generation's data but I do not train the system this time.I directly test it, If it does much better than A (before A was trained). I think in some simplified ways it has qualified to be a rationalist AI model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conclusion:&#xA;Unlike most scientists, I think that AI models CAN be RATIONALIST in the future. I also believe that they, in their own way, may even have a sense of right and wrong. However i also believe that all their &quot;intuitive information&quot; can be traced back to an &quot;empiricist observation based model&quot; which passed on what was learnt over generations (exactly how it has occured in human beings, according to me). &lt;/p&gt;&#xA;" OwnerUserId="21578" LastActivityDate="2019-01-20T09:41:54.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10086" PostTypeId="2" ParentId="9966" CreationDate="2019-01-20T12:53:16.067" Score="0" Body="&lt;p&gt;You can try it with &lt;a href=&quot;https://www.tandfonline.com/doi/abs/10.1080/01621459.1985.10478157&quot; rel=&quot;nofollow noreferrer&quot;&gt;ACE&lt;/a&gt; (alternating conditional expectations) - an algorithm that searches for transformations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta(y) = f_1(x_1)+f_2(x_2)+...+f_n(x_n)$$&lt;/span&gt;&#xA;The functions &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt; are estimated from data. I'll give an example in R here. There is also a &lt;a href=&quot;https://github.com/partofthething/ace&quot; rel=&quot;nofollow noreferrer&quot;&gt;package in Python&lt;/a&gt; that does ACE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's generate some data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np &amp;lt;- 100 # number of points&#xA;R &amp;lt;- runif(np, min = 0.9, max = 1.1)&#xA;alpha &amp;lt;- runif(np, min = 0.0, max = 2*pi)&#xA;x &amp;lt;- R*cos(alpha)&#xA;y &amp;lt;- R*sin(alpha)&#xA;par(mfrow=c(1,1))&#xA;plot(x,y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Lz7o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Lz7o.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ACE to estimate &lt;span class=&quot;math-container&quot;&gt;$\theta(y) = f(x)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(acepack)&#xA;a &amp;lt;- ace(x,y, delrsq = 0.0001)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See the transforms, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;par(mfrow=c(1,2))&#xA;plot(a&lt;span class=&quot;math-container&quot;&gt;$x,a$&lt;/span&gt;tx)&#xA;plot(a&lt;span class=&quot;math-container&quot;&gt;$y,a$&lt;/span&gt;ty)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8bDc3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8bDc3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They look like parabolas, so let's fit them.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xx&amp;lt;-drop(a&lt;span class=&quot;math-container&quot;&gt;$x)&#xA;yy&amp;lt;-drop(a$&lt;/span&gt;tx)&#xA;&#xA;plot(xx,yy)&#xA;m.x &amp;lt;- lm(yy ~ xx+I(xx^2))&#xA;xnew=sort(xx)&#xA;lines(xnew, predict(m.x, list(xx=xnew)),col=&quot;red&quot;,lwd=2)&#xA;&#xA;xx&amp;lt;-drop(a&lt;span class=&quot;math-container&quot;&gt;$y)&#xA;yy&amp;lt;-drop(a$&lt;/span&gt;ty)&#xA;&#xA;plot(xx,yy)&#xA;m.y &amp;lt;- lm(yy ~ xx+I(xx^2))&#xA;m.y$coefficients&#xA;xnew=sort(xx)&#xA;lines(xnew, predict(m.y, list(xx=xnew)),col=&quot;red&quot;,lwd=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ecDXM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ecDXM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red parabolas don't go through the transforms exactly but we don't need it. We use them only as hints to find the exact relations. We can tweak the parameters later. The parameters of our approximate fits are &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m.x$coefficients&#xA;&#xA;(Intercept)          xx     I(xx^2) &#xA; 1.31459869  0.07527254 -2.55098259&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which means &lt;span class=&quot;math-container&quot;&gt;$f(x) \approx  1.3-2.56 x^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m.y$coefficients&#xA; (Intercept)           xx      I(xx^2) &#xA;-1.342572495  0.001216412  2.791683219&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which means &lt;span class=&quot;math-container&quot;&gt;$\theta(y) \approx -1.3 + 2.8 y^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, we have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$-1.3 + 2.8 y^2 \approx 1.3-2.56 x^2$$&lt;/span&gt;&#xA;or&#xA;&lt;span class=&quot;math-container&quot;&gt;$$2.56 x^2 + 2.8 y^2 \approx 2.6 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here you can recover &lt;span class=&quot;math-container&quot;&gt;$x^2+y^2 = 1$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="15524" LastActivityDate="2019-01-20T12:53:16.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10088" PostTypeId="2" ParentId="10013" CreationDate="2019-01-20T14:49:32.590" Score="2" Body="&lt;p&gt;The adjective &lt;em&gt;bad&lt;/em&gt; isn't mathematically descriptive. A better term is sub-optimal, which implies the state of learning might appear optimal based on current information but the optimal solution from among all possibilities is not yet located.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a graph representing a loss function, one of the names to measure disparity between the current learning state and the optimal. Some papers use the term &lt;em&gt;error&lt;/em&gt;. In all learning and adaptive control cases, it is the quantification of disparity between current and optimal states. This is a 3D plot, so it only visualizes the loss as a function of two real number parameters. There can be thousands, but the two will suffice to visualize the meaning of local versus global minima. Some may recall this phenomenon from pre-calculus or analytic geometry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If pure gradient descent is used with an initial value to the far left, the local minimum in the loss surface would be located first. Climbing the slope to test the loss value at the global minimum further to the right would not generally occur. The gradient will in most cases cause the next iteration in learning algorithms to travel down hill, thus the term &lt;em&gt;gradient descent&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a simple case beyond just the visualization of two parameters, since there can be thousands of local minima in the loss surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2tS9r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2tS9r.png&quot; alt=&quot;Error Surface Showing How Global Optimum Can be Missed&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many strategic approaches to improve the speed and reliability in the search for the global minimum loss. These are just a few.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Descent using gradient, or more exactly, the Jacobian&lt;/li&gt;&#xA;&lt;li&gt;Possibly further terms of the multidimensional Taylor series, the next in line being related to curvature, the Hessian&lt;/li&gt;&#xA;&lt;li&gt;Injection of noise, which is at an intuitive (and somewhat oversimplified) level is like shaking the graph so a ball representing the current learning state might bounce over a ridge or peak and land (by essentially trial and error) the global minimum &amp;mdash; Simulated annealing is a materials science analogy and involves simulating the injection of Brownian (thermal) motion&lt;/li&gt;&#xA;&lt;li&gt;Searches from more than one starting position&lt;/li&gt;&#xA;&lt;li&gt;Parallelism, with or without an intelligent control strategy, to try multiple initial learning states and hyper-parameter settings&lt;/li&gt;&#xA;&lt;li&gt;Models of the surface based on past learning or theoretically known principles so that the global minimum can be projected as trials are used to tune the parameters of the model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Interestingly, the only way to prove that the optimal state is found is to try every possibility by checking each one, which is not nearly feasible in most cases, or by relying on a model from which the global optimal may be determined. Most theoretical frameworks target a particular accuracy, reliability, speed, and minimal input information quantity as part of an AI project, so that no such exhaustive search or model perfection is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practical terms, for example, an automated vehicle control system is adequate when the unit testing, alpha functional testing, and eventually beta functional testing all indicate a lower incidence of both injury and loss of property than when humans drive. It is a statistical quality assurance, as in the case of most service and manufacturing businesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The graph above was developed for another answer, which has additional information for those interested.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai/7982#7982&quot;&gt;If digital values are mere estimates, why not return to analog for AI?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-20T14:49:32.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10089" PostTypeId="2" ParentId="10010" CreationDate="2019-01-20T15:32:16.860" Score="0" Body="&lt;p&gt;&lt;strong&gt;Project Definition&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Labelled data set contains 21 K rows; 1,936 features; and 1 textual label&lt;/li&gt;&#xA;&lt;li&gt;Label can be 1 of 14 possible categories&lt;/li&gt;&#xA;&lt;li&gt;The first feature is a time stamp reflecting exact or approximate 10 minute sampling period&lt;/li&gt;&#xA;&lt;li&gt;Data content not primarily natural language&lt;/li&gt;&#xA;&lt;li&gt;The intention is to learn the function mapping the features to the label&lt;/li&gt;&#xA;&lt;li&gt;Visualization to observe training intermediate and final results&lt;/li&gt;&#xA;&lt;li&gt;Hoping to simplify implementation using already implemented algorithms and development support&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Use of Recurrent Artificial Network Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is correct that recurrent networks are designed for temporally related data. The later variants of the original RNN design are most apt to produce favorable results. One of the most effective of these variants is the GRU network cell, which is well represented in all the main machine learning libraries, and visualization hooks in those libraries are well documented.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Various Meanings of Attention Mechanisms&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The belief that an attention mechanism beyond those built into the RNN design are needed to emphasize important features may be over-complicating the problem.&#xA;The parameters of the GRU and the other RNN variants already focus attention on particular features during learning. Even a basic feed forward network does that, but the MLP (multilayer perceptron) does not recognize feature trends temporally, so the use of RNN variants is smart.&lt;/p&gt;&#xA;&lt;p&gt;There are other kinds of attention mechanisms that are not inside each cell of a network layer. Research into advanced attention based designs that involve oversight, various forms of feedback from the environment, recursion, or generative designs is ongoing. As the question indicates, those are targeted for natural language work. There is also attention based design for motion and facial recognition and automated walking, driving, and piloting systems. They are designed, tested, optimized, and evolving for the purpose of natural language processing or robotics, not 1,936 feature rows. It is unlikely that those systems can be morphed into something any more effective than a GRU network for this project without considerable further R&amp;amp;D.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output Layer and Class Encoding&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The 14 labels should be coded as 14 of the 16 permutations of a 4 bit output prior to training. And the loss function should dissuade the two illegal permutations.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Response to Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[Of the] 1936 features, one of them [is] date-time timestamps and [the] rest [are] numeric. ... Can you please suggest the format of the input? Should I convert each column of feature to a list and create a list of lists or some other way around?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Regardless of what types the library you use expect as inputs, the theory is clear. Features with a finite set of fixed discrete values are ordinals. The magnitude of their information content is given in bits &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; as follows, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the total number of possible discrete values for the feature.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ b = \log_2 p $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is also true of the timestamp, which has a specific possible range and time resolution, where &lt;span class=&quot;math-container&quot;&gt;$t_{\emptyset}$&lt;/span&gt; is the initial timestamp where the project or its data began and &lt;span class=&quot;math-container&quot;&gt;$t_{res}$&lt;/span&gt; is the time of one resolution step.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ b_{timestamp} = \log_2 \frac {t_{max} - t_\emptyset} {t_{res}} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The label also has a range. If the range is a fixed set of permutations, then assign an integer to each, starting with zero, to encode them. If the range of the text is unknown, use a library or utility that converts words or phrases to numbers. One popular one is word2vec.&lt;/p&gt;&#xA;&lt;p&gt;Integrating the features to reduce the number of input bits actually wastes a layer, so don't do that. The total information is given as this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ b_{total} = \sum_{i = 1}^{1,936} b_i $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The features, if they are real numbers, can remain so. The input layer of an artificial network expects a number entering the data flow for each cell. One can change the data type of the numbers to reduce computational complexity if no overflow or other mapping issue will occur. This is where the above information content can be useful in understanding how far one can go in collapsing the information into a smaller computational footprint.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-25T17:46:41.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10090" PostTypeId="1" CreationDate="2019-01-20T16:00:12.600" Score="4" ViewCount="528" Body="&lt;p&gt;How would I go about designing a (relatively) simple AI that discovers and invents random more complex concepts on its own?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say I had a robot car. It doesn't know it's a car. It has several inputs and outputs, such as a light sensor and the drive motors. If it stays in the dark, it's score drops (bad), and if it moves into the light, it's score rises (good). It'd have to discover that it's motor outputs cause the light input to change (because it's moving closer or farther away from a light source), and that brighter light means higher score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, it'd be easier to design an AI that does specifically that, but I want its behaviour discovery system to be more generic, if that makes any sense. Like later on, it could discover a way to fight or cooperate with other robots to increase its score (maybe other robots destroy light sources when they drive over them, and they can be disabled by driving into them), but it'd have to discover this without initially knowing that another robot could possibly exist, how to identify one, what they do, and how to interact with one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I want it to be creative instead of following a 'do whatever is best to increase your score' rule. Like maybe one day it could decide that cooperating with other robots is another way to increase its score (it finds out what love is), but if it's unable to do that, it becomes depressed and stops trying to increase it's score and just sits there and dies. Or it could invent any other completely random and possibly useless behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How hard would it be to make something like this, that essentially builds itself up from a very basic system, provided I give it lots of different kinds of inputs and outputs that it can discover how to use and apply to its own evolving behavior?&lt;/p&gt;&#xA;" OwnerDisplayName="user21586" LastEditorUserId="2444" LastEditDate="2019-10-03T03:07:51.077" LastActivityDate="2019-10-03T03:07:51.077" Title="How to design an AI that discovers more complex concepts on its own?" Tags="&lt;ai-design&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10091" PostTypeId="2" ParentId="9966" CreationDate="2019-01-20T16:37:18.273" Score="0" Body="&lt;p&gt;&lt;strong&gt;Project Overview&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Producing an output of algebraic constraints from a data set involves several concepts.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data modelling&lt;/li&gt;&#xA;&lt;li&gt;The concepts of variance and outliers&lt;/li&gt;&#xA;&lt;li&gt;Searching an abstract algebraic space&lt;/li&gt;&#xA;&lt;li&gt;Bounding regions in analytic geometry&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A reasonable formulation for what it seems the question seeks is the development of a system of algebraic inequalities that define a narrow bounding surface for a set of data vectors. In this list of bounding rules, each element &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; has a left hand expression &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_n$&lt;/span&gt;, an inequality operator &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}_n$&lt;/span&gt;, and a right hand expression &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}_n$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{L}_1 \; \mathcal{O}_1 \; \mathcal{R}_1 \\&#xA;   \mathcal{L}_2 \; \mathcal{O}_2 \; \mathcal{R}_2 \\&#xA;   \Downarrow \\&#xA;   \mathcal{L}_N \; \mathcal{O}_N \; \mathcal{R}_N \\&#xA;   \, \\&#xA;   n \in [1, N] \quad \land \quad \mathcal{O} \subset \{&amp;lt;, &amp;gt;, \leq, \geq\} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; represent the above set of inequalities, let &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; represent the number of points in the input data set for which a proper boundary surface is sought, and let &lt;span class=&quot;math-container&quot;&gt;$V(\mathcal{S})$&lt;/span&gt; represent the volume of space contained by the bounds in the appropriately dimensioned &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mu = \frac {I} {V(\mathcal{S})} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Over-fitting&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searches of this type unavoidably get into the notion of over-fitting, simply learning the data by memorizing it in exhaustive detail. In this case that would be a set of inequalities that narrow the bounds of the system to exactly the set of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; points in the input. That is obviously not the solution sought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Under-fitting&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of under-fitting might be a multidimensional bounding box or ellipsoid. The simple case given in the question is probably an under-fit since a set of data points that fit nicely into a torus but represented as fitting into a ball is probably not the solution sought either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Balance Between Complexity and Triviality&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is searching for a mechanism that searches for a balance between these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Over-fitting by simply spitting the input back out in another form at the output of the process&lt;/li&gt;&#xA;&lt;li&gt;Trivializing the bounds such that little is actually known about the shape of the phase space and the density of points in the bounded space is dysfunctionally low.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Approach Likely to be Substantially Effective&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assembly of several research deliverables will position the project nicely for a highly effective (and possibly highly coveted) result.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An abstract symbol graph that can represent a set of algebraic inequalities &amp;mdash; See below.&lt;/li&gt;&#xA;&lt;li&gt;A function that expresses equation set complexity&lt;/li&gt;&#xA;&lt;li&gt;A store of other functions that can express other features of the equation set as those features are extracted through use&lt;/li&gt;&#xA;&lt;li&gt;A genetic-like algorithm that can mutate this algebraic equation set in the direction toward either more simplicity or more complexity, or in terms of the increase or decrease of other features discovered&lt;/li&gt;&#xA;&lt;li&gt;A function that calculates the number of outliers&lt;/li&gt;&#xA;&lt;li&gt;A function that calculates point density &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; of the &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; points in the bounded space &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; (already provided above)&lt;/li&gt;&#xA;&lt;li&gt;A definition of a hyper-parameter that defines the acceptable range of distributions of deviation for outliers in relation to the bounding surface &amp;mdash; This is one of the more complex elements in this research task.&lt;/li&gt;&#xA;&lt;li&gt;A definition of a hyper-parameter that defines the acceptable range of equation set complexity&lt;/li&gt;&#xA;&lt;li&gt;A reasonable strategy to reduce the time of search&lt;/li&gt;&#xA;&lt;li&gt;A reasonable strategy to maximize the reliability (the probability of achieving at least one answer within the hyper-parameter ranges set&lt;/li&gt;&#xA;&lt;li&gt;A system for generating points inside the bounds of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt;, giving higher concentrations of points as distance from the boundary surface is greater &amp;mdash; This will be needed for training if DQN is used to search.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;AI Design Considerations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most directed graph libraries provide a solid base to represent the variables, operators on them, the left and right hand sides of the inequalities, and the set of inequalities as a whole. Search for &lt;em&gt;graph library&lt;/em&gt; along with the programming language of your choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reasonably quick convergence on a properly balanced model of the bounds of the data cannot occur if the Markov property is assumed. Remembering what mutations of the inequality set are already tested is essential to avoid replication. Actions (mutations in this case) cannot lead to states (inequality set in this case) that have been tested. By tested, in this case, is meant that the point density, outlier range, and balance criteria have been evaluated and the value of the permutation of possibilities can be stored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this problem differs from the use of DQN application for chess and go, some of the search pruning and application of basic feed forward (multilayer perceptron) networks to self-teach based on generated examples may be very applicable. The fields of bioinformatics and robotic control of defense countermeasures are the most related fields of study, and there may be some public exposure of that work. It is a major project to read through all that material to find something that can be adjusted to this project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is clearly applicable is training data generation. Once the system for holding the inequalities are formed, it should be straightforward to create artificially generated instances of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; and then use the above generation strategy to produce &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; input points to go with the label, which will be &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Project Scope&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is one of those projects that, if done well, may cost much in effort but return much in fame, fortune, and (more importantly) pure thrill of furthering the frontier of knowledge.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-20T16:37:18.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10092" PostTypeId="2" ParentId="10044" CreationDate="2019-01-20T17:05:42.823" Score="4" Body="&lt;p&gt;This is a very large question that could be answered in a variety of ways depending on the context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some optimization problems operating under specific conditions you can make theoretical guarantees that your optimization will solve your problem.  A specific example of this is running the gradient descent algorithm on a convex function.  If the function being optimized is &lt;a href=&quot;https://en.wikipedia.org/wiki/Convex_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;convex&lt;/a&gt; then gradient descent is guaranteed to converge to the correct solution.  Given the nice properties of convex functions, there are many types of optimization where you can make these theoretical guarantees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When evolutionary algorithms (and many non-derivative based optimizations) are run, it's much harder to make these types of guarantees.  Often in the literature I've seen people try their functions on several baseline optimization functions and report back the minimum or maximum found.  Many of these are included in Facebook's &lt;a href=&quot;https://github.com/facebookresearch/nevergrad/blob/master/nevergrad/functions/corefuncs.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;nevergrad&lt;/a&gt; optimization library to be used for evaluation.  An example of one of these functions is included below,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b7G0p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b7G0p.png&quot; alt=&quot;The Rastrigin Function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you can prove that your algorithm minimizes several objective functions you could be testing on, there could be other properties that should be considered.  Some of these include the following: the speed at which the algorithm converges; the number of evaluations of the objective function (if not closed form), and even the amount of space in memory which is required for your optimization.  All of these (and many more) factors could come into play when evaluating your algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scientists may use functions like this one to evaluate their optimization and compare it to other algorithms that are similar.  To answer your final question though, &quot;How can I see if my optimization works for &lt;strong&gt;every&lt;/strong&gt; problem&quot;, I would say that's a bit of a loaded question.  Most of the time these optimizations need some sort of constraint to make a generalized guarantee.  I would look at the constraints of your problem and go from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested, &lt;a href=&quot;https://www.researchgate.net/publication/317724973_Best_Practices_for_Comparing_Optimization_Algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's&lt;/a&gt; a good paper that goes much more in depth than I did here.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastEditorUserId="17408" LastEditDate="2019-01-24T20:36:54.703" LastActivityDate="2019-01-24T20:36:54.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10093" PostTypeId="2" ParentId="9909" CreationDate="2019-01-20T17:35:40.100" Score="-1" Body="&lt;p&gt;The question is whether MCTS is an appropriate method given these conditions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Action &lt;span class=&quot;math-container&quot;&gt;$a_t \in A_t$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;State &lt;span class=&quot;math-container&quot;&gt;$s_t \in S_t$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$[a_t, s_t] \Rightarrow s_{t+1}$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$t_i \land i \in [1, 40]$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A(t) \in \text{set} \, A \; \land \; \text{size} (A) = 6 x 10^6$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$S(t) \in \text{set} \, S \; \land \; \text{size} (S) = 1 x 10^8$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transition function is stochastic&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;MCTS focuses on the most promising sub-trees and is therefore asymmetric and well suited for some systems with a high branching factor, but the set sizes can be prohibitive unless some symmetries are exploited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original MCTS algorithm does not converge quickly.  This is why alpha–beta pruning and other search strategies that focus on the minimization of the search space are often used, but they have limitations with regard to branching breadth too. Pruning heuristics can help with reducing the combinatoric explosion but may lead to the most advantageous action options being missed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some variants of MCTS, such as those among the articles below and in the case of AlphaZero used to learn Go from only the game rules, show excellence in search speed, but perhaps not to the degree needed in this case without parallelism in the form of a computing cluster or significant hardware acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excellent characteristic of MCTS is that highly promising actions found early may be selected without exhausting all the sub-tree evaluations, but again, that may not be enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possibly constraining consideration is whether the Marcov property will be upheld, how, and whether it should be.  Another consideration is whether the system involves an opposing intelligent participant.  All sampling or pruning search strategies involve the risk of not reliably identifying a single branch in a sub-tree that leads readily to an irreversible (or difficult to reverse) loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some excellent papers that discuss these considerations and provide a high level of experience in the form of research results. The computational challenge of tree searches with high branching is covered by the articles discussing approximate solvers, an area of high research intensity.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://robotics.itee.uq.edu.au/~hannakur/dokuwiki/papers/isrr13_abt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An online POMDP solver for uncertainty planning in dynamic environment&lt;/em&gt;&lt;/a&gt;, H Kurniawati, V Yadav, 2016&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://robotics.itee.uq.edu.au/dokuwiki/cache/ICRA15_GoldenABT.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An online and approximate solver for pomdps with continuous action space&lt;/em&gt;&lt;/a&gt;, KM Seiler, H Kurniawati, 2015&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1804.02573&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Hindsight is Only 50/50: Unsuitability of MDP based Approximate POMDP Solvers for Multi-resolution Information Gathering&lt;/em&gt;&lt;/a&gt;, S Arora, S Choudhury, S Scherer, 2018&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.aaai.org/ocs/index.php/IJCAI/IJCAI15/paper/download/10985/11024&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;α-min: A Compact Approximate Solver For Finite-Horizon POMDPs&lt;/em&gt;&lt;/a&gt;, Y Dujardin, T Dietterich, I Chades, 2015&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.aaai.org/ocs/index.php/WS/AAAIW16/paper/download/12582/12346&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Reinforcement Learning as a Framework for Ethical Decision Making&lt;/em&gt;&lt;/a&gt;, D Abel, J MacGlashan, ML Littman, 2016&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1807.04920&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;On the Complexity of Iterative Tropical Computation with Applications to Markov Decision Processes&lt;/em&gt;&lt;/a&gt;, N Balaji, S Kiefer, P Novotný, GA Pérez, 2018&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1802.09646&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Optimizing over a Restricted Policy Class in Markov Decision Processes&lt;/em&gt;&lt;/a&gt;, E Banijamali, Y Abbasi-Yadkori, 2018&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.jair.org/index.php/jair/article/download/10944/26075/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Probabilistic inference techniques for scalable multiagent decision making&lt;/em&gt;&lt;/a&gt;, A Kumar, S Zilberstein, M Toussaint, 2015&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-20T19:17:52.353" LastActivityDate="2019-01-20T19:17:52.353" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10095" PostTypeId="1" CreationDate="2019-01-20T19:09:14.530" Score="1" ViewCount="62" Body="&lt;p&gt;I want to develop an AI based object (mainly toy) picker that can clean my kid's room and put toys in toy basket.&#xA;Can somebody help me how to acheive this?&#xA;I want to make a custom solution so that it would be a learning for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21590" LastEditorUserId="21590" LastEditDate="2019-02-01T07:33:34.203" LastActivityDate="2019-02-01T07:33:34.203" Title="How do I make AI based object picker?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;tensorflow&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="4" ClosedDate="2021-01-23T04:15:37.793" ContentLicense="CC BY-SA 4.0" />
  <row Id="10096" PostTypeId="2" ParentId="9808" CreationDate="2019-01-20T19:09:43.473" Score="0" Body="&lt;p&gt;&lt;strong&gt;Minimax&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the question author may already understand, minimax is an approach before it is an algorithm. The goal is to minimize the advantage of a challenger and maximize the advantage of the participant presently applying the minimax approach. This is a subset of a wider strategy of summing benefits and subtracting the sum of costs to derive a net value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alpha-beta Pruning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strategic goal of alpha beta pruning is to produce uncompromized decision making with less work. This goal is usually driven by the cost of computing resources, the impatience of the person waiting on results, or a missed deadline penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principle involved is that if there is a finite range to the net gain that can aggregate as a result of traversing any subtree in a search. It can be proven, using the algebra of inequalities, that no knowledge can be lost in the search if the subtree cannot, under any conditions, show any net advantage over the other options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some Graph Theory to Clarify Pruning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vertices are the graph theory name for what are often called tree nodes. They are represented as &lt;span class=&quot;math-container&quot;&gt;$\nu$&lt;/span&gt; below, and they correspond to states. The connections between vertices are called edges in graph theory. They are represented as &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; below, and they correspond to actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimax net gain thus far accumulated at an vertex can be calculated as the tree is traversed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the limitations to net gain at further depths are applied to these intermediate net gain values, it can be inferred that one edge (action) will, in every case, be advantageous over another edge. If an algorithm reveals that one edge leads to disadvantage in every possible case, then there is no need to traverse that edge. Such dismissal of never-advantageous edges saves computing resources and likely reduces search time. The term pruning is used because edges are like branches in a fruit orchard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Condition of Pruning and the Algorithm To Accomplish It&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initialization for the algorithm begins by setting the two Greek letter variables to their worst case values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \alpha = - \infty \\&#xA;   \beta = \infty $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pruning rule for edge &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; leading to a vertex &lt;span class=&quot;math-container&quot;&gt;$\nu$&lt;/span&gt; is simply this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \alpha_\nu \geq \beta_\nu \Rightarrow \text{prune}(\epsilon_\nu) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source code doesn't usually directly implement this declaration, but rather produces its effect in the algorithm, which can be represented in simplified pseudo-code. The algorithm to do such is simple once recursion is understood. That is an conceptual prerequisite, so learn about recursion first if not already known.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ab_minimax ν, α, β&#xA;&#xA;    if ν.depth = search.max_depth or ν.is_leaf&#xA;        return ν.value&#xA;&#xA;    if ν.is_maximizing&#xA;        for ε in ν.edges&#xA;            x = max(β, ab_minimax ε.child, α, β)&#xA;            if α ≥ β&#xA;                ν.prune ε&#xA;                return α&#xA;        return α&#xA;    else&#xA;        for ε in ν.edges&#xA;            x = min(α, ab_minimax child, α, β)&#xA;            if α ≤ β&#xA;                ν.prune ε&#xA;                return β&#xA;        return β&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-27T04:17:22.177" LastActivityDate="2019-01-27T04:17:22.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10100" PostTypeId="2" ParentId="4917" CreationDate="2019-01-21T00:51:16.163" Score="1" Body="&lt;p&gt;It's been done (essentially). This guy at the following link has used a series of FPGAs to emulate hundreds of 8080s, using them to train a neural network to play Gameboy games.&#xA;&lt;a href=&quot;https://towardsdatascience.com/a-gameboy-supercomputer-33a6955a79a4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/a-gameboy-supercomputer-33a6955a79a4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IBM's True North being used in Darpa's SyNAPSE program is also very close to what you suggest. &lt;a href=&quot;https://en.wikipedia.org/wiki/TrueNorth&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/TrueNorth&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also of interest may be SpiNNaker and Intel Loihi.&lt;/p&gt;&#xA;" OwnerUserId="18787" LastActivityDate="2019-01-21T00:51:16.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10101" PostTypeId="1" CreationDate="2019-01-21T02:57:05.313" Score="1" ViewCount="28" Body="&lt;p&gt;I'm relatively new to neural networks, and I've been trying to program my own Hopfield network. I got it to the point where it can reliably reproduce a single pattern from a completely scrambled starting state, for up to 400 neurons (potentially more, my computer takes a while for anything bigger than that). When I try it with more than 1 stored pattern however, it appears to settle on a spurious state that looks like some combination (sum?) of them rather than any of them in particular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether the method I'm using to generate my patterns, the size of the network, or any conditions in general can cause this to happen. At the moment, I'm using randomly-generated patterns. To test the network, I set it to be identical to one of the patterns, introduce some random noise by flipping a certain number of nodes, and then seeing if it can recreate the pattern. Could anything about that process be causing the network to settle on that combination of states?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="21599" LastActivityDate="2019-01-21T02:57:05.313" Title="Under What Conditions Will a Hopfield Network Tend to Converge to Spurious States?" Tags="&lt;ai-basics&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10102" PostTypeId="1" CreationDate="2019-01-21T03:38:23.373" Score="4" ViewCount="24" Body="&lt;p&gt;Does it help to &amp;quot;pre-classify&amp;quot; natural language inputs using labeled input fields? E.g., &amp;quot;Who,&amp;quot; &amp;quot;What,&amp;quot; &amp;quot;Where,&amp;quot; &amp;quot;When,&amp;quot; &amp;quot;Why,&amp;quot; &amp;quot;How,&amp;quot; and &amp;quot;How much?&amp;quot; Or is a single, monolithic, free-form, long-text input field equally effective and efficient for model training purposes?&lt;/p&gt;&#xA;Scenario 1: Without input labels&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We are three research fellows, Alice, Bob and Charlie at the University of Copenhagen. We want to understand the development of the human visual system. This knowledge will help in the prevention and treatment of certain vision problems in children. Further, the rules that guide development in the visual system can be applied to other systems within the brain. Our work, therefore, has wide application to other developmental disorders affecting the nervous system. We will conduct this research in 2019 under a budget of $15,000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;Scenario 2: With input lables&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Who:&lt;/strong&gt; We are three research fellows, Alice, Bob and Charlie.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What:&lt;/strong&gt; We want to understand the development of the human visual system.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Where:&lt;/strong&gt; At the University of Copenhagen.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;When:&lt;/strong&gt; During the calendar year of 2019.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why:&lt;/strong&gt; This knowledge will help in the prevention and treatment of certain vision problems in children.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How:&lt;/strong&gt; Further, the rules that guide development in the visual system can be applied to other systems within the brain.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How Much:&lt;/strong&gt; The research will cost $15,000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Use Case:&lt;/p&gt;&#xA;&lt;p&gt;I am building an AI/ML recommendation system. Users subscribe to the system to get recommendations of research projects they might like to participate in or fund. There will be many projects from all over the globe. Far too many for a human to sort through and filter. So AI will sort and filter automatically.&lt;/p&gt;&#xA;&lt;p&gt;Will pre-classifying input fields using labels help the training algorithm be more efficient or effective?&lt;/p&gt;&#xA;" OwnerUserId="21598" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-27T11:45:36.483" Title="Natural language recommendation system: to pre-classify inputs or not?" Tags="&lt;classification&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10103" PostTypeId="2" ParentId="10084" CreationDate="2019-01-21T04:52:17.203" Score="0" Body="&lt;p&gt;An AI model can be based on the principles of rationalism but not necessarily so. This is an open question the answer of which may be both. There are three hypotheses that cover the answer space.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intelligence is rational and theorem proving, construction of mathematical models, probabilistic belief models, and rules inference will be the core of future cognitive AI even though in human brains realize these abilities through neural networks.&lt;/li&gt;&#xA;&lt;li&gt;Intelligence is behavioral and rarely rational, although disciplines have developed through a scientific culture to attempt to order the world through mathematics, humans and other animals with brains evolve and learn intelligent behavior through successive attempts at gaining and edge.&lt;/li&gt;&#xA;&lt;li&gt;Intelligence is neither based on rationalism nor emergent network behavior and the intelligent features of the human brain are merely a reflection of intelligence in the cosmic, physical, and biological realities that have existed in the universe long before the development of brains.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;None of the three have been proven and none have been negated. There may be overlaps or gaps between them, but only minor ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current AI models have to be partly empiricist because theory is always developed to explain sets of observations. They must be partially theoretical (but not necessarily rationalist) because the models must be programmed in declarative or algorithmic form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rational thought and rationalism are not the same. Rationalism is a philosophic system that involves determinism, causality, and doubt about purpose and meaning. Rational thought is a type of thought that applies rules of logic and inference to the analysis of data and challenges in control and comprehension. The idea that rational thought always leads to rationalism has counter-examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Bob is a staunch rationalist but loves his mother without reason and refuses to recant either his rationalistic worldview or his love for his mother even though the two positions are in rational conflict.&lt;/li&gt;&#xA;&lt;li&gt;Science states about itself that it cannot draw a conclusion without making observations, it is self-evident that existence does not depend on observation, and therefore science cannot prove the non-existence of one or more things that may exist and thus reality may extend outside the scope of scientific inquiry.&lt;/li&gt;&#xA;&lt;li&gt;That biology emerged contrary to the second law of thermodynamics is proposed to be a matter of luck, which is fantastic to the extreme.&lt;/li&gt;&#xA;&lt;li&gt;Liar's Paradox: Epimenides (VI century BC) was a Cretan angry with his fellow-citizens who suggested, &quot;All Cretans are liars.&quot; Is this statement true or false? Gödel's incompleteness theorems shocked the world of rationalism by proving that not all true things can be proven.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Returning to AI, we can develop systems that conform to rational thought, but they may not be rationalistic. We can develop systems in a vacuum, without any data or sensory input, but it may not be applicable for any practical task. Some have suggested that mathematics can and to some degree has been mostly developed without data or sensory input and is applicable, which favors the third hypothesis above, that intelligence predates human thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In training artificial networks, we know that the drawing of samples for training can't deviate far from the distribution of input when the trained system is deployed and be expected to work, which is strictly empirical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is a good one because the disparity between these last two paragraphs is large, and that gap might make the thirty year predictions to human like thinking computers may again be overly optimistic by orders of magnitude. Maybe not. It's certainly an area of debate and curiosity.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-21T04:52:17.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10104" PostTypeId="2" ParentId="10077" CreationDate="2019-01-21T05:41:58.930" Score="0" Body="&lt;p&gt;If we had a clear definition of what is meant by understanding, this question would be easier to answer. The question is obviously thinking beyond the mechanical understanding imbued into an arithmetic unit in a CPU to be able to add, subtract, multiply, divide, and compare an integer, one of which is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other kinds of understanding can there be?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Conception of counting numbers and that they begin with 1&lt;/li&gt;&#xA;&lt;li&gt;Oneness in the Buddhist sense and that once nirvana is achieved 1 is everything&lt;/li&gt;&#xA;&lt;li&gt;Notion that 1 can be a representation of uniqueness&lt;/li&gt;&#xA;&lt;li&gt;Distinction of shapes that are visual representations of 1 in various languages&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In recognition of one in sensory input the above kinds of understanding correspond to the below AI challenges.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cognitive skill to the degree that an elementary school student can begin to understand the difference between counting numbers and integers.&lt;/li&gt;&#xA;&lt;li&gt;Comprehension of composition and that a person's consciousness may transcend the confines of a single individual's brain networks&lt;/li&gt;&#xA;&lt;li&gt;Ability to understand entity relationships as a database schema designer would&lt;/li&gt;&#xA;&lt;li&gt;Numeric recognition in multiple fonts and for multiple locales&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The top three seem seem far from what AI has thus far achieved. Only the visual representation is available in implementation form and requires training sets that represent the variety of fonts in multiple language locales required by the use cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question did not say, &quot;Does,&quot; but, &quot;Could.&quot; Since brains do understand at various educational levels these things in the top three examples of kinds of understanding of one, it is likely that some combination of semantic nets, deep learning, fuzzy logic engines, difference engines, and other AI forks of research will some day achieve these higher abstractions in comprehension and be able to apply the understanding at these other levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, this would mean that computers may someday be able to do these three things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Develop math rather than mechanically manipulate mathematical expressions and search for theorems or calculate math that has already been placed in closed form&lt;/li&gt;&#xA;&lt;li&gt;Develop spirituality, have conversion experiences, commune with God, and transcend rationalistic conceptions in thinking&lt;/li&gt;&#xA;&lt;li&gt;Develop software so that programming jobs would be obsolete&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-21T05:41:58.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10105" PostTypeId="2" ParentId="10043" CreationDate="2019-01-21T05:59:58.620" Score="1" Body="&lt;p&gt;The combination of NLP and ML is an interesting one. The most interesting work is not the mainstream systems employing feed forward, convolution kernel, and LSTM networks with word-to-vector input normalization or straight audio samples as input data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kinds that are most interesting are the kinds that more closely mimic the systems in which natural language developed, which are those of the language networks in the brain, including auditory, visual, and motor pathways and the kinds of cognitive abilities that support conversations beyond chat with no underlying comprehension or belief.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That some balanced rise and fall of beliefs and semantic associations are part of real conversation has been known for a few decades. The AI research directions along these lines include semantic nets, recursive artificial networks (an advanced type of RNN just entering the scene), and fuzzy belief models. Some of the current perspectives centered on letter, word, and grammar are obsolete from a linguistics point of view. Contemporary linguistics do not assume words are the elementary objects and formal grammar rules describe legal combinations of them. Smaller elements such as word roots, prefixes, suffixes, conjugations, phonetics, adjacency, and the dynamic aspects of language are more accurate modeling elements of real talk and writing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking up some of these terms in an academic article search engine can pull up some of this research over the last two or three decades so that a historical background is clearer. Many of those articles have proposed systems, mathematics to describe the perspective, and sometimes algorithms and results of testing the modality. After spending a few days looking around, it will be easier to pick a master's thesis or a group of them to run past those at a university.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applications for deeper forms of natural language capability beyond idiot talk chatbots is so profoundly needed, there will be little need to investigate the career options for most of these lines of research, provided the feasibility of developing components that fit into products and services is high for that line of research.&lt;/p&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-21T05:59:58.620" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10109" PostTypeId="2" ParentId="9745" CreationDate="2019-01-21T10:25:21.803" Score="1" Body="&lt;p&gt;This is what  I understood so far from the paper in &lt;a href=&quot;https://arxiv.org/abs/1410.2474&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv&lt;/a&gt; named &quot;Genetic Stereo Matching Algorithm with Fuzzy Fitness&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have 2 images (left and right taken by a stereo camera) of size 28x28. &#xA;The images are in grey scale. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The individual becomes a disparity map. Let's say, we look at the pixel at row 2, column 3 and it has a number 5. That means that the pixel at (2,3) on the left image (&lt;em&gt;reference image&lt;/em&gt;) corresponds to the pixel at (2, 8) on the right image (&lt;em&gt;target image&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author of the paper created three classes black, average, and white pixels classes where they take the 0, 127.5, and 255 values respectively in the grey image scale. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the author calculates the likelikehood of each pixel on both left and right images belonging to the same class by calculating the mean and standard deviation of the class in consideration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matching possibility metric is calculated by choosing the max likelikehood of both images belonging to the same class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Author also added Sobel gradient normalization to the fitness value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic operators seem to work in an easy way where you just swap some parts of the individual (disparity map) with another disparity map to create new individuals and the mutation is just randomly changing the number in the disparity map. &lt;/p&gt;&#xA;" OwnerUserId="14863" LastActivityDate="2019-01-21T10:25:21.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10110" PostTypeId="1" CreationDate="2019-01-21T10:39:51.977" Score="2" ViewCount="47" Body="&lt;p&gt;Let's say there's a ball with features position, velocity, acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These three are all concatenated as inputs to my neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have prior knowledge that position is way more predictive than the other features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I weight the position feature much more strongly than the others? Would just applying a large scalar coefficient to it as preprocessing work? Seems unprincipled...&lt;/p&gt;&#xA;" OwnerUserId="21158" LastActivityDate="2019-01-21T11:11:58.957" Title="How to weight important features" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10111" PostTypeId="2" ParentId="10110" CreationDate="2019-01-21T11:06:07.550" Score="4" Body="&lt;p&gt;If you are training a neural network, it should learn correct weights to use the most predictive feature without any interference. For a strongly predictive feature matched with weaker ones, most NNs will learn the stronger association very quickly. I'm not sure whether there is an easy way to add this as a prior. You could do things such as inject the most influential feature in one or more hidden layers, but that does seem ugly, and only really helps if the predictive feature has certain types of relationship to output (it would probably help most if there is a strong linear relationship).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training problems are usually in the opposite direction - how to extract the small amount of information from more noisy and less influential features that may still add up and sometimes contradict the main predictor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing that springs to mind for NNs that may help in this situation (mostly about fixing the problem of using the less reliable features) is the architecture of &lt;a href=&quot;https://en.wikipedia.org/wiki/Residual_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;residual neural networks&lt;/a&gt;. The use of skip connections builds the neural network in such a way that a default &quot;do nothing&quot; &lt;em&gt;identity&lt;/em&gt; function is encouraged as layers are added, and this allows for combining weak but complex relationships with stronger simple ones. A residual NN can do this with less compromise than changing network hyper parameters to find best fit on other NNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, residual neural networks are good for managing different degrees of complexity that manifest within a problem, which is not necessarily the same as different degrees of predictive power between features.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would just applying a large scalar coefficient to it as preprocessing work?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, this would usually be counter-productive for training a neural network. You should instead be normalising all inputs - a common and effective technique for neural networks is to ensure that each feature is scaled and offset so it has mean 0, standard deviation 1.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-21T11:11:58.957" LastActivityDate="2019-01-21T11:11:58.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10114" PostTypeId="1" AcceptedAnswerId="10116" CreationDate="2019-01-21T13:54:08.467" Score="1" ViewCount="869" Body="&lt;p&gt;If &quot;image captioning&quot; is utilized to make a commercial product, what application fields will need this technique? And what is the level of required performance for this technique to be usable?&lt;/p&gt;&#xA;" OwnerUserId="21613" LastEditorUserId="21130" LastEditDate="2019-01-31T20:54:26.113" LastActivityDate="2019-01-31T20:54:26.113" Title="What's the commercial usage of &quot;image captioning&quot;?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10115" PostTypeId="2" ParentId="10102" CreationDate="2019-01-21T16:50:35.247" Score="2" Body="&lt;p&gt;Likely yes!  When you split out the inputs like this, you are &lt;em&gt;adding information&lt;/em&gt;.  How &lt;em&gt;much&lt;/em&gt; this helps is an open question till you build your system, get some data and start training.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, it would be marvelous to have a machine do &lt;em&gt;all&lt;/em&gt; the work straight unstructured text - but you want a functional, easy-to-use website, not a research project of your own.  To that end, do everything that you can to constrain the scope of the problem, and maximize the information available to your model.  For example, you might want to see if you can add researchers using Google Scholar (so you can link to their profiles, and perhaps mine some information that way).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll be somewhat 'data constrained' till you've got a decent number of research proposals and user interactions to learn from.  Tools like &lt;a href=&quot;https://github.com/NervanaSystems/nlp-architect&quot; rel=&quot;nofollow noreferrer&quot;&gt;our NLP architect&lt;/a&gt; may help you get more out of your text (there are some other really cool new-generation ML-for-NLP packages you should also evaluate).&lt;/p&gt;&#xA;" OwnerUserId="17770" LastEditorUserId="4302" LastEditDate="2019-01-27T11:45:36.483" LastActivityDate="2019-01-27T11:45:36.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10116" PostTypeId="2" ParentId="10114" CreationDate="2019-01-21T16:52:50.753" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If &amp;quot;image captioning&amp;quot; is utilized to make a commercial product, what application fields will need this technique?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are several important use case categories for image captioning, but most are components in larger systems, web traffic control strategies, SaaS, IaaS, IoT, and virtual reality systems, not as much for inclusion in downloadable applications or software sold as a product. These are a few examples.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Provision of captions to reduce the number of user operations (keying and selection) required to post a movie or image to improve posting volume of sticky media and therefore improve the position of a web site or page in terms of human interest&lt;/li&gt;&#xA;&lt;li&gt;Provision of captions to provide HTML header and alt attribute content to improve search engine scoring of page for search terms related to the content of the movie or image&lt;/li&gt;&#xA;&lt;li&gt;Testing to see if the content of an image submitted for posting matches the criteria intended by the owner or stake holder for the posting space on the basis of caption rather than directly through CNN categorization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first two are usually monetized in ways such as these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Improvements in online purchase daily volume&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Acquisition of contact lists with consumer or business interests as fields in the list for marketing purposes&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Additional draw of web traffic to enhance ad impression revenue&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What is the level of required performance can this technique be usable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Performance is in terms of these system qualities.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cost of computing resources per caption generated&lt;/li&gt;&#xA;&lt;li&gt;Accuracy of caption generated (which may be quantified in more than one way)&lt;/li&gt;&#xA;&lt;li&gt;Reliability of generation (since the AI component should produce a rating of confidence with the caption string or produce a null or empty string based on a hyper-parameter confidence threshold&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The quantification of cost, accuracy, and reliability is business dependent. Some may have a nasty negative effect on the business if the caption is wrong or missing. Others use cases may not.&lt;/p&gt;&#xA;&lt;p&gt;In some cases the average revenue generated by the caption's presence is already known and known to be small, which requires that the CNN run time and computing resource requirements must be kept below that. In other cases, a Fortune 500 company CTO said, &amp;quot;Do it and send me the budget needed.&amp;quot; In such cases the budget may be, for all practical purposes, unconstrained, not that a system that wastes time and resources is ever desirable, even if only for energy conservation reasons.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-21T16:52:50.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10117" PostTypeId="2" ParentId="9990" CreationDate="2019-01-21T17:36:39.007" Score="0" Body="&lt;p&gt;I'm not sure I understand your question. However responding to your question in the comments. The difference between the two objectives is that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an ordinary GAN, we want to push &lt;span class=&quot;math-container&quot;&gt;$p(G)$&lt;/span&gt; to be as close as possible to &lt;span class=&quot;math-container&quot;&gt;$p(data)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a conditional GAN, we have a context &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. If we imagine for ease of understanding that &lt;span class=&quot;math-container&quot;&gt;$c=[1,2,3]$&lt;/span&gt; is a discrete variable where all the data can be categorised under one of these c values , then we want to:&#xA; push &lt;span class=&quot;math-container&quot;&gt;$p(G|c=1)$&lt;/span&gt; as close as possible to &lt;span class=&quot;math-container&quot;&gt;$p(data|c=1)$&lt;/span&gt;&#xA;push &lt;span class=&quot;math-container&quot;&gt;$p(G|c=2)$&lt;/span&gt; as close as possible to &lt;span class=&quot;math-container&quot;&gt;$p(data|c=2)$&lt;/span&gt;&#xA;push &lt;span class=&quot;math-container&quot;&gt;$p(G|c=3)$&lt;/span&gt; as close as possible to &lt;span class=&quot;math-container&quot;&gt;$p(data|c=3)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="18312" LastActivityDate="2019-01-21T17:36:39.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10119" PostTypeId="1" AcceptedAnswerId="10120" CreationDate="2019-01-21T18:59:11.423" Score="2" ViewCount="138" Body="&lt;p&gt;This corresponds to Exercise 1.1 of &lt;a href=&quot;http://incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RLBook&lt;/a&gt;, and a discussion followed from &lt;a href=&quot;https://ai.stackexchange.com/questions/6669/why-does-self-playing-tictactoe-not-become-perfect/6675?noredirect=1#comment15233_6675&quot;&gt;here&lt;/a&gt;. &#xA;Considering two reward schemes- &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Win = +1, Draw = 0, Loss = -1&lt;/li&gt;&#xA;&lt;li&gt;Win = +1, Draw or Loss = 0&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can we say something about the optimal Q-values?&lt;/p&gt;&#xA;" OwnerUserId="21509" LastEditorUserId="1671" LastEditDate="2019-01-21T21:15:13.173" LastActivityDate="2019-01-25T11:33:02.470" Title="What will Q-values look like in self-play tic-tac-toe?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;combinatorial-games&gt;&lt;self-play&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10120" PostTypeId="2" ParentId="10119" CreationDate="2019-01-21T19:54:58.790" Score="1" Body="&lt;p&gt;Chapter 1 of Sutton &amp;amp; Barto, doesn't introduce the full version Q learning, and you are probably not expected to explain the full distribution of values at that stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably what you are expected to notice is that the maximum Q values out of possible next states - after training/convergence - should represent the agent's best choice of move. What the actual optimal values are depends on how the opponent plays. In self-play it is possible to find optimal play for both players in the game, and thus the Q values represent true optimal play. However what &quot;optimal play&quot; means is dependent on the goals you have set the agent implied by the reward values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any move which leads to a guarantee that a player can force a win regardless of what the opponent does, would have a Q value of +1. If the agent or opponent can force a draw at best, then it will have the Q value of the draw, and if the opponent can force a win (i.e. the current agent would lose), then the move will have the Q value of a loss. This happens because the learning process copies best case values backwards from the end game states to earlier game states that lead to them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a game with two perfect players, and +1, 0, -1 reward system, then each player will on its turn only see the 0 and -1 moves available. That is because there is no way to force a win in tic-tac-toe, and the perfect opponent will always act to block winning moves. The best choice out of 0 or -1 is 0: each player, when acting under its value estimates, will force a draw. There will be states defined that have a value of +1, but they will never appear as a choice to either player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens if you don't make a difference in rewards between drawing and losing? In the extreme case of having win +1, lose or draw 0 against a perfect opponent, then all of the agent's available Q values will always be 0. The agent would then be faced with no way to choose between defensive plays that force a draw and mistake plays that allow the opponent to win. In turn that means some chance that the opponent will win, even when the agent had learned optimal play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When two agents learn through self-play using the +1, 0, 0 reward scheme it gets more complicated. That is because the opponent's behaviour is part of the Q value system. Some positions will have more opportunities for the opponent to make mistakes, and score more highly. A mistake that allows an opponent to force a win will actually score worse, because the opponent will not make mistakes once it has a sure route to a +1 score. So even though the agent apparently cannot tell the difference between a loss and a draw, it should still at least partially learn to avoid losses. In fact, without running the experiment, I am not sure whether this would be enough to still learn optimal play. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I think it would be possible for the +1, 0, 0 agent to still learn optimal play, although maybe more slowly than the +1, 0, -1 system, because any situation that gave an opponent a chance of winning would allow it to pick the move with best score, reducing the first agents score for a move that arrived there - and this difference will be backed up to earlier positions. However, the learning would become unstable as described above, as against a perfect opponent the difference disappears as all the best options are draws or losses, and the agent will start to make mistakes again.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-25T11:33:02.470" LastActivityDate="2019-01-25T11:33:02.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10121" PostTypeId="2" ParentId="10084" CreationDate="2019-01-21T21:13:22.903" Score="1" Body="&lt;p&gt;It depends on how you define &lt;a href=&quot;https://en.wikipedia.org/wiki/Rationality#Economics&quot; rel=&quot;nofollow noreferrer&quot;&gt;rationality&lt;/a&gt;.  In the field of AI, the game theoretic conception is utilized, in sense of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;rational agent&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to note that a decision making agent can be perfectly rational and still be &quot;wrong&quot; (i.e. make sub-optimal decisions.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is primarily because &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;rationality is bounded&lt;/a&gt; for all non-trivial problems.  The quality of decisions depends on quality of information, time and space available to make the decision in relation to the problem size, and the quality of the analytic processes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logic itself has pitfalls, in that any formal system can either be &lt;a href=&quot;https://en.wikipedia.org/wiki/Completeness_(logic)&quot; rel=&quot;nofollow noreferrer&quot;&gt;complete&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Consistency&quot; rel=&quot;nofollow noreferrer&quot;&gt;consistent&lt;/a&gt;, but never both.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say that the current, strong AIs are empiricists, in that the methods are statistical, and based on testing and evaluation of results.  For instance, a Monte Carlo Tree Search (MCTS) game algorithm will play billions of games against itself and analyze the results to make the strongest choice for any given state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if one considers any decision-making agent, no matter how primitive or weak, to be a form of AI, then there can be non-empirical AI in the sense that the processes heuristic as opposed to statistical.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-01-21T21:13:22.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10122" PostTypeId="2" ParentId="7680" CreationDate="2019-01-22T00:48:13.163" Score="4" Body="&lt;p&gt;It's a subtle issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the A3C algorithm in the &lt;a href=&quot;https://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; (p.4 and appendix S3 for pseudo-code), their actor-critic algorithm (same algorithm both episodic and continuing problems) is off by a factor of gamma relative to the actor-critic pseudo-code for episodic problems in the Sutton and Barto book (p.332 of January 2019 edition of &lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://incompleteideas.net/book/the-book.html&lt;/a&gt;).  The Sutton and Barto book has the extra &quot;first&quot; gamma as labeled in your picture. So, either the book or the A3C paper is wrong?  Not really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key is on p. 199 of the Sutton and Barto book:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If there is discounting (gamma &amp;lt; 1) it&#xA;  should be treated as a form of termination, which can be done simply by including&#xA;  a factor of in the second term of (9.2).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The subtle issue is that there are two interpretations to the discounting factor gamma:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A multiplicative factor that puts less weight on distant future rewards.&lt;/li&gt;&#xA;&lt;li&gt;A probability, 1 - gamma, that a simulated trajectory spuriously terminates, at any time step.  This interpretation only makes sense for episodic cases, and not continuing cases.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Literal implementations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Just multiply the future rewards and related quantities (V or Q) in the future by gamma.&lt;/li&gt;&#xA;&lt;li&gt;Simulate some trajectories and randomly terminate (1 - gamma) of them at each time step.  Terminated trajectories give no immediate or future rewards.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The two interpretations of gamma are valid.  But choosing one or the other means you are tackling a different problem.  The math is slightly different and you end up with an extra gamma multiplying &lt;span class=&quot;math-container&quot;&gt;$G \nabla\ln\pi(a|s)$&lt;/span&gt; with the second interpretation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you are at step t=2 and gamma = 0.9, the algorithm for the second interpretation is that the policy gradient is &lt;span class=&quot;math-container&quot;&gt;$\gamma^2 G \nabla\ln\pi(a|s)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0.81 G \nabla\ln\pi(a|s)$&lt;/span&gt;.  This term has 19% less gradient power than the t=0 term for the simple reason that 19% of simulated trajectories have died off by t=2.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the first interpretation of gamma, there is no such 19% decay. The policy gradient is just &lt;span class=&quot;math-container&quot;&gt;$G \nabla\ln\pi(a|s)$&lt;/span&gt; at t=2.  But gamma is still present within &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; to discount the future rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can choose whichever interpretation of gamma, but you have to be mindful of the consequences to the algorithm.  I personally prefer to stick with interpretation 1 just because it's simpler.  So I use the algorithm in the A3C paper, not the Sutton and Barto book.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question was about the REINFORCE algorithm, but I have been discussing actor-critic.  You have the exact same issue related to the two gamma interpretations and the extra gamma in REINFORCE.&lt;/p&gt;&#xA;" OwnerUserId="21625" LastEditorUserId="21625" LastEditDate="2019-01-22T00:57:07.377" LastActivityDate="2019-01-22T00:57:07.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10124" PostTypeId="2" ParentId="9479" CreationDate="2019-01-22T02:25:29.577" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Simulated Annealing vs genetic algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Simulated annealing is a materials science analogy and involves the introduction of noise to avoid search failure due to local minima. See images below. To improve the odds of finding the global minimum rather than a sub-optimal local one, a stochastic element is introduced by simulating Brownian (thermal) motion. Variants of simulated annealing include injection of random numbers with various distributions and the averaging effect of mini-batching (dividing a batch into segments and performing network parameter adjustment after each).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic algorithms are search methods based on principles of mutation, meiosis, symbiosis, test, elimination of inadequacy, and recursion. The advantages of such approaches is the simulation of sexual reproduction, where the possibility of dominant genetic features from two individuals producing a child individual containing the best of both exists. In a population of children, the probability of such a hybrid emerging is higher. Over several generations, even higher than that.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What kind of problems does simulated annealing perform better than genetic algorithms if any?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This question can only be answered well if comparing original, pure versions of both, not the variants that have developed since their introduction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulated annealing or other stochastic gradient descent methods usually work better with continuous function approximation requiring high accuracy, since pure genetic algorithms can only select one of two genes at any given position.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From my experience, genetic algorithm seems to perform better than simulated annealing for most problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Those performance results would be of value to others and should be published as a paper, presented as an open source project, or published creative commons in the appropriate AI venue. At the very minimum, the results could be placed in the above question or an answer to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2tS9r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2tS9r.png&quot; alt=&quot;Error Surface Showing How Global Optimum Can be Missed&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b7G0p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b7G0p.png&quot; alt=&quot;The Rastrigin Function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.jstor.org/stable/1690046&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Optimization by Simulated Annealing&lt;/em&gt;&lt;/a&gt;, Science S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, Science New Series, Vol. 220, No. 4598, May 1983, pp. 671-680&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] N. Metropolis, A. Rosenbluth, M. Rosenbluth., A. Teller. E. Teller, J. Chem. Phys. 21. 1087 110511&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1401.1084.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Optimised simulated annealing for Ising spin glasses&lt;/em&gt;&lt;/a&gt;, 2015, S.V. Isakov et. al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/profile/Chao_Wang213/publication/272753635_A_parallel_simulated_annealing_method_for_the_vehicle_routing_problem_with_simultaneous_pickup-delivery_and_time_windows/links/5ac6e5cea6fdcc8bfc7f86f9/A-parallel-simulated-annealing-method-for-the-vehicle-routing-problem-with-simultaneous-pickup-delivery-and-time-windows.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A parallel simulated annealing method for the vehicle routing problem with simultaneous pickup–delivery and time windows&lt;/em&gt;&lt;/a&gt;, 2014, Chao Wang et. al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1409.7186.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Feature-based tuning of simulated annealing applied to the curriculum-based course timetabling problem&lt;/em&gt;&lt;/a&gt;, 2015, Ruggero Bellio et. al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Images from Other Answers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai/7982#7982&quot;&gt;If digital values are mere estimates, why not return to analog for AI?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/10044/method-to-check-goodness-of-combinatorial-optimization-algorithm-implementation/10092#10092&quot;&gt;Method to check goodness of combinatorial optimization algorithm implementation&lt;/a&gt;, plotted from &lt;a href=&quot;https://github.com/facebookresearch/nevergrad/blob/master/nevergrad/functions/corefuncs.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/facebookresearch/nevergrad/blob/master/nevergrad/functions/corefuncs.py&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-22T02:25:29.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10125" PostTypeId="2" ParentId="9745" CreationDate="2019-01-22T03:59:50.173" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do not understand how genetic algorithms are used in stereo matching.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first paper referenced in the question is &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.473.6886&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Stereo matching using genetic algorithm with adaptive chromosomes&lt;/em&gt;&lt;/a&gt;, Kyu-Phil Han, Kun-Woen Song, Eui-Yoon Chung, Seok-Je Cho, Yeong-HoHac, 2000. It summarizes an approach this way. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;An individual is a disparity set,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A chromosome has a 2D structure for handling image signals efficiently, and&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A fitness function is composed of certain constraints which are commonly used in stereo matching.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The paper also states, &quot;Genetic algorithms are efficient search methods based on principles of population genetics, i.e. mating, chromosome crossover, gene mutation, and natural selection.&quot; The extension to genetic algorithm purity is contained in this statement in the paper.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To improve the convergence, an informed generation, that a higher selected possibility is assigned to the chromosome including a smaller intensity difference, is adopted. The informed generation uses several random generations plus a selection. In a random generation, only a value is randomly generated. Yet, the informed generation of the proposed algorithm selects the gene which has the minimum intensity difference among the randomly generated genes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The question inquires along a few lines.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it mean that an individual is a disparity map with random numbers?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The goal is to produce order from chaos through successive mutations, combinations, and tests, arriving at genetic code that represents distances of objects that correspond to a left region of pixels and a right region of pixels. Those two regions are identified through alignment of brightness categories, with some permissible sprinkling of outliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An individual in this context is a simulation of a sample from a population with a particular aggregation of genetic codes. The codes indicate disparity between matching image features in the left and right images, measured in horizontal pixels.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[Don't] understand how this works and even WHY you should use genetic algorithm on an algorithm that at its simplest form uses 5 for loops.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The five loops referenced in the question, &lt;a href=&quot;https://github.com/davechristian/Simple-SSD-Stereo&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;in the source at github.com/davechristian/Simple-SSD-Stereo&lt;/em&gt;&lt;/a&gt;, are as follows.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for y in range(kernel_half, h - kernel_half):&#xA;    for x in range(kernel_half, w - kernel_half):&#xA;        for offset in range(max_offset):&#xA;            for v in range(-kernel_half, kernel_half):&#xA;                for u in range(-kernel_half, kernel_half):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This exhaustive search has gross shortcomings in relation to the genetic algorithm with informed generation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The simulation of meiosis in genetic algorithms is essentially a search for hybrids where the best genetic sequences of two parents are preserved in the offspring.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Genetic algorithms are easy to scale across computer clusters without developing specialized hardware acceleration.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Components of failed compositions are not eliminated in an exhaustive search, so there is considerable redundancy in trials.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The second paper referenced in the question is &lt;a href=&quot;https://arxiv.org/pdf/1410.2474.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Genetic Stereo Matching Algorithm with Fuzzy Fitness&lt;/em&gt;&lt;/a&gt;, Haythem Ghazouani, 2014. It proposes, &quot;To get around [convergence speed] limitations, we propose a new encoding for individuals, more compact than binary encoding and requiring much less space. This approach and the use of three lightness classes is shared with the methodology of the first paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This second paper introduces fuzzy matching and the use of the Sobel gradient norms to penalize (dismiss) pixels which project onto uniform regions, [and are therefore] less significant pixels. This paper also provides some comparison results with Han's, Dong's, and Nguyen's algorithms as define in the second paper's bibliography.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-22T03:59:50.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10126" PostTypeId="2" ParentId="4917" CreationDate="2019-01-22T04:32:32.657" Score="0" Body="&lt;p&gt;Intel, NVidia, Samsung, Qualcomm, and other corporations are all designing VLSI chips to provide exactly that type of configurable hardware acceleration. The kinds of lithographic elements that are laid out in arrays inside GPU chips, neuromorphic chips, and other VLSI chips that are intended to accelerate AI techniques achieve training speeds well beyond what a general purpose narrow data and address width CPU like the 8080 could achieve, even with the current clock speeds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general relationship is the greater the dedication of the hardware, the faster it can produce results. It is difficult to gain the flexibility of a general purpose CPU without sacrificing the speed of parallel dedicated processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These new chips don't perform hyper-parameter adjustment scheduling, loss function design, or calculus to produce expressions from the chain rule for back propagation, but those things can be uploaded through the bus that connects the chips to a CPU running an operating system. The existing hardware acceleration cores and streaming processors do support parallel performance of forward feeding through parameter matrices and activation functions, convolution kernel iteration, gradient descent, and back-propagation, given the appropriate functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a movement away from serialization of processes as algorithms to a more streamed digital signal processing, placing in parallel some of what algorithms accomplish through iteration, which is serial. Even with caching, serial processing is slower when the number of iterations through an algorithm's loop is measured in orders of magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge has always been to optimize hardware speed without limiting the kinds of processes that can be accelerated by making them parallel. It is not any easy target to hit, since it is a moving one. New AI techniques are developed every month.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-22T04:32:32.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10131" PostTypeId="2" ParentId="10027" CreationDate="2019-01-22T09:22:11.437" Score="1" Body="&lt;p&gt;&lt;strong&gt;Comparing Unlike Objects&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comparison between a person and an artificial network cannot be made on an equal basis. The former is a composition of many things that the later is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike an artificial network sitting in computer memory on a laptop or server, a human being is an organism, from head to toe, living in the biosphere and interacting with other human beings from birth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Human Training&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have latent intelligence in the zygotes that met to form us and solidified as our genetic code during meiosis, but it is not yet trained. It cannot be until the brain grows from its first cells, directed by the genetic expressions of the brain's metabolic, sensory, cognitive, motor control, and immune structure and function. After nine months of growth, a newborn baby's intelligence is not yet exhibited in motion, language, or behavior other than to suck liquid food.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our intelligence begins to emerge after initial basic behavioral training and does not reach the ability to pass a test indicating academic abilities until the corresponding stages of development in a family structure and components of education are complete. These are all observations well studied and documented by those in the field of developmental psychology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Artificial Networks are Not Particularly Neural&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An artificial network is a distant and distorted conceptual offspring of a now obsolete model of how neurons behave in networks. Even when the perceptron was first conceived, it was known that neurons reacted to activation from electrical pulses transmitted across synapses from other neurons arranged in complex micro-structures, not by performing an activation function to a vector-matrix product. The parameter matrix at the input of artificial neurons are summing attenuated signals, not electro-chemically reacting to pulses that may only be roughly aligned in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then, imaging and &lt;em&gt;in vetro&lt;/em&gt; study of neurons are revealing the complexities of neuro-plasticity (genetically directed morphing of the network topology of neurons), the many varieties of cell types, the groupings of cells to form function geometrically, and the involvement of energy metabolism in the axon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the human brain, chemical pathways of dozens of compounds that regulate function and comprise global and regional states and the secretion, transmission, agonist and antagonist reception, interaction, and uptake of those components is under study. There is barely, if at all, an equivalent in the environment of the artificial networks deployed today, although nothing stops us from designing such regulation systems, and some of the most recent work has pushed the envelope in that direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sexual Reproduction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial networks are also not brains inside individuals produced by sexual reproduction, therefore potentially exhibiting in neurological capacity the best of two parents, or the worst. We do not yet spawn artificial networks from genetic algorithms, although that has been thought of and it is likely to be researched again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Adjusting the Basis for Comparison&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, the basis for comparison renders it meaningless, however, with some adjustment based on the above, another similar comparison can be considered that is meaningful and on a more equal basis.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the difference between a college student and an artificial network that has billions of artificial neurons, well configured and attached to five senses and motor control, integrated inside a humanoid robot that has been nurtured and educated like a member of a family and a community for eighteen years since its initial deployment?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We don't know. We can't even simulate such a robotic experience of eighteen years or properly project what might happen with scientific confidence. Many of the AI components of the above are not yet well developed. When they are &amp;mdash; and there is no particularly compelling reason to think they cannot &amp;mdash; then we will find out together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Research that May Provide an Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From further cognitive science development, real time neuron level imaging, work on the genetic expressions out of which brains grow, artificial neuron designs will likely progress beyond perceptrons and the more temporally aware LSTM, B-LSTM, and GRU varieties and the topologies of neuron arrangements may break from their current Cartesian structural limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neurons in a brain are not arranged in orthogonal rows and columns. They form clusters that exhibit closed loop feedback at low structural levels. This can be simulated by a B-LSTM type artificial network cell, but any electrical engineer schooled in digital circuit design understands that simulation and realization are miles apart in efficiency. A signal processor can run thousands of times faster than its simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From development of computer vision, hearing, tactile-motor coordination, olfactory sensing, materials science support, robotic packaging, and miniature power sources far beyond what lithium batteries can produce may come humanoid robots that can learn while interacting. At that time it would probably be easy to find a family that cannot have children that would adopt an artificial child.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scientific Rigor&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Progress in these areas is necessary for such a comparison to be made on a scientific basis and for confidence in the comparison results to be published and pass peer review by serious researchers not interested in media hype, making the right career moves, or hiking their company's stock prices.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-22T09:22:11.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10133" PostTypeId="1" AcceptedAnswerId="10630" CreationDate="2019-01-22T11:01:17.253" Score="4" ViewCount="5084" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/thmqC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/thmqC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They only reference in the paper that the position embeddings are learned, which is different from what was done in ELMo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ELMo paper - &lt;a href=&quot;https://arxiv.org/pdf/1802.05365.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1802.05365.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BERT paper - &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1810.04805.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17451" LastEditorUserId="2444" LastEditDate="2019-11-01T02:40:01.607" LastActivityDate="2020-02-17T13:44:26.610" Title="What are the segment embeddings and position embeddings in BERT?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;bert&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10134" PostTypeId="1" CreationDate="2019-01-22T12:02:32.973" Score="2" ViewCount="38" Body="&lt;p&gt;When dealing with continuous action spaces, a common choice when designing a policy in policy gradient methods is to learn mean and variance of actions for a specific state and then simply sample from the normal distribution defined by the learned mean and variance to get an action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is, is explicit exploration strategy even needed is such cases, because the dose of randomness in actions would come from the sampling itself, on the other hand there could probably be cases where we would be stuck in a local optimum just by sampling.&lt;br&gt;&#xA;My second question is, in case that explicit exploration is needed, how would one approach this problem of exploration for this specific setup.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-01-22T12:02:32.973" Title="How to include exploration in Gaussian policy" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10135" PostTypeId="1" CreationDate="2019-01-22T12:41:02.997" Score="1" ViewCount="37" Body="&lt;p&gt;Problem: Maintaining the products is a big task for a retailer. If we can      estimate using AI to predict which products will sell the most, we can maintain sufficient stocks of the product without stocking excess inventory. This should be tracked with respect to time because I want to estimate the purchasing levels with respect to seasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are building a AI model which can self learn with respect to purchasing data and provide an estimate to of which products should be purchased and in what volumes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What approach should we take? As now we are currently thinking of using LSTM networks. Please give a proper approach and algorithm.&lt;/p&gt;&#xA;" OwnerUserId="21634" LastEditorUserId="1671" LastEditDate="2019-01-25T00:49:24.317" LastActivityDate="2019-01-25T00:49:24.317" Title="how to estimate which item is bought most using AI" Tags="&lt;ai-basics&gt;&lt;datasets&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10136" PostTypeId="1" AcceptedAnswerId="10146" CreationDate="2019-01-22T15:32:52.187" Score="2" ViewCount="574" Body="&lt;p&gt;I'm trying to understand how the size of the hidden state affects the GRU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose I want to make a GRU count. I'm gonna feed it with three numbers, and I expect it to predict the fourth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I choose the size of the hidden state of a GRU?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2020-04-12T18:30:23.043" LastActivityDate="2020-04-12T18:30:55.687" Title="How do I choose the size of the hidden state of a GRU?" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;hidden-layers&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10143" PostTypeId="1" CreationDate="2019-01-22T22:16:08.970" Score="2" ViewCount="70" Body="&lt;p&gt;In model-based reinforcement learning algorithms, the model of the environment is constructed to efficiently use samples, models such as Dyna, and Prioritize Sweeping. Moreover, eligibility trace helps the model learns (action) value functions faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I know if it is possible to combine learning, planning, and eligibility traces in a model to increase its convergence rate? If yes, how it is possible to use eligibility traces in the planning part, like Prioritize Sweeping?&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2019-02-16T19:05:04.947" LastActivityDate="2019-02-16T19:05:04.947" Title="Eligibility trace In Model-based Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;model-based-methods&gt;&lt;prioritized-sweeping&gt;&lt;eligibility-traces&gt;&lt;dyna&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10144" PostTypeId="1" AcceptedAnswerId="10153" CreationDate="2019-01-22T22:51:01.783" Score="2" ViewCount="436" Body="&lt;p&gt;What is the difference between meta-learning and zero-shot learning? Are they synonymous?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen articles where they seem to imply that they are at least very similar concepts.&lt;/p&gt;&#xA;" OwnerUserId="18312" LastEditorUserId="2444" LastEditDate="2020-03-09T14:53:44.393" LastActivityDate="2020-03-09T14:53:44.393" Title="What is the difference between meta-learning and zero-shot learning?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10145" PostTypeId="1" AcceptedAnswerId="10156" CreationDate="2019-01-23T04:49:26.413" Score="1" ViewCount="87" Body="&lt;p&gt;In an attempt at designing a neural network more closely modeled by the human brain, I wrote code before doing the reading. The neuron I have modeled operates on the following method.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Parameters: potential, threshold, activation.&lt;/li&gt;&#xA;&lt;li&gt;[activation] = 0.0&lt;/li&gt;&#xA;&lt;li&gt;Receive inputs, added to [potential].&lt;/li&gt;&#xA;&lt;li&gt;If ([potential] &gt;= [threshold])&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[activation] = [potential]&lt;/li&gt;&#xA;&lt;li&gt;[potential] = 0.0&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Else&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[potential] *= 0.5&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In short, the neuron receives inputs, and decides if it &quot;fires&quot; if the threshold is met. If not, the input sum, or potential, decreases. Inputs are applied by adding their values to the input potentials of the input neurons, and connections multiply neuron activation values by weights before applying them to their destination potentials. The only difference between this an a spiking network is the activation model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am, however, beginning to learn that Spiking Neural Networks (SNNs), the actual biologically-inspired model, operate quite differently. Forgive me if my understanding is terribly flawed. I seem to have the understanding that signals in these networks are sharp sinusoidal wave-forms with between 100 and 300 &quot;spikes&quot; in a subdivision of &quot;time,&quot; given for 1 &quot;second.&quot; These signals are sampled for the &quot;1 second&quot; by the neuron, and processed by a differential equation that determines the activation state of the neuron. Synapses seem to function in the same manner -&gt; multiplying the signal by a weight, but increasing or decreasing the period of the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wish to know what form of neuron activation model I created. I have been unable to find papers that describe a method like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT. The &quot;learnable&quot; parameters of this model are [threshold] of the neuron and [weight] of the connections/synapses.&lt;/p&gt;&#xA;" OwnerDisplayName="user12941" LastEditorDisplayName="user12941" LastEditDate="2019-01-23T11:26:20.937" LastActivityDate="2019-01-24T00:11:56.600" Title="How does one characterize a neural network with threshold-based activation functions?" Tags="&lt;neural-networks&gt;&lt;human-inspired&gt;&lt;spiking-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10146" PostTypeId="2" ParentId="10136" CreationDate="2019-01-23T05:42:36.170" Score="2" Body="&lt;p&gt;Yes, your understanding of the hidden state is correct. But the size of the hidden state is a hyperparameter that needs to found by trial-and-error. There is no closed-form formula or solution which links the size of the hidden state and the problem at hand. But, there are some rules of thumb like to start out with the size of the hidden state to be a power of 2, etc. Keep tuning the hyperparameter until you get very good predictions.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastEditorUserId="2444" LastEditDate="2020-04-12T18:30:55.687" LastActivityDate="2020-04-12T18:30:55.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10147" PostTypeId="2" ParentId="9983" CreationDate="2019-01-23T08:29:22.927" Score="1" Body="&lt;p&gt;This is the proposed way to reverse engineer software using AI.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Program fake_user operates program target_prog in diverse ways to generate a huge and comprehensive data set.&lt;/li&gt;&#xA;&lt;li&gt;The parameters of an artificial network are trained to produce within specified accuracy and reliability criteria a behavioral equivalent of target_prog.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Not only is this possible, but it is becoming standard practice for AI projects other than reverse engineering games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are caveats.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Program target_prog may be of sufficient complexity to exceed the capabilities of existing network designs and convergence techniques.&lt;/li&gt;&#xA;&lt;li&gt;The project may lack access to funds and computing resources to complete the generation and training required to achieve reasonable accuracy, with sufficient reliability, in the time allotted.&lt;/li&gt;&#xA;&lt;li&gt;The expertise of those involved may not be sufficient to produce satisfactory results.&lt;/li&gt;&#xA;&lt;li&gt;Although the source code is not copied and the parameter state achieved through learning contains equivalent functionality, there is no guarantee that a civil liability may not result. Copyright law one or more jurisdictions may be interpreted as a protection against this kind of copying even though the text of the source code was not copied verbatim.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-23T08:29:22.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10148" PostTypeId="1" AcceptedAnswerId="10169" CreationDate="2019-01-23T11:21:58.557" Score="3" ViewCount="551" Body="&lt;p&gt;I have some documents containing some text (machine writing text) that I intend to apply OCR on them in order to extract the text. The problem is that these documents contain a lot of noise but in different ways (some documents have noise in the middle, others in the top...); which means that I can't apply simple thresholding in order to remove the noise (i.e applying simple threshold does not only remove the noise, but it removes some parts of the text). For these reasons, I thought about using AI to do de-noise the documents.&lt;br&gt;&#xA;Does anyone know if it is possible to do that with AI or any alternative way?&lt;/p&gt;&#xA;" OwnerUserId="19059" LastActivityDate="2019-01-24T18:04:20.467" Title="Is it possible to use AI to denoise noised or 'dirty' documents?" Tags="&lt;machine-learning&gt;&lt;ai-community&gt;&lt;optical-character-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10149" PostTypeId="1" AcceptedAnswerId="10152" CreationDate="2019-01-23T15:33:40.350" Score="4" ViewCount="601" Body="&lt;p&gt;DeepMind's paper &lt;a href=&quot;https://www.nature.com/articles/nature24270.epdf?author_access_token=VJXbVjaSHxFoctQQ4p2k4tRgN0jAjWel9jnR3ZoTv0PVW4gB86EEpGqTRDtpIz-2rmo8-KG06gqVobU5NSCFeHILHcVFUeMsbvwS-lxjqQGg98faovwjxeTUgZAUMnRQ&quot; rel=&quot;noreferrer&quot;&gt;&quot;Mastering the game of Go without human knowledge&quot;&lt;/a&gt; states in its &quot;Methods&quot; section on its &quot;Neural network architecture&quot; that the output layer of AlphaGo Zero's policy head is &quot;A fully connected linear layer that outputs a vector of size 19^2+1=362, corresponding to the &lt;em&gt;logit probabilities&lt;/em&gt; for all intersections and the pass move&quot; (emphasis mine).  I am self-trained regarding neural networks, and I have never heard of a &quot;logit probability&quot; before this paper.  I have not been able by searching and reading to figure out what it means.  In fact, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Logit&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia page on logit&lt;/a&gt; seems to make the term a contradiction.  A logit can be converted into a probability using the equation &lt;span class=&quot;math-container&quot;&gt;$p=\frac{e^l}{e^l+1}$&lt;/span&gt;, and a probability can be converted into a logit using the equation &lt;span class=&quot;math-container&quot;&gt;$l=\ln{\frac{p}{1-p}}$&lt;/span&gt;, so the two cannot be the same.  The &lt;a href=&quot;https://github.com/gcp/leela-zero/blob/master/training/caffe/zero.prototxt&quot; rel=&quot;noreferrer&quot;&gt;neural network configuration for Leela Zero&lt;/a&gt;, which is supposed to have a nearly identical architecture to that described in the paper, seems to indicate that the fully connected layer described in the above quote needs to be followed with a softmax layer to generate probabilities (though I am absolutely new to Caffe and might not be interpreting the definitions of &quot;p_ip1&quot; and &quot;loss_move&quot; correctly).  The &lt;a href=&quot;https://medium.com/applied-data-science/alphago-zero-explained-in-one-diagram-365f5abf67e0&quot; rel=&quot;noreferrer&quot;&gt;AlphaGo Zero cheat sheet&lt;/a&gt;, which is otherwise very helpful, simply echoes the phrase &quot;logit probability&quot; as though this is a well-known concept.  I have seen several websites that refer to &quot;logits&quot; on their own (such as &lt;a href=&quot;https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt;), but this is not enough to satisfy me that &quot;logit probability&quot; must mean &quot;a probability generated by passing a logit vector through the softmax function&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a logit probability?  What sources can I read to help me understand this concept better?&lt;/p&gt;&#xA;" OwnerUserId="21674" LastActivityDate="2019-01-23T16:54:27.737" Title="What is a &quot;logit probability&quot;?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10150" PostTypeId="1" CreationDate="2019-01-23T15:54:01.897" Score="2" ViewCount="134" Body="&lt;p&gt;There are many methods and algorithms dealing with planning problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, according to Wikipedia, there are classical planning problems, with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a unique known initial state,&lt;/li&gt;&#xA;&lt;li&gt;duration-less actions,&lt;/li&gt;&#xA;&lt;li&gt;deterministic actions,&lt;/li&gt;&#xA;&lt;li&gt;which can be taken only one at a time,&lt;/li&gt;&#xA;&lt;li&gt;and a single agent.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Classical planning problems can be solved using classical planning algorithms. The STRIPS framework for problem description and solution, using backward chaining) of the GraphPlan algorithm can be mentioned here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If actions are non-deterministic, according to Wikipedia, we have a Markow Decision Process (MDP), with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;duration-less actions,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;nondeterministic actions with probabilities,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;full observability, or partial observability for POMDP&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;maximization of a reward function,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;and a single agent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;MDPs are mostly solved by Reinforcement Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, classical planning problems can also be formulated as MDPs (with state transition probabilities of 1, i.e. deterministic actions), and there are many examples (e.g. some OpenAI Gyms), where these are successfully solved by RL methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two questions: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there some characteristics of a classical planning problem, which makes MDP formulation and Reinforcement Learning a better suiting solution method? Better suiting in the sense that it finds a solution faster or it finds the (near)optimal solution faster.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do graph search methods like A* perform with classical planning problems? Does STRIPS with backward chaining or GraphPlan always outperform A*? Outperform in the sense of finding the optimal solution faster.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2585" LastEditorUserId="2585" LastEditDate="2019-01-24T08:35:57.220" LastActivityDate="2019-06-23T09:02:15.217" Title="How to choose method for solving planning problems?" Tags="&lt;reinforcement-learning&gt;&lt;graph-theory&gt;&lt;planning&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2021-02-06T23:35:32.420" ContentLicense="CC BY-SA 4.0" />
  <row Id="10151" PostTypeId="2" ParentId="10148" CreationDate="2019-01-23T16:52:57.650" Score="3" Body="&lt;p&gt;This is also the topic of Image Processing (which has analytical solutions instead of learning) mostly through predesigned filters. The filter depends on the type of noise, (salt &amp;amp; pepper, Gaussian, etc.) i.e., for salt &amp;amp; pepper choosing the median in a window. There are a lot of denoising research in literature. There are also more recent learning based denoising applications, but it requires data so that you can train.&lt;/p&gt;&#xA;" OwnerUserId="21679" LastActivityDate="2019-01-23T16:52:57.650" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10152" PostTypeId="2" ParentId="10149" CreationDate="2019-01-23T16:54:27.737" Score="4" Body="&lt;p&gt;Indeed I haven't seen the term &quot;logit probability&quot; used in many places other than that specific paper. So, I cannot really comment on why they're using that term / where it comes from / if anyone else uses it, but I can confirm that what they mean by &quot;logit probability&quot; is basically the same thing that is more commonly referred to simply as &quot;logits&quot;: &lt;strong&gt;they are the raw, unbounded scores of which we generally push a vector through a softmax function to generate a discrete probability distribution that nicely adds up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition fits the one you linked from wikipedia (although that link only covers the binary case, and AlphaGo Zero would have &lt;a href=&quot;https://en.wikipedia.org/wiki/Multinomial_logistic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;multinomial logits&lt;/a&gt; since it has more than two outputs for the policy head).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the AlphaGo Zero paper, the described architecture has a &quot;linear output layer&quot; (i.e. no activation function for the outputs, or the identity function as activation function for the outputs, or however you like to describe it) for the policy head. This means that these outputs are essentially unbounded, they could be any real number. We know for sure that these outputs cannot directly be interpreted as probabilities, even if this isn't stated quite explicitly in the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By calling them logits (or logit probabilities for reasons unknown to me), they are essentially implying that these outputs will still be post-processed by a softmax to convert them into a vector that can be interpreted as a discrete probability distribution over the actions, even if they do not explicitly describe a softmax layer as being a part of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is indeed possible that in Leela Zero they decided to make the softmax operation explicitly a part of the Neural Network architecture. Mathematically they end up doing the same thing... the AlphaGo Zero paper implies (by using the word &quot;logit&quot;) that a softmax is used as a &quot;post-processing&quot; step, and in Leela Zero they explicitly make it a part of the Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a couple more sources for the reasoning that usage of the word &quot;logit&quot; basically implies usage of a softmax, though indeed they do not cover the term &quot;logit probability&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stats.stackexchange.com/a/52836/144483&quot;&gt;https://stats.stackexchange.com/a/52836/144483&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/a/34243720/6735980&quot;&gt;https://stackoverflow.com/a/34243720/6735980&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-23T16:54:27.737" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10153" PostTypeId="2" ParentId="10144" CreationDate="2019-01-23T16:58:40.290" Score="2" Body="&lt;p&gt;First see the &lt;a href=&quot;https://en.wikipedia.org/wiki/Meta_learning_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of meta-learning&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Meta-learning is a subfield of machine learning where automatic learning algorithms are applied to metadata about machine learning experiments. As of 2017 the term had not found a standard interpretation, however the main goal is to use such metadata to understand how automatic learning can become flexible in solving learning problems, hence to improve the performance of existing learning algorithms or to learn (induce) the learning algorithm itself, hence the alternative term learning to learn.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;a href=&quot;https://www.quora.com/What-is-zero-shot-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;zero-shot learning&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Zero-shot learning is being able to solve a task despite not having received any training examples of that task. For a concrete example, imagine recognizing a category of object in photos without ever having seen a photo of that kind of object before. If you've read a very detailed description of a cat, you might be able to tell what a cat is in a photograph the first time you see it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see, these are different but meta-learning can be used in zero-shot learning to work better. For example see &lt;a href=&quot;https://openreview.net/forum?id=r1xurn0cKQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, as an instance.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-01-23T16:58:40.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10155" PostTypeId="1" AcceptedAnswerId="10161" CreationDate="2019-01-23T21:24:24.127" Score="2" ViewCount="64" Body="&lt;p&gt;It is said, that the essence of &lt;a href=&quot;https://www.springer.com/us/book/9780817639495&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.springer.com/us/book/9780817639495&lt;/a&gt; &quot;Neural Networks and Analog Computation. Beyond the Turing Limit&quot; is that the continuous/physical/real-valued weights for neural networks can induce super-Turing capabilities. Current digital processors can not implement real-valued neural networks, they can only approximate them. There are very little efforts to build analog classical computers. But it is quite possible that quantum computers will be analogue. So - is there research trend that investigates true real-valued neural networks on analog quantum computers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google is of no use for my efforts, because it does not understand the true meaning of &quot;true real-valued neural network&quot;, it gives just real-value vs complex valued neural networks articles, which are not relevant to my question.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2019-01-24T13:32:33.963" Title="Can analog quantum computer implement real-valued neural networks and hence do hypercomputation?" Tags="&lt;neural-networks&gt;&lt;quantum-computing&gt;&lt;hypercomputation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10156" PostTypeId="2" ParentId="10145" CreationDate="2019-01-24T00:01:09.240" Score="1" Body="&lt;p&gt;The model you describe is a kind of a &lt;a href=&quot;https://publications.idiap.ch/downloads/reports/2006/paugam-idiap-rr-06-11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;leaky integrate-and-fire (LIF) neuron&lt;/a&gt; (see p. 7). It is leaky because the membrane potential decreases steadily in the absence of input. In contrast, in the simple integrate-and-fire (IF) model the membrane potential is retained indefinitely until the neuron spikes, at which point it is reset to 0. However, LIF neurons are usually modelled with &lt;em&gt;exponential&lt;/em&gt; decay of the membrane potential, where you have a time constant &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; and you compute the potential &lt;span class=&quot;math-container&quot;&gt;$P_{t}$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; based on the potential &lt;span class=&quot;math-container&quot;&gt;$P_{t_{last}}$&lt;/span&gt; at time when the last input arrived as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P_{t} = P_{t_{last}} exp(- \frac{t - t_{last}}{\tau})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the same formula as radioactive decay (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_constant&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more details). The idea is that this model is inherently 'aware' of time, whereas the IF model (and your design above) do not factor in the timing of the spikes, so they act like a classical neural network activation. In any case, whether or not a neuron would fire does depend on the firing threshold, so I think that treating the threshold as a learnable parameter is justified - you just have to decide what rules to use for updating it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on what you describe as your understanding of spiking neural networks, it seems that you have been reading about the Hodgkin-Huxley (HH) model (also in that paper I linked to). (Please correct me if I'm wrong.) You are correct in thinking that spikes in the brain are not infinitely narrow like a delta function but more like a very sharp sinusoidal signal, and the HH model faithfully reproduces that. However, the reason why the HH model is not actually used for simulations is that it is computationally very taxing. In practice, in most cases we do not actually care about the state of the neuron &lt;em&gt;between&lt;/em&gt; inputs, as long as your model accurately describes the neuron state and what happens to it &lt;em&gt;when&lt;/em&gt; an input arrives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other models that approximate the HH model very closely but are much faster to simulate (like the &lt;a href=&quot;https://www.izhikevich.org/publications/spikes.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Izhikevich model&lt;/a&gt;). However, the LIF model is very fast and sufficient in most cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="16101" LastEditorUserId="16101" LastEditDate="2019-01-24T00:11:56.600" LastActivityDate="2019-01-24T00:11:56.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10157" PostTypeId="1" CreationDate="2019-01-24T02:07:08.700" Score="1" ViewCount="259" Body="&lt;p&gt;Sorry if my question is at the wrong place, I'm new in this community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I have dataset with total of 1 million images (augmented) that separated in 28 classes. I followed this tutorial &lt;a href=&quot;https://www.tensorflow.org/hub/tutorials/image_retraining&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/hub/tutorials/image_retraining&lt;/a&gt; to do transfer learning using Inception V3 in Tensorflow to create my own model. But I have no strong background in ML or DL so I'm not sure how to tune the parameter correctly for training step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the training source code that I'm using : &lt;a href=&quot;https://github.com/tensorflow/hub/raw/master/examples/image_retraining/retrain.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorflow/hub/raw/master/examples/image_retraining/retrain.py&lt;/a&gt; &lt;br&gt;&#xA;Using the default setting I was able to get 80~84% accuracy with 16.000 steps. I've tried to change the training, validation, test ratio, training_steps, batch. But, still the accuracy is below 90%. So, Im seeking for advice which parameter that should I change to achieve good accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parameter:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;training_steps&lt;/li&gt;&#xA;&lt;li&gt;learning_rate&lt;/li&gt;&#xA;&lt;li&gt;testing_percentage&lt;/li&gt;&#xA;&lt;li&gt;validation_percentage&lt;/li&gt;&#xA;&lt;li&gt;eval_step_interval&lt;/li&gt;&#xA;&lt;li&gt;train_batch_size&lt;/li&gt;&#xA;&lt;li&gt;test_batch_size&lt;/li&gt;&#xA;&lt;li&gt;validation_batch_size&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="20612" LastActivityDate="2019-01-24T02:07:08.700" Title="Tensorflow : Inception V3 Transfer Learning Parameter Tuning" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;computer-vision&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10158" PostTypeId="1" AcceptedAnswerId="10160" CreationDate="2019-01-24T04:22:43.037" Score="6" ViewCount="392" Body="&lt;p&gt;Imagine that I have an artificial neural network with a single hidden layer and that I am using ReLU as my activating function.&#xA;If by change I initialize my bias and my weights in such a form that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;X * W + B &amp;lt; 0&#xA;$$&lt;/span&gt;&#xA;for every input &lt;strong&gt;x&lt;/strong&gt; in &lt;strong&gt;X&lt;/strong&gt; then the partial derivate of the Loss function with respect to W will always be 0! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a setup like the above where the derivate is 0 is it true that an NN won´t learn anything? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If true (the NN won´t learn anything) can I also assume that once the gradient reaches the value 0 for a given weight, that weight won´t ever be updated?&lt;/p&gt;&#xA;" OwnerUserId="21688" LastActivityDate="2019-01-28T17:13:29.753" Title="How can a neural network learn when the derivative of the activation function is 0?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10159" PostTypeId="2" ParentId="2347" CreationDate="2019-01-24T05:12:06.100" Score="1" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dpqqx.jpg&quot; alt=&quot;enter image description here&quot;&gt;&#xA;More likely as a result after the test every human needs to go truh... And if it’s successful you will reach the next level (The real world after the “virtual reality” we are living in now). &lt;/p&gt;&#xA;" OwnerUserId="19978" LastActivityDate="2019-01-24T05:12:06.100" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10160" PostTypeId="2" ParentId="10158" CreationDate="2019-01-24T07:59:21.400" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In a setup like the above where the derivat[iv]e is 0 is it true that an NN won´t learn anything?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are a couple of adjustments to gradients that &lt;em&gt;might&lt;/em&gt; apply if you do this in a standard framework:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Momentum may cause weights to continue changing if any recent ones were non-zero. This is typically implemented as a rolling mean of recent gradients.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Weight decay (aka L2 weight regularisation) is often implemented as an additional gradient term and may adjust weights down even in the absence of signals from prediction errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If either of these extensions to basic gradient descent are active, or anything similar, then it is possible for the neural network to move out of the stationary zone that you have created after a few steps and then continue learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, then yes it is correct that the neural networks weights would not change at all through gradient descent, and the NN would remain unchanged for any of your input values. Your careful initialisation of biases and weights will have created a system that is unable to learn from the given data. This is a known problem with ReLU activation, and can happen to some percentage of artificial neurons during training with normal start conditions. Other activation functions such as sigmoid have similar problems - although the gradient is never zero in many of these, it can be arbitrarily low, so it is possible for parameters to get into a state where learning is so slow that the NN, whilst technically learning &lt;em&gt;something&lt;/em&gt; on each iteration, is effectively stuck. It is not always easy to tell the difference between these unwanted states of a NN and the goal of finding a useful minimum error.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-01-24T08:21:49.527" LastActivityDate="2019-01-24T08:21:49.527" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10161" PostTypeId="2" ParentId="10155" CreationDate="2019-01-24T08:09:53.633" Score="1" Body="&lt;p&gt;&lt;strong&gt;Digital and Analog&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question about analog computing is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Digital circuitry gained popularity as a replacement for analog circuitry during the four decades between 1975 to 2015 due to three compelling qualities.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Greater noise immunity&lt;/li&gt;&#xA;&lt;li&gt;Greater drift immunity (accuracy)&lt;/li&gt;&#xA;&lt;li&gt;No leakage of stored values&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This quickly led to digital signaling standards, architecture of a general purpose computing, and central processing units on a chip. The later, combined with an array of registers to perform elementary operations is the meaning of the word microprocessor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Quanta and Computing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding quantum computing, there have been some interesting proposals to pack digital gates into much smaller volumes, but the notion that a computer can be made of transistors the size of electrons might be a bit fantastic. That's what the term &lt;em&gt;quantum computing&lt;/em&gt; implies. That degree of miniaturization would have to defy principles of particle physics that are very strongly supported by amassed empirical evidence. Among them is Heisenberg's uncertainty principle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All computing involves quanta, but statistically. For a transistor in a digital circuit to be statistically stable, there must be a sufficient number of Si atoms with at least 0.1% molar concentration of the atoms used to dope the Si to create a PN junction. Otherwise the transistor will not switch reliably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lithographic limit of most mass produced VLSI chips is around 7 nm as of this writing. Crystalline Si, nucleus to nucleus, is about .2 nm, so the miniaturization of a stable transistor is near its quantum limit already. Exceeding that limit by a considerable amount destabilizes the digital circuitry. That's a quantum physics limitation, not a lithographic limitation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Projections, Models, and Techniques to Push Limits&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moore's law was simply an approximate model for the chip industry during the period between the invention of the integrated circuit to the physical limitation of the atomic composition of transistors, which we are now approaching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Field effect transistors (FETs) can take the miniaturization only slightly further than the mechanics of PN junctions. 3-D circuitry has theoretical promise, but no repeatable nanotechnology mechanisms have yet been developed to form complex circuitry in the third dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Returning to the Primary Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Placing aside the magical idea that quantum computing will somehow completely revolutionize circuitry, we have a question that is both feasible and predictable.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can an analog computer implement real-valued neural networks and hence do artificial network computation better?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If we define &lt;em&gt;better&lt;/em&gt; in this context as cheaper and faster, while maintaining reliability and accuracy, the answer is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It definitely takes fewer transistors to create the feed forward part of an artificial network using an analog approximation of the closed forms resulting from the calculus of artificial networks than a digital one. Both are approximations. Analog circuitry has noise, and drift and digital circuitry has rounding error. Beyond rounding, digital multiplication is much more complex in terms of circuitry than analog, and multiplication is used quite a bit in artificial network implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Limitation Interplay of Gödel and Turing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea from the tail end of the title of the book this question referenced, &quot;Beyond the Turing Limit,&quot; is also a little fantastic. The thought experiment of Alan Turing leading to the Turing machine and the associated computability theory (including Turing completeness) was not developed to be a limit. Quite the opposite. It was an answer to Gödel's incompleteness theory. People in Turing's time saw the work Gödel's genius as the annoying but indismissable limit threatening the centuries-old vision of using machines to automatically expand human knowledge. To summarize this work, we can state with assurance this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The theory limiting what computers can do is not related to how the numbers are represented in electronic circuit implementations. It is a limitation of information mechanics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These principles are important but have little to do with the limitation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Analog computing&lt;/li&gt;&#xA;&lt;li&gt;Miniaturization&lt;/li&gt;&#xA;&lt;li&gt;Parallel computing&lt;/li&gt;&#xA;&lt;li&gt;Ways that stochastic injection can help&lt;/li&gt;&#xA;&lt;li&gt;How algorithms can be defined in programming languages&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The above has to do with the feasibility of a project for which some person or corporation must pay and the intellectual capacities required to completing it, not the hard limit on what is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining what a super-Turing capability might be would be a dismissal or a dismantling of what mathematicians consider to be well constructed theory. Dismantling or shifting the contextual frame of some computability theory is plausible. Dismissing the work that has been done would be naive and counterproductive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Real Numbers are Now Less Real Than Integers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The compelling idea contained in the question is the reference to continuity, physicality, and the real valued nature of parameters that acquire a learned state during the training of artificial networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To multiply a vector of digital signals by a digital parameter matrix requires many transistors and can require a significant number of clock cycles, even when dedicated hardware is used. Only a few transistors per signal path are required to perform the analog equivalent, and the throughput potential is very high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To say that real values cannot be represented in digital circuits is inaccurate. The IEEE standards for floating point numbers, when processed in a time series, represent real valued signals well. Analog circuits suffer from noise and drift as stated above. Both analog and digital signals only appear to be comprised of real number values. Real numbers are not real except in the world of mathematical models. What we call quantities in the laboratory are essentially measurements of means of distributions. Solidifying and integrating the probabilistic nature of reality into science and technology may be the primary triumph of the twentieth century, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, when dealing in milli-Amps (mA), electric current seems to be a continuous phenomenon, but when dealing with nano-Amps (nA), the quantum nature of electric current begins to appear. This is much like what happens with the miniaturization of the transistor. Real numbers can only be represented in analog circuits through the flow of discrete electrons. The key to the advantage of an analog forward feed in artificial networks is solely that the density of network cells can be considerably higher, reducing the cost of the network in its VLSI space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, real numbers received the name for their type prior to the emergence of quantum physics. The idea that quantities formerly considered real and continuous were actually statistical averages of discrete activities at a quantum level revolutionized the field of thermodynamics and microelectronics. This is something that disturbed Einstein in his later years. In essence, mathematics using real numbers is effective in engineering because it simplifies what physicist now believe are distributions of a large numbers of quantum phenomena occurring in concert.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summarizing the Probable Future of Analog Computing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This phrase from the question is not precisely scientific, even though it points to a strong likelihood.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is quite possible that quantum computers will be analogue. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This modified version is more consistent with scientific fact in the way it is phrased, and is also factual.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is possible that computers dealing with signals at a near quantum level of miniaturization will have a higher proportion of analog circuitry.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This question and its answers have many links to papers and research work regarding analog computing: &lt;a href=&quot;https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai&quot;&gt;If digital values are mere estimates, why not return to analog for AI?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-24T13:32:33.963" LastActivityDate="2019-01-24T13:32:33.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10162" PostTypeId="1" CreationDate="2019-01-24T08:26:49.993" Score="2" ViewCount="63" Body="&lt;p&gt;Unfortunately there is no &lt;code&gt;speech-recognition&lt;/code&gt; or &lt;code&gt;speech-to-text&lt;/code&gt; tag yet so I go with the &lt;code&gt;voice-recognition&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question concerns various datasets for automated speech recognition and how training and test split should be generated. Even more specific, I'd like to know how important it is to separate speakers from the training, test and dev split.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In very imbalanced datasets one might have very few speaker which contribute a lot to the corpus and thus the model may be biased towards those speakers and have generalized less. Is this concern legitimate?&lt;/p&gt;&#xA;" OwnerUserId="20162" LastActivityDate="2019-01-24T08:26:49.993" Title="Training, test, dev split in speech recognition" Tags="&lt;deep-learning&gt;&lt;voice-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10163" PostTypeId="1" CreationDate="2019-01-24T11:47:38.330" Score="3" ViewCount="1047" Body="&lt;p&gt;I am curious if it is possible to do so.&lt;/p&gt;&#xA;&lt;p&gt;For example, if I supply&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$[0, 1, 2, 3, 4, 5]$&lt;/span&gt;, the model should return &amp;quot;natural number sequence&amp;quot;,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$[1,3,5,7,9,11]$&lt;/span&gt;, it should return &amp;quot;natural number with step of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;&amp;quot;,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$[1,1,2,3,5]$&lt;/span&gt;, it should return &amp;quot;Fibonacci numbers&amp;quot;,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$[1,4,9,16,25]$&lt;/span&gt;, it should return &amp;quot;square natural number&amp;quot;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;and so on.&lt;/p&gt;&#xA;" OwnerUserId="21696" LastEditorUserId="2444" LastEditDate="2020-11-13T23:56:55.843" LastActivityDate="2020-11-13T23:56:55.843" Title="Can a machine learning model predict the pattern of given sequence?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;supervised-learning&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10164" PostTypeId="1" CreationDate="2019-01-24T13:28:51.610" Score="3" ViewCount="54" Body="&lt;p&gt;I am trying to write an AI to a game, where there is no real adversary. This means, that only the AI player has choices in which move to perform, his opponent may or may not react to the move the AI player made, but when he reacts, he will always do the one and only single move that he is able to do. The goal of this AI would be, to find a solution to the situation, which results in the least amount of monster activations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To explain this a bit further, I will describe the game in a few words: there is a 3x3 board, on which there are some monsters. These monsters has a prewritten AI, and activate based on prewritten rules, ie, they do not have to make any decision at all. This is done, by an enrage mechanic, meaning, that when a monster hits it's enrage limit, it activates, and performs his single move action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI should control the other side of this board, the hero players. Each hero player has a different number of possible moves, each move dealing an amount of damage to the monsters, and increasing it's enrage value, thus getting him closer to his enrage limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to achieve, is to write an AI, that will perform this fight in the least amount of monster activations as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I've written a minimax algorithm for this, without the &lt;code&gt;min&lt;/code&gt; player. I've done this, by calculating the negative effect of the monsters move, in the maximizing and only players move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI works in the following way: he draws the game tree for a set amount of depth of moves, calculates the bottom move with a heuristic function, selects the highest value from the given depth, and returns the value of this function up one level, then repeat. When he reaches the top of the tree, he performs the move, with the highest quantification value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works, somewhat, but I have a big problem: &lt;strong&gt;As there is no randomness in the game, I was expecting that the greater the depth that he can search forward, the better moves he will find, but this is not always the case, sometimes a greater depth, returns a worse solution then a smaller depth&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what could cause the above error? My quantification function? The weights that I use in the function? Or something else?&lt;/li&gt;&#xA;&lt;li&gt;is &lt;code&gt;minimax&lt;/code&gt; the correct algorithm to use, for a game where there is no real adversarry, or is there any algorithm that will perform better for a game like this?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21700" LastActivityDate="2019-01-24T14:56:34.227" Title="What kind of decision rule algorithm is usable in this situation?" Tags="&lt;minimax&gt;&lt;heuristics&gt;&lt;decision-tree&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10165" PostTypeId="2" ParentId="10158" CreationDate="2019-01-24T14:42:08.207" Score="1" Body="&lt;p&gt;&lt;strong&gt;Learning and Zero Derivatives&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial networks are designed so that even when the partial derivative of a single activation function is zero they can learn. They can also be designed to continue learning when the derivative of the loss&lt;sup&gt;1&lt;/sup&gt; function is zero too. This resilience to a vanishing feedback signal amplitude, by design, determines some of how calculus results are employed in the learning algorithm, hardware acceleration circuitry, or both. By learning behavior is meant the behavior of the changes to the parameters of the network as learning occurs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For many of the activation functions used today, the derivative of the of the activation function is never the real expression 0, but there are such cases. These are examples of when the evaluation of the derivative of the activation function is effectively zero.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All the time for a binary step activation function, which is why they are usually only used for the very last layer of a network to discretize the output.&lt;/li&gt;&#xA;&lt;li&gt;When the input of a ReLU activation function is negative, which is the case given in the question&lt;/li&gt;&#xA;&lt;li&gt;When the granularity of the IEEE representation of the number can no longer support the smallness of the absolute value of the number upon evaluation&lt;/li&gt;&#xA;&lt;li&gt;When the loss is zero&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Nearing Zero&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last condition can easily occur if the result of the loss function output is so close to zero that the digital products of that number, during propagation, rounds to zero in the floating point hardware. Even if not zero, the number can be so small that learning slows to an untenable speed. The learning process either oscillates, in many cases chaotically, because of rounding phenomena or finds a static state and remains there. Again, this does not necessarily require a zero partial in the Jacobian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Familiar Analogy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cognitive equivalent that helps intuition in understanding this but is not at all a great and across-the-board accurate analogy is the mental concept of doubt. The advantages of various directions of change or action to produce change is no longer clear. This is a rough analogy that some can connect to when considering what it means when the gradient is vanishing. When looking at gradient in historical context, where gradient is the slope of a surface in a location where gravity defines which direction is down, a vanishing gradient is a place where no direction seems to be down hill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Flat in One Dimension by Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the question, where an inner layer&lt;sup&gt;2&lt;/sup&gt; is a ReLU activation function, the evaluation of the partial derivative of the loss function with respect to the parameter being adjusted will always be zero if its input is negative. However, this is by design and is one of the reasons ReLU trains fast. When the signal is negative going into the layer at that particular cell, it is thrifty to ignore it. The other cells upstream are then altered through other paths around the deactivated cell with the zero partial. A neuroscientist might smile at the oversimplification, but this is like a missing synapse between two adjacent cells in the brain.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In a setup like the above where the derivative is 0, is it true that a [network] won't learn anything?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is false. Learning will stop if all the derivatives in a layer are zero and no other device, such as curvature, momentum, regularization, and other devices controlled by hyper-parameters, is employed. Even so, zeros across the layer would only affect the adjustment of upstream layers, layers closer to the input. Downstream, convergence activity may continue in such a case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Zero and close to zero values (as well as near overflows) are kinds of saturation conditions, and these are studied carefully in artificial network research, but a single cell with a zero partial will not stop learning and may, in specific cases, ensure its completion and the adequacy of its result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some Calculus&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mathematical terms, if the Jacobian has a zero in one position, the others may remain active indicators of proper adjustment magnitude and direction for the individual parameters. If the Hessian is used or various types of momentum, regularization, or other techniques are employed, zeros across the Jacobian will probably not block upstream learning, which is part of the reason why they are used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Analogy for Momentum&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analogy can again be employed to clarify momentum as a principle, with the caveat that it is again an oversimplification. Beliefs have momentum, so when a belief exists and all other indications of direction for the next step is unsure, most will base their next step upon their beliefs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how all organisms with a brain tend to work and why mouse traps and spider webs can catch. Without viable feedback from which learning can occur, the organism will act based on the momentum in its DNA or networks of its brain. This is usually beneficial, but can lead to loss in specific cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gradients are not fool proof either. The problem of local minima can render pure gradient descent dysfunctional as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Analogy for Curvature&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curvature (as when the Hessian is employed) requires a different analogy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a smart, blind person is on a dry, flat surface, thirsty, and needing water, the gradient may be flat, they may feel with their foot or cane for some indication of curvature. If some down curving feature of the surface is found, that may guide the person to water in more cases than a random step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As hardware acceleration improves, the Hessian, which is to computationally heavy for CPU execution in most cases, may emerge as standard practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, this is simply moving from two terms of the Taylor series expansion in multivariate space to three terms. From a mechanics perspective, it is the inclusion of acceleration to velocity.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Loss or any of these functions that drive behavior in AI systems: Error, disparity, value, benefit, optimality, and others of similar evaluative nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Inner layers in artificial networks are often called hidden layers, but that is not technically correct. Even though the encapsulation of a neural network may hide signals in inner layers from the programmer, it is a low level software design feature to do that, and a bad one. One can usually and definitely should be able to monitor those signals and produce statistics on them. They are not hidden in the mathematics, as in some kind of mathematical mystery, and the only difference between them and the output layer is that the output layer is intended to drive or influence something outside the network.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-28T17:13:29.753" LastActivityDate="2019-01-28T17:13:29.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10168" PostTypeId="2" ParentId="9996" CreationDate="2019-01-24T16:44:52.397" Score="0" Body="&lt;p&gt;&lt;strong&gt;Project Statement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the text of the question it appears that the project is to produce a model of compounds and their properties from validated data so that those properties can be predicted for compounds not yet characterized. The properties include these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of atoms per molecule&lt;/li&gt;&#xA;&lt;li&gt;Number of cycles (rings) per molecule&lt;/li&gt;&#xA;&lt;li&gt;Specific volume at some equilibrium temperature and pressure&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The predictability of such properties may be possible within some boundaries of reliability and accuracy. For instance, just as an example, such a system may produce three significant digits of accuracy for one of the properties for 99.6% of the compounds to which intelligent prediction is applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question author would like to use PLS regression and is aware that selection of type (int64, float, ...) and the standardization of the data is important. The author tried scaling automation, including those below, but is concerned that the R2/Q2 ratio is low, which is a reasonable concern.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;standard scaler&lt;/li&gt;&#xA;&lt;li&gt;normalize&lt;/li&gt;&#xA;&lt;li&gt;minmax scaler&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scaling, Standardizing, and Normalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as compounds can be characterized, the distribution of data for a particular property across a large set of compounds can also be characterized at a higher level of abstraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious is the domain of the features used as input to the learning process. (These same dimensions of the data, which have domains during when they are inputs to the learning process, will have ranges when they are the outputs of the trained network.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain of training data dimensions (both features and labels) can be examine for their characteristics.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Absolute minimum in the data&lt;/li&gt;&#xA;&lt;li&gt;Theoretical minimum&lt;/li&gt;&#xA;&lt;li&gt;Absolute maximum in the data&lt;/li&gt;&#xA;&lt;li&gt;Theoretical maximum&lt;/li&gt;&#xA;&lt;li&gt;Mean(s)&lt;/li&gt;&#xA;&lt;li&gt;Standard deviation(s)&lt;/li&gt;&#xA;&lt;li&gt;General type of distribution (Gaussian, Poisson, inverse exponential, ...)&lt;/li&gt;&#xA;&lt;li&gt;Skew(s)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Combined with the granularity and expected accuracy of each dimension, the type of variable to represent the data for each dimension can be selected.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Integer&lt;/li&gt;&#xA;&lt;li&gt;Fixed point&lt;/li&gt;&#xA;&lt;li&gt;Floating point&lt;/li&gt;&#xA;&lt;li&gt;String&lt;/li&gt;&#xA;&lt;li&gt;Recursive structure&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The size of the variable type and the normalization of the data is not something easy to automate. In fact, experts tend to put it off until after the distribution is well understood, perhaps along with the correlation of input dimensions to understand redundancy versus uniqueness in the data. The goal, when preparing input for learning systems in general, is to not waste bits with redundancy or superfluousness, either across the training examples or across the dimensions of each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this across the entire training set would be a user column or a binary flag that indicates whether the example is a molecule. The user is irrelevant to the training and could actually distort the results, and the entire training set is for molecules, so that flag is redundant across the domain of example indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this between features is molecular weight in combination with the molecular formula. The atomic weights and their quantities for each element within the molecule is within the formula. Molecular weight can be calculated from this information. If the formula is properly represented in the input vector, the addition of an input for molecular weight may actually be counterproductive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a proper encoding and normalization of data, it may be possible to predict crystalline tendencies, thermal properties, electrical properties, and phase change surface in PVT space. It would take experimentation to determine the feasibility of those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the best scenario is when the range of the variable is a superset of the expected long-term domain of the values, but by a minimal margin on both sides and the distribution of values is substantially normal. There is more information about normalization here: &lt;a href=&quot;https://ai.stackexchange.com/questions/9089/gradient-descent-feature-scaling/9174#9174&quot;&gt;Gradient Descent Feature Scaling&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Specific Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These sub-questions are specifically listed.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible that by scaling, some of the very important features lose their significance, and thus contribute less to explaining the variance of the response variable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Only if the scaling is done poorly, not according to the above principles. If, in the preparation of input, information is lost, it is not available to the learning system. Don't do that.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If yes, if I identify some important features (by expert knowledge), is it OK to scale other features but those? Or scale the important features only?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Use the expert knowledge. Notice the difference between the min and max in the data and the theoretical min and max. Take the union of both of these to ensure nothing falls outside the range of the standardization and choice of data type for input. With regard to what is and is not scaled. Don't scale if it is not needed to utilize the above principles, such as a flag that fits perfectly into a Boolean input. Otherwise, scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only use the scaling automation after you've checked the data and know what the automation will do in general before you use it. Scalars are good for automating calculations, not for making choices about data normalization. If and when computers become smarter than us with complex conceptual decision making, then we can delegate cognitive tasks to them.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Some of the features, although not always correlated, have values that are in a similar range (e.g. 100-400), compared to others (e.g. -1 to 10). Is it possible to scale only a specific group of features that are within the same range?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If some dimensions are similar in distribution and extent (min and max), the findings and the methodology applied to one dimension can be reused by the others of similar distributions and extents.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] It is not clear why the number of atoms per molecule would require prediction, since a simple parsing of the compound's formula should provide that number.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-25T12:09:28.590" LastActivityDate="2019-01-25T12:09:28.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10169" PostTypeId="2" ParentId="10148" CreationDate="2019-01-24T18:04:20.467" Score="4" Body="&lt;p&gt;It is possible remove noise from signal if what is considered noise is distinguishable from what is considered signal. This is true of documents, images, audio, or any other type of information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If dirty means wrong content, then probably not. If dirty means mispellings, then probably yes. If dirty means tilted by a few degrees and containing edge artifacts around the border from scanning, then definitely yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most OCR software, such as the open source program &lt;a href=&quot;https://github.com/tesseract-ocr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Tesseract&lt;/em&gt;&lt;/a&gt;, can produce fairly accurate text even with some of the above issues in the document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the documents have noise gradients, they may be difficult to OCR. That is different than a value or intensity gradient, which can be normalized. Using AI to remove noise gradients from documents is a reasonable plan, but there is a limit to any kind of intelligence. A scrambled egg cannot be reconstituted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple test of feasibility is whether a human that has no domain knowledge about the subject of the document can make out the text. If they cannot, then a computer will not be able to either, unless the vocabulary is labeled with the probability of each word appearing in that subject matter and the OCR system is trained that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the noise is on a gray scale so that the darkness of the type is greater than one or two standard deviations of the noise distribution, then the document can be divided by a parameterizable grid with parameterizable brightness and contrast. This can become the model used in conjunction with a basic feed forward network (mini-batched multilayer perceptron) and the OCR software. The idea in this architecture would be to converge on a network behavior that, for the documents in the example set that have been hand adjusted so that the OCR produces good output, could then be used for many more documents with similar noise gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inputs for the artificial network would be statistics about the value distribution for specific sections on a specific page of the document and the labels will be the manually created instructions for gradient based contract and value correction of the document so the OCR proceeds with few errors. Once the network is trained, its output could ideally drive the gradient based corrections. It would probably work if three things are in order.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The noise doesn't completely destroy the availability of the text so that an average human could not make out the words.&lt;/li&gt;&#xA;&lt;li&gt;The training of the network converges to a degree that the OCR software works across the pages for most of the documents.&lt;/li&gt;&#xA;&lt;li&gt;The characteristics of the noise gradients remain consistent, meaning that the training examples are representative of the documents that will be processed later.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19168" LastActivityDate="2019-01-24T18:04:20.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10170" PostTypeId="2" ParentId="10163" CreationDate="2019-01-24T21:07:58.430" Score="1" Body="&lt;p&gt;Those all fit into a single quadratic, auto-correlated model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ x_0 = a \\ x_i = b i^2 + c x_{i-1} + d i + e $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequences can be curve fitted producing a set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; perfect fits of the form &lt;span class=&quot;math-container&quot;&gt;$(a, b, c, d, e)$&lt;/span&gt; given the above model. A rules engine given the correct parameterized rules can produce the most desirable verbal description from among the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; fits in the set. The rules can also be prioritized by a simple feed forward network trained to simulate the most natural selection of string descriptions from any set of fits where &lt;span class=&quot;math-container&quot;&gt;$n &amp;gt; 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will work well for the examples in the question and many more, however, if the sequence &lt;span class=&quot;math-container&quot;&gt;$\{1, 4, 1, 5, 9\}$&lt;/span&gt; is fed into the system, it will produce some weird description based on the quadratic, auto-correlated model it was given rather than, &quot;digits of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to the right of the decimal place.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to produce the most common response a university freshman math student would produce would be to extend the boundaries of AI engineering first. For example, once an AI system is developed that can handle natural language and cognition like a child, several of them can be separately trained in a simulation of primary and secondary school mathematics. The median response for each sequence given to the class of AI students (class made up of artificial students studying math, not class of humans studying AI) will then be a reasonable prediction of what human university freshmen would produce as a median response.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-24T21:07:58.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10171" PostTypeId="2" ParentId="10163" CreationDate="2019-01-25T00:18:40.880" Score="-1" Body="&lt;p&gt;This can be framed as a classification problem where a model is supervised on a dataset containing finite-length number sequences &lt;span class=&quot;math-container&quot;&gt;$x^{(i)}_1, \cdots, x^{(i)}_n$&lt;/span&gt; and the sequence name &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;. For example, the dataset could look like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;([0,1,2,3,4], 0)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;([1,3,5,7,9], 1)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;([1,1,2,3,5], 2)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;([1,4,9,16,25], 3)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where the numbers on the right are integer representations of the sequence name. Given intuitive sequences, plentiful training data, and training examples of reasonable length, this problem would not be that difficult to solve. Sequence models from deep learning, such as recurrent networks (LSTM, GRU) or temporal convolutional networks, are well-suited for tasks such as this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this is only possible within certain constraints. The models are only good at what they are trained to do, so it would be impossible to use them to infer whether a sequence skips by 2 or 3 without having that information explicitly present in the training data. It would be interesting to see whether unsupervised models could detect this sort of information, although I don't think this work has been done in the present.&lt;/p&gt;&#xA;" OwnerUserId="19403" LastActivityDate="2019-01-25T00:18:40.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10172" PostTypeId="1" CreationDate="2019-01-25T00:51:25.580" Score="3" ViewCount="53" Body="&lt;p&gt;Typical Feed Forward Neural Networks require a fixed sized input and output. So when you have variable sized input, it seems to be common practice to pad the input with zero vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does it not seem to be common practice to have a &quot;is_padding&quot; attribute?  That way the network can easily distinguish between padding and actual data?  Especially considering input is commonly centered around 0 by subtracting the mean and using unit variance.&lt;/p&gt;&#xA;" OwnerUserId="20338" LastActivityDate="2019-02-01T23:46:39.540" Title="Using a &quot;is_padding&quot; attribute in your padding instead of simply zero vectors" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10173" PostTypeId="5" CreationDate="2019-01-25T00:53:45.233" Score="0" Body="&lt;p&gt;Experimental tag, under the &quot;social&quot; umbrella.  Related to the &quot;mythology-of-ai&quot;, &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-01-25T01:01:18.687" LastActivityDate="2019-01-25T01:01:18.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10174" PostTypeId="4" CreationDate="2019-01-25T00:53:45.233" Score="0" Body="Use this tag for discussions of press coverage of Artificial Intelligence.  This primarily oriented toward how AI is covered in the media, but may include include fact checking publications (articles, blogs and popular non-fiction.)  Intended to be used in conjunction with the social impacts and industry of AI.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-01-25T01:01:18.687" LastActivityDate="2019-01-25T01:01:18.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10175" PostTypeId="1" CreationDate="2019-01-25T03:53:16.343" Score="2" ViewCount="425" Body="&lt;p&gt;I do understand that there are plenty of mobile apps available for body measurement (e.g. MTailor) or creating 3D model (3dlook).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to find out is how we can use deep learning to achieve the accurate body measurement/3D model with just smart phone camera?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, MTailor can predict one's body measurement quite accurately given the cameara angle/camera distance from the human and human height. Can we do the same using deep learning with some labeled images to achieve the same accurate body measurement prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&#xA;Han&lt;/p&gt;&#xA;" OwnerUserId="21710" LastActivityDate="2019-01-25T03:53:16.343" Title="Deep Learning on how to find out the body measurement (e.g. shoulder length, waist, hips, legs length etc) from mobile camera captured images?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10177" PostTypeId="1" AcceptedAnswerId="10221" CreationDate="2019-01-25T06:39:39.863" Score="2" ViewCount="1481" Body="&lt;p&gt;I'm quite new to the field of computer vision and was wondering what are the purposes of having the boundary boxes in object detection.&lt;/p&gt;&#xA;&lt;p&gt;Obviously, it shows where the detected object is, and using a classifier can only classify one object per image, but my question is that&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If I don't need to know 'where' an object is (or objects are) and just interested in the existence of them and how many there are, is it possible to just get rid of the boundary boxes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If not, how does bounding boxes help detect objects? From what I have figured is that a network (if using neural network architectures) predicts the coordinates of the bounding boxes if there is something in the feature map. Doesn't this mean that the detector already knows where the object is (at least briefly)? So, continuing from question 1, if I'm not interested in the exact location, would training for bounding boxes be irrelevant?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, in architectures like YOLO, it seems that they predict the probability of each class on each grid (e.g. 7 x 7 for YOLO v1). What would be the purpose of bounding boxes in this architecture other than that it shows exactly where the object is? Obviously, the class has already been predicted, so I'm guessing that it doesn't help classify better.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21712" LastEditorUserId="2444" LastEditDate="2021-01-28T23:52:41.970" LastActivityDate="2021-01-28T23:52:41.970" Title="What's the role of bounding boxes in object detection?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;yolo&gt;&lt;bounding-box&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10180" PostTypeId="1" CreationDate="2019-01-25T10:34:07.857" Score="7" ViewCount="2512" Body="&lt;p&gt;I'm studying &lt;em&gt;Artificial Intelligence. A Modern Approach&lt;/em&gt;, Stuart Russell, Peter Norvig, specifically about search and planning arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand the difference between the two terms. I was more confused when I saw that some search problems can be determined in planning way. My professor explained to me in a confusing way that the real difference on the search is that it uses an heuristic function, but my book says that planning use a heuristic too, for relaxing problem (in cap. 10.2.3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read &lt;a href=&quot;https://stackoverflow.com/questions/10282476/what-is-the-difference-between-search-and-planning&quot;&gt;this page&lt;/a&gt; that says in a certain way what I'm saying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is planning and search the problem? If not, what are the differences and how are these problems related?&lt;/p&gt;&#xA;" OwnerUserId="21719" LastEditorUserId="2444" LastEditDate="2019-01-28T12:56:17.467" LastActivityDate="2021-04-18T17:54:14.397" Title="What is the difference between search and planning?" Tags="&lt;terminology&gt;&lt;search&gt;&lt;definitions&gt;&lt;planning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10183" PostTypeId="2" ParentId="10043" CreationDate="2019-01-25T11:23:16.290" Score="1" Body="&lt;p&gt;Not just any NLP+ML will qualify as an excellent choice for a masters thesis. It would be wise to focus on long term needs that won't easily go away. Some research directions are likely to be obsolete in the time it takes to finish a masters. Others will be so well covered by thousands of others interested in the highly publicized combination of NLP+ML, that there is no real advantage in choosing that direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reaching further than simply NPL+ML is the better bet, but it will take significant ability and high motivation. The largest gap that is likely to remain a gap for several years is the depth of comprehension on the input side of machine based natural language. The shallow and domain ignorant comprehension of chatbots is a well known stumbling block to enterprise and consumer use.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Speech comprehension is by far the most valuable challenge before us as researchers and the most highly usable type of machine based natural language handling for the 2030s.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These are the current trends indicating the long term value of the above.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Tacotron 2 and other approaches are well on the way to authentic sounding speech synthesis, so that educational direction will be in less demand.&lt;/li&gt;&#xA;&lt;li&gt;The trend in communications was transitioning, in the first decade of this century, from voice to text. That trend is reversing now, and the trend is very likely to continue to be from text back to voice in the third decade of this century. There are neurological and genetic reasons why that is the case. Voice communication is more deeply embedded in humanity. It appeared much earlier in human history and develops much sooner in children, and those things cannot be changed without changing human DNA.&lt;/li&gt;&#xA;&lt;li&gt;Machine translation is limited by the level of comprehension achievable from natural language input.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Eliminating voice synthesis and text generation from semantic structure, which is difficult to do until the semantic structures to use for that research have been collected into data sets, you have speech comprehension left.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In summary, machine listening provides value far beyond machine hearing, just as human listening provides value far beyond human hearing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To the advantage of this comprehension research there are these veins of knowledge.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The mechanics of the ear and first level of neural processing is well mapped. It involves the cochlear geometry which can be simulated using standard windowing techniques with an FFT followed by the RMS spectrum. The time series of spectra can then be mapped to phonetics.&lt;/li&gt;&#xA;&lt;li&gt;Cognitive science is developing quickly and on scientific basis more deeply than during the age of psychology detached from data. Now there are is genetic research, bioinformatics, neural imaging supporting the understanding of cognition and semantic structures that cognition involves.&lt;/li&gt;&#xA;&lt;li&gt;A more mature field of linguistics has changed the understanding of language from formal grammatical structure (with parts of speech and allowed sentence structures) to the natural speech of real people, which involves word roots, prefix and suffix conventions, conjugation patterns, local language evolution, borrowing of foreign words, colloquialism, and use of accent and intonation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The right person, seeking an M.S. and highly motivated to bring machine based natural language handling to greater depths, the following thesis features would be of great value in the coming decade.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input 1: Multi-voice discussion&lt;/li&gt;&#xA;&lt;li&gt;Input 2: A directed graph representing a semantic net of associations, compositions, and generalizations &lt;/li&gt;&#xA;&lt;li&gt;Output: An updated directed graph indicating the decoration of Input 2 with the semantics contained in the voices of the discussion in Input 1.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Do anything like that, and the opportunities will mushroom faster than cell phones did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not difficult to find work along the lines of the above, but it is in its infancy in some ways, which is why its a great opportunity to get on board early. The ways in which machine listening and comprehension can enhance human experience with computers and further the root field of cybernetics is too numerous to list. Here are a small few of the many research papers in this general direction.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1611.01423&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning continuous semantic representations of symbolic expressions&lt;/em&gt;&lt;/a&gt;, M. Allamanis, P. Chanthirasegaran, P. Kohli, C. Sutton, 2017&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.jair.org/index.php/jair/article/download/11196/26408/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure&lt;/em&gt;&lt;/a&gt;, Dieuwke Hupkes, Sara Veldhoen, Willem Zuidema, 2017&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1803.08869.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;On the difficulty of a distributional semantics of spoken language&lt;/em&gt;&lt;/a&gt;, Grzegorz Chrupała, Lieke Gelderloos, Akos Kadar, Afra Alishahi, 2018&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-25T11:23:16.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10184" PostTypeId="1" AcceptedAnswerId="10187" CreationDate="2019-01-25T11:51:07.573" Score="2" ViewCount="52" Body="&lt;p&gt;Let's propose, that I can define the state of a board in a board game, with 234 neurons. In theory, could I be able to train a neural network, with 468 inputs (two game boards), and 1 output, to tell me which board state is 'better'? The output should give me ~-1 if the second board is better than the first, ~0 if they are equal, and ~1 if the first board is better than the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, what could be the number of ideal neurons on the hidden layers? What could be the ideal number of hidden layers?&lt;/p&gt;&#xA;" OwnerUserId="21700" LastActivityDate="2019-01-25T13:09:38.863" Title="Could a neural network be capable to diferentiate between two boards of a game?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10185" PostTypeId="2" ParentId="10158" CreationDate="2019-01-25T12:31:58.820" Score="1" Body="&lt;p&gt;People often place a batchnorm layer before ReLU. That effectively prevents the problem you have described. &lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-01-25T12:31:58.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10186" PostTypeId="2" ParentId="10177" CreationDate="2019-01-25T12:52:43.070" Score="-1" Body="&lt;p&gt;In principle, you could train the model to output a sigmoid map of coarse object positions (0 -&gt; no object, 1 -&gt; an object center is located here). The map could be subjected to non-maximum suppression and such model could be trained end-to-end. That would be possible, if that's what you are asking.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-01-25T12:52:43.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10187" PostTypeId="2" ParentId="10184" CreationDate="2019-01-25T13:09:38.863" Score="1" Body="&lt;p&gt;For optimal performance, the network complexity should fit the complexity of the game. Since we do not know the latter, your question is not answerable. &lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-01-25T13:09:38.863" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10189" PostTypeId="1" CreationDate="2019-01-25T14:03:41.367" Score="1" ViewCount="132" Body="&lt;p&gt;Currently, we can build the &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence (AI)&lt;/a&gt; approaches that respectively explain their actions within the use of goal trees &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/a.hunter/papers/aaai17.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;. By moving up and down across the tree, it keeps tracking the last and next movements. Therefore, giving the ability to the machine for &quot;explain&quot; the actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explainability regarding human levels, requires some cognitive effort, such as self-awareness, memory retrieval, a theory of mind and so on &lt;a href=&quot;https://arxiv.org/abs/1706.07269&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;. Humans are adept at selecting several causes from an infinite number of causes to be the explanation. However, this selection is influenced by certain cognitive biases. The idea of explanation selection is not new in &lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;eXplainable Artificial Intelligence (XAI)&lt;/a&gt; [3, 4]. But, as far as we are aware, there are currently no studies that look at the cognitive biases of humans as a way to select explanations from a set of causes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite a clear definition and description of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;XAI&lt;/a&gt; field, several questions remain present. The issues are summarized in just one sentence and listed as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, our question is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can we create and build &lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;XAI&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/a.hunter/papers/aaai17.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; Hadoux, Emmanuel, and Anthony Hunter. &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/a.hunter/papers/aaai17.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strategic Sequences of Arguments for Persuasion Using Decision Trees&lt;/a&gt;. AAAI. 2017.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1706.07269&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; Miller, T., 2018. &lt;a href=&quot;https://arxiv.org/abs/1706.07269&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explanation in artificial intelligence: Insights from the social sciences&lt;/a&gt;. Artificial Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.darpa.mil/attachments/XAIProgramUpdate.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt; Gunning, D., 2017. &lt;a href=&quot;https://www.darpa.mil/attachments/XAIProgramUpdate.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explainable artificial intelligence (XAI)&lt;/a&gt;. Defense Advanced Research Projects Agency (DARPA), nd Web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1708.08296&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt; Samek, W., Wiegand, T. and Müller, K.R., 2017. &lt;a href=&quot;https://arxiv.org/abs/1708.08296&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models&lt;/a&gt;. arXiv preprint arXiv:1708.08296.&lt;/p&gt;&#xA;" OwnerUserId="21729" LastEditorUserId="2444" LastEditDate="2019-09-14T12:56:50.670" LastActivityDate="2019-09-14T12:56:50.670" Title="How can we create eXplainable Artificial Intelligence?" Tags="&lt;philosophy&gt;&lt;cognitive-science&gt;&lt;explainable-ai&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ClosedDate="2020-12-27T14:51:44.693" ContentLicense="CC BY-SA 4.0" />
  <row Id="10191" PostTypeId="1" CreationDate="2019-01-25T14:37:37.883" Score="2" ViewCount="64" Body="&lt;p&gt;I am a member of a robotics team that is measuring the amount of reflected IR light to determine the lightness/darkness of a given material. We eventually hope to be able to use this to follow a line using a pre-set algorithm, but the first step is determining whether the material is one of the binary options: light or dark.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a large population of values between 0 and 1023, probably in two distinct groupings, how can I best go about classifying a given point as light or dark?&lt;/p&gt;&#xA;" OwnerUserId="20818" LastEditorUserId="2444" LastEditDate="2019-02-25T15:46:14.077" LastActivityDate="2019-03-27T16:20:38.333" Title="How do I classify measurements into only two classes?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10192" PostTypeId="2" ParentId="9990" CreationDate="2019-01-25T14:40:13.367" Score="1" Body="&lt;p&gt;The question is about a mismatch between the loss function in two papers on GANs. The first paper is &lt;a href=&quot;https://arxiv.org/pdf/1406.2661.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Generative Adversarial Nets&lt;/em&gt;&lt;/a&gt; Ian J. Goodfellow et. al., 2014, and the excerpt image in the question is this.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The adversarial modeling framework is most straightforward to apply when the models are both multilayer perceptrons. To learn the generator’s distribution &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; over data &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, we define a prior on input noise variables &lt;span class=&quot;math-container&quot;&gt;$p_z (z)$&lt;/span&gt;, then represent a mapping to data space as &lt;span class=&quot;math-container&quot;&gt;$G (z; \theta_g)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is a differentiable function represented by a multilayer perceptron with parameters &lt;span class=&quot;math-container&quot;&gt;$\theta_g$&lt;/span&gt;. We also define a second multilayer perceptron &lt;span class=&quot;math-container&quot;&gt;$D (x; \theta_d)$&lt;/span&gt; that outputs a single scalar. &lt;span class=&quot;math-container&quot;&gt;$D (x)$&lt;/span&gt; represents the probability that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; came from the data rather than pg. We train &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to maximize the probability of assigning the correct label to both training examples and samples from &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. We simultaneously train &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; to minimize &lt;span class=&quot;math-container&quot;&gt;$\log (1 − D(G(z)))$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;In other words, &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; play the following two-player minimax game with value function &lt;span class=&quot;math-container&quot;&gt;$V (G, D)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \min_G \, \max_D V (D, G) = \mathbb{E}_{x∼p_{data}(x)} \, [\log \, D(x)] \\&#xA;\quad\quad\quad\quad\quad\quad\quad  + \, \mathbb{E}_{z∼p_z(z)} \, [\log \, (1 − D(G(z)))] \, \text{.} \quad \text{(1)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The second paper is &lt;a href=&quot;https://arxiv.org/pdf/1611.07004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Image-to-Image Translation with Conditional Adversarial Networks&lt;/em&gt;&lt;/a&gt;, Phillip Isola Jun-Yan Zhu Tinghui Zhou Alexei A. Efros, 2018, and the excerpt image in the question is this.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The objective of a conditional GAN can be expressed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{L}_{cGAN} (G, D) = \mathbb{E}_{x, y} \, [\log D(x, y)] \\ \quad\quad\quad\quad\quad\quad\quad + \mathbb{E}_{x, z} \, [\log \, (1 − D(x, G(x, z))], \quad \text{(1)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; tries to minimize this objective against an adversarial &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; that tries to maximize it, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ G^{∗} = \arg \, \min_G \, \max_D \mathcal{L}_{cGAN} (G, D) \, \text{.} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;To test the importance of conditioning the discriminator, we also compare to an unconditional variant in which the discriminator does not observe &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{L}_{GAN} (G, D) = \mathbb{E}_y \, [\log \, D(y)] \\&#xA;\quad\quad\quad\quad\quad\quad\quad + \mathbb{E}_{x, z} \, [\log \, (1 − D(G(x, z))] \, \text{.} \quad \text{(2)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the above &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; refers to the generative network, &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; refers to the discriminative network, and &lt;span class=&quot;math-container&quot;&gt;$G^{*}$&lt;/span&gt; refers to the minimum with respect to &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; of the maximum with respect to &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. As the question author tentatively put forward, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}$&lt;/span&gt; is the expectation with respect to its subscripts.&lt;/p&gt;&#xA;&lt;p&gt;The question of discrepancy is that the right hand sides do not match between the first paper's equation (1) and the second paper's equation (2) which is absent of the condition involving &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;First paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathbb{E}_{x∼p_{data}(x)} \, [\log \, D(x)] \\&#xA;\quad\quad\quad\quad\quad\quad\quad  + \, \mathbb{E}_{z∼p_z(z)} \, [\log \, (1 − D(G(z)))] \, \text{.} \quad \text{(1)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Second paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathbb{E}_y \, [\log \, D(y)] \\&#xA;\quad\quad\quad\quad\quad\quad\quad + \mathbb{E}_{x, z} \, [\log \, (1 − D(G(x, z))] \, \text{.} \quad \text{(2)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The second later paper further states this.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GANs are generative models that learn a mapping from random noise vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; to output image &lt;span class=&quot;math-container&quot;&gt;$y, G : z \rightarrow y$&lt;/span&gt;. In contrast, conditional GANs learn a mapping from observed image &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and random noise vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, to &lt;span class=&quot;math-container&quot;&gt;$y, G : {x, z} \rightarrow y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Notice that there is no &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the first paper and the removal of the condition in the second paper corresponds to the removal of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; as the first parameter of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. This is one of the causes of confusion when comparing the right hand sides. The others are use of variables and degree of explicitness in notation.&lt;/p&gt;&#xA;&lt;p&gt;The tilda &lt;span class=&quot;math-container&quot;&gt;$~$&lt;/span&gt; means &lt;em&gt;drawn according to&lt;/em&gt;. The right hand side in the first paper indicates that the expectation involving &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is based on a drawing according to the probability distribution of the data with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and the expectation involving &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is based on a drawing according to the probability distribution of &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The removal of the observation of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from the second right hand term of the second paper's equation (2), which is the first parameter of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, the replacement of that equation's &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; variable with the now freed up &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; variable, and the acceptance of the abbreviation of the tilda notation used in the first paper then brings both papers into exact agreement.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-25T14:40:13.367" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10194" PostTypeId="1" CreationDate="2018-12-15T17:06:55.313" Score="2" ViewCount="529" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch's example for the REINFORCE algorithm&lt;/a&gt; for reinforcement learning has the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;import gym&#xA;import numpy as np&#xA;from itertools import count&#xA;&#xA;import torch&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;import torch.optim as optim&#xA;from torch.distributions import Categorical&#xA;&#xA;&#xA;parser = argparse.ArgumentParser(description='PyTorch REINFORCE example')&#xA;parser.add_argument('--gamma', type=float, default=0.99, metavar='G',&#xA;                    help='discount factor (default: 0.99)')&#xA;parser.add_argument('--seed', type=int, default=543, metavar='N',&#xA;                    help='random seed (default: 543)')&#xA;parser.add_argument('--render', action='store_true',&#xA;                    help='render the environment')&#xA;parser.add_argument('--log-interval', type=int, default=10, metavar='N',&#xA;                    help='interval between training status logs (default: 10)')&#xA;args = parser.parse_args()&#xA;&#xA;&#xA;env = gym.make('CartPole-v0')&#xA;env.seed(args.seed)&#xA;torch.manual_seed(args.seed)&#xA;&#xA;&#xA;class Policy(nn.Module):&#xA;    def __init__(self):&#xA;        super(Policy, self).__init__()&#xA;        self.affine1 = nn.Linear(4, 128)&#xA;        self.affine2 = nn.Linear(128, 2)&#xA;&#xA;        self.saved_log_probs = []&#xA;        self.rewards = []&#xA;&#xA;    def forward(self, x):&#xA;        x = F.relu(self.affine1(x))&#xA;        action_scores = self.affine2(x)&#xA;        return F.softmax(action_scores, dim=1)&#xA;&#xA;&#xA;policy = Policy()&#xA;optimizer = optim.Adam(policy.parameters(), lr=1e-2)&#xA;eps = np.finfo(np.float32).eps.item()&#xA;&#xA;&#xA;def select_action(state):&#xA;    state = torch.from_numpy(state).float().unsqueeze(0)&#xA;    probs = policy(state)&#xA;    m = Categorical(probs)&#xA;    action = m.sample()&#xA;    policy.saved_log_probs.append(m.log_prob(action))&#xA;    return action.item()&#xA;&#xA;&#xA;def finish_episode():&#xA;    R = 0&#xA;    policy_loss = []&#xA;    rewards = []&#xA;    for r in policy.rewards[::-1]:&#xA;        R = r + args.gamma * R&#xA;        rewards.insert(0, R)&#xA;    rewards = torch.tensor(rewards)&#xA;    rewards = (rewards - rewards.mean()) / (rewards.std() + eps)&#xA;    for log_prob, reward in zip(policy.saved_log_probs, rewards):&#xA;        policy_loss.append(-log_prob * reward)&#xA;    optimizer.zero_grad()&#xA;    policy_loss = torch.cat(policy_loss).sum()&#xA;    policy_loss.backward()&#xA;    optimizer.step()&#xA;    del policy.rewards[:]&#xA;    del policy.saved_log_probs[:]&#xA;&#xA;&#xA;def main():&#xA;    running_reward = 10&#xA;    for i_episode in count(1):&#xA;        state = env.reset()&#xA;        for t in range(10000):  # Don't infinite loop while learning&#xA;            action = select_action(state)&#xA;            state, reward, done, _ = env.step(action)&#xA;            if args.render:&#xA;                env.render()&#xA;            policy.rewards.append(reward)&#xA;            if done:&#xA;                break&#xA;&#xA;        running_reward = running_reward * 0.99 + t * 0.01&#xA;        finish_episode()&#xA;        if i_episode % args.log_interval == 0:&#xA;            print('Episode {}\tLast length: {:5d}\tAverage length: {:.2f}'.format(&#xA;                i_episode, t, running_reward))&#xA;        if running_reward &amp;gt; env.spec.reward_threshold:&#xA;            print(&quot;Solved! Running reward is now {} and &quot;&#xA;                  &quot;the last episode runs to {} time steps!&quot;.format(running_reward, t))&#xA;            break&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am interested in the function &lt;code&gt;finish_episode()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; rewards = (rewards - rewards.mean()) / (rewards.std() + eps)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;makes no sense to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought this might be &lt;a href=&quot;https://www.quora.com/Why-can-reinforcement-of-the-baseline-reduce-variance&quot; rel=&quot;nofollow noreferrer&quot;&gt;baseline reduction&lt;/a&gt;, but I can't see why divide by the standard deviation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it isn't baseline reduction, then why normalize the rewards, and where should the baseline reduction go?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please explain that line&lt;/p&gt;&#xA;" OwnerUserId="21645" OwnerDisplayName="Gulzar" LastEditorUserId="2444" LastEditDate="2020-11-22T01:04:53.233" LastActivityDate="2020-11-22T01:04:53.233" Title="Why is the reward signal normalized in openAI's REINFORCE?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;reward-normalization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-11-22T01:02:25.293" ContentLicense="CC BY-SA 4.0" />
  <row Id="10195" PostTypeId="2" ParentId="10194" CreationDate="2018-12-18T02:12:46.213" Score="3" Body="&lt;p&gt;It's just a hack to normalize the rewards to make optimization less problematic in cases where the reward might have very large/small values. It's fine because scaling the rewards by a constant factor won't change what the optimal policy is. You can think of the mean reward as the baseline. &lt;/p&gt;&#xA;" OwnerUserId="17971" OwnerDisplayName="shimao" LastActivityDate="2018-12-18T02:12:46.213" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10196" PostTypeId="1" AcceptedAnswerId="10204" CreationDate="2019-01-24T13:56:08.333" Score="12" ViewCount="5217" Body="&lt;p&gt;In &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/actor_critic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open AI's actor-critic&lt;/a&gt; and in &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open AI's REINFORCE&lt;/a&gt;, the rewards are being normalized like so&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rewards = (rewards - rewards.mean()) / (rewards.std() + eps)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;on every episode individually&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is probably the &lt;a href=&quot;https://qr.ae/TWGVk2&quot; rel=&quot;nofollow noreferrer&quot;&gt;baseline reduction&lt;/a&gt;, but &lt;a href=&quot;https://ai.stackexchange.com/q/10194/2444&quot;&gt;I'm not entirely sure why they divide by the standard deviation of the rewards&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Assuming this is the baseline reduction, why is this done &lt;em&gt;per episode&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;What if one episode yields rewards in the (absolute, not normalized) range of &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, and the next episode yields rewards in the range of &lt;span class=&quot;math-container&quot;&gt;$[100, 200]$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;This method seems to ignore the absolute difference between the episodes' rewards.&lt;/p&gt;&#xA;" OwnerUserId="21645" OwnerDisplayName="Gulzar" LastEditorUserId="2444" LastEditDate="2020-11-22T00:35:24.887" LastActivityDate="2020-11-22T00:35:24.887" Title="Why does is make sense to normalize rewards per episode in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;variance-reduction&gt;&lt;reward-normalization&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10197" PostTypeId="2" ParentId="9975" CreationDate="2019-01-25T16:10:09.703" Score="3" Body="&lt;p&gt;Consider a continuum of complexity in models.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Trivial:&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$y = x + a$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Simple:&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$y = x \, \log \, (a x + b) + c$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Moderately complex:&lt;/strong&gt; A wind turbine under constant wind velocity&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Very complex:&lt;/strong&gt; Ray tracing of lit 3-D motion scenes to pixels&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Astronomically complex:&lt;/strong&gt; The weather&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now consider a continuum regarding the generality or specificity of models.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Very specific:&lt;/strong&gt; The robot for the Mars mission has an exact mechanical topology, materials call-out, and set of mechanical coordinates contained in the CAD files used to machine the robot's parts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Somewhat specific:&lt;/strong&gt; The formulas guiding the design of an internal combustion engine, which are well known.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Somewhat general:&lt;/strong&gt; The phenomenon is deterministic and the variables and their domains are known.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Very general:&lt;/strong&gt; There's probably some model because it works in nature but we know little more.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are twenty permutations at the above level of granularity. Every one has purpose in mathematical analysis, applied research, engineering, and monetization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some general correlations between input, output, and layer counts.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Higher complexity often corresponds to larger layer count.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Higher i/o dimensionality corresponds to higher width to the corresponding i/o layers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mapping generality to or from specificity generally requires complexity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, to make this answer even less appealing to those who want formula answer they can memorize, ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Each artificial network is a model of an arbitrary function before training and a model of a specific function afterward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Loss functions are models of disparity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An algorithm is a model of a process created by spreading a recursive definition out in time to map into a model of centralized computation called a CPU.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The recursive definition is a model too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is almost nothing in science that is a not a model except ideas or data that are not yet modeled.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-25T18:44:10.077" LastActivityDate="2019-01-25T18:44:10.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10198" PostTypeId="1" CreationDate="2019-01-25T16:39:28.503" Score="0" ViewCount="126" Body="&lt;p&gt;Following &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/actor_critic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch's actor critic&lt;/a&gt;, I understand that the critic is a function mapping from the &lt;strong&gt;state&lt;/strong&gt; space to the reward space, meaning, the critic approximates the &lt;strong&gt;state-value funcion&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, according to &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwia87rurYngAhUH36QKHd5QDCIQFjAAegQIChAC&amp;amp;url=https%3A%2F%2Farxiv.org%2Fpdf%2F1512.07679&amp;amp;usg=AOvVaw23yfpdaGmo9gs3_AIz8edQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; (you don't need to read it, just a glance at the nice picture at page 2 is enough), the critic is a function mapping from the &lt;strong&gt;action&lt;/strong&gt; space to the reward, meaning it approximates the &lt;strong&gt;action value funcion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When people say &quot;actor critic&quot; - what do they mean by &quot;critic&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the term &quot;Critic&quot; ambiguous in RL?&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="21645" LastEditDate="2019-02-07T18:54:32.583" LastActivityDate="2019-02-07T18:54:32.583" Title="What does the Critic network evaluate in Actor Critic?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10199" PostTypeId="2" ParentId="10198" CreationDate="2019-01-25T17:14:27.773" Score="2" Body="&lt;p&gt;The same argument of a state-value function int &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; Section 13.5 can be applied to state-action values. The main takeaway is that a critic's state value (or action-state value) function is used for bootstrapping.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Although the REINFORCE-with-baseline method learns both a policy and a state-value function, we do not consider it to be an actor–critic method because its state-value function is used only as a baseline, not as a critic. That is, it is not used for bootstrapping (updating the value estimate for a state from the estimated values of subsequent states), but only as a baseline for the state whose estimate is being updated. This is a useful distinction, for only through bootstrapping do we introduce bias and an asymptotic dependence on the quality of the function approximation. As we have seen, the bias introduced through bootstrapping and reliance on the state representation is often beneficial because it reduces variance and accelerates learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-01-25T17:14:27.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10200" PostTypeId="2" ParentId="10196" CreationDate="2019-01-25T17:33:19.530" Score="4" Body="&lt;p&gt;We subtract mean from values and divide it with standard deviation to get data with mean of zero and variance of one. The range of values per episode does not matter, it will always make it to have zero mean and variance of one in all cases. If the range is bigger ([100, 200]) then deviation will be bigger as well than for smaller range ([0, 1]) so we will end up dividing by bigger number.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-01-25T17:33:19.530" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10201" PostTypeId="1" AcceptedAnswerId="10887" CreationDate="2019-01-25T17:45:23.587" Score="1" ViewCount="37" Body="&lt;p&gt;If one examines SSD: Single Shot MultiBox Detector code from &lt;a href=&quot;https://github.com/weiliu89/caffe/tree/ssd&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub repository&lt;/a&gt;, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter &lt;code&gt;test batch size&lt;/code&gt;. It is not mentioned in the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with using batches during  network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?&lt;/p&gt;&#xA;" OwnerUserId="21737" LastActivityDate="2019-02-26T07:37:24.623" Title="Using batches in testing" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10202" PostTypeId="2" ParentId="9975" CreationDate="2019-01-25T17:50:15.257" Score="1" Body="&lt;p&gt;In what context are you asking this? It is totally different if you want to perform object detection, regression or, for example, reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first case I would say that main point in using simple vs complex model is size of training data. If you have 1000 training samples you can't expect large network to perform better than simple one.&lt;/p&gt;&#xA;" OwnerUserId="21737" LastActivityDate="2019-01-25T17:50:15.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10203" PostTypeId="1" CreationDate="2019-01-25T19:40:24.863" Score="3" ViewCount="1630" Body="&lt;p&gt;I am in the process of implementing the DQN model from scratch in PyTorch with the target environment of Atari Pong. After a while of tweaking hyper-parameters, I cannot seem to get the model to achieve the performance that is reported in most publications (~ +21 reward; meaning that the agent wins almost every volley). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My most recent results are shown in the following figure. Note that the x axis is episodes (full games to 21), but the total training iterations is ~6.7 million.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3TAEJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3TAEJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specifics of my setup are as follows:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Model&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class DQN(nn.Module):&#xA;    def __init__(self, in_channels, outputs):&#xA;        super(DQN, self).__init__()&#xA;        self.conv1 = nn.Conv2d(in_channels=in_channels, out_channels=32, kernel_size=8, stride=4)&#xA;        self.conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=4, stride=2)&#xA;        self.conv3 = nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, stride=1)&#xA;        self.fc1 = nn.Linear(in_features=64*7*7 , out_features=512)&#xA;        self.fc2 = nn.Linear(in_features=512, out_features=outputs)&#xA;&#xA;    def forward(self, x):&#xA;        x = F.relu(self.conv1(x))&#xA;        x = F.relu(self.conv2(x))&#xA;        x = F.relu(self.conv3(x))&#xA;        x = x.view(-1, 64 * 7 * 7)&#xA;        x = F.relu(self.fc1(x))&#xA;        x = self.fc2(x)&#xA;        return x    # return Q values of each action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Hyperparameters&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;batch size: 32&lt;/li&gt;&#xA;&lt;li&gt;replay memory size: 100000&lt;/li&gt;&#xA;&lt;li&gt;initial epsilon: 1.0&lt;/li&gt;&#xA;&lt;li&gt;epsilon anneals linearly to 0.02 over 100000 steps&lt;/li&gt;&#xA;&lt;li&gt;random warmstart episodes: ~50000&lt;/li&gt;&#xA;&lt;li&gt;update target model every: 1000 steps&lt;/li&gt;&#xA;&lt;li&gt;optimizer = optim.RMSprop(policy_net.parameters(),  lr=0.0025, alpha=0.9, eps=1e-02, momentum=0.0)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Additional info&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;OpenAI gym Pong-v0 environment&lt;/li&gt;&#xA;&lt;li&gt;Feeding model stacks of 4 last observed frames, scaled and cropped to 84x84 such that only the &quot;playing area&quot; is visible.&lt;/li&gt;&#xA;&lt;li&gt;Treat losing a volley (end-of-life) as a terminal state in the replay buffer.&lt;/li&gt;&#xA;&lt;li&gt;Using &lt;a href=&quot;https://pytorch.org/docs/stable/nn.html#smooth-l1-loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;smooth_l1_loss&lt;/a&gt;, which acts as Huber loss&lt;/li&gt;&#xA;&lt;li&gt;Clipping gradients between -1 and 1 before optimizing&lt;/li&gt;&#xA;&lt;li&gt;I offset the beginning of each episode with 4-30 no-op steps as the papers suggest&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Has anyone had a similar experience of getting stuck around 6 - 9 average reward per episode like this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions for changes to hyperparameters or algorithmic nuances would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="19789" LastActivityDate="2019-02-10T04:04:36.467" Title="DQN stuck at suboptimal policy in Atari Pong task" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;python&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10204" PostTypeId="2" ParentId="10196" CreationDate="2019-01-25T19:40:45.833" Score="7" Body="&lt;p&gt;The &quot;trick&quot; of subtracting a (state-dependent) baseline from the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; term in policy gradients to reduce variants (which is what is described in your &quot;baseline reduction&quot; link) is a different trick from the modifications to the rewards that you are asking about. The baseline subtraction trick for variance reduction does not appear to be present in the code you linked to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing that your question is about appears to be standardization of rewards, as described in &lt;a href=&quot;https://ai.stackexchange.com/a/10200/1641&quot;&gt;Brale_'s answer&lt;/a&gt;, to put all the observed rewards in a similar range of values. Such a standardization procedure inherently requires division by the standard deviation, so... that answers that part of your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for why they are doing this on a per-episode-basis... I think you're right, &lt;strong&gt;in the general case&lt;/strong&gt; this seems like a bad idea. If there are rare events with extremely high rewards that only occur in some episodes, and the majority of episodes only experience common events with lower-scale rewards... yes, this trick will likely mess up training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In the specific case of the CartPole environment&lt;/strong&gt; (which is the only environment used in these two examples), this is not a concern. In this implementation of the CartPole environment, the agent simply receives a reward with a value of exactly &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for every single time step in which it manages to &quot;survive&quot;. The &lt;code&gt;rewards&lt;/code&gt; list in the example code is in my opinion poorly-named, because it actually contains &lt;strong&gt;discounted returns&lt;/strong&gt; for the different time-steps, which look like: &lt;span class=&quot;math-container&quot;&gt;$G_T = \sum_{t=0}^{T} \gamma^t R_t$&lt;/span&gt;, where all the individual &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; values are equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; &lt;strong&gt;in this particular environment&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These kinds of values tend to be in a fairly consistent range (especially if the policy used to generate them also only moves slowly), so the standardization that they do may be relatively safe, and may improve learning stability and/or speed (by making sure there are always roughly as many actions for which the probability gets increased as there are actions for which the probability gets decreased, and possibly by making hyperparameters easier to tune).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does not seem to me like this trick would generalize well to many other environments, and personally I think it shouldn't be included in such a tutorial / example.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: I'm quite sure that a per-episode subtraction of the mean returns would be a valid, albeit possibly unusual, &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html#baselines-in-policy-gradients&quot; rel=&quot;noreferrer&quot;&gt;baseline for variance reduction&lt;/a&gt;. It's the subsequent division by standard deviation that seems particularly problematic to me in terms of generalizing to many different environments. &lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-01-26T12:55:08.460" LastActivityDate="2019-01-26T12:55:08.460" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="10205" PostTypeId="2" ParentId="9969" CreationDate="2019-01-25T21:30:08.020" Score="0" Body="&lt;p&gt;Separating spam and ham emails based on text using the ID3 algorithm and the &lt;a href=&quot;http://nlp.cs.aueb.gr/software_and_datasets/Enron-Spam/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enron-Spam&lt;/a&gt;, &lt;a href=&quot;http://nlp.cs.aueb.gr/software_and_datasets/lingspam_public.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ling-Spam&lt;/a&gt;, &lt;a href=&quot;http://nlp.cs.aueb.gr/software_and_datasets/PU123ACorpora.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;PU&lt;/a&gt; data sets is possible but has limits in terms of usability. Let's first get through the questions in the body of this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initial Answers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I begin with this implementation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Begin by studying the training sets and the ID3 algorithm described below. Then find all the implementations and compare them on the basis of what is important for the project, which may include programming language, comparative tests of training speed, and whether examples in the download are close to the spam detection scenario.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Should I use a hash map to have the words I read from the text files and the times I read every single word, ... like HashMap or should I use two arrays maybe?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Using a hash map is a great way to establish a list of words and their frequency in the text, but word counting of the training example texts requires that the same be done when the training is applied in the field, which leaves an easy attack vector for the spammers. They can then use the word counts in the training sets to write spam that uses words common to ham and avoid words common to spam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Adaptive Requirements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using training sets to detect fraudulent or unwanted items has several caveats, including these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Spamers can use them to train GANs to write spam that gets classified as ham.&lt;/li&gt;&#xA;&lt;li&gt;The distinction between ham and spam is subjective, as in user dependent.&lt;/li&gt;&#xA;&lt;li&gt;The distinction between ham and spam is time dependent even within a single user.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Therefore, whether ID3, LSTM, Q-learning, or some other strategy is used, continuous sensitivity to what is happening in the spam world and how the user is reacting to automated spam detection decisions is imperative. Therefore two feedback channels are employed in most highly deployed commercial products and services.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Spam rule or fuzzy rule updates&lt;/li&gt;&#xA;&lt;li&gt;Adjustment of detection based on user exceptions indicated&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Seminal ID3 Papers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ID3 was introduced by J.R. Quinlan in &lt;em&gt;Discovering rules by induction from large collections of examples&lt;/em&gt;, Kluwer Academic 1979, not easily found on the web except in Japanese but referenced by &lt;a href=&quot;http://hunch.net/~coms-4771/quinlan.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Expert systems in the micro electronics age&lt;/em&gt;&lt;/a&gt;, also JR Quinlan, Kluwer Academic 1986. Since Springer acquired Kluwer Academic in 2004 but does not even provide a title listing for either of these as of January 2019, their rights with regard to this excellent work are clearly forfeit per pertinent U.S. and global IP law. This is section 4 of the second of the two public domain papers.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One approach to the induction task above would be to generate all possible decision trees that correctly classify the training set and to select the simplest of them. The number of such trees is finite but very large, so this approach would only be feasible for small induction tasks. ID3 was designed for the other end of the spectrum, where there are many attributes and the training set contains many objects, but where a reasonably good decision tree is required without much computation. It has generally been found to construct simple decision trees, but the approach it uses cannot guarantee that better trees have not been overlooked.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The basic structure of ID3 is iterative. A subset of the training set called the window is chosen at random and a decision tree formed from it; this tree correctly classifies all objects in the window. All other objects in the training set are then classified using the tree. If the tree gives the correct answer for all these objects then it is correct for the entire training set and the process terminates. If not, a selection of the incorrectly classified objects is added to the window and the process continues. In this way, correct decision trees have been found after only a few iterations for training sets of up to thirty thousand objects described in terms of up to 50 attributes. Empirical evidence suggests that a correct decision tree is usually found more quickly by this iterative method than by forming a tree directly from the entire training set. However, O'Keefe (1983) has noted that the iterative framework cannot be guaranteed to converge on a final tree unless the window can grow to include the entire training set. This potential limitation has not yet arisen in practice.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The crux of the problem is how to form a decision tree for an arbitrary collection &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; of objects. If &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is empty or contains only objects of one class, the simplest decision tree is just a leaf labelled with the class. Otherwise, let &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; be any test on an object with possible outcomes &lt;span class=&quot;math-container&quot;&gt;$O_1, O_2 .... O_w$&lt;/span&gt;. Each object in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; will give one of these outcomes for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; produces a partition &lt;span class=&quot;math-container&quot;&gt;$[C_1, C_2 .... C_w\}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; containing those objects having outcome &lt;span class=&quot;math-container&quot;&gt;$O_i$&lt;/span&gt;. This is represented graphically by the tree form of Figure 4. If each subset &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; in this figure could be replaced by a decision tree for &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;, the result would be a decision tree for all of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. Moreover~ so long as two or more &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;'s are nonempty, each &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is smaller than &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. In the worst case, this divide-and-conquer strategy will yield single-object subsets that satisfy the one-class requirement for a leaf. Thus, provided that a test can always be found that gives a non-trivial partition of any set of objects, this procedure will always produce a decision tree that correctly classifies each object in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The choice of test is crucial if the decision tree is to be simple. For the moment, a test will be restricted to branching on the values of an attribute, so choosing a test comes down to selecting an attribute for the root of the tree. The first induction programs in the ID series used a seat-of-the-pants evaluation function that worked reasonably well. Following a suggestion of Peter Gacs, ID3 adopted an information based method that depends on two assumptions. Let &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; contain &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; objects of class &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of class &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;. The assumptions are:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(1) Any correct decision tree for &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; will classify objects in the same proportion as their representation in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. An arbitrary object will be determined to belong to class &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$p / (p + n)$&lt;/span&gt; and to class &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$n / (p + n)$&lt;/span&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(2) When a decision tree is used to classify an object, it returns a class. A decision tree can thus be regarded as a source of a message 'P' or 'N', with the expected information needed to generate this message given by&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ I_{(p, n)} =&#xA;\, - \, \frac {p} {p + n} \log_2 \frac {p} {p + n}&#xA;\, - \, \frac {n} {p + n} \log_2 \frac {n} {p + n} $$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If attribute &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with values &lt;span class=&quot;math-container&quot;&gt;$[A_1, A_2, ... A_v]$&lt;/span&gt; is used for the root of the decision tree, -it will partition C into &lt;span class=&quot;math-container&quot;&gt;$[C_1, C_a, ... C_v]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; contains those objects in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; that have value &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; contain &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt; objects of class &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; of class &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;. The expected information required for the sub-tree for &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$I_{(p_i, n_i)}$&lt;/span&gt;. The expected information required for the tree with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; as root is then obtained as the weighted average&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ E(A) = \sum_{i=1}^{v} \frac {p_i + n_i} {p + n} I(p_i, n_i) $$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where the weight for the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; branch is the proportion of the objects in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; that belong to &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;. The information gained by branching on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is therefore&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \text{gain}(A) = I(p, n) - E(A) $$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A good rule of thumb would seem to be to choose that attribute to branch on which gains the most information3. ID3 examines all candidate attributes and chooses &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to maximize &lt;span class=&quot;math-container&quot;&gt;$\text{gain}(A)$&lt;/span&gt;, forms the tree as above, and then uses the same process recursively to form decision trees for the residual subsets &lt;span class=&quot;math-container&quot;&gt;$C_1, C_E, ... C_v$&lt;/span&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To illustrate the idea, let &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; be the set of objects in Table 1. Of the 14 objects, 9 are of class &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and 5 are of class &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, so the information required for classification is&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; $ I(p, n) =&#xA;  \, - \, \frac {9} {14} \, log_2 \, \frac {9} {14}&#xA;  \, - \, \frac {5} {14} \, log_2 \, \frac {5} {14}&#xA;  = \, 0.940 \, \text{bits} $$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Now consider the outlook attribute with values &lt;span class=&quot;math-container&quot;&gt;$[ \text{sunny}, \text{overcast}, \text{rain} \}$&lt;/span&gt;. Five of the 14 objects in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; have the first value (sunny), two of them from class &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and three from class &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, so&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;span class=&quot;math-container&quot;&gt;$ p_1 = 2 \quad n_1 = 3 \quad I(p_1, h_1) = 0.971$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;and similarly&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;span class=&quot;math-container&quot;&gt;$ P_2 = 4 \quad n_2 = 0 \quad I(p_2, n_2) = 0 \\&#xA;P_3 = 3 \quad n_3 = 2 \quad I(p_3, n_3) = 0.971 $&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The expected information requirement after testing this attribute is therefore&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;span class=&quot;math-container&quot;&gt;$ E (\text{outlook}) = \frac {5} {14} \, I(p_1, n_1) \,&#xA;+ \, \frac {4} {14} \, I(p_2, n_2) \,&#xA;+ \, \frac {5} {14} \, I(p_3, n_3) \,&#xA;= \, 0.694 \, \text{bits} $&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The gain of this attribute is then gain(outlook) = 0.940 - E(outlook) = 0.246 bits. Similar analysis gives gain(temperature) = 0.029 bits, gain(humidity) = 0.151 bits, and gain(windy) = 0.048 bits, so the tree-forming method used in ID3 would choose outlook as the attribute for the root of the decision tree. The objects would then be divided into subsets according to their values of the outlook attribute and a decision tree for each subset would be induced in a similar fashion. In fact, Figure 2 shows the actual decision tree generated by ID3 from this training set.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A special case arises if &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; contains no objects with some particular value &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, giving an empty &lt;span class=&quot;math-container&quot;&gt;$C_j$&lt;/span&gt;. ID3 labels such a leaf as 'null' so that it fails to classify any object arriving at that leaf. A better solution would generalize from the set &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; from which &lt;span class=&quot;math-container&quot;&gt;$C_j$&lt;/span&gt; came, and assign this leaf the more frequent class in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The worth of ID3's attribute-selecting heuristic can be assessed by the simplicity of the resulting decision trees, or, more to the point, by how well those trees express real relationships between class and attributes as demonstrated by the accuracy with which they classify objects other than those in the training set (their predictive accuracy). A straightforward method of assessing this predictive accuracy is to use only part of the given set of objects as a training set, and to check the resulting decision&#xA;  tree on the remainder.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Several experiments of this kind have been carried out. In one domain, 1.4 million Chess positions described in terms of 49 binary-valued attributes gave rise to 715 distinct objects divided 65%:35% between the classes. This domain is relatively complex since a correct decision tree for all 715 objects contains about 150 nodes. When training sets containing 20% of these 715 objects were chosen at random, they produced decision trees that correctly classified over 84% of the unseen objects. In another version of the same domain, 39 attributes gave 551 distinct objects with a correct decision tree of similar size; training sets of 20% of these 551 objects gave decision trees of almost identical accuracy. In a simpler domain (1,987 objects with a correct decision tree of 48 nodes), randomly-selected training sets containing 20% of the objects gave decision trees that correctly classified 98% of the unseen objects. In all three cases, it is clear that the decision trees reflect useful (as opposed to random) relationships present in the data.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This discussion of ID3 is rounded off by looking at the computational requirements of the procedure. At each non-leaf node of the decision tree, the gain of each untested attribute &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; must be determined. This gain in turn depends on the values &lt;span class=&quot;math-container&quot;&gt;$p_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; for each value &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, so every object in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; must be examined to determine its class and its value of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Consequently, the computational complexity of the procedure at each such node is &lt;span class=&quot;math-container&quot;&gt;$O(|C| \cdot |A|)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$|A|$&lt;/span&gt; is the number of attributes above. ID3's total computational requirement per iteration is thus proportional to the product of the size of the training set, the number of attributes and the number of non-leaf nodes in the decision tree. The same relationship appears to extend to the entire induction process, even when several iterations are performed. No exponential growth in time or space has been observed as the dimensions of the induction task increase, so the technique can be applied to large tasks.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[3] Since &lt;span class=&quot;math-container&quot;&gt;$I(p,n)$&lt;/span&gt; is constant for all attributes, maximizing the gain is equivalent to minimizing &lt;span class=&quot;math-container&quot;&gt;$E(A)$&lt;/span&gt;, which is the mutual information Of the attribute A and the class. Pearl (1978a) contains an excellent account of the rationale of information-based heuristics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-25T21:30:08.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10206" PostTypeId="2" ParentId="9905" CreationDate="2019-01-25T22:17:46.640" Score="2" Body="&lt;p&gt;Learning is possible without random thoughts and actions. Knowledge can be encapsulated in predetermined forms and passed through predetermined knowledge transfer mechanisms. Much of civilization is based on these predeterminations. Without them, humanity would be thrown back possibly 120,000 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, initial discovery requires trials and review of their outcome. Purely deterministic identifications of trials is necessarily systematic and the system used may interplay with the phenomena under study in such a way as to miss important cases. Furthermore, when the complexity of the phenomenon is high, the number of trials is often too numerous to check entirely. In this second scenario, random selection of trials is wise for a similar reason illustrated by this simple example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phenomenon has one behavior for even numbers and another for odd. The system for determining trials is to check every factor of 100 to cover the range from 1 to 10,000 in 100 trials. The odd behavior will be inadvertently overlooked.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Intelligence begins once the thoughts/actions are logical rather than purely random based.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For the above reasons, intelligence begins with logic only in determining the domain of trials but, when one of the above two cases apply, is often quickly followed by lack of logic in the selection from that domain. Once models have formed as a result of these initial discovery activities, logical inference is useful again, to combine them in various ways. Through this process, engineering, business, resource planning, and other intelligence related disciplines have improved the living conditions for the species, albeit inconsistently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are two limitations of logic.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Piercingly deep searches require some temporary dismissal of logic.&lt;/li&gt;&#xA;&lt;li&gt;Logic applied by large populations produces inconsistently logical results.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is naive to assume that there were logical faults that produces the inconsistency in this second limitation. There is no logical proof that logic necessarily improves conditions logically across the population that uses it. There may be qualities of the goal sets of many individuals that thwart the vision of logic producing peace and prosperity for all. That belief is not new and no person or political group has been able to make it work.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Intelligence needs intelligence to coexist and a sharing communication network for validation/rejection. ... I believe that we must keep the human intelligence in a parental role for long enough time until at least the AI had fully assimilated our values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That assumes that those values are best. Some might want to agree on those values before letting the AI assimilate them, which, given the insanity evident in human history, could lead to a war. If we give the AI the below four objectives, we could probably live without the remainder of human values.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Survival of the biosphere&lt;/li&gt;&#xA;&lt;li&gt;Human freedom of expression&lt;/li&gt;&#xA;&lt;li&gt;Evenly distributed, high prosperity across the human population&lt;/li&gt;&#xA;&lt;li&gt;Encouragement of the social rule, &quot;Be with others how you would want them to be with you,&quot; between humans, between AI robots, and between AI robots and humans.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This statement dismisses the human record.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The actual danger is to leave the artificial intelligence parenting another AI and loose control of it. ... the purpose should always be to help humans achieve mastery of the environment while ensuring our collective preservation. ... AI should not be left unsupervised as we would not give guns to kids, do we? ... To resume it all AI needs an environment and supervision where to learn and grow. The environment can vary but the supervision must stay in place.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is impossible to supervise all adults and it will later become impossible to supervise all AI in data centers and robots. There are already more processes running in computers than there are people in the world. This is why risk management must be applied proactively, during the research and development of the AI. Some of the questions on this site address that very challenge. If it is not done proactively, it will become untenable and control will already have been passed for a random future regarding our species.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Only develop artificial intelligence that is limited by our own beliefs and values rather than searching for something greater than us.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no universal set of beliefs among humans. There will inescapably be a wide range in AI developments with a wide range of intended purposes. The key is to encourage researchers to think beyond their obsession with getting programs to do amusing and impressive things. AI research must continue to keep one foot in technology but another foot firmly placed in the practical elements of ethics, social science, environmental science, economics, and risk management.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-25T22:17:46.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10208" PostTypeId="1" CreationDate="2019-01-26T03:14:09.527" Score="2" ViewCount="67" Body="&lt;p&gt;I am trying to make a personal ML project where my objective is using a photo from an invoice, for instance, a Walmart invoice, classify it as being a Walmart invoice and extract the total amount spent. I would then save this information in a relational database and infer some statistics about my spendings. The goal would be to classify invoices not only from Walmart but from the most frequent shops where I spend money and then extract the total amount spent. I already do this process manually, I insert my spendings in a relational database. I have a bunch of photos from different invoices that I have recorded over the past year for this purpose (training a model).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms would you guys recommend? From my point of view, I think that I need some natural language processing to extract the total amount spent and maybe a convolutional neural network to classify the invoice as being from a specific store?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21688" LastActivityDate="2019-01-27T02:20:32.890" Title="Approach to classify a photo and extract text from it" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10209" PostTypeId="2" ParentId="9975" CreationDate="2019-01-26T08:06:30.010" Score="1" Body="&lt;p&gt;In a nutshell, if you already have a number of models, you usually should be able to distinguish (intuitively, if you will) between simpler and more complex ones. E.g. based on the number of inputs and number of layers, as you have already indicated in the question. Then, if a simpler model and a more complex model perform the same task, and the complex model does not perform significally better than the simpler one, you should use the simpler model. It's your role to decide what difference in performance would be significant, usually based on your use case. It's Occam's razor in practice (&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Occam%27s_razor&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/Occam%27s_razor&lt;/a&gt;).&#xA;You might learn more practical aspects as part of this free course &lt;a href=&quot;https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6053" LastActivityDate="2019-01-26T08:06:30.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10210" PostTypeId="1" CreationDate="2019-01-26T12:28:51.903" Score="4" ViewCount="116" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/actor_critic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open AI's&lt;/a&gt; code to do a RL task on an environment that I built myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried some network architectures, and they all converge, faster or slower on CartPole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On my environment, the reward seems not to converge, and keeps flickering forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect the neural network is too small, but I want to confirm my belief before going the route of researching the architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I confirm that the architecture is the problem and not anything else in a neural network reinforcement learning task?&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2019-03-03T06:24:59.297" Title="How to identify too small network in reinforcement learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10212" PostTypeId="2" ParentId="10191" CreationDate="2019-01-26T14:04:32.607" Score="3" Body="&lt;p&gt;The way to categorize measurements into two separate populations is through what ML people currently term &lt;em&gt;unsupervised learning&lt;/em&gt;. Such a process is part of the AI tool chest. Statistics is part, but not all, of the mathematics involved in the theory that leads to algorithms that learn without labeling the array of points light or dark in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reflected IR light can be specular or mirror in nature and significant levels of IR emission will also occurs, without raising the surface temperature of materials under most common circumstances. There is also refraction and transmission to be considered in the robotics space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Materials will not exhibit lightness. The material composition, coating, and surface texture will exhibit various reflectivities, emission dependent on the absolute temperature raised to the fourth power&lt;sup&gt;1&lt;/sup&gt;, and transmission of light entering the material from various directions, possibly diffused internally.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We eventually hope to be able to use this to follow a line using a pre-set algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is one of the primary objectives of computer vision and has been since the first digital signal processing in the mid twentieth century.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The first step is determining whether the material is one of the binary options, light or dark.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That may be an experimental first step, but the loss of information from taking a number that discretely represents a cube in horizontal-vertical-frame space, each of which has &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; channels of &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; bits, is not a good computer vision strategy. The total number of bits of IR information is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \tau \beta \; \text{,} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the data loss will be, in percent,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \dfrac {100 \, (\tau \beta - 1)} {\tau \beta} \; \text{.} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Object recognition and thus collision avoidance will be frustrated by that much data loss, even if only one IR spectral range is contained in the cube (&lt;span class=&quot;math-container&quot;&gt;$\tau = 1$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given a large population of values between 0 and 1023, probably in two distinct groupings, how can I best go about classifying a given point as light or dark?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It appears that &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; does equal one and &lt;span class=&quot;math-container&quot;&gt;$\beta = \log_{2} (1023 - 0 + 1) = 10$&lt;/span&gt;, thus the information in bits per cube (pixel in a frame) is 10. If the project, for some reason, requires that 9 bits be discarded in a way to conserve as much of the original 10 bits as possible (perhaps to maintain a particular throughput through a transmission or processing bottleneck), then the way to do it is through a one bit feature extraction, meaning that the output of the artificial network should be one bit. These are a few unsupervised learning options to investigate.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;RBMs (restricted Boltzmann machine)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;K-means clustering&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gaussian mixture models&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Most AI and artificial network frameworks and libraries will have examples of these in the example directory. It is advisable to approach the project like this.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Find web pages or articles using the above as search terms until an explanation that is clear to the team is found.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Study all three for at least a few hours each as a team.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find a library or framework that contains all three.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Install the necessary prerequisites for all three on the lab computer(s).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Produce a working example of the three types so that selection is not a function of which one is easiest to get running (something which would qualify as an research anti-pattern).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Make a choice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Change the working code, one verifiable step at a time into the desired unsupervised learning program.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Eventually, when thoroughly ready, make the jump from the example data set to the robotic IR light frame set.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Stefan-Boltzmann law&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-26T14:04:32.607" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10213" PostTypeId="1" AcceptedAnswerId="10215" CreationDate="2019-01-26T14:11:41.233" Score="2" ViewCount="55" Body="&lt;p&gt;It is known that &lt;em&gt;every potential function&lt;/em&gt; won't alter the optimal policy [&lt;a href=&quot;https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. I lack of understanding why is that.&lt;/p&gt;&#xA;&lt;p&gt;The definition:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R' = R + F,$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$$F = \gamma\Phi(s') - \Phi(s),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where, let's suppose, &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If I have the following setup:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;on the left is my &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;on the right my potential function &lt;span class=&quot;math-container&quot;&gt;$\Phi(s)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;the top left is the start state, the top right is the goal state&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WFhWc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WFhWc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reward for the red route is: &lt;span class=&quot;math-container&quot;&gt;$(0 + (0.9 * 100 - 0)) + (1 + (0.9 * 0 - 100)) = -9$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And the reward for the blue route is: &lt;span class=&quot;math-container&quot;&gt;$(-1 + 0) + (1 + 0) = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, for me, it seems like the blue route is better than the optimal red route and thus the optimal policy changed. Do I have erroneous thoughts here?&lt;/p&gt;&#xA;" OwnerUserId="21685" LastEditorUserId="2444" LastEditDate="2020-11-09T17:25:18.763" LastActivityDate="2020-11-09T17:25:18.763" Title="Why does potential-based reward shaping seem to alter the optimal policy in this case?" Tags="&lt;reinforcement-learning&gt;&lt;reward-functions&gt;&lt;reward-shaping&gt;&lt;potential-reward-shaping&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10214" PostTypeId="1" AcceptedAnswerId="10229" CreationDate="2019-01-26T14:24:55.517" Score="1" ViewCount="63" Body="&lt;p&gt;I’m looking for advice regarding my ML project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a special wristband, I am able to collect a bunch of physiological data from human subjects. I want to develop an application to recognize when these physiological signals change in a meaningful way and only then ask the user how he/she is feeling. This data will later be used for machine learning testing. The problem is, that I am struggling to find appropriate ways to classify current data input as meaningful and ask for information only when relevant user input is to be gathered, not more and not less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, this seems to be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Novelty_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;novelty detection&lt;/a&gt; problem, combined with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary classification&lt;/a&gt; problem. I have to recognize what values coming from the data stream are to be considered &lt;em&gt;normal&lt;/em&gt;, and therefore not bother the user with unnecessary input requests. I would also use novelty detection to recognize the data coming out of the normal zone and ask the user about it. This new data is then not considered novelty anymore, and binary classification will tell if the user is to be asked about his emotions when getting the same data in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, these are my questions:&lt;br&gt;&#xA;-  What do you think about my reasoning of the problem? Do you have other perspectives on how to handle these problems? I have been told this could also be considered an &lt;a href=&quot;https://en.wikipedia.org/wiki/Anomaly_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;anomaly detection&lt;/a&gt; problem, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-  What algorithms would you use to separate normal from more meaningful physiological data? &lt;a href=&quot;https://en.wikipedia.org/wiki/Support-vector_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Support Vector Machines&lt;/a&gt; perhaps? Maybe some &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision theory&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-  Do you know any books or papers on similar matters? Even if I have found some after hours and hours of research, you may be able to point me to something different than those I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth noting that data collection is supposed to be done when no other factors are messing with signal readings, such as sport.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&lt;br&gt;&#xA;Augusto&lt;/p&gt;&#xA;" OwnerUserId="21767" LastActivityDate="2019-01-27T05:55:35.017" Title="Help with Novelty Recognition and Binary Classification for Emotion Recognition" Tags="&lt;machine-learning&gt;&lt;data-science&gt;&lt;decision-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10215" PostTypeId="2" ParentId="10213" CreationDate="2019-01-26T14:30:38.727" Score="2" Body="&lt;p&gt;The same &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt; that you use in the definition &lt;span class=&quot;math-container&quot;&gt;$F \doteq \gamma \Phi(s') - \Phi(s)$&lt;/span&gt; should also be used as the discount factor in computing returns for multi-step trajectories. So, rather than simply adding up all the rewards for your different time-steps for the different trajectories, you should discount them by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; for every time step that expires.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the returns of the blue route are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$0 + (0.9 \times -1) + (0.9^2 \times 0) + (0.9^3 \times 1) = -0.9 + 0.729 = -0.171,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the returns of the red route are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(0 + 0.9 \times 100 - 0) + 0.9 \times (1 + 0.9 \times 0 - 100) = 90 - 89.1 = 0.9.$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-26T14:30:38.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10216" PostTypeId="2" ParentId="10208" CreationDate="2019-01-26T15:48:04.753" Score="1" Body="&lt;p&gt;There are three functional aspects to this project, each of which could be a sub-project, even though they are interrelated. The decoupling of these aspects in the associated R&amp;amp;D will likely improve the rate of development.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Classify images &amp;mdash; in this case, the vendor brand&lt;/li&gt;&#xA;&lt;li&gt;Locate and orient the text&lt;/li&gt;&#xA;&lt;li&gt;OCR the text &amp;mdash; in this case, the total&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are certainly feasible in the particular case of invoices from vendors, so the entire project certainly is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to produce two fields of information from input images.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Vendor code&lt;/li&gt;&#xA;&lt;li&gt;Total in some specific monetary unit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That past invoices, the names of the vendors, and the totals of each invoice are available as training data may be helpful. Whether the volume of training data is sufficient can only be determined through a very complicated application of theory to some of the data metrics or by trial. It is recommended to do both.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What algorithms would you guys recommend?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It would be irresponsible to define an AI design in terms of what people recommend for algorithm. The algorithms that guys recommend are already covered by the path names for files in the algorithms directory paths, the example's directory paths, and the sections of the documentation of any good AI framework. Let's first talk about the design and then the algorithm options, so there is more of a basis for algorithm selection than the largely random reading selections of the members of a site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The models involved and which can be parameterized such that they can be tuned through training are partly defined by the three sub-projects above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this project, there is no reason for the AI to recognize which text on the invoice is the vendor name. If there are a hundred vendor invoice types, the speed in which a single person can identify the key boundaries of each invoice type is orders of magnitude faster than the work necessary to develop a completely general algorithmic approach for automating that work. It would only be resource thrifty to develop such automation if there were thousands of vendors and constantly varying templates for invoices, which is likely in the case of personal finance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the key bounds of the form quadrilaterals (four sided polygons) within which the following objects lie.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Vendor name text box&lt;/li&gt;&#xA;&lt;li&gt;Total amount text box&lt;/li&gt;&#xA;&lt;li&gt;Rectangle around the entire form&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Digitizing the twelve points through one of the labelling programs is a model much more likely to produce a reliable system than the representation of three rectangular bounding boxes and tilts. This is because tilt angle requires at least two adjacent points anyway and the aspect ratio cannot be held constant in real scanning scenarios when the scanner might be replaced or produce different ratios with wear or the smoothness or wear characteristics of the paper invoice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing all three things using a network would require more than one person's invoices for a decade, unless the person is a billionaire compulsive buyer with a team of secretaries doing scanning and data entry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The models are then these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Digitization of a quadrilateral document with varying contrast, brightness, tilt, horizontal pixels per inch, vertical pixels per inch, relative location and size of vendor name, and relative location and size of invoice total&lt;/li&gt;&#xA;&lt;li&gt;Numeric characters and other monetary characters in a rectangular box&lt;/li&gt;&#xA;&lt;li&gt;Brand image for the vendor, which may include a logo and type of arbitrary and possibly unique font&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now we can talk about artificial network approaches.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Adjusting for contrast, brightness, tilt, resolution, locations, and sizes will require a customized input, loss function, and layer arrangement based on the geometry involved. A GRU network may gain some advantages if the documents can be ordered chronologically, since the collection then becomes a time series, the trends of which can be exploited.&lt;/li&gt;&#xA;&lt;li&gt;Monetary values can best be done using OCR libraries.&lt;/li&gt;&#xA;&lt;li&gt;Recognizing the brand is probably best done with CNN as a categorization machine.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The system must indicate if the vendor and total are not found, in which case a new template is indicated, and the twelve points must be digitized for this new type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to do this with one single deep convolutional network, but, again, the data set would need to be augmented. The one other way to do this is to create a filled-out-invoice generator to produce the volume of stochastically variable data across the various variability dimensions listed above to train the deep CNN.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-26T15:48:04.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10217" PostTypeId="2" ParentId="1544" CreationDate="2019-01-26T15:56:43.760" Score="0" Body="&lt;p&gt;Curiosity is used successfully with Random Network Distillation (RND). OpenAI has published a &lt;a href=&quot;https://blog.openai.com/reinforcement-learning-with-prediction-based-rewards/&quot; rel=&quot;nofollow noreferrer&quot;&gt;detailed article&lt;/a&gt; about their approach using this method, which was especially successful with previously unsolved games like Montezuma’s Revenge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While this does not fully answer your question about curiosity being required to build a true AI, it shows that previously unsolved problems became solvable introducing curiosity in the reward system.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2019-01-26T15:56:43.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10218" PostTypeId="1" CreationDate="2019-01-26T16:50:11.450" Score="1" ViewCount="30" Body="&lt;p&gt;I'm running Kera's &lt;code&gt;LSTM&lt;/code&gt; (not &lt;code&gt;CuDNNLSTM&lt;/code&gt;) but I notice my GPU is under load. I need recurrent dropout, so I can only stick with &lt;code&gt;LSTM&lt;/code&gt;. Is the 'normal' &lt;code&gt;LSTM&lt;/code&gt; assisted by GPU? If so, how are &lt;code&gt;LSTM&lt;/code&gt; and &lt;code&gt;CuDNNLSTM&lt;/code&gt; different? I presume &lt;code&gt;CuDNNLSTM&lt;/code&gt; uses the CUDNN API (and &lt;code&gt;LSTM&lt;/code&gt; doesn't?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, is the normal &lt;code&gt;LSTM&lt;/code&gt; supposed to be faster on GPU or CPU?&lt;/p&gt;&#xA;" OwnerUserId="13068" LastEditorUserId="13068" LastEditDate="2019-01-26T16:57:27.230" LastActivityDate="2019-01-26T16:57:27.230" Title="Kera's (normal) LSTM uses the GPU?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10220" PostTypeId="2" ParentId="10208" CreationDate="2019-01-26T18:10:17.680" Score="1" Body="&lt;p&gt;Since many of these problems have been tackled earlier and we have quite a few good tools to handle images and text, the task does not seem to be so very difficult. But then you would only find out after actually trying out the solutions suggested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest the following approach:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use Tesseract and OpenCV to extract text from the images you have saved. You can refer to a good example of using tesseract with python here - &lt;a href=&quot;https://www.pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Evaluate the results to plan and consider your next steps. Refer to the publications linked to the tutorials such as the EAST algorithm which describes the challenges and point to other studies on this topic.  This will help you evaluate the other approaches which have been studied for different cases by other researchers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assuming the text detection has worked well, proceed forward by detecting parts of the document by using row detection (find continuous white-space extending horizontally) and tab stop detection (continuous white-space vertically). Use the Tesseract/openCV detected text blocks for this purpose.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Re-organize the detected text according to the layout identified by rows/columns. Next, recognize and discard the text you may not required, e.g. the logo, other associated details such as phone no of the store, tax breakup, change given, etc. These may be easily picked up by carefully crafted regular expressions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run a spell checker to remove errors in recognition (aspell, pyenchant, etc.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, from the results obtained, you could evaluate the remaining errors and fix them by either writing a set of regular expressions for easily identified OCR mistakes, or, else build a text model to learn and fix similar errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2106" LastEditorUserId="2106" LastEditDate="2019-01-27T02:20:32.890" LastActivityDate="2019-01-27T02:20:32.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10221" PostTypeId="2" ParentId="10177" CreationDate="2019-01-26T18:23:25.590" Score="2" Body="&lt;p&gt;A bounding box is a rectangle superimposed over an image within which all important features of a particular object is expected to reside. It's purpose is to reduce the range of search for those object features and thereby conserve computing resources: Allocation of memory, processors, cores, processing time, some other resource, or a combination of them. For instance, when a convolution kernel is used, the bounding box can significantly limit the range of the travel for the kernel in relation to the input frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When an object is in the forefront of a scene and a surface of that object is faced front with respect to the camera, edge detection leads directly to that surface's outlines, which lead to object extent in the optical focal plane. When edges of object surfaces are partly obscured, the potential visual recognition value of modelling the object, depth of field, stereoscopy, or extrapolation of spin and trajectory increases to make up for the obscurity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A classifier can only classify one object per image&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A collection of objects is an object, and the objects in the collection or statistics about them can be characterized mathematically as attributes of the collection object. A classifier dealing with such a case can produce a multidimensional classification of that collection object, the dimensions of which can correspond to the objects in the collection. Because of that case, the statement is false.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) If I don't need to know 'where' an object is (or objects are) and just interested in the existence of them and how many there are, is it possible to just get rid of the boundary boxes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have sufficient resources or patience to process portions of the frame that don't contain the objects, yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions (2) and (3) are already addressed above, but let's look at them in that context.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2.a) If not, how does bounding boxes help detect objects?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It helps by fulfilling its purpose, to reduce the range of the search. If by thrifty method a bounding shape of any type can be created, then the narrowing of focus can be used to reduce the computing burden on the less thrifty method by eliminating pixels that are not necessary to peruse with more resource-consuming-per-pixel methods. These less thrifty methods may be necessary to recognize surfaces, motion, and obscured edges and reflections so that the detection of object trajectory can be obtained with reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That these thrifty mechanisms to find the region of focus and these less thrifty mechanisms to use that information and then determine activity at higher levels of abstraction are artificial networks of this type or that or use algorithms of this type or that is not relevant yet. First understand the need to reduce computing cost in AI, which is a pervasive concept for anything more complex than tic-tac-toe, and then consider how bounding boxes help the AI engineer and the stakeholders of the engineering project to procure technology that is viable in the market.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2.b) From what I have figured is that a network (if using neural network architectures) predicts the coordinates of the bounding boxes if there is something in the feature map. Doesn't this mean that the detector already knows where the object is (at least briefly)?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;2.c) So continuing from question 1, if I'm not interested in the exact location, would training for bounding boxes be irrelevant?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Cognition is something AI seeks to simulate, and many hope to have robots like in the movies that can help out and be invaluable friends, like TARS in the Nolan brothers 2014 film Interstellar. We're not there. The network knows nothing. It can train a complex connection between an input signal through a series of attenuation matrices and activation functions to produce an output signal statistically consistent with its loss function, value function, or some other criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inner layers of an artificial net may, if not directed to do so, produce something equivalent to a bounding region only if velocity of convergence is present as a factor in its loss or value function. Otherwise there is nothing in the Jacobian leading convergence to reduce its own time to completion. Therefore, the process may complete, but not as well as if cognition steps in and decides that the bounding region will be found first and then used to reduce the total burden of mechanical (arithmetic) operations to find the desired output signal as a function of input signal.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;3) Finally, in architectures like YOLO, it seems that they predict the probability of each class on each grid (e.g. 7 x 7 for YOLO v1). What would be the purpose of bounding boxes in this architecture other than that it shows exactly where the object is? Obviously, the class has already been predicted so I'm guessing that it doesn't help classify better.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Reading the section in &lt;a href=&quot;https://www.mdpi.com/1999-4893/10/4/127/pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Real-Time Chinese Traffic Sign Detection Algorithm Based on Modified YOLOv2&lt;/em&gt;&lt;/a&gt;, J Zhang, M Huang, X Jin, X Li, 2017, may help further comprehension of these principles and their almost universal role in AI, especially the text around their statement, &quot;The Network Architecture of YOLO v2 YOLO employs a single neural network to predict bounding boxes and class probabilities directly from full images in one inference. It divides the input image into S × S grids.&quot; This way you can see the use of these principles in the achievement of specific research goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other such applications can be found by simply reading the article full text available on the right side of &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C10&amp;amp;q=YOLO+algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;an academic search for yolo algorithm&lt;/a&gt; and using ctrl-f to find the word bound.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-01-28T11:16:27.830" LastActivityDate="2019-01-28T11:16:27.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10222" PostTypeId="2" ParentId="9234" CreationDate="2019-01-26T18:29:27.137" Score="1" Body="&lt;p&gt;Either (a) re-recognize it in subsequent frames and then train a network employing a trajectory model to the changing object model parameters or (b) recognize the object and its motion in a single object-trajectory parameterized model from the sequence of sets of detected edges indicating edge movement in the sequence of frames.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-26T18:29:27.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10223" PostTypeId="2" ParentId="1544" CreationDate="2019-01-26T18:34:29.180" Score="1" Body="&lt;p&gt;Curiosity by itself does not improve intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It increases the chances of better understanding a given subject, given that curiosity is coupled with actions in that direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&#xA;I am curious about how to make pancakes and decide to find a recipe but stop at the first instance of an answer with steps to follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curiosity needs to be coupled with the desire to improve a given understanding and be followed by a review of current knowledge with the aim of updating the previousely reached conclusions. Provided that the used logic that judges improvements is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curiosity will not necessarily be benefitial for an improved intelligence.  But to allow for an improved intelligence, curiosity is a mandatory requisite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curiosity is a symptom of an evolving intelligence.&lt;/p&gt;&#xA;" OwnerUserId="21285" LastEditorUserId="21285" LastEditDate="2019-01-26T18:53:12.110" LastActivityDate="2019-01-26T18:53:12.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10227" PostTypeId="2" ParentId="10201" CreationDate="2019-01-27T04:30:31.920" Score="0" Body="&lt;p&gt;Test batch time is referenced in the paper on page 33, Section 3.6, Inference Time and in Table 6. When an example set is used, a portion of it is used for training and another portion is used for testing. In mini-batch configurations the mini batch sizes are also independently configurable between train and test phases in many implementations.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.springer.com/cda/content/document/cda_downloaddocument/9783319464473-c2.pdf?SGWID=0-0-45-1588723-p180264624&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;SSD: Single Shot MultiBox Detector&lt;/em&gt;&lt;/a&gt;, 2016, Wei Liu&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-27T04:30:31.920" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10228" PostTypeId="1" CreationDate="2019-01-27T05:54:39.257" Score="4" ViewCount="1267" Body="&lt;p&gt;In an MLP with ReLU activation functions after each hidden layer (except the final),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the final layer should output positive and negative values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With ReLU intermediary activations, this is still possible because the final layer, despite taking in positive inputs only, can combine them to be negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, would using leaky ReLU allow faster convergence? Because you can pass in negative values as input to the final layer instead of waiting till the final layer to make things negative&lt;/p&gt;&#xA;" OwnerUserId="21158" LastActivityDate="2019-04-02T07:48:20.057" Title="Does leaky relu help learning if the final output needs negative values?" Tags="&lt;neural-networks&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10229" PostTypeId="2" ParentId="10214" CreationDate="2019-01-27T05:55:35.017" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Project&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears from the question that emotional detection and response is the longer term goal of the project and that recognizing potential emotional manifestations in easily detectable physiological metrics is an initial R&amp;amp;D objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mobile device applications are already available to do this, but biometric monitoring via a wristband, excellence in AI design, and marketing excellence could overtake these apps in the marketplace and provide emotional regulation to improve productivity and reduce health and wellness risks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (at least initial) goal seems to be basic binary classification, which simplifies the output, but not the detection, which seems to have the following two criteria.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Acquire biological information related to emotion&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Avoid drawing the user's attention to perform unnecessary tasks&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Biometric Device Challenge&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is definitely a challenge to classifying biometric trends as meaningful in this context.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What do you think about my reasoning of the problem? Do you have other perspectives on how to handle these problems? I have been told this could also be considered an anomaly detection problem, for example.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Novelty detection is not the first milestone in this challenge. Detection of useful features is the prerequisite. Novelty comes into play once changes in the organism can be characterized with some reliability and accuracy (few false positives and false negatives) and the particular user has provided subjective reporting to correspond with some change in the organism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Biological systems have forms of stasis that can be indirectly sensed, including these.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Blood pressure&lt;/li&gt;&#xA;&lt;li&gt;Metabolic rate&lt;/li&gt;&#xA;&lt;li&gt;Multi-ionic salinity&lt;/li&gt;&#xA;&lt;li&gt;Cognitive attention&lt;/li&gt;&#xA;&lt;li&gt;Blood sugar levels&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These affect the dermis at the wrists through perturbations in externally detectable metrics, the inclusion of which depends on the capabilities of the wrist device.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Circumference&lt;/li&gt;&#xA;&lt;li&gt;Coefficient of electrical resistance&lt;/li&gt;&#xA;&lt;li&gt;Surface temperature&lt;/li&gt;&#xA;&lt;li&gt;Multi-ionic salinity&lt;/li&gt;&#xA;&lt;li&gt;Surface moisture&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With blood and urine lab tests, conditions can be controlled, however sampling of those fluids are usually infrequent, leading to the use of ranges to detect anomalies that may be indicative of disease, conditions, or other health risks. There are major challenges to using ranges of metrics at the wrist.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sensitivity to the general and ongoing physical condition of the individual&lt;/li&gt;&#xA;&lt;li&gt;Unique individual responses to emotional states&lt;/li&gt;&#xA;&lt;li&gt;Interference from changes in the temperature, pressure, air movement, and humidity of the environment&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One inroad to detection of the internal metrics of the organisms is the fact that biological stasis is not the same thing as perfect regulation. Biological systems are chaotic. Normal signals from time series sampling of biological metrics are neither constant nor periodic. When changes in the attractors and spectra characterizing the chaotic fluctuation of direct metrics (like electrical resistance) or indirect metrics (like systolic blood pressure) occur, a request for subjective information may be asked from the wearer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some chaotic principles to consider.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Spectra can provide much information about the intensity of the force driving the chaotic fluctuation in the time series, which is likely to correlate with stress.&lt;/li&gt;&#xA;&lt;li&gt;The various ways to evaluate exponents, including the Lyapunov exponent, may be of significant value in correlating patterns in the external, indirect time series to emotional states.&lt;/li&gt;&#xA;&lt;li&gt;Autocorrelation at the primary frequencies involved may reveal changes indicative of significant mental transitions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once a correlation is established, then the novelty detection makes sense, except for an occasional re-acquisition of the correlation, since the wearer may become more internally connected to their physical and emotional states as they use the app, and the answers to inquiries may mature correspondingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Questions Within the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These additional questions were also asked.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What algorithms would you use to separate normal from more meaningful physiological data? Support Vector Machines perhaps? Maybe some decision theory?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A recurrent neural network such as B-LSTM or GRU are commonly excellent at characterizing time series, but there may be a need for more than one network.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One to extract the features of the chaotic patterns in the raw data&lt;/li&gt;&#xA;&lt;li&gt;Another to train to correlate changes in the features of the chaos with subjective reporting&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This second training must be done during &lt;em&gt;in situ&lt;/em&gt; field use of the first already trained network. Since training is not generally viable on a mobile device, there will need to be a client server arrangement where the training data and its resulting trained network can be communicated through secure RestFUL services so the training processes can be asynchronous and leverage GPUs or other hardware acceleration.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do you know any books or papers on similar matters? Even if I have found some after hours and hours of research, you may be able to point me to something different than those I have.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The terms above in a scholarly search will bring up all kinds of study materials. Specifically these&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lyapunov exponent&lt;/li&gt;&#xA;&lt;li&gt;Strange attractors&lt;/li&gt;&#xA;&lt;li&gt;Spectral analysis of biometrics&lt;/li&gt;&#xA;&lt;li&gt;Chaos autocorrelation&lt;/li&gt;&#xA;&lt;li&gt;Phase space&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One good book for all but the middle item is &lt;em&gt;Chaos Theory Tamed&lt;/em&gt;, 1997&#xA;by Garnett P. Williams.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is worth noting that data collection is supposed to be done when no other factors are messing with signal readings, such as sport.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is where higher level detection involving chaotic analysis is much more reliable than ranges. The chaotic features of athletic activity will be distinctly different from those of fear states or sexual arousal.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-27T05:55:35.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10232" PostTypeId="2" ParentId="10228" CreationDate="2019-01-27T09:50:19.457" Score="2" Body="&lt;p&gt;In short, yes Leaky Relu helps in faster convergence if your output requires both positive and negative values. But the catch is that you need to tune the negative slope of Leaky Relu which is a hyperparameter to get better accuracy.&lt;/p&gt;&#xA;" OwnerUserId="9062" LastActivityDate="2019-01-27T09:50:19.457" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10236" PostTypeId="2" ParentId="10228" CreationDate="2019-01-27T11:38:12.923" Score="0" Body="&lt;p&gt;Both the output and the the gradient at the negative part of leaky ReLU is 100 times lower than at the positive part. I doubt that they have any significant impact on training direction and/or on the final output of a trained model unless the model is severely underfitting.&lt;/p&gt;&#xA;" OwnerUserId="21426" LastActivityDate="2019-01-27T11:38:12.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10239" PostTypeId="1" AcceptedAnswerId="10245" CreationDate="2019-01-27T14:31:51.263" Score="1" ViewCount="80" Body="&lt;p&gt;In my input tensor, I would like to use both integer values as well as booleans. For example, if there is a spelling difference between 2 texts, I want to set the value to true, and otherwise false. In the same tensor, I would like to assign a value to, for example, the maximum number of consecutive messages, which will be an integer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I allowed to use 0's and 1's for the booleans together with integers, or will it have any negative impact on the working of the network? The ANN wont see any difference between the binary and nonbinary values, but is it a problem?&lt;/p&gt;&#xA;" OwnerUserId="21788" LastEditorUserId="1641" LastEditDate="2019-02-26T20:56:38.833" LastActivityDate="2019-02-26T20:56:38.833" Title="Nonbinary and binary values in input tensor" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10240" PostTypeId="2" ParentId="10040" CreationDate="2019-01-27T15:11:09.337" Score="1" Body="&lt;p&gt;The financial evaluation of the distributions of costs and benefits of establishing an AI department or group to support other business functions is probably roughly additive. Consider approaching each of these separately and always compare the three most obvious options.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fund the centralization of AI research to delegate and reuse expertise inter-departmentally, such as with IT, HR, and accounting.&lt;/li&gt;&#xA;&lt;li&gt;Fund the decentralization of AI research, such that specific expressions of AI are running independently and in parallel&lt;/li&gt;&#xA;&lt;li&gt;Don't fund AI research&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course there are funding amounts spread over the five years to be determined for the first two and they can be mixed as government is in federalism or as a mid-level corporation that has an accountant in each department that reports up its director but is also responsible to an accounting department.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are a few functions the planning or control can benefit from AI in the categories specified in the question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Optimizing purchasing thrift in conjunction with quality control&lt;/li&gt;&#xA;&lt;li&gt;Optimizing manufacturing thrift in conjunction with quality control&lt;/li&gt;&#xA;&lt;li&gt;New product or services or enhancements with AI involved in their design&lt;/li&gt;&#xA;&lt;li&gt;New product or services or enhancements that run with enclosed AI components&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The substantial overlap is evident in the description of potential benefits.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The investments in AI can only be justified if we can state the monetary benefit.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The department heads won't volunteer any projections because they will then be held to them, and AI has about the same level of success in past projections as photovoltaics (the panels of which were supposed to mostly replace fuels a decade ago) and robots (which were going to be doing our dishes for us around that same time). People are skittish for good reason.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any creative approaches out there to estimate the monetary potential of AI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One creative approach is to estimate the loss to the company if no AI is funded, which may be so dismal that it consumes the entire five year AI budget.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Loss in personnel that don't want to work for a company that doesn't have AI in the job description&lt;/li&gt;&#xA;&lt;li&gt;Loss in credit and confidence because a prospectus without AI is like raising a flag with the bright red letters, &quot;We're not progressive.&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That may not fly, but if it did, the analogy of the departmental accountant and the accounting department to quality control and educate the entire corporate financial mechanism is likely to also fly to the same board or senior management.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The less creative approach is to find the most similar past trends to each of the applications of AI technology to the four items above and use them to project benefit. Cost can be based on the typical infrastructure needs plus the extra fast workstations with hardware acceleration plus the cost of labor, which can be estimated now that the term AI engineer is starting to appear in salary surveys. &lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-27T15:11:09.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10244" PostTypeId="1" AcceptedAnswerId="10246" CreationDate="2019-01-27T18:26:37.263" Score="0" ViewCount="42" Body="&lt;p&gt;Say I want to train a NN that generates outputs of some sort (say, even numbers).&#xA;Note that the network does not classify outputs, but, rather GENERATES the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want let it run forward and generate some number, then either give it a positive reward of 1 for an even number, and a reward of -1 for an odd number, to make i output only even numbers over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What would be an architecture for such a NN?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am getting caught in the part where here is actually no input, and I can't really start with a hidden layer, can I?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am quite confused and would appreciate guidance&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2019-01-27T19:24:00.567" Title="How to (theorically) build a neural network with input of size 0?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10245" PostTypeId="2" ParentId="10239" CreationDate="2019-01-27T18:34:52.153" Score="-1" Body="&lt;p&gt;If the two inputs for the binary values are held to the domain set &lt;span class=&quot;math-container&quot;&gt;$\{0, 1\}$&lt;/span&gt; during training, testing, and use, it will not break the network functionality, although the question is a good one. Why use so many bits to hold one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Theory of Holors: A Generalization of Tensors&lt;/em&gt;, by Parry Moon and Domina Spencer (ISBNs 978-0521019002 and 0521019001), propose holors precisely because of this kind of limitation on the natural heterogeneity of numerical structure. Gibbs, Ricci, Einstein, and others used holors in their mathematical expressions but without the name holor. Holors are are conceptually related to objects in ontology and classes in object oriented design, but they are numbers, so they fit into mathematical expressions as scalars, vectors, matrices, vector fields, and other tensors do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the basis for Coplien's operator overload in C++ and one of the reasons the language is too mutable to be mastered by average applications programmers. Support for what Moon and Spenser called holors is also the basis of decoupled type safety in some of the early LISP object oriented frameworks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current programming paradigm we seen in Java, Python, and other popular languages was set in FORTRAN, which was more attainable for average programmers, so we are stuck with homogeneous structure, even in AI libraries and GPUs. Therefore, once one number requires an IEEE 64 float, bytes and bits will use all 64.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In artificial networks, this only matters in the very first multiplication with a parameter.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-27T18:34:52.153" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10246" PostTypeId="2" ParentId="10244" CreationDate="2019-01-27T19:24:00.567" Score="2" Body="&lt;p&gt;Whether a neural network has learned anything or not, it is a function that maps some input to an output. Training is the process of tweaking the weights so that the output is something that we want. Thus there is always in input of some sorts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you have presented, of generating even numbers, is much like a &lt;a href=&quot;https://skymind.ai/wiki/generative-adversarial-network-gan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Network (GAN)&lt;/a&gt;. In a GAN there are 2 networks: a Generator that tries to generate a sample from a target distribution and a Discriminator that tries to tell real samples from fake samples. The classic analogy being a criminal making counterfeit money and a copy trying to tell what is real money or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generator input is usually a random number (or a matrix of random numbers). The generator then learns to transform a particular random input to a particular point in the target space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question, no there can't be a neural network with 0 inputs as there must always be an input of some kind. Even if the network was to generate a sequence instead of one instance, it would still need something to start with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your example, there would have to be some input for the  network to start with. A really simple NN that could solve your problem might look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_input = [RandomInteger()]&#xA;neuralNetworkWeights = [2]&#xA;result = _input * neuralNetworkWeights&#xA;result is always even&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-01-27T19:24:00.567" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10247" PostTypeId="1" AcceptedAnswerId="10276" CreationDate="2019-01-27T20:02:55.583" Score="2" ViewCount="175" Body="&lt;p&gt;I am trying to implement NEAT algorithm in Python from scratch. However, I am stuck. When a new innovation number is created it has two nodes which represents the connection. Also this innovation number has a weight. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I know that innovation numbers are global variables, in other words when a innovation number is created, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ex. Innovation ID:1 - Node:1 to Node:4 - weight: 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it will have a ID which will be used by other connections to represent the connection between Node:1 to Node:4. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When this innovation is used by another neural network, will it also use the weight of the innovation 1, which is 0.5 in this example?&lt;/p&gt;&#xA;" OwnerUserId="21517" LastEditorUserId="2444" LastEditDate="2019-02-28T22:49:45.980" LastActivityDate="2019-03-30T23:00:42.300" Title="Are innovation weights shared in the NEAT algorithm?" Tags="&lt;neural-networks&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10248" PostTypeId="1" CreationDate="2019-01-27T20:21:00.117" Score="2" ViewCount="39" Body="&lt;p&gt;I'm interested in creating a convolutional neural network or LSTM to locate text in an image. I don't want to OCR the text yet, just find the text regions. Yes, I know Tesseract and other systems can do this, but I want to learn how it works by building my own. All of the tutorials and articles I've seen so far have the CNN output to a classification - &quot;image contains a cat&quot;, &quot;image contains a dog&quot;. Okay that's nice, but it doesn't say anything about where it was found. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me to some information that describes the output layer of a NN that can give location information? Like, x-y co-ordinates of text boxes?&lt;/p&gt;&#xA;" OwnerUserId="21796" LastActivityDate="2019-01-27T20:21:00.117" Title="How does a neural network output text box location data?" Tags="&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10250" PostTypeId="2" ParentId="8559" CreationDate="2019-01-27T22:11:48.173" Score="-2" Body="&lt;p&gt;&lt;strong&gt;Discrete Experiments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first impression is that real numbers (or IEEE floating point approximations of them) don't belong in genetic algorithms because DNA genetics is based on  binary representations based on the presence of either adenine-thymine or cytosine-guanine in a position along a chromosome. Sub-sequences of those bits correspond to specific triggers and resulting protein syntheses. Genetic information is discrete, not continuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is true, but, as the number of instances being represented probabilistically approach infinity, the distance between rational numbers that represent the potential probability values tend toward zero. That is, where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{I}$&lt;/span&gt; is the set of indices for all possible probabilities for a given discrete scenario, we have this limit that tends toward but never reaches continuity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \forall \, j \in \mathbb{I}, \, \lim_{n \rightarrow \infty} P_j - P_{j-1} = 0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as the number of bits in the mantissa of a fixed point number increases, the distance between their values similarly decrease toward the limit of zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the authors state, &quot;Binary code is good for pedagogy,&quot; they probably mean that it is difficult to store a voltage without sampling it and representing it as an integer that can be stored in noise and drift resistant ways using binary encoding and error correction strategies. If this is a correct expansion of what that phrase means, then it makes much sense. In analog circuitry, a voltage may be latched for a few microseconds while it is sampled, capacitor leakage is too high to trust the value much longer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, far more samples are often taken than what is prescribed by the Nyquist criteria so that noise cancellation improves the signal to noise ratio. Once the signal is in digital form, error correction and detection reduces these phenomena significantly. Although we can prove this using mathematics, the hyper-evolution of the evolutionary mechanism itself has proven these facts over a few billion years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, the theory presented from the authors is clear. They are stating three things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Real numbers represented in computers as integers, IEEE floating point numbers, or arbitrary precision numbers using appropriate libraries are discrete approximations of continuous values.&lt;/li&gt;&#xA;&lt;li&gt;DNA represent in binary state-action relationships the action plans based on continuous probability distributions in the environment with which the system interacts.&lt;/li&gt;&#xA;&lt;li&gt;These two facts create a fundamental correlation between genetic information and binary approximations of real numbers in computers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although the first two statements are correct, the third is not. This is the kind of fundamental error common in statements made by people with excellence in mathematics and computer science but somewhat less profoundly educated in the nuances of biology and probability. Consider these discrete events that affect the probabilities that some enzyme or protein should be generated under some particular threshold condition in the cellular machinery.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A lion kills a jackal of two&lt;/li&gt;&#xA;&lt;li&gt;One of the two had a child jackal&lt;/li&gt;&#xA;&lt;li&gt;The other did not&lt;/li&gt;&#xA;&lt;li&gt;A particular combination of AT versus CG bits led to the manufacture of a protein that allowed a faster muscular response, which allowed the remaining jackal to propagate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The function representing the event of capture is not continuous. The probability of a death event between birth and conception of the child jackal is effectively continuous. For a child jackal, the probabilities of their conception is irrelevant. The pertinent function from the perspective of that specific line of jackals outputs exactly one bit. Zero is nonexistence and one is existence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function that may model evolution surrounding this scenario is a continuous appearing function, but it is made up of discrete events that may be conditioned by real numbers but, since injured jackals can still reproduce, do not result in real number outcomes. This is where the authors are not accurately modeling biological reality. A functional representation of selection on the basis of a single genetic bit is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ S_i = S(E, P(G_i, T)) \, \text{,} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the survival bit &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; for a given offspring is an effectively randomized selection from the set &lt;span class=&quot;math-container&quot;&gt;$\{0, 1\}$&lt;/span&gt;, given the environmental scenario in which the parent lived from birth to the spawning of an offspring &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, and the probability of a phenotype &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, which is a function of the set of genetic bits in the genotype &lt;span class=&quot;math-container&quot;&gt;$G_i$&lt;/span&gt; (the bits in question), and the triggers &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; of the expression of that genotype presented by both internal and external organism state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is where the authors clearly diverge from what is known about the genetic machinery in the context of adaptation of a species.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We can use both 101001101011 and 0.9534 to represent an individual (or a chromosome); the former is called a genotype representation of an individual and the latter a phenotype representation of an individual.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The items that lead to the event is not a single real number but a set of discrete genotypes. To encode them as a real number is to obfuscate the discrete separations between multiple discrete genotypes that impact any given risk event, such as a lion, the attack of which can potentially eliminate the genotypes and the entire DNA strand of which they are a part. Neither a real number nor its digital approximation would be a straightforward and efficient representation of separate genetic characteristics that may impact a large number of genetic expressions, protein manufacturing runs, phenotypes, more continuous physical traits, and probabilistic advantages related to risks to the spawning of offspring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the confusion expressed in the question may be founded in a legitimate sense of a flaw in the logic of the presented material. Of course, that does not discount the possible expertise and legitimacy of other information presented in the same text.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-27T22:11:48.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10259" PostTypeId="1" AcceptedAnswerId="10260" CreationDate="2019-01-28T04:52:15.480" Score="5" ViewCount="75" Body="&lt;p&gt;I needed to make a system for recognizing people based on hundreds of text by finding similarities in their written text grammatically or similarities between words they choose for writing, I don't want it so accurate but wanted to know if it is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, finding one person with two account or more on a forum or something in that case (texts already gathered). I'm just wondering if it's possible and what field should I research in for.&lt;/p&gt;&#xA;" OwnerUserId="21811" LastEditorUserId="20322" LastEditDate="2019-05-17T20:27:12.287" LastActivityDate="2021-03-07T08:56:51.660" Title="Is it possible to find same persons based on text analyzing?" Tags="&lt;neural-networks&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10260" PostTypeId="2" ParentId="10259" CreationDate="2019-01-28T05:18:41.047" Score="4" Body="&lt;p&gt;The term you are looking for is &lt;a href=&quot;https://en.wikipedia.org/wiki/Stylometry&quot; rel=&quot;nofollow noreferrer&quot;&gt;stylometry&lt;/a&gt;, which is related to a technique in forensic linguistics called &lt;a href=&quot;https://en.wikipedia.org/wiki/Writeprint&quot; rel=&quot;nofollow noreferrer&quot;&gt;writeprint analysis&lt;/a&gt;. There are many different techniques to perform stylometric analysis, from the very basic 5-feature analysis classifying features such as the lexicon and idiosyncrasies unique to a person to more complex analysis utilizing neural networks and machine learning. Searching online for research papers focusing on stylometry should assist you in finding the best technique for the job.&lt;/p&gt;&#xA;" OwnerUserId="20322" LastEditorUserId="20322" LastEditDate="2021-03-07T08:56:51.660" LastActivityDate="2021-03-07T08:56:51.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10261" PostTypeId="2" ParentId="6274" CreationDate="2019-01-28T05:29:30.023" Score="1" Body="&lt;p&gt;As you want to perform image segmentation, you can use &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net&lt;/a&gt;, which does not have fully connected layers, but it is a &lt;a href=&quot;https://arxiv.org/pdf/1411.4038.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;fully convolutional network&lt;/a&gt;, which makes it able to handle inputs of any dimension. You should read the linked papers for more info.&lt;/p&gt;&#xA;" OwnerUserId="21812" LastEditorUserId="2444" LastEditDate="2020-06-13T20:21:01.373" LastActivityDate="2020-06-13T20:21:01.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10263" PostTypeId="2" ParentId="6699" CreationDate="2019-01-28T06:33:07.797" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/236132090_Face_detection_evaluation_A_new_approach_based_on_the_golden_ratio_PH&quot; rel=&quot;nofollow noreferrer&quot;&gt;Face detection evaluation: A new approach based on the golden ratio Φ&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Face detection is a fundamental research area in computer vision field. Most of the face-related applications such as face recognition and face tracking assume that the face region is perfectly detected. To adopt a certain face detection algorithm in these applications, evaluation of its performance is needed. Unfortunately, it is difficult to evaluate the performance of face detection algorithms due to the lack of universal criteria in the literature. In this paper, we propose a new evaluation measure for face detection algorithms by exploiting a biological property called Golden Ratio of the perfect human face. The new evaluation measure is more realistic and accurate compared to the existing one. Using the proposed measure, five haar-cascade classifiers provided by Intel©OpenCV have been quantitatively evaluated on three common databases to show their robustness and weakness as these classifiers have never been compared among each other on same databases under a specific evaluation measure. A thoughtful comparison between the best haar-classifier and two other face detection algorithms is presented. Moreover, we introduce a new challenging dataset, where the subjects wear the headscarf. The new dataset is used as a testbed for evaluating the current state of face detection algorithms under the headscarf occlusion&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="21815" LastEditorUserId="1671" LastEditDate="2019-01-28T20:35:20.820" LastActivityDate="2019-01-28T20:35:20.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10264" PostTypeId="1" AcceptedAnswerId="10269" CreationDate="2019-01-28T09:27:59.997" Score="2" ViewCount="89" Body="&lt;p&gt;I have the following program for my neural network:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n_steps = 9&#xA;n_inputs = 36&#xA;n_neurons = 50&#xA;n_outputs = 1&#xA;n_layers = 2&#xA;learning_rate = 0.0001&#xA;batch_size =100&#xA;n_epochs = 1000#200 &#xA;train_set_size = 1000&#xA;test_set_size = 1000&#xA;tf.reset_default_graph()&#xA;X = tf.placeholder(tf.float32, [None, n_steps, n_inputs],name=&quot;input&quot;)&#xA;y = tf.placeholder(tf.float32, [None, n_outputs],name=&quot;output&quot;)&#xA;layers = [tf.contrib.rnn.LSTMCell(num_units=n_neurons,activation=tf.nn.relu6, use_peepholes = True,name=&quot;layer&quot;+str(layer))&#xA;         for layer in range(n_layers)]    layers.append(tf.contrib.rnn.LSTMCell(num_units=n_neurons,activation=tf.nn.relu6, use_peepholes = True,name=&quot;layer&quot;+str(layer)))&#xA;multi_layer_cell = tf.contrib.rnn.MultiRNNCell(layers)&#xA;rnn_outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)&#xA;stacked_rnn_outputs = tf.reshape(rnn_outputs, [-1, n_neurons]) &#xA;stacked_outputs = tf.layers.dense(stacked_rnn_outputs, n_outputs)&#xA;outputs = tf.reshape(stacked_outputs, [-1, n_steps, n_outputs])&#xA;outputs = outputs[:,n_steps-1,:]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to know whether my network is fully connected or not?&lt;br&gt;&#xA;When I try to see the variables, I see:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;multi_layer_cell.weights&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[&amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/kernel:0' shape=(86, 200) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/bias:0' shape=(200,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/w_f_diag:0' shape=(50,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/w_i_diag:0' shape=(50,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/w_o_diag:0' shape=(50,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/kernel:0' shape=(100, 200) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/bias:0' shape=(200,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/w_f_diag:0' shape=(50,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/w_i_diag:0' shape=(50,) dtype=float32_ref&amp;gt;,&#xA; &amp;lt;tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/w_o_diag:0' shape=(50,) dtype=float32_ref&amp;gt;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I didn't understood whether each layer is getting the complete inputs or not.&lt;br&gt;&#xA;I want to know whether the following figure is correct for the above code:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ub8Qb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ub8Qb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not then what is the figure for the network? Please let me know.&lt;/p&gt;&#xA;" OwnerUserId="9126" LastEditorUserId="9126" LastEditDate="2019-01-28T09:47:23.183" LastActivityDate="2019-01-28T18:23:25.827" Title="Is my Neural Network program fully connected?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10265" PostTypeId="1" AcceptedAnswerId="10297" CreationDate="2019-01-28T10:16:21.447" Score="2" ViewCount="68" Body="&lt;p&gt;Is there a connection between the approximator network sizes in a RL task and the speed of convergence to an (near) optimal policy or value function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When thinking about this, I came across the following thoughts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If the network would be too small, the problem won't get enough representation and would never be solved, and the network would converge to its final state quickly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the network would be infinitely big (assuming no vanishing gradients and the likes), the network would converge to some (desirable) over-fitting, and the network would converge to its final state very slowly, if at all.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This probably means there is some golden middle ground.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which leads me to the interesting question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4. Assuming training time is insignificant relative to running the environment (like in real life environments), then if a network of size M converges to an optimal policy in average after N episodes, would changing M make a predictable change on N?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any research, or known answer to this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to know that there is no more need to increase the network size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to know if the current network is too large?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: please regard question 4 as the main question here.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="2444" LastEditDate="2019-02-20T16:42:28.743" LastActivityDate="2019-02-20T16:42:28.743" Title="Is there a relation between the size of the neural networks and speed of convergence in deep reinforcement learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10268" PostTypeId="2" ParentId="3161" CreationDate="2019-01-28T14:51:33.270" Score="1" Body="&lt;p&gt;The determination of likelihood of advancements in any science or technology over a decade are dependent upon several technical features of culture. Not all are technical because they are elements in hardware and software design. Some are.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Market forces&lt;/li&gt;&#xA;&lt;li&gt;Perpetuation of traditions in education&lt;/li&gt;&#xA;&lt;li&gt;Feasibility&lt;/li&gt;&#xA;&lt;li&gt;Availability of resources&lt;/li&gt;&#xA;&lt;li&gt;Cost of research&lt;/li&gt;&#xA;&lt;li&gt;Cost of deployment and sustenance&lt;/li&gt;&#xA;&lt;li&gt;Natural cultural balance of curiosity and fear&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The low level cybernetic conversations with service applications like Siri and Cortana were correctly identified in the question, and Google Assistant should be included. If a survey is done of these conversations by working around public relations, perhaps by &lt;a href=&quot;https://duckduckgo.com/?q=funny+google+assistant+conversations&quot; rel=&quot;nofollow noreferrer&quot;&gt;using a competitor search engine to locate PR unfriendly records of conversations&lt;/a&gt;, the low level becomes immediately apparent&lt;sup&gt;1&lt;/sup&gt;. Anyone could confirm this by asking these service applications a set of the popular questions in a set of the StackExchange sites. Most if not all of them would produce answers that would be down voted if not deleted.&lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These service applications are efforts to meet the criteria of Alan Turing's thought experiment, which Turing called &lt;em&gt;The Immitation Game&lt;/em&gt;&lt;sup&gt;3&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How high levels is it going to reach (likely) in future years?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Prediction of AI milestones is not possible without defining first the milestone criteria to be met. Let's pick seven coveted ones.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A service application the answers of which can gain StackExchange reputation faster than humans, which is a more demanding version of Alan Turing's thought experiment&lt;/li&gt;&#xA;&lt;li&gt;A robot that can mow our lawns safely&lt;/li&gt;&#xA;&lt;li&gt;A &lt;a href=&quot;https://ai.stackexchange.com/questions/tagged/autonomous-vehicles&quot;&gt;car without steering wheel or pedals&lt;/a&gt; that can take us to places we typically go as safely as a median human driver can&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/10098/design-for-a-lovable-and-loving-ai&quot;&gt;A-pets&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;Automated theorem proving of currently problematic proofs, and not just verification&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/9745/stereo-matching-using-genetic-algorithm/&quot;&gt;Practical goal driven underwater robotic navigation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;A home tutor that produces a better distribution of college board exam results in students than public or private educational institutions, better meaning higher mean and lower standard deviations&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Based on the seven factors already listed above plus one more, the instantaneous state of AI research and its publication during each moment of the search for probabilities regarding the above seven milestone, some probabilistic predictions can be made about 2029. The bullet items primarily driving the numbers are at the top of this answer&lt;sup&gt;4, 5, 6&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&amp;lt; 1% &amp;mdash; bullet item 2 too low, and 7 too skewed toward fear&lt;/li&gt;&#xA;&lt;li&gt;40% &amp;mdash; bullet item 1 too low&lt;/li&gt;&#xA;&lt;li&gt;50% &amp;mdash; bullet items 1, 2, 3, and 7 all favorable&lt;/li&gt;&#xA;&lt;li&gt;2% &amp;mdash; bullet item 2 too low, and 7 too skewed toward fear&lt;/li&gt;&#xA;&lt;li&gt;&amp;lt; 1% &amp;mdash; bullet items 2 too high, 3 too low, and 7 too skewed toward fear&lt;/li&gt;&#xA;&lt;li&gt;80% &amp;mdash; bullet items 1 and 3&lt;/li&gt;&#xA;&lt;li&gt;30% &amp;mdash; bullet item 3 high, but 1, 2, 3, 5, and 7 are unfavorable&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are a few that some will find unpalatable. Service applications and automatic theorem provers are not at all likely to become smart in the way defined in the milestones within the next ten years. Robots will likely be swimming and assisting with mining, drilling, and troubleshooting. Level 5 automated vehicles may already be driving around town.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnotes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] All intelligent systems, including biological ones, have weaknesses and strengths, and the strengths are not perfectly reliable. For instance AI on a CAD application may select a screw fastener to attach one part to another in a sub-assembly with 100% accuracy 99.99999% of the time, seven 9s of accuracy. Much of the talk of general, strong, and weak AI fails the burden of mathematical rigor or scientific verifiability and is largely fabrication and self-deluding conjecture to get cushy, high profile jobs. When published or aired, it is done so not because of credibility but to get paid for ad impressions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Please don't run this experiment fully, since it would burden moderators that are volunteering to keep the sites clean. Ask the questions but do not post the answers you find. The superficiality will be obvious enough without a focus group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] See the section &lt;em&gt;The Turing Challenge to Cartesian Thought&lt;/em&gt; in the post &lt;a href=&quot;https://ai.stackexchange.com/questions/8219/can-a-brain-be-intelligent-without-a-body&quot;&gt;Can a brain be intelligent without a body?&lt;/a&gt; for more information on Turing's work in this space and its limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] The parenthetic &lt;em&gt;(likely)&lt;/em&gt; in the question in the body of the question is correctly placed. The question's tag &lt;em&gt;predicting-ai-milestones&lt;/em&gt; is also correct. The likelihoods listed will change because, as the search for likelihoods of AI milestones progresses, the eight factors listed, which are involved in the predictions, will change. This answer author may or may not change the predictions accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] The reasons behind these probabilities cannot be fully disclosed for contractual reasons, but what is disclosed is explained where it can be. Furthermore, the results of prediction are based on the information available to this particular answer author. Others may have other numerical numbers to offer based on other quantitative or qualitative information and can disclose what they wish and are legally allowed to do in another answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] There is no guarantee made or implied in this answer.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-28T14:51:33.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10269" PostTypeId="2" ParentId="10264" CreationDate="2019-01-28T17:56:52.160" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Is figure = code?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No.&#xA;Your figure shows a fully connected feed forward network (&lt;a href=&quot;https://en.wikipedia.org/wiki/Multilayer_perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLP&lt;/a&gt;). But in your code you are using a two layer &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory#Peephole_LSTM&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM with peepholes&lt;/a&gt;. For the visualization of LSTMs, blocks are usually used for each layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a figure of the LSTM with peepholes which is the base of the tensorflow implementation (Source: &lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43905.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper, fig. 1&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I0BPu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I0BPu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Why size 86?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input is concatenated with the hidden state:&#xA;&lt;code&gt;n_inputs + n_neurons = 36 + 50 = 86&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Why size 100 in the second layer?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second LSTM layer gets input of size 50 by the first layer (&lt;code&gt;n_neurons&lt;/code&gt;) which is concatenated with the hidden state of the second layer (of size &lt;code&gt;n_neurons = 50&lt;/code&gt;).&#xA;Therefore you get &lt;code&gt;50 + 50 = 100&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Why size 200?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are four weight matrices of size &lt;span class=&quot;math-container&quot;&gt;$86 \times 50$&lt;/span&gt; (fig.: colored circles and the g circle), which seem to be combined to one matrix (&lt;span class=&quot;math-container&quot;&gt;$4 \cdot 50$&lt;/span&gt;) of size &lt;span class=&quot;math-container&quot;&gt;$86 \times 200$&lt;/span&gt; (&lt;code&gt;layer0/kernel&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Why size 50?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The three variables &lt;code&gt;w_f_diag&lt;/code&gt;, &lt;code&gt;w_i_diag&lt;/code&gt; and &lt;code&gt;w_o_diag&lt;/code&gt; are for the peephole connections (fig: dashed lines) and they have the size of &lt;code&gt;n_neurons = 50&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8035" LastEditorUserId="8035" LastEditDate="2019-01-28T18:23:25.827" LastActivityDate="2019-01-28T18:23:25.827" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10272" PostTypeId="1" CreationDate="2019-01-28T20:27:34.760" Score="7" ViewCount="137" Body="&lt;p&gt;Can someone explain what is the process of learning? What does it mean to learn something?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastEditorUserId="2444" LastEditDate="2019-05-27T16:30:07.097" LastActivityDate="2019-05-27T16:30:07.097" Title="What does learning mean?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;theory&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10276" PostTypeId="2" ParentId="10247" CreationDate="2019-01-28T21:30:37.260" Score="3" Body="&lt;p&gt;Quoting &lt;em&gt;Evolving neural networks through augmenting topologies&lt;/em&gt;, p. 10 (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When crossing over, the genes in both genomes with the same innovation numbers are lined up. These genes are called matching genes. Genes that do not match are either disjoint or excess, depending on whether they occur within or outside the range of the other parent’s innovation numbers. They represent structure that is not present in the other genome. In composing the offspring, &lt;strong&gt;genes are randomly chosen from either parent at matching genes&lt;/strong&gt;, whereas all excess or disjoint genes are always included from the more fit parent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Innovation numbers are used to line up genes in different genomes so you can perform crossover on networks with different topologies. Each network can optimise the weights in matching genes in a different way, so the weights are not shared. If they were, crossover would have nothing to contribute towards diversifying the population.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2019-01-28T21:30:37.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10278" PostTypeId="2" ParentId="10272" CreationDate="2019-01-28T22:30:16.573" Score="2" Body="&lt;p&gt;That is a wonderfully fundamental question.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Learning is the use of a system to change another system so that, instead of doing what it did before, which may have been nothing, it does something else.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the human brain, the system is the way that genetic expression caused the directed mutability of that brain so that human intentions and responses to external stimuli would alter. The direction of alteration is based on incentives and deterrents. In classes of children, there are things a teacher may do or a culture in the school that motivate learning. Misbehavior is deterred through other mechanisms. In that case, the first system is the educational process and the second system is the brain's ability to use what it was taught.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even without classes, children learn things that create comfort, so comfort is the internal incentive. It is wired into us as a kind of teacher to dislike groin moisture, so we learn to use bathroom receptacles. It is wired into us to like praise from parents and teachers, so we tend to perform to get it. Things like that are complex losses and rewards and primary systems to incentivize learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning in artificial networks is not nearly as complex as in human brains. In some cases the artificial results are better. In other cases the abilities of the human brain cannot be approached by artificial networks yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The functioning of an artificial network often begins arbitrary and completely useless, but it's a parametric function, meaning the function can be changed by changing numbers called parameters. Each time the function is used to process an example drawn from a data set, the result is evaluated and the evaluation is used to modify the parameters. Care is taken to not over-modify the parameters, otherwise confusion can occur. Mathematicians call this type of system confusion &lt;em&gt;chaos&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeating this carefully directed and moderated process ideally leads to something called &lt;em&gt;convergence&lt;/em&gt;. The learning system is set up so that the result of convergence is a set of parameters that minimize losses, maximize gains, or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes initial learning is not enough and there are other related things to do later.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Adjusting learned behavior to adapt to new conditions or information&lt;/li&gt;&#xA;&lt;li&gt;Unlearning things that no longer produce benefit so that new learning can replace them&lt;/li&gt;&#xA;&lt;li&gt;Relearning things that had faded from memory&lt;/li&gt;&#xA;&lt;li&gt;Developing greater confidence in what was learned so that unlearning requires greater impetus&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are additional technical terms for the above concepts, more details that can be grasped, many categories of learning system types, varieties within those, and the mathematics that was used as a foundation to construct all this and make it work. Because the question was fundamental, those details and technicalities were omitted.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-28T22:30:16.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10279" PostTypeId="2" ParentId="10051" CreationDate="2019-01-28T23:15:25.607" Score="1" Body="&lt;p&gt;It seems that in the reference you provided they use attention to compute weights for the encoder representations based on the context provided by the decoder (Fig. 1). As far as I can tell, attention is applied &lt;em&gt;after&lt;/em&gt; convolution (in fact, after the GLU step), so it does not affect the feature maps directly. Rather, attention is used to select the target words in the decoder. In other papers (e.g., &lt;a href=&quot;http://www.aclweb.org/anthology/Q16-1019&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;), attention is applied directly to the feature maps in a way that is more similar to what you described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your second question, the paper you reference actually provides a &lt;a href=&quot;https://github.com/facebookresearch/fairseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to the source code they used. It is written in Lua (using Torch) instead of PyTorch, probably because PyTorch development was just starting when the paper was published. At any rate, you should be able to follow the Lua code and translate it into PyTorch.&lt;/p&gt;&#xA;" OwnerUserId="16101" LastActivityDate="2019-01-28T23:15:25.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10281" PostTypeId="1" AcceptedAnswerId="10283" CreationDate="2019-01-29T01:57:42.183" Score="2" ViewCount="177" Body="&lt;p&gt;I'm doing transfer learning using Inception on Tensorflow. The code that I used for training is &lt;a href=&quot;https://raw.githubusercontent.com/tensorflow/hub/master/examples/image_retraining/retrain.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://raw.githubusercontent.com/tensorflow/hub/master/examples/image_retraining/retrain.py&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take a look at the Argument Parser section at the bottom of the code, you will find these parameters :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;testing_percentage&lt;/li&gt;&#xA;&lt;li&gt;validation_percentage&lt;/li&gt;&#xA;&lt;li&gt;test_batch_size&lt;/li&gt;&#xA;&lt;li&gt;validation_batch_size&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So far, I understand that testing and validation percentage is the amount of images that we want to train at 1 time. But I don't really understand the use of test batch size and validation batch size. What is the difference between percentage and batch size?&lt;/p&gt;&#xA;" OwnerUserId="20612" LastActivityDate="2019-01-29T08:14:28.800" Title="Testing, Validation Percentage &amp; Test, Validation Batch Size Difference?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10283" PostTypeId="2" ParentId="10281" CreationDate="2019-01-29T08:14:28.800" Score="2" Body="&lt;p&gt;The &lt;strong&gt;percentages refer to the number of samples to use&lt;/strong&gt; (out of full dataset) as the validation and test datasets. So if you pass a dataset that consists of 100,000 samples to the model and set the validation and testing percentages to be 10% each, your model will train on 80,000 samples, validate them on 10,000 and save additional 10,000 samples for the final test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;batch sizes refer to the number of samples in each batch&lt;/strong&gt; during the test and validation evaluations.&#xA;Your model probably can't process 10,000 samples in a single run (due to memory limitations) so during evaluation it breaks the dataset into batches, which are processed sequentially and the result is the mean of all batches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are training, the batch size is an important hyper-parameter which has an affect on the properties and final results of the training process. During test/validation it has no affect and only needs to be small enough for your model to be able to run it (evaluation with different batch sizes will produce the same results). &lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2019-01-29T08:14:28.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10288" PostTypeId="1" CreationDate="2019-01-29T10:26:12.727" Score="0" ViewCount="42" Body="&lt;p&gt;As an experiment, I want to teach an ANN to play the game of &lt;a href=&quot;https://en.wikipedia.org/wiki/Nim&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nim&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The normal game is between two players and played with three heaps of any number of objects. The two players alternate taking any number of objects from any single one of the heaps. The goal is to be the last to take an object.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The game is easily solvable and I already wrote a small bot that can play Nim perfectly to provide data sets for supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am struggling with the design question, how should I output the solution to a specific board state. The answer always consists of two components:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How many stones to take (a more or less arbitrary integer value)&lt;/li&gt;&#xA;&lt;li&gt;Which heap to take the stones from (the index of the heap)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are available design choices in this regard and is there a state-of-the-art design for this type of problem?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2021-04-21T03:03:35.457" Title="How to design an ANN to give an answer that includes two different components?" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10289" PostTypeId="1" AcceptedAnswerId="12354" CreationDate="2019-01-29T11:08:22.457" Score="5" ViewCount="3058" Body="&lt;p&gt;By reading the abstract of &lt;a href=&quot;https://people.orie.cornell.edu/davidr/or474/nn_sas.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks and Statistical Models&lt;/a&gt; paper it would seem that ANNs are statistical models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast &lt;a href=&quot;https://towardsdatascience.com/no-machine-learning-is-not-just-glorified-statistics-26d3952234e3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning is not just glorified Statistics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a more concise/summarized answer with focus on ANNs.&lt;/p&gt;&#xA;" OwnerUserId="21269" LastEditorUserId="9947" LastEditDate="2019-01-29T11:24:22.153" LastActivityDate="2020-03-12T03:05:38.527" Title="Are neural networks statistical models?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;statistical-ai&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10294" PostTypeId="1" CreationDate="2019-01-29T14:23:01.237" Score="1" ViewCount="44" Body="&lt;p&gt;I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S -&amp;gt;&#xA;encoding of S (crisp) -&amp;gt;&#xA;S-L encoder -&amp;gt; S-L decoder -&amp;gt;&#xA;encoding of L (non-crisp, coming from decoder) -&amp;gt;&#xA;L -&amp;gt;&#xA;encoding of L (crisp) -&amp;gt;&#xA;L-T encoder -&amp;gt; L-T decoder -&amp;gt;&#xA;encoding of T (non-crisp, coming from decoder) -&amp;gt;&#xA;T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All of this can be implemented in Pytorch more or less adapting the usual encoder-decoder NMT. So, the layer of interlingua L acts as a somehow symbolic/discrete layer inside the whole S-L-T neural network. My question is - how such system can be trained in end-to-end (S-T) manner? The gradient propagates from the T to the L and at the L one should do some kind of &lt;strong&gt;symbolic gradient&lt;/strong&gt;? I. e. one should be able do compute the difference L1-L2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am somehow confused by such setting. My question is - is there similar networks which contain the symbolic representation as the intermediate layer and how one can train such system. I have heard about policy gradients but are they relevant to my setting?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially - if I denote some neural network by symbols x(Wi)y, then the training of this network means, that I change Wi and x stays intact. I.e. the last member of backpropagation equation has the form d.../dw1. But if I combine (&lt;strong&gt;chain!&lt;/strong&gt;) 2 neural networks x(Wi)y-y(Wj)z, then the the last backpropagation term for the y(Wj)z has the form (d.../dw1+d.../dy) and hence both the w1 and y should be changed/updated by the gradient descent too. So, doesn't some ambiguity arise here? Is such chaining of neural networks possible? Is is possible to train end-to-end chains of neural networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also thinking about use of evolutionary training.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2019-01-29T14:44:13.997" LastActivityDate="2019-01-29T15:41:08.930" Title="Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;gradient-descent&gt;&lt;policy-gradients&gt;&lt;machine-translation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10295" PostTypeId="2" ParentId="10294" CreationDate="2019-01-29T15:41:08.930" Score="1" Body="&lt;p&gt;The chain rule applies here as usual, and the term &lt;em&gt;symbolic gradient&lt;/em&gt; is interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How such might apply will depend much on the nature of the policy layer representation and the connections of that layer or layers to other non-policy layers.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A binary threshold layer may feed a logical inference layer.&lt;/li&gt;&#xA;&lt;li&gt;A sigmoid function might feed a fuzzy logic layer.&lt;/li&gt;&#xA;&lt;li&gt;The logical layer could be a production system adapted to look like a layer.&lt;/li&gt;&#xA;&lt;li&gt;The logical layer might be more like a PLD (programmable logic device).&lt;/li&gt;&#xA;&lt;li&gt;The output of the logical layer might be mapped one to one with a perceptron or an LSTM layer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the example in the question, we can't, from the information given in the question, assume that &lt;em&gt;discrete&lt;/em&gt; means a discrete approximation of a curve, since the term used is &lt;em&gt;discrete/symbolic&lt;/em&gt;. Since the nature of logic cannot be generalized in a differentiable closed form, it may be that it has to be probabilistically represented in closed form so a derivative may be obtained, if that is possible in the specific case under study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back propagation is a feedback mechanism to provide corrective signaling to the portions of the system that must be corrected. The hope is that the direction of the correction during back propagation leads to convergence on the global minimum of loss without severe delay, which is not always easy to make happen, thus Gaussian noise injection explicitly or indirectly through mini-batching, momentum, multiple initialization states, parallel searches with different hyper-parameters, and other techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One technique used quite a bit in analog systems and entering use in digital systems in recent years is the idea of multiple corrective mechanisms. Control theory for instrumentation and countermeasure aeronautics is thick with this concept. For the case given in this question, it can look like this, with &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; representing feed forward layer with some activation function, &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; representing some logical inference container, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the parameter set for the function of the same subscript, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is the evaluation function (error, loss, value, benefit), &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; representing the corrective feedback using the back propagation, the Jacobian, and the chain rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f_1 \quad \rightarrow \;\quad \ell \quad \rightarrow \quad\ f_2 \quad \rightarrow \quad\ \epsilon \\&#xA; p_1 \leftarrow \, b \leftarrow \ell\, ; \quad\quad\quad \; p_2 \, \leftarrow b \leftarrow \epsilon  \\&#xA;\quad\quad\quad\quad\quad \ell \quad\quad \leftarrow \quad\quad\quad\quad\quad \epsilon&#xA; $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the last line &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is simply a fact (piece of information) passed into the a rules engine session or a floating point representation of a feedback signal assigned as a probability to a rule in a fuzzy logical container. In the first half of the second line, the logical unit is responsible for providing the evaluation function output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Norbert Wiener's &lt;em&gt;Cybernetics&lt;/em&gt;, he states the following&lt;sup&gt;1&lt;/sup&gt; with regard to automating the steering of ships via rudder control, from which cybernetics got its name.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the important book of MacColl&lt;sup&gt;2&lt;/sup&gt;, we have an example of a complicated system which can be stabilized by two feedbacks but not by one.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He then procures the differential equations and proves MacColl's conclusion. That conclusion is used in nearly every instrumentation amplifier circuit on a chip sold today, if not all of them. There is no way to get the desired stability without the multiple feedback paths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, it may not be possible to stabilize the convergence of &lt;span class=&quot;math-container&quot;&gt;$f_1, \ell, and f_2$&lt;/span&gt; via a single mechanism of back-propagation using &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; and the Jacobians to descend a gradient.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Norbert Wiener's *Cybernetics, 1948, MIT, p 106 in the 1996 printing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] LeRoy A. MacColl. Fundamental Theory of Servomechanisms. Van Nostrand, New York, 1945&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-29T15:41:08.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10297" PostTypeId="2" ParentId="10265" CreationDate="2019-01-29T16:49:36.540" Score="0" Body="&lt;p&gt;Speed and size, no. That's because speed is dependent on processor clock periods and the effective parallel nature of the particular deep RL design in the environment, which is also dependent on cores and host clustering. Size is not really quantifiable in a way that is meaningful in this relationship because there is a broader and more complex structure of a network that might be trained to produce a value to use in the RL algorithm chosen.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Width for each layer&lt;/li&gt;&#xA;&lt;li&gt;Cell type and possibly activation function for each layer&lt;/li&gt;&#xA;&lt;li&gt;Hyper-parameters&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One can say that number of clock cycles times number of effective parallel processors is correlated to all the above, the overhead of the RL algorithm used, and the size of each data tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few inequalities developed in the PAC (probably approximately correct) framework, so it would not be surprising if there were some bounding rules for the relationship between clock cycles, effective parallelism, data widths, activation functions, and RL algorithms for deep RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Study of the algorithm, perhaps in conjunction with experimentation, may reveal the primary factors, essentially the processing bottlenecks. Further study of the factors involved in controlling loop iteration counts, which cause the bottleneck, could permit the quantification of computing effort required to maintain a particular maximum allowable response time, but that would be specific to a particular design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such might produce a metric that is effectively a count of clock cycles across the system's potentially parallel architecture for a worst case or mean RL action selection. That could then be used to determine the response time for a given system with all of the factors mentioned above fixed, including the priority and scheduling of the processes in each operating system involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a guess. Feel free to critique, since such a formulation is a project far beyond the effort that should be put into answering a question online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ t_r \propto e^{k_v} \, n_v \, \eta_v \, t_v \, \mu_v \, \sum c_v + e^{k_d} \, n_d \, \eta_d \, t_d \, \mu_d \, \sum c_d \; \text{,} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is time, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is tensor complexity, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is number of cores, &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt; is the effective efficiency of the parallel processing, &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the effective overhead cost of the glue code, &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the cycles require of the significant (and probably repetitive) elements in the algorithms, &lt;span class=&quot;math-container&quot;&gt;$t_r$&lt;/span&gt; is the total time to response, and the subscripts &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; designate the variable subscripted as either RL value determination metrics or deep network metrics respectively.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-01-29T16:49:36.540" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10300" PostTypeId="1" AcceptedAnswerId="10330" CreationDate="2019-01-29T22:09:05.903" Score="2" ViewCount="133" Body="&lt;p&gt;I'm trying to train a neural network on evaluating chess positions if rather white (0.0) or black would win (1.0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the input consists of 4 bits per chess field (piece id 0 - 12, equals 64*4). Factors like castling are being ignored for now. Also, all training sets are random positions from popular games where it's white's turn and the desired output is the outcome of the game (0.0, 0.5, 1.0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my input values the right choice?&#xA;How many hidden layers / neurons for each layer should be used and what's the best learning rate?&#xA;What type of NN's and which activation function would you recommend for this project?&lt;/p&gt;&#xA;" OwnerUserId="19783" LastActivityDate="2019-01-31T15:23:00.843" Title="Choosing the right neural network settings" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10301" PostTypeId="1" CreationDate="2019-01-29T23:29:56.810" Score="1" ViewCount="37" Body="&lt;p&gt;I would like to use chatbots for a research project. The chatbots should trigger different types of emotions during a 10 minute period of chatting. I'm thinking about using Mitsuku, Zo or Cleverbot (I prefer using Facebook Messenger or Skype).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option could be to let the participants do random smalltalk chatting. On the other hand, one could let the participants talk about specific topics where the bot is known to react differently (e.g. offensive, funny etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does somebody have a good idea how different emotions can be triggered using such a chatbot (or at least one specific emotion)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, an interesting alternative are game-based chatbots (like Lifeline). Does somebody know a good chatbot, ideally for Facebook Messenger or Skype, which is game-based (e.g. a playable, branching story)? The interaction should be text-based (i.e. writing with the chatbot) and not only pressing buttons. I'm thankful also for recommendations of other chatbots which could be useful.&lt;/p&gt;&#xA;" OwnerUserId="21103" LastActivityDate="2019-01-29T23:29:56.810" Title="How to use chatbots to trigger emotions?" Tags="&lt;intelligent-agent&gt;&lt;chat-bots&gt;&lt;emotional-intelligence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10302" PostTypeId="1" CreationDate="2019-01-30T01:11:23.697" Score="1" ViewCount="39" Body="&lt;p&gt;I am new in the field of genetic algorithms, and I want to learn to use them in practice.&#xA;&lt;strong&gt;How the genetic algorithm work and why it is applied ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="21865" LastEditorUserId="9515" LastEditDate="2019-01-30T17:28:08.980" LastActivityDate="2019-01-30T17:28:08.980" Title="What is the genetic algorithm for?" Tags="&lt;algorithm&gt;&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2019-01-30T17:30:49.690" ContentLicense="CC BY-SA 4.0" />
  <row Id="10303" PostTypeId="1" CreationDate="2019-01-30T08:20:37.823" Score="4" ViewCount="833" Body="&lt;p&gt;From what I understand, Monte Carlo Tree Search Algorithm is a solution algorithm for model free reinforcement learning (RL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model free RL means agent doesnt know the transition and reward model. Thus for it to know which next state it will observe and next reward it will get is for the agent to actually perform an action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my question is:&#xA;if that is the case, then how come the agent knows which state it will observe during the rollout, since rollout is just a simulation, and the agent never actually perform that action ? (it never really &lt;strong&gt;interact&lt;/strong&gt; with the environment: e.g it never really move the piece in a Go game during rollout or look ahead, thus cannot observed anything). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can only assume observing anything when not actually interacting with environment (during simulation) if it knows the transition model as I understand it. The same arguments goes for the rewards during rollout/ simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in this case, doesnt rollout in Monte Carlo Tree Search algorithm suggests that the agent knows the transition model and reward model and thus a solution algorithm for model &lt;strong&gt;based&lt;/strong&gt; reinforcement learning and not model &lt;strong&gt;free&lt;/strong&gt; reinforcement learning ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** it makes sense in Alphago, since the agent is trained to estimate what it would observed. but MCTS (without policy and value netwrok) method assumes that agent knows what it would observed even though no additional training is included.&lt;/p&gt;&#xA;" OwnerUserId="21872" LastEditorUserId="21872" LastEditDate="2019-01-30T09:47:12.167" LastActivityDate="2019-02-08T21:37:12.063" Title="Rollout algorithm like Monte Carlo search suggest model based reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;models&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10305" PostTypeId="2" ParentId="3518" CreationDate="2019-01-30T10:50:50.793" Score="0" Body="&lt;p&gt;Look up tournament selection&#xA;Tournament selection is a method of selecting an individual from a population of individuals in a genetic algorithm.[1] Tournament selection involves running several &quot;tournaments&quot; among a few individuals (or &quot;chromosomes&quot;) chosen at random from the population. The winner of each tournament (the one with the best fitness) is selected for crossover. Selection pressure, a probabilistic measure of a chromosome's likelihood of participation in the tournament based on the participant selection pool size, is easily adjusted by changing the tournament size[why?]. If the tournament size is larger, weak individuals have a smaller chance to be selected, because, if a weak individual is selected to be in a tournament, there is a higher probability that a stronger individual is also in that tournament.&lt;/p&gt;&#xA;" OwnerUserId="21876" LastActivityDate="2019-01-30T10:50:50.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10306" PostTypeId="1" AcceptedAnswerId="10481" CreationDate="2019-01-30T11:39:25.787" Score="4" ViewCount="1426" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;Status&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a few weeks now, I have been working on a Double DQN agent for the &lt;code&gt;PongDeterministic-v4&lt;/code&gt; environment, which you can find &lt;a href=&quot;https://github.com/hridayns/Reinforcement-Learning/tree/master/Atari&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A single training run lasts for about 7-8 million timesteps (about 7000 episodes) and takes me about 2 days, on Google Collab (K80 Tesla GPU and 13 GB RAM). At first, I thought this was normal because I saw a lot of posts talking about how DQNs take a long time to train for Atari games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Revelation&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then after cloning the OpenAI baselines &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;repo&lt;/a&gt;, I tried running &lt;code&gt;python -m baselines.run --alg=deepq --env=PongNoFrameskip-v4&lt;/code&gt;  and this took about 500 episodes and an hour or 2 to converge to a nice score of +18, without breaking a sweat. Now I'm convinced that I'm doing something terribly wrong but I don't know what exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Investigation&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After going through the DQN baseline &lt;a href=&quot;https://github.com/openai/baselines/tree/master/baselines/deepq&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; by OpenAI, I was able to note a few differences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I use the &lt;code&gt;PongDeterministic-v4&lt;/code&gt; environment but they use the &lt;code&gt;PongNoFrameskip-v4&lt;/code&gt; environment &lt;/li&gt;&#xA;&lt;li&gt;I thought a larger &lt;strong&gt;replay buffer size&lt;/strong&gt; was important, so I struggled (with the memory optimization) to ensure it was set to &lt;strong&gt;70000&lt;/strong&gt; but they set it to a mere &lt;strong&gt;10000&lt;/strong&gt;, and still got amazing results.&lt;/li&gt;&#xA;&lt;li&gt;I am using a normal &lt;strong&gt;Double DQN&lt;/strong&gt;, but they seem to be using a &lt;strong&gt;Dueling Double DQN&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Results/Conclusion&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have my doubts about such a huge increase in performance with just these few changes. So I know there is probably something wrong with my existing implementation. Can someone &lt;strong&gt;point me in the right direction&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any sort of help will be appreciated. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21513" LastActivityDate="2019-02-10T03:56:15.743" Title="Each training run for DDQN agent takes 2 days, and still ends up with -13 avg score, but OpenAi baseline DQN needs only an hour to converge to +18?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;open-ai&gt;&lt;deepmind&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10308" PostTypeId="1" AcceptedAnswerId="10326" CreationDate="2019-01-30T13:29:47.810" Score="2" ViewCount="72" Body="&lt;p&gt;In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least square error metric? If that is not the case(its not ALWAYS convex or reason WHY we use LMS), what is the reason then? why this metric changes for deep learning/neural networks but works for regression problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT]: Will this always be a convex function or is there any possibility that it will not be convex?&lt;/p&gt;&#xA;" OwnerUserId="18956" LastEditorUserId="18956" LastEditDate="2019-01-31T17:18:02.067" LastActivityDate="2019-01-31T17:18:02.067" Title="Will LMS always be convex function? If yes, then why do we change it for neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10311" PostTypeId="2" ParentId="10306" CreationDate="2019-01-30T15:17:46.523" Score="3" Body="&lt;p&gt;Dueling architectures create bigger differences in the values of actions in the state space. This is because the state-value &lt;strong&gt;V(s)&lt;/strong&gt; function is estimated separately from the state-action value &lt;strong&gt;Q(s, a)&lt;/strong&gt;. A new quantity, the advantage of an action, can then be defined as &lt;strong&gt;A(s, a) = Q(s, a) - V(s)&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Q function, however, measures the the value&#xA;  of choosing a particular action when in this state. The advantage&#xA;  function subtracts the value of the state from the Q&#xA;  function to obtain a relative measure of the importance of&#xA;  each action.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1511.06581.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dueling Network Architectures for Deep Reinforcement Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To better direct you, here are 2 resources that could really help you understand why those differences are important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/mlreview/speeding-up-dqn-on-pytorch-solving-pong-in-30-minutes-81a1bd2dff55&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speeding up DQN on PyTorch: how to solve Pong in 30 minutes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main points of the blog are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a larger batch size and play several steps before updating&lt;/li&gt;&#xA;&lt;li&gt;Play and train in a separate process&lt;/li&gt;&#xA;&lt;li&gt;Use asyncronous cuda transfers&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/higgsfield/RL-Adventure&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL Adventure&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This github library has easy to follow jupyter notebooks and links to all of the papers.&#xA;It includes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DQN&lt;/li&gt;&#xA;&lt;li&gt;Double DQN&lt;/li&gt;&#xA;&lt;li&gt;Dueling DQN&lt;/li&gt;&#xA;&lt;li&gt;Prioritized Experience Replay&lt;/li&gt;&#xA;&lt;li&gt;Noise Networks for Exploration&lt;/li&gt;&#xA;&lt;li&gt;Distributional RL&lt;/li&gt;&#xA;&lt;li&gt;Rainbow (That network that deepmind made that had so many things in it they couldn't find a good name)&lt;/li&gt;&#xA;&lt;li&gt;Distributional RL with Quantile Regression&lt;/li&gt;&#xA;&lt;li&gt;Hierarchical Deep RL&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-01-30T15:17:46.523" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10312" PostTypeId="2" ParentId="10303" CreationDate="2019-01-30T17:27:28.887" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;From what I understand, Monte Carlo Tree Search Algorithm is a solution algorithm for model free reinforcement learning (RL).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo Tree Search is a &lt;em&gt;planning&lt;/em&gt; algorithm. It can be considered part of RL, in a similar way to e.g. Dyna-Q.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a planning algorithm MCTS does need access to a model of the environment. Specifically it requires a &lt;em&gt;sampling&lt;/em&gt; model, i.e. a model that can accept a state and action, then return a single next state and reward with the same probabilities as the target system. The alternative model, used by other RL techniques such as Value Iteration, is a &lt;em&gt;distribution&lt;/em&gt; model which provides the full distribution of probabilities for rewards and next states. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if that is the case, then how come the agent knows which state it will observe during the rollout, since rollout is just a simulation, and the agent never actually perform that action ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is not the case. The agent knows what it will observe during the simulation because the simulation &lt;em&gt;is&lt;/em&gt; a sampling model.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this case, doesnt rollout in Monte Carlo Tree Search algorithm suggests that the agent knows the transition model and reward model and thus a solution algorithm for model based reinforcement learning and not model free reinforcement learning ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. This is how most planning algorithms work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simulation &lt;em&gt;can&lt;/em&gt; be driven purely by sampling from previous experience, which is how Dyna-Q works*. I would still consider that a model-based approach, and its success depends a lot on how well such a model can be learned. In many cases, variance due to errors in the learned model adversely affects learning. So MCTS works best in environments that can be accurately sampled, because they are strongly rules-driven. For example, board games.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Functionally DynaQ is almost identical to experience replay. So much so, that whether you consider it a planning algorithm or experience replay added to basic Q learning is more a matter of how you present the design of the learning agent - e.g. perhaps a designer wants to focus on the model-learning side more, so has code that explicitly represents the learned model.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-01-30T17:27:28.887" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10313" PostTypeId="1" CreationDate="2019-01-30T17:56:21.073" Score="0" ViewCount="42" Body="&lt;p&gt;Gradient training changes indiscriminately all the weights and nodes of the neural network. But one can imagine the situations when the training should be shaped, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One can put constraints on some of the weights. E.g. human brain contains regions whose inner connections are more dense that external connections with different regions. One can try to mimic this region-shaped structure in neural networks as well and hence one can require that inter-regional weights (in opposit to intra-regional weights) are close to zero (except, possibly, for some channels among regions);&lt;/li&gt;&#xA;&lt;li&gt;One can put constraints on some of the weights in such manner that some layer of neurons have specific structure. E.g. consider the popular encoder-decoder architecture of neural machine translation e.g. &lt;a href=&quot;https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html&lt;/a&gt; We can see that that the whole output of the encoder is expressed as a single layer of neurons which is forwarded to the input of the decoder. So - one can require that the set of all the possible outputs of the encoder (e.g. the possible values of this single layer of the neurons) forms some kind of structure, e.g. some grammar of some &lt;em&gt;interlingua&lt;/em&gt;. This example is for illustration only, I have in mind more complex neural network which has one layer of neurons which indeed should output the encoded words of some interlingua. So, one is required to guid all the weights of the encoder in such manner that this single layer has only allowable values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So - my question is - &lt;strong&gt;are there methods that guide the gradient descent training with additional information about the weights or about the nodes&lt;/strong&gt; (i.e. about the whole subsets of weights that have some impact on specific layer of nodes)? E.g. &lt;strong&gt;about methods that impress the region structure on the neural network or that constrains the values of some nodes to be in specific range only?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, it &lt;strong&gt;is quite easy to include such constraints in evolutionary neural networks&lt;/strong&gt; - one can simply reject the neural networks with weights that violates the constraints. But &lt;strong&gt;is it possible to do this in gradient-like training?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2019-06-26T18:30:27.613" Title="How to shape the weights or nodes during gradient training of neural network? Training with constraints?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;gradient-descent&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10314" PostTypeId="2" ParentId="10303" CreationDate="2019-01-30T17:59:25.407" Score="4" Body="&lt;p&gt;Whether or not MCTS is even a Reinforcement Learning algorithm at all &lt;a href=&quot;https://ai.stackexchange.com/q/7589/1641&quot;&gt;may be up for debate&lt;/a&gt;, but let's assume that we view it as an RL algorithm here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For practical purposes, MCTS really should be considered to be a Model-Based method.&lt;/strong&gt; Below, I'm going to describe how you could view it as a &lt;strong&gt;Model-Free RL approach&lt;/strong&gt; in some way... and then wrap back to why that viewpoint isn't really often useful in practice.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;More specifically, following &lt;a href=&quot;https://jair.org/index.php/jair/article/view/11099/26289&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, we'll think of an MCTS search process as a value-based RL algorithm (it learns estimates of a value function, very much like Sarsa, &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, etc.), which limits itself to learning values for the states that it chooses to represent by nodes in the search tree (this set of states that it chooses to represent gradually grows over time during the search process). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike traditional RL approaches, such an MCTS process doesn't really result in a policy or an exhaustive / generalizable value function estimator that can be extracted after the &quot;training&quot; process and re-used in many different states afterwards. We normally play a move after running MCTS, and then discard everything and start over again for the next move (maybe we'll keep a relevant part of the search tree and reuse that, but that's a minor detail... we certainly won't be able to re-use our search results in another match/game/episode).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;MCTS search process&lt;/strong&gt; itself can be viewed as a &lt;strong&gt;Model-Free RL approach&lt;/strong&gt;; every iteration of the search can be viewed as an actual episode of an &quot;agent&quot; that is collecting experience in a model-free manner in a &quot;real&quot; environment (but not as real as the game for which we're running the complete search process), where this &quot;internal agent&quot; first follows the Selection Policy for a while (e.g. UCB1), and then a Play-out policy for the remainder of the episode (e.g. uniform random). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;internal&quot; agent &quot;inside&quot; the MCTS iterations could be viewed as learning from a model-free RL process. The &lt;strong&gt;main problem with this view in practice&lt;/strong&gt; is that, because MCTS &quot;decides&quot; to have a laser-like focus on a relatively small subset of states (around the root node), this process really only leads to something useful being learned for that state in the root node (and possibly some of the closest children/grandchildren/etc.). We don't really learn something that can easily be re-used in the future in MCTS. &lt;strong&gt;What this means in practice is that we have to be able to re-run the complete &quot;Reinforcement Learning process&quot; (or search) whenever we need to make a decision&lt;/strong&gt; (i.e. every turn in a turn-based game). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is feasible if you have a simulator, or &lt;strong&gt;model&lt;/strong&gt; of the environment, in which you can do the learning... but then we really get back to actually have a &lt;strong&gt;model-based&lt;/strong&gt; approach.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Fun fact: if you like to take the viewpoint of MCTS as a Model-Free RL approach, you could also turn that into a Model-Based approach again by incorporating additional forms of planning/search &quot;inside&quot; the MCTS iterations. For example, you can &lt;a href=&quot;https://ai.stackexchange.com/q/9504/1641&quot;&gt;run little instances of MiniMax inside every MCTS iteration&lt;/a&gt;, and I suppose that would turn the approach into a Model-Based approach again even in this viewpoint.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-01-30T17:59:25.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10315" PostTypeId="1" CreationDate="2019-01-30T18:22:25.850" Score="1" ViewCount="167" Body="&lt;p&gt;I am creating a VAE for time series data using CNNs. The data has 4800 timesteps and 4 features. It is standardized and normalized. The network I am using is implemented in Keras as follows. I have used a MSE reconstruction error:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# network parameters&#xA;(_, seq_len, feat_init) = X_train.shape&#xA;input_shape = (seq_len, feat_init)&#xA;intermediate_dim = 512&#xA;batch_size = 128&#xA;latent_dim = 10&#xA;epochs = 10&#xA;img_chns = 3&#xA;filters = 32&#xA;num_conv = (2, 2)&#xA;epsilon_std = 1&#xA;&#xA;inputs = Input(shape=input_shape)&#xA;conv1 = Conv1D(16, 3, 2, padding='same', activation = 'relu', data_format = 'channels_last')(inputs)&#xA;conv2 = Conv1D(32, 2, 2, padding='same', activation = 'relu', data_format = 'channels_last')(conv1)&#xA;conv3 = Conv1D(64, 2, 2, padding='same', activation = 'relu', data_format = 'channels_last')(conv2)&#xA;flat = Flatten()(conv3)&#xA;hidden = Dense(intermediate_dim, activation='relu')(flat)&#xA;z_mean = Dense(latent_dim, name = 'z_mean')(hidden)&#xA;z_log_var = Dense(latent_dim, name = 'z_log_var')(hidden)&#xA;&#xA;def sampling(args):&#xA;    z_mean, z_log_var = args&#xA;    epsilon = K.random_normal(shape=(K.shape(z_mean)[0], latent_dim),&#xA;                              mean=0., stddev=epsilon_std)&#xA;    return z_mean + K.exp(z_log_var) * epsilon&#xA;&#xA;z = Lambda(sampling, output_shape=(latent_dim,))([z_mean, z_log_var])&#xA;&#xA;decoder_hid = Dense(intermediate_dim, activation='relu')(z)&#xA;decoder_upsample = Dense(38400, activation='relu')(decoder_hid)&#xA;decoder_reshape = Reshape((600,64))(decoder_upsample)&#xA;&#xA;deconv1 = Conv1D(filters=32, kernel_size=2, strides=1,&#xA;             activation=&quot;relu&quot;, padding='same', name='conv-decode1')(decoder_reshape)&#xA;upsample1 = UpSampling1D(size=2, name='upsampling1')(deconv1)&#xA;deconv2 = Conv1D(filters=16, kernel_size=2, strides=1,&#xA;             activation=&quot;relu&quot;, padding='same', name='conv-decode2')(upsample1)&#xA;upsample2 = UpSampling1D(size=2, name='upsampling2')(deconv2)&#xA;deconv3 = Conv1D(filters=8&#xA;                 , kernel_size=2, strides=1,&#xA;             activation=&quot;relu&quot;, padding='same', name='conv-decode3')(upsample2)&#xA;upsample3 = UpSampling1D(size=2, name='upsampling3')(deconv3)&#xA;x_decoded_mean_squash = Conv1D(filters=4&#xA;                 , kernel_size=4, strides=1,&#xA;             activation=&quot;relu&quot;, padding='same', name='conv-decode4')(upsample3)&#xA;&#xA;class CustomVariationalLayer(Layer):&#xA;    def __init__(self, **kwargs):&#xA;        self.is_placeholder = True&#xA;        super(CustomVariationalLayer, self).__init__(**kwargs)&#xA;&#xA;    def vae_loss(self, x, x_decoded_mean_squash):&#xA;        x = K.flatten(x)&#xA;        x_decoded_mean_squash = K.flatten(x_decoded_mean_squash)&#xA;        xent_loss = mse(x, x_decoded_mean_squash)&#xA;        kl_loss = - 0.5 * K.mean(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis=-1)&#xA;        return K.mean(xent_loss + kl_loss)&#xA;&#xA;    def call(self, inputs):&#xA;        x = inputs[0]&#xA;        x_decoded_mean_squash = inputs[1]&#xA;        loss = self.vae_loss(x, x_decoded_mean_squash)&#xA;        self.add_loss(loss, inputs=inputs)&#xA;        return x&#xA;&#xA;outputs = CustomVariationalLayer()([inputs, x_decoded_mean_squash])&#xA;&#xA;# entire model&#xA;vae = Model(inputs, outputs)&#xA;vae.compile(optimizer='adadelta', loss=None)&#xA;vae.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask whether it is possible for the network to nearly perfectly reconstruct the test timeseries when passed through the entire VAE network, but still output junk when using a random Normal input. For further details, here is one of the inputs and outputs when passing a test signal through the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MDd0C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MDd0C.png&quot; alt=&quot;inputs and outputs&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a reconstruction generated purely from a random sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RoPs8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RoPs8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be? Even if there was a posterior collapse, the VAE should still be able to generate a good output sample with a random input. To further test this I decided to split the network into two parts (encoder and decoder), and then pass the test image through it. The encoder and decoder networks were made by simply splitting the trained VAE network as follows:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;idx = 9 &#xA;input_shape = vae.layers[idx].get_input_shape_at(0)&#xA;&#xA;layer_input = Input(shape=(input_shape[1],)) &#xA;&#xA;x = layer_input&#xA;for layer in vae.layers[idx:-1]:&#xA;    x = layer(x)&#xA;&#xA;decoder = Model(layer_input, x)&#xA;decoder.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;idx = 0&#xA;input_shape = vae.layers[idx].get_input_shape_at(0)&#xA;&#xA;layer_input = Input(shape=input_shape)&#xA;&#xA;x = layer_input&#xA;for layer in vae.layers[idx + 1:7]:&#xA;     x = layer(x)&#xA;&#xA;encoder = Model(layer_input, x)&#xA;encoder.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Interestingly, I also got junk output here. I'm not sure how it is possible. If the model itself is getting a near perfect reconstruction, surely just passing an image through the encoder, extracting the latent mean, and then passing that latent mean through the decoder should also create a near perfect image?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something I am missing here?&lt;/p&gt;&#xA;" OwnerUserId="21883" LastActivityDate="2019-01-30T18:22:25.850" Title="How can VAE have near perfect reconstruction but still output junk when using random noise input" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;latent-variable&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10318" PostTypeId="1" CreationDate="2019-01-31T01:33:56.620" Score="3" ViewCount="103" Body="&lt;p&gt;Can I treat a stochastic policy (over a finite action space of size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;) as a deterministic policy (in the set of probability distribution in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that nothing is broken by making this mental translation, except that the &quot;induced environment&quot; now has to take a stochastic action and spit out the next state, which is not hard using on the original environment. Is this legit? If yes, how does this &quot;deterministify then DDPG&quot; approach compare to, for example, A2C?&lt;/p&gt;&#xA;" OwnerUserId="21892" LastEditorUserId="2444" LastEditDate="2019-04-04T16:37:46.727" LastActivityDate="2019-04-04T16:37:46.727" Title="Can I use deterministic policy gradient methods for stochastic policy learning?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10321" PostTypeId="2" ParentId="10300" CreationDate="2019-01-31T09:29:28.703" Score="1" Body="&lt;p&gt;Guessing the winner from a chess position is difficult for classification. In chess, even if you start from the same position, it can give you different result depends on the player's action. So, I recommend you to use Temporal Difference (TD) Learning, the driving concept behind &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some methods in Reinforcement Learning still use a neural net but not for predicting the winner. The prediction in Q-Learning, a popular Reinforcement Learning algorithm, predicts the &quot;value&quot; of choosing a certain action while in a certain position for a player. From those values, a player can choose the best action for the current position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following references might interest you:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning for chess&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep reinforcement learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://upcommons.upc.edu/bitstream/handle/2117/106389/119749.pdf?sequence=1&amp;amp;isAllowed=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Predicting the Outcome of a Chess Game by Statistical and Machine Learning techniques&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="4398" LastEditDate="2019-01-31T15:12:33.170" LastActivityDate="2019-01-31T15:12:33.170" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10322" PostTypeId="1" AcceptedAnswerId="10328" CreationDate="2019-01-31T10:35:34.240" Score="4" ViewCount="1045" Body="&lt;p&gt;I'm doing bachaleor thesis on traffic sign detection using single shot detector called YOLO. These single shot detectors can perform detection of objects in image and so they have specific way of training, ie. training on full images. Thats quite problem for me, because the biggest real dataset with full traffic sign images is &lt;a href=&quot;https://btsd.ethz.ch/shareddata/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Belgian one&lt;/a&gt; with 9000 images in 210 classes, which is unfortunately not enough to train good detector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To overcome this problem, I've created &lt;a href=&quot;https://github.com/kocica/DatasetGenerator&quot; rel=&quot;nofollow noreferrer&quot;&gt;DatasetGenerator&lt;/a&gt;, which does quite good job in generating synthetic datasets, you can see in the &lt;a href=&quot;https://github.com/kocica/DatasetGenerator/tree/master/results&quot; rel=&quot;nofollow noreferrer&quot;&gt;results directory&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently I came across GAN's which can (besides others) generate or extend existing dataset and I would like to use these networks to compare with my dataset generator. I've tried &lt;a href=&quot;https://medium.com/ai-society/gans-from-scratch-1-a-deep-introduction-with-code-in-pytorch-and-tensorflow-cb03cdcdba0f&quot; rel=&quot;nofollow noreferrer&quot;&gt;this introduction to GANs&lt;/a&gt; succesfully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is it's unsupervised learning and so there are no annotations. It means it's able to extend my dataset of traffic signs, but the generated dataset won't be annotated at all, which is problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Is there any way how to use GAN's to extend my dataset of full traffic sign images with annotations of traffic sign class and position? Actually the class is not important, because I can do it separately for each class, but what matters is the position of traffic sign in generated image.&lt;/p&gt;&#xA;" OwnerUserId="18760" LastEditorUserId="18760" LastEditDate="2020-03-07T08:39:44.073" LastActivityDate="2020-03-07T08:39:44.073" Title="Using GAN's to generate dataset for CNN training" Tags="&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10325" PostTypeId="2" ParentId="10322" CreationDate="2019-01-31T11:57:59.670" Score="0" Body="&lt;p&gt;You could add the desired traffic sign location to the latent vector and then arrange that the generator incurs loss if the traffic sign is not at the right place in the generated image. &lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-01-31T11:57:59.670" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10326" PostTypeId="2" ParentId="10308" CreationDate="2019-01-31T12:17:15.077" Score="2" Body="&lt;p&gt;Square loss is fine for regression, since minimizing it is the same as maximizing the likelihood of the model parameters (under assumption that the error is Gaussian). However, if the model directly produces probabilities, then it is natural to use these probabilities directly within the loss. Hence, in all classification models we prefer to minimize negative log-likelihood of the correct class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that choosing a natural loss leads to several practical advantages. In particular, applying a quadratic loss after a sigmoid activation leads to very poor gradients when the sigmoid is saturated in the wrong direction. The negative log-likelihood loss has no such problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This issue is not specific for neural networks. Logistic regression has used the negative log likelihood loss ever since 1958.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastEditorUserId="21726" LastEditDate="2019-01-31T17:17:14.343" LastActivityDate="2019-01-31T17:17:14.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10327" PostTypeId="1" CreationDate="2019-01-31T12:52:26.223" Score="2" ViewCount="402" Body="&lt;p&gt;I recently came across a paper on &lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42945.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Ranking&lt;/a&gt;. I was wondering whether this could be used to classify book covers as book titles. (For example, if I had a picture for the cover of the second HP book, the classifier would return Harry Potter and the Chamber of Secrets.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say I have a dataset of book covers along with the book titles in text. Could that data set be used for this deep ranking algorithm, or is there a much better way to approach my problem? I'm quite new to this whole thing, and this one of my first projects in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm trying to create is a mobile app where people can take a picture of a book cover, have an algorithm/neural net classify the title of the book, and then have some other algorithm connect that to the book's Goodreads page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="21905" LastEditorUserId="16565" LastEditDate="2019-02-01T13:34:17.180" LastActivityDate="2019-02-02T23:08:19.650" Title="Deep Ranking/Best way to classify book covers?" Tags="&lt;image-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10328" PostTypeId="2" ParentId="10322" CreationDate="2019-01-31T13:03:24.213" Score="3" Body="&lt;p&gt;I think you'll &lt;a href=&quot;https://machinelearning.apple.com/2017/07/07/GAN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;enjoy this work from Apple&lt;/a&gt; on improving the realism of synthetic images.  Essentially what you need to do is generate a synthetic image &lt;em&gt;then&lt;/em&gt; have your GAN &lt;em&gt;modify&lt;/em&gt; the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this loss function is going to take a little work!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Making synthetic data realistic enough to allow models to generalize successfully in the real world is a very active and exciting area of research, not least with respect to robotics, and so the work you are doing now should make you very attractive indeed to the right employer.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-01-31T13:03:24.213" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10329" PostTypeId="1" AcceptedAnswerId="10334" CreationDate="2019-01-31T13:23:24.273" Score="3" ViewCount="1782" Body="&lt;p&gt;I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, the agent performed an action based on the state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, and, because of that, the agent won or lost and there is no &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; to transition to.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;So, how do you update the Q function with the very last reward in that scenario, given that the state hasn't actually changed?&lt;/em&gt; That case &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; would equal &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; even though an action was performed and the agent received a reward (they ultimately won or lost, so quite important update to make!).&lt;/p&gt;&#xA;&lt;p&gt;Do I add extra inputs to the state &amp;quot;agent won&amp;quot; and &amp;quot;game finished&amp;quot;, and that's the difference between &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; for the final Q update?&lt;/p&gt;&#xA;&lt;p&gt;To make clear, this is in reference to a multi-agent/player system.  So, the final action the agent takes could have a cost/reward associated with it, but the subsequent actions other agents then take could further determine a greater gain or loss for this agent and whether it wins or loses. So, the final state and chosen action, in effect, could generate different rewards without the agent taking further actions.&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="2444" LastEditDate="2020-11-01T14:54:39.633" LastActivityDate="2020-11-01T14:54:39.633" Title="How to deal with the terminal state in SARSA in a multi-agent setting?" Tags="&lt;reinforcement-learning&gt;&lt;implementation&gt;&lt;multi-agent-systems&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10330" PostTypeId="2" ParentId="10300" CreationDate="2019-01-31T15:23:00.843" Score="2" Body="&lt;p&gt;Easy ones first:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Activations are going to be RELU all the way down, until your final softmax layer (win probability?) (because empirically, RELU does great on most problems, except when your model is an RNN, and it makes the gradient explode, or in the final layer of a regression model - numerical stability etc.).&lt;/li&gt;&#xA;&lt;li&gt;You probably to structure this with &lt;em&gt;some&lt;/em&gt; layers of convolutions with max pooling between them, then 1 or 2 fully connected layers (FC) near the end (because if you only have FC layers, you probably won't have enough data to train them)&lt;/li&gt;&#xA;&lt;li&gt;Well worth trying some 1D convolutions (which cleverly combine channels from convolutions created by previous layers).&lt;/li&gt;&#xA;&lt;li&gt;Learning rate: take the SGD default to begin with, then tune later. Problem dependent, I'm afraid!  The returns to tuning can be large though (my kids twigged to this quite quickly when playing with a toy problem).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now the hard bit - encoding your input:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Categorical encoding using a single integer per board position could cause your model some grief (it is an input that &quot;looks like&quot; a real number, but of course it isn't, and values that seem numerically close may represent pieces with radically different abilities (perhaps the code for King is 1 and Queen is 2 and Bishop is 3, but all those pieces have such different attributes).&lt;/li&gt;&#xA;&lt;li&gt;I would strongly consider thinking of each piece/player combo as a &quot;colour channel&quot; - a 64 cell grid where each value is either zero or one (so, one channel for White's pawns, another for White's knights, and so on).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Finally, those labels of yours: do think about what you'll do with &lt;em&gt;drawn&lt;/em&gt; games - perhaps you have 3 possible outcomes, not 2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would all be fascinated to hear how you get on - I hope you'll write your work up in some form (and that you'll come back and complain/praise our advice, as appropriate!).&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-01-31T15:23:00.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10331" PostTypeId="2" ParentId="10327" CreationDate="2019-01-31T16:11:48.633" Score="1" Body="&lt;p&gt;I read some papers talked about it you can take a look on them maybe help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/602807/deep-neural-network-learns-to-judge-books-by-their-covers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Neural Network Learns to Judge Books by Their Covers&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cs229.stanford.edu/proj2015/127_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classification of Book Genres By Cover and Title&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21907" LastActivityDate="2019-01-31T16:11:48.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10332" PostTypeId="2" ParentId="5939" CreationDate="2019-01-31T17:04:21.113" Score="1" Body="&lt;p&gt;I think context is important here. Using tactics like those used by Scotland Yard for over a century is probably the best way. Establishing alibis, realistic time lines, motives. For a legal setting, it would be possible to prove these images were fake using methods like this. From an I.T. perspective, it may be possible to pinpoint an origin for these images. If thousands of duplicitous images came from a single origin, then any images from this origin are suspect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, in general, we should retrain ourselves to not believe everything we see. There are so many methods for faking images, that photography can no longer be considered to be the best evidence of an event occurring. We should not ignore all images, but instead seek outside concurrence of facts before jumping to conclusions. If all facts point to an event happening, then that photograph is likely to be real. &lt;/p&gt;&#xA;" OwnerUserId="16959" LastEditorUserId="1641" LastEditDate="2019-01-31T19:42:35.673" LastActivityDate="2019-01-31T19:42:35.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10333" PostTypeId="2" ParentId="10210" CreationDate="2019-01-31T17:05:53.013" Score="1" Body="&lt;h1&gt;Check the function loss.&lt;/h1&gt;&#xA;&lt;p&gt;It might be that your environment is impossible to learn. However, most likely the network simply can't handle it. By measuring the loss during the learning stage, if you find it is always very high and does not decrease, it's a strong indication this might be the issue.&lt;/p&gt;&#xA;&lt;p&gt;Because the network is too simple, when you optimize for some states, you ruin others. There is not formal way to find out if this is the case, but since the same algorithm works elsewhere, it's either a problem of your environment, or of the network.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-01-31T17:05:53.013" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10334" PostTypeId="2" ParentId="10329" CreationDate="2019-01-31T17:49:33.080" Score="2" Body="&lt;p&gt;The SARSA update rule looks like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(S, A) \gets Q(S, A) + \alpha \left[ R + \gamma Q(S', A') \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Very similar, the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update rule looks like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(S, A) \gets Q(S, A) + \alpha \left[ R + \gamma \max_{A'} Q(S', A') \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Both of these update rules are formulated for single-agent Markov Decision Processes. Sometimes you can make them work reasonably ok in &lt;em&gt;multi-agent&lt;/em&gt; settings, but it is crucial to remember that these update rules should still always be implemented &amp;quot;from the perspective&amp;quot; of a single learning agent, who is oblivious to the presence of other agents and pretends them to be a part of the environment.&lt;/p&gt;&#xA;&lt;p&gt;What this means is that the states &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; that you provide in update rules &lt;strong&gt;really must both be states in which the learning agent is allowed to make the next move&lt;/strong&gt; (with the exception being that &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; is permitted to be a terminal game state.&lt;/p&gt;&#xA;&lt;p&gt;So, suppose that you have three subsequent states &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$S_3$&lt;/span&gt;, where the learning agent gets to select actions in states &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S_3$&lt;/span&gt;, and the opponent gets to select an action in state &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;. &lt;strong&gt;In the update rule, you should completely ignore&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;. This means that you should take &lt;span class=&quot;math-container&quot;&gt;$S = S_1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$S' = S_3$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Following the reasoning I described above literally may indeed lead to a tricky situation with rewards from transitioning into terminal states, since technically every episode there will be only one agent that directly causes the transition into a terminal state. This issue (plus also some of my explanation above being repeated) is discussed in the &lt;a href=&quot;https://ai.stackexchange.com/q/6573/1641&quot;&gt;&amp;quot;How to see terminal reward in self-play reinforcement learning?&amp;quot;&lt;/a&gt; question on this site.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-11-01T14:34:21.977" LastActivityDate="2020-11-01T14:34:21.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10336" PostTypeId="1" CreationDate="2019-01-31T22:27:12.157" Score="0" ViewCount="518" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/actor_critic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;This PyTorch implementation of the actor-critic algorithm&lt;/a&gt; calculates the losses like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;actor_loss = -log_prob * discounted_reward&#xA;policy_loss = F.smooth_l1_loss(value, torch.tensor([discounted_reward]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both are different from &lt;a href=&quot;https://medium.freecodecamp.org/an-intro-to-advantage-actor-critic-methods-lets-play-sonic-the-hedgehog-86d6240171d&quot; rel=&quot;nofollow noreferrer&quot;&gt;the regular formulas&lt;/a&gt; which are, in the case of the actor loss (parameterized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$log[\pi_\theta(s_t,a_t)]Q_w(s_t,a_t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and, in the case of the critic loss (parameterized by &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s_t,a_t) + \gamma Q_w(s_{t+1},a_{t+1}) - Q_w(s_{t},a_{t}),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r(s_t,a_t)$&lt;/span&gt; is the immediate reward following taking the action.&lt;/p&gt;&#xA;&lt;p&gt;For the actor, &amp;quot;the immediate critic evaluation of the transition&amp;quot; was replaced with &amp;quot;the discounted reward&amp;quot;. For the critic, the discounted evaluation of the value from the next state &lt;span class=&quot;math-container&quot;&gt;$r(s_t,a_t) + \gamma Q_w(s_{t+1},a_{t+1})$&lt;/span&gt; was replaced by &amp;quot;the discounted reward&amp;quot;. The &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; loss is then calculated, effectively discarding the sign of the (equation) loss.&lt;/p&gt;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why did they make these changes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is the sign discarded for the critic loss?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="2444" LastEditDate="2020-11-01T16:23:42.703" LastActivityDate="2020-11-01T16:27:38.797" Title="Why is this PyTorch implementation of the actor-critic algorithm inconsistent with the mathematical formulas?" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;pytorch&gt;&lt;actor-critic-methods&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10339" PostTypeId="1" CreationDate="2019-02-01T05:36:14.553" Score="0" ViewCount="87" Body="&lt;p&gt;I am working on a project where the Neural Network weights must be quantized on 8 or 16 bits for an embedded platform, thus I will lose some precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since our platform does not have floating point arithmetic we need to quantize the weights. By quantizing i mean taking the max absolute value of the weights and divide it by the maximum signed number representable on 8 or 16 bits. this operation will give us a quantization factor &lt;span class=&quot;math-container&quot;&gt;$(qf)$&lt;/span&gt;.&#xA;the final quantized weights will be integer&lt;span class=&quot;math-container&quot;&gt;$(value * qf)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my weights are very sparse and have a very bad distribution, I lose more precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, to the left here is the distribution of weights for one layer, and to the right is the distribution of weights after I added to the loss function the Kurtosis and skew measures of the weights, and it improved a bit the shape of the distribution while keeping the same accuracy, even a bit higher.&lt;a href=&quot;https://i.stack.imgur.com/XPQOG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XPQOG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody have any other suggestions? Has anyone tackled this problem before?&lt;/p&gt;&#xA;" OwnerUserId="21918" LastEditorUserId="9947" LastEditDate="2019-04-19T13:47:45.813" LastActivityDate="2019-04-19T13:47:45.813" Title="Tips for keeping the distribution of weights normal" Tags="&lt;neural-networks&gt;&lt;hardware&gt;&lt;quantification&gt;" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10340" PostTypeId="1" CreationDate="2019-02-01T06:44:28.800" Score="1" ViewCount="322" Body="&lt;p&gt;Is the role played by activation function significant only during the training of neural network or they play their role during testing (after training we supply data for prediction) the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that a linear line cannot separate data scattered in complex manner but&#xA;Then why we don't used simple polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;why specifically sigmoid, or tanh or ReLu what exactly they are doing ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What Activation functions do when we are supplying data during training and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And when we supply test data once we have trained the network and we input test data for prediction?&lt;/p&gt;&#xA;" OwnerUserId="21642" LastEditorUserId="21642" LastEditDate="2019-02-01T08:23:37.180" LastActivityDate="2019-10-29T09:02:10.280" Title="What role the activation function plays in the forward pass and how it is different from backpropagation" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2021-01-22T00:20:53.733" ContentLicense="CC BY-SA 4.0" />
  <row Id="10341" PostTypeId="2" ParentId="10340" CreationDate="2019-02-01T06:59:41.220" Score="2" Body="&lt;p&gt;Activation function is a non-linear function. Operation in a neuron without activation function is just a linear function. If we don't put activation function between operation of neurons, then the function &quot;Layer&quot; is useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example if you have two layer network, when you are doing forward-propagation, your output (without activation function) of your first layer will be calculated as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O_1 = W_1X+b_1 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your output of your second layer will be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O_2 = W_2O_1+b_2 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we substitute &lt;span class=&quot;math-container&quot;&gt;$O_1$&lt;/span&gt;, so the output of your second layer can be calculated as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O_2 = W_2(W_1X+b_1)+b_2 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or simply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O_2 = W_2W_1X+W_2b_1+b_2 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we train neural network to optimize the value of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; (we train to find the best value of it) so instead of training neural network with two layers, we actually just train a one layer network. From the latter formula we can said &lt;span class=&quot;math-container&quot;&gt;$W_2W_1 = W_3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_2b_1+b_2 = b_3$&lt;/span&gt; so our two layer network is just another linear model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O_2 = W_3X+b_3 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't want that, we add layers to get more complex model. That's why we use Activation function that is non-linear function. To prevent our deep model is just become a simple linear function.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-02-01T06:59:41.220" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10342" PostTypeId="2" ParentId="10336" CreationDate="2019-02-01T07:45:00.743" Score="1" Body="&lt;p&gt;There is no 'regular' formula for calculating policy loss, the regular thing when calculating policy gradient is to multiply gradient with advantage function which can be many things. Look at section 2 of &lt;a href=&quot;https://arxiv.org/pdf/1506.02438.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper for coverage on basic advantage functions. Also, the expected discounted reward is the same thing as the state-action value function (Q value).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q^\pi(s_t, a_t) = \mathbb{E}_{s_{t+1:\infty}, a_{t+1:\infty}}[\sum_{l=0}^\infty \gamma^lr_{t+l}]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, the variations you posted roughly calculate the same thing.&lt;/p&gt;&#xA;&lt;p&gt;Regarding the negative sign, in policy gradient methods, we want to maximize our performance function which has the following form:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\theta) = \sum_\limits{\substack{a}} \pi(a \mid s, \theta)A(s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, the higher our performance the better it is. When people write code, they use minimizers that minimize the objective function, but, in this case, we want to maximize it, so maximizing the objective function is the same thing as minimizing the negative objective function therefore the negative sign.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="2444" LastEditDate="2020-11-01T16:27:38.797" LastActivityDate="2020-11-01T16:27:38.797" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10343" PostTypeId="2" ParentId="10228" CreationDate="2019-02-01T10:18:47.137" Score="2" Body="&lt;p&gt;The answer depends on a case to case basis. It may so happen that a dataset performs very well on ReLu but takes more number of iterations to converge on leaky ReLu's or PReLu's or vice-versa. There are 2 arguments to consider here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReLu is the &lt;strong&gt;most non-linear&lt;/strong&gt; among all other type of ReLu's, and by this not so mathematical term I mean to say that it has the steepest drop in slope at 0, as compared to any other type of modified ReLu's.&lt;/li&gt;&#xA;&lt;li&gt;ReLu's omit negative values which can be a significant problem with context to data normalisation. As this &lt;a href=&quot;https://www.youtube.com/watch?v=wEoyxE0GP2M&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=6&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; &lt;code&gt;(~10:00)&lt;/code&gt; from Stanford explains how data normalisation is necessary in context of &lt;strong&gt;&lt;code&gt;signs&lt;/code&gt;&lt;/strong&gt; of weight updates, so we can very roughly say any form of Leaky ReLU's somewhat normalise the data.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So theoretically speaking (might not be mathematically rigorous) iff all the inputs have a positive correlation to the output(input increases, output also increases), ReLu should work very well and converge faster. Whereas if there is negative correlation as well then Leaky ReLu's might work better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that unless someone gives definitive mathematical relations of what's going on inside a NN when it is being trained, its hard to tell which will work well and which will not except from intuition.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-04-02T07:48:20.057" LastActivityDate="2019-04-02T07:48:20.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10344" PostTypeId="1" CreationDate="2019-02-01T10:20:44.013" Score="4" ViewCount="572" Body="&lt;p&gt;I've watched this video of the recent contest of AlphaStar Vs Pro players of StarCraft2, and during the discussion David Silver of DeepMind said that they train AlphaStar on TPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how is it possible to utilise a GPU or TPU for reinforcement learning when the agent would need to interact with an environment, in this case is the StarCraft game engine?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment with my training of a RL agent I need to run it on my CPU, but obviously I'd love to utilise the GPU to speed it up. Does anyone know how they did it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the part where they talk about it, if anyone is interested:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=cUTMhmVh1qs&amp;amp;t=7030s&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=cUTMhmVh1qs&amp;amp;t=7030s&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-02-01T16:10:52.227" Title="How does DeepMind perform reinforcement learning on a TPU?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deepmind&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10345" PostTypeId="1" CreationDate="2019-02-01T10:24:15.850" Score="2" ViewCount="231" Body="&lt;p&gt;Suppose we have a data set consists of columns&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TransactionId, CardNo, TransactionDate&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;then how can we calculate the customer purchase interval (means if customer A purchased on Jan 1st and after 10 days he again purchased, and then he again purchased after 15 days.) and how to predict the next visit of customer A by analyzing the purchasing intervals of customer A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16770" LastActivityDate="2019-03-05T20:55:25.323" Title="predict customer visit" Tags="&lt;python&gt;&lt;forecasting&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10347" PostTypeId="2" ParentId="3494" CreationDate="2019-02-01T12:04:56.220" Score="4" Body="&lt;p&gt;It's a mix of many factors that together make it a very good option to develop cognitive systems.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Quick development&lt;/li&gt;&#xA;&lt;li&gt;Rapid prototyping&lt;/li&gt;&#xA;&lt;li&gt;Friendly syntax with almost human-level readability&lt;/li&gt;&#xA;&lt;li&gt;Diverse standard library and multi-paradigm&lt;/li&gt;&#xA;&lt;li&gt;It can be used as a frontend for performant backends written in compiled languages such as C/C++.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Existing performant numerical libraries, such as numpy and others already do the intensive bulk work for you which lets you focus more on architectural aspects of your system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, there is a very big community and ecosystem around Python, which results in a diverse set of available tools oriented to diffent kind of tasks. &lt;/p&gt;&#xA;" OwnerUserId="9623" LastActivityDate="2019-02-01T12:04:56.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10348" PostTypeId="2" ParentId="1877" CreationDate="2019-02-01T13:47:48.127" Score="0" Body="&lt;p&gt;Typical learning algorithms can be stated as a search problem, where we want to find the best possible solution, that successfully solves a particular task, among all the available candidate solutions in the solution space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is often the case where we can't find the best one or it is too hard to find it and thus we compromise with a sub-optimal solution.&lt;/p&gt;&#xA;" OwnerUserId="9623" LastActivityDate="2019-02-01T13:47:48.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10349" PostTypeId="1" CreationDate="2019-02-01T14:00:40.307" Score="1" ViewCount="19" Body="&lt;p&gt;I have a sensor that reads electromagnetic field strength from each position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the field is stable and unique for each position. So the reading is simply a function of the position like this: &lt;code&gt;reading = emf(x,y,z)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reading consists of 3 numbers (not position).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the inverse function of &lt;code&gt;emf&lt;/code&gt; function. Which means I want to find function &lt;code&gt;pos&lt;/code&gt; that is defined like this: &lt;code&gt;x,y,z = pos(reading)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have access to both &lt;code&gt;emf&lt;/code&gt; and &lt;code&gt;pos&lt;/code&gt; function. I think that I want to gradually estimate the &lt;code&gt;pos&lt;/code&gt; function using a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have input &lt;code&gt;reading&lt;/code&gt; and acceleration &lt;code&gt;ax,ay,az&lt;/code&gt; of the sensor through space from an IMU. The acceleration is not so accurate.&#xA;I want to use these 2 inputs to help me figure out the position of the sensor over time. You can assume that the starting position is at 0,0,0 on the first reading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, input is &lt;code&gt;reading&lt;/code&gt; and &lt;code&gt;ax,ay,az&lt;/code&gt; on each timestep, output will be adjustment on the weights of &lt;code&gt;pos&lt;/code&gt; function or output will be position directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been reading about &lt;strong&gt;SLAM&lt;/strong&gt; (simultaneous localization and mapping) algorithm and I think that it might help in my case because my problem is probabilistic. &#xA;If I know accurately the acceleration, I would not need any probability, but the acceleration is not accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to know how do I model this problem in term of &lt;strong&gt;SLAM&lt;/strong&gt;?&#xA;I don't have a camera to do vision-based SLAM though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why I think this is tractable? If the first reading is &lt;code&gt;1,1,1&lt;/code&gt; and the position is at origin &lt;code&gt;0,0,0&lt;/code&gt;, and I move the sensor, the position can drift because the sensor has never seen other reading before, but after I go back to origin, the reading will be &lt;code&gt;1,1,1&lt;/code&gt; again so the sensor should report the origin &lt;code&gt;0,0,0&lt;/code&gt; as output. During the movement of the sensor, the algorithm should filter the acceleration so that all the previous positions makes sense.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-02-01T14:00:40.307" Title="How to use SLAM on other sensor other than camera?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;probability&gt;&lt;bayes&gt;&lt;mapping-space&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10351" PostTypeId="2" ParentId="10344" CreationDate="2019-02-01T16:10:52.227" Score="3" Body="&lt;p&gt;In their &lt;a href=&quot;https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, they link to (among many other papers) their &lt;a href=&quot;https://arxiv.org/pdf/1802.01561.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;IMPALA paper&lt;/a&gt;. Now, the blog post only links to that paper with text implying that they're using the &lt;em&gt;&quot;off-policy actor-critic reinforcement learning&quot;&lt;/em&gt; described in that paper, but one of the major points of the IMPALA paper is actually an efficient, large-scale, distributed RL setup. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, until we get more details (for example in their paper that's currently under review), our best guess would be that they're also using a similar kind of distributed RL setup as described in the IMPALA paper. As depicted in Figures 1 and 2, they &lt;strong&gt;decouple actors&lt;/strong&gt; (machines running code to generate experience, e.g. by playing StarCraft) &lt;strong&gt;and learners&lt;/strong&gt; (machines running code to learn/train/update weights of neural network(s)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would assume that their TPUs are definitely being used by the Learner (or, likely, multiple Learners). StarCraft 2 itself won't benefit from running on TPUs (and probably would be impossible to even get to run on them in the first place), because the game logic likely doesn't depend on large-scale, dense matrix operations (the kinds of operations that TPUs are optimized for). So, the StarCraft 2 game itself (which only needs to run for the &quot;actors&quot;, not for the &quot;learners&quot;) is almost certainly running on CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actors will still have to run forwards passes through Neural Networks in order to select actions. I would assume that their Actors are still equipped with either GPUs or TPUs to do this more quickly than a CPU would be capable of, but the more expensive backwards passes are not necessary here; only the Learners need to perform those. &lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-01T16:10:52.227" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10352" PostTypeId="1" AcceptedAnswerId="10362" CreationDate="2019-02-01T17:49:20.307" Score="1" ViewCount="102" Body="&lt;p&gt;In &lt;a href=&quot;https://datascience.stackexchange.com/q/20535/10640&quot;&gt;experience replay&lt;/a&gt;, the update rule follows the loss:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_i(\theta_i) = \mathbb{E}_{(s_t, a_t, r_t, s_{t+1}) \sim U(D)} \left[ \left(r_t + \gamma \max_{a_{t+1}} Q(s_{t+1}, a_{t+1}; \theta_i^-) - Q(s_t, a_t; \theta_i)\right)^2 \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I can't get my head around the order of calculation of the terms in that equation:&lt;/p&gt;&#xA;&lt;p&gt;An experience element is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, r_t, s_{t+1} )$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; is the state at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; is the action taken from &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; is the reward received by taking that action from &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; at time&#xA;&lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; is the next state&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://qr.ae/pNb5Z7&quot; rel=&quot;nofollow noreferrer&quot;&gt;the on policy case&lt;/a&gt;, &lt;strong&gt;as I understand it&lt;/strong&gt;, Q of the equation above is the same Q, which is the only approximator.&lt;/p&gt;&#xA;&lt;p&gt;As I understand the algorithm, at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; we save an experience&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, r_t, s_{t+1} )$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then, later, at time &lt;span class=&quot;math-container&quot;&gt;$t+x$&lt;/span&gt;, we attempt to learn from that experience.&lt;/p&gt;&#xA;&lt;p&gt;However, at the time of saving the experience, &lt;span class=&quot;math-container&quot;&gt;$Q(s_t,a_t)$&lt;/span&gt; was something different than at the time of attempting to learn from that experience, because the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; have since changed. This could actually be written as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q_t(s_t, a_t) \neq Q_{t+x}(s_t,a_t)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Because the Q value is different, I don't see how the reward signal at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is of any relevance for &lt;span class=&quot;math-container&quot;&gt;$Q_{t+x}(s_t,a_t)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$t+x$&lt;/span&gt;, the time of learning.&lt;/p&gt;&#xA;&lt;p&gt;Also, it is likely that following a policy that is derived from &lt;span class=&quot;math-container&quot;&gt;$Q_t$&lt;/span&gt; would lead to &lt;span class=&quot;math-container&quot;&gt;$a_{t}$&lt;/span&gt;, whereas following a policy that is derived from &lt;span class=&quot;math-container&quot;&gt;$Q_{t+x}$&lt;/span&gt; would not.&lt;/p&gt;&#xA;&lt;p&gt;I don't see in the experience replay algorithm that the Q value &lt;span class=&quot;math-container&quot;&gt;$Q_t(s_t, a_t)$&lt;/span&gt; is saved, so I must assume that is not.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why does calculating the Q value again at a later time make sense FOR THE SAME SAVED REWARD AND ACTION?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="2444" LastEditDate="2020-11-01T10:36:49.693" LastActivityDate="2020-11-01T10:36:49.693" Title="When are Q values calculated in experience replay?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;experience-replay&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10354" PostTypeId="2" ParentId="10352" CreationDate="2019-02-01T19:11:22.233" Score="0" Body="&lt;p&gt;Once the algorithm reaches a point where a reward (or penalty) is gained, then a slowly decaying amount of that reward is given to the Q-value of whichever action was derived from a particular state for each time step back through the game (or however far back you want to apply that reward).&#xA;You will have stored the state and Q-values for each time step during the game, allowing you to update the appropriate Q-values with the new reward.&#xA;These state / updated Q-values (from many runs) become your training data for the next iteration of your training algorithm. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-02-01T19:11:22.233" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10357" PostTypeId="2" ParentId="10339" CreationDate="2019-02-01T22:30:58.790" Score="1" Body="&lt;p&gt;Quantizing of weights is used very often in implementing neural networks with the tensorflow library on mobile devices. The reason is, the such microcontrollers doesn't have a floating point unit and the developer is trying to increase the overall performance. Quantizing means, to map float values to integer values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Integer Float&#xA;0      -5.0&#xA;128    0.0&#xA;256    5.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem isn't trivial, because the float point numbers can have a certain minimum, maximum and precision. Maintaining the original distribution is also an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid such kind of problems, the best idea is to implement a floating point library from scratch. In the stackbased Forth language this can be realized in under 400 bytes. It was described in the paper “Willi Stricker: Forth Floating Point Word-Set without Floating Point Stack, 2012”&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2019-02-01T22:30:58.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10358" PostTypeId="5" CreationDate="2019-02-01T22:37:32.490" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-01T22:37:32.490" LastActivityDate="2019-02-01T22:37:32.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10359" PostTypeId="4" CreationDate="2019-02-01T22:37:32.490" Score="0" Body="Use for questions about autonomous weapons systems specifically.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-01T22:37:32.490" LastActivityDate="2019-02-01T22:37:32.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10360" PostTypeId="1" CreationDate="2019-02-02T08:06:41.317" Score="2" ViewCount="612" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://core.ac.uk/download/pdf/143454908.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BayesChess: A computer chess program based on Bayesian networks&lt;/a&gt; (Fernandez, Salmeron; 2008)&lt;/p&gt;&#xA;&lt;p&gt;It is a chess-playing engine using Bayesian networks. The following is mentioned about the heuristic function in section 3.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Here the heuristic is defined in terms of 838 parameters.&lt;/p&gt;&#xA;&lt;p&gt;There are 5 parameters indicating the value of each piece (pawn,&#xA;queen, rook, knight, and bishop -the king is not evaluated, as it must&#xA;always be on the board), 1 parameter for controlling whether the king&#xA;is under check, 64 parameters for evaluating the location of each&#xA;piece on each square on the board (i.e., a total of 786 parameters,&#xA;corresponding to 64 squares × 6 pieces each colour × 2 colours) and&#xA;finally 64 more parameters that are used to evaluate the position of&#xA;the king on the board during the endgame.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The above sentence contains the parameters used by the heuristic function. But I didn't find the actual definition. What is the actual definition of the heuristic function?&lt;/p&gt;&#xA;" OwnerUserId="21964" LastEditorUserId="2444" LastEditDate="2020-09-29T22:40:17.307" LastActivityDate="2020-09-29T22:40:17.307" Title="What is the definition of a heuristic function in the BayesChess paper?" Tags="&lt;definitions&gt;&lt;papers&gt;&lt;heuristics&gt;&lt;chess&gt;&lt;bayesian-network&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10361" PostTypeId="2" ParentId="10360" CreationDate="2019-02-02T08:42:41.943" Score="0" Body="&lt;p&gt;Heuristic just means that it is hand constructed by a human. Suppose you have the value of each piece given an initial value. That is a heuristic because it was just defined by a human. But if the bayesian process is going to modify that initial value given by the human that would be a non-heuristic thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there is no function definition to search for except to find what the initial parameters given by the human are.&lt;/p&gt;&#xA;" OwnerUserId="21724" LastActivityDate="2019-02-02T08:42:41.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10362" PostTypeId="2" ParentId="10352" CreationDate="2019-02-02T09:11:20.390" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Because the Q value is different, I don't see how the reward signal at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is of any relevance for &lt;span class=&quot;math-container&quot;&gt;$Q_{t+x}(s_t,a_t)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$t+x$&lt;/span&gt;, the time of learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; value for any single step is not dependent on &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; or the current policy. It is purely dependent on &lt;span class=&quot;math-container&quot;&gt;$(s_t,a_t)$&lt;/span&gt;. That means you can use the Q update equation to use it to calculate &lt;em&gt;new&lt;/em&gt; TD targets, by combining your knowledge of the old reward signal and the value functions of the latest policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value &lt;span class=&quot;math-container&quot;&gt;$r_t + \gamma \max_{a_{t+1}} Q(s_{t+1}, a_{t+1}; \theta_i^-)$&lt;/span&gt; is the TD target in the loss function you show. There are many possible variations to calculate the TD target value in RL, with different properties. The one you are using is &lt;em&gt;biased&lt;/em&gt; towards initial random values unrelated to the problem, and also deliberately biased towards older Q values to prevent runaway feedback. It is also low variance compared to other possibilities, and simpler to calculate.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, it is likely that following a policy which is derived from &lt;span class=&quot;math-container&quot;&gt;$Q_t$&lt;/span&gt; would lead to &lt;span class=&quot;math-container&quot;&gt;$a_{t}$&lt;/span&gt;, whereas following a policy which is derived from &lt;span class=&quot;math-container&quot;&gt;$Q_{t+x}$&lt;/span&gt; would not. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes that is correct. However, you don't care about that for a single step, you just care about calculating a better estimate for &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; regardless of whether you would choose &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with the current policy. That is what an action value measures - the utility of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;em&gt;thereafter&lt;/em&gt; following a given policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a strength of experience replay, that you are constantly refining your estimates of off-policy action values to determine which is the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does become more of an issue when you want to use longer trajectories in the update step (which you may want to do to reduce bias of your TD target estimate). A series of steps in history &lt;span class=&quot;math-container&quot;&gt;$s_t,a_t,r_t ... s_{t+1}, a_{t+1},r_{t+1}...s_{t+n}$&lt;/span&gt; may not have the same chance of occurring under the latest policy as it did when it was stored in memory. For the first step &lt;span class=&quot;math-container&quot;&gt;$s_t,a_t$&lt;/span&gt; again you don't care if it is one you would currently take because the point is to refine your estimate of that action value. However, if you want to use &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}, r_{t+2}$&lt;/span&gt; etc plus &lt;span class=&quot;math-container&quot;&gt;$s_{t+n}$&lt;/span&gt; to create a TD target, then you have to care whether your current policy and the one used to populate the history table would be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a problem if you want to use more sophisticated estimates of TD target that use multiple sample steps along with experience replay. There are some approaches you can take to allow for this, such as &lt;em&gt;importance sampling&lt;/em&gt;. For a single step update mechanism you don't need to worry about it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I don't see in the experience replay algorithm that the Q value &lt;span class=&quot;math-container&quot;&gt;$Q_t(s_t, a_t)$&lt;/span&gt; is saved, so I must assume that is is not.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is correct. You must &lt;em&gt;re-calculate&lt;/em&gt; the TD target from a more up-to-date policy to get better estimates of the action value. With experience replay, you are not interested in collecting a history of values of Q. Instead you are interested in the history of state transitions and rewards.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why does calculating the Q value again at a later time make sense FOR THE SAME SAVED REWARD AND ACTION?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because it &lt;em&gt;will&lt;/em&gt; change, due to the learning process summarising effects of state transitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a toy example, consider a maze solver with traps (negative rewards) and treasures (positive rewards). At one point in history, the agent finds itself in a location and its policy told it to move into a trap on the next step. The agent would initially score location and steps leading up to it with negative Q values. Later it discovers through exploration that there is also some treasure if it takes a different turning towards the end of the the same series of steps. With experience replay, and re-calculating Q values each time, it can figure out which section of that path should be converted to high scores because they lead to treasure as well as the trap and now the agent has a better policy for the end of the path it has better estimates of value there.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-02T09:45:29.497" LastActivityDate="2019-02-02T09:45:29.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10363" PostTypeId="2" ParentId="3494" CreationDate="2019-02-02T11:27:54.953" Score="3" Body="&lt;p&gt;Python has rich library, it is also object oriented, easy to program. It can be also used as frontend language. That's why it is used in artificial intelligence. Rather than AI it is also used in machine learning, soft computing, NLP programming and also used as web scripting or in Ethical hacking. &lt;/p&gt;&#xA;" OwnerUserId="21965" LastEditorUserId="21965" LastEditDate="2019-02-28T23:18:36.397" LastActivityDate="2019-02-28T23:18:36.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10364" PostTypeId="1" CreationDate="2019-02-02T14:14:36.803" Score="2" ViewCount="285" Body="&lt;p&gt;Up to now, I have been using (my version of) &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/actor_critic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;open AI's code&lt;/a&gt;, with the suggested CartPole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been using Monte Carlo methods, which, for cartpole, seemed to work fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to move to temporal difference, Cartpole seems to fail to learn (with simple TD method) (or I stopped it too soon, but still unacceptable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that is the case because in Cartpole, for every timestamp, we get a reward of 1, which has very little immediate information about weather or not the action was good or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Which gym environment is the simplest that would probably work with TD learning?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by simplest I mean that there is no need for a large NN to solve it. No conv nets, no RNNS. just a few small layers of a fully connected NN, just like in cartpole. something I can train on my home cpu, just to see it starting to converge.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2019-02-02T17:12:40.057" Title="Which Openai Gym environment should I use to test a Temporal Difference RL Agent?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10365" PostTypeId="2" ParentId="3749" CreationDate="2019-02-02T15:15:18.943" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;h1&gt;&lt;a href=&quot;https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lex Fridman interviews&lt;/a&gt;&lt;/h1&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These are some of the best interviews on the field that I have found on the internet.&lt;/p&gt;&#xA;&lt;p&gt;They are part of the MIT course 6.S099: Artificial General Intelligence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2h8Ze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2h8Ze.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h1&gt;Andrew Ng's &lt;a href=&quot;https://www.youtube.com/watch?v=-eyhCTvrEtE&amp;amp;list=PLfsVAYSMwsksjfpy8P2t_I52mugGeA5gR&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heroes of deep learning&lt;/a&gt; series&lt;/h1&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The focus is on deep learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z4eXp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z4eXp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Yannic Kilcher has some fascinating youtube videos:&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/channel/UCZHmQk67mSJgfCCTn7xBfew&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yannic's channel&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/utuz7wBGjKM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Check out this one!&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-16T09:41:33.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10367" PostTypeId="2" ParentId="10364" CreationDate="2019-02-02T17:12:40.057" Score="1" Body="&lt;p&gt;Cartpole will work just fine with a single-step TD method - e.g. Q learning - and a simple neural network. &lt;em&gt;Configured well&lt;/em&gt;, a simple DQN model should solve it in a few minutes. You can also forgo the neural network and use discretised states, or tile coding etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you cannot do is plug a neural network estimator into basic Q learning and make no other adjustments. That is because the bootstrap process in TD learning will create a runaway feedback driven from the random and incorrect return estimates in the initialised NN. You have to use something like DQN to get it to work at all. This is different to Monte Carlo approach, where you can generally just plug in a NN in place of a Q table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basics of DQN are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use experience replay. Store &lt;span class=&quot;math-container&quot;&gt;$(S, A, R, S')$&lt;/span&gt; values from each step, and when training the NN, draw a small random batch (e.g. maybe 16 samples) from this table, re-estimate their Q values to calculate TD target e.g.  &lt;span class=&quot;math-container&quot;&gt;$R + \gamma\text{max}_{a'} Q(S',a')$&lt;/span&gt; and use that to create a minibatch to train the NN &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a &quot;delayed&quot; target network for estimating TD target. This can just be a snapshot of your learning network, taken once every N steps, where N is typically anything from 100 to 100000.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Further details of what &lt;em&gt;configured well&lt;/em&gt; means varies from problem to problem. With DQN you will want to play with hyper parameters of experience history size, time to keep frozen copy of network (for bootstrap estimates), batch size of experience table samples, amount of exploration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In CartPole though, there will be a wide range of acceptable hyperparameter values. So I suspect you have just plugged in a NN in place of the Q table and wonder why it doesn't work.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I assume that is the case because in Cartpole, for every timestamp, we get a reward of 1, which has very little immediate information about weather or not the action was good or not.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes this is part of what makes some learning goals become harder challenges in sequential-decision control systems. However, this was also the case for your Monte Carlo method, and in general this relates to the &lt;a href=&quot;http://www.bcp.psych.ualberta.ca/~mike/Pearl_Street/Dictionary/contents/C/creditassign.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;credit assignment problem&lt;/a&gt; in RL.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-02-02T17:12:40.057" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10368" PostTypeId="1" AcceptedAnswerId="10379" CreationDate="2019-02-02T17:30:53.470" Score="6" ViewCount="1554" Body="&lt;p&gt;&lt;a href=&quot;https://amreis.github.io/ml/reinf-learn/2017/11/02/reinforcement-learning-eligibility-traces.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD lambda&lt;/a&gt; is a way to interpolate between TD(0) - bootstrapping over a single step, and, TD(max), bootstrapping over the entire episode length, or, Monte Carlo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading the link above, I see that an eligibility trace is kept for each state in order to calculate its &quot;contribution to the future&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if we use an approximator, and not a table for state-values, then can we still use eligibility traces? If so, how would the loss (and thus the gradients) be calculated? Specifically, I would like to use actor-critic (or advantage actor-critic).&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="2444" LastEditDate="2020-06-04T16:59:48.010" LastActivityDate="2020-08-13T14:19:14.743" Title="Can TD($\lambda$) be used with deep reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;temporal-difference-methods&gt;&lt;eligibility-traces&gt;&lt;td-lambda&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10369" PostTypeId="2" ParentId="9614" CreationDate="2019-02-02T19:16:44.553" Score="9" Body="&lt;p&gt;An important thing we're going to need is what is called the &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html#expected-grad-log-prob-lemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Expected Grad-Log-Prob Lemma here&amp;quot;&lt;/a&gt; (proof included on that page), which says that (for any &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta}(a_t \mid s_t) \right] = 0.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Taking the analytical expression of the gradient (from, for example, slide 9) as a starting point:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{aligned}&#xA;\nabla_{\theta} J(\theta) &amp;amp;= \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \left( \sum_{t=1}^T \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \right) \left( \sum_{t=1}^T r(s_t, a_t) \right) \right] \\&#xA;%&#xA;&amp;amp;= \sum_{t=1}^T \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=1}^T r(s_{t'}, a_{t'}) \right] \\&#xA;%&#xA;&amp;amp;= \sum_{t=1}^T \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=1}^{t-1} r(s_{t'}, a_{t'}) + \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=t}^T r(s_{t'}, a_{t'}) \right] \\&#xA;%&#xA;&amp;amp;= \sum_{t=1}^T \left( \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=1}^{t-1} r(s_{t'}, a_{t'}) \right] \\&#xA;+ \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=t}^T r(s_{t'}, a_{t'}) \right] \right) \\&#xA;\end{aligned}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;At the &lt;span class=&quot;math-container&quot;&gt;$t^{th}$&lt;/span&gt; &amp;quot;iteration&amp;quot; of the outer sum, the random variables&#xA;&lt;span class=&quot;math-container&quot;&gt;$ \sum_{t'=1}^{t-1} r(s_{t'}, a_{t'}) $&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) $&lt;/span&gt;&#xA;are independent (we assume, by definition, the action only depends on the most recent state), which means we are allowed to split the expectation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta} J(\theta) = \sum_{t=1}^T \left( \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \sum_{t'=1}^{t-1} r(s_{t'}, a_{t'}) \right] \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \right] \\&#xA;+ \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=t}^T r(s_{t'}, a_{t'}) \right] \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first expectation can now be replaced by &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; due to the lemma mentioned at the top of the post:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\nabla_{\theta} J(\theta)&#xA;%&#xA;&amp;amp;= \sum_{t=1}^T \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \sum_{t'=t}^T r(s_{t'}, a_{t'}) \right]  \\&#xA;%&#xA;&amp;amp;= \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \sum_{t=1}^T \nabla_{\theta} \log \pi_{\theta} (a_t \mid s_t) \left( \sum_{t'=t}^T r(s_{t'}, a_{t'}) \right). \\&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The expression on slide 18 of the linked slides is an unbiased, sample-based estimator of this gradient:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta} J(\theta) \approx \frac{1}{N} \sum_{i=1}^N \sum_{t=1}^T \nabla_{\theta} \log \pi_{\theta} (a_{i, t} \mid s_{i, t}) \left( \sum_{t'=t}^T r(s_{i, t'}, a_{i, t'}) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;For a more formal treatment of the claim that we can pull &lt;span class=&quot;math-container&quot;&gt;$\sum_{t'=1}^{t-1} r(s_{t'}, a_{t'})$&lt;/span&gt; out of an expectation due to the Markov property, see this page: &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/extra_pg_proof1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spinningup.openai.com/en/latest/spinningup/extra_pg_proof1.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="42533" LastEditDate="2020-11-25T20:14:21.823" LastActivityDate="2020-11-25T20:14:21.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10370" PostTypeId="2" ParentId="10327" CreationDate="2019-02-02T23:08:19.650" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Could that data set be used for this deep ranking algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes you can! I think there are at least two approaches for this task:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;First, solve using Image Classification. If you want to use Deep Learning, you can use Deep Convolutional Neural Network to create classifier that decide the image is HP book or not. You can read papers mentioned on Mahmoud's answer. But the problem is you need a very very large dataset, to make a good classifier you can't just provide one image for one book so if you have thousand title of books (or more) you need train your model with very hugh dataset. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using image similarity or content based image retrieval (CBIR), there is a &lt;a href=&quot;https://stackoverflow.com/questions/843972/image-comparison-fast-algorithm&quot;&gt;good discussion in Stackoverflow&lt;/a&gt; about this topic, there are a lot of techniques include Deep Ranking, perceptual hash, and others. One of their difference is Deep Ranking use fewer feature engineering technique than others. In my opinion using image similarity technique is better approach than using image classification (it's also compared in Deep Ranking paper) because some methods will be faster and didn't required a lot of dataset.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You also can read &lt;a href=&quot;https://towardsdatascience.com/building-a-similar-images-finder-without-any-training-f69c0db900b5&quot; rel=&quot;nofollow noreferrer&quot;&gt;another simple reference of image similarity&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-02-02T23:08:19.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10371" PostTypeId="1" AcceptedAnswerId="10388" CreationDate="2019-02-02T23:24:15.137" Score="5" ViewCount="1562" Body="&lt;p&gt;In my country, the Expert System class is mandatory, if you want to take the AI specialization in most universities. In class, I learned how to make a rule-based system, &lt;a href=&quot;https://en.wikipedia.org/wiki/Forward_chaining&quot; rel=&quot;nofollow noreferrer&quot;&gt;forward chaining&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Backward_chaining&quot; rel=&quot;nofollow noreferrer&quot;&gt;backward chaining&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;However, I have read somewhere on the web that &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;expert systems&lt;/a&gt; are no longer used.&lt;/p&gt;&#xA;&lt;p&gt;Is that true? If yes, why? If not, where are they being used? With the rise of machine learning, they may not be as used as before, but is there any industry or company that still uses them today?&lt;/p&gt;&#xA;&lt;p&gt;Please, provide some references to support your claims.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2021-01-22T03:40:37.533" LastActivityDate="2021-01-22T03:40:37.533" Title="Is the expert system still in use today?" Tags="&lt;reference-request&gt;&lt;applications&gt;&lt;symbolic-ai&gt;&lt;expert-system&gt;&lt;rule-based-systems&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10372" PostTypeId="1" AcceptedAnswerId="10373" CreationDate="2019-01-31T23:35:08.680" Score="2" ViewCount="383" Body="&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/20535/what-is-experience-replay-and-what-are-its-benefits&quot;&gt;Experience replay&lt;/a&gt; is buffer (or a &quot;memory&quot;) of transactions &lt;span class=&quot;math-container&quot;&gt;$e_t = (s_t, a_t, r_t, s_{t+1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.freecodecamp.org/an-intro-to-advantage-actor-critic-methods-lets-play-sonic-the-hedgehog-86d6240171d&quot; rel=&quot;nofollow noreferrer&quot;&gt;The equations for calculating the loss in actor critic&lt;/a&gt; are an &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;actor&lt;/em&gt; loss (parameterized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) &lt;span class=&quot;math-container&quot;&gt;$$log[\pi_\theta(s_t,a_t)]Q_w(s_t,a_t)$$&lt;/span&gt; and a &lt;em&gt;critic&lt;/em&gt; loss (parameterized by &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;) &lt;span class=&quot;math-container&quot;&gt;$$r(s_t,a_t) + \gamma Q_w(s_{t+1},a_{t+1}) - Q_w(s_{t},a_{t}).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it, there are two more elements that need to be saved for later use:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The expected Q value at the time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$Q_w(s_{t},a_{t})$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The log probability for action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$log[\pi_\theta(s_t,a_t)]$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If they are not saved, how will we be able to later calculate the loss for learning? I didn't see anywhere stating to save those, and I must be missing something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do these elements need to be saved or not?&lt;/p&gt;&#xA;" OwnerUserId="21645" OwnerDisplayName="Gulzar" LastEditorUserId="21645" LastEditDate="2019-02-06T14:41:45.123" LastActivityDate="2019-02-06T14:41:45.123" Title="What information should be cached in experience replay for actor-critic?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10373" PostTypeId="2" ParentId="10372" CreationDate="2019-02-01T19:13:11.330" Score="4" Body="&lt;p&gt;The loss function is estimated in every batch training cycle. Gradients of the loss are computed and propagation back to the network happens in every cycle. This means that you use a small batch (e.g. 100 instances) from the replay memory, and by having the states you can input them to the respective network and have the &lt;span class=&quot;math-container&quot;&gt;$Q(s)$&lt;/span&gt; for every state in your batch. Then you estimate the loss and run backpropagation and the networks' weights are getting updated. You continue gathering experience by interacting with the environment and after a threshold that you specify you repeat the cycle by re-sampling a new batch from the memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a suggestion: Start moving towards Asynchronous/synchronous methods for RL and use one network with different &quot;heads&quot; for Actor and Critic. Then you use one loss function plus the experience that now is collected from multiple instances of your agent-environment interaction (in contrast to your description in which one agent collects and stores experience from one instance).&lt;/p&gt;&#xA;" OwnerUserId="36093" OwnerDisplayName="Constantinos" LastActivityDate="2019-02-01T19:13:11.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10374" PostTypeId="1" CreationDate="2019-02-03T05:37:55.170" Score="2" ViewCount="72" Body="&lt;p&gt;I've come up with an idea on how we could use a combination of Deep Learning and body sensors to create a walking talking living humanoid. Here goes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we will recruit 1 billion people and have them wear a special full face mask and suit. This suit will contain touch sensors along the skin, cameras, smell sensors, taste sensors on the mask, basically every data and information that a human receives will be collected electronically, whether it is what they see, what they smell what they feel and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These suits will also have potentiometers and other sensors to measure the movement made by the person. Every hand movement, leg movement, muscle movement will be recorded and saved in a database as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 50 years or so, all collected input and output data from every single person who participated in this experiment will be saved in a computer. We then create a neural network and then train it on the input and output data from the database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, we create a robot that has motorized muscles and hand/leg joints that are the same specs as to our previous suits and also the touch, smell, sight and other sensors integrated inside of this robot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once everything is ready, we will load the trained neural network onto the robot and switch it on. During inference, the neural network will take data from sensors all over the robot's body and translate it into movement in legs, hands, and muscle.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Could these techniques in conjunction with the data collection I describe, produce and AGI android?  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Essentially, how feasible is current technology to produce a robot that will behave, speak, live, move like a normal human being?&lt;/p&gt;&#xA;" OwnerUserId="21980" LastEditorUserId="1671" LastEditDate="2019-02-05T22:21:01.580" LastActivityDate="2019-02-05T22:21:01.580" Title="An idea I had to create the first Humanoid using Deep Learning" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;theory&gt;&lt;data-science&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10377" PostTypeId="2" ParentId="145" CreationDate="2019-02-03T19:58:21.840" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is AIXI really a big deal in artificial general intelligence research?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, it is a great &lt;em&gt;theoretical&lt;/em&gt; contribution to AGI. AFAIK, it is the most serious attempt to build a theoretical framework or foundation for AGI. Similar works are Schmidhuber's &lt;a href=&quot;ftp://ftp.idsia.ch/pub/juergen/gmAGI.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gödel Machines&lt;/a&gt; and &lt;a href=&quot;https://soar.eecs.umich.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOAR architecture&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AIXI is an abstract and non-&lt;a href=&quot;https://en.wikipedia.org/wiki/Anthropomorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;anthropomorphic&lt;/a&gt; framework for AGI which builds on top of the reinforcement learning field, without a few usual assumptions (e.g., without the Markov and &lt;a href=&quot;https://arxiv.org/abs/1705.10557&quot; rel=&quot;nofollow noreferrer&quot;&gt;ergodicity&lt;/a&gt; assumptions, which guarantees that the agent can easily recover from any mistakes it made in the past). Even though some optimality properties of AIXI have been proved, it is (Turing) &lt;a href=&quot;https://en.wikipedia.org/wiki/Computable_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;uncomputable&lt;/a&gt; (it cannot be run on a computer), and so it is of very limited practical usefulness. Nonetheless, in the Hutter's book &lt;a href=&quot;http://hutter1.net/ai/uaibook.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability&lt;/a&gt; (2005), where several properties of AIXI are rigorously proved, a computable but intractable version of AIXI, AIXItl, is also described. Furthermore, in the paper &lt;a href=&quot;https://arxiv.org/abs/0909.0801&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Monte Carlo AIXI Approximation&lt;/a&gt; (2009), by Joel Veness et al., a computable and &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#tractable_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;tractable&lt;/a&gt; approximation of AIXI is introduced. So, there have been some attempts to make AIXI practically useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article &lt;a href=&quot;https://jan.leike.name/AIXI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is AIXI? — An Introduction to General Reinforcement Learning&lt;/a&gt; (2015), by Jan Leike, which is one of the contributors to the development and evolution of the AIXI framework, gives a gentle introduction to the AIXI agent. See also &lt;a href=&quot;https://plato.stanford.edu/entries/artificial-intelligence/aixi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The AIXI Architecture&lt;/a&gt; at the Stanford Encyclopedia of Philosophy for a possibly gentler introduction to AIXI. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can it be thought of as a central concept for the field?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, the introduction of AIXI and related research has contributed to the evolution of the AGI field. There have been several discussions and published papers, after its introduction in 2000 by Hutter in the paper &lt;a href=&quot;https://arxiv.org/abs/cs/0004001&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Theory of Universal Artificial Intelligence based on Algorithmic Complexity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. section 7, &quot;Examples of Superintelligences&quot;, of the paper &lt;a href=&quot;https://intelligence.org/files/AGI-HMM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence and the Human Mental Model&lt;/a&gt; (2012), by Roman V. Yampolskiy and Joshua Fox. See also &lt;a href=&quot;https://wiki.lesswrong.com/wiki/AIXI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wiki.lesswrong.com/wiki/AIXI&lt;/a&gt; which contains a discussion regarding a few problems related to AIXI, which need to be solved or possibly avoided in future AGI frameworks. Furthermore, see also &lt;a href=&quot;https://www.lesswrong.com/posts/TtYuY2QBug3dn2wuo/the-problem-with-aixi&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://alife.co.uk/essays/on_aixi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; articles.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, why don't we have more publications on this subject (or maybe we have and I'm not aware of them)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There have been several publications, mainly by Marcus Hutter and associated researchers. You can see Marcus Hutter's publications on the following web page: &lt;a href=&quot;http://www.hutter1.net/official/publ.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.hutter1.net/official/publ.htm&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in contributing to this theory, there are several ways. If you are mathematically well educated, you can attempt to solve some of the problems described &lt;a href=&quot;http://www.hutter1.net/ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (which are also mentioned in the Hutter's 2005 book mentioned above). Furthermore, you can also contribute to new approximations or improvements of existing approximations of the AIXI agent. Finally, you can build your new AGI framework by avoiding the problems associated with the AIXI framework. See also &lt;a href=&quot;https://cecs.anu.edu.au/people/marcus-hutter&quot; rel=&quot;nofollow noreferrer&quot;&gt;projects promoted by Hutter&lt;/a&gt;. It may be a good idea to also take into account e.g. Gödel Machines and related work, before attempting to introduce a new framework (provided you are capable of it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that this theory has not attracted more people probably because it is highly technical and mathematical (so it is not very easy to understand unless you have a very solid background in reinforcement learning, probability theory, etc.). I also think that most people (in the AI community) are not interested in theories, but they are mainly guided by practical and useful results.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="20044" LastEditDate="2019-10-21T14:34:24.360" LastActivityDate="2019-10-21T14:34:24.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10378" PostTypeId="1" CreationDate="2019-02-03T20:05:06.860" Score="3" ViewCount="117" Body="&lt;p&gt;As in sigmoid function when input x is very large or very small the curve is flat that means low gradient descent but when it is in between the slope is more so,&#xA;My question is how this thing helps us in neural network.&lt;/p&gt;&#xA;" OwnerUserId="21642" LastActivityDate="2019-02-04T01:05:55.140" Title="How sigmoid funtion helps us in reducing error in neural networks?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10379" PostTypeId="2" ParentId="10368" CreationDate="2019-02-03T23:09:44.587" Score="7" Body="&lt;p&gt;Eligibility traces is a method of weighting between temporal-difference &amp;quot;targets&amp;quot; and Monte-Carlo &amp;quot;returns&amp;quot;. In practice, for example, instead of using the one-step TD target, &lt;span class=&quot;math-container&quot;&gt;$r_t + \gamma V (s_{t+1})$&lt;/span&gt;, as in the temporal difference update &lt;span class=&quot;math-container&quot;&gt;$V (s_t) \leftarrow V (s_t) + \alpha (r_t + \gamma V (s_{t+1}) − V (s_t))$&lt;/span&gt;, you use the so-called &amp;quot;lambda&amp;quot; (&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) target, which is a target that balances between the TD target and the Monte Carlo return. So, in practice and intuitively, eligibility traces is just a way of using a more &amp;quot;appropriate&amp;quot; target while learning. In general, you need to perform these updates (e.g., the TD update above) &amp;quot;online&amp;quot;, i.e. while you explore or exploit the environment.&lt;/p&gt;&#xA;&lt;p&gt;In theory, you could use a deep neural network to represent your value function (or your policy), while using eligibility traces. It would be similar to not using them: you would just use a different target.&lt;/p&gt;&#xA;&lt;p&gt;However, deep RL (that is, RL which uses deep neural networks to represent e.g. value functions) training needs to be performed using i.i.d. data, in order to prevent overfitting, which often means that they can't be trained online or need to use &amp;quot;tricks&amp;quot; like the &amp;quot;experience replay&amp;quot; (used in the paper &lt;a href=&quot;https://www.nature.com/articles/nature14236.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human-level control through deep reinforcement learning&lt;/a&gt;). Note that, in RL, successive states are often very correlated (e.g. two successive frames of a video would be very correlated).&lt;/p&gt;&#xA;&lt;p&gt;In theory and similarly, you would still be able to use eligibility traces with the actor-critic method, but not with the asynchronous advantage actor-critic method. See the section 2.3 of the paper &amp;quot;&lt;a href=&quot;https://arxiv.org/pdf/1810.09967.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Eligibility Traces for Deep Reinforcement Learning&lt;/a&gt;&amp;quot; (2018) by Brett Daley and Christopher Amato, for more info.&lt;/p&gt;&#xA;&lt;p&gt;In this same paper, an approach is introduced to efficiently combine eligibility traces with deep neural networks. The authors propose DQN(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;), which is the DQN architecture combined with eligibility traces, where the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; return is computed in an &amp;quot;efficient&amp;quot; (and recursive) way, instead of the &amp;quot;usual&amp;quot; way. Since they use a DQN, they also use an &amp;quot;experience replay&amp;quot; buffer (or memory), where they also store the efficiently computed &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; target (in addition to the usual rewards). Furthermore, they also eliminate the need for the &amp;quot;target&amp;quot; network used in the standard DQN. You can have a look at algorithm 1 of the same paper to see how they improve the parameters of the network, which represents the Q function, in the case of the DQN(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) model. See the section 3.1 of the same paper for more details regarding this model.&lt;/p&gt;&#xA;&lt;p&gt;They also introduce A3C(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;), which combines asynchronous advantage actor-critic (A3C) with eligibility traces. See the section 3.2 for more details.&lt;/p&gt;&#xA;&lt;p&gt;Note that there have been other proposals for combining eligibility traces with deep learning. You can have a look at the literature.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-13T14:19:14.743" LastActivityDate="2020-08-13T14:19:14.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10380" PostTypeId="1" AcceptedAnswerId="10417" CreationDate="2019-02-04T00:33:49.347" Score="1" ViewCount="106" Body="&lt;p&gt;I wasn't sure how to title this question so pardon me please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may have seen at least one video of those &quot;INSANE A.I created simulation of {X} doing {Y &amp;amp; Z} like the following ones:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A.I learns how to play Mario&#xA;A.I swaps faces of {insert celebrity} in this video after 16hrs.&#xA;etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know what I have to learn to be able to create for example a program that takes xyz-K images of a person as training data and changes it with another person's face in a video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or create a program that on a basic level creates a simulation of 2 objects orbiting /attracting each other /colliding like this:&#xA;&lt;img src=&quot;https://i.imgur.com/X6TXNUa.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What field/topic is that?&#xA; I suspect deep learning but I'm not sure. I'm currently learning machine learning with Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm struggling because linear regression &amp;amp; finances /stock value prediction is really not interesting compared to teaching objects in games to do archive something or create a program that tries to read characters from images.&lt;/p&gt;&#xA;" OwnerUserId="21988" LastEditorUserId="1671" LastEditDate="2019-02-06T02:48:09.620" LastActivityDate="2019-02-06T05:59:17.517" Title="Which field to study to learn &amp; create a.i generated simulations?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;data-science&gt;&lt;ai-field&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10381" PostTypeId="2" ParentId="10378" CreationDate="2019-02-04T01:05:55.140" Score="2" Body="&lt;p&gt;There are several nice things about using the logistic function and a few drawbacks as well.  I'm not going to discuss all of them, but here is a small synopsis.&lt;/p&gt;&#xA;&lt;h1&gt;Nice Properties&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/czEqL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/czEqL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;For networks which calculate probabilities, it brings the input features into a nice 0 to 1 range.&lt;/h3&gt;&#xA;&lt;p&gt;Since the logistic function maps inputs to the range (0,1), it is commonly used as an activation function in neural networks which require some sort of classification.  For example, if we had a convolution neural network (CNN) that classified if an image given to it was a dog or not, it could have an output neuron that returned 0 if the image was not a dog or 1 if it was.  When the CNN is given an image with inputs of varying dimensions and ranges, it's more intuitive to deal with the input features in the realm of 0 to 1 instead of 0 to 255 or whatever input format you're dealing with.&lt;/p&gt;&#xA;&lt;h3&gt;It is smooth and differentiable.&lt;/h3&gt;&#xA;&lt;p&gt;When performing back propagation, it's much easier to deal with differential functions.  Otherwise you'd have to approximate the gradient which may lead to unfavorable results.&lt;/p&gt;&#xA;&lt;h1&gt;A Vanishing Issue&lt;/h1&gt;&#xA;&lt;h3&gt;At extreme values, the gradient vanishes.&lt;/h3&gt;&#xA;&lt;p&gt;The OP was also hinting that at very positive and very negative values of X, derivative of the logistic function could become close to 0.  For large input values this may hinder learning for extreme values.  If you think of gradient decent, the jump that gradient descent makes is only as strong as its magnitude of the gradient that it requires.  Analytically speaking, this also hurts back propagation because learning is calculated using the chain rule, which ends up multiplying several gradients together.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-02-04T01:05:55.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10384" PostTypeId="2" ParentId="10043" CreationDate="2019-02-04T08:14:58.783" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is doing a M.S. in the combination of Natural Language Processing and Machine Learning worth it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think it's not about on what topic you'll take, but it depends on how you will do it, where you will study, and other factors. I'll try to give you both perspective from industrial and academical:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Pros (take MS in NLP):&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you feel didn't get the basic in your undergrad degree, MS degree will refresh your basic knowledge of AI or NLP.&lt;/li&gt;&#xA;&lt;li&gt;If you do research &lt;strong&gt;seriously&lt;/strong&gt;, it will give you a nice portfolio that you have done an advance research on NLP. But if you just do research only to get your degree with minimum quality, I don't think it can give you any additional value.&lt;/li&gt;&#xA;&lt;li&gt;You'll have a chance to know more researchers, lecturers, that may will give you a chance to get another job / research project in the future (I got job once from my ex-lecturers)&lt;/li&gt;&#xA;&lt;li&gt;Many companies give you a chance to do Master while working part-time / full-time as long you can manage your time.&lt;/li&gt;&#xA;&lt;li&gt;If you want to continue become a researcher, then doing MS is required so you can continue your study to PhD. Then with PhD degree you should get a deeper knowledge of NLP (as you've done many research on NLP).&lt;/li&gt;&#xA;&lt;li&gt;I saw many companies that focus on AI/NLP &lt;strong&gt;only recruit&lt;/strong&gt; people with PhD degree for data scientist/NLP researcher position.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Cons (don't take MS in NLP):&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are many other companies that &lt;strong&gt;don't see your academic degree&lt;/strong&gt;. I know some friends that become senior programmer without any university degree. As long you can pass their test as NLP engineer, then you will get the job.&lt;/li&gt;&#xA;&lt;li&gt;In this modern era, You can learn a lot via online course, read published paper, etc., &lt;strong&gt;but&lt;/strong&gt; you'll need a lot of motivation for this.&lt;/li&gt;&#xA;&lt;li&gt;In my country (Indonesia), sometimes some NLP projects are simple that you don't need advance methods (LSTM, RNN, Seq2seq, etc.) So you can learn by yourself from the internet.&lt;/li&gt;&#xA;&lt;li&gt;You can invest your time for studying advance topic NLP rather than re-learn your basic knowledge on your semesters in Master's program&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the opportunities that can be opened by having such a Master's?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my country, I think having Master's or not is not really different except you might have deeper skills that you can show off to the interviewer. But if you got PhD you'll have more opportunities to do research on NLP as researcher either in industry or university.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conclusions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It depends on how you will do it: if you commit to take MS degree seriously, learn a lot in the program, and do advance research on NLP, then I'll recommend you to take MS&lt;/li&gt;&#xA;&lt;li&gt;It depends on where you will study: if the master's program not really focus teaching NLP, you may waste some of your times to learn something that doesn't intersect at all with NLP area.&lt;/li&gt;&#xA;&lt;li&gt;It depends on what you want to be: if you want to become NLP researcher (both in industry or university) I'll recommend you to take Master, then try to get your PhD so yo can get deeper knowldege on NLP.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-02-04T08:14:58.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10387" PostTypeId="1" CreationDate="2019-02-04T13:59:19.120" Score="1" ViewCount="272" Body="&lt;p&gt;I have a convolutional encoder (a CNN) consisting of DenseBlocks and a total of 50 layers (cf. FC-DenseNet103). The receptive field of the encoder (after last layer) is 660 according to Tensorflow function compute_receptive_field_from_graph_def(..)) whereas the input image is 64x64 pixels. Obviously the receptive field is way too big. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can the receptive field be reduced to say 46 but the capacity of the encoder be more or less kept at the same level? By capacity I simply mean the number of parameters of the model. The capacity requirement is justified due to the complex dataset to be processed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using less layers or smaller kernels reduces the receptive field size but also the capacity. Should I then just increase the number of filters in the remaining layers in order to keep the capacity?&lt;/p&gt;&#xA;" OwnerUserId="20191" LastEditorUserId="20191" LastEditDate="2019-02-04T20:03:27.053" LastActivityDate="2020-10-27T16:00:30.060" Title="Reduce receptive field size of CNN while keeping its capacity?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10388" PostTypeId="2" ParentId="10371" CreationDate="2019-02-04T14:07:58.163" Score="2" Body="&lt;p&gt;I would say Expert Systems is still being taught. For instance, if you look at some of the open courses like &lt;a href=&quot;https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/lecture-videos/lecture-3-reasoning-goal-trees-and-rule-based-expert-systems/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIT's&lt;/a&gt;, there are still lectures on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, looking at the &lt;a href=&quot;http://www.clipsrules.net/AboutCLIPS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLIPS&lt;/a&gt; documentation, you will find a couple of examples of usage from 2005. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I suspect is that Expert Systems are now embedded with &quot;normal systems&quot; in practice. Hence it may be difficult to distinguish from systems used on a daily basis for diagnostics, etc. and not as popular as before.&lt;/p&gt;&#xA;" OwnerUserId="15812" LastActivityDate="2019-02-04T14:07:58.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10391" PostTypeId="1" CreationDate="2019-02-04T18:18:53.410" Score="1" ViewCount="74" Body="&lt;p&gt;I have got multi-class object detector. One model accuracy evaluation of detection consists of: &lt;strong&gt;mAP, FP, FN, TP&lt;/strong&gt; for each class divided to two graphs and looks like this (I've used &lt;a href=&quot;https://github.com/rafaelpadilla/Object-Detection-Metrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;this repo&lt;/a&gt; for evaluation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rBRlb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rBRlb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I've got many of these evaluations (multiple times these two graphs for different models) and I would like to easily compare all these trained models (results) and put them to &lt;strong&gt;one&lt;/strong&gt; graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've searched through the whole Internet, but wasn't able to find suitable method of placing all the values to one graph. Also, the values of these three classes can be put together (eg. result mAP for this evaluation would be (75+ 68+ 66) / 3 = ~70%), so I would have just single value of each mAP, FN, FP, TP for one whole model evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What comes to my mind is the following graph (or maybe some kind of plot):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mYQLs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mYQLs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; It may not make sense to place mAP together with TP, etc. into one graph, but I would like to have all these values together to easily compare all the model evaluations. Also I am not really looking for a script, I can do the graph manually from values, but script would be more helpful. What really matters is, how to create meaningful graph with all the data :). If the post is more suitable for different kind of site, please, let me know.&lt;/p&gt;&#xA;" OwnerUserId="18760" LastEditorUserId="9947" LastEditDate="2019-02-04T18:32:12.960" LastActivityDate="2019-02-04T18:32:12.960" Title="How to create meaningful multiple object detection evaluation comparison graph?" Tags="&lt;neural-networks&gt;&lt;graphs&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10392" PostTypeId="2" ParentId="8227" CreationDate="2019-02-04T21:39:44.070" Score="10" Body="&lt;p&gt;I agree with Felicity's answer, except the part where he says that &amp;quot;C# doesn't look hopeful for keep pace.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;There are new frameworks such as ML.Net being developed right now and the new version of C# (C# 7.0-8.0) focus on performance while keeping simplicity. I cannot find the link right now, but there was a post on their GitHub explaining their new vision. By the way, the .Net Core performance is really close to C++. For these reasons, I think C# is a language that will be more and more popular within the next years in the AI industry.&lt;/p&gt;&#xA;&lt;p&gt;The downside of using C# is that you really need to know the language deeply in order to take full advantage of it. Personally, it took me ~2 years of full-time work to be really good at it while Python only took me two weeks. I think many people in the AI industry doesn't want to spend too much time on learning an language, so they prefer to opt for Python. &lt;strong&gt;But for references, C# is actually MUCH FASTER than Python as of February 4th, 2019.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Python vs Java (Python loses in every test): &lt;a href=&quot;https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python.html&quot; rel=&quot;noreferrer&quot;&gt;https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;.Net Core (C#) vs Java (C# wins in every tests): &lt;a href=&quot;https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html&quot; rel=&quot;noreferrer&quot;&gt;https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;.Net Core (C#) vs C++ (C# loses in every tests, but not by much): &lt;a href=&quot;https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharpcore-gpp.html&quot; rel=&quot;noreferrer&quot;&gt;https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharpcore-gpp.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;.Net Core 2.0 (2.1 is faster) vs other languages in Amazon Lambda: &lt;a href=&quot;https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581&quot; rel=&quot;noreferrer&quot;&gt;https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Also, I have developed in both languages, and I have found that it is much easier to develop good development practices using C# development tools than Python (this is a personal opinion - it's not a fact)&lt;/p&gt;&#xA;&lt;p&gt;Choosing a language for AI development really depends on multiples factors, so giving a general answer here is difficult.&lt;/p&gt;&#xA;" OwnerUserId="22011" LastEditorUserId="40035" LastEditDate="2020-08-02T14:27:18.613" LastActivityDate="2020-08-02T14:27:18.613" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10393" PostTypeId="1" CreationDate="2019-02-04T22:27:50.903" Score="1" ViewCount="17" Body="&lt;p&gt;Is it possible to sample from a distribution inside a neural network forward function? Assume that there is a NN and a sample is needed to be derived from it at every forward-pass to randomly set a layer-specific hyper-parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this operation differentiable&lt;/p&gt;&#xA;" OwnerUserId="10569" LastActivityDate="2019-02-04T22:27:50.903" Title="Sample from a distribution inside a NN layer" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10394" PostTypeId="1" CreationDate="2019-02-04T22:50:24.823" Score="1" ViewCount="30" Body="&lt;p&gt;Are there chatbots for Facebook Messenger or Skype available which are game-based, i.e. with which it is possible to play a short funny game? It should be possible to play the game for at least 10 minutes in sequence and it should be writing based and not based on clicking at boxes. That means the agent should be pretty clever, like Microsoft Zo bot, but instead of conducting random smalltalk a game should be played.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, are there bots for Facebook Messenger or Skype available which are nasty and unfriendly, i.e. which are offending?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you a lot in advance. I'm thankful for any help.&lt;/p&gt;&#xA;" OwnerUserId="21103" LastEditorUserId="1671" LastEditDate="2019-02-06T02:18:25.013" LastActivityDate="2019-02-06T02:18:25.013" Title="Game-based or nasty chatbot for Facebook Messenger or Skype" Tags="&lt;game-ai&gt;&lt;intelligent-agent&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10398" PostTypeId="2" ParentId="8227" CreationDate="2019-02-05T06:43:26.163" Score="1" Body="&lt;p&gt;I did use both languages for a few things and I really do like python because it’s just so neat and tidy. But if you have never coded before it might be a little hard for you to learn. (to truly learn; basics are easy)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest the following: If you have learned a different language already (coded more than &quot;hello world&quot;) then python might be the right deal otherwise start with a different language to learn a few basics before you are going to use and learn python. C# is a good choice then. It is important to learn how to code nice and tidy, how to comment und to understand what’s happening before stumbling into a field of &quot;AI&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to answer the question: Python it is! C# is maybe a choice for companies but I think even there it would not actually be used to code &quot;AI&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BUT: AI or in all honesty Neural Networks... that's  a pretty heavy topic as you wander the field. It has a lot of math and special coding in it and it will keep you &quot;tangled&quot;. Make time to learn the scientific wrappers and it does not really matter where you start!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish you luck and all the best! Have fun however you decide! &#xA;PS: if you are going to use python make sure to use tensorflow and keras in the beginning it just simplifies the start tremendously. And as an IDE I would personally choose PyCharm ;-)&lt;/p&gt;&#xA;" OwnerUserId="21604" LastEditorUserId="21604" LastEditDate="2019-02-05T06:52:46.887" LastActivityDate="2019-02-05T06:52:46.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10400" PostTypeId="2" ParentId="10003" CreationDate="2019-02-05T09:56:53.143" Score="0" Body="&lt;p&gt;Yes there is! If a model generalizes well to the test set, we already know that it has found &lt;em&gt;some&lt;/em&gt; useful features. However, the latent representation of the data may still be &quot;entangled&quot; - a single element of the latent vector may actually encode information about multiple attributes of the input, or a single attribute may be spread across multiple elements.  We usually prefer a representation in which the features are represented by the axes of the latent space - a &quot;disentangled&quot; representation.  For example, if we were encoding faces, it would be nice to have an axis for smiling/not, another for masculine/feminine, and ao on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pushing models to learn &quot;clean&quot; (disentangled) representations is an active sub-field of machine learning research with practical applications (like interpretability, but also because it makes it easier for &quot;downstream&quot; models to learn their tasks, e.g. a control policy in reinforcement learning system taking as input a learned representation from a &lt;a href=&quot;https://worldmodels.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;world model&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where to begin?  Start with L2 regularisation to push your network to &quot;spend&quot; those weights wisely (more weights close to zero =&gt; sparser latent vector) and work your way up from there.  &lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-02-05T09:56:53.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10402" PostTypeId="2" ParentId="10387" CreationDate="2019-02-05T11:02:22.190" Score="1" Body="&lt;p&gt;One way to keep the capacity while reducing the receptive field size is to add 1x1 conv layers instead of 3x3 (I did so within the DenseBlocks, there the first layer is a 3x3 conv and now followed by 4 times a 1x1 conv layer instead of the original 3x3 convs (which increase the receptive field)). In doing that, the number of parameters can be kept at a similar level. While 1x1 convolutions are good to add non-linearity, they are not useful to learn spatial information where neighboring pixels/values correlate. This might turn out to be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, experiments will have to show if this idea of adding 1x1 conv layers to keep the capacity proves useful or not in learning spatial features on a natural image data set.&lt;/p&gt;&#xA;" OwnerUserId="20191" LastActivityDate="2019-02-05T11:02:22.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10403" PostTypeId="1" AcceptedAnswerId="11297" CreationDate="2019-02-05T11:34:29.887" Score="7" ViewCount="475" Body="&lt;p&gt;The Softsign (a.k.a. ElliotSig) activation function is really simple: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f(x) = \frac{x}{1+|x|} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is bounded &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt;, has a first derivative, it is monotonic, and it is computationally extremely simple (easy for, e.g., a GPU).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why it is not widely used in neural networks? Is it because it is not infinitely derivable?&lt;/p&gt;&#xA;" OwnerUserId="15017" LastEditorUserId="2444" LastEditDate="2019-03-18T20:42:53.317" LastActivityDate="2019-03-18T20:42:53.317" Title="Why isn't the ElliotSig activation function widely used?" Tags="&lt;machine-learning&gt;&lt;activation-function&gt;&lt;performance&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10404" PostTypeId="1" CreationDate="2019-02-05T11:47:26.437" Score="1" ViewCount="363" Body="&lt;p&gt;I want to customize the 'Pendulum-v0' environment such that the action (the torque) from previous time step as well as from the current timestep serve as the inputs in the Env.step() function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem statement is that I want to generate torque from the controller which has a white Gaussian noise of magnitude 1 and then filter it with the torque generated in the previous timestep as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tor_ = tor_c + a*WGN ;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tor(t) = lambda*tor_ + (1-lambda)*tor(t-1) ;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L37&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L37&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see that the 'u' is an array of some numbers as the input which is afterward clipped from -max_torque to max_torque and then only the first element is taken as the torque value to calculate the states and the reward function for the given time-step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what does the value of the other elements signify? Are they&#xA;torque values from the previous time steps or is it that the length of the 'u' array is just 1 and its value is restricted between -max_torque to max_torque?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, I just wanna access the action (the torque value) from the previous time-step. Is it possible ? If yes, how?&lt;/p&gt;&#xA;" OwnerUserId="22023" LastActivityDate="2019-02-05T11:47:26.437" Title="Input for the Env.step() in the 'Pendulum-v0' environment" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="8" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10406" PostTypeId="1" CreationDate="2019-02-05T14:23:05.253" Score="2" ViewCount="132" Body="&lt;p&gt;In a neural network, each neuron represents some part of the input. For example, in the case of a MNIST digit, consider the stem of the number 9. Each neuron in the NN represents some part of this digit.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What determines which neuron will represent which part of the digit?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible that if we pass in the same input multiple times, each neuron can represent different parts of the digit?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How is this related to the back-propagation algorithm and chain rule? Is it the case that, before training the neural network, each neuron doesn't really represent anything of the input, and, as training proceeds, neurons start to represent some part of the input?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19583" LastEditorUserId="2444" LastEditDate="2019-05-08T13:23:53.187" LastActivityDate="2021-05-27T20:06:00.677" Title="Which neuron represents which part of the input?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;features&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10412" PostTypeId="1" CreationDate="2019-02-05T20:18:41.510" Score="2" ViewCount="39" Body="&lt;p&gt;I am trying to build a film review classifier where I determine if a given review is positive or negative (w/ Python). I'm trying to avoid any other ML libraries so that I can better understand the processes. Here is my approach and the problems that I am facing:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I mine thousands of film reviews as training sets and classify them as positive or negative. &lt;/li&gt;&#xA;&lt;li&gt;I parse through my training set and for each class, I build an array of unique words. &lt;/li&gt;&#xA;&lt;li&gt;For each document, I build a vector of TF-IDF values where the vector size is my number of unique words.&lt;/li&gt;&#xA;&lt;li&gt;I use a Gaussian classifier to determine: &lt;span class=&quot;math-container&quot;&gt;$$P(C_i|w)=P(C_i)P(w|C)=P(C_i)*\dfrac{1}{\sqrt{2\pi}\sigma_i}e^{-(1/2)(w-\mu_i)^T\sigma_i^{-1}(w-\mu_i)}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the my document in a vector, &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is a particular class, &lt;span class=&quot;math-container&quot;&gt;$\mu_i$&lt;/span&gt; is the mean vector and &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt; is my covariance matrix. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This approach &lt;em&gt;seems&lt;/em&gt; to makes sense. My problem is that my algorithm is much too slow. As an example, I have sampled over 1,500 documents and I have determined over 40,000 unique words. This mean that each of my document vectors has 40,000 entries and if I were to build a covariance matrix, it would have dimensions 40,000 by 40,000. Even I were able to generate the entirety of &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt;, but then I would have to compute the matrix product in the exponent, which will take an extraordinarily long time just to classify one document. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have experimented with a multinomial approach, which is working well. I very curious on how to make this work more efficiently. I realise the matrix multiplication runtime can't be improved, and I was hoping for insight on how others are able to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some things I have tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Filtered any stop words (but this still leaves me with tens of thousands of words)&lt;/li&gt;&#xA;&lt;li&gt;Estimated &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt; by summing over a couple of documents. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22017" LastActivityDate="2019-02-05T20:18:41.510" Title="My naive (ha!) Gaussian Naive Bayes classifier is too slow" Tags="&lt;python&gt;&lt;bayes&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10413" PostTypeId="1" CreationDate="2019-02-05T20:29:30.240" Score="4" ViewCount="266" Body="&lt;p&gt;Could advanced AI decide to kill all humans? &lt;/p&gt;&#xA;" OwnerUserId="22035" LastEditorUserId="1671" LastEditDate="2019-02-05T21:48:40.390" LastActivityDate="2020-12-02T20:01:40.257" Title="Could artificial general intelligence harm humanity?" Tags="&lt;control-problem&gt;&lt;superintelligence&gt;&lt;mythology-of-ai&gt;&lt;neo-luddism&gt;&lt;value-alignment&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ClosedDate="2021-01-23T04:40:24.113" ContentLicense="CC BY-SA 4.0" />
  <row Id="10414" PostTypeId="1" CreationDate="2019-02-05T21:22:43.773" Score="2" ViewCount="123" Body="&lt;p&gt;I just finished the three-part series of &lt;a href=&quot;https://www.coursera.org/learn/probabilistic-graphical-models-3-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probabilistic Graphical Models&lt;/a&gt; courses from Stanford over on Coursera. I got in to them because I realized there is a certain class of problem for which the standard supervised learning approaches don't apply, for which graph search algorithms don't work, problems that don't look like RL control problems, that don't even exactly look like the kind of clustering I came to call &quot;unsupervised learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my AI courses in the &lt;a href=&quot;https://www.gatech.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Institute&lt;/a&gt;, we talked briefly about Bayes Nets, but it was almost as if professors considered that preamble to hotter topics like Neural Nets. Meanwhile I heard about &quot;Expectation Maximization&quot; and &quot;Inference&quot; and &quot;Maximum Likelihood Estimation&quot; all the time, like I was supposed to know what they were talking about. It frustrated me not to be able to remember statistics well enough to &lt;em&gt;feel&lt;/em&gt; these things, so I decided to fill the hole by delving deeper in to PGMs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Throughout, Koller gives examples of how to apply PGMs to things like image segmentation and speech recognition, examples that seem completely dated now because we have CNNs and LSTMs, even &lt;a href=&quot;https://alexgkendall.com/computer_vision/bayesian_deep_learning_for_safe_ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep nets that encode notions of uncertainty about their beliefs&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I gather PGMs are good when:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You know the structure of the problem and can encode domain knowledge that way.&lt;/li&gt;&#xA;&lt;li&gt;You need a generative model.&lt;/li&gt;&#xA;&lt;li&gt;You want to learn more than just one X -&gt; Y mapping, when you instead need a more general-purpose model that can be queried from several sides to answer different kinds of questions.&lt;/li&gt;&#xA;&lt;li&gt;You want to feed the model inputs that look more like probability distributions than like samples.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What else are they good for? Have they not been outstripped by more advanced methods for lots of problems now? In which domains or for which specific kinds of problem are they still king? How are they complementary to modern advanced methods? Should I dedicate time to reading any of Koller &amp;amp; Friedman's massive tome on this subject? How dated is this set of MOOCs?&lt;/p&gt;&#xA;" OwnerUserId="18196" LastEditorUserId="18196" LastEditDate="2019-02-05T21:27:55.593" LastActivityDate="2019-02-05T21:27:55.593" Title="How do PGMs factor in to modern ML?" Tags="&lt;machine-learning&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10415" PostTypeId="2" ParentId="10360" CreationDate="2019-02-06T02:36:41.683" Score="2" Body="&lt;p&gt;Heuristics can be understood aas rules.  Typically heuristics are thought of as problem-specific strategies.  &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expert systems&lt;/a&gt; were an early form of AI that utilized rules-based decisions.  In a game-playing context, heuristics can be &lt;a href=&quot;https://en.wikipedia.org/wiki/Strategy_(game_theory)#Pure_and_mixed_strategies&quot; rel=&quot;nofollow noreferrer&quot;&gt;pure strategies&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A heuristic function would be one that includes a some predefined decision rules.  Russell and Norvig have a nice chapter on informed (heuristic) search strategies. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Heuristic functions are the most common form in which additional knowledge of the&#xA;  problem is imparted to the search algorithm.&#xA;  &lt;br&gt; &lt;sub&gt;Artificial Intelligence: A Modern Approach; 3.5 &lt;a href=&quot;https://faculty.psau.edu.sa/filedownload/doc-7-pdf-a154ffbcec538a4161a406abf62f5b76-original.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;h(n) as opposed to f(n) or g(n)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-07T17:52:04.937" LastActivityDate="2019-02-07T17:52:04.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10417" PostTypeId="2" ParentId="10380" CreationDate="2019-02-06T05:59:17.517" Score="1" Body="&lt;p&gt;You need to define &quot;simulation&quot; more specific. Playing Mario, Swapping face on image/video, or generating simulation of objects that are orbiting use different techniques.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Playing Mario or &quot;AI that playing game&quot;: the AI agent trained on available environment (Mario game, so the environment is not generated) and learn the best sequential actions to achieve the goal. It runs the game thousand times, when it did a wrong action then it gets &quot;penalties&quot; that improve its knowledge. The algorithm that can be used is &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning&lt;/a&gt;, but some earlier paper use &lt;a href=&quot;https://www.youtube.com/watch?v=qv6UVOQ0F44&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithm to generate the best action&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Face swap: It's close to computer vision area, some methods that I know use &lt;a href=&quot;http://openaccess.thecvf.com/content_ICCV_2017/papers/Korshunova_Fast_Face-Swap_Using_ICCV_2017_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Style Transfer principle&lt;/a&gt; (Convolutional Neural Network) to make transformation of face of one image to another image. You can read the basic of &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;style transfer here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generating physical movement: I don't know too much about this topic but I know there are some papers talk about this, &lt;a href=&quot;https://github.com/google/FluidNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fluid Net&lt;/a&gt; from Google workers and &lt;a href=&quot;https://ge.in.tum.de/publications/2017-sig-chu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; from TU Munchen. At a glance they also use CNN to improve the result but the main simulation came from &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler_equations_(fluid_dynamics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler Fluid Equation&lt;/a&gt;. So if you need to generate object that orbiting, I think you need to find equations that models that movement.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-02-06T05:59:17.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10418" PostTypeId="1" CreationDate="2019-02-06T07:33:01.187" Score="3" ViewCount="138" Body="&lt;p&gt;I work with neural networks for real-time image processing on embedded softwares and I tested different architectures (Googlenet, Mobilenet, Resnet, custom networks...) and different hardware solutions (boards, processors, AI accelerators...). I noticed that the performance of the system, in terms of inference time, does not depend only on the processor but also on other factors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;For example&lt;/em&gt;, I have two boards from different manifacturers, B1 (with a cheap processor) and B2 (with a better processor), and two neural networks, N1 (very light with regular convolutions and fully connected layers) and N2 (very large, with inception modules and many layers). The inference time for N1 is better on B1, while for N2 it is better on N2. Moreover, it happens that, as the software is executed, the inference time changes over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: &lt;strong&gt;in an embedded system, what are the aspects that impact on the inference time, and how?&lt;/strong&gt; I am interested not only in the hardware features but also in the neural network architecture (convolutional filter size, types of layers and so on).&lt;/p&gt;&#xA;" OwnerUserId="16671" LastActivityDate="2019-02-07T14:44:51.260" Title="What are the aspects that most impact on the inference time for neural networks in embedded systems?" Tags="&lt;neural-networks&gt;&lt;hardware&gt;&lt;performance&gt;&lt;embedded-design&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10419" PostTypeId="1" CreationDate="2019-02-06T10:43:37.067" Score="2" ViewCount="117" Body="&lt;p&gt;I trained some Gaussian process model with the Python library GPFlow on a dataset consisting of &lt;span class=&quot;math-container&quot;&gt;$(X, Y)$&lt;/span&gt;, inputs and outputs, in a regression setting. This model gives me pretty good predictions in the sense that the relative error is small almost everywhere. I want to use the uncertainty as well, which is given in a GPFlow setting in the form of a standard deviation (STD) associated with every prediction. Here's my problem: I normalised both inputs and outputs before training (separately) using sklearn's StandardScaler (effectively making the data normally distributed with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; mean and unit STD). So the STD given by the model pertains to the scaled data. How do I &quot;rescale&quot; the uncertainty estimates of the GP to the actual data? Using the inverse_transform function of the output scaler makes little sense. This issue might be easier solvable if I scaled with a MinMaxScaler (squishing all data points into the unit interval) by dividing by the length of the range of the original output set (at least I think it works that way). But how about the case of the StandardScaler? Any insights will be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="16901" LastActivityDate="2021-02-12T04:06:40.497" Title="GPFlow: Gaussian Process Uncertainty Quantification" Tags="&lt;machine-learning&gt;&lt;python&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10420" PostTypeId="1" CreationDate="2019-02-06T13:48:43.677" Score="3" ViewCount="55" Body="&lt;p&gt;How we should train a CNN model when training dataset contains only limited number of cases, and the trained model is supposed to predict class (label) for several other cases, which has not seen before? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing there was hidden independent features describing the label repeated in previously seen cases of dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example let's consider we want to train a model to movement time series signals so it can predict some sort of activities (labels), and we have long record of movement signals (e.g. hours) for limited number of persons (e.g. 30) during various type of activities (e.g. 5), we may say these signals carry three type of hidden features:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Noise-features:&lt;/strong&gt; Common features between every persons/activities &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Case-features:&lt;/strong&gt; features mostly correlated with persons&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Class-features:&lt;/strong&gt; features mostly correlated with activities&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We want to train the model such it learn mostly &lt;code&gt;Class-features&lt;/code&gt; and eliminate 1st and 2nd types of features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conventional types of supervised-learning CNN learns all features how dataset represents them. In my test the model learned those 30 person activities very well, but on new persons it only predict randomly (i.e. 20% success). Over-fitted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems there are three straight workaround to this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extracting &lt;code&gt;class-features&lt;/code&gt; and using a shallow classifier.&lt;/li&gt;&#xA;&lt;li&gt;Increasing dataset wideness by recording signal on other persons: it can get so expensive or impossible in some situations. &lt;/li&gt;&#xA;&lt;li&gt;Signal augmentation: by augmenting signals such it does not change &lt;code&gt;Class-features&lt;/code&gt;, and making augmented &lt;code&gt;Case-features&lt;/code&gt;. it seems to me harder than 1st workaround. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any other workaround on this type of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example specific type of training to use, to learn the model how different cases similarly follow &lt;code&gt;class-features&lt;/code&gt; during class changes, eliminating &lt;code&gt;case-features&lt;/code&gt; which varies case by case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for very long question!&lt;/p&gt;&#xA;" OwnerUserId="22055" LastEditorUserId="22055" LastEditDate="2019-02-06T21:41:02.330" LastActivityDate="2019-02-06T21:41:02.330" Title="How to train CNN such it eliminate dependent features and focuses on independent ones?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;datasets&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10421" PostTypeId="1" AcceptedAnswerId="10426" CreationDate="2019-02-06T15:04:21.750" Score="3" ViewCount="499" Body="&lt;p&gt;In actor critic, &lt;a href=&quot;https://medium.freecodecamp.org/an-intro-to-advantage-actor-critic-methods-lets-play-sonic-the-hedgehog-86d6240171d&quot; rel=&quot;nofollow noreferrer&quot;&gt;The equations for calculating the loss in actor critic&lt;/a&gt; are an &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;actor&lt;/em&gt; loss (parameterized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$log[\pi_\theta(s_t,a_t)]Q_w(s_t,a_t)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a &lt;em&gt;critic&lt;/em&gt; loss (parameterized by &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s_t,a_t) + \gamma Q_w(s_{t+1}, a_{t+1}) - Q_w(s_{t}, a_t).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is &lt;a href=&quot;https://datascience.stackexchange.com/questions/20535/what-is-experience-replay-and-what-are-its-benefits&quot;&gt;bootstrapping in experience replay&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_i(\theta_i) = \mathbb{E}_{(s, a, r, s') \sim U(D)} \left[ \left(r + \gamma \max_{a'} Q(s', a'; \theta_i^-) - Q(s, a; \theta_i)\right)^2 \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that bootstrapping is comparable to the critic loss, except that the &lt;span class=&quot;math-container&quot;&gt;$max$&lt;/span&gt; operation is lacking from the critic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;As i see it, (correct me if I'm wrong)&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(s_t,a_t) = V(s_{t+1}) + r_t$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; is the actual action that had been taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The critic, as I understand, estimates &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What exactly is the critic calculating?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What In actor critic outputs &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1},a_{t+1})$&lt;/span&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me like the critic calculates the average next state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; value, over all possible actions, with their corresponding probabilities, yielding&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a_t) = r_t + \sum_{a_{t+1} \in A}P(a_{t+1}|s_t)V(s_{t+1})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which would mean that in order to get &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1}, a_{t+1})$&lt;/span&gt; for the above formula, I would need to calculate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1}, a_{t+1}) = r_{t+1} + \sum_{a_{t+2} \in A}P(a_{t+2}|s_{t+1})V(s_{t+2})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$V(s_{t+2})$&lt;/span&gt; is the critic output on &lt;span class=&quot;math-container&quot;&gt;$s_{t+2}$&lt;/span&gt;, a state we get to by taking action &lt;span class=&quot;math-container&quot;&gt;$a_{t+1}$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; but I am not sure that is indeed the meaning of the critic output and still it is unclear to me how to get &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1}, a_{t+1})$&lt;/span&gt; from actor critic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If indeed that is what's being calculated, then why is it mathematically true that an improvement is being made? Or why does it make sense (even if not mathematically always true)?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Practical use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use actor critic with experience replay in an environment with a large action space (could be continuous). Therefore, I cannot use the &lt;span class=&quot;math-container&quot;&gt;$max$&lt;/span&gt; term. I need to understand the correct equation for the critic loss, and why it works.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="21645" LastEditDate="2019-02-06T16:53:22.487" LastActivityDate="2019-02-06T18:07:28.307" Title="Meaning of Actor Output in Actor Critic Reinforcement Learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10422" PostTypeId="1" AcceptedAnswerId="10427" CreationDate="2019-02-06T15:06:38.913" Score="2" ViewCount="100" Body="&lt;p&gt;Imagine that the agent receives a positive reward upon reaching a state . Once the state  has been reached the positive reward associated with it vanishes and appears somewhere else in the state space, say at state ′. The reward associated to ′ also vanishes when the agent visits that state once and re-appears at state . This goes periodically forever. Will discounted Q-learning converge to the optimal policy in this setup? Is yes, is there any proof out there, I couldn't find anything.&lt;/p&gt;&#xA;" OwnerUserId="22060" LastEditorUserId="2444" LastEditDate="2019-02-15T17:40:55.420" LastActivityDate="2019-02-15T17:40:55.420" Title="Will Q-learning converge to the optimal state-action function when the reward periodically changes?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10425" PostTypeId="1" CreationDate="2019-02-06T17:48:39.643" Score="3" ViewCount="32" Body="&lt;p&gt;I am reviewing my Neural Network lectures and I have a doubt: My book's (Haykin) batch PTA describes a cost function which is defined over the set of the misclassified inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have always been taught to use MSE &amp;lt; X as a stopping condition for the training process. Is the batch case different? Should I use as stopping condition size(misclassified) &amp;lt; Y (and as a consequence when the weight change is very little)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, the book uses the same symbol for both the training set and the misclassified input set. Does this mean that my training set changes each epoch? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Than you for any answer!&lt;/p&gt;&#xA;" OwnerUserId="21676" LastEditorUserId="21676" LastEditDate="2019-02-06T21:00:08.693" LastActivityDate="2019-02-06T21:00:08.693" Title="Batch PTA stopping condition" Tags="&lt;neural-networks&gt;&lt;theory&gt;&lt;perceptron&gt;&lt;computational-learning-theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10426" PostTypeId="2" ParentId="10421" CreationDate="2019-02-06T18:07:28.307" Score="4" Body="&lt;p&gt;When using the loss function for the critic described in your question, the Actor-Critic is an &lt;strong&gt;on-policy&lt;/strong&gt; approach (as are most Actor-Critic methods). Your intuition as to what it is learning seems to be quite close, but the notation/terminology is not quite on point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First it's important to realize that the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; critic is an &lt;strong&gt;estimator&lt;/strong&gt;, we're training it to &lt;strong&gt;estimate&lt;/strong&gt; state-action values. You could say that we are training it such that it can hopefully provide accurate estimates of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_w^{\pi} (s_t, a_t) \approx \mathbb{E}_{\pi} \left[ r_t + \gamma V^{\pi}(s_{t+1}) \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll notice I've added quite a number of symbols there in comparison to your &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a_t) = r_t + V(s_{t+1})$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have added the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; superscript to &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;; this denotes the &lt;strong&gt;behaviour policy&lt;/strong&gt;, which is the policy that we're using to generate experience. In on-policy methods, this is equal to the &lt;strong&gt;target policy&lt;/strong&gt; (the policy for which we're learning to predict values). Adding this superscript makes explicit the fact that we're learning expected returns for states and state-action pairs that are only accurate under the assumption that we continue following the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; policy from state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; onwards.&lt;/li&gt;&#xA;&lt;li&gt;I added the discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, which is probably just a tiny detail you forgot.&lt;/li&gt;&#xA;&lt;li&gt;I added &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt; to indicate that we're trying to estimate an &lt;strong&gt;expectation under &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;&lt;/strong&gt; (and the environment's dynamics).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, the critic is trained to estimate &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s, a)$&lt;/span&gt;, which can intuitively be interpreted as the long-term discounted rewards that we expect to collect when executing &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and selecting actions according to the distribution &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; subsequently. It definitely still is trying to estimate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values for state-action pairs, not just &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; values for states alone.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;What In actor critic outputs &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1},a_{t+1})$&lt;/span&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In practice, when using the loss function described in your question, &lt;span class=&quot;math-container&quot;&gt;$a_{t+1}$&lt;/span&gt; really simply is a single action selected in an actual trajectory of experience by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. The trained network simply takes &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; as input, and the output corresponding to &lt;strong&gt;a single action&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$a_{t+1}$&lt;/span&gt; &lt;strong&gt;as selected by the policy&lt;/strong&gt; is used as the value for &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1},a_{t+1})$&lt;/span&gt; in the update rule. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The update rule does not involve any sum over all actions, multiplied with their probabilities. The &quot;trick&quot; is that we do not just run the update rule a single time, but we expect to generate lots (sometimes millions) of trajectories as experience, and we repeatedly run the update rule. In different trajectories, we'll experience the different actions &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; as samples with approximately the correct frequencies, and &lt;strong&gt;in expectation&lt;/strong&gt; we'll have proper update targets (except for potential bias resulting from function approximation).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to use actor critic with experience replay in an environment with a large action space&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Actor-Critic method in your question is, as I mentioned above, &lt;strong&gt;on-policy&lt;/strong&gt;. This means that the experience used in update rules has to be generated according to exactly the same policy for which you are also learning value estimates. This is incompatible with the idea of experience replay, because old trajectories stored in a replay buffer were generated by older versions of your policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;strong&gt;off-policy&lt;/strong&gt; Actor-Critic methods which can correct for the mismatch in distributions and use experience replay, but these are going to be quite a bit more complicated. Examples are &lt;a href=&quot;https://arxiv.org/abs/1611.01224&quot; rel=&quot;nofollow noreferrer&quot;&gt;ACER&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1802.01561&quot; rel=&quot;nofollow noreferrer&quot;&gt;IMPALA&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-06T18:07:28.307" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="10427" PostTypeId="2" ParentId="10422" CreationDate="2019-02-06T18:13:15.203" Score="3" Body="&lt;p&gt;No, it will not converge in the general case (maybe it might in extremely convenient special cases, not sure, didn't think hard enough about that...). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically everything in Reinforcement Learning theory (including convergence proofs) relies on the &lt;strong&gt;Markov property&lt;/strong&gt;; the assumption that the current state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; includes all relevant information, that the history leading up to &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; is no longer relevant. In your case, this property is violated; it is important to remember whether or not we visited &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; more recently than &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose if you &quot;enhance&quot; your states such that they include that piece of information, then it should converge again. This means that you'd essentially double your state-space. For every state that you have in your &quot;normal&quot; state space, you'd have to add a separate copy that would be used in cases where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; was visited more recently than &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-02-06T18:28:50.813" LastActivityDate="2019-02-06T18:28:50.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10429" PostTypeId="1" CreationDate="2019-02-06T23:29:15.037" Score="2" ViewCount="38" Body="&lt;p&gt;I'm seeing a lot of examples of neuroevolution techniques involving games or robot problems. Can neuroevolution be used for solving tasks other than games? For example, how could you transform a CSV file of psychological data to determine the best life actions you can get from a self-report questionnaire?&lt;/p&gt;&#xA;" OwnerUserId="22070" LastEditorUserId="2444" LastEditDate="2019-07-07T19:17:57.167" LastActivityDate="2019-07-07T19:17:57.167" Title="Can neuroevolution be used for solving tasks other than games?" Tags="&lt;neural-networks&gt;&lt;applications&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10430" PostTypeId="1" AcceptedAnswerId="10433" CreationDate="2019-02-06T23:51:42.447" Score="2" ViewCount="93" Body="&lt;p&gt;A lot of research has been done to create the optimal (or &quot;smartest&quot;) RL agent, using methods such as A2C. An agent can now beat humans at playing Go, Chess, Poker, Atari Games, DOTA, etc. But I think these kind of agents will never be a friend of humans, because humans won't play with a agent that always beats them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could we create an agent that doesn't outperform humans, but it has the human level skill, so that when it plays agains a human, the human is still motivated to beat it?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2019-02-15T17:37:00.600" LastActivityDate="2019-02-15T17:37:00.600" Title="How do we create a good agent that does not outperform humans?" Tags="&lt;reinforcement-learning&gt;&lt;philosophy&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10431" PostTypeId="1" AcceptedAnswerId="10604" CreationDate="2019-02-07T06:32:54.797" Score="8" ViewCount="1548" Body="&lt;p&gt;I have two Machine Learning models (I use LSTM) that have a different result on the validation set (~100 samples data):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Model A: Accuracy: ~91%, Loss: ~0.01&lt;/li&gt;&#xA;&lt;li&gt;Model B: Accuracy: ~83%, Loss: ~0.003&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The size and the speed of both models are almost the same. So, which model should I choose?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2021-01-29T01:50:43.190" LastActivityDate="2021-01-29T01:50:43.190" Title="Should I choose a model with the smallest loss or highest accuracy?" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;loss&gt;&lt;accuracy&gt;&lt;validation&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10432" PostTypeId="2" ParentId="5497" CreationDate="2019-02-07T08:55:21.403" Score="1" Body="&lt;p&gt;Yes. The mutation can either disable or enable a gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's in the original NEAT implementation released by Dr. Kenneth O. Stanley.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Declared in &lt;code&gt;genetics.h&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void mutate_toggle_enable(int times); /* toggle genes on or off */&#xA;void mutate_gene_reenable();  /* Find first disabled gene and enable it */&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/soft-view.php?SoftID=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nn.cs.utexas.edu/soft-view.php?SoftID=4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nn.cs.utexas.edu/?neat-c&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nn.cs.utexas.edu/?neat-c&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20193" LastActivityDate="2019-02-07T08:55:21.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10433" PostTypeId="2" ParentId="10430" CreationDate="2019-02-07T09:20:44.870" Score="3" Body="&lt;p&gt;You basically have to degrade the result, assuming that the machine always finds the best move. There are a number of possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;restrict the depth of searching&lt;/em&gt;. In early chess programs I believe that was the main way of regulating the difficulty. You stop the evaluation of moves after a particular depth in your search tree has been reached. This would be equivalent to only looking ahead two moves instead of twenty.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;set a time limit&lt;/em&gt;. This is somewhat similar the restricting the depth of the search, but more generally applicable. If your algorithm accumulates candidate moves, and the general tendency is to get to the better moves after first finding a number of weaker ones, then you can stop at a given point in time and return what you have found then.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;distort available information&lt;/em&gt;. This might not be that applicable to games such a chess, but you could restrict the information the machine has available for evaluating moves. Something like the &quot;Fog of War&quot; often used in strategy games. With incomplete information it is harder to find a good move, though it is not impossible, which makes it more challenging than, say, restricting the depth of search too much.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;sub-optimal evaluation function&lt;/em&gt;. If you have a function that evaluates the quality of a move, simply fudge that function to not return the best value. Perhaps add a random offset to the return value to make it less deterministic/predictable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are probably other methods as well; the tricky part is to tread the fine line between appearing to be a weaker (but consistent) player, and just being a random number generator.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-02-07T09:20:44.870" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10436" PostTypeId="1" AcceptedAnswerId="10463" CreationDate="2019-02-07T11:34:08.653" Score="2" ViewCount="99" Body="&lt;p&gt;I have been researching about &lt;strong&gt;determining some key points on an image&lt;/strong&gt;, in this case I'm gonna use &lt;strong&gt;cloth (top side of human body)&lt;/strong&gt; pictures. I want to detect some corner points on those. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bN9go.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bN9go.png&quot; alt=&quot;Points on a t-shirt&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have &lt;strong&gt;two&lt;/strong&gt; solutions on my mind. &lt;strong&gt;One&lt;/strong&gt; CNN with transpose layers resulting in &lt;strong&gt;heatmap&lt;/strong&gt; where I can get points. &lt;strong&gt;The second&lt;/strong&gt; is to get &lt;strong&gt;24 number&lt;/strong&gt; as output from the model &lt;strong&gt;meaning 12(x,y) point&lt;/strong&gt;. I don't know which one will be better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In face point detection, they use the second method. In human pose estimation, they use method one.&lt;/strong&gt; So what do you suggest me to use? or do you have any new ideas? Thanks&lt;/p&gt;&#xA;" OwnerUserId="16864" LastEditorUserId="16864" LastEditDate="2019-02-07T11:40:19.067" LastActivityDate="2019-02-08T17:41:12.000" Title="Key Point Extraction the best method?" Tags="&lt;deep-learning&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10440" PostTypeId="2" ParentId="10418" CreationDate="2019-02-07T14:44:51.260" Score="1" Body="&lt;p&gt;You can expect that the inference time will strongly depend on particular hardware and software present on your platform. First, GPU equipped devices (eg NVidia TX) will outperform non-GPU equipped devices (eg. Intel Movidius). Second, software support (eg. cudnn, TensorRT) will make dramatic further impact. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, we have measured the inference time of two convolutional models. The model A requires 250% more floating point operations than the model B. Yet, the two models take roughly the same time to evaluate on our device, since the layers of model A are better optimized in software. Conclusion: algorithmic complexity and practical execution time on a particular computing platform are not bound to be proportional any more.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-07T14:44:51.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10441" PostTypeId="1" CreationDate="2019-02-07T15:22:17.117" Score="1" ViewCount="70" Body="&lt;p&gt;In the book &quot;Reinforcement Learning: An Introduction&quot;, by Sutton and Barto, they provided the &quot;Q-learning prioritized sweeping&quot; algorithm, in which the model saves the next state and the immediate reward, for each state and action, that is, &lt;span class=&quot;math-container&quot;&gt;$Model(S_{t},A_{t}) \leftarrow S_{t+1}, R_{t+1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we want to use &quot;SARSA prioritized sweeping&quot;, should we save &quot;next state, immediate reward, and next action&quot;, that is, &lt;span class=&quot;math-container&quot;&gt;$Model(S_{t},A_{t}) \leftarrow  S_{t+1}, R_{t+1}, A_{t+1}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2019-02-07T16:41:44.647" LastActivityDate="2019-02-07T17:16:01.583" Title="What should be saved in SARSA prioritized sweeping?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10442" PostTypeId="1" CreationDate="2019-02-07T15:38:58.217" Score="5" ViewCount="255" Body="&lt;p&gt;In this &lt;a href=&quot;https://www.youtube.com/watch?v=Jw3ZnWFjDfM&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;, the lecturer states that &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; are equivalent representations of the reward function. Intuitively, this is the case, according to the same lecturer, because &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; can be made to represent the state and the action. Furthermore, apparently, the Markov decision process would change depending on whether we use one representation or the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a formal proof that shows that these representations are equivalent. Moreover, how exactly would the Markov decision process change if we use one representation over the other? Finally, when should we use one representation over the other and why are there three representations? I suppose it is because one representation may be more convenient than another in certain cases: which cases? How do you decide which representation to use?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T17:02:46.500" LastActivityDate="2021-01-27T16:14:02.840" Title="How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;proofs&gt;&lt;notation&gt;&lt;reward-functions&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10443" PostTypeId="2" ParentId="10441" CreationDate="2019-02-07T17:10:13.300" Score="3" Body="&lt;p&gt;SARSA is an on-policy method. Your historical choices of action will have been made using older Q values, and thus from a different policy. In addition, the action that was taken at the time may not have been typical for the agent (it may have been exploring). So you don't usually want to re-use historical action choices to calculate TD targets in single-step SARSA, because these may introduce bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Provided you are performing single-step SARSA, then you can re-generate the action choice sampling from the current best policy. This is similar to generating the max Q value in the TD target value &lt;span class=&quot;math-container&quot;&gt;$R_{t+1} + \text{max}_{a'} Q(S_{t+1},a')$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could do this using a regular SARSA sampling the action choice:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R_{t+1} + Q(S_{t+1}, a' \sim \pi(\cdot|S_{t+1}) )$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you could use Expected SARSA and take a weighted mean of all possible actions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R_{t+1} + \sum_{a' \in \mathcal{A}(S_{t+1})} \pi(a'|S_{t+1})Q(S_{t+1}, a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically these would both be on-policy with respect to evaluating the TD Target, but off-policy with respect to the distribution of &lt;span class=&quot;math-container&quot;&gt;$S_t, A_t$&lt;/span&gt; that you are running the update for. That's already the case due to prioritised sweeping focussing more updates on certain transitions, but could be a big difference when using a neural network to approximate Q. Bear in mind that making TD learning methods off-policy can have a negative impact on stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to process multi-step updates, then you do have to reference &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt; and adjust for when the historical data makes a different action choice than your current estimate of the best policy. This would commonly use &lt;em&gt;importance sampling&lt;/em&gt;. This is true for Q-learning as well however, so there would still be no difference in what you store between Q-learning and SARSA.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-07T17:16:01.583" LastActivityDate="2019-02-07T17:16:01.583" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10444" PostTypeId="2" ParentId="10442" CreationDate="2019-02-07T17:49:41.827" Score="7" Body="&lt;p&gt;In general the different reward functions &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; are &lt;em&gt;not&lt;/em&gt; equivalent mathematically, so you will not find any formal proof.&lt;/p&gt;&#xA;&lt;p&gt;It is possible for the functions to resolve to the same value in a specific MDP, if, for instance, you use &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; and the value returned only depends on &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s') = R(s)$&lt;/span&gt;. This is not true in general, but as the reward functions are often under your control, it can be the case quite often.&lt;/p&gt;&#xA;&lt;p&gt;For instance, in scenarios where the agent's goal is to reach some pre-defined state, as in the grid world example from the video, then there is no difference between &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;. Given that is the case, for those example problems you may as well use &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;, as it simplifies the expressions that you need to calculate for algorithms like Q-learning.&lt;/p&gt;&#xA;&lt;p&gt;I think the lecturer did not mean &amp;quot;equivalent&amp;quot; in the mathematical sense, but in the sense that future lectures will use one of the functions, and a lot of what you will learn is going to be much the same as if you had used a different reward function.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Finally, when should we use one representation over the other and why are there three representations?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Typically, I don't use any of those representations by default. I tend to use Sutton &amp;amp; Barto's &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; notation for combined state transitions and rewards. That expression returns probability of transitioning to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and receiving reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; when starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. For discrete actions, you can re-write the expectation of the different functions &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; in terms of this function as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[R(s)] = \sum_{a \in \mathcal{A}(s)}\sum_{s' \in \mathcal{S}}\sum_{r \in {R}}rp(s', r|s, a)\qquad*$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[R(s,a)] = \sum_{s' \in \mathcal{S}}\sum_{r \in {R}}rp(s', r|s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[R(s,a, s')] = \sum_{r \in {R}}r\frac{p(s', r|s, a)}{p(s'|s, a)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think this is one way to see how the functions in the video are closely related.&lt;/p&gt;&#xA;&lt;p&gt;Which one would you use? Depends on what you are doing. If you want to simplify an equation or code, then use the simplest version of the reward function that fits with the reward scheme you set up for the goals of the problem. For instance, if there is one goal state to exit a maze, and an episode ends as soon as this happens, then you don't care how you got to that state or what the previous state was, and can use &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In practice, what happens if you use a different reward function is that you need to pay attention to where it appears in things like the Bellman equation for theoretical treatments. When you get to implement model-free methods like Q-learning, &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt; or its variants don't really appear except in the theory.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;* This is not technically correct in all cases. The assumption I have made is that &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt; is a reward granted at the point of &lt;em&gt;leaving&lt;/em&gt; state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and is independent of how the state is left and where the agent ends up next.&lt;/p&gt;&#xA;&lt;p&gt;If this was a fixed reward for &lt;em&gt;entering&lt;/em&gt; state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, regardless of how, then it could be written around &lt;span class=&quot;math-container&quot;&gt;$R(s')$&lt;/span&gt; as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[R(s')] = \sum_{s \in \mathcal{S}}\sum_{a \in \mathcal{A}(s)}\sum_{r \in {R}}rp(s', r|s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;i.e. by summing all the rewards that end up at &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-01-27T16:14:02.840" LastActivityDate="2021-01-27T16:14:02.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10445" PostTypeId="2" ParentId="10442" CreationDate="2019-02-07T19:43:01.373" Score="4" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt; denote a probability distribution over rewards that our agent may get in some MDP as a reward for entering a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The easiest case is to demonstrate that we can also choose to write this as &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt;: simply take &lt;span class=&quot;math-container&quot;&gt;$\forall a: R(s, a) = R(s)$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$\forall a \forall s': R(s, a, s') = R(s)$&lt;/span&gt;, as also described in &lt;a href=&quot;https://ai.stackexchange.com/a/10444/1641&quot;&gt;Neil's answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; denote a probability distribution over rewards that our agent may get as a reward for executing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The easy case of demonstrating equivalence to &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; is already handled above, but can we also construct an MDP in which we only use the &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt; notation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way I can think of to do so (may not be the cleanest way) would be to construct a new MDP with a bunch of &quot;dummy&quot; states &lt;span class=&quot;math-container&quot;&gt;$z(s, a)$&lt;/span&gt;, such that executing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in the original MDP deterministically leads to a dummy state &lt;span class=&quot;math-container&quot;&gt;$z(s, a)$&lt;/span&gt; in the new MDP. Note that I write &lt;span class=&quot;math-container&quot;&gt;$z(s, a)$&lt;/span&gt; to make the connection back to the original MDP explicit, but this is a completely independent MDP and you should view it as just a state &quot;&lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the reward distribution &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; that was associated with the state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; in the original MDP can be written as &lt;span class=&quot;math-container&quot;&gt;$R(z(s, a))$&lt;/span&gt;, which is now only a function of the state in the new MDP. In this dummy state &lt;span class=&quot;math-container&quot;&gt;$z(s, a)$&lt;/span&gt; in the new MDP, every possible action &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; should have exactly the same transition probabilities towards new states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; as the original transition probabilities for executing &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; back in the original MDP. This guarantees that the same policy has the same probabilities of reaching certain states in both MDPs; only in our new MDP the agent is forced to transition through these dummy states in between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you also have a discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; in the original MDP, I guess you should use a discount factor &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\gamma}$&lt;/span&gt; in the new MDP, because every step in the original MDP requires two steps (one step into a dummy state, and one step out of it again) in the new MDP.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The final case for &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; could be done in a very similar way, but it would get even more complicated to write out formally. The intuition would be the same though. Above, we pretty much &quot;baked&quot; state-action pairs &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; from the original MDP into additional dummy states, such that in the new MDP we have states that &quot;carry the same amount of information&quot; as a full state-action pair in the original MDP. For the &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; case, you'd need to devise an even uglier solution with even more information &quot;baked into&quot; dummy states, such that you can treat full &lt;span class=&quot;math-container&quot;&gt;$(s, a, s')$&lt;/span&gt; triples as single &lt;span class=&quot;math-container&quot;&gt;$z(s, a, s')$&lt;/span&gt; states in the new MDP.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, when should we use one representation over the other and why are there three representations? I suppose it is because one representation may be more convenient than another in certain cases: which cases? How do you decide which representation to use?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would recommend always using the &lt;strong&gt;simplest representation&lt;/strong&gt; that happens to be &lt;strong&gt;sufficient&lt;/strong&gt; to describe how the rewards work in your environment in a natural way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you have a two-player zero-sum game where terminal game states give a reward of &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for losses, draws, or wins, it is sufficient to use the &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt; notation; the reward depends on the terminal game state reached, not on how it was reached. Another example would be a maze with a specific goal position, as described in Neil's answer. You could use the more complex &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; notations... but there wouldn't be much of a point in doing so really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an environment in which the reached state and the played action both have influence on the reward distribution, then it's much more sensible to just use the &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; notation rather than trying to define a massively overcomplicated MDP with dummy states as I've tried to do above. An example would be... let's say we're playing a quizz, where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; denotes the current question, and different actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; are different answers that the agent can give. Then it's natural to model the problem as an MDP where &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; is only positive if &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the correct answer to the question &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-07T19:43:01.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10446" PostTypeId="1" AcceptedAnswerId="10455" CreationDate="2019-02-07T22:33:01.607" Score="2" ViewCount="215" Body="&lt;p&gt;I'm building a customer assistant chatbot in Python. So, I am modelling this problem as a text classification task. I have available more or less 7 hundred sentences of an average length of 15 words (unbalanced class).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think, knowing that I have to do an oversampling, is this dataset large enough?&lt;/p&gt;&#xA;" OwnerUserId="20780" LastEditorUserId="2444" LastEditDate="2020-05-22T01:24:18.437" LastActivityDate="2020-05-22T01:24:18.437" Title="Is a dataset of roughly 700 sentences of an average length of 15 words enough for text classification?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;datasets&gt;&lt;chat-bots&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10447" PostTypeId="1" AcceptedAnswerId="10449" CreationDate="2019-02-07T22:33:27.293" Score="7" ViewCount="5359" Body="&lt;p&gt;I have a dataset which I have loaded as a data frame in Python. It consists of 21392 rows (the data instances, each row is one sample) and 1972 columns (the features). The last column i.e. column 1972 has string type labels (14 different categories of target labels). I would like to use a CNN to classify the data in this case and predict the target labels using the available features. This is a somewhat unconventional approach though it seems possible. However, I am very confused on how the methodology should be as I could not find any sample code/ pseudo code guiding on using CNN for Classifying non-image data, either in Tensorflow or Keras. Any help in this regard will be highly appreciated. Cheers!&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2021-04-13T15:54:51.953" Title="How to use CNN for making predictions on non-image data?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10449" PostTypeId="2" ParentId="10447" CreationDate="2019-02-08T00:52:25.103" Score="6" Body="&lt;p&gt;You can use CNN on any data, but it's &lt;strong&gt;recommended&lt;/strong&gt; to use CNN only on data that have spatial features (It might still work on data that doesn't have spatial features, see DuttaA's comment below).&lt;/p&gt;&#xA;&lt;p&gt;For example, in the image, the connection between pixels in some area gives you another feature (e.g. edge) instead of a feature from one pixel (e.g. color). So, as long as you can shaping your data, and your data have spatial features, you can use CNN.&lt;/p&gt;&#xA;&lt;p&gt;For Text classification, there are connections between characters (that form words) so you can use CNN for &lt;a href=&quot;https://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;text classification in character level&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For Speech recognition, there is also a connection between frequencies from one frame with some previous and next frames, so you can also use &lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/CNN_ASLPTrans2-14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNN for speech recognition&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If your data have spatial features, just reshape it to a 1D array (for example in text) or 2D array (for example in Audio). Tensorflow's function &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/conv1d&quot; rel=&quot;nofollow noreferrer&quot;&gt;conv1d&lt;/a&gt; and &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/conv2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;conv2d&lt;/a&gt; are general function that can be used on any data. It look the data as an array of floating-point, not as image/audio/text.&lt;/p&gt;&#xA;&lt;p&gt;But if your data doesn't have spatial features, for example, your features are &lt;code&gt;price&lt;/code&gt;, &lt;code&gt;salary&lt;/code&gt;, &lt;code&gt;status_marriage&lt;/code&gt;, etc. I think you don't need CNN, and using CNN won't help.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="36737" LastEditDate="2021-04-13T15:54:51.953" LastActivityDate="2021-04-13T15:54:51.953" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10450" PostTypeId="1" AcceptedAnswerId="10453" CreationDate="2019-02-08T01:47:33.333" Score="3" ViewCount="631" Body="&lt;p&gt;In my understanding, Q-learning gives you a deterministic policy. However, can we use some technique to build a meaningful stochastic policy from the learned Q values? I think that simply using a softmax won't work. &lt;/p&gt;&#xA;" OwnerUserId="22105" LastEditorUserId="22105" LastEditDate="2019-02-08T14:45:11.230" LastActivityDate="2019-02-08T14:45:11.230" Title="Can Q-learning be used to derive a stochastic policy?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;stochastic-policy&gt;&lt;deterministic-policy&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10453" PostTypeId="2" ParentId="10450" CreationDate="2019-02-08T08:19:41.780" Score="2" Body="&lt;p&gt;No it is not possible to use Q-learning to build a deliberately stochastic policy, as the learning algorithm is designed around choosing solely the maximising value at each step, and this assumption carries forward to the action value update step &lt;span class=&quot;math-container&quot;&gt;$Q_{k+1}(S_t,A_t) = Q_k(S_t,A_t) + \alpha(R_{t+1} +\gamma\text{max}_{a'}Q_k(S_{t+1},a') - Q_k(S_t,A_t))$&lt;/span&gt; - i.e. the assumption is that the agent will always choose the highest Q value, and that in turn is used to calculate the TD target values. If you use a stochastic policy as the target policy, then the assumption is broken and the Q table (or approximator) would not converge to estimates of action value for the policy*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy produced by Q-learning can only be treated as stochastic when there is more than one maximum action value in a particular state - in which case you can select equivalent maximising values using any distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory you could use the Q values to derive various distributions, such as a Boltzmann distribution, or softmax as you suggest (you will want to include some weighting factor to make softmax work in general). These can work nicely for the behaviour policy, for further training, and as an alternative to the more common &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy approach. However, they are not optimal policies, and &lt;em&gt;the training algorithm will not adjust the probabilities in any meaningful way related to the problem you want to solve&lt;/em&gt;. You can set a value for e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, or have more sophisticated action choice with more parameters, but no value-based method can provide a way to change those parameters to make action choice optimal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cases where a stochastic policy would perform better - e.g. Scissor, Paper, Stone versus an opponent exploiting patterns in the agent's behaviour - then value based methods provide no mechanism to learn a correct distribution, and they typically fail to learn well in such environments. Instead you need to look into policy gradient methods, where the policy function is learned directly and can be stochastic. The most basic policy gradient algorithm is REINFORCE, and variations on Actor-Critic such as A3C are quite popular.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* You could get around this limitation by using an estimator that does work with a stochastic target policy, e.g. SARSA or Expected SARSA. Expected SARSA can even be used off-policy to learn one stochastic policy's Q values whilst behaving differently. However, neither of these provide you with the ability to change the probability distribution towards an optimal one.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-08T12:40:23.427" LastActivityDate="2019-02-08T12:40:23.427" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10454" PostTypeId="1" CreationDate="2019-02-08T08:32:50.123" Score="1" ViewCount="35" Body="&lt;p&gt;Let assume that we have dataset of variables (random events), I apriori would like to set dependency conditions between some of them and perform structure learning to figure out the rest of the net. How it can be done practically (e.g. some libs like bnlearn etc.) or at least in theory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was trying to google it but haven't found anything related&lt;/p&gt;&#xA;" OwnerUserId="22113" LastActivityDate="2021-03-13T15:05:15.477" Title="How to perform structure learning for Bayes Net given already partially constructed net?" Tags="&lt;machine-learning&gt;&lt;bayes&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10455" PostTypeId="2" ParentId="10446" CreationDate="2019-02-08T09:19:36.653" Score="2" Body="&lt;p&gt;It depends on the number of classes; we are getting good results with about 40 training examples per class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good way to get an idea about this is to run a test with an increasing set of training data, evaluating the result as you go along. Obviously, with a small set (eg 3 sentences per class), it will be very poor, but the accuracy should quickly increase and then stabilise at a higher level. With larger amounts of data you will probably only find a small increase or no change at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Collecting this data would not only give you confidence in your conclusion, it would also be a good supporting argument when you have to ask for more training data, or have to justify the poor performance of the classifier if you do find the data set is too small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, set up an automated &lt;a href=&quot;https://www.openml.org/a/estimation-procedures/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;10-fold cross validation&lt;/a&gt;, feed an increasing amount of your available data into it, sit back, and graph the results.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-02-08T09:19:36.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10456" PostTypeId="1" CreationDate="2019-02-08T10:52:11.697" Score="2" ViewCount="84" Body="&lt;p&gt;I have chromosomes with floating-point representation with values between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. For example&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$p_1 = [0.1, 0.2, 0.3]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_2 = [0.5, 0.6, 0.7]$&lt;/span&gt; be two parents. Both comply with the set of constraints. In my case, the major constraint is &lt;span class=&quot;math-container&quot;&gt;$$ p_1[1]*p_1[2] - k*p_1[0] \geq 0 $$&lt;/span&gt; for any chromosome &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt;. For the example above we can take &lt;span class=&quot;math-container&quot;&gt;$k=0.3$&lt;/span&gt;, which renders &lt;span class=&quot;math-container&quot;&gt;$c_2$&lt;/span&gt; infeasible.&lt;/p&gt;&#xA;&lt;p&gt;However, the children produced by 1 point crossover, we get &lt;span class=&quot;math-container&quot;&gt;$c_1 = [0.1, 0.6, 0.7]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_2 = [0.5, 0.2, 0.3]$&lt;/span&gt; out of which 1 or both may not comply with the given constraints.&lt;/p&gt;&#xA;&lt;p&gt;A similar scenario can also occur with a small perturbation of values due to mutation strategy. Correct me if I am wrong in the belief that such kind of scenarios might arise irrespective of the strategy employed for crossover and mutation.&lt;/p&gt;&#xA;&lt;p&gt;What are the options to handle such kinds of cases?&lt;/p&gt;&#xA;" OwnerUserId="22115" LastEditorUserId="2444" LastEditDate="2021-01-30T22:00:36.617" LastActivityDate="2021-01-30T22:00:36.617" Title="How to handle infeasibility caused due to crossover and mutation in genetic algorithm for optimization?" Tags="&lt;genetic-algorithms&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;&lt;constrained-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10457" PostTypeId="2" ParentId="10456" CreationDate="2019-02-08T15:37:28.287" Score="1" Body="&lt;p&gt;You have two broad categories of options, prevention and repair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prevention means defining a crossover and mutation operator that try to be more intelligent about respecting the constraints. Suppose you have an encoding where each individual is a list of integers, and the constraint is that there can't be duplicates. You might define a crossover operator that did something like the following. For each position in the offspring, choose randomly from one parent such that, if possible, you choose a value that hasn't already appeared in the offspring. If both parents have values at that position that are already used in the offspring, choose a random value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That operator would avoid violating the constraint in the first place, while attempting as much as possible to have the offspring inherit information from the parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to just let the operator violate constraints and then deal with the ramifications afterward. You don't go into details about what your constraints actually are, just that &lt;code&gt;c1=[0.1, 0.6, 0.7]&lt;/code&gt; violates them. Let's say the constraint is that the third position should not be more than 4x greater than the first one. OK, so then let's take this offspring and adjust either the first or third item. Maybe we make the new individual into &lt;code&gt;c1=[0.2, 0.6, 0.7]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often, you want some element of randomness in either option so that you don't strongly bias the production of new search points. In my example, don't always make the first element larger. Sometimes, make the third element smaller or produce some random combination of both to repair the constraint violation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, both options typically strongly benefit from domain knowledge. Design an operator that understands your problem and tries to intelligently solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastActivityDate="2019-02-08T15:37:28.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10458" PostTypeId="2" ParentId="10447" CreationDate="2019-02-08T16:05:53.400" Score="4" Body="&lt;p&gt;The convolutional models are a method of choice when your problem is translation invariant (or covariant). In image classification, the image should be classified into class 'cow' if a cow is present in any part of the image. In text classification, different orders of phrases and sentences result in related meaning. In speech recognition, the same syllable is used at different places to build different words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your problem, you should check whether some subsequences of your 1791 columns give rise to the same meaning although they are located at different places within the sample. If the answer is positive, then convolutional layers are likely going to improve the performance.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-08T16:05:53.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10459" PostTypeId="2" ParentId="2279" CreationDate="2019-02-08T16:10:11.447" Score="0" Body="&lt;p&gt;Object detection models work in a very similar fashion to what you have proposed. They output dense predictions at reduced resolutions. Each prediction fires if an object center is located within the respective region of the image. Of course, there are various further developments, but the main idea is exactly that.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-08T16:10:11.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10461" PostTypeId="1" CreationDate="2019-02-08T16:45:52.937" Score="2" ViewCount="443" Body="&lt;p&gt;I've used a table to represent the Q function, while an agent is being trained to catch the cheese without touching the walls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first and last row (and column) of the matrix are associated with the walls. I placed in last cell a cheese that agent must catch while being training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I've done it with dynamic states and, when necessary, I resized matrix with new states. I've used four actions (up, left, right and down). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like now to use an ANN to represent my Q function. How do I do that? What should be the input and output of such neural network?&lt;/p&gt;&#xA;" OwnerUserId="17405" LastEditorUserId="2444" LastEditDate="2019-02-10T20:27:56.790" LastActivityDate="2019-02-10T20:27:56.790" Title="How do I convert table-based to neural network-based Q-learning?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10462" PostTypeId="1" CreationDate="2019-02-08T17:39:21.540" Score="1" ViewCount="468" Body="&lt;p&gt;LSTM is supposed to be the right tool to capture path-dependency in time-series data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided to run a simple experiment (simulation) to assess the extent to which LSTM is better able to &lt;em&gt;understand&lt;/em&gt; path-dependency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The setting is very simple. I just simulate a bunch (N=100) paths coming from 4 different data generating processes. Two of these processes represent a &lt;em&gt;real&lt;/em&gt; increase and a &lt;em&gt;real&lt;/em&gt; decrease, while the other two &lt;em&gt;fake&lt;/em&gt; trends that eventually revert to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following plot shows the simulated paths for each category:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cqa2v.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cqa2v.png&quot; alt=&quot;Simulated paths&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The candidate machine learning algorithm will be given the first 8 values of the path ( t in [1,8] ) and will be trained to predict the subsequent movement over the last 2 steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the feature vector is &lt;code&gt;X = (p1, p2, p3, p4, p5, p6, p7, p8)&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the target is &lt;code&gt;y = p10 - p8&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I compared LSTM with a simple Random Forest model with 20 estimators. Here are the definitions and the training of the two models, using Keras and scikit-learn:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# LSTM&#xA;model = Sequential()&#xA;model.add(LSTM((1), batch_input_shape=(None, H, 1), return_sequences=True))&#xA;model.add(LSTM((1), return_sequences=False))&#xA;model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])&#xA;history = model.fit(train_X_LS, train_y_LS, epochs=100, validation_data=(vali_X_LS, vali_y_LS), verbose=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Random Forest&#xA;RF = RandomForestRegressor(random_state=0, n_estimators=20)&#xA;RF.fit(train_X_RF, train_y_RF);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results are the summarized by the following scatter plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/81fRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/81fRi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the Random Forest model is clearly outperforming the LSTM. The latter seems to be not able to distinguish between the &lt;em&gt;real&lt;/em&gt; and the &lt;em&gt;fake&lt;/em&gt; trends.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Do you have any idea to explain why this is happening?&lt;/h3&gt;&#xA;&#xA;&lt;h3&gt;How would you modify the LSTM model to make it better at this problem?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Some remarks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The data points are divided by 100 to make sure gradients do not explode&lt;/li&gt;&#xA;&lt;li&gt;I tried to increase the sample size, but I noticed no differences&lt;/li&gt;&#xA;&lt;li&gt;I tried to increase the number of epochs over which the LSTM is trained, but I noticed no differences (the loss becomes stagnant after a bunch of epochs)&lt;/li&gt;&#xA;&lt;li&gt;You can find the code I used to run the experiment &lt;a href=&quot;https://pastebin.com/QL2DuKDE&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22129" LastEditorUserId="22129" LastEditDate="2019-02-08T22:22:00.840" LastActivityDate="2019-02-08T22:22:00.840" Title="Experiment shows that LSTM does worse than Random Forest... Why?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10463" PostTypeId="2" ParentId="10436" CreationDate="2019-02-08T17:41:12.000" Score="0" Body="&lt;p&gt;The 2nd method would make sense only if your object is at the same position in all test images. You would have such situation if you operated on crops located by a separate object detection algorithm. This happens to be the case in facial key-point detection. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 1st method would be much more robust to various object poses since it is translation covariant by design. If a keypoint is detected at location A, it will be equally well detected at any other position with the same set of parameters.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-08T17:41:12.000" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10464" PostTypeId="2" ParentId="10461" CreationDate="2019-02-08T17:48:56.533" Score="1" Body="&lt;p&gt;A neural network (NN) is a &quot;function approximator&quot;, that is, it is a model that can be used to approximate functions. In fact, a neural network with at least one hidden layer is a &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;universal&quot; function approximator&lt;/a&gt; (that is, it can approximate any function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mathematics, a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is usually represented as a mapping of the form &lt;span class=&quot;math-container&quot;&gt;$f: D \rightarrow C$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; are respectively the &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain_of_a_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;domain&lt;/a&gt; (inputs) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Codomain&quot; rel=&quot;nofollow noreferrer&quot;&gt;codomain&lt;/a&gt; (outputs) of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; means that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; &quot;maps&quot; &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. A NN (with at least one hidden layer) can thus approximate any function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; of this form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your context, the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; table is a function: it is a mapping between states and actions (inputs) and Q values (outputs), which are the &quot;expected future cumulative reward&quot; (that you will obtain if you take a certain action from a certain state and then continue to follow the same policy). The Q function can thus more formally be denoted by &lt;span class=&quot;math-container&quot;&gt;$Q: (S, A) \rightarrow \mathcal{R}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the &quot;space of states&quot; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the &quot;space of actions&quot; in your problem. Initially, your Q table does not contain the correct (optimal) values. However, after training (or learning), hopefully, your Q table will be an approximation to the optimal Q function for your specific problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you then represent this table using a NN? What should be the inputs and outputs of the NN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose that your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; table is implemented as matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$M[s, a]$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value for the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. So, in this case, the combination of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the input, whereas &lt;span class=&quot;math-container&quot;&gt;$M[s, a]$&lt;/span&gt; is the output of your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To represent this table as a NN, you can thus have as input of the NN the state and action, and as output the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value. You will then train the NN (using e.g. back-propagation) to learn the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, given a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; as input, for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a \in \mathcal{A}$&lt;/span&gt;. So, during your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning algorithm, instead of using &lt;span class=&quot;math-container&quot;&gt;$M[s, a]$&lt;/span&gt; to represent &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, you will simply use the current output of your neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, in practice, I think, you will likely encounter problems while training the NN, if you update the weights of your NN at every time step (because e.g. NNs do not cope well with correlated data, and, in general, the &quot;experience&quot; data you will obtain from time step to time step will be highly correlated). Anyway, this is the basic idea of how to use a NN to represent a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function. There are other ways, but this is the simplest one, at least, conceptually.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-08T21:10:38.877" LastActivityDate="2019-02-08T21:10:38.877" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="10465" PostTypeId="1" CreationDate="2019-02-08T19:37:12.240" Score="1" ViewCount="53" Body="&lt;p&gt;I have a data analysis problem that I can reduce to one similar to analyzing the trajectories in the images below. These images show the tracks of subatomic particles interacting in a bubble chamber. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's pretty obvious that by eye, easily discernible patterns can be seen. I want very much to know more about how classification and segmentation can be done using neural networks for this type of image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These images are binary. The trajectory is either at a point in the image or it isn't. As can be seen, trajectories cross over one another, Some data appears to be missing in otherwise smooth curves, at arbitrary points along those curves. (My data may be more sparse in this respect.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical paper on bubble chamber analysis that I would find deals with the analysis of the physics after trajectories have been classified and segmented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone identify some papers that address this or something similar in the context of neural networks? I am not able to find anything recent on automated methods at all, but my google fu may not be up to the challenge. (By the way, I am less interested in some of the parametric methods like Hough Transforms. I'd like to focus on the neural approach.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I posted &lt;a href=&quot;https://ai.stackexchange.com/questions/10036/pixel-level-detection-of-each-object-of-the-same-class-in-an-image&quot;&gt;this previous question&lt;/a&gt; which wasn't quite as specific as this one. I hope there is some available research in this area related to physics that might give me some insights that are more directly related to my problem.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g936O.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g936O.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uyuk6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uyuk6.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8439" LastActivityDate="2019-02-08T19:37:12.240" Title="Bubble Chamber Image Analysis Using Neural Network" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10466" PostTypeId="1" AcceptedAnswerId="10498" CreationDate="2019-02-08T19:56:50.297" Score="1" ViewCount="221" Body="&lt;p&gt;I'm a beginner in machine learning and I was trying to make a test neural network for digits recognition from scratch using Numpy. I used MNIST dataset for training and testing. Input layer have 28*28 neurons which correspond to each pixel of image that must be recognized. Output layer have 10 neurons which correspond to each digit (0-9), and return values from 0 to 1 which mean chance that the corresponding digit is displayed on the image. Class Layer represents a separate layer and contains links to previous and next layer (if prevLayer is None, the current layer is input layer; if nextLayer is None, the current layer is output). The forward() method is responsible for passing data through neural network. The backprop() method is responsible for training of neural network via Backpropagation algorithm. A Layer object contains weights (W) between previous and current layer (input layer object doesn't contain weights). 'data_in' property contains a vector of calculated values before passing them into activation function. Property 'data' contains the values after activation function. But, unfortunately, it doesn't work: returned value of loss function doesn't decrease during training and neural network returns the same result during testing. I assume that bugs might be associated with backprop() and softmax_derivatime() methods. I tried in vain to find all bugs. Here's my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def ReLU(x):&#xA;    return np.maximum(0, x)&#xA;&#xA;def ReLU_derivative(x):&#xA;    return np.greater(x, 0).astype(int)&#xA;&#xA;def softmax(x):&#xA;    shift = x - np.max(x)&#xA;    return np.exp(shift) / np.sum(np.exp(shift))&#xA;&#xA;def softmax_derivative(x):&#xA;    sm_array = softmax(x)&#xA;    J = np.zeros((x.size, x.size))&#xA;    for i in range(x.size):&#xA;        for j in range(x.size):&#xA;            delta = np.equal(i, j).astype(int)&#xA;            J[j, i] = sm_array[0][i] * (delta - sm_array[0][j])&#xA;    return J&#xA;&#xA;class Layer:&#xA;    def __init__(self, size, prev_layer=None):&#xA;        self.size = size&#xA;        self.prevLayer = prev_layer&#xA;        self.nextLayer = None&#xA;        self.data = None&#xA;        self.data_in = None&#xA;        if prev_layer is not None:&#xA;            self.prevLayer.nextLayer = self&#xA;            self.W = np.random.random((self.prevLayer.size, size))&#xA;            self.W_bias = np.array([np.random.random(size)])&#xA;        else:&#xA;            self.W = None&#xA;            self.W_bias = None&#xA;&#xA;    def forward(self):&#xA;        if self.prevLayer is not None:&#xA;            self.data_in = np.dot(self.prevLayer.data, self.W)&#xA;            self.data_in += np.dot([[1]], self.W_bias)&#xA;            if self.nextLayer is not None:&#xA;                self.data = ReLU(self.data_in)&#xA;                self.nextLayer.forward()&#xA;            else:&#xA;                self.data = softmax(self.data_in)&#xA;        else:&#xA;            self.nextLayer.forward()&#xA;&#xA;    def backprop(self, expected_output=None, prev_delta=None):&#xA;        if prev_delta is None:&#xA;            #print(self.data_in)&#xA;            delta = np.dot(-(expected_output - self.data), softmax_derivative(self.data_in))&#xA;            delta_bias = delta&#xA;        else:&#xA;            delta = np.dot(prev_delta, self.nextLayer.W.T) * ReLU_derivative(self.data_in)&#xA;            delta_bias = np.dot(prev_delta, self.nextLayer.W_bias.T) * ReLU_derivative(self.data_in)&#xA;        training_velocity = 0.1&#xA;        W_dif = np.dot(self.prevLayer.data.T, delta) * training_velocity&#xA;        W_bias_dif = np.dot([[1]], delta_bias) * training_velocity&#xA;        if self.prevLayer.prevLayer is not None:&#xA;            self.prevLayer.backprop(prev_delta=delta)&#xA;        self.W -= W_dif&#xA;        self.W_bias -= W_bias_dif&#xA;&#xA;f_images = open(&quot;train-images.idx3-ubyte&quot;, &quot;br&quot;)&#xA;f_images.seek(4)&#xA;f_labels = open(&quot;train-labels.idx1-ubyte&quot;, &quot;br&quot;)&#xA;f_labels.seek(8)&#xA;images_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;rows_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;cols_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;&#xA;input_layer = Layer(rows_number*cols_number)&#xA;hidden_layer1 = Layer(rows_number*cols_number*7//10, input_layer)&#xA;hidden_layer2 = Layer(rows_number*cols_number*7//10, hidden_layer1)&#xA;output_layer = Layer(10, hidden_layer2)&#xA;digits = np.array([np.zeros(10)])&#xA;&#xA;input_image = np.array([np.zeros(rows_number * cols_number)])&#xA;for k in range(images_number):&#xA;    for i in range(rows_number):&#xA;        for j in range(cols_number):&#xA;            input_image[0][i*cols_number+j] = int.from_bytes(f_images.read(1), byteorder='big') / 255.0 * 2 - 1&#xA;    input_layer.data = input_image&#xA;    input_layer.forward()&#xA;    current_digit = int.from_bytes(f_labels.read(1), byteorder='big')&#xA;    digits[0][current_digit] = 1&#xA;    output_layer.backprop(expected_output=digits)&#xA;    print(np.sum((digits - output_layer.data)**2)/2)&#xA;    digits[0][current_digit] = 0&#xA;    if((k+1) % 1000 == 0):&#xA;        print(str(k+1) + &quot; / &quot; + str(images_number))&#xA;f_images.close()&#xA;f_labels.close()&#xA;&#xA;f_images = open(&quot;t10k-images.idx3-ubyte&quot;, &quot;br&quot;)&#xA;f_images.seek(4)&#xA;f_labels = open(&quot;t10k-labels.idx1-ubyte&quot;, &quot;br&quot;)&#xA;f_labels.seek(8)&#xA;images_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;rows_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;cols_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;&#xA;for k in range(images_number):&#xA;    for i in range(rows_number):&#xA;        for j in range(cols_number):&#xA;            input_image[0][i*cols_number+j] = int.from_bytes(f_images.read(1), byteorder='big')&#xA;    input_layer.data = input_image&#xA;    input_layer.forward()&#xA;    current_digit = int.from_bytes(f_labels.read(1), byteorder='big')&#xA;    print(output_layer.data)&#xA;&#xA;f_images.close()&#xA;f_labels.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would appreciate for any help. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="21567" LastEditorUserId="21567" LastEditDate="2019-02-10T16:06:42.957" LastActivityDate="2019-02-10T22:31:52.487" Title="A neural network for digits recognition doesn't work (MNIST, Numpy)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;object-recognition&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2019-02-16T19:38:45.047" ContentLicense="CC BY-SA 4.0" />
  <row Id="10467" PostTypeId="1" CreationDate="2019-02-08T21:04:14.843" Score="3" ViewCount="123" Body="&lt;p&gt;Suppose that we are doing machine translation. We have a conditional language model  with attention where we are are trying to predict a sequence &lt;span class=&quot;math-container&quot;&gt;$y_1, y_2, \dots, y_J$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, \dots x_I$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$$P(y_1, y_2, \dots, y_{J}|x_1, x_2, \dots x_I) = \prod_{j=1}^{J} p(y_j|v_j, y_1, \dots, y_{j-1})$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$v_j$&lt;/span&gt; is a context vector that is different for each &lt;span class=&quot;math-container&quot;&gt;$y_j$&lt;/span&gt;. Using an RNN with a encoder-decoder structure, each element &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; of the input sequence and &lt;span class=&quot;math-container&quot;&gt;$y_j$&lt;/span&gt; of the output sequence is converted into an embedding &lt;span class=&quot;math-container&quot;&gt;$h_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt; respectively: &lt;span class=&quot;math-container&quot;&gt;$$h_i = f(h_{i-1}, x_i) \\ s_j = g(s_{j-1},[y_{j-1}, v_j])$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is some function of the previous input state &lt;span class=&quot;math-container&quot;&gt;$h_{i-1}$&lt;/span&gt; and the current input word &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is some function of the previous output state &lt;span class=&quot;math-container&quot;&gt;$s_{j-1}$&lt;/span&gt;, the previous output word &lt;span class=&quot;math-container&quot;&gt;$y_{j-1}$&lt;/span&gt; and the context vector &lt;span class=&quot;math-container&quot;&gt;$v_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we want the process of predicting &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt; to &quot;pay attention&quot; to the correct parts of the encoder states (context vector &lt;span class=&quot;math-container&quot;&gt;$v_j$&lt;/span&gt;). So: &lt;span class=&quot;math-container&quot;&gt;$$v_j = \sum_{i=1}^{I} \alpha_{ij} h_i$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\alpha_{ij}$&lt;/span&gt; tells us how much weight to put on the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; state of the source vector when predicting the &lt;span class=&quot;math-container&quot;&gt;$j^{th}$&lt;/span&gt; word of the output vector. Since we want the &lt;span class=&quot;math-container&quot;&gt;$\alpha_{ij}$&lt;/span&gt;s to be probabilities, we use a softmax function on the similarities between the encoder and decoder states: &lt;span class=&quot;math-container&quot;&gt;$$\alpha_{ij} = \frac{\exp(\text{sim}(h_i, s_{j-1}))}{\sum_{i'=1}^{I} \exp(\text{sim}(h_i, s_{j-1}))}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in additive attention, the similarities of the encoder and decoder states are computed as: &lt;span class=&quot;math-container&quot;&gt;$$\text{sim}(h_i, s_{j}) = \textbf{w}^{T} \text{tanh}(\textbf{W}_{h}h_{i} +\textbf{W}_{s}s_{j})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\textbf{w}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\textbf{W}_{h}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\textbf{W}_{s}$&lt;/span&gt; are learned attention parameters using a  one-hidden layer feed-forward network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the intuition behind this definition? Why use the &lt;span class=&quot;math-container&quot;&gt;$\text{tanh}$&lt;/span&gt; function? I know that the idea is to use one layer of a neural network to predict the similarities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Added.&lt;/strong&gt; This description of machine translation/attention is based on the Coursera course &lt;a href=&quot;https://www.coursera.org/learn/language-processing/lecture/1nQaG/attention-mechanism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Processing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22131" LastEditorUserId="22131" LastEditDate="2019-02-08T22:26:46.283" LastActivityDate="2019-02-08T22:26:46.283" Title="What is the intuition behind the calculation of the similarity between encoder and decoder states?" Tags="&lt;recurrent-neural-networks&gt;&lt;attention&gt;&lt;machine-translation&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10471" PostTypeId="1" AcceptedAnswerId="10489" CreationDate="2019-02-09T10:17:59.980" Score="4" ViewCount="508" Body="&lt;p&gt;I am studying a knowledge base (KB) from the book &amp;quot;Artificial Intelligence: A Modern Approach&amp;quot; (by Stuart Russell and Peter Norvig) and from &lt;a href=&quot;https://www.cs.princeton.edu/courses/archive/fall16/cos402/lectures/402-lec12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this series of slides&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A formula is satisfiable if there is some assignment to the variables that makes the formula evaluate to true. For example, if we have the boolean formula &lt;span class=&quot;math-container&quot;&gt;$A \land B$&lt;/span&gt;, then the assignments &lt;span class=&quot;math-container&quot;&gt;$A=\text{true}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B=\text{true}$&lt;/span&gt; make it satisfiable. Right?&lt;/p&gt;&#xA;&lt;p&gt;But what does it mean for a KB to be consistent? The definition (given at slide 14 of &lt;a href=&quot;https://www.cs.princeton.edu/courses/archive/fall16/cos402/lectures/402-lec12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this series of slides&lt;/a&gt;) is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a KB is consistent with formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$M(KB \cup \{ f \})$&lt;/span&gt; is non-empty (there is a world in which KB is true and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is also true).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can anyone explain this part to me with an example?&lt;/p&gt;&#xA;" OwnerUserId="21719" LastEditorUserId="2444" LastEditDate="2021-01-21T18:44:20.720" LastActivityDate="2021-01-21T18:44:20.720" Title="When is a knowledge base consistent?" Tags="&lt;definitions&gt;&lt;logic&gt;&lt;knowledge-representation&gt;&lt;norvig-russell&gt;&lt;knowledge-base&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10472" PostTypeId="1" CreationDate="2019-02-09T11:24:19.650" Score="1" ViewCount="169" Body="&lt;p&gt;I've recently started reading a book about deep learning. The book is titled &quot;Grokking Deep Learning&quot; (by Andrew W Trask). In chapter 3 (pages 44 and 45), it talks about multiplying vectors using dot product and element-wise multiplication. For instance, taking 3 scalar inputs (vector) and 3 vector weights (matrix) and multiplying. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, when multiplying vectors the size needs to be identical. The concept I have a hard time understanding is multiplying vectors by a matrix. The book gives an example of an 1x4 vector being multiplied by 4x3 matrix. The output is an 1x3 vector. I'm am confused because I assumed multiplying vector by matrix needs the same number of columns, but I have read that the matrices need rows equal to the vectors columns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I do not have an equal number of columns, how does my deep learning algorithm multiply each input in my vector by a corresponding weight?&lt;/p&gt;&#xA;" OwnerUserId="22145" LastEditorUserId="2444" LastEditDate="2019-02-13T02:35:43.733" LastActivityDate="2019-02-13T02:35:43.733" Title="How are vectors and matrices multiplied in supervised machine learning?" Tags="&lt;deep-learning&gt;&lt;math&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10473" PostTypeId="2" ParentId="10472" CreationDate="2019-02-09T12:13:37.543" Score="1" Body="&lt;p&gt;In general, when people do not explicitly state it, a vector &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^n$&lt;/span&gt; is usually considered a &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Row_and_column_vectors&quot; rel=&quot;nofollow noreferrer&quot;&gt;column vector&lt;/a&gt;&quot;, that is, you can think of it as the matrix &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^{n \times 1}$&lt;/span&gt; (that is, a matrix with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; rows and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; column). &#xA;If it is not explicitly stated and you assume that the given vector is a column vector, but the dimensions do not match, then you should check that the dimensions match if you consider the given vector as a &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Row_and_column_vectors&quot; rel=&quot;nofollow noreferrer&quot;&gt;row vector&lt;/a&gt;&quot; (because it might be the case that the author is implicitly considering the vectors as row vectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, you can multiply a vector &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^n$&lt;/span&gt; by a matrix &lt;span class=&quot;math-container&quot;&gt;$A \in \mathbb{R}^{n \times m}$&lt;/span&gt; &lt;em&gt;from the left&lt;/em&gt;, that is, you can do &lt;span class=&quot;math-container&quot;&gt;$v^T A$&lt;/span&gt;. Here, I considered &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; has a column vector (that is, &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^{n \times 1}$&lt;/span&gt;), even though I have not explicitly stated it: &lt;em&gt;you can and you often need to deduce this from the context&lt;/em&gt;! If I transpose &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, I obtain &lt;span class=&quot;math-container&quot;&gt;$v^T \in \mathbb{R}^{1 \times n}$&lt;/span&gt;, and, thus, you can see that you can indeed perform the operation &lt;span class=&quot;math-container&quot;&gt;$v^T A = u \in \mathbb{R}^{1 \times m}$&lt;/span&gt;. Note that, at this point, &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, the vector resulting from the operation &lt;span class=&quot;math-container&quot;&gt;$v^T A$&lt;/span&gt;, is actually considered a matrix, but you can still use it as a vector, if you need and that is permitted according to the mathematical rules of the operations you need to perform. Note also that I cannot multiply &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; &lt;em&gt;from the right&lt;/em&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, because there is no way of making the dimensions match. Have a look at &lt;a href=&quot;https://math.stackexchange.com/q/1615793/168764&quot;&gt;this question&lt;/a&gt;, if you do not know how to multiply a vector by a matrix from the left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, you can multiply &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^n$&lt;/span&gt; by the matrix &lt;span class=&quot;math-container&quot;&gt;$B \in \mathbb{R}^{m \times n}$&lt;/span&gt; only from the right. If &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is a column vector (that is, &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^{n \times 1}$&lt;/span&gt;), you need to do &lt;span class=&quot;math-container&quot;&gt;$B v \in \mathbb{R}^{m \times 1}$&lt;/span&gt;, but, if &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is a row vector (that is, &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^{1 \times n}$&lt;/span&gt;), you will first need to transpose it, so that you can perform the operation: &lt;span class=&quot;math-container&quot;&gt;$B v^T\in \mathbb{R}^{m \times 1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, note that, if you multiply a vector by a matrix from the left, that vector needs to be a &quot;row vector&quot;, so, if you initially assume that the vector is a column vector (or that is explicitly stated), you will need to transpose it first, before the multiplication. However, if the vector is already a row vector, you won't have to transpose it. Similarly, if you multiply a vector by a matrix from the right, you will need a column vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, you can multiply a vector either from the left or right of a matrix, but you need to make sure that the dimensions match: if you multiply from the left, you will need to check that the dimensions of the vector match the number of the rows of the matrix; if you multiply the vector from the right of the matrix, you will need to check that the dimensions of the vector match the number of columns of the matrix.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-09T14:23:42.413" LastActivityDate="2019-02-09T14:23:42.413" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="10474" PostTypeId="1" CreationDate="2019-02-09T14:48:50.033" Score="6" ViewCount="2466" Body="&lt;p&gt;In the context of RL, there is the notion of &lt;strong&gt;on-policy&lt;/strong&gt; and &lt;strong&gt;off-policy&lt;/strong&gt; algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of &lt;strong&gt;online&lt;/strong&gt; and &lt;strong&gt;offline&lt;/strong&gt; learning.&lt;/p&gt;&#xA;&lt;p&gt;What is the relation (including the differences) between online learning and on-policy algorithms? Similarly, what is the relation between offline learning and off-policy algorithms?&lt;/p&gt;&#xA;&lt;p&gt;Finally, is there any relation between online (or offline) learning and off-policy (or on-policy) algorithms? For example, can an on-policy algorithm perform offline learning? If yes, can you explain why?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-16T15:27:52.287" LastActivityDate="2021-01-16T15:27:52.287" Title="What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;on-policy-methods&gt;&lt;online-learning&gt;&lt;batch-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10476" PostTypeId="1" AcceptedAnswerId="10477" CreationDate="2019-02-09T15:33:02.100" Score="3" ViewCount="188" Body="&lt;p&gt;The update rules for Q-learning and SARSA each are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q Learning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_t,a_t)←Q(s_t,a_t)+α[r_{t+1}+γ\max_{a'}Q(s_{t+1},a')−Q(s_t,a_t)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SARSA:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_t,a_t)←Q(s_t,a_t)+α[r_{t+1}+γQ(s_{t+1},a_{t+1})−Q(s_t,a_t)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the theory that SARSA performs 'on-policy' updates, and Q-learning performs 'off-policy' updates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I perform Q-learning by calculating the target thusly: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;target = reward + self.y * np.max(self.action_model.predict(state_prime))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here you can see I pick the maximum for the Q-function for state prime (i.e. greedy selection as defined by maxQ in the update rule). If I were to do a SARSA update and use the same on-policy as used when selecting an action, e.g. ϵ-greedy, would I basically change to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if np.random.random() &amp;lt; self.eps:&#xA;    target = reward + self.y * self.action_model.predict(state_prime)[random.randint(0,9)]&#xA;else:&#xA;    target = reward + self.y * np.max(self.action_model.predict(state_prime))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So sometimes it will pick a random future reward based on my epsilon greedy policy?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-02-09T18:55:45.803" Title="How do updates in SARSA and Q-learning differ in code?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10477" PostTypeId="2" ParentId="10476" CreationDate="2019-02-09T18:28:26.480" Score="4" Body="&lt;p&gt;Picking actions and making updates should be treated as separate things. For Q-learning you also need to explore by using some exploration strategy (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Steps for Q-learning:&lt;br&gt;&#xA;1) initialize state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;br&gt;&#xA;For every step of the episode:&lt;br&gt;&#xA;2) choose action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; by some exploratory policy (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy) from state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;br&gt;&#xA;3) take action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and observe &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt;&lt;br&gt;&#xA;4) do the update &lt;span class=&quot;math-container&quot;&gt;$Q(S, A) = Q(S, A) + \alpha(R + \gamma*\max_aQ(S', a) - Q(S, A))$&lt;/span&gt;&lt;br&gt;&#xA;5) update the state &lt;span class=&quot;math-container&quot;&gt;$S = S'$&lt;/span&gt; and keep looping from step 2 until the end of episode&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Steps for Sarsa:&lt;br&gt;&#xA;1) initialize state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;br&gt;&#xA;2) initialize first action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; by some exploratory policy (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy)&lt;br&gt;&#xA;For every step of the episode:&lt;br&gt;&#xA;3) take action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and observe &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt;&lt;br&gt;&#xA;4) choose action &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; by some exploratory policy (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy)&lt;br&gt;&#xA;5) do the update &lt;span class=&quot;math-container&quot;&gt;$Q(S, A) = Q(S, A) + \alpha(R + \gamma * Q(S', A') - Q(S, A))$&lt;/span&gt;&lt;br&gt;&#xA;6) update state and action &lt;span class=&quot;math-container&quot;&gt;$S = S'$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A = A'$&lt;/span&gt; and keep looping from step 3 until end of the episode&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-02-09T18:55:45.803" LastActivityDate="2019-02-09T18:55:45.803" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10478" PostTypeId="2" ParentId="9667" CreationDate="2019-02-09T20:56:31.710" Score="1" Body="&lt;p&gt;Yes, if you follow the original implementation the children will inherit the topology from the most fit parent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that the goal is to obtain a good population, maintaining the genetic diversity high but at the same time selecting the best individuals from the population; so, in theory you are allowed to give the topology you prefer to the children.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here there is an example of an alternative topology inheritance in which a child gets the genes that lead to an excess node while another child gets only the genes that create a new connection. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9JUlF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9JUlF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2019-02-09T20:56:31.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10479" PostTypeId="1" AcceptedAnswerId="10480" CreationDate="2019-02-09T23:56:24.377" Score="2" ViewCount="92" Body="&lt;p&gt;I'm using Q-learning to train an agent to play a board game (e.g. chess, draughts or go). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent takes an action while in state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, but then what is the next state (that is,  &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt;)? Is &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; now the board with the piece moved as a result of taking the action, or is &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; the state the agent encounters after the other player has performed his action (i.e. it's this agent's turn again)?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="2444" LastEditDate="2019-02-10T21:18:12.610" LastActivityDate="2019-02-10T21:18:12.610" Title="What is the next state for a two-player board game?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10480" PostTypeId="2" ParentId="10479" CreationDate="2019-02-10T02:06:54.060" Score="2" Body="&lt;p&gt;If your opponent has fixed knowledge (it doesn't learn), then the next state after your agent did an action is the state when your turn is back. So the actions of other players are considered as an environment reaction to your actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if your opponent can learn, you may create a &lt;a href=&quot;https://arxiv.org/pdf/1711.00832.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-agent Reinforcement Learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2019-02-10T09:56:31.720" LastActivityDate="2019-02-10T09:56:31.720" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10481" PostTypeId="2" ParentId="10306" CreationDate="2019-02-10T03:56:15.743" Score="3" Body="&lt;p&gt;Although what @Jaden said may be true by itself, it does not really serve to answer my question as I have seen after conducting numerous experiments, and finally reaching close to Dueling Network performance using a normal Double DQN (DDQN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made the following changes to my code after closely examining the OpenAI baselines code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Used &lt;code&gt;PongFrameskip-v4&lt;/code&gt; instead of &lt;code&gt;PongDeterministic-v4&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Used a small replay buffer of size &lt;strong&gt;10000&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;During a &lt;code&gt;step_update()&lt;/code&gt; or &lt;code&gt;replay()&lt;/code&gt; call, changed the condition for a  &lt;code&gt;return&lt;/code&gt; from &lt;code&gt;buffer_fill_size &amp;lt; learn_start&lt;/code&gt; to &lt;code&gt;t &amp;lt; learn_start&lt;/code&gt;, where &lt;code&gt;t&lt;/code&gt; is the current timestep, &lt;code&gt;buffer_fill_size&lt;/code&gt; is the current size of buffer that has been filled up with experience tuples, and &lt;code&gt;learn_start&lt;/code&gt; is the number of timesteps to wait before starting to learn from the experience collected.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Made sure that the &lt;code&gt;make_atari()&lt;/code&gt; wrapper function is also called on the &lt;code&gt;env&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENV_GYM = 'PongFrameskip-v4'&#xA;env = make_atari(ENV_GYM)&#xA;env = wrap_deepmind(env, frame_stack=True, scale=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These wrappers may be implemented from scratch or can be obtained from the &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI baseline Atari wrappers&lt;/a&gt;. I personally used the latter since there is no point in reinventing the wheel.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biggest step that I overlooked, or rather didn't pay much attention to was the input preprocessing. These few changes improved my DDQN from an average score saturation at -13 in almost 5000 episodes to +18 in about 700-800 episodes. That is indeed a huge difference. You can check out my implementation &lt;a href=&quot;https://github.com/hridayns/Reinforcement-Learning/tree/master/Atari&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21513" LastActivityDate="2019-02-10T03:56:15.743" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10482" PostTypeId="2" ParentId="10203" CreationDate="2019-02-10T04:04:36.467" Score="1" Body="&lt;p&gt;Incase you still haven't been able to resolve the problem, here's a link to the &lt;a href=&quot;https://ai.stackexchange.com/a/10481/21513&quot;&gt;answer&lt;/a&gt; to my own question, which has the step-wise changes I made to achieve a +18 average score saturation using just a 10000 replay buffer size and a normal Double DQN (DDQN), trained for about 700-800 episodes. The updated code can also be found &lt;a href=&quot;https://github.com/hridayns/Reinforcement-Learning/tree/master/Atari&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No fancy changes like Prioritized Replay Buffer or any secret hyperparameter changes are required. It's usually something simple, like a small problem with the input preprocessing step.&lt;/p&gt;&#xA;" OwnerUserId="21513" LastActivityDate="2019-02-10T04:04:36.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10484" PostTypeId="1" CreationDate="2019-02-10T10:22:22.583" Score="1" ViewCount="97" Body="&lt;p&gt;I am playing with a deep Q-learning algorithm in my own environment. The network can perform well as long as there is only one enemy. My agent can perform the following actions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;do_nothing&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;prepare_for(e)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;attack(e)&lt;/code&gt; &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;e&lt;/code&gt; is some enemy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of two enemies, the action vector has 5 elements:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|   0       |      1          |      2      |        3         |     4      |&#xA;-----------------------------------------------------------------------------&#xA;|do_nothing | prepare_for(e1) |  attack(e1) |  prepare_for(e2) | attack(e2) |&#xA;-----------------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After a couple of episodes, the agent always starts picking the first &lt;code&gt;do_nothing&lt;/code&gt; action, which is not desired. Changing reward for &lt;code&gt;do_nothing&lt;/code&gt; action is not helping, even using significantly higher negative reward, than for other actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no problem with the environment with only one enemy. (Only using columns 0, 1, 2). I feel like my action encoding can be the issue, but I can't figure it out, how to fix it. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="22162" LastEditorUserId="2444" LastEditDate="2019-02-10T21:18:55.280" LastActivityDate="2019-02-10T21:18:55.280" Title="Deep Q-learning is not performing well when there are several enemies" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10485" PostTypeId="2" ParentId="10484" CreationDate="2019-02-10T10:30:27.690" Score="1" Body="&lt;p&gt;1) Vary the number of enemies during the training process.&#xA;2) Use other generalisation methods that are appropriate for neural networks such as dropout or L1 and L2 regularisation. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-02-10T10:30:27.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10489" PostTypeId="2" ParentId="10471" CreationDate="2019-02-10T13:10:40.177" Score="2" Body="&lt;p&gt;I will first recapitulate the key concepts which you need to know in order to understand the answer to your question (which will be very simple, because I will just try to clarify what is given as a &amp;quot;definition&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;In logic, a &lt;strong&gt;formula&lt;/strong&gt; is e.g. &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\lnot f$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f \land g$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; can be e.g. the &lt;strong&gt;proposition&lt;/strong&gt; (or &lt;em&gt;variable&lt;/em&gt;) &amp;quot;today it will rain&amp;quot;.  So, in a (propositional) formula, you have &lt;em&gt;propositions&lt;/em&gt;, i.e. sentences like &amp;quot;today it will rain&amp;quot;, and logical &lt;strong&gt;connectives&lt;/strong&gt;, i.e. symbols like &lt;span class=&quot;math-container&quot;&gt;$\land$&lt;/span&gt; (i.e. logical AND), which logically &lt;em&gt;connect&lt;/em&gt; these sentences. The propositions like &amp;quot;today it will rain&amp;quot; can often be denoted by a single (capital) letter like &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$f \land g$&lt;/span&gt; is the combination of two formulae (where &lt;em&gt;formulae&lt;/em&gt; is the plural of &lt;em&gt;formula&lt;/em&gt;). So, for example, suppose that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is composed of the propositions &amp;quot;today it will rain&amp;quot; (denoted by &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;) &lt;em&gt;or&lt;/em&gt; &amp;quot;my friend will visit me&amp;quot; (denoted by &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is defined as &amp;quot;I will play with my friend&amp;quot; (denoted by &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;). Then the formula &lt;span class=&quot;math-container&quot;&gt;$f \land g = (P \lor Q) \land S$&lt;/span&gt;. In general, you can combine formulae in any logically appropriate way.&lt;/p&gt;&#xA;&lt;p&gt;In this context, a &lt;strong&gt;model&lt;/strong&gt; is an &lt;em&gt;assignment&lt;/em&gt; to each variable in a formula. For example, suppose &lt;span class=&quot;math-container&quot;&gt;$f = P \lor Q$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$w = \{ P=0, Q = 1\}$&lt;/span&gt; is a &lt;em&gt;model&lt;/em&gt; for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, that is, each variable (e.g. &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;) is assigned either &amp;quot;true&amp;quot; (&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) or &amp;quot;false&amp;quot; (&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) but &lt;em&gt;not&lt;/em&gt; both. (Note that &lt;a href=&quot;https://ai.stackexchange.com/a/25460/2444&quot;&gt;the word &lt;em&gt;model&lt;/em&gt; may be used to refer to different concepts depending on the context&lt;/a&gt;; again, in this context, you can simply think of a model as &lt;em&gt;an assignment of values to the variables in a formula&lt;/em&gt;.)&lt;/p&gt;&#xA;&lt;p&gt;Suppose now we define &lt;span class=&quot;math-container&quot;&gt;$I(f, w)$&lt;/span&gt; to be a &lt;em&gt;function that receives the formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and the model &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; as input, and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; returns either &amp;quot;true&amp;quot; (&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) or &amp;quot;false&amp;quot; (&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;)&lt;/em&gt;. In other words, &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is a function that &lt;em&gt;automatically&lt;/em&gt; tells us if &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is evaluated to true or false given the assignment &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can now define &lt;span class=&quot;math-container&quot;&gt;$M(f)$&lt;/span&gt; to be a &lt;em&gt;set of assignments (or models) to the formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is true&lt;/em&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is a set and not just an assignment (or model). This set can be empty, it can contain one assignment or it can contain any number of assignments: it depends on the formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;: in some cases, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is empty and, in other cases, it may contain say &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; valid assignments to &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, where by &amp;quot;valid&amp;quot; I mean that these assignments make &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; evaluate to &amp;quot;true&amp;quot;. For example, suppose we have formula &lt;span class=&quot;math-container&quot;&gt;$f = A \land \lnot A$&lt;/span&gt;. Then you can try to assign any value to &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; will never evaluate to true. In that case, &lt;span class=&quot;math-container&quot;&gt;$M(f)$&lt;/span&gt; is an empty set, because there is no assignment to the variables (or propositions) of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; which make &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; evaluate to true.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;knowledge base&lt;/strong&gt; is a set of formulae &lt;span class=&quot;math-container&quot;&gt;$\text{KB} = \{ f_1, f_2, \dots, f_n \}$&lt;/span&gt;. So, for example, &lt;span class=&quot;math-container&quot;&gt;$f_2 = $&lt;/span&gt; &amp;quot;today it will rain&amp;quot; and &lt;span class=&quot;math-container&quot;&gt;$f_3 = $&lt;/span&gt; &amp;quot;I will go to school AND I will have lunch&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;We can now define &lt;span class=&quot;math-container&quot;&gt;$M(\text{KB})$&lt;/span&gt; to be the &lt;em&gt;set of assignments to the formulae in the knowledge base &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; such that all formulae are true&lt;/em&gt;. If you think of the formulae in &lt;span class=&quot;math-container&quot;&gt;$KB$&lt;/span&gt; as &amp;quot;facts&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$M(\text{KB})$&lt;/span&gt; is an assignment to these formulae in &lt;span class=&quot;math-container&quot;&gt;$KB$&lt;/span&gt; such that these facts hold or are true.&lt;/p&gt;&#xA;&lt;p&gt;In this context, we then say that a particular knowledge base (i.e., a set of formulae as defined above), denoted by &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt;, is &lt;em&gt;consistent with formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$M(\text{KB} \cup \{ f \})$&lt;/span&gt; is a non-empty set&lt;/em&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\cup$&lt;/span&gt; means the &lt;em&gt;union&lt;/em&gt; operation between sets: note that (as we defined it above) &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; is a set, and &lt;span class=&quot;math-container&quot;&gt;$\{ f \}$&lt;/span&gt; means that we are making a set out of the formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, so we are indeed performing an &lt;em&gt;union&lt;/em&gt; operation on sets.&lt;/p&gt;&#xA;&lt;p&gt;So, what does it mean for a knowledge base to be consistent? First of all, the consistency of a knowledge base &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; is defined &lt;em&gt;with respect to&lt;/em&gt; another formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. Recall that a knowledge base is a set of formulae, so we are defining the consistency of a set of formulae with respect to another formula.&lt;/p&gt;&#xA;&lt;p&gt;When is then a knowledge base &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; consistent with a formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;? When &lt;span class=&quot;math-container&quot;&gt;$M(\text{KB} \cup \{ f \})$&lt;/span&gt; is a non-empty set. Recall that &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is an assignment to the variables in its input such that its inputs evaluate to true. So, &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; is consistent with &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; when there is a set of assignments of values to the formulae in &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; and an assignment of values to the variables in &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; such that &lt;em&gt;both&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; are true. In other words, &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; is consistent with &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; when both all formulae in &lt;span class=&quot;math-container&quot;&gt;$\text{KB}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; can be true &lt;em&gt;at the same time&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T18:44:09.463" LastActivityDate="2021-01-21T18:44:09.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10490" PostTypeId="2" ParentId="10471" CreationDate="2019-02-10T13:11:34.750" Score="0" Body="&lt;p&gt;Here is a (very) brief &lt;a href=&quot;https://en.wikipedia.org/wiki/Consistency_(knowledge_bases)&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia article on consistency in KB's&lt;/a&gt;, which should answer your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A KB is consistent, if it does not contain any contradictions, ie &lt;span class=&quot;math-container&quot;&gt;$\lnot a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; are not both derivable from it. Which is pretty much common sense if you think about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have a formula &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, for example &quot;A is a trout &lt;span class=&quot;math-container&quot;&gt;$\land$&lt;/span&gt; A lays eggs&quot;, and my KB contains &quot;fish lay eggs&quot; and &quot;a trout is a fish&quot;, then, if &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is true, ie trout do lay eggs, that formula is consistent with my KB, which states that trout are fish and that fish lay eggs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: for a more formalised version, see nbro's answer.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-02-10T13:11:34.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10491" PostTypeId="2" ParentId="10474" CreationDate="2019-02-10T14:49:11.147" Score="11" Body="&lt;p&gt;The concepts of on-policy vs off-policy and online vs offline are separate, but do interact to make certain combinations more feasible. When looking at this, it is worth also considering the difference between &lt;em&gt;prediction&lt;/em&gt; and &lt;em&gt;control&lt;/em&gt; in Reinforcement Learning (RL).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Online vs Offline&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;These concepts are not specific to RL, many learning systems can be categorised as online or offline (or somewhere in-between).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Online&lt;/strong&gt; learning algorithms work with data as it is made available. Strictly online algorithms improve incrementally from each piece of new data as it arrives, then discard that data and do not use it again. It is not a requirement, but it is commonly desirable for an online algorithm to forget older examples over time, so that it can adapt to non-stationary populations. Stochastic gradient descent with back-propagation - as used in neural networks - is an example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Offline&lt;/strong&gt; learning algorithms work with data in bulk, from a dataset. Strictly offline learning algorithms need to be re-run from scratch in order to learn from changed data. Support vector machines and random forests are strictly offline algorithms (although researchers have constructed online variants of them).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of the two types, &lt;em&gt;online&lt;/em&gt; algorithms are more general in that you can easily construct an offline algorithm from a strictly online one plus a stored dataset, but the opposite is not true for a strictly offline algorithm. However, this does not necessarily make them superior - often compromises are made in terms of sample efficiency, CPU cost or accuracy when using an online algorithm. Approaches such as mini-batches in neural network training can be viewed as attempts to find a middle ground between online and offline algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience replay, a common RL technique, used in Deep Q Networks amongst others, is another in-between approach. Although you &lt;em&gt;could&lt;/em&gt; store all the experience necessary to fully train an agent in theory, typically you store a rolling history and sample from it. It's possible to argue semantics about this, but I view the approach as being a kind of &quot;buffered online&quot;, as it requires low-level components that can work online (e.g. neural networks for DQN).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;On-policy vs Off-Policy&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;These are more specific to control systems and RL. Despite the similarities in name between these concepts and online/offline, they refer to a different part of the problem.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;On-policy&lt;/strong&gt; algorithms work with a single policy, often symbolised as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, and require any observations (state, action, reward, next state) to have been generated using that policy. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Off-policy&lt;/strong&gt; algorithms work with two policies (sometimes effectively more, though never more than two per step). These are a policy being learned, called the target policy (usually shown as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;), and the policy being followed that generates the observations, called the behaviour policy (called various things in the literature - &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;, Sutton and Barto call it &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; in the latest edition).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A very common scenario for off-policy learning is to learn about best guess at optimal policy from an exploring policy, but that is not the definition of off-policy.&lt;/li&gt;&#xA;&lt;li&gt;The primary difference between observations generated by &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and the target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is which actions are selected on each time step. There is also a secondary difference which can be important: The population distribution of both states and actions in the observations can be different between &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; - this can have an impact for function approximation, as cost functions (for e.g. NNs) are usually optimised over a population of data.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In both cases, there is no requirement for the observations to be processed strictly online or offline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to the relationship between online and offline learning, off-policy is always a strict generalisation of on-policy. You can make any off-policy algorithm into an equivalent on-policy one by setting &lt;span class=&quot;math-container&quot;&gt;$\pi = b$&lt;/span&gt;. There is a sense in which you can do this by degrees, by making &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; closer to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (for instance, reducing &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; in an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy behaviour policy for &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the fully greedy policy). This can be desirable, as off-policy agents do still need to observe states and actions that occur under the target policy - if that happens rarely because of differences between &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, then learning about the target policy will happen slowly.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Prediction vs Control&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This can get forgotten due to the focus on search for optimal policies.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;prediction&lt;/strong&gt; problem in RL is to estimate the value of a particular state or state/action pair, given an environment and a policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The (optimal) &lt;strong&gt;control&lt;/strong&gt; problem in RL is to find the best policy given an environment. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Solving the control problem when using value-based methods involves both estimating the value of being in a certain state (i.e. solving the prediction problem), and adjusting the policy to make higher value choices based on those estimates. This is called &lt;a href=&quot;http://www.incompleteideas.net/book/ebook/node46.html&quot; rel=&quot;noreferrer&quot;&gt;generalised policy iteration&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main thing to note here is that the prediction problem is stationary (all long-term expected distributions are the same over time), whilst the control problem adds non-stationary target for the prediction component (the policy changes, so does the expected return, distribution of states etc) &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Combinations That Work&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Note this is not about choice of algorithms. The strongest driver for algorithm choice is on-policy (e.g. SARSA) vs off-policy (e.g. Q-learning). The same core learning algorithms can often be used online or offline, for prediction or for control.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Online, on-policy prediction. A learning agent is set the task of evaluating certain states (or state/action pairs), and learns from observation data as it arrives. It should always act the same way (it may be observing some other control system with a fixed, and maybe unknown policy).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Online, off-policy prediction. A learning agent is set the task of evaluating certain states (or state/action pairs) from the perspective of an arbitrary fixed target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (which must be defined to the agent), and learns from observation data as it arrives. The observations can be from any behaviour policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; - depending on the algorithm being used, it may be necessary to have &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; defined as well as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Offline, on-policy prediction. A learning agent is set the task of evaluating certain states (or state/action pairs), and is given a dataset of observations from the environment of an agent acting using some fixed policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Offline, off-policy prediction. A learning agent is set the task of evaluating certain states (or state/action pairs) from the perspective of an arbitrary fixed target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (which must be defined to the agent), and is given a dataset of observations from the environment of an agent acting using some other policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Online, on-policy control. A learning agent is set the task of behaving optimally in an environment, and learns from each observations as it arrives. It will adapt its own policy as it learns, making this a non-stationary problem, and also importantly &lt;em&gt;making its own history of observations off-policy&lt;/em&gt; data. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Online, off-policy control. A learning agent is set the task of behaving optimally in an environment. It may behave and gain observations from a behaviour policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, but learns a separate optimal target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. It is common to link &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; - e.g. for &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to be deterministic greedy policy with respect to estimated action values, and for &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; to be &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy with respect to the same action values.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Offline, on-policy control. This not really possible in general, as an on-policy agent needs to be able to observe data about its current policy. As soon as it has learned a policy different to that in the stored dataset, then all the data becomes off-policy to it, and the agent has no valid source data. In some cases you might still be able to get something to work.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Offline, off-policy control. A learning agent is set the task of learning an optimal policy from a store dataset of observations. The observations can be from any behaviour policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; - depending on the algorithm being used, it may be necessary to have &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; defined as well as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As you can see above, only one combination &lt;em&gt;offline, on-policy control&lt;/em&gt;, causes a clash.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a strong skew towards online learning in RL. Approaches that buffer some data, such as Monte Carlo control, experience replay or Dyna-Q do mix-in some of the traits of offline learning, but still require a constant supply of new observations plus forget older ones. Control algorithms imply non-stationary data, and these require online forgetting behaviour from the estimators - another online learning trait.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, mixing in a little &quot;offline&quot; data in experience replay can cause some complications for on-policy control algorithms. The experience buffer can contain things that are technically off-policy to the latest iteration of the agent. How much of a problem this is in practice will vary.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-10T17:27:19.910" LastActivityDate="2019-02-10T17:27:19.910" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10492" PostTypeId="1" AcceptedAnswerId="10495" CreationDate="2019-02-10T16:34:44.907" Score="4" ViewCount="166" Body="&lt;p&gt;In the context of reinforcement learning, a policy, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, is often defined as a function from the space of states, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt;, to the space of actions, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$\pi : \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt;. This function is the &quot;solution&quot; to a problem, which is represented as a Markov decision process (MDP), so we often say that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a solution to the MDP. In general, we want to find the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt; for each MDP &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;, that is, for each MDP &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;, we want to find the policy which would make the agent behave optimality (that is, obtain the highest &quot;cumulative future discounted reward&quot;, or, in short, the highest &quot;return&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is often the case that, in RL algorithms, e.g. Q-learning, people often mention &quot;policies&quot; like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, greedy, soft-max, etc., without ever mentioning that these policies are or not solutions to some MDP. It seems to me that these are two different types of policies: for example, the &quot;greedy policy&quot; always chooses the action with the highest expected return, no matter which state we are in; similarly, for the &quot;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy&quot;; on the other hand, a policy which is a solution to a MDP is a map between states and actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is then the relation between a policy which is the solution to a MDP and a policy like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy? Is a policy like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy a solution to any MDP? How can we formalise a policy like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy in a similar way that I formalised a policy which is the solution to a MDP?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that &quot;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy&quot; can be called a policy, because, in fact, in algorithms like Q-learning, they are used to select actions (i.e. they allow the agent to behave), and this is the fundamental definition of a policy.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-02-11T08:05:10.490" Title="What is the relation between a policy which is the solution to a MDP and a policy like $\epsilon$-greedy?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;markov-decision-process&gt;&lt;policies&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10493" PostTypeId="1" CreationDate="2019-02-10T16:46:50.767" Score="2" ViewCount="56" Body="&lt;p&gt;Can traditional neural networks be combined with spiking neural networks? And can there be training algorithms for such hybrid network? Does such hybrid network model biological brains? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand, brains contain only spiking networks and traditional networks are more or less crude approximation of them. But we can imagine that evolutionary computing can surpass the biological evolution and so the new structure can be created that are better than mind. And that is why the question about such tradition-spiking hybrid neural networks should be interesting.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-02-10T19:20:42.250" LastActivityDate="2019-02-10T19:20:42.250" Title="Can traditional neural networks be combined with spiking neural networks?" Tags="&lt;neural-networks&gt;&lt;spiking-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10495" PostTypeId="2" ParentId="10492" CreationDate="2019-02-10T17:58:44.270" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;for example, the &quot;greedy policy&quot; always chooses the action with the highest expected return, no matter which state we are in&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &quot;no matter which state we are in&quot; there is generally not true; in general, the expected return depends on the state we are in &lt;strong&gt;and&lt;/strong&gt; the action we choose, not just the action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, I wouldn't say that a policy is a mapping from states to actions, but a mapping from states to &lt;strong&gt;probability distributions over actions&lt;/strong&gt;. That would only be equivalent to a mapping from states to actions for deterministic policies, not for stochastic policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that our agent has access to (estimates of) value functions &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for state-action pairs, the greedy and &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policies can be described in precisely the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\pi_g (s, a)$&lt;/span&gt; denote the probability assigned to an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; by the greedy policy. For simplicity, I'll assume there are no ties (otherwise it would in practice be best to randomize uniformly across the actions leading to the highest values). This probability is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi_g (s, a) = &#xA;\begin{cases}&#xA;1, &amp;amp; \text{if } a = \arg\max_{a'} Q(s, a') \\&#xA;0, &amp;amp; \text{otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\epsilon} (s, a)$&lt;/span&gt; could denote the probability assigned by an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy strategy, with probabilities given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi_{\epsilon} (s, a) = &#xA;\begin{cases}&#xA;(1 - \epsilon) + \frac{\epsilon}{\vert \mathcal{A}(s) \vert}, &amp;amp; \text{if } a = \arg\max_{a'} Q(s, a') \\&#xA;\frac{\epsilon}{\vert \mathcal{A}(s) \vert}, &amp;amp; \text{otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\vert \mathcal{A}(s) \vert$&lt;/span&gt; denotes the size of the set of legal actions in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-02-11T08:05:10.490" LastActivityDate="2019-02-11T08:05:10.490" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10496" PostTypeId="1" CreationDate="2019-02-10T18:39:19.517" Score="1" ViewCount="21" Body="&lt;p&gt;I have continual simulated data of million sentences of two simulated persons talking to each other in a room and I want to model one of the persons speech. Now, during this period things in the room can change. Let's say, one of them says &quot;Where is the book?&quot; The other one responds &quot;I placed the book on the bookshelf&quot;. Now during time, the position of the book changes, so the question Where is the book? does not have stationary answer i.e the answer changes during time. However, in general the answer has to be &quot;The book is at some_location&quot; and not something else. Also, the mentioning that the book is placed on the bookshelf can be sometimes 10, 100 or 1000 sentences before the question &quot;Where is the book?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you approach this kind of problem? Since the window can be too large I can not split data into training samples of 10, 100 or 1000 sentences. My guess is that I should use BPTT + LSTM and train in one shot without shuffling the data. I am not sure this is feasible, so I will greatly appreciate your help! I have also my doubts what if &quot;Where is the book?&quot; appears 20 sentences after (instead of 10,100 and 1000) in the test set (which is not same as the training set)? Also, should I use Reinforcement Learning (since I can generate the data) or Supervised learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="20378" LastEditorUserId="20378" LastEditDate="2019-02-10T18:46:47.927" LastActivityDate="2019-02-10T18:46:47.927" Title="How to train chat bot on infinite non-stationary data?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10498" PostTypeId="2" ParentId="10466" CreationDate="2019-02-10T22:31:52.487" Score="1" Body="&lt;p&gt;It seems I've solved the issue. There was several mistakes:&lt;br&gt;&#xA;1. I've generated random weights from 0 to 1. As a result, too big numbers passed through softmax function (&gt;10000), and the function wasn't calculated correctly. I divided each initial weight on the number of neurons in previous layer and solved the issue.&lt;br&gt;&#xA;2. I've calculated separate delta for biases while delta must be the same for main weights and biases.&lt;br&gt;&#xA;If anyone is interested, here is the correct code (83% and 89% precision after first and second launch):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def ReLU(x):&#xA;    return np.maximum(0, x)&#xA;&#xA;def ReLU_derivative(x):&#xA;    return np.greater(x, 0).astype(int)&#xA;&#xA;def softmax(x):&#xA;    shift = x - np.max(x)&#xA;    return np.exp(shift) / np.sum(np.exp(shift))&#xA;&#xA;def softmax_derivative(x):&#xA;    sm_array = softmax(x)&#xA;    J = np.zeros((x.size, x.size))&#xA;    for i in range(x.size):&#xA;        for j in range(x.size):&#xA;            delta = np.equal(i, j).astype(int)&#xA;            J[j, i] = sm_array[0][i] * (delta - sm_array[0][j])&#xA;    return J&#xA;&#xA;class Layer:&#xA;    def __init__(self, size, prev_layer=None):&#xA;        self.size = size&#xA;        self.prevLayer = prev_layer&#xA;        self.nextLayer = None&#xA;        self.data = None&#xA;        self.data_in = None&#xA;        if prev_layer is not None:&#xA;            self.prevLayer.nextLayer = self&#xA;            self.W = np.random.random((self.prevLayer.size, size)) / (self.prevLayer.size + 1)&#xA;            self.W_bias = np.random.random((1, size)) / (self.prevLayer.size + 1)&#xA;        else:&#xA;            self.W = None&#xA;            self.W_bias = None&#xA;&#xA;    def forward(self):&#xA;        if self.prevLayer is not None:&#xA;            self.data_in = np.dot(self.prevLayer.data, self.W)&#xA;            self.data_in += np.dot([[1]], self.W_bias)&#xA;            if self.nextLayer is not None:&#xA;                self.data = ReLU(self.data_in)&#xA;                self.nextLayer.forward()&#xA;            else:&#xA;                self.data = softmax(self.data_in)&#xA;        else:&#xA;            self.nextLayer.forward()&#xA;&#xA;    def backprop(self, expected_output=None, prev_delta=None):&#xA;        if prev_delta is None:&#xA;            delta = np.dot(-(expected_output - self.data), softmax_derivative(self.data_in))&#xA;        else:&#xA;            delta = np.dot(prev_delta, self.nextLayer.W.T) * ReLU_derivative(self.data_in)&#xA;        training_velocity = 0.1&#xA;        W_dif = np.dot(self.prevLayer.data.T, delta) * training_velocity&#xA;        W_bias_dif = np.dot([[1]], delta) * training_velocity&#xA;        if self.prevLayer.prevLayer is not None:&#xA;            self.prevLayer.backprop(prev_delta=delta)&#xA;        self.W -= W_dif&#xA;        self.W_bias -= W_bias_dif&#xA;&#xA;f_images = open(&quot;train-images.idx3-ubyte&quot;, &quot;br&quot;)&#xA;f_images.seek(4)&#xA;f_labels = open(&quot;train-labels.idx1-ubyte&quot;, &quot;br&quot;)&#xA;f_labels.seek(8)&#xA;images_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;rows_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;cols_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;&#xA;input_layer = Layer(rows_number*cols_number)&#xA;hidden_layer1 = Layer(rows_number*cols_number*7//10, input_layer)&#xA;hidden_layer2 = Layer(rows_number*cols_number*7//10, hidden_layer1)&#xA;output_layer = Layer(10, hidden_layer2)&#xA;digits = np.array([np.zeros(10)])&#xA;&#xA;print(&quot;Training:&quot;)&#xA;input_image = np.array([np.zeros(rows_number * cols_number)])&#xA;for k in range(images_number):&#xA;    for i in range(rows_number):&#xA;        for j in range(cols_number):&#xA;            input_image[0][i*cols_number+j] = int.from_bytes(f_images.read(1), byteorder='big') / 255.0&#xA;    input_layer.data = input_image&#xA;    input_layer.forward()&#xA;    current_digit = int.from_bytes(f_labels.read(1), byteorder='big')&#xA;    digits[0][current_digit] = 1&#xA;    output_layer.backprop(expected_output=digits)&#xA;    digits[0][current_digit] = 0&#xA;    if((k+1) % 1000 == 0):&#xA;        print(str(k+1) + &quot; / &quot; + str(images_number))&#xA;f_images.close()&#xA;f_labels.close()&#xA;&#xA;f_images = open(&quot;t10k-images.idx3-ubyte&quot;, &quot;br&quot;)&#xA;f_images.seek(4)&#xA;f_labels = open(&quot;t10k-labels.idx1-ubyte&quot;, &quot;br&quot;)&#xA;f_labels.seek(8)&#xA;images_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;rows_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;cols_number = int.from_bytes(f_images.read(4), byteorder='big')&#xA;&#xA;print(&quot;\r\nTesting:&quot;)&#xA;correct = 0&#xA;for k in range(images_number):&#xA;    for i in range(rows_number):&#xA;        for j in range(cols_number):&#xA;            input_image[0][i*cols_number+j] = int.from_bytes(f_images.read(1), byteorder='big')&#xA;    input_layer.data = input_image&#xA;    input_layer.forward()&#xA;    current_digit = int.from_bytes(f_labels.read(1), byteorder='big')&#xA;    if np.argmax(output_layer.data[0]) == current_digit:&#xA;        correct += 1&#xA;    if((k+1) % 1000 == 0):&#xA;        print(str(k+1) + &quot; / &quot; + str(images_number))&#xA;&#xA;print(&quot;\r\nCorrect: &quot; + str(correct) + &quot; / &quot; + str(images_number))&#xA;f_images.close()&#xA;f_labels.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21567" LastActivityDate="2019-02-10T22:31:52.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10499" PostTypeId="2" ParentId="4456" CreationDate="2019-02-11T01:34:48.157" Score="2" Body="&lt;p&gt;According to &lt;em&gt;&lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI – Kinds of RL Algorithms&lt;/a&gt;&lt;/em&gt;, algorithms which use a model of the environment, i.e. a function which predicts state transitions and rewards, are called &lt;strong&gt;model-based&lt;/strong&gt; methods, and those that don’t are called &lt;strong&gt;model-free&lt;/strong&gt;. This model can either have been given the agent or learned by the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a model allows the agent to plan by thinking ahead, seeing what would happen for a range of possible choices, and explicitly deciding between its options. This may be useful when faced with problems that require more long-term thinking. One way to perform planning is by using some kind of tree search, for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo tree search&lt;/a&gt; (MCTS), or—which I suspect could also be used—&lt;a href=&quot;https://en.wikipedia.org/wiki/Rapidly-exploring_random_tree#Variants_and_improvements_for_motion_planning&quot; rel=&quot;nofollow noreferrer&quot;&gt;variants&lt;/a&gt; of the rapidly exploring random tree (RRT). See e.g. &lt;em&gt;&lt;a href=&quot;https://deepmind.com/blog/agents-imagine-and-plan/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agents that imagine and plan&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent can then distill the results from planning ahead into a learned policy – this is known as expert iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A model can also be used to create a simulated, or &quot;imagined,&quot; environment in which the state is updated by using the model, and make the agent learn inside of that environment, such as in &lt;em&gt;&lt;a href=&quot;https://worldmodels.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Models&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many real-world scenarios, the ground-truth model of the environment is not available to the agent. If an agent wants to use a model in this case, it has to learn the model, which can be challenging for several reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are however cases in which the agent uses a model that is already known and consequently doesn't have to learn the model, such as in &lt;a href=&quot;https://deepmind.com/blog/alphazero-shedding-new-light-grand-games-chess-shogi-and-go/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero&lt;/a&gt;, where the model comes in form of the rules of the game.&lt;/p&gt;&#xA;" OwnerUserId="9220" LastActivityDate="2019-02-11T01:34:48.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10500" PostTypeId="1" CreationDate="2019-02-11T03:34:20.913" Score="1" ViewCount="61" Body="&lt;p&gt;In the diagram below, there are three variables: &lt;code&gt;X3&lt;/code&gt; is a function of (depends on) &lt;code&gt;X1&lt;/code&gt; and &lt;code&gt;X2&lt;/code&gt;, &lt;code&gt;X2&lt;/code&gt; also depends on &lt;code&gt;X1&lt;/code&gt;. More specifically, &lt;code&gt;X3 = f(X1, x2)&lt;/code&gt; and &lt;code&gt;X2 = g(X1)&lt;/code&gt;. Therefore, &lt;code&gt;X3 = f(X1, g(X1))&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Av2hx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Av2hx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the probabilistic distribution of &lt;code&gt;X1&lt;/code&gt; is known, is it possible to derive the the probabilistic distribution of &lt;code&gt;X3&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="22184" LastEditorUserId="2444" LastEditDate="2019-02-11T21:19:25.473" LastActivityDate="2019-07-21T03:19:29.167" Title="Can we derive the distribution of a random variable based on a dependent random variable's distribution?" Tags="&lt;machine-learning&gt;&lt;statistical-ai&gt;&lt;bayes&gt;&lt;probability-distribution&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10502" PostTypeId="2" ParentId="10413" CreationDate="2019-02-11T07:32:58.000" Score="0" Body="&lt;p&gt;Artificial General Intelligence (AGI) is machines being able to do any intellectual task that a human being can do, probably better than human beings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such machines could possibly out-compete inferior humans in every field possible may it be in jobs, arts, politics. Like &lt;strong&gt;Yuval Noah Harari&lt;/strong&gt; often mentions, this will create a new class of people called useless class, who cannot perform anything better than machines. This is definitely a very big challenge for human beings that co-exist with machines that have AGI. But, like always, it is possible that human beings will evolve and can embrace this challenge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lot of predictions on how the society will be with the arrival of AGI, and the predictions span many possibilities. Some are not ready to agree that AGI is possible. Some agree that AGI is possible, but not in near future (may not be in this century). Some predictions are very scary though saying AGI will be a very big scare to mankind. Anything is possible, and we have very limited information to predict the future at this point of time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to the question of whether AGI will decide to kill all humans? It is quite possible. It all depends on how our thought and technical leaders decide on how we want to pursue AI. That is the reason why a lot of discussions/debates are happening now a days about ethics in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many good movies/series also tried to depict how societies will look like with AGI. Some of the movies i like and able to relate with knowledge i have are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The Matrix Trilogy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;iRobot.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Person of Interest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Seeing these movies and giving deeper thought, you can find your own answer.&#xA;Also watch videos and read books by Yuval Noah Harari, for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, one thing i can surely predict is that, we are a long way before we can realize AGI. We are still making baby steps in that direction. But, fact is we have already started in the right direction. &lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-02-11T07:32:58.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10504" PostTypeId="2" ParentId="10413" CreationDate="2019-02-11T07:47:14.043" Score="1" Body="&lt;p&gt;This is a very, very controversial topic. But here's my opinion on the topic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI won't harm humanity (at least, to the level that a lot of people think it will).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Kartik mentioned in his answer, AGI (Artificial General Intelligence) is the concept of AI that's become so &quot;generally&quot; intelligent, that it can do everything we as humans can, and more, but do it better, more efficiently, more accurately, you name it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd say that AGI is (so far) a fictional scenario. Why? Because of the physical limitation of computing that wouldn't let us get there. In order to simulate natural, physical intelligence, you wouldn't be able to use gradient descent, you would need to simulate physical interactions of neurons. Simulating even a single molecule on the fastest supercomputers takes a very long time, because the overhead introduced with taking the real, physical world, transposing it onto mathematics, and running those mathematics on electronics, are too much. So simulating the complete working of just a small cluster of neurons would be almost infeasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you might ask: &quot;what if we were to get enough computing power in the future?&quot;. It's a valid question, however, there's one problem: we're already (possibly) quite close to the physical limit of how small we can get our transistors before Quantum Tunnelling kicks in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'd say it's impossible because of a few key reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Current algorithms are intrinsically guided by human-generated data (even reinforcement learning algorithms need human-engineered rewards to understand what's good and what's bad). Therefore, there's no such thing as a self-guided objective for machine learning today.&lt;/li&gt;&#xA;&lt;li&gt;Computing power isn't there yet, and most probably, will never get near the level required for AGI.&lt;/li&gt;&#xA;&lt;li&gt;Humans have an innate bias to &quot;humanize&quot; non-living entities. Language, for example, is a great representation of the way our minds work on a fundamental, subconscious level. When we say &quot;my computer isn't co-operating with me&quot;, or something along those lines, we don't realize that computers are big calculators with a screen and lots of predefined operations to run.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One thing I don't disagree with: Machine Learning will replace a lot of jobs - but not nearly as many as you may think! It'll take time - if Tesla eventually comes out with a fully-autonomous semi-truck, it's not like the whole industry will adopt to it overnight. Humans, and our genome, will have time to adapt and evolve overtime. We'll free up more people, over time, to get into fields like healthcare, education, and entertainment. When we don't have the option to get jobs like cashiers, drivers, etc., we'll be forced to get more advanced jobs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just how technology has worked, and will continue to work for as long as humanity exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People, however, can use AI/ML for negative purposes and bad things. Face swapping, automated social engineering, etc., are all great examples of this. But with powerful unregulated/illegal technology, comes even more powerful regulated/legal technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out in the comments, transistors are not the only solution. There are alternatives like Quantum Computing, that enable you to remove that mathematical overhead, and directly run your operations on qubits. However, there's still a lot of work to be done to see if technology like AGI or self-guided machine learning is possible even with quantum computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of this is certain - anything I've just mentioned can change in the next few weeks, months, or years.&lt;/p&gt;&#xA;" OwnerUserId="5240" LastEditorUserId="5240" LastEditDate="2019-02-11T07:58:37.623" LastActivityDate="2019-02-11T07:58:37.623" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10506" PostTypeId="2" ParentId="10413" CreationDate="2019-02-11T08:46:42.557" Score="0" Body="&lt;p&gt;Experienced people on this subject sometimes start explaining the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fermi_paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fermi Paradox&lt;/a&gt;. How is it possible that given the apparent huge size of the universe, we still have no evidence or other human life? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways to respond to this &quot;paradox&quot;. One of them is that always that a civilization reaches some level of technology, it destroy itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there are personalities such as Ray Kurzwail (director of technology of Google), that mantains that we may reach the point in which, instead of destroying ourselves, we create something more intelligent than us, which at the same time, could create an intelligence bigger than itself (see his book, Singularity, which talks about &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;the hypothesis with the same name&lt;/a&gt;). Given that, we could just replace natural selection by conscious improvement, and expand ourselves spread in the universe very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can not know which is the future, but in my opinion we are not in a stationary point but going towards one of these destinies. The autodestruction of the humanity is plausible, but who knows if putting good constraints on artificial intelligence, and using it appropiately, we can follow rather the pathway defended by Kurzwail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case in which artificial intelligence becomes one of the causes of humanity aniquilation, this could happen because some AI with enough resources to do it becomes uncontrolled. But I think that it is more plausible to think that it would happen because of a human decision, and rather the AI be just the tool employed to do it. I think that because there is a debate that uses to go along with this one, and it is whether technology will be enough to overcome human needs, or population growth will always take the lead (see the &lt;a href=&quot;https://en.wikipedia.org/wiki/Malthusian_trap&quot; rel=&quot;nofollow noreferrer&quot;&gt;Malthusian trap&lt;/a&gt;), and till now, the last option is the one which has occured.&lt;/p&gt;&#xA;" OwnerUserId="6114" LastEditorUserId="6114" LastEditDate="2019-02-11T09:15:12.283" LastActivityDate="2019-02-11T09:15:12.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10508" PostTypeId="1" CreationDate="2019-02-11T10:38:55.857" Score="1" ViewCount="106" Body="&lt;p&gt;What is the difference between automatic transcription and automatic speech recognition? Are they the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my following interpretation correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Automatic transcription&lt;/em&gt;: it converts the speech to text by looking at the whole spoken input &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Automatic speech recognition&lt;/em&gt;: it converts the speech to text by looking into word by word choices&lt;/p&gt;&#xA;" OwnerUserId="22195" LastEditorUserId="2444" LastEditDate="2019-04-21T13:17:37.423" LastActivityDate="2021-05-10T17:07:10.670" Title="What is the difference between automatic transcription and automatic speech recognition?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;speech-synthesis&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10511" PostTypeId="2" ParentId="10508" CreationDate="2019-02-11T15:26:33.487" Score="0" Body="&lt;p&gt;They are both the same. There are different algorithms to recognise speech, but essentially they all aim to identify the content of the spoken input and convert it into written text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Automatic transcription is then done, whereas the output of more general ASR is often passed on to further processing, such as recognising entities or commands expressed in the speech.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-02-11T15:26:33.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10516" PostTypeId="2" ParentId="10500" CreationDate="2019-02-11T22:43:41.570" Score="0" Body="&lt;p&gt;No, it is not possible. We could derive the most probable &lt;span class=&quot;math-container&quot;&gt;$x_3$&lt;/span&gt; by calculating the maximum likelihood: &lt;span class=&quot;math-container&quot;&gt;$x^*_3=\underset{x_3}{\arg\max} p(x_1,x_2|x_3)$&lt;/span&gt;. We are unable to calculate this as you only stated that there is a correlation, but we don't know how it looks like.&lt;/p&gt;&#xA;" OwnerUserId="3986" LastActivityDate="2019-02-11T22:43:41.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10517" PostTypeId="1" CreationDate="2019-02-12T05:15:40.137" Score="1" ViewCount="66" Body="&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt;, the discounted return at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is defined as: &lt;span class=&quot;math-container&quot;&gt;$$ G_t \triangleq R_t+\gamma R_{t+1}+\gamma^{2}R_{t+2} + \cdots = \sum_{j=1}^{\infty} \gamma^{k}R_{t+k}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; is the reward at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \gamma &amp;lt; 1$&lt;/span&gt; is a discount factor. Let the state-value function &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; be defined as: &lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) \triangleq \mathbb{E}[G_t|S_{t}=s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, it is the expected discounted return given that we start in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with some policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Then  &lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) = \mathbb{E}_{\pi}[R_t+\gamma G_{t+1}|S_{t}=s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \sum_{a} \pi(a|s) \sum_{s',r} p(r,s'|s,a)[r+\ \gamma v_{\pi}(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;. Are the states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; drawn from a from a joint probability distribution &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt;? In other words, if you are in an&#xA;  initial state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, take an action &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the random&#xA;  state you would end up in according to the probability distribution&#xA;  &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also let &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt;, the action-value function be defined as: &lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) \triangleq \mathbb{E}_{\pi}[G_t|S_t = s, A_t = a]$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$=\sum_{s',r} p(r,s'|s,a)[r+\ \gamma v_{\pi}(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question 2.&lt;/strong&gt; What are the advantages of looking at &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; versus &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22228" LastEditorUserId="22916" LastEditDate="2019-03-18T14:22:24.930" LastActivityDate="2019-03-18T14:22:24.930" Title="Is the next state drawn from the joint distribution of the previous state and action?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10518" PostTypeId="2" ParentId="10517" CreationDate="2019-02-12T07:54:51.020" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;. Are the states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; drawn from a from a joint probability distribution &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt;? In other words, if you are in an&#xA;  initial state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, take an action &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the random&#xA;  state you would end up in according to the probability distribution&#xA;  &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is tricky, because you don't show a definition of &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt;. My first thought was that you meant the transition matrix &lt;span class=&quot;math-container&quot;&gt;$P_{ss'}^a$&lt;/span&gt;, but that doesn't fit with the phrase &lt;em&gt;joint probability distribution&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really mean &lt;em&gt;joint probability distribution&lt;/em&gt; then the answer is generally &quot;no&quot;, because &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt; should be the probability of observing state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; when taking a random sampled time step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P_{sa} = \pi(a|s)\rho_{\pi}(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\rho_{\pi}(s)$&lt;/span&gt; is the distribution of states under policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Note this makes no reference to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are ways that this could relate to the distribution of &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. Probably the most direct relationship would be when there is a deterministic environment, thus knowing &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; would determine &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. If, in addition to that, each &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; could &lt;em&gt;only&lt;/em&gt; be reached from a single &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; combination, then knowing &lt;span class=&quot;math-container&quot;&gt;$P_{sa}$&lt;/span&gt; would also give you knowledge of &lt;span class=&quot;math-container&quot;&gt;$P_{s'}$&lt;/span&gt; - this is not the same thing as the question is asking though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you &lt;em&gt;did&lt;/em&gt; mean the transition matrix &lt;span class=&quot;math-container&quot;&gt;$P_{ss'}^a$&lt;/span&gt; instead in the question, then the answer is yes, because&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P_{ss'}^a = \sum_r p(r,s'|s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question 2.&lt;/strong&gt; What are the advantages of looking at &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; versus &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The main advantage is that you can derive a policy more easily from &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi'(s) = \text{argmax}_a q_{\pi}(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare with deriving a policy using &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi'(s) = \text{argmax}_a \sum_{s',r} p(r, s'|s,a)(r + \gamma v_{\pi}(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that these policies are not necessarily the same as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; on which the &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; values are evaluated. In fact this is a common situation whilst searching for an optimal policy, and it is possible to show that &lt;span class=&quot;math-container&quot;&gt;$\pi'(s)$&lt;/span&gt; will result in same or higher returns as &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; across all states . . . the proof of this is called the &lt;a href=&quot;http://www.incompleteideas.net/book/ebook/node42.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;policy improvement theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important thing about the first equation using &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is that it does not involve using the MDP model &lt;span class=&quot;math-container&quot;&gt;$p(r, s'|s,a)$&lt;/span&gt; directly. This is the basis of model-free RL. Whilst the version using &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is more complex (taking more computation) and requires that you know &lt;span class=&quot;math-container&quot;&gt;$p(r,s'|s,a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main disadvantage of looking at &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; is that it has a larger dimension, it maps &lt;span class=&quot;math-container&quot;&gt;$S \times A \rightarrow \mathbb{R}$&lt;/span&gt;, as opposed to using &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; which maps &lt;span class=&quot;math-container&quot;&gt;$S \rightarrow \mathbb{R}$&lt;/span&gt;. So it can take longer to get good approximations of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; compared to &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-12T09:46:10.613" LastActivityDate="2019-02-12T09:46:10.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10525" PostTypeId="1" CreationDate="2019-02-12T10:33:11.910" Score="4" ViewCount="314" Body="&lt;p&gt;Suppose we have a deterministic environment where knowing &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; determines &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. Is it possible to get two different rewards &lt;span class=&quot;math-container&quot;&gt;$r\neq r'$&lt;/span&gt; in some state &lt;span class=&quot;math-container&quot;&gt;$s_{\text{fixed}}$&lt;/span&gt;? Assume that &lt;span class=&quot;math-container&quot;&gt;$s_{\text{fixed}}$&lt;/span&gt; is a fixed state I get to after taking the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. Note that we can have situations where in multiple iterations we have: &lt;span class=&quot;math-container&quot;&gt;$$(s,a) \to (s_1, r_1) \\ (s,a) \to (s_{\text{fixed}}, r_1) \\ (s,a) \to (s_{\text{fixed}}, r_2) \\ (s,a) \to (s_3, r_3) \\ \vdots$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, would &lt;span class=&quot;math-container&quot;&gt;$r_1 =r_2$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="22236" LastEditorUserId="2444" LastEditDate="2020-11-02T14:12:29.380" LastActivityDate="2020-11-02T14:12:29.380" Title="Can the rewards be stochastic when the transition model is deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10526" PostTypeId="2" ParentId="10525" CreationDate="2019-02-12T10:46:35.830" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is, would &lt;span class=&quot;math-container&quot;&gt;$r_1 =r_2$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's usually up to you as the designer of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually when you declare that you have &quot;a deterministic environment&quot;, you imply that both &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; are fixed values depending on &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt;. So in your examples, you would expect your observations to also have &lt;span class=&quot;math-container&quot;&gt;$r_1 = r_2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is &lt;em&gt;possible&lt;/em&gt; to define a MDP where transition to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is deterministic, but &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is not. For instance, you could define reward in a game equal to the sum of a number of dice rolled, with better rewards (on average) resulting in more dice. This is still a valid MDP and can be solved using RL techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A real-world example of this might be managing a queue of work, where you want to minimise lead time, but don't know for certain how long each task will take. Your state progression moves deterministically - you have a queue of pending tasks, current tasks and workers, and assigning a task to a worker is completely deterministic. However, you don't know how efficiently tasks will be performed until after they are done, so you don't know the reward perfectly from the assignment (whether you can treat this as random or hidden state is a more complex issue - it is often pragmatic to treat such unknown data as random though).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-13T07:43:06.287" LastActivityDate="2019-02-13T07:43:06.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10529" PostTypeId="1" AcceptedAnswerId="10538" CreationDate="2019-02-12T12:32:36.097" Score="3" ViewCount="140" Body="&lt;p&gt;From &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the reinforcement learning book section 13.3&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ex4a5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ex4a5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using pytorch, I need to calculate a loss, and then the gradient is calculated internally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to obtain the loss from equations which are stated in the form of an iterative update with respect to the gradient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta \leftarrow \theta + \alpha\gamma^tG\nabla_{\theta}ln\pi(A_t|S_t,\theta)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the loss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in general, what would be the loss if the update rule were&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta \leftarrow \theta + \alpha C\nabla_{\theta}g(x|\theta)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some general (derivable) function &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; parameterized by theta?&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2021-04-20T01:51:16.357" Title="How to obtain a formula for loss, when given an iterative update rule in gradient descent?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;loss-functions&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10531" PostTypeId="1" AcceptedAnswerId="10537" CreationDate="2019-02-12T15:25:24.263" Score="2" ViewCount="456" Body="&lt;p&gt;&lt;a href=&quot;http://The%20Reinforcement%20Learning%20Book%20by%20Richard%20Sutton%20et%20al&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Reinforcement Learning Book by Richard Sutton et al, section 13.5&lt;/a&gt; shows an online actor critic algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sOtRd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sOtRd.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do the weights updates depend on the discount factor via &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the more we get closer to the end of the episode, the less we value our newest experience &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems odd to me. I thought discounting in the  recursive formula of &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; itself is enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why does the weights update become less significant as the episode progresses?&lt;/strong&gt;&#xA;Note this is not eligibility traces, as those are discussed separately, later in the same episode.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2019-02-12T17:43:25.973" Title="In online one step actor critic, why does the weights update become less significant as the episode progresses?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;discount-factor&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10534" PostTypeId="1" AcceptedAnswerId="10583" CreationDate="2019-02-12T15:49:24.840" Score="1" ViewCount="49" Body="&lt;p&gt;We recently founded a company in the area of &lt;strong&gt;additive manufacturing&lt;/strong&gt;. Our development focuses on making the process easier and liberate time to the user, which I personally believe in as the ultimate promise of any computerised process, including AI, for the public. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have gone through the usual channels including regular job offers and personal contacts, visiting universities and meetups. However there appears to be a lack of participation of &lt;em&gt;real&lt;/em&gt; specialists in most of those gatherings as the ones interested in learning about AI are doing just that, and don't show up somewhere where we can find them, although we believe to have a very interesting task at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears to be remarkably hard to find specialists in the field, as AI specialists are either employed by huge multinational companies (instead of startups), have no interest in additive manufacturing (because it implies something else than only code) and the challenge of machines (making machines) or just do not pop up anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How would a specialist of AI look for someone else in the specific field of additive manufacturing/motion planning/creative strategy and its employment?&lt;/strong&gt; &lt;em&gt;(Or course in the case they do not already know someone.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="22244" LastEditorUserId="1671" LastEditDate="2019-02-14T04:00:47.090" LastActivityDate="2019-02-15T08:52:22.413" Title="How to find AI specialists interested in additive Manufacturing" Tags="&lt;ai-field&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10535" PostTypeId="2" ParentId="9563" CreationDate="2019-02-12T16:26:41.420" Score="2" Body="&lt;p&gt;I think we would consider regularization and downsampling better in this way:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;dropout&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;it puts some input value (neuron) for the next layer as 0, which makes the current layer a sparse one. So it reduces the dependence of each feature in this layer.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;pooling layer&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;the downsampling directly remove some input, and that makes the layer &quot;smaller&quot; rather than &quot;sparser&quot;. The difference can be subtle but clear enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's the root reason why the former also affect the evaluation/test process but the later does not. &lt;/p&gt;&#xA;" OwnerUserId="22249" LastActivityDate="2019-02-12T16:26:41.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10536" PostTypeId="1" CreationDate="2019-02-12T17:13:50.103" Score="1" ViewCount="455" Body="&lt;p&gt;Is there any research in this area?&lt;/p&gt;&#xA;" OwnerUserId="22251" LastEditorUserId="1671" LastEditDate="2019-02-13T02:18:05.090" LastActivityDate="2019-02-15T14:18:47.947" Title="Are there profitable hedge funds using AI?" Tags="&lt;machine-learning&gt;&lt;research&gt;&lt;algorithmic-trading&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10537" PostTypeId="2" ParentId="10531" CreationDate="2019-02-12T17:37:46.687" Score="1" Body="&lt;p&gt;This &quot;decay&quot; of later values is a direct consequence of the episodic formula for the objective function for REINFORCE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\theta) = v_{\pi_\theta}(s_0)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the expected return from the first state of the episode. This is equation 13.4 in the book edition that you linked in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if there is any discounting, we care less about rewards seen later in the episode. We mainly care about how well the agent will do from its starting position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not true for all formulations of policy gradients. There are other, related, choices of objective function. We can formulate the objective function as caring about the returns from any distribution of states, but in order to define it well, we do need to describe the weighting/distribution somehow, it should be relevant to the problem, and we want to be able to get approximate samples of &lt;span class=&quot;math-container&quot;&gt;$\nabla J(\theta)$&lt;/span&gt; for policy gradient to work. The algorithm you are asking about is specifically for improving policy for episodic problems. Note you can set &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; for these problems, so the decay is not necessarily required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside (because someone is bound to ask): Defining &lt;span class=&quot;math-container&quot;&gt;$J(\theta)$&lt;/span&gt; with respect to all states equally weighted could lead to difficulties  e.g. the objective would take less account of a policy's ability to avoid undesirable states, and it would require a lot of samples from probably irrelevant states in order to estimate it. These difficulties would turn up as a hard  to calculate (or maybe impossible) expectation for &lt;span class=&quot;math-container&quot;&gt;$\nabla J(\theta)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-12T17:43:25.973" LastActivityDate="2019-02-12T17:43:25.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10538" PostTypeId="2" ParentId="10529" CreationDate="2019-02-12T18:09:22.193" Score="3" Body="&lt;p&gt;You can find an implementation of the REINFORCE algorithm (as defined in your question) in PyTorch at the following URL: &lt;a href=&quot;https://github.com/JamesChuanggg/pytorch-REINFORCE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JamesChuanggg/pytorch-REINFORCE/&lt;/a&gt;. First of all, I would like to note that a policy can be represented or implemented as a neural network, where the input is the state (you are currently in) and the output is a &quot;probability distribution over the actions you can take from that state received as input&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Python module &lt;a href=&quot;https://github.com/JamesChuanggg/pytorch-REINFORCE/blob/master/reinforce_discrete.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JamesChuanggg/pytorch-REINFORCE/blob/master/reinforce_discrete.py&lt;/a&gt;, the policy is defined as a neural network with 2 linear layers, where the first linear layer is followed by a ReLU activation function, whereas the second is followed by a soft-max. In that same Python module, the author also defines another class called &lt;code&gt;REINFORCE&lt;/code&gt;, which creates a &lt;code&gt;Policy&lt;/code&gt; object (in the &lt;code&gt;__init__&lt;/code&gt; method) and defines it as property of that class. The class &lt;code&gt;REINFORCE&lt;/code&gt; also defines two methods &lt;code&gt;select_action&lt;/code&gt; and &lt;code&gt;update_parameters&lt;/code&gt;. These two methods are called from the &lt;a href=&quot;https://github.com/JamesChuanggg/pytorch-REINFORCE/blob/master/main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;main.py&lt;/code&gt;&lt;/a&gt; module, where the main loop of the &lt;code&gt;REINFORCE&lt;/code&gt; algorithm is implemented. In that same main loop, the author declares lists &lt;code&gt;entropies&lt;/code&gt;, &lt;code&gt;log_probs&lt;/code&gt; and &lt;code&gt;rewards&lt;/code&gt;. Note that these lists are re-initialized at ever episode. A &quot;log_prob&quot; and an &quot;entropy&quot; is returned from the &lt;code&gt;select_action&lt;/code&gt; method, whereas a &quot;reward&quot; is returned from the environment after having executed one environment step. The environment is provided by the OpenAI's Gym library. The lists &lt;code&gt;entropies&lt;/code&gt;, &lt;code&gt;log_probs&lt;/code&gt; and &lt;code&gt;rewards&lt;/code&gt; are then used to update the parameters, i.e. they are used by the method &lt;code&gt;update_parameters&lt;/code&gt; defined in the class &lt;code&gt;REINFORCE&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's see better now what these methods, &lt;code&gt;select_action&lt;/code&gt; and &lt;code&gt;update_parameters&lt;/code&gt;, actually do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;select_action&lt;/code&gt; first calls the &lt;code&gt;forward&lt;/code&gt; method of the class &lt;code&gt;Policy&lt;/code&gt;, which returns the output of the forward pass of the NN (i.e. the output of the soft-max layer), so it returns the probabilities of selecting each of the available actions (from the state given as input). It then selects the probability associated with the first action (I guess, it picks the probabilities associated with the action with the highest probabilities), denoted by &lt;code&gt;prob&lt;/code&gt; (in the source code). Essentially, what I've described so far regarding this &lt;code&gt;select_action&lt;/code&gt; method is the computation of &lt;span class=&quot;math-container&quot;&gt;$\pi(A_t \mid S_t, \theta)$&lt;/span&gt; (as shown in the pseudocode of your question). Afterwards, in the same method &lt;code&gt;select_action&lt;/code&gt;, the author also computes the log of that probability I've just mentioned above (i.e. the one associated with the  action with the highest probability, i.e. the log of &lt;code&gt;prob&lt;/code&gt;), denoted by &lt;code&gt;log_prob&lt;/code&gt;. In that same method, the entropy (as defined in &lt;a href=&quot;https://datascience.stackexchange.com/a/20301/10640&quot;&gt;this answer&lt;/a&gt;) is calculated. In reality, the author calculates the entropy using only one distribution (instead of two): more specifically, the entropy is calculated as follows &lt;code&gt;entropy = -(probs*probs.log()).sum()&lt;/code&gt;. In fact, the entropy loss function usually requires the ground-truth labels (as explained in the answer I linked you to above), but, in this case, we do not have ground-truth labels (given that we are performing RL and not supervised learning). Nonetheless, I can't really tell you why the entropy is calculated like this, in this case. Finally, the method &lt;code&gt;select_action&lt;/code&gt; then return &lt;code&gt;action[0], log_prob, entropy&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, I would like to note that the method &lt;code&gt;update_parameters&lt;/code&gt; is called only at the end of each episode (in the &lt;code&gt;main.py&lt;/code&gt; module). In that same method, a variable called &lt;code&gt;loss&lt;/code&gt; is first initialized to zero. In that method, we then iterate the list of rewards for the current episode. Inside that loop of the &lt;code&gt;update_parameters&lt;/code&gt; method, the return, &lt;code&gt;R&lt;/code&gt; is calculated. &lt;code&gt;R&lt;/code&gt; is also multiplied by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;. On each time step, the loss is then calculated as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss = loss - (log_probs[i]*(Variable(R).expand_as(log_probs[i])).cuda()).sum() - (0.0001*entropies[i].cuda()).sum()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;loss&lt;/code&gt; is calculated by subtracting the previous &lt;code&gt;loss&lt;/code&gt; with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(log_probs[i]*(Variable(R).expand_as(log_probs[i])).cuda()).sum() - (0.0001*entropies[i].cuda()).sum()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;log_probs&lt;/code&gt; are the log probabilities calculated in the &lt;code&gt;select_action&lt;/code&gt; method. &lt;code&gt;log_probs&lt;/code&gt; is the part &lt;span class=&quot;math-container&quot;&gt;$\log \pi(A_t \mid S_t, \theta)$&lt;/span&gt; of the update rule of your pseudocode. &lt;code&gt;log_probs&lt;/code&gt; are then multiplied by the return &lt;code&gt;R&lt;/code&gt;. We then sum the result of this multiplication over all elements of the vector. We then subtract this just obtained result by the entropies multiplied by 0.0001. I can't really tell you why the author decided to implement the loss in this way. I would need to think about it a little more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following article may also be useful: &lt;a href=&quot;https://pytorch.org/docs/stable/distributions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/docs/stable/distributions.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-02-12T18:09:22.193" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10540" PostTypeId="1" CreationDate="2019-02-12T19:22:41.717" Score="2" ViewCount="310" Body="&lt;p&gt;I am a beginner, just started studying around NLP, specifically various language models. So far, my understanding is that -  the goal is to understand/produce natural language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far the methods I have studied speak about correlation of words, using correct combination to make a meaningful sentence. I also have the sense that the language modeling does not really care about the punctuation marks (or did I miss it?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I am curious is there a way they can classify sentence types such as Declarative, Imperative, Interrogative or Exclamatory?&lt;/p&gt;&#xA;" OwnerUserId="22254" LastEditorUserId="1671" LastEditDate="2019-02-14T03:52:18.573" LastActivityDate="2019-08-14T00:02:15.243" Title="Is there a way to understand the type of a sentence?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;computational-linguistics&gt;&lt;semantics&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10541" PostTypeId="2" ParentId="10536" CreationDate="2019-02-13T02:16:33.177" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/High-frequency_trading&quot; rel=&quot;nofollow noreferrer&quot;&gt;High-frequency trading&lt;/a&gt; is where you see it being used, essentially, decision-making algorithms analyzing and making transactions in microseconds.  It accounts for a significant percentage of market activity, and has been considered a source of greater market volatility (See: &lt;a href=&quot;https://en.wikipedia.org/wiki/High-frequency_trading#Flash_trading&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flash Crashes&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can bet that hedge funds are evaluating every form of AI for trading and predicting market trends in general, but, unlike academics (and a segment of the tech sector) I doubt they consider it in their interests to publish methods which competitors could then utilize.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's difficult to source information on exactly what is being looked into and utilized in the financial sector because there is a lot of marketing noise (exaggerated claims, &#xA; unreliable sources) but I did find these articles: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.technologyreview.com/s/600695/will-ai-powered-hedge-funds-outsmart-the-market/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Will AI-Powered Hedge Funds Outsmart the Market? (Tech Review; 2/4/2016)&lt;/a&gt;&lt;br&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://www.bloomberg.com/news/articles/2018-09-07/ai-hedge-fund-sentient-is-said-to-shut-after-less-than-two-years&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;AI Hedge Fund Is Said to Liquidate After Less Than Two Years (Bloomberg; 9/7/2018)&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My gut tells me that Machine Learning methods will surpass humans in these kinds of decisions before too long (educated guess), and only increase in utility as the dataset these algorithms draw from grows, because it's ultimately a statistical problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;AI is related to Game Theory, the study of economic decision-making, in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;utility&lt;/a&gt;.  Game Theory might be said to have had its greatest success in computing in general, via minimax, but has traditionally been much harder to apply in real world economics.  Minimax can be utilized in machine learning, and it's actually hard to think of any economic decision-making that wouldn't utilize it in some form.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-13T18:16:15.353" LastActivityDate="2019-02-13T18:16:15.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10543" PostTypeId="5" CreationDate="2019-02-13T02:28:08.840" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/High-frequency_trading&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/High-frequency_trading&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-13T02:28:08.840" LastActivityDate="2019-02-13T02:28:08.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10544" PostTypeId="4" CreationDate="2019-02-13T02:28:08.840" Score="0" Body="For questions about algorithmic trading (stocks, commodities bonds, etc.) and predictive financial algorithms in general." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-13T02:28:08.840" LastActivityDate="2019-02-13T02:28:08.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10545" PostTypeId="1" CreationDate="2019-02-13T07:07:05.523" Score="3" ViewCount="210" Body="&lt;p&gt;For example, you train on dataset 1 with an adaptive optimizer like Adam. Should you reload the learning schedule etc from the end of training on dataset1 when attempting transfer to dataset2? Why or why not?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastActivityDate="2019-03-15T14:31:14.633" Title="Should you reload the optimizer for transfer learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10546" PostTypeId="1" AcceptedAnswerId="10547" CreationDate="2019-02-13T07:30:20.597" Score="2" ViewCount="3037" Body="&lt;p&gt;I am purchasing Titan RTX GPU. Everything seems fine with that except float32 &amp;amp; float64 performance which seems lower vis-a-vis some of its counter parts. I wanted to understand if single precision and double precision performance of GPU affect deep learning training or efficiency ? We work mostly with images, however not limited to that.&lt;/p&gt;&#xA;" OwnerUserId="17980" LastEditorUserId="9947" LastEditDate="2019-02-13T08:01:44.247" LastActivityDate="2019-02-14T09:03:08.523" Title="Does fp32 &amp; fp64 performance of GPU affect deep learning model training?" Tags="&lt;deep-learning&gt;&lt;hardware&gt;&lt;hardware-evaluation&gt;&lt;gpu&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2020-05-13T21:40:59.513" ContentLicense="CC BY-SA 4.0" />
  <row Id="10547" PostTypeId="2" ParentId="10546" CreationDate="2019-02-13T07:58:46.397" Score="2" Body="&lt;p&gt;First off I would like to post this comprehensive &lt;a href=&quot;http://timdettmers.com/2018/11/05/which-gpu-for-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; which makes comparison between all kinds of NVIDIA GPU's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most popular deep learning library TensorFlow by default uses 32 bit floating point precision. The choice is made as it helps in 2 causes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lesser memory requirements&lt;/li&gt;&#xA;&lt;li&gt;Faster calculations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;64 bit is only marginally better than 32 bit as very small gradient values will also be propagated to the very earlier layers. But the trade-off for the gain in performance vs (the time for calculations + memory requirements + time for running through so many epochs so that those small gradients actually do something) is not worth it. There are state of art CNN architectures, which insert gradients midpoint and has very good performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So overall 32 bit performance is the one which should really matter for deep learning, unless you are doing a very very high precision job (which still would hardly matter as small differences due to 64 bit representation is literally erased by any kind of softmax or sigmoid). So 64 bit might increase your accuracy classification by &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;&amp;lt; 1 {\%}$&lt;/span&gt; and will only become significant over very large datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as raw specs go the TITAN RTX in comparison to 2080Ti, TITAN will perform better than 2080Ti in fp64 (as its memory is double than 2080Ti and has higher clock speeds, BW, etc) but a more practical approach would be to use 2 2080Ti's coupled together, giving a much better performance for price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Side Note:&lt;/strong&gt; Good GPU's require good CPU's. It is difficult to tell whether a given CPU will bottleneck a GPU as it entirely depends how the training is being performed (whether data is fully loaded in GPU then training occurs, or continuous feeding from CPU takes place.)&#xA;Here are a few links explaining the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wepc.com/tips/cpu-gpu-bottleneck/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CPU and GPU Bottleneck: A Detailed Explanation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://timdettmers.com/2018/12/16/deep-learning-hardware-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Full Hardware Guide to Deep Learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-02-13T08:24:28.817" LastActivityDate="2019-02-13T08:24:28.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10549" PostTypeId="1" AcceptedAnswerId="10573" CreationDate="2019-02-13T08:33:16.277" Score="2" ViewCount="213" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1801.01290.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor&lt;/a&gt;, they define the loss function for the policy network as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_\pi(\phi)=\mathbb E_{s_t\sim \mathcal D}\left[D_{KL}\left(\pi_\phi(\cdot|s_t)\Big\Vert {\exp(Q_\theta(s_t,\cdot)\over Z_\theta(s_t)}\right)\right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Applying the reparameterization trick, let &lt;span class=&quot;math-container&quot;&gt;$a_t=f_\phi(\epsilon_t;s_t)$&lt;/span&gt;, then the objective could be rewritten as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_\pi(\phi)=\mathbb E_{s_t\sim \mathcal D, \epsilon \sim\mathcal N}[\log \pi_\phi(f_\phi(\epsilon_;s_t)|s_t)-Q_\theta(s_t,f_\phi(\epsilon_t;s_t))]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;They compute the gradient of the above objective as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\phi J_\pi(\phi)=\nabla_\phi\log\pi_\phi(a_t|s_t)+(\nabla_{a_t}\log\pi_\phi(a_t|s_t)-\nabla_{a_t}Q(s_t,a_t))\nabla_\phi f_\phi(\epsilon_t;s_t)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The thing confuses me is the first term in the gradient, where does it come from? To my best knowledge, the second large term is already the gradient we need, why do they add the first term?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="2444" LastEditDate="2020-11-23T01:49:56.180" LastActivityDate="2020-11-23T01:49:56.180" Title="What is the gradient of the objective function in the Soft Actor-Critic paper?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;gradient-descent&gt;&lt;actor-critic-methods&gt;&lt;soft-actor-critic&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10550" PostTypeId="2" ParentId="10540" CreationDate="2019-02-13T09:27:33.180" Score="1" Body="&lt;p&gt;You can generally identify the mood of a verb by looking at grammatical structures; you don't need any language model for it. The three major moods in English are declarative, interrogative, and imperative. Assuming English is the language you will be working with, here are some questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does he like coffee?&lt;/li&gt;&#xA;&lt;li&gt;Is this a piece of chocolate?&lt;/li&gt;&#xA;&lt;li&gt;When did you go there?&lt;/li&gt;&#xA;&lt;li&gt;How is that possible?&lt;/li&gt;&#xA;&lt;li&gt;Have you got any cheese?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Apart from the obvious marker '?', these examples all start with either an auxiliary verb or a &lt;em&gt;wh&lt;/em&gt;-word, so are fairly easy to recognise with a simple lookup. The one exception I can think of is an imperative (&lt;em&gt;Do be quiet, please&lt;/em&gt;), where the &lt;em&gt;do&lt;/em&gt; is followed by a further verb, which wouldn't be the case with a question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imperatives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Go to school now!&lt;/li&gt;&#xA;&lt;li&gt;Eat up your vegetables!&lt;/li&gt;&#xA;&lt;li&gt;Do shut up.&lt;/li&gt;&#xA;&lt;li&gt;Have a go at it!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These start with a main verb in the base form, or with an auxiliary followed by a verb/not a pronoun.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you identified all interrogative and imperative sentences, all the remaining ones should be declarative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, all you would need is a small list of auxiliary verb, pronouns, and &lt;em&gt;wh&lt;/em&gt;-words, and with a bit of simple string matching you should get most of the way there. Undoubtedly there will be some exceptions, but there shouldn't be too many of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other languages there will be similar structures; or there might be explicit markers (eg in Hawai'ian an imperative starts with the marker 'e', as in &lt;em&gt;Hele 'oe ma ka hale&lt;/em&gt; &quot;you go to the house&quot; vs &lt;em&gt;E hele 'oe ma ka hale&lt;/em&gt; &quot;Go to the house!&quot;) &lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-02-13T09:27:33.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10551" PostTypeId="1" CreationDate="2019-02-13T10:38:29.573" Score="1" ViewCount="46" Body="&lt;p&gt;An activation function is a function from &lt;span class=&quot;math-container&quot;&gt;$R \rightarrow R$&lt;/span&gt;. It takes as input the inner products of weights and activations in the previous layer. It outputs the activation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A softmax however, is a function that takes input from &lt;span class=&quot;math-container&quot;&gt;$R^p$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the number of possible outcomes that need to be classified. Therefore, strictly speaking, it cannot be an activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet everywhere on the net it says the softmax is an activation function. Am I wrong or are they?&lt;/p&gt;&#xA;" OwnerUserId="22273" LastActivityDate="2019-02-13T14:26:05.213" Title="Is it a great misconception that the softmax is an activation function?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10552" PostTypeId="2" ParentId="10551" CreationDate="2019-02-13T10:46:22.313" Score="1" Body="&lt;p&gt;I see no problem in regarding the softmax as a particular activation function which takes a vector input and produces a vector output. In fact, the sigmoid function can be viewed as a two-dimensional softmax in which one of the two inputs is hardwired to zero while the corresponding output is neglected.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastEditorUserId="21726" LastEditDate="2019-02-13T14:26:05.213" LastActivityDate="2019-02-13T14:26:05.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10553" PostTypeId="2" ParentId="10545" CreationDate="2019-02-13T10:58:12.800" Score="3" Body="&lt;p&gt;When doing transfer learning it makes sense to have different update policies for &quot;inherited&quot; parameters and the &quot;new&quot; parameters. &quot;Inherited&quot; parameters are pre-trained on dataset1 and they typically form the front end of the deep model. The &quot;new&quot; parameters are trained from scratch and they typically produce the desired predictions on dataset2. It would be sensible to restart the learning schedule for the &quot;new&quot; parameters. However, most often we would avoid doing that for &quot;inherited&quot; parameters in order to avoid catastrophic forgetting.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-13T10:58:12.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10554" PostTypeId="2" ParentId="10546" CreationDate="2019-02-13T11:05:08.520" Score="1" Body="&lt;p&gt;Deep models are very tolerant to arithmetic underflow. You can hope for neglectable differences in prediction accuracy between FP32 and FP16 models. Check this &lt;a href=&quot;https://arxiv.org/pdf/1710.03740.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; for concrete results.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastEditorUserId="21726" LastEditDate="2019-02-14T09:03:08.523" LastActivityDate="2019-02-14T09:03:08.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10555" PostTypeId="1" CreationDate="2019-02-13T11:27:58.547" Score="2" ViewCount="104" Body="&lt;p&gt;I've noticed that when modelling a continuous action space, the default thing to do is to estimate a mean and a variance where each is parameterized by a neural network or some other model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also often see that it is one network &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; models both. The REINFORCE objective can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla \mathcal{J}(\theta) = \mathbb{E}_{\pi} [\nabla_\theta \log \pi(a_t|s_t) * R_t] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For discrete action space this makes sense since the output of the network is determined by a softmax. However, if we explicitly model the output of the network as a Gaussian, then the gradient of the log likelihood is of a different form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi_\theta(a_t|s_t) = Normal(\mu_\theta(s_t), \Sigma_\theta(s_t))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the log is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\log \pi_\theta(a_t | s_t) =  -\frac{1}{2} (a_t-\mu_\theta)^\top \Sigma^{-1}_\theta(a_t-\mu_\theta) + \log 2 \pi \det({\Sigma_\theta})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the slides provided here (slide 18):&#xA;&lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/pg.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/pg.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IF the variance is held constant, then we can solve this analytically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_\theta \log \pi_\theta(a_t|s_t) = (a_t - \mu_\theta) \Sigma^{-1} \phi(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, are things always modelled assuming a constant variance? If it's not constant then we have to account for the inverse of the covariance matrix &#xA;as well as the determinant? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've taken a look at code online and from what I've seen, most of them assume the variance is constant. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;@NielSlater&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the reparameterization trick we would use a normal distribution with fixed parameters 0 and 1. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a_t \sim \mu_\theta(s_t) + \Sigma_\theta(s_t) * Normal(0, 1) $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the same as if we &lt;em&gt;had&lt;/em&gt; actually sampled directly from a distribution, &lt;span class=&quot;math-container&quot;&gt;$ \pi_\theta(a_t | s_t) = Normal(\mu_\theta(s_t), \Sigma_\theta(s_t))$&lt;/span&gt; and let's us calculate the corresponding &lt;span class=&quot;math-container&quot;&gt;$\log \pi_\theta(a_t|s_t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta \log \pi_\theta(a_t | s_t)$&lt;/span&gt; without having to differentiate through the actual density.&lt;/p&gt;&#xA;" OwnerUserId="7858" LastEditorUserId="7858" LastEditDate="2019-02-13T14:43:54.743" LastActivityDate="2019-02-15T09:47:37.460" Title="Calculating gradient for log policy when variance is not constant" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10556" PostTypeId="1" AcceptedAnswerId="10558" CreationDate="2019-02-13T11:28:44.470" Score="2" ViewCount="166" Body="&lt;p&gt;I wonder how self-driving cars determine the path to follow. Yes, there's GPS, but GPS can have hiccups and a precision larger than expected. Suppose the car is supposed to turn right at an intersection on the inner lane, how is the exact path determined? How does it determine the trajectory of the inner lane?&lt;/p&gt;&#xA;" OwnerUserId="13068" LastActivityDate="2019-06-29T15:33:30.490" Title="How do self-driving cars construct paths?" Tags="&lt;autonomous-vehicles&gt;&lt;path-planning&gt;&lt;path-finding&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10558" PostTypeId="2" ParentId="10556" CreationDate="2019-02-13T14:21:12.710" Score="2" Body="&lt;p&gt;As you say, GPS is not precise enough for the purpose (until recently it was only accurate within 5m or so, since 2018 there are receivers that have an accuracy of about 30cm). Instead, autonomous vehicles have a &lt;a href=&quot;https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles&quot; rel=&quot;nofollow noreferrer&quot;&gt;multitude of sensors&lt;/a&gt;, mostly cameras and radar, which record the surrounding area and monitor the road ahead. Due to them being flat, mostly one colour, and often with lines or other markers on them, roads are usually fairly easy to spot, which is why most success has been made driving on roads as opposed to off-road. Once you know exactly where you are and where you want to go, computing the correct trajectory is then just a matter of maths and physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an academic paper on the subject of trajectory planning see &lt;a href=&quot;https://www.archives-ouvertes.fr/hal-01139316/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Local Trajectory Planning and Tracking of Autonomous Vehicles, Using Clothoid Tentacles Method&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It quickly becomes more complex when other road users and obstacles are taken into account; here machine learning is used to identify stationary and movable objects at high speed from the sensor input. Reacting to the input is a further problem, and one reason why there aren't any self-driving cars on the roads today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all on &lt;a href=&quot;https://en.wikipedia.org/wiki/Self-driving_car#Levels_of_driving_automation&quot; rel=&quot;nofollow noreferrer&quot;&gt;driving automation level 2&lt;/a&gt; and above; on the lower levels things are somewhat easier. For example, the latest model Nissan LEAF has an automatic parking mode, where the car self-steers, &lt;a href=&quot;https://autoweek.com/article/green-cars/new-nissan-leaf-will-feature-automatic-self-parking&quot; rel=&quot;nofollow noreferrer&quot;&gt;guided by camera images and sonar&lt;/a&gt;, but still requires the driver to indicate the final position of the vehicle. Apart from that, it is fully automatic.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-02-13T15:04:21.553" LastActivityDate="2019-02-13T15:04:21.553" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10559" PostTypeId="1" CreationDate="2019-02-13T17:04:40.857" Score="1" ViewCount="86" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Show which literals can be inferred from the following knowledge bases, using both reasoning patterns and truth tables. Show all steps in your reasoning and explain your answers.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1) P &amp;amp; Q&lt;br&gt;&#xA;  2) Q →R v S&lt;br&gt;&#xA;  3) P → ~R &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is from my reasoning pattern tutorial, my text book shows similar question except that there is a single literal with workings, so I can somehow read through but I'm not familiar with some terms. I don't understand how I can infer all literals with the above information. I also don't fully understand what is and-elimination, modus ponens and unit resolution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anyone who is kind enough to use the above question as an example so that I can have a clearer picture?&lt;/p&gt;&#xA;" OwnerUserId="22282" LastActivityDate="2019-02-13T17:04:40.857" Title="Logic questions: reasoning pattern, Infer literals, unit resolution, and-elimination etc" Tags="&lt;logic&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10560" PostTypeId="2" ParentId="9024" CreationDate="2019-02-13T17:14:14.137" Score="3" Body="&lt;p&gt;The Q values are updated using a &amp;quot;greedy policy&amp;quot; because, in the Q-learning algorithm, the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator is used to determine the &amp;quot;target&amp;quot;, which is denoted by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator is used to take the &amp;quot;greedy&amp;quot; action, that is, the action associated with the highest Q value: &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; means that we are selecting the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value, associated with the (next) state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;, which corresponds to the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1}, a)$&lt;/span&gt; is the highest (with respect to other possible actions from &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Note that the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function receives as input a state and an action. So, for each state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, we have an action (among all possible actions, &lt;span class=&quot;math-container&quot;&gt;$a_1, a_2, \dots$&lt;/span&gt;, which we can take from the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;), denote it by &lt;span class=&quot;math-container&quot;&gt;$a^*$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$Q(s, a^*) &amp;gt; Q(s, a_1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Q(s, a^*) &amp;gt; Q(s, a_2)$&lt;/span&gt;, etc. In the expression &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt;, we are basically selecting &lt;span class=&quot;math-container&quot;&gt;$Q(s, a^*)$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$s = S_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To explain all the components of this target, I will first have to explain other parts of the Q-learning algorithm. Hopefully, after this explanation, you will be able to understand why Q-learning uses the greedy policy to update the Q values. I will first describe the Q-learning algorithm, you will then, hopefully, understand why it is called a &amp;quot;off-policy&amp;quot; algorithm (which is, IMHO, a quite unintuitive and confusing term to describe what &amp;quot;off-policy&amp;quot; actually means).&lt;/p&gt;&#xA;&lt;p&gt;Here's the &lt;a href=&quot;https://github.com/udacity/rl-cheatsheet/blob/master/cheatsheet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qymVD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qymVD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Q-learning algorithm estimates (and returns) the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function associated with a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (which needs to be passed as input). Intuitively, it does that by simulating an &amp;quot;agent&amp;quot; which takes actions in the environment, &amp;quot;observes&amp;quot; the impact (in terms of rewards received and states visited, after having taken those actions) of those actions on the environment, and then attempts to infer the optimal state-action value (or &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;) function (that is, the optimal Q function that is associated with the optimal policy, which would give the agent the &amp;quot;highest amount of reward&amp;quot; in the long run), given its interaction with the environment.&lt;/p&gt;&#xA;&lt;p&gt;Q-learning proceeds in episodes. So, initially, you need to pass the number of episodes as input. You can think of episodes as iterations (like in any iterative optimisation algorithm). However, in the context of RL, an episode is a little bit more specific: the start and end of an episode are associated with specific states of the environment: the episode starts when the agent is in a &amp;quot;starting state&amp;quot; and ends when it is in an &amp;quot;ending (or goal) state&amp;quot;. In the pseudocode above, &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt; is the starting state.&lt;/p&gt;&#xA;&lt;p&gt;In the pseudocode above, at the beginning of each episode, we initialise &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; represents the time step of a specific episode. Note also that the agent &amp;quot;moves&amp;quot; to the starting state, &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;, at the beginning of each episode.&lt;/p&gt;&#xA;&lt;p&gt;We then have the following loop, which terminates when the agent reaches a goal state (denoted, in the pseudocode, by &amp;quot;terminal&amp;quot;, i.e. &amp;quot;terminal state&amp;quot; or &amp;quot;goal state&amp;quot; are synonyms in this context):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/66r6E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/66r6E.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, at each episode, we run the loop above. The block of code inside this loop contains the main logic of the Q-learning algorithm. On each iteration of this inner loop, the agent chooses an action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; (the action at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; of the current episode) using any policy (which &amp;quot;ensures that all states are sufficiently visited&amp;quot;: &lt;em&gt;you can ignore this for now, given that this answer is already quite long!&lt;/em&gt;). In this case, the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy is used. How does this &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy work? If you look at the pseudocode above, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is initialised at beginning of each episode. In the pseudocode above, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; can change from episode to episode, but assume, for simplicity, that, at every episode, it is a fixed small number (e.g. &lt;span class=&quot;math-container&quot;&gt;$0.01$&lt;/span&gt;). The statement &amp;quot;Choose action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; using policy derived from &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; (e.g., &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy)&amp;quot; means that, with probability &lt;span class=&quot;math-container&quot;&gt;$1 - \epsilon$&lt;/span&gt;, the &amp;quot;greedy action&amp;quot; is chosen (from the current state, which is, at the beginning of the episode, &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;), and, with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, a random action is taken. What is the greedy action in this case? It is the action, in the current state, which is associated with the highest Q value (given the current estimate of the Q value). The greedy action is exactly the same action as the action &lt;span class=&quot;math-container&quot;&gt;$a^*$&lt;/span&gt; (as I explained above). The difference is that, in this case, we choose &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; using the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy: so, most of the times, we choose the greedy action, but, sometimes, we can also choose a random action. Note that the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy is indeed a policy, because it allows the agent to choose an action, given a state: this is roughly the definition of a policy.&lt;/p&gt;&#xA;&lt;p&gt;The agent then executes the just chosen action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; in the environment, and it observes the &amp;quot;impact&amp;quot; of this action on the environment, which is determined by how the environment responds to this action: the response consists of a reward, &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;, and a next state, &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To recapitulate, the agent chooses an action using the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy, executes this action on the environment, and it observes the response (that is, a reward and a next state) of the environment to this action. This is the part of the Q-learning algorithm where the agent interacts with the environment in order to gather some info about it, so as to be able to estimate the Q function.&lt;/p&gt;&#xA;&lt;p&gt;After that, the agent can update its estimate of the Q function associated with the policy given as input &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. It does that using the following update rule&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\color{orange}{Q(S_t, A_t)} \leftarrow \color{red}{Q(S_t, A_t)} + \alpha ([\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}] - \color{red}{Q(S_t, A_t)})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; is the current state (of the current episode) the agent is in, &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; is the action chosen using the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy (as described above), and &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; are respectively the next state and rewards, which, collectively, are the response of the environment to the just taken action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, how is the estimate of this &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function updated?&lt;/p&gt;&#xA;&lt;p&gt;First of all, I would like to note that, if you look at the beginning of the pseudocode above, &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; is initialized arbitrarily for all states &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt; and for all actions &lt;span class=&quot;math-container&quot;&gt;$a \in \mathcal{A}$&lt;/span&gt;: it can e.g. be initialised to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; can e.g. be implemented as a matrix (or 2-dimensional array) &lt;span class=&quot;math-container&quot;&gt;$M \in \mathbb{R}^{|\mathcal{S}| \times |\mathcal{A}|}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$M[s, a] = Q(s, a)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}|$&lt;/span&gt; is the number of states in your problem and &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}|$&lt;/span&gt; the number of actions.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, note that the symbol &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; means &amp;quot;assignment&amp;quot; (like assignment to a variable, in the context of programming). So, in the update rule above, we are assigning to &lt;span class=&quot;math-container&quot;&gt;$\color{orange}{Q(S_t, A_t)}$&lt;/span&gt; (which will be the next or updated estimate of the Q value for the current state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; and the just taken action from that state &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;) the value &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)} + \alpha (\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)} - \color{red}{Q(S_t, A_t)})$&lt;/span&gt;. Let's break this value down.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; (on the right side of the assignment) is the estimate of the Q value for the state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; before the assignment. So, we are summing &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha (\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)} - \color{red}{Q(S_t, A_t)})$&lt;/span&gt;, and then we assign it to &lt;span class=&quot;math-container&quot;&gt;$\color{orange}{Q(S_t, A_t)}$&lt;/span&gt; again.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; is what is often called the &amp;quot;target&amp;quot;. Q-learning is a &amp;quot;temporal-difference&amp;quot; (TD) algorithm, and TD algorithms update estimates of the value or action-value functions based on the difference between the current estimate, in the case of Q-learning it is denoted by &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; (on the right side of the &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt;), and a &amp;quot;target&amp;quot;. So, in the Q-learning algorithm, &lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; is the target. We can roughly think of it as &amp;quot;the value that &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; should have been&amp;quot;. So, in a certain way, we are performing &amp;quot;supervised learning&amp;quot;, where &lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; would be the &amp;quot;ground-truth&amp;quot; label and &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; the current estimate, and so &lt;span class=&quot;math-container&quot;&gt;$[\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}] - \color{red}{Q(S_t, A_t)}$&lt;/span&gt; would be the &amp;quot;error&amp;quot; (or &amp;quot;loss&amp;quot;): in fact, it is often called the &amp;quot;TD error&amp;quot;. However, note that this is not really supervised learning, because &lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; is &lt;strong&gt;not&lt;/strong&gt; a ground-truth (it is partially an estimate, because of the part &lt;span class=&quot;math-container&quot;&gt;$\gamma \color{blue}{\max_{a}Q(S_{t+1}, a)} $&lt;/span&gt;, and it partially a ground-truth, because of &lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;To recapitulate, &lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; is the &amp;quot;target&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; is the current estimate, and &lt;span class=&quot;math-container&quot;&gt;$[\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}] - \color{red}{Q(S_t, A_t)}$&lt;/span&gt; is the &amp;quot;error&amp;quot;. We are thus summing the &amp;quot;error&amp;quot; (weighted by the hyper-parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, which is, in this case, often called the &amp;quot;learning rate&amp;quot;) and the current estimate &lt;span class=&quot;math-container&quot;&gt;$\color{red}{Q(S_t, A_t)}$&lt;/span&gt; in order to produce the new estimate &lt;span class=&quot;math-container&quot;&gt;$\color{orange}{Q(S_t, A_t)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the target, you can see that we are multiplying the &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;. This is a hyper-parameter (a parameter which often needs to be chosen by the programmer before the algorithm is executed). It controls the contribution of &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; to the &amp;quot;target&amp;quot;: that is, how much of &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; we want to include in the target. Recall that I've just said above that the &amp;quot;target&amp;quot; is composed of the reward &lt;span class=&quot;math-container&quot;&gt;$\color{green}{R_{t+1}}$&lt;/span&gt; (which is a &amp;quot;ground-truth&amp;quot; or real-world &amp;quot;experience&amp;quot;, because it is directly received from the environment) and &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; (which actually uses an estimate of the Q function, that is, it uses &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1}, a)$&lt;/span&gt;). So, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; controls the contribution of an estimate to the &amp;quot;ground-truth&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;As I said at the beginning of this answer, &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; can be thought of as the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value associated with the next state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; (which was observed by the agent after he has taken the action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;) and associated with the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1}, a)$&lt;/span&gt; is the highest among all other possible actions from state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;. In other words, &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt; can be thought of the estimate of the Q value associated with the next state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; and the &amp;quot;greedy action&amp;quot; taken from that same state.&lt;/p&gt;&#xA;&lt;p&gt;I would like to note that, at this point, the agent is &lt;strong&gt;not&lt;/strong&gt; really taking actions in the environment. Nonetheless, people, in the RL community, often call Q-learning an &amp;quot;off-policy&amp;quot; algorithm because&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It uses the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy to interact with the environment (this is often called the &amp;quot;behaviour policy&amp;quot;). In this case, actions are really taken, and the responses of the environment are really produced, observed and used to update estimates of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It uses a &amp;quot;target&amp;quot; that is based on an estimate which is &amp;quot;greedy&amp;quot; (i.e. it uses &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Given that Q-learning uses estimates of the form &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\max_{a}Q(S_{t+1}, a)}$&lt;/span&gt;, Q-learning is often considered to be performing updates to the Q values, as if those Q values were associated with the &amp;quot;greedy policy&amp;quot;, that is, the policy that always chooses the action associated with highest Q value. So, you will often hear that Q-learning finds a &amp;quot;target policy&amp;quot; (i.e. the policy that is derived from the last estimate of the Q function) that is &amp;quot;greedy&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I would like to emphasize that, in the Q-learning algorithm, we have several episodes, and, at each episode, the agent interacts with the environment by taking actions and observing the responses of the environment, so as to produce an estimate of the action-value function (i.e. the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-29T13:45:52.863" LastActivityDate="2020-11-29T13:45:52.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10562" PostTypeId="1" CreationDate="2019-02-13T20:37:00.007" Score="1" ViewCount="99" Body="&lt;p&gt;I'm new to the whole world of AI and neural networks, and I'm looking to get started with training a CNN on a cloud service like AWS or GCP but I'm totally overwhelmed by the choices for VM instances. In GCP, there are options that seem to have the same amount of memory but are classifed as standard, highmem, or highcpu -- what's the difference? As well what would be some reasons to opt for having a GPU as well? &lt;/p&gt;&#xA;" OwnerUserId="22286" LastEditorUserId="1671" LastEditDate="2019-02-14T04:06:38.603" LastActivityDate="2019-07-04T17:43:52.623" Title="Choosing an instance in AWS or GCP for training a CNN" Tags="&lt;neural-networks&gt;&lt;getting-started&gt;&lt;software-evaluation&gt;&lt;cloud-services&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10563" PostTypeId="1" CreationDate="2019-02-13T22:52:57.520" Score="1" ViewCount="86" Body="&lt;p&gt;I am trying to write a genetic algorithm that generates 100 items, assigning random weights and utilities to them. And then try to pick items how out these 100 items while maximising the utility and not picking items over 500ks. The program should return an array of boolean values, where true represents items to be picked and false represents item not to be picked. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help with this or point me to a link of something that has been written like this before?&lt;/p&gt;&#xA;" OwnerUserId="22288" LastEditorUserId="2444" LastEditDate="2019-02-20T18:21:30.357" LastActivityDate="2019-02-20T18:21:30.357" Title="How do I write a genetic algorithm to solve the knapsack problem?" Tags="&lt;genetic-algorithms&gt;&lt;java&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10564" PostTypeId="2" ParentId="10562" CreationDate="2019-02-14T00:11:49.470" Score="0" Body="&lt;p&gt;In general, if you want to get a decent machine for training a neural network you need two things:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) A modern GPU&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) A big pile of RAM&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GPU is particularly important as it can greatly speed up the matrix calculations required for the training process. Your training will be between 10 -&gt; 100x faster with a GPU instance. GCP also offer TPU instances, which allege to be even faster than GPU instances.&#xA;If you get a GPU machine, you will need to make sure that you are using a GPU-enabled version of your machine learning library. For example, Tensorflow comes in two flavours - CPU-only and GPU. See &lt;a href=&quot;https://www.tensorflow.org/install/gpu&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Tensorflow docs&lt;/a&gt; for more.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-02-14T00:11:49.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10565" PostTypeId="1" AcceptedAnswerId="10605" CreationDate="2019-02-14T03:04:01.177" Score="2" ViewCount="484" Body="&lt;p&gt;My understanding of the main idea behind A2C / A3C is that we run small segments of an episode to estimate the return using a trainable value function to compensate for the unseen final steps of the episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I can see how this could work in continuing tasks with relatively dense rewards, where you can still get some useful immediate rewards from a small experience segment, does this approach work for episodic tasks where the reward is only delivered at the end? For example, in a game where you only know if you win or lose at the end of the game, does it still make sense to use the A2C / A3C approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not clear to me how the algorithm could get any useful signal to learn anything if almost every experience segment has zero reward, except for the last one. This would not be a problem in a pure MC approach for example, except for the fact that we might need a lot of samples. However, it's not clear to me that arbitrarily truncating episode segments like in A2C / A3C is a good idea in this case.&lt;/p&gt;&#xA;" OwnerUserId="22297" LastEditorUserId="2444" LastEditDate="2019-02-15T17:29:37.227" LastActivityDate="2019-02-15T19:47:58.540" Title="Are A2C or A3C suitable for episodic tasks where the reward is delivered only at the end of the episode?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10566" PostTypeId="1" CreationDate="2019-02-14T08:30:40.563" Score="2" ViewCount="199" Body="&lt;p&gt;One of the most common misconceptions about reinforcement learning (RL) applications is that, once you deploy them, they continue to learn. And, usually, I'm left having to explain this. As part of my explanations, I like to show where it is being used and where not.&lt;/p&gt;&#xA;&lt;p&gt;I've done a little bit of research on the topic, but the descriptions seem fairly academic, and I'm left with the opinion that reinforcement learning is &lt;strong&gt;not&lt;/strong&gt; really suitable for financial services in regulated markets.&lt;/p&gt;&#xA;&lt;p&gt;Am I wrong? If so, I would like to know where RL is being used? Also, in those cases, are these RL algorithms adapting to new data over time? How do you ensure they are not picking up on data points or otherwise making decisions that are considered to be unacceptable?&lt;/p&gt;&#xA;" OwnerUserId="19484" LastEditorUserId="2444" LastEditDate="2020-11-13T23:46:35.453" LastActivityDate="2020-11-14T02:17:02.713" Title="Where are reinforcement algorithms used in financial services?" Tags="&lt;reinforcement-learning&gt;&lt;applications&gt;&lt;ai-safety&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10567" PostTypeId="1" CreationDate="2019-02-14T08:47:05.807" Score="1" ViewCount="121" Body="&lt;p&gt;Which representation is most biologically plausible for actor nodes? For example, actions represented across several output nodes which may be either&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;mutually exclusive with each other (e.g., go north, go south, etc),&#xA;achieved by winner-takes-all. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;NOT mutually exclusive with each other (e.g. left leg forward, right leg forward); these actions may occur concurrently. To go north, the correct combination of nodes must be active.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Similarly which representation is most plausible for critic output nodes?&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;A single output node that outputs a real number representing the&#xA;reward. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A set of output nodes each representing a separate value, achieved by winner-takes-all.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Or do other representations better align with real brains ?&lt;/p&gt;&#xA;" OwnerUserId="22305" LastEditorUserId="2444" LastEditDate="2019-02-15T00:53:02.033" LastActivityDate="2019-02-15T08:32:37.753" Title="What is the most biologically plausible representation for the actor and critic?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10568" PostTypeId="1" CreationDate="2019-02-14T09:31:25.837" Score="1" ViewCount="22" Body="&lt;p&gt;&lt;a href=&quot;https://distill.pub/2017/feature-visualization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feature visualization&lt;/a&gt; allows to better understand neural networks by generating images that maximize the activation of a specific neuron, and therefore understand what are the abstract features that produce a high activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The examples that I saw so far are related to classification tasks. So my question is: can these concepts be applied to other convolutional neural network tasks, like semantic segmentation or image embedding (triplet loss)? What can I expect if I apply visualization algorithms to these networks?&lt;/p&gt;&#xA;" OwnerUserId="16671" LastActivityDate="2019-02-14T09:31:25.837" Title="Feature visualization on neural networks which are not for classification" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10572" PostTypeId="1" AcceptedAnswerId="10609" CreationDate="2019-02-14T13:46:38.480" Score="0" ViewCount="226" Body="&lt;p&gt;I have to calculate the affluence in localities of Metro city. To calculate affluence, I am considering a parameter per capita income. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I can get a dataset of it? What are other parameters I should consider for the problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any guidance will be fruitful for me.&lt;/p&gt;&#xA;" OwnerUserId="15368" LastEditorUserId="2444" LastEditDate="2019-02-15T20:54:21.607" LastActivityDate="2019-02-15T23:24:26.520" Title="Parameters to calculate affluence in localities of Metro city" Tags="&lt;datasets&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10573" PostTypeId="2" ParentId="10549" CreationDate="2019-02-14T15:09:59.653" Score="3" Body="&lt;p&gt;I'll give it a go here and try to answer your question, I'm not sure if this is entirely correct, so if someone thinks that it isn't please correct me.&lt;br&gt;&#xA;I'll disregard expectation here to make things simpler. First, note that policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; depends on parameter vector &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; and function &lt;span class=&quot;math-container&quot;&gt;$f_\phi(\epsilon_t;s_t)$&lt;/span&gt;, and value function &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; depends on parameter vector &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and same function &lt;span class=&quot;math-container&quot;&gt;$f_\phi(\epsilon_t;s_t)$&lt;/span&gt;. Also, one important thing that authors mention in the paper and you didn't mention is that this solution is approximate gradient not the true gradient.&lt;br&gt;&#xA;Our goal is to calculate gradient of objective function &lt;span class=&quot;math-container&quot;&gt;$J_\pi$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, so disregarding the expectation we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi J_\pi (\phi) = \nabla_\phi \log\pi(\phi,f_\phi (\epsilon_t;s_t)) - \nabla_\phi Q(s_t,\theta,f_\phi (\epsilon_t;s_t))$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's see the gradient of first term on right hand side. To get the full gradient we need to calculate derivative w.r.t to both variables, &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_\phi (\epsilon_t;s_t)$&lt;/span&gt;, so we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi \log\pi(\phi,f_\phi (\epsilon_t;s_t)) = \frac {\partial \log\pi(\phi,f_\phi (\epsilon_t;s_t))}{\partial \phi} + \frac{\partial \log\pi(\phi,f_\phi (\epsilon_t;s_t))}{\partial f_\phi(\epsilon_t;s_t)} \frac{\partial f_\phi(\epsilon_t;s_t)}{\partial \phi}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where approximation comes, they replace &lt;span class=&quot;math-container&quot;&gt;$f_\phi (\epsilon_t;s_t)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; in some places and we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi \log\pi(\phi,f_\phi (\epsilon_t;s_t)) \approx \frac {\partial \log\pi(\phi,a_t)}{\partial \phi} + \frac{\partial \log\pi(\phi,a_t)}{\partial a_t} \frac{\partial f_\phi(\epsilon_t;s_t)}{\partial \phi}$&lt;/span&gt;&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi \log\pi(\phi,f_\phi (\epsilon_t;s_t)) \approx \nabla_\phi \log\pi(\phi,a_t) + \nabla_{a_t} \log\pi(\phi,a_t) \nabla_\phi f_\phi (\epsilon_t;s_t)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second term in first expression on right hand side we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi Q(s_t,\theta,f_\phi (\epsilon_t;s_t)) = \frac {\partial Q(s_t,\theta,f_\phi (\epsilon_t;s_t))}{\partial \phi} + \frac{\partial Q(s_t,\theta,f_\phi (\epsilon_t;s_t))}{\partial f_\phi(\epsilon_t;s_t)} \frac{\partial f_\phi(\epsilon_t;s_t)}{\partial \phi}$&lt;/span&gt;&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi Q(s_t,\theta,f_\phi (\epsilon_t;s_t)) \approx \frac {\partial Q(s_t,\theta,a_t)}{\partial \phi} + \frac{\partial Q(s_t,\theta,a_t)}{\partial a_t} \frac{\partial f_\phi(\epsilon_t;s_t)}{\partial \phi}$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fist term on right hand side is 0 because &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; does not depend on &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; so we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_\phi Q(s_t,\theta,f_\phi (\epsilon_t;s_t)) \approx \nabla_{a_t}Q(s_t, \theta,a_t)\nabla_\phi f_\phi(\epsilon_t;s_t)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you add up things and you get the final result.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-02-14T15:30:03.607" LastActivityDate="2019-02-14T15:30:03.607" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10575" PostTypeId="1" AcceptedAnswerId="11069" CreationDate="2019-02-14T17:03:57.140" Score="3" ViewCount="509" Body="&lt;p&gt;In reinforcement learning, we often define two functions, the &lt;em&gt;state-value function&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^\pi(s) = \mathbb{E}_{\pi} \left[\sum_{k=0}^{\infty} &#xA;\gamma^{k}R_{t+k+1} \Bigg| S_t=s \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the &lt;em&gt;state-action-value function&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q^\pi(s,a) = \mathbb{E}_{\pi}\left[\sum_{k=0}^{\infty} \gamma^{k}R_{t+k+1}\Bigg|S_t=s, A_t=a \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt; means that these functions are defined as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectation&lt;/a&gt; with respect to a fixed policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; of what is often called the &lt;em&gt;return&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^{\infty} \gamma^{k}R_{t+k+1}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is a &lt;em&gt;discount factor&lt;/em&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{t+k+1}$&lt;/span&gt; is the &lt;em&gt;reward&lt;/em&gt; received from the environment (while the agent interacts with it) from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; onwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, both the &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; functions are defined as expectations of the return (or the &lt;em&gt;cumulative future discounted reward&lt;/em&gt;), but these expectations have different &quot;conditions&quot; (or are conditioned on different variables). The &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; function is the expectation (with respect to a fixed policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;) of the return given that the current state (the state at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;) is &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function is the expectation (with respect to a fixed policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;) of the return conditioned on the fact that the current state the agent is in is &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and the action the agent takes at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the &lt;em&gt;Bellman optimality equation&lt;/em&gt; for &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; (the optimal value function) can be expressed as the Bellman optimality equation for &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi^*}$&lt;/span&gt; (the optimal state-action value function associated with the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;) as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;V^*(s) = \max_{a \in \mathcal{A}(s)} Q^{\pi^*}(s, a)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is actually shown (or proved) at page 76 of the book &quot;Reinforcement Learning: An Introduction&quot; (1st edition) by Andrew Barto and Richard S. Sutton.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other functions, apart from the &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; functions defined above, in the RL context? If so, how are they related?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I've heard of the &quot;advantage&quot; or &quot;continuation&quot; functions. How are these functions related to the &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; functions? When should one be used as opposed to the other? Note that I'm &lt;em&gt;not just&lt;/em&gt; asking about the &quot;advantage&quot; or &quot;continuation&quot; functions, but, if possible, any existing function that is used in RL that is similar (in purpose) to these mentioned functions, and how they are related to each other. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T14:02:10.033" LastActivityDate="2020-11-23T14:13:36.230" Title="Apart from the state and state-action value functions, what are other examples of value functions used in RL?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;reference-request&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10578" PostTypeId="1" CreationDate="2019-02-14T21:44:23.633" Score="1" ViewCount="85" Body="&lt;p&gt;&lt;strong&gt;TL;DR: read the bold. The rest are details&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am trying to implement&lt;/strong&gt; &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning:An Introduction, section 13.5&lt;/a&gt; &lt;strong&gt;myself&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JDK5k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JDK5k.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAi's cartpole&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The algorithm seems to be learning something useful&lt;/strong&gt; (and not random), as shown in these graphs (different zoom on the same run):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wkyXW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wkyXW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZWAxU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZWAxU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sxkmf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sxkmf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which show the reward per episode (y axis is the &quot;time alive&quot;, x axis is episode number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, as can be seen, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The learning does not seem to stabilize.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It looks like every time the reward maxes out (200), it immediately drops.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My relevant code for reference&lt;/strong&gt; (inspired by &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/actor_critic.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;pytorch's actor critic&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;note: &lt;em&gt;in this question, &lt;code&gt;xp_batch&lt;/code&gt; is ONLY THE VERY LAST (s, a, r, s'), meaning experience replay is not in use in this code&lt;/em&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actor and critic are both distinct neural networks which &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def learn(self, xp_batch):#in this question, xp_batch is ONLY THE VERY LAST (s, a, r, s')&#xA;    for s_t, a_t, r_t, s_t1 in xp_batch:&#xA;        expected_reward_from_t = self.critic_nn(s_t)&#xA;        probs_t = self.actor_nn(s_t)&#xA;        expected_reward_from_t1 = torch.tensor([[0]], dtype=torch.float)&#xA;        if s_t1 is not None:  # s_t is not a terminal state, s_t1 exists.&#xA;            expected_reward_from_t1 = self.critic_nn(s_t1)&#xA;&#xA;        m = Categorical(probs_t)&#xA;        log_prob_t = m.log_prob(a_t)&#xA;&#xA;        delta = r_t + self.args.gamma * expected_reward_from_t1 - expected_reward_from_t&#xA;&#xA;        loss_critic = delta * expected_reward_from_t&#xA;        self.critic_optimizer.zero_grad()&#xA;        loss_critic.backward(retain_graph=True)&#xA;        self.critic_optimizer.step()&#xA;&#xA;        delta.detach()&#xA;        loss_actor = delta * log_prob_t&#xA;        self.actor_optimizer.zero_grad()&#xA;        loss_actor.backward()&#xA;        self.actor_optimizer.step()&#xA;&#xA;def select_action(self, state):&#xA;    probs = self.actor_nn(state)&#xA;    m = Categorical(probs)&#xA;    action  = m.sample()&#xA;    return action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions are:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Am I doing something wrong, or is this to be expected&lt;/strong&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I know this can be improved with eligibility traces/experience replay+off policy learning. &lt;strong&gt;Before making those upgrades, I want to make sure the current results make sense.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2019-02-14T21:44:23.633" Title="How to make episode ending &quot;good&quot; in reinforcement learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10579" PostTypeId="1" CreationDate="2019-02-15T04:49:44.107" Score="4" ViewCount="123" Body="&lt;p&gt;In value iteration, we have a model of the environment's dynamics, i.e &lt;span class=&quot;math-container&quot;&gt;$p(s', r \mid s, a)$&lt;/span&gt;, which we use to update an estimate of the value function.&lt;/p&gt;&#xA;&lt;p&gt;In the case of temporal-difference and Monte Carlo methods, we do not use &lt;span class=&quot;math-container&quot;&gt;$p(s', r \mid s, a)$&lt;/span&gt;, but then how do these methods work?&lt;/p&gt;&#xA;" OwnerUserId="22337" LastEditorUserId="2444" LastEditDate="2020-11-16T21:18:49.660" LastActivityDate="2020-11-16T21:18:49.660" Title="How do temporal-difference and Monte Carlo methods work, if they do not have access to model?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10580" PostTypeId="1" AcceptedAnswerId="10587" CreationDate="2019-02-15T06:36:26.550" Score="2" ViewCount="205" Body="&lt;p&gt;What is the common representation used for the state in articulated robot environments? My first guess is that it's a set of the angles of every joint. Is that correct? My question is motivated by the fact that one common trick that helps training neural nets in general is to normalize the inputs, like setting mean = 0 and std dev = 1, or scaling all the input values to &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, which could be easily done in this case too if all the inputs are angles in &lt;span class=&quot;math-container&quot;&gt;$[0, 2 \pi]$&lt;/span&gt;. But, what about distances? Is it common to, for example, use as input some distance of the agent to the ground, or a distance to some target position? In that case, the scale of the distances can be arbitrary and vary a lot. What are some common ways to deal with that?&lt;/p&gt;&#xA;" OwnerUserId="22297" LastActivityDate="2019-02-15T10:37:28.507" Title="how to normalize the state space for articulated robot environments?" Tags="&lt;reinforcement-learning&gt;&lt;robotics&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10581" PostTypeId="2" ParentId="10579" CreationDate="2019-02-15T07:05:51.960" Score="5" Body="&lt;p&gt;The main idea is that you can estimate &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s)$&lt;/span&gt;, the value of a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; under a given policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, even if you don't have a model of the environment, by visiting that state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; after that state. If you repeat this process many times, you'll get many samples of trajectories starting at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with some total return associated with them. If you average them, you'll have a sample-based estimate of &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s)$&lt;/span&gt;. In a similar way, you can do a sample-based approach to estimate &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt;, where you start at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, take some arbitrary action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, and then continue following &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. After you collect many samples, you can also average the returns to get a sample-based estimate of &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt;. Once you have a good enough estimate of &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt;, you can see that it's easy to improve the current policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to get a new policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; that always chooses the best action on any given state: &lt;span class=&quot;math-container&quot;&gt;$\pi'(s) = \underset{a}{argmax}\ Q^\pi(s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure I described in the last paragraph where you sample an entire trajectory and wait until the end of the episode to estimate a return is the Monte Carlo approach. In contrast, TD exploits the recursive nature of the Bellman equation to learn as you go, even before the episode ends. For example, you can estimate the value function like this: &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s_0) = r_0 + \gamma V^\pi(s_1)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$r_0$&lt;/span&gt; comes from experience and instead of collecting more &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; until the end of the episode, you rely on your estimate of the value of the next state (which you also get from experience). At the beginning, your estimates might be random, but after many iterations the process converges to good estimates. The same idea of exploiting this recursive structure can be used to estimate &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt;. I guess this approach is known as temporal difference because you sample &lt;span class=&quot;math-container&quot;&gt;$r_0$&lt;/span&gt; which is in some sense the temporal difference between &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22297" LastEditorUserId="22297" LastEditDate="2019-02-15T08:54:43.203" LastActivityDate="2019-02-15T08:54:43.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10582" PostTypeId="2" ParentId="10567" CreationDate="2019-02-15T08:32:37.753" Score="1" Body="&lt;p&gt;For the actor, I'd say the 'not mutually exclusive' option is more biologically plausible in the context of muscle systems, where the actions can be seen as simultaneous muscle activations. Maybe at a higher level, an agent thinks of the action as 'go north' or 'go south', but the final outputs which have to control muscles at a lower level have to represent simultaneous muscle activations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the critic, I'd say the 'single output node' is more biologically plausible. Agents perceive the world in the form of high dimensional inputs, such as images. The approach where a value function is learned in a tabular fashion where you know the value for every single state doesn't really scale very well and is limited to small discrete state spaces. For biological agents, it makes sense to have a function that senses the current state of the environment and outputs a single number that represents the value, which gives the agent an idea of how things are going so far given the actions it took in the past.&lt;/p&gt;&#xA;" OwnerUserId="17312" LastActivityDate="2019-02-15T08:32:37.753" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10583" PostTypeId="2" ParentId="10534" CreationDate="2019-02-15T08:52:22.413" Score="1" Body="&lt;p&gt;You should probably compile a list of Universities/Instituitions that specialise in Additive Manufacturing, and target graduating students or researchers looking for a new challenge. This may prove more fruitful than targeting the normal channels. You may have to search worldwide, but of course this adds the issue of work visa etc.&#xA;There is no easy way to solve this....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yo can try posting on the &lt;a href=&quot;https://academia.stackexchange.com/&quot;&gt;Academia&lt;/a&gt; forum too.&lt;/p&gt;&#xA;" OwnerUserId="15812" LastActivityDate="2019-02-15T08:52:22.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10584" PostTypeId="1" CreationDate="2019-02-15T09:24:46.600" Score="3" ViewCount="97" Body="&lt;p&gt;In section &quot;&lt;em&gt;5.2 Monte Carlo Estimation of Action Values&lt;/em&gt;&quot; of the second edition of the reinforcement learning book by Sutton and Barto, this is stated:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If a model is not available, then it is particularly useful to estimate action values (the values of state– action pairs) rather than state values. With a model, state values alone are sufficient to determine a policy; one simply looks ahead one step and chooses whichever action leads to the best combination of reward and next state, as we did in the chapter on DP.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, I don't see how this is true in practice. I can see how it'd work trivially for discrete state and action spaces with deterministic environment dynamics, because we could compute &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \underset{a}{\text{argmax}}\ V(\text{step}(s, a))$&lt;/span&gt; by just looking at all possible actions and choosing the best one. As soon as I think about continuous state and action spaces with stochastic environment dynamics, computing the &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}$&lt;/span&gt; seems to be become very complicated and impractical. For the particular case of continuous states and discrete actions, I think estimating an action value might be more practical to do even if a forward model of the environment dynamics is available, because the &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}$&lt;/span&gt; becomes easier (I'm especially thinking of the approach taken in deep Q learning). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct in thinking this way or is it true that if a model is available it's not useful to estimate action values if state values are already available?&lt;/p&gt;&#xA;" OwnerUserId="17312" LastEditorUserId="2444" LastEditDate="2019-06-22T19:28:50.643" LastActivityDate="2019-06-22T19:28:50.643" Title="Why is the state value function sufficient to determine the policy if a model is available?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10585" PostTypeId="2" ParentId="10555" CreationDate="2019-02-15T09:47:37.460" Score="2" Body="&lt;p&gt;It's true that computing the log prob of a sample from a gaussian requires inverting a matrix and dealing with the determinant in the general case of a full covariance matrix. If you wanted to backprop through the log prob, you'd need to backprop through these operations, which by the way are available as differentiable operations in pytorch. See the gaussian distribution class which deals with these operations: &lt;a href=&quot;https://github.com/pytorch/pytorch/blob/master/torch/distributions/multivariate_normal.py#L177&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pytorch/pytorch/blob/master/torch/distributions/multivariate_normal.py#L177&lt;/a&gt; &#xA;You can see there's a &lt;code&gt;half_log_det&lt;/code&gt; and although you won't find an explicit matrix inversion, you'll see that it uses a differentiable triangular linear solve (&lt;code&gt;torch.trtrs&lt;/code&gt;) at some point, which is essentially doing a matrix inversion in a more efficient way.&lt;/p&gt;&#xA;" OwnerUserId="17312" LastActivityDate="2019-02-15T09:47:37.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10586" PostTypeId="1" CreationDate="2019-02-15T10:28:54.233" Score="5" ViewCount="985" Body="&lt;p&gt;I often see the terms episode, trajectory and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following paragraphs, I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think &lt;em&gt;episode&lt;/em&gt; has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of &lt;span class=&quot;math-container&quot;&gt;$(s, a, r)$&lt;/span&gt; sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With &lt;em&gt;trajectory&lt;/em&gt;, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think &lt;em&gt;rollout&lt;/em&gt; is somewhere in between, since I commonly see it used to refer to a sampled sequence of &lt;span class=&quot;math-container&quot;&gt;$(s, a, r) $&lt;/span&gt;from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.&lt;/p&gt;&#xA;" OwnerUserId="12640" LastEditorUserId="2444" LastEditDate="2019-02-15T17:21:45.107" LastActivityDate="2019-03-17T21:00:55.150" Title="What is the difference between an episode, a trajectory and a rollout?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10587" PostTypeId="2" ParentId="10580" CreationDate="2019-02-15T10:37:28.507" Score="4" Body="&lt;p&gt;This paper might provide some answers &lt;a href=&quot;https://arxiv.org/pdf/1810.05762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1810.05762.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the observations / states they used not only angles, but also velocities, heights and positions (Table 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 4.2 Learning algorithm you can see that they mention this, which is related to your question about normalization:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Additionally, for stability we whiten the current observations by&#xA;  maintaining online statistics of mean and standard deviation from the&#xA;  history of past observations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12640" LastActivityDate="2019-02-15T10:37:28.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10588" PostTypeId="2" ParentId="10431" CreationDate="2019-02-15T10:43:21.157" Score="5" Body="&lt;p&gt;You should choose the model A. The loss is just a differentiable proxy for accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the situation should be examined in more detail. If the higher loss is due to the data term, examine the data which produce high loss and check for presence of overfitting or incorrect labels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the higher loss is due to a regularizer then reducing the regularization factor may further improve the results.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastEditorUserId="21726" LastEditDate="2019-02-15T18:17:09.667" LastActivityDate="2019-02-15T18:17:09.667" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10589" PostTypeId="2" ParentId="10431" CreationDate="2019-02-15T11:07:42.003" Score="1" Body="&lt;p&gt;It depends on your application! Imagine a binary classifier that is always very &quot;confident&quot; - it always assigns P=100% to Class A and 0% to Class B, or &lt;em&gt;vice versa&lt;/em&gt; (sometimes wrong, never uncertain!).  Now imagine a &quot;humble&quot; model that is perhaps fractionally less accurate, but whose probabilities are actually meaningful (when it says &quot;Class A with probability 70%&quot; it is wrong 30% of the time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, both losses are quite small, so we probably prefer the more accurate one.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-02-15T11:07:42.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10591" PostTypeId="1" CreationDate="2019-02-15T13:20:50.677" Score="12" ViewCount="2044" Body="&lt;p&gt;Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, if the environment is &lt;em&gt;deterministic&lt;/em&gt; (that is, if the agent is in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and takes action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, then the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is always the same, no matter which time step), then the optimal policy should also be deterministic (that is, it should be a map from states to actions, and not to a probability distribution over actions).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-15T14:22:21.487" LastActivityDate="2019-02-16T02:37:23.903" Title="Is the optimal policy always stochastic if the environment is also stochastic?" Tags="&lt;reinforcement-learning&gt;&lt;stochastic-policy&gt;&lt;deterministic-policy&gt;&lt;policies&gt;&lt;environment&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10592" PostTypeId="2" ParentId="10584" CreationDate="2019-02-15T13:25:20.850" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;With a model, state values alone are sufficient to determine a policy; one simply looks ahead one step and chooses whichever action leads to the best combination of reward and next state, as we did in the chapter on DP.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As soon as I think about continuous state and action spaces with stochastic environment dynamics, computing the &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}$&lt;/span&gt; seems to be become very complicated and impractical.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For stochastic dynamics the calculations would be more complex, but will often be quite tractable. It depends on size of the distribution, and ease of calculating probabilities to make the correct weighted sums. Instead of &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}(s)|$&lt;/span&gt; calls to &lt;span class=&quot;math-container&quot;&gt;$Q(s,*)$&lt;/span&gt; you will need to make roughly &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S'}(s)| \times |\mathcal{A}(s)|$&lt;/span&gt; calls to &lt;span class=&quot;math-container&quot;&gt;$V(s')$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S'}(s) \in \mathcal{S}$&lt;/span&gt; are all possible states that might result from the starting state. In the worst case this is &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}| \times |\mathcal{A}|$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Despite the additional work here, you have been more efficient earlier. By only evaluating &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, you have removed the action dimension, which otherwise splits up your estimates. Your value function estimates will, all else being equal, converge faster because of this. So this is sometimes a compromise you might be willing to make.&lt;/p&gt;&#xA;&lt;p&gt;For continuous states, this might still be practical, if both action choices and possible transitions are still discrete.&lt;/p&gt;&#xA;&lt;p&gt;Once either action space is continuous, or transitions are continuous over some probability density function, then finding the maximising action via the model becomes impractical.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-02-15T13:31:32.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10593" PostTypeId="2" ParentId="10591" CreationDate="2019-02-15T13:47:00.093" Score="7" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An optimal policy is generally deterministic unless:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Important state information is missing (a POMDP). For example, in a map where the agent is not allowed to know its exact location or remember previous states, and the state it is given is not enough to disambiguate between locations. If the goal is to get to a specific end location, the optimal policy may include some random moves in order to avoid becoming stuck. Note that the environment in this case could be deterministic (from the perspective of someone who can see the whole state), but still lead to requiring a stochastic policy to solve it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is some kind of minimax game theory scenario, where a deterministic policy can be punished by the environment or another agent. Think scissors/paper/stone or prisoner's dilemma.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Intuitively, if the environment is deterministic (that is, if the agent is in a state  and takes action , then the next state ′ is always the same, not matter which time step), then the optimal policy should also be deterministic (that is, it should be a map from states to actions, and not to a probability distribution over actions).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That seems reasonable, but you can take that intuition further with any method based on a value function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If you have found an optimal value function, then acting greedily with respect to it &lt;strong&gt;is&lt;/strong&gt; the optimal policy.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above statement is just a natural language re-statement of the Bellman optimality equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v^*(s) = \text{max}_a \sum_{r,s'}p(r,s'|s,a)(r+\gamma v^*(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. the optimal values are obtained when always choosing the action that maximises reward plus discounted value of next step. The &lt;span class=&quot;math-container&quot;&gt;$\text{max}_a$&lt;/span&gt; operation is deterministic (if necessary you can break ties for max value deterministically with e.g. an ordered list of actions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, any environment that can be modelled by a MDP and solved by a value-based method (e.g. value iteration, Q-learning) has an optimal  policy which is deterministic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible in such an environment that the optimal solution may not be stochastic at all (i.e. if you add any randomness to the deterministic optimal policy, the policy will become strictly worse). However, when there are ties for maximum value for one or more actions in one or more states then there are multiple equivalent optimal and deterministic policies. You may construct a stochastic policy that mixes these in any combination, and it will also be optimal.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-15T15:13:03.083" LastActivityDate="2019-02-15T15:13:03.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10594" PostTypeId="2" ParentId="10591" CreationDate="2019-02-15T13:54:52.390" Score="5" Body="&lt;p&gt;I would say no. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-armed bandit problem&lt;/a&gt;. So, you have &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; arms which all have a probability of giving you a reward (1 point, for example), &lt;span class=&quot;math-container&quot;&gt;$p_i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; being between 1 and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. This is a simple stochastic environment: this is a one state environment, but it is still an environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But obviously the optimal policy is to choose the arm with the highest &lt;span class=&quot;math-container&quot;&gt;$p_i$&lt;/span&gt;. So this is not a stochastic policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, if you are in an environment where you play against other agent (a game theory setting), your optimal policy will certainly be stochastic (think of a poker game, for example). &lt;/p&gt;&#xA;" OwnerUserId="8912" LastEditorUserId="2444" LastEditDate="2019-02-15T16:04:27.027" LastActivityDate="2019-02-15T16:04:27.027" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10595" PostTypeId="2" ParentId="10536" CreationDate="2019-02-15T14:18:47.947" Score="2" Body="&lt;p&gt;Machine learning based hedge funds are currently the worst performing slice through the industry. Large quant funds also claim to use ML but their performance is also very poor and worsening over time. Mostly what they say they are dong is not really what they are actually doing. I know of cases where funds claimed to be doing AI but actually its just simple technical analysis. Successful applications are more in the area of data collation of sentiment using NLP or feature extraction using dimensional reduction but these are not actually part of the trading strategies themselves. The only ML influenced trading I know is HFT trading of the order book which is a well define problem divorced from the actual price series itself. Apart from that 99% of ML use in the hedge fund industry is solely as snake oil for their marketing departments. Out of 10,000 or so hedge funds world wide only 0.25% generate alpha according to academic studies. That's 25 out of 10,000 ... and many of those will be cheating like SAC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental problem is that ML requires data that is high dimensional, highly structured and is low noise and has stationary dynamics and statistical moments. Unfortunately financial price series are low dimensional, unstructured and noise dominated (negative SNR) and exhibit multi-level non-stationarity of the underlying processes and statistical moments in a manner highly related to multi fractals. It is hard to imagine a time series less suitable for machine learning. &lt;/p&gt;&#xA;" OwnerUserId="17764" LastActivityDate="2019-02-15T14:18:47.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10596" PostTypeId="2" ParentId="10591" CreationDate="2019-02-15T15:58:00.120" Score="0" Body="&lt;p&gt;I'm thinking of a probability landscape, in which you find yourself as an actor, with various unknown peaks and troughs.  A good deterministic approach is always likely to lead you to the nearest local optimum, but not necessarily to the global optimum.  To find the global optimum, something like an MCMC algorithm would allow to stochastically accept a temporarily worse outcome in order to escape from a local optimum and find the global optimum. My intuition is that in a stochastic environment this would also be true.&lt;/p&gt;&#xA;" OwnerUserId="22355" LastActivityDate="2019-02-15T15:58:00.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10599" PostTypeId="5" CreationDate="2019-02-15T17:13:23.453" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-14T23:19:31.273" LastActivityDate="2019-12-14T23:19:31.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10600" PostTypeId="4" CreationDate="2019-02-15T17:13:23.453" Score="0" Body="For questions related to the concept of Partially Observable Markov Decision Process (POMDP), which is a generalization of the Markov Decision Process (MDP) to the cases where information about the states is incomplete (or partially observable)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-14T23:19:31.273" LastActivityDate="2019-12-14T23:19:31.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10603" PostTypeId="1" AcceptedAnswerId="10631" CreationDate="2019-02-15T19:30:11.267" Score="8" ViewCount="310" Body="&lt;p&gt;This is not meant to be negative or a joke but rather looking for a productive solution on AI development, engineering and its impact on human life:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lately with my Google searches, the AI model keeps auto filling the ending of my searches with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;“...in Vietnamese”&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;/p&gt;&#xA;&#xA;&lt;p&gt;“...in a Vietnamese home”&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is I have never searched for that but because of my last name the model is creating this context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other issue is that I’m a halfy and my dad is actually third generation, I grew up mainstream American and don’t even speak Vietnamese. I’m not even sure what a Vietnamese home means. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My buddy in a similar situation of South Asian and noticed the same exact thing more so with YouTube recommended videos. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We already have enough issues in the US with racism, projections of who others expect us to be based on any number of things, stereotyping and putting people in boxes to limit them - I truly believe AI is adding to the problem, not helping. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we fix this. Moreover, how can we use AI to bring out peoples true self, talents and empower and free them them to create their life how they like ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is huge potential here to harness AI in ways that can bring us more freedom, joy and beauty so people can be the whole of themselves and with who they really are. Then meet peoples needs, wishes, dreams and hope. Given them shoulders to stand on to create their reality, not live someone else's projection of themselves.&lt;/p&gt;&#xA;" OwnerUserId="22368" LastEditorUserId="2444" LastEditDate="2019-08-09T12:28:55.203" LastActivityDate="2019-08-09T12:28:55.203" Title="How is it that AI can become biased, and what are the proposals to mitigate this?" Tags="&lt;philosophy&gt;&lt;human-like&gt;&lt;social&gt;&lt;algorithmic-bias&gt;" AnswerCount="3" CommentCount="8" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10604" PostTypeId="2" ParentId="10431" CreationDate="2019-02-15T19:40:51.510" Score="3" Body="&lt;p&gt;You should note that both your results are consistent with a &quot;true&quot; probability of 87% accuracy, and your measurement of a difference between these models is not statistically significant. With an 87% accuracy applied at random, then there is approx 14% chance of getting the two extremes of accuracy you have observed by chance if samples are chosen randomly from the target population, and models are different enough make errors effectively at random. This last assertion is usually not true though, so you can relax a little - that is, unless you took different random slices for cross-validation in each case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;100 test cases is not really enough to discern small differences between models. I would suggest using k-fold cross-validation in order to reduce errors in your accuracy and loss estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it is critical to check that the cross-validation split was identical in both cases here. If you have used auto-splitting with a standard tool and not set the appropriate RNG seed, then you may have got a different set each time, and your results are just showing you variance due to the validation split which could completely swamp any differences between the models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, assuming the exact same dataset was used each time, and it was representative sample of your target population, then on average you should expect the one with the best metric to have the highest chance of &lt;em&gt;being&lt;/em&gt; the best model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you should really do is decide which metric to base the choice on &lt;em&gt;in advance of the experiment&lt;/em&gt;. The metric should match some business goal for the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you are trying to choose after the fact, you should go back to the reason you created the model in the first place and see if you can identify the correct metric. It might not be either accuracy or loss.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-02-15T19:40:51.510" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10605" PostTypeId="2" ParentId="10565" CreationDate="2019-02-15T19:47:58.540" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My understanding of the main idea behind A2C / A3C is that we run small segments of an episode to estimate the return using a trainable value function to compensate for the unseen final steps of the episode.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems fairly accurate. The important thing to note is that the trainable value function is trained to predict &lt;strong&gt;values&lt;/strong&gt; (specifically, &lt;strong&gt;advantage&lt;/strong&gt; values of state-action pairs in the case of A2C / A3C, where the first A stands for &quot;advantage&quot;). These value estimates can intuitively be understood as estimates of long-term (discounted) rewards, they're not just short-term rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, initially when the agent only observes a reward at the end of a long trajectory, only state-action pairs close to the end will receive credit for that reward. For example, when using &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns, approximately only the last &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; state-action pairs receive credit. However, in the next episode, that longer-term reward will already become &quot;visible&quot; in the form of an advantage value prediction when you're still &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps away from the end, and then that update can again get propagated back &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps further into the history of state-action pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My explanation above is &lt;strong&gt;very informal&lt;/strong&gt;... there are all kinds of nuances that I skipped over. Use of function approximation is likely to speed up the propagation of reward observations through the space of state-action pairs even more, and of course in reality things won't be as &quot;clean&quot; as getting the propagation to get &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps further in the next episode in comparison to the previous episode, since selected actions and random state transitions can be different... but hopefully it gets the idea across.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-15T19:47:58.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10606" PostTypeId="2" ParentId="10586" CreationDate="2019-02-15T20:11:05.780" Score="4" Body="&lt;p&gt;I don't really think there are fixed, different definitions for all those terms that everyone agrees upon. In most contexts they're going to be quite interchangeable, and if anyone is really using them in a context where they are supposed to have crucially important, different meanings, they should probably precisely define them right there.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of &lt;span class=&quot;math-container&quot;&gt;$(s,a,r)$&lt;/span&gt; sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Agreed with this.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can't really think of cases where it's sensible to talk about trajectories with tuples shuffled into an arbitrary order. I'd still think of trajectories as having to be in the &quot;correct&quot; order in which they were experienced. But I do agree that trajectories can be little samples (for instance, little sequences of experience that we store in an experience replay buffer). So, every full episode would be a (long) trajectory, but not every trajectory is a full episode (a trajectory can just be a small part of an episode).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think rollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of &lt;span class=&quot;math-container&quot;&gt;$(s,a,r)$&lt;/span&gt; from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd say that... often a rollout should have a &quot;terminal&quot; state as ending, but maybe not a true &quot;initial&quot; state of an episode as start. We might be in the middle of an episode, and then say that we &quot;roll out&quot;, which to me implies that we keep going until the end of an episode. I don't think this term is as common as the other two in Reinforcement Learning, but more common in search / planning literature (in particular, Monte Caro Tree Search). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, when I'm working with MCTS I often like to put a limit on my rollouts where I cut them off if no terminal state was reached yet... so that isn't exactly a crisp definition either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to how commonly-used this term is specifically in MCTS, and other Monte-Carlo-based algorithm, I also associate a greater degree of randomness with the term &quot;rollout&quot;. When I hear &quot;episode&quot; or &quot;trajectory&quot;, I can envision a highly sophisticated, &quot;intelligent&quot; policy being used to select actions, but when I hear &quot;rollout&quot; I am inclined to think of a greater degree of randomness being incorporated in the action selection (maybe uniformly random, or maybe with some cheap-to-compute, simple policy for biasing away from uniformity). Again, that's really just an association I have in my mind with the term, and not a crisp definition.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-15T20:11:05.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10607" PostTypeId="1" CreationDate="2019-02-15T22:19:54.047" Score="0" ViewCount="93" Body="&lt;p&gt;I'd like to use machine learning to guess a mathematical pattern: the input are certain polynomials in four variables &lt;span class=&quot;math-container&quot;&gt;$q_1,q_2,q_3,q_4$&lt;/span&gt;, the output can be zero or one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Allowed polynomials are such that (i) all their non-zero coefficients are equal to one, (ii) they do not contain monomials of the form &lt;span class=&quot;math-container&quot;&gt;$q_1^j$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$j \geq 0$&lt;/span&gt;, and (iii) if an allowed polynomial contains a monomial &lt;span class=&quot;math-container&quot;&gt;$m=q_1^a q_2^b q_3^c q_4^d$&lt;/span&gt; for some non-negative integers &lt;span class=&quot;math-container&quot;&gt;$a,b,c,d$&lt;/span&gt;, then it also contains &lt;span class=&quot;math-container&quot;&gt;$m'=q_1^{a-1} q_2^b q_3^c q_4^d$&lt;/span&gt;, provided this does not violate (ii) and &lt;span class=&quot;math-container&quot;&gt;$a \geq 1$&lt;/span&gt;; similarly for &lt;span class=&quot;math-container&quot;&gt;$b \to b-1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c \to c-1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$d \to d-1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example batch, given by pairs {input, output}: &lt;span class=&quot;math-container&quot;&gt;$\{q_2,1\},\{q_3,1\},\{q_4,1\}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a second batch: &lt;span class=&quot;math-container&quot;&gt;$\{q_2+q_1 q_2,0\},\{q_2+q_2^2,0\},\{q_2+q_3,1\},\{q_3+q_1 q_3,0\},\{q_3+q_3^2,0\},\{q_2+q_4,1\},\{q_3+q_4,1\},\{q_4+q_1 q_4,0\},\{q_4+q_4^2,1\}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can construct larger and larger batches using Mathematica, and I'd like to know how to practically go from here, to instructing an AI to guess a simple function of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;'s that reproduces the behavior, namely that can guess the correct output for previously unknown admissible polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the typical batch size and computational power required for such a program to succeed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to use a function &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; from the space of allowed polynomials &lt;span class=&quot;math-container&quot;&gt;$\mathcal P$&lt;/span&gt; to the set &lt;span class=&quot;math-container&quot;&gt;$\mathbb Z_2=\{0,1\}$&lt;/span&gt;, of the form &lt;span class=&quot;math-container&quot;&gt;$\phi:\mathcal P \to \mathbb Z_2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$p=\sum_{i \in I} m_i \mapsto \phi(p):= \sum_{i \in I' \subset I} m_i|_1 \mod 2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m_i|_1$&lt;/span&gt; means the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th monomial inside &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; evaluated at &lt;span class=&quot;math-container&quot;&gt;$q_1=q_2=q_3=q_4=1$&lt;/span&gt;, and come up with the form of &lt;span class=&quot;math-container&quot;&gt;$I'$&lt;/span&gt; as function of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice there's no linear structure on &lt;span class=&quot;math-container&quot;&gt;$\mathcal P$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remark: of course instead of polynomials one could use punctured solid partitions.&lt;/p&gt;&#xA;" OwnerUserId="22371" LastEditorUserId="22371" LastEditDate="2019-02-18T16:01:50.273" LastActivityDate="2019-02-18T16:01:50.273" Title="Using AI to guess a mathematical pattern of certain polynomials in four variables: practical challenge" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;implementation&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10609" PostTypeId="2" ParentId="10572" CreationDate="2019-02-15T23:24:26.520" Score="1" Body="&lt;p&gt;Affluence could encompass several parameters:&#xA;Income;&#xA;Wealth (property ownership);&#xA;Life expectancy;&#xA;Access to services such as education and health;&#xA;Access to clean natural resources;&#xA;Low levels of criminality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Property prices in each locality might be easy to obtain from real estate agent sources&#xA;Ratings for schools or medical facilities in each area might be published&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, where public statistics are collected on a locality, they will be related in one way or another to affluence. A useful strategy might be to collect as many of these diverse data sets as possible, and to learn a composite affluence score from the data. It is very likely that all of these parameters will be correlated to a greater or lesser degree, and so you could accurately learn about affluence from a small number of these parameters.&lt;/p&gt;&#xA;" OwnerUserId="22355" LastActivityDate="2019-02-15T23:24:26.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10611" PostTypeId="1" AcceptedAnswerId="10614" CreationDate="2019-02-16T00:21:40.397" Score="3" ViewCount="335" Body="&lt;p&gt;I am in the process of writing my own basic machine learning library in Python as an exercise to gain a good conceptual understanding. I have successfully implemented backpropagation for activation functions such as &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; and the sigmoid function. However, these are normalised in their outputs. A function like ReLU is unbounded so its outputs can blow up really fast. In my understanding, a classification layer, usually using the SoftMax function, is added at the end to squash the outputs between 0 and 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does backpropagation work with this? Do I just treat the SoftMax function as another activation function and compute its gradient? If so, what is that gradient and how would I implement it? If not, how does the training process work? If possible, a pseudocode answer is preferred.&lt;/p&gt;&#xA;" OwnerUserId="22373" LastActivityDate="2019-02-16T15:06:03.950" Title="How does backpropagation with unbounded activation functions such as ReLU work?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10614" PostTypeId="2" ParentId="10611" CreationDate="2019-02-16T10:41:08.120" Score="3" Body="&lt;p&gt;Backprop through ReLU is easier than backprop through sigmoid activations. For positive activations, you just pass through the input gradients as they were. For negative activations you just set the gradients to 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding softmax, the easiest approach is to consider it a part of the negative log-likelihood loss. In other words, I am suggesting to directly derive gradients of that loss with respect to the softmax input. The result is very elegant and extremely easy to implement. Try to derive that yourself!&lt;/p&gt;&#xA;" OwnerUserId="21726" LastEditorUserId="21726" LastEditDate="2019-02-16T15:06:03.950" LastActivityDate="2019-02-16T15:06:03.950" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10615" PostTypeId="1" AcceptedAnswerId="10616" CreationDate="2019-02-16T14:04:33.143" Score="1" ViewCount="1609" Body="&lt;p&gt;This is an excerpt taken from &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; (pg. 3):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Another key feature of reinforcement learning is that it explicitly considers the whole problem of a goal-directed agent interacting with an uncertain environment. This is in contrast with many approaches that address subproblems without addressing how they might fit into a larger picture. For example, we have mentioned that much of machine learning research is concerned with supervised learning without explicitly specifying how such an ability would finally be useful. Other researchers have developed theories of planning with general goals, but without considering planning's role in real-time decision-making, or the question of where the predictive models necessary for planning would come from. Although these approaches have yielded many useful results, their focus on&#xA;isolated subproblems is a significant limitation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have an idea of supervised learning (SL), but what exactly does the author mean by &lt;em&gt;planning&lt;/em&gt;? And how is the RL approach different from planning and SL?&lt;/p&gt;&#xA;&lt;p&gt;(Illustration with an example would be nice).&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-11-21T13:13:29.887" LastActivityDate="2020-11-21T13:15:24.327" Title="What is &quot;planning&quot; in the context of reinforcement learning, and how is it different from RL and SL?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;supervised-learning&gt;&lt;planning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10616" PostTypeId="2" ParentId="10615" CreationDate="2019-02-16T15:40:36.297" Score="6" Body="&lt;p&gt;The concept of &amp;quot;planning&amp;quot; is not just related to RL. In general (as the name suggests), planning consists in creating a &amp;quot;plan&amp;quot; which you will use to reach a &amp;quot;goal&amp;quot;. The goal depends on the context or problem. For example, in robotics, you can use a &amp;quot;planning algorithm&amp;quot; (e.g. Dijkstra's algorithm) in order to find the path between two points on a map (given e.g. the map as a graph).&lt;/p&gt;&#xA;&lt;p&gt;In RL, planning usually refers to the use of a model of the environment in order to find a policy that hopefully will help the agent to behave optimally (that is, obtain the highest amount of return or &amp;quot;future cumulative discounted reward&amp;quot;). In RL, the problem (or environment) is usually represented as a Markov Decision Process (MDP). The &amp;quot;model&amp;quot; of the environment (or MDP) refers to the &lt;em&gt;transition probability distribution&lt;/em&gt; (and &lt;em&gt;reward function&lt;/em&gt;) associated with the MDP. If the transition model (and reward function) is known, you can use an algorithm that exploits it to (directly or indirectly) find a policy. This is the usual meaning of planning in RL. A common planning algorithm in RL is e.g. &lt;em&gt;value iteration&lt;/em&gt; (which is a dynamic programming algorithm).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Other researchers have developed theories of planning with general goals, but without considering planning's role in real-time decision- making, or the question of where the predictive models necessary for planning would come from.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Planing is often performed &amp;quot;offline&amp;quot;, that is, you &amp;quot;plan&amp;quot; before executing. While you're executing the &amp;quot;plan&amp;quot;, you often do not change it. However, often this is not desirable, given that you might need to change the plan because the environment might also have changed. Furthermore, the authors also point out that planning algorithms often have a few limitations: in the case of RL, a &amp;quot;model&amp;quot; of the environment is required to plan.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For example, we have mentioned that much of machine learning research is concerned with supervised learning without explicitly specifying how such an ability would finally be useful.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think the authors simply want to say that supervised learning is usually used to solve specific problems. The solutions to supervised problems often are not directly applicable to other problems, so this makes them limited.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Another key feature of reinforcement learning is that it explicitly considers the whole problem of a goal-directed agent interacting with an uncertain environment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In RL, there is the explicit notion of a &amp;quot;goal&amp;quot;: there is an agent that interacts with an environment in order to achieve its goal. The goal is often to maximize the &amp;quot;return&amp;quot; (or &amp;quot;future cumulative discounted reward&amp;quot;, or, simply, the reward in the long run).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How is RL different from planning and supervised learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;RL and planning (in RL) are quite related. In RL, the problem is similar to the one in planning (in RL). However, in RL, the transition model and reward function of the MDP (which represents the environment) are usually unknown. Therefore, the only way of finding or estimating an optimal policy that will allow the agent to (near-optimally) behave in this environment is to interact with the environment and gather some info regarding its &amp;quot;dynamics&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;RL and supervised learning (SL) are quite different. In SL, there isn't usually the explicit concept of &amp;quot;agent&amp;quot; or &amp;quot;environment&amp;quot; (and their interaction), even though it might be possible to describe supervised learning in that way (see &lt;a href=&quot;https://ai.stackexchange.com/q/14167/2444&quot;&gt;this question&lt;/a&gt;). In supervised learning, during the training or learning phase, a set of inputs and the associated expected outputs is often provided. Then the &amp;quot;objective&amp;quot; is to find a map between inputs and outputs, which generalizes to inputs (and corresponding outputs) that have not been observed during the learning phase. In RL, there isn't such a set of inputs and associated expected outputs. In RL, there is just a scalar signal emitted by the environment, at each time step, which roughly indicates how well the agent is currently performing. However, the goal of the agent is not just to obtain rewards, but to behave optimally (in the long run).&lt;/p&gt;&#xA;&lt;p&gt;In short, in RL, there is the explicit notion of agent, environment and goal, and the reward is the only signal which tells the agent how well it is performing, but the reward does not tell the agent which actions it should take at each time step. In supervised learning, the objective is to find a function that maps inputs to the corresponding outputs, and this function is learned by providing explicit examples of such mappings during the training phase.&lt;/p&gt;&#xA;&lt;p&gt;There are some RL algorithms (like the temporal-difference ones), which could roughly be thought of as self-supervised learning algorithms, where the agent learns from itself (or from the experience it has gained by interacting with the environment). However, even in these cases, the actions that the agent needs to take are not explicitly taught.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T13:15:24.327" LastActivityDate="2020-11-21T13:15:24.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10617" PostTypeId="1" AcceptedAnswerId="10619" CreationDate="2019-02-16T16:09:31.217" Score="1" ViewCount="183" Body="&lt;p&gt;I am trying to understand how RNNs are used for sequence modelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a &lt;a href=&quot;https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial here&lt;/a&gt;, it mentions that if you want to translate say a sentence from English to French you can use an encoder-decoder set-up as they described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However what if you want to do a sequence to sequence modelling where your inputs and outputs are of the same domain but you just want to predict the next output of a sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if I want to use sequence modelling to learn the sine function. So say I have 20 y-coordinates from &lt;span class=&quot;math-container&quot;&gt;$y = sin(x)$&lt;/span&gt; from 20 evenly spaced out x-coordinates and I want to predict the next 10 or so y-coordinates. Would I use an encoder-decoder setup here? &lt;/p&gt;&#xA;" OwnerUserId="18312" LastEditorUserId="2444" LastEditDate="2019-02-21T11:17:40.413" LastActivityDate="2019-02-21T11:17:40.413" Title="Do I need an encoder-decoder architecture to predict the next item of a sequence?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;machine-translation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10618" PostTypeId="2" ParentId="10615" CreationDate="2019-02-16T16:11:48.580" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;automated planning&lt;/a&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Automated planning and scheduling, sometimes denoted as simply AI Planning,&lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; is a branch of artificial intelligence that concerns the realization of strategies or action sequences, typically for execution by intelligent agents, autonomous robots and unmanned vehicles. Unlike classical control and classification problems, the solutions are complex and must be discovered and optimized in multidimensional space. Planning is also related to decision theory.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In known environments with available models, planning can be done offline. Solutions can be found and evaluated prior to execution. In dynamically unknown environments, the strategy often needs to be revised online. Models and policies must be adapted. Solutions usually resort to iterative trial and error processes commonly seen in artificial intelligence. These include dynamic programming, &lt;strong&gt;reinforcement learning&lt;/strong&gt; and combinatorial optimization. Languages used to describe planning and scheduling are often called action languages.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, the planning is some strategies or actions to reach from the start state to the goal state. As you found in the above, one of the solutions for planning could be RL (depends on the problem). Hence, MDP is a specific case of planning and it is more general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the difference of RL and supervised learning you can see &lt;a href=&quot;https://datascience.stackexchange.com/a/38902/27665&quot;&gt;this post&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The main difference is to do with how &quot;correct&quot; or optimal results are learned:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;In Supervised Learning, the learning model is presented with an input and desired output. It learns &lt;em&gt;by example&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;In Reinforcement Learning, the learning &lt;em&gt;agent&lt;/em&gt; is presented with an environment and must guess correct output. Whilst it receives feedback on how good its guess was, it is never told the correct output (and in addition the feedback may be delayed). It learns &lt;em&gt;by exploration&lt;/em&gt;, or trial and error.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-02-16T17:02:52.353" LastActivityDate="2019-02-16T17:02:52.353" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10619" PostTypeId="2" ParentId="10617" CreationDate="2019-02-16T16:51:37.753" Score="2" Body="&lt;p&gt;You don't need to Encoder-Decoder here. When using seq2seq learning for text (for example, for translation) you need encoder-decoder to encode the words into the numeric vectors and decode the vectors into the words. Therefore, for your numerical case, you don't need an encoder or decoder to train the RNN.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2019-02-21T11:16:11.087" LastActivityDate="2019-02-21T11:16:11.087" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10620" PostTypeId="1" AcceptedAnswerId="23647" CreationDate="2019-02-16T18:59:07.733" Score="1" ViewCount="353" Body="&lt;p&gt;In reinforcement learning (RL), there are model-based and model-free algorithms. In short, model-based algorithms use a transition model (e.g. a probability distribution) and the reward function, even though they do not necessarily compute (or estimate) them. On the other hand, model-free algorithms do not use such a transition model or reward function, but they directly estimate e.g. the state or state-action value functions by interacting with the environment, which allows the agent to infer the dynamics of the environment.&lt;/p&gt;&#xA;&lt;p&gt;Given that model-based RL algorithms do not necessarily estimate or compute the transition model or reward function, in the case these are unknown, how can they be computed or estimated (so that they can be used by the model-based algorithms)? In general, what are examples of algorithms that can be used to estimate the transition model and reward function of the environment (represented as either an MDP, POMDP, etc.)?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T11:49:31.490" LastActivityDate="2020-11-09T12:22:19.570" Title="How can we estimate the transition model and reward function?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;&lt;inverse-rl&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10621" PostTypeId="1" CreationDate="2019-02-16T19:15:53.957" Score="1" ViewCount="34" Body="&lt;p&gt;&lt;strong&gt;Introduction:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notion that various social complex systems (e.g. society, family, business company, state, etc) could be regarded as ones exhibiting consistent traits of behaviour of their own - suggesting that they are entities unto themselves, some sort of organisms on their own or even intelligent entities on their own - is not new. I have personally stumbled upon papers of scholars who straightforwardly speak of such systems as if they were already proven to be singular entities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That kind of assumption has entered the vernacular, as well, long time ago - e.g. &quot;the state wants to...&quot; , &quot;society responds to conflict by...&quot;, &quot;the family dynamics seeks balance through...&quot;, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we could even assume at one point, that such social systems are not only organisms of their own, but even some sort of &lt;strong&gt;artificial intelligence entities&lt;/strong&gt; (as long as we could see them as an artificial product of human activity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are generally used to seeing ourselves as conscious entities and we are also good at exploring entities of less complexity than ourselves. But when it comes to entities which consists of us as mere components, we are not ready to mentally process that idea - it sounds as either too abstract or too sci-fi (think of Stanisław Lem's work).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;While the average Joe could easily say &lt;em&gt;&quot;The state wants to...&quot; or &quot;Society responds to...&quot;, etc&lt;/em&gt;, how exactly do we prove (or at least gather some sort of supporting evidence) that a complex social system really exhibits a behaviour of its own? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Under what conditions could we regard it as some sort of spontaneously born artificial intelligence? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If that were true, how could we predict if that AI would procreate and bring about other social system forms which are also entities unto themselves? How could we possibly become aware if that has already happened?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22390" LastEditorUserId="22390" LastEditDate="2019-02-18T12:56:58.083" LastActivityDate="2019-02-18T12:56:58.083" Title="Complex systems constituting an entity unto itself" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;artificial-consciousness&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10622" PostTypeId="1" CreationDate="2019-02-16T19:21:08.867" Score="1" ViewCount="252" Body="&lt;p&gt;I am reading about the actor-critic architecture. I am confused about how the actor determines the action using the value (or future reward) from the critic network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below you have the most popular picture of actor-critic network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pKKmS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pKKmS.png&quot; alt=&quot;Actor Critic Network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the input of the actor network is only the &quot;state&quot; variable (&lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;), it has nothing to do with the critic network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from the equation below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OKy4l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OKy4l.png&quot; alt=&quot;Policy Gradient with Critic Network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the actor seems to be related to critic network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does actor network has two inputs, state variable and future reward (output from critic network), or only the state variable?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the actor network does take future reward as input, how does it use it? Only during the training stage, or in the action making stage?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a &quot;policy iteration&quot; procedure happens during decision making stage, i.e. for every state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, policy network will make several attempts with critic network, and output the best policy? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22393" LastEditorUserId="2444" LastEditDate="2019-02-17T02:11:45.480" LastActivityDate="2019-02-17T02:11:45.480" Title="How the actor use the output from the critic to make action in actor-critic network?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10623" PostTypeId="1" AcceptedAnswerId="10624" CreationDate="2019-02-16T20:02:58.273" Score="67" ViewCount="59074" Body="&lt;p&gt;What is self-supervised learning in machine learning? How is it different from supervised learning?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T02:46:44.143" LastActivityDate="2020-11-20T02:46:44.143" Title="What is self-supervised learning in machine learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;supervised-learning&gt;&lt;self-supervised-learning&gt;&lt;feature-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="28" ContentLicense="CC BY-SA 4.0" />
  <row Id="10624" PostTypeId="2" ParentId="10623" CreationDate="2019-02-16T20:02:58.273" Score="72" Body="&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&lt;p&gt;The term &lt;em&gt;self-supervised learning&lt;/em&gt; (SSL) has been used (sometimes differently) in different contexts and fields, such as representation learning [&lt;a href=&quot;https://arxiv.org/pdf/1708.07860.pdf&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;], neural networks, robotics [&lt;a href=&quot;https://arxiv.org/pdf/1809.07207.pdf&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;], natural language processing, and reinforcement learning. In all cases, the basic idea is to &lt;strong&gt;automatically&lt;/strong&gt; generate some kind of supervisory signal to solve some task (typically, to learn representations of the data or to automatically label a dataset).&lt;/p&gt;&#xA;&lt;p&gt;I will describe what SSL means more specifically in three contexts: representation learning, neural networks and robotics.&lt;/p&gt;&#xA;&lt;h2&gt;Representation learning&lt;/h2&gt;&#xA;&lt;p&gt;The term self-supervised learning has been widely used to refer to techniques that do not use human-annotated datasets to learn (visual) representations of the data (i.e. representation learning).&lt;/p&gt;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&lt;p&gt;In [&lt;a href=&quot;https://arxiv.org/pdf/1708.07860.pdf&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;], two patches are randomly selected and cropped from an unlabelled image and the goal is to predict the relative position of the two patches. Of course, we have the relative position of the two patches once you have chosen them (i.e. we can keep track of their centers), so, in this case, this is the automatically generated supervisory signal. The idea is that, to solve this task (known as a &lt;strong&gt;pretext&lt;/strong&gt; or &lt;strong&gt;auxiliary&lt;/strong&gt; task in the literature [&lt;a href=&quot;https://arxiv.org/pdf/1905.01235.pdf&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1505.05192.pdf&quot; rel=&quot;noreferrer&quot;&gt;4&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1803.07728.pdf&quot; rel=&quot;noreferrer&quot;&gt;5&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1901.09005.pdf&quot; rel=&quot;noreferrer&quot;&gt;6&lt;/a&gt;]), the neural network needs to learn features in the images. These learned representations can then be used to solve the so-called &lt;strong&gt;downstream&lt;/strong&gt; tasks, i.e. the tasks you are interested in (e.g. object detection or semantic segmentation).&lt;/p&gt;&#xA;&lt;p&gt;So, you first learn representations of the data (by SSL pre-training), then you can transfer these learned representations to solve a task that you actually want to solve, and you can do this by fine-tuning the neural network that contains the learned representations on a labeled (but smaller dataset), i.e. you can use SSL for transfer learning.&lt;/p&gt;&#xA;&lt;p&gt;This example is similar to the example given &lt;a href=&quot;https://ai.stackexchange.com/a/13749/2444&quot;&gt;in this other answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Neural networks&lt;/h3&gt;&#xA;&lt;p&gt;Some neural networks, for example, &lt;strong&gt;autoencoders&lt;/strong&gt; (AE) [&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-46672-9_10&quot; rel=&quot;noreferrer&quot;&gt;7&lt;/a&gt;] are sometimes called self-supervised learning tools. In fact, you can train AEs without images that have been manually labeled by a human. More concretely, consider a de-noising AE, whose goal is to reconstruct the original image when given a noisy version of it. During training, you actually have the original image, given that you have a dataset of uncorrupted images and you just corrupt these images with some noise, so you can calculate some kind of distance between the original image and the noisy one, where the original image is the supervisory signal. In this sense, AEs are self-supervised learning tools, but it's more common to say that AEs are unsupervised learning tools, so SSL has also been used to refer to unsupervised learning techniques.&lt;/p&gt;&#xA;&lt;h2&gt;Robotics&lt;/h2&gt;&#xA;&lt;p&gt;In [&lt;a href=&quot;https://arxiv.org/pdf/1809.07207.pdf&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;], the training data is &lt;strong&gt;automatically&lt;/strong&gt; but approximately labeled by finding and exploiting the relations or correlations between inputs coming from different sensor modalities (and this technique is called SSL by the authors). So, as opposed to representation learning or auto-encoders, in this case, an actual labeled dataset is produced automatically.&lt;/p&gt;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&lt;p&gt;Consider a robot that is equipped with a &lt;em&gt;proximity sensor&lt;/em&gt; (which is a &lt;em&gt;short-range&lt;/em&gt; sensor capable of detecting objects in front of the robot at short distances) and a &lt;em&gt;camera&lt;/em&gt; (which is &lt;em&gt;long-range&lt;/em&gt; sensor, but which does not provide a direct way of detecting objects). You can also assume that this robot is capable of performing &lt;a href=&quot;https://en.wikipedia.org/wiki/Odometry&quot; rel=&quot;noreferrer&quot;&gt;odometry&lt;/a&gt;. An example of such a robot is &lt;a href=&quot;https://github.com/jeguzzi/mighty-thymio&quot; rel=&quot;noreferrer&quot;&gt;Mighty Thymio&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Consider now the task of detecting objects in front of the robot at longer ranges than the range the proximity sensor allows. In general, we could train a CNN to achieve that. However, to train such CNN, in supervised learning, we would first need a labelled dataset, which contains labelled images (or videos), where the labels could e.g. be &amp;quot;object in the image&amp;quot; or &amp;quot;no object in the image&amp;quot;. In supervised learning, this dataset would need to be manually labelled by a human, which clearly would require a lot of work.&lt;/p&gt;&#xA;&lt;p&gt;To overcome this issue, we can use a self-supervised learning approach. In this example, the basic idea is to associate the output of the proximity sensors at a time step &lt;span class=&quot;math-container&quot;&gt;$t' &amp;gt; t$&lt;/span&gt; with the output of the camera at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (a smaller time step than &lt;span class=&quot;math-container&quot;&gt;$t'$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;More specifically, suppose that the robot is &lt;em&gt;initially&lt;/em&gt; at coordinates &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; (on the plane), at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. At this point, we still do not have enough info to label the output of the camera (at the same time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;). Suppose now that, at time &lt;span class=&quot;math-container&quot;&gt;$t'$&lt;/span&gt;, the robot is at position &lt;span class=&quot;math-container&quot;&gt;$(x', y')$&lt;/span&gt;. At time step &lt;span class=&quot;math-container&quot;&gt;$t'$&lt;/span&gt;, the output of the proximity sensor will e.g. be &amp;quot;object in front of the robot&amp;quot; or &amp;quot;no object in front of the robot&amp;quot;. Without loss of generality, suppose that the output of the proximity sensor at &lt;span class=&quot;math-container&quot;&gt;$t' &amp;gt; t$&lt;/span&gt; is &amp;quot;no object in front of the robot&amp;quot;, then the label associated with the output of the camera (an image frame) at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; will be &amp;quot;no object in front of the robot&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-01T13:58:51.847" LastActivityDate="2020-08-01T13:58:51.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10625" PostTypeId="1" AcceptedAnswerId="10637" CreationDate="2019-02-16T21:15:07.833" Score="3" ViewCount="71" Body="&lt;p&gt;In this tutorial from Jeremy Howard: &lt;a href=&quot;https://pytorch.org/tutorials/beginner/nn_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is torch.nn really?&lt;/a&gt; he has an example towards the end where he creates a CNN for mnist. In &lt;code&gt;nn.Conv2d&lt;/code&gt; he makes the inchannels and outchannels: &lt;code&gt;(1,16), (16,16), (16,10)&lt;/code&gt;. I get that the last one has to be 10 because there are 10 classes and we want 'probabilities' of each class. But why go up to 16 first? How do you choose this value? And why not just go from 1 to 10, 10 to 10, and 10 to 10? Does this have to do with the kernel_size and stride?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the images are &lt;code&gt;28x28&lt;/code&gt; so I can't see any correlation between these values and 16 either.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Mnist_CNN(nn.Module):&#xA;    def __init__(self):&#xA;        super().__init__()&#xA;        self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1)&#xA;        self.conv2 = nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1)&#xA;        self.conv3 = nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1)&#xA;&#xA;    def forward(self, xb):&#xA;        xb = xb.view(-1, 1, 28, 28)&#xA;        xb = F.relu(self.conv1(xb))&#xA;        xb = F.relu(self.conv2(xb))&#xA;        xb = F.relu(self.conv3(xb))&#xA;        xb = F.avg_pool2d(xb, 4)&#xA;        return xb.view(-1, xb.size(1))&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12983" LastActivityDate="2019-02-17T17:07:13.017" Title="Mnist CNN Architecture" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10628" PostTypeId="2" ParentId="9141" CreationDate="2019-02-17T05:31:04.100" Score="24" Body="&lt;p&gt;For newbies, NO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sentence generation requires sampling from a language model, which gives the probability distribution of the next word given previous contexts. But BERT can't do this due to its bidirectional nature.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For advanced researchers, YES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can start with a sentence of all [MASK] tokens, and generate words one by one in arbitrary order (instead of the common left-to-right chain decomposition). Though the text generation quality is hard to control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the technical report &lt;a href=&quot;https://arxiv.org/abs/1902.04094&quot; rel=&quot;noreferrer&quot;&gt;BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model&lt;/a&gt;, its &lt;a href=&quot;https://sites.google.com/site/deepernn/home/blog/amistakeinwangchoberthasamouthanditmustspeakbertasamarkovrandomfieldlanguagemodel&quot; rel=&quot;noreferrer&quot;&gt;errata&lt;/a&gt; and the &lt;a href=&quot;https://github.com/nyu-dl/bert-gen&quot; rel=&quot;noreferrer&quot;&gt;source code&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In summary:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you would like to do some research in the area of decoding with&#xA;BERT, there is a huge space to explore&lt;/li&gt;&#xA;&lt;li&gt;If you would like to generate&#xA;high quality texts, personally I recommend you to check &lt;a href=&quot;https://blog.openai.com/better-language-models/&quot; rel=&quot;noreferrer&quot;&gt;GPT-2&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22399" LastEditorUserId="22399" LastEditDate="2019-10-13T09:55:34.357" LastActivityDate="2019-10-13T09:55:34.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10630" PostTypeId="2" ParentId="10133" CreationDate="2019-02-17T09:16:51.480" Score="2" Body="&lt;p&gt;These embeddings are nothing more than token embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You just randomly initialize them, then use gradient descent to train them, just like what you do with token embeddings.&lt;/p&gt;&#xA;" OwnerUserId="22399" LastActivityDate="2019-02-17T09:16:51.480" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="10631" PostTypeId="2" ParentId="10603" CreationDate="2019-02-17T11:28:53.970" Score="12" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Lately with my Google searches, the AI model keeps auto filling the ending of my searches with:&lt;/p&gt;&#xA;&lt;p&gt;“...in Vietnamese”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I can see how this would be annoying.&lt;/p&gt;&#xA;&lt;p&gt;I don't think Google's auto-complete algorithm and training data is publicly available. Also it changes frequently as they work to improve the service. As such, it is hard to tell what exactly is leading it to come up with this less-than-useful suggestion.&lt;/p&gt;&#xA;&lt;p&gt;Your suspicion that it has something to do with Google's service detecting your heritage seems plausible.&lt;/p&gt;&#xA;&lt;p&gt;The whole thing is based around statistical inference. At no point does any machine &amp;quot;know&amp;quot; what Vietnamese - or in fact any of the words in your query  - actually &lt;em&gt;means&lt;/em&gt;. This is a weakness of pretty much all core NLP work in AI, and is called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;noreferrer&quot;&gt;grounding problem&lt;/a&gt;. It is why, for instance, that samples of computer generated text produce such surreal and comic material. The rules of grammar are followed, but semantics and longer term coherence are a mess.&lt;/p&gt;&#xA;&lt;p&gt;Commercial chatbot systems work around this with a lot of bespoke coding around some subject area, such as booking tickets, shopping etc. These smaller domains are possible for human developers to &amp;quot;police&amp;quot;, connecting them back to reality, and avoiding the open-ended nature of the whole of human language. Search engine text autocomplete however, cannot realistically use this approach.&lt;/p&gt;&#xA;&lt;p&gt;Your best bets are probably:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Wait it out. The service will improve. Whatever language use statistics are at work here are likely change over time. Your own normal use of the system without using the suggestions will be part of that data stream of corrections.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Send a complaint to Google. Someone, somewhere in Google will care about these results, and view them as errors to be fixed.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Neither of these approaches guarantee results in any time frame sadly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We already have enough issues in the US with racism, projections of who others expect us to be based on any number of things, stereotyping and putting people in boxes to limit them - I truly believe AI is adding to the problem, not helping.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are not alone in having these worries. The statistics-driven nature of machine learning algorithms and use of &amp;quot;big data&amp;quot; to train them means that machines are exposing bias and prejudice that are long buried in our language. These biases are picked up by machinery then used by companies that don't necessarily want to reflect those attitudes.&lt;/p&gt;&#xA;&lt;p&gt;A similar example occurs in natural language processing models with word embeddings. A very interesting feature of LSTM neural networks that learn statistical language models is that you can look at &lt;a href=&quot;https://towardsdatascience.com/introduction-to-word-embedding-and-word2vec-652d0c2060fa&quot; rel=&quot;noreferrer&quot;&gt;word embeddings&lt;/a&gt;, mathematical representations of words, and do &amp;quot;word math&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W(king) - W(man) + W(woman) \approx W(queen)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W(he) - W(male) + W(female) \approx W(she)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is very cool, and implies that the learned embeddings really are capturing semantics up to some depth. However, the same model can produce results like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W(doctor) - W(male) + W(female) \approx W(nurse)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This doesn't reflect modern sensibilities of gender equality. There is obviously a deep set reason for this, as it has appeared from non-prejudiced statistical analysis of billions of words of text from all sorts of sources. Regardless of this though, engineers responsible for these systems would prefer that their models did not have these flaws.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How can we fix this. Moreover, how can we use AI to bring out peoples true self, talents and empower and free them them to create their life how they like ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Primarily by recognising that statistical ML and AI doesn't inherently have prejudice or any agenda at all. It is reflecting back ugliness already in the world. The root problem is to fix people (beyond scope of this answer, if I had solid ideas about this I would not be working in software engineering, but in something more people-focussed).&lt;/p&gt;&#xA;&lt;p&gt;However, we can remove some of the unwanted bias from AI systems. Broadly the steps toward this go:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Recognise that a particular AI system has captured and is using unwanted gender, racial, religious etc bias.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reach a consensus about how an unbiased model should behave. It must still be useful for purpose.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Add the desired model behaviour into the training and assessment routines of the AI.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For instance in your case, there are possibly some users of Google's system who would prefer to read articles in Vietnamese, or have English translated into Vietnamese, and are finding it awkward that the default assumption is that everything should be presented in English. These users don't necessarily need to use the search text for this, but presumably are for some reason. A reasonable approach is to figure out how their needs could be met without spamming &amp;quot;in Vietnamese&amp;quot; on the end of every autocomplete suggestion, and perhaps in general move suggestions to localise searches by cultural differences out of autocomplete into a different part of the system.&lt;/p&gt;&#xA;&lt;p&gt;For the case of gender bias in NLP systems, &lt;a href=&quot;https://www.coursera.org/lecture/nlp-sequence-models/debiasing-word-embeddings-zHASj&quot; rel=&quot;noreferrer&quot;&gt;Andrew Ng's Coursera course on RNNs&lt;/a&gt; shows how this can be achieved using the embeddings themselves. Essentially it can be done by identifying a bias direction from a set of words (e.g. &amp;quot;he/she&amp;quot;, &amp;quot;male/female&amp;quot;), and removing deviations in that direction for most other words, preserving it only for words where it is inherently OK to reflect the differences (such as &amp;quot;king&amp;quot; and &amp;quot;queen&amp;quot; for gender bias).&lt;/p&gt;&#xA;&lt;p&gt;Each case of unwanted bias though needs to be discovered by people and oversight of this as a political and social issue, not primarily a technical one.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-02-17T15:14:23.997" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10632" PostTypeId="5" CreationDate="2019-02-17T12:00:48.190" Score="0" Body="&lt;p&gt;Supervised learning is a machine learning technique where a function which maps inputs to outputs is learned using a labelled training dataset. A good learned function should be able to generalise to unseen (during the training phase) data.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:34:02.047" LastActivityDate="2019-02-18T23:34:02.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10633" PostTypeId="4" CreationDate="2019-02-17T12:00:48.190" Score="0" Body="For questions related to supervised learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:48.010" LastActivityDate="2019-02-18T23:33:48.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10634" PostTypeId="1" CreationDate="2019-02-17T12:29:10.860" Score="5" ViewCount="113" Body="&lt;p&gt;I have a system (like a bank) that people (customers) are entered into the systems by a Poisson process, so the time between the arrival of people (two consecutive customers) will be a random variable. The state of the problem is related to just the system (bank), and the action, made inside the system, can be e.g. offering the customer promotion or not (just based on the state of the system, not the status of customers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To model the problem through RL, 1) it is possible to discretize time horizon into very short time interval (for example 5 minutes as a stage) such that in each time interval, just a single customer enter to our system. On the other hand, 2) it is possible that stages are defined as the time when a customer enters our system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is the second approach an semi-MDP (SMDP)? If I want to solve it with RL, should I use &lt;em&gt;hierarchical RL&lt;/em&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the first approach, if a customer enters in a time interval, it is easy to update the Q values. However, what should we do, if we are in state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and take action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, but no customer enters our system, so we do not receive any reward for the pair of &lt;span class=&quot;math-container&quot;&gt;$(S, A)$&lt;/span&gt;? There would be no difference if we would take action &lt;span class=&quot;math-container&quot;&gt;$A_{1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{2}$&lt;/span&gt;, and so on. This can happen for several consecutive time intervals. I think it is more challenging when we consider eligibility traces.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2019-02-18T18:11:26.787" LastActivityDate="2019-03-20T20:01:13.420" Title="Should I model my problem as a semi-MDP?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;semi-mdp&gt;&lt;hierarchical-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10637" PostTypeId="2" ParentId="10625" CreationDate="2019-02-17T17:07:13.017" Score="2" Body="&lt;p&gt;I understand your question as: &quot;How did the author select the number of neurons in their hidden layer?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of neurons in the hidden layer is how you can control the complexity of the function you are trying to generate to map the inputs to an output. The more neurons in the hidden layer the more complex the function thus you can capture more intricate decision barriers. However, the more complex function is harder to optimize and will lead to lower performance scores. The goal here is to find the right tradeoff to maximize your performance. You can tune the number of hidden neurons as a hyper-parameter using cross-validation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There isn't any formula to determine the number of neurons you will need, however you can get an intuition based on the number of inputs and outputs you will have. Generally, you want more hidden neurons than input and output neurons. Since most people are programmers who are writing neural networks, we are used to working with units in &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt;. Thus, 16 is chosen over 10, and 32 would be chosen over 28.&lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2019-02-17T17:07:13.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10641" PostTypeId="1" CreationDate="2019-02-17T19:37:42.553" Score="4" ViewCount="403" Body="&lt;p&gt;I'm working on my own implementation of NEAT algorithm based on the original 2002 paper called &quot;&lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.gecco02_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Reinforcement Learning through Evolving Neural Network Topologies&lt;/a&gt;&quot; (by Kenneth O. Stanley and Risto Miikkulainen). The way the algorithm is designed it may generate loops in connection of hidden layer. Which obviously will cause difficulties in calculating the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have searched and came across two types of approaches. One set like this &lt;a href=&quot;https://ai.stackexchange.com/q/6231/2444&quot;&gt;example&lt;/a&gt; claim that the value should be calculated like a time series usually seen in RNNs and the circular nodes should use &quot;old&quot; values as their &quot;current&quot; output. But, this seems wrong since the training data is not always ordered and the previous value has nothing to do with current one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second group like this &lt;a href=&quot;https://ai.stackexchange.com/q/6329/2444&quot;&gt;example&lt;/a&gt; claim that the structure should be pruned with some method to avoid loops and cycles. This approach apart from being really expensive to do, is also against the core idea of the algorithm. Deleting connections like this may cause later structural changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I my self have so far tried setting the unknown forward values as 0 and this hides the connection (as whatever weight it has will have no effect on the result) but have failed also for two reasons. One is my networks get big quickly destroying the &quot;smallest network required&quot; idea and also not good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct approach? &lt;/p&gt;&#xA;" OwnerUserId="6522" LastEditorUserId="2444" LastEditDate="2019-02-17T20:57:51.213" LastActivityDate="2019-12-18T03:38:05.850" Title="How do you implement NEAT by taking into account the loops?" Tags="&lt;neat&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10642" PostTypeId="2" ParentId="10603" CreationDate="2019-02-17T21:50:37.897" Score="2" Body="&lt;p&gt;The key I think is teaching the algorythm by providing better data. The only thing an AI can use is the data available for itself. Figuring out whatever it can is not bias, as it's based on objective facts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it knows 98% of Nguyens are interested in X, knowing nothing else about you personally, showing you X might be good. If you consistently click on downvote/not interested, etc. buttons on the site, your personal data will override the default, and you won't see X anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a user you could give better reviews for better results, and as a developer you can give better ways to get this: by logging what you click on, search, and showing &quot;not interested/interested/upvote/downvote/like&quot; etc. buttons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I'm using youtube from different, unlinked machines/browsers, and I get different suggestions from all of these, probably because I've trained the AI with different data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use services with less intrusive data collection, e.g. duckduckgo, bitchute, etc.&lt;/p&gt;&#xA;" OwnerUserId="22418" LastActivityDate="2019-02-17T21:50:37.897" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10643" PostTypeId="1" CreationDate="2019-02-17T21:58:59.010" Score="2" ViewCount="106" Body="&lt;p&gt;Lets say we have a oracle &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; that, given any function &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; and desired output &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;,  can find an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; that causes &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; to output &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; if it exists, or otherwise returns nil. I.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S(F, y) = x \implies F(x) = y$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$S(F, y) = nil \implies !\exists  x \hspace{10px}s.t.\hspace{10px} F(x) = y$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; takes &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; millisecond to run (plus the amount of time it takes to read the input and write the output), regardless of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is allowed to include calls to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; in itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly with this we can solve any NP-Complete problem in constant time (plus the amount of time it takes to read the input and write the output), and in fact we can go further and efficiently solve any optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def IsMin(Cost, MeetsConstraints, x):&#xA;  def HasSmaller(y):&#xA;    return MeetsConstraints(x) and Cost(y) &amp;lt; Cost(x) and y != x&#xA;  return MeetsConstraints(x) and S(HasSmaller, True) == nil&#xA;&#xA;def FindMin(Cost, MeetsConstraints):&#xA;  def Helper(x):&#xA;    return IsMin(Cost, MeetsConstraints, x)&#xA;  return S(Helper, True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which means we can do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def FindSmallestRecurrentNeuralNetworkThatPerfectlyFitsData(Data):&#xA;  def MeetsConstraints(x):&#xA;    return IsRecurrentNeuralNetwork(x) and Error(x, Data) == 0&#xA;  return FindMin(NumParamaters, MeetsConstraints)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And something similar for any other kind of model (random forest, random ensemble of functions, etc.). We can even solve the halting problem with this, which probably means that there is some proof similar to the halting problem proof that shows such an oracle could not exist. Lets assume this exists anyway, as a thought experiment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm not sure how to take it from here to something that achieves endless self improvement. What exactly the &quot;singularity&quot; even means I suppose is tricky to define formally, but I'm interested in any simple definitions, even if they don't quite capture it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sidenote, here is one more function we can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IsEquivalent(G, H):&#xA;    def Helper(x):&#xA;      return G(x) != H(x)&#xA;    return P(Helper, True) == nil&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6378" LastEditorUserId="6378" LastEditDate="2019-07-07T06:12:20.480" LastActivityDate="2019-07-07T06:12:20.480" Title="Is a very powerful oracle sufficient to trigger the AI singularity?" Tags="&lt;algorithm&gt;&lt;optimization&gt;&lt;singularity&gt;" AnswerCount="0" CommentCount="8" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10644" PostTypeId="1" CreationDate="2019-02-17T23:18:35.250" Score="2" ViewCount="369" Body="&lt;p&gt;The &quot;AI Singularity&quot; or &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technological Singularity&lt;/a&gt;&quot; is a vague term that roughly seems to refer to the idea of:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans can design algorithms&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Humans can improve algorithms&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Eventually algorithms we design might end up being as good as humans at designing and improving algorithms&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This might lead to these algorithms designing better versions of themselves, eventually becoming far more intelligent than humans. This improvement would continue to grow at an increasing rate until we reach a &quot;singularity&quot; where an AI is capable of making technological progress at a rate far faster than we could ever imagine&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Also known as an &lt;a href=&quot;https://wiki.lesswrong.com/wiki/Intelligence_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligence Explosion&lt;/a&gt;. This rough idea has been heavily debated as to its feasibility, how fast it'll take (if it does happen), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm not aware of any formal definitions of the concept of &quot;singularity&quot;. Are there any? If not, do we have close approximations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen &lt;a href=&quot;https://en.wikipedia.org/wiki/AIXI&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIXI&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;G&amp;ouml;del machine&lt;/a&gt;, but these both require some &quot;reward signal&quot; &amp;mdash; it is unclear to me what reward signal one should choose to bring about a singularity, or really how those models are even relevant here. Because even if we had an oracle that can solve any formal problem given to it, it's unclear to me how we could use that to cause a singularity to happen &lt;a href=&quot;https://ai.stackexchange.com/questions/10643/is-a-very-powerful-oracle-sufficient-to-trigger-the-ai-singularity?noredirect=1&amp;amp;lq=1&quot;&gt;(see this question for more discussion on that note)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6378" LastEditorUserId="1671" LastEditDate="2019-02-18T23:14:40.500" LastActivityDate="2020-07-25T09:06:30.757" Title="Can we define the AI singularity mathematically?" Tags="&lt;math&gt;&lt;learning-algorithms&gt;&lt;singularity&gt;" AnswerCount="4" CommentCount="8" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10645" PostTypeId="2" ParentId="10644" CreationDate="2019-02-18T00:19:09.093" Score="1" Body="&lt;p&gt;Here is one idea. I'll start with a more specific &quot;mathematical singularity&quot;, defined as an algorithm that can do the following in N hours or less (for all &lt;span class=&quot;math-container&quot;&gt;$N &amp;gt;= 1$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;State equivalent versions (up to notional differences) of all mathematical theorems/conjectures that humans will read and understand in N*20 years after 2018 that can be stated formally in &lt;a href=&quot;http://us.metamath.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metamath&lt;/a&gt; (this in an arbitrary choice, but Metamath is general enough to include quantum logic and extensions of ZFC so it seems like a decent place to start with. Feel free to instead use Coq, Isabelle, Lean, etc. instead if you prefer), assuming those humans never have access to a &quot;mathematical singularity&quot; capable algorithm and their mathematical community continues living and functioning intellectually in a manner similar in capacity to how it did in 2018&lt;/li&gt;&#xA;&lt;li&gt;Of those problems, provide correct proofs (these may not be readable, that's ok) of all of those that will be solved by those humans in N*20 years.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This of course does not fully capture all mathematical progress that humans will make in those years: a big component missing is &quot;readable proofs&quot; and concepts that can't be captured in metamath. But it is something that is theoretically formal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this doesn't include any &quot;continual improvement&quot;, what I am referring to here is simply a threshold such that when an algorithm passes it, I think it is sufficently powerful enough to be considered as &quot;intelligent enough&quot; that it has reached close to singularity levels of intelligence. Feel free to adjust the (20 years) constant in your head to match your  preferred threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not going to accept this answer because it is lacking &quot;continual improvement&quot;, but I brought it up because if we can't figure out how to define it mathematically, perhaps simply having &quot;sufficient criteria&quot; in various domains could be a good start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I suppose that the singularity typically involves an assumption of the development of an intelligence that is superior to human society. This implies that it is capable of at least doing the things that our society does, so there is probably a good argument to be made here that &quot;proof accessibility&quot; and &quot;method teachability&quot; are vital to this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, if we think of the current state of the field of calculus, it has gone from an arcane topic only understood by a few field experts, to now being readily accessible and teachable to high school students. While that didn't require proving any new major mathematical theorems, one could argue that much of our technological progress didn't come until advanced mathematical machinery developed (calculus) became accessible to a wide range of people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was going to make an argument about how &quot;the difference is that computers can learn quicker: they can read through massive proofs very quickly&quot;. But I suppose that depends on the architecture of whatever kind of &quot;thing&quot; is achieving the singularity. I.e., here is a (non-exhaustive) list two possible outcomes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There is only one &quot;mind&quot; that is achieving all of this. In that case, that mind has all the knowledge it needs and it doesn't need to teach anyone to progress further, so this point is sorta irrelevant. However, I can still see an argument for &quot;teachability&quot; if we want to utilize this vast amount of knowledge the AI has gained in human society, if possible.&lt;/li&gt;&#xA;&lt;li&gt;There is a simulated &quot;society&quot; of virtual minds that are interacting with each other, that, together, achieve the mathematical singularity. If a single &quot;mind&quot; in this &quot;society&quot; isn't able to easily use and understand the work done by another mind, then the point of &quot;teachability&quot; is very important to prevent individual minds from having to continually recreate the wheel, so to speak.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Without our biological limitations these digital minds may have very different &quot;teaching&quot; methods, but I think here is the ideal additional requirement for a &quot;mathematical singularity&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;These proofs must be (eventually, perhaps not until spending quite a bit of time) accessible to a graduate mathematician, via proving pdf textbooks (or other similar teaching materials) that cover the same material that human mathematical textbooks would have covered after N*20 years in a way that is accessible to the typical graduate mathematician.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However we have now lost some formality in this: textbooks usually contain lots of exposition and analogies that are difficult to formally measure and may not even be relevant for the AI. Here is an alternate option that is not as good, but still close:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;The algorithm must present its results in a form that can be used by any other algorithm that also can achieve the &quot;mathematical singularity&quot; to &quot;skip ahead&quot; to N*20 years, and then immediately continue progress from there.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However this criteria has a trivial exploit: an algorithm might as well just provide a 'save state' and a 'program' to run that save state. Conceivably any algorithm that can achieve the mathematical singularity is at least capable of executing code, so providing a 'save state' and 'program' passes this criteria without making it at all accessible (The caveat here is if it uses some sort of model of computation that requires special hardware such as quantum computing or black hole computing to prevent slowdown, but that's besides the point)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I prefer this alternative:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;These proofs must be similar in length as the (formalized versions of) proofs the human academic community would have made in those 20*N years&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&quot;length&quot; is tricky here: it is possible to prove a very difficult theorem very succinctly by simply referencing a very powerful lemma. But here is one example metric:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$length(Proof) = lengthInSymbols(Proof)+\sum_{symbol \in Proof} \frac{length(symbol)}{numberOfTimesUsedInOtherProofs(symbol)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &quot;Other Proofs&quot; is the set of all proofs read and understood by humans in those N*20 years, and &quot;symbols&quot; refers to things such as &quot;Green's Theorem&quot; or &quot;&lt;span class=&quot;math-container&quot;&gt;$\in$&lt;/span&gt;&quot;. Hopefully the idea is apparent here: if something is used frequently in many proofs, it is a &quot;common technique&quot; that isn't vital to that proof, and thus doesn't contribute as much to the &quot;length&quot; of that proof. Finding a potentially more suitable metric here seems like a much more tractable problem then defining the mathematical singularity itself and I suspect this is studied elsewhere more, so I'll leave it at this for now.&lt;/p&gt;&#xA;" OwnerUserId="6378" LastEditorUserId="6378" LastEditDate="2019-02-19T22:29:03.790" LastActivityDate="2019-02-19T22:29:03.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10646" PostTypeId="1" CreationDate="2019-02-18T01:02:16.977" Score="2" ViewCount="1225" Body="&lt;p&gt;I am generating images that consist of points where the object's location is where the most overlap of points occurs. &lt;a href=&quot;https://i.stack.imgur.com/7FyHl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7FyHl.png&quot; alt=&quot;Generated image example&quot;&gt;&lt;/a&gt;&#xA;In this example, the object location is (25, 51). I am trying to train a model to just finds the location, I don't care about classification. Additionally, the shape of the overlapping points where the object is located never changes and will always be that shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is a good model for this objective?&lt;/strong&gt; &#xA;Many of the potential models I've been looking at (CNN, YOLO, and RCNN) are more concerned with classification than location. Should I search the image for the overlapping dots, create a bound box around them, then retrieve the boxes coordinates? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks :)&lt;/p&gt;&#xA;" OwnerUserId="22422" LastEditorUserId="22422" LastEditDate="2019-02-18T04:26:01.277" LastActivityDate="2019-02-18T22:45:46.023" Title="Find object location (x, y) in an image" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10647" PostTypeId="2" ParentId="10646" CreationDate="2019-02-18T01:40:46.043" Score="0" Body="&lt;p&gt;Neural networks are not only used for classification but also for &lt;strong&gt;regression&lt;/strong&gt;. It seems that a CNN would be a good solution for this problem with 2 output neurons each of them providing a number within the range of your frame. &lt;/p&gt;&#xA;" OwnerUserId="5925" LastActivityDate="2019-02-18T01:40:46.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10649" PostTypeId="1" CreationDate="2019-02-18T11:53:46.140" Score="5" ViewCount="135" Body="&lt;p&gt;I have some trouble understanding the benefits of Bayesian networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct that the key benefit of the network is that one does not need to use &#xA;chain rule of probability in order to calculate joint distributions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, using the chain rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(A_1, \dots, A_n) = \prod_{i=1}^n (A_i \mid \cap_{j=1}^{i-1} A_j)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;leads to the same result as the following (assuming the nodes are structured by an Bayesian network)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(A_1, \dots, A_n) = \prod_{i=1}^n P(A_i \mid \text{parents}(A_i))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="22433" LastEditorUserId="2444" LastEditDate="2019-02-18T17:07:38.850" LastActivityDate="2019-02-18T17:07:38.850" Title="What are the main benefits of using Bayesian networks?" Tags="&lt;math&gt;&lt;bayes&gt;&lt;bayesian-network&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10650" PostTypeId="1" CreationDate="2019-02-18T12:44:06.557" Score="3" ViewCount="49" Body="&lt;p&gt;I would like to implement a variant of policy iteration that can choose one or more actions in each state. An example would be to heal and move in the game of Doom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parameterizing the power set of all single actions would be one idea, but I was wondering if somebody achieved good results on a similar problem, perhaps by simply defining some lower bound on the output layer and taking all actions with values larger than that bound (i.e. with actions and activation values {shoot=0.2, heal=0.51, move=0.6, jump=0.4} I would choose heal and move if the bound was 0.5)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea was to collect these actions iteratively, i.e. choosing an action from a softmax output based on the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (taking action &quot;healing&quot;) and then constructing and using some temporary state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; to evaluate that state to find another action (e.g. &quot;moving&quot;). This would require some dummy action that is just used to signal the end of that iteration procedure (i.e. choosing action &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt; will not add any other action to the set &lt;span class=&quot;math-container&quot;&gt;$\{ \text{healing}, \text{moving} \}$&lt;/span&gt;, but it will lead to the execution of those two actions and transition to the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22161" LastEditorUserId="2444" LastEditDate="2019-02-20T18:16:46.657" LastActivityDate="2019-02-20T18:16:46.657" Title="Choosing more than one action in a parameterized policy" Tags="&lt;reinforcement-learning&gt;&lt;policy-iteration&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10651" PostTypeId="1" CreationDate="2019-02-18T14:03:44.797" Score="1" ViewCount="115" Body="&lt;p&gt;I asked a question a while ago here and since then I've been solving the issues within my code but I have just one question... This is the formula for updating the Q-Matrix in Q-Learning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_t, a_t) = Q(s_t, a_t) + \alpha \times (R+Q(s_{t+1}, max_a)-Q(s_t, a_t))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I saw a Q-Learning example that uses a different formula, which I'm applying to my own problem and I'm getting good results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_t, a_t) = R(s_t,a_t) + \alpha \times Q(s_{t+1}, max_a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this valid?&lt;/p&gt;&#xA;" OwnerUserId="22088" LastEditorUserId="1641" LastEditDate="2019-02-18T20:46:40.370" LastActivityDate="2019-02-18T20:46:40.370" Title="Is there more than one Q-matrix update formula?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10652" PostTypeId="2" ParentId="10651" CreationDate="2019-02-18T14:31:30.960" Score="3" Body="&lt;p&gt;No, your second statement does not correctly implement the Q-learning update rule, which the first statement correctly implements.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-02-18T14:31:30.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10653" PostTypeId="2" ParentId="10651" CreationDate="2019-02-18T14:32:33.270" Score="1" Body="&lt;p&gt;Your second code snippet is equivalent to this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{k+1}(s,a)  \leftarrow r + \alpha \text{max}_{a'} Q_k(s', a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks like a simplified Value Iteration update to me, where you have incorrectly switched &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; (the learning rate) for &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (the discount rate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full Value Iteration update based on action values looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{k+1}(s,a)  \leftarrow \sum_{r,s'} p(r,s'|s,a)(r + \gamma \text{max}_{a'} Q_k(s', a'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is almost the same as your equation when you have a deterministic environment (so you can directly predict single values &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s, a$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, it will sort of work with certain assumptions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You want a specific discount rate, or don't particularly care about predicting values, just finding a close-to-optimal policy&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The environment is deterministic&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The further away you are from those assumptions, the worse fit the simpler update method will be to your problem. It is definitely not Q-learning either way.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-02-18T14:32:33.270" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10654" PostTypeId="5" CreationDate="2019-02-18T14:43:44.230" Score="0" Body="&lt;p&gt;REINFORCE was introduced in 1988 and 1992 by Ronald J. Williams, respectively, in the papers &quot;&lt;em&gt;Toward a theory of reinforcement-learning connectionist systems&lt;/em&gt;&quot; and &quot;&lt;a href=&quot;http://www-anw.cs.umass.edu/~barto/courses/cs687/williams92simple.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning&lt;/em&gt;&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:59.083" LastActivityDate="2019-02-18T23:33:59.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10655" PostTypeId="4" CreationDate="2019-02-18T14:43:44.230" Score="0" Body="For questions related to the REINFORCE algorithm (or update rule), which is a policy gradient algorithm, that is, an algorithm which estimates the policy directly (that is, without first estimating any value function)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:35:56.300" LastActivityDate="2019-02-18T23:35:56.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10656" PostTypeId="5" CreationDate="2019-02-18T14:54:49.657" Score="0" Body="&lt;p&gt;&quot;&lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt;&quot; (by Andrew Barto and Richard S. Sutton) is often considered or cited as the most comprehensive introductory manual to the field of RL, by two of the greatest contributors to the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two editions have been published so far. &lt;a href=&quot;https://mitpress.mit.edu/books/reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;The first edition&lt;/a&gt; was published in 1998 and &lt;a href=&quot;https://mitpress.mit.edu/books/reinforcement-learning-second-edition&quot; rel=&quot;nofollow noreferrer&quot;&gt;the second&lt;/a&gt; in 2018. You can find some material related to this book (including some drafts) at the following URL: &lt;a href=&quot;http://incompleteideas.net/book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://incompleteideas.net/book/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:24.207" LastActivityDate="2019-02-18T23:33:24.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10657" PostTypeId="4" CreationDate="2019-02-18T14:54:49.657" Score="0" Body="For questions related to the book &quot;Reinforcement Learning: An Introduction&quot; (by Andrew Barto and Richard S. Sutton)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:08.780" LastActivityDate="2019-02-18T23:33:08.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10658" PostTypeId="1" AcceptedAnswerId="11357" CreationDate="2019-02-18T15:49:40.563" Score="7" ViewCount="182" Body="&lt;p&gt;In a neural network, the number of neurons in the hidden layer corresponds to the complexity of the model generated to map the inputs to output(s).  More neurons creates a more complex function (and thus the ability to model more nuanced decision barriers) than a hidden layer with less nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what of the hidden layers?  What do more hidden layers correspond to in terms of the model generated?&lt;/p&gt;&#xA;" OwnerUserId="22424" LastActivityDate="2019-03-21T20:34:32.980" Title="To what does the number of hidden layers in a neural network correspond?" Tags="&lt;neural-networks&gt;&lt;models&gt;&lt;hidden-layers&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10659" PostTypeId="2" ParentId="10658" CreationDate="2019-02-18T15:56:04.677" Score="1" Body="&lt;p&gt;More hidden layers will just escalate the possibilities amount the neurons, including the solutions from the previous hidden layers. (I will edit this once I am at home and provide you with a good link I found some time ago)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile maybe this will help you &lt;a href=&quot;https://stats.stackexchange.com/questions/63152/what-does-the-hidden-layer-in-a-neural-network-compute&quot;&gt;https://stats.stackexchange.com/questions/63152/what-does-the-hidden-layer-in-a-neural-network-compute&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22439" LastActivityDate="2019-02-18T15:56:04.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10660" PostTypeId="1" AcceptedAnswerId="10662" CreationDate="2019-02-18T16:18:35.610" Score="1" ViewCount="89" Body="&lt;p&gt;Hello I am new to reinforcement learning and robotics. So far I have an understanding of the concept on 2D world. You can make agent move one step in one direction. However, how do you define movement action of a robot arm? I am a bit lost over here. Any useful links or keywords would be very appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="22442" LastActivityDate="2019-02-18T17:12:22.560" Title="Robot Arm Deep Q Learning Actions" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;robotics&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10661" PostTypeId="2" ParentId="10634" CreationDate="2019-02-18T16:34:28.357" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;To model the problem through RL, &lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;it is possible to discretize time horizon into very short time interval (for example 5 minutes as a stage) such that in each time interval, just a single customer enter to our system.  &lt;/li&gt;&#xA;  &lt;li&gt;On the other hand, it is possible that stages are defined as the time when a customer enters our system.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) Is the second approach an SMDP and if I want to solve it with RL, I should use Hierarchical RL?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The second approach as you described it does not sound like an SMDP to me... it sounds to me like a regular MDP where the discrete time steps in the MDP do not have any meaningful connection anymore to real-world time in the real-world problem being modelled. &lt;strong&gt;This is the approach I would recommend taking if long periods of &quot;inactivity&quot; in between customers are irrelevant&lt;/strong&gt; and/or &lt;strong&gt;if you do not expect your actions to have any influence on the duration of those periods of inactivity&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If you do expect your actions to have meaningful influence on the duration of inactivity, I would not recommend this approach.&lt;/strong&gt; For example, your actions may make a customer angry and reduce their likelihood of quickly returning. Or your actions may indirectly affect the likelihood of other customers visiting due to interaction between your population of customers, they may or may not recommend your bank to their friends based on your actions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2) In the first approach, if a customer enters in a time interval, it is easy to update &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values. However, what should we do, if we are in state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and take action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; but, no customer enter our system so we do not receive any reward for the pair of &lt;span class=&quot;math-container&quot;&gt;$(S, A)$&lt;/span&gt; there would be no difference if we would take action &lt;span class=&quot;math-container&quot;&gt;$A_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_2$&lt;/span&gt; and so on. This can happen for several consecutive time interval. I think it is more challenging when we consider eligibility trace.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would actually argue that your first approach can be treated as an SMDP, and that it may be a good idea to do so (personally I prefer the &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370299000521&quot; rel=&quot;nofollow noreferrer&quot;&gt;Options framework&lt;/a&gt;, it's a bit more general than SMDPs, a bit more flexible, and a bit closer to &quot;standard&quot; RL terminology; there are plenty of free versions of that paper available on goole too).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever you encounter a customer, you could select an &lt;em&gt;option&lt;/em&gt; (or &quot;macro-action&quot;) in the SMDP, which executes the one &quot;real&quot; action for your current customer once, and is subsequently forced to automatically select &lt;em&gt;no-op&lt;/em&gt; actions (which do nothing) until the next customer arrives; that is the point in time where your previous option ends, and a new option can be selected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An RL algorithm that operates on the SMDP / hierarchical level, i.e. one that only learns to optimize the selection of options / macro-actions and completely skips over all the forced &lt;em&gt;no-op&lt;/em&gt; actions, would suffice. This is basically the approach described in &lt;a href=&quot;http://proceedings.mlr.press/v28/silver13.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Concurrent Reinforcement Learning from Customer Interactions&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-18T16:34:28.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10662" PostTypeId="2" ParentId="10660" CreationDate="2019-02-18T17:12:22.560" Score="1" Body="&lt;p&gt;It depends a lot on the hardware of your robot arm. &#xA;Assuming that your servos have encoder information, if you have access to servos that have limited control like &quot;rotate left/rotate right&quot; functionality, you can phrase the your action space to be [&quot;move left&quot;, &quot;stop&quot;, &quot;move right&quot;]. In this way you can implement a discrete action space with 3 actions per servo and have an agent learn to move the servos around the space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your servos are connected to each other in an elbow/shoulder configuration, you can have a 9 discrete action setup essentially making a box of cardinal directions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up+Left----------Up-------Up+Right&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Left--------------Stop---------Right&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Down+Left-----Down-----Down+Right&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have 3 or more servos, you can still use the same idea of discrete actions but the number of discrete actions grows by a factor of 3 with each servo as your action space is now the cross product of all of the other servos. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can use a &quot;multi-headed&quot; agent where each head chooses actions for a certain servo but there are pros and cons for both depending on your usecase.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you have more advanced servos like &lt;a href=&quot;http://www.robotis.us/dynamixel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamixels&lt;/a&gt; which have high quality encoders, you'll have access to more advanced controls schemes. For instance, Dynamixels allow you to give actions in encoder space, angle space, and even velocity space. For example, you could give the action of &quot;go to encoder value of 500&quot; or &quot;go to 90 degrees&quot; or &quot;move .5 radians/second&quot;. All of these approaches are useful for certain tasks. For humans controlling the arm using a joystick, the velocity based is the most intuitive and, in my experience, the same is true for RL agents using continuous control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using continuous control, you should normalize all of your action spaces within your agent then &quot;unnormalize (?)&quot; them before giving the actions to your servos.&#xA;For instance if your servo velocity ranges from -3.5 rad/sec to +3.5 rad/sec, have your agent select actions in the range of [-1,1] then multiply by 3.5 to get the velocity.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In either case, one thing that should be noted is that you &lt;strong&gt;give your robotic arm enough time to actually perform the action that the agent selected&lt;/strong&gt;. &lt;strong&gt;If not&lt;/strong&gt; you will see your robot &quot;jitter&quot; back and forth quickly as your agent selects actions randomly. This is bad for a few reasons but most importantly because &lt;strong&gt;it might break your servos&lt;/strong&gt;. To overcome this issue, give each action a little more time to be &quot;actualized&quot; by your robot. This can either come in the form of adding a delay in your code while your servos do &quot;the thing&quot; or by using an &quot;iterations since last action update&quot; counter to get a new action from the agent after a certain number of iterations. Not only is this better for your hardware, this also leads to better exploration of your state space as your agent can move through the state space encountering similar states more frequently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A last thing to be aware of is to set hard coded limits on the servos so that your agent doesn't kill your servos by banging the arm against a table or something. Finding these safe bounds isn't always easy as multiple jointed configurations can have multiple ways of hitting the bounds and it might require a forward dynamics model to find these limits. &#xA;If you have only 2 servos, it should be pretty easy to find though :).&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-02-18T17:12:22.560" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10663" PostTypeId="1" CreationDate="2019-02-18T17:49:40.567" Score="2" ViewCount="136" Body="&lt;p&gt;I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain why updating the gradient directly is necessary? &lt;/p&gt;&#xA;" OwnerUserId="22132" LastEditorUserId="22296" LastEditDate="2019-02-20T03:53:15.887" LastActivityDate="2019-02-27T09:13:36.550" Title="Why is gradient ascent necessary when training Actor Critic agents?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10664" PostTypeId="5" CreationDate="2019-02-18T18:14:23.567" Score="0" Body="&lt;p&gt;For more info, have a look at the paper &quot;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370299000521&quot; rel=&quot;nofollow noreferrer&quot;&gt;Between MDPs and semi-MDPs: A framework for Temporal Abstraction in Reinforcement Learning&lt;/a&gt;&quot; (by Richard S.Sutton, Doina Precup and Satinder Singh).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:51.983" LastActivityDate="2019-02-18T23:33:51.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10665" PostTypeId="4" CreationDate="2019-02-18T18:14:23.567" Score="0" Body="For questions related to semi-Markov Decision Processes (SMDP)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:44.720" LastActivityDate="2019-02-18T23:33:44.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10666" PostTypeId="5" CreationDate="2019-02-18T18:18:51.050" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:24:44.173" LastActivityDate="2020-11-23T13:24:44.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10667" PostTypeId="4" CreationDate="2019-02-18T18:18:51.050" Score="0" Body="For questions related to the concept of return in reinforcement learning, which is defined as the future cumulative (discounted) reward or, in simple words, the reward in the long run." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:24:44.173" LastActivityDate="2020-11-23T13:24:44.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10668" PostTypeId="5" CreationDate="2019-02-18T18:20:49.890" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-25T10:59:31.767" LastActivityDate="2020-11-25T10:59:31.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10669" PostTypeId="4" CreationDate="2019-02-18T18:20:49.890" Score="0" Body="For questions about OpenAI's gym library, which provides a set of APIs to access different types of environments to train reinforcement learning agents." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-25T10:59:31.767" LastActivityDate="2020-11-25T10:59:31.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10670" PostTypeId="5" CreationDate="2019-02-18T18:22:21.310" Score="0" Body="&lt;p&gt;Have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Temporal_difference_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Temporal_difference_learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:19.890" LastActivityDate="2019-02-18T23:33:19.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10671" PostTypeId="4" CreationDate="2019-02-18T18:22:21.310" Score="0" Body="For questions related to the temporal-difference reinforcement learning (RL) algorithms, which is a class of model-free (that is, they do not use the transition and reward function of the MDP) RL algorithms which learn by bootstrapping from the current estimate of the value function (that is, they use one estimate to update another estimate)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-11T22:41:23.153" LastActivityDate="2019-07-11T22:41:23.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10672" PostTypeId="5" CreationDate="2019-02-18T18:23:04.700" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-18T18:23:04.700" LastActivityDate="2019-02-18T18:23:04.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10673" PostTypeId="4" CreationDate="2019-02-18T18:23:04.700" Score="0" Body="For questions related to the Monte Carlo methods in reinforcement learning and other AI sub-fields.  (&quot;Monte Carlo&quot; refers to random sampling of the search space.)" OwnerUserId="2444" LastEditorUserId="1671" LastEditDate="2019-02-18T23:35:49.063" LastActivityDate="2019-02-18T23:35:49.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10674" PostTypeId="1" CreationDate="2019-02-18T18:26:51.483" Score="1" ViewCount="24" Body="&lt;p&gt;I am toying around with creating a probability of win calculator for proposals that we do. the information on each proposal is housed in our corporate SharePoint (which I am the admin) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to pull directly from SharePoint as the data source rather than have to export to xls then upload each time the data updates?&lt;/p&gt;&#xA;" OwnerUserId="22446" LastActivityDate="2019-02-18T18:26:51.483" Title="Azure ML studio pull directly from sharepoint" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10675" PostTypeId="1" AcceptedAnswerId="11041" CreationDate="2019-02-18T18:47:52.793" Score="3" ViewCount="154" Body="&lt;p&gt;In some newer robotics literature, the term system identification is used in a certain meaning. The idea is not to use a fixed model, but to create the model on the fly. So it is equal to a model-free system identification. Perhaps a short remark for all, who doesn't know what the idea is. System identification means, to create a prediction model, better known as a forward numerical simulation. The model takes the input and calculates the outcome. It's not exactly the same like a physics engine, but both are operating with a model in the loop which is generating the output in realtime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what is policy learning? Somewhere, I've read that policy learning is equal to online system identification. Is that correct? And if yes, then it doesn't make much sense, because reinforcement learning has the goal to learn a policy. A policy is something which controls the robot. But if the aim is to do system identification, than the policy is equal to the prediction model. Perhaps somebody can lower the confusion about the different terms ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt; Q-learning is a good example for reinforcement learning. The idea is to construct a q-table and this table controls the robot movements. But, if online-system-identification is equal to policy learning and this is equal to q-learning, then the q-table doesn't contains the servo signals for the robot, but it provides only the prediction of the system. That means, the q-table is equal to a box2d physics engine which can say, what x/y coordinates the robot will have. This kind of interpretation doesn't make much sense. Or does it make sense and the definition of a policy is quite different?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorDisplayName="user11571" LastEditDate="2019-02-18T18:56:06.737" LastActivityDate="2019-03-11T19:23:14.953" Title="Is policy learning and online system identification the same?" Tags="&lt;control-problem&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10676" PostTypeId="5" CreationDate="2019-02-18T21:36:59.347" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-18T21:36:59.347" LastActivityDate="2019-02-18T21:36:59.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10677" PostTypeId="4" CreationDate="2019-02-18T21:36:59.347" Score="0" Body="For questions related to the book &quot;Artificial Intelligence: A Modern Approach&quot; by Peter Norvig and Stuart J. Russell." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:33:34.180" LastActivityDate="2019-02-18T23:33:34.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10678" PostTypeId="5" CreationDate="2019-02-18T21:37:29.817" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-18T21:37:29.817" LastActivityDate="2019-02-18T21:37:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10679" PostTypeId="4" CreationDate="2019-02-18T21:37:29.817" Score="0" Body="For questions related to deep reinforcement learning (DRL), that is, RL combined with deep learning. More precisely, deep neural networks are used to represent e.g. value functions or policies." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:49:11.817" LastActivityDate="2019-02-21T01:49:11.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10680" PostTypeId="5" CreationDate="2019-02-18T21:38:14.250" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-18T21:38:14.250" LastActivityDate="2019-02-18T21:38:14.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10681" PostTypeId="4" CreationDate="2019-02-18T21:38:14.250" Score="0" Body="For questions related to policies (as defined in reinforcement learning or other AI sub-fields)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-18T23:34:13.893" LastActivityDate="2019-02-18T23:34:13.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10682" PostTypeId="1" CreationDate="2019-02-18T22:10:39.050" Score="3" ViewCount="143" Body="&lt;p&gt;Fuzzy logic is typically used in control theory and engineering applications, but is it connected fundamentally to classification systems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once I have a trained neural network (multiple inputs, one output), I have a nonlinear function that will turn a set of inputs  into a number that will estimate how close my set of given inputs are to the trained set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my output number characterizes &quot;closeness&quot; to the training set as a continuous number, isn't this kind of inherently some sort of fuzzy classifier? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a deep connection here in the logic, or am I missing something? &lt;/p&gt;&#xA;" OwnerUserId="20685" LastActivityDate="2020-10-08T08:08:06.970" Title="Is fuzzy logic connected to neural networks?" Tags="&lt;neural-networks&gt;&lt;fuzzy-logic&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10683" PostTypeId="2" ParentId="10658" CreationDate="2019-02-18T22:40:24.617" Score="1" Body="&lt;p&gt;It was proven a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal approximation theorem&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More layers can't improve something that can already do &quot;everything&quot;. But adding more layers reduces the number of necessary neurons, and reduces computing power needed for the network as well.&lt;/p&gt;&#xA;" OwnerUserId="22418" LastActivityDate="2019-02-18T22:40:24.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10684" PostTypeId="2" ParentId="10646" CreationDate="2019-02-18T22:45:46.023" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is a good model for this objective?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I will try to give another perspective: Solve it without machine learning model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is try to find the most overlapping point. If the image above is image that you used in your case, you can solve it directly by applying some computer vision algorithms.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to create some &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_image&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary image&lt;/a&gt; based on the color of the dots. if you are not sure with the available colors on your image, you can list of pixel colors that not black and white uniquely at first. So if there are four colors you need to generate four different binary image. Create a simple condition or a complex one, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if pixel[i,j]=red then&#xA;    pixel[i,j]=white&#xA;else&#xA;    pixel[i,j]=black&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Get its location by searching &quot;the white&quot; over your image or &lt;a href=&quot;https://www.learnopencv.com/blob-detection-using-opencv-python-c/&quot; rel=&quot;nofollow noreferrer&quot;&gt;use blob detection method&lt;/a&gt; (it'll be a little bit tricky if the actual image always have different axis scale). You can save it as a list of coordinate of each color.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;What happen if you can't see the dot fully because it's overlapping with another dot? Find the pattern. In your image, dots appear with certain pattern. If you can find two consecutive dots horizontally and vertically, you can predict the position of all your dots.&lt;/li&gt;&#xA;&lt;li&gt;Find the most overlapping position from your list. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The result may more accurate than using machine learning model&lt;/li&gt;&#xA;&lt;li&gt;Faster, you don't need to train it first&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Finding dot's location in image with different axis will be difficult, but it's still solvable&lt;/li&gt;&#xA;&lt;li&gt;It'll be difficult to predict the pattern if many dots are missing because overlapped by other dots&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-02-18T22:45:46.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10685" PostTypeId="2" ParentId="1885" CreationDate="2019-02-19T00:32:00.383" Score="3" Body="&lt;p&gt;I see two main issues with this suggestion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One: digital circuits take up a lot less space, and they're easier to design, so you can put together a bigger system this way. (not to mention connecting separate chips within a system) This is mainly because in digital circuits your tolerances can be a lot loose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bigger one is: we still don't know how neurons work. Artificial neural networks somewhat resemble the natural one, but they behave differently. There are various ion channels, there are electric signals, and with these neurons stimulate each other, and if one's threshold is reached, it fires a spike. When it's reached again soon, you can see a burst in the signal. As far as I know, researchers don't yet know what function you need to implement to simulate it. The closest ANN is the spiking neural network, but it's not very useful in practice.&lt;/p&gt;&#xA;" OwnerUserId="22418" LastActivityDate="2019-02-19T00:32:00.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10686" PostTypeId="1" CreationDate="2019-02-19T02:50:17.833" Score="2" ViewCount="90" Body="&lt;p&gt;I have a neural network that is already trained to predict two continuous outputs from a set of 7 continuous features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to apply the network to predict one of the input features, given other 6 features and the two outputs?&lt;/p&gt;&#xA;" OwnerUserId="22452" OwnerDisplayName="Abdallah Atef" LastEditorUserId="22296" LastEditDate="2019-02-19T04:54:28.690" LastActivityDate="2021-03-11T01:01:30.133" Title="Is it possible to use a trained neural network to predict a feature, given other features and output?" Tags="&lt;matlab&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10687" PostTypeId="5" CreationDate="2019-02-19T08:10:18.457" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:10:18.457" LastActivityDate="2019-02-19T08:10:18.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10688" PostTypeId="4" CreationDate="2019-02-19T08:10:18.457" Score="0" Body="For questions about model-based reinforcement learning methods (or algorithms). An example of a model-based algorithm is Dyna-Q, which estimates a model of the environment (i.e. the transition function of the associated Markov decision process)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T18:50:29.887" LastActivityDate="2020-10-02T18:50:29.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10689" PostTypeId="5" CreationDate="2019-02-19T08:11:04.100" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:11:04.100" LastActivityDate="2019-02-19T08:11:04.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10690" PostTypeId="4" CreationDate="2019-02-19T08:11:04.100" Score="0" Body="For questions about model-free reinforcement learning methods (or algorithms). An example of a model-free algorithm is Q-learning, which does not use the transition function (i.e. the model) of the environment (or Markov decision process)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T18:48:45.357" LastActivityDate="2020-10-02T18:48:45.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10691" PostTypeId="5" CreationDate="2019-02-19T08:11:52.783" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:11:52.783" LastActivityDate="2019-02-19T08:11:52.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10692" PostTypeId="4" CreationDate="2019-02-19T08:11:52.783" Score="0" Body="For questions related to notation (in general)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:14:23.447" LastActivityDate="2019-02-19T21:14:23.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10693" PostTypeId="5" CreationDate="2019-02-19T08:12:55.690" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:12:55.690" LastActivityDate="2019-02-19T08:12:55.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10694" PostTypeId="4" CreationDate="2019-02-19T08:12:55.690" Score="0" Body="For questions related to the reinforcement learning technique called &quot;eligibility traces&quot;, which combines temporal-difference and Monte Carlo methods." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:13:50.350" LastActivityDate="2019-02-19T21:13:50.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10695" PostTypeId="5" CreationDate="2019-02-19T08:13:36.237" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:13:36.237" LastActivityDate="2019-02-19T08:13:36.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10696" PostTypeId="4" CreationDate="2019-02-19T08:13:36.237" Score="0" Body="For questions related to the concept of environment in reinforcement learning and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-13T21:26:50.157" LastActivityDate="2019-07-13T21:26:50.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10697" PostTypeId="5" CreationDate="2019-02-19T08:14:23.843" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:14:23.843" LastActivityDate="2019-02-19T08:14:23.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10698" PostTypeId="4" CreationDate="2019-02-19T08:14:23.843" Score="0" Body="For questions related to the unsupervised learning technique called &quot;clustering&quot;." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:14:26.750" LastActivityDate="2019-02-19T21:14:26.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10699" PostTypeId="5" CreationDate="2019-02-19T08:14:55.650" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:14:55.650" LastActivityDate="2019-02-19T08:14:55.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10700" PostTypeId="4" CreationDate="2019-02-19T08:14:55.650" Score="0" Body="For questions related to the mathematical concept of &quot;expectation&quot; or &quot;expected value&quot;." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:13:43.907" LastActivityDate="2019-02-19T21:13:43.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10701" PostTypeId="5" CreationDate="2019-02-19T08:15:30.077" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:15:30.077" LastActivityDate="2019-02-19T08:15:30.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10702" PostTypeId="4" CreationDate="2019-02-19T08:15:30.077" Score="0" Body="For questions related to the Q function, a.k.a. state-action value function, a.k.a. the &quot;quality&quot; function (as defined in reinforcement learning), which is used in algorithms such as Q-learning or SARSA." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:14:11.117" LastActivityDate="2019-02-19T21:14:11.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10703" PostTypeId="5" CreationDate="2019-02-19T08:17:26.790" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:17:26.790" LastActivityDate="2019-02-19T08:17:26.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10704" PostTypeId="4" CreationDate="2019-02-19T08:17:26.790" Score="0" Body="For questions related to the V function, a.k.a. state value function (as defined in reinforcement learning), which is used in algorithms such as value iteration." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:13:02.500" LastActivityDate="2019-02-19T21:13:02.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10705" PostTypeId="5" CreationDate="2019-02-19T08:18:46.353" Score="0" Body="&lt;p&gt;See this question &lt;a href=&quot;https://ai.stackexchange.com/q/10623/2444&quot;&gt;What is self-supervision in machine learning?&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:12:53.390" LastActivityDate="2019-02-19T21:12:53.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10706" PostTypeId="4" CreationDate="2019-02-19T08:18:46.353" Score="0" Body="For questions related to self-supervised learning (SSL), which typically refers to techniques that automatically generate the supervisory learning signal. SSL can be used for representation learning, so it can be useful for transfer learning too. Some people consider SSL a sub-field of unsupervised learning given that many (if not all) SSL techniques do not require a human to manually annotate the inputs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T12:59:16.313" LastActivityDate="2020-11-20T12:59:16.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10707" PostTypeId="5" CreationDate="2019-02-19T08:23:42.633" Score="0" Body="&lt;p&gt;On-policy RL algorithms use their current approximation of the policy they attempt to estimate in order to interact with the environment (to gain experience and further update their approximation). An example of an on-policy algorithm is SARSA.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:14:39.063" LastActivityDate="2019-02-19T21:14:39.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10708" PostTypeId="4" CreationDate="2019-02-19T08:23:42.633" Score="0" Body="For questions related to the &quot;on-policy&quot; reinforcement learning algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:13:58.987" LastActivityDate="2019-02-19T21:13:58.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10709" PostTypeId="5" CreationDate="2019-02-19T08:26:21.300" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-11T10:58:14.490" LastActivityDate="2019-08-11T10:58:14.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10710" PostTypeId="4" CreationDate="2019-02-19T08:26:21.300" Score="0" Body="For questions related to off-policy reinforcement learning algorithms, which estimate a policy (the target policy) while using another policy (the behavior policy), during the learning process, which ensures that all states are sufficiently explored. An example of an off-policy algorithm is Q-learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-11T10:58:14.490" LastActivityDate="2019-08-11T10:58:14.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10711" PostTypeId="5" CreationDate="2019-02-19T08:28:19.873" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:28:19.873" LastActivityDate="2019-02-19T08:28:19.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10712" PostTypeId="4" CreationDate="2019-02-19T08:28:19.873" Score="0" Body="For questions related to Bayesian networks, which are e.g. used to study causality (or causation) in AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:13:55.487" LastActivityDate="2019-02-19T21:13:55.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10713" PostTypeId="5" CreationDate="2019-02-19T08:29:20.817" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:29:20.817" LastActivityDate="2019-02-19T08:29:20.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10714" PostTypeId="4" CreationDate="2019-02-19T08:29:20.817" Score="0" Body="For questions related to the Markov property or Markov assumption (that is, the assumption that the &quot;future is independent of the past, given the present&quot;), which underlies e.g. most reinforcement learning algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-31T20:14:00.243" LastActivityDate="2019-10-31T20:14:00.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10715" PostTypeId="5" CreationDate="2019-02-19T08:30:34.303" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-19T08:30:34.303" LastActivityDate="2019-02-19T08:30:34.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10716" PostTypeId="4" CreationDate="2019-02-19T08:30:34.303" Score="0" Body="For questions related to the concept of &quot;optimal policy&quot; in reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-19T21:14:33.273" LastActivityDate="2019-02-19T21:14:33.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10717" PostTypeId="5" CreationDate="2019-02-19T08:31:38.257" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-03-26T10:43:01.333" LastActivityDate="2021-03-26T10:43:01.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10718" PostTypeId="4" CreationDate="2019-02-19T08:31:38.257" Score="0" Body="For questions related to the concept of a stochastic policy (as defined in reinforcement learning), which is a function from a state to a probability distribution over actions (from that state)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-03-26T10:43:01.333" LastActivityDate="2021-03-26T10:43:01.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10719" PostTypeId="1" AcceptedAnswerId="10720" CreationDate="2019-02-19T09:14:56.147" Score="3" ViewCount="557" Body="&lt;p&gt;I am learning Deep RL following this tutorial: &lt;a href=&quot;https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand everything but one detail:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image shows the difference between a classic Q learning table and a DNN.&#xA;It states that a Q table needs a state-action pair as input and outputs the corresponding Q value whereas a Deep Q network needs the state as feature input and outputs the Q value for each action that can be made in that state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But shouldn't the state AND the action together be the input to the network and the network just outputs a single Q value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A9ulk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A9ulk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22460" LastEditorUserId="2444" LastEditDate="2021-02-05T14:28:27.390" LastActivityDate="2021-02-05T14:28:27.390" Title="Why does Deep Q Network outputs multiple Q values?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10720" PostTypeId="2" ParentId="10719" CreationDate="2019-02-19T09:27:11.333" Score="4" Body="&lt;p&gt;I think this was just a &quot;clever&quot; design choice. You can actually design a neural network (NN), to represent your Q function, which receives as input the state and an action and outputs the corresponding Q value. However, to obtain &lt;span class=&quot;math-container&quot;&gt;$\max_aQ(s', a)$&lt;/span&gt; (which is a term of the update rule of the &lt;a href=&quot;https://ai.stackexchange.com/a/10560/2444&quot;&gt;Q-learning algorithm&lt;/a&gt;) you would need a &quot;forward pass&quot; of this network for each possible action from &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. By having a NN that outputs the Q value for each possible action from a given &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, you will just need one forward pass of the NN to obtain &lt;span class=&quot;math-container&quot;&gt;$\max_aQ(s', a)$&lt;/span&gt;, that is, you pick the highest Q value among the outputs of your NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1708.05866.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Brief Survey of Deep Reinforcement Learning&lt;/a&gt; (by Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage and Anil Anthony Bharath), at page 7, section &quot;Value functions&quot; (and subsection &quot;Function Approximation and the DQN&quot;), it's written &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It was designed such that the final fully connected layer outputs &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s,\cdot)$&lt;/span&gt; for all action values in a discrete set of actions — in this case, the various directions of the joystick and the fire button. This not only enables the best action, &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}_a Q^\pi(s, a)$&lt;/span&gt;, to be chosen after a single forward pass of the network, but also allows the network to more easily encode action-independent knowledge in the lower, convolutional layers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-20T11:32:01.903" LastActivityDate="2019-02-20T11:32:01.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10722" PostTypeId="2" ParentId="10682" CreationDate="2019-02-19T10:04:53.487" Score="2" Body="&lt;p&gt;They are unrelated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a possibility of interpreting fuzzy values as probabilities, but strictly speaking they are different: fuzzy values are &lt;em&gt;vague&lt;/em&gt;, while probabilities reflect &lt;em&gt;likelihood&lt;/em&gt; (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia entry for Fuzzy Logic&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While rolling a particular number on a six-side die has a probability of &lt;span class=&quot;math-container&quot;&gt;$1 \over 6$&lt;/span&gt;, a roll can actually only ever have one outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fuzzy value &quot;quite old&quot; can simultaneously be member of a number of fuzzy sets with different degrees of membership, eg &quot;young&quot; with 0.001, &quot;adolescent&quot; with 0.1, &quot;old&quot; with 0.4, &quot;ancient&quot; with 0.7. Unless it is &quot;defuzzified&quot;, it is simultaneously contained in all the sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defuzzyfication is a way of interpreting the result of a series of fuzzy operations and finding the set that best matches, but it is not a clearly defined process such as picking a random number according to a set of probabilities (or rolling the die).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure that the sum of all fuzzy set membership values of any given fuzzy value has to add up to 1.0; whereas this condition has to hold for probabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT: to clarify - probabilities are not a set; I refer here to all possible outcomes of a random event which have a certain probability of being realised. The sum of all possible event probabilities has to be 1.0]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternative interpretation for your application could be the &lt;em&gt;confidence&lt;/em&gt; that the input set is identical to the training set. Which could be a fuzzy value if you wanted to do something else with it, eg by combining it with other fuzzy variables.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-02-19T11:04:28.217" LastActivityDate="2019-02-19T11:04:28.217" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10723" PostTypeId="1" CreationDate="2019-02-19T10:14:12.223" Score="1" ViewCount="75" Body="&lt;p&gt;Some (stock market) traders have the ability to produce a high percentage of winning trades (80%+, positive return) over years. I had the chance to look into real money trades of two such traders and I also got trading instructions from them for research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the interesting part is that if you strictly follow their rules then you usually end up with more losers than winners on the long run. But after a while you get some kind of subconscious &quot;feeling&quot; for winners which also shows in the results. I assume that this &quot;feeling&quot; is a hidden function which can be modeled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is there work about how to model such &quot;gut feeling&quot; and subconscious knowledge by means of machine learning (especially with little training data sets)? Is there relevant literature about this topic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&lt;/p&gt;&#xA;" OwnerUserId="22465" LastActivityDate="2019-02-21T11:11:45.267" Title="Modelling gut-feeling/subconscious knowledge of stock market traders" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10724" PostTypeId="1" CreationDate="2019-02-19T10:42:00.217" Score="2" ViewCount="118" Body="&lt;p&gt;I'm looking into using PPO implementations like OpenAi's &lt;a href=&quot;https://spinningup.openai.com/en/latest/user/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpinningUp&lt;/a&gt; and &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Baselines&lt;/a&gt;. However, I fear that these implementations require packages which are not available for Windows. So I'm wondering if in general Linux should be used for working with DRL algorithms. Also, I'm not planing on using OpenAI's gym environments. Environments, created with the ml-agents toolkit from Unity and its gym wrapper are going to be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is your opinion on this? Should I rather work on Linux or Windows? And what do you think about alternatives such as: VM on Windows with Ubuntu or Subsystem on Windows with Ubuntu (WSL)?&lt;/p&gt;&#xA;" OwnerUserId="22467" LastEditorUserId="1671" LastEditDate="2019-02-19T20:51:00.887" LastActivityDate="2019-02-19T20:51:00.887" Title="Is Windows a bad choice for DRL?" Tags="&lt;reinforcement-learning&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="4" ClosedDate="2020-11-21T22:08:13.247" ContentLicense="CC BY-SA 4.0" />
  <row Id="10727" PostTypeId="2" ParentId="10723" CreationDate="2019-02-19T11:42:33.470" Score="1" Body="&lt;p&gt;You could perhaps model gut feeling or subconscious bias as a prior in a Bayesian context, and then try to learn from the data how much to modify/moderate the bias in each individual case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there is another issue with the problem you outlined.  We might expect it to be normal to see more losers than winners in the long run: trading is a zero-sum game where the house always takes its cut. The trick to being a successful trading algorithm seems to be to make the losers small (cut them early) and the winners big (let them run).&lt;/p&gt;&#xA;" OwnerUserId="22355" LastActivityDate="2019-02-19T11:42:33.470" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10728" PostTypeId="2" ParentId="10724" CreationDate="2019-02-19T12:51:34.860" Score="2" Body="&lt;p&gt;TLDR Use whatever environment you are most comfortable with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today the choice of operating system is of much less importance than few years ago. Generally speaking, you will have more application choices under Linux, however, some of these choices will be less than spectacular :-) Windows works better with some hardware. Hence, under Linux, it is a very good idea to check the forums before buying or installing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have played with OpenAI gym under Linux for some time and it worked as advertised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early versions of many important pieces of deep learning software get released for Linux only. For instance, PyTorch and Tensorflow were available for Linux at least one year before the release of Windows ports.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-19T12:51:34.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10729" PostTypeId="2" ParentId="4509" CreationDate="2019-02-19T15:11:25.013" Score="1" Body="&lt;p&gt;First, a quick background on me.  I was a pre-med student who graduated undergrad with a Biophysics degree.  After some hard work and smart decision making, I am now a AI/ML software engineer with a Master's in Computer Science (specialty in Machine Learning).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it important to have my own expertise to work in the fields?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, absolutely, but not necessarily in a professional context.  You don't need to have been employed as a machine learning software engineer, but do need to show proficiency with the field.  Which is a great segue to the second part of your question...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, what kinds of subjects do I have to study right now?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Their is no one subject you should give your focus.  Machine learning is a combination of many different fields, and it would not be very efficient to focus on just one before diving into more thorough practice.  Instead, tutorials and practice are the name of the game.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=aircAruvnKk&amp;amp;list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&quot; rel=&quot;nofollow noreferrer&quot;&gt;3Blue1Brown&lt;/a&gt; on Youtube gives great tutorials, especially on neural nets&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.khanacademy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Khan Academy&lt;/a&gt; is a godsend when it comes to math tutorials.  Linear Algebra and Probability/Statistics are the best to start with, I'd say.  But Multivariable Calculus and Differential Equations are ultimately used as well.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.udacity.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Udacity&lt;/a&gt; is a great tutorial site that even offers &quot;nanodegree&quot; programs to give you more hands on experience in artificial intelligence and machine learning.  It is free if you just want to view the videos.  &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAIGym&lt;/a&gt; is a great place to practice Reinforcement Learning&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.kaggle.com/learn/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle&lt;/a&gt; has great tutorials on machine learning and their contests provide great practice with supervised/unsupervised learning.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Complement your development in the theory and mathematical background with hands-on development and practice to achieve the best results.  You mention a specific focus on MDP's, with which the Udacity tutorials and OpenAIGym would both give great practice.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in a Master's Degree, I can't recommend Georgia Tech's Online Master's in Computer Science (&lt;a href=&quot;https://www.omscs.gatech.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OMSCS&lt;/a&gt;) enough.  It is a great education, and (when I was enrolled in 2015) required no GRE and only cost about $8000.00&lt;/p&gt;&#xA;" OwnerUserId="22424" LastActivityDate="2019-02-19T15:11:25.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10731" PostTypeId="2" ParentId="10686" CreationDate="2019-02-19T16:42:16.460" Score="1" Body="&lt;p&gt;From your question, it appears that you would like to use other features in your data to predict one of the features. I am not sure I understood your question clearly, but anyways, either, you would be using the feature you want to predict as the output of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you want to use the output of the network and other features to predict the new output, I think possibly you are trying to use a Recurrent Neural Network based approach, in which the past output is taken into account for future predictions. My personal experience with RNN's is that they are really good at learning such dependencies, as the RNN cells consider the present input, as well as the past output in modelling such tasks. If your problem is a sequence to sequence prediction task, I would definitely suggest trying RNN's (especially LSTMs, ideal for learning long term temporal dependencies) but ultimately, there is No Free Lunch. So try different approaches and see which one works for you best. Good Luck!&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2019-02-19T16:42:16.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10732" PostTypeId="1" AcceptedAnswerId="10733" CreationDate="2019-02-19T19:46:09.030" Score="1" ViewCount="230" Body="&lt;p&gt;I am working through the famous RL textbook by Sutton &amp;amp; Barto. Currently, I am on the &lt;a href=&quot;http://www.incompleteideas.net/book/ebook/node44.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration chapter&lt;/a&gt;. To gain better understanding, I coded up a small example, inspired by &lt;a href=&quot;http://www.incompleteideas.net/book/ebook/node41.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the following&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There is a rat (R) in a grid. Every square is accessible. The goal is to find the cheese (C). However, there is also a trap (T). The game is over whenever the rat either find the cheese or is trapped (these are my terminal states). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8CqfK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8CqfK.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rat can move up, down, left, and right (always by one square). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I modeled the reward as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1 for every step&#xA;5 for finding the cheese&#xA;-5 for getting trapped&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used value iteration for this and it worked out quite nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now I would like to add another cheese to the equation. In order to win the game, the rat has to collect &lt;em&gt;both&lt;/em&gt; cheese pieces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nAlf9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nAlf9.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure how to model this scenario. I don't think it will work when I use both cheese squares and the trap square as terminal states, with rewards for both cheese squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I model this scenario? Should I somehow combine the two cheese states into one?&lt;/p&gt;&#xA;" OwnerUserId="10448" LastEditorUserId="2444" LastEditDate="2019-02-20T18:27:09.527" LastActivityDate="2019-02-20T18:33:53.733" Title="How do I apply the value iteration algorithm when there are two goal states?" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10733" PostTypeId="2" ParentId="10732" CreationDate="2019-02-19T20:30:18.177" Score="3" Body="&lt;p&gt;What you could do is to trigger environment termination when rat either:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;steps into the trap  &lt;/li&gt;&#xA;&lt;li&gt;picks both cheese pieces  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The problem with such setup is that, when the rat picks a single piece, it would move one step to the side, and then it would come back to the same cheese spot so it would keep exploiting the same spot indefinitely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to that would be to simply remove the cheese piece once the rat picks it up, so that it can't exploit it indefinitely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly, another problem would arise which is partial observability: Markov property wouldn't be fulfilled because the current action wouldn't depend on the current state solely, it would be important whether cheese piece was picked before or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to that would be to make environment fully observable. You could accomplish that by expanding the amount of information about your current state. Before, only your position on the grid was important, but now you would also add state features that tell you whether cheese piece at specific position was picked or not. You would basically add a flag for each cheese piece that has value of 1 if piece was picked, or value of 0 if it wasn't. That way you could remove cheese piece it rat picks it, and you would still have full information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this setup would work.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="2444" LastEditDate="2019-02-20T18:33:53.733" LastActivityDate="2019-02-20T18:33:53.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10734" PostTypeId="1" CreationDate="2019-02-20T00:29:52.990" Score="4" ViewCount="96" Body="&lt;p&gt;Does anyone work out ways of relating trained networks by symbolic logic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&#xA;If I train a network on pictures of dogs, and I train a network on pictures of shirts. You could imagine that the simplest way of (without going through the process from scratch), identifying &quot;dog AND shirt&quot; would be to perform an AND operation on the last output of the individual cat &amp;amp; dog neural nets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &quot;dog AND shirt&quot; would amount to AND'ing the output of two nets (Which I believe is described &lt;a href=&quot;https://ai.stackexchange.com/questions/9701/combining-different-trained-neural-networks&quot;&gt;here&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this operation AND could be replaced with a more complicated operation. And in principle I could &quot;train&quot; a network to act as one of these operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance maybe I could figure out the net that describes some changeable output &quot;X&quot; being &quot;on the shirt.&quot; This would be sort of like a &quot;functional&quot; in mathematics (in which we are operating are considering the behavior of a network whos input could be any network).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I can figure out this &quot;functional&quot; then I would be able to use is symbolically and determine queries like &quot;dog on the shirt&quot;? - &quot;cat on the shirt&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like to me there's a lot of sense to turn specific neural networks into more &quot;abstract&quot; objects - and that there would be a lot of power in doing so. &lt;/p&gt;&#xA;" OwnerUserId="20685" LastActivityDate="2019-03-17T22:31:40.493" Title="Symbolic &quot;math&quot; using trained networks" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10735" PostTypeId="2" ParentId="4328" CreationDate="2019-02-20T08:50:11.773" Score="1" Body="&lt;p&gt;In &lt;em&gt;data mining&lt;/em&gt;, we can use &lt;em&gt;machine learning&lt;/em&gt; (ML) (with the help of unsupervised learning algorithms) to recognize patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Pattern recognition&lt;/em&gt; is a process of recognizing patterns such as images or speech. We can recognise patterns using ML. For example, once a neural net is trained, using ML algorithms, it can be used for pattern recognition. Other methods, even ones not related to ML and data mining, can be used for pattern recognition, such as a fully handcrafted pattern recognition system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;data mining is mostly associated with statisticians, &lt;/li&gt;&#xA;&lt;li&gt;ML is mostly associated with computer scientists whereas,&lt;/li&gt;&#xA;&lt;li&gt;pattern recognition is mostly associated with engineers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2019-02-20T09:49:55.310" LastActivityDate="2019-02-20T09:49:55.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10738" PostTypeId="1" CreationDate="2019-02-20T10:06:35.263" Score="3" ViewCount="74" Body="&lt;p&gt;The &lt;a href=&quot;https://www.gov.uk/health-and-social-care/national-health-service&quot; rel=&quot;nofollow noreferrer&quot;&gt;National Health Service (NHS)&lt;/a&gt; wrote down several principles in a document &lt;a href=&quot;https://www.gov.uk/government/publications/code-of-conduct-for-data-driven-health-and-care-technology/initial-code-of-conduct-for-data-driven-health-and-care-technology&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code of conduct for data-driven health and care technology&lt;/a&gt; (updated 18 July 2019). I am concerned with principle 7.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h3&gt;Show what type of algorithm is being developed or deployed, the ethical examination of how the data is used, how its performance will be validated and how it will be integrated into health and care provision&lt;/h3&gt;&#xA;&lt;p&gt;Demonstrate the learning methodology of the algorithm being built.&#xA;Aim to show in a clear and transparent way how outcomes are validated.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But how exactly can outcomes be shown in a clear and transparent way how outcomes are validated?&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="2444" LastEditDate="2020-07-15T12:20:09.457" LastActivityDate="2021-04-11T16:03:45.510" Title="How can it be shown clearly and transparently that the outcomes of data-driven health and care technology are validated?" Tags="&lt;social&gt;&lt;ethics&gt;&lt;healthcare&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10739" PostTypeId="1" CreationDate="2019-02-20T10:42:03.103" Score="1" ViewCount="43" Body="&lt;p&gt;I am training a video prediction model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the loss plots, the model convergences very fast while the final loss is not small enough and the generation is not good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, I have test the &lt;code&gt;lr=1e-04&lt;/code&gt;and &lt;code&gt;lr=1e-05&lt;/code&gt;, the loss plots drop down a little more slowly, but it's still not ideal. But I think &lt;code&gt;lr=1e-05&lt;/code&gt;should be small enough, isn't it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I fix my model or the hyper parameters?&lt;/p&gt;&#xA;" OwnerUserId="22495" LastActivityDate="2019-02-20T10:42:03.103" Title="Why is the learning rate is already very small (1e-05) while the model convergences too fast?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10740" PostTypeId="1" CreationDate="2019-02-20T11:09:53.073" Score="1" ViewCount="28" Body="&lt;p&gt;I am trying to understand if there is any difference in the the interpretation of accuracy and loss on synthetic data vs real data.&lt;/p&gt;&#xA;" OwnerUserId="22497" LastEditorUserId="22498" LastEditDate="2019-02-20T14:58:58.323" LastActivityDate="2019-03-22T16:01:19.213" Title="Loss/accuracy on Synthetic data" Tags="&lt;data-science&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10741" PostTypeId="2" ParentId="10740" CreationDate="2019-02-20T13:10:19.700" Score="1" Body="&lt;p&gt;No, there is no difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you are likely not able to extrapolate results obtained from synthetic data to expect identical or similar results in real life to unless you have &lt;strong&gt;very&lt;/strong&gt; compelling reasons to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without a more specific question, I'm afraid a more specific answer is not possible.&lt;/p&gt;&#xA;" OwnerUserId="22498" LastActivityDate="2019-02-20T13:10:19.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10743" PostTypeId="2" ParentId="10623" CreationDate="2019-02-20T14:30:03.233" Score="5" Body="&lt;p&gt;Self-supervised visual recognition is often applied to representation learning. Here we first learn features on unlabeled data (representation learning), and then learn the real model on features extracted from the labeled data. This especially makes sense when we have a lot of unlabeled data and few labeled data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The features can be learned by solving so called &lt;em&gt;pretext&lt;/em&gt; tasks. Examples of pretext tasks are to predict rotation of a jittered image, to recognize jittered instances of a same image, or to predict spatial relationship of image patches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice overview and interesting results can be found in &lt;a href=&quot;https://arxiv.org/pdf/1901.09005.pdf&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; recent paper.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-20T14:30:03.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10744" PostTypeId="5" CreationDate="2019-02-20T15:59:21.413" Score="0" Body="&lt;p&gt;In reinforcement learning, a deterministic policy is a function from a state to a single action.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:51.420" LastActivityDate="2019-02-21T01:50:51.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10745" PostTypeId="4" CreationDate="2019-02-20T15:59:21.413" Score="0" Body="For questions related to the concept of a &quot;deterministic policy&quot; (as defined in reinforcement learning)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:49:15.783" LastActivityDate="2019-02-21T01:49:15.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10746" PostTypeId="5" CreationDate="2019-02-20T16:02:58.063" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:02:58.063" LastActivityDate="2019-02-20T16:02:58.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10747" PostTypeId="4" CreationDate="2019-02-20T16:02:58.063" Score="0" Body="For questions related to the family of reinforcement learning algorithms denoted by &quot;actor-critic&quot;, where there is an actor (a policy) and a critic (a value function)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:53.990" LastActivityDate="2019-02-21T01:50:53.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10748" PostTypeId="5" CreationDate="2019-02-20T16:03:56.533" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:03:56.533" LastActivityDate="2019-02-20T16:03:56.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10749" PostTypeId="4" CreationDate="2019-02-20T16:03:56.533" Score="0" Body="For questions related to the &quot;experience replay&quot; buffer (as used in the Deep Q Network and similar works)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:00.440" LastActivityDate="2019-02-21T01:50:00.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10750" PostTypeId="5" CreationDate="2019-02-20T16:04:48.730" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;https://pytorch.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:11.540" LastActivityDate="2019-02-21T01:50:11.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10751" PostTypeId="4" CreationDate="2019-02-20T16:04:48.730" Score="0" Body="For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-21T19:00:22.437" LastActivityDate="2020-08-21T19:00:22.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10752" PostTypeId="5" CreationDate="2019-02-20T16:06:42.690" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-07T17:43:33.363" LastActivityDate="2021-02-07T17:43:33.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10753" PostTypeId="4" CreationDate="2019-02-20T16:06:42.690" Score="0" Body="For questions related to the (automated) planning problem, which is the problem of finding a plan, i.e. a sequence of actions to move from an initial state to a goal state or a policy (a function from states to actions), and planning algorithms. There are different ways to define a planning problem (such as PDDL) and solve a planning problem (e.g. GraphPlan). In reinforcement learning, planning consists in finding a policy that solves an MDP." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-07T17:43:33.363" LastActivityDate="2021-02-07T17:43:33.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10754" PostTypeId="5" CreationDate="2019-02-20T16:07:31.203" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:07:31.203" LastActivityDate="2019-02-20T16:07:31.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10755" PostTypeId="4" CreationDate="2019-02-20T16:07:31.203" Score="0" Body="For questions related to the concept of &quot;creativity&quot; in AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:49:45.550" LastActivityDate="2019-02-21T01:49:45.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10756" PostTypeId="5" CreationDate="2019-02-20T16:09:16.607" Score="0" Body="&lt;p&gt;See this question &lt;a href=&quot;https://ai.stackexchange.com/q/7390/2444&quot;&gt;What is the difference between Actor-Critic and Advantage Actor-Critic?&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:49:22.867" LastActivityDate="2019-02-21T01:49:22.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10757" PostTypeId="4" CreationDate="2019-02-20T16:09:16.607" Score="0" Body="For questions related to the advantage actor-critic algorithms (that is, actor-critic algorithms that use the &quot;advantage&quot; function)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:14.447" LastActivityDate="2019-02-21T01:50:14.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10758" PostTypeId="5" CreationDate="2019-02-20T16:10:36.883" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:10:36.883" LastActivityDate="2019-02-20T16:10:36.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10759" PostTypeId="4" CreationDate="2019-02-20T16:10:36.883" Score="0" Body="For questions related to hierarchical reinforcement learning algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:49:26.363" LastActivityDate="2019-02-21T01:49:26.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10760" PostTypeId="5" CreationDate="2019-02-20T16:12:31.893" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:12:31.893" LastActivityDate="2019-02-20T16:12:31.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10761" PostTypeId="4" CreationDate="2019-02-20T16:12:31.893" Score="0" Body="For questions related to the concept of value (or performance, or quality, or utility) function (as defined in reinforcement learning and other AI sub-fields). An example of this type of functions is the Q function (used e.g. in the Q-learning algorithm), also known as the state-action value function, given that $Q: S \times A \rightarrow \mathbb{R}$, where $S$ and $A$ are respectively the set of states and actions of the environment." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-04T10:55:56.977" LastActivityDate="2020-10-04T10:55:56.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10762" PostTypeId="5" CreationDate="2019-02-20T16:16:01.523" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:16:01.523" LastActivityDate="2019-02-20T16:16:01.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10763" PostTypeId="4" CreationDate="2019-02-20T16:16:01.523" Score="0" Body="For questions related to the Atari games, which are often used in reinforcement learning (RL) as standard problems to test new RL algorithms or methods." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:06.377" LastActivityDate="2019-02-21T01:50:06.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10764" PostTypeId="1" CreationDate="2019-02-20T16:17:21.213" Score="2" ViewCount="923" Body="&lt;p&gt;I trained a neural network on the &lt;a href=&quot;https://www.unsw.adfa.edu.au/unsw-canberra-cyber/cybersecurity/ADFA-NB15-Datasets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UNSW-NB15 dataset&lt;/a&gt;, but, during training, I am getting spikes in the loss function. The algorithms see part of this UNSW dataset a single time. The loss function is plotted after every batch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yoGv8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yoGv8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other datasets, I don't experience this problem.  I've tried different optimizers and loss functions, but this problem remains with this dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the &lt;code&gt;fit_generator()&lt;/code&gt; function from Keras. Is there anyone experience this problem using Keras with this function? &lt;/p&gt;&#xA;" OwnerUserId="22514" LastEditorUserId="2444" LastEditDate="2019-12-18T22:26:03.887" LastActivityDate="2020-01-17T23:01:15.123" Title="Why am I getting spikes in the values of the loss function during training?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;keras&gt;&lt;loss-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10765" PostTypeId="5" CreationDate="2019-02-20T16:20:51.463" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:20:51.463" LastActivityDate="2019-02-20T16:20:51.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10766" PostTypeId="4" CreationDate="2019-02-20T16:20:51.463" Score="0" Body="For questions related to calculus (developed, among others, by Newton and Leibniz), in the context of AI (and, in particular, machine learning)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T01:50:26.617" LastActivityDate="2019-02-21T01:50:26.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10767" PostTypeId="5" CreationDate="2019-02-20T16:21:27.183" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-20T16:21:27.183" LastActivityDate="2019-02-20T16:21:27.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10768" PostTypeId="4" CreationDate="2019-02-20T16:21:27.183" Score="0" Body="For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-25T21:18:23.300" LastActivityDate="2019-09-25T21:18:23.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10769" PostTypeId="5" CreationDate="2019-02-20T18:18:01.680" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T13:30:31.430" LastActivityDate="2020-04-13T13:30:31.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10770" PostTypeId="4" CreationDate="2019-02-20T18:18:01.680" Score="0" Body="For questions related to the policy iteration (PI) algorithm, which is a dynamic programming algorithm that is used to solve a Markov decision process (MDP), given the transition model and reward function (i.e. the model) of the MDP. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T13:30:31.430" LastActivityDate="2020-04-13T13:30:31.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10772" PostTypeId="1" CreationDate="2019-02-20T20:54:53.377" Score="3" ViewCount="233" Body="&lt;p&gt;This question is related to &lt;a href=&quot;https://ai.stackexchange.com/q/7640/2444&quot;&gt;What does &amp;quot;stationary&amp;quot; mean in the context of reinforcement learning?&lt;/a&gt;, but I have a more specific question to clarify the difference between a non-stationary policy and a state that includes time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that, in general, a non-stationary policy is a policy that doesn't change. My first (probably incorrect) interpretation of that was that it meant that the state shouldn't contain time. For example, in the case of game, we could encode time as the current turn, which increases every time the agent takes an action. However, I think even if we include the turn in the state, the policy is still non-stationary so long as sending the same state (including turn) to the policy produces the same action (in case of a deterministic policy) or the same probability distribution (stochastic policy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the notion of stationarity assumes an additional implicit background state that counts the number of times we have evaluated the policy, so a more precise way to think about a policy (I'll use a deterministic policy for simplicity) would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \pi : \mathbb{N} \times S \rightarrow \mathbb{N} \times A $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \pi : (i, s_t) \rightarrow (i + 1, s_{t+1}) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead of &lt;span class=&quot;math-container&quot;&gt;$\pi : S \rightarrow A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, here is the question: &lt;strong&gt;Is it true that a stationary policy must satisfy this condition?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \forall i, j \in \mathbb{N}, s \in S, \pi (i, s) = \pi(j, s) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the policy must output the same result no matter when we evaluate it (either the ith or jth time). Even if the state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; contains a counter of the turn, the policy would still be non-stationary because for the same state (including turn), no matter how many times you evaluate it, it will return the same thing. Correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final note, I want to contrast the difference between a state that includes time, with the background state I called &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in my definition of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. For example, when we run an episode of 3 steps, the state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; will contain 0, 1, 2, and the background counter of number of the policy &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; will also be set to 2. Once we reset the environment to evaluate the policy again, the turn, which we store in the state, will go back to 0, but the background number of evaluations won't reset and it will be 3. My understanding is that in this reset is when we could see the non-stationarity of the policy in action. If we get a different result here it's a non-stationary policy, and if we get the same result it's a stationary policy, and such property is independent of whether or not we include the turn in the state. Correct?&lt;/p&gt;&#xA;" OwnerUserId="12640" LastEditorUserId="12640" LastEditDate="2019-02-24T22:41:12.190" LastActivityDate="2019-02-24T22:41:12.190" Title="What is the difference between a non-stationary policy and a state that stores time?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10773" PostTypeId="1" AcceptedAnswerId="10780" CreationDate="2019-02-20T21:20:58.600" Score="2" ViewCount="71" Body="&lt;p&gt;I am confused about how neural networks weigh different features or inputs.&lt;/p&gt;&#xA;&lt;p&gt;Consider this example. I have 3 features/inputs: an image, a dollar amount, and a rating.  However, since one feature is an image, I need to represent it with very high dimensionality, for example with &lt;span class=&quot;math-container&quot;&gt;$128 \times 128 = 16384$&lt;/span&gt; pixel values.  (I am just using 'image' as an example, my question holds for any feature that needs high dimensional representation: word counts, one-hot encodings, etc.)&lt;/p&gt;&#xA;&lt;p&gt;Will the &lt;span class=&quot;math-container&quot;&gt;$16384$&lt;/span&gt; 'features' representing the image completely overwhelm the other 2 features that are the dollar amount and rating?  Ideally, I would think the network would consider each of the three true features relatively equally.  Would this issue naturally resolve itself in the training process? Would training become much more difficult of a task?&lt;/p&gt;&#xA;" OwnerUserId="22525" LastEditorUserId="2444" LastEditDate="2020-11-21T20:24:35.780" LastActivityDate="2020-11-21T20:30:39.293" Title="How do neural networks weigh multiple inputs/features of different dimensionality?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;features&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10774" PostTypeId="1" CreationDate="2019-02-20T23:14:31.183" Score="2" ViewCount="99" Body="&lt;p&gt;The definition machine learning is as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A computer program is said to learn from experience E with respect to some  task T and performance measure P, if its performance at task T, as measured by P, improves with experience E.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here, we talk about what it means for a program to learn rather than a machine, and a program and a machine aren't equivalent, so this is a definition about &quot;program learning&quot; rather than &quot;machine learning&quot;. How is this consistent?&lt;/p&gt;&#xA;" OwnerUserId="22528" LastEditorUserId="2444" LastEditDate="2019-02-21T22:03:52.010" LastActivityDate="2019-02-22T05:26:19.683" Title="Is the definition of machine learning by Mitchell in his book &quot;Machine Learning&quot; valid?" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10775" PostTypeId="1" CreationDate="2019-02-20T23:48:48.603" Score="2" ViewCount="176" Body="&lt;p&gt;I have a dataset with hundreds of thousands of training examples. There are 27 input variables and one output variable which is always a 0 or a 1, based on whether an event happened or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network therefore has 27 inputs and 1 output. I want the network's output to be a confidence guess of how likely the event is to happen, for example if the output is 0.23 then that represents that the network thinks the event has a 23% chance of happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using back propagation to train the neural network. It does appear to work well and the network outputs a higher number when the event is more likely and a lower number when the event is less likely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be a valid concern that my training data only has 0 or 1 values as outputs, when this is not truly what I want the network to output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern comes from the fact that back propagation attempts to reduce the square of the error between the network's output, and the value of the output in the training data, which is always a 0 or a 1. Because it is the &lt;strong&gt;square&lt;/strong&gt; of the error it is trying to reduce, I'm concerned that it's probability output may not be a linear mapping to the true probability of the event happening based on the 27 inputs it is seeing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a valid concern? And are there any techniques I can use to get a neural network to output a linear confidence guess between 0 and 1 when my test data only has outputs of 0 or 1?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the sigmoid activation function for all of my neurons, would there be a better choice of activation function for this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Thanks to Xpector's answer, I now understand that not all back propagation aims to reduce the square of the error, it depends on the loss function used. I am including a part of the back propagation code I have used here which calculates the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var neuronOutput = layerOutputs[i];&#xA;var error = (neuronOutput - desiredOutput[i]);&#xA;errors[i] = error * Maths.SigmoidDerivative(neuronOutput);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is from an open source RProp implementation. I am not sure what loss function is being used here.&lt;/p&gt;&#xA;" OwnerUserId="21524" LastEditorUserId="21524" LastEditDate="2019-02-21T11:40:23.803" LastActivityDate="2019-02-21T11:40:23.803" Title="Training a neural network to output the conditional probability of an event when the training data output is only binary" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10776" PostTypeId="1" CreationDate="2019-02-20T23:54:50.313" Score="1" ViewCount="46" Body="&lt;p&gt;I wanna solve a &lt;strong&gt;problem of regression&lt;/strong&gt; to predict a factor. I decide to go with &lt;strong&gt;Deep Neural Networks&lt;/strong&gt; as solution for my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The features in this problem represent loop characteristic such us loop nest level, loop sizes. The loops hold also instruction (operations) that in itself represent many characteristics like number of variables, loads, stores, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those instructions maybe positions in the innermost loop or in the middle or under the outermost loop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We extract here characteristics of Computations in Tiramisu language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we have two iterator variables:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var i(&quot;i&quot;, 0, 20), j(&quot;j&quot;, 0, 30);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and we have the following computation declaration:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;computation S(&quot;S&quot;, {i,j}, 4);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to writing the following &lt;code&gt;C&lt;/code&gt; code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; for (i=0; i&amp;lt;20; i++)&#xA;      for (j=0; j&amp;lt;30; j++)&#xA;         S(i,j) = 4;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The aspect of receptivity here we can have something like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; computation S(&quot;S&quot;, {i,j}, 4+M); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &quot;M&quot; is computation also.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We considered those features to represent Computations in Tiramisu language.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/** Computations=loops **/&#xA;   &quot;nest_level&quot; : 3,   // Number of nest levels&#xA;   &quot;perfect_nested&quot; : 1 ,  // 1 if the loop is perfectly nested , 0 instead&#xA;   &quot;loops_sizes&quot; : [200,100,300] // Sizes of for loops &#xA;   &quot;lower_bound&quot; : [5,0,0], // Bounds of the iterator (in this e.g [2, 510])&#xA;   &quot;upper_bound&quot; : [205,100,300], &#xA;   &quot;nb_intern_if&quot; : 1000, //number of if statements in the nest&#xA;   &quot;nb_exec_if&quot; : 300, // Estimation of number if &#xA;   &quot;prec_if&quot; : 1,  // 1=true if the nest is preceded by if statement  &#xA;   &quot;nb_dependencies_intern&quot; : 5, // number of dependencies between loops levels in the nest &#xA;   // &quot;dependencies_extern&quot; : , // number of extern nest dependencies  &#xA;    &quot;nb_computations&quot; : 3,  // number of operations (computations) in the nest &#xA;    //std::map&amp;lt;std::string, computation_features *&amp;gt; computations_features; // list of operations Features in the nest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this to represent operations: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/** Instructions **/&#xA;&quot;n&quot; : 1, &amp;lt;-- Number of computations&#xA;    &quot;compt_array&quot; : [&#xA;      {&#xA;              // Should we add to which level should belong the instructions ?&#xA;&#xA;              &quot;comp_id&quot; : 1,  // Unique id for the instructions&#xA;              &quot;nb_var&quot; : 5,   // Number of the variables in the instructions&#xA;              &quot;nb_const&quot; : 2, // Number of constantes in the instructions&#xA;              &quot;nb_operands&quot; : 3, // Number of operands of the operatiion ( including direct values)&#xA;              &quot;histograme_loads&quot; :  [2,1,5,8], // number of load ops. i.e. acces to inputs per type&#xA;              &quot;histograme_stores&quot; :  [2,1,5,8], // number of load ops. i.e. acces to inputs per type&#xA;              &quot;nb_library_call&quot; : 5;  // number of the computation library_calls &#xA;              &quot;wait_library_argument&quot; : 2, // number of ar &#xA;              &quot;operations_histogram&quot; : [ // number of arithmetic operations per type&#xA;                    [0, 2, 0, 0],  // p_int32&#xA;                    [0, 0, 0, 0],  // float, for example&#xA;                    [0, 0, 0, 0],  // ...&#xA;                    [0, 0, 0, 0],&#xA;                    [0, 0, 0, 0],&#xA;                    [0, 0, 0, 0], // ...&#xA;                    [0, 0, 0, 0]  // boolean    &#xA;              ]              &#xA;      }&#xA;  ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We may also represent iterator as a characteristic of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem in those features we have: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Loops (Computations) can hold many operations ==&gt; the size of operation vector is variable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instructions (Operations) can be in the level 2, 3 under the innermost &#xA;I mean we can have this situation: &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;for (i=0; i &amp;lt; 20; i++)&#xA;    S(i, j) = 4;&#xA;for (j=0; j &amp;lt; 30; j++)&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or this one:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;for (i=0; i&amp;lt;20; i++)&#xA;      for (j=0; j&amp;lt;30; j++)&#xA;         S(i,j) = 4;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Or many other situations with many instructions ==&gt; their is dependencies between the position of the instruction and the level (iterator) in which it is, in the other way operation hold the id of the iterator :§.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;The operation on itself can be composed with another Computation(Loop nest) which on itself hold instruction and so forth ==&gt; Resistivity.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After some research i have found that that DNN has fixed input size. RNN, recursive NN can handle with varying length of inputs. But what about others &lt;/p&gt;&#xA;&#xA;&lt;p&gt;how should I present that as input?&lt;/p&gt;&#xA;" OwnerUserId="22526" LastEditorUserId="22296" LastEditDate="2019-02-21T01:49:40.607" LastActivityDate="2019-02-21T01:49:40.607" Title="How to handle varying length of inputs that represent dependencies and recursivity in deep neural networks in case of regression?" Tags="&lt;neural-networks&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10777" PostTypeId="2" ParentId="10774" CreationDate="2019-02-21T01:58:55.780" Score="1" Body="&lt;p&gt;Think of a computer as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Machine&lt;/a&gt;--this idea is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Model_of_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;model of computation&lt;/a&gt;, and all of modern computing is based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing-Church thesis&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine and program can be interchangeable—at the end of the day, it's all &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithms&lt;/a&gt;, whether hard coded in the form of microchip, or in the form of software.  (Any microchip can be emulated as software.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pre-modern computers were mechanical in popular sense. Examples include Babbage's &#xA; &lt;a href=&quot;https://en.wikipedia.org/wiki/Difference_engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;difference engine&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Mechanical_calculator&quot; rel=&quot;nofollow noreferrer&quot;&gt;mechanical calculators&lt;/a&gt; in general.  These led to &lt;a href=&quot;https://en.wikipedia.org/wiki/Mechanical_calculator#Programmable_mechanical_calculators&quot; rel=&quot;nofollow noreferrer&quot;&gt;programmable mechanical calculators&lt;/a&gt; and electromechanical computers such as IBM's &lt;a href=&quot;https://en.wikipedia.org/wiki/Harvard_Mark_I&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harvard Mark I&lt;/a&gt;, based on Babbage's notion of an &lt;a href=&quot;https://en.wikipedia.org/wiki/Analytical_Engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analytic Engine&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, Machine Learning may also connote software that runs on a machine (hardware) as opposed to human/animal learning, which utilizes a biological medium.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The etymology of &quot;mechanics&quot;, and by extension, &quot;machine&quot; is worth looking at.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Dmechanisma&quot; rel=&quot;nofollow noreferrer&quot;&gt;mechanisma&lt;/a&gt; (in the sense of &quot;mechanism&quot;) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Dmechanicus&quot; rel=&quot;nofollow noreferrer&quot;&gt;mechanicus&lt;/a&gt; (in the sense of &quot;belonging to mechanics&quot;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Dmechanema&quot; rel=&quot;nofollow noreferrer&quot;&gt;mechanema&lt;/a&gt; (in the sense of &quot;a trick&quot;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Mechanema&lt;/em&gt; is instructive in that math and engineering can be understood as the &quot;trick to doing things&quot;. (&lt;a href=&quot;https://www.etymonline.com/word/machination&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machination&lt;/a&gt; is not pejorative in this sense, as &quot;noun of action from past participle stem of &lt;em&gt;machinari&lt;/em&gt; 'contrive skillfully, to design; to scheme, to plot,'&quot;—one can plot the course of a celestial body.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Early uses of the &lt;em&gt;mech-&lt;/em&gt; root&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aristotle's &lt;a href=&quot;https://en.wikipedia.org/wiki/Mechanics_(Aristotle)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Μηχανικά&lt;/a&gt; (&lt;a href=&quot;http://penelope.uchicago.edu/Thayer/E/Roman/Texts/Aristotle/Mechanica*.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Mechanical Problems&quot;&lt;/a&gt;*)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euler's &lt;a href=&quot;http://www.17centurymaths.com/contents/euler/mechvol1/preface.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mechanica&lt;/a&gt; (Euler was referring to &lt;a href=&quot;https://en.wikipedia.org/wiki/Classical_mechanics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classical Mechanics&lt;/a&gt;, an analytic and predictive science made possible by calculus, which is based on functions.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-22T01:18:04.333" LastActivityDate="2019-02-22T01:18:04.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10778" PostTypeId="1" CreationDate="2019-02-21T03:44:13.630" Score="2" ViewCount="335" Body="&lt;p&gt;How to detect liveness of face using face landmark points? &#xA;I am getting face landmarks from android camera frames. And I want to detect liveness using these landmark points. &#xA;How to tell if a human is making a specific movement that can be useful for liveness detection? &lt;/p&gt;&#xA;" OwnerUserId="22532" LastEditorUserId="32410" LastEditDate="2021-04-19T22:43:13.863" LastActivityDate="2021-05-20T00:01:30.193" Title="Face liveness detection using face landmark points" Tags="&lt;machine-learning&gt;&lt;object-detection&gt;&lt;object-recognition&gt;&lt;face-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10779" PostTypeId="2" ParentId="10617" CreationDate="2019-02-21T06:56:37.267" Score="1" Body="&lt;p&gt;Why you need encoder and decoder here, this is not a use case for this. If you want to convert one sequence to another sequence then you can use encoder-decoder. You can use simple seq2seq learning for below purpose.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sequence   &lt;/li&gt;&#xA;&lt;li&gt;Sequence Prediction &lt;/li&gt;&#xA;&lt;li&gt;Sequence Classification&lt;/li&gt;&#xA;&lt;li&gt;Sequence Generation Sequence to Sequence Prediction&#xA;You can use below simple seq2seq method to predicate the next sequence &#xA;Sequence prediction attempts to predict elements of a sequence on the basis of the preceding elements.&#xA;&lt;a href=&quot;https://machinelearningmastery.com/sequence-prediction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/sequence-prediction/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2019-02-21T06:56:37.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10780" PostTypeId="2" ParentId="10773" CreationDate="2019-02-21T08:48:47.957" Score="1" Body="&lt;p&gt;As stated in your example, the three features are: an image, a price, a rating. Now, you want to build a model that uses all of these features and the simplest way to do is to feed them directly into the neural network, but it's inefficient and fundamentally flawed, due to the following reasons:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In the first dense layer, the neural network will try to combine raw pixel values linearly with price and rating, which will produce features that are meaningless for inference.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It could perform well just by optimizing the cost function, but the model performance will be nowhere as good as it could perform with a good architecture.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, the neural network doesn't care if the data is a raw pixel value, price, or rating: it would just optimize it to produce the desired output. That is why it is necessary to design a suitable architecture for the given problem.&lt;/p&gt;&#xA;&lt;p&gt;Possible architecture for your given example :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Separate your raw features, i.e. pixel value, and high-level data, i.e. price and rating&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stack 2-3 dense layers for raw features (to find complex patterns in images)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stack 1-2 dense layers for high-level features&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Combine them together in a final dense layer&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If you want to de-emphasize the importance of the image, just connect the first dense layer 16,384 to another layer having fewer connections, say 1024, and have more connections from the high-level data, say 2048.&lt;/p&gt;&#xA;&lt;p&gt;So, again, here's the possible architecture&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Raw features -&amp;gt; dense layer (16384) -&amp;gt; dense layer (1024)&lt;/li&gt;&#xA;&lt;li&gt;High-level features -&amp;gt; dense layer (2048)&lt;/li&gt;&#xA;&lt;li&gt;Combine 1 and 2 with another dense layer&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22540" LastEditorUserId="2444" LastEditDate="2020-11-21T20:30:39.293" LastActivityDate="2020-11-21T20:30:39.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10783" PostTypeId="2" ParentId="10775" CreationDate="2019-02-21T10:47:34.443" Score="2" Body="&lt;p&gt;To sort the terms a bit, back propagation (repeatedly applying the chain rule of differentiation) is an implementation of stochastic gradient descent, which is an implementation of optimization. It is not necessarily the square of the error that is reduced by optimization - it is up to us to choose the loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function should ideally mirror the utility function (what you are truly want to optimize), but it needs a useful derivative. For example, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function#0-1_loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;0-1 loss&lt;/a&gt; hasn't one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that what is called &lt;em&gt;linear confidence guess&lt;/em&gt; in the question is the conditional probability of the event, given the input. Then the &lt;a href=&quot;https://gombru.github.io/2018/05/23/cross_entropy_loss/&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary cross entropy&lt;/a&gt; would be a valid choice of a loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://www.deeplearningbook.org/contents/optimization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeplearningbook.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The negative log-likelihood allows the model to estimate the conditional probability of the classes, given the input, and if the model can do that well, then it can pick the classes that yield the least classiﬁcation error in expectation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22544" LastActivityDate="2019-02-21T10:47:34.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10784" PostTypeId="2" ParentId="10773" CreationDate="2019-02-21T10:53:30.810" Score="0" Body="&lt;p&gt;The answer by ssh is correct. Your results could be further improved i) by extracting image features by a convolutional (instead of fully connected) architecture, and ii) by exploiting transfer learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To exploit transfer learning you i) pick some widely used model, eg. ResNet-18, ii) initialize it with ImageNet pretrained parameters, iii) replace its fully connected layer (the one that produces 1000-D softmax input) with your own randomly initialized fully connected layer. If you are interested, have a look at detailed &lt;a href=&quot;https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;instructions&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-21T10:53:30.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10785" PostTypeId="2" ParentId="10723" CreationDate="2019-02-21T11:11:45.267" Score="0" Body="&lt;p&gt;It sounds like a supervised learning from not-so-many samples. &lt;a href=&quot;https://stats.stackexchange.com/questions/393288/learning-useful-semantic-representations-of-data&quot;&gt;Representation/metric learning&lt;/a&gt; could be of some help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two books by Bob Volman that help quite a bit to make subconscious conscious. &quot;Understanding Price Action&quot; and &quot;Forex Price Action Scalping&quot;. Despite sales-driven titles, this is the best non-scam attempt to formalize discretionary trading I ever saw. (Even though, one will become a one-trick pony at most, but on the other hand it's enough). If you google a bit, you'll find a dropbox with countless price charts commented by author.&lt;/p&gt;&#xA;" OwnerUserId="22544" LastActivityDate="2019-02-21T11:11:45.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10786" PostTypeId="5" CreationDate="2019-02-21T11:21:57.140" Score="0" Body="&lt;p&gt;For more details, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Simulated_annealing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T22:01:26.577" LastActivityDate="2019-02-21T22:01:26.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10787" PostTypeId="4" CreationDate="2019-02-21T11:21:57.140" Score="0" Body="For questions related to the simulated annealing algorithm (SA), which is a probabilistic algorithm that attempts to find the global optimum of a function. SA can e.g. be used to solve the travelling salesman problem (TSP)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T22:01:06.483" LastActivityDate="2019-02-21T22:01:06.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10788" PostTypeId="1" CreationDate="2019-02-21T11:22:08.810" Score="3" ViewCount="154" Body="&lt;p&gt;In the paper &quot;&lt;a href=&quot;https://arxiv.org/pdf/1612.00563.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-critical sequence training for image captioning&lt;/a&gt;&quot;, on page 3, they define the loss function (of the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) of an image captioning system as the negative expected reward of a generated sequence of words (Equation (3)): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)],$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$w^s = (w_1^s,..., w_T^s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w^s_t$&lt;/span&gt; is the word sampled from the model at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivation of the gradient of &lt;span class=&quot;math-container&quot;&gt;$L(\theta)$&lt;/span&gt; concludes with Equation (7), where the gradient of &lt;span class=&quot;math-container&quot;&gt;$L(\theta)$&lt;/span&gt; is approximated with a single sample &lt;span class=&quot;math-container&quot;&gt;$w^s \sim p_\theta$&lt;/span&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}L(\theta) \approx -(r(w^s) - b) \ \triangledown_{\theta}  log \ p_{\theta}(w^s),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a reward baseline and &lt;span class=&quot;math-container&quot;&gt;$p_\theta(w^s)$&lt;/span&gt; is the probability that sequence &lt;span class=&quot;math-container&quot;&gt;$w^s$&lt;/span&gt; is sampled from the model. Up until here I understand what's going on. However, then they proceed with defining the partial derivative of &lt;span class=&quot;math-container&quot;&gt;$L(\theta)$&lt;/span&gt; w.r.t. the input of the softmax function &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; (final layer):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}L(\theta) = \sum^T_{t=1} \frac{\partial L(\theta)}{\partial s_t} \frac{\partial s_t}{\partial \theta}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still understand the equation above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And Equation (8):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L(\theta)}{\partial s_t} \approx (r(w^s) - b) (p_\theta(w_t| h_t) - 1_{w^s_t}),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$1_{w^s_t}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; everywhere, but &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; at the &lt;span class=&quot;math-container&quot;&gt;$w^s_t$&lt;/span&gt;'th entry. How do you arrive at Equation (8)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm happy to provide more information if necessary. In the paper &quot;&lt;a href=&quot;https://arxiv.org/pdf/1511.06732.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence level training with recurrent neural networks&lt;/a&gt;&quot;, on page 7, they derive a similar result.&lt;/p&gt;&#xA;" OwnerUserId="22545" LastEditorUserId="22545" LastEditDate="2019-02-21T13:49:01.617" LastActivityDate="2019-03-23T14:03:50.590" Title="How is equation 8 derived in the paper &quot;Self-critical sequence training for image captioning&quot;?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;papers&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10789" PostTypeId="2" ParentId="9919" CreationDate="2019-02-21T12:18:16.133" Score="0" Body="&lt;p&gt;My understanding is now that the author's formula is deliberate. It seeks to learn a worst-case maximizing policy. The formula I instead suggest would, I believe, instead be &lt;a href=&quot;http://www.jmlr.org/papers/volume4/hu03a/hu03a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nash Q learning&lt;/a&gt; where the agent seeks to learn to play a Nash equilibrium.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After debugging, I have gotten &lt;a href=&quot;https://github.com/pb4git/Nash-DQN-CSB-Article&quot; rel=&quot;nofollow noreferrer&quot;&gt;good results&lt;/a&gt; with the second formula but cannot speak for the original Minimax Q learning one.&lt;/p&gt;&#xA;" OwnerUserId="21311" LastEditorUserId="21311" LastEditDate="2019-02-21T12:26:02.880" LastActivityDate="2019-02-21T12:26:02.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10791" PostTypeId="2" ParentId="10788" CreationDate="2019-02-21T13:08:54.930" Score="2" Body="&lt;p&gt;First of all you made a mistake, equation 8 in the paper is defined with &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L(\theta)}{\partial s_t}$&lt;/span&gt; not &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L(\theta)}{\partial\theta}$&lt;/span&gt;.&lt;br&gt;&#xA;Loss is defined as:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use definition of expectation (for discrete case): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X] = \sum\limits_{i} p_i(x_i)x_i$&lt;/span&gt;&lt;br&gt;&#xA;we get  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L(\theta) = -\sum\limits_{w^s} p_\theta(w^s)r(w^s)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;probability &lt;span class=&quot;math-container&quot;&gt;$p_\theta(w_t)$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$softmax(s_t)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$S(s_t)$&lt;/span&gt;. If we calculate derivative of &lt;span class=&quot;math-container&quot;&gt;$L(\theta)$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, we are basically calculating derivative of softmax &lt;span class=&quot;math-container&quot;&gt;$S(s_t)$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be shown and i won't derive it here that derivative of softmax function is  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial S(s_i)}{\partial s_j} = S(s_i)(1_{ij} - S(s_j))$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$1_{ij}$&lt;/span&gt; here is defined as yours. If we plug this derivative into derivative of  &lt;span class=&quot;math-container&quot;&gt;$L(\theta)$&lt;/span&gt; and replace &lt;span class=&quot;math-container&quot;&gt;$S(s_t)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$p_\theta(w_t)$&lt;/span&gt; (remember that probability distribution is defined as softmax here) we get:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L(\theta)}{\partial s_t} = -\sum\limits_{w^s} p_\theta(w^s)(1_{w^s_t} - p_\theta(w_t))r(w^s)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we extract the - from the equation above, use definition of expectation again and subtract baseline from reward and we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L(\theta)}{\partial s_t} = \mathbb{E}_{w^s \sim p_{\theta}}[(p_\theta(w_t) - 1_{w^s_t})(r(w^s) - b)]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;now, if we ignore expectation we get:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L(\theta)}{\partial s_t} \approx (p_\theta(w_t) - 1_{w^s_t})(r(w^s) - b)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I probably butchered notation is some places, but base idea should be clear&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-02-21T13:08:54.930" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10792" PostTypeId="2" ParentId="10778" CreationDate="2019-02-21T13:38:38.153" Score="0" Body="&lt;p&gt;One approach would be to obtain labeled (&quot;yes/no&quot;) samples, e.g. via Mechanical Turk, it costs though) and approach it as supervised learning task. E.g. a neural network will discover itself through the learning what features (landmark positions) are helpful for detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There should be examples around for mood detection etc.&lt;/p&gt;&#xA;" OwnerUserId="22544" LastActivityDate="2019-02-21T13:38:38.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10793" PostTypeId="2" ParentId="10764" CreationDate="2019-02-21T14:20:03.220" Score="1" Body="&lt;p&gt;The spikes could be caused by many reasons: insufficient model capacity, incorrect label, buggy input parsing, ... Finding out the culprit requires some detective work. For instance, you could apply the learned model to the whole train set and manually examine the datapoints which result in the highest loss. Alternatively, you could compare the learning outcomes of different models (both weaker and stronger). &lt;/p&gt;&#xA;" OwnerUserId="21726" LastActivityDate="2019-02-21T14:20:03.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10794" PostTypeId="1" CreationDate="2019-02-21T16:27:51.170" Score="2" ViewCount="15" Body="&lt;p&gt;I'm creating a schedule for a summer camp. Because of the high risk of rain, the higher priority activities need to be attempted first, so there is more time for later attempts if need be (temporarily ignoring the schedule in that situation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Camp takes place over four days. My current idea is to map the days to a set of numbers (4, 3, 2, 1), and get a correlation between these numbers and the priorities of activities. But I'm not certain this is the best way to do it, nor what the best way of correlating the two are. I'm also not sure how I would factor this correlation in with the fitness function, along with the priorities themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I proceed?&lt;/p&gt;&#xA;" OwnerUserId="5526" LastActivityDate="2019-02-21T16:27:51.170" Title="Judging a genetic algorithm's priority-based schedules by how far ahead the higher priority things are done" Tags="&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10795" PostTypeId="5" CreationDate="2019-02-21T16:29:51.900" Score="0" Body="&lt;p&gt;Time complexity can, in some sense, be understood as time measured in computational operations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more precise definition, see: &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Time Complexity (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Additional Resources:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=moPtwq_cVH8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Erik Demaine's lecture on Computational Complexity (MIT)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="1671" LastEditDate="2019-02-21T22:09:16.880" LastActivityDate="2019-02-21T22:09:16.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10796" PostTypeId="4" CreationDate="2019-02-21T16:29:51.900" Score="0" Body="For questions related to the time complexity (e.g. in Big-O notation) of AI and ML algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T22:01:34.047" LastActivityDate="2019-02-21T22:01:34.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10797" PostTypeId="1" AcceptedAnswerId="10799" CreationDate="2019-02-21T17:00:37.000" Score="2" ViewCount="51" Body="&lt;p&gt;In an unknown environment, how do I avoid an agent to tend to terminate its trajectory in a negative state when time needs to be taken into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the following example to make my question clear:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A mouse (M) starts in the bottom left of its world&lt;/li&gt;&#xA;&lt;li&gt;Its goal is to reach the cheese (C) in the top right (+5 reward) while also avoiding the trap (T) (-5 reward)&lt;/li&gt;&#xA;&lt;li&gt;It should do this as quickly as possible, so for every timestep it also receives a penalty (-1 reward)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If the grid world is sufficiently large, it may actually take the mouse many actions to reach the cheese. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a scenario where the mouse may choose to prefer the trap (-1*small + -5 cumulative reward) versus the cheese (-1*large + 5 cumulative reward)?  Is this avoidable?  How does this translate to an unknown environment where the number of time steps required to reach the positive terminal state is unknown?&lt;/p&gt;&#xA;" OwnerUserId="22525" LastEditorUserId="2444" LastEditDate="2019-02-21T18:25:48.223" LastActivityDate="2019-03-23T20:22:15.007" Title="How do I avoid an agent to tend to terminate in a negative state when time needs to be taken into account?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10798" PostTypeId="1" AcceptedAnswerId="10907" CreationDate="2019-02-21T19:55:43.213" Score="5" ViewCount="3010" Body="&lt;p&gt;Sutton and Barto state in the 2018-version of &quot;Reinforcement Learning: An Introduction&quot; in the context of Expected SARSA (p. 133) the following sentences:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Expected SARSA is more complex computationally than Sarsa but, in return, it eliminates the variance due to the random selection of &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt;. Given the same amount of experience we might expect it to perform slightly better than Sarsa, and indeed it generally does.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have three questions concerning this statement:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why is the action selection random with Sarsa? Isn't it on-policy and therefore &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy?&lt;/li&gt;&#xA;&lt;li&gt;Because Expected-Sarsa is off-policy the experience it learns from can be from any policy that at least explores everything in the limit e.g. random action-selection with equal probabilities for every action. How can Exected-Sarsa learning from such policy be &lt;em&gt;generally&lt;/em&gt; better than normal Sarsa learning from an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy, especially with the same amount of experience?&lt;/li&gt;&#xA;&lt;li&gt;Probably more general: How can on-policy and off-policy algorithms be compared in such way (e.g. through variance) even though their concepts and assumptions are so different?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21299" LastEditorUserId="2444" LastEditDate="2019-02-21T20:19:05.507" LastActivityDate="2019-02-27T09:00:51.383" Title="Expected SARSA vs SARSA in &quot;RL: An Introduction&quot;" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;sarsa&gt;&lt;expected-sarsa&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10799" PostTypeId="2" ParentId="10797" CreationDate="2019-02-21T19:57:09.600" Score="2" Body="&lt;p&gt;This is a common problem in reward shaping. You want a certain behavior from you agent but its challenging to describe it completely in terms of rewards. This situation you are describing is challenging specifically because as the grid world grows, the chance of randomly stumbling onto the goal state becomes less likely AKA the problem of exploration.&#xA;There are a few techniques that can be used to address this problem though, here are some.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;0) This is an emergent property of the environment and gamma&lt;/strong&gt; (0 based because its more immediate to your problem:p)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If gamma is small, your agent will value rewards that are in its immediate future more highly whereas as gamma approaches 1, the agent values rewards that are further in its future. In your grid world, the the size of the grid affects how this gamma affects your agent. Like in your example, if your grid was 100x100, if the trap was close to the agent, you would have to have a gamma closer to 0 in order for your agent to avoid the trap because its worse than moving to a cell that isn't a trap. This is interesting because the whole purpose of gamma is to increase the weighted value of temporally distant rewards but when you make the trap more favorable than the goal, going to the trap is the optimal strategy. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Include more observation data to your model&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't always a possibility but depending on your application and what you have available, you may be able to give your agent missing information that might be necessary to its ability to solve your task. For instance, in your infinite grid example, you may include the distance between the agent and the goal or the direction of the goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Include a reward that helps to shape the direction of progress.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could easily create an infinite grid world where there isn't actually a goal but rather a continuing task where the agent has to cover distance in a desired direction while avoiding obstacles. How would you approach this problem? Perhaps a reward that specifically looks at the number of cells visited by the agent in the desired direction over time aka the discrete analog to velocity. Of course this is still dependent on a know direction but thinking about how to one might handle the &quot;limit&quot; of your environment as it grows (e.g., adding more and more cells to the grid world) helps to give an intuition of what your agent could be missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3) Use curiosity based approaches&lt;/strong&gt;&#xA;Following from 2), if the direction isn't known, one thing to consider is rather than giving a penalty for each timestep, instead incentivize the agent to be faster, rewarding visiting &quot;infrequently visited&quot; states. As the task requires that the agent performs as quickly as possible, remaining or returning to a previously visited state clearly doesn't benefit the agent. Taking this notion of rewarding &quot;visiting infrequently/unyet visited/ states&quot; further results in the recent research topic of using curiosity to have RL agents that have novel exploration strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there are many (often debated) ways to define curiosity, they all share an idea of giving the agent a bonus when it has entered a state that has not been visited before. A paper that gives a good recap of curiosity methods and also introduces a novel approach is &lt;a href=&quot;https://blog.openai.com/reinforcement-learning-with-prediction-based-rewards/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Network Distillation&lt;/a&gt; from OpenAI.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-02-21T19:57:09.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10800" PostTypeId="5" CreationDate="2019-02-21T20:20:04.517" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-21T20:20:04.517" LastActivityDate="2019-02-21T20:20:04.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10801" PostTypeId="4" CreationDate="2019-02-21T20:20:04.517" Score="0" Body="For questions related to the reinforcement learning (on-policy) algorithm called SARSA, which stands for (s, a, r, s', a')." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T22:01:21.363" LastActivityDate="2019-02-21T22:01:21.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10802" PostTypeId="5" CreationDate="2019-02-21T20:21:29.610" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-21T20:21:29.610" LastActivityDate="2019-02-21T20:21:29.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10803" PostTypeId="4" CreationDate="2019-02-21T20:21:29.610" Score="0" Body="For questions related to the reinforcement learning algorithm called &quot;expected SARSA&quot; (as described in the book &quot;Reinforcement Learning: An Introduction&quot;, by Sutton and Barto, 2nd edition)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-21T22:01:23.863" LastActivityDate="2019-02-21T22:01:23.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10804" PostTypeId="5" CreationDate="2019-02-21T22:47:32.083" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-22T01:18:24.570" LastActivityDate="2019-02-22T01:18:24.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10805" PostTypeId="4" CreationDate="2019-02-21T22:47:32.083" Score="0" Body="For questions related to the &quot;expectation-maximisation&quot; (EM) algorithm (which is used in several contexts in AI)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-25T22:00:37.407" LastActivityDate="2019-02-25T22:00:37.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10806" PostTypeId="1" CreationDate="2019-02-22T01:30:15.920" Score="1" ViewCount="1623" Body="&lt;p&gt;I've been looking at various bounding box algorithms, like the three versions of RCNN, SSD and YOLO, and I have noticed that not even the original papers include pseudocode for their algorithms. I have built a CNN classifier and I am attempting to incorporate bounding box regression, though I am having difficulties in implementation. I was wondering if anyone can whip up some pseudocode for any bounding box classifier or a link to one (unsuccessful in my search) to aid my endeavor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I do know that there are many pre-built and pre-trained versions of these object classifiers that I can download from various sources, I am interested in building it myself.&lt;/p&gt;&#xA;" OwnerUserId="22563" LastEditorUserId="1671" LastEditDate="2020-03-04T23:57:08.720" LastActivityDate="2020-03-04T23:57:08.720" Title="Pseudocode for CNN with Bounding Box and Classifier" Tags="&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;&lt;pseudocode&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10808" PostTypeId="2" ParentId="3731" CreationDate="2019-02-22T08:54:00.887" Score="0" Body="&lt;p&gt;While adjusting the learning rate during the training is certainly interesting, so is finding of a good initial learning rate, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1506.01186&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cyclical Learning Rates for Training Neural Networks&lt;/a&gt;, &lt;a href=&quot;https://docs.fast.ai/basic_train.html#lr_find&quot; rel=&quot;nofollow noreferrer&quot;&gt;fast.ai (pytorch) implementation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a good practitioner's &lt;a href=&quot;https://github.com/fastai/fastai/blob/master/courses/dl2/training_phase.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;overview of learning rate schedules&lt;/a&gt; (python, but rather readable and with nice plots).&lt;/p&gt;&#xA;" OwnerUserId="22544" LastActivityDate="2019-02-22T08:54:00.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10809" PostTypeId="2" ParentId="5393" CreationDate="2019-02-22T09:01:53.270" Score="1" Body="&lt;p&gt;You might find &lt;a href=&quot;https://stats.stackexchange.com/questions/352036/&quot;&gt;&quot;What should I do when my neural network doesn't learn?&quot;&lt;/a&gt; and &lt;a href=&quot;https://blog.slavv.com/37-reasons-why-your-neural-network-is-not-working-4020854bd607&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Reasons why your Neural Network is not working&quot;&lt;/a&gt; interesting. &lt;/p&gt;&#xA;" OwnerUserId="22544" LastActivityDate="2019-02-22T09:01:53.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10810" PostTypeId="1" CreationDate="2019-02-22T09:02:25.867" Score="1" ViewCount="58" Body="&lt;p&gt;I am using rasa nlu for training an NLU system to detect intents and slots. Now, some languages have word endings with their nouns (like Finnish, e.g. &quot;in Berlin&quot; -&gt; &quot;Berliinissä&quot;). I have tried to annotate the characters in the training data as entities, but then I run the model, it doesn't detect the characters inside the word. When those characters are a separate word, only then they're detected. I am unable to think of an implementation to effectively detect named entities within a word. Suggestions needed.&lt;/p&gt;&#xA;" OwnerUserId="22574" LastEditorUserId="22574" LastEditDate="2019-02-24T07:10:55.897" LastActivityDate="2019-02-24T07:10:55.897" Title="Detect named entities inside words using spaCy" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10811" PostTypeId="1" CreationDate="2019-02-22T09:07:45.573" Score="1" ViewCount="32" Body="&lt;p&gt;Is there any guidance available for training on very noisy data, when Bayes error rate (lowest possible error rate for any classifier) is high? For example, I wonder if deliberately (not due to memory or numerical stability limitations) lowering the batch size or learning rate could produce a better classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found so far some general recommendations, not specific for noisy data: &lt;a href=&quot;https://stats.stackexchange.com/questions/164876/tradeoff-batch-size-vs-number-of-iterations-to-train-a-neural-network&quot;&gt;Tradeoff batch size vs. number of iterations to train a neural network&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22544" LastEditorUserId="22544" LastEditDate="2019-02-24T19:59:29.450" LastActivityDate="2019-02-24T19:59:29.450" Title="Any guidance on learning rate / batch size for noisy data (high Bayes error rate)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10812" PostTypeId="1" AcceptedAnswerId="10818" CreationDate="2019-02-22T09:28:24.537" Score="12" ViewCount="7601" Body="&lt;p&gt;I came across this 2 algorithms but I cannot understand the difference between these  2 both in terms of implementation as well as intuitionally.&#xA;So what difference does the second point in both the slides referring to?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kHuKM.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kHuKM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dOhO5.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dOhO5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-06-28T02:29:06.000" Title="What is the difference between First-Visit Monte-Carlo and Every-Visit Monte-Carlo Policy Evaluation?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10813" PostTypeId="1" CreationDate="2019-02-22T12:11:54.140" Score="1" ViewCount="77" Body="&lt;p&gt;I'm going through the paper &lt;a href=&quot;https://arxiv.org/pdf/1505.05424.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Weight Uncertainty in Neural Networks&lt;/em&gt;&lt;/a&gt; by Google Deepmind. In the final line of the proof of proposition 1, the integral and the derivative are swapped. Then the derivative is taken. But this somehow yields 2 derivatives of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. I thought that this was the result of a product rule applied to &lt;span class=&quot;math-container&quot;&gt;$q(\epsilon)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(w,\theta)$&lt;/span&gt; and then the chain rule. But that does not yield the same outcome as &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial q(\epsilon)}{\partial \theta} = 0 $&lt;/span&gt;. My question is: does anyone understand how the equation in the last line comes about?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/U7BeO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U7BeO.png&quot; alt=&quot;Proposition 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22273" LastActivityDate="2019-02-22T13:02:14.230" Title="Problem with Proposition 1 of Google Deepmind's 'Weight uncertainty in Neural Networks'" Tags="&lt;neural-networks&gt;&lt;bayesian-network&gt;&lt;expectation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10814" PostTypeId="2" ParentId="10454" CreationDate="2019-02-22T12:24:58.400" Score="0" Body="&lt;p&gt;Hill Climbing algo or Constraint-based structure learning algorithms implemented accept &lt;em&gt;whitelist&lt;/em&gt; or &lt;em&gt;blacklist&lt;/em&gt; arguments permitting or prohibitting some arcs.&lt;/p&gt;&#xA;" OwnerUserId="22113" LastActivityDate="2019-02-22T12:24:58.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10815" PostTypeId="2" ParentId="10813" CreationDate="2019-02-22T12:54:29.510" Score="2" Body="&lt;p&gt;we start with  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial \theta} \mathbb E_{q(\mathbf w\mid\theta)}[f(\mathbf w, \theta)]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using definition of expectation for continuous case:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb E[X] = \int xp(x) dx$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the first equation we get:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial \theta} \int f(\mathbf w, \theta)q(\mathbf w \mid \theta) d\mathbf w$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we swap &lt;span class=&quot;math-container&quot;&gt;$q(\mathbf w \mid \theta) d\mathbf w$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$q(\epsilon)d\epsilon$&lt;/span&gt; and we get:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial \theta} \int f(\mathbf w, \theta)q(\epsilon)d\epsilon$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;using definition of expectation again we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial \theta} \mathbb E_{q(\epsilon)}[f(\mathbf w,\theta)]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i believe dominant convergence theorem let's us interchange expectation and derivative and we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb E_{q(\epsilon)}[\frac{\partial}{\partial \theta}f(\mathbf w,\theta)]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that depends on two variables &lt;span class=&quot;math-container&quot;&gt;$\mathbf w$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. In order to get a full derivative of such function we need to take derivative of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with respect to both variables. We have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb E_{q(\epsilon)}[\frac{\partial f(\mathbf w,\theta)}{\partial \theta} + \frac{\partial f(\mathbf w,\theta)}{\partial \mathbf w} \frac{\partial \mathbf w}{\partial \theta}]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First term inside expectation is derivative of function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; directly, and in the second term &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is derivated with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf w$&lt;/span&gt;, but we need to apply chain rule to get full derivative with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; &lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-02-22T13:02:14.230" LastActivityDate="2019-02-22T13:02:14.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10818" PostTypeId="2" ParentId="10812" CreationDate="2019-02-22T14:45:46.643" Score="20" Body="&lt;p&gt;The first-visit and the every-visit Monte-Carlo (MC) algorithms are both used to solve the &lt;strong&gt;&lt;em&gt;prediction&lt;/em&gt; problem&lt;/strong&gt; (or, also called, &quot;evaluation problem&quot;), that is, the problem of estimating the value function associated with a given (as input to the algorithms) fixed (that is, it does not change during the execution of the algorithm) policy, denoted by &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. In general, even if we are given the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, we are not necessarily able to find the exact corresponding value function, so these two algorithms are used to &lt;strong&gt;estimate&lt;/strong&gt; the value function associated with &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, we care about the value function associated with &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; because we might want or need to know &quot;how good it is to be in a certain state&quot;, if the agent behaves in the environment according to the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, assume that the value function is the state value function (but it could also be e.g. the state-action value function), denoted by &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; is the &lt;em&gt;expected return&lt;/em&gt; (or, in other words, &lt;em&gt;expected cumulative future discounted reward&lt;/em&gt;), starting from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (at some time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;) and then following (after time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;) the given policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Formally, &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s) = \mathbb{E}_\pi [ G_t \mid S_t = s ]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=0}^\infty \gamma^k R_{t+k+1}$&lt;/span&gt; is the &lt;strong&gt;return&lt;/strong&gt; (after time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of MC algorithms, &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; is often defined as &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^T  R_{t+k+1}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$T \in \mathbb{N}^+$&lt;/span&gt; is the last time step of the episode, that is, the sum goes up to the final time step of the episode, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. This is because MC algorithms, in this context, often assume that the problem can be naturally split into &lt;strong&gt;episodes&lt;/strong&gt; and each episode proceeds in a discrete number of &lt;strong&gt;time steps&lt;/strong&gt; (from &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I defined it here, the return, in the case of MC algorithms, is only associated with a single episode (that is, it is the return of one episode). However, in general, the expected return can be different from one episode to the other, but, for simplicity, we will assume that the expected return (of all states) is the same for all episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To recapitulate, the first-visit and every-visit MC (prediction) algorithms are used to estimate &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;, for all states &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt;. To do that, at every episode, these two algorithms use &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to behave in the environment, so that to obtain some knowledge of the environment in the form of sequences of states, actions and rewards. This knowledge is then used to estimate &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;. &lt;em&gt;How is this knowledge used in order to estimate &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt;?&lt;/em&gt; Let us have a look at the pseudocode of these two algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q8YCg.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q8YCg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$N(s)$&lt;/span&gt; is a &quot;counter&quot; variable that counts the number of times we visit state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; throughout the entire algorithm (i.e. from episode one to &lt;span class=&quot;math-container&quot;&gt;$num\_episodes$&lt;/span&gt;). &lt;span class=&quot;math-container&quot;&gt;$\text{Returns(s)}$&lt;/span&gt; is a list of (undiscounted) returns for state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is more useful for you to read the pseudocode (which should be easily translatable to actual code) and understand what it does rather than explaining it with words. Anyway, the basic idea (of both algorithms) is to generate trajectories (of states, actions and rewards) at each episode, keep track of the returns (for each state) and number of visits (of each state), and then, at the end of all episodes, average these returns (for all states). This average of returns should be an approximation of the expected return (which is what we wanted to estimate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The differences of the two algorithms are highlighted in &lt;span class=&quot;math-container&quot;&gt;$\color{red}{\text{red}}$&lt;/span&gt;. The part &quot;&lt;em&gt;If state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; is &lt;strong&gt;not&lt;/strong&gt; in the sequence &lt;span class=&quot;math-container&quot;&gt;$S_0, S_1, \dots, S_{t-1}$&lt;/span&gt;&lt;/em&gt;&quot; means that the associated block of code will be executed only if &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; is &lt;strong&gt;not&lt;/strong&gt; part of the sequence of states that were visited (in the episode sequence generated with &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;) before the time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. In other words, that block of code will be executed only if it is the first time we encounter &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; in the sequence of states, action and rewards: &lt;span class=&quot;math-container&quot;&gt;$S_0, A_0, R_1, S_1, A_1, R_2 \ldots, S_{T-1}, A_{T-1}, R_T$&lt;/span&gt; (which can be collectively be called &quot;episode sequence&quot;), with respect to the time step and not the way the episode sequence is processed.  Note that a certain state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; might appear more than once in &lt;span class=&quot;math-container&quot;&gt;$S_0, A_0, R_1, S_1, A_1, R_2 \ldots, S_{T-1}, A_{T-1}, R_T$&lt;/span&gt;: for example, &lt;span class=&quot;math-container&quot;&gt;$S_3 = s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S_5 = s$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do not get confused by the fact that, within each episode, we proceed from the time step &lt;span class=&quot;math-container&quot;&gt;$T-1$&lt;/span&gt; to time step &lt;span class=&quot;math-container&quot;&gt;$t = 0$&lt;/span&gt;, that is, we process the &quot;episode sequence&quot; backwards. We are doing that only to more conveniently compute the returns (given that the returns are &lt;em&gt;iteratively&lt;/em&gt; computed as follows &lt;span class=&quot;math-container&quot;&gt;$G \leftarrow G + R_{t+1}$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, intuitively, in the first-visit MC, we only update the &lt;span class=&quot;math-container&quot;&gt;$\text{Returns}(S_t)$&lt;/span&gt; (that is, the list of returns for state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;, that is, the state of the episode at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;) the first time we encounter &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; in that same episode (or trajectory). In the every-visit MC, we update the list of returns for the state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; every time we encounter &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; in that same episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more info regarding these two algorithms (for example, the convergence properties), have a look at section 5.1 (on page 92) of the book &quot;&lt;a href=&quot;http://incompleteideas.net/book/bookdraft2018mar21.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;&quot; (2nd edition), by Andrew Barto and Richard S. Sutton.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-22T18:57:14.263" LastActivityDate="2019-02-22T18:57:14.263" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10822" PostTypeId="1" AcceptedAnswerId="10834" CreationDate="2019-02-22T16:29:59.450" Score="1" ViewCount="538" Body="&lt;p&gt;I have a reinforcement learning agent with both a positive and a negative terminal state.  After each episode during training, I am recording whether a success or failure occurred, and then I can compute a running ratio of success to failure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am seeing a phenomenon where, at some point in time, my agent achieves a reasonably high success rate (~80%) for a 100-episode running average.  However, with further training, it seems to 'train itself out' of this behavior and ends the training sequence with a very low success rate (~10-20%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using an epsilon-greedy strategy whereby epsilon decays linearly from 1.0 to 0.1 for the first 10% of episodes and then remains at 0.1 for the remaining 90%.  As such, the 'training out' appears to occur some time where exploration only occurs with 10% probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be causing this undesirable behavior? How can I combat it?&lt;/p&gt;&#xA;" OwnerUserId="22525" LastEditorUserId="2444" LastEditDate="2019-02-22T17:06:19.537" LastActivityDate="2019-02-23T00:44:14.250" Title="What is happening when a reinforcement learning agent trains itself out of desired behavior?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10823" PostTypeId="1" CreationDate="2019-02-22T16:51:21.723" Score="3" ViewCount="117" Body="&lt;p&gt;I have a questionnaire consisting of over 10 questions. The questionnaire is being answered by a lot of people, which I have manually graded. Each question can give the user up to 10 points depending on how they have answered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that my dataset is big enough, how would I go about using a neural network to automatically grade these questions for me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used a convolutional neural network before for image classification, but, when dealing with text classification, where should I start? Is there some sort of tutorial out there that covers this with a similar example?&lt;/p&gt;&#xA;" OwnerUserId="22580" LastEditorUserId="2444" LastEditDate="2020-03-11T02:25:24.643" LastActivityDate="2020-08-08T03:06:15.653" Title="How can I train a neural network to grade the user's answers to a questionnaire?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10824" PostTypeId="2" ParentId="10810" CreationDate="2019-02-22T17:36:17.827" Score="1" Body="&lt;p&gt;You could use a tool to decompose composite word. There are many open source softwares for various languages. For example : &lt;a href=&quot;https://github.com/hiroshi-manabe/german-word-splitter&quot; rel=&quot;nofollow noreferrer&quot;&gt;german-word-splitter&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22581" LastActivityDate="2019-02-22T17:36:17.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10825" PostTypeId="2" ParentId="10806" CreationDate="2019-02-22T18:05:21.147" Score="0" Body="&lt;h1&gt;In General&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Each of those projects has open source code on github that you can look at.  If you do some quick googling you'll find the software for these basic regressors exist for different deep learning frameworks.  Usually the these types of projects only include pseduo code for the custom or complicated layers that are involved in the detector.  There isn't much of a need to include pseudo code for a simple convolution because it's a well established operation.  I just included a few links here, but if you look around there's a bunch of implementations in different frameworks that you'll find.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;SSD&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/amdegroot/ssd.pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/amdegroot/ssd.pytorch&lt;/a&gt;&#xA;&lt;a href=&quot;https://github.com/weiliu89/caffe/tree/ssd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/weiliu89/caffe/tree/ssd&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;FasterRCNN (Better version of RCNN)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorpack/tensorpack/tree/master/examples/FasterRCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorpack/tensorpack/tree/master/examples/FasterRCNN&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;YOLO V3&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;V3 -- &lt;a href=&quot;https://itnext.io/implementing-yolo-v3-in-tensorflow-tf-slim-c3c55ff59dbe&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://itnext.io/implementing-yolo-v3-in-tensorflow-tf-slim-c3c55ff59dbe&lt;/a&gt;&#xA;V1 -- &lt;a href=&quot;https://github.com/hizhangp/yolo_tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hizhangp/yolo_tensorflow&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Aside&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;As an aside, unless you're trying to do novel research or fit a specific function usually these detectors work pretty well out of the box.  Running a base level detector and them modifying for your application is usually a safer route unless you have a deep understanding of how these networks work.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2019-02-22T18:05:21.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10826" PostTypeId="1" AcceptedAnswerId="11002" CreationDate="2019-02-22T18:51:34.127" Score="1" ViewCount="175" Body="&lt;p&gt;Say I have a set of data generated by someone. It could be either bytes from a photo, or readings from bio-sensors, and I have a huge amount of said information, from many people or subjects. Which AI algorithms could be used to learn that a set of data belongs to a subject. I would have the information map that a huge set of data belongs to  Bob, and another belongs to Alice to train the system.&lt;/p&gt;&#xA;" OwnerUserId="22586" LastActivityDate="2019-03-04T18:16:33.570" Title="Which AI algorithm to use to identify a subject from many unknown factors" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10830" PostTypeId="1" AcceptedAnswerId="11206" CreationDate="2019-02-22T20:55:03.887" Score="3" ViewCount="1446" Body="&lt;p&gt;I'm trying to replicate the DeepMind paper results, so I implemented my own DQN. I left it training for more than 4 million frames (more than 2000 episodes) on SpaceInvaders-v4 (OpenAI-Gym) and it couldn't finish a full episode. I tried two different learning rates (0.0001 and 0.00125) and seems to work better with 0.0001, but the median score never raises above 200.&#xA;I'm using a double DQN.&#xA;Here is my code and some photos of the graphs I'm getting each session. &#xA;Between sessions I'm saving the network weights; I'm updating the target network every 1000 steps.&#xA;I can't see if I'm doing something wrong, so any help would be appreciated. I'm using the same CNN construction as the DQN paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the &lt;strong&gt;action selection&lt;/strong&gt; function; it uses a batch  of 4  80x80 processed experiences in grayscale to select the action (s_batch means for state batch):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def action_selection(self, s_batch):&#xA;        action_values = self.parallel_model.predict(s_batch)&#xA;        best_action = np.argmax(action_values)&#xA;        best_action_value = action_values[0, best_action]&#xA;        random_value = np.random.random()&#xA;&#xA;        if random_value &amp;lt; AI.epsilon:&#xA;            best_action = np.random.randint(0, AI.action_size)&#xA;        return best_action, best_action_value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my &lt;strong&gt;training function&lt;/strong&gt;. It uses the past experiences as training; I tried to implement that if it lose any life, it wouldn't get any extra rewards, so in theory, the agent would try to not die:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def training(self, replay_s_batch, replay_ns_batch):&#xA;        Q_values = []&#xA;        batch_size = len(AI.replay_s_batch)&#xA;        Q_values = np.zeros((batch_size, AI.action_size))&#xA;&#xA;        for m in range(batch_size):&#xA;&#xA;            Q_values[m] = self.parallel_model.predict(AI.replay_s_batch[m].reshape(AI.batch_shape))&#xA;            new_Q = self.parallel_target_model.predict(AI.replay_ns_batch[m].reshape(AI.batch_shape))&#xA;            Q_values[m, [item[0] for item in AI.replay_a_batch][m]] = AI.replay_r_batch[m]&#xA;&#xA;            if np.all(AI.replay_d_batch[m] == True):&#xA;                Q_values[m, [item[0] for item in AI.replay_a_batch][m]] = AI.gamma * np.max(new_Q)    &#xA;&#xA;        if lives == 0:&#xA;            loss = self.parallel_model.fit(np.asarray(AI.replay_s_batch).reshape(batch_size,80,80,4), Q_values, batch_size=batch_size, verbose=0)&#xA;&#xA;        if AI.epsilon &amp;gt; AI.final_epsilon:&#xA;            AI.epsilon -= (AI.initial_epsilon-AI.final_epsilon)/AI.epsilon_decay&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;replay_s_batch it's a batch of (batch_size) experience replay states (packs of 4 experiences), and replay_ns_batch it's full of 4 next states. The batch size is 32.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here are some results, after training:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VqRoj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VqRoj.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;In blue, the loss (I think it's correct; it's near-zero). Red dots are the different match scores (as you can see, it does sometimes really good matches). In green, the median (near 190 in this training, with learning rate = 0.0001)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Mf72r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Mf72r.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &#xA;Here is the last training, with lr = 0.00125; the results are worse (it's median it's about 160). Anyway the line it's almost straight, I don't see any variation in any case.&#xA;So anyone can point me to the right direction? I tried a similar approach with pendulum and it worked properly. I know that with Atari games it takes more time but a week or so I think it's enough, and it seems to be stuck.&#xA;In case someone need to see another part of my code just tell me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; With the suggestions provided, I modified the action_selection function. Here it is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def action_selection(self, s_batch):&#xA;    if np.random.rand() &amp;lt; AI.epsilon:&#xA;        best_action = env.action_space.sample()&#xA;    else:&#xA;        action_values = self.parallel_model.predict(s_batch)&#xA;        best_action = np.argmax(action_values[0])&#xA;    return best_action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To clarify my last edit: with action_values you get the q values; with best_action you get the action which corresponds to the max q value. Should I return that or just the max q value?&lt;/p&gt;&#xA;" OwnerUserId="9818" LastEditorUserId="9818" LastEditDate="2019-02-23T19:11:36.850" LastActivityDate="2019-03-13T18:59:34.400" Title="My DQN is stuck and can't see where the problem is" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10831" PostTypeId="2" ParentId="10823" CreationDate="2019-02-22T20:57:27.450" Score="1" Body="&lt;p&gt;use the embedding layers which are mainly used for text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;input the question number and the text that the student wrote to the algorithm. Make the problem into a regression of a number from 0 to 10 ( or a classification of class 10, see which one gives better performance).  So the NN will look at text given a question number and try to figure out which 'points' should it get. It will be a supervised training problem since you already have examples of text rated from 0 to 10. &lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2019-02-22T20:57:27.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10833" PostTypeId="1" CreationDate="2019-02-22T23:50:17.017" Score="1" ViewCount="41" Body="&lt;p&gt;Here is what I understand (what I think I understand).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We first train out model on our images using transfer learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we have a pre-trained model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each image in out dataset, we compute &lt;strong&gt;selective search&lt;/strong&gt; on it, which makes 2000 region proposals.&#xA;    These 2000 region proposals are feed through our pre-trained NN ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However we only collect the output (feature maps) from the last convolution layer. These outputs are saved to a hard-disk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These feature maps are fed into a SVM for another round of training, were another label, &quot;no object&quot; is added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also have regression model that trains based on the window coordinates that we also annotated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we have SVN and a regression model (two models) that we train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)Is the above correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Are each of these 2000 region proposals hand-labeled (correct label (cat, dog etc) or no-object) before feeding it into the SVM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Is the regression model tied into the SVM model? Basically out loss is a combination of both regression coords and SVM classification?&lt;/p&gt;&#xA;" OwnerUserId="3460" LastActivityDate="2019-02-23T09:36:14.207" Title="Having trouble understanding some of the details of R-CNN (first one)" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10834" PostTypeId="2" ParentId="10822" CreationDate="2019-02-23T00:29:30.463" Score="2" Body="&lt;p&gt;This is a common problem and does have a name. It is called &quot;&lt;a href=&quot;https://ourarchive.otago.ac.nz/handle/10523/1765&quot; rel=&quot;nofollow noreferrer&quot;&gt;catastrophic forgetting&lt;/a&gt;&quot; (link is just to a paper I found randomly when searching for the term).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What could be causing this undesirable behavior? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It happens only when using function approximation for value functions (e.g. a neural network trained to learn Q values), and is caused by the agent's own success. After a while the only samples that you will train with will be near-optimal high return cases, and a neural network will optimise the Q value approximation only for that recent data in order to get the best loss. This will usually mean much poorer predictions on unseen but different data, as the network overfits to states and actions only seen in the optimal policy. Eventually the agent will explore into an area where its predictions are way off. Then, because Q learning also uses its own predictions to &lt;em&gt;bootstrap&lt;/em&gt; new Q values, this can start a runaway feedback process where the agent starts to choose a suboptimal path through the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inside the hidden layers and weights, the neural network may have lost the ability to differentiate well between the states on the old, worse path and the newer better ones. It didn't need to differentiate, because it stopped needing to reduce loss on any data about the old states. So it will also start incorrectly associating the now poor results with the more optimal paths. It will behave at least partially as if the correct policy was set by the overfit Q predictions, but the values need adjusting - so as well as (correctly) reducing its value predictions of the suboptimal paths it has just encountered, it will also (incorrectly) reduce the value predictions of the optimal paths. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes, the swing back to receiving lower returns during this process is so strong and/or incorrectly associated with the high return states along with the low ones, that the agent never properly recovers. Other times, the agent can re-learn fully and you get random oscillations over time between optimal and non-optimal agent behaviour.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I combat it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is still an ongoing area of research. Here are a couple of things that have worked for me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Low learning rates, and defences against sudden large gradients (e.g. gradient clipping).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Regularisation. Sadly dropout seems not to work in RL, but weight decay is still useful to prevent over-fitting, and it also helps combat catastrophic forgetting because it prevents bootstrap estimates of long-unseen state/action combinations from returning radically different Q values to the rest of the system.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Keep some early experience around from when the agent was still performing badly - this allows the agent to still train with some bad cases and prevents the Q function predicting that &quot;everything is awesome&quot; because it still has examples to learn from where this is not the case.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For simple environments, such as inverted pendulum, just keeping some very early fully random behaviour in the experience replay table is enough. For instance if you have a table with 10000 observations (of &lt;span class=&quot;math-container&quot;&gt;$s, a, r, s'$&lt;/span&gt;), keep 1000 of the first experiences in that table and don't discard them when the table is full. For more complex environments, this is not so useful, as the early random behaviour is too far removed from what the agent learns. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The DQN &quot;rainbow&quot; paper uses &lt;em&gt;prioritized experience replay&lt;/em&gt; to focus on areas where Q value predictions from the NN are not matching the observations. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-02-23T00:44:14.250" LastActivityDate="2019-02-23T00:44:14.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10836" PostTypeId="2" ParentId="10833" CreationDate="2019-02-23T09:17:28.247" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes, your description is mostly aligned with the original &lt;a href=&quot;https://arxiv.org/abs/1311.2524&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. However, Figure 1 shows they use features from the second fully connected layer (not the last convolutional layer).&lt;/li&gt;&#xA;&lt;li&gt;Yes. The training is performed by exploiting hand-annotated bounding boxes. If a proposal has IoU less than 0.3, then it is considered as a negative example (cf. section 2.3).&lt;/li&gt;&#xA;&lt;li&gt;No. Appendix C explains that box regression parameters are learned separately from the SVM model.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21726" LastEditorUserId="21726" LastEditDate="2019-02-23T09:36:14.207" LastActivityDate="2019-02-23T09:36:14.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10837" PostTypeId="2" ParentId="10772" CreationDate="2019-02-23T11:43:48.630" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So, here's is the question: &lt;strong&gt;Is it true that a non-stationary policy must satisfy this condition?&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \forall i, j \in \mathbb{N}, s \in S, \pi (i, s) = \pi(j, s) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With your custom notation (which certainly isn't common, but seems reasonable)... I assume you meant to say that a &lt;strong&gt;stationary&lt;/strong&gt; policy must satisfy that condition, rather than that a &lt;strong&gt;non-stationary&lt;/strong&gt; policy must satisfy that condition. In that case, yeah, that seems correct to me. A stationary policy would satisfy that condition, and a non-stationary one wouldn't.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Wrapping back to the more usual notation, where &lt;span class=&quot;math-container&quot;&gt;$\pi(S, A)$&lt;/span&gt; denotes a probability of selecting &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (which also still covers the case of a deterministic policy, which would simply assign a probability of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to a single action, and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to all others)... I think it's still interesting to consider the case where we decide to &quot;bake&quot; a time-step counter into the state representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this notation, for two different time steps &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t'$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$t \neq t'$&lt;/span&gt;, I'd say that a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is &lt;strong&gt;stationary&lt;/strong&gt; if and only if:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(S_t, A_t) = \pi(S_{t'}, A_{t'}) \text{ if } S_t = S_{t'} \wedge A_t = A_{t'}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if we decide to include &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the state-representation, the case that &lt;span class=&quot;math-container&quot;&gt;$S_t = S_{t'}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$t \neq t'$&lt;/span&gt; will actually &lt;strong&gt;never hold within the same episode&lt;/strong&gt;, states at different time steps will always automatically be different from each other if the time step is one of the &quot;features&quot; encoded in the state. So, within a single episode a policy with a &quot;time-aware&quot; state representation will always automatically be stationary because there cannot be any repeating states. Of course, if you start looking across multiple different episodes, this can change; this is what you're doing when you write:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Once we reset the environment to evaluate the policy again, the turn, which we store in the state, will go back to 0, but the background number of evaluations won't reset and it will be 3.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you chose to &lt;strong&gt;also&lt;/strong&gt; embed that &quot;episode counter&quot; into the state representation, you would also no longer have any state repetitions at all anymore across different episodes (I don't think doing this would ever be a good idea though).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-02-23T11:43:48.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10838" PostTypeId="2" ParentId="10772" CreationDate="2019-02-23T12:12:09.787" Score="1" Body="&lt;p&gt;I think you're overthinking it. I've never seen a formalisation of the concept of &quot;stationary policy&quot; (apart from yours). However, in general, &quot;stationary&quot; means that something does not change (over time). In the context of reinforcement learning, you can interpret it in such a way that it is consistent with the context where you find this expression, unless you decide to formalise this expression (like you've tried in this question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it might be useful to differentiate between the learning phase and the &quot;inference&quot; (or &quot;behaviour&quot;) phase, even though these two might interleave in the RL context, that is, you might be using a policy (to behave in the real-world) even though you're still attempting to find the best policy (and I am not just referring to on-policy algorithms, but, in general, you might be using a policy to behave in the real-world which is sub-optimal while your learning algorithm, like Q-learning, is attempting to find the best policy). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the learning phase, the policy will keep changing over time (anyway), because you still haven't found the optimal policy. So, you could call the policy derived from the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values, during the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning algorithm, a non-stationary policy, because it keeps changing (because the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values also keep changing). However, it is often the case that these are considered different policies associated with different approximations of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also call a policy that changes in response to changes of the (dynamics of the) environment a non-stationary policy. In this case, you would call such an environment a non-stationary environment (because e.g. its transition model might keep changing over time). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that you describe when you compare the state &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; with states that encode time only arises because of your definition of state &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. You can also reset &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; when you reset the environment.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-24T13:27:28.380" LastActivityDate="2019-02-24T13:27:28.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10839" PostTypeId="1" CreationDate="2019-02-23T13:30:07.443" Score="8" ViewCount="2863" Body="&lt;p&gt;In reinforcement learning, successive states (actions and rewards) can be correlated. An &lt;a href=&quot;http://www.incompleteideas.net/lin-92.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;experience replay&lt;/a&gt; buffer was used, in the &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN architecture&lt;/a&gt;, to avoid training the neural network (NN), which represents the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, with correlated (or non-independent) data. In statistics, the i.i.d. (independently and identically distributed) assumption is often made. See e.g. &lt;a href=&quot;https://stats.stackexchange.com/q/213464/82135&quot;&gt;this question&lt;/a&gt;. &lt;a href=&quot;https://qr.ae/TUfiQ2&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is another related question. In the case of humans, if consecutive data points are correlated, we may learn slowly (because the differences between those consecutive data points are not sufficient to infer more about the associated distribution).&lt;/p&gt;&#xA;&lt;p&gt;Mathematically, why exactly do (feed-forward) neural networks (or multi-layer perceptrons) require i.i.d. data (when being trained)? Is this only because &lt;a href=&quot;https://www.ijcai.org/Proceedings/07/Papers/121.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;we use back-propagation to train NNs&lt;/a&gt;? If yes, why would back-propagation require i.i.d. data? Or is actually the optimisation algorithm (like gradient-descent) which requires i.i.d. data? Back-propagation is just the algorithm used to compute the gradients (which is e.g. used by GD to update the weights), so I think that back-propagation isn't really the problem.&lt;/p&gt;&#xA;&lt;p&gt;When using recurrent neural networks (RNNs), we apparently do not make this assumption, given that we expect consecutive data points to be highly correlated. So, why do feed-forward NNs required the i.i.d. assumption but not RNNs?&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for a rigorous answer (ideally, a proof) and not just the intuition behind it. If there is a paper that answers this question, you can simply link us to it.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T12:38:57.043" LastActivityDate="2021-04-06T07:01:47.587" Title="Why exactly do neural networks require i.i.d. data?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;statistical-ai&gt;&lt;experience-replay&gt;&lt;iid&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10842" PostTypeId="1" CreationDate="2019-02-23T17:01:19.063" Score="4" ViewCount="49" Body="&lt;p&gt;I was reading the paper by Kalchbrenner et al. titled &lt;a href=&quot;https://arxiv.org/pdf/1404.2188.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Convolutional Neural Network for Modelling Sentences&lt;/a&gt; and I am struggling to understand their definition of convolutional layer.&lt;/p&gt;&#xA;&lt;p&gt;First, let's take a step back and describe what I'd expect the 1D convolution to look like, just as defined in &lt;a href=&quot;https://arxiv.org/pdf/1408.5882.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yoon Kim (2014)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;sentence. A sentence of length n (padded where necessary) is represented as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_{1:n} = x_1 \oplus  x_2 \oplus \dots ⊕ x_n,$&lt;/span&gt; (1)&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\oplus$&lt;/span&gt; is the concatenation operator. In general, let &lt;span class=&quot;math-container&quot;&gt;$x_{i:i+j}$&lt;/span&gt; refer to the concatenation of words &lt;span class=&quot;math-container&quot;&gt;$x_i, x_{i+1}, \dots, x_{i+j}$&lt;/span&gt;. A convolution operation involves a filter &lt;span class=&quot;math-container&quot;&gt;$w \in \mathbb{R}^{hk}$&lt;/span&gt;, which is applied to a window of h words to produce a new feature. For example, a feature ci is generated from a window of words &lt;span class=&quot;math-container&quot;&gt;$x_{i:i+h−1}$&lt;/span&gt; by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c_i = f(w \cdot x_{i:i+h−1} + b)$&lt;/span&gt; (2).&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$b \in \mathbb{R}$&lt;/span&gt; is a bias term and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a non-linear function such as the hyperbolic tangent. This filter is applied to each possible window of words in the sentence &lt;span class=&quot;math-container&quot;&gt;$\{x_{1:h}, x_{2:h+1}, \dots, x_{n−h+1:n}\}$&lt;/span&gt; to produce a feature map&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c = [c_1, c_2, \dots, c_{n−h+1}]$&lt;/span&gt;, (3)&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$c \in \mathbb{R}^{n−h+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Meaning a single feature detector transforms every window from the input sequence to a single number, resulting in &lt;span class=&quot;math-container&quot;&gt;$n-h+1$&lt;/span&gt; activations.&lt;/p&gt;&#xA;&lt;p&gt;Whereas in Kalchbrenner's paper, the convolution is described as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If we temporarily ignore the pooling layer, we may state how one computes each d-dimensional column a in the matrix a resulting after the convolutional and non-linear layers. Define &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; to be the matrix of diagonals:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$M = [diag(m:,1), \dots, diag(m:,m)]$&lt;/span&gt; (5)&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; are the weights of the d filters of the wide convolution. Then after the first pair of a convolutional and a non-linear layer, each column &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the matrix &lt;strong&gt;a&lt;/strong&gt; is obtained as follows, for some index &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7EwxO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7EwxO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is a column of first order features. Second order features are similarly obtained by applying Eq. 6 to a sequence of first order features &lt;span class=&quot;math-container&quot;&gt;$a_j, \dots, a_{j+m'−1}$&lt;/span&gt; with another weight matrix &lt;span class=&quot;math-container&quot;&gt;$M'$&lt;/span&gt;. Barring pooling, Eq. 6 represents a core aspect of the feature extraction function and has a rather general form that we return to below. Together with pooling, the feature function induces position invariance and makes the range of higher-order features variable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As described in &lt;a href=&quot;https://stats.stackexchange.com/questions/345977/what-does-the-matrix-m-diagm-1-ldots-diagm-m-look-like&quot;&gt;this question&lt;/a&gt;, the matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; has dimensionalty of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$d * m$&lt;/span&gt; and the vector of concatenated &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;'s has dimensionality &lt;span class=&quot;math-container&quot;&gt;$d * m$&lt;/span&gt;. Thus the multiplication produces a vector of dimensionality d (for a single convolution of a single window!).&lt;/p&gt;&#xA;&lt;p&gt;Architecture visualization from the paper seems to confirm this understanding:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3iHN7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3iHN7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The two matrices in the second layer represent two feature maps. Each feature map has dimensionality &lt;span class=&quot;math-container&quot;&gt;$(s + m - 1) \times d$&lt;/span&gt;, and not &lt;span class=&quot;math-container&quot;&gt;$(s + m - 1)$&lt;/span&gt; as I would expect.&lt;/p&gt;&#xA;&lt;p&gt;Authors refer to a &amp;quot;conventional&amp;quot; model where feature maps have only one dimension as Max-TDNN and differentiate it from their own.&lt;/p&gt;&#xA;&lt;p&gt;As the authors point out, feature detectors in different rows are fully independent from each other until the top layer. Thus they introduce the Folding layer, which merges each pair of rows in the penultimate layer (by summation), reducing their number in half (from &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$d/2$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e1Dyn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e1Dyn.png&quot; alt=&quot;architecture&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sorry for the prolonged introduction, here are my two main questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the possible motivation for this definition of convolution (as opposed to Max-TDNN or e.g. Yoon Kim's model)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the Folding layer, why is it satisfying to only have dependence between pairs of corresponding rows? I don't understand the gain over no dependence at all.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22602" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-02-24T11:12:38.093" Title="What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10843" PostTypeId="2" ParentId="10839" CreationDate="2019-02-23T20:35:41.423" Score="10" Body="&lt;p&gt;There is an assumption behind the theory training a neural network, that also applies to many other supervised learning methods, that a training sample is representative of the data set as a whole - that it has been sampled fairly from the population that the learning algorithm has been set up to approximate.&lt;/p&gt;&#xA;&lt;p&gt;The term i.i.d. stands for &amp;quot;independent and identically distributed&amp;quot;. If you pick trajectories from RL then the sampling (per individual* record as used to train with) is not independent. Even if you choose the start of the trajectory randomly, you will have made one random choice then your remaining choices are made according to the trajectory - technically they are chosen by the policy and environment dynamics over a single step, which is usually &lt;em&gt;not&lt;/em&gt; enough to make the second, third etc steps from a trajectory fully independent random samples from the training population. To make the sampled observations independent, you should make a new random choice, and to be identically distributed that choice has to be made fairly over the whole dataset.&lt;/p&gt;&#xA;&lt;p&gt;If a subset of samples drawn for a mini-batch (or as consecutive individual samples) are correlated for an online algorithm like gradient descent, it causes a problem. For the algorithm to converge towards a globally optimal solution, it needs errors and the gradients they generate to be unbiased samples of the &amp;quot;true&amp;quot; function gradients across the loss function. Whilst correlated data from a trajectory does not do this at all - it exhibits a strong sampling bias, causing weight updates to move consistently in the wrong direction.&lt;/p&gt;&#xA;&lt;p&gt;You can demonstrate this effect trying to learn any simple function like &lt;span class=&quot;math-container&quot;&gt;$y = x^2$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;. Training a NN with randomly selected &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; pairs results in far better results than training it serially with ~2000 records starting &lt;span class=&quot;math-container&quot;&gt;$(-1.000, 1.000) (-0.999, 0.998) (-0.998, 0.996)$&lt;/span&gt; etc&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So, why do feed-forward NNs required the i.i.d. assumption but not RNNs?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;RNNs do need i.i.d. data, but the &amp;quot;unit&amp;quot; of sampling here is each sequence.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;* By &amp;quot;individually&amp;quot; I mean in terms of how they are used in the training process. A &amp;quot;unit&amp;quot; here is the smallest set of records that leads to a complete measure of loss in your optimisation.&lt;/p&gt;&#xA;&lt;p&gt;It is ok to draw longer trajectories at random if your smallest unit of measurable error is from a trajectory, as in Monte Carlo or TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) for RL, or for RNNs. There can be subtle problems with this - the trajectories still have to be i.i.d. as in they represent fair samples from the assumed population of trajectories.&lt;/p&gt;&#xA;&lt;p&gt;If you are working at the level of longer trajectories, typically this reduces systemic bias: In RL it reduces bias from initial conditions due to bootstrap process. For RNNs I am less sure what this would mean, but suspect there is some equivalent. Typically, it also increases variance, meaning you may need more training samples. There is a bias/variance trade-off, which is why setting &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; somewhere between 0 and 1 in TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) is often the optimal choice. Note in RL this is a different source of bias and variance than discussed when considering number of parameters and regularisation in supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-04-06T07:01:47.587" LastActivityDate="2021-04-06T07:01:47.587" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10844" PostTypeId="2" ParentId="10839" CreationDate="2019-02-23T20:39:08.683" Score="7" Body="&lt;p&gt;Suppose that we have some optimization criterion &lt;span class=&quot;math-container&quot;&gt;$J(x)$&lt;/span&gt;, which we aim to optimize (maybe maximize, maybe minimize), which we can compute for a single example &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an &quot;ideal world&quot;, where we have no restrictions on computation time and memory, we would generally want to run training algorithms on the complete &quot;ground truth&quot; population. For example, if we're training a model (may be a DNN, but may also be some other kind of Machine Learning model), we'd ideally train it on the complete population of all &quot;real-world&quot; images that have ever been produced, ever will be produced, or ever could be produced (and, of course, all with accurate labels). We could write our full optimization criterion as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{x \in \mathcal{P}} J(x),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{P}$&lt;/span&gt; denotes the complete population. If we have a model that we would like to train using gradient descent (such as a Neural Network), this means we have to compute the gradient with respect to some trainable parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta} \sum_{x \in \mathcal{P}} J(x).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In practice, we do not have this ideal scenario, we do not have access to the complete population &lt;span class=&quot;math-container&quot;&gt;$\mathcal{P}$&lt;/span&gt;. Often, we find ourselves approximating the population with a (hopefully rather large) dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;. This could be a collection of images like ImageNet, or in (Deep) Reinforcement Learning it could be a large experience replay buffer. &lt;strong&gt;If the dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; is an accurate representation of the complete population's distribution&lt;/strong&gt;, we can estimate the gradient of the objective we ultimately care about (computed over the complete population) by the gradient computed over the dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta} \sum_{x \in \mathcal{P}} J(x) \approx \nabla_{\theta} \sum_{x \in \mathcal{D}} J(x).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Collecting such a dataset rather than the complete population is often actually feasible in practice. If we can afford to compute the objective/gradient over such a complete dataset, that's great. &lt;strong&gt;Note that in such a case, the data can already be viewed as being i.i.d.: it's the best approximation we have of a single, complete distribution (the population distribution)&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;However, computing the objective / gradient over a large complete dataset is often still prohibitively expensive in terms of computation time. This is one of the reasons (not the only one) why we often use minibatches &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Then, we approximate the gradient of the objective of the dataset (which is itself an approximation of the gradient for the complete population) by computing it only over a minibatch &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta} \sum_{x \in \mathcal{P}} J(x) \approx \nabla_{\theta} \sum_{x \in \mathcal{D}} J(x) \approx \nabla_{\theta} \sum_{x \in \mathcal{B}} J(x).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we're repeatedly (over many, many training iterations) going to use such an approximation to take gradient descent steps, it is crucial that the gradient computed over this minibatch is actually an accurate approximation of the &quot;true&quot; gradient over the complete population; &lt;strong&gt;if it's not an accurate approximation, we're optimizing the wrong objective!&lt;/strong&gt; In my opinion, this means that we have an even stronger requirement for our minibatch than just wanting it to be identically distributed. It's not sufficient for our minibatch to be sampled from any arbitrary identical distribution. &lt;strong&gt;We want the instances in our minibatch to be sampled from one very specific identical distribution; the dataset/population distribution! If they're not all sampled from that particular distribution, they're an unreliable approximation of the objective we truly care about&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, you may wonder if it wouldn't be possible for multiple biased, &quot;unrepresentative&quot; minibatches with &quot;opposite&quot; biases to &quot;cancel each other out&quot; if they're used in different, subsequent gradient descent steps. Couldn't we first run a bunch of gradient descent steps on minibatches that only contain images of dogs, and afterwards &quot;cancel out&quot; any errors by also running a bunch of updates on minibatches containing only images of cats?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're lucky, it might work sometimes, but it's unreliable. One problem is that the first iterations using only dog images may cause you to end up in a poor area of the &quot;parameter space&quot;, which you would never have ended up in if had used a correct mix of cat and dog images all along. Escaping that poor area may be much more difficult than simply ensuring you never reach it. Another problem is that it would be extremely difficult to find the &quot;correct&quot; number of gradient descent updates to run with cat images. Run too few, you're still stuck recognising only dogs. Run too many, and you may forget how to recognise dogs at all and only start recognising cats. Things like momentum in more sophisticated optimizers will exacerbate this issue.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that I don't think the requirement for i.i.d. batches is necessarily unique to gradient descent. Other learning techniques may have the same requirement (maybe for similar reasons, maybe for different reasons).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-02-24T14:51:25.967" LastActivityDate="2019-02-24T14:51:25.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10845" PostTypeId="1" CreationDate="2019-02-23T21:44:58.993" Score="1" ViewCount="35" Body="&lt;p&gt;Is it possible to create a voice-recognition virtual assistant using local storage?&lt;/p&gt;&#xA;" OwnerUserId="22612" LastActivityDate="2019-02-23T21:44:58.993" Title="Create a virtual assistant using local storage" Tags="&lt;storage&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10846" PostTypeId="1" CreationDate="2019-02-23T23:22:55.103" Score="2" ViewCount="53" Body="&lt;h3&gt;Abstract&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I wish to design a neural network that will categorize messages based on criteria I have predefined. It should feature the ability to be proactively trained as it continues its lifecycle. This means a human can intervene in its categorization attempts and determine whether or not it was correct and have it adjust its weights accordingly (without having to retrain all over again).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Inputs&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It is know that ALL input will follow these rules:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Always of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; length&lt;/li&gt;&#xA;&lt;li&gt;All messages are transformed to eliminate unnecessary complexity&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here's a brief overview of how an example message might be processed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with a message &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;That's an interesting perspective. I think that you should consider adding more details to your point about the cat being too silly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The text is then transformed so that extra details are removed:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;thats an interesting perspective i think that you should consider adding more details to your point about the cat being too silly&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then it's converted into a vector (appending &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to reach length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;) ready to be processed by the neural network:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;[116, 104, 97, 116, 115, 32, 97, . . . , 0, 0, 0, 0]&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Ouputs&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In my network, I wish all the outputs to be weighted on how well they fit in each category. I &lt;strong&gt;need&lt;/strong&gt; multiple outputs. I'm not really focusing on one particular category per-say, but how well the message fits in all of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the input &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; I used as an example, I'd expect the outputs to look &lt;em&gt;something&lt;/em&gt; like this after my vector has fed-forward:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;Suggestive:  0.89042&lt;br&gt;Opinionated: 0.68703&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The weight values for each output indicate the strength of the category in the overall message.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From message &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;That's an interesting perspective.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Would weigh the opinionated category as &lt;span class=&quot;math-container&quot;&gt;$0.68703$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think that you should consider adding more details to your point about the cat being too silly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Would weigh the suggestive category as &lt;span class=&quot;math-container&quot;&gt;$0.89042$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Summary and Questions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'm interested in the architectural design choices of a network that would support my feature set. The main goal is to be able to train my network to categorize messages based on pre-trained (and live-trained) data. I'd like to know things like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What type of Neural Network I should use for this purpose? I've researched &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent&lt;/a&gt; networks; which have been mentioned to be good at processing sequences (ie. messages).&lt;/li&gt;&#xA;&lt;li&gt;What considerations should I account for when creating this network?.&lt;/li&gt;&#xA;&lt;li&gt;How can the overall network support live-training so I can tell my network when its wrong and have it 'correct' itself without having to retrain completely? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22614" LastActivityDate="2019-02-23T23:22:55.103" Title="How can my Neural Network categorize message strings?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10847" PostTypeId="1" AcceptedAnswerId="10851" CreationDate="2019-02-24T06:24:58.143" Score="2" ViewCount="558" Body="&lt;p&gt;I understand what an admissible heuristic is, I just don't know how to tell whether one heuristic is admissible or not. So, in this case, I'd like to know why Nilsson's sequence score heuristic function isn't admissible.&lt;/p&gt;&#xA;" OwnerUserId="21832" LastEditorUserId="2444" LastEditDate="2021-02-04T21:44:08.250" LastActivityDate="2021-02-04T21:44:08.250" Title="Why isn't Nilsson's Sequence Score an admissible heuristic function?" Tags="&lt;proofs&gt;&lt;admissible-heuristic&gt;&lt;heuristic-functions&gt;&lt;8-puzzle-problem&gt;&lt;nilssons-heuristic-function&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10849" PostTypeId="1" CreationDate="2019-02-24T08:58:13.917" Score="0" ViewCount="79" Body="&lt;p&gt;I have a complex wargame already developed in a aging Objective-C and I would like to improve the AI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have built the logic for self-play, fitness evaluation and evolution&#xA;The hard-time is the ability to run a lot of experiences of self-play with limited ressources (single Mac). Time is a factor but also memory. I am facing some random crashed after a given number of games&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I was wondering&lt;/strong&gt; &#xA;- if people have faced the same issues with a large number of runs with objective-C &#xA;- if other people have tried to use Genetic Programming or Reinforcement Learning with Objective-C or C#&lt;/p&gt;&#xA;" OwnerUserId="20886" LastEditorUserId="20886" LastEditDate="2019-02-24T09:46:58.880" LastActivityDate="2019-02-26T11:58:47.247" Title="Genetic programming with Objective-C" Tags="&lt;game-ai&gt;&lt;genetic-programming&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-17T17:39:43.717" ContentLicense="CC BY-SA 4.0" />
  <row Id="10850" PostTypeId="1" CreationDate="2019-02-24T11:14:33.063" Score="3" ViewCount="91" Body="&lt;p&gt;I have the execution of the Monte Carlo Tree Search (MCTS) below. I need to expand it, but I don't understand steps 1 and 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does it go to the first node and then make a new node, instead of going to the deepest left leaf? I thought it needs to go to the most probable leaf.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y4qe8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y4qe8.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In step 1 of MCTS, it added a new node. Now, there are 9 cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qsO5s.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qsO5s.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In step 2 of MCTS, it added a new node. Now, there are 2 cases under the first node:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9D1b6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9D1b6.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17603" LastEditorUserId="2444" LastEditDate="2019-11-19T22:47:21.307" LastActivityDate="2019-11-19T22:47:21.307" Title="Understanding an execution of the Monte Carlo tree search algorithm" Tags="&lt;algorithm&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10851" PostTypeId="2" ParentId="10847" CreationDate="2019-02-24T11:49:23.113" Score="1" Body="&lt;p&gt;I will use the 8-puzzle game to show you why Nilson's sequence score heuristic function is not admissible.  In the 8-puzzle game, you have a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; board of (numbered) squares as follows.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+---+---+---+&#xA;| 0 | 1 | 2 |&#xA;+---+---+---+&#xA;| 7 | 8 | 3 |&#xA;+---+---+---+&#xA;| 6 | 5 | 4 |&#xA;+---+---+---+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The numbers in these squares are just used to refer to the specific squares (to avoid saying the &quot;middle square&quot; or the &quot;upper-left square&quot;). So, when I say square 1, I refer to the upper-left square. In these games, you have 8 &quot;tiles&quot;. Let's denote these tiles by &lt;span class=&quot;math-container&quot;&gt;$A, B, C, D, E, F, G$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;. So, in this game, there is always one square which is free (or empty), given that there are &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; squares. The goal of this game is to reach the following configuration of tiles&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+---+---+---+&#xA;| A | B | C |&#xA;+---+---+---+&#xA;| H |   | D |&#xA;+---+---+---+&#xA;| G | F | E |&#xA;+---+---+---+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that, in the case of the 8-puzzle game, a &quot;state&quot; is a configuration of the board. So, the following two board configurations are two distinct states&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+---+---+---+&#xA;|   | A | C |&#xA;+---+---+---+&#xA;| H | B | D |&#xA;+---+---+---+&#xA;| G | F | E |&#xA;+---+---+---+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+---+---+---+&#xA;|   | C | A |&#xA;+---+---+---+&#xA;| H | B | D |&#xA;+---+---+---+&#xA;| G | F | E |&#xA;+---+---+---+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rules of the 8-puzzle are simple. You can move one tile (at a time) from its current position (or square) to another position, provided that the destination square is free. You can only move a tile horizontally and vertically (and one square at a time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will not explain in this answer how the Nilson's sequence score heuristic works. &lt;a href=&quot;https://stackoverflow.com/a/10607141/3924118&quot;&gt;Here&lt;/a&gt; is a explanation of how is used in the case of the 8-puzzle game. &lt;em&gt;You should read this explanation and make sure you understand how Nilson's heuristic works before proceeding!&lt;/em&gt; You can also find an explanation of how this heuristic works in the book &quot;Principles of Artificial Intelligence&quot; (at &lt;a href=&quot;https://books.google.ch/books?id=4JwN5DTpcqkC&amp;amp;pg=PA85&amp;amp;lpg=PA85&amp;amp;dq=Nilsson%27s+sequence+score&amp;amp;source=bl&amp;amp;ots=KA9Xb018fZ&amp;amp;sig=ACfU3U3c5yuf9bWmc42kICTLZIv65aZTOQ&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=2ahUKEwib8M76ktTgAhVM-yoKHQxXDWoQ6AEwCXoECAMQAQ#v=onepage&amp;amp;q=Nilsson&amp;#39;s%20sequence%20score&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;page 85&lt;/a&gt;), by Nils J. Nilsson (1982).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why isn't then Nilson's sequence score admissible?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A heuristic function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is admissible if &lt;span class=&quot;math-container&quot;&gt;$h(n) \leq h^*(n)$&lt;/span&gt;, for all states &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in the state space, where &lt;span class=&quot;math-container&quot;&gt;$h^*(n)$&lt;/span&gt; is the actual distance to reach the goal (which is often unknown in practice, hence the need to use heuristics to estimate such distance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that admissibility is a property that must hold for &lt;strong&gt;all&lt;/strong&gt; states. So, if we find a state where the condition above is not satisfied for the Nilson's sequence score heuristic function, then we show that the Nilson's sequence score is not admissible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us create the following state&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+---+---+---+&#xA;| A | B | C |&#xA;+---+---+---+&#xA;|   | H | D |&#xA;+---+---+---+&#xA;| G | F | E |&#xA;+---+---+---+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this state only differs from the goal state by one move: if we move the tile &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; to square &lt;span class=&quot;math-container&quot;&gt;$7$&lt;/span&gt;, then we reach the goal state. So, the actual distance to reach the goal state is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; move. But what does the Nilson's score function tell us regarding the distance of this state to the goal state? You can see from the algorithm presented &lt;a href=&quot;https://stackoverflow.com/a/10607141/3924118&quot;&gt;in this answer&lt;/a&gt; to compute the Nilson's sequence score (of a board configuration) that the score of the configuration (or state) above would be more than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (you can immediately see this because you need to multiply by &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;). Therefore, Nilson's sequence score overestimated the distance to the goal (at least for one state), thus it cannot be admissible (by definition).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-24T13:24:40.513" LastActivityDate="2019-02-24T13:24:40.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10852" PostTypeId="5" CreationDate="2019-02-24T12:00:33.780" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-11T00:57:04.757" LastActivityDate="2021-02-11T00:57:04.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10853" PostTypeId="4" CreationDate="2019-02-24T12:00:33.780" Score="0" Body="For questions related to the A* (or A) search algorithm, which is a very famous state-space search algorithm and widely taught in Computer Science and Artificial Intelligence. A* is a best-first search algorithm that is guaranteed to find the optimal solution given an admissible heuristic function, so A* is also an informed search algorithm, as opposed to e.g. depth-first search, which is an uninformed search algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-11T00:57:04.757" LastActivityDate="2021-02-11T00:57:04.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10855" PostTypeId="1" CreationDate="2019-02-24T15:56:06.650" Score="2" ViewCount="182" Body="&lt;p&gt;I am developing an AI tool for &lt;strong&gt;anomaly detection in a distributed system&lt;/strong&gt;.  The system supports an interface that combines several individual logs into a single log file generating approx. 7000 entries/min. The logs entries are partially system generated (d-Bus, IPC, ….)  and human written statements (Status not received, initialized successfully, ….). The developers use the generated log for debugging. The entries have been configured to have a similar format depending on the generated system (timestamp, ids, component, context, verbosity level, description, ….). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Background:&lt;/em&gt;&lt;br&gt;&#xA;1. The history of the identified anomalies is minimal and not archived.&lt;br&gt;&#xA;2. Not many similar event templates in log files.&lt;br&gt;&#xA;3. Software execution rules are not clearly documented.&lt;br&gt;&#xA;4. The log events are co-related.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the recommended algorithms (Statistical, NLP, ML, Neural networks) that can be used to efficiently perform pattern extraction on the entries and identify existing and new anomalous behavior?&lt;/p&gt;&#xA;" OwnerUserId="22635" LastEditorUserId="22635" LastEditDate="2019-03-02T12:38:43.420" LastActivityDate="2021-03-21T16:02:35.260" Title="Anomaly Detection in distributed system using generated log file" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10857" PostTypeId="2" ParentId="258" CreationDate="2019-02-24T16:54:49.547" Score="0" Body="&lt;p&gt;A computational model that attempts to closely mimic the human neural networks is Numenta's &lt;a href=&quot;https://numenta.org/hierarchical-temporal-memory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical temporal memory&lt;/a&gt; (which has not yet received much attention from  the machine learning community). In their models, they explicitly model and implement dendrites and other biological concepts.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-16T18:39:59.510" LastActivityDate="2019-04-16T18:39:59.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10859" PostTypeId="2" ParentId="10855" CreationDate="2019-02-24T17:47:18.337" Score="0" Body="&lt;p&gt;In the paper &quot;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0925231217309864&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised real-time anomaly detection for streaming data&lt;/a&gt;&quot; (by Subutai Ahmad, Alexander Lavin, Scott Purdy and Zuha Agha), 2017, an algorithm for anomaly detection (particularly suited for cases where a stream of data is continuously provided) is described. This algorithm is based on Numenta's Hierarchical Temporal Memory model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've actually never used it, but I know that Numenta's work is particularly suited for anomaly detection. You can have a look at it and see if it fits your needs. Have also a look at the &lt;a href=&quot;https://github.com/numenta/NAB&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numenta Anomaly Benchmark&lt;/a&gt; (NAB).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-02-24T17:47:18.337" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="10860" PostTypeId="1" CreationDate="2019-02-24T18:08:09.497" Score="1" ViewCount="113" Body="&lt;p&gt;The blocked N-queens is a variant of the N-queens problem. In the blocked N-queens problem, we also have a NxN chess board and N queens. Each square can hold at most one queen. Some squares on the board are blocked and can not hold any queens. Conditionality is that queens do not dare to attack each other. At the entrance to this problem are the queues and blocked areas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I model the blocked N queens problem as a search problem, so that I can apply search algorithms like BFS?&lt;/p&gt;&#xA;" OwnerUserId="22634" LastEditorUserId="2444" LastEditDate="2019-02-24T19:26:46.973" LastActivityDate="2019-07-25T18:03:10.387" Title="How do I model the blocked N queens problem as a search problem?" Tags="&lt;search&gt;&lt;problem-solving&gt;&lt;breadth-first-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10863" PostTypeId="2" ParentId="10860" CreationDate="2019-02-24T19:23:03.537" Score="1" Body="&lt;p&gt;In general, the process of modelling a problem as a search problem consists in creating a graph which contains nodes, which represent the possible states in your problem, and edges, which represent the relations between these states (that is, you will have an edge between nodes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; if it is possible to go from state &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to state &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and vice-versa, in your problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the 8 (or, in general, N) queens problem (or blocked 8 queens problem), the states are all possible configurations of the board (that is, all possible combinations of the positions of all the 8 queens). In the graph that represents this problem, you will have a node for all possible valid configurations, where a valid configuration is one that is allowed by the rules of the game. You will also have an edge between configurations (or states) &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; if you can go from state &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to state &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. For example, configurations where you have two queens on the same square or where you have a queen in a blocked area will not be represented in your graph as a node (or state).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the 8 queens problem, you actually have more than one solution, so you will have more than one goal state (or configuration of the board). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using the BFS, the goal will then be to find a path from the initial configuration of the queens (that is, the initial state) to one of these &lt;a href=&quot;https://en.wikipedia.org/wiki/Eight_queens_puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;18 (or 12) goal states&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the 8 queens problem, there are &lt;a href=&quot;https://en.wikipedia.org/wiki/Eight_queens_puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;4,426,165,368 possible configurations&lt;/a&gt; (so you will likely not be able to draw this graph).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-24T22:36:32.657" LastActivityDate="2019-02-24T22:36:32.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10864" PostTypeId="1" CreationDate="2019-02-24T20:03:04.503" Score="2" ViewCount="1394" Body="&lt;p&gt;I want to build a DNN model that I will later integrate into a C++ program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard that PyTorch model is hard to load it on C++ and the integration requires extra code, and it's complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have searched a bit on the internet and I have found this post: &lt;a href=&quot;https://pytorch.org/tutorials/advanced/cpp_export.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loading a PyTorch Model in C++&lt;/a&gt;. I am still unsure whether or not I will have any problems in practice if I opt for PyTorch.&lt;/p&gt;&#xA;" OwnerUserId="22526" LastEditorUserId="22526" LastEditDate="2019-02-27T20:27:57.697" LastActivityDate="2019-02-27T20:27:57.697" Title="How to export Pytorch Deep Neural Networks trained model to C++ program to use it from C++?" Tags="&lt;pytorch&gt;&lt;c++&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-04-01T14:12:07.397" ContentLicense="CC BY-SA 4.0" />
  <row Id="10865" PostTypeId="1" CreationDate="2019-02-25T00:57:21.860" Score="2" ViewCount="150" Body="&lt;p&gt;What are (good) daily life examples of SAT problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've thought about this one. The problem of placing a bunch of different kinds of glasses in a shared cabinet in such a way that some constraints would be satisfied, such as putting the longer ones in the back of the shorter ones, so it will be easier to take them when we need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a good one, or can you think of any other better one?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastEditorUserId="2444" LastEditDate="2019-11-28T23:24:43.510" LastActivityDate="2019-11-28T23:24:43.510" Title="What are daily life examples of SAT problems?" Tags="&lt;problem-solving&gt;&lt;satisfiability&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10868" PostTypeId="2" ParentId="10864" CreationDate="2019-02-25T09:08:23.080" Score="1" Body="&lt;p&gt;PyTorch now has a &lt;a href=&quot;https://pytorch.org/cppdocs/frontend.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;C++ frontend&lt;/a&gt;. I haven't tried it, but I'm sure you could use that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option, which is more production-tested, is using a message passing framework such as &lt;a href=&quot;http://zeromq.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZeroMQ&lt;/a&gt; to communicate requests and results between Python and C++ executables.&lt;/p&gt;&#xA;" OwnerUserId="16035" LastActivityDate="2019-02-25T09:08:23.080" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10869" PostTypeId="1" CreationDate="2019-02-25T09:51:23.810" Score="7" ViewCount="539" Body="&lt;p&gt;You may have heard of GPT2, a new language model. It has recently attracted attention from the general public as the foundation that published the paper, &lt;a href=&quot;https://blog.openai.com/better-language-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI&lt;/a&gt;, ironically refused to share the whole model fearing dangerous implications. Along the paper, they also published a manifesto to justify their choice: &amp;quot;Better Language Models and Their Implications&amp;quot;. And soon a lot of media were publishing articles discussing the choice and its effectiveness to actually prevent bad implications. I am not here to discuss the ethical components of this choice but the actual performance of the model.&lt;/p&gt;&#xA;&lt;p&gt;The model got my attention too and I downloaded the small model to play with. To be honest I am far from impressed by the results. Some times the first paragraph of the produced text appears to make sense, but nine times out of ten it is giberish by the first or the second sentence. Exemples given in the paper seems to be &amp;quot;Lucky&amp;quot; outputs, cherry picked by human hands. Overall, the paper may suffer from a very strong publication bias.&lt;/p&gt;&#xA;&lt;p&gt;However, most article we can read on the internet seems to take its powerfulness for granted. &lt;a href=&quot;https://www.technologyreview.com/s/612975/ai-natural-language-processing-explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The MIT technology review&lt;/a&gt; wrote:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The language model can write like a human&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.theguardian.com/technology/2019/feb/14/elon-musk-backed-ai-writes-convincing-news-fiction&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Guardian&lt;/a&gt; wrote&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When used to simply generate new text, GPT2 is capable of writing plausible passages that match what it is given in both style and subject. It rarely shows any of the quirks that mark out previous AI systems, such as forgetting what it is writing about midway through a paragraph, or mangling the syntax of long sentences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The model appears generally qualified as a &amp;quot;breakthrough&amp;quot;. These writings do not match my personal experimentation as produced texts are rarely consistent / syntactically correct.&lt;/p&gt;&#xA;&lt;p&gt;My question is: without the release of the whole model for ethical reasons, how do we know if the model is really that powerful?&lt;/p&gt;&#xA;" OwnerUserId="22654" LastEditorUserId="2444" LastEditDate="2020-09-22T11:26:05.377" LastActivityDate="2020-09-22T11:26:05.377" Title="How do we know if GPT-2 is a better language model?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10871" PostTypeId="2" ParentId="10865" CreationDate="2019-02-25T14:32:06.450" Score="1" Body="&lt;p&gt;SAT problems are decision problems that can be categorised as &lt;em&gt;NPC&lt;/em&gt;. This informally means although there has not been any solution that can solve these problem in the polynomial order, the solutions of such problems can be satisfied in &lt;span class=&quot;math-container&quot;&gt;$O(n^c)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About your question, first, you should see your problem has exponential space and cannot be solved in polynomial order; after that, you have to investigate whether its solutions can be satisfied in polynomial time or not. An easy way to model this is to consider the boolean table and an expression to satisfy. Your problem can be considered as the boolean equation and different possibilities of variables can be possible solutions that you may want to verify. &lt;/p&gt;&#xA;" OwnerUserId="11599" LastActivityDate="2019-02-25T14:32:06.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10873" PostTypeId="5" CreationDate="2019-02-25T21:26:14.200" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T20:27:39.327" LastActivityDate="2020-12-06T20:27:39.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10874" PostTypeId="4" CreationDate="2019-02-25T21:26:14.200" Score="0" Body="For questions related to the crossover (aka recombination) operator in the context of evolutionary algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T20:27:39.327" LastActivityDate="2020-12-06T20:27:39.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10875" PostTypeId="5" CreationDate="2019-02-25T21:27:23.953" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T20:15:58.577" LastActivityDate="2020-12-06T20:15:58.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10876" PostTypeId="4" CreationDate="2019-02-25T21:27:23.953" Score="0" Body="For questions about methods (or operators) to mutate individuals (or chromosomes) in the context of evolutionary algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T20:15:58.577" LastActivityDate="2020-12-06T20:15:58.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10877" PostTypeId="5" CreationDate="2019-02-25T21:29:05.543" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-25T21:29:05.543" LastActivityDate="2019-02-25T21:29:05.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10878" PostTypeId="4" CreationDate="2019-02-25T21:29:05.543" Score="0" Body="For questions related to the dynamic programming paradigm in the context of AI (and, in particular, reinforcement learning)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-25T22:00:39.950" LastActivityDate="2019-02-25T22:00:39.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10879" PostTypeId="5" CreationDate="2019-02-25T21:32:52.317" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_climbing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hill_climbing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-25T22:00:22.370" LastActivityDate="2019-02-25T22:00:22.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10880" PostTypeId="4" CreationDate="2019-02-25T21:32:52.317" Score="0" Body="For questions related to the family of algorithms called &quot;hill climbing&quot; (in the context of AI)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-02-25T22:00:53.303" LastActivityDate="2019-02-25T22:00:53.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10881" PostTypeId="5" CreationDate="2019-02-25T21:38:02.580" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-02-25T21:38:02.580" LastActivityDate="2019-02-25T21:38:02.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10882" PostTypeId="4" CreationDate="2019-02-25T21:38:02.580" Score="0" Body="For questions like &quot;What is the difference between X and Y?&quot;." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-22T17:07:49.317" LastActivityDate="2019-06-22T17:07:49.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10883" PostTypeId="5" CreationDate="2019-02-25T22:03:48.000" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Satisfiability&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Satisfiability&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-25T22:03:48.000" LastActivityDate="2019-02-25T22:03:48.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10884" PostTypeId="4" CreationDate="2019-02-25T22:03:48.000" Score="0" Body="For questions about the mathematical notion of satisfiability." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-02-25T22:03:48.000" LastActivityDate="2019-02-25T22:03:48.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10885" PostTypeId="5" CreationDate="2019-02-25T22:05:57.510" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-02-17T17:57:56.310" LastActivityDate="2021-02-17T17:57:56.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10886" PostTypeId="4" CreationDate="2019-02-25T22:05:57.510" Score="0" Body="For questions about constraint satisfaction problems (CSPs), which are usually described by a set of variables, a set of domains, and a set of constraints. For more details, see, for example, chapter 6 of the book &quot;Artificial Intelligence: A Modern Approach&quot; (3rd Edition) by Stuart Russell and Peter Norvig." OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-02-17T17:57:56.310" LastActivityDate="2021-02-17T17:57:56.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10887" PostTypeId="2" ParentId="10201" CreationDate="2019-02-26T07:37:24.623" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is usually just for memory use limitation vs speed of assessment. Larger batches evaluate faster on parallelised systems such as GPUs, but use more memory to process. Test results should be identical, with same size of dataset and same model, regardless of batch size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically you would set batch size at least high enough to take advantage of available hardware, and after that as high as you dare without taking the risk of getting memory errors. Generally there is less to gain than with training optimisation though, so it is not worth spending a huge amount of time optimising the batch size to each model you want to test. In most code I have seen, users pick a moderate &quot;safe&quot; value that will speed up testing but doesn't risk failing if you wanted to add a few layers to the model and check what that does.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-02-26T07:37:24.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10888" PostTypeId="1" CreationDate="2019-02-26T08:07:03.373" Score="1" ViewCount="1929" Body="&lt;p&gt;Why do we use the seed function in the 'Pendulum-v0' environment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L25&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L25&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22023" LastEditorUserId="22023" LastEditDate="2019-03-01T00:13:43.247" LastActivityDate="2019-12-28T22:01:40.203" Title="What is the use of the seed function in the gym Environment 'Pendulum-v0'?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;reinforce&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10890" PostTypeId="1" CreationDate="2019-02-26T09:45:47.713" Score="3" ViewCount="110" Body="&lt;p&gt;I have heard of the concepts of &lt;em&gt;learning by analogy&lt;/em&gt; (which is quite self-explanatory), &lt;em&gt;inductive learning&lt;/em&gt; and &lt;em&gt;explanation-based learning&lt;/em&gt;. I tried to learn about &lt;a href=&quot;http://users.cs.cf.ac.uk/Dave.Marshall/AI2/node144.html#SECTION000161000000000000000&quot; rel=&quot;nofollow noreferrer&quot;&gt;inductive learning&lt;/a&gt; and &lt;a href=&quot;http://users.cs.cf.ac.uk/Dave.Marshall/AI2/node148.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;explanation-based learning&lt;/a&gt;, but I don't understand them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you explain all these three concepts? What are the differences between them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A link to some explanatory article/notes/blog post are appreciated too.&lt;/p&gt;&#xA;" OwnerUserId="22673" LastEditorUserId="2444" LastEditDate="2019-03-28T14:28:09.880" LastActivityDate="2019-03-28T14:28:09.880" Title="What are the differences between learning by analogy, inductive learning and explanation based learning?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10892" PostTypeId="1" CreationDate="2019-02-26T10:23:02.940" Score="2" ViewCount="36" Body="&lt;p&gt;How to detect presence of object (highly occluded) in a scene?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are specific features (small patterns, etc), which allow to say that object is present, but it is not enough for detection for YOLO or RPCNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to detect small specific pattern in a whole image efficiently?&lt;/p&gt;&#xA;" OwnerUserId="22677" LastActivityDate="2019-02-26T10:23:02.940" Title="Presence of object (highly occluded vehicle) in a scene" Tags="&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10894" PostTypeId="2" ParentId="10849" CreationDate="2019-02-26T11:58:47.247" Score="2" Body="&lt;p&gt;I have not touched Obj-C, but I've played with evolution in PHP, which wasn't designed for that at all. If a slow script language on my 10-year old desktop PC can do it, Obj-C should be able to handle that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some tricks:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This is a game, so - I assume - you've disabled all the graphics. A headless program is ideal for training. Waste no CPU cycles on stuff you don't need!&lt;/li&gt;&#xA;&lt;li&gt;Use more threads/instances. Threads around the number of cores should be ideal.&lt;/li&gt;&#xA;&lt;li&gt;Watch out for memory leaks! Even languages with GC can have problems with this when used in a wrong way. But you'll need to fix these bugs anyway. You can monitor memory consumption when you run it, and if it increases more-or-less steadily, that's a problem. There are various tools against memleaks under different systems/languages, I guess Obj-C has something similar. (these tools log every allocation and release, and list everything at exit that's left)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22418" LastActivityDate="2019-02-26T11:58:47.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10895" PostTypeId="1" CreationDate="2019-02-26T14:01:34.993" Score="1" ViewCount="233" Body="&lt;p&gt;I want to apply the concept that exists in the &lt;a href=&quot;https://dialogflow.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dialogflow&lt;/a&gt; API in my e-commerce website.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get some references in this regard :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Tokenization&lt;/li&gt;&#xA;&lt;li&gt;Part Of Speech&lt;/li&gt;&#xA;&lt;li&gt;Named Entity Recognition&lt;/li&gt;&#xA;&lt;li&gt;Rule based &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I just saw that I just didn't understand how to implement it on the website. &#xA;so I still don't know how the truth is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;please give me a method or explanation that can help me create a chatbot for ecommerce that can give action when a user asks for a product and wants to place an order or something else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please give me some explanation or method or references :(&lt;/p&gt;&#xA;" OwnerUserId="22686" LastActivityDate="2019-03-07T07:01:03.077" Title="How to make chatbot using NLP like Dialogflow?" Tags="&lt;chat-bots&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10896" PostTypeId="2" ParentId="5769" CreationDate="2019-02-26T15:25:34.630" Score="0" Body="&lt;p&gt;Just to make two details absolutely clear:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; 2D input channels going to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; 2D output channels.  The total number of 2D &lt;span class=&quot;math-container&quot;&gt;$3\times3$&lt;/span&gt; filter weights is actually &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt;.  But how is the 3D convolution affected, i.e., if every input channel contributes one 2D layer to every output channel, then each output channel is composed initially of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; 2D layers, how are they combined?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This tends to be glossed over in almost every publication I've seen, but the key concept is the &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt; 2D output channels are interleaved with each other to form the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; output channels, like shuffled card decks, before being summed together.  This is all logical when you realize that along the channel dimensions of a convolution (which is never illustrated), you actually have a fully connected layer!  Every input 2D channel, multiplied by a unique &lt;span class=&quot;math-container&quot;&gt;$3\times 3$&lt;/span&gt; filter, yields a 2D output layer contribution to a single output channel.  Once combined, every output layer is a combination of every input layer &lt;span class=&quot;math-container&quot;&gt;$\times$&lt;/span&gt; a unique filter.  It's an all to all contribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to convince yourself of this is to imagine what happens in other scenarios and see that the computation becomes degenerate - that is, if you don't interleave and recombine the results, then the different outputs wouldn't actually do anything - they'd have the same effect as a single output with combined weights.&lt;/p&gt;&#xA;" OwnerUserId="22689" LastEditorUserId="1641" LastEditDate="2019-03-03T17:57:20.323" LastActivityDate="2019-03-03T17:57:20.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10898" PostTypeId="1" CreationDate="2019-02-26T20:44:25.120" Score="1" ViewCount="182" Body="&lt;p&gt;I have just found the paper and documentation about GAN 2.0, the new face creator from Nvidia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the website &lt;a href=&quot;https://thispersondoesnotexist.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://thispersondoesnotexist.com/&lt;/a&gt; they have used this approach to create realistic faces. Unfortunately, the website does not exist anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there another webpage demonstrating the new face creator from Nvidia?&lt;/p&gt;&#xA;" OwnerUserId="21103" LastActivityDate="2019-02-27T03:36:09.970" Title="New face generator from Nvidia" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10899" PostTypeId="2" ParentId="10898" CreationDate="2019-02-27T01:49:55.767" Score="0" Body="&lt;p&gt;There is a Youtube video: &lt;a href=&quot;https://www.youtube.com/watch?v=zrVDfX9XFpE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nvidia - AI for Generating High-Resolution Images - Tero Karras&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know about webpages.&lt;/p&gt;&#xA;" OwnerUserId="22418" LastActivityDate="2019-02-27T01:49:55.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10900" PostTypeId="2" ParentId="10898" CreationDate="2019-02-27T03:36:09.970" Score="1" Body="&lt;p&gt;You can find a &lt;a href=&quot;https://web.archive.org/web/*/thispersondoesnotexist.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;decent sized collection on archive.org&lt;/a&gt;. Just browse through the snapshots and they'll contain a few images. It probably doesn't contain every single one, but it has quite a decent set to start from. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's one I found:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9h6jSb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9h6jSb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22614" LastActivityDate="2019-02-27T03:36:09.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10903" PostTypeId="1" CreationDate="2019-02-27T06:07:37.127" Score="2" ViewCount="64" Body="&lt;p&gt;Why are the nodes (or neurons) in neural networks depicted as circles?&lt;/p&gt;&#xA;&lt;p&gt;What is the difference between a circle and a box in diagrams of neural networks?&lt;/p&gt;&#xA;" OwnerUserId="22707" LastEditorUserId="2444" LastEditDate="2021-01-12T00:53:19.603" LastActivityDate="2021-01-12T00:53:19.603" Title="Why are the nodes (or neurons) in neural networks depicted as circles?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10904" PostTypeId="1" AcceptedAnswerId="10905" CreationDate="2019-02-27T06:17:02.450" Score="5" ViewCount="1453" Body="&lt;p&gt;I have already implemented a relatively simple DQN on Pacman.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to clearly understand the difference between a DQN and the techniques used by AlphaGo zero/AlphaZero and I couldn't find a place where the features of both approaches are compared. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also sometimes, when reading through blogs, I believe different terms might in fact be the same mathematical tool which adds to the difficulty of clearly understanding the differences. For example, variations of DQN e.g. Double DQN also uses two networks like alpha zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has someone a good reference regarding this question ? Be it a book or an online ressource.&lt;/p&gt;&#xA;" OwnerUserId="18845" LastEditorUserId="2444" LastEditDate="2019-02-28T22:46:00.120" LastActivityDate="2019-03-13T09:19:57.707" Title="What is the difference between DQN and AlphaGo Zero?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;alphazero&gt;&lt;deep-rl&gt;&lt;alphago-zero&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10905" PostTypeId="2" ParentId="10904" CreationDate="2019-02-27T08:08:12.353" Score="6" Body="&lt;p&gt;DQN and AlphaZero do not share much in terms of implementation.&lt;/p&gt;&#xA;&lt;p&gt;However, they are based on the same Reinforcement Learning (RL) theoretical framework. If you understand terms like MDP, reward, return, value, policy, then these are interchangeable between DQN and AlphaZero. When it comes to implementation, and what each part of the system is doing, then this is less interchangeable. For instance two networks you have read about in AlphaZero are the policy network and value network. Whilst double DQN alternates between two value networks.&lt;/p&gt;&#xA;&lt;p&gt;Probably the best resource that summarises both DQN and AlphaZero, and explains how they extend the basic RL framework in different ways is Sutton &amp;amp; Barto's &lt;a href=&quot;http://www.incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning: An Introduction (second edition)&lt;/a&gt; - Chapter 16 sections 5 and 6 cover the designs of DQN Atari, AlphaGo and AlphaZero in some depth.&lt;/p&gt;&#xA;&lt;p&gt;In brief:&lt;/p&gt;&#xA;&lt;h2&gt;DQN Atari&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is model-free&lt;/li&gt;&#xA;&lt;li&gt;Uses an action value estimator for &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; values, based on a Convolutional Neural Network (CNN)&lt;/li&gt;&#xA;&lt;li&gt;Uses experience replay and temporarily frozen target network to stabilise learning process&lt;/li&gt;&#xA;&lt;li&gt;Uses a variety of tricks to simplify and standardise the state description and reward structure so that the exact same design and hyperparameters work across multiple games, demonstrating that it is a general learner.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;AlphaZero&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is model based (although &lt;em&gt;some&lt;/em&gt; of the learning is technically model-free, based on samples of play)&lt;/li&gt;&#xA;&lt;li&gt;Uses a policy network (estimating &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;) and a state value network (estimating &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;), based on CNNs. In practice for efficiency the NN for these share many layers and parameters, so how many &amp;quot;networks&amp;quot; there are depends how you want to count them.&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The earlier AlphaGo version had 4 separate networks, 3 variations of policy network - used during play at different stages of planning - and one value network.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Is designed around self-play&lt;/li&gt;&#xA;&lt;li&gt;Uses Monte Carlo Tree Search (MCTS) as part of estimating returns - MCTS is a planning algorithm critical to AlphaZero's success, and there is no equivalent component in DQN&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-03-13T09:19:57.707" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10906" PostTypeId="2" ParentId="10903" CreationDate="2019-02-27T08:19:43.973" Score="1" Body="&lt;p&gt;I guess the reason is that we call them nodes and nodes are usually depicted using circles in graph theory. We model them as graphs for both forward and backward paths.&lt;/p&gt;&#xA;" OwnerUserId="11599" LastActivityDate="2019-02-27T08:19:43.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10907" PostTypeId="2" ParentId="10798" CreationDate="2019-02-27T09:00:51.383" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is the action selection random with Sarsa? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A policy could be stochastic. In the case of SARSA, it is stochastic because of the use of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Isn't it on-policy and therefore ϵ-greedy?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't quite understand the question. SARSA is on-policy evaluation with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy. Q-learning is off-policy evaluation with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy is just a way to turn an action-value function into a policy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Because Expected-Sarsa is off-policy the experience it learns from can be from any policy ... How can Exected-Sarsa learning from such policy be generally better than normal Sarsa learning from an ϵ-greedy policy, especially with the same amount of experience?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is unfair to compare different natures of experiences because off-policy experience contains &lt;em&gt;less&lt;/em&gt; useful information. Thus, both SARSA and Expected SARSA should use their own on-policy experience for comparison. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While Expected SARSA update step guarantees to reduce the expected TD error, SARSA could only achieve that in expectation (taking many updates with sufficiently small learning rate). Through this perspective, there is little doubt that Expected SARSA should be better.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Probably more general: How can on-policy and off-policy algorithms be compared in such way (e.g. through variance) even though their concepts and assumptions are so different?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Same as the previous answer, it is unfair to compare them without the same quality of experiences.&lt;/p&gt;&#xA;" OwnerUserId="9793" LastActivityDate="2019-02-27T09:00:51.383" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10908" PostTypeId="2" ParentId="10663" CreationDate="2019-02-27T09:13:36.550" Score="1" Body="&lt;p&gt;It might seem to give the same update direction but would it converge to desirable policy parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.&lt;/p&gt;&#xA;" OwnerUserId="9793" LastActivityDate="2019-02-27T09:13:36.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10909" PostTypeId="1" CreationDate="2019-02-27T09:25:53.430" Score="5" ViewCount="239" Body="&lt;p&gt;In the context of Deep Q Network, a target network is usually utilized. The target network is a slow changing network with a changing rate as its hyperparameter. This includes both replacement update every &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; iterations and slowly update every iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the rate is hard to fine tune manually, is there an alternative technique that can eliminate the use of target network or at least makes it less susceptible to the changing rate?&lt;/p&gt;&#xA;" OwnerUserId="9793" LastEditorUserId="9793" LastEditDate="2019-02-27T12:19:41.997" LastActivityDate="2019-04-10T11:42:11.207" Title="Is there an alternative to the use of target network?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10910" PostTypeId="1" CreationDate="2019-02-27T10:45:11.330" Score="0" ViewCount="896" Body="&lt;p&gt;I am trying to solve part b of the exercise 3.6 (page 113) from the book &lt;a href=&quot;https://faculty.psau.edu.sa/filedownload/doc-7-pdf-a154ffbcec538a4161a406abf62f5b76-original.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;More specifically, I need to give a complete problem formulation (that is precise enough to be implemented) for the following problem.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A 3-foot-tall monkey is in a room where some bananas are suspended from the 8-foot ceiling. He would like to get the bananas. The room contains two stackable, movable, climbable 3-foot-high crates.&lt;/p&gt;&#xA;&lt;p&gt;Give the initial state, goal test, successor function, and cost function for each of the following. Choose a formulation that is precise enough to be implemented.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="19448" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-01T12:43:25.623" Title="How can I solve part b of exercise 3.6 from the book &quot;Artificial Intelligence: A Modern Approach&quot;?" Tags="&lt;ai-design&gt;&lt;search&gt;&lt;homework&gt;&lt;norvig-russell&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10911" PostTypeId="1" CreationDate="2019-02-27T11:42:50.690" Score="1" ViewCount="187" Body="&lt;p&gt;Suppose I want to build a neural network regression model that takes one input and return one output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the training data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.1 =&amp;gt; 0.1&#xA;0.2 =&amp;gt; 0.2&#xA;0.1 =&amp;gt; -0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will see that there are 2 inputs &lt;code&gt;0.1&lt;/code&gt; that matches to different output values &lt;code&gt;0.1&lt;/code&gt; and &lt;code&gt;-0.1&lt;/code&gt;. So what will happen with most machine learning models is that they will predict the average when &lt;code&gt;0.1&lt;/code&gt; is fed to the model. E.g. the output of &lt;code&gt;0.1&lt;/code&gt; will be &lt;code&gt;(0.1 + (-0.1))/2 = 0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this &lt;code&gt;0&lt;/code&gt; as an average answer is an incorrect answer. I want the model to be telling me that the input is ambiguous/insufficient to infer the output. Ideally, the model would report it as a form of confidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I report predictability confidence from the input?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The application that I find very useful in many areas is that I could then later ask the model to show me inputs that are easy to predict and inputs that are ambiguous. This would make me able to collect the data that are making sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way I know is to train the model then check the error on each training data, if it's high then it probably means that the input is ambiguous. But if you know any other papers or better techniques, I would be appreciated to know that!&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-09-11T00:44:00.763" Title="How to make machine learning model that reports ambiguity of the input?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;probability&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10912" PostTypeId="2" ParentId="10082" CreationDate="2019-02-27T11:47:12.553" Score="8" Body="&lt;p&gt;Generally researchers (&lt;a href=&quot;https://cs.adelaide.edu.au/%7Ezbyszek/Papers/trading.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ghandar&lt;/a&gt; et al, &lt;a href=&quot;https://cs.adelaide.edu.au/%7Ezbyszek/papers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michalewicz&lt;/a&gt;, &lt;a href=&quot;https://www.researchgate.net/publication/3903854_A_genetic_fuzzy_expert_system_for_stock_market_timing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lam&lt;/a&gt;) have used the profit or return on investment (ROI) as a reward (fitness) function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ROI = \frac{ \left[\sum_{t=1}^T (Price_t - sc) \times I_s(t) \right] - \left[ \sum_{t=1}^T (Price_t + bc) \times I_b(t) \right] }{ \left[ \sum_{t=1}^T (Price_t + bc) \times I_b(t) \right] }$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$I_b(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I_s(t)$&lt;/span&gt; are equal to one if a rule signals a buy and sell, respectively, and zero otherwise; &lt;span class=&quot;math-container&quot;&gt;$sc$&lt;/span&gt; represents the selling cost and &lt;span class=&quot;math-container&quot;&gt;$bc$&lt;/span&gt; the buying cost. ROI is the difference between final bank balance and starting bank balance after trading.&lt;/p&gt;&#xA;&lt;p&gt;You are correct, that the machine learning algorithm will then be influenced by spikes just before a sell.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.aiecon.org/conference/2008/CIEF/Evaluation%20of%20Fitness%20Functions%20for%20Evolved%20Stock%20Market%20Forecasting/s98028571.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nicholls et al&lt;/a&gt; showed that using the average profit or area under the trade resulted in better performing trading rules. This approach was used by &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.6892&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schoreels et al&lt;/a&gt;. This approach focuses on being in the market to capitalize on profit. It does not penalize the trading rule when it is in the market and the market is going down. The accumulated asset value (AAV) is defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$AAV = \frac{\sum_{i=1}^N [(Price_s - sc) - (Price_b + bc)]}{N}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is a buy and sell trading event, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of buy and sell events, &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; the day the sale took place, and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the day the purchase took place.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://hdl.handle.net/2263/68477&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nicholls MSc thesis&lt;/a&gt; [available April 2019] showed that the fitness function used by &lt;a href=&quot;https://www.cs.montana.edu/courses/spring2007/536/materials/Lopez/genetic.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Allen and Karjalainen&lt;/a&gt; is the preferred fitness function when evolving trading rules for the JSE using evolutionary programs.&lt;/p&gt;&#xA;&lt;p&gt;Allen and Karjalainen used a fitness function based on the compounded excess returns over the buy-and-hold (buy the first day, sell the last day) strategy. The excess return is given by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Delta r = r - r_{bh}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the continuously compounded return of the trading rule is computed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r = \sum_{t=1}^T r_i I_b(t) + \sum_{t=1}^T r_f I_s(t) + n\log\left(\frac{1-c}{1+c'}\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the return for the buy-and-hold strategy is calculated as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r_{bh} = \sum_{t=1}^T r_t + \log\left(\frac{1-c}{1+c'}\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the above,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r_i = \log P_t - \log P_{t-1}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is the daily close price for a given day &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; denotes the one-way transaction cost; &lt;span class=&quot;math-container&quot;&gt;$r_f$&lt;/span&gt; is the risk free cost when the trader is not trading, &lt;span class=&quot;math-container&quot;&gt;$I_b(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I_s(t)$&lt;/span&gt; are equal to one if a rule signals buy and sell, respectively, and zero otherwise; &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; denotes the number of trades and &lt;span class=&quot;math-container&quot;&gt;$r_{bh}$&lt;/span&gt; represents the returns of a buy-and-hold, while &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; represents the returns of the trader.&lt;/p&gt;&#xA;&lt;p&gt;A fixed trading cost of &lt;span class=&quot;math-container&quot;&gt;$c = 0.25\%$&lt;/span&gt; of the transaction was defined but this could be anything like a STATE fee + Brocker fee + Tax, and might even be 2 different values, one for buying and one for selling. Which was the approach used by Nicholls. The continuously compounded return function rewards an individual when the share value is dropping and the individual is out of the market. The continuously compounded return function penalises the individual when the market is rising and the individual is out of the market.&lt;/p&gt;&#xA;&lt;p&gt;I would recommend that you use the compounded excess return over the buy and hold strategy as your reward function.&lt;/p&gt;&#xA;" OwnerUserId="20508" LastEditorUserId="38401" LastEditDate="2020-07-06T05:03:32.030" LastActivityDate="2020-07-06T05:03:32.030" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10913" PostTypeId="2" ParentId="10064" CreationDate="2019-02-27T12:02:55.730" Score="1" Body="&lt;p&gt;I would use the proposed fitness function defined in your other &lt;a href=&quot;https://ai.stackexchange.com/questions/10082/suitable-reward-function-for-trading-buy-and-sell-orders&quot;&gt;StackExchange&lt;/a&gt; question. I would then inflate the buying price used in the equation by \$100, and leave the selling price as the price the shares were sold for. This would only reward a trading rule when profit is greater than \$100.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to minimise spike it might be best to normalise the data going into your learning algorithm, for example use an x day moving average instead of the actual share price.&lt;/p&gt;&#xA;" OwnerUserId="20508" LastActivityDate="2019-02-27T12:02:55.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10914" PostTypeId="2" ParentId="10911" CreationDate="2019-02-27T12:34:52.303" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://scottlocklin.wordpress.com/2016/12/05/predicting-with-confidence-the-best-machine-learning-idea-you-never-heard-of/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Predicting with confidence: the best machine learning idea you never heard of&lt;/a&gt; by Scott Locklin might provide you an idea.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The name of this basket of ideas is “conformal prediction.”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22544" LastActivityDate="2019-02-27T12:34:52.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10915" PostTypeId="1" CreationDate="2019-02-27T13:30:04.210" Score="2" ViewCount="31" Body="&lt;p&gt;I am reading the paper &quot;&lt;a href=&quot;https://i.stack.imgur.com/qZGnM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation&lt;/a&gt;&quot;, where the tangent vector is calculated for the given curve &lt;span class=&quot;math-container&quot;&gt;$s(P,\alpha)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$\alpha=0$&lt;/span&gt; by differentiating with respect to &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial s(P,\alpha)}{\partial\alpha}$&lt;/span&gt;. For the curve, I have taken one &lt;span class=&quot;math-container&quot;&gt;$2D$&lt;/span&gt; image and I am rotating it with matrix &lt;span class=&quot;math-container&quot;&gt;$R=\left[\matrix{cos(\alpha)\space -sin(\alpha)\\sin(\alpha) \space\space\space\space\space cos(\alpha)} \right]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As my image is fixed, the curve is just a function of &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;. Therefore, to find the tangent vector, what I am doing is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I am rotating the image by the matrix &lt;span class=&quot;math-container&quot;&gt;$R^{'}$&lt;/span&gt; which is &lt;span class=&quot;math-container&quot;&gt;$R^{'}=\left[\matrix{-sin(\alpha)\space -cos(\alpha)\\cos(\alpha) \space\space\space\space\space -sin(\alpha)} \right]$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This rotates the image by &lt;span class=&quot;math-container&quot;&gt;$90$&lt;/span&gt; degree, which is not the expected result.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have done the same exercise by differentiating numerically and I am getting the expected answer which is as follows:&lt;br&gt;&lt;a href=&quot;https://i.stack.imgur.com/oqYUc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oqYUc.png&quot; alt=&quot;Picture &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;=0&quot;&gt;&lt;/a&gt; &#xA;&lt;br&gt;&lt;a href=&quot;https://i.stack.imgur.com/qZGnM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qZGnM.png&quot; alt=&quot;Tangent vector plot at &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;=0&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, help me to understand my mistake in taking derivative of the matrix and multiplying it with image.&lt;/p&gt;&#xA;" OwnerUserId="22712" LastEditorUserId="2444" LastEditDate="2019-02-27T14:30:14.817" LastActivityDate="2019-02-27T14:30:14.817" Title="Calculating tangent vector of curve s(P,$\alpha$) at given point $\alpha$ = 0" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;math&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10916" PostTypeId="1" CreationDate="2019-02-27T15:49:03.373" Score="3" ViewCount="53" Body="&lt;p&gt;How do you efficiently choose the hyper-parameters of a neural network (e.g. the learning rate, number of layer, weights, etc.)?&lt;/p&gt;&#xA;" OwnerUserId="22717" LastEditorUserId="2444" LastEditDate="2019-02-27T16:47:36.857" LastActivityDate="2019-02-27T16:47:36.857" Title="How do you efficiently choose the hyper-parameters of a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10917" PostTypeId="1" CreationDate="2019-02-27T16:15:43.010" Score="1" ViewCount="29" Body="&lt;p&gt;I hope this question is not too broad or general. I have a very large set of images all of which contain text (some have more, some less). All of them have been tagged as containing, say, English text or Korean. I wonder if convolutional neural networks would be a good approach to classify these images as containing English vs. Korean. Or is there any existing literature/method that does this already. Crucially though, I am not interested in &quot;understanding&quot; the text, so this is not an NLP task but, I suppose, a task of classifying orthographies in the images. &lt;/p&gt;&#xA;" OwnerUserId="22719" LastActivityDate="2021-03-20T22:08:15.793" Title="Using convnet to classify language of text contained in images" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10918" PostTypeId="2" ParentId="10916" CreationDate="2019-02-27T16:34:53.750" Score="1" Body="&lt;p&gt;In deep learning era, there are two possible choices. Caveat approach and Panda approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Caviar Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this approach, it is supposed that you have a very powerful cluster system that enables you to run different models simultaneously on different nodes. In this manner, you construct a d-dimensional space which each corresponds to a special hyperparameter. Then you can have a grid approach to partition the space and for each intersection of the grids, you can have a possible initialisation although there are some better ways in order to find better initialisations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Panda Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is widely used among students due to using desktop computers. In this approach, you usually try to find an initialisation of hyperparameters based on your experience or the other architectures' which are available and you try to refine them step by step by using cross-validation.&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;&#xA;&lt;p&gt;To answer your question, the efficiency depends on your computing power. For more details, you can take a look at the &lt;a href=&quot;https://www.coursera.org/learn/deep-neural-network&quot; rel=&quot;nofollow noreferrer&quot;&gt;contents of the third week&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11599" LastEditorUserId="11599" LastEditDate="2019-02-27T16:43:42.053" LastActivityDate="2019-02-27T16:43:42.053" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10919" PostTypeId="1" AcceptedAnswerId="10920" CreationDate="2019-02-27T17:27:13.727" Score="2" ViewCount="106" Body="&lt;p&gt;What is the fundamental difference between NN for classifying data and generating data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most examples show how neural networks can be used to classify data. Like is it an image of a dog or a cat. However, there are applications where NN are used to create images and even write short stories. &lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="19413" LastEditDate="2019-02-28T08:19:09.827" LastActivityDate="2019-02-28T08:19:09.827" Title="What is the fundamental difference between neural networks for classifying and generating data" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10920" PostTypeId="2" ParentId="10919" CreationDate="2019-02-27T18:14:34.940" Score="2" Body="&lt;p&gt;The formal name for this difference is &amp;quot;generative&amp;quot; vs &amp;quot;discriminative&amp;quot; models.&lt;/p&gt;&#xA;&lt;p&gt;By default, a supervised learning process using a simple feed-forward neural network and a set of training data with expected answers will produce a discriminative model. It is hard to use such a model to generate content directly.&lt;/p&gt;&#xA;&lt;p&gt;Differences between discriminative and generative models tend to be at the architecture level or deeper, although a few generative techniques are simple variants or re-purposing of discriminative networks. There is no single &amp;quot;fundamental&amp;quot; difference in terms of NN design. About the only common theme is that generative models are more complex and harder to work with than discriminative ones.&lt;/p&gt;&#xA;&lt;h2&gt;Images&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Probably the most popular currently for image generation are &lt;a href=&quot;https://skymind.ai/wiki/generative-adversarial-network-gan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks or GANs&lt;/a&gt;. These train a Generator to create an image from random inputs, alongside a Discriminator that tries to spot fakes compared to real images. Training them together results in a generator network that gets progressively better at creating fake data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also suitable for image generation are &lt;a href=&quot;http://kvfrans.com/variational-autoencoders-explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Autoencoders (VAEs)&lt;/a&gt; which learn essentially by compressing a set of images down into a small representation, and as a result become able to &amp;quot;uncompress&amp;quot; similar representations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are also VAEGANs, which combine VAEs and GANs&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are other models which generate examples from a dataset. For example, Restricted Boltzmann Machines (RBMs) are similar to neural networks, but the neurons fire randomly, and RBMs require a different training process to NNs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GANs, VAEs, VAEGANs, RBMs can be used to generate data in any simple non-sequential datasets, they are not restricted to just images, &lt;a href=&quot;https://www.theverge.com/2018/12/17/18144356/ai-image-generation-fake-faces-people-nvidia-generative-adversarial-networks-gans&quot; rel=&quot;nofollow noreferrer&quot;&gt;but recent work with GANs for example has excelled there&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For a different kind of generator, you can look at &lt;a href=&quot;https://hackernoon.com/dl06-deepdream-with-code-5f735052e21f&quot; rel=&quot;nofollow noreferrer&quot;&gt;how Deep Dream works&lt;/a&gt;. The interesting thing here is that it is a modification of a feed forward network that has been trained by supervised learning. Essentially to run Deep Dream, you take an existing image and change it so that it maximises some internal part(s) of the existing neural network, &amp;quot;training&amp;quot; the image as if it were a set of weights.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The difference between Deep Dream and other generative techniques is that it does not generate content that could be in the training set - it is no good for creating &amp;quot;realistic&amp;quot; images.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Text&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Recurrent Neural Networks, and specifically Long Short-Term Memory (LSTM) networks can be used to model word sequences by training them to predict the next word. Because the resulting model is probabilistic (e.g. chance of next work being &amp;quot;Hello&amp;quot; is ~0.001%) then a simple method to make this generative is to sample from the predictions randomly, and then feeding them back into the sequence to see what it will predict next. There is &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a great blog post about this technique by Andrei Karpathy&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Audio and Music&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://deepmind.com/blog/wavenet-generative-model-raw-audio/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind's WaveNet architecture&lt;/a&gt; is similar to a CNN predicting the next samples of audio from a given input. It can be used in speech or music generation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;LSTMs are a popular choice here, as well as for language models.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;One important caveat. Nearly all these models are hard to understand, work with and train, compared to simpler supervised techniques. Probably the easiest to get to grips with if you want to understand them well enough to implement your own versions, are Deep Dream, VAEs and LSTMs.&lt;/p&gt;&#xA;&lt;p&gt;It is also worth noting that there are many other ways to generate content than neural networks. For instance, sound and music generation has a long history completely independent of recent AI developments.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-02-27T18:32:11.667" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10922" PostTypeId="1" CreationDate="2019-02-27T19:13:19.600" Score="3" ViewCount="91" Body="&lt;p&gt;How would a quantum computer potentially facilitate artificial consciousness, assuming it is possible? &lt;/p&gt;&#xA;" OwnerUserId="22725" LastEditorUserId="2444" LastEditDate="2019-02-28T15:39:07.467" LastActivityDate="2019-02-28T15:39:07.467" Title="How would a quantum computer potentially facilitate artificial consciousness, assuming it is possible?" Tags="&lt;philosophy&gt;&lt;research&gt;&lt;theory&gt;&lt;artificial-consciousness&gt;&lt;quantum-computing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10923" PostTypeId="2" ParentId="4456" CreationDate="2019-02-27T19:17:26.970" Score="3" Body="&lt;h2&gt;Model-Free RL&lt;/h2&gt;&#xA;&lt;p&gt;In Model-Free RL, the agent does not have access to a model of the environment. By environment I mean a function which predicts state transition and rewards.&lt;/p&gt;&#xA;&lt;p&gt;As of the time of writing, model-free methods are more popular and have been researched extensively.&lt;/p&gt;&#xA;&lt;h2&gt;Model-Based RL&lt;/h2&gt;&#xA;&lt;p&gt;In Model-Based RL, the agent has access to a model of the environment.&lt;/p&gt;&#xA;&lt;p&gt;Main advantage is that this allows the agent to plan ahead by thinking ahead. Agents distill the results from planning ahead into a learned policy. A famous example of Model-Based RL is &lt;a href=&quot;https://arxiv.org/abs/1712.01815&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The main downside is that many times a ground-truth representation of the environment is not usually available.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Below is a non-exhaustive taxonomy of RL algorithms, which may help you to visualize better the RL landscape.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oxKqm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oxKqm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14390" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-02-27T19:17:26.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10924" PostTypeId="1" CreationDate="2019-02-28T03:56:42.890" Score="2" ViewCount="2804" Body="&lt;p&gt;The use of target network is to reduce the chance of value divergence which could happen with off-policy samples trained with &lt;em&gt;semi-gradient&lt;/em&gt; objectives. In Deep Q network, semi-gradient TD is used and with experience replay the training could diverge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Target network is a slow changing network designed to slowly track the main value network. In &lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mnih 2013&lt;/a&gt;, it was designed to match the main network every &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; steps. There is another way which slowly updates the weight in the direction to match the main network every step. To someone, the latter is called &lt;em&gt;Polyak updates&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done some very limited experiments and seen that with the same update rate, e.g. &lt;span class=&quot;math-container&quot;&gt;$N=10$&lt;/span&gt;, Polyak update would update with the rate of 0.1, I usually see Polyak updates to give &lt;em&gt;smoother&lt;/em&gt; progress and converge faster. My experiments are by no means conclusive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would thence ask if it is known which one to perform better, converge faster or has smoother progress, in a wider range of tasks and settings?&lt;/p&gt;&#xA;" OwnerUserId="9793" LastActivityDate="2019-02-28T12:54:08.330" Title="In DQN, updating target network every N steps or slowly update every step is better?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10925" PostTypeId="2" ParentId="10922" CreationDate="2019-02-28T12:15:58.340" Score="1" Body="&lt;p&gt;Well, this is quite deep question. I would recommend you to have a look on the UK physicist Roger Penrose: &lt;a href=&quot;https://en.wikipedia.org/wiki/Roger_Penrose&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Roger_Penrose&lt;/a&gt;&#xA;He has some books on the theme of consciousness and quantum mechanics. There are however critics of his ideas. One of them is that the quantum effects are too small to direct any considerable influence on the brain. &lt;/p&gt;&#xA;" OwnerUserId="22741" LastActivityDate="2019-02-28T12:15:58.340" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10926" PostTypeId="2" ParentId="10924" CreationDate="2019-02-28T12:54:08.330" Score="1" Body="&lt;p&gt;Most of papers and my experience support hard update, once per N steps. N is usually very bug, range in between 10^4 to 10^6. DQN training is slow. But that depend on the problem. If you DQN converge with soft update with weight ~.1 (N~10) your problem could be very simple.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-02-28T12:54:08.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10927" PostTypeId="2" ParentId="10904" CreationDate="2019-02-28T13:04:20.260" Score="1" Body="&lt;p&gt;You can actually combine AlphaZero-like approach with DQN:&#xA;&lt;a href=&quot;https://github.com/imagry/aleph_star&quot; rel=&quot;nofollow noreferrer&quot;&gt;A* + DQN&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-02-28T13:04:20.260" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10928" PostTypeId="2" ParentId="5274" CreationDate="2019-02-28T19:20:47.457" Score="2" Body="&lt;p&gt;In addition to Jaden's excellent answer &quot;no one is trying to actually make a “conscious” AI because we don’t know what that word means yet&quot; I'd like to add that the word &quot;yet&quot; there is highly optimistic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's highly problematic and likely impossible to distinguish between a conscious being and a being that behaves exactly as if it was conscious. Philosophers have been struggling with that for centuries; some even espoused solipsism, which is a &quot;I live in the Matrix&quot; philosophy. In particular, how can you tell whether your childhood friend or your spouse or anybody else is a conscious being rather than an embodiment of AI that acts exactly as a conscious being would?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible, of course, to go &quot;if it walks like a duck and quacks as a duck then it's a duck&quot; way. In that case a Turing Test passing AI would be automatically considered conscious. However, most people wouldn't accept the duck criteria of consciousness; otherwise they would very soon have to call their Alexa operated household appliances conscious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My two cents are basically the same as Jaden's, except that I'm more pessimistic about ever understanding what consciousness is. &lt;/p&gt;&#xA;" OwnerUserId="22751" LastEditorUserId="22751" LastEditDate="2019-02-28T21:31:00.040" LastActivityDate="2019-02-28T21:31:00.040" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10930" PostTypeId="1" CreationDate="2019-02-28T23:57:07.183" Score="1" ViewCount="92" Body="&lt;p&gt;I have a bayesian network, which has the following data: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(S) = 0.07$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(A) = 0.01$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F \mid S,A) = 1.0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F \mid S, \lnot A) = 0.7$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F \mid \lnot S, A) = 0.9$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F \mid \lnot S, \lnot A) = 0.1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I'm asked to get &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S)$&lt;/span&gt;. Is it possible? How can I deduce it? &lt;/p&gt;&#xA;" OwnerUserId="22760" LastEditorUserId="2444" LastEditDate="2019-03-01T09:52:04.583" LastActivityDate="2019-04-03T12:01:40.307" Title="Is it possible to compute $P( F \mid S )$ given $P(F \mid S,A)$, $P(F \mid S, \lnot A)$?" Tags="&lt;math&gt;&lt;bayes&gt;&lt;bayesian-network&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10932" PostTypeId="2" ParentId="10909" CreationDate="2019-03-01T04:24:38.987" Score="6" Body="&lt;p&gt;&lt;em&gt;I have done some research and would like to share.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Generally to eliminate the use of target network one needs to show that training would be &lt;em&gt;stable&lt;/em&gt; under off-policy semi-gradient.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two approaches that might work:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Experience reweighting&lt;/li&gt;&#xA;&lt;li&gt;Constrained optimization&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Experience reweighting&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the simplest idea is to use importance sampling ratio (&lt;a href=&quot;https://www.researchgate.net/publication/2371944&quot; rel=&quot;nofollow noreferrer&quot;&gt;Precup 2001&lt;/a&gt;) to multiply each sample. This would correct the off-policy sample distribution to be on-policy distribution. It has been shown (&lt;a href=&quot;http://jmlr.org/papers/volume17/14-488/14-488.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton 2016&lt;/a&gt;) that on-policy samples lead to stability for semi-gradient. However, this line of work that corrects for sample distribution has high variance and does not work well in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another line of work aimed to partially correct the distribution only to the extent that would be provably stable is called &lt;strong&gt;Emphatic TD&lt;/strong&gt; (&lt;a href=&quot;http://jmlr.org/papers/volume17/14-488/14-488.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton 2016&lt;/a&gt;). The distribution is still mostly off-policy but is proved to be stable under linear function approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Constrained optimization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general wisdom comes from the fact that updating a value would also alter the target value since they share the same set of parameters. This problem is called over-generalization. To reduce this, &lt;a href=&quot;http://www.cs.utexas.edu/~pstone/Papers/bib2html-links/NIPS17-ishand.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Durugkar&lt;/a&gt; suggests that constraining the target Q-value after parameter update to be steady helps reduce divergence. &lt;a href=&quot;https://arxiv.org/abs/1903.08894&quot; rel=&quot;nofollow noreferrer&quot;&gt;Achiam 2019&lt;/a&gt; provides a good overview of the problem, and suggests a way to make sure that the update is non-expansion (which should prevent divergence). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sidenote&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some works have shown convergence under off-policy samples but only in tabular case. It takes much more to show that it is stable in function approximation (even a linear one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Works like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Q(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;)&lt;/strong&gt; (&lt;a href=&quot;https://arxiv.org/abs/1602.04951&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harutyunyan 2016&lt;/a&gt;), which augments the value function with a correction term&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Retrace(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;)&lt;/strong&gt; (&lt;a href=&quot;http://arxiv.org/abs/1606.02647&quot; rel=&quot;nofollow noreferrer&quot;&gt;Munos 2016&lt;/a&gt;), which extends the former by truncating the importance sampling ratio&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;are shown to work &lt;em&gt;only&lt;/em&gt; in tabular case. &lt;/p&gt;&#xA;" OwnerUserId="9793" LastEditorUserId="9793" LastEditDate="2019-04-10T11:42:11.207" LastActivityDate="2019-04-10T11:42:11.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10933" PostTypeId="1" CreationDate="2019-03-01T06:07:50.643" Score="1" ViewCount="35" Body="&lt;p&gt;i am new to machine learning. i'm trying to identify driving pattern through accelerometer and gyroscope sensor. i have been collecting the data of both the sensors and have been storing them in .csv extension. i am not able to identify a pattern in the datasets since it has a lot of datas. i have three independent variables of accelerometer and with that i need to identify the sudden acceleration and sudden breaking and i have three independent variables of gyroscope and i need to identify aggressive turns. can you suggest as to how i need to analyse the pattern and find a algorithm which suits my requirement. this is how the dataset is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L380x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L380x.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="22736" LastEditDate="2019-03-01T09:40:47.947" LastActivityDate="2019-03-01T09:40:47.947" Title="identifying pattern in datasets" Tags="&lt;classification&gt;&lt;datasets&gt;&lt;unsupervised-learning&gt;&lt;autonomous-vehicles&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10934" PostTypeId="1" CreationDate="2019-03-01T08:48:01.860" Score="1" ViewCount="36" Body="&lt;p&gt;Is there a method (not a table of recommendations!) that could tell me what activation function to choose if the outputs of the neural network have some interpretation? For example, these can be the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution, and so on.&lt;/p&gt;&#xA;" OwnerUserId="11359" LastEditorUserId="2444" LastEditDate="2020-11-06T02:33:03.377" LastActivityDate="2020-11-06T02:33:03.377" Title="How do I choose the activation function of the output layer of a neural network (based on theoretical motivations)?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10935" PostTypeId="1" CreationDate="2019-03-01T08:53:51.203" Score="1" ViewCount="42" Body="&lt;p&gt;I'm currently doing a research project related to Distributed Tracing. My research has led me to a point where I think ML might be suited for our problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for papers that are similar to this (even if they have other applications):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to match packets exiting a black-box system (outputs) to packets that enter a black box (inputs). I can do that easily in a non concurrent setting which should help  me grow a training set (maybe for supervised learning), but I need an algorithm that, in a concurrent setting, can separate the different request &quot;flows&quot; if you will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest thing to what I'm looking for is &quot;Aguilera, Marcos K., et al. &quot;Performance debugging for distributed systems of black boxes.&quot; ACM SIGOPS Operating Systems Review. Vol. 37. No. 5. ACM, 2003.&quot; but it's mostly suited for finding the dependency graph of the system, which I already know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="22763" LastActivityDate="2020-03-25T14:02:25.300" Title="Machine Learning papers for matching packets to request flows" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10936" PostTypeId="1" CreationDate="2019-03-01T09:15:44.577" Score="0" ViewCount="152" Body="&lt;p&gt;A generative adversarial network (GAN) takes a vector of numbers as input and generates an image, based on the input. Each element of the vector causes some feature of the image to change, but the mapping between input and output is not clear, as often happens in deep learning. What is the best way to study the correlation between the vector elements and the output image features? The first approach that comes to mind is to manually change every element and check the result, however I am not sure that this is the best solution.&lt;/p&gt;&#xA;" OwnerUserId="16671" LastActivityDate="2019-03-01T10:07:47.017" Title="How to study the correlation between GAN's input vector and output image" Tags="&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10938" PostTypeId="2" ParentId="10930" CreationDate="2019-03-01T09:50:02.363" Score="0" Body="&lt;p&gt;I do not think you can compute &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S=s)$&lt;/span&gt; only using your given probabilities (and no further independence assumption between your random variables). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, note that &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S=s)$&lt;/span&gt; is the probability of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; (being equal to one of the values that &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; can attain), given that the value of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Not also that &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S)$&lt;/span&gt; is a shorthand for &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S=s)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, by the &lt;em&gt;law of total probability&lt;/em&gt;, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(A) &#xA;&amp;amp;= P(A, B) + P(A, B^c) \\&#xA;&amp;amp;= P(A|B)P(B) + P(A|B^c)P(B^c) \\&#xA;&amp;amp;= P(B|A)P(A) + P(B|A^c)P(A^c)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$B^c$&lt;/span&gt; is the complement of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (in case &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are sets or events).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in your specific case, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(F \mid S) &#xA;&amp;amp;= P(F \mid S, A)P(A \mid S) + P(F \mid S, \lnot A)P(\lnot A \mid S) \\&#xA;&amp;amp;= P(F \mid S, A) \frac{P(A, S)}{P(S)} + P(F \mid S, \lnot A)\frac{P(\lnot A, S)}{P(S)} \\&#xA;&amp;amp;= \frac{P(F, S, A)}{P(A, S)} \frac{P(A, S)}{P(S)} + \frac{P(F,S, \lnot A)}{P(\lnot A, S)}\frac{P(\lnot A, S)}{P(S)} \\&#xA;&amp;amp;= \frac{P(F, S, A)}{P(S)} + \frac{P(F,S, \lnot A)}{P(S)} \\&#xA;&amp;amp;= \frac{1}{P(S)} (P(F, S, A) + P(F,S, \lnot A) ) \\&#xA;&amp;amp;= \frac{P(F, S)}{P(S)} &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S, A), P(S)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(F \mid S, \lnot A)$&lt;/span&gt;, but we do not have &lt;span class=&quot;math-container&quot;&gt;$P(A, S)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(F, S, A)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$P(F, S)$&lt;/span&gt; (and I think we cannot retrieve them from your given probabilities).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I will be happy to be corrected, if this conclusion is wrong. Maybe I'm not seeing another way of computing it now.)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-01T13:34:34.267" LastActivityDate="2019-03-01T13:34:34.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10939" PostTypeId="2" ParentId="10936" CreationDate="2019-03-01T10:07:47.017" Score="1" Body="&lt;p&gt;Let me first provide a brief introduction first &lt;/p&gt;&#xA;&#xA;&lt;p&gt;GANs as VAEs are generative models which means they learn exactly what you described: to map a typically small dimensional vector/tensor into a higher dimensional one, which in your case is (interpreted as) an image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These 2 generative models differ in the actual learning strategy which results typically in making GAN outperform VAE in realistic image generation (but it’s better not to get into the reasons for this here)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in principle the Generator Network in GAN works in a similar way to the Decoder Network in VAE so as their &lt;strong&gt;output space&lt;/strong&gt; is interpreted as an &lt;strong&gt;image&lt;/strong&gt;, you can interpret their &lt;strong&gt;input space&lt;/strong&gt; as a sort of &lt;strong&gt;semantic space&lt;/strong&gt; hence the values you are setting define the semantic of the output image (in fact you observe the features change)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to say the generative model learns a  &lt;strong&gt;Semantic to Appearance Mapping&lt;/strong&gt; hence it works the opposite way of the classifier model which learns an &lt;strong&gt;Appearance to Semantic Mapping&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately as it typically happens in NN this mapping are definitely hard to understand for humans, nevertheless this is an active area of research &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So after this introduction, aimed at giving you an essential understanding of the problem, I can suggest you some papers to get into more details like this one &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1811.10597&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN Dissection: Visualizing and Understanding Generative Adversarial Networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case you found it not that clear, I’m planning to write a summary of this paper and publish it on my Medium so you could check it there (I’ll update this answer or add a comment to notify about it)&lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-03-01T10:07:47.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10940" PostTypeId="1" AcceptedAnswerId="10957" CreationDate="2019-03-01T11:15:59.197" Score="0" ViewCount="820" Body="&lt;p&gt;I've pieced together this A3C w/ PPO Gym Pendulum example, but I'm finding after a while, when attempting to get the action from the model, I get a NaN return:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = self.sess.run(self.sample_op, {self.tfs: s})[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It runs okay for a while, but then errors. To me that implies perhaps a invalid update happens at some point that corrupts the model. But I've put debugging output in the code and everything appears to be fine - it's not submitting any NaNs or outliers to the model as far as I can see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a bit of playing around I find that if I comment out the code to update the actor model then the code executes fine. Obviously it doesn't learn much, but it appears it's &lt;em&gt;something&lt;/em&gt; to do with this update:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[self.sess.run(self.atrain_op, {self.tfs: s, self.tfa: a, self.tfadv: adv}) for _ in range(UPDATE_STEP)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone see what the problem might be?  I've been playing around with this for hours and it's got me stumped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In code example below I set &lt;code&gt;N_WORKERS = 1&lt;/code&gt; so the debug is easier to read, but if you want to hit the error faster then increase that figure to the number of CPU cores you have.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot;&#xA;Dependencies:&#xA;tensorflow 1.8.0&#xA;gym 0.9.2&#xA;&quot;&quot;&quot;&#xA;&#xA;import tensorflow as tf&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import gym, threading, queue, math&#xA;&#xA;EP_MAX = 600&#xA;EP_LEN = 200&#xA;N_WORKER = 1                # parallel workers&#xA;GAMMA = 0.9                 # reward discount factor&#xA;A_LR = 0.0001               # learning rate for actor&#xA;C_LR = 0.0001               # learning rate for critic&#xA;MIN_BATCH_SIZE = 64         # minimum batch size for updating PPO&#xA;UPDATE_STEP = 15            # loop update operation n-steps&#xA;EPSILON = 0.2               # for clipping surrogate objective&#xA;GAME = 'Pendulum-v0'&#xA;&#xA;env = gym.make(GAME)&#xA;S_DIM = env.observation_space.shape[0]&#xA;A_DIM = 1 # not available in pendulum env.action_space.n&#xA;&#xA;&#xA;class PPONet(object):&#xA;    def __init__(self):&#xA;        self.sess = tf.Session()&#xA;        self.tfs = tf.placeholder(tf.float32, [None, S_DIM], 'state')&#xA;        self.tfa = tf.placeholder(tf.float32, [None, ], 'action')&#xA;        self.tfadv = tf.placeholder(tf.float32, [None, 1], 'advantage')        &#xA;&#xA;        # critic        &#xA;        with tf.variable_scope('critic'):&#xA;            w_init = tf.random_normal_initializer(0., .1)&#xA;            lc = tf.layers.dense(self.tfs, 200, tf.nn.relu, kernel_initializer=w_init, name='layer1-critic')&#xA;            self.v = tf.layers.dense(lc, 1)&#xA;&#xA;        with tf.variable_scope('ctrain'):&#xA;            self.tfdc_r = tf.placeholder(tf.float32, [None, 1], 'discounted_r')&#xA;            self.advantage = self.tfdc_r - self.v&#xA;            self.closs = tf.reduce_mean(tf.square(self.advantage))&#xA;            self.ctrain_op = tf.train.AdamOptimizer(C_LR).minimize(self.closs)&#xA;&#xA;        # actor&#xA;        pi, pi_params = self._build_anet('pi', trainable=True)&#xA;        oldpi, oldpi_params = self._build_anet('oldpi', trainable=False)&#xA;&#xA;        with tf.variable_scope('sample_action'):&#xA;            self.sample_op = tf.squeeze(pi.sample(1), axis=0)       # choosing action&#xA;&#xA;        with tf.variable_scope('update_oldpi'):&#xA;            self.update_oldpi_op = [oldp.assign(p) for p, oldp in zip(pi_params, oldpi_params)]&#xA;&#xA;        with tf.variable_scope('loss'):&#xA;            with tf.variable_scope('surrogate_pp'):&#xA;                ratio = pi.prob(self.tfa) / oldpi.prob(self.tfa)&#xA;                surr = ratio * self.tfadv&#xA;&#xA;            self.aloss = -tf.reduce_mean(tf.minimum(        # clipped surrogate objective&#xA;                surr,&#xA;                tf.clip_by_value(ratio, 1. - EPSILON, 1. + EPSILON) * self.tfadv))&#xA;&#xA;        with tf.variable_scope('atrain'):&#xA;            self.atrain_op = tf.train.AdamOptimizer(A_LR).minimize(self.aloss)&#xA;&#xA;        self.sess.run(tf.global_variables_initializer())&#xA;&#xA;    def update(self):&#xA;        global GLOBAL_UPDATE_COUNTER&#xA;        while not COORD.should_stop():&#xA;            if GLOBAL_EP &amp;lt; EP_MAX:&#xA;                UPDATE_EVENT.wait()                     # wait until get batch of data&#xA;                self.sess.run(self.update_oldpi_op)     # copy pi to old pi&#xA;                data = [QUEUE.get() for _ in range(QUEUE.qsize())]      # collect data from all workers&#xA;                data = np.vstack(data)&#xA;                s, a, r = data[:, :S_DIM], data[:, S_DIM: S_DIM + 1].ravel(), data[:, -1:]&#xA;                adv = self.sess.run(self.advantage, {self.tfs: s, self.tfdc_r: r})&#xA;&#xA;                print(&quot;Updating: s: {}, a: {}, r: {}, adv: {}&quot;.format(s, a, r, adv))&#xA;&#xA;                # update actor and critic in a update loop&#xA;                [self.sess.run(self.atrain_op, {self.tfs: s, self.tfa: a, self.tfadv: adv}) for _ in range(UPDATE_STEP)] #ERR &#xA;                [self.sess.run(self.ctrain_op, {self.tfs: s, self.tfdc_r: r}) for _ in range(UPDATE_STEP)]&#xA;                UPDATE_EVENT.clear()        # updating finished&#xA;                GLOBAL_UPDATE_COUNTER = 0   # reset counter&#xA;                ROLLING_EVENT.set()         # set roll-out available&#xA;&#xA;    def _build_anet(self, name, trainable): # Build the current &amp;amp; hold structure for the policies &#xA;        with tf.variable_scope(name):&#xA;            l1 = tf.layers.dense(self.tfs, 200, tf.nn.relu, trainable=trainable)&#xA;            mu = 2 * tf.layers.dense(l1, A_DIM, tf.nn.tanh, trainable=trainable, name = 'mu_'+name)&#xA;            sigma = tf.layers.dense(l1, A_DIM, tf.nn.softplus, trainable=trainable,name ='sigma_'+name )&#xA;            norm_dist = tf.distributions.Normal(loc=mu, scale=sigma) # Loc is the mean&#xA;        params = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope=name) # Collects the weights of the layers l1, mu / 2, sigma&#xA;        return norm_dist, params    &#xA;&#xA;    def choose_action(self, s):&#xA;        s = s[np.newaxis, :]&#xA;        print(&quot;Action s: {}&quot;.format(s))&#xA;        a = self.sess.run(self.sample_op, {self.tfs: s})[0]&#xA;        if math.isnan(a):&#xA;            print(&quot;Action is NaN - stopping&quot;)&#xA;            exit()&#xA;        return np.clip(a, -2, 2)&#xA;&#xA;    def get_v(self, s):&#xA;        if s.ndim &amp;lt; 2: s = s[np.newaxis, :]&#xA;        return self.sess.run(self.v, {self.tfs: s})[0, 0]&#xA;&#xA;&#xA;class Worker(object):&#xA;    def __init__(self, wid):&#xA;        self.wid = wid&#xA;        self.env = gym.make(GAME).unwrapped&#xA;        self.ppo = GLOBAL_PPO&#xA;&#xA;    def work(self):&#xA;        global GLOBAL_EP, GLOBAL_RUNNING_R, GLOBAL_UPDATE_COUNTER&#xA;        while not COORD.should_stop():&#xA;            s = self.env.reset()            &#xA;            ep_r = 0&#xA;            buffer_s, buffer_a, buffer_r = [], [], []&#xA;            for t in range(EP_LEN):&#xA;                if not ROLLING_EVENT.is_set():                  # while global PPO is updating&#xA;                    ROLLING_EVENT.wait()                        # wait until PPO is updated&#xA;                    buffer_s, buffer_a, buffer_r = [], [], []   # clear history buffer, use new policy to collect data&#xA;&#xA;                a = self.ppo.choose_action(s)&#xA;                s_, r, done, _ = self.env.step(a)&#xA;                print(&quot;Step returns: s_: {}, r: {}, done: {}&quot;.format(s_, r, done))&#xA;                #self.env.render()&#xA;                buffer_s.append(s)&#xA;                buffer_a.append(a)&#xA;                buffer_r.append((r+8)/8)    # normalize reward, find to be useful&#xA;                s = s_&#xA;                ep_r += r&#xA;&#xA;                GLOBAL_UPDATE_COUNTER += 1                      # count to minimum batch size, no need to wait other workers&#xA;                if t == EP_LEN - 1 or GLOBAL_UPDATE_COUNTER &amp;gt;= MIN_BATCH_SIZE or done:&#xA;                    if done:&#xA;                        v_s_ = 0                                # end of episode&#xA;                    else:&#xA;                        v_s_ = self.ppo.get_v(s_)&#xA;&#xA;                    discounted_r = []                           # compute discounted reward&#xA;                    for r in buffer_r[::-1]:&#xA;                        v_s_ = r + GAMMA * v_s_&#xA;                        discounted_r.append(v_s_)&#xA;                    discounted_r.reverse()&#xA;&#xA;                    bs, ba, br = np.vstack(buffer_s), np.vstack(buffer_a), np.array(discounted_r)[:, None]&#xA;                    buffer_s, buffer_a, buffer_r = [], [], []&#xA;                    QUEUE.put(np.hstack((bs, ba, br)))          # put data in the queue&#xA;                    if GLOBAL_UPDATE_COUNTER &amp;gt;= MIN_BATCH_SIZE:&#xA;                        ROLLING_EVENT.clear()       # stop collecting data&#xA;                        UPDATE_EVENT.set()          # globalPPO update&#xA;&#xA;                    if GLOBAL_EP &amp;gt;= EP_MAX:         # stop training&#xA;                        COORD.request_stop()&#xA;                        break&#xA;&#xA;                    if done: break&#xA;&#xA;            # record reward changes, plot later&#xA;            if len(GLOBAL_RUNNING_R) == 0: GLOBAL_RUNNING_R.append(ep_r)&#xA;            else: GLOBAL_RUNNING_R.append(GLOBAL_RUNNING_R[-1]*0.9+ep_r*0.1)&#xA;            GLOBAL_EP += 1&#xA;            print('{0:.1f}%'.format(GLOBAL_EP/EP_MAX*100), '|W%i' % self.wid,  '|Ep_r: %.2f' % ep_r,)&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    GLOBAL_PPO = PPONet()&#xA;    UPDATE_EVENT, ROLLING_EVENT = threading.Event(), threading.Event()&#xA;    UPDATE_EVENT.clear()            # not update now&#xA;    ROLLING_EVENT.set()             # start to roll out&#xA;    workers = [Worker(wid=i) for i in range(N_WORKER)]&#xA;&#xA;    GLOBAL_UPDATE_COUNTER, GLOBAL_EP = 0, 0&#xA;    GLOBAL_RUNNING_R = []&#xA;    COORD = tf.train.Coordinator()&#xA;    QUEUE = queue.Queue()           # workers putting data in this queue&#xA;    threads = []&#xA;    for worker in workers:          # worker threads&#xA;        t = threading.Thread(target=worker.work, args=())&#xA;        t.start()                   # training&#xA;        threads.append(t)&#xA;    # add a PPO updating thread&#xA;    threads.append(threading.Thread(target=GLOBAL_PPO.update,))&#xA;    threads[-1].start()&#xA;    COORD.join(threads)&#xA;&#xA;    # plot reward change and test&#xA;    plt.plot(np.arange(len(GLOBAL_RUNNING_R)), GLOBAL_RUNNING_R)&#xA;    plt.xlabel('Episode'); plt.ylabel('Moving reward'); plt.ion(); plt.show()&#xA;    env = gym.make('CartPole-v0')&#xA;    while True:&#xA;        s = env.reset()&#xA;        for t in range(1000):&#xA;            env.render()&#xA;            s, r, done, info = env.step(GLOBAL_PPO.choose_action(s))&#xA;            if done:&#xA;                break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-03-04T10:07:18.733" Title="Getting NaN from A3C PPO model" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;actor-critic-methods&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2019-03-18T16:22:23.587" ContentLicense="CC BY-SA 4.0" />
  <row Id="10941" PostTypeId="2" ParentId="2598" CreationDate="2019-03-01T11:42:58.187" Score="1" Body="&lt;p&gt;There is a fundamental difference between Generative Models and Discriminative Models, to simplify it we could say that &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;given &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; Semantic Space (low dimensionality) and&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;given &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; Appearance Space (high dimensionality, e.g. space of &lt;span class=&quot;math-container&quot;&gt;$W \times H$&lt;/span&gt; images)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;the Discriminator learns the function &lt;span class=&quot;math-container&quot;&gt;$f(A) \rightarrow S$&lt;/span&gt; and the Generator learns the function &lt;span class=&quot;math-container&quot;&gt;$g(S) \rightarrow A$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to this formalism we can say that &lt;span class=&quot;math-container&quot;&gt;$g \neq f^{-1}$&lt;/span&gt; and a simple reason for this is &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is not invertible because it is not injective, in fact 2 different horror movie posters will be assigned the same label “horror movie” (assuming the network works well :) )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you have to find another way to define a mapping from &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and this kind of job is related to the Generative Models &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly GAN are state of the art in image generation but technically they are not the only choice, e.g. you could use a VAE as the Decoder Subnetwork is a generative model &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However from a practical point of view do not expected VAE results would that good: you’ll probably get blurred stuff like this one &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://jaan.io/images/variational-autoencoder-faces.jpg&quot; alt=&quot;VAE1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so go for GAN but beware that training them properly is definitely not easy &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-03-01T11:42:58.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10942" PostTypeId="1" CreationDate="2019-03-01T11:46:15.743" Score="2" ViewCount="413" Body="&lt;p&gt;Usually, when I evaluate() a model, I would get a single loss that is already averaged over all samples. How do I get the loss per each sample and return all of them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. if my dataset has 100 samples, I want to get 100 losses, for each of the sample instead of 1 averaged loss.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastEditorUserId="20819" LastEditDate="2019-12-19T08:42:40.770" LastActivityDate="2021-01-12T10:05:30.603" Title="How do I get multiple loss per sample in keras evaluate?" Tags="&lt;keras&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ClosedDate="2021-01-12T10:16:40.157" ContentLicense="CC BY-SA 4.0" />
  <row Id="10943" PostTypeId="2" ParentId="10930" CreationDate="2019-03-01T12:49:24.773" Score="0" Body="&lt;p&gt;I believe you can deduce it. Using the product rule:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(x,y) = p(x\mid y)p(y)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F\mid S) = \frac{P(F,S)}{P(S)}$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have &lt;span class=&quot;math-container&quot;&gt;$P(S)$&lt;/span&gt; and we do not have &lt;span class=&quot;math-container&quot;&gt;$P(F,S)$&lt;/span&gt;, but we can use the addition rule:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(x) = \sum_\limits y p(x, y)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F, S) = P(F,S,A) + P(F,S,\lnot A)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;first term on the right side, using the product rule, is:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F,S,A) = P(F\mid S,A)P(S,A)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; are not conditionally dependent so we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F,S,A) = P(F\mid S, A)P(S)P(A)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the second term on the right side, following the same logic, we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F,S,\lnot A) = P(F\mid S, \lnot A)P(S)P(\lnot A)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have all the needed members so in the end we have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(F\mid S) = \frac{P(F\mid S, A)P(S)P(A) + P(F\mid S, \lnot A)P(S)P(\lnot A)}{P(S)}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-03-01T12:49:24.773" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10944" PostTypeId="1" AcceptedAnswerId="10945" CreationDate="2019-03-01T14:09:32.353" Score="11" ViewCount="2150" Body="&lt;p&gt;I'm interested in knowing whether there exist any neural network, that solves (with  &gt;=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.&lt;/p&gt;&#xA;" OwnerUserId="22365" LastEditorUserId="2444" LastEditDate="2019-03-01T15:58:35.497" LastActivityDate="2019-03-01T17:43:18.420" Title="Are there neural networks with very few nodes that decently solve non-trivial problems?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10945" PostTypeId="2" ParentId="10944" CreationDate="2019-03-01T15:35:52.330" Score="13" Body="&lt;p&gt;Even if it’s impossible to answer this question properly, as &lt;em&gt;non trivial&lt;/em&gt; is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1804.02171&quot; rel=&quot;noreferrer&quot;&gt;Smallest Neural Network to Learn the Ising Criticality&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you have a general idea of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ising_model&quot; rel=&quot;noreferrer&quot;&gt;Ising Model&lt;/a&gt; I think the problem of identifying the critical temperature from a data driven perspective can be considered as &lt;em&gt;non trivial&lt;/em&gt; and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper &lt;a href=&quot;https://www.nature.com/articles/nphys4035&quot; rel=&quot;noreferrer&quot;&gt;Machine learning phases of matter&lt;/a&gt; from 2017, to only 2 Hidden Neurons &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just my cents: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;reducing the neurons, while keeping good performance, should help in terms of &lt;strong&gt;neural processing interpretability&lt;/strong&gt; which is notoriously obscure and its complexity grows (exponentially) with the number of neurons&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="1963" LastEditDate="2019-03-01T17:43:18.420" LastActivityDate="2019-03-01T17:43:18.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10946" PostTypeId="2" ParentId="10917" CreationDate="2019-03-01T18:17:44.937" Score="0" Body="&lt;p&gt;This sounds like a fairly straightforward task, with low risks. I think the proper term is that you're you are trying to detect the &lt;em&gt;script&lt;/em&gt;, which would be either Latin ( &quot;English&quot;) or Hangul (&quot;Korean&quot;). The chance is that you end up learning &lt;em&gt;fonts&lt;/em&gt;, though. &lt;/p&gt;&#xA;" OwnerUserId="16378" LastActivityDate="2019-03-01T18:17:44.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10947" PostTypeId="1" AcceptedAnswerId="11044" CreationDate="2019-03-01T18:45:46.870" Score="4" ViewCount="203" Body="&lt;p&gt;I was reading an article on &lt;a href=&quot;https://chatbotsjournal.com/why-i-should-should-not-invest-in-chatbot-df952cbabf10&quot; rel=&quot;nofollow noreferrer&quot;&gt;Medium&lt;/a&gt; and wanted to make it clear whether a bot created on IBM Watson is an intelligent one or unintelligent.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Simply put, there are 2 types of chatbots — unintelligent ones that act using predefined conversation flows (algorithms) written by developers building them and intelligent ones that use machine learning to interact with users.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22676" LastActivityDate="2019-03-06T01:39:18.747" Title="Does IBM Watson use machine learning?" Tags="&lt;machine-learning&gt;&lt;chat-bots&gt;&lt;watson&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10948" PostTypeId="1" AcceptedAnswerId="10955" CreationDate="2019-03-01T19:33:32.927" Score="1" ViewCount="98" Body="&lt;p&gt;The reinforcement learning paradigm has the aim to determine the optimal actions for a robot. A typical example is a maze finding robot, but reinforcement learning can also be used for training a robot to play the pong game. The principle is based on a reward function. If the robot is able to solve a problem, he gets a score from the underlying game engine. The score can be positive, if the robot reaches the end of a maze, or it can be negative, if he is colliding with an obstacle. The principle itself is working quite well, that means for simpler applications it is possible to train a robot to play a game with reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chatbots are a different category of artificial intelligence. They are working not with actions but with natural language. Person #1 is opening a dialogue with “Hi, I'm Alice”, while person #2 is responding with “Nice to meet you”. What is missing here is an underlying game which is played. There is no reward available for printing out a certain sentence. In some literature the problem of language grounding was discussed seriously, but with an unclear result. It seems, that a classical game for example pong, and a chatbot conversation doesn't have much in common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to combine Reinforcement Learning with chatbot design? The problem is, that a speech-act should be connected to a reward. That means, a well formulated sentence gets +10 points but a weak sentence gets -10 points. How can this be evaluated?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2019-03-01T22:07:48.297" Title="Is reinforcement learning possible for chatbots?" Tags="&lt;reinforcement-learning&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10949" PostTypeId="2" ParentId="7109" CreationDate="2019-03-01T20:27:16.387" Score="2" Body="&lt;p&gt;Basically, if you read the full paper (especially, the abstract and the section 7), you find that the main accomplishment remains a marginal contribution on top of dropout.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you see the empirical results on Table 5 (of the page 5) of the maxout's &lt;a href=&quot;https://arxiv.org/pdf/1302.4389.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;, you find that the misclassification rate is only very, very slightly lower than that of dropouts. (2.47 % instead of 2.78%)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That could explain the relatively lower interest in the work.&lt;/p&gt;&#xA;" OwnerUserId="20745" LastActivityDate="2019-03-01T20:27:16.387" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10950" PostTypeId="1" AcceptedAnswerId="10962" CreationDate="2019-03-01T20:46:51.720" Score="4" ViewCount="69" Body="&lt;p&gt;The following plot shows error function output based on system weights.&#xA;Two equal local minima are shown in green pointers. Note that the red dots are not related to the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the right one generalize better compared to the left one? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My assumption is that for the right minimum if the weights change, the overall error increases less compared to the left minimum point. Would this somehow mean the system does better generalization if the right one is chosen as the optimum minimum?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pDEFF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pDEFF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22779" LastEditorUserId="22779" LastEditDate="2019-03-01T21:47:14.563" LastActivityDate="2019-03-02T10:37:56.403" Title="Could error surface shape be useful to detect which local minima is better for generalization?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;loss-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10951" PostTypeId="1" AcceptedAnswerId="10968" CreationDate="2019-03-01T20:56:32.490" Score="1" ViewCount="72" Body="&lt;p&gt;I find myself wondering if there exists a data structure with the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stores information&lt;/li&gt;&#xA;&lt;li&gt;Conforms to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Encoding_specificity_principle&quot; rel=&quot;nofollow noreferrer&quot;&gt;encoding specificity principle&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22781" LastEditorUserId="2444" LastEditDate="2019-10-22T21:10:57.417" LastActivityDate="2019-10-22T21:13:05.710" Title="Does a data structure that models the encoding specificity principle in memory exist?" Tags="&lt;classification&gt;&lt;biology&gt;&lt;memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10952" PostTypeId="2" ParentId="10947" CreationDate="2019-03-01T20:59:07.990" Score="0" Body="&lt;p&gt;A quick glance at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Watson_(computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM Watson wikipedia page&lt;/a&gt; reveals that it does indeed use machine learning. Watson is a complex computing system that uses a variety of cutting edge techniques and concepts such as natural language processing and machine learning.&lt;/p&gt;&#xA;" OwnerUserId="22781" LastActivityDate="2019-03-01T20:59:07.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10953" PostTypeId="1" AcceptedAnswerId="10967" CreationDate="2019-03-01T21:30:38.590" Score="1" ViewCount="135" Body="&lt;p&gt;I want to try and compare different optimization methods in some datasets. I know that in scikit-learn there are some corresponding functions for the grid and random search optimizations. However, I also need a package (or multiple ones) for different recent Bayesian optimization methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any good and stable ones to use? Which packages do you recommend? (If any recent for grid/random search, it is also okay.)&lt;/p&gt;&#xA;" OwnerUserId="22784" LastEditorUserId="2444" LastEditDate="2020-03-12T15:08:58.013" LastActivityDate="2020-03-12T15:08:58.013" Title="Are there Python packages for recent Bayesian optimization methods?" Tags="&lt;python&gt;&lt;hyperparameter-optimization&gt;&lt;resource-request&gt;&lt;bayesian-optimization&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-12-19T19:03:22.043" ContentLicense="CC BY-SA 4.0" />
  <row Id="10954" PostTypeId="1" AcceptedAnswerId="10961" CreationDate="2019-03-01T21:47:19.977" Score="1" ViewCount="52" Body="&lt;p&gt;The following plot shows error function output based on system weights.&#xA;Two equal local minima are shown in green pointers. Note that the red dots are not related to the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the amount of convex in the local minima, is there any way to opt between these two local minima?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7rMRC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7rMRC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22779" LastEditorUserId="2444" LastEditDate="2019-03-01T22:45:34.327" LastActivityDate="2019-03-02T09:06:44.530" Title="Which local minima to choose according to the shape of the error surface?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10955" PostTypeId="2" ParentId="10948" CreationDate="2019-03-01T22:07:48.297" Score="2" Body="&lt;p&gt;The problem is indeed that the 'rules' of conversations are not as fixed as the rules for games. However, you can make use of  descriptive formalism from Discourse Analysis, called &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Adjacency_pairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;adjacency pairs&lt;/a&gt;&lt;/em&gt;. These describe regularities between utterances on a local level, for example greeting/reply, which would match your &quot;Hi, I'm Alice&quot; and &quot;Nice to meet you&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to be able to classify utterances by your chat bot according to a set of possible responses, and then you can see if a valid response is produced for any given utterance. If the user asks a &lt;em&gt;question&lt;/em&gt;, then a &lt;em&gt;greeting&lt;/em&gt; will not be a good answer, but a &lt;em&gt;statement&lt;/em&gt; could be, if it was a response to the question. This is leaving aside the content and focuses merely on the formal characteristics of the utterance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more about the topic, have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Conversation_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conversation Analysis&lt;/a&gt;, which is the linguistic field dealing with the subject.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-03-01T22:07:48.297" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10956" PostTypeId="2" ParentId="10950" CreationDate="2019-03-01T23:28:35.510" Score="2" Body="&lt;p&gt;I don't think that the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalization_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalization&lt;/a&gt; is (directly) related to the &quot;shape&quot; of the function close to the point where it attains a minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept of generalisation refers to when a trained model is able &quot;&lt;a href=&quot;https://stats.stackexchange.com/q/385312/82135&quot;&gt;perform&lt;/a&gt; well&quot; on unseen data (that is, data not seen during the training phase). If a trained model does not generalise well, then it might have &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;overfitted&quot; or &quot;underfitted&quot;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overfitting means that the model performs well on the training data, but not on other data. In other words, in the case of overfitting, the model learns the structure of the training data, but not of the whole population (of interest) or it learns the &quot;noise&quot; (that is, the data which is not part of the population of interest) present in the training data. This also implies that the training data is not a &quot;good&quot; sample of the population, that is, it doesn't &quot;summarise&quot; well all characteristics of the population. In practice, this is often the case (for large populations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Underfitting occurs when the model is not even able to learn enough about the training data during the training phase. For example, underfitting occurs when you try to train a linear model but the data does not have a linear relationship.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, being able to generalise (or not) depends on the model (including the number of parameters), but also on the data you are given.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the right one generalize better compared to the left one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The shape of the function will only be useful during the training phase. More specifically, in this case, you might reach one of the two minima faster than the other (depending also on the optimisation algorithms, model, etc., that you use). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to note that, in machine learning, we often minimise a function of functions (which in mathematics is called a &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Functional_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;functional&lt;/a&gt;&quot;). Why is that? For &quot;simplicity&quot;, consider a simple neural network (NN) model (e.g. a multi-layer perceptron). We usually train such NN using gradient descent (or one of its variants) by minimising a function (e.g. the mean squared error). Essentially, when we train such a NN, we want to find the function (which is represented by the parameters of the NN) that minimises e.g. the MSE. In this case, the mean squared error (MSE) is the functional we attempt to minimise. Note that the MSE is a function of the parameters of the current NN and that a NN is a model that represents a function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's go back to your question. If, during the training phase, you get one minimum rather than the other, you will get different NNs (say &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;). Note, again, that the point where our MSE function attains one its minima, in this context of training a NN (and often, in general, in machine learning) is a function (and not just a scalar). I know that, at the beginning (and if you're not a math guy), it might be difficult to think of functions as points (actually, more precisely, we should call them &quot;vectors&quot;) that minimise other functions, but this concept exists and actually underlies a great amount of machine learning techniques (e.g. training NNs using back-propagation by minimising a cost function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;So, why do I think that we can't say much about the generalisation ability of NNs &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that we have access, after the training phase, to both NNs &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (that is, the ones that correspond to the &quot;points&quot; where your function attains the two (visible) minima. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; will perform equally well on the training data (or even validation data). We might then conclude that both generalise in the same way. But this might be a wrong conclusion because both the training and the validation data (as I mentioned above) might not be good samples of the population. So, we can't really say which NN (or &quot;minima&quot;), &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, generalises better. They might generalise more or less in the same way (according to their performance e.g. on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets&quot; rel=&quot;nofollow noreferrer&quot;&gt;validation dataset&lt;/a&gt;) or not (because the validation dataset is not a good sample of the population).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the concept of generalisation is a little bit more complex than function minimisation, because it is also related to data. In machine learning, we often minimise a &lt;strong&gt;functional&lt;/strong&gt; (a function whose parameters are functions). If your functional attains the same minimum at two different points (actually, functions), during training, one might be faster to reach than the other. However, if you have two NNs that represent these two minima, we can't say much about their generalisation ability. Essentially, we can just hope that the validation dataset is a good sample of the population.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-02T10:37:56.403" LastActivityDate="2019-03-02T10:37:56.403" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10957" PostTypeId="2" ParentId="10940" CreationDate="2019-03-01T23:29:57.870" Score="1" Body="&lt;p&gt;I found the error: division by zero when calculating the ratio, here.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ratio = pi.prob(self.tfa) / oldpi.prob(self.tfa)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I changed to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ratio = tf.divide(pi.prob(self.tfa), tf.maximum(oldpi.prob(self.tfa), 1e-5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="20352" LastEditDate="2019-03-04T10:07:18.733" LastActivityDate="2019-03-04T10:07:18.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10958" PostTypeId="1" AcceptedAnswerId="10964" CreationDate="2019-03-02T06:02:53.117" Score="2" ViewCount="952" Body="&lt;p&gt;I just read about the concept of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Parse_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;parse tree&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In my understanding, a valid parse tree of a sentence needs to be validated by a linguistic expert. So, I concluded, a sentence only has one parse tree.&lt;/p&gt;&#xA;&lt;p&gt;But, is that correct? Is it possible a sentence has more than one valid parse tree (e.g. constituency-based)?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2021-01-18T15:48:21.090" LastActivityDate="2021-01-18T15:48:30.780" Title="Can a sentence have different parse trees?" Tags="&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10959" PostTypeId="2" ParentId="10909" CreationDate="2019-03-02T06:51:31.513" Score="1" Body="&lt;p&gt;Never tried it, but there couple of approaches which may or may not help: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Distributional DQN ( not C51, other one, cant find ref right now typing from phone) with several output heads, chosen randomly&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiple agents learning randomly on each other with some regularizer to pervent collapse to same net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both approach essentially try to&quot;hide&quot; or smear target.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-03-04T15:23:17.423" LastActivityDate="2019-03-04T15:23:17.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10960" PostTypeId="2" ParentId="10958" CreationDate="2019-03-02T06:56:22.500" Score="1" Body="&lt;p&gt;Grammars in NLP basically correspond to &lt;a href=&quot;https://en.wikipedia.org/wiki/Context-free_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;Context-free Grammars(CFG)&lt;/a&gt; in formal Language theory. And, in case the CFG corresponding to the NLP task is &lt;a href=&quot;https://en.wikipedia.org/wiki/Ambiguous_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;ambiguous&lt;/a&gt;, then corresponding to a single sentence (more formally &lt;em&gt;derivation&lt;/em&gt;), there can be multiple Parse Trees.&lt;br&gt;&#xA;Hence, it depends on the grammar whether there can be more than one valid parse tree.  &lt;/p&gt;&#xA;" OwnerUserId="14404" LastActivityDate="2019-03-02T06:56:22.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10961" PostTypeId="2" ParentId="10954" CreationDate="2019-03-02T07:36:35.337" Score="1" Body="&lt;p&gt;Just sharing my thoughts on this: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A local minimum in the loss function is just a synthetic measure expressing how well the model is performing on the target dataset (I'm assuming it's the test set)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Letting aside the absolute values, you could also consider the steepness of that region as follows: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the first LM is at the bottom of steep region which means the related parametrization is quite &lt;em&gt;unstable&lt;/em&gt; as if you change it slightly the performance drop quickly (loss increases quickly) so you could interpret this &#xA;as a parametrization with not great generalization capability (it is not robust against small changes): always remember that what you are actually looking for is model able to generalize well when &lt;em&gt;in production&lt;/em&gt; (i.e. after you have deployed it so when training, validation, test is finished) and test set is typically a small proxy for the data observed in production (but it really depends on the final application itself)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the second LM is related to a much less steep region: even if you change the parametrization slightly you do not observe a big performance change which could be interpreted as a more &lt;em&gt;stable&lt;/em&gt; parametrization hence with better possibility to generalize on unseen data &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="1963" LastEditDate="2019-03-02T09:06:44.530" LastActivityDate="2019-03-02T09:06:44.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10962" PostTypeId="2" ParentId="10950" CreationDate="2019-03-02T08:25:17.603" Score="4" Body="&lt;p&gt;In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;looking at the LF steepness we observe the Left LM is in a steeper region than the Right LM: how can this be interpreted? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'd interpret the LF Steepness as a Measure of Parametrization Stability: in fact perturbating the Parametrization slightly has a bigger effect on Left LM observed performance rather than the Right LM one &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;when the NN is running &quot;in production&quot; typically the parametrization is fixed (in most typical application, weights are not changed out of the training phase) so you should not be concerned about parametrization stability, &#xA;however I'm one of those believing the &lt;a href=&quot;http://www.bioinf.jku.at/publications/older/3304.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flat Minima - Schmidhuber 1997&lt;/a&gt; idea that Local Minima Flatness is connected to generalization property &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However it is important to observe this is still a very open and interesting question as in &lt;a href=&quot;https://arxiv.org/abs/1703.04933&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sharp Minima Can Generalize For Deep Nets&lt;/a&gt; Dinh 2017 et al. demonstrated &#xA;it is not just about flatness, because reparametrization of the NN model, though preserving minima locations, change its shape so basically sharp minima might be transformed into flat ones without affecting network performance &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-03-02T08:25:17.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10964" PostTypeId="2" ParentId="10958" CreationDate="2019-03-02T09:29:25.477" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;But, is that correct? Is it possible a sentence has more than one valid parse tree (e.g. constituency-based)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The fact that a single sequence of words can be parsed in different ways depending on context (or &amp;quot;grounding&amp;quot;) is a common basis of miscommunication, misunderstanding, innuendo and jokes.&lt;/p&gt;&#xA;&lt;p&gt;One classic NLP-related &amp;quot;joke&amp;quot; (around longer than modern AI and NLP) is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Time flies like an arrow.&lt;/p&gt;&#xA;&lt;p&gt;Fruit flies like a banana.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are actually &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_flies_like_an_arrow;_fruit_flies_like_a_banana&quot; rel=&quot;nofollow noreferrer&quot;&gt;several valid parse trees&lt;/a&gt; for even these simple sentences. Which ones come &amp;quot;naturally&amp;quot; will depend on context - anecdotally I only half got the joke when I was younger, because I did not know there were such things as &lt;em&gt;&lt;a href=&quot;https://entomology.ca.uky.edu/ef621&quot; rel=&quot;nofollow noreferrer&quot;&gt;fruit flies&lt;/a&gt;&lt;/em&gt;, so I was partly confused by literal (but still validly parsed, and &lt;em&gt;somewhat&lt;/em&gt; funny) meaning that all fruit can fly about as well as a banana does.&lt;/p&gt;&#xA;&lt;p&gt;Analysing these kinds of ambiguous sentences leads to the &lt;a href=&quot;http://www.scholarpedia.org/article/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;grounding problem&lt;/a&gt; - the fact that without some referent for symbols, a grammar is devoid of meaning, even if you know the rules and can construct valid sequences. For instance, the above joke works partly because the nature of time, when referred in a particular way (singular noun, not as a possession or property of another object), leads to a well-known metaphorical reading of the first sentence.&lt;/p&gt;&#xA;&lt;p&gt;A statistical ML parser could get both sentences correct through training on many relevant examples (or trivially by including the examples themselves with correct parse trees). This has not solved the grounding problem, but may be of practical use for any machine required to handle natural language input and map it to some task.&lt;/p&gt;&#xA;&lt;p&gt;I did check a while ago though, and most Parts Of Speech taggers in Pythons NLTK get both sentences wrong - I suspect because resolving sentences like those above and AI &amp;quot;getting language jokes&amp;quot; is not a high priority compared to more practical uses for chatbots/summarisers, etc.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-01-18T15:48:30.780" LastActivityDate="2021-01-18T15:48:30.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10965" PostTypeId="1" AcceptedAnswerId="10978" CreationDate="2019-03-02T10:17:24.927" Score="3" ViewCount="496" Body="&lt;p&gt;Why would someone use a neuroevolution algorithm, such as NEAT, over other machine learning algorithms? What situation would only apply to an algorithm such as NEAT, but no other machine learning algorithm?&lt;/p&gt;&#xA;" OwnerUserId="22802" LastEditorUserId="2444" LastEditDate="2019-07-07T19:22:57.853" LastActivityDate="2019-07-07T19:22:57.853" Title="Why would someone use NEAT over other machine learning algorithms?" Tags="&lt;machine-learning&gt;&lt;neat&gt;&lt;comparison&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10967" PostTypeId="2" ParentId="10953" CreationDate="2019-03-02T11:19:30.723" Score="1" Body="&lt;p&gt;Apart from the &lt;a href=&quot;https://scikit-optimize.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scikit-Optimize&lt;/a&gt; package related to &lt;a href=&quot;https://scikit-learn.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scikit-Learn&lt;/a&gt;, following are some of the packages related to Bayesian optimization:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sheffieldml.github.io/GPyOpt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPyOpt&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pygpgo.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyGPGO&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/hyperopt/hyperopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyperopt&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/fmfn/BayesianOptimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;bayesian-optimization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/befelix/SafeOpt&quot; rel=&quot;nofollow noreferrer&quot;&gt;safeopt&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/automl/RoBO&quot; rel=&quot;nofollow noreferrer&quot;&gt;RoBO&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14404" LastActivityDate="2019-03-02T11:19:30.723" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10968" PostTypeId="2" ParentId="10951" CreationDate="2019-03-02T13:20:32.413" Score="1" Body="&lt;p&gt;Beyond the basic requirement of storage, it seems you are looking for some explicitly defined data organization providing a query interface which allows specifying, alongside the fundamental query params, &#xA;a context and is able to use both these information items (instead of the first one only) to boost the query.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this interpretation of mine is correct, the context-aware query database comes to my mind, here are some examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.microsoft.com/en-us/research/publication/context-aware-query-classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Context-Aware Query Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper, we propose a logical model and an abstract query language as a foundation of context-aware database management systems. The model is a natural extension of the relational model in which contexts are first class citizens and can be described at different levels of granularity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-04957-6_7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Querying Context-Aware Databases&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="2444" LastEditDate="2019-10-22T21:13:05.710" LastActivityDate="2019-10-22T21:13:05.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10970" PostTypeId="2" ParentId="5325" CreationDate="2019-03-02T14:31:39.037" Score="1" Body="&lt;p&gt;The prediction made by linear regression can simply be thought of as a vector dot product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\overrightarrow{x}^T \cdot \overrightarrow{y}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of those two vectors is the &quot;data&quot; for one case (like a row in your data matrix), the other is a vector of the model's parameters, which is usually called &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{\theta}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{\beta}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the case shown by yourself, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h(x) = \theta_0 + \theta_1 \cdot x$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often we add a row of ones to the beginning of the data matrix, that way we are consistent in the sense that the &lt;span class=&quot;math-container&quot;&gt;$\theta_0 = 1 \cdot \theta_0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way we arrive at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h(x) = \overrightarrow{\theta}^T \cdot  \overrightarrow{x}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="22553" LastActivityDate="2019-03-02T14:31:39.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10971" PostTypeId="1" CreationDate="2019-03-02T14:43:03.937" Score="1" ViewCount="52" Body="&lt;p&gt;I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict &lt;span class=&quot;math-container&quot;&gt;$0.5*sin(t + N)$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$0.5*sin(t)$&lt;/span&gt; as an exercise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tweaked the model, changing the output layer activation function, weight initialization, number of memory blocks/cells, etc. However, I still couldn't manage to correct the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the output range is much smaller than desired, &lt;span class=&quot;math-container&quot;&gt;$[-0.2, 0.2]$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$[-0.5, 0.5]$&lt;/span&gt;. The output also is slightly delayed, meaning it is predicting &lt;span class=&quot;math-container&quot;&gt;$sin(t + N - 1)$&lt;/span&gt; for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something that I'm missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, for output layer activation function as a centered logistic from &lt;span class=&quot;math-container&quot;&gt;$(-1, 1)$&lt;/span&gt;, the validation output looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ay7By.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ay7By.png&quot; alt=&quot;Validation output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training output looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ds6oX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ds6oX.png&quot; alt=&quot;Training output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional information:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Topology: 1 input layer, 1 hidden layer each with 5 memory blocks each with 1 cell, 1 output layer each with 1 regular neuron.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Alpha: 1&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Weights: generated with normal distribution, from &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Output layer activation function used: logistic &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, centered logistic, tanh, ReLU, leaky ReLU, &lt;span class=&quot;math-container&quot;&gt;$f(x) = x$&lt;/span&gt; (identity)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16609" LastEditorUserId="16565" LastEditDate="2019-03-03T23:20:24.267" LastActivityDate="2019-03-03T23:20:24.267" Title="Predicting sine using LSTM: Small output range and delayed output?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10972" PostTypeId="2" ParentId="3720" CreationDate="2019-03-02T20:41:32.513" Score="0" Body="&lt;p&gt;I would say that, while it &lt;em&gt;does&lt;/em&gt; count, it only counts in certain circumstances, as trying to use trial and error with already predetermined data as its objective to output isn't AI, as it already has data that is fine as-is. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say your AI is trying to point at a precise location but is only given the current accuracy of its positioning. It could check every location and find the best of all of them, or, it could do this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Correct the up/down(vertical) positioning:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start moving up.&lt;/li&gt;&#xA;&lt;li&gt;If the accuracy gets better:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Keep moving up.&lt;/li&gt;&#xA;&lt;li&gt;Else move down.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Continue until the accuracy starts to get weaker.&lt;/li&gt;&#xA;&lt;li&gt;Move back just a tiny bit to increase again.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Correct the left/right(horizontal) positioning:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start moving right.&lt;/li&gt;&#xA;&lt;li&gt;If the accuracy gets better:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Keep moving right.&lt;/li&gt;&#xA;&lt;li&gt;Else move left.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Continue until the accuracy starts to get weaker&lt;/li&gt;&#xA;&lt;li&gt;Move back just a tiny bit to increase again.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;You're done!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is AI, as it learns what to and what not to do, and figures out a location based off of the accuracy of its positioning alone. So while the &quot;trial and error&quot; method &lt;em&gt;is&lt;/em&gt; AI, it only counts when it has &lt;strong&gt;no predetermined calculation&lt;/strong&gt; to figure out the result &lt;strong&gt;without said trial and error&lt;/strong&gt;. Trying to find numbers of Pi, for example, while it &lt;em&gt;is&lt;/em&gt; technically trial and error, it uses math functions and requires multiple inputs to calculate its output, and, in the end, it only partly uses trial and error, therefore, it is not AI in the end.&lt;/p&gt;&#xA;" OwnerUserId="22816" LastActivityDate="2019-03-02T20:41:32.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10973" PostTypeId="1" CreationDate="2019-03-02T21:08:57.293" Score="3" ViewCount="117" Body="&lt;p&gt;To make A2C into A3C you make it asynchronous. From what I understand the 'correct' way to do that is to thread off workers with a copy of the policy and critic, and then return the state/action/reward tuples to the main thread, which then performs the gradients updates on the main policy and critic, and then repeat the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand why the copying would be necessary in a distributed environment, but if I were to always run it locally then could I just perform the updates on a global variable of the policy and critic, i.e. avoid the need for copying? Provided the concurrency of the updates was handled correctly, would that be fine?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="20352" LastEditDate="2019-03-02T23:18:11.650" LastActivityDate="2021-04-12T17:02:40.217" Title="Can A3C update the policy / critic on a local machine without needing to copy?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10974" PostTypeId="2" ParentId="10210" CreationDate="2019-03-03T06:24:59.297" Score="0" Body="&lt;p&gt;Loss not decreasing during the training does not signify that network don't train. It may mean network is in the exploration mode. I have observed that situation - accumulated reward growing steadily, which mean network is training well, but loss is not decreasing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you already know solution to some (simpler or other) version of your environment, you can train network in supervised manner to reproduce that solution. If network unable reproduce existing solution that is strong indication that network is too small or otherwise not good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other reason for accumulated reward oscillation could be network overfitting on latest training samples. In that case bigger replay buffer or more slow update of target network(if target network is used) may help.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-03-03T06:24:59.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10975" PostTypeId="1" AcceptedAnswerId="15406" CreationDate="2019-03-03T08:39:20.207" Score="2" ViewCount="258" Body="&lt;p&gt;So I was wondering, why I have only encountered square loss function also known as MSE. The only nice property of MSE I am so far aware of is its convex nature. But then all equations of the form &lt;span class=&quot;math-container&quot;&gt;$x^{2n}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is an integer belongs to the same family.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what makes MSE the most suitable candidate among this entire family of curves? Why do other curves in the family, even though having steeper slopes, &lt;span class=&quot;math-container&quot;&gt;$(x &amp;gt;1) $&lt;/span&gt;, which might result in better optimisation, not used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture to what I mean where red is &lt;span class=&quot;math-container&quot;&gt;$x^4$&lt;/span&gt; and green is &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/muwB1.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/muwB1.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-03-03T09:39:33.463" LastActivityDate="2020-06-08T19:01:40.217" Title="Why is MSE used over other quadratic loss functions?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;loss-functions&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10976" PostTypeId="2" ParentId="10975" CreationDate="2019-03-03T09:24:10.107" Score="-1" Body="&lt;p&gt;There is another variant for MSE. You can employ the absolute value of the difference of your hypothesis and the expected output. &lt;code&gt;MSE&lt;/code&gt; and the absolute difference version, each have a property. The interpretation of the &lt;code&gt;MSE&lt;/code&gt; is that you have the area of squares which at first are large but after training the predicted outputs and real outputs get similar to each other which means the area of squares has been diminished after training. For absolute value, the interpretation is that the difference you are going to reduce is a length and after training, you diminish the length of the errors. I don't say what you've said is not possible. It is possible and it does have an interpretation, maybe to diminish the volume of hypercubes, but the point is that the differentiation of such cost functions are a bit complex and it may lead to update rules which are longer to be calculated. &lt;/p&gt;&#xA;" OwnerUserId="11599" LastEditorUserId="11599" LastEditDate="2019-03-03T09:35:53.460" LastActivityDate="2019-03-03T09:35:53.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10977" PostTypeId="1" CreationDate="2019-03-03T10:31:23.247" Score="2" ViewCount="57" Body="&lt;p&gt;We have been assigned a project, in which we have to create a chatbot which will ask question, take the replies, analyse them and give an approximate assessment of the current emotional state of the person. There are two aspects of the project, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Training the bot to choose the next question based on the previous response &lt;/li&gt;&#xA;&lt;li&gt;And analysing the responses individually, to detect . the sentiment.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What technology would we have to use and what would be the steps to accomplishing the tasks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="22822" LastActivityDate="2021-03-22T15:04:55.543" Title="What would be the steps to create an sentiment analysis chatbot?" Tags="&lt;machine-learning&gt;&lt;sentiment-analysis&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10978" PostTypeId="2" ParentId="10965" CreationDate="2019-03-03T11:25:55.043" Score="1" Body="&lt;p&gt;The main difference leading to strengths and weaknesses of NEAT algorithm, is that it does not use any gradient calculations. That means for NEAT, neither the cost function, nor the activation functions of the neurons are required to be differentiable. In some circumstances - e.g. where agents directly compete, and you can select them for next generation if they win versus one or more opponents - you may not even need a cost function to optimise. Quite often the cost function can be a simple number generated from a complex evaluation of the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, NEAT can be used in situations where the formula for a cost function, in terms of single feed-forward runs of the network, is not very clear. It can also be used to explore activation functions such as step functions or stochastic firing neurons, where gradient based methods are difficult to impossible to apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEAT can perform well for simple control scenarios, as a policy network that outputs actions given some sensor inputs. For example, it is popular to use it to create agents for racing games or simulated robotics controllers. When used as a policy-based controller, NEAT is in competition with Reinforcement Learning (RL), and has basic similarities with policy gradient methods - the nature of the controller is similar and often you could use the same reward/fitness function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As NEAT is already an evolution-inspired algorithm, it also fits well with &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_life&quot; rel=&quot;nofollow noreferrer&quot;&gt;a-life&lt;/a&gt; code as the &quot;brains&quot; of simulated creatures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main disadvantage of NEAT is slow convergence to optimal results, especially in complex or challenging environments. Gradient methods can be much faster, and recent advances in Deep RL Policy Gradients (algorithms like A3C and DDPG) means that RL can tackle much more complex environments than NEAT. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest to use NEAT when:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The problem is easy to assess - either via a measurement of performance or via competition between agents - but might be hard to specify as a loss function&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A relatively small neural network should be able to approximate the target function&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a goal to assess a non-differentiable activation function&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you are looking at a sequential control problem, and could use a standard feed-forward neural network to approximate a policy, it is difficult to say in advance whether NEAT or some form of Deep RL would be better. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-03T11:34:42.223" LastActivityDate="2019-03-03T11:34:42.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10979" PostTypeId="2" ParentId="10977" CreationDate="2019-03-03T11:56:26.450" Score="0" Body="&lt;p&gt;There are some different approaches to learning this sequential acting. First, you can use RL (reinforcement learning) and define some rewards over the action of the user over that question, to explore and exploit the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, you can use RNN to predict the preferred next question from a history of data. In contrast with RL, you need many instances to train the network to predict better next answer more accurately. &lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-03-03T11:56:26.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10981" PostTypeId="1" AcceptedAnswerId="10996" CreationDate="2019-03-03T13:05:42.127" Score="5" ViewCount="173" Body="&lt;p&gt;Is there some type of neural network that changes the number of neurons while training? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this idea, the network can increase or decrease the number of neurons when the complexity of the inputs increases or decreases.&lt;/p&gt;&#xA;" OwnerUserId="19783" LastEditorUserId="2444" LastEditDate="2019-07-07T22:35:15.653" LastActivityDate="2019-07-07T22:35:15.653" Title="Is there a neural network with a varying number of neurons?" Tags="&lt;neural-networks&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10982" PostTypeId="1" CreationDate="2019-03-03T13:07:18.340" Score="7" ViewCount="5029" Body="&lt;p&gt;In section 3.6 of the &lt;a href=&quot;https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf&quot; rel=&quot;noreferrer&quot;&gt;OpenAI GPT-2 paper&lt;/a&gt; it mentions summarising text based relates to this, but the method is described in very high-level terms:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To induce summarization behavior we add the text &lt;code&gt;TL;DR:&lt;/code&gt; after the article and generate 100 tokens with Top-k random sampling (Fan et al., 2018) with k=2 which reduces repetition and encourages more abstractive summaries than greedy decoding. We use the first 3 generated sentences in these 100 tokens as the summary.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Given a corpus of text, in concrete code terms (python preferred), how would I go about generating a summary of it?&lt;/p&gt;&#xA;" OwnerUserId="2424" LastActivityDate="2019-06-10T08:46:42.107" Title="How do I use GPT-2 to summarise text?" Tags="&lt;open-ai&gt;&lt;text-summarization&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10983" PostTypeId="1" AcceptedAnswerId="10985" CreationDate="2019-03-03T13:36:06.670" Score="2" ViewCount="182" Body="&lt;p&gt;Hi guys I've been studying such combination where the idea is to replace the classic descendant gradient for an algorithm that is less sensitive to local optimum, so the PSO tries to select the best weight and bias for the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I've seen everywhere that only one hidden layer is used (no one explains why just one layer is being used) and all those codes break when I try to use more than one hidden layer, so the questions are:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Can't PSO be used to optimize an Artificial Neural Network with more than one hidden layer?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In that case, why is that?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15764" LastEditorUserId="16565" LastEditDate="2019-03-03T16:00:30.940" LastActivityDate="2019-03-03T17:18:07.213" Title="Training an artificial neural network using PSO" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;optimization&gt;&lt;swarm-intelligence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10985" PostTypeId="2" ParentId="10983" CreationDate="2019-03-03T17:18:07.213" Score="1" Body="&lt;p&gt;Particle Swarm Optimization can be used to optimize a neural network with more than one hidden layer. Instead of optimizing a single weight matrix, and two bias vectors, you are just optimizing more of them. However, PSO is not often used for larger neural networks, because, particle swarm optimization is not all that efficient at working with a large amount of data, which you need to train larger neural networks. Particle Swarm Optimization involves taking many particles, and evaluating the fitness in all of them. If you have let’s say 1,000,000 training examples, you are doing a lot more error calculations, even if you used a small batch size, than if you used another technique lake Backproppogation, or a related technique. Backproppogation and related techniques are used much more in larger neural networks because they are more efficient at working with larger sets of data. As for why the code that you have been using breaks with networks with more than one hidden layer, I cannot explain, but it is fairly easy to write your own basic implementation of PSO to train multi layer neural networks.&lt;/p&gt;&#xA;" OwnerUserId="4631" LastActivityDate="2019-03-03T17:18:07.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10987" PostTypeId="1" CreationDate="2019-03-03T18:32:27.577" Score="1" ViewCount="47" Body="&lt;p&gt;I'm wanting to conduct game theoretic analyses of ongoing conflict situations (e.g. the US/North Korea negotiations; Syrian conflict; etc)  as reported in the news media. I believe that AI may help me do this by helping me to pick out from the text: the parties involved; the issues over which they are in conflict; the choices they have; their preferences.  However I'm not sure whether to approach this using 'modern' 'deep learning' approaches or to try something along the lines of the classic work by Schank, deJong etc. who used the notion of scripts (and sketchy scripts) in their work with conceptual dependency approaches.  Does anyone have comments, suggestions that may guide my work please?&lt;/p&gt;&#xA;" OwnerUserId="22832" LastActivityDate="2019-03-03T18:32:27.577" Title="How can/should I use AI to populate a game (in the game theory sense) from text input" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;game-theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10988" PostTypeId="1" CreationDate="2019-03-03T18:46:48.937" Score="2" ViewCount="190" Body="&lt;p&gt;I've learned that you can use two perceptrons to ultimately create a classifier for non-linearly separable data. I'm trying to understand how / if  these two perceptrons converge to two different decision boundaries though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z9iMR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z9iMR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how the second perceptron creates a different decision boundary when it has the same input as the first perceptron? I know the weights can be initialized differently but does this second perceptron classify something else? Shouldn't the decision boundaries be converge to be the the same ultimately after training?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gqiw6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gqiw6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22833" LastActivityDate="2020-04-08T14:48:34.510" Title="How do two perceptrons produce different linear decision boundaries when learning?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10989" PostTypeId="1" CreationDate="2019-03-03T19:41:22.113" Score="2" ViewCount="165" Body="&lt;p&gt;After going through both the &quot;&lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illustrated Transformer&lt;/a&gt;&quot; and &quot;&lt;a href=&quot;http://nlp.seas.harvard.edu/2018/04/03/attention.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Annotated Transformer&lt;/a&gt;&quot; blog posts, I still don't understand how the sinusoidal encodings are representing the position of elements in the input sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the fact that since each row (input token) in a matrix (entire input sequence) has a unique waveform as its encoding, each of which can be expressed as a linear function of any other element in the input sequence, then the transformer can learn relations between these rows via linear functions? &lt;/p&gt;&#xA;" OwnerUserId="8062" LastEditorUserId="2444" LastEditDate="2019-11-01T03:12:24.280" LastActivityDate="2019-11-01T03:12:24.280" Title="How do the sine and cosine functions encode position in the transformer?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;attention&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10992" PostTypeId="2" ParentId="10988" CreationDate="2019-03-03T21:43:50.520" Score="2" Body="&lt;p&gt;I think your confusion comes from the fact that you are calling those two hidden nodes &quot;perceptrons&quot;. You shouldn't call the hidden nodes in your network perceptrons. You should call them &quot;nodes&quot; or &quot;neurons&quot;, although the term &quot;multilayer perceptron&quot; comes exactly from that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should think of a perceptron as something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DWFga.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DWFga.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A perceptron simply computes a linear combination of the inputs using the corresponding weights. A single perceptron can (only) &lt;strong&gt;linearly&lt;/strong&gt; separate a set of points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you have two &quot;stacked perceptrons&quot; (like the hidden nodes in your network), then, provided that both &quot;perceptrons&quot; (or hidden nodes) &lt;em&gt;only compute a linear combination of the inputs&lt;/em&gt;, the output of each hidden node can be interpreted as line (like the violet and green lines in your picture).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I don't understand how the second perceptron creates a different decision boundary when it has the same input as the first perceptron?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is because the two perceptrons will eventually have different weights. You can see from your diagram that &lt;span class=&quot;math-container&quot;&gt;$p_0$&lt;/span&gt; has weights &lt;span class=&quot;math-container&quot;&gt;$w_{0, 0}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{0, 1}$&lt;/span&gt;, whereas perceptron &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; has weights &lt;span class=&quot;math-container&quot;&gt;$w_{1, 1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{1, 0}$&lt;/span&gt;. Note that weights are learned, during training, depending on your loss function (at the output node &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt;) and the labeled dataset (that you use to train that simple neural network).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I know the weights can be initialized differently but does this second perceptron classify something else?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you consider the two hidden nodes (in your network) separately, then (as I said above) the output of these two hidden nodes can be interpreted as a line, and these lines might be different (as illustrated in your picture), because these two hidden nodes will eventually learn something different about the input. However, you should also think of the output of two hidden nodes as contributing to the input of the output node. So, in a certain way, you're combining the decisions of the two hidden nodes (or, as you call them, &quot;perceptrons&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you will need to introduce &quot;non-linearity&quot; in those hidden perceptrons in order to be able to non-linearly separate a set of points.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Shouldn't the decision boundaries both converge to the same ultimately after training?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, not necessarily. The decision boundary found by the perceptron learning algorithm depends on the initialization of the weights and the order that the inputs are presented. See chapter 4 (specifically, pages 192-196) of &lt;a href=&quot;http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern Recognition and Machine Learning&lt;/a&gt; by C. Bishop. Two perceptrons would be guaranteed to converge to the same solution if the loss function was convex (i.e. there's only one optimum).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-08T14:48:34.510" LastActivityDate="2020-04-08T14:48:34.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10995" PostTypeId="2" ParentId="10339" CreationDate="2019-03-04T06:56:59.633" Score="1" Body="&lt;p&gt;Then work with quantized network it's good idea to normalize block of them them with  floating point scaling factor (16 bit float for example).&#xA;The weight channel and input channel will have represenation S_f * w_n where S_f - scalar float and w_n - low-bit fixed or integer tensor or vector. Then calculating dot product put it into high-bit fixed/integer variable or float and recalculate scaling factor and low-bit representation for output. That is only if you can implement floating point (or high bit fixed) operations of cause. For low-bit representation you should clamp values to S_f*[-2^n:2^n] and that could be non-trivial. Good idea to use statistics of weights/inputs for it.&#xA;There are dozen of papers on quantization of neural networks with different approaches, you may want to read some.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-03-04T07:02:15.520" LastActivityDate="2019-03-04T07:02:15.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10996" PostTypeId="2" ParentId="10981" CreationDate="2019-03-04T10:37:41.157" Score="4" Body="&lt;p&gt;Yes, NEAT (NeuroEvolution of Augmenting Topologies) increases the number of neurons during training. More specifically, NEAT uses evolution to introduce new neurons and connections during training, and - just as evolution - if the mutation performs poorly, gets eliminated after a few generations. This way overall performance increases over time while it keeps your network size (and computing power to run it) minimal. There's also a way to add convolution to this algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper that introduced NEAT&lt;/a&gt;, the authors mention a completely different algorithm they tried, optimizing network hyper-parameters by choosing reasonable random values and training it multiple times. That could decrease that number as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there is a trick to temporarily turn off specific parts of a network, which supposedly helps with overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A ReLU+backpropagation based network can &quot;turn off&quot; parts of the network during training, because constant 0's derivative is constant 0. In practice, you decrease the number of neurons (that is considered bad though, that's when leaky ReLU and PReLU is used instead).&lt;/p&gt;&#xA;" OwnerUserId="22418" LastEditorUserId="2444" LastEditDate="2019-03-04T21:00:40.673" LastActivityDate="2019-03-04T21:00:40.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10997" PostTypeId="1" AcceptedAnswerId="11005" CreationDate="2019-03-04T10:59:52.900" Score="3" ViewCount="59" Body="&lt;p&gt;How accurate are neuro-evolution algorithms (such as NEAT) in modelling real organism evolution?&lt;/p&gt;&#xA;" OwnerUserId="22802" LastEditorUserId="2444" LastEditDate="2019-07-07T19:20:58.043" LastActivityDate="2019-07-07T19:20:58.043" Title="How accurate are neuroevolution algorithms in modelling organism evolution?" Tags="&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10998" PostTypeId="2" ParentId="10930" CreationDate="2019-03-04T11:21:58.930" Score="1" Body="&lt;p&gt;This is a bit of a puzzle but you can compute a reasonable narrow limit even without knowing whether or not &lt;span class=&quot;math-container&quot;&gt;$P(S,A) = P(S) P(A)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start with the contingency table relating &lt;span class=&quot;math-container&quot;&gt;$P(S, A)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(S,\neg A)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(\neg S, A)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(\neg S,\neg A)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$P(S)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(A)$&lt;/span&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{cc|c}&#xA;P( S,A)&amp;amp; P(\neg S,A) &amp;amp; P(A) \\&#xA;P(S,\neg A)&amp;amp; P(\neg S,\neg A) &amp;amp; P(\neg A) \\ \hline&#xA;P(S)&amp;amp; P(\neg S) &amp;amp; 1 \\&#xA;\end{array} \quad \rightarrow \quad \begin{array}{cc|c}&#xA;x&amp;amp; 0.01-x &amp;amp; 0.01 \\&#xA;0.07-x&amp;amp; 0.92+x &amp;amp; 0.99 \\ \hline&#xA;0.07&amp;amp; 0.93 &amp;amp; 1 \\&#xA;\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;note that the cells must be between zero and one thus &lt;span class=&quot;math-container&quot;&gt;$0 \leq x \leq 0.01$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Then with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{array}{rcrl}P(F|S) &amp;amp;= &amp;amp; &amp;amp; P(F|S,A) P(A|S) + P(F| S,\neg A) P(\neg A|S)\\&#xA;&amp;amp;=&amp;amp;&amp;amp; 1.0 \frac{x}{0.07} +  0.7 \frac{0.07-x}{0.07} \\&#xA;&amp;amp; = &amp;amp;&amp;amp; 0.7 + 4 \frac{2}{7} x  \end{array}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$0.7 \leq P(F|S) \leq 0.743$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To solve &lt;span class=&quot;math-container&quot;&gt;$P(F|S)$&lt;/span&gt; exactly you need to narrow down &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; more precisely. Possibilities are:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you know &lt;span class=&quot;math-container&quot;&gt;$P(F)$&lt;/span&gt; then you could use &lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{rcrl}P(F) &amp;amp;= &amp;amp; &amp;amp; P(F|S,A) P(S,A) \\ &amp;amp;&amp;amp; +&amp;amp; P(F|\neg S,A) P(\neg S,A) \\ &amp;amp;&amp;amp;+&amp;amp; P(F| S,\neg A) P(S, \neg A)\\&amp;amp;&amp;amp;+&amp;amp;P(F|\neg S,\neg A) P(\neg S,\neg A)\\&#xA;&amp;amp;=&amp;amp;&amp;amp; 1.0 (x) + 0.7 (0.07 - x) + 0.9 (0.01-x)+0.1(0.92+x) \\&#xA;&amp;amp; = &amp;amp;&amp;amp; 0.15-0.5 x \end{array}$$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you know that &lt;span class=&quot;math-container&quot;&gt;$S \perp \! \! \! \perp A$&lt;/span&gt; then you can use &lt;span class=&quot;math-container&quot;&gt;$x = P(S,A) = P(S)P(A) = 0.0007$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22854" LastEditorUserId="22854" LastEditDate="2019-03-04T11:43:04.060" LastActivityDate="2019-03-04T11:43:04.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11000" PostTypeId="1" CreationDate="2019-03-04T16:07:19.913" Score="1" ViewCount="1534" Body="&lt;p&gt;The (discrete and continuous) Fourier transform (FT) is used in signal processing in order to convert a signal (or function) in a certain domain (e.g. the time domain) to another domain (e.g., frequency domain). There are several resources on the web that attempt to explain the FT at different levels of complexity. See e.g. &lt;a href=&quot;https://math.stackexchange.com/a/11114/168764&quot;&gt;this answer&lt;/a&gt; or &lt;a href=&quot;https://www.youtube.com/watch?v=mkGsMWi_j4Q&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=spUNpyF58BY&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; Youtube videos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are examples of (real-world) applications of the Fourier transform to AI? I am looking for answers that explain the reason behind the use of the FT in the given application. I suppose that there are several applications of the FT to e.g. ML (data analysis) and robotics. I am looking for specific examples.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-11-02T15:19:20.483" Title="What are examples of applications of the Fourier transform to AI?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;applications&gt;&lt;robotics&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11002" PostTypeId="2" ParentId="10826" CreationDate="2019-03-04T18:16:33.570" Score="1" Body="&lt;p&gt;You have a set of different types of data available for each of your subjects, and given one set you'd like to &lt;em&gt;classify&lt;/em&gt; which subject it belongs to. This looks like a supervised classificiation problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most popular classifiers for supervised learning are neural networks. Now given the heterogenous nature of your data types, a simple approach would be to use separate classifiers for each type of data. For example, a convolution neural net for the image data, and a simple feed forward net for the biosensor data. Another thing you try is a multi channel approach, where towards the input side you have multiple channels for the different types of data, and the final few layers are fully connected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/lHSJO.jpg&quot; alt=&quot;A multi channel CNN&quot;&gt;&#xA;The image has only CNNs for the multi channel part but you could have one channel as a simple feed forward net while another one as having conv layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you wish to classify the data as belonging to a subject on the basis of just one of the data types from the set, then you should have separate classifiers for all types. In that case it might be worthwhile to look into classifiers other than neural nets like multi class logistic regression which might be simpler to work with for a particular data type. &lt;/p&gt;&#xA;" OwnerUserId="22855" LastActivityDate="2019-03-04T18:16:33.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11003" PostTypeId="2" ParentId="11000" CreationDate="2019-03-04T19:04:35.937" Score="1" Body="&lt;p&gt;The filtering (or convolution) process in Convolutional Neural Networks can be implemented using Fourier Transforms. Convolutions in image (or spatial) space are equivalent to multiplications in frequency space, and multiplications can be performed much more efficiently than convolutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm for using Fourier Transforms to calculate a convolution between an image and a filter kernel is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming a 100x100 input image, and a 3x3 filter kernel (such as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sobel_operator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sobel Edge-Detection filter&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculate the 2D Fourier Transform (FT) of the image (&lt;a href=&quot;http://www.fftw.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;C library&lt;/a&gt;, &lt;a href=&quot;https://docs.scipy.org/doc/numpy-1.13.0/reference/routines.fft.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python library&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pad the filter kernel so that it is the same size as the image, and that the filter kernel is centred at image coordinate (0, 0). This means that some parts of the kernel will be placed in the other corners of the padded filter image. For example, consider a 3x3 kernel where the filter coefficients are [[a, b, c], [d, e, f], [g, h, i]], in this case the coefficients map to pixel positions on the padded kernel as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a -&gt; (99, 99)    //bottom right&lt;/li&gt;&#xA;&lt;li&gt;b -&gt; (0, 99)     //bottom left&lt;/li&gt;&#xA;&lt;li&gt;c -&gt; (1, 99)&lt;/li&gt;&#xA;&lt;li&gt;d -&gt; (99, 0) //top right&lt;/li&gt;&#xA;&lt;li&gt;e -&gt; (0, 0) //top left&lt;/li&gt;&#xA;&lt;li&gt;f -&gt; (1, 0)   &lt;/li&gt;&#xA;&lt;li&gt;g -&gt; (99, 1)&lt;/li&gt;&#xA;&lt;li&gt;h -&gt; (0, 1)&lt;/li&gt;&#xA;&lt;li&gt;i -&gt; (1, 1)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Set all the other padded filter kernel pixels to zero&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the 2D Fourier Transform of the padded filter kernel.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perform an element-wise multiplication of the image FT and the padded filter kernel FT. Note that this is a complex multiplication as the FT consists of complex numbers. The result of this multiplication is therefore also an array of complex numbers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the Inverse Fourier Transform of the result from step 4. This will return a real valued image which is the original image convolved with the filter kernel.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The advantage for performing the kernel convolutions in this way is that it is much faster than performing an element-wise convolution directly. This speeds up the training process for neural networks with Convolutional layers.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="2444" LastEditDate="2019-03-06T09:18:29.177" LastActivityDate="2019-03-06T09:18:29.177" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11004" PostTypeId="1" CreationDate="2019-03-04T21:13:16.217" Score="2" ViewCount="73" Body="&lt;p&gt;I am doing some experimentation on neural networks, and for that I am trying to program a plain &lt;a href=&quot;https://en.wikipedia.org/wiki/Optical_character_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;OCR&lt;/a&gt; task. I have learned CNNs are the best choice ,but for the time being and due to my inexperience, I wanna go step by step and start with feedforward nets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my training data is a set of roughly 400 16*16 images extracted from a script that draws every alphabet char in a tiny image for a small set of fonts registered in my computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the test data set is extracted from the same procedure, but for all fonts in my computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, results are quite bad. Get an accuracy of aprox. 45-50%, which is very poor... but that's not my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that I can't get the MSE below 0.0049, no matter what hidden layer distribution I apply to the net. I have tried with several architectures and all comes down to this figure. Does that mean the net cannot learn any further given the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This MSE value however throws this poor results too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Tensorflow API directly, no keras or estimators and for a list of 62 recognizable characters these are examples of the architectures I have used: [256,1860,62] [256,130,62] [256,256, 128 ,62] [256,3600,62] ....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But never get the MSE below 0.0049, and still results are not over 50%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hints are greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="22869" LastEditorUserId="2444" LastEditDate="2019-03-05T10:04:03.283" LastActivityDate="2021-05-23T20:08:15.847" Title="Attempting to solve a optical character recognition task using a feed-forward network" Tags="&lt;feedforward-neural-network&gt;&lt;optical-character-recognition&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11005" PostTypeId="2" ParentId="10997" CreationDate="2019-03-04T23:02:12.290" Score="1" Body="&lt;p&gt;Modelling genetic code persistence, mutation, and meiosis for neurological features or any other biological features is not the greatest of challenges. Modelling the rest of the organism that can support neural growth and change, the electrochemistry of the many types of neurons, and the environment that favors the emergence of learned neural behaviors is challenging and still far beyond the current level of technology.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-03-04T23:02:12.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11006" PostTypeId="2" ParentId="11000" CreationDate="2019-03-04T23:20:26.887" Score="1" Body="&lt;p&gt;To be precise, a discrete Fourier transform can be used to transform a finite set of samples between frequency and time domains. A continuous Fourier transform can be applied in calculus to an expression or a set of equations (through the appropriate techniques) or used to develop algorithms, but digital systems are not continuous, so there is no way to directly integrate in a digital computer except symbolically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the features contained in time series or pixels are more easily extracted from spectral information or frequency domain complex numbers than from the time series, an FFT can produce the data in a form that supports easy extraction. For streams containing indefinitely long time series, a windowing scheme may be used.  The common cases are for sound recognition and music recognition. FFTs are also used in the finite math involved in cryptography and especially cryptanalysis (code breaking). In robotics, FFTs would most likely be used in vibration analysis to reduce wear and dampen oscillation.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-03-04T23:20:26.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11008" PostTypeId="1" CreationDate="2019-03-05T01:17:00.723" Score="7" ViewCount="199" Body="&lt;p&gt;It is possible that the view of what is impressive enough in computer behavior to be called intelligence changes with each decade as we adjust to what capabilities are made available in products and services.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-03-08T15:58:50.987" LastActivityDate="2020-03-08T15:58:50.987" Title="Would the people of the 19th Century call our conventional software today artificial intelligence?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;history&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11009" PostTypeId="1" CreationDate="2019-03-05T01:30:25.717" Score="1" ViewCount="75" Body="&lt;p&gt;Conjecture 1: The smartest chess playing system is the one that wins the tournament.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conjecture 2: The computer system that imitates human dialog is as smart as the human whose dialog was imitated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are legacy views from the twentieth century.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are they correct?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Response to Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is not about agents. It is about indicators, or metrics if you wish. The question is whether the established game championship and Turing's thought experiment, the dialog centered imitation game, are correctly designed metrics to establish the extent of intellectual ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That someone voted to close this question on the basis of broadness is irrelevant. This question and ones like it are important questions for AI researchers and enthusiasts to ask about the set of capabilities we call intelligence, which we attempt to realize in artificial systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may also be a tendency to think of questions like this as seeking an opinion. It is certainly true that our opinions are not useful to the furtherance of AI, and opinion is not what this question invite when it asks, &quot;Is this [pair of conjectures that guides much AI research] correct?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Correctness must draw on mathematical rigor leading to a yes or a no. Some reasoning leading to the conclusion of correctness or incorrectness must be given. The answer may be yes in one context and no in another, but this is not because of the broadness of the question but because of the broadness of the many useless opinions and possibly counterproductive conjecture that remains from early thinking about what intelligence is and how to gauge its presence in computing machinery or in people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions about intelligence from this author are intended to narrow the terms and concepts so that they can be expressed mathematically. Only with a mathematically terse and elegant set of fundamentals can AI progress as a field in the sciences. Some interesting opinion will probably need to be discarded and a logically constructed consensus must replace broadness and ambiguity for this to occur.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-03-06T06:46:51.553" LastActivityDate="2019-03-06T06:46:51.553" Title="Are winning and imitating the primary indicators of intelligence?" Tags="&lt;theory&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11010" PostTypeId="2" ParentId="3032" CreationDate="2019-03-05T03:48:08.447" Score="1" Body="&lt;p&gt;I think there are at least three points that you need to think before implement Hill-Climbing (HC) algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the initial state. In HC, people usually use a &quot;temporary solution&quot; for the initial state. You can use an empty knapscak, but I prefer to randomly pick items and put it in the knapsack as the initial state. You can use a binary array to save the &quot;pickup status&quot; of each item such that if &lt;span class=&quot;math-container&quot;&gt;$a_i=1$&lt;/span&gt; then you pick the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th item and if &lt;span class=&quot;math-container&quot;&gt;$a_i=0$&lt;/span&gt; then you don't pick the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th item.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next is the heuristic function. A function that evaluate your current state. Actually, you can define it freely (like the initial state) as long as the function can evaluate which state is better than the others. You can use a simple one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;h =&#xA;    \begin{cases}&#xA;      -1, &amp;amp; \text{if}\ \sum (w_i \times a_i) &amp;gt; \text{MaxWeight} \\&#xA;      \sum (v_i \times a_i), &amp;amp; \text{otherwise}&#xA;    \end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; is the weight and the value of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th item respectively. One of the disadvantage of this function is if you have more than one invalid state (a state with the total weight is more than the limit), they have the same value, which is &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last is the method to generate the neighbors of the current state. Once again, you are also free to define the method that you will use. I think your proposed actions are good enough:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;reverse the status of an item (1 to 0 or 0 to 1). That means I add or reduce the item on the current state.&lt;/li&gt;&#xA;&lt;li&gt;swap the status two items that have different value. Formally swap &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_j$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$ i \neq j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ a_i \neq a_j$&lt;/span&gt;. That means I swap the item that I picked before with the item that I didn't pick before&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then, after you define everything you can run the algorithm as explained in &lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_climbing#Pseudocode&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;currentNode = startNode;&#xA;loop do&#xA;    L = NEIGHBORS(currentNode);&#xA;    nextEval = -INF;&#xA;    nextNode = NULL;&#xA;    for all x in L &#xA;        if (EVAL(x) &amp;gt; nextEval)&#xA;            nextNode = x;&#xA;            nextEval = EVAL(x);&#xA;    if nextEval &amp;lt;= EVAL(currentNode)&#xA;        //Return current node since no better neighbors exist&#xA;        return currentNode;&#xA;    currentNode = nextNode;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-03-05T03:48:08.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11013" PostTypeId="2" ParentId="11009" CreationDate="2019-03-05T07:27:35.700" Score="0" Body="&lt;p&gt;To some extend I would say yes they are true and there is not that much more to say, but true intelligence and thereby the thing we call smart can only be achieved in a full extend if the machine learns to think and behave not only by imitation but by true learning. &#xA;A really intelligent Software might me capable of winning every Go game (Go is more difficult than chess as probably everybody knows by now). Would you say this machine is more intelligent than the one that beats you in chess? I'd personally say no. Both are only designed to perform one task and that one alone. But what would happen if the chess computer is somehow learning (without human interference) to play Go? Would we call it intelligent or at least more intelligent than the one only playing Go? Why would we call it more intelligent. &#xA;Same is for the imitation. I would say that most machines are not capable of imitating a human interaction to the spot, if not all, but lets start with such a computer. One that can overcome the touring test by imitating a human. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pretty smart, I guess. But: What if an other machine is capable of deliberately failing the touring test? it would not have learned this by the hand of a human, since every human would design it to complete the test. Imagine a machine that can tell it is better to not be identified as intelligent / smart on its own. It might be an advantage to do so. &#xA;True intelligence can not be obtained by just imitating. Every child and every chessplayer knows that. &#xA;For example: if you play a game of chess and black moves just as white does, white will always win. &lt;/p&gt;&#xA;" OwnerUserId="21604" LastActivityDate="2019-03-05T07:27:35.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11014" PostTypeId="1" CreationDate="2019-03-05T09:22:06.333" Score="6" ViewCount="110" Body="&lt;p&gt;In the paper &lt;em&gt;Mastering the game of Go with deep neural networks and tree search&lt;/em&gt;, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Feature       #of planes Description &#xA;Stone colour  3          Player stone/opponent stone/empty &#xA;Ones          1          A constant plane ﬁlled with 1 &#xA;Turns since   8          How many turns since a move was played &#xA;Liberties     8          Number of liberties (empty adjacent points) &#xA;Capture size  8          How many opponent stones would be captured &#xA;Self-atari size 8        How many of own stones would be captured &#xA;Liberties after move 8   Number of liberties after this move is played &#xA;Ladder capture 1         Whether a move at this point is a successful ladder capture &#xA;Ladder escape 1          Whether a move at this point is a successful ladder escape &#xA;Sensibleness  1          Whether a move is legal and does not ﬁll its own eyes &#xA;Zeros         1          A constant plane ﬁlled with 0 &#xA;Player color  1          Whether current player is black&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's more, I don't understand the sharp sign here. Does it mean &quot;the number&quot;? But one number is enough to represent &quot;the number of turns since a move was played&quot;, why eight?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="22886" LastActivityDate="2021-01-05T20:30:43.180" Title="Why is a constant plane of ones added into the input features of AlphaGo?" Tags="&lt;machine-learning&gt;&lt;alphago&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11015" PostTypeId="2" ParentId="10345" CreationDate="2019-03-05T10:04:57.210" Score="2" Body="&lt;p&gt;What interests us in this problem are only the intervals for 1 person. Lets say that we want to train a neural network on recognizing the simple pattern in date differences. This would mean that we could train the neural network on series of purchase histories of multiple people. That means that one possible input is the previous intervals: in your case the previous 2 intervals (10 days, 15 days). That is a very small number and it will be hard to recognise a pattern for 3 consecutive purchases, but lets disregard that. After all, you can always experiment with different input sizes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For training, you can take multiple customers, where of all of them you know when 4 purchases took place. You calculate the intervals, thus for each customer you get 3 intervals. As input you give the network 2, you let the network predict the 3rd one, and then you let the network learn off its mistake (you can do that because you know what the 3rd one was in reality, that's how you train the network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Provided that the network isn't too complex or too simple (by that I mean that a hidden layer could be necessary, and you need to give it sufficient neurons) than the network might give a decent approximation. The only problem is, as Manuel Rodriguez said, that the network would indeed only learn to make predictions based on the corectness of its previous ones on a large group of people. Perfect predictions thus won't be possible, but you can minimize the error by making your dataset and input larger, so the network has more information to work on.&lt;/p&gt;&#xA;" OwnerUserId="21788" LastActivityDate="2019-03-05T10:04:57.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11016" PostTypeId="1" AcceptedAnswerId="11018" CreationDate="2019-03-05T10:30:06.413" Score="2" ViewCount="1323" Body="&lt;p&gt;I wanted to clarify the term 'acting greedily'. What does it mean? Does it correspond to the immediate reward, future reward or both combined? I want to know the actions that will be taken in 2 cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; is known and &lt;span class=&quot;math-container&quot;&gt;$R_s$&lt;/span&gt; is also known (only).&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s, a)$&lt;/span&gt; is known and &lt;span class=&quot;math-container&quot;&gt;$R_s^a$&lt;/span&gt; is also known (only).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-03-05T10:34:05.417" LastActivityDate="2019-03-05T16:49:17.883" Title="What does 'acting greedily' mean?" Tags="&lt;reinforcement-learning&gt;&lt;greedy-ai&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11017" PostTypeId="2" ParentId="11016" CreationDate="2019-03-05T11:13:53.590" Score="2" Body="&lt;p&gt;In general, a greedy &quot;action&quot; is an action that would lead to an immediate &quot;benefit&quot;. For example, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dijkstra's algorithm&lt;/a&gt; can be considered a greedy algorithm because at every step it selects the node with the smallest &quot;estimate&quot; to the initial (or starting) node. In reinforcement learning, a greedy action often refers to an action that would lead to the immediate highest &lt;strong&gt;reward&lt;/strong&gt; (&lt;em&gt;disregarding possible future rewards&lt;/em&gt;). However,  a greedy action can also mean the action that would lead to the highest possible &lt;strong&gt;return&lt;/strong&gt; (that is, the greedy action can also be considered an action that takes into account not just immediate rewards but also future ones).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, I think that the &quot;greedy action&quot; can mean different things, depending on weather you use the reward function or the value functions, that is, you can act greedily with respect to the reward function or the value functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to note that you are using a different notation for the reward function for each of the two value functions, but this does not need to be the case. So, your reward function might be expressed as &lt;span class=&quot;math-container&quot;&gt;$R_s^a$&lt;/span&gt; even if you use &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;. I will use the notation &lt;span class=&quot;math-container&quot;&gt;$R_s^a$&lt;/span&gt; for simplicity of the explanations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you have access to the reward function for a given state and action, &lt;span class=&quot;math-container&quot;&gt;$R^a_s = r(s, a)$&lt;/span&gt;, then the greedy action (with respect to the reward function &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;) would just be the action from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with the highest &lt;strong&gt;reward&lt;/strong&gt;. So, formally, we can define it as &lt;span class=&quot;math-container&quot;&gt;$a_\text{greedy} = \arg \max_a r(s, a)$&lt;/span&gt; (both in the case of the state or state-action value functions: it does not matter if you have one or the other value function). In other words, if you have access to the reward function (in that form), you can act greedily from any state without needing to access the value functions: you have a &quot;model&quot; of the rewards that you will obtain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, a)$&lt;/span&gt; (that is, the state-action value function for a &lt;em&gt;fixed&lt;/em&gt; policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;), then, at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the greedy action (with respect to &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, a)$&lt;/span&gt;) from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$a_\text{greedy} = \arg \max_{a}q_\pi(s, a)$&lt;/span&gt;. If you then take action &lt;span class=&quot;math-container&quot;&gt;$a_\text{greedy}$&lt;/span&gt; in the environment, you would obtain the highest discounted future reward (that is, the &lt;strong&gt;return&lt;/strong&gt;), according the &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, a)$&lt;/span&gt;, which might actually not be the highest &lt;em&gt;possible&lt;/em&gt; return from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, a)$&lt;/span&gt; might not be the optimal state-action value function. If &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, a) = q_{\pi^*}(s, a)$&lt;/span&gt; (that is, if you have the optimal state-action value function), then, if you execute &lt;span class=&quot;math-container&quot;&gt;$a_\text{greedy}$&lt;/span&gt; in the environment, you will theoretically obtain the highest possible return from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you had the optimal value function (the value function associated with the optimal policy to act in your environment), then the following equation holds &lt;span class=&quot;math-container&quot;&gt;$v_*(s) = \max_{a} q_{\pi^*}(s, a)$&lt;/span&gt;. So, in that case, &lt;span class=&quot;math-container&quot;&gt;$a_\text{greedy} = \arg \max_{a}q_{\pi^*}(s, a)$&lt;/span&gt; would also be the greedy action if you had &lt;span class=&quot;math-container&quot;&gt;$v_*(s)$&lt;/span&gt;. If you only have &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; (without e.g. the Q function), I don't think you can act greedily (that is, there is no way of knowing which action is the greedy action from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; by just having the value of state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;: this is actually why we often estimate the Q functions for &quot;control&quot;, i.e. acting in the environment).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-05T16:49:17.883" LastActivityDate="2019-03-05T16:49:17.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11018" PostTypeId="2" ParentId="11016" CreationDate="2019-03-05T11:14:02.563" Score="3" Body="&lt;p&gt;In RL, the phrase &amp;quot;acting greedily&amp;quot; is usually short for &amp;quot;acting greedily with respect to the value function&amp;quot;. Greedy local optimisation turns up in other contexts, and it is common to specify what metric is being maximised or minimised. The value function is most often the discounted sum of expected future reward, and also the metric used when defining a policy as &amp;quot;acting greedily&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to define a policy as acting greedily with respect to immediate expected reward, but not the norm. As a special case, when the discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt; then it is the same as acting greedily with respect to the value function.&lt;/p&gt;&#xA;&lt;p&gt;When rewards are sparse (a common situation in RL), acting greedily with respect to expected immediate reward is not very useful. There is not enough information to make a decision.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I want to know the actions that will be taken in 2 cases:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; is known and &lt;span class=&quot;math-container&quot;&gt;$R_s$&lt;/span&gt; is also known (only).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To act greedily in RL, you would use the value function &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s')$&lt;/span&gt; - the value function of the next states. To do so, you need to know the environment dynamics - to go with the notation &lt;span class=&quot;math-container&quot;&gt;$R_s$&lt;/span&gt; you should also know the transition matrix &lt;span class=&quot;math-container&quot;&gt;$P_{ss'}^a$&lt;/span&gt; - the probability of transitioning from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; whilst taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi'(s) = \text{argmax}_a \sum_{s'} P_{ss'}^a(R_{s} + \gamma v_{\pi}(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;This assumes &lt;span class=&quot;math-container&quot;&gt;$R_{s}$&lt;/span&gt; is your immediate reward for &lt;em&gt;leaving&lt;/em&gt; state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Substitute &lt;span class=&quot;math-container&quot;&gt;$R_{s'}$&lt;/span&gt; if the reward matrix is for entering state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; instead.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The policy gained from acting greedily with respect to &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; is &lt;em&gt;not&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, it is (a usually improved) policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s, a)$&lt;/span&gt; is known and &lt;span class=&quot;math-container&quot;&gt;$R_s^a$&lt;/span&gt; is also known (only).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To act greedily in RL, you would use the value function &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s, a)$&lt;/span&gt;, and this is much simpler:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi'(s) = \text{argmax}_a q_{\pi}(s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Again, the policy gained from acting greedily with respect to &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt; is &lt;em&gt;not&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, it is (a usually improved) policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-03-05T11:14:02.563" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11023" PostTypeId="2" ParentId="11016" CreationDate="2019-03-05T12:24:06.007" Score="1" Body="&lt;p&gt;Acting greedily means that the search is not forward thinking and limits its decisions solely on immediate return. It is not quite the same as what is meant in human social contexts in that greed in that context can involve forward thinking strategies that sacrifice short term losses for long term gain. In the typical machine search lingo, greed is myopic (short-sighted).&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-03-05T12:24:06.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11024" PostTypeId="2" ParentId="11004" CreationDate="2019-03-05T12:27:09.167" Score="0" Body="&lt;p&gt;If you are serious about OCR systems and not just trying to learn CNN, start with a known good OSS product like tesseract. OCR some 1200 DPI grayscale documents, reverse engineer it to some degree so that you understand the design, join the development team, and pick up a ticket or two. Once you get into pull requests for a real system that is in constant use for a while, you will be an expert. Starting from scratch would make sense after you are an expert.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-03-05T12:27:09.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11027" PostTypeId="2" ParentId="8751" CreationDate="2019-03-05T13:25:05.560" Score="1" Body="&lt;p&gt;Before I answer your question, it is important to frame what a GA is, so please allow me to cover some history.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wiley.com/en-us/Evolutionary+Computation%3A+Toward+a+New+Philosophy+of+Machine+Intelligence%2C+3rd+Edition-p-9780471669517&quot; rel=&quot;nofollow noreferrer&quot;&gt;Friedman and Fraser&lt;/a&gt; performed some of the earliest evolutionary computation experiments. Fraser presented the case of diploid organisms represented by &lt;em&gt;binary strings&lt;/em&gt; of &lt;em&gt;a given length&lt;/em&gt;. Each bit in the string represented a gene. Fraser proposed the process of single-point crossover for binary string reproduction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;American computer scientist &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;John Henry Holland&lt;/a&gt; and his colleagues at the University of Michigan formalised the evolutionary algorithms originally implemented by Bremermann, Fraser, Friedman and Friedberg.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Holland's formal algorithm is known as a &lt;em&gt;Genetic Algorithm&lt;/em&gt; (GA). A GA has a &lt;em&gt;fixed length chromosome structure&lt;/em&gt;. Traditionally it was binary based. However due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Hamming_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;hamming&lt;/a&gt; distances of various bit strings a GA using a binary encoded chromosome string can get stuck at local &lt;a href=&quot;https://en.wikipedia.org/wiki/Maxima_and_minima&quot; rel=&quot;nofollow noreferrer&quot;&gt;minima&lt;/a&gt;. Holland experimented with integer based GAs too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting back to your question, theoretically and in its strictest sense, a Genetic Algorithm has a fixed length chromosome vector. The primary reason for this is that each gene represents something. If it was variable then the meaning of the gene must be encoded into the chromosome, which makes evolution more difficult. However, there are other chromosome structures that are variable length. The chromosome structure itself &lt;em&gt;implies&lt;/em&gt; meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.genetic-programming.com/johnkoza.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Koza&lt;/a&gt; experimented with an alternative version of a GA called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic programming&lt;/a&gt; (&lt;strong&gt;GP&lt;/strong&gt;). A GP replaces the fixed length vector structure used in a genetic algorithm with a &lt;strong&gt;variable length&lt;/strong&gt;, non-linear, hierarchical abstract data structure, known as a &lt;em&gt;tree&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tree consists of branch nodes and leaf nodes. Leaf nodes have no children and implement a number or constant from a terminal set. A leaf node is also known as a terminal node. A branch node is connected to other branch nodes or leaf nodes and implement an operator from a functional set. A branch node is also known as a non-terminal node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;tree&lt;/em&gt; structure naturally accommodates &lt;strong&gt;a game rule&lt;/strong&gt;. A tree is not restricted to a fixed length. A tree is a natural structure to represent hierarchical decisions rules, and a tree can store both binary, logical, and numeric functions within its nodes. Logical functions allow the GP to evolve conditional execution of game play functions not easily implemented using a fixed length vector. Tree does need a traversal path and grammar (syntax and semantics).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In short to answer your questions&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A GA (fixed length) is generally not used for games, but rather a GP (variable with decisions).  &lt;/li&gt;&#xA;&lt;li&gt;A GA is not the most suitable method for this kind of problem as it is fixed&#xA;length, and each gene corresponds to a rule or value.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would implement a GP, where the operators of mutation and crossover are adapted for a GP. I would also use the game itself as a fitness function, where a pool of individuals either using a single population, or co-evolution evolve players that gradually increase their score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paragraphs above were taken from Nicholls' MSc thesis. His thesis was on stock market trading rule generation which is similar to game rule generation. His thesis is available &lt;a href=&quot;http://hdl.handle.net/2263/68477&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (Only available after April 2019).&lt;/p&gt;&#xA;" OwnerUserId="20508" LastEditorUserId="20508" LastEditDate="2019-03-06T08:16:22.553" LastActivityDate="2019-03-06T08:16:22.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11030" PostTypeId="2" ParentId="10345" CreationDate="2019-03-05T15:17:36.823" Score="2" Body="&lt;h1&gt;An 'AI'* is only as smart as the information you give it&lt;/h1&gt;&#xA;&lt;p&gt;You've got to add your own knowledge of the situation into this. Currently we have a transaction id which only really tells us that there is a transaction, a card number (identifying a user, I assume) and a date.&lt;/p&gt;&#xA;&lt;p&gt;The date can probably tell you most - what day of the week was it? What season (most sales experience some seasonality)? What time of day?&lt;/p&gt;&#xA;&lt;p&gt;Comparing several dates can then tell you things like the average gap - deviation on that average.&lt;/p&gt;&#xA;&lt;p&gt;You can use machine learning models to tell you how good these variables are at predicting the next visit day but you have to create these variables first, the model won't know about seasonality or its effect on sales of ice cream, umbrellas or winter jumpers so you have to use your knowledge of your customer base to pass the right variables to the model.&lt;/p&gt;&#xA;&lt;p&gt;You might also want to consider the product purchased - if you can see that information - someone who buys a pint of milk or a loaf of bread will probably return for the same goods on a weekly basis (or whenever they run out) but someone who bought a set of screw drivers and returned for a hammer a week later is unlikely to return for the same goods.&lt;/p&gt;&#xA;&lt;p&gt;The vast majority of work done for most predictive systems is in creating your variables and providing something to train on which will hold the pertinent information.&lt;/p&gt;&#xA;&lt;p&gt;*I'm assuming here that you're working with a machine learning model&lt;/p&gt;&#xA;" OwnerUserId="22897" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-03-05T20:55:25.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11032" PostTypeId="1" CreationDate="2019-03-05T17:58:07.273" Score="1" ViewCount="33" Body="&lt;p&gt;Are there any (well validated) approaches for applying pathfinding algorithms on a graph following specific rules?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific: I want to introduce a graph with coloured edges. The idea is to apply a well known pathfinding algorithm (such as Dijkstra) on the graph given the rule: &quot;only black and red edges&quot;.&lt;/p&gt;&#xA;" OwnerUserId="19413" LastActivityDate="2019-03-05T18:25:49.640" Title="Which pathfinding algorithms can be applied on coloured graphs?" Tags="&lt;path-finding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11033" PostTypeId="2" ParentId="11008" CreationDate="2019-03-05T18:16:42.227" Score="4" Body="&lt;p&gt;My sense is that they would, based on a high-level take of &lt;a href=&quot;https://en.wikipedia.org/wiki/Charles_Babbage&quot; rel=&quot;nofollow noreferrer&quot;&gt;Babbage&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Ada_Lovelace&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lovelace&lt;/a&gt;'s view of the potential capability of the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Analytical_Engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;analytic engine&lt;/a&gt;&quot;.  If Babbage's Tic-Tac-Toe machine had been built, I am sure that would have been regarded as machine intelligence.  &lt;a href=&quot;https://history-computer.com/ModernComputer/Relays/Condon.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nimatron&lt;/a&gt; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Edward_Condon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edward Condon&lt;/a&gt;) may have been the first game AI, and the capability seems similar to what Babbage was envisioning.  Certainly the bogus &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/The_Turk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turk&lt;/a&gt;&quot; chess-playing hoax was considered a machine intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conventional software could connote a form of automation, and I think any form of automation, particularly where the operations are &quot;under the covers&quot;, would have been considered a form of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I think the current idea of only regarding &quot;strong statistical AI&quot; (Machine Learning) as AI is inherently flawed because of the concept of utility.  Intelligence is a spectrum, being a relative measure of problem solving strength, and artificial merely connotes a thing intentionally or skillfully constructed.  The Russell &amp;amp; Norvig definition seems to hew to this viewpoint.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-05T18:36:43.657" LastActivityDate="2019-03-05T18:36:43.657" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11034" PostTypeId="2" ParentId="8747" CreationDate="2019-03-05T18:22:42.810" Score="1" Body="&lt;p&gt;Remember that any machine learning model works good only when there is a &quot;rule&quot; or a correlation between modeling data and modeled data. When there is not, even the best algorithm will not predict/classify correctly. I am not saying that this must be the case, but probably you have come pretty close to the physical limit of what can be achieved using this data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you should consider adding or extracting new features to your model.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-03-05T18:22:42.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11035" PostTypeId="2" ParentId="11009" CreationDate="2019-03-05T18:25:46.370" Score="1" Body="&lt;p&gt;&lt;strong&gt;Winning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My feeling is that utility is the standard of determining intelligence, and merely connotes the strength of a decision making algorithm.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Winning is a metric involving strength in relation to other agents, but may not always result in optimal global utility.  (Here I'm thinking of a game AI that always wins.  In a game like Chess, a subset of serious players may appreciate this, but it's unclear that this would be any kind of fun for the average player.  Similarly, an unbeatable AI in a game like Civilization or Stellaris would be very poorly received by most players.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;http://julian.togelius.com/Lantz2017Depth.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Depth In Strategic Games&lt;/a&gt; they formalize the term &quot;strategy ladder&quot;. A simple definition of this concept is a series of AIs that are &quot;meaningfully weak&quot;.  Specifically, players learn to beat them and in so doing learn different levels of strategy, ascending up ladder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, user engagement is the ultimate goal, not the AI winning every game! &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Imitation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing makes a compelling case that imitation is an indication of intelligence (self-awareness and &quot;true comprehension&quot; as strictly corollary.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like to reduce this idea to Captcha, because it involves utility.  At some point Captcha may no longer work because the average bot is a &quot;smart&quot; in solving them as the lowest-common denominator human.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same goes for chat bot--it they can successfully imitate humans, they can be considered as smart as those humans in regard to chatting.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion:&lt;/strong&gt; Both winning and imitation are indicators of intelligence, whether machine-based of biological.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-05T18:44:20.440" LastActivityDate="2019-03-05T18:44:20.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11036" PostTypeId="2" ParentId="11032" CreationDate="2019-03-05T18:25:49.640" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The idea is to apply a well known pathfinding algorithm (such as Dijkstra) on the graph given the rule: &quot;only black and red edges&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Remove all the non-black-and-red edges from the graph first, then run it through any off-the-shelf pathfinding implementation.  Or implement your own, and have it completely ignore the non-black-and-red edges.&lt;/p&gt;&#xA;" OwnerUserId="19452" LastActivityDate="2019-03-05T18:25:49.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11037" PostTypeId="2" ParentId="8778" CreationDate="2019-03-05T18:44:27.790" Score="0" Body="&lt;p&gt;I have encountered similar problem while trying to predict forex prices. I understand it this way:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The data based on which you try to model price differences are so poor that the lowest error is achieved by zeroing the predicted values&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, due to poor &quot;correlation&quot; in data, zero values are the best solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice would be to look for more correlated data to be used.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="16565" LastEditDate="2019-03-06T12:27:39.840" LastActivityDate="2019-03-06T12:27:39.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11038" PostTypeId="1" AcceptedAnswerId="11045" CreationDate="2019-03-05T18:59:04.550" Score="1" ViewCount="62" Body="&lt;p&gt;I am interested in a framework for learning the similarity of different input representations based on some common context. I have looked into word2vec, SVD and siamese networks, all of which are similar to what I want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose we have some customers we are sending different advertisements to, and I would like to create a system to map offers to customers. I am thinking in the lines of creating a customer representation, and a representation of the offers, and feeding them in parallel to a neural network that has a label of whether they acted on the advertisement or not. The idea is that I should be able to locate the best offer for any customer given these representations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked into siamese networks and word2vec, both are close to what I want. The problem differs slightly in that for the siamese networks, it tends to be identical parallel networks, which I don't want because my inputs are not equivalent. And for word2vec the vectors tend to be in the same domain, while I want to apply this in a more general setting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has any resources on a similar problem statement, I would be very interested in it.&lt;/p&gt;&#xA;" OwnerUserId="22906" LastEditorUserId="2444" LastEditDate="2019-04-03T09:10:43.237" LastActivityDate="2019-04-03T09:10:43.237" Title="Learning similarities between customers and offers representation" Tags="&lt;deep-learning&gt;&lt;knowledge-representation&gt;&lt;word2vec&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11041" PostTypeId="2" ParentId="10675" CreationDate="2019-03-05T20:03:33.477" Score="5" Body="&lt;p&gt;From the book &lt;em&gt;Reinforcement Learning, An Introduction (R. Sutton, A. Barto)&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The term &lt;strong&gt;system identification&lt;/strong&gt; is used in adaptive control for what we&#xA;  call &lt;strong&gt;model-learning&lt;/strong&gt; (e.g., Goodwin and Sin, 1984; Ljung and S&#xA;  ̈oderstrom, 1983; Young, 1984).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Model-learning&lt;/strong&gt; refers to the act of learning the model (environment). Reinforcement Learning can be divided into two types:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Model-based&lt;/strong&gt; - first we build a model of an environment and then do the control.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Model-free&lt;/strong&gt; - we do not try to model the behaviour of the environment. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Policy learning&lt;/strong&gt; is the act of learning optimal policy. You can do it in two ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;On-policy learning&lt;/strong&gt; - learn about the policy &lt;span class=&quot;math-container&quot;&gt;$π$&lt;/span&gt; by sampling from the same policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Off-policy learning&lt;/strong&gt; - learn about the policy &lt;span class=&quot;math-container&quot;&gt;$π$&lt;/span&gt; from the experience sampled from some other policy (e.g. watching different agent playing a game).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22835" LastActivityDate="2019-03-05T20:03:33.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11042" PostTypeId="1" CreationDate="2019-03-05T20:39:26.673" Score="1" ViewCount="20" Body="&lt;p&gt;I would like to train a constrained neural network. I found a paper on this: &lt;a href=&quot;https://papers.nips.cc/paper/4-constrained-differential-optimization.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://papers.nips.cc/paper/4-constrained-differential-optimization.pdf&lt;/a&gt;.&#xA;However, I don't really understand how to change my feedforward neural network. I don't understand exactly the role of the new output neurons, which serve as lagrangian multipliers. Can someone explain this to me in more detail? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which steps in backpropagation do I have to change?&lt;/p&gt;&#xA;" OwnerUserId="19195" LastEditorUserId="2444" LastEditDate="2019-03-05T21:29:56.373" LastActivityDate="2019-03-05T21:29:56.373" Title="What is the purpose of the new neurons in the constrained neural network?" Tags="&lt;neural-networks&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11043" PostTypeId="1" AcceptedAnswerId="12471" CreationDate="2019-03-05T23:22:24.627" Score="2" ViewCount="377" Body="&lt;p&gt;I'm looking to build from scratch an implementation of the &lt;a href=&quot;https://www.cs.toronto.edu/~hinton/absps/ws.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;wake-sleep algorithm&lt;/a&gt;, for unsupervised learning with neural networks. I plan on doing this in Python in order to better understand how it works. In order to facilitate my task, I was wondering if anyone could point me to an existing (open-source) implementation of this concept. I'm also looking for articles or, in general, resources that could facilitate this task.&lt;/p&gt;&#xA;" OwnerUserId="22781" LastEditorUserId="2444" LastEditDate="2019-11-18T18:49:13.673" LastActivityDate="2019-11-18T18:54:56.627" Title="Where can I find an implementation of the wake-sleep algorithm?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;unsupervised-learning&gt;&lt;resource-request&gt;&lt;wake-sleep-algorithm&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11044" PostTypeId="2" ParentId="10947" CreationDate="2019-03-06T01:39:18.747" Score="2" Body="&lt;p&gt;Yes, it does and at many parts of the solution. For one of the core components - intent detection - Intento did a benchmark comparing IBM Watson and other similar products: &lt;a href=&quot;https://www.slideshare.net/KonstantinSavenkov/nlu-intent-detection-benchmark-by-intento-august-2017&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.slideshare.net/KonstantinSavenkov/nlu-intent-detection-benchmark-by-intento-august-2017&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outside of intent detection, there are other areas where AI techniques help - e.g. disambiguation, bootstrapping a bot from chat logs etc. Specifically for IBM Watson, you can learn more here: &lt;a href=&quot;https://www.ibm.com/cloud/watson-assistant/features/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ibm.com/cloud/watson-assistant/features/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22914" LastActivityDate="2019-03-06T01:39:18.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11045" PostTypeId="2" ParentId="11038" CreationDate="2019-03-06T03:23:45.240" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The idea is that I should be able to locate the best offer for any customer given these representations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you need a &lt;a href=&quot;https://en.wikipedia.org/wiki/Recommender_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recommender System&lt;/a&gt;. As you want to map the offers to customers based one their representation you can check &lt;a href=&quot;https://en.wikipedia.org/wiki/Recommender_system#Content-based_filtering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Content-Based Recommender System&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method is by taking pattern from a customer history and try to find similarities with the new offers. You can use many techniques for a recommender system, from a simple &lt;a href=&quot;https://towardsdatascience.com/recommender-engine-under-the-hood-7869d5eab072&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-IDF&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/pdf/1707.07435.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; for more complex problems.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-03-06T03:23:45.240" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11047" PostTypeId="1" CreationDate="2019-03-06T08:57:54.400" Score="1" ViewCount="55" Body="&lt;p&gt;I would like to find a way to isolate the speech of each of the people in an audio record so I can create a file of that form :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;   {&#xA;       &quot;voice_fingerprint&quot;: &quot;701066EDD3A0A40A2F53F61EAFD0E6AB&quot;,&#xA;       &quot;sentences&quot;: {&#xA;           {&#xA;               &quot;sentence&quot;: &quot;do you like red apples&quot;,&#xA;               &quot;position&quot;: 1.39 // Seconds. Time position in the audio record&#xA;           },&#xA;           {&#xA;               &quot;sentence&quot;: &quot;and how do you feel about time shifts&quot;,&#xA;               &quot;position&quot;: 7.21&#xA;           }&#xA;       }&#xA;   },&#xA;   {&#xA;       &quot;voice_fingerprint&quot;: &quot;8FFEA051AF3E3FB9A80A51A98FE05896&quot;,&#xA;       &quot;sentences&quot;: {&#xA;           {&#xA;               &quot;sentence&quot;: &quot;yes I do like them&quot;,&#xA;               &quot;position&quot;: 4.54&#xA;           },&#xA;           {&#xA;               &quot;sentence&quot;: &quot;i feel well about traveling&quot;,&#xA;               &quot;position&quot;: 10.18&#xA;           }&#xA;       }&#xA;   }&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This may be an interview record.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem &lt;strong&gt;IS NOT&lt;/strong&gt; the Speech to Text, but to &lt;strong&gt;isolate the two people's sentences&lt;/strong&gt;. Preferably in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you ever worked on this ? Do you have any hints ?&lt;/p&gt;&#xA;" OwnerUserId="22924" LastActivityDate="2020-02-13T18:02:40.313" Title="Isolate the speech of two people in an audio record with two people only" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11048" PostTypeId="2" ParentId="10050" CreationDate="2019-03-06T10:08:16.690" Score="1" Body="&lt;p&gt;If you simulate many trajectories and receive many estimates of the two returns you're interested in, you could empirically compare their &lt;a href=&quot;https://en.wikipedia.org/wiki/Variance#Sample_variance&quot; rel=&quot;nofollow noreferrer&quot;&gt;sample variances&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the variance of ordinary importance sampling is in general unbounded.&#xA;If you're wanting some theoretical bounds on the variance of importance sampling estimates, I'd start with &lt;em&gt;weighted importance sampling&lt;/em&gt;, whose variance converges to zero (&lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt;, section 5.5). &lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-06T10:08:16.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11049" PostTypeId="2" ParentId="9925" CreationDate="2019-03-06T10:34:08.733" Score="2" Body="&lt;p&gt;This is a learnable behavior, given enough data. &#xA;We would expect the an error to backpropagate to &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; whenever its use harmed classification accuracy. In this case, that would be whenever &lt;span class=&quot;math-container&quot;&gt;$|w|&amp;gt;0$&lt;/span&gt;. In general, I'm not sure how long this would take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the speed of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;'s convergence to zero would benefit from &lt;a href=&quot;http://cs231n.github.io/neural-networks-2/#reg&quot; rel=&quot;nofollow noreferrer&quot;&gt;regularization&lt;/a&gt;, which is often basically a penalty on the magnitude of your network weights added to the loss function you're optimizing. If &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; truly doesn't matter to classification, then regularization will definitely drive it to zero.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-10T08:34:35.833" LastActivityDate="2019-03-10T08:34:35.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11050" PostTypeId="1" AcceptedAnswerId="11147" CreationDate="2019-03-06T10:41:05.043" Score="2" ViewCount="321" Body="&lt;p&gt;Following &lt;a href=&quot;https://arxiv.org/pdf/1704.03732.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-learning from Demonstrations&lt;/a&gt;, I'd like to avoid potentially unsafe behavior during early learning by making use of supervised learning with demonstration data. However, the &lt;a href=&quot;https://github.com/AurelianTactics/dqfd-with-keras/blob/master/keras_dqfd.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementation&lt;/a&gt; I'm following still uses an environment. Can I train my agent without an environment at all?&lt;/p&gt;&#xA;" OwnerUserId="22930" LastEditorUserId="22916" LastEditDate="2019-03-11T08:10:48.217" LastActivityDate="2019-03-11T10:04:56.840" Title="Is there a way to train an RL agent without any environment?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11051" PostTypeId="2" ParentId="11043" CreationDate="2019-03-06T10:45:51.243" Score="1" Body="&lt;p&gt;I don't know if you are looking for something in a library, but I've found &lt;a href=&quot;https://github.com/jackklys/wake-sleep-modifications/blob/master/wakesleep/ws.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; in a public Github (I've not checked deeply if it fits for you).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that's what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="22930" LastActivityDate="2019-03-06T10:45:51.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11055" PostTypeId="1" CreationDate="2019-03-06T13:23:32.007" Score="13" ViewCount="3185" Body="&lt;p&gt;I am working on creating an RL-based AI for a certain board game. Just as a general overview of the game so that you understand what it's all about: It's a discrete turn-based game with a board of size &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; depending on the number of players). Each player gets an &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; number of pieces, which the player must place on the board. In the end, the one who has the least number of pieces wins. There are of course rules as to how the pieces can be placed so that not all placements are legal at every move.&lt;/p&gt;&#xA;&lt;p&gt;I have the game working in an OpenAI's gym environment (i.e. control by &lt;code&gt;step&lt;/code&gt; function), have the board representation as the observation, and I have defined the reward function.&lt;/p&gt;&#xA;&lt;p&gt;The thing I am struggling with right now is to meaningfully represent the action space.&lt;/p&gt;&#xA;&lt;p&gt;I looked into how AlphaZero tackles chess. The action space there is &lt;span class=&quot;math-container&quot;&gt;$8*8*73 = 4672$&lt;/span&gt;: for every possible tile on the board, there are 73 movement-related modalities. So, for every move, the algorithm comes up with 4672 values, the illegal ones are set to zero and non-zero ones are re-normalized.&lt;/p&gt;&#xA;&lt;p&gt;Now, I am not sure if such an approach would be feasible for my use-case, as my calculations show that I have a theoretical cap of ~30k possible actions (&lt;span class=&quot;math-container&quot;&gt;$n * n * m$&lt;/span&gt;) if using the same way of calculation. I am not sure if this would still work on, especially considering that I don't have the DeepMind computing resources at hand.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, my question: &lt;em&gt;Is there any other way of doing it apart from selecting the legal actions from all theoretically possible ones?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The legal actions would be just a fraction of the ~30k possible ones. However, at every step, the legal actions would change because every new piece determines the new placement possibilities (also, the already placed pieces are not available anymore, i.e. action space generally gets smaller with every step).&lt;/p&gt;&#xA;&lt;p&gt;I am thinking of games, like Starcraft 2, where action space must be larger still and they demonstrate good results, not only by DeepMind but also by private enthusiasts with for example DQN.&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any ideas, hints, or readings!&lt;/p&gt;&#xA;" OwnerUserId="21278" LastEditorUserId="2444" LastEditDate="2021-01-11T23:57:37.147" LastActivityDate="2021-01-11T23:57:37.147" Title="How to deal with a huge action space, where, at every step, there is a variable number of legal actions?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;game-ai&gt;&lt;action-spaces&gt;&lt;board-games&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11056" PostTypeId="1" CreationDate="2019-03-06T13:35:58.420" Score="3" ViewCount="69" Body="&lt;p&gt;Some formulations of humanism already grant moral consideration to sentient non-human animals (e.g. &lt;a href=&quot;https://humanists.international/what-is-humanism/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://humanists.international/what-is-humanism/&lt;/a&gt;). Does humanism also extend to granting rights to AGIs, should they become sentient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sentientism is a closely related philosophy that makes this explicit &lt;a href=&quot;https://en.wikipedia.org/wiki/Sentientism&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Sentientism&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22935" LastEditorUserId="22935" LastEditDate="2019-03-07T16:23:14.293" LastActivityDate="2019-03-07T18:26:36.317" Title="Does humanism grant moral consideration to sentient artificial general intelligences?" Tags="&lt;philosophy&gt;&lt;ethics&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11057" PostTypeId="1" AcceptedAnswerId="11133" CreationDate="2019-03-06T14:07:16.067" Score="19" ViewCount="6171" Body="&lt;p&gt;In mathematics, the word &lt;a href=&quot;https://en.wikipedia.org/wiki/Operator_(mathematics)&quot; rel=&quot;noreferrer&quot;&gt;operator&lt;/a&gt; can refer to several distinct but related concepts. An operator can be defined as a function between two vector spaces, it can be defined as a function where the domain and the codomain are the same, or it can be defined as a function from functions (which are vectors) to other functions (for example, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Differential_operator&quot; rel=&quot;noreferrer&quot;&gt;differential operator&lt;/a&gt;), that is, a high-order function (if you are familiar with functional programming).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the &lt;em&gt;Bellman operator&lt;/em&gt; in reinforcement learning (RL)? Why do we even need it? How is the Bellman operator related to the Bellman equations in RL?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-11T10:11:50.917" LastActivityDate="2020-08-12T17:40:35.970" Title="What is the Bellman operator in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;math&gt;&lt;bellman-equations&gt;&lt;bellman-operators&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="11058" PostTypeId="1" CreationDate="2019-03-06T14:21:30.907" Score="1" ViewCount="583" Body="&lt;p&gt;I have 10 variables as like below&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_1=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_2=2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_3=3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_4=4$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_5=5$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_6=6$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_7=7$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_8=8$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_9=9$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V_{10}=10$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Note: Each variable can have any value&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now I want to select the best 3 variables combination as like below&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_1V_3V_4$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$V_{10}V_1V_7$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$V_5V_3V_9$&lt;/span&gt; etc.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The best combination is nothing but the sum of the values of 3 variables in the combination.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Combination 1(&lt;span class=&quot;math-container&quot;&gt;$V_1V_2V_3$&lt;/span&gt;) : 1+2+3 &lt;span class=&quot;math-container&quot;&gt;$\Rightarrow$&lt;/span&gt; &lt;strong&gt;6&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Combination 2(&lt;span class=&quot;math-container&quot;&gt;$V_8V_9V_{10}$&lt;/span&gt;) : 8+9+10 &lt;span class=&quot;math-container&quot;&gt;$\Rightarrow$&lt;/span&gt; &lt;strong&gt;27&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the above example &lt;strong&gt;Combination 2(&lt;span class=&quot;math-container&quot;&gt;$V_8V_9V_{10}$&lt;/span&gt;)&lt;/strong&gt; has the highest sum value. So the &lt;strong&gt;Combination 2(&lt;span class=&quot;math-container&quot;&gt;$V_8V_9V_{10}$&lt;/span&gt;)&lt;/strong&gt; is the best combination here.&lt;/p&gt;&#xA;&lt;p&gt;Like this, if I have a large number of variables means which machine learning algorithm selects the best combination in all the sense.&lt;/p&gt;&#xA;&lt;p&gt;Suggest me the best machine learning algorithm for selecting the best variable combinations. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="22931" LastEditorUserId="2444" LastEditDate="2021-03-28T01:26:50.643" LastActivityDate="2021-03-28T01:27:04.743" Title="What is the best machine learning algorithm to select best 3 variable combinations?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11059" PostTypeId="1" AcceptedAnswerId="11064" CreationDate="2019-03-06T14:44:40.287" Score="4" ViewCount="414" Body="&lt;p&gt;The question already has some &lt;a href=&quot;https://stackoverflow.com/questions/37370015/what-is-the-difference-between-value-iteration-and-policy-iteration&quot;&gt;answer&lt;/a&gt;. But I am still finding it quite unclear (also does &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; here mean &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; ?):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qM4a2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qM4a2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The few things I do not understand are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why the difference between 2 iterations if we are acting greedily in each of them? As per many sources 'Value Iteration' does not have an explicit policy, but here we can see the policy is to act greedily to current &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;What exactly does Policy Improvement mean? Are we acting greedily only at a particular state at a particular iteration &lt;strong&gt;OR&lt;/strong&gt; once we act greedily on a particular state we keep on acting greedily on that state and other states are added iteratively until in all states we act greedily?&lt;/li&gt;&#xA;&lt;li&gt;We can intuitively understand that acting greedily w.r.t &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; will lead to &lt;span class=&quot;math-container&quot;&gt;$v^*(s)$&lt;/span&gt; eventually, but does using Policy Iteration eventually lead to &lt;span class=&quot;math-container&quot;&gt;$v^*(s)$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; I have been thinking of all the algorithms in context of &lt;strong&gt;Gridworld&lt;/strong&gt;, but if you think there is a better example to illustrate the difference you are welcome.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-03-06T16:41:23.103" LastActivityDate="2019-03-06T20:16:16.243" Title="A few questions regarding the difference between policy iteration and value iteration" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-02-05T10:23:15.260" ContentLicense="CC BY-SA 4.0" />
  <row Id="11062" PostTypeId="1" CreationDate="2019-03-06T17:28:51.890" Score="1" ViewCount="37" Body="&lt;p&gt;I don't know if this it's possible but nowadays as almost everything is possible I am asking to see if anyone has any idea.&lt;/p&gt;&#xA;&lt;p&gt;The problem is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Regularly I have to import files (CSV, XML, Excel, ...) to an SQL Server Database and after I import the files I have to map the content of the columns of the imported files in a .sql Script to run and insert in a table in another DB (this table is the same for all the files imported).&lt;/p&gt;&#xA;&lt;p&gt;I spend a lot of time on these mappings as you can imagine, I want to know if it's possible through Artificial Intelligence or Machine Learning one solution that can make the import of the initial data to the final DB, the initial import I can make it automatically the problem is only the import of the imported files to the final DB.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: The initial files don't have the same columns names, each file can have different columns names.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Sorry for the long post but I don't know if it is possible something like that. I searched and don't find anything.&lt;/p&gt;&#xA;" OwnerUserId="22940" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-03-07T09:07:40.323" Title="Automation the import of files to Database" Tags="&lt;machine-learning&gt;&lt;automation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11064" PostTypeId="2" ParentId="11059" CreationDate="2019-03-06T20:16:16.243" Score="3" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; does not mean &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; here. &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; is a policy that represents probability distribution over action space for a specific state. &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; is a state-action pair value function that tells us how much reward do we expect to get by taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; onwards.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the value iteration on the right side with this update formula:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v(s) \leftarrow \max_\limits{a} \sum_\limits{s'}p(s'\mid s, a)[r(s, a, s') + \gamma v(s')]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have an implicit &lt;strong&gt;greedy deterministic&lt;/strong&gt; policy that updates value of state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; based on the greedy action that gives us the biggest expected return. When the value iteration converges to its values based on greedy behaviour after &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; iterations we can get the explicit optimal policy with:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \arg \max_\limits{a} \sum_\limits{s'} p(s'\mid s, a)[r(s, a, s') + \gamma v(s')]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;here we are basically saying that the action that has highest expected reward for state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; will have probability of 1, and all other actions in action space will have probability of 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the policy evaluation on the left side with this update formula:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v(s) \leftarrow \sum_\limits{s'}p(s'\mid s, \pi(s))[r(s, \pi(s), s') + \gamma v(s')]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have an explicit policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; that is &lt;strong&gt;not greedy&lt;/strong&gt; in general case in the beginning. That policy is usually randomly initialized so the actions that it takes will not be greedy, it means we can start with policy that takes some pretty bad actions. It also does not need to be deterministic but I guess in this case it is. Here we are updating value of state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; according to the current policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;br&gt;&#xA;After policy evaluation step ran for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; iterations we start with the policy improvement step:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \arg \max_\limits{a} \sum_\limits{s'} p(s'\mid s, a)[r(s, a, s') + \gamma v(s')]$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;here we are greedily updating our policy based on the values of states that we got through policy evaluation step. It is guaranteed that our policy will improve but it is not guaranteed that our policy will be optimal after only one policy improvement step. After improvement step we do the evaluation step for new improved policy and after that we again do the improvement step and so on until we converge to the optimal policy&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-03-06T20:16:16.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11069" PostTypeId="2" ParentId="10575" CreationDate="2019-03-07T02:00:16.780" Score="5" Body="&lt;p&gt;&lt;strong&gt;Advantage function&lt;/strong&gt;: &lt;span class=&quot;math-container&quot;&gt;$A(s,a) = Q(s,a) - V(s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;More interesting is the &lt;strong&gt;General Value Function (GVF)&lt;/strong&gt;, the expected sum of the (discounted) future values of some arbitrary signal, not necessarily reward. It is therefore a generalization of value function &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;. The GVF is defined on page 459 of the 2nd edition of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's RL book&lt;/a&gt; as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi,\gamma,C}(s) =\mathbb{E}\left[\left.\sum_{k=t}^\infty\left(\prod_{i=t+1}^k \gamma(S_i)\right)C_{k+1}\right\rvert S_t=s, A_{t:\infty}\sim\pi\right]$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$C_t \in \mathbb{R}$&lt;/span&gt; is the signal being summed over time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\gamma(S_t)$&lt;/span&gt; is a function &lt;span class=&quot;math-container&quot;&gt;$\gamma: \cal{S}\to[0,1]$&lt;/span&gt; allowing the discount rate to depend upon the state. Sutton and Barto call it the termination function. &lt;a href=&quot;http://reinforcement-learning.ml/papers/pgmrl2018_sherstan.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some&lt;/a&gt; call it the continuation function.&lt;/p&gt;&#xA;&lt;p&gt;Also of note are the &lt;strong&gt;differential value functions&lt;/strong&gt;. These are used in the continuing, undiscounted setting. Because there is no discounting, the expected sum of future rewards is unbounded. Instead, we optimize the expected differential reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}-r(\pi)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt; is the average reward under policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi,\,diff}(s) = \sum_a \pi(a|s) \sum_{s',r} p(s',r|s,a)\left[r-r(\pi)+ v_{\pi,\,diff}(s')\right]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_{*,\,diff}(s) = \max_a \sum_{s',r} p(s',r|s,a)\left[r-\max_\pi r(\pi)+ v_{*,\,diff}(s')\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The differential value functions assume that a single fixed value of &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt; exists. That is, they assume the MDP is &amp;quot;ergodic.&amp;quot; See section 10.3 of Sutton and Barto for details.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="2444" LastEditDate="2020-11-23T14:03:51.547" LastActivityDate="2020-11-23T14:03:51.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11070" PostTypeId="1" CreationDate="2019-03-07T02:18:47.097" Score="2" ViewCount="106" Body="&lt;p&gt;I would like to start programming a multi task reinforcement learning model. For this, I need not just one maze or grid world (or just model-based), but many with different reward functions. So, I am wondering if exists a dataset or a generator for such thing, or do I need to code everything by my self?&lt;/p&gt;&#xA;" OwnerUserId="22949" LastEditorUserId="2444" LastEditDate="2019-03-07T08:51:25.057" LastActivityDate="2019-03-07T08:51:25.057" Title="Is there any grid world dataset or generator for reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;model-based-methods&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11071" PostTypeId="2" ParentId="11070" CreationDate="2019-03-07T03:06:45.373" Score="2" Body="&lt;p&gt;Depending on your needs and the size of the project, you might be better off making a custom set of environments. If you'd rather not do that, though, you should take a look at OpenAI's &lt;a href=&quot;https://github.com/openai/coinrun&quot; rel=&quot;nofollow noreferrer&quot;&gt;CoinRun environment&lt;/a&gt;. A high-level description can be found in their &lt;a href=&quot;https://blog.openai.com/quantifying-generalization-in-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;RandomMazes&quot; version of this environment might be useful to you. And if you want to make the mazes smaller, you can redefine MAX_MAZE_DIFFICULTY in coinrun.cpp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, although the levels are procedurally generated, reward is only ever given when the agent picks up the single coin. So, this might not be as much variety in reward function as you wanted.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-07T03:11:48.257" LastActivityDate="2019-03-07T03:11:48.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11073" PostTypeId="2" ParentId="10895" CreationDate="2019-03-07T07:01:03.077" Score="1" Body="&lt;p&gt;You are right that you will have to implement the above steps before building any chat bot, this is because your computer doesn't understand text like you and I do. Therefore, text has to be pre-processed(converted) into a format that the computer can work with Tokenization, Parts of Speech Tagging and Named Entity Recognition are such pre-processing techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rule based means when you create rules for deciding how should your chat bot behave in a given situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have a look at this blog which teaches how to deploy a chat bot using DialogFlow on Slack - &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2018/03/how-to-build-an-intelligent-chatbot-for-slack-using-dialogflow-api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.analyticsvidhya.com/blog/2018/03/how-to-build-an-intelligent-chatbot-for-slack-using-dialogflow-api/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22956" LastActivityDate="2019-03-07T07:01:03.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11074" PostTypeId="1" AcceptedAnswerId="11075" CreationDate="2019-03-07T07:50:52.097" Score="2" ViewCount="1256" Body="&lt;p&gt;I'm studying how SPP (Spatial, Pyramid, Pooling) works. SPP was invented to tackle the fix input image size in CNN. According to the original paper &lt;a href=&quot;https://arxiv.org/pdf/1406.4729.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1406.4729.pdf&lt;/a&gt;, the authors say: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;convolutional layers do not require a fixed image size and can&#xA;  generate feature maps of any sizes. On the other hand, the&#xA;  fully-connected layers need to have fixed size/length input by their&#xA;  definition. Hence, the fixed size constraint comes only from the&#xA;  fully-connected layers, which exist at a deeper stage of the network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why does a fully connected layer only accepts a fixed input size (but convolutional layers don't)? What's the real reason behind this definition?&lt;/p&gt;&#xA;" OwnerUserId="20612" LastEditorUserId="2444" LastEditDate="2019-03-07T08:45:49.117" LastActivityDate="2019-03-07T10:28:09.877" Title="Why does a fully connected layer only accept a fixed input size?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11075" PostTypeId="2" ParentId="11074" CreationDate="2019-03-07T09:19:46.780" Score="1" Body="&lt;p&gt;A convolutional layer is a layer where you slide a kernel or filter (which you can think of as a small square matrix of weights, which need to be learned during the learning phase) over the input. In practice, when you need to slide this kernel, you will often need to specify the &quot;padding&quot; (around the input) and &quot;stride&quot; (with which you convolve the kernel on the input), in order to obtain the desired output (size). So, even if you receive inputs of different sizes, you can change these values, like the padding or the stride, in order to produce a valid output (size). In this sense, I think, we can say that convolutional layers accept inputs of (almost) any size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of feature maps does not depend on the kernel or input (sizes). The number of features maps is determined by the number of different kernels that you will use to slide over the input. If you have &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; different kernels, then you will have &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; different feature maps. The number of kernels is often a hyper-parameter, so you can change it (as you please).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fully connected (FC) layer requires a fixed input size by design. The programmer decides the number of input units (or neurons) that the FC layer will have. This hyper-parameter often does not change during the learning phase. So, yes, FC often accept inputs of fixed size (also because they do not adopt techniques like &quot;padding&quot;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-03-07T09:19:46.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11076" PostTypeId="2" ParentId="11074" CreationDate="2019-03-07T10:28:09.877" Score="0" Body="&lt;p&gt;It doesn't have to be so. Fully connected layer could be considered as convolutional layer with input image of 1 pixel and spatial kernel size of 1 pixel. So 1-pixel kernel convolutional layer is effectively the same as fully connected layer attached to each pixel. That is idea behind &quot;Fully Convolutional Networks&quot;. If you want &quot;true&quot;, 1-pixel fully connected layer after convolutional layer (with variable input size) all you have to do is to put average pooling layer (or other type of pooling layer)  before fully connected layer. That way fully connected layer can accept variable input size.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-03-07T10:28:09.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11077" PostTypeId="1" CreationDate="2019-03-07T11:06:06.593" Score="2" ViewCount="122" Body="&lt;p&gt;I have series of sensors (around 4k) and each sensor will measure the amplitudes at each point.Suppose I train the neural network with sufficent set of 4k values (N * 4k shape). The machine will find a pattern in the series of values.If the values stray away from the pattern (that is anomaly)  it can detect the point and will be able to say that anomaly is in the 'X'th  sensor.Is this possible.If so what kind of neural network should I use?&lt;/p&gt;&#xA;" OwnerUserId="21936" LastActivityDate="2019-03-12T16:20:50.353" Title="Finding anomaly detection by pattern matching in a set of continous data" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11079" PostTypeId="1" AcceptedAnswerId="11082" CreationDate="2019-03-07T12:27:21.053" Score="4" ViewCount="118" Body="&lt;p&gt;Which Python packages do you recommend for random search hyperparameter optimization to use? Is there any recent and good one (better than the one in scikit-learn)?&lt;/p&gt;&#xA;" OwnerUserId="22784" LastEditorUserId="2444" LastEditDate="2020-03-12T15:12:53.057" LastActivityDate="2020-10-26T01:10:52.340" Title="Are there Python packages for random search hyper-parameter optimisation?" Tags="&lt;python&gt;&lt;hyperparameter-optimization&gt;&lt;resource-request&gt;&lt;scikit-learn&gt;&lt;bayesian-optimization&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-10-26T08:28:05.523" ContentLicense="CC BY-SA 4.0" />
  <row Id="11081" PostTypeId="1" AcceptedAnswerId="11086" CreationDate="2019-03-07T15:52:59.950" Score="3" ViewCount="466" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Excercise 3.5 The equastions in Section 3.1 are for the continuing&#xA;  case and need to be modified (very slightly) to apply to episodic&#xA;  tasks. Show that you know the modifications needed by giving the&#xA;  modified version of (3.3).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\displaystyle\sum_{s^{\prime} \in S} \displaystyle\sum_{r \in R} = p(s^{\prime}, r &#xA;| s,a) = 1$&lt;/span&gt; , for all &lt;span class=&quot;math-container&quot;&gt;$s\in S, a \in A(s)$&lt;/span&gt;    (3.3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it just about final states? So for &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; when S is not final? &lt;/p&gt;&#xA;" OwnerUserId="22826" LastEditorUserId="22826" LastEditDate="2019-03-07T16:19:21.257" LastActivityDate="2019-04-17T18:23:17.717" Title="Difference in continuing and episodic cases in Sutton and Barto - Introduction to RL, exercise 3.5" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;sutton-barto&gt;&lt;markov-chain&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11082" PostTypeId="2" ParentId="11079" CreationDate="2019-03-07T15:56:54.220" Score="1" Body="&lt;p&gt;Welcome to AI.SE Enes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think by random search, you are referring to so-called &quot;black-box optimization&quot;. Random search is sometimes used as a name for this, but BBO is a more common name, and might be easier to search for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;em&gt;many&lt;/em&gt; BBO techniques. 'random search' is usually used to refer to a hill-climbing algorithm where you start at a random location, sample from 'adjacent' points in the input space, and then pick the best one to move to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scikit-optimize.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scikit-optimize&lt;/a&gt; is fairly easy to use package, and implements some of the most reasonable algorithms (gaussian process is particularly strong on most problems). The package's website includes many good examples showing how to use the methods the package provides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not happy with those methods, then you are likely entering 'unsupported land'. There are lots of more complex, esoteric, methods, but I don't believe they are well consolidated into a single package (or even a single language). There's a nice collection on the &lt;a href=&quot;http://archimedes.cheme.cmu.edu/?q=dfocomp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sahindis lab's website&lt;/a&gt;, but those are in a mixture of C and Fortran. There are also a huge number of algorithms submitted to recurring competitions, like &lt;a href=&quot;https://bbcomp.ini.rub.de/#downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; at GECCO each year, but the implementations of these algorithms may not be available at all, or might be available only through the websites of their respective authors. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-07T15:56:54.220" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11083" PostTypeId="2" ParentId="11058" CreationDate="2019-03-07T16:07:21.947" Score="1" Body="&lt;p&gt;What you have is an instance of a &lt;em&gt;local search&lt;/em&gt; or &lt;em&gt;black-box optimization&lt;/em&gt; problem.&lt;/p&gt;&#xA;&lt;p&gt;There are many AI algorithms to do this, though as commenters have noted, with only 10 binary variables, you're happier to just look at all &lt;span class=&quot;math-container&quot;&gt;$2^{10}$&lt;/span&gt; of them.&lt;/p&gt;&#xA;&lt;p&gt;If you had, say, 60, then looking at them all is no longer a good idea (&lt;span class=&quot;math-container&quot;&gt;$2^{60}$&lt;/span&gt; is very large).&lt;/p&gt;&#xA;&lt;p&gt;The simplest BBO technique is called hill climbing. It looks like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick a random subset of the variables to be true.&lt;/li&gt;&#xA;&lt;li&gt;For each variable, determine the effect of flipping that variable from true to false in the current assignment.&lt;/li&gt;&#xA;&lt;li&gt;Select the 'flip' that leads to the largest value of your function.&lt;/li&gt;&#xA;&lt;li&gt;If the 'flip' has a larger value of your function than the current assignment, GOTO 2.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise, return the current assignment.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This amounts to: 'Pick an assignment at random, then keep making small changes that improve it until you can't anymore. Claim that this is the best solution.'&lt;/p&gt;&#xA;&lt;p&gt;Clearly, this algorithm will not recover the globally best solution with certainty, but you could run it many times, and eventually, it will find it with high probability.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;em&gt;complete&lt;/em&gt; algorithm, that will always uncover the global optimum, is &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulated annealing&lt;/a&gt;. This algorithm has a &lt;em&gt;temperature&lt;/em&gt; parameter, usually denoted T, which changes according to a &lt;em&gt;temperature schedule&lt;/em&gt;. If you start T large 'enough', and decrease it slowly 'enough', then it will find the global optimum. Unfortunately, 'enough' depends a lot on the shape of the function you are optimizing, which you generally cannot observe. There are some common guesses for T, like multiplying it by, e.g. 0.9999 at each step in the search, that seem to work okay in practice.&lt;/p&gt;&#xA;&lt;p&gt;Both of these algorithms are widely available in packages.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="36737" LastEditDate="2021-03-28T01:27:04.743" LastActivityDate="2021-03-28T01:27:04.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11084" PostTypeId="1" CreationDate="2019-03-07T16:11:56.240" Score="4" ViewCount="92" Body="&lt;p&gt;I'm writing a virtual environment for a 4-player card game named estimation, and will use deep reinforcement learning to teach an agent to play it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each player gets a hand of 13 cards, and the first phase is for each player to estimate the number of tricks they will collect. The highest player gets to start first,and then after each round, the player who collects the trick starts the next. So basically the first phase is for bidding and the next phase consists of 13 rounds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The state input I'll use will include all the cards that have been played, the goal and collected tricks, and the available cards. The output for each round will be a vector of length 54, containing all the cards and then the available card with the highest probability would be played.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first I thought that the bidding phase should use the same input but with zeros everywhere except the available hand, and the output would exclude all the cards with no numbers like king, queen or jack. But then the ability to dash (estimate that you'll collect 0 tricks) wouldn't be available. Also I don't think it would work really well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I just use two NNs for each phase, or what should I do? Also if anyone has any advice on things I need to watch out for, I'd really appreciate it if they shared them.&lt;/p&gt;&#xA;" OwnerUserId="22971" LastActivityDate="2019-03-09T09:01:46.283" Title="Teaching a neural network to play a card game with two phases" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11086" PostTypeId="2" ParentId="11081" CreationDate="2019-03-07T16:41:06.427" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it just about final states? So for &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; when S is not final? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You are thinking the right way, but to represent what you mean you don't need to write out &quot;when &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is not final&quot; - although that would be fine (and is used in some places), there is a more concise way of saying that given to you by the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this is a formal exercise from the book, I don't want to write out an answer that could be cut&amp;amp;paste for all students.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead I suggest you take a look at the notations section at the beginning of the book, and find how Sutton &amp;amp; Barto use different set labels for all states including terminal states, and all states excluding terminal states. Also, check carefully which of those sets needs to be summed over.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-03-07T16:41:06.427" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11087" PostTypeId="1" AcceptedAnswerId="11091" CreationDate="2019-03-07T16:59:53.727" Score="3" ViewCount="92" Body="&lt;p&gt;Let's use Excercise 3.8 from Sutton, Barto - Introduction to RL:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.5$&lt;/span&gt; and following sequence of rewards is received&#xA;  &lt;span class=&quot;math-container&quot;&gt;$R_1=-1$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$R_2=2$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$R_3=6$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$R_4=3$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$R_5=2$&lt;/span&gt; , with &lt;span class=&quot;math-container&quot;&gt;$T=5$&lt;/span&gt; . What&#xA;  are &lt;span class=&quot;math-container&quot;&gt;$G_0, G_1, ..., G_5?$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There isn't &lt;span class=&quot;math-container&quot;&gt;$G_5$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$R_5$&lt;/span&gt; is last reward. Am I understanding it right? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_4 = 2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_3 = 3 + 0.5*2 = 4$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_2 = 6+0.5*4 = 8$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_1 = 2+0.5*8 = 6$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_0 = -1 +0.5*6 = 2$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="22826" LastEditorUserId="2444" LastEditDate="2019-03-07T17:43:48.340" LastActivityDate="2019-03-07T19:48:32.480" Title="Given specific rewards, how can I calculate the returns for each time step?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11088" PostTypeId="5" CreationDate="2019-03-07T17:44:12.343" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-03-07T17:44:12.343" LastActivityDate="2019-03-07T17:44:12.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11089" PostTypeId="4" CreationDate="2019-03-07T17:44:12.343" Score="0" Body="For questions related to the concept of reward, for example, in the context of reinforcement learning and Markov decision processes. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T12:51:01.047" LastActivityDate="2021-01-20T12:51:01.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11090" PostTypeId="2" ParentId="11056" CreationDate="2019-03-07T18:20:05.747" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does humanism also extend to granting rights to AGIs, should they become sentient?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think that is going to be something that will divide self-identifying humanists into a few different groups, based on understanding about what AGI is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The front page you link includes &quot;aiming toward the welfare and fulfillment of living things&quot;, which does not explicitly include non-living sentient things. However, the humanist philosophy naturally extends to granting rights and protections based on reasoning and scientific evidence to anything that can be argued as needing them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The big problems are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Recognising and measuring sentience.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Attaching moral value to events that happen to artificial entities, where subjective measures such as pain, stress, happiness, fulfilment might not be relevant.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These measurement problems already cause a lot of variability in moral stances towards farming animals for instance. These biological creatures share enough with humans that even though we don't fully understand the nature of their subjective experience, it is possible to extrapolate with &lt;em&gt;some&lt;/em&gt; backing from scientific studies. For example, we can measure whether a creature suffers stress or pain and reacts similarly to a human, and even if we don't understand what that means subjectively (what it feels like to be such an animal), a combination of Occam's Razor and Precautionary Principle can be argued:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Occam's Razor&lt;/strong&gt; the simplest interpretation is that if an animal is based on same cellular structure, has same brain regions for experiencing pain and reward, uses same stress hormones etc, that its subjective experience is likely somehow comparable to a humans. This allows us to apply empathy to an animal's situation with a guess that doing so is not 100% inaccurate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Precautionary Principle&lt;/strong&gt; the consequences for inhumanely treating a sentient creature are morally worse than the waste of humanely treating a non-sentient creature (provided doing the latter is not causing undue suffering to some other thing). So faced with a lack of precise knowledge, there is still a strong rational ethics argument for extending rights and humane treatment if there is any uncertainty.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When it comes to artificially created entities, we lose the first argument of Occam's Razor, and can only get it back with a better understanding of what sentience actually is, and which parts of it we should value. We need a much firmer theory here than with animals, because there is so little shared between our construction and that of a human-built machine. In addition, just as there are many different kinds of organism which exhibit different amounts of apparent intelligence and possible sentience, there will be different versions of AGI, probably starting with purpose-built non-narrow AIs that combine features that are practically useful and make them &lt;em&gt;seem&lt;/em&gt; intelligent (parsing human speech and responding appropriately, which Google Home, Alexa, Siri already do to some depth) with some common-sense understanding of broad task portfolios. These things are near future, and most developers who work on them would not consider them sentient - however, they will be designed to act as if they were, because that makes them practical and usable, which in turn will make it much harder to argue Occam's Razor due to similarity in behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We still keep the second argument, but it is confounded by not understanding what the analogs are for &lt;em&gt;humane&lt;/em&gt; and &lt;em&gt;inhumane&lt;/em&gt; when dealing with artificial entities. It is unlikely that anyone would have a problem with switching off a running instance of the next 2 or 3 generations of Siri assistants - but at what point does that become deliberate incapacitation of a sentient being, and does it even matter if the system is designed to be switched on and off, and expresses no particular desire to be in either state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, the humanist philosophy &lt;em&gt;could&lt;/em&gt; extend to cover the welfare of artificial general intelligences, but it is far from clear whether it can go there in practice. Until we have more complete theoretical models of AGIs, or practical working devices, then it is premature to consider it. Most debate is going to be based on imagined qualities of AGI, and any two people discussing this are likely to be starting from different assumptions about what those are. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-07T18:26:36.317" LastActivityDate="2019-03-07T18:26:36.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11091" PostTypeId="2" ParentId="11087" CreationDate="2019-03-07T19:48:32.480" Score="1" Body="&lt;p&gt;Perfect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To back up your intuition about there not being a &lt;span class=&quot;math-container&quot;&gt;$G_5$&lt;/span&gt;, refer to the definition of discounted return in the periodic case (3.11). &#xA;&lt;span class=&quot;math-container&quot;&gt;$$G_t \doteq \sum_{k=t+1}^T \gamma^{k-t-1} R_k$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll see that &lt;span class=&quot;math-container&quot;&gt;$G_5$&lt;/span&gt; would be written as a sum with no terms in it, since &lt;span class=&quot;math-container&quot;&gt;$T=5$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-07T19:48:32.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11092" PostTypeId="1" CreationDate="2019-03-07T22:28:01.463" Score="1" ViewCount="75" Body="&lt;p&gt;Dialects differ a lot between cities in my country, Syria. People sometimes express themselves using different local phrases and idioms which refer to the same topic. So, I came up with the idea of creating an Android application shows a limited set of sentences or expressions while asking you to express them in the local dialect of your region &lt;strong&gt;orally&lt;/strong&gt;, after that this application tries to figure out what your dialect is. For a short period of time, I'm going to launch an Android application in order to collect the needed dataset which will be a new contribution. First of all, I need some helpful answers to my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In general, is a period of 6 months enough for such a project to be done by only one student who is a &lt;strong&gt;beginner in this field&lt;/strong&gt; or it is harder than it seems?&lt;/li&gt;&#xA;&lt;li&gt;Are the libraries and tools needed to do this project available for free?&lt;/li&gt;&#xA;&lt;li&gt;I know that more training data leads to more accurate results. In order to obtain good results, what is the estimated minimum number of training data needed for this model?&lt;/li&gt;&#xA;&lt;li&gt;How do you advise me to begin?&lt;/li&gt;&#xA;&lt;li&gt;How much is my suggested project relevant to the project attached in this &lt;a href=&quot;https://www.tensorflow.org/tutorials/sequences/audio_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;kindly write down your suggested edits and recommendations if any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; for the 5th question: also see &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1707/1707.09917.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;" OwnerUserId="20449" LastEditorUserId="20449" LastEditDate="2019-03-09T09:57:00.077" LastActivityDate="2019-03-09T09:57:00.077" Title="Dialects classification using deep learning" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;applications&gt;&lt;voice-recognition&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11093" PostTypeId="2" ParentId="10603" CreationDate="2019-03-07T22:31:22.660" Score="4" Body="&lt;p&gt;Another fallacy that appears common to most search engines is that anything a person searches on is an aspect of their own identity. I once searched on walk-in tubs for a very elderly relative, and was followed all over the web by ads for aids for the infirm elderly. Users who recognize that Google uses their searches to build their profile can alter their searches accordingly. It's also fun to mess with Google's model. Try searching on &quot;dragon images&quot; and see how fast Google and advertisers decide you are a teenage female. Have fun with it. Do your best to turn Google's model of you into self-contradictory garbage. &lt;/p&gt;&#xA;" OwnerUserId="22984" LastActivityDate="2019-03-07T22:31:22.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11095" PostTypeId="2" ParentId="4650" CreationDate="2019-03-08T05:26:01.637" Score="0" Body="&lt;p&gt;AI can also be used by Telcos to:&#xA;(1) Improve Usage &amp;amp; Retention efforts: Make relevant up-sell and cross-sell offers to the right users at the right time.&#xA;(2) Make segmentation more granular: make bespoke recommendations based on a user’s behavioral patterns and content preferences and assessing which call &amp;amp; data packages best suit different customer segments thereby increasing sales success rates&lt;/p&gt;&#xA;" OwnerUserId="22991" LastActivityDate="2019-03-08T05:26:01.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11096" PostTypeId="2" ParentId="11077" CreationDate="2019-03-08T07:38:42.023" Score="2" Body="&lt;p&gt;Without knowing the kind of data and the process generating it it's hard to give a definite answer.&#xA;In general, I would attempt a network that has as inputs the actual sensor readings, and outputs the expected readings. You train this network by presenting data with errors added as inputs, and correct readings as outputs.  It should learn to guess the correct (non-anomalous) readings from a set of actual readings, and you can find the sensors with anomalies by taking the difference between actual and predicted readings.&#xA;Depending on the kind of anomalies you expect (misreadings such as fluke zero or max values, or actually anomalous states of the system being measured) your training data should be set up to have samples of such anomalies.&#xA;If there are temporal correlations (for example, temperature readings that change slowly) using a RNN might be helpful but its dimensioning heavily depends on the nature of your measured system.&#xA;External factors influencing the system (state of a heating mechanism, time of day/year, etc.) could be added at inputs for better predictions.&#xA;At the end of the day, trial and error is your friend. Start with a simple network and see how well it behaves. Go for more complex networks when you see where the limitations of the simple solution are.&lt;/p&gt;&#xA;" OwnerUserId="22993" LastActivityDate="2019-03-08T07:38:42.023" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11098" PostTypeId="2" ParentId="11077" CreationDate="2019-03-08T10:56:08.900" Score="2" Body="&lt;p&gt;Instead of using a neural network, simply sample as many non-anomalous readings from each sensor as you can. If the distribution of the readings from each sensor is approximately normal (check the &lt;a href=&quot;https://www.itl.nist.gov/div898/handbook/eda/section3/eda35b.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;skew and kurtosis&lt;/a&gt; values for the samples from each sensor) then you can work out mean and standard deviation of the samples and, for any future samples, the value of a particular measurement on a normal probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If your measurements do not have a normal distribution, then you can often apply a transform of some sort to the data make it normal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, let's say that you have measured a few thousand typical samples for one of your sensors, confirmed that the distribution is normal and calculated the mean and standard deviation of those samples. You can now calculate where on the normal curve your new sample 'x' would be using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def gaussian(x, mean, std):&#xA;    # check for very small standard deviations&#xA;    if(2 * std ** 2 == 0):&#xA;        return 0.0&#xA;&#xA;    return (1.0/(math.sqrt(2*math.pi) * std)) * math.exp(-((x-mean)**2)/(2*std**2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At this point, you can make a decision if the value of the of a particular measurement on that distribution is so low as to be anomalous, and inform the user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you have probably realised that working out exactly &lt;em&gt;how&lt;/em&gt; low a value should be to be considered anomalous might be tricky, and you are correct. The solution is to see if you can get some &lt;em&gt;real&lt;/em&gt; anomalous data and use that to set the thresholds. Obviously, this might be rather a job if you have 4000 independent sensors...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more about Anomaly Detection then I recommend that you take a look at &lt;a href=&quot;https://www.youtube.com/playlist?list=PLwgXNx7TiGV6UH3aEzmdZwzFRwvEnRb0N&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's introductory lecture series.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="12509" LastEditDate="2019-03-12T16:20:50.353" LastActivityDate="2019-03-12T16:20:50.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11099" PostTypeId="1" CreationDate="2019-03-08T11:57:23.460" Score="2" ViewCount="32" Body="&lt;p&gt;I frequently need to translate product names for hundreds of similar products -- and I have a list of past product names. Is it possible to train AI to review past translations and translate? It doesn't have any special grammar, simply the name (with some industry-specific usage that a general machine translator can't do.) What would I need to do to get started?&lt;/p&gt;&#xA;" OwnerUserId="23000" LastEditorUserId="1671" LastEditDate="2019-11-06T21:35:12.763" LastActivityDate="2019-11-06T21:35:12.763" Title="Translate product names with AI" Tags="&lt;natural-language-processing&gt;&lt;machine-translation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11100" PostTypeId="1" CreationDate="2019-03-08T12:52:43.333" Score="5" ViewCount="101" Body="&lt;p&gt;I am working on a MLP neural networks, using supervised learning (2 classes and multi-class classification problems). For the hidden layers, I am using &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; (which produces an output in the range &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;) and for the output layer a softmax (which gives the probability distribution between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;). As I am working with supervised learning, should be my targets output between 0 and 1, or &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (because of the &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; function), or it does not matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function is quadratic (MSE).&lt;/p&gt;&#xA;" OwnerUserId="19268" LastEditorUserId="2444" LastEditDate="2019-03-08T13:55:40.093" LastActivityDate="2019-08-05T21:50:32.407" Title="What should the range of the output layer be when performing classification?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;activation-function&gt;&lt;supervised-learning&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11102" PostTypeId="2" ParentId="6699" CreationDate="2019-03-08T14:45:38.270" Score="1" Body="&lt;p&gt;The use of the Golden Ratio is an interesting suggestion which has intrigued many lovers of the mathematical beauty represented in nature and in AI. The problem lies in the foundations of the AI applications. For example, in designing algorithms for recognizing naturally occurring phenomena, such as face recognition or human body movements (See  &lt;a href=&quot;https://www.intechopen.com/books/machine-learning-and-biometrics/a-human-body-mathematical-model-biometric-using-golden-ratio-a-new-algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.intechopen.com/books/machine-learning-and-biometrics/a-human-body-mathematical-model-biometric-using-golden-ratio-a-new-algorithm&lt;/a&gt;) it is suitable. However, for non natural occurrences, the ratio is limited since the data is usually random or chaotic. However, in order to create a master algorithm for the future which encompasses all the best of the current AI algorithms, the use of mathematical concepts such as the golden ration and fractals will be vital. Watch this space...&lt;/p&gt;&#xA;" OwnerUserId="23002" LastActivityDate="2019-03-08T14:45:38.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11103" PostTypeId="1" CreationDate="2019-03-08T17:37:58.113" Score="1" ViewCount="136" Body="&lt;p&gt;Is there a way to run C64, Nintendo Gameboy, or other platforms in an OpenAI's Gym-like environment? It seems that we can run only Atari games.&lt;/p&gt;&#xA;" OwnerUserId="23009" LastEditorUserId="2444" LastEditDate="2020-09-22T16:47:26.920" LastActivityDate="2020-09-22T16:47:26.920" Title="Is there a way to run other platforms (other than Atari) in an OpenAI's Gym-like environment?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11104" PostTypeId="2" ParentId="11103" CreationDate="2019-03-08T18:03:45.767" Score="1" Body="&lt;p&gt;Ok, I found &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.openai.com/gym-retro/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.openai.com/gym-retro/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but there may be other platforms?&lt;/p&gt;&#xA;" OwnerUserId="23009" LastActivityDate="2019-03-08T18:03:45.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11105" PostTypeId="2" ParentId="10675" CreationDate="2019-03-08T18:45:41.243" Score="2" Body="&lt;p&gt;System Identification and policy learning are two completely different aspects of a system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;System Identification is basically finding out the transfer functions, the hardware parameters, the relationships and nature of behavior for different components that determine the results, when acted upon by a control signal. Generally, it is the hardware manufacturers who have all configuration details in their datasheets and they are either used as direct system parameters or used to derive other. Online system identification is the process of determining the set of parameters not with already available measurements but by using data coming through in real time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy learning is the process of correlating the actions to results and discerning what actions are good or bad. Policy learning is about determining the control strategy that shall produce desired results given all the circumstances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;SI is like curve fitting and determine the equation of the curve (already knowing the polynomial degree because you need to know the structure of the parameters you are trying to estimate) on already available data while policy learning is using a closed loop system to repeatedly update your control signals till you find one that satisfies your performance and operational desires. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In robotics context, a robot manipulator is supposed to have the Mass(H), Coriolis(C) and Gravity(G) matrix that define the dynamics of the system, basically relating the physics of the robot to the applied torque on the joints and the tip as shown in the equation below. Online parameter identification would mean using the torque, the known structure of the HCG matrices (H is a n x n matrix, n being the DOF and so on) dynamic equation and the then determine the numerical values. Similar online parameter identification is also done for friction components like the Static and Coulomb friction and the coefficients of Viscous friction. Least squares method is often used for the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{H(q)\ddot{q}}(t) + \mathbf{C(q, \dot{q})\dot{q}}(t) + \mathbf{B\dot{q} + g(q) = \tau}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy learning in terms fo RL is basically learning the set of actions that will produce desired good behavior. Q-learning is model free learning, so there is no predicted behavior to be obtained based on the inputs. Here, the inputs and simulated, results are obtained, they are given a degree of belief (positive &amp;amp; negative and high &amp;amp; low rewards) depending upon what part and how much of the desired result are they producing. Over time, the policy learned is finally the sequence of actions that should be run to get to desired result. The Q-table does not have anything to do with the system identification which is a modeling step, it is rather a control step. So, for the arm, the learned policy would be what joints in what sequence should be actuated to what angles to complete a pick and place task.&lt;/p&gt;&#xA;" OwnerUserId="22988" LastEditorUserId="22835" LastEditDate="2019-03-11T19:23:14.953" LastActivityDate="2019-03-11T19:23:14.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11106" PostTypeId="2" ParentId="11100" CreationDate="2019-03-08T20:05:36.407" Score="1" Body="&lt;p&gt;For this particular classification problem, I would recommend you using a softmax function whose output range is [0,1].&lt;br&gt;&#xA;The sum of all outputs should be 1, so an advantage of using a softmax function is that you get a percentage of how confident the network is in this classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side note: As DuttaA has commented, cross entropy loss is a better loss function than the quadratic mean squared error.&lt;/p&gt;&#xA;" OwnerUserId="23012" LastActivityDate="2019-03-08T20:05:36.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11107" PostTypeId="1" CreationDate="2019-03-08T20:08:39.423" Score="1" ViewCount="39" Body="&lt;p&gt;Here intelligence is defined as any analytic or decision making process, regardless of strength (utility), and, potentially, any process of computation that produces output, regardless of the medium. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea that AI is part of an evolutionary process, with humans as merely a vehicle of the next dominant species, has been a staple of Science Fiction for many decades.  It informs our most persistent mythologies related to AI.  (Recent examples include Terminator/Skynet, Westworld, Ex Machina, and Alien:Covenant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm driving at here is that, although the concept of Neural Networks have been around since the 1940's, they have only recently demonstrated strong utility, so it's not an unreasonable assumption to identify Moore's Law as the limiting factor.  (i.e. it is only recently that we have had sufficient processing power and memory to achieve this utility.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the idea of AI is ingrained into information technology once automation becomes possible.  Babbage's Difference Engine led to the idea of an Analytic Engine, and the game of Tic-Tac-Toe was proposed as a means of demonstrating intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm driving at here is that the idea of analysis and decision making are so fundamental in regard to information technology, that it is difficult to see functions that don't involve them.  And, if the strength of analysis and decision making is largely a function of computing power:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can intelligence be understood as a naturally occurring function of information technology?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-08T21:54:40.597" LastActivityDate="2019-03-08T21:54:40.597" Title="Is Intelligence a naturally occurring function of Information Technology?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;soft-question&gt;&lt;mythology-of-ai&gt;&lt;intelligence&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11108" PostTypeId="2" ParentId="11107" CreationDate="2019-03-08T20:39:52.027" Score="1" Body="&lt;p&gt;Maybe, but it depends to a very large degree on the choice of definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the biggest challenges for AI researchers, neuroscientists, philosophers, and psychologists, has been that the layperson's understanding of &lt;em&gt;intelligence&lt;/em&gt; does not appear to correspond to a well-defined concept. This point was most famously exploited by John R. Searle in his paper &lt;em&gt;Minds Brains and Programs&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider your definition again carefully, and notice that &lt;em&gt;decision&lt;/em&gt; is unbound. Does a rock rolling down a hill make a decision when it 'decides' to roll to the left? If not, Searle and his ilk would argue that a computer isn't making a decision when it &lt;em&gt;decides&lt;/em&gt; to execute the next instruction. It is simply obeying deterministic laws of physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To escape this problem and still claim that machines are intelligent, you can either chose to believe that rocks make decisions (a view called &lt;em&gt;panpsychism&lt;/em&gt;), or that we should &lt;em&gt;behave&lt;/em&gt; as though rocks make decisions because doing so gives us a lot of explanatory power (the intentional stance), or claim that humans are no different from the rock or the computer (eliminative materialism). The last option leaves you with the problem of explaining subjective experiences though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excellent book that covers the mainstream arguments in this vein is Haugeland's &lt;em&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0262581531&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mind Design II&lt;/a&gt;&lt;/em&gt;. All the arguments I've outlined here are covered in great detail within. Among modern AI researchers, I'd say there is a split between those who embrace something like the intentional stance and those who embrace eliminative materialism. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-08T20:39:52.027" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11109" PostTypeId="1" CreationDate="2019-03-08T20:41:23.087" Score="4" ViewCount="50" Body="&lt;p&gt;As far as I know, a &lt;em&gt;problem representation&lt;/em&gt; is the formulation of the problem in a way that it can be programmed and therefore solved (for example, you can represent the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-queens problem by using an array of &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &lt;em&gt;problem modelling&lt;/em&gt; mean? What is the difference between a problem representation and problem modelling?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastEditorUserId="2444" LastEditDate="2019-11-12T17:53:36.780" LastActivityDate="2019-11-12T17:53:36.780" Title="What is the difference between a problem representation and problem modelling?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11111" PostTypeId="2" ParentId="11109" CreationDate="2019-03-08T20:45:26.863" Score="4" Body="&lt;p&gt;I would say these terms are often used interchangeably in AI. When they differ, I would say that problem modeling means finding a mathematical description of the problem, while problem representation means finding a particular way to represent that mathematical formalism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a list of numbers can be stored (represented) with a linked list, and array list, a hash table, or a self-balancing tree. All of them can produce a faithful model of the list, but if what you want to do is find the order that elements were entered in, the array list or linked list is far faster and more natural. If what you want to do is determine whether certain pieces of information are present in the list, the hash table is fastest. If what you want to do is find ranges of similar elements, the tree is fastest. Essentially, representational choices are engineering problems, while modelling choices are scientific problems.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-08T20:45:26.863" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11112" PostTypeId="1" AcceptedAnswerId="11113" CreationDate="2019-03-08T21:21:16.363" Score="2" ViewCount="76" Body="&lt;p&gt;I was reading in the article &lt;em&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0022249609000042&quot; rel=&quot;nofollow noreferrer&quot;&gt;A tutorial on partially observable Markov decision processes&lt;/a&gt;&lt;/em&gt; (p. 120), by Michael L. Littman, that &lt;span class=&quot;math-container&quot;&gt;$\sum_{z \in Z}O(a, s',z) =1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the action, &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; the next possible state and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; a certain/specific observation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How come that the observation function &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; adds up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in POMDP?&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="2444" LastEditDate="2019-03-28T17:40:31.440" LastActivityDate="2019-03-28T17:40:31.440" Title="Does the observation function for POMDP always add up to 1?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11113" PostTypeId="2" ParentId="11112" CreationDate="2019-03-08T21:39:28.627" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O(a, s', z) = \mathbb{P}(z \mid a, s')$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_probability_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional probability distribution&lt;/a&gt;, so it always needs to sum up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. You should interpret &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; as the probability of observation &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, given that the agent took action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and landed in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; is thus &lt;strong&gt;not&lt;/strong&gt; a &lt;a href=&quot;https://en.wikipedia.org/wiki/Joint_probability_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;joint distribution&lt;/a&gt;, even though the notation &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; might suggest it. In this case, &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; simply means that &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; is a function of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to see a proof that conditional probability distributions sum up to 1, have a look at &lt;a href=&quot;https://math.stackexchange.com/q/2084313/168764&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-08T21:51:46.307" LastActivityDate="2019-03-08T21:51:46.307" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11116" PostTypeId="1" AcceptedAnswerId="11804" CreationDate="2019-03-09T03:05:14.073" Score="3" ViewCount="42" Body="&lt;p&gt;In &lt;a href=&quot;https://www.tensorflow.org/tutorials/sequences/audio_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial, they build a speech recognition model to classify a one-second audio clip as one of ten predefined words. Suppose that we modified this problem as the following: Given an Arabic dataset, we aim to build a &lt;em&gt;dialects&lt;/em&gt; recognition model to classify a &lt;em&gt;two&lt;/em&gt;-second audio clip as one of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; local dialects using ten predefined &lt;em&gt;sentences&lt;/em&gt;. I.e. for each of these ten sentences, there are &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; different phrases and idioms which refer to the same meaning&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt;. Now how can I take advantage of the mentioned tutorial to solve the modified problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$*$&lt;/span&gt; The &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; different phrases and idioms for each sentence are not predefined.&lt;/p&gt;&#xA;" OwnerUserId="20449" LastActivityDate="2019-04-13T20:58:33.217" Title="How much the dialects recognition and speech recognition are relevant?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;voice-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11117" PostTypeId="1" CreationDate="2019-03-09T05:25:36.703" Score="4" ViewCount="109" Body="&lt;p&gt;What are the characteristics which make a function difficult for the Neural Network to approximate? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, one might think uneven functions might be difficult to approximate, but uneven functions just contain some high frequency terms (which in case of sigmoid is easy to approximate &lt;span class=&quot;math-container&quot;&gt;$\frac{1} {(1 + e ^ {-(w*x + b)})}$&lt;/span&gt;, by increasing the value of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;). So uneven data might not be diffcult to approximate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is what makes a function truly difficult for approximation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: By approximation I do not mean things which can be changed by changing the  training method (changing training set size, methods, optimisers). By approximation I mean things which require hyperparameters (size, structure, etc) of a NN to be changed to approximate to a certain level significantly easily.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-05-10T14:41:28.817" LastActivityDate="2019-05-10T14:41:28.817" Title="What characteristics make it difficult for a Neural Network to approximate a function?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11118" PostTypeId="1" CreationDate="2019-03-09T05:57:32.520" Score="1" ViewCount="40" Body="&lt;p&gt;Can ANN with only one neuron in output layer be trained in a way that output neuron’s value (0-1) can be representation of some real value, like for example height.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words,can neural network given the inputs predict the height of a person by outputing values from 0 to 1. Zero being the 50 cm and 1 being 250cm.Or it will always gravitate to 0 or 1?Can it predict a height of 150cm (0.5) ?&lt;/p&gt;&#xA;" OwnerUserId="23022" LastActivityDate="2019-03-11T11:16:28.103" Title="ANN’s single output representing value" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11119" PostTypeId="1" CreationDate="2019-03-09T06:42:21.613" Score="2" ViewCount="219" Body="&lt;p&gt;I'm writing an AI for a board game, and previously I would just create a value maximizing state machine and tune the factors one at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the issues with this is getting apparent. My last AI did not look into the future, hurting it's long term chances, and manual tuning of weights is proving to be a chore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked into minimax algorithms, but with a non perfect information game and elements of random chance I'm not too confident it will be effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also looked into traditional neural networks, but evaluating board states is tricky and the game does not split into moves well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The game I'm writing the AI for is Ticket To Ride, but I would appreciate tips for any board game with similar mechanics.&lt;/p&gt;&#xA;" OwnerUserId="23024" LastActivityDate="2019-03-09T11:52:22.073" Title="Game AI design for a multiplayer random board game?" Tags="&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11120" PostTypeId="2" ParentId="11118" CreationDate="2019-03-09T08:18:43.917" Score="1" Body="&lt;p&gt;Yes, this is possible. The only restriction on the range of output values is due to the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;activation function&lt;/a&gt;, if there is one. If, for example, the activation function is a &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;sigmoid&lt;/a&gt;, the output would be restricted to values between 0 and 1. You can of course choose an activation function best suited to your problem, even if that means having none. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, if you really want the output of your network to be between 50 and 250, you could use a scaled and shifted sigmoid as your activation: 200*sig(w*x+b)+50. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, as was pointed out in comments below, having a single output neuron may not be the best architecture for this problem.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-11T11:16:28.103" LastActivityDate="2019-03-11T11:16:28.103" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11122" PostTypeId="2" ParentId="11084" CreationDate="2019-03-09T09:01:46.283" Score="1" Body="&lt;p&gt;I've played this game. If I remember correctly, a successful strategy (for winning as many tricks you estimated to win) involves continually evaluating how well you're doing. If you're doing very poorly relative to your initial estimation, that's valuable information to have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you should have one network that outputs--at every time step--both an estimate of future tricks won and an action of which card to play. This has the additional advantage of giving you more estimation experience with which to train. After all rounds, you'll know how far off each of your estimations were throughout the game. This should speed up learning on that part of the game.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-09T09:01:46.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11123" PostTypeId="2" ParentId="11119" CreationDate="2019-03-09T11:23:41.480" Score="1" Body="&lt;p&gt;Many successful game-playing engines use some form of search to look ahead and plan the next move. This is possible even in stochastic environments by building a tree with probabilities assigned, or by simulating the randomness during planning and relying on lots of samples to get reasonable estimates of value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many approaches possible. However, as Ticket to Ride is a relatively simple game that you could automate to play to the end very quickly (1000s of times per second), then you could look into &lt;a href=&quot;https://int8.io/monte-carlo-tree-search-beginners-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search&lt;/a&gt; (MCTS) as a mechanism for the game-playing agents to look ahead and adjust their play to current odds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basics of Monte Carlo Tree Search:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On its turn, the agent builds up an evaluation tree local to its current state - each node of the tree is a game state, augmented with statistics from MCTS (how often node was chosen, how many times agent managed to win eventually from that node) and each link is a choice that a player makes. This is built up slowly, typically just adding one node per full simulated game.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For look-ahead search &quot;inside&quot; the tree, the agent explores options according to their relative promise, selecting moves to simulate according to its best estimates of value at the time. It needs to explore alternatives to make sure it doesn't miss anything, and there are a few variations of doing this. One successful approach to action selection inside the tree is Upper Confidence Bound applied to Trees (or UCT for short).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When play gets to a leaf node of the tree, the tree might be expanded slightly, and then afterwards the agent simulates plays of the game either to the end or to a robust evaluation point. This later play might be entirely random - or might still be guided probabilistically by some heuristic - and is called a rollout. The important thing here is to play quickly to get samples just to roughly assess the position at the leaf of the tree.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The evaluation resulting from rollout is fed back into the tree, which builds up sampled statistics of the most promising variations of play.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After a certain number of repeats of the whole algorithm (local tree search, expanding the tree, rollout and update), the agent picks the most promising next step from the tree. At this point it may just discard the whole tree - in a game with lots of randomness, this is probably what you would do now, as things you were considering as just probabilities before will have become historical fact, and the chances are that re-running the tree build from the next start position will be more accurate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are a few variants possible, including combinations with neural networks as used in AlphaZero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic principle is to sample from many choices, focusing over time on what looks &quot;best&quot; to each player based on statistics of those choices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Ticket To Ride is a card game where cards can be drawn from an unseen deck, and it would be considered cheating to simulate and rollout with the actual deck in play, you would need to have realistic shuffles of the remaining unknown cards used during rollout. It may still work quite well without the &lt;em&gt;realistic&lt;/em&gt; part and for speed just assume a random unlimited deck of cards (because re-shuffling on each imagined rollout would be costly).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-09T11:52:22.073" LastActivityDate="2019-03-09T11:52:22.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11124" PostTypeId="1" CreationDate="2019-03-09T13:22:19.807" Score="1" ViewCount="25" Body="&lt;p&gt;There are many, many literary works in the public domain, along with human translations, many of which have entered the public domain as well. (Public domain = easily available)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for me to advance my knowledge of e.g. Japanese, I'd like to read texts in Japanese using a yet to be written tool, and when I encounter an unknown word/phrase/sentence, I'd like to just click on a position in the text and be transferred to the corresponding position in the e.g. English translation (or original) of the text in question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's also assume we have access to a dictionary that translates a fair amount of words between those two languages (and there are of course free dictionaries for many language pairs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What ways are there to use AI toolkits plus some wiring and perhaps scripting/programming, to auto-correlate the positions of two versions of the same text, in two languages?&lt;/strong&gt; The results do not - and in fact in many cases cannot - be perfect, but they should be roughly correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware that this is still not a straightforwards task, as there are complicating factors like inflection of verbs and other grammatical properties that make the use of dictionary tools much harder. Also, translators will often translate to words that don't have that mapping in any dictionary. Then there is the fact that words aren't delimited by spaces in languages like my example language Japanese - (but if it is easier to work with only space-separated languages like, say, Spanish, or Russian, I'd like to hear answers to this simpler problem as well). Also the order of words and even hole sub-clauses differs from language to language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple, non-AI approximation would be to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;figure out at what relative position in the source language text the user clicked (e.g. at the character on position 50.234%)&lt;/li&gt;&#xA;&lt;li&gt;then go to that same relative position 50.234% in the target language text&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This approximation could perhaps be used as the starting point for the AI, which would then use words and dictionaries to make the results more accurate.&lt;/p&gt;&#xA;" OwnerUserId="23027" LastActivityDate="2019-03-10T16:08:33.820" Title="Use AI to auto-correlate the words of human-translated texts?" Tags="&lt;algorithm&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11127" PostTypeId="1" CreationDate="2019-03-09T16:30:14.757" Score="1" ViewCount="56" Body="&lt;p&gt;I would love to know if an AI model could come up with certain theories of the old like Pythagoras' theorem, Euclid's formulations,&#xA;Newton's gravity, Einstein's theories if provided and trained with sufficient amount of observable data available at those period of time. If this is possible can unsolved conjectures be proved by AI? Or even better can AI develop new theories or will it  fail to come up with even basic mathematical operations by itself?&lt;/p&gt;&#xA;" OwnerUserId="23034" LastActivityDate="2019-03-09T17:47:01.493" Title="Can AI come up with scientific theories of past when provided with sufficient data available at that time?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;automated-theorem-proving&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11128" PostTypeId="1" CreationDate="2019-03-09T16:38:17.100" Score="3" ViewCount="34" Body="&lt;p&gt;Can AI transform natural language text describing real scenarios to visual images and videos ? How does as AI interprets say a Harry Potter story if it has to reproduce it in form of videos ? Would be useful if anyone can help me with the required literature for understanding text to image transformation by AI &lt;/p&gt;&#xA;" OwnerUserId="23034" LastActivityDate="2019-03-09T16:38:17.100" Title="How would an AI visualize a story written in natural language?" Tags="&lt;natural-language-processing&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11129" PostTypeId="1" CreationDate="2019-03-09T17:29:17.463" Score="3" ViewCount="22" Body="&lt;p&gt;I have a NN I'd like to train using supervised learning. Some samples of the training set, however, have better &quot;quality&quot; than others, so I'd like the algorithm to pay &quot;special attention&quot; to them. As general question, how to take this into account in implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being more specific, I'm working with OpenCV and noticed that the train method apparently have such parameter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cv2.ANN_MLP.train(inputs, outputs, sampleWeights[, sampleIdx[, params[, flags]]]) → retval&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;sampleWeights&lt;/strong&gt; – (RPROP only) Optional floating-point vector of weights for each sample. Some samples may be more important than&#xA;  others for training. You may want to raise the weight of certain&#xA;  classes to find the right balance between hit-rate and false-alarm&#xA;  rate, and so on.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However OpenCV documentation is unclear on this, so how to handle this parameter?&lt;/p&gt;&#xA;" OwnerUserId="13087" LastActivityDate="2019-03-09T17:29:17.463" Title="Backpropagation: how to take into account different samples quality" Tags="&lt;machine-learning&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11130" PostTypeId="2" ParentId="11127" CreationDate="2019-03-09T17:47:01.493" Score="1" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some good examples of this are &lt;a href=&quot;https://www.wired.com/2009/04/newtonai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lipson's&lt;/a&gt; work using evolutionary models, and Wu &amp;amp; Tegmark's work on a &lt;a href=&quot;https://arxiv.org/pdf/1810.10525.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;theory-based life-long learner&lt;/a&gt;, and &lt;a href=&quot;https://arxiv.org/pdf/1807.10300.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iten et al.'s&lt;/a&gt; work with deep neural networks. There are many, many, other research papers in this area, and there is a lot more work that is ongoing. The endgame for a lot of this work is the hope that we can synthesize new theories automatically.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-09T17:47:01.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11131" PostTypeId="1" CreationDate="2019-03-09T18:54:43.750" Score="2" ViewCount="307" Body="&lt;p&gt;I'm trying create neural network to predict moves in a card game.  I am looking for recommendations on encoding the game state to my input layer.  It's a complex turn based collectible card game (think Magic the Gathering).  I need to represent cards being in various areas of the game board (deck, discard pile, hand, etc).  It seems difficult to assign cards to these areas because the number of cards in these areas is never constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of an approach where I assign each card in the game to being in a specific card area.  The number of total cards in the game should be constant (let's assume that).  This approach I feel should give me a potentially less sparse input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, With this approach what is the best way to handle card duplicates?  Let's say I have 3 copies of the exact same card in my deck.  Maybe 1 of the copies is in my hand and 2 in my discard pile.  It does not matter which of the 3 copies is in my hand because they are all the same exact card.  There is now multiple ways to represent this same exact game state in my network because each individual card has it's own state.  To me this does not seem good.  How much will this effect my neural network's ability to learn the game?&lt;/p&gt;&#xA;" OwnerUserId="23037" LastActivityDate="2021-04-01T08:25:24.943" Title="How to encode card game state into neural network input" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11132" PostTypeId="2" ParentId="11131" CreationDate="2019-03-10T01:24:36.230" Score="1" Body="&lt;p&gt;I am not sure what you mean by&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;There is now multiple ways to represent this same exact game state in my network because each individual card has it's own state. To me this does not seem good.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We can think of game state as a column vector of the form &lt;span class=&quot;math-container&quot;&gt;$1Xn$&lt;/span&gt;. In this case, you can formalize cards held according to some encoding. This can be anything from a Gödel Numbering to simple integer assignment. Here is an example for poker in python: &lt;a href=&quot;https://pypi.org/project/treys/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pypi.org/project/treys/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the example of duplicates you would have a column vector that simply has duplicate indices. I.e: {241, 424, &lt;strong&gt;112, 112&lt;/strong&gt;, 455}$&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="43351" LastEditDate="2021-04-01T08:25:24.943" LastActivityDate="2021-04-01T08:25:24.943" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11133" PostTypeId="2" ParentId="11057" CreationDate="2019-03-10T04:57:41.727" Score="22" Body="&lt;p&gt;The notation I'll be using is from &lt;a href=&quot;https://web.archive.org/web/20171118120953/http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/MDP.pdf&quot; rel=&quot;noreferrer&quot;&gt;two&lt;/a&gt; &lt;a href=&quot;https://web.archive.org/web/20181127083007/http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/DP.pdf&quot; rel=&quot;noreferrer&quot;&gt;different&lt;/a&gt; lectures by David Silver and is also informed by &lt;a href=&quot;http://chercheurs.lille.inria.fr/%7Eghavamza/RL-EC-Lille/Lecture3.pdf&quot; rel=&quot;noreferrer&quot;&gt;these slides&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The expected Bellman equation is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \sum_{a\in \cal{A}} \pi(a|s) \left(\cal{R}_s^a + \gamma\sum_{s' \in \cal{S}} \cal{P}_{ss'}^a v_\pi(s')\right) \tag 1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If we let&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\cal{P}_{ss'}^\pi = \sum\limits_{a \in \cal{A}} \pi(a|s)\cal{P}_{ss'}^a \tag 2$$&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\cal{R}_{s}^\pi = \sum\limits_{a \in \cal{A}} \pi(a|s)\cal{R}_{s}^a \tag 3$$&lt;/span&gt;&#xA;then we can rewrite &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \cal{R}_s^\pi + \gamma\sum_{s' \in \cal{S}} \cal{P}_{ss'}^\pi v_\pi(s') \tag 4$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can be written in matrix form&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left.&#xA;\begin{bmatrix}&#xA;v_\pi(1) \\&#xA;\vdots \\&#xA;v_\pi(n)&#xA;\end{bmatrix}=&#xA;\begin{bmatrix}&#xA;\cal{R}_1^\pi \\&#xA;\vdots \\&#xA;\cal{R}_n^\pi&#xA;\end{bmatrix}&#xA;+\gamma&#xA;\begin{bmatrix}&#xA;\cal{P}_{11}^\pi &amp;amp; \dots &amp;amp; \cal{P}_{1n}^\pi\\&#xA;\vdots &amp;amp; \ddots &amp;amp; \vdots\\&#xA;\cal{P}_{n1}^\pi &amp;amp; \dots &amp;amp; \cal{P}_{nn}^\pi&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;v_\pi(1) \\&#xA;\vdots \\&#xA;v_\pi(n)&#xA;\end{bmatrix}&#xA;\right. \tag 5$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or, more compactly,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi = \cal{R}^\pi + \gamma \cal{P}^\pi v_\pi \tag 6$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notice that both sides of &lt;span class=&quot;math-container&quot;&gt;$(6)$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional vectors. Here &lt;span class=&quot;math-container&quot;&gt;$n=|\cal{S}|$&lt;/span&gt; is the size of the state space. We can then define an operator &lt;span class=&quot;math-container&quot;&gt;$\cal{T}^\pi:\mathbb{R}^n\to\mathbb{R}^n$&lt;/span&gt; as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\cal{T^\pi}(v) = \cal{R}^\pi + \gamma \cal{P}^\pi v \tag 7$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for any &lt;span class=&quot;math-container&quot;&gt;$v\in \mathbb{R}^n$&lt;/span&gt;. This is the expected Bellman operator.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, you can rewrite the Bellman optimality equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_*(s) = \max_{a\in\cal{A}} \left(\cal{R}_s^a + \gamma\sum_{s' \in \cal{S}} \cal{P}_{ss'}^a v_*(s')\right) \tag 8$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;as the Bellman optimality operator&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\cal{T^*}(v) = \max_{a\in\cal{A}} \left(\cal{R}^a + \gamma \cal{P}^a v\right) \tag 9$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Bellman operators are &amp;quot;operators&amp;quot; in that they are mappings from one point to another within the vector space of state values, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Rewriting the Bellman equations as operators is useful for proving that certain dynamic programming algorithms (e.g. policy iteration, value iteration) converge to a unique fixed point. This usefulness comes in the form of a body of existing work in operator theory, which allows us to make use of special properties of the Bellman operators.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, the fact that the Bellman operators are &lt;a href=&quot;https://en.wikipedia.org/wiki/Contraction_%28operator_theory%29&quot; rel=&quot;noreferrer&quot;&gt;contractions&lt;/a&gt; gives the useful results that, for any policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and any initial vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{k\to\infty}(\cal{T}^\pi)^k v = v_\pi \tag{10}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{k\to\infty}(\cal{T}^*)^k v = v_* \tag{11}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt; is the value of policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_*$&lt;/span&gt; is the value of an optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;. The proof is due to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Banach_fixed-point_theorem&quot; rel=&quot;noreferrer&quot;&gt;contraction mapping theorem&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2020-08-12T17:40:35.970" LastActivityDate="2020-08-12T17:40:35.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11134" PostTypeId="1" AcceptedAnswerId="11138" CreationDate="2019-03-10T06:18:15.020" Score="2" ViewCount="95" Body="&lt;p&gt;I notice with the recent revelation of severe limitations in some AI domains such as self driving cars that NNets behave with the same sort of errors as in simpler models. Ie: They may be ~100% accuracte on test data however if you throw in a test sample that is slightly different to anything it's been trained on, it can throw the net off completely. This seems to be the case with self driving cars where NNets are miss-classifying modified/grafitied Stop Signs, unable to cope with rain or snow flakes, or birds appearing on the road, etc. Something it's never seen before in a unique climate may cause it to make completely unpredictable predictions. These specific examples may be circumvented by training on modified Stop Signs, or with rain and birds, but that avoids the point: that NN's seem very limited when it comes to generalizing to an environment that is completely unique to its training samples. And this makes sense of course given the way NNs train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current solution seems to be to manually find out these new things that confuse the netowrk and label them as additional training data. But that isn't an AI at all. That isn't true generalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think part of the problem to blame is the term &quot;AI&quot; in and of itself. When all we're doing is finding a global minima to a theoretical ideal function at some perfect point before over-fitting our training data, it's obvious that the NNet cannot generalize anymore than what is possible within its training set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought one way that might be possible to get around this is if rather than being static &quot;one unique calculation at a time&quot; if instead NNets could remember the last one or two predictions they made, and then their current prediction, and use the result of that to then make a more accurate prediction. Ie: A very basic form of short term memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By doing this perhaps the NN could see that the rain drops or snow flakes aren't static objects, but are simply moving noise. It could determine this by looking at its last couple of predictions and see how those objects move. Certainly this would require immense additional computation overhead, but I'm just looking to the future in terms of when processing power increases how NNs might evolve further. Similar to how NNs were already defined many decades ago but were never a &quot;thing&quot; due to the lack of computational power, could this be the same case with something like short term memory? That we lack the practical computational power for it but that perhaps we could theoretically implement it somehow for sometime in the future when we do have it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this short term memory thing would only be useful for when a classification is related to the prior classifications, like with self driving cars. It's important to know what was observed a few seconds ago in real life when driving, likewise it could be important for an NN to know what was predicted a few live classifications ago. This might also have use in object detection, ie: perhaps a representation could be learned for a moving figure in the distance. Speed could now become a representation in the hidden layers and used in assistance for identification of distant objects, something not possible when using a set of single static weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this whole thing would involve somehow getting around the problem of training weights on a live model for the most recent sample. Or alternatively perhaps the weights could still remain static but we'd use two or three different models of weights to represent time somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless I can't help but see short term memory of some form as being a requirement if AI is to not be &quot;so stupid&quot; when it observes something unique, and if it's to ever classify things based on time and recent observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious if there's any research papers or other sources that explore any aspect of incorporating time using multiple recent classifications or something else that could be considered a short term memory of sorts to help reach a more general form of generalization that the NNet doesn't necessarily see in its training, ie: making it able to avoid noise using time as a feature to help it do so, or using time from multiple recent classifications as a way to estimate speed and use that as a feature? I'd appreciate the answer to include some specific experiment or methodology as to how this sort of thing might be added to NNets (or list it as a source), or if this is not an area of active research, why not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="6328" LastEditorUserId="6328" LastEditDate="2019-03-10T10:11:23.493" LastActivityDate="2019-03-10T17:17:21.697" Title="Neural nets need a short term memory of some sort" Tags="&lt;theory&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11135" PostTypeId="1" CreationDate="2019-03-10T10:28:40.553" Score="1" ViewCount="82" Body="&lt;p&gt;I am working lately on batch normalization for the last couple of days, but I just can't seem to solve it on how to apply it. So, to give the full code to you, I will just send the implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation DDPG Code: &lt;a href=&quot;https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow/blob/master/contents/9_Deep_Deterministic_Policy_Gradient_DDPG/DDPG_update2.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow/blob/master/contents/9_Deep_Deterministic_Policy_Gradient_DDPG/DDPG_update2.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and fun thing is that this github creator actually also has a batch normalization implementation: &lt;a href=&quot;https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/502_batch_normalization.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/MorvanZhou/Tensorflow-Tutorial/blob/master/tutorial-contents/502_batch_normalization.py&lt;/a&gt;. However, I don't get how he does it, since his videos are in Chinese. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a lot of research so far on how to implement it, however I usually get ValueErrors for &quot;NoneType&quot; or &quot;False_Fn has to return a value&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The name differences between everything makes me even more confused on where what should be. Also, to avoid confusion, I am trying to add this version of batch normalisation: &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any idea, suggestion on how to implement this, more than welcome.&lt;/p&gt;&#xA;" OwnerUserId="23050" LastEditorUserId="2444" LastEditDate="2019-03-10T10:47:03.037" LastActivityDate="2019-03-10T10:47:03.037" Title="How do I apply batch normalization to my algorithm?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11136" PostTypeId="2" ParentId="4680" CreationDate="2019-03-10T14:38:17.503" Score="3" Body="&lt;p&gt;I would start with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kendall_PoseNet_A_Convolutional_ICCV_2015_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2017/papers/Clark_VidLoc_A_Deep_CVPR_2017_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_ICCV_2017/papers/Walch_Image-Based_Localization_Using_ICCV_2017_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image-based localization using LSTMs for structured feature correlation&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w17/Melekhov_Image-Based_Localization_Using_ICCV_2017_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image-based Localization using Hourglass Networks&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1901.05376.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;UAN: Unified Attention Network for Convolutional Neural Networks&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1805.08443.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scene Coordinate and Correspondence Learning for Image-Based&#xA;Localization&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can also search for articles using the 7-scenes dataset (&lt;a href=&quot;https://ieeexplore.ieee.org/document/6619221&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images&lt;/a&gt;) which is pretty standard &lt;/p&gt;&#xA;" OwnerUserId="23049" LastEditorUserId="1671" LastEditDate="2019-03-11T20:49:33.250" LastActivityDate="2019-03-11T20:49:33.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11137" PostTypeId="2" ParentId="11124" CreationDate="2019-03-10T16:08:33.820" Score="1" Body="&lt;p&gt;You are basically describing the way &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Translate#Method_of_translation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Translate&lt;/a&gt; works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been a lot of research in text alignment in the area of multi-lingual corpus linguistics. An early paper (with sourcode) is Gale and Church's &lt;em&gt;A Program for Aligning Sentences in Bilingual Corpora&lt;/em&gt; (&lt;a href=&quot;https://s3.amazonaws.com/tm-town-nlp-resources/Gale-Church+alignment.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In linguistics these are called &lt;a href=&quot;https://en.wikipedia.org/wiki/Parallel_text&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallel texts&lt;/a&gt;. On the wikipedia page you will find links to a number of alignment programs which attempt to solve that issue.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-03-10T16:08:33.820" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11138" PostTypeId="2" ParentId="11134" CreationDate="2019-03-10T17:17:21.697" Score="3" Body="&lt;p&gt;What you're describing is called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrent neural network&lt;/a&gt;. There are a large number of designs in this family that all have the ability to remember recent inputs and use them in the processing of future inputs. The &quot;Long Short Term Memory&quot; or &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt; architecture was one of the most successful in this family.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are actually very widely used in things like self-driving cars too, so, on their own, they are not enough to overcome the brittleness of current models.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-10T17:17:21.697" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11139" PostTypeId="1" AcceptedAnswerId="16241" CreationDate="2019-03-10T17:29:15.723" Score="2" ViewCount="1395" Body="&lt;p&gt;I'm looking for neural network architecture that excel in counting objects. For example, CNN that can output the number of balls (or any other object) in a given image.&#xA;I already found articles about crowd counting, I'm looking for articles about different types of objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank&lt;/p&gt;&#xA;" OwnerUserId="23049" LastActivityDate="2019-11-03T06:57:41.307" Title="Count number of objects in image using CNN" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11140" PostTypeId="2" ParentId="11139" CreationDate="2019-03-10T18:30:42.397" Score="1" Body="&lt;p&gt;You can run YOLO, then count the number of occurrences.&lt;/p&gt;&#xA;" OwnerUserId="4042" LastActivityDate="2019-03-10T18:30:42.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11141" PostTypeId="1" CreationDate="2019-03-10T20:54:06.810" Score="4" ViewCount="69" Body="&lt;p&gt;I want to model an SMDP such that time is discretized and the transition time between the two states follows an exponential distribution and there would be no reward between the transition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I know what are the differences between &lt;span class=&quot;math-container&quot;&gt;$Q(\lambda)$&lt;/span&gt; and Q-learning for this problem (SMDP)? I actually want to extend the pseudo-code presented &lt;a href=&quot;https://stackoverflow.com/q/40862578/3924118&quot;&gt;here&lt;/a&gt; to an SMDP problem with discretization of time horizon.&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2019-03-10T21:24:10.793" LastActivityDate="2019-03-11T17:43:24.233" Title="How to apply or extend the $Q(\lambda)$ algorithm to semi-MDPs?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;semi-mdp&gt;&lt;eligibility-traces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11142" PostTypeId="1" CreationDate="2019-03-10T22:05:40.670" Score="0" ViewCount="215" Body="&lt;p&gt;When training a AI RL agent to play a game there'll be situations where the AI cannot perform certain actions lest they violate the game rules. That's easy to handle, and I can set illegal actions to some large negative amount so when doing an argmax they won't be selected.  Or if I use softmax I can set probabilities of illegal actions to zero and then re-calculate softmax on the remaining legal states. Indeed, I believe this is what David Silver was referring to when asked this question at a presentation/lecture of AlphaZero:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=Wujy7OzvdJk&amp;amp;t=2404s&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=Wujy7OzvdJk&amp;amp;t=2404s&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But doing so changes the output from the network and surely changes things when performing the backprop once a reward is known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one handle that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would I set the illegal actions to the mean of the legal actions, or zero...?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="20352" LastEditDate="2019-03-11T12:42:28.323" LastActivityDate="2019-03-11T12:42:28.323" Title="How to back-propagate illegal actions for policy gradient learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="11143" PostTypeId="1" CreationDate="2019-03-11T01:14:01.363" Score="1" ViewCount="94" Body="&lt;p&gt;I am currently trying to create a One-Shot network using the Siamese architecture for an object that isn't a face. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is, in normal Face Recognition the detecting gadget (e.g. Smartphone) knows which image it should compare the face, currently trying to unlock the tool, to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, I don't know the object's identity so I would have to compare every other object in the database to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better, more efficient way of checking the identity of the object without comparing it to any other object. A normal classification doesn't work in my case, because new unknown objects can be added any time.&lt;/p&gt;&#xA;" OwnerUserId="23063" LastActivityDate="2019-03-11T01:14:01.363" Title="Siamese Network for unknown object" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11144" PostTypeId="1" CreationDate="2019-03-11T07:31:54.857" Score="1" ViewCount="42" Body="&lt;p&gt;I need to extract personal information about a person from a list of documents and summarize it to the user. If there are 2 people with the same name, the correct person should be identified. If the person has a nickname, that also needs to be identified. The input to the program can be the name of the person, address, organization name etc.&#xA;I have extracted named entities like person, org, location etc from the text using NLTK library. &#xA;The output after extracting the named entities is mentioned below,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[('Michael', 'NNP', 'B-PERSON'), ('Joseph', 'NNP', 'B-PERSON'), ('Jackson', 'NNP', 'I-PERSON'), ('was', 'VBD', 'O'), ('born', 'VBN', 'O'), ('in', 'IN', 'O'), ('Gary', 'NNP', 'B-GPE'), (',', ',', 'O'), ('Indiana', 'NNP', 'B-GPE')....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to extract relationships between those entities.&lt;/p&gt;&#xA;" OwnerUserId="23069" LastEditorUserId="23069" LastEditDate="2019-03-11T08:44:26.103" LastActivityDate="2019-03-11T08:44:26.103" Title="Extract personal information about a person from a list of documents and summarize it" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11145" PostTypeId="2" ParentId="3189" CreationDate="2019-03-11T08:16:41.883" Score="1" Body="&lt;p&gt;Machines will never be conscious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's try this theoretical thought exercise. You memorize a whole bunch of shapes. Then, you memorize the order the shapes are supposed to go in, so that if you see a bunch of shapes in a certain order, you would &quot;answer&quot; by picking a bunch of shapes in another proper order. &lt;strong&gt;Now, did you just learn any meaning behind any language? Programs manipulate symbols this way.&lt;/strong&gt; (previously, people have either skirted this question or never had a satisfactory answer)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above was my reformulation of Searle's rejoinder to System Reply to his Chinese Room Argument.&lt;/p&gt;&#xA;" OwnerUserId="23071" LastActivityDate="2019-03-11T08:16:41.883" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11146" PostTypeId="2" ParentId="7926" CreationDate="2019-03-11T08:27:55.390" Score="0" Body="&lt;p&gt;No, because there is no utility in building a &quot;libertarian free AI&quot; as far as I know of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is another tool. What is the purpose in building an AI with such a distinction? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for that question is this. Let's say you want an AI to accomplish some kind of task you want machine assistance in. That's what tools do- assisting with tasks. &lt;em&gt;What exactly would this task be that a &quot;non-libertarian free AI&quot; couldn't achieve but a &quot;libertarian free AI&quot; could?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23071" LastActivityDate="2019-03-11T08:27:55.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11147" PostTypeId="2" ParentId="11050" CreationDate="2019-03-11T10:04:56.840" Score="2" Body="&lt;p&gt;There are many techniques for training an RL agent without explicitly interacting with an environment, some of which are cited in the paper you linked. Heck, even using experience replay like in the foundational &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN paper&lt;/a&gt; is a way of doing this. However, while many models utilize some sort of pre-training for the sake of safety or speed, there are a couple of reasons why an environment is &lt;em&gt;also&lt;/em&gt; used whenever possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually, your RL agent will be placed in an environment to take its own actions. This is why we train RL agents. I'm assuming that, per your question, learning does not happen during this phase. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Maybe your agent encounters a novel situation&lt;/strong&gt;&lt;br&gt;&#xA;Hopefully, the experience your agent learns from is extensive enough to include every possible state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; that your agent will ever encounter. If it isn't, your agent won't have learned about these situations, and it will always perform suboptimally in them. This lack of coverage over the state-action space could be caused by stochasticity or nonstationarity in the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Maybe the teacher isn't perfect&lt;/strong&gt;&lt;br&gt;&#xA;If you don't allow your agent to learn from its own experience, it will only ever perform as well as the agent that collected the demonstration data. That's an upper bound on performance that we have no reason to set for ourselves.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-11T10:04:56.840" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11150" PostTypeId="1" AcceptedAnswerId="11157" CreationDate="2019-03-11T11:07:21.470" Score="1" ViewCount="105" Body="&lt;p&gt;I am dealing with intent classification task on an Italian customer service data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've more or less 1.5k sentences and 29 classes (imbalanced).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the literature, a good choice is to generate synthetic data, oversampling, or undersampling the training data, using for example the &lt;a href=&quot;https://en.wikipedia.org/wiki/Oversampling_and_undersampling_in_data_analysis#SMOTE&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMOTE algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want use a cross-validation mechanism (&lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-validation_(statistics)#k-fold_cross-validation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stratified &lt;em&gt;k&lt;/em&gt;-fold&lt;/a&gt;) to be more confident in the obtained result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that accuracy is not the right metric to take into account, I should use precision, recall and confusion matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to combine cross validation &lt;em&gt;k&lt;/em&gt;-fold and oversampling or undersampling techniques?&lt;/p&gt;&#xA;" OwnerUserId="20780" LastEditorUserId="2193" LastEditDate="2019-03-11T15:08:00.590" LastActivityDate="2019-03-11T18:13:08.523" Title="Multi class text classification with imbalanced data" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11152" PostTypeId="1" CreationDate="2019-03-11T16:02:04.947" Score="4" ViewCount="298" Body="&lt;p&gt;Reading the high-level descriptions of &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;backpropagation&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Predictive_coding&quot; rel=&quot;nofollow noreferrer&quot;&gt;predictive coding&lt;/a&gt;, they don't sound so drastically different. What is the key difference between these techniques?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently reading the following paper if that helps ground the explanation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1706.02444.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Predictive Coding-based Deep Dynamic Neural Network for Visuomotor&#xA;Learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20955" LastEditorUserId="2444" LastEditDate="2019-03-12T15:36:31.373" LastActivityDate="2019-03-12T15:36:31.373" Title="What is the difference between backpropagation and predictive coding?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11153" PostTypeId="2" ParentId="11152" CreationDate="2019-03-11T16:35:30.547" Score="1" Body="&lt;p&gt;I would say that these concepts are quite different, even though they might have a few things in common (or might be vaguely related). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/8985/2444&quot;&gt;Back-propagation&lt;/a&gt; is an &lt;em&gt;algorithm&lt;/em&gt; used (in machine learning) to compute the gradient of a function with respect to its parameters. This gradient is then used by algorithms, like gradient descent, to update the parameters of the model (e.g. a neural network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roughly, &lt;a href=&quot;https://theness.com/neurologicablog/index.php/the-brain-and-predictive-coding/&quot; rel=&quot;nofollow noreferrer&quot;&gt;predictive coding&lt;/a&gt; is a general (neuroscience) &lt;em&gt;theory&lt;/em&gt; of how the brain builds an internal model of the external world, of how it continuously predicts the sensory inputs (from the world) using its current model of the external world, and of how it updates this internal model once the sensory inputs are actually received.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could think of the output of an ML model, while it is being trained (using gradient descent with back-propagation), as a prediction associated with the inputs. However, note that the output of the model is often not a prediction of the actual input, but e.g. a label (which is, nonetheless, associated with the input). Furthermore, it is the model that produces an output and not the back-propagation algorithm (even though the back-propagation algorithm is often used to train such models, e.g. neural networks). We could think of back-propagation as the way of updating these predictions associated with the inputs, but, anyway, it is well known (and accepted) that our brain does not perform back-propagation, but we learn in an &lt;a href=&quot;https://en.wikipedia.org/wiki/Hebbian_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;associative fashion&lt;/a&gt; (Hebbian learning).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-11T16:52:11.107" LastActivityDate="2019-03-11T16:52:11.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11154" PostTypeId="1" CreationDate="2019-03-11T17:29:07.163" Score="1" ViewCount="43" Body="&lt;p&gt;Glados {&lt;a href=&quot;https://en.wikipedia.org/wiki/GLaDOS&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLaDOS&lt;/a&gt; is responsible for testing and maintenance in the Aperture Science research facility in all titles. While GLaDOS initially appears in the first game to simply be a voice that guides the player, her words and actions become increasingly malicious as she makes her intentions clear. The second game, as well as the Valve created comic Lab Rat, reveals that she was mistreated by the scientists and used a neurotoxin to kill the scientists in the laboratory}&lt;/p&gt;&#xA;" OwnerUserId="23085" LastEditorUserId="1671" LastEditDate="2019-03-11T20:30:15.677" LastActivityDate="2019-03-11T20:30:15.677" Title="Can glados be real?" Tags="&lt;mythology-of-ai&gt;&lt;popular-culture&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11155" PostTypeId="2" ParentId="11141" CreationDate="2019-03-11T17:43:24.233" Score="1" Body="&lt;p&gt;If you really just want an SMDP-version of the algorithm, which only needs to be capable of operating on the &quot;high-level&quot; time scale of macro-actions, you can relatively safely take the original pseudocode of whatever MDP-based algorithm you like, replace every occurrence of &quot;action&quot; with &quot;macro-action&quot;, and you're pretty much done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only caveat I can think of in the case of &lt;span class=&quot;math-container&quot;&gt;$Q(\lambda)$&lt;/span&gt; is that the &quot;optimal&quot; value for &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is probably somewhat related to the amount of time that expires... so intuitively I'd expect it to be best if the value for &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; decreases as the amount of time expired during execution of the last macro-action increases. A constant &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; probably still works fine as well though.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you actually want your algorithm to also be aware of lower-time-scale MDP underlying an SMDP, and not only treat macro-actions as &quot;large actions&quot; and be done with it... I'd recommend looking into the Options framework. There you get interesting ideas like intra-option updates, which may allow you to also perform learning whilst larger macro-actions (or options) are still in progress. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last time I looked there hasn't been a lot of work involving the combination of eligibility traces and options, but there has been some: &lt;a href=&quot;http://ifaamas.org/Proceedings/aamas2018/pdfs/p1008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eligibility Traces for Options&lt;/a&gt;. This paper doesn't specifically apply the algorithm you mentioned (&lt;span class=&quot;math-container&quot;&gt;$Q(\lambda)$&lt;/span&gt;), but it does discuss a bunch of other -- much more recent, and likely better -- off-policy algorithms with eligibility traces.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-03-11T17:43:24.233" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11157" PostTypeId="2" ParentId="11150" CreationDate="2019-03-11T18:13:08.523" Score="0" Body="&lt;p&gt;It is straightforward to combine k-fold cross-validation with a technique like oversampling or undersampling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, apply the balance-restoration technique to your training data. Then parametrize a model using k-fold cross-validation on the re-balanced training data. In Scikit learn, I believe you can even bundle these actions together into a single 'pipeline' object to make it easier to manipulate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Precision/recall is probably a fine starting place for measuring performance.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-11T18:13:08.523" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11159" PostTypeId="2" ParentId="10360" CreationDate="2019-03-11T19:46:52.113" Score="3" Body="&lt;p&gt;By far the most common form of heuristic evaluation functions for Chess-playing (or, really, any game-playing) agents are simple linear functions. At least when we're talking about handcrafted features that's the case, of course all the hype with Deep Neural Networks in more recent years is different. So, when it's not specified in a paper like this exactly what their heuristic evaluation function looks like, you can relatively safely assume it's just a linear function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With linear function, I mean that you have vectors of features &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\phi}(s)$&lt;/span&gt; for your states &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and a vector of weights &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt;, and the evaluation &lt;span class=&quot;math-container&quot;&gt;$f(s)$&lt;/span&gt; of a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is simply given by the dot product (summing up all the multiplications of feature values with their corresponding weights):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(s) = \boldsymbol{\phi}(s)^{\top} \boldsymbol{\theta} = \sum_i \phi_i(s) \times \theta_i(s),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the subscript &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; indicates taking the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; element of a vector.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-03-11T19:46:52.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11160" PostTypeId="5" CreationDate="2019-03-11T20:34:29.230" Score="0" Body="&lt;p&gt;Myths about automata have been with us since at least Ancient Greece [See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Talos&quot; rel=&quot;nofollow noreferrer&quot;&gt;Talos]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial Intelligence and predictive science has been a staple of modern Science Fiction since at least Isaac Asimov.  [See: &lt;a href=&quot;https://en.wikipedia.org/wiki/I,_Robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;I, Robot&lt;/a&gt;; &lt;a href=&quot;https://en.wikipedia.org/wiki/Foundation_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundation Series&lt;/a&gt;]&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-11T20:34:29.230" LastActivityDate="2019-03-11T20:34:29.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11161" PostTypeId="4" CreationDate="2019-03-11T20:34:29.230" Score="0" Body="Use for questions about AI in popular culture.  (Terminator/Skynet as an example.)  Includes movies/tv, novels &amp; comic books, and even canonical mythology (Talos)." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-11T20:34:29.230" LastActivityDate="2019-03-11T20:34:29.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11163" PostTypeId="1" CreationDate="2019-03-12T01:11:11.133" Score="1" ViewCount="30" Body="&lt;p&gt;According to &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PER&lt;/a&gt;, we have to multiply the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; error &lt;span class=&quot;math-container&quot;&gt;$\delta_i$&lt;/span&gt; by the importance sampling ratio to correct the bias introduced by the imbalance sampling of PER, where importance sampling ratio is defined&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_i=\left({1\over N}{1\over P(i)}\right)^\beta&#xA;$$&lt;/span&gt;&#xA;in which &lt;span class=&quot;math-container&quot;&gt;$1/N$&lt;/span&gt; is the probability of drawing a sample uniformly from the buffer, and &lt;span class=&quot;math-container&quot;&gt;$P(i)$&lt;/span&gt; is the probability of drawing a sample from PER.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if we have to do the same to the target of the actor when we apply PER to DDPG. That is, multiplying &lt;span class=&quot;math-container&quot;&gt;$-Q(s_i, \mu(s_i))$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the outcome of the actor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, it is necessary. And I've done some experiments in the gym environment &lt;a href=&quot;https://github.com/openai/gym/wiki/BipedalWalker-v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;BipedalWalker-v2&lt;/a&gt;. The results, however, is quite confusing: I constantly get better performance when I do not apply importance sampling to the actor. Why would this be the case?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="2444" LastEditDate="2019-04-04T16:37:20.530" LastActivityDate="2019-04-04T16:37:20.530" Title="Should we multiply the target of actor by the importance sampling ratio when prioritized replay is applied to DDPG?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;experience-replay&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11165" PostTypeId="1" AcceptedAnswerId="11185" CreationDate="2019-03-12T09:57:37.507" Score="2" ViewCount="123" Body="&lt;p&gt;I wonder if it would be possible to know the size of a room using image, I don't see anything about this subject, do you have some idea how it could be done?&lt;/p&gt;&#xA;" OwnerUserId="23107" LastEditorUserId="2444" LastEditDate="2019-03-12T11:02:33.867" LastActivityDate="2019-03-13T02:58:19.773" Title="How could we estimate the square footage of a room from an image?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11166" PostTypeId="1" CreationDate="2019-03-12T10:29:49.193" Score="17" ViewCount="3058" Body="&lt;p&gt;What is geometric deep learning (GDL)?&lt;/p&gt;&#xA;&lt;p&gt;Here are a few sub-questions&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How is it different from deep learning?&lt;/li&gt;&#xA;&lt;li&gt;Why do we need GDL?&lt;/li&gt;&#xA;&lt;li&gt;What are some applications of GDL?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-06T14:24:17.047" LastActivityDate="2021-02-03T01:27:19.697" Title="What is geometric deep learning?" Tags="&lt;deep-learning&gt;&lt;definitions&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="20" ContentLicense="CC BY-SA 4.0" />
  <row Id="11167" PostTypeId="5" CreationDate="2019-03-12T10:36:09.487" Score="0" Body="&lt;p&gt;For more info, have a look at this question &lt;a href=&quot;https://ai.stackexchange.com/q/11166/2444&quot;&gt;What is geometric deep learning?&lt;/a&gt; or the paper &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geometric deep learning: going beyond Euclidean data&lt;/a&gt; (2017) by Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, Pierre Vandergheynst.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-12T20:34:27.997" LastActivityDate="2019-03-12T20:34:27.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11168" PostTypeId="4" CreationDate="2019-03-12T10:36:09.487" Score="0" Body="For questions related to geometric deep learning, which is the application of deep learning techniques to non-Euclidean data (e.g. graphs and manifolds)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-18T12:43:13.190" LastActivityDate="2019-06-18T12:43:13.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11169" PostTypeId="1" CreationDate="2019-03-12T10:46:08.563" Score="5" ViewCount="597" Body="&lt;p&gt;What is a graph neural network (GNN)?&lt;/p&gt;&#xA;&lt;p&gt;Here are some sub-questions&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How is a GNN different from a NN?&lt;/li&gt;&#xA;&lt;li&gt;How exactly is a GNN related to graphs?&lt;/li&gt;&#xA;&lt;li&gt;What are the components of a GNN? What are the inputs and outputs of GNNs?&lt;/li&gt;&#xA;&lt;li&gt;How can GNNs be trained? Can we also use gradient descent with back-propagation to train GNNs?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-06T11:52:54.917" LastActivityDate="2020-11-06T11:52:54.917" Title="What is a graph neural network?" Tags="&lt;deep-learning&gt;&lt;definitions&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11171" PostTypeId="1" CreationDate="2019-03-12T13:31:25.977" Score="0" ViewCount="24" Body="&lt;p&gt;I have an agent that will play in a multi-agent environment, and I'd like that agent to play against other agents that have been previously trained. The model for each agent was saved using &lt;code&gt;tf.train.Saver()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the neatest way to load the opponents models, bearing in mind each agent will (I suspect) need its own independent Session and graph, otherwise I'll end up with duplicate model elements on the same graph and TensorFlow will throw an error.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems rather clunky to need lots of graphs and sessions, so I was wondering if there was a neater way to achieve this?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-03-12T13:31:25.977" Title="Loading multiple trained models for use in multi-agent environment" Tags="&lt;python&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11172" PostTypeId="1" AcceptedAnswerId="21874" CreationDate="2019-03-12T13:57:59.017" Score="4" ViewCount="795" Body="&lt;p&gt;How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, rather than sliding this kernel (e.g. using loops), we can perform the convolution operation &quot;in one step&quot; using a matrix-vector multiplication, where the matrix is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Circulant_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;circulant matrix&lt;/a&gt; containing shifted versions of the kernel (as rows or columns) and the vector is the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly can this operation be performed? I am looking for a detailed step-by-step answer that shows how the convolution operation (as usually presented) can be performed using a matrix-vector multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the usual way the convolution operations are implemented in CNNs?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-14T14:03:27.817" LastActivityDate="2020-11-07T00:33:53.540" Title="How can the convolution operation be implemented as a matrix multiplication?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;implementation&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11173" PostTypeId="1" CreationDate="2019-03-12T16:00:54.587" Score="1" ViewCount="71" Body="&lt;p&gt;The book by Sutton and Barto discussed in section 11.8 that the convergence of off-policy TD function approximation can be improved by correcting for distribution of states encountered. The section seems to be written in haste and doesn't do a good job in explaining why will &lt;span class=&quot;math-container&quot;&gt;$M_t$&lt;/span&gt;, the emphasis, help in getting a state distribution closer to the target policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding of on-policy distribution is not clear at the moment. I think it is the distribution of states encountered under the target policy (the policy for which we want to state-action/state values). The importance sampling ratio corrects for update distribution (by multiplying the correction term with the ratio), but how is &lt;span class=&quot;math-container&quot;&gt;$M_t$&lt;/span&gt; helping in correcting for state distribution?&lt;/p&gt;&#xA;" OwnerUserId="21509" LastActivityDate="2019-03-13T07:29:41.120" Title="On-policy distribution for Emphatic TD" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11174" PostTypeId="1" AcceptedAnswerId="11194" CreationDate="2019-03-12T16:30:55.220" Score="7" ViewCount="1431" Body="&lt;p&gt;I have been using &lt;a href=&quot;https://github.com/openai/retro&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Retro&lt;/a&gt; for awhile, and I wanted to experiment with two player games. By two player games, I mean co-op games like &quot;Tennis-Atari2600&quot; or even Pong, where 2 agents are present in one environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a parameter for &lt;code&gt;players&lt;/code&gt; in the OpenAI documentation, but setting this variable to &lt;code&gt;2&lt;/code&gt; does nothing in terms of the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do you properly implement this? Can this even be done?&lt;/strong&gt; &#xA;The end goal is to have 2 separate networks per one environment.&lt;/p&gt;&#xA;" OwnerUserId="23119" LastEditorUserId="1847" LastEditDate="2019-03-13T09:43:52.320" LastActivityDate="2020-06-28T23:17:12.837" Title="2 Player Games in OpenAI Retro" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11177" PostTypeId="2" ParentId="6231" CreationDate="2019-03-12T17:00:35.483" Score="3" Body="&lt;p&gt;I can think of two possible ways of enforcing NEAT to create a feed forward network. One elegant one and one a little more cumbersome one;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Only allow the &quot;add connection&quot; mutation to connect a node with another node that have a higher maximum distance from an input node. This should result in feed forward network, without much extra work. (Emergent properties are great!)&lt;/li&gt;&#xA;&lt;li&gt;Run as you did and create a fully connected network with NEAT and then prune it during a forward pass. After creating the network, run through it and remove connections that try to connect to a node already used in the forward pass (example 3-&gt;5). Alternatively just remove unused input connections to nodes during the forward pass. Given how NEAT mutates, it should not be possible that you remove a vital connection and cut the the network in two. This property of NEAT make sure your signal will always be able to reach the output, even if you remove those &quot;backwards pointing&quot; connections. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I believe these should work, however i have not tested them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original NEAT paper assumed a feed forward ANN, even though its implementation as described would result in a fully connected network. I think it was just an assumption of the paradigm they worked in. The confusion is fully understandable.&lt;/p&gt;&#xA;" OwnerUserId="23118" LastActivityDate="2019-03-12T17:00:35.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11178" PostTypeId="1" CreationDate="2019-03-12T17:10:04.383" Score="1" ViewCount="189" Body="&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt;, the task is to learn to predict which words are most likely to be near each other in some long corpus of text. For each word &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in the corpus, the model outputs the probability distribution &lt;span class=&quot;math-container&quot;&gt;$P(O=o|C=c)$&lt;/span&gt; of how likely each other word &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; in the vocabulary is to be within a certain number of words away from &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. We call &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; the &quot;center word&quot; and &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; the &quot;outside word&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We choose the softmax distribution as the output of our model: &lt;span class=&quot;math-container&quot;&gt;$$P(O=o|C=c) = \frac{\exp(\textbf{u}_{0}^{T} \textbf{v}_{c})}{\sum_{w \in \text{Vocab}} \exp(\textbf{u}_{w}^{T} \textbf{v}_c)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\textbf{u}_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\textbf{v}_c$&lt;/span&gt; are vectors that represent the outside and center words respectively. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question.&lt;/strong&gt; What do the vectors  &lt;span class=&quot;math-container&quot;&gt;$\textbf{u}_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\textbf{v}_c$&lt;/span&gt; look like? Are they just one-hot-encodings? Do we need to learn them&#xA;  too? Why is this useful?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="23120" LastEditorUserId="2444" LastEditDate="2019-04-16T22:26:26.883" LastActivityDate="2019-04-16T22:26:26.883" Title="What do the vectors of the center and outside word look like in word2vec?" Tags="&lt;natural-language-processing&gt;&lt;word2vec&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11179" PostTypeId="5" CreationDate="2019-03-12T19:39:35.700" Score="0" Body="&lt;p&gt;For more info, have a look e.g. at &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_search_(optimization)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Local_search_(optimization)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-12T21:36:32.013" LastActivityDate="2019-03-12T21:36:32.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11180" PostTypeId="4" CreationDate="2019-03-12T19:39:35.700" Score="0" Body="For questions related to local search algorithms used in AI (e.g. 2-opt or hill climbing)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-12T21:36:52.380" LastActivityDate="2019-03-12T21:36:52.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11181" PostTypeId="1" CreationDate="2019-03-12T19:51:22.200" Score="2" ViewCount="1768" Body="&lt;p&gt;I want to tackle the problem of detecting similar objects in an image. To illustrate the problem consider this photo of some Lego bricks as my &quot;input&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pIBRs.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pIBRs.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The detection routine should identify similar objects. So for the given input, it should e.g. identify the following output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jHtR8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jHtR8.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So an object might appear none to multiple times in the input image. For example, there are only two bricks marked with a blue cross, but three bricks marked with a red cross.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be assumed that all objects are of similar kind, so e.g. only Lego bricks or a heap of sweets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial idea is to apply a two-phased approach:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Extract all objects in input image.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compare those extracted objects and find similar ones.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is this a valid approach or is there already some standard way of solving this kind of problem? Can you give me some pointers how to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="23123" LastActivityDate="2019-03-13T07:47:07.023" Title="How do I detect similar objects in an image?" Tags="&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11182" PostTypeId="1" AcceptedAnswerId="11183" CreationDate="2019-03-12T21:09:52.673" Score="1" ViewCount="335" Body="&lt;p&gt;I am trying to understand the value iteration method for Markov Decision Process(MDP) and I was referring ot UC Berkeley's slides titled &lt;a href=&quot;https://people.eecs.berkeley.edu/~pabbeel/cs287-fa12/slides/mdps-exact-methods.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Processes&#xA;and&#xA;Exact Solution Methods&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On slide no.9, we start with the first step :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5Jyh2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Jyh2.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok! So, we have the information about the transition function (desribed elaborately in slide no.5 as well), the &lt;strong&gt;resting reward is 0&lt;/strong&gt; and &lt;strong&gt;discount of 0.9&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this, I am able to compute the &lt;em&gt;utility value&lt;/em&gt; of the cell left to terminal state with R = +1 (Green cell). The action that is going to be most rewarding at this cell is moving forward, so putting the values in the equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.0 + 0.9(0.8*1 + 0.1*0 + 0.1*0)&#xA;=0.72&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which seems to be correct:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LCQzx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LCQzx.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, using the same algorithm I am able to compute the value of the cells adjacent to this newly obtained utility cell value. However, I really do not know how did they update the value from &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;0.72 -&gt; 0.78&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;in the next slide:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8lgsJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8lgsJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried searching at various sites and seen some videos but most of them stop at first iteration assuming the next step is same as it is a recursive equation (And it should have been so!) but I am stuck at this!&lt;/p&gt;&#xA;" OwnerUserId="23126" LastActivityDate="2019-09-15T18:22:19.567" Title="Unable to understand the second iteration update in value iteration algorithm for solving MDP" Tags="&lt;reinforcement-learning&gt;&lt;ai-basics&gt;&lt;value-iteration&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11183" PostTypeId="2" ParentId="11182" CreationDate="2019-03-12T22:05:07.810" Score="1" Body="&lt;p&gt;First thing to know is that, in this case, values for the gridworld in new iteration are completely calculated with respect to the old values from the previous iteration. Value of &lt;span class=&quot;math-container&quot;&gt;$0.78$&lt;/span&gt; is got like this:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$0.9 \cdot (0.8 \cdot 1 + 0.1 \cdot 0.72 + 0.1 \cdot 0) = 0.7848 \approx 0.78$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;term &lt;span class=&quot;math-container&quot;&gt;$0.8 \cdot 1$&lt;/span&gt; is for going to the right with probability of &lt;span class=&quot;math-container&quot;&gt;$0.8$&lt;/span&gt; and getting reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;term &lt;span class=&quot;math-container&quot;&gt;$0.1 \cdot 0.72$&lt;/span&gt; is for going up with probability of &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt;, we hit the wall and stay in the same field which value is &lt;span class=&quot;math-container&quot;&gt;$0.72$&lt;/span&gt; (from previous iteration)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;term &lt;span class=&quot;math-container&quot;&gt;$0.1 \cdot 0$&lt;/span&gt; is for going down with probability of &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt;, even though value of that field in the image is &lt;span class=&quot;math-container&quot;&gt;$0.43$&lt;/span&gt; we take the value from previous iteration which is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-03-12T22:05:07.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11185" PostTypeId="2" ParentId="11165" CreationDate="2019-03-13T02:58:19.773" Score="0" Body="&lt;p&gt;Welcome to AI.SE Hadrien!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible approach is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Gather many &lt;em&gt;example&lt;/em&gt; images of rooms for which you know the square footage. Record the square footage of each room together with each image.&lt;/li&gt;&#xA;&lt;li&gt;Pick a machine learning model that is well suited to learning relationships between images and numerical outputs, like a Convolutional Neural Network.&lt;/li&gt;&#xA;&lt;li&gt;Train a machine learning model using an &lt;em&gt;optimisation&lt;/em&gt; algorithm, like &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient decent&lt;/a&gt;. In the case of training a CNN, this algorithm starts by setting up the network with randomly chosen connection strengths between different 'simulated' neurons. It then exposes the network to an input image, and observes the number the network outputs in response. The algorithm then makes small adjustments to the strengths of the connections in the network, so that if the network were exposed to the image a second time, it would output a number that was less wrong (i.e. closer to the correct square footage). By repeating this process many thousands of times, and with many images, the network eventually becomes quite good at guessing the square footage of new images that it hasn't seen before.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In practice, the network is quite likely to pick up on &lt;em&gt;any&lt;/em&gt; patterns in the images that correlate with square footage, not necessarily the ones you want. For example, it might pick up on, say, the fact that humans in the picture are usually a good object to guess at the scale of the rest of the room.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-13T02:58:19.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11187" PostTypeId="2" ParentId="11173" CreationDate="2019-03-13T06:48:28.987" Score="1" Body="&lt;p&gt;I don't think the section was written in haste. I think they just didn't have space to include the whole proof. It's a bit involved, so they just gave concepts.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://armahmood.github.io/files/SMW-emphasis-2016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Emphatic Approach to the Problem&#xA;of Off-policy Temporal-Difference Learning&lt;/a&gt; gives a proof of stability. &#xA;At least parts of it should seem familiar if you've read Sutton and Barto's proof of the convergence of linear TD(0) on page 206 of their 2nd edition RL book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v40/Yu15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Convergence of Emphatic Temporal-Difference Learning&lt;/a&gt; gives a proof of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I confess that I don't understand these papers well enough to give a summary. If you eventually do, I would greatly appreciate an update.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-13T07:29:41.120" LastActivityDate="2019-03-13T07:29:41.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11188" PostTypeId="2" ParentId="11181" CreationDate="2019-03-13T07:39:04.863" Score="1" Body="&lt;p&gt;Disclaimer: I have never used Siamese networks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would approach this problem in two steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First: train object detector and train it for eligible classes of object, for example using &lt;a href=&quot;https://github.com/pjreddie/darknet/wiki/YOLO:-Real-Time-Object-Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yolo&lt;/a&gt; architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use pretrained object detector and finetune it for your classes of objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second: extract a lot of bounding boxes of eligible objects of the same classes from your dataset and train Siamese network on their subimages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Siamese_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siamese network&lt;/a&gt; output similarity measure between two objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your pipleline would look like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Run object detetctor&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each two pair of objects of the same class rescale bounding boxes and run Siamese network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check if the similarity distance between two objects is less then threshold (tunable hyperparameter)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-03-13T07:47:07.023" LastActivityDate="2019-03-13T07:47:07.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11189" PostTypeId="1" AcceptedAnswerId="11193" CreationDate="2019-03-13T08:29:19.037" Score="1" ViewCount="32" Body="&lt;p&gt;I am making an AI model to predict monthly retail sales of a motor cycle spare parts shop, for that to be possible I have to first create a dataset. The problem I am facing is what features should the dataset have?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already did some research on some other datasets but still I want to know specifically what features should it have other than Date, Product Name, Quantity, Net amount, Gross amount..?&lt;/p&gt;&#xA;" OwnerUserId="23140" LastEditorUserId="9608" LastEditDate="2019-03-13T20:58:40.717" LastActivityDate="2019-03-13T20:58:40.717" Title="What features should a dataset to predict monthly retail sales for a motorcycle spare parts shop have?" Tags="&lt;datasets&gt;&lt;feature-selection&gt;&lt;data-mining&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11190" PostTypeId="2" ParentId="11178" CreationDate="2019-03-13T08:38:28.710" Score="1" Body="&lt;p&gt;No, the word vectors are not one-hot encodings. Yes, they are learned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of the word2vec model is actually to learn dense, semantically meaningful encodings for words. That is, if your words are &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;-dimensional vectors, then each word's position in this vector space &lt;strong&gt;says something about what that word means&lt;/strong&gt;. This is because word2vec learns to represent words in similar ways if they are frequently close together in your corpus. It implements the idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Distributional_semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;distributional similarity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The task of predicting an &quot;outside word&quot; given a &quot;center word&quot; accomplishes all of this in an indirect way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A naive objective function to maximize for word2vec is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$J = \prod_{t=1}^L \prod_{-m \leq j \leq m\\ \quad j\neq 0} p(\textbf{u}_{t+j}|\textbf{v}_t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the length of your corpus, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the &quot;radius&quot; from each center word you want to consider, &lt;span class=&quot;math-container&quot;&gt;$\textbf{u}_{t+j}$&lt;/span&gt; is an outside word, and &lt;span class=&quot;math-container&quot;&gt;$\textbf{v}_t$&lt;/span&gt; is a center word. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we let &lt;span class=&quot;math-container&quot;&gt;$p(\textbf{u}_{t+j}|\textbf{v}_t)$&lt;/span&gt; be the softmax distribution, then maximizing &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; means maximizing the inner product &lt;span class=&quot;math-container&quot;&gt;$\textbf{u}_{t+j}^T\textbf{v}_t$&lt;/span&gt; in the softmax's numerator. Maximizing that inner product means making center words as close as possible to their neighboring words, giving you some semantically meaningful word vectors to use in your downstream NLP tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=ERibwqs9p38&quot; rel=&quot;nofollow noreferrer&quot;&gt;This lecture&lt;/a&gt; from Stanford's CS224N goes into more detail.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-13T08:52:55.447" LastActivityDate="2019-03-13T08:52:55.447" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11193" PostTypeId="2" ParentId="11189" CreationDate="2019-03-13T10:41:33.890" Score="0" Body="&lt;p&gt;This is the prototypical problem in AI/ML known as feature-selection. If you do not have too many features, typically one would just use them all(with the exception of features that are known to to be correlated), in this case you would want to perform feature engineering wherever possible as well. One the other hand, if you have lots of features, and some are likely to be useless, you would use a feature-selection technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many many methods for doing this. One can simply use their intuitive grasp as their problem domain at the simplest level. However, there are also many algorithmic ways of performing it. Some examples of this are a low variance filter, or using a ensemble(classifier or regression depending on the problem) which can then be used to order features according to their derived importance(I personally like this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A search of &quot;feature selection methods in ML&quot; will yield many potential ways to accomplish your goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as what to grab if you don't have many, as many as possible wrt domain knowledge.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-03-13T10:41:33.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11194" PostTypeId="2" ParentId="11174" CreationDate="2019-03-13T11:09:57.557" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/openai/retro&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Retro&lt;/a&gt; is an extension to &lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Gym&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, it does not support multiple agents in multi-player environments. Instead, the environment is always presented from the perspective of a single agent that needs to solve a control problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When there is more than one agent in a Gym problem, everything other than the first agent is considered part of the environment. Typically player 2 in 2-player problems is controlled by whatever game AI already exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can work around this in a couple of ways, both involve you modifying existing environments, requiring a solid understanding of the integration and wrapper layers in Gym so that you can make correct edits. However you &lt;em&gt;might&lt;/em&gt; find environments where either of these are done for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, within the Retro environments, some multiplayer environments are supported using the second option below.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;1. Supply additional player(s) as part of the environment&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You could modify the environment code so that it accepts a parameter which is the agent used to control other players. Typically such an agent would not be set up to learn, it could just be some earlier version of the agent that is learning. You would need to code the interface between the supplied agent and the game engine - taking care to allow for its potentially different view of success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a Gym environment has been set up for you (or any human) to compete against your own agent using the keyboard, that is already halfway there to this solution. You could take a look at where the Gym environment has been set up for this, and modify the code to allow for automated input as opposed to action selection by keyboard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach probably works best for competitive games where you can use a random selection of previous agents inserted into the environment to train a next &quot;generation&quot;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2. Present actions for multiple players as if there was a single player&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Provided the game emulator allows you to do this, you can edit the code so that multiple players are controlled at once. The Gym environment does not specify &lt;em&gt;how&lt;/em&gt; you construct your action choices, so you are then free to split the action vector up so that parts of it are chosen by one trained agent and parts of it by another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may need to work on different views of the environment and/or reward signal in this case. For instance in a competitive game (or a co-operative game with competitive elements such as personal scores) then you will need to add custom code to re-interpret reward values compared to single player game. For something simple like Pong, that could just be that Player 2 receives the negative of Player 1's reward signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach probably works best for cooperative games where success is combined. In that case, you have the choice of writing any number of separate agents that control different aspects of the action space - that would include having agents that separately controlled P1 and P2 game controller inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each agent would still effectively treat its partner as a part of the environment, so you may want to mix training scenarios so that agents for each controller did not get too co-dependent. For instance, if you hope to step in to control P1, you probably don't want the AI running P2 to have too specific expectations of P1's role and behaviour in the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This second option is what &lt;a href=&quot;http://retro.readthedocs.io/en/latest/python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Retro interface does&lt;/a&gt;. For instance Pong supports this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;env = retro.make(game='Pong-Atari2600', players=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I verified this using the sample code in the documentation, which controls both paddles, and receives the reward signal as a Python &lt;code&gt;list&lt;/code&gt; mostly &lt;code&gt;[0.0, 0.0]&lt;/code&gt; but whenever a player scores it becomes &lt;code&gt;[-1.0, 1.0]&lt;/code&gt; or &lt;code&gt;[1.0, -1.0]&lt;/code&gt; depending on which player got the point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will only be supported on retro games where someone has made the effort to make it work as above. If you have a game which does not do this, but in theory supports multiplayer on the ROM, you will need to look at and alter either the AI retro wrapper or the emulator or both.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-15T19:04:34.423" LastActivityDate="2019-03-15T19:04:34.423" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11196" PostTypeId="1" CreationDate="2019-03-13T11:20:27.733" Score="5" ViewCount="132" Body="&lt;p&gt;I am looking for a source that really discusses the classic rules of learning in depth. So classical conditioning, operant conditioning, imitation learning... I have found an infinite number of books that supposedly discuss these topics, but have not yet found anything that summarizes all the important findings on this topic. I am familiar with the basics, but I am explicitly interested in a detailed presentation of the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me a good source about the different forms of classical learning in mammals? I consider &quot;Reinforcement Learning: An Introduction&quot; comprehensive and detailed regarding RL. A comparable book on biological learning systems would be great. Sources in German and English would fit.&lt;/p&gt;&#xA;" OwnerUserId="17658" LastEditorUserId="1847" LastEditDate="2019-03-14T07:44:41.517" LastActivityDate="2019-03-18T12:47:02.480" Title="Looking for comprehensive material on learning in mammals" Tags="&lt;reinforcement-learning&gt;&lt;biology&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11198" PostTypeId="2" ParentId="11189" CreationDate="2019-03-13T15:36:12.210" Score="0" Body="&lt;p&gt;I would think you would want the brands or model numbers each part fits, and also data on the number of each model or brand sold each year.  E.g. you need to stock more parts for a Honda than for an Indian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over time your system should be able to learn what demand patterns are for each part, but to &quot;prime the pump&quot; some sort of information on number of consumers for each part would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would also want to know what the time needed to restock is for each part.&lt;/p&gt;&#xA;" OwnerUserId="23143" LastActivityDate="2019-03-13T15:36:12.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11200" PostTypeId="1" CreationDate="2019-03-13T16:36:18.250" Score="2" ViewCount="42" Body="&lt;p&gt;My group is working on a ML model that can work with little data (and bad accuracy) as long as there actually is little data available but can easily be extended as soon as said data becomes available (Think of interpolating existing models and then creating an individual predictor). This is due to a business requirement of the relevant application (new plants get installed over time but need to be integrated seamlessly). Transforming from a coarse yet cheap prediction to an accurate but expensive prediction takes labor effort that the company can invest as desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there research areas that take into account such &quot;evolutionary transformation processes&quot; which enable a tradeoff between costs and accuracy? What are the right keywords to look for? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for keywords/papers/communities.&lt;/p&gt;&#xA;" OwnerUserId="23158" LastEditorUserId="2444" LastEditDate="2019-06-14T23:01:10.350" LastActivityDate="2019-07-15T00:01:10.110" Title="Are there communities dealing with costs-vs-accuracy tradeoffs in Machine Learning?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11201" PostTypeId="2" ParentId="11166" CreationDate="2019-03-13T17:39:23.130" Score="7" Body="&lt;p&gt;The article &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geometric deep learning: going beyond Euclidean data&lt;/a&gt; (by Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, Pierre Vandergheynst) provides an overview of this relatively new sub-field of deep learning. It answers all the questions asked above (and more). If you are familiar with deep learning, graphs, linear algebra and calculus, you should be able to follow this article.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What is geometric deep learning (GDL)&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;This article describes GDL as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, the inputs to these GDL models are graphs (or representations of graphs), or, in general, any &lt;a href=&quot;https://ai.stackexchange.com/q/11226/2444&quot;&gt;non-Euclidean data&lt;/a&gt;. To be more concrete, the input to these models (e.g. &lt;a href=&quot;https://ai.stackexchange.com/q/11169/2444&quot;&gt;graph neural networks&lt;/a&gt;) are e.g. feature vectors associated with the nodes of the graphs and matrices which describe the graph structure (e.g. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Adjacency_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;adjacency matrix&lt;/a&gt; of the graphs).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Why are e.g. graphs non-Euclidean data?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;A graph is a non-Euclidean structure because e.g. distances between nodes are not well defined. Yes, you can have graphs with weights associated with the edges, but not all graphs have this property.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What classes of problems does GDL address?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In GDL, there are two classes of problems that are often tackled:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;characterise the structure of the data (e.g. of a graph)&lt;/li&gt;&#xA;&lt;li&gt;analyse functions defined on a given non-Euclidean domain&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;These classes of problems are related, given that the structure of the graph imposes certain properties on the functions that can be defined on it. Furthermore, these properties of these functions can also convey information about the structure of the graph.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What are applications of GDL?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;An example of an application where this type of data (graphs) arises is in the context of social networks, where each user can be associated with a vertex of the social graph and the characteristics (or features) of each user (e.g. number of friends) can be represented as a feature vector (which can then be associated with the corresponding vertex of a graph). In this context, the goal might e.g. be to determine different groups of users in the social network (i.e. clustering).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Why can't we simply use deep learning methods (like CNNs) when the data is non-Euclidean?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are several problems that arise when dealing with non-Euclidean data. For example, operations like convolution are not (usually) defined on non-Euclidean data. More concretely, the relative position of nodes is not defined on graphs (but this would be required to perform the usual convolution operation): in other words, it is meaningless to talk about a vertex that is e.g. on the left of another vertex. In practice, it means that we can't simply use the usual CNN when we are given non-Euclidean data. There have been attempts to generalise the convolution operation to graphs (or to approximate it). The field is still quite new, so there will certainly be new developments and breakthroughs.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-03T01:27:19.697" LastActivityDate="2021-02-03T01:27:19.697" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11202" PostTypeId="1" CreationDate="2019-03-13T18:03:26.477" Score="1" ViewCount="55" Body="&lt;p&gt;Suppose we want to classify a review as good (&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) or bad (&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;). We have a training data set of &lt;span class=&quot;math-container&quot;&gt;$10,000$&lt;/span&gt; reviews. Also, suppose we have a vocabulary of &lt;span class=&quot;math-container&quot;&gt;$100,000$&lt;/span&gt; words &lt;span class=&quot;math-container&quot;&gt;$w_1, \dots, w_{100,000}$&lt;/span&gt;. So the data is a matrix of dimension &lt;span class=&quot;math-container&quot;&gt;$100,000 \times 10,000$&lt;/span&gt;. Let's represent each of the words in the reviews using a bag-of-words approach over tf-idf values. Also, we normalize the rows such that they sum to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In a logistic regression approach, would we have &lt;span class=&quot;math-container&quot;&gt;$10,000$&lt;/span&gt; different logistic regression models as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \log \left(\frac{p}{1-p} \right)_{1} = \beta_{0_{1}} + \beta_{1_{1}}w_{11} + \dots + \beta_{100,000_{1}}w_{100,000} \\ \vdots \\ \log \left(\frac{p}{1-p} \right)_{10,000} = \beta_{0_{10,000}} + \beta_{1_{10,000}}w_{11} + \dots + \beta_{100,000_{10,000}}w_{100,000}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So are we estimating &lt;span class=&quot;math-container&quot;&gt;$100,000 \times 10,000$&lt;/span&gt; coefficients?&lt;/p&gt;&#xA;" OwnerUserId="23120" LastEditorUserId="2444" LastEditDate="2020-12-04T17:35:09.997" LastActivityDate="2020-12-04T17:35:09.997" Title="How many parameter would there be in a logistic regression model used to classify reviews into &quot;good&quot; or &quot;bad&quot;?" Tags="&lt;natural-language-processing&gt;&lt;weights&gt;&lt;logistic-regression&gt;&lt;binary-classification&gt;&lt;bag-of-words&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11203" PostTypeId="1" CreationDate="2019-03-13T18:08:28.337" Score="1" ViewCount="117" Body="&lt;p&gt;I want to make an AI with deep learning which can adapt itself from user to user. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have food combiner AI which suggests a food to eat with another food as you give as input. This is the most personalized case I found to ask here. For example the AI suggested a food for me. However, the food AI suggested for me might not be good choice for another person. So another person will let the AI know like &quot;I don't like that food to eat with this. Etc. When the user let the AI know that, It should affect AI's further combination food suggestions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I build that AI? Where should I start from? which area or topics should I research? &lt;/p&gt;&#xA;" OwnerUserId="16864" LastEditorUserId="2444" LastEditDate="2019-03-13T20:51:18.713" LastActivityDate="2019-03-13T21:08:15.427" Title="How create an AI that continuously adapts to different users?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11204" PostTypeId="1" CreationDate="2019-03-13T18:10:49.323" Score="1" ViewCount="58" Body="&lt;p&gt;Currently I'm feeding spectrogram of audio to the CNN with 3 convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each convolution is followed by a max pool of filter size 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First -&gt; 5x5x4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second - &gt; 5x5x8&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third - &gt; 5x5x16&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and final layer is a fully connected with 512 unit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But while training with dropout of 0.25, getting train accuracy of 0.97 with 150 iterations.&#xA;and on test data accuracy is just 0.60.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tell me how to improve the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes both train and test data come from same distribution.&lt;/p&gt;&#xA;" OwnerUserId="18428" LastActivityDate="2019-04-12T22:02:15.450" Title="How to reduce over-fitting on training set?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11206" PostTypeId="2" ParentId="10830" CreationDate="2019-03-13T18:59:34.400" Score="4" Body="&lt;p&gt;After some research and reading &lt;a href=&quot;https://datascience.stackexchange.com/questions/24921/why-random-sample-from-replay-for-dqn?rq=1&quot;&gt;this&lt;/a&gt; post, I see where my problem was: I was introducing a full consecutive batch of experiences, selected randomly, yes, but the experiences in the batch were consecutives.&#xA;After redoing my experience selection method, my DQN is actually working and has reached about +200 points after 400000 experiences (about 500 episodes; only 2-3 hours or training!). Before I couldn't reach that score after days of training.&#xA;I'll let it train to see if there are something I can improve. Thanks to everyone who tried to help me! I let this answer here just in case someone has the same problem as me. &lt;/p&gt;&#xA;" OwnerUserId="9818" LastActivityDate="2019-03-13T18:59:34.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11208" PostTypeId="1" CreationDate="2019-03-13T19:33:08.647" Score="3" ViewCount="54" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Max_Tegmark&quot; rel=&quot;nofollow noreferrer&quot;&gt;Max Tegmark&lt;/a&gt; discusses the topic consciousness in his book &lt;a href=&quot;https://en.wikipedia.org/wiki/Life_3.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Life 3.0&lt;/a&gt; and comes to the conclusion, that consciousness is substrate independent. If his analysis is correct, it should be possible to create artificial consciousness. &lt;a href=&quot;https://en.wikipedia.org/wiki/Integrated_information_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;The integrated information theory (IIT)&lt;/a&gt;, while currently only just a theory, also points in this direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to the question, which fields of AI research are currently actively engaged in this domain, if any. So far I've only found research concerning consciousness in neuroscience and discussions of experts in philosophy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any projects publicly known concerning artificial consciousness or organizations who are active in this regard?&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2019-03-14T02:26:27.853" Title="Which fields of AI are actively researching consciousness?" Tags="&lt;artificial-consciousness&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11209" PostTypeId="1" CreationDate="2019-03-13T20:00:59.333" Score="3" ViewCount="55" Body="&lt;p&gt;I am reading the paper &lt;a href=&quot;http://martin.zinkevich.org/publications/regretpoker.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Regret Minimization in Games with Incomplete&#xA;Information&lt;/a&gt; on CFR algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On page 4, the paper defines &lt;span class=&quot;math-container&quot;&gt;$R^{T,+}_{i,\text{imm}}=\max\{R^{T}_{i,\text{imm}}, 0\}$&lt;/span&gt; after equation (5). I am confused why it is necessary? It seems to me that since in the definition of &lt;span class=&quot;math-container&quot;&gt;$R^{T}_{i,\text{imm}}$&lt;/span&gt; the regret is computed with respect to the optimal action.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As everything is in expectation, is mixed-action going to make any difference?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Isn't &lt;span class=&quot;math-container&quot;&gt;$R^{T}_{i,\text{imm}}$&lt;/span&gt; always nonzero already?&lt;/p&gt;&#xA;" OwnerUserId="23163" LastEditorUserId="1671" LastEditDate="2019-03-13T21:01:56.900" LastActivityDate="2019-03-13T21:01:56.900" Title="Negative counterfactual regret" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;game-theory&gt;&lt;poker&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11210" PostTypeId="2" ParentId="11203" CreationDate="2019-03-13T21:08:15.427" Score="1" Body="&lt;p&gt;Assuming you have enough training data and representational capacity, you can give each user a unique identifier and concatenate that with the other inputs to the neural net. As users give more feedback, the network will learn in the usual way since each situation (e.g. food-user pair) is represented by a unique input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd consider this a general, brute-force approach, and it may not be suited to your application. Others' links to recommendation systems might be more useful. It depends on what your task is and what your constraints are.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-13T21:08:15.427" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11211" PostTypeId="2" ParentId="11204" CreationDate="2019-03-13T21:09:28.773" Score="2" Body="&lt;p&gt;The problem of overfitting is given that your model is being too flexible with the training data. The approaches you could take will be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Download pretrained models (VGG-16) and use transfer learning.&lt;/li&gt;&#xA;&lt;li&gt;Increase the value of your dropout (e.g. from 0.25 to 0.50).&lt;/li&gt;&#xA;&lt;li&gt;Use data augmentation for your images.&lt;/li&gt;&#xA;&lt;li&gt;Reduce the number of fully connected layers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12006" LastActivityDate="2019-03-13T21:09:28.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11212" PostTypeId="1" AcceptedAnswerId="11404" CreationDate="2019-03-13T21:17:24.320" Score="1" ViewCount="137" Body="&lt;p&gt;I'm trying to replicate the DeepMind DQN paper. I'm using OpenAI's Gym. I'm trying to get a decent score with Space Invaders (using &lt;code&gt;SpaceInvaders-v4&lt;/code&gt; environment). I checked the actions available with &lt;code&gt;env.unwrapped.get_action_meanings()&lt;/code&gt;, and I get this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['NOOP', 'FIRE', 'RIGHT', 'LEFT', 'RIGHTFIRE', 'LEFTFIRE']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Checking the number of actions with &lt;code&gt;env.action_space.n&lt;/code&gt; gives me a number of 6 actions.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;RIGHTFIRE&lt;/code&gt; and &lt;code&gt;LEFTFIRE&lt;/code&gt; actions, I suppose, aren't used, given that they &lt;em&gt;seem&lt;/em&gt; to do the same as &lt;code&gt;LEFT&lt;/code&gt; and &lt;code&gt;RIGHT&lt;/code&gt;, am I right?&lt;/p&gt;&#xA;&lt;p&gt;If so, restricting the action size to the 4 first actions would improve my learning?&lt;/p&gt;&#xA;" OwnerUserId="9818" LastEditorUserId="2444" LastEditDate="2020-12-27T13:56:24.453" LastActivityDate="2020-12-27T13:57:19.517" Title="Should I ignore the actions RIGHTFIRE and LEFTFIRE in the SpaceInvaders environment?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;gym&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-12-28T16:05:20.037" ContentLicense="CC BY-SA 4.0" />
  <row Id="11213" PostTypeId="2" ParentId="11208" CreationDate="2019-03-14T02:26:27.853" Score="4" Body="&lt;p&gt;I think we still have a long way to go before any progress is made on artificial consciousness. However, researchers are taking inspiration from traits of human consciousness. One relevant paper is &lt;a href=&quot;https://arxiv.org/abs/1802.07740&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Theory of Mind&lt;/a&gt; by DeepMind. They show that their model can (at least to some extent) represent the desires, beliefs, and intentions of agents that it observes. It even passes a form of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sally%E2%80%93Anne_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sally-Anne test&lt;/a&gt;, showing that it can represent the &lt;em&gt;false beliefs&lt;/em&gt; of an agent. &lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-14T02:26:27.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11214" PostTypeId="1" CreationDate="2019-03-14T02:31:53.637" Score="1" ViewCount="222" Body="&lt;p&gt;I just recently got into machine learning, and have been hitting a lot of obstacles understanding the algorithms involved in the programmings. My issue isnt with the programming, but how they're translated from math to code. ML is popular with python, and that's okay, but i dont like python, and i dont want to have to learn &lt;em&gt;it&lt;/em&gt; to be able to use the programming language of &lt;em&gt;my&lt;/em&gt; choice, to do the &lt;em&gt;exact&lt;/em&gt; same thing but in a way i feel comfortable (i dont care if python is popular for math majors, because it's easier for them to understand -- it isnt for me, when nothing being done is explained thoroughly.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to decipher this model&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TX5OS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TX5OS.png&quot; alt=&quot;Algorithm logic&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is the breakdown for the algorithm model&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ACOJ3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ACOJ3.jpg&quot; alt=&quot;Learning Vector Quantization&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is the math i was able to decipher for this particular model&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lkBjE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lkBjE.jpg&quot; alt=&quot;Math Equation&quot;&gt;&lt;/a&gt;&#xA;(Left is terminology and their usage, the middle in black was something to do with programming arrays... below it is the equation used in bottom left, but more elaborate, and underneath that is a image that says the same thing the algorithm is doing.. because sometimes pictures are easier to understand that words &lt;code&gt;VectorArray(Value) * VectorArray(Weight) + SingleUnit(Bias) = Neuron(Node)&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then everything stops at the middle layer of the second image. How do i get the full output to give me a &lt;code&gt;yes&lt;/code&gt; or &lt;code&gt;no&lt;/code&gt; response?&#xA;How do i enter in the variables and tables to go thru the math steps?&#xA;Is my understanding correct, or am i lacking somewhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5638/hand-computing-feed-forward-and-back-propagation-of-neural-network&quot;&gt;This user is also sharing the same algorithm but our math dont look the same&lt;/a&gt;&#xA;How do i go from what i have, to what [s]he has?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of all of these questions, i'm going to write everything into a programming script, that'll use a different language from python (and i would need to manually create resources from scratch, because no one else thinks machine learning should be done in other languages -- it seems...). I want to be able to understand the process itself, without just doing cookie-cutter actions (tools made by users for those too lazy to do the work -- which circumvent the learning/understanding process of what's going on behind scenes).&lt;/p&gt;&#xA;" OwnerUserId="23171" LastEditorUserId="23171" LastEditDate="2019-03-14T07:54:43.193" LastActivityDate="2019-03-14T07:54:43.193" Title="How to translate algorithm from logic to equation, and back?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="11215" PostTypeId="1" CreationDate="2019-03-14T04:07:41.590" Score="0" ViewCount="31" Body="&lt;p&gt;I am building a mobile camera app that needs to show a real time preview of the processed image. So i was thinking of reducing the image size to 100 x 100. But as for the weights and number of layers. Is there a maximum known value for the processing time to be less than 20 fps.&lt;/p&gt;&#xA;" OwnerUserId="22876" LastActivityDate="2019-03-14T04:07:41.590" Title="Maximum number of nodes for CNN for mobile phones" Tags="&lt;convolutional-neural-networks&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11216" PostTypeId="2" ParentId="11200" CreationDate="2019-03-14T04:42:54.553" Score="1" Body="&lt;p&gt;Welcome to AI.SE ks.and1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you're describing could be related to several areas, but since you are looking only for some keywords, I'll suggest some:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You might be interested in &lt;a href=&quot;https://en.wikipedia.org/wiki/Anytime_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;anytime learning&lt;/a&gt;, which might better be called 'learning that can stop at any time'. A good example of this approach in machine learning is anytime learning for decision trees, as summarized in &lt;a href=&quot;http://www.jmlr.org/papers/volume8/esmeir07a/esmeir07a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Esmeir &amp;amp; Markovitch's 2007 JMLR paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Apart from this, it sounds like you are just describing the regular process of retraining a model when you get more data. For most algorithms, this just amounts to changing the input to your program so that it points at a larger file, containing more records than last time, and maybe buying a faster GPU or CPU to train the model with if it's taking too long with the extra data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hopefully, this can help you get started. If you have some more details about your problem, I might be able to suggest some more specific resources.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-14T04:42:54.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11218" PostTypeId="1" AcceptedAnswerId="11224" CreationDate="2019-03-14T05:02:29.717" Score="2" ViewCount="83" Body="&lt;p&gt;I need to create model which will find suspicious entries or anomalies in a network, whose characteristics or features are the &lt;code&gt;asset_id&lt;/code&gt;, &lt;code&gt;user_id&lt;/code&gt;, IP accessed from and &lt;code&gt;time_stamp&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which unsupervised anomaly detection algorithms or models should I use to solve this task?&lt;/p&gt;&#xA;" OwnerUserId="23174" LastEditorUserId="2444" LastEditDate="2019-03-14T10:47:21.273" LastActivityDate="2019-03-14T10:47:21.273" Title="Which unsupervised anomaly detection algorithms are there?" Tags="&lt;unsupervised-learning&gt;&lt;anomaly-detection&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11219" PostTypeId="1" AcceptedAnswerId="11221" CreationDate="2019-03-14T05:49:49.220" Score="4" ViewCount="1618" Body="&lt;p&gt;I read top articles on Google Search about Deep Q-Learning:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://skymind.ai/wiki/deep-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://skymind.ai/wiki/deep-reinforcement-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning#Deep_Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Learning page on wikipedia&lt;/a&gt;*&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term &quot;Deep Q-Learning&quot; on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*&lt;sub&gt;it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning &lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2019-03-15T03:50:47.987" LastActivityDate="2019-11-05T16:26:10.043" Title="Do we have to use CNN for Deep Q Learning?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11220" PostTypeId="1" CreationDate="2019-03-14T06:37:24.773" Score="0" ViewCount="60" Body="&lt;p&gt;I am not new to AI and did some work for few months but completely new to text to audio. Yes I used text to audio tools a decade back... but I would like to know where exactly we stand in terms of text to audio today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already did some research, it seems like traditional way of text to audio is fading away and speech cloning seems to be emerging but my impression on this might be completely wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the current open source text-to-audio libraries?&lt;/p&gt;&#xA;" OwnerUserId="23176" LastEditorUserId="23176" LastEditDate="2019-03-14T12:17:04.083" LastActivityDate="2020-01-09T15:04:00.483" Title="What are the current open source text-to-audio libraries?" Tags="&lt;audio-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11221" PostTypeId="2" ParentId="11219" CreationDate="2019-03-14T06:47:16.953" Score="7" Body="&lt;p&gt;No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: &lt;a href=&quot;http://github.com/keon/deep-q-learning/blob/master/dqn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;github.com/keon/deep-q-learning/blob/master/dqn.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="2444" LastEditDate="2019-03-15T13:38:47.007" LastActivityDate="2019-03-15T13:38:47.007" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11224" PostTypeId="2" ParentId="11218" CreationDate="2019-03-14T09:56:03.567" Score="1" Body="&lt;p&gt;If you are OK to use python, thy novelty-detection with sklearn:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scikit-learn.org/stable/modules/outlier_detection.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/modules/outlier_detection.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23104" LastActivityDate="2019-03-14T09:56:03.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11225" PostTypeId="2" ParentId="11218" CreationDate="2019-03-14T09:56:20.197" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://numenta.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical Temporal Memory&lt;/a&gt; is a model &lt;a href=&quot;https://numenta.com/assets/pdf/whitepapers/Numenta%20White%20Paper%20-%20Science%20of%20Anomaly%20Detection.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;well suited for anomaly detection&lt;/a&gt;. It is also pretty interesting and different from currently typical Deep Learning models. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2019-03-14T09:56:20.197" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11226" PostTypeId="1" AcceptedAnswerId="20628" CreationDate="2019-03-14T10:20:51.617" Score="12" ViewCount="7377" Body="&lt;p&gt;What is non-Euclidean data?&lt;/p&gt;&#xA;&lt;p&gt;Here are some sub-questions&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Where does this type of data arise? I have come across this term in the context of geometric deep learning and graph neural networks.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apparently, graphs and manifolds are non-Euclidean data. Why exactly is that the case?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the difference between non-Euclidean and Euclidean data?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How would a dataset of non-Euclidean data look like?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-06T14:38:52.167" LastActivityDate="2020-11-06T14:38:52.167" Title="What is non-Euclidean data?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;definitions&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="11227" PostTypeId="1" AcceptedAnswerId="11240" CreationDate="2019-03-14T10:39:13.217" Score="1" ViewCount="40" Body="&lt;p&gt;I have features like state, city and location. Currently i am inserting this in respective tables in DB and tranform it using its primary key.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg. country,state,city = IN,Maharastra,Pune = 2,5,10 (primary key in DB)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;transforming it as 002005010&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this approach correct? If not suggest me correct one&lt;/p&gt;&#xA;" OwnerUserId="23174" LastActivityDate="2019-03-14T18:39:07.727" Title="transform Location data into int which will be used as input to ML model" Tags="&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11229" PostTypeId="1" CreationDate="2019-03-14T10:55:33.147" Score="3" ViewCount="63" Body="&lt;p&gt;I couldn't find GUI for precise &quot;artificial neural-network alike&quot; structures, which could supports neuron naming, synapse naming, import of external functions or code fragments and debugging. It would be ideal if synapses also could pass not only float values, but user-defined structures too. Optimization, GPU-computing is irrelevant (and probably impossible with such features).&#xA;Is such thing exists? I'm thinking about writing one myself, for my needs. And most probably I will...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do I need such features? For testing concept of construction of kind-of-neural logic programming. I'm sure there is also should exists programming languages with such paradigm, but I also don't know how to find them, google would mostly give me info about common artificial neural networks.&lt;/p&gt;&#xA;" OwnerUserId="23183" LastEditorUserId="23183" LastEditDate="2019-03-14T12:22:29.410" LastActivityDate="2019-03-14T12:22:29.410" Title="Is there any GUI for per-neuron editing" Tags="&lt;neural-networks&gt;&lt;programming-languages&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11231" PostTypeId="1" CreationDate="2019-03-14T12:52:07.787" Score="1" ViewCount="38" Body="&lt;p&gt;There are several version of DDQN floating around. Sutton gives one that is a simple symmetric random update of the two Q functions. I think other papers (Silver paper for example) use a kind of delayed and split update rule. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything systematic describing the properties of the bias corrections and their respective advantages?&lt;/p&gt;&#xA;" OwnerUserId="23001" LastEditorUserId="2444" LastEditDate="2019-03-14T14:59:00.233" LastActivityDate="2019-03-14T14:59:00.233" Title="Comparison and understanding of different version of DDQN?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11232" PostTypeId="1" CreationDate="2019-03-14T12:53:22.040" Score="0" ViewCount="21" Body="&lt;p&gt;How to Serve a Deep Q Network using Tensorflow Serving. I have built a Deep Q Network using Multilayer Perceptron. Is it possible to serve it?&lt;/p&gt;&#xA;" OwnerUserId="23185" LastActivityDate="2019-03-14T12:53:22.040" Title="How to serve a deep q network using tensorflow serving?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;q-learning&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11234" PostTypeId="1" CreationDate="2019-03-14T13:03:06.537" Score="2" ViewCount="109" Body="&lt;p&gt;I have a log file of the format,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Index, Date, Timestamp, Module, App, Context, Session, Verbosity level, Description&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The log file can be considered as a master log, which consists of individual logs from several modules constituting a distributed system. The individual log can be identified using the corresponding Module+App+Context tags. The verbosity level(Info, Warn, Error, …) and the descriptions(system generated + print statements added by developers) contain further information on the log events necessary for debugging. I need to perform an &lt;strong&gt;unsupervised anomaly detection&lt;/strong&gt; with the log file as input. The output should be the functionality and timestamp of the identified anomalies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the log is mostly textual, I plan to use NLP algorithm (Bag of words/TF-IDF) to convert the data into word vectors and then perform a generative learning method to identify the normal pattern. Can someone suggest if my approach in the right direction? Which headers of the log file would be relevant for the word-vector representation and further analysis?&lt;/p&gt;&#xA;" OwnerUserId="23161" LastActivityDate="2019-03-14T13:03:06.537" Title="How to perform unsupervised anomaly detection from log file with mostly textual data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;word2vec&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11235" PostTypeId="1" CreationDate="2019-03-14T14:15:54.450" Score="2" ViewCount="227" Body="&lt;p&gt;I am reading the BERT paper &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I look at the attention mechanism, I don't understand why in the BERT encoder we have an intermediate layer between the attention and neural network layers with a bigger output (&lt;span class=&quot;math-container&quot;&gt;$4*H$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is the hidden size).&#xA;Perhaps it is the layer normalization, but, by looking at the code, I'm not certain. &lt;/p&gt;&#xA;" OwnerUserId="23154" LastEditorUserId="2444" LastEditDate="2019-11-01T02:36:04.297" LastActivityDate="2019-11-01T02:36:04.297" Title="Why does the BERT encoder have an intermediate layer between the attention and neural network layers with a bigger output?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;papers&gt;&lt;attention&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11236" PostTypeId="1" CreationDate="2019-03-14T14:43:58.590" Score="0" ViewCount="89" Body="&lt;p&gt;Suppose we are using word2vec and have embeddings of individual words &lt;span class=&quot;math-container&quot;&gt;$w_1, \dots, w_{10}$&lt;/span&gt;. Let's say we wanted to analyze &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; grams or &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; grams.&lt;/p&gt;&#xA;&lt;p&gt;Why would adding all the possible embeddings, &lt;span class=&quot;math-container&quot;&gt;$\binom{10}{2}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\binom{10}{3}$&lt;/span&gt;, be &amp;quot;worse&amp;quot; than using 1D-convolutions?&lt;/p&gt;&#xA;" OwnerUserId="23120" LastEditorUserId="2444" LastEditDate="2020-12-04T10:20:03.043" LastActivityDate="2021-05-03T11:06:20.757" Title="Why would adding all the possible embeddings be &quot;worse&quot; than using 1D-convolutions?" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;word2vec&gt;&lt;1d-convolution&gt;&lt;n-gram&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11238" PostTypeId="2" ParentId="11202" CreationDate="2019-03-14T17:23:48.803" Score="2" Body="&lt;p&gt;Nope! Our number of coefficients will be driven by the vocabulary, and we'll use each of those 10K samples to estimate values for those coefficients - so, 'just'  100K samples.  However, word frequency in human languages follows a Zipf distribution =&gt; most of those words will be rare, seen in only a few samples (=&gt; won't even be able to determine whether they are useful or not, let alone get good values for coefficients).  For an application like this one, you would probably find most of the value from a few hundred words.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-03-14T17:23:48.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11239" PostTypeId="2" ParentId="6527" CreationDate="2019-03-14T17:47:45.863" Score="0" Body="&lt;p&gt;Your first stop should be &lt;a href=&quot;http://newsletter.ruder.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sebastian Ruder's NLP newsletter&lt;/a&gt;, a monthly publication that will keep you current with the cutting-edge research in this field - progress is so rapid that monthly is the right frequency.  Not saying the @FauChrisian approach is wrong, but it's not obviously connected to the research that I see.  The Stanford course that @quintumnia mentions is a good bet too - again, progress is so rapid that books (and most courses) can't keep up. Fast.ai is also highly recommended - the founder is very active in the field.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-03-14T17:47:45.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11240" PostTypeId="2" ParentId="11227" CreationDate="2019-03-14T18:39:07.727" Score="1" Body="&lt;p&gt;In order to accurately input location data into a machine learning model it really depends on what your goal is and what type of algorithm you are working with.  If you are working with a strictly numerical algorithm and your data seems to be spread far apart, it might be easier to convert your country-state-city location to a longitude, latitude feature where the exact value is the centroid location of the given city.  This &lt;a href=&quot;https://www.kaggle.com/camnugent/geospatial-feature-engineering-and-visualization&quot; rel=&quot;nofollow noreferrer&quot;&gt;kaggle post&lt;/a&gt; has a good writeup of how to run build features for geo-spatial data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you have only a small number of different locations per identifier (country, city, state) you could just represent them as seperate location classes and subclasses like you would for machine learning detection model.  You could think &quot;vehicle&quot; -&gt; &quot;bike&quot; -&gt; &quot;road-bike&quot; similarly to &quot;country&quot; -&gt; &quot;state&quot; -&gt; &quot;city&quot;.  In this case, you would want to look at hierachical methods for machine learning and see some ways they represent their data.  Although I would say this method is more frowned upon for larger datasets, for smaller datasets it might be a better option.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2019-03-14T18:39:07.727" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11243" PostTypeId="1" CreationDate="2019-03-14T20:47:32.873" Score="4" ViewCount="115" Body="&lt;p&gt;What is a generalized MDP?&lt;/p&gt;&#xA;&lt;p&gt;Here are a few sub-questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How is it different than a &amp;quot;regular&amp;quot; MDP?&lt;/li&gt;&#xA;&lt;li&gt;How does it generalize the notion of an MDP?&lt;/li&gt;&#xA;&lt;li&gt;Why do we need a generalized MDP?&lt;/li&gt;&#xA;&lt;li&gt;Do generalized MDPs have some practical usefulness or they are just theoretical tools?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-07T11:10:37.003" LastActivityDate="2021-01-07T11:10:37.003" Title="What is a generalized MDP?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;markov-decision-process&gt;&lt;generalized-mdp&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11244" PostTypeId="1" CreationDate="2019-03-14T20:50:46.340" Score="3" ViewCount="87" Body="&lt;p&gt;In certain reinforcement learning (RL) proofs, the operators involved are assumed to be &lt;em&gt;non-expansive&lt;/em&gt;. For example, on page 6 of the paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.71.1316&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalized Markov Decision Processes: Dynamic-programming and Reinforcement-learning Algorithms&lt;/a&gt; (1997), Csaba Szepesvari and Michael L. Littman state&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When &lt;span class=&quot;math-container&quot;&gt;$0 \leq \gamma &amp;lt; 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\otimes$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\oplus$&lt;/span&gt; are non-expansions, the generalized Bellman equations have a unique optimal solution, and therefore, the optimal value function is well defined.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On page 7 of the same paper, the authors say that max is non-expansive. Moreover, on page 33, the authors assume &lt;span class=&quot;math-container&quot;&gt;$\otimes$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\oplus$&lt;/span&gt; are non-expansions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a non-expansive operator? Why is the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; (and the &lt;span class=&quot;math-container&quot;&gt;$\min$&lt;/span&gt;), which is, for example, used in Q-learning, a non-expansive operator?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-15T14:49:32.393" LastActivityDate="2019-09-15T14:49:32.393" Title="Why is the max a non-expansive operator?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;math&gt;&lt;proofs&gt;&lt;papers&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11245" PostTypeId="2" ParentId="11236" CreationDate="2019-03-14T21:00:58.380" Score="0" Body="&lt;p&gt;N-grams are defined as sets of n contiguous words. We use n-grams because they are more useful than random combinations of words across the sentence. Intuitively, combinations of nearby words have more semantic meaning than combinations of distant words.&lt;/p&gt;&#xA;&lt;p&gt;Also, using all possible combinations of n embeddings would take much longer, especially since (1D) convolutions are such efficient operations.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="2444" LastEditDate="2020-12-04T10:22:25.773" LastActivityDate="2020-12-04T10:22:25.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11246" PostTypeId="1" AcceptedAnswerId="11251" CreationDate="2019-03-14T21:12:04.403" Score="5" ViewCount="2559" Body="&lt;p&gt;Apparently, we can solve an MDP (that is, we can find the optimal policy for a given MDP) using a linear programming formulation. What's the basic idea behind this approach? I think you should start by explaining the basic idea behind a linear programming formulation and which algorithms can be used to solve such constrained optimisation problems.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-06-09T20:56:36.353" Title="How can we use linear programming to solve an MDP?" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;markov-decision-process&gt;&lt;linear-programming&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11247" PostTypeId="5" CreationDate="2019-03-14T21:23:35.413" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-03-14T21:23:35.413" LastActivityDate="2019-03-14T21:23:35.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11248" PostTypeId="4" CreationDate="2019-03-14T21:23:35.413" Score="0" Body="For questions related to the linear programming optimisation technique used in the context of AI (e.g. in the context of RL, in order to solve an MDP)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-15T19:11:37.913" LastActivityDate="2019-03-15T19:11:37.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11249" PostTypeId="2" ParentId="11220" CreationDate="2019-03-14T21:26:31.437" Score="0" Body="&lt;p&gt;There is one by Mozilla called Deep Voice &#xA;And another python library called pocket-sphinx &lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2019-03-14T21:26:31.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11251" PostTypeId="2" ParentId="11246" CreationDate="2019-03-15T02:42:16.287" Score="2" Body="&lt;p&gt;This question seems to be addressed directly in &lt;a href=&quot;http://www.cs.cmu.edu/afs/cs/academic/class/15780-s16/www/slides/mdps.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the slides&lt;/a&gt; of the deck you linked to in the comment under your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assume you have a complete model of the MDP (transitions, rewards, etc.).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For any given state, we have the assumption that the state's true value is reflected by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^*(s) = r + \gamma \max_{a \in A}\sum_{s' \in S} P(s' | s,a) \cdot V^*(s')$$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That is, the true value of the state is the reward we accrue for being in it, plus the expected future rewards of acting optimally from now until infinitely far into the future, discounted by the factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, which captures the idea that reward in the future is less good than reward now.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In Linear Programming, we find the minimum or maximum value of some function, subject to a set of constraints. We can do this efficiently if the function can take on continuous values, but the problem becomes NP-Hard if the values are discrete. You would usually do this using something like the &lt;a href=&quot;http://web.tecnico.ulisboa.pt/mcasquilho/compute/_linpro/TaylorB_module_c.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Branch &amp;amp; Bound algorithm&lt;/a&gt;. These are widely available in fast implementations. GLPK is a decent free library. IBM's CPLEX is faster, but expensive.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We can represent the problem of finding the value of a given state as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{minimize}_V \ V(s)$$&lt;/span&gt;&#xA;subject to the constraints:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \geq r + \gamma\sum_{s' \in S} P(s' | s,a)*V(s'),\; \forall a\in A, s \in S$$&lt;/span&gt;&#xA;It should be apparent that if we find the smallest value of &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; that matches this requirement, then that value would make exactly one of the constraints tight.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you formulate your linear program by writing a program like the one above for &lt;em&gt;every&lt;/em&gt; state and then minimize &lt;span class=&quot;math-container&quot;&gt;$\sum_{s\in S} V(s)$&lt;/span&gt;, subject to the union of all the constraints from all these sub-problems you have reduced the problem of learning a value function to solving the LP.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-06-09T20:56:36.353" LastActivityDate="2020-06-09T20:56:36.353" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11252" PostTypeId="2" ParentId="11166" CreationDate="2019-03-15T09:15:07.867" Score="17" Body="&lt;p&gt;To complete the &lt;a href=&quot;https://ai.stackexchange.com/a/11201/22176&quot;&gt;first answer&lt;/a&gt; that is rather graph oriented, I will write a little about deep learning on manifolds, which is quite general in terms of GDL thanks to the nature of manifolds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the description of GDL through the explanation of what are DL on graphs and manifolds, in opposition to DL on euclidean domains, comes from the 2017 paper &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;noreferrer&quot;&gt;Geometric deep learning: going beyond Euclidean data&lt;/a&gt; (this paper is excellent at clarifying both the intuition and the mathematics of what I'm writing).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1. In case you don't know what a manifold is&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As the previously cited paper puts it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Roughly, a manifold is a space that is locally Euclidean. One of the&#xA;  simplest examples is a spherical surface modeling our planet: around a&#xA;  point, it seems to be planar, which has led generations of people to&#xA;  believe in the flatness of the Earth. Formally speaking, a&#xA;  (differentiable) d-dimensional manifold X is a topological space where&#xA;  each point x has a neighborhood that is topologically equivalent&#xA;  (homeomorphic) to a d-dimensional Euclidean space, called the tangent&#xA;  space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Good other not-so-technical &lt;a href=&quot;https://stats.stackexchange.com/a/289488/241224&quot;&gt;explanation&lt;/a&gt; on stats.stackexchange&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other &lt;a href=&quot;https://en.wikipedia.org/wiki/Manifold#Motivating_examples&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia examples&lt;/a&gt; to develop not too abstract understanding&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very shortly put, it's an interesting mathematical set on which to work (different kinds exist, see papers at the end of this answer for DL related manifolds uses). By work, you can typically understand that you &lt;em&gt;constrain the neural net parameters to the manifold&lt;/em&gt; you chose (e.g. training with parameters constrained on a hypersphere, among the &lt;a href=&quot;https://arxiv.org/abs/1805.08308&quot; rel=&quot;noreferrer&quot;&gt;geomstats paper&lt;/a&gt; examples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your data can also be represented thanks to a practical manifold. For example, you can choose to work on images and videos by representing the samples using Symmetric Positive Definite (SPD) matrices (see this &lt;a href=&quot;https://arxiv.org/abs/1605.06182&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;), the space of SPD matrices being a manifold itself.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2. Why bother learning on manifolds ?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Defining a clearer/better adapted set (understand that it's a sort of constraint!) on which to learn parameters and features can make it simpler to formally understand what your model is doing, and can lead to better results. I see it as a part of the effort of deep learning formalization. One could say &lt;strong&gt;you're looking for the best information geometry for your task&lt;/strong&gt;, the one that best captures the desirable data distribution properties.To develop this intuition consider the &lt;a href=&quot;https://www.kaggle.com/apapiu/manifold-learning-and-autoencoders&quot; rel=&quot;noreferrer&quot;&gt;solar system analogy for manifold learning of this Kaggle kernel&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Perhaps a good analogy here is that of a solar system: the surface of our planets are the manifolds we're interested in, one for each digit. Now say you're on the surface of the earth which is a 2-manifold and you start moving in a random direction (let's assume gravity doesn't exist and you can go through solid objects). If you don't understand the structure of earth you'll quickly find yourself in space or inside the earth. But if you instead move within the local earth (say spherical) coordinates you will stay on the surface and get to see all the cool stuff.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This analogy reminds us of the spherical surface planet model from &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;noreferrer&quot;&gt;Bronstein's paper&lt;/a&gt; already quoted above. This paper also describes a typical case for which manifolds are interesting: where graphs (the other example of GDL/DL on non euclidean data) are better at handling data from social or sensor networks, manifolds are good at modeling 3D objects endowed with properties like color texture in computer vision.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;3. Regarding deep neural networks on manifolds&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I would advise reading the &lt;a href=&quot;https://github.com/geomstats/geomstats&quot; rel=&quot;noreferrer&quot;&gt;geomstats&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/abs/1805.08308&quot; rel=&quot;noreferrer&quot;&gt;associated paper&lt;/a&gt;, which does a great job at showing what it is and how it can be used, along with example codes (e.g. MNIST on hyperspheres manifold example code &lt;a href=&quot;https://github.com/geomstats/applications/blob/master/deep_learning/mnist_hypersphere.py&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;). This library implements manifolds and associated metrics on Keras. The choice of metrics is essential to understand the point of working on manifolds: it's because you need to work on an adapted mathematical set (&lt;em&gt;ie&lt;/em&gt; with the right properties) with an adapted distance definition (&lt;em&gt;so that the measure actually means something when considering the problem you're trying to solve&lt;/em&gt;) that you switch to working on manifolds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to dive in the details and examples of deep learning on manifolds here are some papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1608.04233&quot; rel=&quot;noreferrer&quot;&gt;A Riemannian Network for SPD Matrix Learning&lt;/a&gt;: new backpropagation to learn SPD matrices on Riemannian manifolds&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1711.06540&quot; rel=&quot;noreferrer&quot;&gt;Learning a Robust Representation via a Deep Network on Symmetric Positive Definite Manifolds&lt;/a&gt;: using SPD matrices to aggregate convolutional features&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;4. Why &lt;em&gt;Riemannian&lt;/em&gt; manifolds ?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;TL;DR: you need a metric to do machine learning&lt;/em&gt; (otherwise, how could you evaluate how much you actually learned !)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still based on &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;noreferrer&quot;&gt;Bronstein's paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On each tangent space, we define an inner product [...]. This inner&#xA;  product is called a Riemannian metric in differential geometry and&#xA;  allows performing local measurements of angles, distances, and&#xA;  volumes. A manifold equipped with a metric is called a Riemannian&#xA;  manifold.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;5. What's the relation between a Riemannian manifold and a Euclidean space ?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Still based on &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;noreferrer&quot;&gt;Bronstein's paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a Riemannian manifold can be realized as a subset of a Euclidean space&#xA;  (in which case it is said to be &lt;em&gt;embedded&lt;/em&gt; in that space) by using the&#xA;  structure of the Euclidean space to induce a Riemannian metric.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I leave the details to the paper, otherwise this answer will never end.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;6. Answers to questions in comments&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Will only answer once I think I've found a relatively well-argued answer, so won't answer everything at once.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Isn't manifold learning just a way of dimensionality reduction? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't think so, it isn't &lt;em&gt;just that&lt;/em&gt;. I haven't seen any dimensional reduction constraint (yet ?) in the papers I've read (cf. geomstats again). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://github.com/geomstats/applications/blob/master/deep_learning/mnist_hypersphere.py&quot; rel=&quot;noreferrer&quot;&gt;hypersphere/MNIST geomstats code example&lt;/a&gt;, you can see the chosen manifold dimension &lt;code&gt;hypersphere_dimension = 17&lt;/code&gt;. Since we're working with MNIST data I guess this would mean a dimension reduction in this particular case. I admit I would need to check exactly what that dimension implies on the neural net architecture, I haven't discussed my understanding of this yet.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Disclaimer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm still developing a more rigorous mathematical understanding of manifolds, and shall update this post to make additional necessary clarifications: exactly what can be considered as a manifold in a traditional deep learning context, why do we use the word manifold when speaking about the hidden state of auto-encoders (see the previously cited Kaggle kernel that quotes Goodfellow's book on this). All of this if the perfectly clear answer doesn't show up here before !&lt;/p&gt;&#xA;" OwnerUserId="22176" LastEditorUserId="22176" LastEditDate="2019-03-19T08:41:01.907" LastActivityDate="2019-03-19T08:41:01.907" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11253" PostTypeId="2" ParentId="5230" CreationDate="2019-03-15T09:52:07.880" Score="1" Body="&lt;p&gt;Knime is my favorite - it is the Gartner Leader Quadrant - available for free.  Also consider Rattle (for R) and Gretl for econometrics/stats.  These are good tools for the Citizen Data Scientist.&lt;/p&gt;&#xA;" OwnerUserId="23209" LastActivityDate="2019-03-15T09:52:07.880" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11257" PostTypeId="2" ParentId="10545" CreationDate="2019-03-15T14:31:14.633" Score="1" Body="&lt;p&gt;Adam reduces the learning rate over time. When you change to the new training data, you want to reset the learning rate. But Adam might not be the best choice for the second round of training - it can make big changes to the inherited weights, which prevents the transfer of previous learning. It can be good to switch to simple SGD for the second round.&lt;/p&gt;&#xA;" OwnerUserId="12269" LastActivityDate="2019-03-15T14:31:14.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11258" PostTypeId="2" ParentId="11244" CreationDate="2019-03-15T14:49:37.053" Score="2" Body="&lt;p&gt;In laymen's terms, a non-expansive operator is a function that brings points closer together or at least no further apart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a non-expansive operator is the function &lt;span class=&quot;math-container&quot;&gt;$f(x) = x/2$&lt;/span&gt;. The two numbers &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; are a distance of &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; apart. The two output numbers &lt;span class=&quot;math-container&quot;&gt;$f(0) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(5) = 2.5$&lt;/span&gt; are 2.5 apart (which is smaller than &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; apart). It is easy to see that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; brings everything closer together except when the two input numbers are the same: in which case, the distance between the outputs of the function at those numbers is at least no further apart than distance between the two input numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt;  is a two-input function (or n-input, but the intuition should be clear from the 2-input case).  We can think of max as a function that maps &lt;em&gt;pairs&lt;/em&gt; of numbers &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; to single numbers (picking whichever of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is larger).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that we chose to measure the distance between pairs using &lt;a href=&quot;https://en.wikipedia.org/wiki/Euclidean_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euclidean distance&lt;/a&gt;, and the distance between single numbers using the Euclidean distance as well. Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distance between (0,0) and (3,3) is &lt;span class=&quot;math-container&quot;&gt;$\sqrt{3^2 + 3^2} = \sqrt{18}$&lt;/span&gt;. The distance between &lt;span class=&quot;math-container&quot;&gt;$\max(0,0) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\max(3, 3) = 3$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\sqrt{{(\max(0,0) - \max(3, 3))}^2} = \sqrt{9} = 3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider the general case. The Euclidean distance between the 2D points &lt;span class=&quot;math-container&quot;&gt;$(a, b)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(c, d)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\sqrt{(a-c)^2 + (b-d)^2}$&lt;/span&gt;. There are four cases to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Suppose that &lt;span class=&quot;math-container&quot;&gt;$a\geq b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c \geq d$&lt;/span&gt;. In this case, the distance between max(a,b) and max(c,d) is just |a-c|, which is clearly &lt;em&gt;at most&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\sqrt{(a-c)^2 + (b-d)^2}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Suppose that &lt;span class=&quot;math-container&quot;&gt;$a\leq b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c \leq d$&lt;/span&gt;. In this case, the distance is |b-d|, which is also at most the original distance.&lt;/li&gt;&#xA;&lt;li&gt;Suppose that &lt;span class=&quot;math-container&quot;&gt;$a\geq b$&lt;/span&gt; but &lt;span class=&quot;math-container&quot;&gt;$c \leq d$&lt;/span&gt;. Then the distance is |a-d|. Suppose that &lt;span class=&quot;math-container&quot;&gt;$a &amp;gt; d$&lt;/span&gt;. Since &lt;span class=&quot;math-container&quot;&gt;$d &amp;gt; c$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$|a-d| &amp;lt;= \sqrt{(a-c)^2 + (b-d)^2}$&lt;/span&gt; since |a-d|&amp;lt;=&lt;span class=&quot;math-container&quot;&gt;$\sqrt{(a-c)^2}$&lt;/span&gt;, and a symmetric argument holds for the case &lt;span class=&quot;math-container&quot;&gt;$d &amp;gt; a$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a\leq b$&lt;/span&gt; but &lt;span class=&quot;math-container&quot;&gt;$c \geq d$&lt;/span&gt;, we can construct an argument identical to the one for case 3 above.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since max is always bringing things closer together, or at least, no further apart, it is a non-expansive operator.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2019-03-15T15:53:00.423" LastActivityDate="2019-03-15T15:53:00.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11259" PostTypeId="2" ParentId="11226" CreationDate="2019-03-15T16:14:41.980" Score="9" Body="&lt;p&gt;Non-Euclidian geometry can be generally boiled down to the phrase&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the shortest path between 2 points isn't necessarily a straight line.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Or, put in a way that lends itself very much to machine learning,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;things that are similar to each other are not necessarily close if one uses Euclidean distance as a metric (aka the &lt;a href=&quot;https://en.wikipedia.org/wiki/Triangle_inequality&quot; rel=&quot;nofollow noreferrer&quot;&gt;triangle inequality&lt;/a&gt; doesn't hold).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You mention graphs and manifolds as being non-Euclidian, but, really, the majority of problems being worked on don't have Euclidian data. Take the below images for example:&lt;/p&gt;&#xA;&lt;p&gt;Clearly, 2 of the images are more similar to each other than the third one is, but if we looked at the pixels alone, the Euclidean distance between the pixel values don't represent this similarity.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P6mH2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P6mH2.png&quot; alt=&quot;2 good boys and a rad hampster&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If there was a function, &lt;span class=&quot;math-container&quot;&gt;$F(\text{image})$&lt;/span&gt;, that mapped images to a space of values where similar images produced values that were closer together, we could better understand the data, infer some statistics about the distributions, and make predictions on data we have yet to see. This is what classic techniques of image recognition have done and it's also what modern machine learning is doing. Taking data and mapping it to a space such that the &lt;a href=&quot;https://en.wikipedia.org/wiki/Triangle_inequality&quot; rel=&quot;nofollow noreferrer&quot;&gt;triangle inequality&lt;/a&gt; holds.&lt;/p&gt;&#xA;&lt;p&gt;Let's look at a more concrete example, some points I drew in MSPaint.&#xA;On the left is some space that we are interested in where points have 2 classes (red or blue). Even though there are points that are close to each other, they may have different colors/classes. Ideally, we could have a function that converts these points to some space where we can draw a line to separate these 2 classes. In general, there would be many lines, or hyper-planes in dimensions &amp;gt; 3, but the goal is to transform the data so that it will be &amp;quot;linearly separable&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S81eT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S81eT.png&quot; alt=&quot;Some points I drew in MSPaint.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To conclude, non-Euclidian data is everywhere.&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-07-10T11:50:16.627" LastActivityDate="2020-07-10T11:50:16.627" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11261" PostTypeId="1" AcceptedAnswerId="11263" CreationDate="2019-03-15T17:44:50.580" Score="2" ViewCount="288" Body="&lt;p&gt;Two words can be similar if they co-occur &quot;a lot&quot; together. They can also be similar if they have similar vectors. This similarity can be captured using cosine similarity. Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; be a &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; matrix counting how often &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; occurs with &lt;span class=&quot;math-container&quot;&gt;$w_k$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i,k = 1, \dots, n$&lt;/span&gt;. Since computing the cosine similarity between &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_k$&lt;/span&gt; might be expensive, we approximate &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; using truncated SVD with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; components as: &lt;span class=&quot;math-container&quot;&gt;$$A \approx W_k \Sigma W^{T}_{k} = CD$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$$C = W_{k} \Sigma \\ D = W^{T}_{k}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where are the cosine similarities between the words &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_k$&lt;/span&gt; captured? In the &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; matrix or the &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; matrix?&lt;/p&gt;&#xA;" OwnerUserId="23220" LastEditorUserId="2444" LastEditDate="2019-03-15T18:06:55.537" LastActivityDate="2021-01-03T20:30:01.217" Title="Which matrix represents the similarity between words when using SVD?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11263" PostTypeId="2" ParentId="11261" CreationDate="2019-03-16T00:17:06.180" Score="0" Body="&lt;p&gt;You can find some material &lt;a href=&quot;https://scikit-learn.org/stable/modules/decomposition.html#truncated-singular-value-decomposition-and-latent-semantic-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/0909.4061.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; but the idea (at least in this case) is the following: consider the full SVD decomposition of the symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$A = W \Delta W^T$&lt;/span&gt;. We want to calculate the cosine similarity between the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th column (aka word) &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;-th column &lt;span class=&quot;math-container&quot;&gt;$a_j$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$a_k = A e_k$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$e_k$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-th vector of the canonical basis of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;. Let's call &lt;span class=&quot;math-container&quot;&gt;$\cos(a_i,a_j)$&lt;/span&gt; the cosine between &lt;span class=&quot;math-container&quot;&gt;$a_i,a_j$&lt;/span&gt;. Then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\cos(a_i,a_j) = \cos(Ae_i,Ae_j) = \cos(W \Delta W^T e_i,W \Delta W^T e_j) = \cos(\Delta W^T e_i,\Delta W^T e_j)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the last equality holds because &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is an orthogonal matrix (and so &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is &lt;a href=&quot;https://en.wikipedia.org/wiki/Conformal_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;conformal&lt;/a&gt;, i.e. it preserves angles). So you can calculate the cosine similarity between the columns of &lt;span class=&quot;math-container&quot;&gt;$\Delta W^T$&lt;/span&gt;. A &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-truncated SVD gives a well-enough approximation. In &lt;a href=&quot;https://stats.stackexchange.com/q/134282/82135&quot;&gt;general&lt;/a&gt;, columns of &lt;span class=&quot;math-container&quot;&gt;$W \Delta$&lt;/span&gt; and rows of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; have different meanings!&lt;/p&gt;&#xA;" OwnerUserId="23224" LastEditorUserId="2444" LastEditDate="2021-01-03T20:30:01.217" LastActivityDate="2021-01-03T20:30:01.217" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11266" PostTypeId="1" CreationDate="2019-03-16T08:14:47.073" Score="2" ViewCount="119" Body="&lt;p&gt;I don't know anything about neural networks, but I got the information that making a chatbot with a neural network is very good. Is this really true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do I need to know in order to build a chat bot using neural networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have understood a little about text preprocessing (such as tokenization, stemming, stop word lists, named entity recognition), but when I want to try classification using the neural network I am confused.&lt;/p&gt;&#xA;" OwnerUserId="22686" LastEditorUserId="2444" LastEditDate="2019-04-15T20:25:44.523" LastActivityDate="2020-01-10T21:03:20.437" Title="How do I use neural networks to implement a chatbot?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="9" ClosedDate="2021-01-26T21:03:19.180" ContentLicense="CC BY-SA 4.0" />
  <row Id="11275" PostTypeId="2" ParentId="10540" CreationDate="2019-03-16T22:08:59.450" Score="0" Body="&lt;p&gt;If you can classify words then you can easily classify sentences. One of interesting problems you can solve then is »what are allowed sentence forms?« How can you classify words? By searching for features that are common between them. These features are all possible truths that are true for a word, like with what other words it appears in same sentence, what is their spatial relation, what is frequency of appearance.&lt;/p&gt;&#xA;" OwnerUserId="12251" LastActivityDate="2019-03-16T22:08:59.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11277" PostTypeId="1" CreationDate="2019-03-16T22:24:45.287" Score="2" ViewCount="158" Body="&lt;p&gt;I was reading about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbol_grounding_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;grounding problem&lt;/a&gt; after seeing it mentioned in another answer today. The article states that, in order to avoid the &quot;infinite regress&quot; of defining all words with other words, we must ground the meaning of some words in the &quot;sensorimotor.&quot; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To be grounded, the symbol system would have to be augmented with nonsymbolic, sensorimotor capacities—the capacity to interact autonomously with that world of objects, events, actions, properties and states that its symbols are systematically interpretable (by us) as referring to.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Obviously, this made me think of Reinforcement Learning. But I'm not exactly sure what counts as &quot;interaction.&quot; Would this necessarily imply an MDP-like formulation with rewards, state transitions, etc? Or could some form of grounding be accomplished with supervised learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems like a pretty fundamental problem of AI. Does anyone know of research being done on grounding words/symbols within an RL agent?&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-17T10:12:25.127" LastActivityDate="2019-03-17T17:43:38.947" Title="Is Reinforcement Learning the future of Natural Language Processing?" Tags="&lt;reinforcement-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11279" PostTypeId="1" CreationDate="2019-03-17T01:21:18.437" Score="0" ViewCount="72" Body="&lt;p&gt;The last week I've been looking for freelancers who are able to do this project for me but they weren't that experienced in it, so I would like to know whether my idea is complicated or is it their lake of experience.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; The facial recognition system will be installed on a vertical screen where a camera would be attached to it and it will be assigned on the entrance of the room.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Once a visitor come to the entrance, and looks at the screen, a text on the screen would say: &amp;quot; Welcome! It seems like it's your first visit! Please enter your name then a keyboard should popup&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; The visitor would enter his first name into the keyboard, and it would be saved alongside his face in the database, and it would say thank you, {name}.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;4.&lt;/strong&gt; If the same visitor visits again, the system should say: &amp;quot; Welcome back {name}, happy to see you again&amp;quot;!&lt;/p&gt;&#xA;" OwnerUserId="23252" LastEditorUserId="32410" LastEditDate="2021-01-19T16:19:23.953" LastActivityDate="2021-01-19T16:19:23.953" Title="Facial Recognition + Database + Compare &amp; Identify - is it complicated?" Tags="&lt;machine-learning&gt;&lt;facial-recognition&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2021-01-20T11:28:05.020" ContentLicense="CC BY-SA 4.0" />
  <row Id="11280" PostTypeId="1" AcceptedAnswerId="11282" CreationDate="2019-03-17T08:56:00.903" Score="0" ViewCount="63" Body="&lt;p&gt;I'm implementing PPO myself strictly follow the steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;sample transitions&lt;/li&gt;&#xA;&lt;li&gt;randomly shuffle the sampled transitions&lt;/li&gt;&#xA;&lt;li&gt;compute gradients and update networks using the sampled transitions&lt;/li&gt;&#xA;&lt;li&gt;drop transitions and repeat the above steps &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I observe a strange phenomenon that randomly shuffling transitions makes the algorithm perform significantly worse than keeping it as it is. This is very strange to me. To my best understanding, neural networks perform badly when the input data are correlated. To decorrelate transitions, algorithms like DQN introduce replay buffer and randomly sample from it. But this seems not the same story to policy-based methods. I'm wondering why policy-based methods do not require to decorrelate the input data?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastActivityDate="2019-03-17T10:10:38.300" Title="Why don't we decorrelate transitions for policy-based data?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11282" PostTypeId="2" ParentId="11280" CreationDate="2019-03-17T09:56:09.450" Score="3" Body="&lt;p&gt;We do decorrelate training experience, even for policy gradient methods. This is because decorrelation helps training data be more like IID data, which helps with the convergence of SGD-like optimizers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shuffling is done on line 151 of OpenAI's &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/ppo2/ppo2.py#L151&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;baselines&quot; implementation of PPO&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to guess that there's a bug somewhere in your implementation. If you're using the same truncated advantage estimation as in the &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO paper&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;&amp;amp;\hat{A}_t = \delta_t + (\gamma\lambda)\delta_{t+1}+\dots+(\gamma\lambda)^{T-t+1}\delta_{T-1}\\&#xA;&amp;amp;\text{where}\quad\delta_t=r_t+\gamma V(s_{t+1})-V(s_t)&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;make sure you're not shuffling experience before computing your advantage estimates &lt;span class=&quot;math-container&quot;&gt;$\hat{A}_1,\dots,\hat{A}_T$&lt;/span&gt;. Each of these estimates is a function of several unbroken steps of experience. After you compute your advantage estimates, then create the tuples &lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, \hat{A}_t)$&lt;/span&gt; and shuffle and sample from those. I think that's all the information you need  per time step for constructing their objective function.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-17T10:10:38.300" LastActivityDate="2019-03-17T10:10:38.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11284" PostTypeId="1" CreationDate="2019-03-17T11:18:03.417" Score="0" ViewCount="40" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Floorball&quot; rel=&quot;nofollow noreferrer&quot;&gt;Floorball&lt;/a&gt; is a type of floor hockey. During the game, substitutions can be made.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The team is also allowed to change players any time in the game; usually, they change the whole team. Individual substitution happens sometimes, but it usually happens when a player is exhausted or is hurt. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would like to use an RNN to predict when the next substitution will happen for a team. However, I have no pre-existing dataset to train on. Is there a way that I can start predicting without a dataset and continually improve accuracy as more games are played?&lt;/p&gt;&#xA;" OwnerUserId="23258" LastEditorUserId="22916" LastEditDate="2019-03-18T16:45:56.873" LastActivityDate="2019-03-18T16:45:56.873" Title="Is there a RNN that can predict the next substitute in a floorball match?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11285" PostTypeId="1" AcceptedAnswerId="20646" CreationDate="2019-03-17T11:56:36.493" Score="19" ViewCount="5546" Body="&lt;p&gt;In general, the word &amp;quot;latent&amp;quot; means &amp;quot;hidden&amp;quot; and &amp;quot;to embed&amp;quot; means &amp;quot;to incorporate&amp;quot;. In machine learning, the expressions &amp;quot;hidden (or latent) space&amp;quot; and &amp;quot;embedding space&amp;quot; occur in several contexts. More specifically, an &lt;a href=&quot;https://en.wikipedia.org/wiki/Word_embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;embedding&lt;/a&gt; can refer to a vector representation of a word. An &lt;a href=&quot;https://en.wikipedia.org/wiki/Embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;embedding space&lt;/a&gt; can refer to a subspace of a bigger space, so we say that the subspace is &lt;em&gt;embedded&lt;/em&gt; in the bigger space. The word &amp;quot;latent&amp;quot; comes up in contexts like &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;hidden Markov models (HMMs)&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;auto-encoders&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What is the difference between these spaces? In some contexts, do these two expressions refer to the same concept?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T13:17:41.663" LastActivityDate="2020-12-31T17:43:22.203" Title="What is the difference between latent and embedding spaces?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;word-embedding&gt;&lt;latent-variable&gt;&lt;embeddings&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11286" PostTypeId="1" AcceptedAnswerId="11291" CreationDate="2019-03-17T12:25:06.293" Score="1" ViewCount="356" Body="&lt;p&gt;So I am simulating a Tic Tac Toe game with a human opponent. The way the RL trains is through policy/value iterations for a fixed number of iterations all specified by the user. Now whether the human player has turn 1 or turn 2 will decide the starting state (1st move by human or empty). Now the starting states for the 1st case can differ as the human can make 9 different moves. So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In Tic Tac Toe what is the effect of starting state in the State and Action Value function? &lt;/li&gt;&#xA;&lt;li&gt;Does it converge to the same stable value for all starting states?&lt;/li&gt;&#xA;&lt;li&gt;Will the value functions change if the starting players are changed? (human vs RL to RL vs human)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; I will be enumerating all states since there are approximately 20000 states which I believe is not a big number and thus convergence should not be a problem.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-03-17T16:23:18.053" LastActivityDate="2019-03-17T17:39:29.453" Title="Importance of starting state and player in RL for Tic Tac Toe" Tags="&lt;reinforcement-learning&gt;&lt;gaming&gt;" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="11287" PostTypeId="2" ParentId="11285" CreationDate="2019-03-17T12:50:06.820" Score="6" Body="&lt;p&gt;When it comes to normal layman terms &quot;latent space&quot; means it cannot be accessed, thus we have no direct control over it. We can only manipulate it indirectly, while &quot;Embeddings&quot; can be obtained directly. We can use deterministic operations or transformations to convert an object into its corresponding embedding space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no marked difference between these 2 terms as far as Machine Learning is concerned. If we look at this famous &lt;a href=&quot;https://arxiv.org/pdf/1406.5298.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on Variational Autoencoders, we can see the words has been used interchangeably. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, I would consider the word (in the context of Machine Learning only) &lt;strong&gt;latent&lt;/strong&gt; as a more general term than &lt;strong&gt;Embedding&lt;/strong&gt;. Embeddings will refer to a more specific object (in context of ML), for example the embedding of &lt;span class=&quot;math-container&quot;&gt;$word_1$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$embedding_1$&lt;/span&gt;. Whereas, we can use the term latent to describe broader terms like &lt;strong&gt;latent space&lt;/strong&gt;, &lt;strong&gt;latent representation&lt;/strong&gt;, &lt;strong&gt;latent variables&lt;/strong&gt; (latent variables of a word is same as an embedding of a word).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After digging some more I found some what of a formal definition of Latent Variables in &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; by Goodfellow:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Latent Variables&lt;/strong&gt; - A latent variable is a random&#xA;variable that we cannot observe directly. The component identity variable &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; of the&#xA;mixture model provides an example. Latent variables may be related to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; through&#xA;the joint distribution, in this case, &lt;span class=&quot;math-container&quot;&gt;$P(x, c) = P(x | c)P(c)$&lt;/span&gt;. The distribution &lt;span class=&quot;math-container&quot;&gt;$P(c)$&lt;/span&gt;&#xA;over the latent variable and the distribution &lt;span class=&quot;math-container&quot;&gt;$P(x | c)$&lt;/span&gt; relating the latent variables&#xA;to the visible variables determines the shape of the distribution &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt;, even though&#xA;it is possible to describe &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; without reference to the latent variable. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also a &lt;a href=&quot;https://www.utc.fr/~bordesan/dokuwiki/_media/en/transe_nips13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; cited by Goodfellow while discussing embeddings has the following excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Following the success of user/item clustering or matrix factorization techniques in collaborative filtering to represent non-trivial similarities between the connectivity patterns of entities in single relational data, most existing methods for multi-relational data have been designed within the framework of relational learning from latent attributes, as pointed out by; that is, by learning and&#xA;  operating on latent representations (or embeddings) of the constituents (entities and relationships).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So clearly these are somewhat interchangeable terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my interpretation would be that embeddings are helpful more explicitly (more visible, latent variables are meant to be hidden), that is we can construct a new data-set from it and use various ML methods on it, whereas latent variables are something not useful explicitly (it is a part of a bigger problem we are trying to solve). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: In the context of HMM's the term better suitable is hidden state and not latent space. Thus, in a HMM (from Wiki) The adjective hidden refers to the state sequence through which the model passes, not to the parameters of the model; the model is still referred to as a hidden Markov model even if these parameters are known exactly.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-05-24T12:19:18.560" LastActivityDate="2019-05-24T12:19:18.560" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11289" PostTypeId="2" ParentId="11279" CreationDate="2019-03-17T15:08:22.760" Score="1" Body="&lt;p&gt;Depending on the level of accuracy/speed you want your system to have, you could try using an &lt;a href=&quot;https://github.com/ageitgey/face_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;existing python AP&lt;/a&gt;I. First thing I found on google, the README looks adapted to your project. Otherwise ask your question with more accurate wanted specifications perhaps.&lt;/p&gt;&#xA;" OwnerUserId="22176" LastActivityDate="2019-03-17T15:08:22.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11290" PostTypeId="1" CreationDate="2019-03-17T15:47:30.717" Score="4" ViewCount="61" Body="&lt;p&gt;I am developing an algorithm that, in certain moment, must explore an exponential number of objects derived from a graph:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for o in my_graph.getDerivedObjects():&#xA;  if hasPropertyX(o):&#xA;    process(o)&#xA;    break;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If one of the derived objects has property &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, then the algorithm process it and then stops. The theory ensures that at least one of these derived objects has property &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. Now, I strongly suspect that there is a strong correlation between some topological aspects of the graph, and which derived objects actually have property &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. I want to &lt;em&gt;predict&lt;/em&gt; some of the derived objects that have property &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; using Machine Learning. So, the idea is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Predict a derived object &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; that supposedly has property &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; - or maybe predict &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of them for some number &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If any of them is useful, I use them. If not, I run the exponential algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of course, this isn't an optimization in the worst-case complexity of the algorithm. Also, I suppose I should also develop some statistical tests in order to show that the prediction algorithm actually works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this type of optimizations common? Could you please provide some examples? The literature on the subject would also be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="22365" LastEditorUserId="2444" LastEditDate="2019-08-18T22:26:10.777" LastActivityDate="2019-08-18T22:26:10.777" Title="Can machine learning be used to improve the average case complexity of an algorithm?" Tags="&lt;algorithm&gt;&lt;applications&gt;&lt;time-complexity&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11291" PostTypeId="2" ParentId="11286" CreationDate="2019-03-17T16:48:23.517" Score="1" Body="&lt;p&gt;My understanding - from comments on the question - is that you are looking to train a Reinforcement Learning agent on the game of Tic Tac Toe (perhaps just in theory), where the agent should learn to play against a &quot;human&quot; opponent. In practice you may want a model of a human opponent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the RL agent will be presented with a board state, it will take an action (to put its mark on an empty place in the grid) and either:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Win immediately, receiving a positive reward. It is common to use +1 in a game like Tic Tac Toe, so I will assume that later.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lose on opponent's turn, receiving a negative reward (assumed -1 later in the answer), as opponent makes a move that causes it to win. This is effectively &quot;immediately&quot; in terms of time steps, the agent does not get to act afterwards. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Receive zero reward, and a new board state that includes the opponent's move&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Receive zero reward, and the game ends in a draw&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In all cases, the opponent is considered part of the environment. That makes the opponent behaviour &lt;em&gt;critical&lt;/em&gt; to the value function and choice of optimal play. Training versus different opponents can result in very different state values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For training to be stable, the opponent &lt;em&gt;should&lt;/em&gt; behave with the same probability of action choices for each interim state that it observes. That includes it behaving deterministically, even optimally, purely randomly or anything in-between provided the probability distribution is fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the above context, it is possible to give sound answers to your questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In Tic Tac Toe what is the effect of starting state in the State and Action Value function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Each state of the board, or each state/action pair if you want to track action values, should converge to a value, depending on the agent's &lt;em&gt;estimated, expected&lt;/em&gt; result at the end of the episode. As there is only a single reward possible at the end of each game, this will vary between -1 and +1. If either the agent or the opponent can make mistakes at random, then non-zero values between -1 and +1 are possible. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it converge to the same stable value for all starting states?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That depends on behaviour of the opponent. In the scenario you are working with, the agent may &lt;em&gt;not&lt;/em&gt; learn to play optimally in general, instead it will learn to gain optimal results against the supplied opponent. If the opponent can make mistakes, then moves which take advantage of that fact will have higher values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without a detailed description of the opponent it is not possible to make many statements about the actual state values and action values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Against a perfect opponent, with the RL going second, it should converge to state values which are all zero and action values which are all zero or -1 for moves which would be mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Against a completely random opponent, I would have to run it to be sure, but I would expect state values to have 3 different values, all slightly positive, depending if opponent chose middle, edge or corner cases - each of these would have slightly different chance of leading to a win for the agent going forward.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will the value functions change if the starting players are changed?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Due to the turn-based nature of the game, all the states observed would be different depending on who was the first player. When the agent goes first it will get to score the empty grid and action values of any position it would like to make a first mark in - it gets to see state after turn 0 on time step 1, after turn 2 on time step 2, after turn 4 on time step 3 etc. When the agent goes second it will get to see and evaluate the outpt of other turns - turn 1 at t=1, turn 3 at t=2, turn 5 at t=3 etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means the sets of states and state/action pairs for each case (RL first or RL second) are disjoint, and you cannot ask if one agent has the same value for a specific state as the other agent - it simply won't know about the other agent's values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you train a single agent, sometimes starting first, sometimes starting second, the two sets of values never interact with each other directly - in an enumerated table, as per the question, this is not at all, but if an agent uses function approximation such as neural networks, then they can affect each other.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-17T17:39:29.453" LastActivityDate="2019-03-17T17:39:29.453" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11293" PostTypeId="1" AcceptedAnswerId="11294" CreationDate="2019-03-17T19:05:22.787" Score="1" ViewCount="86" Body="&lt;p&gt;I am trying to understand how weights are actually gotten. What is generating them in a neural network? What is the algorithm that gives them certain values?&lt;/p&gt;&#xA;" OwnerUserId="23262" LastEditorUserId="22916" LastEditDate="2019-03-17T20:08:08.840" LastActivityDate="2019-03-17T21:44:44.843" Title="What determines the values of weights in a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;neurons&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11294" PostTypeId="2" ParentId="11293" CreationDate="2019-03-17T19:48:10.363" Score="5" Body="&lt;p&gt;Typically, weights are randomly initialized. Then, as the model is &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neural_network#Optimization&quot; rel=&quot;noreferrer&quot;&gt;optimized&lt;/a&gt; for its given task, those weights are steadily made &quot;better&quot; as determined by the network's &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;noreferrer&quot;&gt;loss function&lt;/a&gt;. This is also referred to as &quot;training&quot; the neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By far the most popular way of updating weights in a neural net is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;noreferrer&quot;&gt;backpropagation&lt;/a&gt; algorithm, most simply with &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&quot; rel=&quot;noreferrer&quot;&gt;stochastic gradient descent&lt;/a&gt; (SGD). Essentially, the algorithm determines how much each individual weight contributed to the network's loss. It then updates that weight in the direction that would reduce the loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend going through Michael Nielsen's &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/&quot; rel=&quot;noreferrer&quot;&gt;online book&lt;/a&gt; to learn the basics.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-17T19:55:17.307" LastActivityDate="2019-03-17T19:55:17.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11295" PostTypeId="2" ParentId="11293" CreationDate="2019-03-17T21:44:44.843" Score="0" Body="&lt;p&gt;I agree with @PhilipRaeisghasem, in most architectures, weights are initialized in a random manner. However, some research papers suggest applying a random normal distribution initialization to the weights in the case of Convolutional Neural Networks (for computer vision).&lt;/p&gt;&#xA;" OwnerUserId="20934" LastActivityDate="2019-03-17T21:44:44.843" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11296" PostTypeId="2" ParentId="10734" CreationDate="2019-03-17T22:31:40.493" Score="3" Body="&lt;p&gt;I think you would be interested in &lt;a href=&quot;https://arxiv.org/pdf/1711.03902.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural-Symbolic Learning and Reasoning&lt;/a&gt;, a recent survey on the intersection between connectionist models (e.g. neural nets) and symbolic reasoning. It's a long paper, but it has a lot in it that is relevant to your question, if you're willing to dig. For example, it talks about learning networks that implement boolean logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yzaeK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yzaeK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This&#xA;  network searches for satisfying solutions for the weighted conjunctive normal form (CNF):&lt;span class=&quot;math-container&quot;&gt;$\text{(¬X ∨ ¬Y ∨ Z) ∧ (X ∨ Y )}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The closest thing I could find to the &quot;functionals&quot; you mentioned was a discussion on learning &quot;relations&quot; with neural networks. For this, it cited &lt;a href=&quot;https://epdf.tips/neural-symbolic-cognitive-reasoning3f30592b99b4240b1790fcf45c13e1e260069.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural-Symbolic Cognitive Reasoning&lt;/a&gt;, a book also about the intersection between neural networks and symbolic reasoning. In Chapter 10, it gives the example of teaching a neural network to learn the relation of &quot;grandparent.&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Our goal is to learn a description of grandparent from examples of the father, mother, and grandparent relations such as &lt;em&gt;father&lt;/em&gt;(charles, william), &lt;em&gt;mother&lt;/em&gt;(elisabeth, charles), and &lt;em&gt;grandparent&lt;/em&gt;(elisabeth, william).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I could imagine something like this being done with the example inputs being image features from a CNN. Perhaps then you could start learning functionals like &quot;on the shirt.&quot;&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-17T22:31:40.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11297" PostTypeId="2" ParentId="10403" CreationDate="2019-03-17T23:08:19.130" Score="5" Body="&lt;p&gt;I can't speak for individual researchers, but I can guess why the community as a whole hasn't adopted this activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt; is just so incredibly cheap. This benefit continues to grow as networks grow deeper. Also, they work reasonably well. As pointed out in &lt;a href=&quot;https://arxiv.org/pdf/1710.05941.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Searching for Activation Functions&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the performance improvements of the other activation functions tend to be inconsistent across different models and datasets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Even if a new activation function did provide a meager improvement in performance across the board, I wouldn't be surprised if ReLU were still commonly used. It's the default for a lot of machine learning software already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also research isn't ordinarily about eeking out one more percentage point in accuracy on a specific task. If I were entering in a competition, I might experiment with activation functions. But even then I'd rather use ReLU and save a little time while prototyping my architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by @DuttaA in comments, softsign could potentially replace sigmoid and tanh in situations where a bounded output is desired. I haven't seen anyone compare them before, but softsign would at least be much faster. I'm guessing this replacement hasn't happened because of tradition and exposure. Not because of softsign's lack of infinite derivatives. I don't know if this happening would make softsign &quot;widely used&quot;, but it would be something.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-03-18T09:32:03.817" LastActivityDate="2019-03-18T09:32:03.817" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11298" PostTypeId="1" CreationDate="2019-03-18T02:26:26.940" Score="1" ViewCount="64" Body="&lt;p&gt;I am attempting to train a network to do something I thought would be a relatively simple case to learn with: identify whether the back of a scanned vintage postcard has one of 'no postage stamp', a '1 cent stamp', or a '2 cent stamp.' The images are 250px by about 150px, RGB color, and there are about two thousand of them. Ballpark 75% of them are no-stamp, 20% 1-cent, and 10% 2-cent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I attempt to train the network it seems like it is starting at 70 +/- 1 % accurate and hovers in that range for 50 epochs, never improving. I'm not sure I'm reading the metrics correctly, though, as this doesn't seem quite right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I set this up by following the tutorial on the Keras blog: &lt;a href=&quot;https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't implemented the latter part of the tutorial, where a pre-trained network is used, because I haven't found one that seems like it would be a similar problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My training and validation sets are here: &lt;a href=&quot;https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my Google Colab Jupyter notebook is here: &lt;a href=&quot;https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="23264" LastActivityDate="2019-03-18T23:04:02.620" Title="Why doesn't my image classification network get better with training?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11302" PostTypeId="1" AcceptedAnswerId="11305" CreationDate="2019-03-18T08:47:45.250" Score="1" ViewCount="53" Body="&lt;p&gt;For the case of crowd density estimation using CNN, using datasets like shanhaiTech or UCF, why there hasn't been attempts to tackle this type of task as a classification problem? All current papers I've seen are related to regression, not classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I have the crowd images labeled based on their crowd density (low, moderate, high), and I'm not interested in the count, but the density class (low, moderate, high), can't I train the network to classify the data based on these classes as a classification network?&lt;/p&gt;&#xA;" OwnerUserId="23268" LastEditorUserId="2444" LastEditDate="2019-03-18T16:39:17.823" LastActivityDate="2019-03-18T16:39:46.237" Title="Why don't we perform classification of crowd density?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11305" PostTypeId="2" ParentId="11302" CreationDate="2019-03-18T09:40:42.153" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, if I have the crowd images labeled based on their crowd density (low, moderate, high), and I'm not interested in the count, but the density class (low, moderate, high), can't I train the network to classify the data based on these classes as a classification network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes you can, all you need is enough correctly labelled training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good rule of thumb is if a human expert can assign the correct label from an image (and purely from the image, not using extra information) then it is a realistic goal to train a CNN to perform the same labelling. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;why there hasn't been attempts to tackle this type of task as a classification problem.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Probably because there are no natural, and likely no widely accepted, classes in this case (I may be wrong, maybe some international society has defined classes you could use). If you use a regression, you can map it to a particular problem case - e.g. sending an alert to someone responsible for traffic and safety when crowd density hits some threshold - by setting numerical boundaries to your classes. Using classification and mapping back the other way is harder.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-03-18T16:39:46.237" LastActivityDate="2019-03-18T16:39:46.237" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11306" PostTypeId="2" ParentId="5458" CreationDate="2019-03-18T12:41:44.480" Score="3" Body="&lt;p&gt;I just want to provide this intuition &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;this NN consists of a 2 steps detection pipeline (the region proposal and regression + classification in parallel) exploring a certain range of scales and aspect ratios for proposals&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;as the proposed region is rectangular and the objects of interests have not a rectangular appearance strictly speaking, but a mostly rectangular one (with different ARs), considering the resulting detection bboxes areas some pixels will belong to a certain relevant object appearance (e.g. pixels of a pedestrian, car, …) while other pixels in that BBox won’t be related to that semantic&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;as this is not semantic segmentation we can now know what pixel is actually relevant, however the &lt;em&gt;objectness&lt;/em&gt; and &lt;em&gt;backgroundness&lt;/em&gt; can provide a rough measure of this in an aggregated way&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Let’s assume at some point in your processing pipeline, you have 2 regions containing the same car but at different scales: so ideally both of them should be associated with good confidence to CarID but it’s clear one will contain much more background pixels than the other one and you'd like this to be represented as an additional measure &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-03-18T12:41:44.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11307" PostTypeId="2" ParentId="11196" CreationDate="2019-03-18T12:47:02.480" Score="5" Body="&lt;p&gt;I recommend &lt;a href=&quot;https://www.taylorfrancis.com/books/9781317709947&quot; rel=&quot;noreferrer&quot;&gt;Animal Learning and Cognition: An Introduction&lt;/a&gt; by John Pearce.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As an introduction to the field, it's comprehensive. It covers all the topics you mentioned and more. &lt;a href=&quot;https://content.taylorfrancis.com/books/download?dac=C2010-0-34629-9&amp;amp;isbn=9781317709947&amp;amp;format=googlePreviewPdf&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt; is a free preview of the book.&lt;/li&gt;&#xA;&lt;li&gt;It's well-regarded, having over 500 citations &lt;a href=&quot;https://scholar.google.com/scholar?q=Animal%20Learning%20and%20Cognition&amp;amp;hl=en&amp;amp;as_sdt=0&amp;amp;as_vis=1&amp;amp;oi=scholart&quot; rel=&quot;noreferrer&quot;&gt;on Google Scholar&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;It's relatively modern. Published in 2008, it discusses experimental findings that you won't be able to find in some of the other classics published in the 70s or 80s.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although it does not specifically focus on mammals, all of the ideas extend to mammals.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-18T12:47:02.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11308" PostTypeId="2" ParentId="11219" CreationDate="2019-03-18T14:12:18.063" Score="-2" Body="&lt;p&gt;The approximator can be any artificial neural network architecture, including deep fully-connected networks.&lt;/p&gt;&#xA;" OwnerUserId="23276" LastEditorUserId="23276" LastEditDate="2019-03-22T09:10:32.753" LastActivityDate="2019-03-22T09:10:32.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11309" PostTypeId="1" CreationDate="2019-03-18T16:08:15.043" Score="2" ViewCount="31" Body="&lt;p&gt;I've got classification problem on image, I have 10 classes and when I fine tuned my model on it (I tried VGG, Xception, resnet etc) I have approximatly 83% validation accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if doing lot of binary model with 1 class represented and the other as 'other' and then using them to classify my image would be good and efficient ? (I obtain more than 90% val acc for each class doing this)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Except for memory consumption and training time does this method have drawback ?&lt;/p&gt;&#xA;" OwnerUserId="23107" LastActivityDate="2019-03-18T16:08:15.043" Title="Is making lot of 1 versus other model efficient?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11310" PostTypeId="1" CreationDate="2019-03-18T19:00:35.433" Score="2" ViewCount="88" Body="&lt;p&gt;Consider a shop owner who has to deal with having to buy for one week from a different supplier with several different brands. Another week a brand is removed or added from the market. Yet another week, the manager decides to skip three assortments of fruit soda and exchange them with a different selection of three assortments of fruit sods, and so on and so forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is dealing with such issues possible with existing public implementations of reinforcement learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only research that I saw dealing with dynamically changing stuff like this is the DeepMind FTW bot playing Quake capture the flag. It deals with changing layouts to the map being played, but the implementation is not public and it doesn't resemble the inventory management situation I outlined.&lt;/p&gt;&#xA;" OwnerUserId="21724" LastEditorUserId="2444" LastEditDate="2019-03-18T20:23:59.450" LastActivityDate="2019-03-18T20:23:59.450" Title="Reinforcement learning for inventory management with dynamic changes to available products" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11311" PostTypeId="2" ParentId="11298" CreationDate="2019-03-18T21:52:41.733" Score="1" Body="&lt;p&gt;The answer (&lt;a href=&quot;https://stackoverflow.com/questions/42081257/keras-binary-crossentropy-vs-categorical-crossentropy-performance&quot;&gt;as detailed quite thoroughly here&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/45799474/keras-model-evaluate-vs-model-predict-accuracy-difference-in-multi-class-nlp-ta/45834857#45834857&quot;&gt;here&lt;/a&gt;) is that specifying&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(optimizer = 'rmsprop', loss = 'binary_crossentropy', metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;causes keras to guess, incorrectly, that because I am using &lt;code&gt;binary_crossentropy&lt;/code&gt; for the loss function, that I would want to use &lt;code&gt;binary_accuracy&lt;/code&gt; as the way of reporting the accuracy metrics. Apparently, one should specify that one wants the &lt;code&gt;categorical_accuracy&lt;/code&gt; metrics if one, as I do, has more than two classes.&lt;/p&gt;&#xA;" OwnerUserId="23264" LastEditorUserId="2444" LastEditDate="2019-03-18T23:04:02.620" LastActivityDate="2019-03-18T23:04:02.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11313" PostTypeId="1" AcceptedAnswerId="11319" CreationDate="2019-03-18T23:16:00.460" Score="5" ViewCount="163" Body="&lt;p&gt;I have read a lot about RL recently. As far as I understood, most RL applications have much more states than there are actions to choose from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about using RL for a problem where I have got a lot of actions to choose from, but only very few states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give a handy example:&#xA;The algo should render (for whatever reason) a sentence with three words. I always want to have a sentence with three words, but I have many words to choose from. After choosing the words, I get some sort of reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are RL algorithms an efficient way to solve this?&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about using policy gradients with an ε-greedy algorithm to explore a lot of the possible actions before exploiting the knowledge gained.&lt;/p&gt;&#xA;" OwnerUserId="23288" LastEditorUserId="2444" LastEditDate="2019-03-18T23:24:28.367" LastActivityDate="2019-03-19T08:59:47.867" Title="Reinforcement Learning with more actions than states" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;greedy-ai&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11314" PostTypeId="5" CreationDate="2019-03-18T23:30:22.490" Score="0" Body="&lt;p&gt;For more info, see this tutorial: &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="1641" LastEditDate="2019-05-02T19:22:26.127" LastActivityDate="2019-05-02T19:22:26.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11315" PostTypeId="4" CreationDate="2019-03-18T23:30:22.490" Score="0" Body="For questions related to reinforcement learning algorithms often referred to as &quot;policy gradients&quot; (or &quot;policy gradient algorithms&quot;), which attempt to directly optimise a parameterised policy (without first attempting to estimate value functions) using gradients of an objective function with respect to the policy's parameters." OwnerUserId="2444" LastEditorUserId="1641" LastEditDate="2019-05-02T19:23:53.533" LastActivityDate="2019-05-02T19:23:53.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11316" PostTypeId="1" CreationDate="2019-03-19T07:23:56.957" Score="0" ViewCount="32" Body="&lt;p&gt;I Want to cite the reference paper of The models appeared in this webpage:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ruotianluo/ImageCaptioning.pytorch/blob/master/models/__init__.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ruotianluo/ImageCaptioning.pytorch/blob/master/models/&lt;strong&gt;init&lt;/strong&gt;.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But which papers do these four models 'all_img', 'att2in2', 'adaattmo' and 'topdown' originate from?&#xA;and Which of the above models are created by the author of the github page?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21613" LastActivityDate="2019-03-19T07:23:56.957" Title="What are the reference papers of image captioning model 'all_img', 'att2in2', 'adaattmo' and 'topdown'?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11317" PostTypeId="1" CreationDate="2019-03-19T07:46:36.403" Score="1" ViewCount="116" Body="&lt;p&gt;This is a simple version of NIM: Two players alternately remove one, two or three coins from a stack initially containing 5 coins. The player who picks up the last coin loses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does alpha-beta pruning look like on the game tree for this game?&lt;/p&gt;&#xA;" OwnerUserId="23298" LastEditorUserId="1847" LastEditDate="2019-03-19T09:52:48.520" LastActivityDate="2019-03-19T09:52:48.520" Title="Understanding alpha-beta pruning for simplified NIM" Tags="&lt;game-ai&gt;&lt;minimax&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11318" PostTypeId="1" CreationDate="2019-03-19T08:30:30.273" Score="2" ViewCount="104" Body="&lt;p&gt;I am writing a field report on AI. I was wondering what the technological challenges are that AI is facing today. I have written the following so far.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AI needs common sense like a human common&lt;/li&gt;&#xA;&lt;li&gt;AI needs curiosity &lt;/li&gt;&#xA;&lt;li&gt;AI needs to understand cause and effect&lt;/li&gt;&#xA;&lt;li&gt;AI needs to be creative&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there any other hardware-tech related obstacles?&lt;/p&gt;&#xA;" OwnerUserId="23300" LastEditorUserId="2444" LastEditDate="2019-03-20T08:59:28.327" LastActivityDate="2019-03-20T08:59:28.327" Title="What are the technological challenges that AI faces today?" Tags="&lt;machine-learning&gt;&lt;artificial-consciousness&gt;&lt;hardware&gt;&lt;challenges&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11319" PostTypeId="2" ParentId="11313" CreationDate="2019-03-19T08:32:53.960" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;As far as I understood, most RL applications have much more states than there are actions to choose from.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, this is quite common, but in no way required by the underlying theory of &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Processes (MDPs)&lt;/a&gt;. The most extreme version of the opposite thing - with one state (or effectively no state, as state is not relevant)  - are k-armed bandit problems, where an agent tries to find a single best long-term action in general from a selection of actions. These problems typically would not use RL algorithms such as Q-learning or policy-gradients. However, that is partly because they are described with different goals in mind (e.g. minimising &quot;regret&quot; or simply gaining as much reward as possible &lt;em&gt;during the learning process&lt;/em&gt;), and RL algorithms will work to solve them, albeit less efficiently than algorithms designed to work on bandit problems directly.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am thinking about using RL for a problem where I have got a lot of actions to choose from, but only very few states.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That should work, provided your problem is still a MDP. That means for instance that the state evolves according to rules depending on which action was taken in which starting state. If the state evolution is instead arbitrary or random, then you may have a contextual bandit problem instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an important difference here between:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a large number of entirely different actions, each with different results, which need enumeration and have to be explored separately&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a large action space due to measurable values which are part of an action, such as how much force to apply in a motor&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The former will require lots of exploration, since any specific combination of action and state could be the ideal. With the latter, you can use that fact that numerical values that are similar will often give similar results, which will make generalisation via function approximation (e.g. neural networks) work efficiently in your favour.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The algo should render (for whatever reason) a sentence with three words. I always want to have a sentence with three words, but I have many words to choose from. After choosing the words, I get some sort of reward&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems more like the first bullet-point above, although that may depend if a natural language model could be applied for example. E.g. if &quot;This is good&quot; and &quot;This is great&quot; would produce similar rewards in a specific state, then there is maybe some benefit to generalisation, although I am not quite sure where you would fit this knowledge - possibly in a generator for a sentence vector as the &quot;raw&quot; action and then have a LSTM-based language model produce the actual action from that vector, similar to seq2seq translation models.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are RL algorithms an efficient way to solve this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but whether or not it is the most efficient will depend on other factors, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether the environment is stochastic or deterministic with regard to both reward and state progression. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether state progression is key to obtaining the best rewards in the long term (e.g. there is some &quot;goal&quot; or &quot;best&quot; state that can only be reached by a certain route).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What the actual size of the MDP is &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}| \times |\mathcal{A}|$&lt;/span&gt;. Small MDPs can be fully enumerated, allowing you to estimate action values in a simple table. If you have 10 states and 1,000,000 discrete actions, with no real pattern of actions mapping to results, then a big 10 million entry table will actually be reasonably efficient.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Competitive algorithms to RL here might be global search and optimisation ones, such as genetic algorithms. The more arbitrary and deterministic your environment is, the more likely it is that an exhaustive search will find your optimal policy faster than RL.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am thinking about using policy gradients with an ε-greedy algorithm to explore a lot of the possible actions before exploiting the knowledge gained.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This should be fine. Exploration here is definitely important, but finding the sweet spot for the right amount of it will be hard, and depend on other traits of the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to use something like &lt;a href=&quot;https://banditalgs.com/2016/09/18/the-upper-confidence-bound-algorithm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;upper confidence bound action selection&lt;/a&gt; or simply optimistic initial values, in order to ensure exploration does not miss certain actions. An epsilon greedy approach will miss a certain fraction of actions over time, and the expectation of that fraction grows smaller progressively more slowly, so it may be possible to miss an important action for a long time if you rely on being able to randomly select it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To give a handy example: The algo should render (for whatever reason) a sentence with three words. I always want to have a sentence with three words, but I have many words to choose from. After choosing the words, I get some sort of reward.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would consider modelling this as sequences of 3 actions, each of which chooses a word, with the state being a start token (whatever the state you already have) plus the sentence so far, and on every 3 words the environment is consulted to reset the state to the next start token and to gather a reward (rewards for interim steps would be zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this immediately makes the state space much larger than the action space, as your state includes history of up to two actions. If you had 10 different start states, and 100 word choices, then your state space would be 101,010 and action space 100.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will fit available designs of RL algorithms, and allow for learning some internal language modelling if it is relevant. It will reduce your need to model sentence construction outside of the agent. Most importantly, if &quot;good&quot; or &quot;bad&quot; sentences tend to start or end with certain words, and you use function approximation, then the algorithm may discover combinations more efficiently than iterating over all sentences as if they were completely independent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-19T08:59:47.867" LastActivityDate="2019-03-19T08:59:47.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11322" PostTypeId="1" CreationDate="2019-03-19T11:51:12.200" Score="0" ViewCount="319" Body="&lt;p&gt;I have the following part of code -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;model.add(Generic_act_func())&#xA;print('layer 1 output')&#xA;print(model.layers[1].output) #Here layers[1] is Generic_act_func() layer specified above&#xA;model.add(Lambda(lambda x: tf.spectral.irfft2d(x)))&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which when run produces the following output and error respectively -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;layer 1 output&#xA;Tensor(&quot;generic_act_func_1/Complex:0&quot;, shape=(?, 28, 28, 9), dtype=complex64)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Error -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Tensor conversion requested dtype complex64 for Tensor with dtype float32: 'Tensor(&quot;lambda_1/Placeholder:0&quot;, shape=(?, 28, 28, 9), dtype=float32)'&#xA;at line&#xA;&#xA;model.add(Lambda(lambda x: tf.spectral.irfft2d(x)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;tf.spectral.irfft2d() requires a complex64 input which the previous layer i.e., Generic_act_func() layer is already producing (as printed by layer 1 output), hence I am not able to figure out why this error. Thanks in advance for any help for solving the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit - Could there be a mistake wrt passing previous layer outputs to the Lambda layer? Like I have used a variable 'x' in the Lambda function assuming the previous layer output will be captured in this variable. Is this the correct way of adding the Lambda layer? (I am a beginner in Keras, so will be really grateful for any help)&lt;/p&gt;&#xA;" OwnerUserId="23306" LastEditorUserId="23306" LastEditDate="2019-03-20T06:36:42.970" LastActivityDate="2019-03-20T06:36:42.970" Title="What is the meaning of ValueError in Keras? - 'Tensor conversion requested dtype complex64 for Tensor with dtype float32'" Tags="&lt;computer-vision&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11323" PostTypeId="1" CreationDate="2019-03-19T12:28:04.870" Score="0" ViewCount="17" Body="&lt;p&gt;I’m in this problem and haven’t found a sound solution to it. Been like 20 days now. I have a dataset that looks like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X=image&#xA;&#xA;Y1= current_zoom (0,25,50,75)&#xA;&#xA;Y2= predicted_zoom (0,25,50,75)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;y1 will have equal images for all classes. Also, I will know X and y1 when I test the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y2 will have variation with it because it has the predicted zoom level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to train on MTL model, I used two outputs from the model - y1 and y2. Now, y2 overfits (still not sure why, but my best guess is class imbalance). And y1 accuracy is around 0.99 in validation. Now the thing is when I deploy this on production, I’ll always have the current zoom (y1) with the image. So, I wanted to incorporate this to my model. First, I tried with two inputs and one output model, but loss was too much. Here, I concatenated the y1 to the output of last conv layer before it flattens and goes to dense. And second I tried was to concatenate y1 after flattening the output from last conv. Both didn’t work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how can I with data like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Models used: resnet-18, vgg, alexnet&lt;/p&gt;&#xA;&#xA;&lt;p&gt;size of data: approx 7000 images in total.&lt;/p&gt;&#xA;" OwnerUserId="15633" LastActivityDate="2019-03-19T12:28:04.870" Title="how to work with multi-labels or two inputs and a output" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11324" PostTypeId="1" CreationDate="2019-03-19T15:50:12.097" Score="0" ViewCount="19" Body="&lt;p&gt;Let's say I have a number of videos, and I want to train an SSD/YOLO (or FRCNN) to detect objects. In the case of a large amount of videos,  there will be a lot of frames extracted and transferred to images. Can you take, for example, only the fifth frame everytime and thus lower the amount of memory required? If the frames contain a similar info, we can skip some and not impact the results? This is mainly to train faster..&lt;/p&gt;&#xA;" OwnerUserId="23311" LastActivityDate="2019-03-19T15:50:12.097" Title="Consecutive frames can be discarded when training an SSD/YOLO?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;datasets&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11325" PostTypeId="1" CreationDate="2019-03-19T15:59:36.103" Score="2" ViewCount="45" Body="&lt;p&gt;What if there are multiple goals? For example, let's consider the bit-flipping environment as described in the paper &lt;a href=&quot;https://arxiv.org/pdf/1707.01495.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;HER&lt;/a&gt; with one small change: now, the goal is not some specific configuration, but let's say for the last &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; bits (e.g. &lt;span class=&quot;math-container&quot;&gt;$m=2$&lt;/span&gt;), I do not really care if there is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the paper, there is section 3.2 multi-goal RL, where they mention an example with two-dimensional coordinates (&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;), but they are interested only in the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; coordinate, so they only use the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; coordinate as a goal.&lt;/p&gt;&#xA;&lt;p&gt;Applying this strategy to my example would result in cutting the last &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; bits from the goal and only use the other bits. Is this logic correct?&lt;/p&gt;&#xA;&lt;p&gt;Another approach I could think of would be to train with all possible goal configurations, as there are not many in my case. But this seems impractical as the number of goal configurations grows.&lt;/p&gt;&#xA;" OwnerUserId="22162" LastEditorUserId="2444" LastEditDate="2020-11-20T18:44:56.217" LastActivityDate="2020-11-20T18:44:56.217" Title="How does Hindsight Experience Replay cope with multiple goals?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;state-spaces&gt;&lt;hindsight-experience-replay&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11326" PostTypeId="1" AcceptedAnswerId="11358" CreationDate="2019-03-19T17:03:32.260" Score="1" ViewCount="108" Body="&lt;p&gt;I have a 2D plane, with a fixed height and width of 10M. The plane has an agent (or robot) in the point &lt;span class=&quot;math-container&quot;&gt;$(1, 2.2)$&lt;/span&gt;, and an &lt;em&gt;electric outlet&lt;/em&gt; in the point &lt;span class=&quot;math-container&quot;&gt;$(8.2, 9.1)$&lt;/span&gt;. The plane has a series of obstacles.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ucSQA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ucSQA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there an algorithm to find the shortest path between the agent and the goal?&lt;/p&gt;&#xA;&lt;p&gt;And if the point has a fixed wingspan? For example, that the space between &lt;strong&gt;O&lt;/strong&gt; and &lt;strong&gt;N&lt;/strong&gt; is smaller than the agent, and then the agent cannot cross?&lt;/p&gt;&#xA;" OwnerUserId="22082" LastEditorUserId="2444" LastEditDate="2021-01-23T00:56:05.097" LastActivityDate="2021-01-23T00:56:05.097" Title="How can I calculate the shortest path between two 2d vector points in an environment with obstacles?" Tags="&lt;search&gt;&lt;robotics&gt;&lt;a-star&gt;&lt;shortest-path-problem&gt;&lt;path-finding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11327" PostTypeId="1" CreationDate="2019-03-19T19:09:31.427" Score="3" ViewCount="38" Body="&lt;p&gt;Nowadays, there is too much data for humans to work on alone, and it is very normal for data analysts to use AI techniques to treat and process these data so it can lead to a faster and more accurate result. But many data analysts and decision-makers still don't trust AI methods or techniques and are reluctant to use them. How can we encourage them to accept or prefer these AI solutions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if AU gives advice to solve a problem, then decision-makers must trust the results and data analysts must trust the mechanism, so that decision-makers can be confident in the AI work and also data analysts can concentrate their activities on added value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we encourage data analysts and decision-makers to adopt AI?&lt;/p&gt;&#xA;" OwnerUserId="23315" LastEditorUserId="2444" LastEditDate="2019-11-06T22:48:12.167" LastActivityDate="2019-11-06T22:48:12.167" Title="How can we encourage data analysts and decision-makers to adopt AI?" Tags="&lt;applications&gt;&lt;ai-community&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11328" PostTypeId="1" CreationDate="2019-03-19T19:25:00.353" Score="1" ViewCount="86" Body="&lt;p&gt;Suppose we want to predict context words &lt;span class=&quot;math-container&quot;&gt;$w_{i-h}, \dots, w_{i+h}$&lt;/span&gt; given a target word &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; for a window size &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; around the target word &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;. We can represent this as: &lt;span class=&quot;math-container&quot;&gt;$$p(w_{i-h}, \dots, w_{i+h}|w_i) = \prod_{-h \leq k \leq h, \ k \neq 0} p(w_{i+k}|w_i)$$&lt;/span&gt; where we model the probabilities of a word &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; given another word &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$$p(u|v) = \frac{\exp(\left&amp;lt;\phi_u, \theta_v \right&amp;gt;)}{\sum_{u' \in W} \exp(\left&amp;lt;\phi_{u'}, \theta_v \right&amp;gt;)}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\phi_u, \theta_v$&lt;/span&gt; are some vector representations for words &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; respectively and &lt;span class=&quot;math-container&quot;&gt;$\left&amp;lt;\phi_u, \theta_v \right&amp;gt;$&lt;/span&gt; is the dot product between these vector representations (which represents some sort of similarity between the words) and &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is a matrix of all the words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Skip-Gram Negative Sampling, we want to learn the embeddings &lt;span class=&quot;math-container&quot;&gt;$\phi_u, \theta_v$&lt;/span&gt; that maximize the following: &lt;span class=&quot;math-container&quot;&gt;$$\sum_{u \in W} \sum_{v \in W} n_{uv} \log \sigma(\left&amp;lt;\phi_u, \theta_v \right&amp;gt;) +k \mathbb{E}_{\bar{v}} \log \sigma(-\left&amp;lt;\phi_u, \theta_{\bar{v}} \right&amp;gt;)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question.&lt;/strong&gt; How exactly does this work? For example, suppose &lt;span class=&quot;math-container&quot;&gt;$k=5$&lt;/span&gt;, the target word &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\text{apple}$&lt;/span&gt; and we want to find&#xA;  &lt;span class=&quot;math-container&quot;&gt;$p(\text{pie}| \text{apple})$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$n_{uv} = 10$&lt;/span&gt; (number of times pie&#xA;  co-occurs with apple). Then we sample &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; random words &lt;span class=&quot;math-container&quot;&gt;$\bar{v}$&lt;/span&gt; that&#xA;  did not occur with &lt;span class=&quot;math-container&quot;&gt;$\text{apple}$&lt;/span&gt; and whichever term in the sum is&#xA;  bigger is the one we predict? For example, if the first term in the&#xA;  sum is larger than the second term then we would predict that&#xA;  &lt;span class=&quot;math-container&quot;&gt;$p(\text{pie}| \text{apple}) \approx 1$&lt;/span&gt;? Otherwise we predict that&#xA;  &lt;span class=&quot;math-container&quot;&gt;$p(\text{pie}| \text{apple}) \approx 0$&lt;/span&gt;? Is this the correct&#xA;  intuition?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source.&lt;/strong&gt; &lt;a href=&quot;https://www.coursera.org/learn/language-processing/lecture/A4eQD/explicit-and-implicit-matrix-factorization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; at around the 10:05 mark.&lt;/p&gt;&#xA;" OwnerUserId="23220" LastEditorUserId="2444" LastEditDate="2019-04-16T22:23:58.053" LastActivityDate="2019-04-16T22:23:58.053" Title="Skip-Gram Model Training" Tags="&lt;natural-language-processing&gt;&lt;word2vec&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11330" PostTypeId="2" ParentId="11327" CreationDate="2019-03-19T19:50:11.673" Score="2" Body="&lt;p&gt;Not all of the mistrust aimed at AI systems is unjustified, particularly when it comes to neural networks and other such systems that rely on large training data sets. There are a number of high profile cases, facial recognition being one that has often (understandably) received a lot of flak, where improperly configured training data has resulted in skewed and questionable results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to foster more trust in the systems, it will require better tools for analyzing how they are approaching problems and reaching decisions, as well as determining if there are holes in the training data. It will require a community working in the field that gives a lot more thought and care to how they approach their training data and what unintended biases they may be introducing by forgetting something than has often been displayed presently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, some people are just distrustful of new technology, but I think it's more interesting to address the more legitimate concerns. &lt;/p&gt;&#xA;" OwnerUserId="15114" LastActivityDate="2019-03-19T19:50:11.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11331" PostTypeId="2" ParentId="11318" CreationDate="2019-03-19T20:46:52.937" Score="2" Body="&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_creativity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Creativity&lt;/a&gt;&lt;/strong&gt; is not an unassailable challenge (depending on who you talk to;) &lt;a href=&quot;https://www.technologyreview.com/s/612913/a-philosopher-argues-that-an-ai-can-never-be-an-artist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Philosophers have claimed&lt;/a&gt; algorithms can't be creative, but Marcel Duchamp, one of the most significant artists in modernity, famously stated that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;All artists are not chess players, but all chess players are artists&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This would seem to have been validated by commentators referring to &lt;a href=&quot;https://www.wired.com/2016/03/sadness-beauty-watching-googles-ai-play-go/&quot; rel=&quot;nofollow noreferrer&quot;&gt;move 37 in game 2 of  Lee Sedol's match with AlphaGo as &quot;beautiful&quot;&lt;/a&gt;  (In the game of Go, aesthetics are considered in regard to strategy, not just outcomes.)  The takeaway is it was a choice humans would never have considered, because the structure of our brains is different, and thus our approach to creativity different, than automata.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current algorithmic creativity is a function of monte carlo methods, which utilize randomness, Monte Carlo Tree Search as a major method. &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCTS&lt;/a&gt; has great utility in intractable models such as non-trivial combinatorial games, which produce complexity akin to nature, so it's not surprising it's slowing extending to real world applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue with computational creativity is it is still not as efficient as human creativity, requiring a great deal of processing power for non-trivial problems.  (This was why it took so long for a computer to beat the best human a Chess--human insight seems rooted in semantics/understanding.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Procedural content generation is an area of research that is steadily progressing, and includes &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.297.2821&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;games&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_composition&quot; rel=&quot;nofollow noreferrer&quot;&gt;music&lt;/a&gt; and &lt;a href=&quot;https://www.christies.com/features/A-collaboration-between-two-artists-one-human-one-a-machine-9332-1.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;visual art&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithmic Bias&lt;/a&gt;&lt;/strong&gt; is the most pressing issue facing the field of AI and Machine Learning methods specifically, which are statistical.  (If the dataset is incorrect, imcomplete or biases, the output will be biased.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-03-19T21:12:54.507" LastActivityDate="2019-03-19T21:12:54.507" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11332" PostTypeId="1" CreationDate="2019-03-19T20:47:49.283" Score="0" ViewCount="518" Body="&lt;p&gt;I am training a model using the DNN Regressor estimator from the Tensorflow API to predict prices based on 1035 features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataset contains a little over 500 millions inputs and none of the target output is negative. All prices are positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training for 3 epochs, my NN is still making negative predictions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this normal ? Shouldn't the network learn that prices are always positive &lt;/p&gt;&#xA;" OwnerUserId="23318" LastActivityDate="2019-03-19T20:47:49.283" Title="Neural Network Regression predicting negative values" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11333" PostTypeId="1" AcceptedAnswerId="11441" CreationDate="2019-03-19T21:18:47.090" Score="3" ViewCount="45" Body="&lt;p&gt;Agent can have reasoning skills (prediction, taking calculated guesses, etc.) and those skills can help reinforcement learning of this agent. Of course, reinforcement learning itself can help to develop reasoning skills. Are there research that explores this impact of reasoning and consciousness on the effectivenes of reinforcement learning. Or maybe people just sit and wait such skills to emerge during reinforcement learning?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2019-03-25T13:18:21.157" Title="How agent's reasoning skills can improve its reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;artificial-consciousness&gt;&lt;reasoning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11335" PostTypeId="1" CreationDate="2019-03-20T01:18:15.157" Score="1" ViewCount="31" Body="&lt;p&gt;I would like to use a 3D convolutional network on a 2000x2000x2000 volume for segmentation. I know I can break the volume into chunks that can fit in VRAM, but I was wondering if there was a way to analyze the entire 3D volume at once. &lt;/p&gt;&#xA;" OwnerUserId="23323" LastActivityDate="2019-03-20T01:18:15.157" Title="Very Large 3D Input Size" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11336" PostTypeId="1" CreationDate="2019-03-20T02:19:57.160" Score="1" ViewCount="390" Body="&lt;p&gt;If we have classified 1000 people's faces; how do we ensure the network tells us when it encounters a new person?&lt;/p&gt;&#xA;" OwnerUserId="23324" LastActivityDate="2021-01-08T11:06:55.747" Title="How do we classify an unrecognised face in face recognition?" Tags="&lt;image-recognition&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11337" PostTypeId="1" CreationDate="2019-03-20T04:51:46.497" Score="1" ViewCount="367" Body="&lt;p&gt;I am writing my first LSTM network and I would really appreciate if someone can tell me if it is right (the loss seems to go down very slowly and before playing around with hyper parameters I want to make sure that the code is actually doing what I want). The code is meant to go through some time series and label each point according to some categories. In the version I am putting here there are just 2 categories: 0, if the value of the point is 1 and 1 otherwise (I know it’s a bit weird, but I didn’t choose the labels). So this is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;from torch.autograd import Variable&#xA;import torch.optim as optim&#xA;import numpy as np&#xA;from fastai.learner import *&#xA;&#xA;torch.manual_seed(1)&#xA;torch.cuda.set_device(0)&#xA;&#xA;bs = 2&#xA;&#xA;x_trn = torch.tensor([[1.0000, 1.0000],&#xA;        [1.0000, 0.9870],&#xA;        [0.9962, 0.9848],&#xA;        [1.0000, 1.0000]]).cuda()&#xA;&#xA;y_trn = torch.tensor([[0, 0],&#xA;        [0, 1],&#xA;        [1, 1],&#xA;        [0, 0]]).cuda()&#xA;&#xA;n_hidden = 5&#xA;n_classes = 2&#xA;&#xA;class TESS_LSTM(nn.Module):&#xA;    def __init__(self, nl):&#xA;        super().__init__()&#xA;        self.nl = nl&#xA;        self.rnn = nn.LSTM(1, n_hidden, nl)&#xA;        self.l_out = nn.Linear(n_hidden, n_classes)&#xA;        self.init_hidden(bs)&#xA;&#xA;    def forward(self, input):&#xA;        outp,h = self.rnn(input.view(len(input), bs, -1), self.h)&#xA;        return F.log_softmax(self.l_out(outp),dim=1)&#xA;&#xA;    def init_hidden(self, bs):&#xA;        self.h = (V(torch.zeros(self.nl, bs, n_hidden)),&#xA;                  V(torch.zeros(self.nl, bs, n_hidden)))&#xA;&#xA;model = TESS_LSTM(1).cuda()&#xA;&#xA;loss_function = nn.NLLLoss()&#xA;&#xA;optimizer = optim.Adam(model.parameters(), lr=0.01)&#xA;&#xA;for epoch in range(10000):  &#xA;    model.zero_grad()&#xA;    tag_scores = model(x_trn)&#xA;    loss = loss_function(tag_scores.reshape(4*bs,n_classes), y_trn.reshape(4*bs))&#xA;    loss.backward()&#xA;    optimizer.step()&#xA;&#xA;    if epoch%1000==0:&#xA;        print(&quot;Loss at epoch %d = &quot; %epoch, loss)&#xA;&#xA;print(model(x_trn), y_trn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The (super reduced in size) time series should be &lt;code&gt;[1,1, 0.9962,1]&lt;/code&gt;, with labels &lt;code&gt;[0,0,1,0]&lt;/code&gt; and &lt;code&gt;[1, 0.9870, 0.9848,1]&lt;/code&gt; with labels &lt;code&gt;[0,1,1,0]&lt;/code&gt; and the batch size should be 2. I really hope I didn’t mess up the dimensionalities, but I tried to make it in a shape accepted by the LSTM. This is the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Loss at epoch 0 = tensor(1.3929, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;)&#xA;Loss at epoch 1000 = tensor(0.8939, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 2000 = tensor(0.8664, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 3000 = tensor(0.8390, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 4000 = tensor(0.8339, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 5000 = tensor(0.8288, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 6000 = tensor(0.8246, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 7000 = tensor(0.8202, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;) &#xA;Loss at epoch 8000 = tensor(0.8143, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;)&#xA;Loss at epoch 9000 = tensor(0.8108, device='cuda:0', grad_fn=&amp;amp;lt;NllLossBackward&amp;amp;gt;)&#xA;&#xA;(tensor([[[-9.0142e-01, -1.2631e+01],&#xA;          [-9.3762e-01, -9.6707e+00]],&#xA;&#xA;         [[-1.3467e+00, -3.9542e+00],&#xA;          [-2.2005e+00, -7.6977e-01]],&#xA;&#xA;         [[-2.4500e+01, -1.9363e-02],&#xA;          [-2.3349e+01, -6.2210e-01]],&#xA;&#xA;         [[-1.0969e+00, -2.1953e+01],&#xA;          [-6.9776e-01, -1.8608e+01]]], device='cuda:0',&#xA;        grad_fn=&amp;lt;LogSoftmaxBackward&amp;gt;), tensor([[0, 0],&#xA;         [0, 1],&#xA;         [1, 1],&#xA;         [0, 0]], device='cuda:0'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The loss doesn’t go down too fast (I expected it to overfit and go really close to zero). The actual values are okish (the smaller one is always the right one), but they can be definitely improved. Can someone tell me if my code is doing what I want (and maybe suggest why is the loss still big - maybe I need a smaller LR?)&lt;/p&gt;&#xA;" OwnerUserId="22839" LastActivityDate="2019-03-21T15:50:24.007" Title="LSTM is not converging" Tags="&lt;long-short-term-memory&gt;&lt;convergence&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11339" PostTypeId="2" ParentId="11336" CreationDate="2019-03-20T06:14:34.157" Score="0" Body="&lt;p&gt;You can't really make the network tell you if a face is new (unless you actually train a network with that particular purpose, maybe). Ideally, if you feed a new face to a trained network, the output activations will be pretty low in all the possible categories (faces that the network has already seen) i.e. no particular category will signal a high probability. In that case, you can just set a cut, such that if the maximum activation is below a certain threshold, it means that the NN encountered a new face. But things are not that easy in practice. If the NN encounters a face it has already seen, but the image is weird (blurry, the face doesn't cover much of the actual image, there are multiple faces in the image etc.) you could get the same output as in the case of a new face. At the same time, for a NN images are just probability distributions, so it could happen that a new face, which for us, humans, looks nothing like the others, to look pretty similar to an old one, from the NN point of view, hence giving a high activation in a single particular output neuron. &lt;/p&gt;&#xA;" OwnerUserId="22839" LastActivityDate="2019-03-20T06:14:34.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11340" PostTypeId="1" CreationDate="2019-03-20T09:46:16.050" Score="5" ViewCount="66" Body="&lt;p&gt;This is my first project using machine learning so I'm looking for some guidance. I am extending a model-based testing (MBT) system to a learning-based testing system by integrating a machine learning algorithm, for automation purposes. The MBT system executes tests on some system under test (SUT) and generates test verdicts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ML algorithm I want to integrate, is to take test verdicts from the MBT system as input and gain understanding of the SUT's behaviour, in order to generate new test cases. The test verdict (input) is a text-file, containing information about the previous test and whether it passed or failed. The output is a new test case, also a text file, containing variable values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking that supervised learning would be suitable since the input file contains both variable values (features) and the test verdict (class). However, I have doubts since I am not looking to solve a classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate ideas of what type of algorithm I should use (supervised/unsupervised/reinforcement etc.), and where I could find such (open-source) algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adam&lt;/p&gt;&#xA;" OwnerUserId="23331" LastEditorUserId="23331" LastEditDate="2019-03-20T10:08:43.060" LastActivityDate="2019-03-20T10:08:43.060" Title="Choosing Machine Learning Algorithm: Learning-Based Testing" Tags="&lt;machine-learning&gt;&lt;model-based-methods&gt;&lt;open-source&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11341" PostTypeId="2" ParentId="11328" CreationDate="2019-03-20T10:55:55.553" Score="1" Body="&lt;p&gt;Almost, but no. When you maximize that objective function, you do so by adjusting the parameters &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. After you're done with training, you can use your word embeddings for other NLP tasks. You don't, however, do any prediction directly from the skip-gram model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To maximize the first term, co-occuring words must have large inner products. That is, they must be &quot;similar&quot;. To maximize the second term**, the randomly sampled words must have a small inner product with &lt;span class=&quot;math-container&quot;&gt;$\phi_u$&lt;/span&gt;. That is, they must be &quot;dissimilar&quot; to &lt;span class=&quot;math-container&quot;&gt;$\phi_u$&lt;/span&gt;. Moving these word embeddings around in the vector space to make some words similar and others not is the only thing that happens during skip-gram training. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** &lt;span class=&quot;math-container&quot;&gt;$\sigma(-x)=1-\sigma(x)$&lt;/span&gt;, so maximizing &lt;span class=&quot;math-container&quot;&gt;$\sigma(-x)$&lt;/span&gt; is minimizing &lt;span class=&quot;math-container&quot;&gt;$\sigma(x)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-20T10:55:55.553" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11342" PostTypeId="1" AcceptedAnswerId="11343" CreationDate="2019-03-20T10:59:56.310" Score="1" ViewCount="2453" Body="&lt;p&gt;I am fine-tuning a VGG16 model on 20 classes with 500k images I was wondering how do you chose the size of the dense layer (the one before the prediction layer which has a size 20). I would prefer not to do a grid search seeing how long it take to train my model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also how many Dense layer should I put after my global average pooling ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;base_model = keras.applications.VGG16(weights='imagenet', include_top=False)&#xA;&#xA;  x = base_model.output&#xA;  x = GlobalAveragePooling2D()(x)&#xA;  x = Dense(???, activation='relu')(x)&#xA;  x = Dropout(0.5, name='drop_fc1')(x)&#xA;  prediction_layer = Dense(class_number, activation='softmax')(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't see particular rules about how its done, are there any ?&#xA;Is it link with the size of the convolution layer ?&lt;/p&gt;&#xA;" OwnerUserId="23107" LastEditorUserId="23107" LastEditDate="2019-03-20T11:07:30.250" LastActivityDate="2021-02-18T12:17:39.980" Title="How to chose dense layer size?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;hidden-layers&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11343" PostTypeId="2" ParentId="11342" CreationDate="2019-03-20T11:20:55.723" Score="3" Body="&lt;p&gt;It's depend more on number of classes. For 20 classes 2 layers 512 should be more then enough. If you want to experiment you can try also  2 x 256 and 2 x 1024. Less then 256 may work too, but you may underutilize power of previous conv layers.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-03-20T11:20:55.723" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11345" PostTypeId="1" CreationDate="2019-03-20T13:47:59.457" Score="1" ViewCount="95" Body="&lt;p&gt;So i have been playing around with &lt;code&gt;neat-python&lt;/code&gt;. I made a program, applying neat, to play pinball on the Atari 2600. The code for that can be found in the file &lt;code&gt;test2.py&lt;/code&gt; &lt;a href=&quot;https://github.com/niallmandal/atari_pinball_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now based on that, I would like to do the same, but on a 2 player game. I have already set up the environment to play a 2 player game, which PONG using OpenAI Retro. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have no clue how to do, is run 2 nets at the same time, on the same observation. The way that &lt;code&gt;neat-python&lt;/code&gt; works, is you get the observation from a single function that goes through each genome and runs the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you create 2 &lt;code&gt;eval_genome&lt;/code&gt; functions that can take in the same observation real-time? This means that they train based off of the same images and environmenrs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Help?&lt;/p&gt;&#xA;" OwnerUserId="23119" LastActivityDate="2021-03-06T17:04:40.447" Title="Running 2 NEAT nets on the same observations" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;neat&gt;&lt;open-ai&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11347" PostTypeId="1" CreationDate="2019-03-20T16:55:39.990" Score="3" ViewCount="116" Body="&lt;p&gt;There are several (family of) algorithms that can be used to cluster a set of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;-dimensional points: for example, &lt;a href=&quot;https://en.wikipedia.org/wiki/K-means_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;k-means&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/K-medoids&quot; rel=&quot;nofollow noreferrer&quot;&gt;k-medoids&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical clustering&lt;/a&gt; (agglomerative or divisive). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is graph-based clustering? Are we clustering the nodes or edges of a graph instead of a set of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;-dimensional (as e.g. in k-means)? Couldn't we just use k-means to also cluster a set of nodes?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-03-21T10:25:47.193" Title="What is graph clustering?" Tags="&lt;unsupervised-learning&gt;&lt;clustering&gt;&lt;k-means&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11348" PostTypeId="2" ParentId="11347" CreationDate="2019-03-20T16:55:39.990" Score="4" Body="&lt;p&gt;In graph clustering, we want to cluster the nodes of a given graph, such that nodes in the same cluster are highly connected (by edges) and nodes in different clusters are poorly or not connected at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple (hierarchical and divisive) algorithm to perform clustering on a graph is based on first finding the &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_spanning_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimum spanning tree&lt;/a&gt; of the graph (using e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Kruskal%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kruskal's algorithm&lt;/a&gt;), &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. It then proceeds in iterations. At each iteration, we remove from &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; the edge with the highest weight. Given that &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tree_(data_structure)&quot; rel=&quot;nofollow noreferrer&quot;&gt;tree&lt;/a&gt;, the removal of an edge from &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; will create a &lt;a href=&quot;http://mathworld.wolfram.com/Forest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;forest&lt;/a&gt; (with connected components). So, after the removal of the edge of highest weight from &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, we will have two connected components. These two connected components will represent two clusters. So, after one iteration, we will have two clusters. At the next iteration, we remove the edge with the second highest weight, and this will create other connected components, and so on, until, possibly, all nodes are in their own cluster (that is, all edges have been removed from &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several limitations of this algorithm. For example, it only considers the edges of the initial graph that are shared with &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; (that is, it only considers the edges of the minimum spanning tree). It also requires the edges of the graph to be weighted. It does not require the number of clusters to be known in advance (like any other hierarchical clustering algorithm), but we still need to choose the optimal number of clusters (after the algorithm has terminated). We can do that in several ways. A way to do it would be to have a threshold value &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; that is used to decide when we should stop removing edges from &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;: more specifically, we will keep removing edges from &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; until the next highest weight is higher than this threshold &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are numerous applications of this type of clustering. For example, we might want to discover certain groups of people in social networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://www.leonidzhukov.net/hse/2016/networks/papers/GraphClustering_Schaeffer07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph clustering&lt;/a&gt; (by Satu Elisa Schaeffer, 2007) provides a readable and quite detailed overview of this field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are algorithms based on k-means that can also work on graphs. See e.g. &lt;a href=&quot;http://digital.csic.es/bitstream/10261/30134/1/Graph-based%20k-means.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph-based k-means Clustering: A Comparison of the Set Median versus the Generalized Median Graph&lt;/a&gt; (by Ferrer et al.).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-21T10:25:47.193" LastActivityDate="2019-03-21T10:25:47.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11350" PostTypeId="1" AcceptedAnswerId="11352" CreationDate="2019-03-20T21:53:38.110" Score="5" ViewCount="1126" Body="&lt;p&gt;I have read a lot about RL algorithms, that update the action-value function at each step with the currently gained reward. The requirement here is, that the reward is obtained after each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a case, where I have three steps, that have to be passed in a specific order. At each step the agent has to make a choice between a range of actions. The actions are specific for each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an example for my problem:&#xA;I want the algorithm to render a sentence of three words. For the first word the agent may choose a word out of &lt;code&gt;['I', 'Trees']&lt;/code&gt;, the second word might be &lt;code&gt;['am', 'are']&lt;/code&gt; and the last word could be chosen from &lt;code&gt;['nice', 'high']&lt;/code&gt;. After the agent has made its choices, the reward is obtained once for the whole sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Does anyone know which algorithms to use in this kind of problem?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To give a bit more detail on what I already tried:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought that using value iteration would be an reasonable approach to test. My problem here is that I don't know how to assign the discounted reward for the chosen operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example after the last choice I get a reward of 0.9. But how do I update the value for the first action (choosing out of I and Trees in my example)?&lt;/p&gt;&#xA;" OwnerUserId="23288" LastEditorUserId="1847" LastEditDate="2019-03-21T07:36:47.693" LastActivityDate="2020-04-10T15:29:02.847" Title="Reinforcement Learning with long term rewards and fixed states and actions" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11351" PostTypeId="2" ParentId="11350" CreationDate="2019-03-20T22:49:12.213" Score="4" Body="&lt;p&gt;You don't need to have a reward on every single timestep, reward at the end is enough. Reinforcement learning can deal with temporal credit assignment problem, all algorithms are designed to work with it. Its enough to define a reward at the end where you, for example, give a reward of 1 if sentence is satisfactory or -1 if it isn't. Regular tabular Q-learning would easily solve the toy problem that you gave as an example.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-03-20T22:49:12.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11352" PostTypeId="2" ParentId="11350" CreationDate="2019-03-21T08:04:15.573" Score="2" Body="&lt;p&gt;You are describing a straightforward Markov Decision Process that could be solved by almost any Reinforcement Learning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have read a lot about RL algorithms, that update the action-value function at each step with the currently gained reward. The requirement here is, that the reward is obtained after each step.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is true. However, the reward value can be zero, and it is quite common to have most states and actions result in zero reward with only some specific ones returning a non-zero value that makes a difference to the goals of the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When this is more extreme - e.g. only one non-zero reward per thousand steps - this is referred to as &quot;sparse rewards&quot; and can be a problem to handle well. Your three steps then reward situation is nowhere near this, and not an issue at all.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I thought that using value iteration would be an reasonable approach to test. My problem here is that I don't know how to assign the discounted reward for the chosen operations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Value iteration should be fine for your test problem, providing the number of choices for words is not too high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way you assign discounted rewards is to use the Bellman equation as an update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s) \leftarrow \text{max}_{a}[\sum_{s',r} p(s',r|s,a)(r + \gamma v(s'))]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a deterministic environment you can simplify the sum, as &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s, a)$&lt;/span&gt; will be 1 for a specific combination of &lt;span class=&quot;math-container&quot;&gt;$s', r$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't matter that &lt;span class=&quot;math-container&quot;&gt;$r = 0$&lt;/span&gt; for the first two steps. The Bellman equation links time steps, by connecting &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v(s')$&lt;/span&gt;. So, over many repetitions of value iteration's main loop, the episode end rewards get copied - with the discount applied - to their predecessor states. Very quickly in your case you will end up with values for the start states.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example after the last choice I get a reward of 0.9. But how do I update the value for the first action (choosing out of I and Trees in my example)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You don't do it directly in a single step. What happens is repeating the value iteration loop copies the best values to their predecessor states, one possible time step at a time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the first loop through all states, you will run an update something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(\text{'I'}) \leftarrow 0 + \gamma v(\text{'I am'})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;span class=&quot;math-container&quot;&gt;$v(\text{'I am'}) = 0$&lt;/span&gt; initially, so it will learn nothing useful in this first loop. However, it will also learn in the same loop:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(\text{'I am'}) \leftarrow 0 + \gamma v(\text{'I am nice'})$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So assuming &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v(\text{'I am nice'}) = 0.9$&lt;/span&gt;, and that &quot;I am high&quot; scores less than 0.9, then it will set &lt;span class=&quot;math-container&quot;&gt;$v(\text{'I am'}) = 0.81$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the next loop through states in value iteration, it will then set &lt;span class=&quot;math-container&quot;&gt;$v(\text{'I'}) = 0.727$&lt;/span&gt; (assuming &quot;I am&quot; beats &quot;I are&quot; for maximum value)&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-21T08:11:38.287" LastActivityDate="2019-03-21T08:11:38.287" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11353" PostTypeId="1" CreationDate="2019-03-21T08:15:52.560" Score="1" ViewCount="387" Body="&lt;p&gt;I'm learning a bit about the use of the &lt;a href=&quot;https://surprise.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;Surprise library&lt;/a&gt; and I have a set of data with users and ratings. I'm training a network with this library, using KNNBasic and KNNWithMeans, this last algorithm is the same as KNN but averages the ratings before calculating the distances between the points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I don't use any measure of similarity, i.e. using the two algorithms with the default parameters, KNNBasic predicts the results better than using KNNWithMeans. But if I train the nets using subsets, 10 folds, where the algorithm iterates over 9 oh them for training and the other one for validating, KNNWithMeans gives better results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know why this can happen? Why KNNBasic is better in the first case, and increasing the number of folds is better KNNWithMeans?&lt;/p&gt;&#xA;" OwnerUserId="23347" LastEditorUserId="23347" LastEditDate="2019-03-22T07:39:38.047" LastActivityDate="2019-03-22T07:39:38.047" Title="KnnBasic vs KnnWithMeans" Tags="&lt;algorithm&gt;&lt;ai-basics&gt;&lt;training&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11355" PostTypeId="2" ParentId="11337" CreationDate="2019-03-21T15:50:24.007" Score="1" Body="&lt;p&gt;I tried to play with your code and found changing loss function to the cross_entropy alternate of negative log-likelihood makes the difference between 2000th epoch's loss and  9000th epoch's loss is greater about 0.2 alternate of 0.09&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to change optimizer and learning rate but no loss didn't improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can explore the &lt;a href=&quot;https://gist.github.com/DevMahmoud10/4d17cc4eb5031b106836349130e9d842&quot; rel=&quot;nofollow noreferrer&quot;&gt;modified code&lt;/a&gt; may help you with another idea&lt;/p&gt;&#xA;" OwnerUserId="21907" LastActivityDate="2019-03-21T15:50:24.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11356" PostTypeId="1" CreationDate="2019-03-21T16:53:19.580" Score="1" ViewCount="72" Body="&lt;p&gt;I have trained a recurrent neural network based on 1 stack of LSTM cells. I use it to solve a classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The RNN cell has 48 hidden states. The output of the last unfolded LSTM cell is linearly projected into two dimensions corresponding to two mutually exclusive classes. I train with softmax cross-entropy loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that both my train and test sets are &lt;em&gt;mislabeled(!)&lt;/em&gt; to a certain extent. Possibly about 10% of items labelled as class 1 are actually class 0. And the same hold in other direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What puzzles me is this. Every time I train the network from scratch and plot a precision recall curve for the validation set in the end. And every time it is different! Especially in the very beginning in the range that corresponds to high precision levels. Why is this happening (this instability)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried various numbers of epochs, training rates, number of hidden states in LSTM. Every time it is the same, but for some combinations of these parameters the variability is less (e.g. 8 out of 10 train/test runs i see more or less the same precision recall curve and 2 times it is severely different and worse).&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2019-03-21T16:53:19.580" Title="Why validation performance is unstable for my LSTM based model (labelling problems)?" Tags="&lt;classification&gt;&lt;training&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11357" PostTypeId="2" ParentId="10658" CreationDate="2019-03-21T17:32:54.910" Score="2" Body="&lt;p&gt;I believe, this &lt;a href=&quot;http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; and this &lt;a href=&quot;http://penkovsky.com/post/neural-networks-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; written by me well address your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, this is a very interesting question that you ask. It deserves some more explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Fully connected networks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more layers you add, the more &quot;nonlinear&quot; your network becomes. For instance, in the case of &lt;a href=&quot;https://www.gwern.net/docs/ai/1988-lang.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;two spirals problem&lt;/a&gt;, which requires a &quot;highly nonlinear separation&quot;, the &lt;a href=&quot;https://www.gwern.net/docs/ai/1988-lang.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;first known architecture&lt;/a&gt; to solve the problem was pretty advanced for that time: it had 3 hidden layers and also it had skip-connections (very early &lt;em&gt;ResNet&lt;/em&gt; in 1988). Back then, computing was a way less powerful and training methods with momentum were not known. Nevertheless, thanks to the multilayer architecture, the problem was solved. &lt;a href=&quot;http://penkovsky.com/post/neural-networks-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;, however, I was able to train a single-hidden layer network to solve the spirals problem using Adam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Convolutional nets (CNNs)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting partial case of neural networks are CNNs. They restrict the architecture of the first layers, known as &lt;em&gt;convolutional&lt;/em&gt; layers, so that there is a much smaller number of trainable parameters due to the weights sharing. What we have learned from computer vision, moving towards the end of CNNs layers, their receptive fields become larger. That means that the subsequent CNN layers &quot;see&quot; more than their predecessors. Conceptually, first CNN layers can recognize simpler features such as edges and textures, whereas final CNN layers contain information about more abstract objects such as trees or faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Recurrent nets (RNNs)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNNs are networks with layers which receive some of their outputs as inputs. Technically, a single recurrent layer is equivalent to an infinite (or at least large) number of ordinary layers. Thanks to that recurrence, RNNs retain an internal state (memory). Therefore, it is much more difficult to answer your question in the case of recurrent nets. What is known, due to their memory, RNNs are more like programs, and thus are in principle more complex than other neural networks. Please let me know if you find an answer to your question in the last case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the higher number of hidden layers may help to structure a neural network. Thanks to the recent developments such as &lt;a href=&quot;https://arxiv.org/abs/1512.03385&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNets&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation_through_time&quot; rel=&quot;nofollow noreferrer&quot;&gt;backpropagation through time&lt;/a&gt;, it is possible to train neural networks with a large number of hidden layers.&lt;/p&gt;&#xA;" OwnerUserId="23360" LastEditorUserId="23360" LastEditDate="2019-03-21T20:34:32.980" LastActivityDate="2019-03-21T20:34:32.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11358" PostTypeId="2" ParentId="11326" CreationDate="2019-03-21T17:37:10.530" Score="2" Body="&lt;p&gt;The usual way to solve this kind of problem is to construct a &lt;em&gt;configuration space&lt;/em&gt;: extruding all the polygonal obstacles by sliding the polygon corresponding to the robot around them (&lt;a href=&quot;https://www.cs.cmu.edu/~motionplanning/lecture/Chap3-Config-Space_howie.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;some slides&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exterior vertices of the configuration space can then be used as input to a path-planning algorithm, such as A*.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2019-03-21T17:37:10.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11359" PostTypeId="2" ParentId="11290" CreationDate="2019-03-21T17:56:33.007" Score="3" Body="&lt;p&gt;To the best of my knowledge, there haven't yet been many academic publications in this area, which could be broadly said to fall within &lt;em&gt;Search-Based Software Engineering&lt;/em&gt;. Here are the ones I know of.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Jerry Swan and Nathan Burles. &lt;em&gt;Templar - A Framework for Template-Method Hyper-Heuristics&lt;/em&gt;. In: Genetic Programming - 18th European Conference, EuroGP 2015, Copenhagen, Denmark, April 8-10, 2015, Proceedings. 2015, pp. 205–216. DOI: 10.1007/978-3-319-16501-1_17.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This paper describes 'Hyper-quicksort', a quicksort variant that uses Machine Learning (ML) to generate a pivot function:&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A. E. I. Brownlee, N. Burles, and J. Swan. &lt;a href=&quot;http://eprints.whiterose.ac.uk/117916/1/07935484_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Search-Based Energy Optimization of Some Ubiquitous Algorithms&lt;/em&gt;&lt;/a&gt;. In: IEEE Transactions on Emerging Topics in Computational Intelligence 1.3 (2017), pp. 188–201. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This paper uses ML to generate energy-efficient variants of some widely-used algorithms&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;David R. White, Leonid Joffe, Edward Bowles, and Jerry Swan. &lt;em&gt;&lt;a href=&quot;http://eprints.whiterose.ac.uk/117963/1/optimizing_akka_concurrency.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Parameter Tuning of Concurrent Divide and Conquer Algorithms in Akka&lt;/a&gt;&lt;/em&gt;. ISBN: 978-3-319-55792-2.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This paper uses ML to optimise the FFT, matrix multiplication and quicksort for concurrency&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Nathan Burles, Edward Bowles, Alexander E. I. Brownlee, Zoltan A. Kocsis, Jerry Swan, and Nadarajen Veerapen. &lt;em&gt;Object-Oriented Genetic Improvement for Improved Energy Consumption in Google Guava&lt;/em&gt;. DOI: 10.1007/978-3-319-22183-0_20.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This one optimises Google Guava for energy consumption&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Zoltan A. Kocsis, Geoff Neumann, Jerry Swan, Michael G. Epitropakis, Alexander E. I. Brownlee, Saemundur O. Haraldsson, and Edward Bowles. &lt;em&gt;&lt;a href=&quot;https://www.researchgate.net/publication/264155239_Repairing_and_Optimizing_Hadoop_hashCode_Implementations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Repairing and Optimizing Hadoop hashCode Implementations&lt;/a&gt;&lt;/em&gt;. In: Search-Based Software Engineering: 6th International Symposium, SSBSE 2014, DOI: 10.1007/978-3-319-09940-8_22.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This one fixes broken hashCodes by using ML to generate new ones&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There was also one (from Microsoft Research, I think) entitled something like &quot;The case for self-adjusting data structures&quot;. I'll add an edit if I can find it.&lt;/p&gt;&#xA;" OwnerUserId="42" LastEditorUserId="2444" LastEditDate="2019-08-18T22:24:15.890" LastActivityDate="2019-08-18T22:24:15.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11360" PostTypeId="1" CreationDate="2019-03-21T18:23:47.867" Score="1" ViewCount="66" Body="&lt;p&gt;Suppose that we have unlabeled data. That is, all we have are a collection of emails and want to determine whether any of them is spam or not. Let's say we have &lt;span class=&quot;math-container&quot;&gt;$1,000$&lt;/span&gt; rules to determine whether a particular email is spam or not. For example, one rule could be that a sender's email address should not contain the text &lt;code&gt; no_reply &lt;/code&gt;. If an email does contain this text, then it would be classified as spam.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What are the advantages/disadvantages of a rules-based approach for detecting spam vs. a non-rules-based approach/unsupervised methods for detecting spam?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Would there even be a point in constructing a non-rules based model given that we already have a rules-based model? Could we use the rules-based model to create some labeled training data and then apply supervised techniques?&lt;/p&gt;&#xA;" OwnerUserId="23362" LastEditorUserId="2444" LastEditDate="2020-12-11T14:33:21.380" LastActivityDate="2021-05-10T16:00:55.087" Title="Are there any advantages of using rules-based approaches versus models for detecting spam?" Tags="&lt;comparison&gt;&lt;unsupervised-learning&gt;&lt;anomaly-detection&gt;&lt;binary-classification&gt;&lt;rule-based-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11361" PostTypeId="1" AcceptedAnswerId="11362" CreationDate="2019-03-21T21:38:12.243" Score="2" ViewCount="187" Body="&lt;p&gt;I want to solve a problem using Reinforcement Learning on a 20x20 board. An agent (a mouse) has to get the highest possible rewards as fast as possible by collecting cheese, which there are 10 in total. The agent has a fixed amount of time for solving this problem, namely 500 steps per game. The problem is, the cheeses will be randomly assigned to the fields on the board, the agent knows however, where the cheeses is located.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way how this could be solved using only Reinforcement Learning (and not training for an absurd amount of time)? Or is the only way to solve this problem to use algorithms like the A*-algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried many different (deep)-q-learning models, but it always failed miserably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I could not get any meaningful behavior after 6 hours of learning while using a GTX 950M. Maybe my implementation was off, but i don't think so. &lt;/p&gt;&#xA;" OwnerUserId="23005" LastEditorUserId="23005" LastEditDate="2019-03-21T22:19:07.793" LastActivityDate="2019-03-22T08:10:57.177" Title="Can Reinforcement Learning solve problems, where certain elements in the environement are randomly located?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11362" PostTypeId="2" ParentId="11361" CreationDate="2019-03-21T21:54:37.797" Score="2" Body="&lt;p&gt;Yes you can use RL for this. The trick is to include the location of the cheese as part of the state description. So as well as up to 400 states for the mouse location, you have (very roughly) &lt;span class=&quot;math-container&quot;&gt;$400^{10}$&lt;/span&gt; possible cheese locations, meaning you have &lt;span class=&quot;math-container&quot;&gt;$400^{11}$&lt;/span&gt; states in total. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you are going to want some function approximation if you want to use RL - you would probably train using a convolutional neural network, with an &quot;image&quot; of the board including mouse and cheese positions, plus DQN to select actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viewed like this, a game where the mouse tries to get the cheese in minimal time seems on the surface a lot simpler than many titles in the Atari game environments, which DQN has been shown to solve well for many games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would probably use two image channels - one for mouse location and one for cheese location. A third channel perhaps for walls/obstacles if there are any.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or is the only way to solve this problem to use algorithms like the A*-algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A* plus some kind of sequence optimisation like a travelling salesman problem (TSP) solver would probably be optimal if you have been presented the problem and asked to solve it any way you want. With only 11 locations to resolve - mouse start plus 10 cheese locations - then you can brute force the movement combinations in a few seconds on a modern CPU, so that part may not be particularly exciting (whilst TSP solvers can get more involved and interesting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interesting thing about RL is &lt;em&gt;how&lt;/em&gt; it will solve the problem. RL is a learning algorithm - the purpose of implementing it is to see what it takes for the machine to gain knowledge towards a solution. Whilst A* and combinatorial optimisers are where &lt;em&gt;you&lt;/em&gt; have knowledge of how to solve the problem and do so as optimally as possible based on a higher level analysis. The chances are high that an A*/optimiser solution would be more robust, quicker to code, and quicker to run than a RL solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is nothing inherently wrong with either approach, if all you want to do is solve the problem at hand. It depends on your goals for why you are bothering with the problem in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could even combine A* and RL if you really wanted to. A* to find the paths, then RL to decide best sequence using the paths as part of the input to the CNN. The A* analysis of routes would likely help the RL stage a lot - add them as one or more additional channels.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-21T22:12:00.300" LastActivityDate="2019-03-21T22:12:00.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11363" PostTypeId="1" AcceptedAnswerId="11625" CreationDate="2019-03-21T22:24:49.743" Score="2" ViewCount="391" Body="&lt;p&gt;I know the difference between &lt;em&gt;content-based&lt;/em&gt; and &lt;em&gt;collaborative filtering&lt;/em&gt; approach in recommender systems. I also know some of the articles said collaborative filtering have some advantages than content-based, some of them also suggest to use both method (hybrid) to make a better system recommendation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a specific case where the use of one method (content-based, specifically) is better than another? Because if there is no case at all, why both methods are considered to be on the same &quot;level&quot;, why not focus on just one method? For example, focus on collaborative filtering or hybrid method (as an extension for collaborative filtering).&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2019-04-03T08:30:26.390" LastActivityDate="2019-04-03T15:16:10.700" Title="When is content-based more appropriate than collaborative filtering?" Tags="&lt;research&gt;&lt;definitions&gt;&lt;comparison&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11364" PostTypeId="2" ParentId="9470" CreationDate="2019-03-22T01:11:52.597" Score="1" Body="&lt;p&gt;All answers above explain Yolo and Keras relation very well, I just want to add minor information. Yolo V3 comes in several different models. The faster the model, it has lower accuracy and the slower the model, it has better accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can simply choose which model is the most suitable for you (trade off between accuracy and speed)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tdoVx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tdoVx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download the cfg file and the weights for each model on official website : &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/darknet/yolo/&lt;/a&gt; . &lt;br&gt;&#xA;Or you can use AlexeyAb's repo which the most popular forked darknet version with several improvements : &lt;a href=&quot;https://github.com/AlexeyAB/darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/AlexeyAB/darknet&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20612" LastActivityDate="2019-03-22T01:11:52.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11366" PostTypeId="1" CreationDate="2019-03-22T01:38:14.430" Score="1" ViewCount="36" Body="&lt;p&gt;I'm a Rails developer with a lot of web experience, but none (still) in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working in a web text editor that judges use to writing their sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to start to use AI to help the judge rule the case, either based on his own previous rulings, either based on his colleagues rulings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The judge would provide the text for the plaintiffs and defendants petitions, and based on these two inputs the system would recommend previous rulings that apply for the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have a considerable dataset of judges rulings inside the database, and they can be easily attached to the plaintiffs and defendants petitions for training (so this plaintiff petition + this defendant petition = this ruling). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is specially challenging because the complaints can contain different subjects combined into the same petition; but the fact is that many offices use the same standardized petitions, as the defendants do as well, so I think the system can have a great chance of prediction success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms or strategies should I start studying to tackle this problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any similar articles, white papers or repositories that could help in my goal?&lt;/p&gt;&#xA;" OwnerUserId="23370" LastEditorUserId="23370" LastEditDate="2019-03-22T02:29:19.187" LastActivityDate="2019-03-22T08:09:34.633" Title="Algorithms and strategies to help judges rule cases" Tags="&lt;natural-language-processing&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11369" PostTypeId="2" ParentId="11361" CreationDate="2019-03-22T07:44:47.067" Score="0" Body="&lt;p&gt;A assume here OP is familiar with DQN basics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Standard&quot; way to solve this problem with Deep RL would be using convolutional DQN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make net from 2 to 4 convolutional layers with 1-2 fully connected on top. &#xA;The trick here is how you output Q-values. Input is board with cheese, without information about mouse. Instead net should output Q for every action from every field on the board (every possible position of mouse), that mean you output 20x20x(number_of_moves from the field) Q values. That would make net quite big, but that is most reliable way for DQN. For each move form the replay buffer only one Q value updated (gradient produced) with Time Difference equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because only one value form 20x20x(number_of_moves) updated per sample you need quite big replay buffer and a lot of training. After each episod (complete game) cheese should be randomly redistributed. Episodes should be mixed up in replay buffer, training on 1 episod is a big No.&#xA;Hopefully that should at least give you direction in which do research/development. Warning: DQN is slow to train, and with such big action space (20 x 20 x number_of_moves) could require million or tens of millions of even more moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you don't want such big action space, is to use actor-critic architecture (or policy gradient, actor-critic  is a kind of policy gradient). Actor-critic network have small action space, with only number_of_moves outputs. On the down size complexity of method is much higher, and behavior could be difficult to predict or analyze. However if action space is too big it could be preferable solution. Practical issues and implementations of actor-critic is too huge area to go in depth here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: There is another way with lesser action space for DQN, but somehow less reliable and possibly more slow: shift the board in such way that mouse is in the center of the board and pad invalid parts with zero (size of the new board should be x2). In that case only number_of_moves should be in output. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-03-22T08:10:57.177" LastActivityDate="2019-03-22T08:10:57.177" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11370" PostTypeId="2" ParentId="11366" CreationDate="2019-03-22T08:09:34.633" Score="2" Body="&lt;p&gt;Genuine success in this area would be beyond the state-of-the-art in research, since it likely requires analogising from relational knowledge extracted from text. In recent years, techniques for working with natural language have tended to be statistical, and are therefore somewhat deficient in this respect. You could look at 'bag of words'/latent semantic analysis approaches, but they are likely to generate many false positives unless a lot of ad hoc conditions are added manually. More recent work on 'treenets' &lt;a href=&quot;https://www.ijcai.org/proceedings/2018/557&quot; rel=&quot;nofollow noreferrer&quot;&gt;(paper here)&lt;/a&gt; is more structurally informed, but is still a relatively new area.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2019-03-22T08:09:34.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11371" PostTypeId="2" ParentId="11360" CreationDate="2019-03-22T09:02:50.930" Score="0" Body="&lt;p&gt;Yes, there would be a point. Assuming your rule set is accurate, then you can use data classified with it to train a model. This model can be expected to be more robust and properly categorise emails that your rule-set will not handle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why? Machine learning algorithms generally work on features, and identify relationships between those features that lead to a classification decision. A human rule author basically does the same, but they might not notice subtle relationships; a good ML algorithm, however, might pick those up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you could have a hybrid model, where you first use your rule-based classifier, and anything that does not get classified is then run through the ML classifier.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-03-22T09:02:50.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11373" PostTypeId="1" AcceptedAnswerId="11379" CreationDate="2019-03-22T09:07:26.303" Score="2" ViewCount="802" Body="&lt;p&gt;Can Deep Learning be applied to Computational Fluid Dynamics (CFD) to develop turbulence models that are less computationally expensive compared to traditional CFD modeling?&lt;/p&gt;&#xA;" OwnerUserId="23276" LastActivityDate="2019-03-22T14:42:35.523" Title="Can Deep Learning be applied to Computational Fluid Dynamics" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11374" PostTypeId="1" AcceptedAnswerId="11391" CreationDate="2019-03-22T10:45:48.367" Score="7" ViewCount="161" Body="&lt;p&gt;I've started working on anomaly detection in Python. My dataset is a time series one. The data is being collected by some sensors which record and collect data on semiconductor making machines.&lt;/p&gt;&#xA;&lt;p&gt;My dataset looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ContextID   Time_ms Ar_Flow_sccm    BacksGas_Flow_sccm&#xA;7289973 09:12:48.502    49.56054688 1.953125&#xA;7289973 09:12:48.603    49.56054688 2.05078125&#xA;7289973 09:12:48.934    99.85351563 2.05078125&#xA;7289973 09:12:49.924    351.3183594 2.05078125&#xA;7289973 09:12:50.924    382.8125    1.953125&#xA;7289973 09:12:51.924    382.8125    1.7578125&#xA;7289973 09:12:52.934    382.8125    1.7578125&#xA;7289999 09:15:36.434    50.04882813 1.7578125&#xA;7289999 09:15:36.654    50.04882813 1.7578125&#xA;7289999 09:15:36.820    50.04882813 1.66015625&#xA;7289999 09:15:37.904    333.2519531 1.85546875&#xA;7289999 09:15:38.924    377.1972656 1.953125&#xA;7289999 09:15:39.994    377.1972656 1.7578125&#xA;7289999 09:15:41.94     388.671875  1.85546875&#xA;7289999 09:15:42.136    388.671875  1.85546875&#xA;7290025 09:18:00.429    381.5917969 1.85546875&#xA;7290025 09:18:01.448    381.5917969 1.85546875&#xA;7290025 09:18:02.488    381.5917969 1.953125&#xA;7290025 09:18:03.549    381.5917969 14.453125&#xA;7290025 09:18:04.589    381.5917969 46.77734375&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I have to do is to apply some unsupervised learning technique on each and every parameter column individually and find any anomalies that might exist in there. The &lt;code&gt;ContextID&lt;/code&gt; is more like a product number.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know which unsupervised learning techniques can be used for this kind of task at hand, since the problem is a bit unique:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It has temporal values&lt;/li&gt;&#xA;&lt;li&gt;Since it has temporal values, each product will have many (similar or different) values as it can be seen in the dataset above.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23380" LastEditorUserId="2444" LastEditDate="2020-09-18T10:12:14.023" LastActivityDate="2020-09-18T10:12:14.023" Title="Which unsupervised learning technique can be used for anomaly detection in a time series?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;time-series&gt;&lt;anomaly-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11375" PostTypeId="1" AcceptedAnswerId="23427" CreationDate="2019-03-22T10:57:57.820" Score="11" ViewCount="5951" Body="&lt;p&gt;Coming from a process (optimal) control background, I have begun studying the field of deep reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;Sutton &amp;amp; Barto (2015) state that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;particularly important (to the writing of the text) have been the contributions establishing and developing the relationships to the theory of optimal control and dynamic programming&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;With an emphasis on the elements of reinforcement learning - that is, policy, agent, environment, etc., what are the key differences between (deep) RL and optimal control theory?&lt;/p&gt;&#xA;&lt;p&gt;In optimal control we have, controllers, sensors, actuators, plants, etc, as elements. Are these different names for similar elements in deep RL? For example, would an optimal control plant be called an environment in deep RL?&lt;/p&gt;&#xA;" OwnerUserId="23276" LastEditorUserId="2444" LastEditDate="2021-02-12T20:03:23.763" LastActivityDate="2021-02-12T20:03:23.763" Title="What is the difference between reinforcement learning and optimal control?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;sutton-barto&gt;&lt;control-theory&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11376" PostTypeId="2" ParentId="3942" CreationDate="2019-03-22T11:03:41.840" Score="1" Body="&lt;p&gt;Predicting what happens post-singularity is simply not possible as we cannot attempt to model let alone conceptualise a mind far more complex than ours. If that is a difficult concept to get your head around, consider how far an insect's central nervous system could go in understanding human behaviour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That fact alone is an argument against the likelihood of success for attempting any type of control. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in terms of 'defending' against a post-singularity mind well before it happens (ie now) there are 2 solutions, with only the first offering a good likelihood of success albeit only for as long as everyone cooperates: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;identify the technology types that are anticipated to enable the singularity,  register them as 'instruments of human extinction' and regulate them accordingly ; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ensure technology in human augmentation is sufficiently advanced&#xA;to enable human-mediated guidance/fusion during the exponential rise&#xA;in computation that will occur prior to the singularity event.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In any case, as mentioned, it is impossible to predict the behaviour of a post-singularity mind and even a human hybrid will similarly be unpredictable due to its exponentially increased cognitive/computational complexity.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting consideration is the possibility that numerous singularity-level minds have already spawned in other parts of the universe (based on the likelihood of other civilisations a) existing and b) reaching that level of technological advancement).     &lt;/p&gt;&#xA;" OwnerUserId="23379" LastActivityDate="2019-03-22T11:03:41.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11379" PostTypeId="2" ParentId="11373" CreationDate="2019-03-22T14:29:29.563" Score="1" Body="&lt;p&gt;Read the paper &lt;a href=&quot;https://www.cambridge.org/core/journals/journal-of-fluid-mechanics/article/deep-learning-in-fluid-dynamics/F2EDDAB89563DE5157FC4B8342AD9C70/core-reader&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep learning in fluid dynamics&lt;/a&gt; (by J. Nathan Kutz), and you will find your answer.&lt;/p&gt;&#xA;" OwnerUserId="22603" LastEditorUserId="2444" LastEditDate="2019-03-22T14:42:35.523" LastActivityDate="2019-03-22T14:42:35.523" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11380" PostTypeId="1" CreationDate="2019-03-22T14:44:10.680" Score="2" ViewCount="29" Body="&lt;p&gt;Geoffrey Hinton's Coursera MOOC was recently discontinued:&#xA;&lt;a href=&quot;https://twitter.com/geoffreyhinton/status/1085325734044991489?lang=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/geoffreyhinton/status/1085325734044991489?lang=en&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The videos however are still available at both on Youtube and on Hinton's webpage:&#xA;&lt;a href=&quot;https://www.cs.toronto.edu/~hinton/coursera_lectures.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.toronto.edu/~hinton/coursera_lectures.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in his MOOC Hinton also had some papers as required (or recommended, I can't really remember) readings after each lecture. They were generally old, seminal papers which provided some good insights and intuitions and were worth reading IMHO for learning purposes. I couldn't find these papers as a reading list online. Does anyone have this list?&lt;/p&gt;&#xA;" OwnerUserId="23386" LastActivityDate="2019-03-22T14:44:10.680" Title="Hinton's reading list from the removed Coursera MOOC" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11381" PostTypeId="1" AcceptedAnswerId="11440" CreationDate="2019-03-22T15:11:47.503" Score="4" ViewCount="78" Body="&lt;p&gt;&lt;a href=&quot;https://amturing.acm.org/award_winners/pearl_2658896.cfm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Judea Pearl won the 2011 Turing Award&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For fundamental contributions to artificial intelligence through the development of a calculus for probabilistic and causal reasoning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He is credited with the invention of Bayesian networks and a framework for causal inference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why should we study causation and causal inference in artificial intelligence? How would causation integrate into other topics like machine learning? There are facts or relations that cannot be retrieved from data (e.g. cause-effect relations), which is the driving force of ML. Are there other reasons for studying causation?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-11T20:44:59.843" Title="Why should we study causation in artificial intelligence?" Tags="&lt;causation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11382" PostTypeId="2" ParentId="9296" CreationDate="2019-03-22T15:58:04.450" Score="2" Body="&lt;p&gt;Biological organisms (such as animals or plants) are the main examples of &lt;em&gt;intelligent&lt;/em&gt; systems that we are aware of (excluding artificially intelligent systems, so as not to discuss whether current AI systems are really intelligent or not). Consequently, biological life is often an inspiration for AI researchers to develop AI systems.&lt;/p&gt;&#xA;&lt;p&gt;There are numerous examples of AI systems that have been introduced  (at least, partially) &lt;em&gt;based on&lt;/em&gt; or just &lt;em&gt;inspired by&lt;/em&gt; the biology. Here are a few examples.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Reinforcement learning&lt;/strong&gt; is based on a similar way that animals (such as dogs or &lt;a href=&quot;https://www.youtube.com/watch?v=QKSvu3mj-14&quot; rel=&quot;nofollow noreferrer&quot;&gt;pigeons&lt;/a&gt;) can learn. For more details, see &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto's book&lt;/a&gt; (especially chapters 14 and 15).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Artificial neural networks&lt;/strong&gt; are very approximative models of human neural networks.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Genetic algorithms&lt;/strong&gt; are roughly based on Darwin's theory of evolution.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Ant colony optimization&lt;/strong&gt; algorithms (and, in general, &lt;strong&gt;swarm intelligence&lt;/strong&gt;) are based on the way real ants (and, respectively, &lt;a href=&quot;https://en.wikipedia.org/wiki/Swarm_behaviour&quot; rel=&quot;nofollow noreferrer&quot;&gt;biological swarms&lt;/a&gt;) behave. (There is even &lt;a href=&quot;https://www.youtube.com/watch?v=XeovS8-FauM&amp;amp;ab_channel=PhoeneticComposition-Topic&quot; rel=&quot;nofollow noreferrer&quot;&gt;a rap song dedicated to ants&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There are probably other examples that don't come to my mind right now. See also &lt;a href=&quot;https://ai.stackexchange.com/q/20540/2444&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/258/2444&quot;&gt;this&lt;/a&gt; questions.&lt;/p&gt;&#xA;&lt;p&gt;There are cases where AI discoveries have also helped the development of biology or related fields (such as neuroscience and psychology). For instance, &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=26&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto (on page 4)&lt;/a&gt; write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Of all the forms of machine learning, reinforcement learning is the closest to the kind of learning that humans and other animals do, and many of the core algorithms of reinforcement learning were originally inspired by biological learning systems. Reinforcement learning has also given back, both through a psychological model of animal learning that better matches some of the empirical data, and through an influential model of parts of the brain's reward system.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T17:21:35.017" LastActivityDate="2020-11-16T17:21:35.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11386" PostTypeId="2" ParentId="9296" CreationDate="2019-03-22T17:36:44.880" Score="1" Body="&lt;h3&gt;Evolutionary game theory and evolutionary algorithms&lt;/h3&gt;&#xA;&lt;p&gt;I see the connection arising mostly thought &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Game Theory&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Algorithms&lt;/a&gt;.  Evolutionary algorithms are an analog of natural selection, where successive generations of a given decision making agent are more optimized than previous generations.  Like organisms in nature, this process uses &amp;quot;reproduction, mutation, recombination and &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_selection&quot; rel=&quot;nofollow noreferrer&quot;&gt;selection&lt;/a&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;There are a couple of recent articles from Quanta Magazine.  One, &lt;a href=&quot;https://www.quantamagazine.org/the-math-that-tells-cells-what-they-are-20190313/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;The Math That Tells Cells What They Are&amp;quot;&lt;/a&gt; discusses mathematical optimization as the core function of fundamental biological systems.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Through evolution, these cells have figured out how to implement &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayes' trick&lt;/a&gt; using regulatory DNA.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Natural selection [seems to be] pushing the system hard enough so that it ... reaches a point where the cells are performing at the limit of what physics allows.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This second quote is exactly the goal of Artificial Intelligence, where utility is limited by physics (computing resources).  One way for an algorithm to increase utility is to increase computing power, but the other method is to refine the algorithm to make strong decisions more efficiently.  (&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCTS&lt;/a&gt; vs. &lt;a href=&quot;https://en.wikipedia.org/wiki/Brute-force_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brute Force&lt;/a&gt; where a model is &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractable&lt;/a&gt;, as an example.)&lt;/p&gt;&#xA;&lt;p&gt;A second article &lt;a href=&quot;https://www.quantamagazine.org/computer-science-and-biology-explore-algorithmic-evolution-20181129/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Mathematical Simplicity May Drive Evolution’s Speed&amp;quot;&lt;/a&gt; talks about &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithms&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Creationists love to insist that evolution had to assemble upward of 300 amino acids in the right order to create just one medium-size human protein. With 20 possible amino acids to occupy each of those positions, there would seemingly have been more than 20300 possibilities to sift through, a quantity that renders the number of atoms in the observable universe inconsequential.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The game of Go on a 19x19 board has a similar quality--the number of potential gamestates is vastly exceeds the number of atoms in the universe, and, even if the entire universe were converted to &lt;a href=&quot;https://en.wikipedia.org/wiki/Computronium&quot; rel=&quot;nofollow noreferrer&quot;&gt;computronium&lt;/a&gt;, the game would still be intractable.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The fatal flaw in their argument is that evolution didn’t just test sequences randomly: The process of natural selection winnowed the field. Moreover, it seems likely that nature somehow also found other shortcuts, ways to narrow down the vast space of possibilities to smaller, explorable subsets more likely to yield useful solutions.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This would also be an accurate description the process of &lt;a href=&quot;https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;pruning a search space&lt;/a&gt;.  The article concludes that, although there is still much research to be conducted:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;“The idea of thinking about life as evolving software is fertile.”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The process of optimization in nature and in computer science is similar in spirit, if not in fact.&lt;/p&gt;&#xA;&lt;h3&gt;Automata as a form of artificial life&lt;/h3&gt;&#xA;&lt;p&gt;The second factor may arise out of the mythology of AI, via speculative fiction.  In science fiction, the idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life&quot; rel=&quot;nofollow noreferrer&quot;&gt;automata as a form of artificial life&lt;/a&gt; is persistent.  Shows &amp;amp; films like &lt;a href=&quot;https://en.wikipedia.org/wiki/Westworld_(TV_series)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Westworld&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Blade_Runner_2049#Plot&quot; rel=&quot;nofollow noreferrer&quot;&gt;BladeRunner&lt;/a&gt;, and the Alien franchise, with &lt;a href=&quot;https://en.wikipedia.org/wiki/David_8&quot; rel=&quot;nofollow noreferrer&quot;&gt;David the Android&lt;/a&gt; as a prime example of a superior, artificial species that may supplant humanity, are extremely popular. These are all based on Phillip K Dick's ideas explicated in &lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;Do Androids Dream of Electric Sheep&lt;/a&gt;, the plot of which turns on evolutionary game theory, &lt;em&gt;written about 5 years before the field was formalized!&lt;/em&gt; (Dick's influence can even be seen in Google's &lt;a href=&quot;https://bladerunner.fandom.com/wiki/Nexus-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Nexus&amp;quot;&lt;/a&gt; naming convention for their Android phone;)  Underneath all of this is also the idea that Artificial Intelligence itself is a function of nature, with humans as merely the vehicle for the next form of dominant life.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-11-17T11:24:57.520" LastActivityDate="2020-11-17T11:24:57.520" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11387" PostTypeId="2" ParentId="7838" CreationDate="2019-03-22T19:24:37.550" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is artificial intelligence?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This question is ambiguous. I will address the two less ambiguous but related questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the goal of the AI field?&lt;/li&gt;&#xA;&lt;li&gt;What is &lt;strong&gt;an&lt;/strong&gt; artificial intelligence?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;What is the goal of the AI field?&lt;/h2&gt;&#xA;&lt;p&gt;In the article &lt;a href=&quot;http://www-formal.stanford.edu/jmc/whatisai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is artificial intelligence?&lt;/a&gt; (2007), John McCarthy, one of the founders of artificial intelligence and who also coined the expression &lt;em&gt;artificial intelligence&lt;/em&gt;, writes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Artificial intelligence&lt;/strong&gt; is the science and engineering of making intelligent machines, especially intelligent computer programs.&lt;/p&gt;&#xA;&lt;p&gt;It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Therefore, the goal of the AI field is to create &lt;em&gt;intelligent&lt;/em&gt; programs (or machines). So, he defines the goal of the field based on the concept of &lt;em&gt;intelligence&lt;/em&gt;, which he defines as follows.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Intelligence&lt;/strong&gt; is the computational part of the ability to achieve goals in the world. Varying kinds and degrees of intelligence occur in people, many animals and some machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, we could conclude that the goal of the AI field is to create programs (or machines) that achieve goals in the world to different extents.&lt;/p&gt;&#xA;&lt;p&gt;This definition is of intelligence is reasonable, but maybe not formal and rigorous enough. In &lt;a href=&quot;https://ai.stackexchange.com/a/13100/2444&quot;&gt;this answer&lt;/a&gt;, I report a possibly more sound definition of intelligence given by Hutter and Legg.&lt;/p&gt;&#xA;&lt;h2&gt;What is an artificial intelligence?&lt;/h2&gt;&#xA;&lt;p&gt;Nowadays, most people distinguish two types of artificially intelligent systems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Narrow AI&lt;/a&gt; (aka weak AI): a system that solves a very specific problem (e.g. playing go)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.scholarpedia.org/article/Artificial_General_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial general intelligence&lt;/a&gt; (aka strong AI): a system that can solve multiple problems&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This distinction started with philosophical arguments, such as &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Chinese room argument&lt;/a&gt;, where the ability of a computer to &lt;em&gt;understand&lt;/em&gt; the actual problem was questioned. Nowadays, there are multiple successful cases of narrow AIs (e.g. AlphaGo), but there isn't yet a truly AGI system. This is mainly due to the fact that more people have been (probably wisely) focusing on solving specific problems rather than solving the &amp;quot;holy grail&amp;quot; problem of the AI field, i.e. create an AGI, which seems to be a lot more difficult than creating narrow AI systems. (Anyway, the creation of an AGI could actually arise from solutions to  these specific problems, so maybe we are already creating the tools needed to build an AGI, without realizing it). See &lt;a href=&quot;https://ai.stackexchange.com/q/74/2444&quot;&gt;What is the difference between strong-AI and weak-AI?&lt;/a&gt; for more details about the difference between narrow AI and strong AI.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-30T20:36:08.370" LastActivityDate="2020-07-30T20:36:08.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11388" PostTypeId="1" CreationDate="2019-03-22T20:42:21.047" Score="2" ViewCount="63" Body="&lt;p&gt;Do we have to consider if (s is given) an action a can lead to s' when defining a reward function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific:&#xA;Let's say I have a 1D Map like: &lt;code&gt;|A|B|C|D|&lt;/code&gt;&#xA;To define a reward function, I simply defined a matrix for every action, where the columns and rows represent the states (A-D) and the entries represent the reward. But I made it even simpler. As reaching a specific state gives a reward of 1 I just assigned the reward to a column (C). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{matrix} -1&amp;amp;0&amp;amp;1&amp;amp;0\\ -1&amp;amp;0&amp;amp;1&amp;amp;0\\-1&amp;amp;0&amp;amp;1&amp;amp;0\\-1&amp;amp;0&amp;amp;1&amp;amp;0 \end{matrix}&#xA;$$&lt;/span&gt;&#xA;However, lets say the matrix is specified for the action &quot;going right&quot;. Now there are entries reading: &lt;code&gt;D ==&amp;gt; going right ==&amp;gt; C getting reward of 1&lt;/code&gt;. This is actually not possible. However, I thought the transition function would handle this issue since I will define there what is possible and what is not. But anyway it is said that for a horizon of one the immidiate reward is given and the transition function isn't even considered. This leads to arbitrary result. So do I have to consider the &quot;physics&quot; of my world?&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="19413" LastEditDate="2019-03-22T23:10:07.127" LastActivityDate="2021-04-11T15:01:10.120" Title="Do we have to consider the feasability of an action when defining the reward function of a MDP?" Tags="&lt;rewards&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11389" PostTypeId="1" AcceptedAnswerId="11400" CreationDate="2019-03-22T21:54:33.107" Score="3" ViewCount="51" Body="&lt;p&gt;Consider an &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterative deepening search&lt;/a&gt; using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Transposition_table&quot; rel=&quot;nofollow noreferrer&quot;&gt;transposition table&lt;/a&gt;. Whenever the transposition table is full, what are common strategies applied to replace entries in the table?&lt;/p&gt;&#xA;&lt;p&gt;I'm aware of two strategies:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace the oldest entry whenever a new one is found.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace an entry if the new one has a higher depth.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm curious about other replacement approaches.&lt;/p&gt;&#xA;" OwnerUserId="22369" LastEditorUserId="2444" LastEditDate="2021-02-13T14:07:02.397" LastActivityDate="2021-02-13T14:07:02.397" Title="What are the common techniques one could use to deal with collisions in a transposition table?" Tags="&lt;search&gt;&lt;dynamic-programming&gt;&lt;iddfs&gt;&lt;transposition-table&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11390" PostTypeId="1" CreationDate="2019-03-23T02:00:07.433" Score="1" ViewCount="28" Body="&lt;p&gt;I'm working on a project where I need to forecast sales data where I have history of 1 year (2017) daily data. I am new on Artificial Intelligence topic and after searching for a while, I think ARIMA or Multiple Linear Regression is a good model for seasonal forecasting (Correct Me If I'm Wrong). But then I think that my history data is exclusive for 2017 because on 2018 and 2019, holiday date is changing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What model I have to used to forecast based on new holiday setup? Is ARIMA or Multiple Linear Regression still can be used? Or I need another model? Where do I have to start on this?&lt;/p&gt;&#xA;" OwnerUserId="23393" LastActivityDate="2019-03-23T02:00:07.433" Title="What Model Used for Forecasting Sales with Dynamic Holiday" Tags="&lt;neural-networks&gt;&lt;linear-regression&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11391" PostTypeId="2" ParentId="11374" CreationDate="2019-03-23T02:15:13.447" Score="3" Body="&lt;p&gt;So if I understood correctly:&#xA;You have data from 2 sensors in time: Ar flow and BackGas Flow (SCCM, what is that?)&#xA;You have that data for multiple products.&lt;/p&gt;&#xA;&lt;p&gt;1 - Since it is relatively low dimensional, you may try using raw data with K-Means or Self Organizing Maps.&lt;/p&gt;&#xA;&lt;p&gt;2 - If you searching for anomalies in time, you might try using feature engineering with things like flow change:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;i) Take 2 points in time and &amp;quot;derivate&amp;quot; (Point A - Point B)/(Time A - Time B)&lt;/li&gt;&#xA;&lt;li&gt;ii) Take the result for A and repeat the process as a second derivative, try that for many levels of &amp;quot;derivative&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;iii) Machine learning application for sensor failure detection in polymerization process. In: Simpósio Brasileiro de Telecomunicações e Processamento de Sinais.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;3 - You might want to check other Time-Series related research, even on regression and classification, since they may give you ideas on relevant features or approaches:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;i) An enseble approach to time dependent classification (10.1109/ICMLA.2018.00164)&lt;/li&gt;&#xA;&lt;li&gt;ii) Multivariate Time Series for Data-driven Endpoint Prediction in the Blast Oxygen Furnace (DOI: 10.1109/ICMLA.2018.00231)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;4 - Note that anomalies are outliers, so analysing data distant from the mean of any features you might engineer from the time. An model based on RBF kernels might pick up some information.&lt;/p&gt;&#xA;&lt;p&gt;5 - Since you don't know the amount of clusters you can try hierarchical clustering.&lt;/p&gt;&#xA;&lt;p&gt;6 - Don't forget to talk to people in your field about what to expect from this.&lt;/p&gt;&#xA;&lt;p&gt;If you can elaborate on the type of anomalies you're looking for I may come with better ideas. This is pretty general tips.&lt;/p&gt;&#xA;&lt;p&gt;I may have misunderstood what you wanted to cluster:&lt;/p&gt;&#xA;&lt;p&gt;7 - If you want to detect anomalies in:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;i) &lt;strong&gt;products behavior&lt;/strong&gt; tips above were mostly for it and you might use all 23 features in short time intervals and cluster like this:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;    For each product in dataset {&#xA;&#xA;      samples = {} # empty set&#xA;&#xA;      For each time interval in product {&#xA;&#xA;        features in interval add to samples&#xA;&#xA;      }&#xA;&#xA;      perform clustering method using samples&#xA;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This algorithm would help cluster periods of time where the product behavied weirdly&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;ii) &lt;strong&gt;sensors behavior&lt;/strong&gt; then for something like that you might try tip 8&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;iii) &lt;strong&gt;products itself&lt;/strong&gt; create a feature vector with all avaiable information for each product, then cluster that. Since it is composed of variable time interval and 23 features per product (plus time) you might want to use a bit of dimensionality reduction (PCA should work fine, since it gives you the direction of greater variance and you're look for that). Using a regressor to model products behavior and cluster that could all be usefull (create a regressor in time for each product to model its behavior, then cluster the weights of the regressors as the product representation)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;8 - Let's say you want to identify sensors behavior, it might be helpful to model the sensors output as a function &lt;strong&gt;f&lt;/strong&gt;(time, extravariables) and do regression over it (try linear, then try non-linear). Points in time that the regressor had bad performance predicting the answer might indicate that that behavior is anomalous&lt;/p&gt;&#xA;" OwnerUserId="23392" LastEditorUserId="1847" LastEditDate="2020-09-18T07:26:47.107" LastActivityDate="2020-09-18T07:26:47.107" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11394" PostTypeId="1" CreationDate="2019-03-23T06:26:44.727" Score="2" ViewCount="919" Body="&lt;p&gt;I have a set of topics and each topic consists of a set of words. I want to make meaningful English sentences from these words. Each topic consist of 5 to 10 words and these words are relevant to each other, like &lt;code&gt;{code, language, test, write and function}&lt;/code&gt; and &lt;code&gt;{class, public, method, string, int}&lt;/code&gt; are two sets. I want to generate a sentence from these set of words using API.&lt;/p&gt;&#xA;" OwnerUserId="23399" LastEditorUserId="2444" LastEditDate="2019-08-20T22:40:34.193" LastActivityDate="2020-06-26T01:02:40.187" Title="How can I make meaningful English sentences from given set of words?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11399" PostTypeId="2" ParentId="11388" CreationDate="2019-03-23T09:35:49.353" Score="1" Body="&lt;p&gt;Your issue is related to how you are representing your rewards, and not anything to worry about for the MDP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have chosen to use a matrix to represent your reward function, which maps &lt;span class=&quot;math-container&quot;&gt;$(s,s')$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;. If some transition &lt;span class=&quot;math-container&quot;&gt;$s \rightarrow s'$&lt;/span&gt; doesn't happen, then it doesn't matter at all what you put there for the reward value for &lt;span class=&quot;math-container&quot;&gt;$(s,s')$&lt;/span&gt;. It is only because you have decided to use a matrix for this that you even need to think about it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;complete&lt;/em&gt; reward function would map &lt;span class=&quot;math-container&quot;&gt;$s, a, s'$&lt;/span&gt; to a scalar value, and can also include a random factor. You don't need to use all the factors - a MDP can use any or all of them, and all the MDP theory is still correct. In your case, it looks like arriving in a &quot;goal state&quot; &lt;span class=&quot;math-container&quot;&gt;$s' = C$&lt;/span&gt; is what triggers a +1 reward. So instead of a matrix you could define a function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(s,a,s') = \begin{cases}&#xA;    -1,&amp;amp; \text{if } s' = A\\&#xA;    1,&amp;amp; \text{if } s' = C\\&#xA;    0,              &amp;amp; \text{otherwise}&#xA;\end{cases} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is equally valid in the MDP as your matrix, and avoids the issue of storing data about impossible transitions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I thought the transition function would handle this issue since I will define there what is possible and what is not.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes that should be the case. If your transition function does not allow for certain state changes, then there is no need to handle them in any particular way in the reward function.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But anyway it is said that for a horizon of one the imm[e]diate reward is given and the transition function isn't even considered&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure where you have read this, but it does not relate to your representation issue or resolving &quot;impossible&quot; state transitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using a model-based method, such as Policy Iteration or Value Iteration, then you do use the transition function and reward function - more or less directly in the form of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bellman_equation#Example&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bellman equations&lt;/a&gt;. In this case, the transition function will assign a weight of 0 to the impossible transitions, so the reward value doe not matter (and efficient code would likely skip even looking up or calculating the reward in that case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using a model-free method, such as Q learning, then the &lt;em&gt;agent&lt;/em&gt; does not use the transition function or the reward function directly. They are part of the environment that it is learning. However, any code for the simulated environment has to implement the physics of your world, and that includes using the transition function to resolve what happens when the agent takes an action. In that case, the simulated model of the environment would prevent the agent ever experiencing impossible transitions, so there is never any need to calculate the reward for them or allow for them as edge cases in a reward function (you might still choose to check validity and/or return something as defensive programming of the reward function, but it is not required for the MDP and reinforcement learning to work).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-23T11:40:11.823" LastActivityDate="2019-03-23T11:40:11.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11400" PostTypeId="2" ParentId="11389" CreationDate="2019-03-23T09:49:36.470" Score="2" Body="&lt;p&gt;The term you're looking for is &quot;replacement schemes&quot;. As far as I'm aware, the primary reference on this is still &lt;a href=&quot;https://www.researchgate.net/publication/2737817_Replacement_Schemes_for_Transposition_Tables&quot; rel=&quot;nofollow noreferrer&quot;&gt;Replacement Schemes for Transposition Tables&lt;/a&gt;, although it is a fairly old paper from 1994.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll very briefly summarize the seven different schemes listed in this paper, but full text of the paper is also freely available and contains more info:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Deep&lt;/strong&gt;: preserve position for which the deepest subtree below it was searched.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;New&lt;/strong&gt;: Always replace an old entry with the newest entry.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Old&lt;/strong&gt;: Opposite of New. Paper mentions only including it for the sake of completeness, implies that it's probably not a good scheme.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Big1&lt;/strong&gt;: similar to Deep, but uses the size (in number of nodes) of subtree searched, rather than just its depth. If the same position (i.e. table entry) occurs multiple times in a subtree, it is only counted once.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;BigAll&lt;/strong&gt;: same as above, but really counts the number of nodes rather than number of positions (so a position that occurs multiple times is counted multiple times).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;TwoDeep&lt;/strong&gt;: In this scheme, the TT is modified to contain two positions per table entry, rather than just one position. You could view it as the table having two &quot;layers&quot;. In this scheme, a new position is moved into the first &quot;slot&quot; if it has the deepest subtree (as in the Deep scheme), moving the previous position in the first slot into the second slot. If a new position doesn't have a new deepest subtree, it is instead moved into the second slot.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;TwoBig1&lt;/strong&gt;: Similar to the above, but using Big1-style replacement rather than Deep-style replacement.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If I recall correctly, the Two-layered schemes tend to perform best (of course they do require approximately twice as much memory for the same number of bits per key).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-03-23T09:49:36.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11401" PostTypeId="1" CreationDate="2019-03-23T12:38:31.063" Score="3" ViewCount="517" Body="&lt;p&gt;There are several variants of the &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN&lt;/a&gt; model. For example, &lt;a href=&quot;https://arxiv.org/abs/1509.06461&quot; rel=&quot;nofollow noreferrer&quot;&gt;double DQN&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1511.06581&quot; rel=&quot;nofollow noreferrer&quot;&gt;duelling DQN&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1511.05952&quot; rel=&quot;nofollow noreferrer&quot;&gt;prioritized DQN&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1803.00933&quot; rel=&quot;nofollow noreferrer&quot;&gt;distributed prioritized DQN&lt;/a&gt;, &lt;a href=&quot;https://www.ijcai.org/proceedings/2018/0337.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;episodic memory DQN&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow noreferrer&quot;&gt;asynchronous n-step DQN&lt;/a&gt; and &lt;a href=&quot;https://www.researchgate.net/publication/326679264_Survey_of_Deep_Q-Network_variants_in_PyGame_Learning_EnvironmentSurvey_of_Deep_Q-Network_variants_in_PyGame_Learning_Environment&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple DQN&lt;/a&gt;. What are the differences between all these variants? What are their advantages and disadvantages? When should I use one over the other? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for an answer that (briefly) describes all the variants (that we are aware of) and then compares them.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-23T12:52:23.623" LastActivityDate="2019-08-20T20:00:55.143" Title="What are the differences between the DQN variants?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;comparison&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11404" PostTypeId="2" ParentId="11212" CreationDate="2019-03-23T15:04:42.993" Score="0" Body="&lt;p&gt;I found that we shouldn't eliminate those actions, because &lt;code&gt;RIGHTFIRE&lt;/code&gt; and &lt;code&gt;LEFTFIRE&lt;/code&gt; do &lt;code&gt;RIGHT&lt;/code&gt; + &lt;code&gt;FIRE&lt;/code&gt; and &lt;code&gt;LEFT&lt;/code&gt; + &lt;code&gt;FIRE&lt;/code&gt;, respectively. So they are different actions. I post this just to clarify in case someone faces the same doubt.&lt;/p&gt;&#xA;" OwnerUserId="9818" LastEditorUserId="2444" LastEditDate="2020-12-27T13:57:19.517" LastActivityDate="2020-12-27T13:57:19.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11405" PostTypeId="1" CreationDate="2019-03-23T15:53:23.880" Score="11" ViewCount="2538" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;noreferrer&quot;&gt;Autoencoders&lt;/a&gt; are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder and a decoder (which can be separate neural networks). Dimensionality reduction can be useful in order to deal with or attenuate the issues related to the curse of dimensionality, where data becomes sparse and it is more difficult to obtain &quot;statistical significance&quot;. So, autoencoders (and algorithms like PCA) can be used to deal with the curse of dimensionality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we care about dimensionality reduction specifically using autoencoders? Why can't we simply use PCA, if the purpose is dimensionality reduction? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we need to decompress the latent representation of the input if we just want to perform dimensionality reduction, or why do we need the decoder part in an autoencoder? What are the use cases? In general, why do we need to compress the input to later decompress it? Wouldn't it be better to just use the original input (to start with)?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-25T19:04:52.107" Title="What are the purposes of autoencoders?" Tags="&lt;machine-learning&gt;&lt;autoencoders&gt;&lt;dimensionality-reduction&gt;&lt;curse-of-dimensionality&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11406" PostTypeId="2" ParentId="11405" CreationDate="2019-03-23T15:53:23.880" Score="3" Body="&lt;p&gt;A use case of autoencoders (in particular, of the decoder or generative model of the autoencoder) is to denoise the input. This type of autoencoders, called &lt;em&gt;denoising autoencoders&lt;/em&gt;, take a partially corrupted input and they attempt to reconstruct the corresponding uncorrupted input. There are several applications of this model. For example, if you had a corrupted image, you could potentially recover the uncorrupted one using a denoising autoencoder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Autoencoders and PCA are related:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;an autoencoder with a single fully-connected hidden layer, a linear activation function and a squared error cost function trains weights that span the same subspace as the one spanned by the principal component loading vectors, but that they are not identical to the loading vectors. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For more info, have a look at the paper &lt;a href=&quot;https://arxiv.org/pdf/1804.10253.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;From Principal Subspaces to Principal Components with Linear Autoencoders&lt;/a&gt; (2018), by Elad Plaut. See also &lt;a href=&quot;https://stats.stackexchange.com/a/120096/82135&quot;&gt;this answer&lt;/a&gt;, which also explains the relation between PCA and autoencoders.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-23T15:59:38.497" LastActivityDate="2019-03-23T15:59:38.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11407" PostTypeId="1" AcceptedAnswerId="11428" CreationDate="2019-03-23T16:03:49.737" Score="5" ViewCount="114" Body="&lt;p&gt;&lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2&lt;/a&gt; relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the intersection of two lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I can't understand is how the iterations of on-line backpropagation are perpendicular to the (current) constraint. More specifically, how is &lt;span class=&quot;math-container&quot;&gt;$\frac12(x_1w_1 + x_2w_2 - y)^2$&lt;/span&gt;'s gradient, &lt;span class=&quot;math-container&quot;&gt;$(x_1,x_2)$&lt;/span&gt;, normal to the constraint &lt;span class=&quot;math-container&quot;&gt;$x_1w_1 + x_2w_2 = y$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="14892" LastEditorUserId="23790" LastEditDate="2019-04-07T20:52:00.720" LastActivityDate="2019-04-07T20:52:00.720" Title="Are on-line backpropagation iterations perpendicular to the constraint?" Tags="&lt;backpropagation&gt;&lt;math&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11408" PostTypeId="2" ParentId="11401" CreationDate="2019-03-23T18:33:00.403" Score="1" Body="&lt;p&gt;I assume OP understand basics of DQN - Time Difference (TD), target network and replay buffer. My experience is just my observations, it not necessarily generalize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of those variants:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Double DQN: small difference in Time Difference propagation. Instead of propagating for maximum action of target network, Q taken form target network propagated for maximum action for current network. This claim to reduce the extrapolation error of TD, which prone to overestimation. My experience: not really helping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dueling DQN: difference in network architecture - Q separated into Advantage (Q(a)-Value) and Value (max_a Q(a)) which calculated separately and summed. My experience: not really helping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prioritized DQN: instead of uniform sampling of replay buffer it is sampled with probability depended on the error of TD. My experience: really helping, must have for any implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Episodic memory DQN: this is really ambitious method. It combine memory-augmented network (differentiable dictionary, idea similar to Neural Turing Machine) with DQN. Difficult to implement correctly. My experience is ambiguous - I tried it in much simplified form, it was not quite clear to me if it was helping. I intend to return to it later. It also incur considerable performance cost. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Missing here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1707.06887&quot; rel=&quot;nofollow noreferrer&quot;&gt;C51 architecture&lt;/a&gt; - this could be important advance. It's attempt to get rid of regression in DQN(regressions are hard for  NN) and replace it with classification. Could be costly from performance point of view. Some reports claim it no worse then best performing Deep RL methods (with exception of tree-based method like Alpha Zero).&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-03-23T18:33:00.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11409" PostTypeId="2" ParentId="10764" CreationDate="2019-03-23T19:00:39.097" Score="0" Body="&lt;p&gt;Andrew Ng explains with great details in &lt;a href=&quot;https://www.coursera.org/lecture/deep-neural-network/understanding-mini-batch-gradient-descent-lBXu8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Course&lt;/a&gt; appears in the image below.&#xA;He also focuses on some corners can cause this problem :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;some mislabeled examples in the dataset.&lt;/li&gt;&#xA;&lt;li&gt;the size of your mini batch or change it with GD.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EGqOx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EGqOx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21907" LastActivityDate="2019-03-23T19:00:39.097" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11410" PostTypeId="2" ParentId="11394" CreationDate="2019-03-23T20:42:55.660" Score="1" Body="&lt;p&gt;Two Approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Naive Bayes&lt;/li&gt;&#xA;&lt;li&gt;LSTM&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Train Naive Bayes on a whole dataset learning the probability of the next word given a word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can even go with any LSTM approaches, but I'd bet on Naive Bayes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eg:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;text: hello how are you hello how are you hello No how&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to get the suggestion of next word depending on current word - hello&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p(how | hello) = 3/4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p(No | hello) = 1/4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;take argmax of probabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also remember to smooth, and train on huge dataset. Training is just finding the probabilities before hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps ;)&lt;/p&gt;&#xA;" OwnerUserId="20522" LastActivityDate="2019-03-23T20:42:55.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11411" PostTypeId="2" ParentId="10778" CreationDate="2019-03-23T20:49:57.923" Score="0" Body="&lt;p&gt;BioID has a liveness detection algorithm that you can test &lt;a href=&quot;https://www.bioid.com/liveness-detection/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and it is free!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get their model at &lt;a href=&quot;https://github.com/BioID-GmbH/BWS-Android/&quot; rel=&quot;nofollow noreferrer&quot;&gt;their GitHub&lt;/a&gt;. I think it is open-source (check the license).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their model relies on facial landmarking. They also have algorithms for:&#xA;- Face Verification&#xA;- Photo Verification&#xA;- Cloud Based Solutions in BWS&lt;/p&gt;&#xA;" OwnerUserId="23392" LastActivityDate="2019-03-23T20:49:57.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11412" PostTypeId="2" ParentId="11405" CreationDate="2019-03-23T21:29:29.433" Score="2" Body="&lt;p&gt;PCA is a linear method that creates a transformation that is capable of changing the vectors projections (changing axis)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since PCA looks for the direction of maximum variance it usually have high discriminativity &lt;strong&gt;BUT&lt;/strong&gt; it does not guaranteed that the direction of most variance is the direction of most discriminativity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LDA is a linear method that creates a transformation that is capable of finding the direction that is most relevant to decide if a vector belong to class A or B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PCA and LDA have non-linear Kernel versions that might overcome their linear limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Autoencoders can perform dimensionality reduction with other kinds of loss function, can be non-linear and might perform better than PCA and LDA for a lot of cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is probably no best machine learning algorithm to do anything, sometimes Deep Learning and Neural Nets are overkill for simple problems and PCA and LDA might be tried before other, more complex, dimensionality reductions.&lt;/p&gt;&#xA;" OwnerUserId="23392" LastActivityDate="2019-03-23T21:29:29.433" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11413" PostTypeId="2" ParentId="7879" CreationDate="2019-03-23T22:05:20.000" Score="0" Body="&lt;p&gt;Offline tracking requires knowlege of the future of an object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For real time applications such as the ones you cited, offline tracking might be impeditive given that the method needs to wait for future frames.&lt;/p&gt;&#xA;" OwnerUserId="23392" LastActivityDate="2019-03-23T22:05:20.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11414" PostTypeId="2" ParentId="10973" CreationDate="2019-03-23T23:50:00.453" Score="0" Body="&lt;p&gt;You might want to check out this paper relating to Phizaz's comment: &lt;a href=&quot;https://arxiv.org/pdf/1602.01783.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Methods for Deep Reinforcement Learning&lt;/a&gt; (specifically search for Hogwild).&lt;/p&gt;&#xA;" OwnerUserId="21180" LastEditorUserId="2444" LastEditDate="2019-03-24T13:38:52.290" LastActivityDate="2019-03-24T13:38:52.290" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11415" PostTypeId="2" ParentId="11405" CreationDate="2019-03-24T00:26:08.930" Score="6" Body="&lt;p&gt;It is important to think about what sort of patterns in the data are being represented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that you have a dataset of greyscale images, such that every image is a uniform intensity. As a human brain you'd realise that every element in this dataset can be described in terms of a single numeric parameter, which is that intensity value. This is something that PCA would work fine for, because each of the dimensions (we can think of each pixel as a different dimension) is perfectly linearly correlated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose instead that you have a dataset of black and white 128x128px bitmap images of centred circles. As a human brain you'd quickly realise that every element in this dataset can be fully described by a single numeric parameter, which is the radius of the circle. That is a very impressive level of reduction from 16384 binary dimensions, and perhaps more importantly it's a semantically meaningful property of the data. However, PCA probably won't be able to find that pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question was &quot;Why can't we simply use PCA, if the purpose is dimensionality reduction?&quot; The simple answer is that PCA is the simplest tool for dimensionality reduction, but it can miss a lot of relationships that more powerful techniques such as autoencoders might find. &lt;/p&gt;&#xA;" OwnerUserId="23413" LastActivityDate="2019-03-24T00:26:08.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11418" PostTypeId="1" CreationDate="2019-03-24T05:26:49.420" Score="4" ViewCount="128" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/11418/why-q2-is-a-more-or-less-independant-estimate-in-twin-delayed-ddpg-td3&quot;&gt;Twin Delayed Deep Deterministic (TD3)&lt;/a&gt; policy gradient is inspired by both double Q-learning and double DQN. In double Q-learning, I understand that Q1 and Q2 are independent because they are trained on different samples. In double DQN, I understand that target Q and current Q are relatively independent because their parameters are quite different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in TD3, Q1 and Q2 are trained on exactly the same target. If their parameters are initialized the same, there will be no difference in their output and the algorithm will be equal to DQN. The only source of independence/difference of Q2 to Q1 I can tell is the randomness in the initialization of their parameters. But with training on the same target, I thought this independence will become smaller and smaller as they converge to the same target values. So I don't quite understand why TD3 works in combating overestimation in Q-learning.&lt;/p&gt;&#xA;" OwnerUserId="23420" LastEditorUserId="2444" LastEditDate="2020-04-20T11:17:50.187" LastActivityDate="2021-05-15T13:08:40.797" Title="Why Q2 is a more or less independant estimate in Twin Delayed DDPG (TD3)?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11419" PostTypeId="1" CreationDate="2019-03-24T06:52:59.050" Score="0" ViewCount="43" Body="&lt;p&gt;can i compare MAE and MSE loss results of a regression CNN with categorical_crossentropy loss of a classification CNN if they both have similar tasks? is yes how to?&lt;/p&gt;&#xA;" OwnerUserId="23268" LastActivityDate="2019-03-24T08:29:48.243" Title="Can we compare MAE MSE results with categorical_crossentropy?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;linear-regression&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11421" PostTypeId="1" CreationDate="2019-03-24T08:40:01.430" Score="5" ViewCount="140" Body="&lt;p&gt;I am training a modified VGG-16 to classify crowd density (empty, low, moderate, high). 2 dropout layers were added at the end on the network each one after one of the last 2 FC layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;network settings:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;training data contain 4381 images categorized under 4 categories (empty, low, moderate, high), 20% of the training data is set for validation. test data has 2589 images.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;training is done for 50 epochs.(training validation accuracy drops after&#xA;50 epochs)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;lr=0.001, decay=0.0005, momentum=0.9&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;loss= categorical_crossentropy&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;augmentation for (training, validation and testing data): rescale=1./255, brightness_range=(0.2,0.9), horizontal_flip&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With the above-stated settings, I get the following results: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;training evaluation loss: 0.59, accuracy: 0.77&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;testing accuracy 77.5 (correct predictions 2007 out of 2589)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding this, I have two concerns: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there anything else I could do to improve accuracy for both training and testing? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I know if this is the best accuracy I can get?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23268" LastEditorUserId="2444" LastEditDate="2020-06-14T22:20:46.387" LastActivityDate="2020-06-14T22:20:46.387" Title="How do I improve accuracy and know when to stop training?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;performance&gt;&lt;vgg&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11422" PostTypeId="5" CreationDate="2019-03-24T10:19:47.120" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-03-24T10:19:47.120" LastActivityDate="2019-03-24T10:19:47.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11423" PostTypeId="4" CreationDate="2019-03-24T10:19:47.120" Score="0" Body="For questions related to AI methods of dimensionality reduction (e.g. PCA or autoencoders)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-25T19:22:37.717" LastActivityDate="2019-03-25T19:22:37.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11424" PostTypeId="5" CreationDate="2019-03-24T10:20:56.473" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Curse_of_dimensionality&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Curse_of_dimensionality&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-21T17:17:56.700" LastActivityDate="2019-07-21T17:17:56.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11425" PostTypeId="4" CreationDate="2019-03-24T10:20:56.473" Score="0" Body="For questions related to the concept of &quot;curse of dimensionality&quot;, which refers to the problem of an exponential increase in volume which occurs when adding extra dimensions to the Euclidean (or input) space. In machine learning and statistics, the curse of dimensionality implies that more data is required to achieve statistical significance, as the number of dimensions of the input increases. The expression was introduced by Richard Bellman in 1957." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-21T17:17:56.700" LastActivityDate="2019-07-21T17:17:56.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11427" PostTypeId="1" CreationDate="2019-03-24T19:37:20.193" Score="7" ViewCount="269" Body="&lt;p&gt;&lt;strong&gt;Background Context:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the past I've heavily applied various &quot;code quality metrics&quot; to statically analyze code to provide an inkling of how &quot;maintainable&quot; it is and using things like the Maintainability Index alluded to &lt;a href=&quot;https://quandarypeak.com/2015/02/measuring-software-maintainability/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a problem that I face is whether a language has libraries that effectively measure such metrics - only then is it usable else it's rather subjective/arbitrary. Given the plethora of languages that one has to deal with in an enterprise system, this is can get rather unwieldy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Proposed Idea:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Build and train an Artificial Neural Network that &quot;ingests a folder of code&quot; (i.e., all files within that folder/package are assumed to house the &quot;project&quot; whose quality metrics we'd like to compute). This may again be language dependent but let's assume it exists for a language that I'm having the hardest time with (for measuring &quot;maintainability&quot;): Scala.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using numeric metrics like &lt;code&gt;McCabe's complexity&lt;/code&gt; or &lt;code&gt;Cyclomatic complexity&lt;/code&gt; maybe &quot;convention&quot; but are not entirely relevant. Few things like class/method length are almost always relevant no matter the language. Thus, providing a few &quot;numeric metrics&quot; + abstract notion of readability by subjective evaluation to train an ANN would be a good balance of &quot;inputs&quot; to the ANN. The output being either a classification of maintainability like &lt;code&gt;low, medium, high&lt;/code&gt; etc., or a number between &lt;code&gt;0 and 1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this been tried and are there any references? I spent some time digging via Google Scholar but didn't find anything &quot;usable&quot; or worthwhile. It's okay if it's not Scala, but have ANNs been used for measuring code quality (i.e., static analysis) and what are the benefits or disadvantages of something like this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;PS: Hopefully, the question isn't too broad, but if so, please let me know in the comments and I'll try make it as specific as possible.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23432" LastActivityDate="2019-03-25T15:19:15.470" Title="Are there existing examples of using neural networks for static code analysis?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;quality-control&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11428" PostTypeId="2" ParentId="11407" CreationDate="2019-03-24T20:38:03.707" Score="1" Body="&lt;p&gt;Answer by &lt;a href=&quot;https://math.stackexchange.com/users/248286/theo-bendit&quot;&gt;Theo Bandit&lt;/a&gt; &lt;a href=&quot;https://math.stackexchange.com/questions/3160982/how-is-x-1-x-2-normal-to-x-1w-1-x-2w-2-y/3160988#3160988&quot;&gt;at maths stackexchange&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you choose two points &lt;span class=&quot;math-container&quot;&gt;$(w_1, w_2), (v_1, v_2)$&lt;/span&gt; along this line, then&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$(x_1, x_2) \cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$&lt;/span&gt;&#xA;  That is, the direction &lt;span class=&quot;math-container&quot;&gt;$(x_1, x_2)$&lt;/span&gt; is perpendicular to any vector lying along the line, i.e. &lt;span class=&quot;math-container&quot;&gt;$(x_1, x_2)$&lt;/span&gt; is normal to the line.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14892" LastActivityDate="2019-03-24T20:38:03.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11429" PostTypeId="2" ParentId="9141" CreationDate="2019-03-24T21:15:53.543" Score="5" Body="&lt;p&gt;this experiment by Stephen Mayhew suggests that BERT is lousy at sequential text generation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://mayhewsw.github.io/2019/01/16/can-bert-generate-text/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mayhewsw.github.io/2019/01/16/can-bert-generate-text/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;although he had already eaten a large meal, he was still very hungry&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As before, I masked “hungry” to see what BERT would predict. If it could predict it correctly without any right context, we might be in good shape for generation.&lt;/p&gt;&#xA;&lt;p&gt;This failed. BERT predicted “much” as the last word. Maybe this is because BERT thinks the absence of a period means the sentence should continue. Maybe it’s just so used to complete sentences it gets confused. I’m not sure.&lt;/p&gt;&#xA;&lt;p&gt;One might argue that we should continue predicting after “much”. Maybe it’s going to produce something meaningful. To that I would say: first, this was meant to be a dead giveaway, and any human would predict “hungry”. Second, I tried it, and it keeps predicting dumb stuff. After “much”, the next token is “,”.&lt;/p&gt;&#xA;&lt;p&gt;So, at least using these trivial methods, BERT can’t generate text.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="23433" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-03-24T21:15:53.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11431" PostTypeId="1" CreationDate="2019-03-24T23:20:19.493" Score="4" ViewCount="40" Body="&lt;p&gt;I was looking around, a promising approach was this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mpralat/notesRecognizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mpralat/notesRecognizer&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the problem is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it doesn't seem good enough. One should be able to read musical notes with lower quality images. You can see in her: &quot;bad&quot; images folder just tiny variations of lightning can already cause her problems with her high res images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;others are here, they all use high res sharp images:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/suyalcinkaya/music-note-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/suyalcinkaya/music-note-recognition&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/suyalcinkaya/music-note-recognition/blob/master/input_images/im2s.JPG?raw=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/suyalcinkaya/music-note-recognition/blob/master/input_images/im2s.JPG?raw=true&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nikolalsvk/note-play&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nikolalsvk/note-play&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nikolalsvk/note-play/blob/master/images/notes-1.png?raw=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nikolalsvk/note-play/blob/master/images/notes-1.png?raw=true&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now this is unsatisfying if you want to snap a photo of some tunes, and want them to be recognized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so what could one do to achieve a good solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about treating the musical notes just like written letters. The computer can easily learn written characters with the arabic symbols. I wonder though, how easy would it be for a none arabic? e.g. in chinese or japanese, several characters combine into one. The same applies to music notes, they can be connected and form something slightly different through that e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XtB0y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XtB0y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P6X9K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P6X9K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in contrast to just simple notes like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ds7tf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ds7tf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what would be a good approach to try ones luck, with interpreting those symbols successfully, even for slightly low res images or bit blurry deformed images. I'm not saying to read out a symfony out of a thumbnail. But less than optimal captures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;any subjective ideas or comments are more than welcome&lt;/p&gt;&#xA;" OwnerUserId="23435" LastActivityDate="2019-03-25T06:01:46.900" Title="An approach on reading musical notes from photos" Tags="&lt;image-recognition&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11432" PostTypeId="2" ParentId="11421" CreationDate="2019-03-25T01:38:06.183" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there anything else I could do to improve accuracy for both training and testing? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Try to use a more complex model: ResNet, DenseNet, etc.&lt;/li&gt;&#xA;&lt;li&gt;Try to use other optimizers: Adam, Adadelta, etc.&lt;/li&gt;&#xA;&lt;li&gt;Tune your hyperparameters (e.g. change your learning rate, momentum, rescale factor, convolution size, number of feature maps, epochs, neurons, FC layers)&lt;/li&gt;&#xA;&lt;li&gt;Try to analyze your data, with ~75% and 4 categories, is that possible there is one category that difficult to classified?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In essence, you have to do a lot of experiments with your model until you think &quot;it is enough&quot; (if you have a deadline). If you don't have a hard deadline, you can keep improving updating your ML model.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I know if this is the best accuracy I can get?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, you can't until you compare it with other models/hyperparameters. If you do more experiments (some ways like the one I mentioned above) or compare with other people's experiments that using the same data, you'll find which one is the best. For an academic paper, for example, you need to compare at least 3 to 4 models that similar or experiment with hundreds of different hyperparameters combination.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-03-25T01:38:06.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11433" PostTypeId="1" AcceptedAnswerId="11451" CreationDate="2019-03-25T04:26:05.393" Score="3" ViewCount="74" Body="&lt;p&gt;I am trying to reproduce the results for the simple grid-world environment in [1]. But it turns out that using a dynamically learned PBA makes the performance worse and I cannot obtain the results shown in Figure 1 (a) in [1] (with the same hyperparameters). Here is the result I got:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/14m4g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/14m4g.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I found is that the learning procedure is stuck due to bad PBA in the early stages of training. Without PBA, Sarsa can converge well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did anyone try the method before? I am really puzzled and how the authors obtain these good results? There are some top conference papers using the same method in [1], for example, [2] and [3].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]  &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/download/9893/9923&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expressing Arbitrary Reward Functions as Potential-Based Advice&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2]  &lt;a href=&quot;http://www.ifaamas.org/Proceedings/aamas2016/pdfs/p429.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning from demonstration for shaping through inverse reinforcement learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3]  &lt;a href=&quot;https://ai.vub.ac.be/~tbrys/publications/Brys2015AAMAS.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy Transfer using Reward Shaping&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the method itself defective or anything wrong with my code? Here is part of my codes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import copy&#xA;import numpy as np&#xA;import pandas as pd&#xA;&#xA;def expert_reward(s, action):&#xA;    if (action == RIGHT) or (action == DOWN):&#xA;        return 1.0&#xA;    return 0.0&#xA;&#xA;class DynamicPBA:&#xA;    def __init__(self, actions, learning_rate=0.1, reward_decay=0.99):&#xA;        self.lr = learning_rate&#xA;        self.gamma = reward_decay&#xA;        self.actions = actions&#xA;        self.q_table = pd.DataFrame(columns=self.actions, dtype=np.float64) #q table for current time step&#xA;        self.q_table_ = pd.DataFrame(columns=self.actions, dtype=np.float64) #q table for the current time step&#xA;        self.check_state_exist(str((0,0)))&#xA;&#xA;    def learn(self, s, a, r, s_, a_): #(s,a) denotes current state and action, r denotes reward, (s_, a_) denotes the next state and action&#xA;        self.check_state_exist(s_)&#xA;        q_predict = self.q_table.loc[s, a]&#xA;        q_target = r + self.gamma * self.q_table.loc[s_, a_]&#xA;        self.q_table.loc[s, a] = self.q_table.loc[s, a] + self.lr * (q_target - q_predict)&#xA;&#xA;    def update(self):&#xA;        self.q_table = copy.deepcopy(self.q_table_)&#xA;&#xA;    def check_state_exist(self, state):&#xA;        if state not in self.q_table.index:&#xA;            # append new state to q table&#xA;            self.q_table = self.q_table.append(&#xA;                pd.Series(&#xA;                    [0]*len(self.actions),&#xA;                    index=self.q_table.columns,&#xA;                    name=state,&#xA;                    )&#xA;                )&#xA;            self.q_table_ = self.q_table_.append(&#xA;                pd.Series(&#xA;                    [0]*len(self.actions),&#xA;                    index=self.q_table_.columns,&#xA;                    name=state,&#xA;                    )&#xA;                )&#xA;&#xA;#######Main part&#xA;&#xA;RL = SarsaTable(actions=list(range(len(actions_dict))), reward_decay=0.99, learning_rate=0.05)&#xA;expert = DynamicPBA(actions=list(range(len(actions_dict))), learning_rate=0.1, reward_decay=0.99)&#xA;for episode in range(100):&#xA;    # initial observation&#xA;    s = (0,0)&#xA;    env.reset(s)&#xA;    action = RL.choose_action(str(s))&#xA;&#xA;    r_episode_s = 0&#xA;    r_episode = 0&#xA;&#xA;    current_step = 0&#xA;    while True:        &#xA;&#xA;        # RL take action and get next observation and reward&#xA;        s_, _, reward, status = env.step(action)&#xA;        current_step += 1&#xA;&#xA;        action_ = RL.choose_action(str(s_))&#xA;        # update dynamic potentials&#xA;        expert_r = -expert_reward(s, action) &#xA;        expert.learn(str(s), action, expert_r, str(s_), action_)&#xA;&#xA;        # compute PBA&#xA;        F = expert.gamma * expert.q_table_.loc[str(s_), action_] - expert.q_table.loc[str(s), action]&#xA;&#xA;        #update expert PBA table&#xA;        expert.update()&#xA;&#xA;        RL.learn(str(s), action, reward+F, str(s_), action_, status)&#xA;&#xA;        # swap observation&#xA;        s = s_&#xA;        action = action_&#xA;&#xA;        # break while loop when end of this episode&#xA;        if status != 'not_over':&#xA;            break&#xA;        if current_step&amp;gt;10000:&#xA;            print(episode, r_episode, r_episode_s, current_step)&#xA;            break     &#xA;        # learning rate decay&#xA;        RL.lr = RL.lr*0.999&#xA;#     expert.update()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23429" LastEditorUserId="2444" LastEditDate="2020-11-09T17:35:54.807" LastActivityDate="2020-11-09T17:35:54.807" Title="Expressing Arbitrary Reward Functions as Potential-Based Advice (PBA)" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-shaping&gt;&lt;inverse-rl&gt;&lt;potential-reward-shaping&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11435" PostTypeId="2" ParentId="11431" CreationDate="2019-03-25T06:01:46.900" Score="-1" Body="&lt;p&gt;refer to &lt;a href=&quot;https://www.kaggle.com/c/digit-recognizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST on kagle&lt;/a&gt;&#xA;to train model to recognize given set of notes in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B2oQ3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B2oQ3.png&quot; alt=&quot;your example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you start training model will understand pattern depending on bits in your image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;issue comes with blurry images when last 2 sets of pics with minute change. will be difficult task for computer to distinguish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;there is one sure shot method which works almost every time but is resource intensive and needs plethora of data to train. is DNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is my take on your question.&lt;/p&gt;&#xA;" OwnerUserId="9991" LastActivityDate="2019-03-25T06:01:46.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11436" PostTypeId="1" CreationDate="2019-03-25T09:19:38.147" Score="4" ViewCount="156" Body="&lt;p&gt;I am no expert in the field of AI so I apologize if this is a simple/easy question. I was trying to implement a network similar to OpenAI's for another game and I noticed that I did not fully understand how the network worked.&lt;/p&gt;&#xA;&lt;p&gt;Below is the image of OpenAI five's network&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1uQGT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1uQGT.jpg&quot; alt=&quot;OpenAI's model&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Basic question&lt;/h1&gt;&#xA;&lt;p&gt;How is the data concatenated/ what is the dimension of the data right after the concatenation before the LSTMs or just before entering the LSTMs?&#xA;Below are my thoughts which I provide for clarification's sake.&lt;/p&gt;&#xA;&lt;h2&gt;1st Interpretation&lt;/h2&gt;&#xA;&lt;p&gt;In the blue area for units, my initial understanding was that for each visible unit, the output of the max-pool is concatenated along the columns. So, assuming the number of rows is 1(as a 1d max-pool is being applied for each unit), the number of columns is n and there are N visible units, the final size of the matrix when concatenated is &lt;span class=&quot;math-container&quot;&gt;$(1,n\cdot N)$&lt;/span&gt; with few extra columns given by the pickups and the like as shown on the left-hand side of the model.&lt;/p&gt;&#xA;&lt;h3&gt;Problem with this interpretation&lt;/h3&gt;&#xA;&lt;p&gt;As the number of units a player can see per each turn is not constant, under this interpretation, I suspect that the fully connected layer after the concatenation layer cannot do its job as matrix multiplication becomes impossible with a variable number of columns.&lt;/p&gt;&#xA;&lt;h3&gt;Possible solution&lt;/h3&gt;&#xA;&lt;p&gt;One possible solution to this is to set a maximum to the number of observed units as &lt;span class=&quot;math-container&quot;&gt;$N_{max}$&lt;/span&gt; and pad with constants if some units are not observed. Is this the case?&lt;/p&gt;&#xA;&lt;h2&gt;2nd Interpretation&lt;/h2&gt;&#xA;&lt;p&gt;My 2nd interpretation is that the data is concatenated along the rows. In this case, I can see that the data can pass through a fully connected layer because the number of columns can remain constant. Under this assumption, I decided that right before going through the LSTM, the data is reshaped to (batch size, number of rows, number of columns).&lt;/p&gt;&#xA;&lt;h3&gt;Problems with this interpretation&lt;/h3&gt;&#xA;&lt;p&gt;While I found this interpretation to be more appealing, I noticed that under this train of thought, the LSTM is used just to associate the input data and is not associated with time(the time step for the LSTM is simply the next row of data rather than actual time). I know that this is not especially a problem but I thought that there is no special need to use an LSTM here as in this second interpretation, the order of the data holds no special meaning. But is this the case?&lt;/p&gt;&#xA;&lt;p&gt;I apologize in advance for any unclear points. Please tell me in the comments and I'll try to clarify as best as I can!&lt;/p&gt;&#xA;" OwnerUserId="23443" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2021-04-17T07:59:11.480" Title="How did the OpenAI 5 for Dota concatenate units?" Tags="&lt;deep-learning&gt;&lt;open-ai&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11438" PostTypeId="1" CreationDate="2019-03-25T09:55:07.263" Score="2" ViewCount="635" Body="&lt;p&gt;I have been looking for BERT for many tasks. I would like to compare the performance to answer an FAQ, using BERT semantic similarity and BERT Q/A. &#xA;However, I'm not sure it is a good idea to use semantic similarity for this task. If it is, do you think it is possible to find a dataset to fine-tune my algorithm? &lt;/p&gt;&#xA;" OwnerUserId="23154" LastEditorUserId="2444" LastEditDate="2019-11-01T02:37:57.027" LastActivityDate="2020-03-30T05:03:20.393" Title="Is it a good idea to use BERT to answer a FAQ with semantic similarity?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;applications&gt;&lt;bert&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11439" PostTypeId="1" CreationDate="2019-03-25T11:22:41.433" Score="2" ViewCount="292" Body="&lt;p&gt;I'm interested in building a (deep) RL agent for solving a continuous problem (which splits something into portions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all examples I've seen so far, e.g., &lt;a href=&quot;https://github.com/openai/gym/wiki/Leaderboard#lunarlandercontinuous-v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;solving the continuous lunar lander&lt;/a&gt;, always a &lt;span class=&quot;math-container&quot;&gt;$\tanh$&lt;/span&gt; output layer activation was used, which produces values between &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this just because it fits the use case or is this a general rule for RL agents with continuous action spaces?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if I just want values between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;? Could I simply use a &lt;span class=&quot;math-container&quot;&gt;$\operatorname{softmax}$&lt;/span&gt; activation for my output layer?&lt;/p&gt;&#xA;" OwnerUserId="19928" LastEditorUserId="2444" LastEditDate="2019-05-15T15:11:45.990" LastActivityDate="2019-05-16T03:41:37.453" Title="Regarding the output layer's activation function for continuous action space problems" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11440" PostTypeId="2" ParentId="11381" CreationDate="2019-03-25T12:59:08.513" Score="3" Body="&lt;p&gt;There are two intermixed elements in your question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why have people already studied causality in AI?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why would people like to continue studying causality in AI?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt;: AI systems can't function in the real world without some way of understanding uncertainty. The best ways we have to understand uncertainty don't work well unless we also understand causal structure. Getting most of the way towards this understanding was a huge accomplishment for AI research, but there's still a big problem waiting to be solved here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For part 1, let's think about where research into causality comes from. Pearl originally set out to solve a major problem plaguing AI researchers in the 1980s: reasoning under uncertainty. This area is so central to AI systems that it has its own, large, conference: &lt;a href=&quot;http://auai.org/uai2019/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UAI&lt;/a&gt;. The need for reasoning under uncertainty arose because the real world is filled with uncertainty. Consider even a simple problem, like have a robot navigate an empty room. Even if the robot knows its initial position exactly, it is not likely to know its true position for long. The robots' wheels might have a certain specification, but wheels stick and slip (friction), and they do so unpredictably (e.g. maybe one part of the floor didn't get polished as much as the others). The robot might have sensors, but sensors are imperfect. Does a light sensor value of 0.25 mean a wall is near, or just that the sun is coming through the window at an unfortunate angle? Or maybe just that the shade of paint is slightly different there? Does our acoustic sensor reading of 1.35 mean that we're 1.35 meters from the wall, or did the signal get reflected, and return to us by a different path?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of Pearl's major contributions was showing how we can use the rules of probability to reason about these events correctly. Although others had done the basics of this long ago, Pearl proposed the idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Networks&lt;/a&gt;. &lt;a href=&quot;http://robots.stanford.edu/papers/thrun.landmark.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thrun&lt;/a&gt; and many others used these techniques to solve problems like the robot navigation task discussed above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A problem with Bayesian Networks is that, if built incorrectly, they lose much of their efficiency benefits. The correct construction is usually the one that best captures known causal relations between factors. Further, algorithms for inference in a Bayesian Network can not easily answer questions about counterfactuals. They cannot tell our robot what &lt;em&gt;would&lt;/em&gt; happen in a world where certain actions were taken. They can only say whether certain behaviors are likely co-occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These outstanding issues led Pearl to work on causality and causal networks. It is important for our systems to be able to answer counterfactual questions, because often the answers to those questions determine how the system aught to act. Pearl's do-calculus (nicely summarized &lt;a href=&quot;https://www.inference.vc/untitled/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and in Pearl's &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/046509760X&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Book of Why&lt;/a&gt;) has solved this problem, and shown how causes can often be inferred from observation alone. It's super exciting!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully you're now convinced that Causality was worth studying within AI. You might now wonder why it's &lt;em&gt;still&lt;/em&gt; worth studying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main open problem right now is that to do causal reasoning, we need to already have a network that describes causal relations. These networks are hard to construct by hand, so we'd like to be able to &lt;em&gt;learn&lt;/em&gt; the structures of these networks. Unfortunately, this looks like a very hard problem, both to solve exactly (it's &lt;a href=&quot;https://complexityzoo.uwaterloo.ca/Complexity_Zoo:Symbols#sharpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;#P-hard&lt;/a&gt; if I recall correctly), and to approximate. There are no known general algorithms, at least as of 2018. Getting this would be a major breakthrough in AI research, and more generally, in statistics, philosophy of science, and epistemology. Even getting something that mostly worked, most of the time, would be huge. &lt;a href=&quot;https://cs.uwaterloo.ca/~vanbeek/Research/research_ml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Peter Van Beek&lt;/a&gt; has been doing some work on this recently, which could be a good starting place if you want to read more.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-06-11T20:44:59.843" LastActivityDate="2019-06-11T20:44:59.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11441" PostTypeId="2" ParentId="11333" CreationDate="2019-03-25T13:18:21.157" Score="1" Body="&lt;p&gt;It sounds like you are describing a synthesis of two competing ways to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDP problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reinforcement learning, we solve the MDP problem by having the agent move around its environment, observe rewards and transitions in response to the actions it takes, and build a model of the relationship between actions and rewards that allows it to maximize rewards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An older approach is to give the agent facts about the world that can be encoded as logical rules. The agent then uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Unification_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;unification&lt;/a&gt; to reason about the consequences of actions within that framework of rules. The agent then takes actions that maximize the rewards it can expect, given the rules and the information at hand. A problem with this approach was that it did not work well in problem domains with probabilistic rules (i.e. X &lt;em&gt;usually&lt;/em&gt; happens when action Y is taken). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A hybrid approach, somewhere between these two is the use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process#Value_iteration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Value Iteration or Policy Iteration&lt;/a&gt; methods. These are so-called &quot;model-based&quot; reinforcement learning algorithms (although, I would tend to say that makes them something other than true reinforcement learning...). Like the older logic-based approaches, they start by writing down a series of rules that fully describe how things happen in the world, and then derive the logical consequences of those rules to compute the best actions the agent could take. Like reinforcement learning however, they are able to account for probabilistic rules and probabilistic rewards. If you had an exact description of a game of chance, you could write it down as an MDP, and then solve it exactly using these techniques. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Importantly, value &amp;amp; policy iteration methods are not feasible if the state and action spaces are very large (as they often are), and are &lt;em&gt;really&lt;/em&gt; not feasible if the MDP is not known exactly (i.e. if you don't know all the rules of the game in advance). That's where reinforcement learning shines. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-25T13:18:21.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11442" PostTypeId="1" AcceptedAnswerId="11448" CreationDate="2019-03-25T13:48:27.580" Score="2" ViewCount="293" Body="&lt;p&gt;From what I know, AI/ML uses a large amount of data to train an algorithm to solve problems. But since it’s an algorithm, I was wondering if it's possible to export it. If I trained an AI with R, could I export a mathematical algorithm that could be imported by other users to use in their application, whether it’s written in R or another language?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it’s like I’ve discovered a secret message decoding method. I don’t need to share the whole program for others to decode it. I just need to tell them the steps (algorithm) to decode it, and they can implement it in whatever application they want. &lt;/p&gt;&#xA;" OwnerUserId="23017" LastEditorUserId="2444" LastEditDate="2020-06-15T14:00:24.003" LastActivityDate="2020-06-15T14:00:24.003" Title="Can I export a trained machine learning model so that others case use it?" Tags="&lt;machine-learning&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11443" PostTypeId="2" ParentId="11442" CreationDate="2019-03-25T14:56:14.520" Score="0" Body="&lt;p&gt;Yes, once you've trained a model you'll have the details of that model in your workspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B_Naive = naiveBayes(train_set[,-c(1)],train_set[,1]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Will give you an object B_naive that can be 'exported'. These are the parameters of the model, you'll still need the naïve bayes library (or whichever library).&lt;/p&gt;&#xA;" OwnerUserId="22897" LastActivityDate="2019-03-25T14:56:14.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11445" PostTypeId="2" ParentId="11427" CreationDate="2019-03-25T15:19:15.470" Score="3" Body="&lt;p&gt;There's certainly literature on a related topic: code smell detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;code smell&quot; is a sign that code has a maintenance problem, and hints at the presence of technical debt. It is reasonable to suppose that code with a lot of smells is lower quality. Code smells include things like giant classes, high cyclomatic complexity, and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/s10664-015-9378-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fontana et al.&lt;/a&gt; have a good 2016 survey comparing different ML methods for detecting code smells. A &lt;a href=&quot;https://scholar.google.com/scholar?cites=3238500405772448084&amp;amp;as_sdt=40005&amp;amp;sciodt=0,10&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;reverse citation search&lt;/a&gt; on that paper uncovers many other papers that seem relevant, including:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rasool et al. 2015, &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/pdf/10.1002/smr.1737&quot; rel=&quot;nofollow noreferrer&quot;&gt;A review of code smell mining techniques&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;da Silva Maldonado et al. 2017 &lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7820211&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using Natural Language Processing to Automatically Detect Self-Admitted Technical Debt&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Maneerat &amp;amp; Muenchaisri, 2011, &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/5930143&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bad-smell prediction from software design model using machine learning techniques&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Mansoor et al. 2017, &lt;a href=&quot;https://link.springer.com/article/10.1007/s11219-016-9309-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-objective code-smells detection using good and bad design examples&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This seems like a pretty well-studied area. I wasn't able to find a cross-language model though, but I suspect one may well exist.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-25T15:19:15.470" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11447" PostTypeId="1" CreationDate="2019-03-25T15:49:31.813" Score="1" ViewCount="17" Body="&lt;p&gt;I have a dataset about (240000,23). For my task, I have to use an unsupervised learning method and apply it on every single column separately in order to detect anomalies that might exist. I have pre-processed the data and I am visualizing the TimeElapsed vs the parameter in Python (The graphs look like the one shown in an earlier post by me &lt;a href=&quot;https://stackoverflow.com/questions/54201622/how-to-obtain-multiple-windows-containing-multiple-graphs-using-matplotlib&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is a way wherein after the graphs are plotted, the graphs are compared with each other and then clustered together based on their similarities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: If I have temporal data of about 200 products, I plot the graphs of all the 200 products (as can be seen in the link provided above) and these 200 graphs must be compared with each other and must be separately plotted as 200 different points on a scatter plot (by using some unsupervised learning techniques) based on how similar are the graphs of different products to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if the code that I have would be helpful in guiding me, but the code that I have is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;np.random.seed(1234)&#xA;&#xA;dataset = pd.read_csv('Temporal_DataTable.CSV', header = 0)&#xA;dataset != 0&#xA;(dataset != 0).any(axis=0)&#xA;dataset = dataset.loc[:, (dataset != 0).any(axis=0)]&#xA;&#xA;dataset['product_number'] = pd.factorize(dataset.Context)[0]+1&#xA;dataset['index'] = pd.factorize(dataset.Context)[0]+1&#xA;&#xA;cols = list(dataset.columns.values)&#xA;cols.pop(cols.index('StepID'))&#xA;cols.pop(cols.index('Context'))&#xA;cols.pop(cols.index('product_number'))&#xA;cols.pop(cols.index('index'))&#xA;dataset = dataset[['index','product_number','Context','StepID']+cols]&#xA;dataset = dataset.set_index('index')&#xA;&#xA;max_time_ID_without_drop = dataset.groupby(['product_number'])['TimeElapsed', 'StepID'].max()&#xA;avg_time_without_drop = np.average(max_time_ID_without_drop['TimeElapsed'])&#xA;&#xA;dataset_drop = dataset.drop(index = [128, 133, 140, 143, 199])&#xA;&#xA;max_time_ID_with_drop = dataset_drop.groupby(['product_number'])['TimeElapsed', 'StepID'].max()&#xA;avg_time_with_drop = np.average(max_time_ID_with_drop['TimeElapsed'])&#xA;&#xA;dataset = dataset.drop(columns=['TimeStamp'])&#xA;dataset_drop = dataset_drop.drop(columns=['TimeStamp'])&#xA;&#xA;grouped = dataset.groupby('product_number')&#xA;ncols = 4&#xA;nrows = int(np.ceil(grouped.ngroups/40))&#xA;for i in range(10):&#xA;    fig, axes = plt.subplots(figsize=(12,4), nrows = nrows, ncols = ncols)&#xA;    for (key, ax) in zip(grouped.groups.keys(), axes.flatten()):&#xA;        grouped.get_group((20*i)+key).plot(x='TimeElapsed', y=['Flow_Ar-EDGE'], ax=ax, sharex = True, sharey = True)&#xA;        ax.set_title('product_number=%d'%((20*i)+key))&#xA;        ax.get_legend().remove()&#xA;        handles, labels = ax.get_legend_handles_labels()&#xA;        fig.legend(handles, labels, loc='upper center')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for the help.&lt;/p&gt;&#xA;" OwnerUserId="21233" LastActivityDate="2019-03-25T15:49:31.813" Title="Is there a way to compare the similarities among different graphs and then cluster them using Unsupervised learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;unsupervised-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11448" PostTypeId="2" ParentId="11442" CreationDate="2019-03-25T16:46:53.723" Score="2" Body="&lt;p&gt;If the 'algorithm' you're talking about is a neural network, then you can distribute the learned parameters/weights to anyone who wants to use it. This is how neural nets are normally 'exported': without all of the training data used to create them. Actually, this is done with many kinds of models (&lt;a href=&quot;https://en.wikipedia.org/wiki/Parametric_family&quot; rel=&quot;nofollow noreferrer&quot;&gt;parameterized ones&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to 'decode' the model, users would only have to know its structure. In the case of a neural network, they'd need to know the size of each layer, what activation functions were used, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not possible with every type of ML model, however. Specifically, &lt;a href=&quot;https://en.wikipedia.org/wiki/Nonparametric_statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-parametric models&lt;/a&gt; and 'lazy' models make use of training data at inference time. They wouldn't be useful without their training data. Classifying an input by finding its &lt;a href=&quot;https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;k nearest neighbors&lt;/a&gt;, for example, would require having the training data.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-03-25T16:46:53.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11449" PostTypeId="2" ParentId="11439" CreationDate="2019-03-25T16:56:35.600" Score="1" Body="&lt;p&gt;the use of Tanh is purely because it fits the described problem (Especially for values that are min-max normalized). I have worked on couple of professional RL projects ( specifically with actions in the continuous space) and I did not use tanh at all. Hope that helped :)&lt;/p&gt;&#xA;" OwnerUserId="23455" LastEditorUserId="23455" LastEditDate="2019-05-16T03:41:37.453" LastActivityDate="2019-05-16T03:41:37.453" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11451" PostTypeId="2" ParentId="11433" CreationDate="2019-03-25T19:47:58.837" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the method itself defective or anything wrong with my code?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There does indeed appear to be an issue with the code, the publications are fine (I know most of those authors and would very much trust their writing too :) ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first issue I see, and likely the most important, is that the &lt;code&gt;update()&lt;/code&gt; calls of &lt;code&gt;DynamicPBA&lt;/code&gt; frequently update the contents of &lt;code&gt;self.q_table&lt;/code&gt; to those of &lt;code&gt;self.q_table_&lt;/code&gt;, &lt;strong&gt;but the contents of &lt;code&gt;self.q_table_&lt;/code&gt; are never updated&lt;/strong&gt;. So, your &lt;code&gt;q_table&lt;/code&gt; is essentially always filled with a bunch of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; values that never get a chance to properly learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did not check the rest of the implementation in detail, but at a glance it all looks fine to me. So I guess that changing the last line of &lt;code&gt;learn()&lt;/code&gt; to update an entry in &lt;code&gt;self.q_table_&lt;/code&gt; rather than &lt;code&gt;self.q_table&lt;/code&gt; should go a long way towards fixing your issue.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-03-25T19:47:58.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11453" PostTypeId="1" AcceptedAnswerId="11454" CreationDate="2019-03-25T22:03:20.690" Score="2" ViewCount="91" Body="&lt;p&gt;From this &lt;a href=&quot;http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;, I read that &quot;to accurately classify data with neural networks, &lt;strong&gt;wide&lt;/strong&gt; layers are sometimes necessary.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have seen many implementations and discussions on deep-learning, such as &lt;a href=&quot;https://skymind.ai/wiki/neural-network&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, mention the concept of &lt;strong&gt;depth&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference in the context of neural networks?  How does width vs depth impact a neural network's performance?&lt;/p&gt;&#xA;" OwnerUserId="22424" LastActivityDate="2019-03-25T22:11:24.217" Title="What does &quot;Wide&quot; vs. &quot;Deep&quot; mean in the context of Neural Networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11454" PostTypeId="2" ParentId="11453" CreationDate="2019-03-25T22:11:24.217" Score="6" Body="&lt;p&gt;The width refers to the number of neurons in a layer. The depth refers to the number of layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at the following question regarding the impact of these hyper-parameters on the performance of the neural network: &lt;a href=&quot;https://stats.stackexchange.com/q/214360/82135&quot;&gt;https://stats.stackexchange.com/q/214360/82135&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-03-25T22:11:24.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11455" PostTypeId="1" AcceptedAnswerId="11463" CreationDate="2019-03-25T22:46:28.447" Score="6" ViewCount="729" Body="&lt;p&gt;As far as I know, Stochastic Gradient Descent is an optimization algorithm which belongs to the the category of algorithms where hyper-parameters have to be defined beforehand. They are useful in many cases, but there are some cases that the adaptive learning algorithms (like AdaGrad or Adam) might be preferable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When are algorithms like Adam and AdaGrad preferred over SGD? What are the cons and pros of adaptive algorithms, like Adam, when we compare them with learning algorithms like SGD?&lt;/p&gt;&#xA;" OwnerUserId="23460" LastEditorUserId="2444" LastEditDate="2019-03-26T14:15:08.243" LastActivityDate="2019-03-26T14:15:08.243" Title="When should we use algorithms like Adam as opposed to SGD?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11456" PostTypeId="2" ParentId="11421" CreationDate="2019-03-26T07:53:32.193" Score="3" Body="&lt;p&gt;One option is not mentioned by &lt;strong&gt;malioboro&lt;/strong&gt; is getting more data. Getting bigger dataset is almost always improve training results. If it's too hard to obtain more labeled data you can use data augmentation on existing data - small random transformations while keeping the same label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For images most common methods of augmentations are (applying padding if needed):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;small random zooming in/out &lt;/li&gt;&#xA;&lt;li&gt;small random shifts of image&lt;/li&gt;&#xA;&lt;li&gt;adding random noise to image&lt;/li&gt;&#xA;&lt;li&gt;small random change in brightness, contrast, color balance and similar parameters&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are more complex methods, but they are dependent on specific of dataset/goal of training&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stopping: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you should stop training if you see training error is not decreasing - your have reached (possibly local) minima.&lt;/li&gt;&#xA;&lt;li&gt;you should stop if testing or validation error start to increase - your method is overfitting (data augmentation can help in that case)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-03-26T07:59:20.003" LastActivityDate="2019-03-26T07:59:20.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11459" PostTypeId="2" ParentId="11407" CreationDate="2019-03-26T08:47:59.307" Score="1" Body="&lt;p&gt;The equation &lt;span class=&quot;math-container&quot;&gt;$\frac12(x_1w_1 + x_2w_2 - y)^2$&lt;/span&gt; is called the &lt;span class=&quot;math-container&quot;&gt;$Error (E)$&lt;/span&gt; (assuming &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to be continuous which is not the case in case of classifiers). If you write this equation in Physics or Maths it represents a family of curves in 4D (the curves are continuous but for visualisation we will assume it to be a family of curves).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a representative equation of what it would have looked like had the error been &lt;span class=&quot;math-container&quot;&gt;$\frac12(x_1w_1 - y)^2$&lt;/span&gt; a 3D curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bLAbH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bLAbH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a scalar quantity which represents the value of error at different places for different values of &lt;span class=&quot;math-container&quot;&gt;$w1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w2$&lt;/span&gt;. Now gradient of a scalar is defined as &lt;span class=&quot;math-container&quot;&gt;$\nabla F$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is a scalar, on doing this operation you get a vector, which is perpendicular to the &lt;strong&gt;equi-potential&lt;/strong&gt; or more suitably &lt;strong&gt;equi-error&lt;/strong&gt; surface, i.e. if you trace all the points which give the same error, you will get a curve, and its gradient at any point is the vector perpendicular to the curve at that given point. There are many proofs for this but here is a very simple and nice &lt;a href=&quot;https://math.stackexchange.com/questions/599488/why-gradient-vector-is-perpendicular-to-the-plane&quot;&gt;proof&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets look at the equation of the constraint &lt;span class=&quot;math-container&quot;&gt;$x_1w_1 + x_2w_2 = y$&lt;/span&gt;. In case of a 3D error curve, the constraint is giving us a &lt;strong&gt;plane&lt;/strong&gt; which is &lt;strong&gt;parallel&lt;/strong&gt; to the &lt;strong&gt;tangential plane&lt;/strong&gt; of the &lt;strong&gt;equi-error&lt;/strong&gt; surface at a given point. You can look at this &lt;a href=&quot;http://www.math.ucla.edu/~ronmiech/Calculus_Problems/32A/chap12/section4/793d3/793_3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;method&lt;/a&gt; of how to find tangential planes and derive the plane yourself, where &lt;span class=&quot;math-container&quot;&gt;$z = Error(E)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are your &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus it is quite clear that the gradient will be perpendicular to the constraint, and this is the reason we use gradients because according to mathematics if you move in a direction perpendicular to an equi-potential surface you get the maximum change than any other direction for same &lt;span class=&quot;math-container&quot;&gt;$dl$&lt;/span&gt; movement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would highly suggest you check out these &lt;a href=&quot;https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/gradient-and-directional-derivatives/v/gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;videos&lt;/a&gt; on gradient from Khan academy. This will hopefully give you a more intuitive understanding of why we do what we do in Neural Networks.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-03-26T08:47:59.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11460" PostTypeId="1" CreationDate="2019-03-26T09:27:52.700" Score="4" ViewCount="41" Body="&lt;p&gt;This question came after I connected 2 pieces of information :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I recently listened to &lt;a href=&quot;https://futureoflife.org/2019/01/30/ai-alignment-podcast-the-byzantine-problem-poisoning-and-distributed-machine-learning-with-el-mahdi-el-mhamdi-beneficial-agi-2019/?cn-reloaded=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Byzantine Generals’ Problem, Poisoning, and Distributed Machine Learning with El Mahdi El Mhamdi (Beneficial AGI 2019)&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Today, I was laughing with a friend at &quot;quantum computing&quot; being mentioned in a job offer. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Considering the probabilistic nature of results in quantum computers, what would be the advantages to using byzantine resistant neural networks on quantum computer? Has this already been attempted?&lt;/p&gt;&#xA;" OwnerUserId="23466" LastEditorUserId="23466" LastEditDate="2019-07-17T09:29:57.097" LastActivityDate="2019-07-17T09:29:57.097" Title="Can there be applications of byzantine neural networks on quantum computers?" Tags="&lt;neural-networks&gt;&lt;quantum-computing&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11461" PostTypeId="1" CreationDate="2019-03-26T09:40:36.617" Score="0" ViewCount="66" Body="&lt;p&gt;I need a pathfinding algorithm that considers the history of visited nodes and varies its path depending on some rules (like already visited). Are there good approaches serving this purpose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific: Let's say I have a graph representing a map. I want to find a Route from B to D: Once I am in D following B-&gt;C-&gt;D, I want to calculate a new path, let's say to A. The shortest path would be D-&gt;C-&gt;B-&gt;A. But I want a path with unvisited nodes even if it's longer than the shortest possible path. The new path should be the shortest among all possible paths according to the rules.&#xA;Another example is the game &quot;snake&quot;. Seeing the grid as a graph I cannot visit already visited nodes as fas as the corpus (of the snake) is in that node (or I have visited the node t time steps ago)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the problem is too specific and I have to implement some basic pathfinding algorithm in a wider algorithm.&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="19413" LastEditDate="2019-03-26T15:53:49.180" LastActivityDate="2019-03-26T16:25:46.187" Title="Are there any pathfinding algorithms that take customized rules into account when determining the shortest path?" Tags="&lt;path-finding&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11462" PostTypeId="1" CreationDate="2019-03-26T09:57:58.457" Score="1" ViewCount="114" Body="&lt;p&gt;I try to implement RL to a case something like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This game consist of several rounds. Every round the players need to generate a maze that consists of rooms. There are around 1000 different available rooms with different properties. At the beginning of a round, each player will be given 10 rooms one-by-one (the sequence is same for each player) from 1000 that available, then he/she try to create a maze from the taken rooms (by arranging each room). After the maze is done there is a Game Master who will judge the maze (give a score between 0-100). The player never know how the Game Master judges the maze, it can be judged based on the level of difficulty that is produced, the order of the room we compose, or others. The player who got the best score for the given rooms will win this round.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;in this case, I have around 100,000 &quot;perfect&quot; mazes that have been created from different room combination and got a perfect score. I use this maze as episodes and try to train RL-Agent to find the pattern of how the Game Master judges a maze. For your information, there are rooms that not exist in the 100,000 perfect mazes, but I hope the RL-Agent can use its properties to find similar rooms that exist in  the &quot;perfect&quot; mazes, and make it as a reference&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This case is different from other RL environments that I've ever met before, generating an episode is not an easy task because it needs an expert to validate it (The Game Master). So you could say, I can only build the RL agent using that 100,000 episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, even though it only consists of 100,000 episodes, my case has millions of states, so I plan to use Q-Learning with Neural Net as approximator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in this case, am I still need the Experience Replay process (I am afraid I don't need it because of the small number of available episodes)? &lt;/li&gt;&#xA;&lt;li&gt;has this case ever happened before? What is the best approach to deal with cases where the number of episodes is limited?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2019-05-03T03:22:29.857" LastActivityDate="2019-05-03T03:22:29.857" Title="Reinforcement Learning with limited number of episodes" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="11463" PostTypeId="2" ParentId="11455" CreationDate="2019-03-26T10:01:46.080" Score="5" Body="&lt;p&gt;Empirically, I observed that algorithms like Adam and RMSProp tended to give me a final higher performance (in my case, the accuracy) on (the validation dataset) with respect to SGD. However, I also observed that Adam and RMSProp are highly sensitive to certain values of the learning rate (and, sometimes, other hyper-parameters like the batch size) and they can catastrophically fail to converge if e.g. the learning rate is too high. On the other hand, in general, SGD have not led me to the highest performance, but they did not catastrophically fail (at least, as much as Adam and RMSProp) in my experiments (even when using quite different hyper-parameters). I noticed that the learning rate (and the batch size) are the hyper-parameters that mainly affect the performance of all these algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experiments, I use SGD without momentum and I used the (PyTorch) default values of Adam and RMSProp. I only compared SGD with Adam and RMSProp, on the relatively simple task of recognising MNIST digits. You can have a look at this repository &lt;a href=&quot;https://github.com/nbro/comparative-study-between-optimizers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nbro/comparative-study-between-optimizers&lt;/a&gt;, which contains the code I used to perform these experiments. You also have the instructions there to perform the experiments (if you want).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-03-26T10:01:46.080" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11464" PostTypeId="1" CreationDate="2019-03-26T10:33:47.713" Score="3" ViewCount="310" Body="&lt;p&gt;I was given the following problem to solve.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given a circular trail divided by &lt;span class=&quot;math-container&quot;&gt;$n&amp;gt; 2$&lt;/span&gt; segments labeled &lt;span class=&quot;math-container&quot;&gt;$0 \dots n-1$&lt;/span&gt;. In the beginning, an agent is at the start of segment number &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (the edge between segments &lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) and the agent's speed (&lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;) is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; segments per minute.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;At the start of each minute the agent take one of three actions:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;strong&gt;speed up&lt;/strong&gt;: the agent's speed increases by &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; segment per minute.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;slow down&lt;/strong&gt;:  the agent's speed decreases by &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; segment per minute.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;keep the same speed&lt;/strong&gt;: stay at the same speed.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;The action &lt;strong&gt;slow down&lt;/strong&gt; cannot be used if the current speed is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; segments per minute.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The cost of each action is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The goal of the agent is to drive around the trail k times (&lt;span class=&quot;math-container&quot;&gt;$1 \leq k$&lt;/span&gt;)  and then park in the beginning spot (at speed 0 of course). The agent needs to do that in the minimum amount of actions.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The heuristic given is: &lt;em&gt;If agent is in segment &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; then: &lt;span class=&quot;math-container&quot;&gt;$n-z$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$z \not = 0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I need to find if the given heuristic is admissible (or complete) and consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Regarding consistency: A heuristic is consistent if its estimate is always &lt;span class=&quot;math-container&quot;&gt;$\leq$&lt;/span&gt; estimated distance from any given neighbour vertex to goal plus cost of reaching a goal. So, in the given problem, it is consistent because (&lt;span class=&quot;math-container&quot;&gt;$n&amp;gt;2$&lt;/span&gt;), so heuristic function is well defined and because of circular trail divided by &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; segments with a constant price of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for each action, then the given definition of consistency holds because estimated distance from any given neighbour vertex to goal can be looked on as a difference between segments until reaching a goal and it is consistent because again the function is well defined.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Regarding admissibility: An admissible heuristic is one that the cost to reach goal is never more than the lowest possible cost from the current point to reach the goal. I am not sure if the given heuristic is admissible because it does not help much to know the difference between trail size (&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; = segment size) and current place. But it does not create flaws, so it is probably admissible. I am not sure this is a proof.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is my idea correct? How could I write it as a proof?&lt;/p&gt;&#xA;" OwnerUserId="23467" LastEditorUserId="2444" LastEditDate="2019-11-10T16:53:00.210" LastActivityDate="2019-11-10T16:53:00.210" Title="How do I find whether this heuristic is or not admissible and consistent?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;heuristics&gt;&lt;admissible-heuristic&gt;&lt;consistent-heuristic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11465" PostTypeId="1" CreationDate="2019-03-26T11:04:09.213" Score="1" ViewCount="42" Body="&lt;p&gt;I have a column with links to websites and another column with keywords from those websites. I have to find a map between these two, such that for a new input, which is a website's URL, I can generate the keywords associated with the contents of the website.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given the URL &lt;code&gt;chocolate.com&lt;/code&gt;, the keywords could be &lt;code&gt;milk&lt;/code&gt; and &lt;code&gt;dark&lt;/code&gt;. We can tell from this example that my keywords are types of chocolates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, if the URL is &lt;code&gt;career.com&lt;/code&gt;, then the keywords could be &lt;code&gt;IT&lt;/code&gt; and &lt;code&gt;medicine&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a feeling that some sort of supervised neural networks could be used here.   Which approach would be most suited here?&lt;/p&gt;&#xA;" OwnerUserId="23468" LastEditorUserId="2444" LastEditDate="2019-11-06T23:12:26.647" LastActivityDate="2019-11-06T23:12:26.647" Title="How can I generate keywords associated with a website given its URL?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;supervised-learning&gt;&lt;text-summarization&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11466" PostTypeId="1" CreationDate="2019-03-26T13:14:18.797" Score="1" ViewCount="102" Body="&lt;p&gt;I was going through &lt;a href=&quot;https://github.com/ianxxiao/reinforcement_learning_project/blob/master/code/training.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; implementation of Reinforcement learning where model is being trained to manage the number of bikes at a station.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, line 78 represents the loop over all episodes (if I understood correctly). In line 92, the DQN Agent is defined meaning after each episode, the agent will be reset to default parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But shouldn't we define the model before the loop starts because the after each episode, won't all the previous learning be lost if we initialize the class object in each iteration? &#xA;Am I misinterpreting anything?&lt;/p&gt;&#xA;" OwnerUserId="19244" LastActivityDate="2019-03-26T13:14:18.797" Title="Do we need to reset the DQN network after every episode?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11467" PostTypeId="2" ParentId="11461" CreationDate="2019-03-26T16:25:46.187" Score="1" Body="&lt;p&gt;Yes, this is easily solved using the A* algorithm. Once your agent has visited a particular node, increase the cost of that node to infinity and recalculate the path.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-03-26T16:25:46.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11468" PostTypeId="2" ParentId="11464" CreationDate="2019-03-26T20:36:19.383" Score="2" Body="&lt;p&gt;Welcome to AI.SE @hpr16!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your understanding of when a heuristic is admissible is correct, but your heuristic is inadmissible. An admissible heuristic must always underestimate the cost to move from a given &lt;em&gt;state&lt;/em&gt; to a goal state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that states in the search are &lt;em&gt;not&lt;/em&gt; the same as positions on the circle in your problem. A state needs to capture all the information about the current environment the agent is in. In your problem, agents have a speed as well as a position. A state must, therefore, contain both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why your heuristic is inadmissible because the agent can move (n-z) segments in less than n-z steps: it can speed up, and do them in, for example, (n-z)/2 steps, by moving with speed 2. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-26T20:36:19.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11469" PostTypeId="1" CreationDate="2019-03-26T20:43:07.783" Score="5" ViewCount="68" Body="&lt;p&gt;In the documentary about the match, it is said that after losing the 4th game, AlphaGo came back stronger and started to play in a weird way (not human-like) and it was pretty impossible to be beaten. Why and how did that happen? &lt;/p&gt;&#xA;" OwnerUserId="21832" LastEditorUserId="1671" LastEditDate="2019-03-26T20:56:47.093" LastActivityDate="2019-03-27T04:01:42.990" Title="Why didn't champion of the Go game manage to win the last game against AlphaGo, after winning the 4th one?" Tags="&lt;game-ai&gt;&lt;soft-question&gt;&lt;alphago&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11471" PostTypeId="1" CreationDate="2019-03-27T03:11:54.003" Score="1" ViewCount="46" Body="&lt;p&gt;I was researching about hierarchical object detection, and end up reading that Yolo v3 is the state of art for that kind of tasks, besides, the inference time make it one of the best for run it on live video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what I have in mind, is to run a pose estimation technology over the live video (LikeOpenPose), then focus only in the rectangles near the hands of the estimated pose in order to detect the object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the previous approach sounds good, but I feel like I'm not taking advantage of the temporal features on the video, for example, YoloV3 could not be very sure that someone has a cellphone with only the rectangle of the hands, but if I add up, the movement of the estimated pose (hand near to the head for several frames), I could be more sure that he has a phone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I cannot find a paper, approach or something close to the idea I have on mind, so I was wondering if someone here could give me a little clue about what path should I follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help!&lt;/p&gt;&#xA;" OwnerUserId="23491" LastActivityDate="2019-03-27T03:11:54.003" Title="Live video object detection with pose estimation" Tags="&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11472" PostTypeId="2" ParentId="11418" CreationDate="2019-03-27T03:58:43.457" Score="0" Body="&lt;p&gt;I emailed the author of the paper and he replied that randomness in the parameter initialization is the only difference between Q1 and Q2. This difference is enough in practice. Moreover, TD3 method is more concerned with overestimation induced by function approximation error rather than stochasticity in the environment.&lt;/p&gt;&#xA;" OwnerUserId="23420" LastActivityDate="2019-03-27T03:58:43.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11473" PostTypeId="2" ParentId="11469" CreationDate="2019-03-27T04:01:42.990" Score="3" Body="&lt;p&gt;The technique used by AlphaGo is &quot;Monte Carlo Tree Search&quot;, combined with a very well trained neural network. The network's job is to estimate the quality of different board states and moves. This estimation is deterministic. If you show AlphaGo the same board on two different occasions, it thinks it is exactly as good (or bad) on both occasions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search&lt;/a&gt; however, is a &lt;em&gt;randomized&lt;/em&gt; algorithm. So as a simplified explanation, the way AlphaGo decides which moves to make is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Look at the current board.&lt;/li&gt;&#xA;&lt;li&gt;Pick a random move, and imagine what the board would look like if you made that move.&lt;/li&gt;&#xA;&lt;li&gt;Pick a random move for your opponent, and imagine what the board would look like if they made that move.&lt;/li&gt;&#xA;&lt;li&gt;Keep doing steps 2 &amp;amp; 3 for a while, so that we're imagining being at some board many moves in the future along a random line of play.&lt;/li&gt;&#xA;&lt;li&gt;Ask the neural network how good this board state is.&lt;/li&gt;&#xA;&lt;li&gt;Repeat steps 1-5 many times. Then make whichever move led to the best lines of play &lt;em&gt;on average&lt;/em&gt;, to make right now.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What this means is, AlphaGo won't always play the same way, because it doesn't actually consider every move explicitly. It just thinks about enough lines of play to be pretty confident about whether one move is better than another. This is actually not so far removed from how humans play most of the middle of games like these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in game 4, essentially, &lt;strong&gt;Sedol got lucky&lt;/strong&gt;. The random lines of play that AlphaGo chose to look at did not capture some critical facts about one or more board states. This led it to make a mistake. If you asked it to play the same game through again, it might not make the same mistakes (it might think about different random lines of play, that &lt;em&gt;do&lt;/em&gt; capture the critical facts it missed in the first game). Further, it might choose to play slightly differently on other moves, which could have a big impact on the rest of the game. These two factors prevented Sedol from simply playing game 4 over again.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-27T04:01:42.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11474" PostTypeId="1" CreationDate="2019-03-27T07:34:42.463" Score="1" ViewCount="75" Body="&lt;p&gt;I am trying to create a chatbot whose dialogue policy model will be trained through reinforcement learning. Dialogue Policy is responsible for selecting the action to take based on the given state of the conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All implementations I see for RL are trained from an environment taken from &lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gym&lt;/a&gt; or created manually. These environments provide the next state, rewards etc to the model based on which it is trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I am creating a dialogue policy model which will be trained through real user conversations, I cannot provide a &quot;pre-defined&quot; environment which can provide the states and rewards. I am planning to train it myself by talking to it and providing rewards and next state (which I think is called interactive learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I was not able to find any implementations, tutorials or articles on interactive learning. I am not able to figure out how to create such a model, how to take care of the episodes, sessions etc. This will be a continuous learning that will go on for months maybe. I have to save the model each day and continue training the next day by loading the model from that same state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone guide me in the correct direction on how to approach this? Any githubs links, articles, tutorials of such implementations will be highly appreciated. I am aware this question seems too broad, but some hints will be very helpful for a newbie like me.&lt;/p&gt;&#xA;" OwnerUserId="19244" LastActivityDate="2019-03-27T18:14:25.177" Title="How to build a DQN agent which can be trained through interactive learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;chat-bots&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11478" PostTypeId="1" AcceptedAnswerId="11494" CreationDate="2019-03-27T12:13:29.973" Score="5" ViewCount="318" Body="&lt;p&gt;Can neural networks change or evolve other neural networks? Also, could evolutionary algorithms be applied to evolve neural networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that we have neural networks A and B. The neural network B changes the neural network A. If B &quot;successfully&quot; changed it, NN A will survive.&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2019-07-28T19:57:57.953" LastActivityDate="2019-07-29T14:26:14.290" Title="Can neural networks evolve other neural networks?" Tags="&lt;neural-networks&gt;&lt;evolutionary-algorithms&gt;&lt;agi&gt;&lt;neuroevolution&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11479" PostTypeId="1" CreationDate="2019-03-27T12:59:53.067" Score="3" ViewCount="45" Body="&lt;p&gt;SVM is designed for two-class classification problem. If the data is not linear-separable, a kernel function is used. I want to know  if there is exists any method that will indicate if the data is linearly separable or not. &lt;/p&gt;&#xA;" OwnerUserId="23501" LastActivityDate="2019-03-27T12:59:53.067" Title="Is there any formal test for linear separability of 2-class data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11480" PostTypeId="1" CreationDate="2019-03-27T13:01:19.460" Score="6" ViewCount="1291" Body="&lt;p&gt;I don't know what people mean by 'vanilla policy gradient', but what comes to mind is REINFORCE, which is the simplest policy gradient algorithm I can think of. Is this an accurate statement?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By REINFORCE I mean this surrogate objective&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{1}{m} \sum_i \sum_t log(\pi(a_t|s_t)) R_i $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I indices over the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; episodes and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; over time steps, and &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; is the total reward of the episode. It's also common to replace &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; with something else, like a baselined version &lt;span class=&quot;math-container&quot;&gt;$R_i - b$&lt;/span&gt; or use the future return, potentially also with a baseline &lt;span class=&quot;math-container&quot;&gt;$G_{it} - b$&lt;/span&gt;. However, I think even with these modifications to the multiplicative term, people would still call this 'vanilla policy gradient'. Is that correct?&lt;/p&gt;&#xA;" OwnerUserId="17312" LastEditorUserId="2444" LastEditDate="2019-05-31T21:54:23.450" LastActivityDate="2019-05-31T21:54:23.450" Title="Is REINFORCE the same as 'vanilla policy gradient'?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;policy-gradients&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11481" PostTypeId="1" CreationDate="2019-03-27T13:30:57.687" Score="0" ViewCount="383" Body="&lt;p&gt;I'm trying to implement my own DQN. So far I think my code is good, but my Q-values (I'm getting the mean of all the values for every episode) tends to converge near-zero but negatively. It is normal? Or there is something wrong in my implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My exploration vs explotation greedy strategy goes from 1.0 to 0.1 in 1 million steps (as DeepMind does), my learning rate is 0.00025 and my gamma 0.99. I read &lt;a href=&quot;https://rubenfiszel.github.io/posts/rl4j/2016-08-24-Reinforcement-Learning-and-DQN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that &#xA;&quot;The mean Q-values should smoothly converge towards a value proportionnal to the mean expected reward.&quot;&#xA;So, it's my agent expecting a negative reward? If so, how can i fix it?&#xA;Here is a graph of the first training session:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YiVRh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YiVRh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;You can see how the Q-values tend to converge near-zero after about 1300 episodes (1120000 steps aproximately). Actually it's showing values like -0.0117, -0.0145, etc.&#xA;Also, the agent seems very &quot;static&quot; after epsilon gets near 0.1, and when it reaches it doesn't move so much. (I'm training with &lt;code&gt;PongDeterministic-v4&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="9818" LastActivityDate="2019-05-09T14:01:45.313" Title="DQN Q-mean values converge negatively" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11483" PostTypeId="1" CreationDate="2019-03-27T13:47:21.110" Score="2" ViewCount="57" Body="&lt;p&gt;In order to update the belief state in a POMDP, the following formula is used:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$b'(s')=\frac{O(a, s', z) \sum_{s\in S} b(s)T(s, a, s')}{\mathbb{P}(z \mid b, a)}$$&lt;/span&gt;&#xA;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is a specific state in the set of states &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$b'(s')$&lt;/span&gt; is the updated belief state of being in the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$T(s, a, s') = \mathbb{P}(s' \mid s, a)$&lt;/span&gt; is the propability (function) of having been in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and ending up in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; by taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O(a, s', z) = \mathbb{P}(z \mid s', a)$&lt;/span&gt; the probability (function) of observing &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, performing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and ending up in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(z \mid b, a)$&lt;/span&gt; is defined as follows &lt;span class=&quot;math-container&quot;&gt;$\sum_{s \in S}b(s)\sum_{s' \in S} T(s, a, s')O(a, s', z)$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Looking at &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(z \mid b, a)$&lt;/span&gt; it is possible that the result is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. This would be the case if the agent is in a state where no further actions are possible. But, in that case, there is a problem with updating &lt;span class=&quot;math-container&quot;&gt;$b'(s')$&lt;/span&gt;, since this causes a zero division. Is this a common problem and is the only possibility to avoid that a programming solution like an if-statement? Or is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(z \mid b, a)$&lt;/span&gt; always non-zero?&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="2444" LastEditDate="2019-03-28T14:52:48.933" LastActivityDate="2019-03-28T21:11:41.690" Title="Can the normalization factor for the belief state update be zero?" Tags="&lt;reinforcement-learning&gt;&lt;pomdp&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11485" PostTypeId="1" AcceptedAnswerId="11486" CreationDate="2019-03-27T15:06:57.007" Score="2" ViewCount="282" Body="&lt;p&gt;As my first AI model I have decided to make an AI model to predict multiplication of two numbers EX - [2,4] = [8]. I wrote the following code, but the loss is very high, around thousands, and it's very inaccurate. How do I make it more accurate?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;&#xA;data = torch.tensor([[2,4],[3,6],[3,3],[4,4],[100,5]],dtype=torch.float)&#xA;values = torch.tensor([[8],[18],[9],[16],[500]],dtype=torch.float)&#xA;lossfun = torch.nn.MSELoss()&#xA;model=Net()&#xA;optim = torch.optim.Adam(model.parameters(),lr=0.5)&#xA;&#xA;class Net(nn.Module):&#xA;    def __init__(self):&#xA;&#xA;        super(Net,self).__init__();&#xA;&#xA;        self.fc1 = nn.Linear(in_features=2,out_features=3)&#xA;        self.fc2 = nn.Linear(in_features=3,out_features=6)&#xA;        self.out = nn.Linear(in_features=6,out_features=1)&#xA;&#xA;    def forward(self,x):&#xA;        x = self.fc1(x)&#xA;        x = F.relu(x)&#xA;        x = self.fc2(x)&#xA;        x = F.relu(x)&#xA;        x = self.out(x)&#xA;        return x&#xA;for epoch in range(1000):&#xA;&#xA;    y_pred=model.forward(data)&#xA;&#xA;    loss = lossfun(y_pred,values)&#xA;&#xA;    print(loss.item())&#xA;&#xA;    loss.backward()&#xA;&#xA;    optim.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: I am a newbie in AI and ML.&lt;/p&gt;&#xA;" OwnerUserId="23507" LastEditorUserId="16229" LastEditDate="2019-03-30T15:51:48.930" LastActivityDate="2019-03-30T15:51:48.930" Title="Heavy loss and inaccurate answer in pytorch" Tags="&lt;loss-functions&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11486" PostTypeId="2" ParentId="11485" CreationDate="2019-03-27T16:46:32.800" Score="2" Body="&lt;p&gt;There are a few things you could do to improve this NN, but are probably worth covering in different questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your main problem though is that you forgot to reset the gradient after each training batch. You need to call &lt;code&gt;optim.zero_grad()&lt;/code&gt; in order to do this, at the start of each training loop. Otherwise, using PyTorch, the gradient values keep accumulating inside the model's training data (sometimes you want this effect if you are adding gradients from multiple sources, that's why PyTorch is not clearing them automatically for you).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, a learning rate of 0.5 is very high for the Adam optimiser - it is very common to leave it at the default value because Adam is an adaptive optimiser that will adjust step sizes depending on gradients seen so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a working version of your code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;&#xA;class Net(nn.Module):&#xA;    def __init__(self):&#xA;&#xA;        super(Net,self).__init__();&#xA;&#xA;        self.fc1 = nn.Linear(in_features=2,out_features=3)&#xA;        self.fc2 = nn.Linear(in_features=3,out_features=6)&#xA;        self.out = nn.Linear(in_features=6,out_features=1)&#xA;&#xA;    def forward(self,x):&#xA;        x = self.fc1(x)&#xA;        x = torch.relu(x)&#xA;        x = self.fc2(x)&#xA;        x = torch.relu(x)&#xA;        x = self.out(x)&#xA;        return x&#xA;&#xA;data = torch.tensor([[2,4],[3,6],[3,3],[4,4],[100,5]],dtype=torch.float)&#xA;values = torch.tensor([[8],[18],[9],[16],[500]],dtype=torch.float)&#xA;&#xA;lossfun = torch.nn.MSELoss()&#xA;model=Net()&#xA;optim = torch.optim.Adam(model.parameters(),lr=0.001)&#xA;&#xA;for epoch in range(20000):&#xA;    optim.zero_grad()&#xA;    y_pred=model.forward(data)&#xA;    loss = lossfun(y_pred,values)&#xA;    if (epoch % 1000 == 0):&#xA;        print(loss.item())&#xA;    loss.backward()&#xA;    optim.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This version can be tweaked to quite easily reach 0 loss for your data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has not in really learned how to multiply two values. The approximation to multiplying will be very weak as there is very little data. However, playing with some very basic data and a simple NN is a first step towards understanding details like this . . .&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-27T16:53:38.730" LastActivityDate="2019-03-27T16:53:38.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11487" PostTypeId="1" AcceptedAnswerId="13040" CreationDate="2019-03-27T17:29:05.870" Score="4" ViewCount="216" Body="&lt;p&gt;Suppose I have a deep feed-forward neural network with sigmoid activation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; already trained on a dataset &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. Let's consider a training point &lt;span class=&quot;math-container&quot;&gt;$x_i \in S$&lt;/span&gt;. I want to analyze the entries of a hidden layer &lt;span class=&quot;math-container&quot;&gt;$h_{i,l}$&lt;/span&gt;, where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h_{i,l} = \sigma(W_l ( \sigma (W_{l-1} \sigma( \dots \sigma ( W_1 \cdot x_i))\dots).        $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition would be that, since gradient descend has passed many times on the point &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; updating the weights at every iteration, the entries of every hidden layer computed on &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; would be either very close to zero or very close to one (thanks to the effect of the sigmoid activation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this true? Is there a theoretical result in the literature which shows anything similar to this? Is there an empirical result which shows that?  &lt;/p&gt;&#xA;" OwnerUserId="21338" LastEditorUserId="2444" LastEditDate="2019-03-27T21:19:55.887" LastActivityDate="2019-06-24T23:32:56.393" Title="How do intermediate layers of a trained neural network look like?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11489" PostTypeId="2" ParentId="11474" CreationDate="2019-03-27T18:14:25.177" Score="1" Body="&lt;p&gt;I suggest you to start reading this one on &lt;a href=&quot;https://github.com/google-research/bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt; and this one on &lt;a href=&quot;https://github.com/openai/gpt-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT-2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am aware this question seems too broad, but some hints will be very helpful for a newbie like me.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure you want to create your chatbot using RL architecture at all. But if you want to implement such an idea the right way to go with that is by using iterative approach and starting from a really simple base:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Define a list of actions for your agent. Might be 10-words&#xA;vocabulary with simple one-word answers. &lt;/li&gt;&#xA;&lt;li&gt;Define you simple environment. It might have 20 different states (e.g. greetings/conversation starters &lt;code&gt;hey&lt;/code&gt;, &lt;code&gt;hello&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Map good responses (&lt;code&gt;hello&lt;/code&gt; -&gt; &lt;code&gt;hi&lt;/code&gt;) to positive rewards and &lt;code&gt;-1&lt;/code&gt; for the rest of them.&lt;/li&gt;&#xA;&lt;li&gt;If you want to move forward with DQN try a simple Q-learning algorithm on your simple environment and train agent to properly response to those.&lt;/li&gt;&#xA;&lt;li&gt;Once you have done the above you can make your vocabulary bigger&lt;/li&gt;&#xA;&lt;li&gt;Now replace your table of input words which gives you a state for your network with NN/word2vec/any other NLP approach which will convert your input sentence to a state.&lt;/li&gt;&#xA;&lt;li&gt;Manually provide a reward as a response for what you agent did according to learned policy.&lt;/li&gt;&#xA;&lt;li&gt;Do that infinite amount of time and your agent probably will figure out how to respond properly.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: DQN uses discrete action space. So in such a case you will have only limited or REALLY HUGE actions space. It's all possible combinations between all possible words in you vocabulary (assuming &lt;em&gt;action&lt;/em&gt; maps to some &lt;em&gt;sentence&lt;/em&gt;)&lt;/p&gt;&#xA;" OwnerUserId="23434" LastActivityDate="2019-03-27T18:14:25.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11490" PostTypeId="1" CreationDate="2019-03-27T19:42:47.843" Score="1" ViewCount="53" Body="&lt;p&gt;In the paper &quot;Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps&quot;, &lt;a href=&quot;https://arxiv.org/abs/1312.6034&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1312.6034&lt;/a&gt;, at part 3, there is a first-order Taylor expansion(formula number 3 &amp;amp; 4) that I can't understand the logic behind it and how they are obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those formulas are about computing saliency map in convolutional neural networks.&lt;/p&gt;&#xA;" OwnerUserId="10051" LastActivityDate="2019-03-27T19:42:47.843" Title="Image-Specific Class Saliency Visualisation" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11491" PostTypeId="1" CreationDate="2019-03-28T03:07:34.627" Score="4" ViewCount="218" Body="&lt;p&gt;I am attempting to implement YOLO v3 in Tensorflow-Keras from scratch, with the aim of training my own model on a custom dataset. By that, I mean &lt;strong&gt;without using pretrained weights&lt;/strong&gt;. I have gone through all three papers for YOLOv1, YOLOv2(YOLO9000) and YOLOv3, and find that although Darknet53 is used as a feature extractor for YOLOv3, I am unable to point out the complete architecture which extends after that - the &quot;detection&quot; layers talked about &lt;a href=&quot;https://itnext.io/implementing-yolo-v3-in-tensorflow-tf-slim-c3c55ff59dbe&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. After a lot of reading on blog posts from Medium, kdnuggets and other similar sites, I ended up with a few significant questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Have I have missed the complete architecture of the detection layers (that extend after Darknet53 used for feature extraction) in YOLOv3 paper somewhere?&lt;/li&gt;&#xA;&lt;li&gt;The author seems to use different image sizes at different stages of training. Does the network automatically do this upscaling/downscaling of images?&lt;/li&gt;&#xA;&lt;li&gt;For preprocessing the images, is it really just enough to resize them and then normalize it (dividing by 255)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please be kind enough to point me in the right direction. I appreciate the help!&lt;/p&gt;&#xA;" OwnerUserId="21513" LastEditorUserId="21513" LastEditDate="2019-03-28T03:18:11.627" LastActivityDate="2019-03-28T03:18:11.627" Title="YOLO v3 complete architecture" Tags="&lt;tensorflow&gt;&lt;computer-vision&gt;&lt;keras&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11494" PostTypeId="2" ParentId="11478" CreationDate="2019-03-28T09:16:46.827" Score="5" Body="&lt;p&gt;Yes, this is an active area of research as we speak. Both using classic algorithms (decision trees, random forests, Bayesian ensembles) as well as neural networks. This can also be done via evolutionary algorithms. I have personally used them for hyperparameter tuning in a few cases where squeezing out a couple of extra points of accuracy was key. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in fact what Google is doing with their AutoML system. They are using neural networks for architecture search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a Github repo with some interesting papers and links on the topic you are describing: &lt;a href=&quot;https://github.com/hibayesian/awesome-automl-papers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hibayesian/awesome-automl-papers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="2444" LastEditDate="2019-07-07T22:32:18.870" LastActivityDate="2019-07-07T22:32:18.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11496" PostTypeId="2" ParentId="11480" CreationDate="2019-03-28T09:48:05.583" Score="2" Body="&lt;p&gt;By vanilla policy gradient, I think what is meant is normally any arbitrary policy gradient for the purposes of formalization(or whatever is trying to be communicated).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let &lt;span class=&quot;math-container&quot;&gt;$J(\theta)$&lt;/span&gt; be any policy objective function. Then our policy gradient would be &lt;span class=&quot;math-container&quot;&gt;$\begin{equation} \nabla_\theta J(\theta)  \end{equation}$&lt;/span&gt; where the change in our policy is &lt;span class=&quot;math-container&quot;&gt;$\Delta \theta = \alpha \nabla_\theta J(\theta)  $&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; being our step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will, of course, vary as notational convention and term definition can change between practitioners.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-03-28T09:48:05.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11500" PostTypeId="1" CreationDate="2019-03-28T12:56:12.917" Score="3" ViewCount="136" Body="&lt;p&gt;Can AI be used as a tool to investigate our minds? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise, what am I specifically asking for here are examples of discoveries on artificial intelligence (so algorithms, programs and computers that try to implement intelligent systems) that brought to light facts about intelligence and cognition &lt;em&gt;in general&lt;/em&gt;. Have this ever happened? Is it frequent? How influential and important were these discoveries, if any?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible example of what I mean could be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Physical_symbol_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSSH&lt;/a&gt;, which states that a formal system is sufficient to simulate general intelligent behaviour. I believe that this is relevant to Cognitive Science in general because it entails our understanding of this phenomena. (Of course, this is just an hypotesis, but I believe that its importance in the AI debate makes it a really compelling result).&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-03-28T14:53:11.307" LastActivityDate="2021-01-18T04:05:20.483" Title="Can AI research lead to new findings in general cognitive science?" Tags="&lt;philosophy&gt;&lt;cognitive-science&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11504" PostTypeId="1" CreationDate="2019-03-28T16:08:25.933" Score="5" ViewCount="169" Body="&lt;p&gt;I am very beginner to this world. I still learning the basics of Machine learning and AI but i have a problem at hand and i am not sure which technique or Algorithm can be applied on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on Click-Fraud detection in advertising. I need to predict fraud and learn new frauds with ML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dataset I have is the view and click logs from adserver(Service Provider). This data have some fields few of them are listed below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;auction_log_bid_id&quot;: null, &#xA;&quot;banner&quot;: 9407521, &#xA;&quot;browser&quot;: 0, &#xA;&quot;campaign&quot;: 2981976, &#xA;&quot;city&quot;: 94965, &#xA;&quot;clickword&quot;: null, &#xA;&quot;content_unit&quot;: 4335438, &#xA;&quot;country&quot;: 1, &#xA;&quot;external_profiledata&quot;: {}, &#xA;&quot;external_user_id&quot;: null, &#xA;&quot;flash_version&quot;: null, &#xA;&quot;id&quot;: 6665230893362053181, &#xA;&quot;ip_address&quot;: &quot;80.187.103.98&quot;, &#xA;&quot;is_ssl&quot;: true, &#xA;&quot;keyword&quot;: &quot;string&quot;&#xA;&quot;mobile_device&quot;: -1, &#xA;&quot;mobile_device_class&quot;: -1, &#xA;&quot;network&quot;: 268, &#xA;&quot;new_user_id&quot;: 6665230893362118717, &#xA;&quot;operating_system&quot;: 14, &#xA;&quot;profile_data&quot;: {}, &#xA;&quot;referrer&quot;: null, &#xA;&quot;screen_resolution&quot;: null, &#xA;&quot;server_id&quot;: 61, &#xA;&quot;state&quot;: 7, &#xA;&quot;target_url&quot;: &quot;string&quot;&#xA;&quot;timestamp&quot;: 1551870000, &#xA;&quot;type&quot;: &quot;CLICK_COMMAND&quot;, &#xA;&quot;user_agent&quot;: &quot;Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16D57&quot;, &#xA;&quot;user_id&quot;: null, &#xA;&quot;view_log_id&quot;: null&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are other fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to analyse these logs to find patterns for possible frauds but i am not sure where to start and which technique to use. e.g. Supervised, Unsupervised Semi-Supervised or Reinforcement Learning.&lt;/p&gt;&#xA;" OwnerUserId="23531" LastEditorUserId="23531" LastEditDate="2019-03-29T11:12:23.627" LastActivityDate="2021-04-17T09:59:26.007" Title="How to detect frauds in advertising business using machine learning?" Tags="&lt;machine-learning&gt;&lt;models&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11505" PostTypeId="2" ParentId="11483" CreationDate="2019-03-28T18:44:39.777" Score="3" Body="&lt;p&gt;I think that the normalisation factor is &lt;em&gt;assumed&lt;/em&gt; to be non-zero. So, in practice, I guess, you must eventually check that &lt;span class=&quot;math-container&quot;&gt;$P(z \mid b, a)$&lt;/span&gt; is non-zero (even though, I guess, it will likely never be zero because of round-off errors in computers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula to calculate &lt;span class=&quot;math-container&quot;&gt;$b'(s')$&lt;/span&gt; comes from its definition, which is based on Bayes' theorem, where the denominator is assumed to be non-zero (in general).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of &lt;span class=&quot;math-container&quot;&gt;$b'(s')$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid z, a, b)$&lt;/span&gt;, that is, the new belief &lt;span class=&quot;math-container&quot;&gt;$b'$&lt;/span&gt; of being in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is defined as the probability of landing in the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, given that we have observed &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, have taken action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from the previous state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and we had the previous belief &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. We will expand this definition, but first let us recall a few &lt;a href=&quot;https://github.com/nbro/probability_cheatsheet/blob/master/probability_cheatsheet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability definitions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that &lt;span class=&quot;math-container&quot;&gt;$P(A, B) = P(A \mid B) P(B) = P(B \mid A) P(A)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; can actually be multiple events (that is, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; could actually be the intersection of multiple events). In other words, suppose we want to calculate &lt;span class=&quot;math-container&quot;&gt;$P(A, B, C)$&lt;/span&gt;, we can actually consider e.g. &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; as one event. Let &lt;span class=&quot;math-container&quot;&gt;$(B \cap C) = (A, B) = D$&lt;/span&gt; (note that the notation &lt;span class=&quot;math-container&quot;&gt;$(A, B)$&lt;/span&gt; means the &quot;intersection&quot; of events &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, in the case &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are events). Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(A, B, C) &#xA;&amp;amp; = P(A, (B, C)) \\&#xA;&amp;amp;= P(A, D) \\&#xA;&amp;amp;= P(A|D)P(D) \\ &#xA;&amp;amp;= P(A|B, C)P(B, C) \\&#xA;&amp;amp;= P(A|B, C)P(B|C)P(C) \\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, this idea generalises to more variables/events.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that &lt;span class=&quot;math-container&quot;&gt;$\frac{P(A, B)}{P(B)} = P(A \mid B)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, we are prepared to expand &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid z, a, b)$&lt;/span&gt; and understand its expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can expand &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid z, a, b)$&lt;/span&gt; as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(s' \mid z, a, b)&#xA;&amp;amp;= \frac{P(s', z, a, b)}{P(z, a, b)}\\[0.7em]&#xA;&amp;amp;= \frac{P(s', z, a, b)}{P(z \mid b, a)P(a|b)P(b)}\\[0.7em]&#xA;&amp;amp;= \frac{P(z \mid s', a, b) P(s' \mid a, b) P(a | b) P(b)}{P(z \mid b, a)P(a|b)P(b)} \\[0.7em]&#xA;&amp;amp;= \frac{P(z \mid s', a, b) P(s' \mid a, b)}{P(z \mid b, a)}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It then turns out that (I will maybe explain this more in detail later) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(s' \mid z, a, b)&#xA;&amp;amp;= \frac{P(z \mid s', a, b) P(s' \mid a, b)}{P(z \mid b, a)} \\[0.7em]&#xA;&amp;amp;=\frac{O(a, s', z) \sum_{s\in S} b(s)T(s, a, s')}{\sum_{s \in S}b(s)\sum_{s' \in S} T(s, a, s')O(a, s', z)} \\[0.7em]&#xA;&amp;amp;=b'(s')&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, by assumption, &lt;span class=&quot;math-container&quot;&gt;$P(z \mid b, a) = \sum_{s \in S}b(s)\sum_{s' \in S} T(s, a, s')O(a, s', z)$&lt;/span&gt; must be different from zero for the equality above to hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see this from a very simply example of the Bayes' theorem. Let &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$&lt;/span&gt;. Now, intuitively, &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt; (which is what we want to calculate using Bayes' theorem) means the probability of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; occurring given that &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; has occurred, which means that &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; couldn't have had a probability of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; of happening if we wanted to calculate &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$P(B)$&lt;/span&gt; couldn't have been zero if we wanted to calculate &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt; using Bayes' theorem. We can also apply this reasoning to the definition of &lt;span class=&quot;math-container&quot;&gt;$b'(s')$&lt;/span&gt; above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, note also that, in the normalisation factor &lt;span class=&quot;math-container&quot;&gt;$$\sum_{s \in S}b(s)\sum_{s' \in S} T(s, a, s')O(a, s', z),$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$b(s)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$T(s, a, s')$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; are probability distributions, which means that not all terms of &lt;span class=&quot;math-container&quot;&gt;$b(s)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$T(s, a, s')$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; can be zero, for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (given they must sum up to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that &lt;span class=&quot;math-container&quot;&gt;$\sum_{s' \in S} T(s, a, s')O(a, s', z)$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Convex_combination&quot; rel=&quot;nofollow noreferrer&quot;&gt;convex combination&lt;/a&gt; of all &lt;span class=&quot;math-container&quot;&gt;$O(a, s', z)$&lt;/span&gt; (for all &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;), where the coefficients are &lt;span class=&quot;math-container&quot;&gt;$T(s, a, s')$&lt;/span&gt;. The normalisation factor is also a convex combination where the coefficients are &lt;span class=&quot;math-container&quot;&gt;$b(s)$&lt;/span&gt; (for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;). &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-28T21:11:41.690" LastActivityDate="2019-03-28T21:11:41.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11506" PostTypeId="2" ParentId="3389" CreationDate="2019-03-28T21:23:52.587" Score="2" Body="&lt;p&gt;I'm an undergraduate researcher at Prairie View A&amp;amp;M university. I just spent a few weeks tweaking an MLPRegressor model to predict the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th prime number. It recently stumbled into a super low minimum, where the first &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; extrapolations outside of the training data produced error less than &lt;span class=&quot;math-container&quot;&gt;$.02$&lt;/span&gt; percent. Even at &lt;span class=&quot;math-container&quot;&gt;$300000$&lt;/span&gt; primes out, it was about &lt;span class=&quot;math-container&quot;&gt;$.5$&lt;/span&gt; percent off. My model was simple: &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; hidden layers, trained on a single processor for less than 2 hours.&lt;/p&gt;&#xA;&lt;p&gt;To me, it begs the question, &amp;quot;Is there a reasonable function that produces the nth prime number?&amp;quot; Right now, the algorithms become computationally very taxing for extreme &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. Check out the time gaps between the most recent largest primes discovered. Some of them are years apart. I know it's been proven that if such a function exists, it will not be polynomial.&lt;/p&gt;&#xA;" OwnerUserId="23542" LastEditorUserId="2444" LastEditDate="2020-11-14T13:53:36.487" LastActivityDate="2020-11-14T13:53:36.487" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11507" PostTypeId="1" CreationDate="2019-03-28T21:45:10.530" Score="1" ViewCount="43" Body="&lt;p&gt;I am working on a supervised machine learning problem where I have more than 10 probably 50 or 100 predicting label categories. Which type of model can be used to work on this type of problem in anaconda python.&lt;/p&gt;&#xA;" OwnerUserId="23544" LastEditorUserId="23544" LastEditDate="2019-03-29T13:09:26.597" LastActivityDate="2019-03-29T20:36:41.443" Title="What types of machine learning model would fit?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11508" PostTypeId="1" CreationDate="2019-03-28T21:51:28.413" Score="0" ViewCount="39" Body="&lt;p&gt;I have an RC car with a camera, I have implemented so that i can detect lanes on my track (think like a nascar track). I want to get this car to be able to go around the track autonomous. But I am quite unsure what my next step should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either I can do an algorithm that detects so that I stay in the middle of the lane (if i get to close to either of the lines I steer towards the center). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or perhaps go around the track manually and save the coordinates of the detected lines as well as the actions I take (steering) and try a DQN approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to minimize my 'trial and error' time a little here. Perhaps there are some important steps in between here, or a solution that I have not thought of (i.e. am I missing something)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing this as a proof of concept therefore I can only spend maximum of a month on this, so what would you do here?&lt;/p&gt;&#xA;" OwnerUserId="22349" LastActivityDate="2019-03-28T21:51:28.413" Title="Next step after lane detection in vehicle automation" Tags="&lt;computer-vision&gt;&lt;dqn&gt;&lt;autonomous-vehicles&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11509" PostTypeId="2" ParentId="11478" CreationDate="2019-03-28T23:51:32.793" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/11494/2444&quot;&gt;This answer&lt;/a&gt; points at some of the more modern approaches. This has been around for a long time in the form of &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeAT: Neuroevolution of Augmenting Topologies&lt;/a&gt;, originally described in Kenneth Stanley's 2002 paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NeAT is available as a package for many languages, including &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/neat_overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python&lt;/a&gt;, &lt;a href=&quot;http://nn.cs.utexas.edu/?jneat&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java&lt;/a&gt;, and &lt;a href=&quot;http://nn.cs.utexas.edu/keyword?neat-c&quot; rel=&quot;nofollow noreferrer&quot;&gt;C++&lt;/a&gt;. The algorithm works as a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic programming&lt;/a&gt;. A population of networks is generated with simple, random, topologies. Then they are evaluated according to a loss function for a specific task. The poorly performing networks are discarded, and the better performing ones are intermixed to generate new variations. This process is iterated until the user wishes it to stop, and typically results in gradual improvement of average population performance against the loss function.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2019-07-28T19:57:16.013" LastActivityDate="2019-07-28T19:57:16.013" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11510" PostTypeId="1" AcceptedAnswerId="11531" CreationDate="2019-03-28T23:55:54.163" Score="1" ViewCount="66" Body="&lt;p&gt;This is my first post so please forgive me for any mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on an object detection algorithm that can detect abnormalities in an x-ray. As a prototype, I will be using yolov3 (more about yolo here: '&lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/darknet/yolo/&lt;/a&gt;')&#xA;However, one radiologist mentioned that in order to produce a good result you need to take into account the demographics of the patient.&#xA;In order to do that, my neural network must take into account both text an an image. Some suggestions have been made by other people for this question. For example, someone recommended taking the result of a convolution neural network and a seperate text neural network.&#xA;Here is an image for clarification:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ctYuj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ctYuj.png&quot; alt=&quot;Here is an image for clarification&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image Credits: This image (&lt;a href=&quot;https://cdn-images-1.medium.com/max/1600/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cdn-images-1.medium.com/max/1600/1&lt;/a&gt;*oiLg3C3-7Ocklg9_xubRRw.jpeg) from Christopher Bonnett's article (&lt;a href=&quot;https://blog.insightdatascience.com/classifying-e-commerce-products-based-on-images-and-text-14b3f98f899e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.insightdatascience.com/classifying-e-commerce-products-based-on-images-and-text-14b3f98f899e&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, please refer to above-mentioned article. It has explained how e-commerce products can be classified into various category hierarchies using both image and text data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a when convolution  neural network is mention it usssualy means it is used for classification instead of detection &#xA;&lt;a href=&quot;https://www.quora.com/What-is-the-difference-between-detection-and-classification-in-computer-vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/What-is-the-difference-between-detection-and-classification-in-computer-vision&lt;/a&gt; (Link for comparison between detection and classification)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, when I am using yolov3, how would it work. Would I be using yolov3 output vector which would be like this format &#xA;class, center_x, center_y, width and height&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is how would the overall structure of my neural network be like if I have both image and text as input while using yolov3.&#xA;Thank you for taking the time to read this.&lt;/p&gt;&#xA;" OwnerUserId="23546" LastActivityDate="2019-03-31T17:05:14.057" Title="Detecting abnormalities in x-rays while taking into account demographics of a patient -automated" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;automation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11511" PostTypeId="1" CreationDate="2019-03-29T00:44:19.540" Score="4" ViewCount="25" Body="&lt;p&gt;A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is difference than NCE Loss, which doesn't use a softmax at all, it uses a logistic binary classifier for the context/labels. In NLP, 'Negative Sampling' basically refers to the NCE-based approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More details here: &lt;a href=&quot;https://www.tensorflow.org/extras/candidate_sampling.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/extras/candidate_sampling.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested both and they both give pretty much the same results. But in word embedding literature, they always use NCE loss, and never sampled softmax.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any reason why this is? The sampled softmax seems like the more obvious solution to prevent applying a softmax to all the classes, so I imagine there must be some good reason for the NCE loss.&lt;/p&gt;&#xA;" OwnerUserId="18358" LastEditorUserId="2444" LastEditDate="2019-04-16T22:24:32.703" LastActivityDate="2019-04-16T22:24:32.703" Title="Why does all of NLP literature use noise contrastive estimation loss for negative sampling instead of sampled softmax loss?" Tags="&lt;natural-language-processing&gt;&lt;word2vec&gt;&lt;word-embedding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11513" PostTypeId="1" CreationDate="2019-03-29T04:25:36.700" Score="1" ViewCount="23" Body="&lt;p&gt;The model that we develop in artificial intelligence.What is its purpose,and what training data is relevant to it.&lt;/p&gt;&#xA;" OwnerUserId="23529" LastActivityDate="2019-05-25T07:41:00.003" Title="What is the goal of the model and is the training data relevant to that?" Tags="&lt;training&gt;&lt;generative-model&gt;&lt;goal-based&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11515" PostTypeId="1" AcceptedAnswerId="11516" CreationDate="2019-03-29T06:15:05.543" Score="2" ViewCount="1714" Body="&lt;p&gt;Since Keras API as defined as layers, how would it be used to implement the word2vec?&lt;/p&gt;&#xA;" OwnerUserId="23549" LastActivityDate="2019-11-11T02:50:59.070" Title="How to implement word2vec using Tensorflow 2.0 keras API?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;word2vec&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2020-08-10T17:02:18.780" ContentLicense="CC BY-SA 4.0" />
  <row Id="11516" PostTypeId="2" ParentId="11515" CreationDate="2019-03-29T06:38:41.773" Score="2" Body="&lt;p&gt;Your question is really broad.. But there are many manuals and tutorials which describe how to do it step by step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal favorite would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://realpython.com/python-keras-text-classification/#using-pretrained-word-embeddings&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Text Classification With Python and Keras&lt;/a&gt; - It has a very detailed explanation of every step of the implementation while remaining practical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also try:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://adventuresinmachinelearning.com/word2vec-keras-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Word2Vec Keras tutorial&lt;/a&gt; - This one focuses more on understanding how word2vec embedding works, but it also contains an implementation. &lt;/p&gt;&#xA;" OwnerUserId="20399" LastActivityDate="2019-03-29T06:38:41.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11517" PostTypeId="1" AcceptedAnswerId="11529" CreationDate="2019-03-29T07:12:37.200" Score="4" ViewCount="169" Body="&lt;p&gt;According to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Physical_symbol_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page of the physical symbol system hypothesis (PSSH)&lt;/a&gt;, this hypothesis seems to be a vividly debated topic in philosophy of AI. But, since it's about formal systems, shouldn't it be already disproven by Gödel's theorem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question arises specifically because the PSSH was elaborated in the 1950s, while Gödel came much earlier, so at the time the Incompleteness theorems were already known; in which way does the PSSH deal with this fact? How does it &quot;escape&quot; the theorem? Or, in other words, how can it try to explain intelligence given the deep limitations of such formal systems?&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2020-05-13T10:39:25.713" LastActivityDate="2020-12-11T11:30:41.493" Title="Shouldn't Gödel's incompleteness theorems disprove the physical symbol system hypothesis?" Tags="&lt;philosophy&gt;&lt;symbolic-ai&gt;&lt;theory-of-computation&gt;&lt;incompleteness-theorems&gt;&lt;physical-symbol-system-hypothesis&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11518" PostTypeId="2" ParentId="10090" CreationDate="2019-03-29T07:32:20.547" Score="3" Body="&lt;p&gt;This is not an answer. I couldn't comment, so here are some remarks about your question: This is a very broad question, and considered The Holy Grail for building artificially intelligent systems - meaning that &lt;a href=&quot;http://people.idsia.ch/~juergen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;some scientists&lt;/a&gt; have been dreaming about this since time immemorial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some homework is warranted from your side; you could have offered some of your solutions or maybe identified the multiple layers to your query, as they invoke concepts from many fields within the study of AI (or AGI to be general).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the following layers are taken up around the big-words in the question, though rhetorically - &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;On evolution&lt;/em&gt; - what registers as evolution (is only genetic mutation evolution or should it involve some form of natural selection or seeking a niche for increasing the chances of survival or etc). How would evolution of software look like? Should it be able to modify its own code in the process of evolution? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;On concept&lt;/em&gt; - &lt;a href=&quot;https://plato.stanford.edu/entries/concepts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;what constitutes a concept&lt;/a&gt;, identifying from the environment a concept by calculating its relevance (w.r.t other concepts), coming with a process of selecting the concept for use in a given environment (natural or artificial).Referring to the example in the question, is the road a &lt;em&gt;relevant concept&lt;/em&gt; or the trees and the sky and the bees pollinating some flowers along the roadside? What is a &lt;em&gt;more fundamental concept&lt;/em&gt; - the trees or the bees, and how does one measure that?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;On rewards&lt;/em&gt; - for us humans, rewards are maximization of survival of genetic machinery they carry (translating to reproductive success). what reward system should we come up that machines could use to increase their chances of survival in the physical world? what value should be put on a reward for moving the car on a straight line, or moving from darkness towards light? Shouldn't moving from darkness to light be more fundamental action (concept) and therefore rewarded higher than moving along the straight line? However, given that the car has learned to move from darkness to light, shouldn't the value of the reward be lowered so that it can learn other actions/concepts?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As can be seen from this brief detailing, there are layers within layers. Therefore, it is proper to establish now that there are no simple answer to this essay-requiring soul-searching time-eating heavy-research-wanting question. However appropriate directions can be given about work that is being done by some of the most prominent scientists of our times. The &lt;em&gt;thing&lt;/em&gt; that one is looking for are known as &lt;strong&gt;Universal Problems Solvers&lt;/strong&gt;, such as &lt;a href=&quot;http://people.idsia.ch/~juergen/goedelmachine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gödel&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine&lt;/a&gt; (by &lt;a href=&quot;http://people.idsia.ch/~juergen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jürgen Schmidhuber&lt;/a&gt;) and &lt;a href=&quot;https://wiki.lesswrong.com/wiki/AIXI&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIXI&lt;/a&gt; - &lt;a href=&quot;http://hutter1.net/ai/uaibook.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence (AI) based on Solomonoff's distribution ξ&lt;/a&gt; (by &lt;a href=&quot;http://hutter1.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markus Hutter&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a quote lifted from &lt;a href=&quot;https://en.wikipedia.org/wiki/AIXI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page on AIXI&lt;/a&gt; that is pretty self-explanatory on how it maximizes the rewards over time.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AIXI is a reinforcement learning agent. It maximizes the expected&#xA;  total rewards received from the environment. Intuitively, it&#xA;  simultaneously considers every computable hypothesis (or environment).&#xA;  In each time step, it looks at every possible program and evaluates&#xA;  how many rewards that program generates depending on the next action&#xA;  taken. The promised rewards are then weighted by the subjective belief&#xA;  that this program constitutes the true environment. This belief is&#xA;  computed from the length of the program: longer programs are&#xA;  considered less likely, in line with Occam's razor. AIXI then selects&#xA;  the action that has the highest expected total reward in the weighted&#xA;  sum of all these programs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Gödel Machine goes further - it allows the agent to modify its own code that allows it to maximise the rewards of its actions - which is to modify its own code that allows it to maximize the rewards of its actions - as so on. This is kind of a recursive definition, that simulates evolution (a rapid one) by choosing to evolve that code/state of the agent that converges towards a code that is superior to the code that the agent is currently running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a quote lifted from &lt;a href=&quot;http://people.idsia.ch/~juergen/gmsummary.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;page on summary of Gödel machine&lt;/a&gt;. See that Hutter is also referenced (discovered of AIXI above). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Our Gödel machine will never get worse than its initial problem&#xA;  solving strategy, and has a chance of getting much better, provided&#xA;  the nature of the given problem allows for a provably useful rewrite&#xA;  of the initial strategy, or of the proof searcher. The Gödel machine&#xA;  may be viewed as a &lt;em&gt;self-referential universal problem solver that can&#xA;  formally talk about itself, in particular about its performance. It&#xA;  may &quot;step outside of itself&quot; (Hofstadter, 1979) by rewriting its&#xA;  axioms and utility function or augmenting its hardware, provided this&#xA;  is provably useful&lt;/em&gt;. Its conceptual simplicity notwithstanding, the&#xA;  Gödel machine explicitly addresses the `Grand Problem of Artificial&#xA;  Intelligence' by optimally dealing with limited resources in general&#xA;  environments, and with the possibly huge (but constant) slowdowns&#xA;  buried by previous approaches (Hutter, 2001, 2002) in the widely used&#xA;  but sometimes misleading O()-notation of theoretical computer science.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The main limitation of the Gödel machine is that it cannot profit from&#xA;  self-improvements whose usefulness it cannot prove in time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/cs/0004001.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reading&lt;/a&gt; &lt;a href=&quot;ftp://ftp.idsia.ch/pub/juergen/gm6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the papers&lt;/a&gt; of the stated research would certainly be useful in finding answers to some of the layers in the question. There is also good work going on in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI community&lt;/a&gt; that somewhat aligns with the direction. Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="5750" LastActivityDate="2019-03-29T07:32:20.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11519" PostTypeId="1" AcceptedAnswerId="11530" CreationDate="2019-03-29T08:30:21.667" Score="0" ViewCount="702" Body="&lt;p&gt;How could we solve the TSP using a hill-climbing approach?&lt;/p&gt;&#xA;" OwnerUserId="19448" LastEditorUserId="2444" LastEditDate="2020-11-23T21:32:51.353" LastActivityDate="2020-11-23T21:32:51.353" Title="How could we solve the TSP using a hill-climbing approach?" Tags="&lt;applications&gt;&lt;optimization&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;travelling-salesman-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11520" PostTypeId="1" CreationDate="2019-03-29T08:44:52.503" Score="1" ViewCount="19" Body="&lt;p&gt;I’ve created a variational autoencoder to encode 1-dimensional arrays. The encoding is done through 3 1d-convolutional layers. Then, after the sampling trick, I reconstruct the series using 3 fully connected layers. Here are my questions in case some can shed some light on it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it would be better if I use 1d-deconvolutional layers instead of fully connected, but I cannot understand precisely why.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It’s because it would bring better results? But then, if the FC layer is complex enough should be able to archive the same results, right?&lt;/li&gt;&#xA;&lt;li&gt;It’s because would be more efficient? This is, would get the same results as a complex enough FC layer but with less training and parameters?&lt;/li&gt;&#xA;&lt;li&gt;Or it’s because of other reasons that I’m missing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="22066" LastActivityDate="2019-03-29T08:44:52.503" Title="One dimension deconvolutions or fully connected layers?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11522" PostTypeId="2" ParentId="11500" CreationDate="2019-03-29T09:01:31.190" Score="1" Body="&lt;p&gt;This is about &lt;em&gt;hard AI&lt;/em&gt; and &lt;em&gt;soft AI&lt;/em&gt;: proponents of hard AI work on systems that simulate the way human cognition works, with the eventual (hypothetical) goal of replicating it. This presupposes that you know how cognition works, and presumably you will learn about it as you attempt to replicate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Soft AI, on the other hand, tries to emulate the outcomes only. For example, Weizenbaum's &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;ELIZA&lt;/em&gt;&lt;/a&gt; is clearly on this side, as it uses simple pattern matching, and does not 'understand' anything about the conversations it is having.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, we don't even know fully what it means to 'understand' something, and building working systems is not really possible with a hard approach. Hence, soft AI is more common, as researchers are usually measured by their outcomes rather than their ideas. As far as I am aware, the hard AI approach has been all but abandoned long ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As current AI seems to be dominated by statistical approaches, I doubt that we can find out many useful things about cognition this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting side-note: it seems to me that the capabilities of modern AI systems have developed away from human capabilities. A three-year-old can do some things that a sophisticated AI system cannot do, but in some areas (chess, translation, ...) the capabilities of AI systems surpass what humans are capable of. Maybe imitation is indeed not the right way to approach AI.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-03-29T09:01:31.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11523" PostTypeId="1" CreationDate="2019-03-29T10:43:00.937" Score="5" ViewCount="154" Body="&lt;p&gt;As a piece of computer code, it's tempting to think that artificially intelligent systems can't have feelings. So, can an artificial intelligence suffer?&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="2444" LastEditDate="2019-03-31T12:43:25.360" LastActivityDate="2019-03-31T12:43:25.360" Title="Can an artificial intelligence suffer?" Tags="&lt;philosophy&gt;&lt;emotional-intelligence&gt;&lt;artificial-life&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11524" PostTypeId="2" ParentId="11523" CreationDate="2019-03-29T11:17:57.497" Score="4" Body="&lt;p&gt;AFAIK, pain is produced by the nervous system (after the realease of chemicals), so no AI suffers. AI does not (yet) possess a life (and I don't think this will ever happen).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A definition of &lt;em&gt;life&lt;/em&gt; from the dictionary:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the condition that distinguishes animals and plants from inorganic matter, including the capacity for growth, reproduction, functional activity, and continual change preceding death.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The current AI definitely does not possess a life according to this definition. AI is inorganic (like a rock): it is based on silicon (not carbon atoms).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-29T11:24:46.650" LastActivityDate="2019-03-29T11:24:46.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11525" PostTypeId="2" ParentId="3176" CreationDate="2019-03-29T11:57:41.187" Score="1" Body="&lt;p&gt;There are several expressions that are &lt;em&gt;often&lt;/em&gt; used as synonyms for &lt;em&gt;artificial intelligence&lt;/em&gt;, but, nowadays, the most common ones are likely &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;machine intelligence&lt;/em&gt;&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;computational intelligence&lt;/em&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, these expressions are not well defined, so not everyone will agree that they are interchangeable, but we can all agree that these fields (either if we consider them the same or not) are quite related to each other (and they overlap).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, these fields also evolve over time and they embrace techniques from other fields, which makes it more difficult to define them. More concretely, initially, AI was mainly based on the manipulation of symbols and logic, but nowadays AI is mainly &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistics&lt;/a&gt; and, in particular, &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep learning&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the expression &lt;em&gt;artificial intelligence&lt;/em&gt; was apparently coined after the term &lt;a href=&quot;https://en.wikipedia.org/wiki/Cybernetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;cybernetics&lt;/em&gt;&lt;/a&gt;, which some people might consider the first serious attempt to building &lt;em&gt;intelligent&lt;/em&gt; systems.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-23T12:35:21.547" LastActivityDate="2019-06-23T12:35:21.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11526" PostTypeId="2" ParentId="11517" CreationDate="2019-03-29T13:12:09.267" Score="0" Body="&lt;p&gt;I think your conceptualization of this is a bit off. All PSSH states is &quot;A physical symbol system has the necessary and sufficient means for general intelligent action.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gödel's theorems state 2 basic things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Any sufficiently powerful formal system cannot prove its own consistency.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are theorems in any sufficiently powerful formal system that cannot be proved within the system. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;PSSH doesn't have too much to do with Gödel. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-03-29T13:12:09.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11527" PostTypeId="2" ParentId="11504" CreationDate="2019-03-29T13:16:03.140" Score="0" Body="&lt;p&gt;There are a couple of different ways you can go about this depending on what kind of data you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have labels or can separate the normal data from the fraudulent, you can perform either binary classification, or likely more usefully, anomaly detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Anomaly Detection(which is typically now done via autoencoder) you train your model on the normal data, so it learns a compressed representation of that 'signal', from there it will be able to detect any sample that does not fit the learned representation(in theory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to a tutorial in keras: &lt;a href=&quot;https://medium.com/@curiousily/credit-card-fraud-detection-using-autoencoders-in-keras-tensorflow-for-hackers-part-vii-20e0c85301bd&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-03-29T13:16:03.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11528" PostTypeId="1" CreationDate="2019-03-29T13:27:52.300" Score="1" ViewCount="88" Body="&lt;p&gt;Where or for what could genetic algorithms (GA) be used in the context of project management (PM)? I thought about task dispatching, but I'm looking for other potential uses of GAs in the context of PM.&lt;/p&gt;&#xA;" OwnerUserId="23557" LastEditorUserId="2444" LastEditDate="2021-01-06T22:11:17.863" LastActivityDate="2021-02-05T23:04:35.473" Title="Where or for what could genetic algorithms be used in the context of project management?" Tags="&lt;genetic-algorithms&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11529" PostTypeId="2" ParentId="11517" CreationDate="2019-03-29T13:28:25.387" Score="3" Body="&lt;p&gt;The PSSH is often attacked via either Godel's theorems or Turing's incomputability theorem.&lt;/p&gt;&#xA;&lt;p&gt;However, both attacks have an implicit assumption: that to be intelligent is to be able to decide undecidable questions. It's really not clear that this is so.&lt;/p&gt;&#xA;&lt;p&gt;Consider what Godel's theorems say, in essence:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&amp;quot;powerful&amp;quot; formal systems cannot prove, using only techniques from within the system, that they are self-consistent.&lt;/li&gt;&#xA;&lt;li&gt;There are statements that are true that cannot be proven within a given &amp;quot;powerful&amp;quot; formal system.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Suppose that we allow both of those facts. The missing step in the argument is the following statements:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You need to be able to prove the consistency of your own reasoning system to be considered intelligent.&lt;/li&gt;&#xA;&lt;li&gt;You need to be able to correct reason out a proof of all true statements to be considered intelligent.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The main problem is, under this definition, humans are probably not considered intelligent! I certainly have no way to prove that my reasoning is sound and self-consistent. Moreover, it is objectively not so! I frequently believe contradictory things at the same time.&lt;/p&gt;&#xA;&lt;p&gt;I also am not able to reason out proofs of all the statements that appear to be true, and it seems entirely plausible that I cannot do so because of the inherent limitations of the logical systems I'm reasoning with.&lt;/p&gt;&#xA;&lt;p&gt;This is a contradiction. The overall argument was that one of these 4 statements is false:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Godel's theorems say symbol systems lack some important properties.&lt;/li&gt;&#xA;&lt;li&gt;Intelligent things have the properties that Godel says symbol systems lack.&lt;/li&gt;&#xA;&lt;li&gt;Humans are intelligent.&lt;/li&gt;&#xA;&lt;li&gt;Humans can't do the things Godel says symbol systems can't do.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Some authors (like &lt;a href=&quot;https://en.wikipedia.org/wiki/John_Searle&quot; rel=&quot;nofollow noreferrer&quot;&gt;John Searle&lt;/a&gt;) might argue the false premise is 4. Most modern AI researchers would argue that the false premise is 2. Since intelligence is a bit nebulous, which view is correct may rely on metaphysical assumptions, but most people agree on premises 1 &amp;amp; 3.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-12-11T11:30:41.493" LastActivityDate="2020-12-11T11:30:41.493" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11530" PostTypeId="2" ParentId="11519" CreationDate="2019-03-29T15:28:17.930" Score="1" Body="&lt;p&gt;I will give you a basic idea of an approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea behind hill climbing algorithms is to find local neighbouring solutions to the current one and, eventually, replace the current one with one of these neighbouring solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you first need to model your problem in a way such that you can find neighbouring solutions to the current solution (as efficiently as possible). But what is a solution to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;TSP&lt;/a&gt;? It is a sequence of nodes (or vertices), such that the first node is equal to the last one (given that the travelling salesman needs to return to its initial position), no other vertex is repeated, and all vertices of the graph are included. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, given a sequence of nodes &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, \dots, x_{n}, x_1$&lt;/span&gt;, how can you create a neighbouring solution that is valid, that is, there is not repeated vertex (apart from the initial and the last one) and all vertices are included? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: recall that, in the case of TSP, we can assume that every node (or city) is connected to every other node.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-29T15:33:52.450" LastActivityDate="2019-03-29T15:33:52.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11531" PostTypeId="2" ParentId="11510" CreationDate="2019-03-29T17:16:04.607" Score="0" Body="&lt;p&gt;First of all you don't need &quot;text&quot; input as in Christopher Bonnett's blog. Your case is more easy - demographic is table data, which can be expressed as vector of numeric values. This data should be processed - pushed through one or two fully connected layers. The trick is where, to what part of yolo to concatenate results of processing of this vector. Because it's vector data it should concatenated to fully connected layer. Where exactly should be found by experiments, but as starting point it could be concatenated to before-last (before output) fully connected layer (I think for yolo it's 4096-size layer). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall I'd say that is not a trivial task. It require some experience with deep learning, good understanding of yolo design, yolo algorithm and a lot of experimentation, both with architecture and hyperparameters. Probably worth solid paper. Good luck.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-03-29T17:16:04.607" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11532" PostTypeId="2" ParentId="11528" CreationDate="2019-03-29T19:46:28.180" Score="-1" Body="&lt;p&gt;Project Management is most like model to game theory. You can find a &lt;a href=&quot;https://pdfs.semanticscholar.org/3b96/427b507c429c9db96fdb3de8bc5edfe190ca.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;article here&lt;/a&gt;, as a cooperative game, and apply &lt;a href=&quot;https://en.wikipedia.org/wiki/Opportunity_cost&quot; rel=&quot;nofollow noreferrer&quot;&gt;opportunity cost&lt;/a&gt; to your budget, only you need to apply your own rules of the game.&lt;/p&gt;&#xA;" OwnerUserId="10983" LastEditorUserId="2444" LastEditDate="2021-01-06T21:47:24.433" LastActivityDate="2021-01-06T21:47:24.433" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11533" PostTypeId="2" ParentId="11507" CreationDate="2019-03-29T20:36:41.443" Score="1" Body="&lt;p&gt;Welcome to AI.SE @Par!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you have might be either a &lt;em&gt;multi-label&lt;/em&gt; or a &lt;em&gt;multi-class&lt;/em&gt; classification problem. If the classes are disjoint (each example belongs to just 1 of the 50 classes), it's a multi-class problem. If not (so each example can belong to several classes at once), it's a multilabel problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-label classification is usually handled by training a separate model for each of the labels. If you want to label a new point, you then ask each model what it thinks, and assign the union of the labels that the models suggest together. An alternative approach is to use something like a neural network, which can have many outputs. You can then have one output neuron for each possible label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-class classification &lt;em&gt;can&lt;/em&gt; be addressed by using the multilabel techniques, but this is usually not a good idea. The three main approaches that are used are &quot;one-v-one&quot;, &quot;one-v-all&quot;, and &quot;many-v-many&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In 1-v-1, we train one model to discriminate each &lt;em&gt;pair&lt;/em&gt; of classes (n(n+1)/2 models in total for n classes). To classify a new point, we ask each model which class it belongs to and assign a &quot;vote&quot; to the class the model returns. The class with the most votes overall is selected as the label for the new point. In case of a tie, we can report several possible answers to the user.&lt;/li&gt;&#xA;&lt;li&gt;In 1-v-all, we train one model to discriminate each class from &lt;em&gt;all the other classes&lt;/em&gt; (n models in total for n classes). To classify a new point, we ask each model whether it belongs to the model's primary class or not. Ideally, just one model claims the new point. If more than one does, we can report a tie, or use some notion of classifier confidence to select a winner.&lt;/li&gt;&#xA;&lt;li&gt;In many-v-many, we train &lt;em&gt;k&lt;/em&gt; models each of which is tasked with discriminating some subset of the classes from all the rest. To classify a new point, we ask all of these models to label the point. We then pick the class that that is most consistent with the results of all the models. This can also be done by deliberately constructing the models to form an &lt;a href=&quot;https://en.wikipedia.org/wiki/Error_correction_code&quot; rel=&quot;nofollow noreferrer&quot;&gt;error correcting code&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So which approach should you use? Well, Rifkin &amp;amp; Klautau's 2004 &lt;a href=&quot;http://www.jmlr.org/papers/volume5/rifkin04a/rifkin04a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;JMLR paper&lt;/a&gt; argues convincingly that the answer is to use one-versus-all classification. This is also pretty easy to do in most packages. For example, in Python's ScikitLearn, you can do it with &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OneVsRest&lt;/a&gt;. If you're not sure what to try, that's probably a safe bet.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-03-29T20:36:41.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11534" PostTypeId="1" CreationDate="2019-03-29T23:50:12.453" Score="1" ViewCount="75" Body="&lt;p&gt;&lt;em&gt;&lt;a href=&quot;https://en.wiktionary.org/wiki/super#Etymology_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Super&lt;/a&gt;&lt;/em&gt; comes from the Latin and means &amp;quot;above&amp;quot;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;University of Oxford philosopher Nick Bostrom defines superintelligence as &amp;quot;any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest&amp;quot;. &lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;(wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Bostrom's definition could be taken to imply this is a quantitative measure of degrees as a numeric relationship. (Under this definition, we have achieved narrow superintelligence, reduced to competency in a single task.)&lt;/p&gt;&#xA;&lt;p&gt;Gibson, famously, sheds light on an another aspect via &lt;a href=&quot;https://williamgibson.fandom.com/wiki/Wintermute&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wintermute&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://williamgibson.fandom.com/wiki/Neuromancer_(AI)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neuromancer&lt;/a&gt;, where, once superintelligence is achieved, the AI just f-'s off and does it's own thing, motivations beyond human comprehensions.  (Essentially, &amp;quot;next-level&amp;quot; thinking.) The second measure is discrete and ordinal.&lt;/p&gt;&#xA;&lt;p&gt;Is superintelligence a function of strength or a category?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-22T01:38:18.877" LastActivityDate="2021-01-22T01:38:18.877" Title="Is superintelligence a function of strength or a category?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;agi&gt;&lt;superintelligence&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11535" PostTypeId="1" AcceptedAnswerId="11549" CreationDate="2019-03-29T23:57:01.980" Score="3" ViewCount="119" Body="&lt;p&gt;I'm working on understanding VAEs, mostly through video lectures of Stanford cs231n, in particular &lt;a href=&quot;https://youtu.be/5WoItGTWV54?t=2543&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture 13&lt;/a&gt; tackles on this topic and I think I have a good theoretical grasp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when looking at actual code of implementations, such as &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/examples/variational_autoencoder.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this code&lt;/a&gt;  from &lt;a href=&quot;https://blog.keras.io/building-autoencoders-in-keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; of VAEs I see some differences which I can't quite understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please take a look at this VAE &lt;a href=&quot;https://imgur.com/a/xK75jxL&quot; rel=&quot;nofollow noreferrer&quot;&gt;architecture visualization&lt;/a&gt; from the class, specifically the decoder part. From the way it is presented here I understand that the decoder network outputs mean and covariance for the data &lt;em&gt;distribution&lt;/em&gt;. To get an actual output (i.e. image) we need to sample from the distribution that is parametrized by mean and covariance - the outputs of the decoder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you look at the code from the Keras blog VAE implementation, you will see that there is no such thing. A decoder takes in a sample from latent space and &lt;strong&gt;directly&lt;/strong&gt; maps its input (sampled z) to an output (e.g. image), not to parameters of a distribution from which an output is to be sampled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something or does this implementation not correspond to the one presented in the lecture? I've been trying to make sense of it for quite some time now but still can't seem to understand the discrepancy. &lt;/p&gt;&#xA;" OwnerUserId="21278" LastEditorUserId="2444" LastEditDate="2019-03-31T18:44:58.253" LastActivityDate="2019-04-02T09:10:57.480" Title="Do we also need to model a probability distribution for the decoder of a VAE?" Tags="&lt;computer-vision&gt;&lt;generative-model&gt;&lt;autoencoders&gt;&lt;latent-variable&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11536" PostTypeId="2" ParentId="9834" CreationDate="2019-03-30T00:56:55.470" Score="2" Body="&lt;p&gt;IMO, the greatest &quot;strength&quot; of HTM is that it is modeled after the human neocortex, which is the most intelligent thing we know of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to understand the importance of this simple idea one must contrast it with the most familiar form of AI - Neural Networks (NNs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Traditional Neural Network AI has been under development for a long time and has many more people working on it than HTM. NNs are capable of performing a bewildering number of tasks, and the list of its accomplishments grows with every passing day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, NNs are &lt;em&gt;not thinking&lt;/em&gt;. They perform their magic only after being trained on (typically) massive amounts of training data. Training a NN is essentially an advanced form of curve-fitting. If your training data encompasses closely enough what it encounters in new data then it will likely perform very well. However, if it encounters something new (which is sometimes difficult to know beforehand) then it can fail abysmally, and often in a way that humans would never fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example I heard about was on a NN trained on millions of images that could briefly describe what was in new images it had never seen before. It performed fabulously - something like 95-97% accuracy. However, when it was shown an image of a baby holding a toothbrush, it said, &quot;A boy holding a baseball bat.&quot; This is not a human-like error. Humans know the difference between a boy and a baby, and a bat and a toothbrush. This is just an example, but it reveals a fundamental problem of NNs - they are not thinking. Useful? Yes. Thinking? No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back to HTM. HTM is new and currently has only a handful of researchers working on it. It is &quot;better&quot; than NNs in only a small number of cases - it has a long way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if by &quot;strengths&quot; you're thinking about what tasks can currently be done better with HTM than with NNs, then most people should still chose NNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if by &quot;strengths&quot; you're thinking about what has the best chance of achieving general intelligence someday, then I would say hands-down it is HTM.&lt;/p&gt;&#xA;" OwnerUserId="5927" LastActivityDate="2019-03-30T00:56:55.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11537" PostTypeId="2" ParentId="11523" CreationDate="2019-03-30T02:11:59.073" Score="0" Body="&lt;p&gt;I like nbro's answer. I think another way to look at it is that we have an AI/ML algorithm where we provide some sort of policy or function that we want for it to minimize (or maximize - just multiply by -1). Based off of the AI's objective, if it gathers/collects information from it's environment that it might not find &quot;satisfactory&quot; to it's mission, it might &quot;suffer&quot; -&gt; but really it is not being able to accomplish it's goals. BTW, this is all very &quot;hand-wavy&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a very over simplistic analogy to our case. From a philosophical perspective, if I am a hedonistic organism whose only goal is to seek &quot;happiness&quot; or maximize the &quot;happiness function&quot; -&gt; physical/mental/etc. pain would not allow me to satisfy my objective/goal of being happy -&gt; perhaps I might &quot;suffer&quot;. Again, I am deliberately framing this and also making many assumptions like a human's goal is to maximize their happiness -&gt; this may or may not be the case depending on your personal views. &lt;/p&gt;&#xA;" OwnerUserId="15428" LastActivityDate="2019-03-30T02:11:59.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11538" PostTypeId="2" ParentId="11523" CreationDate="2019-03-30T03:24:34.823" Score="0" Body="&lt;p&gt;At subatomic levels, there might be millions of short love and hate stories going on every nanosecond. Tragedies and absolute massacres, where atoms, which have been together for centuries are parted never to meet again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But so long as, there is no &quot;observer&quot; who can coherently observe all these, the concept of pain can't get fully realized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This AI would need to be &quot;alive&quot; and comprehend the events around it. It may or may not have an objective (function) but it'll need to go through pain, distress or hardship.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A intelligent device protecting itself against virus attack, using antivirus, might have to struggle, but the comprehension of either impending doom or constant battle perhaps doesn't dawn on it, so even though it suffers, it does not grasp it. Even if it does grasp it, it's not able to communicate that distress to us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we code the computer to shout for help and cry when a virus attacks, and it does so when virus attacks, then we would just &quot;address&quot; the issue and won't believe it's sadness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it'll be really tough for AI to convince us that it's suffering. Hence I'll turn a blind eye to it. It doesn't suffer. &lt;/p&gt;&#xA;" OwnerUserId="23481" LastActivityDate="2019-03-30T03:24:34.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11539" PostTypeId="1" CreationDate="2019-03-30T05:31:04.133" Score="7" ViewCount="565" Body="&lt;p&gt;Often times I see the term deep reinforcement learning to refer to RL algorithms that use neural networks, regardless of whether or not the networks are deep.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO&lt;/a&gt; is often considered a deep RL algorithm, but using a deep network is not really part of the algorithm. In fact, the example they report in the paper says that they used a network with only 2 layers.&lt;/p&gt;&#xA;&lt;p&gt;This SIGGRAPH project (&lt;a href=&quot;https://xbpeng.github.io/projects/DeepMimic/2018_TOG_DeepMimic.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills&lt;/a&gt;)  has the name &lt;em&gt;deep&lt;/em&gt; in it and the title even says 'deep reinforcement learning', but if you read the paper, you'll see that their network uses only 2 layers.&lt;/p&gt;&#xA;&lt;p&gt;Again, the paper &lt;a href=&quot;https://arxiv.org/pdf/1812.11103.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Walk via Deep Reinforcement Learning&lt;/a&gt; by researchers from Google and Berkeley, contains &lt;em&gt;deep RL&lt;/em&gt; in the title, but if you read the paper, you'll see they used 2 hidden layers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cc.gatech.edu/%7Eaclegg3/projects/learning-dress-synthesizing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Another SIGGRAPH project with deep RL in the title&lt;/a&gt;. And, if you read it, surprise, 2 hidden layers.&lt;/p&gt;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1801.01290.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor&lt;/a&gt;, if you read table 1 with the hyperparameters, they also used 2 hidden layers.&lt;/p&gt;&#xA;&lt;p&gt;Is it standard to just call deep RL to any RL algorithm that uses a neural net?&lt;/p&gt;&#xA;" OwnerUserId="17312" LastEditorUserId="2444" LastEditDate="2020-07-01T13:05:48.777" LastActivityDate="2020-07-01T13:07:52.930" Title="Is reinforcement learning using shallow neural networks still deep reinforcement learning?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11542" PostTypeId="1" CreationDate="2019-03-30T08:57:53.047" Score="1" ViewCount="175" Body="&lt;p&gt;If the AI goal is to serve humans and protect them (if this ever happens) and AI someday realizes that humans destroy themselves, will it try to control people for their own good, that is, will it control man's will to not destroy himself?&lt;/p&gt;&#xA;" OwnerUserId="23569" LastEditorUserId="2444" LastEditDate="2019-03-30T11:23:54.263" LastActivityDate="2020-01-02T21:45:06.870" Title="If the AI goal is the protection of humans, will it always pursue this goal?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;ethics&gt;&lt;ai-safety&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11543" PostTypeId="1" CreationDate="2019-03-30T10:23:56.050" Score="0" ViewCount="29" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;TL;DR below.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You entered a online mini video game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a fast-paced hypercasual game. But it's a ranked one - top 10 get prizes. The leaderboard resets and prizes are handed out every couple of days. But it's too time-consuming and brain-exhausting to game to the top spots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So... you decided to write an AI. At first, the prototype failed to start at all due to runtime bugs. With some coding, it went from nowhere to a snail-crawling speed but stil scoring nothing at all. With more experiment, it finally got a few hundred score. You did some research and optimized the crap out of it then ran it again, it scored a whopping 3500 points, leaving behind the second place just several hundreds of points. Hours later, you logged in again and found yourself overwhelmed in joy that you just won the biggest prize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ding! You got an idea: &quot;Why not just wait until just several hours before the leaderboard closes then I run the AI?&quot;. Meanwhile, you did some research on the game: some's been banned due to reaching an unbelievably high scores (ten-thousands points plus) messaging to the mod claiming they did it all by themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fast-forward, it's time! Closing all the browser tabs and your code editor hoping to score higher, you realized your AI may have reached &lt;a href=&quot;https://en.wikipedia.org/wiki/Progress_in_artificial_intelligence#Current_performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;superhuman level&lt;/a&gt; when it scored 5000 points, almost 2 times that of the second place, twice, before you had to shut it down afraid of being banned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few minutes after sharing the screenshot of your highscore with your friends, with them telling you &quot;You're gunna be banned for sure, man!&quot;, you checked in again and... it happened, you've been banned! In panic, you asked yourself: &quot;what I've done wrong?&quot;. Topping the leaderboard in just 2 runs? Or twice in a row?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You entered the game under another name, this time developing a play strategy to not get banned. What would you do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Playing a hypercasual game&lt;/li&gt;&#xA;&lt;li&gt;Want to top the leaderboard for prizes&lt;/li&gt;&#xA;&lt;li&gt;Leaderboard resets every couple of days&lt;/li&gt;&#xA;&lt;li&gt;Wrote a AI to do that&lt;/li&gt;&#xA;&lt;li&gt;AI slowly got better&lt;/li&gt;&#xA;&lt;li&gt;Won once leaving the second place a few hundred points&lt;/li&gt;&#xA;&lt;li&gt;Found out other players who got banned of an unbelievably highscore&lt;/li&gt;&#xA;&lt;li&gt;AI reached &lt;a href=&quot;https://en.wikipedia.org/wiki/Progress_in_artificial_intelligence#Current_performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;superhuman level&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Ran AI just a few hours before the leaderboard closes&lt;/li&gt;&#xA;&lt;li&gt;Scored almost twice the points of second place&lt;/li&gt;&#xA;&lt;li&gt;Got banned&lt;/li&gt;&#xA;&lt;li&gt;Enter the game again with better scheme&lt;/li&gt;&#xA;&lt;li&gt;What scheme?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Suggestions/What I'm doing&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Climb up the leaderboard over a couple-of-day period&lt;/li&gt;&#xA;&lt;li&gt;Stop the AI around a manually given score range randomly at some point&lt;/li&gt;&#xA;&lt;li&gt;Run only few times spaced out every few hours&lt;/li&gt;&#xA;&lt;li&gt;Surpass the top score only a few hundred points&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I'm not doing&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intentionally not get the Top 1 spot&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Extra Alternate Problem&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This time, it's not a hypercasual game but a board game (think Go). Players are matched with each other. Winner gets points. You write a neural network and train it to play the game automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be suspicious to the mod? (I doubt it) Given that it's much more complicated than a hypercasual game as it's a board game, matching with real players, points go up slowly.&lt;/p&gt;&#xA;" OwnerUserId="23571" LastActivityDate="2019-03-30T10:23:56.050" Title="Superhuman AI to disguise as human players" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;theory&gt;&lt;human-like&gt;&lt;gaming&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11544" PostTypeId="2" ParentId="11542" CreationDate="2019-03-30T11:17:45.360" Score="1" Body="&lt;p&gt;If the main goal of AI (which I assume you mean an AGI) is to protect humans and AI will be effective, then AI will always attempt to pursue its main goal (otherwise the assumption of its effectiveness does not hold), even at the expense of other less important goals that it might have. However, if the destruction of a human (or a group of humans) protected or avoided the destruction of other humans, then AI would face a dilemma. In that case, I think it is hard to predict the actions of the AI. Will it act rationally or irrationally? What would it mean for the AI to act rationally? Which parameters will it take into account? Only the number of deaths, or will take into account the future and weight the importance of the lives? How will it define the importance of a human life?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-30T11:22:50.817" LastActivityDate="2019-03-30T11:22:50.817" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11546" PostTypeId="1" AcceptedAnswerId="11561" CreationDate="2019-03-30T11:47:15.987" Score="0" ViewCount="192" Body="&lt;p&gt;Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that &quot;if some NN exist, then edit any NN&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence.&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="23500" LastEditDate="2019-03-31T13:59:53.300" LastActivityDate="2019-03-31T22:02:29.490" Title="Can the inputs and outputs of a neural network be a neural network?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11547" PostTypeId="2" ParentId="11534" CreationDate="2019-03-30T11:48:43.743" Score="1" Body="&lt;p&gt;I am reading Bostrom's book &quot;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1501227742&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superintelligence&lt;/a&gt;&quot;. I have only read the first 2 chapters, but I think he doesn't want to define super-intelligence is a precise way, but he leaves the reader the option to define it in a &quot;sensible&quot; way. However, I think that, in his thoughts, there's the (clear) assumption that a super-intelligence will necessarily need to be general, so a super-intelligence will be an AGI.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-03-30T11:48:43.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11549" PostTypeId="2" ParentId="11535" CreationDate="2019-03-30T16:06:18.110" Score="1" Body="&lt;p&gt;Thanks @nbro for pointing this out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pictorial architecture in the slides uses the Gaussian loss, which when coupled with Maximum Likelihood Estimation gives the squared error loss (not to remove any tractability issues). The main reason we do the encoder Gaussian trick is to force the latent variable &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; to be normal so that we can apply &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$Divergence$&lt;/span&gt; to optimise an otherwise intractable integral. You can get a better intuition and reasoning in this &lt;a href=&quot;https://www.youtube.com/watch?v=uaaqyVS9-rM&amp;amp;t=5s&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pictorial architecture is basically taking the Gaussian loss so that the final loss becomes the squared error loss effectively. Also the loss term used in your blog link is exactly the same loss term used in the &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;, but the blog is using CE loss (it is the more common loss used for classification). I am not sure how they are using the CE loss though, since it is only valid for &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; values and AFAIK MNIST data-set has grayscale images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not exactly sure how they implement the randomness of the Gaussian loss in the decoder structure, but in the simplest of cases they just take the MSE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out this blog on &lt;a href=&quot;http://bjlkeng.github.io/posts/variational-autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VAE's&lt;/a&gt; (where they have taken the mean &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; which they have abbreviated as mean, I have not checked their implementation detail to know what they exactly mean by that) and also this answer on &lt;a href=&quot;https://datascience.stackexchange.com/questions/47061/transform-an-autoencoder-to-a-variational-autoencoder/47078#comment54228_47078&quot;&gt;Data Science&lt;/a&gt; on implementation of VAE's&lt;a href=&quot;https://datascience.stackexchange.com/questions/47061/transform-an-autoencoder-to-a-variational-autoencoder/47078#comment54228_47078&quot;&gt;4&lt;/a&gt; (both of which gives a more general form of loss). Also for the exact Mathematics check out Appendix C of the original paper.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-04-02T09:10:57.480" LastActivityDate="2019-04-02T09:10:57.480" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11550" PostTypeId="1" CreationDate="2019-03-30T17:02:03.673" Score="2" ViewCount="156" Body="&lt;p&gt;In machine learning, there are several metrics to assess the quality of the models: accuracy, precision, recall, f measure, ROC (AUC), etc. There are cases when certain metrics are more appropriate than others. For example, accuracy is not a good metric when there is class imbalance in the dataset (although there are ways of attenuate this issue, e.g. resampling). ROC curves are typically used in binary classifications problems, even though they can also be used in multi-class classification problems (with some tricks). The confusion matrix is also a nice way to visualise the performance of the model (in the case of multi-class classification).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for an as much as possible &lt;em&gt;comprehensive&lt;/em&gt; (which can also be concise) explanation of when a metric (at least, the ones mentioned above) can or should (or not) be used, depending on the dataset (balanced or unbalanced), type of problem (classification or regression), type of classification (binary or multi-class), etc. You can eventually add more details and metrics.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-30T17:10:28.363" LastActivityDate="2019-03-30T17:10:28.363" Title="When to use which metric in machine learning?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;supervised-learning&gt;&lt;metric&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11553" PostTypeId="1" CreationDate="2019-03-31T00:08:42.173" Score="2" ViewCount="290" Body="&lt;p&gt;I've implemented A2C. I'm now wondering why would we have multiple actors walk around the environment and gather rewards, why not just have a single agent run in an environment vector? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally think this will be more efficient since now all actions can be calculated together by only going through the network once. I've done some tests, and this seems to work fine in my test. One reason I can think of to use multiple actors is implementing the algorithm across many machines, in which case we can have one agent on a machine. What else reason should we prefer multiple actors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example of environment vector based on OpenAI's gym&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class GymEnvVec:&#xA;    def __init__(self, name, n_envs, seed):&#xA;        self.envs = [gym.make(name) for i in range(n_envs)]&#xA;        [env.seed(seed + 10 * i) for i, env in enumerate(self.envs)]&#xA;&#xA;    def reset(self):&#xA;        return [env.reset() for env in self.envs]&#xA;&#xA;    def step(self, actions):&#xA;        return list(zip(*[env.step(a) for env, a in zip(self.envs, actions)]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="2444" LastEditDate="2020-05-11T12:39:31.710" LastActivityDate="2020-05-11T12:39:31.710" Title="What is the difference between A2C and running an agent in an environment vector?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;gym&gt;&lt;advantage-actor-critic&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11554" PostTypeId="1" CreationDate="2019-03-31T03:04:51.420" Score="1" ViewCount="103" Body="&lt;p&gt;I have billions of anonymized location coordinates of people movement collected from app. I want to improve user experience by using location data. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For example identify if user is at home or at office so that what they view in app changes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Where will they be tomorrow at particular hour -  so that I can suggest to secure their homes from IOT device if they are out.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding the first point, I tried to use the following rule: at night they are at home, and at day they are at work or school. Regarding the 2nd point, I have no idea how to proceed. Is there any way I could use AI to predict future location and home location?&lt;/p&gt;&#xA;" OwnerUserId="23578" LastEditorUserId="2444" LastEditDate="2019-03-31T20:31:42.523" LastActivityDate="2019-03-31T20:36:31.950" Title="How to predict human future location?" Tags="&lt;machine-learning&gt;&lt;models&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11556" PostTypeId="2" ParentId="11553" CreationDate="2019-03-31T08:09:27.980" Score="1" Body="&lt;p&gt;I believe if you run a single agent in multiple parallel environments many times you will get similar actions in similar states, the reason behind multiple agents is that you will have different agents with different parameters and you can also have different explicit exploration policies so your exploration will be better and you will learn more from environment (see more state space). With single agent you can't really achieve that, you would have a single exploration policy, single parameter set for the agent and most of the time you would be seeing similar states (at least after a while). You would be speeding up your learning process but that's just because you're running multiple environments in parallel (compared to the regular actor-critic or Q-learning). I think quality of learning would be better with multiple different actors.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-03-31T08:09:27.980" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11557" PostTypeId="1" CreationDate="2019-03-31T10:03:05.573" Score="2" ViewCount="227" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://openreview.net/pdf?id=B1DmUzWAW&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Simple Neural Attentive Meta-Learner&lt;/a&gt;, the authors mentioned right before Section 3.1:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we preserve the internal state of a SNAIL across episode boundaries, which allows it to have memory that spans multiple episodes. The&#xA;  observations also contain a binary input that indicates episode termination.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As far as I can understand, SNAIL uses temporal convolutions to aggregate contextual information, from which causal attention learns to distill specific pieces of information. Temporal convolutions does not seems to maintain any internal state, and neither does the attention mechanism they use after &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. This makes me wonder: &quot;What is the internal state of a SNAIL?&quot;&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="8689" LastEditDate="2019-04-01T02:33:01.623" LastActivityDate="2019-09-28T14:02:45.483" Title="What is the internal state of a Simple Neural Attentive Meta-Learner(SNAIL)?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11558" PostTypeId="2" ParentId="11535" CreationDate="2019-03-31T11:04:43.933" Score="0" Body="&lt;p&gt;The VAE architecture from the cs231n class is just a more general version of the code Keras provides, in which the covariance matrix is &lt;span class=&quot;math-container&quot;&gt;$\mathbf 0$&lt;/span&gt;. You can see this from the reparametrization trick&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;x&amp;amp;=\mu+\Sigma\epsilon\\&#xA;&amp;amp;=\mu&amp;amp;\mathrm{if}\ \Sigma=0&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="8689" LastActivityDate="2019-03-31T11:04:43.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11559" PostTypeId="2" ParentId="11539" CreationDate="2019-03-31T12:04:49.787" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is it standard to just call deep RL to any RL algorithm that uses a neural net?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, it seems to have become standard practice to label RL + any NN &amp;quot;Deep Reinforcement Learning&amp;quot;. It is not a formalised term.&lt;/p&gt;&#xA;&lt;p&gt;The whole &amp;quot;Deep Learning&amp;quot; movement started this decade is as much a marketing term as a scientific one. It &lt;em&gt;is&lt;/em&gt; however based on the discovery of real improvements in neural network architecture and training approaches.&lt;/p&gt;&#xA;&lt;p&gt;You may find that some (or even most) of these shallower networks will use improvements designed in the last decade or so, and also associated with deeper networks, such as Xavier initialization, ReLU activation, the Adam optimizer.&lt;/p&gt;&#xA;&lt;p&gt;As a personal opinion, I would say that, if a published experiment uses just 1 or 2 hidden layers, and does not make use of any of these recent advances, then the &amp;quot;Deep&amp;quot; label is almost entirely a branding exercise. There were advances with such networks much longer ago. For instance &lt;a href=&quot;https://cling.csd.uwo.ca/cs346a/extra/tdgammon.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the TD-Gammon paper&lt;/a&gt; is from 1995. For TD-Gammon, the authors used reinforcement learning and a NN with one hidden layer to create a Backgammon player that played better than any human player. This was well before &amp;quot;Deep Learning&amp;quot; was a term used to describe such networks, and the term &amp;quot;Deep Reinforcement Learning&amp;quot; does not appear in that paper.&lt;/p&gt;&#xA;&lt;p&gt;However, because &amp;quot;Deep Learning&amp;quot; is such a loose branding term, there is also an argument that all these older approaches, and pretty much all neural networks with hidden layers, should be included. &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning#Definition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia's definition for Deep Learning&lt;/a&gt; says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Deep learning is a class of machine learning algorithms that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input.&lt;/li&gt;&#xA;&lt;li&gt;learn in supervised (e.g., classification) and/or unsupervised (e.g., pattern analysis) manners.&lt;/li&gt;&#xA;&lt;li&gt;learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Using that definition would include all the papers you cite.  You don't need a 50 layer Resnet architecture to qualify. And the branding exercise makes more sense under that definition, because the newly invented techniques have made such systems that much more viable and worthy of investment (of time &amp;amp; effort as well as financially).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-03-31T19:30:06.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11560" PostTypeId="2" ParentId="11539" CreationDate="2019-03-31T12:16:44.997" Score="4" Body="&lt;p&gt;Even after several years of success of deep learning systems (i.e. neural networks trained with gradient descent and back-propagation), as far as I know, there is not yet a consensus on what constitutes a deep neural network. Some people could use a neural network with 2 hidden layers and call it deep (like in your case), but other people may just dedicate the adjective &lt;em&gt;deep&lt;/em&gt; to refer to neural networks with 10, 100, or more hidden layers. In fact, there are some good reasons to associate the term &lt;em&gt;deep&lt;/em&gt; only to neural networks that have a significant number of hidden layers (e.g. 100): for example, the exploding (or vanishing) gradient problem does not typically arise if you only have one hidden layer but can easily occur with many (e.g. 100) hidden layers.&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless, a neural network with at least one hidden layer can approximate any continuous function, given enough (but finite number of) units (or neurons) in the layers. See the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt;. For this reason, we could start denoting any such neural network as deep, but, although this rule would exclude perceptrons (which can only approximate linear functions, and nobody would probably call them deep anyway), this rule would be a bit redundant or useless (i.e. we may just not use the adjective &lt;em&gt;deep&lt;/em&gt; to start with).&lt;/p&gt;&#xA;&lt;p&gt;In your case, the rule that the authors are using seems to be the following: if it contains more hidden layers than the bare minimum (i.e. 1) to approximate any continuous function, then let's denote it as &lt;em&gt;deep&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-01T13:07:52.930" LastActivityDate="2020-07-01T13:07:52.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11561" PostTypeId="2" ParentId="11546" CreationDate="2019-03-31T16:31:01.130" Score="0" Body="&lt;p&gt;A neural network essentially is a function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{y} = f(\mathbf{x}, \mathbf{\theta})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; is a vector input, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; are changeable or learnable parameters, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; is a vector output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some variations of this in practice, as you can make special arrangements of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;, or use an internal state and feedback loops to allow either or both &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; to be sequences. However, the above function is basically what a neural network is; how it works beyond that summary are details that you can study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;supervised learning&lt;/a&gt;, you are interested in fixed sizes/shapes of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}, \mathbf{y}, \mathbf{\theta}$&lt;/span&gt; and trying to find a value of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(\mathbf{x}, \mathbf{\theta}) \approx g(\mathbf{x})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$g(\mathbf{x})$&lt;/span&gt; is some &quot;true&quot; function that you care about, and can find or generate examples of, but typically don't fully know. The value &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; is called the &lt;em&gt;parameters&lt;/em&gt; of the neural network. In addition to these parameters, there are also &lt;em&gt;hyper-parameters&lt;/em&gt; of the neural network, which include how many neurons there are in each layer, the valid connections between them, which non-linear function is applied after summing connections between them, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are learning algorithms used to find &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; - the many variations of gradient descent being the most popular. Some algorithms - mainly evolutionary approaches - can also vary hyper-parameters, although a more common approach is to repeatedly find &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; using gradient descent, and vary hyper-parameters in different learning trials, using some metric of performance using test data to pick the best one.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that &quot;if some NN exist, then edit any NN&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes - partially. This is a data representation issue. To use it as an input, you would need to express the state of a neural network as a vector - or sequence of vectors - for the input, and the output/edit would also need to be a vector. Probably the simplest way to do this would be to use one network directly output a fixed length vector for &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; of the target network given an existing value of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt;. That would not allow you to change connections or layer sizes etc, but it would be a very straightforward way to express &quot;one neural network altering another&quot; (ignoring whether this was in any way useful for a task).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the output was a full representation of the new network, then you would have a function that took as input the definition of one neural network, and output the definition of another network. It would be up to you to convert to/from implemented neural networks and the representations for input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the output &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; was an &quot;edit&quot; for a change &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \rightarrow \mathbf{x}'$&lt;/span&gt;, then you would have to decide what edits are allowed, design the representation and write code that applied the edit (the NN would not actually make any changes to another NN by itself). There is no standard way to do this, although there are things you could base this on (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The big unanswered question with both of the approaches though is what your &quot;true function&quot; &lt;span class=&quot;math-container&quot;&gt;$g(\mathbf{x})$&lt;/span&gt; is supposed to be. Having a neural network that represents a neural network generator or edit function is only half of the problem. You also need a way to either generate &quot;correct&quot; outputs to learn from, or a way to assess outputs against a goal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal cannot simply be &quot;make a valid edit&quot;, as the number of valid edits that will do nothing useful vastly outnumbers the number of edits that have some specific purpose. This is a similar issue to the fact that there are roughly &lt;span class=&quot;math-container&quot;&gt;$2^{8000000}$&lt;/span&gt; valid 1MB files, but only a small proportion of those will be valid image files, and a smaller proportion still will be valid images that represent a natural image that could be taken by a camera. Neural networks that generate natural images therefore must be trained using natural images as a reference, otherwise they will tend to produce meaningless static-like noise.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is very broadly compatible with an &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_life&quot; rel=&quot;nofollow noreferrer&quot;&gt;Articial Life&lt;/a&gt; approach to AI. Although there are two differences between what you are proposing and typical A-life approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Evolutionary algorithms need some measure of fitness, in order to select the best performing individuals to take forward. A-life solves this by implementing a very open environment that makes no direct judgement on outputs of functions, but allows virtual creatures that collect enough resources (defined in the environment) to procreate. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Your suggestion contains no hint that you are thinking of any kind of measure of success or fitness for either the editing NN or target NN. You will need some measure of fitness at least for the target network (and maybe the editing network too) if you intend to use evolutionary algorithms&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A-life typically does not treat the evolutionary algorithm itself (the editing or the NNs) as a learning goal. You will not see the results of a good or bad editor until many simulations have passed, so this &quot;meta-search&quot; is likely to be incredibly slow.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A-life simulations are typically already quite slow to reach behaviour which is &lt;em&gt;interesting&lt;/em&gt; (because it has emerged without direction from the developer), but usually quite &lt;em&gt;simple&lt;/em&gt; for the given environment such as predators chasing prey.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From what we know of the evolution of life, a simple feedback mechanism of RNA molecules editing other RNA molecules - the &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/RNA_world&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA world&lt;/a&gt;&quot; - is considered a likely pre-life step. This has some parallels with what you are suggesting - and this or something similar perhaps, has resulted in intelligent beings such as ourselves. So your idea perhaps has some merit in a theoretical sense. However, it took biology billions of years to go from such a basic stage to self-aware creatures, and this took the full processing power of large numbers of atoms interacting in ways that even all the computers in the world could not simulate in a fraction of real time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To speed things up, and turn the idea into something feasible, you would need to look into viable environments and evaluations that would focus the learning towards direct measures of intelligence. Also, you would do well to compare your idea about NNs editing NNs with learning approaches that don't use such a feedback loop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no theory that suggests a NN that can edit or output other NNs would offer any advantage for research into strong AI, compared to other search methods. The idea is basically &quot;AI alchemy&quot; - an idea of an experiment that could perhaps be done, but without any theory backing it as being better or worse than other ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I would expect the search for a NN which is a good NN editor for a NN which has some other task, to be too slow to be useful when faced with very broad tasks such as exhibiting high level reasoning.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-03-31T22:02:29.490" LastActivityDate="2019-03-31T22:02:29.490" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="11562" PostTypeId="2" ParentId="11510" CreationDate="2019-03-31T17:05:14.057" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;you need to take into account the demographics of the patient&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How, exactly? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it a difference of, say, threshold? In this case you can do this serially (as @mirror2image mentions): process the image and then conclude by comparing the size of what you saw to, say, an age-dependent threshold.&lt;/li&gt;&#xA;&lt;li&gt;Or has the whole processing to be different? In the extreme, you would not wait until the very end before asking whether the patient is a man if you are looking for prostate cancer.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To design the model, you need enough medical understanding to make such choices. The model can handle the parameters, but you have to choose the architecture.&lt;/p&gt;&#xA;" OwnerUserId="23584" LastActivityDate="2019-03-31T17:05:14.057" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11563" PostTypeId="1" CreationDate="2019-03-31T18:10:45.487" Score="2" ViewCount="79" Body="&lt;p&gt;I am trying to understand the similarities and differences between: (i) the UCT algorithm in Kocsis and Szepesvári (2006); (ii) the UCT algorithm in Section 3.3 of Browne et al (2012); (iii) the MCTS algorithm in Silver et al. (2016); (iv) the MCTS algorithm in Silver et al. (2017).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be really grateful for some help identifying the similarities and differences in these papers, I am doing some research and really struggling right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i) &lt;a href=&quot;http://ggp.stanford.edu/readings/uct.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ggp.stanford.edu/readings/uct.pdf&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) &lt;a href=&quot;http://mcts.ai/pubs/mcts-survey-master.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mcts.ai/pubs/mcts-survey-master.pdf&lt;/a&gt; (Section 3.3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(iii) &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(iv) &lt;a href=&quot;https://deepmind.com/documents/119/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/documents/119/agz_unformatted_nature.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23589" LastEditorUserId="2444" LastEditDate="2019-04-01T21:53:00.140" LastActivityDate="2019-04-01T21:53:00.140" Title="Similarities and differences between UCT algorithms in (i), (ii), (iii) and (iv)?" Tags="&lt;algorithm&gt;&lt;monte-carlo-tree-search&gt;&lt;alphago&gt;&lt;alphazero&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11565" PostTypeId="1" AcceptedAnswerId="11583" CreationDate="2019-03-31T20:18:51.570" Score="1" ViewCount="185" Body="&lt;p&gt;I am following &lt;a href=&quot;https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html#6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this TensorFlow JS tutorial&lt;/a&gt; where you load car data. The data looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[{x:100, y:20}, {x:80, y:33}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;X is the horsepower of a car, Y is the expected miles per gallon usage. After creating the model I save it locally using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;async function saveModel(){&#xA;    await model.save('downloads://cars-model');&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Next, I load the model in a separate project, to make predictions without needing the original data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEW PROJECT&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;async function app(){&#xA;    let model = await tf.loadLayersModel('./cars-model.json');&#xA;    console.log(&quot;car model is loaded!&quot;);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I expect to be able to run &lt;code&gt;predict&lt;/code&gt; here, on a single number (say, 120)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.predict(tf.tensor2d([120], [1, 1]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTION&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the number 120 needs to be normalised to a number between 0-1, just like the training data was. But how do I know the &lt;code&gt;inputMin, inputMax, labelMin, labelMax&lt;/code&gt; values from the loaded model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To un-normalise the prediction (in this case 0.6) I also need those original values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I normalise/un-normalise data when loading a model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;original prediction code uses label and input values from the original data&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function testModel(model, inputData, normalizationData) {&#xA;    const { inputMax, inputMin, labelMin, labelMax } = normalizationData;&#xA;&#xA;    // Generate predictions for a uniform range of numbers between 0 and 1;&#xA;    // We un-normalize the data by doing the inverse of the min-max scaling &#xA;    // that we did earlier.&#xA;    const [xs, preds] = tf.tidy(() =&amp;gt; {&#xA;&#xA;        const xs = tf.linspace(0, 1, 100);&#xA;        const preds = model.predict(xs.reshape([100, 1]));&#xA;&#xA;        const unNormXs = xs&#xA;            .mul(inputMax.sub(inputMin))&#xA;            .add(inputMin);&#xA;&#xA;        const unNormPreds = preds&#xA;            .mul(labelMax.sub(labelMin))&#xA;            .add(labelMin);&#xA;&#xA;        // Un-normalize the data&#xA;        return [unNormXs.dataSync(), unNormPreds.dataSync()];&#xA;    });&#xA;&#xA;&#xA;    const predictedPoints = Array.from(xs).map((val, i) =&amp;gt; {&#xA;        return { x: val, y: preds[i] }&#xA;    });&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11620" LastEditorUserId="11620" LastEditDate="2019-04-01T13:06:30.747" LastActivityDate="2020-09-16T13:48:06.490" Title="How do I normalise/un-normalise data when loading a model?" Tags="&lt;tensorflow&gt;&lt;models&gt;&lt;javascript&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11566" PostTypeId="2" ParentId="11554" CreationDate="2019-03-31T20:30:23.320" Score="1" Body="&lt;p&gt;Your problem is often called (in the literature) &lt;em&gt;human mobility prediction&lt;/em&gt;. There has been some research in this area. Have a look at it on the web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you might want to use any statistical or machine learning model that uses the historical data to predict the future. For example, you could try to use an hidden Markov model (for both point 1 and 2). However, before that, you might also need to do some feature engineering, if you only have locations (and not e.g. the time of the day when those locations were recorded).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-03-31T20:36:31.950" LastActivityDate="2019-03-31T20:36:31.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11567" PostTypeId="1" CreationDate="2019-03-31T21:07:24.583" Score="9" ViewCount="251" Body="&lt;p&gt;I've pondered this for a while without developing an intuition for the math behind the cause of this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what causes a model to need a low learning rate?&lt;/p&gt;&#xA;" OwnerUserId="20257" LastActivityDate="2019-04-02T03:19:05.670" Title="What causes a model to require a low learning rate?" Tags="&lt;machine-learning&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11570" PostTypeId="2" ParentId="2324" CreationDate="2019-04-01T00:31:33.550" Score="0" Body="&lt;p&gt;This depends on the definition(s) of AGI and ASI. Both are currently ill-defined. Most researchers in AGI follow their own &lt;a href=&quot;https://arxiv.org/pdf/0706.3639.pdf%20a%20collection%20of%20definitions%20of%20intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition&lt;/a&gt; of AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least one researcher believes that there is no such thing as ASI. This is because the basic principles of said AGI always stay the same. It may be learning processes, the core logic(s) and the control logic (reasoning systems are divided into control systems and logic systems, the control system(s) decide which derivations are fruitful).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ASI may be defined as a search for any combination of these (just a subset which come to my mind):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;search for better algorithms&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;heuristics&lt;/li&gt;&#xA;&lt;li&gt;better contemporary (NN) architectures&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;learning mechanisms&lt;/li&gt;&#xA;&lt;li&gt;solving techniques&lt;/li&gt;&#xA;&lt;li&gt;higher subjective beauty&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;better compression of knowledge&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;better subsystems&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;NN and in general architectures&lt;/li&gt;&#xA;&lt;li&gt;better embedded AGI's    &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;faster solving capabilities of known problems&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There are limitations to any sort of (recursive) self improvement however.&#xA;Examples of these are&#xA; * the score of AlphaGo and AlphaGo-Zero plateaus after a long enough training period&#xA; * supercompilation of a supercompiled program yields no improved program after a few iterations&#xA; * ...&#xA;Note here that these are examples about weak-AI and may not apply to AGI - but it is very likely in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So the level of worry depends on the plausible (or followed) definition of AGI and the assumptions of the mechanisms an AGI may employ.&lt;/p&gt;&#xA;" OwnerUserId="21141" LastActivityDate="2019-04-01T00:31:33.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11571" PostTypeId="1" CreationDate="2019-04-01T00:36:19.820" Score="1" ViewCount="32" Body="&lt;p&gt;Description logic is a fragment of first order logic, but description logic is decidable and first order logic not decidable. Why is that? what is the role of variables in first order logic to make it undecidable?&lt;/p&gt;&#xA;" OwnerUserId="23590" LastEditorUserId="2444" LastEditDate="2019-04-01T13:25:36.273" LastActivityDate="2019-04-01T13:25:36.273" Title="Why is description logic decidable but first order logic is not decidable?" Tags="&lt;logic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11574" PostTypeId="1" CreationDate="2019-04-01T03:39:55.907" Score="2" ViewCount="39" Body="&lt;p&gt;I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using all this information for categorization?  I see two options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Concatenate the text from all fields, preceding each field content with a special token. Run concatenated text through LSTM.&lt;/li&gt;&#xA;&lt;li&gt;Train one model per field. Concatenate output from each model in a hidden layer and pass into subsequent layers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are the benefits of each of the approaches? Is there an alternative I'm missing?&lt;/p&gt;&#xA;" OwnerUserId="23599" LastActivityDate="2019-04-01T03:39:55.907" Title="Multi-field text input for LSTM" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;hidden-layers&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11575" PostTypeId="1" CreationDate="2019-04-01T04:53:30.843" Score="5" ViewCount="135" Body="&lt;p&gt;I’m looking to match two pieces of text - e.g. IMDb movie descriptions and each person’s description of the type of movies they like. I have an existing set of ~5000 matches between the two. I particularly want to overcome the cold-start problem: what movies to recommend to a new user? When a new movie comes out, to which users should it be recommended? I see two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run each description of a person through an LSTM; do the same for each movie description; concatenate the results for some subset of possible combinations of people and movies, and attach to a dense net to then predict whether it’s a match or not&lt;/li&gt;&#xA;&lt;li&gt;Attempt to augment collaborative filtering with the output from running the movie description and person description through a text learner.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are these tractable approaches?&lt;/p&gt;&#xA;" OwnerUserId="23599" LastEditorUserId="16565" LastEditDate="2019-04-04T20:24:16.393" LastActivityDate="2019-05-09T21:27:16.143" Title="Cold start collaborative filtering with NLP" Tags="&lt;natural-language-processing&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11576" PostTypeId="1" AcceptedAnswerId="11592" CreationDate="2019-04-01T06:29:19.637" Score="7" ViewCount="1639" Body="&lt;p&gt;Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about the random forest?&lt;/p&gt;&#xA;" OwnerUserId="23601" LastEditorUserId="2444" LastEditDate="2019-11-21T03:05:15.120" LastActivityDate="2020-12-05T03:28:10.943" Title="Are decision tree learning algorithms deterministic?" Tags="&lt;decision-tree&gt;&lt;random-forest&gt;&lt;id3-algorithm&gt;&lt;c4.5-algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11579" PostTypeId="2" ParentId="11008" CreationDate="2019-04-01T10:46:59.163" Score="5" Body="&lt;p&gt;They would probably have followed the same sequence we do:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;be amazed at the capabilities,&lt;/li&gt;&#xA;&lt;li&gt;ask how it is done,&lt;/li&gt;&#xA;&lt;li&gt;wonder whether this is really intelligence and (or) point out our narrow the performance was,&lt;/li&gt;&#xA;&lt;li&gt;require more next time to be impressed again.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23584" LastActivityDate="2019-04-01T10:46:59.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11580" PostTypeId="2" ParentId="11571" CreationDate="2019-04-01T11:34:23.330" Score="1" Body="&lt;p&gt;This has to do with the fact that you can define arithmetic inside the axiomatic system or not. In description logic you cannot speak about arithmetic sentences and in first order logic you do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you look at the proof of incompleteness you will understand this in depth. This demonstration depends on an arithmetic coding of statements, and this representation is fundamental to obtain the conclusion.&lt;/p&gt;&#xA;" OwnerUserId="23608" LastActivityDate="2019-04-01T11:34:23.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11583" PostTypeId="2" ParentId="11565" CreationDate="2019-04-01T13:02:27.730" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I convert this to a number between 0-1 without having access to the original car data?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You save the normalisation parameters (typically an offset and a multiplier for each column), and consider that part of the model. Typically you do this when you originally scale training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you want to re-use the model, as well as loading the neural network architecture and weights, you need to load the normalisation parameters in order to re-use those too and scale your inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When tutorials present a self-contained neural network that loads training data, builds a model, then tests that model, all in the same process, then often this step is not shown. However, saving the normalisation data is important, and basically should be considered part of the model, even though it is not directly part of the neural network parameters or hyper-parameters.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-01T13:02:27.730" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11585" PostTypeId="2" ParentId="11567" CreationDate="2019-04-01T13:27:29.457" Score="5" Body="&lt;p&gt;Gradient Descent is a method to find the optimum parameter of the hypothesis or minimize the cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UW6mo.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UW6mo.png&quot; alt=&quot;formula&quot;&gt;&lt;/a&gt;&#xA;where alpha is learning rate &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the learning rate is high then it can overshoot the minimum and can fail to minimize the cost function.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/w7Y3c.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w7Y3c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;hence result in a higher loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qwqsM.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qwqsM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Gradient descent can only find local minimum so, the lower learning rate may result in bad performance. To do so, it is better to start with the random value of the hyperparameter can increase model training time but there are advanced methods such as adaptive gradient descent can manage the training time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of optimizer for the same task but no optimizer is perfect. It depends on some factors&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;size of training data: as the size of the training data increases training time for model increases. If you want to go with less training model time you can choose a higher learning rate but may result in bad performance.&lt;/li&gt;&#xA;&lt;li&gt;Optimizer(gradient descent) will be slow down whenever the gradient is small then it is better to go with a higher learning rate.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;PS. It is always better to go with different rounds of gradient descent&lt;/p&gt;&#xA;" OwnerUserId="15368" LastEditorUserId="15368" LastEditDate="2019-04-01T14:11:39.637" LastActivityDate="2019-04-01T14:11:39.637" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11586" PostTypeId="2" ParentId="5428" CreationDate="2019-04-01T18:05:53.573" Score="2" Body="&lt;p&gt;You can contribute to AGI research in several ways.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Write papers for the &lt;a href=&quot;http://agi-conf.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence conference&lt;/a&gt;, which is peer-reviewed, or other AGI conferences or journals, or just submit them to e.g. &lt;a href=&quot;https://arxiv.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arxiv&lt;/a&gt;. For example, you can write papers about AGI algorithms or architectures.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Contribute to (open source) AGI systems, like &lt;a href=&quot;https://github.com/opencog&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt;, &lt;a href=&quot;https://github.com/brohrer/becca&quot; rel=&quot;nofollow noreferrer&quot;&gt;BECCA&lt;/a&gt; or &lt;a href=&quot;https://github.com/opennars/opennars&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenNARS&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Implement AGI algorithms, mechanisms or architectures, and put them, for example, on Github.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21141" LastEditorUserId="2444" LastEditDate="2019-11-22T22:55:58.013" LastActivityDate="2019-11-22T22:55:58.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11587" PostTypeId="2" ParentId="11563" CreationDate="2019-04-01T18:10:04.220" Score="1" Body="&lt;p&gt;The algorithm in the &lt;strong&gt;2012 survey article&lt;/strong&gt; (your second link) is the &lt;strong&gt;most common / standard implementation&lt;/strong&gt;. Whenever someone mentions using MCTS or UCT, without explicitly stating any other info, it's safe to assume that that pseudocode is what they're using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper by &lt;strong&gt;Kocsis and Szepesvári from 2006&lt;/strong&gt; (your first link) is (one of) the &lt;strong&gt;original publication(s) on UCT&lt;/strong&gt;. It is very similar to the &quot;standard&quot; implementation as described in the survey paper. If I recall correctly, the only important difference is that the algorithm as described in 2006 keeps track of at which point in time during an episode a reward is observed, and accounts for that timing in the backpropagation phase (i.e. if a reward is observed at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in an episode, credit for that reward is not assigned to states/action after time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;). It also used a discounting factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; to discount the importance of reward depending on temporal distance, which is uncommon otherwise in MCTS literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of those differences are due to the original, 2006 paper having more of a &quot;Markov decision process&quot; or &quot;Reinforcement Learning&quot; flavour, whereas otherwise MCTS has especially become popular in AI for (board) games where we often by default assume that there is only a single nonzero reward (win or lose) at the end of every episode anyway, which makes those differences less meaningful.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Both of the AlphaGo papers (AlphaGo and AlphaGo Zero) use a &quot;foundation&quot; of MCTS that is mostly similar to the one from the 2012 survey article. The core components are all the same. Both of those systems add on a lot of (very important, and some quite complex) bells and whistles though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going off the top of my head (should be mostly accurate, but best details are in the original source of course!) &lt;strong&gt;AlphaGo&lt;/strong&gt; (your third link) added Neural Networks trained in various ways to output policies (mapping from states to probability distributions over actions) and value functions (mapping from states to &quot;value estimates&quot;, or estimates of win percentage). Trained policy networks were used in a variant of the &quot;selection&quot; phase (no longer UCB1 strategy as in UCT) to bias selection. A different (simpler function, not a deep net) policy was used to run play-outs (no long uniform at random action selection as in UCT). A combination of observed reward at end of play-out + value function estimate at start of play-out was used for backpropagation (no longer only backpropagating reward observed in play-out as in UCT).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;AlphaGo Zero&lt;/strong&gt; is, if we're purely looking at the MCTS part, quite similar to AlphaGo. The Neural Networks were a bit different (a single one, with multiple outputs for policy + value), and play-outs were no longer used at all (just immediate backpropagation of value function estimates after MCTS' selection + expansion phases). Apart from that, the primary differences going from AlphaGo to AlphaGo Zero were in the learning process used to train the Neural Networks really, not so much in the MCTS side of things.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-04-01T18:10:04.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11588" PostTypeId="1" CreationDate="2019-04-01T18:41:45.310" Score="3" ViewCount="131" Body="&lt;p&gt;I am taking AI this semester and we have a semester project that will last 4 weeks. We can choose just about anything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what are some possible semester projects that can be finished in a 4-week time-frame?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some background information: I am a graduate student in CS, but this is my first AI course. My research area is in the space of data mining and analytics. I am open to doing anything that seems interesting and creative. &lt;/p&gt;&#xA;" OwnerUserId="23622" LastEditorUserId="2444" LastEditDate="2020-06-16T10:47:57.887" LastActivityDate="2020-06-16T13:14:04.727" Title="What are some possible projects that can be finished in a 4-week time-frame?" Tags="&lt;academia&gt;&lt;homework&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11589" PostTypeId="2" ParentId="11588" CreationDate="2019-04-01T23:26:55.333" Score="7" Body="&lt;p&gt;Welcome to AI.SE @Kate_Catelena!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I teach AI courses at the undergraduate level, and so have seen a lot of semester projects over the years. Here are some templates that often lead to exciting outcomes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Pick a &lt;em&gt;new&lt;/em&gt; board or card game, and write a program to play it. Your course has probably covered Adversarial Search, and may also have covered Monte Carlo Tree Search, or self-play reinforcement learning approaches. These projects are often fun to mark and creative because they are easy enough to be well done, and yet there are always new, exciting, domains to apply these algorithms to. Some examples of past projects that I thought were neat were an AI to play the boardgame Tac (mostly A* Search), and an AI to play the card game Love Letter (mostly Counter-factual minimax regret, the algorithm used to solve poker).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pick a question that you would like to know the answer to, that could be addressed with machine learning. Then implement your own ML algorithm (decision tree learners are fairly easy), gather your own data, and show a result. Examples of interesting projects I've seen in the past are using ML to find out which of a number of factors most strongly influenced a students' subjective quality of sleep; and which items are most commonly purchased along with camping supplies (using association rule mining).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Anything involving reinforcement learning. RL projects are always neat to see if accompanied by a visualization showing the learner's behavior at different stages. A strong past project involved a student simply replicating Sutton &amp;amp; Barto's Acrobot experiments with their own implementation of the SARSA-Lambda algorithm.  Other things that might be neat include making a trainable &quot;pet&quot; that the user can influence, or solving games using self-play.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Theoretical results might seem intimidating but are often more accessible than one might think, especially if your discrete math skills are strong. I have had many student projects where the student went away to look at theory papers in ML or Multiagent Systems, found a suggestion in the future work sections that wasn't a big result, but that was fairly easy to prove, and proved it. Sometimes these are even publishable. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replications. Go find an interesting AI paper (use scholar.google.com), and then see if you can do exactly what the authors suggest and if you get the same result or not. Then, if you have time, see if you can improve on the results. These are often most interesting when you find a paper written in a different field that uses AI. Often the authors of such papers know less about AI than you do, and so it can be fairly easy to improve on their results. I have had several students do projects like this to great effect.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Those categories are a bit vague, but remember: AI touches almost anything. Pick your favorite hobby, and see whether you can relate it to AI using one of the approaches above. Nothing makes a project stand out like one that applies AI to solve some real issues in an exciting domain. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-01T23:26:55.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11590" PostTypeId="2" ParentId="11588" CreationDate="2019-04-02T00:50:25.013" Score="2" Body="&lt;p&gt;Here are some possible options&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Music Generation using GA/MA&lt;/li&gt;&#xA;&lt;li&gt;Open AI's gym projects&lt;/li&gt;&#xA;&lt;li&gt;2048 on RL and search algorithms&lt;/li&gt;&#xA;&lt;li&gt;Fixing bugs in the source code of some AI software project&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20522" LastEditorUserId="2444" LastEditDate="2020-06-16T13:14:04.727" LastActivityDate="2020-06-16T13:14:04.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11592" PostTypeId="2" ParentId="11576" CreationDate="2019-04-02T03:47:56.720" Score="5" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Generally, yes. Most decision tree learners, like the common ID3 and C4.5/C5.0 algorithms, are deterministic. At each step, the learners consider all possible features that have not yet been used to split the data, and find the splits that maximize some function (e.g. information gain). There is no randomness (or pseudo-randomness) in this process.&lt;/p&gt;&#xA;&lt;p&gt;The exceptions to this would be if you used randomness to break ties (rather than, say, using the index of each feature, as is common), but this would be an unusual modification.&#xA;What about the random forest?&lt;/p&gt;&#xA;&lt;p&gt;Like the name suggests, random forests &lt;em&gt;do&lt;/em&gt; make use of randomness, or at least, pseudo-randomness. If we're only concerned about whether or not the algorithm is deterministic in the usual sense of the word (at least, within computer science), the answer is &lt;strong&gt;no&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you start the same random forest learning algorithm, with the same datasets, at two different times (or, using two different seeds for your pseudorandom number generator), you will get two different forests. This is because the algorithm selects random subsets of the features and/or datapoints to learn on, and, if different seeds are used, the subsets will be different each time.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-12-05T03:28:10.943" LastActivityDate="2020-12-05T03:28:10.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11593" PostTypeId="1" AcceptedAnswerId="11601" CreationDate="2019-04-02T06:28:09.127" Score="1" ViewCount="155" Body="&lt;p&gt;I'm trying to generate images at minimum of size 128 x 128 with a Generative Adversarial Network. I already tried a SAGAN pytorch implementation, but I'm not very happy with results. The images look cool but and I see some correct shape but without explanation you wouldn't know what the images are about. I have a dataset of 4000 images. Lightness, colors and shapes vary a lot, but they are similar in style and on what they portray. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;With a Google Cloud V100 GPU the GAN would run a week to two with default parameters. Does this sound realistic time for this kind of dataset? It's definitely not feasible for me.&lt;/li&gt;&#xA;&lt;li&gt;Is 4000 images enough to train a GAN from scratch?&lt;/li&gt;&#xA;&lt;li&gt;Is there any implementation with pytorch/keras that would be good to get nice results with?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3579" LastActivityDate="2019-04-02T16:01:08.697" Title="How to get good results with GAN and some thousands of images?" Tags="&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11595" PostTypeId="1" CreationDate="2019-04-02T11:15:33.117" Score="1" ViewCount="77" Body="&lt;p&gt;I've been working on genetic algorithms &amp;amp; evolutionary strategies for a while now in a research context. Across the vast majority of the articles and content I've read, every single one of them will either use Python, Matlab, or Java/C++ to build &amp;amp; benchmark their algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an objective reason for these languages to be the single ones used in a research environment? Mainly in contrast with other languages like C#, or Javascript, that are almost never used (despite being some of the most used programming languages in other areas), whereas it would definitely be possible to code in practice all current algorithms in them.&lt;/p&gt;&#xA;" OwnerUserId="23637" LastActivityDate="2019-12-28T17:00:14.143" Title="Is there a reason evolutionary algorithms are language-bound in research material?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;programming-languages&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11596" PostTypeId="1" AcceptedAnswerId="11604" CreationDate="2019-04-02T12:11:14.293" Score="1" ViewCount="87" Body="&lt;p&gt;I am working on a DDQN with 5 LSTM layers and 3 actions as output and state space of 21 features. I am dividing the dataset into episodes of 720 timesteps, for each episode the agent acts greedily for the first 480 steps without training, collecting a replay memory, and then update the parameters each step for the subsequent 240 steps using a window size (of 96 steps) randomly sampled from the replay memory (that always saves the last 480). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that so far the agent learned the optimal policy just once, and it looks like this&lt;a href=&quot;https://i.stack.imgur.com/CFyfg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CFyfg.png&quot; alt=&quot;Optimal Q-values&quot;&gt;&lt;/a&gt;&#xA;(on the test set, training is off) where, as you can see, the agent dynamically changes its evaluation of the state and acts greedily accordingly. All works fine and the performances are optimal, however, I have to slightly change the normalization of the database and rerun the training to get new parameters fitted to the new database. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to get to the same result has been proven impossible so far, (even keeping all settings the same!) because most of the time the agent learns to keep its q-values static such as &lt;a href=&quot;https://i.stack.imgur.com/lNut3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lNut3.png&quot; alt=&quot;that&quot;&gt;&lt;/a&gt;. &#xA;Note: this is an extract of the end of an episode and the beginnig of a new one, the noisy behaviour at the extreme is due to the model being trained for each step, in the middle the training is off and the agent acts greedily (as I explained above). The problem is that the learned parameters give rise to static q values that do not change while the state does and inevitably make the agent stuck in suboptimal strategy, as they never changes actions, even on longer sequences on the test set. The middle part of the second picture, where trainig is off, should look like the first picture, however, I am unable to get back to that optimal behaviour, even keeping all the parameters as they were. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea on what can be the cause of this anomalous behaviour? &lt;/p&gt;&#xA;" OwnerUserId="23638" LastActivityDate="2019-04-02T16:32:12.813" Title="DQN Q-values are static" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;long-short-term-memory&gt;&lt;dqn&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11599" PostTypeId="1" CreationDate="2019-04-02T14:28:09.703" Score="1" ViewCount="36" Body="&lt;p&gt;I was thinking about what I can do for my thesis in upcoming academic semester, and I came across an idea. The idea is like: &quot;If there is any kind of system that generates website designs itself.&quot; If no, then I can go for it, and I will be lucky if anyone has the same idea. We can collaborate. In case, if there's any project or system (Open source or not) that can do this or has been initiated in this context, I want to contribute solely. If anyone has any clue or knowledge over this kind of system, please do inform me. As I haven't done anything like this before, I want to learn. Any kind of suggestion or assistance on this idea will be so helpful for me.&lt;/p&gt;&#xA;" OwnerUserId="23643" LastEditorUserId="2444" LastEditDate="2019-04-02T16:23:01.167" LastActivityDate="2019-04-02T16:23:01.167" Title="Is there any system that generates website designs?" Tags="&lt;research&gt;&lt;open-source&gt;&lt;creativity&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11600" PostTypeId="2" ParentId="11595" CreationDate="2019-04-02T15:51:58.003" Score="2" Body="&lt;p&gt;I would say there are quite a few different reasons for this, with the proportion of each dependent on a given researcher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I use python for the vast majority of what I do. And for me, it is due to a few different factors:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I was already familiar with python, and it is a simple, high-level abstraction. This is probably the reason a lot of people set it and forget it, particularly with python. It allows them to focus on the ideas and implementation of said ideas, without worrying about all the junk that comes with trying to write a program in a faster language like C++&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The vast majority of ML/DS packages are only or primarily supported via python. I think this is probably the main reason for most in the field, as even if one can implement the architecture in a faster language, the time to do so would likely even out when taking into account the time required to prototype a given model. Tensorflow and others are supported for other languages but do not see the same level of dev support.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The ability to deploy models to multiple platforms without headache. When working in an environment where the work is also applied, the ability to deploy a given model without too much debugging cannot be understated. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are just a few of the main ones, and as I said the reasons for a particular language over another is primarily a preferential one, and can even vary by requirements(i.e speed)&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-02T15:51:58.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11601" PostTypeId="2" ParentId="11593" CreationDate="2019-04-02T16:01:08.697" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;With a Google Cloud V100 GPU the GAN would run a week to two with&#xA;  default parameters. Does this sound realistic time for this kind of&#xA;  dataset? It's definitely not feasible for me.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, V100s are quite beefy. You shouldn't even need a week. Obviously this is based on my experience with various problems, rather than a concrete calculation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is 4000 images enough to train a GAN from scratch?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For the size you want to generate, it is still on the edge of what would constitute a decent training set. You will get some results(depending on architecture) but will probably want to grab more data if at all possible.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any implementation with pytorch/keras that would be good to&#xA;  get nice results with?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would check out this link: &lt;a href=&quot;https://github.com/eriklindernoren/Keras-GAN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eriklindernoren/Keras-GAN&lt;/a&gt;. It has some nice implementations in Keras. As far as the particular GAN to use, I would start out with a vanilla GAN that fits your purposes and focus on toying with hyperparameters, and if that doesn't work, look into one of the other variations that correlate well with your problem.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-02T16:01:08.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11602" PostTypeId="2" ParentId="11599" CreationDate="2019-04-02T16:10:58.870" Score="1" Body="&lt;p&gt;The closest research that I am aware of is in artificial intelligence designed user interfaces(&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370210000822&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;,&lt;a href=&quot;https://arxiv.org/abs/1705.07962&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scope of which vary tremendously. Some teams are trying to generate UIs based on some user-defined parameters, others are trying to generate based off of images(as in the second link). I think part of the reason research is focused here is two-fold: One, we aren't very good at generating UIs automatically yet, and 2, it is an exponentially harder problem to develop a backend that has integrated calls and features required for a fully-featured website or application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that being said, one could adapt this research into things like one-page sites or other simple implementations that might bear fruit.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-02T16:10:58.870" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11604" PostTypeId="2" ParentId="11596" CreationDate="2019-04-02T16:32:12.813" Score="0" Body="&lt;p&gt;It is likely converging to a far worse local optimum from which it can't recover, so yes I would guess if all else is the same, that is where the issue would be. I would first try to adjust hyperparameters from a static seed for weight initialization.  ε and α, are likely good places to start. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that fails, adjust some architecture params like the number of units or layers in the network as well as the starting seed value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seemingly have run into a core problem at the heart of ML research which is reproducibility, which is often predicated by initialization.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-02T16:32:12.813" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11605" PostTypeId="2" ParentId="2226" CreationDate="2019-04-02T16:40:21.590" Score="2" Body="&lt;p&gt;In the reinforcement learning setting, an agent interacts with an environment in (discrete) time steps, which are incremented after the agent takes an action, receives a reward and the &quot;system&quot; (the environment and the agent) moves to a new state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, at time step &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; (the first time step), the environment (including the agent) is in some state &lt;span class=&quot;math-container&quot;&gt;$s_t = s_0$&lt;/span&gt;, takes an action &lt;span class=&quot;math-container&quot;&gt;$a_t = a_0$&lt;/span&gt; and receives and reward &lt;span class=&quot;math-container&quot;&gt;$r_t = r_0$&lt;/span&gt; and the environment (including the agent) moves to a next state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1} = s_{0 + 1} = s_1$&lt;/span&gt;, which will also be the state that the environment will be in at the next time step, &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;, hence the notation &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt;. Here, the subscripts &lt;span class=&quot;math-container&quot;&gt;$_t$&lt;/span&gt; refer to the time step associated with those &quot;entities&quot; (state, action and rewards). So, after one time step (or after &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;), the agent will be in state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; and the new time step will be &lt;span class=&quot;math-container&quot;&gt;$t + 1 = 0 + 1 = 1$&lt;/span&gt;. So, we are now at time step &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt; (because we have just incremented the time step) and the agent is in state &lt;span class=&quot;math-container&quot;&gt;$s_{t} = s_1$&lt;/span&gt;. The previously described interaction then repeats: the agent takes an action &lt;span class=&quot;math-container&quot;&gt;$a_{t} = a_1$&lt;/span&gt;, gets the reward &lt;span class=&quot;math-container&quot;&gt;$r_t = r_1$&lt;/span&gt; and the environment moves to the state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1} = s_{1+1} = s_{2}$&lt;/span&gt;, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your summation, we are just discounting the rewards using a value denoted by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (which is usually between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;), that is often called the &quot;discount factor&quot;. That summation represents the summation of the rewards the agent will received starting (in this case) from time step &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;. We could also just have &lt;span class=&quot;math-container&quot;&gt;$r_1 + r_2 + r_3 + \dots $&lt;/span&gt;, but, for technical or mathematical reasons, we often &quot;discount&quot; the rewards, that is, we multiply them by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (raised to a power associated with the time step that reward will be received).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above description, I said that, at some time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent takes an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and receives a reward &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt;. However, it is often the case that the reward received after taken an action at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is denoted by &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}$&lt;/span&gt;. I think this is a little confusing, but not conceptually &quot;wrong&quot;, because one might think that the reward for having performed an action at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is only received at the next time step. (You should get used to slightly different notations and terminology. At the beginning, it is not easy to understand, if the notation is not precise and consistent across sources, but you will get used to it, the more you learn about the topic, in the same way that you get used to a new language).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-02T16:52:50.700" LastActivityDate="2019-04-02T16:52:50.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11607" PostTypeId="5" CreationDate="2019-04-02T20:09:20.130" Score="0" Body="&lt;p&gt;An interdisciplinary subfield involcing computer science, statistics, databases and machine learning. It utilizes AI to extract information from large data, and to transform it into formats better suiting the requirements.&lt;/p&gt;&#xA;" OwnerUserId="2255" LastEditorUserId="2255" LastEditDate="2019-04-04T20:38:57.350" LastActivityDate="2019-04-04T20:38:57.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11608" PostTypeId="4" CreationDate="2019-04-02T20:09:20.130" Score="0" Body="The process of discovering patterns in large data sets by AI." OwnerUserId="2255" LastEditorUserId="2255" LastEditDate="2019-04-04T20:29:38.997" LastActivityDate="2019-04-04T20:29:38.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11609" PostTypeId="1" AcceptedAnswerId="11611" CreationDate="2019-04-02T20:57:54.037" Score="6" ViewCount="139" Body="&lt;p&gt;I coded a tic tac toe program, but I don't know if I can call it artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what I did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a random player, which always makes random valid moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then there is the AI player, which will receive input before every move, that input is the state of the board, and all the posibilities.&#xA;The AI, will try any move that it hasn't tried before. But if it knows every possibility, it will select the one that has the higher value.&#xA;This value is assigned by the outcome of the match, if the match was won ,+1, 0 for draw, -1 for lose.&#xA;Every move, will be stored in a database, or updated if it's known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually it will know every possible move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also added a threshold to compare the best moves, so it really select the best move. Example, two moves with a value of 100, the AI will keep trying them both, randomly until one has surpased the other by the threshold, say 50.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It takes about 20.000 games to make the AI perfect, it never loses a game, just draws and wins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new to AI, and I'm wondering, could this be really considered Artificial Intelligence?&#xA;And how is this different from a Neural Network approach? (I'm been reading about it, but I still don't quite get it.)&lt;/p&gt;&#xA;" OwnerUserId="23389" LastActivityDate="2019-04-27T11:51:59.593" Title="Can this tic tac toe program be considered AI?" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11610" PostTypeId="1" CreationDate="2019-04-02T22:28:55.777" Score="1" ViewCount="42" Body="&lt;p&gt;Now I'm using tensorflow.keras to implement the &lt;code&gt;FCN-16s&lt;/code&gt;,&lt;a href=&quot;https://i.stack.imgur.com/HF6Pu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HF6Pu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;this picture may be different with others, you should focus this it add &lt;code&gt;pool4&lt;/code&gt; to &lt;code&gt;2x conv7&lt;/code&gt; not &lt;code&gt;2x pool5&lt;/code&gt; in &lt;code&gt;FCN-16s&lt;/code&gt;. Actually the &lt;code&gt;conv7's&lt;/code&gt; output_shape is &lt;code&gt;(w/32,h/32,classes_number)&lt;/code&gt; in my implementation. I know &lt;code&gt;conv7&lt;/code&gt; should multiply 2 to achieve the same shape (1st,2nd) with &lt;code&gt;pool4&lt;/code&gt;. But my question is the &lt;code&gt;pool4&lt;/code&gt;'s shape is &lt;code&gt;(w/16,h/16,512)&lt;/code&gt;, ps 512 is the output_channel of the last conv-layer. &lt;code&gt;pool4&lt;/code&gt; and &lt;code&gt;2x conv7&lt;/code&gt; have not same shape(3rd), how to add?&lt;/p&gt;&#xA;&lt;p&gt;And another question is what are the train-labels for the FCN, I think a train-label is just a image which has the same width and same height with input_image and has classes_number channel to show each pixel's possibility belong to each class. This idea is derived from the conv7's output_shape, is it right? I hope you can give me more and clear explanations. Thanks!!!&lt;/p&gt;&#xA;" OwnerUserId="23656" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-02T22:28:55.777" Title="how to add the pool4 to the 2 x conv7 in FCN-16s using keras?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11611" PostTypeId="2" ParentId="11609" CreationDate="2019-04-03T01:27:49.797" Score="7" Body="&lt;p&gt;This is basically reinforcement learning. The state space contains your moves, and the value function are the value you store at the end. And your rewards are the end results. And you have episodic game. It is an AI method. Consider looking at value iteration, policy iteration, SARSA, Q-learning. The difference between neural network method and yours is you are not doing function approximation with neural network for the value function, you are doing tabular method.&lt;/p&gt;&#xA;" OwnerUserId="4042" LastActivityDate="2019-04-03T01:27:49.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11612" PostTypeId="1" AcceptedAnswerId="11615" CreationDate="2019-04-03T02:40:29.227" Score="6" ViewCount="2286" Body="&lt;p&gt;Can Q-learning (and SARSA) be directly used in a Partially Observable Markov Decision Process (POMDP)? If not, why not? My intuition is that the policies learned will be terrible because of partial observability. Are there ways to transform these algorithms so that they can be easily used in a POMDP?&lt;/p&gt;&#xA;" OwnerUserId="4042" LastEditorUserId="12509" LastEditDate="2019-04-03T11:39:05.597" LastActivityDate="2019-04-03T11:43:06.913" Title="Can Q-learning be used in a POMDP?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;pomdp&gt;&lt;markov-decision-process&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11613" PostTypeId="1" CreationDate="2019-04-03T02:45:21.020" Score="2" ViewCount="230" Body="&lt;p&gt;I am struggling to understand the use of the Convolutional Sequence to Sequence (Conv-Seq2Seq) model.  The image below is take directly from the paper and is the nearly canonical diagram of the parallel training procedure.  After puzzling over it for quite some time, it has come to seem straight forward to me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An input sentence of N tokens can be encoded in one step because the input sentence exists prior to the start of training, and therefore the token-wise convolution can be trivially parallelized. (Compare to RNN encoders, which require N steps)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;During training&lt;/strong&gt;, an output sentence can similarly be parallelized in the decoder because &lt;strong&gt;during training&lt;/strong&gt;, the entire output sentence is known.&lt;/li&gt;&#xA;&lt;li&gt;Therefore, &lt;strong&gt;during training&lt;/strong&gt;, the attention function can be fully parallelized in the two dimensional array of dot products shown below&lt;/li&gt;&#xA;&lt;li&gt;Finally, &lt;strong&gt;during training&lt;/strong&gt;, the attention is used to weight the input embeddings and encodings, combined with the output training encodings (as such) and the final output assembled. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is clearly not the case after the network is trained and evaluation input sequences are translated without reference outputs.  I understand from various resources (including but not limited to Gehring's conference presentation) that post-training, output sequences are generated token by token in a fashion vaguely similar to earlier architectures, but I cannot find a clear description of that process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I speculate that this is because the parallel training routine was so revolutionary at the time, that the focus of the publications was rightly on the training routines.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please help me understand the post-training generation algorithm, &lt;em&gt;if possible in terms of the training diagram?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current non-confident understanding is that the sentence below would be handled something like the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Prime the decoder with a default token string of &lt;code&gt;&amp;lt;p&amp;gt; &amp;lt;p&amp;gt; &amp;lt;s&amp;gt;&lt;/code&gt;, this results (due to convolution) in a single decoder encoding (as such) input into the attention function, and would hopefully generate the single token &lt;code&gt;&amp;lt;'Sie'&amp;gt;&lt;/code&gt; as output&lt;/li&gt;&#xA;&lt;li&gt;Restart the decoder with the token string &lt;code&gt;&amp;lt;p&amp;gt; &amp;lt;p&amp;gt; &amp;lt;s&amp;gt; &amp;lt;'Sie'&amp;gt;&lt;/code&gt; which would generate two inputs into the attention function and hopefully output &lt;code&gt;&amp;lt;'Sie'&amp;gt; &amp;lt;'stimmen'&amp;gt;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Proceed with lengthening input sentences until the final token generated output ends with a &lt;code&gt;&amp;lt;/s&amp;gt;&lt;/code&gt; token signifying the end of the sentence. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If that is a correct understanding, can someone confirm it?  If close, can someone correct me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eIVS1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eIVS1.png&quot; alt=&quot;Fig 1, Gehring, et al&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1705.03122.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Sequence to Sequence Learning&lt;/a&gt;, Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin, 2017&lt;/p&gt;&#xA;" OwnerUserId="15020" LastActivityDate="2021-04-14T15:47:04.483" Title="Convolutional Sequence to Sequence Learning: Training vs Generation" Tags="&lt;convolutional-neural-networks&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11615" PostTypeId="2" ParentId="11612" CreationDate="2019-04-03T09:02:08.157" Score="3" Body="&lt;p&gt;The usual (as presented in &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2018mar21.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;) &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning and SARSA algorithms use (and update) a function of a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;. These algorithms assume that the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is known. However, in POMDP, at each time step, the agent does not know the current state, but it maintains a &quot;belief&quot; (which, mathematically, is represented as a probability distribution) in what the current state might be, so it cannot maintain (an approximation of) the function &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;. Hence, the usual Q-learning and SARSA algorithms shouldn't be directly applicable to a POMDP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is often used in the contexts where observations emitted by the environment (or transformations of the raw observations) are used to build the current state (which is assumed to be Markov, even if it is not). For example, in the &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original DQN&lt;/a&gt;, the action taken at the current step and the raw observation and the reward emitted by the environment (after this action is taken) are combined to produce the current (Markov) state. It might not be the case that the way they combine the action, the reward and the observation is sufficient to fully describe the current state (which might not even be Markov).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this report, &lt;a href=&quot;http://cs229.stanford.edu/proj2015/363_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning with POMDPs&lt;/a&gt;, the author attempts to use Q-learning in a POMDP setting. He suggests to represent a function, either &lt;span class=&quot;math-container&quot;&gt;$Q(b, a)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$Q(h, a)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the &quot;belief&quot; over the states and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the history of previously executed actions, using neural networks. So, the resulting parameterized functions would be denoted by &lt;span class=&quot;math-container&quot;&gt;$Q(b, a; \theta)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$Q(h, a; \theta)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is a vector representing the parameters of the corresponding neural network. Essentially, the author uses a DQN (with an &lt;em&gt;experience replay&lt;/em&gt; buffer and &lt;em&gt;target network&lt;/em&gt;), but the results are not great: &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values converge, but policies do not and they are not robust (in that they are sensitive to small perturbations).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-03T11:43:06.913" LastActivityDate="2019-04-03T11:43:06.913" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11617" PostTypeId="1" CreationDate="2019-04-03T09:55:20.223" Score="1" ViewCount="702" Body="&lt;p&gt;I will explain my question in relation to chess, but it should be relevant for other games as well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short terms: Is it possible to combine the techniques used by AlphaZero with those used by, say, Stockfish? And if so, has it been attempted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have only a brief knowledge about how AlphaZero works, but from what I've understood, it basically takes the board state as input to a neural net, possibly combined with monte carlo methods, and outputs a board evaluation or prefered move. To me, this really resembles the heuristic function used by traditional chess engines like stockfish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from this I will conclude (correct me if I'm wrong) that AlphaZero evaluates the current position, but uses a very powerful heuristic. Stockfish on the other hand searches through lots of positions from the current one first, and then uses a less powerful heuristic when a certain depth is reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it therefore possible to combine these approaches by first using alpha-beta pruning, and then using AlphaZero as some kind of heuristic when the max depth is reached? To me it seems like this would be better than just evaluating the current position like (I think) AlphaZero does. Will it take too much time to evaluate? Or is it something I have misunderstood? If it's possible, has anyone attempted it? &lt;/p&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2020-05-22T22:55:43.980" Title="Combining deep reinforcement learning with alpha-beta pruning" Tags="&lt;deep-learning&gt;&lt;chess&gt;&lt;alphazero&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11618" PostTypeId="1" CreationDate="2019-04-03T10:36:45.377" Score="1" ViewCount="36" Body="&lt;p&gt;&lt;strong&gt;TLTR:&lt;/strong&gt;&#xA;I'm developing a CNN for a classification task. The data contains multiple classes some of which are very similar to each other and I know these meta-classes. In such a situation is it a good approach to use 2 Levels of CNNs:&#xA;1. Level detect the meta-classes.&#xA;2. Level detect the classes within the classified meta class (of Level 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&#xA;&lt;strong&gt;Example:&lt;/strong&gt;&#xA;Suppose I try to classify the following 9 classes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apple Tree, Plum Tree, Cherry Tree, Sports car, SUV car, Coupe car, Dog, Cat, Wolf&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I could of course use one network on these classes and get a classification output for all of them. But the output (softmax) percentage e.g. for an apple tree would be for probably high for any tree class.&#xA;Thus is it a good approach to train and use 2 level of CNNs, like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Level classify tree, car, animal&#xA;--&gt; Trained with all images&lt;/li&gt;&#xA;&lt;li&gt;Level classify what kind of tree, car, animal --&gt; trained only with the subsample of trees, cars, animals&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So images are checked by CNN Level 1 and then depending on its classification with appropriate CNN Level 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this a good approach ?&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it help in terms of prediction quality/accuracy of the subclasses?&lt;/li&gt;&#xA;&lt;li&gt;Is it easier for an CNN to detect the specific features of a subclass if the input is limited (like in level 2) ?&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or use another approach ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&#xA;Swad&lt;/p&gt;&#xA;" OwnerUserId="23665" LastEditorUserId="23665" LastEditDate="2019-04-03T13:05:11.527" LastActivityDate="2019-04-03T13:05:11.527" Title="Classification of classes within meta-classes" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11619" PostTypeId="5" CreationDate="2019-04-03T12:03:10.140" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Markov_decision_process&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-01T19:38:26.087" LastActivityDate="2019-09-01T19:38:26.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11620" PostTypeId="4" CreationDate="2019-04-03T12:03:10.140" Score="0" Body="For questions related to the Markov Decision Process (MDP), which is a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision-maker." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-01T19:38:26.087" LastActivityDate="2019-09-01T19:38:26.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11621" PostTypeId="1" CreationDate="2019-04-03T13:15:15.903" Score="2" ViewCount="43" Body="&lt;p&gt;I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across language models &lt;a href=&quot;https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT&lt;/a&gt;, GPT-2 and &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt;. I learned that they can be used for generation purposes. But I did not find anywhere whether they can generate sentences given only a word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am inclined to use GPT for my task, but I am not sure how to proceed with it. I wanted to know whether it is possible or not? It would be helpful if anyone can help me give a start in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="19244" LastEditorUserId="2444" LastEditDate="2019-11-01T02:58:47.083" LastActivityDate="2019-11-01T02:58:47.083" Title="How can I generate a document from a single word using GPT or BERT?" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;&lt;language-model&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11622" PostTypeId="1" CreationDate="2019-04-03T13:28:05.830" Score="2" ViewCount="33" Body="&lt;p&gt;I have a small dataset (117 training examples) and many features (4005). Each of the training examples is binary labeled (healthy / diseased). Each feature represents the connectivity between two different brain regions.&lt;br&gt;&#xA;The goal is to assign subjects to one of the two groups based on their brain activity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What methods are there for generating new artificial training examples based on the existing training examples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example I could think of would be &lt;a href=&quot;http://rikunert.com/SMOTE_explained&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMOTE&lt;/a&gt;. However, this technique is usually only used to balance unbalanced datasets. This would not be necessary for my set, since it has about the same number of training examples for both label classes.&lt;/p&gt;&#xA;" OwnerUserId="23672" LastActivityDate="2019-04-03T13:28:05.830" Title="What methods are there to generate artificial training examples based on existing training examples?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11623" PostTypeId="1" CreationDate="2019-04-03T13:57:01.983" Score="3" ViewCount="125" Body="&lt;p&gt;The Turing award is sometimes called Computer Sceince's Nobel Prize. &lt;a href=&quot;https://amturing.acm.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This year's award&lt;/a&gt; goes to Bengio, Hinton, and LeCun for their work on artificial neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual work contributed by these authors is, of course, quite technical. It centers around the development of deep neural networks, convolutional neural networks, and effective training techniques. The lay press will tend to simplify these results to the point that they lose meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have a concise, and yet still precise, explanation of their contributions to share with a lay audience. So, what is a simplified way to explain the contributions of these researchers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have my own ideas and will add them if no other satisfactory answer appears. For a &quot;lay&quot; audience, I want to assume someone who had taken a college level course in something scientific but not necessarily computer science. Explanations that are suitable for those with even less background are better still though, as long as they don't lose too much precision.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="22365" LastEditDate="2019-04-03T18:15:39.653" LastActivityDate="2019-04-03T18:15:39.653" Title="What is a simplified way to explain why the AI researchers Bengio, Hinton, and Lecun, won the 2019 Turing Award?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;soft-question&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11624" PostTypeId="2" ParentId="11623" CreationDate="2019-04-03T14:38:57.120" Score="6" Body="&lt;p&gt;The &lt;a href=&quot;https://www.acm.org/media-center/2019/march/turing-award-2018&quot; rel=&quot;noreferrer&quot;&gt;related ACM article&lt;/a&gt; describes a few specific technical contributions, which led the ACM to award them.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h3&gt;Geoffrey Hinton&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;Backpropagation&lt;/strong&gt;: In a 1986 paper, &amp;quot;Learning Internal Representations by Error Propagation&amp;quot;, co-authored with David Rumelhart and Ronald Williams, Hinton demonstrated that the backpropagation algorithm allowed neural nets to discover their own internal representations of data, making it possible to use neural nets to solve problems that had previously been thought to be beyond their reach. The backpropagation algorithm is standard in most neural networks today.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Boltzmann Machines&lt;/strong&gt;: In 1983, with Terrence Sejnowski, Hinton invented Boltzmann Machines, one of the first neural networks capable of learning internal representations in neurons that were not part of the input or output.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Improvements to convolutional neural networks&lt;/strong&gt;: In 2012, with his students, Alex Krizhevsky and Ilya Sutskever, Hinton improved convolutional neural networks using rectified linear neurons and dropout regularization. In the prominent ImageNet competition, Hinton and his students almost halved the error rate for object recognition and reshaped the computer vision field.&lt;/p&gt;&#xA;&lt;h3&gt;Yoshua Bengio&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;Probabilistic models of sequences&lt;/strong&gt;: In the 1990s, Bengio combined neural networks with probabilistic models of sequences, such as hidden Markov models. These ideas were incorporated into a system used by AT&amp;amp;T/NCR for reading handwritten checks, were considered a pinnacle of neural network research in the 1990s, and modern deep learning speech recognition systems are extending these concepts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;High-dimensional word embeddings and attention&lt;/strong&gt;: In 2000, Bengio authored the landmark paper, &amp;quot;A Neural Probabilistic Language Model&amp;quot;, that introduced high-dimension word embeddings as a representation of word meaning. Bengio's insights had a huge and lasting impact on natural language processing tasks including language translation, question answering, and visual question answering. His group also introduced a form of attention mechanism which led to breakthroughs in machine translation and form a key component of sequential processing with deep learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Generative adversarial networks&lt;/strong&gt;: Since 2010, Bengio's papers on generative deep learning, in particular the Generative Adversarial Networks (GANs) developed with Ian Goodfellow, have spawned a revolution in computer vision and computer graphics. In one fascinating application of this work, computers can actually create original images, reminiscent of the creativity that is considered a hallmark of human intelligence.&lt;/p&gt;&#xA;&lt;h3&gt;Yann LeCun&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;Convolutional neural networks&lt;/strong&gt;: In the 1980s, LeCun developed convolutional neural networks, a foundational principle in the field, which, among other advantages, have been essential in making deep learning more efficient. In the late 1980s, while working at the University of Toronto and Bell Labs, LeCun was the first to train a convolutional neural network system on images of handwritten digits. Today, convolutional neural networks are an industry standard in computer vision, as well as in speech recognition, speech synthesis, image synthesis, and natural language processing. They are used in a wide variety of applications, including autonomous driving, medical image analysis, voice-activated assistants, and information filtering.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Improving backpropagation algorithms&lt;/strong&gt;: LeCun proposed an early version of the backpropagation algorithm (backprop), and gave a clean derivation of it based on variational principles. His work to speed up backpropagation algorithms included describing two simple methods to accelerate learning time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Broadening the vision of neural networks&lt;/strong&gt;: LeCun is also credited with developing a broader vision for neural networks as a computational model for a wide range of tasks, introducing in early work a number of concepts now fundamental in AI. For example, in the context of recognizing images, he studied how hierarchical feature representation can be learned in neural networks - a concept that is now routinely used in many recognition tasks. Together with Léon Bottou, he proposed the idea, used in every modern deep learning software, that learning systems can be built as complex networks of modules where backpropagation is performed through automatic differentiation. They also proposed deep learning architectures that can manipulate structured data, such as graphs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-03T14:38:57.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11625" PostTypeId="2" ParentId="11363" CreationDate="2019-04-03T15:16:10.700" Score="1" Body="&lt;p&gt;Some of the cases content-based filtering is useful is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Cold-start problem&lt;/strong&gt;: it happens when no previous information about user history is available to build collaborative filtering, so in this case, we offer to the user some items then recommend based on the similarity between these items and other items in the dataset alternate of recommending any items that maybe not with the user taste. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Transparency&lt;/strong&gt;: collaborative method gives you the recommendation because some unknown users have the same taste as you that cause a problem if your data is biased towards one taste makes new user haven't enough similar users have the same taste, but the content-based method can tell you they recommend the items based on what features, that helps you to determine which factors affect in the recommendation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21907" LastActivityDate="2019-04-03T15:16:10.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11626" PostTypeId="1" CreationDate="2019-04-03T15:42:04.977" Score="4" ViewCount="81" Body="&lt;p&gt;It seems that stacking LSTM layers &lt;a href=&quot;https://machinelearningmastery.com/stacked-long-short-term-memory-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be beneficial for some problem settings&lt;/a&gt; in order to learn higher levels of abstraction of temporal relationships in the data. There is already &lt;a href=&quot;https://ai.stackexchange.com/questions/3156/how-to-select-number-of-hidden-layers-and-number-of-memory-cells-in-an-lstm&quot;&gt;some discussion&lt;/a&gt; on selecting the number of hidden layers and number of cells per layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; Is there any guidance for the relative number of cells from one LSTM layer to a subsequent LSTM layer in the stack? I am specifically interested in problems involving timeseries forecasting (given a stretch of temporal data, predict the trend of that data over some time window into the future), but I'd also be curious to know for other problem settings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say I am stacking 3 LSTM layers on top of each other: LSTM1, LSTM2, LSTM3, where LSTM1 is closer to the input and LSTM3 is closer to the output. Are any of the following relationships expected to improve performance?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;num_cells(LSTM1) &gt; num_cells(LSTM2) &gt; num_cells(LSTM3) &lt;strong&gt;[Sizes decrease input to output]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;num_cells(LSTM1) &amp;lt; num_cells(LSTM2) &amp;lt; num_cells(LSTM3) &lt;strong&gt;[Sizes increase input to output]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;num_cells(LSTM1) &amp;lt; num_cells(LSTM2) &gt; num_cells(LSTM3) &lt;strong&gt;[Middle layer is largest]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Obviously there are other combinations, but those seem to me salient patterns. I know the answer is probably &quot;it depends on your problem, there is no general guidance&quot;, but I'm looking for some indication of what kind of behavior I could expect from these different configurations.&lt;/p&gt;&#xA;" OwnerUserId="20955" LastActivityDate="2019-04-03T15:42:04.977" Title="How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11627" PostTypeId="1" CreationDate="2019-04-03T18:50:29.097" Score="6" ViewCount="2306" Body="&lt;p&gt;In my thesis I dealt with the question how a computer can recognize LEGO bricks. With multiple object detection, I chose a deep learning approach. I also looked at an existing training set of LEGO brick images and tried to optimize it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using Tensorflow's Object Detection API on a dataset of specifically generated images (Created with Blender) I was able to detect 73.3% of multiple LEGO Bricks in one Foto. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the main problems I noticed was, that I tried to distinguish three different 2x4 bricks. However, colors are difficult to distinguish, especially in different lighting conditions. A better approach would have been to distinguish a 2x4 from a 2x2 and a 2x6 LEGO brick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I have noticed that the training set should best consist of &quot;normal&quot; and synthetically generated images. The synthetic images give variations in the lighting conditions, the backgrounds, etc., which the photographed images do not give. However, when using the trained Neural Network, photos and not synthetic images are examined. Therefore, photos should also be included in the training data set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last point that would probably lead to even better results is that you train the Neural Network with pictures that show more than one LEGO brick. Because this is exactly what is required by the Neural Network when it is in use. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Are there other ways I could improve upon this?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Can you see any further potential for improvement for the Neural Network? How would you approach the issue? Do any of my approaches seem poor? How do you solve the problem?)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23682" LastEditorUserId="1671" LastEditDate="2019-04-04T20:42:22.207" LastActivityDate="2020-10-03T12:56:56.950" Title="How to detect LEGO bricks by using a deep learning approach?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;tensorflow&gt;&lt;datasets&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11629" PostTypeId="1" AcceptedAnswerId="11630" CreationDate="2019-04-03T23:36:00.570" Score="2" ViewCount="1020" Body="&lt;p&gt;I am new to Deep Learning.&lt;/p&gt;&#xA;&lt;p&gt;Suppose that we have a neural network with one input layer, one output layer, and one hidden layer. Let's refer to the weights from input to hidden as &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; and the weights from hidden to output as &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;. Suppose that we have initialized &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, and ran them through the neural network via the forward algorithm/pass. Suppose that we have updated &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; via backpropagation.&lt;/p&gt;&#xA;&lt;p&gt;When estimating the ideal weights for &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, do we keep the weights &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; constant when updating &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; via gradient descent given we already calculated &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, or do we allow &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; to update along with &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;So, in the code, which I am trying to do from scratch, do we include &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; in the for loop that will be used for gradient descent to find &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;? In other words, do we simply use the same &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; for every iteration of gradient descent?&lt;/p&gt;&#xA;" OwnerUserId="23687" LastEditorUserId="2444" LastEditDate="2020-11-15T12:45:18.940" LastActivityDate="2020-11-15T12:45:18.940" Title="Does backpropagation update weights one layer at a time?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11630" PostTypeId="2" ParentId="11629" CreationDate="2019-04-04T00:15:33.513" Score="1" Body="&lt;p&gt;The answer is implied in the term &quot;backpropagation&quot;. All gradients are calculated at the same time. That is, the error from your loss function is propagated backwards from the output and through your whole network. This propagation results in an error associated with each weight in the network, which determines how you change each weight in order to minimize your loss function.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;or do we allow &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; to update along with &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. This saves time, since many of the results of intermediate computations used to update &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; can be reused in the update of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://neuralnetworksanddeeplearning.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://neuralnetworksanddeeplearning.com&lt;/a&gt; for a detailed description of the backpropagation algorithm.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-04T01:18:07.873" LastActivityDate="2019-04-04T01:18:07.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11631" PostTypeId="2" ParentId="11627" CreationDate="2019-04-04T02:16:26.867" Score="5" Body="&lt;p&gt;So I am assuming that you are trying to &lt;em&gt;detect&lt;/em&gt; a lego brick from the image. One idea is that you can use transfer learning. Leveraging a pre-trained machine learning model is called transfer learning. The underlying idea behind transfer learning is that one takes a well-trained model from one dataset or domain, and applies it to a new one.  François Chollet has written a very comprehensive guide to transfer learning (&lt;a href=&quot;https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html&quot; rel=&quot;noreferrer&quot;&gt;https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I admit I took some of this information from Christopher Bonnett's article named Classifying e-commerce products based on images and text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also suggest using the Lego brick dataset from Kaggle on this link: &lt;a href=&quot;https://www.kaggle.com/joosthazelzet/lego-brick-images&quot; rel=&quot;noreferrer&quot;&gt;https://www.kaggle.com/joosthazelzet/lego-brick-images&lt;/a&gt;&#xA;It has over 12,700 lego brick images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If processing power is a problem, you can use Amazon Web Services for cloud computing. It is inexpensive for small scale operations like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course for the object detection, you can always increase the number of convolution layers. However, if you have too many layers, you should also include residual blocks/residual network. This would allow neural networks even with over a thousand layers to operate effectively. This video should help you understand how residual networks work (&lt;a href=&quot;https://www.youtube.com/watch?v=ahkBkIGdnWQ&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=ahkBkIGdnWQ&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, make sure not to overfit during your training and if you do follow the residual network idea, you should also include upsampling in your convolution neural network( More in here: &lt;a href=&quot;https://towardsdatascience.com/up-sampling-with-transposed-convolution-9ae4f2df52d0&quot; rel=&quot;noreferrer&quot;&gt;https://towardsdatascience.com/up-sampling-with-transposed-convolution-9ae4f2df52d0&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helped and good luck on your endeavor. &lt;/p&gt;&#xA;" OwnerUserId="23546" LastActivityDate="2019-04-04T02:16:26.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11633" PostTypeId="1" CreationDate="2019-04-04T04:02:07.013" Score="3" ViewCount="41" Body="&lt;p&gt;In Section 10.4 of Sutton and Barto's &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL book&lt;/a&gt;, they argue that the discount rate &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; has no effect in continuing settings. They show (at least for one objective function) that the average of the discounted return is proportional to the undiscounted average reward &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt; under the given policy.&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt; They then advocate using average rewards rather than the usual returns of the discounted setting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never encountered someone using average rewards (and no discounting) in the wild, though. Am I just ignorant of some use case, or is pretty much everyone sticking to discounting anyways?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(\pi)=\sum_s \mu_\pi (s) \sum_a \pi(a|s) \sum_{s',r}p(s',r|s,a)r$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mu_\pi$&lt;/span&gt; is the stationary state distribution while following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt;Their proof did use the fact that the MDP was ergodic. I'm not sure how often that assumption holds in practice.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-04-04T04:02:07.013" Title="Does everyone still use discount rates?" Tags="&lt;reinforcement-learning&gt;&lt;discount-factor&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11634" PostTypeId="1" CreationDate="2019-04-04T04:19:09.840" Score="1" ViewCount="35" Body="&lt;p&gt;Is any classifier not subject to fooling as in &lt;a href=&quot;http://www.evolvingai.org/fooling&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree that the question is related to the other one as commented by Philip. But I guess it is not completely a duplication as pointed out by hisairnessag3. What I wanted to ask is that any classifiers inherently do not subject (or less prone) to attack. I have a feeling that non-linear classifiers should be less susceptible to attack. Btw, any benchmark on say, simple K-nearest neighbor classifiers is available?&lt;/p&gt;&#xA;" OwnerUserId="23688" LastEditorUserId="23688" LastEditDate="2019-04-04T20:07:12.727" LastActivityDate="2019-05-05T00:01:26.280" Title="Is any classifier not subject (or less susceptible) to fooling?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11636" PostTypeId="1" CreationDate="2019-04-04T04:48:12.557" Score="2" ViewCount="132" Body="&lt;p&gt;I am working with a dataset where each input sample is a matrix, and the output corresponding to each input is also a matrix (of shape &lt;code&gt;(400, 10)&lt;/code&gt;). The &lt;strong&gt;input samples do not have translation invariance&lt;/strong&gt;. Each output image has shape &lt;code&gt;(16, 16)&lt;/code&gt; . The output matrices have translation invariance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I want to build a neural network which can learn how to predict the output images&lt;/strong&gt; from the aforementioned data. It seems to me that one needs to think of a &lt;strong&gt;neural network here which does regression on the output images&lt;/strong&gt; to learn. Presently, I am using &lt;strong&gt;1000 data samples for learning in the neural network&lt;/strong&gt; (input samples and corresponding output images). &lt;strong&gt;What is the best way to build a neural network for this task?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently, I am using multi-layer perceptron (MLP) with mean square error (MSE) loss for this task. I flatten the input matrices before I feed them into the MLP, and use a standard MLP with multiple hidden layers (4-5) with many hidden units for this task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While a visual inspection shows that the true and predicted output images are in relatively good agreement for training data, I find a &lt;strong&gt;mismatch between true and predicted (reconstructed) output images for validation data&lt;/strong&gt;. The bottom plot shows pictures of true and predicted (reconstructed) output images for training and validation data for chosen samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ns9FP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ns9FP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently, I am using training and validation loss curves (with respect to iteration) to measure performance. I want to have a robust metric for comparison which can tell me whether the prediction is a random image or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I get the model to generalize to the validation set better?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Python code that I am using for this MLP and the required data can be found &lt;a href=&quot;https://drive.google.com/file/d/1_Ls2Q6m-Df96JjhumHchy25kpddfCZSt/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://drive.google.com/file/d/1YaHmP-amz14Ev4nh5rsPi4l7XDDGq3RL/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; respectively.&lt;/p&gt;&#xA;" OwnerUserId="22566" LastEditorUserId="2444" LastEditDate="2019-04-04T10:14:23.660" LastActivityDate="2019-04-04T10:14:23.660" Title="How to build a neural network that can learn to predict output images?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11637" PostTypeId="1" CreationDate="2019-04-04T08:12:22.690" Score="1" ViewCount="48" Body="&lt;p&gt;IQN paper (&lt;a href=&quot;https://arxiv.org/abs/1806.06923&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1806.06923&lt;/a&gt;) uses distributional bellman target:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \delta^{\tau,\tau'}_t = r_t + \gamma Z_{\tau'}(x_{t+1}, \pi_{\beta}(x_{t+1})) - Z_{\tau}(x_t, a_t) $$&lt;/span&gt;&#xA;And optimizes: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ L = \frac{1}{N'} \sum^{N}_i \sum^{N'}_j \rho^\kappa_{\tau_i} \delta^{\tau_i,\tau_j}_t $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But similar quantiles can be got just from Q values, when doing so:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \delta^\tau_t = r_t + \gamma \frac{1}{N'} \sum_{j}^{N'} Z_{\tau_j}(x_{t+1}, \pi_{\beta}(x_{t+1})) - Z_\tau(x_t, a_t) \\ = r_t + \gamma Q (x_{t+1}, \pi_\beta(x_{t+1})) - Z_\tau(x_t, a_t) $$&lt;/span&gt;&#xA;optimizing:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ L = \sum^N_i \rho^{\kappa}_{\tau_i} \delta^{\tau_i}_t $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both lead to similar performance on CartPole env. The loss function of the 2nd one is more simpler and intuitive (atleast to me). So i was thinking if there are any obvious reason why authors didin't use it?&lt;/p&gt;&#xA;" OwnerUserId="18808" LastEditorUserId="18808" LastEditDate="2019-04-04T10:40:30.217" LastActivityDate="2019-04-04T11:27:03.897" Title="IQN bellman target: using Z vs using Q" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;atari-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11638" PostTypeId="2" ParentId="11637" CreationDate="2019-04-04T11:27:03.897" Score="1" Body="&lt;p&gt;Replacement you suggest is replacement of random variable by its expectation in forward part of TD. It would make IQN into modification of &lt;a href=&quot;https://arxiv.org/abs/1707.06887&quot; rel=&quot;nofollow noreferrer&quot;&gt;C51&lt;/a&gt; with randomly sampled function approximator instead of discrete distribution. Both distribution produced and especially exploration behavior with your replacement would be very different. The authors of paper explicitly said that &quot;more randomness&quot; in their opinion benefit training, so reducing randomness would go aginst spirit of the paiper.  That they produce similar results on single toy test mean very little. IQN could be better then C51 or it could be worse then C51 but single toy example is not enough to say they are close. Nevertheless I agree that IQN looks overly complex and may require more training time, C51 approach could be more practical. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-04-04T11:27:03.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11639" PostTypeId="1" CreationDate="2019-04-04T14:22:51.743" Score="2" ViewCount="36" Body="&lt;p&gt;Policy learning refers to mapping an agent state onto an action to maximize reward. A linear policy, such as the one used in the &lt;a href=&quot;https://arxiv.org/abs/1803.07055&quot; rel=&quot;nofollow noreferrer&quot;&gt;Augmented Random Search paper&lt;/a&gt;, refers to learning a linear mapping between state and reward.&lt;/p&gt;&#xA;&lt;p&gt;When the entire state changes at each time-step, for example in the &lt;a href=&quot;https://gym.openai.com/envs/MountainCarContinuous-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continuous Mountain Car OpenAI Gym&lt;/a&gt;, the position and speed of the car changes at each time-step.&lt;/p&gt;&#xA;&lt;p&gt;However, assume we also wanted to communicate the constant position of one or more goals. By &amp;quot;constant&amp;quot;, I mean does not change within a training episode, but may change between episodes. For example, if there was a goal on the left &lt;em&gt;and&lt;/em&gt; right of the Mountain Car.&lt;/p&gt;&#xA;&lt;p&gt;Are there examples of how this constant/static information be communicated from the environment other than appending the location of the two goals to the state vector? Can static/constant state be differentiated from state which changes with each action?&lt;/p&gt;&#xA;" OwnerUserId="23703" LastEditorUserId="2444" LastEditDate="2020-11-01T16:55:45.640" LastActivityDate="2020-11-01T16:55:45.640" Title="How can certain information about the goal be given to the RL learning algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11640" PostTypeId="1" CreationDate="2019-04-04T14:40:34.553" Score="6" ViewCount="4653" Body="&lt;p&gt;I'm learning DDPG algorithm by following the following link: &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html#the-policy-learning-side-of-ddpg&quot; rel=&quot;noreferrer&quot;&gt;Open AI Spinning Up document on DDPG&lt;/a&gt;, where it is written&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In order for the algorithm to have stable behavior, the replay buffer should be large enough to contain a wide range of experiences, but it may not always be good to keep everything. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does this mean? Is it related to the tuning of the parameter of the batch size in the algorithm?&lt;/p&gt;&#xA;" OwnerUserId="23707" LastEditorUserId="2444" LastEditDate="2019-04-04T18:59:24.590" LastActivityDate="2020-10-16T23:28:42.070" Title="How large should the replay buffer be?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;experience-replay&gt;&lt;ddpg&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11642" PostTypeId="2" ParentId="11639" CreationDate="2019-04-04T15:16:02.103" Score="4" Body="&lt;p&gt;I see three main ways to do this, which one makes more sense will depend on your application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One is to append that information to the state/observations like you mentioned. While this information is static for a particular episode, it will be different across episodes and the policy should learn to condition the actions it chooses based on this information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another would be to leave goal information out entirely and force the agent to learn a policy that works when the goal is unknown. This will likely be more difficult to learn and you may end up with a policy that moves to the average of all goals or explores and tries them all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third option and probably the most natural is to have some context cue that the agent can observe (e.g. often in lab experiments with rats, a cue is placed on a wall that lets the rat know which way to go to get a reward). This is similar to the first method, except that the cue has to be observed rather than given directly. For the Mountain Car example, this could be an extra signal that the agent only sees in a particular location, such as the bottom of the valley or when it moves close to a particular side.&lt;/p&gt;&#xA;" OwnerUserId="23710" LastActivityDate="2019-04-04T15:16:02.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11643" PostTypeId="1" AcceptedAnswerId="11653" CreationDate="2019-04-04T15:26:30.523" Score="1" ViewCount="647" Body="&lt;p&gt;I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the forward implementation of the flatten layer, but is the backward just reshaping it or not? If yes, can I just call a simple NumPy's &lt;code&gt;reshape&lt;/code&gt; function to reshape it?&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="2444" LastEditDate="2020-12-16T14:54:51.507" LastActivityDate="2020-12-16T14:54:51.507" Title="How should I implement the backward pass through a flatten layer of a CNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11644" PostTypeId="2" ParentId="6579" CreationDate="2019-04-04T15:41:13.910" Score="3" Body="&lt;p&gt;In reinforcement learning (RL), an &lt;em&gt;agent&lt;/em&gt; interacts with an &lt;em&gt;environment&lt;/em&gt; in time steps. At each time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent and the environment are in some state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;. From that state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, the agent chooses and executes an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and the environment emits a reward &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; (which values the just taken action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;). Finally, the agent and the environment move to the next state, &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt;. This interaction proceeds until either the agent dies or some other termination criterion is met. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the agent is to obtain the highest amount of reward in the long run (that is, not just in the next time step, but in all successive time steps). To do that, ideally, the agent needs to find a way of behaving &quot;optimally&quot;. In RL, the behaviour of the agent is called a &quot;policy&quot;. An optimal policy is a policy that allows the agent to obtain the (expected) highest amount of reward in the long run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, we can describe a full and finite (in terms of time steps) interaction between an agent and an environment as a sequence (sometimes called a &quot;rollout&quot; or a &quot;trajectory&quot;) of states, actions, rewards and next states. So, a rollout might look like this &lt;span class=&quot;math-container&quot;&gt;$$(s_t, a_t, r_t, s_{t+1}, a_{t+1}, r_{t+1}, s_{t+2}, \dots, s_{T-1}, a_{T-1}, r_{T-1}, a_{T}),$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is the last time step of the interaction between the agent and the environment. During the interaction between the agent and the environment, the agent might decide to store this &quot;experience&quot; in a &quot;buffer&quot; (e.g. an array), so that it can use it later (you will see below a use case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The elements of this type of sequences are often temporally &lt;em&gt;correlated&lt;/em&gt;. What does this mean? For example, suppose that states are frames of a video game (that is, each frame is a different state). In this context, successive frames (or states) are &lt;em&gt;similar&lt;/em&gt; to each other, which mathematically means that they are correlated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out that neural networks (NNs) are able to approximate (almost) any function. In RL, a policy is also a function: it is a function from a state to an action (or &lt;a href=&quot;https://ai.stackexchange.com/q/12274/2444&quot;&gt;probability distribution over actions&lt;/a&gt;). So, we can represent a policy using a NN. (Deep RL is essentially a combination of traditional RL algorithms, like Q-learning, with NNs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, it also turns out that training a NN using back-propagation with data that is temporally correlated might lead the NN not to capture the essential characteristics of the data, which, in practice (during training), means that we are not able to find the NN that represents the optimal policy (or another function, e.g. &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, used later to retrieve the policy). In such cases, we often say that the training of the NN is not stable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of RL, the data used to train such types of neural networks (which represent the policy, or other functions that are used in RL) are the &quot;rollouts&quot;, which contain elements that are often temporally correlated. Hence, we can't just feed the NN with a rollout, in the same order that the elements (states, actions, rewards and next states) are collected. So, in order to use uncorrelated data to train an NN, we can randomly take tuples of the form &lt;span class=&quot;math-container&quot;&gt;$\langle s_h, a_h, r_h, s_{h+1} \rangle$&lt;/span&gt; from the rollout (where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is some time step between &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;). For example, suppose we take (or &quot;sample&quot;) &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; tuples &lt;span class=&quot;math-container&quot;&gt;$\langle s_7, a_7, r_7, s_{8} \rangle$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\langle s_{97}, a_{97}, r_{97}, s_{98} \rangle$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\langle s_{2}, a_{2}, r_{2}, s_{3} \rangle$&lt;/span&gt;. Given that these elements have been observed at quite different points in time, they are likely to be less correlated than e.g. &lt;span class=&quot;math-container&quot;&gt;$\langle s_7, a_7, r_7, s_{8} \rangle$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\langle s_{8}, a_{8}, r_{8}, s_{9} \rangle$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\langle s_{9}, a_{9}, r_{9}, s_{10} \rangle$&lt;/span&gt; (which are successive &quot;tuples of experience&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, &quot;experience replay&quot; (or &quot;replay buffer&quot;, or &quot;experience replay buffer&quot;) refers to this technique of feeding a neural network using tuples (of &quot;experience&quot;) which are less likely to be correlated (given that &quot;random sampling&quot; procedure). The &quot;buffer&quot; part refers to a data structure (e.g. an array or list) that stores the trajectory (or rollout), that is, it stores the &quot;experience&quot; of the agent (hence the name &quot;experience&quot;). The &quot;replay&quot; refers to the fact that this &quot;experience&quot; is reused (or &quot;replayed&quot;) by randomly sampling from it to train the NN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also this question &lt;a href=&quot;https://ai.stackexchange.com/q/10839/2444&quot;&gt;Why exactly do neural networks require i.i.d. data?&lt;/a&gt; regarding the fact that NNs often require i.i.d. data.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-10T20:43:54.363" LastActivityDate="2019-06-10T20:43:54.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11645" PostTypeId="2" ParentId="11640" CreationDate="2019-04-04T16:24:25.587" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In order for the algorithm to have stable behavior, the replay buffer should be large enough to contain a wide range of experiences, but it may not always be good to keep everything.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The larger the experience replay, the less likely you will sample correlated elements, hence the more stable the training of the NN will be. However, a large experience replay also requires a lot of memory and it might slow training. So, there is a trade-off between training stability (of the NN) and memory requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors of the linked article state (right after the sentence above)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you only use the very-most recent data, you will overfit to that and things will break; if you use too much experience, you may slow down your learning. This may take some tuning to get right.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-10T17:45:45.407" LastActivityDate="2019-07-10T17:45:45.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11646" PostTypeId="5" CreationDate="2019-04-04T16:30:23.697" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-04T20:24:24.067" LastActivityDate="2019-04-04T20:24:24.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11647" PostTypeId="4" CreationDate="2019-04-04T16:30:23.697" Score="0" Body="For questions related to the reinforcement learning algorithm called Deep Deterministic Policy Gradient (DDPG)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-04T20:24:21.470" LastActivityDate="2019-04-04T20:24:21.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11648" PostTypeId="1" AcceptedAnswerId="11652" CreationDate="2019-04-04T17:43:07.613" Score="2" ViewCount="147" Body="&lt;p&gt;What is the motivation behind using a deterministic policy? Given that the environment is uncertain, it seems stochastic policy makes more sense. &lt;/p&gt;&#xA;" OwnerUserId="23707" LastEditorUserId="2444" LastEditDate="2019-04-04T19:12:14.107" LastActivityDate="2019-04-04T19:32:49.473" Title="What is the motivation behind using a deterministic policy?" Tags="&lt;reinforcement-learning&gt;&lt;deterministic-policy&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11651" PostTypeId="2" ParentId="11634" CreationDate="2019-04-04T18:42:29.457" Score="1" Body="&lt;p&gt;I would say this is not necessarily a duplicate but quite similar to some other questions. However, I will answer the question posed here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a theoretical level, what you are asking is there any algorithm that cannot be tricked into predicting the wrong class?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is: &lt;strong&gt;No&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is analogous to asking whether there is a perfect architecture for an arbitrary classification problem, and that answer is quite obviously not. It would also not likely to be terribly difficult to show this is provably the case, at least for a class of algorithms(i.e connectionist models).&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-04T18:42:29.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11652" PostTypeId="2" ParentId="11648" CreationDate="2019-04-04T19:04:38.950" Score="3" Body="&lt;p&gt;You're right! &lt;em&gt;Behaving&lt;/em&gt; according to a deterministic policy while still learning would be a terrible idea in most cases (with the exception of environments that &quot;do the exploring for you&quot;; see comments). But deterministic policies are learned off-policy. That is, the experience used to learn the deterministic policy is gathered by behaving according to a stochastic behavior policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under some reasonable assumptions--like that the environment is fully-observed and is stationary--an optimal deterministic policy always exists. The proof can be found in chapter 6 of &quot;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316887&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process -- Discrete Stochastic Dynamic Programming&lt;/a&gt;&quot; by Martin L. Puterman. The same cannot be said for stochastic polices. For this kind of environment (even if it's stochastic) an optimal policy is hardly ever stochastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, a motivation for wanting to learn a deterministic policy is often because we know that there is an optimal deterministic policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible your question was also tangentially about off-policy learning. &quot;Why learn a deterministic policy directly (off-policy) when we can just use something like decaying &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy?&quot; Briefly, off-policy learning is very powerful and general. It's necessary in any algorithm that uses experience replay, for example. A discussion about the merits of off-policy learning is probably best left to another question, but reading Section 5.5 of Sutton and Barto's &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL book&lt;/a&gt; should get you started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, directly learning a deterministic policy could be more computationally efficient if using &lt;a href=&quot;http://proceedings.mlr.press/v32/silver14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;deterministic policy gradients&lt;/a&gt;. In the setting with continuous state and action spaces, the deterministic policy gradient exists and has a simpler expectation than the stochastic policy gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic policy gradient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align*}&#xA;\nabla_\theta J(\pi_\theta) &amp;amp;= \int_\mathcal{S} \rho^\pi (s) \int_\mathcal{A} \nabla_\theta \pi_\theta (a|s) Q^\pi (s,a) \text{d}a\text{d}s\\&#xA;&amp;amp; = \mathbb{E}_{s\sim \rho^\pi, a\sim\pi_\theta}[\nabla_\theta \log \pi_\theta (a|s) Q^\pi (s,a)]&#xA;\end{align*}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deterministic policy gradient:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align*}&#xA;\nabla_\theta J(\mu_\theta) &amp;amp;= \int_\mathcal{S} \rho^\mu (s) \nabla_\theta \mu_\theta (s) \nabla_a Q^\mu (s,a)|_{a=\mu_\theta (s)}\text{d}s\\&#xA;&amp;amp; = \mathbb{E}_{s\sim \rho^\mu}[\nabla_\theta \mu_\theta (s) \nabla_a Q^\mu (s,a)|_{a=\mu_\theta (s)}]&#xA;\end{align*}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the expectation in the deterministic policy gradient isn't over the action space. Estimating this expectation would require many fewer samples in the setting of a continuous, high-dimensional action space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To recap:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Optimal policies are often deterministic, not stochastic&lt;/li&gt;&#xA;&lt;li&gt;Learning deterministic policies directly (off-policy) is powerful and general&lt;/li&gt;&#xA;&lt;li&gt;It can also be more efficient if in a continuous, high-dimensional action space&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-04T19:32:49.473" LastActivityDate="2019-04-04T19:32:49.473" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11653" PostTypeId="2" ParentId="11643" CreationDate="2019-04-04T19:52:23.580" Score="1" Body="&lt;p&gt;Yes, a simple reshape would do the trick. A flattening layer is just a tool for reshaping data/activations to make them compatible with other layers/functions. The flattening layer doesn't change the activations themselves, so there is no special backpropagation handling needed other than changing back the shape.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-04-04T19:52:23.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11655" PostTypeId="1" CreationDate="2019-04-04T20:10:23.970" Score="4" ViewCount="579" Body="&lt;p&gt;That has been discussion on &lt;a href=&quot;https://towardsdatascience.com/what-is-wrong-with-convolutional-neural-networks-75c2ba8fbd6f&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. Maybe from Hinton himself. And I heard that many max pooling layers have been replaced by conv layers in recent years, is that true?&lt;/p&gt;&#xA;" OwnerUserId="23688" LastActivityDate="2019-04-05T05:55:17.730" Title="Is max pooling really bad?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11656" PostTypeId="5" CreationDate="2019-04-04T21:09:42.693" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-04T21:09:42.693" LastActivityDate="2019-04-04T21:09:42.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11657" PostTypeId="4" CreationDate="2019-04-04T21:09:42.693" Score="0" Body="For questions about how AI and AI-related subjects are reported in the news media.  Can involve unclear or misleading reporting, requests for clarification of popular articles, or the subject of public perception of AI in the news media in general. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-04T21:09:42.693" LastActivityDate="2019-04-04T21:09:42.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11659" PostTypeId="1" CreationDate="2019-04-04T22:25:48.643" Score="0" ViewCount="16" Body="&lt;p&gt;I want to restore missing parts from a character. You can see some examples after the pages are edited with Scantailor:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Flduc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Flduc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I want to keep the same font, so I don't want just to OCR and create the pages of the scanned book with some new font.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Can I use machine learning to solve this problem? &lt;/p&gt;&#xA;" OwnerUserId="23715" LastActivityDate="2019-04-04T22:25:48.643" Title="Denoising and Improving the Quality of Scanned Books" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11661" PostTypeId="1" CreationDate="2019-04-05T03:26:43.337" Score="1" ViewCount="24" Body="&lt;p&gt;I wanted to get some opinions from the community for a certain problem that I will be approaching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is to provide feedback to a user based on a image of the upper male torso. The image would either reflect something positive like increasing muscle mass or decreasing muscle mass or both and gaining adipose tissue would be seen as negative as well as muscle atrophy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the users input such as (sleep data, food, training routine) among some other data I would like to provide feedback such as &quot;no John, this exercise has not yielded desirable results&quot; or &quot;a combination of your recent dietary change has caused strength loss&quot; obviously this is a complex issue and has a lot of interconnected variables and potentials but you get the idea high-level at least and if you don't - Please ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my idea so far would be to use a CNN that holds the picture of the torso, using a softmax function we could run this through a model to estimate bodyfat and  doing the same  with a model trained on muscle mass using those two models we could paint a pretty accurate picture of someones physique if they're going in the right direction or not; we could then go on to analyse what that user may have done/has not done to yield a result - Obviously there would be connected models here and many different combinations of algorithms applied such as CNN, RNN and others. Really curious to hear your response(s) thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="23726" LastEditorUserId="23726" LastEditDate="2019-04-05T13:45:16.197" LastActivityDate="2019-04-05T13:45:16.197" Title="Architecture and Use of Different Algorithms for Health Goal Feedback" Tags="&lt;machine-learning&gt;&lt;architecture&gt;&lt;model-based-methods&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11663" PostTypeId="2" ParentId="11655" CreationDate="2019-04-05T04:46:10.343" Score="1" Body="&lt;p&gt;Max pooling isn't bad, it just depends of what are you using the convnet for. For example if you are analyzing objects and the position of the object is important you shouldn't use it because the translational variance; if you just need to detect an object, it could help reducing the size of the matrix you are passing to the next convolutional layer. So it's up to the application you are going to use your CNN. &lt;/p&gt;&#xA;" OwnerUserId="9818" LastActivityDate="2019-04-05T04:46:10.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11664" PostTypeId="2" ParentId="11655" CreationDate="2019-04-05T05:55:17.730" Score="3" Body="&lt;p&gt;In addition to JCP's answer I would like to add some more detail. At best, max pooling is a less than optimal method to reduce feature matrix complexity and therefore over/under fitting and improve model generalization(for translation invariant classes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However as JCP begins to hit on.. there are problems with this method. Hinton perhaps sums the issues in &lt;a href=&quot;https://www.youtube.com/watch?v=rTawFwUvnLE&quot; rel=&quot;nofollow noreferrer&quot;&gt;his talk here&lt;/a&gt; on what is wrong with CNNs. This also serves as motivation for his novel architecture capsule networks or just capsules. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As he talks about, the main problem is not translational variance per se but rather pose variance. CNNs with max pooling are more than capable of handling simple transformations like flips or rotation without too much trouble. The problem comes with complicated transforms, as features learned about a chair facing forwards, will not be too helpful towards class representation if the real-world examples contain chairs upside down, to the side, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However there is much work being done here, mostly constrained to 2 areas. Those being, novel architectures/methods and inference of the 3d structure from images(via CNN tweaks). This problem was one of the bigger motivators for researchers throughout the decades, even &lt;a href=&quot;https://en.wikipedia.org/wiki/David_Marr_(neuroscientist)&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Marr&lt;/a&gt; with his primal sketches.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-05T05:55:17.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11666" PostTypeId="1" CreationDate="2019-04-05T06:34:04.157" Score="4" ViewCount="208" Body="&lt;p&gt;In computer vision is very common to use supervised tasks, where datasets have to be manually annotated by humans. Some examples are object classification (class labels), detection (bounding boxes) and segmentation (pixel-level masks). These datasets are essentially pairs of inputs-outputs which are used to train Convolutional Neural Networks to learn the mapping from inputs to outputs, via gradient descent optimization. But animals don't need anybody to show them bounding boxes or masks on top of things in order for them to learn to detect objects and make sense of the visual world around them. This leads me to think that brains must be performing some sort of self-supervision to train themselves to see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does current research say about the learning paradigm used by brains to achieve such an outstanding level of visual competence? Which tasks do brains use to train themselves to be so good at processing visual information and making sense of the visual world around them? Or said in other words: how does the brain manage to train its neural networks without having access to manually annotated datasets like ImageNet, COCO, etc. (i.e. what does the brain use as ground truth, what is the loss function the brain is optimizing)? Finally, can we apply these insights in computer vision?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update: I posted a related question on Psychology &amp;amp; Neuroscience StackExchange, which I think complements the question I posted here: &lt;a href=&quot;https://psychology.stackexchange.com/questions/21971/is-the-visual-cortex-of-a-newborn-baby-immediately-capable-of-object-detection-o&quot;&gt;check it out&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12746" LastEditorUserId="12746" LastEditDate="2019-04-11T21:03:17.333" LastActivityDate="2019-04-11T21:03:17.333" Title="Which loss function is the brain optimizing in order to learn advanced visual skills without expert/human supervision?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;cognitive-science&gt;&lt;biology&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11667" PostTypeId="1" AcceptedAnswerId="11675" CreationDate="2019-04-05T08:34:13.667" Score="4" ViewCount="1361" Body="&lt;p&gt;I am new to deep learning and trying to understand the concept of back-propagation. I have a doubt about when the back-propagation is applied.  Assume that I have a training data set of 1000 images for handwritten letters,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is back-propagation applied immediately after getting the output for each input or after getting the output for all inputs in a batch?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is back-propagation applied &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times till the neural network gives a satisfactory result for a single data point before going to work on the next data point?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23734" LastEditorUserId="2444" LastEditDate="2020-11-30T00:05:28.550" LastActivityDate="2020-11-30T00:05:28.550" Title="Is back-propagation applied for each data point or for a batch of data points?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;stochastic-gradient-descent&gt;&lt;batch-gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11668" PostTypeId="2" ParentId="11666" CreationDate="2019-04-05T08:34:14.050" Score="3" Body="&lt;p&gt;I think you are slightly confusing 2 problems. 1 being classification of meta visual elements and the other being the visual system itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our visual system, when it comes to processing information, has had billions of years of iteration(training), so that at birth(and before), we are already tuned for the processing of visual stimuli, as well as have the mechanisms to decipher objects in our spatial field of view. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These two papers(&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S006528060860065X&quot; rel=&quot;nofollow noreferrer&quot;&gt;L1&lt;/a&gt;, &lt;a href=&quot;https://www.researchgate.net/profile/Adrian_Horridge/publication/20270930_The_Evolution_of_Visual_Processing_and_the_Construction_of_Seeing_Systems/links/574ce89808aec988526a266d.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;L2&lt;/a&gt;), have a great deal of information about the evolution of our visual system and its processing. The second speculates on the connection of said evolution and the construction of &quot;seeing systems&quot; very interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further inquiry on this in particular, check out &lt;a href=&quot;https://en.wikipedia.org/wiki/David_Marr_(neuroscientist)&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Marr&lt;/a&gt;. He was probably the most influential early computer vision mind. He still is mentioned in many top-down AGI and computer vision research projects to this day.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-05T08:34:14.050" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11669" PostTypeId="1" CreationDate="2019-04-05T09:08:56.710" Score="1" ViewCount="704" Body="&lt;p&gt;Forward KL Divergence (also known as cross entropy loss) is a standard loss function in supervised learning problems. I understand why it is so: matching a known a trained distribution to a known distribution fits &lt;span class=&quot;math-container&quot;&gt;$P \log(P/Q)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is the known distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why isn't the reverse KL divergence commonly used in supervised learning?&lt;/p&gt;&#xA;" OwnerUserId="23723" LastEditorUserId="2444" LastEditDate="2019-04-05T12:21:04.883" LastActivityDate="2019-04-08T08:19:02.583" Title="Why isn't the reverse KL divergence commonly used in supervised learning?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;loss-functions&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11671" PostTypeId="1" CreationDate="2019-04-05T10:06:49.173" Score="0" ViewCount="40" Body="&lt;p&gt;Conventional NVIDIA GPUs, such as the Titan and or GT1080, are used to train AI models.  Why would a Jetson Nano board be less than ideal as a substitute for a conventional GPU?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;CONTEXT&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I would like to run simple concept demonstrator training exercises with and without a GPU: that being said, I am not interested in buying a desktop and outfitting it with a $1000+ GPU for said exercise.&lt;/p&gt;&#xA;" OwnerUserId="18819" LastEditorUserId="18819" LastEditDate="2019-04-05T10:59:14.080" LastActivityDate="2019-04-05T10:59:14.080" Title="Why is a Nvidia Single Board computer less than ideal for AI model training?" Tags="&lt;training&gt;&lt;gpu&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11672" PostTypeId="1" CreationDate="2019-04-05T10:47:53.840" Score="3" ViewCount="165" Body="&lt;p&gt;I read that functions are used as activation functions only when they are differentiable. What about the unit step activation function? So, is there any other reason a function can be used as an activation function (apart from being differentiable)?&lt;/p&gt;&#xA;" OwnerUserId="23501" LastEditorUserId="2444" LastEditDate="2019-04-05T19:45:31.297" LastActivityDate="2020-06-25T20:22:04.743" Title="What kind of functions can be used as activation functions?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2020-11-09T09:55:53.243" ContentLicense="CC BY-SA 4.0" />
  <row Id="11673" PostTypeId="2" ParentId="11671" CreationDate="2019-04-05T10:48:13.960" Score="1" Body="&lt;p&gt;Jetson Nano specs Price:$99:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;128-core Maxwell GPU &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Quad-core ARM A57 @ 1.43 GHz &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;472 GFLOPS max&#xA;   compute&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Titan V(for example) Price:$2500 :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;640 tensor cores&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;5120 cuda cores&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;110000 GFLOPS max compute&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Some simple maths gets us to a speed of 11,800 GFLOPS for 25 of these perfectly synced in parallel(not likely or gonna be fun). Vs the 110000 it just isn't even a question.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-05T10:48:13.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11674" PostTypeId="1" CreationDate="2019-04-05T11:29:49.410" Score="2" ViewCount="61" Body="&lt;p&gt;Suppose a model &lt;code&gt;M&lt;/code&gt; classifies apples and oranges. Can &lt;code&gt;M&lt;/code&gt; be extended to classify a third class of objects, e.g., pears, such that the new images for 'retraining' only have pears annotated and apples and oranges ignored? That is, since &lt;code&gt;M&lt;/code&gt; already classifies apples and oranges, can the old weights be somehow preserved and let the retraining focus specifically on learning about pears?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods such as fine-tuning and learning without forgetting seem to require all objects in the new images annotated though.&lt;/p&gt;&#xA;" OwnerUserId="13068" LastActivityDate="2019-04-06T23:52:04.330" Title="Extending a neural network to classify new objects" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11675" PostTypeId="2" ParentId="11667" CreationDate="2019-04-05T14:18:28.067" Score="5" Body="&lt;h1&gt;Short answers&lt;/h1&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is back-propagation applied immediately after getting the output for each input or after getting the output for all inputs in a batch?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can perform back-propagation using (or after) only one training &lt;em&gt;input&lt;/em&gt; (also known as &lt;em&gt;data point&lt;/em&gt;, &lt;em&gt;example&lt;/em&gt;, &lt;em&gt;sample&lt;/em&gt; or &lt;em&gt;observation&lt;/em&gt;) or multiple ones (a &lt;em&gt;batch&lt;/em&gt;). However, the loss function to train the neural network is &lt;em&gt;slightly&lt;/em&gt; different in both cases.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is back-propagation applied &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times till the neural network gives a satisfactory result for a single data point before going to work on the next data point?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If we use only one example, we usually do not wait until the neural network gives satisfactory results for a single input-label pair &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;, but we keep feeding it with several input-label pairs, one after the other (and each time updating the parameters of the neural network using back-propagation), without &lt;em&gt;usually&lt;/em&gt; caring whether the neural network already produces a satisfactory output for an input-label pair before feeding it with the next (although you could also do that).&lt;/p&gt;&#xA;&lt;h1&gt;Long answer (to the 1st question)&lt;/h1&gt;&#xA;&lt;p&gt;In case you want to know more about the first question, keep reading!&lt;/p&gt;&#xA;&lt;h2&gt;What is back-propagation?&lt;/h2&gt;&#xA;&lt;p&gt;Back-propagation is the (automatic) process of &lt;strong&gt;differentiating&lt;/strong&gt; the &lt;strong&gt;loss function&lt;/strong&gt;, which we use to train the neural network, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;, with respect to all of the parameters (or weights) of the same neural network. If you collect the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; parameters of the neural network in a vector&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\boldsymbol{\theta} = &#xA;\begin{bmatrix}&#xA;\theta_1\\&#xA;\vdots \\&#xA;\theta_N&#xA;\end{bmatrix},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;then the derivative of the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; is called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;gradient&lt;/strong&gt;&lt;/a&gt;, which is a vector that contains the &lt;strong&gt;partial derivatives&lt;/strong&gt; of the loss function with respect to each single scalar parameter of the network, &lt;span class=&quot;math-container&quot;&gt;$\theta_i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, N$&lt;/span&gt;, that is, the gradient looks something like this&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla \mathcal{L} =&#xA;\begin{bmatrix}&#xA;\frac{\partial \mathcal{L}}{ \partial \theta_1}\\&#xA;\vdots \\&#xA;\frac{\partial \mathcal{L}}{ \partial \theta_N}&#xA;\end{bmatrix},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the symbol &lt;span class=&quot;math-container&quot;&gt;$\nabla$&lt;/span&gt; denotes the gradient of the function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Loss functions&lt;/h2&gt;&#xA;&lt;p&gt;The specific loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; that is used to train the neural network depends on the problem that we need to solve. For example, if we need to solve a &lt;strong&gt;regression problem&lt;/strong&gt; (i.e. predict a real number), then the &lt;strong&gt;mean squared error (MSE)&lt;/strong&gt; can be used. If we need to solve a &lt;strong&gt;classification problem&lt;/strong&gt; (i.e. predict a class), the &lt;strong&gt;cross-entropy (CE)&lt;/strong&gt; (aka &lt;em&gt;negative log-likelihood&lt;/em&gt;) may be used instead.&lt;/p&gt;&#xA;&lt;h2&gt;Example&lt;/h2&gt;&#xA;&lt;p&gt;Let us assume that we need to solve a regression problem, so we choose the mean squared error (MSE) as the loss function. For simplicity, let's also assume that the neural network, denoted by &lt;span class=&quot;math-container&quot;&gt;$f_{\boldsymbol{\theta}}$&lt;/span&gt;, contains only one output neuron and contains no biases.&lt;/p&gt;&#xA;&lt;h3&gt;Stochastic gradient descent&lt;/h3&gt;&#xA;&lt;p&gt;Given an input-label pair &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i) \in D$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is a dataset of input-label pairs), the &lt;strong&gt;squared error&lt;/strong&gt; function (not the &lt;em&gt;mean&lt;/em&gt; squared error yet!) is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}_i(\boldsymbol{\theta}) = \frac{1}{2} (f_{\boldsymbol{\theta}}(x_i) - y_i)^2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f_{\boldsymbol{\theta}}(x_i) = \hat{y}_i$&lt;/span&gt; is the output of the neural network for the the data point &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; (which depends on the specific values of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is the corresponding ground-truth &lt;strong&gt;label&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As the name suggests, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_i$&lt;/span&gt; (where the subscript &lt;span class=&quot;math-container&quot;&gt;$_i$&lt;/span&gt; is only used to refer to the specific input-label pair &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;) measures the squared &lt;strong&gt;error&lt;/strong&gt; (i.e. some notion of distance) between the current prediction (or output) of the neural network, &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt;, and the &lt;em&gt;expected&lt;/em&gt; output for the given input &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We can differentiate &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_i(\boldsymbol{\theta})$&lt;/span&gt; with respect to the parameters of the neural network, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt;. However, given that the details of back-propagation can easily become tedious, I will not describe them here. You can find more details &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, let me assume that we have a computer program that is able to compute &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L}_i$&lt;/span&gt;. At that point, we can perform one step of the &lt;strong&gt;gradient descent&lt;/strong&gt; algorithm&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol{\theta} \leftarrow \boldsymbol{\theta} - \gamma \nabla \mathcal{L}_i, \label{1}\tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the &lt;strong&gt;learning rate&lt;/strong&gt; and &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; is the &lt;em&gt;assignment&lt;/em&gt; operator. Note that &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L}_i$&lt;/span&gt; have the same dimensions, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, I have just shown you that you can update the parameters of the neural network using only one input-label pair, &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;. This way of performing GD with only one input-label pair is known as &lt;strong&gt;stochastic gradient descent (SGD)&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Batch (or mini-batch) gradient descent&lt;/h3&gt;&#xA;&lt;p&gt;In practice, for several reasons (including learning instability and inefficiency), it is rarely the case that you update the parameters using only one input-label pair &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;. Instead, you use multiple input-label pairs, which are collected in a so-called &lt;strong&gt;batch&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B = \{(x_1, y_i), \dots, (x_M, y_M) \},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the size of &lt;strong&gt;batch&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, which is also known as &lt;strong&gt;mini-batch&lt;/strong&gt; when &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is smaller than the total number of input-label pairs in the training dataset, i.e. when &lt;span class=&quot;math-container&quot;&gt;$|B| = M &amp;lt; |D|$&lt;/span&gt;. If you use mini-batches, typical values for &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$32$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$256$&lt;/span&gt; (yes, powers of 2: can you guess why?). See &lt;a href=&quot;https://ai.stackexchange.com/q/8560/2444&quot;&gt;this question for other details&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In this case, the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_M(\boldsymbol{\theta})$&lt;/span&gt; is defined as as the &lt;strong&gt;mean&lt;/strong&gt; (or &lt;em&gt;average&lt;/em&gt;) of the squared errors for single input-label pairs, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_i(\boldsymbol{\theta})$&lt;/span&gt;, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathcal{L}_M(\boldsymbol{\theta}) &#xA;&amp;amp;= \frac{1}{M} \sum_{i=1}^M \mathcal{L}_i(\boldsymbol{\theta}) \\&#xA;&amp;amp;= \frac{1}{M} \sum_{i=1}^M \frac{1}{2} (f_{\boldsymbol{\theta}}(x_i)-y_i)^2 \\&#xA;&amp;amp;= \frac{1}{M} \frac{1}{2} \sum_{i=1}^M (f_{\boldsymbol{\theta}}(x_i)-y_i)^2.&#xA;\end{align}&lt;/span&gt;&#xA;The normalisation factor, &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{M}$&lt;/span&gt;, can be thought of as &lt;em&gt;averaging out&lt;/em&gt; the losses of all input-label pairs. Note also that we can take the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; out of the summation because it is a constant with respect to the variable of the summation, &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In this case, let me also assume that we are able to compute (using back-propagation) the gradient of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_M$&lt;/span&gt;, so that we can perform a gradient descent (GD) update (using a batch of examples)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol{\theta} \leftarrow \boldsymbol{\theta} - \gamma \nabla \mathcal{L}_M \label{2} \tag{2}&#xA;$$&lt;/span&gt;&#xA;The only thing that changes with respect to the GD update in equation \ref{1} is the loss function, which is now &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_M$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is known as &lt;strong&gt;mini-batch&lt;/strong&gt; (or &lt;strong&gt;batch&lt;/strong&gt;) &lt;strong&gt;gradient descent&lt;/strong&gt;. In the case &lt;span class=&quot;math-container&quot;&gt;$M = |D|$&lt;/span&gt;, this is simply known as &lt;strong&gt;gradient descent&lt;/strong&gt;, which is a term that can also be used to refer to any of its variants (including SGD or mini-batch GD).&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;You may also be interested in &lt;a href=&quot;https://ai.stackexchange.com/a/20380/2444&quot;&gt;this answer&lt;/a&gt;, which provides more details about mini-batch gradient descent.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-29T23:56:30.773" LastActivityDate="2020-11-29T23:56:30.773" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11676" PostTypeId="1" AcceptedAnswerId="11682" CreationDate="2019-04-05T14:23:21.970" Score="4" ViewCount="198" Body="&lt;p&gt;In Sutton &amp;amp; Barto's book (2nd ed) page 149, there is the equation 7.11&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tWlV2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tWlV2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having a hard time understanding this equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would have thought that we should be moving &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; towards &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; would be corrected by importance sampling, but only &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, not &lt;span class=&quot;math-container&quot;&gt;$G-Q$&lt;/span&gt;, therefore I would have thought that the correct equation would be of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q \leftarrow Q + \alpha (\rho G - Q)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and not&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q \leftarrow Q + \alpha \rho (G - Q)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't get why the entire update is weighted by &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; and not only the sampled return &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22003" LastEditorUserId="2444" LastEditDate="2019-04-05T14:33:50.430" LastActivityDate="2019-04-05T20:44:26.283" Title="Understanding the n-step off-policy SARSA update" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;off-policy-methods&gt;&lt;temporal-difference-methods&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11678" PostTypeId="1" CreationDate="2019-04-05T16:13:24.633" Score="1" ViewCount="309" Body="&lt;p&gt;I am new to NLP realm. If you have an input text &quot;The price of orange has increased&quot; and output text &quot;Increase the production of orange&quot;. Can we make our RNN model to predict the output text? Or what algorithm should I use?&lt;/p&gt;&#xA;" OwnerUserId="23743" LastEditorUserId="2444" LastEditDate="2019-04-05T16:29:01.640" LastActivityDate="2020-05-06T07:03:50.380" Title="Which algorithm should I use to map an input sentence to an output sentence?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11679" PostTypeId="1" CreationDate="2019-04-05T18:23:46.233" Score="17" ViewCount="1942" Body="&lt;p&gt;The tabular Q-learning algorithm is guaranteed to find the optimal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt;, provided the following conditions (the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;noreferrer&quot;&gt;Robbins-Monro conditions&lt;/a&gt;) regarding the learning rate are satisfied&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{t} \alpha_t(s, a) = \infty$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{t} \alpha_t^2(s, a) &amp;lt; \infty$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha_t(s, a)$&lt;/span&gt; means the learning rate used when updating the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value associated with state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; at time time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$0 \leq  \alpha_t(s, a) &amp;lt; 1$&lt;/span&gt; is assumed to be true, for all states &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, given that &lt;span class=&quot;math-container&quot;&gt;$0 \leq  \alpha_t(s, a) &amp;lt; 1$&lt;/span&gt;, in order for the two conditions to be true, all state-action pairs must be visited infinitely often: this is also stated in the book &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2018mar21.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;, apart from the fact that this should be widely known and it is the rationale behind the usage of the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy (or similar policies) during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A complete proof that shows that &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning finds the optimal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function can be found in the paper &lt;a href=&quot;http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf&quot; rel=&quot;noreferrer&quot;&gt;Convergence of Q-learning: A Simple Proof&lt;/a&gt; (by Francisco S. Melo). He uses concepts like &lt;a href=&quot;https://en.wikipedia.org/wiki/Contraction_mapping&quot; rel=&quot;noreferrer&quot;&gt;contraction mapping&lt;/a&gt; in order to define the optimal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function (see also &lt;a href=&quot;https://ai.stackexchange.com/q/11057/2444&quot;&gt;What is the Bellman operator in reinforcement learning?&lt;/a&gt;), which is a fixed point of this contraction operator. He also uses a theorem (n. 2) regarding the random process that converges to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, given a few assumptions. (The proof might not be easy to follow if you are not a math guy.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a neural network is used to represent the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, do the convergence guarantees of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning still hold? Why does (or not) Q-learning converge when using function approximation? Is there a formal proof of such non-convergence of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning using function approximation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for different types of answers, from those that give just the intuition behind the non-convergence of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning when using function approximation to those that provide a formal proof (or a link to a paper with a formal proof).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-18T16:01:26.307" LastActivityDate="2020-12-19T13:14:29.677" Title="Why doesn't Q-learning converge when using function approximation?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;proofs&gt;&lt;function-approximation&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11680" PostTypeId="2" ParentId="11672" CreationDate="2019-04-05T19:19:34.603" Score="2" Body="&lt;p&gt;Not completely sure your question. Do you mean&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q. why should we use activation function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ans: we need to introduce non-linearity to the network. Otherwise, multiple layers are no difference from single layer network. (It is obvious as we write things in matrix form, and say when we have two layers with weights &lt;span class=&quot;math-container&quot;&gt;$W_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_2$&lt;/span&gt;, the two layer is no difference from a single layer with weight &lt;span class=&quot;math-container&quot;&gt;$W_2 W_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q. why they need to be differentiable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ans: Just for sake that we can back-propagate gradients back to earlier layers. Note that back-propagation is nothing but the chain rule in calculus. Say &lt;span class=&quot;math-container&quot;&gt;$f(\cdot)$&lt;/span&gt; is an activation function in one layer and the output of that activation function is &lt;span class=&quot;math-container&quot;&gt;$\bf y$&lt;/span&gt; and the input is &lt;span class=&quot;math-container&quot;&gt;${\bf u}=W \bf x$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\bf x$&lt;/span&gt; is output from the previous layer and mix with weights &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; in current layer. Of course, the final loss &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; will depend on &lt;span class=&quot;math-container&quot;&gt;${\bf y} = f({\bf u})= f(W {\bf x})$&lt;/span&gt;. Say, loss &lt;span class=&quot;math-container&quot;&gt;$L=g(\bf y)$&lt;/span&gt; somehow. To train the weights &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, we have to find the gradient &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial W}$&lt;/span&gt; so that we can adjust weight &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; to minimize &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. But &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial W}=\frac{\partial g({\bf y})}{\partial W}=\frac{\partial g({\bf y})}{\partial \bf y}\frac{\partial {\bf y}}{\partial {\bf u}}\frac{\partial {\bf u}}{\partial W}$&lt;/span&gt;. Each of these product terms can be computed locally and will be accumulatively multiplied as we apply backprop.  And note that the middle term &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial {\bf y}}{\partial {\bf u}}=\frac{\partial f({\bf u})}{\partial {\bf u}}$&lt;/span&gt; is just &quot;derivative&quot; of &lt;span class=&quot;math-container&quot;&gt;$f(\cdot)$&lt;/span&gt;, thus we require the activation function to be differentiable and &quot;informative&quot;/non-zero (at least most of the time). Note that ReLU is not differentiable everywhere and that is why researchers (at least Yoshua Bengio) worried about that when they first tried to adopt ReLU. You may check out the interview of Bengio by Andrew Ng for that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q. why step function is a bad activation function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ans: Note that step function is differentiable almost everywhere but is not &quot;informative&quot; though. For places (flat region) where it is differentiable, the derivative is simply zero. Consequently, any later layer gradient (information) will get cut off as it passes through a step function activation function.&lt;/p&gt;&#xA;" OwnerUserId="23688" LastEditorUserId="23688" LastEditDate="2019-04-05T19:25:06.237" LastActivityDate="2019-04-05T19:25:06.237" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11681" PostTypeId="2" ParentId="11679" CreationDate="2019-04-05T19:25:49.813" Score="12" Body="&lt;p&gt;Here's an intuitive description answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Function approximation can be done with any parameterizable function. Consider the problem of a &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; space where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the positive reals, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, and the true Q-function is &lt;span class=&quot;math-container&quot;&gt;$Q(s, 0) = s^2$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$Q(s, 1)= 2s^2$&lt;/span&gt;, for all states. If your function approximator is &lt;span class=&quot;math-container&quot;&gt;$Q(s, a) = m*s + n*a + b$&lt;/span&gt;, there exists no parameters which can accurately represent the true &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function (we're trying to fit a line to a quadratic function). Consequently, even if you chose a good learning rate, and visit all states infinitely often, your approximation function will never converge to the true &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here's a bit more detail:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Neural networks &lt;em&gt;approximate&lt;/em&gt; functions. A function can be approximated to greater or lesser degrees by using more or less complex polynomials to approximate it. If you're familiar with Taylor Series approximation, this idea should seem pretty natural. If not, think about a function like a sine-wave over the interval [0-&lt;span class=&quot;math-container&quot;&gt;$\pi/2$&lt;/span&gt;). You can approximate it (badly) with a straight line. You can approximate it better with a quadratic curve. By increasing the degree of the polynomial we use to approximate the curve, we can get something that fits the curve more and more closely.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural networks are &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;noreferrer&quot;&gt;universal function approximators&lt;/a&gt;. This means that, if you have a function, you can also make a neural network that is deep or wide enough that it can approximate the function you have created to an arbitrarily precise degree. However, any specific network topology you pick will be unable to learn &lt;em&gt;all&lt;/em&gt; functions, unless it is infinitely wide or infinitely deep. This is analogous to how, if you pick the right parameters, a line can fit any two points, but not any 3 points. If you pick a network that is of a certain finite width or depth, I can always construct a function that needs a few more neurons to fit properly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Q-learning's bounds hold only when the representation of the Q-function is &lt;em&gt;exact&lt;/em&gt;. To see why, suppose that you chose to approximate your Q-function with a linear interpolation. If the true function can take any shape at all, then clearly the error in our interpolation can be made unboundedly large simply by constructing a XOR-like Q-function function, and no amount of extra time or data will allow us to reduce this error. If you use a function approximator, and the true function you try to fit is &lt;em&gt;not&lt;/em&gt; something that the function can approximate arbitrarily well, then your model will not converge properly, even with a well-chosen learning rate and exploration rate. Using the terminology of computational learning theory, we might say that the convergence proofs for Q-learning have implicitly assumed that the true Q-function is a member of the hypothesis space from which you will select your model. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-04-06T00:11:18.947" LastActivityDate="2019-04-06T00:11:18.947" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11682" PostTypeId="2" ParentId="11676" CreationDate="2019-04-05T20:44:26.283" Score="3" Body="&lt;p&gt;Multiplying the entire update by &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; has the desirable property that experience affects &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; less when the behavior policy is unrelated to the target policy. In the extreme, if the trajectory taken has zero probability under the target policy, then &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; isn't updated at all, which is good. Alternatively, if only &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is scaled by &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, taking zero probability trajectories would artificially drive &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; to zero.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-04-05T20:44:26.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11683" PostTypeId="1" CreationDate="2019-04-06T00:50:34.073" Score="7" ViewCount="196" Body="&lt;p&gt;Does anyone know what specific tasks the OpenCog environment is capable of performing? I have glanced though their wiki and a few of the pages on Goertzel's site and the AI.SE. So far I could only find some technical documentation regarding theory and engineering, but nothing on concrete results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the technical description of AtomSpaces it seems that OpenCog is capable of some &quot;representational inference&quot;, but I haven't come across any sources that concretely describes what it is capable of doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently there is some collaboration between Sophia the Robot and OpenCog, but to what extent I am unclear. I am aware however that the dialogue functions is powered by ChatScript (though I also suspect that the high profile interviews Sophia gives are completely scripted...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone provide concrete examples or evidence of OpenCogs' functional behavior. Like transcripts of chat, examples of reasoning, video or demonstrations of its emotion-emulating; and not just claims of functions.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-04-23T04:06:30.840" LastActivityDate="2020-04-23T04:06:30.840" Title="Concrete examples of OpenCog's functionality" Tags="&lt;applications&gt;&lt;agi&gt;&lt;open-cog&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11684" PostTypeId="1" CreationDate="2019-04-06T03:15:00.873" Score="2" ViewCount="53" Body="&lt;p&gt;I have not seen a neuron that uses both a bias and a threshold. Why is this? &lt;/p&gt;&#xA;" OwnerUserId="23501" LastEditorUserId="22916" LastEditDate="2019-04-06T19:55:00.393" LastActivityDate="2019-04-06T19:55:00.393" Title="Can a neuron have both a bias and a threshold?" Tags="&lt;neural-networks&gt;&lt;neurons&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11685" PostTypeId="1" CreationDate="2019-04-06T06:52:05.100" Score="2" ViewCount="104" Body="&lt;p&gt;Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning. &lt;/p&gt;&#xA;" OwnerUserId="23501" LastActivityDate="2019-04-06T17:29:52.440" Title="At which point we have to stop post pruning in decision tree?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11686" PostTypeId="2" ParentId="11684" CreationDate="2019-04-06T07:03:27.060" Score="4" Body="&lt;p&gt;I assume you're talking about a &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;perceptron&lt;/a&gt; threshold function. One definition of it with an explicit threshold is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\textbf{x})= &#xA;\begin{cases}&#xA;1&amp;amp; \text{if } \textbf{w}\cdot\textbf{x} &amp;gt; t\\&#xA;0&amp;amp; \text{otherwise}&#xA;\end{cases}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another form with a bias is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\textbf{x})= &#xA;\begin{cases}&#xA;1&amp;amp; \text{if } \textbf{w}\cdot\textbf{x} + b &amp;gt; 0\\&#xA;0&amp;amp; \text{otherwise}&#xA;\end{cases}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these forms are of course equivalent if you set &lt;span class=&quot;math-container&quot;&gt;$b=-t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's nothing stopping you from using a perceptron definition with both a bias and a threshold:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\textbf{x})= &#xA;\begin{cases}&#xA;1&amp;amp; \text{if } \textbf{w}\cdot\textbf{x} +b &amp;gt; t\\&#xA;0&amp;amp; \text{otherwise}&#xA;\end{cases}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is also equivalent to the other two forms. We can rewrite this as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\textbf{x})= &#xA;\begin{cases}&#xA;1&amp;amp; \text{if } \textbf{w}\cdot\textbf{x} &amp;gt; t'\\&#xA;0&amp;amp; \text{otherwise}&#xA;\end{cases}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$t'=t-b$&lt;/span&gt; is the new threshold. Or, we could rewrite it as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\textbf{x})= &#xA;\begin{cases}&#xA;1&amp;amp; \text{if } \textbf{w}\cdot\textbf{x} + a&amp;gt; 0\\&#xA;0&amp;amp; \text{otherwise}&#xA;\end{cases}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$a=b-t$&lt;/span&gt; is the new bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You never see definitions of this function with both a threshold and a bias because it has simpler forms.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-04-06T07:03:27.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11687" PostTypeId="5" CreationDate="2019-04-06T08:42:42.637" Score="0" Body="&lt;p&gt;For more info, have a look at &lt;a href=&quot;https://opencog.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://opencog.org/&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenCog&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/OpenCog&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-11T21:02:44.100" LastActivityDate="2019-04-11T21:02:44.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11688" PostTypeId="4" CreationDate="2019-04-06T08:42:42.637" Score="0" Body="For questions related to Open Cog, which is a project that aims to build an open source artificial intelligence framework." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-11T21:03:01.377" LastActivityDate="2019-04-11T21:03:01.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11690" PostTypeId="2" ParentId="11679" CreationDate="2019-04-06T09:16:00.397" Score="5" Body="&lt;p&gt;As far as I'm aware, it is still somewhat of an open problem to get a really clear, formal understanding of exactly why / when we get a lack of convergence -- or, worse, sometimes a danger of divergence. It is typically attributed to the &lt;strong&gt;&quot;deadly triad&quot;&lt;/strong&gt; (see 11.3 of the second edition of Sutton and Barto's book), the combination of:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Function approximation, AND&lt;/li&gt;&#xA;&lt;li&gt;Bootstrapping (using our own value estimates in the computation of our training targets, as done by &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning), AND&lt;/li&gt;&#xA;&lt;li&gt;Off-policy training (&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is indeed off-policy).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That only gives us a (possibly non-exhaustive) description of cases in which we have a lack of convergence and/or a danger of divergence, but still doesn't tell us &lt;strong&gt;why&lt;/strong&gt; it happens in those cases.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/11681/1641&quot;&gt;John's answer&lt;/a&gt; already provides the intuition that part of the problem is simply that the use of function approximation can easily lead to situations where your function approximator isn't powerful enough to represent the true &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt; function, there may always be approximation errors that are impossible to get rid of without switching to a different function approximator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I think this intuition does help to understand why the algorithm cannot guarantee convergence to the optimal solution, but I'd still intuitively expect it to maybe be capable of &quot;converging&quot; to some &quot;stable&quot; solution that is the best possible approximation given the restrictions inherent in the chosen function representation. Indeed, this is what we observe in practice when we switch to on-policy training (e.g. Sarsa), at least in the case with linear function approximators.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My own intuition with respect to this question has generally been that an important source of the problem is &lt;strong&gt;generalisation&lt;/strong&gt;. In the tabular setting, we have completely isolated entries &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; pairs. Whenever we update our estimate for one entry, it leaves all other entries unmodified (at least initially -- there may be some effects on other entries in future updates due to bootstrapping in the update rule). Update rules for algorithms like &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning and Sarsa may sometimes update towards the &quot;wrong&quot; direction if we get &quot;unlucky&quot;, but &lt;strong&gt;in expectation&lt;/strong&gt;, they generally update towards the correct &quot;direction&quot;. Intuitively, this means that, in the tabular setting, &lt;strong&gt;in expectation&lt;/strong&gt; we will slowly, gradually fix any mistakes in any entries in isolation, without possibly harming other entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With function approximation, when we update our &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; estimate for one &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; pair, it can potentially also affect &lt;strong&gt;all&lt;/strong&gt; of our other estimates for &lt;strong&gt;all&lt;/strong&gt; other state-action pairs. Intuitively, this means that we no longer have the nice isolation of entries as in the tabular setting, and &quot;fixing&quot; mistakes in one entry may have a risk of adding new mistakes to other entries. However, like John's answer, this whole intuition would really also apply to on-policy algorithms, so it still doesn't explain what's special about &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning (and other off-policy approaches).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A very interesting recent paper on this topic is &lt;a href=&quot;https://papers.nips.cc/paper/8200-non-delusional-q-learning-and-value-iteration&quot; rel=&quot;noreferrer&quot;&gt;Non-delusional Q-learning and Value Iteration&lt;/a&gt;. They point out a problem of &quot;delusional bias&quot; in algorithms that combine function approximation with update rules involving a &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator, such as Q-learning (it's probably not unique to the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator, but probably applies to off-policy in general?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is as follows. Suppose we run this &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update for a state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a) \gets Q(s, a) + \alpha \left[ \max_{a'} Q(s', a') - Q(s, a) \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value estimate &lt;span class=&quot;math-container&quot;&gt;$\max_{a'} Q(s', a')$&lt;/span&gt; used here is based on the assumption that we execute a policy that is greedy with respect to older versions of our &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; estimates over a -- possibly very long -- trajectory. As already discussed in some of the previous answers, our function approximator has a limited representational capacity, and updates to one state-action pair may affect value estimates for other state-action pairs. This means that, after triggering our update to &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, &lt;strong&gt;our function approximator may no longer be able to simultaneously express the policy that leads to the high returns that our &lt;span class=&quot;math-container&quot;&gt;$\max_{a'} Q(s', a')$&lt;/span&gt; estimate was based on&lt;/strong&gt;. The authors of this paper say that the algorithm is &quot;delusional&quot;. It performs an update under the assumption that, down the line, it can still obtain large returns, but it may no longer actually be powerful enough to obtain those returns with the new version of the function approximator's parameters.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Finally, another (even more recent) paper that I suspect is relevant to this question is &lt;a href=&quot;https://arxiv.org/abs/1902.10250&quot; rel=&quot;noreferrer&quot;&gt;Diagnosing Bottlenecks in Deep Q-learning Algorithms&lt;/a&gt;, but unfortunately I have not yet had the time to read it in sufficient detail and adequately summarise it.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-04-06T09:16:00.397" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11691" PostTypeId="1" CreationDate="2019-04-06T10:03:57.300" Score="2" ViewCount="453" Body="&lt;p&gt;I want to see if I can make my Software Defined Radio, SDR, to classify unknown radio signals with the help of an artificial neural network. That is, my SDR outputs a sequence of complex numbers (IQ-data), which I want to use to determine if the receieved signal is, for instance, FM or AM modulated. This approach was used in a paper (&lt;a href=&quot;https://arxiv.org/pdf/1712.04578.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1712.04578.pdf&lt;/a&gt;) and they created and used a freely downloadable dataset (&lt;a href=&quot;https://github.com/sofwerx/deepsig_datasets/blob/master/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sofwerx/deepsig_datasets/blob/master/README.md&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being new to both SDR and Deep Learning I have now tried for a couple of months to create an LSTM network, train it on the dataset and then use it for classification, but have sadly failed. I have concluded that it is likely due to the fact that I do not seem to understand how the dataset is structured. I have not been able to find any documentation concerning this dataset (other than a text file with a list of the modulation forms used in the dataset) though. My hope is that someone on this forum has some prior experience to share about how to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dataset (DEEPSIG DATASET: RADIOML 2016.10A) is split in three matrixes, X: 2x1024x2555904 cells, Y: 24x2555904 cells and Z: 1x2555904 cells. My belief has up until now been that &quot;X&quot; contains the complex time series, i.e. one row for the real component and one row for the imaginary component both in a sequence of 1024 samples. The &quot;Y&quot; to contain the corresponding 24 classes, whereas the &quot;Z&quot; somehow contain the signal to noise level for each signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training and deploying the network in Matlab I found out that it does not even manage to classify the local FM radio station. I then looked at the data I had used for training and plotted one of (what I thought to be) the FM sample sequences from the dataset in the complex plane, but I did not get the &quot;circle&quot; I had expected from a frequency or phase modulated signal. I am thus totally lost, the dataset has been used for a high class scientific paper so the problem lies with me, and my lack of understanding. I apologize for the long and still very unconcise question, but I don't want to infer any of my own misconceptions into the query. Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="23760" LastActivityDate="2019-12-18T21:59:24.333" Title="Deep Learning for radio signal classification with DeepSig dataset" Tags="&lt;datasets&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11692" PostTypeId="2" ParentId="11669" CreationDate="2019-04-06T11:34:47.823" Score="1" Body="&lt;p&gt;It’s not an exhaustive answer to your question, but here some aspects that might be helpful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common problem in supervised learning where you will see KL-divergence used are classification tasks. Very often in those cases the data points in the training set are assumed to belong to a single class &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is from some index set &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;. The class membership is represented by one-hot encoding, which corresponds to a distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$p_i = 1$&lt;/span&gt; for the class the data point belongs to and &lt;span class=&quot;math-container&quot;&gt;$p_j = 0$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;. The loss function mostly used in this problems is the average categorical cross-entropy &lt;span class=&quot;math-container&quot;&gt;$\langle E_P[-log(Q)]\rangle_{\rm data} = \langle H(P) - D(P || Q)\rangle_{\rm data}$&lt;/span&gt;, which in the case of single-class membership reduces to the binary cross-entropy. Because only one probability in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is non-zero in this case, the cross-entropy simplifies to &lt;span class=&quot;math-container&quot;&gt;$\langle -\log(q_i)\rangle_{\rm data}$&lt;/span&gt;. The reverse KL-Divergence &lt;span class=&quot;math-container&quot;&gt;$D(Q || P)$&lt;/span&gt; is strictly speaking not even defined in this case, as &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; needs to be absolute continuous w.r.t. &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are of course cases were class membership is probabilistic also in the training data and the above argument doesn't apply in full. From a more conceptual point of view, &lt;a href=&quot;https://datascience.stackexchange.com/questions/9302/the-cross-entropy-error-function-in-neural-networks&quot;&gt;this thread&lt;/a&gt; summaries nicely the relation of cross-entropy to maximizing the likelihood of the observed data under the model. Using the reverse cross-entropy would maximize the likelihood of a typical sample drawn from the model under the (empirical) distribution of the observed data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A small side note when you read the above link is that in the above described classification problem we have a probability model for the class membership for &lt;em&gt;each&lt;/em&gt; point in the input space, this is why the average cross-entropy is minimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classification problems are just one example of supervised learning, so this answer might not fully cover your question.&lt;/p&gt;&#xA;" OwnerUserId="23746" LastEditorUserId="23746" LastEditDate="2019-04-08T08:19:02.583" LastActivityDate="2019-04-08T08:19:02.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11693" PostTypeId="2" ParentId="11685" CreationDate="2019-04-06T17:29:52.440" Score="2" Body="&lt;p&gt;There are a variety of conditions we can use when deciding whether to prune a sub-tree or not after generating a decision tree model. There are three common approaches. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We can prune branches with less &lt;em&gt;support&lt;/em&gt; than a specific threshold. These are branches which were constructed using very few points from the training data.&lt;/li&gt;&#xA;&lt;li&gt;We can prune branches where the information gain from a split (or any other splitting measure we are using, like GINI), is smaller than a threshold.&lt;/li&gt;&#xA;&lt;li&gt;You can do what is done in &lt;a href=&quot;https://books.google.com/books?id=HExncpjbYroC&amp;amp;pg=PA41&amp;amp;lpg=PA41&amp;amp;dq=Quinlan%20confidence%20factor&amp;amp;source=bl&amp;amp;ots=nMocgYr32m&amp;amp;sig=ACfU3U0L7rhk3hkcT34X48q1AaS2LMOp9A&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=2ahUKEwiptsTn_rvhAhUC8YMKHVADAZsQ6AEwCXoECAkQAQ#v=onepage&amp;amp;q=Quinlan%20confidence%20factor&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quinlan's 4.5 &amp;amp; C5.0 learners&lt;/a&gt; (which are the standard approaches; J48 is another implementation of the same algorithm). Quinlan performs a Chi-squared-like test for the relationship between the attribute we split upon and the target attribute. If the relationship is statistically significant, then the split is preserved. If not, it is not. The &quot;confidence factor&quot; parameter found in most implementations of these algorithms corresponds to the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; value used in determining whether the relationship is considered significant. This approach captures the idea that we should prefer to keep branches that have few datapoints, but a very strong signal, or that have a weak signal, but very many datapoints supporting the pattern, since both cases are less likely to be overfitting that cases where we have weak signals &lt;em&gt;and&lt;/em&gt; small numbers of points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-06T17:29:52.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11694" PostTypeId="1" CreationDate="2019-04-06T18:44:13.170" Score="2" ViewCount="43" Body="&lt;p&gt;I trained a recurrent neural network (if it matters - it contains three CuDNNLSTM cells and 3 Dense layers, Dropout = 0.2). The result of data preparation is one array of ~330.000 sequences. Each contains 256 time steps and 24 features in each time step. This array is normalized, shuffled and balanced. Then it is split into two arrays - train array contains 90% of data (so ~297k) and validation array contains ~10%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During training process (Adam optimizer, 128 or 256 batches) max accuracy of validation data set is 90%. Epoch accuracy is 94%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I run my additional validation test script with more realistic, unbalanced data set. The accuracy of test is just ~50%. I checked this second test if it is correct and there is no errors in the code, but when I run it on data, that was included in the training set, the accuracy was 87%, so it looks good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is going on? I suppose, that wrong architecture is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a graph of epoch accuracy during training:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aBGPz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aBGPz.png&quot; alt=&quot;Epoch accuracy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a graph of validation accuracy during training&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CZAw1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CZAw1.png&quot; alt=&quot;validation accuracy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for support.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today I trained the network one more time. I used the unbalanced data from second test as validation data set in traininge process. You can see the results below.I stopped training after 29 epochs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Blue:&lt;/strong&gt; balanced valitadion data set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Orange:&lt;/strong&gt; unbalanced validation data set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Epoch accuracy:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lXjFm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lXjFm.png&quot; alt=&quot;Epoch accuracy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Validation accuracy&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eKOBj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eKOBj.png&quot; alt=&quot;Validation accuracy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Validation loss&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FB5r0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FB5r0.png&quot; alt=&quot;Validation loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't look good at all.&lt;/p&gt;&#xA;" OwnerUserId="21171" LastEditorUserId="21171" LastEditDate="2019-04-07T15:11:26.233" LastActivityDate="2019-04-07T15:11:26.233" Title="RNN: Different test results on balanced and unbalanced data" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;long-short-term-memory&gt;&lt;intelligence-testing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11695" PostTypeId="2" ParentId="11683" CreationDate="2019-04-06T23:41:24.293" Score="3" Body="&lt;p&gt;It depends what you mean by &quot;what OpenCog can do?&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCog, at a high-level, is a loosely coupled collection of various theoretical and variants of conventional methods aimed at constructing the beginnings of an AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, it's purely applied uses are fairly limited. It can do some typical NLP and ML tasks if used correctly, albeit, it will almost always be ineffective when compared to a problem specific solution. However, this is understandable, as OpenCog is not geared at solving narrow problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCog, as well as Goertzel's other projects(SingularityNet, AGI society), are broad, top-down attempts at formulating an AGI system.&#xA;Outside of AGI circles they are also not oft referenced or researched(although, I am utilizing some aspects of the system in work I am currently crafting). &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-04-06T23:41:24.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11696" PostTypeId="2" ParentId="11674" CreationDate="2019-04-06T23:52:04.330" Score="2" Body="&lt;p&gt;Yes this is standard transfer learning. Using a trained model, we can freeze the first N hidden layers of a classifier, except for the last few. This will allow our previous relevant training to be retained whilst also being able learn new features and target new classes.&lt;/p&gt;&#xA;&lt;p&gt;We will then initialize a new output layer that works our new context(i.e sigmoid, 1 node for binary classifier). Everything is now set to resume training on our new y_targets.&lt;/p&gt;&#xA;&lt;p&gt;Take a look at &lt;a href=&quot;https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; for some more info on transfer learning.&lt;/p&gt;&#xA;&lt;p&gt;If you want no perturbance to your past learning, I would recommend freezing all previous hidden layers and then tacking some additional ones on.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-06T23:52:04.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11701" PostTypeId="1" CreationDate="2019-04-07T13:47:59.910" Score="2" ViewCount="43" Body="&lt;p&gt;There are lot of researches about face detection in pictures, but is it the only way one can say &quot;this person I'm looking for is here in this picture&quot;? Aren't there algorithms that you can provide with information like lateral or back pictures of a person and making calculations on the height, the width, the anatomic distance between parts of its body, the analysis of the hair can determine: &quot;there's a X per cent chance this is the person you're looking for&quot;? Is it possible to accomplish?&lt;/p&gt;&#xA;" OwnerUserId="15764" LastEditorUserId="2444" LastEditDate="2019-04-07T16:37:10.513" LastActivityDate="2019-04-07T16:37:10.513" Title="Algorithms to indentify people in pictures without using face recognition" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11702" PostTypeId="1" CreationDate="2019-04-07T14:06:11.917" Score="1" ViewCount="111" Body="&lt;p&gt;There's been a lot of apocalyptic hype (on the media) about &lt;a href=&quot;http://www.cognimem.com/products/chips-and-modules/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CM1K&lt;/a&gt; and other similar technologies. Elon Musk has voiced fears about it taking over the world, which unfortunately are not unfounded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding of these devices is that there's no software, that they more or less act on their own volition, which makes them almost actually alive. But I've never used one, so that's probably not entirely correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone here ever used a &lt;a href=&quot;https://www.general-vision.com/neuromem/cm1k/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CM1K&lt;/a&gt; (or something similar)? If so, what exactly does it do? &lt;/p&gt;&#xA;" OwnerDisplayName="user23786" LastEditorUserId="9947" LastEditDate="2019-04-09T11:34:26.560" LastActivityDate="2019-04-09T11:34:26.560" Title="What does the CM1K chip (which uses ZISC) exactly do?" Tags="&lt;pattern-recognition&gt;&lt;hardware&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11703" PostTypeId="1" CreationDate="2019-04-07T16:07:04.643" Score="1" ViewCount="43" Body="&lt;p&gt;I have multiple image sequences, each of which contains an animation of two moving dots. The trajectory of the dots in a sequence is always cyclic (not necessarily circular). There are two types of sequences. In some sequences the two dots are moving in phase but in the other sequences they are moving out of phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to classify these two types of sequences using a neural network? What is the simple and feasible neural network structure for this classification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example set of animations. The in-phase patterns are in the above row and the out-of-phase patterns are in the bottom row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/trDhC.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/trDhC.gif&quot; alt=&quot;rotating dots in-phase and out-of-phase&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23789" LastEditorUserId="23789" LastEditDate="2019-04-21T07:46:19.947" LastActivityDate="2019-04-21T07:46:19.947" Title="What is the feasible neural network structure that can learn to identify types of trajectory of moving dots?" Tags="&lt;recurrent-neural-networks&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11704" PostTypeId="1" CreationDate="2019-04-07T16:07:58.180" Score="1" ViewCount="20" Body="&lt;p&gt;A simple word search seems too simple to solve with a computer AI. But what I'm interested is how human's solve it. They build up strategies over the course of solving the puzzle. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) first just look to see if any words &quot;jump out&quot;.&#xA;2) look for the horizontal words.&#xA;3) look for the letter &quot;O&quot;&#xA;4) look for the letter &quot;p&quot; next to a letter &quot;o&quot;&#xA;5) methodically look along rows and columns and diagonals. &#xA;6) cross off words&#xA;7) take the first letter of a word in the list and put it in memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things of that sort. I would like to build such a program. With built in search capabilities, some of which are faster than others. And the AI can combine different search methods and try to solve the puzzle as fast as possible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should also store strategies that work. Or think about why strategies work sometimes and not others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to read a bit more about AI and strategies if you know any good references?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-04-07T16:07:58.180" Title="Has any research been done to solve word searches with AI?" Tags="&lt;ai-design&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11705" PostTypeId="2" ParentId="7926" CreationDate="2019-04-07T17:03:04.133" Score="1" Body="&lt;p&gt;&quot;there's no such thing as true random numbers anyway.&quot; that's all you really need to deter any idea of AI on any computer. Any software or set of functions on a computer is pretty much (right now at least) all set code, by humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the an execution of actions based on variables not listed is not artificial intelligence, it's just simpler code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any REAL Artificial Intelligence will not be made on a board of 1's and 0's, that's defeating the purpose, all of the actions are predetermined (even if they are extended intricately to cover many possibilities) so they have no chance to create something deterministic.&#xA;real independent intellect is most likely (in my eyes) found, not made.&lt;/p&gt;&#xA;" OwnerUserId="23791" LastActivityDate="2019-04-07T17:03:04.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11706" PostTypeId="1" CreationDate="2019-04-07T17:45:38.380" Score="4" ViewCount="42" Body="&lt;p&gt;Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is received only when a particular temporal logic formula is satisfied (LTL or CTL formula). However, i cannot find any work about &lt;em&gt;learning&lt;/em&gt; which rewards correspond to which temporally extended behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my searches, I came across k-order MDPs (which are non-Markovian). I did not find RL research done on k-order MDPs.&lt;/p&gt;&#xA;" OwnerUserId="23792" LastEditorUserId="16920" LastEditDate="2019-04-08T09:23:29.140" LastActivityDate="2019-04-08T09:23:29.140" Title="What research has been done on learning non-Markovian reward functions?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;markov-property&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11707" PostTypeId="2" ParentId="8270" CreationDate="2019-04-07T17:48:56.173" Score="0" Body="&lt;p&gt;In case that you want to connect a placeholder to a new layer you can do as below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example&#xA;&#xA;x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)&#xA;&#xA;x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1&#xA;&#xA;con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you can feed your new layer with above con which is only corresponds from the input 1 to 5.&#xA;you can apply same technique to a layer instead of tf.placeholder&lt;/p&gt;&#xA;" OwnerUserId="23795" LastEditorUserId="30565" LastEditDate="2019-10-26T19:46:35.140" LastActivityDate="2019-10-26T19:46:35.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11708" PostTypeId="1" CreationDate="2019-04-07T19:34:07.950" Score="1" ViewCount="234" Body="&lt;p&gt;If I have a DQN, and I care A LOT about future rewards (moreso than current rewards), can I set gamma to a number greater than 1? Like 1.1 perhaps?&lt;/p&gt;&#xA;" OwnerUserId="23719" LastActivityDate="2019-04-08T06:39:54.933" Title="Can gamma be greater than 1 in a DQN?" Tags="&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11709" PostTypeId="2" ParentId="11708" CreationDate="2019-04-07T21:01:52.513" Score="1" Body="&lt;p&gt;You can't! As if you have a &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; the specified sum for the q-learning will diverge! (goes to infinity in the future steps for &lt;span class=&quot;math-container&quot;&gt;$\gamma^n$&lt;/span&gt;). To know more about that please more scrutinize on the specified formula for the q-learning.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-04-07T21:01:52.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11710" PostTypeId="1" AcceptedAnswerId="11712" CreationDate="2019-04-08T04:40:34.507" Score="2" ViewCount="43" Body="&lt;p&gt;I've been reading a lot of tutorials on DQNs for cartpole. In many of them, they have the funnel layer of the neural net be a linear activation.&#xA;Why is this?&#xA;Is it just a choice made by the implementer?&#xA;Is this Choice specific to cartpole, or do most control task dqns use it?&#xA;Thanks. &lt;/p&gt;&#xA;" OwnerUserId="23803" LastActivityDate="2019-04-08T07:18:07.650" Title="Why do DQNs use linear activations on cartpole?" Tags="&lt;neural-networks&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11711" PostTypeId="2" ParentId="11708" CreationDate="2019-04-08T06:39:54.933" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \gamma $&lt;/span&gt; goes up to 1, but cannot be greater than or equal to 1 (this would make the discounted reward infinite).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discount factor &lt;span class=&quot;math-container&quot;&gt;$ \gamma $&lt;/span&gt;  determines the importance of future rewards. A factor of 0 will make the agent &quot;myopic&quot; (or short-sighted) by only considering current rewards, while a factor approaching 1 will make it strive for a long-term high reward. If the discount factor meets or exceeds 1, the action values may diverge. For &lt;span class=&quot;math-container&quot;&gt;$ \gamma =1$&lt;/span&gt;, without a terminal state, or if the agent never reaches one, all environment histories become infinitely long, and utilities with additive, undiscounted rewards generally become infinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source:&lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Q-learning&lt;/a&gt;&#xA;&lt;a href=&quot;https://cs.stanford.edu/people/karpathy/reinforcejs/puckworld.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cs.stanford.edu/people/karpathy/reinforcejs/puckworld.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21181" LastActivityDate="2019-04-08T06:39:54.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11712" PostTypeId="2" ParentId="11710" CreationDate="2019-04-08T06:54:12.597" Score="2" Body="&lt;p&gt;Q learning predicts the action value, &lt;span class=&quot;math-container&quot;&gt;$q(s, a)$&lt;/span&gt; for taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The action value is usually the discounted sum of all future rewards. In general it can take any scalar value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DQN uses a neural network to approximate &lt;span class=&quot;math-container&quot;&gt;$q(s, a)$&lt;/span&gt;. Although you might use this to select an action (thus think of the problem as a &lt;em&gt;classification&lt;/em&gt;), the NN has to perform &lt;em&gt;regression&lt;/em&gt; to predict the action values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is most common to use a linear final layer, and mean squared error loss in DQN, to match this regression task. So yes, you will find most control DQNs will make the same choice as the cartpole example that you are looking at.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-04-08T07:18:07.650" LastActivityDate="2019-04-08T07:18:07.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11713" PostTypeId="2" ParentId="10406" CreationDate="2019-04-08T07:43:17.503" Score="0" Body="&lt;p&gt;The determination of what each neuron represents is dictated by the initial weights of the network. As you may know, the common practice is to initialize the weights randomly. This means that given the same input data, the network may switch what each neuron means. It may also function differently, taking longer or shorter times to train. However, if you set the random seed to some constant, you can make the network deterministic.&lt;/p&gt;&#xA;" OwnerUserId="23803" LastActivityDate="2019-04-08T07:43:17.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11714" PostTypeId="2" ParentId="9895" CreationDate="2019-04-08T07:46:32.920" Score="1" Body="&lt;p&gt;It is possible! Here is an article by Adobe where they explain how they do it: &lt;a href=&quot;https://theblog.adobe.com/spotting-image-manipulation-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://theblog.adobe.com/spotting-image-manipulation-ai/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm for this would almost certainly be a Convolutional Neural Net trained on a dataset of real and manipulated images (labeled as such).&lt;/p&gt;&#xA;" OwnerUserId="23803" LastActivityDate="2019-04-08T07:46:32.920" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11716" PostTypeId="1" AcceptedAnswerId="11730" CreationDate="2019-04-08T09:08:16.597" Score="6" ViewCount="537" Body="&lt;p&gt;I'm training both DQN and double DQN in the same environment, but DQN performs significantly better than double DQN. As I've seen &lt;a href=&quot;https://arxiv.org/pdf/1509.06461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the double DQN paper&lt;/a&gt;, double DQN should perform better than DQN. Am I doing something wrong or is it possible?&lt;/p&gt;&#xA;" OwnerUserId="22930" LastEditorUserId="2444" LastEditDate="2020-11-04T21:19:05.593" LastActivityDate="2021-01-25T16:28:34.750" Title="Can DQN perform better than Double DQN?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11717" PostTypeId="1" CreationDate="2019-04-08T10:04:21.293" Score="3" ViewCount="180" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1507.06527&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Recurrent Q-Learning for Partially Observable MDPs&lt;/a&gt;, the DRQN is described as DQN with the first &lt;em&gt;post-convolutional&lt;/em&gt; fully-connected layer replaced by a recurrent LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have DQN implementation with only two dense layers. I want to change this into DRQN with the first layer as an LSTM and leave the second dense layer untouched. If I understood correctly, I would also need to change the input data appropriately. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other things that need to be modified in order to make DRQN work?&lt;/p&gt;&#xA;" OwnerUserId="22162" LastEditorUserId="2444" LastEditDate="2019-04-09T17:16:00.457" LastActivityDate="2019-04-09T17:16:00.457" Title="What can be considered a deep recurrent neural network?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;long-short-term-memory&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11718" PostTypeId="1" AcceptedAnswerId="11721" CreationDate="2019-04-08T13:09:17.977" Score="2" ViewCount="194" Body="&lt;p&gt;I am currently using Nvidia GTX1050 with 640 CUDA cores and 2GB GDDR5 for Deep Neural Network training. I want to buy a new GPU for training, but I am not sure how much performance improvement I can get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is a way to roughly calculate the training performance improvement by just comparing GPUs' specification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming all training parameters are the same. I wonder if I can roughly assume the training performance improvement is X times because the CUDA core number and memory size increased X times?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, Is RTX2070 with 2304 CUDA cores and 8GB GDDR6 roughly 4 times faster than GTX1050? And is RTX2080Ti with 4352 CUDA cores and 11GB GDDR6 roughly 7 times faster than GTX1050?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="21213" LastActivityDate="2019-04-10T10:39:20.223" Title="Can I calculate the training performance of GPUs by comparing their specification?" Tags="&lt;training&gt;&lt;gpu&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11719" PostTypeId="1" AcceptedAnswerId="11722" CreationDate="2019-04-08T13:21:05.430" Score="0" ViewCount="67" Body="&lt;p&gt;I'm a bit of a CNN newbie, and I'm trying to train one to image classify pictures of pretty similar looking particles. I'm making the inputs and labels by hand from a set of 48x48 grayscale images, and labeling them with a one-hot vector based on their position in the sequence (for example, the 400/1000th image might have a one-hot in the 4th position if I have 10 categories in the run). I'm using sigmoidal output activation and categorical cross entropy loss. I've played around with a few different optimizers, as well. I'm implementing in python keras. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, although I have pretty good accuracy numbers for the training and validation, when I actually look at the outputs being produced, it generally gives multiple categories, which is not at all what I want. For example, if I have 6 categories and a label of 3, it might give the following probability vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[ .99 .98  1.0  .99  0.02  0.05 ]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was my understanding that categorical cross entropy would not allow this type of categorization, and yet it is prevalent in my code. I am under the impression that I'm doing something fundamentally wrong, but I cant figure out what. Any help would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="23812" LastEditorUserId="22916" LastEditDate="2019-04-11T21:02:53.103" LastActivityDate="2019-04-11T21:02:53.103" Title="CNN output generally has more than one category in one-hot categorization?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;training&gt;&lt;categorical-data&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11720" PostTypeId="2" ParentId="6699" CreationDate="2019-04-08T13:28:38.760" Score="0" Body="&lt;p&gt;I mean you can predict these sequences quite easily(with varying levels of accuracy) just by using LSTMs in the time series forecasting context. Obviously, as the number of digits you give it increases, it will increase the prediction accuracy of the next in the sequence(with some caveats), as we can think of neural networks more generally as connectionist function approximators(nonlinear in almost all cases). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as direct applications in AI, I suppose not beyond mathematical modeling and economic/financial modeling as these sorts of sequences emerge from a vast majority of pure and applied mathematical concepts. This research is quite relevant and ongoing(&lt;a href=&quot;https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/95WR01955&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;,&lt;a href=&quot;https://content.iospress.com/articles/algorithmic-finance/af176&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;,&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1143891&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2019-04-08T15:36:51.133" LastActivityDate="2019-04-08T15:36:51.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11721" PostTypeId="2" ParentId="11718" CreationDate="2019-04-08T13:32:50.140" Score="2" Body="&lt;p&gt;A lot matters when it comes to comparison of GPU's, I will give you a broad overview of the matter (it is not possible to go into exact details as huge number of factors are actually involved):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cores&lt;/strong&gt; - Number of CUDA cores increases means the parallelism has increased, thus multiple calculations can be done in parallel &lt;strong&gt;but&lt;/strong&gt; is of no significance if your algorithm is inherently sequential. Then CUDA cores will not matter. Your library will parrallelize what it can parallelize and will take that many CUDA cores only, the rest will remain idle.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Memory&lt;/strong&gt; - Memory is useful if you are working on data whose one instance requires huge memory (like pictures). Thus with greater memory you can load greater amount of data at the same time and the cores will process on that. If memory is too low then cores want data but it is not getting it (basically data available in RAM's is the fuel while cores are the engine, you cannot run jets on fuel tank the size of car, it will consume time to constantly refill the fuel). But according to convention of Machine Learning one should load only small mini-batches at a time.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Micro-architecture&lt;/strong&gt; - Lastly, architecture matters. I do not exactly know how but NVIDIA's RTX is faster for deep learning than GTX. NVIDIA has two affordable architectures (Pascal - GTX and Turing - RTX). Thus even for exactly same specs Turing architecture will run faster for deep learning. But for more details you can explore NVIDIA's website on what architecture specialises in what. For example NVIDIA P-series is good for CAD purposes. Also there are some very high end GPU's using Tesla architecture.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So AFAIK, these are the factors that matter. The library you will be using also matters, as a lot depends on how the library unrolls your program and maps them on several GPU's. Also related these 2 answers I previously gave:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5452/cpu-preferences-and-specifications-for-a-multi-gpu-deep-learning-setup/5456#5456&quot;&gt;CPU preferences and specifications for a multi GPU deep-learning setup&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/10546/does-fp32-fp64-performance-of-gpu-affect-deep-learning-model-training/10547#10547&quot;&gt;Does fp32 &amp;amp; fp64 performance of GPU affect deep learning model training?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-04-10T10:39:20.223" LastActivityDate="2019-04-10T10:39:20.223" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11722" PostTypeId="2" ParentId="11719" CreationDate="2019-04-08T14:41:06.667" Score="1" Body="&lt;p&gt;When you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-04-08T14:41:06.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11724" PostTypeId="1" CreationDate="2019-04-08T15:59:38.870" Score="0" ViewCount="66" Body="&lt;p&gt;It must be a very stupid question, but since I have not such sufficient know ledge storage and having no more time to search the answer of it, I have to put it here to ask for help. I generated a training dataset of images of simple geometric shapes as triangles, squares, diamonds etc. by programs and constructed a CNN with two convolutional layers and one pooling layer also a final fully connected layer to learn the classifications of these shapes.&#xA;But the network just does not to learn it. I mean the loss just does not decrease. What is the cause?&lt;/p&gt;&#xA;" OwnerUserId="14948" LastActivityDate="2019-04-08T15:59:38.870" Title="CNN doesn't learning simple geometric patterns" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11725" PostTypeId="1" CreationDate="2019-04-08T16:11:48.077" Score="0" ViewCount="35" Body="&lt;p&gt;How can the bias, &lt;span class=&quot;math-container&quot;&gt;$b_0$&lt;/span&gt;, and the coefficient for the single input value, &lt;span class=&quot;math-container&quot;&gt;$b_1$&lt;/span&gt;, be calculated in logistic regression?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to calculate these coefficients with pure Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody provide some example or formulas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Problem was solved with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://machinelearningmastery.com/logistic-regression-for-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/logistic-regression-for-machine-learning/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.stat.cmu.edu/~cshalizi/402/lectures/14-logistic-regression/lecture-14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.stat.cmu.edu/~cshalizi/402/lectures/14-logistic-regression/lecture-14.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21708" LastEditorUserId="21708" LastEditDate="2019-04-08T20:06:40.873" LastActivityDate="2019-04-08T20:06:40.873" Title="How can the bias and the coefficient be calculated in logistic regression?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11726" PostTypeId="1" AcceptedAnswerId="11727" CreationDate="2019-04-08T16:29:47.503" Score="1" ViewCount="303" Body="&lt;p&gt;I am new to deep learning. I have doubts on modifying bias values during back propagation. My doubts are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does the back propagation algorithm modifies the weigh values and bias values in the same pass?&lt;/li&gt;&#xA;&lt;li&gt;How does the algorithm decide whether it has to change the weight value or bias value to reduce the error in a pass?&lt;/li&gt;&#xA;&lt;li&gt;Will the learning rate same for bias and weights? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="23734" LastActivityDate="2019-04-08T16:57:43.260" Title="When is bias values updated in back propagation?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11727" PostTypeId="2" ParentId="11726" CreationDate="2019-04-08T16:51:59.377" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the back propagation algorithm modifies the weigh values and bias values in the same pass?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How does the algorithm decide whether it has to change the weight value or bias value to reduce the error in a pass?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It differentiates the loss function (like MSE) with respect to the weights and biases, that is, it finds the &lt;em&gt;partial derivative&lt;/em&gt; of the loss function with respect to each of the parameters. You can think of the partial derivative of the loss function with respect to one of the parameters of the model as representing the &quot;contribution&quot; of that parameter to the loss of the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The partial derivatives of the loss function with respect to each of the parameters (weights and biases) is collectively called the &lt;em&gt;gradient&lt;/em&gt;, which is thus a vector of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; partial derivatives, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of parameters of the model.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will the learning rate same for bias and weights?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is usually the same, but, in theory, nobody prevents you from updating the biases using a different learning rate, so you could use a different learning rate to update the biases and weights. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-08T16:57:43.260" LastActivityDate="2019-04-08T16:57:43.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11728" PostTypeId="2" ParentId="11716" CreationDate="2019-04-09T04:00:04.243" Score="0" Body="&lt;p&gt;That may happen when the value of the state is bad. You can find the example and explain about that in the link below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this:&lt;a href=&quot;https://medium.freecodecamp.org/improvements-in-deep-q-learning-dueling-double-dqn-prioritized-experience-replay-and-fixed-58b130cc5682&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.freecodecamp.org/improvements-in-deep-q-learning-dueling-double-dqn-prioritized-experience-replay-and-fixed-58b130cc5682&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21181" LastActivityDate="2019-04-09T04:00:04.243" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11729" PostTypeId="1" CreationDate="2019-04-09T05:10:55.027" Score="2" ViewCount="72" Body="&lt;p&gt;Gradient descent can get stuck into local optimum. Which techniques are there to reach global optimum?&lt;/p&gt;&#xA;" OwnerUserId="23501" LastEditorUserId="2444" LastEditDate="2019-04-27T16:18:50.447" LastActivityDate="2019-04-27T16:18:50.447" Title="How can we reach global optimum?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11730" PostTypeId="2" ParentId="11716" CreationDate="2019-04-09T07:35:29.923" Score="1" Body="&lt;p&gt;There is no thorough proof, theoretical or experimental that Double DQN is better then vanilla DQN. There are a lot of different tasks, paper and later experiments only explore some of them. What practitioner can take out of it is that &lt;em&gt;on some tasks&lt;/em&gt; DDQN is better. That's the essence of Deep Mind's &quot;Rainbow&quot; approach - drop a lot of different methods into bucket and take best results. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-04-09T07:35:29.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11731" PostTypeId="2" ParentId="11729" CreationDate="2019-04-09T07:44:20.537" Score="1" Body="&lt;p&gt;In Deep Learning there are several methods to improve &quot;stuck&quot; gradient - decrease learning rate, use cyclic learning rate - cycle it from bigger to smaller value. More radical method is completely reinitialize last or two last (before loss) layers of the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In non-Deep Learning ML out of those only decrease learning rate will work, but there is plethora Numerical Optimization methods to help, like second-order methods - variation of Gauss-Newton, or methods specific to the problem which may include projective methods, alternate directions, conjugate gradients &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization#Optimization_algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;etc&lt;/a&gt;. There are a lot of methods which are better then gradient descent for non-Deep Learning optimization.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-04-09T08:35:22.947" LastActivityDate="2019-04-09T08:35:22.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11732" PostTypeId="1" CreationDate="2019-04-09T08:59:57.053" Score="1" ViewCount="17" Body="&lt;p&gt;yesterday I tried to transform a picture in the artistic style using &lt;strong&gt;CNNs based on A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge&lt;/strong&gt; using a recent Torch implemenation,as it is explained here :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mbartoli/neural-animation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mbartoli/neural-animation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it started the conversion correctly,the problem is that the process is very time consuming,after 1 hour of elaboration a simple picture was not fully transformed. And I have to trasform &lt;strong&gt;1615&lt;/strong&gt; pictures. What's the solution here ? Can I use the &lt;strong&gt;Google Cloud Platform&lt;/strong&gt; to make this operation faster ? Or some other kind of &lt;strong&gt;Cloud&lt;/strong&gt; service ? Using my Home PC is not the right solution. If I can use the cloud power,how can I configure everything ? let me know,thanks.&lt;/p&gt;&#xA;" OwnerUserId="23825" LastActivityDate="2019-04-09T09:23:06.560" Title="Using the cloud service to trasform a picture using a neural algorithm?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;google-cloud&gt;&lt;cloud-services&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11738" PostTypeId="1" CreationDate="2019-04-09T15:33:33.017" Score="2" ViewCount="19" Body="&lt;p&gt;I want to train a neural network to predict what my favourite home-work route will be for a particular day. I have these features for routes on a day: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;temperature, humidity, congestion, distance, duration&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have come up with this concept of training/testing a network:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//&#xA;// training features result in route 1,2 or 3&#xA;//&#xA;Network.train([30,10,12,20,12] , 1)&#xA;Network.train([20,10,22,20,14] , 3)&#xA;Network.train([23,10,2,20,10] , 2)&#xA;Network.train([20,10,22,20,12] , 2)&#xA;&#xA;//&#xA;// On a new day, predict which route the user is most likely to take:&#xA;//&#xA;var route = Network.test([25,8,12,22,12])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is: is this a viable approach? Can I make relevant predictions this way if I have enough training data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I generate an outcome between 1 and 3 this way? &lt;/p&gt;&#xA;" OwnerUserId="11620" LastEditorUserId="1671" LastEditDate="2019-11-06T21:34:41.257" LastActivityDate="2019-11-06T21:34:41.257" Title="How to predict a preferred route based on weather and distance" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11739" PostTypeId="2" ParentId="1885" CreationDate="2019-04-09T16:05:32.497" Score="0" Body="&lt;p&gt;my hunch (and this is strictly a hunch) is that building a human brain on a chip is actually alot easier than you might think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my pet theory is that biological neurons are horribly slow, clumsy, and error-prone devices (at least mines are :lol:), but that the human brain overcomes this limitation by increasing the degree of parallelism several orders of magnitude over the current chip technology; and to that end it requires ~1.0e+11 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but the chip removes these limitations, and when the neurons have instantaneous relays, then you dont need nearly so many of them. if thats correct, then a human brain on a chip could probably run in only a few million neurons as opposed to 1.0e+11 inside the skull.&lt;/p&gt;&#xA;" OwnerDisplayName="user23786" LastActivityDate="2019-04-09T16:05:32.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11740" PostTypeId="1" CreationDate="2019-04-09T17:11:03.973" Score="3" ViewCount="111" Body="&lt;p&gt;Could a better algorithm other than Monte Carlo be used for the AlphaGo computer? Why didn't the DeepMind team think of choosing another kind of algorithm rather than spending time on their neural nets?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastEditorUserId="21832" LastEditDate="2019-04-09T23:44:08.303" LastActivityDate="2019-06-10T20:31:40.723" Title="Why is Monte Carlo used as the tree search algorithm for AlphaGo?" Tags="&lt;monte-carlo-tree-search&gt;&lt;alphago&gt;&lt;monte-carlo-methods&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11741" PostTypeId="1" CreationDate="2019-04-09T17:21:45.803" Score="4" ViewCount="44" Body="&lt;p&gt;I'm working in a computer vision project, where the goal is to detect some specific parasites, but now that I have the images, I noticed that they have a watermark that specifies the microscope graduation.&#xA;I have some ideas of how to remove this noise, like detecting the numbers and replace for the most common background or split the image but if I split the image I'll lose information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would like to hear some recommendations and guidelines of experts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I added an example image below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lU8LQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lU8LQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23836" LastEditorUserId="23836" LastEditDate="2019-04-09T17:34:49.240" LastActivityDate="2019-04-09T17:34:49.240" Title="How do I denoize a microscopic image?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11743" PostTypeId="1" CreationDate="2019-04-09T19:10:46.760" Score="2" ViewCount="436" Body="&lt;p&gt;I've decided to make my bachelor thesis in RL. I am currently struggling to find a good problem. I am interested in multi-agent RL with the dilemma between selfishness and cooperation.&lt;/p&gt;&#xA;&lt;p&gt;I only have 2 months to complete this and I'm afraid that multi-agent RL is too difficult and I don't have the knowledge and time to nicely learn this topic.&lt;/p&gt;&#xA;&lt;p&gt;What are some simple open problems in multi-agent reinforcement learning that would be suited for a bachelor's thesis?&lt;/p&gt;&#xA;&lt;p&gt;I've only done applied the Q-learning algorithm to solve a text-based environment in OpenAI's gym.&lt;/p&gt;&#xA;" OwnerUserId="23838" LastEditorUserId="2444" LastEditDate="2021-01-23T00:24:38.280" LastActivityDate="2021-01-23T00:24:56.177" Title="What are some simple open problems in multi-agent RL that would be suited for a bachelor's thesis?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;research&gt;&lt;academia&gt;&lt;multi-objective-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11744" PostTypeId="1" CreationDate="2019-04-09T19:59:37.873" Score="3" ViewCount="33" Body="&lt;p&gt;How one can model physiological reward mechanisms occuring in the brain using artificial neural networks? E.g. are there efforts to use the notion of dopamine or similar substances in the artificial neural networks. Maybe introduction of the physiological reward mechanism can lead to the emergence of consciousness or at least enhance the effectiveness of reinforcement learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially - how neural network models reward? People's brain perceive money as the ultimate reward because almost everything other can be bought by this. So - mental perception of owning money gives reward. But how this notion of reward is modeled in artificial neural networks? How networks know that some money is assigned to the network's account and so, the network should feel happy and rewarded and should strive to repeat successful behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am reading &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5293493/pdf/elife-21492.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5293493/pdf/elife-21492.pdf&lt;/a&gt; and I hope that it will move me in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is quite confusing. The old-school neural networks expect that there are 2 separate phases: training and inference. So, the network receives all the feedback (let it be called reward) in the training phase and network receives nothing in the inference phase. But maybe network should receive some reward during acting-inference phase as well, kind a lifelong learning.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2019-04-09T20:19:47.167" LastActivityDate="2019-04-09T20:19:47.167" Title="Models of reward (possibly mimicking dopamine) in artificial neural networks?" Tags="&lt;neural-networks&gt;&lt;brain&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11745" PostTypeId="1" CreationDate="2019-04-09T20:55:12.253" Score="3" ViewCount="28" Body="&lt;p&gt;One Nash equilibrium of every GANs model has is when the generator creates  perfect samples indistinguishable from the training data and the discriminator just output 1 with probability 1/2. And I think this is the desirable outcome since we are most interested in the generator part of the GAN model. I know that we probably try to converge to this equilibrium with some hacks in training such as &quot;mode collapse avoidance&quot; and so on. But is there any theoretical work trying to go in another ways (say, by reduce the number of Nash equilibria somehow?)&lt;/p&gt;&#xA;" OwnerUserId="23688" LastActivityDate="2019-04-09T20:55:12.253" Title="How do we ensure that training GANs will fall in the desirable Nash equilibrium?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11746" PostTypeId="1" CreationDate="2019-04-09T23:51:08.227" Score="2" ViewCount="33" Body="&lt;p&gt;How can we use the ability of AlphaGo Zero computer, to do something in any other life important related field? Is it possible to make something important besides having created something so smart that can play mind games way better than humans?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastActivityDate="2019-04-10T07:04:46.153" Title="How do the achievements met in the gaming field (ex. AlphaGo Zero) impact other fields of application?" Tags="&lt;machine-learning&gt;&lt;gaming&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11747" PostTypeId="1" CreationDate="2019-04-10T02:18:19.513" Score="3" ViewCount="22" Body="&lt;p&gt;I've a rather simple question for a school project. We're developing a GA solution for the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chromosome: A location with lat-lon coords. There are two types of locations - up to 15 waypoints from user input, and a dataset of about 3-400 stations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gene: A route consisting of all waypoints (incl. a fixed start and end) + 1 station.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fitness function: Shortest path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stop condition: Run duration - configurable, default 3 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're discussing two possible implementations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A problem set of all waypoints and &lt;strong&gt;all&lt;/strong&gt; stations, kinda like soccer team assignment or nurse rostering design. Run GA &lt;strong&gt;once&lt;/strong&gt; on all of that.&lt;/li&gt;&#xA;&lt;li&gt;A problem set of all waypoints and &lt;strong&gt;one&lt;/strong&gt; station, do TSP. Run GA for &lt;strong&gt;number-of-stations-in-dataset&lt;/strong&gt; iterations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which is better, in terms of design, efficiency, and performance?&lt;/p&gt;&#xA;" OwnerUserId="23844" LastActivityDate="2019-04-10T02:18:19.513" Title="Shortest route GA: One loop through one dataset vs multiple loops through subsets of the same data?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11748" PostTypeId="1" CreationDate="2019-04-10T05:11:27.350" Score="2" ViewCount="1044" Body="&lt;p&gt;I have set of topics generated using LDA and  like {code, language, test , write, function}, {class, public, method, string, int} etc and I want to make meaningful sentence/sentences from these words using  api or libraries.  How do I implement this with NLTK and(or) Machine Learning? Any suggestions as to how I should go about this?&lt;/p&gt;&#xA;" OwnerUserId="23509" LastActivityDate="2020-01-05T12:03:00.353" Title="How to make meaningful sentences from a set of words?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11749" PostTypeId="2" ParentId="11746" CreationDate="2019-04-10T07:04:46.153" Score="1" Body="&lt;p&gt;Yes it's created something important. Until Alpha(Go) Zero all (or almost all) of &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; approach to &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning&lt;/a&gt; was based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Temporal_difference_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Time Difference&lt;/a&gt; loss function. The weakness of Time Difference loss function was that it was essentially training on itself, that is data produced by the same method was used as part of &lt;a href=&quot;https://en.wikipedia.org/wiki/Nonlinear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;regression&lt;/a&gt; target. That was producing the problem of &quot;extrapolation error&quot; - solution would blow up, or oscillate wildly. There was attempts to mitigate that problem (n-steps algorithms), but they weren't helping much. Alpha Zero combined Deep Network with tree search instead (&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search&lt;/a&gt;). Tree search algorithm produced wide and long fields of high precision data (&lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning#Value_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;value function&lt;/a&gt;), with network influence on value much diminished. That way network was training mostly not on itself, but on data produced by tree, and tree search itself was accelerated greatly by network (using it as heuristics). The whole happens to be much more then sum of parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is not limited to broad game or RL theory. It may work for any problem for which high-precision simulator could be built. Essentially if problem allow Monte Carlo Tree Search, or other tree search which could be augmented with heuristic, Alpha Zero approach would probably work on it. Of cause Alpha Zero approach is computationally expensive, so it's not always efficient to apply it.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-04-10T07:04:46.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11750" PostTypeId="2" ParentId="3548" CreationDate="2019-04-10T08:37:10.757" Score="0" Body="&lt;p&gt;As they suggested good resources and there are many resources but I will recommend you to start with &lt;a href=&quot;https://www.forbes.com/sites/bernardmarr/2019/03/04/what-is-the-best-book-on-artificial-intelligence-ai/#522d65781674&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Is The Best Book On Artificial Intelligence (AI)? &lt;/a&gt; , &lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/ml-intro&quot; rel=&quot;nofollow noreferrer&quot;&gt;&#xA;Introduction to Machine Learning&lt;/a&gt;  and &lt;a href=&quot;https://github.com/davidbailey/tum/blob/master/handbook/README.md#artificial-intelligence---machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence - Machine Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this video  summary which knowledge of math that you need &lt;a href=&quot;https://youtu.be/8onB7rPG4Pk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematics of Machine Learning&lt;/a&gt; And this link explain &lt;a href=&quot;https://towardsdatascience.com/mathematics-for-ai-all-the-essential-math-topics-you-need-ed1d9c910baf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematics for AI: All the essential math topics you need&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, you can see these links, these my favorite &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=w2OtwL5T1ow&amp;amp;list=PLE6Wd9FR--EdyJ5lbFl8UuGjecvVw66F6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to machine learning and the Slides available&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;list=PL7-jPKtc4r78-wCZcQn5IqyuWhBZ8fOxT&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning Course by David Silve and the Slides available&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=vOppzHpvTiQ&amp;amp;list=PL2-dafEMk2A7YdKv4XfKpfbTH5z6rEEj3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intro to Deep Learning! by Siraj Raval and the codes available&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/channel/UCfzlCWGWYyIQ0aLC5w48gBQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Machine Learning Tutorial with Python and the codes available&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="21181" LastEditDate="2019-04-13T19:08:23.863" LastActivityDate="2019-04-13T19:08:23.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11752" PostTypeId="1" CreationDate="2019-04-10T09:17:39.993" Score="0" ViewCount="46" Body="&lt;p&gt;I want to create a semantic search for a set documents with terms that could not appear on the set. Is there some code for that? I'm working on python.&lt;/p&gt;&#xA;" OwnerUserId="23854" LastActivityDate="2019-04-10T09:17:39.993" Title="Semantic search engine for a set of documents in python" Tags="&lt;natural-language-processing&gt;&lt;python&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11755" PostTypeId="1" CreationDate="2019-04-10T10:02:36.443" Score="3" ViewCount="174" Body="&lt;p&gt;Before the release of BERT, we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that's being predicted to indirectly &quot;see itself&quot; in a multi-layer model. How does this happen? &lt;/p&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="2444" LastEditDate="2019-11-01T03:00:51.950" LastActivityDate="2019-11-01T03:00:51.950" Title="How does bidirectional encoding allow the predicted word to indirectly &quot;see itself&quot;?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11756" PostTypeId="2" ParentId="11748" CreationDate="2019-04-10T10:32:35.160" Score="1" Body="&lt;p&gt;How do you define &quot;meaningful&quot;? Generally, you would start from concepts and meanings, and then realise them in syntactic structures using lexical items (words). You seem to want to start in the middle somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For turning a semantic representation into a valid sentence, you would use a generator; these are often based on grammars. Examples exist which take a grammar, fill in random words, and create a syntactically well-formed sentence; often they will, however, be rather non-sensical or meaningless. Have a look at this site which &lt;a href=&quot;http://zompist.com/syntax.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;describes the Syntax Construction Kit&lt;/a&gt;. The author, Mark Rosenfelder, links to a number of toy programs which do exactly that. Just substitute his lexicon with the list of words created by your LDA process. See for example this &lt;a href=&quot;http://zompist.com/gtg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;generator based on generative grammar&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-04-10T10:32:35.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11757" PostTypeId="2" ParentId="11678" CreationDate="2019-04-10T12:18:15.757" Score="1" Body="&lt;p&gt;In your case you can use either RNN (especially BiLSTM with ELMo and attention mechanism for better accuracy) or Transformer based architectures (the best of them today is BERT). But for both cases you need data to train the model (i.e sequences of input/output like in your question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the best choice in your case is BERT as it's achieving state of the art performance in most NLP tasks and is already pretrained so you don't need a massive data to retrain the model. Also, BERT is pretrained on &quot;Next Sentence Prediction&quot; and allows &quot;2 separate sentences&quot; as input which helps a lot in your case. The only drawback, compared to other methods, is that you need fine-tuning the model so it's not a completly &quot;ready to use&quot; model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;For more information:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the github repository of BERT: &lt;a href=&quot;https://github.com/google-research/bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT-repo&lt;/a&gt;. For quick explanation check: &lt;a href=&quot;http://jalammar.github.io/illustrated-bert/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Illustrated BERT&lt;/a&gt;. For detailed explanation see BERT paper: &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT-paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="23350" LastEditDate="2019-04-10T14:02:29.650" LastActivityDate="2019-04-10T14:02:29.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11759" PostTypeId="1" AcceptedAnswerId="16437" CreationDate="2019-04-10T13:53:28.567" Score="2" ViewCount="1324" Body="&lt;p&gt;Let's suppose I have a set of heuristics &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; = {&lt;span class=&quot;math-container&quot;&gt;$h_1, h_2, ..., h_N$&lt;/span&gt;}.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If all heuristics in &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; are &lt;strong&gt;admissible&lt;/strong&gt;, does that mean that a heuristic that takes the &lt;span class=&quot;math-container&quot;&gt;$\min(H)$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$\max(H)$&lt;/span&gt; for that matter) is also admissible?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If all heuristics in &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; are &lt;strong&gt;consistent&lt;/strong&gt;, does that mean that a heuristic that takes the &lt;span class=&quot;math-container&quot;&gt;$\min(H)$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$max(H)$&lt;/span&gt; for that matter) is also consistent?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm thinking about a search problem in a bi-dimensional grid that every iteration of an algorithm, the agent will have to find a different goal. Therefore, depending on the goal node, a certain heuristic can possibly better guide the agent than the others (hence the use of &lt;span class=&quot;math-container&quot;&gt;$\min$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="22369" LastEditorUserId="2444" LastEditDate="2019-11-11T15:25:12.967" LastActivityDate="2019-11-11T17:03:57.083" Title="Is the minimum and maximum of a set of admissible and consistent heuristics also consistent and admissible?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;heuristics&gt;&lt;admissible-heuristic&gt;&lt;consistent-heuristic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11760" PostTypeId="1" AcceptedAnswerId="11764" CreationDate="2019-04-10T15:42:57.157" Score="3" ViewCount="884" Body="&lt;p&gt;I am training a modified VGG16 network for classification (adding 0.5 dropout after each of the last FC layers). In the following plot I am training for a small number of epochs as an example, and it shows the accuracy and loss curves of training process on both training and validation datasets. My training set size is &lt;span class=&quot;math-container&quot;&gt;$1725$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$429$&lt;/span&gt; for validation. Also I am training with &lt;code&gt;weights=None&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZZGbM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZZGbM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about the validation curves, why do not they appear to be as smooth as the training ones? Is this normal during the training stage?&lt;/p&gt;&#xA;" OwnerUserId="23268" LastEditorUserId="9947" LastEditDate="2019-04-10T17:57:04.163" LastActivityDate="2019-04-10T17:57:04.163" Title="Why are not validation accuracy and loss as smooth as train accuracy and loss?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11761" PostTypeId="1" CreationDate="2019-04-10T16:18:36.993" Score="3" ViewCount="82" Body="&lt;p&gt;I'm trying to tackle the problem of feature selection as an RL problem, inspired by the paper &lt;a href=&quot;https://hal.inria.fr/inria-00484049/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feature Selection as a One-Player Game&lt;/a&gt;. I know Monte-Carlo tree search (MCTS) is hardly RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I used MCTS for this problem, where nodes are subsets of features and edges are (&quot;adding a feature to the subset&quot;) and it does converge to the optimal subset slowly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a clever way to speed up the convergence of MCTS, besides parallelizing the rollout phase?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adding nodes to the tree takes time and memory for datasets with a large number of features, for 10000 features, it takes up all my RAM (8GB) from the second iteration, (although it runs for 2000+ iterations for a dataset with 40 features which doesn't make sense to me). Is this expected or is my implementation likely wrong? Are there any workarounds for this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are your opinions on using MCTS for this task? Can you think of a better approach? The main problem of this approach is running the SVM as an evaluation function which may make the algorithm impractically slow for large datasets (a large number of training examples).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I was thinking of trying to come up with a heuristic function to evaluate subsets instead of the SVM. But I'm kind of lost and don't know how to do that. Any help would be really appreciated. &lt;/p&gt;&#xA;" OwnerUserId="23866" LastEditorUserId="2444" LastEditDate="2019-04-10T17:21:30.833" LastActivityDate="2019-04-10T17:21:30.833" Title="Feature Selection using Monte Carlo Tree Search" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;feature-selection&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11762" PostTypeId="1" CreationDate="2019-04-10T17:33:43.483" Score="2" ViewCount="133" Body="&lt;p&gt;What are the key differences between &lt;a href=&quot;https://en.wikipedia.org/wiki/Cellular_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;cellular neural networks&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolutional neural networks&lt;/a&gt; in terms of working principle, implementation, potential performance, and applicability?&lt;/p&gt;&#xA;" OwnerUserId="23868" LastEditorUserId="22916" LastEditDate="2019-04-11T01:58:06.170" LastActivityDate="2019-04-11T01:58:06.170" Title="What are the key differences between cellular neural network and convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11763" PostTypeId="1" AcceptedAnswerId="11765" CreationDate="2019-04-10T17:54:27.137" Score="4" ViewCount="127" Body="&lt;p&gt;I want to make a network, specifically a CNN for image recognition, that takes an input, processes it the same way for several layers, and then at some point splits before coming to two different outputs. Is it possible to create a network such as this? It would look something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input -&gt; Conv -&gt; Pool -&gt; Conv -&gt; Pool ---------&gt; Dense -&gt; Output 1 &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                  ||&#xA;&#xA;                                     ----&amp;gt; Dense -&amp;gt; Output 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I.E. it splits off after the second pooling layer into separate fully connected layers. Of course, it has to train to both outputs, so that it is producing minimal error on both separate outputs using these common convolutional layers. Also, I am using Python Keras, and it would help if there was some way to do this using Keras in some way. Thank you!  &lt;/p&gt;&#xA;" OwnerUserId="23812" LastActivityDate="2019-04-11T05:17:19.863" Title="Is it possible to make a 'forked path' neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11764" PostTypeId="2" ParentId="11760" CreationDate="2019-04-10T17:55:39.697" Score="3" Body="&lt;p&gt;You are training your model on the train set and only validating your model on CV set, thus your weights are getting exclusively optimised according to the loss of Training Set (in a continuous manner) and thus always decreasing. We do not have such guarantees with the CV set, which is the entire purpose of Cross Validation in the first place. Ideally it gives you an unbiased measure of how well your model and its trained weights will perform in the real world. Thus even if it performs well in Training Set, the loss can still go up in CV set which is what you are seeing in your graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speaking in layman terms, even if you do all the sums of a single exercise given in a book, your performance might not be the same in a model paper. You do another exercise from another book, there is no guarantee your previous concepts will stick and you may do even worse in the model paper. Same thing is happening here where exercises are your training set and model papers are to evaluate your learning.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-04-10T17:55:39.697" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11765" PostTypeId="2" ParentId="11763" CreationDate="2019-04-10T19:21:09.883" Score="4" Body="&lt;p&gt;Keras Functional APIs can help you define complex models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the documentation here :&#xA;&lt;a href=&quot;https://keras.io/getting-started/functional-api-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/getting-started/functional-api-guide/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# prev_layer is the layer you want to be forked&#xA;fork1 = Dense(32, activation='relu')(prev_layer)&#xA;fork2 = Dense(32, activation='relu')(prev_layer)&#xA;&#xA;# you do some operations on fork1 to get output1&#xA;# and on fork2 to get output2&#xA;&#xA;model = Model(inputs=input_layer, outputs=[output1, output2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21229" LastEditorUserId="21229" LastEditDate="2019-04-11T05:17:19.863" LastActivityDate="2019-04-11T05:17:19.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11766" PostTypeId="1" CreationDate="2019-04-10T19:22:24.713" Score="2" ViewCount="33" Body="&lt;p&gt;Please read the following &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/svm/plot_svm_scale_c.html#sphx-glr-auto-examples-svm-plot-svm-scale-c-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; of the Sklearn documentation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The figure shown there (see below) illustrates why C should be scaled when using a SVM with 'l1' penalty, whereas it shouldn't be scaled C when using one with 'l2' penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bWQZr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bWQZr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scaling however does not change the scores of the models examined within the GridSearch. So what exactly is this scaling-step good for?&lt;/p&gt;&#xA;" OwnerUserId="23672" LastActivityDate="2019-04-10T19:22:24.713" Title="What is the benefit of scaling the hyperparameter C of an SVM?" Tags="&lt;hyper-parameters&gt;&lt;regularization&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11768" PostTypeId="1" CreationDate="2019-04-11T02:34:42.343" Score="3" ViewCount="497" Body="&lt;p&gt;I think I don't understand group convolutions well.&lt;/p&gt;&#xA;&lt;p&gt;Say you have 2 groups. This means that the number of parameters would be reduced in half. So, assuming you have an image and 100 channels, with a filter size of &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt;, you would have 900 parameters (ignore the bias for this example). If you separate this into 2 groups, if I understand it well, you would have 2 groups of 50 channels.&lt;/p&gt;&#xA;&lt;p&gt;This can be made faster, by running the 2 groups in parallel, but how does the number of parameters get halved? Isn't each group having &lt;span class=&quot;math-container&quot;&gt;$50*9=450$&lt;/span&gt; parameters, so, in total, you still have 900 parameters? Do they mean that the number of parameters that the backpropagation goes over (in each branch) gets halved?&lt;/p&gt;&#xA;&lt;p&gt;Because overall, I don't see how it can get reduced. Also, is there a downside in using more groups (even going to 100 groups of 1 channel each)?&lt;/p&gt;&#xA;" OwnerUserId="23871" LastEditorUserId="2444" LastEditDate="2021-01-23T23:29:37.523" LastActivityDate="2021-01-23T23:29:37.523" Title="How is the number of parameters reduced in the group convolution?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11769" PostTypeId="1" CreationDate="2019-04-11T06:36:25.450" Score="1" ViewCount="181" Body="&lt;p&gt;I'm attempting to implement the actor-critic algorithm on Matlab using Radial Basis Function, Local Linear Regression, and shallow Neural Network for inverted pendulum system.&#xA;the state space and the action space are continuous.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;states are the angle x_1 wrapped into [-pi pi] and the angle velocity x_2 in [-8*pi 8*pi]&lt;/li&gt;&#xA;&lt;li&gt;the continuous action u, which is bound between [-3 3].&lt;/li&gt;&#xA;&lt;li&gt;reward function is quadrat rho=x'Q x+u'R u where Q=diag(1,5) and R=0.1 &lt;/li&gt;&#xA;&lt;li&gt;the desired point is upright position  [0 0]'&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;some notes will be added&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the used solver is ode45.&lt;/li&gt;&#xA;&lt;li&gt;the sampling time 0.03.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;it explores random u every step, with normal distribution zero mean sigma=1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;model of the system (to save place the parameters of the model are not written)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; function dy =pendulum(y,u)&#xA;dy(1,1)=y(2);&#xA;dy(2,1)=1/J*(M*g*l*sin(y(1))-(b+K^2/R)*y(2)+K/R*u);&#xA;end_function&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;function to calculate RBF: the idea is to define centers and widths for N RBFs which cover the entire state space to approximate the value function and policy separately. The RBF is normalized.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function phi=RBF(x,C,B,N)         % x:state, C: centres, B: width, N: nombre of used RBfs&#xA; Phi_vec=[];&#xA; Phi_sum=0;&#xA; for i=1:N                        % loop for to calculate the vector phi&#xA;     Phi_i=exp(-1/2*(x-C(i,:)')'*B^(-1)*(x-C(i,:)'));  % gaussian function&#xA;     Phi_vec=[Phi_vec;Phi_i];                    % not normalized phi vector&#xA;     Phi_sum=Phi_sum+Phi_i; % sum for normalisation&#xA;&#xA; end&#xA; phi=Phi_vec/Phi_sum; % normalized phi vector&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; % after tuning the learning rate for actor and critic alpha_a and alpha_c &#xA;&#xA; %  every step the following updates shall be carried out: &#xA;&#xA; %% generally&#xA;&#xA; %  Value function V=Theta_O'*RBF(x,C,B,N)&#xA;&#xA; %  policy pi= Theta_v'*RBF(x,C,B,N)&#xA;&#xA;&#xA; % determine u(k) with exploration term&#xA; u(k)=Theta_V'*RBF(x,C,B,N)+Delta_U(k-1)&#xA;&#xA; %% aplly u(k) and gain x(k+1)&#xA;&#xA; [t,y] = ode45(@(t,y) pendulum(y,u(k)),tspan,x(k)');&#xA;        :&#xA; x(k+1,1)= wrapToPi(x(k+1,1)); % wrpping to pi&#xA;&#xA;&#xA; % determine Temporal difference Error &#xA; Delta(k)=r(k)+gamma*Theta_O'*RBF(x(k),C,B,N)-Theta_O'*RBF(x(k-1),C,B,N);&#xA;&#xA; % eligibility trace&#xA; z=lamda*gamma*z+RBF(x,C,B,N);&#xA;&#xA; % Critic update&#xA; Theta_O=Theta_O+alpha_c*Delta(k)*z;&#xA;&#xA; %actor update&#xA; Theta_V=Theta_V+alpha_a*Delta(k)*Delta_U(k- 1)*RBF(x,C,B,N);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22209" LastEditorUserId="22209" LastEditDate="2019-04-11T15:26:48.753" LastActivityDate="2019-04-11T15:26:48.753" Title="Actor-critic algorithm using gaussian Radial Basis Function, Local Linear Regression and shallow Neural Network" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;linear-regression&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11771" PostTypeId="1" CreationDate="2019-04-11T10:58:59.290" Score="1" ViewCount="69" Body="&lt;p&gt;I am modelling a process with 4 input parameters &lt;code&gt;x1 x2 x3 x4&lt;/code&gt;. The output of the process is 2 variable &lt;code&gt;y1 y2&lt;/code&gt;that varies with length and time. &#xA;I also have data from experiments basically recording the trends in the two output variables as I vary my input variable. &#xA;So far I have only seen neural network examples which will take input &lt;code&gt;x1 x2 x3 x4 t&lt;/code&gt; &#xA;(t  is time) and predict &lt;code&gt;y1 y2&lt;/code&gt; at said time t (no consideration of location). I would however like to also like to see variation with length as well at a given time t &lt;code&gt;[y1a y1b... y1z; y2a y2b... y2z]&lt;/code&gt; where (a, b...z) are location points at an incremental distance &lt;code&gt;dh&lt;/code&gt; from the start.&#xA;Any help is appreciated. TIA&lt;/p&gt;&#xA;" OwnerUserId="23883" LastEditorUserId="23883" LastEditDate="2019-04-11T11:24:33.623" LastActivityDate="2019-09-12T04:05:45.633" Title="Can I use neural networks for a problem (in description)?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11773" PostTypeId="1" CreationDate="2019-04-11T13:49:55.527" Score="1" ViewCount="33" Body="&lt;p&gt;I am training a convLSTM with a dropout layer (with prob 0.5). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I train over more than 5 epochs I notice that the network starts to overfit: my validation set loss becomes stationary while the train loss keeps going down with every epoch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if I train for 20 or more epochs the gap between the validation and train loss is quite substantial. At the same time precision-recall curve becomes much more stable (i.e. monotonic) if i train with a large number of epochs (e.g. 20). Why is that? Is this behaviour a common occurrence?&lt;/p&gt;&#xA;" OwnerUserId="11417" LastActivityDate="2019-04-11T13:49:55.527" Title="Why does precision-recall curve become more stable when neural net begins to overfit?" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11774" PostTypeId="1" CreationDate="2019-04-11T14:13:56.783" Score="2" ViewCount="56" Body="&lt;p&gt;I am in education and I'm wondering about the use of chatbot-like tools to facilitate automated discussions among students.  The chatbot domain of knowledge would be purposely restricted to a specific unit of learning.  The chatbot must have flexible knowledge representation because discussing the causes of wars in one grade will be a different discussion than discussing the causes of wars in an older grade.  I'm wondering how to construct the knowledge base of facts and question prompts to ask students (to keep the inquiry moving forward).  I also wonder about the chatbot gradually getting &quot;smarter&quot; as the discussion progresses.  For example, I don't want the chatbot to give too most useful information too soon and shortcircuit the learning process.  Said differently, I think the chatbot should respond with more sophisticated information as the discussion progresses.  I also think the prompting questions would be different/deeper as the discussion progresses.  My question is more of a request for feedback on:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The potential of the idea?&lt;/li&gt;&#xA;&lt;li&gt;Is the scope for a proof of concept deliverable within my abilities or should I seek additional resources?&lt;/li&gt;&#xA;&lt;li&gt;What back-end tool should I use?  I'm most concerned about finding a visual knowledge representation tool.&lt;/li&gt;&#xA;&lt;li&gt;Is there prior product work done in this area?&lt;/li&gt;&#xA;&lt;li&gt;Is there prior research done in this area?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your thoughts!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. I am reviewing this question thread --&gt; &lt;a href=&quot;https://ai.stackexchange.com/questions/4432/build-conversational-ai/4919#4919&quot;&gt;Build Conversational AI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23886" LastEditorUserId="1671" LastEditDate="2019-04-11T21:04:42.793" LastActivityDate="2019-04-11T21:04:42.793" Title="ChatBot applications [Academia]" Tags="&lt;natural-language-processing&gt;&lt;ai-basics&gt;&lt;chat-bots&gt;&lt;knowledge-representation&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11776" PostTypeId="1" CreationDate="2019-04-11T23:18:09.190" Score="3" ViewCount="50" Body="&lt;p&gt;I'm trying to make deep q-learning agent from &lt;a href=&quot;https://keon.io/deep-q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keon.io/deep-q-learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My environment looks like this:&#xA;&lt;a href=&quot;https://imgur.com/a/OnbiCtV&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/OnbiCtV&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see my agent is a circle and there is one gray track with orange lines (reward gates). The bolder line is an active gate. &#xA;The orange line from the circle pointing to his direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent has constant velocity and it can turn left/right 10 degrees or do nothing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the next image are agents sensors&#xA;&lt;a href=&quot;https://imgur.com/a/Qj7Kesi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/Qj7Kesi&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are rotating with the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The states are a distance from the agent to active gate and lengths of sensors.&#xA;In total there are 1+7 states and it is q-learning neural net input dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actions are turn left, turn right and do nothing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reward function returns 25 when the agent intersects reward gate; 125 when agent intersects the last gate; -5 if agent intersects track border&#xA;If none of this, reward function compare the distance from the agent to the active gate for current state and next state:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If current state distance &gt; next state distance:&#xA; return 0.1&#xA;else&#xA; return -0.1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, DQNAgent has negative, positive and neutral memory.&#xA;If reward is -5, (state, action, reward, next_state, done) go to the negative memory, &#xA;if reward is &gt;= 25, to positive &#xA;else to neutral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is because when I'm forming minibatch for training, I'm taking 20 random samples from neutral memory, 6 from positive and 6 from negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every time when agent intersects track border or when he is stuck for more than 30 seconds, I'm doing training (replay) and agent starts from the beginning.&#xA;This is my model &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(64, input_dim=self.state_size,activation='relu', &#xA;                  kernel_initializer=VarianceScaling(scale=2.0)))&#xA;model.add(Dense(32, &#xA;    activation='relu',kernel_initializer=VarianceScaling(scale=2.0)))&#xA;model.add(Dense(self.action_size, activation='linear'))&#xA;model.compile(loss=self._huber_loss,&#xA;                  optimizer=Adam(lr=self.learning_rate))&#xA;return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried different kinds of model, a different number of neurons per layer, other activation and loss functions, dropout, batch normalization, and this model works the best for now&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried different reward values&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I tried to use static sensors (they are not rotating with the agent)&#xA;&lt;a href=&quot;https://imgur.com/a/8eDtQIF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/8eDtQIF&lt;/a&gt; (green lines on the photo)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes my agent manages to intersect a few gates before hits the border. Rarely he manages to traverse half of the track and once, with this settings, he traversed two laps before he stuck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More often, he is only rotating in one place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the problem lays in state representation or reward function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be appreciated&lt;/p&gt;&#xA;" OwnerUserId="23894" LastActivityDate="2019-04-11T23:18:09.190" Title="Deep Q-Learning agent poor performing actions. Need help optimizing" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;&lt;keras&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11778" PostTypeId="2" ParentId="11768" CreationDate="2019-04-12T00:58:05.953" Score="1" Body="&lt;p&gt;The number of parameters is &lt;code&gt;filters*input_channels*output_channels&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Groups are formed among input and output channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So instead of &lt;code&gt;input_channels*output_channels&lt;/code&gt; with two groups you get &lt;code&gt;(input_channels/2)*(output_channels/2) + (input_channels/2)*(output_channels/2)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="16886" LastActivityDate="2019-04-12T00:58:05.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11779" PostTypeId="2" ParentId="8796" CreationDate="2019-04-12T01:06:44.450" Score="0" Body="&lt;p&gt;I am probably not qualified to answer your question, but I will attempt. At the very least I hope to start a discussion about the matter because I am unsure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (linear) Kalman filter uses the error Covariance as it's main contributor to the Kalman gain. We usually notice the Error covariance decreases over time which can be thought of as the decrease in uncertainty. In the information filter the inverse is true. Our &quot;information matrix&quot; increases with more measurements, because our information increases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally the measurement update of the information state vector &lt;code&gt;y&lt;/code&gt; resembles the way we simply add information. Since &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y(i+1) = y(i) + i(i+1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;here we are simply adding information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I reason and remember the Information filter. I'm not sure where the name came from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally:&#xA;&lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=2&amp;amp;ved=2ahUKEwiS6sa4--fhAhWJXbwKHQFVD7YQFjABegQIBRAB&amp;amp;url=https%3A%2F%2Fwww.crcpress.com%2FDecentralized-Estimation-and-Control-for-Multisensor-Systems%2FMutambara%2Fp%2Fbook%2F9780849318658&amp;amp;usg=AOvVaw0xRFF6uPGPXf5MHFPwPzYK&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arthur Mutambara&lt;/a&gt; mentions: &quot;The information filter essentially tracks information about the states and not the states them selves&quot;. This may be more correct and concise. &lt;/p&gt;&#xA;" OwnerUserId="23895" LastEditorUserId="23895" LastEditDate="2019-04-24T05:14:12.243" LastActivityDate="2019-04-24T05:14:12.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11780" PostTypeId="1" CreationDate="2019-04-12T03:02:26.847" Score="2" ViewCount="40" Body="&lt;p&gt;I have some ecological data on the confirmed presence of a certain animal. I have data on the:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Date&#xA;Relevant metadata about the site &#xA;Simple metrics on the animal&#xA;A complete weather record for the site. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm assuming the presence of this animal is driven by weather events, and that the metadata about the site may be an important factor for patterns in the data. What my data ends up looking like is this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Date of observation&#xA;Date&amp;lt;-as.POSIXct(c(&quot;2015-01-01&quot;,&quot;2015-01-11&quot;,&quot;2015-01-19&quot;,&quot;2015-02-04&quot;,&quot;2015-02-12&quot;,&quot;2015-02-23&quot;,&quot;2015-04-01&quot;,&quot;2015-04-10&quot;,&quot;2015-04-16&quot;,&quot;2015-04-20&quot;))&#xA;&#xA;#Data about animal &#xA;Size&amp;lt;-c(1,1,1,2,2,3,4,1,2,5)&#xA;Color&amp;lt;-c(&quot;B&quot;,&quot;B&quot;,&quot;R&quot;,&quot;R&quot;,&quot;R&quot;,&quot;R&quot;,&quot;B&quot;,&quot;B&quot;,&quot;B&quot;,&quot;Y&quot; )&#xA;Length&amp;lt;-c(1,10,12,4,5,2,1,2,7,12)&#xA;&#xA;#Weather Data&#xA;AirTempDayOf&amp;lt;-c(20,40,20,23,24,25,24,25,25,22)&#xA;WindSpeedDayOf&amp;lt;-c(2,3,2,3,4,3,2,3,4,5)&#xA;AirTempDayBefore&amp;lt;-c(21,40,22,23,24,24,24,27,25,22)&#xA;WindSpeedDayBefore&amp;lt;-c(2,5,2,6,4,3,6,3,2,5)&#xA;AirTemp2DayBefore&amp;lt;-c(21,45,22,23,34,24,24,23,25,23)&#xA;WindSpeed2DayBefore&amp;lt;-c(8,5,3,6,4,7,6,3,2,6)&#xA;&#xA;#Metadata about site&#xA;Type&amp;lt;-c(&quot;Forest&quot;,&quot;Forest&quot;,&quot;Forest&quot;,&quot;Forest&quot;,&quot;Forest&quot;,&quot;Beach&quot;&quot;Beach&quot;&quot;Beach&quot;,&quot;Swamp&quot;,&quot;Swamp&quot;)&#xA;Population&amp;lt;-c(20,30,31,23,32,43,23,43,23,33)&#xA;Use&amp;lt;-c(&quot;Industrial&quot;,&quot;Commercial&quot;,&quot;Industrial&quot;,&quot;Commercial&quot;,&quot;Industrial&quot;,&quot;Commercial&quot;,&quot;Industrial&quot;,&quot;Commercial&quot;,&quot;Industrial&quot;,&quot;Commercial&quot;,)&#xA;&#xA;&#xA;DF&amp;lt;-data.frame(Date,Size,Color,Length,AirTempDayOf, WindSpeedDayOf,AirTempDayBefore,WindSpeedDayBefore,AirTemp2DayBefore,WindSpeed2DayBefore)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I don't have is absence data, so I can't make any assumptions about when an organism was not at the site. I'd like to look for patterns in weather that my be driving the arrival of this organism, but all I have is data on when the organism was spotted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to apply some sort of machine learning to look for patterns that may be driving the arrival of this animal? If I don't have absence data, I'm assuming I cant. I've looked into pseudo-absence models, but I don't know how they might apply here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I can't use machine learning to look at drivers for the presence of these animals, is it possible to use ML to look at possible weather patterns that may be associated with some of the metadata about the site? For example, weather patterns that may be associated with Forrest vs Beach habitats? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I usually use R for my stats, so any answers including R packages would be helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, note that this is just an example dataset above. I don't expect to find any patterns in the above data, and my actual dataset is much larger. But any code developed for the above data should be applicable&lt;/p&gt;&#xA;" OwnerUserId="23896" LastEditorUserId="23896" LastEditDate="2019-04-12T12:26:55.153" LastActivityDate="2019-09-09T14:02:30.330" Title="Machine learning to find drivers of an event with presence-only data (no absence)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;r&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11781" PostTypeId="1" CreationDate="2019-04-12T05:02:52.887" Score="6" ViewCount="449" Body="&lt;p&gt;Projected Bellman error has shown to be stable with linear function approximation. The technique is not at all new. I can only wonder why this technique is not adopted to use with non-linear function approximation (e.g. DQN)? Instead, a less theoretical justified target network is used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could come up with two possible explanations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It doesn't readily apply to non-linear function approximation case (some work needed)&lt;/li&gt;&#xA;&lt;li&gt;It doesn't yield a good solution. This is the case for &lt;em&gt;true&lt;/em&gt; Bellman error but I'm not sure about the projected one.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9793" LastEditorUserId="2444" LastEditDate="2019-05-10T14:48:20.443" LastActivityDate="2019-10-09T10:50:28.163" Title="Why don't people use projected Bellman error with deep neural networks?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11782" PostTypeId="2" ParentId="11678" CreationDate="2019-04-12T05:28:59.370" Score="0" Body="&lt;p&gt;sounds like a job for Sequence-to-sequence or seq2seq&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example &lt;a href=&quot;https://google.github.io/seq2seq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tf-seq2seq &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you basically use an RNN as an encoder (reduces and sequence of words to a vector), and another RNN as a decoder (takes the encoded vector as in input to generation a new sequence of words).&lt;/p&gt;&#xA;" OwnerUserId="16886" LastActivityDate="2019-04-12T05:28:59.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11783" PostTypeId="1" AcceptedAnswerId="11784" CreationDate="2019-04-12T08:16:52.143" Score="1" ViewCount="457" Body="&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return is defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$G_t^\lambda = (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1}G_{t:t+n}$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+n} = R_{t+1}+\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n})$$&lt;/span&gt;&#xA;is the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step return from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we use this definition to rewrite &lt;span class=&quot;math-container&quot;&gt;$G_t^\lambda$&lt;/span&gt; recursively?&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-12T08:28:55.680" LastActivityDate="2019-04-14T07:44:15.380" Title="How can the $\lambda$-return be defined recursively?" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;return&gt;&lt;eligibility-traces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11784" PostTypeId="2" ParentId="11783" CreationDate="2019-04-12T08:16:52.143" Score="5" Body="&lt;p&gt;To rewrite &lt;span class=&quot;math-container&quot;&gt;$G_t^\lambda$&lt;/span&gt; recursively, our goal is to define it in terms of&#xA;&lt;span class=&quot;math-container&quot;&gt;$$G_{t+1}^\lambda = (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1}G_{t+1:t+n+1}.\tag{0}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return is a weighted average of all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns. We will split up the summation by  pulling out the one-step return &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+1}$&lt;/span&gt; and the first step's reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align*}&#xA;G_t^\lambda &amp;amp;= (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1}G_{t:t+n} \tag{1}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= (1-\lambda)\lambda^0G_{t:t+1} + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}G_{t:t+n}\tag{2}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= (1-\lambda)\left(R_{t+1}+\gamma\hat{v}(S_{t+1})\right)\\&#xA;&amp;amp;\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(R_{t+1}+\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{3}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= (1-\lambda)\left(R_{t+1}+\gamma\hat{v}(S_{t+1})\right) + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1} R_{t+1}\\&#xA;&amp;amp;\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{4}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= \gamma(1-\lambda)\hat{v}(S_{t+1}) + (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1} R_{t+1}\\&#xA;&amp;amp;\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{5}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1}\\&#xA;&amp;amp;\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{6}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1}\\&#xA;&amp;amp;\qquad + \gamma\lambda(1-\lambda)\sum_{n=2}^\infty \lambda^{n-2}(R_{t+2}+\dots +\gamma^{n-2}R_{t+n} + \gamma^{n-1}\hat{v}(S_{t+n}))\tag{7}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1}\\&#xA;&amp;amp;\qquad + \gamma\lambda(1-\lambda)\sum_{m=1}^\infty \lambda^{m-1}(R_{t+2}+\dots +\gamma^{m-1}R_{t+m+1} + \gamma^{m}\hat{v}(S_{t+m+1}))\tag{8}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1} + \gamma\lambda(1-\lambda)\sum_{m=1}^\infty \lambda^{m-1}G_{t+1:t+m+1}\tag{9}\\&#xA;&amp;amp;\\&#xA;&amp;amp;= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1} + \gamma\lambda G_{t+1}^\lambda \tag{10}\\&#xA;\end{align*}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ $&lt;/span&gt;&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt; pulls out the one-step return from the summation.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt; expands the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(4)$&lt;/span&gt; pulls out the remaining first step rewards.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(5)$&lt;/span&gt; combines first step rewards.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(6)$&lt;/span&gt; simplifies the &lt;a href=&quot;https://en.wikipedia.org/wiki/Geometric_series#Formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;geometric series&lt;/a&gt;.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(7)$&lt;/span&gt; pulls a factor of &lt;span class=&quot;math-container&quot;&gt;$\gamma\lambda$&lt;/span&gt; out of the summation.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(8)$&lt;/span&gt; makes the substitution &lt;span class=&quot;math-container&quot;&gt;$m=n-1$&lt;/span&gt;.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(9)$&lt;/span&gt; uses the definition of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step return.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(10)$&lt;/span&gt; uses the definition of the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result can be verified in equation &lt;span class=&quot;math-container&quot;&gt;$(12.18)$&lt;/span&gt; of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's RL book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-14T07:44:15.380" LastActivityDate="2019-04-14T07:44:15.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11785" PostTypeId="1" CreationDate="2019-04-12T08:37:06.037" Score="1" ViewCount="102" Body="&lt;p&gt;I have searched on how Google or any map provider calculates distance between two coordinates. The closest I could find is &lt;a href=&quot;http://www.movable-type.co.uk/scripts/latlong.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haversine formula&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I draw a straight line between two points, then Haversine formula can be helpful. But since no one will travel straight and typically move through the streets, I want to know if there are any methods to calculate turn by turn points and see how to find multiple ways to travel to the destination from the source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now my idea is&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have the two coordinates within a map window.&lt;/li&gt;&#xA;&lt;li&gt;Make an algorithm detect the white lines (path) in the window.&lt;/li&gt;&#xA;&lt;li&gt;Make it understand how they are connected.&lt;/li&gt;&#xA;&lt;li&gt;Feed it to an algorithm to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Travelling Salesman Problem&lt;/a&gt; to find the best path between them.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But these things see very memory and process intensive. Even with the knowledge that Google has the powerhouse to process, to serve so many directions and distance matrix in fractions of seconds in amazing. I want to know if there are different approaches to this?&lt;/p&gt;&#xA;" OwnerUserId="9170" LastEditorUserId="2193" LastEditDate="2019-04-12T10:14:55.897" LastActivityDate="2019-06-14T19:02:46.063" Title="How do map providers like Google calculate the distance between two coordinates and find turn by turn directions?" Tags="&lt;reinforcement-learning&gt;&lt;object-recognition&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11786" PostTypeId="2" ParentId="11785" CreationDate="2019-04-12T10:12:55.377" Score="0" Body="&lt;p&gt;Obviously the way Google store their information is not published, but from the &lt;a href=&quot;https://developers.google.com/maps/documentation/directions/intro&quot; rel=&quot;nofollow noreferrer&quot;&gt;Directions API&lt;/a&gt; I would make the following educated guesses:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The roads/paths are stored as a graph database&lt;/li&gt;&#xA;&lt;li&gt;Each path has additional information: type of road, transport link, etc.&lt;/li&gt;&#xA;&lt;li&gt;Geocoordinates or placenames are mapped onto graph nodes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finding a route then is a problem of finding the best path through the graph. This will be easier if you provide waypoints (which effectively split a long route into several shorter ones). As you have the physical coordinates, you can use something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A* algorithm&lt;/a&gt; to traverse the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your question I assume that you'd want to work with map images, rather than a pre-processed graph. I would think that this is not feasible, and that you have to do this conversion into a graph first. You can probably semi-automate this by using image processing to identify roads, but ultimately this is probably something that has to be done at least partially with human intervention. You don't want to drive for ages only to find out there was a one-way street which you did not spot from the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, usually satnavs are aware of speed limits along the path. This again has to be added either manually or automated (by recognising traffic signs along the route, eg from a street view photography car). So image data alone is not sufficient.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-04-12T10:12:55.377" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11787" PostTypeId="1" AcceptedAnswerId="11791" CreationDate="2019-04-12T11:42:10.900" Score="8" ViewCount="182" Body="&lt;p&gt;I am trying to understand how alpha zero works, but there is one point that I have problems understanding, even after reading several different explanations. As I understand it (see for example &lt;a href=&quot;https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png&quot; rel=&quot;noreferrer&quot;&gt;https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png&lt;/a&gt;), alpha zero does not perform rollouts. So instead of finishing a game, it stops when it hits an unknown state, uses the neural network to compute probabilities for different actions as well as the value of this state (&quot;probability of winning&quot;), and then propagates the new value up the tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasoning is that this is much cheaper, since actually completing the game would take more time then just letting the neural network guess the value of a state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this requires that the neural network is decent at predicting the value of a state. But in the beginning of training, obviously it will be bad at this. Moreover, since the monte carlo tree search stops as soon as it hits a new state, and the number of different game states is very large, it seems to me that the simulation will rarely manage to complete a game. And for sure, the neural network can not improve unless it actually completes a significant number of games, because that is only real feedback that tells the agent if it is doing good or bad moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only plausible explanation I can come up with is:  If the neural network would be essentially random in the beginning, well then for sure the large number of game states would prevent the tree search from ever finishing if it restarts as soon as it hits a previously unknown game state, so this can not be the case. So perhaps, maybe even if the neural network is bad in the beginning, it will not be very &quot;random&quot;, but still be quite biased towards some paths. This would mean that the search would be biased to some smaller set of states among the vast number of different game states, and thus it would tend to take the same path more than once and be able to complete some games and get feedback. Is this &quot;resolution&quot; correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem I have though with the above &quot;resolution&quot;, is that according to the algorithm, it should favor exploration in the beginning, so it seems that in the beginning it will be biased towards choosing previously not taken actions. This makes it even more seem like the tree search will never be able to complete a game and thus the neural net would not learn.&lt;/p&gt;&#xA;" OwnerUserId="23910" LastActivityDate="2019-06-10T16:12:04.993" Title="How can alpha zero learn if the tree search stops and restarts before finishing a game?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11788" PostTypeId="2" ParentId="11780" CreationDate="2019-04-12T13:14:53.637" Score="1" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you have only one type of data, cluster analysis may be a good choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also try '&lt;a href=&quot;https://en.wikipedia.org/wiki/One-class_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;1-class learning&lt;/a&gt;' approaches, although I have found these to be unreliable in the past. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a cluster analysis algorithm in R is &lt;a href=&quot;https://uc-r.github.io/kmeans_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;kmeans&lt;/a&gt;. There are many others. These approaches will reveal points that typify large portions of the dataset. By examining 'typical' cases, and how they differ, you can spot potential causal factors to test experimentally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a 1-class learning algorithm is a one-class svm. Most svm libraries will accept data of a single class and do the right thing with it. &lt;a href=&quot;https://stackoverflow.com/questions/27375517/one-class-classification-with-svm-in-r&quot;&gt;Here's&lt;/a&gt; an example with R's e1071 package.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-12T13:14:53.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11789" PostTypeId="1" CreationDate="2019-04-12T13:38:21.127" Score="1" ViewCount="53" Body="&lt;p&gt;Heterogeneity: Based on the heterogeneity of agents MAS can be divided into two categories namely: homogeneous and heterogeneous. Homogeneous MAS include agents that all have the same characteristics and functionalities, while heterogeneous MAS include agents with diverse features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I read in this &lt;a href=&quot;https://arxiv.org/abs/1812.11794&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; that these methods can deal with the heterogeneity of agents MAS :&#xA;The dueling double deep Q-network &lt;strong&gt;(DDDQN)&lt;/strong&gt; and Independent Deep Q-Network &lt;strong&gt;(IDQN)&lt;/strong&gt;: first approach to address heterogeneous multi-agent learning in urban traffic control.&#xA;deep Q-network &lt;strong&gt;(DQN)&lt;/strong&gt;: To handle heterogeneity,  each agent has different experience replay memory and different network policy. &#xA;The asynchronous advantage actor-critic &lt;strong&gt;(A3C)&lt;/strong&gt; algorithm is used to learn optimal policy for each agent, which can be extended to multiple heterogeneous agents.  So, Can someone tell me What is the best method to deal with heterogeneous multi-agent system MAS?&lt;/p&gt;&#xA;" OwnerUserId="21181" LastActivityDate="2019-04-12T13:38:21.127" Title="What is the best method to deal with heterogeneous multi agent system MAS?" Tags="&lt;deep-learning&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11791" PostTypeId="2" ParentId="11787" CreationDate="2019-04-12T16:40:22.923" Score="2" Body="&lt;p&gt;You're right that AlphaGo Zero doesn't perform rollouts in its MCTS. It does complete many, many games, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realize that AlphaGo Zero only iterates MCTS 1,600 times before taking an action. The next state resulting from that action becomes the root of future search trees. Since a typical game of Go only lasts a few hundred moves, the board will very quickly reach a terminal state.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of this is dependent on the initial performance of the neural network. The neural net can be incredibly bad; actions/moves will still be taken at the same frequency. Of course, since AlphaGo Zero trains with self-play, one of its two selves in a game will usually be the winner (ties are possible). So the neural net will improve over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend going over the &quot;Self-Play Training Pipeline&quot; section of &lt;a href=&quot;http://discovery.ucl.ac.uk/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-12T19:07:00.153" LastActivityDate="2019-04-12T19:07:00.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11792" PostTypeId="1" CreationDate="2019-04-12T17:06:47.410" Score="2" ViewCount="402" Body="&lt;p&gt;I've been trying to read Sutton &amp;amp; Barto book chapter 5.1, but I'm still a bit confused about the procedure of using Monte Carlo policy evaluation (p.92), and now I just cant proceed anymore coding a python solution, because I feel like I don't fully understand how the algorithm works, so that the pseudocode example in the book doesn't seem to make much sense to me anymore. (the orange part)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done the chapter 4 examples with the algorithms coded already, so I'm not totally unfamiliar with these, but somehow I must have misunderstood the Monte Carlo prediction algorithm from chapter 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MAJZ8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MAJZ8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My setting is a 4x4 gridworld where reward is always -1. &lt;/li&gt;&#xA;&lt;li&gt;Policy is currently equiprobable randomwalk. If an action would take the newState (s') into outside the grid, then you simply stay in place, but action will have been taken, and reward will have been rewarded. &lt;/li&gt;&#xA;&lt;li&gt;Discount rate will be 1.0 (no discounting). &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Terminal states should be two of them, (0,0) and (3,3) at the corners.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;On page 92 it shows the algorithm pseudocode and I feel as though I coded my episode generating function correctly thusfar. I have it such that, the results are that I always start in the same starting state (1,1) coords in the gridworld.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Currently, I have it so that if you started always in state (1,1), then a possible randomly generate episode could be as follows (in this case also optimal walk). Note that I currently have the episodes in form of list of tuple (s, a, r). Where s will also be a tuple (row,column), but a = string such as &quot;U&quot; for up, and r is reward always -1. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;so that a possible episode could be like: [( (1,1), &quot;U&quot;, -1 ), ( (0,1), &quot;L&quot;, -1 )] So that the terminal state is always excluded, so that the last state in episode will be the state immediately close to terminal state. Just like the pseudocode describes that you should exclude the terminal state S_T. &#xA;But, the random episode could have been one where there are repeating states such as [( (1,1), &quot;U&quot;, -1), ( (0,1), &quot;U&quot;, -1 ), ( (0,1), &quot;U&quot;, -1 ), ( (0,1), &quot;L&quot;, -1 )]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I made the loop for each step of episode, such as follows: once you have the episodeList of tuples, iterate for each tuple, in reversed order. I think this should give the correct amount of iterations there...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;G can be updated as described in pseudocode.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;currently the Returns(S_t) datastructure that I have, will be a dictionary where the keys are state tuples (row,col), and the values are empty lists in the beginning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have a feeling that I'm calculating the average into V(S_t) incorrectly because I origianlly thought that you could even omit the V(S_t) step totally from the algorithm, and only afterwards compute for a separate 2D array V[r,c] for each state get the sum of the appropriate list elements (accessed from the dict), and divide that sum by the amount of episodes that you ran???&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But I don't suddently know how to implement the first visit check in the algorithm. Like, I literally don't understand what it is actually checking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And furthermore I don't understand how the empirical mean is now supposed to be calculates in the monte carlo algorithm where there is the V(s_t) = average( Returns(S_t) )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will also post my python code thusfar.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import numpy.linalg as LA&#xA;import random&#xA;&#xA;# YOUR CODE&#xA;&#xA;&#xA;&#xA;rows_count = 4&#xA;columns_count = 4&#xA;V = np.zeros((rows_count, columns_count))&#xA;reward = -1 #probably not needed&#xA;directions = ['up', 'right', 'down', 'left'] #probably not needed&#xA;maxiters = 10000&#xA;eps = 0.0000001&#xA;k = 0 # &quot;memory counter&quot; of iterations inside the for loop, note that for loop i-variable is regular loop variable&#xA;&#xA;rows = 4&#xA;cols = 4&#xA;&#xA;#stepsMatrix = np.zeros((rows_count, columns_count)) &#xA;&#xA;&#xA;&#xA;&#xA;&#xA;def isTerminal(r,c):      #helper function to check if terminal state or regular state&#xA;    global rows_count, columns_count&#xA;    if r == 0 and c == 0: #im a bit too lazy to check otherwise the iteration boundaries        &#xA;        return True       #so that this helper function is a quick way to exclude computations&#xA;    if r == rows_count-1 and c == columns_count-1:&#xA;        return True&#xA;    return False&#xA;&#xA;def getValue(row, col):    #helper func, get state value&#xA;    global V&#xA;    if row == -1: row =0   #if you bump into wall, you bounce back&#xA;    elif row == 4: row = 3&#xA;    if col == -1: col = 0&#xA;    elif col == 4: col =3&#xA;&#xA;    return V[row,col]&#xA;&#xA;def getState(row,col):&#xA;    if row == -1: row =0   #helper func for the exercise:1&#xA;    elif row == 4: row = 3&#xA;    if col == -1: col = 0&#xA;    elif col == 4: col =3&#xA;    return row, col&#xA;&#xA;&#xA;def makeEpisode(r,c):  #helper func for the exercise:1&#xA;## return the count of steps ??&#xA;#by definition, you should always start from non-terminal state, so&#xA;#by minimum, you need at least one action to get to terminal state&#xA;    stateWasTerm = False&#xA;    stepsTaken = 0&#xA;    curR = r&#xA;    curC = c&#xA;    while not stateWasTerm:&#xA;&#xA;        act = random.randint(0,3)&#xA;        if act == 0: ##up&#xA;            curR-=1&#xA;        elif act == 1: ##right&#xA;            curC+=1&#xA;        elif act == 2: ## down&#xA;            curR+=1&#xA;        else:##left&#xA;            curC-=1&#xA;        stepsTaken +=1&#xA;        curR,curC = getState(curR,curC)&#xA;        stateWasTerm = isTerminal(curR,curC)&#xA;    return stepsTaken&#xA;&#xA;&#xA;V = np.zeros((rows_count, columns_count))&#xA;episodeCount = 100&#xA;reward = -1&#xA;y = 1.0 #the gamma discount rate&#xA;&#xA;&#xA;#use dictionary where key is stateTuple, &#xA;#and value is stateReturnsList&#xA;#after algorithm for monte carlo policy eval is done, &#xA;#we can update the dict into good format for printing&#xA;#and use numpy matrix&#xA;returnsDict={} &#xA;for r in range(4):&#xA;    for c in range(4):&#xA;        returnsDict[(r,c)]=[]&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;#&quot;&quot;&quot;first-visit montecarlo episode generation returns the episodelist&quot;&quot;&quot;&#xA;def firstMCEpisode(r,c):&#xA;    global reward&#xA;    stateWasTerm = False&#xA;    stepsTaken = 0&#xA;    curR = r&#xA;    curC = c&#xA;    episodeList=[  ]&#xA;&#xA;    while not stateWasTerm:&#xA;&#xA;        act = random.randint(0,3)&#xA;        if act == 0: ##up&#xA;            r-=1&#xA;            act=&quot;U&quot;&#xA;        elif act == 1: ##right&#xA;            c+=1&#xA;            act=&quot;R&quot;&#xA;        elif act == 2: ## down&#xA;            r+=1&#xA;            act=&quot;D&quot;&#xA;        else:##left&#xA;            c-=1&#xA;            act=&quot;L&quot;&#xA;        stepsTaken +=1&#xA;&#xA;        r,c = getState(r,c)&#xA;        stateWasTerm = isTerminal(r,c)&#xA;        episodeList.append( ((curR,curC), act, reward) )&#xA;        if not stateWasTerm:&#xA;&#xA;            curR = r&#xA;            curC = c&#xA;&#xA;&#xA;    return episodeList&#xA;&#xA;&#xA;kakka=0 #for debug breakpoints only!&#xA;#first-visit Monte Carlo with fixed starting state in the s(1,1) state&#xA;for n in range(1, episodeCount+1):&#xA;&#xA;    epList = firstMCEpisode(1,1)&#xA;    G = 0&#xA;    for t in reversed( range( len(epList) )):&#xA;        G = y*G + reward #NOTE! reward is always same -1&#xA;        S_t = epList[t][0] #get the state only, from tuple&#xA;&#xA;        willAppend = True&#xA;        for j in range(t-1):&#xA;            tmp = epList[j][0]&#xA;            if( tmp == S_t ):&#xA;                willAppend =False&#xA;                break&#xA;        if(willAppend):&#xA;            returnsDict[S_t].append(G)&#xA;            t_r = S_t[0] #tempRow from S_t&#xA;            t_c =S_t[1] #tempCol from S_t&#xA;            V[t_r, t_c] = sum( returnsDict[S_t] ) / n&#xA;&#xA;&#xA;kakka = 3 #for debug breakpoints only!&#xA;print(V)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="2444" LastEditDate="2020-04-16T19:25:52.737" LastActivityDate="2020-04-16T19:25:52.737" Title="Difficulty understanding Monte Carlo policy evaluation (state-value) for gridworld" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;policy-evaluation&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11793" PostTypeId="1" AcceptedAnswerId="13038" CreationDate="2019-04-12T20:53:48.807" Score="10" ViewCount="1119" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FwEtk.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FwEtk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the meaning of &lt;span class=&quot;math-container&quot;&gt;$V(D,G)$&lt;/span&gt;? How do we get these expectation parts?&#xA;I was trying to understand it following this article: &lt;a href=&quot;https://danieltakeshi.github.io/2017/03/05/understanding-generative-adversarial-networks/&quot; rel=&quot;noreferrer&quot;&gt;Understanding Generative Adversarial Networks &lt;em&gt;(D.Seita)&lt;/em&gt;&lt;/a&gt;, but, after many tries, I still can't understand how he got from &lt;span class=&quot;math-container&quot;&gt;$\sum_{n=1}^{N} \log D(x)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}(\log(D(x))$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23918" LastEditorUserId="2444" LastEditDate="2019-04-15T09:58:33.980" LastActivityDate="2021-04-10T01:13:30.993" Title="Understanding notation of Goodfellow's GAN objective function" Tags="&lt;machine-learning&gt;&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11794" PostTypeId="5" CreationDate="2019-04-12T21:17:37.363" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative adversarial network (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Nets&lt;/a&gt; (Goodfellow, et al.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-12T21:17:37.363" LastActivityDate="2019-04-12T21:17:37.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11795" PostTypeId="4" CreationDate="2019-04-12T21:17:37.363" Score="0" Body="Generative Adversarial Networks" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-12T21:17:37.363" LastActivityDate="2019-04-12T21:17:37.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11796" PostTypeId="5" CreationDate="2019-04-12T21:49:58.340" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-12T21:49:58.340" LastActivityDate="2019-04-12T21:49:58.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11797" PostTypeId="4" CreationDate="2019-04-12T21:49:58.340" Score="0" Body="For questions related to Hebbian learning (or Hebb's rule), which is a local and incremental learning rule that is inspired by biological learning systems (such as the human brain). Put simply, Hebbian learning is based on the idea that the connection between two neurons is strengthened if these two neurons fire together." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-10T14:48:04.290" LastActivityDate="2019-08-10T14:48:04.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11798" PostTypeId="2" ParentId="11743" CreationDate="2019-04-13T00:59:53.807" Score="4" Body="&lt;p&gt;I have several undergraduates working on multiagent deep RL problems for their theses, but most of them have been working for 8-9 months. 2 might be a stretch.&lt;/p&gt;&#xA;&lt;p&gt;Good multiagent deep RL problems for a bachelor's thesis might look something like:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick an older video game, which has been studied using Deep RL, but not in depth. Right now my students have been liking Nintendo 64 games.&lt;/li&gt;&#xA;&lt;li&gt;Read the papers that study this game already.&lt;/li&gt;&#xA;&lt;li&gt;Pick one of the described approaches and reproduce the paper's results in your own system.&lt;/li&gt;&#xA;&lt;li&gt;Pick one of the parameters that the paper does not explore changing, and see what happens as you change it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This probably does not lead to a publishable result, but it is real science and can make for a fine undergraduate thesis.&lt;/p&gt;&#xA;&lt;p&gt;A slightly harder project, which may require more time, would be to examine the &amp;quot;future work&amp;quot; sections of these papers, and perform one of the experiments suggested there. These experiments often lead to small publishable results.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-23T00:24:56.177" LastActivityDate="2021-01-23T00:24:56.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11799" PostTypeId="1" AcceptedAnswerId="11819" CreationDate="2019-04-13T15:19:42.393" Score="3" ViewCount="40" Body="&lt;p&gt;In Decision Tree or Random Forest, each tree has a collection of decision nodes (in which each node has a threshold value) and a class labels (or regression values). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that threshold values are used for comparison with a corresponding feature value. As far as I know, the comparison is performed either &quot;&amp;lt;&quot;, &quot;&gt;&quot;  or &quot;==&quot; predicate.&#xA;Anything else for the functions taking threshold value and a feature value as inputs??&lt;/p&gt;&#xA;" OwnerUserId="23924" LastActivityDate="2019-04-15T00:58:28.497" Title="What are possible functions assigned on decision nodes for decision tree prediction?" Tags="&lt;decision-tree&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11800" PostTypeId="1" CreationDate="2019-04-13T16:57:34.497" Score="1" ViewCount="46" Body="&lt;p&gt;I am attempting to implement an agent that learns to play in the Pong environment, the environment was created in PyGame and I return the pixel data and score at each frame. I use a CNN to take a stack of the last 4 frames as input and predicts the best action to take, I also make use of training on a minibatch of experiences from an experience replay at each timestep.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen an implementation where the game returned a reward of 10 for each time the bot returns the ball and -10 for each time the bot misses the ball. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether it would be better to reward the bot significantly for managing to get the ball passed the opponent, ending the episode. I was thinking of rewarding 10 for winning the episode, -10 for missing the ball and 5 for returning the ball. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if my approach is sensible, has any glaring problems or if I need to provide more information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="18208" LastActivityDate="2019-04-13T16:57:34.497" Title="Deciding the rewards for different actions in Pong for a DQN agent" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;rewards&gt;&lt;reward-clipping&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11801" PostTypeId="1" CreationDate="2019-04-13T18:17:07.947" Score="1" ViewCount="29" Body="&lt;p&gt;Facing with a multi-class classification task, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are &lt;strong&gt;ROC&lt;/strong&gt; and &lt;strong&gt;Precision-Recall&lt;/strong&gt; (One-vs-All-Rest) curves useful to evaluate and visualize the performance of a model? &#xA;or &lt;strong&gt;Confusion matrix&lt;/strong&gt;, &lt;strong&gt;Precision&lt;/strong&gt;, &lt;strong&gt;Recall&lt;/strong&gt;, &lt;strong&gt;F-score&lt;/strong&gt; (micro and macro) are enough?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think about?&lt;/p&gt;&#xA;" OwnerUserId="20780" LastActivityDate="2019-04-13T18:17:07.947" Title="Evaluation metrics multi-class classification (ROC- PR curves)" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11802" PostTypeId="1" CreationDate="2019-04-13T18:23:28.700" Score="5" ViewCount="627" Body="&lt;p&gt;Image recognition, in the context of machine vision, is the ability of software to identify objects, places, people, writing and actions in images. Computers can use machine vision technologies in combination with a camera and artificial intelligence software to achieve image recognition. So, why is image recognition a key function of AI?&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="11539" LastEditDate="2020-04-25T19:57:23.067" LastActivityDate="2020-04-25T19:57:23.067" Title="Why is image recognition a key function of AI?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;philosophy&gt;&lt;artificial-life&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11803" PostTypeId="1" AcceptedAnswerId="11805" CreationDate="2019-04-13T18:28:22.733" Score="9" ViewCount="211" Body="&lt;p&gt;I came across an article, &lt;a href=&quot;http://www.incompleteideas.net/IncIdeas/BitterLesson.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Bitter Truth&lt;/a&gt;, via the &lt;a href=&quot;https://www.youtube.com/watch?v=wEgq6sT1uq8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Two Minute Papers&lt;/a&gt; YouTube Channel. Rich Sutton says...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are &lt;strong&gt;search&lt;/strong&gt; and &lt;strong&gt;learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What is the difference between search and learning here? My understanding is that learning is a form of search -- where we iteratively search for some representation of data that minimizes a loss function in the context of deep learning.&lt;/p&gt;&#xA;" OwnerUserId="22866" LastEditorUserId="2444" LastEditDate="2020-09-12T13:15:49.100" LastActivityDate="2020-09-12T13:15:49.100" Title="What is the difference between search and learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;philosophy&gt;&lt;search&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11804" PostTypeId="2" ParentId="11116" CreationDate="2019-04-13T20:58:33.217" Score="1" Body="&lt;p&gt;The tutorials you link are not much relevant, there are already existing implementations of your exact problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;a href=&quot;https://github.com/swshon/dialectID_e2e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/swshon/dialectID_e2e&lt;/a&gt;, there are many other similar implementations on github.&lt;/p&gt;&#xA;" OwnerUserId="3459" LastActivityDate="2019-04-13T20:58:33.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11805" PostTypeId="2" ParentId="11803" CreationDate="2019-04-13T21:10:42.090" Score="8" Body="&lt;p&gt;In the context of AI:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Search&lt;/strong&gt; refers to Simon &amp;amp; Newell's &lt;a href=&quot;https://en.wikipedia.org/wiki/General_Problem_Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Problem Solver&lt;/a&gt;, and it's many (many) descendant algorithms. These algorithms take the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a. Represent a current state of some part of the world as a vertex in a graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b. Represent, connected to the current state by edges, all states of the world that could be reached from the current state by changing the world with a single action, and represent all subsequent states in the same manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;c. Algorithmically find a sequence of actions that leads from a current state to some more desired goal state, by walking around on this graph.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;An example of an application that uses search is Google Maps. Another is Google Flights.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Learning&lt;/strong&gt; refers to any algorithm that refines a belief about the world through the exposure to experiences or to examples of others' experiences. Learning algorithms do not have a clear parent, as they were developed separately in many different subfields or disciplines. A reasonable taxonomy is the &lt;a href=&quot;https://medium.com/42ai/the-5-tribes-of-the-ml-world-670ebce96b4c&quot; rel=&quot;nofollow noreferrer&quot;&gt;5 tribes&lt;/a&gt; model. Some learning algorithms actually use search within themselves to figure out how to change their beliefs in response to new experiences! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a learning algorithm used today is &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning&lt;/a&gt;, which is part of the more general family of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt; algorithms. Q-learning works like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a. The learning program (usually called the &lt;em&gt;agent&lt;/em&gt;) is given a representation of the current state of the world, and a list of actions that it could choose to perform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b. If the agent has not seen this state of the world before, it assigns a random number to the reward it expects to get for performing each action. It stores this number as &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;, its guess at the quality of performing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in-state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;c. The agent looks at &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; for each action it could perform. It picks the best action with some probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; and otherwise acts randomly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d. The action of the agent causes the world to change and may result in the agent receiving a reward from the environment. The agent makes a note of whether it got a reward (and how much the reward was), and what the new state of the world is like. It then adjusts its belief about the quality of performing the action it performed in the state it used to be in, so that its belief about the quality of that action is closer to the reality of the reward it got, and the quality of where it ended up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e. The agent repeats steps b-d forever. Over time, its beliefs about the quality of different state/action pairs will converge to match reality more and more closely.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;An example of an application that uses learning is AI.SEs recommendations, which are made by a program that likely analyzes the relationships between different combinations of words in pairs of posts, and the likelihood that someone will click on them. Every time someone clicks on them, it learns something about whether listing a post as related is a good idea or not. Facebook's feed is another everyday example. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-05-18T22:49:25.160" LastActivityDate="2019-05-18T22:49:25.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11806" PostTypeId="2" ParentId="3573" CreationDate="2019-04-14T00:04:36.373" Score="1" Body="&lt;p&gt;Although it is not a rigorous proof, &lt;a href=&quot;https://en.wikipedia.org/wiki/Marvin_Minsky&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marvin Minsky's&lt;/a&gt; book, &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0671657135&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Society of Mind&lt;/a&gt; gives us a blueprint for creating a &quot;mind&quot; (general intelligence).  In his book, he posits that by combining mindless components (&quot;agents&quot;) together in various competing and cooperative structures, we can create actual minds.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMHO, the recent popularity of &lt;a href=&quot;https://medium.com/@rrfd/boosting-bagging-and-stacking-ensemble-methods-with-sklearn-and-mlens-a455c0c982de&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boosting, Bagging, Stacking, and other ensemble techniques&lt;/a&gt; will eventually evolve (through research) into Marvin Minsky's &quot;agent&quot; metaphor. Subsequently, as we learn to make these agents compete and cooperate (looks like this has recently begun with &lt;a href=&quot;https://medium.com/datadriveninvestor/a-leap-into-the-future-generative-adversarial-networks-96a780ed8ee6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks&lt;/a&gt;), we will be able to write &quot;programs&quot; that mimic (or surpass) the human mind.&lt;/p&gt;&#xA;" OwnerUserId="17741" LastEditorUserId="17741" LastEditDate="2019-04-14T00:58:02.360" LastActivityDate="2019-04-14T00:58:02.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11808" PostTypeId="2" ParentId="4245" CreationDate="2019-04-14T08:28:48.880" Score="3" Body="&lt;h1&gt;Theory&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;Encoder&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In general, an Encoder is a mapping &lt;span class=&quot;math-container&quot;&gt;$f : X \rightarrow Y $&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; Input Space and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; Code Space &lt;/li&gt;&#xA;&lt;li&gt;In case of Neural Networks, it is a &lt;strong&gt;Generative Model&lt;/strong&gt; hence a function which is able to compute a &lt;strong&gt;Representation&lt;/strong&gt; out of some input (like GAN)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The point is: how would you train such an encoder network ? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The general answer is: it depends on what you want your &lt;strong&gt;code&lt;/strong&gt; to be and ultimately depends on what kind of problem the NN has to solve, so let's pick one  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Signal Compression&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The goal is to learn a compressed representation for your input that allows to reconstruct the original input minimizing the loss of information &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case hence you want the dimensionality of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; to be lower than the dimensionality &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; which in the NN case means the code space will be represented by less neurons than the input space &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Autoencoder&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Focusing on the Signal Compression problem, what we want to build is a system which is able to &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;take a given signal with size &lt;code&gt;N&lt;/code&gt; bytes &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;compress it into another signal with size &lt;code&gt;M&amp;lt;N&lt;/code&gt; bytes &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;reconstruct the original signal, starting from the compressed representation, as good as possible &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To be able to achiebve this goal, we need basically 2 components &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;an Encoder which compresses its input, performing the &lt;span class=&quot;math-container&quot;&gt;$f : X \rightarrow Y$&lt;/span&gt; mapping &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a Decoder which decompresses its input, performing the &lt;span class=&quot;math-container&quot;&gt;$f: Y \rightarrow X$&lt;/span&gt; mapping &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We can approach this problem with the Neural Network Framework, defining an Encoder NN and a Decoder NN and training them &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to observe this kind of problem can be effectively approached with the convenient learning strategy of &lt;strong&gt;unsupervised learning&lt;/strong&gt; : there is no need to spend any &lt;em&gt;human work&lt;/em&gt; (expensive) to build a supervision signal as the original input can be used for this purpose &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means we have to build a NN which operates essentially between 2 spaces &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; Input Space &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; Latent or Compressed Space &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The general idea behind the training is to make a certain input go along the encoder + decoder pipeline and then compare the reconstruction result with the original input with some kind of loss function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To define this idea a bit more formally &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The final autoencoder mapping is &lt;span class=&quot;math-container&quot;&gt;$f : X \rightarrow Y \rightarrow X$&lt;/span&gt; with &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; input &lt;/li&gt;&#xA;&lt;li&gt;the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; encoded input or latent representation of the input &lt;/li&gt;&#xA;&lt;li&gt;the &lt;span class=&quot;math-container&quot;&gt;$\hat x$&lt;/span&gt; reconstructed input &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Eventually you will get an architecture similar to &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://cdn-images-1.medium.com/max/1600/1*44eDEuZBEsmG_TCAKRI3Kw@2x.png&quot; alt=&quot;AE1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can train this architecture in an unsupervised way, using a loss function like &lt;span class=&quot;math-container&quot;&gt;$f : X \times X \rightarrow \mathbb{R}$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$f(x, \hat x)$&lt;/span&gt; is the loss associated to the &lt;span class=&quot;math-container&quot;&gt;$\hat x$&lt;/span&gt; reconstruction compared with the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; input which is also the ideal result &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Code&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Now let's add a simple example in Keras related to the MNIST Dataset &lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;from keras.layers import Input, Dense &#xA;from keras.models import Model &#xA;&#xA;# Defines spaces sizes &#xA;&#xA;## MNIST 28x28 Input &#xA;space_in_size = 28*28&#xA;&#xA;## Latent Space &#xA;space_compressed_size = 32 &#xA;&#xA;# Defines the Input Tensor &#xA;in_img = Input(shape=(space_in_size,))&#xA;&#xA;encoder = Dense(space_compressed_size, activation='relu')(in_img)&#xA;&#xA;decoder = Dense(space_in_size, activation='sigmoid')(encoder)&#xA;&#xA;autoencoder = Model(in_img, decoder)&#xA;&#xA;autoencoder.compile(optimizer='adam', loss='binary_crossentropy')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="1963" LastEditDate="2019-04-14T10:52:09.873" LastActivityDate="2019-04-14T10:52:09.873" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="11809" PostTypeId="1" AcceptedAnswerId="11821" CreationDate="2019-04-14T13:04:26.620" Score="9" ViewCount="250" Body="&lt;p&gt;Given a large problem, value iteration and other table based approaches seem to require too many iterations before they start to converge. Are there other reinforcement learning approaches that better scale to large problems and minimize the amount of iterations in general?&lt;/p&gt;&#xA;" OwnerUserId="23288" LastEditorUserId="2444" LastEditDate="2019-04-14T18:38:52.930" LastActivityDate="2019-04-19T01:55:44.767" Title="Are there reinforcement learning algorithms that scale to large problems?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11810" PostTypeId="2" ParentId="4245" CreationDate="2019-04-14T14:24:09.233" Score="1" Body="&lt;p&gt;As an addition to NicolaBernini's answer. Here is a full listing which should work with a Python 3 installation that includes Tensorflow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot;MNIST autoencoder&quot;&quot;&quot;&#xA;&#xA;from tensorflow.python.keras.layers import  Input, Dense, Flatten, Reshape&#xA;from tensorflow.python.keras.models import Model &#xA;from tensorflow.python.keras.datasets import mnist&#xA;import matplotlib.pyplot as plt&#xA;from matplotlib.pyplot import figure&#xA;&#xA;&quot;&quot;&quot;## Load the MNIST dataset&quot;&quot;&quot;&#xA;&#xA;(x_train, y_train), (x_test, y_test) = mnist.load_data()&#xA;&#xA;&quot;&quot;&quot;## Define the autoencoder model&quot;&quot;&quot;&#xA;&#xA;## MNIST 28x28 Input &#xA;image_shape = (28,28)&#xA;&#xA;## Latent Space &#xA;space_compressed_size = 25 &#xA;&#xA;in_img = Input(shape=image_shape)&#xA;img = Flatten()(in_img)&#xA;encoder = Dense(space_compressed_size, activation='elu')(img)&#xA;decoder = Dense(28*28, activation='elu')(encoder)&#xA;reshaped = Reshape(image_shape)(decoder)&#xA;autoencoder = Model(in_img, reshaped)&#xA;autoencoder.compile(optimizer='adam', loss='mean_squared_error')&#xA;&#xA;&quot;&quot;&quot;## Train the autoencoder&quot;&quot;&quot;&#xA;&#xA;history = autoencoder.fit(x_train, x_train, epochs=10, shuffle=True, validation_data=(x_test, x_test))&#xA;&#xA;&quot;&quot;&quot;## Plot the training curves&quot;&quot;&quot;&#xA;&#xA;plt.plot(history.history['loss'])&#xA;plt.plot(history.history['val_loss'])&#xA;plt.legend(['loss', 'val_loss'])&#xA;plt.show()&#xA;&#xA;&quot;&quot;&quot;## Generate some output images given some input images. This will allow us to see the quality of the reconstruction for the current value of ```space_compressed_size```&quot;&quot;&quot;&#xA;&#xA;rebuilt_images = autoencoder.predict([x_test[0:10]])&#xA;&#xA;&quot;&quot;&quot;## Plot the reconstructed images and compare them to the originals&quot;&quot;&quot;&#xA;&#xA;&#xA;figure(num=None, figsize=(8, 32), dpi=80, facecolor='w', edgecolor='k')&#xA;plot_ref = 0&#xA;&#xA;for i in range(len(rebuilt_images)):&#xA;&#xA;  plot_ref += 1&#xA;  plt.subplot(len(rebuilt_images), 3, plot_ref)&#xA;&#xA;  if i==0:&#xA;    plt.title(&quot;Reconstruction&quot;)&#xA;&#xA;  plt.imshow(rebuilt_images[i].reshape((28,28)), cmap=&quot;gray&quot;)&#xA;&#xA;  plot_ref += 1&#xA;  plt.subplot(len(rebuilt_images), 3, plot_ref)&#xA;&#xA;  if i==0:&#xA;    plt.title(&quot;Original&quot;)&#xA;&#xA;  plt.imshow(x_test[i].reshape((28,28)), cmap=&quot;gray&quot;)&#xA;&#xA;  plot_ref += 1&#xA;  plt.subplot(len(rebuilt_images), 3, plot_ref)&#xA;&#xA;  if i==0:&#xA;    plt.title(&quot;Error&quot;)&#xA;&#xA;  plt.imshow(abs(rebuilt_images[i] - x_test[i]).reshape((28,28)), cmap=&quot;gray&quot;)&#xA;&#xA;plt.show(block=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have changed the loss function of the training optimiser to &quot;mean_squared_error&quot; to capture the grayscale output of the images. &#xA;Change the value of &#xA;&lt;code&gt;space_compressed_size&lt;/code&gt;&#xA;to see how that effects the quality of the image reconstructions.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="12509" LastEditDate="2019-04-19T22:53:19.563" LastActivityDate="2019-04-19T22:53:19.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11811" PostTypeId="2" ParentId="11802" CreationDate="2019-04-14T14:50:52.620" Score="2" Body="&lt;p&gt;Image recognition is an important application of AI techniques, as images usually act as sensory input for further problems to be solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a self-driving car needs to take into account its environment; it needs to recognise the path/road it is driving on, obstacles, other traffic, traffic signs, etc. All this is visual input which needs to be recognised. Without image recognition a self-driving car would not be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A machine that operates in a warehouse picking goods: in order to handle the goods, eg with a robotic arm, it needs to recognise where the goods are and what they look like, so that the arm can be controlled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A system controlling a CCTV camera should be able to distinguish an intruder from an animal that happens to just walk past. Again, with image recognition and classification this could be achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An AI system will react to sensory input from the 'outside' world, and act on it. The type of sensory input depends on the application: it could be a numerical value (like a bank balance), language (text or speech), tactile (eg a robotic hand), or visual.  If it is a stock brokering system, image recognition is not needed. Machine translation would also not need it. But for many real-world interactions, having 'eyes' is either necessary or at the very least useful.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-05-22T15:59:59.063" LastActivityDate="2019-05-22T15:59:59.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11812" PostTypeId="1" AcceptedAnswerId="11908" CreationDate="2019-04-14T15:01:29.503" Score="2" ViewCount="111" Body="&lt;p&gt;I'm building a generative adversarial network that generates images based on an input image. From the literature I've read on GANs, it seems that the generator takes in a random variable and uses it to generate an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to have the generator receive an input image, would it no longer be a GAN? Would the discriminator be extraneous?&lt;/p&gt;&#xA;" OwnerUserId="23941" LastEditorUserId="23941" LastEditDate="2019-04-14T15:07:20.647" LastActivityDate="2019-04-21T13:19:57.953" Title="How important is it that the generator of a generative adversarial network doesn't take in information about input classes?" Tags="&lt;keras&gt;&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11813" PostTypeId="1" AcceptedAnswerId="11814" CreationDate="2019-04-14T18:47:58.480" Score="1" ViewCount="107" Body="&lt;p&gt;I need to understand the logic of below FOL statement.&#xA;Can someone help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall x \exists y \forall z (z \neq y \iff f(x) \neq z)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this imply that x, y and z cannot be same or f(x) has no value?&lt;/p&gt;&#xA;" OwnerUserId="22322" LastEditorUserId="1671" LastEditDate="2019-10-15T19:25:18.527" LastActivityDate="2019-10-15T19:25:18.527" Title="Meaning of the statement $\forall x \exists y \forall z (z \neq y \iff f(x) \neq z)$" Tags="&lt;logic&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11814" PostTypeId="2" ParentId="11813" CreationDate="2019-04-14T19:57:34.443" Score="4" Body="&lt;p&gt;The statement is  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;for all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, there exists a value of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; such that for all &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;,&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$z\neq y$&lt;/span&gt; if and only if &lt;span class=&quot;math-container&quot;&gt;$z \neq f(x)$&lt;/span&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be simplified:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;&amp;amp; &amp;amp; \forall x \exists y \forall z (z\neq y \iff z \neq f(x))\\&#xA;&amp;amp;\implies &amp;amp; \forall x \exists y \forall z (z=y \iff z = f(x))\\&#xA;&amp;amp;\implies &amp;amp; \forall x \exists y \forall z (y = f(x))\\&#xA;&amp;amp;\implies &amp;amp; \forall x \exists y (y = f(x))\\&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we denote the set of all values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and the set of all values of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, then this tells us that the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; maps every &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to a &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;.  That is, &lt;span class=&quot;math-container&quot;&gt;$f: X \to Y$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-04-14T19:57:34.443" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11816" PostTypeId="1" AcceptedAnswerId="11818" CreationDate="2019-04-14T22:13:13.363" Score="7" ViewCount="588" Body="&lt;p&gt;What loss function is most appropriate when training a model with target values that are probabilities? For example, I have a 3-output model. I want to train it with a feature vector &lt;span class=&quot;math-container&quot;&gt;$x=[x_1, x_2, \dots, x_N]$&lt;/span&gt; and a target &lt;span class=&quot;math-container&quot;&gt;$y=[0.2, 0.3, 0.5]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like something like cross-entropy doesn't make sense here since it assumes that a single target is the correct label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would something like MSE (after applying softmax) make sense, or is there a better loss function?&lt;/p&gt;&#xA;" OwnerUserId="17681" LastEditorUserId="2444" LastEditDate="2019-04-15T10:11:24.777" LastActivityDate="2019-04-15T10:11:24.777" Title="What loss function to use when labels are probabilities?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;loss-functions&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11817" PostTypeId="1" AcceptedAnswerId="11837" CreationDate="2019-04-14T22:19:49.867" Score="3" ViewCount="48" Body="&lt;p&gt;The proof of the consistency of the per-decision importance sampling estimator assumes the independence of &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\pi(A_t|S_t)}{b(A_t|S_t)}R_{t+1}\quad\text{ and }\quad \prod_{k=t+1}^{T-1}\frac{\pi(A_k|S_k)}{b(A_k|S_k)}.$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the proof of Theorem 1 in &quot;&lt;a href=&quot;https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1079&amp;amp;context=cs_faculty_pubs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eligibility Traces for Off-Policy Policy Evaluation&lt;/a&gt;&quot;.&lt;br&gt;&#xA;The result is also stated in Equation (5.14) of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's RL book&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing that this is itself a consequence of an assumption of independence between &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\pi(A_t|S_t)}{b(A_t|S_t)}\quad\text{ and }\quad \frac{\pi(A_{t+1}|S_{t+1})}{b(A_{t+1}|S_{t+1})}.$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how this assumption can be justified. Consider the extreme case of a nearly deterministic policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and deterministic MDP dynamics. It would seem to me that the two values above are then surely &lt;strong&gt;not&lt;/strong&gt; independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="22916" LastActivityDate="2019-04-15T20:12:17.390" Title="Are successive actions independent?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;importance-sampling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11818" PostTypeId="2" ParentId="11816" CreationDate="2019-04-14T22:38:31.020" Score="8" Body="&lt;p&gt;Actually, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;noreferrer&quot;&gt;cross-entropy&lt;/a&gt; loss function would be appropriate here, since it measures the &amp;quot;distance&amp;quot; between a distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; and the &amp;quot;true&amp;quot; distribution &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You are right, though, that using a loss function called &amp;quot;cross_entropy&amp;quot; in many APIs would be a mistake. This is because these functions, as you said, assume a one-hot label. You would need to use the general cross-entropy function,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H(p,q)=-\sum_{x\in X} p(x) \log q(x).$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that one-hot labels would mean that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(x) = &#xA;\begin{cases}&#xA;1 &amp;amp; \text{if }x \text{ is the true label}\\&#xA;0 &amp;amp; \text{otherwise} &#xA;\end{cases}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which causes the cross-entropy &lt;span class=&quot;math-container&quot;&gt;$H(p,q)$&lt;/span&gt; to reduce to the form you're familiar with:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H(p,q) = -\log q(x_{label})$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-14T22:38:31.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11819" PostTypeId="2" ParentId="11799" CreationDate="2019-04-15T00:58:28.497" Score="2" Body="&lt;p&gt;For a binary split, there are only three possible operations (or arguably only two if you consider one-hot encoding). Any other kind of split would simply not be binary. Almost every tree-based model is restricted to &lt;em&gt;binary&lt;/em&gt; splits, due to the combinatorial explosion when considering ternary or even more complex splits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you could write your own algorithm that uses recursive non-binary splits. Beware though that you'll be facing the same difficulty that has led the vast majority of algorithms to be limited to strictly binary splits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;https://stats.stackexchange.com/q/12187/176202&quot;&gt;this related question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12996" LastActivityDate="2019-04-15T00:58:28.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11820" PostTypeId="2" ParentId="11771" CreationDate="2019-04-15T02:08:03.923" Score="1" Body="&lt;p&gt;To answer the titular question first: &lt;em&gt;Yes&lt;/em&gt;, of course you can. Whether a neural network can give you better results than a simpler model, however, depends on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How complex/non-linear the relationship between your input and output variables is;&lt;/li&gt;&#xA;&lt;li&gt;Whether the neural network you have specified is able to learn this relationship efficiently;&lt;/li&gt;&#xA;&lt;li&gt;How much training data you have.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With the information you have provided in the question, there's really no telling how it will perform against a simpler model, so I would advice trying both. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how your output varies over time and location, just include both in your model. In case of a linear regression model, accounting for spatiotemporal autocorrelation could be done with a mixed model using an appropriate covariance structure. The challenge for a neural network would be how to specify one that can learn this type of relationship (for starters, read up on RNNs for longitudinal data). &lt;/p&gt;&#xA;" OwnerUserId="12996" LastActivityDate="2019-04-15T02:08:03.923" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11821" PostTypeId="2" ParentId="11809" CreationDate="2019-04-15T05:53:19.907" Score="5" Body="&lt;p&gt;This is a big question. I'm not going to try to cover the state-of-the-art, but I'll try to cover some of the main ideas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Function Approximation&lt;/strong&gt; [1]&lt;br&gt;&#xA;An essential strategy for scaling up RL algorithms is to reduce the effective size of your state and/or action space through function approximation. For example, you could parameterize your value function using fewer parameters than there are states. Optimization would then take place in the much smaller parameter space, which can be substantially faster. Note that using function approximation almost always loses you any convergence guarantees you would have had otherwise in the tabular setting. It has been very successful in practice, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sampling&lt;/strong&gt;[2]&lt;br&gt;&#xA;Value iteration and other dynamic programming algorithms sweep through the entire state space when computing value functions. Sample-based approaches instead update value functions for states as they are visited. These include Monte Carlo and Temporal Difference methods. Sampling allows us to focus on a subset of the states, freeing us from the computation required to get accurate value estimates of potentially irrelevant states. This is essential in real-world settings, where almost all possible states of the world are irrelevant or even impossible to reach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sample Efficiency/Experience Replay&lt;/strong&gt;&lt;br&gt;&#xA;All else equal, a sample efficient agent is one that learns more with the same experience. Doing this reduces learning time, especially if the time bottleneck is in interacting with the environment. One basic way of improving sample efficiency is to store and reuse experience with something like the experience replay buffer popularized in the &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN paper&lt;/a&gt;. Another, more recent, algorithm called &lt;a href=&quot;https://arxiv.org/pdf/1707.01495.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience Replay&lt;/a&gt; improves sample efficiency by allowing the agent to learn more from its failures (trajectories with no reward).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Model-Based Methods&lt;/strong&gt;[3]&lt;br&gt;&#xA;While technically also about sample efficiency (maybe all of these points are?), model-based methods are important enough to warrant their own section. Usually, the MDP dynamics aren't known to the agent beforehand. Learning and maintaining an estimate of the MDP is therefore often a good idea. If an agent can use its internal model of the world to simulate experience, it can then learn from that simulated experience (called &lt;em&gt;planning&lt;/em&gt;) in addition to learning from actual experience. Because simulated experience is much cheaper to gather than actual experience, this can reduce the time needed to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Search&lt;/strong&gt;[4]&lt;br&gt;&#xA;If our value estimates were perfect, then behaving optimally would only be a matter of moving to the neighboring state with the highest value. This is hardly ever the case, though, so we would like to make decisions more intelligently. One way, called &lt;em&gt;forward search&lt;/em&gt;, is to use a model to consider many possible trajectories starting from the current state. The most popular and successful example of forward search is Monte Carlo Tree Search (MCTS), which was famously used in &lt;a href=&quot;http://discovery.ucl.ac.uk/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero&lt;/a&gt;. Because search allows us to make better decisions given imperfect value estimates, we can focus on more promising trajectories, saving time and computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Exploration&lt;/strong&gt;&lt;br&gt;&#xA;Only ever taking what we think is the &quot;best&quot; action in a given state is usually not a very good idea. When sampling trajectories through large state and/or action spaces, this strategy can completely fail. Taking exploratory actions can help ensure that high-value states are discovered at all. Deciding when to explore and which actions to take is active area of research. In general, though, exploratory actions are ones that reduce an agent's uncertainty of the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Injecting Human Knowledge&lt;/strong&gt;&lt;br&gt;&#xA;Finally, and maybe obviously, reducing the time complexity of an RL algorithm can be accomplished by giving the agent more information about the world. This can be done in many ways. If using linear function approximation, for example, an agent could be given useful information through the features it uses. If it makes use of a model, the model could be initialized with reasonable priors for the reward and transition probability distributions. &quot;Reward shaping&quot;, the practice of manually engineering a (dense) reward function to facilitate learning a specific task, is a more general approach. An agent could also learn directly from human demonstrations with &lt;a href=&quot;https://www.academia.edu/160777/Algorithms_for_inverse_reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverse reinforcement learning&lt;/a&gt; or &lt;a href=&quot;https://www.cs.cmu.edu/~mmv/papers/09ras-survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;imitation learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;br&gt;&#xA;All references not already linked to are chapters out of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's RL book&lt;/a&gt;.&lt;br&gt;&#xA;[1] Linear function approximation is discussed in depth in Chapter 9.&lt;br&gt;&#xA;[2] Monte Carlo and Temporal Difference methods are discussed in Chapters 5 and 6.&lt;br&gt;&#xA;[3] Model-based methods are discussed in the first part of Chapter 8.&lt;br&gt;&#xA;[4] MCTS and search in general are discussed in the second half of Chapter 8.  &lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-19T01:55:44.767" LastActivityDate="2019-04-19T01:55:44.767" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11822" PostTypeId="1" CreationDate="2019-04-15T07:47:20.967" Score="1" ViewCount="162" Body="&lt;p&gt;I'm wondering if AI now can help us abstract summary or general idea of long article, for example novel or historical stories, or abstract most important keyword from sentence;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you please tell me if any of this kind of project is done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish I can improve my reading speed and effectiveness with AI help.&lt;/p&gt;&#xA;" OwnerUserId="23855" LastActivityDate="2020-09-07T19:33:40.000" Title="Can AI help summarize article or abstract sentence keyword?" Tags="&lt;natural-language-processing&gt;&lt;artificial-neuron&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11823" PostTypeId="2" ParentId="11822" CreationDate="2019-04-15T08:29:17.303" Score="1" Body="&lt;p&gt;Yes. Text summarisation has been a research topic in (computational) linguistics for literally decades. Have a look at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_summarization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page on Automatic Summarisation&lt;/a&gt; for an overview.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are basically various different approaches: either, selecting salient sentences (or parts of sentences) which represent the gist of the text, or, trying to 'understand' the text and generating new sentences. The former is generally easier, and works on any text, while the latter would probably be able to produce better results, but is more complicated and would not work on any text, as it would be specific to a particular topic.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-04-15T08:29:17.303" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11824" PostTypeId="1" CreationDate="2019-04-15T08:33:47.020" Score="0" ViewCount="41" Body="&lt;p&gt;Is there an AI application that can produce syntactically (and semantically) correct sentences given a bag of words? For example, suppose I am given the words &quot;cat&quot;, &quot;fish&quot;, and &quot;lake&quot;, then one possible sentence could be &quot;cat eats fish by the lake&quot;.&lt;/p&gt;&#xA;" OwnerUserId="23855" LastEditorUserId="2444" LastEditDate="2019-04-15T09:45:42.450" LastActivityDate="2019-04-15T09:45:42.450" Title="How do I create syntactically correct sentences given several words?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11825" PostTypeId="1" CreationDate="2019-04-15T09:42:09.253" Score="1" ViewCount="56" Body="&lt;p&gt;In natural language processing, we can convert words to vectors (or word embeddings). In this vector space, we can measure the similarity between these word embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we create a vector space where word spelling and pronunciation can be easily compared? For example, &quot;apple&quot; and &quot;ape&quot;, &quot;start&quot; and &quot;startle&quot; are very similar, so they should also be similar in this new vector space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am eventually looking for a library that can do this out of the box. I would like to avoid implementing this myself.&lt;/p&gt;&#xA;" OwnerUserId="23855" LastEditorUserId="2444" LastEditDate="2019-04-16T22:18:48.397" LastActivityDate="2019-04-18T14:17:47.110" Title="How can we create a vector space where word spelling and pronunciation can be easily compared?" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11826" PostTypeId="5" CreationDate="2019-04-15T09:55:10.433" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-15T09:55:10.433" LastActivityDate="2019-04-15T09:55:10.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11827" PostTypeId="4" CreationDate="2019-04-15T09:55:10.433" Score="0" Body="For questions related to the concept of function approximation. For example, questions that involve the use of a neural network (which is a function approximator) in the context of RL in order to approximate a value function or questions that are related to universal approximation theorems." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-03T18:57:10.010" LastActivityDate="2019-08-03T18:57:10.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11828" PostTypeId="5" CreationDate="2019-04-15T09:55:57.630" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-15T09:55:57.630" LastActivityDate="2019-04-15T09:55:57.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11829" PostTypeId="4" CreationDate="2019-04-15T09:55:57.630" Score="0" Body="For questions related to clustering (a usual unsupervised learning technique)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-16T18:34:39.523" LastActivityDate="2019-04-16T18:34:39.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11830" PostTypeId="5" CreationDate="2019-04-15T09:57:42.820" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-15T09:57:42.820" LastActivityDate="2019-04-15T09:57:42.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11831" PostTypeId="4" CreationDate="2019-04-15T09:57:42.820" Score="0" Body="For questions of the form &quot;What is the relation (or relationship) between X and Y?&quot;, which are question used to disambiguate between two or more AI topics, terms or expressions." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-16T18:34:32.227" LastActivityDate="2019-04-16T18:34:32.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11833" PostTypeId="1" CreationDate="2019-04-15T14:35:28.560" Score="2" ViewCount="66" Body="&lt;p&gt;Do we have cross-language vector space for word embedding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When measure similarity for apple/Pomme/mela/Lacus/苹果/りんご, they should be the same&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If would be great if there's available internet service of neuron network which already be trained by multiple language&lt;/p&gt;&#xA;" OwnerUserId="23855" LastEditorUserId="2444" LastEditDate="2019-04-16T22:22:42.363" LastActivityDate="2019-04-18T15:00:27.223" Title="Do we have cross-language vector space for word embedding?" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11835" PostTypeId="1" CreationDate="2019-04-15T16:47:17.947" Score="2" ViewCount="690" Body="&lt;p&gt;I am solving a classification problem with CNN. The number of classes is 5. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can I decide the number of neurons in the FC layer before the softmax layer?&lt;/li&gt;&#xA;&lt;li&gt;Is it &lt;span class=&quot;math-container&quot;&gt;$N * 5$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of classes?&lt;/li&gt;&#xA;&lt;li&gt;Is there any documentation for deciding the number of neurons in the FC layer (before SoftMax layer)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23734" LastEditorUserId="23734" LastEditDate="2019-04-16T00:47:44.650" LastActivityDate="2019-04-16T09:33:57.877" Title="How do I choose the number of neurons in the fully-connected layer before the softmax layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11836" PostTypeId="2" ParentId="11835" CreationDate="2019-04-15T18:20:11.730" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I decide the number of neurons in the FC layer before the softmax layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Train different network architectures (with different numbers of neurons in the last FC layer). Use cross-validation - a set of data you have not trained on - to measure the performance of the network, using a metric that you have decided beforehand is a good proxy for your experiment goals. For instance, you might choose getting the highest classification accuracy as your goal - but might choose something different for an unbalanced dataset, because 90% accuracy is not meaningful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is not usually a good reason to over-tune your network. Trying some variations with 1.5 x or 2 x geometric series (e.g. 5, 10, 20, 40 neurons in layer) is probably enough to find a good hidden layer size.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it &lt;span class=&quot;math-container&quot;&gt;$N * 5$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of classes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, in general. That may work just fine for your problem though.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any documentation for deciding the number of neurons in the FC layer (before the softmax layer)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not really. The most direct thing to do is find solutions that work well for similar problems to yours, and adapt as necessary. So you can search for any similar problem domain and see what the researchers used there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is not much theory to guide you here. However, if it helps with your intuition, more neurons can make a better fit to higher frequency variations in the target function you are learning, whilst more layers will mean better handling of complex function spaces (e.g. where rules about the mapping between input and output can be made according to combining simpler rules) - provided in both cases that you have enough training data that it is possible for the NN to learn the function approximately. These are not strictly defined traits of functions for machine learning, so many NN users will work with this intuitive view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If in doubt, assuming this is not an image, NLP or other well-studied problem, then I might just guess at e.g. 64 neurons per hidden layer, and try 1, 2, and 3 hidden layers as a starting point (all the same size). I cannot say if that will work for you and your problem, but it might help get past the &quot;blank page effect&quot; and start you training and testing some variations.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-04-16T09:33:57.877" LastActivityDate="2019-04-16T09:33:57.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11837" PostTypeId="2" ParentId="11817" CreationDate="2019-04-15T19:53:39.730" Score="1" Body="&lt;p&gt;This is a consequence of the Markovian assumption, which underpins all of RL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Markovian assumption says that it doesn't matter &lt;em&gt;how&lt;/em&gt; we reached a given state, only &lt;em&gt;that&lt;/em&gt; we reached it, when deciding how likely it is that we move to subsequent states. This naturally implies that our choice of actions must also depend only on the current state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct that the assumption is somewhat unrealistic. However, it usually can be made to yield a reasonable approximation of the true dynamics without involving too many state variables. By solving this approximation, we hope to find a policy that also works well in the real problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example. In robot navigation, the real dynamics of the robot &lt;em&gt;do&lt;/em&gt; depend on where the robot has been in the past, because as the robot's battery drains, the voltage levels it outputs will change slightly, and its wheels may become more prone to slippage. Thus, logically, our choice of action should change based on both the current state and our choice of actions in previous states (which drained the battery to a greater or lesser degree). However, if we try to incorporate this into the model, we'll end up having a combinatorial explosion in size of the dynamics function &lt;span class=&quot;math-container&quot;&gt;$P(S_{t+1} | S_t, a_{t}, S_{t-1}, a_{t-1} ... S_0, a_0)$&lt;/span&gt; that actually captures this process (in particular, it will now be a function of 2t inputs). This, in turn, will necessitate a combinatorial blowup in the complexity of our policy (it will increase in complexity by an &lt;em&gt;exponent&lt;/em&gt; of 2t). To keep things tractable, we accept that the dynamics will become lossy, or we can add some extra detail to the local state (e.g. a battery level) to capture the more complex dynamics in a Markovian way. Either way though, we'll be back to a world where future actions won't depend on past ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be clearer, if we assume that &lt;span class=&quot;math-container&quot;&gt;$P(S_{t+1} | S_t) = P(S_{t+1} | S_{t}, S_{t-i})$&lt;/span&gt; for any &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, then the other relationships you mention should not seem surprising. That assumption &lt;em&gt;is&lt;/em&gt; the Markovian assumption. It converts the state transition function into a Matrix that represents a Markov Chain. If we don't make that assumption, then most RL algorithms do not apply to our problem.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-04-15T20:12:17.390" LastActivityDate="2019-04-15T20:12:17.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11839" PostTypeId="5" CreationDate="2019-04-15T20:30:05.033" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Game Theory (wiki)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-15T20:30:05.033" LastActivityDate="2019-04-15T20:30:05.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11840" PostTypeId="4" CreationDate="2019-04-15T20:30:05.033" Score="0" Body="For questions about the field and methods of Evolutionary Game Theory. (Distinct from &quot;evolutionary/genetic in that it EGT does not require recombination.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-15T20:30:05.033" LastActivityDate="2019-04-15T20:30:05.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11842" PostTypeId="1" CreationDate="2019-04-16T00:07:03.750" Score="3" ViewCount="79" Body="&lt;p&gt;I am trying to deploy a machine learning solution online into an application for a client.  One thing they requested is that the solution must be able to learn online because the problem may be non-stationary and they want the solution to track the non-stationarity of the problem.  I thought about this problem a lot, would the following work?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set the learning rate (step-size parameter) for the neural network at a low fixed value so that the most recent training step is weighted more.&lt;/li&gt;&#xA;&lt;li&gt;Update the model only once per day, in a mini-batch fashion.  The mini-batch will contain data from the day, mixed with data from the original data set to prevent catastrophic interference.  By using a mini batch update, I am not prone to biasing my model to the latest examples, and completely forgetting the training examples from months ago.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Would this set-up be &quot;stable&quot; for online/incremental machine learning?  Also, should I set up the update step so it samples data from all distributions of my predicted variable uniformly so it gets an &quot;even&quot; update (i.e., does not overfit to the most probabilistic predicted value)?&lt;/p&gt;&#xA;" OwnerUserId="17706" LastEditorUserId="2444" LastEditDate="2019-04-16T10:01:06.413" LastActivityDate="2019-04-16T10:01:06.413" Title="What are stable ways of doing online machine learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;incremental-learning&gt;&lt;online-learning&gt;" AnswerCount="0" CommentCount="9" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11843" PostTypeId="2" ParentId="8943" CreationDate="2019-04-16T01:56:50.377" Score="3" Body="&lt;p&gt;In the case of UCS, the &lt;em&gt;evaluation function&lt;/em&gt; (that is, the function that is used to select the next node to &lt;em&gt;expand&lt;/em&gt;) is &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the cost of the path from the initial node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, while in the case of the greedy BFS it is &lt;span class=&quot;math-container&quot;&gt;$f(n) = h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the heuristic function that estimates the cost of the path from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node. In other words, in the case of UCS, nodes are expanded only using the experience (in the form of &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;), while, in the case of GBFS, nodes are expanded only using the estimate of the cost to the goal. Note that, in both cases, the node that is chosen to be expanded is the one with the smallest &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt; value.&lt;/p&gt;&#xA;" OwnerUserId="23973" LastEditorUserId="2444" LastEditDate="2019-07-06T20:41:47.510" LastActivityDate="2019-07-06T20:41:47.510" CommentCount="0" CommunityOwnedDate="2020-11-08T12:02:37.637" ContentLicense="CC BY-SA 4.0" />
  <row Id="11845" PostTypeId="1" AcceptedAnswerId="11848" CreationDate="2019-04-16T06:50:50.500" Score="2" ViewCount="537" Body="&lt;p&gt;Why does informed search more efficiently finds a solution than an uninformed search? &lt;/p&gt;&#xA;" OwnerUserId="23299" LastEditorUserId="2444" LastEditDate="2019-06-22T12:54:34.513" LastActivityDate="2019-06-22T12:54:34.513" Title="Why is informed search more efficient than uninformed search?" Tags="&lt;search&gt;&lt;comparison&gt;&lt;efficiency&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11846" PostTypeId="2" ParentId="7998" CreationDate="2019-04-16T07:44:43.477" Score="2" Body="&lt;p&gt;I had a similar problem with a 2D convolution on a hexagonal grid while working on a diffusion problem and stumbled upon this question. Rather than using cube coordinates, you could use &lt;a href=&quot;https://www.redblobgames.com/grids/hexagons/#coordinates-doubled&quot; rel=&quot;nofollow noreferrer&quot;&gt;doubled coordinates&lt;/a&gt;, which I found much easier to save in a 2D array. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example kernel that only changes the direct neighbours and the cell itself would be this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kernel = np.array([[0,   0.1, 0,   0.1, 0  ],&#xA;                   [0.1, 0,   0.4, 0,   0.1],&#xA;                   [0,   0.1, 0,   0.1, 0  ]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know much about Keras, but I assume that this is possible there as well.&lt;/p&gt;&#xA;" OwnerUserId="23978" LastActivityDate="2019-04-16T07:44:43.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11847" PostTypeId="1" AcceptedAnswerId="11850" CreationDate="2019-04-16T12:35:58.513" Score="0" ViewCount="205" Body="&lt;p&gt;How can the A* algorithm be optimized?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references that shows the optimization of A* algorithm are also appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23299" LastEditorUserId="2444" LastEditDate="2019-04-16T15:33:27.223" LastActivityDate="2019-04-16T15:33:27.223" Title="How can the A* algorithm be optimized?" Tags="&lt;algorithm&gt;&lt;optimization&gt;&lt;search&gt;&lt;a-star&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11848" PostTypeId="2" ParentId="11845" CreationDate="2019-04-16T13:53:37.353" Score="1" Body="&lt;p&gt;There are several informed and uninformed search algorithms. They do not all have the same time and space complexity (which also depends on the specific implementation). I could come up with an informed search algorithm that is highly inefficient in terms of time or space complexity. So, in general, informed search algorithm are not more efficient than uninformed ones, in terms of space and time complexity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, given that informed search algorithms use &quot;domain knowledge&quot; (that is, an heuristic function that estimates the distance to the goal nodes), in practice, they tend to find the goal node more rapidly, given a more informed &quot;heuristic&quot; (which needs to be admissible in order to find the optimal solution). For example, in theory, &lt;a href=&quot;https://www.cs.auckland.ac.nz/courses/compsci709s2c/resources/Mike.d/astarNilsson.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A*&lt;/a&gt; has exponential time and space complexities (with respect to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Branching_factor&quot; rel=&quot;nofollow noreferrer&quot;&gt;branching factor&lt;/a&gt; and the depth of the tree), but, in practice, it tends to perform decently well. It tends to have an &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm#cite_note-nilsson-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;effective branching factor&lt;/a&gt; (that is, the branching factor for specific problem instances) quite &quot;small&quot; (for several problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a more informed heuristic? Intuitively, it is an heuristic that more rapidly focuses on the promising parts of the search space. Let's denoted by &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the heuristic function. If &lt;span class=&quot;math-container&quot;&gt;$h(n)=0$&lt;/span&gt;, for all nodes &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, then this is an admissible heuristic, because it always underestimates the distance to the goal (that is, it always returns &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;). However, it is a quite uninformed heuristic: either if you are at the start or goal nodes, the estimation is always the same (so you cannot distinguish the start and goal nodes, in terms of estimates). Given two admissible heuristics &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; is more informed than &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$h_1(n) \leq h_2(n)$&lt;/span&gt;, for all nodes &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An uninformed search algorithm performs an exhaustive search. There are several ways of performing such exhaustive search (e.g. breadth-first or depth-first), which are more efficient than others (depending on the search space or problem). Given that they perform an exhaustive search, they tend to explore &quot;uninteresting&quot; parts of the search space. Hence, in practice, they might be more inefficient than informed search algorithms (that is, they might require more time to find the solution).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-16T14:05:51.040" LastActivityDate="2019-04-16T14:05:51.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11849" PostTypeId="2" ParentId="11847" CreationDate="2019-04-16T14:45:43.790" Score="1" Body="&lt;p&gt;The first step of optimisation is to measure where inside the implementation most time is spent -- you don't actually optimise the algorithm itself, but a specific implementation of it. This step should give you an overview of where you can make improvements. Speculative changes usually don't do much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several aspects of A* which would be candidates. You are keeping a list of paths currently under investigation. If it is very long, processing might take more time. You could try and prune this list (though it might mean that you won't actually find the best path anymore). Or you could investigate efficient ways of storing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each step you need to compute the 'cost' of each path. This could be cached to stop you from repeating calculations. The heuristic function could be slow -- again something to investigate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me stress again that you need to measure the performance first. There is no point in randomly trying to make things faster; you could waste a lot of time for very little gain. And typically with optimisation, readability of the code suffers, and it becomes harder to maintain. You might find that other parts of your system take a lot longer than the actual path finding, so your time might better be spent on those.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-04-16T14:45:43.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11850" PostTypeId="2" ParentId="11847" CreationDate="2019-04-16T15:24:47.187" Score="1" Body="&lt;p&gt;Check below reference url for A* algorithms ...&#xA;&lt;a href=&quot;https://takinginitiative.wordpress.com/2011/05/02/optimizing-the-a-algorithm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://takinginitiative.wordpress.com/2011/05/02/optimizing-the-a-algorithm/&lt;/a&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Heap_%28data_structure%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Heap_%28data_structure%29&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23985" LastActivityDate="2019-04-16T15:24:47.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11851" PostTypeId="2" ParentId="8554" CreationDate="2019-04-16T15:43:55.533" Score="1" Body="&lt;p&gt;In the announced problem, most of the transitions aren't possible, so most the terms of equations (3.3) and (3.4) from the book will end up being 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my understanding,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;p(s'= high | s = high, a = search) &amp;amp;= \sum_{r \in \{0, -3, r_{search}, r_{wait}\}} p(s'=high, r | s = high, a = search) \\&#xA;&amp;amp;= p(s'=high, r =0 | s = high, a = search) \\&#xA;&amp;amp;+p(s'=high, r = -3 | s = high, a = search) \\&#xA;&amp;amp;+p(s'=high, r = r_{search}| s = high, a = search) \\&#xA;&amp;amp;+p(s'=high, r = r_{wait} | s = high, a = search)&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem states that if the agent has high batteries and it chooses to search, then there is no chance that it ends up having a negative reward (&lt;span class=&quot;math-container&quot;&gt;$r= -3$&lt;/span&gt;), thus its transition probability is 0 by definition: &lt;span class=&quot;math-container&quot;&gt;$p(s'=high, r = -3 | s = high, a = search) = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the same logic to all other terms, we get,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;p(s'= high | s = high, a = search) &amp;amp;= \sum_{r \in \{0, -3, r_{search}, r_{wait}\}} p(s'=high, r | s = high, a = search) \\&#xA;&amp;amp;= p(s'=high, r = r_{search}| s = high, a = search) \\&#xA;&amp;amp;= \alpha&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks weird. I not 100% sure that that's the solution, because the question would not make much of sense (the table would've been quite the same).&lt;/p&gt;&#xA;" OwnerUserId="23986" LastEditorUserId="23986" LastEditDate="2019-04-16T15:49:36.083" LastActivityDate="2019-04-16T15:49:36.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11852" PostTypeId="2" ParentId="11833" CreationDate="2019-04-16T17:38:24.237" Score="2" Body="&lt;p&gt;You can try to read about MUSE (Multilingual Unsupervised and Supervised Embeddings) by Facebook. You can read it &lt;a href=&quot;https://github.com/facebookresearch/MUSE&quot; rel=&quot;nofollow noreferrer&quot;&gt;from its Github&lt;/a&gt; or &lt;a href=&quot;https://code.fb.com/ml-applications/under-the-hood-multilingual-embeddings/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. They also provide the FastText dictionary format (.vec file) for some languages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their &lt;a href=&quot;https://arxiv.org/pdf/1710.04087.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; shows how it aligns the vector of words from two different languages:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zs1EF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zs1EF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-04-16T17:38:24.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11854" PostTypeId="5" CreationDate="2019-04-16T22:48:25.583" Score="0" Body="&lt;p&gt;In natural language processing, there are several techniques to create &lt;a href=&quot;https://en.wikipedia.org/wiki/Word_embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;word embeddings&lt;/a&gt;: for example, &lt;a href=&quot;https://nlp.stanford.edu/projects/glove/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GloVe&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Word2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-17T12:50:03.387" LastActivityDate="2019-04-17T12:50:03.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11855" PostTypeId="4" CreationDate="2019-04-16T22:48:25.583" Score="0" Body="For questions related to word embeddings, which are vector representations of words." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-17T12:50:23.587" LastActivityDate="2019-04-17T12:50:23.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11856" PostTypeId="5" CreationDate="2019-04-16T22:49:50.037" Score="0" Body="&lt;p&gt;For more details, have a look at &lt;a href=&quot;https://nlp.stanford.edu/projects/glove/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nlp.stanford.edu/projects/glove/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-17T12:51:17.913" LastActivityDate="2019-04-17T12:51:17.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11857" PostTypeId="4" CreationDate="2019-04-16T22:49:50.037" Score="0" Body="For questions related to the GloVe (Global Vectors for Word Representation), which is an unsupervised learning algorithm for obtaining vector representations for words." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-17T12:51:11.423" LastActivityDate="2019-04-17T12:51:11.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11858" PostTypeId="5" CreationDate="2019-04-16T22:50:59.340" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-16T22:50:59.340" LastActivityDate="2019-04-16T22:50:59.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11859" PostTypeId="4" CreationDate="2019-04-16T22:50:59.340" Score="0" Body="For questions related to incremental learning algorithms, which are algorithms that attempt to learn new information without forgetting all the previously learned one. Incremental learning is often a synonym for continual (or continuous) learning and lifelong learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-06T00:02:40.993" LastActivityDate="2019-08-06T00:02:40.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11860" PostTypeId="5" CreationDate="2019-04-16T22:52:31.313" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-16T22:52:31.313" LastActivityDate="2019-04-16T22:52:31.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11861" PostTypeId="4" CreationDate="2019-04-16T22:52:31.313" Score="0" Body="For questions related to online learning algorithms, that is, algorithms that learn while e.g. the associated agent interacts with an environment." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-17T12:49:43.943" LastActivityDate="2019-04-17T12:49:43.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11862" PostTypeId="5" CreationDate="2019-04-16T22:53:44.827" Score="0" Body="&lt;p&gt;For more details, see, for example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2836213/&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Causal Inference&lt;/a&gt; (2010) Judea Pearl, &lt;a href=&quot;https://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Causal inference in statistics: An overview&lt;/a&gt; (2009) by Judea Pearl or &lt;a href=&quot;https://plato.stanford.edu/entries/causation-probabilistic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probabilistic Causation&lt;/a&gt; at &lt;em&gt;Stanford Encyclopedia of Philosophy&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="2444" LastEditDate="2019-08-15T11:44:38.903" LastActivityDate="2019-08-15T11:44:38.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11863" PostTypeId="4" CreationDate="2019-04-16T22:53:44.827" Score="0" Body="For questions related to causation (or causality), which is the field that studies the relationship between cause and effect, where the cause is partly responsible for the effect and the effect is partly dependent on the cause." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-15T21:25:57.907" LastActivityDate="2019-08-15T21:25:57.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11864" PostTypeId="5" CreationDate="2019-04-16T22:54:27.037" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-16T22:54:27.037" LastActivityDate="2019-04-16T22:54:27.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11865" PostTypeId="4" CreationDate="2019-04-16T22:54:27.037" Score="0" Body="For questions related to the iterative deepening A* search algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-17T12:50:29.160" LastActivityDate="2019-04-17T12:50:29.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11866" PostTypeId="1" CreationDate="2019-04-17T04:38:33.790" Score="8" ViewCount="1982" Body="&lt;p&gt;In section 3.2.1 of &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt; the claim is made that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dot-product attention is identical to our algorithm, except for the scaling factor of &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{\sqrt{d_k}}$&lt;/span&gt;. Additive attention computes the compatibility function using a feed-forward network with a single hidden layer. While the two are similar in theoretical complexity, dot-product attention is much faster and more space-efficient in practice, since it can be implemented using highly optimized matrix multiplication code.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It does not make sense why dot product attention would be faster. Additive attention is nearly identical computation wise; the main difference is &lt;span class=&quot;math-container&quot;&gt;$Q + K$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$Q K^T$&lt;/span&gt; in dot product attention. &lt;span class=&quot;math-container&quot;&gt;$Q K^T$&lt;/span&gt; requires at least as many addition operations as &lt;span class=&quot;math-container&quot;&gt;$Q + K$&lt;/span&gt;, so how can it possibly be faster?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2019-04-17T09:17:41.703" LastActivityDate="2021-01-06T23:38:49.820" Title="Why is dot product attention faster than additive attention?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11867" PostTypeId="2" ParentId="9925" CreationDate="2019-04-17T09:36:10.390" Score="2" Body="&lt;p&gt;I am going to refer to the expected output as &quot;the price of the house&quot; or simply as &quot;price&quot; to make the answer easier to understand but this applies to any other scenario as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer &lt;strong&gt;part 1&lt;/strong&gt; of your question, if the correlation between &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and the price of the house is 0 or negligible, then it is very likely that varying &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; while keeping &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; constant will result in almost the same price being predicted in a trained network. This certainly seems like a learnable statistical characteristic.&#xA;There are some caveats though. Firstly, it depends on how complex your network is. How learnable are the correlations between &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and the price? Assuming most other factors like these fall into place, I would say that it is &lt;em&gt;quite likely&lt;/em&gt; that &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; will be decoupled from the output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Part 2&lt;/strong&gt; of your question is a little trickier to explain. Let us consider a simpler scenario where we use logistic regression. Logistic regression is, in essence, a network with sigmoid output and no hidden layer.  The output is the sigmoid over a linear combination of all inputs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us consider the example as follows -&lt;br&gt;&#xA;Two data points have the same or similar values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and expected output, but &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; varies considerably. The coefficient of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; in the linear combination has a finite, non-zero value. Because of this, the input to the sigmoid will differ even though the resultant price in both cases should be the same or similar.&lt;br&gt;&#xA;The loss function's value will increase because of the discrepancy between expected and predicted values in the above example.&lt;br&gt;&#xA;In general, the change in the value of the coefficient of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is a product of learning rate and the differential of the loss function with respect to it.&#xA;The change in its value will now be such that the magnitude of the coefficient of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; decreases to enforce the condition noted in the above examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is difficult to predict at what point a more complex network will learn to ignore &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;. It could happen that the weights at the input layer itself might converge to zero. Or the weights in successive layers could converge such that their linear combination gives no importance to the input w. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to address another point. You mentioned that - &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For a classical neural network, the network has no &quot;memory,&quot; so it might be very difficult for the network to realize that w is a worthless input parameter. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This first part about having no memory is true. However, the network does not need to remember all the past values. The trainable parameters of a network are basically learning the statistical distribution of the input data and map it to the expected output.  They are trying to model a mathematical function that satisfies as many such training samples as possible.&#xA;The network, through these parameters, has stored an abstraction of the behaviour of the training data. So even though it does not remember every training sample, it does remember the general correlation between input and expected output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An oversimplified analogy would be, you as a human don't remember the multiplication of every number by 2. Yet, if I ask you what is the product of 123 and 2, you can find the product because you just know how the &quot;multiply by 2&quot; function works in general.   Similarly, the network builds an intuition of what the expected output should look like in general, by mapping it to a function whose parameters can be modified.&lt;/p&gt;&#xA;" OwnerUserId="23994" LastActivityDate="2019-04-17T09:36:10.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11868" PostTypeId="2" ParentId="11081" CreationDate="2019-04-17T15:36:48.543" Score="0" Body="&lt;p&gt;I found myself turning in cycles for a while, so to clarify Neil Slater's answer,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the beginning of the book, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; means &quot;set of non-terminal states&quot; and &lt;span class=&quot;math-container&quot;&gt;$S^+$&lt;/span&gt; means &quot;set of all states, including the terminal ones&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{s^{\prime} \in S} \sum_{r \in R} p(s^{\prime}, r | s,a) = 1, \forall s \in S, a \in A(s) \tag{3.3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, in eq. 3.3 when we define that &lt;span class=&quot;math-container&quot;&gt;$\forall s \in S$&lt;/span&gt;, we say that that once in a terminal state, the formula does not apply (which is obvious because no action is ever available in a terminal state by definition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does not however constraint the probability in how to &quot;get&quot; in a terminal state, and that is the key to answer the question.&lt;/p&gt;&#xA;" OwnerUserId="23986" LastEditorUserId="22916" LastEditDate="2019-04-17T18:23:17.717" LastActivityDate="2019-04-17T18:23:17.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11869" PostTypeId="2" ParentId="3288" CreationDate="2019-04-17T15:43:35.970" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/5187/23994]&quot;&gt;eggie5&lt;/a&gt; actually has a good solution for you. This approach is a tried and tested way to solve the same problem you are trying to solve.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you still want to concatenate the images and do this your way, you should &lt;strong&gt;concatenate the images along the channel dimension&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, by combining two &lt;span class=&quot;math-container&quot;&gt;$200\times 100 \times c$&lt;/span&gt; feature vectors (where c is the number of channels) you should get a single &lt;span class=&quot;math-container&quot;&gt;$200\times 100 \times 2c$&lt;/span&gt; feature vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kernels of the next convolution look through all the channels of the feature vector &lt;span class=&quot;math-container&quot;&gt;$x \times x$&lt;/span&gt; pixels at a time.&lt;br&gt;&#xA;If we combine along the channel dimension, it becomes easier for the network to compare pixel values at corresponding positions in both images. Since the objective is to predict similarity or dissimilarity, this is ideal for us.&lt;/p&gt;&#xA;" OwnerUserId="23994" LastEditorUserId="23994" LastEditDate="2019-04-18T23:44:20.160" LastActivityDate="2019-04-18T23:44:20.160" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11870" PostTypeId="1" AcceptedAnswerId="11873" CreationDate="2019-04-17T17:35:34.047" Score="5" ViewCount="1041" Body="&lt;p&gt;I have a CNN with the regression task of a single scalar.&#xA;I was wondering if an additional task of reconstructing the image (used for learning visual concepts), seen in a DeepMind &lt;a href=&quot;https://youtu.be/yV698Fi2XzE&quot; rel=&quot;nofollow noreferrer&quot;&gt;presentation&lt;/a&gt; with the loss and re-parametrization trick of Variational Autoencoder, might help the principal task of regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can imagine some convolutions with the role of feature extraction with some output X (let's say a vector of 256 values), that X goes into the VAE which computes Z and then the reconstructed image. And then the original regression task would take either X or Z in order to compute that scalar value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone tried such an approach, is it worth the work?&#xA;Thank you&lt;/p&gt;&#xA;" OwnerUserId="21918" LastEditorUserId="9947" LastEditDate="2019-04-17T21:48:15.307" LastActivityDate="2019-04-19T13:39:02.323" Title="Variational Autoencoder task for better feature extraction" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;unsupervised-learning&gt;&lt;autoencoders&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11871" PostTypeId="1" CreationDate="2019-04-17T18:15:29.557" Score="2" ViewCount="35" Body="&lt;p&gt;I have trained  a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%). &#xA;now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)&lt;/p&gt;&#xA;" OwnerUserId="23268" LastActivityDate="2019-04-18T05:47:54.210" Title="how to benefit from previous training weights in training again to increase accuracy?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11872" PostTypeId="2" ParentId="11812" CreationDate="2019-04-17T19:59:15.260" Score="1" Body="&lt;p&gt;If you're building a straight &quot;vanilla&quot; generative adversarial network, it's best to understand the network as a statistical engine:  You are training the generator &lt;em&gt;on samples of a statistical distribution&lt;/em&gt;.  (And you're training the discriminator to distinguish between &quot;ground truth&quot; images, and images from that generator.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you replace the input noise with another image... well.  Strictly speaking, it is probably still a generative adversarial network, if you're still doing everything else the same.  It is still a generator and a discriminator, acting in an adversarial fashion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you've radically altered the input distribution, so there is a good chance that you're no longer accomplishing what you want to accomplish unless you're being very careful and clever. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there are GAN variants which do take images rather than noise as inputs.  See the wonderful &lt;a href=&quot;https://junyanz.github.io/CycleGAN/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on CycleGANs by Zhu, et al, along with a substantial body of followup literature. And note that CycleGANs use not one, but two discriminators, so even here the discriminator is necessary.&lt;/p&gt;&#xA;" OwnerUserId="15020" LastEditorUserId="15020" LastEditDate="2019-04-18T18:15:47.993" LastActivityDate="2019-04-18T18:15:47.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11873" PostTypeId="2" ParentId="11870" CreationDate="2019-04-17T22:12:05.827" Score="2" Body="&lt;p&gt;I have not worked on this but I think I can give you a theoretical perspective of using VAE's. Regression is a Supervised Learning task and is basically a mapping from Input to Output where the Neural Net will approximate the function &lt;span class=&quot;math-container&quot;&gt;$f(input) = output$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VAE's on the other hand are good for finding how a latent variable affects the output. For example, if you have a task of training on a persons facial emotions, and if your latent space contains 2 variables &lt;span class=&quot;math-container&quot;&gt;$z_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z_2$&lt;/span&gt; then you might find varying &lt;span class=&quot;math-container&quot;&gt;$z_1$&lt;/span&gt; varies the amount of smile on the face, while varying &lt;span class=&quot;math-container&quot;&gt;$z_2$&lt;/span&gt; might give the amount of drooping of eyes. I suggest you check this &lt;a href=&quot;https://www.youtube.com/watch?v=5WoItGTWV54&amp;amp;t=2945s&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; from Stanford at ~44:00 to see this actually happens or check this &lt;a href=&quot;https://www.jeremyjordan.me/variational-autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;. So VAE's might have been useful if your output contained more features which would vary according to variations in latent variables, but a single scalar output can only tell you about the rate of effect on varying a latent variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if your job is only for better regression, Auto-Encoders are the better alternative, since it has an inherent &lt;a href=&quot;https://deepai.org/machine-learning-glossary-and-terms/denoising-autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;de-noising&lt;/a&gt; ability and sufficient training might help in de-noising the input, and thus provide better results if classified on the basis of latent variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approach, which I think is kind of similar (to your thinking) has been proposed by Kingma, et al. for Semi Supervised learning in this &lt;a href=&quot;https://arxiv.org/pdf/1406.5298.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. The paper has very poor description of the method so I would suggest you check out this &lt;a href=&quot;http://bjlkeng.github.io/posts/semi-supervised-learning-with-variational-autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;. They have used an additional classifier for reconstruction of the original input and trained the classifier when labels are present.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-04-19T13:39:02.323" LastActivityDate="2019-04-19T13:39:02.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11874" PostTypeId="2" ParentId="11866" CreationDate="2019-04-18T04:00:34.207" Score="2" Body="&lt;p&gt;The additive attention method that the researchers are comparing to corresponds to a neural network with 3 layers (it is not actually straight addition). Computing this will involve one multiplication of the input vector by a matrix, then by another matrix, and then the computation of something like a softmax. Smart implementation of a dot-product will not break out the whole matrix multiplication algorithm for it, and it will basically be a tight, easily parallelized loop.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-06T23:38:49.820" LastActivityDate="2021-01-06T23:38:49.820" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11875" PostTypeId="2" ParentId="11871" CreationDate="2019-04-18T05:38:38.783" Score="2" Body="&lt;p&gt;First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, there are several methods and considerations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use &quot;initial&quot; value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply &quot;nothing change&quot; in your architecture.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or, you can use the concept of the &lt;em&gt;selffer&lt;/em&gt; network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using &quot;initial&quot; value of hyperparameters&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can read more about &lt;a href=&quot;https://arxiv.org/pdf/1411.1792.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the transfer learning concept (or &lt;em&gt;selffer&lt;/em&gt; network)&lt;/a&gt; to get the most appropriate method for your case. There is also a paper about &lt;a href=&quot;https://arxiv.org/pdf/1803.10232.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;incremental training on CNN&quot;&lt;/a&gt; that I think is similar with the &lt;em&gt;selffer&lt;/em&gt; network but with some modifications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2019-04-18T05:47:54.210" LastActivityDate="2019-04-18T05:47:54.210" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11876" PostTypeId="2" ParentId="11825" CreationDate="2019-04-18T06:39:53.837" Score="1" Body="&lt;p&gt;If you only need the vector space as a way to obtain a similarity measure, you may want to consider a distance measure instead. Similarity and distance are inversely related: identical words have maximum similarity or zero distance, and as the similarity decreases, the distance increases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Wagner%E2%80%93Fischer_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wagner-Fischer algorithm&lt;/a&gt; computes the edit distance between two strings of characters. This edit distance takes into acccount insertions and deletions, as in your examples, but also substitutions (for example &quot;gray&quot; vs. &quot;grey&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article linked above includes pseudocode that should translate easily to actual code.&lt;/p&gt;&#xA;" OwnerUserId="24014" LastEditorUserId="24014" LastEditDate="2019-04-18T14:17:47.110" LastActivityDate="2019-04-18T14:17:47.110" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11878" PostTypeId="1" AcceptedAnswerId="11879" CreationDate="2019-04-18T08:16:04.020" Score="2" ViewCount="59" Body="&lt;p&gt;I'm working on implementing a Q-Learning algorithm for a 2 player board game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HJXIZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HJXIZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I encountered what I think may be a problem. When it comes time to update the Q value with the Bellman equation (above), the last part states that for the maximum expected reward, one must find the highest q value in the new state reached, &lt;code&gt;s'&lt;/code&gt;, after making action &lt;code&gt;a&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it seems like the I never have q values for state &lt;code&gt;s'&lt;/code&gt;. I suspect &lt;code&gt;s'&lt;/code&gt; can only be reached from P2 making a move. It may be impossible for this state to be reached as a result of an action from P1. Therefore, the board state &lt;code&gt;s'&lt;/code&gt; is never evaluated by P2, thus its Q values are never being computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will try to paint a picture of what I mean. Assume P1 is a random player, and P2 is the learning agent.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;P1 makes a random move, resulting in state &lt;code&gt;s&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;P2 evaluates board &lt;code&gt;s&lt;/code&gt;, finds the best action and takes it, resulting in state &lt;code&gt;s'&lt;/code&gt;. In the process of updating the Q value for the pair &lt;code&gt;(s,a)&lt;/code&gt;, it finds &lt;code&gt;maxQ'(s', a) = 0&lt;/code&gt;, since the state hasn't been encountered yet.&lt;/li&gt;&#xA;&lt;li&gt;From &lt;code&gt;s'&lt;/code&gt;, P1 again makes a random move.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As you can see, state &lt;code&gt;s'&lt;/code&gt; is never encountered by P2, since it is a board state that appears only as a result of P2 making a move. Thus the last part of the equation will always result in &lt;code&gt;0 - current Q value&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I seeing this correctly? Does this affect the learning process? Any input would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="24018" LastActivityDate="2019-04-18T09:26:30.120" Title="Maximum Q value for new state in Q-Learning never exists" Tags="&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11879" PostTypeId="2" ParentId="11878" CreationDate="2019-04-18T09:26:30.120" Score="1" Body="&lt;p&gt;Your problem is with how you have defined &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next state for an agent is &lt;em&gt;not&lt;/em&gt; the state that the agent's action immediately puts the environment into. It is the state when it next takes an action. For some, more passive, environments, these are the same things. But for many environments they are not. For instance, a robot that is navigating a maze may take an action to move forward. The next state does not happen immediately at the point that it starts to take the action (when it would still be in the same position), but at a later time, after the action has been processed by the environment for a while (and the robot is in a new position), and the robot is ready to take another action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in your 2-player game example using regular Q learning, the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; for P2 is &lt;em&gt;not&lt;/em&gt; the state immediately after P2's move, but the state after P1 has also played its move in reaction. From P2's perspective, P1 is part of the environment and the situation is no different to having a stochastic environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you take this perspective on what &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is, then Q learning will work as normal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you should note that optimal behaviour against a specific opponent - such as a random opponent - is not the same as optimal play in a game. There are other ways to apply Reinforcement Learning ideas to 2-player games. Some of them can use the same approach as above - e.g. train two agents, one for P1 and one for P2, with each treating the other as if it were part of the environment. Others use different ways of reversing the view of the agent so that it can play versus itself more directly - in those cases you &lt;em&gt;can&lt;/em&gt; treat each player's immediate output as &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, but you need to modify the learning agent. A simple modification to Q learning is to alternate between taking &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{min}_{a'}$&lt;/span&gt; depending on whose turn you are evaluating (and assuming P1's goal is to maximise their score while P2's goal is to minimise P1's score - and by extension maximise their own score in any zero-sum game)&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-18T09:26:30.120" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11880" PostTypeId="1" CreationDate="2019-04-18T11:59:14.253" Score="0" ViewCount="51" Body="&lt;p&gt;I am trying to train an ANN to control a 7 Degrees-Of-Freedom arm. It should reach a target avoiding a single obstacle. Given my modeling of the situation, my input layer is composed of 12 nodes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;5 nodes for the 5 joint states &lt;/li&gt;&#xA;&lt;li&gt;3 nodes for the cartesian coordinates of the target&lt;/li&gt;&#xA;&lt;li&gt;3 nodes for the cartesian coordinates of the obstacle&lt;/li&gt;&#xA;&lt;li&gt;1 node for the radius of the obstacle (it's a spherical object).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have already tried training the ANN with DQN. I want to try neuroevolution (NEAT, in particular) and see how the results compare. I am using &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT-python&lt;/a&gt;. As seen in &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/dsilva.msreport06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, this should be feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am having trouble choosing the best fitness function and also some other hyperparameters, namely the population size. (I am also puzzled by the extremely long training time for a single generation, but that's another story.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the &lt;strong&gt;fitness function&lt;/strong&gt;. I have tried to replicate what I have done with DQN. So, basically, my function evaluates a genome (so, an ANN) as follows (pseudocode):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;counter = 0&#xA;reapeat for NUM_OF_EPISODES times:&#xA;    generate a random target&#xA;    generate an obstacle which lies about halfway from the end-effector to the target&#xA;    repeat for TIMEOUT times:&#xA;        use the ANN to decide the next_action and execute it&#xA;        if OBSTACLE_REACHED or TARGET_REACHED, stop &#xA;    counter += 0.3 * relative_distance + 0.2 * relative_path + 0.5 * didntHitObstacle()&#xA;fitness = counter / NUM_OF_EPISODES&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, humanly speaking, for each ANN we try to execute &lt;code&gt;NUM_OF_EPISODES&lt;/code&gt; times (how many times should be enough? 100 times seems ok but it get's really slow) a scenario. In this scenario, we use the ANN to search for the target, and if we reach it or the obstacle, we stop. Now for each one of these scenarios, we should &quot;rank&quot; how well the ANN performed (see the counter += ... part). But how should I do this? My idea (stolen from the paper above) was to compute something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.3 * relative_distance + 0.2 * relative_path + 0.5 * didntHitObstacle()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, basically, we see how much we are closer to the target compared to when we started, how &quot;short&quot; the path was (compared to the ideal straight line start point-to-target), and whether we did or did not hit the target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this function make sense? My concern is mainly about how we deal with the obstacle: 50% of the fitness. Is it correct? I am asking this because I am receiving poor results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem that I have is &lt;strong&gt;population size&lt;/strong&gt;. Of course, the bigger the better, but this thing takes &lt;em&gt;a lot&lt;/em&gt; to train. How big is ok, in your experience?&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2019-07-07T20:47:50.157" LastActivityDate="2019-07-07T20:47:50.157" Title="How do I choose an appropriate fitness function and hyper-parameters to train a 7-DOF arm?" Tags="&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;robotics&gt;&lt;neuroevolution&gt;" AnswerCount="0" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11882" PostTypeId="2" ParentId="11833" CreationDate="2019-04-18T15:00:27.223" Score="1" Body="&lt;p&gt;For cross-language word representation the trend now is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;ELMoForManyLangs&lt;/strong&gt;: &lt;a href=&quot;https://github.com/HIT-SCIR/ELMoForManyLangs&quot; rel=&quot;nofollow noreferrer&quot;&gt;git_repo&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/pdf/1802.05365.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original_paper_March_2018&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;MUSE by Facebook&lt;/strong&gt;: &lt;a href=&quot;https://github.com/facebookresearch/MUSE&quot; rel=&quot;nofollow noreferrer&quot;&gt;git_repo&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/pdf/1710.04087.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original_paper_January_2018&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Remember that you can also do the task in 2 steps: &#xA;&lt;strong&gt;Translate&lt;/strong&gt; the words to a reference language (e.g english), then &lt;strong&gt;represent&lt;/strong&gt; each one of them using any word representation model (in the reference language). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2-steps option is also good as specific-language word representation models are more accurate, and there are a bench of easy-to-use libraries for single-language translation (&lt;em&gt;i.e &lt;a href=&quot;https://pypi.org/project/py-translator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;py-translator&lt;/a&gt;&lt;/em&gt;) and representation (&lt;em&gt;i.e &lt;a href=&quot;https://tfhub.dev/google/universal-sentence-encoder/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal sentence encoder by Google&lt;/a&gt;&lt;/em&gt;).    &lt;/p&gt;&#xA;" OwnerUserId="23350" LastActivityDate="2019-04-18T15:00:27.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11883" PostTypeId="1" CreationDate="2019-04-18T17:29:34.560" Score="2" ViewCount="318" Body="&lt;p&gt;I want to train an AI to detect the class (i.e. suit and rank) of playing cards. Playing cards from different decks may use slightly different shapes or colors to represent these attributes, and I want the system to work across many decks. I bought many different decks, scanned and labeled them. Next up would be to create training data with an augmentation library. I found two examples of how other people did that:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tooploox.com/blog/card-detection-using-yolo-on-android&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image detection using YOLO algorithm and poker cards&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=pnntrewH0xg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing card detection with YOLO&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The problem is that I want my AI to be able to detect multiple cards of the same class in one picture. In the examples above, they put a label on the top left corner of a card. This makes sense since it is a very good indicator of what class the card is in. Unfortunately, every card has two of these labels.&lt;/p&gt;&#xA;&lt;p&gt;I think their solution returns &amp;quot;detected&amp;quot; if any instance of a given label is found. But many cards with the same label could be in the same picture, and I am not sure if I can detect the quantity of the class with this solution. For example, if there are 2 Ace of Clubs in a picture, I would like my system to output &amp;quot;2&amp;quot;, rather than &amp;quot;detected&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Do you think it is feasible to mark the whole card, prepare the data accordingly, and train an AI that detects the count as well?&lt;/p&gt;&#xA;" OwnerUserId="24028" LastEditorUserId="16909" LastEditDate="2021-01-13T00:31:14.160" LastActivityDate="2021-02-12T01:05:40.607" Title="How to detect multiple playing cards of the same class with a neural network?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;yolo&gt;&lt;data-labelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11884" PostTypeId="1" AcceptedAnswerId="11887" CreationDate="2019-04-18T20:24:54.033" Score="2" ViewCount="22" Body="&lt;p&gt;I quite often find projects using pre-trained model and using them as a starting point for their new model that learns something novel from thier dataset or on-live learning process - e.g. using a webcam or live audio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this quite usual and recommended to speed up training a model? For example using a model trained on ImageNet as a first layer to your model that will categorise faces specifically.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2019-04-18T22:57:22.557" Title="Is it mostly the case to train with available models" Tags="&lt;training&gt;&lt;models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11885" PostTypeId="1" CreationDate="2019-04-18T20:58:27.530" Score="2" ViewCount="954" Body="&lt;p&gt;I've been trying to implement policy improvement for Q(s,a) function as per Sutton&amp;amp;Barto reinforcement learning book. The original algorithm with first-visit MonteCarlo is pictured below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ODu0o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ODu0o.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I remember the book earlier mentioning that the every-visit variation simply omits the first-visit check &quot;unless the pair blah,blah...&quot; but otherwise the algorithms should be the same (???)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial policy in the very first iteration (first episode), should be equiprobable randomwalk. In general terms, if action takes you outside the border of the gridworld (4x4), then you simply bounce back into where you started from, but reward will have been given, and action will have been taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have verified that my code gets stuck (sometimes) in the episode generation portion of my code in foreverloop for some reason, early on in the code (iterations amount in the outerloop is small). Even though, I thought I followed the pseudocode rather well, but it's really annoying that sometimes the code gets stuck in foreverloop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason must be that for some reason my code updates from the equiprobable randomwalk policy =&gt; into deterministic policy but in a wrong way, such that it can create foreverloops in the episode generation after the first episode has ran (it must have ran the entire first episode with equiprobable randomwalk). The below picture shows that if you get into any state in the marked box, you cannot get out from there and get stuck in episode generation...&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ID7Qn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ID7Qn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the random generator seed it lucky, then it can usually get &quot;over the hump&quot; and proceed to make the required number of iterations in the outerloop, and output an optimal policy for the gridworld.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the python code (from my experience I ran it in debugger and had to restart it a couple of times, but usually it will show quite fast that it gets stuck into the episode generation, and cannot proceed in the iterations in outerloop.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import numpy.linalg as LA&#xA;import random&#xA;from datetime import datetime&#xA;&#xA;random.seed(datetime.now())&#xA;rows_count = 4&#xA;columns_count = 4&#xA;&#xA;def isTerminal(r, c):  # helper function to check if terminal state or regular state&#xA;    global rows_count, columns_count&#xA;    if r == 0 and c == 0:  # im a bit too lazy to check otherwise the iteration boundaries&#xA;        return True  # so that this helper function is a quick way to exclude computations&#xA;    if r == rows_count - 1 and c == columns_count - 1:&#xA;        return True&#xA;    return False&#xA;&#xA;&#xA;maxiters = 100000&#xA;reward = -1&#xA;actions = [&quot;U&quot;, &quot;R&quot;, &quot;D&quot;, &quot;L&quot;]&#xA;V = np.zeros((rows_count, columns_count))&#xA;returnsDict={}&#xA;QDict={}&#xA;actDict={0:&quot;U&quot;,1:&quot;R&quot;,2:&quot;D&quot;,3:&quot;L&quot;}&#xA;policies = np.array([ ['T','A','A','A'],&#xA;                     ['A','A','A','A'],&#xA;                     ['A','A','A','A'],&#xA;                     ['A','A','A','T'] ])&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&quot;&quot;&quot;returnsDict, for each state-action pair, maintain (mean,visitedCount)&quot;&quot;&quot;&#xA;for r in range(rows_count):&#xA;    for c in range(columns_count):&#xA;        if not isTerminal(r, c):&#xA;            for act in actions:&#xA;                returnsDict[ ((r, c), act) ] = [0, 0] ## Maintain Mean, and VisitedCount for each state-action pair&#xA;&#xA;&#xA;&#xA;&quot;&quot;&quot; Qfunc, we maintain the action-value for each state-action pair&quot;&quot;&quot;&#xA;for r in range(rows_count):&#xA;    for c in range(columns_count):&#xA;        if not isTerminal(r, c):&#xA;            for act in actions:&#xA;                QDict[ ((r,c), act) ] = -9999  ## Maintain Q function value for each state-action pair&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;def getValue(row, col):  # helper func, get state value&#xA;    global V&#xA;    if row == -1:&#xA;        row = 0  # if you bump into wall, you bounce back&#xA;    elif row == 4:&#xA;        row = 3&#xA;    if col == -1:&#xA;        col = 0&#xA;    elif col == 4:&#xA;        col = 3&#xA;    return V[row, col]&#xA;&#xA;def getRandomStartState():&#xA;    illegalState = True&#xA;&#xA;    while illegalState:&#xA;        r = random.randint(0, 3)&#xA;        c = random.randint(0, 3)&#xA;        if (r == 0 and c == 0) or (r == 3 and c == 3):&#xA;            illegalState = True&#xA;        else:&#xA;            illegalState = False&#xA;    return r, c&#xA;&#xA;def getState(row, col):&#xA;    if row == -1:&#xA;        row = 0  # helper func for the exercise:1&#xA;    elif row == 4:&#xA;        row = 3&#xA;    if col == -1:&#xA;        col = 0&#xA;    elif col == 4:&#xA;        col = 3&#xA;    return row, col&#xA;&#xA;&#xA;&#xA;def getRandomAction():&#xA;    global actDict&#xA;    return actDict[random.randint(0, 3)]&#xA;&#xA;&#xA;def getMeanFromReturns(oldMean, n, curVal):&#xA;    newMean = 0&#xA;    if n == 0:&#xA;        raise Exception('Exception, incrementalMeanFunc, n should not be less than 1')&#xA;    elif n == 1:&#xA;        return curVal&#xA;    elif n &amp;gt;= 2:&#xA;        newMean = (float) ( oldMean + (1.0 / n) * (curVal - oldMean) )&#xA;        return newMean&#xA;&#xA;&#xA;&quot;&quot;&quot;get the best action &#xA;returns string action&#xA;parameter is state tuple (r,c)&quot;&quot;&quot;&#xA;def getArgmaxActQ(S_t):&#xA;    global QDict&#xA;    qvalList = []&#xA;    saList = []&#xA;&#xA;    &quot;&quot;&quot;for example get together&#xA;    s1a1, s1a2, s1a3, s1a4&#xA;    find which is the maxValue, and get the action which caused it&quot;&quot;&quot;&#xA;    sa1 = (S_t, &quot;U&quot;)&#xA;    sa2 = (S_t, &quot;R&quot;)&#xA;    sa3 = (S_t, &quot;D&quot;)&#xA;    sa4 = (S_t, &quot;L&quot;)&#xA;    saList.append(sa1)&#xA;    saList.append(sa2)&#xA;    saList.append(sa3)&#xA;    saList.append(sa4)&#xA;&#xA;    q1 = QDict[sa1]&#xA;    q2 = QDict[sa2]&#xA;    q3 = QDict[sa3]&#xA;    q4 = QDict[sa4]&#xA;    qvalList.append(q1)&#xA;    qvalList.append(q2)&#xA;    qvalList.append(q3)&#xA;    qvalList.append(q4)&#xA;&#xA;    maxQ = max(qvalList)&#xA;    ind_maxQ = qvalList.index(maxQ)  # gets the maxQ value and the index which caused it&#xA;&#xA;    &quot;&quot;&quot;when we have index of maxQval, then we know which sa-pair&#xA;    gave that maxQval =&amp;gt; we can access that action from the correct sa-pair&quot;&quot;&quot;&#xA;    argmaxAct = saList[ind_maxQ][1]&#xA;    return argmaxAct&#xA;&#xA;&quot;&quot;&quot;QEpisode generation func&#xA;returns episodeList&#xA;parameters are starting state, starting action&quot;&quot;&quot;&#xA;def QEpisode(r, c, act):&#xA;    global reward&#xA;    global policies&#xA;&#xA;    &quot;&quot;&quot;NOTE! r,c will both be local variables inside this func&#xA;    they denote the nextState (s') in this func&quot;&quot;&quot;&#xA;    stateWasTerm = False&#xA;    stepsTaken = 0&#xA;    curR = r&#xA;    curC = c&#xA;    episodeList = [ ((r, c), act, reward) ]  # add the starting (s,a) immediately&#xA;&#xA;    if act == &quot;U&quot;:  ##up&#xA;        r -= 1&#xA;    elif act == &quot;R&quot;:  ##right&#xA;        c += 1&#xA;    elif act == &quot;D&quot;:  ## down&#xA;        r += 1&#xA;    else:  ##left&#xA;        c -= 1&#xA;    stepsTaken += 1&#xA;    r, c = getState(r, c)  ## check status of the newState (s')&#xA;    stateWasTerm = isTerminal(r, c)  ## if status was terminal stop iteration, else keep going into loop&#xA;&#xA;    if not stateWasTerm:&#xA;        curR = r&#xA;        curC = c&#xA;&#xA;    while not stateWasTerm:&#xA;        if policies[curR, curC] == &quot;A&quot;:&#xA;            act = getRandomAction()  ## &quot;&quot;&quot;get the random action from policy&quot;&quot;&quot;&#xA;        else:&#xA;            act = policies[curR, curC]  ## &quot;&quot;&quot;get the deterministic action from policy&quot;&quot;&quot;&#xA;&#xA;        if act == &quot;U&quot;:  ## up&#xA;            r -= 1&#xA;        elif act == &quot;R&quot;:  ## right&#xA;            c += 1&#xA;        elif act == &quot;D&quot;:  ## down&#xA;            r += 1&#xA;        else:  ## left&#xA;            c -= 1&#xA;        stepsTaken += 1&#xA;&#xA;        r, c = getState(r, c)&#xA;        stateWasTerm = isTerminal(r, c)&#xA;        episodeList.append( ((curR, curC), act, reward) )&#xA;        if not stateWasTerm:&#xA;            curR = r&#xA;            curC = c&#xA;&#xA;    return episodeList&#xA;&#xA;&#xA;&#xA;&#xA;print(&quot;montecarlo program starting...\n&quot;)&#xA;&quot;&quot;&quot; MOnte Carlo Q-function, exploring starts, every-visit, estimating Pi ~~ Pi* &quot;&quot;&quot;&#xA;for iteration in range(1, maxiters+1): ## for all episodes&#xA;&#xA;    print(&quot;curIter == &quot;, iteration)&#xA;    print(&quot;\n&quot;)&#xA;    if iteration % 20 == 0: ## get random seed periodically to improve randomness performance&#xA;        random.seed(datetime.now())&#xA;&#xA;&#xA;&#xA;    for r in range(4):&#xA;        for c in range(4):&#xA;            if not isTerminal(r,c):&#xA;                startR = r&#xA;                startC = c&#xA;                startAct = getRandomAction()&#xA;&#xA;&#xA;   ## startR, startC = getRandomStartState() ## get random starting-state, and starting action equiprobably&#xA;  ##  startAct = getRandomAction()&#xA;                sequence = QEpisode(startR, startC, startAct)  ## generate Q-sequence following policy Pi, until terminal-state (excluding terminal)&#xA;                G = 0&#xA;&#xA;                for t in reversed(range(len(sequence))): ## iterate through the timesteps in reversed order&#xA;                    S_t = sequence[t][0] ## use temp variables as helpers&#xA;                    A_t = sequence[t][1]&#xA;                    R_t = sequence[t][2]&#xA;                    G += R_t ## increment G with reward, NOTE! the gamma discount factor == 1.0&#xA;                    visitedCount = returnsDict[S_t, A_t][1] ## use temp visitedcount&#xA;                    visitedCount += 1&#xA;&#xA;                    if visitedCount == 1: ## special case in iterative mean algorithm, the first visit to any state-action pair&#xA;                        curMean = 9999&#xA;                        curMean = getMeanFromReturns(curMean, visitedCount, G)&#xA;                        returnsDict[S_t, A_t][0] = curMean ## update mean&#xA;                        returnsDict[S_t, A_t][1] = visitedCount ## update visitedcount&#xA;                    else:&#xA;                        curMean = returnsDict[S_t, A_t][0] ## get temp mean from returnsDict&#xA;                        curMean = getMeanFromReturns(curMean, visitedCount, G) ## get the new temp mean iteratively&#xA;                        returnsDict[S_t, A_t][1] = visitedCount ## update visitedcount&#xA;                        returnsDict[S_t, A_t][0] = curMean ## update mean&#xA;&#xA;&#xA;                    QDict[S_t, A_t] = returnsDict[S_t, A_t][0] ## update the Qfunction with the new mean value&#xA;                    tempR = S_t[0] ## temp variables simply to disassemble the tuple into row,col&#xA;                    tempC = S_t[1]&#xA;                    policies[tempR, tempC] = getArgmaxActQ(S_t) ## update policy based on argmax_a[Q(S_t)]&#xA;&#xA;&#xA;print(&quot;optimal policy with Monte-Carlo, every visit was \n&quot;)&#xA;print(&quot;\n&quot;)&#xA;print(policies)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the updated &quot;hacky fix&quot; code that seems to get the algorithm &quot;over the hump&quot; without getting stuck into foreverloop with deterministic policy. My teacher had recommended that you don't need to update policy at every step in this kind of Monte Carlo, so you could have made the policy updates at periodic intervals using python's modulo operator on the iterationsCount or something.&#xA;Also, I had the bright idea that the Sutton&amp;amp;Barto book described that all state-action pairs must be visited, very large amount of times, for the exploring starts pre-condition of the algorithm to be fulfilled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I then decided to enforce the algorithm to have run at least once for all state-action pairs so you start from each state-action pair deterministically one-by-one (for each episode actually). This would still be run with the old randomwalk policy in this early exploration phase, where we are gathering data into the returnsDict, and Qdict, but not yet improving deterministic policy.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import numpy.linalg as LA&#xA;import random&#xA;from datetime import datetime&#xA;&#xA;random.seed(datetime.now())&#xA;rows_count = 4&#xA;columns_count = 4&#xA;&#xA;def isTerminal(r, c):  # helper function to check if terminal state or regular state&#xA;    global rows_count, columns_count&#xA;    if r == 0 and c == 0:  # im a bit too lazy to check otherwise the iteration boundaries&#xA;        return True  # so that this helper function is a quick way to exclude computations&#xA;    if r == rows_count - 1 and c == columns_count - 1:&#xA;        return True&#xA;    return False&#xA;&#xA;&#xA;&#xA;&quot;&quot;&quot;NOTE about maxiters!!!&#xA;the Monte-Carlo every visit algorithm implements total amount of iterations with formula&#xA;totalIters = maxiters * nonTerminalStates * possibleActions&#xA;totalIters = 5000 * 14 * 4&#xA;totalIters = 280000&#xA;&#xA;in other words, there will be 5k iterations per each state-action pair&#xA;in other words there will be an early exploration phase where policy willnot be updated,&#xA;but the gridworld will be explored with randomwalk policy, gathering Qfunc information, &#xA;and returnDict information.&#xA;&#xA;in early phase there will be about 27 iterations for each state-action pair during,&#xA;non-policy-updating exploration &#xA;(maxiters * explorationFactor) / (stateACtionPairs) = 7500 *0.2 /56&#xA;&#xA;after that early exploring with randomwalk,&#xA;then we act greedily w.r.t. the Q-function, &#xA;for the rest of the iterations to get the optimal deterministic policy&#xA;&quot;&quot;&quot;&#xA;maxiters = 7500&#xA;explorationFactor = 0.2 ## explore that percentage of the first maxiters rounds, try to increase it, if you get stuck in foreverloop, in QEpisode function&#xA;reward = -1&#xA;actions = [&quot;U&quot;, &quot;R&quot;, &quot;D&quot;, &quot;L&quot;]&#xA;V = np.zeros((rows_count, columns_count))&#xA;returnsDict={}&#xA;QDict={}&#xA;actDict={0:&quot;U&quot;,1:&quot;R&quot;,2:&quot;D&quot;,3:&quot;L&quot;}&#xA;policies = np.array([ ['T','A','A','A'],&#xA;                     ['A','A','A','A'],&#xA;                     ['A','A','A','A'],&#xA;                     ['A','A','A','T'] ])&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&quot;&quot;&quot;returnsDict, for each state-action pair, maintain (mean,visitedCount)&quot;&quot;&quot;&#xA;for r in range(rows_count):&#xA;    for c in range(columns_count):&#xA;        if not isTerminal(r, c):&#xA;            for act in actions:&#xA;                returnsDict[ ((r, c), act) ] = [0, 0] ## Maintain Mean, and VisitedCount for each state-action pair&#xA;&#xA;&#xA;&#xA;&quot;&quot;&quot; Qfunc, we maintain the action-value for each state-action pair&quot;&quot;&quot;&#xA;for r in range(rows_count):&#xA;    for c in range(columns_count):&#xA;        if not isTerminal(r, c):&#xA;            for act in actions:&#xA;                QDict[ ((r,c), act) ] = -9999  ## Maintain Q function value for each state-action pair&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;def getValue(row, col):  # helper func, get state value&#xA;    global V&#xA;    if row == -1:&#xA;        row = 0  # if you bump into wall, you bounce back&#xA;    elif row == 4:&#xA;        row = 3&#xA;    if col == -1:&#xA;        col = 0&#xA;    elif col == 4:&#xA;        col = 3&#xA;    return V[row, col]&#xA;&#xA;def getRandomStartState():&#xA;    illegalState = True&#xA;&#xA;    while illegalState:&#xA;        r = random.randint(0, 3)&#xA;        c = random.randint(0, 3)&#xA;        if (r == 0 and c == 0) or (r == 3 and c == 3):&#xA;            illegalState = True&#xA;        else:&#xA;            illegalState = False&#xA;    return r, c&#xA;&#xA;def getState(row, col):&#xA;    if row == -1:&#xA;        row = 0  # helper func for the exercise:1&#xA;    elif row == 4:&#xA;        row = 3&#xA;    if col == -1:&#xA;        col = 0&#xA;    elif col == 4:&#xA;        col = 3&#xA;    return row, col&#xA;&#xA;&#xA;&#xA;def getRandomAction():&#xA;    global actDict&#xA;    return actDict[random.randint(0, 3)]&#xA;&#xA;&#xA;def getMeanFromReturns(oldMean, n, curVal):&#xA;    newMean = 0&#xA;    if n == 0:&#xA;        raise Exception('Exception, incrementalMeanFunc, n should not be less than 1\n')&#xA;    elif n == 1:&#xA;        return curVal&#xA;    elif n &amp;gt;= 2:&#xA;        newMean = (float) ( oldMean + (1.0 / n) * (curVal - oldMean) )&#xA;        return newMean&#xA;&#xA;&#xA;&quot;&quot;&quot;get the best action &#xA;returns string action&#xA;parameter is state tuple (r,c)&quot;&quot;&quot;&#xA;def getArgmaxActQ(S_t):&#xA;    global QDict&#xA;    qvalList = []&#xA;    saList = []&#xA;&#xA;    &quot;&quot;&quot;for example get together&#xA;    s1a1, s1a2, s1a3, s1a4&#xA;    find which is the maxValue, and get the action which caused it&quot;&quot;&quot;&#xA;    sa1 = (S_t, &quot;U&quot;)&#xA;    sa2 = (S_t, &quot;R&quot;)&#xA;    sa3 = (S_t, &quot;D&quot;)&#xA;    sa4 = (S_t, &quot;L&quot;)&#xA;    saList.append(sa1)&#xA;    saList.append(sa2)&#xA;    saList.append(sa3)&#xA;    saList.append(sa4)&#xA;&#xA;    q1 = QDict[sa1]&#xA;    q2 = QDict[sa2]&#xA;    q3 = QDict[sa3]&#xA;    q4 = QDict[sa4]&#xA;    qvalList.append(q1)&#xA;    qvalList.append(q2)&#xA;    qvalList.append(q3)&#xA;    qvalList.append(q4)&#xA;&#xA;    maxQ = max(qvalList)&#xA;    ind_maxQ = qvalList.index(maxQ)  # gets the maxQ value and the index which caused it&#xA;&#xA;    &quot;&quot;&quot;when we have index of maxQval, then we know which sa-pair&#xA;    gave that maxQval =&amp;gt; we can access that action from the correct sa-pair&quot;&quot;&quot;&#xA;    argmaxAct = saList[ind_maxQ][1]&#xA;    return argmaxAct&#xA;&#xA;&#xA;&#xA;&#xA;&quot;&quot;&quot;QEpisode generation func&#xA;returns episodeList&#xA;parameters are starting state, starting action&quot;&quot;&quot;&#xA;def QEpisode(r, c, act):&#xA;&#xA;    &quot;&quot;&quot;ideally, we should not get stuck in the gridworld...but,&#xA;    but sometiems when policy transitions from the first episode's policy == randomwalk,&#xA;    then, on second episode sometimes we get stuck in foreverloop in episode generation&#xA;    usually the only choice then seems to restart the entire policy into randomwalk ??? &quot;&quot;&quot;&#xA;&#xA;    global reward&#xA;    global policies&#xA;&#xA;    &quot;&quot;&quot;NOTE! r,c will both be local variables inside this func&#xA;    they denote the nextState (s') in this func&quot;&quot;&quot;&#xA;    stepsTaken = 0&#xA;    curR = r&#xA;    curC = c&#xA;    episodeList = [ ((r, c), act, reward) ]  # add the starting (s,a) immediately&#xA;&#xA;    if act == &quot;U&quot;:  ##up&#xA;        r -= 1&#xA;    elif act == &quot;R&quot;:  ##right&#xA;        c += 1&#xA;    elif act == &quot;D&quot;:  ## down&#xA;        r += 1&#xA;    elif act == &quot;L&quot;:  ##left&#xA;        c -= 1&#xA;    stepsTaken += 1&#xA;    r, c = getState(r, c)  ## check status of the newState (s')&#xA;    stateWasTerm = isTerminal(r, c)  ## if status was terminal stop iteration, else keep going into loop&#xA;&#xA;    if not stateWasTerm:&#xA;        curR = r&#xA;        curC = c&#xA;&#xA;    while not stateWasTerm:&#xA;        if policies[curR, curC] == &quot;A&quot;:&#xA;            act = getRandomAction()  ## &quot;&quot;&quot;get the random action from policy&quot;&quot;&quot;&#xA;        else:&#xA;            act = policies[curR, curC]  ## &quot;&quot;&quot;get the deterministic action from policy&quot;&quot;&quot;&#xA;&#xA;        if act == &quot;U&quot;:  ## up&#xA;            r -= 1&#xA;        elif act == &quot;R&quot;:  ## right&#xA;            c += 1&#xA;        elif act == &quot;D&quot;:  ## down&#xA;            r += 1&#xA;        else:  ## left&#xA;            c -= 1&#xA;        stepsTaken += 1&#xA;&#xA;        r, c = getState(r, c)&#xA;        stateWasTerm = isTerminal(r, c)&#xA;        episodeList.append( ((curR, curC), act, reward) )&#xA;        if not stateWasTerm:&#xA;            curR = r&#xA;            curC = c&#xA;        if stepsTaken &amp;gt;= 100000:&#xA;            raise Exception(&quot;Exception raised, because program got stuck in MC Qepisode generation...\n&quot;)&#xA;&#xA;&#xA;    return episodeList&#xA;&#xA;&#xA;&#xA;&#xA;print(&quot;montecarlo program starting...\n&quot;)&#xA;&quot;&quot;&quot; MOnte Carlo Q-function, exploring starts, every-visit, estimating Pi ~~ Pi* &quot;&quot;&quot;&#xA;&#xA;&quot;&quot;&quot;It appears that the Qfunction apparently can be unreliable in the early episodes rounds, so we can avoid getting &#xA;stuck in foreverloop because of unreliable early episodes, BUT...&#xA;&#xA;we gotta delay updating the policy, until we have explored enough for a little bit...&#xA;so our Qfunction has reliable info inside of it, to base the decision on, later...&quot;&quot;&quot;&#xA;Q_function_is_reliable = False ## variable shows if we are currently updating the policy, or just improving Q-function and exploring&#xA;&#xA;&#xA;for iteration in range(1, maxiters+1): ## for all episodes&#xA;&#xA;    print(&quot;curIter == &quot;, iteration, &quot;, QfunctionIsReliable == &quot;, Q_function_is_reliable )&#xA;    print(&quot;\n&quot;)&#xA;    if iteration % 20 == 0: ## get random seed periodically to improve randomness performance&#xA;        random.seed(datetime.now())&#xA;&#xA;    for r in range(4):  ## for every non-terminal-state&#xA;        for c in range(4):&#xA;            if not isTerminal(r,c):&#xA;                startR = r&#xA;                startC = c&#xA;                for act in actions: ## for every action possible&#xA;                    startAct = act&#xA;                    sequence = QEpisode(startR, startC, startAct)  ## generate Q-sequence following policy Pi, until terminal-state (excluding terminal)&#xA;                    G = 0&#xA;&#xA;                    for t in reversed(range(len(sequence))): ## iterate through the timesteps in reversed order&#xA;                        S_t = sequence[t][0] ## use temp variables as helpers&#xA;                        A_t = sequence[t][1]&#xA;                        R_t = sequence[t][2]&#xA;                        G += R_t ## increment G with reward, gamma discount factor is zero&#xA;                        visitedCount = returnsDict[S_t, A_t][1]&#xA;                        visitedCount += 1&#xA;&#xA;                       ## if (S_t, A_t, -1) not in sequence[:t]: ## This is how you COULD have done the first-visit MC, but we do every-visit now...&#xA;                        if visitedCount == 1: ## special case in iterative mean algorithm, the first visit to any state-action pair&#xA;                            curMean = 9999&#xA;                            curMean = getMeanFromReturns(curMean, visitedCount, G)&#xA;                            returnsDict[S_t, A_t][0] = curMean ## update mean&#xA;                            returnsDict[S_t, A_t][1] = visitedCount ## update visitedcount&#xA;                        else:&#xA;                            curMean = returnsDict[S_t, A_t][0] ## get temp mean from returnsDict&#xA;                            curMean = getMeanFromReturns(curMean, visitedCount, G) ## get the new temp mean iteratively&#xA;                            returnsDict[S_t, A_t][1] = visitedCount ## update visitedcount&#xA;                            returnsDict[S_t, A_t][0] = curMean ## update mean&#xA;&#xA;&#xA;                        QDict[S_t, A_t] = returnsDict[S_t, A_t][0] ## update the Qfunction with the new mean value&#xA;                        tempR = S_t[0] ## temp variables simply to disassemble the tuple into row,col&#xA;                        tempC = S_t[1]&#xA;&#xA;                        if iteration &amp;gt;= round(maxiters * explorationFactor): ## ONLY START UPDATING POLICY when we have reliable estimates for Qfunction, that is when iteration &amp;gt; maxiter/10&#xA;                            Q_function_is_reliable = True&#xA;                            policies[tempR, tempC] = getArgmaxActQ(S_t) ## update policy based on argmax_a[Q(S_t)]&#xA;&#xA;&#xA;print(&quot;optimal policy with Monte-Carlo, every visit was \n&quot;)&#xA;print(&quot;\n&quot;)&#xA;print(policies)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="23915" LastEditDate="2019-04-19T09:42:33.403" LastActivityDate="2020-06-03T16:38:55.097" Title="Monte-Carlo, every-visit gridworld, exploring starts, python code gets stuck in foreverloop in episode generation" Tags="&lt;python&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11887" PostTypeId="2" ParentId="11884" CreationDate="2019-04-18T22:57:22.557" Score="1" Body="&lt;p&gt;Yes, it is recommended to start with pre-trained model if you don't have high-end hardware. You can use a pre-trained model for fine-tuning (their trained weight as your initial weight) or use it as feature extractor (you remove few last layers, and then train it).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why we need a pre-trained network?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Because training a good deep model takes a lot of time and needs a lot of hardware. Some good models, like DenseNet, ResNet, even VGG16 need days of training. You can read from &lt;a href=&quot;https://arxiv.org/pdf/1409.1556.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG original paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;On a system equipped with four NVIDIA Titan Black GPUs, training a single net took 2–3 weeks depending on the architecture.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sometimes our case is similar to the dataset of pre-trained models out there. For example, if we need to classify some images of flower type, even our case has different class, we can use some first Convolutional layers of pre-trained model and use it as a feature extractor.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You can read more about transfer learning from &lt;a href=&quot;https://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; or &lt;a href=&quot;http://cs231n.github.io/transfer-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-18T22:57:22.557" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11888" PostTypeId="1" AcceptedAnswerId="13162" CreationDate="2019-04-19T00:22:01.707" Score="2" ViewCount="223" Body="&lt;p&gt;At the appendix A of paper &quot;&lt;a href=&quot;https://arxiv.org/abs/1810.01257&quot; rel=&quot;nofollow noreferrer&quot;&gt;near-optimal representation learning for hierarchical reinforcement learning&lt;/a&gt;&quot;, the authors express the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-discounted state visitation frequency &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; of policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d=(1-\gamma)A_\pi(I-\gamma^cP_\pi^c)^{-1}\mu\tag 1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've simplifed the notation for easy reading, hoping it does not introduce any error. In the above definition, &lt;span class=&quot;math-container&quot;&gt;$P_\pi^c$&lt;/span&gt; the &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-step transition matrix under the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, i.e., &lt;span class=&quot;math-container&quot;&gt;$P_{\pi}^c=P_\pi(s_{c}|s_0)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; a Dirac &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; distribution centered at start state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A_\pi=I+\sum_{k=1}^{c-1}\gamma^kP_\pi^k\tag 2&#xA;$$&lt;/span&gt;&#xA;They further give the every-&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-steps &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-discounted state frequency of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d^c_\pi=(1-\gamma^c)(I-\gamma^cP_\pi^c)^{-1}\mu\tag 3&#xA;$$&lt;/span&gt;&#xA;To my best knowledge, &lt;span class=&quot;math-container&quot;&gt;$A_\pi$&lt;/span&gt; seems to be the unnormalized &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-discounted state frequency, but I cannot really make sense of the rest.&#xA;I'm hoping that someone can shed some light on these definitions.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Thank @Philip Raeisghasem for pointing out the paper &lt;a href=&quot;https://arxiv.org/abs/1705.10528&quot; rel=&quot;nofollow noreferrer&quot;&gt;CPO&lt;/a&gt;. Here's what I've gotten from that.&#xA;Applying the sum of the geometric series to Eq.&lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt;, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A={(I-\gamma^cP_\pi^c)(I-\gamma P_\pi)^{-1}}\tag4&#xA;$$&lt;/span&gt;&#xA;Plugging Eq.&lt;span class=&quot;math-container&quot;&gt;$(4)$&lt;/span&gt; back into Eq.&lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt;, we get the same result as Eq.&lt;span class=&quot;math-container&quot;&gt;$(18)$&lt;/span&gt; in the CPO paper:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d=(1-\gamma)(I-\gamma P_\pi)^{-1}\mu\tag 5&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$(1-\gamma)$&lt;/span&gt; normalizes all weights introduced by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; so that they are summed to one. However, I'm still confused. Here are the questions I have&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Eq.&lt;span class=&quot;math-container&quot;&gt;$(5)$&lt;/span&gt; indicates Eq.&lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; is the state frequency in the infinite horizon. But I do not understand why we have it in the hierarchical policy. To my best knowledge, policies here are low-level, which means they are only valid in a short horizon (&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; steps, for example). Computing state frequency in the infinite horizon here seems confusing.&lt;/li&gt;&#xA;&lt;li&gt;What should I make of &lt;span class=&quot;math-container&quot;&gt;$d_\pi^c$&lt;/span&gt; defined in Eq.&lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt;, originally from Eqs.&lt;span class=&quot;math-container&quot;&gt;$(26)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(27)$&lt;/span&gt; in the paper? The authors define them as every-&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-steps &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-discounted state frequencies of policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. But I do not see why it is the case. To me, they are more like the consequence of Eq.&lt;span class=&quot;math-container&quot;&gt;$(30)$&lt;/span&gt; in the paper.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sorry if anyone feels that this update makes this question too broad. This is kept since I'm not so sure whether I can get a satisfactory answer without these questions.&#xA;Any partial answer will be sincerely appreciated. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="8689" LastEditDate="2019-07-02T03:06:51.657" LastActivityDate="2019-07-11T01:59:00.557" Title="Intuition behind $\gamma$-discounted state frequency" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11889" PostTypeId="1" CreationDate="2019-04-19T04:37:06.437" Score="2" ViewCount="119" Body="&lt;p&gt;I have looked at the documentation for the NEAT Python API found &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/xor_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where it's written&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The error for each genome is &lt;span class=&quot;math-container&quot;&gt;$1-\sum_i(e_i-a_i)^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have not yet learned calculus, so I can't understand this formula. So, can someone please explain what the calculation means?&lt;/p&gt;&#xA;" OwnerUserId="24036" LastEditorUserId="2444" LastEditDate="2021-01-14T16:50:51.340" LastActivityDate="2021-01-14T17:43:58.430" Title="What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?" Tags="&lt;math&gt;&lt;neat&gt;&lt;neuroevolution&gt;&lt;notation&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11890" PostTypeId="2" ParentId="11885" CreationDate="2019-04-19T08:55:33.980" Score="3" Body="&lt;p&gt;Your implementation of Monte Carlo Exploring Starts algorithm appears to be working as designed. This is a problem that can occur with some deterministic policies in the gridworld environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible for your policy improvement step to generate such a policy, and there is no recovery from this built into the algorithm. First visit and every visit variants will converge differently to the true action values, however neither offers an improvement here. It is most likely that your loops are occurring via state/action pairs that did not occur in the first episode, so the value estimates are default 0, which then looks like the best choice when creating the deterministic policy*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Sutton &amp;amp; Barto to demonstrate Monte Carlo ES, the authors choose an environment where such loops are impossible (a simplified Blackjack game). They then quickly move on to removing the need for exploring starts by using &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policies. So this issue is not covered in detail, although there are assertions in a couple of places that it must be possible to complete episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To resolve this whilst still using Monte Carlo ES, you will need to alter the environment so that such loops are not possible. The simplest change is to terminate the episode if it gets too long. This is hacky, because done simply on the gridworld it violates the Markov property (because now the time step should technically be part of the state if you want to predict value). However, it will get you out of the immediate problem, and provided you set the termination point high enough - e.g. 100 steps for your small gridworld - the agent should still discover the optimal policy and associated action values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;timeout&quot; patch is also used by many OpenAI Gym environments, because although most other algorithms can find their way out of infinite loops like this, they can still suffer slow learning from over-long episodes as a result.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* This leads to a possible fix of initialising your Q values pessimistically - e.g. with -50 starting value. That should help the first deterministic policy join up to the terminal state - although if you make it &lt;em&gt;strictly&lt;/em&gt; deterministic and resolve value ties deterministically too, then even this may not be enough. You might want to give that a try to verify what I am saying here, although I would recommend the timeout hack instead, as that is a more general solution. Pessimistic start values are bad for exploration when using other algorithms.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-04-19T09:13:26.187" LastActivityDate="2019-04-19T09:13:26.187" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11892" PostTypeId="5" CreationDate="2019-04-19T15:13:49.930" Score="0" Body="&lt;p&gt;For more details, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/AIXI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AIXI&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-19T19:21:24.850" LastActivityDate="2019-04-19T19:21:24.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11893" PostTypeId="4" CreationDate="2019-04-19T15:13:49.930" Score="0" Body="For questions related to AIXI, which is a theoretical mathematical formalism for artificial general intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-19T19:21:34.690" LastActivityDate="2019-04-19T19:21:34.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11894" PostTypeId="1" CreationDate="2019-04-19T16:47:09.540" Score="1" ViewCount="45" Body="&lt;p&gt;I am trying classify CIFAR10. The CNN that I generated over fits when the accuracy reaches ~77%. The code and the plot is given below. I tried DropOut, Batch Normalization and L2 Regularization. But the accuracy does not go beyond ~77.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I identify the areas to be corrected to reduce over fitting? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;convolutional_model = Sequential()&#xA;&#xA;# 32&#xA;convolutional_model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3), kernel_regularizer=regularizers.l2(.0002)))&#xA;convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))&#xA;convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))&#xA;&#xA;# 64&#xA;convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))&#xA;convolutional_model.add(Conv2D(128, (3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(.0002)))&#xA;convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))&#xA;&#xA;convolutional_model.add(Flatten())&#xA;convolutional_model.add(Dropout(0.5))&#xA;&#xA;convolutional_model.add(Dense(128, activation='relu'))&#xA;convolutional_model.add(Dense(10, activation='softmax'))&#xA;&#xA;print(convolutional_model.summary())&#xA;convolutional_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])&#xA;es = EarlyStopping(monitor='val_loss', mode='min', verbose=2, patience=8)&#xA;history = convolutional_model.fit(X_Train_Part, Y_Train_Part, epochs=200, verbose=2,validation_data=(X_Train_Validate, Y_Train_Validate), callbacks=[es])&#xA;&#xA;scores = convolutional_model.evaluate(X_Test, Y_Test, verbose=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TAOQ5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TAOQ5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23734" LastEditorUserId="23734" LastEditDate="2019-04-20T01:38:04.987" LastActivityDate="2019-04-20T10:01:54.327" Title="How to identify the areas to reduce over fitting?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11895" PostTypeId="1" CreationDate="2019-04-19T17:05:52.470" Score="3" ViewCount="96" Body="&lt;p&gt;Th Bayes' rule specifies how an agent should update its belief in a proposition based on a new piece of evidence. Suppose an agent has a current belief in proposition &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; based on evidence &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; already observed, given by &lt;span class=&quot;math-container&quot;&gt;$P(h \mid k)$&lt;/span&gt;, and subsequently observes &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In artificial intelligence, where can we use the Bayes' rule? I am unable to understand this concept.&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="2444" LastEditDate="2019-04-19T18:21:14.213" LastActivityDate="2019-04-20T16:42:08.487" Title="Where can we use the Bayes' theorem in artificial intelligence?" Tags="&lt;applications&gt;&lt;bayes-theorem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11897" PostTypeId="5" CreationDate="2019-04-19T18:42:00.763" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-19T18:42:00.763" LastActivityDate="2019-04-19T18:42:00.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11898" PostTypeId="4" CreationDate="2019-04-19T18:42:00.763" Score="0" Body="For questions related to Bayes' theorem (or rule) used in the AI context." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-19T19:21:28.647" LastActivityDate="2019-04-19T19:21:28.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11899" PostTypeId="2" ParentId="11889" CreationDate="2019-04-19T23:33:17.980" Score="0" Body="&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\sum$&lt;/span&gt; means that they take a sum of the squared difference of each pair of expected/predicted values (&lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt;) and actual values (&lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;p&gt;That gives them an error metric of how far off they are from their desired result. The goal is generally to optimize the algorithms against such an error function, in this case, to get it as close to one as possible.&lt;/p&gt;&#xA;" OwnerUserId="24055" LastEditorUserId="2444" LastEditDate="2021-01-14T16:49:34.477" LastActivityDate="2021-01-14T16:49:34.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11900" PostTypeId="1" AcceptedAnswerId="11947" CreationDate="2019-04-20T01:13:46.130" Score="2" ViewCount="379" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT paper&lt;/a&gt;, section 4.2 covers the SQuAD training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, there are two extra parameters trained, they are two vectors with the same dimension as the hidden size, so the same dimensions as the contextualized embeddings in BERT. They are S (for start) and E (for End). For each, a softmax is taken with S and each of the final contextualized embeddings to get a score for the correct Start position. And the same thing is done for E and the correct end position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get up to this part. But I am having trouble figuring out how the did the labeling and final loss calculations, which is described in this paragraph&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;and the maximum scoring span is used as the prediction. The training objective is the loglikelihood of the correct start and end positions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What do they mean by &quot;maximum scoring span is used as the prediction&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, how does that play into &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The training objective is the loglikelihood of the correct start and end positions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From this source: &lt;a href=&quot;https://ljvmiranda921.github.io/notebook/2017/08/13/softmax-and-the-negative-log-likelihood/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ljvmiranda921.github.io/notebook/2017/08/13/softmax-and-the-negative-log-likelihood/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says the log-likelihood is only applied to the correct classes. So, we are only calculating the softmax for the correct positions only, not any of the incorrect positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this interpretation is correct, then the loss will be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Loss = -Log( Softmax(S*T(predictedStart) / Sum(S*Ti) ) -Log( Softmax(E*T(predictedEnd) / Sum(S*Ti) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18358" LastEditorUserId="2444" LastEditDate="2019-11-01T02:48:35.613" LastActivityDate="2019-11-01T02:48:35.613" Title="Understanding how the loss was calculated for the SQuAD task in BERT paper" Tags="&lt;natural-language-processing&gt;&lt;loss-functions&gt;&lt;papers&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11901" PostTypeId="2" ParentId="10508" CreationDate="2019-04-20T06:07:01.080" Score="0" Body="&lt;p&gt;automatic transcription is a system to change phonemes to graphemes automatically, it's more like syllabel recognition, used for build speech recognition in another language base on stable and existing version of speech recognition.&#xA;see this picture to more understanding: &lt;a href=&quot;https://static1.squarespace.com/static/5382de75e4b092b699c496fd/t/54d860a2e4b07462f1a11816/1423466661326/phonemes-graphemes-letters.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this picture&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one think that you must understood, sound is different with meaning, computer need tools to understood the sound, it's speech or music or another form of sound. all speech sound already mapped in International Phonetic Alphabet (IPA).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and with little computation to combine the graphemes (call dictionary), resulting understanding speech of specific language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;example: you already know the sound of &quot;spoon&quot; or 'key', the grapheme is split by s and 'p-o-on'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in other language like Indonesia's language, another developers use this grapheme identification system (transcription system) to build a speech recognition, for words 'meskipun'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in grapheme, 'meskipun' words is: 'mehs-key-poon'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with a simple computation, we can make computer understood the 'meskipun' words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;only said if &quot;mesh + key + poon&quot; show, the words is 'meskipun'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;meskipun (=althought, english)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The big problem in building speech recognition with using automated transcription system, there is no 100% similar for IPA map in every language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the developers should use several 'transfer learning' database of language to make their speech recognition have more high accuracy. except, he decided to build it from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;automated speech recognition is an end to end speech recognition that set to understood a specific speech language, which in the middle of the system contain automatic transcription system.&lt;/p&gt;&#xA;" OwnerUserId="24058" LastEditorUserId="24058" LastEditDate="2019-04-20T06:20:58.370" LastActivityDate="2019-04-20T06:20:58.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11902" PostTypeId="2" ParentId="11894" CreationDate="2019-04-20T07:45:52.087" Score="1" Body="&lt;p&gt;It is a difficult task to identify the areas to be corrected to improve accuracy unless that area is looking into your face. By that I mean unless the regularizing parameter has unusually large or small values it is difficult to pin down and identify which regularizer to tweak. You need to do a grid search or random search over the hyper-parameter space to come up with an optimal combination of hyper parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There could be a lot of things that you could consider to reduce over fitting. Some of them are &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; regularizers, Dropout, depth of the network, number of neurons in the layers, the optimizer,  etc. &lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-04-20T10:01:54.327" LastActivityDate="2019-04-20T10:01:54.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11903" PostTypeId="1" CreationDate="2019-04-20T10:19:55.007" Score="1" ViewCount="53" Body="&lt;p&gt;I am trying to develop a time series model using &lt;strong&gt;autoregression&lt;/strong&gt;. The data set is like as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INDEX MAXIMA&#xA;  0   0.743&#xA;  1   0.837&#xA;  2   0.838&#xA;  4   0.896&#xA;  5   1.014&#xA;  6   1.003&#xA;  7   1.01&#xA;  8   1.101&#xA;  9   1.097&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Maxima point is given is the largest points on each curve. Basically, I have to perform &lt;em&gt;multi-step forecasting&lt;/em&gt; (at least 9 steps ahead). I've done it using the &lt;code&gt;recursive approach&lt;/code&gt;. but the accuracy of the prediction getting worse as it reaches the end.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Predicted Result&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KfI06.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KfI06.png&quot; alt=&quot;Prediction graph click here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;PYTHON CODE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using the AR model from &lt;a href=&quot;https://www.statsmodels.org/devel/generated/statsmodels.tsa.ar_model.AR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;stats model&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# fit model for MAX VALUE&#xA;  model = AR(data)&#xA;  model_fit = model.fit()&#xA;  yhat_max = model_fit.predict(len(data),len(data]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For obtaining an accurate prediction, What changes should be done in the approach? or Do I have to change the model?&lt;/p&gt;&#xA;&lt;p&gt;Any kind of help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24006" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-20T11:09:43.587" Title="Auto-regression - Reduce error in prediction" Tags="&lt;python&gt;&lt;linear-regression&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11904" PostTypeId="2" ParentId="11895" CreationDate="2019-04-20T10:39:59.050" Score="2" Body="&lt;p&gt;A toy example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose an agent has information about the reliability of fire alarms. It may know how likely it is that an alarm will work if there is a fire i.e. &lt;span class=&quot;math-container&quot;&gt;$P(alarm | fire)$&lt;/span&gt;. But, if the problem is tweaked a bit and if the agent must know the probability that there is a fire, given that there is an alarm, it can use Bayes' rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(fire | alarm) = [P(alarm | fire) ×P(fire )] / P(alarm )$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(alarm | fire)$&lt;/span&gt; is the probability that the alarm worked, assuming that there was a fire. It is a measure of the alarm's reliability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expression &lt;span class=&quot;math-container&quot;&gt;$P(fire)$&lt;/span&gt; is the probability of a fire given no other information. It is a measure of how fire-prone the building is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(alarm)$&lt;/span&gt; is the probability of the alarm sounding, given no other information.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-04-20T16:42:08.487" LastActivityDate="2019-04-20T16:42:08.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11905" PostTypeId="2" ParentId="11903" CreationDate="2019-04-20T11:00:17.437" Score="1" Body="&lt;p&gt;The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the past of the process contains no information on the development of the process in the &lt;em&gt;distant&lt;/em&gt; future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, this might be the reason that you are getting poor forecasts. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What changes should be done in the approach? or Do I have to change the model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You might want to move to ARIMA models. See how they perform. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have some other time series that might act as explanatory variables then you might want to augment the dataset and use the ARIMAX model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third option would be to try out RNN if you have lots of data. You can also try hybrid models like ARIMA and RNN together. &lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-04-20T11:09:43.587" LastActivityDate="2019-04-20T11:09:43.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11907" PostTypeId="2" ParentId="11889" CreationDate="2019-04-20T12:00:25.603" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$1-\sum_i(e_i-a_i)^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum$&lt;/span&gt; - there just means sum. It is the greek letter for S.  You can rewrite the above formula as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$1 -[(e_1 - a_1)^2+(e_2-a_2)^2+(e_3-a_3)^2+\ldots ]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum$&lt;/span&gt; just helps us avoid writing dozens of &lt;span class=&quot;math-container&quot;&gt;$+$&lt;/span&gt; signs. Read &lt;a href=&quot;https://en.wikipedia.org/wiki/Summation&quot; rel=&quot;nofollow noreferrer&quot;&gt;more here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What they are doing here is taking the difference of expected value &lt;span class=&quot;math-container&quot;&gt;$e_1$&lt;/span&gt; and the actual value &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; for the 1st example, and so on. The difference can be positive (&lt;span class=&quot;math-container&quot;&gt;$e_1 &amp;gt; a_1$&lt;/span&gt;) or negative (&lt;span class=&quot;math-container&quot;&gt;$e_1 &amp;lt; a_1$&lt;/span&gt;), so usually we square the difference to make it positive number.&lt;/p&gt;&#xA;&lt;p&gt;The rest is there in the docs. Try putting in concrete imagined values for &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21690" LastEditorUserId="2444" LastEditDate="2021-01-14T17:43:58.430" LastActivityDate="2021-01-14T17:43:58.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11908" PostTypeId="2" ParentId="11812" CreationDate="2019-04-20T14:42:45.883" Score="0" Body="&lt;p&gt;&lt;strong&gt;Short Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generative networks in generative network arrangements do not learn about input images directly. Their input during training is feedback from the discriminative network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Theory in Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The seminal paper, &lt;em&gt;Generative Adversarial Networks&lt;/em&gt;, Goodfellow, Pouget-Abadie, Mirza, Xu, Warde-Farley, Ozair, Courville, and Bengio, June 2014, states, &quot;We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models ...&quot; The two models are defined as MLPs (multilayer perceptrons) in the paper.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generative model, G&lt;/li&gt;&#xA;&lt;li&gt;Discriminative model, D&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These two models are interconnected such that they form a negative feedback loop.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;G is trained to capture the feature relational distribution of a set of examples and generate new examples based on that relational distribution well enough to fool D.&lt;/li&gt;&#xA;&lt;li&gt;D is trained to differentiate G's mocks from the set of externally procured examples.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Applying Concepts&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If G were to receive input images, their presence would merely frustrate network training, in that the goal of the training would likely be inadequately defined. The objective of the convergence of G, stated above, is not the learning of how to process the images to produce some other form of output. Its objective in the generative approach is to learn how to generate well, an entirely incompatible objective with either image evaluation or image processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Information&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, one image is not nearly enough. There must be a sufficiently large set of example images for training to converge at all and then many more to expect the convergence to be both accurate and reliable. The PAC (probably approximately correct) learning analysis framework may be helpful to determine how many examples are needed for a specific case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Essential Discriminator&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discriminator is essential to the generative approach because the feedback loop referenced above is essential to the convergence mechanism. The bidirectional interdependence between G and D is what allows a balanced approach toward accuracy in the feature relational distribution. That accuracy facilitates the human perception that the generated images fit adequately within a cognitive class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Response to Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The attempt to use a generative approach, &quot;To paint in gaps in an image,&quot; is reasonable. In such a case, using Goodfellow's nomenclature, G would be generating the missing pixels and D would be trying to discriminate between G's gap filling and the pixels that were in the scenes in the regions of gaps prior to their introduction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two additional requirements in the scenario of filling in pixels.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;G must be strongly incentivized against allowing a large gradient between the generated pixels and the adjacent non-gap pixels, unless that gradient is appropriate to the scene, as in the case of an object edge, or a change in reflectivity, such as a surface abrasion or the edge of a spray painted shape.&lt;/li&gt;&#xA;&lt;li&gt;D must train using the entire image, which means the examples should be images without gaps, the gaps must be introduced in a way that matches the expected distribution of features of gaps that may be encountered later, and the result of G must be superimposed over the full image to produce the input arising from G and discriminated from the original by D.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is recommended to begin with a standard GAN design, create a test for it (in TDD fashion), implement it, experiment with it, and otherwise become familiar with it and the mathematics involved. Most important to understand is how the balance between G's convergence and D's convergence is obtained in the loss (a.k.a. error or disparity) functions for each, and what concepts of feedback are employed using those functions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does your point about input images frustrating network training apply to this kind of problem, or just to GANNs that generate from scratch?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It applies to both.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would I have to have the generator compare the original image with the generated image and pick which one it thinks is better in order to deal with the &quot;adequately defined&quot; issue? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;D compares, not G. That is the delegated arrangement. It is not that other arrangements cannot work. They may. But Goodfellow and the others understood what worked in artificial networks long before they discovered a new approach, and they likely worked out the math of that approach and diagrammed it, perhaps on a white board, long before they typed a single line of code.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-04-21T13:19:57.953" LastActivityDate="2019-04-21T13:19:57.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11909" PostTypeId="2" ParentId="11883" CreationDate="2019-04-20T15:37:14.843" Score="0" Body="&lt;p&gt;Although it was not crystal clear, we'll assume that by, &quot;Multiple cards of the same class in one picture,&quot; is meant that multiple cards of identical suit and rank will be grouped together in the same example image but each card in the image will be selected from a unique deck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That arrangement would only be fruitful if the objective of training was to classify, flag, or otherwise analyze the same kind of groupings after training. Otherwise, it would likely be most productive and efficient to first devise a way to divide up the images by deck so that the focus of learning is detection of the three dimensions or perhaps just the first two, depending on the intended use of the trained network.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rank&lt;/li&gt;&#xA;&lt;li&gt;Suit&lt;/li&gt;&#xA;&lt;li&gt;Style&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-04-20T15:37:14.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11912" PostTypeId="1" CreationDate="2019-04-20T20:36:44.257" Score="1" ViewCount="209" Body="&lt;p&gt;Say i want to train a neural network with 10 classes as outputs and use &lt;code&gt;categorical_cross_entropy&lt;/code&gt; as a loss function in keras. This will try&#xA;to fit the training data as best as possible irregardless of the outcome (i.e. value). If I want to take value into account, I have to use something like a policy gradient RL algorithm. How do I formulate the loss of policy gradient algorithm in this case ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard categorical cross entropy loss function is as follows where y_ = true value, and y = predicted value:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   loss = -mean( y_ * log(y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am thinking to just multiply the true value by the reward and still use the &lt;code&gt;categorical cross entropy&lt;/code&gt; of keras i.e.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   y_ = y_ * reward&#xA;   loss = -mean( y_* log(y) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is my interpretation correct ?&lt;/p&gt;&#xA;" OwnerUserId="20456" LastEditorUserId="24073" LastEditDate="2019-04-21T09:19:18.293" LastActivityDate="2019-04-21T09:19:18.293" Title="Policy gradient loss for neural network training" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11914" PostTypeId="1" AcceptedAnswerId="11920" CreationDate="2019-04-21T08:20:59.010" Score="4" ViewCount="48" Body="&lt;p&gt;Let's say you have an input which can take one of 10 different unique values. How would you encode it?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Have input length 10 and one-hot encode it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have 1 input but normalise the value between the input range.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Would the end result be the same?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="2444" LastEditDate="2019-04-21T13:02:49.870" LastActivityDate="2020-04-23T15:27:36.600" Title="How should I encode a categorical input?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11916" PostTypeId="5" CreationDate="2019-04-21T12:53:32.380" Score="0" Body="&lt;p&gt;For more info, have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Action_model_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Action_model_learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-22T19:57:19.567" LastActivityDate="2019-04-22T19:57:19.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11917" PostTypeId="4" CreationDate="2019-04-21T12:53:32.380" Score="0" Body="For questions related to the &quot;action model learning&quot;, which is an area of machine learning concerned with creation and modification of software agent's knowledge about effects and preconditions of the actions that can be executed within its environment." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-22T19:57:27.927" LastActivityDate="2019-04-22T19:57:27.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11918" PostTypeId="5" CreationDate="2019-04-21T12:54:02.987" Score="0" Body="&lt;p&gt;Have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Control_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Control_theory&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-22T19:57:11.647" LastActivityDate="2019-04-22T19:57:11.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11919" PostTypeId="4" CreationDate="2019-04-21T12:54:02.987" Score="0" Body="For questions related to control theory and its relation to reinforcement learning and other artificial intelligence sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-27T14:47:42.813" LastActivityDate="2019-07-27T14:47:42.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11920" PostTypeId="2" ParentId="11914" CreationDate="2019-04-21T13:08:38.553" Score="3" Body="&lt;p&gt;As with many &lt;em&gt;general&lt;/em&gt; questions about how to represent features, or best learn from them, the answer is &quot;it depends&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If there is a natural sequence to the separate items, and that sequence is informative somehow for the prediction, then the classes may work best as a single feature which takes discrete values, scaled to the network. A good example might be predicting house prices where one of the features is property tax bands (in locations which have these such as UK Council Tax).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the classes have no natural sequence to them relating to the problem, then one-hot-encoding is usually a better interpretation. An example of this might be predicting the price of a car based on the manufacturer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In both cases, a neural network with enough layers and connections - and enough data - could resolve a more difficult representation, and make little effective difference between the representations in practice. However, if you know something useful about the feature, it is normal to choose the most &quot;natural&quot; representation based on that knowledge, and you will often see a small improvement by making the correct choice. In the first case, the neural network can benefit from requiring less parameters, and learn more efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not sure, then choosing one-hot versus a single scaled input should be an experiment which you perform as part of your hyper parameter tuning, along with any other feature engineering that you are not sure of.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-21T13:08:38.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11921" PostTypeId="1" CreationDate="2019-04-21T14:24:27.937" Score="2" ViewCount="16" Body="&lt;p&gt;I am working on a CNN model for image classification. Currently a single AWS instance (p2xlarge) used for model training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have one more AWS instance and distribute the load across these 2 AWS instances (p2xlarge). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I can create cluster between 2 AWS instances so that I can train the model in parallel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly direct me to the right stackexchange site if this is not the right one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you,&#xA;KK&lt;/p&gt;&#xA;" OwnerUserId="17831" LastEditorUserId="17831" LastEditDate="2019-04-23T06:11:11.310" LastActivityDate="2019-04-23T06:11:11.310" Title="Cluster 2 AWS instances for deep learning model training" Tags="&lt;deep-learning&gt;&lt;gpu&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11924" PostTypeId="1" AcceptedAnswerId="11996" CreationDate="2019-04-21T18:02:13.173" Score="0" ViewCount="60" Body="&lt;p&gt;I understand that gamma is an important factor in determining the rewards for a deep Q agent, however during testing of my network I am noticing that the agent is outputting more actions to &quot;do nothing&quot; as it learns more about it's given data set (financial stock data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried tweaking the gamma at different levels ranging from 0 - 1 and everywhere in between, however as the agent continues to learn, the times between actions is getting longer, and longer. This behaviour is undesirable, and it is preferable that the agent be making more often, short-term actions even if they result in reduced reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any tips on how to achieve this? Would a minus gamma have adverse effects on the network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR:&#xA;Time between actions becoming increasingly long over time, would prefer an agent that makes many actions over long-term ones.&lt;/p&gt;&#xA;" OwnerUserId="20893" LastEditorUserId="20893" LastEditDate="2019-04-24T15:07:14.617" LastActivityDate="2019-04-24T15:07:14.617" Title="Encourage Deep Q to seek short-term reward" Tags="&lt;deep-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11926" PostTypeId="1" CreationDate="2019-04-21T18:24:21.503" Score="1" ViewCount="16" Body="&lt;p&gt;I need to train an LSTM on some light curves, in order to find a signal (there are 2 classes signal and background). However the signal (data points corresponding to signal) is around 100 times less frequent than the background so I have a huge class imbalance and in the end all points are labelled as background. I tried to use focal loss, but it doesn't help. Is there a way to make it work?&lt;/p&gt;&#xA;" OwnerUserId="23871" LastActivityDate="2019-04-21T18:24:21.503" Title="Training LSTM with class imbalance" Tags="&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11927" PostTypeId="1" AcceptedAnswerId="11932" CreationDate="2019-04-21T18:32:46.213" Score="4" ViewCount="193" Body="&lt;p&gt;Nowadays, robots or artificial agents often only perform the specific task they have been programmed to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will we be able to build an artificial intelligence that feels empathy, that understands the emotions and feelings of humans, and, based on that, act accordingly?&lt;/p&gt;&#xA;" OwnerUserId="23538" LastEditorUserId="4302" LastEditDate="2019-05-18T22:07:38.403" LastActivityDate="2019-09-18T21:31:55.023" Title="Will we be able to build an artificial intelligence that feels empathy?" Tags="&lt;ai-design&gt;&lt;philosophy&gt;&lt;emotional-intelligence&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11928" PostTypeId="2" ParentId="52" CreationDate="2019-04-21T19:15:49.277" Score="0" Body="&lt;p&gt;Yes, it is possible. The field of &lt;em&gt;deep reinforcement learning&lt;/em&gt; is all about using deep neural networks (that is, neural networks with at least one hidden layer) to &lt;em&gt;approximate&lt;/em&gt; value functions (such as the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function) or policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at the paper &lt;a href=&quot;https://arxiv.org/abs/1708.05866&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Brief Survey of Deep Reinforcement Learning&lt;/a&gt; that gives a brief survey of the field.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-04-21T19:15:49.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11929" PostTypeId="1" AcceptedAnswerId="11931" CreationDate="2019-04-21T19:23:33.580" Score="6" ViewCount="562" Body="&lt;p&gt;Reading Sutton and Barto, I see the following in describing policy gradients:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LnCdQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LnCdQ.jpg&quot; alt=&quot;policy grad &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, but conceptually I'm not sure I understand how the gradient is computed, since we need some loss function to compute the gradient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen a good PyTorch article, but I still don't understand the meaning of this gradient conceptually, and I don't know what I'm looking to implement. Any intuition that you could provide would be helpful. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="22916" LastEditDate="2019-04-22T03:51:44.270" LastActivityDate="2019-04-22T05:17:31.570" Title="How is the policy gradient calculated in REINFORCE?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;sutton-barto&gt;&lt;notation&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11931" PostTypeId="2" ParentId="11929" CreationDate="2019-04-22T03:23:46.850" Score="5" Body="&lt;p&gt;The first part of this answer is a little background that might bolster your intuition for what's going on. The second part is the more practical and direct answer to your question.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The gradient is just the generalization of the derivative to multivariable functions. The gradient of a function at a certain point is a vector that points in the direction of the steepest increase of that function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, we take a derivative/gradient of some loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; because we want to minimize that loss. So we update our parameters in the direction &lt;em&gt;opposite&lt;/em&gt; the direction of the gradient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta_{t+1} = \theta_{t} - \alpha\nabla_{\theta_t} \mathcal{L} \tag{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In policy gradient methods, we're not trying to minimize a loss function. Actually, we're trying to &lt;em&gt;maximize&lt;/em&gt; some measure &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; of the performance of our agent. So now we want to update parameters in the same direction as the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta_{t+1} = \theta_{t} + \alpha\nabla_{\theta_t} J \tag{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the episodic case, &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the value of the starting state. In the continuing case, &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the average reward. It just so happens that a nice theorem called the Policy Gradient Theorem applies to both cases. This theorem states that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;\nabla_{\theta_t}J(\theta_t) &amp;amp;\propto \sum_s \mu(s)\sum_a q_\pi (s,a) \nabla_{\theta_t} \pi (a|s,\theta_t)\\&#xA;&amp;amp;=\mathbb{E}_\mu \left[ \sum_a q_\pi (s,a) \nabla_{\theta_t} \pi (a|s,\theta_t)\right].&#xA;\end{align}\tag{3}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rest of the derivation is in your question, so let's skip to the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;\theta_{t+1} &amp;amp;= \theta_{t} + \alpha G_t \frac{\nabla_{\theta_t}\pi(A_t|S_t,\theta_t)}{\pi(A_t|S_t,\theta_t)}\\&#xA;&amp;amp;= \theta_{t} + \alpha G_t \nabla_{\theta_t} \ln \pi(A_t|S_t,\theta_t)&#xA;\end{align}\tag{4}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember, &lt;span class=&quot;math-container&quot;&gt;$(4)$&lt;/span&gt; says exactly the same thing as &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt;, so REINFORCE just updates parameters in the direction that will most increase &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;. (Because we sample from an expectation in the derivation, the parameter step in REINFORCE is actually an unbiased estimate of the maximizing step.) &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alright, but how do we actually get this gradient? Well, you use the chain rule of derivatives (backpropagation). Practically, though, both Tensorflow and PyTorch can take all the derivatives for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow, for example, has a minimize() method in its Optimizer class that takes a loss function as an input. Given a function of the parameters of the network, it will do the calculus for you to determine which way to update the parameters in order to minimize that function. But we don't want to minimize. We want to maximize! So just include a negative sign. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our case, the function we want to minimize is &#xA;&lt;span class=&quot;math-container&quot;&gt;$$-G_t\ln \pi(A_t|S_t,\theta_t).$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This corresponds to stochastic gradient descent (&lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; is not a function of &lt;span class=&quot;math-container&quot;&gt;$\theta_t$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to do minibatch gradient descent on each episode of experience in order to get a better (lower variance) estimate of &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_t} J$&lt;/span&gt;. If so, you would instead minimize &#xA;&lt;span class=&quot;math-container&quot;&gt;$$-\sum_t G_t\ln \pi(A_t|S_t,\theta_t),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\theta_t$&lt;/span&gt; would be constant for different values of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; within the same episode. Technically, minibatch gradient descent updates parameters in the average estimated maximizing direction, but the scaling factor &lt;span class=&quot;math-container&quot;&gt;$1/N$&lt;/span&gt; can be absorbed into the learning rate.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-22T05:17:31.570" LastActivityDate="2019-04-22T05:17:31.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11932" PostTypeId="2" ParentId="11927" CreationDate="2019-04-22T03:28:37.193" Score="5" Body="&lt;p&gt;Let us describe a very simple system that does something we could label as empathyc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A chatbot answers &quot;I am sorry to hear that. What happened?&quot; when we type &quot;I feel bad&quot;, and it replies &quot;I am glad to hear that. Fancy some music?&quot; when we type &quot;I feel good&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Somehow, it perceives a human emotion, and acts accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Planes fly but they do not fly as birds. Similarly, we can expect artificial empathy from an AI, not necessarily natural empathy as we feel it.&lt;/p&gt;&#xA;" OwnerUserId="24014" LastEditorUserId="24014" LastEditDate="2019-04-22T06:54:03.953" LastActivityDate="2019-04-22T06:54:03.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11933" PostTypeId="1" CreationDate="2019-04-22T07:05:48.920" Score="3" ViewCount="116" Body="&lt;p&gt;I have been reading &lt;a href=&quot;https://deepmind.com/blog/alphago-zero-learning-scratch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article on AlphaGo&lt;/a&gt; and one sentence confused me a little bit, because I'm not sure what it exactly means. The article says: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What exactly is the input to AlphaGo's neural network? What do they mean by &quot;just white and black stones as input&quot;? What kind of information is the neural network using? The position of the stones?&lt;/p&gt;&#xA;" OwnerUserId="24093" LastEditorUserId="2444" LastEditDate="2020-06-08T20:11:11.140" LastActivityDate="2020-06-08T20:11:11.140" Title="What is the input to AlphaGo's neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;architecture&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11934" PostTypeId="2" ParentId="11933" CreationDate="2019-04-22T08:13:41.297" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The input to the neural network is a &lt;span class=&quot;math-container&quot;&gt;$19 × 19 × 17$&lt;/span&gt; image stack&#xA;  comprising &lt;span class=&quot;math-container&quot;&gt;$17$&lt;/span&gt; binary feature planes. &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; feature planes &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt; consist of binary values indicating the&#xA;  presence of the current player’s stones (&lt;span class=&quot;math-container&quot;&gt;$X^i_t = 1$&lt;/span&gt; if intersection &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; contains a stone of the player’s&#xA;  colour at time-step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;; &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if the intersection is empty, contains an opponent stone, or if &lt;span class=&quot;math-container&quot;&gt;$t &amp;lt; 0$&lt;/span&gt;). A&#xA;  further &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; feature planes, &lt;span class=&quot;math-container&quot;&gt;$Y_t$&lt;/span&gt;&#xA;  , represent the corresponding features for the opponent’s stones. The&#xA;  final feature plane, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, represents the colour to play, and has a constant value of either &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; if black&#xA;  is to play or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if white is to play. These planes are concatenated together to give input features&#xA;  &lt;span class=&quot;math-container&quot;&gt;$s_t = [&#xA;X_t, Y_t, X_{t−1}, Y_{t−1}, ..., X_{t−7}, Y_{t−7}, C]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This and all the other architecture details can be found in the &quot;Neural Network Architecture&quot; section in &lt;a href=&quot;http://discovery.ucl.ac.uk/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-04-22T08:20:16.013" LastActivityDate="2019-04-22T08:20:16.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11936" PostTypeId="1" CreationDate="2019-04-22T09:00:45.757" Score="3" ViewCount="223" Body="&lt;p&gt;I am trying to use Deep-Q-Learning to learn an ANN which controls a 7-DOF robotic arm. The robotic arm must avoid an obstacle and reach a target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented a number of state-of-art techinques to try to improve the ANN performance. Such techniques are: PER, Double DQN, adaptive discount factor, sparse reward. I have also tried Dueling DQN but it performed poorly. I have also tried a number of ANN architectures and it looks like that 2 hidden layers with 128 neurons is the best one so far. My input layer is 12 neurons, the output 10 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as you can see from the image down here, at a certain point the DQN stops learning and gets stuck at around 80% of success rate. I don't understand why it gets stuck, because in my opinion we could reach an higher success rate, 90% at least, but I just can't get out of that &quot;local minimum&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, my question is: &lt;strong&gt;What are some techniques I can try to unstuck a DQN from something that looks like a local minimum?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LZ5jz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LZ5jz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;note:&lt;/strong&gt; the success rate in this picture is computed as the number of successes in the last 100 runs.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-04-22T09:34:43.903" LastActivityDate="2019-04-22T09:34:43.903" Title="DQN Agent not learning anymore - what can I do to fix this?" Tags="&lt;deep-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11937" PostTypeId="1" AcceptedAnswerId="12040" CreationDate="2019-04-22T09:09:17.073" Score="0" ViewCount="611" Body="&lt;p&gt;I wrote a convolutional neural network for the MNIST dataset with Numpy from scratch. I am currently trying to understand every part and calculation.&#xA;But one thing I noticed was the &quot;just positive&quot; derivative of the ReLU function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network structure is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(Input 28x28)&lt;/li&gt;&#xA;&lt;li&gt;Conv Layer (filter count = 6, filter size = 3x3,  stride = 1)&lt;/li&gt;&#xA;&lt;li&gt;Max Pool Layer (Size 2x2) with RELU&lt;/li&gt;&#xA;&lt;li&gt;Conv Layer (filter count = 6, filter size = 3x3,  stride = 1)&lt;/li&gt;&#xA;&lt;li&gt;Max Pool Layer (Size 2x2) with RELU&lt;/li&gt;&#xA;&lt;li&gt;Dense (128)&lt;/li&gt;&#xA;&lt;li&gt;Dense (10)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I noticed, when looking at the gradients, that the ReLU derivative is always (as it should be) positive. But is it right that the filter weights are always decreasing their weights? Or is there any way they can increase their weight?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever I look at any of the filter's values, they decreased after training. Is that correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, I am using stochastic gradient descent with a fixed learning rate for training.&lt;/p&gt;&#xA;" OwnerUserId="24096" LastEditorUserId="24096" LastEditDate="2019-04-23T01:35:17.927" LastActivityDate="2019-04-28T13:13:01.497" Title="How should the values of the filters of a CNN change?" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;relu&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11941" PostTypeId="1" CreationDate="2019-04-22T12:00:35.690" Score="2" ViewCount="158" Body="&lt;p&gt;I was trying to implement a DQN without experience reply memory, and the agent is not learning anything at all. I know from readings that experience reply is used for stabilizing gradients. But how important is experience reply in DQN and similar RL algorithms? If the model needs to learn from memory, why don't we use a recurrent network, which has inbuilt memory to it? What is the advantage of experience reply over a recurrent memory?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2019-04-22T12:05:29.007" LastActivityDate="2019-04-22T12:05:29.007" Title="Why experience reply memory in DQN instead of a RNN memory?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;long-short-term-memory&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11942" PostTypeId="1" AcceptedAnswerId="11946" CreationDate="2019-04-22T13:05:10.057" Score="2" ViewCount="128" Body="&lt;p&gt;I was wondering whether there is an AI system which could be used to resolve the class clashes problem which mostly happens in universities. In almost every university students face this problem, where two or more courses that many students want to take together get scheduled at the same time. Does anyone know about a system which resolves this issue or someone who works on this problem?&lt;/p&gt;&#xA;" OwnerUserId="24095" LastEditorUserId="16909" LastEditDate="2019-04-22T18:37:16.287" LastActivityDate="2019-04-22T18:37:16.287" Title="Is there any AI system for finding the best way to schedule university classes?" Tags="&lt;planning&gt;&lt;symbolic-ai&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11944" PostTypeId="1" CreationDate="2019-04-22T17:09:39.753" Score="1" ViewCount="80" Body="&lt;p&gt;The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can artificial intelligence create the next wonder material?&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="1671" LastEditDate="2019-05-22T21:13:36.260" LastActivityDate="2019-05-29T20:28:21.570" Title="Can artificial intelligence create the next wonder material?" Tags="&lt;materials-science&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11945" PostTypeId="1" AcceptedAnswerId="11952" CreationDate="2019-04-22T18:22:41.017" Score="1" ViewCount="110" Body="&lt;p&gt;There are several science fiction movies where the robots rebel against their creators: for example, the &lt;em&gt;Terminator&lt;/em&gt;'s series or &lt;em&gt;I Robot&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the future, is it possible that robots will rebel against their human creators (like in the mentioned movies)?&lt;/p&gt;&#xA;" OwnerUserId="24095" LastEditorUserId="1671" LastEditDate="2019-04-22T21:04:13.617" LastActivityDate="2020-06-27T22:06:26.483" Title="Will robots rebel against their human creators?" Tags="&lt;control-problem&gt;&lt;mythology-of-ai&gt;&lt;neo-luddism&gt;&lt;ai-takeover&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11946" PostTypeId="2" ParentId="11942" CreationDate="2019-04-22T18:35:10.577" Score="3" Body="&lt;p&gt;Welcome to AI.SE @Israr Ali.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem of scheduling a timetable is an example of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_satisfaction_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;constraint satisfaction problem&lt;/a&gt;, a topic long studied in AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many possible techniques to apply to this kind of problem. They can be organized into three broad categories:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Global search algorithms, like &lt;a href=&quot;https://en.wikipedia.org/wiki/Backtracking&quot; rel=&quot;nofollow noreferrer&quot;&gt;backtracking search&lt;/a&gt; can be used to try and find an assignment of times to classes that results in no conflicts at all. With proper heuristics, these can be quite fast in practice, even though their worst-case runtimes are poor. Russell &amp;amp; Norvig's &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI: A Modern Approac&lt;/a&gt;h has a very good overview of these approaches in Chapter 6. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Local search algorithms, like &lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_climbing&quot; rel=&quot;nofollow noreferrer&quot;&gt;hill climbing search&lt;/a&gt; start by assigning every class a randomly selected time, and then making small adjustments that monotonically decrease the number of conflicts (e.g. swapping the times of two classes). These are often faster than backtracking approaches and are especially so if you are willing to accept a &quot;pretty good&quot; schedule that might still have fewer conflicts than an optimal one. Chapter 4 of AI: A Modern Approach provides a good introduction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Planning approaches, particularly non-classical algorithms like &lt;a href=&quot;https://en.wikipedia.org/wiki/Graphplan&quot; rel=&quot;nofollow noreferrer&quot;&gt;GraphPlan&lt;/a&gt; can also be used for these domains. They make use of the structure of planning or scheduling problems in particular to re-frame the search problems addressed by Backtracking techniques. By using this domain-specific representation, they are able to achieve very high-quality solutions very quickly. Chapter 11 of AI: A Modern Approach covers Graph Plan in some detail, and could be a good introduction to these more specialized techniques.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-22T18:35:10.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11947" PostTypeId="2" ParentId="11900" CreationDate="2019-04-22T19:58:58.523" Score="1" Body="&lt;p&gt;These answers are based on my personal understanding of Bert from both the &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and &lt;a href=&quot;https://github.com/google-research/bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;official_implementation&lt;/a&gt;, hope it will help:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;What do they mean by &quot;maximum scoring span is used as the prediction&quot;?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you know in SQuAD the input sequence is divided to 2 parts: Question and Document (from which we extract the answer if possible). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the input length exceeds the &lt;code&gt;max_seq_length&lt;/code&gt; parameter, in this case the document is truncated to as many parts as needed and we end up having more than one input for the same question/document. Mention that the question is replicated in all the resulting inputs (see &lt;a href=&quot;https://github.com/google-research/bert/blob/master/run_squad.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;line_350&lt;/a&gt; for details). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in such cases, in order to determine the &lt;strong&gt;predicted span&lt;/strong&gt; among all the generated-inputs we use the &lt;strong&gt;maximum scoring&lt;/strong&gt; &lt;em&gt;i.e&lt;/em&gt; &lt;code&gt;(max_start + max_end) / 2 = ( max(Softmax(S*Ti)) + max(Softmax(E*Ti)) ) / 2&lt;/code&gt; of all the inputs related to the same question.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;The training objective is the loglikelihood of the correct start and end positions&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The loss is the average of the start_position loss &lt;code&gt;start_loss&lt;/code&gt; and the end_postition loss &lt;code&gt;end_loss&lt;/code&gt;. Each loss is computed in the same way: after applying the Softmax to the final output (i.e &lt;code&gt;S*Ti&lt;/code&gt; or &lt;code&gt;E*Ti&lt;/code&gt;) we use the real start/end postitions to compute the loss (see code below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://github.com/google-research/bert/blob/master/run_squad.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;run_squad.py&lt;/a&gt; in Bert_repo:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; def compute_loss(logits, positions):&#xA;        one_hot_positions = tf.one_hot(positions, depth=seq_length, dtype=tf.float32)&#xA;        log_probs = tf.nn.log_softmax(logits, axis=-1)&#xA;        loss = -tf.reduce_mean(tf.reduce_sum(one_hot_positions * log_probs, axis=-1))&#xA;        return loss&#xA;&#xA; start_positions = features[&quot;start_positions&quot;]&#xA; end_positions = features[&quot;end_positions&quot;]&#xA;&#xA; start_loss = compute_loss(start_logits, start_positions)&#xA; end_loss = compute_loss(end_logits, end_positions)&#xA;&#xA; total_loss = (start_loss + end_loss) / 2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="23350" LastEditDate="2019-04-22T21:33:07.277" LastActivityDate="2019-04-22T21:33:07.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11948" PostTypeId="2" ParentId="11944" CreationDate="2019-04-22T20:00:14.950" Score="1" Body="&lt;p&gt;Artificial intelligence has already gotten to the point where machines are as good as humans at many tasks, and even outperform humans at several more. The general goal of AI is to improve machines to be as good or better than humans at a wide variety of tasks. There's no theoretical reason why materials science is any different; we may well someday have a machine that can out-think all human materials scientists. There's not much reason to suppose that humans will always reign supreme at any particular task (especially objective ones like science), so yes, it's entirely possible that the next &quot;wonder material&quot; will be invented by a machine.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2019-04-22T20:00:14.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11949" PostTypeId="1" CreationDate="2019-04-22T20:45:37.747" Score="2" ViewCount="290" Body="&lt;p&gt;Suppose I have a standard image classification problem (i.e. CNN is shown a single image and predicts a single classification for it). If I were to use bounding boxes to surround the target image (i.e. convert this into an object detection problem), would this increase classification accuracy purely through the use of the bounding box? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious if the neural network can be &quot;assisted&quot; by us when we show it bounding boxes as opposed to just showing it the entire image and letting it figure it all out by itself.&lt;/p&gt;&#xA;" OwnerUserId="6328" LastEditorUserId="2444" LastEditDate="2021-01-28T23:49:48.163" LastActivityDate="2021-02-28T02:04:05.367" Title="Can bounding boxes further improve the performance of a CNN classifier?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;object-recognition&gt;&lt;bounding-box&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11952" PostTypeId="2" ParentId="11945" CreationDate="2019-04-23T00:51:28.480" Score="1" Body="&lt;p&gt;I don't like to be a killjoy, but this question seems premature (that's why it's hd the &quot;mythology of AI&quot; tag added to it). The kinds of emergent artificial general intelligence depicted in the movies you mention are in science fiction films because they are science fiction. &lt;a href=&quot;https://nickbostrom.com/papers/survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Most AI researchers do not think they are likely to appear anytime soon&lt;/a&gt;. The overwhelming majority of researchers think the most likely times for such a system to appear are &quot;More than 50 years [from now]&quot; or &quot;Never&quot;. In part, this is because AI researchers thought we were close to such systems for several decades, despite failing to create them. This suggests that making an artificial general intelligence is much harder than we might expect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite AGI being a long show, there's a lot of recent interest in the AI research community in the social impact of our technologies. The study of these systems is called &quot;ethical AI&quot;, and this is the path that the research community as a whole has &lt;a href=&quot;https://sigai.acm.org/aimatters/blog/2017/09/20/new-conference-aaaiacm-conference-on-ai-ethics-and-society/&quot; rel=&quot;nofollow noreferrer&quot;&gt;begun to embark on&lt;/a&gt;. A promising approach is to model the process by which humans decide to treat each other well, in the hopes of creating programs act according to that process. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-23T00:51:28.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11953" PostTypeId="1" CreationDate="2019-04-23T01:57:24.070" Score="11" ViewCount="409" Body="&lt;p&gt;It seems that deep neural networks and other neural network based models are dominating many current areas like computer vision, object classification, reinforcement learning, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there domains where SVMs (or other models) are still producing state-of-the-art results?&lt;/p&gt;&#xA;" OwnerUserId="22525" LastEditorUserId="2444" LastEditDate="2019-04-23T14:08:00.547" LastActivityDate="2019-09-28T14:35:26.720" Title="What are the domains where SVMs are still state-of-the-art?" Tags="&lt;machine-learning&gt;&lt;support-vector-machine&gt;&lt;state-of-the-art&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11954" PostTypeId="2" ParentId="11953" CreationDate="2019-04-23T02:18:33.203" Score="7" Body="&lt;p&gt;Deep Learning and Neural Networks are getting most of the focus because of recent advances in the field and most experts believe it to be the future of solving machine learning problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But make no mistake, classical models still produce exceptional results and in certain problems, they can produce better results than deep learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linear Regression is still by far the most used machine learning algorithm in the world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It’s difficult to identify a specific domain where classical models always perform better as the accuracy is very much determined on the shape and quality of the input data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So algorithm and model selection is always a trade-off. It’s a somewhat accurate statement to make that classical models still perform better with smaller data sets. However, a lot of research is going into improving deep learning model performance on less data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most classical models require less computational resources so if your goal is speed then its much better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, classical models are easier to implement and visualize which can be another indicator for performance, but it depends on your goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have unlimited resources, a massive observable data set that is properly labeled and you implement it correctly within the problem domain then deep learning is likely going to give you better results in most cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in my experience, the real-world conditions are never this perfect&lt;/p&gt;&#xA;" OwnerUserId="24107" LastEditorUserId="24107" LastEditDate="2019-04-23T02:44:20.910" LastActivityDate="2019-04-23T02:44:20.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11955" PostTypeId="2" ParentId="11949" CreationDate="2019-04-23T03:18:03.823" Score="0" Body="&lt;p&gt;Another way to ask the question is: Does sound get clearer when you remove the background noise? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The obvious answer is yes and in the case of image classification, the answer is also generally yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases reducing the noise (irrelevant pixels) will strengthen the signal (activations) the neural network is trying to find. &lt;/p&gt;&#xA;" OwnerUserId="24107" LastActivityDate="2019-04-23T03:18:03.823" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11956" PostTypeId="1" AcceptedAnswerId="11991" CreationDate="2019-04-23T04:12:28.513" Score="1" ViewCount="99" Body="&lt;p&gt;How do I choose the search algorithm for a particular task? Which criteria should I take into account?&lt;/p&gt;&#xA;" OwnerUserId="23299" LastEditorUserId="2444" LastEditDate="2019-04-24T20:08:11.357" LastActivityDate="2019-04-24T20:08:11.357" Title="How do I choose the search algorithm for a particular task?" Tags="&lt;algorithm&gt;&lt;search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11957" PostTypeId="1" CreationDate="2019-04-23T05:30:13.557" Score="1" ViewCount="29" Body="&lt;p&gt;I've been working on Hinton's matrix capsule networks for several months. I searched each corner of the internet. But I couldn't find anyone that can reproduce Hinton's &lt;a href=&quot;https://openreview.net/pdf?id=HJWLfGWRb&quot; rel=&quot;nofollow noreferrer&quot;&gt;matrix capsule network&lt;/a&gt;. Can anyone get the reported accuracy on SmallNORB and Cifar10 dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I know Hinton's another paper on capsule networks &lt;a href=&quot;https://papers.nips.cc/paper/6975-dynamic-routing-between-capsules.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing Between Capsules&lt;/a&gt; is reproducible. Please, do not confuse the two papers.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastEditorUserId="2444" LastEditDate="2020-06-09T11:38:06.090" LastActivityDate="2020-06-09T11:38:06.090" Title="Is anyone able to reproduce Hinton's matrix capsule networks?" Tags="&lt;deep-learning&gt;&lt;capsule-neural-network&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11959" PostTypeId="2" ParentId="4320" CreationDate="2019-04-23T05:52:13.563" Score="2" Body="&lt;p&gt;That is a very deep question. There was series of papers recently proving the convergence of gradient descent for overparameterized deep networks (for example, &lt;a href=&quot;https://arxiv.org/abs/1811.03804&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gradient Descent Finds Global Minima of Deep Neural Networks&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1811.03962&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Convergence Theory for Deep Learning via Over-Parameterization&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/abs/1811.08888&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks&lt;/a&gt;). All of the proofs assume that the initial weights are assigned randomly according to a Gaussian distribution. The main reasons this initial distribution is important for the proofs are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Random weights make the ReLU operators in each layer statistically compressive mapping (up to a linear transformation).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Random weights preserve separation of input for any input distribution - that is if input samples are distinguishable network propagation will not make them indistinguishable.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Those properties are very difficult to reproduce with deterministically generated initial weight matrices, and even if they are reproducible with deterministic matrices NULL-space (from which we can generate adversarial examples) would likely make the method less useful in practice. More importantly, preservation of those properties during gradient descent would likely make method impractical. But overall it's very difficult but not impossible, and may warrant some research in that direction. In analogous situation, there were some results for &lt;a href=&quot;https://arxiv.org/abs/1202.1234&quot; rel=&quot;nofollow noreferrer&quot;&gt;Restricted Isometry Property for deterministic matrices&lt;/a&gt; in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Compressed_sensing&quot; rel=&quot;nofollow noreferrer&quot;&gt;compressed sensing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="16909" LastEditDate="2021-01-22T00:58:37.820" LastActivityDate="2021-01-22T00:58:37.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11964" PostTypeId="1" AcceptedAnswerId="11967" CreationDate="2019-04-23T09:18:11.477" Score="1" ViewCount="98" Body="&lt;p&gt;Could we even use reinforcement learning with big datasets? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or in RL does the agent built its own dataset ?&lt;/p&gt;&#xA;" OwnerUserId="24003" LastEditorUserId="1847" LastEditDate="2019-04-23T10:16:52.177" LastActivityDate="2019-04-23T10:30:15.710" Title="Is there any example of using Q-learning with big data?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11966" PostTypeId="1" AcceptedAnswerId="11972" CreationDate="2019-04-23T10:11:10.767" Score="1" ViewCount="71" Body="&lt;p&gt;Could changing the order of convolution layers in a CNN improve accuracy or training time?&lt;/p&gt;&#xA;" OwnerUserId="24003" LastEditorUserId="2444" LastEditDate="2019-04-23T13:29:48.050" LastActivityDate="2019-04-23T15:18:40.130" Title="Does changing the order of the convolution layers in a CNN have any impact?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11967" PostTypeId="2" ParentId="11964" CreationDate="2019-04-23T10:30:15.710" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Or in RL does the agent built its own dataset?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Essentially this is the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL is a very general learning mechanism, based on trial-and-error. You &lt;em&gt;could&lt;/em&gt; create an environment where the agent's goal is to correctly predict classification or regression problems, where the agent is rewarded for getting close to the correct prediction. However, these problems are most often better addressed using supervised learning techniques. Using RL in such cases will most of the time make the learning slower, less efficient and less accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relationship between RL and supervised learning is more about how RL generates the target data for learning. In some cases, such as Deep Q Learning (DQN), you can see quite clearly from looking at the algorithm or code, that the RL agent contains a supervised learning component. The &quot;supervised learning&quot; in DQN learns a regression problem to predict the action values for the current agent's target policy. Whilst the outer RL logic that contains this supervised learning model, is built around testing and changing that behaviour in order to act optimally in the longer term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using RL to predict from a big data set would involve using the same kind of supervised learning model on the inside, with RL using that to guess at the correct result, and sometimes choosing to guess at some other random prediction instead. This guessing process is extra processing, plus it will add noise and variance to the error signal used to train the model. In a few cases, this might even be helpful, but in the majority of cases studied for regression and classification tasks, it will result in a poorer training process, less accurate model, or both.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-23T10:30:15.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11969" PostTypeId="5" CreationDate="2019-04-23T14:03:10.997" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-04-23T14:03:10.997" LastActivityDate="2019-04-23T14:03:10.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11970" PostTypeId="4" CreationDate="2019-04-23T14:03:10.997" Score="0" Body="For questions related to &quot;state of the art&quot; (SOTA) models in machine learning and, in general, AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-23T19:57:06.150" LastActivityDate="2019-04-23T19:57:06.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11971" PostTypeId="1" AcceptedAnswerId="11989" CreationDate="2019-04-23T15:07:09.970" Score="1" ViewCount="76" Body="&lt;p&gt;&lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=16&quot; rel=&quot;nofollow noreferrer&quot;&gt;Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1&lt;/a&gt; calculates the variance of the output of a hidden neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Raul Rojas says that &quot;for binary vectors we have &lt;span class=&quot;math-container&quot;&gt;$E[x_i^2] = \frac{1}{3}$&lt;/span&gt;&quot; where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is the input value transported through each edge to a node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't quite get how he reaches this result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time :)&lt;/p&gt;&#xA;" OwnerUserId="14892" LastActivityDate="2019-04-24T11:18:10.333" Title="Binary vector expected value" Tags="&lt;neural-networks&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11972" PostTypeId="2" ParentId="11966" CreationDate="2019-04-23T15:18:40.130" Score="0" Body="&lt;p&gt;Conventionally, CNN layers downsample over and over, which enables them to capture details at different levels of abstractions. Usually, it is observed that the initial layers do nothing more than detecting edges, or filtering color channels; the combinations of these edges are what we perceive as 'features'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you reverse the order, you essentially are changing sampling modes down the line.&#xA;CNNs detect by 'downsampling' the inputs and therefore 'extracting' features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may not work as expected!&lt;/p&gt;&#xA;" OwnerUserId="24122" LastActivityDate="2019-04-23T15:18:40.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11973" PostTypeId="2" ParentId="11953" CreationDate="2019-04-23T15:31:36.133" Score="14" Body="&lt;p&gt;State-of-the-art is a tough bar, because it's not clear how it should be measured. An alternative criteria, which is akin to state-of-the-art, is to ask &lt;em&gt;when&lt;/em&gt; you might prefer to try an SVM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVMs have several advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Through the kernel trick, the runtime of an SVM does not increase significantly if you want to learn patterns over many non-linear combinations of features, rather than the original feature set. In contrast, a more modern approach like a deep neural network will need to get deeper or wider to model the same patterns, which will increase its training time.&lt;/li&gt;&#xA;&lt;li&gt;SVMs have an inherent bias towards picking &quot;conservative&quot; hypotheses, that are less likely to overfit the data, because they try to find maximum margin hypotheses. In some sense, they &quot;bake-in&quot; Occam's razor.&lt;/li&gt;&#xA;&lt;li&gt;SVMs have only two hyperparameters (the choice of kernel and the regularization constant), so they are very easy to tune to specific problems. It is usually sufficient to tune them by performing a simple grid-search through the parameter space, which can be done automatically.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;SVMs also have some disadvantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;SVMs have a runtime that scales cubically in the number of datapoints you want to train on (i.e. &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt; runtime)&lt;sup&gt;1&lt;/sup&gt;. This does not compare well with, say, a typical training approach for a deep neural network which runs in &lt;span class=&quot;math-container&quot;&gt;$O(w*n*e)$&lt;/span&gt; time, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of data points, &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; is the number of training epochs, and &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the number of weights in the network. Generally &lt;span class=&quot;math-container&quot;&gt;$w, e &amp;lt;&amp;lt; n$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;To make use of the Kernel trick, SVMs cache a value for the kernelized &quot;distance&quot; between any two pairs of points. This means they need &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; memory. This is far, far, more trouble than the cubic runtime on most real-world sets. More than a few thousand datapoints will leave most modern servers &lt;a href=&quot;https://en.wikipedia.org/wiki/Thrashing_(computer_science)&quot; rel=&quot;noreferrer&quot;&gt;thrashing&lt;/a&gt;, which increases effective runtime by several orders of magnitude. Together with point 1, this means SVMs will tend to become unworkably slow for sets beyond maybe 5,000-10,000 datapoints, at the upper limit. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All of these factors point to SVMs being relevant for exactly one use case: small datasets where the target pattern is thought, apriori, to be some regular, but highly non-linear, function of a large number of features. This use case actually arises fairly often. A recent example application where I found SVMs to be a natural approach was building predictive models for a target function that was known to be the result of interactions between pairs of features (specifically, communications between pairs of agents). An SVM with a quadratic kernel could therefore efficiently learn conservative, reasonable, guesses.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; There are approximate algorithms that will solve the SVM faster than this, as noted in the other answers.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-09-26T23:10:13.717" LastActivityDate="2019-09-26T23:10:13.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11975" PostTypeId="1" CreationDate="2019-04-23T15:53:56.627" Score="1" ViewCount="80" Body="&lt;p&gt;I am following this &lt;a href=&quot;https://github.com/ray0809/weakly-supervised-object-localization&quot; rel=&quot;nofollow noreferrer&quot;&gt;link : Weakly-supervised-object-localization&lt;/a&gt; to create heatmap of the region in an image where the CNN looks to identify the class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per the above mentioned repository , following steps are done :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 1 : Trained a model on my custom dataset of 3 classes &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def creat_model():&#xA;    inputs = Input(shape=(299,299,3))&#xA;    #Notic : preprocess is different in each Model&#xA;    resize = Lambda(Resize,(299,299,3))(inputs)&#xA;    #normal = Lambda(preprocess_input,(256,256,3))(resize)&#xA;    base_model = InceptionV3(weights='imagenet', include_top=False)&#xA;&#xA;    conv = base_model(resize)    #conv = base_model(resize)&#xA;    GAV = GlobalAveragePooling2D()(conv)&#xA;    outputs = Dense(3,activation='softmax')(GAV)&#xA;    model = Model(inputs,outputs)&#xA;    model.compile(optimizer='sgd',&#xA;                  loss='categorical_crossentropy',&#xA;                  metrics=['accuracy'])&#xA;    return model&#xA;&#xA;&#xA;&#xA;def training(model):&#xA;    earlystopping = EarlyStopping()&#xA;    modelchenkpoint = ModelCheckpoint('model_best_only',save_best_only=True)&#xA;    model.fit(trainX,trainY,&#xA;              batch_size=32,&#xA;              nb_epoch=1,&#xA;              validation_data=(testX,testY),&#xA;              callbacks=[earlystopping,modelchenkpoint])&#xA;&#xA;    model.save('model_save_file.h5')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Step 2 : Get convolutional feature map using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_conv(model,test_imgs):&#xA;    #Using InceptionV3's output&#xA;    #base_model = Model(model.input,model.get_output_at.output)&#xA;    inputs = Input(shape=(299,299,3))&#xA;    resize = Lambda(Resize,(299,299,3))(inputs)&#xA;    inception_v3 = model.get_layer('inception_v3')&#xA;    outputs = inception_v3(resize)&#xA;    new_model = Model(inputs,outputs)&#xA;    #new_model.save('conv.npy')&#xA;    print('Loading the conv_features of test_images .......')&#xA;    conv_features = new_model.predict(test_imgs)&#xA;    print('Loading the conv_features done!!!')&#xA;    print(conv_features)&#xA;    return conv_features&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Step 3 : Saved weights.npy , conv_features.npy and predict_label.npy  as numpy arrays using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv_features = get_conv(model,testX)&#xA;    np.save('conv_features', conv_features)&#xA;    print('Predict the labels of test_images .......')&#xA;    predict_label = model.predict(testX)&#xA;    np.save('predict_label', predict_label)&#xA;    print('Extraction the weight between GAV and dense(2048x10) .......')&#xA;    w = model.get_weights()[-2]&#xA;    np.save('weight', w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now I want to use this information and test this on a random input image and get the result as shown in the&lt;/strong&gt; &lt;a href=&quot;https://github.com/ray0809/weakly-supervised-object-localization&quot; rel=&quot;nofollow noreferrer&quot;&gt;above github page&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code shown in the github page for doing this ( I assume ) is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np import cv2 import tensorflow as tf&#xA;&#xA;W = np.zeros((10000,2048),dtype='float32')&#xA;&#xA;w = np.load('weight.npy') conv = np.load('conv_features.npy') predict&#xA;= np.load('predict_label.npy')&#xA;&#xA;&#xA;predict_label = (np.argmax(predict,axis=1)).astype('uint8')&#xA;#this W meas the coefficients of linear combination:10000x2048 W = np.transpose(w)[predict_label,:] W = np.expand_dims(W,axis=2)&#xA;&#xA;&#xA;#W and conv are too large temp1 = W[0:10] temp2 = conv[0:10]&#xA;&#xA;del W del conv&#xA;&#xA;&#xA;&#xA;&#xA;sess = tf.Session()&#xA;&#xA;weight = tf.placeholder(dtype='float32',shape=temp1.shape,name='weight') features = tf.placeholder(dtype='float32',shape=temp2.shape,name='features')&#xA;&#xA;resize_features = tf.image.resize_bicubic(features,size=(299,299)) resize_features = tf.reshape(resize_features,[-1,299*299,2048])&#xA;&#xA;#Hotmap = tf.batch_matmul(resize_features,weight) Hotmap = tf.matmul(resize_features,weight) Hotmap = tf.reshape(Hotmap,[-1,299,299])&#xA;&#xA;result_map = sess.run(Hotmap,feed_dict={weight:temp1,features:temp2})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Could anyone please help me to use this code and show the result on an input image as shown in the github repository ? Or any relevant information regarding this ?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="24123" LastActivityDate="2019-04-23T15:53:56.627" Title="Simple weakly supervised Object localizetion using keras. How to visualize the results?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11976" PostTypeId="1" CreationDate="2019-04-23T16:50:10.383" Score="4" ViewCount="236" Body="&lt;p&gt;What is an artificial neural network in artificial intelligence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is apparently used to find patterns in data and it is loosely inspired by human neural networks.&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="1671" LastEditDate="2019-05-29T21:19:57.020" LastActivityDate="2020-04-21T11:17:01.597" Title="What is an artificial neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11977" PostTypeId="1" CreationDate="2019-04-23T17:01:42.867" Score="0" ViewCount="42" Body="&lt;p&gt;I'm trying to figure out what's the best way to get a trained model that does facial detection to run on a camera that's connected to a computer and perform inference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I don't need the device to train the model, is there a low cost solution that can still do at least 10fps on a 1280x720 image?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any device that can run the model and have camera and computer all built in a small device?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, any suggestion on what's the best way to put one together and deploy it in production environment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11493" LastActivityDate="2019-05-31T10:51:08.797" Title="Edge AI device to run inference" Tags="&lt;ai-design&gt;&lt;hardware&gt;&lt;ai-development&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11978" PostTypeId="2" ParentId="11976" CreationDate="2019-04-23T17:59:30.183" Score="2" Body="&lt;p&gt;An (artificial) neural network (NN) can be described at different levels of detail, but, essentially, an NN is a &lt;em&gt;data structure&lt;/em&gt; (or model), which behaves like a &lt;strong&gt;function&lt;/strong&gt;: that is, it can receive one or more &lt;em&gt;inputs&lt;/em&gt; and it produces one or more &lt;em&gt;outputs&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the NNs can be thought of as &lt;em&gt;graph-like&lt;/em&gt; data structures that contain nodes which are connected by &lt;em&gt;weighted&lt;/em&gt; edges, where some of the nodes are associated with the inputs and some of the nodes are associated with the outputs. The remaining nodes are often called &quot;hidden nodes&quot;. However, there are NNs which might not be easily understood as a graph-like structure: for example, convolutional neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of nodes and the connections between them defines the architecture of the NN and, intuitively, its &quot;expressivity&quot;, that is, the number or type of functions it can approximate. Each node in an NN often computes a &quot;function&quot; of its inputs: for example, it could sum the inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weights of the edges connecting the nodes can change, during a &quot;learning or training phase&quot;, so that the NN produces the desired outputs given certain inputs. For example, we can train an NN to &quot;approximate&quot; the function &lt;span class=&quot;math-container&quot;&gt;$f(x) = x$&lt;/span&gt;. Thus, an NN is often described as a &quot;function  approximator&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several algorithms that can be used to &quot;train&quot; an NN to approximate a desired (and often unknown) function. The most common one is based on an optimisation algorithm (like gradient descent) and the back-propagation algorithm. In order to approximate a function, an NN requires data, often in the form of pairs of inputs and outputs, &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the input and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the output of the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (that we want to approximate). Intuitively, after the learning phase, an NN that receives an input similar to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, denoted by &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt;, will then produce an output similar to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of tasks that can be thought of as functions. For example, the task of machine translation can be thought of as a function &lt;span class=&quot;math-container&quot;&gt;$f(x) = y$&lt;/span&gt;, where the input, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, is a sentence in one language (e.g. English) and the output, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, is a sentence in another language (e.g. Chinese). In this case, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is unknown (to a lot of people), but an NN can still learn (from the data) an approximation of it. Therefore, NNs have wide applicability and are thus quite useful.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-04-23T17:59:30.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11979" PostTypeId="1" AcceptedAnswerId="11982" CreationDate="2019-04-23T19:12:53.727" Score="6" ViewCount="966" Body="&lt;p&gt;Gradient descent works on the equation of &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean squared error&lt;/a&gt;, which is an equation of a parabola &lt;span class=&quot;math-container&quot;&gt;$y=x^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We often say that weight adjustment in a neural network by gradient descent algorithm can hit a local minima and get stuck in there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is local minima on the equation of a parabola possible, where the slope is always parabolic?&lt;/p&gt;&#xA;" OwnerUserId="11789" LastEditorUserId="2444" LastEditDate="2019-04-25T21:09:05.203" LastActivityDate="2019-04-29T18:43:48.860" Title="How is local minima possible in gradient descent?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11980" PostTypeId="1" CreationDate="2019-04-23T19:31:01.550" Score="1" ViewCount="38" Body="&lt;p&gt;Note: I am NOT asking for general advantages of neuroevolution over standard approaches (e.g.: architecture search, parallelization), I am asking for examples of tasks in which, currently, neuroevolved networks outperform ANNs trained with gradient-based techniques. Of course, this is not opinion based, as I am asking for examples based on facts.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2019-07-07T19:16:01.040" LastActivityDate="2019-07-07T19:16:01.040" Title="What are some examples of tasks in which, currently, neuroevolution outperforms gradient-based approaches?" Tags="&lt;neural-networks&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11981" PostTypeId="2" ParentId="11979" CreationDate="2019-04-23T19:44:56.560" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is, how is local minima possible on the equation of a parabola, where the slope is always parabolic !&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A parabola has one minimum, and no separate local minima. So it isn't possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However . . .&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gradient descent works on the equation of mean squared error, which is an equation of a parabola &lt;span class=&quot;math-container&quot;&gt;$y=x^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Just because the loss function is a parabola with respect to the direct input, does not mean that the loss function is a parabola with respect to the parameters that indirectly cause that error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact it only remains true for linear functions. When considering linear regression &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = \sum_i w_i x_i + b$&lt;/span&gt;, there is only one global minimum (with specific values of &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; or specific vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt;), and your assertion is true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you add nonlinear activations, as in neural networks, then the relationship between error function and parameters of the model becomes far more complex. For the last/output layer you can carefully choose a loss function so that this cancels out - you can keep your single global minimum for logistic regression and softmax regression. However, one or more hidden layers, and all bets are off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact you can prove quite easily that a neural network with a hidden layer must have multiple stationary points (not necessarily local minima). The outline of the proof is to note that there must be multiple equivalent solutions, since in a fully-connected network you can re-arrange the nodes into any order, move the weights to match, and it will be a new solution with exactly the same behaviour, including the same loss on the dataset. So a neural network with one hidden layer with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; nodes must have &lt;span class=&quot;math-container&quot;&gt;$n!$&lt;/span&gt; absolute minimums. There is no way for these to exist without other stationary points in-between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is theory to suggest that most of the stationary points found in practice will not be local minima, but saddle points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, &lt;a href=&quot;http://liacs.leidenuniv.nl/assets/PDF/TechRep/tr94-19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this is an analysis of saddle points in a simple XOR approximator&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-23T19:44:56.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11982" PostTypeId="2" ParentId="11979" CreationDate="2019-04-23T19:49:21.737" Score="5" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(x) = x^2$&lt;/span&gt; is indeed a parabola and thus has just one optimum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i (y_i - f(x_i))^2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}$&lt;/span&gt; are the inputs, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{y}$&lt;/span&gt; the corresponding labels and the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the model (e.g. a neural network), is &lt;em&gt;not necessarily&lt;/em&gt; a parabola. In general, it is only a parabola if &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a constant function and the sum is over one element. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that &lt;span class=&quot;math-container&quot;&gt;$f(x_i) = c, \forall i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$c \in \mathbb{R}$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i (y_i - c)^2$&lt;/span&gt; will only change as a function of one variable, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{y}$&lt;/span&gt;, as in the case of &lt;span class=&quot;math-container&quot;&gt;$g(x) = x^2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is a function of one variable, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. In that case, &lt;span class=&quot;math-container&quot;&gt;$(y_i - c)^2$&lt;/span&gt; will just be a shifted version (either to the right or left depending on the sign of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;) of &lt;span class=&quot;math-container&quot;&gt;$y_i^2$&lt;/span&gt;, so, for simplicity, let's ignore &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. So, in the case &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a constant function, then &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i y_i^2$&lt;/span&gt;, which is a sum of parabolas &lt;span class=&quot;math-container&quot;&gt;$y_i^2$&lt;/span&gt;, which is called a &lt;em&gt;paraboloid&lt;/em&gt;. In this case, the paraboloid corresponding to &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i y_i^2$&lt;/span&gt; will only have one optimum, just like a parabola. Furthermore, if the sum is just over one &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i y_i^2 = y^2$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{y} = y$&lt;/span&gt;), then the MSE becomes a parabola. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other cases, the MSE might not be a parabola or have just one optimum. For example, suppose that &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_i = 1$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$\forall i$&lt;/span&gt;), then &lt;span class=&quot;math-container&quot;&gt;$h(x) = (1 - x^2)^2$&lt;/span&gt;  looks as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ykEmA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ykEmA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has two minima (and one maximum): at &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;. We can find the two minima of this function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; using calculus: &lt;span class=&quot;math-container&quot;&gt;$h'(x) = -4x(1 - x^2)$&lt;/span&gt;, which becomes zero when &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, we only considered one term of the sum. If we considered the sum of terms of the form of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, then we could even have more &quot;complicated&quot; functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, given that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; can be arbitrarily complex, then also &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y})$&lt;/span&gt;, which is a function of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, can also become arbitrarily complex and have multiple minima. Given that neural networks can implement arbitrarily complex functions, then &lt;span class=&quot;math-container&quot;&gt;$\text{MSE}(\boldsymbol{x}, \boldsymbol{y})$&lt;/span&gt; can easily have multiple minima. Moreover, the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (e.g. the neural network) changes during the training phase, which might introduce more complexity, in terms of which functions the MSE can be and thus which (and how many) optima it can have.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-27T00:39:06.020" LastActivityDate="2019-04-27T00:39:06.020" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11983" PostTypeId="2" ParentId="11937" CreationDate="2019-04-23T20:14:46.117" Score="0" Body="&lt;p&gt;The weights of the filters do not always and necessarily decrease. Consider the extreme case when you initialise them to &lt;span class=&quot;math-container&quot;&gt;$-\infty$&lt;/span&gt; and you want to approximate a function different than the one the CNN represents initially with all weights set to &lt;span class=&quot;math-container&quot;&gt;$-\infty$&lt;/span&gt;. You will have to increase one or more weights. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-04-23T20:14:46.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11984" PostTypeId="2" ParentId="6317" CreationDate="2019-04-24T01:42:38.157" Score="0" Body="&lt;p&gt;The main difference between on-policy and off-policy is how to get samples and what policy we optimize. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In off-policy deterministic actor-critic, the trajectories are samples from beta distribution (also called behavior policy), not the policy we are optimized (that is &lt;span class=&quot;math-container&quot;&gt;$\mu_{\theta}$&lt;/span&gt;). However, in the on-policy actor-critic, the action &lt;span class=&quot;math-container&quot;&gt;$a_{t+1}$&lt;/span&gt; is sampled from target policy &lt;span class=&quot;math-container&quot;&gt;$\mu_{\theta}$&lt;/span&gt; and the policy we optimized is also the &lt;span class=&quot;math-container&quot;&gt;$\mu_{\theta}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25129" LastEditorUserId="2444" LastEditDate="2019-04-24T09:29:01.420" LastActivityDate="2019-04-24T09:29:01.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11985" PostTypeId="1" AcceptedAnswerId="12002" CreationDate="2019-04-24T03:42:24.370" Score="2" ViewCount="42" Body="&lt;p&gt;Let's say I've trained a CNN that is predicting/inferring live samples that it hasn't seen before. In the event the network makes a correct prediction, would including this as a new sample in its training set increase the model accuracy even further when re-training the network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unsure about this since it seems as though the network has already learnt the necessary features for making the correct prediction, so adding it as a new training sample might be redundant. On the other hand it might also reinforce to the network that it's on the right track, perhaps giving it further confidence to generalize with whatever features its learnt in regards to that class, that it might be able to apply to the same class in other images it might otherwise make an incorrect prediction with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I'm thinking of this is that manually labeling each image is a time-consuming process, however if a simple &quot;Correct/Incorrect&quot; popup box was presented after the network made a live prediction, then it's simply a matter of clicking a single button to generate a new labelled training sample, which would be a far easier labeling task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how useful would it be to do something like this?&lt;/p&gt;&#xA;" OwnerUserId="6328" LastActivityDate="2019-04-24T19:26:43.057" Title="Does reinforcing correct predictions increase model accuracy further?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11986" PostTypeId="1" CreationDate="2019-04-24T08:31:55.223" Score="1" ViewCount="19" Body="&lt;p&gt;We have various types of data features with different temporal scale. For example, some of them describe the state per second while others may describe the state per day or per month from another aspect. The former features are dense on the time scale and latter features are sparse. Simply concatenate them into one feature vector seems not proper. Is there any typical method in machine learning can handle with problem ?&lt;/p&gt;&#xA;" OwnerUserId="20587" LastActivityDate="2019-04-24T08:31:55.223" Title="How to combine features with different temporal scale in machine learning" Tags="&lt;machine-learning&gt;&lt;feature-selection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11987" PostTypeId="1" AcceptedAnswerId="11997" CreationDate="2019-04-24T09:08:25.843" Score="3" ViewCount="409" Body="&lt;p&gt;I am now working on training an alphazero player for a board game. The implementation of board game is mine, MCTS for alphazero was taken elsewhere. Due to complexity of the game, it takes a much longer time to self-play than to train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you know, alphazero has 2 heads: value and policy. In my loss logging I see that with time, the value loss is decreasing pretty significantly. However, the policy loss only demonstrates fluctuation around its initial values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe someone here has run into similar problems? I would like to know if its my implementation problem (but then the value loss is decreasing) or just a matter of not enough data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, perhaps importantly, the game has ~17k theoretically possible moves, but only 80 at max are legal at any single state (think chess - a lot of possibles but very few are actually legal at any given time). Also, if MCTS has 20 simulations, then the improved probabilities vector (against which we train our policy loss) will have at most 20 non-zero entries. My idea was that it might be hard for the network to learn such sparse vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any ideas!&lt;/p&gt;&#xA;" OwnerUserId="21278" LastActivityDate="2019-04-24T15:56:48.053" Title="Alphazero policy head loss not decreasing" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;alphago&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11988" PostTypeId="1" CreationDate="2019-04-24T11:07:44.487" Score="0" ViewCount="410" Body="&lt;p&gt;As far as I know, in PDDL, an environment is designed as well as the initial state described. When we describe the target state, the solver creates some sort of a graph. How is the graph built and what are the keys (keywords) in PDDL referring to?&lt;/p&gt;&#xA;&lt;p&gt;I know that there are many flavours of PDDL, but let's go with the standard or the most common version of PDDL.&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="2444" LastEditDate="2021-01-23T00:13:53.403" LastActivityDate="2021-01-23T00:13:53.403" Title="How does a PDDL solver find a solution for a given problem?" Tags="&lt;planning&gt;&lt;pddl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11989" PostTypeId="2" ParentId="11971" CreationDate="2019-04-24T11:18:10.333" Score="2" Body="&lt;p&gt;Some lines above the author says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;hence we can assume &lt;span class=&quot;math-container&quot;&gt;$X \sim \mathcal{N}(0,1)$&lt;/span&gt; with the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; domain being &lt;strong&gt;continuous&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then he says the input vector is assumed to be binary which changes the domain &lt;strong&gt;from continuous to discrete&lt;/strong&gt; so we can discretize it assuming &lt;span class=&quot;math-container&quot;&gt;$-1 \le X \le 1$&lt;/span&gt; is mapped into zero and &lt;span class=&quot;math-container&quot;&gt;$X&amp;lt; -1 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X &amp;gt; 1$&lt;/span&gt; are mapped to 1 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally according to the &lt;a href=&quot;https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;68-95-99.7 Rule&lt;/a&gt; we can compute &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally probably the author rounds this up to &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{3} \simeq 0.33$&lt;/span&gt; &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-04-24T11:18:10.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11990" PostTypeId="2" ParentId="11988" CreationDate="2019-04-24T12:36:54.003" Score="2" Body="&lt;p&gt;The question doesn't really make sense: &lt;a href=&quot;https://en.wikipedia.org/wiki/Planning_Domain_Definition_Language&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDDL&lt;/a&gt; is a &lt;em&gt;description language&lt;/em&gt; that is used to formulate a problem. This description then is the &lt;em&gt;input&lt;/em&gt; to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;planner&lt;/a&gt;; how the planner arrives at the intended solution is not related to the PDDL description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling#Algorithms_for_planning&quot; rel=&quot;nofollow noreferrer&quot;&gt;a number of planning algorithms&lt;/a&gt;, and you can implement any of them to make use of a PDDL description. The output of the solver is a plan, which is usually an ordered sequence of actions, and a tree or graph structure might be a good way of capturing this.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-04-24T12:36:54.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11991" PostTypeId="2" ParentId="11956" CreationDate="2019-04-24T12:49:01.533" Score="3" Body="&lt;p&gt;The choice of the most appropriate search algorithm for a particular task is often based (but not exclusively) on its &lt;em&gt;time complexity&lt;/em&gt;, &lt;em&gt;space complexity&lt;/em&gt;, &lt;em&gt;termination&lt;/em&gt; (if the algorithm always halts), &lt;em&gt;optimality guarantees&lt;/em&gt; (if the algorithm is guaranteed to find the optimal solution), &lt;em&gt;available implementations&lt;/em&gt; (as software libraries) and (if known) the &lt;em&gt;actual performance&lt;/em&gt; (for such particular task). There are algorithms that have the same time or space complexities, but that, in practice, have different performance, which also depends on the problem being solved and implementations (which might or not be optimised). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider a search tree where you have a finite number of nodes and paths. Suppose that solutions lie at the leaves of this tree and that you are interested in one solution, but not necessarily the optimal one. In that case, in practice, DFS might find solutions faster than BFS, because of its nature and the task being solved. However, in the case you have a tree with infinite-length paths, then DFS might not terminate. In that case, proceeding layer by layer (BFS) might be a more appropriate strategy.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-04-24T12:49:01.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11992" PostTypeId="1" AcceptedAnswerId="12123" CreationDate="2019-04-24T14:15:02.803" Score="5" ViewCount="690" Body="&lt;p&gt;Following the &lt;strong&gt;DQN algorithm&lt;/strong&gt; with experience replay:&lt;/p&gt;&#xA;&lt;p&gt;Store transition &lt;span class=&quot;math-container&quot;&gt;$\left(\phi_{t}, a_{t}, r_{t}, \phi_{t+1}\right)$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; Sample random minibatch of transitions &lt;span class=&quot;math-container&quot;&gt;$\left(\phi_{j}, a_{j}, r_{j}, \phi_{j+1}\right)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; Set&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{j}=\left\{\begin{array}{cc}r_{j} &amp;amp; \text { if episode terminates at j+1} \\ r_{j}+\gamma \max _{d^{\prime}} \hat{Q}\left(\phi_{j+1}, a^{\prime} ; \theta^{-}\right) &amp;amp; \text {otherwise }\end{array}\right.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Perform a gradient descent step on &lt;span class=&quot;math-container&quot;&gt;$\left(y_{j}-Q\left(\phi, a_{j} ; \theta\right)\right)^{2}$&lt;/span&gt; with respect to the network parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We calculate the &lt;span class=&quot;math-container&quot;&gt;$loss=(Q(s,a)-(r+Q(s+1,a)))^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Assume I have positive but changing rewards. Meaning, &lt;span class=&quot;math-container&quot;&gt;$r&amp;gt;0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thus, since the rewards are positive, by calculating the loss, I notice that almost always &lt;span class=&quot;math-container&quot;&gt;$Q(s)&amp;lt; Q(s+1)+r$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, the network learns to always increase the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function , and eventually, the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function is higher in same states in later learning steps.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;How can I stabilize the learning process?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="25141" LastEditorUserId="36737" LastEditDate="2021-04-04T15:14:29.787" LastActivityDate="2021-04-04T15:14:29.787" Title="How to stop DQN Q function from increasing during learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;loss-functions&gt;&lt;value-functions&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11993" PostTypeId="2" ParentId="11992" CreationDate="2019-04-24T14:45:21.490" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use discount factor gamma less then one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use finite time horizon - only for states which are no farther away then T time steps reward propagate back&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use sum of rewords averaged over time for Q&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All of those are legitimate approaches.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-04-24T14:45:21.490" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11994" PostTypeId="1" CreationDate="2019-04-24T14:53:35.293" Score="1" ViewCount="55" Body="&lt;p&gt;I have a big amount of light curves (image below). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aKmwP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aKmwP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to label the points as signal or background (the signal appears usually periodically, several times, for a given light curve). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, I want to identify the downward spikes (class label = 1) from the background (class label = 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the data is not labeled. I tried labeling it by hand, and using a bi-directional LSTM succeeds in labeling the data points properly. However, there are thousands of light curves and labeling all of them would take very long. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good unsupervised approach to do this (unsupervised LSTM maybe, but any other method that might work on time series would do just fine)? &lt;/p&gt;&#xA;" OwnerUserId="23871" LastEditorUserId="2444" LastEditDate="2020-04-12T19:09:01.907" LastActivityDate="2020-04-12T19:09:01.907" Title="Is there an LSTM-based unsupervised learning algorithm to label a dataset of curves?" Tags="&lt;long-short-term-memory&gt;&lt;datasets&gt;&lt;unsupervised-learning&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11995" PostTypeId="2" ParentId="11992" CreationDate="2019-04-24T15:03:49.347" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Therefore,the network learns to always increase the Q function , and eventually the Q function is higher in same states in later learning steps &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If your value function keeps increasing in later steps that means that the network is still learning those Q-values, you shouldn't necessarily prevent that. Your Q-values won't increase forever even if the rewards are always positive. You basically have a regression problem here and when the value of &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; becomes very close to the predicted value of  &lt;span class=&quot;math-container&quot;&gt;$r+Q(s',a)$&lt;/span&gt; value of &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; will stop increasing by itself.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-04-24T15:03:49.347" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11996" PostTypeId="2" ParentId="11924" CreationDate="2019-04-24T15:04:46.740" Score="0" Body="&lt;p&gt;Thanks for the responses - after taking into account the 'risk' the network experiences when working with unstable financial data I &lt;em&gt;modified the activation layers from RelU to Sigmoid&lt;/em&gt; - this has lead to a considerable improvement to both results (profit!) and additionally the rate at which actions are carried out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will also edit the above question to provide further details.&lt;/p&gt;&#xA;" OwnerUserId="20893" LastActivityDate="2019-04-24T15:04:46.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11997" PostTypeId="2" ParentId="11987" CreationDate="2019-04-24T15:56:48.053" Score="2" Body="&lt;p&gt;The loss of the policy head here is really quite different from losses in, for instance, more &quot;conventional&quot; Supervised Learning approaches (where we typically expect/hope to see a relatively steady decrease in loss function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this AlphaZero setup, the target that we're updating the policy head towards is itself changing during the training process. When we improve our policy, we expect the MCTS &quot;expert&quot; to also be improved, which may lead to a different distribution of MCTS visit counts, which in turn may lead to a different update target for the policy head from previous update targets. So it's perfectly fine if our &quot;loss&quot; &lt;strong&gt;increases&lt;/strong&gt; sometimes, we may still actually be performing better. The loss is useful for the computation of our gradient, but otherwise it doesn't have much use -- it certainly cannot be used as an accurate indicator of performance / learning progress.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;but only 80 at max are legal at any single state (think chess - a lot of possibles but very few are actually legal at any given time). Also, if MCTS has 20 simulations, then the improved probabilities vector (against which we train our policy loss) will have at most 20 non-zero entries.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This can be a problem yes. The fact that the majority of moves are not legal at any point in time is not a problem, but if you only have 20 MCTS simulations for a branching factor of 80... that's certainly a problem. The easiest fix would be to simply keep MCTS running for longer, but obviously that's going to take more computation time. If you cannot afford to do this for every turn of self-play, you could try:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;using only a low MCTS iteration count for some moves, not adding these distributions to the training data for the policy head&lt;/li&gt;&#xA;&lt;li&gt;using a larger MCTS iteration count for some other moves, and only using the distributions of these moves as training data for the policy head&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This idea is described in more detail in Subsection 6.1 of &lt;a href=&quot;https://arxiv.org/abs/1902.10565&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accelerating Self-Play Learning in Go&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-04-24T15:56:48.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11999" PostTypeId="1" CreationDate="2019-04-24T18:15:44.087" Score="8" ViewCount="911" Body="&lt;p&gt;Semantics means the meaning and interpretation of words, signs, and sentence structure. Semantics largely determine our reading comprehension, how we understand others, and even what decisions we make as a result of our interpretations. Semantics can also refer to the branch of study within linguistics that deals with language and how we understand meaning. This has been a particularly interesting field for philosophers as they debate the essence of meaning, how we build meaning, how we share meaning with others, and how meaning changes over time.so actually in artificial intelligence what do semantic analysis used for ? &lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="11539" LastEditDate="2020-04-25T19:57:17.030" LastActivityDate="2020-04-25T19:57:17.030" Title="In Artificial Intelligence, what are semantic analyses used for?" Tags="&lt;semantics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12000" PostTypeId="2" ParentId="11999" CreationDate="2019-04-24T18:32:00.787" Score="1" Body="&lt;p&gt;In supervised learning the semantic gets injected into the NN via the supervision signal &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a typical pedestrian detection NN trained in a supervised learning way, has no knowledge about what the label &quot;pedestrian&quot; actually means before training, this semantic is injected during training by means of the supervision signal &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However this &quot;top-down&quot; approach (based on a high level supervision signal) is not the only way semantic can be got from data because data itself, in its representation space, has a structure (like a manifold in its container space) which can be automatically discovered by NN in an unsupervised learning way like for &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1711.02209&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised Learning of Semantic Audio Representations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/8354271&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised Clustering Guided Semantic Segmentation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-04-24T18:32:00.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12001" PostTypeId="1" CreationDate="2019-04-24T19:09:25.247" Score="1" ViewCount="26" Body="&lt;p&gt;Having two point clouds, the second being a transformation of the first, how could I utilize a neural network in order to solve the pose (transformation in terms of x, y, z, rx, ry, rz) of the second point cloud?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the point clouds can be rather large (~200,000 points), I think it'd be best to first select regions from each point cloud and see if their geometries are similar (still researching the optimal method for this). If they are not similar, I'd choose two new points. If they are similar, I'd use those two regions when implementing the neural network to discern the pose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My preliminary research has led me to believe that a Siamese neural network may work in this scenario but I'm not sure if there are better alternatives. One of my goals is to accomplish this without relying on Iterative Closest Point. Any and all insight is appreciated. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="25147" LastActivityDate="2019-04-24T19:09:25.247" Title="Point Cloud Alignment using a Neural Network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12002" PostTypeId="2" ParentId="11985" CreationDate="2019-04-24T19:26:43.057" Score="1" Body="&lt;p&gt;Your suggestion is risky. It &lt;em&gt;might&lt;/em&gt; make improvements to your classifier, but it may also reduce generalisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two conflicting factors in play are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Adding data points to a model &lt;em&gt;which has capacity to learn more detail&lt;/em&gt; can improve its performance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Training from a different distribution of data points than your target population can reduce its performance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From the first point, you need your model to be able to accept the new data. This can be harder to achieve than you might think at first - if you have tuned some regularisation parameters by using cross-validation, then you may have at least in part saturated the model's capacity in order to prevent over-fitting. That means fitting to new data could require re-tuning your hyper-parameters as well - but you probably won't need to start from scratch, just search nearby.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is hard to tell the impact from the second point. There is not going to be a general answer here, too much depends on how the data is arranged for your problem. My gut feeling is that you will notice an improvement to the model initially, but that there will be diminishing returns due to the self-selection of already-correct data points.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should definitely keep back a cross-validation set of data distributed as it is in production, that has been properly and fully labelled in order to assess this work. This will be your way to assess whether generalisation is improving using your approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Worst case: You may need to go back and re-label the mistakes in order to significantly improve performance of your model. So I would suggest part of your auto-labelling process should store the misclassified items somewhere so you can re-visit them and spend the extra effort. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly, even collecting more properly labelled data is not a guaranteed fix - some models already have enough data. You can check whether that might applies in your case by training with different sizes of training set (from the available training data you have so far), and seeing what the trend in performance is when you increase the training set size. In fact, you should probably do this &lt;em&gt;first&lt;/em&gt;, before you invest significant effort in collecting more data.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-24T19:26:43.057" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12003" PostTypeId="2" ParentId="11999" CreationDate="2019-04-24T22:24:46.210" Score="3" Body="&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Processing&lt;/a&gt; (NLP), you are dealing with knowledge/intents/... expressed in linguistic structures. For example, if you have a text understanding system, you would read the text, usually sentence by sentence, analyse the syntactic structure, and then the meaning (composed of the word meaning and the structural meaning, plus the pragmatic meaning (based on the situation/context)); in other words, looking at the semantic content of a sentence helps you understand what it is about. An AI system can then process the content towards the goal of understanding the text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In natural language generation, a semantic representation is turned into text. Here the process in reversed: the meaning of the sentence is known, but the appropriate words and sentence structures have to be chosen by the generator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are two examples where semantic analysis is used in NLP; there are plenty of others, as meaning is central to the processing of natural language. &lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-04-24T22:24:46.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12005" PostTypeId="1" CreationDate="2019-04-25T02:24:42.983" Score="0" ViewCount="67" Body="&lt;p&gt;What is the goal of a constraint solver? How are constraints propagated in a constraint satisfaction search?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references are also appreciated. &lt;/p&gt;&#xA;" OwnerUserId="23299" LastEditorUserId="2444" LastEditDate="2020-09-08T22:45:16.723" LastActivityDate="2020-09-08T22:45:16.723" Title="What is the goal of a constraint solver?" Tags="&lt;definitions&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12006" PostTypeId="2" ParentId="12005" CreationDate="2019-04-25T03:15:33.720" Score="2" Body="&lt;p&gt;You want Chapter 6 of Russell &amp;amp; Norvig's &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI: A Modern Approach&lt;/a&gt;, for a starting place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of a constraint solver is to find an assignment of &lt;em&gt;values&lt;/em&gt; to &lt;em&gt;variables&lt;/em&gt; such that every variable is assigned a value, but none of a list of &lt;em&gt;constraints&lt;/em&gt; are violated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example problem that a constraint solver can solve is the problem of assigning students to groups for a project. The variables are the names of the students. Each variable has a set of possible values it could take on that correspond to the names of the groups. The constraints prohibit assigning certain students to the same group (perhaps they hate each other), or require certain students to be assigned to the same group (perhaps they like each other). More complex constraints are also possible (e.g. at least 2 of these 5 students need to be in the same group).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constraint propagation is the idea that several constraints might logically imply a stronger constraint. As a simple example, suppose that student &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; must be assigned to group &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. That's one constraint. Suppose further that student &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; must be assigned to the same group as &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. That's another constraint. Together, however, they imply that &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; must be assigned to group &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most constraint solvers operate by iteratively trying to assign values to variables and then backtracking when a constraint is violated. When assignments are made, different constraint propagation methods make different decisions about how much computational effort to spend deriving the logical consequences of constraints, as opposed to simply trying more assignments of values to variables until the consequences become apparent (one of the existing constraints gets violated). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest form of constraint propagation is called &lt;a href=&quot;https://ktiml.mff.cuni.cz/~bartak/constraints/propagation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;forward checking&lt;/a&gt;. After each speculative assignment of a value to a variable, forward checking looks at all the constraints the variable is involved in. It then derives the logical consequences of the assignments. For example, continuing with the student problem from above, if &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; was assigned value &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, then forward checking would notice the constraint &lt;span class=&quot;math-container&quot;&gt;$S=T$&lt;/span&gt;, and would, effectively, add the constraint &lt;span class=&quot;math-container&quot;&gt;$T=1$&lt;/span&gt; (it actually does something a little different, but it has this effect). As another example, if the assignment &lt;span class=&quot;math-container&quot;&gt;$T=2$&lt;/span&gt; were made, forward checking would notice that &lt;span class=&quot;math-container&quot;&gt;$S=T$&lt;/span&gt;, and would add the constraint &lt;span class=&quot;math-container&quot;&gt;$S=2$&lt;/span&gt;. It would also detect the contradiction that &lt;span class=&quot;math-container&quot;&gt;$S=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S=2$&lt;/span&gt; are both constraints, and would declare the assignment &lt;span class=&quot;math-container&quot;&gt;$T=2$&lt;/span&gt; invalid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More advanced constraint propagation methods can reason about these kinds of logical consequences &lt;em&gt;before&lt;/em&gt; assignment are made. For example, they could derive that &lt;span class=&quot;math-container&quot;&gt;$T=1$&lt;/span&gt; before the program starts. Wikipedia has a good summary of these, starting with &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_consistency#Arc_consistency&quot; rel=&quot;nofollow noreferrer&quot;&gt;arc consistency&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-25T03:15:33.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12007" PostTypeId="1" AcceptedAnswerId="12009" CreationDate="2019-04-25T12:28:33.710" Score="1" ViewCount="92" Body="&lt;p&gt;I've been unsure about a principle of Q-Learning, I was hoping someone could clear it up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a new state is encountered, and thus there are no existing Q values, and that the algorithm decides to exploit, and not explore, how is the move chosen, since all the values are 0?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it chosen randomly? This intuitively would make sense, since after this, the state-move pair would have a value  and thus the matrix would get filled up throughout the iterations. But I just want to make sure I understand this correctly...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="24018" LastActivityDate="2019-04-25T15:06:15.077" Title="Picking a random move in exploitation in Q-Learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12008" PostTypeId="1" CreationDate="2019-04-25T12:56:02.830" Score="1" ViewCount="74" Body="&lt;p&gt;I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I share the encoder with the actor and the critic, my network isn't learning anything:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Encoder(nn.Module):&#xA;  def __init__(self, state_dim):&#xA;    super(Encoder, self).__init__()&#xA;&#xA;    self.l1 = nn.Linear(state_dim, 512)&#xA;&#xA;  def forward(self, state):&#xA;    a = F.relu(self.l1(state))&#xA;    return a&#xA;&#xA;class Actor(nn.Module):&#xA;  def __init__(self, state_dim, action_dim, max_action):&#xA;    super(Actor, self).__init__()&#xA;&#xA;    self.l1 = nn.Linear(state_dim, 128)&#xA;    self.l3 = nn.Linear(128, action_dim)&#xA;&#xA;    self.max_action = max_action&#xA;&#xA;  def forward(self, state):&#xA;    a = F.relu(self.l1(state))&#xA;    # a = F.relu(self.l2(a))&#xA;    a = torch.tanh(self.l3(a)) * self.max_action&#xA;    return a&#xA;&#xA;class Critic(nn.Module):&#xA;  def __init__(self, state_dim, action_dim):&#xA;    super(Critic, self).__init__()&#xA;&#xA;    self.l1 = nn.Linear(state_dim + action_dim, 128)&#xA;    self.l3 = nn.Linear(128, 1)&#xA;&#xA;  def forward(self, state, action):&#xA;    state_action = torch.cat([state, action], 1)&#xA;&#xA;    q = F.relu(self.l1(state_action))&#xA;    # q = F.relu(self.l2(q))&#xA;    q = self.l3(q)&#xA;    return q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I use different encoder for the actor and different for the critic, it learn properly.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Actor(nn.Module):&#xA;def __init__(self, state_dim, action_dim, max_action):&#xA;    super(Actor, self).__init__()&#xA;&#xA;    self.l1 = nn.Linear(state_dim, 400)&#xA;    self.l2 = nn.Linear(400, 300)&#xA;    self.l3 = nn.Linear(300, action_dim)&#xA;&#xA;    self.max_action = max_action&#xA;&#xA;def forward(self, state):&#xA;    a = F.relu(self.l1(state))&#xA;    a = F.relu(self.l2(a))&#xA;    a = torch.tanh(self.l3(a)) * self.max_action&#xA;    return a&#xA;&#xA;class Critic(nn.Module):&#xA;  def __init__(self, state_dim, action_dim):&#xA;    super(Critic, self).__init__()&#xA;&#xA;    self.l1 = nn.Linear(state_dim + action_dim, 400)&#xA;    self.l2 = nn.Linear(400, 300)&#xA;    self.l3 = nn.Linear(300, 1)&#xA;&#xA;  def forward(self, state, action):&#xA;    state_action = torch.cat([state, action], 1)&#xA;&#xA;    q = F.relu(self.l1(state_action))&#xA;    q = F.relu(self.l2(q))&#xA;    q = self.l3(q)&#xA;    return q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Im pretty sure its becuase of the optimizer. In the shared encoder code, I define it as foolow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.actor_optimizer = optim.Adam(list(self.actor.parameters())+&#xA;                                      list(self.encoder.parameters()))&#xA;self.critic_optimizer = optim.Adam(list(self.critic.parameters()))&#xA;                                         +list(self.encoder.parameters()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the seperate encoder, its just:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.actor_optimizer = optim.Adam((self.actor.parameters()))&#xA;self.critic_optimizer = optim.Adam((self.critic.parameters()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;two optimizers must be becuase of the actor critic algorithm, in which the loss of the actor is the value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I combine two optimizers to optimize correctly the encoder?&lt;/p&gt;&#xA;" OwnerUserId="25141" LastActivityDate="2021-03-21T13:05:10.530" Title="How to properly optimize shared network between actor and critic?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;actor-critic-methods&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="14" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12009" PostTypeId="2" ParentId="12007" CreationDate="2019-04-25T15:06:15.077" Score="3" Body="&lt;p&gt;It depends on the implementation of the software package that you are using. If you call a function that returns the maximum value and all values are the same then it might return the value at first index or some other one. The point is it doesn't matter which action is chosen since all of them are the best at the same time. So it's basically random but you should treat it as if you are trying to pick the best action.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-04-25T15:06:15.077" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12010" PostTypeId="1" CreationDate="2019-04-25T15:39:38.677" Score="0" ViewCount="100" Body="&lt;p&gt;I am trying to understand &lt;a href=&quot;https://aboveintelligent.com/ml-cnn-translation-equivariance-and-invariance-da12e8ab7049&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;, but I get confused by the definitions and the differences. What's definition of equivariant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I remove all the pooling layers from a CNN, will it make the network to detect features in pixel resolution? For example, detecting the local maximum of a pixel. For example, can a CNN be designed to return True for the following case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i5t7p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i5t7p.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And False for the shifted window:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2l7Gh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2l7Gh.png&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second case it returns false because the 3x3 submatrix isn't centered (yellow dash line) around the local maximum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will an architecture that is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.layers import Dense, Conv2D, Flatten&#xA;model = Sequential()&#xA;model.add(Conv2D(128, kernel_size=2, activation=’relu’, padding='same', input_shape=(3,3,1)))&#xA;model.add(Conv2D(64, kernel_size=2, activation=’relu’, padding='same'))&#xA;model.add(Flatten())&#xA;model.add(Dense(10, activation=’softmax’))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;be able to differentiate between the tiling of the larger grayscale image?&lt;/p&gt;&#xA;" OwnerUserId="12975" LastEditorUserId="12975" LastEditDate="2019-04-28T07:50:33.303" LastActivityDate="2019-04-28T07:50:33.303" Title="How can I suppress a CNN’s translation invariant or translation equivariant?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12011" PostTypeId="1" CreationDate="2019-04-25T16:43:43.963" Score="1" ViewCount="107" Body="&lt;p&gt;Is there a Hebb neural network? What kind of functions can it implement? Or, are there multiple &quot;Hebb networks&quot;, that is, neural networks that learn in a Hebbian fashion?&lt;/p&gt;&#xA;" OwnerUserId="23299" LastEditorUserId="2444" LastEditDate="2019-04-28T12:28:38.207" LastActivityDate="2019-04-28T12:57:24.533" Title="Is there a Hebb neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;terminology&gt;&lt;comparison&gt;&lt;hebbian-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12012" PostTypeId="2" ParentId="12010" CreationDate="2019-04-25T16:58:46.670" Score="1" Body="&lt;h2&gt;Question 1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To make it simple&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You have a transformation &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and an operator &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; acting on a given input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Let's say you do this experiment&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;compute &lt;span class=&quot;math-container&quot;&gt;$y_{1} = C(T(x))$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;compute &lt;span class=&quot;math-container&quot;&gt;$y_{2} = C(x)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can get three different results:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y_{1} = y_{2}$&lt;/span&gt; then you can say the operator is invariant with respect to the given transformation&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y_{1} = T(y_{2})$&lt;/span&gt; then you can say the operator is equivariant to the given transformation as applying it to the input basically reflects its effect completely on the output&lt;/li&gt;&#xA;&lt;li&gt;none of the 2&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Questions 2&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Q2.1&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Spatial Pooling is not the only way to perform dimensionality reduction. You can achieve it even simply applying the Conv Kernel with no padding.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, let's say your input is a &lt;code&gt;WxHxC&lt;/code&gt; tensor and you are applying a kernel which is &lt;code&gt;KxKxC&lt;/code&gt; the result will have spatial domain size &lt;code&gt;(W-(K/2))x(H-(K/2))&lt;/code&gt; with &lt;code&gt;K/2&lt;/code&gt; the integer truncation so if &lt;code&gt;K=5&lt;/code&gt; then &lt;code&gt;K/2=2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can reduce the spatial domain with stride.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Q2.2&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It seems to me you are talking about a sort of Non Max Suppression Operator rather an emergent behaviour, but certainly with the proper supervision signal you can train a CNN to do this work (even if practically it does not make sense as you can explicitly define it).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="16229" LastEditDate="2019-04-26T11:48:58.593" LastActivityDate="2019-04-26T11:48:58.593" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12016" PostTypeId="2" ParentId="6478" CreationDate="2019-04-26T07:21:31.433" Score="1" Body="&lt;p&gt;From your matrix definitions, the issue is S always singular so it can never be inverted &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I reimplemented the computation with numpy and here are the numbers &lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H = np.array([[1.0, 0.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0]])&#xA;&#xA;P = np.identity(5) * 1000&#xA;&#xA;R = np.array([[100, 0, 0, 0], [0, 100, 0, 0], [0,0,0,0], [0,0,0,0]])&#xA;&#xA;S=H.dot(P.dot(np.transpose(H)))+R&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;S&lt;/code&gt; is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[1100.,    0.,    0.,    0.],&#xA;       [   0., 1100.,    0.,    0.],&#xA;       [   0.,    0.,    0.,    0.],&#xA;       [   0.,    0.,    0.,    0.]])&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically the definition of &lt;code&gt;H&lt;/code&gt; with its all zeros lines cancels part of the information (make it unobservable) and you also do not have any observation noise component for them &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-04-26T07:21:31.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12017" PostTypeId="1" AcceptedAnswerId="12018" CreationDate="2019-04-26T09:04:08.863" Score="1" ViewCount="246" Body="&lt;p&gt;I have data with about 100 numerical features and a multi-labelling that encodes ownership of a certain product (i.e. my labels are of the form &lt;span class=&quot;math-container&quot;&gt;$[x_i, i=1, \dots, n]$&lt;/span&gt; where n is the number of products and &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is either 0 or 1). My neural network approach to this currently looks like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(1024, activation='relu', input_dim=X_train.shape[1]))&#xA;model.add(Dense(1024, activation='relu'))&#xA;model.add(Dense(1024, activation='relu'))&#xA;model.add(Dense(1024, activation='relu'))&#xA;model.add(Dense(y_train.shape[1], activation='softmax'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in Keras (i.e. a couple of dense layers with ReLu activation, then an output layer with softmax).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is: will the network consider labels of the other products when considering a probability to assign to the label of one product? I would like that to happen, but I can't quite grasp whether it does (my suspicion is no).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new to multilabel classification and relatively new to NN in general, so I hope this isn't too inept a question.  &lt;/p&gt;&#xA;" OwnerUserId="25190" LastEditorUserId="16565" LastEditDate="2019-05-20T08:46:03.147" LastActivityDate="2019-05-20T08:46:03.147" Title="Multi-label classification (e.g. as used in Keras) using DNN" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12018" PostTypeId="2" ParentId="12017" CreationDate="2019-04-26T09:30:15.683" Score="1" Body="&lt;p&gt;Firstly you should use sigmoid in your last layer instead of softmax. Softmax returns a probability distribution, meaning that when one labels probability increases the other will decrease, which is not always the case. Secondly in order for keras to use all the labels you should use binary_crossentropy as the loss function. &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-04-26T09:30:15.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12019" PostTypeId="1" AcceptedAnswerId="12025" CreationDate="2019-04-26T10:20:29.403" Score="3" ViewCount="193" Body="&lt;p&gt;I have a genetic algorithm that maximizes a fitness function with two variables f(X,Y).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been running the algorithm with various parameters in mutation and crossover probability (0.1, 0.2, ...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I dont have much theoretical knowledge of GA, how could I proceed in order to find the optimal values for mutation and crossover probability, and if necessary the optimal population size ?&lt;/p&gt;&#xA;" OwnerUserId="25194" LastEditorUserId="10135" LastEditDate="2019-05-05T12:46:52.967" LastActivityDate="2019-05-05T12:46:52.967" Title="How to find optimal mutation probability and crossover probability?" Tags="&lt;algorithm&gt;&lt;genetic-algorithms&gt;&lt;mutation-operators&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12020" PostTypeId="1" CreationDate="2019-04-26T12:36:13.383" Score="1" ViewCount="1110" Body="&lt;p&gt;I have an idea for a new type of AI for two-player games with alternating turns, like chess, checkers, connect four, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A little background: Traditionally engines for such games have used the minimax algorithm combined with a heuristic function when a certain depth has been reached to find the best moves. In recent days engines using reinforcement learning, etc (like AlphaZero in chess) have increased popularity, and become as strong as or stronger than the traditional minimax engines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach is to combine these ideas, to some level. A minimax tree with alpha-beta pruning will be used, but instead of considering every move in a position, these moves will be evaluated with a neural net or some other machine learning method, and the moves which seem least promising will not be considered further. The more interesting moves are expanded like in traditional minimax algorithms, and the same evaluation are again done for these nodes' children.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pros and cons are pretty obvious: By decreasing the breadth (number of moves in a position), the computation time will be reduced, which again can increase the search depth. The downside is that good moves may not be considered, if the machine learning method used to evaluate moves are not good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could of course hope that the position evaluation itself (from the neural net, etc) is good enough to pick the best move, so that no minimax is needed. However, combining the two approaches will hopefully make better results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A big motivation for this approach is that it resembles how humans act when playing games like chess. One tends to use intuition (which will be what the neural net represents in this approach) to find moves which looks interesting. Then one will look more thoroughly at these interesting moves by calculating moves ahead. However, one does not do this for all moves, only those which seem interesting. The idea is that a computer engine can play well by using the same approach, but can of course calculate much faster than a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate the performance gain: The size of a minimax tree is about b^d, where b is the average number of moves possible in each position, and d is the search depth. If the neural net can reduce the size of considered moves b to half, the new complexity will be (b/2)^d. If d is 20, that means reducing the computation time by approx. 1 million.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does anyone see any obvious flaws about this idea, which I might have missed?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Has it been attempted before? I have looked a bit around for information about this, but haven't found anything. Please give me some references if you know any articles about this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you think the performance of such a system could compete with those of pure minimax or those using deep reinforcement learning?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Exactly how the neural net will be trained, I have not determined yet, but there should be several options possible.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastActivityDate="2021-02-09T22:08:20.880" Title="Minimax combined with machine learning to determine if a path should be explored" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;minimax&gt;&lt;chess&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12021" PostTypeId="1" AcceptedAnswerId="12033" CreationDate="2019-04-26T13:30:05.243" Score="4" ViewCount="534" Body="&lt;p&gt;I want to create a simple game which basically consists of 2d circles shooting smaller circles at each other (to make hitbox detection easier for the start). My goal is to create an ai which adapts its own behaviour to the player‘s. For that, i want to use a NN as brain. Every frame, the NN is fed with the same inputs as the player and his output is compared to the players output. (outputs in this case are pressed keys like the up-arrow) &#xA;As inputs, I want to use a couple different important factors:&#xA;for example, the direction of the enemy player as number from 0 to 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to input the direction, size and speed of enemy’s and own projectiles and this is where my problem lies. If there was only one bullet per player, it would be easy but I want the number of bullets to be variable so the number of input neurons would have to be variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approaches:&#xA;1) use a big amount of neurons and set unused ones to 0&#xA;( not elegant at all)&#xA;2) Instead of specific values, just use all the pixels‘ rgb values as inputs (would limit the game as colours would deliver all the information) (+factors like speed and direction would probably not have any impact)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more promising approach to this problem ?&#xA;I hope you can give me some inspiration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a difference in ranging input values between 0/1 or -1/1 ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance, Mo&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: In case there aren‘t enough questions for you, is there a way to make the NN remember things ? For example, if I added a mechanic to the game which involves holding a key, I would add an input neuron which inputs 1 if the certain key is pressed and 0 if it isn‘t but I doubt that would work.&lt;/p&gt;&#xA;" OwnerUserId="25201" LastEditorUserId="25201" LastEditDate="2019-04-26T13:44:05.560" LastActivityDate="2019-04-28T15:22:34.793" Title="Neural Network with varying inputs (for a game ai)" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;neurons&gt;&lt;java&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12023" PostTypeId="1" CreationDate="2019-04-26T13:58:43.330" Score="0" ViewCount="110" Body="&lt;p&gt;Which libraries can be used for image caption generation?&lt;/p&gt;&#xA;" OwnerUserId="24076" LastEditorUserId="2444" LastEditDate="2019-04-26T16:23:45.617" LastActivityDate="2020-09-19T03:58:23.480" Title="Which libraries can be used for image caption generation?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;generative-model&gt;" AnswerCount="2" CommentCount="3" ClosedDate="2021-02-16T10:40:07.030" ContentLicense="CC BY-SA 4.0" />
  <row Id="12024" PostTypeId="1" CreationDate="2019-04-26T14:50:28.840" Score="1" ViewCount="59" Body="&lt;p&gt;Suppose we have a data set &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; that is split as &lt;span class=&quot;math-container&quot;&gt;$X_{\text{train}}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X_{\text{val}}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{\text{test}}$&lt;/span&gt; and the outcome variable is binary. Let's say we train three different models (logistic regression, random forest, and a support vector machine) using &lt;span class=&quot;math-container&quot;&gt;$X_{\text{train}}$&lt;/span&gt;. We then get predictions for &lt;span class=&quot;math-container&quot;&gt;$X_{\text{val}}$&lt;/span&gt; using each of the three models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://qr.ae/TWpjIz&quot; rel=&quot;nofollow noreferrer&quot;&gt;stacking&lt;/a&gt;, is it correct to say that we train a logistic regression model on a data set of dimension &lt;span class=&quot;math-container&quot;&gt;$|X_{\text{val}}| \times 3$&lt;/span&gt; with the predicted values and actual values of the validation set? This logistic regression model is then used to predict outcomes for data in &lt;span class=&quot;math-container&quot;&gt;$X_{\text{test}}$&lt;/span&gt;? &lt;/p&gt;&#xA;" OwnerUserId="25207" LastEditorUserId="2444" LastEditDate="2019-04-28T12:22:41.437" LastActivityDate="2021-05-17T18:08:50.893" Title="Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12025" PostTypeId="2" ParentId="12019" CreationDate="2019-04-26T21:08:50.317" Score="3" Body="&lt;p&gt;As @Oliver Mason says, picking the parameters that control the behavior of a GA (which are sometimes called &quot;hyperparameters&quot;) is historically more of an art than a science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evolutionary computation literature has many theories about the merits of high vs. low mutation, and high vs. low crossover. Most practitioners I have worked with use either high crossover, low mutation (e.g. Xover = 80%, mutation = 5%), or moderate crossover, moderate mutation (e.g. Xover = 40%, mutation = 40%). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more recent years, the field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyperparameter optimization&lt;/a&gt; has emerged and focuses on developing automatic approaches to picking these parameters. A very simple example of hyperparameter optimization is the &lt;a href=&quot;https://scikit-learn.org/stable/modules/grid_search.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GridSearchCV function&lt;/a&gt; in ScikitLearn. This systematically tries every combination of, say, 10 crossover values with evey one of 10 mutation values, and reports on which one works best. It uses &lt;a href=&quot;https://machinelearningmastery.com/k-fold-cross-validation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross Validation&lt;/a&gt; to prevent overfitting during this process. A more complex approach is &lt;a href=&quot;https://towardsdatascience.com/a-conceptual-explanation-of-bayesian-model-based-hyperparameter-optimization-for-machine-learning-b8172278050f&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Hyperparameter Optimization&lt;/a&gt;, which performs a sort of optimal experiment design to uncover the best values using as few tests as possible. This approach has been quite successful in tuning the hyperparamters of deep neural networks, for example.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-04-26T21:08:50.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12026" PostTypeId="1" CreationDate="2019-04-27T05:28:22.510" Score="1" ViewCount="249" Body="&lt;p&gt;I need help with ExpectiMinimax problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start a game.&lt;/li&gt;&#xA;&lt;li&gt;The first player flips a coin.&lt;/li&gt;&#xA;&lt;li&gt;The second player flips a coin.&lt;/li&gt;&#xA;&lt;li&gt;The first player decides if he wants to flip another coin.&lt;/li&gt;&#xA;&lt;li&gt;The second player decides if he wants to flip another coin.&lt;/li&gt;&#xA;&lt;li&gt;Game over.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The winner is the player who has earned more points.&#xA;The points are computed as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the player flips ones and got the head - 1 point.&lt;/li&gt;&#xA;&lt;li&gt;If the player flips ones and got the tail - 2 points.&lt;/li&gt;&#xA;&lt;li&gt;If the player flips twice and got 2 heads - 4 points.&lt;/li&gt;&#xA;&lt;li&gt;If the player flips twice and got 2 tails - 4 points.&lt;/li&gt;&#xA;&lt;li&gt;If the player flips twice and got one head and one tail - 0 points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I need to draw the ExpectiMinimax tree associated with this problem, and write the value of each node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I draw the tree properly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yncjr.png&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;ExpectiMinimax tree&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yncjr.png&quot; alt=&quot;ExpectiMinimax tree&quot; title=&quot;ExpectiMinimax tree&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25217" LastEditorUserId="16909" LastEditDate="2019-10-16T23:45:47.663" LastActivityDate="2021-03-10T04:03:40.303" Title="Is this ExpectiMinimax Tree correctly drawn?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;expectiminimax&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12027" PostTypeId="2" ParentId="7803" CreationDate="2019-04-27T06:39:56.280" Score="2" Body="&lt;p&gt;Deep Learning is not a generalization of Hopfield networks. Deep Learning is a &amp;quot;generalization&amp;quot; of the neural networks/connectionism field started by Rumelhart and McClelland.&lt;/p&gt;&#xA;&lt;p&gt;There are two kinds of neural networks:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Directed (Perceptron, MLP, ConvNets, RNNs, etc.)&lt;/li&gt;&#xA;&lt;li&gt;Undirected (Hopfield Nets, Boltzmann Machines, Energy-based models, etc.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any of these can be made deep. As you said, Boltzmann machines are the probabilistic version of Hopfield Networks, and there has been a lot more work on deepifying these models than Hopfield nets: Deep Boltzmann machines, Deep Belief Networks, and deep energy models. Hinton is really the guy you want to read to learn about these models, but you can have a look at &lt;a href=&quot;http://www.icml-2011.org/papers/557_icmlpaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; which compares the three models.&lt;/p&gt;&#xA;&lt;p&gt;Not sure about the Gestalt organisation. I guess I'll leave that to your interpretation.&lt;/p&gt;&#xA;" OwnerUserId="8637" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-04-27T06:39:56.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12028" PostTypeId="2" ParentId="11953" CreationDate="2019-04-27T11:11:03.260" Score="0" Body="&lt;p&gt;Totally agree with @John's answer. Will try and complement that with some more points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some advantages of SVMs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) SVM is defined by a convex optimisation problem for which there are &lt;a href=&quot;http://cs229.stanford.edu/notes/cs229-notes3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;efficient methods to solve, like SMO&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Effective in high dimensional spaces and also in cases where number of dimensions is greater than the number of samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;c) Uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d) &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Different Kernel functions can be specified for the decision function.&lt;/a&gt;. In its simplest form, the kernel trick means transforming data into another dimension that has a clear dividing margin between classes of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The disadvantages of support vector machines include:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) If the number of features is much greater than the number of samples, avoiding over-fitting in choosing Kernel functions and regularization term is crucial. Kernel models can be quite sensitive to &lt;a href=&quot;http://jmlr.csail.mit.edu/papers/volume11/cawley10a/cawley10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;over-fitting the model selection criterion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) SVMs do not directly provide probability estimates. In many classification problems you actually want the probability of class membership, so it would be better to use a method like Logistic Regression, rather than post-process the output of the SVM to get probabilities.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-09-28T14:35:26.720" LastActivityDate="2019-09-28T14:35:26.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12029" PostTypeId="1" CreationDate="2019-04-27T11:35:27.807" Score="4" ViewCount="5626" Body="&lt;p&gt;The AI must predict the next number in a given sequence of incremental integers (with no obvious pattern) using Python but so far I don't get the intended result!&#xA;I tried changing the learning rate and iterations but so far no luck!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example sequence: [1, 3, 7, 8, 21, 49, 76, 224]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expected result: 467&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Result found : 2,795.5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cost: 504579.43&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I've done so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;# Init sequence&#xA;data =\&#xA;    [&#xA;        [0, 1.0], [1, 3.0], [2, 7.0], [3, 8.0],&#xA;        [4, 21.0], [5, 49.0], [6, 76.0], [7, 224.0]&#xA;    ]&#xA;&#xA;X = np.matrix(data)[:, 0]&#xA;y = np.matrix(data)[:, 1]&#xA;&#xA;def J(X, y, theta):&#xA;    theta = np.matrix(theta).T&#xA;    m = len(y)&#xA;    predictions = X * theta&#xA;    sqError = np.power((predictions-y), [2])&#xA;    return 1/(2*m) * sum(sqError)&#xA;&#xA;dataX = np.matrix(data)[:, 0:1]&#xA;X = np.ones((len(dataX), 2))&#xA;X[:, 1:] = dataX&#xA;&#xA;# gradient descent function&#xA;def gradient(X, y, alpha, theta, iters):&#xA;    J_history = np.zeros(iters)&#xA;    m = len(y)&#xA;    theta = np.matrix(theta).T&#xA;    for i in range(iters):&#xA;        h0 = X * theta&#xA;        delta = (1 / m) * (X.T * h0 - X.T * y)&#xA;        theta = theta - alpha * delta&#xA;        J_history[i] = J(X, y, theta.T)&#xA;     return J_history, theta&#xA;print('\n'+40*'=')&#xA;&#xA;# Theta initialization&#xA;theta = np.matrix([np.random.random(), np.random.random()])&#xA;&#xA;# Learning rate&#xA;alpha = 0.02&#xA;&#xA;# Iterations&#xA;iters = 1000000&#xA;&#xA;print('\n== Model summary ==\nLearning rate: {}\nIterations: {}\nInitial &#xA;theta: {}\nInitial J: {:.2f}\n'&#xA;  .format(alpha, iters, theta, J(X, y, theta).item()))&#xA;print('Training model... ')&#xA;&#xA;# Train model and find optimal Theta value&#xA;J_history, theta_min = gradient(X, y, alpha, theta, iters)&#xA;print('Done, Model is trained')&#xA;print('\nModelled prediction function is:\ny = {:.2f} * x + {:.2f}'&#xA;  .format(theta_min[1].item(), theta_min[0].item()))&#xA;print('Cost is: {:.2f}'.format(J(X, y, theta_min.T).item()))&#xA;&#xA;# Calculate the predicted profit&#xA;def predict(pop):&#xA;    return [1, pop] * theta_min&#xA;&#xA;# Now&#xA;p = len(data)&#xA;print('\n'+40*'=')&#xA;print('Initial sequence was:\n', *np.array(data)[:, 1])&#xA;print('\nNext numbers should be: {:,.1f}'&#xA;  .format(predict(p).item()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; Another method I tried but still giving wrong results&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from sklearn import datasets, linear_model&#xA;&#xA;# Define the problem&#xA;problem = [1, 3, 7, 8, 21, 49, 76, 224]&#xA;&#xA;# create x and y for the problem&#xA;&#xA;x = []&#xA;y = []&#xA;&#xA;for (xi, yi) in enumerate(problem):&#xA;    x.append([xi])&#xA;    y.append(yi)&#xA;&#xA;x = np.array(x)&#xA;y = np.array(y)&#xA;# Create linear regression object&#xA;regr = linear_model.LinearRegression()&#xA;regr.fit(x, y)&#xA;&#xA;# create the testing set&#xA;x_test = [[i] for i in range(len(x), 3 + len(x))]&#xA;&#xA;# The coefficients&#xA;print('Coefficients: \n', regr.coef_)&#xA;# The mean squared error&#xA;print(&quot;Mean squared error: %.2f&quot; % np.mean((regr.predict(x) - y) ** 2))&#xA;# Explained variance score: 1 is perfect prediction&#xA;print('Variance score: %.2f' % regr.score(x, y))&#xA;&#xA;# Do predictions&#xA;y_predicted = regr.predict(x_test)&#xA;&#xA;print(&quot;Next few numbers in the series are&quot;)&#xA;for pred in y_predicted:&#xA;    print(pred)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3894" LastEditorUserId="3894" LastEditDate="2019-04-27T21:37:38.573" LastActivityDate="2019-05-09T10:43:17.573" Title="Use Machine Learning/Artificial Intelligence to predict next number (n+1) in a given sequence of random increasing integers" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;python&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12031" PostTypeId="2" ParentId="12029" CreationDate="2019-04-27T17:27:30.323" Score="3" Body="&lt;p&gt;I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like &lt;span class=&quot;math-container&quot;&gt;$y = 2x$&lt;/span&gt; or maybe affine function like &lt;span class=&quot;math-container&quot;&gt;$y = 2x + 5$&lt;/span&gt;. So you would have a sequence like:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$2, 4, 6, 8 ...$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$7, 9, 11, 13, ...$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you manage to get that working try a nonlinear function like &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; for example.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions &#xA;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal approximation theorem&lt;/a&gt;) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-04-27T17:27:30.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12032" PostTypeId="2" ParentId="12021" CreationDate="2019-04-27T19:11:02.067" Score="2" Body="&lt;p&gt;The most generic approach is to input all pixels as you have suggested. A CNN would be the best architecture for that. To provide information like speed or velocity, you can feed more than one frame to the CNN (e.g. the last 5 frames or whatever provides enough information). The CNN can learn movement information by comparing those images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to store additional information (like an inventory item), an input neuron for each value would be an option. You can also look up LSTM (long-short term memory) models, but for your specific situation a hardcoded neuron would be the easier solution.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastActivityDate="2019-04-27T19:11:02.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12033" PostTypeId="2" ParentId="12021" CreationDate="2019-04-28T03:12:19.397" Score="2" Body="&lt;p&gt;I recommend preprocessing images and feeding pixel values of several combined images. Some ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Preprocess all images to grayscale if possible. It’ll reduce the number of input neurons. (As long as this step doesn’t introduce large overhead)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Select some &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; value such that 0 &amp;lt; &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; &amp;lt; 1. Generate (ie. Select from your game) &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; sequential images. For the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th image in the sequence, multiply every pixel value by &lt;span class=&quot;math-container&quot;&gt;$\gamma^{n-k-1}$&lt;/span&gt;. This assumes we index &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; starting at zero. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sum the pixel values of all processed images with clip ~ [0, 255] (for a valid range of values)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This will yield a single image where stationary pixels will be summed to create brighter / more saturated spots, where moving objects will have “shadows” or “tails” which are faded with each time step (&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the “fading factor” so to speak). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Image input:&lt;/em&gt; As long as all values are on a comparable scale, it shouldn’t really matter whether inputs are on range [-1, 1] or [0, 1], but since you’ll be using pixel values, they will all be positive. So normalizing the pixel values will yield a range [0, 1].  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; this kind of processing can probably be done iteratively with greater efficiency by summing, then multiplying by gamma at each time step. Then you can implement it online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider what you want the OUTPUT of the network to be. If you want the agent to take an action after processing the inputs, your output later should consist of one neuron per discrete action (ie, each “button” that can be pushed). I will limit my answer to discrete actions since that is likely the most useful answer for this question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you asked about if the network can “remember things,” like “holding down a key.”  This question is a bit vague, but let me try to answer. It sounds like you were considering using this as an INPUT to the network. In theory, you could use a similar implementation (ie. At every time step measure if the button was pressed. Perhaps use 1 if pressed 0 otherwise. Decay at every time step and sum. With n time steps, the sum will have a max value of &lt;span class=&quot;math-container&quot;&gt;$\sum_{n}(\gamma^{n-1})$&lt;/span&gt;). Remember to decay by &lt;span class=&quot;math-container&quot;&gt;$n-k-1$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; starting at zero. You  don’t have to actually decay this value, but decaying by a factor of gamma helps the network know if for instance the button was pressed near the first frame or the last frame. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, I don’t know if you want to use this as an input. If the AI is meant to have more information than the opponent, than I suppose you can. But then the agent will not be learning from the same information as the opponent. Also, if holding the button produces a clearly visible effect, that information would be captured in the images already, so might be a redundant input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These ideas are not the only implementation, but can get you going. It sounds like you’ll need a measure of reward and likely need to structure this as an RL problem. The details of that are beyond the scope of this post and I don’t want to get too afield of your original question. Just note that comparing to the players output may not give you the results you want, and even if it did, your network will be limited to learning only to mimic the other player. Using a measure of reward will allow your agent to theoretically advance beyond the skill of its opponent by taking actions that maximize reward, even if the opponent would not have thought to take that action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2019-04-28T15:22:34.793" LastActivityDate="2019-04-28T15:22:34.793" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12034" PostTypeId="1" CreationDate="2019-04-28T03:47:35.933" Score="7" ViewCount="66374" Body="&lt;p&gt;These guys here: &lt;a href=&quot;https://www.patreon.com/AiAngel&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.patreon.com/AiAngel&lt;/a&gt; are saying that they've created a AI who can chat and stream. As the so-called administrator &quot;Rogue&quot; said:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MEGez.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MEGez.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this chat/streamer bot are no fake. &#xA;Also, there's more about the dynamics of this chat/streamer bot on youtube:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=WyFwjHQhlgo&amp;amp;t=463s&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=WyFwjHQhlgo&amp;amp;t=463s&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=GtvivssqLhE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=GtvivssqLhE&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering that videos I realy think that this bot is totaly fake. I mean, I think that even the most advanced AI bot do not get even close to a real conversation like this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, of course that you can say that this is a artistic project or something, but the people behind all of this  are on Patreon, and the people who are paying to these guys possibly are getting totally fooled, which is a serious thing when we're talking about real money. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is AIAngel a real bot? (With this question I'm spreading this possible fake to community)  &lt;/p&gt;&#xA;" OwnerUserId="25232" LastEditorUserId="1671" LastEditDate="2019-04-29T18:16:15.597" LastActivityDate="2020-04-16T19:32:39.150" Title="Is &quot;AIAngel&quot; (Patreon) a fake?" Tags="&lt;chat-bots&gt;&lt;turing-test&gt;&lt;ai-hoaxes&gt;" AnswerCount="4" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12035" PostTypeId="2" ParentId="11953" CreationDate="2019-04-28T05:31:09.047" Score="-1" Body="&lt;p&gt;For datasets of low-dimensional tabular data. DNN are not efficient on low-dimensional input because of huge overparametrisation. So even if dataset is huge in size, but each sample is low-dimensional SVM would beat DNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally if data is tabular and the correlation between fields of the sample is weak and noisy, SVM may still beat DNN even for high dimensional data, but that depend on specific of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I can't recall any specific papers on subject, so it's mostly common sense reasoning, you don't have to trust it. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-04-28T05:31:09.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12037" PostTypeId="1" CreationDate="2019-04-28T10:41:00.447" Score="3" ViewCount="31" Body="&lt;p&gt;Anyone knows a resources (papers, articles and especially repositories) regarding controlling multiple units with RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The controlled units should not be fixed, for example in Real Time Strategy the agent builds various units (workers, soldiers ...) and later controls them. During the game various units could die and new ones are built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think good contemporary example is AlphaStar, while OpenAI Five controls just a single agent. This might be incorrect since I've never played those games.&lt;/p&gt;&#xA;" OwnerUserId="25236" LastActivityDate="2019-04-28T10:41:00.447" Title="Code examples of controlling multiple units with RL" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12038" PostTypeId="2" ParentId="12011" CreationDate="2019-04-28T12:46:53.917" Score="4" Body="&lt;p&gt;In machine learning, the idea behind &lt;a href=&quot;https://en.wikipedia.org/wiki/Hebbian_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hebbian learning&lt;/a&gt; is to strengthen (or weaken) the connection (the weight) between the neurons that have similar (or, respectively, dissimilar) outputs, where &quot;similar&quot; can be defined in different ways (e.g. it could be based on the sign of the output of the neurons). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hebbian learning is a more biologically plausible way of learning than back-propagation, because it is a &quot;local learning strategy&quot; (you locally update the connections and not all the connections of the model at the same time), as opposed to back-propagation, which is a &quot;global learning strategy&quot; (where all connections are usually updated at once, given a &quot;global error&quot; of the network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several neural networks (or models) that can learn in a Hebbian fashion: for example, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hopfield_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hopfield network&lt;/a&gt; or &lt;a href=&quot;https://numenta.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numenta&lt;/a&gt;'s &lt;em&gt;temporal memory&lt;/em&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-28T12:57:24.533" LastActivityDate="2019-04-28T12:57:24.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12039" PostTypeId="2" ParentId="11976" CreationDate="2019-04-28T12:52:55.327" Score="2" Body="&lt;p&gt;You can find a lot of content regarding the mathematical definitions. Let's take it simpler:&#xA;It is a universal function approximator.&#xA;Think about this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5Midf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Midf.png&quot; alt=&quot;Linear Regression&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can approximate this locus with a line, two parameters: slope and offset.&#xA;That's the simplest you can do, with a straight line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, Think about this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K6ouY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K6ouY.png&quot; alt=&quot;3D Regression&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You approximated it with planes and curvatures, didn't you?&#xA;That's a little gist of what Neural Networks do. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They approximate functions with given I/O.&lt;/li&gt;&#xA;&lt;li&gt;They find patterns.&lt;/li&gt;&#xA;&lt;li&gt;They find behaviours in data (image, text, speech, numbers).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We think it is artificial intelligence, and it has worked out pretty well so far.&lt;/p&gt;&#xA;" OwnerUserId="24122" LastActivityDate="2019-04-28T12:52:55.327" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12040" PostTypeId="2" ParentId="11937" CreationDate="2019-04-28T13:13:01.497" Score="0" Body="&lt;p&gt;All weight matrices in a Neural Network, adapt to map input and output. ReLU, as you pointed out, doesn't give negative derivatives. You're right. But notice the weight update equation in Backpropagation, it uses a multitude of parameters like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Error value&lt;/li&gt;&#xA;&lt;li&gt;Activation from the previous layer&lt;/li&gt;&#xA;&lt;li&gt;Weight matrix of the previous layer&lt;/li&gt;&#xA;&lt;li&gt;The pre-activation cache of the previous layer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These can be positive or negative. Hence weight updations dW and db can be +ve or -ve, allowing for decreasing (or increasing) value of weights.&lt;/p&gt;&#xA;" OwnerUserId="24122" LastActivityDate="2019-04-28T13:13:01.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12041" PostTypeId="2" ParentId="12024" CreationDate="2019-04-28T13:56:17.307" Score="0" Body="&lt;p&gt;The predictions of each of your initial models will become a feature to feed the meta learner. If you use &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; initial models, then for each example you will feed the meta learner &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; features, each feature being a prediction from one of the initial models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this doesn’t mean the size of your dataset increases. Instead, each member of &lt;span class=&quot;math-container&quot;&gt;$X_{val}$&lt;/span&gt; will be represented by &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; features, with the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th feature being equal to the initial prediction of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th initial predictor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.medium.com/nHYrgZmXfW&quot; rel=&quot;nofollow noreferrer&quot;&gt;This medium link&lt;/a&gt; recommends a package that can help you build a pipeline. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: in a sense if you are caching the intermediate predictions of each initial predictor, you’ll end up with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times as many data points as there are entries in &lt;span class=&quot;math-container&quot;&gt;$X_{val}$&lt;/span&gt;. But these are simply each going to be a feature of another data point, so the dataset hasn’t really increased in size. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2019-04-28T15:02:15.880" LastActivityDate="2019-04-28T15:02:15.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12042" PostTypeId="1" AcceptedAnswerId="12047" CreationDate="2019-04-28T16:55:21.243" Score="12" ViewCount="2403" Body="&lt;p&gt;Surprisingly, this wasn't asked before - at least I didn't find anything besides some &lt;a href=&quot;https://ai.stackexchange.com/q/11717/2444&quot;&gt;vaguely related questions.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what is a recurrent neural network, and what are their advantages over regular (or feed-forward) neural networks?&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2020-06-02T11:38:59.227" LastActivityDate="2020-06-02T11:38:59.227" Title="What is a recurrent neural network?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;definitions&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12043" PostTypeId="2" ParentId="12042" CreationDate="2019-04-28T17:39:11.357" Score="9" Body="&lt;p&gt;A &lt;em&gt;recurrent neural network&lt;/em&gt; (RNN) is an &lt;a href=&quot;https://ai.stackexchange.com/a/11978/2444&quot;&gt;artificial neural network&lt;/a&gt; that contains backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network (FFNN). The adjective &quot;recurrent&quot; thus refers to this backward or self-connections, which create loops in these networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An RNN can be trained using &lt;em&gt;back-propagation through time&lt;/em&gt; (BBTT), such that these backward or self-connections &quot;memorise&quot; previously seen inputs. Hence, these connections are mainly used to track temporal relations between elements of a sequence of inputs, which makes RNNs well suited to sequence prediction and similar tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several RNN models: for example, RNNs with LSTM or GRU units. LSTM (or GRU) is an RNN whose single units perform a more complex transformation than a unit in a &quot;plain RNN&quot;, which performs a linear transformation of the input followed by the application of a non-linear function (e.g. ReLU) to this linear transformation. In theory, &quot;plain RNN&quot; are as powerful as RNNs with LSTM units. In practice, they suffer from the &quot;vanishing and exploding gradients&quot; problem. Hence, in practice, LSTMs (or similar sophisticated recurrent units) are used.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-04-28T17:39:11.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12044" PostTypeId="1" CreationDate="2019-04-29T02:51:14.563" Score="3" ViewCount="48" Body="&lt;p&gt;I am looking for good introductory and advanced books on unsupervised learning. I have already read books like &lt;em&gt;Probabilistic Graphical Models&lt;/em&gt; from D. Kholler and &lt;em&gt;Pattern Recognition and Machine Learning&lt;/em&gt; from C. M. Bishop. I am also very familiar with the Ph.D. thesis of K. P. Murphy, on Dynamic Bayesian Networks.&lt;/p&gt;&#xA;&lt;p&gt;I have read all of the above mostly for the probability aspects, not really the applications to AI and ML. I would like to know what are the good books (or references) for unsupervised learning, that focuses on practical exercises and examples instead of deep and abstract concepts.&lt;/p&gt;&#xA;" OwnerUserId="25262" LastEditorUserId="2444" LastEditDate="2021-01-17T19:31:01.160" LastActivityDate="2021-01-17T19:31:01.160" Title="What are examples of good reference books on unsupervised learning?" Tags="&lt;reference-request&gt;&lt;unsupervised-learning&gt;&lt;books&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12046" PostTypeId="1" AcceptedAnswerId="12060" CreationDate="2019-04-29T07:03:44.927" Score="1" ViewCount="461" Body="&lt;p&gt;I recently watched a YouTube video (sorry, can't remember the link) where (a very talented) someone created what they called a &lt;em&gt;&quot;static AI&quot;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Somewhere in the video they said something along the lines of:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;&quot;this is a static AI, it's very simple and not dynamic at all&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does this mean? What's the difference between a static AI and a dynamic AI?&lt;/p&gt;&#xA;" OwnerUserId="25274" LastEditorUserId="1671" LastEditDate="2019-10-15T19:24:28.503" LastActivityDate="2019-10-15T19:24:28.503" Title="What's the difference between a static AI and a dynamic AI?" Tags="&lt;terminology&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12047" PostTypeId="2" ParentId="12042" CreationDate="2019-04-29T07:12:07.653" Score="6" Body="&lt;p&gt;Recurrent neural networks (RNNs) are a class of artificial neural network&#xA;architecture inspired by the cyclical connectivity of neurons in the brain. It uses iterative function loops to store information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Difference with traditional Neural networks using pictures from &lt;a href=&quot;https://www.cs.toronto.edu/~graves/preprint.pdf&quot; rel=&quot;noreferrer&quot;&gt;this book&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ULBqq.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ULBqq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, an RNN:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qobNg.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qobNg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the difference -- feedforward neural networks' connections&#xA;do not form cycles. If we relax this condition, and allow cyclical&#xA;connections as well, we obtain recurrent neural networks (RNNs). You can see that in the hidden layer of the architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While the difference between a multilayer perceptron and an RNN may seem&#xA;trivial, the implications for sequence learning are far-reaching. An MLP can only&#xA;map from &lt;strong&gt;input to output vectors&lt;/strong&gt;, whereas an RNN can in principle map from&#xA;the &lt;strong&gt;entire history of previous inputs to each output&lt;/strong&gt;. Indeed, the equivalent&#xA;result to the universal approximation theory for MLPs is that an RNN with a&#xA;sufficient number of hidden units can approximate any measurable sequence-to-sequence&#xA;mapping to arbitrary accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Important takeaway:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The recurrent connections allow a 'memory' of previous inputs to persist in the&#xA;network's internal state, and thereby influence the network output.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Talking in terms of advantages is not appropriate as they both are state-of-the-art and are particularly good at certain tasks. A broad category of tasks that RNN excel at is:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Sequence Labelling&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The goal of sequence labelling is to assign sequences of labels, drawn from a fixed alphabet, to sequences of input data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex: Transcribe a sequence of acoustic features with spoken words (speech recognition), or a sequence of video frames with hand gestures (gesture recognition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the sub-tasks in sequence labelling are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sequence Classification&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Label sequences are constrained to be of length one. This is referred to as sequence classification, since each input sequence is assigned to a single class. Examples of sequence classification task include the identification of a single spoken work and the recognition of an individual&#xA;handwritten letter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Segment Classification&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Segment classification refers to those tasks where the target sequences consist&#xA;of multiple labels, but the locations of the labels -- that is, the positions of the input segments to which the labels apply -- are known in advance.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-04-29T13:54:41.947" LastActivityDate="2019-04-29T13:54:41.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12048" PostTypeId="1" CreationDate="2019-04-29T08:07:22.677" Score="4" ViewCount="75" Body="&lt;p&gt;In Section 4.3 of paper &lt;a href=&quot;https://arxiv.org/abs/1802.10567&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning by Playing - Solving Sparse Reward Tasks from Scratch&lt;/a&gt;, the authors define Retrace as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q^{ret}=\sum_{j=i}^\infty\left(\gamma^{j-i}\prod_{k=i}^jc_k\right)[r(s_j,a_j)+\delta_Q(s_i,s_j)],\\&#xA;\delta_Q(s_i,s_j)=\mathbb E_{\pi_{\theta'}(a|s)}[Q^\pi(s_i,\cdot;\phi')]-Q^\pi(s_j,a_j;\phi')\\&#xA;c_k=\min\left(1,{\pi_{\theta'}(a_k|s_k)\over b(a_k|s_k)}\right)&#xA;$$&lt;/span&gt;&#xA;where I omit &lt;span class=&quot;math-container&quot;&gt;$\mathcal T$&lt;/span&gt; for simplicity.&#xA;I'm quite confused about the definition of &lt;span class=&quot;math-container&quot;&gt;$Q^{ret}$&lt;/span&gt;, which seems not consistent with Retrace define in &lt;a href=&quot;https://arxiv.org/abs/1606.02647&quot; rel=&quot;nofollow noreferrer&quot;&gt;Safe and efﬁcient off-policy reinforcement learning&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal RQ(x,a):=Q(x,a)+\mathbb E_\mu[\sum_{t\ge0}\gamma^t\left(\prod_{s=1}^tc_s\right)(r_t+\gamma\mathbb E_\pi Q(x_{t+1},\cdot)-Q(x_t,a_t)]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I make of &lt;span class=&quot;math-container&quot;&gt;$Q^{ret}$&lt;/span&gt; in the first paper?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="9947" LastEditDate="2020-04-11T17:01:35.510" LastActivityDate="2020-04-11T17:01:35.510" Title="Why is there inconsistency in the definitions of the retrace?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12049" PostTypeId="1" CreationDate="2019-04-29T12:02:38.847" Score="1" ViewCount="313" Body="&lt;p&gt;I need to train an autoencoder in Keras with the JPG images I took myself.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;&#xA;#1st convolution layer&#xA;model.add(Conv2D(16, (3, 3), padding='same', data_format='channels_first', input_shape=(3,224,224)))&#xA;model.add(Activation('relu'))&#xA;model.add(MaxPooling2D(pool_size=(2,2), padding='same'))&#xA;&#xA;#2nd convolution layer&#xA;model.add(Conv2D(2,(3, 3), padding='same'))&#xA;model.add(Activation('relu'))&#xA;model.add(MaxPooling2D(pool_size=(2,2), padding='same'))&#xA;&#xA;#-------------------------&#xA;&#xA;#3rd convolution layer&#xA;model.add(Conv2D(2,(3, 3), padding='same'))&#xA;model.add(Activation('relu'))&#xA;model.add(UpSampling2D((2, 2)))&#xA;&#xA;#4rd convolution layer&#xA;model.add(Conv2D(16,(3, 3), padding='same'))&#xA;model.add(Activation('relu'))&#xA;model.add(UpSampling2D((2, 2)))&#xA;&#xA;#-------------------------&#xA;&#xA;model.add(Conv2D(1,(3, 3), padding='same'))&#xA;model.add(Activation('sigmoid'))&#xA;&#xA;model.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which generates  a model as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;conv2d_1 (Conv2D)            (None, 16, 224, 224)      448       &#xA;_________________________________________________________________&#xA;activation_1 (Activation)    (None, 16, 224, 224)      0         &#xA;_________________________________________________________________&#xA;max_pooling2d_1 (MaxPooling2 (None, 16, 112, 112)      0         &#xA;_________________________________________________________________&#xA;conv2d_2 (Conv2D)            (None, 2, 112, 112)       290       &#xA;_________________________________________________________________&#xA;activation_2 (Activation)    (None, 2, 112, 112)       0         &#xA;_________________________________________________________________&#xA;max_pooling2d_2 (MaxPooling2 (None, 2, 56, 56)         0         &#xA;_________________________________________________________________&#xA;conv2d_3 (Conv2D)            (None, 2, 56, 56)         38        &#xA;_________________________________________________________________&#xA;activation_3 (Activation)    (None, 2, 56, 56)         0         &#xA;_________________________________________________________________&#xA;up_sampling2d_1 (UpSampling2 (None, 2, 112, 112)       0         &#xA;_________________________________________________________________&#xA;conv2d_4 (Conv2D)            (None, 16, 112, 112)      304       &#xA;_________________________________________________________________&#xA;activation_4 (Activation)    (None, 16, 112, 112)      0         &#xA;_________________________________________________________________&#xA;up_sampling2d_2 (UpSampling2 (None, 16, 224, 224)      0         &#xA;_________________________________________________________________&#xA;conv2d_5 (Conv2D)            (None, 1, 224, 224)       145       &#xA;_________________________________________________________________&#xA;activation_5 (Activation)    (None, 1, 224, 224)       0         &#xA;=================================================================&#xA;Total params: 1,225&#xA;Trainable params: 1,225&#xA;Non-trainable params: 0&#xA;_________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I compile and train as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(optimizer='adadelta', loss='binary_crossentropy')&#xA;batch_size = 16 #16&#xA;&#xA;train_datagen = ImageDataGenerator(rescale=1./255)&#xA;train_generator = train_datagen.flow_from_directory(&#xA;    'cropped/',&#xA;    target_size=(224, 224),&#xA;    batch_size=batch_size,&#xA;    class_mode='categorical'&#xA;    )&#xA;test_datagen = ImageDataGenerator(rescale=1./255)&#xA;validation_generator = test_datagen.flow_from_directory(&#xA;    'cropped/',&#xA;    target_size=(224, 224),&#xA;    batch_size=batch_size,&#xA;    class_mode='categorical'&#xA;    )&#xA;&#xA;model.fit_generator(&#xA;        train_generator,&#xA;        steps_per_epoch=1000,&#xA;        epochs=20,&#xA;        validation_data=validation_generator,&#xA;        validation_steps=1000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I end up with the error message:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ValueError: Error when checking target: expected activation_5 to have&#xA;  4 dimensions, but got array with shape (16, 2)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Should I use &lt;code&gt;Conv2D&lt;/code&gt; instead?&lt;/p&gt;&#xA;" OwnerUserId="25211" LastActivityDate="2019-04-29T12:40:57.330" Title="Autoencoder for color images in Keras backed by MXNet" Tags="&lt;keras&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12051" PostTypeId="2" ParentId="12049" CreationDate="2019-04-29T12:40:57.330" Score="1" Body="&lt;p&gt;It was not about the model, but about ImageDataGenerator. I found the answer here: &lt;a href=&quot;https://stackoverflow.com/a/51673998/493080&quot;&gt;https://stackoverflow.com/a/51673998/493080&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25211" LastActivityDate="2019-04-29T12:40:57.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12052" PostTypeId="1" CreationDate="2019-04-29T13:20:11.437" Score="1" ViewCount="22" Body="&lt;p&gt;I'm doing a model to detect duplicate in my database (there is a lot of features that can be different but mean same object in the end)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have my feature vector for my duplicate dataset which contains score, distance and relation between 2 identical object that I labelised&#xA;such as (0, 0, 123, 14000, 5, 10, 0, 0, -1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since duplicate are a rare event I was wondering what size of dataset should I use for non duplicate features, since I want my model to learn about the disparity of the multiple features I have I though I should have like 10 times the number of example of non-duplicate and in my model change the weight of duplicate class by multiplying by 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that a good thing to do or is it better to take 50%/50% of duplicate/non duplicate features for my model ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, should I apply filter to chose my non-duplicate dataset in order to have like close object on some features but different on other. Or should I take them randomly among all the data I have ?&lt;/p&gt;&#xA;" OwnerUserId="23107" LastActivityDate="2019-04-29T13:20:11.437" Title="Size of dataset for feature vector with rare event" Tags="&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12053" PostTypeId="1" CreationDate="2019-04-29T15:30:20.570" Score="3" ViewCount="116" Body="&lt;p&gt;I have recently started working on a control problem using a Deep Q Network as proposed by DeepMind (&lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1312.5602&lt;/a&gt;). Initially, I implemented it without Experience Replay. The results were very satisfying, although after implementing ER, the results I got were relatively bad. Thus I started experimenting with BATCH SIZE and MEMORY CAPACITY.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;(1) I noticed that if I set BATCH SIZE = 1 and MEMORY CAPACITY = 1 i.e. the same as doing normal online learning as previously, the results are then (almost) the same as initially. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(2) If I increased CAPACITY and BATCH SIZE e.g. CAPACITY = 2000 and BATCH SIZE = 128, the Q Values for all actions tend to converge to very similar negative values.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A small negative reward -1 is received for every state transition except of the desired state which receives +10 reward. My gamma is 0.7. Every state is discrete and the environment can transition to a number of X states after action a, with every state in X having a significant probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Receiving a positive reward is very rare as getting to a desired state can take a long time. Thus, when sampling 128 experiences if 'lucky' only a small amount of experiences may have a positive reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, when doing mini-batch training we average the loss over all the samples and then update the DQN I was wondering whether generally the positive rewards can become meaningless as they are 'dominated' by the negative ones. Which means that this would result in a very slower convergence to actual values ? And also justifies the the convergence to similar negative values as in (2) ? Is this something expected? I am looking to implement. Prioritised ER as a  potential solution to this, but is there something wrong inn the above logic? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this does makes sense. Please forgive me if I make a wrong assumption above as I am new to the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: The problem seemed to be that indeed finding rewards very rarely would result in sampling almost never, especially at the begging of training, which in turn resulted in very slow convergence to the actual Q values. The problem was successfully solved using Prioritised ER -but I believe any form of careful Stratified Sampling would result in good results&lt;/p&gt;&#xA;" OwnerUserId="25288" LastEditorUserId="25288" LastEditDate="2019-05-01T20:05:52.423" LastActivityDate="2019-05-31T21:03:15.380" Title="Experience Replay Not Always Giving Better Results" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="12054" PostTypeId="2" ParentId="12053" CreationDate="2019-04-29T17:22:26.527" Score="1" Body="&lt;p&gt;What you describe sounds to me like a problem inherent to off policy learning, and what you describe seems to me to be a reasonable interpretation of what may be happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you implemented experience replay with capacity = 1 and batch_size = 1 you said you got “almost” the same results as before. There are probably two reasons for this being “almost” the same. One is simply the random initializations of the networks, so as you train, you will potentially converge around the same point but not exactly to it (also the stochastic nature of generating the training samples).  The other reason might be to do with what has already occurred each time you update the target net weights, so your error terms may differ slightly at each point in time, but asymptotically converge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is essentially following an on policy training, so every training sample is following a trajectory (they are all sequential states). Eventually, given enough time, this trajectory will reach a goal state and be rewarded. The reward will be propagated through the network and effect the backed up values of other states will be updated as well. So essentially each episode ends with a reward (I presume), and the average reward given per episode is proportional to the average length of an episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When increasing capacity and batch size &gt; 1, we move to true off policy training. When sampling, the updates are not following a trajectory. And as such, there is no guarantee that we EVER sample any positive reward (although, I’m sure you will at some point). So, if we are averaging over the rewards in the updates, the average reward given per “episode” is no longer proportional to the average episode length (and the idea of episode starts to lose some of its relevance - since we aren’t following a trajectory). This the effect of the reward on all other states is not in proportion to what it was when following an on-policy trajectory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try some hacks to investigate, for example try making your positive reward &gt; batch_size. Or, if you have some statistics on how often your goal state is being sampled, perhaps scale it up by that factor. (Or if you know the size of your state space, make your reward greater than that).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.freecodecamp.org/improvements-in-deep-q-learning-dueling-double-dqn-prioritized-experience-replay-and-fixed-58b130cc5682&quot; rel=&quot;nofollow noreferrer&quot;&gt;This blog post&lt;/a&gt; offers some more elegant refinements, like prioritized ER which you mentioned. But it would be interesting to see the effect of scaling up your reward will overcome the effect of averaging over many negative rewards. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2019-04-29T19:08:26.440" LastActivityDate="2019-04-29T19:08:26.440" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12055" PostTypeId="2" ParentId="12034" CreationDate="2019-04-29T18:06:40.973" Score="2" Body="&lt;p&gt;Without being able to interact with the bot, a Turing test is impossible.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Based on the manner in which this supposed bot is presented, in videos as opposed to an interactive medium where users can interact with the bot, &lt;strong&gt;the only reasonable conclusion is hoax.&lt;/strong&gt;  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This assessment is supported by the overly sexualized rendering of the bot combined with requests for donations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In order for this project to be considered legitimate, the creators would have to be more transparent about the methods and allow the general public, or at least reliable experts, to interact with the bot.&lt;/strong&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Compare to IBM's &lt;a href=&quot;https://en.wikipedia.org/wiki/Tay_(bot)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tay&lt;/a&gt; and &lt;a href=&quot;https://www.zo.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zo&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Turing Tests &amp;amp; Pornbots&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nature of Turing tests is that they are subjective.  A bot that an adult human could easily recognize as non-human, a child might perceive as human--the child does not have the requisite knowledge to form a strategy to expose the bot as an automaton.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an idea that &quot;pornbots&quot; have been passing the Turing Test for many years now, predicated on the hormonal imperatives of those interacting with the bots, which greatly inhibits their judgement.  (The purpose of pornbots is to get consumers to spend money.  Their continued existence suggests they do have utility in this regard, and that utility can be seen as a confirmation of the bots passing the text in a specific context.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Automation Hoaxes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the most famous machine intelligence hoaxes was &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Turk&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Turk&lt;/a&gt;, presented as a Chess playing automaton.  The supposed machine astounded late 18th century audiences with its skill, but it was later revealed to be a human in a box.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-04-29T18:06:40.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12056" PostTypeId="5" CreationDate="2019-04-29T18:15:16.797" Score="0" Body="&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Turk&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Turk&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Also: &lt;a href=&quot;https://io9.gizmodo.com/historys-greatest-robot-hoaxes-5053731&quot; rel=&quot;nofollow noreferrer&quot;&gt;History's Greatest Robot Hoaxes (io9)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-29T18:15:16.797" LastActivityDate="2019-04-29T18:15:16.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12057" PostTypeId="4" CreationDate="2019-04-29T18:15:16.797" Score="0" Body="Use for questions related to claims or presentations related to AI capabilities that may be hoaxes." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-04-29T18:15:16.797" LastActivityDate="2019-04-29T18:15:16.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12059" PostTypeId="1" CreationDate="2019-04-30T01:15:00.733" Score="4" ViewCount="222" Body="&lt;p&gt;There are multiple ways to implement parallelism in reinforcement learning. One is to use parallel workers running in their own environments to collect data in parallel, instead of using replay memory buffers (this is how A3C works, for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are methods, like PPO, that use batch training on purpose. How is parallelism usually implemented for algorithms that still use batch training?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are gradients accumulated over parallel workers and the combined? Is there another way? What are the benefits of doing parallelism one way over another?&lt;/p&gt;&#xA;" OwnerUserId="25294" LastEditorUserId="2444" LastEditDate="2020-02-20T12:08:15.313" LastActivityDate="2020-02-20T20:06:12.830" Title="How is parallelism implemented in RL algorithms like PPO?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;implementation&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12060" PostTypeId="2" ParentId="12046" CreationDate="2019-04-30T02:04:48.647" Score="3" Body="&lt;p&gt;My sense is they're probably talking about the difference between symbolic AI (aka &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOFAI&lt;/a&gt;)  and learning algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GOFAI typically uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;heuristics&lt;/a&gt;, which are static—essentially fixed rules governing decision making.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statistical AI, aka &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt;, is dynamic and can analyze and evaluate the environment to form its own decision rules.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes a trained NN that is no longer learning is said to have created it's own heuristics.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-04-30T02:04:48.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12062" PostTypeId="1" CreationDate="2019-04-30T08:53:34.283" Score="1" ViewCount="34" Body="&lt;p&gt;I'd like to measure the difference between 2 grid-worlds to determine the generalization capacity of my agent using tabular Q-learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example (OpenAI Frozen Lake) :&lt;br&gt;&#xA;SFFF&lt;br&gt;&#xA;FHFH&lt;br&gt;&#xA;FFFH&lt;br&gt;&#xA;HFFG  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and  :&lt;br&gt;&#xA;SFFG&lt;br&gt;&#xA;FHFH&lt;br&gt;&#xA;FFFH&lt;br&gt;&#xA;HFFH&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are not very different but the tabular policy that I found on the first environement will completely fail on the new environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A correct distance should to measure the policy on the first environment and compute the norm between this one and the optimal policy (not found with RL) of the new environment. Is it accurate ?&#xA;I think this is a bit strange because measure the difference between 2 policies is the intrinsic answer.. How can I measure accurately 2 environments, not forgetting the transition matrix of the environment ?&lt;/p&gt;&#xA;" OwnerUserId="23838" LastEditorUserId="23838" LastEditDate="2019-04-30T09:09:03.100" LastActivityDate="2019-04-30T09:09:03.100" Title="Measure grid-world environments difference for reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;policies&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12065" PostTypeId="1" AcceptedAnswerId="12067" CreationDate="2019-04-30T14:26:46.283" Score="9" ViewCount="1153" Body="&lt;p&gt;What are the areas/algorithms that belong to reinforcement learning?&lt;/p&gt;&#xA;&lt;p&gt;TD(0), Q-Learning and SARSA are all temporal-difference algorithms, which belong to the reinforcement learning area, but is there more to it?&lt;/p&gt;&#xA;&lt;p&gt;Are the dynamic programming algorithms, such as policy iteration and value iteration, considered as part of reinforcement learning? Or are these just the basis for the temporal-difference algorithms, which are the only RL algorithms?&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="2444" LastEditDate="2021-03-11T10:06:16.217" LastActivityDate="2021-03-11T10:06:16.217" Title="What algorithms are considered reinforcement learning algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;temporal-difference-methods&gt;&lt;planning&gt;&lt;dynamic-programming&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12066" PostTypeId="1" AcceptedAnswerId="12102" CreationDate="2019-04-30T14:51:25.057" Score="3" ViewCount="87" Body="&lt;p&gt;&lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=16&quot; rel=&quot;nofollow noreferrer&quot;&gt;Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1&lt;/a&gt; calculates the standard deviation of the output of a hidden neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma^2 = \sum^n_{i=0}E[w_i^2]E[x_i^2]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try what I get is (&lt;a href=&quot;https://ai.stackexchange.com/questions/11971/binary-vector-expected-value&quot;&gt;with &lt;span class=&quot;math-container&quot;&gt;$E[x_i^2] = \frac{1}{3}$&lt;/span&gt;&lt;/a&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_i \in [-\alpha, \alpha]$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma^2 = \sum^n_{i=0}E[w_i^2]E[x_i^2]&#xA;= [n\frac{(\alpha-(-\alpha))^2}{12}][n\frac{1}{3}]=n^2\alpha^2\frac{1}{9}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma = \sqrt{n^2\alpha^2\frac{1}{9}}=n\alpha\frac{1}{3}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But Raul Rojas concludes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma = \frac{1}{3}\sqrt{n}\alpha&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing some implicance of the law of large numbers use for the input to the node?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time :)&lt;/p&gt;&#xA;" OwnerUserId="14892" LastEditorUserId="1671" LastEditDate="2019-05-02T00:18:40.127" LastActivityDate="2019-05-18T07:53:32.840" Title="Standard deviation of the total input to a neuron" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12067" PostTypeId="2" ParentId="12065" CreationDate="2019-04-30T14:54:56.310" Score="11" Body="&lt;p&gt;The dynamic programming algorithms (like &lt;em&gt;policy iteration&lt;/em&gt; and &lt;em&gt;value iteration&lt;/em&gt;) are often presented in the context of reinforcement learning (in particular, in the book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; by Barto and Sutton) because they are very related to reinforcement learning algorithms, like &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning. They are all based on the assumption that the environment can be modelled as an MDP.&lt;/p&gt;&#xA;&lt;p&gt;However, dynamic programming algorithms require that the &lt;em&gt;transition model&lt;/em&gt; and &lt;em&gt;reward functions&lt;/em&gt; of the underlying MDP are known. Hence, they are often referred to as &lt;strong&gt;planning&lt;/strong&gt; algorithms, because they can be used to find a policy (which can be thought of as &lt;strong&gt;plan&lt;/strong&gt;) given the &amp;quot;dynamics&amp;quot; of the environment (which is represented by the MDP). They just exploit the given &amp;quot;physical rules&amp;quot; of the environment, in order to find a policy. This &amp;quot;exploitation&amp;quot; is referred to as a &lt;em&gt;planning algorithm&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning and similar algorithms do not require that the MDP is known. They attempt to find a policy (or value function) by interacting with the environment. They eventually infer the &amp;quot;dynamics&amp;quot; of the underlying MDP from experience (that is, the interaction with the environment).&lt;/p&gt;&#xA;&lt;p&gt;If the MDP is not given, the problem is often referred to as the &lt;strong&gt;full reinforcement learning problem&lt;/strong&gt;. So, algorithms like &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning or SARSA are often considered reinforcement learning algorithms. The dynamic programming algorithms (like policy iteration) do not solve the &amp;quot;full RL problem&amp;quot;, hence they are not always considered RL algorithms, but just planning algorithms.&lt;/p&gt;&#xA;&lt;p&gt;There are several categories of RL algorithms. There are temporal-difference, Monte-Carlo, actor-critic, model-free, model-based, on-policy, off-policy, prediction, control, policy-based or value-based algorithms. These categories can overlap. For example, &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is a temporal-difference (TD), model-free, off-policy, control and value-based algorithm: it is based on an temporal-difference (TD) update rule, it doesn't use a model of the environment (model-free), it uses a behavioural policy that is different than the policy it learns (off-policy), it is used to find a policy (control) and it attempts to approximate a value function rather than directly the policy (value-based).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-08T14:27:39.593" LastActivityDate="2021-02-08T14:27:39.593" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12068" PostTypeId="1" CreationDate="2019-04-30T15:36:39.950" Score="6" ViewCount="1034" Body="&lt;p&gt;Previously I have learned that the &lt;code&gt;softmax&lt;/code&gt; as the output layer coupled with the &lt;code&gt;log-likelihood cost function&lt;/code&gt; (the same as the the &lt;code&gt;nll_loss&lt;/code&gt; in pytorch) can solve &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap3.html#softmax&quot; rel=&quot;noreferrer&quot;&gt;the learning slowdown problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, while I am learning the pytorch mnist tutorial, I'm confused that why the combination of the &lt;code&gt;log_softmax&lt;/code&gt; as the output layer and the &lt;code&gt;nll_loss&lt;/code&gt;(the negative log likelihood loss) as the loss function was used (&lt;a href=&quot;https://github.com/pytorch/examples/blob/master/mnist/main.py#L26&quot; rel=&quot;noreferrer&quot;&gt;L26&lt;/a&gt; and &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/mnist/main.py#L34&quot; rel=&quot;noreferrer&quot;&gt;L34&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that when &lt;code&gt;log_softmax&lt;/code&gt;+&lt;code&gt;nll_loss&lt;/code&gt; was used, the test accuracy was 99%, while when &lt;code&gt;softmax&lt;/code&gt;+&lt;code&gt;nll_loss&lt;/code&gt; was used, the test accuracy was 97%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm confused that what's the advantage of &lt;code&gt;log_softmax&lt;/code&gt; over &lt;code&gt;softmax&lt;/code&gt;? How can we explain the performance gap between them? Is &lt;code&gt;log_softmax&lt;/code&gt;+&lt;code&gt;nll_loss&lt;/code&gt; always better than &lt;code&gt;softmax&lt;/code&gt;+&lt;code&gt;nll_loss&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="25305" LastActivityDate="2020-01-26T04:04:35.510" Title="What's the advantage of log_softmax over softmax?" Tags="&lt;loss-functions&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12069" PostTypeId="1" CreationDate="2019-04-30T15:40:31.133" Score="2" ViewCount="236" Body="&lt;p&gt;It is mentioned by &lt;a href=&quot;https://arxiv.org/abs/1902.10250&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fu 2019&lt;/a&gt; that overfitting might have a negative effect on training DQN. They showed that with either &lt;em&gt;early stopping&lt;/em&gt; or &lt;em&gt;experience replay&lt;/em&gt; this effect could be reduced. The first is reducing overfitting, the latter is increasing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't only have negative effects on the returns though, my test shows that it has a negative effect on &lt;em&gt;value errors&lt;/em&gt; as well (diff. between predict V and ground truth V). I observed frequently with limited data that the training diverged almost 100% of the time (on small nets). Since increasing the amount of data could reduce the chance of divergence, I think this is an effect from overfitting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overfitting should mean low training loss, however, my observation is that there is a strong correlation between &lt;em&gt;TD loss&lt;/em&gt; and &lt;em&gt;value error&lt;/em&gt;. That is if I see a jump in TD loss, I could expect to see a jump in value error around that moment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or it is not overfitting because it is not really &lt;em&gt;fit&lt;/em&gt; (i.e. high loss) but over-optimization that is for sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question is why? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two points:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If it is overfitting, overfitting should have a positive effect because remembering values for all training states correctly is hardly a bad thing. (In fact, my training data is a &lt;em&gt;superset&lt;/em&gt; of my testing data, so remembering should be fine.)&lt;/li&gt;&#xA;&lt;li&gt;If it doesn't fit, this begs a question what over-optimization really does. It doesn't seem to fit, but it does have a negative effect. How could that be?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9793" LastActivityDate="2019-04-30T15:40:31.133" Title="Why overfitting is bad in DQN?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12070" PostTypeId="2" ParentId="12065" CreationDate="2019-04-30T16:06:16.180" Score="3" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; the authors suggest that the &lt;em&gt;topic&lt;/em&gt; of reinforcement learning covers analysis and solutions to problems that can be framed in this way:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reinforcement learning, like many topics whose names end with “ing,” such as machine&#xA;  learning and mountaineering, is simultaneously a problem, a class of solution methods&#xA;  that work well on the problem, and the field that studies this problem and its solution&#xA;  methods. It is convenient to use a single name for all three things, but at the same time&#xA;  essential to keep the three conceptually separate. In particular, the distinction between&#xA;  problems and solution methods is very important in reinforcement learning; failing to&#xA;  make this distinction is the source of many confusions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Markov decision processes are intended to include just&#xA;  these three aspects—sensation, action, and goal—in their simplest possible forms without&#xA;  trivializing any of them. Any method that is well suited to solving such problems we&#xA;  consider to be a reinforcement learning method.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, to answer your questions, the simplest take on this is yes there is more (much more) to RL than the classic value-based optimal control methods of SARSA and Q-learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Including DP and other &quot;RL-related&quot; algorithms in the book allows the author to show how closely related the concepts are. For example, there is little in practice that differentiates Dyna-Q (a planning algorithm closely related to Q-learning) from  experience replay. Calling one strictly &quot;planning&quot; and the other &quot;reinforcement learning&quot; and treating them as separate can reduce insight into the topic. In many cases there are hybrid methods or even a continuum between what you may initially think of as RL and &quot;not RL&quot; approaches. Understanding this gives you a toolkit to modify and invent algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, the book is not the sole arbiter of what is and isn't reinforcement learning. Ultimately this is just a classification issue, and it only matters if you are communicating with someone and there is a chance for misunderstanding. If you name which algorithm you are using, it doesn't really matter whether the person you are talking to thinks it is RL or not RL. It matters what the problem is and how you propose to solve it.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-30T16:06:16.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12071" PostTypeId="1" CreationDate="2019-04-30T16:25:06.407" Score="3" ViewCount="189" Body="&lt;p&gt;In most of RL algorithms I saw, there is a &lt;strong&gt;coefficient that reduces&lt;/strong&gt; actions exploration over time, to help convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in Actor-Critic, or other algorithms (A3C, DDPG, ...) used in continuous action spaces, the different implementation I saw (mainly &lt;strong&gt;using Ornstein Uhlenbeck process&lt;/strong&gt;) is correlated over time, but not decreased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The action noises are clipped into a range of [-1, 1] and are added to policies that are between [-1, 1] too. So, I don't understand how it could work in environments with hard-to-obtain rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thought about this ?&lt;/p&gt;&#xA;" OwnerUserId="23818" LastActivityDate="2019-04-30T16:25:06.407" Title="Should noise (such as OU) be decreased over time in actor / critic algorithms?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12072" PostTypeId="1" AcceptedAnswerId="12079" CreationDate="2019-04-30T17:32:28.253" Score="1" ViewCount="579" Body="&lt;p&gt;One way to speed up a neural network is to prune the network and reducing number of neurons in each layer. What are the other methods to speed up inference?&lt;/p&gt;&#xA;" OwnerUserId="25307" LastActivityDate="2019-05-08T18:04:22.470" Title="What are the various methods for speeding up neural network for inference?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12073" PostTypeId="5" CreationDate="2019-04-30T17:40:13.787" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;https://academic.oup.com/database/article/doi/10.1093/database/bay101/5116160&quot; rel=&quot;nofollow noreferrer&quot;&gt;A survey of ontology learning techniques and application&lt;/a&gt; (2018) for an introduction to the concept of ontology and ontology learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-08T22:58:09.033" LastActivityDate="2019-09-08T22:58:09.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12074" PostTypeId="4" CreationDate="2019-04-30T17:40:13.787" Score="0" Body="For questions related to ontologies, their role in artificial intelligence (AI), or the use of AI for developing ontologies (a field called ontology learning)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-08T22:57:12.833" LastActivityDate="2019-09-08T22:57:12.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12075" PostTypeId="5" CreationDate="2019-04-30T17:49:15.220" Score="0" Body="&lt;p&gt;For more info, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Liquid_state_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Liquid_state_machine&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-30T18:53:05.820" LastActivityDate="2019-04-30T18:53:05.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12076" PostTypeId="4" CreationDate="2019-04-30T17:49:15.220" Score="0" Body="For questions related to the Liquid State Machine (LSM) neural network model." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-04-30T18:53:08.433" LastActivityDate="2019-04-30T18:53:08.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12079" PostTypeId="2" ParentId="12072" CreationDate="2019-05-01T02:08:07.263" Score="2" Body="&lt;p&gt;These are some ways to speed up inference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Reduction of float precision: This is done post-training. According to work in this segment, a very little accuracy is sacrificed for a huge benefit in memory usage reduction. Also it speeds up inference. float32 -&gt; float8.&#xA;Reference paper: &lt;a href=&quot;https://arxiv.org/pdf/1502.02551&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1502.02551&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using ReLU or similar small compute power activations: Benefits are obvious when you don't need to compute heavy exponents, like in tanh or sigmoid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Binary Neural Architecture: This is new. This taps upon the ability to use binary valued weights and activations (1 bit) compared to 32 bit float counterparts. Estimation and learning is done through POPCNT and XNOR operations (for matrix products) and STE (straight through estimator) for backpropagation. You do have to sacrifice for a large number of neurons to learn the same features, &lt;em&gt;but, the speed is on average 7x faster&lt;/em&gt;. Reference paper: &lt;a href=&quot;https://pjreddie.com/media/files/papers/xnor.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pjreddie.com/media/files/papers/xnor.pdf&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://software.intel.com/en-us/articles/binary-neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.intel.com/en-us/articles/binary-neural-networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hardware standpoint: Using specialised hardware to compute matrix products, pretty standard examples are GPUs and TPUs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="24122" LastEditorUserId="24122" LastEditDate="2019-05-08T18:04:22.470" LastActivityDate="2019-05-08T18:04:22.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12080" PostTypeId="2" ParentId="12068" CreationDate="2019-05-01T02:45:51.337" Score="1" Body="&lt;p&gt;The short answer is yes, log_softmax + nll_loss will work better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don’t know the implementation details under the hood in PyTorch, but see the screenshot below from the &lt;a href=&quot;https://pytorch.org/docs/stable/nn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BixQf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BixQf.jpg&quot; alt=&quot;img&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16343" LastActivityDate="2019-05-01T02:45:51.337" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12081" PostTypeId="1" CreationDate="2019-05-01T03:53:26.863" Score="2" ViewCount="155" Body="&lt;p&gt;I have googled for a long time for rotated object detection datasets. Most of papers focused on rotated object detection using DOTA, HRSC2016 or coco text detection dataset. Some researcher also collect their own datasets but almost all of their theme is areal object detection. Is there any others dataset focus on rotated object detection? &lt;/p&gt;&#xA;" OwnerUserId="25318" LastActivityDate="2020-03-29T12:58:32.783" Title="Is there any other rotated object detection datasets?" Tags="&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12082" PostTypeId="2" ParentId="11977" CreationDate="2019-05-01T06:01:19.227" Score="1" Body="&lt;p&gt;Since you said detection, you could do this on OpenCV: HAAR Cascade Detection for faces. It's swift, and should fit your use case. I've seen such scripts running on small Tello drones and gets job done relatively well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Definitely there is hardware built around for this purpose. Look for Nvidia Jetson Nano and Jetson TK1/TX1. It's got sufficient juice to take even 30fps IMO.&lt;/p&gt;&#xA;" OwnerUserId="24122" LastEditorUserId="2444" LastEditDate="2019-05-31T10:51:08.797" LastActivityDate="2019-05-31T10:51:08.797" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12083" PostTypeId="1" AcceptedAnswerId="12088" CreationDate="2019-05-01T08:09:32.053" Score="2" ViewCount="67" Body="&lt;p&gt;Can a normal neural network work as good as a convolutional network? If yes, how much more time and neurons would it need compared to a CNN?&lt;/p&gt;&#xA;" OwnerUserId="19783" LastActivityDate="2019-05-01T11:40:03.137" Title="Is it possible for a NN to reach the same results as CNNs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12085" PostTypeId="1" CreationDate="2019-05-01T08:27:56.603" Score="1" ViewCount="25" Body="&lt;p&gt;Without considering lyrics, there are two type of songs: songs that have a distinct tune which you can easily remember, and songs that are the opposite. Can we design a network to identify the songs that have a distinct tune? It's very hard to find good music, I can't get a song that has a distinct tune for even 100+ songs which consumes a lot of time. &lt;/p&gt;&#xA;" OwnerUserId="25322" LastActivityDate="2019-05-01T08:27:56.603" Title="Is it possible to use AI to find music that have a distinct tune?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12086" PostTypeId="1" AcceptedAnswerId="12090" CreationDate="2019-05-01T08:39:45.717" Score="2" ViewCount="31" Body="&lt;p&gt;If I'd previously trained a model and saved the whole training session (using TensorFlow &lt;code&gt;tf.train.Saver.save()&lt;/code&gt;), and then later restored the session (using TensorFlow &lt;code&gt;tf.train.Saver.restore()&lt;/code&gt;) but then wanted to train using different hyperparameters (batch/mini-batch and learning rate), then I assume the optimizer would need to be reset. Is that the only part of the previous session that would need to reset?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-05-01T12:04:36.490" Title="Changing the batch, mini-batch and learning rate for a previously-trained model" Tags="&lt;ai-design&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12087" PostTypeId="2" ParentId="6229" CreationDate="2019-05-01T10:01:59.147" Score="-1" Body="&lt;p&gt;Without knowing any complex theories, let me talk about some real thing. We make AI to make our life better, so we have to consider economy factors, i.e. the AI system has to be low cost/high energy efficiency to be practically used--the economy always choose the most economic products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't produce an economic conscious machine based on solid state circuits which is the AI that we are always talking about. We have to base AI on bio-systems. Bio-system is the best architecture for consciousness. To see this point you can compare the size/power consumption/cost of human brain and a super computer (which still can't produce conscious for now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why? Because both bio system and solid state circuits are based on the same set of atoms. The way bio system using atoms is much higher efficiency than solid state circuit to implement consciousness. On the other hand, solid state circuits is a much higher efficiency way to use the atoms to implement calculating or even &quot;intelligent work&quot; like machine-vision (a 10W computer like Tegra TX1 can analyze 100+ images per second, people only a few with a 10W brain).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think even without considering economics factor, solid state circuit won't implement a conscious machine someday, since we are already near the end of the game of micronization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So from the economics point of view, if some day there is true AI that is conscious, it will be based on bio-system&lt;/strong&gt; i.e. based on bio engineering to design new species that can be educated to  be communicable to human being.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we may never know how brain produce consciousness (like we don't know why neuron networks works), then we don't know how to design a brain that is both conscious and learns humanity. Even so we can try, i.e. design many different species to see what result we get. Indeed, in this way, I think the hard problem is not to design a brain that is both conscious and learns humanity, but to design a brain that is conscious but &lt;strong&gt;can't&lt;/strong&gt; learns humanity, since &lt;strong&gt;it is highly possible that if you successfully design a conscious brain, then it will learn humanity, too&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The even harder problem is, how to design species that is conscious, can learn humanity, but is not lazy/greedy and don't have the idea of rights&lt;/strong&gt;. If they are lazy/greedy and have the idea of rights like us, then finally they will fight and get human rights. &lt;strong&gt;If so ,they are not AI that work for us as we imagine, they are just new version of us&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So I predict the steps to AI are&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, use bio engineering to design new species that lives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, design new species that have a brain that can be as conscious as human being, which is highly possible to be able to learn humanity, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, design a specie that is conscious as human being, but is not lazy/greedy, and never ask for a right, and can always stay so after a long time living with human being which are lazy/greedy. &lt;strong&gt;I think the work will stop here i.e. without knowing the source of greedy/lazy and after a lot of try out, we still can't get a working specie that is not greedy/lazy. This also means that the possibility to get to real AI that serve us is very low&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forth, design a specie that have the feature above, plus they are happy about their life all the time or have no feelings except for love for human being. Do our self really have humanity if we get the third and without the fourth? May be we are not as conscious/intelligent/humanity as we think we re. If we don't really have humanity, how can we ask AI to?&lt;/p&gt;&#xA;" OwnerUserId="25322" LastActivityDate="2019-05-01T10:01:59.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12088" PostTypeId="2" ParentId="12083" CreationDate="2019-05-01T11:19:12.393" Score="2" Body="&lt;p&gt;&lt;em&gt;NNs won't be able to reach the performance of CNNs, in general.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By a Neural Network, I am assuming you're pointing to 'vanilla' vector based neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take the MNIST dataset, it performs almost similar in both NNs and CNNs; the reason being digits of almost same size, and similar spatial drawings. To put it easy, all the digits roughly take up the same area as the other 60k. If you were to zoom in our zoom out these digits, the trained NN may not perform well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plain NNs lack the ability to extract 'position independent' features. A cat is a cat to your eye, no matter whether you saw it in the center of an image, or left corner. CNNs use 'filters' to extract 'templates' of cat. Hence CNNs can 'localize' what it is searching for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could say, NNs look at the 'whole' data to make a sense, CNNs look at the 'special' features at certain parts to make a sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is also a reason why CNNs are popular. NNs are suited for their own applications. Every input node in a vectorized Neural Networks represent a feature, and that feature is tied to a purpose throughout training, i.e. its position is fixed throughout the training.&lt;/p&gt;&#xA;" OwnerUserId="24122" LastActivityDate="2019-05-01T11:19:12.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12089" PostTypeId="2" ParentId="12083" CreationDate="2019-05-01T11:40:03.137" Score="0" Body="&lt;p&gt;Yes. In theory, a single layer neural network can compute any function. In practice, such a network would have to be much larger than a CNN with equivalent functionality and would therefore be much harder to train. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-05-01T11:40:03.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12090" PostTypeId="2" ParentId="12086" CreationDate="2019-05-01T12:04:36.490" Score="1" Body="&lt;p&gt;Yes as far as I know you only need to reset the optimizer.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-05-01T12:04:36.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12091" PostTypeId="2" ParentId="11557" CreationDate="2019-05-01T12:05:28.797" Score="1" Body="&lt;p&gt;Here's what I understand, welcome to point out any mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When starting a new episode(but still in the same task), SNAIL does not clear its batches. Instead, it makes decisions based on the current observation and observation-action pairs from the previous episode. In this way, it keeps knowledge of the previous episode whereby achieving few-shot learning in the test time.&lt;/p&gt;&#xA;" OwnerUserId="8689" LastActivityDate="2019-05-01T12:05:28.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12092" PostTypeId="1" AcceptedAnswerId="12093" CreationDate="2019-05-01T13:14:22.413" Score="2" ViewCount="30" Body="&lt;p&gt;I've never worked with very large models that require weeks or months of training, but in such a situation, what happens if you want to add extra features inputs, do you need to re-train the entire model again from scratch, or how is it handled?  E.g. if Tesla added an extra sensor to its cars, would it need to re-train its network again from scratch to include this extra sensor input?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-05-01T14:46:54.053" Title="Adding input features - is complete re-training required?" Tags="&lt;ai-design&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12093" PostTypeId="2" ParentId="12092" CreationDate="2019-05-01T14:46:54.053" Score="2" Body="&lt;p&gt;I'll try to explain how I would do it and the intuition behind it, feel free to correct me if something doesn't make sense.&#xA;Lets consider you have an input of shape F where F is the number of features. If you were to construct a simple feed forward neural network you'll need to multiply the input with a weight matrix of shape (F, hidden_dim). Now if we want to add one more feature, the input will be of shape F+1 and the multiplication with the first layer will not work. What we could do to overcome this problem is to pad the weight matrix with an extra 0. In theory the new network should be able to reproduce the results of the first because it ignores the new feature. Now if we want to learn the &quot;importance&quot; of the new feature we could train the model to do so. I assume that the training time should be significantly lower since some of the weights might not need to be updated.    &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-05-01T14:46:54.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12094" PostTypeId="2" ParentId="3372" CreationDate="2019-05-01T15:28:56.383" Score="0" Body="&lt;p&gt;I realise this question was asked a couple of years ago, but I think it's worth reflecting that the OP asked an insightful question, as indeed the gaming industry is now actively working on the questions and ideas raised by the OP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, huge sums of money are now being invested in the AI field within the gaming industry, given the increase to productivity it would produce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might find this YouTube video interesting, which discusses the topic of producing game elements from description, and ray-tracing techniques.  It's not produced from a AI research/developer-perspective, but certainly a useful overview of where things stand mid-2019&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=fQlQQSsC47g&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=fQlQQSsC47g&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-05-01T15:28:56.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12095" PostTypeId="1" CreationDate="2019-05-01T16:29:02.923" Score="4" ViewCount="307" Body="&lt;p&gt;It this podcast between Oriol Vinyals and Lex Friedman: &lt;a href=&quot;https://youtu.be/Kedt2or9xlo?t=1769&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/Kedt2or9xlo?t=1769&lt;/a&gt;, at 29:29, Oriol Vinyals refers to a paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you look at research in computer vision where it makes a lot of sense to treat images as two dimensional arrays...&#xA;  There is actually a very nice paper from Facebook. I forgot who the&#xA;  authors are but I think [it's] part of Kaiming He's group. And what&#xA;  they do is they take an image, which is a 2D signal, and they&#xA;  actually take pixel by pixel, and scramble the image, as if it&#xA;  was a just a list of pixels, crucially they encode the position of the pixels&#xA;  with the XY coordinates. And this is a new architecture which we&#xA;  incidentally also use in Starcraft 2 called the transformer, which is&#xA;  a very popular paper from last year which yielded very nice results in&#xA;  machine translation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Do you know which paper he is referring to? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing maybe he is talking about &lt;a href=&quot;https://arxiv.org/abs/1711.07971&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-local neural networks&lt;/a&gt;, but I'm probably guessing wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit: after reviewing the recent publications of Kaiming He (&lt;a href=&quot;http://kaiminghe.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://kaiminghe.com/&lt;/a&gt;), maybe I'm guessing right. Any thoughts?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1741" LastEditorUserId="1741" LastEditDate="2019-05-01T16:42:22.740" LastActivityDate="2019-05-13T17:27:44.853" Title="Name of paper for encoding/representing XY coordinates in deep learning" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12097" PostTypeId="1" AcceptedAnswerId="12130" CreationDate="2019-05-02T01:23:50.440" Score="3" ViewCount="133" Body="&lt;p&gt;In &lt;a href=&quot;https://int8.io/monte-carlo-tree-search-beginners-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; toward the end, the author writes the following:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j2Dsm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j2Dsm.jpg&quot; alt=&quot;pic_0&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fPf77.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fPf77.jpg&quot; alt=&quot;pic&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of my question, let’s assume that a terminal state gives a reward of +1 for a win and -1 for a loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the author says “for any two consecutive nodes this perspective is opposite,” does that mean that if &lt;span class=&quot;math-container&quot;&gt;$Q_i$&lt;/span&gt; is positive (for example, 4) for player A at a given node, the same node will have the negative of that value for player B (-4 in my hypothetical)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to compute two statistics to store the node value (one for each player) or can I simply store statistics for one player and flip the sign at every consecutive node?&lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2019-05-02T01:51:21.120" LastActivityDate="2019-05-04T13:38:26.523" Title="How does Monte Carlo Tree Search UCT exploitation value change based on perspective?" Tags="&lt;monte-carlo-tree-search&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12098" PostTypeId="1" CreationDate="2019-05-02T02:14:29.780" Score="1" ViewCount="38" Body="&lt;p&gt;Recently OpenAI removes their board game environments. (It may be possible to install an older version to get access to them, but I haven’t downgraded). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a list of repositories or resources of board game or similar environments that can be used to practice RL implementations? Things like checkers, chess, backgammon, or even grid world and mazes would be excellent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A running list might be useful for many in this community.  &lt;/p&gt;&#xA;" OwnerUserId="16343" LastActivityDate="2019-05-02T02:14:29.780" Title="What are a list of board game environments for RL practice?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;environment&gt;&lt;open-source&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12099" PostTypeId="1" CreationDate="2019-05-02T03:08:37.743" Score="4" ViewCount="168" Body="&lt;p&gt;In the Constraint Propagation in CSP, it is often stated that &lt;em&gt;pre-processing&lt;/em&gt; can solve the whole problem, so no search is required at all. And the key  idea is &lt;strong&gt;local consistency&lt;/strong&gt;. What does this actually mean?&lt;/p&gt;&#xA;" OwnerUserId="23299" LastEditorUserId="2444" LastEditDate="2019-05-02T13:00:02.243" LastActivityDate="2021-05-13T17:06:10.637" Title="What is local consistency in constraint satisfaction problems?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12100" PostTypeId="1" CreationDate="2019-05-02T04:35:40.697" Score="2" ViewCount="87" Body="&lt;p&gt;Is it possible to use t-SNE, PCA or UMAP to find separating hyperplane?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have data points in high dimensional space and we want to phase separate it into two sets of points? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to use t-SNE, PCA or UMAP for such a task?&lt;/p&gt;&#xA;" OwnerUserId="12975" LastEditorUserId="12975" LastEditDate="2019-05-16T02:40:10.097" LastActivityDate="2019-05-16T02:40:10.097" Title="Using UMAP, PCA or t-SNE to find the separating hyperplane?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12101" PostTypeId="1" CreationDate="2019-05-02T07:39:49.367" Score="2" ViewCount="130" Body="&lt;p&gt;I want to formulate the following sentence using FOL; I want also to know whether there any contradictions in it, or if it is consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming human relations are binary:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;All Human Relations are Utilitarian (Human relations are Utilitarian&#xA;  only when people in those relations are selfish and calculative).&#xA;  I am a human. People are Humans. Unselfish people are nice. Therefore, I am nice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22322" LastEditorUserId="23527" LastEditDate="2019-05-02T08:46:52.580" LastActivityDate="2019-05-02T08:46:52.580" Title="Formulation of a sentence using FOL" Tags="&lt;logic&gt;&lt;knowledge-representation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12102" PostTypeId="2" ParentId="12066" CreationDate="2019-05-02T10:07:30.893" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If n different edges with associated weights &lt;span class=&quot;math-container&quot;&gt;$w_1, w_2, . . . , w_n$&lt;/span&gt; point to this node, then after selecting weights&#xA;  with uniform probability from the interval &lt;span class=&quot;math-container&quot;&gt;$[−α, α]$&lt;/span&gt;, the expected total input&#xA;  to the node is &lt;span class=&quot;math-container&quot;&gt;$$\sum_iw_ix_i$$&lt;/span&gt;.&#xA;  By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that the input to a node  is &lt;span class=&quot;math-container&quot;&gt;$$w_1x_1 + w_2x_2 + ... + w_nx_n $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variance of the total input to a node is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma^2 = \sum^n_{i=1}E[w_i^2]E[x_i^2] = n(E[w_1^2]E[x_1^2])&#xA;= n([\frac{(\alpha-(-\alpha))^2}{12}][\frac{1}{3}])=n\alpha^2\frac{1}{9}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;since inputs and weights are uncorrelated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have taken out the summation and replaced it with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times the variance of an input and a weight -- this can be done because both are &lt;a href=&quot;https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables&quot; rel=&quot;nofollow noreferrer&quot;&gt;independent and identically distributed&lt;/a&gt; in their own right and are independent of each other too. Variance of &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; is the same as variance of &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I missing some implicance of the law of large numbers use for the input to the node?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Implication of Law of Large Numbers:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the &lt;em&gt;random variable&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$E(w_ix_i)$&lt;/span&gt; in  &lt;span class=&quot;math-container&quot;&gt;$$E(\sum^n_i(w_ix_i)^2)$$&lt;/span&gt; is written as product of the random variables &lt;span class=&quot;math-container&quot;&gt;$E(w_i)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E(x_i)$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$$\sum^n_{i=1}E[w_i^2]E[x_i^2]$$&lt;/span&gt;. This can only be written when the two random variables are independent. And the switching of expectation and summation is due to the &lt;a href=&quot;https://brilliant.org/wiki/linearity-of-expectation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;linearity of expectation property&lt;/a&gt;.&#xA;How do we get independence here? LLN comes to the rescue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That comes from the fact that -- according to the law of large numbers -- the &lt;em&gt;random variable&lt;/em&gt; (total input) follows a normal distribution which implies the two random variables i.e. the weights and the inputs also follow normal distribution -- and &lt;a href=&quot;https://en.wikipedia.org/wiki/Normally_distributed_and_uncorrelated_does_not_imply_independent&quot; rel=&quot;nofollow noreferrer&quot;&gt;it is also given that the two random variables are uncorrelated and thus they are independent.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-05-18T07:53:32.840" LastActivityDate="2019-05-18T07:53:32.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12103" PostTypeId="1" CreationDate="2019-05-02T10:25:44.270" Score="1" ViewCount="333" Body="&lt;p&gt;I have skimmed through a bunch of deep learning books, but I have not yet understood whether we must use the experience replay buffer with the A3C algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approached I used is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have some threaded agents that play their own copy of an enviroment; they all use a shared NN to predict 'best' move given a current state;&lt;/li&gt;&#xA;&lt;li&gt;At the end of each episode, they push the episode (in the form of a list of steps) in a shared memory&lt;/li&gt;&#xA;&lt;li&gt;A separated thread reads from the shared memory and executes the train step on the shared NN, training it episode after episode.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this an appropriate approach? More specifically, do I need to sample data to train the NN from the shared memory? Or should I push in the shared memory each step, just when it's done by a single agent? In this last case, I wonder how I could calculate discounted rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm afraid that with my current approach I'm doing nothing more that presenting n episodes to the NN, with the hope that each agent explores the enviroment differently from other agents (so that NN is presented a richer variety of states).&lt;/p&gt;&#xA;" OwnerUserId="25352" LastEditorUserId="2444" LastEditDate="2019-05-02T13:09:43.720" LastActivityDate="2019-05-02T13:09:43.720" Title="Do we need to use the experience replay buffer with the A3C algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;deep-rl&gt;&lt;actor-critic-methods&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12105" PostTypeId="5" CreationDate="2019-05-02T13:28:14.723" Score="0" Body="&lt;p&gt;For more info, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Image_segmentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Image_segmentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-02T19:21:21.173" LastActivityDate="2019-05-02T19:21:21.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12106" PostTypeId="4" CreationDate="2019-05-02T13:28:14.723" Score="0" Body="For questions related to image segmentation (in computer vision and related AI fields)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-02T19:22:30.587" LastActivityDate="2019-05-02T19:22:30.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12108" PostTypeId="5" CreationDate="2019-05-02T15:01:09.850" Score="0" Body="&lt;p&gt;For more info, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Transfer_learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-02T19:24:04.890" LastActivityDate="2019-05-02T19:24:04.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12109" PostTypeId="4" CreationDate="2019-05-02T15:01:09.850" Score="0" Body="For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-02T19:21:29.837" LastActivityDate="2019-05-02T19:21:29.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12110" PostTypeId="1" CreationDate="2019-05-02T15:31:08.017" Score="2" ViewCount="54" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1502.05477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trust Region Policy Optimization (TRPO) paper&lt;/a&gt;, on page 10, it is stated&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An informal overview is as follows.  Our proof relies on the notion of  coupling,  where  we  jointly  define  the  policies &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt;so  that  they  choose  the  same  action  with  high  probability &lt;span class=&quot;math-container&quot;&gt;$(1−\alpha)$&lt;/span&gt;.   Surrogate loss &lt;span class=&quot;math-container&quot;&gt;$L_π(\hat\pi)$&lt;/span&gt; accounts for the the advantage of &lt;span class=&quot;math-container&quot;&gt;$\hat\pi$&lt;/span&gt; the first time that it disagrees with &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;,  but not subsequent disagreements. Hence, the error in &lt;span class=&quot;math-container&quot;&gt;$L_\pi$&lt;/span&gt; is due to two or more disagreements between &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat\pi$&lt;/span&gt;, hence, we get an &lt;span class=&quot;math-container&quot;&gt;$O(\alpha^2)$&lt;/span&gt; correction term, where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the probability of disagreement.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't see how this holds? In what way does &lt;span class=&quot;math-container&quot;&gt;$L_\pi$&lt;/span&gt; account for one disagreement? Surely when &lt;span class=&quot;math-container&quot;&gt;$\hat\pi$&lt;/span&gt; disagrees with &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; you will have different trajectories under expectation for each so then &lt;span class=&quot;math-container&quot;&gt;$L_\pi$&lt;/span&gt; immediately is different from &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any insight. I understand the proof given, but I wanted to try and capture this intuition.&lt;/p&gt;&#xA;" OwnerUserId="25153" LastEditorUserId="2444" LastEditDate="2019-05-02T15:58:11.030" LastActivityDate="2019-05-02T15:58:11.030" Title="How does the TRPO surrogate loss account for the error in the policy?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12111" PostTypeId="5" CreationDate="2019-05-02T16:00:54.710" Score="0" Body="&lt;p&gt;For more info, have a look at the paper that introduced this algorithm &lt;a href=&quot;https://arxiv.org/pdf/1502.05477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trust Region Policy Optimization&lt;/a&gt;, by John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan and Pieter Abbeel, 2017.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-06T05:16:08.393" LastActivityDate="2019-05-06T05:16:08.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12112" PostTypeId="4" CreationDate="2019-05-02T16:00:54.710" Score="0" Body="For questions related to the reinforcement learning algorithm called Trust Region Policy Optimization (TRPO)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-06T05:15:54.907" LastActivityDate="2019-05-06T05:15:54.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12114" PostTypeId="1" CreationDate="2019-05-02T18:44:59.150" Score="1" ViewCount="159" Body="&lt;p&gt;Today, AI is mainly driven by own-profit-oriented companies (e.g. Facebook, Amazon, Google). Admittedly, there's a lot of AI in the health sector (even in the public health sector) and there's a lot of AI in the sustainability sector &amp;ndash; but also mostly driven by obviously own-profit-oriented companies (e.g. Tesla, Uber, Google).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other side, one often hears from hard-core economists that &lt;a href=&quot;https://www.wikiwand.com/en/Planned_economy&quot; rel=&quot;nofollow noreferrer&quot;&gt;centrally planned (= public-profit-oriented) economies&lt;/a&gt; (or economic principles) are &quot;the work of the devil&quot; - and that they failed all over history (sometimes for understandable reasons).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But intelligently planning global economic processes and applying these plans with the help of state-of-the-art AI - given the huge amounts of really big data available, and given the argument that globalization is finally for the benefit of all - would seem to be a rewarding endeavour, at least for parts of the AI community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why isn't this endeavour undertaken more decidedly? (Or is it?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do I find approaches to apply AI to global economic processes? (Not only describing and understanding but mainly planning and executing?)&lt;/p&gt;&#xA;" OwnerUserId="25362" LastEditorUserId="2444" LastEditDate="2020-03-09T21:24:05.097" LastActivityDate="2021-05-02T16:38:01.657" Title="What approaches are there to apply AI to global economic processes?" Tags="&lt;applications&gt;&lt;social&gt;&lt;ethics&gt;&lt;game-theory&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12115" PostTypeId="1" AcceptedAnswerId="12119" CreationDate="2019-05-02T19:49:34.240" Score="1" ViewCount="44" Body="&lt;p&gt;Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem to be meant for either pure text or pure numerical data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are those that can handle data with the dual data types, but those are generally not flexible for wrangling. For example, for pickle you have to load the entire thing into memory if you want to wrangle anything. You can just append directly to the disk like you can with hdf5, which can be very helpful for huge datasets which can not be all loaded into memory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, any alternatives to Pandas for wrangling Huge datasets? Sometimes you can't load all the data into Pandas without causing a memory crash.&lt;/p&gt;&#xA;" OwnerUserId="18358" LastActivityDate="2020-11-18T06:30:59.810" Title="What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?" Tags="&lt;natural-language-processing&gt;&lt;datasets&gt;&lt;data-science&gt;&lt;structured-data&gt;&lt;data-mining&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12116" PostTypeId="1" CreationDate="2019-05-02T21:15:42.313" Score="5" ViewCount="124" Body="&lt;p&gt;This might be a trivial question but I couldn't find any reliable answers on the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost all the neural network architectures for self-driving cars that I have seen on the internet have a feedforward network, previous frames will not help in making the current decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read somewhere that Tesla uses two last frames captured to make a decision, even then 2 frames will not be that useful in this case. &#xA;This might not be very helpful when predicting things ie.. lane cut-ins, as the system needs to observe the vehicle (that is going to cut in) behavior such as turn indicator, vehicle veering towards center lane over time in order to predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain if this is the way production self-driving card such as Tesla work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DOtqF.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DOtqF.png&quot; alt=&quot;Feed forward CNN&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or Is it something like the below?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/83beA.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/83beA.png&quot; alt=&quot;Something like this from my imagination&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or are they using something like Many to one Recurrent net where inputs are CNN vectors of previous few frames and output is the control?&lt;/p&gt;&#xA;" OwnerUserId="25366" LastActivityDate="2019-05-11T15:49:40.887" Title="Are self-driving cars using single frame or multiple frame to make decision?" Tags="&lt;autonomous-vehicles&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12117" PostTypeId="2" ParentId="12116" CreationDate="2019-05-02T23:30:14.790" Score="1" Body="&lt;p&gt;It varies quite substantially between different self-driving paradigms(rather obviously) but for the most part the vast majority of implementations are using a variety of different reference frames in order to make predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, Tesla's Autopilot is being fed many different camera feeds as well as radar and ultrasonic signals that are processed in a variety of temporal contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While, for the most part, all of these programs are very tight-lipped, we can make a variety of assumptions based on the information available and educated assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with many large, complex ML/AI systems, there is a large amount of compartmentalization where many different connectionist(or sometimes classic) models are combined(à la &lt;a href=&quot;https://ai.google/research/pubs/pub45530&quot; rel=&quot;nofollow noreferrer&quot;&gt;youtube recommendation system&lt;/a&gt;). Tesla is likely utilizing recurrent and convolutional networks where particular modules(combinations of models) are deciding on specific contexts(temporal or signal-based). These outputs are then most likely fed into an actor network which makes real time decisions. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-05-02T23:30:14.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12118" PostTypeId="1" CreationDate="2019-05-03T03:15:23.973" Score="3" ViewCount="137" Body="&lt;p&gt;Why don't people use nonlinear activation functions after projecting the query key value in attention?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like doing this would lead to much-needed nonlinearity, otherwise, we're just doing linear transformations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This observation applies to the transformer, additive attention, etc.&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2019-11-01T03:14:05.543" LastActivityDate="2019-11-01T03:14:05.543" Title="Why don't people use nonlinear activation functions after projecting the query key value in attention?" Tags="&lt;neural-networks&gt;&lt;attention&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12119" PostTypeId="2" ParentId="12115" CreationDate="2019-05-03T06:44:32.197" Score="2" Body="&lt;p&gt;There are different possible ways to handle huge datasets:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If the data is too big to be fully uploaded to RAM, you can iterate over it in Pandas. You can find a brief explanation in &lt;a href=&quot;https://towardsdatascience.com/why-and-how-to-use-pandas-with-large-data-9594dda2ea4c&quot; rel=&quot;nofollow noreferrer&quot;&gt;the article &lt;code&gt;Why and How to Use Pandas with Large Data&lt;/code&gt;&lt;/a&gt;, section &lt;code&gt;1. Read CSV file data in chunk size&lt;/code&gt;. Or add more RAM (or use powerful server hardware), if you want to continue using single machine.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the data is really big, probably it's better to store and process it by multiple computers using special software, A specific tool depends on what you want to do with the data:&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2.1. You can even keep using Pandas: there is &lt;a href=&quot;https://stackoverflow.com/a/47426521/5308802&quot;&gt;an extension named Dask&lt;/a&gt; which wraps interfaces of Python iterables, NumPy, and Pandas to run on multiple machines. Also, there are other similar tools.&lt;/li&gt;&#xA;&lt;li&gt;2.2. A more independent yes simple approach is to use some analytical, distributed DBMS like Google BigQuery or Cassandra.&lt;/li&gt;&#xA;&lt;li&gt;2.3. And if you need something more powerful and complex, MapReduce systems like Apache Hadoop, Spark can be utilised. Moreover, there are many &lt;a href=&quot;https://livebook.manning.com/book/machine-learning-in-action/chapter-15/135&quot; rel=&quot;nofollow noreferrer&quot;&gt;articles&lt;/a&gt; and &lt;a href=&quot;https://papers.nips.cc/paper/2006/file/77ee3bc58ce560b86c2b59363281e914-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;scientific papers&lt;/a&gt; that describes usage of MapReduce for the machine learning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, as always, it is a matter of your opportunities and your aims of dealing with the data.&lt;/p&gt;&#xA;" OwnerUserId="18397" LastEditorUserId="18397" LastEditDate="2020-11-18T06:30:59.810" LastActivityDate="2020-11-18T06:30:59.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12123" PostTypeId="2" ParentId="11992" CreationDate="2019-05-03T09:37:05.753" Score="0" Body="&lt;p&gt;I changed the rewards to be negative and positive by substructing the mean reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to improve the Q function boundries.&lt;/p&gt;&#xA;" OwnerUserId="25141" LastActivityDate="2019-05-03T09:37:05.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12125" PostTypeId="1" CreationDate="2019-05-03T14:11:27.203" Score="2" ViewCount="49" Body="&lt;p&gt;I trained a neural network which makes a regression to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Poincar%C3%A9_disk_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Poincarè Disk Model&lt;/a&gt; with radius &lt;span class=&quot;math-container&quot;&gt;$r = 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to optimize using the &lt;em&gt;hyperbolic distance&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{arcosh} \left( 1 + \frac{2|pq|^2|r|^2}{(|r|^2 - |op|^2)(|r|^2 - |oq|^2)} \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$|op|$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|oq|$&lt;/span&gt; are the distances of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and respectively &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; to the centre of the disk, &lt;span class=&quot;math-container&quot;&gt;$|pq|$&lt;/span&gt; the distance between &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$|r|$&lt;/span&gt; the radius of the boundary circle of the disk and  &lt;span class=&quot;math-container&quot;&gt;$\operatorname{arcosh}$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_hyperbolic_functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverse hyperbolic function&lt;/a&gt; of hyperbolic cosine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there is a problem&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In the Poincarè Disk Model with &lt;span class=&quot;math-container&quot;&gt;$r = 1$&lt;/span&gt;, the distance is defined only for vectors which have norm less than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A neural network does not know this rule, so it can predict vectors with norm greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, I tried to use the distance defined in a space with &lt;span class=&quot;math-container&quot;&gt;$r = 2$&lt;/span&gt;, and it works very well for the learning task, but I'm doubtful because the distance doesn't scale in a linear way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will there be unwanted effects, in your opinion?&lt;/p&gt;&#xA;" OwnerUserId="25377" LastEditorUserId="2444" LastEditDate="2019-05-03T21:02:19.750" LastActivityDate="2019-05-03T21:02:19.750" Title="Should I use the hyperbolic distance loss in the case of Poincarè Disk Model?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;loss-functions&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12126" PostTypeId="1" CreationDate="2019-05-03T22:17:52.547" Score="1" ViewCount="82" Body="&lt;p&gt;Suppose I have a lot of scans of hardcopy documents, in the form of jpegs. Some of them are potentially scans of driver's licenses or identification cards. I wonder what would be a good way to identify those scans that contain driver's license/ID cards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thought I had was training a model or use an existing pretrained model that can detect faces. However, if the data set I have has a lot of scans of photos of people, it would cause false positives. So I am not sure how I might approach this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thought would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="22719" LastActivityDate="2019-06-04T18:06:30.743" Title="How to identify whether images contain driver's licenses or ID cards" Tags="&lt;image-recognition&gt;&lt;classification&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12127" PostTypeId="1" CreationDate="2019-05-04T07:27:45.790" Score="7" ViewCount="144" Body="&lt;p&gt;I'm trying to develop skills to deal with very small amounts of labeled samples (250 labeled/20000 total, 200 features) by practicing on Kaggle &amp;quot;Don't Overfit&amp;quot; &lt;a href=&quot;https://www.kaggle.com/c/overfitting/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset&lt;/a&gt; (Traget_Practice have provided all 20,000 Targets). I've read a ton of papers and articles on this topic, but everything I've tried did not improve simple regularized SVM results (best accuracy was 75 and AUC was 85) or any other algorithm result (LR, K-NN, NaiveBayes, RF, MLP). I believe the result can be better (on Leaderboard they go even over AUC 95)&lt;/p&gt;&#xA;&lt;p&gt;What I've tried without success:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Remove outliers&lt;/strong&gt;  I've tried to remove 5%-10% outliers with EllipticEnvelope and with IsolationForest.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Feature Selection&lt;/strong&gt; I've tried RFE (with or without CV) + L1/L2 regularised LogisticRegression, and SelectKBest (with chi2).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Semi-Supervised techniques&lt;/strong&gt; I've tried co-training with different combinations of two complementary algorithms and :100-100: split features. I've also tried LabelSpreading, but I don't know how to provide the most uncertain samples (I tried predictions from other algorithms, but there were many mislabeled samples, hence was unsuccessful).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Ensembling Classifiers&lt;/strong&gt; StackingClassifier with all possible combinations of algorithms and this also didn't improve the result (the best is the same as SVM accuracy 75 and AUC 85).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Can anyone give me advice on what I'm doing wrong or what else to try?&lt;/p&gt;&#xA;" OwnerUserId="25393" LastEditorUserId="2444" LastEditDate="2021-01-31T18:34:08.043" LastActivityDate="2021-03-02T19:06:00.813" Title="How to deal with a small amount of labeled samples?" Tags="&lt;machine-learning&gt;&lt;semi-supervised-learning&gt;&lt;data-labelling&gt;&lt;labeled-datasets&gt;&lt;unlabeled-datasets&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12128" PostTypeId="1" CreationDate="2019-05-04T08:44:31.557" Score="1" ViewCount="17" Body="&lt;p&gt;If we have a neural network that learns the generative model for P(A, B, C)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And now, we want to learn the generative model for P(A, B, C, D)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any theory that says learning P(A,B,C) and then composing it with P(D | A,B,C) is faster than learning P(A,B,C,D) from scratch?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="10135" LastEditDate="2019-05-04T15:17:42.783" LastActivityDate="2019-05-04T15:17:42.783" Title="Theoretical grounding for ease of training with a prior" Tags="&lt;neural-networks&gt;&lt;theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12129" PostTypeId="1" CreationDate="2019-05-04T09:08:23.283" Score="1" ViewCount="76" Body="&lt;p&gt;I was learning about GAN when the term &quot;Label Smoothing&quot; appears. From the video tutorial that I watch, they use the term &quot;label smoothing&quot; to change the binary labels when calculating the loss of discriminator network. Instead of using 0 or 1 label they use 0 or 0.9 labels. What is the main purpose of this label smoothing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've skimmed through the &lt;a href=&quot;https://arxiv.org/pdf/1512.00567.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;, there is a lot of maths, that honestly, I got difficulties in understanding it. But I notice this paragraph in there:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We propose a mechanism for encouraging the model to be less confident. While this may not be desired if the goal is to maximize the log-likelihood of training labels, it does regularize  the  model  and  makes  it  more  adaptable&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And it gives me another question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;why &quot;this may not be desired if the goal is to maximize the log-likelihood of training labels&quot;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;what do they mean with &quot;adaptable&quot;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-05-04T09:08:23.283" Title="What is the intuition behind the Label Smoothing?" Tags="&lt;generative-adversarial-networks&gt;&lt;regularization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12130" PostTypeId="2" ParentId="12097" CreationDate="2019-05-04T13:32:59.637" Score="2" Body="&lt;p&gt;In UCT, the value of Q(vi) / N(vi) is bounded between 0 and 1. Normally when applying MCTS to 2-player games, what happens is the following:&lt;br&gt;&#xA;N(vi) corresponds to the total number of games simulated in node vi.&lt;br&gt;&#xA;Q(vi) corresponds to the total number of games simulated and won in node vi.&lt;br&gt;&#xA;So in each simulation Q(vi) will add +1 to the winning player and +0 to the losing player.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a tree representation of a 2-player game, each level will represent player A and player B alternately. &lt;strong&gt;So you will add 1 to every other node&lt;/strong&gt;. I don't think it makes sense to add negative values to Qi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question:&#xA;&lt;strong&gt;You only need to store N and Q for each node&lt;/strong&gt;, where N is the total number of times that node was in the path of a simulated node and Q is the number of times it won, out of those simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can give you the example of the way I implemented that part, in a board game I saved in the state class a variable which said who the next player was ( 1 or 2). So I would know that in that state (each node is a state) the player who made the last move was the other one (by the formula: player = 3 - next_player), and I would add 1 to the Qi of those nodes. &lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-05-04T13:38:26.523" LastActivityDate="2019-05-04T13:38:26.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12131" PostTypeId="5" CreationDate="2019-05-04T15:15:40.347" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-23T15:27:36.010" LastActivityDate="2019-10-23T15:27:36.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12132" PostTypeId="4" CreationDate="2019-05-04T15:15:40.347" Score="0" Body="For questions related to the concept of dropout, which refers to the dropping out units in a neural network (NN), during the training of the NN, so that to avoid overfitting. The dropout method is a regularisation technique, which was introduced in &quot;Dropout: A Simple Way to Prevent Neural Networks from Overfitting&quot; (2014) by Nitish Srivastava et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-23T15:27:36.010" LastActivityDate="2019-10-23T15:27:36.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12133" PostTypeId="1" CreationDate="2019-05-04T16:03:30.890" Score="2" ViewCount="106" Body="&lt;p&gt;I've heard the expression &quot;Gaussian kernel&quot; in several contexts (e.g. in the kernel trick used in SVM). A &lt;a href=&quot;https://datascience.stackexchange.com/q/17352/10640&quot;&gt;Gaussian kernel&lt;/a&gt; usually refers to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian function&lt;/a&gt; (that is, a function similar to the probability density function of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian distribution&lt;/a&gt;) that is used to measure the similarity between two vectors (or numbers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is this Gaussian function called a &quot;kernel&quot;? Why not just calling it a Gaussian (similarity) function? Does it have something to do with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_(algebra)&quot; rel=&quot;nofollow noreferrer&quot;&gt;kernel of a linear transformation&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-04T16:08:34.100" LastActivityDate="2021-02-10T19:01:10.103" Title="Why do we use the word &quot;kernel&quot; in the expression &quot;Gaussian kernel&quot;?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;math&gt;&lt;history&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12134" PostTypeId="2" ParentId="12100" CreationDate="2019-05-04T19:02:15.120" Score="3" Body="&lt;p&gt;All the methods are basically manifold methods which are used to squeeze hyper dimensions to two or three dimensions with a certain amount of information loss. So whatever you &quot;see&quot; with these methods are not real and deceiving. You may see a separation of data points in 3D but when they are mapped onto the actual dimensions it may be complete rubbish. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By definition, these methods do not provide a way to find separating hyperplane. They just tell you some condensed information about the whole data. It is generally advised to use methods like t-SNE with caution. &lt;/p&gt;&#xA;" OwnerUserId="25400" LastActivityDate="2019-05-04T19:02:15.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12135" PostTypeId="1" CreationDate="2019-05-04T23:40:15.413" Score="5" ViewCount="167" Body="&lt;p&gt;Suppose I have an MDP &lt;span class=&quot;math-container&quot;&gt;$(S, A, p, R)$&lt;/span&gt; where the &lt;span class=&quot;math-container&quot;&gt;$p(s_j|s_i,a_i)$&lt;/span&gt; is uniform, i.e given an state &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; and an action &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; all states &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt; are equally probable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to find an optimal policy for this MDP. Can I just apply the usual methods like policy gradients, actor-critic to find the optimal policy for this MDP? Or is there something I should be worried about?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least, in theory, it shouldn't make any difference. But I'm wondering are there any practical considerations I should be worried about? Should the discount factor, in this case, be high?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward function here depends both on states and actions and is not uniformly random.&lt;/p&gt;&#xA;" OwnerUserId="25403" LastEditorUserId="1847" LastEditDate="2019-05-05T16:55:39.960" LastActivityDate="2019-05-05T17:05:59.460" Title="Reinforcement learning with uniformly random dynamics" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12136" PostTypeId="1" CreationDate="2019-05-05T00:11:04.380" Score="1" ViewCount="21" Body="&lt;p&gt;I am trying to figure out how ALS works when minimizing the following formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\\ \\$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{min}_{\lbrace b_u,b_i \rbrace} \sum_{(u,i)\in \mathcal{K}} (r_{ui} -  \bar{r} - b_u - b_i )^2 + \lambda_{1}(\sum_{u} b_u^{2} +\sum_{i} b_i^{2})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\\ \\$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wdqbK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wdqbK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{Question 1}$&lt;/span&gt;:&#xA;I would like to know how does Alternating Least Squares work in this case. How does it minimize the equation on the picture? The idea of the whole equation that needs to be minimized, I think, it is like when we do a simple linear regression and we have to fit the line. Am I right? In the Lineal Regression we do &lt;span class=&quot;math-container&quot;&gt;$(y - \hat y)^2$&lt;/span&gt;. In the case of the paper we do &lt;span class=&quot;math-container&quot;&gt;$(r_{ui} - \mu -b_{i}-b_{u})^2 +\lambda(...)$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just in case I leave the link: &lt;a href=&quot;http://courses.ischool.berkeley.edu/i290-dm/s11/SECURE/a1-koren.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25405" LastEditorUserId="25405" LastEditDate="2019-05-05T06:58:08.657" LastActivityDate="2019-05-05T06:58:08.657" Title="Estimating Baselines using ALS" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;linear-algebra&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12137" PostTypeId="2" ParentId="9526" CreationDate="2019-05-05T04:06:17.533" Score="1" Body="&lt;p&gt;The Chinese room argument is such a big deal because it takes the concept of the Turing machine and Turing's conception of the electronic digital computer (so-called) as a practical version of the universal Turing machine, and shows that the resulting concept of machine computation does not allow the creation of an internal semantics (knowledge). Explaining why this is the case is is a bit of a fraught task, though. &lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2019-05-05T04:06:17.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12141" PostTypeId="2" ParentId="12135" CreationDate="2019-05-05T06:42:57.857" Score="2" Body="&lt;p&gt;Convergence guarantees for basic RL algorithms like policy gradient / actor-critic methods make no assumptions about the dynamics of the MDP. So, theoretically, you don't need to change much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically, when the number of possible trajectories from any given state is so high, the return from each state will have high variance. This means you'll have to collect much more experience for your estimates of expected return to converge to their true values. Intuitively, an environment with high uncertainty requires the agent to do more knowledge-gathering to behave optimally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My real advice to you depends on what exactly you're trying to do. If you want to have the kind of agent that &lt;strong&gt;could&lt;/strong&gt; learn to behave well in an extremely random environment, then all you need to worry about is giving it enough experience to learn from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Your agent should also take a little longer before deciding it's &quot;confident&quot; in its evaluation of different states. That is, don't behave greedily before you're sure your estimates are accurate. Explore adequately. This advice is only relevant if your MDP dynamics aren't actually completely uniform.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, however, you want to train an RL agent &lt;strong&gt;specifically&lt;/strong&gt; to solve a problem formulated as an MDP with &lt;strong&gt;uniformly&lt;/strong&gt; random dynamics, then I would tell you to not waste your time. We know before spending the computation that all policies would be equally good/bad in this setting. Since actions are irrelevant to the environment, it would be inefficient to deploy an RL agent that will only learn that which action it takes doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As noted in the comments, the last paragraph is only true when reward from each state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; is also uniformly random. If it is not, just being aware of the high variance and giving your agent a lot of experience should do the trick.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-05-05T08:51:17.900" LastActivityDate="2019-05-05T08:51:17.900" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12142" PostTypeId="1" AcceptedAnswerId="12147" CreationDate="2019-05-05T08:45:40.797" Score="20" ViewCount="1688" Body="&lt;p&gt;With the increasing complexity of reCAPTCHA, I wondered about the existence of some problem, that only a human will ever be able to solve (or that AI won't be able to solve as long as it doesn't reproduce exactly the human brain).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the distorted text used to be only possible to solve by humans. Although...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The computer now got the [distorted text] test right 99.8%, even in the most challenging situations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems also obvious that the distorted text can't be used for real human detection anymore. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also like to know whether an algorithm can be employed for the creation of such a problem (as for the distorted text), or if the originality of a human brain is necessarily needed. &lt;/p&gt;&#xA;" OwnerUserId="25411" LastEditorUserId="2444" LastEditDate="2019-05-05T13:09:20.900" LastActivityDate="2019-06-04T11:34:15.387" Title="Problems that only humans will ever be able to solve" Tags="&lt;agi&gt;&lt;problem-solving&gt;&lt;intelligence-testing&gt;&lt;ai-complete&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12143" PostTypeId="2" ParentId="12135" CreationDate="2019-05-05T08:49:28.517" Score="2" Body="&lt;p&gt;When the next state selection is not driven by any meaningful dynamics i.e. it is independent of starting state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action taken &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, but the rewards received do depend somehow on the &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, then the MDP you describe also fits with something called a &lt;a href=&quot;https://towardsdatascience.com/contextual-bandits-and-reinforcement-learning-6bdfeaece72a&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contextual Bandit Problem&lt;/a&gt; where there is no control over state due to action choice, and thus no incentive to choose actions other than for their potential for immediate reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any algorithm capable of solving a full MDP can also be put to use attempting to solve a contextual bandit problem, as the MDP framework is a strictly more general case of the contextual bandit problem, and can model such an environment. However, this is typically going to be inefficient, as MDP solvers make no assumptions about state transition dynamics and need to experience and learn them. Whilst if you start with an algorithm designed to solve a contextual bandit problem, you have the assumption of randomised state built in to the algorithm, it does not need to be learned, and the learning process should be more efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you only have RL solvers available, you can reduce variance and get the same effective policy by setting discount factor, &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for some reason you still want or need a long-term discounted value prediction from your policy, you can take the mean predicted value of some random states (or even of all the states if there are few enough of them) and multiply by &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{1-\gamma}$&lt;/span&gt; for whatever discount factor you want to know it for. Or if predicting for a time horizon, just multiply by number of steps to the horizon.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-05T17:05:59.460" LastActivityDate="2019-05-05T17:05:59.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12144" PostTypeId="1" CreationDate="2019-05-05T10:05:47.557" Score="6" ViewCount="522" Body="&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;noreferrer&quot;&gt;Prioritized Experience Replay&lt;/a&gt; paper gives two different ways of sampling from the replay buffer. One, called &quot;proportional prioritization&quot;, assigns each transition a priority proportional to its TD-error. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$p_i = |\delta_i|+\epsilon$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other, called &quot;rank-based prioritization&quot;, assigns each transition a priority inversely proportional to its rank.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p_i = 1/\text{rank}(i)$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(i)$&lt;/span&gt; is the rank of transition &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; when the replay buffer is sorted according to &lt;span class=&quot;math-container&quot;&gt;$|\delta_i|$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper goes on to show that the two methods give similar performance for certain problems.&lt;br&gt;&#xA;Are there times when I should choose one sampling method over the other?&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-05-05T10:22:17.427" LastActivityDate="2020-11-26T02:06:51.080" Title="Which kind of prioritized experience replay should I use?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12145" PostTypeId="2" ParentId="12144" CreationDate="2019-05-05T10:05:47.557" Score="5" Body="&lt;p&gt;The authors of that paper hypothesized that rank-based prioritization would be more robust to outliers. They suggested that rank-based sampling would be preferred for this reason. However, as they noted later, the fact that DQN clips rewards anyways weakens this argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're going to use someone else's ready-made code for your prioritized experience replay, then you'll probably end up using proportional sampling. Every implementation I've been able to find, including OpenAI's &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/deepq/replay_buffer.py&quot; rel=&quot;noreferrer&quot;&gt;baselines implementation&lt;/a&gt;, uses proportional sampling. If you were going to write your own, proportional sampling might be preferred for being simpler to implement (so less error-prone).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing these two sampling methods is complicated by the fact that rank-based sampling involves a hyperparameter that determines how often you sort your replay buffer. The authors of the original PER paper only sorted every &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; timesteps, giving the nice amortized time of &lt;span class=&quot;math-container&quot;&gt;$O(\log n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the size of the replay buffer. Sampling takes constant time, so sampling a minibatch of size &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; takes &lt;span class=&quot;math-container&quot;&gt;$O(k)$&lt;/span&gt; time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proportional sampling doesn't involve sorting, but it does need to maintain a &lt;a href=&quot;https://www.fcodelabs.com/2019/03/18/Sum-Tree-Introduction/&quot; rel=&quot;noreferrer&quot;&gt;sum tree&lt;/a&gt; structure, which takes &lt;span class=&quot;math-container&quot;&gt;$O(\log n)$&lt;/span&gt; time each time we add to the buffer. Sampling also takes &lt;span class=&quot;math-container&quot;&gt;$O(\log n)$&lt;/span&gt; time, so sampling a minibatch of size &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; takes &lt;span class=&quot;math-container&quot;&gt;$O(k\log n)$&lt;/span&gt; time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we only sort our replay buffer every &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; timesteps, then rank-based sampling is faster. However, because the buffer is almost always only approximately sorted, the distribution we sample from is only a rough estimate of the distribution we wanted. It's not clear that this estimation would be accurate enough to be performant when the replay buffer is scaled up in size past the &lt;span class=&quot;math-container&quot;&gt;$n=10^6$&lt;/span&gt; transitions used in the paper. I haven't seen a study that compares performance for different frequencies of sorting and different buffer sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, rank-based sampling might be faster, but it also might not work as well. Adjustment of the sorting frequency hyperparameter might be necessary. The simpler and surer approach would be to use proportional sampling with clipped TD-errors.&lt;/p&gt;&#xA;" OwnerUserId="22916" LastEditorUserId="22916" LastEditDate="2019-05-05T10:11:18.867" LastActivityDate="2019-05-05T10:11:18.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12147" PostTypeId="2" ParentId="12142" CreationDate="2019-05-05T13:06:05.197" Score="13" Body="&lt;p&gt;Informally, &lt;a href=&quot;https://en.wikipedia.org/wiki/AI-complete&quot; rel=&quot;noreferrer&quot;&gt;AI-complete&lt;/a&gt; problems are the most difficult problems for an AI. The concept is &lt;a href=&quot;http://www.aaai.org/Papers/Symposia/Spring/2007/SS-07-05/SS07-05-026.pdf&quot; rel=&quot;noreferrer&quot;&gt;not mathematically defined yet&lt;/a&gt;, as e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/NP-completeness&quot; rel=&quot;noreferrer&quot;&gt;NP-complete&lt;/a&gt; problems. However, intuitively, these are the problems that require a human-level or &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;general&lt;/em&gt; intelligence&lt;/a&gt; to be solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Real&lt;/em&gt; natural language understanding is &lt;em&gt;believed&lt;/em&gt; to be an AI-complete problem (this is also discussed in the paper &lt;a href=&quot;https://arxiv.org/abs/1901.02918&quot; rel=&quot;noreferrer&quot;&gt;Making AI Meaningful Again&lt;/a&gt; by Jobst Landgrebe and Barry Smith, 2019). There are a lot more AI-complete problems. For example, problems that involve emotions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="26146" LastEditDate="2019-06-04T11:34:15.387" LastActivityDate="2019-06-04T11:34:15.387" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12148" PostTypeId="5" CreationDate="2019-05-05T13:22:16.370" Score="0" Body="&lt;p&gt;For more info, see &lt;a href=&quot;https://en.wikipedia.org/wiki/AI-complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AI-complete&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-07T15:41:03.447" LastActivityDate="2019-05-07T15:41:03.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12149" PostTypeId="4" CreationDate="2019-05-05T13:22:16.370" Score="0" Body="For questions related to AI-complete (or AI-hard) problems, which, informally, are the problems, in AI, that require a human-level or general intelligence to be solved." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-07T15:41:00.690" LastActivityDate="2019-05-07T15:41:00.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12150" PostTypeId="1" CreationDate="2019-05-05T15:09:38.190" Score="2" ViewCount="146" Body="&lt;p&gt;Given the Chinese room argument, and given the development in chatbots and machine learning, isn't Turing test superseded by some other way of evaluating AI's inteligence? Would an positive result of a Turing test provide any value, besides telling that a machine is good at conversations (but possibly nothing more)?&lt;/p&gt;&#xA;" OwnerUserId="25414" LastActivityDate="2019-05-05T16:36:47.390" Title="Is the Turing test still relevant, as of 2019?" Tags="&lt;turing-test&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12151" PostTypeId="2" ParentId="12150" CreationDate="2019-05-05T16:30:09.873" Score="2" Body="&lt;p&gt;The Turing test is a good test for AI applications like Siri and Alexa (or, in general, intelligent personal assistants), but it doesn't test a lot of features (e.g. vision) that an artificial general intelligence requires. There are several arguments against the Turing test (and it is definitely not flawless), but is it a &lt;em&gt;necessary&lt;/em&gt; (but insufficient) test for AGI. The Turing test is still quite relevant: e.g. if IPAs passed the test, people would use them more often and they would be a lot more useful.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-05T16:36:47.390" LastActivityDate="2019-05-05T16:36:47.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12152" PostTypeId="2" ParentId="7548" CreationDate="2019-05-05T16:58:55.973" Score="2" Body="&lt;p&gt;Jobst Landgrebe and Barry Smith, in the paper &lt;a href=&quot;https://arxiv.org/abs/1901.02918&quot; rel=&quot;nofollow noreferrer&quot;&gt;Making AI meaningful again&lt;/a&gt; (2019), argue that machine learning is not sufficient to build an AI that is able to fully (like humans) understand language. They state that current machine learning (or stochastic) approaches might not take into account a lot of contexts while understanding language (e.g. at machine translation), because of the eventual lack of associated data or because of the big number of possible solutions to a language understanding problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If understanding language is a necessary skill to pass the Turing test, then current machine learning is not sufficient to build an AI that will pass the Turing test (according to them).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-05T17:30:50.730" LastActivityDate="2019-05-05T17:30:50.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12153" PostTypeId="2" ParentId="12126" CreationDate="2019-05-05T17:33:44.780" Score="1" Body="&lt;p&gt;Why not make a detector/classifier to look for the text &quot;Drivers Licence&quot; or some very generic keywords related to license? As you mentioned images of people may be present in any sort of documents. Looking for text which is super specific to IDs/drivers license seem to be a better way. &lt;/p&gt;&#xA;" OwnerUserId="25400" LastActivityDate="2019-05-05T17:33:44.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12154" PostTypeId="1" AcceptedAnswerId="12306" CreationDate="2019-05-05T18:08:03.360" Score="4" ViewCount="150" Body="&lt;p&gt;In my understanding, the mind arises from a physical system, the brain. I see that there is a big research under the topic of simulating physical systems efficiently (especially in quantum computing). Hence, in theory, we could achieve AGI by simulating the physical brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any research I should look into regarding this topic? I would like to hear if it is possible, why, why not, what are the limitaions, how far we are from achieving this, and anything else, really. Also, I would like to read something about my first assumption (&quot;the mind arises from a physical system, the brain&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have searched AI.SE but I've found only related questions, so I don't think this is duplicate. For reference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/6052/why-are-we-asking-how-can-we-simulate-the-brain&quot;&gt;Why are we asking, “How can we simulate the brain?”&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/1314/how-powerful-a-computer-is-required-to-simulate-the-human-brain&quot;&gt;How powerful a computer is required to simulate the human brain?&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/1394/are-there-any-artificial-neuromorphic-systems-which-can-mimic-the-brain&quot;&gt;Are there any artificial neuromorphic systems which can mimic the&#xA;brain?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/2192/is-it-possible-to-build-human-brain-level-artificial-intelligence-based-on-neuro&quot;&gt;Is it possible to build human-brain-level artificial intelligence based on neuromorphic chips and neural networks?&lt;/a&gt; (this one is pretty similar but I think it's slighly different, and it doesn't answer my question).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note: I am not asking for the possibility NOW, but in general, so telling me that &quot;we don't know the brain enough&quot; is not on topic.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-05-05T18:16:01.870" LastActivityDate="2021-01-26T01:40:02.607" Title="Is there any paper, article or book that analyzes the feasibility of acheiving AGI through brain-simulation?" Tags="&lt;agi&gt;&lt;human-like&gt;&lt;brain&gt;&lt;human-inspired&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12156" PostTypeId="2" ParentId="12142" CreationDate="2019-05-05T18:44:56.283" Score="5" Body="&lt;p&gt;This is more of a comment and philosophical opinion, but I don’t believe that there are any problems an AI couldn’t solve, that a human can.  Being new to this forum, I cannot make it a comment on the question (and it would probably be too long) — I preemptively ask for your forgiveness.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;AI Eventually Will Mimic Humans (and surpass them)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Humans by nature are logical. Logic is learned  or hardwired, and influenced by observation and chemical impulses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as an AI can be trained to act like a human, it will be able to act like one. Currently, those behaviors are limited to technology (space, connections, etc), which the human brain has been optimized to rule out or disregard certain “fluff” automatically enabling it certain &lt;em&gt;super&lt;/em&gt; capabilities. For instance, not everything seen is registered through the brain; often, the brain performs differential comparisons and updates for changes to reduce processing time and energy.  It will only be a matter of time before AI can also be programmed to behave this way, or technological advancements will allow it to not need some of this function, which will allow it to leapfrog humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the current state, we recognize humans are sometimes irrational or inconsistent. In those cases, AI could mimic human limitations with configured randomization patterns, but again, there really won’t be a need since it can be programmed and learn those patterns automatically (if necessary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It all comes down to consumption of data, retention of information, and learned corrections. So, there is no problem that a human can perform (to my knowledge) that AI couldn’t theoretically ever perform. Even in the case of chemistry. As we are manufacturing foods and organs, an AI could also, theoretically, one day reproduce and survive via biological functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of the question being binary about human capability vs that of artificial intelligence, I’d be more interested to see what people think are the more challenging things humans can do, which will take AI time to accomplish.&lt;/p&gt;&#xA;" OwnerUserId="25424" LastActivityDate="2019-05-05T18:44:56.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12157" PostTypeId="1" CreationDate="2019-05-05T20:25:17.603" Score="2" ViewCount="17" Body="&lt;p&gt;In section 3 of &lt;a href=&quot;https://web.eecs.umich.edu/~baveja/Papers/IJCAIGSLL2016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; the author outlines how GARB was adapted to reduce the variance in updating parameters to an internal reward function estimator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read it a number of times and understand it up through the end of the explanation of GARB. The author then goes on to explain how they use backprop to implement this procedure, which is the point at which I stop understanding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/InOjt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/InOjt.jpg&quot; alt=&quot;pic1&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wW79K.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wW79K.jpg&quot; alt=&quot;pic2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an open source implementation available to look at? I can’t figure out if &lt;span class=&quot;math-container&quot;&gt;$g_T$&lt;/span&gt; is actually computed and used or not? And not certain how the internal reward gradient is calculated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insight you can provide would be helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; after reading it several times and several related papers, I think I have more understanding but not quite there. So my main questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) are we keeping a full eligibility trace vector with the dimensionality of the vector equal to the number of parameters in &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (all NN params)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) do we use the gradient calculation via backdrop at every step to calculate &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) do we have to maintain &lt;span class=&quot;math-container&quot;&gt;$3 * \theta$&lt;/span&gt; parameters, one for theta, one for &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; and one for &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) what then is the procedure at terminal &lt;span class=&quot;math-container&quot;&gt;$g_T$&lt;/span&gt; to update the parameters? A simple element wise matrix operation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) How often to update the parameters of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2019-05-06T14:38:07.353" LastActivityDate="2019-05-06T14:38:07.353" Title="How is GARB implemented in PGRD-DL to calculate gradients w.r.t. internal rewards?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;rewards&gt;&lt;monte-carlo-methods&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12158" PostTypeId="1" CreationDate="2019-05-05T21:15:03.413" Score="1" ViewCount="34" Body="&lt;p&gt;I'm currently using google speech to text api to transcribe audio in real time (police scanner audio dispatches). The audio quality isn't great and I've been putting in key words to try to help train it. Is there a way to either use google, amazon aws, IBM Watson, to create a model based on past audio dispatches where I can manually type in what was said to help train it? It seems like putting in key words won't really cut it. Any other suggestions to help make it more accurate?  &lt;/p&gt;&#xA;" OwnerUserId="25432" LastActivityDate="2019-05-05T21:15:03.413" Title="Speech to text models" Tags="&lt;machine-learning&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12159" PostTypeId="1" CreationDate="2019-05-06T04:37:58.027" Score="1" ViewCount="174" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;: I have seen lots of people asking whether multiplication and pseudo-random sequences can be approximated by a NN without providing whether the inputs and outputs are bounded or not, and people have answered it (lot of upvotes) based on conventional NN knowledge. without taking into consideration the aforementioned fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; How good/Is it possible by a Neural Network to approximate an unbounded function provided it is trained on a subset of the number line and the test inputs are significantly outside the subset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a Neural Network do regression for an unbounded function? To me it is impossible if the output function is sigmoid, since the best approximation (basis of all signal decomposition and reconstruction schemes) of a function the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier Series&lt;/a&gt; (&lt;span class=&quot;math-container&quot;&gt;$\star$&lt;/span&gt;) demands &lt;a href=&quot;https://gradestack.com/Circuit-Theory-and/Fourier-Series-and/Dirichlet-s-Conditions/19351-3926-40569-study-wtw&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dirichlet's condition&lt;/a&gt; to be satisfied, one of which more or less states that the value should be absolutely integrable (&lt;span class=&quot;math-container&quot;&gt;$\int_{-\infty}^{\infty}|f(x)|^2dx &amp;lt; \infty$&lt;/span&gt;). The sigmoid can be more or less thought in terms of a sinusoidal function as its value is bounded like a sinusoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if the output function used is ReLu then the output is unbounded. But still it is just some linear combination of weights gone through some non-linear functions (in the previous layers which at best might be linearly unbounded if previous layers are ReLu). So one can assume, that even though the Neural Net can approximate an unbounded linear function, can it approximate an unbounded Polynomial function or an exponential function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\star$&lt;/span&gt; Although the regression problem might seem more suitable to Fourier Transform analogy than Fourier Series, I have used the FS analogy based on the fact that FT output is is continuous function as opposed to FS (in NN regression we are adding outputs of several nodes, similar to what we do in FS where &lt;span class=&quot;math-container&quot;&gt;$number_{nodes} &amp;lt;&amp;lt; \infty$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-05-10T15:38:27.227" LastActivityDate="2019-05-10T15:38:27.227" Title="Neural Networks VS Unbounded Outputs" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12160" PostTypeId="1" CreationDate="2019-05-06T06:32:31.823" Score="4" ViewCount="440" Body="&lt;p&gt;While I've been able to solve MountainCar-v0 using Deep Q learning, no matter what I try I can't solve this enviroment using policy-gradient approaches. As far as I learnt searching the web, this is a really hard enviroment to solve, mainly because the agent is given a reward only when it reaches the goal,which is a rare event. I tried to apply the so called &quot;reward engineering&quot;, more or less substituting the reward given by the enviroment with a reward based upon the &quot;energy&quot; of the whole system (kinetic plus potential energy), but despite this no luck.&#xA;I ask you:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is correct to assume that MountainCar-v0 is beyond the current state of the art A3C algorithm, so that it requires some human intervention to suggest the agent the policy to follow, for example adopting reward engineering?&lt;/li&gt;&#xA;&lt;li&gt;could anyone provide any hint about which reward function could be used, provided that reward engineering is actually needed ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="25352" LastActivityDate="2019-10-22T22:01:15.160" Title="A3C fails to solve MountainCar-v0 enviroment (implementation by OpenAi gym)" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12161" PostTypeId="1" AcceptedAnswerId="12180" CreationDate="2019-05-06T08:03:54.360" Score="3" ViewCount="114" Body="&lt;p&gt;There are different kinds of machine learning algorithms, both univariate and multivariate, that are used for time series forecasting: for example ARIMA, VAR or AR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is it harder (compared to classical models like ARIMA) to achieve good results using neural network based algorithms (like ANN and RNN) for multi step time series forecasting?&lt;/p&gt;&#xA;" OwnerUserId="24006" LastEditorUserId="2444" LastEditDate="2019-05-07T13:32:59.060" LastActivityDate="2019-05-07T13:50:51.917" Title="Why is it harder to achieve good results using neural network based algorithms for multi step time series forecasting?" Tags="&lt;deep-learning&gt;&lt;prediction&gt;&lt;comparison&gt;&lt;forecasting&gt;&lt;time-series&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12163" PostTypeId="1" CreationDate="2019-05-06T08:08:29.923" Score="2" ViewCount="60" Body="&lt;p&gt;I read several papers and articles where it is suggested that transposed convolution with 2 strides is better than upsampling then convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However implementing such model with the transposed convolution resulted in heavy checkboard effect, where the whole generated image is just a pattern of squares and no learning takes place. How to properly implement it without totally messing up the generation? With the upsampling+convolution I got okay result but I want to improve my model. I am trying to generate images based on the CelebA dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use keras with tf and I used the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.add(Conv2DTranspose(256, 5, 2, padding='same'))&#xA;&#xA;model.add(LeakyReLU(alpha=0.2))&#xA;&#xA;model.add(BatchNormalization(momentum=0.9))&#xA;&#xA;&#xA;&#xA;model.add(Conv2DTranspose(128, 5, 2, padding='same'))&#xA;&#xA;model.add(LeakyReLU(alpha=0.2))&#xA;&#xA;model.add(BatchNormalization(momentum=0.9))&#xA;&#xA;&#xA;&#xA;model.add(Conv2DTranspose(64, 5, 2, padding='same'))&#xA;&#xA;model.add(LeakyReLU(alpha=0.2))&#xA;&#xA;model.add(BatchNormalization(momentum=0.9))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here I try to turn a 4x4 image into a 32x32. Later it will be turned into a 64x64 image with 1 or 3 channels depending on the image. However I get the following pattern always. Some tweaking usually leads to some other pattern but it does not really change:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.redd.it/oxefl4nwvfw21.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Checkboard effect&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your answers in advance&lt;/p&gt;&#xA;" OwnerUserId="25438" LastActivityDate="2019-05-06T08:08:29.923" Title="Transposed convolution as upsampling in DCGAN" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12167" PostTypeId="1" CreationDate="2019-05-06T13:34:27.300" Score="1" ViewCount="41" Body="&lt;p&gt;I have a python notebook in &lt;strong&gt;IBM Watson Studio&lt;/strong&gt; for a time series forecasting purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it takes two input data&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;MAX &lt;/li&gt;&#xA;&lt;li&gt;MIN&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and it returns two lists of data having 10 steps ahead prediction of both &lt;code&gt;MAX&lt;/code&gt; and &lt;code&gt;MIN&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to make it available for an external application. I have searched for the solution like API creation but didn't get any source. can anyone suggest me the proper way or helpful resources to do it?&lt;/p&gt;&#xA;" OwnerUserId="24006" LastEditorUserId="24006" LastEditDate="2019-10-31T04:44:38.003" LastActivityDate="2019-10-31T04:44:38.003" Title="Deploy notebook in IBM Watson" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;watson&gt;&lt;cloud-services&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12168" PostTypeId="1" AcceptedAnswerId="12175" CreationDate="2019-05-06T14:27:50.723" Score="4" ViewCount="66" Body="&lt;p&gt;So, I have a dataset which has around 1388 unique products and I have to do unsupervised learning on them in order to find anomalies (high/low peaks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data below just represents one product. The &lt;code&gt;ContextID&lt;/code&gt; is the product number, and the &lt;code&gt;StepID&lt;/code&gt; indicates different stages in the making of the product.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    ContextID   BacksGas_Flow_sccm  StepID  Time_ms&#xA;427 7290057 1.7578125   1   09:20:15.273&#xA;428 7290057 1.7578125   1   09:20:15.513&#xA;429 7290057 1.953125    2   09:20:15.744&#xA;430 7290057 1.85546875  2   09:20:16.814&#xA;431 7290057 1.7578125   2   09:20:17.833&#xA;432 7290057 1.7578125   2   09:20:18.852&#xA;433 7290057 1.7578125   2   09:20:19.872&#xA;434 7290057 1.7578125   2   09:20:20.892&#xA;435 7290057 1.7578125   2   09:20:22.42&#xA;436 7290057 16.9921875  5   09:20:23.82&#xA;437 7290057 46.19140625 5   09:20:24.102&#xA;438 7290057 46.19140625 5   09:20:25.122&#xA;439 7290057 46.6796875  5   09:20:26.142&#xA;440 7290057 46.6796875  5   09:20:27.162&#xA;441 7290057 46.6796875  5   09:20:28.181&#xA;442 7290057 46.6796875  5   09:20:29.232&#xA;443 7290057 46.6796875  5   09:20:30.361&#xA;444 7290057 46.6796875  5   09:20:31.381&#xA;445 7290057 46.6796875  5   09:20:32.401&#xA;446 7290057 46.6796875  5   09:20:33.431&#xA;447 7290057 46.6796875  5   09:20:34.545&#xA;448 7290057 46.6796875  5   09:20:34.761&#xA;449 7290057 46.6796875  5   09:20:34.972&#xA;450 7290057 46.6796875  5   09:20:36.50&#xA;451 7290057 46.6796875  5   09:20:37.120&#xA;452 7290057 46.6796875  7   09:20:38.171&#xA;453 7290057 46.6796875  7   09:20:39.261&#xA;454 7290057 46.6796875  7   09:20:40.280&#xA;455 7290057 46.6796875  12  09:20:41.429&#xA;456 7290057 46.6796875  12  09:20:42.449&#xA;457 7290057 46.6796875  12  09:20:43.469&#xA;458 7290057 46.6796875  12  09:20:44.499&#xA;459 7290057 46.6796875  12  09:20:45.559&#xA;460 7290057 46.6796875  12  09:20:45.689&#xA;461 7290057 47.16796875 12  09:20:46.710&#xA;462 7290057 46.6796875  12  09:20:47.749&#xA;463 7290057 46.6796875  15  09:20:48.868&#xA;464 7290057 46.6796875  15  09:20:49.889&#xA;465 7290057 46.6796875  16  09:20:50.910&#xA;466 7290057 46.6796875  16  09:20:51.938&#xA;467 7290057 24.21875    19  09:20:52.999&#xA;468 7290057 38.76953125 19  09:20:54.27&#xA;469 7290057 80.46875    19  09:20:55.68&#xA;470 7290057 72.75390625 19  09:20:56.128&#xA;471 7290057 59.5703125  19  09:20:57.247&#xA;472 7290057 63.671875   19  09:20:58.278&#xA;473 7290057 70.5078125  19  09:20:59.308&#xA;474 7290057 71.875  19  09:21:00.337&#xA;475 7290057 69.82421875 19  09:21:01.358&#xA;476 7290057 69.23828125 19  09:21:02.408&#xA;477 7290057 69.23828125 19  09:21:03.548&#xA;478 7290057 72.4609375  19  09:21:04.597&#xA;479 7290057 73.4375 19  09:21:05.615&#xA;480 7290057 73.4375 19  09:21:06.647&#xA;481 7290057 73.4375 19  09:21:07.675&#xA;482 7290057 73.4375 19  09:21:08.697&#xA;483 7290057 73.4375 19  09:21:09.727&#xA;484 7290057 74.21875    19  09:21:10.796&#xA;485 7290057 75.1953125  19  09:21:11.827&#xA;486 7290057 75.1953125  19  09:21:12.846&#xA;487 7290057 75.1953125  19  09:21:13.865&#xA;488 7290057 75.1953125  19  09:21:14.886&#xA;489 7290057 75.1953125  19  09:21:15.907&#xA;490 7290057 75.9765625  19  09:21:16.936&#xA;491 7290057 75.9765625  19  09:21:17.975&#xA;492 7290057 75.9765625  19  09:21:18.997&#xA;493 7290057 75.9765625  19  09:21:20.27&#xA;494 7290057 75.9765625  19  09:21:21.55&#xA;495 7290057 75.9765625  19  09:21:22.75&#xA;496 7290057 75.9765625  19  09:21:23.95&#xA;497 7290057 76.85546875 19  09:21:24.204&#xA;498 7290057 76.85546875 19  09:21:25.225&#xA;499 7290057 76.85546875 19  09:21:25.957&#xA;500 7290057 76.85546875 19  09:21:26.984&#xA;501 7290057 75.9765625  19  09:21:27.995&#xA;502 7290057 75.9765625  19  09:21:29.2&#xA;503 7290057 76.7578125  19  09:21:30.13&#xA;504 7290057 76.7578125  19  09:21:31.33&#xA;505 7290057 76.7578125  19  09:21:32.59&#xA;506 7290057 76.7578125  19  09:21:33.142&#xA;507 7290057 76.7578125  19  09:21:34.153&#xA;508 7290057 75.87890625 19  09:21:34.986&#xA;509 7290057 75.87890625 19  09:21:35.131&#xA;510 7290057 75.87890625 19  09:21:35.272&#xA;511 7290057 75.87890625 19  09:21:35.451&#xA;512 7290057 76.7578125  19  09:21:36.524&#xA;513 7290057 76.7578125  19  09:21:37.651&#xA;514 7290057 76.7578125  19  09:21:38.695&#xA;515 7290057 76.7578125  19  09:21:39.724&#xA;516 7290057 76.7578125  19  09:21:40.760&#xA;517 7290057 76.7578125  19  09:21:41.783&#xA;518 7290057 76.7578125  19  09:21:42.802&#xA;519 7290057 76.7578125  19  09:21:43.822&#xA;520 7290057 76.7578125  19  09:21:44.862&#xA;521 7290057 76.7578125  19  09:21:45.884&#xA;522 7290057 76.7578125  19  09:21:46.912&#xA;523 7290057 76.7578125  19  09:21:47.933&#xA;524 7290057 76.7578125  19  09:21:48.952&#xA;525 7290057 76.7578125  19  09:21:49.972&#xA;526 7290057 76.7578125  19  09:21:51.72&#xA;527 7290057 77.5390625  19  09:21:52.290&#xA;528 7290057 77.5390625  19  09:21:52.92&#xA;529 7290057 77.5390625  19  09:21:53.361&#xA;530 7290057 77.5390625  19  09:21:54.435&#xA;531 7290057 76.66015625 19  09:21:55.602&#xA;532 7290057 76.66015625 19  09:21:56.621&#xA;533 7290057 72.94921875 22  09:21:57.652&#xA;534 7290057 3.90625 24  09:21:58.749&#xA;535 7290057 2.5390625   24  09:21:59.801&#xA;536 7290057 2.1484375   24  09:22:00.882&#xA;537 7290057 2.05078125  24  09:22:01.259&#xA;538 7290057 2.1484375   24  09:22:01.53&#xA;539 7290057 1.953125    24  09:22:02.281&#xA;540 7290057 1.953125    24  09:22:03.311&#xA;541 7290057 2.1484375   24  09:22:04.331&#xA;542 7290057 2.1484375   24  09:22:05.351&#xA;543 7290057 1.953125    24  09:22:06.432&#xA;544 7290057 1.85546875  24  09:22:07.519&#xA;545 7290057 1.7578125   24  09:22:08.549&#xA;546 7290057 1.85546875  24  09:22:09.710&#xA;547 7290057 1.7578125   24  09:22:10.738&#xA;548 7290057 1.85546875  24  09:22:11.798&#xA;549 7290057 1.953125    24  09:22:12.820&#xA;550 7290057 1.85546875  1   09:22:13.610&#xA;551 7290057 1.85546875  1   09:22:14.629&#xA;552 7290057 1.953125    1   09:22:15.649&#xA;553 7290057 1.85546875  2   09:22:16.679&#xA;554 7290057 1.85546875  2   09:22:17.709&#xA;555 7290057 1.85546875  2   09:22:18.729&#xA;556 7290057 1.953125    2   09:22:19.748&#xA;557 7290057 1.85546875  2   09:22:20.768&#xA;558 7290057 1.7578125   3   09:22:21.788&#xA;559 7290057 1.7578125   3   09:22:22.808&#xA;560 7290057 1.85546875  3   09:22:23.829&#xA;561 7290057 1.953125    3   09:22:24.848&#xA;562 7290057 1.85546875  3   09:22:25.898&#xA;563 7290057 1.953125    3   09:22:27.39&#xA;564 7290057 1.953125    3   09:22:28.66&#xA;565 7290057 1.7578125   3   09:22:29.87&#xA;566 7290057 1.85546875  3   09:22:30.108&#xA;567 7290057 1.7578125   3   09:22:31.129&#xA;568 7290057 1.953125    3   09:22:32.147&#xA;569 7290057 1.85546875  3   09:22:33.187&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use the following code to plot a graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lineplot = X.loc[X['ContextID'] == 7290057]&#xA;x_axis = lineplot.values[:,3]&#xA;y_axis = lineplot.values[:,1]&#xA;&#xA;plt.figure(1)&#xA;plt.plot(x_axis, y_axis)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the &lt;strong&gt;graph:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bIFus.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bIFus.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this graph, the peaks (marked in red circles) are the anomalies which need to be detected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And when I have a graph like this: No anomalies must be caught since there are no undesirable peaks.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/EHaqK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EHaqK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using &lt;code&gt;OneClassSVM&lt;/code&gt;, but I am somehow not satisfied with the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know which unsupervised learning algorithm can be used for such a task at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="23380" LastActivityDate="2019-05-07T08:41:09.563" Title="Which unsupervised learning algorithm can be used for peaks detection?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;unsupervised-learning&gt;&lt;anomaly-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12169" PostTypeId="1" CreationDate="2019-05-06T17:07:15.197" Score="3" ViewCount="40" Body="&lt;p&gt;I would like to train a neural network with the OpenAI Gym Taxi and see how it would react on a new map. Is it possible to insert a new map in the OpenAI Gym Taxi V.2?&lt;/p&gt;&#xA;" OwnerUserId="25448" LastEditorUserId="16565" LastEditDate="2019-05-07T08:26:26.807" LastActivityDate="2019-05-07T08:26:26.807" Title="Is it possible to insert a new map in the OpenAI Gym Taxi V.2?" Tags="&lt;open-ai&gt;&lt;academia&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12171" PostTypeId="1" CreationDate="2019-05-06T19:20:08.353" Score="4" ViewCount="154" Body="&lt;p&gt;I am building a video analytics program for counting moving things in a video. I am detecting bicycles and nothing else. I run object detection using the SSD mobile-net model in all the frames and store the bounding box coordinates (x,y,w,h) of each detection to a CSV file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a video, I have a CSV file of one row each for a frame and each row has multiple detections of D1, D2, D3,.., Dn. Each detection has the bounding box coordinates as values. D1 is x,y,w,h.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the x,y values of each detection, I am trying to find the direction of the bicycles and if the bicycle crosses the whole frame to do a UP/DOWN count.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I count/track(Don't want to use classic tracking algos) these bounding boxes moving in the video?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see LSTM/RNN coming up in my search results when I search for video analytics. Being a noob, I am not able to find any tutorial that suits my needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to check if my approach towards the problem is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to use the classical tracking solutions for two reasons&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I feel the tracking and counting conditions that I program in Python is always leaky/fails in certain conditions, hence I want to see how AI manages to count the objects.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The video stream I am using has heavy distortion on the objects that I track, hence the shape and size of the object changes drastically within 10s/20s of frames.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion towards other better approaches is much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 1: The area of view under the camera is fixed. And we expect the bicycles to move from one entry side. Lets assume that the view and entry/exit is like shown in this video &lt;a href=&quot;https://www.youtube.com/watch?v=tW7Pl3bSzR4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=tW7Pl3bSzR4&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22093" LastEditorUserId="22093" LastEditDate="2019-05-12T18:32:18.570" LastActivityDate="2019-05-13T23:20:58.130" Title="Object IN/OUT counting using CNN+RNN" Tags="&lt;computer-vision&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12172" PostTypeId="1" CreationDate="2019-05-06T20:48:25.060" Score="6" ViewCount="131" Body="&lt;p&gt;I'm interested in the industrial use of GDL (see &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/abs/1611.08097&lt;/a&gt;). Is it used in industry? That is, does any company have access to non-Euclidean data and process it directly instead of converting it to a more standard format?&lt;/p&gt;&#xA;" OwnerUserId="22365" LastEditorUserId="2444" LastEditDate="2019-05-06T20:54:37.843" LastActivityDate="2019-05-16T15:39:40.437" Title="Do you know any examples of geometric deep learning used in industry?" Tags="&lt;applications&gt;&lt;geometric-deep-learning&gt;&lt;real-world&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12173" PostTypeId="1" CreationDate="2019-05-07T05:11:43.027" Score="2" ViewCount="38" Body="&lt;p&gt;I have searched but found that some similarity measures are for continuous data and some are for categorical data. But i want to know the similarity measures which are use for both data, continuous and categorical?&lt;/p&gt;&#xA;" OwnerUserId="23501" LastActivityDate="2019-05-07T05:21:52.430" Title="what are the similarity measure use for both continuous and categorical data?" Tags="&lt;machine-learning&gt;&lt;search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12174" PostTypeId="2" ParentId="12173" CreationDate="2019-05-07T05:21:52.430" Score="1" Body="&lt;p&gt;Sometimes continuous data can be represented as parametric distribution with distribution parameters as variable, essentially continuous stochastic process. In that case cross-entropy would work on that type of continuous data.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-05-07T05:21:52.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12175" PostTypeId="2" ParentId="12168" CreationDate="2019-05-07T08:41:09.563" Score="3" Body="&lt;p&gt;If your anomalies are simply peaks, why should you be using machine learning methods? You could use peak detection algorithms for the purpose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still insist on ML, isolation forest is a good try. &lt;/p&gt;&#xA;" OwnerUserId="25400" LastActivityDate="2019-05-07T08:41:09.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12176" PostTypeId="1" CreationDate="2019-05-07T09:00:29.800" Score="1" ViewCount="21" Body="&lt;p&gt;I want to make 8 fold cross validation from the dataset. The dataset is the musical onsets annotated which has txt format at each songs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3bsjJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3bsjJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to make the another folder like called splits (the name of folder) which is contains 8 fold of songs. Each fold will be written as a txt format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rrELH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rrELH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g2jpM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g2jpM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need some code references to make 8 fold like that. Do I need use sklearn Kfold? And how to save each song like stated in the third picture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18885" LastActivityDate="2019-05-07T09:00:29.800" Title="N fold Cross Validation" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12180" PostTypeId="2" ParentId="12161" CreationDate="2019-05-07T12:44:46.773" Score="3" Body="&lt;p&gt;ANNs &amp;amp; RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyper-parameter optimization&lt;/a&gt;. Because neural networks are so flexible, it is not clear, at the outset, which &lt;a href=&quot;https://cdn-images-1.medium.com/max/2400/1*cuTSPlTq0a_327iTPJyD-Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;arrangement of neurons&lt;/a&gt; will be most effective to solve a given problem. It is also not clear how fast the network should learn from new signals, what sorts of activation functions to use in the different layers of the network, and which of &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/&quot; rel=&quot;nofollow noreferrer&quot;&gt;several possible regularization methods&lt;/a&gt; might be best. Making these decisions well requires either years of practice and experience, or a lot of trial and error (or, maybe both!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, a regression-based method like ARMA will typically have just a couple of simple hyperparameters, each of which has a clear, intuitive, meaning. This means that an untrained practitioner can probably get an ARMA result that is close to the result of a trained practitioner using ARMA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially: neural networks are brittle and sensitive to the choice of hyper-parameters, while regression generally is not.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-05-07T12:44:46.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12182" PostTypeId="1" AcceptedAnswerId="12198" CreationDate="2019-05-07T13:26:01.993" Score="1" ViewCount="1623" Body="&lt;p&gt;What is a temporal feature, what features make something temporal in nature? Is this problem agnostic? How does it change from different fields of study?&lt;/p&gt;&#xA;" OwnerUserId="22093" LastEditorUserId="2444" LastEditDate="2019-05-08T00:02:24.320" LastActivityDate="2019-05-08T00:07:31.807" Title="What is a temporal feature?" Tags="&lt;machine-learning&gt;&lt;definitions&gt;&lt;signal-processing&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12183" PostTypeId="1" CreationDate="2019-05-07T13:41:03.393" Score="3" ViewCount="371" Body="&lt;p&gt;&lt;em&gt;I hope this question is ok here, but since I have found a tag which deals with these issues (profession), I'll ask away. I also hope this may be useful to other people with similar doubts, since I am failing to find valuable information on this topic online.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am interested in the theoretical side of CS, such as computability, logic, complexity theory and formal methods. At the same time, I am deeply fascinated by Artificial Intelligence and the questions it poses to our understanding of the notion of intelligence and what does it mean to be a human being.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, is AI a more &quot;applied&quot;/engineeristic field, or are there theoretical aspects to research in?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short: &lt;strong&gt;If I prefer formal/theoretical compsci, is AI a bad career choice?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note: I am asking this because I am a CS undergrad considering getting into a AI MSc).&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-05-07T13:51:38.157" LastActivityDate="2019-05-08T04:32:16.963" Title="If I am interested in theoretical computer science, is AI a bad choice?" Tags="&lt;profession&gt;" AnswerCount="2" CommentCount="8" ClosedDate="2020-05-04T12:18:14.227" ContentLicense="CC BY-SA 4.0" />
  <row Id="12185" PostTypeId="2" ParentId="12161" CreationDate="2019-05-07T13:50:51.917" Score="2" Body="&lt;p&gt;Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be numerically unstable and statistical inefficient (so convergence might be slow). See &lt;a href=&quot;https://arxiv.org/pdf/1505.02417.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards stability and optimality in stochastic gradient descent&lt;/a&gt; for more details and a solution (AI-SGD).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-05-07T13:50:51.917" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12186" PostTypeId="2" ParentId="12183" CreationDate="2019-05-07T14:14:11.387" Score="2" Body="&lt;p&gt;Any &quot;serious&quot; AI program is full of theoretical and mathematical foundations (you will study plenty of statistics and optimisation methods) anyway, but I would say that much useful AI today is an applied or engineering area. Anyhow, you will need to be comfortable with a lot of mathematical details (especially, linear algebra and calculus). If you're more interested in statistics, optimisation or robotics, you should go for AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you study &quot;pure&quot; computer science, you should also have one or two courses related to AI (at least, one ML course). If you are more interested in traditional CS algorithms, data structures, software engineering, operating systems, compilers, theory of computation, computer networking, programming languages and/or databases, then you should go for CS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, before enrolling in a master's program, you should really have a look at the details of the courses they offer. Furthermore, you might also take into account that, during your studies, you might change idea regarding one subject.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-07T14:46:09.953" LastActivityDate="2019-05-07T14:46:09.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12187" PostTypeId="1" AcceptedAnswerId="12192" CreationDate="2019-05-07T14:53:05.343" Score="3" ViewCount="880" Body="&lt;p&gt;How fast does Monte Carlo Tree Search converge? Is there proof that it does converge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does it compare to Temporal Difference learning in terms of convergence speed (assuming the evaluation step is a bit slow)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to exploit the information gathered during the simulation phase to accelerate MCTS?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry if too many questions, if you have to choose one, please choose the last question.&lt;/p&gt;&#xA;" OwnerUserId="23866" LastEditorUserId="2444" LastEditDate="2019-06-10T19:01:25.867" LastActivityDate="2019-06-10T19:01:25.867" Title="How fast does Monte Carlo tree search converge?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;comparison&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12189" PostTypeId="1" AcceptedAnswerId="12190" CreationDate="2019-05-07T15:50:03.360" Score="2" ViewCount="405" Body="&lt;p&gt;Is reinforcement learning problem adaptable to the setting when there is only one - final - reward. I am aware of problems with sparse and delayed rewards, but what about only one reward and a quite long path?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2020-11-02T22:02:13.713" LastActivityDate="2020-11-03T13:37:38.540" Title="Can reinforcement learning be used for tasks where only one final reward is received?" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;sparse-rewards&gt;&lt;credit-assignment-problem&gt;&lt;delayed-rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12190" PostTypeId="2" ParentId="12189" CreationDate="2019-05-07T15:59:34.377" Score="10" Body="&lt;p&gt;RL can be used for cases where you have &lt;strong&gt;sparse rewards&lt;/strong&gt; (i.e. at almost every step all rewards are zero), but, in such a setting, the experience the agent receives during the trajectory does not provide much information regarding the quality of the actions.&lt;/p&gt;&#xA;&lt;p&gt;Games can be often formulated as episodic tasks. For example, you could formulate a chess match as an episode and you could give a (non-zero) reward only at the end of the match. However, in this specific case, it will be hard for the RL to understand which moves have mainly contributed to the reward received, which is known as the &lt;a href=&quot;https://ai.stackexchange.com/q/12908/2444&quot;&gt;credit assignment problem&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can solve the issue of sparse rewards with &lt;strong&gt;reward shaping&lt;/strong&gt; (in particular, &lt;a href=&quot;https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;potential-reward shaping&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The term &amp;quot;delayed rewards&amp;quot; may also refer to the cases where you receive only one reward at the end of the episode, although it may more usually refer to scenarios where the reward at one-time step is only received later (for some reason).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-03T13:37:38.540" LastActivityDate="2020-11-03T13:37:38.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12191" PostTypeId="1" CreationDate="2019-05-07T17:23:47.053" Score="3" ViewCount="293" Body="&lt;p&gt;Apologies for the lengthy title.&#xA;My question is about the weight update rule for logistic regression using stochastic gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have just started experimenting on Logistic Regression. I came across two weight update expressions and did not know which one is more accurate and why they are different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The first Method:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: (Book) Artificial Intelligence: A Modern Approach by Norvig, Russell&#xA; on page 726-727:&#xA;using the L2 loss function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QA8O9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QA8O9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FwVLC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FwVLC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&#xA;g stands for the logistic function&#xA;g' stands for g's derivative&#xA;w stands for weight&#xA;hw(x) represents the logistic regression hypothesis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The other method:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source (Paper authored by Charles Elkan): Logistic Regression and Stochastic Gradient Training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be found &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.3611&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZCpRd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZCpRd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25463" LastEditorUserId="1671" LastEditDate="2019-10-15T19:24:20.640" LastActivityDate="2019-10-15T19:24:20.640" Title="What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12192" PostTypeId="2" ParentId="12187" CreationDate="2019-05-07T19:38:23.603" Score="3" Body="&lt;p&gt;Yes, Monte Carlo tree search (MCTS) has been proven to converge to optimal solutions, under assumptions of infinite memory and computation time. That is, at least for the case of perfect-information, deterministic games / MDPs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe some other problems were covered too by some proofs (I could intuitively imagine the proofs holding up for non-deterministic games as well, depending on implementation details)... but the classes of problems I mentioned above are what I'm sure about. The initial, classic proofs can be found in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sztaki.hu/%7Eszcsaba/papers/ecml06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit based Monte-Carlo Planning&lt;/a&gt; by Levente Kocsis, Csaba Szepesvári&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sztaki.hu/~szcsaba/papers/cg06-ext.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improved Monte-Carlo Search&lt;/a&gt; by Levente Kocsis, Csaba Szepesvári, Jan Willemson&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Much more recently the paper &lt;a href=&quot;https://arxiv.org/abs/1902.05213&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Reinforcement Learning Using Monte Carlo Tree Search with Supervised Learning: Non-Asymptotic Analysis&lt;/a&gt; appeared on arXiv, in which I saw it is mentioned that there may have been some flaws in those original papers, but they also seem to be able to fix it and add more theory for the more &quot;modern&quot; variants which combine (deep) learning approaches inside MCTS.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It should be noted that, as is typically the case, all those convergence proofs are for the case where you spend an infinite amount of time running your algorithm. In the case of MCTS, you can intuitively think of the proofs only starting to hold once your algorithm has manage to build up the &lt;strong&gt;complete search tree&lt;/strong&gt;, and then on top of that had sufficient time to run through &lt;strong&gt;all the possible paths in the tree&lt;/strong&gt; sufficiently often for the correct values to backpropagate. This is unlikely to be realistic for most interesting problems (and if it is feasible, a simpler breadth-first search algorithm may be a better choice).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How does it compare to Temporal Difference learning in terms of convergence speed (assuming the evaluation step is a bit slow)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you're thinking of a standard, tabular TD learning approach like Sarsa... &lt;a href=&quot;https://www.jair.org/index.php/jair/article/view/11099&quot; rel=&quot;nofollow noreferrer&quot;&gt;such approaches actually turn out to be very closely related to MCTS&lt;/a&gt;. In terms of convergence speed, I'd say the important differences are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MCTS focusses on &quot;learning&quot; for a single state, the root state; all efforts are put towards obtaining an accurate value estimate for that node (and its direct children), whereas typical TD implementations are about learning immediately for the complete state-space. I suppose the &quot;focus&quot; of MCTS could improve its convergence speed for that particular state...&lt;/li&gt;&#xA;&lt;li&gt;but the fact that the search tree (which can be viewed as its &quot;table&quot; of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values as you'd see in Sarsa or &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning) only slowly grows can also be a disadvantage, in comparison to tabular TD learning approaches which start out with a complete table that covers the complete state space.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that papers such as the last one I linked above show how MCTS can also actually use Temporal Difference learning for its backing up of values through the tree... so looking at it from a &quot;MCTS vs TD learning&quot; angle doesn't really make too much sense when you consider that TD learning can be used inside MCTS.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a way to exploit the information gathered during the simulation phase to accelerate MCTS?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are lots and lots of ideas like that tend to improve performance empirically. It will be difficult to say much about them in theory though. Some examples off the top of my head:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All Moves As First (AMAF)&lt;/li&gt;&#xA;&lt;li&gt;Rapid Action Value Estimation (RAVE, also see GRAVE)&lt;/li&gt;&#xA;&lt;li&gt;Move Average Sampling Technique (MAST)&lt;/li&gt;&#xA;&lt;li&gt;N-Gram Selection Technique (NST)&lt;/li&gt;&#xA;&lt;li&gt;Last-Good-Reply policy&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many of them can be found in &lt;a href=&quot;http://mcts.ai/pubs/mcts-survey-master.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this survey paper&lt;/a&gt;, but it is somewhat old now (from 2012), so it doesn't include all the latest stuff.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-05-07T19:38:23.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12195" PostTypeId="2" ParentId="5774" CreationDate="2019-05-07T21:06:39.653" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X9Pih.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X9Pih.gif&quot; alt=&quot;Animation of backpropagation with stride &amp;gt; 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backpropagation with stride &gt; 1 involves dilation of the gradient tensor with stride-1 zeroes. I created a &lt;a href=&quot;https://medium.com/@mayank.utexas/backpropagation-for-convolution-with-strides-8137e4fc2710&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; that describes this in greater detail.&lt;/p&gt;&#xA;" OwnerUserId="25475" LastActivityDate="2019-05-07T21:06:39.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12196" PostTypeId="2" ParentId="12183" CreationDate="2019-05-07T21:32:27.167" Score="3" Body="&lt;p&gt;There are certainly results in theoretical computer science / pure math with deep implications for AI. But to my knowledge these results typically aren't labeled as results of artificial intelligence, but as something more congruent in that particular field (For example in CS, we might say &quot;agent with unbounded computational power&quot;; in math might say some statement is &quot;decidable/undecidable&quot; with respect to some system). Of course they still matter in the field of AI, but you need to know what you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See my question &lt;a href=&quot;https://ai.stackexchange.com/questions/3209/what-are-some-implications-of-g%C3%B6dels-theorems-on-ai-research&quot;&gt;What are some implications of Gödel's theorems on AI research?&lt;/a&gt; for some examples. Or you can look up &lt;a href=&quot;https://intelligence.org/research-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIRI's research guide&lt;/a&gt; for a better idea of what existing work is out there that links formal math / CS to AI research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point to raise is that there is no good definition of AI in fields outside of normal discourse (or even within, perhaps), so its difficult to decide what discussions pertains to the study of AI. Questions like whether ZFC with/without choice is expressive enough might not be on the mind of most AI researchers, but could still have some implications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer you question more directly, there is certainly a field of study regarding theoretical AI. Regarding whether or not its a good choice is something for you to decide, but it is (in my humble and not-very-well-educated opinion) very difficult field that isn't very popular, and has not seen major progress in many years.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2019-05-08T04:32:16.963" LastActivityDate="2019-05-08T04:32:16.963" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12197" PostTypeId="2" ParentId="11517" CreationDate="2019-05-07T22:04:12.257" Score="2" Body="&lt;p&gt;Although there seems to be an apt analogy between Gödel's theorems and the PSHH, there is nothing formal linking the two together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More concretely, Gödel's theorems are about systems that decide certain &quot;truths&quot; about mathematics, but unless I am mistaken, the PSSH doesn't imply that the symbol system of the mind needs to decide truths. Though implicitly us humans do decide facts about math, there isn't a formal interpretation of how that might be done in the PSHH, thus Gödel's theorems do not apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;a href=&quot;https://ai.stackexchange.com/a/11529/2444&quot;&gt;this answer&lt;/a&gt; is still good, under the assumption that the formal system we are talking about does indeed decide certain truths about math.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2020-05-13T10:13:13.470" LastActivityDate="2020-05-13T10:13:13.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12198" PostTypeId="2" ParentId="12182" CreationDate="2019-05-08T00:01:23.177" Score="2" Body="&lt;p&gt;In general, the expression &amp;quot;temporal feature&amp;quot; might refer to any feature that is associated with or changes over time.&lt;/p&gt;&#xA;&lt;p&gt;However, in the context of signal processing, a temporal feature might refer to any feature of the data before being transformed to the &lt;a href=&quot;https://dsp.stackexchange.com/a/56595/40095&quot;&gt;Fourier, frequency or spectral&lt;/a&gt; domain, using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier transform&lt;/a&gt;. In this context, the domain of the untransformed data is often called &amp;quot;time domain&amp;quot; (as opposed to the &amp;quot;frequency&amp;quot; or &amp;quot;spectral&amp;quot; domain, which is the domain of the transformed data), even though it might not be strictly associated with or defined as a function of time. For example, in image processing, an image can be interpreted as a 2D signal. The domain of an image can be referred to as the &amp;quot;time domain&amp;quot;, even though it is usually and more correctly referred to as the &amp;quot;spatial domain&amp;quot; (given that an image can be thought of as a function from a pixel, which is defined by two numbers &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, to a value, e.g. a grayscale value). You can transform this image, using the Fourier transform, to the spectral domain. In that case, the domain of the result of the transformation can be referred to as the &amp;quot;spectral domain&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;http://m.mr-pc.org/ismir16/website/articles/159_Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Temporal Features Using a Deep Neural Network and its Application to Music Genre Classification&lt;/a&gt;, the authors define spectral and temporal features&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Extracting features from audio that are relevant to the task at hand is a very important step in many &lt;em&gt;music information retrieval&lt;/em&gt; (MIR) applications, and the choice of features has a huge impact on the performance. For the past decades, numerous features have been introduced and successfully applied to many different kinds of MIR systems. These audio features can be broadly categorized into two groups: 1) spectral and 2) temporal features.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Spectral features&lt;/em&gt; (SFs) represent the spectral characteristics of music in a relatively short period of time. In a musical sense, it can be said to reveal the timbre or tonal characteristics of music. Some of popular SFs include: spectral centroid, spectral spread, spectral flux, spectral flatness measure, mel-frequency cepstral coefficients (MFCCs) and chroma.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, &lt;em&gt;temporal features&lt;/em&gt; (TFs) describe the relatively long-term dynamics of a music signal over time such as temporal transition or rhythmic characteristics. These include zero-crossing rate (ZCR), temporal envelope, tempo histogram, and so on.&lt;/p&gt;&#xA;&lt;p&gt;The two groups are not mutually exclusive, however, and many MIR applications use a combination of many different features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These definitions are given in the context of signal processing (as mentioned above).&lt;/p&gt;&#xA;&lt;p&gt;To conclude, the meaning of &amp;quot;temporal feature&amp;quot; might change depending on the context. Hence, you should interpret it given the context, but it is almost always associated with time (in some way).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-05-08T00:07:31.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12199" PostTypeId="5" CreationDate="2019-05-08T00:03:09.580" Score="0" Body="&lt;p&gt;For more info, have a look e.g. at &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Feature_(machine_learning)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-08T23:01:21.217" LastActivityDate="2019-05-08T23:01:21.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12200" PostTypeId="4" CreationDate="2019-05-08T00:03:09.580" Score="0" Body="For questions related to features in the context of machine learning and, in general, AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-08T23:01:18.663" LastActivityDate="2019-05-08T23:01:18.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12201" PostTypeId="2" ParentId="5085" CreationDate="2019-05-08T13:31:47.927" Score="0" Body="&lt;p&gt;I think your intuition about a classifier being the wrong approach is a good one.  This looks like a great use-case for word vectors, a &quot;self-supervised&quot; learning technique that maps tokens (e.g. &quot;dog&quot;) to vectors (which might have anywhere between 50 - 500 dimensions).  Facebook open-sourced a particular excellent tool for training word vectors &lt;a href=&quot;https://github.com/facebookresearch/fastText&quot; rel=&quot;nofollow noreferrer&quot;&gt;called FastText&lt;/a&gt;; you could use this to embed tokens and hashtags alike into a word embedding space.  You should find that the vector for &quot;dog&quot; ends &quot;close to&quot; (small cosine distance).  Given a word, you can easily look up its vector (after training on your corpus, of course), but how to find other vectors that are close to it?  If you want to do better than &quot;brute force&quot; and you need to check against a large number of (vectors for) hashtags, you should consider using &lt;a href=&quot;https://github.com/facebookresearch/faiss&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facebook's excellent FAISS library for fast similarity search&lt;/a&gt; to find the closest hashtags.&lt;/p&gt;&#xA;" OwnerUserId="17770" LastActivityDate="2019-05-08T13:31:47.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12202" PostTypeId="2" ParentId="5085" CreationDate="2019-05-08T13:35:43.560" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Here is a good approach to achieve the task you want:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1-&lt;/strong&gt; Compute the Vector representation (i.e embeddings) of all the words you want to include. There are many algorithms out there to achieve this task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ifqc4m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ifqc4m.png&quot; alt=&quot;Step 1- Compute Embeddings&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2-&lt;/strong&gt; Choose the #words corresponding to your input word (e.g dog) by applying K-Nearest Neighbors (KNN) or similar algorithms. You basically compute the distances using the embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jI8Ptm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jI8Ptm.jpg&quot; alt=&quot;Step 2 - Apply KNN&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Steps Detailed:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1-&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In NLP we represent human language as a vector of values instead of a set of characters in order to process it. To do so there are 3 approaches in the literature:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;- Word Level Embeddings:&lt;/strong&gt; Represent each word as a vector of values.&#xA;   Algorithms: &lt;a href=&quot;https://medium.freecodecamp.org/how-to-get-started-with-word2vec-and-then-how-to-make-it-work-d0a2fca9dad3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Word2Vec by Google&lt;/a&gt; (&lt;a href=&quot;https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;), &lt;a href=&quot;https://blog.manash.me/how-to-use-pre-trained-word-vectors-from-facebooks-fasttext-a71e6d55f27&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastText by Facebook&lt;/a&gt;, &lt;a href=&quot;https://nlp.stanford.edu/projects/glove/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GloVe by Stanford University&lt;/a&gt; (&lt;a href=&quot;https://nlp.stanford.edu/pubs/glove.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;- Character Level Embeddings:&lt;/strong&gt; Represent each character as a vector of values. Algorithms: &lt;a href=&quot;https://towardsdatascience.com/elmo-contextual-language-embedding-335de2268604&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELMo&lt;/a&gt; (&lt;a href=&quot;https://arxiv.org/abs/1802.05365&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;- Sentence Level Embeddings:&lt;/strong&gt; Represent a sentence as a vector of values. Algorithms: &lt;a href=&quot;https://tfhub.dev/google/universal-sentence-encoder/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Sentence Encoder by Google&lt;/a&gt; (&lt;a href=&quot;https://arxiv.org/abs/1803.11175&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case I suggest to use &lt;a href=&quot;https://nlp.stanford.edu/projects/glove/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GloVe&lt;/a&gt; or &lt;a href=&quot;https://towardsdatascience.com/elmo-contextual-language-embedding-335de2268604&quot; rel=&quot;nofollow noreferrer&quot;&gt;ElMo&lt;/a&gt; if you have only words and &lt;a href=&quot;https://tfhub.dev/google/universal-sentence-encoder/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Sentence Encoder&lt;/a&gt; if you have words and sentences.  . Compute all your words embeddings and move to the next step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2-&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that you have your embeddings, compute the distances between all your words &lt;em&gt;(use Euclidian, Minkowski or any other distance)&lt;/em&gt;. Notice that the computation may take some time but will only be executed once. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now each time you have a word (e.g dog) you apply the &lt;strong&gt;KNN&lt;/strong&gt; algorithm using the computed distances and you will get the most related words to this word. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: No need to compute distances and apply KNN if you use Universal Sentence Encoder as the similarity is easily computed using a dot product of the embeddings. See my quick implementation example &lt;a href=&quot;https://ai.stackexchange.com/questions/4965/sentence-similarity-in-python/12232#12232&quot;&gt;here&lt;/a&gt; for details.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="23350" LastEditDate="2019-05-24T11:02:01.823" LastActivityDate="2019-05-24T11:02:01.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12203" PostTypeId="2" ParentId="10406" CreationDate="2019-05-08T13:42:06.687" Score="0" Body="&lt;p&gt;It is the combination of the output of all neurons that determines the output of the neural network. In the case of convolutional neural networks (CNNs), the term &quot;feature&quot; is used because it is associated with the &lt;a href=&quot;https://qr.ae/TWIcfO&quot; rel=&quot;nofollow noreferrer&quot;&gt;feature maps&lt;/a&gt; (or &lt;a href=&quot;https://stats.stackexchange.com/a/292064/82135&quot;&gt;activation maps&lt;/a&gt;) and &lt;a href=&quot;https://qr.ae/TWIcfL&quot; rel=&quot;nofollow noreferrer&quot;&gt;filters&lt;/a&gt; (or kernels) of the CNN. However, this terminology might not be accurate (because these might not be features in the intuitive sense) and it is used more to &lt;em&gt;interpret&lt;/em&gt; the inner workings of the CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training, the weights of the neural network are fixed (unless you perform online and continual training), so the output of each neuron will be the same given the same input, thus the output of the neural network will also be the same (unless there is some random operation being performed). During training, the weights and thus the output of each neuron (and of the neural network) often change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The contribution of each neuron to the output of the neural network is determined by the weights of the connections between the neurons, which change during training and can be initialised in different ways, which might affect differently the final weights (after training).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There several &lt;a href=&quot;http://cs231n.github.io/understanding-cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ways of visualising the contribution of each neuron to the output of the neural network&lt;/a&gt;. See also this article &lt;a href=&quot;https://it.mathworks.com/help/deeplearning/examples/visualize-features-of-a-convolutional-neural-network.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visualize Features of a Convolutional Neural Network&lt;/a&gt; and the paper &lt;a href=&quot;https://arxiv.org/pdf/1311.2901.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visualizing and Understanding Convolutional Networks&lt;/a&gt; (2013).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-08T15:37:40.790" LastActivityDate="2019-05-08T15:37:40.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12205" PostTypeId="2" ParentId="11575" CreationDate="2019-05-08T14:16:42.697" Score="2" Body="&lt;p&gt;From what I understood you will not have any cold start problem because you basically process the user preferences description against movies descriptions to get recommendations. So you don't use other users feedback at any time of the process which is not collaborative filtering.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here is instead the approach I would suggest in your case to get movies recommendations for each user:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the similarity between the user description and each movie description. This can be done using the &lt;strong&gt;&lt;a href=&quot;https://tfhub.dev/google/universal-sentence-encoder/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Sentence Encoder&lt;/a&gt;&lt;/strong&gt;. It's an 2018 &lt;a href=&quot;https://arxiv.org/abs/1803.11175&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; by Google which represents any sentence as a vector of 215 values &lt;em&gt;(i.e embeddings)&lt;/em&gt;. &#xA;The semantic similarity between 2 sentences is computed using the dot product of their embeddings. Fortunately, the implementation was integrated to Tensorflow Hub and easily be used (see &lt;a href=&quot;https://ai.stackexchange.com/questions/4965/sentence-similarity-in-python&quot;&gt;my answer here&lt;/a&gt; for details).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose the highest similarity values and recommend the corresponding movies to the user. Notice that you can still use this approach along with a collaborative filtering one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="23350" LastEditDate="2019-05-09T21:27:16.143" LastActivityDate="2019-05-09T21:27:16.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12206" PostTypeId="1" AcceptedAnswerId="12207" CreationDate="2019-05-08T15:05:25.370" Score="0" ViewCount="350" Body="&lt;p&gt;In Sutton's RL:An introduction 2nd edition it says the following(page 203):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;State aggregation is a simple form of generalizing function&#xA;  approximation in which states are grouped together, with one estimated&#xA;  value (one component of the weight vector w) for each group. The value&#xA;  of a state is estimated as its group’s component, and when the state&#xA;  is updated, that component alone is updated. State aggregation is a&#xA;  special case of SGD (9.7) in which the gradient, rv̂(S t ,w t ), is 1&#xA;  for S t ’s group’s component and 0 for the other components.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and follows up with a theoretical example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, imagining my original state space is [1,100000], why can't I just say that the new state space is [1, 1000] where each of these numbers corresponds to an interval: so 1 to [1,100], 2 to [101,200], 3 to [201,300], and so on, and then just apply the normal TD[0] formula, instead of using the weights?     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main problem with their approach is the last sentence:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;in which the gradient, rv̂(S t ,w t ), is 1&#xA;  for St’s group’s component and 0 for the other components.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;if v(s,w) is the linear combination of a feature vector and the weights (w), how does the gradient of that function can be 1 for a state and 0 for others? There are not as many w as states or groups of states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g.: if my feature vector is 5 numbers between 0 and 100. For example (55,23,11,44,99) for a specific state, how do you choose a specific group of states for state aggregation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe what I'm not understanding is the feature vector. If we have a state space that is[1,10000] as in the random walk, what can be the feature vector? does it have the same size as the number of groups after state aggregation?&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-05-08T15:55:05.107" LastActivityDate="2019-05-08T16:10:16.210" Title="State aggregation methods" Tags="&lt;reinforcement-learning&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12207" PostTypeId="2" ParentId="12206" CreationDate="2019-05-08T16:03:44.177" Score="1" Body="&lt;p&gt;Using the book's random walk example, If you have a state space with &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; states and you divide them into &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; groups, each of those groups will have &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; neighboring states. The function for approximation will be&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;v(\mathbf w) = x_1w_1 + x_2w_2 + ... + x_{10}w_{10}&#xA;\end{equation}&lt;/span&gt;&#xA;Now, when you pick a state, the feature vector will be a one-hot encoded vector with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; that is placed in a position that depends on in which group does the chosen state belong. For example if you have state &lt;span class=&quot;math-container&quot;&gt;$990$&lt;/span&gt; that state belongs in group &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; so the feature vector will be&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mathbf x_t = [0, 0, ..., 0, 1]^T&#xA;\end{equation}&lt;/span&gt;&#xA;what this means is that the only weight that will be updated is weight &lt;span class=&quot;math-container&quot;&gt;$w_{10}$&lt;/span&gt; because gradients for all other weights will be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (that's because features for those weights are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;)&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-05-08T16:10:16.210" LastActivityDate="2019-05-08T16:10:16.210" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12208" PostTypeId="2" ParentId="1508" CreationDate="2019-05-08T16:16:24.017" Score="3" Body="&lt;p&gt;You should look at &lt;a href=&quot;https://arxiv.org/pdf/1506.03134.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pointer networks&lt;/a&gt;. It is still not perfect for the case, but it should be more applicable than LSTMs and MLPs because they learn in an output space of size equal to the input, rather than a fixed input dim that you would get using LSTMs in sequence to sequence or direct MLP. By design though they are meant for problems with replacement. Sorting when done sequentially is without, so to remedy this in the case of a pointer network, you could mask outputs that have already been chosen before the final normalization step (such as softmax)&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2019-05-13T15:44:52.717" LastActivityDate="2019-05-13T15:44:52.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12209" PostTypeId="1" AcceptedAnswerId="12211" CreationDate="2019-05-08T16:39:42.997" Score="1" ViewCount="42" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a robotic arm controlled by a DQN + a python script I wrote.&#xA;The DQN receives the 5 joint states, the coordinates of a target, the coordinates of the obstacle and outputs the best action to take (in terms of joint rotations).&#xA;The python script checks if the action suggested by the DQN is safe. If it is, it performs it. Otherwise, it performs the second highest-ranking action from the DQN; and so on. If no action is possible, collision: we fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During training, this python functionality wasn't present: the arm learned how to behave without anything else to correct his behaviour. With this addition on the already-trained network, the performance raised from 78 to 95%.&#xA;Now my advisor (bachelor's thesis) asked me to leave the external controller on during training to check whether this improves learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what happens during training; at each step:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the ANN selects an action&lt;/li&gt;&#xA;&lt;li&gt;if it is legal, the python script executes it, otherwise it chooses another one.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now... On which action should I perform backprop? The one proposed by the arm or the one which was really executed? (So, which action should I perform backprop on?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am really confused. On the one hand, the arm &lt;strong&gt;did choose&lt;/strong&gt; an action so my idea was that we should, in fact, learn on THAT action. On the other hand, during the exploration phase (&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; greedy), we backprop on the action which was randomly selected and executed, with no interest on what was the output of the arm. So, it would be rational too, in this case, to perform backprop on the action really executed; so the one chosen by the script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the right thing to do here?&lt;/strong&gt;. &lt;em&gt;(Bonus question: is it reasonable to train with this functionality on? Wouldn't it be better to train the Network by itself, and then later, enhance its performance with this functionality?)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-05-08T16:47:10.813" LastActivityDate="2019-05-08T18:19:09.347" Title="DQN Agent helped by a controller: on which action should I perform backprop?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12210" PostTypeId="1" CreationDate="2019-05-08T16:58:41.877" Score="1" ViewCount="29" Body="&lt;p&gt;So when using semi-gradient td(0) you need to convert your state representation into a feature vector that represents the state and as far as I know, should not be correlated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the input on the ANN of a DQN the same? should it be a feature vector that represents the state? What considerations should one have when creating such vector?&lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2019-05-08T16:58:41.877" Title="DQN ANN input vs Linear function approximator feature vector" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12211" PostTypeId="2" ParentId="12209" CreationDate="2019-05-08T18:19:09.347" Score="2" Body="&lt;p&gt;Q-learning - which DQN is based on - is an off-policy reinforcement learning (RL) method. That means it can learn a &lt;em&gt;target policy&lt;/em&gt; of optimal control whilst acting using a different &lt;em&gt;behaviour policy&lt;/em&gt;. In addition, provided you use single step learning (as opposed to n-step or Q(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) variants), you don't need to worry much about the details of the behaviour policy. It is more efficient to learn from behaviour policies closer to the current best guess at optimal, but possible to learn from almost anything, including random behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it doesn't really matter too much if you change the behaviour during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the script is actually doing more than just changing the behaviour. It is consistently filtering out state/action pairs that you have decided should never be taken, as a domain expert. This has two major consequences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It reduces the search space by whatever fraction of state/actions are now denied by your safety script.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It prevents the agent from ever learning about certain state/action pairs, as they are never experienced.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first point means that your learning should in theory be more efficient. As for how much, you will have to try and see. It might only be a small amount if the problem states and actions are unlikely to be reached during exploration from near-optimal behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second point means that your agent will never learn by itself to avoid the problem state/action combinations. So you will always need to use the safety script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact you can view the safety script as a modification to the environment (as opposed to a modification to the agent), if its decisions are strict and consistent. Filtering available actions is a standard mechanism in RL when action space may vary depending on state.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On which action should I perform backprop? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In DQN, you don't &quot;perform backprop&quot; on an action. Instead you either use directly or store some observed data about the step: &lt;span class=&quot;math-container&quot;&gt;$s, a, r, s'$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the start state, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; the action taken, &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; the immediate reward, and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; the resulting state. You then update the the current action value estimate(s) based on a TD target &lt;span class=&quot;math-container&quot;&gt;$\delta = r + \gamma \text{max}_{a'} Q(s,a')$&lt;/span&gt; either online or from the experience table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Q-learning learns, it updates the estimate for the action value &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; - and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is taken from the actual behaviour (otherwise the agent will update its estimate of an action that it didn't take). So your answer here is to use - or more likely store in the experience table - the action actually taken. If the action recommended as optimal at that time is different, ignore the difference, what matters is the observed experience.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-05-08T18:19:09.347" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12213" PostTypeId="1" CreationDate="2019-05-08T20:50:07.520" Score="1" ViewCount="21" Body="&lt;p&gt;I have some variable length input vectors for my own use case of a 'stylistic transfer'-esque process, and I am wondering if anyone knows of a way to engineer an input that maps to a 0 element in embedding space. This would be an element that simply holds space but would be readily overlaid with vector addition of another embedded input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My rationale is that I could pad the inputs with these zero elements to mask what I don't care about and have a semantically meaningful vector addition in the embedding space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if I could permute some training examples with a chosen value which all map to the same output and this would allow a neural net to learn such a feature.&lt;/p&gt;&#xA;" OwnerUserId="12849" LastActivityDate="2019-05-08T20:50:07.520" Title="Creating a zero element in embedding space" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;word-embedding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12214" PostTypeId="1" CreationDate="2019-05-08T22:06:36.610" Score="1" ViewCount="114" Body="&lt;p&gt;I am making a school project where I should develop any kind of game where I can have one reactive agent and one agent based on machine learning competing with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My game consists of a salesmen problem. Basically, I have 3 types of entities, consumers, salesmen, and hotspots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The consumers are represented by the person with a green background. There are 8 of them. They basically move around the whole game using random walks and they tend o aggregate on the HotSpots (the orange icon with the router in it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The salesmen are represented by the person with the dark grey background. One of them is controlled by a reactive agent that has in it some rules that I programmed and the other one is controlled by my DQN model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The salesmen have 5 available actions, move &lt;strong&gt;up&lt;/strong&gt;, &lt;strong&gt;right&lt;/strong&gt;, &lt;strong&gt;down&lt;/strong&gt;, &lt;strong&gt;left&lt;/strong&gt; or &lt;strong&gt;sell&lt;/strong&gt;.&#xA;When they choose to sell the simulation will try to sell to the closest consumer in a predetermined max range. If no consumers exist in that range or if the consumer rejects to buy then the sell fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started training a Deep Q Network that I built using TensorFlow. As input features, I am giving the agent current position, the position of each consumer and a boolean saying if the consumer was recently asked to buy or not (consumers that were asked to buy something will reject future offers for a determined amount of time with 100% probability).&#xA;For the output layer, I have 5 nodes, one for each available action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a screenshot of the game:&#xA;The red number on the right-bottom corner of each agent represents their total utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xQ16I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xQ16I.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I decided to give the agents the following rewards:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;SELL_SUCCESSED_REWARD = 3&lt;/strong&gt; - The agents receives 3 points for each success sell. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;SELL_FAILED_REWARD = -0.010&lt;/strong&gt; - The agent loses 0.01 points for each failed sell&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;MOVING_REWARD = -0.001&lt;/strong&gt; - The agent loses 0.001 points for each move&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;NOT_MOVING_REWARD = -0.0125&lt;/strong&gt; - The agent loses 0.0125 points for standing in the same position (ie. not moving or trying to move against a wall)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I started training my agent but I seems to do not learn anything! I left it training for around 3 hours and I could not see any improvement. I tried different activation functions, batch sizes, exploration rates etc but no noticiable effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Can a DQN learn in this type of enviroment where there are a lot of random walks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes what could be my problem? Not enought training time? Bad input features? Bad implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the files with my implementation of DQN:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Agent: &lt;a href=&quot;https://github.com/daniel3303/aasma-project/blob/master/src/Agent/DeepLearningAgent/DeepLearningAgent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/daniel3303/aasma-project/blob/master/src/Agent/DeepLearningAgent/DeepLearningAgent.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training: &lt;a href=&quot;https://github.com/daniel3303/aasma-project/blob/master/train_dqn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/daniel3303/aasma-project/blob/master/train_dqn.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="21688" LastActivityDate="2019-05-08T22:06:36.610" Title="DQN not able to learn in a game where other agents perform random walks" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12216" PostTypeId="1" CreationDate="2019-05-09T02:44:54.127" Score="3" ViewCount="146" Body="&lt;p&gt;I have found various references describing Naive Bayes and they all demonstrated that it used MLE for the calculation. However, this is my understanding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(y=c|x)$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\propto$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$P(x|y=c)P(y=c)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the class the model may classify &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; as.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that's all, we can infer &lt;span class=&quot;math-container&quot;&gt;$P(x|y=c)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(c)$&lt;/span&gt; from the data. I don't see where the MLE shows its role.&lt;/p&gt;&#xA;" OwnerUserId="25509" LastEditorUserId="2444" LastEditDate="2019-06-14T20:08:03.510" LastActivityDate="2019-12-03T12:20:51.110" Title="What is the relationship between MLE and naive Bayes?" Tags="&lt;comparison&gt;&lt;naive-bayes&gt;&lt;maximum-likelihood&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12217" PostTypeId="1" AcceptedAnswerId="12218" CreationDate="2019-05-09T02:59:36.227" Score="2" ViewCount="336" Body="&lt;p&gt;I would provide a sound signal of about 2-3 seconds to my neural network. I have trained my network with a single word, like if I speak &quot;Hello&quot; the network may tell if &quot;Hello&quot; is spoken or not, but some other word like &quot;World&quot; is spoken, it will say &quot;Hello&quot; is not spoken. I just want classification of sound if its a specific command or word.&#xA;What is the best way to do this, I am not a that much advanced in DNN, I only know about NN and CNN, I want to know if there is some research paper or tutorial, or need some explanation about the work.&lt;/p&gt;&#xA;" OwnerUserId="25510" LastActivityDate="2019-05-09T04:52:33.373" Title="How to do speech recognition on a single word" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;voice-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12218" PostTypeId="2" ParentId="12217" CreationDate="2019-05-09T04:52:33.373" Score="2" Body="&lt;p&gt;If you have fixed length speech data you can detect the content using only CNN. You can see that problem as a binary classification (&lt;code&gt;1&lt;/code&gt; if the spoken word is correct, &lt;code&gt;0&lt;/code&gt; otherwise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But first, you need to make the input length is fixed. For example, you use 2 seconds as the fixed length. If the recorded speech is more than 2 seconds, you need to crop it, and if the recorded speech is less than 2 seconds you can pad it with 0 values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, You can either use &lt;a href=&quot;https://ronan.collobert.com/pub/matos/2015_cnnspeech_interspeech.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;raw data (time-domain)&lt;/a&gt; or &lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/CNN_ASLPTrans2-14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transform your data using some features extractors method (FFT, MFCC, or MFSC)&lt;/a&gt;. Then, use CNN as you use it to classify the image. You can assume the graphic of the sound wave as a 2D image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, If your data have a variety of length, you can combine CNN to detect each phoneme then combine it as a sequence using RNN or HMM. You can read this method also in the mentioned papers.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-05-09T04:52:33.373" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12219" PostTypeId="2" ParentId="12216" CreationDate="2019-05-09T06:32:02.193" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;And that's all, we can infer P(x|y=c) and P(c) from the data. I don't see where the MLE shows its role.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Maximum likelihood estimate is used for this very purpose, i.e. to estimate the conditional probability &lt;span class=&quot;math-container&quot;&gt;$p(x_j \mid y)$&lt;/span&gt; and marginal probability &lt;span class=&quot;math-container&quot;&gt;$p(y)$&lt;/span&gt; .&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In Naive Bayes Algorithm ,using the properties of &lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional probability&lt;/a&gt;, we can estimate the joint probability&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(y, x_1, x_2, \dots, x_n) =  p(x_1, x_2, \dots, x_n \mid y) \; p(y)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the same as&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(y=c|x)\propto p(x|y=c)p(y=c)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we need to estimate in here, are the conditional &lt;span class=&quot;math-container&quot;&gt;$p(x_j \mid y)$&lt;/span&gt; and marginal &lt;span class=&quot;math-container&quot;&gt;$p(y)$&lt;/span&gt; probabilities, and we use maximum likelihood for this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise: Assume that the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a feature vector &lt;span class=&quot;math-container&quot;&gt;$(x_1, x_2, ...x_n)$&lt;/span&gt; - which means nothing more than the data point is a vector. In Naive Bayes Classification the assumption is that these features are independent of each other conditional on the class. So, in that case this term &#xA;&lt;span class=&quot;math-container&quot;&gt;$$p(x|y=c)$$&lt;/span&gt; is replaced by &lt;span class=&quot;math-container&quot;&gt;$$\prod_{i=1}^{n} p(x_i|y=c)$$&lt;/span&gt; -- this is because of the independence assumption that lets us simply multiply the probabilities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem of estimating the class to which the data point belongs reduces to the problem of maximizing the  &lt;em&gt;likelihood&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$$\prod_{i=1}^{n} p(x_i|y=c)$$&lt;/span&gt; -- which means assigning the data &lt;span class=&quot;math-container&quot;&gt;$(x_1, x_2, ...x_n)$&lt;/span&gt;  to the class &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; for which this likelihood is highest. This will give the same result as MLE which takes the form  of maximizing this quantity -- &lt;span class=&quot;math-container&quot;&gt;$$\prod_{i=1}^{n} p(x_i|\theta)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the assumed parameters.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-12-03T12:20:51.110" LastActivityDate="2019-12-03T12:20:51.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12220" PostTypeId="2" ParentId="12029" CreationDate="2019-05-09T09:34:11.717" Score="2" Body="&lt;p&gt;Like others said you can't approximate this with a linear regression model.&lt;br&gt;&lt;br&gt;&#xA;A PRM that approximates a solution could give you the following:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$y = 0.948 + x + 0.00085*x^6$&lt;/span&gt; ~ &lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$y = 237/250 + x + (17/20000)*x^6$&lt;/span&gt;&lt;br&gt;&#xA;For &lt;span class=&quot;math-container&quot;&gt;$x = 9$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y \simeq 462$&lt;/span&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y = 0.9258 + x + 0.00086*x^6$&lt;/span&gt;&#xA;For &lt;span class=&quot;math-container&quot;&gt;$x = 9$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y \simeq 466.965$&lt;/span&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;UPDATE&lt;/code&gt;&lt;br&gt;&#xA;An approximation of course, may be in the range of:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$y = 2^{(x + 1)} - 2^x$&lt;/span&gt; -the model you propose- &lt;br&gt;&#xA;&lt;code&gt;Goodness of fit: 0.968475 and Mean Square Error = 685.111&lt;/code&gt;&lt;br&gt;&lt;br&gt;&#xA;Based on this range a better approximation would be:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$y = 2^x + (-1/2)*x^2$&lt;/span&gt;&lt;br&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$R^2$&lt;/span&gt;&lt;br&gt;&#xA;&lt;code&gt;Goodness of fit = 0.995&lt;/code&gt; &lt;br&gt;&#xA;&lt;code&gt;Mean Square Error: 89.0278&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="25410" LastEditorUserId="25410" LastEditDate="2019-05-09T10:43:17.573" LastActivityDate="2019-05-09T10:43:17.573" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12226" PostTypeId="1" CreationDate="2019-05-09T13:02:53.130" Score="0" ViewCount="905" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uc8C7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uc8C7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to train a PPO agent and my average rewards graph looks like this.  Could this indicate that it's stuck at a local maximum?  Do I need to promote exploring by increasing the entropy or does this look like a bug with my implementation?  Also, my action space is continuous.  Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hyperparameters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning rate = 0.01&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Entropy Coefficient = 0.01&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Value Function Loss Coefficient = 0.5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gamma/Discount Factor = 0.995&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MiniBatch Size = 512&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Epochs = 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clip Epsilon = 0.1&lt;/p&gt;&#xA;" OwnerUserId="23494" LastActivityDate="2020-11-06T01:02:35.173" Title="Reinforcement learning with PPO: rewards decreasing" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12232" PostTypeId="2" ParentId="4965" CreationDate="2019-05-09T21:19:15.330" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;The best approach at this time (2019):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The most efficient approach now is to use &lt;a href=&quot;https://tfhub.dev/google/universal-sentence-encoder/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Universal Sentence Encoder by Google&lt;/strong&gt;&lt;/a&gt; (&lt;a href=&quot;https://arxiv.org/abs/1803.11175&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper_2018&lt;/a&gt;) which computes semantic similarity between sentences using the dot product of their embeddings &lt;em&gt;(i.e learned vectors of 215 values)&lt;/em&gt;. Similarity is a float number between 0 &lt;em&gt;(i.e no similarity)&lt;/em&gt; and 1 &lt;em&gt;(i.e strong similarity).&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation is now integrated to Tensorflow Hub and can easily be used. Here is a ready-to-use code to compute the similarity between 2 sentences. Here I will get the similarity between &lt;strong&gt;&lt;em&gt;&quot;Python is a good language&quot;&lt;/em&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;em&gt;&quot;Language a good python is&quot;&lt;/em&gt;&lt;/strong&gt; as in your example.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Code example:&lt;/strong&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Requirements: Tensorflow&amp;gt;=1.7 tensorflow-hub numpy&#xA;&#xA;import tensorflow as tf&#xA;import tensorflow_hub as hub&#xA;import numpy as np&#xA;&#xA;module_url = &quot;https://tfhub.dev/google/universal-sentence-encoder-large/3&quot; &#xA;embed = hub.Module(module_url)&#xA;sentences = [&quot;Python is a good language&quot;,&quot;Language a good python is&quot;]&#xA;&#xA;similarity_input_placeholder = tf.placeholder(tf.string, shape=(None))&#xA;similarity_sentences_encodings = embed(similarity_input_placeholder)&#xA;&#xA;with tf.Session() as session:&#xA;  session.run(tf.global_variables_initializer())&#xA;  session.run(tf.tables_initializer())&#xA;  sentences_embeddings = session.run(similarity_sentences_encodings, feed_dict={similarity_input_placeholder: sentences})&#xA;  similarity = np.inner(sentences_embeddings[0], sentences_embeddings[1])&#xA;  print(&quot;Similarity is %s&quot; % similarity)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Similarity is 0.90007496 #Strong similarity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23350" LastActivityDate="2019-05-09T21:19:15.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12233" PostTypeId="1" CreationDate="2019-05-10T03:59:18.300" Score="1" ViewCount="24" Body="&lt;p&gt;I'm a grad student from EE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, basically, there's an electrical circuit that is supposed to output &quot;0&quot; or &quot;1&quot; by exactly 50 to 50 chance. It generates a number of big arrays of 0s and 1s, each of which amounts to more than 4,000 of the numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But because these arrays are physically generated in a fab, &lt;strong&gt;I assume it might develop some dependencies among numbers and some output could be predicted by more than 50% chance&lt;/strong&gt;. For example, due to some variations in the process, &quot;1&quot; can be more likely to come than &quot;0&quot; after a sequence of &quot;001100&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then let's say I make a simple deep neural network which takes 7 inputs and gives 1 output. I simply slice my array by 8 numbers, 7 of which are given to the input and the last one is used as a label (the true answer). I train my simple DNN using all these sliced numbers and it will learn some sequences. Finally, I apply my NN to a test set, and if it predicts the next number with an accuracy of more than 50%, that proves my assumption, and if it doesn't that is also good for me because it says my circuits are good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it work?&lt;/p&gt;&#xA;" OwnerUserId="25540" LastEditorUserId="9947" LastEditDate="2019-05-10T05:12:31.787" LastActivityDate="2019-05-10T05:12:31.787" Title="Would this NN for my chip outputs work?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="12234" PostTypeId="1" AcceptedAnswerId="12235" CreationDate="2019-05-10T04:41:27.073" Score="2" ViewCount="29" Body="&lt;p&gt;I have seen in several jupyter notebooks people initializing the NN weights using:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.random.randn(D, M) / np.sqrt(D)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other times they just do:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.random.randn(D, M)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the advantage of dividing the Gaussian distribution by the squared root of the number of neurons in the layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="21688" LastActivityDate="2019-05-10T04:53:38.247" Title="Why is it so common to initialize weights with a Guassian distribution divided by the square root of number of neurons in a layer?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12235" PostTypeId="2" ParentId="12234" CreationDate="2019-05-10T04:53:38.247" Score="2" Body="&lt;p&gt;I think they use the Xavier/Glorot's initialization method. You can read from &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We initialized the biases to be 0 and the weights &lt;span class=&quot;math-container&quot;&gt;$W_{ij}$&lt;/span&gt; at each layer with the following commonly used heuristic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_{ij} \sim U [ -\frac{1}{\sqrt{n}}, \frac{1}{\sqrt{n}}] $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$U[−a, a]$&lt;/span&gt; is the uniform distribution in the interval &lt;span class=&quot;math-container&quot;&gt;$(−a, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the size of the previous layer (the number of columns of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Some people use this as some reports said this initialization method lead to better result&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-05-10T04:53:38.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12236" PostTypeId="1" CreationDate="2019-05-10T07:06:40.253" Score="2" ViewCount="41" Body="&lt;p&gt;I have a large data set that doesn't fit in memory and would have to use something like Keras's &lt;code&gt;model.fit_generator&lt;/code&gt; if I would like to train the model on all of the available data. The problem is that my data load time is larger than a single epoch and I would hate to incur that data load cost for each epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative approach that yields some value is to load as much data as possible, train the model for a few hundred epochs, then load the next portion of the data and reiterate for the same amount of epochs. And repeat this until all my data is &quot;seen&quot; by the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively I understand that this is sub-optimal as the model will tend to optimize for the latest portion of the data and &quot;forget&quot; the previous data but I would like a more in-depth explanation of the downsides of that method and if there are any ways to overcome them.&lt;/p&gt;&#xA;" OwnerUserId="25542" LastEditorUserId="25542" LastEditDate="2019-05-11T06:42:35.433" LastActivityDate="2019-05-11T06:42:35.433" Title="Difference between retraining on different portions of data and training initially on larger data set" Tags="&lt;training&gt;&lt;datasets&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12238" PostTypeId="2" ParentId="6678" CreationDate="2019-05-10T10:45:46.347" Score="0" Body="&lt;p&gt;Yes, your intuition is correct. The effect of this problem is that the generator can no longer improve its output to marginally fool the discriminator - the discriminator isn't buying any of the generated output. In this case, the generator gets stuck in a local minimum and typically produces nonsense results.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-05-10T10:45:46.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12239" PostTypeId="1" AcceptedAnswerId="12241" CreationDate="2019-05-10T12:15:31.887" Score="5" ViewCount="1059" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt; states that a &lt;a href=&quot;https://en.wikipedia.org/wiki/Feedforward_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;feed-forward neural network &lt;/a&gt; with a single hidden layer containing a finite number of neurons can approximate any continuous function (provided some assumptions on the activation function are met).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other machine learning model (apart from any neural network model) that has been &lt;strong&gt;proved&lt;/strong&gt; to be an &lt;em&gt;universal&lt;/em&gt; function approximator (and that is potentially comparable to neural networks, in terms of usefulness and applicability)? If yes, can you provide a link to a research paper or book that shows the proof?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar questions have been asked in the past in other places (e.g. &lt;a href=&quot;https://qr.ae/TWI6KL&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://qr.ae/TWI62P&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://cstheory.stackexchange.com/q/7894/34637&quot;&gt;here&lt;/a&gt;), but they do not provide links to papers or books that show the proofs.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T11:26:22.847" LastActivityDate="2020-08-30T17:28:01.803" Title="Which machine learning models are universal function approximators?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;function-approximation&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12240" PostTypeId="1" CreationDate="2019-05-10T13:22:04.883" Score="1" ViewCount="55" Body="&lt;p&gt;I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look at the relevant part of the paper below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1xSku.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1xSku.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the input to the kernel X is k*d (say k=5 words of d=300 embedding dimenisonality). Therefore the input is 5*300. In a computer vision task a kernel would slide over parts of the image, in NLP you usually see kernel taking up the whole width of the input matrix. So I would expect kernel to be m*d (e.g. 3*300 - slide over 3 words and look at their whole embeddiings).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the kernel here is of dimensionality 2d x kd which in our hypothetical example would be 600*1500. I don't understand how this massive kernel would slide over an input that is by far lower dimensional (5*300). In computer vision you could zero-pad the input, but here zero-padding would basically turn the input matrix into mostly zeros with only a handful of meaningful numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for shedding some light on it!&lt;/p&gt;&#xA;" OwnerUserId="21278" LastActivityDate="2021-05-14T17:05:00.457" Title="Convolutional Sequence to Sequence Learning kernel parameters" Tags="&lt;convolutional-neural-networks&gt;&lt;natural-language-processing&gt;&lt;machine-translation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12241" PostTypeId="2" ParentId="12239" CreationDate="2019-05-10T13:49:03.203" Score="2" Body="&lt;h3&gt;Support vector machines&lt;/h3&gt;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://www.researchgate.net/publication/2531186_A_Note_on_the_Universal_Approximation_Capability_of_Support_Vector_Machines&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Note on the Universal Approximation Capability of Support Vector Machines&lt;/a&gt; (2002) B. Hammer and K. Gersmann investigate the universal function approximation capabilities of SVMs. More specifically, the authors show that SVMs with standard kernels (including Gaussian, polynomial, and several dot product kernels) can approximate any measurable or continuous function up to any desired accuracy. Therefore, SVMs are universal function approximators.&lt;/p&gt;&#xA;&lt;h3&gt;Polynomials&lt;/h3&gt;&#xA;&lt;p&gt;It is also widely known that we can approximate any continuous function with polynomials (see the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stone-Weierstrass theorem&lt;/a&gt;). You can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Polynomial_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;polynomial regression&lt;/a&gt; to fit polynomials to your labelled data.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-30T17:28:01.803" LastActivityDate="2020-08-30T17:28:01.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12242" PostTypeId="2" ParentId="12240" CreationDate="2019-05-10T14:00:36.113" Score="0" Body="&lt;p&gt;They are doing a matrix multiplication: consider &lt;span class=&quot;math-container&quot;&gt;$y = Ax, y \in \mathbb{R}^m, x \in \mathbb{R}^n, A \in M_\mathbb{R}(m,n)$&lt;/span&gt;. In the paper &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a concatenation of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; elements of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^d$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is long &lt;span class=&quot;math-container&quot;&gt;$kd$&lt;/span&gt;; &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is long &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23224" LastActivityDate="2019-05-10T14:00:36.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12244" PostTypeId="1" CreationDate="2019-05-10T17:58:28.847" Score="1" ViewCount="84" Body="&lt;p&gt;I searched through the internet but couldn't find a reliable article that answers this question.&lt;/p&gt;&#xA;&lt;p&gt;Can we use Autoencoders for unsupervised CNN feature learning of unlabeled images like the below&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JC9DO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JC9DO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;and use the encoder part of the Auto-encoder for Transfer learning of few labeled images from the dataset? as shown below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/flVLi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/flVLi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I believe this will reduce the labeling work and increase the accuracy of a model.&lt;/p&gt;&#xA;&lt;p&gt;However, I have concerns like the more cost in computing, failing to learn all required features, etc..&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if any employed this method in large scale learning such as image-net.&lt;/p&gt;&#xA;&lt;p&gt;PS: Pardon if it is Trivial or Vague as I am new to the field of AI and computer vision.&lt;/p&gt;&#xA;" OwnerUserId="25366" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-05-10T17:58:28.847" Title="Can we use Autoencoders for unsupervised CNN feature learning?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;autoencoders&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12245" PostTypeId="1" CreationDate="2019-05-10T19:30:49.353" Score="1" ViewCount="18" Body="&lt;p&gt;I am currently working on learning the features provided by a pre-trained network for image retrieval. Currently I take the features provided by the pre-trained network, use global max pooling to essentially provide me with a vector and then use fully connected layers to learn the feature vector. This has provided good results, although prone to over-fitting, particularly without dropout. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible/would it be beneficial to use a 1D convolutional layer instead of the fully connected layers to learn the features? Bearing in mind this is essentially still image data that has just been transformed. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.add(GlobalAveragePooling2D(input_shape=input_shape))&#xA;model.add(Dense(256, activation=&quot;relu&quot;))&#xA;model.add(Dropout(0.2))&#xA;model.add(Dense(256, activation=&quot;relu&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how to try this practically in Keras as 1D convolutional layers only seem to accept a 3 dimensional input tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions welcome!&lt;/p&gt;&#xA;" OwnerUserId="18577" LastActivityDate="2019-05-10T19:30:49.353" Title="Learning Features from a Pre-trained Network" Tags="&lt;convolutional-neural-networks&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12247" PostTypeId="1" AcceptedAnswerId="12249" CreationDate="2019-05-10T23:43:24.807" Score="1" ViewCount="812" Body="&lt;p&gt;Can you describe this reward system in more detail?&lt;/p&gt;&#xA;&lt;p&gt;I understand that the environment sends a signal indicating whether or not the action taken by the agent was 'good' or not, but it seems too simple.&lt;/p&gt;&#xA;&lt;p&gt;Basically, can you detail the nitty-gritty workings of this system? I dunno, I may just be overthinking things.&lt;/p&gt;&#xA;" OwnerUserId="25560" LastEditorUserId="2444" LastEditDate="2021-01-16T04:18:18.467" LastActivityDate="2021-01-16T04:18:18.467" Title="What is the reward system of reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12249" PostTypeId="2" ParentId="12247" CreationDate="2019-05-11T00:40:09.840" Score="2" Body="&lt;p&gt;In this case, the word &quot;system&quot; refers to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov decision process&lt;/a&gt; (MDP), which is the mathematical model used to represent the reinforcement learning (RL) problem or, in general, a decision making problem. Recall that, in RL, the problem consists in finding an (optimal) policy, which is a policy that allows the agent to collect the highest amount of reward (in the long run). Hence, in RL, the MDP is the problem and the optimal policy (for that MDP) is the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MDP is composed of the set of states of the environment &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, the set of possible actions that the RL agent can take &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, a transition function, &lt;span class=&quot;math-container&quot;&gt;$P(s_{t+1}=s'\mid s_{t}=s, a_t = a)$&lt;/span&gt;, which is a probability distribution and describes the dynamics of the environment, and a reward function &lt;span class=&quot;math-container&quot;&gt;$R_a(s', s)$&lt;/span&gt;, which is a function that describes the &quot;reward system&quot; of the environment. &lt;span class=&quot;math-container&quot;&gt;$R_a(s', s)$&lt;/span&gt; can be thought of as the reward (signal) that the agent receives after having taken action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and having landed in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the reward system is just this function &lt;span class=&quot;math-container&quot;&gt;$R_a(s', s)$&lt;/span&gt;, which is often the hardest function to define when modelling an RL problem. However, in certain cases, this function can be easily defined. For example, in the case of chess, you could define &lt;span class=&quot;math-container&quot;&gt;$R_a(s', s) = 1$&lt;/span&gt;, if &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the terminal state (checkmate), else &lt;span class=&quot;math-container&quot;&gt;$R_a(s', s) = 0$&lt;/span&gt;. Nonetheless, we could define the reward function for a chess environment differently. For example, we could give a reward of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; for certain &quot;clever&quot; moves. So, the reward function needs to be defined (by the programmer) in order to solve an RL problem and it highly affects the way the agent will learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward function can also be denoted by &lt;span class=&quot;math-container&quot;&gt;$R_a(s)$&lt;/span&gt;, which can be thought of as the reward that the agent receives after having taken action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (no matter which state it lands in), or &lt;span class=&quot;math-container&quot;&gt;$R(s)$&lt;/span&gt;, which can be thought of as the reward the agent receives either when it enters or exits state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-11T00:45:40.447" LastActivityDate="2019-05-11T00:45:40.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12250" PostTypeId="1" CreationDate="2019-05-11T05:42:13.527" Score="1" ViewCount="56" Body="&lt;p&gt;The demand is to locate the invoice within a camera captured image about that invoice. The invoice is always a white paper with printed black or blue characters, tables and red stamps. Sometimes the background behind the invoice is dark(about 60% of all the samples), but others are not. Sometimes there is shadow on it.&#xA;The question is how to detect the vertices, edges and corners of the invoice in this image? What algorithms should be applied? The android application CamScanner seems to have this function but not is effective every time. So does the CIDetector interface on iOS. What algorithms does CamScanner or iOS use(traditional or deep)?&lt;/p&gt;&#xA;" OwnerUserId="14948" LastEditorUserId="14948" LastEditDate="2019-05-14T02:04:15.950" LastActivityDate="2019-05-14T02:04:15.950" Title="How to locate the invoice within a camera captured image?" Tags="&lt;image-recognition&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12251" PostTypeId="1" AcceptedAnswerId="12253" CreationDate="2019-05-11T06:38:06.143" Score="2" ViewCount="67" Body="&lt;p&gt;I came across this formula in Sutton And Barto: RL an Intro (2nd Edition) equation number 4.7 (page number 78).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; are deterministic policies and &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, \pi'(s)) \geq v_\pi(s)$&lt;/span&gt; then the policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; is as good or better than &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE on Convention:&lt;/strong&gt; As per the convention of the book goes, I think they are using rewards for state to action to state transition sequence rather than state to action transition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why are they comparing state value function to action value function?&lt;/li&gt;&#xA;&lt;li&gt;Isn't it obvious the above equation might hold true (provided we select the best action among the possible actions) even for the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; since then the equation will change to &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, \pi(s)) \geq v_\pi(s)$&lt;/span&gt; and we know &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s) = \sum_{a \in \mathcal{A}(s)} \pi(a|s)q_\pi(s, a)$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What is the inconsistency here?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-05-11T10:58:02.860" LastActivityDate="2019-05-11T10:58:02.860" Title="Possible inconsistency in the Policy Improvement equation" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;policies&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12253" PostTypeId="2" ParentId="12251" CreationDate="2019-05-11T08:39:43.910" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are they comparing state value function to action value function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is because &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; measure the same quantity at different stages of the trajectory. By comparing the values at the same &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and modifying how &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is selected, the proof makes assertions about how that choice impacts the value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to recall that &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; measures the expected future reward when starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, and that &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; measures the expected future reward when starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;em&gt;thereafer&lt;/em&gt; following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. When &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is chosen using a deterministic &lt;span class=&quot;math-container&quot;&gt;$a = \pi(s)$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s) = q_{\pi}(s,\pi(s))$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Isn't it obvious the above equation might hold true &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;inequality&lt;/em&gt; (it is not an equation) does strictly hold true, because the &lt;em&gt;equation&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s) = q_{\pi}(s,\pi(s))$&lt;/span&gt; is true. However, that is not terribly useful, it doesn't prove anything new.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is interesting is if you change the decision for &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(provided we select the best action among the possible actions) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You cannot do that. The policy decides the actions, by definition. Selecting a better action according to the action value function, and showing what that does is precisely what the proof is showing.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;since then the equation will change to &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, \pi(s)) \geq v_\pi(s)$&lt;/span&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_\pi(s, \pi(s)) = v_\pi(s)$&lt;/span&gt; - yes your inequality holds, but is equivalent to not changing anything.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;and we know &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s) = \pi(a|s)q_\pi(s, \pi(s))$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here you have switched to using a non-deterministic policy and a deterministic policy in the same statement (you also have not defined &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; properly), making your equation badly formed. The correct form would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \sum_{a \in \mathcal{A}(s)} \pi(a|s)q_\pi(s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not a relevant form for the initial policy improvement theorem. However it does become relevant when the theory is adapted to show improving &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policies later.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;NOTE on Convention:&lt;/strong&gt; As per the convention of the book goes, I think they are using rewards for state to action to state transition sequence rather than state to action transition.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is not a convention in the book. In the book, the reward distribution is generally given as joint distribution with state transition in &lt;span class=&quot;math-container&quot;&gt;$p(s', r | s, a)$&lt;/span&gt; - this is a very generic approach that models any discrete MDP, regardless of how the rewards are described as associated with any of &lt;span class=&quot;math-container&quot;&gt;$s, a, s'$&lt;/span&gt; or a random factor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly the convention you are referring to is labelling the immediate reward as &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;, so that a small section of trajectory starting from state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; might be &lt;span class=&quot;math-container&quot;&gt;$S_t, A_t, R_{t+1}, S_{t+1}$&lt;/span&gt; . . . some other texts will use &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; here. However, that is not relevant to the policy improvement theorem, it would hold the same under either convention, just with slight relabelling of the reward indices.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-05-11T08:39:43.910" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12254" PostTypeId="2" ParentId="7755" CreationDate="2019-05-11T09:30:11.193" Score="1" Body="&lt;p&gt;Normally, the set of actions that the agent can execute does not change over time, but some actions can become impossible in different states (for example, not every move is possible in any position of the TicTacToe game).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look as example at pice of code &lt;a href=&quot;https://github.com/haje01/gym-tictactoe/blob/master/examples/base_agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/haje01/gym-tictactoe/blob/master/examples/base_agent.py&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ava_actions = env.available_actions()&#xA;action = agent.act(state, ava_actions)&#xA;state, reward, done, info = env.step(action)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25551" LastActivityDate="2019-05-11T09:30:11.193" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12255" PostTypeId="1" CreationDate="2019-05-11T11:11:43.090" Score="11" ViewCount="3985" Body="&lt;p&gt;Many examples work with a table-based method for Q-learning. This may be suitable for a discrete state (observation) or action space, like a robot in a grid world, but is there a way to use Q-learning for continuous spaces like the control of a pendulum?&lt;/p&gt;&#xA;" OwnerUserId="19413" LastEditorUserId="2444" LastEditDate="2020-11-21T21:17:25.590" LastActivityDate="2020-11-21T21:31:01.930" Title="Can Q-learning be used for continuous (state or action) spaces?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;continuous-action-spaces&gt;&lt;continuous-state-spaces&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12257" PostTypeId="2" ParentId="12255" CreationDate="2019-05-11T11:32:06.907" Score="7" Body="&lt;h1&gt;Q-learning for continuous state spaces&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Yes, this is possible, provided you use some mechanism of approximation. One approach is to discretise the state space, and that doesn't have to reduce the space to a small number of states. Provided you can sample and update enough times, then a few million states is not a major problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with large state spaces it is more common to use some form of function approximation for the action value. This is often noted &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta)$&lt;/span&gt; to show that it is both an estimate (the circumflex over &lt;span class=&quot;math-container&quot;&gt;$\hat{q}$&lt;/span&gt;) and that you are learning some function parameters (&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;). There are broadly two popular approaches to Q-learning using function approximation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Linear function approximation over a processed version of the state into features. A lot of variations to generate features have been proposed and tested, including Fourier series, tile coding, radial basic functions. The advantage of these methods are that they are simple, and more robust than non-linear function approximations. Which one to choose depends on what you state space represents and how the value function is likely to vary depending on location within the state space.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural network function approximation. This is essentially what Deep Q Networks (DQN) are. Provided you have a Markov state description, you scale it to work sensibly with neural networks, and you follow other DQN best practices (experience replay table, slow changing target network) this can work well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Q-learning for continuous action spaces&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Unless you discretise the action space, then this becomes very unwieldy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that, given &lt;span class=&quot;math-container&quot;&gt;$s,a,r,s'$&lt;/span&gt;, Q-learning needs to evaluate the TD target:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{target}(s,a) = r + \gamma \text{max}_{a'} \hat{q}(s',a',\theta)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process for evaluating the maximum becomes less efficient and less accurate the larger the space that it needs to check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For somewhat large action spaces, using double Q-learning can help (with two estimates of Q, one to pick the target action, the other to estimate its value, which you alternate between on different steps) - this helps avoid maximisation bias where picking an action because it has the highest value and then using that highest value in calculations leads to over-estimating value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For very large or continuous action spaces, it is not usually practical to check all values. The alternative to Q-learning in this case is to use a policy gradient method such as Actor-Critic which can cope with very large or continuous action spaces, and does not rely on maximising over all possible actions in order to enact or evaluate a policy.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Controlling a pendulum&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For a discrete action space e.g. applying one of a choice of forces on each time step, then this can be done using a DQN approach or any other function approximation. The classic example here might be an environment like Open AI's &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v1/&quot; rel=&quot;noreferrer&quot;&gt;CartPole-v1&lt;/a&gt; where the state space is continuous, but there are only two possible actions. This can be solved easily using DQN, it is something of a beginner's problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding continuous action space ends up with something like the &lt;a href=&quot;https://gym.openai.com/envs/Pendulum-v0/&quot; rel=&quot;noreferrer&quot;&gt;Pendulum-v0&lt;/a&gt; environment. This can be solved to some degree using DQN and discretising the action space (to e.g. 9 different actions). However, it is possible to make more optimal solutions using an Actor-Critic algorithm like A3C.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-11T13:17:59.623" LastActivityDate="2019-05-11T13:17:59.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12258" PostTypeId="2" ParentId="12255" CreationDate="2019-05-11T11:39:10.367" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q-Learning for continuous state space&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Reinforcement learning algorithms (e.g Q-Learning) can be applied to both discrete and continuous spaces. If you understand how it works in discrete mode, then you can easily move to continuous mode. That's why in the literature all the introductory material focuses on discrete mode, as it's easier to model (table, grid, etc.)&lt;/p&gt;&#xA;&lt;p&gt;Supposing you have a discrete number of actions, &lt;strong&gt;the only difference&lt;/strong&gt; in a continuous space is that you will be modeling the state each &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; amount of time (&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; being a number you can choose depending on your use case). So, basically, you end up with a discrete space, but probably with an infinite number of states. You apply then the same approach you learned for discrete mode.&lt;/p&gt;&#xA;&lt;p&gt;Let's take the example of self-driving cars, at each &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;ms (e.g &lt;span class=&quot;math-container&quot;&gt;$X=1$&lt;/span&gt;), you'll be computing the state of the car which are your input features (e.g direction, orientation, rotation, distance to the pavement, relative position on the lane, etc.) and take a decision of the action to take as in discrete mode. The approach is the same in other use cases, like playing games, walking robots, and so on.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note (continuous action space):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you have continuous actions, then in almost all use cases the best approach is to discretize your actions. I can't think of an example where discretizing your actions will lead to a considerable deficiency.&lt;/p&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="2444" LastEditDate="2020-11-21T21:31:01.930" LastActivityDate="2020-11-21T21:31:01.930" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12259" PostTypeId="2" ParentId="12171" CreationDate="2019-05-11T12:28:22.003" Score="1" Body="&lt;p&gt;&lt;strong&gt;Assessing the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Based on the x,y values of each detection, I am trying to find the direction of the bicycles and if the bicycle crosses the whole frame to do a UP/DOWN count.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It appears from the question that there is an interest in both counting bicycles and determining the direction of travel of each. from within the frames of a video stream or file. We can assume that each bicycle has at least one rider. It also appears that it is not a problem involving a fixed optical system positioned to point at a path that is tangential to the optical path, which would make the problem much easier, locking the approximate distance of the bicycle wheels to the optical system to near constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use of the SSD mobile-net model seems reasonable as a starting point for developing expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Starting With ML Design Basics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider the purpose of CNN and RNN designs.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The purpose of a convolutional network is to deal equally with regions in a multi-dimensional array of values in discrete samples of \R^n during an adaptive (learning) process.&lt;/li&gt;&#xA;&lt;li&gt;The purpose of a recurrent network is to adapt to (learn) potentially complex temporal (time-wise) trends in potentially complex nonlinear systems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Understand that the SSD class of algorithms do not do what natural visual systems do. They do not zoom in and out on independent objects within the network seamlessly. They cannot note that a base ball player is running to first base and a ball is coming from the catcher at the same time, requiring &lt;strong&gt;independent&lt;/strong&gt; conceptual zoom operations within the neural network. This cannot be done with a zoom lens. That is why the Director of Photography is such a key role in movie making. The visual data must contribute well to the story telling, using lighting, camera orientation, panning, zooming, and depth of focus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although one can create several bicycle concept classes to cover various bicycle sizes, orientations relative to the optics, and distances away, there are limitations to this approach, which can be diminished with circuit parallelism in hardware. Multi-threading and serial evaluation can, depending on resources and patience factors, increase training time beyond what is practical. The challenge is to create seamlessness between low level concept classes of a bicycle in a frame as the bicycle angle and distance changes relative to the optical path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deeper into Details&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The, &quot;heavy distortion on the objects,&quot; could be a show-stopper if the root cause of the distortion mentioned is poor resolution in the time, horizontal, or vertical dimensions. The most significant and consistent image-oriented feature of a bicycle is two ellipses (not always circles) in close horizontal proximity and even closer vertical proximity &amp;mdash; the two wheels. The wheels need to be recognizable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two general categories of networks were mentioned in the question, CNNs and RNNs, which, in general are the two most relevant overall categories of components in a visual system that recognizes motion. We have some nomenclature in the question, which begins the mathematical theory behind the design of the training of the networks and the real time requirements on those network components once the network components are trained.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... each ... frame ... has multiple detections of D1, D2, D3, ..., Dn&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ D_i \land i \in {1, 2, ..., n} \\&#xA;\Downarrow \\&#xA;D_{xywh} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above nomenclature presumably refers to a post-learning detection of concept classes &lt;span class=&quot;math-container&quot;&gt;$C_a, C_b, C_d$&lt;/span&gt;, where there is a many-to-one relationship between the above numeric indices for detections and these letter indices for the concepts of a bicycle to be recognized. Each concept class &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; might correspond to a particular recognizable bicycle feature set given a particular range of distances to the optics and orientation of the wheels relative to the direction of light rays between the wheels and the camera. The designer, considering this correspondence cannot dismiss the turning of the front wheel. We cannot assume that the eccentricity of the visual representations of the two wheels will be the same, since the bicycle may be turning. Even in this more complex case, the ellipses are likely the two most differentiating features of bicycles in common scenes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may be a good time to point out that tricycle recognition may require the recognition of an entirely distinct set of concept classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also notice that, if the optics (camera) is at a drastically different altitude than the wheels of the bicycles, such as images from a drone or a camera on a tall pole, the problem is a different one. This is the intensely effective quality of natural vision systems. Over millions of years, the kind of training that recognizes a bicycle from a drone video stream having only been trained to recognize a bicycle from ground level has emerged. Nature's ability to apply cognitive abilities to visual sequence recognition to use in trajectory prediction is not yet realized in software and hardware and the main problem in automated vehicle piloting and driving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Two Different Output Requirements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two somewhat distinct problems that must be considered in analysis of the project requirements. The output could be either of these two, depending on whether counting bicycles is the primary goal or whether metrics of travel is its own independent objective.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Unit vector &lt;span class=&quot;math-container&quot;&gt;$\vec{r}$&lt;/span&gt;, presumably in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; as a pixel vector, essentially a normalized vector of the first derivative of pixel position with respect to time. The center of the bike, in this case, would be based on the features of the bike in the field of view.&lt;/li&gt;&#xA;&lt;li&gt;Unit vector &lt;span class=&quot;math-container&quot;&gt;$\vec{r}$&lt;/span&gt;, presumably in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; as a geocoordinate unit vector, essentially a normalized vector of the first derivative of position with respect to time. The center of the bike, in this case, would be based on the features of the bike in geo-space without the altitude coordinate.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approaches to AI Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common, but inefficient, artificial network approach is to locate the bicycle in each frame with a CNN and then use one of the progressive RNN types (either a GRU or a b-LSTM network) to recognize motion trends. One of the largest drawbacks is that you may have many concept classes that represent adjacent size-distance-orientation concepts (kernel based recognition models) of a bicycle to train into the CNN. If the bike is traveling toward or away from the optics at some angle, then the disappearance of the bike from &lt;span class=&quot;math-container&quot;&gt;$D_a$&lt;/span&gt; and its appearance in &lt;span class=&quot;math-container&quot;&gt;$D_b$&lt;/span&gt; needs to be &lt;em&gt;construed as the contiguous motion of one bike&lt;/em&gt;. This is not an easy challenge but is heavily covered in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is recommended to use web searches designed to search scholarly articles, not dummies guides, which are not reliable. There many academic publications that can be found with &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C10&amp;amp;q=image+recognition+changing+distance+orientation&quot; rel=&quot;nofollow noreferrer&quot;&gt;the search&lt;/a&gt; term, &quot;Image recognition changing distance orientation.&quot; Looking at old articles from the 90s will provide a good historical context. Looking at new ones from the last three years will provide a survey of the current state of research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other Questions Within the Primary Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original types of recurrent networks are essentially for historical context. The dominant in-field recurrent network successes are often of the LSTM, b-LSTM, or GRU types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The language (Scala, Python, Java, C, C++) is not particularly relevant if you are delegating the training to a GPU (which always runs C/C++ code),&#xA;so it may be unwise to consider reliability concerns as a primary driver for programming language selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding, &quot;How AI manages to count the objects,&quot; AI doesn't &amp;mdash; not at the current state of technology. There is no one approach or algorithm across AI technology that dominates over all other approaches for all domains, into which bicycles can be plugged in and it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the AI engineer designs how the objects will be counted based on the characteristics of the objects to be counted, the meta-features of the incoming stream or data set, and the specifics of the recognition challenge. This is again because the wider capabilities of natural vision systems using the more sophisticated neural nets in animals and people has not yet been invented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final Recommendations Regarding System Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The division between the use of kernels, in the CNN context, and the use of one of the recurrent network types is critical. If the engineer tries to delegate too much to the kernel, the above issue of bicycle distance to the optics and turning corners is exacerbated, because kernel operations do not lend themselves well to orientation and distance complexity. However, the CNN approach is excellent for the most upstream operations, such as edge detection and primitive object detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the recurrent network (of the more advanced types mentioned above) detect the bicycles as their distance and orientation in relation to the optical path changes, unless you have a sizable GPU farm that will perform CNN operations covering many distance and orientation ranges in parallel. Even if you do or have the patience of a saint, it may be best to delegate total bicycle recognition to the recurrent network anyway, since this is likely closer to the way natural systems do it and the modeling of bicycle travel between distance and orientation categories can be made more naturally seamless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recapping the comment above in this context, the problem complexity would be much lower if the bicycles were on a path, could not turn corners, and must travel either right to left or left to right within a speed range governed by the flow of traffic.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Response to Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding Edit 1,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Edit 1: The area of view under the camera is fixed. And we expect the bicycles to move from one entry side. Lets assume that the view and entry/exit is like shown in this &lt;a href=&quot;https://www.youtube.com/watch?v=tW7Pl3bSzR4&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;, &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;YouTube.com reports, &quot;This video does not exist.&quot; That the camera is fixed, had been assumed in the writing of this answer because no camera trajectory described in the question. Had the expectation that bicycles will move from one entry side of the frame been included in the question, the answer would have addressed that case, but there was no hint of that requirement prior to Edit 1. Nonetheless, much of the content in this answer to the more general case still applies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding practicality, let's differentiate &lt;em&gt;practical&lt;/em&gt; from &lt;em&gt;prefabricated&lt;/em&gt;. The problem originally described before Edit 1 has no prefabricated solution into which one can plug bicycle data and tweak a few parameters to achieve success. In fact, the general interest on the web in seeking prefabricated solutions is usually met with the practical reality that such plug and play cases in machine learning are rare. Most of the time an approach that involves design and experimentation is usually the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those that have hired and managed human beings know that even the hoped-for high level AI of the future, although possibly quite practical, may be as prefabricated as idealized. For instance, hiring an EE does not mean that the electrical engineering department will immediately see a practical improvement in throughput. Management, training, and workflow design will still be prerequisites to employee effectiveness. To have a practical &lt;strong&gt;and reliable&lt;/strong&gt; bicycle counting, some comprehension of concepts to guide initial experimentation, design, and tuning of the training and use scenarios will likely be necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the author of the question has one of those very rare cases where bicycles travel in exactly one direction and in series, with no foot traffic, trikes, walking pedestrians, or pets, then AI is not necessary at all. A simple LED and photo-transistor with a passive low pass filter and a digital input to a counter circuit will perform a reasonable count. However, if two bicycles might pass in parallel, then we are back with the camera, the need for concept classes, and challenges much like those discussed in detail above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding steps to a solution, if approaching from the side of a technology to investigate, &lt;a href=&quot;https://ai.stackexchange.com/questions/9829/concrete-example-for-q-learning/9878#9878&quot;&gt;this answer&lt;/a&gt; includes a recommendable sequence, although there is usually quite a bit of overlap and occasional backtracking in actual practice. If this is not a learning exercises and the bicycle problem is one that is actually needed to operate in the field, then the above answer is the correct background to comprehend what it is the machine is required to perform. Following that comprehension would be the investigation of various designs and algorithms, beginning with a search of scholarly articles using the key phrases in the original answer.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-05-13T23:20:58.130" LastActivityDate="2019-05-13T23:20:58.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12260" PostTypeId="2" ParentId="11976" CreationDate="2019-05-11T14:04:12.027" Score="1" Body="&lt;p&gt;&lt;strong&gt;Defining Contemporary Artificial Networks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of &lt;em&gt;an artificial neural network&lt;/em&gt; is a challenge. The most traveled sites on the Internet do not provide any adequate formal definitions and include much misconception. Wikipedia's articles contain some truth but read much like a blog in places, with too much ambiguity and misconception to be considered authoritative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question in the body of this question is phrased in the wider field of AI rather than in the narrower sub-field of machine learning, which is excellent.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;What is an artificial neural network in artificial intelligence?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's the question to be answered in this particular post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer given in the question is a synopsis of common understanding in a single sentence.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;It is apparently used to find patterns in data&#xA;and it is loosely inspired by human neural networks.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Common Misconception&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This synopsis sentence is an accurate representation of common understanding, but it that common conception is inaccurate in a few ways.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There is very little in current artificial networks inspired specifically from human brains. Insect brains could just as easily be construed as the inspiration. Current artificial networks do not yet simulate the cognitive abilities of the cerebral cortex of people.&lt;/li&gt;&#xA;&lt;li&gt;Pattern recognition can be accomplished with an artificial network, but that is not their defining characteristic. When machine learning is directed at the task of recognizing a pattern, it is not the same as targeting the AlphaZero at winning a Chess or Go tournament. It is not the same as a child recognizing her or his mother's face when walking among many adults at the mall. The concept class of a specific pattern is trained into the network by providing a loss function that compares the network output with an expectation from labels, some external signal, or a correlative model.&lt;/li&gt;&#xA;&lt;li&gt;Current artificial networks don't actually recognize anything. They can be trained to do many things that are tiny subsets of recognition, but they don't see and understand that what they are seeing is a car, pedestrian, missile, submarine, tapped hole into which a screw can be screwed, or crumb to be vacuumed up. They don't think, &quot;Oh no, and ICBM. I'd better send out some countermeasures and notify the President.&quot; They don't know what a missile is, but they can detect the features of one, having been trained to do so, and perform some action they were trained to perform.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What Then is an Artificial Network in the Context of AI?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever definition we pick, it must apply to unadorned feed forward networks, CNNs, attention based networks, GRU cell networks, and other designs inspired by the original Perceptrons and CNNs, developed prior to the discovery of usable gradient descent and back-propagation mathematics and corresponding software technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our fitting of artificial networks into the context of AI is limited by the lack of cross-cultural or academic consensus on what intelligence is. Because of that, we cannot yet produce a formal definition on what qualifies as artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Allan Turing's &lt;em&gt;Imitation Game&lt;/em&gt; was intended by him to be a thought experiment to consider what we think of as intelligence when in conversation. He never intended it to be a definition or a test of intelligence, but rather a way to test natural language and perhaps cognitive ability in machines against a human. If the machine could not be differentiated from the human in a double blind experiment, then the machine was, at at least a superficial level, acting human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Definition By Universal Features of Artificial Networks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider what is common to all contemporary artificial networks.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The idea of interconnected cells that together carry a signal, often in sequences of layers but not necessarily so, producing a structure that is conducive to learning an output behavior as a function of input stimuli: &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y} = f(\mathcal{X})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is a representation of input stimuli and $\mathcal{Y} is a representation of output behavior. &amp;mdash; If the cells are, as is the typical convention, arranged in layers, signals propagate from the input layer to the output layer to produce a behavior. Each layer contain cells identical to one another except for the values of signal attenuation parameters (usually called &lt;em&gt;weights&lt;/em&gt; or simply &lt;em&gt;parameters&lt;/em&gt;), which can be initialized in some way and then change in subsequent learning iterations based improvement strategies that have been shown to work both theoretically and empirically.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The mathematical concept of convergence on an optimum set of output behaviors in response to input stimuli, which we call training, and which relies on a numerical computation that indicates the degree of optimization &amp;mdash; This can be called the error function, loss function, deviation function, pain function, or disparity function. The inverse function can be called a gain function, value function, pleasure function, accuracy function, or some other similar name. In systems theory terms, these are names of a feedback device, a corrective signal that causes a well designed learning system to adapt its behavior to hopefully achieve the optimum or sufficiently close to it to satisfy the system requirements. It is a hope because artificial networks are not guaranteed to converge on an optimum in every situation. There may be a deficiency in the training data, its presentation to the learning system, computing resources, overall approach, or hyper-parameter.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cell non-linearity &amp;mdash; not representable as a first degree polynomial of the form &lt;span class=&quot;math-container&quot;&gt;$y = A \, \vec{x} + B$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; is the cell input vector from immediately upstream cells in the network and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the cell output &amp;mdash; This is a critical requirement to permit the learning of nontrivial behavioral complexity. Although this first degree polynomial representation is the representation of a typical attenuation at the front end of the cell, at which time &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a parameter vector or matrix and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is an offset, the right hand side of the expression is enclosed in what is normally called an activation function. This name arose from the obsolete conception of neurons, when it was thought that some learned synaptic attenuation of signals (the weights in &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;) and some learned threshold (-&lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;) determined when a neuron fired. This is now known to be a gross oversimplification, for reasons about two thirds of the way down in &lt;a href=&quot;https://ai.stackexchange.com/questions/5955/how-are-artificial-neural-networks-and-the-biological-neural-networks-similar-an/7267#7267&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although there are many other characteristics that are common across many artificial network types, such as gradient descent, back-propagation, learning rate strategies, the set of commonly used activation functions, hyper-parameters, batching, data set analytics, approaches to training and use, and libraries used, none of these are absolute requirements on a design that makes it an artificial network. These three sum up the universal requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fitting Artificial Networks in AI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see that components with the above three characteristics have been used effectively in many systems that bear marks of intelligence. Some of the most notable cases can be enumerated.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The ability to prevail in the games of chess and go&lt;/li&gt;&#xA;&lt;li&gt;The ability to identify and locate objects of certain types in an image&lt;/li&gt;&#xA;&lt;li&gt;The ability to map trajectories or expression changes in a movie&lt;/li&gt;&#xA;&lt;li&gt;The ability to generate images of certain types&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The question of whether these abilities are examples of intelligence is debated. Artificial networks cannot get honest and explain to some humans why it made a particular game move, how it determined what pixels were what object, or what it had in mind when producing a new living room interior design. Artificial networks are not yet cognitive in the way a cognitive scientist uses that term, although no one has proved that they cannot one day develop cognitive abilities.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.pnas.org/content/pnas/112/8/E846.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence of ion channel genome content in early animal evolution&lt;/a&gt;, &#xA;Benjamin J. Liebeskind, David M. Hillis, and Harold H. Zakon, 2014&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.pnas.org/content/91/11/5033&quot; rel=&quot;nofollow noreferrer&quot;&gt;A measure for brain complexity: relating functional segregation and integration in the nervous system&lt;/a&gt;, G Tononi, O Sporns, and G M Edelman, PNAS, 1994&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5955/how-are-artificial-neural-networks-and-the-biological-neural-networks-similar-an/7267#7267&quot;&gt;How artificial networks and biological neural systems are dissimilar&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-05-11T14:04:12.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12261" PostTypeId="2" ParentId="12116" CreationDate="2019-05-11T15:49:40.887" Score="0" Body="&lt;p&gt;&lt;strong&gt;Visualization Appreciated&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The diagrams are nicely thought out. As you refine your comparative design visualizations, you might use something like Inkscape to draw them for web publication, whether or not you decide to submit a paper to a publisher or license your ideas Creative Commons.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Web Research Realities&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reliability of answers from the internet are a function of the author, how much time she or he took in researching the question themselves, what search terms are used, and where the search term is entered. This question falls under the more challenging to research, cases where the must reliable answers are in the results sets of experiments run in proprietary government and corporate research centers.&lt;/p&gt;&#xA;&lt;p&gt;If the GM or Tesla or Toyota or DoD research results were posted on line, someone would likely be fired, sued, and possibly jailed and a team of lawyers would hunt down all references to the posted information, employing international agreements and backbone level content filters to eliminate the dissemination of secrets.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A Better Research Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can determine with a fairly high confidence that a decision based on a single frame is much less likely to lead to collision avoidance, beginning with a simple thought experiment.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Two kids are playing creatively and decide to make up a game that involves a ball. One of the rules of the game is that each player can only run backward, not forward or sideways. That's what makes the same fun for these kids. It's silly because everyone wants to run forward to get to the ball. The rest of the rules of the game are not particularly relevant.&lt;/p&gt;&#xA;&lt;p&gt;A machine driver is processing an image and goes to make the decision. In this case, the term decision indicates the use of a rules based system, whether fuzzy and probabilistic or of the former pure Boolean type, but that is not particularly relevant either. This thought experiment applies to a learned response arising from the training of an architecture built primarily of artificial networks as well.&lt;/p&gt;&#xA;&lt;p&gt;Consider now the case where no decision or learned response is invoked on the basis of the direction of the ball travel or its proximity to the street and path of the vehicle being driven because the kids are facing away from the ball. It would not be reasonable to assume that the training data included this spontaneously made-up game. The result of the combination of child creativity and single frame selection leads to a delayed collision avoidance tactic, at best.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In contrast, if two or three frames are included in the analysis, the feature of the ball moving toward the street and the children, regardless of their orientation with respect to the ball and the street, may be detected as a feature of the entire system through which the vehicle is driving.&lt;/p&gt;&#xA;&lt;p&gt;This is one of astronomical number of examples where the training without the temporal dimension will lead to a much higher likelihood of improper trajectory projection from pixel data on the basis of any reasonable training set than had training and use of the pixel data included the temporal dimension.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Mathematical Analysis&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;When the results of trials with real vehicles move from the domain of corporate intellectual property and the domain of governmental national security artifacts into the public domain, we will see our empirical evidence. Until then we can rest on theory. The above thought experiment and others like it can be represented. Consider the hypothesis of&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ P_{\mathcal{A} = \mathcal{E}(S)} &amp;gt; P_{\mathcal{A} = \mathcal{E}(\vec{S})} \; \text{,}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$P_c$&lt;/span&gt; is the probability given condition &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is the actuality (&lt;em&gt;posteriori&lt;/em&gt;) and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{E}$&lt;/span&gt; is the expectation function applied to instantaneous sensory information &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; on the left hand side of the inequality versus the recent history of instantaneous sensory information &lt;span class=&quot;math-container&quot;&gt;$\vec{s}$&lt;/span&gt; on the right hand side.&lt;/p&gt;&#xA;&lt;p&gt;If we bring intermediate actions (decisioning after each frame is acquired) into the scope of this question, then we may pose the hypothesis in a way that involves Markov's work on causality and prediction in chains of events.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The accuracy of decisioning based on optical acquisition in the context of collision avoidance is higher without the Markov property, in that historical optically acquired data in conjunction with new optically acquired data will produce better trajectory oriented computational collision avoidance results than without historical data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Either of these might take considerable work to prove, but they are both likely to be provable in probabilistic terms with a fairly reasonable set of mathematical constraints placed on the system up front. We know this because the vast majority of thought experiences show an advantage to a vector of frames over a single frame for the determination of actions that are most likely to avoid a collision.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Design&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As is often the case, convolution kernel use in a CNN is likely to be the best design to recognize edge, contour, reflectivity, and texture features in collide-able object detection.&lt;/p&gt;&#xA;&lt;p&gt;Assembly of trajectories (as a somewhat ethereal internal intermediate result) and subsequent determination of beeping, steering, acceleration, breaking, or signaling is likely best handled by recurrent networks of some type, the most publicly touted being b-LSTM or GRU based networks. The attention based handling and preemption discussed in many papers regarding real time system control are among the primary candidates for eventual common use among the designs. This is because changes in focus is common during human driving operations and is even detectable in birds and insects.&lt;/p&gt;&#xA;&lt;p&gt;The simple case is when an ant detects one of these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A large insurmountable object&lt;/li&gt;&#xA;&lt;li&gt;A predator&lt;/li&gt;&#xA;&lt;li&gt;A morsel of good smelling food&lt;/li&gt;&#xA;&lt;li&gt;Water&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The mode of behavior switches, probably in conjunction with the neural pathways for sensory information, when a preemptive stimulus is detected. Humans pilot and drive aircraft and motor vehicles this way too. When you pilot or drive next, bring into consciousness what you have learned unconsciously and this preemptive detection and change of attention and focus of task will become obvious.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-05-11T15:49:40.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12262" PostTypeId="1" CreationDate="2019-05-11T17:43:44.460" Score="0" ViewCount="187" Body="&lt;p&gt;From what I've seen, neural networks take a set of atomic inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want an input to be a variable array, i.e. a group of people (with unique IDs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I didn't care about their ID, I could simply feed a count of people as an input - but I do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thought are to have some form of forward-feeding, and iterate through each person, using their ID as an input to a node representing &quot;insert entered room&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this make sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same ID would be used in different nodes at different points. I.e to a &quot;user left room&quot; or &quot;user sat down&quot;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Also, how would one model two users interacting, atomically for inputs? I'm guess use 2 inputs representing &quot;interacter A&quot; and &quot;interacter B&quot;&lt;/p&gt;&#xA;" OwnerUserId="25574" LastActivityDate="2019-05-11T17:43:44.460" Title="How would you feed a neural network a variable sized array as an input?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="12263" PostTypeId="1" CreationDate="2019-05-11T19:48:00.137" Score="3" ViewCount="90" Body="&lt;p&gt;I am wondering how am I supposed to train a model using actor/critic algorithms in &lt;strong&gt;environments with opponents&lt;/strong&gt;. I tried the followings (using A3C and DDPG):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Play against &lt;strong&gt;random player&lt;/strong&gt;. I had rather good results, but not as good as expected since most interesting states cannot be reached with a random opponent.&lt;/li&gt;&#xA;&lt;li&gt;Play against &lt;strong&gt;list of specific AIs&lt;/strong&gt;. Results were excellent against those AIs, but very bad with never seen opponents&lt;/li&gt;&#xA;&lt;li&gt;Play against &lt;strong&gt;itself&lt;/strong&gt;. Seemed the best to me, but I could not get any convergence due to non-stationary environment. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any thought or advice about this would be very welcome.&lt;/p&gt;&#xA;" OwnerUserId="23818" LastActivityDate="2019-05-16T18:41:42.183" Title="Training actor-critic algorithms in games with opponents" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12264" PostTypeId="1" AcceptedAnswerId="12299" CreationDate="2019-05-12T00:15:10.860" Score="6" ViewCount="610" Body="&lt;p&gt;How do you actually decide what reward value to give for each action in a given state for an environment?&lt;/p&gt;&#xA;&lt;p&gt;Is this purely experimental and down to the programmer of the environment? So, is it a heuristic approach of simply trying different reward values and see how the learning process shapes up?&lt;/p&gt;&#xA;&lt;p&gt;Of course, I understand that the reward values have to make sense, and not just put completely random values, i.e. if the agent makes mistakes then deduct points, etc.&lt;/p&gt;&#xA;&lt;p&gt;So, am I right in saying it's just about trying different reward values for actions encoded in the environment and see how it affects the learning?&lt;/p&gt;&#xA;" OwnerUserId="25360" LastEditorUserId="2444" LastEditDate="2021-01-20T12:49:21.510" LastActivityDate="2021-01-20T12:49:21.510" Title="How do we define the reward function for an environment?" Tags="&lt;reinforcement-learning&gt;&lt;reward-functions&gt;&lt;reward-design&gt;&lt;reward-shaping&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12266" PostTypeId="1" CreationDate="2019-05-12T00:23:20.273" Score="9" ViewCount="5613" Body="&lt;p&gt;What's the differences between &lt;a href=&quot;https://www.scihive.org/paper/1905.03670&quot; rel=&quot;noreferrer&quot;&gt;semi-supervised learning and self-supervised visual representation learning&lt;/a&gt;, and how they are connected? &lt;/p&gt;&#xA;" OwnerUserId="12975" LastEditorUserId="12975" LastEditDate="2019-05-12T07:32:55.153" LastActivityDate="2020-08-04T19:02:26.430" Title="What is the relation between semi-supervised and self-supervised visual representation learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;supervised-learning&gt;&lt;self-supervised-learning&gt;&lt;semi-supervised-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12267" PostTypeId="2" ParentId="12266" CreationDate="2019-05-12T01:16:51.387" Score="8" Body="&lt;h3&gt;Semi-supervised learning&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Semi-supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semi-supervised learning&lt;/a&gt; is the collection of machine learning techniques where there are two datasets: a labelled one and an unlabelled one.&lt;/p&gt;&#xA;&lt;p&gt;There are two main problems that can be solved using semi-supervised learning:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;transductive learning (i.e. label the given unlabelled data) and&lt;/li&gt;&#xA;&lt;li&gt;inductive learning (generalization) (i.e. find a function that maps inputs to outputs, like classification).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Self-supervised learning&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/10624/2444&quot;&gt;Self-supervised learning&lt;/a&gt; (SSL) is a machine learning approach where the supervisory signal is &lt;strong&gt;automatically&lt;/strong&gt; generated. More precisely, SSL can either refer to&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;learn data representations (i.e. learn to represent the data) by solving a so-called &lt;strong&gt;pretext&lt;/strong&gt; (or auxiliary) task, in a self-supervised fashion, i.e. you automatically generate the supervised signal from the unlabelled&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;automatically label an unlabelled dataset by exploiting data coming from different sensors (this is the usual definition of SLL in the context or robotics)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;What is the relationship between the two?&lt;/h3&gt;&#xA;&lt;p&gt;SSL (for data representation) can be considered a semi-supervised learning approach, if you fine-tune the learned data representations with a labeled dataset to solve supervised learning problem (i.e. the so-called &lt;strong&gt;downstream&lt;/strong&gt; task), which you will probably do otherwise the data representations are pretty useless. Read &lt;a href=&quot;https://ai.stackexchange.com/a/22860/2444&quot;&gt;this answer&lt;/a&gt; for other details.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-04T19:02:26.430" LastActivityDate="2020-08-04T19:02:26.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12268" PostTypeId="1" AcceptedAnswerId="12269" CreationDate="2019-05-12T02:41:42.447" Score="1" ViewCount="635" Body="&lt;p&gt;I have the following code (below), where an agent uses Q-learning (RL) to play a simple game.&lt;br&gt;&#xA;What appears to be questionable for me in that code is the fixed learning rate. When it's set low, it's always favouring the old Q-value over the learnt/new Q-value (which is the case in this code example) and vice-versa when it's set high. My thinking was shouldn't the learning rate be dynamic, i.e. it should start high because at the beginning we don't have any values in the Q-table and the agent is simply choosing the best actions it encounters. So we should be favouring the new Q-values over the existing ones (in the Q-table, which there's no values, just zeros at the start). Over time (say every &lt;code&gt;n&lt;/code&gt; number of episodes), ideally we decrease the learning rate to reflect that overtime the values in the Q-table are getting more and more accurate (with the help of the Bellman equation to update the values in the Q-table). So lowering the learning rate will start to favour the existing value in the Q-table over the new ones. I'm not sure if my logic has gaps and flaws, but I'm putting it out there in the community to get feedback from experienced/experts opinions. &#xA;Just to make things easier, the line to refer to, in the code below (for updating the Q-value using the learning rate) is under the comment: &lt;code&gt;# Update Q-table for Q(s,a) with learning rate&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import gym&#xA;import random&#xA;import time&#xA;from IPython.display import clear_output&#xA;&#xA;env = gym.make(&quot;FrozenLake-v0&quot;)&#xA;&#xA;action_space_size = env.action_space.n&#xA;state_space_size = env.observation_space.n&#xA;&#xA;q_table = np.zeros((state_space_size, action_space_size))&#xA;&#xA;num_episodes = 10000&#xA;max_steps_per_episode = 100&#xA;&#xA;learning_rate = 0.1&#xA;discount_rate = 0.99&#xA;&#xA;exploration_rate = 1&#xA;max_exploration_rate = 1&#xA;min_exploration_rate = 0.01&#xA;exploration_decay_rate = 0.001&#xA;&#xA;&#xA;rewards_all_episodes = []&#xA;&#xA;for episode in range(num_episodes):&#xA;    # initialize new episode params&#xA;    state = env.reset()&#xA;&#xA;    done = False &#xA;    rewards_current_episode = 0 &#xA;&#xA;    for step in range(max_steps_per_episode):&#xA;&#xA;        # Exploration-exploitation trade-off&#xA;        exploration_rate_threshold = random.uniform(0, 1)&#xA;        if exploration_rate_threshold &amp;gt; exploration_rate:&#xA;            action = np.argmax(q_table[state,:])&#xA;        else:&#xA;            action = env.action_space.sample()&#xA;&#xA;        new_state, reward, done, info = env.step(action)&#xA;&#xA;        # Update Q-table for Q(s,a) with learning rate&#xA;        q_table[state, action] = q_table[state, action] * (1 - learning_rate) + \&#xA;            learning_rate * (reward + discount_rate * np.max(q_table[new_state, :]))&#xA;&#xA;        state = new_state&#xA;        rewards_current_episode += reward&#xA;&#xA;        if done == True:&#xA;            break&#xA;&#xA;&#xA;    # Exploration rate decay&#xA;    exploration_rate = min_exploration_rate + \&#xA;        (max_exploration_rate - min_exploration_rate) * np.exp(-exploration_decay_rate*episode)&#xA;&#xA;    rewards_all_episodes.append(rewards_current_episode)&#xA;&#xA;# Calculate and print the average rewards per thousand episodes&#xA;rewards_per_thousands_episodes = np.array_split(np.array(rewards_all_episodes), num_episodes/1000)&#xA;count = 1000&#xA;print(&quot;******* Average reward per thousands episodes ************&quot;)&#xA;for r in rewards_per_thousands_episodes:&#xA;    print(count, &quot;: &quot;, str(sum(r/1000)))&#xA;    count += 1000&#xA;&#xA;# Print updated Q-table&#xA;print(&quot;\n\n********* Q-table *************\n&quot;)&#xA;print(q_table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25360" LastActivityDate="2019-05-12T08:42:44.197" Title="Static or dynamic learning rate (Q-learning)" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12269" PostTypeId="2" ParentId="12268" CreationDate="2019-05-12T08:31:23.343" Score="2" Body="&lt;p&gt;Yes you can decay the learning rate in Q-learning, and yes this should result in more accurate Q-values in the long term for many environments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this is something that is harder to manage than in supervised learning, and might not be as useful or necessary. The issues you need to be concerned about are:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Non-stationarity&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The target values in value-based optimal control methods are non-stationary. The TD target derived from the Bellman equation is never quite a sample from the optimal action value until the very final stages of learning. This is an issue both due to iterative policy improvements and the bootstrap nature of TD learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reducing learning rate before you reach optimal control could delay finding the optimal policy. In general you want the learning rate to be just low enough that inaccuracies due to over/undershooting the correct value don't prevent or delay differentiating between actions for whatever the interim policy is.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Policy indifference to accurate action values&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Q-learning's policy is based on &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \text{argmax}_{a} Q(s,a)$&lt;/span&gt;. To obtain an optimal policy, what you care about is identifying the highest-valued action, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. As such, it is possible in some environments to achieve optimal control whilst Q values are far from accurate, provided the highest-valued action still has the highest estimate of expected return.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether or not you need highly accurate Q values to determine a policy depends on the environment, how similar actions are to each other. For long-term forecasts (i.e. &lt;span class=&quot;math-container&quot;&gt;$\gamma \approx 1$&lt;/span&gt;) where individual time steps make little difference, this is a more important detail, as there will be only minor differences between action values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your goal is to get highly accurate predictions of action values, then the above does not necessarily apply.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Some deterministic environments can use a high learning rate&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This only applies to some, simpler environments, but you should definitely bear this in mind when working with tabular solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, it is possible to apply tabular Q-learning to Tic Tac Toe with a learning rate of &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt; - essentially replacing each estimate with a new latest estimate - and it works just fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other, more complex environments, this would be a problem and the algorithm would not converge. But clearly, adding a learning rate decay is not a &lt;em&gt;general&lt;/em&gt; requirement.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The learning rate decay schedule is a hyper parameter&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There is no generic schedule that could apply to all environments and be equally effective in them. For an optimal approach, you would need to run a search over possible decay schedules, and the most efficient learning rate decay would apply only to the environment that you tested. It would likely apply well enough to similar environments that it could be an advantage to know it in future.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;In complex environments, there are other working solutions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In practice, for environments where neural networks are used, an approach called Deep Q Networks (DQN) is used. It is common to use a conservative/low learning rate with this due to stability issues of combining neural networks and off-policy reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, you will quite often see an adaptive gradient optimiser - e.g. Adam or RMSProp - used with DQN. These already adjust gradient step sizes based on recent gradients observed in the neural network during training, so there is less need to add a dynamic learning rate schedule on top. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supervised learning challenges, such as ImageNet &lt;em&gt;do&lt;/em&gt; sometimes add a learning rate schedule over adaptive optimisers, resulting in further improvements. So it might help with DQN, but the other caveats above could still apply.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-12T08:42:44.197" LastActivityDate="2019-05-12T08:42:44.197" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12270" PostTypeId="1" AcceptedAnswerId="12272" CreationDate="2019-05-12T11:15:54.263" Score="3" ViewCount="395" Body="&lt;p&gt;Consider the grid world problem in RL. Formally, policy in RL is defined as &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;. If we are solving grid world by policy iteration then the following pseudocode is used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QU6Z8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QU6Z8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is related to the policy improvement step. Specifically, I am trying to understand the following update rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) \leftarrow arg max_a \sum_{s'}p(s'|s,a)[r(s,a,s') + \gamma v(s')] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can have 2 interpretations of this update rule.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In this step, we check which action (say, going right for a particular state) has the highest reward and assign going right a probability of 1 and rest actions a probability of 0. Thus, in PE step, we will always go right for all iterations for that state even if it might not be the most rewarding function after certain iterations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We keep the policy improvement step in mind, and, while doing the PE step, we update the &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;, based on the action giving highest reward (say, for 1st iteration &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt;, going right gives the highest reward, we update based on that, while, for &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt;, we see going left gives the highest reward, and update our value based on that likewise. Thus action changes depending on maximum reward).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For me, the second interpretation is very similar to value iteration. So, which one is the correct interpretation of a policy iteration?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-05-01T12:34:18.487" LastActivityDate="2020-05-01T12:34:18.487" Title="Understanding the update rule for the policy in the policy iteration algorithm" Tags="&lt;reinforcement-learning&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;&lt;policy-improvement&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12271" PostTypeId="1" CreationDate="2019-05-12T13:54:47.517" Score="3" ViewCount="123" Body="&lt;p&gt;I was trying to figure out how to create a solver to the puzzle of putting 11 pieces in a board (8 x 8). I created the game in &lt;a href=&quot;http://www.xams.com.br/quebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.xams.com.br/quebra&lt;/a&gt;. It is possible to turn the piece 90 degrees each time counterclockwise (&lt;em&gt;Girar&lt;/em&gt;) and mirror it vertically (&lt;em&gt;Inverter&lt;/em&gt;), and so the piece can assume 8 forms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When clicking in solver button (&lt;em&gt;Resolver&lt;/em&gt;), it tries to put pieces randomly in board in brutal force method (it spends a &lt;strong&gt;LOT&lt;/strong&gt; of time). Using this method, I was not able to achieve the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to try something smarter than this and having a machine learning algorithm for this would be great. I don´t know how to formulate the problem. How would you start this please?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;I am still building the page and there is so much to improve. You need to click the center of 3x3 matrix where you want to put the piece.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT2:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eq7iY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eq7iY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16837" LastEditorUserId="2444" LastEditDate="2019-05-12T23:44:37.587" LastActivityDate="2019-05-20T00:18:34.183" Title="How do I solve the problem of positioning 11 pieces into a 8x8 puzzle?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12272" PostTypeId="2" ParentId="12270" CreationDate="2019-05-12T14:07:53.083" Score="2" Body="&lt;p&gt;A policy can be stochastic or deterministic. A deterministic policy is a function of the form &lt;span class=&quot;math-container&quot;&gt;$\pi_{\text{deterministic}}: S \rightarrow A$&lt;/span&gt;, that is, a function from the set of states to the set of actions. A stochastic policy is a map of the form &lt;span class=&quot;math-container&quot;&gt;$\pi_{\text{stochastic}} : S \rightarrow P(A)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$P(A)$&lt;/span&gt; is a set of probability distributions (&lt;span class=&quot;math-container&quot;&gt;$P(A) = \{ p_{s_1}(A), p_{s_2}(A), \dots, p_{s_{|S|}}(A) \}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p_{s_i}(A)$&lt;/span&gt; is a probability distribution over the set of actions &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; for the state &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|S|$&lt;/span&gt; is the size of the set of states of the environment) over the set of actions &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. A deterministic policy can be interpreted as a stochastic policy that gives the probability of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to one of the available actions (and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to the remaining actions), for each state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of value iteration (VI) and policy iteration (PI), the policy is deterministic, both in the policy evaluation (PE) or policy improvement (PI) steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the PE step, for a &lt;strong&gt;specific&lt;/strong&gt; state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (inside the for loop), you use &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt;, that is, you assume that the action taken in the &lt;em&gt;specific&lt;/em&gt; state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the greedy action, because of the &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}$&lt;/span&gt; in the PI, according to the current policy. However, in general, &lt;span class=&quot;math-container&quot;&gt;$\pi_k(s_i) \neq \pi_k(s_j)$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the current iteration step (of PE and PI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The update rule&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi_{k+1}(s) \leftarrow arg max_a \sum_{s'}p(s' \mid s, a)[r(s, a, s') + \gamma v_k(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;computes the action that is &lt;strong&gt;expected&lt;/strong&gt; to give the highest return, given the current and fixed value function &lt;span class=&quot;math-container&quot;&gt;$(v_k$&lt;/span&gt;). Have a look at the definition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectation&lt;/a&gt; for discrete random variables: it is defined as a weighted sum (like in the update rule above, where the weights are &lt;span class=&quot;math-container&quot;&gt;$p(s' \mid s, a)$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Value iteration is a shorter version of policy iteration. In VI, rather than performing a PI step for each state of the environment, when performing PE, the action that is &lt;em&gt;expected&lt;/em&gt; to give the highest return is chosen. In VI, the policy is updated only once and at the end of policy evaluation. In PI, you alternate between PE and PI, and, at each PI, you update the policy.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-12T16:19:28.950" LastActivityDate="2019-05-12T16:19:28.950" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="12273" PostTypeId="1" CreationDate="2019-05-12T18:42:16.040" Score="1" ViewCount="39" Body="&lt;p&gt;I'm building a web application that collects &lt;a href=&quot;https://schema.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;schema.org&lt;/a&gt; data from different webshops as Amazon, Shopify, etc. It collects data every 6h and shows the current and lowest price. It is used for monitoring products and buying at the lowest price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to &lt;strong&gt;recognize products from different shops as the same product.&lt;/strong&gt; Every shop has its own title for the same product. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Google Pixel 2 64GB Clearly White (Unlocked) Smartphone &#xA;Google Pixel 2 GSM/CDMA Google Unlocked (Clearly White, 64GB, US warranty) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Problems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;don't have a lot of data (only products chosen by the user)&lt;/li&gt;&#xA;&lt;li&gt;needs to support every new product that app doesn't have data history&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25589" LastEditorUserId="16565" LastEditDate="2019-05-13T08:37:34.733" LastActivityDate="2019-05-13T08:37:34.733" Title="String matching algorithm for product recognition" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;similarity&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12274" PostTypeId="1" AcceptedAnswerId="12275" CreationDate="2019-05-12T18:50:32.373" Score="3" ViewCount="4075" Body="&lt;p&gt;In reinforcement learning, there are the concepts of stochastic (or probabilistic) and deterministic policies. What is the difference between them?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-12T18:59:23.210" LastActivityDate="2021-01-21T15:55:37.613" Title="What is the difference between a stochastic and a deterministic policy?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policies&gt;&lt;deterministic-policy&gt;&lt;stochastic-policy&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12275" PostTypeId="2" ParentId="12274" CreationDate="2019-05-12T18:50:32.373" Score="7" Body="&lt;p&gt;A &lt;strong&gt;deterministic policy&lt;/strong&gt; is a function of the form &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{d}}: S \rightarrow A$&lt;/span&gt;, that is, a function from the set of states of the environment, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, to the set of actions, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. The subscript &lt;span class=&quot;math-container&quot;&gt;$_{\mathbb{d}}$&lt;/span&gt; only indicates that this is a &lt;span class=&quot;math-container&quot;&gt;${\mathbb{d}}$&lt;/span&gt;eterministic policy.&lt;/p&gt;&#xA;&lt;p&gt;For example, in a grid world, the set of states of the environment, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, is composed of each cell of the grid, and the set of actions, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, is composed of the actions &amp;quot;left&amp;quot;, &amp;quot;right&amp;quot;, &amp;quot;up&amp;quot; and &amp;quot;down&amp;quot;. Given a state &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; is, with probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, always the same action (e.g. &amp;quot;up&amp;quot;), unless the policy changes.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;stochastic policy&lt;/strong&gt; is often represented as a &lt;em&gt;family&lt;/em&gt; of &lt;a href=&quot;https://brilliant.org/wiki/conditional-probability-distribution/&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional probability distributions&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(A \mid S)$&lt;/span&gt;, from the set of states, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, to the set of actions, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. A probability distribution is a function that assigns a probability for each &lt;a href=&quot;https://en.wikipedia.org/wiki/Event_(probability_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;event&lt;/a&gt; (in this case, the events are actions in certain states) and such that the sum of all the probabilities is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A stochastic policy is a family and not just one conditional probability distribution because, for a fixed state &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(A \mid S = s)$&lt;/span&gt; is a possibly distinct conditional probability distribution. In other words, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(A \mid S) = \{ \pi_{\mathbb{s}}(A \mid S = s_1), \dots, \pi_{\mathbb{s}}(A \mid S = s_{|S|})\}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(A \mid S = s)$&lt;/span&gt; is a conditional probability distribution over actions given that the state is &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|S|$&lt;/span&gt; is the size of the set of states of the environment.&lt;/p&gt;&#xA;&lt;p&gt;Often, in the reinforcement learning context, a stochastic policy is misleadingly denoted by &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a \in A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; are respectively a specific action and state, so &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt; is just a number and not a conditional probability distribution. A single conditional probability distribution can be denoted by &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(A \mid S = s)$&lt;/span&gt;, for some fixed state &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt;. However, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt; can also denote a family of conditional probability distributions, that is, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(A \mid S) = \pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt;, if &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; are arbitrary. Alternatively, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt; are just (dummy or input) variables of the function &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt; (i.e. p.m.f. or p.d.f.): this is probably the most sensible way of interpreting &lt;span class=&quot;math-container&quot;&gt;$\pi_{\mathbb{s}}(a \mid s)$&lt;/span&gt; when you see this notation (see also &lt;a href=&quot;https://ai.stackexchange.com/a/16842/2444&quot;&gt;this answer&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In the particular case of games of chance (e.g. poker), where there are sources of randomness, a deterministic policy might not always be appropriate. For example, in poker, not all information (e.g. the cards of the other players) is available. In those circumstances, the agent might decide to play differently depending on the round (time step). More concretely, the agent could decide to go &amp;quot;all-in&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$\frac{2}{3}$&lt;/span&gt; of the times whenever it has a hand with two aces and there are two uncovered aces on the table and decide to just &amp;quot;raise&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{3}$&lt;/span&gt; of the other times.&lt;/p&gt;&#xA;&lt;p&gt;A deterministic policy can be interpreted as a stochastic policy that gives the probability of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to one of the available actions (and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to the remaining actions), for each state.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T15:55:37.613" LastActivityDate="2021-01-21T15:55:37.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12276" PostTypeId="1" CreationDate="2019-05-12T19:02:49.683" Score="2" ViewCount="57" Body="&lt;p&gt;The Q-learning does not guarantee convergence for continuous state space problems (&lt;a href=&quot;https://ai.stackexchange.com/questions/11679/why-doesnt-q-learning-converge-when-using-function-approximation&quot;&gt;Why doesn&amp;#39;t Q-learning converge when using function approximation?&lt;/a&gt;). In that case, is there an algorithm which can guarantee convergence? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking at model-based RL specifically iLQR but all the solutions I find are for the continuous action space problem.&lt;/p&gt;&#xA;" OwnerUserId="19541" LastEditorUserId="2444" LastEditDate="2019-05-12T19:07:42.540" LastActivityDate="2019-05-12T19:07:42.540" Title="Are there reinforcement learning algorithms that ensure convergence for continuous state space problems?" Tags="&lt;reinforcement-learning&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12277" PostTypeId="1" CreationDate="2019-05-12T19:13:15.043" Score="0" ViewCount="294" Body="&lt;p&gt;I'm trying to make a dark image brighter using CNN-UNet architecture.&lt;/p&gt;&#xA;&lt;p&gt;When I train the network, I get the following results:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WXKjz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WXKjz.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;When I cut the features in half for pruning, and do full train again, I get the following result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OLtsv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OLtsv.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are those bubble artifacts near the light.&lt;/p&gt;&#xA;&lt;p&gt;Why are they happening?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I locate the problematic neurons that cause it?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I train the network (change the loss or anything) to not generate those made up bubbles?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25141" LastEditorUserId="2444" LastEditDate="2021-02-03T10:09:21.947" LastActivityDate="2021-03-05T11:07:20.620" Title="Artifacts After pruning Unet CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;image-processing&gt;&lt;u-net&gt;&lt;image-denoising&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12279" PostTypeId="1" CreationDate="2019-05-13T05:02:36.080" Score="1" ViewCount="32" Body="&lt;p&gt;I am training an agent to play a simple game using double deep q learning. However, the variance in agent performance is very high, even for agents trained with same model parameters. For example, I can train agent A and agent B using the exact parameters and agent A beats B 800 to 200.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I understand why this is happening, when training starts the model is initialized with different weights, and this leads the model to find different local max/min. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above makes it difficult to find optimal parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the strategies to reduce this variance? What parameters should I look at tweaking?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;More details about the environment:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a two player game (Zombie Dice); however, in my implementation so far the agents are learning to maximize expected score on their turn, so the actions and score of the opponent is ignored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variance is higher when I am using purely greedy strategy with no exploitation at all. Though it exists in both cases. I would say roughly 2/3 wins for stronger side with greedy and 3/5 with exploration out of 1000 matches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environment is stochastic; I have not done many assessment runs maybe 20 or 30, it is mostly eyeballing, but the differences are fairly large; therefore, I am confident that this is not due to chance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tested the models against themselves, and I get scores very close to 50/50. However, two different models trained with same parameters give results very different from 50/50. I tested this with models trained with different types of parameters and it is generally the same problem.&lt;/p&gt;&#xA;" OwnerUserId="25248" LastEditorUserId="1847" LastEditDate="2019-05-14T06:51:59.043" LastActivityDate="2019-05-14T06:51:59.043" Title="High variance in performance of q-learning agents trained with same parameters" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12281" PostTypeId="2" ParentId="12277" CreationDate="2019-05-13T08:26:55.133" Score="0" Body="&lt;p&gt;Did you retrain the network after pruning the weights?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, 1) the bubbles are probably appearing because of that, and 2) you definitely should retrain your network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did you prune output features only or weights inside the network also?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard procedure is to retrain the network after pruning, and the best results can be achieved with iterative pruning: you prune small part of weights (the loss of accyracy is inevitable here), then retrain to get the accuracy back, and repeat until you won't be able to get the accuracy back to high level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;a href=&quot;https://arxiv.org/abs/1611.06440&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pruning Convolutional Neural Networks for Resource Efficient Inference&lt;/a&gt; is an article about (among others) iterative pruning.&lt;/p&gt;&#xA;" OwnerUserId="25520" LastActivityDate="2019-05-13T08:26:55.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12282" PostTypeId="2" ParentId="11785" CreationDate="2019-05-13T08:39:18.563" Score="1" Body="&lt;p&gt;Roads maps are well defined and you can access them online. For example you can visit OpenStreetMap to download road network of a given region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a road network definition contains nodes (junctions) with lat/lon coordinates and edges between nodes (roads). An edge is defined by a connection of two nodes. Edges can represent one-way roads: you need to define double ways as two edges going in both ways. Haversine formula gives you the straight distance between two nodes: the length of the edge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition of a road network gives you a graph (in the mathematical point of view) and is largely covered by the literature. You can search the graph and compute many of its parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a road network definition you can run shortest path algorithm to find the closest way between two nodes. From a given coordinate you can either pick the closest node or get the closest edge (and compute distance to the edge and the coordinates of the perpendicular projection point). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms like A* and Dijkstra are not so time consuming and you can find shortest path efficiently.&lt;/p&gt;&#xA;" OwnerUserId="25555" LastEditorUserId="12509" LastEditDate="2019-05-15T18:29:03.030" LastActivityDate="2019-05-15T18:29:03.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12283" PostTypeId="1" CreationDate="2019-05-13T13:43:31.060" Score="1" ViewCount="591" Body="&lt;p&gt;I am designing a neural network using Deep Q-Learning, which teaches an agent how to play Snake (The classic Nokia game from the 90'ies). The goal of the game is to navigate the snake on a playing field (2D), and to eat a randomly placed fruit. As the Snake eats the fruit, it grows in length. The game ends if the snake hits the game border, or it self, so as the number of fruits consumed increases, so does the difficulty of navigating without hitting something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have trained the Snake game on a 10x10 playing field using the following inputs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;x direction of the snake&lt;/li&gt;&#xA;&lt;li&gt;y direction of the Snake &lt;/li&gt;&#xA;&lt;li&gt;The fraction of playing field occupied by the snake itself &lt;/li&gt;&#xA;&lt;li&gt;A bool which says if the fruit is in front of the snake (from it's current direction)&lt;/li&gt;&#xA;&lt;li&gt;A bool which says if the fruit is to the left of the snake (from&#xA;it's current direction) &lt;/li&gt;&#xA;&lt;li&gt;A bool which says if collision (game over) in front of the snake is possible &lt;/li&gt;&#xA;&lt;li&gt;A bool which says if collision to the right is possible&lt;/li&gt;&#xA;&lt;li&gt;A bool which says if collision to the left is possible&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With this choice of inputs I am getting the Snake agent to work reasonably well, and score until it plateaus out. I have examined the cases where the snake dies, and it all happens when it has no way of escaping, for example, it turns around and blocks its own path, until it finally has no where to go. This is more likely to happen as the Snake increases in lenght.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking on how I could improve this performance. It seems to me, that the reason the Snake can make a self-blocking turn is because of the inputs. Since the path it takes is clear, there's no information, that the next path is not, or that continuing further down will eventually lead to game over. If the Snake agent was aware of all obstacles in each step, i.e. the entire state-space, then I could imagine that would help train the network towards finding the optimal path without ending up blocking itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I have made the Snake game myself, I can return to the agent a matrix, or an unfolded vector, which contains the inputs for each column / row on the playing field. A blocked cell would be set to 0, a free cell 1, the fruit cell has value 0.75 and the snake head (which moves) could be assigned value 0.25. After trying this approach, I have to say I was unsuccesful. The snake ends up just turning in circles, even if I use the same reward system as the 8 input case shown above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am therefore trying to understand what is happening here. Am I missing information? I would think the full 10x10 state space would give me exactly enough information to lead to the correct evaluation of the next path. I would very much like to hear someone elses input to this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="25606" LastActivityDate="2019-05-14T07:47:22.513" Title="Choice of inputs features for Snake game" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12284" PostTypeId="1" CreationDate="2019-05-13T16:12:59.710" Score="2" ViewCount="281" Body="&lt;p&gt;Monte Carlo (MC) methods are methods that use some form of randomness or sampling. For example, we can use an MC method to approximate the area of a circle inside a square: we generate random 2D points inside the square and count the number of points inside and outside the circle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reinforcement learning, an MC method is a method that &quot;samples&quot; experience (in the form of &quot;returns&quot;) from the environment, in order to approximate e.g. a value function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A temporal-difference algorithm, like &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, also performs some form of sampling: it chooses an action using a stochastic policy (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy) and observes a reward and the next state.  So, couldn't &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning also be considered a MC method? Can an MC method be model-based?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-13T16:20:35.523" LastActivityDate="2019-05-13T18:10:49.223" Title="What is the relation between Monte Carlo and model-free algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;comparison&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12285" PostTypeId="1" CreationDate="2019-05-13T16:50:39.160" Score="0" ViewCount="31" Body="&lt;p&gt;I know case-based reasoning has four stages: retrieve, retain, re-use and revise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Used for solving new problems by adapting solutions that were used to solve old problems, like car issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantages of it are that solutions are proposed quickly and there's no need to start from scratch, but is there a real-world problem where using this would not be suitable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just trying to better my understanding of it.&lt;/p&gt;&#xA;" OwnerUserId="25609" LastEditorUserId="2444" LastEditDate="2019-05-13T17:41:08.847" LastActivityDate="2019-05-13T17:41:08.847" Title="Are there real-world problems where case-based reasoning is not suitable?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12286" PostTypeId="1" CreationDate="2019-05-13T17:17:53.707" Score="1" ViewCount="26" Body="&lt;p&gt;Consider a problem where we have a finite number of states and actions. Given a state and an action, we can easily know the reward and the next state (deterministic). The state space is really large and has no final state.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was a paper that for a problem of this type used TD(0) by filling the value table and chose its actions by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a (r(s,a) + \gamma V(s_{(t+1)}))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read somewhere that is OK to use prediction algorithms when the model is well-described with the objective of choosing best actions and not only evaluating the policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the purpose, advantages and disadvantages of using TD prediction here instead of a TD control algorithms (and just saving the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; table)? If it was about space, you still have to store a table with all the rewards for each state-action pair, right? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if I was able to explain myself very well as I was trying to keep it short, if some clarification is needed please tell me.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="2444" LastEditDate="2019-05-13T17:38:29.977" LastActivityDate="2019-05-13T17:38:29.977" Title="Why should we use TD prediction as opposed to TD control algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12287" PostTypeId="2" ParentId="12284" CreationDate="2019-05-13T17:19:56.667" Score="2" Body="&lt;p&gt;In Reinforcement Learning (RL), the use of the term Monte Carlo has been slightly adjusted by convention to refer to only a few specific things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more general use of &quot;Monte Carlo&quot; is for simulation methods that use random numbers to sample - often as a replacement for an otherwise difficult analysis or exhaustive search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In RL, Monte Carlo methods are generally taken to be non-bootstrapping sample-based approaches to estimating returns. This is a labelling convention within RL - probably because someone called an initial model-free learner a &quot;Monte Carlo method&quot;, and the name stuck whilst many refinements and new ideas have since been published under different names.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The historical use of the term is important, since if you mention you are using &quot;Monte Carlo Control&quot;, it usually means a very specific subset of methods within RL, to most readers.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, couldn't -learning also be considered a MC method? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the &lt;em&gt;general&lt;/em&gt; sense perhaps. The argument is perhaps stronger for it if the environment is being simulated on a computer for the agent to learn from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you start to unilaterally call Q-learning a MC method, you are probably going to just confuse people who have learned the conventions in RL.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can an MC method be model-based?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, yes, because the model can be sampled for planning, and a policy can also be separately sampled - so it is possible to run an MC method with or without a model - depending on whether you sample from the model taking &quot;virtual actions&quot; (for e.g. planning or refining your agent) or from the environment taking real actions. Many RL techniques blur the line between online learning and planning. For instance, using a simulated environment or historical data can be framed as planning for the real environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo Tree Search is an example of a model-based technique using the term &quot;Monte Carlo&quot; within RL frameworks. It is used famously within DeepMind's AlphaZero in order to refine a policy and value estimates during self-play.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the relation between Monte Carlo and [other] model-free algorithms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the context of RL, Monte Carlo is presented as one way to estimate expected &lt;em&gt;utility&lt;/em&gt; (or &lt;em&gt;return&lt;/em&gt;) - by sampling from the environment and policy until the a complete trajectory is available:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) = \mathbb{E}_{\pi}[\sum_{k=0}^{T-t} \gamma^k R_{t+1+k} | S_t = s ]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MC is contrasted with Temporal Difference (TD) approaches, such as Q-learning, which sample &lt;em&gt;bootstrap&lt;/em&gt; estimates using the Bellman Equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) = \mathbb{E}_{\pi}[R_{t+1} + \gamma v_{\pi}(S_{t+1}) | S_t = s ]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two approaches can be combined in various ways, including TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) methods. With  TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;), if you set &lt;span class=&quot;math-container&quot;&gt;$\lambda = 0$&lt;/span&gt; then the algorithm is identical to single-step TD learning, and if you set &lt;span class=&quot;math-container&quot;&gt;$\lambda = 1$&lt;/span&gt; then it is very similar to a Monte Carlo method. Often setting it to some intermediate value is more efficient than either extreme.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-13T18:10:49.223" LastActivityDate="2019-05-13T18:10:49.223" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12288" PostTypeId="2" ParentId="12095" CreationDate="2019-05-13T17:27:44.853" Score="2" Body="&lt;p&gt;So this paper is by google, but is very similar where they use 2D positional embeddings and perform MHA on the flattened image. Are you talking about &lt;a href=&quot;https://arxiv.org/pdf/1904.09925.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Augmented Convolutional Networks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-13T17:27:44.853" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12289" PostTypeId="1" CreationDate="2019-05-13T20:13:47.060" Score="2" ViewCount="89" Body="&lt;p&gt;Our customer runs a tour agency. He has an excel spreadsheet containing the following information for people that have contacted them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Customer name, country, tour duration (requested by customer), tour date, number of people in the tour (usually from 1 to 3), price given to the customer, answer: accepted/rejected (indicates if customer accepted or rejected the price given by the tour agency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My customer wants a predictor or tool that can let him enter the details given by future customers, e.g:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of participants,&#xA;Tour duration,&#xA;Country (not sure if necessary?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the system will return the best price to charge the customer (so he won't reject the proposal but pay the maximum possible). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be that the tour agency owner will enter the price and the system will answer &quot;Customer will accept that price&quot; or &quot;Customer will reject that price&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this even possible? I think it may be done using neural networks trained with the previous answers from customers that the tour agency owner has in his excel spreadsheet? &lt;/p&gt;&#xA;" OwnerUserId="25614" LastActivityDate="2019-05-14T13:35:41.070" Title="Predict best price using neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12290" PostTypeId="2" ParentId="4683" CreationDate="2019-05-13T21:21:24.660" Score="7" Body="&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/12043/2444&quot;&gt;Recurrent neural networks&lt;/a&gt;&lt;/strong&gt; (RNNs) are artificial neural networks (ANNs) that have one or more recurrent (or cyclic) connections, as opposed to just having feed-forward connections, like a feed-forward neural network (FFNN). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These cyclic connections are used to keep track of temporal relations or dependencies between the elements of a sequence. Hence, RNNs are suited for sequence prediction or related tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the picture below, you can observe an RNN on the left (that contains only one hidden unit) that is equivalent to the RNN on the right, which is its &quot;unfolded&quot; version. For example, we can observe that &lt;span class=&quot;math-container&quot;&gt;$\bf h_1$&lt;/span&gt; (the hidden unit at time step &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;) receives both an input &lt;span class=&quot;math-container&quot;&gt;$\bf x_1$&lt;/span&gt; and the value of the hidden unit at the previous time step, that is, &lt;span class=&quot;math-container&quot;&gt;$\bf h_0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7HI1R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7HI1R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cyclic connections (or the weights of the cyclic edges), like the feed-forward connections, are learned using an optimisation algorithm (like gradient descent) often combined with back-propagation (which is used to compute the gradient of the loss function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convolutional neural networks&lt;/strong&gt; (CNNs) are ANNs that perform one or more &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolution&lt;/a&gt; (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-correlation&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-correlation&lt;/a&gt;) operations (often followed by a &lt;a href=&quot;https://en.wikipedia.org/wiki/Downsampling_(signal_processing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;down-sampling&lt;/a&gt; operation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convolution is an operation that takes two functions, &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt;, as input and produces a third function, &lt;span class=&quot;math-container&quot;&gt;$\bf g = f \circledast h$&lt;/span&gt;, where the symbol &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt; denotes the convolution operation. In the context of CNNs, the input function &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt; can e.g. be an image (which can be thought of as a function from 2D coordinates to RGB or grayscale values). The other function &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; is called the &quot;kernel&quot; (or filter), which can be thought of as (small and square) matrix (which contains the output of the function &lt;span class=&quot;math-container&quot;&gt;$\bf &#xA; h$&lt;/span&gt;). &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt; can also be thought of as a (big) matrix (which contains, for each cell, e.g. its grayscale value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of CNNs, the &lt;em&gt;convolution&lt;/em&gt; operation can be thought of as dot product between the kernel &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; (a matrix) and several parts of the input (a matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the picture below, we perform an &lt;a href=&quot;https://en.wikipedia.org/wiki/Hadamard_product_(matrices)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;element-wise multiplication&lt;/em&gt;&lt;/a&gt; between the kernel &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; and part of the input &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt;, then we sum the elements of the resulting matrix, and that is the value of the convolution operation for that specific part of the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EiujB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EiujB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more concrete, in the picture above, we are performing the following operation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\sum_{ij}&#xA;\left(&#xA;\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0\\&#xA;1 &amp;amp; 1 &amp;amp; 0\\&#xA;1 &amp;amp; 1 &amp;amp; 1&#xA;\end{bmatrix}&#xA;\otimes&#xA;\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 1\\&#xA;0 &amp;amp; 1 &amp;amp; 0\\&#xA;1 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}&#xA;\right)&#xA;=&#xA;\sum_{ij}&#xA;\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; 0\\&#xA;1 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}&#xA;= 4&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\otimes$&lt;/span&gt; is the element-wise multiplication and the summation &lt;span class=&quot;math-container&quot;&gt;$\sum_{ij}$&lt;/span&gt; is over all rows &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and columns &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; (of the matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute all elements of &lt;span class=&quot;math-container&quot;&gt;$\bf g$&lt;/span&gt;, we can think of the kernel &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; as being slided over the matrix &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the kernel function &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; can be fixed. However, in the context of CNNs, the kernel &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; represents the learnable parameters of the CNN: in other words, during the training procedure (using e.g. gradient descent and back-propagation), this kernel &lt;span class=&quot;math-container&quot;&gt;$\bf h$&lt;/span&gt; (which thus can be thought of as a matrix of weights) changes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of CNNs, there is often more than one kernel: in other words, it is often the case that a sequence of kernels &lt;span class=&quot;math-container&quot;&gt;$\bf h_1, h_2, \dots, h_k$&lt;/span&gt; is applied to &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt; to produce a sequence of convolutions &lt;span class=&quot;math-container&quot;&gt;$\bf g_1, g_2, \dots, g_k$&lt;/span&gt;. Each kernel &lt;span class=&quot;math-container&quot;&gt;$\bf h_i$&lt;/span&gt; is used to &quot;detect different features of the input&quot;, so these kernels are different from each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;down-sampling&lt;/em&gt; operation is an operation that reduces the input size while attempting to maintain as much information as possible. For example, if the input size is a &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$\bf f = \begin{bmatrix} 1 &amp;amp; 2 \\ 3 &amp;amp; 0 \end{bmatrix}$&lt;/span&gt;, a common down-sampling operation is called the &lt;em&gt;max-pooling&lt;/em&gt;, which, in the case of &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt;, returns &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; (the maximum element of &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CNNs are particularly suited to deal with high-dimensional inputs (e.g. images), because, compared to FFNNs, they use a smaller number of learnable parameters (which, in the context of CNNs, are the kernels). So, they are often used to e.g. classify images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the fundamental difference between RNNs and CNNs?&lt;/strong&gt; RNNs have recurrent connections while CNNs do not necessarily have them. The fundamental operation of a CNN is the convolution operation, which is not present in a standard RNN.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-04T17:20:10.343" LastActivityDate="2019-06-04T17:20:10.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12291" PostTypeId="2" ParentId="12289" CreationDate="2019-05-13T22:31:59.270" Score="1" Body="&lt;p&gt;This is possible.... but there's no reason to use a neural network! Your best bet on a problem like this is likely to use a logistical regression for the yes/no aspect of the question and a linear regression (or combination of linear regressions) to answer the pricing question - there are also ways of simply using linear regressions and setting up cutoffs to answer the yes/no question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reality is that the accuracy of such a model/series of models would depend entirely on the quality and quantity of the data, but it's unlikely that in this case a neural network would provide a better result than smart usage of simpler models.&lt;/p&gt;&#xA;" OwnerUserId="25382" LastActivityDate="2019-05-13T22:31:59.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12292" PostTypeId="2" ParentId="4328" CreationDate="2019-05-14T00:57:24.273" Score="3" Body="&lt;p&gt;Terms in a field are sometimes defined unambiguously. For instance, we know what &lt;em&gt;convergence&lt;/em&gt; means when communicating about machine learning algorithms in academic publications because it has a formal definition in an older field, mathematics. However, the term &lt;em&gt;machine learning&lt;/em&gt; is defined ambiguously across academic publications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Perspectives on Machine Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some see it as a branch of applied probability and statistics involving models with curvature (not usefully approximated by a first degree polynomial) and the application of those principles in digital computing. Some see it as an extension of the work of James Watt and Le Roy MacColl the application of the feedback control concept to digital control. Some see it as the natural result of the pioneering AI work of Norbert Wiener and John Von Neumann, where the adaptive qualities of nature, including neurochemistry, are simulated with the intention of birthing artificial life.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some don't see that deeply into ML and imagine that its a set of classes and libraries the mastery over which will make for a great career. As shallow as that may seem, that concept may be as true as the other three deeper conceptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Perspectives on Data Mining&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;data mining&lt;/em&gt; is like that. Each book, and sometimes each chapter within the same book, seems to have its own distinct conception of the verb &lt;em&gt;mining&lt;/em&gt;. Although these definitions have some similarity, the term is nothing like the term &lt;em&gt;convergence&lt;/em&gt; or even &lt;em&gt;database&lt;/em&gt; in IT or &lt;em&gt;melody&lt;/em&gt; in music.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfolding the metaphor contained in the two words of the term, data mining is digging up data, and perhaps that's a satisfactory definition for the most general use of the term. The information sought is not on the surface, like diamonds dropped onto the ground, but rather underneath and covered with other materials so that one has to survey, dig, and process to get past the worthless material and reveal the gems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This term has another vantage point. In systems theory there is an important distinction between noise and signal. In data science, what an electrical engineer would call the signal is the listing of statistics, table, graphic, or other visualization miner's client needs to make management decisions. The noise is everything obscuring the signal through complexity, volume, or prominence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Perspectives on Pattern Recognition&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;pattern recognition&lt;/em&gt; is perhaps the most ambiguous because neither of the two words arose in a scientific context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early uses of the word &lt;em&gt;pattern&lt;/em&gt; in English (and its equivalent in other languages) are related to shelter construction, farming, or early textiles. The notion that the shape of a letter or other symbol or a sequence of phonetic elements that make up a spoken word were patterns only arose recently. Much of the early and current work in pattern recognition involving computers had to do with converting natural language expressions into some functional machine representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term pattern is also ambiguous because of gestalt, the dependence of perception on the orientation of the recognizer at the time of recognition. A sand castle may have an architecture to an architect, a chemical composition to a chemist, an indication of civilization to the starving passengers of a boat adrift, an obstacle in the way to a crayfish, and an imaginary home for a child.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To a mathematician, it may be a three dimensional form with particular surface topology, feature curvature, and dimensions. To a physicist, there may be no significant difference between the sand castle and the seagull flying over it or the air in between them (unless the sand castle is the triumph of the physicists own child).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The orientation of the machine is even more a constraint on the emulation of some aspect of human perception than demonstrated in gestalt psychology experiments. The human can adjust perception when a new kind of pattern or structure is pointed out. Until AI progresses further, that kind of experience, where the computer would say, &quot;Oh, yes. Now I see the old woman in the picture of the young woman,&quot; is only realizable in software to the most primitive degree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taken literally, the term &lt;em&gt;recognition&lt;/em&gt; means the repetition of a cognitive event, but that is not what we mean when we say, &quot;I recognize that,&quot; in common speech. We usually mean that a mental search for some set of sensory features (not necessarily any more a pattern than anything else in the sensory stream) is identified and associated with some internal object or concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common use of a convolutional network (CNN) is neither of these. It is usually used to categorize objects or as a feature extracting sensory front end to a much larger AI design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overlap and Associations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all these ambiguities present, some overlap may be apparent, in that some AI activities may thoroughly involve two or all three of these terms. Certainly some associations between the three terms are obvious.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When mining data, we may be looking for a particular kind of structure in a sea of data and have a particular search strategy to narrow the search and make it manageable for computing resources available. The test use during the search may be called pattern recognition.&lt;/li&gt;&#xA;&lt;li&gt;In machine learning, we may train a network of artificial cells to assist in locating data or features in data that are meaningful to the stakeholders in the project. That would be using ML for data mining projects.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A large number of other associations between the three terms can be made. Which ones would appear most prominent to the expert would depend on the scientific, research, and career orientation of the expert.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Not Sufficient Overlap to be Synonyms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be difficult however to declare any two of the three to be synonymous. The three arose out of different kinds of research and from different orientations. Only some of that etymology is preserved in the terms themselves.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-05-20T07:21:51.233" LastActivityDate="2019-05-20T07:21:51.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12294" PostTypeId="1" AcceptedAnswerId="12308" CreationDate="2019-05-14T02:56:59.147" Score="1" ViewCount="510" Body="&lt;p&gt;I'm looking for some general advice here before I dive in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in creating a new environment for OpenAI gym to provide some slightly more challenging continuous control problems than the ones currently in their set of &lt;a href=&quot;https://gym.openai.com/envs/#classic_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classic Control&lt;/a&gt; environments.  The intended users of this new environment are just me and members of a meetup group I am in but obviously I want everyone to be able to access, install and potentially modify the same environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the easiest way to do this?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I simply import and sub-class the OpenAI &lt;code&gt;gym.Env&lt;/code&gt; module similar&#xA;to the way &lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;cartpole.py&lt;/a&gt; does.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or do I need to create a whole new&#xA;PIP package as &lt;a href=&quot;https://github.com/openai/gym/blob/master/docs/creating-environments.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; suggests?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, before I invest a lot of time on this, has anyone already created a cart-pole system environment where the goal is to stabilize the full state (not just the vertical position of the pendulum)?  (I tried googling and couldn't find any variants on the original &lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;cart-pole-v1&lt;/a&gt; but I'm suspicious as I can't be the first person to make modified versions of some of these classic control environments).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, (I realize this question is a bit open-ended) but hoping for some good advice that will point me in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="25618" LastEditorUserId="25618" LastEditDate="2019-05-14T03:02:50.083" LastActivityDate="2019-06-13T17:02:35.977" Title="Advice on creating a new environment using OpenAI Gym" Tags="&lt;python&gt;&lt;open-ai&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12295" PostTypeId="1" CreationDate="2019-05-14T06:16:39.420" Score="2" ViewCount="40" Body="&lt;p&gt;I am trying to estimate the real world distance (in metres) between two points in a perspective image using an uncalibrated camera. However, the dimensions of an object in the image are known. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of using pixel per metre, but, being a perspective image, that did not seem like a viable approach. I believe I need the camera matrix from the image (maybe using the known object) and compute the 3D coordinates of the point, then simply compute the Euclidean distance between the points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is the right approach, how may I do it? If there are any alternatives to this approach, kindly let me know.&lt;/p&gt;&#xA;" OwnerUserId="25623" LastEditorUserId="2444" LastEditDate="2019-05-14T18:23:35.450" LastActivityDate="2019-05-14T18:23:35.450" Title="Estimate distance between points in perspective image" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12296" PostTypeId="2" ParentId="12264" CreationDate="2019-05-14T07:09:48.103" Score="2" Body="&lt;p&gt;Yes, you are right. It is somehow an arbitrary choice, although you should consider the reasonable numerical ranges of your activation functions if you decide to go beyond the values +/- 1.&lt;/p&gt;&#xA;&lt;p&gt;You can also have a think about whether you want to add a small reward for the agent reaching states that are &lt;em&gt;near&lt;/em&gt; the goal, if you have an environment where such states are discernable.&lt;/p&gt;&#xA;&lt;p&gt;If you want to have a more machine learning approach to reward values, consider using an Actor-Critic arrangement, in which a second network learns reward values for non-goal states (by observing the results of agent exploration), although you still need to determine end-state values according to your handcrafted heuristic.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="2444" LastEditDate="2020-10-07T17:12:00.043" LastActivityDate="2020-10-07T17:12:00.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12297" PostTypeId="2" ParentId="12283" CreationDate="2019-05-14T07:35:09.030" Score="1" Body="&lt;p&gt;Before we start to tweak with you Agent-Environment setup, there are couple of important things to note&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Q-Learning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-learning as a fundamental is a &lt;strong&gt;greedy&lt;/strong&gt; approach based on &lt;strong&gt;Action Value&lt;/strong&gt; functions. When I say action value what I mean to say is this&#xA;&lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a) = p(\hat{s},r | s, a)[r + \gamma(V(\hat{s}))]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means according to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; policy the &lt;strong&gt;Action value&lt;/strong&gt; for taking action &quot;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;&quot; at state &quot;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&quot; is the probability of taking reaching state &quot;&lt;span class=&quot;math-container&quot;&gt;$\hat{s}$&lt;/span&gt;&quot; and getting a reward &quot;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&quot; given the current state is &quot;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&quot; and action taken is &quot;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;&quot; multiplied by sum of immediate reward &quot;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&quot; and &lt;strong&gt;expected&lt;/strong&gt; future reward given by future state value(s) &quot;&lt;span class=&quot;math-container&quot;&gt;$V(\hat{s})$&lt;/span&gt;&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note the &lt;span class=&quot;math-container&quot;&gt;$p(\hat{s},r | s, a)$&lt;/span&gt; is only available in &lt;strong&gt;&lt;em&gt;deterministic modelling technique (model based methods)&lt;/em&gt;&lt;/strong&gt; and this part is not available for &lt;strong&gt;&lt;em&gt;stochastic modelling techniques (model-free methods)&lt;/em&gt;&lt;/strong&gt;. For model-free methods we execute multiple &lt;strong&gt;experiments&lt;/strong&gt; and take the &lt;em&gt;expected&lt;/em&gt; value in accordance with &lt;strong&gt;Law of Large Numbers&lt;/strong&gt;. Which is what we do in common &lt;em&gt;Deep Reinforced Learning&lt;/em&gt; implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above formulation for future rewards can be expanded in series using the state-value formulation as &lt;span class=&quot;math-container&quot;&gt;$V(s) = \pi(a,s)q_\pi(s,a)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the policy we are following aka probability of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; when in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is it termed &lt;em&gt;Greedy&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because while using the fundamental of multiple experiments, with every action I take the immediate reward, and the &lt;strong&gt;max&lt;/strong&gt; q_value (&lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;) associated with the &lt;strong&gt;next_state&lt;/strong&gt; to update my current q_value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In Q-learning the future state reward is what you calculate by submitting the &lt;strong&gt;&lt;em&gt;next_state&lt;/em&gt;&lt;/strong&gt; to the network and taking &lt;strong&gt;&lt;em&gt;max&lt;/em&gt;&lt;/strong&gt; of the &lt;strong&gt;&lt;em&gt;q_values&lt;/em&gt;&lt;/strong&gt; attained. (Hence, Greedy!!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Crux of the above&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said all the above (Duh!), what is in it for us?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Q-Learning, your reward is key for your model learn, i.e. &quot;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&quot; &lt;strong&gt;immediate reward&lt;/strong&gt; and the &lt;strong&gt;future reward&lt;/strong&gt; (&lt;span class=&quot;math-container&quot;&gt;$V(\hat{s})$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;When the state-space is limited&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tic-tac-toe_variants#Numerical_Tic-tac-toe&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical tic-tac-toe game&lt;/a&gt;, in such cases it is ideal to have discrete reward system, such as the one suggested by you. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When the state-space is very large (the problem statement you are tackling)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such cases, it is ideal to have a &lt;strong&gt;continuous&lt;/strong&gt; reward mechanism, i.e. rewards should be more variable in nature, like continuous values.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suggestions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can define the reward function in the environment has the &lt;strong&gt;10&lt;span class=&quot;math-container&quot;&gt;$\sqrt{2}$&lt;/span&gt; - Euclidean distance&lt;/strong&gt; between the face of your &lt;em&gt;snake&lt;/em&gt; and the &lt;em&gt;fruit&lt;/em&gt; it is trying to reach. The reason I am doing it (10&lt;span class=&quot;math-container&quot;&gt;$\sqrt{2}$&lt;/span&gt; minus) is because the maximum distance could be 10&lt;span class=&quot;math-container&quot;&gt;$\sqrt{2}$&lt;/span&gt; (considering it is 10X10 board). This would help you to have a more variable/continuous rewards for your model to learn with (since you are using neural networks for training them). As your &lt;em&gt;Snake&lt;/em&gt; goes closer to the &lt;em&gt;fruit&lt;/em&gt; this value will keep on increasing. One cavet, if by taking an action, your snake reaches fruit, you can provide it a specific constant (extra) reward, and if it hits its own self, or the borders you penalize it with a specific constant. So that the model understand better.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can define your state (input vector to your NN) as 100 dimension array (10X10 grid) with each element signifying if there a object in the that area or not. &lt;strong&gt;1&lt;/strong&gt; means it is fruit, &lt;strong&gt;0&lt;/strong&gt; means it is &lt;em&gt;snakes&lt;/em&gt; own body and &lt;strong&gt;0.5&lt;/strong&gt; means it is free for movement. The idea is to limit the input values of the vectors between 0~1.  As your agent takes action, the values in the state will keep on changing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The output of your network should have 3 neurons, each signifying the &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;&lt;/strong&gt; of each of action (front/right/left movement). For &lt;em&gt;get_action&lt;/em&gt; function you will get the action pertaining to &lt;em&gt;max(q_value)&lt;/em&gt;, and keep on adding things in your &lt;strong&gt;memory d_que&lt;/strong&gt;. Subsequently you can trigger &lt;strong&gt;model_train&lt;/strong&gt; methods at every episode/or even at after every step.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lastly, the hyper parameters would be the discount factor &lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;&lt;/em&gt; you choose and the learning rate &lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;&lt;/em&gt; you use for the NN network. That's something you will have to experiment. Along the depth of your ANN. I think a single hidden layer should be enough.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the last layer of your NN, please use a &lt;em&gt;linear&lt;/em&gt; activation function, and not a &lt;em&gt;relu&lt;/em&gt; activation function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Please run many-many-many experiments, something in millions for optimum training. There could be multiple ways of monitoring your model's learning process. How many games won, how much &lt;em&gt;fruit&lt;/em&gt; eating it is able to achieve in 1 game etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope the above helps.&lt;/p&gt;&#xA;" OwnerUserId="25617" LastEditorUserId="25617" LastEditDate="2019-05-14T07:47:22.513" LastActivityDate="2019-05-14T07:47:22.513" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12299" PostTypeId="2" ParentId="12264" CreationDate="2019-05-14T08:23:20.867" Score="3" Body="&lt;p&gt;In Reinforcement Learning (RL), a reward function is &lt;em&gt;part of the problem definition&lt;/em&gt; and should:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Be based primarily on the goals of the agent.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Take into account any combination of starting state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, action taken &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, resulting state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and/or a random amount (a constant amount is just a random amount with a fixed value having probability 1). You should not use other data than those four things, but you also do not have to use any of them. This is important, as using any other data stops your environment from being a Markov Decision Process (MDP).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Given the first point, as direct and simple as possible. In many situations this is all you need. A reward of +1 for winning a game, 0 for a draw and -1 for losing is enough to fully define the goals of most 2-player games.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In general, have positive rewards for things you want the agent to achieve or repeat, and negative rewards for things you want the agent to avoid or minimise doing. It is common for instance to have a fixed reward of -1 per time step when the objective is to complete a task as fast as possible.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In general, reward 0 for anything which is not &lt;em&gt;directly&lt;/em&gt; related to the goals. This allows the agent to learn for itself whether a trajectory that uses particular states/actions or time resources is worthwhile or not.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Be scaled for convenience. Scaling all rewards by a common factor does not matter at a theoretical level, as the exact same behaviour will be optimal. In practice you want the sums of reward to be easy to assess by yourself as you analyse results of learning, and you also want whatever technical solution you implement to be able to cope with the range of values. Simple numbers such as +1/-1 achieve that for basic rewards.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Ideally, you should avoid using heuristic functions that reward an agent for interim goals or results, as that inserts your opinion about how the problem should be solved into the system, and may not in fact be optimal given the goals. In fact you can view the purpose of value-based RL is learning a good heuristic function (the value function) from the more sparse reward function. If you already had a good heuristic function then you may not need RL at all.&lt;/p&gt;&#xA;&lt;p&gt;You may need to compare very different parts of the outcome in a single reward function. This can be hard to balance correctly, as the reward function is a single scalar value and &lt;em&gt;you&lt;/em&gt; have to define what it means to balance between different objectives within a single scenario. If you do have very different metrics that you want to combine then you need to think harder about what that means:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Where possible, flatten the reward signal into the same units and base your goals around them. For instance, in business and production processes if may be possible to use currency as the units of reward and convert things such as energy used, transport distance, etc., into that currency.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For highly negative/unwanted outcomes, instead of assigning a negative reward, consider whether a constraint on the environment is more appropriate.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You may have opinions about valid solutions to the environment that you want the agent to use. In which case you can extend or modify the system of rewards to reflect that - e.g. provide a reward for achieving some interim sub-goal, even if it is not directly a result that you care about. This is called &lt;em&gt;reward shaping&lt;/em&gt;, and can help in practical ways in difficult problems, but you have to take extra care not to break things.&lt;/p&gt;&#xA;&lt;p&gt;There are also more sophisticated approaches that use multiple value schemes or no externally applied ones, such as hierarchical reinforcement learning or intrinsic rewards. These may be necessary to address more complex &amp;quot;real life&amp;quot; environments, but are still subject of active research. So bear in mind that all the above advice describes the current mainstream of RL, and there are more options the deeper you research the topic.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is this purely experimental and down to the programmer of the environment. So, is it a heuristic approach of simply trying different reward values and see how the learning process shapes up?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Generally no. You should base the reward function on the analysis of the problem and your learning goals. And this should be done at the start, before experimenting with hyper-parameters which define the learning process.&lt;/p&gt;&#xA;&lt;p&gt;If you are trying different values, especially different &lt;em&gt;relative&lt;/em&gt; values between different aspects of a problem, then you may be changing what it means for the agent to behave optimally. That might be what you want to do, because you are looking at how you want to frame the original problem to achieve a specific behaviour.&lt;/p&gt;&#xA;&lt;p&gt;However, outside of &lt;a href=&quot;https://towardsdatascience.com/inverse-reinforcement-learning-6453b7cdc90d&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;inverse reinforcement learning&lt;/em&gt;&lt;/a&gt;, it is more usual to want an optimal solution to a well-defined problem, as opposed to a solution that matches some other observation that you are willing to change the problem definition to meet.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So, am I right in saying it's just about trying different reward values for actions encoded in the environment and see how it affects the learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is usually not the case.&lt;/p&gt;&#xA;&lt;p&gt;Instead, think about how you want to &lt;em&gt;define&lt;/em&gt; the goals of the agent. Write reward functions that encapsulate those goals. Then focus on changes to the agent that allow it to better learn how to achieve those goals.&lt;/p&gt;&#xA;&lt;p&gt;Now, you can do it the way round, as you suggest. But what you are doing, in that case, is changing the problem definition, and seeing how well a certain kind of agent can cope with solving each kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-10-07T17:09:51.130" LastActivityDate="2020-10-07T17:09:51.130" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12303" PostTypeId="1" CreationDate="2019-05-14T14:07:44.433" Score="2" ViewCount="12" Body="&lt;p&gt;I'm building CNN network of Image to Image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training, I have some bad results in part of the Image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to &lt;strong&gt;find the neurons&lt;/strong&gt; that &lt;strong&gt;most influenced those pixels&lt;/strong&gt; and do retraining only for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen some previous works about visualizing networks, like here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/utkuozbulak/pytorch-cnn-visualizations&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/utkuozbulak/pytorch-cnn-visualizations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But they are only finding activations maps or visualizing with softmax in the output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do that for Image to Image?&lt;/p&gt;&#xA;" OwnerUserId="25144" LastActivityDate="2019-05-14T14:07:44.433" Title="How to debug and find neurons that most influenced a pixel in the output image?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12306" PostTypeId="2" ParentId="12154" CreationDate="2019-05-14T15:59:22.350" Score="2" Body="&lt;p&gt;&lt;strong&gt;The Title Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there any paper, article or book that analyzes the feasibility of achieving AGI through brain-simulation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. There are various analyses that have been published. We have some early work like &lt;a href=&quot;http://www-formal.stanford.edu/jmc/mcchay69.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Some Philosophical Problems From the Standpoint of Artificial Intelligence&lt;/em&gt;&lt;/a&gt;, John McCarthy and Patrick J. Hayes, Stanford University, 1969. And there is more recent and optimistic work like &lt;a href=&quot;https://www.researchgate.net/profile/Prof_Dr_Hugo_De_GARIS/publication/226000160_Artificial_Brains/links/55d1e55308ae2496ee658634.pdf#page=143&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Essentials of General Intelligence: The Direct Path to Artificial General Intelligence&lt;/em&gt;&lt;/a&gt;, Peter Voss, 2007. There are refutations of the concept of General Intelligence, artificial or not, in some posts here (listed below) and quite a bit of academic scrutiny of the assumptions behind the AGI concept and the offspring of it: The Singularity. Concern about The Singularity is a social phenomenon, not a scientific one. It began with the depiction of the emergence of a super-intelligence as an inevitability in science fiction and in sensational media.&lt;/p&gt;&#xA;&lt;p&gt;But mathematically terse proofs of AGI feasibility are not found in the media or in the Terminator franchise started by the wonderful imagination of James Cameron. This is not to say that world domination cannot fall into the hands of a new species created by humans. In fact, Jaques Ellul, in his &lt;em&gt;Techological Society&lt;/em&gt; points out that in many ways humans are already subservient to technology — that the creation of humankind is already autonomous and dominant. His 449 page heap of evidence is witty and quite convincing.&lt;/p&gt;&#xA;&lt;p&gt;Academic publications either&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;(a) Assume feasibility and discuss approaches to its design or&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;(b) Illuminate caveats in the idea of a universally intelligent system&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Where is the Mind?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Whether the entirety of the mind arises from a physical system is a question that dates back to René Descartes and Gottfried Leibniz. Current conceptions of intelligence tend to deny the interaction and interdependence of the human brain, the rest of the human, and the biosphere into which the human fits without extreme expenditure to produce a floating, underwater, or underground simulation of the biosphere. The mind may center in the brain, but any brain disconnected from the biosphere and the circulatory system for even a short time is completely useless. A think tank is not a bunch of smart people in a sensory deprivation tank. Quite the opposite. Early experimentation with think tanks revealed the importance of mental health of the participants through diversion and physical exercise so that the various forms of neurological stasis could be maintained.&lt;/p&gt;&#xA;&lt;p&gt;It is quite questionable whether computers without bodies will ever learn as much of general use (that can be put to use in the biosphere) without moving about and interaction with the biosphere. It is highly likely that the mind is more like Carl Jung sees it, as in the brain and elsewhere. Ludwik Fleck, in a somewhat Jungian fashion, speaks of science as the product of a collective in his &lt;em&gt;Genesis and Development of a Scientific Fact&lt;/em&gt;, a good read that throws considerable light on the folly of most people's naive conception of facts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Quantum Computing Buzz&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;When Intel's founder, Robert Noyce, brought in Gordon Moore in 1975, it was because of various statistical observations he had made, including what is now known as Moore's Law, a curve that on semi-log paper in 1965 looked linear. Since then, the business assumption in Silicon Valley has been that compactness and speed would double frequently and consistently. In the last few years, speed has stopped doubling and compactness seems to have reached its limit around 7 nm. The transistors in this VLSI technology have only a handful of atoms to represent the semiconductor topologies that permit gating, the fundamental activity of digital circuitry.&lt;/p&gt;&#xA;&lt;p&gt;In essence, to continue the Silicon Valley boon, it has to become Particle Valley, using carbon allotropes or other nano-tech to build computers where Moore's Law is not overturned by the limitations of the structure of the universe. Whether this can be done requires its own feasibility study. Already, SSD (solid state drive) technology relies heavily on probability. Errors are occurring in great proportions and error correcting techniques are used to keep them manageable. That's because, as we reach for designs closer to the quantum level, Brownian motion or the light electromagnetic interference can upset the digital operation intended.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Challenges Versus Show Stoppers&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As with many trends in the history of human endeavors, determining what is a challenge and what is a show stopper is not clear until viewing in hindsight. That is why people are asking the various questions in this Stack Exchange listed in the question.&lt;/p&gt;&#xA;&lt;p&gt;Stating that mentioning our partial knowledge of brain operation is off topic for this question might have been appropriate if we were trying to invent something that we had not already seen in operation in nature. However, that is not the case. We call it artificial intelligence because we think that we are intelligent and want to extend that facility we find in ourselves. The following fields, in addition to quantum physics and machine learning, are thus necessarily relevant to the feasibility of a more generally usable fabricated intelligent system.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neuro-chemistry in general&lt;/li&gt;&#xA;&lt;li&gt;Genetics as it applies to neurological network structuring and cell activity&lt;/li&gt;&#xA;&lt;li&gt;Correlation between DNA coding and human mental performance and stability&lt;/li&gt;&#xA;&lt;li&gt;Applied psychology&lt;/li&gt;&#xA;&lt;li&gt;Addiction science&lt;/li&gt;&#xA;&lt;li&gt;Cognitive science&lt;/li&gt;&#xA;&lt;li&gt;Mathematics in general and probability and statistics specifically&lt;/li&gt;&#xA;&lt;li&gt;Control theory&lt;/li&gt;&#xA;&lt;li&gt;Philosophy, especially in regard to whether intelligence actually exists and the ideas of culturally determined knowledge and belief&lt;/li&gt;&#xA;&lt;li&gt;Language and linguistics&lt;/li&gt;&#xA;&lt;li&gt;Symbology&lt;/li&gt;&#xA;&lt;li&gt;Conceptions of a thought collective&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Not only do we not know enough in these areas, but we know very little. There is also considerable evidence that humanity tends to forget some of the things it once knew. Take away our petroleum and we may find out just how much we forgot.&lt;/p&gt;&#xA;&lt;p&gt;Whether any of the things we have lost from our general knowledge are prerequisites to solving the challenges of universally intelligent system design is unknown. A simple example is that many computing science graduates do not know anything about Kurt Gödel's incompletness theorems and how they led to Alan Turing's completeness theorem. Many use back-propagation in their Python code but don't have any conception that it is a corrective feedback distribution scheme.&lt;/p&gt;&#xA;&lt;p&gt;On the flip side, we have no strong argument to draw the conclusion that there is some theoretical limitation as solid as the second law of thermodynamics, one of the apparently insurmountable hard stops built into our physical reality on and off planet. We have no proof that intelligence does not exist and is rather an anthrocentric fantasy. We have no proof that whatever limitations humans have cannot be overcome by copying the intellectual life of human beings into human creations or using intellect to create greater intellect in machines.&lt;/p&gt;&#xA;&lt;p&gt;What we know, as smart as we think we are, is greatly outweighed by what we don't know. Anyone in real research work will agree.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Refutations of AGI Philosophy&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some of the many refutations of AGI that appear throughout academia. Some articles include these.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://intelligence.org/2013/08/11/what-is-agi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;What is AGI?&lt;/em&gt;&lt;/a&gt;, Luke Muehlhauser, 2013&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.theguardian.com/science/2012/oct/03/philosophy-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Philosophy will be the key that unlocks artificial intelligence&lt;/em&gt;&lt;/a&gt;, David Deutsch, 2012, The Guardian&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.econtalk.org/rodney-brooks-on-artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;On Artificial Intelligence&lt;/em&gt;&lt;/a&gt;, Rodney Brooks, 2018, The Library of Economics and Liberty&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These are some posts here that question various assumptions prevalent in popular AI media.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/1774/human-like-general-intelligence-vs-task-specific-algorithms-when-people-realiz/3682#3682&quot;&gt;Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2021-01-26T01:40:02.607" LastActivityDate="2021-01-26T01:40:02.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12307" PostTypeId="1" CreationDate="2019-05-14T16:28:21.080" Score="2" ViewCount="660" Body="&lt;p&gt;I understand the way we build and train a model, but all of the online courses I've found end with this. I can't find any course explaining the process of utilizing the trained model to address the problem.&lt;/p&gt;&#xA;&lt;p&gt;Is there any course out there that explains the whole process from data collection, model building, and utilizing the model to solve the real-world problems?&lt;/p&gt;&#xA;" OwnerUserId="25628" LastEditorUserId="2444" LastEditDate="2020-07-05T18:02:36.310" LastActivityDate="2020-07-05T18:02:36.310" Title="After a model has been trained, how do I use it to address the real-world problems?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;models&gt;&lt;ai-development&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12308" PostTypeId="2" ParentId="12294" CreationDate="2019-05-14T16:37:21.317" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/53241029/1609514&quot;&gt;This stackoverflow post&lt;/a&gt; provides a good answer.  It recommends creating the environment as a new package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the &lt;a href=&quot;https://github.com/openai/gym/blob/master/docs/creating-environments.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;updated link&lt;/a&gt; to the instructions on OpenAI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that's what I did and it's honestly not that much work and then you can import the environment into your script like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import gym&#xA;import gym_foo&#xA;env = gym.make('foo-v0')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@apoddar573/making-your-own-custom-environment-in-gym-c3b65ff8cdaa&quot; rel=&quot;nofollow noreferrer&quot;&gt;This blog post&lt;/a&gt; also describes the process in a little bit more detail.&lt;/p&gt;&#xA;" OwnerUserId="25618" LastActivityDate="2019-05-14T16:37:21.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12309" PostTypeId="2" ParentId="12307" CreationDate="2019-05-14T17:45:58.247" Score="6" Body="&lt;p&gt;Using a machine learning or AI-powered model once it has been built and tested, is not &lt;em&gt;directly&lt;/em&gt; an AI issue, it is just a development issue. As such, you won't find many machine learning tutorials that focus on this part of the work. But they do exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence it is the same as integrating any other function, which might be in a third-party library:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Package the new function so that it can be called from your production system (this may be the hardest part)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Decide where in the code of your production system to call the new function. In your example case maybe after form is completed describing an incident you could link the top recommended KB articles from the new ticket.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Design and check the associated user experience/UI as appropriate for your development team (in small projects you may skip this step and just implement)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Change your production code to call the packaged function. In any professional development team, this part will have multiple stages, but not really relevant to the question - if you are the ML specialist and delivering your new model to an existing team, you need to talk to them about both the packaging part and the steps involved here.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Often, machine learning models come with a whole bunch of dependencies that the rest of a system does not have. There are a variety of solutions for that, depending on the libraries that you are using, whether you are using a cloud PaaS service etc. You could just build a Docker image to hold all the AI parts and call it passing the input data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deploying ML models to production as a job is often a task for Machine Learning Engineer roles. There are courses and articles covering the practical aspects of these steps, if you search for them. Here are a couple:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.coursera.org/learn/deploying-machine-learning-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deploying Machine Learning Models&lt;/a&gt; from Coursera&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A blog article on Medium: &lt;a href=&quot;https://towardsdatascience.com/there-are-two-very-different-ways-to-deploy-ml-models-heres-both-ce2e97c7b9b1&quot; rel=&quot;nofollow noreferrer&quot;&gt;There are two very different ways to deploy ML models . . .&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have no affiliation with any of the above, have not read articles or taken the courses, and am not able to make any recommendation, even if you told me the technologies you were using for ML and in production currently.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;After deploying to production . . .&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Your work is not necessarily done. You will want to monitor behaviour of the system. During integration, you should of added logging, or some way to get feedback of performance in the wild.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if this is an ML system, does the accuracy seen in testing hold up against real life? Is the target population drifting? If the system makes interventions - by e.g. suggesting links, or showing categories to end users - how well are those working in practice? Is the performance and responsiveness of the system fast enough when the service is under load?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is more reactive AI system, that takes actions itself, you will similarly want to monitor what it is supposed to optimise, or sample its output for errors and quality control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this feedback can go back into a new iteration of design and integration. How to incorporate that will depend a lot on the nature of the system and what you discover, so could be the subject of further questions.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-15T07:06:18.737" LastActivityDate="2019-05-15T07:06:18.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12310" PostTypeId="1" AcceptedAnswerId="12323" CreationDate="2019-05-14T20:50:07.443" Score="2" ViewCount="30" Body="&lt;p&gt;I am looking at LSTM example &lt;a href=&quot;https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/?unapproved=485429&amp;amp;moderation-hash=4df84fbc7e6d58bc190deee3b48ef6dd#comment-485429&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am not sure how to modify the setup if I have forecast available (assuming perfect forecast) for TEMP: Temperature and PRES: Pressure at time t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. &lt;code&gt;pollution_t = fn(TEMP_t, PRES_t, TEMP_t_1, PRES_t_1, ..., othervariables and lagged values)&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;_t represents the perfect forecast available&lt;/p&gt;&#xA;&#xA;&lt;p&gt;_t_1 represents the variable values at previous time steps&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I am looking for something of the following setup:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bTMDy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bTMDy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25634" LastEditorUserId="25634" LastEditDate="2019-05-14T21:19:59.223" LastActivityDate="2019-05-15T15:40:35.550" Title="Modifying LSTM to include forecast" Tags="&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12311" PostTypeId="1" CreationDate="2019-05-14T22:06:54.437" Score="2" ViewCount="722" Body="&lt;p&gt;I've got a Lego Mindstorms EV3 with &lt;a href=&quot;https://www.ev3dev.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EV3DEV&lt;/a&gt; and EV3-Python installed. I wanted to try out artificial intelligence with the robot, and my first project is going to be to get the robot to try and recognize some images (using convolutions) and do an action related to the image it has seen. However, I can't find a way to use Tensorflow (or any AI module for that matter) on an EV3. Does anyone know how to incorporate Tensorflow or any other modules into the EV3? Help would be gladly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="25639" LastEditorUserId="2444" LastEditDate="2019-05-14T22:09:58.027" LastActivityDate="2020-07-23T14:22:46.667" Title="How do I integrate Tensorflow with EV3DEV?" Tags="&lt;python&gt;&lt;tensorflow&gt;&lt;robotics&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12312" PostTypeId="2" ParentId="12311" CreationDate="2019-05-15T00:25:32.437" Score="1" Body="&lt;p&gt;There is a YouTube video &lt;a href=&quot;https://www.youtube.com/watch?v=wSSh9iV70ng&quot; rel=&quot;nofollow noreferrer&quot;&gt;LEGO EV3 Raspberry Pi Tensorflow Sorting Machine&lt;/a&gt; by &lt;a href=&quot;https://www.youtube.com/channel/UCOziWS9PkSVwB0XoIE14rDw&quot; rel=&quot;nofollow noreferrer&quot;&gt;ebswift&lt;/a&gt; that should help you although you will need a Raspberry Pi.  From the abstract:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This is a sorting machine based on the EV3 45544 education kit sorting machine.  The colour sorting camera is substituted with a Raspberry Pi with the v2 camera.  The EV3 is controlled over wifi via RPyC and the object recognition work is done via Tensorflow.&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A viewer asked&lt;/strong&gt;: Can you share links about how to train a model on the PC and move the trained model to the Pi?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ebswifft replied&lt;/strong&gt;:&#xA;Glad you like it :).  Training the model on the PC is following the guide for Tensorflow for Poets &lt;a href=&quot;https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/&lt;/a&gt;.  All you do is install Tensorflow on the Raspberry Pi, clone the github repository onto it and copy the model you trained from the codelabs article onto the Raspberry Pi to run the classifier as per step 5 of the codelabs article.  Onboard image capture on the Raspberry Pi is just done using picamera.  I used a tensorflow version that a user compiled for the Raspberry Pi from here: &lt;a href=&quot;https://github.com/samjabrahams/tensorflow-on-raspberry-pi/issues/92&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samjabrahams/tensorflow-on-raspberry-pi/issues/92&lt;/a&gt;.  I might do up more of a general step-by-step sometime on my site, I'll report back if I can get that up and running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also &lt;a href=&quot;https://github.com/robodhhb/BrickClassifi3r&quot; rel=&quot;nofollow noreferrer&quot;&gt;BrickClassifi3r&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This Lego Mindstorms EV3 robot uses a neural network to recognize a cube, a cylinder or a small cube put on a conveyor belt. See the video how it works. Each object on the conveyor belt is scanned by an IR-sensor every 40ms for about a second. The resulting data are 24 distance values representing one of the three objects. This data is fed into the neural network on the robot to classify the object within 180ms. In a test with 300 objects it reaches 95,6% accuracy. The neural network has been trained before by a machine learning algorithm with TensorFlow on a PC using a set of 375 training examples - 125 examples for each object.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://github.com/aljazfrancic/ev3-myo&quot; rel=&quot;nofollow noreferrer&quot;&gt;ev3-myo&lt;/a&gt; project uses LEGO Mindstorms EV3, TRACK3R, a Myo armband and TensorFlow for gesture recognition.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2019-05-15T01:13:05.493" LastActivityDate="2019-05-15T01:13:05.493" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12313" PostTypeId="1" AcceptedAnswerId="12316" CreationDate="2019-05-15T00:27:48.573" Score="6" ViewCount="1196" Body="&lt;p&gt;I am trying to understand why attention models are different than just using neural networks. Essentially the optimization of weights or using gates for protecting and controlling cell state (in recurrent networks), should eventually lead to the network focusing on certain parts of the input/source. So what is attention mechanism really adding to the network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A potential answer in the case of Encoder-Decoder RNNs is:  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1409.0473.pdf&quot; rel=&quot;noreferrer&quot;&gt;The most important distinguishing feature of this approach from the&#xA;  basic encoder–decoder is that it does not attempt to encode a whole&#xA;  input sentence into a single fixed-length vector. Instead, it encodes&#xA;  the input sentence into a sequence of vectors and chooses a subset of&#xA;  these vectors adaptively while decoding the translation. This frees a&#xA;  neural translation model from having to squash all the information of&#xA;  a source sentence, regardless of its length, into a fixed-length&#xA;  vector. We show this allows a model to cope better with long&#xA;  sentences.&lt;br&gt;&#xA;  - Neural Machine Translation by Jointly Learning to Align and Translate&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which made sense and the paper says that it worked better for NMT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A previous study indicated that breaking down the sentence into phrases could lead to better results:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1409.1257.pdf&quot; rel=&quot;noreferrer&quot;&gt;In this paper, we propose a way to address this issue by automatically&#xA;  segmenting an input sentence into phrases that can be easily&#xA;  translated by the neural network translation model. Once each segment&#xA;  has been independently translated by the neural machine translation&#xA;  model, the translated clauses are concatenated to form a final&#xA;  translation. Empirical results show a significant improvement in&#xA;  translation quality for long sentences.&lt;br&gt;&#xA;   - Overcoming the Curse of Sentence Length for Neural Machine&#xA;  Translation using Automatic Segmentation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which paved the way for further research resulting in attention models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was also going through an article on &lt;a href=&quot;https://medium.com/octavian-ai/attention-is-not-quite-all-you-need-cb605ef3aff6&quot; rel=&quot;noreferrer&quot;&gt;Attention is not quite all you need&lt;/a&gt; where it said something similar:  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An LSTM has to learn to sequentially retain past values together in a&#xA;  single internal state across multiple RNN iterations, whereas&#xA;  attention can recall past sequence values at any point in a single&#xA;  forward pass.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and a more curated blog on the family of attention mechanism gives insight on how different ways have been formulated for implementing the concept: &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html&quot; rel=&quot;noreferrer&quot;&gt;Attention? Attention!&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I want to know how attention mechanism is formulated for this task (aforementioned) or in general. A detailed mathematical insight would be helpful, probably somewhat on these lines: &lt;a href=&quot;https://srome.github.io/Understanding-Attention-in-Neural-Networks-Mathematically/&quot; rel=&quot;noreferrer&quot;&gt;Understanding Attention in NN mathematically&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4573" LastActivityDate="2019-05-15T03:44:55.880" Title="A mathematical explanation of Attention Mechanism" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12315" PostTypeId="1" CreationDate="2019-05-15T03:11:44.070" Score="3" ViewCount="177" Body="&lt;p&gt;&lt;strong&gt;Dataset Description&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on famous ABIDE Autism Datasets. The dataset is very big in a sense that it has more than 1000 subjects containing half of them as autisitic and other half as healthy controls.The Dataset is taken from 17 sites across the word and each site used a varying time dimension when recording the subjects fMRI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use this dataset for classification task but only issue is time varying subjects as features set are fixed to 200 so you can say that I have subjects dimensions like 150 x200, 75 x 200 , 300 x 200... so on. So what are advanced AI or deep learning techniques that I can use to fix this time dimension for every subjects or can anybody suggest some deep learning framework or model that I could use to fix these varying time dimensions across subjects?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Effort&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have applied PCA to the time dimension and fixed them to 50 and tried other numbers also but it did not produce good accuracy for classification&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to use only specific time points from every subject like take only 40 time points from every subject to fix the dimension but again it did not work as definitely filtering some time series data on every subject would loose crucial information.&lt;/p&gt;&#xA;" OwnerUserId="25642" LastEditorUserId="25642" LastEditDate="2019-05-15T03:20:45.860" LastActivityDate="2021-02-08T15:02:30.223" Title="How to fix time dimension in time varying data-sets using deep learning model for classification?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;ai-community&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12316" PostTypeId="2" ParentId="12313" CreationDate="2019-05-15T03:44:55.880" Score="2" Body="&lt;p&gt;There's plenty, but keep in mind that these articles do not describe the same approach. They simply have attention shifting automation as part of their approaches and therefore must detect a need for shift and execute it in a way that improves speed, accuracy, reliability or some combination of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no one dominant attention approach and probably will not be. In fact, the earliest attention mechanism in common use in machines was likely the electromechanical fire alarm. In digital systems, it would be a vacuum tube electric eye driving an intruder alert followed by the first hardware interrupts in transistor microprocessor boards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sophistication of hardware interrupts in contemporary computer systems is probably higher than attention mechanisms in neural nets as of this writing, but that may change. Currently the dictionary definition of attention is the only constraint we can place on these newer approaches in artificial networks.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The four working links in the question (one is broken) lead to articles with some mathematical description, so the question half answers itself.&lt;/li&gt;&#xA;&lt;li&gt;This article, &lt;a href=&quot;https://pdf.sciencedirectassets.com/271585/1-s2.0-S0004370200X00331/1-s2.0-S0004370297000659/main.pdf?x-amz-security-token=AgoJb3JpZ2luX2VjEOr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQCVfJlD30DAXF7w%2BJZTxVddGdAUgsgNTlP3Db8WMtDDlAIgUU7ITEmLZveMDUkKDLVV50AoHNwUQ0hqDzLJR48UKA4q4wMI4v%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwwNTkwMDM1NDY4NjUiDOiCgCnrXcO4FYPkDCq3A9%2Bvw3GIADb%2FYrKeeImglZyeVuKDHCj5kt2lG7Zskv4scJPYJ4CH7CGL2A127Os3RvLzkHT87NUqZIUkKm%2FQPYSxJ2Cn6dmhmVgr2G44OcRT72nd84hBMN%2BYFzy3bcDcZax9AeM%2Br9fGEa1XtkDmsiMsvLfGEbByZ%2FoEZV0Kn5sxJBTBKaD5oHiKmdCDttXf9bzn0nZYEa2WA3BOzrQ8gBmtml%2BtdUa2arQXlvtsDJelhL5zjjPgJzJkOxGDLxOrEl28seWX75VNsE7010HYbpbteohQEFeAqLmmF0KLzQutYsjpfEHxeOmDuM9uzMlFYMlXdQA923o6dg%2F5YReXwon3fa84w%2FKn6cGztbDxt%2BS0n60P7FJnjNAZDqHwChSmKL3%2F61Pbie3lnfT7RAP%2B9kfKPIgfctjkjByVd2%2FB7y93X36wK%2FUwqcgnycNbfvdtY6rBGnqLXcEKG82l4bzVqTHQS8sYd791XxXa6dhw0jbAn%2Fk5aBM0ofELo4b67jfERsgj7oy4KwofjrH1WY6ExU5vMP%2FWSDPK0yPXayPWlQrOzsF0vWggkMUJ52A3jw%2FHvyU4DLotTNAwitLt5gU6tAHA%2Fy0ooqh979HVQpd5%2FGlH5eFEkiSVVRdi76GZWckrduAd1pSMPpfgKmIJfh0MDcWXUGZCx%2FR3qhq1nIzUjcgao3Iw6UYkI2smvO%2B0MyRnHxw%2BaVcWRImDeGTDhqtqfT9Y7ucVGPTDA4SYkH7nuWHxktkhSj8LtlT5Gk0iFdzMJRWIPCQx2aMf32l9qp%2FFDzpxn9gwPVickccSOHdjX9Y11MOyPVcydZ9L6ph3pntb2wd%2Fy8c%3D&amp;amp;AWSAccessKeyId=ASIAQ3PHCVTY3MVAATUK&amp;amp;Expires=1557889652&amp;amp;Signature=BzzLZHNG66%2FkecI%2FtjnJCvPRF8Y%3D&amp;amp;hash=9801a5a49f67b2342e82c00f4d9b746eb2951194644f233a0588e915f8d22bda&amp;amp;host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&amp;amp;pii=S0004370297000659&amp;amp;tid=spdf-74d42e48-3195-473c-ab3e-61994c3c81c0&amp;amp;sid=0ea5c4719d60084fe42944c4fd4f100bc2c0gxrqa&amp;amp;type=client&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Dynamic relevance: vision-based focus of attention using artificial neural networks&lt;/em&gt;&lt;/a&gt;,&#xA;Shumeet Baluja, Dean Pomerleau, 1995; revised 1996,&#xA;(&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370297000659&quot; rel=&quot;nofollow noreferrer&quot;&gt;abstract&lt;/a&gt;),&#xA;has only a few mathematical expressions but is filled with system&#xA;theory that could be described mathematically with a little reading&#xA;and thought.&lt;/li&gt;&#xA;&lt;li&gt;Same with &lt;a href=&quot;http://www.diva-portal.org/smash/get/diva2:987589/FULLTEXT01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Self-Organization of an Artificial Neural Network&#xA;Subjected to Attention Shift Impairments and Familiarity&#xA;Preference, Characteristics Studied in Autism&lt;/em&gt;&lt;/a&gt;, Lennart Gustafsson and Andrew P. Papliński, 2004.&lt;/li&gt;&#xA;&lt;li&gt;This one has clear and comprehensive mathematical treatment: &lt;a href=&quot;https://www.degruyter.com/downloadpdf/j/amcs.2012.22.issue-2/v10006-012-0034-5/v10006-012-0034-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Backpropagation Generalized Delta Rule for the Selective Attention Sigma-if Artificial Neural Network&lt;/em&gt;&lt;/a&gt;, Maciej Huk, 2012&lt;/li&gt;&#xA;&lt;li&gt;This one too: &lt;a href=&quot;https://doi.org/10.1371/journal.pone.0072035&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Small and Dim Target Detection via Lateral Inhibition Filtering and Artificial Bee Colony Based Selective Visual Attention&lt;/em&gt;&lt;/a&gt;, Haibin Duan, Yimin Deng, Xiaohua Wang, Chunfang Xu, 2013&lt;/li&gt;&#xA;&lt;li&gt;There is some here, but the visualizations and concepts are so theoretically relevant that one might not miss the more extensive mathematics: &lt;a href=&quot;http://sprott.physics.wisc.edu/pubs/paper451.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Using chaotic artificial neural networks to model memory in the brain&lt;/em&gt;&lt;/a&gt;, Zainab Aram, Sajad Jafari, Jun Ma, Julien C. Sprott, Sareh Zendehrouh, Viet-Thanh Pham&lt;/li&gt;&#xA;&lt;li&gt;Solid math coverage here: &lt;a href=&quot;https://iris.polito.it/retrieve/handle/11583/2303844/52344/cuda_v10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Parallel Implementation of Artificial Neural Network Training&lt;/em&gt;&lt;/a&gt;, Stefano Scanzio, Sandro Cumani, Roberto Gemello, Franco Mana, P. Laface, 2010, (&lt;a href=&quot;https://www.researchgate.net/publication/220732611_Parallel_implementation_of_artificial_neural_network_training&quot; rel=&quot;nofollow noreferrer&quot;&gt;abstract&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Also here: &lt;a href=&quot;https://arxiv.org/pdf/1412.7755.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Multiple Object Recognition with Visual Attention&lt;/em&gt;&lt;/a&gt;, Jimmy Lei Ba, Volodymyr Mnih, Koray Kavukcuoglu, 2015 ICLR&lt;/li&gt;&#xA;&lt;li&gt;The mathematics for the very popular DiSAN is here: &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/viewFile/16126/16099&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding&lt;/em&gt;&lt;/a&gt;, Tao Shen, Jing Jiang, Tianyi Zhou, Shirui Pan, Guodong Long, Chengqi Zhang, 2018&lt;/li&gt;&#xA;&lt;li&gt;Mathematics and the algorithm is provided for this unique approach too: &lt;a href=&quot;https://web.cs.wpi.edu/~xkong/publications/papers/kdd18b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Graph Classification using Structural Attention&lt;/em&gt;&lt;/a&gt;, John Boaz Lee, Ryan Rossi, Xiangnan Kong, 2018&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It would be interesting to develop a taxonomy of attention approaches in AI, as that has probably not yet been done. It would take quite a study to see if any of the above bullet items match up with either of the two articles referenced in the question.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-05-15T03:44:55.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12318" PostTypeId="1" CreationDate="2019-05-15T09:32:24.957" Score="1" ViewCount="91" Body="&lt;p&gt;I setupped a small drone simulator using PhysX, the time step is at 200 hz, while motors update like regular ESCs (at 50 Hz). I computed the inertia matrix, tweaked a bit mass of components to be real, air drag, gravity etc. After a first partial success in tuning a PID algorithm I got bored to find and hunt perfect values, and started thinking to tune it with a NN, but then I thinked, why using PID at all if NN can find better solutions?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started playing with NN, &lt;strong&gt;but then I realized I have no traning data.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure I could do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NN.Train(input,expectedOutput);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but what is actually the expected output? To be useful it has to be the thrust force of propellers tuned to keep input (position and orientation) stable in the desired place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But actually I don't (Want to) know in advance the thrust that every single propeller has.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is a simulation it is ok spending some computing time between each simulation step (eventually I'll try to optimize it later to fit it in a microcontroller)..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is it possible given a regular NN implementation where I can select number of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input neurons&lt;/li&gt;&#xA;&lt;li&gt;Hidden neurons&lt;/li&gt;&#xA;&lt;li&gt;Output neurons&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;find a way to train my model live? I need a way to tell the NN &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;hei this time you performed X keep going or go the opposite way.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1863" LastActivityDate="2019-05-15T16:26:19.913" Title="Drone training, how to train without training data?" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12319" PostTypeId="2" ParentId="12318" CreationDate="2019-05-15T10:01:37.943" Score="3" Body="&lt;p&gt;It's debatable whether neural networks can find a better solution than PID, if your goal is to simply keep the output around a certain reference point PID should do a perfect job pretty much. If you really want to use &quot;intelligent&quot; control with NN you can look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt;.&lt;br&gt;&#xA;Few interesting papers that directly adress your problem:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/~jordan/papers/ng-etal03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autonomous helicopter flight via reinforcement learning&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://heli.stanford.edu/papers/nips06-aerobatichelicopter.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Application of Reinforcement Learning to Aerobatic Helicopter Flight&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.465.8751&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autonomous Helicopter Control Using Reinforcement Learning Policy Search Methods&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also find more in the references of these papers.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-05-15T10:01:37.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12321" PostTypeId="1" AcceptedAnswerId="12346" CreationDate="2019-05-15T15:15:16.280" Score="4" ViewCount="111" Body="&lt;p&gt;Ok, due to &lt;a href=&quot;https://ai.stackexchange.com/questions/12318/drone-training-how-to-train-without-training-data&quot;&gt;previous question&lt;/a&gt; I was pointed to use reinfrocement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far what I understood from random websites is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;there is a Q(s,a) function involved&lt;/li&gt;&#xA;&lt;li&gt;I can assume my neural network ~ Q(s,a)&lt;/li&gt;&#xA;&lt;li&gt;my simulation has a state (N input variables)&lt;/li&gt;&#xA;&lt;li&gt;my actor can perform M possible actions (M output variables)&lt;/li&gt;&#xA;&lt;li&gt;at each step of the simulation my actor perform just the action corresponding to the &lt;code&gt;max(outputs)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;(in my case the actions are 1/2/3 % increase or decrease to propellers thrust force.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;From this &lt;a href=&quot;https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt; I found that at some point I have to&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Estimate outputs Q[t] (or so called q-values)&lt;/li&gt;&#xA;&lt;li&gt;Estimate outputs over next state Q[t+1]&lt;/li&gt;&#xA;&lt;li&gt;Let the backpropagation algorithm perform error correction only on the action performed on next state.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The last 3 points are not clear at all to me, infact I don't have yet the next state what I do instead is&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Estimate previous outputs Q[t-1]&lt;/li&gt;&#xA;&lt;li&gt;Estimate current outputs Q[t]&lt;/li&gt;&#xA;&lt;li&gt;Let backpropagation fix the error for max q value only&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Actually for code I use just this library which is simple enough to allow me understand what happens inside:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Hagsten/NeuralNetwork/tree/master/NeuralNetwork&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeuralNetwork library&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initializing the neural network (with N input neurons, N+M hidden neurons and M output neurons) is as simple as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Network network = new NeuralNetwork( N, N+M, M);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I think to understand there is the need for an arbitrary reward function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public double R()&#xA;{&#xA;     double distance = (currentPosition - targetPosition).VectorMagnitude();&#xA;     if(distance&amp;lt;100)&#xA;         return 100-distance; // the nearest the greatest the reward&#xA;     return -1; // too far&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then what I do is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// init step&#xA;var previousInputs = ReadInputs();&#xA;UpdateInputs();&#xA;var currentInputs = ReadInputs();&#xA;&#xA;//Estimate previous outputs Q[t-1]&#xA;previousOutputs = network.Query( previousInputs );&#xA;&#xA;//Estimate current outputs Q[t]&#xA;currentOutputs = network.Query( currentInputs);&#xA;&#xA;// compute modified max value&#xA;int maxIndex = 0;&#xA;double maxValue = double.MinValue;&#xA;SelectMax( currentOutputs, out maxValue, out maxIndex);&#xA;&#xA;// apply the modified max value to PREVIOUS outputs&#xA;previousOutputs[maxIndex] = R() + discountValue* currentOutputs[maxIndex];&#xA;&#xA;//Let backpropagation fix the error for max q value only&#xA;network.Train( previousInputs, previousOutputs);&#xA;&#xA;// advance simulation by 1 step and see what happens &#xA;RunPhysicsSimulationStep(1/200.0);&#xA;DrawEverything();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it doesn't seem to work very nice. I let simulation running for over one hour without success. Probably I'm reading the algorithm in a wrong way.&lt;/p&gt;&#xA;" OwnerUserId="1863" LastEditorUserId="1863" LastEditDate="2019-05-15T16:54:09.290" LastActivityDate="2019-05-16T11:03:40.107" Title="Q-learning, am I interpreting correctly $Q(s,a) = r + \gamma \max_{a'} Q(s',a')$?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12322" PostTypeId="1" AcceptedAnswerId="12342" CreationDate="2019-05-15T15:22:40.597" Score="3" ViewCount="692" Body="&lt;p&gt;I found Sentiment Analysis and Emotion Recognition as two different categories on &lt;a href=&quot;https://paperswithcode.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paperswithcode.com&lt;/a&gt;. Should both be the same as my understanding? If not what's the difference?&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-05-15T22:20:08.487" LastActivityDate="2020-05-02T21:16:49.730" Title="What is the difference between Sentiment Analysis and Emotion Recognition?" Tags="&lt;definitions&gt;&lt;comparison&gt;&lt;emotional-intelligence&gt;&lt;sentiment-analysis&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12323" PostTypeId="2" ParentId="12310" CreationDate="2019-05-15T15:40:35.550" Score="2" Body="&lt;p&gt;You need a bidirectional LSTM. The temporal direction on the original LSTM and on GRU type cell connectivity is forward in time. Fortunately, the bidirectional class is available in Keras.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential&#xA;from keras.layers import LSTM&#xA;from keras.layers import TimeDistributed&#xA;from keras.layers import Bidirectional&#xA;...&#xA;model = Sequential()&#xA;model.add(Embedding(max_features, 128, input_length=maxlen))&#xA;model.add(Bidirectional(LSTM(64)))&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(1, activation='sigmoid'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the network has causality from both history and forecast. This is only useful if the forecasts reduce entropy, which means they have to be known effective forecasts that when fed to earlier cells (in the time domain) will enhance accuracy and confidence metrics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the loss function (for its derivative) is provided, it must return the error at &lt;span class=&quot;math-container&quot;&gt;$t = t(i)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the index of the last sample in the sequence before the values become forecasts. This is the main distinction when the sequence includes both history and forecast.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2019-05-15T15:40:35.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12324" PostTypeId="1" AcceptedAnswerId="15913" CreationDate="2019-05-15T15:42:09.150" Score="4" ViewCount="529" Body="&lt;p&gt;What is the point of having multiple LSTM units in a single layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surely if we have a single unit it should be able to capture (remember) all the data anyway and using more units in the same layer would just make the other units learn exactly the same historical features?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've even shown myself empirically that using multiple LSTMs in a single layer improves performance, but in my head it still doesn't make sense, because I don't see what is it that other units are learning that others aren't? Is this sort of similar to how we use multiple filters in a single CNN layer?&lt;/p&gt;&#xA;" OwnerUserId="25659" LastEditorUserId="2444" LastEditDate="2019-05-15T22:17:06.743" LastActivityDate="2019-10-15T02:43:12.250" Title="Why do we need multiple LSTM units in a layer?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12325" PostTypeId="2" ParentId="12322" CreationDate="2019-05-15T16:04:53.493" Score="1" Body="&lt;p&gt;It's just as in common use. Sentiment is a superset of emotion, further including feelings, intuition, and rational conclusions drawn from past experience. Analysis is also a superset of recognition, which also includes taking emotions like contentment, happiness, humor affected, anger, rage, and passion and determining what sequences of them mean in the context of the scene or senario.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2019-05-15T16:04:53.493" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12326" PostTypeId="2" ParentId="12318" CreationDate="2019-05-15T16:26:19.913" Score="3" Body="&lt;p&gt;Training is not possible without training data, but it is not necessary to have a data set before the project begins. One can be generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That initial training when no data is available is tenuous at best is why parents run alongside the bicycle when their children are first learning to ride and why the training wheels are elevated so the parents can know when balance has been achieved and they can be removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach is to use a joystick and fly around in the simulation, leaving an audit trail. The collision, crash, and successful flight data will be valuable as labels. The other more sophisticated approach is to use a flight evaluation model attached to the flight model. How you rate the flights in the evaluation will determine the results of training. Metrics such as jostle, fuel consumption, time of flight, and near collisions are common in evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you use continuous learning such as reinforcement learning algorithms and libraries, you will need a flight model to train if you want anything close to optimal flight. The benefit of continuous learning in a real business context is that passengers can be paying customers as the safety and thrift of the flight business continues to improve. Reinforcement is only one kind of system that does that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, pilots don't actually use Markov chains and evaluate advantage of one option over another when piloting. It is a much more continuous type of control and there are attention mechanisms involving field of view and instrumentation that changes the control function under certain risk conditions. If you are interested in field-usable drone automation, these other aspects of flight control will need to be understood and included in your experimentation eventually. Better sooner than later if you are serious about the field.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2019-05-15T16:26:19.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12328" PostTypeId="5" CreationDate="2019-05-15T18:32:17.883" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-15T18:32:17.883" LastActivityDate="2019-05-15T18:32:17.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12329" PostTypeId="4" CreationDate="2019-05-15T18:32:17.883" Score="0" Body="For questions related to the &quot;black box&quot; nature of certain kinds of machine learning, where the internal decision making process is unknown." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-15T18:32:17.883" LastActivityDate="2019-05-15T18:32:17.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12330" PostTypeId="1" CreationDate="2019-05-15T20:40:40.257" Score="4" ViewCount="104" Body="&lt;p&gt;First I will clarify the context, I have to learn new technologies for my bachelor thesis. I am making a mobile application similar to Flappy Bird, except it's voice controlled. The idea is to have this app as a practice tool for people with voice problems (monotony). The bird flies upwards when the user makes a high pitch, and downwards when it's a low pitch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The app part is pretty much complete. But my project also consists of a website where the vocal coach can follow his user's progress. I am saving all the user's game history in a database. More specifically, every time the bird passes through a pipe opening, I save the pitch values that he correctly produced. I also save the pitch value that made him lose the game, along with the pitch value that he was supposed to produce to not lose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having all these data in my database, this is where I'd like to add AI as a new technology for my project. I thought of a nice feature where it would calculate the user's strong and weak points. For example it would say that a user is strong at high pitches, but is not good at low pitches. But since I don't know anything about AI, I'm not sure to what extent this is possible and have no idea where to start. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate if someone could share their knowledge with me by pointing me to existing libraries/frameworks! I'd most preferably include the AI code inside my API that is directly connected to my database, so I can directly return the calculated data to my website. My API is a SpringBoot app, so I guess I would need Java AI libraries?&lt;/p&gt;&#xA;" OwnerUserId="25662" LastEditorUserId="1671" LastEditDate="2019-10-15T19:24:09.267" LastActivityDate="2020-11-09T03:01:25.153" Title="AI method for evaluating user performance based on audio pitch re: public speaking" Tags="&lt;software-evaluation&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12332" PostTypeId="5" CreationDate="2019-05-15T21:11:59.093" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-15T21:11:59.093" LastActivityDate="2019-05-15T21:11:59.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12333" PostTypeId="4" CreationDate="2019-05-15T21:11:59.093" Score="0" Body="Use for questions about game AI for games that utilize randomness.  (This can include imperfect information games.)  Distinct from non-chance, perfect information games." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-15T21:11:59.093" LastActivityDate="2019-05-15T21:11:59.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12335" PostTypeId="1" AcceptedAnswerId="12336" CreationDate="2019-05-16T00:37:12.737" Score="0" ViewCount="47" Body="&lt;p&gt;I'm using 10-fold cross validation on all models. Here you can see both plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q4kwH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q4kwH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5pMek.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5pMek.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I am using k-fold cross validation, is it okay to name it &quot;validation error vs training error&quot; or &quot;test error vs training error&quot; would be better? &lt;/p&gt;&#xA;" OwnerUserId="25405" LastEditorUserId="2444" LastEditDate="2019-05-16T01:03:02.080" LastActivityDate="2019-05-16T01:03:02.080" Title="Should I call the error &quot;validation error&quot; or &quot;test error&quot; during cross validation?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;overfitting&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12336" PostTypeId="2" ParentId="12335" CreationDate="2019-05-16T00:50:33.237" Score="1" Body="&lt;p&gt;The expression &quot;validation error vs training error&quot; is likely more appropriate because the data you use during cross validation that is not the training data is often considered the validation data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The test data is the data you use to test your model after having performed e.g. cross-validation. The test dataset should be independent of both the training and validation datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a convention, and some people use the terms &quot;validation data&quot; and &quot;test data&quot; interchangeably, because, in a way, during cross-validation you are also &quot;testing&quot; (or, to disambiguate, &quot;validating&quot;) the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also this answer: &lt;a href=&quot;https://stats.stackexchange.com/a/401702/82135&quot;&gt;https://stats.stackexchange.com/a/401702/82135&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-05-16T00:50:33.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12337" PostTypeId="5" CreationDate="2019-05-16T01:01:39.753" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-05-11T10:52:33.763" LastActivityDate="2021-05-11T10:52:33.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12338" PostTypeId="4" CreationDate="2019-05-16T01:01:39.753" Score="0" Body="For questions related to the cross-validation techniques (e.g. k-fold cross-validation or leave-one-out cross-validation) used in machine learning to assess the quality (e.g. average accuracy) of the models." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-05-11T10:52:33.763" LastActivityDate="2021-05-11T10:52:33.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12340" PostTypeId="1" AcceptedAnswerId="12362" CreationDate="2019-05-16T05:06:39.407" Score="1" ViewCount="158" Body="&lt;p&gt;Below you have the plots of the training and validation errors for two different models. Both plots show the RMSE values for the validation dataset versus the number of training epochs. It is observed that models get lower RMSE value as training progresses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model associated with the first plot is performing quite well. The gap is quite narrowed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oMDvC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oMDvC.png&quot; alt=&quot;First Plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the model associated with this second plot is doing pretty good, but not as well as the other. The gap is much broader. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Suc5q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Suc5q.png&quot; alt=&quot;Second Plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model of the first plot was trained using a data set containing 1 million of ratings, while the second one used only 100K. I'm implementing the collaborative filtering (CF) algorithm. I am optimising it using SGD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LFPzs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LFPzs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are any of these models overfitting or underfitting?&lt;/p&gt;&#xA;" OwnerUserId="25405" LastEditorUserId="25405" LastEditDate="2019-05-17T02:03:48.390" LastActivityDate="2020-12-19T23:40:05.173" Title="Which model is better given their training and validation errors?" Tags="&lt;machine-learning&gt;&lt;overfitting&gt;&lt;recommender-system&gt;&lt;cross-validation&gt;&lt;underfitting&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12341" PostTypeId="1" CreationDate="2019-05-16T06:16:22.003" Score="3" ViewCount="47" Body="&lt;p&gt;I have a set of 15 unique playing cards from a deck of 52 playing cards. A given state is represented by the respective card values in the set of 15 cards, where the card value is a prime number associated with that card. For example, &lt;code&gt;AH&lt;/code&gt; is represented by &lt;code&gt;3&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I represent a single state for the NN? Should it be a list of the 15 prime numbers representing the list of cards? I was hoping that I could represent a single state as the &lt;strong&gt;sum&lt;/strong&gt; of each of all 15 prime numbers and then throw that sum through a sigmoid function. My concern, however, is that the NN will &lt;em&gt;lose&lt;/em&gt; information if I reduce the dimension of the state to a single attribute (even if that attribute is unique to that state - &lt;a href=&quot;https://en.wikipedia.org/wiki/Goldbach%27s_conjecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;the sum of &lt;code&gt;n&lt;/code&gt; prime numbers is unique compared to the sum of any other &lt;code&gt;n&lt;/code&gt; prime numbers&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How important is the dimensionality of each state for Deep Q Learning? I'd really appreciate even some general direction.&lt;/p&gt;&#xA;" OwnerUserId="25671" LastEditorUserId="2444" LastEditDate="2019-05-16T13:51:35.863" LastActivityDate="2019-05-16T13:51:35.863" Title="How do I represent a multi-dimensional state using a neural network?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12342" PostTypeId="2" ParentId="12322" CreationDate="2019-05-16T08:01:14.560" Score="1" Body="&lt;p&gt;Sentiment in this context refers to evaluations, typically positive/negative/neutral. &lt;a href=&quot;https://en.wikipedia.org/wiki/Sentiment_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sentiment Analysis&lt;/a&gt; can be applied to product reviews, to identify if the reviewer liked the product or not. This has (in principle) got nothing to do with emotions as such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Emotion_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emotion recognition&lt;/a&gt; would typically work on conversational data (eg from  conversations with chatbots), and it would attempt to recognise the emotional state of the user -- angry/happy/sad...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the same can overlap: if the user is happy, they will typically express positive sentiments on something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also: emotion recognition goes beyond text (eg facial expressions), whereas sentiment analysis mostly works with textual data only.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-05-02T21:16:49.730" LastActivityDate="2020-05-02T21:16:49.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12343" PostTypeId="1" AcceptedAnswerId="12389" CreationDate="2019-05-16T08:51:45.540" Score="5" ViewCount="533" Body="&lt;p&gt;I have a dataset as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pJIKM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pJIKM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(and the table extends to include an extra 146 columns for companies 4-149)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm I could use effectively to find similar patterns in sales from the other companies when compared to my company? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of using k-means clustering, but as I'm dealing with 150 companies here it would likely become a bit of a mess, and I don't think linear regression would work here.&lt;/p&gt;&#xA;" OwnerUserId="25675" LastEditorUserId="2444" LastEditDate="2019-05-19T14:49:31.417" LastActivityDate="2019-09-18T11:35:32.653" Title="Is there a machine learning algorithm to find similar sales patterns?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;linear-regression&gt;&lt;pattern-recognition&gt;&lt;k-means&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12344" PostTypeId="2" ParentId="12341" CreationDate="2019-05-16T09:27:52.910" Score="2" Body="&lt;p&gt;In my humble opinion, it seems like it is important to have them separated, if having a certain card can influence the result in some way that is not its prime value, instead of not only using the sum. But it depends on the game and its rules. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If having 5 cards of hearts in the set of 15 cards makes you win the game, then if you only represent the state as the sum of the value of the cards, the DQN will never learn that it was not the whole 15 cards that caused you to win, but only a specific part of the set.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2019-05-16T09:27:52.910" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12345" PostTypeId="1" AcceptedAnswerId="12369" CreationDate="2019-05-16T09:32:32.857" Score="2" ViewCount="108" Body="&lt;p&gt;In LeNet 5's first layer, the number of feature maps is equal to the number of kernels. However, the second convolutional layer has a depth different from the 3rd layer. Does the filter size dictate the number of feature maps?&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2019-05-17T13:28:21.927" LastActivityDate="2019-05-17T13:40:47.743" Title="Is the number of feature maps equal to the number of kernels in the LeNet 5 architecture?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;features&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12346" PostTypeId="2" ParentId="12321" CreationDate="2019-05-16T10:09:09.837" Score="2" Body="&lt;p&gt;What normally happens in DQN is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the NN is used to estimate an approximation of Q for each state-action pair by using a vector of weights:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a,w) \simeq Q(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two options: You can either feed it a certain state, and get as output the value of each possible action, or, you can input a state-action pair and get as output the value of that pair. (basically the NN gives you the policy)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After having this output you can now use your environment to simulate the next state. So basically you apply the action given by the NN (maybe using &lt;span class=&quot;math-container&quot;&gt;$\epsilon -greedy$&lt;/span&gt;), and you get the next state, and your next reward from your environment by observing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does the Q function comes in all of this? The Q-function &#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = Q(s,a) + \alpha [r(s,a) + \gamma  max_a{Q(s',a)} - Q(s,a)]$$&lt;/span&gt;&#xA;that in standard Q-learning is used to update the values after each simulation step, is now used as the Target of the neural network, to make the back propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some considerations you have to make to help this converge, mainly experience replay and fixed Q-targets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when you say &quot; I don't have yet the next state&quot;, is because you need to simulate the action that is selected to get this next state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; The main thing you might be missing is that the right part of the equation you present in the title is called TD-Target, and it is not used to make the updates, you might see it as the final result you want to achieve &lt;span class=&quot;math-container&quot;&gt;$Q^*(s,a)$&lt;/span&gt; (the optimal values, after convergence). But to make the update you need to use the weighted function with the previous value of &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; as I showed above, using a learning rate &lt;span class=&quot;math-container&quot;&gt;$0&amp;lt; \alpha &amp;lt; 1$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-05-16T11:03:40.107" LastActivityDate="2019-05-16T11:03:40.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12348" PostTypeId="1" CreationDate="2019-05-16T11:34:49.153" Score="1" ViewCount="86" Body="&lt;p&gt;At the moment I am working on a vehicle counting &amp;amp; classification project. &#xA;For a specific part in the project I need to get back only the completely visible vehicles from my input data (images).  I am wondering if this could be done (more) automatically in the following way:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;zoom in such that only approximately one van would be visible&lt;/li&gt;&#xA;&lt;li&gt;divide the vehicles into two categories: truncated and non-truncated&lt;/li&gt;&#xA;&lt;li&gt;train on these two classes&lt;/li&gt;&#xA;&lt;li&gt;After training and testing, use the model to find the completely visible vehicles. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So the main question is, is it possible that this would give sufficient results or should I try to find another solution?&lt;/p&gt;&#xA;" OwnerUserId="23473" LastActivityDate="2021-02-04T19:07:30.647" Title="Divide classes into truncated and non-truncated objects" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12349" PostTypeId="2" ParentId="12343" CreationDate="2019-05-16T13:33:13.567" Score="2" Body="&lt;p&gt;I would recommend a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical cluster algorithm&lt;/a&gt;, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make the cut, you can decide how many clusters you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A great resource on this topic is Kaufman, L., &amp;amp; Roussew, P. J. (1990). &quot;Finding Groups in Data - An Introduction to Cluster Analysis&quot;. John Wiley &amp;amp; Sons&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-05-16T13:33:13.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12350" PostTypeId="2" ParentId="12348" CreationDate="2019-05-16T13:33:53.347" Score="0" Body="&lt;p&gt;It could work. I think itll be hard to find labelled bounding box data describing full/truncated cars so this is a good idea for self labelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theres probably also some other ways for you to get labelled data without the crowd sourcing or self labelling. Take a large labelled car dataset with bounding boxes (COCO has some, and if your good with aerial imagery theres alot more out there), and just enforce that any car bounding boxes that are touching the boundary are truncated, while ones fully encompassed in the imaged are not, since with high probability that will be the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-16T13:33:53.347" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12353" PostTypeId="2" ParentId="12348" CreationDate="2019-05-16T14:54:58.830" Score="0" Body="&lt;p&gt;Your approach should work well, you just have to test different neural networks &lt;strong&gt;architectures&lt;/strong&gt; to see the most accurate one. I suggest you focus on &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;CNNs&lt;/strong&gt;&lt;/a&gt; architectures and you'll be good to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want to spend much time labelling your data you can &lt;strong&gt;only label non-truncated vehicles&lt;/strong&gt;. Everything else is not a &quot;non-truncated vehicle&quot;. This is even better because some images may contain persons, roads, animals... and your model will get confused if he's only trained to know truncated vehicles and non truncated vehicles. The other model will instead know how non-truncated vehicles look like and consider everything else as &quot;not a non-truncated vehicle&quot;. And this is exactly what you need in your use-case.&lt;/p&gt;&#xA;" OwnerUserId="23350" LastActivityDate="2019-05-16T14:54:58.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12354" PostTypeId="2" ParentId="10289" CreationDate="2019-05-16T14:57:00.340" Score="1" Body="&lt;p&gt;&lt;em&gt;What is a statistical model?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Anthony C. Davison (in the book &lt;a href=&quot;https://fac.ksu.edu.sa/sites/default/files/statistical_models.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistical Models&lt;/a&gt;), a statistical model is a probability distribution constructed to enable inferences to be drawn or decisions made from data. The probability distribution represents the variability of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are all neural networks statistical models?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All neural networks that construct a probability distribution to draw inferences from the data or to make decisions from the data are statistical models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variational auto-encoders (VAEs) construct a probability distribution (e.g. a Gaussian) to draw inferences, so VAEs can be considered &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical models&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also &lt;a href=&quot;https://arxiv.org/pdf/1505.05424.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural networks&lt;/a&gt;, which are neural networks that maintain a probability distribution (usually, a Gaussian) for each unit (or neuron) of the neural network, rather than only a point estimate. BNNs can thus also be considered statistical models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, for example, MLPs do not necessarily construct any probability distribution, so they are not necessarily statistical models. However, note that MLPs can be used to represent the parameters of a distribution. For example, you could train an MLP to represent the mean of a Gaussian distribution. See e.g. &lt;a href=&quot;https://arxiv.org/pdf/1802.04364.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Junction Tree Variational Autoencoder for Molecular Graph Generation&lt;/a&gt; for an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, not all neural networks are statistical models (at least, according to the definition by Davison).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-11T23:35:18.130" LastActivityDate="2020-03-11T23:35:18.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12355" PostTypeId="2" ParentId="12172" CreationDate="2019-05-16T15:26:41.373" Score="1" Body="&lt;p&gt;Recursive artificial networks are a type aligned well with GDL, and it is an increasingly popular one. Using non-orthogonal geometries is natural when using directed graphs as semantic association, data flow, or composition networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This more free-form approach enters into &lt;code&gt;industry&lt;/code&gt; slowly because most programming languages use orthogonal structures designed around multidimensional arrays and nested loops to iterate through them, made popular in the FORTRAN era. It is common that a student learns to loop through an array before learning to call a function or test and branch. Computer science is, in some ways, entrenched in the orthogonal structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To call the two categories Euclidean and non-euclidean is inaccurate. The initial trend in machine learning followed a Cartesian paradigm, one locked into 90° angles in data structures. The free-form graph is non-cartesian but still just as Euclidean, as Euclid worked with angles other than 90° frequently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apple and Google are likely candidates for this seepage of recursive networks into industry data centers, for NLP applications. Language associations are free-form even though these structures are serialized for speech and written text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cognition is also non-cartesian. Graph based optimization (pre-GDL) has been part of cognitive research since the 1970s, especially in the LISP space at MIT and CMU and the U.S. Naval Research facilities for strategic analysis, &lt;a href=&quot;https://projecteuclid.org/download/pdf_1/euclid.aos/1176343791&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Optimal strategies for a class of constrained sequential problems&lt;/em&gt;&lt;/a&gt; by JB Kadane, HA Simon, The Annals of Statistics, 1977. Algorithm 2 is a convergence strategy, but without layers of cells, yet the searching concepts of gradient are already present in the maximal and minimal operations in the search in steps &lt;em&gt;c&lt;/em&gt; and &lt;em&gt;f&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aeronautics and auto industry is a likely candidates for seepage into the embedded computing for automated driving and flight. Most of this is either company confidential or classified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/7547/is-topological-sophistication-necessary-to-the-furtherance-of-ai/8529#8529&quot;&gt;This answer&lt;/a&gt; to the question about topological sophistication provides some background why non-euclidean space may be a more natural way to map associations, flow, and composition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some practical applications.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/5954-convolutional-networks-on-graphs-for-learning-molecular-fingerprints.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Convolutional networks on graphs for learning molecular fingerprints&lt;/em&gt;&lt;/a&gt; by DK Duvenaud, D Maclaurin, J Iparraguirre, Advances in neural, 2015&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/JHEP01%282019%29057.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;CD-aware recursive neural networks for jet physics&lt;/em&gt;&lt;/a&gt; by Gilles Louppe, Kyunghyun Cho, Cyril Becot, Kyle Cranmer, 2019&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1905.04497&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Stability Properties of Graph Neural Networks&lt;/em&gt;&lt;/a&gt; by Fernando Gama, Joan Bruna, Alejandro Ribeiro&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1811.11465&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Isospectralization, or how to hear shape, style, and correspondence&lt;/em&gt;&lt;/a&gt; by Luca Cosmo, Mikhail Panine, Arianna Rampini, Maks Ovsjanikov, Michael M. Bronstein, Emanuele Rodolà &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1902.06673&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Fake News Detection on Social Media using Geometric Deep Learning&lt;/em&gt;&lt;/a&gt;, by Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, Michael M. Bronstein&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We've used recursive networks to align complex models with data acquired during laboratory experimentation and hope to publish on that approach as part of another publication. It is effective, since the relationships between features in the real world form squares and cubes only by chance, so it is not very common. Research into materials and energy cannot be confined by Cartesian conceptions, even if the graphs published are on Cartesian coordinate axes, so that others with traditional analytic geometry backgrounds can understand the graphs quickly.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-05-16T15:39:40.437" LastActivityDate="2019-05-16T15:39:40.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12356" PostTypeId="2" ParentId="12263" CreationDate="2019-05-16T18:41:42.183" Score="1" Body="&lt;p&gt;The specific approaches you mentioned (A3C, DDPG), and usually also other Actor-Critic methods in general, are approaches for the standard &lt;strong&gt;single-agent&lt;/strong&gt; Reinforcement Learning (RL) setting. When trying to apply such algorithms to settings that are actually multi-agent settings, it is indeed common to encounter the problems you describe. They can all be viewed as your agent &quot;overfitting&quot; to the opponent they're training against:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Play against random player. I had rather good results, but not as good as expected since most interesting states cannot be reached with a random opponent.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this case, your agent will &quot;overfit&quot; against random opponents. It will likely become capable of beating them relatively easily, and to some extent this may also still translate to relatively weak non-random agents (like simple heuristic approaches), but it is unlikely to generalise to strong opponents.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Play against list of specific AIs. Results were excellent against those AIs, but very bad with never seen opponents&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here you pretty much exactly give a description of what I mean when I say &quot;overfitting&quot; against the opponent you're training against.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Play against itself. Seemed the best to me, but I could not get any convergence due to non-stationary environment. &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When training using self-play like this, there is still a danger of overfitting against &quot;itself&quot;. This may turn out to work well in some games, but in other games, there are indeed risk of instability / lack of convergence. This can, for example, be due to the agent &quot;rotating&quot; through a circle of strategies that beat each other. Intuitively, you could think of Rock-Paper-Scissors. A randomly-initialised agent may, for example, have a slight tendency to play Rock more often than the others. Self-play will then lead to a strategy that primarily plays Paper, to beat the current &quot;Rock&quot; strategy. Continued self-play training can then figure out that a tendency to play more Scissors will be strong against itself. Etc., these strategies can keep rotating forever.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The standard approach for self-play training in games is the one popularised by &lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero&lt;/a&gt; and &lt;a href=&quot;https://science.sciencemag.org/content/362/6419/1140&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero&lt;/a&gt;, and also around the same time &lt;a href=&quot;https://papers.nips.cc/paper/7120-thinking-fast-and-slow-with-deep-learning-and-tree-search&quot; rel=&quot;nofollow noreferrer&quot;&gt;independently published with results on the game of Hex&lt;/a&gt;. That last paper coined the term &quot;Expert Iteration&quot; for the approach, which I like to use. The basic idea of Expert Iteration is to have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A policy that we're training (parameterised by some function approximator like a Deep Neural Net)&lt;/li&gt;&#xA;&lt;li&gt;A tree search algorithm, like Monte Carlo tree search (MCTS).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These two components can then iteratively improve each other. The trained policy can be used to guide the search behaviour of MCTS. MCTS can be used to generate a new distribution over actions which, thanks to additional search effort, is typically expected to be a little bit better than then trained policy. That slightly better distribution is then used as a training target for the policy, using a Cross-Entropy loss. This means that the policy is trained to mimic the MCTS search behaviour, and also used to improve that same MCTS search behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This training procedure has been found to lead to strong results. One of the prevailing hypotheses is that the additional search performed by MCTS can help to stabilise training, and avoid overfitting to the &quot;self&quot; opponent; the search algorithm can actually &quot;reason&quot; about what a strong opponent (not necessarily the &quot;self&quot; agent) could do.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The Expert Iteration training procedure I described above often works well, but it doesn't really answer your question about training Actor-Critic approaches in games with opponents... because it's not really an Actor-Critic algorithm!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1905.05809&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimates&lt;/a&gt;, we (I'm first author on this paper) propose an approach where the policy in self-play is trained using a training target based on the value estimates of MCTS, rather than the visit counts of MCTS (which is what we use in the standard Expert Iteration framework). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if it necessarily should exactly be called an Actor-Critic algorithm, but you could intuitively view it as an &quot;Actor-Critic&quot; approach where MCTS is the critic. As mentioned right before Section IV, the resulting gradient estimator also turns out to be very similar to that of the &quot;Mean Actor Critic&quot;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If Expert-Iteration-style solutions are not an option (for example because tree search is infeasible), I would suggest taking a look at some of the following Policy-Gradient-based approaches (and possibly later research that cites these papers):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1709.04326&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning with Opponent-Learning Awareness&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1904.03646&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy Gradient Search: Online Planning and Expert Iteration without Search Trees&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-05-16T18:41:42.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12357" PostTypeId="1" CreationDate="2019-05-16T18:50:27.717" Score="1" ViewCount="251" Body="&lt;p&gt;So I´m currently implementing my first neural network using GRUs as a model and Keras as an implementation since it´s pretty highlevel.&#xA;My problem is about the classification of 8 hour long timeseries with 11 different events with 1 second timesteps or to be more specific: sleep recordings.&lt;br&gt;&#xA;Since the GRU can´t handle the whole timeseries at once, I split it in 500 timepoint pieces with 50 seconds (10%) overlap.  Also since I don´t have much data, after the training/test split I´m oversampling the train data by duplicating the underrepresented classes up to 10 times. So at the end I get a set of ca. 14.000 training snippets and ca 1.800 to test the model. That to the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next thing is the model, represented with the Keras Code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;verbose, epochs, batch_size = 2, 50, 1200  # 1200 batch_size was the maximum the GPU can handle&#xA;    n_timesteps, n_features, n_outputs = trainX.shape[1], trainX.shape[2], trainy.shape[1]&#xA;&#xA;    model = Sequential()&#xA;    model.add(GRU(128, input_shape=(n_timesteps, n_features), return_sequences=True, dropout=0.7,&#xA;                  kernel_regularizer=regularizers.l2(0.01), activation=&quot;relu&quot;))&#xA;    model.add(&#xA;        GRU(64, return_sequences=True, go_backwards=False, dropout=0.7, kernel_regularizer=regularizers.l2(0.01),&#xA;            activation=&quot;relu&quot;))&#xA;    model.add(&#xA;        GRU(32, return_sequences=True, go_backwards=False, dropout=0.7, kernel_regularizer=regularizers.l2(0.01),&#xA;            activation=&quot;relu&quot;))&#xA;    model.add(&#xA;        GRU(16, return_sequences=True, go_backwards=False, dropout=0.7, kernel_regularizer=regularizers.l2(0.01),&#xA;            activation=&quot;relu&quot;))&#xA;    model.add(TimeDistributed(Dense(units=16, activation='relu')))&#xA;    model.add(Flatten())&#xA;    model.add(Dense(n_outputs, activation='softmax'))&#xA;&#xA;    # define custom optimizer with configurable learning rate&#xA;    sgd = optimizers.SGD(lr=0.1, momentum=0.9, decay=0.0, nesterov=False)&#xA;    model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So as a short summary:&lt;br&gt;&#xA;I am using 4 GRU layers with a &quot;pyramid&quot; shape, so the network has to be more specific towards the end. Also one fully connected layer as some kind of &quot;adapter&quot; and one output layer with the size of my features. I am using SGD as an optimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results are always pretty bad, here are the loss and accuracy plots of an example run featuring 25 epochs:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/gierx.png?s=250&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IL81q.png?s=250&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IL81q.png?s=250&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the huge dropout each stage, it still seems to be overfitting. Also as you can see, the accuracy of test and train is not changing after the 2nd epoch. The result is this sad looking confusion-matrix, showing only one class beeing detected by the network, which is not even the one with the highest amount of data in the dataset:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[231   0   0   0   0   0   0   0]&#xA; [ 55   0   0   0   0   0   0   0]&#xA; [647   0   0   0   0   0   0   0]&#xA; [141   0   0   0   0   0   0   0]&#xA; [444   0   0   0   0   0   0   0]&#xA; [  0   0   0   0   0   0   0   0]&#xA; [118   0   0   0   0   0   0   0]&#xA; [ 74   0   0   0   0   0   0   0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So what are possible approaches in order to:&lt;br&gt;&#xA;a) fight the overfitting,&lt;br&gt;&#xA;b) just one class beeing detected and&lt;br&gt;&#xA;c) fix the stationary accuracy to be growing again?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I´m pretty new to neural networks I am thankful for your time and effort!&lt;/p&gt;&#xA;" OwnerUserId="25687" LastEditorUserId="25687" LastEditDate="2019-05-16T19:11:06.003" LastActivityDate="2019-06-15T20:03:30.427" Title="Train and Test Accuracy of GRU network not increasing after 2nd epoch" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;keras&gt;&lt;loss-functions&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12358" PostTypeId="2" ParentId="12357" CreationDate="2019-05-16T19:02:09.727" Score="1" Body="&lt;p&gt;Couple reccomendations:&lt;br&gt;&#xA;1) I dont think your overfitting, your test loss is not ever increasing and is staying reasonbly proportional to train loss -- This may indicate that whatever loss your using is not a good indicator of the metric of interest (in this case, it seems you want that to be accuracy, but data is imbalnced so maybe look at avg precision?)&lt;br&gt;&#xA;2) Use a lr scheduler, or something like a ReduceLROnPlateau callback to reduce the lr once the loss converges&lt;br&gt;&#xA;3) Adding more sources of the underrepresented class is valid but i reccomend just using class weighting. Effectively its the same (as an expectation) and will save you train time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good Luck!&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-16T19:02:09.727" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12360" PostTypeId="1" AcceptedAnswerId="12424" CreationDate="2019-05-16T19:30:30.733" Score="1" ViewCount="416" Body="&lt;p&gt;Understandably RNNs are very good at solving problems involving audio, video and text processing due to the arbitrary input's length of this sort of data.&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand is why RNNs are also superior at predicting time series data and why we use them over simple MLP DNNs.&lt;/p&gt;&#xA;&lt;p&gt;Say I wanted to predict what the value in the time series is at &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;. I would take a window of, let's say, &lt;span class=&quot;math-container&quot;&gt;$t-50, t-49, \dots, t$&lt;/span&gt;, and then feed loads of sampled training data into a network. I could either choose to have a single LSTM unit remembering the entire window and basing the predictions on that, or I could simply make a 50 neuron wide MLP network.&lt;/p&gt;&#xA;&lt;p&gt;What exactly is it about RNNs that makes them better in this scenario or any scenario for that matter?&lt;/p&gt;&#xA;&lt;p&gt;I understand that the LSTM would have substantially fewer weights (in this scenario) and should be less computationally intensive, but, apart from that, I don't see any difference in these two methods.&lt;/p&gt;&#xA;" OwnerUserId="25659" LastEditorUserId="2444" LastEditDate="2020-10-02T19:19:40.270" LastActivityDate="2020-10-02T19:22:16.633" Title="Why are RNNs better than MLPs at predicting time series data?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12361" PostTypeId="2" ParentId="12360" CreationDate="2019-05-16T19:47:05.307" Score="0" Body="&lt;p&gt;RNNs have the ability to hold a state, that means the model can learn which information it wants to save and what to delete based on ordering and how you designed the creation and passing of the state (probably worth looking into what an LSTM is), while this would be alot more difficult for a sliding MLP to do (you can think of a sliding MLP as a stateless RNN). Also a sliding MLP would be alot more computation since it needs to recompute an entire context window for each new output, while an RNN could just use the previous state and only do processing on a new singular input or a smaller context window.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-16T19:47:05.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12362" PostTypeId="2" ParentId="12340" CreationDate="2019-05-17T00:46:14.250" Score="1" Body="&lt;p&gt;I would say that your intuition is correct: the model associated with the first plot is likely to generalise more than the one associated with the second plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, it doesn't seem that your model has overfitted the training data. Overfitting often occurs when the training error keeps decreasing but the validation error starts to increase. In both your plots, both the training and validation errors keep decreasing (even if slowly, after a while).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Underfitting occurs when your model hasn't learned enough even about your training data. The smaller the training and validation error, the more likely your model has not underfitted, but the value of RMSE depends on the range of your inputs. See e.g. &lt;a href=&quot;https://stats.stackexchange.com/q/56302/82135&quot;&gt;What are good RMSE values?&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also this article &lt;a href=&quot;https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overfitting and Underfitting With Machine Learning Algorithms&lt;/a&gt; for a general overview of the concepts of overfitting and underfitting.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T01:00:46.747" LastActivityDate="2019-05-17T01:00:46.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12365" PostTypeId="1" CreationDate="2019-05-17T12:25:42.863" Score="1" ViewCount="120" Body="&lt;p&gt;Right now, I am trying to synthesize training images for a CNN and due to the nature of the application, there is a finite number of sample images to learn from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From other research, I expect to be using about 200,000 training images at a resolution of 1280*720, which with 3 channel at 8 bits will take about 550 GB to save uncompressed. This number can and probably will rise in the future, meaning more memory that I will need to provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine that there are applications that required even more training data with higher complexity and that there are solutions to handling that such as compression techniques and the like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Are there solutions for the memory management beyond compressing the images with JPEG and such besides generating and instantly consuming the pictures without saving them to permanent memory?&lt;/p&gt;&#xA;" OwnerUserId="25702" LastEditorUserId="2444" LastEditDate="2019-06-30T16:02:45.487" LastActivityDate="2021-03-23T10:03:55.480" Title="How to manage large amounts of image data for training?" Tags="&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;memory&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12366" PostTypeId="1" AcceptedAnswerId="12367" CreationDate="2019-05-17T12:26:56.863" Score="1" ViewCount="42" Body="&lt;p&gt;I've recently been reading up on CNNs and this part of the architecture is really confusing me. &#xA;Assume, I have an input of size [32*32*3] and pass it to a convolution layer. Now, if my kernel size were to be [5*5*3] and the depth of my convolution layer were to be 1, only one feature map would be produced for the image. Here, each neuron would have a 75 weights (+1 bias). If I wanted to calculate multiple feature maps in this layer, say 3, is each local section (in this example [5*5*3]) of the image looked on by three different neurons and each of their weights trained individually? And what would be the output volume of this layer?&lt;/p&gt;&#xA;" OwnerUserId="25704" LastActivityDate="2019-05-17T13:48:14.760" Title="Convolution layer neurons when extracting multiple feature maps" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;features&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12367" PostTypeId="2" ParentId="12366" CreationDate="2019-05-17T12:48:59.957" Score="0" Body="&lt;p&gt;Each feature map (or kernel) is independent of each other. If you had &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; of these filters, your output shape would be &lt;span class=&quot;math-container&quot;&gt;$(28, 28, 3)$&lt;/span&gt; (given the appropriate amount of padding and stride) with a total of &lt;span class=&quot;math-container&quot;&gt;$75*3=225$&lt;/span&gt; trainable weights.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-05-17T13:48:14.760" LastActivityDate="2019-05-17T13:48:14.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12368" PostTypeId="2" ParentId="12345" CreationDate="2019-05-17T12:56:34.727" Score="0" Body="&lt;p&gt;The # of kernels will be the channel length, looking at the image you posted in your comment from &lt;a href=&quot;https://medium.com/@shahariarrabby/lenet-5-alexnet-vgg-16-from-deeplearning-ai-2a4fa5f26344&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt;, I do not understand where you see the inconsistency.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-17T12:56:34.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12369" PostTypeId="2" ParentId="12345" CreationDate="2019-05-17T13:18:25.587" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MAArI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MAArI.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, each kernel has the same depth as the depth of the input &lt;a href=&quot;https://en.wikipedia.org/wiki/Cuboid&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuboid&lt;/a&gt;. In the architecture above, we have an (input) cuboid of dimension &lt;span class=&quot;math-container&quot;&gt;$14 \times 14 \times 6$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt; is the depth, which is followed by an (output) cuboid of dimension &lt;span class=&quot;math-container&quot;&gt;$10 \times 10 \times 16$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; is the depth, which is the number of (output) feature maps (or channels) and it is also equal to the number of kernels applied to the (input) cuboid of dimensions &lt;span class=&quot;math-container&quot;&gt;$14 \times 14 \times 6$&lt;/span&gt;. Each of these &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; kernels has a depth of &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt;. So, each of these &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; kernels has a shape of &lt;span class=&quot;math-container&quot;&gt;$5\times 5 \times 6$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the depth of the input cuboid can be different than the depth of the output cuboid, which is often equal to the number of kernels applied to the input cuboid (but this is not always the case). Furthermore, the depth of &lt;em&gt;each&lt;/em&gt; kernel (applied to the input cuboid) has (often) the same depth as the input cuboid.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T13:40:47.743" LastActivityDate="2019-05-17T13:40:47.743" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12370" PostTypeId="1" CreationDate="2019-05-17T14:53:12.187" Score="1" ViewCount="246" Body="&lt;p&gt;I am currently working with a small dataset of 20x300. Since I have so few data points, I was wondering if I could use an approach similar to leave-one-out cross-validation but for testing.&lt;/p&gt;&#xA;&lt;p&gt;Here's what I was thinking:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;train/test split the data, with only one data point in the test set.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;train the model on training data, potentially with grid-search/cross-validation&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;use the best model from step 2 to make a prediction on the one data point and save the prediction in an array&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;repeat the previous steps until all the data points have been in the test set&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate your preferred metric of choice (accuracy, f1-score, auc, etc) using these predictions&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The pros of this approach would be to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You don't have to split the data into train/test so you can train&#xA;with more data points.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The cons would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;This approach suffers from potential(?) data leakage.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You are calculating an accuracy metric from a bunch of predictions that potentially came from different models, due to the grid searches, so I'm not sure how accurate it is going to be.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have tried the standard approaches of train/test splitting, but since I need to take out at least 5 points for testing, then I don't have enough points for training and the ROC AUC becomes very bad.&lt;/p&gt;&#xA;&lt;p&gt;I would really appreciate some feedback about whether this approach is actually feasible or not and why.&lt;/p&gt;&#xA;" OwnerUserId="25708" LastEditorUserId="2444" LastEditDate="2021-05-11T10:47:33.060" LastActivityDate="2021-05-11T10:47:33.060" Title="Should I use leave-one-out cross-validation for testing?" Tags="&lt;training&gt;&lt;cross-validation&gt;&lt;testing&gt;&lt;loocv&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12371" PostTypeId="2" ParentId="12365" CreationDate="2019-05-17T15:30:26.963" Score="1" Body="&lt;p&gt;I suggest you &lt;strong&gt;fine-tune an existing model&lt;/strong&gt;. Knowledge transfer models in many image processing tasks are now open sourced and you can build your model on top of them. Also, knowledge transfer models are trained on a large datasets and can quickly converge to your case-study with a little of task-specific extra training. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This way you will use few data to tune the model which leads to less&#xA;  memory use and less training time. You will also take advantage from a&#xA;  ready-to-use architecture and get accurate results.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Depending on your case-study you can choose from this &lt;em&gt;&lt;a href=&quot;https://modelzoo.co/category/computer-vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;list of computer vision pre-trained models.&lt;/strong&gt;&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23350" LastActivityDate="2019-05-17T15:30:26.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12372" PostTypeId="5" CreationDate="2019-05-17T16:39:44.017" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/&lt;/a&gt; for more info regarding overfitting and underfitting ML models.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:17.850" LastActivityDate="2019-05-17T20:27:17.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12373" PostTypeId="4" CreationDate="2019-05-17T16:39:44.017" Score="0" Body="For questions related to the concept of underfitting in machine learning, which occurs when a machine learning model is not able to learn." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-20T22:50:14.783" LastActivityDate="2020-03-20T22:50:14.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12374" PostTypeId="5" CreationDate="2019-05-17T16:42:10.170" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Time_series&lt;/a&gt; or &lt;a href=&quot;https://machinelearningmastery.com/time-series-forecasting-supervised-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/time-series-forecasting-supervised-learning/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:26:56.900" LastActivityDate="2019-05-17T20:26:56.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12375" PostTypeId="4" CreationDate="2019-05-17T16:42:10.170" Score="0" Body="For questions related to time series analysis or forecasting in the context of AI and, in particular, ML." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:15.247" LastActivityDate="2019-05-17T20:27:15.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12376" PostTypeId="5" CreationDate="2019-05-17T16:43:03.160" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Semi-supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Semi-supervised_learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:25.303" LastActivityDate="2019-05-17T20:27:25.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12377" PostTypeId="4" CreationDate="2019-05-17T16:43:03.160" Score="0" Body="For questions related to the machine learning technique called semi-supervised learning, which is a combination of supervised and unsupervised learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:23.093" LastActivityDate="2019-05-17T20:27:23.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12378" PostTypeId="5" CreationDate="2019-05-17T16:44:12.557" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-05-17T16:44:12.557" LastActivityDate="2019-05-17T16:44:12.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12379" PostTypeId="4" CreationDate="2019-05-17T16:44:12.557" Score="0" Body="For questions related to conditional probability e.g. in the context of Bayesian inference or networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:28.573" LastActivityDate="2019-05-17T20:27:28.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12380" PostTypeId="5" CreationDate="2019-05-17T16:47:05.643" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Ensemble_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Ensemble_learning&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:20.580" LastActivityDate="2019-05-17T20:27:20.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12381" PostTypeId="4" CreationDate="2019-05-17T16:47:05.643" Score="0" Body="For questions related to the random forest (or random decision forests), which is an ensemble machine learning technique (that is, an ML technique that uses or combines different models)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:36.573" LastActivityDate="2019-05-17T20:27:36.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12382" PostTypeId="5" CreationDate="2019-05-17T16:48:10.433" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Recommender_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Recommender_system&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:26:47.007" LastActivityDate="2019-05-17T20:26:47.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12383" PostTypeId="4" CreationDate="2019-05-17T16:48:10.433" Score="0" Body="For questions related to recommender systems in the context of machine learning and data mining." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:26:53.657" LastActivityDate="2019-05-17T20:26:53.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12384" PostTypeId="5" CreationDate="2019-05-17T16:50:16.293" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Naive_Bayes_classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Naive_Bayes_classifier&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:26:43.853" LastActivityDate="2019-05-17T20:26:43.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12385" PostTypeId="4" CreationDate="2019-05-17T16:50:16.293" Score="0" Body="For questions related to the naive Bayes, which is a machine learning (or statistics) technique that is based on the Bayes' theorem." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:00.050" LastActivityDate="2019-05-17T20:27:00.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12386" PostTypeId="5" CreationDate="2019-05-17T16:56:05.070" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-05-17T16:56:05.070" LastActivityDate="2019-05-17T16:56:05.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12387" PostTypeId="4" CreationDate="2019-05-17T16:56:05.070" Score="0" Body="For questions related to the liquid state machine model, which is a spiking neural network, which is a neural network that more closely mimics a biological neural network (with respect to traditional neural networks, like multi-layer perceptrons)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-17T20:27:33.933" LastActivityDate="2019-05-17T20:27:33.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12388" PostTypeId="1" CreationDate="2019-05-17T17:05:22.153" Score="4" ViewCount="54" Body="&lt;p&gt;I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm having trouble in regards to reward shaping. I've tried multiple different reward functions, but they all end up the same way: the agent quickly learns to sit (i.e. lifting its torso), but then gets stuck on this local optimum forever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas or advice on how to best design a good reward function for this scenario?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few reward functions I've tried so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;current_height / goal_height&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;current_height / goal_height - 1&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;current_height / goal_height - reward_prev_timestep&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;(current_height / goal_height)^N (tried multiple different values of N)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25712" LastEditorUserId="2444" LastEditDate="2020-10-07T17:22:44.487" LastActivityDate="2020-10-07T17:22:44.487" Title="How define a reward function for a humanoid agent whose goal is to stand up from the ground?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;reward-shaping&gt;&lt;reward-design&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12389" PostTypeId="2" ParentId="12343" CreationDate="2019-05-17T17:20:40.760" Score="2" Body="&lt;p&gt;If I understand correctly you want to find companies with similar patterns to yours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start with measuring &lt;code&gt;cosine similarity&lt;/code&gt; between your company and others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is really easy with Python, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [21]: from sklearn.metrics.pairwise import cosine_similarity&#xA;&#xA;In [22]: cosine_similarity([[1,4,2,6], [1,9,5,4]])&#xA;Out[22]:&#xA;array([[1.        , 0.84794633],&#xA;       [0.84794633, 1.        ]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that if size of sale matters to you, this is not the right approach, as cosine similarity is magnitude invariant:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [23]: from sklearn.metrics.pairwise import cosine_similarity&#xA;&#xA;In [24]: cosine_similarity([[1,4,2,6], [10,90,50,40]])&#xA;Out[24]:&#xA;array([[1.        , 0.84794633],&#xA;       [0.84794633, 1.        ]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25248" LastEditorUserId="25248" LastEditDate="2019-09-18T11:35:32.653" LastActivityDate="2019-09-18T11:35:32.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12390" PostTypeId="1" CreationDate="2019-05-17T18:24:30.060" Score="5" ViewCount="529" Body="&lt;p&gt;I am new in the field of AI. I am working to create the flappy bird using Genetic Algorithm. After reading and seeing some examples, I saw that most implementations use a Neural Network + Genetic Algorithm and after certain generations, you achieve a very good agent that survives very long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently struggling to implement the Neural Network since I have never taken a Machine Learning course.&#xA;On many examples that I have read neural networks require training inputs and outputs. For the flappy bird, I can't think of output since you don't really know if the action of flapping will benefit you or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example that I followed, Synaptic.js is used and it is pretty straight-forward. However, in Python, I can't find a simple library that will initialize randomly and adjust the weights and biases depending on the good agents that survive longer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the right way to implement this Neural Network without having a training dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to create Flappy Bird without using Neural Networks, just Genetic Algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example in Javascript that I am referring to: &lt;a href=&quot;https://github.com/ssusnic/Machine-Learning-Flappy-Bird&quot; rel=&quot;noreferrer&quot;&gt;Flappy Bird Using Machine Learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25714" LastActivityDate="2020-04-07T09:03:59.717" Title="How to implement a neural network for Flappy Bird in Python?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;genetic-algorithms&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12392" PostTypeId="5" CreationDate="2019-05-18T00:01:21.623" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-05-18T00:01:21.623" LastActivityDate="2019-05-18T00:01:21.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12393" PostTypeId="4" CreationDate="2019-05-18T00:01:21.623" Score="0" Body="For questions related to regression (both linear and non-linear) in the context of machine learning and AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-19T17:05:46.393" LastActivityDate="2019-05-19T17:05:46.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12396" PostTypeId="1" CreationDate="2019-05-18T08:57:10.483" Score="3" ViewCount="356" Body="&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&lt;br&gt;&#xA;I've been reading research papers on how to solve a peg solitaire using graph searching, but all the papers kind of assume you know how to do the reduction(polynomial time conversion) from the peg solitaire to the graph, which I do not, but this is how I assumed it was done.&lt;br&gt;&#xA;For those of you unfamiliar:&lt;br&gt;&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=Bt6GpGvUNeQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=Bt6GpGvUNeQ&lt;/a&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/c9rQB.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c9rQB.jpg&quot; alt=&quot;Peg Solitaire&quot;&gt;&lt;/a&gt;&#xA;The goal is to only have one peg on the board and you get rid of pegs by jump one peg over another. A peg can only jump if the it's jumping into an empty space as shown in the picture above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I've tried:&lt;/strong&gt;&lt;br&gt;&#xA;I had the idea of converting the problem to a tree where each node represents the state after an action is taken and each edge represents the action taken. So the root node would the initial state which is the board shown above then it's children would be the state of the board after any of the possible legal actions that can be taken. So for example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YJ4BR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YJ4BR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Then the children of each of those nodes would be the possible moves for them and you can find a solution once you've reached depth 31 in the tree because there are 32 pegs and you win the game where there's only 1 left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the right approach? It feels a little too abstract because I'd have to represent the edges as peg moves, but that's weird cause they're usually numbers or constraint.&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2019-05-18T14:48:49.590" LastActivityDate="2019-06-17T16:02:55.317" Title="Using graph searching to solve peg solitaire?" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;graphs&gt;&lt;graph-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12397" PostTypeId="1" AcceptedAnswerId="17692" CreationDate="2019-05-18T09:41:11.200" Score="3" ViewCount="701" Body="&lt;p&gt;I've been doing some reading about GANs, and although I've seen several excellent examples of &lt;em&gt;implementations&lt;/em&gt;, the descriptions of why those patterns were chosen isn't clear to me in many cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a very high level, the purpose of the &lt;strong&gt;discriminator&lt;/strong&gt; in a GAN is establish a &lt;strong&gt;loss function&lt;/strong&gt; that can be used to train the &lt;strong&gt;generator&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ie. Given the random input to the generator, the discriminator should be able to return a probability of the result being a 'real' image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the discriminator is perfect the probability will always be zero, and the loss function will have no gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you iterate:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;generate random samples&lt;/li&gt;&#xA;&lt;li&gt;generate output from the generator&lt;/li&gt;&#xA;&lt;li&gt;evaluate the output using the discriminator&lt;/li&gt;&#xA;&lt;li&gt;train the generator&lt;/li&gt;&#xA;&lt;li&gt;update the discriminator to be more accurate by training it on samples from the real distribution and &lt;strong&gt;output from the generator&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem, and what I don't understand, is point 5 in the above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do you use the output of the generator?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I absolutely understand that you need to iterate on the accuracy of the discriminator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start with it needs to respond with a non-zero value for the effectively random output from the generator, and slowly it needs to converge towards correctly classifying images at 'real' or 'fake'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to achieve this we iterate, training the generator with images from the real distribution, pushing it towards accepting 'real' images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...and with the images from the generator; but I don't understand why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Effectively, you have a set of real images (eg. 5000 pictures of faces), that represent a sample from the latent space you want the GAN to converge on (eg. all pictures of faces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the argument goes: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the generator is trained iteratively closer and closer to &lt;strong&gt;generating images from the latent space&lt;/strong&gt;, the discriminator is iteratively trained to &lt;strong&gt;recognise from the latent space&lt;/strong&gt;, as though it had a &lt;em&gt;much larger sample size&lt;/em&gt; than the 5000 (or whatever) sample images you started with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...ok, but that's daft.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;whole point&lt;/em&gt; of DNN's is that given a sample you can train it to recognise images from the latent space the samples represent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never seen a DNN where the first step was 'augment your samples with extra procedurally generated fake samples'; the only reason to do this would be if you can only recognise samples in the input set, ie. your network is over-fitted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as a specific example, why can't you incrementally train the discriminator on samples of ('real' * epoch/iterations + 'noise' * 1 - epoch/iterations), where 'noise' is just a random input vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your discriminator will then &lt;em&gt;necessarily&lt;/em&gt; converge towards recognising real images, as well as offering a meaningful gradient to the generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What benefit does feeding the output of the generator in offer over this?&lt;/p&gt;&#xA;" OwnerUserId="25722" LastEditorUserId="25722" LastEditDate="2019-05-19T04:26:23.197" LastActivityDate="2020-01-27T09:46:41.667" Title="Why use the output of the generator to train the discriminator in a GAN?" Tags="&lt;generative-adversarial-networks&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12399" PostTypeId="2" ParentId="12370" CreationDate="2019-05-18T10:34:12.223" Score="1" Body="&lt;p&gt;Concerning &lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-fold Cross Validation&lt;/em&gt;, I like to think of it by considering two extremes you can do: &lt;em&gt;Leave-One-Out Cross-Validation&lt;/em&gt; where you leave one sample each time and train your model on the remaining &lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt;, and &lt;em&gt;2-fold Cross Validation&lt;/em&gt; at which you split your dataset in half and train (and validate) two models on two different halves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important aspect when choosing &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is a &lt;em&gt;bias-variance tradeoff&lt;/em&gt;. Note that in &lt;em&gt;LOOCV&lt;/em&gt; you train each model using almost as many samples as there are available (&lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt;), so the validation step should give you an &lt;em&gt;unbiased estimate&lt;/em&gt; of the real test error. However, each model in &lt;em&gt;LOOCV&lt;/em&gt; is trained each time on &lt;strong&gt;almost exactly same dataset!&lt;/strong&gt;. This has important consequences, since the output of each model is highly correlated with each other. Since the mean of highly correlated variables has big variance, &lt;em&gt;LOOCV&lt;/em&gt; will suffer from huge variance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, in &lt;em&gt;2-fold CV&lt;/em&gt; the models share no common samples, so they are not correlated and therefore their outputs have low variance. But since we train each model using only a half of available samples, the procedure will have a high bias (the estimated test error will be off from the true test error).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What to do in this scenario? Choose something in the middle. Usually &lt;span class=&quot;math-container&quot;&gt;$k=5$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k=10$&lt;/span&gt; should be a good choice.&lt;/p&gt;&#xA;" OwnerUserId="22835" LastActivityDate="2019-05-18T10:34:12.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12400" PostTypeId="2" ParentId="12396" CreationDate="2019-05-18T14:46:56.563" Score="2" Body="&lt;p&gt;Your approach seems reasonable to me. The edges do not necessarily have to be numbers, but, if you wish, you could also encode the actions as numbers. For example, the weight of an edge could represent the &quot;cost&quot; of the corresponding action. If there's no natural cost associated with an action, then you can add a unit cost for each action.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-18T15:59:15.077" LastActivityDate="2019-05-18T15:59:15.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12401" PostTypeId="2" ParentId="12397" CreationDate="2019-05-18T23:01:22.040" Score="2" Body="&lt;p&gt;The main reason that the discriminator is trained concurrently with the generator is to provide (at least in theory) a smooth and gradual learning signal for the generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we trained the discriminator on only the input data, then, assuming our training algorithm converges well, it should quickly converge to a fixed model. The generator can then learn to fool this fixed model, but it will likely still be easy to spot the generator's fakes for a human. For example, the discriminator may have learned that, coincidentally, in the sample you provided, all images of trucks have a fully white pixel in the top left corner. If it learns that pattern, the generator can fool it by generating noise with a white pixel. Once the generator has learned this pattern, all learning stops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, suppose that the discriminator is repeatedly re-trained on a mixture of real and generated examples. The discriminator will be forced to learn more complex patterns than &quot;white pixel in upper left&quot;, which improves its quality beyond the raw patterns in the sample data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The converse relationship is also true. If the generator is trained only on the training data, it will also likely pick out only the most obvious patterns. These patterns are likely to create many local minima in the weight space for the network. However, if the error signal from the discriminator is fed to the generator, then the generator must adapt: in effect, we are telling it &quot;making the top left pixel white is not good enough to fool observers. Find more complex patterns&quot;. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-05-18T23:01:22.040" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12402" PostTypeId="1" AcceptedAnswerId="12406" CreationDate="2019-05-18T23:07:43.073" Score="7" ViewCount="247" Body="&lt;p&gt;For discrete action spaces, what is the purpose of the actor in actor-critic algorithms? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current understanding is that the critic estimates the future reward given an action, so why not just take the action that maximizes the estimated return?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial guess at the answer is the exploration-exploitation problem, but are there other, more important/deeper reasons? Or am I underestimating the importance of exploration vs. exploitation as an issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It just seems to me that if you can accurately estimate the value function, then you have solved the RL challenge. &lt;/p&gt;&#xA;" OwnerUserId="25732" LastEditorUserId="2444" LastEditDate="2020-05-03T19:03:56.323" LastActivityDate="2020-05-03T19:03:56.323" Title="What is the purpose of the actor in actor-critic algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12404" PostTypeId="1" CreationDate="2019-05-19T04:27:59.313" Score="4" ViewCount="114" Body="&lt;p&gt;Update on 2019-05-19:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about teaching AI to solve the problem, not letting AI teach a human developer to solve a problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Original post:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a software developer but very new to AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today my friend and I chatted about the development of AI. One topic was about implementing the capability of &quot;given a problem, analyzing the problem and designing a solution&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we are both software developers, we used a simple example problem in our discussion to see how AI might possibly find a solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Print the following three lines on the console:&#xA;&#xA;*&#xA;***&#xA;*****&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My friend and I thought we may use some &lt;a href=&quot;https://en.wikipedia.org/wiki/Formal_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;formal method&lt;/a&gt; to describe &lt;strong&gt;WHAT&lt;/strong&gt; we want but &lt;strong&gt;NOT&lt;/strong&gt; how we implement it. It's the AI's job to figure out the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we came to the question I'm asking here: Since my friend and I are both outsiders of AI research, we don't know if there is any existing research (we believe such research must have existed somewhere) that teaches AI to &lt;strong&gt;analyze the problem (which is formally defined) and design a solution using the given tools&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For us human beings, our analysis of the problem and designing might look like the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Let me choose a programming language. For example, C.&lt;/li&gt;&#xA;&lt;li&gt;Let me see what tools I have in the chosen programming language. Oh, here they are:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;putchar(ch)&lt;/code&gt; which prints a single character on the console.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;printf(str)&lt;/code&gt; which prints a string on the console.&lt;/li&gt;&#xA;&lt;li&gt;for-loop; if-else; support of subroutines; etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;I see the result has three lines of characters: line 1, 2, and 3.&lt;/li&gt;&#xA;&lt;li&gt;I see the numbers of '*' in the three lines are an arithmetic progression and there is a connection of line number and character number: given the line number &lt;em&gt;i&lt;/em&gt;, the character number is &lt;em&gt;2*i-1&lt;/em&gt;, where &lt;em&gt;i&lt;/em&gt; is 1, 2, and 3. This is repetition and I can use a for-loop.&lt;/li&gt;&#xA;&lt;li&gt;Each line is the repetition of '*' so I may implement a function to do this.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void print_line(int N) {&#xA;  for (int i = 0; i &amp;lt; N; i++) {&#xA;    putchar('*');&#xA;  }&#xA;  putchar('\n');&#xA;}&#xA;&#xA;int main(int argc, char * argv[]) {&#xA;  for (int i = 1; i &amp;lt;=3; i++) {&#xA;    print_line(2 * i - 1);&#xA;  }&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, I may design a naive solution of using &lt;code&gt;printf()&lt;/code&gt; three times and hard-code each string:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;printf(&quot;*\n&quot;);&#xA;printf(&quot;***\n&quot;);&#xA;printf(&quot;*****\n&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We think an AI that can do this may follow a similar analyzing and designing approach as a human developer does. In general, we think:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This AI should have a toolbox using which it can solve some problems (possibly not all problems). In my example above, this toolbox may be a programming language and its corresponding library.&lt;/li&gt;&#xA;&lt;li&gt;This AI should have the knowledge about some concepts (such as &lt;em&gt;console&lt;/em&gt; and &lt;em&gt;string&lt;/em&gt; in the example above) and their relationships.&lt;/li&gt;&#xA;&lt;li&gt;This AI should have the knowledge that connects the toolbox and the concepts, so the AI knows how a tool can manipulate the properties of a concept.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Most importantly&lt;/strong&gt;, this AI should have the capability of figuring out one or more paths that connect the input to the desired output, using the toolbox. This process, we think, needs the capability of &quot;analysis&quot; and &quot;design&quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Excuse us if the description is still vague. My friend and I are both new to AI so, in fact, we don't even know if &quot;analysis&quot; and &quot;design&quot; are the proper words to use. We will be glad to clarify if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, we did some quick search about such AI:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://info.askbayou.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayou by Rice University&lt;/a&gt; doesn't look like understanding the problem, either.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.microsoft.com/en-us/research/publication/deepcoder-learning-write-programs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepCoder&lt;/a&gt; uses Deep Learning and I doubt whether it understands the problem, either.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://github.com/primaryobjects/AI-Programmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-Programmer&lt;/a&gt; uses genetic algorithms to generate the desired string in BrainFuck. But this AI doesn't look like understanding the problem. It looks like a trial-and-error with feedback.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4817" LastEditorUserId="4817" LastEditDate="2020-03-16T22:53:34.600" LastActivityDate="2020-04-16T01:01:50.377" Title="Is there research about teaching AI to &quot;analyze the problem and design a solution&quot;?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;research&gt;&lt;human-like&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12406" PostTypeId="2" ParentId="12402" CreationDate="2019-05-19T08:13:29.037" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For discrete action spaces, what is the purpose of the actor in Actor-Critic algorithms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In brief, it is the policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;. The critic (a state action function &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;) is not used to derive a policy, and in &quot;vanilla&quot; Actor-Critic cannot be used in this way at all unless you have the full distribution model of the MDP.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It just seems to me that if you can accurately estimate the value function, then you have solved the RL challenge.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Often this can be the case, and this is how e.g. Q-learning works, where the value function is more precisely the action value function, &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuous or very large action spaces can cause a problem here, in that maximising over them is impractical. If you have a problem like that to solve, it is often an indicator that you should use a policy gradient method such as Actor-Critic. However, you have explicitly asked about &quot;discrete action spaces&quot; here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue with your statement &quot;if you can accurately estimate the value function&quot; is that you have implicitly assumed the the learning is complete. You are looking at the final output of the algorithm, after it has converged, and asking why not use just half of its output. Whilst choosing between RL algorithms is more often related to &lt;em&gt;how&lt;/em&gt; they learn, how efficiently they learn, and how they behave during the learning process.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My current understanding is that the critic estimates the future reward given an action, so why not just take the action that maximizes the estimated return?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The critic is &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; so does not take actions into account. You can of course learn &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; instead or as well - and some policy gradient methods &lt;em&gt;also&lt;/em&gt; estimate the action value, such as Advantage Actor-Critic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference is in how data gathered from experience is applied to changing the policy. Recall at any step during reinforcement learning, that all estimates are based around a best guess at values of a &lt;em&gt;current&lt;/em&gt; target policy. Policy gradient methods directly improve the policy, whilst value-based methods have an &lt;em&gt;implied&lt;/em&gt; policy based on the same learned value functions. The &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}_a q(s,a)$&lt;/span&gt; approach used in Q-learning or SARSA gives a simple but crude mapping from values to policy, and means that the trajectory through policy space taken whilst learning is different when comparing policy gradient methods and value-based methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following are notable differences affecting performance between policy-based methods and value-based methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Policy-gradient methods tend to make small changes to policy on each step, the policy changes smoothly, adjusting probabilities between choice of different actions by small amounts. This in turn means that the targets for estimated value functions change slowly. In comparison, when a value based method finds a new maximising action, it can make a big change to the policy, making the updates required less smooth.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Policy-gradient methods can learn to balance stochastic policies, whilst value-based methods assume deterministic policies. So if you have a situation like paper, scissor stone game where the ideal policy is random choice between actions, you should use policy gradients. This is a bit niche, but is worth noting.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Function approximators (typically neural networks) are easier to train on simpler target functions. The policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and action value function &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; are quite different views of the same problem, and it can be the case that one has a simple form whilst the other is more complex. This can make a big practical difference on which algorithms learn fastest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For smaller, discrete action spaces, there is not always a clear choice between policy gradient and value-based methods. You could use either and it may be worth doing an experiment to find the most efficient algorithm for certain classes of problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-19T15:28:39.503" LastActivityDate="2019-05-19T15:28:39.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12408" PostTypeId="1" CreationDate="2019-05-19T13:28:32.363" Score="1" ViewCount="186" Body="&lt;p&gt;I asked a &lt;a href=&quot;https://ai.stackexchange.com/questions/11286/importance-of-starting-state-and-player-in-rl-for-tic-tac-toe&quot;&gt;question&lt;/a&gt; relating to TicTacToe playing in RL. From the answer it seems to me a lot is dependent on the opponent (rightly so, if we write down the Expectation Equations). My question is (in the context of TicTacToe or Chess):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How to make the RL player a perfect player (TicTactoe) or an expert (chess). As far as TTT is concerned when playing against a perfect player an RL will become perfect conditioned on the clause that the opponent is perfect. So will this hold true if the same RL algo, with its learned values are used to play some other lesser perfect players?&lt;/li&gt;&#xA;&lt;li&gt;The question can be extended to the scenario, can a RL player with pre-trained values (assume from a perfect or expert opponent) be used in any scenario with best results?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; The problem is more severe in chess, since experts will use some kind of opening moves which will not match with say a random player and thus finding values for those states becomes a problem, since we have not encountered it during training time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Footnote:&lt;/strong&gt; Any resources on Game Playing RL is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-05-19T14:37:28.403" Title="Adversarial Game playing using RL" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12409" PostTypeId="2" ParentId="12408" CreationDate="2019-05-19T14:37:28.403" Score="1" Body="&lt;p&gt;I would say a good way to make a good agent would be making it play against itself. As you go through several episodes, with a good exploration and exploitation balance, both will gradually learn and converge to Q*(s,a).´&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So will this hold true if the same RL algo, with its learned values&#xA;  are used to play some other lesser perfect players?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As long as the states that are played  (or approximations if you are using Function approximation methods) were simulated enough times during  training, it will play well against any kind of opponent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are training against a completely perfect opponent and you are not using function approximation, I believe you could get to an incomplete Q(s,a) table and as such not be able to predict the best play when facing certain states.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2019-05-19T14:37:28.403" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12410" PostTypeId="1" CreationDate="2019-05-19T15:54:27.450" Score="1" ViewCount="205" Body="&lt;p&gt;I'm having serious issues with the implementation of the LRP algorithm for neural networks in MATLAB. The challenge is to implement the equations correctly. I'm trying to implement the deep-Taylor &lt;span class=&quot;math-container&quot;&gt;$\alpha_1\beta_0$&lt;/span&gt; version of the LRP. I'm testing it on a feed-forward full-connected neural network with one hidden layer trained on the MNIST dataset. I start initializing the relevances of the last layer with the predictions of the network:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R{l} = nn.layers{nn.lastLayer}.output;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, i need to perform the following equation through the hidden layers (in my case is just one):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$R_i^l = \sum_j \frac{a_i^l \cdot w_{ij}^{+}}{\sum_k a_j^{l+1} \cdot w_{kj}^{+}} \cdot R_j^{l+1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to a paper, i've followed some suggestions and correctly implemented that equation as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for l = nn.numOfLayers-1 : -1 : 1&#xA;        z = max(nn.layers{l+1}.W, 0) * nn.layers{l}.output;&#xA;        s = R{l+1} ./ z;&#xA;        c = (max(nn.layers{l+1}.W, 0))' * s;&#xA;        R{l} = nn.layers{l}.output .* c;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I hope the syntax is clear, &quot;nn&quot; is the neural network struct and other fields should be intuitive to understand. The elements are columns, it means that outputs &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$numberOfNeurons \times 1$&lt;/span&gt;. The weights are matrices such that on the rows there are the neurons of the next layer, and on the columns the neurons of the current layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my problem is, how do i compute the relevances for the first layer? (input layer) Because, in the input layer i haven't the &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; values (that are the outputs transformed by the activation function). So i've looked also in a paper and found that the following equation can be used to compute the relevances of the first layer (the layer of interest):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R_i^1 = \sum_j \frac{w_{ij}^{2}}{\sum_k w_{kj}^{2}} \cdot R_j^{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried different ways to implement that in matlab, but the results seems wrong. For instance, i've tried with the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i = 1 : inputSize&#xA;            scalar = 0;&#xA;            for j = 1 : nn.layers{1}.layerSize&#xA;                norm = 0;&#xA;                for k = 1 : inputSize&#xA;                    norm = norm + nn.layers{1}.W(j,k)^2;&#xA;                end&#xA;                scalar = (nn.layers{1}.W(j,i)^2 * R{1}(j))/norm;&#xA;                Rfin(i) = Rfin(i) + scalar;&#xA;            end&#xA;        end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then return &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. These are the results i should see:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nuQZW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nuQZW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is an example of what i get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2vhWr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2vhWr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know what am i doing wrong? Do you have ideas on how to implement correctly in MATLAB the second equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="25751" LastActivityDate="2020-03-07T21:49:47.407" Title="Difficulties to implement the layer-wise relevance propagation in MATLAB" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12411" PostTypeId="1" CreationDate="2019-05-19T19:52:06.153" Score="4" ViewCount="87" Body="&lt;p&gt;I've been studying Branch and Bound's graph algorithm and I hear it always finds the optimal path because it uses previously found solutions to find others&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I haven't been able to find a proof of why it finds the optimal path. (In fact, most sites kind of do a bad job generalizing the algorithm itself.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the proof that this algorithm always finds the optimal path in the case of a graph with 1 or more goal nodes?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2020-05-26T16:43:19.510" LastActivityDate="2021-02-20T18:04:50.503" Title="What is the proof that the branch and bound algorithm always finds optimal path in a graph?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;graphs&gt;&lt;branch-and-bound&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12412" PostTypeId="1" CreationDate="2019-05-19T20:10:32.770" Score="2" ViewCount="30" Body="&lt;p&gt;I appreciate that there are many ways to arrange the memory in a NN, and that the numerical representations may be with bytes to floats depending on an implementation.  What is typical amount of memory required for the &quot;application side&quot; for the better NN programs such as Alpha Zero or an automated driving AI?  How much does it matter?&lt;/p&gt;&#xA;" OwnerUserId="25756" LastActivityDate="2019-05-19T20:10:32.770" Title="How much Physical memory does Alpha Zero's Neural Net require?" Tags="&lt;neural-networks&gt;&lt;memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12413" PostTypeId="2" ParentId="12411" CreationDate="2019-05-19T21:13:59.753" Score="-1" Body="&lt;p&gt;My attempt was proof by contradiction. We can assume B&amp;amp;B found a sub-optimal path, but that would create a contradiction, because the only way B&amp;amp;B would miss an optimal path is if it skipped it completely (this part I don't know how to prove) or the related part of the search space was pruned.&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2019-05-19T22:20:44.360" LastActivityDate="2019-05-19T22:20:44.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12414" PostTypeId="1" CreationDate="2019-05-19T21:15:24.323" Score="1" ViewCount="198" Body="&lt;p&gt;I have a &lt;code&gt;.csv&lt;/code&gt; file called &lt;code&gt;ratings.csv&lt;/code&gt; with the following structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;userID, movieID, rating&#xA;3,      12,      5&#xA;2,      7,       6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rating scale goes from 0 to 5 stars. I want to be able to plot the sparsity of the matrix like it's done in the following picture: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jtSLT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jtSLT.jpg&quot; alt=&quot;pictiure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, ratings scale goes from 0 to 5 on the right. It is a very well thought plot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have Matlab, Python, R etc. Could you come up with something and help me? I’ve tried hard but I cannot find the way to do it.&lt;/p&gt;&#xA;" OwnerUserId="25405" LastEditorUserId="2444" LastEditDate="2019-05-20T14:03:41.343" LastActivityDate="2019-05-20T14:03:41.343" Title="How do I plot a matrix of ratings?" Tags="&lt;machine-learning&gt;&lt;recommender-system&gt;&lt;data-visualization&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12415" PostTypeId="1" CreationDate="2019-05-19T22:26:41.723" Score="3" ViewCount="37" Body="&lt;p&gt;Can &lt;span class=&quot;math-container&quot;&gt;$\Phi$&lt;/span&gt; measure (computed rigorously or approximately) of &lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004654&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integrated Information Theory&lt;/a&gt; serve as reward for self-evolving/learning reinforcement learning system and hence we let this system to become/evolve into conscious system?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-05-19T22:39:26.757" LastActivityDate="2019-05-19T22:39:26.757" Title="Can $\Phi$ measure of Integrated Information Theory serve as reward for reinforcement learning system?" Tags="&lt;reinforcement-learning&gt;&lt;artificial-consciousness&gt;&lt;integration&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12416" PostTypeId="2" ParentId="12271" CreationDate="2019-05-19T22:28:22.290" Score="3" Body="&lt;p&gt;Using your app, I was able to find a (spoiler alert!) &lt;a href=&quot;https://imgur.com/FUBfstq&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; manually. At least now you know your puzzle is solvable and you did not waste your money :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems your app has a bug, though. I was unable to put the last piece, as shown in the picture. I was wondering if your solver, as it stands, will ever find a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the idea. It may be useful for your solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The board has 8x8=64 squares. Each piece will occupy 5 squares and you want to fit 11 pieces, so the final position will have 9 empty squares. Divide the board in two 8x4 rectangles, left and right. Now, it seems only fair that one rectangle should have 5 empty squares leaving the other with 4; with that in mind I've proceeded to fill the right part first and only then the left. After some trial and error, I got lucky.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if you can reach all solutions with this method. Notice that, in the solution given above, the bottom rectangle will end up with 6 empty squares. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to write an efficient solver either. For starters:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Build up a list of bad configurations: small sets of pieces/positions such that whenever you reach them, you know there is no hope.&lt;/li&gt;&#xA;&lt;li&gt;At each step, put a piece in such way it minimizes the number of forced empty squares. &lt;/li&gt;&#xA;&lt;li&gt;A variant of the idea above: divide the board in four 4x4 parts; it seems reasonable each part should have at least one empty square but no so many; so, look for solutions forcing these parts to have (1,2,3,3) or (2,2,2,3) empty squares. &lt;/li&gt;&#xA;&lt;li&gt;Lookup whether this puzzle is known. Names that come to mind: Martin Gardner, Ian Stewart, Sam Lloyd. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can't say you will ever be able to see a list of all possible solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nice puzzle, nice app that one you wrote, I've had a good time. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="25759" LastEditorUserId="25759" LastEditDate="2019-05-20T00:18:34.183" LastActivityDate="2019-05-20T00:18:34.183" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12417" PostTypeId="2" ParentId="12414" CreationDate="2019-05-19T22:46:45.390" Score="2" Body="&lt;p&gt;You're looking for a &lt;a href=&quot;https://en.wikipedia.org/wiki/Heat_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;heatmap&lt;/a&gt;. Check out e.g. &lt;a href=&quot;https://stackoverflow.com/q/33282368/3924118&quot;&gt;https://stackoverflow.com/q/33282368/3924118&lt;/a&gt; (if you like Python more than the others). See also &lt;a href=&quot;https://seaborn.pydata.org/generated/seaborn.heatmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-20T14:02:52.760" LastActivityDate="2019-05-20T14:02:52.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12418" PostTypeId="1" AcceptedAnswerId="12445" CreationDate="2019-05-20T01:40:49.383" Score="4" ViewCount="175" Body="&lt;p&gt;I've been trying to learn backpropagation for CNNs. I read several articles like &lt;a href=&quot;https://medium.com/@pavisj/convolutions-and-backpropagations-46026a8f5d2c&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; and &lt;a href=&quot;https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;. They all say that to compute the gradients for the filters, you just do a convolution with the input volume as input and the error matrix as the kernel. After that, you just subtract the filter weights by the gradients(multiplied by the learning rate). I implemented this process but it's not working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a simple example that I tried: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input volume (randomised)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1 -1  0&#xA; 0  1  0&#xA; 0 -1  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, we want the filter to only pick up the top left 4 elements. So the target output will be: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1  0(supposed to be -1, but ReLU is applied) &#xA; 0  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We know that the desired filter is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1  0&#xA; 0  0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But we pretend that we don't know this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We first randomise a filter: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1 -1&#xA; 1  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output right now is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 3  0 &#xA;-2  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Apply ReLU: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 3  0&#xA; 0  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Error (target - output): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-2  0&#xA; 0  0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use error as kernel to compute gradients: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-2  2&#xA; 0 -2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Say the learning rate is 0.5, then the new filter is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2 -2&#xA;1  2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is still wrong! It's not improving at all. If this process is repeated, it won't learn the desired filter. So I must have understood the math wrong. So what's the problem here? &lt;/p&gt;&#xA;" OwnerUserId="25745" LastEditorUserId="25745" LastEditDate="2019-05-21T06:05:06.607" LastActivityDate="2019-05-23T01:05:26.910" Title="How are filters weights updated for a CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12419" PostTypeId="2" ParentId="12315" CreationDate="2019-05-20T01:47:44.517" Score="0" Body="&lt;p&gt;In deep learning, it is very common to use Recurrent Neural Networks (RNNs) to handle time-series data with varying input sequence lengths. Check out the &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNN Wikipedia&lt;/a&gt; for more detail.&lt;/p&gt;&#xA;" OwnerUserId="25732" LastActivityDate="2019-05-20T01:47:44.517" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12420" PostTypeId="2" ParentId="12414" CreationDate="2019-05-20T03:34:29.500" Score="1" Body="&lt;p&gt;I did it!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = importdata('u.data');&#xA;user_id = A(:, 1);&#xA;movie_id = A(:, 2);&#xA;rating = A(:, 3);&#xA;&#xA;% Build matrix R and w (weights matrix)&#xA;R = zeros(943, 1682);&#xA;w = zeros(943, 1682);&#xA;for i=1:100000&#xA;    R(user_id(i), movie_id(i)) = rating(i);&#xA;    w(user_id(i), movie_id(i)) = 1;&#xA;end&#xA;&#xA;&#xA;m = HeatMap(R)&#xA;ax = hm.plot; % 'ax' will be a handle to a standard MATLAB axes.&#xA;colorbar('Peer', ax); % Turn the colorbar on&#xA;caxis(ax, [0 5]); % Adjust the color limits&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qn3yn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qn3yn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25405" LastActivityDate="2019-05-20T03:34:29.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12421" PostTypeId="1" AcceptedAnswerId="12432" CreationDate="2019-05-20T05:58:09.850" Score="5" ViewCount="634" Body="&lt;p&gt;I am trying to solve for &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; using &lt;a href=&quot;https://en.wikipedia.org/wiki/Temporal_difference_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;temporal-difference learning&lt;/a&gt;. More specifically, I am trying to figure out what &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; I need, such that &lt;span class=&quot;math-container&quot;&gt;$\text{TD}(\lambda)=\text{TD}(1)$&lt;/span&gt;, after one iteration. But I get the incorrect value of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my implementation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.optimize import fsolve,leastsq&#xA;import numpy as np&#xA;&#xA;&#xA;&#xA; class TD_lambda:&#xA;        def __init__(self, probToState,valueEstimates,rewards):&#xA;            self.probToState = probToState&#xA;            self.valueEstimates = valueEstimates&#xA;            self.rewards = rewards&#xA;            self.td1 = self.get_vs0(1)&#xA;&#xA;        def get_vs0(self,lambda_):&#xA;            probToState = self.probToState&#xA;            valueEstimates = self.valueEstimates&#xA;            rewards = self.rewards&#xA;            vs = dict(zip(['vs0','vs1','vs2','vs3','vs4','vs5','vs6'],list(valueEstimates)))&#xA;&#xA;            vs5 = vs['vs5'] + 1*(rewards[6]+1*vs['vs6']-vs['vs5'])&#xA;            vs4 = vs['vs4'] + 1*(rewards[5]+lambda_*rewards[6]+lambda_*vs['vs6']+(1-lambda_)*vs['vs5']-vs['vs4'])&#xA;            vs3 = vs['vs3'] + 1*(rewards[4]+lambda_*rewards[5]+lambda_**2*rewards[6]+lambda_**2*vs['vs6']+lambda_*(1-lambda_)*vs['vs5']+(1-lambda_)*vs['vs4']-vs['vs3'])&#xA;            vs1 = vs['vs1'] + 1*(rewards[2]+lambda_*rewards[4]+lambda_**2*rewards[5]+lambda_**3*rewards[6]+lambda_**3*vs['vs6']+lambda_**2*(1-lambda_)*vs['vs5']+lambda_*(1-lambda_)*vs['vs4']+\&#xA;                                (1-lambda_)*vs['vs3']-vs['vs1'])&#xA;            vs2 = vs['vs2'] + 1*(rewards[3]+lambda_*rewards[4]+lambda_**2*rewards[5]+lambda_**3*rewards[6]+lambda_**3*vs['vs6']+lambda_**2*(1-lambda_)*vs['vs5']+lambda_*(1-lambda_)*vs['vs4']+\&#xA;                                (1-lambda_)*vs['vs3']-vs['vs2'])&#xA;            vs0 = vs['vs0'] + probToState*(rewards[0]+lambda_*rewards[2]+lambda_**2*rewards[4]+lambda_**3*rewards[5]+lambda_**4*rewards[6]+lambda_**4*vs['vs6']+lambda_**3*(1-lambda_)*vs['vs5']+\&#xA;                                        +lambda_**2*(1-lambda_)*vs['vs4']+lambda_*(1-lambda_)*vs['vs3']+(1-lambda_)*vs['vs1']-vs['vs0']) +\&#xA;                    (1-probToState)*(rewards[1]+lambda_*rewards[3]+lambda_**2*rewards[4]+lambda_**3*rewards[5]+lambda_**4*rewards[6]+lambda_**4*vs['vs6']+lambda_**3*(1-lambda_)*vs['vs5']+\&#xA;                                        +lambda_**2*(1-lambda_)*vs['vs4']+lambda_*(1-lambda_)*vs['vs3']+(1-lambda_)*vs['vs2']-vs['vs0'])&#xA;            return vs0&#xA;&#xA;        def get_lambda(self,x0=np.linspace(0.1,1,10)):&#xA;            return fsolve(lambda lambda_:self.get_vs0(lambda_)-self.td1, x0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The expected output is: &lt;span class=&quot;math-container&quot;&gt;$0.20550275877409016$&lt;/span&gt;, but I am getting &lt;code&gt;array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot understand what am I doing incorrectly.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TD = TD_lambda(probToState,valueEstimates,rewards)&#xA;TD.get_lambda()&#xA;# Output : array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am just using TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) for state 0 after one iteration. I am not required to see where it converges, so I don't update the value estimates.&lt;/p&gt;&#xA;" OwnerUserId="25768" LastEditorUserId="2444" LastEditDate="2020-06-04T17:05:30.827" LastActivityDate="2020-06-04T17:05:30.827" Title="Why am I getting the incorrect value of lambda?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;markov-decision-process&gt;&lt;temporal-difference-methods&gt;&lt;td-lambda&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12422" PostTypeId="2" ParentId="3343" CreationDate="2019-05-20T07:28:34.753" Score="0" Body="&lt;p&gt;﻿AbuShawar &amp;amp; Atwell state:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A chatbot is a conversational agent that interacts with the users turn by turn using natural language. Different chatbots or human-computer dialogue systems have been developed using spoken or text communication and have been applied in different domains such as: Linguistic research, language education, customer service, web site help, and for fun.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Theirs and other papers convey some of the many contemporary approaches to chatbot training as of this writing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/cf51/309780bf46a9c6aa49678288436a88880511.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Automatic Extraction of Chatbot Training Data from Natural Dialogue Corpora&lt;/em&gt;&lt;/a&gt;, Bayan AbuShawar, Eric Atwell, 2016&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, most chatbots are restricted to knowledge that is manually in their files, and to a specific natural language which is written or spoken.&#xA;  This paper presents the program we developed to convert a machine readable text (corpus) to a specific chatbot format,&#xA;  which is then used to retrain a chatbot and generate a chat which is closer to human language.&#xA;  Different corpora were used: dialogue corpora such as the British National Corpus of English (BNC);&#xA;  the holy book of Islam Quran which is a monologue corpus where verse and following verse are turns; and the FAQ where questions and answers are pair of turns.&#xA;  The main goal of this automation process is the ability to generate different chatbot prototypes that spoke different languages based on corpus. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ruizhang.info/publications/PAKDD2018_Context-Uncertainty-Aware%20Chatbot.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Context-Uncertainty-Aware Chatbot Action Selection via Parameterized Auxiliary Reinforcement Learning&lt;/em&gt;&lt;/a&gt;, Chuandong Yin, Rui Zhang, Jianzhong Qi, Yu Sun, and Tenglun Tan, 2018&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We propose a context-uncertainty-aware chatbot and a reinforcement learning (RL) model to train the chatbot. The proposed model&#xA;  is named Parameterized Auxiliary Asynchronous Advantage Actor Critic&#xA;  (PA4C). We utilize a user simulator to simulate the uncertainty of users utterance confidence in a&#xA;  conversation context. Compared with naive rule-based approaches, our&#xA;  chatbot trained via the PA4C model avoids hand-crafted action selection&#xA;  and is more robust to user utterance variance. The PA4C model optimizes conventional RL models with action parameterization and auxiliary tasks for chatbot training, which address the problems of a large&#xA;  action space and zero-reward states. We evaluate the PA4C model over&#xA;  training a chatbot for calendar event creation tasks. Experimental results show that our model outperforms the state-of-the-art RL models&#xA;  in terms of success rate, dialogue length, and episode reward.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://patentimages.storage.googleapis.com/a0/39/78/44f84fbda71d34/US20190034828A1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Supervised Learning System Training Using Chatbot Interaction&lt;/em&gt;&lt;/a&gt;, United States Patent Application Publication 0034828 A1, International Business Machines Corporation, Armonk, NY, US, 2019&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A computer - implemented method comprising receiving and analyzing a data point to determine parameters of the data point,&#xA;  generating an alert ticket based on the analysis of the data point, communicating, via a chatbot, at least some information contained in the alert ticket to one or more users,&#xA;  and categorizing, via the chatbot, the data point that resulted in the alert ticket based on behavior of a device that generated the data point.&#xA;  Jonathan A . Cagadas, Alexander D. Lewitt, Simon D. Mikulcik, Karan Shukla, Leigh A. Williamson&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.aclweb.org/anthology/W18-6707&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Two-Step Training and Mixed Encoding-Decoding for Implementing a Generative Chatbot with a Small Dialogue Corpus&lt;/em&gt;&lt;/a&gt;, Jintae Kim, Hyeon-Gu Lee, Harksoo Kim, Yeonsoo Lee, Young-Gil Kim, 2016&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Generative chatbot models based on sequence-to-sequence networks can generate natural conversation interactions if a huge dialogue corpus is used as training data.&#xA;  However, except for a few languages such as English and Chinese, it remains difficult to collect a large dialogue corpus.&#xA;  To address this problem, we propose a chatbot model using a mixture of words and syllables as encoding-decoding units.&#xA;  In addition, we propose a two-step training method, involving pre-training using a large non-dialogue corpus and re-training using a small dialogue corpus.&#xA;  In our experiments, the mixture units were shown to help reduce out-of-vocabulary (OOV) problems.&#xA;  Moreover, the two-step training method was effective in reducing grammatical and semantical errors in responses when the chatbot was trained using a small dialogue corpus (533,997 sentence pairs).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1802.00757.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Submodularity-Inspired Data Selection for Goal-Oriented Chatbot Training Based on Sentence Embeddings&lt;/em&gt;&lt;/a&gt;, Mladen Dimovski, Claudiu Musat, Vladimir Ilievski, Andreea Hossmann, Michael Baeriswyl, 2018&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Spoken language understanding (SLU) systems,&#xA;  such as goal-oriented chatbots or personal assistants, rely on an initial natural language understanding (NLU) module to determine the intent and&#xA;  to extract the relevant information from the user&#xA;  queries they take as input. SLU systems usually&#xA;  help users to solve problems in relatively narrow&#xA;  domains and require a large amount of in-domain&#xA;  training data. This leads to significant data availability issues that inhibit the development of successful systems. To alleviate this problem, we propose a technique of data selection in the low-data&#xA;  regime that enables us to train with fewer labeled&#xA;  sentences, thus smaller labelling costs.&#xA;  We propose a submodularity-inspired data ranking function, the ratio-penalty marginal gain, for&#xA;  selecting data points to label based only on the&#xA;  information extracted from the textual embedding&#xA;  space. We show that the distances in the embedding&#xA;  space are a viable source of information that can be&#xA;  used for data selection. Our method outperforms&#xA;  two known active learning techniques and enables&#xA;  cost-efficient training of the NLU unit. Moreover,&#xA;  our proposed selection technique does not need&#xA;  the model to be retrained in between the selection&#xA;  steps, making it time efficient as well.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-05-20T07:28:34.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12423" PostTypeId="1" CreationDate="2019-05-20T09:00:52.617" Score="1" ViewCount="55" Body="&lt;p&gt;I have a fairly large dataset consisting of different images with and without persons that I want to use for a project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I only want the pictures that contain faces, and it is best if there is only a crop of the face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already looked at Facenet and Openface, but I thought that there must be a simpler already trained solution just to sort the dataset so I can get started with my own project.&lt;/p&gt;&#xA;" OwnerUserId="25770" LastEditorUserId="2444" LastEditDate="2019-06-28T11:02:27.577" LastActivityDate="2020-03-06T07:55:23.107" Title="Are there tools to help clean a large dataset so that it only contains faces?" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12424" PostTypeId="2" ParentId="12360" CreationDate="2019-05-20T09:53:01.590" Score="1" Body="&lt;p&gt;In an RNN, the output of the previous state is passed as an input to the current state. Intuitively, there is a temporal (time-based) relationship in the way in which input is processed in an RNN. It can understand how the current state was achieved on the basis of the previous values, i.e value at time-step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is a result of value at time-steps &lt;span class=&quot;math-container&quot;&gt;$t-1, t-2$&lt;/span&gt;, and so on.&lt;/p&gt;&#xA;&lt;p&gt;In a DNN, there is no temporal relationship in the way input is processed. Values at time-steps &lt;span class=&quot;math-container&quot;&gt;$t, t-1, t-2, \dots $&lt;/span&gt; are all treated distinctly and not as a continuation of the previous time-step values.&lt;/p&gt;&#xA;" OwnerUserId="25704" LastEditorUserId="2444" LastEditDate="2020-10-02T19:22:16.633" LastActivityDate="2020-10-02T19:22:16.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12425" PostTypeId="2" ParentId="7397" CreationDate="2019-05-20T10:35:54.320" Score="0" Body="&lt;p&gt;Let me give an example of where epsilon-greedy comes unstuck: Imagine you have a environment with a very large branching-factor, like Go; if you were using epsilon-greedy for your exploration then you may find levels higher up the search tree are very well explored because they're hit on more regularly and so you'd want to select more greedily for those areas that are well explored, but further down the tree where actions are not explored you'd want to encourage more level of random exploration. Epsilon-greedy doesn't enable you to do that; it's one probability for all situations.  So it's fine to use where the state-space is quite small, but not for large state-spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actor-critic methods, such as PPO, use the entropy (the measure of randomness) of the policy to form part of the loss function which inherently encourages exploration. To elaborate on this; early in training you'd expect high levels of entropy as all actions may have near-equal probability of being selected, but as the model explores the actions and garners rewards, it will gradually favour taking the actions which lead to higher rewards, and so the entropy will decrease as training progresses and it gradually starts to behave more greedily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the entropy is added to the loss (or reward in case of RL, i.e. negative loss), the agent will get a higher reward if it were to select an action with low probability but ended up getting a high reward. That will have the effect of pushing up its probability of being re-selected again in future, while pushing down the previously more-favoured action probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is good because it ensures there is always some element of curiosity, and it means it will act greedily in areas of the state-space it's already well explored, but continue to explore areas of the state-space which it's unfamiliar, which is why (in my opinion) it's a superior method than using epsilon-greedy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, what I personally do is once training is converged, which I define as the agent isn't garnering higher rewards and there's very low entropy in the policy, then I add some small amount of epsilon-greedy just to force some level of exploration.&lt;/p&gt;&#xA;" OwnerUserId="20352" LastActivityDate="2019-05-20T10:35:54.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12427" PostTypeId="2" ParentId="12423" CreationDate="2019-05-20T10:42:18.933" Score="1" Body="&lt;p&gt;I don't know of a tool but you could write a simple script to detect faces and crop it. It's quite simple with the &lt;strong&gt;Haar cascade in openCV&lt;/strong&gt; to detect faces and use inbuilt functions to crop your image based on the size of the detected face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps !&lt;/p&gt;&#xA;" OwnerUserId="25704" LastActivityDate="2019-05-20T10:42:18.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12428" PostTypeId="2" ParentId="12423" CreationDate="2019-05-20T10:57:31.430" Score="1" Body="&lt;p&gt;I have found a script that does what i need:&#xA;&lt;a href=&quot;https://github.com/leblancfg/autocrop&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/leblancfg/autocrop&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25770" LastActivityDate="2019-05-20T10:57:31.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12429" PostTypeId="1" CreationDate="2019-05-20T11:10:51.767" Score="2" ViewCount="90" Body="&lt;p&gt;I'm studying convolutional neural networks from the following article &lt;a href=&quot;https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take a grayscale image, the value of the pixel will be between 0 and 255. Now, if we apply a filter to our &quot;new image&quot;, can we have pixels whose values are not included in this range? In this case, how can we create the convolved image?&lt;/p&gt;&#xA;" OwnerUserId="25772" LastEditorUserId="2444" LastEditDate="2019-05-20T14:05:51.713" LastActivityDate="2021-03-10T18:06:20.660" Title="What are the value of the pixels of the convolved image?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12430" PostTypeId="2" ParentId="12429" CreationDate="2019-05-20T12:23:23.923" Score="0" Body="&lt;p&gt;The convolved image can be considered a feature map, where each new neuron represents some indication (or lack-there-of) of a feature in some receptive field of the original image, so no it does not need to be a valid image in the output.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you specifically care for it to be an image as an output, you can do a couple of things:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) normalize the produced feature map to some set range that youre working in (0-255 or 0-1)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) make the filter a valid probability distribution, and you know the output will stay in the same range as the input (ex: Gaussian filters) &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-20T12:23:23.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12431" PostTypeId="2" ParentId="12315" CreationDate="2019-05-20T12:48:49.527" Score="0" Body="&lt;p&gt;One way to tackle this problem is to identify the longest sequence in the training dataset and than expand (0 padding) all the other inputs to match that size. &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-05-20T12:48:49.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12432" PostTypeId="2" ParentId="12421" CreationDate="2019-05-20T12:55:17.600" Score="4" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$TD(\lambda)$&lt;/span&gt; return has the following form:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;G_t^\lambda = (1 - \lambda) \sum_{n=1}^{\infty} \lambda^{n-1} G_{t:t+n}&#xA;\end{equation}&lt;/span&gt;&#xA;For you MDP &lt;span class=&quot;math-container&quot;&gt;$TD(1)$&lt;/span&gt; looks like this:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;G &amp;amp;= 0.64 (r_0 + r_2 + r_4 + r_5 + r_6) + 0.36(r_1 + r_3 + r_4 + r_5 + r_6)\\&#xA;G &amp;amp;\approx 6.164&#xA;\end{align}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$TD(\lambda)$&lt;/span&gt; looks like this:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;G_0^\lambda = (1-\lambda)[\lambda^0 G_{0:1} + \lambda^1 G_{0:2} + \lambda^2 G_{0:3} + \lambda^3 G_{0:4} + \lambda^4 G_{0:5} ]&#xA;\end{equation}&lt;/span&gt;&#xA;Now each &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; term separately:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;G_{0:1} &amp;amp;= 0.64(r_0 + v_1) + 0.36(r_1 + v_2) \approx 7.864\\&#xA;G_{0:2} &amp;amp;= 0.64(r_0 + r_2 + v_3) + 0.36(r_1 + r_3 + v_3) \approx -5.336\\&#xA;G_{0:3} &amp;amp;= 0.64(r_0 + r_2 + r_4 + v_4) + 0.36(r_1 + r_3 + r_4 + v_4) \approx 25.864\\&#xA;G_{0:4} &amp;amp;= 0.64(r_0 + r_2 + r_4 + r_5 + v_5) + 0.36(r_1 + r_3 + r_4 + r_5 + v_5) \approx -11.936\\&#xA;G_{0:5} &amp;amp;= 0.64(r_0 + r_2 + r_4 + r_5 + r_6 + v_6) + 0.36(r_1 + r_3 + r_4 + r_5 + r_6 &#xA; + v_6) \approx -0.336&#xA;\end{align}&lt;/span&gt;&#xA;Finally, we need to find &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; so that the return is equal to &lt;span class=&quot;math-container&quot;&gt;$TD(1)$&lt;/span&gt; return, we have:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;6.164 = (1 - \lambda)[7.864 - 5.336\lambda + 25.864\lambda^2 - 11.936\lambda^3 - 0.336\lambda^4]&#xA;\end{equation}&lt;/span&gt;&#xA;When you solve this equation, one of the solutions is  &lt;span class=&quot;math-container&quot;&gt;$0.205029$&lt;/span&gt; which is close to what you needed to get considering the numerical errors. Your problem was that you only considered probability in first state, but that decision prolongs to future states as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by bewestphal, this is not a full solution, it misses one crucial step to get it fully correct. Hint for it can be found in his &lt;a href=&quot;https://ai.stackexchange.com/a/12606/20339&quot;&gt;answer&lt;/a&gt; and that's the correct solution.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-06-01T08:29:59.993" LastActivityDate="2019-06-01T08:29:59.993" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12433" PostTypeId="1" AcceptedAnswerId="12457" CreationDate="2019-05-20T14:16:37.620" Score="5" ViewCount="139" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1905.04994&quot; rel=&quot;nofollow noreferrer&quot;&gt;Governance by Glass-Box: Implementing Transparent Moral Bounds for AI Behaviour&lt;/a&gt;, the authors seem to be presenting a &lt;a href=&quot;https://en.wikipedia.org/wiki/Black_box&quot; rel=&quot;nofollow noreferrer&quot;&gt;black box&lt;/a&gt; method of testing. Are these ideas really new? Weren't these ideas already proposed in &lt;a href=&quot;https://www.researchgate.net/publication/291179099_Translating_Values_into_Design_Requirements&quot; rel=&quot;nofollow noreferrer&quot;&gt;Translating Values into Design Requirements&lt;/a&gt; (by Ibo Van de Poel)? Black-box testing had already been proposed much earlier .&lt;/p&gt;&#xA;" OwnerUserId="25777" LastEditorUserId="25777" LastEditDate="2019-05-22T14:07:07.007" LastActivityDate="2019-05-22T14:07:07.007" Title="Are the ideas in the paper &quot;Governance by Glass-Box: Implementing Transparent Moral Bounds for AI Behaviour&quot; novel?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;ai-basics&gt;&lt;ai-community&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12434" PostTypeId="1" CreationDate="2019-05-20T15:59:51.887" Score="5" ViewCount="256" Body="&lt;p&gt;I have come across something that IBM offers called neural Architecture search. You feed it a data set and it outputs an initial neural Architecture that you can train. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is &lt;em&gt;neural architecture search&lt;/em&gt; (NAS) performed? Do they use heuristics or is this meta machine learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any papers on NAS, I would appreciate if you can provide a link to them.&lt;/p&gt;&#xA;" OwnerUserId="25780" LastEditorUserId="2444" LastEditDate="2019-07-07T22:46:56.443" LastActivityDate="2019-11-11T14:16:45.350" Title="How is neural architecture search performed?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;neuroevolution&gt;&lt;hyperparameter-optimization&gt;&lt;neural-architecture-search&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12435" PostTypeId="2" ParentId="12434" CreationDate="2019-05-20T17:05:54.080" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_architecture_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural architecture search (NAS)&lt;/a&gt; is a method of automating the design (that is, the choice of the values of the hyper-parameters) of artificial neural networks. There are different approaches to search the space of neural network architectures. For example, you can use reinforcement learning or evolutionary (or genetic) algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out the paper &lt;a href=&quot;https://arxiv.org/pdf/1611.01578.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Architecture Search with Reinforcement Learning&lt;/a&gt; (2017), by Barret Zoph and Quoc V. Le, where the authors train, using reinforcement learning (specifically, &lt;a href=&quot;http://www-anw.cs.umass.edu/~barto/courses/cs687/williams92simple.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;REINFORCE&lt;/a&gt;), a recurrent neural network (the &quot;controller&quot;) to generate (convolutional and recurrent) neural network architectures, so that to maximise the expected accuracy of the generated architectures on a validation dataset. They achieve some good results using this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://arxiv.org/pdf/1802.03268.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Neural Architecture Search via Parameter Sharing&lt;/a&gt; (2018), by Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le and Jeff Dean (which thus includes some of the authors of &lt;a href=&quot;https://arxiv.org/pdf/1611.01578.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NAS&lt;/a&gt;), which is similar to &lt;a href=&quot;https://arxiv.org/pdf/1611.01578.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NAS&lt;/a&gt;, but more efficient, hence the acronym ENAS (efficient NAS).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-18T23:28:03.637" LastActivityDate="2019-07-18T23:28:03.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12436" PostTypeId="5" CreationDate="2019-05-20T17:22:19.743" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_architecture_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Neural_architecture_search&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-20T20:40:57.870" LastActivityDate="2019-05-20T20:40:57.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12437" PostTypeId="4" CreationDate="2019-05-20T17:22:19.743" Score="0" Body="For questions related to the concept of neural (network) architecture search (NAS), which is a way of automating the design (that is, the hyper-parameters) of a neural network. NAS is related to neuroevolution, given that neuroevolution can be used to perform NAS, but neuroevolution is not the only way of performing NAS. For example, reinforcement learning can also be used to perform NAS." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-07T22:45:23.990" LastActivityDate="2019-07-07T22:45:23.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12438" PostTypeId="5" CreationDate="2019-05-20T17:23:42.973" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hyperparameter_optimization&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-20T20:41:00.833" LastActivityDate="2019-05-20T20:41:00.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12439" PostTypeId="4" CreationDate="2019-05-20T17:23:42.973" Score="0" Body="For questions related to the concept of hyper-parameter optimization, that is, the task of finding the best hyper-parameters for a particular learning algorithm (e.g. gradient descent) or model (e.g. a multi-layer neural network) using an optimization method (e.g. Bayesian optimization or genetic algorithms)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-20T12:06:56.737" LastActivityDate="2019-07-20T12:06:56.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12441" PostTypeId="1" CreationDate="2019-05-20T19:17:34.923" Score="4" ViewCount="155" Body="&lt;p&gt;Will the machine become so powerful that it begins to think to a point where it is more capable than the humans? Notice, I said capable, not powerful. No doubt computers are more powerful at giving us answers faster than human brainpower, but are they more capable? so can artificial intelligence take over the world?.&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="1671" LastEditDate="2019-05-20T21:26:14.160" LastActivityDate="2019-05-29T21:19:13.640" Title="Can artificial intelligence take over the world?" Tags="&lt;superintelligence&gt;&lt;mythology-of-ai&gt;&lt;ai-takeover&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12442" PostTypeId="2" ParentId="12441" CreationDate="2019-05-20T20:52:29.523" Score="3" Body="&lt;p&gt;The world seems to be operating under the &quot;law of maximum intelligence&quot; (I made this up), which is that the species with the most intelligence get to realise its preferences the most strongly. Put another way, the more intelligent you are, the more the world takes the shape of what you want; and this happens through your actions upon the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is tautological if what you mean by &quot;intelligence&quot; is &quot;ability to act well&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, both humans and chimps want to be happy, but you get zoos and not chimps paradise, because zoos are optimised to please humans, not chimps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you grant this law, then all you need is a &quot;greater than humanity&quot; level of intelligence, and an entity with &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;that level of intelligence&lt;/a&gt; should have its preferences met.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to whever or not this will happen, I’ll quote Sam Harris in &lt;a href=&quot;https://youtu.be/qwDnvA7mQMQ?t=4m2s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this little talk&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;All you have to grant to get your fears [about AI] up and running is&#xA;  that:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;We will continue to make progress in hardware and software design&#xA;  (unless we destroy ourselves some other way), and &lt;/li&gt;&#xA;  &lt;li&gt;There’s nothing&#xA;  magical about the wetware we have running inside our heads, and that&#xA;  an intelligent machine could be build of other material. […]&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Once you’ve granted those two propositions, you’ll now be hard pressed&#xA;  to find some hand hold with which to resist yor slide into real&#xA;  concern about where this is all going.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To go back to the question: yes if you have an entity/agent that is sufficiently intelligent, and with a goal that would directly or as a side-effect cause it to take over the world.&#xA;Note that this is a real concern because of &lt;a href=&quot;https://en.wikipedia.org/wiki/Instrumental_convergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;instrumental convergence&lt;/a&gt;. &lt;a href=&quot;https://www.youtube.com/watch?v=dY3zDvoLoao&quot; rel=&quot;nofollow noreferrer&quot;&gt;Good talk on this topic by Nates Soares&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also recommend the lesswrong forums: &lt;a href=&quot;https://wiki.lesswrong.com/wiki/AI_takeoff&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wiki.lesswrong.com/wiki/AI_takeoff&lt;/a&gt; and Nick Bostrom: &lt;a href=&quot;https://nickbostrom.com/superintelligence.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nickbostrom.com/superintelligence.html&lt;/a&gt; (or his book also titled Superintelligence)&lt;/p&gt;&#xA;" OwnerUserId="1741" LastEditorUserId="1741" LastEditDate="2019-05-20T20:58:40.557" LastActivityDate="2019-05-20T20:58:40.557" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12443" PostTypeId="2" ParentId="12434" CreationDate="2019-05-20T21:17:26.960" Score="6" Body="&lt;p&gt;You could say that NAS fits into the domain of Meta Learning or Meta Machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've pulled the NAS papers from my notes, this is a collection of papers/lectures that I personally found very interesting. It's sorted in rough chronological descending order, and &lt;code&gt;***&lt;/code&gt; means influential / must read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quoc V. Le and Barret Zoph are to good authors on the topic.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1901.11117v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;The Evolved Transformer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1904.01569.pdf&quot; rel=&quot;noreferrer&quot;&gt;Exploring Randomly Wired Neural Networks for Image Recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1810.05749v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;GRAPH HYPERNETWORKS FOR NEURAL ARCHITECTURE SEARCH&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1808.02822v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;Backprop Evolution&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.00559v3.pdf&quot; rel=&quot;noreferrer&quot;&gt;Progressive Neural Architecture Search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;*** &lt;a href=&quot;https://arxiv.org/pdf/1806.09055v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;DARTS: Differentiable Architecture Search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;*** &lt;a href=&quot;https://arxiv.org/pdf/1802.03268.pdf&quot; rel=&quot;noreferrer&quot;&gt;Efficient Neural Architecture Search via Parameter Sharing - ENAS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;*** &lt;a href=&quot;https://arxiv.org/pdf/1712.00559v2.pdf&quot; rel=&quot;noreferrer&quot;&gt;Progressive Neural Architecture Search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1805.07440v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=OR-IKyP4ZpI&quot; rel=&quot;noreferrer&quot;&gt;Automatic Machine Learning - Prof. Frank Hutter&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=sROrvtXnT7Q&quot; rel=&quot;noreferrer&quot;&gt;Google Brain - Neural Architecture Search - Quoc Le&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;*** &lt;a href=&quot;https://arxiv.org/pdf/1802.01548.pdf&quot; rel=&quot;noreferrer&quot;&gt;Regularized Evolution for Image Classifier Architecture Search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1803.00684v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;Autostacker: A Compositional Evolutionary Learning System&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1801.01952v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;Generating Neural Networks with Neural Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.07420.pdf&quot; rel=&quot;noreferrer&quot;&gt;Finding Competitive Network Architectures Within a Day Using UCT&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.oreilly.com/ideas/neuroevolution-a-different-kind-of-deep-learning&quot; rel=&quot;noreferrer&quot;&gt;Neuroevolution: A different kind of deep learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.00548.pdf&quot; rel=&quot;noreferrer&quot;&gt;Evolving Deep Neural Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=TERCdog1ddE&quot; rel=&quot;noreferrer&quot;&gt;Pieter Abbeel: Deep Learning-to-Learn Robotic Control&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;*** &lt;a href=&quot;https://arxiv.org/pdf/1708.05344.pdf&quot; rel=&quot;noreferrer&quot;&gt;SMASH: One-Shot Model Architecture Search through HyperNetworks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1741" LastActivityDate="2019-05-20T21:17:26.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12444" PostTypeId="1" CreationDate="2019-05-20T21:34:18.660" Score="1" ViewCount="4085" Body="&lt;p&gt;First of all, I don't play games at all, and I am still quite new to deep learning. I was using AlexNet (transfer learning, actually) in MATLAB to classify images in my current laptop (i5-3230, without any GPU). And it was taking roughly 25-30 hours to finish training, and I don't dare using GooLeNet.&lt;/p&gt;&#xA;&lt;p&gt;I have to buy a laptop. &lt;strong&gt;How fast will be if I use GPU?&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And I am not that rich to buy 2080 or 1080ti that easily. Actually, I am poor and I have struggled financially, if the price exceeds $1200. I have the intention to do research on signal and medical image processing using deep learning in the future.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Will it be enough for me to buy a laptop with GTX 1060 6GB, or is it essential/overkill to buy one with RTX2060?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Feel free to give any other option that could fill my requirements.&lt;/p&gt;&#xA;" OwnerUserId="25788" LastEditorUserId="2444" LastEditDate="2020-09-29T22:18:21.697" LastActivityDate="2020-09-29T22:59:52.183" Title="How fast is a GPU compared to a CPU for training deep learning models, and which GPU should I buy?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;matlab&gt;&lt;hardware-evaluation&gt;&lt;gpu&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ClosedDate="2021-01-22T03:06:42.933" ContentLicense="CC BY-SA 4.0" />
  <row Id="12445" PostTypeId="2" ParentId="12418" CreationDate="2019-05-21T06:15:20.557" Score="2" Body="&lt;p&gt;You have made many wrong assumptions in this question. First &lt;strong&gt;theoretically&lt;/strong&gt; speaking,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Filters do not work in the way to 'pick up elements' (they work on principle of edge detection).&lt;/li&gt;&#xA;&lt;li&gt;You have assumed only a single combination of filter weights will give the desired output (assuming continuous weights not binary). This is especially in prominence in the problem of Regularization where we want to choose a set of weights without over-fitting data.&lt;/li&gt;&#xA;&lt;li&gt;The error you used looks very similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptron&lt;/a&gt; update rule (squared error gives the same derivative, but make sure you are not confusing the two).&lt;/li&gt;&#xA;&lt;li&gt;Backpropagation through 'dead ReLu's' is not possible (see this &lt;a href=&quot;https://datascience.stackexchange.com/questions/5706/what-is-the-dying-relu-problem-in-neural-networks&quot;&gt;answer&lt;/a&gt; for more details).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, let us check &lt;strong&gt;mathematically&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Input Volume:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    1 &amp;amp; -1 &amp;amp; 0 \\&#xA;    0 &amp;amp; 1 &amp;amp; 0 \\&#xA;    0 &amp;amp; -1 &amp;amp; 1 \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Desired Output:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    1 &amp;amp; -1 \\&#xA;    0 &amp;amp; 1  \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, in this step you desire an output which is negative (element (0,-1)), but you are forward propagating through a ReLu which is cutting off the negative part, thus the gradients have no way to communicate or update the required negative.&#xA;Basically,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ wx \rightarrow ReLu \rightarrow y$&lt;/span&gt; is happening and if 'x' is a negative number then &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is always &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; thus &lt;span class=&quot;math-container&quot;&gt;$(target-y)$&lt;/span&gt; is always &lt;span class=&quot;math-container&quot;&gt;$target$&lt;/span&gt; and hence whatever the value of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; error remains constant, and if we want to backpropagate (assuming squared error) then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{d}{dw} (target - y)^2 = 2*(target - y)*\frac{d}{dw}y = -2*(target - y)*0 = 0$&lt;/span&gt; (Remember from the ReLu output graph slope is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; in the negative region).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Now, you randomise a filter:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    1 &amp;amp; -1 \\&#xA;    1 &amp;amp; 1  \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;apply ReLu and get the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    3 &amp;amp; 0 \\&#xA;    0 &amp;amp; 1  \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;again you have chosen you target to have a negative number which is not possible in the case of ReLu activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But continuing you get the error as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    -2 &amp;amp; -1 \\&#xA;    0 &amp;amp; 0  \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;use error to compute gradients (which again you have calculated wrong,by Backpropagating through ReLu's with ) values and also missed the minus sign associated with the output, but you have compensated it by adding it to the &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; whereas the convention is to subtract):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    1 &amp;amp; 2 \\&#xA;    1 &amp;amp; -2  \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And get the new filter as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \begin{matrix}&#xA;    0.5 &amp;amp; 0 \\&#xA;    0.5 &amp;amp; 0  \\&#xA;    \end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a pretty good approximation of the desired filter (even though the previous steps have wrong assumptions, but it does not matter much, since what you essentially did was use a linear activation function which will work, if you go through enough iterations). So basically you are using a Linear Filters and the details are too hodge podge for me to go into, so I will suggest some resources for you to see ReLu backpropagation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/19272/deep-neural-network-backpropogation-with-relu&quot;&gt;Deep Neural Network - Backpropogation with ReLU&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-05-21T11:50:12.427" LastActivityDate="2019-05-21T11:50:12.427" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12446" PostTypeId="1" CreationDate="2019-05-21T07:07:56.920" Score="1" ViewCount="23" Body="&lt;p&gt;I am trying to learn utility functions for ships through their AIS data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a lot of data available and plan on focusing on fishing boats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've researched a lot of IRL algorithms but I'm not sure if I missed a important one that could be applied.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I've found this paper &lt;a href=&quot;https://journals.sagepub.com/doi/citedby/10.1177/0278364917722396&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.sagepub.com/doi/citedby/10.1177/0278364917722396&lt;/a&gt;, but I'm not sure if this is really applicable.&#xA;I would need to transform the ais data as trajectorys, add material from openseamap and plot this to images of trajectories. Or did I misunderstand the paper completely?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My other found approach would be selecting features as position, distance to other vessels and others. And then try to apply continous max entropy deep inverse reinforcement learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there another approach that may be easier in your eyes?&lt;/p&gt;&#xA;" OwnerUserId="25796" LastActivityDate="2019-05-21T07:07:56.920" Title="Learning utility function for AIS data" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;utility-based&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12447" PostTypeId="1" CreationDate="2019-05-21T07:30:15.377" Score="2" ViewCount="47" Body="&lt;p&gt;I'm trying to make use of sensor data from VOC, Humidity, Age, Sampling rate and use them as NN input data. Below are the technical questions I'm struggling to find out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each training set, I have 2500 data points for VOC and Humidity. However, for the age and sampling rate, I have only one for each. I'm wondering if it would work to just put 5002(=2500(VOC)+ 2500(hum)+1(age) + 1 (sampling rate)) input data points to the layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Note: sampling rate is in the data set because the VOC and HUM data have different data points for each training set. I reduced the data points by sampling from the original data points. However, I know for sure that timespan does matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me out! A good reference is also a huge welcome!&lt;/p&gt;&#xA;" OwnerUserId="25797" LastActivityDate="2019-05-21T07:30:15.377" Title="How to deal with Neural network input data with different length and type" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12448" PostTypeId="1" CreationDate="2019-05-21T08:37:45.133" Score="5" ViewCount="231" Body="&lt;p&gt;What is a simple turn-based game, that can be used to validate a Monte-Carlo Tree Search code and it's parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before applying it to problems where I do not have a possiblity to validate its moves for correctness, I would like to implement a test case, that makes sure that it behaves as expected, especially when there are some ambiguities between different implementations and papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I built a &lt;em&gt;connect-four game&lt;/em&gt; in which to MCTS-AI play against each other and an &lt;em&gt;iterated prisoners dilemma&lt;/em&gt; implementation, in which a MCTS-AI plays against common strategies like Tit-for-Tat, but I am still not sure if there is a real good interpretation if the MCTS-AI finds the best strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative would be a &lt;em&gt;tic-tac-toe&lt;/em&gt; game, but MCTS will exhaust the whole search space within little steps, so it is hard to tell how the implementation will perform on other problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, expanding a full game tree does not tell you if any states before the full expansion are following the best MCTS strategy.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Example:&lt;/em&gt;&#xA;You can alternate in the expand step of player 1's tree between &lt;em&gt;optimize for player 1&lt;/em&gt; and &lt;em&gt;optimize for player 2&lt;/em&gt;, assuming that player 2 will not play the best move for player 1, but the best move for himself. Not doing so would result in an optimistic game tree, that may work in some cases, but probably is not the best choice for many games, while it would be useful for cooperative games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the game tree is fully expanded, you can find the best move, even when the order of the expand steps was not optimal, so using a game that can be fully expanded is no good test to validate the in-between steps.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Is there a simple to implement game, that can be used for validation, in which you can reliably check for each move, if the AI did find the expected move?&lt;/p&gt;&#xA;" OwnerUserId="25798" LastEditorUserId="25798" LastEditDate="2019-05-22T09:55:11.893" LastActivityDate="2019-06-12T08:41:02.370" Title="What is a simple game for validation of MCTS?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="12449" PostTypeId="2" ParentId="11438" CreationDate="2019-05-21T09:04:47.653" Score="1" Body="&lt;p&gt;Maybe the following article can help you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1905.02851&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance (2019)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They evaluate their model in localgovFAQ and StackExchange datasets.&lt;/p&gt;&#xA;" OwnerUserId="25789" LastEditorUserId="1671" LastEditDate="2019-05-21T18:29:45.637" LastActivityDate="2019-05-21T18:29:45.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12450" PostTypeId="1" CreationDate="2019-05-21T13:19:08.540" Score="3" ViewCount="232" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1710.06234.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nonlinear Interference Mitigation via Deep Neural Networks&lt;/a&gt;, the the following network is illustrated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network structure is &lt;a href=&quot;https://i.stack.imgur.com/1rsp2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1rsp2.png&quot; alt=&quot;neural network structure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network parameters are &lt;span class=&quot;math-container&quot;&gt;$\theta = \{W_1^{1},...,W_1^{l-1},W_2^{1},...,W_2^{l-1},W^{l},\alpha_1,...,\alpha_{l-1}\}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$W_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_2$&lt;/span&gt; are linear matrices and &lt;span class=&quot;math-container&quot;&gt;$\rho^{(i)}(x)=xe^{-j\alpha_i|x|^2}$&lt;/span&gt; is element-wise nonlinear function (&lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the index of layer). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where should I add this &lt;span class=&quot;math-container&quot;&gt;$\rho^{(i)}(x)$&lt;/span&gt;? Is it possible to learn the parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;? I don't think it is the same idea as activation function since it is positioned in the middle of two linear matrices... Or can it be added as embedding layer?&lt;/p&gt;&#xA;" OwnerUserId="25806" LastEditorUserId="2444" LastEditDate="2019-05-21T19:55:35.587" LastActivityDate="2019-05-21T20:07:25.053" Title="Can you learn parameters in nonlinear function?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="15" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12451" PostTypeId="1" CreationDate="2019-05-21T14:28:36.720" Score="2" ViewCount="65" Body="&lt;p&gt;Is there any conmprehensive paper or site that describes an overview of the known black box testing techniques?&lt;/p&gt;&#xA;" OwnerUserId="25777" LastEditorUserId="1671" LastEditDate="2019-05-21T18:06:44.967" LastActivityDate="2019-05-21T18:28:30.243" Title="What are the known ways to test AI black boxes?" Tags="&lt;ai-design&gt;&lt;theory&gt;&lt;reference-request&gt;&lt;black-box&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2021-01-29T21:18:55.840" ContentLicense="CC BY-SA 4.0" />
  <row Id="12452" PostTypeId="2" ParentId="12451" CreationDate="2019-05-21T16:49:09.440" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.semanticscholar.org/paper/On-the-applicability-of-Artificial-Intelligence-in-Khanna/42628977cb7a02d36f74672cb2d1d374638314b0&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the applicability of Artificial Intelligence in Black Box Testing (Khanna, 2017)&lt;/a&gt; might be a good place to start. It classifies some black box testing techniques based on the AI branch it belongs to.&lt;/p&gt;&#xA;" OwnerUserId="22365" LastEditorUserId="1671" LastEditDate="2019-05-21T18:28:30.243" LastActivityDate="2019-05-21T18:28:30.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12453" PostTypeId="1" CreationDate="2019-05-21T18:44:19.703" Score="1" ViewCount="41" Body="&lt;p&gt;I think reinforcement learning would be a good fit for this problem, but I am not sure of how to deal with a seemingly infinite number of actions. In the beginning of each game (generic RTS game), the player places down units anywhere on the map. Then as the game progresses, the player can move units around by selecting on them and clicking on a valid location on the map. They must take into consideration things like distance and travel time. An AI agent must do the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I represent these actions? It’s not as simple as selecting ‘up’, ‘down’, ‘right’,...etc. Should the agent just randomly pick locations on the map? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any papers or implementations I can look at to help me get started?&lt;/p&gt;&#xA;" OwnerUserId="25810" LastActivityDate="2019-05-21T18:44:19.703" Title="Action spaces for an RTS game" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;game-ai&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12454" PostTypeId="2" ParentId="12450" CreationDate="2019-05-21T20:07:25.053" Score="1" Body="&lt;p&gt;In general, you can learn any parameter of the network, provided you can find the partial derivative of the loss function with respect to the desired parameter. Given that &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is assumed to be differentiable (as the authors state in the paper), you can take the partial derivative of the loss function with respect to the parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper, &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is a non-linear function (that is, a function that is not &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear&lt;/a&gt;, e.g. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;sigmoid function&lt;/a&gt;) that applies element-wise to its input. So, if you pass a vector to this &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, you will get a vector of the same shape out of it. The authors do not explicitly call it an &quot;activation function&quot;, but &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; does an analogous job of an activation function, that is, it introduces non-linearity. Furthermore, in this architecture, &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is also followed by a matrix. In general, this is not forbidden, even though it is not common. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, in each layer of a neural network, you can have several different learnable parameters or weights. A parameter is learnable if you can differentiate the loss function with respect to it. You can have more than one weight matrix. For example, recurrent neural networks have usually more than one weight matrix associated with each layer: one matrix is associated with the feed-forward connections and the other matrix is associated with the recurrent connections. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-05-21T20:07:25.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12455" PostTypeId="1" CreationDate="2019-05-21T20:39:52.907" Score="2" ViewCount="130" Body="&lt;p&gt;I am using the following library:&#xA;&lt;a href=&quot;https://github.com/vishnugh/evo-NEAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/vishnugh/evo-NEAT&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which seems to be a pretty simple NEAT-implementation. &#xA;Therefore I am using the following Config:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;package com.evo.NEAT.com.evo.NEAT.config;&#xA;&#xA;/**&#xA; * Created by vishnughosh on 01/03/17.&#xA; */&#xA;public class NEAT_Config {&#xA;&#xA;    public static final int INPUTS = 11;&#xA;    public static final int OUTPUTS = 2;&#xA;    public static final int HIDDEN_NODES = 100;&#xA;    public static final int POPULATION =300;&#xA;&#xA;    public static final float COMPATIBILITY_THRESHOLD = Float.MAX_VALUE;&#xA;    public static final float EXCESS_COEFFICENT = 1;&#xA;    public static final float DISJOINT_COEFFICENT = 1;&#xA;    public static final float WEIGHT_COEFFICENT = 5;&#xA;&#xA;    public static final float STALE_SPECIES = 2;&#xA;&#xA;&#xA;    public static final float STEPS = 0.1f;&#xA;    public static final float PERTURB_CHANCE = 0.9f;&#xA;    public static final float WEIGHT_CHANCE = 0.5f;&#xA;    public static final float WEIGHT_MUTATION_CHANCE = 0.5f;&#xA;    public static final float NODE_MUTATION_CHANCE = 0.1f;&#xA;    public static final float CONNECTION_MUTATION_CHANCE = 0.1f;&#xA;    public static final float BIAS_CONNECTION_MUTATION_CHANCE = 0.1f;&#xA;    public static final float DISABLE_MUTATION_CHANCE = 0.1f;&#xA;    public static final float ENABLE_MUTATION_CHANCE = 0.2f ;&#xA;    public static final float CROSSOVER_CHANCE = 0.1f;&#xA;&#xA;    public static final int STALE_POOL = 10;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, there are way too much species (about 60). I do not know how to reduce this number, given the fact that the COMPATIBILITY_THRESHOLD is already maximized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's my fault?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am not using: &lt;a href=&quot;http://nn.cs.utexas.edu/keyword?stanley:ec02&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nn.cs.utexas.edu/keyword?stanley:ec02&lt;/a&gt; &#xA;since this algorithm seems not to work in a changing environment (where fitness can vary hardly)&lt;/p&gt;&#xA;" OwnerUserId="19062" LastEditorUserId="1847" LastEditDate="2019-05-22T06:02:38.730" LastActivityDate="2019-09-13T14:11:35.257" Title="How to reduce amount of species in NEAT?" Tags="&lt;evolutionary-algorithms&gt;&lt;neat&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12456" PostTypeId="2" ParentId="12448" CreationDate="2019-05-21T21:00:04.693" Score="4" Body="&lt;p&gt;A good choice might be smaller-scale games of Go, like a 9x9 board. This was the original application domain MCTS was designed for, and the &lt;a href=&quot;http://www.ideanest.com/vegos/MonteCarloGo.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; by Brugmann from 1993 details parameters that should lead to an agent that can play above beginner level in what is today a minuscule amount of computational time, in a scaled-down 9x9 grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go is a good choice for a benchmark because most learning algorithms fail at it pretty badly. The fact that MCTS worked here was a major breakthrough at the time, and helped cement it as a technique for game playing. If your algorithm is not working properly, it is therefore unlikely that it can learn to play Go at the level described in Brugmann's paper.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-05-21T21:00:04.693" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12457" PostTypeId="2" ParentId="12433" CreationDate="2019-05-21T21:09:07.913" Score="2" Body="&lt;p&gt;Poel's paper on &lt;em&gt;Translating Values into Design Requirements&lt;/em&gt; articulates a framework for mapping abstract values and norms into concrete design constraints that an engineer could work with. The example used in the paper is mapping beliefs about animal welfare to design constraints on chicken coups. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The newer paper by Tubella et al. on &lt;em&gt;Governance by Glass-Box&lt;/em&gt; builds on Poel's idea (and in fact, cite's Poel several times). It basically suggests that we should use Poel's design process, but that in something like an AI system, we also need to use an &quot;observation phase&quot; to validate the system, because unlike the problem of engineering a chicken coup, an AI system may &lt;em&gt;appear&lt;/em&gt; to have met design constraints, but routinely violate them in production. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you're right that the Tubella et al. paper is essentially proposing the combination of Poel's framework for translating values into design constraints with the old idea of black-box testing, but this combination itself appears to be a new, if modest, contribution.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-05-21T21:09:07.913" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12458" PostTypeId="1" CreationDate="2019-05-20T16:02:26.653" Score="1" ViewCount="75" Body="&lt;p&gt;I’ve been thinking about this for a few days and can’t tell if this would feel morally just to an average user&lt;/p&gt;&#xA;" OwnerDisplayName="John" LastEditorUserId="1671" LastEditDate="2019-05-23T20:02:34.730" LastActivityDate="2019-05-29T10:43:51.900" Title="Would it be ethical to use AI to determine a user’s gender from the content they upload, without them knowing?" Tags="&lt;applications&gt;&lt;ethics&gt;&lt;social&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12459" PostTypeId="2" ParentId="12458" CreationDate="2019-05-20T16:28:14.977" Score="1" Body="&lt;p&gt;Ethics aside, I think this could potentially create a number of different issues. What if your algorithm guesses wrong and I'm stuck with a UI that's targeting incorrectly? What if my wife and I share an account? What if I'm a (&lt;em&gt;insert orientation here&lt;/em&gt;) (&lt;em&gt;insert gender here&lt;/em&gt;) who [coaches, supports] a [men's, women's] [volleyball, football] team? People are very diverse, and you could make many mistakes. Would you then provide tools for the user to correct these mistakes? How would you be able to do so without the user being offended?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, I see many fewer issues that might arise from a section like the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Let's get to know each other!&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If you'd like, we can help tune your profile to match your interests.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;You can start by selecting some of your interests from the list below, or try searching for your own.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;kbd&gt;Fishing&lt;/kbd&gt; &lt;kbd&gt;Cooking&lt;/kbd&gt; &lt;kbd&gt;Soccer&lt;/kbd&gt; &lt;kbd&gt;Video Gaming&lt;/kbd&gt; &lt;kbd&gt;Social Media&lt;/kbd&gt;&#xA;  &lt;a href=&quot;https://ux.stackexchange.com/questions/125809&quot;&gt;&lt;code&gt;Search for my interest...&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://ux.stackexchange.com/questions/125809&quot;&gt;Skip this section &amp;rarr;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You could continue with other questions that might actually be relevant to your shaping your UI or application, allowing the user to omit/delete details for any or all questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being purely an opt-in experience prevents any issues with the user feeling like your app may be &quot;talking about them&quot; behind their back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, specifically at this point in time, I think it's wise to be transparent about what data your application knows (or &lt;em&gt;thinks&lt;/em&gt; it knows) about your users.&lt;/p&gt;&#xA;" OwnerDisplayName="maxathousand" LastActivityDate="2019-05-20T16:28:14.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12462" PostTypeId="1" CreationDate="2019-05-22T07:36:30.033" Score="3" ViewCount="150" Body="&lt;p&gt;In the book &amp;quot;Reinforcement Learning: An Introduction&amp;quot; (2018) Sutton and Barto explain, &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=243&quot; rel=&quot;nofollow noreferrer&quot;&gt;on page 221&lt;/a&gt;, a form of tile coding using &lt;em&gt;hashing&lt;/em&gt;, to reduce memory consumption.&lt;/p&gt;&#xA;&lt;p&gt;I have two questions about that:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can this approach reduce memory consumption? Doesn't it just depend on the number of tiles (you have to store one weight for each tile)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They state that there is only a &amp;quot;little loss of performance&amp;quot;. In my understanding, the sense of tile coding (and coarse coding) is, that near-by states have many tiles in common and far-away states have only few tilings in common. With tilings &amp;quot;randomly spread throughout the state space&amp;quot; this isn't the case. How does this &lt;strong&gt;not&lt;/strong&gt; influence performance?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21299" LastEditorUserId="2444" LastEditDate="2021-01-21T02:40:51.807" LastActivityDate="2021-01-21T02:40:51.807" Title="When using hashing in tile coding, why are memory requirements reduced and there is only a little loss of performance?" Tags="&lt;reinforcement-learning&gt;&lt;function-approximation&gt;&lt;sutton-barto&gt;&lt;computational-complexity&gt;&lt;tile-coding&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12463" PostTypeId="1" CreationDate="2019-05-22T07:49:14.167" Score="0" ViewCount="1211" Body="&lt;p&gt;Are there any open sourced algorithms that can take a couple of images as an input and generate a new, similar image based on that input? Or are there any resources where I can learn to create such an algorithm?&lt;/p&gt;&#xA;" OwnerUserId="25821" LastActivityDate="2019-05-22T09:07:16.780" Title="Algorithm that creates new images based on other images" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;&lt;image-generation&gt;&lt;art-aesthetics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12464" PostTypeId="1" CreationDate="2019-05-22T07:54:13.650" Score="1" ViewCount="45" Body="&lt;p&gt;I want my models to be accessible only by my programs. How do I encrypt and decrypt my model when I run inference on it? Is there any existing technology that is widely used?&lt;/p&gt;&#xA;" OwnerUserId="22093" LastEditorUserId="2444" LastEditDate="2021-03-03T10:00:28.277" LastActivityDate="2021-03-03T10:00:28.277" Title="How do I encrypt and decrypt my model when I run inference on it?" Tags="&lt;machine-learning&gt;&lt;models&gt;&lt;security&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12465" PostTypeId="1" CreationDate="2019-05-22T08:29:41.703" Score="1" ViewCount="215" Body="&lt;p&gt;I'm currently working on a group project where we need to find a pattern in a given dataset. The dataset is a collection of X, Y, Z values of a gyroscope from someone who is walking. If you plot these values you'll get a result like this.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/d2k4d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d2k4d.png&quot; alt=&quot;Plotted data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is how our dataset looks like.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XIhDF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XIhDF.png&quot; alt=&quot;Dataset&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are new to AI and ML so we first did some general research like understanding how matrices work and how to do some basic predictions with frameworks like TensorFlow and PyTorch. Now we want to start on this problem. What we need to do is to find a pattern in the dataset and count how many times this pattern appears in this dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We started of with some none AI functions to count, we managed to do that but the way we counted will probably only work on this specific dataset. So that's why we decided to do this with AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would love to hear as many different approaches as possible to count the steps since we are still learning.&lt;/p&gt;&#xA;" OwnerUserId="25820" LastActivityDate="2020-02-16T18:04:33.660" Title="Recognize pattern in dataset" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;pattern-recognition&gt;&lt;supervised-learning&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12466" PostTypeId="2" ParentId="12463" CreationDate="2019-05-22T08:44:29.047" Score="2" Body="&lt;p&gt;I'm not an expert on that so you could probably get a better answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure to understand what you're looking for. Are the couple of images about the same thing? Like pictures of cats and you want to generate a new cat based on these pictures? If that's what you want, you could probably take a look at &lt;strong&gt;Generative Adversarial Network&lt;/strong&gt; (GAN) : &lt;a href=&quot;http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction&lt;/a&gt;.&#xA;A GAN is made up of a Generator and a Discriminator. The goal of the discriminator is to distinguish the real data from the generated data. And the goal of the generator is to improve its generated data to look similar to real data. Then, if there are different cat images in your dataset, the generator will learn to create a new cat based on that dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If what you're looking for is to take different images like a cat and a dog and generate a &quot;catdog&quot;, you can take a look at &lt;strong&gt;Variational AutoEncoder&lt;/strong&gt; (VAE). For example you can train two different VAE (Encoder/Decoder). One for cats, and one for dogs. Then you take the encoder of dogs and the decoder of cats. That what I saw one day, not sure if it really works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Correct me if I'm wrong&lt;/p&gt;&#xA;" OwnerUserId="25166" LastEditorUserId="25166" LastEditDate="2019-05-22T09:07:16.780" LastActivityDate="2019-05-22T09:07:16.780" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12467" PostTypeId="2" ParentId="12441" CreationDate="2019-05-22T09:03:42.710" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Let me give you a simpler and a two letter answer No. However, not everyone may agree, as many people, including the renowned late physicist Stephen Hawking, have raised concern that “efforts to create thinking machines pose a threat to our very existence”.&lt;/p&gt;&#xA;&lt;p&gt;In my opinion, AI is obviously a powerful technology to improve our lives. In fact, the results are there for everyone to see ranging from Apple SIRI to self-driving cars. I think all these fears are a result of Sci-Fi movies.&lt;/p&gt;&#xA;&lt;p&gt;Everything we create is to enhance our human intelligence with artificial intelligence via machine learning and deep learning, so we can surpass the limitations of human minds.&lt;/p&gt;&#xA;&lt;p&gt;AI is a boon to humanity and not a curse that might harm us in the future or take over the world&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://qr.ae/TWNVmZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Will AI take over humans one day? (Quora)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25672" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-05-29T21:19:13.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12468" PostTypeId="1" CreationDate="2019-05-22T09:07:58.227" Score="1" ViewCount="107" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformer&lt;/a&gt; (adopted in &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt;), we normalize the attention weights (dot product of keys and queries) using a softmax in the Scaled Dot-Product mechanism. It is unclear to me whether this normalization is performed on each row of the weight matrix or on the entire matrix. In &lt;a href=&quot;https://www.tensorflow.org/alpha/tutorials/text/transformer&quot; rel=&quot;nofollow noreferrer&quot;&gt;the TensorFlow tutorial&lt;/a&gt;, it is performed on each row (axis=-1), and in the &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/official/transformer/model/attention_layer.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;official TensorFlow code&lt;/a&gt;, it is performed on the entire matrix (axis=None). The paper doesn't give many details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, both methods can make sense, but they have a strong impact. If on each row, then each value will have a roughly similar norm, because the sum of its weights is 1. If on the entire matrix, some values might be &quot;extinguished&quot; because all of its weights can be very close to zero. &lt;/p&gt;&#xA;" OwnerUserId="7783" LastEditorUserId="2444" LastEditDate="2019-11-01T02:43:59.707" LastActivityDate="2019-11-01T02:43:59.707" Title="How are the attention weights normalised in the transformer?" Tags="&lt;tensorflow&gt;&lt;machine-translation&gt;&lt;bert&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12469" PostTypeId="1" CreationDate="2019-05-22T10:20:35.140" Score="1" ViewCount="196" Body="&lt;p&gt;Recently I simulated the Gambler's Problem in RL:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qNMCK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qNMCK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the problem is, the curve does not at all appear the way as given in the book. The &quot;best policy&quot; curve appears a lot more undulating than it is shown based on the following factors:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sensitivity (i.e. the threshold for which you decide the state values have converged).&lt;/li&gt;&#xA;&lt;li&gt;Probability of heads (expected).&lt;/li&gt;&#xA;&lt;li&gt;Depending the value of sensitivity it also depends on whether I find the policy by finding the action (bet) which cause the maximum return by using &lt;span class=&quot;math-container&quot;&gt;$&amp;gt;$&lt;/span&gt; or by using &lt;span class=&quot;math-container&quot;&gt;$&amp;gt;=$&lt;/span&gt; in the following code i.e:&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; initialize maximum = -inf&#xA; best_action = None&#xA; loop over states:&#xA;    loop over actions of the state:&#xA;       if(action_reward&amp;gt;maximum):&#xA;          best_action = action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;Also note that if we make the final reward as 101 instead of 100 the curve becomes more uniform. This problem has also been noted in the following thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the actual intuitive explanation behind such a behaviour of the solution. Also here is the &lt;a href=&quot;http://www.incompleteideas.net/book/gamblers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt; where this problem is discussed.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1671" LastEditDate="2019-06-14T19:00:02.310" LastActivityDate="2019-06-14T19:00:02.310" Title="The problem with the Gambler's Problem in RL" Tags="&lt;reinforcement-learning&gt;&lt;probability&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12470" PostTypeId="1" CreationDate="2019-05-22T11:54:54.033" Score="5" ViewCount="138" Body="&lt;p&gt;As a layman in AI, I want to get an idea of how big data players, like Facebook, &lt;strong&gt;model&lt;/strong&gt; individuals (of which they have so many data).&lt;/p&gt;&#xA;&lt;p&gt;There are two scenarios I can imagine:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural networks build clusters of individuals by pure and &amp;quot;unconscious&amp;quot; big data analysis (not knowing, trying to understand and naming the clusters and &amp;quot;feature neurons&amp;quot; on intermediate levels of the network) with the only aim to predict some decisions of the members of these clusters with highest possible accuracy.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Letting humans analyze the clusters and neurons (trying to understand what they mean) they give names to them and possibly add human-defined &amp;quot;fields&amp;quot; (like &amp;quot;is an honest person&amp;quot;) if these were not found automatically, and whose values are then calculated from big data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The second case would result in a specific psychological model of individuals with lots of &amp;quot;human-understandable&amp;quot; dimensions.&lt;/p&gt;&#xA;&lt;p&gt;In case there is such a model, I would be very interested to know as much about it as possible.&lt;/p&gt;&#xA;&lt;p&gt;What can be said about this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there most probably such a model (that is kept as a secret e.g. by Facebook)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Has somehow tried to guess how it may look like?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there leaked parts of the model?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My aim is to know and understand by which categories Facebook (as an example) classifies its users.&lt;/p&gt;&#xA;" OwnerUserId="25362" LastEditorUserId="2444" LastEditDate="2020-06-24T13:58:01.793" LastActivityDate="2021-03-21T17:08:28.897" Title="How do big companies, like Facebook, model individuals and their interaction?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;social&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12471" PostTypeId="2" ParentId="11043" CreationDate="2019-05-22T14:49:23.757" Score="3" Body="&lt;p&gt;I found the following detailed and well documented &lt;a href=&quot;https://github.com/vadim0x60/wakesleep/blob/master/WakeSleep.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python notebook&lt;/a&gt;, which uses only NumPy.&lt;/p&gt;&#xA;" OwnerUserId="25825" LastEditorUserId="2444" LastEditDate="2019-11-18T18:54:56.627" LastActivityDate="2019-11-18T18:54:56.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12472" PostTypeId="1" AcceptedAnswerId="12480" CreationDate="2019-05-22T15:24:37.517" Score="15" ViewCount="3238" Body="&lt;p&gt;When designing solutions to problems such as the Lunar Lander on &lt;a href=&quot;https://gym.openai.com/envs/LunarLander-v2/&quot; rel=&quot;noreferrer&quot;&gt;OpenAIGym&lt;/a&gt;, Reinforcement Learning is a tempting means of giving the agent adequate action control so as to successfully land.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what are the instances in which control system algorithms, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/PID_controller&quot; rel=&quot;noreferrer&quot;&gt;PID controllers&lt;/a&gt;, would do just an adequate job as, if not better than, Reinforcement Learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions such as &lt;a href=&quot;https://ai.stackexchange.com/questions/11375/what-is-the-difference-between-reinforcement-learning-and-optimal-control&quot;&gt;this one&lt;/a&gt; do a great job at addressing the theory of this question, but do little to address the practical component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an Artificial Intelligence engineer, what elements of a problem domain should suggest to me that a PID controller is insufficient to solve a problem, and a Reinforcement Learning algorithm should instead be used (or vice versa)?&lt;/p&gt;&#xA;" OwnerUserId="22424" LastEditorUserId="22424" LastEditDate="2019-05-22T17:02:39.760" LastActivityDate="2019-05-22T23:56:44.683" Title="When should I use Reinforcement Learning vs PID Control?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;control-theory&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="12473" PostTypeId="2" ParentId="12469" CreationDate="2019-05-22T16:23:34.887" Score="3" Body="&lt;p&gt;The intuitive explanation is that there are many equally good &quot;optimal&quot; policies. This is mentioned at the end of the example problem description you posted. My gut says that the family of optimal policies would be &lt;em&gt;any&lt;/em&gt; policy from the double/nothing family. So, for example, if you bet 25 on the first bet instead of 50, I think your overall chances of winning should be the same as if you bet 50, it'll just take longer in expectation. The resulting family of policies will look more undulating than the one in the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Neil notes, for low values of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, the probability that you win a gamble, it is the case that there is a unique optimal policy. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-05-23T00:53:46.963" LastActivityDate="2019-05-23T00:53:46.963" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12474" PostTypeId="2" ParentId="12465" CreationDate="2019-05-22T19:05:49.897" Score="1" Body="&lt;p&gt;For such time-series data that has a significant amount of periodicity, I would recommend converting data to the frequency domain and performing various spectral analysis methods as @firion has already mentioned. For example, you could perform Fourier Analysis and study the individual components and identify patterns there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it generally not recommended to perform the normal pattern extraction approaches to time-series data as they fail to understand the temporal relationship between subsequent data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="25704" LastActivityDate="2019-05-22T19:05:49.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12476" PostTypeId="2" ParentId="8878" CreationDate="2019-05-22T20:52:15.023" Score="1" Body="&lt;p&gt;We don't really need artificial intelligence, but it is proving ever more useful.  This is a function of what is known as utility--the capability of an algorithm to perform a task adequately.  The new utility of AI has upsides and downsides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Weak AI: Less capable than a human&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the lower end of the scale, there might be situation where a human would be better, but the work is so dangerous, or expensive for humans, that we use automatons instead.  (Space exploration is a good example.  The AI on a deep space probe or &lt;a href=&quot;http://Home/News/How%20does%20Mars%20rover%20Curiosity&amp;#39;s%20new%20AI%20system%20work?%201.5K%20How%20does%20Mars%20rover%20Curiosity&amp;#39;s%20new%20AI%20system%20work?&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mars rover&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Semi-Strong AI: About as capable as a human&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm defining &quot;semi-strong&quot; for this answer. Here we have AI or automation that can do tasks as well as humans, such as on an assembly line, but where automation is more efficient.  (This type of automation spurred the &lt;a href=&quot;https://en.wikipedia.org/wiki/Luddite&quot; rel=&quot;nofollow noreferrer&quot;&gt;Luddite movement&lt;/a&gt;, responding to the loss of human jobs due to automation.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Machine Learning continues to get more effective, the range of tasks that AI can perform as well as humans will surely grow.  This might lead to unprecedented levels of persistent human unemployment, but some argue that automation will also create new opportunities for humans and mostly eliminate repetitive, less fulfilling work.  (See also: &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_unemployment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technological unemployment&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Strong AI: Exceeds human capability&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strong AI is the &quot;holy grail&quot;, and some believe it will lead to a technological &quot;singularity&quot; in which smarter machines make ever smarter machines. (No one knows:)  Nevertheless, Machine Learning has demonstrated greater than human capability in a number of tasks, and the range of such tasks will surely grow.  (&lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; was a milestone because the game of Go is unsolvable and notoriously difficult for AI, prior to AlphaGo.  Now it's unclear if unmodified humans will ever again be able to beat strong AI at these types of games.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although game AIs are not directly useful, except for recreational purposes, the methods used to create them can be extended to real-world problems.  There are many forms of Machine Learning, not restricted to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCTS&lt;/a&gt;, with &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Algorithms&lt;/a&gt; as another type, also recently demonstrating strong utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strong AI is useful because it has greater utility than humans.  It is desirable because it increases efficiency and expected return on investment.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-22T21:00:51.460" LastActivityDate="2019-05-22T21:00:51.460" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12477" PostTypeId="5" CreationDate="2019-05-22T21:14:20.450" Score="0" Body="&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Materials_science&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Materials_science&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-22T21:14:20.450" LastActivityDate="2019-05-22T21:14:20.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12478" PostTypeId="4" CreationDate="2019-05-22T21:14:20.450" Score="0" Body="For questions related to applications of AI in materials science." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-22T21:14:20.450" LastActivityDate="2019-05-22T21:14:20.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12479" PostTypeId="1" CreationDate="2019-05-22T21:45:21.910" Score="1" ViewCount="101" Body="&lt;p&gt;How would one go about inputting multiple high dimensionality categorical columns using TensorFlow's &lt;a href=&quot;https://www.tensorflow.org/guide/feature_columns&quot; rel=&quot;nofollow noreferrer&quot;&gt;Embedding Feature Columns&lt;/a&gt;? Does that even make sense to do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: for a car price predictor, the features are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Make&lt;/li&gt;&#xA;&lt;li&gt;Model&lt;/li&gt;&#xA;&lt;li&gt;Trim&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25834" LastActivityDate="2019-05-22T21:45:21.910" Title="Multiple embedding layers?" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;word-embedding&gt;&lt;features&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12480" PostTypeId="2" ParentId="12472" CreationDate="2019-05-22T23:56:44.683" Score="7" Body="&lt;p&gt;I think the comments are basically on the right track.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PID controllers are useful for finding optimal policies in continuous dynamical systems, and often these domains are also used as benchmarks for RL, precisely because there is an easily derived optimal policy. However, in practice, you'd obviously prefer a PID controller for any domain in which you can easily design one: the controller's behaviors are well understood, while RL solutions are often difficult to interpret. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where RL shines is in tasks where we know what good behaviour looks like (i.e., we know the reward function), and we know what sensor inputs look like (i.e. we can completely and accurately describe a given state numerically), but we have little or no idea what we actually want the agent to do to achieve those rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a good example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If I wanted to make an agent to &lt;strong&gt;maneuver a plane&lt;/strong&gt; from in front of an enemy plane with known movement patterns to behind it, using the least amount of fuel, I'd much prefer to &lt;strong&gt;use a PID controller&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I wanted to make an agent to &lt;strong&gt;control a plane&lt;/strong&gt; and shoot down an enemy plane with enough fuel left to land, but without a formal description of how the enemy plane might attack (perhaps a human expert will pilot it in simulations against our agent), I'd much &lt;strong&gt;prefer RL&lt;/strong&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-05-22T23:56:44.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12481" PostTypeId="2" ParentId="12465" CreationDate="2019-05-23T00:51:37.040" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;since you are on ML forum, recognition of sequences does RNN.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you wond belive, i currently work on similar stuff. Start about thinking of algo to find repititions in string, like : &quot;ababcab&quot; returns ('ab' : 0,2,5) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;and yes you could do Fourier Analysis but thats not ML method at all&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-05-23T00:51:37.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12482" PostTypeId="1" CreationDate="2019-05-23T00:57:08.200" Score="0" ViewCount="21" Body="&lt;p&gt;The difference in &lt;strong&gt;clasterisation&lt;/strong&gt; task is, that the classes of objects are not given , but must be determined from the dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usual usecase for NN is like having images of cats and dogs, and the right answers in form like [1,0] for cat, [0, 1] for dog , to backpropagate from those results. If the right answers are unknown , how NN will learn then? Also there might be unknown amount of classes and the output vector could &quot;grow&quot; somehow&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-05-23T00:57:08.200" Title="Is there any known visual neural network, capable of image clasterisation?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12483" PostTypeId="2" ParentId="11944" CreationDate="2019-05-23T01:01:38.733" Score="0" Body="&lt;p&gt;The wonder materials are made by &lt;strong&gt;physical engines&lt;/strong&gt; what are generally &lt;strong&gt;not part of AI&lt;/strong&gt;. But i have some weird idea about AI that it could be capable even fully do scientific research process, including &lt;strong&gt;all management&lt;/strong&gt; of research, not just island calculations, as those  are getting more complicated for humans. &lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="1671" LastEditDate="2019-05-29T20:27:03.823" LastActivityDate="2019-05-29T20:27:03.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12487" PostTypeId="1" CreationDate="2019-05-23T10:45:07.830" Score="4" ViewCount="162" Body="&lt;p&gt;There is plenty of literature describing LSTMs in a lot of detail and how to use them for multi-variate or uni-variate forecasting problems. What I couldn't find though, is any papers or discussions describing time series forecasting where we have correlated forecast data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example is best to describe what I mean. Say I wanted to predict number of people at a beach for the next 24 hours and I want to predict this in hourly granularity. This quantity of people would clearly depend on the past quantity of people at the beach as well as the weather. Now I can make an LSTM architecture of some sort to predict these future quantities based upon what happened in the past quite easily. But what if I now have access to weather forecasts for the next 24 hours too? (and historical forecast data too).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The architecture I came up with looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Eqegn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Eqegn.png&quot; alt=&quot;Forecast architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I train the left upper branch on forecast data, then train the right upper branch on out-turn data, then freeze their layers to and join them to form the final network in the picture and train that on both forecasts and out-turns. (when I say train, the output is always the forecast for the next 24 hours). This method does in fact have better performance than using forecasts or out-turns alone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess my question is, has anyone seen any literature around on this topic and/ or knows a better way to solve these sort of multivariate time series forecasting problems and is my method okay or completely flawed?&lt;/p&gt;&#xA;" OwnerUserId="25659" LastEditorUserId="2444" LastEditDate="2020-01-31T02:37:23.247" LastActivityDate="2020-03-01T03:02:15.233" Title="How should I design the LSTM architecture for multivariate time series forecasting problems?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12488" PostTypeId="1" CreationDate="2019-05-23T11:27:12.730" Score="2" ViewCount="27" Body="&lt;p&gt;Is it possible to recognize the height and width of the sails of different kitesurfers and windsurfers taken from public webcams? And show these information on video in real time? Or on screenshots?&lt;/p&gt;&#xA;" OwnerUserId="25851" LastActivityDate="2019-05-23T11:27:12.730" Title="Sails size recognition" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12490" PostTypeId="1" AcceptedAnswerId="12502" CreationDate="2019-05-23T15:36:42.400" Score="8" ViewCount="1550" Body="&lt;p&gt;Can the decoder in a transformer model be parallelized like the encoder?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder, this is not possible (in both training and testing), as self-attention is calculated based on previous timestep outputs. Even if we consider some techniques, like teacher forcing, where we are concatenating expected output with obtained, this still has a sequential input from the previous timestep. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, apart from the improvement in capturing long-term dependencies, is using a transformer-decoder better than say an LSTM, when comparing purely on the basis of parallelization?&lt;/p&gt;&#xA;" OwnerUserId="25859" LastEditorUserId="2444" LastEditDate="2019-11-01T03:02:12.243" LastActivityDate="2021-04-14T07:32:02.637" Title="Can the decoder in a transformer model be parallelized like the encoder?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;transformer&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12495" PostTypeId="2" ParentId="11944" CreationDate="2019-05-23T20:06:42.267" Score="0" Body="&lt;p&gt;Some &quot;wonder materials&quot; may themselves be a limited form of AI--specifically nanotechnology/nanomachines that make local decisions, based on stimulus and their environment, related to their function!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Molecular_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular Machine&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be a very good thing, in the sense of &lt;a href=&quot;https://en.wikipedia.org/wiki/Utility_fog&quot; rel=&quot;nofollow noreferrer&quot;&gt;utility fog&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Smartdust&quot; rel=&quot;nofollow noreferrer&quot;&gt;smartdust&lt;/a&gt;, or a very bad thing in the sense of &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Grey_goo&quot; rel=&quot;nofollow noreferrer&quot;&gt;grey goo&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-05-29T20:28:21.570" LastActivityDate="2019-05-29T20:28:21.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12497" PostTypeId="1" CreationDate="2019-05-24T05:54:37.987" Score="1" ViewCount="283" Body="&lt;p&gt;I am not sure I understand what is the advantage of using a VAE's over a deterministic Auto Encoder? For example, assuming we have just 2 labels, a deterministic Auto Encoder will always map a given image to the same latent vector. However, one expects that after the training, the 2 classes will form separate clusters in the encoder space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the VAE an image is mapped to an encoding vector probabilistically. However, one still ends up with 2 separate clusters. Now, if one passes a new image (at the test time), in both cases the network should be able to place that new image in one of the 2 clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are these 2 clusters created using the VAE better than the ones from the deterministic case? &lt;/p&gt;&#xA;" OwnerUserId="23871" LastEditorUserId="9947" LastEditDate="2019-05-24T06:35:34.587" LastActivityDate="2019-06-24T21:00:40.073" Title="Why are VAE's useful?" Tags="&lt;neural-networks&gt;&lt;autoencoders&gt;&lt;probability&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12498" PostTypeId="2" ParentId="12497" CreationDate="2019-05-24T06:33:41.073" Score="0" Body="&lt;p&gt;VAE's are not used for classification. They are used for inference or as Generative Models, while AE's can be used as data re-constructors (as you described above), de-noisers, classifiers. So the difference is generation of new data vs re-construction of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VAE's map the inputs to a hidden space, where each variable is enforced to have probability distribution which is given by &lt;span class=&quot;math-container&quot;&gt;$N(0,1)$&lt;/span&gt; i.e. the standard Normal Distribution. Once we have trained a VAE we will now, use only the decoder part to generate new models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GkyX5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GkyX5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Source: Stanford University CS231n slides&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume there is an x_axis and a y_axis on the bottom and the left. Let the x_axis represent &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and y_axis &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; which are our hidden variables. By varying &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; you can see what happens. Increasing &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; changes face angle while increasing &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; changes eye droop. Thus we can generate new data by varying the features in a  latent representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For better understanding I highly recommend you check out these links:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.jeremyjordan.me/variational-autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational autoencoders.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bjlkeng.github.io/posts/variational-autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Autoencoders - Brian Keng&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=uaaqyVS9-rM&amp;amp;t=23s&quot; rel=&quot;nofollow noreferrer&quot;&gt;VAE - Ali Ghodsi&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=5WoItGTWV54&amp;amp;t=2817s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Models - CS231n&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-05-24T06:33:41.073" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12499" PostTypeId="1" AcceptedAnswerId="12500" CreationDate="2019-05-24T07:04:47.830" Score="10" ViewCount="5433" Body="&lt;p&gt;I just learned about GAN and I'm a little bit confused about the naming of Latent Vector.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;First, In my understanding, a definition of a latent variable is a random variable that can't be measured directly (we needs some calculation from other variables to get its value). For example, knowledge is a latent variable. Is it correct?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And then, in GAN, a latent vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is a random variable which is an input of the generator network. I read in some tutorials, it's generated using only a simple random function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = np.random.uniform(-1, 1, size=(batch_size, z_size))&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then how are the two things related? why don't we use the term &quot;a vector with random values between -1 and 1&quot; when referring &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; (generator's input) in GAN?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-05-24T20:50:20.150" Title="Why is it called Latent Vector?" Tags="&lt;terminology&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12500" PostTypeId="2" ParentId="12499" CreationDate="2019-05-24T07:31:13.117" Score="5" Body="&lt;p&gt;It is called a Latent variable because you cannot access it during train time (which means manipulate it), In a normal Feed Forward NN you cannot manipulate the values output by hidden layers. Similarly the case here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term originally came from &lt;a href=&quot;https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine&quot; rel=&quot;noreferrer&quot;&gt;RBM's&lt;/a&gt; (they used term hidden variables). The interpretation of hidden variables in the context of RBM was that these hidden nodes helped to model the interaction between 2 input features (if both activate together, then the hidden unit will also activate). This principle can be traced to Hebb's rule which states &quot;Neurons that fire together, wire together.&quot; Thus RBM's were used to find representation of models, in a space (generally lower dimensional than than the original). This is the principal used in &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;noreferrer&quot;&gt;Auto Encoder's&lt;/a&gt; also. Thus as you can see we are explicitly, not modelling the interaction between 2 features, how the process is occurring is &quot;hidden&quot; from us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the term latent basically can be attributed to the following ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We map higher dimensional data to a lower dimensional data with no prior convictions of how the mapping will be done. The NN trains itself for the best configuration.&lt;/li&gt;&#xA;&lt;li&gt;We cannot manipulate this lower dimensional data. Thus it is &quot;hidden from us.&lt;/li&gt;&#xA;&lt;li&gt;As we do not know what each dimension means, it is &quot;hidden&quot; from us.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-05-24T07:31:13.117" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12502" PostTypeId="2" ParentId="12490" CreationDate="2019-05-24T10:57:08.330" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Can the decoder in a transformer model be parallelized like the&lt;/strong&gt;&#xA;&lt;strong&gt;encoder?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Generally NO:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your understanding is completely right. In the decoder, the output of each step is fed to the bottom decoder in the next time step, just like an &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Also, like in LSTMs, the self-attention layer needs to attend to earlier positions in the output sequence in order to compute the output. Which makes straight parallelisation impossible.&lt;/p&gt;&#xA;&lt;p&gt;However, when decoding during training, there is a frequently used procedure which doesn't take the previous output of the model at step t as input at step t+1, but rather takes the ground truth output at step t. This procudure is called 'Teacher Forcing' and makes the decoder parallelised during training. You can read more about it &lt;a href=&quot;https://machinelearningmastery.com/teacher-forcing-for-recurrent-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;And For detailed explanation of how Transformer works I suggest reading this article: &lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Illustrated Transformer&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is using a transformer-decoder better than say an lstm when comparing&lt;/strong&gt;&#xA;&lt;strong&gt;purely on the basis of parallelization?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;YES:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Parallelization is the main drawback of &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNNs&lt;/a&gt; in general. In a simple way, RNNs have the ability to memorize but not parallelize while &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNNs&lt;/a&gt; have the opposite. Transformers are so powerful because they &lt;strong&gt;combine both&lt;/strong&gt; parallelization (at least partially) and memorizing.&lt;/p&gt;&#xA;&lt;p&gt;In Natural Language Processing for example, where RNNs are used to be so effective, if you take a look at &lt;a href=&quot;https://gluebenchmark.com/leaderboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLUE leaderboard&lt;/a&gt; you will find that most of the world leading algorithms today are Transformer-based (e.g &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT by GOOGLE&lt;/a&gt;, &lt;a href=&quot;https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT by OpenAI&lt;/a&gt;..)&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;For better understanding of why Transformers are better than CNNs I suggest reading this Medium article: &lt;a href=&quot;https://towardsdatascience.com/transformers-141e32e69591&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Transformers Work.&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="23350" LastEditDate="2021-04-14T07:32:02.637" LastActivityDate="2021-04-14T07:32:02.637" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12504" PostTypeId="2" ParentId="12497" CreationDate="2019-05-24T11:08:47.010" Score="1" Body="&lt;p&gt;The main difference between a variational auto-encoder (VAE) and an auto-encoder is that the VAE is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/12354/2444&quot;&gt;statistical model&lt;/a&gt;  while an &lt;a href=&quot;https://ai.stackexchange.com/q/11405/2444&quot;&gt;auto-encoder (AE)&lt;/a&gt; is just a data compressor and decompressor (it is just a function approximator), so an AE is not a statistical model. They both have an encoder and a decoder and they both convert the inputs to a latent representation, but their inner workings are different and their purpose is also slightly different. An &lt;a href=&quot;https://stats.stackexchange.com/q/408398/82135&quot;&gt;AE could be considered a generative model&lt;/a&gt;, in the sense that it generates objects that are similar to the input (but they might not be the same as the input). However, it is &lt;em&gt;not&lt;/em&gt; a generative model according to the usual &lt;em&gt;statistical&lt;/em&gt; definition of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative model&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of an &lt;em&gt;AE&lt;/em&gt;, given an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (e.g. an image), the encoder produces &lt;strong&gt;one&lt;/strong&gt; latent vector &lt;span class=&quot;math-container&quot;&gt;$z_x$&lt;/span&gt;, which can be decoded into &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; (another image which should be similar or related to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;). Compactly, this can be presented as &lt;span class=&quot;math-container&quot;&gt;$x'=f(z_x=g(x))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the encoder and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the decoder. This operation is deterministic, in the sense that, given the same &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, the same &lt;span class=&quot;math-container&quot;&gt;$z_x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; are produced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a &lt;em&gt;VAE&lt;/em&gt;, given an input &lt;span class=&quot;math-container&quot;&gt;$x \in \mathcal{X}$&lt;/span&gt; (e.g. an image), more than one latent vector, &lt;span class=&quot;math-container&quot;&gt;$z_{x}^i \in \mathcal{Z}$&lt;/span&gt;, can be produced, because the encoder part of the VAE models the inputs as a probability distribution. In other words, the encoder of a VAE assumes that the inputs &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; are drawn from a probability distribution, which can be denoted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X} \sim q$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is some computable probability distribution (e.g. a Gaussian distribution with mean zero, &lt;span class=&quot;math-container&quot;&gt;$\mu =0$&lt;/span&gt;, and a finite standard deviation, &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;). &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is defined over &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Z}$&lt;/span&gt; (the space of possible latent vectors). More specifically, &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is a conditional probability distribution, which can be denoted by &lt;span class=&quot;math-container&quot;&gt;$q(z \mid x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the encoder of a VAE, while being trained, creates a probability distribution, which, in practice, can be represented by neural networks, that is, you can train a neural network to represent the parameters of e.g. a Gaussian distribution (which is defined by two parameters: the mean and standard deviation). You can thus more precisely denote &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$q_\phi$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; are the parameters of a neural network that e.g. output two numbers: the mean and a standard deviation of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a probability distribution &lt;span class=&quot;math-container&quot;&gt;$q_\phi$&lt;/span&gt;, we can sample several &lt;span class=&quot;math-container&quot;&gt;$z_{x}^i$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$q_\phi$&lt;/span&gt;. Why is this useful? Given the same input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (e.g. a graphical representation of a molecule &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;), we can sample e.g. &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; latent vectors &lt;span class=&quot;math-container&quot;&gt;$z_{x}^1, z_{x}^2, \dots, z_{x}^N$&lt;/span&gt;, which can all be decoded (e.g. into &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; new molecules). Why would this be useful? For example, in the case you need to generate molecules which have a similar structure to the input molecule &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. Why would you need this? In some cases, molecules with a similar structure can have similar effects, but could e.g. be more easy or inexpensive to &lt;a href=&quot;https://en.wikipedia.org/wiki/Chemical_synthesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;synthesise&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why can we generate &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; new molecules out of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; latent vectors, where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; can be greater than &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;? We have not yet mentioned the details of a decoder of a VAE. The decoder of a VAE also models its inputs, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Z}$&lt;/span&gt;, as a probability distribution defined over &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt;, which can be denoted by &lt;span class=&quot;math-container&quot;&gt;$p_\theta(x \mid z)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is a latent vector. In other words, given the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th latent vector sampled from the latent distribution (the distribution of the encoder), &lt;span class=&quot;math-container&quot;&gt;$z_{x}^i$&lt;/span&gt;, we can sample more than one reconstructed input from the probability distribution of the decoder. Hence, a VAE often constructs two probability distributions: one that is associated with the encoder, &lt;span class=&quot;math-container&quot;&gt;$q_\phi(z \mid x)$&lt;/span&gt;, and one that is associated with the decoder, &lt;span class=&quot;math-container&quot;&gt;$p_\theta(x \mid z)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that VAEs create these distributions, they are considered &lt;a href=&quot;https://ai.stackexchange.com/a/12354/2444&quot;&gt;statistical models&lt;/a&gt;. An auto-encoder has no notion of a probability distribution. An AE only maps inputs to outputs deterministically, so it is a function approximator (given that it approximates a function that compresses the input and then another function that decompresses it). VAEs are thus a lot more flexible than AEs.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-25T20:06:09.987" LastActivityDate="2019-05-25T20:06:09.987" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12505" PostTypeId="2" ParentId="12499" CreationDate="2019-05-24T11:29:21.507" Score="5" Body="&lt;p&gt;&lt;em&gt;Latent&lt;/em&gt; is a synonym for &lt;em&gt;hidden&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is it called a hidden (or latent) variable? For example, suppose that you observe the behaviour of a person or animal. You can only observe the behaviour. You cannot observe the internal state (e.g. the mood) of this person or animal. The mood is a hidden variable because it cannot be observed directly (but only indirectly through its consequences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example of statistical model that is highly based on the notion of latent variables is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;hidden Markov model&lt;/a&gt; (HMM). If you understand the HMM, you will understand the concept of a hidden variable.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-24T20:50:20.150" LastActivityDate="2019-05-24T20:50:20.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12506" PostTypeId="1" CreationDate="2019-05-24T12:05:46.863" Score="4" ViewCount="157" Body="&lt;p&gt;&lt;sub&gt;I also asked this question &lt;a href=&quot;https://cs.stackexchange.com/questions/109807/multi-agent-sokoban-solvers-state-of-the-art&quot;&gt;here&lt;/a&gt; but I'm repeating it on this SE because I feel it is more relevant. No intention to spam.&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am researching into coding a solver for a variant of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sokoban&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sokoban&lt;/a&gt; game with multiple agents, restrictions (eg. colors of stones, goals) and relaxations (push AND pull possible, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By researching online I have found classic papers in the field, like the Rolling Stone paper (by Andreas Junghanns and Jonathan Schaeffer) and &lt;em&gt;Sokoban: Enhancing general single-agent search methods using domain knowledge&lt;/em&gt; from the same authors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These solutions seem to be outdated and I am currently structuring my solver per the notes of the two most performant solvers: &lt;a href=&quot;http://www.sokobano.de/wiki/index.php?title=Sokoban_solver_%22scribbles%22_by_Brian_Damgaard_about_the_YASS_solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;YASS&lt;/a&gt; and &lt;a href=&quot;https://cs.stackexchange.com/questions/109807/multi-agent-sokoban-solvers-state-of-the-art&quot;&gt;Sokolution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the research, I've done these two seem to be my best bet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is apparent that they are not enough by themselves to solve a multi-agent environment. Those solvers are made for a single agent. So far, I have failed to find useful multi-agent proposals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, my question is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What can be considered state-of-the-art in order to:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;coordinate multiple agents with different goals, and&lt;/li&gt;&#xA;&lt;li&gt;plug a solver's solution in and validate/edit it?&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;What are some search terms I can use to research this further?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you very much&lt;/p&gt;&#xA;" OwnerUserId="25888" LastEditorUserId="16565" LastEditDate="2019-05-25T21:09:31.783" LastActivityDate="2019-05-25T21:09:31.783" Title="Multi Agent Sokoban Search Solvers state of the art" Tags="&lt;research&gt;&lt;intelligent-agent&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12507" PostTypeId="2" ParentId="11285" CreationDate="2019-05-24T12:22:09.307" Score="2" Body="&lt;p&gt;The expression &quot;latent space&quot; explicitly indicates that the space is associated with the mathematical concept of an hidden (or latent) variable, which cannot be observed directly, but only indirectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expression &quot;embedding space&quot; refers to a vector space that represents an original space of inputs (e.g. images or words). For example, in the case of &quot;word embeddings&quot;, which are vector representations of words. It can also refer to a latent space because a latent space can also be a space of vectors. However, an embedding space is not necessarily an hidden space. It is just another (vector) representation of another space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These two expressions can be used interchangeably, also because the expression &quot;embedding space&quot; is often not formally defined.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-05-24T12:22:09.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12508" PostTypeId="1" CreationDate="2019-05-24T13:06:30.177" Score="7" ViewCount="161" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm#Description&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia article&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If the heuristic &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; satisfies the additional condition &lt;span class=&quot;math-container&quot;&gt;$h(x) \leq d(x, y) + h(y)$&lt;/span&gt; for every edge &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; of the graph (where &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; denotes the length of that edge), then &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is called monotone, or consistent. In such a case, &lt;span class=&quot;math-container&quot;&gt;$A^*$&lt;/span&gt; can be implemented more efficiently — roughly speaking, no node needs to be processed more than once (see closed set below) — and &lt;span class=&quot;math-container&quot;&gt;$A^*$&lt;/span&gt; is equivalent to running Dijkstra's algorithm with the reduced cost &lt;span class=&quot;math-container&quot;&gt;$d'(x, y) = d(x, y) + h(y) − h(x).$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can someone intuitively explain why the reduced cost is of this form ?&lt;/p&gt;&#xA;" OwnerUserId="25729" LastEditorUserId="2444" LastEditDate="2020-11-19T12:53:35.557" LastActivityDate="2020-12-19T13:04:35.440" Title="A* is similar to Dijkstra with reduced cost" Tags="&lt;comparison&gt;&lt;search&gt;&lt;heuristics&gt;&lt;a-star&gt;&lt;dijkstras-algorithm&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12509" PostTypeId="1" CreationDate="2019-05-24T15:27:04.913" Score="1" ViewCount="35" Body="&lt;p&gt;I want to be able to improve my lower level device specific programming abilities to assist in future endeavors. Examples would be learning to write custom tensorflow operations in C++ optimized to work on GPUS. Does anyone know good resources to find tutorials showing which packages to use for which devices and etc.?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approach other than just reading source and replicating until understanding would be nice. &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-24T15:27:04.913" Title="Any good resources for learning programming GPU level operations?" Tags="&lt;c++&gt;&lt;gpu&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12510" PostTypeId="1" AcceptedAnswerId="12512" CreationDate="2019-05-24T17:39:06.653" Score="3" ViewCount="117" Body="&lt;p&gt;No matter what I google or what paper I read, I can't find an answer to my question. In a deep convolutional neural network, let's say AlexNet (Krizhevsky, 2012), filters' weights are learned by means of back-prop.&lt;/p&gt;&#xA;&lt;p&gt;But how are kernels themselves selected? I know kernels had been used in image processing long before CNNs, hence I'd imagine there would be a set of filters based on kernels (see, for example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_(image_processing)#Details&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;) that are proven to be effective for edge detection and the likes.&lt;/p&gt;&#xA;&lt;p&gt;Reading around the web, I also found something about &amp;quot;randomly generated kernels&amp;quot;. Does anyone know if and when this practice is adopted?&lt;/p&gt;&#xA;" OwnerUserId="25893" LastEditorUserId="2444" LastEditDate="2020-12-11T11:32:29.553" LastActivityDate="2020-12-11T15:13:30.177" Title="Are filters fixed or learned?" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;filters&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12511" PostTypeId="2" ParentId="12510" CreationDate="2019-05-24T18:00:11.893" Score="1" Body="&lt;p&gt;If you're looking for filters with known effect, the Gaussian filters do smoothing, the Gabor filters are useful for edge detection, etc.&lt;/p&gt;&#xA;&lt;p&gt;Usually, in deep learning models where things are trained from scratch, the filters are randomly initialized and then learned by the model's training scheme. For the most part, without using any of the well-known kernels mentioned above.&lt;/p&gt;&#xA;&lt;p&gt;Clarification on for the most part: so filters aren't initialized with the goal of knowing exactly which feature it will activate on, but what will assist the training procedure. Recently, people have even trained ResNets with or without &lt;em&gt;batch normalization&lt;/em&gt; by finding good initial points -- it's an ongoing field of research.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2020-12-11T14:42:40.140" LastActivityDate="2020-12-11T14:42:40.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12512" PostTypeId="2" ParentId="12510" CreationDate="2019-05-24T20:39:44.520" Score="4" Body="&lt;h3&gt;What are filters in image processing?&lt;/h3&gt;&#xA;&lt;p&gt;In the context of image processing (and, in general, signal processing), the &lt;em&gt;kernels&lt;/em&gt; (also known as &lt;em&gt;filters&lt;/em&gt;) are used to perform some specific operation on the image. For example, you can use a &lt;a href=&quot;https://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian filter&lt;/a&gt; to smooth the image (including its edges).&lt;/p&gt;&#xA;&lt;h3&gt;What are filters in CNNs?&lt;/h3&gt;&#xA;&lt;p&gt;In the context of convolutional neural networks (CNNs), the filters (or kernels) are the &lt;strong&gt;learnable&lt;/strong&gt; parameters of the model.&lt;/p&gt;&#xA;&lt;p&gt;Before training, the kernels are usually randomly initialised (so they are not usually hardcoded). During training, depending on the loss or error of the network (according to the loss function), the kernels (or filters) are updated, so that to minimise the loss (or error). After training, they are typically fixed. Incredibly, the filters learned by CNNs can be similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gabor_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gabor filter&lt;/a&gt; (which is thought to be related to our visual system [&lt;a href=&quot;http://www.cns.nyu.edu/%7Etony/vns/readings/olshausen-field-1996.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]). See &lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;figure 9.19 of chapter 9 (p. 365) of the Deep Learning book by Goodfellow et al&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The number of kernels that are applied to a given input (and more than one kernel is often applied) in a CNN is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyper-parameter&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;What are the differences and similarities?&lt;/h3&gt;&#xA;&lt;p&gt;In both contexts, the words &amp;quot;kernel&amp;quot; and &amp;quot;filter&amp;quot; are roughly synonymous, so they are often used interchangeably. Furthermore, in both cases, the kernels are related to the &lt;a href=&quot;https://ai.stackexchange.com/a/22000/2444&quot;&gt;convolution (or cross-correlation) operation&lt;/a&gt;. More specifically, the application of a filter, which is a function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, to an input, which is another function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, is equivalent to the convolution of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. In mathematics, this is often denoted by &lt;span class=&quot;math-container&quot;&gt;$f \circledast h = g$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt; is the convolution operator and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the result of the convolution operation and is often called the &lt;em&gt;convolution&lt;/em&gt; (of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;). In the case of image processing, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the filtered image. In the case of CNNs, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is often called an &lt;a href=&quot;https://stats.stackexchange.com/a/292064/82135&quot;&gt;activation map&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;Take a look at &lt;a href=&quot;https://ai.stackexchange.com/a/21967/2444&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/21401/2444&quot;&gt;this&lt;/a&gt; answers for more details about CNNs and the convolution operation, respectively.&lt;/p&gt;&#xA;&lt;p&gt;You may also want to have a look at &lt;a href=&quot;http://cs231n.github.io/understanding-cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visualizing what ConvNets learn&#xA;&lt;/a&gt; for some info about the visualization of the kernels learned during the training of a CNN.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-11T15:13:30.177" LastActivityDate="2020-12-11T15:13:30.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12513" PostTypeId="1" AcceptedAnswerId="12515" CreationDate="2019-05-24T21:23:13.537" Score="1" ViewCount="379" Body="&lt;p&gt;If I use a desktop PC with a GPU, how long it might take to train face recognition deep neural network on let's say dataset of 2.6 million images and 2600 identities? I guess it should depend on various properties (e.g., type of the DNN). But I am just looking for a rough estimation. Is it a matter of hours/days or years?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="25902" LastEditorUserId="1671" LastEditDate="2019-10-15T19:23:51.960" LastActivityDate="2019-10-15T19:23:51.960" Title="How long it takes to train face recognition deep neural network? (rough estimation)" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;hardware&gt;&lt;facial-recognition&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12515" PostTypeId="2" ParentId="12513" CreationDate="2019-05-25T03:44:05.007" Score="2" Body="&lt;p&gt;Training time depends on a lot of parameters. Some of them are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Size of each image (resolution)&lt;/li&gt;&#xA;&lt;li&gt;Color/Monochrome image (color image has 3 times data if you consider RGB image)&lt;/li&gt;&#xA;&lt;li&gt;Like you mentioned on the type of DNN.&#xA;No. of layers of DNN.&#xA;No. of neurons in each layer. &lt;/li&gt;&#xA;&lt;li&gt;Total no. of images in the dataset. (2.6 million here)&lt;/li&gt;&#xA;&lt;li&gt;GPU you are using (you didn't mention which GPU you are using. There are GPUs with wide range of capabilities, to predict the time, you need to know the exact specs of GPU). &lt;/li&gt;&#xA;&lt;li&gt;Training time also depends on RAM on your machine and also how fast host PC can transfer data to GPU for processing.&lt;/li&gt;&#xA;&lt;li&gt;Since you mentioned face recognition, i am assuming you are using CNN, but if you again use fully connected network, training time will change and will obviously increase manifold.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Your task of classification and your database is mostly similar to ILSVRC that uses imagenet database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Making some reasonable assumptions for the parameters you didn't mention, i feel your task is similar to ILSVRC and i am predicting the training time will be a few days.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are the links which mention time of training for ILSVRC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://mxnet-tqchen.readthedocs.io/en/latest/tutorials/imagenet_full.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mxnet-tqchen.readthedocs.io/en/latest/tutorials/imagenet_full.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are the details of imagenet database for your comparison&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/ImageNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/ImageNet&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-05-25T03:44:05.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12516" PostTypeId="1" AcceptedAnswerId="12517" CreationDate="2019-05-25T05:08:26.117" Score="10" ViewCount="901" Body="&lt;p&gt;I'm trying to learn neural networks by watching &lt;a href=&quot;https://www.youtube.com/watch?v=tIeHLnjs5U8&amp;amp;t=17s&quot; rel=&quot;noreferrer&quot;&gt;this series of videos&lt;/a&gt; and implementing a simple neural network in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's one of the things I'm wondering about: I'm training the neural network on sample data, and I've got 1,000 samples. The training consists of gradually changing the weights and biases to make the cost function result in a smaller cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; Should I be changing the weights/biases on every single sample before moving on to the next sample, or should I first calculate the desired changes for the entire lot of 1,000 samples, and only then start applying them to the network?&lt;/p&gt;&#xA;" OwnerUserId="25904" LastEditorUserId="2444" LastEditDate="2020-11-30T00:05:50.993" LastActivityDate="2020-11-30T00:05:50.993" Title="Is neural networks training done one-by-one?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;&lt;stochastic-gradient-descent&gt;&lt;batch-gradient-descent&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2019-11-06T00:44:33.640" ContentLicense="CC BY-SA 4.0" />
  <row Id="12517" PostTypeId="2" ParentId="12516" CreationDate="2019-05-25T06:01:37.007" Score="11" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Should I be changing the weights/biases on every single sample before moving on to the next sample, &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can do this, it is called &lt;em&gt;stochastic&lt;/em&gt; gradient descent (SGD) and typically you will shuffle the dataset before working through it each time.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;or should I first calculate the desired changes for the entire lot of 1,000 samples, and only then start applying them to the network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can do this, it is called &lt;em&gt;batch&lt;/em&gt; gradient descent, or in some cases (especially in older resources) just assumed as the normal approach and called gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each approach offers advantages and disadvantages. In general:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;SGD makes each update sooner in terms of amount of data that has been processed. So you may need less epochs before converging on reasonable values.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SGD does more processing per sample (because it updates more frequently), so is also slower in the sense that it will take longer to process each sample. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SGD can take less advantage of parallelisation, as the update steps mean you have to run each data item serially (as the weights have changed and error/gradient results are calculated for a specific set of weights).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SGD individual steps make typically only very rough guesses at the correct gradients to change weights in. This is both a disadvantage (performance of the NN against the objective on the training set can decrease as well as increase) and an advantage (there is less likelihood of getting stuck in a local stationary point due the &quot;jitter&quot; these random differences cause).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What happens in practice is that most software allows you to compromise between batch processing and single-sample processing, to try and get the best performance and update characteristics. This is called mini-batch processing, which involves:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Shuffling the dataset at the start of each epoch.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Working through the shuffled data, N items per time where N might vary from maybe 10 to 1000, depending on the problem and any constraints on the hardware. A common decision is to process the largest batch size that the GPU acceleration allows to run in parallel.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the update required for each small batch, then apply it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is nowadays the most common update method that most neural network libraries assume, and they almost universally will accept a batch size parameter in the training API. Most of the libraries will still call simple optimisers that do that SGD; technically it is true, the gradients calculated are still somewhat randomised due to not using the full batch, but you may find this called mini-batch gradient descent in some older papers.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-26T07:47:33.887" LastActivityDate="2019-05-26T07:47:33.887" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12518" PostTypeId="2" ParentId="11513" CreationDate="2019-05-25T07:41:00.003" Score="2" Body="&lt;p&gt;Model means you can say a Prototype we make regarding to our task.&#xA;As we first train our model on some observed or you can say &lt;code&gt;bench-marked data&lt;/code&gt; ; called as TRaining phase of model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we apply that model to our problem (test data) you can say in order to evaluate how  much well you have trained your model.&#xA;Training data we use related to our task or use bench marked data for any model of AI.&#xA;hope you find this answer useful .&lt;/p&gt;&#xA;" OwnerUserId="23048" LastActivityDate="2019-05-25T07:41:00.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12523" PostTypeId="1" AcceptedAnswerId="12526" CreationDate="2019-05-25T11:52:33.930" Score="2" ViewCount="176" Body="&lt;p&gt;Im implementing a neural network framework from scratch in C++ as a learning exercise. There is one concept I don't see explained anywhere clearly: How do you go from your last convolutional/pooling/something layer which is &quot;3 dimensional&quot; to your first fully connected layer in the network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many sources say, that you should flatten the data. Does this mean that you should just simply create a &lt;span class=&quot;math-container&quot;&gt;$1D$&lt;/span&gt; vector with a size of &lt;span class=&quot;math-container&quot;&gt;$N*M*D$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$N*M$&lt;/span&gt; is the last conv. layer's size, and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is the number of activation maps in that layer) and put the numbers in it one by one in some arbitrary order?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the case I understand how to propagate further down the line, but how does backprogation work here? Just put the values in reverse order into the activation maps?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read, that you can do this &quot;flattening&quot; as a tensor contraction. How does that work exactly?&lt;/p&gt;&#xA;" OwnerUserId="25910" LastEditorUserId="9947" LastEditDate="2019-05-25T13:04:30.233" LastActivityDate="2019-05-25T13:09:34.400" Title="Convolutional layer to Fully Connected Layer implementation" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;c++&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12524" PostTypeId="1" AcceptedAnswerId="12525" CreationDate="2019-05-25T12:02:33.703" Score="4" ViewCount="56" Body="&lt;p&gt;In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the &#xA;loss function as below make me really confused. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to understand &lt;span class=&quot;math-container&quot;&gt;$x,y \sim p_{data}$&lt;/span&gt; means a sample &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; sampled from original dataset distribution (or &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the ground truth label).&#xA;The loss function in formula 5.101 seems to be correct for my understanding. Actually, the formula 5.101 is derived from 5.100 by adding the regularization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the notation &lt;span class=&quot;math-container&quot;&gt;$x,y \sim \hat{p}_{data}$&lt;/span&gt; in formula 5.96 and 5.100 is really confusing to me whether the loss function is defined correctly (kinda typo error or not). If not so, could you help me to refactor the meaning of two notations, are they similar and correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CnAr7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CnAr7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zzp5P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zzp5P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FIDcS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FIDcS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Many thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="25911" LastEditorUserId="2444" LastEditDate="2019-05-25T12:24:56.213" LastActivityDate="2019-05-25T13:00:33.477" Title="Being confused of distribution notations in Deep Learning book" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12525" PostTypeId="2" ParentId="12524" CreationDate="2019-05-25T12:55:12.567" Score="3" Body="&lt;p&gt;At page 130 of the same book, the author states that &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_\text{data}$&lt;/span&gt; is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Empirical_distribution_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;empirical distribution&lt;/a&gt; defined by the training data. Similarly, at page 129, he states that &lt;span class=&quot;math-container&quot;&gt;$p_\text{data}$&lt;/span&gt; is the true distribution that generates the set &lt;span class=&quot;math-container&quot;&gt;$\mathbb{X} = \{ \boldsymbol{x}^{(1)}, \dots, \boldsymbol{x}^{(m)} \}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_\text{data}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_\text{data}$&lt;/span&gt;? You can think of &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_\text{data}$&lt;/span&gt; as a histogram that is calculated from the set &lt;span class=&quot;math-container&quot;&gt;$\mathbb{X}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_\text{data}$&lt;/span&gt; as the true distribution from which the elements in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{X}$&lt;/span&gt; are drawn. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The subscript &lt;span class=&quot;math-container&quot;&gt;${\boldsymbol{x}, y \sim \hat{p}_\text{data}}$&lt;/span&gt; in the expectation &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\boldsymbol{x}, y \sim \hat{p}_\text{data}}$&lt;/span&gt; indicates that the expectation is taken with respect to the samples drawn from the empirical distribution &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_\text{data}$&lt;/span&gt;. In other words, you will optimise the objective function &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; using the training data. Have a look at &lt;a href=&quot;https://stats.stackexchange.com/q/72613/82135&quot;&gt;this question&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The subscript &lt;span class=&quot;math-container&quot;&gt;${\boldsymbol{x}, y \sim p_\text{data}}$&lt;/span&gt; in the expectation of formula &lt;span class=&quot;math-container&quot;&gt;$5.101$&lt;/span&gt; is a typo. In fact, in this &lt;a href=&quot;https://www.deeplearningbook.org/contents/ml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;online version of the book&lt;/a&gt;, at page 151, the subscript of the expectation is  &lt;span class=&quot;math-container&quot;&gt;${\boldsymbol{x}, y \sim \hat{p}_\text{data}}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-25T13:00:33.477" LastActivityDate="2019-05-25T13:00:33.477" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12526" PostTypeId="2" ParentId="12523" CreationDate="2019-05-25T13:03:09.793" Score="1" Body="&lt;p&gt;Yes, you are correct (I think it is quite easily implementable in C++ with pointers). The arbitrary order is to be maintained though, since Fully Connected Neural Nets are &quot;&lt;a href=&quot;https://stats.stackexchange.com/questions/208936/what-is-translation-invariance-in-computer-vision-and-convolutional-neural-netwo&quot;&gt;Translationally Invariant&lt;/a&gt;&quot; i.e. you have to make sure if pixel &lt;span class=&quot;math-container&quot;&gt;$(1,5,6)$&lt;/span&gt; is being supplied to node &lt;span class=&quot;math-container&quot;&gt;$38$&lt;/span&gt; or being indexed as &lt;span class=&quot;math-container&quot;&gt;$37$&lt;/span&gt; as a single datapoint to be input to a Fully Connected Neural Network, then from then on it must be fixed (cannot put say pixel &lt;span class=&quot;math-container&quot;&gt;$(1,6,5 )$&lt;/span&gt; in node &lt;span class=&quot;math-container&quot;&gt;$38$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backpropagation works the same as it always works, it is tough to give a verbal explanation so I will give you this picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mzTFi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mzTFi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, basically if you visualise like this you understand you have to differentiation will propagate, the &quot;flattening&quot; is only reshaping the value lookup table it is not changing the way the values affect final loss, so if you take gradient w.r.t each values and then convert it back to a &lt;span class=&quot;math-container&quot;&gt;$3D$&lt;/span&gt; map same way as before and then propagate the gradients as you were doing in previous layers.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-05-25T13:09:34.400" LastActivityDate="2019-05-25T13:09:34.400" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12527" PostTypeId="1" CreationDate="2019-05-25T13:44:26.573" Score="2" ViewCount="87" Body="&lt;p&gt;I myself am not new to NLP, but for some reason I am unable to grasp purity of BERT. I have seen a ton of blogs, github repos, but none could clarify BERT usage to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be helpful if you could provide two things :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A clear implementation of BERT, preferably in ipython notebook.&lt;/li&gt;&#xA;&lt;li&gt;Some papers on BERT excluding the original paper by google.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25512" LastEditorUserId="25512" LastEditDate="2019-12-18T07:09:22.760" LastActivityDate="2019-12-18T07:09:22.760" Title="Bert super easy implementation" Tags="&lt;natural-language-processing&gt;&lt;python&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12528" PostTypeId="1" CreationDate="2019-05-25T14:51:20.180" Score="2" ViewCount="20" Body="&lt;p&gt;I am new to machine learning and I would like to seek some advice/help for directions on implementing a binary classification for a series of data and tell if it is a straight line or a not? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example I have the following data e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training data:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0,2,4  -- straight line &lt;/li&gt;&#xA;&lt;li&gt;0,10,5  -- not a straight line&lt;/li&gt;&#xA;&lt;li&gt;0,99,99  -- not a straight line&lt;/li&gt;&#xA;&lt;li&gt;0,1,2  -- straight line&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My Validation data would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0,60,120 -- straight line&lt;/li&gt;&#xA;&lt;li&gt;0,120,60 -- not a straight line&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25913" LastActivityDate="2019-05-25T14:51:20.180" Title="Binary classification for a series of data (using Keras) to tell if it is a straight line or not a straight line" Tags="&lt;machine-learning&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12529" PostTypeId="1" CreationDate="2019-05-25T14:53:36.397" Score="1" ViewCount="282" Body="&lt;p&gt;I'm working on an Advantage A2C implementation, and I just finished creating the value network &lt;span class=&quot;math-container&quot;&gt;$\hat{V_{\phi}}$&lt;/span&gt;. I train this network with the standard MSE loss of discounted rewards-to-go:&lt;span class=&quot;math-container&quot;&gt;$$\|\hat{V_{\phi}}(s_{t'}) - \sum_{t=t'}^{T}\gamma^{t-t'} r(s_t, a_t)\|^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to evaluate and assess the performance and ability of the value network as I train, especially to see how that relates and interacts with the changes and improvements in the policy, however I'm not sure how to do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first instinct was to track the loss after each batch of experiences, but this doesn't work. As the policy improves, episodes last longer, and the value loss increases. I understand why this happens, as it is harder to predict the rewards-to-go when the length of the future is more undetermined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this, I tried dividing the loss that I'm computing by the total number of steps in the batch of episodes. However, the loss is still increasing as the policy improves (as the episodes get longer). Why is this still happening? Then, is there anything I can do to get a better assessment of the quality of the value network?&lt;/p&gt;&#xA;" OwnerUserId="25732" LastActivityDate="2019-10-28T09:23:15.560" Title="A2C Critic Loss Interpretation" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12531" PostTypeId="1" CreationDate="2019-05-25T18:24:22.527" Score="2" ViewCount="50" Body="&lt;p&gt;In simple words, what is the prediction accuracy? What is it based on? How does it help? When is it used?&lt;/p&gt;&#xA;" OwnerUserId="23529" LastEditorUserId="2444" LastEditDate="2019-05-25T19:29:46.883" LastActivityDate="2019-05-25T19:29:46.883" Title="What is the prediction accuracy?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12533" PostTypeId="2" ParentId="12531" CreationDate="2019-05-25T19:07:50.407" Score="1" Body="&lt;p&gt;The accuracy of a model is defined as the number of correct &lt;a href=&quot;https://en.wikipedia.org/wiki/Prediction&quot; rel=&quot;nofollow noreferrer&quot;&gt;predictions&lt;/a&gt; divided by the total number of predictions. The expression &quot;prediction accuracy&quot; refers to the accuracy of the model at doing predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that you observe the sequence of numbers &lt;span class=&quot;math-container&quot;&gt;$1, 1, 2, 3$&lt;/span&gt;. What would be a likely next number of this sequence? The operation of guessing this number is called a prediction. These are numbers of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Fibonacci_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fibonacci sequence&lt;/a&gt;, so a model could predict &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; as the next number of this sequence. However, it could also predict e.g. &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; because the complete sequence could be &lt;span class=&quot;math-container&quot;&gt;$1, 1, 2, 3, 2, 1, 1$&lt;/span&gt;. Which prediction is correct? It depends on the problem. Do we want the model to predict the numbers of the Fibonacci sequence or of another sequence? Suppose that we want the model to learn the Fibonacci sequence and the models predicts another number different than &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;, then this will be considered an incorrect prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A model that produces incorrect predictions is useless. So, we want models that have high accuracy. However, accuracy is not always the best measure of a model's &lt;a href=&quot;https://stats.stackexchange.com/q/385312/82135&quot;&gt;performance&lt;/a&gt;. See e.g. &lt;a href=&quot;https://stats.stackexchange.com/q/312780/82135&quot;&gt;Why is accuracy not the best measure for assessing classification models?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several related terms and concepts, such as inference, forecasting or deduction. They are sometimes used interchangeably, even though they might have slightly different meanings or definitions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-25T19:17:45.617" LastActivityDate="2019-05-25T19:17:45.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12534" PostTypeId="1" CreationDate="2019-05-25T23:17:17.500" Score="0" ViewCount="107" Body="&lt;p&gt;I have been offered a job with a software firm which does  high level custom software. I believe i am the first person there with some AI background and that very soon now many of their projects will require some AI.&#xA;I am not a lead, i don't think i have the skills, technical or managerial. Is it possible for a software development team with one person with some AI background, to learn to implement various AI techniques succesfully?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own background is that i studied AI but with a focus on cognition (ie: use AI to better understand cognition, human or otherwise). Afterwards i learned the craft of software development (Java webapps) and worked in this field until i found a PhD position. &#xA;During my PhD i dabbled a bit with machine learning, academically. Soon i focussed again on the more cognition oriented approach, and looked for ways of formalising knowledge from literature so it could be used inside a custom classifier to better analyse a new dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The funding ran out before i could finish and i returned to software development.&#xA;Ideally i would love a job where i could join a AI-oriented team, and learn more about production-level AI on the job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The job i have been offered is with a firm that makes custom software. They are starting a new project which should employ AI to help human operators fulfil a complex task. Another project uses very costly calculation and one ML component. The data it works on is public and google has announced it will have a go at the data. I expect that when google is done, there will be a scramble to use ML for all the components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general i think that 5 years from now all their software projects will require some form of AI. If i were running this software company i would go and look for a lead developer who had proven themselves in AI. Then i would task this person with interviewing to find 2 'luitenants'.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the chances of a software development team including one&#xA;  member who knows about machine learning in an academic context,&#xA;  getting up-to-speed and delivering production level software&#xA;  incorporating AI?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="25929" LastEditorUserId="7800" LastEditDate="2019-05-29T17:51:34.180" LastActivityDate="2019-05-30T05:09:09.197" Title="Single AI-person in software firm about to adopt AI, good idea?" Tags="&lt;machine-learning&gt;&lt;software-development&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12535" PostTypeId="2" ParentId="11803" CreationDate="2019-05-26T04:16:46.237" Score="0" Body="&lt;p&gt;One way to think of the difference between search and learning is that search usually entails a search key, and an algorithm hunts through the structure looking for a match between the key and an already-existing item. Whereas learning is the creation of the structure in the first place. But search and learning are related in that on receipt of an input (say from one or more sensors) the structure is initially searched to see if the input already exists, but if it doesn't then current input (when certain conditions are met) is added to the structure, and learning follows a failure of search. &lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2019-05-26T04:16:46.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12536" PostTypeId="2" ParentId="10272" CreationDate="2019-05-26T04:31:07.007" Score="0" Body="&lt;p&gt;Turing had much difficulty explaining how a computer could learn, and in his 1950 paper  asked, How could a machine learn? Its possible behavior is completely defined by its rules of operation (program), whatever the machine's history (past, present, future) might be. His proposed solution was ephemerally valid rules, whatever they might be - and he doesn't say. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So learning can be understood as the machine acquiring new behaviors but not by the behaviors being programmed into the machine by a human. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps a better way to look at this is causally. A human can define the causation (possible behavior) of a computer by programming it into the machine. But this is a case of the human using their knowledge to define how the machine will react to situations. Learning is the case where the machine itself acquires now behaviors or possible behaviors (not by a human using their knowledge of the world). And any such alleged intrinsic learning can easily be tested. If it helps the machine survive in a complex and hostile world, then it really is learning.  &lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2019-05-26T04:31:07.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12540" PostTypeId="1" CreationDate="2019-05-26T13:32:12.817" Score="2" ViewCount="409" Body="&lt;p&gt;Copy from my reddit post: &#xA;(Sorry if this does not fit here, please tell me and i delete it)&#xA;Help regarding &#xA;I'm working on an implementation of PPO, which i plan to use in my (Bachelors) Thesis. To test whether my implementation works, i want to use the LunarLanderContinuous-v2 Environment. Now my implementation seems to work just fine, but plateues much too early - At an average reward of ~ -1.8 reward per timestep, where the goal should be somewhere around ~ +2,5 reward per timestep. As the implementation generally learns i am somewhat confused, as to why it then pleateus so early.&#xA;Some details regarding my implementation, also here is the github repo:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I use parallelized environments via openai's subproc_vecenv&lt;/li&gt;&#xA;&lt;li&gt;I use the Actor Critic Version of PPO&lt;/li&gt;&#xA;&lt;li&gt;I use Generalized Advantage Estimation as my Advantage term&lt;/li&gt;&#xA;&lt;li&gt;I only use finished runs (every run used in training has reached a terminal state)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Even though Critic loss in the graphic below looks small it is actually rather large, as the rewards are normalized and therefor the value targets are actually rather small&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Critic seemingly predicts a value independent of the state it is fed - that is it predicts for every state just the average over all the values. That seems like harsh underfitting, which is weird as the network is already rather large for the problem in my opinion. But this seems to be the most likely cause for the problem in my opinion.&lt;a href=&quot;https://i.stack.imgur.com/Yu4Bu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yu4Bu.png&quot; alt=&quot;Training Progress&quot;&gt;&lt;/a&gt;&#xA;Edit1: Added image&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25945" LastEditorUserId="25945" LastEditDate="2019-05-27T21:47:01.047" LastActivityDate="2019-05-27T21:47:01.047" Title="Implementation of PPO - Value Loss not converging, return plateauing" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;advantage-actor-critic&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12542" PostTypeId="1" CreationDate="2019-05-26T19:07:36.687" Score="3" ViewCount="37" Body="&lt;p&gt;Given enough experiment data on time taken for objects to fall to earth from different heights, one can create various models that will accurately predict the time it will take for an object falling at any height (in the inner the atmosphere, this is a toy example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this simple example the model is deterministic, it will always produce an output given an input regardless of the amount of data over a small threshold—something akin to Newton’s gravity equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try modelling something like the stock-market (the other extreme end of the scale) and the predictions will never reliably converge on a predictable accurate model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any way of knowing whether your domain will yield a deterministic or non-deterministic model or not?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2019-05-26T19:07:36.687" Title="How to know when a Environment will yield a deterministic model" Tags="&lt;models&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12543" PostTypeId="1" CreationDate="2019-05-26T19:23:58.417" Score="2" ViewCount="53" Body="&lt;p&gt;In every day life, it seems that we all have various habits and actions that we perform. For example, we wake up and check our email/facebook etc. on our phones. We don't look at are current state right now, and consider the values of all the possible future trajectories. We basically choose the action that maximizes our &quot;reward&quot; at our current state. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question.&lt;/strong&gt; Is it practical to randomly initialize actions &lt;span class=&quot;math-container&quot;&gt;$a \in A$&lt;/span&gt;, states &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; and a policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and update this according to some algorithm (e.g. REINFORCE, exploration, etc.) to achieve some desired&#xA;  goal in your life? This could be done, for example, by uniformly sampling a random number in the interval &lt;span class=&quot;math-container&quot;&gt;$(0,1)$&lt;/span&gt; and acting according to a policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;. For example, suppose your goal is to get married,&#xA;  get a new house etc. What would be appropriate return functions in&#xA;  this case? Return is usually defined as the immediate reward plus the&#xA;  discounted cumulative future rewards. But is this the right definition for&#xA;  practical problems like marriage/dating, buying a house, etc.? Would you define return as &lt;span class=&quot;math-container&quot;&gt;$R_{\text{total}} = R_{\text{marriage}}+R_{\text{buying house}}$&lt;/span&gt; in our example, where each of those individual returns are typical immediate plus discounted rewards and try to maximize that? Or would it be better to maximize &lt;span class=&quot;math-container&quot;&gt;$R_{\text{marriage}}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{\text{buying house}}$&lt;/span&gt; individually?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="25953" LastEditorUserId="25953" LastEditDate="2019-05-26T20:19:04.257" LastActivityDate="2019-05-26T20:19:04.257" Title="Reinforcement Learning in Real Life/Practical Terms" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12544" PostTypeId="1" AcceptedAnswerId="12545" CreationDate="2019-05-26T22:04:46.983" Score="3" ViewCount="127" Body="&lt;p&gt;I'm interested in ant colony optimization algorithms and bee algorithms,but I'm confused what are the applications of these algorithms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you suggest me some examples of applications can I work on?&lt;/p&gt;&#xA;" OwnerUserId="25940" LastEditorUserId="2444" LastEditDate="2019-05-26T23:20:11.043" LastActivityDate="2019-05-26T23:47:52.917" Title="What are the applications of ant colony optimization algorithms?" Tags="&lt;applications&gt;&lt;swarm-intelligence&gt;&lt;ant-colony-optimization&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12545" PostTypeId="2" ParentId="12544" CreationDate="2019-05-26T23:09:45.260" Score="2" Body="&lt;p&gt;The first ant colony optimisation algorithm was introduced by &lt;a href=&quot;http://iridia.ulb.ac.be/~mdorigo/HomePageDorigo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marco Dorigo&lt;/a&gt; in the report &lt;em&gt;Positive Feedback as a Search Strategy&lt;/em&gt; (1991) and his PhD thesis &lt;em&gt;Optimization, Learning and Natural Algorithms&lt;/em&gt; (1992). He's still one of the leading figures in the field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Swarm_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;swarm intelligence&lt;/a&gt; (having also written or co-written several papers and books). Another important person that contributed to ACO algorithms is &lt;a href=&quot;http://people.idsia.ch/~luca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Luca Gambardella&lt;/a&gt; (co-director of &lt;a href=&quot;http://www.idsia.ch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IDSIA&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ACO algorithms. They are all based on the way real ants behave, that is, by leaving a substance called &quot;pheromone&quot; on the ground in order to communicate. More specifically, the amount of pheromone is associated with value (e.g. food): more pheromone means more value. (It should now be clear the reason behind the queues real ants form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A list of ACO algorithms can be found at &lt;a href=&quot;http://iridia.ulb.ac.be/~mdorigo/ACO/publications.html#THIRD&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://iridia.ulb.ac.be/~mdorigo/ACO/publications.html&lt;/a&gt;. For reproducibility, here's a (non-exhaustive) list:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ant System&lt;/li&gt;&#xA;&lt;li&gt;Elitist Ant System&lt;/li&gt;&#xA;&lt;li&gt;Ant-Q&lt;/li&gt;&#xA;&lt;li&gt;Ant Colony System&lt;/li&gt;&#xA;&lt;li&gt;Max-Min Ant System&lt;/li&gt;&#xA;&lt;li&gt;Rank-based Ant System&lt;/li&gt;&#xA;&lt;li&gt;ANTS&lt;/li&gt;&#xA;&lt;li&gt;Hyper Cube - ACO&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ACO algorithms have been applied to &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial&lt;/a&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/NP-completeness&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP-complete&lt;/a&gt;&lt;/strong&gt; (e.g. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;travelling salesman problem&lt;/a&gt;) problems. ACO algorithms are thus a collection of meta-heuristic and probabilistic algorithms (in the same family of &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulated annealing&lt;/a&gt;) to tackle often considered &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractable problems&lt;/a&gt;. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;related Wikipedia article&lt;/a&gt; contains a more exhaustive section dedicated to the applications of these algorithms. ACO algorithms are often combined with &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_search_(optimization)&quot; rel=&quot;nofollow noreferrer&quot;&gt;local search algorithms&lt;/a&gt; (like the 2-opt or 3-opt).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you to start with the travelling salesman problem, which was the first application of these algorithms. You can have a look at the reference implementations at &lt;a href=&quot;http://iridia.ulb.ac.be/~mdorigo/ACO/aco-code/public-software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://iridia.ulb.ac.be/~mdorigo/ACO/aco-code/public-software.html&lt;/a&gt;, where you can also find software to solve specific tasks (not just the TSP, such as maximum clique problems).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-26T23:47:52.917" LastActivityDate="2019-05-26T23:47:52.917" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12546" PostTypeId="5" CreationDate="2019-05-26T23:21:25.393" Score="0" Body="&lt;p&gt;For more info, have a look at &lt;a href=&quot;http://iridia.ulb.ac.be/~mdorigo/ACO/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://iridia.ulb.ac.be/~mdorigo/ACO/&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-29T17:51:13.640" LastActivityDate="2019-05-29T17:51:13.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12547" PostTypeId="4" CreationDate="2019-05-26T23:21:25.393" Score="0" Body="For questions related to the ant colony optimization (ACO) algorithms, which are population-based and stochastic metaheuristics that can be used to find approximate solutions to difficult optimization problems, such as the TSP. In ACO, artificial ants (the agents) iteratively search for good solutions to a given optimization problem. To apply ACO, the optimization problem is transformed into the problem of finding the best path on a weighted graph." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T01:39:53.077" LastActivityDate="2019-11-01T01:39:53.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12548" PostTypeId="1" AcceptedAnswerId="12582" CreationDate="2019-05-27T06:28:42.280" Score="2" ViewCount="96" Body="&lt;p&gt;There is this problem I have encountered, I was trying to classify the pixels from input image into classes, sort of like segmentation, using a encoder-decoder CNN. The “interested” pixels usually locate in the top right corner of the input image, but the input images are too big, which I have to slice them in patches, by doing this, each input patch loses its “which region of the whole picture it’s from” information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using pytorch, I thought of manually add this patch location info into the input, but then it will be convoluted, which does make sense to me since it's not a part of an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new to this, not sure if I'm thinking the whole thing right, how should I manually add this info into the input correctly or if there is some keywords I can do some researches, in order to let the CNN taking position into account? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="25963" LastActivityDate="2019-05-29T12:42:35.913" Title="How to add some data input in a CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;autoencoders&gt;&lt;pytorch&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12551" PostTypeId="1" CreationDate="2019-05-27T11:45:00.813" Score="1" ViewCount="336" Body="&lt;p&gt;I'm about to create an OpenAI Gym environment for a flight simulator. I'm wondering, how to cope with the fact, that the result and reward for some action needs a considerable time to advance through the system due to the inherent time constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the easy example Gym-environments (e. g. cartpole, or some games) the step can anstantly be executed and the resulting reard can be calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my continuous control system (aka flight simulator), There is some reaction time needed, until I can calculate the result from my action. E. g. When I pull the stick, it takes some time, until the aircraft lifts it's nose. So there is a considerable delay (maybe in the seconds ballpark) from commanding the action to the environment, and the earliest observation of that result and it's corresponding reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I cope with that. As far as I understood, the &lt;code&gt;env.step(action)&lt;/code&gt; function blocks till it comes back with a new observation and a corresponding reward.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I cope with long lasting reward caclulations?&lt;/li&gt;&#xA;&lt;li&gt;Is it possible to have overlapping actions somehow? E. g. command a new action every 100ms, but get the reward for that action only 1 second later. In this case there would be always 10 rewards pending.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I hope I made my point clear. Don't hesitate to ask for further details in the comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hints are welcome. Is there anything to read out in the wild dealing with a similar issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Felix&lt;/p&gt;&#xA;" OwnerUserId="25972" LastActivityDate="2019-05-27T15:08:59.973" Title="OpenAI Gym interface when reward calculation is delayed? (continuous control with considerable reaction time)" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12552" PostTypeId="1" AcceptedAnswerId="12572" CreationDate="2019-05-27T11:49:55.327" Score="3" ViewCount="97" Body="&lt;p&gt;I am trying to formulate and solve the following problem of image mutation. Suppose I am trying to insert an object image into a &quot;background&quot; image of several objects, and I will need to look for a &quot;sweet spot&quot; to insert the image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WTxFV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WTxFV.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am tentatively trying to formulate the problem into a reinforcement learning process, with the following elements:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;0. initial stage:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a background image where the location of objects within the image has been marked (let's suppose we have a perfect object detector)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;another image of a new object, let's say, a human&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. action space:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;location (x, y) for the object image to be inserted; in that sense the action space is &lt;strong&gt;quite&lt;/strong&gt; &lt;strong&gt;large&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. environment:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;each step I will have a new image to &quot;learn from&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An oracle function &lt;strong&gt;F&lt;/strong&gt; returns 1 or 0 (roughly one computation of &lt;strong&gt;F&lt;/strong&gt; takes 30 seconds). &#xA; This function tells me the latest synthesized image hits the &quot;sweet spot&quot; or not (1 means hit). If so, I will stop the search and return the image.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. constraint:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the newly inserted object shouldn't overlap with the original objects in the figure.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While my gut feeling is that this problem is somehow similar to the classic &quot;maze escape&quot; problem which can be solved well with reinforcement learning, the action space seems &lt;strong&gt;quite large&lt;/strong&gt; in this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here are my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In case I would like to formulate this &quot;beautify&quot; image problem into a &quot;deep&quot; reinforcement learning problem, how can I learn from such large action space? Or is it really suitable for a reinforcement learning process? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I somehow subsume the &quot;non-overlapping&quot; constraint into the oracle function &lt;strong&gt;F&lt;/strong&gt;? If so, how should I decide the reward score? Any principled or empirical way of deciding so? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-05-28T20:01:54.500" Title="beautify an image with reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12553" PostTypeId="2" ParentId="12516" CreationDate="2019-05-27T12:56:09.767" Score="1" Body="&lt;p&gt;Ideally, you need to update weights by going over all the samples in the dataset. This is called as &lt;strong&gt;Batch Gradient Descent&lt;/strong&gt;. But, as the no. of training examples increases, the computation becomes huge and training will be very slow. With the advent of deep learning, training size is in millions and computation using all training examples is very impractical and very slow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where, two optimization techniques became prominent. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Mini-Batch Gradient Descent&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Stochastic Gradient Descent (SGD)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In mini-batch gradient descent, you use a batch size that is considerably less than total no. of training examples and update your weights after passing through these examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In stochastic gradient descent, you update the weights after passing through each training example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to advantages and disadvantages of the three methods we discussed.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Batch gradient descent gradually converges to the global minimum but&#xA;it is slow and requires huge computing power.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stochastic gradient descent converges fast but not to the global&#xA;minimum, it     converges somewhere near to the global minimum and&#xA;hovers    around that point, but doesn't converge ever to the global&#xA;minimum.    But, the converged point in Stochastic gradient descent&#xA;is good    enough for all practical purposes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mini-Batch gradient is a trade-off the above two methods. But, if you&#xA;have a vectorized implementation of the weights updation and you&lt;br&gt;&#xA;are training with a multi-core setup or submitting the training to&lt;br&gt;&#xA;multiple machines, this is the best method both in terms of time for &#xA;training and convergence to global minimum.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can plot the cost function, w.r.t the no. of iterations to understand the difference between convergence in all the 3 types of gradient descent. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Batch gradient descent plot falls smoothly and slowly and gets stabilized and&#xA;gets to global minimum. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stochastic gradient descent plot will have    oscillations, will fall&#xA;fast but hovers around global minimum.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are some blogs where there is detailed explanation of advantages, disadvantages of each method and also graphs of how cost function changes for all the three methods with iterations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://adventuresinmachinelearning.com/stochastic-gradient-descent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://adventuresinmachinelearning.com/stochastic-gradient-descent/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/gentle-introduction-mini-batch-gradient-descent-configure-batch-size/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/gentle-introduction-mini-batch-gradient-descent-configure-batch-size/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="20760" LastEditDate="2019-05-27T13:01:40.860" LastActivityDate="2019-05-27T13:01:40.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12556" PostTypeId="2" ParentId="12513" CreationDate="2019-05-27T14:41:54.490" Score="1" Body="&lt;p&gt;YELP Dataset (200k images) used to take 5 hr for training to identify Five (5) classes on GPU - Nvidia 1080 Ti with 11 GB RAM. So I guess in your case it will take days. Again it will depend on the type of your GPU configuration and type of Architecture you will be using.  &lt;/p&gt;&#xA;" OwnerUserId="25978" LastActivityDate="2019-05-27T14:41:54.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12557" PostTypeId="2" ParentId="12551" CreationDate="2019-05-27T15:08:59.973" Score="1" Body="&lt;p&gt;It's not on your end, as a creator of flight simulator, to worry about what action should get the credit for the reward that happened some time after the action was taken. You should return the reward when the actual event happens not when the action that caused it happened. It's the job of the reinforcement learning agent to figure that out. For example if you want to give a reward when airplane nose is at 45 degrees from the horizontal axis, you should return the reward when that event actually happens, RL agent should figure out that the crucial action happened some time ago. This may be difficult for the agent but its up to the user to use a proper algorithm and proper exploration strategy to solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-05-27T15:08:59.973" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12558" PostTypeId="1" AcceptedAnswerId="12559" CreationDate="2019-05-27T18:48:30.883" Score="2" ViewCount="329" Body="&lt;p&gt;What is the definition of machine learning? What are the advantages of machine learning? &lt;/p&gt;&#xA;" OwnerUserId="24095" LastEditorUserId="2444" LastEditDate="2020-11-14T15:25:35.057" LastActivityDate="2021-03-11T10:01:25.157" Title="What is machine learning?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12559" PostTypeId="2" ParentId="12558" CreationDate="2019-05-27T19:04:42.560" Score="5" Body="&lt;h3&gt;What is machine learning?&lt;/h3&gt;&#xA;&lt;p&gt;Machine learning (ML) has been defined by multiple people in similar (or related) ways.&lt;/p&gt;&#xA;&lt;p&gt;Tom Mitchell, in his book &lt;a href=&quot;http://www.cs.ubbcluj.ro/%7Egabis/ml/ML-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt; (1997), defines an ML &lt;strong&gt;algorithm/program&lt;/strong&gt; (or machine learner) as follows.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A computer program is said to &lt;em&gt;learn&lt;/em&gt; from &lt;strong&gt;experience&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; with respect to some class of tasks &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and performance measure &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, if its performance at tasks in &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, as measured by &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, improves with experience &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a quite reasonable definition, given that it describes algorithms such as gradient descent, Q-learning, etc.&lt;/p&gt;&#xA;&lt;p&gt;In his book &lt;a href=&quot;http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning: A Probabilistic Perspective&lt;/a&gt; (2012), Kevin P. Murphy defines the machine learning &lt;strong&gt;field/area&lt;/strong&gt; as follows.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a set of methods that can automatically detect patterns in &lt;strong&gt;data&lt;/strong&gt;, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data!)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Without referring to algorithms or the field, &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shai Shalev-Shwartz and Shai Ben-David&lt;/a&gt; define machine learning as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The term machine learning refers to the automated detection of meaningful patterns in &lt;strong&gt;data&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In all these definitions, the core concept is &lt;strong&gt;data&lt;/strong&gt; or &lt;strong&gt;experience&lt;/strong&gt;. So, any &lt;a href=&quot;https://edutechlearners.com/download/Introduction_to_algorithms-3rd%20Edition.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithm&lt;/a&gt; that &lt;em&gt;automatically&lt;/em&gt; detects patterns in &lt;strong&gt;data&lt;/strong&gt; (of any form, such as textual, numerical, or categorical) to solve some task/problem (which often involves more data) is a (machine) learning algorithm.&lt;/p&gt;&#xA;&lt;p&gt;The tricky part of this definition, which often causes a lot of misconceptions about what ML is or can do, is probably &lt;em&gt;automatically&lt;/em&gt;: this does not mean that the learning algorithm is completely autonomous or independent from the human, given that the human, in most cases, still needs to define a &lt;em&gt;performance measure&lt;/em&gt; (and other parameters, including the learning algorithm itself) that guides the learning algorithm towards a set of solutions to the problem being solved.&lt;/p&gt;&#xA;&lt;p&gt;As a &lt;a href=&quot;https://ai.stackexchange.com/a/26664/2444&quot;&gt;field&lt;/a&gt;, ML could be defined as the study and application of ML algorithms (as defined by Mitchell's definition).&lt;/p&gt;&#xA;&lt;h3&gt;Sub-categories&lt;/h3&gt;&#xA;&lt;p&gt;Murphy and many others often divide machine learning into three main sub-categories&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;supervised learning&lt;/strong&gt; (or predictive), where the goal is to learn a mapping from inputs &lt;span class=&quot;math-container&quot;&gt;$\textbf{x}$&lt;/span&gt; to outputs &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, given a labeled set of input-output pairs&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://mlg.eng.cam.ac.uk/zoubin/papers/ul.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;unsupervised learning&lt;/strong&gt;&lt;/a&gt; (or descriptive), where the goal is to find &amp;quot;interesting patterns&amp;quot; in the data&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;reinforcement learning&lt;/strong&gt;&lt;/a&gt;, which is useful for learning how to act or behave when given an occasional reward or punishment signals&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, there are many other possible sub-categories (or taxonomies) of machine learning techniques, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608014002135&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;deep learning&lt;/strong&gt;&lt;/a&gt; (i.e. the use of neural networks to approximate functions and related learning algorithms, such as gradient descent) or&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/nature14541&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;probabilistic machine learning&lt;/strong&gt;&lt;/a&gt; (machine learning techniques that provide uncertainty estimation)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://academic.oup.com/nsr/article/5/1/44/4093912&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;weakly supervised learning&lt;/strong&gt;&lt;/a&gt; (i.e. SL where labeling information may not be completely accurate)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/papers/OLsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;online learning&lt;/strong&gt;&lt;/a&gt; (i.e. learning from a single data point at a time rather than from a dataset of multiple data points)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These sub-categories can also be combined. For example, deep learning can be performed online or offline.&lt;/p&gt;&#xA;&lt;h3&gt;Related fields&lt;/h3&gt;&#xA;&lt;p&gt;There is also a related field known as &lt;strong&gt;computational&lt;/strong&gt; (or &lt;strong&gt;statistical&lt;/strong&gt;) &lt;strong&gt;learning theory&lt;/strong&gt;, which concerned with the &lt;em&gt;theory of learning&lt;/em&gt; (from a computational and statistical point of view). So, in this field, we are interested in questions like &amp;quot;How many samples do we need to approximately compute this function with a certain error?&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Of course, given that machine learning is a set of algorithms and techniques that are data- or experience-driven, one may wonder what the difference between machine learning and &lt;strong&gt;statistics&lt;/strong&gt; is. In fact, in many cases, they are very similar and ML adopts many statistical concepts, and you may even read on the web that machine learning is just &lt;em&gt;glorified statistics&lt;/em&gt;. ML and statistics often tackle the same problem, but from a different perspective or with slightly different approaches (and the terminology may slightly change from one field to the other). If you are interested in a more detailed explanation of their difference, you could read &lt;a href=&quot;https://www.nature.com/articles/nmeth.4642&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistics versus machine learning&lt;/a&gt; (2018) by Danilo Bzdok et al.&lt;/p&gt;&#xA;&lt;h3&gt;What is machine learning good for?&lt;/h3&gt;&#xA;&lt;p&gt;ML can potentially be used to (at least partially) automate tasks that involve data and pattern recognition, which were previously performed only by humans (e.g. translation from one human language, such as English, to another, such as Italian). However, machine learning cannot automate all tasks: for example, it cannot infer causal relations from the data (which often must be done by humans), unless you include &lt;strong&gt;causal inference&lt;/strong&gt; as part of machine learning. If you are interested in causal inference, you could take a look at the paper &lt;a href=&quot;http://proceedings.mlr.press/v6/pearl10a/pearl10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Causal Inference&lt;/a&gt; by Judea Pearl (&lt;a href=&quot;https://amturing.acm.org/award_winners/pearl_2658896.cfm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Award for his work in causal inference!&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-03-11T10:01:25.157" LastActivityDate="2021-03-11T10:01:25.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12561" PostTypeId="1" CreationDate="2019-05-28T00:52:13.103" Score="1" ViewCount="109" Body="&lt;p&gt;I am building CSP NQueens solver and apply the AC-3 algorithm.But,the domain reduction doesn't occur and gets more search time.How does it happen?Is it a nature of AC-3 algorithm?&lt;/p&gt;&#xA;" OwnerUserId="21779" LastActivityDate="2019-05-28T00:52:13.103" Title="Can AC-3 algorithm solve N-Queens problem?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;problem-solving&gt;&lt;combinatorial-games&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12562" PostTypeId="1" CreationDate="2019-05-28T02:09:26.493" Score="1" ViewCount="30" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Given &lt;span class=&quot;math-container&quot;&gt;$F_1,F_2,..,F_n$&lt;/span&gt; as set of final exams of subjects taken by students &lt;span class=&quot;math-container&quot;&gt;$S_1,..,S_k$&lt;/span&gt; in h slots such that no student takes two exams in a single slot.Here the objective is to maximize the number of exams taken by a student in a single slot.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I reduced the problem to graph coloring problem where the nodes of the graph are the final exams of subjects and edges would be the students.  The chromatic number of graph would be h.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure how to represent edges as they can me multiple also as same exams are taken by multiple students and would it have any impact on constraint satisfaction formulation?&lt;/p&gt;&#xA;" OwnerUserId="25984" LastEditorUserId="25984" LastEditDate="2019-05-28T09:39:02.110" LastActivityDate="2019-05-28T09:39:02.110" Title="Write Constraint Satisfaction Formulation for problem" Tags="&lt;graphs&gt;&lt;graph-theory&gt;&lt;constraint-satisfaction-problems&gt;&lt;satisfiability&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12563" PostTypeId="1" CreationDate="2019-05-28T07:32:34.133" Score="1" ViewCount="37" Body="&lt;p&gt;Is it better to train one neural network for a dispersed labeled data with large number of classes or first classify data by unsupervised learning then train each part by a separate NN?&#xA;I mean by unsupervised learning we help each NN to classify in lower dispersed data with lower number of labels.&#xA;So for test data  the class of data is found by unsupervised learning then the final label is found by the network associated with that class.&#xA;Does this question generally have an answer or it depends on data and needs to be answered in practice?&lt;/p&gt;&#xA;" OwnerUserId="25988" LastActivityDate="2019-05-28T07:32:34.133" Title="One end to end Neural network or many task-specific ones?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12564" PostTypeId="1" CreationDate="2019-05-28T09:16:04.140" Score="2" ViewCount="110" Body="&lt;p&gt;I want to develop a fraud detection application in the stock market Using Blockchain technology, we have some pattern that defines the anomaly for use of supervised machine learning but there is one question remain:&lt;br&gt;&#xA;What is role of machine learning and Blockchain to detect anomaly like a fraud?&lt;/p&gt;&#xA;" OwnerUserId="25452" LastEditorUserId="25452" LastEditDate="2019-06-07T19:36:07.153" LastActivityDate="2019-06-07T19:36:07.153" Title="Application of Blockchain in Fraud detection in stock market" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;anomaly-detection&gt;&lt;blockchain&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12565" PostTypeId="1" CreationDate="2019-05-28T09:45:44.620" Score="5" ViewCount="514" Body="&lt;p&gt;When training a neural network, we often run into the issue of overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is it possible to put overfitting to use? Basically, my idea is, instead of storing a large dataset in a database, you can just train a neural network on the &lt;em&gt;entire&lt;/em&gt; dataset until it overfits as much as possible, then retrieve data &quot;stored&quot; in the neural network like it's a hashing function.&lt;/p&gt;&#xA;" OwnerUserId="23941" LastActivityDate="2019-06-06T11:01:52.903" Title="Is it possible for a neural network to be used to compress data?" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12566" PostTypeId="2" ParentId="12565" CreationDate="2019-05-28T10:26:27.470" Score="6" Body="&lt;p&gt;The &lt;a href=&quot;https://ai.stackexchange.com/a/12504/2444&quot;&gt;&lt;em&gt;auto-encoder&lt;/em&gt;&lt;/a&gt; (AE) can be used to learn a compressed representation (a vectorised hash value) of each observation in the training dataset, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, which can then be used to later retrieve the original (or similar) observation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;variational auto-encoder&lt;/em&gt;&lt;/a&gt; (VAE), a &lt;a href=&quot;https://ai.stackexchange.com/a/12354/2444&quot;&gt;statistical&lt;/a&gt; variation of AE, can also be used to generate objects &lt;em&gt;similar&lt;/em&gt; to the observations (or inputs) in the training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other data compressor models, for example, &lt;a href=&quot;http://artem.sobolev.name/posts/2016-07-11-neural-variational-inference-variational-autoencoders-and-Helmholtz-machines.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Helmholtz machine&lt;/a&gt;, which precedes both the AE and VAE.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-29T23:16:24.783" LastActivityDate="2019-05-29T23:16:24.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12567" PostTypeId="2" ParentId="12565" CreationDate="2019-05-28T13:42:37.613" Score="1" Body="&lt;p&gt;Train a network that has large input and small output. Turn it upside down (yes, you may do that). By giving the small outputs, corresponging to input, the ideally- trained network will generate those large data. &#xA;But you see in all compression there will be data lost, so generated data will be &lt;em&gt;slightly&lt;/em&gt; :DD different then original dataset. So its suitable for statistical data, like images, whatever, but not for structured like text or the most unsuitable example - program source code.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-05-28T13:42:37.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12568" PostTypeId="1" CreationDate="2019-05-28T14:25:06.190" Score="2" ViewCount="65" Body="&lt;p&gt;For logistic regression, the &lt;em&gt;&lt;strong&gt;Cost&lt;/strong&gt;&lt;/em&gt; function is defined as:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;Cost(h_{\theta}(x)-y) = -ylog(h_{\theta}(x))-(1-y)log(1-h_{\theta}(x))&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now have a nonlinear function&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; h_{\theta}^{(i)}(x)=xe^{-j\theta_i|x|^2}&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; denotes the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th training sample. How should I define cost function for this particular nonlinear function? &lt;/p&gt;&#xA;" OwnerUserId="25806" LastEditorUserId="2444" LastEditDate="2019-05-29T23:13:00.297" LastActivityDate="2019-05-29T23:13:00.297" Title="How to define cost function for custom nonlinear functions?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12569" PostTypeId="1" CreationDate="2019-05-28T14:44:04.090" Score="3" ViewCount="29" Body="&lt;p&gt;I am tentatively trying to train a deep reinforcement learning model the maze escaping task, and each time it takes one image as the input (e.g., a different &quot;maze&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have about &lt;span class=&quot;math-container&quot;&gt;$10K$&lt;/span&gt; different maze images, and the ideal case is that after training &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; mazes, my model would do a good job to quickly solve the puzzle in the rest &lt;span class=&quot;math-container&quot;&gt;$10K$&lt;/span&gt; - &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; images.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing to inquire some good idea/empirical evidences on how to select a good &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; for the training task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in general, how should I estimate and enhance the ability of &quot;transfer learning&quot; of my reinforcement model? Make it more generalized? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice or suggestions would be appreciate it very much. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-05-28T14:44:04.090" Title="Training a reinforcement learning model with multiple images" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;deep-rl&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12570" PostTypeId="1" AcceptedAnswerId="15733" CreationDate="2019-05-28T16:40:29.937" Score="2" ViewCount="88" Body="&lt;p&gt;In the book &quot;Reinforcement Learning: An Introduction&quot; (2018) Sutton and Barto define the prediction objective (&lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt;) as follows (page 199):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\overline{VE}\doteq\sum_{s\epsilon S} \mu(s)[v_{\pi}(s)-\hat{v}(s,w)]^2$$&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; is the true value of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(s,w)$&lt;/span&gt; is the approximation of it. Furthermore it is stated that this is &quot;often used in plots&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How do I get the true value &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;? And if there is a way to obtain the value, why would I need to approximate it?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="21299" LastEditorUserId="2444" LastEditDate="2019-10-05T08:52:15.277" LastActivityDate="2019-10-05T11:00:54.360" Title="How do we get the true value in the prediction objective in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;sutton-barto&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12571" PostTypeId="2" ParentId="12570" CreationDate="2019-05-28T18:27:52.987" Score="0" Body="&lt;p&gt;I may be wrong about this but my best interpretation without having access to the book is that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How do I get the true value vπ(s)?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The True value I think is whatever the most correct answer is for the prediction. This should be training data. Some companies like facebook spend a lot of money to hire people to create hand-detailed data to fill in this value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;And if there is a way to obtain the value, why would I need to approximate it?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are approximating it to test the accuracy of your model - your prediction. It seems to me this equation is only necessary when training the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result of this is your total error between all your predictions. The lower the value, the better your model.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Mean_squared_error&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1720" LastActivityDate="2019-05-28T18:27:52.987" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12572" PostTypeId="2" ParentId="12552" CreationDate="2019-05-28T19:44:56.280" Score="3" Body="&lt;p&gt;The purpose of &lt;em&gt;Reinforcement Learning&lt;/em&gt; is to maximize some notion of &lt;strong&gt;cumulative reward&lt;/strong&gt;, leading me to the point (1) :&#xA;as far as I understand, there is no timesteps in your problem and the &quot;reward&quot; is immediate.&#xA;Thus, I don't think reinforcement is suitable here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On an other hand, in &lt;em&gt;supervised learning&lt;/em&gt;, &lt;strong&gt;linear regression&lt;/strong&gt; is the task of approximating a mapping function (f) from input variables (X) to a &lt;strong&gt;continuous output variable&lt;/strong&gt; (y).&#xA;It has much in common with your case. If I am not wrong, you are trying to approximate a function that maps image data to (x,y) coordinates&#xA;(the &quot;sweet spot&quot;). So, I think regression would be a better way to go for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could either &lt;strong&gt;generate a dataset&lt;/strong&gt; at first, with images data and associated (x,y) coordinates for only spots validated by your function F, and then train a regression predictive model. Or you could train your model with &lt;strong&gt;online learning&lt;/strong&gt;, by generating batch of images and sweet spot coordinates at each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning the point (2), it will highly depend on how is made your F function. Since overlapping spots cannot be sweet spots, the simplest would&#xA;be to make your F function return 0 for those spots.&lt;/p&gt;&#xA;" OwnerUserId="23818" LastEditorUserId="23818" LastEditDate="2019-05-28T20:01:54.500" LastActivityDate="2019-05-28T20:01:54.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12573" PostTypeId="1" CreationDate="2019-05-29T01:53:08.997" Score="2" ViewCount="51" Body="&lt;p&gt;I am working on a project that involves using a ConvNet to identify screws. I am able to train from scratch a ConvNet based on the first version of the inception network, but shallower (only 3 inception modules), and at the moment classifying only 45 different screws (the goal is to cover a significant part of a catalog containing ~ 4000 different itens).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My training set consists of rectangular grayscale images of the screws (150 x 300 pixels), approx. 700 images for each class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The prototype of this model has been working pretty well with 45 classes (test set accuracy ~98%), but I am starting to worry about two things:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Many screws in the catalog have similar shapes, but different sizes, so the production model will have to be able to infer the scale of the objects. This is important because future users will image screws with different smartphone cameras, yielding different screw sizes in the images fed to the ConvNet. I haven't been able to find much about this in the literature. And from what I have read about ConvNets, they are good at detecting shapes, which mean that two objects, the first 1 meter long and the other 1 centimeter long, would be considered &quot;equal&quot; by a ConvNet if they appeared similar in an image. One (not very elegant) solution I imagined would be to include a scale in the training images, by means either of a ruler or a common object (a coin, for example). Anyway, I wonder if this problem has a simple solution, since I believe many people might have faced it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) All of the notorious ConvNets I know of are trained with the ImageNet dataset, which comprises 1000 different classes. My screw dataset will ultimately have more than that. Is that an issue? Assuming I have the hardware resources to train very large fully connected layers and softmax output layers, is there an upper bound to the number of classes a ConvNet can identify?&lt;/p&gt;&#xA;" OwnerUserId="26003" LastActivityDate="2019-05-29T01:53:08.997" Title="Object size identification and maximum number of classes with convolutional neural networks" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12574" PostTypeId="2" ParentId="7573" CreationDate="2019-05-29T06:39:05.447" Score="0" Body="&lt;p&gt;Self-conscious AIs are important because consciousness is what makes us human. And scientists are trying to develop AIs that are closer to humans and human behaviour. The thought process of humans is very complicated and to have it developed in AIs could really replace us with robots that have consciousness. The only reason AIs are not taking up work that involves decision-making because they do not have consciousness.&lt;/p&gt;&#xA;" OwnerUserId="26007" LastActivityDate="2019-05-29T06:39:05.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12575" PostTypeId="2" ParentId="12458" CreationDate="2019-05-29T06:47:32.270" Score="0" Body="&lt;p&gt;Aren't we already doing things with technology that are not ethical? But to answer your question, yes, it will be unethical to use gender to target a user. It will mean that the AI will be fed a list of things to look for for a certain gender which is again gender-bias. I find targetting users unethical but using gender basis would really be something else. I am not sure if I am on the right track but ads are run keeping in mind the target audience which involves the age, gender, etc. That is done by humans but still.&lt;/p&gt;&#xA;" OwnerUserId="26007" LastActivityDate="2019-05-29T06:47:32.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12576" PostTypeId="1" CreationDate="2019-05-29T08:19:38.493" Score="3" ViewCount="188" Body="&lt;p&gt;Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly &amp;quot;normal&amp;quot; data points. Then new data can be considered anomalous if the new data has a large reconstruction error, i.e. it was hard to fit the features as in the normal data.&lt;/p&gt;&#xA;&lt;p&gt;Even if the training is supervised by learning to reconstruct the same data, how is the reconstruction error computed for the new data?&lt;/p&gt;&#xA;" OwnerUserId="26009" LastEditorUserId="2444" LastEditDate="2021-02-17T15:21:16.050" LastActivityDate="2021-03-19T19:04:15.440" Title="How can auto-encoders compute the reconstruction error for the new data?" Tags="&lt;unsupervised-learning&gt;&lt;autoencoders&gt;&lt;anomaly-detection&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12577" PostTypeId="1" CreationDate="2019-05-29T08:49:17.190" Score="5" ViewCount="3901" Body="&lt;p&gt;I am a newbie in reinforcement learning working on a college project. The project is related to optimizing the hardware power. I am running proprietary software in Linux distribution (16.04). The goal is to use reinforcement learning and optimize the power of the System (keeping the performance degradation of the software as minimum as possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this, I need to create a custom environment for my reinforcement learning. From reading different materials, I could understand that I need to make my software as a custom environment from where I can retrieve the &lt;code&gt;state&lt;/code&gt; features. &lt;code&gt;Action&lt;/code&gt; space may include instructions to Linux to change the power (I can use some predefined set of power options).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proprietary software is a cellular network and the &lt;code&gt;state&lt;/code&gt; variables include &lt;code&gt;latency&lt;/code&gt; or &lt;code&gt;throughput&lt;/code&gt;. To control the power &lt;code&gt;action space&lt;/code&gt;, &lt;a href=&quot;https://github.com/kentcz/rapl-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;rapl-tools&lt;/a&gt; can be used to control CPU power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just started working on this project and everything seems blurry. What is the best way to make this work? Is there some tutorials or materials that would help me make things clear. Is my understanding of creating a custom environment for reinforcement learning true?&lt;/p&gt;&#xA;" OwnerUserId="25885" LastEditorUserId="25885" LastEditDate="2019-05-29T12:24:29.107" LastActivityDate="2019-10-26T13:02:30.347" Title="How to create a custom environment for reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12578" PostTypeId="2" ParentId="12576" CreationDate="2019-05-29T08:51:34.233" Score="-1" Body="&lt;p&gt;By Unsupervised they refer to the learning process, which is Unsupervised. Although, it does not appear so, but Unsupervised loosely means you work with data points and data points only without the use of any information about those data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have idea about how AE's are used for anomaly detection, but from the point of view of AE's, I will explain what might be happening. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In AE's the hidden representations (if hidden units are less than visible units) the weights for the encoder network are not independent of each other, thus the values coming out of a hidden node after encoding operation are not independent of each other, which if speaking mathematically will mean each hidden node is modelling complex probabilistic interactions between the input features. Thus, the AE encoder learns that if the given feature &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; is of this magnitude, it also expects &lt;span class=&quot;math-container&quot;&gt;$x_2,...x_n$&lt;/span&gt; to be of certain other magnitudes. Similarly, the decoder also learns how to decode this complex interactions to original data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point to note is that in AE's with hidden nodes less than input nodes, if encoder function is given by &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; and decoder by &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; then the operation you are doing is &lt;span class=&quot;math-container&quot;&gt;$$d(e(x))=x$$&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$$e^{-1}(e(x))=x$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Decoder is reversing the inputs as &quot;learned&quot; and &quot;generalized&quot; from inputs, and not by &quot;learning&quot; to perform the inverse operation (this might be the case where &lt;span class=&quot;math-container&quot;&gt;$nodes_{hidden}$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\geq$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$nodes_{input}$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, say there is an anomaly. The encoder, outputs something which the decoder will see does not match the way it should match with other hidden nodes output (normally if hidden node1 is outputing &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; the decoder expects &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; from hidden node 2, but now the decoder is getting &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; but not getting &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; instead getting &lt;span class=&quot;math-container&quot;&gt;$h_2 + \epsilon$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is noise and as a result fails to understand and decode the hidden representation properly.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-05-29T08:51:34.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12579" PostTypeId="1" AcceptedAnswerId="12598" CreationDate="2019-05-29T10:08:39.777" Score="2" ViewCount="287" Body="&lt;p&gt;In the paper, &lt;a href=&quot;https://www.aclweb.org/anthology/C18-1139&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contextual String Embeddings for Sequence Labeling&lt;/a&gt;, the authors state that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P(x_{0:T}) = \prod_{t=0}^T P(x_t|x_{0:t-1})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They also state that, in the LSTM architecture, the conditional probability &lt;span class=&quot;math-container&quot;&gt;$P(x_{t}|x_{0:t})$&lt;/span&gt; is approximately a function of the network output &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P(x_{t}|x_{0:t}) \approx \prod_{t=0}^{T} P(x_t|h_t;\theta)&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is this equation true?&lt;/p&gt;&#xA;" OwnerUserId="26012" LastEditorUserId="2444" LastEditDate="2019-05-30T14:57:23.767" LastActivityDate="2019-05-30T16:46:16.487" Title="Why can we approximate the joint probability distribution using the output vector of an LSTM?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;probability-distribution&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12580" PostTypeId="2" ParentId="12458" CreationDate="2019-05-29T10:43:51.900" Score="0" Body="&lt;p&gt;The general principle is that data belongs to the person who generated it, and permission should be sought before you use someone else's possessions.&#xA;So, it is ethical to use it to infer whatever you like about a person, provided that you ask permission first. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-05-29T10:43:51.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12581" PostTypeId="2" ParentId="12577" CreationDate="2019-05-29T12:42:18.670" Score="4" Body="&lt;p&gt;&lt;em&gt;This answer assumes that your &quot;proprietary software&quot; is a simulation of, or controller for a real environment.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Yes you will very likely need to write software to represent your environment in some standard way as a Reinforcement Learning (RL) environment. Depending on details, this may be trivially easy or it might be quite involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An environment in RL must have the following traits in general, in order to interface with RL agent software:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A state representation. This will typically be an object or array of data that matches sensor readings from the real environment. It is important to RL that the state has the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; so that predictions of value can be accurate. For some environments that will mean calculating derived values from observations, or representing a combined history of last few observations from sensors as the state.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The state can either be held inside an internal representation of the environment, which is a typical object-oriented approach, or it can be passed around as a parameter to other functions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A simple state might just be a fixed size array of numbers representing important traits of the environment, scaled between -1 and 1 for convenience when using it with neural networks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An action representation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A simple action representation could just be an integer which identifies which of N actions has been chosen, starting from 0. This allows for a basic index lookup when checking value function estimates.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A reward function. This is part of a problem definition, and you may want to have that code as part of the environment or part of the agent or somewhere in-between depending on how likely it is to change - e.g. if you want to run multiple experiments that optimise different aspects of control but in the same environment, you may make a totally separate reward calculation module that you combine at a high level with the agent and environment code.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/12299/1847&quot;&gt;Reward function design is a complex subject&lt;/a&gt;. However, the basics are that this should always be a single real valued number (i.e. a floating point number in most programming languages).&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A time step function. This should take an action choice, and should update the state for a time step - returning the next state, and the immediate reward. If the environment is real, then the code will make actual changes (e.g. move robot arm), potentially wait for the time step to elapse, then read sensors to get the next state and calculate reward. If the environment is simulated, then the code should call some internal model to calculate the next state. This function should call the proprietary software you have been provided for your task.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If actions available depend on the current state, then code for that could live in the environment simulation or the agent, or be some helper function that the agent can call, so it can filter the actions before choosing one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are working in Python, to help make this more concrete, and follow an existing design, see &lt;a href=&quot;https://stackoverflow.com/questions/45068568/how-to-create-a-new-gym-environment-in-openai&quot;&gt;&quot;How to create a new gym environment in OpenAI?&quot;&lt;/a&gt; on Stack Overflow. The Open AI environments all follow the same conventions for environment definitions, which helps when writing agents to solve them. I also recommend finding an Open AI Gym environment that seems similar to your problem, seeing how that works and trying to train an agent to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may still be work to match your environment to an agent that can solve it. That depends on what agent software you are using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you write your own agent software, it helps to separate out the environment code like this. The environment is the problem to solve, and the RL agent is one way to search for a solution to it. Keeping those parts separate is a useful design that will allow you to try different types of agent and compare their performance for example.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-05-29T12:57:00.323" LastActivityDate="2019-05-29T12:57:00.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12582" PostTypeId="2" ParentId="12548" CreationDate="2019-05-29T12:42:35.913" Score="1" Body="&lt;p&gt;If your interest is positional information, encode it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could include learning an embedding for each position and leveraging that in your model. You could also use an approach to hard-encode rather than learn it (kinda like adding sinusoids in the transformer paper &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is All You Need&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;an example of a paper that encodes the 2D positional info: &lt;a href=&quot;https://arxiv.org/pdf/1904.09925.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Augmented Convolutional Networks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-29T12:42:35.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12583" PostTypeId="2" ParentId="8270" CreationDate="2019-05-29T13:00:34.010" Score="0" Body="&lt;p&gt;I am working on something similar (if not the same thing) in JS right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Network structure: The network is a 2d array. Each element in the 2d dimension is an object holding 2 arrays, starts and ends. The starts array holds the outgoing connections. The ends array holds the incoming connections. This way, an end on one node is a reference to a start on another node in the next layer. Each element in starts and ends has 2 variables, weight and value. I've tried setting the value on the node itself (array[x][x].value instead of array[x][y].starts[z].value) but it didn't work out. The value must be carried to the next activation somehow - so I put it directly on the &quot;synapse&quot;. (Note that nodes, or neurons, are just conceptual here. I'm just referring to the cell in the array (array[x][y] &amp;lt;- node)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Connections:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I have an &quot;addLink&quot; function that sets the connections. It takes sr, sc, er, ec as arguments (start/end row, start/end column). It creates the &quot;Link&quot; object and places it into two arrays (array[sr][sc].starts and array[er][ec].ends). You would just need an array, or external files... some list of start and end coordinates for each link that you want to form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My connection solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented &quot;max outgoing connections&quot; for each node, as well as defining &quot;possible connection ranges&quot; for each node. Basically, I define a box size (yMin, yMax, xMin, xMax), it adds the node's coordinates (and inverts for a second box on the other side) to create an array of possible connections and &quot;randomly&quot; chooses a number of nodes from those connections up to max connections. I call it getReachableCells. This allows me to form connections across multiple layers. I also define the probabilities of which connections are chosen (further connections are less likely; one cell to the right is the most likely.) In the future, I plan to (rarely) make one of those reachable cells an input node on another network. I've made a 2d-array utility class that allows me to scale, consolidate, and prune the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I plan on creating signalling events that triggers growth and death of nodes and connections. For example, if an input-values array's (x_data) length is greater than the number of input neurons, this will trigger a growth signal. If it happens several times (if the demand is high), the array will be scaled and a new node created. Synapses will have connection strengths that are strengthened with use and weakened with lack of use - eventually being removed if the connection strength hits 0. (I started as a game programmer.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I made a render class that shows all of the connections as lines on a canvas as well as functions that label inputs and outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that I'm currently facing: Backpropagation. Since I have multiple outputs, I need to adjust the weights associated with each output. I call this an output's &quot;chain&quot;. I want to be able to grab the chain during the forward pass so that I don't have to loop over the entire network again for every output to find it, but it's looking like that's the only way. If anyone has any insight on this, please let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/cTukD8e.png&quot; alt=&quot;my nn&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="26020" LastEditorUserId="26020" LastEditDate="2019-05-29T13:13:32.600" LastActivityDate="2019-05-29T13:13:32.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12587" PostTypeId="1" CreationDate="2019-05-29T20:53:45.710" Score="1" ViewCount="12" Body="&lt;p&gt;I am  new to big data theory, and during the past 3 days, I took an official big data course with some of the best instructors available in my country in this domain. The things was little bit obscure for me as I have an engineering background but with no knowledge in AI techniques and domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After getting an intro on big data and the 5Vs (Volume, Velocity, ...), we got an intro on hadoop and hadoop ecosystem tools (Hive, Pig, ...). Then a simple example on how to run MapReduce Java script on small data file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to make things clear with me, are hive, pig and other hadoop ecosystem tools, are tools to break up my large data files from different sources and servers into fast-readable files, by which we create new tables with our required fields to use them later on in machine learning scripts and feature extractions ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. by fast readable files I mean, using a scripting tools that normal relational database tools like SQL and oracle don't have it on huge data sets (1 terrabytes and above) to manage and get info from it as fast as possible ?&lt;/p&gt;&#xA;" OwnerUserId="26028" LastActivityDate="2019-05-29T20:53:45.710" Title="Are hadoop ecosystem tools main goal is to break up large data sets into fast readable files?" Tags="&lt;machine-learning&gt;&lt;data-science&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12588" PostTypeId="1" CreationDate="2019-05-29T22:49:36.580" Score="1" ViewCount="36" Body="&lt;p&gt;Basickly, NN is approximator of any statistical relation, but we operate the common sence where entities may be rather strickt known, so maybe its possible to construct AI, representing it as a chain of algorithms?&#xA;Means, by (very big and tricky) common (turing-complete) programm ??&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-05-29T22:49:36.580" Title="Must strong AI be only neural network based, or its possible with common programm?" Tags="&lt;agi&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12590" PostTypeId="1" CreationDate="2019-05-29T23:12:22.280" Score="3" ViewCount="35" Body="&lt;p&gt;I've seen GANs that do things like convert an image to a painting or this GAN here &lt;a href=&quot;https://make.girls.moe/#/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://make.girls.moe/#/&lt;/a&gt; that takes in a set of characteristics and generates a waifu with those characteristics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding of a GAN is that the generator upsamples random noise and the discriminator detects if an image is within the real or fake. So if the generator say, generates a waifu with the wrong hair color, how would the discriminator know?&lt;/p&gt;&#xA;" OwnerUserId="23941" LastActivityDate="2019-05-29T23:12:22.280" Title="How do GANs create an image with specific characteristics?" Tags="&lt;neural-networks&gt;&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12591" PostTypeId="1" CreationDate="2019-05-30T04:57:42.483" Score="1" ViewCount="123" Body="&lt;p&gt;I'm trying to work out an approach to balancing my dataset, which is a subset of a google openimages - some classes are represented orders of magnitude more than others and I am hesistant to simply throw away data. The approach I want to try next is roughly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Add a single instance of every image to the training set.&#xA;Crop out instances of the majority class from entire dataset and then add these cropped images to the dataset. keep adding these cropped images to training set until number of instances of 2nd majority class is close to number of instances of 1st majority class.&#xA;Crop out instances of second majority class and repeat (2)&#xA;keep going until all images have been cropped down to the minority class and dataset is roughly balanced.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This approach would mean that no instance of any class would be repeated more often than any other instance of the same class. My concern is that the minority classes will end up being predominantly represented in training by heavily cropped images. I'm not sure but I don't think this would matter in a two-stage object detector but I'm concerned that it might be a problem for a one-stage detector if, for some classes, it was predominantly trained on images that were cropped to various degrees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative to doing this would be undersampling ie. just throwing away most of the data which I'm hesistant to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, it would be really great to get some opinions on this in the context of Yolov3? Is yolov3 sufficiently robust at scale-invariance that training on many examples that are cropped/larger than what they are likely to be at test time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm fairly commited to some sort of oversampling because of the results in this paper which found best results generally from oversampling with CNN classifiers as opposed to undersampling, threshold adjustment etc. &lt;a href=&quot;https://arxiv.org/pdf/1710.05381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1710.05381.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="21583" LastActivityDate="2019-05-30T04:57:42.483" Title="Will balancing dataset of images for object detection for a single-shot OD (Yolov3-spp) by cropping lower the quality of the model?" Tags="&lt;datasets&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12592" PostTypeId="2" ParentId="12534" CreationDate="2019-05-30T05:09:09.197" Score="2" Body="&lt;p&gt;I think you should be fine if you are using other people's tried and tested algorithms and have a reasonable understanding of machine learning best practices - how to clean and balance your dataset and how to effectively measure performance. Maybe if you are trying to come up with new algorithms / DNN architectures you will struggle.&lt;/p&gt;&#xA;" OwnerUserId="21583" LastActivityDate="2019-05-30T05:09:09.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12594" PostTypeId="1" AcceptedAnswerId="12597" CreationDate="2019-05-30T09:59:09.537" Score="1" ViewCount="107" Body="&lt;p&gt;I am tentatively reusing a codebase of &lt;a href=&quot;https://github.com/prasoon2211/pacman-RL/blob/master/pacman_vanilla_DQN/pacman_agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;pacman&lt;/a&gt; to train my own deep reinforcement learning model. While most of the components seems reasonable and understandable to me, there are two things that seem obscure to me:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How to decide the size of the replay memory? Currently, since I set the total step of learning as 4000 (note that in the referred codebase this value is set as 4000000), so I just proportionally decrease the &lt;code&gt;replay_memory_size&lt;/code&gt; as 400. Would that make sense? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the return value &lt;code&gt;epsilon&lt;/code&gt; when calling function &lt;code&gt;PiecewiseSchedule&lt;/code&gt;? I also proportionally decrease its parameters as follows:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        epsilon = PiecewiseSchedule([(0, 1.0),&#xA;                                     (40, 1.0), # since we start training at 10000 steps&#xA;                                     (80, 0.4),&#xA;                                     (200, 0.2),&#xA;                                     (400, 0.1),&#xA;                                     (2000, 0.05)], outside_value=0.01)&#xA;        replay_memory = PrioritizedReplayBuffer(replay_memory_size, replay_alpha)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the original function call is like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        epsilon = PiecewiseSchedule([(0, 1.0),&#xA;                                     (10000, 1.0), # since we start training at 10000 steps&#xA;                                     (20000, 0.4),&#xA;                                     (50000, 0.2),&#xA;                                     (100000, 0.1),&#xA;                                     (500000, 0.05)], outside_value=0.01)&#xA;        replay_memory = PrioritizedReplayBuffer(replay_memory_size, replay_alpha)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And in general, what is the principle (guideline) behind setting a good size of &quot;replay memory&quot; and calling function &lt;code&gt;PiecewiseSchedule&lt;/code&gt;? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="25973" LastEditorUserId="12509" LastEditDate="2019-05-30T14:32:34.647" LastActivityDate="2019-05-30T15:50:02.477" Title="Understanding the configuration of replay memory and epsilon in deep reinforcement learning" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12596" PostTypeId="1" AcceptedAnswerId="12611" CreationDate="2019-05-30T10:52:10.080" Score="2" ViewCount="37" Body="&lt;p&gt;I'd like to build an application for tracking the position of a given animal (e.g. a cat) in a series of images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any off-the-shelf API I could use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Azure has some Vision APIs, but it seems to me they can't be used to get the position of something in an image.&lt;/p&gt;&#xA;" OwnerUserId="26043" LastEditorUserId="2444" LastEditDate="2019-05-30T16:32:50.827" LastActivityDate="2019-05-31T12:18:18.500" Title="Which API can I use for tracking the position of animal in one or more images?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12597" PostTypeId="2" ParentId="12594" CreationDate="2019-05-30T12:41:35.310" Score="4" Body="&lt;p&gt;Replay memory is a recording of the games that the agents has played. It is this data that is used to train the neural network (or whatever machine learning method you are using).&#xA;The training is performed by looking at the games played and the resulting reward signal (in Pacman, this might be your score), and then learning which game strategies worked well and which did not.&#xA;It is difficult to say exactly how big the replay memory should be, this is something that you will need to experiment with. The larger it is, then the more examples will be available to be learned from during a particular training period, but of course it takes longer to acquire them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Epsilon is a parameter that governs the trade-off between &lt;em&gt;exploration&lt;/em&gt; and &lt;em&gt;exploitation&lt;/em&gt;. By this, I mean that for each action that your agent takes, it needs to decide whether to do the best thing that it has already discovered to do (&lt;em&gt;exploitation&lt;/em&gt;), or try something new (&lt;em&gt;exploration&lt;/em&gt;). Usually, a value of epsilon near to 1 means that the agent will choose to explore more, and a value closer to 0 will mean that the agent will exploit more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can imagine therefore that at the beginning of an agent's training you might want epsilon ~1, because it has lots of new strategies to try to discover, but as it gets better you would reduce it to ~0 so that it can explore strategies that are only reachable after a long sequence of game steps.&#xA;Naturally, when the agent is fully trained and ready to be deployed then epsilon would = 0.&#xA;In the example that you posted, you can see that Pacman starts off doing nothing but &lt;em&gt;exploring&lt;/em&gt;, but after 500000 games it &lt;em&gt;exploits&lt;/em&gt; 95% of the time. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="12509" LastEditDate="2019-05-30T15:50:02.477" LastActivityDate="2019-05-30T15:50:02.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12598" PostTypeId="2" ParentId="12579" CreationDate="2019-05-30T14:44:53.203" Score="1" Body="&lt;p&gt;In section 2.1 of the &lt;a href=&quot;https://www.aclweb.org/anthology/C18-1139&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, the authors state that the goal of &lt;em&gt;character-level language model&lt;/em&gt; is to estimate the following &lt;a href=&quot;https://en.wikipedia.org/wiki/Joint_probability_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;joint probability distribution&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(\boldsymbol{x}_{0:T}) = P(\boldsymbol{x}_{0}, \boldsymbol{x}_{1}, \dots, \boldsymbol{x}_{T}),$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a joint probability distribution of all characters of a sequence of &lt;span class=&quot;math-container&quot;&gt;$T+1$&lt;/span&gt; characters, where &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_i$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th character of the sequence. Ideally, the joint probability distribution, &lt;span class=&quot;math-container&quot;&gt;$P(\boldsymbol{x}_{0:T})$&lt;/span&gt;, should put more mass (or density) on the combination of &lt;span class=&quot;math-container&quot;&gt;$T-1$&lt;/span&gt; characters that are more likely (in a given language). For example, in English, the combination of the characters &quot;the&quot; is more likely than the combination &quot;bibliopole&quot;. Hence, &lt;span class=&quot;math-container&quot;&gt;$P(\text{t}, \text{h}, \text{e})$&lt;/span&gt; should be higher than &lt;span class=&quot;math-container&quot;&gt;$P(\text{b}, \text{i}, \text{b}, \text{l}, \text{i}, \text{o}, \text{p}, \text{o}, \text{l}, \text{e})$&lt;/span&gt;, where, in this case, &lt;span class=&quot;math-container&quot;&gt;$T=2$&lt;/span&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$P(\boldsymbol{x}_{0:T})$&lt;/span&gt; is the actual character-level language model, which is represented by a joint probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly and intuitively, the conditional probability distribution &lt;span class=&quot;math-container&quot;&gt;$$P(\boldsymbol{x}_{t} \mid \boldsymbol{x}_{0:t-1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tells us the probability of the next character of the sequence &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_{t}$&lt;/span&gt; given (that is, having observed) the previous characters of the sequence &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_{0:t-1} = (\boldsymbol{x}_{0}, \dots, \boldsymbol{x}_{t-1})$&lt;/span&gt;. In general, the next character of a sequence &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_{t}$&lt;/span&gt; can depend on all previous characters of the same sequence, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_{0:t-1}$&lt;/span&gt;. If we are able to learn the conditional model &lt;span class=&quot;math-container&quot;&gt;$P(\boldsymbol{x}_{t} \mid \boldsymbol{x}_{0:t-1})$&lt;/span&gt;, given a sequence of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; characters, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_{0:t-1}$&lt;/span&gt;, we can sample the next character according to the same conditional probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that, given two events (or random variables) &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, the joint distribution of them is defined as &lt;span class=&quot;math-container&quot;&gt;$P(A, B) = P(A \mid B)P(B) = P(B \mid A)P(A)$&lt;/span&gt;, which gives rise to the Bayes' theorem. This can be easily generalised to multiple variables. More specifically, if you consider &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to a set of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; events (rather than just one), that is, &lt;span class=&quot;math-container&quot;&gt;$B=B_1 \cap B_2\cap \dots \cap B_N = B_1, B_2, \dots, B_N$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$P(A, B) = P(A \mid B)P(B)$&lt;/span&gt; still holds, but we can further decompose it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(A, B) = P(A, B_1, B_2, \dots, B_N)&#xA;&amp;amp;= P(A \mid B_1, B_2, \dots, B_N)P(B_1, B_2, \dots, B_N) \\&#xA;&amp;amp;= P(A \mid B_1, B_2, \dots, B_N)P(B_1 \mid B_2, \dots, B_N) P(B_2, \dots, B_N) \\&#xA;&amp;amp;= \cdots&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule_(probability)&quot; rel=&quot;nofollow noreferrer&quot;&gt;chain rule (or product rule)&lt;/a&gt; of probability. Essentially, we apply the rule &lt;span class=&quot;math-container&quot;&gt;$P(A, B) = P(A \mid B)P(B)$&lt;/span&gt; recursively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analogously, in the &lt;a href=&quot;https://www.aclweb.org/anthology/C18-1139&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, the authors apply this chain rule to express the joint distribution &lt;span class=&quot;math-container&quot;&gt;$P(\boldsymbol{x}_{0:T})$&lt;/span&gt; as a product of conditional probability distributions, that is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(\boldsymbol{x}_{0:T}) = \prod_{t=0}^T P(\boldsymbol{x}_{t}\mid \boldsymbol{x}_{0:t-1})&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the LSTM, the &lt;em&gt;vector&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{h}_t$&lt;/span&gt; is supposed to keep track of the past. More specifically, in the case of the character-level language model, we train an LSTM-based RNN, so that &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{h}_t$&lt;/span&gt; is approximately equal to &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}_{0:t-1}$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{h}_t \approx \boldsymbol{x}_{0:t-1}$&lt;/span&gt;. Hence, the joint probability distribution of the characters above can be now be approximately defined as a function of the vector &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{h}_t$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(\boldsymbol{x}_{0:T}) \approx \prod_{t=0}^T P(\boldsymbol{x}_{t}\mid \boldsymbol{h}_t; \boldsymbol{\theta})&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; are the parameters of the LSTM-based RNN. Note that this is not an equality but an &lt;em&gt;approximation&lt;/em&gt;. Intuitively, we train the LSTM so that it learns the interactions of the past characters.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-30T16:46:16.487" LastActivityDate="2019-05-30T16:46:16.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12600" PostTypeId="1" CreationDate="2019-05-30T18:10:08.487" Score="1" ViewCount="126" Body="&lt;p&gt;How can I implement a GAN network for text (review) generation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, can someone guide me to resource (code) to help in text generation? &lt;/p&gt;&#xA;" OwnerUserId="26049" LastEditorUserId="2444" LastEditDate="2019-05-30T21:21:12.610" LastActivityDate="2019-11-01T19:02:50.450" Title="How can I implement a GAN network for text (review) generation?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;generative-adversarial-networks&gt;&lt;software-evaluation&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12601" PostTypeId="1" AcceptedAnswerId="12602" CreationDate="2019-05-30T20:29:22.730" Score="3" ViewCount="85" Body="&lt;p&gt;Can you please point me to some resources about image genereation besides GANs?&#xA;Are there any other techniques throughout history?&#xA;How did idea of image generation evolved and how it started?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried googling &quot;image generation before gans&quot; and similar alternatives but without any success.&lt;/p&gt;&#xA;" OwnerUserId="23918" LastEditorUserId="2444" LastEditDate="2019-05-30T21:18:25.587" LastActivityDate="2019-05-30T21:18:25.587" Title="Other deep learning image generation techniques besides GANs?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;generative-model&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12602" PostTypeId="2" ParentId="12601" CreationDate="2019-05-30T21:09:45.723" Score="4" Body="&lt;p&gt;There are several &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative models&lt;/a&gt; that have been proposed before or roughly at the same time of the &lt;a href=&quot;https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN&lt;/a&gt; (2014). For example, the &lt;a href=&quot;http://proceedings.mlr.press/v5/salakhutdinov09a/salakhutdinov09a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep Boltzman machine&lt;/a&gt; (2009), &lt;a href=&quot;http://proceedings.mlr.press/v32/bengio14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep generative stochastic network&lt;/a&gt; (2014) or &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;variational auto-encoder&lt;/a&gt; (2014).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-30T21:11:31.520" LastActivityDate="2019-05-30T21:11:31.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12604" PostTypeId="1" AcceptedAnswerId="12609" CreationDate="2019-05-31T02:47:46.293" Score="3" ViewCount="84" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original prioritized experience replay paper&lt;/a&gt;, the authors track &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt; in every state transition tuple (see line 6 in algorithm below):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dSnQZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dSnQZ.png&quot; alt=&quot;algorithm&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do the authors track this at every time step?  Also, many blog posts and implementations leave this out (including I believe the &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/deepq/replay_buffer.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI implementation&lt;/a&gt; on github).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain explicitly how &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt; is used in this algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: I understand the typical use of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; as a discount factor. But typically gamma remains fixed. Which is why I’m curious as to the need to track it. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="16343" LastEditDate="2019-05-31T12:13:33.577" LastActivityDate="2019-05-31T12:13:33.577" Title="Why do authors track $\gamma_t$ in Prioritized Experience Replay Paper?" Tags="&lt;dqn&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12605" PostTypeId="1" AcceptedAnswerId="12608" CreationDate="2019-05-31T04:14:01.397" Score="2" ViewCount="317" Body="&lt;p&gt;Is it possible to combine or create conditional statements of 0 and 1, and optimize with an evolutionary algorithm (given that all computers use a binary system)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be an algorithm that maps input and output to 0 and 1, or a conditional statement that edits a conditional statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a binary conditional statement is &lt;code&gt;if 11001 then 01110&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as molecules are combined to form living beings, we could begin with the most fundamental operations (0 and 1, if then) to develop intelligence.&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2019-06-20T00:03:33.713" LastActivityDate="2019-06-20T00:04:27.613" Title="Can we evolve 0 and 1?" Tags="&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12606" PostTypeId="2" ParentId="12421" CreationDate="2019-05-31T05:36:58.267" Score="1" Body="&lt;p&gt;The previous answer from Brale is &lt;em&gt;mostly&lt;/em&gt; correct but is missing a large detail to get the precise answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this is a question from a GT course homework, I only want to leave pointers so those seeking help can understand the required concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;() equation is a summation over &lt;em&gt;infinite&lt;/em&gt; K-steps (&lt;sub&gt;0:1&lt;/sub&gt; -&gt; &lt;sub&gt;0:∞&lt;/sub&gt;) and should be included in our equation of () = (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every k-step estimator which included steps past the termination point will equal the sum of the rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Including these values into the summation will show a pattern, making infinite summation equation solvable.&lt;/p&gt;&#xA;" OwnerUserId="26058" LastActivityDate="2019-05-31T05:36:58.267" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12607" PostTypeId="1" CreationDate="2019-05-31T06:06:22.240" Score="2" ViewCount="120" Body="&lt;p&gt;I read that a mix of &quot;greedy&quot; and &quot;random&quot; are ideal for stochastic local search (SLS), but I'm not sure why. It mentioned that the greedy finds the local minima and the randomness avoids getting trapped by the minima. What is the minima and how can you get trapped? Also, how does randomness avoid this? It seems like if it's truly random there's always a chance of ending up searching solutions that lead to dead ends multiple times (which seems like a waste of processing and avoidable)?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2019-05-31T13:05:51.307" LastActivityDate="2019-05-31T15:05:55.600" Title="Why is a mix of greedy and random usually &quot;best&quot; for stochastic local search?" Tags="&lt;optimization&gt;&lt;search&gt;&lt;problem-solving&gt;&lt;local-search&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12608" PostTypeId="2" ParentId="12605" CreationDate="2019-05-31T08:07:37.570" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to combine or create conditional statements of 0 and 1, and optimize with an evolutionary algorithm (given that all computers use a binary system)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, evolutionary algorithms are very general, and can be used to modify almost any source data structure, including logic trees or even executable code, provided you have some measure of fitness to optimise for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be taken to a highly abstract level, such as the paper &lt;a href=&quot;https://ieeexplore.ieee.org/document/4425074&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolution of evolution: Self-constructing Evolutionary Turing Machine case study&lt;/a&gt; where an evolutionary algorithm is used to optimise other evolutionary algorithms which solve tasks using generic models of computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are two important caveats:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There needs to be a measurement phase to establish the fitness of the algorithm. This can be very complex, depending on the problem you attempt to solve.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Genetic algorithms may be very general optimisers (capable of finding optimal solutions for problems when other algorithms may fail), but may also be very inefficient and slow depending on the size of the allowed genomes, and how the search space is structured relative to available genetic operations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;begin with the most fundamental operations (0 and 1, if then) to develop intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Provided the fitness measure allows for the expression of intelligence, then this seems theoretically possible - in the sense that there are no known reasons why a sufficiently complex logical machine could not be intelligent by any measure we have of abstract intelligence (excluding measures deliberately constructed to exclude computational models such as &quot;intelligence is the capability of a living system . . .&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, such a project faces some barriers which currently look insurmountable:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no formal measure of general intelligence to use as a fitness function. This could be worked around using an e-life approach of providing a suitable rich virtual environment and allowing agents to compete for resources in the hope that the most competitive agents would exhibit intelligent behaviour - but that begs the question of how you could recognise and select those agents through any objective measure.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any environment rich enough to select for general intelligence, whilst simulating low-level agent logic is likely to require a lot of computation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Our one example of evolving basic building blocks into beings we call intelligent took billions of years, whilst processing billions upon billions of separate evolving entities at any one time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These last two points imply a computational cost far beyond current technology, so there is no route to actually running this experiment for real. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-06-20T00:04:27.613" LastActivityDate="2019-06-20T00:04:27.613" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="12609" PostTypeId="2" ParentId="12604" CreationDate="2019-05-31T08:26:08.763" Score="4" Body="&lt;p&gt;In some cases we may wish to have a discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt; which depends on time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (or depends on state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and/or action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;, leading to an indirect dependence on time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;). Indeed we do not usually do this, but &lt;a href=&quot;https://arxiv.org/abs/1902.02893&quot; rel=&quot;noreferrer&quot;&gt;it does happen sometimes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that, from a theoretical point of view, it was very easy of the authors to make their algorithm more flexible/general and also support this (somewhat rare) case of time-varying discount factor. If it had been very complicated for them to support this option, they may have chosen not to; but if it's trivial to do so, well, why not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practical implementations will often indeed ignore that possibility if they're not using it, and can avoid including &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt; values in the replay buffer altogether if it is known to be a constant &lt;span class=&quot;math-container&quot;&gt;$\gamma_t = \gamma$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. As far as I can see, in the experiments discussed in this paper they also only used a fixed, constant &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-05-31T08:26:08.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12610" PostTypeId="2" ParentId="12470" CreationDate="2019-05-31T11:03:51.513" Score="0" Body="&lt;p&gt;This article may shed some light on this question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.propublica.org/article/facebook-doesnt-tell-users-everything-it-really-knows-about-them&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Facebook Doesn’t Tell Users Everything It Really Knows About Them&lt;/strong&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25362" LastActivityDate="2019-05-31T11:03:51.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12611" PostTypeId="2" ParentId="12596" CreationDate="2019-05-31T12:18:18.500" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.fast.ai&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;FastAI&quot;&gt;FastAI&lt;/a&gt; is the most “out of the box” API for this type of task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For video examples (and a little theory) check out the MOOC section of their site. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://course.fast.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Deep Learning&lt;/a&gt; and &lt;a href=&quot;http://course18.fast.ai/part2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cutting Edge Deep Learning&lt;/a&gt; are the two sections most relevant to you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want a working implementation check out &lt;a href=&quot;https://github.com/qfgaohao/pytorch-ssd&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GitHub repo&lt;/a&gt; that implements SSD for your purposes. I can’t say how simple the API is but it does what you are seeking (in pure PyTorch).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;note&lt;/strong&gt;: FastAI was originally built on top of PyTorch (although they are expanding out now), so you’ll be using PyTorch and need a rough idea of how to work with tensors. Most of the challenges are already implemented in their api. Installing their library will automatically install PyTorch. &lt;/p&gt;&#xA;" OwnerUserId="16343" LastActivityDate="2019-05-31T12:18:18.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12612" PostTypeId="1" AcceptedAnswerId="12615" CreationDate="2019-05-31T12:54:50.300" Score="5" ViewCount="202" Body="&lt;p&gt;The MSE can be defined as &lt;span class=&quot;math-container&quot;&gt;$(\hat{y} - y)^2$&lt;/span&gt;, which should be equal to &lt;span class=&quot;math-container&quot;&gt;$(y - \hat{y})^2$&lt;/span&gt;, but I think their derivative is different, so I am confused of what derivative will I use for computing my gradient. Can someone explain for me what term to use?&lt;/p&gt;&#xA;" OwnerUserId="26070" LastEditorUserId="2444" LastEditDate="2019-05-31T16:33:15.313" LastActivityDate="2019-06-01T23:03:10.153" Title="Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?" Tags="&lt;math&gt;&lt;gradient-descent&gt;&lt;loss-functions&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12613" PostTypeId="2" ParentId="12607" CreationDate="2019-05-31T12:58:10.533" Score="0" Body="&lt;p&gt;The greedy action is the action that maximises some quantity in the present or near future. A stochastic action is an action that is &lt;a href=&quot;https://en.wikipedia.org/wiki/Randomness&quot; rel=&quot;nofollow noreferrer&quot;&gt;random&lt;/a&gt; (it can be the greedy action or any other possible action).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that you are hungry. You can either choose to eat pizza, salad, fruits or fish. Pizza is your favourite food. On the other hand, you don't like much salad, fruits and fish, but you know that they are healthier than pizza. If you choose to eat pizza, then this is a greedy action. What quantiy are you maximising if you choose pizza? Your current happiness. If you randomly pick one of those (either pizza, salad, fruits or fish), then this is a stochastic (or random) action. The stochastic action can also happen to be pizza, but, in the next day, it might not be pizza, and, in general, it will not always be pizza.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that you always choose pizza. What's going to happen? In the long run, you will get fatter and your health will deteriorate. However, everytime you choose pizza, you will get happier in that moment (local maximum). If you randomly choose the food every time you want to eat, then it is more likely that you will eat also salad, fruits and fish. In the long run, this could be more beneficial for your health, hence this could avoid you getting trapped in the local maximum (being happy in the moment that you eat, but unhappier later in life because of the possible health problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of artificial intelligence, the ideas are the same. There are several algorithms that use stochastic actions in order to avoid getting trapped in local extrema. For example, &lt;a href=&quot;https://pdfs.semanticscholar.org/e893/4a942f06ee91940ab57732953ec6a24b3f00.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulated annealing&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/a/12545/2444&quot;&gt;ant colony optimisation algorithms&lt;/a&gt;, &lt;a href=&quot;https://www.researchgate.net/publication/33784417_Learning_From_Delayed_Rewards&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning&lt;/a&gt; (using &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy) or &lt;a href=&quot;https://epubs.siam.org/doi/pdf/10.1137/1018105&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithms&lt;/a&gt;. An example of local search (or greedy) algorithm is &lt;a href=&quot;https://www.cs.ubc.ca/~hutter/previous-earg/EmpAlgReadingGroup/TSP-JohMcg97.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2-opt&lt;/a&gt; (for the TSP problem).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-05-31T14:51:42.457" LastActivityDate="2019-05-31T14:51:42.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12614" PostTypeId="2" ParentId="12612" CreationDate="2019-05-31T13:23:16.203" Score="2" Body="&lt;p&gt;The derivative is the same as far as I understand it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is constant and &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; is the variable the result will be:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$((\hat{y} - y)^2)' = 2(\hat{y} - y)$&lt;/span&gt;&lt;br&gt;&#xA;and for the other formula:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$((y - \hat{y})^2)' = -2(y - \hat{y})$&lt;/span&gt;&lt;br&gt;&#xA;which is the same.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2019-05-31T13:23:16.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12615" PostTypeId="2" ParentId="12612" CreationDate="2019-05-31T13:27:46.253" Score="7" Body="&lt;p&gt;The derivative of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the model and &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = f(x)$&lt;/span&gt; is the output of the model, is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{d}{d \hat{y}} \mathcal{L_1}&#xA;&amp;amp;= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\&#xA;&amp;amp;= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\&#xA;&amp;amp;= 2(\hat{y} - y) (1) \\&#xA;&amp;amp;= 2(\hat{y} - y) &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L_2}(y, x) = (y - \hat{y})^2 = (y - f(x))^2$&lt;/span&gt; w.r.t &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{d}{d \hat{y}} \mathcal{L_2}&#xA;&amp;amp;= \frac{d}{d \hat{y}} (y - \hat{y})^2 \\&#xA;&amp;amp;= 2(y -\hat{y}) \frac{d}{d \hat{y}} (y -\hat{y}) \\&#xA;&amp;amp;= 2(y - \hat{y})(-1)\\&#xA;&amp;amp;= -2(y - \hat{y})\\&#xA;&amp;amp;= 2(\hat{y} - y)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the derivatives of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L_1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L_2}$&lt;/span&gt; are the same.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-01T23:03:10.153" LastActivityDate="2019-06-01T23:03:10.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12616" PostTypeId="1" CreationDate="2019-05-31T14:13:07.597" Score="1" ViewCount="40" Body="&lt;p&gt;Imagine you have access to a dataset of pairs &lt;span class=&quot;math-container&quot;&gt;$(s, \hat{\pi}(s))$&lt;/span&gt; where s is a state in a high dimension continuous space &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; is a probabilistic distribution on a large &lt;strong&gt;discrete&lt;/strong&gt; space &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; (size around &lt;span class=&quot;math-container&quot;&gt;$10^{9}$&lt;/span&gt;), and &lt;span class=&quot;math-container&quot;&gt;$\hat{\pi}(s)$&lt;/span&gt; is a sample from this distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A detail is important : &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is not a distribution right, it's &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; who is one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is simple : how can an algorithm learn to, given a state s, sample from &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="26071" LastActivityDate="2019-05-31T14:13:07.597" Title="How to learn to sample?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12617" PostTypeId="2" ParentId="12607" CreationDate="2019-05-31T15:05:55.600" Score="1" Body="&lt;p&gt;As an example of local/global minima, imagine being on a rugged, mountainous landscape, and you want to find the lowest point within some area. For a greedy search, every step you take will take you downhill. If you go downhill long enough, you'll eventually find a flat spot, which is a minimum - from here, there's no step you can take that will get you any lower. However, there's a nearby ridge, which if you crossed it, you could continue downhill to find an even lower spot, the global minimum (the true lowest point). Using your greedy approach, you'll never go uphill to cross the ridge, so you'll be stuck in the local minimum forever. If you occasionally take random steps (other than directly downhill), you have the opportunity to cross ridges that separate local minima, and you have a better chance of finding the global minimum. You are correct that in many cases, the random step won't help you cross a ridge, and will just take you up a mountain in the wrong direction, which is a waste of time. But unless we allow the algorithm to &quot;explore&quot; a bit, it will be content that the first minimum it finds is the best one, and will never get to the bottom.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2019-05-31T15:05:55.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12619" PostTypeId="1" CreationDate="2019-05-31T15:49:45.397" Score="4" ViewCount="466" Body="&lt;p&gt;Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2,  Instead of cleaning all of learned memory, and relearn handwriting data 0, 1, and 2 at the same time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can CNN do the same thing? Can CNN learn something new, but keep the previous memory? If yes, the efficiency could be high.  Right now, I have to give all of data at the same time, the efficiency is very very low.&lt;/p&gt;&#xA;" OwnerUserId="26072" LastEditorUserId="2444" LastEditDate="2019-05-31T15:59:50.780" LastActivityDate="2019-05-31T16:21:04.090" Title="Can a CNN be trained incrementally?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;incremental-learning&gt;&lt;online-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12620" PostTypeId="2" ParentId="12612" CreationDate="2019-05-31T15:54:08.133" Score="7" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The MSE can be defined as &lt;span class=&quot;math-container&quot;&gt;$(\hat{y} - y)^2$&lt;/span&gt;, which should be equivalent to &lt;span class=&quot;math-container&quot;&gt;$(y - \hat{y})^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They are not just &quot;equivalent&quot;. It is actually the exact same function, with two different ways to write it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(\hat{y} - y)^2 = (\hat{y} - y)(\hat{y} - y) = \hat{y}^2 -2\hat{y}y + y^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(y - \hat{y})^2 = (y -\hat{y})(y - \hat{y}) = y^2 -2y\hat{y} + \hat{y}^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are exactly the same function. Not just &quot;equivalent&quot; or &quot;equivalent everywhere&quot;, but actually the same function. It is therefore no surprise that any derivative is also the same - including the partial derivative with respect to &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; which is what you typically use to drive gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two ways of writing the function is because it is a square and thus has two factorisations. When you write it as a square you can choose which form to use for the inner term.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Which function [form] should I use to compute the gradient?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can use either form, it does not matter. They represent the same function and have the same gradient.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-06-01T08:14:11.513" LastActivityDate="2019-06-01T08:14:11.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12621" PostTypeId="2" ParentId="12619" CreationDate="2019-05-31T16:21:04.090" Score="4" Body="&lt;p&gt;You are looking for &lt;a href=&quot;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;incremental (or online) learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A CNN can be trained incrementally. For example, in the paper &lt;a href=&quot;https://pdfs.semanticscholar.org/8f21/c99d8257c79baf22c211ed17a2224574b524.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Incremental Learning of Convolutional Neural Networks&lt;/a&gt;, the authors propose an incremental learning algorithm (inspired by &lt;a href=&quot;https://en.wikipedia.org/wiki/AdaBoost&quot; rel=&quot;nofollow noreferrer&quot;&gt;AdaBoost&lt;/a&gt; and &lt;a href=&quot;https://www.researchgate.net/publication/2489080_Learn_An_Incremental_Learning_Algorithm_for_Supervised_Neural_Networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learn++&lt;/a&gt;, which is another incremental learning algorithm for supervised learning of neural networks) for CNNs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, note that incremental learning is a challenging task, given the  &lt;em&gt;stability-plasticity dilemma&lt;/em&gt;: a completely &lt;em&gt;stable&lt;/em&gt; model, in order to keep being stable, will attempt to preserve the existing knowledge, so it will not learn new knowledge; similarly, a completely &lt;em&gt;plastic&lt;/em&gt; model, in order to keep being plastic, it will keep forgetting previously acquired knowledge so that to learn new information.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-05-31T16:21:04.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12622" PostTypeId="1" AcceptedAnswerId="12624" CreationDate="2019-05-31T16:55:03.883" Score="2" ViewCount="944" Body="&lt;p&gt;I wonder if Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) use any machine learning or deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in AR, the virtual objects are brought into the real world, does this process involve any object detection and localization?&lt;/p&gt;&#xA;" OwnerUserId="21213" LastEditorUserId="2444" LastEditDate="2019-05-31T17:00:01.650" LastActivityDate="2019-06-01T12:27:06.847" Title="Do VR, AR and MR use any machine learning or deep learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12624" PostTypeId="2" ParentId="12622" CreationDate="2019-05-31T18:31:24.590" Score="0" Body="&lt;h1&gt;TL;DR&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Yes for all the 3 cases &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Details&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;VR&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Even if the environment is virtual there is another thing to perceive which or better who is real: the user &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep Learning can be used for things like: eye tracking, gesture recognition, voice recognition, ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, in games,  you could think about more complex applications like understanding the way the user plays the game and react accordingly &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;AR and MR&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In addition to what has been said for VR, in these cases you also have to integrate the environment in the final user experience so you ultimately have to perceive it and there multiple specific applications: object detection, planes detection, depth estimation, Visual Odometry (e.g. as feature detection module), SLAM, ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These days Deep Learning is achieving Starte-Of-The-Art (SOTA) in many different tasks far beyond the classification and detection, but one things is research so published papers and so on and another thing is a product &#xA;so to sum up every VR, MR, AR vendor is probably relying on an hybrid solution where traditional algorithms and deep learning coexist but so far it seems the trend is deep learning is eating the traditional algos up day after day &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;References&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.logikk.com/articles/deep-learning-in-vr-and-ar/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The rising force of deep learning in VR and AR&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.smartdatacollective.com/virtual-reality-machine-learning-go-hand-in-hand/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Virtual Reality And Machine Learning Go Hand In Hand&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="1963" LastEditDate="2019-06-01T08:51:29.830" LastActivityDate="2019-06-01T08:51:29.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12625" PostTypeId="1" AcceptedAnswerId="12626" CreationDate="2019-05-31T18:59:56.977" Score="1" ViewCount="27" Body="&lt;p&gt;good day&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a problem where i have 9 data points that are collected every minute for 40 minutes, and by the 40th minute the solution would be either end up being black or white.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to set up a neural network; which would take the live input of every minute; and I was hoping within the 25-30 minute mark to predict the outcome of what the results would be at 40 minute; which is a classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have over 3000 historical runs of this experiment; each containing 40 rows of 9 columns data per experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What network would I need to set up; so that it can learn from each run at every minute mark per experiment with the results; and then set it up for live input, when the experiment is running again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i feel like I might need more than one system to accomplish this; any help in pointing me towards the right path would be greatly appreciated&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using python (keras) to try to solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="26075" LastActivityDate="2019-05-31T19:25:25.697" Title="What type of network for a repeated experiment" Tags="&lt;neural-networks&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12626" PostTypeId="2" ParentId="12625" CreationDate="2019-05-31T19:25:25.697" Score="0" Body="&lt;p&gt;So it seems any time series modeling would do the trick. If you new to neural nets and seems you want to play with keras, maybe start by throwing it into a simple LSTM.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-05-31T19:25:25.697" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12627" PostTypeId="1" CreationDate="2019-05-31T20:09:58.620" Score="4" ViewCount="419" Body="&lt;p&gt;I am running a basic DQN (Deep Q-Network) on the Pong environment. Not a CNN, just a 3 layer linear neural net with ReLUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to work for the most part, but at some point, my model suffers from catastrophic performance loss:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/smEex.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/smEex.png&quot; alt=&quot;Catastrophic forgetting&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is really the reason for that?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the common ways to avoid this? Clipping the gradients? What else? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Reloading from previous successful checkpoints feels more like a hack, rather than a proper solution to this issue.)&lt;/p&gt;&#xA;" OwnerUserId="26077" LastEditorUserId="2444" LastEditDate="2019-11-19T02:56:45.097" LastActivityDate="2019-11-19T02:56:45.097" Title="What could be causing the drastic performance drop of the DQN model on the Pong environment?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12629" PostTypeId="2" ParentId="11480" CreationDate="2019-05-31T21:22:45.747" Score="1" Body="&lt;p&gt;OpenAI has a series of Spinning Up pages on their website to educate people about AI. One of those defines Vanilla Policy Gradiant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/vpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vanilla Policy Gradiant via OpenAI&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the bottom of the page are reference papers that further discuss gradiants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether this is definitive for Vanilla Policy Gradients or not I do not know, but if many others refer to OpenAI for learning this subject their definition will spread.&lt;/p&gt;&#xA;" OwnerUserId="26079" LastActivityDate="2019-05-31T21:22:45.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12630" PostTypeId="1" CreationDate="2019-06-01T00:58:30.793" Score="2" ViewCount="87" Body="&lt;p&gt;I read this article: &quot;&lt;a href=&quot;https://docdro.id/2FVna0t&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards Autonomous Data Ferry Route Design through Reinforcement Learning&lt;/a&gt;&quot; by Daniel Henkel and Timothy X Brown. It specifies an infinite horizon problem where they use as a reward function for TD[0] the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;r(s,a) = - \int_{t_0}^{t_1} (Ft +N_0)e^{-\beta t} dt&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$N_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; are constant, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; is used to adjust the discount factor and &lt;span class=&quot;math-container&quot;&gt;$t_0 , t_1$&lt;/span&gt; are the initial and final time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then they proceed to use &lt;span class=&quot;math-container&quot;&gt;$e^{-\beta t}$&lt;/span&gt; as the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (discount factor) in the TD[0] update formula and in the policy formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why is the discount factor in the infinite horizon problem &lt;span class=&quot;math-container&quot;&gt;$e^{-\beta t}$&lt;/span&gt;, and why is it used as &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; update, since it is a variable factor&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in the formula of the TD[0] update they don't subtract &lt;span class=&quot;math-container&quot;&gt;$\alpha V(s)$&lt;/span&gt;. They do:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;V_{t+1}(s) = V_t(s) + \alpha( r(s,a) + e^{-\beta t_a} V_t(s'))&#xA;\end{equation}&lt;/span&gt;&#xA;I really think this is a mistake, and the values of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; will explode without it, even in an infinite horizon problem. &lt;em&gt;Am I correct ? Is &lt;span class=&quot;math-container&quot;&gt;$- V(s)$&lt;/span&gt; missing inside the brackets?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if someone is willing and has the time to directly read this part of the article and enlighten me on if &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt; represent the initial and final time of an action OR &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; is always 0 and &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt; is the duration of the action, I would appreciate it. I ask this because from what is written in the paper &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; seems to be the current time in the simulation, but I'm afraid that would just decay too fast and after some actions the reward would be close to 0. It is really not well explained and I'm a bit confused. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time if you got this far reading. Any guideline answer will be very much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="2444" LastEditDate="2019-06-01T10:41:44.753" LastActivityDate="2019-06-01T10:41:44.753" Title="Infinite horizon in Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;temporal-difference-methods&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12631" PostTypeId="1" CreationDate="2019-06-01T01:02:55.923" Score="2" ViewCount="1170" Body="&lt;p&gt;I am trying to train a deep learning model to predict an 8*2 matrix. The predicted matrix would have complex values and the input matrix would be real numbers. Can it be done? Thank you for your time.&lt;/p&gt;&#xA;" OwnerUserId="26083" LastActivityDate="2019-06-01T08:32:51.337" Title="How can I train a deep learning model to predict a matrix?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12632" PostTypeId="1" AcceptedAnswerId="12635" CreationDate="2019-06-01T02:26:03.443" Score="4" ViewCount="89" Body="&lt;p&gt;I am having a hard time converting line 6 of the prioritized experience replay algorithm from &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt; into plain English (see below):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8Q53w.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Q53w.jpg&quot; alt=&quot;algorithm&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I understand that new transitions (not visited before) are given maximal priority. On line 6 this would be done for every transition in an initial pass since the history is initialized as empty on line 2.&lt;/p&gt;&#xA;&lt;p&gt;I’m having trouble with the notation &lt;span class=&quot;math-container&quot;&gt;$p_t = \text{max}_{i&amp;lt;t}  p_i$&lt;/span&gt;. Can someone please state this in plain English? If &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; = 4 for example, then &lt;span class=&quot;math-container&quot;&gt;$p_t$&lt;/span&gt; = 4? How is this equal to max&lt;span class=&quot;math-container&quot;&gt;$_{i&amp;lt;t}  p_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It seems in my contrived example here, max&lt;span class=&quot;math-container&quot;&gt;$_{i&amp;lt;t}  p_i$&lt;/span&gt; would be 3. I must be misreading this notation.&lt;/p&gt;&#xA;" OwnerUserId="16343" LastEditorUserId="2444" LastEditDate="2020-11-01T19:55:33.190" LastActivityDate="2020-11-01T19:55:33.190" Title="What does the notation $p_t = \text{max}_{i&lt;t} p_i$ mean in algorithm 1 of the prioritized experience replay paper?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12633" PostTypeId="2" ParentId="12631" CreationDate="2019-06-01T08:18:54.097" Score="1" Body="&lt;p&gt;You could use a CNN or Fully Connected Network and output a matrix of size 8*2*2. The first 8*2 matrix is the real number and the second is the imaginary number. Example code below uses keras.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;input: numpy array of shape(batch_size,input_dim_0,input_dim_1,1)&#xA;Y: keras tensor of shape(batch_size,8,2,2)&#xA;&#xA;'''&#xA;&#xA;model = Sequential()&#xA;model.add(Conv2d(64,3,input_dim =(input_dim_0,input_dim_1,1))&#xA;model.add(Conv2d(64,3)&#xA;model.add(Conv2d(2,3)&#xA;&#xA;model.compile(loss='MSE',metrixs=['accuracy'])&#xA;model.fit(input,Y)&#xA;&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-06-01T08:32:51.337" LastActivityDate="2019-06-01T08:32:51.337" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12635" PostTypeId="2" ParentId="12632" CreationDate="2019-06-01T10:16:49.260" Score="3" Body="&lt;p&gt;From my interpretation what it means is that &lt;span class=&quot;math-container&quot;&gt;$p_t$&lt;/span&gt; is the priority value associated with each transition and &lt;span class=&quot;math-container&quot;&gt;$p_t = max_{i&amp;lt;t} p_i $&lt;/span&gt; means that the priority of transition number &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; will be the maximum between the values of the priorities of the previous elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: since &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; is initialized to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, all the new experiences will be too:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;p_2 = max\{p_1\} = 1,&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;p_3 = max\{p_1,p_2\} = 1,&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;p_4 = max\{p_1,p_2,p_3\} = 1.&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-06-01T10:22:55.153" LastActivityDate="2019-06-01T10:22:55.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12636" PostTypeId="2" ParentId="12622" CreationDate="2019-06-01T12:27:06.847" Score="0" Body="&lt;p&gt;Certainly in Virtual Reality the new Deep Convolutional Neural Networks algorithms applied in image processing are playing a role! Object detection, segmentation and semantic imaging are all features of these new ways of image processing. Augmented Reality then uses some advanced Optics features to accomplish its job. And so on...&lt;/p&gt;&#xA;" OwnerUserId="25375" LastActivityDate="2019-06-01T12:27:06.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12639" PostTypeId="1" AcceptedAnswerId="12641" CreationDate="2019-06-01T14:41:50.517" Score="5" ViewCount="367" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://incompleteideas.net/papers/sutton-88-with-erratum.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to predict by the methods of temporal differences&lt;/a&gt; (p. 15), the weights in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Temporal_difference_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;temporal difference learning&lt;/a&gt; are updated as given by the equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Delta w_t&#xA;= \alpha \left(P_{t+1} - P_t\right) \sum_{k=1}^{t}{\lambda^{t-k} \nabla_w P_k}&#xA;\tag{4}&#xA;\,.$$&lt;/span&gt;&#xA;When &lt;span class=&quot;math-container&quot;&gt;$\lambda = 0$&lt;/span&gt;, as in TD(0), how does the method learn? As it appears, with &lt;span class=&quot;math-container&quot;&gt;$\lambda = 0$&lt;/span&gt;, there will never be a change in weight and hence no learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing anything?&lt;/p&gt;&#xA;" OwnerUserId="25768" LastEditorUserId="2444" LastEditDate="2019-06-03T23:04:12.393" LastActivityDate="2019-06-03T23:04:12.393" Title="Understanding the equation of TD(0) in the paper &quot;Learning to predict by the methods of temporal differences&quot;" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12640" PostTypeId="1" AcceptedAnswerId="12651" CreationDate="2019-06-01T14:53:15.223" Score="2" ViewCount="110" Body="&lt;p&gt;I am using deep reinforcement learning to solve a classic maze escaping task, similar to the implementation provided &lt;a href=&quot;https://github.com/prasoon2211/pacman-RL/blob/master/pacman_vanilla_DQN/pacman_agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, except the following three key differences:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;instead of using a &lt;code&gt;numpy&lt;/code&gt; array as the input of a standard maze escaping task, I am feeding the model with an image at each step; the image is a 1300 * 900 RGB image, so it is not too small.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;reward: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;each valid move has a small negative reward (penalize long move) &lt;/li&gt;&#xA;&lt;li&gt;each invalid move has a big negative reward (run into other objects or boundaries) &lt;/li&gt;&#xA;&lt;li&gt;Each blocked move has the minimal reward (not common) &lt;/li&gt;&#xA;&lt;li&gt;Find the remote detectors’ defect has a positive reward (5)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I tweaked the parameters of replay memory, reduced the size of the replay memory buffer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding the implementation, I basically do not change the agent setup except the above items, and I implemented my &lt;code&gt;env&lt;/code&gt; to wrap my customized maze. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is that, the accumulated reward (first 200 rounds of successful escaping) does not increase:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kUoJU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kUoJU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the number of steps it takes to escape one maze is also stable somewhat:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZxftI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZxftI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my question, on which aspect I could start to look at to optimize my problem? Or is it still too early and I will need to train more time?&lt;/p&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-06-03T08:00:37.083" Title="Reward does not increase for a maze escaping problem with DQN" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12641" PostTypeId="2" ParentId="12639" CreationDate="2019-06-01T15:07:06.217" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When lambda = 0 as in TD(0), how does the method learn? As it appears, with lambda = 0, there will never be a change in weight and hence no learning. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the detail that you're missing is that one of the terms in the sum (the final &quot;iteration&quot; of the sum, the case where &lt;span class=&quot;math-container&quot;&gt;$k = t$&lt;/span&gt;) has &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; raised to the power &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and anything raised to the power &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (even &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) is equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. So, for &lt;span class=&quot;math-container&quot;&gt;$\lambda = 0$&lt;/span&gt;, your update equation becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Delta w_t = \alpha \left( P_{t+1} - P_t \right) \nabla_w P_t,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is basically a one-step update (just like Sarsa).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-06-01T15:07:06.217" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12642" PostTypeId="1" CreationDate="2019-06-01T17:30:01.860" Score="1" ViewCount="257" Body="&lt;p&gt;I want to implement super-resolution and deblurring on images from text documents. Which is the best approach? Are there any Git-hub links which will help me to start? I am new to the field. Any help would be appreciated. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="21797" LastEditDate="2019-06-04T14:19:11.857" LastActivityDate="2019-06-04T14:19:11.857" Title="Super Resolution on text documents" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12643" PostTypeId="2" ParentId="7222" CreationDate="2019-06-01T21:54:03.680" Score="0" Body="&lt;p&gt;Its definitely simpler task as NLP or mashine learning, its &lt;strong&gt;keyword&lt;/strong&gt; based. And you have a little bit wrong view on that. See your example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Example: java-developer, java web engineer and java software developer shall all be mapped to the profession java engineer.&lt;/em&gt;&#xA;Not at all : java web and java are &lt;strong&gt;different&lt;/strong&gt; jobs, while java software developer = java-developer and word software means nothing, cause java already stand for software. The info can NOT be mined from texts like job applications -&gt; you have no link to a sence what title is , and better just create the mapping by hand - its not so long. Then, just look in text for key words and ignore other words&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-01T21:54:03.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12646" PostTypeId="1" CreationDate="2019-06-02T14:01:49.667" Score="3" ViewCount="71" Body="&lt;p&gt;I started to learn reinforcement learning a few days ago. And I want to use that to solve resource allocation problem something like given a constant number, find the best way to divide it into several real numbers each is non-negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, to divide the number 1 into 3 real numbers, the allocation can be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[0.2, 0.7, 0.1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[0.95, 0.05, 0]&#xA;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to represent the action space because each allocation is 3-dimensional and each dimension is real-valued and each other correlated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In actor-critic architecture, is it possible to have 3 outputs activated by softmax in the actor's network each represents one dimension in the allocation?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Appended:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a playlist of videos. A user can switch to the next video at any time. More buffer leads to better viewing experience but more bandwidth loss if user switches to the next video. I want to optimize the smoothness of playback with minimal bandwidth loss. At each time step, the agent decides the bandwidth allocation to download current video and the next 2 videos. So I guess the state will be the bandwidth, user's behavior and the player situation.&lt;/p&gt;&#xA;" OwnerUserId="26099" LastEditorUserId="26099" LastEditDate="2019-06-02T15:56:07.350" LastActivityDate="2019-06-02T15:56:07.350" Title="How to represent action space in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12647" PostTypeId="1" CreationDate="2019-06-02T19:47:24.853" Score="2" ViewCount="80" Body="&lt;p&gt;As far as I understand, &lt;a href=&quot;https://www.youtube.com/watch?v=RLWuzLLSIgw&quot; rel=&quot;nofollow noreferrer&quot;&gt;beam search&lt;/a&gt; is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?&lt;/p&gt;&#xA;" OwnerUserId="12746" LastActivityDate="2020-02-19T12:43:51.787" Title="Does the human brain use beam search for text generation?" Tags="&lt;natural-language-processing&gt;&lt;brain&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12648" PostTypeId="1" CreationDate="2019-06-02T20:45:36.050" Score="1" ViewCount="118" Body="&lt;p&gt;I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: I divided it into consecutive batches of 96 steps (the window size that my NN gets as input) and I normalized each batch respectively with Z-score. However, I am unable to extend these results to an online setting, as online I only have access to the &lt;em&gt;last 96&lt;/em&gt; elements, and thus I can only normalize according to the &lt;em&gt;last 96&lt;/em&gt;. This small difference actually causes a sharp decrease in the performance of my DQN, as the weights of the NN were perfectly tuned for the first normalization but are not great with the online normalized database. In a nutshell, the problem is that only every 96 steps the first normalized database and the online one are the same, for all steps in between this is not happening. I have the weights for the first one, but I cannot find a way to exploit them for the online one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have tried so far with the online database:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If I normalize every last 96 steps, and act for every new step (as it should be), the performances are quite bad.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I normalize every last 96 steps, and act just every 96 steps (repeating the same action in between), the agent is actually picking the optimal action every 96 steps (like in the offline setting), so the results are somewhat decent but far from optimal for the long period between the actions. If I try with shorter periods, like 48, performances decrease sharply as it only acts optimally every 2 actions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't know if there is a way to tune the optimal weights for the online database, acting directly on them without going through training again. It would be nice to understand why the NN picks its actions at each step in the optimal setting, so that I would be able to follow its strategy, but I'm not aware if it's possible to actually deduct this from the analysis of weights and features, especially for a multi-layer LSTM network. &#xA;Otherwise, I was thinking about something like normalizing the online database directly through similarities with the old batches of 96 (using their mean and std) or something like that. Anything that would help reducing the time between optimal actions to around 50-60 steps instead of 96 would be enough to provide a nearly optimal strategy, so at this point, I would consider any kind of (unelegant) method to get what I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if any of these is feasible, but retraining the agent is very difficult as every single time but once the agent got stuck in suboptimal strategies, this is why I am trying to get around this problem using the optimal weights I have instead of retraining. &lt;/p&gt;&#xA;" OwnerUserId="23638" LastActivityDate="2019-06-02T20:45:36.050" Title="Online normalization of database for DQN" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;dqn&gt;&lt;online-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12649" PostTypeId="1" AcceptedAnswerId="12650" CreationDate="2019-06-03T05:44:59.120" Score="3" ViewCount="73" Body="&lt;p&gt;I was hoping someone could just confirm some intuition about how convolutions&#xA;work in convolutional neural networks. I have seen all of the tutorials on&#xA;applying convolutional filters on an image, but most of those tutorials &#xA;focus on one channel images, like a 128 x 128 x 1 image. I wanted to clarify&#xA;what happens when we apply a convolutional filter to RGB 3 channel images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this is not a unique question, I think a lot of people ask this question as well. It is just that there seem to be so many answers out there, each with their own variations, that it is hard to find a consistent answer. I included a post below that seems to comport with what my own intuition, but I was hoping one of the experts on SE could help validate the layer arithmetic, to make sure my intuition was not off. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/6934/convolutional-neural-nets-and-reduction-of-the-layers&quot;&gt;Convolutional neural nets and reduction of the layers&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider an Alexnet network with 5 convolutional layers and 3 fully connected&#xA;layers. I borrowed the network from this &lt;a href=&quot;https://medium.com/@smallfishbigsea/a-walk-through-of-alexnet-6cbd137a5637&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt;.  &lt;strong&gt;Now, say the input is 227 x 227, and the filter is specified&#xA;as 11 x 11 x 96 with stride 4. That means there are 96 filters each with dimensions 11x11x3, right?&lt;/strong&gt;&#xA;So there are a total of 363 parameters per filter--excluding the bias term--&#xA;and there are 96 of these filters to learn. So the 363*96 = 34848 filter values are learned &#xA;just like the weights in the fully connected layers right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second question deals with the next convolutional network layer. &lt;strong&gt;In the next &#xA;layer I will have an image that is 55 x 55 x 96 image. In this case, would the&#xA;filter be 5x5x96--since there are now 96 feature maps on the image?&lt;/strong&gt; So that means&#xA;that each individual filter would need to learn 5x5x96 = 2400 filter values (weights),&#xA;and that across all 256 filters this would mean 614,400 filter values? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just wanted to make sure that I was understanding exactly what is being learned&#xA;at each level. &lt;/p&gt;&#xA;" OwnerUserId="15765" LastActivityDate="2019-06-03T07:47:53.283" Title="Neural Nets: CNN confirming layer/filter arithmetic" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12650" PostTypeId="2" ParentId="12649" CreationDate="2019-06-03T07:47:53.283" Score="3" Body="&lt;p&gt;Your first point is correct. The filters are stored in 4d arrays, with dimensions of (height, width, input channels, filter number) . The order may differ.&#xA;Your second point is correct too. The filtered result get stacked together so the output dimensions are (Height,width,filter numbers) the next layer's filters are of size( filter width, filter width, last layer's filter number). Your understanding of CNN is correct. If you want additional resources on CNN, you can try Andrew Ng's class on CNN in Coursera. Hope you can learn more about CNN.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-06-03T07:47:53.283" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12651" PostTypeId="2" ParentId="12640" CreationDate="2019-06-03T08:00:37.083" Score="1" Body="&lt;p&gt;You should use an algorithm to try doing a solve for the maze optimally, maybe A* algorithm. If the optimal steps is also in the range of your network, your network may have reached it's best. If the optimal step is much less, you can try increasing the step penalty and increasing the reward for reaching the end. Hope you can succeed in this problem.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-06-03T08:00:37.083" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12652" PostTypeId="2" ParentId="12127" CreationDate="2019-06-03T08:04:02.737" Score="0" Body="&lt;p&gt;In that particular competition, you can try using GAN to generate new data or adding noise to existing data. You can also use K-means algorithm. You can try using a smaller network and remove bias. May be you can use logistic regression to compare the result. You can also use a PCA method.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-06-03T08:04:02.737" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12653" PostTypeId="2" ParentId="12600" CreationDate="2019-06-03T08:13:53.127" Score="0" Body="&lt;p&gt;For the resources, you can refer to this:&#xA;&lt;a href=&quot;https://becominghuman.ai/generative-adversarial-networks-for-text-generation-part-1-2b886c8cab10&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://becominghuman.ai/generative-adversarial-networks-for-text-generation-part-1-2b886c8cab10&lt;/a&gt;&#xA;If you want to generate text review for specific score, you can input a noise vector and the score to the generator. You could also make a vector filled with the number of score and add noise to that vector instead. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-06-03T08:13:53.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12654" PostTypeId="1" CreationDate="2019-06-03T08:33:06.490" Score="4" ViewCount="109" Body="&lt;p&gt;How do I identify monologues and dialogues in a conversation (or transcript) using natural language processing? How do I distinguish between the two?&lt;/p&gt;&#xA;" OwnerUserId="26113" LastEditorUserId="2444" LastEditDate="2019-06-03T12:31:41.633" LastActivityDate="2019-06-07T13:23:54.230" Title="How do I identify a monologue or dialogue in a conversation?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;machine-translation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12655" PostTypeId="1" CreationDate="2019-06-03T09:39:45.660" Score="1" ViewCount="53" Body="&lt;p&gt;I have a corpus of a domain data in form of 10-15 books pdf and some articles and my end-goal is to make a question-answering system particular to that domain.&#xA;For that, I would need a dataset on Q/A which I can use on top of something like SQuAD(&lt;strong&gt;S&lt;/strong&gt;tanford &lt;strong&gt;Qu&lt;/strong&gt;estion &lt;strong&gt;A&lt;/strong&gt;nswering &lt;strong&gt;D&lt;/strong&gt;ataset) for domain-specific knowledge&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My stuck point is &lt;em&gt;how to convert this corpus into a usable question-answering dataset&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current strategy is something &lt;strong&gt;AllenAI&lt;/strong&gt; has been working with. A list of their research papers on it can be found &lt;a href=&quot;https://allenai.org/aristo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand they use a combination of &lt;em&gt;Knowledge Extraction&lt;/em&gt;, &lt;em&gt;Natural Language Understanding&lt;/em&gt;, and &lt;em&gt;Inference&lt;/em&gt; to get the job done. But I cannot find any good practical implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find a good resource?&lt;/p&gt;&#xA;" OwnerUserId="17530" LastEditorUserId="12853" LastEditDate="2019-06-05T10:07:09.660" LastActivityDate="2019-06-05T10:07:09.660" Title="Generate QA dataset from large text corpus" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12656" PostTypeId="1" CreationDate="2019-06-03T10:45:37.737" Score="4" ViewCount="119" Body="&lt;p&gt;When we use BERT embeddings for a classification task, would we get different embeddings every time we pass the same text through the BERT architecture? If yes, is it the right way to use the embeddings as features? Ideally, while using any feature extraction technique, features values should be consistent. How do I handle this if we want BERT to be used as a feature extractor?&lt;/p&gt;&#xA;" OwnerUserId="26115" LastEditorUserId="2444" LastEditDate="2019-11-01T02:46:04.260" LastActivityDate="2020-07-28T04:07:00.697" Title="Will BERT embedding be always same for a given document when used as a feature extractor" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12657" PostTypeId="1" AcceptedAnswerId="12809" CreationDate="2019-06-03T10:53:12.873" Score="1" ViewCount="170" Body="&lt;p&gt;The Back propagation through time on recurrent layer is defined similar to  normal one, means somethin like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;self.deltas[x] =  self.deltas[x+1].dot(self.weights[x].T) * self.layers[x] * (1- self.layers[x])&lt;/code&gt; where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;self.deltas[x+1]&lt;/code&gt; is error from prevous layer, &lt;code&gt;self.weights[x]&lt;/code&gt; is weights map and &lt;code&gt;self.layers[x](1- self.layers[x])&lt;/code&gt; is bakwards activation of sigmoid function where &lt;code&gt;self.layers[x]&lt;/code&gt; is vector of sigmoid. But while normal backpropagation the values are there, while BPTT i can not take the current &lt;code&gt;self.layers[x]&lt;/code&gt; : i need the previous ones, right ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So unlike normal BP, do i need extra store old weights and layers, for example in circular queue, and then apply the formula where &lt;code&gt;self.deltas[x+1]&lt;/code&gt; is layer from next time ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not realy implementation, just basic understanding in order to can implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets see the picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NwXQ9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NwXQ9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are : self.layers[0] = &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt;, self.layers[&lt;em&gt;1&lt;/em&gt;] = &lt;span class=&quot;math-container&quot;&gt;$h_{t+1}$&lt;/span&gt; , self.layers[2] = &lt;span class=&quot;math-container&quot;&gt;$o_{t+1}$&lt;/span&gt;, in order to perform backprop &lt;span class=&quot;math-container&quot;&gt;$h_{t+1}$&lt;/span&gt; -&gt; &lt;span class=&quot;math-container&quot;&gt;$h_{t}$&lt;/span&gt; -&gt; &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt;...  &lt;strong&gt;I DO NEED to have layers &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt; ,&lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt;... and weights &lt;span class=&quot;math-container&quot;&gt;$v_{t+1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt;...  EXTRA stored in additional to the network &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt; -&gt; &lt;span class=&quot;math-container&quot;&gt;$h_{t+1}$&lt;/span&gt; -&gt; &lt;span class=&quot;math-container&quot;&gt;$o_{t+1}$&lt;/span&gt;, right?&lt;/strong&gt; &#xA;Thats all the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And i do not need to store previous outputs &lt;span class=&quot;math-container&quot;&gt;$o[t, o_{t-1}, etc..]$&lt;/span&gt;, because backprop from them ot-&gt;ht, etc was &lt;em&gt;already&lt;/em&gt; calculated. &lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="25496" LastEditDate="2019-06-10T17:00:22.490" LastActivityDate="2019-06-13T12:19:39.980" Title="Do you need to store prevous values of weights and layers on recurrent layer while BPTT?" Tags="&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12659" PostTypeId="1" CreationDate="2019-06-03T14:34:41.017" Score="28" ViewCount="6415" Body="&lt;p&gt;As an AI layman, till today I am confused by the promised and achieved improvements of automated translation.&lt;/p&gt;&#xA;&lt;p&gt;My impression is: &lt;strong&gt;there is still a very, very far way to go.&lt;/strong&gt; Or are there other explanations why the automated translations (offered and provided e.g. by Google) of quite simple Wikipedia articles still read and sound mainly silly, are hardly readable, and only very partially helpful and useful?&lt;/p&gt;&#xA;&lt;p&gt;It may depend on personal preferences (concerning readability, helpfulness, and usefulness), but &lt;strong&gt;my personal expectations&lt;/strong&gt; are disappointed sorely.&lt;/p&gt;&#xA;&lt;p&gt;The other way around: Are Google's translations nevertheless readable, helpful, and useful &lt;strong&gt;for a majority of users&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Or does Google have reasons to &lt;strong&gt;retain its achievements&lt;/strong&gt; (and not to show to the users the best they can show)?&lt;/p&gt;&#xA;" OwnerUserId="25362" LastEditorUserId="2444" LastEditDate="2021-01-18T12:29:33.707" LastActivityDate="2021-01-18T12:29:33.707" Title="What is the actual quality of machine translations?" Tags="&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;machine-translation&gt;&lt;google-translate&gt;" AnswerCount="9" CommentCount="1" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12660" PostTypeId="2" ParentId="12656" CreationDate="2019-06-03T15:29:43.173" Score="1" Body="&lt;p&gt;BERT is deterministic. There is no variation unless you parse your tokens differently in succeeding runs. Here is the original paper the model architecture is based off of &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformer Paper&lt;/a&gt;. Note that in every layer, the only operations used for the most part are matrix multiplications, concatenations, basic ops, and layer normalizations, all of which are deterministic.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-03T15:29:43.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12661" PostTypeId="2" ParentId="12659" CreationDate="2019-06-03T15:43:11.623" Score="7" Body="&lt;p&gt;Google's translations &lt;em&gt;can&lt;/em&gt; be useful, especially if you know that the translations are not perfect and if you just want to have an initial idea of the meaning of the text (whose Google's translations can sometimes be quite misleading or incorrect). I wouldn't recommend Google's translate (or any other non-human translator) to perform a serious translation, unless it's possibly a common sentence or word, it does not involve very long texts and informal language (or slang), the translations involve the English language or you do not have access to a human translator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://translate.google.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Translate&lt;/a&gt; currently uses a &lt;a href=&quot;https://arxiv.org/pdf/1609.08144.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural machine translation system&lt;/a&gt;. To evaluate this model (and similar models), the &lt;a href=&quot;https://en.wikipedia.org/wiki/BLEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLEU metric&lt;/a&gt; (a scale from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; corresponds to the human gold-standard translation) and side-by-side evaluations (a human rates the translations) have been used. If you use only the BLEU metric, the machine traslations are quite poor (but the BLEU metric is also not a perfect evaluation metric, because there's often more than one translation of a given sentence). However, GNMT reduces the translation errors compared to &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_machine_translation&quot; rel=&quot;nofollow noreferrer&quot;&gt;phrase-based machine translation (PBMT)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1901.02918.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Making AI Meaningful Again&lt;/a&gt;, the authors also discuss the difficulty of the task of translation (which is believed to be an &lt;a href=&quot;https://ai.stackexchange.com/a/12147/2444&quot;&gt;AI-complete problem&lt;/a&gt;). They also mention the &lt;a href=&quot;https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer&lt;/a&gt; (another state-of-the-art machine translation model), which achieves quite poor results (evaluated using the BLEU metric).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, machine translation is a hard problem and current machine translation systems definitely do not perform as well as a professional human translator.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-06T22:51:54.737" LastActivityDate="2019-06-06T22:51:54.737" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12662" PostTypeId="2" ParentId="12659" CreationDate="2019-06-03T17:03:14.483" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I wrong and Google's translations are nevertheless readable, helpful and useful for a majority of users?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, they are somewhat helpful and allow you to translate faster.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or does Google have reasons to retain its greatest achievements (and not to&#xA;  show to the users the best they can show)? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Maybe, I don't know. If you search for info, Google does really do a lot of horrible stupid stuff, like learning from what users say on the internet, taking unsuitable data as trusted input data sets. &lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="2193" LastEditDate="2019-06-04T08:19:23.543" LastActivityDate="2019-06-04T08:19:23.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12663" PostTypeId="2" ParentId="12659" CreationDate="2019-06-03T17:15:14.583" Score="22" Body="&lt;p&gt;Who claimed that machine translation is as good as a human translator? For me, as a professional translator who makes his living on translation for 35 years now, MT means that my daily production of human quality translation has grown by factor 3 to 5, depending on complexity of the source text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot agree that the quality of MT goes down with the length of the foreign language input. That used to be true for the old systems with semantic and grammatical analyses. I don't think that I know all of the old systems (I know Systran, a trashy tool from Siemens that was sold from one company to the next like a Danaer's gift, XL8, Personal Translator and Translate), but even a professional system in which I invested 28.000 DM (!!!!) failed miserably. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the sentence: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On this hot summer day I had to work and it was a pain in the ass.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;can be translated using several MT tools to German.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.linguatec.de/en/translation/personal-translator-professional-20/&quot; rel=&quot;noreferrer&quot;&gt;Personal Translator 20&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Auf diesem heißen Sommertag musste ich arbeiten, und es war ein Schmerz im Esel.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.online-translator.com/&quot; rel=&quot;noreferrer&quot;&gt;Prompt&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An diesem heißen Sommertag musste ich arbeiten, und es war ein Schmerz im Esel.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.deepl.com/en/translator&quot; rel=&quot;noreferrer&quot;&gt;DeepL&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An diesem heißen Sommertag musste ich arbeiten und es war eine Qual.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Google: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An diesem heißen Sommertag musste ich arbeiten und es war ein Schmerz im Arsch.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Today, Google usually presents me with readable, nearly correct translations and DeepL is even better. Just this morning I translated 3500 words in 3 hours and the result is flawless although the source text was full of mistakes (written by Chinese).&lt;/p&gt;&#xA;" OwnerUserId="26120" LastEditorUserId="2444" LastEditDate="2019-06-03T17:47:27.307" LastActivityDate="2019-06-03T17:47:27.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12664" PostTypeId="2" ParentId="5577" CreationDate="2019-06-03T18:14:38.733" Score="0" Body="&lt;p&gt;For the formats above you could write a one normal CFG parser what would extract AST tree. That actualy you want as output?&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-03T18:14:38.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12665" PostTypeId="1" CreationDate="2019-06-03T18:24:02.590" Score="1" ViewCount="58" Body="&lt;p&gt;Im trying to implement CNN for small images classification (36x36x1) (grayscale). I've checked every forward/backward pass function on small example, and still my cnn is not doin any progress on training. Tests were done on learning rate [0.001 - 0.01].&#xA;Network structure:&#xA;Conv -&gt; Relu -&gt; Conv -&gt; Relu -&gt; MaxPooling -&gt; Conv -&gt; Relu -&gt; Conv -&gt; Relu -&gt; MaxPooling -&gt; Flattening -&gt; sigmoid -&gt; Fully connected -&gt; sigmoid -&gt;Fully connected -&gt; softmax&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a mistake in forwardPass/ backwardPass function?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def forwardPass(self, X):&#xA;    &quot;&quot;&quot;&#xA;        :param X:   batch of images (Input value of entire convolutional neural network)&#xA;                    image.shape = (m,i,i,c) - c is number of channels&#xA;                    for current task, first input c = 1 (grayscale)&#xA;                    example: for RGB c = 3&#xA;                    m - batch size&#xA;                    X.shape M x I x I x C&#xA;&#xA;        :return :   touple(Z, inValues)&#xA;                    Z - estimated probability of every class&#xA;                    Z.shape M x K x 1&#xA;&#xA;&#xA;    &quot;&quot;&quot;&#xA;&#xA;    W = self.weights&#xA;&#xA;    inValues = {&#xA;        'conv': [],&#xA;        'fullyconnect': [],&#xA;        'mask' : [],&#xA;        'pooling' : [],&#xA;        'flatten' : [],&#xA;        'sigmoid' : [],&#xA;        'relu' : []&#xA;    }&#xA;&#xA;    &quot;&quot;&quot;&#xA;        Current structure:&#xA;        Conv -&amp;gt; Relu -&amp;gt; Conv -&amp;gt; Relu -&amp;gt; MaxPooling -&amp;gt; Conv -&amp;gt; Relu -&amp;gt; Conv -&amp;gt; Relu -&amp;gt; MaxPooling -&amp;gt; Flattening -&amp;gt; &#xA;        -&amp;gt; sigmoid -&amp;gt; Fully connected -&amp;gt; sigmoid -&amp;gt;Fully connected -&amp;gt; softmax&#xA;    &quot;&quot;&quot;&#xA;&#xA;&#xA;&#xA;    inValues['conv'].append(X)&#xA;    Z = self.convolution_layer(X, W['conv'][0]);z = Z&#xA;&#xA;    inValues['relu'].append(z)&#xA;    Z = self.relu(z);z =Z&#xA;&#xA;    inValues['conv'].append(z)&#xA;    Z = self.convolution_layer(z, W['conv'][1]);z = Z&#xA;&#xA;&#xA;    inValues['relu'].append(z)&#xA;    Z = self.relu(z);z =Z&#xA;&#xA;&#xA;    inValues['pooling'].append(z)&#xA;    Z, mask = self.max_pooling(z);z = Z&#xA;    inValues['mask'].append(mask)&#xA;&#xA;&#xA;&#xA;    inValues['conv'].append(z)&#xA;    Z = self.convolution_layer(z, W['conv'][2]);z = Z&#xA;&#xA;    inValues['relu'].append(z)&#xA;    Z = self.relu(z);z = Z&#xA;&#xA;    inValues['conv'].append(z)&#xA;    Z = self.convolution_layer(z, W['conv'][3]);z = Z&#xA;&#xA;    inValues['relu'].append(z)&#xA;    Z = self.relu(z);z = Z&#xA;&#xA;    inValues['pooling'].append(z)&#xA;    Z, mask = self.max_pooling(z);z = Z&#xA;    inValues['mask'].append(mask)&#xA;&#xA;&#xA;    inValues['flatten'].append(z)&#xA;    Z = self.flattening(z);z = Z&#xA;&#xA;&#xA;    inValues['sigmoid'].append(z)&#xA;    Z = self.sigmoid(z); z = Z&#xA;&#xA;&#xA;    inValues['fullyconnect'].append(z)&#xA;    Z = self.fullyConnected_layer(z, W['fullyconnect'][0]); z = Z&#xA;&#xA;&#xA;    #dropout here later&#xA;&#xA;    inValues['sigmoid'].append(z)&#xA;    Z = self.sigmoid(z); z = Z&#xA;&#xA;&#xA;    inValues['fullyconnect'].append(z)&#xA;    Z = self.fullyConnected_layer(z, W['fullyconnect'][1]);z = Z&#xA;&#xA;&#xA;    Z = self.softmax(z)&#xA;&#xA;&#xA;    return Z, inValues&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Backpropagation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; def backwardPass(self, y, Y, inValues):&#xA;&#xA;    &quot;&quot;&quot;&#xA;&#xA;    :param Y: estimated probability of all K classes&#xA;                ( Y.shape = M x K x 1 )&#xA;    :param y: True labels for current&#xA;                M x K x 1&#xA;    :param inValues: Dictionary with input values of conv/ff layers&#xA;                     example: inValues['conv'][1] - Values encountered during feedForward on input of Conv layer with index 1&#xA;    :return:  Gradient of weights in respect to L&#xA;    &quot;&quot;&quot;&#xA;&#xA;    np.set_printoptions(suppress=True)&#xA;    W = self.weights&#xA;&#xA;    G = {&#xA;        'conv' : [],&#xA;        'fullyconnect' : []&#xA;    }&#xA;&#xA;&#xA;    Z = self.softmax_backward(Y, y); z = Z&#xA;&#xA;&#xA;&#xA;    Z, dW, dB = self.fullyConnected_layer_backward(z, W['fullyconnect'][1],inValues['fullyconnect'][1]);z = Z&#xA;    weight = {&#xA;        'W': dW,&#xA;        'B': dB&#xA;    }&#xA;    G['fullyconnect'].append(weight)&#xA;&#xA;&#xA;&#xA;    Z = self.sigmoid_deriv(z, inValues['sigmoid'][1]); z = Z&#xA;&#xA;    Z, dW, dB = self.fullyConnected_layer_backward(z, W['fullyconnect'][0],inValues['fullyconnect'][0]);z = Z;&#xA;    weight = {&#xA;        'W': dW,&#xA;        'B': dB&#xA;    }&#xA;    G['fullyconnect'].append(weight)&#xA;&#xA;&#xA;    Z = self.sigmoid_deriv(z, inValues['sigmoid'][0]);z=Z&#xA;&#xA;&#xA;    Z = self.flattening_backward(z, inValues['flatten'][0]); z = Z&#xA;&#xA;&#xA;    Z = self.max_pooling_backward(z,inValues['mask'][1]); z = Z&#xA;&#xA;&#xA;    Z = z * self.relu(inValues['relu'][3], deriv=True); z = Z&#xA;&#xA;    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][3],inValues['conv'][3]); z = Z&#xA;    weight = {&#xA;        'W': dW,&#xA;        'B': dB&#xA;    }&#xA;    G['conv'].append(weight)&#xA;&#xA;    Z = z * self.relu(inValues['relu'][2], deriv=True);z = Z&#xA;&#xA;&#xA;    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][2],inValues['conv'][2]); z = Z&#xA;    weight = {&#xA;        'W': dW,&#xA;        'B': dB&#xA;    }&#xA;    G['conv'].append(weight)&#xA;&#xA;&#xA;    Z = self.max_pooling_backward(z,inValues['mask'][0]);z = Z&#xA;&#xA;&#xA;    Z = z * self.relu(inValues['relu'][1], deriv=True);z = Z&#xA;&#xA;    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][1],inValues['conv'][1]); z = Z&#xA;    weight = {&#xA;        'W': dW,&#xA;        'B': dB&#xA;    }&#xA;    G['conv'].append(weight)&#xA;&#xA;    Z = z * self.relu(inValues['relu'][0], deriv=True);z = Z&#xA;&#xA;    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][0],inValues['conv'][0]); z = Z&#xA;    weight = {&#xA;        'W': dW,&#xA;        'B': dB&#xA;    }&#xA;    G['conv'].append(weight)&#xA;&#xA;    G['conv'].reverse()&#xA;    G['fullyconnect'].reverse()&#xA;&#xA;    return G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;update:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def update(self, alfa, W, G):&#xA;&#xA;    W['fullyconnect'][0]['W'] -= alfa * np.sum(G['fullyconnect'][0]['W'],axis=0)&#xA;    W['fullyconnect'][1]['W'] -= alfa * np.sum(G['fullyconnect'][1]['W'],axis=0)&#xA;    W['fullyconnect'][0]['B'] -= alfa * np.sum(G['fullyconnect'][0]['B'],axis=0)&#xA;    W['fullyconnect'][1]['B'] -= alfa * np.sum(G['fullyconnect'][1]['B'],axis=0)&#xA;&#xA;    W['conv'][0]['W'] -= alfa * np.sum(G['conv'][0]['W'],axis=0)&#xA;    W['conv'][1]['W'] -= alfa * np.sum(G['conv'][1]['W'],axis=0)&#xA;    W['conv'][2]['W'] -= alfa * np.sum(G['conv'][2]['W'],axis=0)&#xA;    W['conv'][3]['W'] -= alfa * np.sum(G['conv'][3]['W'],axis=0)&#xA;    W['conv'][0]['B'] -= alfa * np.sum(G['conv'][0]['B'],axis=0)&#xA;    W['conv'][1]['B'] -= alfa * np.sum(G['conv'][1]['B'],axis=0)&#xA;    W['conv'][2]['B'] -= alfa * np.sum(G['conv'][2]['B'],axis=0)&#xA;    W['conv'][3]['B'] -= alfa * np.sum(G['conv'][3]['B'],axis=0)&#xA;&#xA;    return W&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26122" LastActivityDate="2019-06-03T18:24:02.590" Title="Convolutional neural network debugging" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12666" PostTypeId="1" AcceptedAnswerId="12668" CreationDate="2019-06-03T19:35:23.407" Score="2" ViewCount="75" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider the following data with one input (x) and one output (y):&lt;br&gt;&#xA;  (x=1, y=2)&lt;br&gt;&#xA;  (x=2, y=1)&lt;br&gt;&#xA;  (x=3, y=2)&lt;br&gt;&#xA;  Apply linear regression on this data, using the hypothesis &lt;span class=&quot;math-container&quot;&gt;$h_Θ(x) = Θ_0 + Θ_1 x$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Θ_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Θ_1$&lt;/span&gt; represent the parameters to be learned. Considering the initial values &lt;span class=&quot;math-container&quot;&gt;$Θ_0$&lt;/span&gt;= 1.0, and &lt;span class=&quot;math-container&quot;&gt;$Θ_1$&lt;/span&gt; = 0.0, and learning rate 0.1, what will be the values of &lt;span class=&quot;math-container&quot;&gt;$Θ_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Θ_1$&lt;/span&gt; after the first three iterations of Gradient Descent&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From least squares method I took the derivative with respect to &lt;span class=&quot;math-container&quot;&gt;$Θ_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Θ_1$&lt;/span&gt; and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new &lt;span class=&quot;math-container&quot;&gt;$Θ_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Θ_1$&lt;/span&gt; values.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am getting &lt;span class=&quot;math-container&quot;&gt;$Θ_0$&lt;/span&gt; as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve&lt;/p&gt;&#xA;" OwnerUserId="25984" LastActivityDate="2019-06-03T20:55:12.187" Title="Calculating Parameter value Using Gradient Descent for Linear Regression Model" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12667" PostTypeId="1" CreationDate="2019-06-03T20:50:38.097" Score="0" ViewCount="101" Body="&lt;p&gt;Stories like &lt;a href=&quot;https://aiweirdness.com/post/185339301987/once-again-a-neural-net-tries-to-name-cats&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one are quite popular these days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of training a neural net to do something silly like this may sound trivial to experts like you,  but for a novice like me it could be an interesting learning experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there novice-friendly software I could play with to train a neural net to do something like this or is there necessarily a steep learning curve?&lt;/p&gt;&#xA;" OwnerUserId="7078" LastEditorUserId="8068" LastEditDate="2019-06-04T18:25:51.557" LastActivityDate="2019-06-19T23:12:50.630" Title="Neural nets for novices" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="1" ClosedDate="2021-02-06T17:48:24.437" ContentLicense="CC BY-SA 4.0" />
  <row Id="12668" PostTypeId="2" ParentId="12666" CreationDate="2019-06-03T20:55:12.187" Score="1" Body="&lt;pre&gt;&lt;code&gt;X = np.array([1,2,3])&#xA;Y = np.array([2,1,2])&#xA;&#xA;params = np.array([1, 0])&#xA;&#xA;def loss(y, yhat):&#xA;    return ((y - yhat)**2).mean()&#xA;&#xA;def model(x):&#xA;    return params[0] + params[1]*x&#xA;&#xA;def loss_grad(y, yhat, x):&#xA;    return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])&#xA;&#xA;lr = .1&#xA;for _ in range(3):&#xA;    yhat = model(X)&#xA;    l = loss(Y, yhat)&#xA;    g = loss_grad(Y, yhat, X)&#xA;    params = params - lr*g&#xA;    print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;outputs &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666&#xA;thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296&#xA;thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-03T20:55:12.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12669" PostTypeId="1" CreationDate="2019-06-03T21:15:02.727" Score="3" ViewCount="30" Body="&lt;p&gt;I read the &lt;a href=&quot;https://arxiv.org/pdf/1503.03832.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facenet&lt;/a&gt; paper and one thing I am not sure about (it might be trivial and I missed it) is how do we give the kick start to the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The embeddings in the beginning are random, so picking hard (or semi-hard) negatives, based on the Euclidean distance, would give random images in the beginning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we hope that over time this will converge to the actual desired hard images? Is it any reason to expect that this convergence will be attained?&lt;/p&gt;&#xA;" OwnerUserId="23871" LastEditorUserId="2444" LastEditDate="2019-06-04T21:02:34.467" LastActivityDate="2019-06-04T21:02:34.467" Title="How do we give a kick start to the Facenet network?" Tags="&lt;convolutional-neural-networks&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12670" PostTypeId="2" ParentId="12667" CreationDate="2019-06-03T21:20:38.567" Score="2" Body="&lt;p&gt;keras is probably the highest level and easiest to go into.&#xA;Here are some &lt;a href=&quot;https://www.pyimagesearch.com/2018/09/10/keras-tutorial-how-to-get-started-with-keras-deep-learning-and-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;keras tutorials&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-03T21:20:38.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12671" PostTypeId="1" AcceptedAnswerId="12672" CreationDate="2019-06-03T22:04:28.960" Score="2" ViewCount="112" Body="&lt;p&gt;As stated in the universal approximation theorem, a neural network can approximate almost any function.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to calculate the closed-form (or analytical) expression of the function that a neural network computes/approximates?&lt;/p&gt;&#xA;&lt;p&gt;Or, alternatively, figure out if the function is linear or non-linear?&lt;/p&gt;&#xA;" OwnerUserId="26130" LastEditorUserId="2444" LastEditDate="2021-01-03T20:18:36.380" LastActivityDate="2021-01-03T20:36:56.073" Title="Is there a way to calculate the closed-form expression of the function that a neural network computes?" Tags="&lt;neural-networks&gt;&lt;function-approximation&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12672" PostTypeId="2" ParentId="12671" CreationDate="2019-06-03T23:24:38.290" Score="2" Body="&lt;p&gt;To check if a function is linear is easy: if you can train one fully connected layer, without activations, of the right dimensions (for a function &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n \rightarrow \mathbb{R}^m$&lt;/span&gt; you need &lt;span class=&quot;math-container&quot;&gt;$nm$&lt;/span&gt; weights aka the matrix corresponding to the linear application), with enough data, to 100% accuracy... then it is linear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The estimated function is explicit: it is given by the architecture of the NN and its weights. I don't think you can hope for an answer like &quot;&lt;span class=&quot;math-container&quot;&gt;$\sin(\pi x + \phi) +4$&lt;/span&gt;&quot; if it's not already in the predictive capacity of the NN. Interpretability of NNs is a hot topic. In what cases symbolic reasoning can lead to a simplified expression?&lt;/p&gt;&#xA;" OwnerUserId="23224" LastActivityDate="2019-06-03T23:24:38.290" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12673" PostTypeId="1" AcceptedAnswerId="12677" CreationDate="2019-06-04T02:11:28.863" Score="1" ViewCount="58" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;A bank wants to decide whether a customer can be given a loan,&#xA;  based on two features related to (i) the monthly salary of the customer, and (ii) his/her&#xA;  account balance. For simplicity, we model the two features with two binary variables &lt;span class=&quot;math-container&quot;&gt;$X1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X2$&lt;/span&gt;&#xA;  and the class &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; (all of which can be either 0 or 1). &lt;span class=&quot;math-container&quot;&gt;$Y=1$&lt;/span&gt; indicates that the customer can be&#xA;  given loan, and Y=0 indicates otherwise.&#xA;  Consider the following dataset having four instances:&#xA;  (&lt;span class=&quot;math-container&quot;&gt;$X1 = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X2 = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Y = 0$&lt;/span&gt;)&#xA;  (&lt;span class=&quot;math-container&quot;&gt;$X1 = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X2 = 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Y = 0$&lt;/span&gt;)&#xA;  (&lt;span class=&quot;math-container&quot;&gt;$X1 = 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X2 = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Y = 0$&lt;/span&gt;)&#xA;  (&lt;span class=&quot;math-container&quot;&gt;$X1 = 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X2 = 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Y = 1$&lt;/span&gt;)&lt;br&gt;&#xA;  Can there be any logistic regression classifier using X1 and X2 as features, that can perfectly classify the given data?  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The approach followed in the question was to calculate respective probabilities for Y=0 and Y=1 respectively.  The value of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; obtained was &lt;span class=&quot;math-container&quot;&gt;$0.25$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(1-p)$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$0.75$&lt;/span&gt;.The &lt;span class=&quot;math-container&quot;&gt;$log(p/1-p)$&lt;/span&gt; is coming as negative. But I am stuck at this place as to which parameter to be calculated which decides that logistic regression would classify/not classify the data ?&lt;/p&gt;&#xA;" OwnerUserId="25984" LastEditorUserId="8068" LastEditDate="2019-06-04T02:29:32.437" LastActivityDate="2019-06-04T11:39:03.000" Title="Checking Whether Given Logistic Regression Classifier classifies data" Tags="&lt;machine-learning&gt;&lt;logistic-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12675" PostTypeId="2" ParentId="12659" CreationDate="2019-06-04T08:17:28.330" Score="2" Body="&lt;p&gt;Apologies for not writing in English. Please find the adapted translation here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give interested people an idea of the quality of MT (DeepL) please see this example from a text I was working on this morning (6,300 words, started at 9 am, delivery today around 1 pm and still find time for this post). I was working on this sentence (201 words) when I posted my comment.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;You further represent, warrant and undertake to ABC that you shall not: (a) Conduct any fraudulent, abusive, or otherwise illegal activity which may be grounds for termination of your right to access or use this Website and/or the Services; or (b) Post or transmit, or cause to be posted or transmitted, any communication or solicitation designed or intended to obtain password, account, or private information from any other user of this Website; or (c) Violate the security of any computer network, crack passwords or security encryption codes, transfer or store illegal material (including material that may be considered threatening or obscene), or engage in any kind of illegal activity that is expressly prohibited; or (d) Run maillist, listserv, or any other form of auto-responder, or &quot;spam&quot; on this Website, or any processes that run or are activated while you are not logged on to this Website, or that otherwise interfere with the proper working of or place an unreasonable load on this Website’s infrastructure; or (e) Use manual or automated software, devices, or other processes to &quot;crawl,&quot; &quot;scrape,&quot; or &quot;spider&quot; any page of this Website; or (f) Decompile, reverse engineer, or otherwise attempt to obtain the source code of this Website.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;DeepL returns this: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sie versichern, garantieren und verpflichten sich gegenüber ABC, dass Sie dies nicht tun werden: (a) betrügerische, missbräuchliche oder anderweitig illegale Aktivitäten durchzuführen, die Anlass für die Beendigung Ihres Rechts auf Zugang oder Nutzung dieser Website und/oder der Dienste sein können; oder (b) Mitteilungen oder Aufforderungen, die dazu bestimmt sind oder sind, Passwörter, Konten oder private Informationen von anderen Nutzern dieser Website zu erhalten, zu posten oder zu übertragen oder posten oder zu posten oder zu übertragen; oder (c) die Sicherheit eines Computernetzwerks verletzen, Passwörter oder Sicherheitsverschlüsselungscodes knacken, illegales Material (einschließlich Material, das als bedrohlich oder obszön angesehen werden kann) übertragen oder speichern oder sich an illegalen Aktivitäten beteiligen, die ausdrücklich verboten sind; oder (d) Mailliste, Listenserver oder jede andere Form von Auto-Responder oder &quot;Spam&quot; auf dieser Website oder Prozesse, die ausgeführt werden oder aktiviert werden, während Sie nicht auf dieser Website angemeldet sind, oder die anderweitig das ordnungsgemäße Funktionieren oder eine unangemessene Belastung der Infrastruktur dieser Website stören; oder (e) manuelle oder automatisierte Software, Geräte oder andere Prozesse verwenden, um eine Seite dieser Website zu &quot;crawlen&quot;, zu kratzen, zu spinnen oder zu spinnen; oder (f) dekompilieren, zurückzuentwickeln oder anderweitig zu versuchen, den Quellcode dieser Website zu erhalten.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It took me about 5 to 10 minutes to adjust this paragraph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a translator, I know that I cannot rely on the machine translation, but I learnt the specifics and capabilities of the different systems over time and I know what to pay attention for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;MT helps me a lot in my work.&lt;/p&gt;&#xA;" OwnerUserId="26142" LastEditorUserId="1671" LastEditDate="2019-06-05T19:35:29.447" LastActivityDate="2019-06-05T19:35:29.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12676" PostTypeId="2" ParentId="12659" CreationDate="2019-06-04T09:04:14.997" Score="4" Body="&lt;p&gt;It really depends on the language pair and the topic of the content. Translating to/from English to any other language usually is the best supported. Translating to and from popular languages works better, for example, translating from English to Romanian is a poorer translation than English to Russian. But translating from English to Russian or Romanian is better than translating Russian to Romanian. And translating Romanian to English is better than translating English to Romanian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you are used to working with translators and you have a passing familiarity with the languages, translation mistakes and the topic, it's easy to understand what was supposed to be there. And, at that point, sometimes its easier to read something translated into your native language for quick scanning than it is to read it in a second language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Less popular languages (for translation not necessarily in number of speakers) are much much closer to literal translations only slightly better than what you personally would do using a dictionary for two languages you do not know.&lt;/p&gt;&#xA;" OwnerUserId="26149" LastActivityDate="2019-06-04T09:04:14.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12677" PostTypeId="2" ParentId="12673" CreationDate="2019-06-04T11:39:03.000" Score="1" Body="&lt;p&gt;check it&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import keras&#xA;from keras.layers import *&#xA;&#xA;X = np.array([[0,0], [0,1], [1,0], [1,1]])&#xA;Y = np.array([[0], [0], [0], [1]])&#xA;&#xA;input = Input(shape=(2,))&#xA;output = Dense(1, activation='sigmoid')(input)&#xA;model = keras.Model(input, output)&#xA;&#xA;model.compile(keras.optimizers.Adam(1e0), 'binary_crossentropy', metrics=['acc'])&#xA;model.fit(X, Y, epochs=10, batch_size=4, verbose=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 1/10&#xA;4/4 [==============================] - 0s 52ms/step - loss: 0.7503 - acc: 0.7500&#xA;Epoch 2/10&#xA;4/4 [==============================] - 0s 817us/step - loss: 0.5142 - acc: 0.7500&#xA;Epoch 3/10&#xA;4/4 [==============================] - 0s 732us/step - loss: 0.4353 - acc: 0.7500&#xA;Epoch 4/10&#xA;4/4 [==============================] - 0s 694us/step - loss: 0.3413 - acc: 1.0000&#xA;Epoch 5/10&#xA;4/4 [==============================] - 0s 633us/step - loss: 0.2817 - acc: 1.0000&#xA;Epoch 6/10&#xA;4/4 [==============================] - 0s 679us/step - loss: 0.2299 - acc: 1.0000&#xA;Epoch 7/10&#xA;4/4 [==============================] - 0s 672us/step - loss: 0.1769 - acc: 1.0000&#xA;Epoch 8/10&#xA;4/4 [==============================] - 0s 721us/step - loss: 0.1412 - acc: 1.0000&#xA;Epoch 9/10&#xA;4/4 [==============================] - 0s 694us/step - loss: 0.1193 - acc: 1.0000&#xA;Epoch 10/10&#xA;4/4 [==============================] - 0s 716us/step - loss: 0.1015 - acc: 1.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...so yes, you can&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note you calculated marginal probabilities, here you want them conditioned on the input variables to actually solve for the parameters&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-04T11:39:03.000" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12678" PostTypeId="2" ParentId="12659" CreationDate="2019-06-04T12:29:58.280" Score="6" Body="&lt;p&gt;You have asked quite a lot of questions, some of which cannot be answered definitively . To give an insight of the &lt;strong&gt;quality&lt;/strong&gt; (and its history) of machine translations I like  to refer to Christopher Manning his 'one sentence benchmark' as presented in his &lt;a href=&quot;https://youtu.be/IxQtK2SjWWM?t=1665&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture&lt;/a&gt;. It contains one Chinese to English example which is compared with Google Translate output. The correct translation for the example would be:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In 1519, six hundred Spaniards landed in Mexico to conquer the Aztec Empire with a population of a few million. They lost two thirds of their soldiers in the first clash.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Google Translate returned the following translations.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;2009&lt;/strong&gt; 1519 600 Spaniards landed in Mexico, millions of people to conquer the Aztec empire, the first two-thirds of soldiers against their loss.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2011&lt;/strong&gt; 1519 600 Spaniards landed in Mexico, millions of people to conquer the Aztec empire, the initial loss of soldiers, two thirds of their encounters.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2013&lt;/strong&gt; 1519 600 Spaniards landed in Mexico to conquer the Aztec empire, hundreds of millions of people, the initial confrontation loss of soldiers two-thirds.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2015&lt;/strong&gt; 1519 600 Spaniards landed in Mexico, millions of people to conquer the Aztec empire, the first two-thirds of the loss of soldiers they clash.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2017&lt;/strong&gt; In 1519, 600 Spaniards landed in Mexico, to conquer the millions of people of the Aztec empire, the first confrontation they killed two-thirds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Whether Google  &lt;strong&gt;retains&lt;/strong&gt; or 'hides' their best results: I doubt it. There are many excellent researchers working in the field of natural language processing (NLP). If Google would have a 'greatest achievement' for translation, the researchers would figure it out sooner or later. (Why would Google hide their 'greatest achievement' anyway? They seem to see the benefit of open source, see the Transformer[1] or BERT[2])&lt;/p&gt;&#xA;&lt;p&gt;NB. For an updated list of state-of-the-art algorithms in NLP, see the&#xA;&lt;a href=&quot;https://rajpurkar.github.io/SQuAD-explorer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SQuAD2.0 leaderboard&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;[1] Vaswani, Ashish, et al. &amp;quot;Attention is all you need.&amp;quot; Advances in neural information processing systems. 2017.&lt;/p&gt;&#xA;&lt;p&gt;[2] Devlin, Jacob, et al. &amp;quot;Bert: Pre-training of deep bidirectional transformers for language understanding.&amp;quot; arXiv preprint arXiv:1810.04805 (2018).&lt;/p&gt;&#xA;" OwnerUserId="26155" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-06-06T07:35:25.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12679" PostTypeId="2" ParentId="12114" CreationDate="2019-06-04T12:53:46.097" Score="2" Body="&lt;h1&gt;It's currently just too complex&lt;/h1&gt;&#xA;&lt;p&gt;The different sources of information are too varied, in economics this is often referred to as a &lt;a href=&quot;http://www.fao.org/3/y5610e/y5610e01.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;local knowledge&lt;/a&gt; problem, which hampers many large scale plans. Humans can react to slight differences like respecting local traditions, landscapes, history but an artificial intelligence would (currently at least) struggle not to generalise over such a large scale as a whole country's economy.&lt;/p&gt;&#xA;&lt;p&gt;The real work in this case (and actually most 'AI' tasks) would be collecting all the necessary data. Here that part job is currently insurmountable.&lt;/p&gt;&#xA;&lt;p&gt;Currently a human lead planned economy would do a better job.&lt;/p&gt;&#xA;" OwnerUserId="22897" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-06-06T09:32:43.207" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12680" PostTypeId="2" ParentId="10562" CreationDate="2019-06-04T13:07:42.847" Score="1" Body="&lt;p&gt;I can't do a better job at explaining than Jeremy Howard's team did. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow &lt;strong&gt;carefully&lt;/strong&gt; and &lt;strong&gt;systematically&lt;/strong&gt; all the steps put together &lt;a href=&quot;https://course.fast.ai/start_gcp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about the different &lt;strong&gt;&lt;em&gt;types of machines&lt;/em&gt;&lt;/strong&gt; &lt;a href=&quot;https://cloud.google.com/compute/docs/machine-types&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, about the possible choices of &lt;strong&gt;&lt;em&gt;GPUs&lt;/em&gt;&lt;/strong&gt; &lt;a href=&quot;https://cloud.google.com/compute/docs/gpus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and about the &lt;strong&gt;&lt;em&gt;regions and zones&lt;/em&gt;&lt;/strong&gt; &lt;a href=&quot;https://cloud.google.com/compute/docs/regions-zones/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow the &lt;a href=&quot;https://www.fast.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastai&lt;/a&gt; course too.&#xA;I can't emphasize how good is it.&#xA;You can find the lessons in the left panel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend you use &lt;a href=&quot;https://colab.research.google.com/notebooks/welcome.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colab&lt;/a&gt; for experiments as it has a &lt;strong&gt;Tesla-K80&lt;/strong&gt; and the cloud computing for something that requires more power. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helped!&lt;/p&gt;&#xA;" OwnerUserId="26150" LastActivityDate="2019-06-04T13:07:42.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12681" PostTypeId="1" CreationDate="2019-06-04T13:12:25.830" Score="1" ViewCount="65" Body="&lt;p&gt;I am wondering if anyone has any experience dealing with vector states, that is, a state that has multiple values. I cannot find any coding examples online and I am trying to implement Q-learning. Any help would be great!&lt;/p&gt;&#xA;" OwnerUserId="26158" LastEditorUserId="2444" LastEditDate="2019-06-04T20:29:46.303" LastActivityDate="2019-06-04T20:29:46.303" Title="How should I deal with vector states in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-11-16T23:09:14.990" ContentLicense="CC BY-SA 4.0" />
  <row Id="12682" PostTypeId="1" CreationDate="2019-06-04T13:42:58.250" Score="3" ViewCount="87" Body="&lt;p&gt;I was reading the following book: &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://neuralnetworksanddeeplearning.com/chap2.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and towards the end of equation 29, there is a paragraph that explains this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KkBdn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KkBdn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I am unsure how the equation below is derived:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DlOfT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DlOfT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26159" LastActivityDate="2019-06-05T11:57:37.860" Title="How does adding a small change to an neuron's weighted input affect the overall cost?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="12683" PostTypeId="1" CreationDate="2019-06-04T14:19:24.770" Score="2" ViewCount="49" Body="&lt;p&gt;This is an Inverse Reinforcement Learning (IRL) problem. I have data (observations) on actions taken by a (real) agent. Given this data I want to estimate the likelihood of the observed actions in a Q-learning agent. Rewards are given by a linear function on a parameter, say alpha.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I want to estimate the alpha that makes the observed actions more likely to be taken by a Q-agent. I read some papers (i.e. Ng &amp;amp; Russel 2004), but I found them rather generalistic.&lt;/p&gt;&#xA;" OwnerUserId="26154" LastActivityDate="2019-06-04T14:19:24.770" Title="Inverse Reinforcement Learning for Markov Games" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12685" PostTypeId="2" ParentId="12682" CreationDate="2019-06-04T15:39:18.310" Score="0" Body="&lt;p&gt;I think that Nielsen just wanted to convey the idea of the back-propagation algorithm using that formula, as you can read from the next paragraph &quot;Now, this demon is a good demon...&quot;, so I don't think that that partial derivative is mathematically correct, provided the partial derivative is still with respect to &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the cost (or loss) function. &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt; is the linear output of neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, which is followed by a non-linear function (e.g. sigmoid), denoted by &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;. So, the actual output of neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\sigma(z_j^l)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The partial derivative of the cost function &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; with respect to this neuron's linear output, &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt;, is &lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial z_j^l} = \frac{\partial C}{\partial z_j^l} 1 = \frac{\partial C}{\partial z_j^l} \frac{\partial z_j^l}{\partial z_j^l}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the the linear output of node &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; is now &lt;span class=&quot;math-container&quot;&gt;$z_j^l + \Delta z_j^l$&lt;/span&gt;, then the partial derivative with respect to &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt; becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{\partial C}{\partial z_j^l} &#xA;&amp;amp;= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)}\frac{\partial (z_j^l + \Delta z_j^l)}{\partial z_j^l}  \\&#xA;&amp;amp;= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)} \left( \frac{\partial z_j^l}{\partial z_j^l} +  \frac{\partial \Delta z_j^l}{\partial z_j^l} \right) \\&#xA;&amp;amp;= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)} \left( 1 +  \frac{\partial \Delta z_j^l}{\partial z_j^l} \right) \\&#xA;&amp;amp;= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)} +  \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)}\frac{\partial \Delta z_j^l}{\partial z_j^l} \\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Delta z_j^l$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt;, but it is not specified how.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-04T15:39:18.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12686" PostTypeId="2" ParentId="12682" CreationDate="2019-06-04T16:01:08.137" Score="1" Body="&lt;p&gt;I believe he's just saying that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial C}{\partial z_j^l} \Delta z_j^l \approx \frac{\partial C}{\partial z_j^l} \partial z_j^l \approx \partial C&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that the change in cost function can be arrived at simply for a small enough perturbation &lt;span class=&quot;math-container&quot;&gt;$\Delta z_j^l$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, taking that line of approximations backwards, the change in the cost function for a given perturbation is just:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\partial C \approx \frac{\partial C}{\partial z_j^l} \partial z_j^l \approx  \frac{\partial C}{\partial z_j^l} \Delta z_j^l&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20639" LastActivityDate="2019-06-04T16:01:08.137" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12687" PostTypeId="2" ParentId="12682" CreationDate="2019-06-04T16:05:29.357" Score="0" Body="&lt;p&gt;The derivative of a function (&lt;span class=&quot;math-container&quot;&gt;$f(x_1,x_2..x_n)$&lt;/span&gt;) w.r.t to one of the variables (&lt;span class=&quot;math-container&quot;&gt;$x_1,x_2..x_n$&lt;/span&gt;) gives us the rate of change of the function w.r.t the rate of change of the variable. This roughly means that by how much will the function value change if we change the variable by a &quot;unit amount&quot; or &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;. (we cannot use the change as &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; as the change needs to be infinitesimally small, this is just a rough explanation) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image shows a tangent line to a curve or function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. The slope of this tangent is given by &lt;span class=&quot;math-container&quot;&gt;$\frac{df(x)}{dx}$&lt;/span&gt; at that particular &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. If you move by a very very small amount in the direction of positive &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; i.e. &lt;span class=&quot;math-container&quot;&gt;$x+\delta x$&lt;/span&gt; the change in the value of the &lt;span class=&quot;math-container&quot;&gt;$f(x)= y $&lt;/span&gt; will almost be the same as the change in the value of the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; of the tangent line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MDjXL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MDjXL.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as per the excerpt the cost function &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a function of &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt;. Thus, it can be written as &lt;span class=&quot;math-container&quot;&gt;$$C = f(z_j^l, .....).$$&lt;/span&gt; So, &lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial z_j^l}$$&lt;/span&gt; indicates how much &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; will vary w.r.t &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt;, i.e. when &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt; is changed by an infinitesimally small amount and thus the formulae:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial z_j^l} \Delta z_j^l$$&lt;/span&gt; where the author assumed &lt;span class=&quot;math-container&quot;&gt;$\Delta z_j^l$&lt;/span&gt; to be very small. This gives the infinitesimal change in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; or gives us &lt;span class=&quot;math-container&quot;&gt;$\Delta C$&lt;/span&gt;, for infinitesimally small change in &lt;span class=&quot;math-container&quot;&gt;$z_j^l$&lt;/span&gt; or any varible affecting the cost function. This can be derived by series expansions too (given below), but this is an intuitive explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An explanation can be given from &lt;strong&gt;&lt;a href=&quot;https://users.math.msu.edu/users/magyar/Math133/11.10-Taylor-Series.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taylor Series Theorem&lt;/a&gt;&lt;/strong&gt; which states: :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; be a function which is analytic at &lt;span class=&quot;math-container&quot;&gt;$x = a$&lt;/span&gt;. Then we can write&#xA;  &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; as the following power series, called the Taylor series of &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;&#xA;  at &lt;span class=&quot;math-container&quot;&gt;$x = a$&lt;/span&gt;, then we can write &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; as:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = f(a) + f'(a)(x-a) + f''(a)\frac{(x-a)^2}{2!} + f'''(a)\frac{(x-a)^3}{3!}... $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if we keep other variables constant and make cost function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; vary only with &lt;span class=&quot;math-container&quot;&gt;$z^l_j$&lt;/span&gt;&#xA;and if we put &lt;span class=&quot;math-container&quot;&gt;$a=z^l_j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x=z^l_j + \Delta z^l_j$&lt;/span&gt; the equation becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(z^l_j + \Delta z^l_j) = f(z^l_j) + f'(z^l_j)(\Delta z^l_j) + f''(z^l_j)\frac{(\Delta z^l_j)^2}{2!} + f'''(a)\frac{(\Delta z^l_j)^3}{3!}... $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which if we ignore the higher order terms of  &lt;span class=&quot;math-container&quot;&gt;$\Delta z^l_j$&lt;/span&gt;, since terms containing &lt;span class=&quot;math-container&quot;&gt;$\Delta z^l_j$&lt;/span&gt; for powers greater than 1 will be negligible compared to &lt;span class=&quot;math-container&quot;&gt;$\Delta z^l_j$&lt;/span&gt; with power 1. Thus the equation now effectively is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(z^l_j + \Delta z^l_j) = f(z^l_j) + f'(z^l_j)(\Delta z^l_j)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(z^l_j + \Delta z^l_j) - f(z^l_j)=   f'(z^l_j)(\Delta z^l_j)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(z^l_j + \Delta z^l_j) - f(z^l_j)=   \frac{\partial f(z^l_j)}{\partial z^l_j}(\Delta z^l_j)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$$f(z^l_j + \Delta z^l_j) - f(z^l_j)$$&lt;/span&gt; can be thought of as &lt;span class=&quot;math-container&quot;&gt;$\Delta C$&lt;/span&gt; or the change in cost function for small change in &lt;span class=&quot;math-container&quot;&gt;$z^l_j$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I have glossed over some requirements for a Taylor Series to be convergent.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-06-05T11:57:37.860" LastActivityDate="2019-06-05T11:57:37.860" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="12688" PostTypeId="2" ParentId="12600" CreationDate="2019-06-04T17:11:34.547" Score="1" Body="&lt;p&gt;As @Clement mentions, &lt;a href=&quot;https://becominghuman.ai/generative-adversarial-networks-for-text-generation-part-1-2b886c8cab10&quot; rel=&quot;nofollow noreferrer&quot;&gt;text_gen_description&lt;/a&gt; gives a good overview!, but the paper &lt;a href=&quot;https://arxiv.org/abs/1609.05473&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqGAN paper&lt;/a&gt; describes the REINFORCE approach more in depth, as they are the first to do it (i believe). This is probably the approach most take now of days when going the GAN route.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that just basic MLE training has shown promise with openAI's GPT2. When i need a text generator, fine tuning one of the provided models is usually my goto.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if your looking for seq gans code base (you asked for example code) here is is: &lt;a href=&quot;https://github.com/LantaoYu/SeqGAN&quot; rel=&quot;nofollow noreferrer&quot;&gt;git repo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good Luck!&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-04T17:11:34.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12689" PostTypeId="1" CreationDate="2019-06-04T18:51:34.267" Score="3" ViewCount="90" Body="&lt;p&gt;I need to manually classify thousands of pictures into discrete categories, say, where each picture is to be tagged either A, B, or C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I want to do this work myself, not outsource / crowdsource / whatever online collaborative distributed shenanigans. Also, I'm currently not interested in active learning. Finally, I don't need to label features inside the images (eg. &lt;a href=&quot;https://cvhci.anthropomatik.kit.edu/~baeuml/projects/a-universal-labeling-tool-for-computer-vision-sloth/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sloth&lt;/a&gt;) just file each image as either A, B, or C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I need a tool that will show me a picture, wait for me to press a single key (0 to 9 or A to Z), save the classification (filename + chosen character) in a simple CSV file in the same directory as the pictures, and show the next picture. Maybe also showing a progress bar for the entire work and ETA estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I go ahead and code it myself, is there anything like this already available?&lt;/p&gt;&#xA;" OwnerUserId="26169" LastEditorUserId="26169" LastEditDate="2019-06-04T21:16:02.863" LastActivityDate="2019-06-05T02:38:31.363" Title="Are there tools to help labelling images?" Tags="&lt;classification&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ClosedDate="2020-11-06T19:22:49.493" ContentLicense="CC BY-SA 4.0" />
  <row Id="12690" PostTypeId="1" AcceptedAnswerId="12694" CreationDate="2019-06-04T19:53:37.460" Score="1" ViewCount="96" Body="&lt;p&gt;I'm new to NEAT, so, please, don't be too harsh. How does NEAT find the most successful generation without gradient descent or gradients?&lt;/p&gt;&#xA;" OwnerUserId="22802" LastEditorUserId="2444" LastEditDate="2019-06-05T21:48:52.127" LastActivityDate="2019-06-05T21:48:52.127" Title="How does NEAT find the most successful generation without gradients?" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12691" PostTypeId="2" ParentId="12689" CreationDate="2019-06-04T20:50:27.220" Score="3" Body="&lt;p&gt;There are a few tools that you can use to annotate (or label) data. For example, &lt;a href=&quot;https://github.com/wkentaro/labelme&quot; rel=&quot;nofollow noreferrer&quot;&gt;labelme&lt;/a&gt; or &lt;a href=&quot;https://labelbox.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Labelbox&lt;/a&gt;. Have a look at &lt;a href=&quot;https://qr.ae/TWGcG1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this question&lt;/a&gt; for more alternatives.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-04T22:26:06.827" LastActivityDate="2019-06-04T22:26:06.827" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12692" PostTypeId="5" CreationDate="2019-06-04T21:03:34.317" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-06-04T21:03:34.317" LastActivityDate="2019-06-04T21:03:34.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12693" PostTypeId="4" CreationDate="2019-06-04T21:03:34.317" Score="0" Body="For questions related to the convergence of AI algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-05T19:34:38.067" LastActivityDate="2019-06-05T19:34:38.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12694" PostTypeId="2" ParentId="12690" CreationDate="2019-06-04T23:13:45.340" Score="3" Body="&lt;p&gt;NEAT is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithm&lt;/a&gt; (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like &lt;a href=&quot;https://ai.stackexchange.com/a/9079/2444&quot;&gt;the crossover or the mutation&lt;/a&gt;, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are neural networks. However, these individuals need first to be encoded into a (compressed) reprentation (for example, a vector) that allows the operations like mutation to be efficiently applied: this representation is often called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Genotype%E2%80%93phenotype_distinction&quot; rel=&quot;nofollow noreferrer&quot;&gt;genotype&lt;/a&gt; (or chromosome).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you decide which individuals are the &lt;a href=&quot;https://en.wikipedia.org/wiki/Survival_of_the_fittest&quot; rel=&quot;nofollow noreferrer&quot;&gt;fittest&lt;/a&gt;? A function called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;fitness function&lt;/a&gt; needs first to be defined. The fitness function measures the fitness (or quality) of the individuals (or solutions). In the case of neural networks, a fitness function could, for example, be the accuracy of the neural networks on a validation dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why don't GAs need gradients? The fitness (or quality) of the solutions is given by the fitness function in the case of GAs, so gradients are not strictly needed, even though they can also be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEAT is a little bit more complex (and it is described in detail in the paper that introduced it &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolving Neural Networks through Augmenting Topologies&lt;/a&gt;), but this is the basic idea behind all genetic algorithms (including NEAT).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-05T21:45:32.850" LastActivityDate="2019-06-05T21:45:32.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12695" PostTypeId="2" ParentId="12689" CreationDate="2019-06-05T02:38:31.363" Score="3" Body="&lt;p&gt;I do not know a specific tool that meets all the mentioned requirements. However, a long time ago, I had to do a very similar task of labeling tons of images into 10 classes. This is how I did this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Used a very basic clustering tool to cluster images into clusters (I set the number of clusters larger than 10 as I new some classes have very different subclasses inside).&lt;/li&gt;&#xA;&lt;li&gt;Moved all images of each cluster to a separate folder. Named each folder after one of the classes/subclasses.&lt;/li&gt;&#xA;&lt;li&gt;Double checked the folders content to make sure there is no outlier or mismatched sample. In the case of wrong labels, I just moved them to the correct folder.&lt;/li&gt;&#xA;&lt;li&gt;Merged subclasses to form the 10 classes that I was interested in.&lt;/li&gt;&#xA;&lt;li&gt;Done!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12853" LastActivityDate="2019-06-05T02:38:31.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12696" PostTypeId="2" ParentId="12659" CreationDate="2019-06-05T03:49:44.677" Score="1" Body="&lt;p&gt;This will be not so much an answer as a commentary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quality depends on several things, including (as Aaron said above) 1) the language pair and 2) the topic, but also 3) the genera and 4) the style of the original, and 5) the amount of parallel text you have to train the MT system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To set the stage, virtually all MT these days is based off of parallel texts, that is a text in two different languages, with one presumably being a translation of the other (or both being a translation of some third language); and potentially using dictionaries (perhaps assisted by morphological processes) as backoff when the parallel texts don't contain particular words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, as others have said, an MT system in no way understands the texts it's translating; it just sees strings of characters, and sequences of words made up of characters, and it looks for similar strings and sequences in texts it's translated before.  (Ok, it's slightly more complicated than that, and there have been attempts to get at semantics in computational systems, but for now it's mostly strings.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Languages vary.  Some languages have lots of morphology, which means they do things with a single word that other languages do with several words.  A simple example would be Spanish 'cantaremos' = English &quot;we will sing&quot;.  And one language may do things that the other language doesn't even bother with, like the informal/formal (tu/ usted) distinction in Spanish, which English doesn't have an equivalent to.  Or one language may do things with morphology that another language does with word order.  Or the script that the language uses may not even mark word boundaries (Chinese, and a few others).  The more different the two languages, the harder it will be for the MT system to translate between them.  The first experiments in statistical MT were done between French and English, which are (believe it or not) very similar languages, particularly in their syntax.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Topic: If you have parallel texts in the Bible (which is true for nearly any pair of written languages), and you train your MT system off of those, don't expect it to do well on engineering texts.  (Well, the Bible is a relatively small amount of text by the standards of training MT systems anyway, but pretend :-).)  The vocabulary of the Bible is very different from that of engineering texts, and so is the frequency of various grammatical constructions.  (The grammar is essentially the same, but in English, for example, you get lots more passive voice and more compound nouns in scientific and engineering texts.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Genera: If your parallel text is all declarative (like tractor manuals, say), trying to use the resulting MT system on dialog won't get you good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Style: Think Hilary vs. Donald; erudite vs. popular.  Training on one won't get good results on the other.  Likewise training the MT system on adult-level novels and using it on children's books.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) Language pair: English has lots of texts, and the chances of finding texts in some other language which are parallel to a given English text are much higher than the chances of finding parallel texts in, say, Russian and Igbo.  (That said, there may be exceptions, like languages of India.)  As a gross generalization, the more such parallel texts you have to train the MT system, the better results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In sum, language is complicated (which is why I love it--I'm a linguist).  So it's no surprise that MT systems don't always work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, human translators don't always do so well, either.  A decade or two ago, I was getting translations of documents from human translators into English, to be used as training materials for MT systems.  Some of the translations were hard to understand, and in some cases where we got translations from two (or more) human translators, it was hard to believe the translators had been reading the same documents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, there's (almost) never just one correct translation; there are multiple ways of translating a passage, which may be more or less good, depending on what features (grammatical correctness, style, consistency of usage,...) you want.  There's no easy measure of &quot;accuracy&quot;.&lt;/p&gt;&#xA;" OwnerUserId="26180" LastActivityDate="2019-06-05T03:49:44.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12697" PostTypeId="1" CreationDate="2019-06-05T04:15:50.713" Score="2" ViewCount="99" Body="&lt;p&gt;I have recently begun researching LSTM networks, as I have finished my GA and am looking to progress to something more difficult. I believe I am using the classic LSTM (if that makes any sense) and have a few questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need LSTM units everywhere in the network? For example, can I only use LSTM units for the first and last layer and use feedforward units everywhere else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I go about implementing bias values into an LSTM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I create a network that predicts the next few words of a sentence, does that mean my outputs should be every possible word that the network could conceivably use?&lt;/p&gt;&#xA;" OwnerUserId="26181" LastEditorUserId="2444" LastEditDate="2019-11-03T03:07:02.030" LastActivityDate="2020-04-01T04:07:10.870" Title="Do I need LSTM units everywhere in the network?" Tags="&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12698" PostTypeId="2" ParentId="12667" CreationDate="2019-06-05T04:50:54.473" Score="3" Body="&lt;p&gt;An intuitive NN playground can be found in &lt;a href=&quot;https://playground.tensorflow.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow Playground&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, check the Google ML crash course for coders as they promised to add more &lt;a href=&quot;https://developers.google.com/machine-learning/practica/&quot; rel=&quot;nofollow noreferrer&quot;&gt;practicals&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12853" LastEditorUserId="12853" LastEditDate="2019-06-19T23:12:50.630" LastActivityDate="2019-06-19T23:12:50.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12699" PostTypeId="1" CreationDate="2019-06-05T06:05:35.527" Score="1" ViewCount="35" Body="&lt;p&gt;I am writing a NN in pytorch and I want to add the derivative of the output with respect to one of the inner layers in the loss. Here is a simple example of what I mean:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class NN(torch.nn.Module):&#xA;&#xA;    def __init__(self):&#xA;        super(Model, self).__init__()&#xA;        self.l1 = torch.nn.Linear(2, 3)&#xA;        self.l2 = torch.nn.Linear(3, 1)&#xA;        self.l3 = torch.nn.Linear(1, 3)&#xA;        self.l4 = torch.nn.Linear(3, 2)&#xA;&#xA;    def forward(self, x):&#xA;        z = torch.sigmoid(self.l1(x))&#xA;        z = torch.sigmoid(self.l2(z))&#xA;        y_pred = torch.sigmoid(self.l3(z))&#xA;        y_pred = self.l4(y_pred)&#xA;        return y_pred&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case (for just one example) I want the loss to be defined as: &lt;code&gt;loss = (y-y_pred)**2+norm(d(y_pred)/dz)&lt;/code&gt; (the last part is not correct from a python point of view). So what I want is to add the derivative of of the output with respect to a given layer to the loss function. I tried some things with hook, but that is not really what I need here. Can someone help me? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="23871" LastActivityDate="2019-06-05T06:05:35.527" Title="Add a layer derivative in the loss function" Tags="&lt;loss-functions&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12700" PostTypeId="2" ParentId="12671" CreationDate="2019-06-05T11:29:00.503" Score="1" Body="&lt;p&gt;The network is the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A network is a function, that is modeled by terms describing the architecture and coefficients that are learned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look at a simple model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = ax+b$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your solver determines &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, and you substitute them into &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and then you're able to calculate &lt;span class=&quot;math-container&quot;&gt;$f(42)$&lt;/span&gt;. The function is linear by definition, but may not be a good fit for your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see if the input data may belong to a linear function can be achieved, when you have a network, that can both fit linear functions and higher order functions. For example try to fit your data to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x)=ax^2+bx+c$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is linear, when &lt;span class=&quot;math-container&quot;&gt;$a=0$&lt;/span&gt;. If the function is quadratic you get &lt;span class=&quot;math-container&quot;&gt;$a\ne 0$&lt;/span&gt; and if it has a higher order than &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; you will get not good fit with any &lt;span class=&quot;math-container&quot;&gt;$a, b, c$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When looking at MLP and similar networks, the question &quot;is it linear&quot; may not be that easy to answer, but the question how to get the function is the same: Your trained network is the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess one option to see if the function approximated by an existing network is linear is to train a second network that only contains linear elements. If it is able to approximate your first network, the first network is linear as well. Of course you will need the right test data, as you may accidentally choose a training set that has linearly correlated output in a non-linear function. This is of course the same problem as always, that your network will always can only be as good as the training data.&lt;/p&gt;&#xA;" OwnerUserId="25798" LastEditorUserId="25798" LastEditDate="2019-06-07T08:13:15.883" LastActivityDate="2019-06-07T08:13:15.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12701" PostTypeId="2" ParentId="12697" CreationDate="2019-06-05T11:44:05.227" Score="1" Body="&lt;p&gt;For question 1) I dont understand what you are getting at... LSTM cells will work on a contiguous block of inputs, where it sequentially uses the states from a previous time step and the new input to generate the next ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;question 2) Please look into the LSTM atchitecture &lt;a href=&quot;https://i.stack.imgur.com/BA9gg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BA9gg.png&quot; alt=&quot;LSTM pic&quot;&gt;&lt;/a&gt;&#xA;As you see, biases are already there, is there somewhere specific you want it, that it isnt?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;question 3) Generally yes, but the normalization step can be expensive (such as softmax), so if you want to get clever, you can use negative sampling or hierarchial softmax-- but generally, you you predict a probability over all possible words given the previous text&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-05T11:44:05.227" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12702" PostTypeId="2" ParentId="4683" CreationDate="2019-06-05T14:08:11.323" Score="3" Body="&lt;p&gt;&lt;strong&gt;CNN vs RNN&lt;/strong&gt; &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A CNN will learn to recognize patterns across space while RNN is useful for solving temporal data problems.&lt;/li&gt;&#xA;&lt;li&gt;CNNs have become the go-to method for solving any image data challenge while RNN is used for ideal for text and speech analysis.&lt;/li&gt;&#xA;&lt;li&gt;In a very general way, a CNN will learn to recognize components of an image (e.g., lines, curves, etc.) and then learn to combine these components to recognize larger structures (e.g., faces, objects, etc.) while an RNN will similarly learn to recognize patterns across time. So a RNN that is trained to convert speech to text should learn first the low level features like characters, then higher level features like phonemes and then word detection in audio clip.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;&lt;strong&gt;CNN&lt;/strong&gt; &lt;br&gt;&#xA;A  convolutional network (ConvNet) is made up of layers.&#xA;In a convolutional network (ConvNet), there are basically three types of layers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Convolution layer&lt;/li&gt;&#xA;&lt;li&gt;Pooling layer&lt;/li&gt;&#xA;&lt;li&gt;Fully connected layer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of these, the convolution layer applies convolution operation on the input 3D tensor. Different filters extract different kinds of features from an image. The below GIF illustrates this point really well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/KpvMabX.gif&quot; alt=&quot;Convolutional Filter in ConvNets&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the filter is the green 3x3 matrix while the image is the blue 7x7 matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many such layers passes through filters in CNN to give an output layer that can again be a NN Fully connected layer or a 3D tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/InqKjHl.jpg&quot; alt=&quot;CNN&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the above example, the input image passes through convolutional layer, then pooling layer, then convolutional layer, pooling layer, then the 3D tensor is flattened like a Neural Network 1D layer, then passed to a fully connected layer and finally a softmax layer. This makes a CNN.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;RNN&lt;/strong&gt; &lt;br&gt;&#xA;Recurrent Neural Network(RNN) are a type of Neural Network where the output from previous step are fed as input to the current step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/lJgJB4A.png&quot; alt=&quot;RNN model&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$x_{t-1}$&lt;/span&gt;, , &lt;span class=&quot;math-container&quot;&gt;$x_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt;  are the values of inputs data that occur at a specific time steps and are fed into the RNN that goes through the hidden layers namely &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt;, , &lt;span class=&quot;math-container&quot;&gt;$h_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{t+1}$&lt;/span&gt; which further produces output &lt;span class=&quot;math-container&quot;&gt;$o_{t-1}$&lt;/span&gt;, , &lt;span class=&quot;math-container&quot;&gt;$o_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$o_{t+1}$&lt;/span&gt; respectively. &lt;/p&gt;&#xA;" OwnerUserId="6604" LastActivityDate="2019-06-05T14:08:11.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12703" PostTypeId="2" ParentId="12659" CreationDate="2019-06-05T15:52:20.773" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Or does Google have reasons to retain its achievements (and not to show to the users the best they can show)&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If they were, then what they're holding back would be &lt;em&gt;amazing&lt;/em&gt;. Google publishes &lt;a href=&quot;https://ai.google/research/pubs/?area=NaturalLanguageProcessing&quot; rel=&quot;nofollow noreferrer&quot;&gt;a lot&lt;/a&gt; of strong papers in Natural Language Processing, including ones that get &lt;a href=&quot;https://arxiv.org/abs/1602.02410&quot; rel=&quot;nofollow noreferrer&quot;&gt;state of the art results&lt;/a&gt; or make &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;significant&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/abs/1409.3215&quot; rel=&quot;nofollow noreferrer&quot;&gt;conceptual&lt;/a&gt; &lt;a href=&quot;https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;breakthroughs&lt;/a&gt;.&#xA;They have also released very useful &lt;a href=&quot;https://catalog.ldc.upenn.edu/LDC2006T13&quot; rel=&quot;nofollow noreferrer&quot;&gt;datasets&lt;/a&gt; and &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tools&lt;/a&gt;. Google is one of the few companies out there that is not only using the cutting edge of current research, but is actively contributing to the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine translation is just a hard problem. A good human translator needs to be &lt;em&gt;fluent&lt;/em&gt; in both languages to do the job well. Each language will have its own idioms and non-literal or context-dependent meanings. Just working from a dual-language dictionary would yield terrible results (for a human or computer), so we need to train our models on existing corpora that exist in multiple languages in order to learn how words are actually used (n.b. hand-compiled phrase translation tables can be used as &lt;em&gt;features&lt;/em&gt;; they just can't be the whole story). For some language pairs, parallel corpora are plentiful (e.g. for EU languages, we have the &lt;a href=&quot;https://www.statmt.org/europarl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;complete proceedings of the European Parliament&lt;/a&gt;). For other pairs, training data is much sparser.  And even if we have training data, there will exist lesser used words and phrases that don't appear often enough to be learned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This used to be an even bigger problem, since synonyms were hard to account for. If our training data had sentences for &quot;The dog caught the ball&quot;, but not &quot;The puppy caught the ball&quot;, we would end up with a low probability for the second sentence. Indeed, significant smoothing would be needed to prevent the probability from being &lt;em&gt;zero&lt;/em&gt; in many such cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The emergence of neural language models in the last 15 years or so has &lt;em&gt;massively&lt;/em&gt; helped with this problem, by allowing words to be mapped to a real-valued semantic space &lt;em&gt;before&lt;/em&gt; learning the connections between words. This allows models to be learned in which words that are close together in meaning are also close together in the semantic space, and thus switching a word for its synonym will not greatly affect the probability of the containing sentence.  &lt;a href=&quot;https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt; is a model that illustrated this very well; it showed that you could, e.g., take the semantic vector for &quot;king&quot;, subtract the vector for &quot;man&quot;, add the vector for &quot;woman&quot;, and find that the nearest word to the resulting vector was &quot;queen&quot;. Once the research in neural language models began in earnest, we started seeing immediate and &lt;em&gt;massive&lt;/em&gt; drops in perplexity (i.e. how confused the models were by natural text) and we're seeing corresponding increases in BLEU score (i.e. quality of translation) now that those language models are being integrated into machine translation systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine translations are still not as good as quality human translations, and quite possibly won't be &lt;em&gt;that&lt;/em&gt; good until we crack fully sapient AI. But good human translators are expensive, while everyone with Internet access has machine translators available. The question isn't whether the human translation is better, but rather how close the machine gets to that level of quality. That gap has been shrinking and is continuing to shrink.&lt;/p&gt;&#xA;" OwnerUserId="2212" LastEditorUserId="2212" LastEditDate="2019-06-05T15:57:55.090" LastActivityDate="2019-06-05T15:57:55.090" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12704" PostTypeId="2" ParentId="10196" CreationDate="2019-06-05T16:04:29.697" Score="3" Body="&lt;p&gt;This question is discussed in detail, in the following NeurIPS 2016 paper by David Silver: &lt;a href=&quot;https://arxiv.org/pdf/1602.07714.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning values across many orders of magnitude&lt;/a&gt;. They also give experimental results over the Atari domain.&lt;/p&gt;&#xA;" OwnerUserId="26209" LastActivityDate="2019-06-05T16:04:29.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12705" PostTypeId="1" CreationDate="2019-06-05T18:45:49.700" Score="6" ViewCount="153" Body="&lt;p&gt;There was a recent informal question on chat about RTS games suitable for AI benchmarks, and I thought it would be useful to ask a question about them in relation to AI research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compact is defined as the fewest mechanics, elements, and smallest gameboard that produces a balanced, intractable, strategic game. &lt;em&gt;(This is important because greater compactness facilitates mathematical analysis.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-05T18:50:57.133" LastActivityDate="2019-06-10T02:45:44.283" Title="What are the most compact Real Time-Strategy Games?" Tags="&lt;game-ai&gt;&lt;rts&gt;&lt;benchmarks&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12706" PostTypeId="5" CreationDate="2019-06-05T18:46:44.307" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Real-time_strategy&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Real-time_strategy&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-05T18:46:44.307" LastActivityDate="2019-06-05T18:46:44.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12707" PostTypeId="4" CreationDate="2019-06-05T18:46:44.307" Score="0" Body="For questions about Real-time Strategy games, often used in AI research." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-05T18:46:44.307" LastActivityDate="2019-06-05T18:46:44.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12708" PostTypeId="5" CreationDate="2019-06-05T18:50:08.137" Score="0" Body="&lt;p&gt;Distinct from &quot;AI Milestones&quot; in that milestones can refer to theories, where benchmarks refer to verified results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Benchmarking&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Benchmarking&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-05T18:50:08.137" LastActivityDate="2019-06-05T18:50:08.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12709" PostTypeId="4" CreationDate="2019-06-05T18:50:08.137" Score="0" Body="For questions related to AI benchmarks--results that validate a specific technique or approach.  Also for question regarding the history of AI achievements, and predictions as to future achievements." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-05T18:50:08.137" LastActivityDate="2019-06-05T18:50:08.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12712" PostTypeId="1" CreationDate="2019-06-06T00:13:06.053" Score="4" ViewCount="340" Body="&lt;p&gt;As far I know, the RNN accepts a sequence as input and can produce as a sequence as output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there neural networks that accept graphs or trees as inputs, so that to represent the relationships between the nodes of the graph or tree?&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="23503" LastEditDate="2020-04-25T18:45:14.760" LastActivityDate="2020-05-26T19:44:30.120" Title="Are there neural networks that accept graphs or trees as inputs?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12714" PostTypeId="2" ParentId="12712" CreationDate="2019-06-06T06:20:05.690" Score="1" Body="&lt;p&gt;There are types of neural networks designed exactly for that purpose. For example, &lt;a href=&quot;https://arxiv.org/pdf/1609.02907.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;graph convolutional networks&lt;/em&gt;&lt;/a&gt; (GCN) by Thomas N. Kipf. The input to the network will be a matrix of size &lt;span class=&quot;math-container&quot;&gt;$N \times F$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of nodes and &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; the number of features (for each node). You then can multiply the feature matrix with the adjacency matrix (each node is going to be a weighted sum of its first-degree neighbors). There are a lot of other variations, such as &lt;a href=&quot;https://arxiv.org/pdf/1511.02136.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;diffusion convolutional networks&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1511.05493.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;gated graph neural networks&lt;/em&gt;&lt;/a&gt;, etc. There is a nice survey that describes most of the recent related work in the field &lt;a href=&quot;https://arxiv.org/pdf/1812.08434.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Neural Networks: A review of methods and applications&lt;/a&gt; by Jie Zhou et al.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2019-07-06T12:42:08.420" LastActivityDate="2019-07-06T12:42:08.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12716" PostTypeId="2" ParentId="12565" CreationDate="2019-06-06T08:16:14.677" Score="1" Body="&lt;p&gt;An Auto-Encoder is probably what you are looking for. AE is a very powerful Neural Network when you want to compress data and get a lower dimensional representation of the data with maximum information retained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested to know how it works -- Think of it as training a NN to predict the data itself. Which means your input and output layers would exactly be the same. So how does this help in compression ? The Hidden Layer -- Here is where the magic happens. The compression comes from the fact that we use lesser number of neurons in the hidden layer than in the i/p layer. Assuming that the NN we build nicely predicts the i/p at the end of the training,  the output of the hidden layers can be thought of as newly engineered features which are less in dimension yet powerful enough to represent the information in original data. &lt;/p&gt;&#xA;" OwnerUserId="26115" LastActivityDate="2019-06-06T08:16:14.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12718" PostTypeId="1" CreationDate="2019-06-06T10:55:03.907" Score="0" ViewCount="79" Body="&lt;p&gt;I'm building a CNN decoder, which mirrors (in reverse) the VGG network structure from Conv-4-1 layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The net seems to be working fine, however, the output looks broken. Please note that the colour distortion is fine, it's the the [255/0 RGB pixels] e.g. green that I'm worrying about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to overfit a single image, but even then I get these hot pixels. Does anyone know why they appear?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/akPst.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/akPst.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My net:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    activation = 'elu'&#xA;&#xA;    input_ = Input((None, None, 512))&#xA;    x = Conv2D(filters=256, kernel_size=self.kernel_size, padding='same', bias_initializer='zeros', activation=activation)(input_)&#xA;&#xA;    x = UpSampling2D()(x)&#xA;    for _ in range(3):&#xA;        x = Conv2D(filters=256, kernel_size=self.kernel_size, padding='same', activation=activation)(x)&#xA;    x = Conv2D(filters=128, kernel_size=self.kernel_size, padding='same', activation=activation)(x)&#xA;&#xA;    x = UpSampling2D()(x)&#xA;    x = Conv2D(filters=128, kernel_size=self.kernel_size, padding='same', activation=activation)(x)&#xA;    x = Conv2D(filters=64, kernel_size=self.kernel_size, padding='same', activation=activation)(x)&#xA;&#xA;    x = UpSampling2D()(x)&#xA;    x = Conv2D(filters=64, kernel_size=self.kernel_size, padding='same', activation=activation)(x)&#xA;    x = Conv2D(filters=3, kernel_size=self.kernel_size, padding='same')(x)&#xA;&#xA;    model = Model(inputs=input_, outputs=x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13149" LastActivityDate="2019-06-06T13:27:37.757" Title="What is wrong with this CNN network, why are there hot pixels?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12719" PostTypeId="2" ParentId="12565" CreationDate="2019-06-06T11:01:52.903" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/questions/4245/what-is-the-difference-between-encoders-and-auto-encoders/11808&quot;&gt;What is the difference between encoders and auto-encoders?&lt;/a&gt; For a working example of a neural network being used to compress (and decompress) images.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-06-06T11:01:52.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12720" PostTypeId="1" CreationDate="2019-06-06T12:42:33.930" Score="0" ViewCount="30" Body="&lt;p&gt;I have built various &quot;successful&quot; GANs or VAEs that can generate realistic images reliably, but in either case the generative step is sampling a latent feature vector from some distribution and running it through a generator/decoder &lt;span class=&quot;math-container&quot;&gt;$G(x) \  s.t. \  x \sim \mathbb{D} $&lt;/span&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt; has a continuous domain (normally distributed is a common choice), and &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt; is a continuous and at-least singly differential function (by construction so it could be optimized with a gradient scheme).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now assume we have 2 images &lt;span class=&quot;math-container&quot;&gt;$y_1, y_2$&lt;/span&gt; generated by &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2 \sim \mathbb{D}$&lt;/span&gt; and that shortest path from &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; is also in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt;'s domain. Crawling along this path, we get a continuous path from &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt; in image space. Wouldnt you assume along this continuous transformation that the images would be a &quot;Garbage&quot; as they are essentially a fusion of &lt;span class=&quot;math-container&quot;&gt;$y_1, y_2,$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\{y_k\}_k$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\{y_k\}_k$&lt;/span&gt; describes a set of &quot;succesful&quot; stops along the way? &#xA; --Especially in cases of single mode distributions (like normal) where all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; on the path must have higher likelihood than atleast one of the paths endpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to summarize my point, how do continuous generative models not with high probability produce garbage when given any 2 &quot;succesful&quot; images, there probably is a high range of &quot;garbage&quot; examples along a path between them in image space?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: by &quot;succesful&quot; i mean the generated image could be considered a draw from the true distribution you are trying to capture by the generator, and by &quot;garbage&quot; i mean ones that are obviously not.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-06T12:42:33.930" Title="Deep Generative Networks Probability of &quot;Success&quot;" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12721" PostTypeId="2" ParentId="12718" CreationDate="2019-06-06T13:27:37.757" Score="3" Body="&lt;p&gt;I've seen this too many times - it's not a problem with your network, it's a problem with matplotlib and how it displays the image. You are probably trying to display a float with range &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;0, 255&amp;gt;$&lt;/span&gt;. When matplotlib sees &lt;code&gt;float&lt;/code&gt; type as input, it assumes a range of &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;0, 1&amp;gt;$&lt;/span&gt;, and thresholds everything outside of that range, and the results you can see.&lt;/p&gt;&#xA;" OwnerUserId="26248" LastActivityDate="2019-06-06T13:27:37.757" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12723" PostTypeId="1" CreationDate="2019-06-06T15:43:33.600" Score="1" ViewCount="32" Body="&lt;p&gt;Suppose I have a set of data that I want to apply a segmented regression to, fitting linearly across the breakpoint. I aim to find the offsets and slopes of either line and the position of the breakpoint that minimizes an error function given the data I have, and then use them as sufficiently close initial guesses to find the exact solutions using a curve fit. I'll elect to choose the bounds as the mins and maxes of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; of my data, and an arbitrary bound for a slope with &lt;span class=&quot;math-container&quot;&gt;$slope = a * \frac{y_{max}-y_{min}}{x_{max}-x_{min}}$&lt;/span&gt; for a suitable &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; where I can safely assume the magnitude of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is greater than any possible slope that realistically represents the data. Let's suppose I define a function (in Python):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; def generate_genetic_Parameters():&#xA;            initial_parameters=[]&#xA;            x_max=np.max(xData)&#xA;            x_min=np.min(xData)&#xA;            y_max=np.max(yData)&#xA;            y_min=np.min(yData)&#xA;            slope=10*(y_max-y_min)/(x_max-x_min)&#xA;&#xA;            initial_parameters.append([x_max,x_min]) #Bounds for module break point&#xA;            initial_parameters.append([-slope,slope]) #Bounds for slopeA&#xA;            initial_parameters.append([-slope,slope]) #Bounds for slopeB&#xA;            initial_parameters.append([y_max,y_min]) #Bounds for offset A&#xA;            initial_parameters.append([y_max,y_min]) #Bounds for offset B&#xA;&#xA;            result=differential_evolution(sumSquaredError,initial_parameters,seed=3)&#xA;&#xA;            return result.x&#xA;&#xA;        geneticParameters = generate_genetic_Parameters() #Generates genetic parameters&#xA;&#xA;fittedParameters, pcov= curve_fit(func, xData, yData, geneticParameters) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will do the trick, but what is the implicit standard of fitness that the differential evolution here deals with?&lt;/p&gt;&#xA;" OwnerUserId="26250" LastActivityDate="2019-06-06T15:43:33.600" Title="What qualifies as 'fitness' for a genetic algorithm that minimizes an error function?" Tags="&lt;python&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12724" PostTypeId="1" AcceptedAnswerId="12726" CreationDate="2019-06-06T16:45:36.437" Score="3" ViewCount="121" Body="&lt;p&gt;In Sutton &amp;amp; Barto's &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;,  in page 83 (101 of the pdf), there is a description of first-visit MC control. In the phase where they update &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, they do an average of all the returns &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; for that state-action pair. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why don't they just update the value with a weight for the value from previous episodes &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and a weight &lt;span class=&quot;math-container&quot;&gt;$1- \alpha$&lt;/span&gt; for the new episode return as it is done in TD-Learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also seen other books (for example, &lt;a href=&quot;https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithms for RL&lt;/a&gt; (page 22) where they update it using &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;. What is the difference?&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="2444" LastEditDate="2020-01-23T11:53:05.987" LastActivityDate="2020-01-23T11:53:05.987" Title="Why is an average of all returns used to update the value in the first-visit MC control?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;sutton-barto&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12726" PostTypeId="2" ParentId="12724" CreationDate="2019-06-06T18:58:41.563" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why don't they just update the value with a weight for the value from previous episodes &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and a weight &lt;span class=&quot;math-container&quot;&gt;$1- \alpha$&lt;/span&gt; for the new episode return as it is done in TD-Learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my opinion, this is a mistake in the book. I went back and checked that this is still the same in the &lt;a href=&quot;http://incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;finished second edition&lt;/a&gt;, and it is still there.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keeping all returns and taking averages works fine in a prediction scenario with a fixed policy, and it is somewhat simpler and more intuitive to explain as &quot;this is the mean value&quot; (efficiency can come later, after comprehension). However, the pseudo-code is incorrect for a control scenario where older returns (that you still refer in the list) will not reflect the current policy, so will be biased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, no-one really uses the MC Control algorithms as written. Nearly always a learning rate parameter, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is used to update estimates. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the difference?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a &lt;em&gt;prediction&lt;/em&gt; scenario, where you want to evaluate a fixed policy, it would be slightly more efficient to use a count of samples for each &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; pair and variable &lt;span class=&quot;math-container&quot;&gt;$\alpha = \frac{1}{N(s,a)}$&lt;/span&gt; which is mathematically identical to keeping a list and taking the average as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a control scenario using a list of old return values is not only memory inefficient, it is also sample inefficient as you would need samples from each improved policy to outnumber the older samples in order to remove sampling bias due to using returns from initial worse policies. A fixed, or slowly decaying &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a simple and efficient way of &quot;forgetting&quot; the old return values as they become less relevant to the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to achieve similar forgetting may be to have a maximum size for each &lt;span class=&quot;math-container&quot;&gt;$Returns(s,a)$&lt;/span&gt; list. However, that is not mentioned in the MC Control pseudocode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I am missing something, as the book has been through significant review process. However, it may be that this detail is overlooked because the &lt;em&gt;basic&lt;/em&gt; MC Control scenarios are not as interesting as all the extensions and combinations with TD, where you will find the book does use a learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; - including when making comparison charts of MC vs TD approaches etc. The explicit list of returns and averaging over them for control only occurs in a couple of places in the text in this pseudocode, and is not mentioned again.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-06-06T19:11:58.547" LastActivityDate="2019-06-06T19:11:58.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12727" PostTypeId="1" CreationDate="2019-06-06T20:10:35.553" Score="1" ViewCount="151" Body="&lt;p&gt;If an NLP system processes a text containing proper nouns like names, trade marks, etc. without knowing anything about the language (ie no lexicon), is it possible to recognise them?&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="2193" LastEditDate="2019-06-07T08:30:15.400" LastActivityDate="2019-12-20T07:16:44.750" Title="How to distinguish between proper nouns and other words in NLP?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12728" PostTypeId="2" ParentId="12727" CreationDate="2019-06-06T21:32:55.633" Score="1" Body="&lt;p&gt;Essentially you would need to train a named entity recognizer (NER) to recognize the names out of the common words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many works that try to use a similar language to the language in question as a pivot to train a full NER model (for example, &lt;a href=&quot;https://www.aclweb.org/anthology/D17-1269&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Cheap Translation for Cross-Lingual Named Entity Recognition&lt;/em&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your task might be slightly simpler, as you are interested only in one class: whether it is a named entity or not. But in general it is very similar to this setup.&lt;/p&gt;&#xA;" OwnerUserId="22794" LastEditorUserId="2193" LastEditDate="2019-06-07T11:55:55.800" LastActivityDate="2019-06-07T11:55:55.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12729" PostTypeId="2" ParentId="12659" CreationDate="2019-06-06T22:28:21.040" Score="1" Body="&lt;p&gt;Surprisingly all the other answers are very vague and try to approach this from the human translator POV. Let's switch over to ML engineer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When creating a translation tool, one of the first questions that we should consider is &lt;em&gt;&quot;How do we measure that our tool works?&quot;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is essentially what the OP is asking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this is not an easy task (some other answers explain why). There is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Evaluation_of_machine_translation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia Article&lt;/a&gt; that mentions different ways to evaluate machine translation results - both human and automatic scores exist (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/BLEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLEU&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/NIST_(metric)&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIST&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/LEPOR&quot; rel=&quot;nofollow noreferrer&quot;&gt;LEPOR&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With rise of neural network techniques, those scores improved significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Translation is a complex problem. There are many things that can go right(or wrong), and computer translation system often ignores some of the subtleties, which stands out for a human speaker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think if we are to think about the future, there are few things that we can rely on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Our techniques are getting better, wider known and tested. This is going to improve the accuracy in the long run.&lt;/li&gt;&#xA;&lt;li&gt;We are developing new techniques which can take into account variables previously ignored or just do a better job.&lt;/li&gt;&#xA;&lt;li&gt;Many of currently existing translation models are often &quot;reused&quot; to translate other languages (for example, try translating &quot;JEDEN&quot; from Polish to Chinese(traditional) using Google Translator - you will end up with &quot;ONE&quot;, which is an evidence pointing out the fact that Google translates Polish to English, and then English to Chinese). &#xA;This is obviously not a good approach - you are going to lose some information in the process - but it's a one that will still work, so companies like Google use it for languages where they don't have enough workpower or data.&#xA;With time, more specialized models will appear, which will improve the situation.&lt;/li&gt;&#xA;&lt;li&gt;Also, as previous point stated, more and more data will only help improving the machine translation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To summarize, this complex problem, although not solved, is certainly on a good way and allows for some impressive results for well-researched language pairs.&lt;/p&gt;&#xA;" OwnerUserId="26259" LastActivityDate="2019-06-06T22:28:21.040" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12731" PostTypeId="2" ParentId="12705" CreationDate="2019-06-07T00:40:15.057" Score="3" Body="&lt;p&gt;This is an interesting question. One good place to start would be the gargantuan catalogue of rather simple RTS flash games and the like. There are many sites that have 100s of such titles, and it would not be too difficult to build the framework for these types of games(there are open source tools that can help as well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as &quot;real&quot; mainstream games, the pickings are rather slim for simple benchmark RTS titles. One that comes to mind is Rome: Total War. At least in exhibition, there is not much to strategize over besides who to attack and from where. Also, as I assume this relates to starcraft, one good simplified predecessor to starcraft would be the Dune series, and in particular Dune 2.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-06-07T00:40:15.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12734" PostTypeId="1" AcceptedAnswerId="16782" CreationDate="2019-06-07T13:23:33.370" Score="1" ViewCount="46" Body="&lt;p&gt;The question is little bit broad, but I could not find any concrete explanation anywhere, hence decided to ask the experts here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have trained a classifier model for binary classification task. Now I am trying to fine tune the model. With different sets of hyperparameters I am getting different sets of accuracy on my train and test set. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1) Train set: 0.99 | Cross-validation set: 0.72&#xA;(2) Train set: 0.75 | Cross-validation set: 0.70&#xA;(3) Train set: 0.69 | Cross-validation set: 0.69&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are approximate numbers. But my point is - for certain set of hyperparameters I am getting more or less similar CV accuracy, while the accuracy on training data varies from overfit to not so much overfit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is - which of these models will work best on future unseen data? What is the recommendation in this scenario, shall we choose the model with higher training accuracy or lower training accuracy, given that CV accuracy is similar in all cases above (in fact CV score is better in the overfitted model)?&lt;/p&gt;&#xA;" OwnerUserId="23993" LastActivityDate="2019-11-26T10:02:43.207" Title="Ideal score of a model on training and cross validation data" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;cross-validation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12735" PostTypeId="2" ParentId="12654" CreationDate="2019-06-07T13:23:54.230" Score="1" Body="&lt;p&gt;There are a few different ways you could frame the problem...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the simplest, yet probably not the most effective, would be to treat it as a supervised classification task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you would gather data, split it into 2 classes(binary). With one dataset consisting of dialogues and the other monologues. Obviously, this framing comes with it's own set of problems. Namely, how do you prepare the dataset, or how do you deal with live examples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there are other ways you could tackle the problem, perhaps the detection context. It all depends on what you feel is the easiest tenable solution given the variables involved in your specific instance.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-06-07T13:23:54.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12736" PostTypeId="1" AcceptedAnswerId="12743" CreationDate="2019-06-07T15:05:02.353" Score="1" ViewCount="17" Body="&lt;p&gt;I wish to be able to detect: pedestrians, cars, traffic lights&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two large datasets:&#xA; - One contains instances and labels of all three classes.&#xA; - The other contains instances of all three but only labels for pedestrians and cars. ie. there are many unlabelled traffic lights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to combine the two datasets and train Yolov3 on it. Will the unlabelled presence of objects of interest significantly affect detection performance of that category?&lt;/p&gt;&#xA;" OwnerUserId="21583" LastActivityDate="2019-06-07T19:39:27.820" Title="Are absence of labels for classes of interest in a vision dataset a big problem?" Tags="&lt;datasets&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12738" PostTypeId="1" CreationDate="2019-06-07T16:42:17.437" Score="2" ViewCount="134" Body="&lt;p&gt;In regression, in order to minimize an error function, a functional form of hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; must be decided upon, and it must be assumed (as far as I'm concerned) that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, the true mapping of instance space to target space, must have the same form as &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; (if &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is linear, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; should be linear. If &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is sinusoidal, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; should be sinusoidal. Otherwise the choice of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; was poor). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, doesn't this require a priori knowledge of datasets that we are wanting to let computers do on their own in the first place? I thought machine learning was letting machines do the work and have minimal input from the human. Are we not telling the machine what general form &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; will take and letting the machine using such things as error minimization do the rest? That seems to me to forsake the whole point of machine learning. I thought we were supposed to have the machine work for us by analyzing data after providing a training set. But it seems we're doing a lot of the work for it, looking at the data too and saying &quot;This will be linear. Find the coefficients &lt;span class=&quot;math-container&quot;&gt;$m, b$&lt;/span&gt; that fit the data.&quot;&lt;/p&gt;&#xA;" OwnerUserId="26250" LastActivityDate="2019-06-10T12:46:54.897" Title="How is regression machine learning?" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;supervised-learning&gt;&lt;regression&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12739" PostTypeId="1" AcceptedAnswerId="12740" CreationDate="2019-06-07T16:56:20.500" Score="3" ViewCount="181" Body="&lt;p&gt;AlphaGo Zero (&lt;a href=&quot;https://deepmind.com/blog/alphago-zero-learning-scratch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/blog/alphago-zero-learning-scratch/&lt;/a&gt;) has several key components that contribute to it's success:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A Monte Carlo Tree Search Algorithm that allows it to better search and learn from the state space of Go&lt;/li&gt;&#xA;&lt;li&gt;A Deep Neural Network architecture that learns the value and policies of given states, to better inform the MCTS.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question is, how is this Reinforcement Learning?  Or rather, what aspects of this algorithm specifically make it a Reinforcement Learning problem?  Couldn't this just be considered a Supervised Learning problem?  &lt;/p&gt;&#xA;" OwnerUserId="22424" LastActivityDate="2019-06-07T17:28:08.770" Title="How Does AlphaGo Zero Implement Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;supervised-learning&gt;&lt;alphago-zero&gt;&lt;go&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12740" PostTypeId="2" ParentId="12739" CreationDate="2019-06-07T17:21:59.100" Score="3" Body="&lt;p&gt;If you learn a policy or a value function from experience (that is, interaction with an environment), that's RL. In the case of AlphaGo, the MCTS is used to acquire the experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL could in fact be considered supervised learning (SL) or, more specifically, &lt;a href=&quot;https://ai.stackexchange.com/q/10623/2444&quot;&gt;self-supervised learning&lt;/a&gt;, where the experience corresponds to the labels in SL, especially nowadays with techniques like &lt;a href=&quot;https://ai.stackexchange.com/q/6579/2444&quot;&gt;experience replay&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-07T17:28:08.770" LastActivityDate="2019-06-07T17:28:08.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12741" PostTypeId="2" ParentId="12738" CreationDate="2019-06-07T19:15:07.777" Score="4" Body="&lt;p&gt;So in a sense you are correct. Using your jargon: linear regression will only &quot;work&quot; if the true function is approximately &lt;span class=&quot;math-container&quot;&gt;$y=h(x)=\beta^{T}x+\beta_0$&lt;/span&gt;. Advantages to using this is that its light, its convex, and all-around easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper models come into play: You allow a learn-able featurization and classification/regression. Think about it this way, the result of your regression is most likely linearly associated with some set of features, they just may not be the ones you are interested in (you can prove this actually with any infinitely wide network :: &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal approx Thm&lt;/a&gt;). Unfortunately we cant use an infinitely dimensional model, so we run with these giant over-parametrized models where we hope the a good function can be described by a sub-structure (only recently are we starting to pay attention, to how these sub-structures form -- look at this &lt;a href=&quot;https://arxiv.org/abs/1803.03635&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the way you bring about thinking about it is a large pit fall for many trying to move forward. Alot of ML people now of days gain success by throwing a function without alot of parameters on a big data problem, but youll see the largest advancements in the field come from a theoretical understanding of the featurization and optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helped&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-07T19:15:07.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12742" PostTypeId="1" CreationDate="2019-06-07T19:32:00.257" Score="2" ViewCount="17" Body="&lt;p&gt;I've read a number of articles on &lt;a href=&quot;https://medium.com/x8-the-ai-community/machine-learning-hardware-1dac168423fd&quot; rel=&quot;nofollow noreferrer&quot;&gt;how GPUs can speed up matrix algebra calculations&lt;/a&gt;, but I'm wondering how calculations are performed when one uses various &lt;a href=&quot;http://crsouza.com/2010/03/17/kernel-functions-for-machine-learning-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kernel functions&lt;/a&gt; in a neural network.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use Sigmoid functions in my neural network, does the computer use the CPU for the Sigmoid calculation, and then the GPU for the subsequent matrix calculations?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is the GPU capable of doing &lt;strong&gt;nonlinear&lt;/strong&gt; calculations in addition to the linear algebra calculations?  If not, how about a simple Kernel function like &lt;a href=&quot;https://medium.com/tinymind/a-practical-guide-to-relu-b83ca804f1f7&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt;?  Can a GPU do the Relu calculation, or does it defer to the CPU?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I'm using &lt;a href=&quot;https://www.tensorflow.org/guide/keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras with a Tensorflow backend&lt;/a&gt;, and would like to know what TensorFlow can and cannot use the GPU for, but I'm also interested in the general case.&lt;/p&gt;&#xA;" OwnerUserId="17741" LastActivityDate="2019-06-07T19:32:00.257" Title="In addition to matrix algebra, can GPU's also handle the various Kernel functions for Neural Networks?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;performance&gt;&lt;gpu&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12743" PostTypeId="2" ParentId="12736" CreationDate="2019-06-07T19:39:27.820" Score="0" Body="&lt;p&gt;So depends on sizes— if the second dataset has a lot of traffic lights without labels and you in a basic setup train on it, it will cause large performance decreases on that class. If it’s small, then it may just be “noise” to the model and it’ll filter it out, and still perform well. My recommendation is to adjust the network slightly and add a flag input saying which dataset the input was from: if from first, train normally. if from the second then ignore all losses pertain to boxes/classes that the network thinks a traffic light was in: that way your labels don’t try to tell the mode l “there’s no traffic light there”&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-07T19:39:27.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12744" PostTypeId="1" CreationDate="2019-06-07T20:59:15.693" Score="1" ViewCount="14" Body="&lt;p&gt;Most examples if not all, are models that have been trained with images that are turned grey. Does this mean that models only detect edges? Why wouldnt you want to keep color so that model could learn that as well?&lt;/p&gt;&#xA;" OwnerUserId="22145" LastActivityDate="2019-06-07T20:59:15.693" Title="Training Haar Cascade model with grey vs color images" Tags="&lt;classification&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12745" PostTypeId="2" ParentId="12738" CreationDate="2019-06-07T22:32:27.477" Score="0" Body="&lt;p&gt;It is just a statistical technique that is used in machine learning and it depends on the nature of the machine learning problem. I think you should be referred to the relation of the statistics and machine learning. These are no the same, but you can see the statistical methods in machine learning methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your specific problem, there are a lot of optimization techniques in AI (not specifically in machine learning). So, I think you should scrutinize more on the problem to find the relation of machine learning, AI, and statistics in this regression example.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-06-07T22:32:27.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12746" PostTypeId="1" CreationDate="2019-06-08T00:45:20.980" Score="1" ViewCount="65" Body="&lt;p&gt;I want to train a neural network on some input data from a probability distribution (say a Gaussian). The loss function would normally be &lt;span class=&quot;math-container&quot;&gt;$-\sum\log(f(x_i))$&lt;/span&gt;, where the sum is over the whole data (or in this case a mini batch) and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the NN function. However I need to enforce the fact that &lt;span class=&quot;math-container&quot;&gt;$\int_0^\infty f(x)dx=1$&lt;/span&gt;, in order for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; to be a real probability distribution. How can I add that to the loss function? Thank you! &lt;/p&gt;&#xA;" OwnerUserId="23871" LastActivityDate="2019-06-08T06:31:04.890" Title="Unit integral condition on the output layer" Tags="&lt;loss-functions&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="12750" PostTypeId="1" CreationDate="2019-06-08T15:50:42.047" Score="2" ViewCount="43" Body="&lt;p&gt;I have some limited experience with MLPs and CNNs. I am working on a project where I've used a CNN to classify &quot;images&quot; into two classes, 0 and 1. I say &quot;images&quot; as they are not actually images in the traditional sense, rather we are encoding a string from a limited alphabet, such that each character has a one-hot encoded row in the &quot;image&quot;. For example, we are using this for a bioinformatics application, so with the alphabet {A, C, G, T} the sequence &quot;ACGTCCAGCTACTTTACGG&quot; would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fYLlG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fYLlG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All &quot;images&quot; are 4x26. I used a CNN to classify pairs of &quot;images&quot; (either using two channels, i.e. 2x4x26 or concatenating two representations as 8x26) according to our criteria with good results. The main idea is for the network to learn how 2 sequences interact, so there are particular patterns that make sense. If we want to detect a reverse complement for example, then the network should learn that A-T and G-C pairs are important. For this particular example, if the interaction is high/probable, the assigned label is 1, otherwise 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now we want to go one step further and have a model that is able to generate &quot;images&quot; (sequences) that respect the same constraints as the classification problem. To solve this, I looked at Generative Adversarial Networks as the tool to perform the generation, thinking that maybe I could adapt the model from the classification to work as the discriminator. I've looked at the &quot;simpler&quot; models such as &lt;a href=&quot;https://arxiv.org/abs/1511.06434&quot; rel=&quot;nofollow noreferrer&quot;&gt;DCGAN&lt;/a&gt; and GAN, with implementations from &lt;a href=&quot;https://github.com/eriklindernoren/Keras-GAN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eriklindernoren/Keras-GAN&lt;/a&gt;, as I've never studied or used a GAN before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say that we want to generate pairs that are supposed to interact, or with the label 1 from before. I've adapted the DCGAN model to train on our 1-labelled encodings and tried different variations for the discriminator and generator, keeping in mind rules of thumb for stability. However, I can't get the model to learn anything significant. For example, I am trying to make the network learn the simple concept of reverse complement, mentioned above (expectation: learn to produce a pair with high interaction, from noise). Initially the accuracy for the discriminator is low, but after a few thousand epochs it increases drastically (very close to 100%, and the generator loss is huge, which apparently is a good thing, as the two models &quot;compete&quot; against each other?). However, the generated samples do not make any sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the generator learns the one-hot encoding above - since early generator output is just noise, it probably learns something like &quot;a single 1 per column is good&quot;, but not the more high level relation between the 1s and 0s. The discriminator probably is able to tell that the early generated outputs are garbage as there are 1s all over the place, but perhaps at some point the generator can match the one-hot encoding and thus the discriminator decides that is not a fake. This would explain the high accuracy, despite the sequences not making sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure of this is the case or not, or if it makes sense at all (I've just started reading about GANs yesterday). Is there a way to capture the high level features of the dataset? I am not interested in just generating something that looks like a real encoding, I'd like to generate something that follows the encoding but also exhibits patterns from the original data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking that maybe pretraining the discriminator would be a good idea, because it would then be able to discern between real-looking encodings for both the 0 and 1 classes. However, the pretraining idea seems frowned upon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd appreciate any ideas and advice. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="26292" LastEditorUserId="26292" LastEditDate="2019-06-08T20:20:23.877" LastActivityDate="2019-06-08T20:20:23.877" Title="Can GANs be used to generate matching pairs to inputs?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;&lt;generative-model&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12751" PostTypeId="1" CreationDate="2019-06-08T16:27:22.713" Score="2" ViewCount="331" Body="&lt;p&gt;I'm trying to make an AI to play my own card game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an OpenAI gym for the game with Dict as an observation space. It is nested dict, so I can't easily replace it with a tuple. I want to pass this observation space as model input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to encode this observation space in some multidimensional array by myself? &#xA;I thought there should be some code in gym/keras-rl that would help me with that task.&lt;/p&gt;&#xA;" OwnerUserId="26102" LastActivityDate="2019-06-08T16:27:22.713" Title="Gym dict space as keras DQN agent input" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;dqn&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12752" PostTypeId="2" ParentId="12738" CreationDate="2019-06-08T17:20:24.523" Score="1" Body="&lt;p&gt;Actually regression comes under the statistical analysis.  As you know many business activity(decision making) relies in the previous trends that can be grabbed from the organizations transaction data. When regression is performed on those organizational data. One can understand what decision can be made. One could even simulate the different conditions when the regression line is generated and to predict the unknown cases, decision maker can pass the numerical values corresponding to the certain phenomena in the operation of the organization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How regression is machine learning?&lt;/strong&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's start from the definition of machine learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn for themselves.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.expertsystem.com/machine-learning-definition/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.expertsystem.com/machine-learning-definition/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As from the definition it becomes clear that machine learning is to know the inner insight about the data without being explicitly programmed. &lt;strong&gt;Doesn't it fills great to know about what my previous trends in the business related transaction data is trying to convey me.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Please note that, in machine learning algorithms like Regression, one is trying to built some relation between the transnational data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So how relation between the data is being built?&lt;/strong&gt;&lt;br&gt;&#xA;Consider you are in the business of selling and buying the house and you want to predict the house price according to the latest trend. So what you have got is the data of house price and the feature of the house.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Feature : house_area, no_of_rooms&lt;br&gt;&#xA;  Target (what you want to predict): Price &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now, you perform regression on those data and you want to find out what would be best price for the house with the feature that is not mentioned in the latest trend's data.&#xA;Suppose general regression becomes like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;price = a * hourse_area + b * no_of_rooms + some_constant&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So in some sense. We're just trying to find the best fit line of the latest trend data with some variables like a, b and some_constant. Isn't it great to find such higher level of details from those trends data to know what would be the house price for non-mentioned data in the so called 'training data'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Choosing the objective function for best mapping?&lt;/strong&gt;&lt;br&gt;&#xA;Suppose relation is sometimes is non-linear. But how my algorithms would know that. In such case, one can use Artificial-Neural Network as it can learn to hypothesize the non-linear training data too. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note: You can learn to simulate the non linear data at: &lt;a href=&quot;https://playground.tensorflow.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://playground.tensorflow.org&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12021" LastEditorUserId="12021" LastEditDate="2019-06-08T17:29:36.110" LastActivityDate="2019-06-08T17:29:36.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12753" PostTypeId="1" CreationDate="2019-06-08T22:58:23.453" Score="1" ViewCount="22" Body="&lt;p&gt;Given a set of historical data points, I am trying to predict a continuous output of which I have no historical record of, therefore the problem is of an unsupervised nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is any method or approach I should take to tackle this problem? Essentially, how to build a model that will provide an output that is not clustered?&lt;/p&gt;&#xA;" OwnerUserId="26296" LastActivityDate="2019-06-08T22:58:23.453" Title="Prediction of values with an unsupervised model" Tags="&lt;models&gt;&lt;unsupervised-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12754" PostTypeId="2" ParentId="12705" CreationDate="2019-06-09T02:22:53.467" Score="3" Body="&lt;p&gt;These are some examples of simple RTS games I know of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://code.fb.com/ml-applications/introducing-elf-an-extensive-lightweight-and-flexible-platform-for-game-research/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELF Mini-RTS&lt;/a&gt; (&lt;a href=&quot;https://github.com/facebookresearch/ELF&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;): &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is very fast; the game environment runs 40,000 frames per second per core on a Macbook Pro. It captures the key dynamics of a real-time strategy game: Both players gather resources, build facilities, explore unknown territory (terrain that is out of sight of the player), and attempt to control regions on the map. In addition, the engine has characteristics that facilitate AI research: perfect save/load/replay, full access to its internal game state, multiple built-in rule-based AIs, visualization for debugging, and a human-AI interface, among others.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/santiontanon/microrts&quot; rel=&quot;nofollow noreferrer&quot;&gt;microRTS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;microRTS is a small implementation of an RTS game, designed to perform AI research. The advantage of using microRTS with respect to using a full-fledged game like Wargus or Starcraft (using BWAPI) is that microRTS is much simpler, and can be used to quickly test theoretical ideas, before moving on to full-fledged RTS games.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.galcon.com/fusion/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Galcon Fusion&lt;/a&gt;: A streamlined RTS game with relatively few actions, such as unit production and when to attack/defend.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3373" LastEditorUserId="3373" LastEditDate="2019-06-10T02:45:44.283" LastActivityDate="2019-06-10T02:45:44.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12755" PostTypeId="1" CreationDate="2019-06-09T10:15:23.850" Score="1" ViewCount="176" Body="&lt;p&gt;From the answers to this question &lt;a href=&quot;https://ai.stackexchange.com/q/5769/2444&quot;&gt;In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?&lt;/a&gt;, I got the fact that each filter has different weights for each input channel. But why should that be the case? What if we apply the same weights to each input channel? Does it work or not?&lt;/p&gt;&#xA;" OwnerUserId="26304" LastEditorUserId="2444" LastEditDate="2019-07-11T21:54:40.943" LastActivityDate="2021-04-02T01:06:50.727" Title="Why should each filter have different weights for each input channel?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12756" PostTypeId="2" ParentId="12755" CreationDate="2019-06-09T14:00:05.123" Score="0" Body="&lt;p&gt;For simplicitly, let's consider only the first convolutional layer, that is, the one applied to the image. If you consider an RGB image, then there are &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; channels: the red channel, the green channel and the blue channel. Thus, a kernel that is applied to this image will also have &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; channels: the red channel, the green channel and the blue channel. In general, the distributions of the intensity of the red, green and blue colors in the image are different, so, in general, the red, green and blue channels of the kernel will also be different because they need to keep track of different information.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-09T14:00:05.123" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12757" PostTypeId="1" CreationDate="2019-06-09T14:06:57.560" Score="1" ViewCount="90" Body="&lt;p&gt;Since the supreme court is always political, why not program 9 AI robots that use different methods to determine whether a law is constitutional and the outcome of cases. How would engineers go about building this? Would it work? &lt;/p&gt;&#xA;" OwnerUserId="26306" LastActivityDate="2019-10-21T21:01:30.207" Title="Make 9 AIs to replace Supreme Court justices" Tags="&lt;robots&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12758" PostTypeId="1" CreationDate="2019-06-09T14:44:13.587" Score="1" ViewCount="61" Body="&lt;p&gt;I am training a neural network which produces the following errors (epoch number on the x axis). I have some questions regrading interpreting it.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;When I say &lt;code&gt;model.predict&lt;/code&gt; is it giving me the result based on the final state (that is epoch 5000)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Towards the end (and some places in the middle) there are places where the training error and validation error are farther apart. Does this mean that the model was over-fitting on those epochs?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on the graph, can one determine that the model was &lt;code&gt;best&lt;/code&gt; at a certain epoch? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does Keras have API methods to retrieve the model at a specific epoch so that I can retrieve the &lt;code&gt;best&lt;/code&gt; model?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jKlil.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jKlil.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26307" LastActivityDate="2019-06-09T14:44:13.587" Title="How can I interpret the following error graph?" Tags="&lt;deep-learning&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12759" PostTypeId="1" CreationDate="2019-06-09T16:17:01.957" Score="0" ViewCount="695" Body="&lt;p&gt;I have a question about how the value and policy heads are used in AlphaZero (not Alphago Zero), and where the leaf nodes are relative to the root node. Specifically, there seem to be several possible interpretations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Policy estimation only. This would be most similar to DFS in which an average evaluation is computed through MCTS rollouts (though as others have noted the AlphaZero implementation actually seem to be deterministic apart from the exploration component, so 'rollout' may not be the most appropriate term) after reaching the . Here each leaf node would be at the end of the game.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value estimation only. It seems that if the value network is to be used effectively, there should be a limit on the depth to which any position is searched, e.g. 1 or 2 ply. If so, what should the depth be?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They are combined in some way. If I understand correctly, there is a limit on the maximum number of moves imposed - so is this really the depth? By which I mean, if the game has still not ended, this is the chance to use the value head to produce the value estimation? The thing is that the paper states that the maximum number of moves for Chess and shogi games was 512, while it was 722 moves for Go. These are extremely deep - evaluations based on these seem to be rather too far from the starting state, even when averaged over many rollouts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My search for answers elsewhere hasn't yielded anything definitive, because they've focused more on one side or the other. For example, &lt;a href=&quot;https://nikcheerla.github.io/deeplearningschool/2018/01/01/AlphaZero-Explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nikcheerla.github.io/deeplearningschool/2018/01/01/AlphaZero-Explained/&lt;/a&gt; the emphasis seems to be on the value estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the Alphazero pseudocode, e.g. from &lt;a href=&quot;https://science.sciencemag.org/highwire/filestream/719481/field_highwire_adjunct_files/1/aar6404_DataS1.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://science.sciencemag.org/highwire/filestream/719481/field_highwire_adjunct_files/1/aar6404_DataS1.zip&lt;/a&gt; the emphasis seems to be on the policy selection. Indeed, it's not 100% clear if the value head is used at all (value seems to return -1 by default).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a gap in my understanding somewhere? Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: To explain this better, here's the bit of pseudocode given that I found slightly confusing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Network(object):&#xA;&#xA;  def inference(self, image):&#xA;    return (-1, {})  # Value, Policy&#xA;&#xA;  def get_weights(self):&#xA;    # Returns the weights of this network.&#xA;    return []&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the (-1,{}) can either be placeholders or -1 could be an actual value and {} a placeholder. My understanding is that they are both placeholders (because otherwise the value head would &lt;em&gt;never&lt;/em&gt; be used), but -1 is the default value for unvisited nodes (this interpretation is taken from here, from the line about First Play Urgency value: &lt;a href=&quot;http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html&lt;/a&gt;). Now, if I understand correctly, &lt;code&gt;inference&lt;/code&gt; is called in both during training and playing by the &lt;code&gt;evaluate&lt;/code&gt; function. So my core question is: how deep into the tree are the leaf nodes (i.e. where the &lt;code&gt;evaluate&lt;/code&gt; function would be called)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the bit of code that confused me. In the official pseudocode as below, the 'rollout' seems to last until the game is over (expansion stops when a node has no children). So this means that under most circumstances you'll have a concrete game result - the player to move doesn't have a single move, and hence has lost (so -1 also makes sense here).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def run_mcts(config: AlphaZeroConfig, game: Game, network: Network):&#xA;  root = Node(0)&#xA;  evaluate(root, game, network)&#xA;  add_exploration_noise(config, root)&#xA;&#xA;  for _ in range(config.num_simulations):&#xA;    node = root&#xA;    scratch_game = game.clone()&#xA;    search_path = [node]&#xA;&#xA;    while node.expanded():&#xA;      action, node = select_child(config, node)&#xA;      scratch_game.apply(action)&#xA;      search_path.append(node)&#xA;&#xA;    value = evaluate(node, scratch_game, network)&#xA;    backpropagate(search_path, value, scratch_game.to_play())&#xA;  return select_action(config, game, root), root&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But under such conditions, the value head &lt;em&gt;still&lt;/em&gt; doesn't get very much action (you'll almost always return -1 at the leaf nodes). There are a couple of exceptions to this. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When you reach the maximum number of allowable moves  - however, this number is a massive 512 for chess &amp;amp; Shogi and 722 for Go, and seems to be too deep to be representative of the 1-ply positions, even averaged over MCTS rollouts.&lt;/li&gt;&#xA;&lt;li&gt;When you are at the root node itself - but the value here isn't used for move selection (though it is used for the backprop of the rewards)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So does that mean that the value head is only used for the backprop part of AlphaZero (and for super-long games)? Or did I misunderstand the depth of the leaf nodes?&lt;/p&gt;&#xA;" OwnerUserId="26309" LastEditorUserId="26309" LastEditDate="2019-06-10T12:17:04.363" LastActivityDate="2019-07-12T23:36:59.593" Title="How does AlphaZero use its value and policy heads in conjunction?" Tags="&lt;reinforcement-learning&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12761" PostTypeId="1" AcceptedAnswerId="12771" CreationDate="2019-06-09T18:01:09.007" Score="1" ViewCount="851" Body="&lt;p&gt;I have way more unlabeled data than labeled data. Therefore I would like to train an Autoencoder using MobileNetV2 as the encoder. Then I will use the pretrained model for the classification of the labeled data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is rather difficult to &quot;invert&quot; the MobileNet architecture to create a Decoder. Therefore, my question is: can I use a different architecture for the decoder, or will this introduce weird artifacts?&lt;/p&gt;&#xA;" OwnerUserId="23063" LastActivityDate="2019-11-11T15:47:29.183" Title="Autoencoder for MobileNetV2" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;autoencoders&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12764" PostTypeId="1" AcceptedAnswerId="12765" CreationDate="2019-06-09T23:29:56.930" Score="1" ViewCount="86" Body="&lt;p&gt;Why do the GAN's loss functions use an expectation (sum + division) instead of a simple sum?&lt;/p&gt;&#xA;" OwnerUserId="26313" LastEditorUserId="2444" LastEditDate="2020-09-27T19:57:25.513" LastActivityDate="2020-09-27T19:57:25.513" Title="Why is an expectation used instead of simple sum in GANs?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12765" PostTypeId="2" ParentId="12764" CreationDate="2019-06-10T00:57:50.627" Score="0" Body="&lt;p&gt;I'm assuming you mean the original loss explained in &lt;a href=&quot;https://arxiv.org/pdf/1406.2661.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original GAN paper&lt;/a&gt;. This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isn't just in GANs, it's in most objective functions used in a widespread of problems.&lt;/p&gt;&#xA;&lt;p&gt;In general practice, though, dividing is good because it works as a normalization. Let's say you train one batch with 10 elements and the next with 8. Wouldn't you want each training example to be weighted equally?&lt;/p&gt;&#xA;&lt;p&gt;Also, even if you use equal-sized batches, then the &amp;quot;division&amp;quot; you mention gets eaten up by the learning rate if you're using some form of gradient scheme.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2020-09-27T19:56:30.567" LastActivityDate="2020-09-27T19:56:30.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12768" PostTypeId="1" CreationDate="2019-06-10T09:08:59.117" Score="2" ViewCount="136" Body="&lt;p&gt;Here's the famous VGG-16 model.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Wgf0y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wgf0y.png&quot; alt=&quot;VGG16&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth?  What's the reason for that?&lt;/p&gt;&#xA;&lt;p&gt;Is there a theory or paper trying to explain this kind of setting?&lt;/p&gt;&#xA;" OwnerUserId="26321" LastEditorUserId="2444" LastEditDate="2020-07-04T20:30:59.830" LastActivityDate="2020-12-01T21:02:15.317" Title="Why do the inputs and outputs of a convolutional layer usually have the same depth?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;convolutional-layers&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12769" PostTypeId="1" CreationDate="2019-06-10T09:52:22.400" Score="4" ViewCount="192" Body="&lt;p&gt;Artificial intelligence is changing everything. AI is going to replace our jobs. So, what are the jobs that AI is going to create or replace? &lt;/p&gt;&#xA;" OwnerUserId="26325" LastEditorUserId="2444" LastEditDate="2020-03-04T22:54:27.850" LastActivityDate="2020-03-05T00:42:43.347" Title="What are the jobs that artificial intelligence will create?" Tags="&lt;social&gt;&lt;profession&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12771" PostTypeId="2" ParentId="12761" CreationDate="2019-06-10T10:24:05.363" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;can I use a different architecture for the decoder, or will this&#xA;  introduce weird artifacts?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you are using &lt;a href=&quot;https://arxiv.org/abs/1505.04597&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-net&lt;/a&gt; -like architecture with skip connection from corresponding encoding to decoding layer outputs of corresponding layers should have the same spatial resolution. There is no other commonly recognized limitations on decoder architecture for convolutional networks.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-06-10T10:24:05.363" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12772" PostTypeId="2" ParentId="12769" CreationDate="2019-06-10T10:41:53.643" Score="2" Body="&lt;p&gt;A report from PwC states that - AI will displace around 7 million and will be creating around 7.2 million new jobs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to MIT researchers –  AI will replace tasks, not jobs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion AI revolution is mainly going to effect non-technological jobs. So, the next generation—of employees should focus on creating career in new roles like AI researchers, Data scientist, Robotic Engineer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/voVp5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/voVp5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;(Image Source : theedgesearch)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the new jobs that AI is going to create:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;AI engineer&lt;/li&gt;&#xA;&lt;li&gt;Big Data scientist&lt;/li&gt;&#xA;&lt;li&gt;Real-Time Data Engineer &lt;/li&gt;&#xA;&lt;li&gt;Machine Linguist&lt;/li&gt;&#xA;&lt;li&gt;Robot counselor&lt;/li&gt;&#xA;&lt;li&gt;Telepresence Specialists&lt;/li&gt;&#xA;&lt;li&gt;Vertical Farming Specialist&lt;/li&gt;&#xA;&lt;li&gt;Limb Engineer&lt;/li&gt;&#xA;&lt;li&gt;Sensor Scientist&lt;/li&gt;&#xA;&lt;li&gt;Virtual Experience Specialist&lt;/li&gt;&#xA;&lt;li&gt;Robotic Engineer&lt;/li&gt;&#xA;&lt;li&gt;Virtual Reality Engineer&lt;/li&gt;&#xA;&lt;li&gt;Genome Specialist&lt;/li&gt;&#xA;&lt;li&gt;Gaming engineer &lt;/li&gt;&#xA;&lt;li&gt;IOT Engineer&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26265" LastEditorUserId="1671" LastEditDate="2019-06-10T17:18:04.913" LastActivityDate="2019-06-10T17:18:04.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12773" PostTypeId="1" CreationDate="2019-06-10T12:19:14.283" Score="1" ViewCount="106" Body="&lt;p&gt;I am interested in creating a neural network-based engine for chess. It uses a &lt;span class=&quot;math-container&quot;&gt;$8 \times 8 \times 73$&lt;/span&gt; output space for each possible move as proposed in the Alpha Zero paper: &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when running the network, the first selected move is invalid. How should we deal with this? Basically, I see two options.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick the next highest outputted move, until it is a valid move. In this case, the network might automatically over time not put illegal moves on top.&lt;/li&gt;&#xA;&lt;li&gt;Process the game as a loss for the player who picked the illegal move. This might have the disadvantage that the network might be 'stuck' on only a few legal moves.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What is the preferred solution to this particular problem?&lt;/p&gt;&#xA;" OwnerUserId="26326" LastEditorUserId="2444" LastEditDate="2019-07-10T20:45:50.837" LastActivityDate="2019-07-10T20:46:24.560" Title="How to deal with invalid output in a policy network?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2019-07-11T22:15:53.363" ContentLicense="CC BY-SA 4.0" />
  <row Id="12774" PostTypeId="2" ParentId="12773" CreationDate="2019-06-10T12:27:27.000" Score="0" Body="&lt;p&gt;You should have a method to generate a possible moves output based on the board state. Use this as a mask before normalization in the policy head.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-10T20:46:24.560" LastActivityDate="2019-07-10T20:46:24.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12775" PostTypeId="2" ParentId="12738" CreationDate="2019-06-10T12:46:54.897" Score="0" Body="&lt;p&gt;What you are asking touches upon two very different approaches to machine learning:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The empirical approach (many people just call this 'machine learning', and some people like to call it 'algorithmic machine learning')&lt;/li&gt;&#xA;&lt;li&gt;The statistical approach (some people like to call this 'statistical machine learning')&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The purely empirical approach is very goal-oriented - think discriminative models that are only used for prediction. You really only care about whether the data fits the training + test data well according to whichever metric you've selected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statistical approach is very process-oriented - you would want to identify the processes that generate the data, the distributions they follow, whether your results are statistically significant, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along this spectrum, most folks fall somewhere into the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you've described is closer to statistical machine learning - to practitioners of the other approach, regression only means that you are trying to predict for a continuous target variable (whereas classification would be for a discrete target variable). Then you might poke around the data a bit, fiddle around with features &amp;amp; hyperparameters, and try out a lot of different regression algorithms, going from OLS, SVMs, nearest neighbour regressors, random forests, gradient boosted trees, and maybe even RNNs, etc. In the extreme case, a purist of this approach wouldn't care about the statistics or whatever the underlying distributions are at all, but only care if the final regression gives good results in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are clear risks with this approach (especially when the underlying assumptions of the models fall apart), it can give good results, especially when the practitioner is a good coder and can try out lots of possibilities very quickly, and even produce novel algorithms. The fact is that maths does sometimes lag development of other fields - Fourier analysis for example, and deep neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another very approximate analogy would be science vs engineering.&lt;/p&gt;&#xA;" OwnerUserId="26309" LastActivityDate="2019-06-10T12:46:54.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12777" PostTypeId="1" CreationDate="2019-06-10T14:17:46.757" Score="1" ViewCount="25" Body="&lt;p&gt;I'm wondering if anyone knows about any projects that break down conversations for an AI. I'll explain what I mean - I've seen a lot of hardcoded responses, but nothing that allows an AI to break a piece of dialogue into components (say that the speaker sounds happy or is trying to be manipulative) and model a response based on this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I envision a coding scheme for different core components of conversation. This would allow an AI to be more dynamic in its responses, and would be closer to actually being able to hold a conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm &lt;em&gt;not&lt;/em&gt; looking for AI generated text, at least not in the sense of some NN or the like being fed a diet of leterature and seeing what it spits out - there's nothing dynamic about that.&lt;/p&gt;&#xA;" OwnerUserId="26334" LastActivityDate="2019-06-10T14:17:46.757" Title="Conversational AI" Tags="&lt;natural-language-processing&gt;&lt;human-like&gt;&lt;concepts&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12778" PostTypeId="1" CreationDate="2019-06-10T14:35:03.723" Score="1" ViewCount="220" Body="&lt;p&gt;I am trying to make my AI win the board game &quot;Catan&quot; against my friends.&#xA;Therefore i am using the python implementation of NEAT. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I changed the values of &lt;code&gt;weight_mutate_power&lt;/code&gt;, &lt;code&gt;response_mutate_power&lt;/code&gt;, &lt;code&gt;bias_mutate_power&lt;/code&gt; and &lt;code&gt;compatibility_threshold&lt;/code&gt; in the config the number of individuals and species exploded (~Doubled every generation and exceeded &lt;code&gt;pop_size&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;weight_mutate_power = 12&#xA;response_mutate_power = 0&#xA;bias_mutate_power = 0.5&#xA;compatibility_threshold = 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Playing with those values I discovered that the rate of the explosion changes in relation to those values (Everything worked fine with standard Values from the documentation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea how to control this behavior? &#xA;My cluster is drowning in genomes...&lt;/p&gt;&#xA;" OwnerUserId="26332" LastActivityDate="2020-05-01T05:27:50.593" Title="Exploding population size in neat-python" Tags="&lt;python&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12779" PostTypeId="1" CreationDate="2019-06-10T14:52:01.243" Score="2" ViewCount="194" Body="&lt;p&gt;I am writing a report where I used a slightly modified version of MCTS (not parallelized). I thought It could be interesting if I could calculate its time complexity. I'd appreciate any help I could get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the rough idea of how it works:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of tree search, I'm using graph search meaning I keep a list of visited nodes in order to avoid adding duplicate nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the expansion phase, I add all child nodes of the current node that aren't present elsewhere in the tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the remaining phases, it's essentially the same as the basic version of MCTS, with a default random policy in the simulation step.&lt;/p&gt;&#xA;" OwnerUserId="23866" LastEditorUserId="2444" LastEditDate="2019-11-19T22:41:45.353" LastActivityDate="2019-11-19T22:41:45.353" Title="What is the time complexity of an unparellelized Monte Carlo tree search?" Tags="&lt;monte-carlo-tree-search&gt;&lt;time-complexity&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12780" PostTypeId="2" ParentId="11787" CreationDate="2019-06-10T16:12:04.993" Score="0" Body="&lt;p&gt;Sorry this is more of a comment than an answer. I'm wondering if you have found a definitive answer to your question, because I've a very related question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm also confused by the AlphaZero algorithm - my explanation for my confusion is specified here: &lt;a href=&quot;https://ai.stackexchange.com/questions/12759/how-does-alphazero-use-its-value-and-policy-heads-in-conjunction&quot;&gt;How does AlphaZero use its value and policy heads in conjunction?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that I think the AlphaZero algorithm is also different from the Alphago Zero algorithm. A lot of sources that I've tried referred to really mix the two together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, there's this function in the official pseudocode, which really confused me:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def run_mcts(config: AlphaZeroConfig, game: Game, network: Network):&#xA;  root = Node(0)&#xA;  evaluate(root, game, network)&#xA;  add_exploration_noise(config, root)&#xA;&#xA;  for _ in range(config.num_simulations):&#xA;    node = root&#xA;    scratch_game = game.clone()&#xA;    search_path = [node]&#xA;&#xA;    while node.expanded():&#xA;      action, node = select_child(config, node)&#xA;      scratch_game.apply(action)&#xA;      search_path.append(node)&#xA;&#xA;    value = evaluate(node, scratch_game, network)&#xA;    backpropagate(search_path, value, scratch_game.to_play())&#xA;  return select_action(config, game, root), root&#xA;&#xA;&#xA;def select_action(config: AlphaZeroConfig, game: Game, root: Node):&#xA;  visit_counts = [(child.visit_count, action)&#xA;                  for action, child in root.children.iteritems()]&#xA;  if len(game.history) &amp;lt; config.num_sampling_moves:&#xA;    _, action = softmax_sample(visit_counts)&#xA;  else:&#xA;    _, action = max(visit_counts)&#xA;  return action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Mainly because, if you look at the definition of &lt;code&gt;expanded&lt;/code&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  def expanded(self):&#xA;    return len(self.children) &amp;gt; 0&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I.e. when there are no more moves, and we are at the end of the game. I wonder what I'm missing here.&lt;/p&gt;&#xA;" OwnerUserId="26309" LastActivityDate="2019-06-10T16:12:04.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12781" PostTypeId="2" ParentId="12768" CreationDate="2019-06-10T16:14:20.947" Score="-1" Body="&lt;p&gt;Keeping the same channel size allows the model to maintain rank but i would say the main reason is convenience. Its easier book keeping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also in many model cases output features need some form of alignment with the input (example being all models using residual units -- &lt;span class=&quot;math-container&quot;&gt;$\hat{x} = F(x) + x$&lt;/span&gt; &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-10T16:14:20.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12783" PostTypeId="1" CreationDate="2019-06-10T18:54:18.250" Score="1" ViewCount="51" Body="&lt;p&gt;I'm trying to write the proof of correctness of Monte Carlo Tree Search. Any help would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23866" LastEditorUserId="2444" LastEditDate="2019-11-19T22:40:36.287" LastActivityDate="2019-11-19T22:40:36.287" Title="Proof of Correctness of Monte Carlo Tree Search" Tags="&lt;monte-carlo-tree-search&gt;&lt;proofs&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12784" PostTypeId="1" CreationDate="2019-06-10T20:25:12.240" Score="2" ViewCount="21" Body="&lt;p&gt;Shouldn't the discriminator and generator work fine even if they don't process data symmetrically? I mean, they don't only receive the final layer results of each other, they don't use data that from hidden layers.&lt;/p&gt;&#xA;" OwnerUserId="23941" LastActivityDate="2019-06-10T20:25:12.240" Title="How important is architectural similarity between the discriminator and generator of a GAN?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12785" PostTypeId="2" ParentId="11740" CreationDate="2019-06-10T20:31:40.723" Score="2" Body="&lt;p&gt;The paper that introduced AlphaGo, &lt;a href=&quot;https://www.nature.com/articles/nature16961.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the game of Go with deep neural networks and tree search&lt;/a&gt;, motivates the use of MCTS&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://hal.inria.fr/inria-00116992/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo tree search (MCTS)&lt;/a&gt; uses Monte Carlo rollouts to estimate the value of each state in a search tree. As more simulations are executed, the search tree grows larger and the relevant values become more accurate. The policy used to select actions during search is also improved over time, by selecting children with higher values. &lt;a href=&quot;http://ggp.stanford.edu/readings/uct.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asymptotically, this policy converges to optimal play, and the evaluations converge to the optimal value function&lt;/a&gt;. &lt;strong&gt;&lt;a href=&quot;https://hal.inria.fr/inria-00149859/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;The strongest current Go programs are based on MCTS&lt;/strong&gt;, enhanced by policies that are trained to predict human expert moves&lt;/a&gt;. These policies are used to narrow the search to a beam of high-probability actions, and to sample actions during rollouts. This approach has achieved strong amateur play.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-10T20:31:40.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12786" PostTypeId="1" CreationDate="2019-06-10T20:31:41.253" Score="2" ViewCount="86" Body="&lt;p&gt;I was working on CNN. I modified the training procedure on runtime.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0hrgz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0hrgz.png&quot; alt=&quot;Validation Loss and Validation Accuracy&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As we can see from the validation loss and validation accuracy, the yellow curve does not fluctuate much. The green curve and red curve fluctuate suddenly to higher validation loss and lower validation accuracy, then goes to the lower validation loss and the higher validation accuracy, especially for the green curve.&lt;/p&gt;&#xA;&lt;p&gt;Is it happening because of overfitting or something else?&lt;/p&gt;&#xA;&lt;p&gt;I am asking it because, after fluctuation, the loss decreases to the lowest point, and also the accuracy increases to the highest point.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone tell me why is it happening?&lt;/p&gt;&#xA;" OwnerUserId="26342" LastEditorUserId="2444" LastEditDate="2020-11-07T11:31:21.073" LastActivityDate="2021-04-06T13:01:30.127" Title="Validation Loss Fluctuates then Decrease alongside Validation Accuracy Increases" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;data-preprocessing&gt;&lt;data-augmentation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12787" PostTypeId="1" CreationDate="2019-06-10T20:57:15.263" Score="2" ViewCount="57" Body="&lt;p&gt;In general, how does one balance the two opposing forces of allowing a layer of a neural network to adapt/learn to its training data, while also forcing the neural network to represent some known structure (e.g., representing a family of functions)? The neural network might find the optimal weights, but those weights might no longer make the layer represent the function I originally intended. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose I want to create a convolutional layer in the middle of my neural network that is a low-pass filter. In the context of the entire network, however, the layer might cease to be a low-pass filter at the end of training because the backpropagation algorithm found a better optimum. How do I allow the weights to be as optimal as possible, while still maintaining the low-pass characteristics I originally wanted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help and suggestions. General tips or pointing to specific literature would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26344" LastActivityDate="2019-06-12T09:42:03.663" Title="Balancing learning and structure in a neural network" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12788" PostTypeId="1" CreationDate="2019-06-10T21:47:56.453" Score="1" ViewCount="50" Body="&lt;p&gt;I have a time-varying input size vector for a RNN. However, I am facing some difficulties understanding how to deal with my network weights when the input changes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say we have a set of natural positive integers&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Gamma=\{1,2,\dots,F\},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$F=100$&lt;/span&gt; for the sake of the example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A valid observation vector of my agent at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; might be &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\gamma_t=[1,3,5,1].&#xA;$$&lt;/span&gt;&#xA;Thus, at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; a set of weights will be produced by my RNN, according to &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt;. Sat that at time &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;, my observation vector changes as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\gamma_{t+1}=[3,5,8],&#xA;$$&lt;/span&gt;&#xA;and there is my problem. If I now continue training my RNN with the previous weights, the output would be inevitably affected. Also, Which weight shall I remove? I see RNN can face the issue but how shall I deal with the previously computed weights? Which one to remove? How to initialize a new one in case the cardinality of &lt;span class=&quot;math-container&quot;&gt;$\gamma_{t+1}$&lt;/span&gt; is higher than that of &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="26345" LastActivityDate="2019-06-11T15:08:56.683" Title="RNN weights when varying the input size" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12789" PostTypeId="2" ParentId="12769" CreationDate="2019-06-11T04:27:13.597" Score="1" Body="&lt;p&gt;As the rate of automation is increasing day by day and companies are keen on AI automation. Surely, AI is going to affect the job market. &lt;a href=&quot;https://economictimes.indiatimes.com/jobs/by-2020-artificial-intelligence-will-create-more-jobs-than-it-eliminates-gartner/articleshow/62053363.cms?from=mdr&quot; rel=&quot;nofollow noreferrer&quot;&gt;According to Granter&lt;/a&gt;, artificial intelligence will create more jobs than it eliminates by 2020. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think it is too early to define what kind of jobs are going to created by AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But its undeniable fact that AI is going to play a key role in companies. Obviously, companies require AI engineers, AI researchers and AI experts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from these jobs, such as robotic engineers, IOT engineers, machine learning specialists are also on-demand.&lt;/p&gt;&#xA;" OwnerUserId="26351" LastEditorUserId="2444" LastEditDate="2020-03-05T00:42:43.347" LastActivityDate="2020-03-05T00:42:43.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12791" PostTypeId="1" CreationDate="2019-06-11T06:22:51.713" Score="1" ViewCount="27" Body="&lt;p&gt;I'm working with a DCGAN, a deep CNN for classifying images with a GAN that competes with the classifier to generate images of what we are classifying. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the project at the moment is to produce AI generated memes in the form of pepe the frog, based on a dataset found on the internet of roughly 2000 images. I scaled them all maintaining aspect ratio as my only form of normalization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I train my data (I've tried many combinations of hyperparameters) upwards of 100k epochs (batch 32) my classifying network gets around 1^-6 average while my GAN approaches nearly 16, yes 16, with a properly defined loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now because my images are typically made with random features, some containing text, others containing full body renditions, turned away from viewport etc... I'm  assuming that this is because of the data I'm training with and it's diverse amount of features, is my reasoning correct? Also if allowed to continue is it possible that the GAN learns to properly generate the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main reason I have come to the above conclusion is that if I train on a few hand picked examples that have similar artistic styles/orientations and such (less than 100) and let it train my GAN will generate decent images, however they have low variability.&lt;/p&gt;&#xA;" OwnerUserId="26352" LastEditorUserId="26352" LastEditDate="2019-06-11T07:10:56.287" LastActivityDate="2019-06-11T07:10:56.287" Title="DCGAN loss determining data normalization problems" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12793" PostTypeId="1" CreationDate="2019-06-11T08:18:25.203" Score="1" ViewCount="50" Body="&lt;p&gt;I am trying to apply deep deterministic policy gradient (DDPG) on a robotic application. My states consist of the joint angle positions of the robot and my actions are also its joint angle positions. Since, DDPG produces a continuous policy output where states are directly mapped onto the actions, can I say that my next state and action will be same? Simplistically, the input of the policy network will be the current state and the output will be the next state?&lt;/p&gt;&#xA;" OwnerUserId="18986" LastActivityDate="2019-06-11T08:18:25.203" Title="Can next state and action be same in Deep Deterministic Policy Gradient?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12798" PostTypeId="2" ParentId="12769" CreationDate="2019-06-11T20:46:33.013" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;AI Product Manager&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is a role that didn't exist a few years ago.  &lt;a href=&quot;https://www.glassdoor.com/Job/product-manager-ai-jobs-SRCH_KO0,18.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a GlassDoor search from today&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Informally, Product Management positions indicate a certain degree of maturity for a new technology.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-06-11T20:46:33.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12799" PostTypeId="2" ParentId="10869" CreationDate="2019-06-11T23:14:18.280" Score="1" Body="&lt;p&gt;You have experimented with a small model (117M parameters). OpenAI has now released the medium sized model (345M parameters). Note that the full model has 1.5B parameters. Also note that GPT-2 has been trained on a tiny fraction of all available text. It's almost guaranteed that a larger model trained on more text will generate better text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have experimented with the 345M model, and I agree with The Guardian assessment. See for yourself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Prompt:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to generate beautiful music using machine learning. Do you&#xA;  think a computer will ever be able to compose something as novel as&#xA;  creations of Mozart, Bach, or Chopin?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Results&lt;/strong&gt; (first four samples, not cherry picked):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Yes, computers will probably be able to do it by 2020. I think the key&#xA;  for us will be to make music that is interesting and beautiful, so we&#xA;  will never really know. My hope would be that computers will become&#xA;  far more efficient over time – something that is already happening in&#xA;  a lot of ways like in big data – while still being able to do more&#xA;  things that they are trained on, like find patterns. I think that's&#xA;  where I'd want to use neural networks in the future.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We don't know, but it will probably happen. That's an interesting&#xA;  question to ask. Of course we have AI, which can do interesting things&#xA;  without human input. It can recognize the world, find objects and&#xA;  images, it understands language, we can make games for it. But if you&#xA;  imagine an AI being able to write music, will it even be good enough?&#xA;  How much better than human players? How much better can someone else&#xA;  do? AI will never really be at its best because it will always have to&#xA;  come up with creative ways to use and play it. To ask a question like&#xA;  it is good enough is to ignore that it is human. You are only allowed&#xA;  to ask good questions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The software will help, as it already can do. But it will also allow&#xA;  more human creators, which is what I think it should do more of to&#xA;  bring music to any genre. In terms of composing, it's a matter of&#xA;  creativity in humans in general. But how does it achieve that? Let's&#xA;  just say that there are fewer humans composing today than ever before.&#xA;  Do you remember the day when, by chance, a computer could write lyrics&#xA;  to Mozart's Sonata No. 4, or the day when a human wrote lyrics to&#xA;  Mozart's La Résistance du Ciel No. 2? That was back in 1960, and it's&#xA;  just not true anymore. As we move forward, we are not going to create&#xA;  music that human musicians cannot compose. What is your goal as a&#xA;  computer music creator?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At the core of this idea is that music is a collection of sounds – it&#xA;  was the very first art form, it was the first music that humans&#xA;  evolved, a collection of sounds. Music is not an abstract thing of&#xA;  sound, but a collection of sounds. You can't write music the way we&#xA;  write and compose books and films, because we write and compose with&#xA;  sounds in mind. This is where machine learning comes in. You can read&#xA;  some books about how to get better music writers or how to improve the&#xA;  quality of their music, but you cannot make a difference with this&#xA;  kind of information at all. What does music teach you? To write and&#xA;  perform beautifully, to convey emotion accurately, and to create&#xA;  lasting and rich relationships with&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="26373" LastActivityDate="2019-06-11T23:14:18.280" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12800" PostTypeId="2" ParentId="12787" CreationDate="2019-06-12T07:25:22.470" Score="1" Body="&lt;p&gt;Extending @mirror2image's comment, if you have a certain metric that allows you to measure how close the intended layer is to a low pass filter (something that compares its output with what a low pass filter would have produced, for example), the simplest way to achieve what you want would be to add a term in your loss function that calculates the value of this metric. This way, each time you do a training step, the network now is not only made to output the correct predictions but is also forced to do so while also keeping that specific layer's behavior as close to a low-pass filter as possible.&#xA;This is the most common way of tweaking the behavior of neural networks and is often encountered in many research papers.&lt;/p&gt;&#xA;" OwnerUserId="16159" LastActivityDate="2019-06-12T07:25:22.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12801" PostTypeId="1" CreationDate="2019-06-12T08:04:25.223" Score="2" ViewCount="164" Body="&lt;p&gt;I have two closed polygons, drawn as connected straight black lines on a white background. I need to classify such images in to three forms&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Two separate polygons&lt;/li&gt;&#xA;&lt;li&gt;One polygon encloses the other&lt;/li&gt;&#xA;&lt;li&gt;The two polygons overlap each other.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The polygons vary in sizes and location on the image, and the image contains only the polygons and the white background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which neural network architecture should I use to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="26382" LastEditorUserId="2444" LastEditDate="2019-06-12T11:15:16.350" LastActivityDate="2019-06-12T11:15:16.350" Title="How do I classify an image that contains only polygons?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12802" PostTypeId="2" ParentId="12448" CreationDate="2019-06-12T08:41:02.370" Score="1" Body="&lt;p&gt;I now currently use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Connect_Four&quot; rel=&quot;nofollow noreferrer&quot;&gt;Connect four&lt;/a&gt; / a &lt;a href=&quot;https://en.wikipedia.org/wiki/M,n,k-game&quot; rel=&quot;nofollow noreferrer&quot;&gt;m,n,k-game&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://boardgamegeek.com/boardgame/1875/isolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Isolation&lt;/a&gt; with 6x8 board&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I currently have, but may remove:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Iterated prisoners dilemma (hard to interpret and I am not sure if MCTS is really the right choice)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I may add sometime:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tic-tac-toe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tic-tac-toe&lt;/a&gt; as very simple verification&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Dots_and_Boxes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dots and Boxes&lt;/a&gt; (looks like it could be an interesting planning problem)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other good ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rules_of_Go#Concise_statement&quot; rel=&quot;nofollow noreferrer&quot;&gt;Go with simplified rules&lt;/a&gt; (thanks John Doucette)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I did not try all ideas, as I just want to verify my framework, that should be used on other problems (that are harder to verify). You can still keep adding answers with good ideas, that may be a good reference for others that want to test their implementations.&lt;/p&gt;&#xA;" OwnerUserId="25798" LastActivityDate="2019-06-12T08:41:02.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12803" PostTypeId="1" AcceptedAnswerId="12842" CreationDate="2019-06-12T08:41:47.570" Score="1" ViewCount="43" Body="&lt;p&gt;We have seen advances in top down, RTS team games like Dota 2 and Starcraft II from companies like OpenAI who developed agents to beat real pro players most of the time. How would similar learning techniques compare to games like Overwatch that require faster reaction times and complex understanding of 3d space and effect? &#xA;Or have we not developed solutions that could be tasked with this problem?&lt;/p&gt;&#xA;" OwnerUserId="26383" LastActivityDate="2019-06-14T08:59:08.460" Title="Feasibility of a team-based FPS AI?" Tags="&lt;game-ai&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12804" PostTypeId="2" ParentId="12787" CreationDate="2019-06-12T09:42:03.663" Score="1" Body="&lt;p&gt;the idea about training that it is allowed for weights (not layer as you wrote) &quot;learn&quot; values what they &quot;want&quot; from general network setting. But actualy i v thinked about that too, and to have weights represent as you want, you can first train a shorter network, while having those weights (1) as very last, so you have maximum control over them, after trained a lot append next layers to those and learning rate for weights (1) make much smaller as for new weights &lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-12T09:42:03.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12805" PostTypeId="1" CreationDate="2019-06-12T09:52:22.193" Score="3" ViewCount="99" Body="&lt;p&gt;Basically, economic decision making is not restricted to mundane finance, the managing of money, but any decision that involves expected utility (some result with some degree of optimality.)  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can Machine Learning algorithms make economic decisions as well as or better than humans?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&quot;Like humans&quot; means understanding classes of objects and their interactions, including agents such as other humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a fundamental level, there must be some physical representation of an object, leading to usage of an object, leading to management of resources that the objects constitute. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may include ability to effectively handle semantic data (NLP) because mcuh of the relevant information is communicated in human languages.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="1671" LastEditDate="2019-06-12T22:00:45.793" LastActivityDate="2019-06-14T17:23:45.967" Title="Can Machine Learning make economic decisions of human quality or better?" Tags="&lt;machine-learning&gt;&lt;economics&gt;" AnswerCount="2" CommentCount="7" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12808" PostTypeId="2" ParentId="12801" CreationDate="2019-06-12T10:39:20.680" Score="-1" Body="&lt;p&gt;To make it easy take small known CNN network like Alex Net, train like : input  is image, output is [1,0,0,] separate, [0,1,0], encloses, [0,0,1] overlap. Cause you task is easy i guess that will be fine. But it can be done without ML just analysing image with constant algorithm by where are the points of poligons.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-12T10:39:20.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12809" PostTypeId="2" ParentId="12657" CreationDate="2019-06-12T11:09:04.690" Score="1" Body="&lt;p&gt;One word answer for your question &quot;&lt;strong&gt;Do you need to store previous values of weights and layers on recurrent layer while BPTT?&lt;/strong&gt;&quot; is &lt;strong&gt;YES&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us go through the details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For training an RNN using BPTT, we need gradients of error w.r.t all three parameters &lt;strong&gt;U&lt;/strong&gt;, &lt;strong&gt;V&lt;/strong&gt;, &lt;strong&gt;W&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notation of my explanation is different from notation in the figure of question.&#xA;My notation is as below:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;V&lt;/strong&gt; - Hidden Layer - Output Layer (gradients of V are independent of&#xA;previous time steps) &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;U&lt;/strong&gt; - Input Layer - Hidden Layer (gradients of U are dependent on previous time steps) &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;W&lt;/strong&gt; - Hidden Layer - Hidden Layer (gradients of W are also dependent on previous time steps)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And for calculating these gradients, we use chain rule of differentiation, the same rule that we used to calculate gradients in a fully connected neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient w.r.t V only depends on current time step (doesn't need any values from previous time step). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradients w.r.t U, W depends on current time step and also all previous time steps (so needs values from all time steps)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, we need to back propagate gradients from current time step all the way to t=0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How this back propagation is different from the back propagation we use in fully connected neural network is that, in fully connected neural network we don't have the concept of t and also we don't share any weights across layers. But, here we share weights across layers and time instants. So, gradients depend on all time instants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note: Be careful with notation difference between several articles. I followed slightly different notation than in the diagram in question.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some links that will help you explore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=RrB605Mbpic&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=RrB605Mbpic&lt;/a&gt; (clearly explains about gradients of all three U, V, W; but notation is different from diagram in question)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/&lt;/a&gt;&lt;a href=&quot;http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ir.hit.edu.cn/~jguo/docs/notes/bptt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ir.hit.edu.cn/~jguo/docs/notes/bptt.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.d2l.ai/chapter_recurrent-neural-networks/bptt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.d2l.ai/chapter_recurrent-neural-networks/bptt.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember, you should understand chain rule of partial derivative very clearly to do the derivation yourself and understand it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, dont think BPTT is separate than BP. It is one and the same. Since neural network architecture in RNN includes time instants and sharing of weights across time instants, just using chain rule on this network makes back propagation also dependent on time and so is the name. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps. Feedback is welcome.&lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="20760" LastEditDate="2019-06-13T12:19:39.980" LastActivityDate="2019-06-13T12:19:39.980" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12810" PostTypeId="1" CreationDate="2019-06-12T12:04:21.097" Score="1" ViewCount="52" Body="&lt;p&gt;I want to use Machine Learning for text classification, more precisely, I want to determine whether a text (or comment) is positive or negative. I can download a dataset with 120 million comments. I read the TensorFlow &lt;a href=&quot;https://www.tensorflow.org/tutorials/keras/basic_text_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; and they also have a text dataset. This dataset is already pre-processed, like the words are converted to integers and the most used words are in the top 10000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I also have to use a pre-processed dataset like them? If yes, does it have to be like the dataset from TensorFlow? And which pages could help me to implement that kind of program? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My steps would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;find datasets&lt;/li&gt;&#xA;&lt;li&gt;preprocess them if needed&lt;/li&gt;&#xA;&lt;li&gt;feed them in the neural network&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="24093" LastEditorUserId="2444" LastEditDate="2019-06-28T11:18:58.743" LastActivityDate="2019-06-28T11:18:58.743" Title="Do I need to use a pre-processed dataset to classify comments?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12811" PostTypeId="2" ParentId="9319" CreationDate="2019-06-12T12:54:54.827" Score="0" Body="&lt;p&gt;For example, suppose I want to approximate the function&#xA;mashine learning is used to approximate an UNKNOWN function. If you do want to do this with NN, you will have a regression task with small full connected network with like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input is 2 values of range 0 ..1&lt;/li&gt;&#xA;&lt;li&gt;a few hidden layers, just start trying with one, of size like 3-6 neurons , activation  sigmoid or tanh&lt;/li&gt;&#xA;&lt;li&gt;need a non-linear function here to approx non linear sin * sin, whatever &lt;/li&gt;&#xA;&lt;li&gt;last layer with no activation function just making a difference from... known value f(x,y)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-12T12:54:54.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12812" PostTypeId="2" ParentId="12810" CreationDate="2019-06-12T12:59:15.247" Score="2" Body="&lt;p&gt;Here's a list of some of the best python libraries for natural language processing.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.nltk.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Toolkit (&lt;code&gt;nltk&lt;/code&gt;)&lt;/a&gt;&#xA;Covers all the basic functions and NLP tools such as tokenization etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://textblob.readthedocs.io/en/dev/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TextBolb&lt;/a&gt;&#xA;This is a good library of beginners, it provides the &lt;code&gt;nltk&lt;/code&gt; toolkit in a simplified format.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://spacy.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spacy&lt;/a&gt;&#xA;It is an advanced library and can be used in production code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can preprocess textual data in a number of ways. It depends on the type of task at hand and the size of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your question, I think you are referring to converting the words to a vector form &lt;code&gt;word2vec&lt;/code&gt;. &lt;a href=&quot;https://code.google.com/archive/p/word2vec/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a massive word2vect list from Google.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also have a look at preprocessing techniques in NLP such as &lt;code&gt;tf-idf&lt;/code&gt; etc.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-12T12:59:15.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12813" PostTypeId="1" CreationDate="2019-06-12T14:28:23.273" Score="5" ViewCount="227" Body="&lt;p&gt;I want to use Reinforcement Learning to optimize the distribution of energy for a peak shaving problem given by a thermodynamical simulation. However, I am not sure how to proceed as the action space is the only thing that really matters, in this sense:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The action space is a &lt;span class=&quot;math-container&quot;&gt;$288 \times 66$&lt;/span&gt;  matrix of &lt;strong&gt;real numbers&lt;/strong&gt; between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. The output of the simulation and therefore my reward depend solely on the distribution of this matrix.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The state space is therefore absent, as the only thing that matters is the matrix on which I have total control. At this stage of the simulation, no other variables are taken into consideration.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am not sure if this problem falls into the tabular RL or it requires approximation. In this case, I was thinking about using a policy gradient algorithm for figuring out the best distribution of the &lt;span class=&quot;math-container&quot;&gt;$288 \times 66$&lt;/span&gt; matrix. However, I do not know how to behave with the &amp;quot;absence&amp;quot; of the state space. Instead of a tuple &lt;span class=&quot;math-container&quot;&gt;$\langle s,a,r,s' \rangle$&lt;/span&gt;, I would just have &lt;span class=&quot;math-container&quot;&gt;$\langle a, r \rangle$&lt;/span&gt;, is this even an RL-approachable problem? If not, how can I reshape it to make it solvable with RL techniques?&lt;/p&gt;&#xA;" OwnerUserId="23638" LastEditorUserId="2444" LastEditDate="2020-11-30T01:46:02.523" LastActivityDate="2020-11-30T01:46:02.523" Title="It is possible to solve a problem with continuous action spaces and no states with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;multi-armed-bandit&gt;&lt;state-spaces&gt;&lt;continuous-action-spaces&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12814" PostTypeId="2" ParentId="12805" CreationDate="2019-06-12T14:37:35.903" Score="0" Body="&lt;p&gt;at this time, as open source - NOT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i guess:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;for a decision make we need a broad input layer/-s of data flows, &lt;/li&gt;&#xA;&lt;li&gt;we need a 20 000-200 000 layers of neural networks or more complex and dynamic architectures&lt;/li&gt;&#xA;&lt;li&gt;we need a deep research of date-time influence for historical data flow&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;what we have at this time:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;only sensors - opencv and object recognition, nlp-tagging, data predicting&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;so, sensors isn't AI, sensors and machine learning is previous experience. it is not ready for the change analysis. &lt;/p&gt;&#xA;" OwnerUserId="15757" LastActivityDate="2019-06-12T14:37:35.903" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12815" PostTypeId="1" CreationDate="2019-06-12T15:31:33.680" Score="4" ViewCount="108" Body="&lt;p&gt;I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is as follows: Let's say we have a story (in the text) that has 10 characters. Can we define them, their characteristics, whole sentences they said, and then analyze emotions within those sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, is it possible to generate an audio version of the story where: the text, in general, is narrated by one voice, each individual character's sentences are read in a different voice generated specifically for that character. Finally is it possible to make the tones of the characters voices change depending on the emotions detected in their sentences?&lt;/p&gt;&#xA;" OwnerUserId="26390" LastEditorUserId="12853" LastEditDate="2019-06-13T07:34:33.423" LastActivityDate="2019-11-10T10:02:20.280" Title="How can I build an AI with NLP that read stories" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12817" PostTypeId="1" CreationDate="2019-06-12T21:17:30.437" Score="3" ViewCount="177" Body="&lt;p&gt;I am trying to build a Deep Q-Network (DQN) agent that can learn to play the game &lt;a href=&quot;https://play2048.co/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2048&lt;/a&gt;. I am orientating myself on other programs and articles that are based on the game snake and it worked well (&lt;a href=&quot;https://towardsdatascience.com/how-to-teach-an-ai-to-play-games-deep-reinforcement-learning-28f9b920440a&quot; rel=&quot;nofollow noreferrer&quot;&gt;specifically this one&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As input state, I am only using the grid with the tiles as numpy array, and as a reward, I use (newScore-oldScore-1) to penalize moves that do not give any points at all. I know that this might not be optimal, as one might as well reward staying alive for as long as possible, but it should be okay for the first step, right? Nevertheless, I am not getting any good results whatsoever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to tweak the model layout, the number of neurons and layers, optimizer, gamma, learning rates, rewards, &lt;em&gt;etc.&lt;/em&gt;. I also tried ending the game after 5 moves and to optimize just for those first five moves but no matter what I do, I don't get any noticeable improvement. I've run it for thousands of games and it just doesn't get better. In fact, sometimes I get worse results than a completely random algorithm, as sometimes it just returns the same output for any input and gets stuck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is, if I am doing anything fundamentally wrong? Do I just have a small stupid mistake somewhere? Is this the wrong approach completely? (I know the game could probably be solved pretty easily without AI, but it seemed like a little fun project) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Jupyter notebook can be seen here &lt;a href=&quot;https://github.com/Drukob/2048DQN&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;. Sorry for the poor code quality, I'm still a beginner and I know I need to start making documentation even for fun little projects...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Drukob&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit:&#xA;some code snippets:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input is formatted as a 1,16 numpy array, also tried normalizing the values or using only 1 and 0 for occupied and empty cells, but that did not help either. Which is why I assumed it's maybe more of a conceptual problem?&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def get_board(self):&#xA;        grid = self.driver.execute_script(&quot;return myGM.grid.cells;&quot;)&#xA;        mygrid = []&#xA;        for line in grid:&#xA;            a = [x['value'] if x != None else 0 for x in line]&#xA;            #a = [1 if x != None else 0 for x in line]&#xA;            mygrid.append(a)&#xA;        return np.array(mygrid).reshape(1,16)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is an index of {0,3}, representing the actions up, down, left or right and it's just the value with the highes prediction score.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prediction = agent.model.predict(old_state)&#xA;predicted_move = np.argmax(prediction)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tried a lot of different model architectures, but settled for a simpler network now, as I have read that unnecessary complex structures are often a problem and unneeded. However, I couldn't find any reliable source for a method, how to get the optimal layout except for experimenting, so I'd be happy to have some more suggestions there.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = models.Sequential()&#xA;        model.add(Dense(16, activation='relu', input_dim=16))&#xA;        #model.add(Dropout(0.15))&#xA;        #model.add(Dense(50, activation='relu'))&#xA;        #model.add(Dropout(0.15))&#xA;        model.add(Dense(20, activation='relu'))&#xA;        #model.add(Dropout(0.15))&#xA;        #model.add(Dense(30, input_dim=16, activation='relu'))&#xA;        #model.add(Dropout(0.15))&#xA;        #model.add(Dense(30, activation='relu'))&#xA;        #model.add(Dropout(0.15))&#xA;        #model.add(Dense(8, activation='relu'))&#xA;        #model.add(Dropout(0.15))&#xA;        model.add(Dense(4, activation='linear'))&#xA;        opt = Adam(lr=self.learning_rate)&#xA;        model.compile(loss='mse', optimizer=opt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26399" LastEditorUserId="1671" LastEditDate="2019-10-15T19:20:25.957" LastActivityDate="2019-10-15T19:20:25.957" Title="Deep Q-Network (DQN) to learn the game 2048" Tags="&lt;python&gt;&lt;game-ai&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12818" PostTypeId="5" CreationDate="2019-06-12T21:47:23.223" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Economics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Economics&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-12T21:47:23.223" LastActivityDate="2019-06-12T21:47:23.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12819" PostTypeId="4" CreationDate="2019-06-12T21:47:23.223" Score="0" Body="For questions involving economics and economic decision-making.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-12T21:47:23.223" LastActivityDate="2019-06-12T21:47:23.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12820" PostTypeId="2" ParentId="12805" CreationDate="2019-06-12T22:10:18.160" Score="1" Body="&lt;p&gt;Consider managing a memory structure as an economic function.  (Where to put, and how to manage, the resources constituted by data.)  This is something computers can do better and faster than any human.  The reason is that the system in which the economic decisions are being made is fully defined.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Routing of packages is a similar, economic function that computers do much better than humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These functions haven't been handled by Machine Learning in the past, but, soon after the AlphaGo milestone, Google found an economic application for Machine Learning.  &lt;a href=&quot;https://www.wired.co.uk/article/google-deepmind-data-centres-efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's DeepMind trains AI to cut its energy bills by 40% (Wired)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it's entirely context dependent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the model increases in complexity and nuanced, utility will be reduced. (In the former case it's a time and space issue related to computational complexity, and in the latter case, often a function of &lt;a href=&quot;https://en.wikipedia.org/wiki/Complete_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;incomplete information&lt;/a&gt; or inability to define parameters.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as the sophistication of the machine learning algorithms increases, and the models continue to be refined, the algorithms will get better and better at managing intractability and incomplete information.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-14T17:23:45.967" LastActivityDate="2019-06-14T17:23:45.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12822" PostTypeId="1" CreationDate="2019-06-13T04:42:55.667" Score="1" ViewCount="54" Body="&lt;p&gt;So, I've read about seq2seq for time-series and it seemed really promising, but when I went to implement it, all the tutorial I've found use the correct output as input to the decoder phase during training, instead of using the actual prediction made by the cell before it. Is there a reason to why not do the later? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been using the tutorial from here: &lt;a href=&quot;https://weiminwang.blog/2017/09/29/multivariate-time-series-forecast-using-seq2seq-in-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://weiminwang.blog/2017/09/29/multivariate-time-series-forecast-using-seq2seq-in-tensorflow/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But all the other tutorials that I've found followed the same principle.&lt;/p&gt;&#xA;" OwnerUserId="26406" LastEditorUserId="12853" LastEditDate="2019-06-13T18:31:41.673" LastActivityDate="2021-03-04T20:04:10.353" Title="Why feeding the correct output as input during training of seq2seq models?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12823" PostTypeId="2" ParentId="12822" CreationDate="2019-06-13T06:23:45.173" Score="0" Body="&lt;p&gt;The reason why you would use the ground truth as input to the decoder is to lean the testing distribution. From what I've seen so far most of the papers are using scheduled sampling (Bengio et al.). Meaning that you will introduce a new term p which will be the probability of the network to feed as input its own prediction. Initially p will be very small so that the network will use the ground truth, but later the more iterations passes by, the probability will decrease and the network will start using its own predictions. &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-06-13T06:23:45.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12827" PostTypeId="1" AcceptedAnswerId="12830" CreationDate="2019-06-13T08:10:58.897" Score="1" ViewCount="213" Body="&lt;p&gt;I am new in RL and I am trying to understand why do we need all these hyperparameters.&#xA;Can somebody explain me why we use them and what are the best values to use for them?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;total_episodes = 50000        # Total episodes&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;total_test_episodes = 100     # Total test episodes&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;max_steps = 99                # Max steps per episode&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;learning_rate = 0.7           # Learning rate&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;gamma = 0.618                 # Discounting rate&lt;/p&gt;&#xA;  &#xA;  &lt;h1&gt;Exploration parameters&lt;/h1&gt;&#xA;  &#xA;  &lt;p&gt;epsilon = 1.0                 # Exploration rate&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;max_epsilon = 1.0             # Exploration probability at start&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;min_epsilon = 0.01            # Minimum exploration probability&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;decay_rate = 0.01             # Exponential decay rate&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am currently working on taxi_v2 problem from GYM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link: &lt;a href=&quot;https://learndatasci.com/tutorials/reinforcement-q-learning-scratch-python-openai-gym/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://learndatasci.com/tutorials/reinforcement-q-learning-scratch-python-openai-gym/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26410" LastEditorUserId="2444" LastEditDate="2019-06-13T14:49:18.813" LastActivityDate="2019-06-14T09:38:12.373" Title="Why are we using all hyperparameters in RL?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12828" PostTypeId="2" ParentId="12815" CreationDate="2019-06-13T08:48:23.007" Score="2" Body="&lt;p&gt;This is quite an ambitious project, and IMHO well beyond the scope of what a single individual can do (within a reasonable time span) at present. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to first analyse the story text to identify the characters. This can already be quite a tricky task, as pronouns and other reference expressions are generally used to make a text less monotonous. If a character is referred to by name, say &lt;em&gt;Jane&lt;/em&gt;, then you can assume that a follow-up &lt;em&gt;the young woman&lt;/em&gt; will refer to her and not a male character mentioned in the same paragraph. But what about &lt;em&gt;the young scientist&lt;/em&gt;? Such expressions can be very opaque, and you'd need a lot of world-knowledge to decode them correctly, as they can refer to any distinctive attribute of the character.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Identifying speech is a bit easier, unless you're talking about indirect speech. &lt;em&gt;Jane was thinking aloud. She wasn't going to be able to do that. It was too hard.&lt;/em&gt; -- is that speech or not? Compare to &lt;em&gt;Jane was thinking aloud: &quot;I am not going to be able to do that. It is too hard.&quot;&lt;/em&gt;, which is the direct speech equivalent. Also, unless you're dealing with a play, most of the text will probably not be speech. For the audio version you will probably only want to deal with direct speech, which is usually (but not always) indicated by quote marks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analysing emotions seems to be comparatively easy if you have reached this stage, though if it is just based on keywords in the speech it probably won't be very accurate. If you can assign any descriptive statements to characters, that might be more successful, though by no means trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generating the text as audio should be straight forward. Most operating systems nowadays have speech synthesis integrated, and you can generally choose different voices, so if your text is marked up properly with which voice should speak which part it would be trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarise: The NLP part is the hardest bit of it. As has been mentioned in the comments already, I don't think it's a problem that machine learning can help with, and I would stick to traditional methods of parsing the text into a structural representation and then applying rules to identify the bits you are interested in. The recognition of emotion might be a subtask that is suitable for ML, but in the past I have only applied pattern matching to similar tasks, so I can't really say much about that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my own experience in text analysis I would think that you might be able to get decent results with a few simple heuristics, but those will likely fail when it becomes a bit more complicated. A lot hinges on the type of story: children's fairy tales might be easier than &lt;em&gt;War and Peace&lt;/em&gt; in that respect.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-06-13T08:48:23.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12830" PostTypeId="2" ParentId="12827" CreationDate="2019-06-13T09:35:01.510" Score="1" Body="&lt;p&gt;In RL, there are episodic and non-episodic tasks (or problems). In episodic tasks, each episode proceeds in time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, most games are episodic tasks. For instance, in a football championship (e.g. Premier League), each football match during the whole season is an episode. In this example, each minute (or second) of a football match can be considered a time step of the the episode (that is, the football match).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameter &lt;code&gt;total_episodes&lt;/code&gt; thus specifies the number of episodes of your RL episodic problem. Similarly, &lt;code&gt;max_steps&lt;/code&gt; specifies the maximum number of time steps per episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we also need &lt;code&gt;total_test_episodes&lt;/code&gt;? In machine learning, when building a model, there are usually two phases: the training phase and the test phases. In the training phase, you use the training dataset to learn the parameters of the model. In the test phase, you test the performance (e.g. the total return, in the case of RL) of the model. Hence, &lt;code&gt;total_test_episodes&lt;/code&gt; is used to specify this hyper-parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In deep RL, a (deep) neural network (NN) is used to represent either a value function or a policy (which is also a function). In machine learning, neural networks are usually trained using an &lt;a href=&quot;https://ai.stackexchange.com/a/8985/2444&quot;&gt;optimisation method, like gradient descent (GD) and the back-propagation (BP), which is used to compute the gradient of the objective function with respect to the parameters of the model&lt;/a&gt;. In the case of deep RL, the parameters of the NN that represents either the value function or policy are also learned using a similar approach. In this context, the learning rate is a hyper-parameter that determines the &quot;strength&quot; of the update step of the optimisation algorithm. More concretely, in the case of GD, the update step is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{\theta}_{n+1} \gets \mathbf{\theta}_{n} - \alpha \nabla f(\mathbf{\theta} _{n})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}_{n+1}$&lt;/span&gt; is a vector containing the parameter of your model (in this case, a NN), &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate and &lt;span class=&quot;math-container&quot;&gt;$\nabla f(\mathbf{\theta} _{n})$&lt;/span&gt; is the gradient of the objective function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. Hence, &lt;code&gt;learning_rate&lt;/code&gt; specifies the value of &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, &lt;code&gt;learning_rate&lt;/code&gt; can actually specify the value of the learning rate of your RL algorithm. For example, in the case &lt;a href=&quot;https://ai.stackexchange.com/a/10560/2444&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning&lt;/a&gt;, the learning rate, which we can denote by &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, also specifies the &quot;strength&quot; of the update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the &lt;code&gt;gamma&lt;/code&gt; is a hyper-parameter of your RL algorithm. For example, in the case &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, the parameter &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (the discount factor) which determines the contribution of the estimate of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value of the next state (while taking the greedy action from that state) to the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value that is being currently updated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is an off-policy RL algorithm, which means that it uses a behaviour policy that is possibly different than the policy it tries to estimate. The usual behaviour policy is the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy (with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; a random action is taken in a certain state and with probability &lt;span class=&quot;math-container&quot;&gt;$1 - \epsilon$&lt;/span&gt; the greedy action is taken). The parameter &lt;code&gt;epsilon&lt;/code&gt; specifies this hyper-parameter. In this context, an initial &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; that specifies the exploration rate is provided. As the training progresses, the estimate of the optimal value function should become more accurate. In that case, you want to explore less and follow more your estimate of the optimal value function, so &lt;code&gt;min_epsilon&lt;/code&gt; is used to specify the lowest &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameter &lt;code&gt;decay_rate&lt;/code&gt; is used to specify the value of the decay rate. Have a look at &lt;a href=&quot;https://stats.stackexchange.com/a/31334/82135&quot;&gt;https://stats.stackexchange.com/a/31334/82135&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-14T09:38:12.373" LastActivityDate="2019-06-14T09:38:12.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12835" PostTypeId="1" CreationDate="2019-06-14T06:36:16.327" Score="5" ViewCount="99" Body="&lt;p&gt;I have a dataset which contains 4000k rows and 6 columns. The goal is to predict travel time demand of a taxi. I have read many articles regarding how to approach the problem. So, every writer tell his own way. The thing which I have concluded from all my readings is that I have to use multiple algorithms and check the accuracy of each one. Then I can ensemble them by averaging or any other approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithms will be best for my problem accuracy-wise? Some links to code will be helpful for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently only have training set of data. After I work on it, it will be evaluated on any testing set by my professor. So, what should I do now? Either split data I have into my own testing and training set or separately generate dummy data as a testing set?&lt;/p&gt;&#xA;" OwnerUserId="26429" LastEditorUserId="2444" LastEditDate="2019-06-14T09:07:18.347" LastActivityDate="2019-06-14T09:52:52.900" Title="What approach should I take to model forecasting problem in machine learning?" Tags="&lt;machine-learning&gt;&lt;models&gt;&lt;forecasting&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12837" PostTypeId="2" ParentId="12813" CreationDate="2019-06-14T07:21:59.717" Score="3" Body="&lt;p&gt;A stateless RL problem can be reduced to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiarmed Bandit (MAB)&lt;/a&gt; problem. In such a scenario, taking an action will not change the state of the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, this is the setting of a conventional MAB problem: at each time step, the agent selects an action to either perform an exploration or exploitation move. It then records the reward of the taken action and updates its estimation/expectation of the usefulness of the action. Then, repeats the procedure (selection, observing, updating). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To chose between exploration and exploitation moves, MAB agents adopt a strategy. The simplest one would probably be &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy which agent chooses the most rewarding actions most of the time (1-&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; probability) or randomly selects an action (&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; probability). &lt;/p&gt;&#xA;" OwnerUserId="12853" LastActivityDate="2019-06-14T07:21:59.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12840" PostTypeId="2" ParentId="12835" CreationDate="2019-06-14T07:54:38.453" Score="3" Body="&lt;p&gt;In general, this type of problem is called a &lt;strong&gt;regression&lt;/strong&gt; problem since the target variable (i.e. travel time) can take any value in a continuous domain. In theory, you can use any regression algorithms (a subset &lt;strong&gt;supervised learning&lt;/strong&gt; techniques) to solve this problem. Some of the most popular ones are &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear regression&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;K-nearest neighbor&lt;/a&gt; (regressor), and &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you observed already, different algorithms result in (sometimes significantly) different results. Also, the parameter configurations (e.g., number of hidden layers in Neural Networks) can make a big difference. Sometimes, ensembling different models can be helpful, but in general, you should try to avoid overfitting (when your model is more complex than your data such that it memorizes the training set instead of learning it!). That may result in a very good performance on your training set but perform very poorly on your professor's testing set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would do is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;exploring the dataset to see what are the contributing factor in travel time (any correlation between the columns).&lt;/li&gt;&#xA;&lt;li&gt;cleaning and preprocessing my dataset (duplicates, null values, outliers)&lt;/li&gt;&#xA;&lt;li&gt;reshaping my dataset if needed (normalizing some columns, merging or splitting columns)&lt;/li&gt;&#xA;&lt;li&gt;dividing my dataset to training and evaluation subsets (so I train on one part and test on the other part to avoid overfitting)&lt;/li&gt;&#xA;&lt;li&gt;choosing a simple baseline, applying and measuring the accuracy metrics.&lt;/li&gt;&#xA;&lt;li&gt;trying to fine tune parameters of my baseline or trying other more advanced techniques.&lt;/li&gt;&#xA;&lt;li&gt;comparing the results and improving any part of the pipeline when necessary (more/less cleaning, parameter tuning, ensembling).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12853" LastEditorUserId="2444" LastEditDate="2019-06-14T09:12:24.380" LastActivityDate="2019-06-14T09:12:24.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12841" PostTypeId="1" AcceptedAnswerId="12845" CreationDate="2019-06-14T08:22:42.710" Score="2" ViewCount="308" Body="&lt;p&gt;iPhone X allows you to look at the TrueDepth camera and reports 52 facial blendshapes like how much your eye is opened, how much your jaw is opened, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to do something similar with other cameras (not TrueDepth), what are my alternative methods? Currently, I just use a simple ConvNet which takes in an image and predict 52 sigmoid values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think could be the underlying technology behind ARKit Face Tracking?&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-06-14T10:40:54.117" Title="How does ARKit's Facial Tracking work?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12842" PostTypeId="2" ParentId="12803" CreationDate="2019-06-14T08:59:08.460" Score="0" Body="&lt;p&gt;To get a better understanding, let's compare a couple of games &lt;a href=&quot;https://en.wikipedia.org/wiki/Dota_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DOTA2&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Overwatch_(video_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overwatch&lt;/a&gt;. At first, both seen similar in terms of &lt;code&gt;multiplayer&lt;/code&gt; and &lt;code&gt;strategy&lt;/code&gt;, but there is a significant difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have mentioned it in the question. &lt;strong&gt;reaction time&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though both the games require strategy and understanding of the surroundings and team play, FPS games are heavily dependent on &lt;code&gt;reaction time&lt;/code&gt; and &lt;code&gt;accuracy&lt;/code&gt;. As much as you try, it's next to impossible to reach the reaction times and accuracy of computer agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In games such as DOTA2, the game is more dependent towards &lt;code&gt;strategy&lt;/code&gt; and how the characters interact with each other (Strengths and Weakness).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it doesn't make sense to train an AI based agent to play FPS games, because the optimal solution is already present and there is no way that AI can reach those levels. It's called &lt;strong&gt;AIM BOT&lt;/strong&gt;. As good as you can get at strategy and planning, team work and reaction times, you can almost never beat it. It's next to 100% accurate and the opponent is basically dead as soon as he's in sight.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-14T08:59:08.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12845" PostTypeId="2" ParentId="12841" CreationDate="2019-06-14T10:40:54.117" Score="3" Body="&lt;p&gt;Facial classification and tracking is easier compared to tracking any other motion. This is due to the fact that the face have a large number of easily identifiable features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Facial tracking is an additional layer on top of facial detection. Facial detection works by finding characteristics such as the cheekbones, chin, nose, eyes etc. These features are easy to detect as they have very specific properties. These points are found with the help of shadows and brightness, for example, the nose and cheekbones are highlighted, whereas the the eyes and lips have a shadows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using these feature points it is possible to create a mesh to construct the face. This has been a classical way to detect and classify faces. The TrueDepth camera helps to make a better structure by providing depth with the help of an IR sensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/40R2b.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/40R2b.jpg&quot; alt=&quot;Facial features and Facial mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Facial tracking makes use of this mesh to study and track changes in the facial structure. For example when you smile, it will detect the cheekbone points changing, similarly when you open your mouth, your lips will move apart.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-14T10:40:54.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12850" PostTypeId="1" CreationDate="2019-06-14T19:01:21.977" Score="1" ViewCount="151" Body="&lt;p&gt;How do I do object detection (or identify the location of an object) if there is only one kind of object, and they are more of less similar size, but the picture does not look like standard scenes (it is detection of drops on a substrate in microscopic images)?  Which software is good for it?&lt;/p&gt;&#xA;" OwnerUserId="5852" LastEditorUserId="2444" LastEditDate="2019-06-14T19:31:39.497" LastActivityDate="2021-03-06T15:08:50.940" Title="How do I perform object detection if there is only one type of object?" Tags="&lt;deep-learning&gt;&lt;object-detection&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12851" PostTypeId="5" CreationDate="2019-06-14T19:02:18.960" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Probability&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-14T19:02:18.960" LastActivityDate="2019-06-14T19:02:18.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12852" PostTypeId="4" CreationDate="2019-06-14T19:02:18.960" Score="0" Body="For question involving probability as related to AI methods.  (This tag is for general usage.  Feel free to utilize in conjunction with the &quot;math&quot; and more specific probability tags.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-14T19:02:18.960" LastActivityDate="2019-06-14T19:02:18.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12853" PostTypeId="2" ParentId="7690" CreationDate="2019-06-14T20:03:51.563" Score="1" Body="&lt;p&gt;This is a great question. Connecting the AI knowledge to the application is a difficult task and its not easy to &lt;strong&gt;test and perfect&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few points regarding autonomous driving algorithms that one should keep in mind.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A lot of data is required before actually applying a model to the real world.&lt;/li&gt;&#xA;&lt;li&gt;The control algorithms are designed for specific tasks, there is no &lt;strong&gt;master&lt;/strong&gt; algorithm that solves the issue.&lt;/li&gt;&#xA;&lt;li&gt;Time is very important, decisions have to be made in real time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The image segmentation you have mentioned in the question is rarely used in current autonomous technologies. The main issue is that &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The model is slow and requires massive computing resources. Due to this is cannot help in real time decision making.&lt;/li&gt;&#xA;&lt;li&gt;A very important parameter &lt;strong&gt;depth&lt;/strong&gt; is missing from the information. This is of utmost importance in autonomous driving technology. A 2D image cannot provide this parameter.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the current technology use &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A range of sensors (LIDAR, Ultrasonic, Cameras etc.) that provide readings 100+ times a second.&lt;/li&gt;&#xA;&lt;li&gt;A lot of the computer vision algorithms and control algorithms are hard coded. There are set parameters within which decisions are made.&lt;/li&gt;&#xA;&lt;li&gt;The control algorithms are designed to work with each set of sensors and the mechanical components of the car.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To get a better look at the way these algorithms are used, you can have a look at &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=FLi_IQgCxbo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanley vision system&lt;/a&gt; winner of the DARPA grand challenge (Stanford)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=tiwVMrTLUWg&quot; rel=&quot;nofollow noreferrer&quot;&gt;How a driverless car sees the road&lt;/a&gt; TEDx&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-14T20:03:51.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12854" PostTypeId="2" ParentId="12850" CreationDate="2019-06-14T21:19:17.440" Score="0" Body="&lt;p&gt;The hardest part is image annotation: here the difference between object recognition and object detection becomes important. If you just want to answer the question &quot;does this image contain object X?&quot;, then you just need to provide as many images that contain object X as possible, together with as many images  that don't contain object X (but are otherwise similar). However if you want to answer the question &quot;Where exactly object X is located in this image?&quot; then you will need to manually provide a bounding box for each instance of object X in each image. Obviously, the second scenario is a lot more labor intensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you've done this part, train either a binary image classifier (typically this will be a convolutional neural network) on your annotated images (split them into train and test partitions), or an object detector (googling &quot;custom object detection&quot; produces lots of code examples how to train it (e.g. &lt;a href=&quot;https://towardsdatascience.com/tutorial-build-an-object-detection-system-using-yolo-9a930513643a&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/tutorial-build-an-object-detection-system-using-yolo-9a930513643a&lt;/a&gt; start with Step 2B).&lt;/p&gt;&#xA;" OwnerUserId="26373" LastEditorUserId="26373" LastEditDate="2019-06-14T21:24:48.067" LastActivityDate="2019-06-14T21:24:48.067" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12856" PostTypeId="2" ParentId="12226" CreationDate="2019-06-14T22:16:44.177" Score="1" Body="&lt;p&gt;I had the same problem where the reward kept decreasing and started to search for answers in the forum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I let the model trained while I search. As the model trained, the reward started to increase. You can see the tensorboard &lt;strong&gt;graph for rewards in validation time&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kk11A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kk11A.png&quot; alt=&quot;Tensorboard graph for validation rewards&quot;&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fall continued until around 100k~ steps and did not change a lot for 250k~ steps. &#xA;After 350k~ th step, it slowly started to increase. &lt;strong&gt;Without knowing your number of steps trained&lt;/strong&gt;, I would suggest training for more steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I read about this (Reward first decreasing and then increasing) in an RL paper, if I find it I will mention it here.&lt;/p&gt;&#xA;" OwnerUserId="26447" LastActivityDate="2019-06-14T22:16:44.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12857" PostTypeId="1" CreationDate="2019-06-15T00:34:08.070" Score="3" ViewCount="63" Body="&lt;p&gt;My data is stock data with features such as stocks' closing prices.I am curious to know &lt;strong&gt;if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' features.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Date  Ticker  Open  High  Low  Close  Interest  Unemp. &#xA;  1/1    AMZN    75    78     73   76     0.015     0.03&#xA;  1/2    AMZN    76    77     72   72     0.016     0.03&#xA;  1/3    AMZN    72    78     76   77     0.013     0.03&#xA;  ...    ...     ...   ...    ...  ...    ...       ...&#xA;  1/1    AAPL    104   105    102  102    0.015     0.03&#xA;  1/2    AAPL    102   107    104  105    0.016     0.03&#xA;  1/3    AAPL    105   115    110  111    0.013     0.03&#xA;  ...    ...     ...   ...    ...  ...    ...      ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see from the table above, daily prices of AMZN and AAPL are different but the Interest and Unemployment rates are the same. Can I feed the data to my neural network like the table above?&lt;/p&gt;&#xA;&lt;p&gt;In other words, can I put the individual stocks' information besides the environment feature such as interest rates?&lt;/p&gt;&#xA;" OwnerUserId="26037" LastEditorUserId="26037" LastEditDate="2021-01-07T21:30:22.680" LastActivityDate="2021-02-06T22:03:55.447" Title="How to make a distinction between item feature and environment feature?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;data-science&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12859" PostTypeId="2" ParentId="5177" CreationDate="2019-06-15T05:02:28.493" Score="-1" Body="&lt;p&gt;I am correct in thinking that you would like an intuitive feel or explanation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After thinking about this for a little bit, I thought I would approach this problem from understanding the K-means algorithm. As you may know the k-means algorithm uses some straight forward calculations. The computer calculates the distance from one point to a center. If you can measure the distance of a point (x1) to center 1 and from point (x1) to center 2 then you are golden. Because next all you do is choose the minimum argument or distance and that becomes the group that the point will belong to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in our example of K-means, let us say a human picks the number of groupings, k=2. Meaning the human thinks there are only two types of 'things'. Therefore all the things, items or whatever that are closer to center 1 are called '1'. The items that are closer to center 2 are now called '2'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our K-means case it chooses the distance which is the shortest. It chooses the minimum value of distance and assigns the future points to '1' or '2' (for example). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, what if you don't want to use distance as your measure but instead you just learned about Gaussian curves and what to call your items as being from Gaussian curve '1' or '2'. We could do that, right? Well, if I used Gaussian curves as my measuring criteria, I could use probability instead of distance (as we did in K-means). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's say we started to look at our points in terms of what is the probability of point 1 being related to center '1'. Let's say we knew the mean and standard deviation of our center '1'. Well we could then use probability to use as our new measuring stick now. So if point x1 is 1 standard deviation away from center '1' its p-value would be (let's say) (P of 1 std.dev.) = ~0.35. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what if we also knew the mean and variance of the second center, '2'. We could then calculate the Probability of our point x1 being part of center '2'.  Let's say that point x1 is 4 standard deviations from center '2'. It's p-value = 0.0002. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore I would choose the MAXIMUM p-values to use to assign my points to my centers, NOT the Minimum that I used in K-means. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really K-means is similar to E.M. except it uses distance not p-values.&#xA;Does that help?&#xA;P.S. Forgive the grammar mistakes, etc I am doing this late for me, lol.&lt;/p&gt;&#xA;" OwnerUserId="12550" LastActivityDate="2019-06-15T05:02:28.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12860" PostTypeId="1" CreationDate="2019-06-15T01:39:04.783" Score="2" ViewCount="67" Body="&lt;p&gt;I have physical model prediction data as well as actual data. From this I can calculate the error of each prediction data point through simple subtraction. I am hoping to train a neural network to be able to assign an error to the input of the physical model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current plan is to normalize the error of each data point and assign it as a label to each model input. So the NN would be trained (and validated)on a 1000 data points with the associated error as a label. Once the model is trained I would be able to input one data point and the output of the neural network would be a single class, that is the error. The purpose this would serve would be to tune the physical prediction model. Would this kind of architecture work? If so, would you recommend a feedforward or RNN? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="26451" OwnerDisplayName="Arthur Shune" LastActivityDate="2019-06-15T05:25:52.713" Title="Neural Network for Error Prediction of a Physics Model?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12863" PostTypeId="2" ParentId="12850" CreationDate="2019-06-15T10:34:45.887" Score="-1" Body="&lt;p&gt;Such applications are usually dealt with by using some image processing and computer vision techniques. As far as I remember, we used Correlation in Digital Signal Processing using MATLAB last semester. The method was simple:&lt;br&gt;&#xA;&lt;strong&gt;METHOD 1 : Using Cross Convolution&lt;/strong&gt;&lt;br&gt;&#xA;- Import the image you want to use&lt;br&gt;&#xA;- Select the object in the image that you want to detect (a drop in your case) using its coordinates&lt;br&gt;&#xA;- After some processing, search the image for the drop, using the selected image as reference. &#xA;(So, MATLAB actually calculates cross convolution of each coordinates of the actual image with that of the 'drop' image)&lt;br&gt;&#xA;&lt;a href=&quot;https://in.mathworks.com/help/images/registering-an-image-using-normalized-cross-correlation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Read this for detailed method&lt;/a&gt;&lt;br&gt;&#xA;This requires you to obtain MATLAB and a normal PC&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can manage to get better hardware or more time (as slower processing demands more time), you can opt for method 2&lt;br&gt;&#xA;&lt;strong&gt;METHOD 2 : Using Deep Learning&lt;/strong&gt;&lt;br&gt;&#xA;- Download loads of images of drops or the object you are looking for&lt;br&gt;&#xA;- Train the model and create a Convolution Neural Network (Hardware and time intensive)&lt;br&gt;&#xA;- Validate the network and test it&lt;br&gt;&#xA;(this method is more widely used and is a offspring of Artificial Intelligence/ Machine Learning . It provides a far more accurate result at the cost of the images, hardware and time you spend on training the model. Nowadays, people use Python for the same purpose as it saves you from the new syntax that you might face using MATLAB. Both are more or the same equally efficient)&lt;br&gt;&#xA;&lt;a href=&quot;https://in.mathworks.com/help/vision/examples/image-category-classification-using-deep-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Read detailed methodology here&lt;/a&gt; and &lt;a href=&quot;https://in.mathworks.com/help/vision/examples/object-detection-using-deep-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="26452" LastActivityDate="2019-06-15T10:34:45.887" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12866" PostTypeId="1" CreationDate="2019-06-15T13:50:26.107" Score="1" ViewCount="32" Body="&lt;h2&gt; Question&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why such a big difference between my &lt;strong&gt;'Train loss'&lt;/strong&gt; and &lt;strong&gt;'Validation loss'&lt;/strong&gt; as shown in the picture below? Is it a signal that my codes are wrong and my trained network is wrong as well?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sQzJQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sQzJQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Some of my codes are as follows:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;DATA_SPLIT_PCT = 0.2&#xA;&#xA;timesteps =  5&#xA;n_features =  20&#xA;&#xA;epochs = 100&#xA;batch = 32&#xA;lr = 0.0001&#xA;&#xA;lstm_autoencoder = Sequential([&#xA;    # Encoder&#xA;    LSTM(8, activation='relu', input_shape=(timesteps, n_features), return_sequences=True),&#xA;    LSTM(4, activation='relu', return_sequences=False),&#xA;    RepeatVector(timesteps),&#xA;&#xA;    # Decoder&#xA;    LSTM(4, activation='relu', return_sequences=True),&#xA;    LSTM(8, activation='relu', return_sequences=True)&#xA;    TimeDistributed(Dense(n_features)),&#xA;])&#xA;&#xA;adam = optimizers.Adam(lr)&#xA;lstm_autoencoder.compile(loss='mse', optimizer=adam)&#xA;&#xA;for stock in stock_list:  # 500 stocks in stock_list&#xA;    lstm_autoencoder_history = lstm_autoencoder.fit(X_train_dict[ticker], X_train_dict[ticker], &#xA;                                                    epochs=epochs, &#xA;                                                    batch_size=batch, &#xA;                                                    validation_data=(X_valid_dict[ticker], X_valid_dict[ticker]),&#xA;                                                    verbose=False).history&#xA;&#xA;plt.plot(lstm_autoencoder_history['loss'], linewidth=2, label='Train')&#xA;plt.plot(lstm_autoencoder_history['val_loss'], linewidth=2, label='Valid')&#xA;plt.show()&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;I used the for loop to feed my data into &lt;code&gt;lstm_autoencoder&lt;/code&gt; network. In the dictionary variable &lt;code&gt;stock_list&lt;/code&gt;, there are 500 stock names such as 'AAPL'. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I plotted &lt;code&gt;lstm_autoencoder_history['loss']&lt;/code&gt; and &lt;code&gt;lstm_autoencoder_history['val_loss']&lt;/code&gt; and it is weird because usually validation loss is higher than train loss.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am curious to know why my plot has smaller amount of validation loss. For your information, I used &lt;strong&gt;Keras&lt;/strong&gt; as my deep learning framework. And since I used Keras, I thought this library would handle the different proportion of training set size and validation set size by averaging the errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26037" LastActivityDate="2019-06-15T13:50:26.107" Title="Why such a big difference in number between training error and validation error?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;keras&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12867" PostTypeId="5" CreationDate="2019-06-15T14:36:12.010" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-15T14:36:12.010" LastActivityDate="2019-06-15T14:36:12.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12868" PostTypeId="4" CreationDate="2019-06-15T14:36:12.010" Score="0" Body="For questions related to explainable artificial intelligence (XAI), also known as interpretable AI, which refers to AI techniques that can be trusted and easily understood by humans, which are particularly relevant in areas like healthcare or self-driving cars. There are several concepts related to XAI, such as accountability, fairness, and transparency." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-13T23:30:04.330" LastActivityDate="2019-08-13T23:30:04.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12870" PostTypeId="1" AcceptedAnswerId="24138" CreationDate="2019-06-15T23:07:47.740" Score="13" ViewCount="1357" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;Explainable artificial intelligence (XAI)&lt;/a&gt; is concerned with the development of techniques that can enhance the interpretability, accountability, and transparency of artificial intelligence and, in particular, machine learning algorithms and models, especially black-box ones, such as artificial neural networks, so that these can also be adopted in areas, like healthcare, where the interpretability and understanding of the results (e.g. classifications) are required.&lt;/p&gt;&#xA;&lt;p&gt;Which XAI techniques are there?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T13:24:40.267" LastActivityDate="2021-04-27T10:51:20.617" Title="Which explainable artificial intelligence techniques are there?" Tags="&lt;reference-request&gt;&lt;ethics&gt;&lt;explainable-ai&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="12871" PostTypeId="2" ParentId="12870" CreationDate="2019-06-15T23:07:47.740" Score="5" Body="&lt;p&gt;There are a few &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/agnostic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;XAI techniques&lt;/a&gt; that are (partially) agnostic to the model to be interpreted&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Layer-wise relevance propagation (LRP), introduced in &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation&lt;/a&gt; (2015)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Local Interpretable Model-agnostic Explanations (LIME), introduced in &lt;a href=&quot;https://arxiv.org/pdf/1602.04938v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Why Should I Trust You?&quot; Explaining the Predictions of Any Classifier&lt;/a&gt; (2016)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Model Agnostic Contrastive Explanations Method (MACEM), introduced in &lt;a href=&quot;https://arxiv.org/pdf/1906.00117.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model Agnostic Contrastive Explanations for Structured Data&lt;/a&gt; (2019)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are also ML models that are not considered black boxes and that are thus &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/simple.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;more interpretable than black boxes&lt;/a&gt;, such as&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;linear models (e.g. linear regression)&lt;/li&gt;&#xA;&lt;li&gt;decision trees&lt;/li&gt;&#xA;&lt;li&gt;naive Bayes (and, in general, Bayesian networks)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For a more complete list of such techniques and models, have a look at the online book &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning: A Guide for Making Black Box Models Explainable&lt;/a&gt;, by Christoph Molnar, which attempts to categorise and present the main XAI techniques.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-30T15:07:40.397" LastActivityDate="2019-06-30T15:07:40.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12872" PostTypeId="1" AcceptedAnswerId="13060" CreationDate="2019-06-15T23:16:55.513" Score="4" ViewCount="173" Body="&lt;p&gt;In the proofs for the original GAN paper, it is written:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$∫_x p_{data}(x) \log D(x)dx+∫_zp(z)\log(1−D(G(z)))dz&#xA;=∫_xp_{data}(x)\log D(x)+p_G(x) \log(1−D(x))dx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen some explanations asserting that the following equality is the key to understanding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E_{z∼p_z(z)}log(1−D(G(z)))=E_{x∼p_G(x)}log(1−D(x))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a consequence of the LOTUS theorem and &lt;span class=&quot;math-container&quot;&gt;$x_g = g(z)$&lt;/span&gt;. Why is &lt;span class=&quot;math-container&quot;&gt;$x_g = g(z)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="26313" LastEditorUserId="8068" LastEditDate="2019-06-17T19:45:26.633" LastActivityDate="2019-06-25T19:28:17.233" Title="How is G(z) related to x in GAN proof?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;generative-adversarial-networks&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12873" PostTypeId="1" CreationDate="2019-06-16T05:30:29.827" Score="1" ViewCount="74" Body="&lt;p&gt;I am trying to implement a DNN to optimize a set of 7 parameters that are used in a robot swarm simulator on the ARGoS platform. the program is a compiled C++ executable that reads the parameters from an XML file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previous experiments to optimize these parameters used GA techniques to find optimal parameter settings for specified distributions, but this technique may not be finding global maximums. We would like to try using a DNN with automated hyper-parameter tuning, specifically Vizier (HyperTune on the Google Cloud Platform) but I am having some trouble designing how this problem is solved with tools like TensorFlow. Using a GA to select for an optimized parameter set is straightforward, but how to use a DNN on TensorFlow, and take advantage of Vizier/HyperTune to tune this DNN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have installed the sim in a docker container that can launch the sim with a parameter set via command-line arguments, and can return the resulting score of the trial, but how I can connect this into a DNN program to optimize the input parameters by maximizing the resulting score is unclear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a novice ML programmer so please pardon erroneous assumptions or methods here. Any help is greatly appreciated! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;note&lt;/em&gt; the reason I am looking to use TF would be to take advantage of the native GCP hyper-parameter tuning tools, which appear to be tailored for TF. &lt;/p&gt;&#xA;" OwnerUserId="26465" LastActivityDate="2019-06-16T05:30:29.827" Title="How to use TensorFlow with hyperparameter tuning to optimize parameters for a robot simulator" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;models&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12874" PostTypeId="1" CreationDate="2019-06-16T06:11:26.947" Score="2" ViewCount="204" Body="&lt;p&gt;&lt;em&gt;Artificial general intelligence&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;is the intelligence of a machine that could successfully perform any intellectual task that a human being can&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would an artificial general intelligence have to be Turing complete?&lt;/p&gt;&#xA;" OwnerUserId="17541" LastEditorUserId="2444" LastEditDate="2021-01-25T23:42:20.287" LastActivityDate="2021-01-25T23:42:20.287" Title="Would an artificial general intelligence have to be Turing complete?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;theory-of-computation&gt;&lt;computational-theory-of-mind&gt;&lt;turing-machine&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12875" PostTypeId="1" AcceptedAnswerId="12879" CreationDate="2019-06-16T09:48:45.293" Score="2" ViewCount="55" Body="&lt;p&gt;I'm majoring in pure linguistics (not computational), and I don't have any basic knowledge regarding computational science or mathematics. But I happen to take &quot;Automatic Speech Recognition&quot; course in my graduate school and struggling with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a question regarding getting the formula for a component of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Forward_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;forward algorithm&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\alpha_t(j) = \sum_{i=1}^{N} P(q_{t-1} = i, q_t=j, o_1^{t-1}, o^t|\lambda)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is a hidden state, &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; is a given observation, and &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; contains transition probability, emission probability and the start/end state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the Markov assumption (the current state is only dependent upon the one right before it) assumed here? I thought so, because it contains &lt;span class=&quot;math-container&quot;&gt;$q_{t-1}=i$&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$q_{t-2}=k$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$q_{t-3}=l$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26470" LastEditorUserId="2444" LastEditDate="2019-06-16T13:10:44.030" LastActivityDate="2019-06-16T13:39:49.683" Title="Is the Markov property assumed in the forward algorithm?" Tags="&lt;algorithm&gt;&lt;math&gt;&lt;markov-property&gt;&lt;hidden-markov-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12876" PostTypeId="5" CreationDate="2019-06-16T13:09:48.797" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hidden_Markov_model&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-16T13:09:48.797" LastActivityDate="2019-06-16T13:09:48.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12877" PostTypeId="4" CreationDate="2019-06-16T13:09:48.797" Score="0" Body="For questions related to the hidden Markov model and related algorithms (such as the forward algorithm)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-16T13:09:48.797" LastActivityDate="2019-06-16T13:09:48.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12878" PostTypeId="1" CreationDate="2019-06-16T13:16:04.567" Score="3" ViewCount="64" Body="&lt;h2&gt;Problem Statement&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I've built a classifier to classify a dataset consisting of n samples and four classes of data. To this end, I've used pre-trained VGG-19, pre-trained Alexnet and even LeNet (with cross-entropy loss). However, I just changed the softmax layer's architecture and placed just four neurons for that (because my dataset includes just four classes). Since the dataset classes have a striking resemblance to each other, this classifier was unable to classify them and I was forced to use other methods.&#xA;During the training section, after some epochs, loss decreased from approximately 7 to approximately 1.2, but there were no changes in accuracy and it was frozen on 25% (random precision). In the best epochs, the accuracy just reached near 27% but it was completely unstable.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;How is it justifiable? If loss reduction means model improvement, why doesn't accuracy increase? How is it possible to the loss decreases near 6 points (approximately from 7 to 1) but nothing happens to accuracy at all?&lt;/p&gt;&#xA;" OwnerUserId="26472" LastEditorUserId="2444" LastEditDate="2020-04-13T16:13:39.800" LastActivityDate="2020-04-13T19:27:24.330" Title="If loss reduction means model improvement, why doesn't accuracy increase?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;training&gt;&lt;loss-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12879" PostTypeId="2" ParentId="12875" CreationDate="2019-06-16T13:39:49.683" Score="0" Body="&lt;p&gt;In general, to formally state that the Markov property holds, you need to have &lt;span class=&quot;math-container&quot;&gt;$P(&#xA;x_t \mid x_{t-1:1}) = P(x_t \mid x_{t-1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you cannot conclude &lt;em&gt;only&lt;/em&gt; from &lt;span class=&quot;math-container&quot;&gt;$P(q_{t-1} = i, q_t=j, o_1^{t-1}, o^t|\lambda)$&lt;/span&gt; that the Markov property holds, because  &lt;span class=&quot;math-container&quot;&gt;$P(q_{t-1} = i, q_t=j, o_1^{t-1}, o^t|\lambda)$&lt;/span&gt; is the &lt;strong&gt;joint&lt;/strong&gt; probability of &lt;span class=&quot;math-container&quot;&gt;$q_{t-1} = i, q_t=j, o_1^{t-1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$o^t$&lt;/span&gt; &lt;em&gt;given&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, the theory of &lt;a href=&quot;http://jedlik.phy.bme.hu/~gerjanos/HMM/node5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;hidden Markov model often assumes that a few properties (including the Markov property) hold&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;the Markov property: &lt;span class=&quot;math-container&quot;&gt;$$P(q_{t+1} \mid q_{t}) = P(q_{t+1} \mid q_{t:1}),$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$q_t$&lt;/span&gt; is the hidden state at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q_{t:1} = q_t, q_{t-1}, \dots, q_1$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the stationarity property: &lt;span class=&quot;math-container&quot;&gt;$$P(q_{t_1 + 1} \mid q_{t_1}) = P(q_{t_2 + 1} \mid q_{t_2}),$$&lt;/span&gt; for any time step &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_2$&lt;/span&gt;. In other words, the &lt;em&gt;state transition probabilities are independent of the actual time at which the transitions takes place&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the output independence property: &lt;span class=&quot;math-container&quot;&gt;$$P(o^{T:1} \mid q_{T:1}) = \prod_{t=1}^T P(o^t \mid q_t , \lambda)$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$o^{T:1} = o^T, o^{T-1}, \dots, o^1$&lt;/span&gt;. In other words, this is the assumption that the output at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$o^t$&lt;/span&gt;, is independent of the outputs at previous time steps.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-16T13:39:49.683" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12881" PostTypeId="1" CreationDate="2019-06-16T14:27:47.113" Score="2" ViewCount="1562" Body="&lt;p&gt;What is the use of softmax function? Why was it used at the end of fully connected layer in convolution neural network?&lt;/p&gt;&#xA;" OwnerUserId="20551" LastEditorUserId="2444" LastEditDate="2019-06-16T14:35:05.233" LastActivityDate="2019-06-16T15:47:52.667" Title="What is the use of softmax function in a CNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12882" PostTypeId="2" ParentId="12881" CreationDate="2019-06-16T14:47:46.653" Score="1" Body="&lt;p&gt;The main purpose of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;softmax function&lt;/a&gt; is to transform the (unnormalised) output of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; units (which is e.g. represented as a vector of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; elements) of a fully-connected layer to a probability distribution (a normalised output), which is often represented as a vector of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; elements, each of which is between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (a probability) and the sum of all these elements is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (a probability distribution). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a classification task, the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th element of the vector produced by the softmax function corresponds to the probability of the input of the network of belonging to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th class (e.g. a dog).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-16T14:47:46.653" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12883" PostTypeId="2" ParentId="12881" CreationDate="2019-06-16T15:47:52.667" Score="1" Body="&lt;p&gt;In short the softmax function helps with multi-classification i.e &lt;em&gt;output of more than one of two possibilities&lt;/em&gt;.&#xA;It works well this the categorical cross entropy.&lt;/p&gt;&#xA;" OwnerUserId="15465" LastActivityDate="2019-06-16T15:47:52.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12884" PostTypeId="2" ParentId="10288" CreationDate="2019-06-16T19:43:48.663" Score="0" Body="&lt;p&gt;Nim is a simple game and it's really simple to build a bot that gives the optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The correct move is to leave an odd number of piles of size 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So when it comes to training an ANN to play a game, there are some this to keep in mind.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fixed play area (This is taken care of as there are only 3 stacks)&lt;/li&gt;&#xA;&lt;li&gt;Providing input to the model so it knows it's current state (You can use a simple array of integers &lt;code&gt;[3, 4, 1]&lt;/code&gt; denoting the current state of the stacks)&lt;/li&gt;&#xA;&lt;li&gt;Providing feedback. An important step to guide the network in the right direction (You already have an optimal bot to do this)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now that you can easily cover all the requirements, it's pretty simple to teach the model.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Input&lt;/strong&gt; - Current state of the model &lt;code&gt;[3, 4, 5]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Output&lt;/strong&gt; - Move that model will make &lt;code&gt;[2, 0, 0]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Final output&lt;/strong&gt; - Here you will have to add another layer with a custom function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the important part, to direct the model. Check if the move made is &lt;code&gt;GOOD&lt;/code&gt;, &lt;code&gt;BAD&lt;/code&gt; or &lt;code&gt;ILLEGAL&lt;/code&gt; and assign a custom output to the same. For example&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;GOOD&lt;/strong&gt; return&lt;code&gt;1&lt;/code&gt; if it matches one of the optimal moves.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;BAD&lt;/strong&gt; return &lt;code&gt;0&lt;/code&gt; if it's not the optimal move and&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;ILLEGAL&lt;/strong&gt; return &lt;code&gt;-1&lt;/code&gt; if the move is not allowed.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Another thing to keep in mind is that this might require a slightly larger network as it as to learn complex functions that are not linearly separable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need to train that model by playing it against itself as you already have the optimal solution. Playing against itself is required only when you are trying to achieve something that better than the current best agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check this out, it's pretty interesting &lt;a href=&quot;https://towardsdatascience.com/today-im-going-to-talk-about-a-small-practical-example-of-using-neural-networks-training-one-to-6b2cbd6efdb3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural network to play snake&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-16T19:43:48.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12885" PostTypeId="2" ParentId="12878" CreationDate="2019-06-16T23:31:08.330" Score="1" Body="&lt;p&gt;&lt;em&gt;Loss reduction means model improvement,&lt;/em&gt; it does not in the wrong setup, wher random choise produces least loss. So it is some critical setup error. What classes do you have?&#xA;I got also thet recently experimenting with an encoder with too narrow coding layer - it just EQUILIZES the output with average values cause this state has minimum loss.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-16T23:31:08.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12886" PostTypeId="1" CreationDate="2019-06-17T05:39:46.347" Score="1" ViewCount="35" Body="&lt;p&gt;I want to develop a regression model using the artificial neural network. For developing such a model I use standardised ( z-score normalised ) data.&lt;/p&gt;&#xA;&lt;p&gt;given below is the sample data set. Here MAX is the real data But I am using MAX-ZS (these values are continues)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jRyeI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jRyeI.png&quot; alt=&quot;Sample Data Set&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So my question is while developing the model do I have to perform further normalization such as Min-Max scaling on my training data?&#xA;Any Kind of help is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="24006" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-06-17T10:21:05.173" Title="Further Normalization of Standardized data - ANN" Tags="&lt;neural-networks&gt;&lt;datasets&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12887" PostTypeId="1" CreationDate="2019-06-17T05:55:15.560" Score="1" ViewCount="14" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/robinsloan/rnn-writer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/robinsloan/rnn-writer&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I preface this by saying I do not know much about this topic, only that I have an intense interest in it, so I'm hoping I can make my questions as clear as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This writing assistant was released with full code and instructions on how to make it work.  I was halfway through this process when I was told, basically, that this could not work on a Windows PC.  Torch, specifically, either doesn't work on PC, or doesn't work very well, and I am hesitant to continue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, does anyone know if that's true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is true, is it theoretically possible to recreate this in a different way that will work on Windows PCs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, has anyone ever done it before, or know how to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a way to make Torch work on PCs, is someone willing to tell me how?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I apologize if this isn't meant for this specific area of discussion.  I just don't know where else to go with my questions that I will get any kind of helpful responses.  Even if it's just telling me where I can go for more pertinent responses, that would still be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help you might be willing to give.  Please let me know if I need to clarify anything.&lt;/p&gt;&#xA;" OwnerUserId="15659" LastActivityDate="2019-06-17T05:55:15.560" Title="Questions regarding rrn-writer by Robin Sloane?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12888" PostTypeId="1" AcceptedAnswerId="12894" CreationDate="2019-06-17T06:38:07.053" Score="4" ViewCount="87" Body="&lt;p&gt;I've read that for MDPs the state transition function &lt;span class=&quot;math-container&quot;&gt;$P_a(s, s')$&lt;/span&gt; is a probability. This seems strange to me for modeling because most environments (like video games) are deterministic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'd like to assert that most systems we work with are deterministic given enough information in the state (i.e. in a video game, if you had the random number seed, you could predict 'rolls', and then everything else follows game logic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my guess for why would MDP state transitions are probabilities is because the state given to the MDP is typically a subset (i.e. from feature engineering) of total information available. That, and of course to model non-deterministic systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="26487" LastEditorUserId="2444" LastEditDate="2019-06-17T12:28:26.970" LastActivityDate="2019-06-17T12:28:26.970" Title="Why are state transitions in MDPs probabilistic rather than deterministic?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12889" PostTypeId="1" CreationDate="2019-06-17T06:49:41.203" Score="2" ViewCount="122" Body="&lt;p&gt;In the book &lt;a href=&quot;https://www.goodreads.com/book/show/2054765.PROLOG&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog Programming for Artificial Intelligence&lt;/a&gt;, a large and intricate chapter (chapter 14) is dedicated to Expert Systems. In these systems, a  knowledge-database is represented through facts and rules in a declarative manner, and then we use the PROLOG inference engine to derive statements and decisions.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering: are there any examples of &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;expert systems&lt;/a&gt; that represent knowledge through a standard Relational Database approach and then extract facts through SQL queries? Is there any research in this area? If not, why is a rule-based approach preferred?&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2021-01-27T21:19:38.077" LastActivityDate="2021-01-27T21:25:00.407" Title="Are Relational DBs and SQL used in Expert Systems?" Tags="&lt;reference-request&gt;&lt;knowledge-representation&gt;&lt;expert-system&gt;&lt;rule-based-systems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12891" PostTypeId="2" ParentId="12114" CreationDate="2019-06-17T08:31:55.150" Score="0" Body="&lt;p&gt;I found this article from James Manyika (MacKinsey) helpful: &lt;a href=&quot;https://www.mckinsey.com/featured-insights/artificial-intelligence/applying-artificial-intelligence-for-social-good&quot; rel=&quot;nofollow noreferrer&quot;&gt;Applying artificial intelligence for social good&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25362" LastActivityDate="2019-06-17T08:31:55.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12892" PostTypeId="1" CreationDate="2019-06-17T08:57:08.333" Score="4" ViewCount="51" Body="&lt;p&gt;So I'm finding AutoML to be pretty interesting but I'm still learning how it all works. I've played with the incredibly broken AutoKeras and got some decent results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, if you are using a NN to optimize the architecture of another network, why not take it another layer deeper and use another network to find the optimum architecture for your Parent network with a grand-parent network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem doesn't necessarily need to expand exponentially as the grand-parent network could do few-shot training sessions on the parent network which itself is doing few or one-shot training. &lt;/p&gt;&#xA;" OwnerUserId="17541" LastEditorUserId="17541" LastEditDate="2019-06-17T09:04:39.710" LastActivityDate="2019-06-17T21:57:43.963" Title="Why not go another layer deeper with Auto-AutoML?" Tags="&lt;ai-design&gt;&lt;automation&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12893" PostTypeId="2" ParentId="12886" CreationDate="2019-06-17T10:21:05.173" Score="1" Body="&lt;p&gt;Data scaling or normalization is a process of making model data in a standard format so that the training is improved, accurate, and faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you just have to scale the data once. Doesn't matter what scaler you are using. Just make sure to initialize the scaler with the &lt;code&gt;training data&lt;/code&gt; and then use the same parameters to scale the &lt;code&gt;test data&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781788397872/5/ch05lvl1sec62/scaling-of-data-in-neural-network-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;z-score normalized&lt;/a&gt; data (&lt;code&gt;MAX-ZS&lt;/code&gt;) can be used directly to train the network.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-17T10:21:05.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12894" PostTypeId="2" ParentId="12888" CreationDate="2019-06-17T10:38:53.270" Score="2" Body="&lt;p&gt;Your understanding is right!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a probabilistic transition function allows the model to explore a bigger search space before making a decision. One of the most important use cases of MDP is in POS tagging in NLP using a Hidden Markov Model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of a deterministic model, search space is limited by the number of transitions and hence at each step, a definite decision is made. This does not take into regard the possibility of relationship between the previous states, rather only deals with the current and next state. These model are good for solving a certain range of tasks like decision trees etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it come to tasks such as weather prediction, there is significance to the historical weather data. In such cases, we cannot make use of a deterministic approach. You always predict the &lt;code&gt;chance&lt;/code&gt; of rainfall etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QJclo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QJclo.jpg&quot; alt=&quot;Weather prediction&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This example can also be extended to predict the weather for future days&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-17T10:38:53.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12896" PostTypeId="1" CreationDate="2019-06-17T13:15:43.810" Score="3" ViewCount="5374" Body="&lt;p&gt;I am planning to use BERT embeddings in the LSTM embedding layer instead of the usual Word2vec/Glove Embeddings. What are the possible ways to do that? &lt;/p&gt;&#xA;" OwnerUserId="26115" LastEditorUserId="2444" LastEditDate="2019-11-01T02:27:38.613" LastActivityDate="2019-11-01T02:27:38.613" Title="Adding BERT embeddings in LSTM embedding layer" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;word-embedding&gt;&lt;long-short-term-memory&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12897" PostTypeId="1" CreationDate="2019-06-17T15:00:47.750" Score="2" ViewCount="33" Body="&lt;p&gt;Do you have any advice, what architecture of neural network is the best for following task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let input be some (complex function), the neural network gains a flow of its values, so I guess there will be some kind of RNN or CNN? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is classifier like &lt;strong&gt;is the function same or not&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the neural network thinks, that the input is still the same function, the output is &lt;strong&gt;0&lt;/strong&gt;. &lt;/li&gt;&#xA;&lt;li&gt;If the input function changes, the output will be &lt;strong&gt;1&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The input function is of course not one value or simple math function (what will be trivial) but may be really sophisticated. So the neural network learns abstraction about same and different over any complex flow ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you approach to that task ?&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="26384" LastEditDate="2019-06-17T15:10:06.127" LastActivityDate="2019-06-17T19:30:02.713" Title="Changes in flow detection neural network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;pattern-recognition&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12898" PostTypeId="2" ParentId="12896" CreationDate="2019-06-17T15:02:39.240" Score="3" Body="&lt;p&gt;Instead of using the &lt;code&gt;Embedding()&lt;/code&gt; layer directly, you can create a new &lt;code&gt;bertEmbedding()&lt;/code&gt; layer and use it instead.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Sample code&#xA;# Model architecture&#xA;&#xA;# Custom BERT layer&#xA;bert_output = BertLayer(n_fine_tune_layers=10)(bert_inputs)&#xA;&#xA;# Build the rest of the classifier &#xA;dense = tf.keras.layers.Dense(256, activation='relu')(bert_output)&#xA;pred = tf.keras.layers.Dense(1, activation='sigmoid')(dense)&#xA;&#xA;model = tf.keras.models.Model(inputs=bert_inputs, outputs=pred)&#xA;model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;model.fit(...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://towardsdatascience.com/bert-in-keras-with-tensorflow-hub-76bcbc9417b&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; will walk you through the entire process of creating the custom BERT layer along with example code. Give it a read.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-17T15:02:39.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12900" PostTypeId="1" CreationDate="2019-06-17T18:51:26.927" Score="1" ViewCount="69" Body="&lt;p&gt;An important property of a reinforcement learning problem is whether the environment of the agent is static, which means that nothing changes if the agent remains inactive. Different learning methods assume in varying degrees that the environment is static. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I check if and (if so) where in the Monte Carlo algorithm, temporal difference learning (TD(0)), the Dyna-Q architecture, and R-Max a static environment is implicitly assumed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I modify the relevant learning methods so that they can in principle adapt to changing environments? (It can be assumed that &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is sufficiently large.)&lt;/p&gt;&#xA;" OwnerUserId="26494" LastEditorUserId="2444" LastEditDate="2019-06-17T19:43:46.423" LastActivityDate="2019-06-17T19:43:46.423" Title="How do I know if the assumption of a static environment is made?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;environment&gt;&lt;dyna&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12901" PostTypeId="2" ParentId="12897" CreationDate="2019-06-17T19:30:02.713" Score="1" Body="&lt;p&gt;A network is able to fit to a certain function over several iteration while training. Now you want the model to be able to detect a change in a list of inputs from the function. This is not possible without first training the model on some data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you want to use a simple function&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Sample function&#xA;f(x) = x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Say you create inputs for function using sets of 3 integers for &lt;code&gt;x&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Data&#xA;f1 = [[1, 2, 3],&#xA;      [4, 5, 6],&#xA;      ...      ]&#xA;&#xA;# Some random values&#xA;f2 = [[1, 4, 19],&#xA;      [16, 35, 36],&#xA;      ...      ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now if you use data &lt;code&gt;f1&lt;/code&gt; labelled as &lt;code&gt;0&lt;/code&gt; and data &lt;code&gt;f2&lt;/code&gt; labelled as &lt;code&gt;1&lt;/code&gt;, in the best case the ANN will only learn to differentiate between data from function &lt;code&gt;f1&lt;/code&gt; and some other data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To detect change, first the model has to fit to a certain function, this requires it to be trained over a number of epochs. Then it will be able to detect if the values don't match the function, but such a model will only be able to differentiate a single function.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-17T19:30:02.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12903" PostTypeId="5" CreationDate="2019-06-17T19:41:54.840" Score="0" Body="&lt;p&gt;For more info, have a look e.g. at &lt;a href=&quot;https://www.cs.cmu.edu/afs/cs/project/jair/pub/volume4/kaelbling96a-html/node29.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.cmu.edu/afs/cs/project/jair/pub/volume4/kaelbling96a-html/node29.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-17T19:41:54.840" LastActivityDate="2019-06-17T19:41:54.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12904" PostTypeId="4" CreationDate="2019-06-17T19:41:54.840" Score="0" Body="For questions related to the reinforcement learning &quot;dyna&quot; architecture." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-17T19:41:54.840" LastActivityDate="2019-06-17T19:41:54.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12907" PostTypeId="2" ParentId="12892" CreationDate="2019-06-17T21:57:43.963" Score="1" Body="&lt;p&gt;Logically it is possible, but you will just end up complicating the entire task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim of AutoML is to provide a drop in solution to the customers. To do this, a trained network decides and generates the model architecture. This is done so that anyone with basic experience is able to integrate the solution into their systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the complicated architectures and networks require experienced data scientist to build, train and deploy. To overcome this bottleneck and make ML accessible to all, AutoML is being developed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So adding another grand-parent network to optimize the autoML network will just complicate the task in terms to computation time and hyper parameter optimization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case we decide to add another network, now the researchers must look at this network and tune it in regards to the inner network and the model both. This means more work and no direct way to understand how the hyper parameters are affecting the final results.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-17T21:57:43.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12908" PostTypeId="1" AcceptedAnswerId="12909" CreationDate="2019-06-18T00:35:55.820" Score="14" ViewCount="10120" Body="&lt;p&gt;In reinforcement learning (RL), the &lt;em&gt;credit assignment problem&lt;/em&gt; (CAP) seems to be an important problem. What is the CAP? Why is it relevant to RL?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-20T10:41:21.480" LastActivityDate="2020-06-20T10:41:21.480" Title="What is the credit assignment problem?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;credit-assignment-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12909" PostTypeId="2" ParentId="12908" CreationDate="2019-06-18T00:35:55.820" Score="21" Body="&lt;p&gt;In reinforcement learning (RL), an agent interacts with an environment in time steps. On each time step, the agent takes an &lt;em&gt;action&lt;/em&gt; in a certain state and the environment emits a &lt;em&gt;percept&lt;/em&gt; or perception, which is composed of a &lt;em&gt;reward&lt;/em&gt; and an &lt;em&gt;observation&lt;/em&gt;, which, in the case of fully-observable MDPs, is the next state (of the environment and the agent). The goal of the agent is to maximise the reward in the long run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (temporal) credit assignment problem (CAP) (discussed in &lt;a href=&quot;https://courses.csail.mit.edu/6.803/pdf/steps.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Steps Toward Artificial Intelligence&lt;/em&gt;&lt;/a&gt; by Marvin Minsky in 1961) is the problem of determining the actions that lead to a certain outcome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in football, at each second, each football player takes an action. In this context, an action can e.g. be &quot;pass the ball&quot;, &quot;dribbe&quot;, &quot;run&quot; or &quot;shoot the ball&quot;. At the end of the football match, the outcome can either be a victory, a loss or a tie. After the match, the coach talks to the players and analyses the match and the performance of each player. He discusses the contribution of each player to the result of the match. The problem of determinig the contribution of each player to the result of the match is the (temporal) credit assignment problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How is this related to RL?&lt;/em&gt; In order to maximise the reward in the long run, the agent needs to determine which actions will lead to such outcome, which is essentially the temporal CAP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why is it called credit assignment problem?&lt;/em&gt; In this context, the word credit is a synonym for value. In RL, an action that leads to a higher final cumulative reward should have more value (so more &quot;credit&quot; should be assigned to it) than an action that leads to a lower final reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why is the CAP relevant to RL?&lt;/em&gt; Most RL agents attempt to solve the CAP. For example, a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning agent attempts to learn an (optimal) value function. To do so, it needs to determine the actions that will lead to the highest value in each state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few variations of the (temporal) CAP problem. For example, the &lt;a href=&quot;http://pages.cs.wisc.edu/~finton/what-rl.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;structural CAP&lt;/em&gt;&lt;/a&gt;, that is, the problem of assigning credit to each structural component (which might contribute to the final outcome) of the system.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-18T00:35:55.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12911" PostTypeId="1" AcceptedAnswerId="12913" CreationDate="2019-06-18T02:17:55.950" Score="6" ViewCount="331" Body="&lt;p&gt;All sources I can find provide a similar explanation to each phase. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Selection Phase, we start at the root and choose child nodes until reaching a leaf.  Once the leaf is reached (assuming the game is not terminated), we enter the Expansion Phase. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Expansion Phase, we expand any number of child nodes and select one of the expanded nodes. Then, we enter the Play-Out Phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my confusion. If we choose to only expand a single node, the nodes that were not expanded will never be considered in future selections as we only select child nodes until a leaf is reached during the Selection Phase. Is this correct? If not, what am I misunderstanding about the Selection Phase?&lt;/p&gt;&#xA;" OwnerUserId="26498" LastEditorUserId="2444" LastEditDate="2019-11-19T20:11:09.380" LastActivityDate="2019-11-19T20:11:09.380" Title="When does the selection phase exactly end in MCTS?" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12913" PostTypeId="2" ParentId="12911" CreationDate="2019-06-18T06:39:47.347" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If we choose to only expand a single node, the nodes that were not expanded will never be considered in future selections as we only select child nodes until a leaf is reached during the Selection Phase. Is this correct? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No this is not correct.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If not, what am I misunderstanding about the Selection Phase?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The selection phase does not end only when you reach a node that has no expanded nodes. It ends when you reach a node that has &lt;em&gt;any&lt;/em&gt; unexpanded nodes. At which point you typically pick one or more nodes you have not yet expanded at that point in the tree, expand them and collect one or more rollout results for them. Variations are possible such as choosing to whether to expand or continue selecting stochastically, or expanding all child nodes at the same time using value estimates to initialise them - the latter is what AlphaZero does.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-06-18T08:32:42.257" LastActivityDate="2019-06-18T08:32:42.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12914" PostTypeId="1" CreationDate="2019-06-18T07:03:29.710" Score="10" ViewCount="353" Body="&lt;p&gt;What are some interesting myths of Artificial Intelligence and what are the facts behind them?&lt;/p&gt;&#xA;" OwnerUserId="26351" LastEditorUserId="2444" LastEditDate="2019-06-18T16:09:23.313" LastActivityDate="2019-06-28T14:17:00.453" Title="What are the common myths associated with Artificial Intelligence?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;mythology-of-ai&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12915" PostTypeId="1" CreationDate="2019-06-18T07:35:45.343" Score="4" ViewCount="360" Body="&lt;p&gt;I was looking at two papers &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1710.10903.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Attention Networks&lt;/a&gt; (GAT) by &lt;em&gt;Petar Veličković&lt;/em&gt; and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1803.07294.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs&lt;/a&gt; by &lt;em&gt;Jiani Zhang&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm trying to implement the second paper and I'm having some troubles understanding the differences between GAT and GaAN. By looking at equation 1 in GaAN paper I can see only two differences from GAT. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first difference is that they are doing a dot product with the initial feature map and &lt;/li&gt;&#xA;&lt;li&gt;Have another fully connected layer to project the result. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there something else that I'm missing?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2019-06-18T12:40:53.540" LastActivityDate="2020-10-06T08:07:27.207" Title="What is the difference between GAT and GaAN?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;attention&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12916" PostTypeId="1" CreationDate="2019-06-18T10:20:21.833" Score="2" ViewCount="65" Body="&lt;p&gt;I have a couple of questions and I was wondering if you could answer them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a bunch of images of the cars, side view only. I want to train the model with those images. My objects of interest are &lt;strong&gt;3 types of trucks&lt;/strong&gt; that have different trailers. I Rarely see two target object at one image (maybe 1 or 2 in every 1000 images). However, I do see other types of cars that I do not want to detect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you think I should tackle this problem as a detection task or classification task? (for example, should I consider multi-label classification or omit those pictures)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I also include other vehicles that I do not want to detect in my training dataset? let say I do not assign bounding box to them but include them in training dataset just to make the system robust. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I trained YOLO with 200 images, sometimes the trained model confused and detected the wrong object that is not in any of classes, this happens when training with 2000 images per class? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this due to a small number of dataset or it is because of not including those images with no bounding boxes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="12853" LastEditDate="2019-06-21T22:20:33.217" LastActivityDate="2019-06-21T22:20:33.217" Title="How to choose our data set wisely?" Tags="&lt;classification&gt;&lt;datasets&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12918" PostTypeId="1" CreationDate="2019-06-18T11:40:03.810" Score="3" ViewCount="88" Body="&lt;p&gt;In Chapter 15 of Russel and Norvig's &lt;em&gt;Artificial Intelligence -- A Modern Approach (Third Edition)&lt;/em&gt;, they describe three basic tasks in temporal inference: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Filtering, &lt;/li&gt;&#xA;&lt;li&gt;Likelihood, and &lt;/li&gt;&#xA;&lt;li&gt;Finding the Most Likely Sequence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question is on the difference between the first and third task.  Finding the Most Likely Sequence determines, given evidences &lt;span class=&quot;math-container&quot;&gt;$e_1,\dots,e_n$&lt;/span&gt;, the most likely sequence of states &lt;span class=&quot;math-container&quot;&gt;$S_1,\dots,S_n$&lt;/span&gt;.  This is done using the Viterbi algorithm.  On the other hand, Filtering provides the probability distribution on states after seeing &lt;span class=&quot;math-container&quot;&gt;$e_1,\dots,e_n$&lt;/span&gt;.  You could then pick the state with the highest probability, call it &lt;span class=&quot;math-container&quot;&gt;$S'_n$&lt;/span&gt;. I am guessing that &lt;span class=&quot;math-container&quot;&gt;$S'_n$&lt;/span&gt; should always be equal to &lt;span class=&quot;math-container&quot;&gt;$S_n$&lt;/span&gt;.  Likewise, you can already do the same after any prefix &lt;span class=&quot;math-container&quot;&gt;$e_1,\dots,e_i$&lt;/span&gt;, again picking the most likely state &lt;span class=&quot;math-container&quot;&gt;$S'_i$&lt;/span&gt;.  I would love to have a simple example where &lt;span class=&quot;math-container&quot;&gt;$S'_1,\dots,S'_n$&lt;/span&gt; is not equal to the sequence &lt;span class=&quot;math-container&quot;&gt;$S_1,\dots,S_n$&lt;/span&gt; produced by the  Viterbi algorithm.&lt;/p&gt;&#xA;" OwnerUserId="26508" LastEditorUserId="26508" LastEditDate="2019-06-18T20:28:30.480" LastActivityDate="2019-07-19T01:02:00.213" Title="Viterbi versus filtering" Tags="&lt;probability&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12919" PostTypeId="1" CreationDate="2019-06-18T13:24:57.493" Score="1" ViewCount="83" Body="&lt;p&gt;The input in word2vec is known word (spellings), each tagged by its ID.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;But if you process real text, there can be not only dictionary words but also proper nouns like human names, trade marks, file names , etc, how to make an input for that? &lt;/li&gt;&#xA;&lt;li&gt;Is you consider some input where items are variables, like the meaning of input would be x = something, and after some time you acces to x value and define some other stuff with it. That would be format for this input, and will this approach work at all?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-18T13:39:24.290" Title="How to handle proper names or variable names in word2vec?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12920" PostTypeId="2" ParentId="12919" CreationDate="2019-06-18T13:39:24.290" Score="1" Body="&lt;p&gt;Word2vec works on the concept of  typical word co-occurrences. This means that it will work well only for words that occur frequently in the dataset. So proper nouns will not play any role in training the model. You can keep the proper nouns as they are or use only the words the occur more frequently than some threshold value based on the size of your dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you use the value stored in variable &lt;code&gt;x&lt;/code&gt; for something, and then change the stored value, it will not reflect anywhere unless you us use the variable &lt;code&gt;x&lt;/code&gt; again somewhere in the program.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Example&#xA;x = &quot;something something&quot;&#xA;print(x + &quot;...&quot;)&#xA;&#xA;# Result&#xA;something something ...&#xA;&#xA;# Changing x&#xA;x = &quot;new value&quot;&#xA;&#xA;# This new value of x will not reflect anywhere in the program&#xA;# Unless you use the variable x again.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-18T13:39:24.290" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12921" PostTypeId="2" ParentId="12914" CreationDate="2019-06-18T15:33:41.973" Score="2" Body="&lt;p&gt;In artificial intelligence, even though not everyone agrees, a common (and maybe the biggest) myth is that of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;intelligence explosion&lt;/a&gt;, which some people claim will happen (without considering physical limits or knowing anything about thermodynamics).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-18T15:33:41.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12922" PostTypeId="2" ParentId="12889" CreationDate="2019-06-18T15:44:25.267" Score="1" Body="&lt;p&gt;A recent research example is the &amp;quot;Grind&amp;quot; system. Take a look at the paper &lt;a href=&quot;https://arxiv.org/pdf/1804.06907.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing FO-Rewritings in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{E} \mathcal{L}$&lt;/span&gt; in Practice: from Atomic to Conjunctive Queries&lt;/a&gt; (2018) by Peter Hansen and Carsten Lutz. Here's the abstract.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A prominent approach to implementing ontology-mediated queries (OMQs) is to rewrite into a first-order query, which is then executed using a conventional SQL database system. We consider the case where the ontology is formulated in the description logic &lt;span class=&quot;math-container&quot;&gt;$\mathcal{E} \mathcal{L}$&lt;/span&gt; and the actual query is a conjunctive query and show that rewritings of such OMQs can be efficiently computed in practice, in a sound and complete way. Our approach combines a reduction with a decomposed backwards chaining algorithm for OMQs that are based on the simpler atomic queries, also illuminating the relationship between first-order rewritings of OMQs based on conjunctive and on atomic queries. Experiments with real-world ontologies show promising results.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="26508" LastEditorUserId="2444" LastEditDate="2021-01-27T21:25:00.407" LastActivityDate="2021-01-27T21:25:00.407" CommentCount="0" CommunityOwnedDate="2021-01-27T21:26:17.943" ContentLicense="CC BY-SA 4.0" />
  <row Id="12923" PostTypeId="2" ParentId="12874" CreationDate="2019-06-18T15:50:51.980" Score="1" Body="&lt;p&gt;My answer is yes, but in a trivial way. The least you would expect from an intelligent agent is that it is able to execute a given Turing machine on a given input. This requires actually no intelligence, just following rules. If however, you are referring to the capability of predicting if the Turing machine will terminate on the given input, that is another matter. I don't think it is reasonable to expect such &quot;undecidable&quot; computational power for an agent to be generally intelligent.&lt;/p&gt;&#xA;" OwnerUserId="26508" LastActivityDate="2019-06-18T15:50:51.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12926" PostTypeId="1" CreationDate="2019-06-18T19:45:18.273" Score="3" ViewCount="424" Body="&lt;p&gt;I am working on a deep reinforcement learning problem. The policy network has the same architecture as the one Deepmind published in 'Playing Atari with Deep Reinforcement Learning'. I am also using Prioritized Experience Replay. In the initial stage the behavior seems to be normal, i.e the agent is learning gradually. However, after a while the rewards suddenly go down by a lot. The TD erros also seem to be going up at the same time. I'm not sure how to interpret this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hypotheses are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The policy network is overfitting&lt;/li&gt;&#xA;&lt;li&gt;Some filters fail to activate thereby misrepresenting the state information&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate if you guys could give me some tips to narrow down this problem debug it. Cheers.&lt;a href=&quot;https://i.stack.imgur.com/gLwue.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gLwue.jpg&quot; alt=&quot;Rewards and Td error&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26514" LastEditorUserId="26514" LastEditDate="2019-06-18T20:10:50.580" LastActivityDate="2021-03-14T07:01:25.123" Title="Deep Reinforcement Learning: Rewards suddenly dip down" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12927" PostTypeId="1" AcceptedAnswerId="12935" CreationDate="2019-06-18T19:57:50.027" Score="3" ViewCount="91" Body="&lt;p&gt;My goal is to understand AlphaZero paper published by deepmind. I'm beginning my journey trying to get the basic intuition of reinforcement learning from the &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; by Barto and Sutton. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per my background, I'm familiar with MDPs, value iteration and policy iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask till what chapter of Barto and Sutton's book is one required to read in order to fully comprehend AlphaZero paper. Monte-Carlo Tree Search is discussed in Chapter-8 of the book. Will it be enough till that? Or would I be needing more resources apart from this book? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="1299" LastActivityDate="2019-06-19T20:49:32.273" Title="Knowledge required for understanding AlphaZero paper" Tags="&lt;reinforcement-learning&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12929" PostTypeId="1" CreationDate="2019-06-18T20:19:17.873" Score="1" ViewCount="81" Body="&lt;p&gt;What do you recommend for recognizing squares in an image?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data needed to know is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Identify concrete squares.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Identify the pixel coordinates of (the edges of) the squares in the image.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A couple of examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;20 squares identified.&lt;/li&gt;&#xA;&lt;li&gt;For each square identify its pixel coordinates.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wq1Pv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wq1Pv.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2 squares identified.&lt;/li&gt;&#xA;&lt;li&gt;First square coordinates: (38, 124, 86, 187).&lt;/li&gt;&#xA;&lt;li&gt;Second square coordinates: (88, 124, 136, 187).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lCf4D.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lCf4D.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26517" LastEditorUserId="1671" LastEditDate="2019-06-19T17:43:47.413" LastActivityDate="2019-06-21T14:38:36.937" Title="Any library recommended for recognizing squares in an image?" Tags="&lt;image-recognition&gt;&lt;software-evaluation&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12930" PostTypeId="1" CreationDate="2019-06-18T23:10:18.457" Score="1" ViewCount="83" Body="&lt;p&gt;I learn that the &lt;strong&gt;Viterbi&lt;/strong&gt; algorithm used for Hidden Markov Model (&lt;strong&gt;HMM&lt;/strong&gt;) can classify a sequence of hidden states from the corresponding observations; Markov Random Field (&lt;strong&gt;MRF&lt;/strong&gt;) and Conditional Random Field (&lt;strong&gt;CRF&lt;/strong&gt;) can also do it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can these algorithms be used to classify a single future state?&lt;/p&gt;&#xA;" OwnerUserId="4042" LastEditorUserId="12853" LastEditDate="2019-06-20T18:11:36.707" LastActivityDate="2021-03-11T22:02:55.237" Title="Can HMM, MRF, or CRF be used to classify the state of a single observation, not the entire observation sequence?" Tags="&lt;markov-chain&gt;&lt;sequence-modelling&gt;&lt;time-series&gt;&lt;hidden-markov-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12931" PostTypeId="5" CreationDate="2019-06-18T23:12:02.217" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-18T23:12:02.217" LastActivityDate="2019-06-18T23:12:02.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12932" PostTypeId="4" CreationDate="2019-06-18T23:12:02.217" Score="0" Body="For questions related to the branch and bound algorithm design paradigm, for discrete and combinatorial optimization problems, as well as mathematical optimization, in the context of AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-18T23:12:02.217" LastActivityDate="2019-06-18T23:12:02.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12933" PostTypeId="1" CreationDate="2019-06-18T23:50:17.177" Score="1" ViewCount="135" Body="&lt;p&gt;I’m looking for some help with my neural network. I’m working on a binary classification on a recurrent neural network that predicts stock movements (up and down) Let’s say I’m studying Eur/Usd, I’m using all the data from 2000 to 2017 to train et I’m trying to predict every day of 2018. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I’m dealing with right now is that my program is giving me different answers every time I run it even without changing anything and I don’t understand why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accuracy during the train from 2000 to 2017 is around 95% but I’ve noticed another issue. When I train it with 1 new data every day in 2018, I thought 2 epochs was enough, like if it doesn’t find the right answer the first time, then it knows what the answer is since the problem is binary, but apparently that doesn’t work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys have any suggestion to &lt;strong&gt;stabilize&lt;/strong&gt; my NN?&lt;/p&gt;&#xA;" OwnerUserId="26522" LastEditorUserId="2444" LastEditDate="2019-06-24T00:37:49.720" LastActivityDate="2021-03-15T13:04:17.447" Title="How can I stabilise a recurrent neural network used for binary classification?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;recurrent-neural-networks&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12934" PostTypeId="2" ParentId="12918" CreationDate="2019-06-19T00:12:21.950" Score="2" Body="&lt;p&gt;Welcome to AI.SE @vdbuss, and great first question!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This point is touched on in Section 15.2.3 (page 576 in my copy), in the second paragraph, and there's a good exercise at the end of the chapter (15.4) that is designed to get you to think through exactly why these are different procedures. If you want to really absorb it, I suggest trying to work out that exercise! If you want the quick answer, read on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic action of filtering is to generate a probability distribution &lt;span class=&quot;math-container&quot;&gt;$P(X_{t+1} | e_{1:t+1})$&lt;/span&gt; using &lt;em&gt;only two pieces of information&lt;/em&gt;, specifically the current state distribution &lt;span class=&quot;math-container&quot;&gt;$P(X_{t} | e_{1:t})$&lt;/span&gt;, and the new piece of evidence &lt;span class=&quot;math-container&quot;&gt;$e_{t+1}$&lt;/span&gt;. So, when computing the most likely sequence, the algorithm &lt;em&gt;cannot&lt;/em&gt; take into account the sequences that are actually &lt;em&gt;possible&lt;/em&gt;, while Vitirbi can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a simple example: suppose I tell you that I'm going to drop you in a maze at one of two locations. I drop you near the top right corner with probability 0.75, and near the bottom left corner with probability 0.25. Suppose further that a &lt;a href=&quot;https://en.wikipedia.org/wiki/Grue_(monster)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grue&lt;/a&gt; is known (with certainty) to live somewhere near the bottom left corner. Using filtering, your maximum aposterori estimate for your location after being dropped in the maze (&lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;) is that you are in the top right corner. You then move 1 step to the right and can see a Grue. Clearly your estimate for your position in the second timestep &lt;span class=&quot;math-container&quot;&gt;$(t=2)$&lt;/span&gt; must be the bottom left, because Grues only live there. But, you definitely can't end up moving to the bottom left by moving right from the top right, so your sequence has probability zero overall, despite using the maximum aposterori estimate for position at every step. To avoid this, Vitirbi uses a linear amount of extra space to select the maximum aposterori &lt;em&gt;sequence&lt;/em&gt;, which in this case is clearly that you are near the bottom left in both timesteps.  &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-06-19T00:12:21.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12935" PostTypeId="2" ParentId="12927" CreationDate="2019-06-19T00:18:36.000" Score="4" Body="&lt;p&gt;The more you read, the more deeply you can understand any paper, but given your stated background, reading the Monte-Carlo Tree Search chapter of Barto &amp;amp; Sutton, plus Gerald Tesauro's &lt;a href=&quot;https://cling.csd.uwo.ca/cs346a/extra/tdgammon.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD-Gammon paper&lt;/a&gt; (which is pretty accessible, and which is the basis for the &lt;em&gt;other&lt;/em&gt; technique used in AlphaZero) should be enough to get a pretty good idea of what they did. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-06-19T20:49:32.273" LastActivityDate="2019-06-19T20:49:32.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12936" PostTypeId="1" CreationDate="2019-06-19T04:25:12.423" Score="1" ViewCount="280" Body="&lt;p&gt;I would like to use the bottleneck layer of U-Net (the last layer of the encoder) to calculate the similarity between two images. For that, I have to somehow flatten the last layer of the encoder. In my opinion, there are two approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Take the last layer which in my case is &lt;span class=&quot;math-container&quot;&gt;$4 \times 4 \times 16$&lt;/span&gt; and flatten it to 1D&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Add a dense before the decoder and then reshape the dense 1D layer into 3D&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the second case, I am not sure how this would affect the network. Arbitrarily reshaping a 1D array into a 3D tensor. Could that introduce weird artifacts? Does someone have experience in a similar problem?&lt;/p&gt;&#xA;" OwnerUserId="23063" LastEditorUserId="2444" LastEditDate="2020-06-13T00:19:04.513" LastActivityDate="2020-06-13T00:19:04.513" Title="How can I use the bottleneck layer of the U-net to calculate the similarity between two images?" Tags="&lt;convolutional-neural-networks&gt;&lt;unsupervised-learning&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12937" PostTypeId="2" ParentId="12874" CreationDate="2019-06-19T07:57:28.803" Score="1" Body="&lt;p&gt;A system is &lt;em&gt;Turing complete&lt;/em&gt; if it can be used to simulate any Turing machine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the Church-Turing thesis (which has not yet been proven), &lt;a href=&quot;https://plato.stanford.edu/entries/church-turing/#MeanCompCompTuriThes&quot; rel=&quot;nofollow noreferrer&quot;&gt;a human brain can compute any function that a Turing machine can (given enough time and space)&lt;/a&gt;, &lt;a href=&quot;http://web.mit.edu/asf/www/PopularScience/Friedman_BrainsAndTuringMachines_2002.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;but the reverse is not necessarily true, given that the human brain might be able to compute more functions than a Turing machine&lt;/a&gt;. Intuively, humans are thus Turing complete (even though, to prove this, you need a formal model of the human), that is, given enough time and space, a human can compute anything that a Turing machine can. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, an AGI, defined as an AI with human-level intelligence, needs to be Turing complete, otherwise there would be at least one function that a human can calculate but the AGI cannot, which would not make it as general as a human.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-19T07:57:28.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12938" PostTypeId="1" CreationDate="2019-06-19T08:28:00.287" Score="2" ViewCount="56" Body="&lt;p&gt;I am looking for an example in which it is simply impossible to use some sort of gradient descent to train a neural network. Is this available? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read quite some papers about gradient-free optimization tools, but they always use it on a network for which you can also use gradient descent. I want to have a situation in which the only option to train the network is by, for example, a genetic algorithm. &lt;/p&gt;&#xA;" OwnerUserId="26531" LastEditorUserId="2444" LastEditDate="2019-06-19T17:52:21.383" LastActivityDate="2019-06-19T17:52:21.383" Title="Neural networks when gradient descent is not possible" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="12939" PostTypeId="2" ParentId="12914" CreationDate="2019-06-19T10:49:35.877" Score="5" Body="&lt;p&gt;As Artificial Intelligence is rapidly invading in our lives the myths around AI is also fabricating rapidly. Before getting into details one need to get clear off from this myths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Myth 1: AI will take away our jobs:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reality:&lt;/strong&gt; AI is not completely different from other technologies and AI will not take away jobs but AI will change the way we work and helps us to increase the productivity by removing monotonous works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Myth 2: Artificial intelligence will take over the world:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reality:&lt;/strong&gt; AI controlling the world. According to me it will not possible unless we give it that power. AI or robots will assist in our work and helps us to solve some tedious works that are difficult for human to solve easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Myth 3: Intelligent machines can learn on their own&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reality:&lt;/strong&gt; It seems that a Intelligent machine can learn by it own. But the fact is that a AI Engineer or AI specialist should develop the algorithm and feed the machine with datasets and instructions and continuous monitoring should be done and most importantly regular update of software should be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Myth 4: Artificial Intelligence, Machine learning and Deep learning all three are same:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reality:&lt;/strong&gt; No not at all. To be clear machine learning is a part of AI and deep learning is the subset of ML. All three- AL, ML and DL are different but they are inter related with each other.&lt;/p&gt;&#xA;" OwnerUserId="26265" LastEditorUserId="75" LastEditDate="2019-06-28T14:17:00.453" LastActivityDate="2019-06-28T14:17:00.453" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12940" PostTypeId="1" CreationDate="2019-06-19T11:15:52.440" Score="3" ViewCount="139" Body="&lt;p&gt;Everyone is afraid of losing their job to robots. Will or does artificial intelligence cause mass unemployment?&lt;/p&gt;&#xA;" OwnerUserId="26351" LastEditorUserId="2444" LastEditDate="2019-06-20T22:10:20.477" LastActivityDate="2019-06-20T22:10:20.477" Title="Will artificial intelligence cause mass unemployment?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;social&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12941" PostTypeId="2" ParentId="12940" CreationDate="2019-06-19T13:57:49.613" Score="0" Body="&lt;p&gt;Up to a point. Some jobs will IMHO not easily be replaced by robots, others more easily. Some could be, but I hope common sense will prevail and stop that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manual jobs: fruit picking, warehouse picking, and cooking are some jobs that need really subtle hand control, and precise handling of fragile items. I think those will be harder to automate than eg car factory robots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Customer facing roles: receptionists have to do a wide variety of tasks. While some of the tasks might be aided by AI systems, a good PA or receptionist cannot easily be replaced. Also, many people would much rather interact with a fellow human than with a machine, at least in some situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judgments: a lot of jobs require judging a situation, balancing risks, and making 'gut' decisions. While AI systems can do them, I think many would still require human intervention. I for one wouldn't like to be sentenced by a robo-judge, or examined by a robo-doctor. True, humans also make mistakes, but they would hopefully err in less potentially disastrous ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Administration: again, many tasks can be assisted, which would lead to a reduction in head count, but a general AI is still off the horizon, so you'd still need humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creative arts: not likely. Would you want to read computer-generated novels, or look at computer-generated paintings?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could go on... in general: I think AI systems will make a lot of tasks faster and easier, so you need fewer people to do them. Some jobs cannot realistically be done by machines in the near-to-mid future, so overall we need not worry too much.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-06-19T13:57:49.613" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12942" PostTypeId="2" ParentId="12940" CreationDate="2019-06-19T14:56:50.467" Score="2" Body="&lt;p&gt;The nuanced, boring answer is that it depends on your definition of AI. Most people wouldn't say that the rule-based systems designed in the 70's are AI. The amazing leaps in machine learning are almost taken for granted as well (think about how normal speech and facial recognition have become). This is known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI effect&lt;/a&gt;; when we become accustomed to the technology, it loses it's 'magical aspect' and is thus no longer labelled as AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since AI is so diverse and difficult to define, the question becomes incredibly abstract. Did Siri cause all secretaries to become unemployed? Did TurboTax replace all accountants? Some parts of AI will affect jobs, or even make them redundant yes. On the other hand, it will give rise to new jobs as well. It is therefore impossible to generalize it as 'AI will cause massive unemployment'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not a new phenomenon, however, it has been part of the human economy ever since the industrial revolution (probably even before that, but I am not a historian). The invention of the car crippled the horse-and-wagon industry, but it brought along new jobs as well. &lt;/p&gt;&#xA;" OwnerUserId="18398" LastActivityDate="2019-06-19T14:56:50.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12943" PostTypeId="2" ParentId="12429" CreationDate="2019-06-19T15:17:34.720" Score="0" Body="&lt;p&gt;I think normalisation into range 0-1 is needed or at least to 5-10 cliping   because the values will become astronomical after many layers. Convolved image has a vector of features for each pixel. Take a RGB for example -&gt; each color is a feature in one pixel, the next map will be like 'horisontal line, vertical line, circle' for one pixel surroundings.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-19T15:17:34.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12944" PostTypeId="2" ParentId="12940" CreationDate="2019-06-19T15:26:13.677" Score="0" Body="&lt;p&gt;Yes , but you should be happy about it. It is like usage of mashines in industry causes &quot;mass unemployment&quot;. Modern tendencies is unfortinatly not like that - industry has not so bit interest in full automatisation cause of huge amout of cheap work force from immigrants - it is a bad factor, not that &quot;ai takes your job&quot;. &#xA;And further, robot what would take your job must not be realy smart - in could be just some doll without AI just making what is in programm...&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-19T15:26:13.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12946" PostTypeId="2" ParentId="12930" CreationDate="2019-06-19T16:53:21.500" Score="0" Body="&lt;p&gt;Yes this is possible and is exactly what a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov process&lt;/a&gt; aims to accomplish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hidden Markov Model&lt;/a&gt; can be considered a simple dynamic Bayesian network. This means that it will generate a probabilistic outcome for a number of states (classes) given a sequence of inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be used for a classification task based on a threshold probability to make a confirmed decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take a simple example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4fmgJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4fmgJ.png&quot; alt=&quot;Example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above example is a model to determine  how a person will be feeling given some sample data of the previous days.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Sample data&#xA;data = ['Healthy', 'Healthy', 'Fever']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using this data, it's possible to calculate the probability of each possible state (outcome).&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Probability after 2 Healthy and 1 Fever day&#xA;P = 0.6 * 0.7 * 0.3&#xA;&#xA;# Final day state 'Fever'&#xA;# Calculating probabilities for each state&#xA;Dizzy  = P * 0.6&#xA;Cold   = P * 0.3&#xA;Normal = P * 0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can calculate the probability of each state and this will be unique for every unique sequence of historical data.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-19T16:53:21.500" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12947" PostTypeId="1" CreationDate="2019-06-19T19:34:09.387" Score="1" ViewCount="17" Body="&lt;p&gt;I have some 64x64 pixels frames from a (simulated) video, with a spaceship moving on a fixed background. The spaceship moves in a straight line with constant velocity from left to right (along the x-axis), and the frames are from equal time intervals.  I can also place the ship at different y positions and let it move. In total I have 8 y positions and 64 frames for each y position (the details don't matter that much). Intuitively, as the background is fixed, and the shape of the ship is the same, all the information to reconstruct the image is found in the x and y position of the spaceship. What I am trying to do is to have a NN with an encoded and a decoder and a bottleneck in the middle and I want that bottleneck to have just 2 neurons. Ideally, the network would learn in these 2 neurons some function of x and y in the encoder, and the decoder would invert that function to give the original image. Here is my NN architecture (in Pytorch):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Rocket_E_NN(nn.Module):&#xA;    def __init__(self):&#xA;        super().__init__()&#xA;&#xA;        self.encoder = nn.Sequential(&#xA;            nn.Conv2d(3, 32, 4, 2, 1),          # B,  32, 32, 32&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(32, 32, 4, 2, 1),          # B,  32, 16, 16&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(32, 64, 4, 2, 1),          # B,  64,  8,  8&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(64, 64, 4, 2, 1),          # B,  64,  4,  4&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(64, 256, 4, 1),            # B, 256,  1,  1&#xA;            nn.ReLU(True),&#xA;            View((-1, 256*1*1)),                 # B, 256&#xA;            nn.Linear(256, 2),             # B, 1&#xA;        )&#xA;&#xA;    def forward(self, x):&#xA;        z = self.encoder(x)&#xA;        return z&#xA;&#xA;class Rocket_D_NN(nn.Module):&#xA;    def __init__(self):&#xA;        super().__init__()&#xA;        self.decoder = nn.Sequential(&#xA;            nn.Linear(2, 256),               # B, 256&#xA;            View((-1, 256, 1, 1)),               # B, 256,  1,  1&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(256, 64, 4),      # B,  64,  4,  4&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(64, 64, 4, 2, 1), # B,  64,  8,  8&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(64, 32, 4, 2, 1), # B,  32, 16, 16&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(32, 32, 4, 2, 1), # B,  32, 32, 32&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(32, 3, 4, 2, 1),  # B, 3, 64, 64&#xA;        )&#xA;&#xA;    def forward(self, z):&#xA;        x = self.decoder(z)&#xA;        return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the example of one of the images that I have (it was much higher resolution but I brought it down to 64x64):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2bnPE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2bnPE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So after training it for around 2000 epoch with a bs of 128, with Adam, trying several LR schedules (going from 1e-3 to 1e-6) I can't get the loss below an RMSE of 0.010-0.015 (the pixel values are between 0 and 1). The reconstructed image looks ok by eye, but I would need a better loss for the purpose of my project. Is there any way I can push the loss lower, or am I asking too much from the NN to distill all the information in these 2 numbers?&lt;/p&gt;&#xA;" OwnerUserId="23871" LastActivityDate="2019-06-19T19:34:09.387" Title="Limits for a bottleneck" Tags="&lt;loss-functions&gt;&lt;autoencoders&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12948" PostTypeId="1" CreationDate="2019-06-19T19:49:11.070" Score="0" ViewCount="97" Body="&lt;p&gt;I am trying to implement a Neural Network for binary classification using python and numpy only. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network structure is as follows:&lt;br&gt;&#xA;input features: 2 [1X2] matrix&lt;br&gt;&#xA;Hidden layer1: 5 neurons [2X5] matrix&lt;br&gt;&#xA;Hidden layer2: 5 neurons [5X5] matrix&lt;br&gt;&#xA;Output layer: 1 neuron [5X1]matrix&lt;br&gt;&#xA;I have used the sigmoid activation function in all the layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets say I use binary cross entropy as my loss function.&#xA;How do I do the back propagation on these matrices to update weights?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Layer():&#xA;    def __init__(self,number_of_neurons,number_of_inputs):&#xA;        self.weights=np.random.rand(number_of_neurons, number_of_inputs)&#xA;        self.bias=np.random.rand(number_of_neurons,1)     &#xA;&#xA;&#xA;class NeralNetwork():&#xA;        def __init__(self, layer1, layer2,layer3):&#xA;            self.layer1 = layer1&#xA;            self.layer2 = layer2&#xA;            self.layer3 = layer3&#xA;&#xA;        def sigmoid(self,x):&#xA;            return 1 / (1 + np.exp(-x))&#xA;&#xA;        def derivative_sigmoid(self,x):&#xA;            return x*(1-x)&#xA;&#xA;        def get_cost_value(self,Y_hat, Y):&#xA;            m = Y_hat.shape[1]&#xA;            cost = -1 / m * (np.dot(Y, np.log(Y_hat).T) + np.dot(1 - Y, np.log(1 - Y_hat).T))&#xA;            return np.squeeze(cost)&#xA;&#xA;        def get_cost_derivative(self,Y_hat,Y):&#xA;            return  - (np.divide(Y, Y_hat) - np.divide(1 - Y, 1 - Y_hat))&#xA;&#xA;        def train(self,inputs,labels,epocs):&#xA;            for epoc in range(1,epocs+1):&#xA;                z1=np.dot(self.layer1.weights,inputs)+self.layer1.bias&#xA;                a1=self.sigmoid(z1)&#xA;                z2=np.dot(self.layer2.weights,a1)+self.layer2.bias&#xA;                a2=self.sigmoid(z2)&#xA;                #print(a2.shape)&#xA;                z3=np.dot(self.layer3.weights,a2)+self.layer3.bias&#xA;                a3=self.sigmoid(z3)&#xA;                #print(a3.shape)&#xA;                if epoc%100 is 0:&#xA;                    print(a3)&#xA;&#xA;                cost=self.get_cost_value(a3,labels)&#xA;                #print(cost)&#xA;&#xA;&#xA;&#xA;                layer3_delta=self.derivative_sigmoid(a3)*self.get_cost_derivative(a3,labels)&#xA;                print(layer3_delta.shape)&#xA;                Dw_layer3=np.dot(layer3_delta,a2.T)&#xA;                Db_layer3=layer3_delta&#xA;                #print(Dw_layer3.shape)&#xA;                layer2_delta=np.dot(self.layer3.weights.T,layer3_delta)*self.derivative_sigmoid(a2)&#xA;                #print(layer2_delta.shape)&#xA;                Dw_layer2=np.dot(layer2_delta,a1.T)&#xA;                Db_layer2=layer2_delta&#xA;&#xA;                layer1_delta=np.dot(self.layer2.weights.T,layer2_delta)*self.derivative_sigmoid(a1)&#xA;                Dw_layer1=np.dot(layer1_delta,inputs.T)&#xA;                Db_layer1=layer1_delta&#xA;                #print(Dw_layer1)&#xA;&#xA;                self.layer1.weights-=((1/epoc)*Dw_layer1)&#xA;                self.layer2.weights-=((1/epoc)*Dw_layer2)&#xA;                self.layer3.weights-=((1/epoc)*Dw_layer3)&#xA;&#xA;                self.layer1.bias-=((1/epoc)*Db_layer1)&#xA;                self.layer2.bias-=((1/epoc)*Db_layer2)&#xA;                self.layer3.bias-=((1/epoc)*Db_layer3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So far I have tried to implement this as shown above. But there seems to be a mistake because, after training, the network doesn't seem to have learned. Please let me know if you have any inputs.&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="21797" LastEditDate="2019-06-23T21:43:54.627" LastActivityDate="2019-06-23T21:43:54.627" Title="Back propagation on matrix of weights" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12949" PostTypeId="1" AcceptedAnswerId="12950" CreationDate="2019-06-19T20:04:17.610" Score="2" ViewCount="74" Body="&lt;p&gt;Is explainable AI more feasible through symbolic AI or soft computing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much each paradigm, symbolic AI and soft computing (or hydrid approaches), adresses explanation and argumentation, where symbolic AI refers e.g. to &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOFAI&lt;/a&gt; or expert systems and soft computing refers to machine learning or probabilistic methods.&lt;/p&gt;&#xA;" OwnerUserId="26542" LastEditorUserId="2444" LastEditDate="2019-06-19T20:45:09.183" LastActivityDate="2019-06-19T20:45:09.183" Title="Is explainable AI more feasible through symbolic AI or soft computing?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;symbolic-ai&gt;&lt;explainable-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12950" PostTypeId="2" ParentId="12949" CreationDate="2019-06-19T20:22:35.197" Score="2" Body="&lt;p&gt;XAI is relevant to &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Black_box&quot; rel=&quot;nofollow noreferrer&quot;&gt;black box&lt;/a&gt;&quot; AI (machine learning methods where the decision making rationale is not apparent, only the structure of the system that led to that decision.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Symbolic AI, GOFAI, and Expert Systems are both explainable and understood, in that the the decision-making process is designed by humans.  (&lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symbolic AI&lt;/a&gt; involves human-readable representations of problems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To directly answer, XAI is not only feasible in the latter cases, it is a prerequisite. The difficulty is in making black box decision-making explainable. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-06-19T20:22:35.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12951" PostTypeId="2" ParentId="12948" CreationDate="2019-06-19T21:19:06.847" Score="-1" Body="&lt;p&gt;The back propagation must be done in two steps.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transfer derivative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the output value, you must calculate its slope. This is simple as you are using the &lt;code&gt;sigmoid&lt;/code&gt; activation function. So the derivative can be calculated as follows&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Calculating slope&#xA;def transfer_derivative(output):&#xA;    return output * (1.0 - output)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Back propagate the error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculate the errors for the various layers of the network and update the weights and bias.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Error calculation&#xA;&#xA;# For output layer&#xA;error = (expected - output) * transfer_derivative(output)&#xA;&#xA;# Hidden layers&#xA;error = (weight_k * error_j) * transfer_derivative(output)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;error_j&lt;/code&gt; is the error signal from the &lt;code&gt;jth&lt;/code&gt; neuron in the output layer, &lt;code&gt;weight_k&lt;/code&gt; is the weight that connects the &lt;code&gt;kth&lt;/code&gt; neuron to the current neuron and output is the output for the current neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally updating weights&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;weight = weight + learning_rate * error * input&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/implement-backpropagation-algorithm-scratch-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a detailed explanation of implementing neural networks from scratch using Python along with sample code. Give it a read.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-19T21:19:06.847" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12952" PostTypeId="2" ParentId="12933" CreationDate="2019-06-19T21:38:38.447" Score="1" Body="&lt;p&gt;Firstly, dealing with the issue that the program gives different answers every time without making any changes can be due to a couple of things.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assigning random values to weights and bias. This can be solved by setting a &lt;code&gt;seed&lt;/code&gt; manually at the start of the program.&lt;/li&gt;&#xA;&lt;li&gt;Make sure you have set the model to the testing mode after training. For some frameworks, this has to be done manually.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Secondly, regarding your expected results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To generate a proper accuracy metric, you will have to sample your dataset into training and testing data, making sure there is now overlap between them. This might be an issue as you have stated training on data till 2017 and then again training on data of 2018.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, don't expect that the model will know that the output is wrong and directly change it because it's binary classification. This is not how neural networks work. The model fits the solution better by gradually updating its weights and biases over a number of iterations. So it will take a number of epochs to learn new trends in the data for 2018.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-19T21:38:38.447" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12953" PostTypeId="1" AcceptedAnswerId="13005" CreationDate="2019-06-20T00:36:00.680" Score="3" ViewCount="42" Body="&lt;p&gt;I'm kind of new to machine learning/AI, but I was wondering if using thresholds/fuzzy logic-like functions and even networks of dependent, stochastic variables that change over time (LTL maybe?), would be ample enough to emulate natural processes like emotions, hunger, maybe even pain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dilemma is whether creating a basic library to do this for the developer community is worth it if everything can be modeled more-or-less mathematically deterministic, even if the formulas are really complicated (see research like: &lt;a href=&quot;https://engineering.stanford.edu/news/virtual-cell-would-bring-benefits-computer-simulation-biology&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://engineering.stanford.edu/news/virtual-cell-would-bring-benefits-computer-simulation-biology&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial reasoning was biological processes are connected to psychological functionality (e.g., being hungry might make someone irritable, but that irritability may wear-off, which triggers different paths of thought but not others).  But these are so inter-dependent that it may be random or it is essentially PRNG, in order to properly simulate the mood fluctuations and biological processes computers don't have but humans do have.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would we be better-off waiting for these complex physical/neurological models to come out?&lt;/p&gt;&#xA;" OwnerUserId="26544" LastEditorUserId="26544" LastEditDate="2019-06-20T01:01:43.290" LastActivityDate="2019-06-23T02:39:31.800" Title="Is There A Need For Stochastic Inputs To Mimic Real-World Biology And Environment?" Tags="&lt;emotional-intelligence&gt;&lt;biology&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12954" PostTypeId="1" CreationDate="2019-06-20T06:51:10.220" Score="0" ViewCount="50" Body="&lt;p&gt;It seems to me that, right now, the key to making a good Machine Learning model is in choosing the right combination of hyper-parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly: Am I right in saying, if a model is able to tune it's own hyper-parameters, we have in some sense achieved a general intelligence system? Or a glimpse of an actual artificial intelligence system? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel the answer lies in what one means by &quot; tune it's own hyper-parameters&quot;. If it means to be able to reach Bayesian levels of performance on that task then theoretically, after the tuning, the model is able to perform at par or better than humans and so it seems the answer would be yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly: I understand that hyper-parameter tuning is done intuitively. But there are a  set of general directions that is discernable looking at results.  Here I am talking about a heuristic approach to perfect a learning model. &#xA;Consider an example:&#xA;Say I hardcode a model to, while training, observe gradient values. If the gradient is too large or the cost is highly oscillatory, then restart training with a smaller learning rate.&#xA;Then obtain metrics on a test set. If it is poor, then again restart training with regularisation or increased regularisation. &#xA;It can also observe various plot behaviours, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is maybe not every trick up a researcher's sleeve can be hardcoded. But a decent level of basic tuning can be done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thirdly: Let us say, we have a reinforcement learning system on top of a supervised learning system. That is an RL network sets some hyper-parameters. The action then is to train with these hyper-parameters. The reward would be the accuracy on the test set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible that such a system could solve the problem of hyper-parameter tuning? &lt;/p&gt;&#xA;" OwnerUserId="17143" LastActivityDate="2019-06-20T07:10:56.820" Title="Evolving Machine Learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2020-06-19T22:58:35.890" ContentLicense="CC BY-SA 4.0" />
  <row Id="12955" PostTypeId="2" ParentId="12954" CreationDate="2019-06-20T07:10:56.820" Score="2" Body="&lt;p&gt;Such a system can and does solve the problem of hyperparameter tuning. &lt;a href=&quot;https://cloud.google.com/automl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's AutoML&lt;/a&gt; does this. &lt;a href=&quot;https://blog.coast.ai/lets-evolve-a-neural-network-with-a-genetic-algorithm-code-included-8809bece164&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is another example that uses a Genetic Algorithm&lt;/a&gt; to breed new neural network structures. &#xA;AutoML has been shown to outperform humans in the rate that it improves network designs. It seems to favour Residual Network style topologies. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-06-20T07:10:56.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12957" PostTypeId="1" CreationDate="2019-06-20T09:58:57.880" Score="3" ViewCount="104" Body="&lt;p&gt;I have the following problem. We have &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; separate &lt;em&gt;discrete&lt;/em&gt; inputs, which can take any integer value between &lt;span class=&quot;math-container&quot;&gt;$-63$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$63$&lt;/span&gt;. The output is also supposed to be a discrete value between &lt;span class=&quot;math-container&quot;&gt;$-63$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$63$&lt;/span&gt;. Another constraint is that the solution should allow for online learning with singular values or mini-batches, as the dataset is too big to load all the training data into memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried the following method, but the predictions are not good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created an MLP or feedforward network with &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; inputs and &lt;span class=&quot;math-container&quot;&gt;$127$&lt;/span&gt; outputs. The inputs are being fed without normalization. The number of hidden layers is &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$[8,16,32,64]$&lt;/span&gt; units in each (respectively). So, essentially, this treats the problem like a sequence classification problem. For training, we feed the non-normalized input along with a one-hot encoded vector for that specific value as output. The inference is done the same way. Finding the hottest output and returning that as the next number in the sequence.&lt;/p&gt;&#xA;" OwnerUserId="26554" LastEditorUserId="2444" LastEditDate="2019-07-20T17:43:54.440" LastActivityDate="2021-04-10T21:06:40.570" Title="Which online machine learning technique to use for multi-class classification problem with multiple inputs?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;classification&gt;&lt;multilayer-perceptron&gt;&lt;online-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12958" PostTypeId="2" ParentId="12957" CreationDate="2019-06-20T13:04:01.453" Score="0" Body="&lt;p&gt;I suggest using Data Stream algorithms to try on your problems since you are asking for &quot;online learning with singular values or minibatches as the dataset is too big too load all the training data into memory.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://moa.cms.waikato.ac.nz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOA&lt;/a&gt; is a good choice for these algorithms. &lt;a href=&quot;https://www.cs.waikato.ac.nz/~abifet/MOA/API/classmoa_1_1classifiers_1_1trees_1_1_hoeffding_tree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hoeffding Trees&lt;/a&gt; is also a good first  choice to try.&lt;/p&gt;&#xA;" OwnerUserId="4300" LastActivityDate="2019-06-20T13:04:01.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12961" PostTypeId="2" ParentId="4647" CreationDate="2019-06-20T17:06:09.680" Score="1" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=1315&amp;amp;context=dltr&quot; rel=&quot;nofollow noreferrer&quot;&gt;Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For&lt;/a&gt;, the authors claim that the &quot;right to explanation&quot; is unlikely to provide a complete remedy to algorithmic harms for at least two reasons&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It is unclear when any explanation-related right can be triggered&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The explanations required by the law, &quot;meaningful information about the&#xA;logic of processing&quot;, might not be provided by current (explainable) AI methods &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The authors then conclude that the &quot;right to explanation&quot; is &lt;em&gt;distracting&lt;/em&gt;, but that other laws of the European Union's General Data Protection Regulation, such as&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;right to be forgotten&lt;/li&gt;&#xA;&lt;li&gt;right to data portability&lt;/li&gt;&#xA;&lt;li&gt;privacy by design&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;might compensate the defects of the &quot;right to explanation&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the &quot;right to explanation&quot; is an attempt to protect citizens against the possible undesirable consequences of the use of AI methods. However, it is not flawless and clearer rights might need to be promulgated.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-20T17:06:09.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12963" PostTypeId="1" AcceptedAnswerId="12968" CreationDate="2019-06-21T05:31:42.897" Score="2" ViewCount="63" Body="&lt;p&gt;I would like to create a neural network that converts text into handwriting for use with a pen plotter. Before I start on this project, I'd like to be sure that artificial intelligence is the best way to do this. A problem that I foresee with this approach is a lack of human like variation in the results. For example, the word &quot;dog&quot;, when inputted into the network, would be the same every time, assuming I'm not missing something. I am interested if there is any way to vary the output of the network in a realistic way, even when the input is exactly the same. Could I use a second network to make the results more random, but also still look human-like? Any thoughts/ideas would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26575" LastActivityDate="2019-06-21T08:22:39.937" Title="How to add variation in the results of a neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;human-like&gt;&lt;handwritten-characters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12965" PostTypeId="1" CreationDate="2019-06-21T06:31:16.227" Score="1" ViewCount="104" Body="&lt;p&gt;I have multiple invoices images which need to classify invoice types such as fright, utility, goods, etc. Is there any way to classify without OCR?&lt;/p&gt;&#xA;" OwnerUserId="26576" LastEditorUserId="12853" LastEditDate="2019-06-21T08:40:24.213" LastActivityDate="2019-06-21T08:40:24.213" Title="Is there any way to classify Document Image without OCR?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12966" PostTypeId="1" CreationDate="2019-06-21T07:18:18.070" Score="0" ViewCount="56" Body="&lt;p&gt;I have to create a Neural Network for regression purpose. Basically, I created a Model which predict next 5 values when we give past 6 values. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/MRb7h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MRb7h.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;I want to make a change in this neural network. For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when giving 6 past values I have to predict the next 10 values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, is there any issue of selecting the number of output dimension greater than the input dimension. Which type of parameters arrangement makes the Neural Network achieve good accuracy? do I have to decide the number of input parameters always greater than output parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in Advance!&lt;/p&gt;&#xA;" OwnerUserId="24006" LastEditorUserId="12853" LastEditDate="2019-06-28T04:34:27.233" LastActivityDate="2019-06-28T04:34:27.233" Title="Decide Number of input Parameters and Output Parameters - ANN" Tags="&lt;neural-networks&gt;&lt;datasets&gt;&lt;artificial-neuron&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12967" PostTypeId="2" ParentId="12966" CreationDate="2019-06-21T08:01:30.053" Score="2" Body="&lt;p&gt;This should be possible given the fact that ANNs have the ability to do the feature engineering and feature selection tasks by themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that given a lesser number of input parameters, the model will be able to generate and select additional features by itself. You will obviously not be able to understand or model these features manually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing to keep in mind is that you will need a large dataset and a number of iterations before you are able to achieve a decent accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, there are networks that can generate image from classes.&#xA;Give &lt;a href=&quot;https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks-explained-3fb6f2367464&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; a read and here is an &lt;a href=&quot;https://www.youtube.com/watch?v=rAbhypxs1qQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; where the output layer is larger than the input layer.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-21T08:01:30.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12968" PostTypeId="2" ParentId="12963" CreationDate="2019-06-21T08:22:39.937" Score="0" Body="&lt;p&gt;I would suggest starting with &lt;a href=&quot;https://skymind.ai/wiki/generative-adversarial-network-gan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks (GAN)&lt;/a&gt;. They usually are capable of adding some randomness to the output to produce different variants. Moreover, &lt;a href=&quot;https://arxiv.org/abs/1411.1784&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conditional GAN&lt;/a&gt;s can generate outcomes regarding the observed condition. Therefore, as you change the condition (as an input to the network) you can get different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples of NN for synthetic handwriting generators:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/sjvasquez/handwriting-synthesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sjvasquez/handwriting-synthesis&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://distill.pub/2016/handwriting/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://distill.pub/2016/handwriting/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12853" LastActivityDate="2019-06-21T08:22:39.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12969" PostTypeId="2" ParentId="12965" CreationDate="2019-06-21T08:34:29.110" Score="1" Body="&lt;p&gt;It is possible to classify invoice scans without passing through an OCR component if they are visually different (they demonstrate different visual features). On the other hand, if the invoices look very similar, then the classifier might not be very accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another challenge would be the number of images you need to train a deep network for image classification (you may start with &lt;a href=&quot;https://www.kernix.com/blog/image-classification-with-a-pre-trained-deep-neural-network_p11&quot; rel=&quot;nofollow noreferrer&quot;&gt;pretrained models&lt;/a&gt; and only perform the finetuning of you do not have enough images). On the other hand, the combination of pretrained OCR models and NLP-based document classifier may not need that many samples for training (for this specific task).&lt;/p&gt;&#xA;" OwnerUserId="12853" LastActivityDate="2019-06-21T08:34:29.110" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12970" PostTypeId="1" AcceptedAnswerId="13178" CreationDate="2019-06-21T09:22:20.317" Score="2" ViewCount="420" Body="&lt;p&gt;Neuroevolution can be used to evolve a network's architecture (and weights, of course). Deep reinforcement learning, on the other hand, has been proven to be extremely powerful at optimising the network weights in order to train really well-performing agents. Can we use the following pipeline?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;search for the best network topology/weights through neuroevolution&lt;/li&gt;&#xA;&lt;li&gt;train the best candidate selected above through DQN or something similar&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This seems reasonable to me, but I haven't found anything on the matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is there any research work that attempts to combine neuroevolution with deep reinforcement learning? Is it feasible? What are the main challenges?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2020-06-10T07:20:55.750" LastActivityDate="2020-06-10T07:20:55.750" Title="Is there any research work that attempts to combine neuroevolution with deep reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;evolutionary-algorithms&gt;&lt;reference-request&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12971" PostTypeId="1" AcceptedAnswerId="12978" CreationDate="2019-06-21T09:37:47.403" Score="13" ViewCount="2175" Body="&lt;p&gt;I recently got a 18-month postdoc position in a math department. It's a position with relative light teaching duty and a lot of freedom about what type of research that I want to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously I was mostly doing some research in probability and combinatorics. But I am thinking of doing a bit more application oriented work, e.g., AI. (There is also the consideration that there is good chance that I will not get a tenure-track position at the end my current position. Learn a bit of AI might be helpful for other career possibilities.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What sort of &lt;strong&gt;mathematical problems&lt;/strong&gt; are there in AI that people are working on? From what I heard of, there are people studying&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deterministic_finite_automaton&quot; rel=&quot;noreferrer&quot;&gt;Deterministic Finite Automaton&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;noreferrer&quot;&gt;Multi-armed bandit problems&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;noreferrer&quot;&gt;Monte Carlo tree search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Community_structure&quot; rel=&quot;noreferrer&quot;&gt;Community detection&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any other examples?&lt;/p&gt;&#xA;" OwnerUserId="26112" LastEditorUserId="2444" LastEditDate="2019-06-21T20:30:54.463" LastActivityDate="2021-01-31T13:40:45.927" Title="What sort of mathematical problems are there in AI that people are working on?" Tags="&lt;research&gt;&lt;math&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="17" ContentLicense="CC BY-SA 4.0" />
  <row Id="12972" PostTypeId="1" AcceptedAnswerId="12983" CreationDate="2019-06-21T09:57:56.960" Score="0" ViewCount="1300" Body="&lt;p&gt;Here I am Showing Two Loss graphs of an Artificial Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Model 1&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WgBhA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WgBhA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Model 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oghdm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oghdm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Blue&lt;/strong&gt; -training loss&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Red&lt;/strong&gt;  -val training loss&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me to analyse these graphs? I read some articles and post but doesn't give me any sense.&lt;/p&gt;&#xA;" OwnerUserId="24006" LastActivityDate="2019-06-21T20:36:21.117" Title="Analysis of Training Loss and Validation Loss Graph" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12973" PostTypeId="1" AcceptedAnswerId="13034" CreationDate="2019-06-21T11:23:35.627" Score="2" ViewCount="85" Body="&lt;p&gt;When describing tensors of higher order I feel like there is an overloading of the term &lt;em&gt;dimension&lt;/em&gt; as it may be used to describe the order of the tensor but also the dimensionality of the... &quot;orders&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume one describes the third-order tensor produced by a convolutional layer and wants to refer to its width and height. Do you say spatial &lt;em&gt;dimensions&lt;/em&gt;? Would you write about the channel &lt;em&gt;dimension&lt;/em&gt;? Or rather the direction? Saying &quot;spatial order&quot; feels really weird. But staying with dimensions makes sentences like &quot;&lt;em&gt;The spatial dimensions are of equal dimensionality&lt;/em&gt;.&quot; (Disclaimer: Obviously you can avoid the issue here by restructuring, but doing this at every occasion does not feel like a satisfactory solution.).&lt;/p&gt;&#xA;" OwnerUserId="26584" LastActivityDate="2019-06-25T16:50:20.420" Title="Describing the order of a tensor" Tags="&lt;terminology&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12974" PostTypeId="1" AcceptedAnswerId="12976" CreationDate="2019-06-21T12:29:00.857" Score="0" ViewCount="20" Body="&lt;p&gt;I am on the hook to measure the prediction results of an object detector. I learned from some tutorials that when testing a trained object detector, for each object in the test image, the following information is provided:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;lt;object&amp;gt;&#xA;    &amp;lt;name&amp;gt;date&amp;lt;/name&amp;gt;&#xA;    &amp;lt;pose&amp;gt;Unspecified&amp;lt;/pose&amp;gt;&#xA;    &amp;lt;truncated&amp;gt;0&amp;lt;/truncated&amp;gt;&#xA;    &amp;lt;difficult&amp;gt;0&amp;lt;/difficult&amp;gt;&#xA;    &amp;lt;bndbox&amp;gt;&#xA;        &amp;lt;xmin&amp;gt;451&amp;lt;/xmin&amp;gt;&#xA;        &amp;lt;ymin&amp;gt;182&amp;lt;/ymin&amp;gt;&#xA;        &amp;lt;xmax&amp;gt;695&amp;lt;/xmax&amp;gt;&#xA;        &amp;lt;ymax&amp;gt;359&amp;lt;/ymax&amp;gt;&#xA;    &amp;lt;/bndbox&amp;gt;&#xA;&amp;lt;/object&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it is still unclear to me 1) how does these information is taken by the object detector to measure the accuracy, and 2) how does the &quot;loss&quot; is computed for this case. Is it something like a strict comparison? For instance, if for the object &quot;date&quot;, I got the following outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;lt;object&amp;gt;&#xA;    &amp;lt;name&amp;gt;date&amp;lt;/name&amp;gt;&#xA;    &amp;lt;pose&amp;gt;Unspecified&amp;lt;/pose&amp;gt;&#xA;    &amp;lt;truncated&amp;gt;0&amp;lt;/truncated&amp;gt;&#xA;    &amp;lt;difficult&amp;gt;0&amp;lt;/difficult&amp;gt;&#xA;    &amp;lt;bndbox&amp;gt;&#xA;        &amp;lt;xmin&amp;gt;461&amp;lt;/xmin&amp;gt;  &amp;lt;---- different&#xA;        &amp;lt;ymin&amp;gt;182&amp;lt;/ymin&amp;gt;&#xA;        &amp;lt;xmax&amp;gt;695&amp;lt;/xmax&amp;gt;&#xA;        &amp;lt;ymax&amp;gt;359&amp;lt;/ymax&amp;gt;&#xA;    &amp;lt;/bndbox&amp;gt;&#xA;&amp;lt;/object&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I will believe that my object detector made something wrong? Or they tolerant some small delta such that if the bounding box has a small drifting, then it's acceptable. But if the &quot;label&quot; is totally wrong, then that's wrong for sure? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is like a &quot;blackbox&quot; to me and it would be great if someone can shed some lights on this. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-06-21T14:16:16.070" Title="From what aspect to measure the performance of an object detector?" Tags="&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12976" PostTypeId="2" ParentId="12974" CreationDate="2019-06-21T14:16:16.070" Score="1" Body="&lt;p&gt;This is a high level explanation: So most object detectors are generally deep neural networks that return a set of boxes: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;box1 : coordinates : confidence&#xA; box2 : coordinates : confidence&#xA; etc...&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and loss is computed different model to model. But i think your specifically curious how most do box comparison and loss. Generally they first check if the box is even relevant to the target. To do this often they calculate the Intersection over Union (IOU) as known as the Jaccard index &lt;a href=&quot;https://en.wikipedia.org/wiki/Jaccard_index&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia link&lt;/a&gt;. This checks how overlapped they are biased on the box sizes. Now if its above some threshold (example: boxes have IOU over .5 caluclate a loss, otherwise loss is just 0) they will compute some objective. Objectives can differ, but ideally its something differentiable. Examples include vectorizing and approximiating the IOU, dice loss, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a quick google search, heres a good tutorial explaining some components commonly used in the field (region proposals, feature extraction, etc) [&lt;a href=&quot;https://www.datacamp.com/community/tutorials/object-detection-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;guide&lt;/a&gt;]&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-21T14:16:16.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12977" PostTypeId="2" ParentId="12929" CreationDate="2019-06-21T14:38:36.937" Score="2" Body="&lt;p&gt;You might want to look into building convolutional neural network (CNN) for object detection using Keras. With plain white squares, it should work pretty good.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastActivityDate="2019-06-21T14:38:36.937" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12978" PostTypeId="2" ParentId="12971" CreationDate="2019-06-21T14:48:44.113" Score="12" Body="&lt;p&gt;In &lt;em&gt;artificial intelligence&lt;/em&gt; (sometimes called &lt;em&gt;machine intelligence&lt;/em&gt; or &lt;em&gt;computational intelligence&lt;/em&gt;), there are several problems that are based on mathematical topics, especially optimization, statistics, probability theory, calculus and linear algebra.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hutter1.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marcus Hutter&lt;/a&gt; has worked on a mathematical theory for &lt;em&gt;artificial general intelligence&lt;/em&gt;, called &lt;a href=&quot;https://jan.leike.name/AIXI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;AIXI&lt;/strong&gt;&lt;/a&gt;, which is based on several mathematical and computation science concepts, such as reinforcement learning, probability theory (e.g. Bayes theorem and related topics) &lt;a href=&quot;http://mathworld.wolfram.com/MeasureTheory.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;measure theory&lt;/a&gt;, &lt;a href=&quot;http://www.scholarpedia.org/article/Algorithmic_information_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithmic information theory&lt;/a&gt; (e.g. Kolmogorov complexity), optimisation, &lt;a href=&quot;https://wiki.lesswrong.com/wiki/Solomonoff_induction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solomonoff induction&lt;/a&gt;, &lt;a href=&quot;http://www.scholarpedia.org/article/Universal_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal Levin search&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Theory_of_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;theory of compution&lt;/a&gt; (e.g. universal Turing machines). His book &lt;a href=&quot;http://lib.slon.pp.ru/ComputerScience/Sequential%20Decisions%20Based%20On%20Algorithmic%20Probability.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability&lt;/a&gt;, which is a highly technical and mathematical book, describes his theory of optimal Bayesian non-Markov reinforcement learning agents. &lt;a href=&quot;https://ai.stackexchange.com/q/21427/2444&quot;&gt;Here&lt;/a&gt; I list other similar works.&lt;/p&gt;&#xA;&lt;p&gt;There is also the research field called &lt;a href=&quot;http://www.learningtheory.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;computational learning theory&lt;/strong&gt;&lt;/a&gt;, which is devoted to studying the design and analysis of machine learning algorithms (from a statistical perspective, known as &lt;a href=&quot;https://www.math.arizona.edu/%7Ehzhang/math574m/Read/vapnik.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;statistical learning theory&lt;/em&gt;&lt;/a&gt;, or algorithmic perspective, &lt;em&gt;algorithmic learning theory&lt;/em&gt;). More precisely, the field focuses on the rigorous study and mathematical analysis of machine learning algorithms using techniques from fields such as probability theory, statistics, optimization, information theory and geometry. Several people have worked on the &lt;a href=&quot;https://ai.stackexchange.com/q/20355/2444&quot;&gt;computational learning theory&lt;/a&gt;, including Michael Kearns and &lt;a href=&quot;https://en.wikipedia.org/wiki/Vladimir_Vapnik&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vladimir Vapnik&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is also a lot of research effort dedicated to approximations (heuristics) of &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial optimization&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/NP-completeness&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP-complete&lt;/a&gt; problems, such as &lt;a href=&quot;http://www.scholarpedia.org/article/Ant_colony_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;ant colony optimization&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is also some work on &lt;a href=&quot;https://en.wikipedia.org/wiki/AI-complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-completeness&lt;/a&gt;, but this has not received much attention (compared to the other research areas mentioned above).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-31T13:40:45.927" LastActivityDate="2021-01-31T13:40:45.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12979" PostTypeId="1" CreationDate="2019-06-21T14:52:35.263" Score="2" ViewCount="166" Body="&lt;p&gt;I have a question about search and planning:&#xA;I still haven't understood the difference from the two, but they seem very similar to me; here is a question I am struggling with: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Having formulated a PDDL problem, transform it into research,&#xA;  emphasizing what the differences are.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Someone can do an example? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attached an example of simple PDDL from my book (I'm using  Russell &amp;amp; Norvig)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/c8wG1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c8wG1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21719" LastEditorUserId="2444" LastEditDate="2021-02-06T13:36:00.620" LastActivityDate="2021-02-06T13:36:00.620" Title="How to transform a PDDL to search?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;planning&gt;&lt;pddl&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12980" PostTypeId="2" ParentId="12971" CreationDate="2019-06-21T15:27:52.157" Score="4" Body="&lt;p&gt;Most of the math work being done in AI that I'm familiar with is already covered in nbro's answer. One thing that I do not believe is covered yet in that answer is &lt;strong&gt;proving algorithmic equivalence&lt;/strong&gt; and/or &lt;strong&gt;deriving equivalent algorithms&lt;/strong&gt;. One of my favourite papers on this is &lt;a href=&quot;https://arxiv.org/abs/1508.04582&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Predict Independent of Span&lt;/a&gt; by Hado van Hasselt and Richard Sutton.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that we may first formulate an algorithm (in math form, for instance some update rules/equations for parameters that we're training) in one way, and then find different update rules/equations (i.e. a different algorithm) for which we can prove that it is equivalent to the first one (i.e. always results in the same output). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical case where this is useful is if the first algorithm is easy to understand / appeals to our intuition / is more convenient for convergence proofs or other theoretical analysis, and the second algorithm is more efficient (in terms of computation, memory requirements, etc.).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-06-21T15:27:52.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12982" PostTypeId="2" ParentId="9474" CreationDate="2019-06-21T16:17:27.343" Score="1" Body="&lt;p&gt;A* is an &lt;strong&gt;informed&lt;/strong&gt; search algorithm. A* &lt;em&gt;informed&lt;/em&gt; because it is based on the use of a &lt;em&gt;heuristic function&lt;/em&gt;, which estimates the distance of each node to the goal, that is, the heuristic function provides &lt;em&gt;information&lt;/em&gt; about the distance from any node to the goal node. The heuristic function can e.g. be the Euclidean distance (in the case this can be defined).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, at each step, A* needs to select the next node to explore. It chooses the node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; with the smallest value of &lt;span class=&quot;math-container&quot;&gt;$$f(n) = g(n) + h(n),$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the actual distance from the starting node to node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the heuristic function that estimates the distance from node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that we want to find the shortest path from Paris to Madrid. If you are already in Madrid, then you can estimate that the distance from Madrid to Madrid is zero, so &lt;span class=&quot;math-container&quot;&gt;$h(\text{Madrid}) = 0$&lt;/span&gt;. However, if you're still in Paris, what is the estimate of the distance from Paris to Madrid? We can e.g. pick up a map of the world and choose &lt;span class=&quot;math-container&quot;&gt;$h(\text{Paris})$&lt;/span&gt; to be the length of the straight segment that goes from Paris to Madrid. Similarly, we can do this for other intermediate cities. There are other ways of estimating this distance, but this is an understandable one, given that people are usually familiar with the Euclidean distance. (This example is a little bit misleading because the Earth is not flat and so the Euclidean geometry does not really apply, but, for simplicity, you can ignore this). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For info regarding the completeness and optimality of A*, have a look at &lt;a href=&quot;https://ai.stackexchange.com/a/8907/2444&quot;&gt;https://ai.stackexchange.com/a/8907/2444&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-21T20:25:45.000" LastActivityDate="2019-06-21T20:25:45.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12983" PostTypeId="2" ParentId="12972" CreationDate="2019-06-21T20:36:21.117" Score="2" Body="&lt;p&gt;Simply &lt;code&gt;model 1&lt;/code&gt; is a better fit compared to &lt;code&gt;model 2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Graph for model 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We notice that the training loss and validation loss aren't correlated. This means the as the training loss is decreasing, the validation loss remains the same of increases over the iterations. This means that the model is not exactly improving, but is instead overfitting the training data. This isn't what we are looking for.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Graph for model 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, there is clearly a health correlation between training loss and the validation loss. They both seem to reduce and stay at a constant value. This means that the model is well trained and is equally good on the training data as well as the hidden data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You should stick with model 2. In case you're going ahead with model 1, make sure to use the chechpoint where both the losses are at a similar value (at around 100 -150 epochs)&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-21T20:36:21.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12984" PostTypeId="2" ParentId="12973" CreationDate="2019-06-21T21:22:39.237" Score="0" Body="&lt;p&gt;By definition, tensors can be of any order (usually named differently if the order is less than three). So, I use &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; to indicate the dimensionality of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th facet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you have three or four-order tensors which each facet has a very specific meaning, naming the orders by terms such as &lt;em&gt;special&lt;/em&gt; or &lt;em&gt;time&lt;/em&gt; would be limiting.&lt;/p&gt;&#xA;" OwnerUserId="12853" LastActivityDate="2019-06-21T21:22:39.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12985" PostTypeId="1" AcceptedAnswerId="12994" CreationDate="2019-06-22T05:53:22.077" Score="1" ViewCount="338" Body="&lt;p&gt;I am quite new to Deep Reinforcement Learning, and I'm trying to define states in a Reinforcement Learning problem. The environment consists of multiple identical elements, and each one of them is characterized by different features of the same type. In other words, let us say we have &lt;span class=&quot;math-container&quot;&gt;$e_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$e_1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$e_2$&lt;/span&gt;. Then, suppose that each one is characterized by features &lt;span class=&quot;math-container&quot;&gt;$f_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f_0$&lt;/span&gt; belongs to &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; belongs to &lt;span class=&quot;math-container&quot;&gt;$\{0, 1, 2, 3, 4, 5\}$&lt;/span&gt;. Then, &lt;span class=&quot;math-container&quot;&gt;$e_0$&lt;/span&gt; will have some value for the features &lt;span class=&quot;math-container&quot;&gt;$f_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt;, and the same goes for &lt;span class=&quot;math-container&quot;&gt;$e_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How can I encode such states?&lt;/p&gt;&#xA;&lt;p&gt;Can I simply vectorize such state by concatenating the different features of each element obtaining &lt;span class=&quot;math-container&quot;&gt;$[f_{0e_0}, f_{1e_0}, f_{0e_1}, f_{1e_1}, f_{0e_2}, f_{1e_2}]$&lt;/span&gt;, or should I use a convolutional architecture of some sort?&lt;/p&gt;&#xA;" OwnerUserId="26605" LastEditorUserId="2444" LastEditDate="2020-10-31T15:26:13.137" LastActivityDate="2020-10-31T15:26:13.137" Title="How can I encode states where the environment consists of multiple identical elements, but each is characterised by different features?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;deep-rl&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="12986" PostTypeId="1" CreationDate="2019-06-22T06:55:32.347" Score="1" ViewCount="24" Body="&lt;p&gt;I am trying to train a SVR but I found that with some combination of features, the trained SVR predict every point in test set to the same value. this problem occurs much more when I use linear kernel than other kernels. The parameters are: C=1, gamma=0.5.&#xA;My question is what leads to this kind of problem. Is there a name for this phenomenon? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="26608" LastActivityDate="2019-06-22T06:55:32.347" Title="why my regression model predict every datapoint to the same value" Tags="&lt;machine-learning&gt;&lt;support-vector-machine&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12987" PostTypeId="1" CreationDate="2019-06-22T07:15:38.327" Score="1" ViewCount="55" Body="&lt;p&gt;I am trying to understand the average precision (AP) metrics in evaluating the performance of deep-learning based object detection models. Suppose we have the following &lt;strong&gt;ground true&lt;/strong&gt; (four objects highlighted by four blue arrows):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kbYCc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kbYCc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have labelled four objects:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;person 25 16 38 56&#xA;person 129 123 41 62&#xA;kite 45 16 38 56&#xA;kite 169 123 41 62&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And when feeding the above image to an object detector, it gives the following outputs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bLFSD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bLFSD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to see that the object detector identified another object with low confidence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;person 0.4 25 16 38 56&#xA;person 0.2 129 123 41 62&#xA;kite 0.3 45 16 38 56&#xA;kite 0.5 169 123 41 62&#xA;kite 0.1 769 823 141 162 &amp;lt;-------- a &quot;kite&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my humble opinion, this is an erroneous behavior of the object detector, which should be counted as a &quot;&lt;em&gt;false positive&lt;/em&gt;&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since the &quot;kite&quot; has a quite low confidence score (0.1), when using the standard &lt;strong&gt;&lt;code&gt;mAP&lt;/code&gt;&lt;/strong&gt; algorithm to compute the performance, I got the following output (I am using code from &lt;a href=&quot;https://github.com/rafaelpadilla/Object-Detection-Metrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to compute the &lt;code&gt;mAP&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AP: 100.00% (kite)&#xA;AP: 100.00% (person)&#xA;mAP: 100.00%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So here are my questions and confusions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;from what kind of design intension, the &lt;code&gt;AP&lt;/code&gt; is designed in a way such that objects with low confidence score are ignored and therefore in this case we are with flying colors. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any metrics that can take this extra &quot;kite&quot; into consideration and therefore would output one &quot;false positive&quot; of the object detection model? I am just thinking that in this way, we can further proceed to improve the accuracy of this model during training.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-06-22T07:15:38.327" Title="Understanding average precision (AP) in measuring object detector performance" Tags="&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12988" PostTypeId="2" ParentId="12971" CreationDate="2019-06-22T07:44:55.943" Score="3" Body="&lt;p&gt;Specifically for mathematical apparatus of Neural Networks - &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;random matrix theory&lt;/a&gt;.  &lt;a href=&quot;https://arxiv.org/abs/1011.3027&quot; rel=&quot;nofollow noreferrer&quot;&gt;Non-asymptotic random matrix theory&lt;/a&gt; was used in some &lt;a href=&quot;https://arxiv.org/abs/1811.08888&quot; rel=&quot;nofollow noreferrer&quot;&gt;proofs of convergence of gradient descent for Neural Networks&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1806.05294&quot; rel=&quot;nofollow noreferrer&quot;&gt;high dimensional random landscapes&lt;/a&gt; in connection to Hessian spectrum have relation to &lt;a href=&quot;https://arxiv.org/abs/1412.0233&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss surfaces of Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Topological_data_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Topological data analysis&lt;/a&gt; is another area of intense research related to ML, AI and &lt;a href=&quot;https://arxiv.org/abs/1905.12200&quot; rel=&quot;nofollow noreferrer&quot;&gt;applied to Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There were some works on &lt;a href=&quot;https://arxiv.org/abs/1805.07091&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tropical Geometry of Neural Networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Homotopy_type_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Homotopy Type Theory&lt;/a&gt; also have &lt;a href=&quot;https://andrewnetwork.github.io/minor/math/2017/07/29/hotti.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;connection to AI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-06-22T07:44:55.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12989" PostTypeId="1" CreationDate="2019-06-22T08:09:08.683" Score="2" ViewCount="92" Body="&lt;p&gt;I'm using a restricted Boltzmann machine (RBM) as an autoencoder. For now, I use a simple architecture of two layers, the input (~100 nodes) and the output (3 nodes) layers.  I'm thinking to add more hidden layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there some improvements in encoding by adding multiple hidden layers? If yes, how can multiple layers improve the encoding?&lt;/p&gt;&#xA;" OwnerUserId="26577" LastEditorUserId="2444" LastEditDate="2020-05-18T12:19:29.957" LastActivityDate="2020-05-18T12:19:29.957" Title="Does the encoding of a restricted Boltzmann machine improve with more layers?" Tags="&lt;hidden-layers&gt;&lt;boltzmann-machine&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12990" PostTypeId="1" CreationDate="2019-06-22T12:44:12.213" Score="2" ViewCount="102" Body="&lt;p&gt;One of if not the most popular programming language for data science and AI today is Python, with R being a frequently cited runner-up. However, both of them are interpreted languages, which do not execute as fast as compiled languages. Why is that the case? The main advantage of AI over humans is in computing speed, and with real world AI applications today handling big data, execution time will already increase considerably as it is. Why aren't compiled languages preferred for this very reason?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure, the key argument and strength going for Python is its vast range of third party libraries available for AI, like scikit etc. but such communities can take root and grow anywhere under the right circumstances. Why did this community end up growing around Python and not a faster, equally common compiled language like C++, C# or Java?&lt;/p&gt;&#xA;" OwnerUserId="23844" LastActivityDate="2019-06-22T19:02:34.507" Title="Why aren't compiled languages as popular as Python in AI?" Tags="&lt;programming-languages&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2020-05-04T12:20:15.187" ContentLicense="CC BY-SA 4.0" />
  <row Id="12991" PostTypeId="1" AcceptedAnswerId="12995" CreationDate="2019-06-22T13:44:12.233" Score="1" ViewCount="1965" Body="&lt;p&gt;While studying artificial intelligence, I have often encountered the term &quot;agent&quot; (often autonomous, intelligent). For instance, in fields such as Reinforcement Learning, Multi-Agent Systems, Game Theory, Markov Decision Processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an intuitive sense, it is clear to me what an agent is; I was wondering whether in AI it had a rigorous definition, perhaps expressed in mathematical language, and shared by the various AI-related fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is an agent in Artificial Intelligence?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2019-06-22T17:04:26.697" LastActivityDate="2019-06-22T18:59:53.643" Title="What is an agent in Artificial Intelligence?" Tags="&lt;ai-basics&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;theory&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12992" PostTypeId="2" ParentId="12990" CreationDate="2019-06-22T13:51:46.040" Score="2" Body="&lt;p&gt;There are a few advantages of interpreted languages (compared to compiled languages)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;platform independence (you only need the interpreter for your platform, even though this is not true if e.g. your interpreted language is only a &lt;a href=&quot;https://en.wikipedia.org/wiki/Wrapper_library&quot; rel=&quot;nofollow noreferrer&quot;&gt;wrapper library&lt;/a&gt; around a library written in another programming language)&lt;/li&gt;&#xA;&lt;li&gt;dynamic typing (no need to specify the types of the variables)&lt;/li&gt;&#xA;&lt;li&gt;dynamic scoping (e.g. you can access variables in other scopes)&lt;/li&gt;&#xA;&lt;li&gt;automatic memory management (but there are compiled languages, like Java, that also have a garbage collector)&lt;/li&gt;&#xA;&lt;li&gt;Rapid prototyping (for various reasons, including dynamic typing), hence software can be written more quickly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hence, the main advantages of an interpreted language compared to a compiled language are &lt;em&gt;flexibility&lt;/em&gt; and &lt;em&gt;dynamism&lt;/em&gt;. Given that AI is still an evolving field, these characteristics are widely appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also at least one disadvantage of interpreted languages (compared to compiled languages)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Slower running times compared to compiled languages, which, once compiled, are quite fast, because they are often compiled to a code that is quickly executable by the machine or virtual machine&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Python and R are widely used in data science and artificial intelligence because of the advantages above, which possibly contributed to the rapid growth of the communities around them and the development of software libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, note that the core of the most common machine learning libraries today, including TensorFlow and PyTorch, is written in a compiled language like C and C++. In the specific case of TensorFlow, Python is just a wrapper library. Consequently, under the hood, the code is not executed by the Python interpreter, but first compiled, which implies that, when you're using e.g. TensorFlow, your code will run (more or less) as fast as if you were using a compiled language like C. A similar argument can be made for libraries like NumPy, where Python is just a wrapper library.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-22T19:02:34.507" LastActivityDate="2019-06-22T19:02:34.507" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12994" PostTypeId="2" ParentId="12985" CreationDate="2019-06-22T15:21:22.690" Score="1" Body="&lt;p&gt;For general advice about state representation, you could check my answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/7763/how-to-define-states-in-reinforcement-learning/7771#7771&quot;&gt;How to define states in reinforcement learning?&lt;/a&gt; - this does not cover your specific issue, but may help with other details such as whether to one-hot-encode and/or scale your discrete features. Also you will want to assess whether any state vector you construct actually contains &lt;em&gt;enough&lt;/em&gt; data for reinforcement learning to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that is all good, then your approach here:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can I simply vectorize such state by concatenating the different features of each element obtaining &lt;span class=&quot;math-container&quot;&gt;$[f_{0e_0}, f_{1e_0}, f_{0e_1}, f_{1e_1}, f_{0e_2}, f_{1e_2}]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;should work with a generic feed-forward neural network with two or more layers. As this is simple to construct, then it is a reasonable place to start just considering development time. If nothing else it can be a benchmark against other representations and model architectures that you may want to try.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;should I use a convolutional architecture of some sort?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That will depend on any structure inherent in your environment. The &lt;em&gt;interesting&lt;/em&gt; thing about your description of the environment is the repetition of &lt;em&gt;meaning&lt;/em&gt; in the values. Each entity &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; appears multiple times, and the units of feature &lt;span class=&quot;math-container&quot;&gt;$f_{0e_0}$&lt;/span&gt; will be the same as &lt;span class=&quot;math-container&quot;&gt;$f_{0e_1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An unfortunate consequence of using a flattened vector is that the information about these repeated entities and their units is lost to the system. Any important details due to the entities being essentially the same of this must be &quot;rediscovered&quot; by the approximator training with many examples. Although neural networks can generalise function results, this does not apply to generalising across different parts of the input unless you choose an architecture designed to help with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically you would use a RNN architecture (such as LSTM) if key information is in the &lt;em&gt;sequence order&lt;/em&gt; of entities, or a CNN architecture if key information is in local patterns between neighbouring entities. By &quot;key information&quot; I mean whether useful relationship to a value function or optimal policy depends on those factors. This might be difficult to assess for some environments with mixed data types and complex meanings for the entity ids - so you may have to experiment with multiple architectures to figure out what factor is more important. However, it is also possible you have some insight based on known good policies, so can head more directly to a promising feature representation and architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learning may benefit if instead of ordering your entities arbitrarily by identity in the vector, that you sort them according to some factor that is relevant to the goals of your agent. I have used this is in a colour matching environment where an agent had to hit a target with a &quot;current&quot; colour selection whilst avoiding incorrect colours - sorting target entities by whether they matched colour and then by angular distance to the agent's aim enabled the agent to learn its task far more efficiently. This sorting may help independently of architecture choice - i.e. you don't need to be using a RNN or CNN to see a benefit with this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing worth considering, but probably not possible here because of the continuous feature &lt;span class=&quot;math-container&quot;&gt;$f_0$&lt;/span&gt;, is whether you can invert the representation so that instead of being based on entities with properties, it is a view over property space populated with entities in certain positions. A CNN processed over a small enough &quot;property space&quot; instead of &quot;entity space&quot; may well perform better - this is why board game RL tends to treat the board positions as defining feature vector indexes which represent which entities are present on them, which is a very sparse structure that can represent far more states than ever occur in real games. The over-specified space works because it can be matched to architectures that generalise efficiently over it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you need to consider different types of symmetry within the system - especially what is meaningful about the id number you have assigned to each entity. If the entity id is not meaningful, or is secondary to some other factor, you have some flexibility in changing the representation to make it meaningful and use architectures that take advantage of how you have injected that domain knowledge into the q function approximator. If the id is meaningful, think how could you best represent the difference between entities with different ids in your environment?&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-06-23T07:30:33.110" LastActivityDate="2019-06-23T07:30:33.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12995" PostTypeId="2" ParentId="12991" CreationDate="2019-06-22T16:20:37.013" Score="3" Body="&lt;p&gt;The acclaimed book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (by Stuart Russell and Peter Norvig) gives a definition of an agent&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This definition is illustrated by the following figure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qIpUW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qIpUW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition (and illustration) of an agent thus does not seem to include the agent as part of the environment, but this is debatable and can be a limitation, given that the environment also includes the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to this definition, humans, robots, and programs are agents. For example, a human is an agent because it possesses sensors (e.g. the eyes) and actuators (e.g. the hands, which, in this case, are also sensors) and it interacts with an environment (the world). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;percept&lt;/strong&gt; (or perception) is composed of all perceptual inputs of the agents. The specific definition of the percept changes depending on the specific agent. For example, in the case of a human, the percept consists of all perceptual inputs from all sense organs of the human (eyes, ears, tongue, skin, and nose). In the case of a robot only equipped with a camera, the percept consists only of the camera frame (at a certain point in time). A &lt;strong&gt;percept sequence&lt;/strong&gt; is a sequence of percepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;strong&gt;action&lt;/strong&gt; is anything that has an effect on the environment. For example, in the case of a legged robot, an action can be &quot;move forward&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An action is chosen by the &lt;strong&gt;agent function&lt;/strong&gt; (which is illustrated by the white box with a black question mark in the figure above), which can also be called &lt;strong&gt;policy&lt;/strong&gt;. The agent function highly determines the intelligent or intellectual capabilities of the agent and differentiates it from other agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, there are different agents depending on the sensors and actuators they possess, but, more importantly, depending on their policy, which highly affects their intellectual characteristics. A possible categorization of agents is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;rational&lt;/strong&gt; agents do the &quot;right&quot; thing (where &quot;right&quot;, of course, depends on the context)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;simple reflex&lt;/strong&gt; agents select actions only based on the current percept (thus ignoring previous percepts)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;model-based reflex&lt;/strong&gt; agents build a model of the world (sometimes called a &lt;em&gt;state&lt;/em&gt;) that is used to deal with cases where the current percept is insufficient to take the most appropriate action&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;goal-based&lt;/strong&gt; agents possess some sort of &lt;em&gt;goal information&lt;/em&gt; that describes situations that are desirable; for example, in the case of a human, a situation that is desirable is to have food&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;utility-based&lt;/strong&gt; agents associate value with certain actions more than others; for example, if you need immediate energy, chocolate might have more value than some vegetable&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;learning&lt;/strong&gt; agents update their e.g. model based on the experience or interaction with the environment&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More details regarding these definitions can be found in section 2 of the book mentioned above (3rd edition). However, note that there are other possible categorizations of agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;reinforcement learning&lt;/strong&gt; (RL) agent (which is a program) is an agent that interacts with an &lt;em&gt;environment&lt;/em&gt; and can learn a policy (a behavioral strategy or agent function) or value (or utility) function (from which the policy can be derived) from this interaction, where the agent takes an &lt;em&gt;action&lt;/em&gt; from the &lt;em&gt;current state&lt;/em&gt; (or model) of the environment and the environments emits a percept, which, in the case of RL, consists of a &lt;em&gt;reinforcement&lt;/em&gt; (or &lt;em&gt;reward&lt;/em&gt;) signal and the &lt;em&gt;next state&lt;/em&gt;. The goal of the RL agent is to maximize the cumulative reward (or reinforcement) signal. An RL agent can thus be considered a rational, goal, utility-based and learning agent. It can also be (or not) a simple reflex and model-based agent.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-22T18:59:53.643" LastActivityDate="2019-06-22T18:59:53.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12996" PostTypeId="2" ParentId="8707" CreationDate="2019-06-22T17:14:06.723" Score="1" Body="&lt;p&gt;A learning agent is any entity that, over time, improves its performance (which can be defined in different ways depending on the context) based on the interaction with the environment (or experience). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The human is an example of a learning agent. For example, a human can learn to ride a bicycle, even though, at birth, no human possesses this skill.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-22T17:14:06.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12997" PostTypeId="1" CreationDate="2019-06-22T17:43:09.073" Score="-1" ViewCount="520" Body="&lt;p&gt;I'm using a DQN Algorithm to play Snake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input of the neural network is a stack of 4 images taken from the games 80x80.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is an array of 4 values, one for every direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the program does not converge and I've a lot of doubts in the replay function, where I train the neural network over a batch of 32 events.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's the snippet:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def replay(self, batch_size):&#xA;&#xA;    minibatch = random.sample(self.memory, batch_size)&#xA;&#xA;    for state, action, reward, next_state, done in minibatch:&#xA;&#xA;        target = reward&#xA;&#xA;        if not done:&#xA;            target = (reward + self.gamma *&#xA;                      np.amax(self.model.predict(next_state)[0]))&#xA;        target_f = self.model.predict(state)&#xA;        target_f[0][action] = target&#xA;        self.model.fit(state, target_f, epochs=1, verbose=0)&#xA;&#xA;    if self.epsilon &amp;gt; self.epsilon_min:&#xA;        self.epsilon *= self.epsilon_decay`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Targets are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;+1&lt;/strong&gt; for eating an apple&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;0&lt;/strong&gt; for doing a movement without dying&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;-1000&lt;/strong&gt; for hitting a wall or the snake hitting himself&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26623" LastEditorUserId="26384" LastEditDate="2019-06-22T19:04:30.020" LastActivityDate="2019-07-20T12:56:04.373" Title="Problem over DQN Algorithm not converging on snake" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13000" PostTypeId="2" ParentId="12997" CreationDate="2019-06-22T18:56:11.383" Score="0" Body="&lt;p&gt;I think the main issue here is that you are trying to train the &lt;code&gt;snake&lt;/code&gt; (network) on images. This will create a lot of issues a there are no set parameters that the model can learn from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From images, there is no logical way to define the boundary, directions and objects on the board. It will be much easier to write a simple computer vision script of game API to provide actual meaningful inputs to the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/today-im-going-to-talk-about-a-small-practical-example-of-using-neural-networks-training-one-to-6b2cbd6efdb3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a great article on building a model to play the snake game. The author also provides the game API for input along with example code to train the snake game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final results from the model&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/du7ZI.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/du7ZI.gif&quot; alt=&quot;Snake agent results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2019-06-22T18:56:11.383" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13003" PostTypeId="1" CreationDate="2019-06-23T00:18:42.423" Score="1" ViewCount="18" Body="&lt;p&gt;CNNs are often used in one of the following scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A known-sized image is encoded to an intermediate format for later use&lt;/li&gt;&#xA;&lt;li&gt;An intermediate or precursor format is decoded into a known-sized image&lt;/li&gt;&#xA;&lt;li&gt;An image is converted into a same-size image&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Usually 3 is done by sticking together 1 and 2.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any papers dealing with convolutional techniques where the image sizes vary?  Not only would the size of input X differ from input Y, but also input X may differ from output Y. The total amount of variation can probably be constrained by the statistics of the dataset, but knowledge of input X does not grant &lt;em&gt;a priori&lt;/em&gt; knowledge of the size of output Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Masking is an obvious solution, but I am hoping for something more elegant if research already exists.  The problem domain need not be images.) &lt;/p&gt;&#xA;" OwnerUserId="15020" LastActivityDate="2019-06-23T01:49:50.273" Title="Convolutional Neural Networks for different-sized Source and Target" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13004" PostTypeId="2" ParentId="13003" CreationDate="2019-06-23T01:49:50.273" Score="2" Body="&lt;p&gt;Yes actually. There have been quite a few different adaptations to convolutional neural networks to do precisely what you are describing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1512.07108&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is an earlier one.&lt;/a&gt; See section 3.2.5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here He et al. create a method known as Spatial Pyramid Pooling(SPP). In this method, you are able construct a fixed length representation regardless of input size by pooling them into &quot;spatial bins&quot; which are proportional to the input size, and thusly do not need to modify input dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few newer methods that improve on this in various capacities, as usual, your solution will be dependent on the problem and other situation-dependent constraints. I suggest you dig deeper into the literature to find the optimal solution in your case.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-06-23T01:49:50.273" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13005" PostTypeId="2" ParentId="12953" CreationDate="2019-06-23T02:39:31.800" Score="1" Body="&lt;p&gt;Quite an interesting question, this sort of computational representation of biological systems are at the forefront of what we are trying to accomplish algorithmically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be tackled via a variety of coordinate frames and problem formulations and could be thought of quite philosophically, as it pokes at some deep questions such as free-will and consciousness more generally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we could take the assumption that based on everything we know physically, the universe is completely deterministic. In this case, implementing deterministic functions for these variables makes quite a bit of sense and seems like the obvious choice. However, we can easily imagine that for at least some of this dynamic variables a stochastic representation would be both simpler(computationally) as well as giving better approximations to what we see in the real-world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it really is a question of how much value you see from implementing a system like this, and which representation you think would be most effective. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-06-23T02:39:31.800" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13006" PostTypeId="2" ParentId="12926" CreationDate="2019-06-23T02:53:33.667" Score="0" Body="&lt;p&gt;Without digging into your diagnostics more deeply, on it's face this seem like a local optima issue. Assuming you are optimizing via GD, there are many local optima that a network or agent in this case can converge to and stay at which will cause symptoms like seeing above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that being said, assuming this is our issue, here are some things you can try:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Regularization, try adding dropout or L2 and see how that affects convergence and learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adjust network architecture, number of layers, nodes, etc. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try a different type of RL(Q learning for example), this will be dependent on your problem of course.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adjust starting seed. Assuming you have a static seed used for weight initialization, you will always converge to the same solution. It could be as simple as adjusting the seed value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If all these steps fail, you likely have a deeper issue at work, and I would suggest coming back after with some additional detail if this does not succeed.  &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-06-23T02:53:33.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13007" PostTypeId="1" CreationDate="2019-06-23T06:52:46.737" Score="1" ViewCount="39" Body="&lt;p&gt;Think Angry Birds kind of game. You need to hit a target at some point by adjusting angle and power. There is infinite number of parabolas that will solve this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is not exactly that but similar, it also has infinite number of solutions. Could anyone please suggest how do I approach this kind of problems using Machine Learning?&lt;/p&gt;&#xA;" OwnerUserId="26632" LastActivityDate="2019-06-23T06:52:46.737" Title="How to approach a problem with infinite solutions" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13008" PostTypeId="1" CreationDate="2019-06-23T14:44:20.287" Score="1" ViewCount="52" Body="&lt;p&gt;Generally, we come across terms such as High Frequency and Low frequency filters in Convolutional Neural Networks (CNN). In regards to this highlighted statement, in 'S1' section of &lt;a href=&quot;http://yosinski.com/media/papers/Yosinski__2015__ICML_DL__Understanding_Neural_Networks_Through_Deep_Visualization__.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper by Jason Yosinski (ref 1), &lt;a href=&quot;https://i.stack.imgur.com/5eE9s.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5eE9s.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; I thought, in order for high and low-frequency filters to produce a similar effect, weight of low-frequency filters should be greater than high-frequency filters. I would like to understand why I am wrong and I will be grateful if anyone can elaborate about High and Low frequency filters in CNN, in general, or in this context. Thank you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref 1: Yosinski, Jason, et al. &quot;Understanding neural networks through deep visualization.&quot; arXiv preprint arXiv:1506.06579 (2015).&lt;/p&gt;&#xA;" OwnerUserId="26639" LastActivityDate="2019-06-23T14:44:20.287" Title="How High and Low frequency filters effect activation in the next layer?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13009" PostTypeId="1" CreationDate="2019-06-23T22:06:15.223" Score="1" ViewCount="913" Body="&lt;p&gt;I need to develop a convolutional neural network whose inputs are 1-channel images, but I dont know how to do it, given that most libraries use 3 channel images. Should I convert my images to RGB? Is there any way to implement a CNN that receive as input 1-channel images?&lt;/p&gt;&#xA;" OwnerUserId="26644" LastEditorUserId="2444" LastEditDate="2019-06-24T00:09:32.280" LastActivityDate="2019-11-21T04:04:30.567" Title="How can I use 1-channel images as input to a CNN?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;supervised-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13010" PostTypeId="1" CreationDate="2019-06-23T23:00:57.680" Score="2" ViewCount="327" Body="&lt;p&gt;I have a bunch of images from different trucks passing the road. Here is an example.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3mHXL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3mHXL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The truck needs to be at a certain distance from the border of the lane. Some of the trucks are way close to the border (that you can see on the shoulder of the road).&lt;/p&gt;&#xA;&lt;p&gt;I want to find a way to measure the distance between the truck and the border of the lane and, more importantly, to detect whether a truck is inside its lane.&lt;/p&gt;&#xA;&lt;p&gt;I would like to solve this problem by training a deep learning-based classifier or image processing techniques. Painting the ground is also possible if I can train a classification algorithm with painted images.&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="2444" LastEditDate="2020-12-11T15:29:44.580" LastActivityDate="2020-12-11T15:29:44.580" Title="How do I determine whether a truck is inside its lane?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;applications&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13011" PostTypeId="5" CreationDate="2019-06-23T23:58:25.887" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Digital_image_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Digital_image_processing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-23T23:58:25.887" LastActivityDate="2019-06-23T23:58:25.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13012" PostTypeId="4" CreationDate="2019-06-23T23:58:25.887" Score="0" Body="For questions related to image processing (in the context of AI)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-23T23:58:25.887" LastActivityDate="2019-06-23T23:58:25.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13013" PostTypeId="2" ParentId="13009" CreationDate="2019-06-24T00:20:26.657" Score="1" Body="&lt;p&gt;The libraries should allow you to specify the number of input channels of the convolutional layer, so no one should prevent you from passing 1-channel images as input to a CNN. For example, in PyTorch, you can specify the number of input channels of the &lt;a href=&quot;https://pytorch.org/docs/stable/nn.html#conv2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Conv2d&lt;/code&gt;&lt;/a&gt; object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your library does not provide such feature, you could convert your 1-channel images to 3-channel images, where e.g. all 3 channels of each image are equal to the only channel of the corresponding original 1-channel image.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-24T00:20:26.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13014" PostTypeId="2" ParentId="3176" CreationDate="2019-06-24T01:12:08.950" Score="1" Body="&lt;p&gt;These are correct. Artificial implies that it runs on artifically made hardware. There is no reason to distinguish between the natural processes what do the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, the term &lt;em&gt;intelligence&lt;/em&gt; is nor realy precise. What is more/ less intelligent or has or has not intelligence among : mowgli, monkey, crow, common game bot, whatever? MAing thing, some learning on data happens here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best alternative would be Mashine learning, but again, that mashine like  &quot;artifically made stuff&quot; does that is irrelevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my definition is:&#xA; Algorithmic Learning.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-06-24T01:12:08.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13015" PostTypeId="2" ParentId="13009" CreationDate="2019-06-24T01:12:58.873" Score="1" Body="&lt;p&gt;If you look at the theory of CNN, the no.of channels in the input layer is also a parameter that user can decide. In fact, if you are working on monochrome (black &amp;amp; white) images, you have to use only one channel in the input layer. All the libraries should provide a way to design an input layer with no. of channels as an option. But, if you are trying to use transfer learning by using some model in the database, then you may be restricted to use 3 channel input because the model is designed so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no need to convert your images to RGB, surely there should be a way provided by the library you are using to use only 1 channel input. Or if you are writing your code, you can design your CNN with only 1 channel input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3 channels is not a rule in CNN&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-06-24T01:12:58.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13016" PostTypeId="2" ParentId="96" CreationDate="2019-06-24T01:22:41.790" Score="1" Body="&lt;p&gt;General structure of an Artificial Neural Network&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input Layer + Hidden Layers + Output Layer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are more hidden layers in the artificial neural network, then the neural network is called as Deep Neural network. How many exactly constitute a deep neural network is a point of debate, but in general, the more the hidden layers, the deep is the neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to why they are so popular or important, many problems like object detection, classification, face recognition, speech recognition got solved with the advent of deep neural networks. It's not a exaggeration to say that, the performance of deep neural networks crossed even the human performance in many of the above mentioned tasks. That means now a computer is the best one to do the above tasks than humans. All the above mentioned problems were lying in research field since almost 5 decades. All of them have been solved to perfection only in the last 4,5 years just because of the success of deep neural networks. That is why they are very popular and important. I mentioned very few problems that i worked on, there are many similar tasks that deep neural networks solved with ease in the last decade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, at this point in time, many people across the world are working on solving innumerable applications using deep neural networks.&lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-06-24T01:22:41.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13017" PostTypeId="1" CreationDate="2019-06-24T05:55:29.570" Score="1" ViewCount="72" Body="&lt;p&gt;Hello I am trying to implement a CRNN with multiple input images (in my context it is 6 images)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tuGN7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tuGN7.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a regression problem and output is two real value.&#xA;And for the CNN block I am thinking of using ResNet50 (identical CNN block to handle 6 input images). Just an abstract code for model architecture and input dimension will be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="19424" LastActivityDate="2019-06-24T05:55:29.570" Title="Keras CRNN implementation with multiple input images" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13018" PostTypeId="1" CreationDate="2019-06-24T07:52:28.370" Score="1" ViewCount="87" Body="&lt;p&gt;Knowledge Representation and Automated Reasoning are two AI subfields which seem to have something to do with reasoning. &#xA;However, I can't find any information online about their relationship. Are they synonyms? Is KR a subfield of AR? What's the difference, if any?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To expand further, I believe representing knowledge is an essential part of &quot;reasoning&quot;: how can you reason without a proper representation of the concepts you want to manipulate?&#xA;At the same time, reasoning seems to be an essential part of KR (we build Knowledge Bases in order to build computer programs which are able to make inferences from them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it seems to me that they are the same field, or at least deeply interrelated, but nobody on the internet seems to explicitly say that; furthermore, in &lt;a href=&quot;https://ai.stackexchange.com/questions/5782/were-the-requirements-to-solve-the-turing-test-in-ai-a-modern-approach-forese&quot;&gt;this&lt;/a&gt; question, they are mentioned separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point of ambiguity is that the wikipedia page of &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;KRR&lt;/a&gt; mentions reasoning and automated reasoning as a part of KR; it even lists &quot;automated theorem proving&quot; (a classical application of AR) as an application of KR.&#xA;But at the same time, we have a separate &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;AR page&lt;/a&gt; which does not mention KR at all.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="2444" LastEditDate="2019-06-24T10:57:39.473" LastActivityDate="2019-06-24T10:57:39.473" Title="What is the difference between Knowledge Representation and Automated Reasoning?" Tags="&lt;terminology&gt;&lt;comparison&gt;&lt;knowledge-representation&gt;&lt;automated-reasoning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13020" PostTypeId="5" CreationDate="2019-06-24T08:42:00.820" Score="0" Body="&lt;p&gt;Automated Reasoning (AR) is a sub-field of Artificial Intelligence concerned with the development of computer programs which can reason completely, or nearly completely, automatically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of applications are automated theorem proving, proof checking, logic programming, circuit design and reasoning under uncertainity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tools used in AR include formal logic, fuzzy logic, bayesian inference and other formal &lt;em&gt;ad-hoc&lt;/em&gt; techniques. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the first example of successful AR is Logic Theorist (LT), a computer program developed in 1956 by Allen Newell, Cliff Shaw and Herbert A. Simon, which eventually proven 38 of the first 52 theorems in Whitehead and Russell's Principia Mathematica, and find new and more elegant proofs for some. It is called &quot;the first artificial intelligence program&quot;.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-06-24T19:59:08.320" LastActivityDate="2019-06-24T19:59:08.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13021" PostTypeId="4" CreationDate="2019-06-24T08:42:00.820" Score="0" Body="For questions about Automated Reasoning concepts and research topics. Automated Reasoning is a sub-field of Artificial Intelligence concerned with the development of computer programs which can reason completely, or nearly completely, automatically." OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-06-24T19:59:13.117" LastActivityDate="2019-06-24T19:59:13.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13022" PostTypeId="2" ParentId="13010" CreationDate="2019-06-24T08:43:05.330" Score="2" Body="&lt;p&gt;One possible approach will be to use an algorithm which detects lines (Ex. Hough lines or any deep neural net trained to detect lanes) and use some threshold range so that we can get the lane and the edges of truck, then after extracting the lines, you can easily find the distance between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you need to experiment out on few images to get the threshold distance that you are expecting the truck to maintain as the real distance and the distance calculated using images are not same &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you want to classify using deep learning, you may need to preprocess the images and send them. As it will become very difficult to directly learn to classify &#xA;based on image, you may need to first detect the lanes, then apply a mask and then send the masked image to your network to make the network to converge. &lt;/p&gt;&#xA;" OwnerUserId="26648" LastActivityDate="2019-06-24T08:43:05.330" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13025" PostTypeId="2" ParentId="8650" CreationDate="2019-06-24T10:08:21.777" Score="1" Body="&lt;p&gt;In some sense yes, in some sense no. The &quot;error&quot; word you are using is not defined in the question. The network can have two type of errors, quite different - &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalization_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalization error&lt;/a&gt; and loss (training error). The lower bound of loss on the dataset obviously depend on the network architecture, because loss is value produced by network as a whole, and some network will be able to produce exactly zero loss (which is likely be overfitting, wich is bad).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generalization error is how well network generalize form training dataset to potentially infinite (or very big) amount of data samples &quot;in the wild&quot;, so generalization error can not be calculated precisely, only estimated. The lower bound on generalization error is studied in Statistical Learning. It always depend on the complexity of the classifier (in our case architecture of network). Statistical Learning theory produce estimations like &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC generalization bound&lt;/a&gt; which connect generalization error with size of the dataset and complexity of the classifer (netwroks in our case) space. &lt;em&gt;However for specific dataset it probably don't make sense to talk about generalization bound outside of context of complexity of classifier space&lt;/em&gt;, because if there is no restriction we can always construct bigger network which would have lesser generalization error (for example by pretraning bigger network on bigger dataset)&#xA;To put it short you have to put restriction on complexity of network space to be able estimate minimum generalization error (and that probably wouldn't be useful anyway, because it wouldn't say anything about if gradient descent would reach that error)&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-06-24T12:17:14.690" LastActivityDate="2019-06-24T12:17:14.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13026" PostTypeId="1" CreationDate="2019-06-24T11:13:58.117" Score="1" ViewCount="382" Body="&lt;p&gt;When using neural networks (NNs), we often normalized the inputs. I think this is done to equally capture the changes in any input feature, that is, if any feature takes huge values and other features take small values, we don't want the NN not to be able to &amp;quot;see&amp;quot; the change in the smaller value.&lt;/p&gt;&#xA;&lt;p&gt;However, what if we cause the NN to become insensitive to the input, that is, the NN is not able to identify changes in the input because the changes are too small?&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="2444" LastEditDate="2020-11-15T18:44:31.843" LastActivityDate="2021-04-14T21:53:50.547" Title="Could the normalisation of the inputs make the neural network insensitive to changes in the inputs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;data-preprocessing&gt;&lt;normalisation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13028" PostTypeId="1" AcceptedAnswerId="13035" CreationDate="2019-06-24T11:38:31.863" Score="2" ViewCount="64" Body="&lt;p&gt;Some pictures contain an elephant, others don't. I know which of the pictures contain the elephant, but I don't know where it is or how does it look like.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I make a neural network which locates the elephant on a picture if it contains one? There are no pictures with more than one elephant.&lt;/p&gt;&#xA;" OwnerUserId="5852" LastEditorUserId="2444" LastEditDate="2019-06-24T13:50:29.510" LastActivityDate="2019-06-24T17:51:57.650" Title="How do I locate a specific object in an image?" Tags="&lt;deep-learning&gt;&lt;object-detection&gt;&lt;semi-supervised-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13029" PostTypeId="2" ParentId="13026" CreationDate="2019-06-24T11:52:44.973" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;When using neural networks (NNs), we often normalized the inputs. I&#xA;think this is done to equally capture the changes in any input&#xA;feature, that is, if any feature takes huge values and other features&#xA;take small values, we don't want the NN not to be able to &amp;quot;see&amp;quot; the change&#xA;in the smaller value.&lt;/p&gt;&#xA;&lt;p&gt;However, what if we cause the NN to become insensitive to the input,&#xA;that is, the NN is not able to identify changes in the input&#xA;because the changes are too small?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We &lt;strong&gt;don't&lt;/strong&gt; normalize the input to make the model less sensitive to small changes in the input (theoretically, given the correct optimization strategy, the model will learn to approximate the smaller-ranged input as well).&lt;/p&gt;&#xA;&lt;p&gt;An example of this would be Convolutional Neural Networks. Traditionally, images were represented with integer values ranging from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$255$&lt;/span&gt;. This means that a given pixel could have &lt;strong&gt;only &lt;span class=&quot;math-container&quot;&gt;$256$&lt;/span&gt; distinct values&lt;/strong&gt;. However, assuming we normalize the input, let's say to &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, this gives the pixel a whole range of values to occupy, making the input &lt;em&gt;more sensitive to changes.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Instead, normalization is done to help with the model's &lt;strong&gt;convergence&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="2444" LastEditDate="2020-11-15T18:39:57.427" LastActivityDate="2020-11-15T18:39:57.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13030" PostTypeId="2" ParentId="1507" CreationDate="2019-06-24T13:05:16.287" Score="2" Body="&lt;p&gt;There is also the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI effect&lt;/a&gt;, that is, the tendency to not consider something an AI once it is well understood. For example, neural networks are not yet fully understood, so people still tend to call them AI. Once we know exactly all the details about neural networks and their inner workings, we might start to consider them just &lt;em&gt;computation&lt;/em&gt;. This is an old philosophical topic that goes back at least to the famous Jacques de Vaucanson's &lt;a href=&quot;https://web.stanford.edu/dept/HPS/DefecatingDuck.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;defecating duck&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Jacquard_loom&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic loom&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-24T13:05:16.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13031" PostTypeId="1" AcceptedAnswerId="13177" CreationDate="2019-06-24T13:51:43.990" Score="1" ViewCount="87" Body="&lt;p&gt;Actually, I am &quot;fresh-water&quot;, and I've never known what is neural network. Now I am trying understand how to design simple neuronetwork for this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to make up such neural network that after learning it could predicate a mark of passed exam (for example, math). There is such factors that influence on a mark:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chosen topic (integral, derivative, series)&lt;/li&gt;&#xA;&lt;li&gt;Perfomance (low, medium, high)&lt;/li&gt;&#xA;&lt;li&gt;Does a student work? (Yes, No, flexible schedule)&lt;/li&gt;&#xA;&lt;li&gt;Have a student ever gotten through a add course? (Yes, No)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output is a mark (&lt;em&gt;A,B,C,D,E,F&lt;/em&gt;)&#xA;I don't know should I add few layers between inputs and output&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ChZiH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ChZiH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Moreover, I have few results from past years:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(integral, low, Yes, No, &lt;strong&gt;E&lt;/strong&gt;) &lt;/li&gt;&#xA;&lt;li&gt;(integral, medium, Yes, Yes, &lt;strong&gt;B&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(series, high, No, Yes, &lt;strong&gt;A&lt;/strong&gt;)&#xA;and so on. What do I need to know else for designing this NN?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26653" LastActivityDate="2020-07-26T04:51:23.110" Title="How to create neural network that predicates result of exam?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13032" PostTypeId="5" CreationDate="2019-06-24T13:54:00.977" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T17:40:19.937" LastActivityDate="2020-06-16T17:40:19.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13033" PostTypeId="4" CreationDate="2019-06-24T13:54:00.977" Score="0" Body="For questions related to object detection (where objects can be e.g. humans, dogs, houses, etc.), whose meaning or definition can vary depending on the context. OD can refer to the task of locating (i.e. finding the coordinates) an object in an image (so, in this case, it would be a synonym for object localization) or the task of locating the object and classifying it (i.e. object localization + object classification)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T17:40:19.937" LastActivityDate="2020-06-16T17:40:19.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13034" PostTypeId="2" ParentId="12973" CreationDate="2019-06-24T14:59:34.470" Score="1" Body="&lt;p&gt;&lt;strong&gt;Ambiguity in Terms&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct that there is something like overloading occurring in tensor terminology in posts and in software libraries. Confusing jargon often appears when those without the mathematical background use mathematical terms. You rarely find this confusion when reading NASA, Cambridge, MIT, or Cal-tech materials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tensors, Rank, Dimensions, and Channels&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A tensor is a grouping of dimensions. The grouping typically represents a relation between quantities describing a system. The order (or rank) of the tensor is the number of dimensions in the grouping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When this mathematical idea is applied formally to electrical engineering, chemical systems, biological systems, or computing systems, we can say a tensor is the grouping of the inputs and outputs of that system, abstracting the relation between them. The order (or rank) of the tensor is the total number of signal channels going to or from the system. The rank is simply the number of inputs plus the number of outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That original conception was lost a little when the term tensor began to be applied to grouping inputs in one tensor and grouping outputs in another tensor, without any connection with their relationship, which is not formally correct. Those are simply vectors in mathematics, represented by arrays in computer programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A labelled data set prepared for machine learning is a discrete tensor formally, however it does not have a mathematical model applied to it yet. When convergence occurs in an artificial network, the result of training is a set of parameters that, if later applied to new inputs, will predict the outputs based on the net and its trained parameters. The configuration of the network cells had become the model and the trained network had become a tensor expression approximating the phenomenon that the training data represents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Specific Example in the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third order tensor cannot have only two dimensions, width and height. It must have three dimensions, possibly width, height, and instantaneous value. Width and height are the independent variables, and instantaneous value is the dependent variable. So we have a rank of &lt;span class=&quot;math-container&quot;&gt;$1 + 2 = 3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no clear and formal definition of what dimensionality means, so it would be useful to discontinue its use until academic textbooks contain a consistent definition, but that's unlikely to occur. It will likely continue to be used by non-mathematicians somewhat arbitrarily. Therefore, stating that two tensors are of equal dimensionality remains ambiguous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Domain and Range&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct terms are domain and range. These terms unambiguously describe the variability of the independent and dependent variables of a relation respectively. In the case of an array of samples describing a tensor relationship between horizontal position, vertical position, and brightness, the horizontal domain corresponds to the width and the vertical domain corresponds to the height.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, one would not say, &quot;Width and height are the spacial dimensions,&quot; or, &quot;width and height are the channel dimensions.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could unambiguously be stated, &quot;Width in this article shall refer to the maximum horizontal pixel position minus its minimum plus one.&quot; Without saying something that complex, albeit accurate, this would be a clear statement.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The fourth convolution layer in the network has a domain of 1280, 768 and a range of IEEE 32 bit floats.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is easily understood and close enough to technically correct to not cause readers with mathematical training to dismiss the writer as uneducated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equality question is now solved. The equality of domains could be stated clearly and unambiguously. Whether it sounds unusual is not particularly relevant.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The horizontal and vertical domains of network layers two and three are equal, both being [1280, 768].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(The reason [1280, 768] is often seen is because compilers of many languages, starting with FORTRAN or maybe even earlier, interpret that as a two dimensional domain for a two dimensional array. So that can be used among software people, although [768, 1280] is more likely used in programs since it is a video tradition to traverse left to right and then top to bottom.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Misuse of the Term Tensor&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is incorrect to say that adding two arrays is an example of tensors manipulated in software. Tensors are more abstract than arrays. If one were to ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Make symbolic manipulations to reduce a complex tensor equation to a simple one,&lt;/li&gt;&#xA;&lt;li&gt;Show that two tensor expressions are equal for given domains, or&lt;/li&gt;&#xA;&lt;li&gt;Determine the correlation of a set of discrete points to a model represented as a tensor equation,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... then one would be performing tensor mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many pieces of software that perform such symbolic manipulations, but assigning a tensor to an input sample in machine learning is not tensor math. Tensor object classes that are not part of a system of symbolic manipulation are simply an abstraction of the typical programming language's array structure, parameterizing rank in the constructor. Such a trivial abstraction is not nearly as deep as symbolic manipulation of tensor expressions.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-06-25T16:50:20.420" LastActivityDate="2019-06-25T16:50:20.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13035" PostTypeId="2" ParentId="13028" CreationDate="2019-06-24T17:51:57.650" Score="1" Body="&lt;p&gt;so assuming your not allowed to use transfer methodologies (like take an already exisiting elephant object detector) my recommendation is to train a CNN classifier (labels are binary-- elephant exist, elephant doesnt exist) and then use strategies founded in like &lt;a href=&quot;https://arxiv.org/abs/1610.02391&quot; rel=&quot;nofollow noreferrer&quot;&gt;grad cam&lt;/a&gt;. Note there does exist a gradcam++ but because you can assure theres only one instance, it isnt necessary and is just more complicated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that since you just need the location and not the pixel specificity, you dont even need to do the guided backprop, but just the relation with respect to the last convoluitional map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick description is that its using the gradient of the class loss w.r.t the last feature map to see which locations helped make the classification, and from there you can upscale to the receptive field that those neurons touch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helped!&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-24T17:51:57.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13038" PostTypeId="2" ParentId="11793" CreationDate="2019-06-24T21:15:06.007" Score="7" Body="&lt;p&gt;To understand this equation first you need to understand the &lt;a href=&quot;https://arxiv.org/pdf/1406.2661.pdf&quot; rel=&quot;noreferrer&quot;&gt;context in which it is first introduced&lt;/a&gt;. We have two neural networks (i.e. &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;) that are playing a minimax game. This means that they have competing goals. Let's look at each one separately:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Generator&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Before we start, you should note that throughout the whole paper the notion of the &lt;a href=&quot;https://datascience.stackexchange.com/a/54349/34269&quot;&gt;data-generating distribution&lt;/a&gt; is used; in short the authors will refer to the samples through their underlying distributions, i.e. if a sample &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is drawn from a distribution &lt;span class=&quot;math-container&quot;&gt;$p_a$&lt;/span&gt;, we'll denote this as &lt;span class=&quot;math-container&quot;&gt;$a \sim p_a$&lt;/span&gt;. Another way to look at this is that &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; &lt;em&gt;follows&lt;/em&gt; distribution &lt;span class=&quot;math-container&quot;&gt;$p_a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generator (&lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;) is a neural network that produces samples from a distribution &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt;. It is trained so that it can bring &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; as close to &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; as possible so that samples from &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; become indistinguishable to samples from &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;. The catch is that it never gets to actually see &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;. Its inputs are samples &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; from a noise distribution &lt;span class=&quot;math-container&quot;&gt;$p_z$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Discriminator&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The discriminator (&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;) is a simple binary classifier that tries to identify which class a sample &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; belongs to. There are two possible classes, which we'll refer to as the &lt;em&gt;fake&lt;/em&gt; and the &lt;em&gt;real&lt;/em&gt;. Their respective distributions are &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; for the &lt;em&gt;real&lt;/em&gt; samples and &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; for the &lt;em&gt;fake&lt;/em&gt; ones (note that &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; is actually the distribution of the outputs of the generator, but we'll get back to this later).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is a simple binary classification task, the discriminator is trained on a binary cross-entropy error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J^{(D)} = H(y, \hat y) = H(y, D(x))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is the cross-entropy &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is sampled either from &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; or from &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; with a probability of &lt;span class=&quot;math-container&quot;&gt;$50\%$&lt;/span&gt;. More formally:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x \sim&#xA;\begin{cases}&#xA;p_{data} \rightarrow &amp;amp; y = 1, &amp;amp; \text{with prob 0.5}\\&#xA;p_g \;\;\;\,\rightarrow &amp;amp; y = 0, &amp;amp; \text{otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We consider &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is sampled from the &lt;em&gt;real&lt;/em&gt; distribution and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if it is sampled from the &lt;em&gt;fake&lt;/em&gt; one. Finally, &lt;span class=&quot;math-container&quot;&gt;$D(x)$&lt;/span&gt; represents the probability with which &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; thinks that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; belongs to &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;. By writing the cross-entropy formula we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H(y, D(x)) = \mathbb{E}_y[-log \; D(x)] = \frac{1}{N} \sum_{i=1}^{N}{ \; y_i \; log(D(x_i))}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the size of the dataset. Since each class has &lt;span class=&quot;math-container&quot;&gt;$N/2$&lt;/span&gt; samples we can split this sum into two parts:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;= - \left[ \frac{1}{N} \sum_{i=1}^{N/2}{ \; y_i \; log(D(x_i))} +  \frac{1}{N} \sum_{i=N/2}^{N} \; (1 - y_i) \; log((1 - D(x_i))) \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first of the two terms represents the the samples from the &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; distribution, while the second one the samples from the &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; distribution. Since all &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; are equally likely to occur, we can convert the sums into expectations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;= - \left[ \frac{1}{2} \; \mathbb{E}_{x \sim p_{data}}[log \; D(x)] + \frac{1}{2} \; \mathbb{E}_{x \sim p_{g}}[log \; (1 - D(x))] \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, we'll ignore &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; from the equations since it's constant and thus irrelevant when optimizing this equation. Now, remember that samples that were drawn from &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; were actually outputs from the generator (obviously this affects only the second term). If we substitute &lt;span class=&quot;math-container&quot;&gt;$D(x), x \sim p_g$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$D(G(z)), z \sim p_z$&lt;/span&gt; we'll get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_D = - \left[\; \mathbb{E}_{x \sim p_{data}}[log \; D(x)] + \; \mathbb{E}_{z \sim p_{z}}[log \; (1 - D(G(z)))] \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the final form of the discriminator loss. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Zero-sum game setting&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The discriminator's goal, through training, is to minimize its loss &lt;span class=&quot;math-container&quot;&gt;$L_D$&lt;/span&gt;. Equivalently, we can think of it as trying to maximize the opposite of the loss:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\max_D{[-J^{(D)}]} = \max_D \left[\; \mathbb{E}_{x \sim p_{data}}[log \; D(x)] + \; \mathbb{E}_{z \sim p_{z}}[log \; (1 - D(G(z)))] \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generator however, wants to maximize the discriminator's uncertainty (i.e. &lt;span class=&quot;math-container&quot;&gt;$J^{(D)}$&lt;/span&gt;), or equivalently minimize &lt;span class=&quot;math-container&quot;&gt;$-J^{(D)}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J^{(G)} = - J^{(D)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the two are tied, we can summarize the whole game through a &lt;strong&gt;value function&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$V(D, G) = -J^{(D)}$&lt;/span&gt;. At this point I like to think of it like we are seeing the whole game through the eyes of the generator. Knowing that &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; tries to maximize the aforementioned quantity, the goal of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_G\max_D{V(D, G)} = \min_G\max_D \left[\; \mathbb{E}_{x \sim p_{data}}[log \; D(x)] + \; \mathbb{E}_{z \sim p_{z}}[log \; (1 - D(G(z)))] \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Disclaimer:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This whole endeavor (on both my part and the authors' part) was to provide a mathematical formulation to training GANs. In practice there are many tricks that are invoked to effectively train a GAN, that are not depicted in the above equations.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-06-24T21:15:06.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13039" PostTypeId="1" CreationDate="2019-06-24T21:53:23.937" Score="0" ViewCount="29" Body="&lt;p&gt;I’m using Keras LSTM layers and building a model that is trained off ethics text. I have a problem of often over fitting (the network basically remembers my input corpus as it is very small). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if anyone has heard of or thought about creating a custom optimizer that takes in the output wordVec but also wordVec’s found next to the same input at a decreasing level of error... bare with me. So instead of the goal to maximise the network towards getting the right next word in a given sequence it is maximising the chance of a correct word across the corpus with the same word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like this may be an complication but thought it was worth asking.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2019-06-24T21:53:23.937" Title="Custom optimizer and word-vector evaluator lstm" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13040" PostTypeId="2" ParentId="11487" CreationDate="2019-06-24T23:32:56.393" Score="2" Body="&lt;h3&gt;tl;dr&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Your intuition is correct.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Why is this a problem?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;saturating&lt;/em&gt; effect of the sigmoid activation function is well documented as it is the main culprit of a problem called &lt;strong&gt;vanishing gradients&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, the derivative of the sigmoid function is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{dσ(z)}{dz} = σ(z) (1 - σ(z))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that very often (especially if we initialize our weights with &lt;strong&gt;large values&lt;/strong&gt;), the output of this neuron will either be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. This causes the gradient to be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, which in turn means that the weights of that neuron can't be updated. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;So let's keep the weights small (in absolute value)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Initially this seems like a good idea, if we keep the weights small we will avoid saturation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://proxy.duckduckgo.com/iu/?u=https%3A%2F%2Ftse1.mm.bing.net%2Fth%3Fid%3DOIP.HXCBO-Wx5XhuY_OwMl0PhwHaE8%26pid%3DApi&amp;amp;f=1&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However there is another problem with sigmoid functions: their gradient has a maximum value of &lt;span class=&quot;math-container&quot;&gt;$0.25$&lt;/span&gt;! This means that if the weight of a neuron is less than &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;, the error will &lt;strong&gt;diminish&lt;/strong&gt; while flowing backward through the net. This becomes progressively worse as we add more layers to the network. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How have we solved this issue&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Naturally researchers tried to find better weight initialization strategies. However this was hard, because as we saw, small weights are bad and large weights are bad. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example is &lt;a href=&quot;https://arxiv.org/pdf/1511.06422.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mishkin et al. 2016&lt;/a&gt; who propose a new initialization strategy, but fail to train a deep neural network with sigmoid activations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another workaround is to use a different learning rate for each layer (&lt;a href=&quot;https://openreview.net/pdf?id=D1VDjyJjXF5jEJ1zfE53&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xu et al. - Revise Saturated Activation Functions&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a while the Machine Learning community realized that sigmoid functions were ill-suited for deep neural networks and adopted ReLU activations, which have fewer drawbacks and scale better. Nowadays, they have become the de-facto choice for deep learning.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Sources&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This problem is known for several years and has been well documented (the earliest I could find was in 1994). This was mostly explored in Recurrent Neural Networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in reading about this, I'd recommend &lt;a href=&quot;https://medium.com/@karpathy/yes-you-should-understand-backprop-e2f06eab496b&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;,  by Andrej Karpathy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more formal sources on this topic:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.bioinf.jku.at/publications/older/ch7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hochreiter et al. - Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.4.7389&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hochreiter et al. - Recurrent Neural Net Learning and Vanishing Gradient&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v28/pascanu13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pascanu et al. - On the difficulty of training recurrent neural networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ai.dinfo.unifi.it/paolo/ps/tnn-94-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bengio et al. - Learning long-term dependencies with gradient descent is difficult&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-06-24T23:32:56.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13041" PostTypeId="1" CreationDate="2019-06-25T00:09:56.490" Score="3" ViewCount="64" Body="&lt;p&gt;I am trying to write self-play RL (NN + MCTS &lt;a href=&quot;http://web.stanford.edu/~surag/posts/alphazero.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://web.stanford.edu/~surag/posts/alphazero.html&lt;/a&gt;) to &quot;solve&quot; a board game. However, I got stuck in designing boardgame same (input layer for NN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What would be the best way to represent each cell, if there are ~10-100 cells in a game which could be occupied by any playing card.  Do one-hot-encoding and get 52 nodes for single cell ([0, 0, 1, ..., 0]) or just divide card_id by total number of cards and get single node for each cell ([0.0576...])?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is it a good/bad practice to help NN by adding additional input that could be derived from the other nodes? For instance, imagine the game where whoever has most red cards wins. Input is 10 cards, and I am adding new input node (number of red cards) to emphasize it. Would that lead to a positive result or doing something like that is bad?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Would it help to reduce the number of illegal moves and increase the performance of NN by creating additional input stating which cards are available now and which are not?&lt;/p&gt;&#xA;" OwnerUserId="26658" LastActivityDate="2019-06-25T00:09:56.490" Title="Designing state representation for board game" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13042" PostTypeId="2" ParentId="13031" CreationDate="2019-06-25T01:57:31.333" Score="0" Body="&lt;p&gt;You &lt;strong&gt;can&lt;/strong&gt; add as many layers (with any arbitrary number of nodes) as you want.&lt;/p&gt;&#xA;&lt;p&gt;Please note that as you add more learning parameters (layers and nodes), your model complexity increases. This means the model can potentially learn a more complex input-output relationship. However, it also increases the risk of &lt;strong&gt;overfitting&lt;/strong&gt;. Overfitting generally happens when the model you build is more complex than the data you have. In such a scenario, the model memorizes the data instead of learning from it. In other words, it can produce a very good result for the same data as it was trained on but cannot generalize well. So, it performs poorly when the inputs are slightly different from what is used to be fed at the training stage.&lt;/p&gt;&#xA;&lt;p&gt;In practice, you may try different architectures and parameter configurations, and measure the generalization capacity of the models (via cross-validation for example) to choose the best model. In English, a generalizable model is the one that performs (almost) equally good on both training/validation and testing sets.&lt;/p&gt;&#xA;" OwnerUserId="12853" LastEditorUserId="12853" LastEditDate="2020-07-26T04:51:23.110" LastActivityDate="2020-07-26T04:51:23.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13043" PostTypeId="1" CreationDate="2019-06-25T02:30:55.860" Score="1" ViewCount="57" Body="&lt;p&gt;I am quite new to text classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using EAST text detection model, I get multiple strings that aren't words and most often have no meaning. For example, IDs, brand names, etc. I would like to classify them into two groups. Which models work the best and how should I preprocess the strings?  I wanted to use Word2Vec, but I think it only works with real words and not with arbitrary strings.&lt;/p&gt;&#xA;" OwnerUserId="23063" LastEditorUserId="2444" LastEditDate="2019-06-25T13:56:19.003" LastActivityDate="2019-06-25T13:56:19.003" Title="How do I classify strings with possibly no meaning?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;word2vec&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13044" PostTypeId="2" ParentId="13043" CreationDate="2019-06-25T08:09:13.923" Score="1" Body="&lt;p&gt;I would just use a dictionary. A simple list lookup would tell you whether it's a recognised word or not. As an added bonus you can add some basic language processing, eg to identify inflected forms without listing them in your dictionary. Or use regular expressions to recognise ID numbers. ML is not really the right tool here. &lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-06-25T08:09:13.923" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13045" PostTypeId="1" CreationDate="2019-06-25T08:33:02.667" Score="2" ViewCount="192" Body="&lt;p&gt;I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does it work? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, starting from the &quot;model.bin&quot; file it retrains the model only on the new corpus updating the old word-vectors, is it right? &lt;/p&gt;&#xA;" OwnerUserId="20780" LastEditorUserId="2444" LastEditDate="2019-06-25T13:54:51.353" LastActivityDate="2019-06-25T13:54:51.353" Title="How does FastText support online learning?" Tags="&lt;classification&gt;&lt;word-embedding&gt;&lt;online-learning&gt;&lt;incremental-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13046" PostTypeId="1" CreationDate="2019-06-25T08:35:24.013" Score="1" ViewCount="48" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R6kHV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R6kHV.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have this slide from my AI class on using a Bayes network to compute a conditional probability. I don't really understand the point of converting the conditional probabilities to factors (besides the fact that it looks weird to marginalize or multiply variables in a CP). It seems kind of arbitrary. Is there some benefit I'm not noticing? &lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2019-06-26T12:31:38.227" LastActivityDate="2019-06-26T12:31:38.227" Title="What is the point of converting conditional probability to factor for Variable Elimination?" Tags="&lt;bayesian-network&gt;&lt;random-variable&gt;&lt;conditional-probability&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13047" PostTypeId="2" ParentId="1507" CreationDate="2019-06-25T09:14:41.037" Score="1" Body="&lt;p&gt;From &quot;&lt;a href=&quot;https://ai100.stanford.edu/sites/g/files/sbiybj9861/f/ai_100_report_0831fnl.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence And Life In 2030: One Hundred Year Study On Artificial Intelligence&lt;/a&gt;&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In fact, the field of AI is a continual endeavor to push forward the frontier of&#xA;  machine intelligence. Ironically, AI suffers the perennial fate of losing claim to its&#xA;  acquisitions, which eventually and inevitably get pulled inside the frontier, a repeating pattern known as the “AI effect” or the “odd paradox”—AI brings a new technology into the common fold, people become accustomed to this technology, it stops being considered AI, and newer technology emerges.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Consequentially, I believe we can not choose a fixed set of requirements for something to be considered AI; rather, at any given moment in history, AI is a set of programs which can achieve something that before was generally considered to be solvable by humans only. As technology evolves, the boundaries keep getting pushed and pushed, and the bar rises higher. Consider chess playing: once chess engines were considered one of the pinnacles of AI, while nowadays such programs are perceived as &quot;blind search&quot; and not truly intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To quote Larry Tesler, &lt;em&gt;Intelligence is whatever machines haven't done yet&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-06-25T09:40:06.520" LastActivityDate="2019-06-25T09:40:06.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13049" PostTypeId="1" CreationDate="2019-06-25T10:40:39.730" Score="6" ViewCount="306" Body="&lt;p&gt;I have heard that there is a huge talent demand in AI, with limited talent supply. Is it true?&lt;/p&gt;&#xA;" OwnerUserId="26351" LastEditorUserId="23527" LastEditDate="2019-06-25T12:10:54.590" LastActivityDate="2019-07-03T01:36:10.357" Title="Is there a shortage of machine learning and artificial intelligence professionals?" Tags="&lt;profession&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13050" PostTypeId="2" ParentId="3920" CreationDate="2019-06-25T11:14:24.483" Score="3" Body="&lt;p&gt;There are several terms or expressions related to such systems, such as &lt;a href=&quot;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;online learning&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;incremental learning&lt;/em&gt;&lt;/a&gt;, &lt;em&gt;continuous learning&lt;/em&gt;, &lt;a href=&quot;https://wiki.continualai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;continual learning&lt;/em&gt;&lt;/a&gt;, and &lt;a href=&quot;https://wiki.continualai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;lifelong learning&lt;/em&gt;&lt;/a&gt;. They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The opposite of online is offline. However, the expression &lt;a href=&quot;https://stats.stackexchange.com/q/70761/82135&quot;&gt;&lt;em&gt;batch learning&lt;/em&gt;&lt;/a&gt; is sometimes used as an antonym for online learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-25T11:14:24.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13051" PostTypeId="1" CreationDate="2019-06-25T13:58:39.803" Score="0" ViewCount="34" Body="&lt;p&gt;I am using python and Xgboost. I have features: activity and location and time stamps of when the activity occurred. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to predict day of week. Is this straight forward, ie y=day of week, X={activity, location}, or am I being naive and I need to do fancy time series things? I'd also like to predict time of day.&lt;/p&gt;&#xA;" OwnerUserId="8385" LastActivityDate="2019-06-25T13:58:39.803" Title="Is predicting day of week straight forward?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13053" PostTypeId="5" CreationDate="2019-06-25T16:26:09.847" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-25T16:26:09.847" LastActivityDate="2019-06-25T16:26:09.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13054" PostTypeId="4" CreationDate="2019-06-25T16:26:09.847" Score="0" Body="For questions related to teaching and learning AI concepts." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-25T16:26:09.847" LastActivityDate="2019-06-25T16:26:09.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13055" PostTypeId="2" ParentId="7073" CreationDate="2019-06-25T17:26:59.360" Score="2" Body="&lt;p&gt;You might also be looking for &lt;a href=&quot;https://en.wikipedia.org/wiki/Active_learning_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;active learning&lt;/a&gt;, where the machine learning algorithm interactively queries the user to label certain unlabeled training examples. Active learning is similar to semi-supervised learning, in which there are labeled and unlabeled examples.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-25T17:26:59.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13056" PostTypeId="2" ParentId="2964" CreationDate="2019-06-25T17:27:31.040" Score="1" Body="&lt;p&gt;I think the answer is most likely no, not in the most notable examples of AI programs (such as Machine Learning). There &lt;em&gt;is&lt;/em&gt; a set of AI techniques which involve &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic programming&lt;/a&gt;, but in that scenario, we have a computer program which automatically codes another program (we can call it &quot;target program&quot;). But the target program is not the program which perfoms the coding; so technically speaking, no, it does not write its &lt;em&gt;own&lt;/em&gt; code. This is an important difference; the programmer still has the task to write the code-generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in automated coding, though, the most notable example is &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt;, a technique which uses an evolutionary algorithm to breed computer programs. As you can see, we have an AI which produces as a result a computer program (which may be or not an AI program); it is not interacting with its own code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final remark, note that automatic coding is a pretty vague term and not all techniques are AI-related (for instance, back then, the first compilers were seen as a form of automatic programming). The most relevant techinque to your question is probably &lt;a href=&quot;https://en.wikipedia.org/wiki/Program_synthesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Program Synthesis&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-06-25T17:33:18.607" LastActivityDate="2019-06-25T17:33:18.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13057" PostTypeId="5" CreationDate="2019-06-25T17:33:51.953" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-25T17:33:51.953" LastActivityDate="2019-06-25T17:33:51.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13058" PostTypeId="4" CreationDate="2019-06-25T17:33:51.953" Score="0" Body="For questions related to active learning, which is a machine learning technique where the user is interactively queried to label certain unlabelled training examples." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-25T17:33:51.953" LastActivityDate="2019-06-25T17:33:51.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13059" PostTypeId="1" CreationDate="2019-06-25T18:05:47.027" Score="3" ViewCount="120" Body="&lt;p&gt;Similar to the recent pushes in Pretrained Language Models (BERT, GPT2, XLNet) I was wondering if such a thrust exists in Computer Vision?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, it seems the community has converged and settled for ImageNet trained classifiers as the &quot;Pretrained Visual Model&quot;. But relative to the data we have access too, shouldn't there exist something stronger? Also, classification as a sole task has its own constrictions on domain transfer (based on the assumption of how these loss manifolds are).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any better visual models for transfer rather than ImageNet successes? If no, why? Is it because of the domains fluidity in shape, resolution, etc., in comparison to text?&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-06-26T12:31:06.767" LastActivityDate="2019-06-26T15:08:58.123" Title="Are there any better visual models for transfer rather than ImageNet?" Tags="&lt;computer-vision&gt;&lt;transfer-learning&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13060" PostTypeId="2" ParentId="12872" CreationDate="2019-06-25T19:28:17.233" Score="1" Body="&lt;p&gt;It's not supposed to be derived from some equation. That is the basic premise under which GANs work. &lt;em&gt;The output of the Generator &lt;span class=&quot;math-container&quot;&gt;$G(z)$&lt;/span&gt; is fed as an input &lt;span class=&quot;math-container&quot;&gt;$x_g$&lt;/span&gt; to the discriminator&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-06-25T19:28:17.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13061" PostTypeId="2" ParentId="6699" CreationDate="2019-06-25T22:19:42.353" Score="0" Body="&lt;p&gt;It should be a short amount of time before we start seeing exponential complexity growth as the target of AI algorithms, likely involving the golden ratio. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Golden_ratio#Relationship_to_Fibonacci_sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Golden_ratio#Relationship_to_Fibonacci_sequence&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are already using the golden ratio to perform quantum computations :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/How-are-quantum-physics-and-the-golden-ratio-connected&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/How-are-quantum-physics-and-the-golden-ratio-connected&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, once we scale parallelization to GPU-like networks of quantum processors, we can then be sure we have entered the territory where AI and the Golden Ratio are inherently more intrinsic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as accelerating the learning models/algorithms with them, we can only hope something so fantastic would be found in the future of AI as well; who knows, maybe they will come out of course through necessity much like the biological computers we all execute and replicate our code from, and it's environment :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6047800/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6047800/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not to say it will be the only property present, as the article concludes the golden ratio is &quot;most likely&quot; only related by &quot;chance&quot;. I'm sure the same random chance by which it relates to black hole entropy, and, everything else it relates to :P. &lt;/p&gt;&#xA;" OwnerUserId="26673" LastEditorUserId="26673" LastEditDate="2019-06-25T22:42:08.303" LastActivityDate="2019-06-25T22:42:08.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13063" PostTypeId="2" ParentId="4703" CreationDate="2019-06-26T07:00:56.820" Score="-2" Body="&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Connect_Four&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia article on  Connect Four&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Connect Four is a solved game. The first player can always win by&#xA;  playing the right moves.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's pretty reasonable that even without training 1st player win more often by small margin.&#xA;With some training  1st player should mostly win.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-06-26T07:00:56.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13064" PostTypeId="2" ParentId="13049" CreationDate="2019-06-26T10:27:05.297" Score="1" Body="&lt;p&gt;In one word – yes, there is! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are following the industry reports then you must know that more than 500,000 jobs are lying vacant because there is a huge shortage of qualified talent in the industry.  It is also quite interesting to note that the market has almost double the number of jobs available than the required talented ML and AI professionals – just in case – there is someone out there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to note is that the countries like US, China, India, Israel, Germany, Switzerland, Canada, France, Spain and Singapore —  have seen a consistent rise in demand for skilled ML and AI professionals. &lt;/p&gt;&#xA;" OwnerUserId="26265" LastEditorUserId="75" LastEditDate="2019-06-28T14:07:07.737" LastActivityDate="2019-06-28T14:07:07.737" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13065" PostTypeId="5" CreationDate="2019-06-26T12:34:02.720" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-26T12:34:02.720" LastActivityDate="2019-06-26T12:34:02.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13066" PostTypeId="4" CreationDate="2019-06-26T12:34:02.720" Score="0" Body="For questions related to the mathematical concept of a random variable (in the context of AI)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-26T12:34:02.720" LastActivityDate="2019-06-26T12:34:02.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13067" PostTypeId="1" CreationDate="2019-06-26T12:55:19.520" Score="1" ViewCount="101" Body="&lt;p&gt;Paper link : &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prioritized Experience Replay&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the blind cliffwalk setup:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why is the number of possible action sequences equal to 2^N? I cant think of sequences more than (N + 1) where one sequence is the sequence of all right actions and the other N sequences are due to wrong actions at each state.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Generally for prioritized experience replay:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The replay memory consists of some transitions which are repeated.In the priority queue I feel that there should only be a single priority for each transition to speed up learning. Is there any advantage of having priority values for each repeated instance of the transition?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit for 2nd question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider algorithm 1 on page 5 of the article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nFRP7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nFRP7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets consider one of the transitions to be repeated in the replay memory. If one of them is sampled (line 9) and the priority updated (line 12). Will the priority update on the other instance of the same transition?&lt;/p&gt;&#xA;" OwnerUserId="26697" LastEditorUserId="26697" LastEditDate="2019-06-27T03:50:05.370" LastActivityDate="2019-06-27T03:50:05.370" Title="Clarifications on &quot;Prioritized Experience Replay&quot; (Deepmind, 2015)" Tags="&lt;deep-learning&gt;&lt;prioritized-sweeping&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13068" PostTypeId="2" ParentId="13067" CreationDate="2019-06-26T14:07:33.803" Score="1" Body="&lt;p&gt;for 1) i think your confusing elements touched vs sequences. At each point for N turns you have 2 possible options, therefore you have &lt;span class=&quot;math-container&quot;&gt;$\prod_{i=1}^N 2$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$2^N$&lt;/span&gt; possible sequences.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for 2) The priorities are updated based on the expected rewards. They do not add new elements each time, they update&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-06-26T14:07:33.803" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13070" PostTypeId="2" ParentId="13059" CreationDate="2019-06-26T15:08:58.123" Score="4" Body="&lt;h3&gt;Why is ImageNet so popular for transfer learning?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Models pre-trained on the ImageNet datasets have been the de-facto choice for many years now. Many popular reasons as to why people think that ImageNet is so effective for transfer learning are the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ImageNet is a truly &lt;strong&gt;large-scale&lt;/strong&gt; dataset that contains over 1 million images, each of which has a decent resolution.&lt;/li&gt;&#xA;&lt;li&gt;ImageNet has a &lt;strong&gt;wide and diverse set of classes&lt;/strong&gt; (1000 in total) ranging from animals to humans, cars, trees, etc. Most Computer Vision tasks operate in similar domains, however there are some notable exceptions (e.g. medical images).&lt;br&gt;&#xA;For example, an object detection model for autonomous driving would benefit from ImageNet transfer learning, as the pre-trained model has seen images with similar content (e.g. roads, people, cars, street signs), even though it tries to solve a different task (i.e. object detection not classification). &lt;/li&gt;&#xA;&lt;li&gt;The above two reasons allow models trained on ImageNet to identify and extract very &lt;strong&gt;generic features&lt;/strong&gt;, especially in their initial layers, that can be effectively re-used.&lt;/li&gt;&#xA;&lt;li&gt;ImageNet has a lot of &lt;strong&gt;similar classes&lt;/strong&gt;. This is an interesting argument because it contradicts the second one. Due to the &lt;em&gt;closeness&lt;/em&gt; of some classes (e.g. multiple breeds of cats), networks learn to extract more fine-grained features.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Another overlooked reason I find very important is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ImageNet has been the &lt;em&gt;benchmark&lt;/em&gt; for performance for image classifiers for years now. When, for example, you are selecting a re-trained ResNet to use, you know that that model is guaranteed to operate at a high level of performance. Other datasets don't have such &lt;strong&gt;notable challenges&lt;/strong&gt; as the ILVRC. That challenge is what make the VGG and ResNet popular in the first case, so it comes natural that people would want to use those weights.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In practice, due to the way in which CNNs identify and extract features from images, they can easily be &quot;transferred&quot; from task to task.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Is it actually better than other datasets?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This question was widely explored by &lt;a href=&quot;https://arxiv.org/pdf/1608.08614.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Huh et al.&lt;/a&gt;, who tried to identify the reasons that made the ImageNet dataset better than other ones for transfer learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short they found out that most of the reasons that people thought made ImageNet so good (i.e. the ones I mentioned above) &lt;strong&gt;weren't necessarily correct&lt;/strong&gt;. Furthermore, the amount and diversity of images and classes required to effectively train a CNN has been &lt;strong&gt;highly overestimated&lt;/strong&gt;. So there is no particular reason people should choose this specific dataset.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Now, to answer your questions:&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was wondering if such a thrust exists in Computer Vision?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, ImageNet is currently established as the de-facto choice, evident by the fact that &lt;a href=&quot;https://keras.io/applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;all 10 keras.applications models&lt;/a&gt; offer weights only for ImageNet.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But relative to the data we have access too, shouldn't there exist something stronger?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is an interesting question, as the consensus things that deep learning models &lt;a href=&quot;https://3qeqpr26caki16dnhd19sv6by6v-wpengine.netdna-ssl.com/wp-content/uploads/2016/08/Why-Deep-Learning-1024x742.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;keep getting better with more data&lt;/a&gt;. There is, however, evidence that indicates otherwise (i.e. that CNN models con't have as much &lt;em&gt;capacity&lt;/em&gt; as we thought). You can read the aforementioned study for more details. In any case, this is still an open research question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if models could get better, though, with more data, it is possible that it still wouldn't matter because ImageNet pre-trained models are &lt;em&gt;strong enough&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;classification as a sole task has its own constrictions on domain transfer &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There have been numerous cases where models initialized from pre-trained ImageNet weights have done well in settings other than classification (e.g. regression, object detection). I'd argue that initialization from ImageNet is almost always better than random initialization.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any better visual models for transfer rather than ImageNet successes? If no, why? Is it because of the domains fluidity in shape, resolution, etc., in comparison to text?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Partly, yes. I think that in comparison to text, images have some useful properties that are exploited through CNNs, which makes their &lt;em&gt;knowledge&lt;/em&gt; more transferable. This claim, however, is based on intuition; I can't back this up somehow.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-06-26T15:08:58.123" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13071" PostTypeId="1" AcceptedAnswerId="13074" CreationDate="2019-06-26T15:41:08.430" Score="3" ViewCount="97" Body="&lt;p&gt;I'm new to the graph convolution network. I wonder what is the main purpose of applying data with graph structure to CNN?&lt;/p&gt;&#xA;" OwnerUserId="26502" LastEditorUserId="2444" LastEditDate="2019-06-27T11:22:49.717" LastActivityDate="2019-06-27T11:22:49.717" Title="What is the purpose and benefit of applying CNN to a graph?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;geometric-deep-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13072" PostTypeId="5" CreationDate="2019-06-26T17:05:55.423" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-26T17:05:55.423" LastActivityDate="2019-06-26T17:05:55.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13073" PostTypeId="4" CreationDate="2019-06-26T17:05:55.423" Score="0" Body="For questions about the &quot;AI business&quot;, the industry of AI, and related subjects." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-06-26T17:05:55.423" LastActivityDate="2019-06-26T17:05:55.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13074" PostTypeId="2" ParentId="13071" CreationDate="2019-06-26T18:10:38.403" Score="2" Body="&lt;p&gt;There are some problems that involve graphs and manifolds (sometimes collectively called &lt;em&gt;non-Euclidean&lt;/em&gt; data), such as &lt;a href=&quot;https://arxiv.org/pdf/1802.04364.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;molecule design and generation&lt;/a&gt;, &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/34/13/i457/5045770&quot; rel=&quot;nofollow noreferrer&quot;&gt;drug repositioning&lt;/a&gt;, social networks analysis, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28412442&quot; rel=&quot;nofollow noreferrer&quot;&gt;brain imaging&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1902.06673.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;fake news detection&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1704.06803.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommender systems&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1809.06166.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;neutrino detection&lt;/a&gt;, computer vision and graphics and &lt;a href=&quot;https://arxiv.org/pdf/1712.00268v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;shape (e.g. hand or face) completion&lt;/a&gt; or generation (generative models).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main benefit of &lt;a href=&quot;https://arxiv.org/pdf/1611.08097.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;geometric deep learning&lt;/em&gt;&lt;/a&gt; (deep learning applied to graphs and manifolds) is that you do not lose the information encoded in the graphs (or manifolds), which, otherwise, you would likely lose because you would need to convert your graphs (or manifolds) to an equivalent vector representation that you can feed into the existing CNN or other standard neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you cannot directly apply the usual convolution operation to graphs, because, for example, graphs do not have the notion of relative positions of the nodes. Furthermore, note that &lt;em&gt;graph networks&lt;/em&gt; have little to do with CNNs, even if they are sometimes called &lt;em&gt;graph convolution networks&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-26T18:29:58.930" LastActivityDate="2019-06-26T18:29:58.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13075" PostTypeId="2" ParentId="10313" CreationDate="2019-06-26T18:30:27.613" Score="1" Body="&lt;p&gt;I have a somewhat similar problem, there's this paper that I was supposed to extend (I can explain how): &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1984-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1984-2&lt;/a&gt;&#xA;What they do is that they have a connection network between input and second layer (giving some biological context to training) I don't know how you can maintain the zero edges during back propagation though.&lt;/p&gt;&#xA;" OwnerUserId="22010" LastActivityDate="2019-06-26T18:30:27.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13076" PostTypeId="1" AcceptedAnswerId="13081" CreationDate="2019-06-27T00:03:31.793" Score="2" ViewCount="57" Body="&lt;p&gt;I am new in this area of Machine Learning and Neural Networks. Currently, I'm taking some courses on Udemy and reading a book about it, but I still have one big question regarding data pre-processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all of those Udemy's lessons, people always use a perfect dataset and ready to input in a model. So all you have to do is run it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How do I know if my dataset is ready for a model?&lt;/em&gt; &lt;em&gt;What do I have to do to make it ready? Which evaluations?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had a few statistics classes in college already and I learned a lot about correlations matrices, autocorrelation functions, and its lags, etc. and I didn't see yet in anywhere someone explaining how can I evaluate my data and then proceed to implement a model to solve my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone could point me a direction, give me some material, show me where I can learn this, anything, it would be really helpful!&lt;/p&gt;&#xA;" OwnerUserId="26707" LastEditorUserId="2444" LastEditDate="2019-06-27T11:31:30.647" LastActivityDate="2019-06-27T11:31:30.647" Title="How do I know if my dataset is ready for a machine learning model?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;data-science&gt;&lt;data-mining&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13077" PostTypeId="1" CreationDate="2019-06-27T00:04:13.403" Score="3" ViewCount="38" Body="&lt;p&gt;When you applying a graph structured data to the graph convolution network,  what are the benefits of using the state information that maintains the graph structure?&lt;/p&gt;&#xA;" OwnerUserId="26502" LastActivityDate="2019-06-27T00:04:13.403" Title="What are the benefits of using the state information that maintains the graph structure?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;graphs&gt;&lt;geometric-deep-learning&gt;&lt;graph-theory&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13078" PostTypeId="2" ParentId="2158" CreationDate="2019-06-27T05:19:58.470" Score="2" Body="&lt;p&gt;LIDARs, especially cheap LIDARs, have problems with reflective surfaces (like metallic paint on cars), strong lights like car headlights, weather(rain, snow, hail, fog), and have a considerably shorter range than comparable in price radars. Of course, they have much better precision, so some hardware stacks for cars are using both.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="2444" LastEditDate="2019-06-27T11:21:13.467" LastActivityDate="2019-06-27T11:21:13.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13080" PostTypeId="1" CreationDate="2019-06-27T07:16:51.563" Score="2" ViewCount="317" Body="&lt;p&gt;I don't understand the proof that &lt;span class=&quot;math-container&quot;&gt;$A^*$&lt;/span&gt; is optimal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proof is by contradiction:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Assume &lt;span class=&quot;math-container&quot;&gt;$A^*$&lt;/span&gt; returns &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; but there exists a &lt;span class=&quot;math-container&quot;&gt;$p'$&lt;/span&gt; that is cheaper. When &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is chosen from the frontier, assume &lt;span class=&quot;math-container&quot;&gt;$p''$&lt;/span&gt; (Which is part of the path &lt;span class=&quot;math-container&quot;&gt;$p'$&lt;/span&gt;) is chosen from the frontier. Since &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; was chosen before &lt;span class=&quot;math-container&quot;&gt;$p''$&lt;/span&gt;, then we have &lt;span class=&quot;math-container&quot;&gt;$\text{cost}(p) + \text{heuristic}(p) \leq \text{cost}(p'') + \text{heuristic}(p'')$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; ends at goal, therefore the &lt;span class=&quot;math-container&quot;&gt;$\text{heuristic}(p) = 0$&lt;/span&gt;. Therefore &lt;span class=&quot;math-container&quot;&gt;$\text{cost}(p) \leq \text{cost}(p'') + \text{heuristic}(p'') \leq \text{cost}(p')$&lt;/span&gt; because heuristics are admissible. Therefore we have a contradiction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am confused: can't we also assume there's a cheaper path that's in a frontier closer to the start node than &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;? Or is part of the proof that's not possible because &lt;span class=&quot;math-container&quot;&gt;$A^*$&lt;/span&gt; would have examined that path because it is like BFS with lowest cost search, so, if there's a cheaper path, it'll be at a further frontier?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2019-11-10T16:59:04.687" LastActivityDate="2019-11-10T16:59:04.687" Title="Understanding the proof that A* search is optimal" Tags="&lt;search&gt;&lt;proofs&gt;&lt;heuristics&gt;&lt;a-star&gt;&lt;admissible-heuristic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13081" PostTypeId="2" ParentId="13076" CreationDate="2019-06-27T08:57:29.227" Score="2" Body="&lt;p&gt;Before jumping to modeling, there are a few tasks a data scientist (or ML/AI practitioner) must do:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Ideation&lt;/strong&gt; (or hypothesizing): Before applying any modeling approach, we need to ask the right questions. We must clearly mention our assumptions and declare how we want to measure the effectiveness of the pipeline. Note that, some tools/algorithms might not fit to the made assumptions or may not lead to the best values in the defined metrics. So, the pipeline must be designed in a way it serves the purpose of answering the defined questions.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.udemy.com/topic/data-cleaning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Data Cleansing&lt;/strong&gt;&lt;/a&gt;: Real-world data sets are usually not clean. They have all sorts of data issues such as missing value, duplicates, outliers, wrong measurements, fragments, inconsistency, etc. Most of the ML techniques are sensitive (to different extents, of course) to such issues. Therefore, the data should be cleaned before any modeling can be done.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Data Wrangling&lt;/strong&gt; (or Feature Engineering): In many cases, the gathered data (even cleaned) is not immediately suitable for any modeling/analysis. For example, we may need to convert the documents of a text corpus to vectors of numbers (via TF-IDF or Embedding techniques) before being able to apply a text classifier simply because our classifier only takes numeric data. Converting measurements to other units, breaking addresses to their components, converting times and dates to different formats or timezones are just a few examples of data wrangling tasks (in a broader context, feature engineering may also refer to dimension reduction or feature selection/projection).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.udemy.com/exploratory-data-analysis-with-pandas-and-python-3x/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Exploratory Data Analysis (EDA)&lt;/strong&gt;&lt;/a&gt;: To do the cleansing, understanding the data set features, and ideation, we often need to explore the given data set using visual (e.g., dashboards and diagrams) or summary statistics tools.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I have no business interest with Udemy. The links are just shared because @pedro-de-sá mentioned they take some courses from Udemy.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12853" LastEditorUserId="2444" LastEditDate="2019-06-27T11:27:49.387" LastActivityDate="2019-06-27T11:27:49.387" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13082" PostTypeId="1" AcceptedAnswerId="13083" CreationDate="2019-06-27T09:19:58.713" Score="5" ViewCount="1048" Body="&lt;p&gt;On &lt;a href=&quot;https://towardsdatascience.com/understanding-semantic-segmentation-with-unet-6be4f42d4b47&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; article, it says that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The UNET was developed by Olaf Ronneberger et al. for Bio Medical Image Segmentation. The architecture contains two paths. First path is the contraction path (also called as the encoder) which is used to capture the context in the image. The encoder is just a traditional stack of convolutional and max pooling layers. The second path is the symmetric expanding path (also called as the decoder) which is used to enable precise localization using transposed convolutions. Thus it is an end-to-end fully convolutional network (FCN), i.e. it only contains Convolutional layers and does not contain any Dense layer because of which it can accept image of any size.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What I don't understand is how an FCN can accept images of any size, while an ordinary object detector, such as YOLO with a dense layer at the very end, cannot accept images of any size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, why can a fully convolutional network accept images of any size?&lt;/p&gt;&#xA;" OwnerUserId="26714" LastEditorUserId="2444" LastEditDate="2020-06-14T10:57:29.030" LastActivityDate="2020-06-14T10:59:44.273" Title="Why can a fully convolutional network accept images of any size?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13083" PostTypeId="2" ParentId="13082" CreationDate="2019-06-27T09:55:56.757" Score="6" Body="&lt;p&gt;The reason is that when using a convolutional layer, you select the size of the &lt;em&gt;filter kernels&lt;/em&gt;, which are independent of the image/layer input size (provided that images smaller than the kernels are padded appropriately).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using a dense layer, you specify the size of the layer itself and the resulting weight matrix is a function of both the size of the dense layer and the upstream layer. This is because each neuron in the upstream layer makes a connection to each neuron in the dense layer. So, if you have 50 neurons in the upstream layer and 20 neurons in the dense layer, then the weight matrix has &lt;span class=&quot;math-container&quot;&gt;$50 \times 20=1000$&lt;/span&gt; values. Those weights are what get determined during the training phase, and so those layer sizes are fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the output of a CNN layer is a number of images/tensors (specified by the number of filters chosen), whose size is determined by the kernel size and any padding option chosen. If those are fed into a dense layer, then that fixes the the size that those images can be (because of the reason given in the previous paragraph). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if no dense layer is used in the whole network, then the input to the first CNN layer can be any size because the weights are just the individual parameters of the filter kernels, and the filter kernels remain the same size regardless of the input tensor size.&lt;/p&gt;&#xA;" OwnerUserId="12509" LastEditorUserId="2444" LastEditDate="2020-06-14T10:59:44.273" LastActivityDate="2020-06-14T10:59:44.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13086" PostTypeId="1" CreationDate="2019-06-27T11:22:10.720" Score="4" ViewCount="75" Body="&lt;p&gt;Thomas Ray's &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=5&amp;amp;ved=2ahUKEwjIl_S1w4njAhV1sHEKHSt3DbYQFjAEegQIAxAC&amp;amp;url=https%3A%2F%2Fwww.cc.gatech.edu%2F~turk%2Fbio_sim%2Farticles%2Ftierra_thomas_ray.pdf&amp;amp;usg=AOvVaw26-NAiCH4NxxWbXKEK20r5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tierra&lt;/a&gt; is a computer program which simulates life.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the linked paper, he argues how this simulation may have real-world applications, showing how his digital organisms (computer programs) evolve in an interesting way: they develop novel ways of replicating themselves and become faster at it (he argues that the evolved organisms employ an algorithm which is 5 times faster than the original one he wrote).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tierra's approach is different from standard GAs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;While in GAs usually there is a set of genomes manipulated, copied and mutated by the program, in Tierra everything is done by the programs themselves: they self-replicate.&lt;/li&gt;&#xA;&lt;li&gt;There is no explicit fitness function: instead, digital organisms compete for energy resources (CPU time) and space resources (memory). &lt;/li&gt;&#xA;&lt;li&gt;Organisms which take a long time to replicate reproduce less frequently, and organisms who create many errors are penalized (they die out faster).&lt;/li&gt;&#xA;&lt;li&gt;Tierran machine language is extremely small: operands included, it only has 32 instructions. Oftentimes, so called RISC instruction sets have a limited set of opcodes, but if you consider the operands, you get billions of possible instructions.&lt;/li&gt;&#xA;&lt;li&gt;Consequentially, Tierran code is less brittle, and you can mutate it without breaking the code. In contrast, usually, if you mutate randomly some machine code, you get a broken program.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I was wondering if we could use this approach to optimize machine code. For instance, let's assume we have some assembly-like program which computes a certain function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. We could link reproduction time with efficiently computing &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, and life-span with correctly computing it. This could motivate programs to find novel and faster ways to compute &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anything similar ever been tried? Could it work? Where should I look into?&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-06-27T11:53:24.503" LastActivityDate="2019-12-22T23:26:57.727" Title="Can we use the Tierra approach to optimize machine code?" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;genetic-programming&gt;&lt;artificial-life&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13087" PostTypeId="1" CreationDate="2019-06-27T11:26:03.860" Score="1" ViewCount="34" Body="&lt;p&gt;I am specifically interested in the topic of edge cases.&lt;/p&gt;&#xA;&lt;p&gt;I have the presentation &lt;a href=&quot;https://users.ece.cmu.edu/%7Ekoopman/lectures/Koopman19_SSS_slides.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edge Cases and Autonomous Vehicle Safety&lt;/a&gt; as a starting point, in particular on page 6:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Machine Learning (inductive training)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;No design insight&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generally inscrutable; prone to gaming and brittleness.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'd like to find more hard data on how ML may do very well until an edge case is encountered.&lt;/p&gt;&#xA;" OwnerUserId="23170" LastEditorUserId="23170" LastEditDate="2020-06-23T12:32:50.030" LastActivityDate="2020-06-23T12:32:50.030" Title="I am looking for research related to the use of AI and ML in automotive and aeronautics safety design" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13088" PostTypeId="1" AcceptedAnswerId="15429" CreationDate="2019-06-27T15:14:23.067" Score="4" ViewCount="2710" Body="&lt;p&gt;In reinforcement learning, there are &lt;a href=&quot;https://ai.stackexchange.com/q/12274/2444&quot;&gt;deterministic and non-deterministic (or stochastic) policies&lt;/a&gt;, but there are also &lt;em&gt;stationary&lt;/em&gt; and &lt;em&gt;non-stationary&lt;/em&gt; policies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between a stationary and a non-stationary policy? How do you formalize both? Which problems (or environments) require a stationary policy as opposed to a non-stationary one (and vice-versa)?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2021-02-25T00:17:37.457" Title="What is the difference between a stationary and a non-stationary policy?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policies&gt;&lt;stationary-policy&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13089" PostTypeId="5" CreationDate="2019-06-27T15:17:52.237" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-27T15:17:52.237" LastActivityDate="2019-06-27T15:17:52.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13090" PostTypeId="4" CreationDate="2019-06-27T15:17:52.237" Score="0" Body="For questions related to the concept of a stationary policy (in reinforcement learning and other AI sub-fields)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-27T15:17:52.237" LastActivityDate="2019-06-27T15:17:52.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13091" PostTypeId="1" CreationDate="2019-06-27T15:51:18.267" Score="3" ViewCount="38" Body="&lt;p&gt;I have been searching for more than one week which learning methods were used in Neurogrid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I only found descriptions of its architecture (chips, circuits, analog and/or digital components, performance results), everything but no clue on how it updates the weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, I think that it cannot be gradient descent (with back-propagation), as the topology of the neurons in a chip, for example in the neurocore of Neurogrid, is a mesh or grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know where I could find this kind of information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TdpST.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TdpST.png&quot; alt=&quot;One neurocore of Neurogrid&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26719" LastEditorUserId="2444" LastEditDate="2019-06-28T11:06:46.857" LastActivityDate="2019-06-28T11:06:46.857" Title="Where could I find information on the learning methods used in Neurogrid?" Tags="&lt;machine-learning&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13093" PostTypeId="1" AcceptedAnswerId="13095" CreationDate="2019-06-27T16:04:28.363" Score="2" ViewCount="55" Body="&lt;p&gt;Quoting from Wikipedia page on &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Turing test, developed by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. Turing proposed that a human evaluator would judge natural language conversations between a human and a machine designed to generate human-like responses. The evaluator would be aware that one of the two partners in conversation is a machine, and all participants would be separated from one another. The conversation would be limited to a text-only channel such as a computer keyboard and screen...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The definition made me wonder if we would term the &lt;strong&gt;captchas&lt;/strong&gt; that show up on Google here it gives textual instructions to the user, bot or not, asking them to select images containing a specific object out of the given set of images.&lt;/p&gt;&#xA;" OwnerUserId="17209" LastEditorUserId="17209" LastEditDate="2019-06-27T16:12:08.823" LastActivityDate="2021-01-24T18:47:38.450" Title="Would you term Google's Captchas as Turing Test?" Tags="&lt;terminology&gt;&lt;turing-test&gt;&lt;google&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13095" PostTypeId="2" ParentId="13093" CreationDate="2019-06-27T16:51:51.187" Score="4" Body="&lt;p&gt;In the &lt;a href=&quot;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard Turing test (or imitation game)&lt;/a&gt;, the interrogator can ask multiple &lt;em&gt;arbitrary&lt;/em&gt; questions, while, in the case of captchas, usually, there's only one question or problem. Additionally, in the Turing test, the interrogator interactively communicates with both the human and the machine. Furthermore, captchas do not test the conversational skills but only the visual ones of the AI. Therefore, the existing captchas are &lt;strong&gt;not&lt;/strong&gt; an example of the &lt;em&gt;standard&lt;/em&gt; Turing test.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-24T18:47:38.450" LastActivityDate="2021-01-24T18:47:38.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13096" PostTypeId="2" ParentId="6099" CreationDate="2019-06-27T16:58:59.240" Score="-1" Body="&lt;p&gt;My interpretation of the question was 'what activation function in an artificial neural network (ANN) is closest to that found in the brain?'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst I agree with the selected answer above, that a single neuron outputs a dirac, if you think of a neuron in an ANN as modelling the output firing rate, rather than the current output, then I believe ReLU might be closest?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jackterwilliger.com/biological-neural-networks-part-i-spiking-neurons/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jackterwilliger.com/biological-neural-networks-part-i-spiking-neurons/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26722" LastActivityDate="2019-06-27T16:58:59.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13097" PostTypeId="1" AcceptedAnswerId="13099" CreationDate="2019-06-27T17:03:30.407" Score="1" ViewCount="112" Body="&lt;p&gt;Both of them deal with data of graph structure like a network community. Is there a big difference there?&lt;/p&gt;&#xA;" OwnerUserId="26502" LastEditorUserId="2444" LastEditDate="2019-06-28T10:46:30.970" LastActivityDate="2019-09-10T16:51:12.093" Title="What are the differences between network analysis and geometric deep learning on graphs?" Tags="&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;graphs&gt;&lt;geometric-deep-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13098" PostTypeId="2" ParentId="13080" CreationDate="2019-06-27T17:34:01.540" Score="2" Body="&lt;p&gt;The key phrase here is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;because heuristics are admissible&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, the heuristics never &lt;em&gt;overestimate&lt;/em&gt; the path length:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$cost(n) + heuristic(n) \le cost(\text{any path going through n})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And since &lt;strong&gt;the frontier is ordered by &lt;span class=&quot;math-container&quot;&gt;$\textbf{cost + heuristic}$&lt;/span&gt;&lt;/strong&gt;, when a completed path &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is dequeued from the frontier, we know that it must necessarily be &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; any path going through some other frontier node &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, because&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$cost(p) = cost(p) + heuristic(p) \le cost(q) + heuristic(q) \le cost(\text{any path going through q})$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="19452" LastActivityDate="2019-06-27T17:34:01.540" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13099" PostTypeId="2" ParentId="13097" CreationDate="2019-06-27T18:44:48.590" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Network_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Network analysis&lt;/a&gt; does not necessarily use deep learning techniques, while geometric deep learning (GDL) on graphs uses only deep learning techniques (that is, you train a neural network using gradient descent or other optimization methods). You can do some network analysis using GDL.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-06-27T18:44:48.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13100" PostTypeId="2" ParentId="7624" CreationDate="2019-06-27T20:20:32.040" Score="4" Body="&lt;p&gt;Over the years, many people have attempted to define intelligence, so there are many definitions of intelligence, but most of them are not formalized. For a big collection of definitions, see the paper &lt;a href=&quot;https://arxiv.org/pdf/0706.3639.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Collection of Definitions of Intelligence&lt;/a&gt; (2007) by Shane Legg and Marcus Hutter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an attempt to formally define intelligence, so that it comprises all forms of intelligence, in the paper &lt;a href=&quot;https://arxiv.org/pdf/0712.3329.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Intelligence: A Definition of Machine Intelligence&lt;/a&gt; (2007), the same Legg and Hutter, after having researched the previously given definitions of intelligence, define intelligence as follows&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Intelligence measures an &lt;a href=&quot;https://ai.stackexchange.com/a/12995/2444&quot;&gt;agent&lt;/a&gt;'s ability to achieve goals in a wide range of environments&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This definition apparently favors systems that are able to solve more tasks (i.e. &lt;a href=&quot;http://www.scholarpedia.org/article/Artificial_General_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGIs&lt;/a&gt;) than systems that are only able to solve a specific task (i.e. &lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;narrow AIs&lt;/a&gt;), but, according to Legg and Hutter, it should summarise the main points of the previously given definitions of intelligence, so it should be a reasonable and quite general definition of intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my blog post &lt;a href=&quot;https://nbro.github.io/blogging/2020/05/20/on-the-definition-of-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the definition of intelligence&lt;/a&gt;, I also talk about this definition, but I suggest that you read the mentioned papers if you are interested in all details.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-30T21:18:53.080" LastActivityDate="2020-05-30T21:18:53.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13101" PostTypeId="2" ParentId="7591" CreationDate="2019-06-27T21:38:10.420" Score="0" Body="&lt;p&gt;If intelligence is defined as utility relation to a task, any algorithm can be said to be intelligent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the task is convincing a human that an algorithm is human, and the algorithm achieves this goal, it can be said to be strongly intelligent in relation to the task (fooling the human.  Here the term strong is used because the algorithm's performance is stronger than the humans.  Strength is relative, and Turing Tests are unavoidably subjective.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this does not mean that the algorithm is generally strongly intelligent, because it may not exceed human capability in &lt;em&gt;all&lt;/em&gt; tasks. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-06-27T21:38:10.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13102" PostTypeId="2" ParentId="7431" CreationDate="2019-06-27T22:06:07.463" Score="1" Body="&lt;p&gt;The paper &lt;a href=&quot;https://content.sciendo.com/view/journals/jagi/5/1/article-p1.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence: Concept, State of the Art, and Future Prospects&lt;/a&gt; (2014), by Ben Goertzel (one of the people that are really still very interested in AGI), surveys the field of artificial general intelligence (AGI), its progress, approaches, mathematical formalisms, engineering, and biology-inspired perspectives, and metrics for assessing AGI.&lt;/p&gt;&#xA;&lt;p&gt;Just to give a little bit more context and whet your appetite, let me briefly describe the different approaches to AGI (section 3, p. 14).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;symbolic&lt;/strong&gt; approach (which is based on the &lt;a href=&quot;http://ai.stanford.edu/users/nilsson/OnlinePubs-Nils/PublishedPapers/pssh.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Physical Symbol System Hypothesis&lt;/a&gt;; examples of this approach are &lt;em&gt;ACT-R&lt;/em&gt; or &lt;em&gt;SOAR&lt;/em&gt;),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;emergentist&lt;/strong&gt; approach (aka &lt;strong&gt;sub-symbolic&lt;/strong&gt;, i.e. the use of neural networks, and similar &lt;em&gt;sub-symbolic&lt;/em&gt; models, from which abstract symbolic processing/reasoning can or is expected to &lt;em&gt;emerge&lt;/em&gt;; so examples of this approach is &lt;em&gt;deep learning&lt;/em&gt;, &lt;em&gt;computational neuroscience&lt;/em&gt;, and &lt;em&gt;artificial life&lt;/em&gt;),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;hybrid&lt;/strong&gt; approach (a combination of the symbolic and sub-symbolic approaches; examples of this approach are &lt;em&gt;CLARION&lt;/em&gt; and &lt;em&gt;CogPrime&lt;/em&gt;), and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;universalist&lt;/strong&gt; approach (examples of this approach are the &lt;em&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/10377/2444&quot;&gt;AIXI&lt;/a&gt;&lt;/em&gt; and &lt;em&gt;Gödel machine&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T13:19:30.280" LastActivityDate="2021-01-18T13:19:30.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13103" PostTypeId="2" ParentId="6985" CreationDate="2019-06-27T23:26:07.133" Score="1" Body="&lt;p&gt;There are a few technical papers and books on the topic&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.136.6387&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Limitations on Learning from Examples&lt;/a&gt; (1988) by Leonard Pitt and Leslie G. Valiant, published in Journal of the ACM&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608005800103&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training a 3-node neural network is NP-complete&lt;/a&gt; (1992) by Avrim L. Blum and Ronald L. Rivest, published in Neural Networks&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.70.7434&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational complexity of neural networks: a survey&lt;/a&gt; (1994) by Pekka Orponen, published in Nordic Journal of Computing&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/374143&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the relative time complexities of standard and conjugate gradient backpropagation&lt;/a&gt; (1994) by J. V. Stone and R. Lister, published in IEEE&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks - A Systematic Introduction&lt;/a&gt; (1996), specifically chapter 10 &lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Complexity of Learning&lt;/a&gt;, a book by Raul Rojas&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://core.ac.uk/download/pdf/82200790.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network&lt;/a&gt; (2014) by Gursel Serpen and Zhenning Gao, published in Complex Adaptive Systems&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://papers.nips.cc/paper/5267-on-the-computational-efficiency-of-training-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the Computational Efficiency of Training Neural Networks&lt;/a&gt; (2014) by Roi Livni, Shai Shalev-Shwartz and Ohad Shamir, published in NIPS 2014&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/He_Convolutional_Neural_Networks_2015_CVPR_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Networks at Constrained Time Cost&lt;/a&gt; (2014) Kaiming He and  Jian Sun, published in IEEE Conference on Computer Vision and Pattern Recognition (2015)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1709.06010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Neural Networks with Two Nonlinear Layers in Polynomial Time&lt;/a&gt; (2018) by Surbhi Goel and Adam Klivans&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, note that gradient descent (and other optimization algorithms) and the &lt;a href=&quot;http://www.cs.cmu.edu/~bhiksha/courses/deeplearning/Fall.2016/pdfs/Werbos.backprop.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;back-propagation algorithm&lt;/a&gt; are &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical algorithms&lt;/a&gt; (that is, they deal with numerical errors), so the time complexity is not the only factor affecting the actual performance of these algorithms and models.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-27T23:35:14.717" LastActivityDate="2019-06-27T23:35:14.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13104" PostTypeId="5" CreationDate="2019-06-27T23:32:42.747" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-27T23:32:42.747" LastActivityDate="2019-06-27T23:32:42.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13105" PostTypeId="4" CreationDate="2019-06-27T23:32:42.747" Score="0" Body="For questions related to the (computational) complexity (e.g. time and space complexity) of AI algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-27T23:32:42.747" LastActivityDate="2019-06-27T23:32:42.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13106" PostTypeId="1" AcceptedAnswerId="13110" CreationDate="2019-06-28T02:59:11.260" Score="3" ViewCount="139" Body="&lt;p&gt;Take AlexNet for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YZce8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YZce8.png&quot; alt=&quot;combined gpu stream of alexnet&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, only the activation function ReLU is used. Due to the fact ReLU cannot be saturated, it instead explodes, like in the following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have a weight matrix of &lt;code&gt;[-1,-2,3,4]&lt;/code&gt; and inputs of &lt;code&gt;[ReLU(4), ReLU(5), ReLU(-2), Relu(-3)]&lt;/code&gt;. The resultant matrix from these will have large numbers for the inputs of &lt;code&gt;ReLU(4)&lt;/code&gt; and &lt;code&gt;ReLU(5)&lt;/code&gt;, and 0 for &lt;code&gt;ReLU(-2)&lt;/code&gt; and &lt;code&gt;ReLU(-3)&lt;/code&gt;. If there are even just a few more layers, the numbers are quick to either explode or be 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this typically combated? How do you keep these numbers towards 0? I understand you can take subtract the mean at the end of each layer, but for a layer that is already in the millions, subtracting the mean will still result in thousands.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2019-06-28T10:51:25.937" LastActivityDate="2019-06-28T10:51:25.937" Title="How are exploding numbers in a forward pass of a CNN combated?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13107" PostTypeId="2" ParentId="11438" CreationDate="2019-06-28T06:10:23.413" Score="0" Body="&lt;p&gt;You can do that and you'll propably find data, yet that depends on the kind of FAQ data you  will apply it on. More importantly, what insight do you gain by comparing two BERT models? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, f you mean with semantic similarity that vector space embeddings are used, even for the retrieval/ranking and not just for re-ranking, then I can tell you that the performance still isn't SOTA. But you can simply use a such a neural semantic model for re-ranking. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are working on that. So if you wanna know more, PM me. &lt;/p&gt;&#xA;" OwnerUserId="19693" LastActivityDate="2019-06-28T06:10:23.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13108" PostTypeId="1" CreationDate="2019-06-28T08:23:08.617" Score="2" ViewCount="25" Body="&lt;p&gt;I want to create a framework that allows GDL to be applied to time-varying graphs.&#xA;I came up with the Erdos-renyi model as an example of a time-varying graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GDL for graphs takes node information as input and takes correspondence with correct data as accuracy. However, how should I deal with time-varying data, even random, and ground-truth data for such graphs? Or is there other better way?&#xA;And is it nonsense to use pseudo-coordinate as input, as it refers to the &lt;a href=&quot;https://arxiv.org/abs/1611.08402&quot; rel=&quot;nofollow noreferrer&quot;&gt;traditional approach&lt;/a&gt; to time-invariant graphs?&#xA;Also, an application of time-varying graphs has been anomaly detection in financial networks. How does this work specifically? Also, please let me know if there are other application examples.&lt;/p&gt;&#xA;" OwnerUserId="26502" LastEditorUserId="26502" LastEditDate="2019-06-28T08:43:02.460" LastActivityDate="2019-06-28T08:43:02.460" Title="Random graph as input in geometric deep learning on time-varying graph" Tags="&lt;machine-learning&gt;&lt;graphs&gt;&lt;geometric-deep-learning&gt;&lt;graph-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13109" PostTypeId="1" AcceptedAnswerId="13111" CreationDate="2019-06-28T09:03:02.233" Score="1" ViewCount="106" Body="&lt;p&gt;As mentioned in the title I'm using 300 Dataset example with 500 feature as an input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I'm training the dataset, I found something peculiar.  Please look at the data shown below.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Iteration 5000 | Cost: 2.084241e-01&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Training Set Accuracy: 100.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CV Set Accuracy: 85.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Test Set Accuracy: 97.500000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Iteration 3000 | Cost: 2.084241e-01&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Training Set Accuracy: 98.958333&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CV Set Accuracy: 85.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Test Set Accuracy: 97.500000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Iteration 1000 | Cost: 4.017322e-01&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Training Set Accuracy: 96.875000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CV Set Accuracy: 85.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Test Set Accuracy: 97.500000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Iteration 500 | Cost: 5.515852e-01&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Training Set Accuracy: 95.486111&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CV Set Accuracy: 90.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Test Set Accuracy: 97.500000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Iteration 100 | Cost: 8.413299e-01&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Training Set Accuracy: 90.625000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CV Set Accuracy: 95.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Test Set Accuracy: 97.500000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Iteration 50 | Cost: 8.483802e-01&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Training Set Accuracy: 90.277778&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CV Set Accuracy: 95.000000&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Test Set Accuracy: 97.500000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The trend is that as the Iteration(cost) increases(cost decreases), the training set accuracy increases as expected, but the CV Set/Test Set Accuracy decreases. My initial thought is that this has to do with precision/bias issue, but I really can't buy it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone know what this entails? Or any reference? &lt;/p&gt;&#xA;" OwnerUserId="25797" LastActivityDate="2019-06-28T09:17:24.363" Title="In NN, as iterations of Gradient descent increases, the accuracy of Test/CV set decreases. how can i resolve this?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13110" PostTypeId="2" ParentId="13106" CreationDate="2019-06-28T09:10:08.543" Score="2" Body="&lt;p&gt;The most effective way to prevent both the forward and backward propagation of &lt;em&gt;exploding&lt;/em&gt; is keeping the weights in a small range. The main way this is accomplished is through their &lt;strong&gt;initialization&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in the case of &lt;a href=&quot;https://arxiv.org/pdf/1502.01852.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;He initialization&lt;/a&gt;, the authors show (given some assumptions) that the variance of the output of the final layer &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; of the network is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Var[y_L] = Var[y_1] \left( \prod_{i=2}^L{\frac{1}{2} \, n_l \, Var[w_l]} \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$n_l$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_l$&lt;/span&gt; are the number of connections and weights of layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;. In order to keep the outputs from exploding the product above should not exponentially magnify its inputs. In order to do this the authors elect to initialize the weights so that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{1}{2} \, n_l \, Var[w_l] = 1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this is helps keeping the outputs from exploding. They then go and prove that the same strategy helps preventing the gradients from exploding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another similar strategy is the so-called &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Glorot (or Xavier) Initialization&lt;/a&gt;.&#xA;These techniques are &lt;strong&gt;extremely effective&lt;/strong&gt; in helping the models converge!&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-06-28T09:10:08.543" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13111" PostTypeId="2" ParentId="13109" CreationDate="2019-06-28T09:17:24.363" Score="1" Body="&lt;p&gt;Training scores improving (loss decreasing and accuracy increasing) whilst the opposite happens with cross validation and test data is a sign of &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting to the training data&lt;/a&gt;. Your neural network is getting worse at generalising and no amount of further training will improve it - in fact the situation will get worse the more you train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the main reason you have CV data sets, to show you when this happens before you try to use your model against the test set or real world data. So it is not &quot;peculiar&quot; at all, but the CV set doing its job for you, allowing you notice something has gone wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To improve on the situation, you need to use some form of &lt;a href=&quot;https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a&quot; rel=&quot;nofollow noreferrer&quot;&gt;regularisation&lt;/a&gt;. The simplest approach here would be to take your model from around 100 epochs (because it has best CV score that I can see). This is early stopping, and is a simple valid regularisation approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatives for neural networks include L2 weight regularisation (also called weight decay) and dropout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to this, your question states that you only have 300 examples, and more features than examples. This is really tricky to generalise (and hard to tell if you have - for instance it looks like you only have 20 CV examples and 40 test examples, these numbers are very low and prone to giving you inaccurate estimates simply due to chance of what entries are in the data sets). I recommend you look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-validation_(statistics)#k-fold_cross-validation&quot; rel=&quot;nofollow noreferrer&quot;&gt;K-fold cross-validation&lt;/a&gt; in order to get more accurate scores, and help you choose the best model for generalisation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-06-28T09:17:24.363" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13112" PostTypeId="5" CreationDate="2019-06-28T11:03:44.920" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Data_pre-processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Data_pre-processing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-28T11:03:44.920" LastActivityDate="2019-06-28T11:03:44.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13113" PostTypeId="4" CreationDate="2019-06-28T11:03:44.920" Score="0" Body="For questions related to the concept of data pre-processing, which includes, for example, cleaning, instance selection, normalization, transformation, feature extraction or selection." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-28T11:03:44.920" LastActivityDate="2019-06-28T11:03:44.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13115" PostTypeId="2" ParentId="1987" CreationDate="2019-06-28T12:21:43.583" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0FQY9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0FQY9.png&quot; alt=&quot;vanilla tensorflow playground spiral solution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an example of vanilla Tensorflow playground with no added features and no modifications.&#xA;The run for Spiral was between 187 to ~300 Epoch, depending. &#xA;I used Lasso Regularization L1 so I could eliminate coefficients. &#xA;I decreased the batch size by 1 to keep the output from over fitting. &#xA;In my second example I added some noise to the data set then upped the L1 to compensate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OXPkd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OXPkd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26733" LastActivityDate="2019-06-28T12:21:43.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13116" PostTypeId="2" ParentId="5410" CreationDate="2019-06-28T13:57:16.310" Score="0" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://content.sciendo.com/view/journals/jagi/5/1/article-p1.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence: Concept, State of the Art, and Future Prospects&lt;/a&gt; (2014) Ben Goertzel gives an overview of the AGI field and its progress. He describes the main approaches (symbolic, emergentist, hybrid and universalist) to AGI and metrics to assess human-level intelligence and partial progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also an older book by Cassio Pennachin and Ben Goertzel called &lt;a href=&quot;https://link.springer.com/book/10.1007/978-3-540-68677-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt; (2007). A chapter of the book, &lt;a href=&quot;https://bilder.buecher.de/zusatz/13/13732/13732357_lese_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contemporary Approaches to Artificial General Intelligence&lt;/a&gt;, gives a brief history of the AGI field.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T14:44:04.563" LastActivityDate="2019-07-06T14:44:04.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13117" PostTypeId="1" AcceptedAnswerId="13118" CreationDate="2019-06-28T14:20:09.490" Score="1" ViewCount="74" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Planning_Domain_Definition_Language&quot; rel=&quot;nofollow noreferrer&quot;&gt;Planning Domain Definition Language (PDDL)&lt;/a&gt; is known for its capabilities of symbolic planning in the state space. A solver will find a sequence of steps to bring the system from a start state to the goal state. A common example of this is &lt;a href=&quot;https://en.wikipedia.org/wiki/Monkey_and_banana_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;the monkey-and-banana problem&lt;/a&gt;. At first, the monkey sits on the ground and, after doing some actions in the scene, the monkey will have reached the banana.&lt;/p&gt;&#xA;&lt;p&gt;The way a PDDL planner works is by analyzing the preconditions and effects of each primitive action. This will answer the question of what happens if a certain action is executed.&lt;/p&gt;&#xA;&lt;p&gt;However, will a PDDL domain description work the other way around as well, not for planning, but for action recognition?&lt;/p&gt;&#xA;&lt;p&gt;I've searched in the literature to get an answer, but all the papers I've found are describing &lt;a href=&quot;https://www.researchgate.net/publication/220003750_Context_Aware_Approach_for_Activity_Recognition_Based-on_Precondition-Effect_Rules&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDDL only as a planning paradigm&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My idea is to use the given precondition and effects as a parser to identify what the monkey is doing and not what he should do. That means, in the example, the robot ape knows by itself how to reach the banana and the AI system has to monitor the actions. The task is to identify a PDDL action that fits the action by the monkey.&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="2444" LastEditDate="2021-02-06T13:54:23.613" LastActivityDate="2021-02-06T13:54:23.613" Title="Can PDDL be utilized for action recognition?" Tags="&lt;planning&gt;&lt;action-recognition&gt;&lt;pddl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13118" PostTypeId="2" ParentId="13117" CreationDate="2019-06-28T16:07:39.283" Score="1" Body="&lt;p&gt;I don't know of any work on this with respect to PDDL, but this is very similar to a conceptual dependency application called SAM (Script Applier Mechanism). &lt;a href=&quot;https://en.wikipedia.org/wiki/Conceptual_dependency_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conceptual Dependency&lt;/a&gt; (CD) models actions using a number of primitives (which could be seen as equivalent to PDDL primitive actions):  PTRANS for physical transfer, PROPEL for application of a physical force to an object, GRASP for grasping an object, etc. Their number varies around 12 or so, depending on the version of CD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stories are described by a sequence of primitive acts, which have slots for actor, object, etc. They are supposed to enable a program to draw inferences about what happens. A common problem when trying to understand stories is that often common knowledge about a situation is omitted; the standard example here is going to a restaurant. It is usually assumed that the listener/reader knows what commonly happens when the protagonist enters a restaurant, so that when they leave without paying this is recognised as something unusual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach used to solve this problem is to encode such knowledge in scripts, sequences of primitive acts. When triggered, eg by &quot;Manuel went to a restaurant&quot;, this script is retrieved, and the following actions are looked for in the script. Anything that is recognised is used to fill gaps in the story, eg sitting down at a table, or looking at a menu. This was the task of the SAM program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically you have a sequence of primitive actions, and you try to recognise a more abstract event &quot;going to a restaurant&quot; from that. Obviously you'd need to have a script to recognise, but one could presumably use this to derive a sequence of more generalised events, such as &quot;retrieving an object from a high place&quot;, or &quot;standing on top on another object&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory of using scripts, plans, and goals to describe human reasoning is detailed in Schank, Roger; Abelson, Robert P. (1977). &lt;em&gt;Scripts, plans, goals and understanding: An inquiry into human knowledge structures.&lt;/em&gt; New Jersey: Erlbaum. ISBN 0-470-99033-3.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-06-28T16:07:39.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13119" PostTypeId="1" AcceptedAnswerId="13123" CreationDate="2019-06-28T17:54:48.887" Score="3" ViewCount="231" Body="&lt;p&gt;There are reinforcement learning papers (e.g. &lt;a href=&quot;https://arxiv.org/pdf/1705.02670.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metacontrol for Adaptive Imagination-Based Optimization&lt;/a&gt;) that use (apparently, interchangeably) the term &lt;em&gt;control&lt;/em&gt; or &lt;em&gt;action&lt;/em&gt; to refer to the effect of the agent on the environment at each time step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any difference between the terms &lt;em&gt;control&lt;/em&gt; or &lt;em&gt;action&lt;/em&gt; or are they (always) used interchangeably? If there is a difference, when is one term used as opposed to the other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term control likely comes from the field of optimal control theory, which is related to reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T22:14:12.097" LastActivityDate="2019-07-23T22:14:12.097" Title="Is there any difference between a control and an action in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;comparison&gt;&lt;control-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13120" PostTypeId="1" CreationDate="2019-06-28T19:45:57.840" Score="3" ViewCount="935" Body="&lt;p&gt;Sometimes when I am training a DC-GAN on an image dataset, similar to the DC-GAN PyTorch example (&lt;a href=&quot;https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html&lt;/a&gt;), either the Generator or Discriminator will get stuck in a large value while the other goes to zero. How should I interpret what is going on right after iteration 1500 in the example loss function image shown below &lt;a href=&quot;https://i.stack.imgur.com/AElrJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AElrJ.png&quot; alt=&quot;DC-GAN Training Loss&quot;&gt;&lt;/a&gt;? Is this an example of mode collapse? Any recommendations for how to make the training more stable? I have tried reducing the learning rate of the Adam optimizer with varying degrees of success. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21952" LastActivityDate="2020-04-08T23:40:30.223" Title="What parameters can be tweaked to avoid a generator or discriminator loss collapsing to zero when training a DC-GAN?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13121" PostTypeId="1" CreationDate="2019-06-29T05:37:44.173" Score="1" ViewCount="18" Body="&lt;p&gt;Reference managers like Zotero or Mendeley allow researchers to categorize papers into hierarchical categories called collections. The User navigates through a listing of these collections when filing a new item. The retrieval time grows something like the logarithm of the number of collections; looking for a collection can quickly become a nuisance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m5Ptq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m5Ptq.png&quot; alt=&quot;A top level view of a collection hierarchy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;sup&gt;&lt;sub&gt; Fig 1. A top level list of collections in the Zotero reference manager &lt;/sub&gt;&lt;/sup&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to reduce navigation time is to allow users to search collections by name. A complementary solution is to provide a view of the currently &quot;hot&quot; collections. The user may interact with a list of suggested collections, or receive relevant completions when typing into a collections search bar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This raises a basic learning problem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$K = K_1, \ \dots, \ K_m$&lt;/span&gt; be the sequence of collections the user has visited (possibly augmented with visit times). Let &lt;span class=&quot;math-container&quot;&gt;$H_m$&lt;/span&gt; be a set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; collections the user is likely to visit next. How can we construct &lt;span class=&quot;math-container&quot;&gt;$H_m$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A technique that does this this might exploit a few important features of this domain:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Project Clusters: Users jump between collections relevant to their current projects&lt;/li&gt;&#xA;&lt;li&gt;Collection Aging: Users tend to focus on new collections, and forget older ones&lt;/li&gt;&#xA;&lt;li&gt;Retrieval Cost: There's a tangible cost to the user (time, distraction) when navigating collections; this applies to the reduced view (the technique might keep &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; as small as possible)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Two ideas so far&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;LIFO Cache&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reduce &lt;span class=&quot;math-container&quot;&gt;$K_{m-1}, K_{m-2},\ \dots$&lt;/span&gt; into the first &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; unique entries which do not match &lt;span class=&quot;math-container&quot;&gt;$K_m$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This heuristic is very simple to implement and requires no learning. It encompasses clusters and aging given suitably large &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. But with large &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; it incurs a retrieval cost of its own.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Markov Chain/Hidden Markov Model&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Use &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; to train a MC or HMM. Build &lt;span class=&quot;math-container&quot;&gt;$H_m$&lt;/span&gt; using estimates from this model.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The simplest version is an order &lt;span class=&quot;math-container&quot;&gt;$k\ $&lt;/span&gt; MC transition matrix built using k-gram statistics of &lt;span class=&quot;math-container&quot;&gt;$K_i$&lt;/span&gt;. This might be sensitive to project clusters, but I don't think it will recognize new collections without a hard coded aging heuristic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not clear on how HMMs would be trained here, and I'm not very taken with the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-gram MC approach. My next task is to read about MCs/HMMs in context of suggestion systems.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Other Models&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I am brand new to suggestion systems. Reading leads are quite welcome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be especially excited about unsupervised techniques, and neural network techniques I could train on a GPU. Apart from improving Zotero, I would like this problem to give me an opportunity to learn about cutting edge techniques.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Valuable Goals&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;An ideal technique would cast light on questions like&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How should we measure the performance of this kind of system? (I suspect cache miss rate is a good metric, as well as the ratio between cache miss rate and cache size)&lt;/li&gt;&#xA;&lt;li&gt;How should we translate these human-centric performance metrics into human independent objective functions for learning?&lt;/li&gt;&#xA;&lt;li&gt;How much better than LIFO can we theoretically expect to do with a more sophisticated technique (say, in terms of cache size for a given cache miss rate)?&lt;/li&gt;&#xA;&lt;li&gt;How can a technique learn patterns like clusters and aging without hand tuned objectives?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am interested in learning theory and building an implementation, so resources with publicly available code would be be preferable. Apart from potentially being overkill for the problem, I would not mind if the final model depends on a GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please forgive me for the long and vague question. I wish I had done more reading before posing this question, but I feel a bit stuck! I hope to get unstuck with some good reading resources. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="26746" LastActivityDate="2019-06-29T05:37:44.173" Title="Predicting Hot Categories In a Reference Manager" Tags="&lt;unsupervised-learning&gt;&lt;prediction&gt;&lt;reference-request&gt;&lt;markov-chain&gt;&lt;hidden-markov-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13122" PostTypeId="2" ParentId="12734" CreationDate="2019-06-29T06:20:10.943" Score="0" Body="&lt;p&gt;Assuming that your cross-validation scores(both on train set and test set) indicate model's prediction performance correctly, you should definitely decide which trained model to use based on your &lt;strong&gt;validation accuracy only,&lt;/strong&gt; regardless your model is overfitted or not. &lt;/p&gt;&#xA;" OwnerUserId="17094" LastActivityDate="2019-06-29T06:20:10.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13123" PostTypeId="2" ParentId="13119" CreationDate="2019-06-29T08:03:29.723" Score="3" Body="&lt;p&gt;There's no difference. As they too often do, ML researchers take concepts from other disciplines, conveniently forget to cite sources and change the terminology, leading to much confusion. RL is a textbook example (pun intended). Optimal control researchers have been studying very similar problems long before RL ones, and used standard symbols and terms (&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; for states, &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; for controls). Then RL researchers came and changed just about everything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the paper &lt;a href=&quot;https://arxiv.org/pdf/1806.09460.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Tour of Reinforcement Learning: The View from Continuous Control&lt;/a&gt; (2018), by Benjamin Recht, which discusses reinforcement learning from a control and optimization perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also this tweet &lt;a href=&quot;https://twitter.com/beenwrekt/status/1134536093980864514?s=21&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/beenwrekt/status/1134536093980864514?s=21&lt;/a&gt; (by Benjamin Recht) regarding the presentation of Sham Kakade.&lt;/p&gt;&#xA;" OwnerUserId="20874" LastEditorUserId="2444" LastEditDate="2019-07-21T17:47:58.627" LastActivityDate="2019-07-21T17:47:58.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13124" PostTypeId="2" ParentId="112" CreationDate="2019-06-29T09:44:39.350" Score="3" Body="&lt;p&gt;Self-driving cars use a combination of both supervised as well as reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Huge amounts of sensor data are recorded in real-time. This data can be used to train all sorts of supervised classifiers, e.g. for predicting rain or switching on lights. You can also set up a model to predict pedestrians and other cars. This is supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement learning can be used in situations positive or negative signals appear when driving a car: Traffic lights, blinking signals from other vehicles and street signs in general. These signals can be used to train a reinforcement model and decide on best actions (adjust speed, steer,..) to get the maximal reward (or better minimize costs of a crash)&lt;/p&gt;&#xA;" OwnerUserId="26747" LastEditorUserId="26747" LastEditDate="2019-06-29T15:24:10.747" LastActivityDate="2019-06-29T15:24:10.747" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13125" PostTypeId="1" AcceptedAnswerId="13127" CreationDate="2019-06-29T10:46:52.620" Score="2" ViewCount="113" Body="&lt;p&gt;I have implemented an epsilon-greedy Monte Carlo reinforcement learning agent like suggested in &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's RL book&lt;/a&gt; (page 101). As far as I understood epsilon-greedy agents so far, the evaluation has to stop at some point to exploit the gained knowledge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand, how to stop the evaluation here, because the policy update is linked to epsilon. So just setting epsilon equal to zero at some point does not seem to make sense to me.&lt;/p&gt;&#xA;" OwnerUserId="23288" LastEditorUserId="2444" LastEditDate="2019-06-29T11:41:03.150" LastActivityDate="2019-08-02T03:03:20.323" Title="How to stop evaluation phase in reinforcement learning with epsilon-greedy Monte Carlo agent?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13126" PostTypeId="2" ParentId="112" CreationDate="2019-06-29T12:03:03.173" Score="3" Body="&lt;p&gt;What you are calling 'analyzing the surroundings' is generally referred to as perception. Self-driving cars sense their surroundings using cameras, radars, lidars often combining or fusing more than one sensor to paint a picture of the environment. A lot of algorithms get used for fusing the sensor data and then deriving an understanding of the surrounding. One such example is semantic scene segmentation of camera data that tries to identify object boundaries in camera images. Typically a fully convolutional neural network is used to achieve this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To the best of my knowledge Google does not disclose the exact algorithms anywhere.   &lt;/p&gt;&#xA;" OwnerUserId="23273" LastActivityDate="2019-06-29T12:03:03.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13127" PostTypeId="2" ParentId="13125" CreationDate="2019-06-29T12:30:58.487" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I do not understand, how to stop the evaluation here, because the policy update is linked to epsilon. So just setting epsilon equal to zero at some point does not seem to make sense to me.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If by evaluation you mean the act of exploring new paths, it does not need to stop by changing epsilon to 0 instantly. Instead, in order to facilitate the convergence of the algorithm, the epsilon can be progressively decreased until it reaches 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not think your question is meant to be related with monte carlo specifically, but if this was not what you wanted to know, please comment.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But would not a epsilon value of 0 lead to an unintended policy update?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, a value of epsilon equal to 0 will make you always choose your action according to the policy. I think what is confusing is that the &quot;update&quot; they make to the policy on the last for, is not an update. It is a statement saying the probability of taking such action. What it means is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, the probability of taking &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;, when &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is 0 is: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 if action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the action with the best value&lt;/li&gt;&#xA;&lt;li&gt;0 if action  &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is not the action with the best value&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What this means is that when epsilon is 0, the action taken will always be the greedy action.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-06-29T14:01:28.383" LastActivityDate="2019-06-29T14:01:28.383" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13133" PostTypeId="1" CreationDate="2019-06-30T07:22:51.407" Score="2" ViewCount="44" Body="&lt;p&gt;I am learning about discretization of the state space when applying reinforcement learning to continuous state space. In &lt;a href=&quot;https://www.youtube.com/watch?v=Uu1J5KLAfTU&amp;amp;feature=youtu.be&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; the instructor, at 2:02, the instructor says that one benefit of this approach (radial basis functions over tile coding) is that &amp;quot;it drastically reduces the number of features&amp;quot;. I am not able to deduce this in the case of a simple 2D continuous state space.&lt;/p&gt;&#xA;&lt;p&gt;Suppose we are dealing with 2D continuous state space, so any state is a pair &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; in the Cartesian space. If we use Tile Coding and select &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; tiles, the resulting encoding will have &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; features, consisting of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; discrete valued pairs &lt;span class=&quot;math-container&quot;&gt;$(u_1, v_1) \dots (u_n, v_n)$&lt;/span&gt; representing the approximate position of &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; in the frames of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; 2-D tiles. If instead we use &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; 2-D circles and encode using the distance of &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; from the center of each circle, we have &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; (continuous) features.&lt;/p&gt;&#xA;&lt;p&gt;Is there a reason to assume that &lt;span class=&quot;math-container&quot;&gt;$m &amp;lt; 2n$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional feature vector will again need discretization, so it is unclear to me how this approach uses fewer features.&lt;/p&gt;&#xA;" OwnerUserId="23273" LastEditorUserId="2444" LastEditDate="2021-01-21T02:43:41.153" LastActivityDate="2021-01-21T02:43:41.153" Title="Does coarse coding with radial basis function generate fewer features?" Tags="&lt;reinforcement-learning&gt;&lt;function-approximation&gt;&lt;features&gt;&lt;tile-coding&gt;&lt;coarse-coding&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13134" PostTypeId="1" CreationDate="2019-06-30T10:15:58.337" Score="1" ViewCount="31" Body="&lt;p&gt;I am trying to do RoI pooling of an image whose RoI bounding box coordinates are known to me. Can anyone provide me the code for doing the same? I have read many repositories but I didn't get any generic code for testing on an image. I want a RoI pooled 7x7 matrix given an image as an input and pretrained VGG 16 model for extracting the feature map.&lt;/p&gt;&#xA;" OwnerUserId="26720" LastActivityDate="2019-06-30T10:15:58.337" Title="Issues related to RoI Pooling in keras" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;keras&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-06-02T22:42:24.767" ContentLicense="CC BY-SA 4.0" />
  <row Id="13135" PostTypeId="1" CreationDate="2019-06-30T15:12:14.500" Score="4" ViewCount="197" Body="&lt;p&gt;At a time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, for a state &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt;, the return is defined as the discounted cumulative reward from that time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an agent is following a policy (which in itself is a probability distribution of choosing a next state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt;), the agent wants to find the value at &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt; by calculating sort of &quot;weighted average&quot; of all the returns from &lt;span class=&quot;math-container&quot;&gt;$S_{t}.$&lt;/span&gt; This is called the expected return.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="26764" LastEditorUserId="2444" LastEditDate="2021-01-22T17:13:15.317" LastActivityDate="2021-02-21T21:04:10.350" Title="What is the difference between return and expected return?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;return&gt;&lt;expectation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13136" PostTypeId="2" ParentId="13135" CreationDate="2019-06-30T15:39:17.697" Score="1" Body="&lt;p&gt;Formally, the &lt;strong&gt;return&lt;/strong&gt; (also known as the &lt;em&gt;cumulative future discounted reward&lt;/em&gt;) can be defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;G_t = \sum_{k=0}^\infty \gamma^k R_{t+k+1},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$0 \leq \gamma \leq 1$&lt;/span&gt; is the discount factor and &lt;span class=&quot;math-container&quot;&gt;$R_{i}$&lt;/span&gt; is the reward at time step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. Here &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; are considered random variables (and r.v.s are usually denoted with capital letters, so I am using the notation used in the book &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;, &lt;strong&gt;2nd edition&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;expected return&lt;/strong&gt; is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;v^\pi(s) &#xA;&amp;amp;= \mathbb{E}\left[G_t \mid S_t = s \right] \\&#xA;&amp;amp;= \mathbb{E}\left[\sum_{k=0}^\infty \gamma^k R_{t+k+1} \bigm\vert S_t = s \right]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In other words, the value of a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (associated with a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;) is equal to the expectation of the return &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; given that &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$v^\pi(s)$&lt;/span&gt; is defined as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_expectation&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional expectation&lt;/a&gt;. Note also that the expected value is usually defined with respect to a random variable, which is the case. Note also that &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; is a random variable, while &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is a realization of this random variable.&lt;/p&gt;&#xA;&lt;p&gt;A policy is &lt;strong&gt;not&lt;/strong&gt; a probability distribution of choosing the next state. A stochastic policy is a family of a conditional probability distribution &lt;em&gt;over actions given states&lt;/em&gt;. There are also deterministic policies. Have a look at this question &lt;a href=&quot;https://ai.stackexchange.com/q/12274/2444&quot;&gt;What is the difference between a stochastic and a deterministic policy?&lt;/a&gt; for more details about the definition of stochastic and deterministic policies.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If an agent is following a policy, the agent wants to find the value at &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt; by calculating a sort of &amp;quot;weighted average&amp;quot; of all the returns from &lt;span class=&quot;math-container&quot;&gt;$S_{t}.$&lt;/span&gt; This is called the expected return.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the case of Monte Carlo Prediction, the value of a state associated with a specific policy, that is, the expected value of the return given a state is approximated with a finite (weighted) average. See e.g. &lt;a href=&quot;https://ai.stackexchange.com/a/10818/2444&quot;&gt;What is the difference between First-Visit Monte-Carlo and Every-Visit Monte-Carlo Policy Evaluation?&lt;/a&gt;. Furthermore, note that the &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectation&lt;/a&gt; of a &lt;em&gt;discrete&lt;/em&gt; random variable is defined as a weighted average.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-22T20:56:33.813" LastActivityDate="2021-01-22T20:56:33.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13137" PostTypeId="5" CreationDate="2019-06-30T17:56:40.707" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;http://www.incompleteideas.net/book/first/ebook/node44.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.incompleteideas.net/book/first/ebook/node44.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-30T17:56:40.707" LastActivityDate="2019-06-30T17:56:40.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13138" PostTypeId="4" CreationDate="2019-06-30T17:56:40.707" Score="0" Body="For questions related to the value iteration algorithm, which is a dynamic programming (DP) algorithm used to solve an MDP, that is, it is used to find a policy given the transition and reward functions of the MDP. Value iteration is related to another DP algorithm called policy iteration." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-06-30T17:56:40.707" LastActivityDate="2019-06-30T17:56:40.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13139" PostTypeId="1" CreationDate="2019-06-30T19:27:23.217" Score="1" ViewCount="32" Body="&lt;p&gt;I have a set of polygons for each image. Those polygons consist of four &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; coordinates. For each image, I need to extract the ones of interest. This could be formulated as an Image Segmentation task where, for example I want to extract the objects of interest, here: &lt;strong&gt;cars&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E7A7y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E7A7y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But since I already get the polygons through a different part of my pipeline I would like to create a simpler machine learning model. The input will not be the image but only the coordinates of the polygons. &#xA;In this model each sample should consist of multiple polygons (those can vary in number) and the model should output the ones of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my mind, I formulated the problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The polygons are the features. Problem: Samples will have varying number of features.&lt;/li&gt;&#xA;&lt;li&gt;The output will consist of the indices of the &quot;features&quot; (polygons) I am interested in.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;First, I created a decision tree and classified each coordinate as &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (not interested in) or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (of interest). But, by doing this, I don't consider the other coordinates that belong to the image. The information of the surrounding is lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone have an idea of how to model this problem without using Image Segmentation?&lt;/p&gt;&#xA;" OwnerUserId="23063" LastEditorUserId="23063" LastEditDate="2019-07-03T04:40:45.577" LastActivityDate="2019-07-03T04:40:45.577" Title="Which model to use when selecting objects of interest?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13140" PostTypeId="1" CreationDate="2019-06-30T21:33:57.707" Score="1" ViewCount="41" Body="&lt;p&gt;I have a unique implementation where I have to process videos with dynamic frame rates (that is the number of frames is different for each video in a batch). I am stacking all the frames in a single tensor and processing the same from there on. This works fine with Conv2D layer but creating a 2D tensor (batch_size, features) by a flattening operation this has to be fed to a Dense layer. I can't find a suitable way to implement this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information on why it should be like this kindly explore: &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiple_instance_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;explore this link&lt;/a&gt;. Instead of the MNIST images, I have multiple videos in a single bag, each with a variable number of frames. &lt;/p&gt;&#xA;" OwnerUserId="17068" LastEditorUserId="12853" LastEditDate="2019-07-01T08:15:07.470" LastActivityDate="2019-07-01T08:15:07.470" Title="Dynamic frames processing with CNN LSTM combination or otherwise" Tags="&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13141" PostTypeId="1" CreationDate="2019-07-01T01:21:34.883" Score="4" ViewCount="34" Body="&lt;p&gt;What are some deep learning models that can use supplementary information other than RGB channels for image segmentation? &lt;a href=&quot;https://i.stack.imgur.com/f8ZjU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f8ZjU.png&quot; alt=&quot;river with indication of flow directions&quot;&gt;&lt;/a&gt; For example imagine a poorly shot image of a river (blue) that shows a gap, and the supplementary information are detailed flow directions (arrows) which help showing the river's true shape (no gap in reality). To get the river shape, most image segmentation models I see such as U-Net only uses RGB channels. &lt;strong&gt;Are there any neural network models that can use this kind of auxiliary information along with RGB channels during training&lt;/strong&gt; for the image segmentation task?&lt;/p&gt;&#xA;" OwnerUserId="26767" LastEditorUserId="2444" LastEditDate="2020-06-13T00:10:06.073" LastActivityDate="2020-06-13T00:10:06.073" Title="What are some neural network models that can use auxiliary info during training for image segmentation?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13143" PostTypeId="1" CreationDate="2019-07-01T04:13:48.613" Score="2" ViewCount="103" Body="&lt;p&gt;I have constructed a CNN that utilizes max-pooling layers. I have found with these layers that, should I remove them, my network performs ideally with every output and gradient at each layer having a variance close to 1. However, if they are included, the variance skyrockets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes sense, of course, as a max-pooling layer takes the maximum of an area, which must incur a positive bias as larger numbers are chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would just like to know what methods are typically used to combat this.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="30725" LastEditDate="2019-11-20T01:55:21.057" LastActivityDate="2019-11-20T01:55:21.057" Title="How is the bias caused by a max pooling layer overcome?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;relu&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13144" PostTypeId="1" CreationDate="2019-07-01T08:55:04.693" Score="2" ViewCount="27" Body="&lt;p&gt;I'm trying to implement this &lt;a href=&quot;http://www.cs.toronto.edu/~urtasun/publications/luo_etal_cvpr18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;approach&lt;/a&gt; for object detection and tracking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this approach, the first step is voxelize each frame to construct a 3D tensor, the second step is to append multiple voxels at the time along a new axis to create a 4D tensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to understand is how to voxelize multiple frames at the time and append them together.&lt;/p&gt;&#xA;" OwnerUserId="26777" LastEditorUserId="2444" LastEditDate="2019-07-01T11:04:58.557" LastActivityDate="2019-07-01T11:04:58.557" Title="How to voxelize multiple frames at the time and append them together?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13145" PostTypeId="1" CreationDate="2019-07-01T10:44:26.793" Score="2" ViewCount="26" Body="&lt;p&gt;I've been struggling to analyize my NN model. I've studied through andrew ng's course, but there are some results that cannot be explained by the course. Is there any useful source on  High Bias vs High variance issue on NN?&lt;/p&gt;&#xA;" OwnerUserId="25797" LastEditorUserId="25797" LastEditDate="2019-07-02T00:29:58.790" LastActivityDate="2019-07-02T00:29:58.790" Title="Is there any useful source on High Bias vs High variance issue on Neural Network?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13146" PostTypeId="1" CreationDate="2019-07-01T11:57:45.210" Score="1" ViewCount="21" Body="&lt;p&gt;Could anyone help to derive equation (15) dL/dx in &lt;a href=&quot;https://arxiv.org/pdf/1802.09769.pdf#page=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks&lt;/a&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="20581" LastActivityDate="2019-07-01T11:57:45.210" Title="L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13147" PostTypeId="1" AcceptedAnswerId="13151" CreationDate="2019-07-01T12:11:40.230" Score="4" ViewCount="587" Body="&lt;p&gt;I'm trying to apply a DQN to a stochastic environment, but I'm having trouble getting it to converge.&lt;/p&gt;&#xA;&lt;p&gt;I found some similar questions asked here, but no solutions yet.&lt;/p&gt;&#xA;&lt;p&gt;I can easily get the DQN to converge on a static environment, but I am having trouble with a dynamic environment where the end is not given.&lt;/p&gt;&#xA;&lt;p&gt;Example: I have made a really simple model of the Frozen Lake (without any holes) - simply navigation from A to B. This navigation works fine when A and B are always the same, but when I shuffle the position of A or B for each session, the DQN cannot converge properly.&lt;/p&gt;&#xA;&lt;p&gt;I am using the grid (3x3, 4x4 sizes) as input neurons. Each with &amp;quot;0&amp;quot; value. I assign the current position &amp;quot;0.5&amp;quot; and the end position &amp;quot;1&amp;quot;. 4x4 grid gives us 16 input neurons. Example of 3x3 grid:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 0.5  0  0 &#xA;  0   0  0 &#xA;  0   0  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a few questions in this regard:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;When training the DQN, how do I apply Q-values? (Or do I really need to? I'm not sure how to correctly &amp;quot;reward&amp;quot; the network. I'm not using any adversarial network or target network at this point.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I train the network using only a short replay memory of the last move, or the last N moves that led to success. Is this the right way to approach this?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I use Keras, and am simply training the network every time it does something right - and ignoring failed attempts. - But is this anywhere near the right approach?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Am I missing something else?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Perhaps I should note that my math skills are not that strong, but I try my best.&lt;/p&gt;&#xA;&lt;p&gt;Any input is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26768" LastEditorUserId="2444" LastEditDate="2020-07-06T00:11:28.503" LastActivityDate="2020-07-06T00:11:28.503" Title="Why does the DQN not converge when the start or goal states can change dynamically?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13148" PostTypeId="1" CreationDate="2019-07-01T12:37:52.300" Score="1" ViewCount="372" Body="&lt;p&gt;In YOLO, why use k-means clustering to determine bounding-box priors ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why if we use standard k-means with Euclidean distance, larger boxes generate more error than smaller boxes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why using &lt;a href=&quot;https://lars76.github.io/object-detection/k-means-anchor-boxes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IOU (Jaccard Index)&lt;/a&gt; can avoid/eliminate such error ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to derive &lt;a href=&quot;https://arxiv.org/pdf/1612.08242.pdf#page=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;d(box, centroid) = 1 - IOU(box, centroid)&lt;/a&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="20581" LastActivityDate="2020-09-04T19:08:35.840" Title="YOLO Architecture - kmeans clustering" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;k-means&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2020-09-04T19:12:19.037" ContentLicense="CC BY-SA 4.0" />
  <row Id="13149" PostTypeId="1" AcceptedAnswerId="13183" CreationDate="2019-07-01T14:35:31.483" Score="3" ViewCount="65" Body="&lt;p&gt;I'm looking for examples of time-varying graph-structured data for time-varying graph CNNs. First, I came up with the idea of infection network. Is there anything more? If possible, I want data that can be easily obtained online.&lt;/p&gt;&#xA;" OwnerUserId="26502" LastEditorUserId="2444" LastEditDate="2019-07-01T18:50:24.880" LastActivityDate="2019-07-03T08:18:28.300" Title="Examples of time-varying graph-structured data in real world" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;geometric-deep-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13150" PostTypeId="1" CreationDate="2019-07-01T14:50:58.557" Score="2" ViewCount="642" Body="&lt;p&gt;I am trying to implement CTC loss in Tensorflow, but &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/ctc_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;their documentation&lt;/a&gt; is pretty limited. So I am not sure how to approach the problem. I found a good example in Theano: &lt;a href=&quot;https://github.com/mohammadpz/CTC-Connectionist-Temporal-Classification/blob/master/ctc_cost.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mohammadpz/CTC-Connectionist-Temporal-Classification/blob/master/ctc_cost.py&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are any other resources that explain the CTC loss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am trying to understand how its forward-backward algorithm works. And what is beam decoder in the case for CTC loss?&lt;/p&gt;&#xA;" OwnerUserId="26787" LastEditorUserId="2444" LastEditDate="2019-07-01T19:14:10.407" LastActivityDate="2019-07-01T19:27:05.480" Title="How does the CTC loss work?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13151" PostTypeId="2" ParentId="13147" CreationDate="2019-07-01T16:28:00.267" Score="2" Body="&lt;p&gt;Your problem is not that the environment is stochastic or dynamic. In fact you are using the terms slightly incorrectly. These terms do not usually refer to the fact that starting state can differ or goal locations can move episode-by-episode. They typically refer to behaviour of state transitions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although in your case you could view the initial state as stochastic, this is not a big deal, and not likely to be the cause of your problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your questions, it seems to me that you are not really running a DQN algorithm yet. It is not 100% clear what your neural network is predicting, but my best guess is that you have 4 outputs to select &quot;best&quot; action and are treating the neural network as a classifier. This training approach seems closest to &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-entropy_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross Entropy Method (CEM)&lt;/a&gt; due to how you are selecting &quot;successful&quot; navigation only.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When training the DQN, how do i apply Q-values?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This question is the most revealing that you are not using DQN. This is too complex to describe in full in an answer, but the basics are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Your neural network (NN) should be estimating Q values. Typically in DQN, you input the state and the NN outputs an array of estimates for Q of each action (although other architectures are possible). This should be a regression problem, so last layer of network needs to be linear.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Current best guess of optimal policy is to run the NN forward and find the maximising action. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In DQN you also have a &quot;behaviour policy&quot; - a simple and popular choice is to use &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy action selection, which just means to take the maximising action (as calculated above), except with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; (some small value, e.g. 0.1) to take a random action.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To figure out your training data to improve the NN, you need Q values to calculate a TD Target. In single-step Q learning that would be &lt;span class=&quot;math-container&quot;&gt;$r + \gamma Q(s',a*)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the immediate reward &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the next state seen, and &lt;span class=&quot;math-container&quot;&gt;$a*$&lt;/span&gt; is the maximising action in that state. You should force &lt;span class=&quot;math-container&quot;&gt;$Q(s', a*) = 0$&lt;/span&gt; (i.e. not use the NN) if &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is a terminal state.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This means you typically need to work with Q values in 2 or 3 places in your inner loop. Your inner loop should look &lt;em&gt;something&lt;/em&gt; like this per time step, given a current state &lt;code&gt;current_state&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Figure out how to act&#xA;current_q_values = NN_predict(current_state)&#xA;current_action = argmax(current_q_values)&#xA;if random() &amp;lt; epsilon:&#xA;  current_action = random_action()&#xA;&#xA;# Take an action&#xA;reward, next_state, done = call_environment(current_state, current_action)&#xA;&#xA;# Remember what happened&#xA;store_in_replay_memory(current_state, current_action, reward, next_state, done)&#xA;&#xA;# Train the NN from memory&#xA;repeat N times: # This can be vectorised for efficiency&#xA;  mem_state, mem_action, mem_reward, mem_next_state, mem_done = sample_replay_memory()&#xA;  mem_q_values = NN_predict(mem_next_state)&#xA;  mem_max_action = argmax(mem_q_values)&#xA;  if done:&#xA;    td_target = mem_reward&#xA;  else&#xA;    td_target = mem_reward + gamma * mem_q_values[mem_max_action]&#xA;  target_q_values = NN_predict(mem_state)&#xA;  target_q_values[mem_action] = td_target&#xA;  NN_train(mem_state, target_q_values)&#xA;&#xA;# Maybe end an episode (this can include generating new map)&#xA;if done:&#xA;  current_state = reset_environment()&#xA;else:&#xA;  current_state = next_state&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can see above, &lt;code&gt;NN_predict&lt;/code&gt; is called three different times to get Q values in slightly different contexts. I have ignored extras such as using a separate target network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I train the network using only a short replay memory of the last move, or the last N moves that led to success. Is this the right way to approach this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is important to include moves that lead to failure so that the NN learns the difference. Typically you will need a replay memory with from a few hundred to a few hundred thousand entries. You would get away with a few hundred perhaps for your simple problem. The idea is to use this training data a bit like a dataset from supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I use Keras, and am simply training the network every time it does something right - and ignoring failed attempts. - But is this anywhere near the right approach?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is not the right approach for DQN, although perhaps could be considered a crude version of CEM.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-07-01T16:28:00.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13152" PostTypeId="1" AcceptedAnswerId="13158" CreationDate="2019-07-01T16:32:46.737" Score="3" ViewCount="320" Body="&lt;p&gt;I need to cluster my points into unknown number of clusters, given the minimal Euclidean distance R between the two clusters. Any two clusters that are closer than this minimal distance should be merged and treated as one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could implement a loop starting from the two clusters and going up until I observe the pair of clusters that are closer to each other than my minimal distance. The upper boundary of the loop is the number of points we need to cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any well known algorithms and approaches estimate the approximate number of centroids from the set of points and required minimal distance between centroids?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using FAISS under Python, but with the right idea I could also implement in C myself.&lt;/p&gt;&#xA;" OwnerUserId="26789" LastEditorUserId="26789" LastEditDate="2019-07-01T16:38:31.547" LastActivityDate="2019-07-01T18:45:40.753" Title="How to compute the number of centroids for K-means clustering algorithm given minimal distance?" Tags="&lt;machine-learning&gt;&lt;k-means&gt;&lt;clustering&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13153" PostTypeId="1" CreationDate="2019-07-01T16:39:47.133" Score="2" ViewCount="16" Body="&lt;p&gt;Is this a scenario that would work well for a ML/Pattern Recognition Model or would it be easier/faster to just filter from a large DB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to create a system that will allow users to identify the appropriate product by specifying certain constraints and preferred features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are millions of possible product configurations. Lets pretend it's boxes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Product Options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Size (From 1mm up to 1m) in 1mm increments&lt;/li&gt;&#xA;&lt;li&gt;Color: choice of 10 colors&lt;/li&gt;&#xA;&lt;li&gt;Material: choice of 3, wood,metal, plastic&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Wood is only available in centimeter units&lt;/li&gt;&#xA;&lt;li&gt;Red is only available in 500 mm and greater&lt;/li&gt;&#xA;&lt;li&gt;Wood is the preferred material&lt;/li&gt;&#xA;&lt;li&gt;Blue is the preferred color&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, we have 30,000 (1000*10*3) possible options.&#xA;Of those, many are not viable such as 533 mm-Red-Wood&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but these configurations similar to the request are possible.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;533 mm-Red-Plastic &lt;/li&gt;&#xA;&lt;li&gt;530 mm-Red-Wood &lt;/li&gt;&#xA;&lt;li&gt;540 mm-Red-Wood&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Notes:&#xA;Our current Rules and code based tool can take anywhere from 0.5 to 2 mins to identify the preferred configuration.&#xA;We can generate a list of all possible configs and whether they are valid or not.&#xA;We estimate 30,000,000 possible configs&#xA;It takes around 0.5 seconds to validate a config so with enough computing power we expect we could do 30M in a few days.&lt;/p&gt;&#xA;" OwnerUserId="26783" LastActivityDate="2019-07-01T16:39:47.133" Title="Product Configuration based on user selection of features and other requirements" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13154" PostTypeId="5" CreationDate="2019-07-01T16:47:09.543" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2019-07-01T16:47:09.543" LastActivityDate="2019-07-01T16:47:09.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13155" PostTypeId="4" CreationDate="2019-07-01T16:47:09.543" Score="0" Body="K-means algorithm groups given set of points (or vectors) into clusters, finding the groups of points that are closer together (by Euclidean distance)." OwnerUserId="26789" LastEditorUserId="26789" LastEditDate="2019-07-01T20:17:03.890" LastActivityDate="2019-07-01T20:17:03.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13156" PostTypeId="1" AcceptedAnswerId="13168" CreationDate="2019-07-01T17:02:00.180" Score="7" ViewCount="591" Body="&lt;p&gt;I was reading the AlphaZero paper &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;noreferrer&quot;&gt;Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm&lt;/a&gt;, and it seems they don't mention Q-Learning anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So does AZ use Q-Learning on the results of self-play or just a Supervised Learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's a Supervised Learning, then why is it said that AZ uses Reinforcement Learning? Is &quot;reinforcement&quot; part primarily a result of using Monte-Carlo Tree Search?&lt;/p&gt;&#xA;" OwnerUserId="26791" LastEditorUserId="2444" LastEditDate="2019-07-01T20:27:57.997" LastActivityDate="2019-07-02T10:16:06.990" Title="Does AlphaZero use Q-Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;supervised-learning&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13157" PostTypeId="2" ParentId="13152" CreationDate="2019-07-01T18:18:55.160" Score="1" Body="&lt;p&gt;If you look at Kaufman &amp;amp; Rousseeuw (1990), &lt;em&gt;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316801&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finding Groups in Data&lt;/a&gt;&lt;/em&gt;, they describe an algorithm to evaluate the quality of clusters in agglomerative clustering. You run the clustering algorithm with a specific value &lt;em&gt;k&lt;/em&gt; for the number of clusters you want, and that routine then gives you a score to reflect the cohesion of the clustering. If you then cluster again with a different value for &lt;em&gt;k&lt;/em&gt;, you will get another score. You repeat this process until you have found a maximum score, and then you have the clustering with the optimum number of clusters.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-07-01T18:18:55.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13158" PostTypeId="2" ParentId="13152" CreationDate="2019-07-01T18:45:40.753" Score="3" Body="&lt;p&gt;Yes, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Silhouette_(clustering)&quot; rel=&quot;nofollow noreferrer&quot;&gt;silhouette&lt;/a&gt; method (which is implemented in &lt;code&gt;sklearn&lt;/code&gt; as &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.metrics.silhouette_score.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;silhouette_score&lt;/code&gt;&lt;/a&gt;) is commonly used to assess the quality of clusters produced by any clustering algorithm (including &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-means or any hierarchical clustering algorithm). Roughly, you can compute the silhouette value for different &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, then you would pick the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; with the highest silhouette value.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-01T18:45:40.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13159" PostTypeId="2" ParentId="13150" CreationDate="2019-07-01T19:11:07.657" Score="2" Body="&lt;p&gt;The paper &lt;a href=&quot;http://www.cs.toronto.edu/~graves/icml_2006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Connectionist Temporal Classification: Labeling Unsegmented Sequence Data with Recurrent Neural Networks&lt;/a&gt; (2006), by Alex Graves et al., mentioned in the linked &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/ctc_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow documentation&lt;/a&gt;, presents and explains the CTC loss and the CTC forward-backward algorithm (in section 4.1). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at this article &lt;a href=&quot;https://distill.pub/2017/ctc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence Modeling With CTC&lt;/a&gt; (2017) by Awni Hannun.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-01T19:27:05.480" LastActivityDate="2019-07-01T19:27:05.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13162" PostTypeId="2" ParentId="11888" CreationDate="2019-07-01T23:53:15.383" Score="1" Body="&lt;p&gt;&lt;strong&gt;Update: I rewrote the first part due to major mistake in the first version&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice: The notation &lt;span class=&quot;math-container&quot;&gt;$P^k$&lt;/span&gt; from Eq.&lt;span class=&quot;math-container&quot;&gt;$(20)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(21)$&lt;/span&gt; in the paper does not mean the kth power of some &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. Instead, &lt;span class=&quot;math-container&quot;&gt;$P^k$&lt;/span&gt; should be thought as the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; step transition probability of a non-homogeneous Markov chain. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;According to the CPO paper, the discounted future state distribution is defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_\pi(s)=(1-\gamma)\sum_{k=0}^\infty \gamma^kProb(s_k=s|\pi,s_0)\mu(s_0)\tag{1}&#xA;$$&lt;/span&gt;&#xA;Consider function form. Let &lt;span class=&quot;math-container&quot;&gt;$Prob_{\pi,k}$&lt;/span&gt; denote the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; step probability transition operator induced by &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;; here &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; can be a hierarchical policy, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; can be larger than &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_\pi=(1-\gamma)\sum_{k=0}^\infty \gamma^kProb_{\pi,k}\mu\tag{2}&#xA;$$&lt;/span&gt;&#xA;Now apply the similar definition as Eq.&lt;span class=&quot;math-container&quot;&gt;$(20)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(21)$&lt;/span&gt; in the paper, let &lt;span class=&quot;math-container&quot;&gt;$P^k_\pi$&lt;/span&gt; denote the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; step transition probability of the non-homogeneous Markov chain induced by the low level policy, with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; smaller or equal to &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;d_\pi&amp;amp;=(1-\gamma)\sum_{m=0}^\infty\gamma^{mc}\sum_{k=0}^{c-1}\gamma^kP^k_\pi(P^c_\pi)^m\mu\\&#xA;&amp;amp;=(1-\gamma)\sum_{k=0}^{c-1}\gamma^kP^k_\pi(\sum_{m=0}^\infty\gamma^{mc}(P^c_\pi)^m)\mu\\&#xA;&amp;amp;=(1-\gamma)A_\pi(I-\gamma^cP^c)^{-1}\mu\tag{3}&#xA;\end{align}&lt;/span&gt;&#xA;which is exactly the form of Eq.&lt;span class=&quot;math-container&quot;&gt;$(22)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(23)$&lt;/span&gt; in the paper, with &lt;span class=&quot;math-container&quot;&gt;$A_\pi$&lt;/span&gt; defined similar as Eq.&lt;span class=&quot;math-container&quot;&gt;$(24)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(25)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The &quot;every-&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-step discounted state frequency&quot; builds on &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt;, but it lumps the &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; steps into one &quot;high level&quot; step where the discount factor is &lt;span class=&quot;math-container&quot;&gt;$\gamma^c$&lt;/span&gt; and the transition operator is &lt;span class=&quot;math-container&quot;&gt;$P_\pi^c$&lt;/span&gt;. Starting with &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt;, replace &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; with 1, we get the &quot;every one step future state distribution&quot;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_\pi=(1-\gamma)(I-\gamma P_\pi)^{-1}\mu\tag{4}&#xA;$$&lt;/span&gt;&#xA;Then replace &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P_\pi$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$(4)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\gamma^c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P_\pi^c$&lt;/span&gt;, we get the &quot;every-&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-step discounted state frequency&quot;, or &quot;every-&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-step future state distribution&quot;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_\pi^c=(1-\gamma^c)(I-\gamma^c P_\pi^c)^{-1}\mu\tag{5}&#xA;$$&lt;/span&gt;&#xA;By the way, I read your blogpost on this paper. It's very helpful for me, thank you!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26796" LastEditorUserId="26796" LastEditDate="2019-07-11T01:59:00.557" LastActivityDate="2019-07-11T01:59:00.557" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13163" PostTypeId="1" CreationDate="2019-07-02T03:55:11.850" Score="2" ViewCount="74" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the authors refer to the application of time-varying graphs as an open problem.　And they say it will be useful for anomaly detection in financial networks, etc. But why is that useful?&lt;/p&gt;&#xA;" OwnerUserId="26502" LastEditorUserId="2444" LastEditDate="2019-07-04T10:17:48.003" LastActivityDate="2019-07-04T10:17:48.003" Title="Why is graph convolution network in time-varying graphs useful for anomaly detection?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;applications&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13164" PostTypeId="2" ParentId="12365" CreationDate="2019-07-02T07:05:47.313" Score="1" Body="&lt;p&gt;Most of the algorithms seem accessing the training set sequentially so the images need not be loaded into memory all at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is technically fully possible to build a workstation with 1 Tb of RAM or more, using a server barebone in a tower form factor (see &lt;a href=&quot;https://www.supermicro.com/en/products/system/4U/7048/SYS-7048A-T.cfm?parts=SHOW&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, for instance, and would support multiple GPUs) but this only makes sense if the image loading is a bottleneck. Current SSDs are rather fast so you need to measure this before spending money on such a beast.&lt;/p&gt;&#xA;" OwnerUserId="26789" LastActivityDate="2019-07-02T07:05:47.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13168" PostTypeId="2" ParentId="13156" CreationDate="2019-07-02T10:16:06.990" Score="6" Body="&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; you mentioned in the comments that you are reading the old, pre-print version of the paper describing AlphaZero on arXiv. My answer will be for the &quot;official&quot;, peer-reviewed, more recent publication in Science (which nbro linked to in his comment). I'm not only focusing on the official version of the paper just because it is official, but also because I found it to be much better / more complete, and I would recommend reading it instead of the preprint if you are able to (I understand it's behind a paywall so it may be difficult for some to get access). The answer to this specific question would probably be identical for the arXiv version anyway.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;No, AlphaZero does not use &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neural network in AlphaZero is trained to minimise the following loss function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(z - \nu)^2 - \pi^{\top} \log \mathbf{p} + c \| \theta \|^2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$z \in \{-1, 0, +1\}$&lt;/span&gt; is the real outcome observed in a game of self-play.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\nu$&lt;/span&gt; is a predicted outcome / value.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a distribution over actions derived from the visit counts of MCTS.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}$&lt;/span&gt; is the distribution over actions output by the network / the policy.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c \| \theta \|^2$&lt;/span&gt; is a regularisation term, not really interesting for this question/answer.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this loss function, the term &lt;span class=&quot;math-container&quot;&gt;$(z - \nu)^2$&lt;/span&gt; is exactly what we would have if we were performing plain Monte-Carlo updates, based on Monte-Carlo returns, in a traditional Reinforcemet Learning setting (with function approximation). Note that there is no &quot;bootstrapping&quot;, no combination of a single-step observed reward plus predicted future rewards, as we would have in &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning. We play out all the way until the end of a game (an &quot;episode&quot;), and use the final outcome &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; for our updates to our value function. That's what makes this Monte-Carlo learning, rather than &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the &lt;span class=&quot;math-container&quot;&gt;$- \pi^{\top} \log \mathbf{p}$&lt;/span&gt; term, this is definitely not &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning (we're directly learning a policy in this part, not a value function).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If it's a Supervised Learning, then why is it said that AZ uses Reinforcement Learning? Is &quot;reinforcement&quot; part primarily a result of using Monte-Carlo Tree Search?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The policy learning looks very similar to Supervised Learning; indeed it's the cross-entropy loss which is frequently used in Supervised Learning (classification) settings. &lt;strong&gt;I'd still argue it is Reinforcement Learning, because the update targets are completely determined by self-play, by the agent's own experience. There are no training targets / labels that are provided externally&lt;/strong&gt; (i.e. no learning from a database of human expert games, as was done in 2016 in AlphaGo).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-02T10:16:06.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13169" PostTypeId="2" ParentId="6748" CreationDate="2019-07-02T12:42:19.403" Score="0" Body="&lt;p&gt;I would recommend not to use TF at all, but use Pytorch instead. Make your test game in python. Even for simple game training can take a lot of time. If you are not afraid of some math and complexity you can start from &lt;a href=&quot;https://github.com/suragnair/alpha-zero-general&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero General&lt;/a&gt; - it have structure which allow to plug any simple two-player deterministic game into it. That project is derivative of famous AlphaZero and on this moment is the most powerful RL method for those type of game. Be warned that even easy 6x6 game of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reversi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Othello&lt;/a&gt; take tree days to train on single GPU with this method.&#xA;After you get some experience with python prototype you can switch to C++ API.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to run any framework on console though would probably be hellish and likely impossible. Most of frameworks use NVIDIA GPU tensor library &quot;cudnn&quot;. I have no idea if you can access it from console. You can run framework on CPU and probably port it too, not using cudnn, but forward run (actual playing, not training) would be ~50 time more slow. If GPU run would be less 1 second then CPU AI would still be playable for turn-based game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can run any framework on linux and on windows and Mac (probably with some problems, but solvable). So for PC-games it's doable, assuming you manage to train python prototype. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-07-02T12:42:19.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13172" PostTypeId="1" CreationDate="2019-07-02T23:02:12.503" Score="6" ViewCount="138" Body="&lt;p&gt;The basic seq-2-seq model consists of 2 parts: a recurrent encoder that compresses a sequence to a vector and decoder that unrolls the vector into the output sequence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IkEwP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IkEwP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the output, &lt;code&gt;w&lt;/code&gt;, &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt;, &lt;code&gt;z&lt;/code&gt; of the decoder used as its input? Shouldn't the hidden state of the RNN from the previous timestamps be enough?&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="2444" LastEditDate="2019-07-02T23:19:15.160" LastActivityDate="2019-07-03T16:07:17.423" Title="In sequence-to-sequence, why is the output of the decoder used as its input?" Tags="&lt;recurrent-neural-networks&gt;&lt;sequence-modelling&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13173" PostTypeId="2" ParentId="12266" CreationDate="2019-07-03T00:05:55.163" Score="5" Body="&lt;p&gt;The previous answer has given a good insight into the difference between two areas. I would like to give more examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Semi-Supervised Learning&lt;/strong&gt; work with improving the data set by adding up new examples. There are iterative systems where we train a model on a given dataset and improve the model further after deploying it on the real world by adding interactions of the real world and their outcomes to further train the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Self-Supervised Learning&lt;/strong&gt; is becoming a very hot topic these days. It has the ability to understand the underline properties of a given dataset with some kind of a supervisory signal (Not exactly a label). &lt;strong&gt;self-Attention&lt;/strong&gt; introduced in Transformers is a modern day popular self-supervised learning. Also, check this tweet from &lt;a href=&quot;https://twitter.com/ylecun/status/1123235709802905600?lang=en&quot; rel=&quot;noreferrer&quot;&gt;Yann Lecun tweet&lt;/a&gt;   &lt;/p&gt;&#xA;" OwnerUserId="14909" LastActivityDate="2019-07-03T00:05:55.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13174" PostTypeId="2" ParentId="13049" CreationDate="2019-07-03T01:28:36.793" Score="0" Body="&lt;p&gt;Let's consider both pieces of that question as we consider the overall one.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a shortage of machine learning professionals?&lt;/li&gt;&#xA;&lt;li&gt;Is there a shortage of artificial intelligence professionals?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Supply and demand are the two words associated with price in the often oversimplified model of price in a free market. We can discuss them, but they are difficult to quantify and their is no particular function that predicts the cost or pay in relation to a job title purely on the basis of the number of job requisitions on the web and the number of resumes that match the job description. It's not that simple, if the question author is looking for financial security.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there is a much larger frequency of job requisitions with job titles such as &quot;Machine Learning Engineer&quot; or &quot;Intelligence Designer&quot;, it is often difficult to state outright that there is a demand. It depends on the perspectives of the requisitioning manager and the professional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many companies are operating already without such professionals and don't really know what they want to do with ML or AI. Calling that &quot;demand&quot; is stretching the meaning of the term somewhat. Others have such talent in house and are really looking for a programmer that can follow directions and still others have some objective they believe can be done in a year but may take fifty. For an experienced engineer or contractor, those are not coveted positions. For those that want to get a paragraph onto a resume with some ML or AI key words in it without lying, they'll do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the supply side, the number of people who have mastered one or more of the ML libraries are growing fast, mostly due to the improved maturity of the libraries, their examples, and the data sets to use them. That does not mean that everyone on the supply side can make code work in robots and data centers to produce competitive products, which is where the supply side is weak for the ML question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial intelligence is such a wide field that includes ML only as a small sub-field, is a much more difficult role to define and fill. It may have the larger future as a career now, partly because of the ML craze, but it is also a less frequent job requisition category and there are fewer resumes that legitimately demonstrate AI research and development background. &lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-07-03T01:36:10.357" LastActivityDate="2019-07-03T01:36:10.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13175" PostTypeId="2" ParentId="13125" CreationDate="2019-07-03T02:44:33.847" Score="0" Body="&lt;p&gt;Before answering how to stop the evaluation phase and begin exploitation of those results, one must first answer when to stop it whereby the balance the project stakeholder wants between quality and cost is found. You won't always find that in books discussing pure research, so your question is an excellent one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm the authors are discussing on that page (101) is based on the policy improvement theorem on page 78, and the appearance of the endless loop in the algorithm in the pseudo-code line &quot;Repeat forever (for each episode)&quot; is obviously worse than useless in a data center if the loop is not terminated, unless it is multi-agent, exploiting multiple threads, processes, virtual hosts, hardware accelerators, cores, or hosts, and the improvements are accessed for exploitation independently or symbiotically using some scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a deployed robot, an endless loop often has a legitimate a use case. &quot;Repeat until shutdown,&quot; might be appropriate in a production algorithm or hardware embodiment if the robot's goal is, &quot;Keep the living room clean.&quot; One must always try to place this theory in context when taking pure research and considering the applied research that may stem from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In real product and service development environments, how the balance is struck between quality of action and cost of determining it depends upon the problem size, expectations of the user or business, and the architecture of the computational resources you have. Consider some of these factors in more detail.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maximum number of rounds of evaluation-exploitation cycles&lt;/li&gt;&#xA;&lt;li&gt;Requirements for precision in terms of optimality&lt;/li&gt;&#xA;&lt;li&gt;Requirements for reliability in terms of completion&lt;/li&gt;&#xA;&lt;li&gt;Distribution of the number branches from nodes&lt;/li&gt;&#xA;&lt;li&gt;Distribution of lengths of possible action traversal sequences to the goal&lt;/li&gt;&#xA;&lt;li&gt;Average cost (in time and energy) of each evaluation&lt;/li&gt;&#xA;&lt;li&gt;Average cost (in time and energy) of each exploitation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In a single thread, single core, von Neumann architecture, as is sometimes the case in an embedded environment, evaluation and exploitation are time sliced. In such a case, evaluation should stop and exploitation should begin when the probability that further evaluation will produce an improved result drops below the cost of further evaluation, based on some estimation of return and cost. This is a function of the above factors, although not a linear one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have considered training an LSTM network to determine the function in the epoch domain (roughly related to the time domain), although it is low on our priority list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an embedded process, a function that approximates return on further evaluation cost can be constructed, based on statistics gathered up to that point in current learning or over a longer period of operations. The function should be fast and inexpensive. In each cycle within the evaluation phase, the function can be evaluated and compared against a configurable probability threshold. Its configuration value can be an educated guess based on the perception of the value of further path exploration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a simulation environment, when more computing resources for parallel processes, exploiting OS or hardware facilities for that, the time slicing is either opaque or nonexistent respectively. In those cases, continuous improvement may be unbounded because the state-action graph is not finite.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-07-03T02:44:33.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13177" PostTypeId="2" ParentId="13031" CreationDate="2019-07-03T03:23:12.037" Score="0" Body="&lt;p&gt;You will need one of two things or both.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A theoretical model of grade probability distribution already verified against data that contains the features listed&lt;/li&gt;&#xA;&lt;li&gt;A data set mapping the features listed to grades (labels) for training an artificial network&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The features can be specified more formally&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chosen topic --- from a finite list of topics, encoded by integer (numbered strings)&lt;/li&gt;&#xA;&lt;li&gt;Performance --- low, medium, or high --- it is not clear from where this judgment comes, but that would need to be clarified and also included in either the verified theoretical model or among the features of the training data&lt;/li&gt;&#xA;&lt;li&gt;Does a student work? --- hours per week --- note that the flexibility of the schedule is not as critical as the hours of the 168 hours of each week not available for sleep, self-care, class time, or study&lt;/li&gt;&#xA;&lt;li&gt;Have a student ever gotten through a add course? --- binary --- this question is unclear in its current phrasing&lt;/li&gt;&#xA;&lt;li&gt;Grade --- converted to numeric value from 0.0 to 4.0, which can be represented as an integer from 00 to 40&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The difficult part will be finding either the verified theoretical model or the data set for training or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a verified theoretical model is found, apply a gradient descent implementation to tune that model, which is a statistics and conversion problem only partly related to AI and machine learning. Otherwise using a model free artificial network will likely allow you to reach your objective. A simple feed forward network with gradient descent and back-propagation and three layers should suffice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many examples in Python, Java, and other languages available online, usually in conjunction with a library that the examples leverage for computation and instructions on how to install the software. There are other questions and answers in this forum that explain what books to buy to get started. In fact, there is a tag specifically for that purpose: &lt;a href=&quot;https://ai.stackexchange.com/questions/tagged/getting-started&quot;&gt;https://ai.stackexchange.com/questions/tagged/getting-started&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-07-03T03:23:12.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13178" PostTypeId="2" ParentId="12970" CreationDate="2019-07-03T03:52:06.547" Score="1" Body="&lt;p&gt;There are 3,810 articles easily found in an academic article search. These are three examples.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.158.5544&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Neuroevolution for reinforcement learning using evolution strategies&lt;/em&gt;&lt;/a&gt; — C Igel — The 2003 Congress on Evolutionary Computation, 2003, ieeexplore.ieee.org — &quot;We apply the CMA-ES, an evolution strategy which efficiently adapts the covariance matrix of the mutation distribution, to the optimization of the weights of neural networks for solving reinforcement learning problems. It turns out that the topology of the networks considerably ...&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.06567&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Neuroevolution strategies for episodic reinforcement learning&lt;/em&gt;&lt;/a&gt; — V Heidrich-Meisner, C Igel — Journal of Algorithms, 2009, Elsevier — &quot;Because of their convincing performance, there is a growing interest in using evolutionary algorithms for reinforcement learning. We propose learning of neural network policies by the covariance matrix adaptation evolution strategy (CMA-ES), a randomized variable-metric ...&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://infoscience.epfl.ch/record/112676/files/FloreanoDuerrMattiussi2008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning&lt;/em&gt;&lt;/a&gt; — FP Such, V Madhavan, E Conti, J Lehman — 2017, arxiv.org — &quot;Deep artificial neural networks (DNNs) are typically trained via gradient-based learning algorithms, namely backpropagation. Evolution strategies (ES) can rival backprop-based algorithms such as Q-learning and policy gradients on challenging deep reinforcement ...&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To clarify the strategy proposed, we can rewrite the approach as a set of design features. Let's not assume that topology formation is based on weights, which diminishes the concept of morphology and topology. If we were to reduce the model to a traditional artificial network of orthogonal topology, it would then not be neuroevolution; it would then be basic machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Search for the best network topology through neuroevolution&lt;/li&gt;&#xA;&lt;li&gt;Train the best candidate selected above through Q-learning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The second item doesn't seem address the relationship between the inputs, outputs, and objectives of neuroevolution designs and those of Q-learning and other reinforcement learning strategies. Q-learning algorithms are not designed to run on feed forward networks in general and certainly not easily mapped to the topologies that may form during neuroevolution. There are probably billions, if not an infinite number, of ways to combine the two strategies, but simplistic concatenation of the two processes is not possible without further research and consideration of how they will interrelate collaboratively toward program goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be useful to search for articles, study, and then formulate your research trajectory. It is recommendable to start with learning about neuroevolution and reinforcement independently, and then start reading articles like the above three. Pour the foundation, let it dry, and then frame the house.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-07-03T03:52:06.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13179" PostTypeId="1" AcceptedAnswerId="13181" CreationDate="2019-07-03T06:05:06.700" Score="2" ViewCount="150" Body="&lt;p&gt;I want to study NN for time-varying directed graphs. However, as this field has developed relatively recently, it is difficult to find new ways. So the question is, is there any NN that can handle such data?&lt;/p&gt;&#xA;" OwnerUserId="26502" LastEditorUserId="26502" LastEditDate="2019-07-03T06:35:27.553" LastActivityDate="2019-07-03T07:32:06.637" Title="Is there a neural network method for time-varying directed graphs?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;graphs&gt;&lt;geometric-deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13180" PostTypeId="1" CreationDate="2019-07-03T07:23:58.870" Score="1" ViewCount="11" Body="&lt;p&gt;I have the following problem: I am doing some research on the accuracy of recommender algorithms that are mostly used nowadays. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, one way to measure their performance is by checking how well they predict a certain value under different sizes of a given dataset, meaning, sparsity in a ratings matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find a way to calculate the root mean square error(or mae), some metric, versus the sparsity in the dataset. As an example, let’s have a look at the picture below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mN1mF.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mN1mF.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that it says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;“RMSE as a function of sparsity. 5000 ratings were removed from the training set(initially containing 80000 ratings) in every iteration. “&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I’m using Python and the Movielens dataset. Do you know how can I achieve this in the mentioned language? Is there any tool to do that? &lt;/p&gt;&#xA;" OwnerUserId="25405" LastActivityDate="2019-07-03T07:23:58.870" Title="Reducing the Number of Training Samples for collaborative filtering in recommender systems" Tags="&lt;machine-learning&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13181" PostTypeId="2" ParentId="13179" CreationDate="2019-07-03T07:32:06.637" Score="3" Body="&lt;p&gt;I'm seeing recent trend of combining RNN/CNN with GNN(graph neural networks) so that both time dependency and topology are captured. I would suggest you to start by looking at DCRNN (Yaguang Li et al.), it's a strong baseline that everyone uses nowadays. Other good resources: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Graph wavenet for deep spatial-temporal graph modeling (Zonghan Wu et al.)&lt;/li&gt;&#xA;&lt;li&gt;Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting (Bing Yu et al.)&lt;/li&gt;&#xA;&lt;li&gt;STANN: A Spatio–Temporal Attentive Neural Network for Traffic Prediction (Zhixiang He et al.) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-07-03T07:32:06.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13182" PostTypeId="2" ParentId="13172" CreationDate="2019-07-03T08:05:57.273" Score="0" Body="&lt;p&gt;In the original seq2seq paper, they used two RNN, one for encoding and one for decoding. In the encoder they need to unroll the inputs to capture the time dependency. Now if we want to pass the hidden state from the encoder to the decoder that means that the decoder hidden state shape needs to match the encoder (aka same architecture). Since the architecture is the same, we can not directly generate a sequence of n samples within the decoder without unrolling it and you can not unroll it without an input.  &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-07-03T08:05:57.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13183" PostTypeId="2" ParentId="13149" CreationDate="2019-07-03T08:18:28.300" Score="2" Body="&lt;p&gt;You can take a look at traffic data for example if you follow &#xA;&lt;a href=&quot;https://github.com/liyaguang/DCRNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;link1&lt;/a&gt;, &lt;a href=&quot;http://www.d.umn.edu/~tkwon/TMCdata/TMCarchive.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link2&lt;/a&gt; you can find 3 publicly available traffic datasets which are already preprocessed. You cold also look at air quality datasets offered by the government &lt;a href=&quot;https://aqs.epa.gov/aqsweb/airdata/download_files.html#Meta&quot; rel=&quot;nofollow noreferrer&quot;&gt;link3&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-07-03T08:18:28.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13185" PostTypeId="2" ParentId="13172" CreationDate="2019-07-03T09:05:35.023" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Shouldn't the hidden state of the RNN from the previous timestamps be enough?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is theoretically enough to generate a sequence. However, allowing an input offers a couple of convenient extras:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Training data for output sequences is used twice - once as input (as previous sequence data), once as target (to establish loss metric). This may help training process as the decoder trains both as a decoder to the new sequence type and as a predictive model over the output sequence semi-independently - i.e. weights from input to RNN layer are affected by error gradients separately to weights between previous hidden state and next state, although the two sets of weights together influence output and next state, so are not fully independent over a sequence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;By allowing input of sequence so far generated, the decoder can work as a generator, where the next item in the sequence does not need to be the maximum probability item, but can be sampled or have rules applied. This allows for approaches such as &lt;a href=&quot;https://medium.com/@dhartidhami/beam-search-in-seq2seq-model-7606d55b21a5&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEAM search&lt;/a&gt;, commonly used in machine translation, which maintains several potential outputs, selecting best one at the end.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have not done the experiment, but I suspect the first item results in faster and better generalisation. The second one is very convenient for natural language generation and similar problems.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-03T16:07:17.423" LastActivityDate="2019-07-03T16:07:17.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13186" PostTypeId="1" CreationDate="2019-07-03T11:09:34.910" Score="1" ViewCount="75" Body="&lt;p&gt;I created an lstm model which predicts multioutput sequeances. It takes variable length sequences as input. These sequences are padded with zero to obtain equal length. Note that the time series are not equally spaced but time stamp is added as predictor. The descrete time series predictors are normalized with mean and standard deviation and run through PCA, the categorical feature is one hot encoded and the ordinal features are integer encoded. So the feature set is a combination of dynamic and static features. The targets are also scaled between [-1,1]. The input layer goes through a masked layer and then to 2 stacked lstm layers and dense layer to predict the targets (see code below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training actually starts well but then the performance starts to saturate. It seems the network also focus more on the 3rd output rather than the first two. This is seen in the validation curve of the third output that follows the training curve perfectly. For the first 2 outputs, the network has a hard time predicting some peak values. I have been tuning the hyper parameters but validation error does not go below a certain value. The longer I train the more the validation curve and training curve separate from each other and overfitting occurs on the first 2 outputs. I tried all the standard initializations and he_initialization seems to work the best. When more data is added there is a slight improvement in validation error but not significant. When adding dropout the validation error is lower than the training error due to noise introduced by dropout in feed forward but there is no significant improvement. Since neural networks tend to converge close to where they are initialized, I was thinking my initialization is not good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wandering if anyone had any suggestions on how to improve the error of this model. I think I will be happy if I can get the validation error somewhere around 0.01.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZKkgk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZKkgk.png&quot; alt=&quot;out1&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/pySzM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pySzM.png&quot; alt=&quot;out2&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8KUhy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8KUhy.png&quot; alt=&quot;out3&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/M4qJl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M4qJl.png&quot; alt=&quot;loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def masked_mse(y_true, y_pred):&#xA;    mask = keras.backend.all(keras.backend.not_equal(y_true, 0.), axis=-1, keepdims=True)&#xA;&#xA;    y_true_ = tf.boolean_mask(y_true, mask)&#xA;    y_pred_ = tf.boolean_mask(y_pred, mask)&#xA;&#xA;    return keras.backend.mean(keras.backend.square(y_pred_ - y_true_))&#xA;&#xA;def rmse(y_true, y_pred):&#xA;    # find timesteps where mask values is not 0.0&#xA;    mask = keras.backend.all(keras.backend.not_equal(y_true, 0.), axis=-1, keepdims=True)&#xA;&#xA;    y_true_ = tf.boolean_mask(y_true, mask)&#xA;    y_pred_ = tf.boolean_mask(y_pred, mask)&#xA;&#xA;    return keras.backend.sqrt(keras.backend.mean(keras.backend.square(y_pred_ - y_true_)))&#xA;&#xA;hl1 = 125&#xA;hl2 = 125&#xA;window_len = 30&#xA;n_features = 50&#xA;batch_size = 128&#xA;&#xA;optimizer = keras.optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0., amsgrad=False)&#xA;dropout = 0.&#xA;input_ = keras.layers.Input(&#xA;        shape=(window_len, n_features)&#xA;    )&#xA;&#xA;# masking is to make sure the model doesn't fit the zero paddings&#xA;masking = keras.layers.Masking(mask_value=0.0)(input_)&#xA;&#xA;# hidden layer 1 with he_normal initializer. &#xA;&#xA;lstm_h1 = keras.layers.LSTM(hl1, dropout=dropout, kernel_initializer='he_normal',&#xA;                        return_sequences=True)(masking)&#xA;&#xA;# hidden layer 2&#xA;lstm_h2 = keras.layers.LSTM(hl2, dropout=dropout, kernel_initializer='he_normal',&#xA;                        return_sequences=True)(lstm_h1)&#xA;&#xA; # dense output layer of single output&#xA; out1 = keras.layers.Dense(1, activation='linear',name='out1')(lstm_h2)&#xA;&#xA; out2 = keras.layers.Dense(1, activation='linear', name='out2')(lstm_h2)&#xA;&#xA; out3 = keras.layers.Dense(1, activation='linear', name='out3')(lstm_h2)&#xA;&#xA; model = keras.models.Model(inputs=input_, outputs=[out1, out2, out3])&#xA;&#xA;&#xA; pi = [rmse]&#xA;&#xA; n_gpus = len(get_available_gpus())&#xA;&#xA; if n_gpus &amp;gt; 1:&#xA;    print(&quot;Using Multiple GPU's ...&quot;)&#xA;    parallel_model = multi_gpu_model(model, gpus=n_gpus)&#xA;&#xA;else:&#xA;    print(&quot;Using Single GPU ...&quot;)&#xA;    parallel_model = model&#xA;&#xA;&#xA; parallel_model.compile(loss=masked_mse, optimizer=optimizer, metrics=pi)&#xA; parallel_model.summary()&#xA;&#xA; checkpoint = keras.callbacks.ModelCheckpoint(&#xA;             file_name+&quot;.hdf5&quot;, monitor='val_loss', verbose=1, &#xA;             save_best_only=True, mode='min', period=10,&#xA;             save_weights_only=True)&#xA;&#xA; save_history = keras.callbacks.CSVLogger(file_name+&quot;.csv&quot;, append=True)&#xA;&#xA; callbacks_list = [checkpoint, save_history]&#xA;&#xA; y_train_reshaped = list(reshape_test(window_len, y_train))&#xA; parallel_model.fit(&#xA;            x_train,&#xA;&#xA;            {&#xA;                 'out1': y_train_reshaped[0],&#xA;                 'out2': y_train_reshaped[1],&#xA;                 'out3': y_train_reshaped[2],&#xA;            },&#xA;&#xA;            epochs=epochs,&#xA;            batch_size=batch_size,&#xA;            verbose=0,&#xA;            shuffle='batch',&#xA;            validation_data=(x_test, list(reshape_test(window_len,y_test))),&#xA;            callbacks=callbacks_list,&#xA;&#xA;        )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26835" LastEditorUserId="26835" LastEditDate="2019-07-04T10:43:33.443" LastActivityDate="2019-07-04T10:43:33.443" Title="Why doesnt my lstm model for time series prediction improve after certain level of performance?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13187" PostTypeId="2" ParentId="13172" CreationDate="2019-07-03T11:51:06.437" Score="3" Body="&lt;p&gt;In seq2seq they model the joint distribution of whatever char/word sequence by decomposing it into time-forward conditionals:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\begin{align*}&#xA;p(w_1,w_2,...,w_n) =&amp;amp; \ p(w_1)*p(w_2|w_1) * \ ... \ * p(w_n|w_1,...,w_{n-1}) \\&#xA; =&amp;amp; \ p(w_1)*\prod_{i=2}^{n}p(w_i|w_{&amp;lt;i})&#xA;\end{align*}&#xA;$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be sampled by samping each of the conditional is ascending order. So thats exactly what theyre trying to imitate. You want the second output dependant on the &lt;strong&gt;sampled&lt;/strong&gt; first output, not its distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why the hidden state is &lt;strong&gt;NOT&lt;/strong&gt; good for modeling this setup because it is a latent representation of the distribution, not a sample of the distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: In training they use ground truth as input by default because its working under the assumption the model shouldve predicted the correct word and if it didnt the gradient of the word/char level loss will reflect that (this is called teacher forcing and has a multitude of pitfalls)&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-03T11:51:06.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13189" PostTypeId="1" CreationDate="2019-07-03T15:41:03.197" Score="2" ViewCount="336" Body="&lt;p&gt;Let's say I have an adjustable loaded die, and I want to train a neural network to give me the probability of each face, depending on the settings of the loaded die.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't mesure its performance on individual die roll, since it does not give me a probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could batch a lot of roll to calculate a probability and use this batch as an individual test case, but my problem does not allow this (let's say the settings are complex and randomized between each roll).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2 ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;train it as a classification problem which output confidence, and hope that the confidence will reflect the actual probability. Sometimes the network would output the correct probability and fail the test, but on average it would tend to the correct probability. However it may require a lot of training and data.&lt;/li&gt;&#xA;&lt;li&gt;batch random rolls together and compare the mean/median/standard deviation of the measured result vs the predictions. It could work but I don't know the good batch size.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="26839" LastActivityDate="2020-11-16T05:21:27.817" Title="How can I train a neural network to give probability of a random event?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13190" PostTypeId="2" ParentId="13189" CreationDate="2019-07-03T17:48:38.393" Score="0" Body="&lt;p&gt;Neural network isnt what you want here. You have a limited number of events and draws from some unknown distribution that you want to recover.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case, just use the empirical probabilities &lt;span class=&quot;math-container&quot;&gt;$p(event_i) = \frac{\# event_i}{total\ events}$&lt;/span&gt; which given enough draws will converge to the true probability. &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-03T17:48:38.393" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13191" PostTypeId="1" CreationDate="2019-07-03T20:08:55.730" Score="2" ViewCount="43" Body="&lt;p&gt;Why do neural networks have bias units? Why is it sometimes okay to opt them out?&lt;/p&gt;&#xA;" OwnerUserId="26843" LastEditorUserId="2444" LastEditDate="2019-07-04T10:01:53.303" LastActivityDate="2019-07-04T10:01:53.303" Title="Why do neural networks have bias units?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;theory&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13192" PostTypeId="1" CreationDate="2019-07-03T20:30:42.363" Score="0" ViewCount="235" Body="&lt;p&gt;I want to develop an AI for continuous space. I reached to DDPG algorithm that takes actions deterministically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If DDPG takes actions deterministically, should the environment also be deterministic? I want non-deterministic, continuous real-world environments. Is DDPG the algorithm I am looking for? Is there any other algorithm for my need?&lt;/p&gt;&#xA;" OwnerUserId="19493" LastEditorUserId="2444" LastEditDate="2019-07-04T10:06:08.217" LastActivityDate="2019-07-04T15:37:26.920" Title="Is DDPG just for deterministic environments?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13193" PostTypeId="1" CreationDate="2019-07-03T21:44:12.283" Score="1" ViewCount="16" Body="&lt;p&gt;I'm working on a deep learning project and have encountered a problem. The images that I'm using are very large and extremely detailed. They also contain a huge amount of necessary visual information, so it's hard to downgrade the resolution. I've gotten around this by slicing my images into 'tiles,' with resolution 512 x 512. There are several thousand tiles for each image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the problem—the annotations are binary and the images are heterogenous. Thus, an annotation can be applied to a tile of the image that has no impact on the actual classification. How can I lessen the impact of tiles that are 'improperly' labeled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thought is to cluster the tiles with something like a t-SNE plot and compare the ratio of the binary annotations for different regions (or 'classes'). I could then assign weights to images based on where it's located and then use that as an extra layer in my training. Very new to all of this, so wouldn't be surprised if that's an awful idea! Just thought I'd take a stab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For background, I'm using transfer learning on Inception v3.&lt;/p&gt;&#xA;" OwnerUserId="26844" LastActivityDate="2019-07-03T21:44:12.283" Title="Binary annotations on large, heterogenous images" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;training&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13194" PostTypeId="1" CreationDate="2019-07-03T22:40:58.993" Score="4" ViewCount="827" Body="&lt;p&gt;I would like to know if it was possible to train a neural network on daily new data. Let me explain this more in detail. Let's say you have daily data from 2010 to 2019. You train your NN on all of it, but, from now on, every day in 2019 you get new data. Is it possible to &quot;append&quot; the training of the NN or do we need to retrain an entire NN with the data from &lt;span class=&quot;math-container&quot;&gt;$2010$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$2019+n$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; the day for every new day?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if it is relevant but my work is on binary classification. &lt;/p&gt;&#xA;" OwnerUserId="26522" LastEditorUserId="2444" LastEditDate="2019-08-03T12:52:45.060" LastActivityDate="2019-12-31T13:03:30.490" Title="Can I train a neural network incrementally given new daily data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;&lt;incremental-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13196" PostTypeId="2" ParentId="13192" CreationDate="2019-07-04T03:40:43.130" Score="2" Body="&lt;p&gt;I am not an expert in this area. But I believe that the word &quot;Deterministic&quot; is for &quot;Policy&quot; in the &quot;Deterministic Policy&quot; Gradient. It does not mean  deterministic environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic policy: Probabilistic(random) action choice for a given state.&lt;br&gt;&#xA;Deterministic policy: one action is chosen for a given state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deterministic Policy Gradient algorithm still can handle a stochastic (and continuous, of cause) environment, but the policy will be deterministic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;in DDPG, the Actor directly maps states to actions (the output of the network directly the output) instead of outputting the probability distribution across a discrete action space&quot;  -&lt;a href=&quot;https://towardsdatascience.com/deep-deterministic-policy-gradients-explained-2d94655a9b7b&quot; rel=&quot;nofollow noreferrer&quot;&gt;towards Data Science&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deterministic Policy Gradient Algorithms by Silver &lt;em&gt;et al&lt;/em&gt;. &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Publications_files/deterministic-policy-gradients.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="23788" LastEditorUserId="23788" LastEditDate="2019-07-04T15:37:26.920" LastActivityDate="2019-07-04T15:37:26.920" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13198" PostTypeId="1" CreationDate="2019-07-04T07:37:07.710" Score="1" ViewCount="30" Body="&lt;p&gt;As I wrote in the title, what are the advantages of time-varying graph CNNs compared to fixed graph? For example, in CORA, which is a graph of citation relations of papers frequently used in graph CNN, what examples are there?&lt;/p&gt;&#xA;" OwnerUserId="26502" LastActivityDate="2019-07-04T07:37:07.710" Title="What are the advantages of time-varying graph CNNs compared to fixed graph?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;graphs&gt;&lt;geometric-deep-learning&gt;&lt;graph-theory&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13199" PostTypeId="1" CreationDate="2019-07-04T08:05:38.950" Score="1" ViewCount="761" Body="&lt;p&gt;i'm trying to implement this &lt;a href=&quot;http://www.cs.toronto.edu/~urtasun/publications/luo_etal_cvpr18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and I'm stuck for quite some time now. Here is the issue: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a 3D tensor and has  &lt;code&gt;(180,200,20)&lt;/code&gt; as dimension and I'm trying to append 5 of them as the paper states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Now that each frame is represented as a 3D tensor, we can append multiple frames’ along a new temporal dimension to create a 4D tensor&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;what I did is I applied the tensorflow command &lt;code&gt;tf.stack()&lt;/code&gt; and now so far so good, I have my input as a 4D tensor and has &lt;code&gt;(5,180,200,20)&lt;/code&gt; as stated in the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thus our input is a 4 dimensional tensor consisting of time, height, X and Y&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now what I'm trying to do is to apply a 1D convolution on this 4D tensor as the paper mentions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;given a 4D input tensor, we first use a 1D convolution with&#xA;  kernel size n on temporal dimension to reduce the temporal dimension from n to 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is this case n = 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here where I got stuck, I created the kernel as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;kernel = tf.Variable(tf.truncated_normal([5,16,16], dtype = tf.float64, stddev = 1e-1, name = 'weights'))&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and tried to apply a 1D convolution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conv = tf.nn.conv1d(myInput4D, kernel, 1 , padding = 'SAME'&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I get this error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Shape must be rank 4 but is rank 5 for 'conv1d_42/Conv2D' (op: 'Conv2D') with input shapes: [5,180,1,200,20], [1,5,16,16]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how &lt;code&gt;1&lt;/code&gt; is added to the dimensions at the &lt;code&gt;index = 2&lt;/code&gt; and &lt;code&gt;index = 0&lt;/code&gt; in the first and second tensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conv = tf.layers.conv1d(myInput4D, filters = 16, kernel_size = 5, strides = 1, padding = 'same)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And get the following error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Input 0 of layer conv1d_4 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [5, 180, 200, 20]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is it possible to apply a 1D convolution on a 4D input and if yes can anyone suggests a way to do so? Because in the Tenssorflow documentations it says the input must be 3D&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;tf.nn.conv1d(&#xA;      value=None,&#xA;      filters=None,&#xA;      stride=None,&#xA;      padding=None,&#xA;      use_cudnn_on_gpu=None,&#xA;      data_format=None,&#xA;      name=None,&#xA;      input=None,&#xA;      dilations=None&#xA;  ) &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;value: A 3D Tensor. Must be of type float16, float32, or float64.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="26777" LastEditorUserId="26777" LastEditDate="2019-07-04T08:46:09.930" LastActivityDate="2021-01-04T19:24:28.077" Title="Applying a 1D convolution for 4D input" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;computer-vision&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13200" PostTypeId="2" ParentId="13189" CreationDate="2019-07-04T08:12:46.533" Score="0" Body="&lt;p&gt;This approach:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;train it as a classification problem which output confidence, and hope that the confidence will reflect the actual probability. Sometimes the network would output the correct probability and fail the test, but on average it would tend to the correct probability.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;will work with some limitations. If you use a classifier with softmax activation and multiclass log loss&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}(\mathbf{\hat{y}},\mathbf{y}) = -\mathbf{y} \cdot  \text{log}(\mathbf{\hat{y}})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\hat{y}}$&lt;/span&gt; is the network output as a vector, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; is the actual output from an individual sample. Your input should be the settings of the die. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimising this loss will converge on approximated probabilities for each discrete output. You can demonstrate this with some simple examples - for instance if you train a network with a single input - a one-hot-encoded die type from the classic D&amp;amp;D dice sets, plus deliberately chosen examples of different results in the right frequencies, you will end up with a classifier that predicts roughly &lt;span class=&quot;math-container&quot;&gt;$p=0.25$&lt;/span&gt; for results of 1,2,3,4 for a d4 and &lt;span class=&quot;math-container&quot;&gt;$p=0.125$&lt;/span&gt; for results of 1,2,3,4,5,6,7,8 for a d8&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it works mathematically. Whether it works for your situation depends on details. You need enough data samples to cover both the distribution of results under each setting, and any complexities of how that distribution varies with the settings. In the limit of wanting very accurate predictions of probability within a complex space you will need a huge number of samples. You should be able to find a compromise between accuracy and generalisation by trying different levels of regularisation - this will be necessary as over-fitting to input/output sample pairs as seen is going to be a serious problem for a neural network trained on this data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you can do to help a classifier learn probabilities is always take some number of samples with the same settings - e.g. 10 or 100 or 1000 each with the same settings - that should guarantee that the network cannot simply converge to predict high &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; values for single outputs as seen, as it will have counter-examples to work with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention that you have 40 dimensions of settings. Whether this is an issue will depend on how the probability distribution varies based on those settings. However, at minimum you should be thinking in terms of millions of samples for training, or possibly a fast on-demand generator that can generate 1000s of new samples per second to train with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can test accuracy by building histograms using some fixed (and as yet unseen) setting and comparing to NN predictions of probabilities of that setting. Even getting accurate test results is likely to require 1000s of samples.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However it may require a lot of training and data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you cannot obtain a very large training set here, then a purely statistical &quot;black box&quot; approach is probably not feasible, regardless of whether you use neural networks, or more raw analysis. What neural networks add is smooth interpolation between different settings values, as a form of approximation. This seems desirable for your problem, as you will never fully explore 40 dimensions of 100 values in the lifetime of the universe - but you need some confidence that minor changes in settings equate to minor changes in probability distributions in most cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's OK to have one or two major shifts across the input space, but if the distribution depends in some cryptographic primitive or similar complex high frequency (over space) and high amplitude on the input variables, there is no way to obtain approximations using statistics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative to statistical approaches is to find some way to break open this black box through analysis. No AI system can do that in general at the moment, so you would rely on human ingenuity.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-04T08:53:17.230" LastActivityDate="2019-07-04T08:53:17.230" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13201" PostTypeId="2" ParentId="13194" CreationDate="2019-07-04T08:52:41.193" Score="2" Body="&lt;p&gt;Yes, this is possible. Continuously extending your training data is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Incremental_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;incremental learning&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to take a look at &lt;a href=&quot;https://machinelearningmastery.com/transfer-learning-for-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt;, in which you reuse a trained model for a different purpose. This is very useful if you have a smaller dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your particular case, you could train a NN once using your data from 2010 to 2019 and use it as a base model. Every time you get new data, you can use transfer learning to slightly re-train this model. Based on parameters such as the number of epochs and the learning rate, you can determine how much of an impact this new data will have. &lt;/p&gt;&#xA;" OwnerUserId="18398" LastActivityDate="2019-07-04T08:52:41.193" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13202" PostTypeId="1" AcceptedAnswerId="13204" CreationDate="2019-07-04T12:54:02.223" Score="1" ViewCount="582" Body="&lt;p&gt;I am trying to build a network able to play snake game. This is my very first attempt to do such stuff. Unfortunately, I've stuck and even have no idea how to reason about the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use reinforcement neural network approach (q-leaning). My network is built on top of Keras. I use 6 input neurons for my snake:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 - is any collision directly behind&lt;/li&gt;&#xA;&lt;li&gt;2 - is any collision directly on the right&lt;/li&gt;&#xA;&lt;li&gt;3 - is any collision directly on the left&lt;/li&gt;&#xA;&lt;li&gt;4 - is snack up front (no matter how far)&lt;/li&gt;&#xA;&lt;li&gt;5 - is a snack on the right side (no matter how far)&lt;/li&gt;&#xA;&lt;li&gt;6 - is a snack on the left side (no matter how far)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;the output has 3 neurons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 - do nothing (go ahead)&lt;/li&gt;&#xA;&lt;li&gt;2 - turn right&lt;/li&gt;&#xA;&lt;li&gt;3 - turn left&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I believe this is a sufficient set of information to make proper decisions. But the snake seems to not even grasp the concept of not hitting the wall - which results with instant death.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the following rewards table:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;100 for getting the snack&lt;/li&gt;&#xA;&lt;li&gt;-100 for hitting wall/tail&lt;/li&gt;&#xA;&lt;li&gt;1 for staying alive (each step)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Snake tends to run randomly no matter how many training iterations it gets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is available on my github: &lt;a href=&quot;https://github.com/ayeo/snake/blob/master/main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ayeo/snake/blob/master/main.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26855" LastActivityDate="2019-07-04T13:34:34.903" Title="Reinforcement learning to play snake - network seems to not get trained at all" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13203" PostTypeId="1" CreationDate="2019-07-04T13:05:30.933" Score="1" ViewCount="16" Body="&lt;p&gt;i just tried to improve my image dataset by inverting the images with a probability of 50% (means white background, black features transforms to black background, white features)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought this will improve the ability to recognize abstract features for my network. Right now, the network does not perform really well. Is inverting the intensity of images too much for a training algorithm to deal with? &lt;/p&gt;&#xA;" OwnerUserId="26857" LastActivityDate="2019-07-04T13:05:30.933" Title="Inverting intensity on images to enhance image dataset" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13204" PostTypeId="2" ParentId="13202" CreationDate="2019-07-04T13:27:34.747" Score="2" Body="&lt;p&gt;I cannot comment much on your setup for inputs and outputs. It seems adequate to get some control, but does not cover the fully Markov state for the game, so I would expect that will limit the agent from ever being truly optimal. I would expect it to learn to play the game though, if you were implementing Q learning with a neural network correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your code, you are implementing a basic Q learning loop. It seems correct. However, this combination of Q learning and neural networks is known not to work - or more accurately, it rarely works this simply. The problem is mainly to do with the network receiving its own initially biased outputs back as new targets, plus receiving updates in correlated form (data on each time step is strongly correlated with data from previous time step). These biases are too large for the Q learning process to overcome, and typically the result is an agent that fixates on a single default action, because it has learned an inflated action value for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is well known in RL research and called &quot;&lt;a href=&quot;https://arxiv.org/abs/1812.02648&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Deadly Triad&lt;/a&gt;&quot; by Sutton &amp;amp; Barto.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual solution to this with Q learing is called DQN or &quot;Deep&quot; Q Learning (&quot;Deep&quot; is in quotes because this should be applied even if you just have a single hidden layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In basic DQN, you need to add the following features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;An experience replay table. Instead of training directly on experience as it is received, instead store &lt;span class=&quot;math-container&quot;&gt;$s, a, r, s'$&lt;/span&gt; in memory. When it is time later in the loop to train the NN for a step, take a random sample of some M items (e.g. 32 items) as a mini-batch, calculate a latest target for them, and train once on the mini batch. You will need logic to only start this training process once you have some minimal amount of experience from behaving randomly (e.g. 500 random steps).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &quot;target network&quot;. When generating target Q values, use a cloned copy of the learning network, and only update this clone every N steps (with N typically set at 1000 or 10000).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These two additions are not really optional, even for really basic environments. You will need to add them to your script.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-04T13:34:34.903" LastActivityDate="2019-07-04T13:34:34.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13206" PostTypeId="1" CreationDate="2019-07-04T14:41:52.150" Score="1" ViewCount="33" Body="&lt;p&gt;Hello Dear StackExchange members, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to make a deep network as shown in the image. I want each 'network 1 to look at the specific part of the input and I don't want to divide my input beforehand in chunks. Is there any such implementation available in any library? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zDX03.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zDX03.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;your help will be highly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best Regards&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ali Raza &lt;/p&gt;&#xA;" OwnerUserId="26860" LastActivityDate="2019-07-04T14:41:52.150" Title="How to implement Multiple Neural network architecture, connected in parallel and series in Keras or Pytorch" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-06-02T22:41:28.407" ContentLicense="CC BY-SA 4.0" />
  <row Id="13207" PostTypeId="2" ParentId="10562" CreationDate="2019-07-04T17:43:52.623" Score="0" Body="&lt;p&gt;Assuming that you have some experience with Jupyter Notebooks then you might find &lt;a href=&quot;https://colab.research.google.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Colaboratory&lt;/a&gt; does what you need. You can select a CPU, GPU or TPU instance to process your model. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-07-04T17:43:52.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13208" PostTypeId="1" CreationDate="2019-07-04T19:54:36.840" Score="1" ViewCount="84" Body="&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node73.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Excerpt&lt;/a&gt; from Sutton and Barto:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The expected value of all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns is guaranteed to improve in a&#xA;  certain way over the current value function as an approximation to the&#xA;  true value function. For any &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, the expected value of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step return&#xA;  using &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is guaranteed to be a better estimate of &lt;span class=&quot;math-container&quot;&gt;$V^{\pi}$&lt;/span&gt; than &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is, in a&#xA;  worst-state sense. That is, the worst error under the new estimate is&#xA;  guaranteed to be less than or equal to &lt;span class=&quot;math-container&quot;&gt;$\gamma^n$&lt;/span&gt; times the worst error under &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xl0tV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xl0tV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is why did they add the &lt;span class=&quot;math-container&quot;&gt;$\gamma^n$&lt;/span&gt; in the RHS of the equation?&#xA;My understanding is that in the LHS we use the &lt;span class=&quot;math-container&quot;&gt;$E_{\pi}$&lt;/span&gt; expectation term to get the estimate of &lt;span class=&quot;math-container&quot;&gt;$V^{\pi}(s)$&lt;/span&gt;, then why is the difference in the second term diminished by &lt;span class=&quot;math-container&quot;&gt;$\gamma^n$&lt;/span&gt;. Can it be explained informally or is it a result of a proof?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="1641" LastEditDate="2019-07-05T08:38:18.347" LastActivityDate="2019-07-05T08:38:18.347" Title="Doubt regarding improvement of State Value by n-step returns" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2021-02-05T15:21:51.647" ContentLicense="CC BY-SA 4.0" />
  <row Id="13209" PostTypeId="1" CreationDate="2019-07-05T04:58:39.447" Score="1" ViewCount="21" Body="&lt;p&gt;I am looking for dataset in tree structure that captures the hierarchy of concepts.&#xA;For example, something like,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                         Entertainment&#xA;                   Movies           Sports&#xA;              Comedy  Thriller   cricket football&#xA;   charlie chaplin              sachin       messy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3015" LastActivityDate="2019-07-05T04:58:39.447" Title="Is there any readily available concept/topic tree?" Tags="&lt;knowledge-representation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13210" PostTypeId="1" CreationDate="2019-07-05T05:04:18.513" Score="2" ViewCount="53" Body="&lt;p&gt;I'm aware of metrics like accuracy (correct predictions / total predictions) for models that classify things. However, I'm working on a model that outputs the probability of a datapoint belonging to one of two classes. What metrics can/should be used to evaluate these types of models?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently using mean squared error, but I would like to know if there are other metrics, and what the advantages/disadvantages of those metrics are.&lt;/p&gt;&#xA;" OwnerUserId="26875" LastEditorUserId="26875" LastEditDate="2019-07-05T07:24:26.243" LastActivityDate="2019-07-05T08:48:31.723" Title="Metrics for evaluating models that output probabilities" Tags="&lt;cross-validation&gt;&lt;metric&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13212" PostTypeId="2" ParentId="13210" CreationDate="2019-07-05T08:11:06.013" Score="1" Body="&lt;p&gt;For a binary classifier, the cross-entropy loss is a natural measure of probability accuracy, if you care about relative probabilities. By that I mean if you care that the estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{p}$&lt;/span&gt; is within some ratio of the true value. So an estimate of &lt;span class=&quot;math-container&quot;&gt;$\hat{p} = 0.1$&lt;/span&gt; is a better estimate if the true value is &lt;span class=&quot;math-container&quot;&gt;$p = 0.2$&lt;/span&gt; than if the true value is &lt;span class=&quot;math-container&quot;&gt;$p = 0.01$&lt;/span&gt; (even though the latter value is closer and would score better under MSE). It also applies that you care that 0.9 is by the same logic &quot;closer&quot; to 0.8 than it is to 0.97. With cross-entropy loss, extreme confidence (predicting close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or close to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) is penalised more heavily when it is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, the loss function (per data point) is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}(\hat{y},y) = -(y\text{log}(\hat{y})+ (1-y)\text{log}(1-\hat{y}))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is likely to be the same loss function as you are using for your objective (or at least it should be), so for tests, simply also use it as your metric*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; is your predicted probability of being in class A, and can be in range &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;. Ideally you have ground truth probabilities and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is also in that range. In which case the only problem is that the &quot;perfect&quot; score is no longer &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; but some positive number. If that bothers you, then you could offset by the perfect score, pre-calculating it on each data set (just set &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = y$&lt;/span&gt; for each item and you will find the minimum possible score).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have ground truth probabilities, but you do have classes, then &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; will either be 0 or 1, and the metric still works. To get an accurate metric, you will need enough samples that the relative frequencies of each class depending on input has a significant effect. That is, you need more data, both training and test, in order to train for accurate probabilities instead of targeting simpler classification accuracy metrics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar logic also works for multi-class probabilities. However, many off-the-shelf libraries use an optimisation in the loss function - assuming only one true class - which makes using probabilities as ground truth impossible. You might therefore need to write your own loss function and gradient functions based on multi-class cross-entropy loss in that case.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* I am making the assumption here that you use standard conventions for noting loss functions (typically per item), cost functions (typically aggregated across a data set and possibly multiple loss functions) and metric functions which don't have to be differentiable or usable as either of the former. The cost function is usually also fed into optimisers as the objective function - i.e. it has the important job of driving parameters to reach a maximum or minimum value. For gradient based solvers, that means it must be differentiable.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-05T08:48:31.723" LastActivityDate="2019-07-05T08:48:31.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13213" PostTypeId="2" ParentId="10806" CreationDate="2019-07-05T10:49:29.300" Score="2" Body="&lt;p&gt;The minimal algorithm for convolution in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; is a four dimensional iteration.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for all vertical kernel positions&#xA;  for all horizontal kernel positions&#xA;    initialize the value at the output position to the bias&#xA;    for all vertical positions in the kernel&#xA;      for all horizontal positions in the kernel&#xA;        add the product of the input value to that of the output position&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; it is a &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; dimensional iteration following this pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimal algorithm for regression of bounding boxes orthogonal with respect to the image grid (no tilting) is this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;until number of boxes reaches max&#xA;  make first guess of two coordinates&#xA;  until number of guesses reaches max or matching criteria is met&#xA;    evaluate guess&#xA;    remember guess and guess results&#xA;    improve on guess based on evaluation results and&#xA;          possibly injected randomness,&#xA;          excluding locations already covered&#xA;    if some intermediate criteria is met&#xA;      change the nature of the guessing, evaluation, and improving&#xA;            as is appropriate for the criteria match&#xA;            (this covers approaches that have multiple phases)&#xA;  if no guess matched criteria&#xA;    break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's approaching concepts from the top down. When approaching from the other direction, reverse engineer the best code. In the case of RCNN, it is unadvisable to find implementations following the first paper expressing the approach. Reading the first paper may be helpful to get the gist of the approach, but reverse engineer the best one, which, in this case, may be &lt;a href=&quot;https://arxiv.org/pdf/1506.01497.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks&lt;/em&gt;&lt;/a&gt;, Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun, 2016. Study the implementation they pushed to git at &lt;a href=&quot;https://github.com/rbgirshick/py-faster-rcnn/tree/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rbgirshick/py-faster-rcnn/tree/master/&lt;/a&gt;. The algorithm is in lib/fast_rcnn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason this algorithm isn't spelled out in their paper or any paper from the first on down through the lineage to their paper is simple.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The pseudo-code above is universal across all convolutions and all bounding box regressions, so that doesn't need to be restated with each approach.&lt;/li&gt;&#xA;&lt;li&gt;The main features of an approach like RCNN, SSD, or YOLO are not algorithmic. They are algebraic expressions of the guess, the evaluation, the improvement upon the guess, and the test for the criteria.&lt;/li&gt;&#xA;&lt;li&gt;The use of objects and functional programming makes the implementation more readable, so it can be easier to read the implementation than read a huge chunk of the above pseudo-code with all the algebra and test branches plugged in.&lt;/li&gt;&#xA;&lt;li&gt;For the above reasons, it is rare that pseudo-code would be used prior to the implementation when the paper is written.&lt;/li&gt;&#xA;&lt;li&gt;The return on investment of reverse engineering from code to pseudo-code is only sufficient motivation if one is going to improve the algorithm and write another paper, and on the way to finishing the prior paper's pseudo-code, the new paper and the new code gets finished first.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since the author of this question seems interested in writing their own code, it may be reasonable to assume the same author may be interested in thinking their own thoughts, so I'll add this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of these algorithms are object recognition. Recognition has to do with cognition, and these approaches do not even touch upon cognitive processing, another branch of AI not related to convolution and probably not closely related to formal regression either. Additionally, bounding boxes are not the way animal vision systems work. Early gestalt experiments in vision indicate a complete independence of human vision from rectilinear formalities. In lay terms, humans and other organisms with vision systems don't have any conception of Cartesian coordinates. We can still read books if tilted slightly relative to the plane passing through our eyes. We don't zoom or tilt in Cartesian coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These facts may not be necessary to comprehend to create an automated vehicle driving system that produces a better safety record than average human drivers, but that is only because humans don't set that bar very high and because cars roll in the plane of the road. These facts are indeed necessary in aeronautic system used in military applications, where nothing is particularly Cartesian and the meaning of horizontal and vertical is ambiguous. For that reason, it is unlikely that bounding boxes will be the edge of vision technology for very long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one wishes to transcend current mediocrity, consider bounding circles with fuzzy boundaries, which would be more like the systems that evolved over millions of biological iterations. If the computer hardware is poorly fit to radial processing, design new hardware in which radial processing is native and in which Cartesian coordinates may be foreign and cumbersome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the classifier, the classifier papers do generally include the algorithm, so those can be found by doing an academic search for the original paper describing the classifier being used.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-07-05T10:55:56.830" LastActivityDate="2019-07-05T10:55:56.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13214" PostTypeId="1" CreationDate="2019-07-05T11:59:18.560" Score="1" ViewCount="36" Body="&lt;p&gt;Given a set of time series data that are generated from different sites where all sites are investigating the same objective but with slightly different protocols. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use &lt;em&gt;adversarial learning&lt;/em&gt; to learn site invariant features for a classification problem, that is, how can adversarial learning be used to minimize experimental differences (e.g. different measurement equipment) so that the learned feature representations from the time series are homogenous for a classification problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have come across &lt;em&gt;multi-domain adversarial learning&lt;/em&gt;, but I'm not sure if this is the best formulation for my problem.&lt;/p&gt;&#xA;" OwnerUserId="26885" LastEditorUserId="2444" LastEditDate="2019-07-06T12:52:35.840" LastActivityDate="2019-07-06T12:52:35.840" Title="Is it possible to use adversarial training to learn invariant features?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;time-series&gt;&lt;feature-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13216" PostTypeId="2" ParentId="7685" CreationDate="2019-07-05T13:16:05.383" Score="1" Body="&lt;p&gt;For everybody getting here from google, like me: the &lt;span class=&quot;math-container&quot;&gt;$\log$&lt;/span&gt; might have been replaced in the loss function, but I think it is still there when taking the gradient of both functions (correct me, if I am wrong):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{aligned}&#xA;\nabla_{\theta} L^{P G}(\theta) &amp;amp;=\nabla_{\theta} \hat{E}_{t}\left[\log \pi_{\theta}\left(a_{t} \mid s_{t}\right) \hat{A}_{t}\right] \\&#xA;&amp;amp;=\hat{E}_{t}\left[\nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) \hat{A}_{t}\right]&#xA;\end{aligned}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\nabla_{\theta} L^{I S}(\theta)=&amp;amp; \nabla_{\theta} \hat{E}_{t} \left[\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \hat{A}_{t}\right] \\&#xA;&amp;amp;=\hat{E}_{t} \left[\nabla_{\theta} \frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \hat{A}_{t}\right] \\&#xA;&amp;amp;=\hat{E}_{t} \left[\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \frac{\nabla_{\theta} \pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta}\left(a_{t} \mid s_{t}\right)} \hat{A}_{t}\right] \\&#xA;&amp;amp;=\hat{E}_{t}\left[\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) \hat{A}_{t}\right]&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, the &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta}\left(a_{t} \mid s_{t}\right) $&lt;/span&gt; in the PG function was replaced with &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} $&lt;/span&gt; whose derivate is the same as the log of the PG function (apart from the proportionality factor).&lt;/p&gt;&#xA;" OwnerUserId="26876" LastEditorUserId="2444" LastEditDate="2020-07-18T15:08:05.990" LastActivityDate="2020-07-18T15:08:05.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13217" PostTypeId="1" AcceptedAnswerId="13409" CreationDate="2019-07-05T13:55:26.420" Score="3" ViewCount="84" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1905.01067.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask&lt;/a&gt;, they learn a mask for the network by setting up the mask parameters as &lt;span class=&quot;math-container&quot;&gt;$M_i = Bern(\sigma(v_i))$&lt;/span&gt;. Where &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the parameter mask (&lt;span class=&quot;math-container&quot;&gt;$f(x;\theta, M) = f(x;M \odot \theta$&lt;/span&gt;), &lt;span class=&quot;math-container&quot;&gt;$Bern$&lt;/span&gt; is a Bernoulli sampler, &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the sigmoid function, and &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; is some trainable parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper, they learn &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; using SGD. I was wondering how they managed to do that, because there isn't a reparameterization trick, as there is for some other distributions I see trained on in the literature (example: normal).&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2019-07-05T17:00:07.237" LastActivityDate="2019-07-17T16:43:34.750" Title="How are the parameters of the Bernoulli distribution learned?" Tags="&lt;machine-learning&gt;&lt;probability-distribution&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13218" PostTypeId="1" CreationDate="2019-07-05T15:11:45.923" Score="1" ViewCount="41" Body="&lt;p&gt;For a demo I need to develop an AI solution to learn how to play pong. I have the following requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Computer needs to play against a human player. Learn while playing the game.&lt;/li&gt;&#xA;&lt;li&gt;Poor AI result in the first game played.&lt;/li&gt;&#xA;&lt;li&gt;A significant improvement must be noticeable within minutes.&lt;/li&gt;&#xA;&lt;li&gt;Develop it within weeks. So open source.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus far I found during my search that I need to use the &lt;code&gt;gym&lt;/code&gt; library as a platform. Pretty much all examples I've seen have used it. Google deepmind is by far the greatest expert in the field and I intend to use their work as it is open-source as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First question is: is there any other solution that meets my requirements and is open source?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some questions about the SimPLe package documentation, since it seems to be the latest produced by deepmind (&lt;a href=&quot;https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/rl/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;). So anyone who has used it:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What do they mean with pretrained policy? Is this one fully capable to compete with a human? Or is it pretrained in the sense that CNN's can be pretrained: Only convolution layer is trained, they will still suck without more training?&lt;/li&gt;&#xA;&lt;li&gt;Second chapter in the readme is called &quot;Train your policy (model-free training)&quot;. Does this mean all the other options are not trained using a human? Using model-free training. From the paper it is clear that model-free training is inferior to World-model training, but if it is the only model I can use with humans training it, I will have to use it.&lt;/li&gt;&#xA;&lt;li&gt;The third chapter describes World Model training (with random trajectories). It is not mentioned whether this uses human input for training or not. Does it? If not, how much time does it take to get a noticeable result (computer bouncing first ball)?&lt;/li&gt;&#xA;&lt;li&gt;There is also an option to learn a Model-based training with pre-trained world models. How fast does this one learn? Does it already have a good performance from the start?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26888" LastActivityDate="2019-07-05T15:11:45.923" Title="Questions performance SimPLe pong for AI demo" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;deep-rl&gt;&lt;deepmind&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13221" PostTypeId="1" CreationDate="2019-07-06T02:43:29.587" Score="3" ViewCount="262" Body="&lt;p&gt;Most image classifiers like Inception-v3 accept images of about size 299 x 299 x 3 as input. In this particular case, I cannot resize the image and lose resolution. Is there an easy solution of dealing with this rather than retraining the model? (Particularly in tensorflow)&lt;/p&gt;&#xA;" OwnerUserId="26900" LastActivityDate="2020-12-22T05:03:28.993" Title="Is there a simple way of classifying images of size differing from the input of existing image classifiers?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;tensorflow&gt;&lt;models&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13223" PostTypeId="1" CreationDate="2019-07-06T09:16:24.047" Score="2" ViewCount="134" Body="&lt;p&gt;Does anyone know a paper or code that does &quot;unsupervised domain adaptation&quot; for regression task?&#xA;I saw most of the papers were benchmarked on classification tasks, not regression.&#xA;I want to do something like training a model to predict a scalar value from an image (e.g. predicting image of a road to steering wheel angle for a self-driving car).&#xA;One of the examples could be training on synthesis data from simulated environment (think GTA) and then trying to predict on real-world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one of the examples of unsupervised domain adaptation algorithm that also has an easy-to-access code with Keras: &lt;a href=&quot;https://github.com/bbdamodaran/deepJDOT&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bbdamodaran/deepJDOT&lt;/a&gt;&#xA;But it's for classification. The author said it can be used for regression but I had to change it. I changed it and it didn't work well so I don't know if it's my fault or the algo is not good for regression. I want to see papers that were benchmarked on regression so I know how well it performs on regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My real use case is to predict facial expression as a value from 0 to 1 like how open is the mouth. The source domain and target domain are real-world images but from different lighting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-07-20T04:22:55.800" Title="Paper &amp; code for &quot;unsupervised domain adaptation&quot; for regression task" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13224" PostTypeId="5" CreationDate="2019-07-06T13:03:38.473" Score="0" Body="&lt;p&gt;For more details, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Feature_learning&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/pdf/1206.5538.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Representation Learning: A Review and New Perspectives&lt;/a&gt; by Yoshua Bengio, Aaron Courville, and Pascal Vincent.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T13:03:38.473" LastActivityDate="2019-07-06T13:03:38.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13225" PostTypeId="4" CreationDate="2019-07-06T13:03:38.473" Score="0" Body="For questions related to feature learning (also known as representation learning), which is a set of techniques that can learn the features associated with the raw data. It is similar to feature engineering, but, in the case of feature learning, the features are learned and not handcrafted." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T13:03:38.473" LastActivityDate="2019-07-06T13:03:38.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13226" PostTypeId="5" CreationDate="2019-07-06T13:06:58.287" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_selection&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Feature_selection&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T13:06:58.287" LastActivityDate="2019-07-06T13:06:58.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13227" PostTypeId="4" CreationDate="2019-07-06T13:06:58.287" Score="0" Body="For questions related to the concept of feature selection (also known as variable selection or attribute selection), which is the process of selecting a subset of relevant features (a.k.a. variables or predictors) for use in model construction." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T13:06:58.287" LastActivityDate="2019-07-06T13:06:58.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13228" PostTypeId="5" CreationDate="2019-07-06T13:18:16.020" Score="0" Body="&lt;p&gt;For more details, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_extraction&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Feature_extraction&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T13:18:16.020" LastActivityDate="2019-07-06T13:18:16.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13229" PostTypeId="4" CreationDate="2019-07-06T13:18:16.020" Score="0" Body="For questions related to the concept of feature extraction, which is a set of techniques used to derive or create features from the existing set of features. Feature extraction is different from feature selection, which is used to select a subset of the existing features." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-06T13:18:16.020" LastActivityDate="2019-07-06T13:18:16.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13230" PostTypeId="1" CreationDate="2019-07-06T16:49:25.543" Score="0" ViewCount="72" Body="&lt;p&gt;I have a piece of code and I don't seem to really understand it but I'd love to get a source/link/material that would help me understand the basic functions in TensorFlow. Are there any recommended resources for learning the same?&lt;/p&gt;&#xA;" OwnerUserId="26845" LastEditorUserId="9947" LastEditDate="2019-07-06T19:46:23.157" LastActivityDate="2019-12-03T20:02:00.347" Title="Is there a place where I can read or watch to get an accurate TensorFlow code wise explanation?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13231" PostTypeId="2" ParentId="13230" CreationDate="2019-07-06T19:44:55.333" Score="2" Body="&lt;p&gt;The best resource for learning TensorFlow 1.9 and earlier is this &lt;a href=&quot;http://web.stanford.edu/class/cs20si/syllabus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;course&lt;/a&gt; by Stanford. Also additional resources for the entire overview of TensorFlow and its comparisons with NumPy has been made in this &lt;a href=&quot;https://www.youtube.com/watch?v=l6K-MFgIEjc&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;. For hands on models check these &lt;a href=&quot;https://www.youtube.com/watch?v=oYbVFhK_olY&amp;amp;list=PLSPWNkAMSvv5DKeSVDbEbUKSsK4Z-GgiP&quot; rel=&quot;nofollow noreferrer&quot;&gt;videos&lt;/a&gt; by Sentdex and also some high level &lt;a href=&quot;https://www.youtube.com/watch?v=er8RQZoX3yk&amp;amp;list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorials&lt;/a&gt; by Hvass Labs.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-07-06T19:44:55.333" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13232" PostTypeId="1" AcceptedAnswerId="13285" CreationDate="2019-07-06T20:47:06.457" Score="2" ViewCount="180" Body="&lt;p&gt;Is there any empirical/theoretical evidence on the effect of initial values of state-action and state values on the training of an RL agent (the values an RL agent assigns to visited states) via MC methods Policy Evaluation and GLIE Policy Improvement?&lt;/p&gt;&#xA;&lt;p&gt;For example, consider two initialization scenarios of &lt;a href=&quot;https://rdrr.io/cran/reinforcelearn/man/WindyGridworld.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Windy Gridworld&lt;/a&gt; problem:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Implementation&lt;/strong&gt;: I have modified the problem along with step penalty to include a non-desired terminal state and a desired terminal state which will be conveyed to the agent as a negative and positive reward state respectively. The implementation takes care that the MC sampling ends at the terminal state and gives out penalty/reward as a state-action value and not state value, since this is a control problem. Also, I have 5 moves: north, south, east, west and stay.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; I am not sure whether this changes the objective of the problem. In the original problem, it was to reduce the number of steps required to reach the final stage.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;We set the reward of reaching the desired terminal state to a value that is higher than the randomly initialized values of the value function; for example, we can set the reward to &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; and initialize the values with random numbers in the range &lt;span class=&quot;math-container&quot;&gt;$[1, 7]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We set the reward of reaching the desired terminal state to a value that is comparable to the randomly initialized values of the value functions; for example, we can set the reward to &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; and initialize the values with random numbers in the range &lt;span class=&quot;math-container&quot;&gt;$[1, 10]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As far as I see, in the first case, the algorithm will easily quickly converge as the reward is very high for the terminal reward state which will skew the agent to try to reach the reward stage.&lt;/p&gt;&#xA;&lt;p&gt;In the second case, this might not be true if the reward state is surrounded by other high reward states, the agent will try to go to those states.&lt;/p&gt;&#xA;&lt;p&gt;The step penalty ensures that the agent finally reaches the terminal state, but will this skew the path of the agent and severely affect its convergence time? This might be problematic in large state spaces since we will not be able to explore the entire state space, but the presence of exploratory constant &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; might derail the training by going to a large false reward state. Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-11-01T16:16:13.873" LastActivityDate="2020-11-01T16:16:13.873" Title="How does the initialization of the value function and definition of the reward function affect the performance of the RL agent?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;weights-initialization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13233" PostTypeId="1" AcceptedAnswerId="13238" CreationDate="2019-07-07T06:14:46.673" Score="13" ViewCount="5163" Body="&lt;p&gt;I listened to a talk by panel consisted of two influential Chinese scientists: &lt;a href=&quot;https://sites.google.com/site/gangwang6/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wang Gang&lt;/a&gt; and &lt;a href=&quot;https://www.crunchbase.com/person/kai-yu#section-overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yu Kai&lt;/a&gt; and others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When being asked about the biggest bottleneck of the development of artificial intelligence in the near future (3 to 5 years), Yu Kai, who has a background in the hardware industry, said that hardware would be the essential problem and we should pay most of our attention to that. He gave us two examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In the early development of the computer, we compare our machines by its chips;&lt;/li&gt;&#xA;&lt;li&gt;Artificial intelligence which is very popular these years would be almost impossible if not empowered by Nvidia's GPU.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The fundamental algorithms existed already in the 1980s and 1990s, but artificial intelligence went through 3 AI winters and was not empirical until we can train models with GPU boosted mega servers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then Dr. Wang commented to his opinions that we should also develop software systems because we cannot build an automatic car even if we have combined all GPUs and computation in the world together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, as usual, my mind wandered off and I started thinking that what if those who can operate supercomputers in the 1980s and 1990s utilized the then-existing neural network algorithms and train them with tons of scientific data? Some people at that time can obviously attempt to build AI systems we are building now. But why did AI become a hot topic and become empirical until decades later? Is it only a matter of hardware, software and data?&lt;/p&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="2193" LastEditDate="2019-07-08T14:32:44.360" LastActivityDate="2019-07-08T14:32:44.360" Title="Why did ML only become viable after Nvidia's chips were available?" Tags="&lt;history&gt;&lt;hardware&gt;&lt;software-development&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13235" PostTypeId="2" ParentId="13233" CreationDate="2019-07-07T07:30:44.723" Score="2" Body="&lt;p&gt;GPUs were ideal for AI boom becouse&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They hit the right time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;AI has been researched for a LONG time. Almost half a century. However, that was all exploration of how would algorithms work and look. When NV saw that the AI is about to go mainstream, they looked at their GPUs and realized that the huge parellel processing power, with relative ease of programing, is ideal for the era that is to be. Many other people realized that too.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GPUs are sort of general purpose accelerators&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;GPGPU is a concept of using GPU parallel processing for general tasks. You can accelerate graphics, or make your algorithm utalize 1000s of cores available on GPU. That makes GPU awesome target for all kinds of use cases including AI. Given that they are already available and are not too hard to program, its ideal choice for accelerating AI algorithms.&lt;/p&gt;&#xA;" OwnerUserId="23262" LastEditorUserId="23262" LastEditDate="2019-07-07T16:04:01.243" LastActivityDate="2019-07-07T16:04:01.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13237" PostTypeId="1" AcceptedAnswerId="13240" CreationDate="2019-07-07T07:49:10.173" Score="2" ViewCount="84" Body="&lt;p&gt;Certain hyper-parameters (e.g. the size of the offspring generation or the definition of the fitness function) and the design (e.g. how the mutation is performed) of evolutionary algorithms usually need to be defined or specified by a human. Could also these definitions be automated? Could we also mutate the fitness function or automatically decide the size of the offspring generation?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2019-07-07T22:53:48.553" LastActivityDate="2019-07-07T22:53:48.553" Title="Can we automate the choice of the hyper-parameters of the evolutionary algorithms?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;hyper-parameters&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13238" PostTypeId="2" ParentId="13233" CreationDate="2019-07-07T09:25:17.173" Score="15" Body="&lt;p&gt;There is a lot of factors for the boom of AI industry. What many people miss though is the boom has mostly been in the Machine Learning part of AI. This can be attributed to various simple reasons along with their comparisons during earlier times:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Mathematics&lt;/strong&gt;:  The maths behind ML algorithms are pretty simple and known for a long time (whether it would work or not was not known though). During earlier times it was not possible to implement algorithms which require high precision of numbers, to be calculated on a chip, in an acceptable amount of time. One of the main arithmetic operations division of numbers still takes a lot of cycles in modern processors. Older processors were magnitude times slower than modern processors (more than 100x), this bottleneck made it impossible to train sophisticated models on contemporary processors.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Precision&lt;/strong&gt;: Precision in calculations is an important factor in ML algorithms. 32 bit precision in processor was made in the 80's and was probably commercially available in the late 90's (&lt;a href=&quot;https://en.wikipedia.org/wiki/Intel_8086&quot; rel=&quot;noreferrer&quot;&gt;x86&lt;/a&gt;), but it was still hella slow than current processors. This resulted in scientists improvising on the precision part and the most basic Perceptron Learning Algorithm invented in the 1960's to train a classifier uses only &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;'s and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;'s, so basically a binary classifier. It was run on &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptron&quot; rel=&quot;noreferrer&quot;&gt;special computers&lt;/a&gt;. Although, it is interesting to note that we have come a full circle and Google is now using &lt;a href=&quot;https://en.wikipedia.org/wiki/Tensor_processing_unit&quot; rel=&quot;noreferrer&quot;&gt;TPU's&lt;/a&gt; with 8-16 bit accuracy to implement ML models with great success.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Parallelization&lt;/strong&gt; : The concept of parallelization of matrix operations is nothing new. It was only when we started to see Deep Learning as just a set of matrix operations we realized that it can be easily parallelized on massively parallel GPU's, still if your ML algorithm is not inherently parallel then it hardly matters whether you use CPU or GPU (e.g. RNN's).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Data&lt;/strong&gt;: Probably the biggest cause in the ML boom. The Internet has provided opportunities to collect huge amounts of data from users and also make it available to interested parties. Since an ML algorithm is just a function approximator based on data, therefore data is the single most important thing in a ML algorithm. The more the data the better the performance of your model.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cost&lt;/strong&gt;: The cost of training a ML model has gone down significantly. So using a Supercomputer to train a model might be fine, but was it worth it? Super computers unlike normal PC's are tremendously resource hungry in terms of cooling, space, etc. A recent &lt;a href=&quot;https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/&quot; rel=&quot;noreferrer&quot;&gt;article&lt;/a&gt; on MIT Technology Review points out the carbon footprint of training a Deep Learning model (sub-branch of ML). It is quite a good indicator why it would have been infeasible to train on Supercomputers in earlier times (considering modern processors consume much lesser power and gives higher speeds).&#xA;Although, I am not sure but I think earlier supercomputers were specialised in &quot;parallel+very high precision computing&quot; (required for weather, astronomy, military applications, etc) and the &quot;very high precison part&quot; is overkill in Machine Learning scenario.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Another important aspect is nowadays everyone has access to powerful computers. Thus, anyone can build new ML models, re-train pre-existing models, modify models, etc. This was quite not possible during earlier times, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this factors has led to a huge surge in interest in ML and has caused the boom we are seeing today. Also check out this &lt;a href=&quot;https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai/&quot;&gt;question&lt;/a&gt; on how we are moving beyond digital processors.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-08T05:04:17.517" LastActivityDate="2019-07-08T05:04:17.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13240" PostTypeId="2" ParentId="13237" CreationDate="2019-07-07T09:38:00.177" Score="1" Body="&lt;p&gt;Yes, you can also automate the choice of certain hyperparameters of the evolutionary algorithm. In this context, this process is called &lt;em&gt;self-adaptation&lt;/em&gt;. There are different ways of performing self-adaptation (depending on the hyper-parameter that needs to self-adapt). See e.g. the chapter &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.60.4624&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-Adaptation in Evolutionary Algorithms&lt;/a&gt; (by Silja Meyer-Nieberg and Hans-Georg Beyer) of the book &lt;a href=&quot;https://link.springer.com/book/10.1007/978-3-540-69432-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parameter Setting in Evolutionary Algorithms&lt;/a&gt; (2007).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-07T09:46:36.060" LastActivityDate="2019-07-07T09:46:36.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13241" PostTypeId="1" AcceptedAnswerId="13242" CreationDate="2019-07-07T09:52:08.340" Score="1" ViewCount="218" Body="&lt;p&gt;Dietterich, who introduced the &lt;a href=&quot;https://arxiv.org/pdf/cs/9905014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;taxi environment&lt;/a&gt; (see p. 9), states the following: In total there “are 500 [distinct] possible states: 25 squares, 5 locations for the passenger (counting the four starting locations and the taxi), and 4 destinations” (Dietterich, 2000, p. 9).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my opinion there are only 25 (grid) * 4 (locations) * 2 (passenger in car) = 200 different states, because for the agent it should be the same task to go to a certain point, regardless of whether it's on its way to pick up or to drop-off. Only the action at the destination is different which would be stored binary (passenger in car or not)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does Dietterich come up with 500 states? &lt;/p&gt;&#xA;" OwnerUserId="21299" LastEditorUserId="21299" LastEditDate="2019-07-07T11:01:24.500" LastActivityDate="2019-07-07T11:01:24.500" Title="Number of states in taxi environment (Dietterich 2000)" Tags="&lt;reinforcement-learning&gt;&lt;combinatorics&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13242" PostTypeId="2" ParentId="13241" CreationDate="2019-07-07T10:13:15.007" Score="2" Body="&lt;p&gt;This is more of a combinatorics than AI question but regradless, the full state information for the environment is:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(taxi \space position, passenger \space position, destination \space position)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 25 possible taxi positions, 5 passenger positions and 4 destination positions making it &lt;span class=&quot;math-container&quot;&gt;$25 \cdot 5 \cdot 4 = 500$&lt;/span&gt;, so the paper is correct.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are also correct but you divided 1 objective into 2 objectives and you have 2 separate policies, a pickup policy and a dropoff policy. So your state information would be for each policy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(taxi \space position, destination \space position)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 25 possible taxi positions and 4 possible destination positions making it &lt;span class=&quot;math-container&quot;&gt;$25 \cdot 4 = 100$&lt;/span&gt;. You have 2 policies so you have &lt;span class=&quot;math-container&quot;&gt;$200$&lt;/span&gt; states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually in the second case, I think you could get away with only 1 policy where you would simply change the destination position once you pick up the passenger so you dont need 2 separate policies and you would have only &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; states&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-07-07T10:19:06.280" LastActivityDate="2019-07-07T10:19:06.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13243" PostTypeId="2" ParentId="13241" CreationDate="2019-07-07T10:20:10.023" Score="2" Body="&lt;p&gt;This . . .&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;because for the agent it should be the same task to go to a certain point, regardless of whether it's on its way to pick up or to drop-off&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;. . . might seem logical/intuitive to a person understanding the task, but it is not mathematically correct. The agent cannot &quot;merge&quot; states because they involve the same behaviour. It &lt;em&gt;must&lt;/em&gt; count differences in state as the combinations are presented. Critically, heading towards the passenger location or heading towards the goal location are not in any way similar to the agent, unless you manipulate the state to make them so*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually the taxi will learn very similar navigation behaviour for picking up and dropping off a passenger. However, using a basic RL agent it learns these very much separately, and must re-learn the navigation rules independently for each combination of passenger and goal location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An agent that learned navigation within the environment, and then combined it into different tasks might be an example of hierarchical reinforcement learning, transfer learning, or curriculum learning. These are more sophisticated learning approaches, but it is quite interesting that even very basic RL problems can demonstrate a use for higher level abstractions. Most agents used on the taxi problem don't do this though, as 500 states is really very easy to &quot;brute force&quot; using the simplest algorithms.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* You could modify the state representation to rationalise the task and make it have less states, similar to your suggestion. For instance, have one &quot;target&quot; location which could either be pickup or drop off, and a boolean &quot;carrying passenger&quot; state component. That would indeed reduce the number of states. However, that has involved you as the problem designer simplifying the problem to make it easier for the agent. Given that this is a toy problem designed as a benchmark to see how different agents perform, by doing that you subvert the purpose of the environment. If you were creating an agent to work on a harder real world problem though, it might be a very good idea to look for symmetries and ways to simplify state representation which would speed up learning.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-07T10:31:55.663" LastActivityDate="2019-07-07T10:31:55.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13244" PostTypeId="2" ParentId="7369" CreationDate="2019-07-07T13:15:11.727" Score="2" Body="&lt;p&gt;The problem isn't the GAN but the implementation of its discriminator which is typically a convolutional neural network (CNN).  CNNs have trouble with sparse data.  They require dense data to learn well.  There are ways to work around this.  See the following for some ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.groundai.com/project/sparse-and-dense-data-with-cnns-depth-completion-and-semantic-segmentation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://medium.com/@fkariminejadasl/sparse-data-in-deep-learning-7380dd19462c&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sparse data in deep learning&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/5125/a16039cabc6320c908a4764f32596e018ad3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SECOND: Sparsely Embedded Convolutional Detection&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2019-07-07T13:15:11.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13247" PostTypeId="1" AcceptedAnswerId="13930" CreationDate="2019-07-07T17:55:33.760" Score="2" ViewCount="105" Body="&lt;p&gt;I am implementing NEAT (Neuroevolution of augmenting topologies) by Stanley, &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Original Paper&lt;/a&gt;. I am facing a problem during crossover of genomes. Suppose two networks with connections&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Genome1 = {&#xA;            (1, Input1, Output), //Numbers represent innovation numbers&#xA;            (2, Input2, Output)&#xA;           } //More Fit&#xA; Genome2 = {&#xA;            (1, Input1, Output),&#xA;            (2, Input2, Output), //Disabled&#xA;            (3, Input2, Hidden1),&#xA;            (4, Hidden1, Output)&#xA;           }&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are crossed over, then the connection (Input2, Output) in the fitter parent has a chance of being disabled,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There’s&#xA;  a preset chance that an inherited gene is disabled if it is disabled in either parent. (Pg. 109, Section 3.2, Figure 4)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and thus producing the following offspring:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Child = {&#xA;          (1, Input1, Output),&#xA;          (2, Input2, Output) //Disabled&#xA;         }&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and thus render the network non functional. Similarly by this chance nodes can get left in a state of uselessness after crossover (As having no outgoing connections or no connections at all). How can this be prevented or am I missing something here?&lt;/p&gt;&#xA;" OwnerUserId="26927" LastActivityDate="2019-08-13T03:58:42.020" Title="Disabling of genes during crossover (NEAT)" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13249" PostTypeId="5" CreationDate="2019-07-07T19:11:41.600" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Neuroevolution&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-07T19:11:41.600" LastActivityDate="2019-07-07T19:11:41.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13250" PostTypeId="4" CreationDate="2019-07-07T19:11:41.600" Score="0" Body="For questions related to neuroevolution (or neuro-evolution) techniques, such as NEAT, that are used to evolve (or train) artificial neural networks (that is, they are used evolve their parameters or topology), inspired by the natural evolution. A neuroevolution algorithm is thus an evolutionary algorithm where the genomes (individuals or chromosomes) are artificial neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-07T22:44:27.293" LastActivityDate="2019-07-07T22:44:27.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13251" PostTypeId="1" CreationDate="2019-07-07T20:45:41.580" Score="2" ViewCount="55" Body="&lt;p&gt;I am looking at a problem which can be distilled as follows:  I have a phenomenon which can be modeled as a probability density function which is &quot;messy&quot; in that it sums to unity over its support but is somewhat jagged and spiky, and does not correspond to any particular textbook function.  It takes considerable amounts of time to generate these experimental density functions, along with conditional data for machine learning, but I have them.  I also have a crude model which runs quickly but performs poorly, i.e., generates poor quality density functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to train a neural network to transform the crude estimated pdfs to something closer to the experimentally generated pdfs, if possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To investigate this, I've further reduced this to the most toy-like toy problem I can think of:  Feeding a narrow, smooth (relatively narrow) normal curve into a 1D convolutional neural network, and trying to transform it to a similar narrow curve with a different mean.  Both input and output have fine enough support (101 points) to be considered as a smooth pdf.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the crux of the problem I think I have: &lt;em&gt;I do not know what a good loss function is for this problem.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L1, L2 and similar losses are useless, given that once the non-zero parts of the pdfs are non-overlapping, it doesn't matter how far apart the means are, the loss remains the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been experimenting with &lt;a href=&quot;https://www.kernel-operations.io/geomloss/api/pytorch-api.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sinkhorn approximations&lt;/a&gt; to optimal transport, to properly capture the intuition of &quot;distance&quot; but somewhat surprisingly these have not been helpful either.  I think part of the problem may be an (unavoidable?) numerical stability issue related to the support, but I would not stake hard money on that assumption.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If support is at percentiles on the [0,1] it is quite instructive (and dismaying) to look at the sinkhorn loss for normal functions with the mean directly on a point of support, vs normal functions with the mean directly between two points of support.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a problem in this vein, are there any recommended loss functions (preferably supported by or easily implement in PyTorch) which might work better?&lt;/p&gt;&#xA;" OwnerUserId="15020" LastEditorUserId="2444" LastEditDate="2019-07-07T21:29:24.223" LastActivityDate="2019-07-07T21:29:24.223" Title="Which loss functions for transforming a density function to another density function?" Tags="&lt;ai-design&gt;&lt;loss-functions&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13252" PostTypeId="5" CreationDate="2019-07-07T21:01:14.647" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Loss_function&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-07T21:01:14.647" LastActivityDate="2019-07-07T21:01:14.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13253" PostTypeId="4" CreationDate="2019-07-07T21:01:14.647" Score="0" Body="For questions related to the concept of loss (or cost) function in the context of machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-07T21:01:14.647" LastActivityDate="2019-07-07T21:01:14.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13254" PostTypeId="1" AcceptedAnswerId="13255" CreationDate="2019-07-07T22:16:09.313" Score="4" ViewCount="2030" Body="&lt;p&gt;According to a lecture about Reinforcement Learning, the concept of options allows searching the state space of an agent much faster. The lecture came from Nptel [1] (National Program on Technology Enhanced Learning) and it was hard to follow the course because many new terms were introduced in a short time. For me, the concept of options sounds a bit like skills [2], which are used for describing high-level actions as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are skills an improvement over options which includes the trajectory, or is both the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking for a certain reason. Normal deep reinforcement learning has the problem, that the agent comes very often to a dead end, for example in the Montezuma's Revenge played at the Atari emulator. And the options framework promises to overcome the issue. But the concept sounds a bit too esoteric, and apart from the Nptel lecture, nobody else has explained the idea. So, is it useful at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] NPTEL, Computer science and Engineering, Reinforcement Learning, Week 10 Hierarchical Reinforcement Learning, &lt;a href=&quot;https://nptel.ac.in/courses/106106143/57&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nptel.ac.in/courses/106106143/57&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Peters, Jan, and Stefan Schaal. &quot;Reinforcement learning of motor skills with policy gradients.&quot; Neural networks 21.4 (2008): 682-697.&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="2444" LastEditDate="2019-07-08T00:01:34.687" LastActivityDate="2019-07-08T00:31:44.633" Title="What are options in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;comparison&gt;&lt;semi-mdp&gt;&lt;hierarchical-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13255" PostTypeId="2" ParentId="13254" CreationDate="2019-07-07T23:49:01.197" Score="5" Body="&lt;p&gt;An &lt;strong&gt;option&lt;/strong&gt; is a generalization of the concept of action. The concept of an &lt;em&gt;option&lt;/em&gt; (or &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.14.2402&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;macro-action&lt;/a&gt;) was introduced in the context of reinforcement learning in the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370299000521&quot; rel=&quot;noreferrer&quot;&gt;Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning&lt;/a&gt; (1998) by Richard S. Sutton, Doina Precup and Satinder Singh, so that to capture the idea that certain actions are composed of other sub-actions. Section 2 of the mentioned paper formally defines the concept of an option (which is actually composed of a policy, a termination condition, and an initiation set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors of the mentioned paper give examples of options&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Examples of options include &lt;em&gt;picking up an object, going to lunch, and traveling to a distant city&lt;/em&gt;, as well as primitive actions such as &lt;em&gt;muscle twitches&lt;/em&gt; and &lt;em&gt;joint torques&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The option &lt;em&gt;picking up an object, going to lunch, and traveling to a distant city&lt;/em&gt; is composed of other sub-actions (e.g. &lt;em&gt;picking up an object&lt;/em&gt;), but is itself an action (or macro-action). A primitive action (e.g. &lt;em&gt;joint torques&lt;/em&gt;) is itself an option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A set of options defined over an MDP constitutes a &lt;a href=&quot;https://pdfs.semanticscholar.org/eb7f/25d8cecf31fae319cf94030966c88f5a8bb4.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;semi-Markov decision process (SMDP)&lt;/strong&gt;&lt;/a&gt;, which are MDPs where the time between actions is not constant but it is variable. In other words, a semi-MDP (SMDP) is an extension of the concept of MDP that is used to deal with problems where there are actions of different levels of abstraction.  For example, consider a footballer that needs to &lt;a href=&quot;https://en.wikipedia.org/wiki/Direct_free_kick&quot; rel=&quot;noreferrer&quot;&gt;take a freekick&lt;/a&gt;. The action &quot;take a freekick&quot; involves a sequence of other actions, like &quot;run towards the ball&quot;, &quot;look at the wall&quot;, etc. The action &quot;take a freekick&quot; takes a variable number of time steps (which depends on the other sub-actions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semi-MDPs are thus used to deal with such problems that involve actions of different levels of abstraction. &lt;strong&gt;Hierarchical reinforcement learning&lt;/strong&gt; (HRL) is a generalization (or extension) of reinforcement learning where the environment is modeled as a semi-MDP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curiously, certain models that have won the &lt;a href=&quot;https://www.robocup.org/&quot; rel=&quot;noreferrer&quot;&gt;RoboCup&lt;/a&gt; (the famous AI football) context are based on the concept of semi-MDPs, options and HRL. See e.g. &lt;a href=&quot;https://github.com/wrighteagle2d/wrighteaglebase&quot; rel=&quot;noreferrer&quot;&gt;WrightEagleBASE&lt;/a&gt;, which use the &lt;a href=&quot;https://aijunbai.github.io/publications/TIST15-Bai.pdf&quot; rel=&quot;noreferrer&quot;&gt;MAXQ-OP (MAXQ online planning) algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semi-MDPs can be converted to MDPs. The picture below (which is a screenshot of figure 1 of the mentioned paper that introduces the &quot;options framework&quot; in RL) illustrates the relationship between semi-MDPs and MDPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/03mVE.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/03mVE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The empty circles (in the middle) are options, while the black circles (at the top) are primitive actions (which are themselves options).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.142.8735&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement learning of motor skills with policy gradients&lt;/a&gt; mentioned in the question, apparently, the term skill is not formally defined. However, I suppose that skills can be represented as options.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-08T00:31:44.633" LastActivityDate="2019-07-08T00:31:44.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13256" PostTypeId="1" AcceptedAnswerId="13259" CreationDate="2019-07-08T09:39:01.317" Score="3" ViewCount="542" Body="&lt;p&gt;I plan to create a neural network using Python, Keras and TensorFlow. All the tutorials I have seen so far are concerned with image recognition. However, the goal of my program would be to take in 10+ inputs and calculate a binary output (true/false) instead. What would you suggest me to learn for this specific purpose?&lt;/p&gt;&#xA;" OwnerUserId="26948" LastEditorUserId="2444" LastEditDate="2019-07-08T10:08:47.577" LastActivityDate="2019-07-08T12:17:42.840" Title="Which loss function should I use for binary classification?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13257" PostTypeId="1" AcceptedAnswerId="13258" CreationDate="2019-07-08T10:29:58.843" Score="6" ViewCount="493" Body="&lt;p&gt;Is there any precedent for using a neuroevolution algorithm, like NEAT, as a way of getting to an initialization of weights for a network that can then be fine-tuned with gradient descent and back-propagation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if this may be a faster way of getting to a global minimum before starting a decent to a local using backpropagation with a large set of input parameters.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="2444" LastEditDate="2019-07-08T11:55:54.577" LastActivityDate="2019-07-08T16:28:03.660" Title="Can neuroevolution be combined with gradient descent?" Tags="&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;gradient-descent&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13258" PostTypeId="2" ParentId="13257" CreationDate="2019-07-08T11:51:54.167" Score="4" Body="&lt;p&gt;The paper &lt;a href=&quot;https://pdfs.semanticscholar.org/5071/d5f447a99f7e4d522f07f7752ca1435e13c9.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Comparison and Combination of Genetic and Gradient Descent Learning in Recurrent Neural Networks: An Application to Speech Phoneme Classification&lt;/a&gt; (2007), by Rohitash Chandra and Christian W. Omlin, uses genetic algorithms to train a recurrent neural network and then uses gradient descent to fine tune the trained model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1810.06773.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks&lt;/a&gt; (2018), by Xiaodong Cui, Wei Zhang, Zoltán Tüske and Michael Picheny, also combines evolutionary algorithms and gradient descent, but, in this case, they alternate between a gradient descent step and an evolution step. This is an example of a &lt;em&gt;evolutionary stochastic gradient descent&lt;/em&gt; (ESGD) method, as opposed to a &lt;a href=&quot;https://arxiv.org/pdf/1711.09846.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;population-based training (PBT)&lt;/a&gt; method, which uses only evolutionary algorithms to train neural networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-08T11:51:54.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13259" PostTypeId="2" ParentId="13256" CreationDate="2019-07-08T12:17:42.840" Score="1" Body="&lt;p&gt;There are several loss functions that you can use for binary classification. For example, you could use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;binary cross-entropy&lt;/strong&gt;&lt;/a&gt; or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hinge_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;hinge&lt;/strong&gt;&lt;/a&gt; loss functions. See, for example, the tutorials &lt;a href=&quot;https://machinelearningmastery.com/binary-classification-tutorial-with-the-keras-deep-learning-library/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Binary Classification Tutorial with the Keras Deep Learning Library&lt;/a&gt; (2016) and &lt;a href=&quot;https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Choose Loss Functions When Training Deep Learning Neural Networks&lt;/a&gt; (2019) by Jason Brownlee. Have also a look at Keras &lt;a href=&quot;https://keras.io/losses/&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation of its available loss functions&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-08T12:17:42.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13260" PostTypeId="1" CreationDate="2019-07-08T13:09:07.893" Score="2" ViewCount="445" Body="&lt;p&gt;I've compared videos manipulated with three different automated face manipulation methods: Deepfakes, Face2Face, and FaceSwap. Surprisingly, I found the output videos quite different: Deepfakes and Face2Face are more real than FaceSwap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the difference between them? (like the algorithm, the way they manipulate, etc)&lt;/p&gt;&#xA;" OwnerUserId="26955" LastEditorUserId="1671" LastEditDate="2019-07-10T21:08:53.330" LastActivityDate="2019-07-10T21:08:53.330" Title="What are the differences between Deepfakes, FaceSwap and Face2Face?" Tags="&lt;terminology&gt;&lt;comparison&gt;&lt;software-evaluation&gt;&lt;deepfakes&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13261" PostTypeId="1" AcceptedAnswerId="13262" CreationDate="2019-07-08T14:05:12.100" Score="11" ViewCount="6347" Body="&lt;p&gt;Let's consider this example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It's John's birthday, let's buy him a kite.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We humans most likely would say the kite is a birthday gift, if asked why it's being bought; and we refer to this reasoning as &lt;em&gt;common sense&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we need this in artificially intelligent agents? I think it could cause a plethora of problems, since a lot of our human errors are caused by these vague assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine an AI ignoring doing certain things because it assumes it has already been done by someone else (or another AI), using its common sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wouldn't that bring human errors into AI systems?&lt;/p&gt;&#xA;" OwnerUserId="26958" LastEditorUserId="2444" LastEditDate="2021-02-07T22:36:16.077" LastActivityDate="2021-02-07T22:36:16.077" Title="Why do we need common sense in AI?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;knowledge-representation&gt;&lt;commonsense-knowledge&gt;" AnswerCount="5" CommentCount="2" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13262" PostTypeId="2" ParentId="13261" CreationDate="2019-07-08T14:41:42.213" Score="18" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Commonsense_knowledge_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Commonsense knowledge&lt;/a&gt; is the collection of premises that everyone, in a certain context (hence common sense knowledge might be a function of the context), takes for granted. There would exist a lot of miscommunication between a human and an AI if the AI did not possess common sense knowledge. Therefore, commonsense knowledge is fundamental to &lt;a href=&quot;https://en.wikipedia.org/wiki/Human%E2%80%93computer_interaction&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;human-AI interaction&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are also premises that every human takes for granted independently of the country, culture, or, in general, context. For example, every human (almost since its birth) has a mechanism for reasoning about naive physics, such as space, time, and physical interactions. If an AI does not possess this knowledge, then it cannot perform the tasks that require this knowledge.&lt;/p&gt;&#xA;&lt;p&gt;Any task that requires a machine to have common sense knowledge (of an &lt;em&gt;average&lt;/em&gt; human) is believed to be &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.369.7112&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-complete&lt;/a&gt;, that is, it requires human-level (or general) intelligence. See section D of &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.369.7112&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-Complete, AI-Hard, or AI-Easy – Classification of Problems in AI&lt;/a&gt; (2012) by Roman V. Yampolskiy.&lt;/p&gt;&#xA;&lt;p&gt;Of course, the problems that arise while humans communicate because of different assumptions or premises might also arise between humans and AIs (that possess commonsense knowledge).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-07T22:32:53.230" LastActivityDate="2021-02-07T22:32:53.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13263" PostTypeId="2" ParentId="13261" CreationDate="2019-07-08T14:48:03.330" Score="10" Body="&lt;p&gt;We need this kind of common sense knowledge if we want to get computers to &lt;em&gt;understand&lt;/em&gt; human language. It's easy for a computer program to analyse the grammatical structure of the example you give, but in order to understand its meaning we need to know the possible contexts, which is what you refer to as &quot;common sense&quot; here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was emphasised a lot in Roger Schank &lt;em&gt;et al.&lt;/em&gt;'s work on computer understanding of stories, and lead to a lot of research into knowledge representation, scripts, plans, goals. One example from Schank's work is &lt;em&gt;Mary was hungry. She picked up a Michelin Guide.&lt;/em&gt; -- this seems like a non-sequitur: if you are hungry, why pick up a book? Until you realise that it is a restaurant guide, and that Mary is presumably planning to go to a restaurant to eat. If you know that going to a restaurant is a potential solution to the problem of being hungry, then you have no problem understanding this story fragment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any story needs common sense to be understood, because no story is completely explicit. Common things are &quot;understood&quot; and aren't explicitly mentioned. Stories relate to human experience, and a story that would make everything explicit would probably read like a computer program. You also need common sense to understand how characters in a story behave, and how they are affected by what is happening. Again, this is very subjective, but it is necessary. Some common sense might be generally applicable, other aspects of it won't be. It's a complex issue, which is why researchers have struggled with it for at least half a century of AI research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this would introduce &quot;human errors&quot; into an AI system. All this is very subjective and culture-specific. Going to a restaurant in the USA is different from going to one in France -- this is why going abroad can be a challenge. And my reading of a story will probably be different from yours. But if you want to simulate human intelligence, you cannot do that without potential human &quot;errors&quot;.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-07-08T14:48:03.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13264" PostTypeId="2" ParentId="13257" CreationDate="2019-07-08T15:13:57.097" Score="1" Body="&lt;p&gt;Yes it can be in addition to the papers that nbro linked to above uber's ai research team has a very interesting combination of sgd and neuroevolution which they have dubbed &quot;safe mutations&quot;. In the algorithm each genome undergoes a bit of sgd to improve its fitness before the speciation, elitism, and reproduction processes. I imagine this has an effect of searching for genomes which are well suited for sgd optimization, and in my opinion does really provide the best of both a worlds. Here is the link to the paper &lt;a href=&quot;https://arxiv.org/abs/1712.06563&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1712.06563&lt;/a&gt; . What I think would be a cool for this combination of the two would be its use in conjunction with the es-hyperneat/hyperneat neuroevolution algorithms in which a small genome cppn encodes large phenotype rnns using the rnns substrate (its structure represented with cartesian coordinates) as the cppns input. If a small amount sgd is used on the rnn's to improve fitness then what you end up with is a cppn is being evolved to encode very general rnn networks that can then be optimized to specific domains via sgd. I like this because then your neuroevolution doesnt occur on a massive rnn and you can create cppns that recognize the general problem you wish to solve if your clever with your fitness evaluation.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-07-08T16:28:03.660" LastActivityDate="2019-07-08T16:28:03.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13266" PostTypeId="1" AcceptedAnswerId="13267" CreationDate="2019-07-08T17:35:13.157" Score="6" ViewCount="501" Body="&lt;p&gt;I perfectly understand that CNN takes into account the local dependency of each pixel to the nearby pixels. In addition, CNNs are spatially invariant which means that they are able to detect the same feature anywhere in the image. These qualities are useful in image classification problems given the nature of the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does a vanilla neural net exactly falls short on these properties? Am I right in claiming that a vanilla neural net has to learn a given feature in every part of the image? This is different than how a CNN does it, which learns the feature once and then detects it anywhere in the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about local pixel dependancy? Why can't a vanilla neural network learn the local dependency by relating one pixel to its neighbors in the 1D input? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, is there more information present while training a CNN that are simply absent when training a normal NN? Or is a CNN just better at optimizing in the space of image classification problems? &lt;/p&gt;&#xA;" OwnerUserId="17582" LastEditorUserId="2444" LastEditDate="2019-07-08T19:46:21.897" LastActivityDate="2019-07-08T19:50:17.240" Title="Can a vanilla neural network theoretically achieve the same performance as CNN?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13267" PostTypeId="2" ParentId="13266" CreationDate="2019-07-08T18:38:55.230" Score="7" Body="&lt;p&gt;All CNNs can be represented as vanilla networks on the flattened image data. Just to do so, you would need &lt;strong&gt;A LOT&lt;/strong&gt; of parameters (most of which would be 0) for what CNNs do freely. You can think of a CNN as reusing a filter on a masked input (whichever receptive field it's looking at whatever point during the convolution) repetitively.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, fully connected layers use &lt;strong&gt;all&lt;/strong&gt; the information, so it can still learn spatial dependence as a CNN does, while CNNs for each neuron only look at a specific receptive field and will reuse that filter for all neurons in that channel. This constraint saves computation and allows wider and deeper models under some budget.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is nice because the hypothesis of why CNN's work are, is that at each point in the network we care about looking at localized features rather than global ones and that creating a composition of these makes it so even if each neuron only relates to a handful of neurons in the previous layer, the receptive field from the initial image can still be quite large if not the whole thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Take away:&lt;/strong&gt; &lt;em&gt;CNNs are an efficient implementation of a vanilla NN, given the locality constraint that each neuron only looks at a small localized subset of neurons from the previous layer&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-08T19:50:17.240" LastActivityDate="2019-07-08T19:50:17.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13268" PostTypeId="1" CreationDate="2019-07-08T22:13:26.890" Score="2" ViewCount="49" Body="&lt;p&gt;I want to build an AI that plays a simple android game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The game is just a one at a time object falling, some times at an angle. The AI needs to recognize the object and to decide whether to swipe left, swipe down, or click on it. The background is changing some times, but the object falling is always on top.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 44 different assets and I have the original full resolution PNG of the objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I approach this?&lt;/p&gt;&#xA;" OwnerUserId="26976" LastEditorUserId="2444" LastEditDate="2019-07-08T22:58:49.233" LastActivityDate="2019-07-08T22:58:49.233" Title="How should I build an AI that quickly detects falling game assets on screen?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13269" PostTypeId="2" ParentId="13261" CreationDate="2019-07-08T23:07:16.507" Score="2" Body="&lt;p&gt;I'll answer this question in several parts:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do AGI systems need to have common sense?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Humans in the wild reason and communicate using common sense more than they do with strict logic, you can see this by noting that it is easier to appeal to someone's emotion than logic. So any system that seeks to replicate human cognition (as in AGI) should also replicate this tendency to use common sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More simply put, we'd wish that our AGI system can speak to us in common sense language simply because that is what we understand best (otherwise we wouldn't understand our friendly AGI would we?). Obtuse theory and strict logic might technically be correct, but don't appeal to our understanding.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Isn't the goal of AGI the create the most cognitively advance system? Why should the &quot;most perfect&quot; AGI system need to deal with such imperfections and impreciseness present in common sense?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;First, it might only &lt;em&gt;appear&lt;/em&gt; to be the case that common sense logic is &quot;irrational&quot;. Perhaps there is a consistent mathematical way to model common sense such that all the subtleties of common sense are represented in a rigour fashion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the early study of Artificial Intelligence started in the study of cognitive science, where researchers tried to replicate &quot;algorithms of the mind&quot;, or more precisely: decidable procedures which replicated human thought. To that extent then, the study of AI isn't to create the &quot;most supreme cognitive agent&quot; but to merely replicate human thought/behavior. Once we can replicate human behavior we can perhaps try to create something super-human by giving it more computational power, but that is not guaranteed.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I still don't see why common sense is needed in AGI systems. Isn't AGI about being the most intelligent and powerful computational system? Why should it care or conform towards the limits of human understanding, which requires common sense?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Perhaps then you have a bit of a misaligned understanding of what AGI entails. AGI doesn't mean unbounded computational power (physically impossible due to physical constraints on computation such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Bremermann%27s_limit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bremermann's limit&lt;/a&gt;) or unbounded intelligence (perhaps physically impossible due to the prior constraint). It usually just means artificial &quot;general intelligence&quot;, general meaning broad and common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considerations about unbounded agents are studied in more detail in fields such as theoretical computer science (type theory I believe), decision theory, and perhaps even set theory, where we are able to pose questions about agents with unbounded computational power. We might say that there are questions even an AGI system with unbounded power can't answer due to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Halting Problem&lt;/a&gt;, but only if the assumptions on those fields map onto the structure of the given AGI, which might not be true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a better understanding of what AGI might entail and its goals, I might recommend two books: Artificial Intelligence: The Very Idea by John Haugeland for a more pragmatic approach (as pragmatic as AI-philosophy can be, and On the Origin of Objects by Brian Cantwell Smith for a more philosophically inclined approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a fun aside, the collection of Zen koan's: The Gateless Gate, includes the following passage: (quoted and edited from &lt;a href=&quot;https://en.wikipedia.org/wiki/Mu_(negative)#The_Mu-koan&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A monk asked Zhaozhou, a Chinese Zen master, &quot;Has a dog Buddha-nature or not?&quot; Zhaozhou answered, &quot;Wú&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Wú (無) translates to &quot;none&quot;, &quot;nonesuch&quot;, or &quot;nothing&quot;, which can be interpreted as to avoid answering either yes or no. This enlightened individual doesn't seek to strictly answer every question, but just to respond in a way that makes sense. It doesn't really matter as to wether the dog has Buddha-nature or not (whatever Buddha-nature means), so the master defaults to absolve the question rather than resolving it.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2019-07-08T23:07:16.507" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13271" PostTypeId="1" CreationDate="2019-07-09T02:59:42.087" Score="1" ViewCount="45" Body="&lt;p&gt;I have a simple neural network for a binary classification.&lt;/p&gt;&#xA;&lt;p&gt;Input features include:&#xA;&lt;strong&gt;age, sex, economic_situation, illness, disability, etc.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Output is simply 1 and 0&lt;/p&gt;&#xA;&lt;p&gt;I would like to order the features for each input from greatest to least impact it had on the classification.&lt;/p&gt;&#xA;&lt;p&gt;An example answer could look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Classification: 1&#xA;1) illness&#xA;2) economic_situation&#xA;3) disability&#xA;4) sex&#xA;5) age&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Another example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Classification: 0 &#xA;1) economic_situation&#xA;2) age&#xA;3) disability&#xA;4) sex&#xA;5) illness&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18881" LastEditorUserId="38892" LastEditDate="2020-07-29T16:58:53.940" LastActivityDate="2021-04-25T18:05:35.547" Title="Obtain the most important input data for binary classification on a neural network" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13272" PostTypeId="2" ParentId="13271" CreationDate="2019-07-09T03:17:44.193" Score="0" Body="&lt;p&gt;Two popular methods I’ve seen done:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1)&lt;/strong&gt; For each feature, remove it and run the model and see the impact it has on the result. The idea is that the larger the impact, the more pertinent it was to the result.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2)&lt;/strong&gt; Look at the gradients magnitude &lt;span class=&quot;math-container&quot;&gt;$|\nabla_f {y} |$&lt;/span&gt;. You can either look at the raw gradient or look at the &lt;a href=&quot;https://www.cs.toronto.edu/~guerzhoy/321/lec/W07/HowConvNetsSee.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;guided back-propagation&lt;/a&gt; which is just the back props  product rule, but you only look at when the nodes positively help trigger a neuron by taking only the positive gradients at each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There’s probably also more methods. Hope this helped.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-09T03:17:44.193" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13273" PostTypeId="1" CreationDate="2019-07-09T03:40:00.743" Score="1" ViewCount="51" Body="&lt;p&gt;Do you think it would be possible to train an AI in such a way as to mimic/simulate someone that is diagnosed as &quot;Special Needs&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why? Most diagnosis and treatments for people today are subjective, sure it's what a group of like-minded professionals has agreed upon as a valid hypothesis, but, at the same time, there is an absence of the absolute. Could we train an AI to become a &quot;special needs&quot; be a starting point in helping find better ways to unlock the potential and understanding of these differences?&lt;/p&gt;&#xA;" OwnerUserId="26982" LastEditorUserId="2444" LastEditDate="2019-07-09T08:59:03.893" LastActivityDate="2019-07-09T08:59:03.893" Title="Can an AI simulate someone that is diagnosed as &quot;Special Needs&quot;?" Tags="&lt;ai-design&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13274" PostTypeId="1" AcceptedAnswerId="13275" CreationDate="2019-07-09T07:17:33.867" Score="2" ViewCount="58" Body="&lt;p&gt;I've recently come across an amazing work for human pose estimation: &lt;a href=&quot;https://arxiv.org/pdf/1802.00434.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DensePose: Dense Human Pose Estimation In The Wild&lt;/a&gt; by Facebook. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this work, they have tackled the task of dense human pose estimation using discriminative trained models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do understand that &quot;correspondence&quot; means how well pixels in one image correspond to pixels in the second image (specifically, here - 2D to 3D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what does &quot;dense&quot; means in this case?&lt;/p&gt;&#xA;" OwnerUserId="26989" LastEditorUserId="2444" LastEditDate="2019-07-09T10:44:15.967" LastActivityDate="2019-07-09T10:44:15.967" Title="What is &quot;dense&quot; in DensePose?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13275" PostTypeId="2" ParentId="13274" CreationDate="2019-07-09T09:26:33.587" Score="0" Body="&lt;p&gt;In computer vision, the adjectives &lt;em&gt;dense&lt;/em&gt; and &lt;em&gt;sparse&lt;/em&gt; are used in a variety of tasks (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Optical_flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimal flow&lt;/a&gt;), but they are commonly used in the context of the &lt;a href=&quot;https://www.mia.uni-saarland.de/zimmer/phd-thesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;correspondence problem&lt;/a&gt;, which is the problem of finding a map (or correspondence) between pixels of two images (e.g. two successive frames of a video). In this context, these adjectives thus refer to the number of pixels of the image that are used to solve this specific task. A &lt;em&gt;dense correspondence&lt;/em&gt; is thus a correspondence between two images using all (or, at least, &lt;em&gt;many&lt;/em&gt;) pixels. In other words, a dense correspondence attempts to map all (or many) pixels of an image to all (or many) pixels of another image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of &lt;a href=&quot;https://arxiv.org/pdf/1802.00434.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DensePose&lt;/a&gt;, the correspondences are between RGB images and surface-based representation of the human body (the figures of the paper illustrates this).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-09T09:32:01.153" LastActivityDate="2019-07-09T09:32:01.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13276" PostTypeId="1" AcceptedAnswerId="13277" CreationDate="2019-07-09T09:46:23.913" Score="2" ViewCount="69" Body="&lt;p&gt;I would like to create a model, that will tell me if one type of object is in an image or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for example, I have a camera and I would like to see when one object gets into the shot.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Object detection: This could be an overkill, because I don't need to know the bounding box around. Also, this means that I would need to label a lot of images, and draw the bounding box to have train data (a lot of time)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Image classification: This doesn't solve the problem, because I don't know what else could not be an object. It would be impossible to train for 2 classes: object / not object. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My idea&lt;/strong&gt; is to have Autoencoder. Train it only on data with the object. Then, if Autoencoder produces a result with a high difference with the original, I detect it as an anomaly - no object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a good approach? Will I have a lot of trouble with different backgrounds?&lt;/p&gt;&#xA;" OwnerUserId="26993" LastEditorUserId="2444" LastEditDate="2019-07-10T20:59:04.330" LastActivityDate="2019-07-10T20:59:04.330" Title="How should I detect an object in a camera image?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;autoencoders&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13277" PostTypeId="2" ParentId="13276" CreationDate="2019-07-09T10:09:12.773" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is this a good approach? Will I have a lot of trouble with different backgrounds?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A lot will depend on the nature of the backgrounds you have, and how well they encode/decode by themselves without the object in frame. My gut feeling is that your system will have poor performance compared to a properly trained classifier, as the autoencoder will naturally have to get good at constructing background elements in order to score well, so unless your object is very consistently always in a similar place with radically different appearance to the background then the autoencoder will get good reconstructions of background-only frames, and your anomaly detection would need to be set too sensitive. That would cause the detector to fail to spot the object when in frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a catch-all answer to &quot;how well will my idea work&quot; with ML projects. You should try it and see. Data science is essentially an empirical approach, and building practical models is an engineering discipline where testing is a core part of the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to test your model, you are going to need a lot of images of just background, including all the kinds of background that you expect the system to be used. Which begs the question, why not collect a range of suitable images without your target object, and use them as the second class?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This doesn't solve the problem, because I don't know what else could not be an object. It would be impossible to train for 2 classes: object / not object.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Actually, it is easy. Just collect suitable images from locations where you expect your system to be used, without the object in frame. These are your &quot;Not Object&quot; class. A &quot;Not Object&quot; does not have to include some substitute foreground object. Although I recommend that you do have some images like this to prevent accidentally creating an &quot;object is in the foreground&quot; detector*. The primary goal should be to be collect data that matches how the model will be used once deployed. That will depend a lot on how much control and consistency you get over the production cameras where the trained detector will be put to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would do this, then train a standard binary classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are still interested in how well your auto-encoder idea could work, you then have plenty of test data to evaluate it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*  This is one area where your auto-encoder idea &lt;em&gt;might&lt;/em&gt; do better than a classifier - reconstructing unfamiliar foreground objects should be hard for it leading to high error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is too difficult to tell in advance whether this effect is strong enough to make your auto-encoder approach better than a classifier.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-07-09T10:51:56.947" LastActivityDate="2019-07-09T10:51:56.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13278" PostTypeId="1" CreationDate="2019-07-09T11:26:14.867" Score="2" ViewCount="41" Body="&lt;p&gt;I am looking to extract causal relations between entities like Drug and Adverse Effect in a document. Are there any proven NLP or AI techniques to handle the same.  Also are there ways to handle cases where the 2 entities may not necessarily co-occur in the same sentence. &lt;/p&gt;&#xA;" OwnerUserId="26115" LastEditorUserId="2444" LastEditDate="2019-07-09T11:45:44.337" LastActivityDate="2019-07-09T11:45:44.337" Title="Models to extract Causal Relationship between entities in a document using Natural Language Processing techniques" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;causation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13279" PostTypeId="2" ParentId="13261" CreationDate="2019-07-09T12:05:25.587" Score="1" Body="&lt;p&gt;Is this common sense, or is this natural language understanding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's been said that natural language understanding is one of the hardest AI tasks. This is one of the examples showing why. The first part of the sentence is related to the second part, that how sentences work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the relevant question is  &lt;em&gt;how&lt;/em&gt; the two parts are related. There are a few standard relations that we encounter, for instance a temporal order. In this specific example, the nature of the relation is closer to a cause-and-effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You see this effect when we insert a word to make this relation explicit:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It's John's birthday, &lt;em&gt;so&lt;/em&gt; let's buy him a kite.&#xA;  or &#xA;  Let's buy John a kite, &lt;em&gt;because&lt;/em&gt; it's his birthday.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a technique for humans to make these implicit relations explicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as curiousdannii notes, you also need the cultural knowledge to understand how a birthdays can be a cause for a present. No amount of common sense helps with that.&lt;/p&gt;&#xA;" OwnerUserId="16378" LastActivityDate="2019-07-09T12:05:25.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13280" PostTypeId="1" CreationDate="2019-07-09T12:52:19.683" Score="1" ViewCount="64" Body="&lt;p&gt;I have an idea for a new mobile app. Here is what I want to accomplish using AI;&lt;br&gt;&#xA;I want to get an &lt;em&gt;image (png format)&lt;/em&gt;, (&lt;em&gt;maybe just byte data too&lt;/em&gt;), from my application (&lt;em&gt;I'm developing with Unity3D/C#&lt;/em&gt;), send this data to AI application; get modified image from ML app and send it back to my app.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What AI is going to do with img?&lt;/strong&gt;&lt;br&gt;&#xA;Imagine you are the user of my app; you are going to draw a picture on your phone.&#xA;Picture will be simple, &lt;em&gt;like a seagull illustrated as an &lt;strong&gt;'M'&lt;/strong&gt; letter&lt;/em&gt;.&lt;br&gt;&#xA;AI program will get your drawing (&lt;em&gt;'M' in this case&lt;/em&gt;), check pixels to give a meaning to &lt;em&gt;'M'&lt;/em&gt;; then draw a more complex drawing that is themed around that &lt;em&gt;M seagull&lt;/em&gt;.&lt;br&gt;&#xA;&lt;em&gt;(Like a drawing with an ocean made by Pixel Art, with rain clouds from Van Gogh and &lt;strong&gt;seagull&lt;/strong&gt; is painted as a surreal bird...)&lt;/em&gt;&lt;br&gt;&#xA;&lt;hr&gt;&#xA;&lt;strong&gt;My general idea about building this AI system...&lt;/strong&gt;&lt;br&gt;&#xA;I'm not sure how to build this AI system, because I can't understand AI/ML completely. How it works on machine, how to implement it using computer, how to write an algorithm, how pre-made libraries like &lt;em&gt;Tensorflow&lt;/em&gt; works... But I'm in a phase of my life that I need to use my time well, and want to build this app while learning.&lt;br&gt;&#xA;I think, I can build an side app to use for &lt;em&gt;analyzing&lt;/em&gt; and &lt;em&gt;modifying&lt;/em&gt; image I get from user. Right know I can write in C, C# and learning JavaScript. I learned Python too (and a few others), but I'm not comfortable using it. (&lt;em&gt;I hate Python&lt;/em&gt;). And didn't write a good program in any other language...&lt;br&gt;&#xA;I thought, I can use &lt;a href=&quot;https://www.tensorflow.org/js?hl=tr&quot; rel=&quot;nofollow noreferrer&quot;&gt;JavaScript&lt;/a&gt; or &lt;a href=&quot;https://www.tensorflow.org/lite/guide/android?hl=tr&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java || C++&lt;/a&gt; but &lt;em&gt;honestly&lt;/em&gt; I don't know how to start and which steps to take. Also after gaining some success I &lt;em&gt;will&lt;/em&gt; want to port app to &lt;strong&gt;iOS&lt;/strong&gt; too... Maybe that can wait...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you give me some examples, guidelines and advices? &lt;em&gt;How to start doing this &lt;strong&gt;while deciding; what is best approach, performance and development-time wise&lt;/strong&gt;?&lt;/em&gt;&lt;br&gt;&#xA;&lt;em&gt;And,&lt;/em&gt; &lt;strong&gt;Is my approach to the problem is a good one?&lt;/strong&gt; &lt;em&gt;&lt;strong&gt;Can you come up with a better solution for my idea that you can tell me to point me in another direction?&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="25858" LastEditorUserId="25858" LastEditDate="2019-07-09T16:21:26.097" LastActivityDate="2019-07-09T16:21:26.097" Title="How can I develop this ML/AI system that I want to use in my new mobile app?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13282" PostTypeId="2" ParentId="6038" CreationDate="2019-07-09T15:14:10.980" Score="1" Body="&lt;p&gt;Liquid State Machines are used in the field of Neuroscience. What you have used is a variant of LSM called &lt;a href=&quot;http://www.scholarpedia.org/article/Echo_state_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Echo State Network&lt;/a&gt; (ESN) used in the field of Machine Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ESN's are pretty simple and superfast compared to normal ML paradigms like Feed Forward NN's or RNN's. ESN's are based on a relatively new paradigm called &lt;a href=&quot;https://en.wikipedia.org/wiki/Reservoir_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reservoir Computing&lt;/a&gt;. ESN's are mainly used in Sequential ML problems, and it is used to overcome the problems RNN face like vanisihing gradient, exploding gradient, long training times, etc. The basic idea is that you have a reservoir of Neurons (basically a Neural Net with fixed and non-trainable weights, but it can be varied to complex structures) which will echo an input into some form based on which a simple Linear Regression model (or maybe a complex NN) will be trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea of an ESN is very simple, note it works on time series data and the conventions are similar to a RNN:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Just like RNN you have a hidden state &lt;span class=&quot;math-container&quot;&gt;$x(n)$&lt;/span&gt; and an input &lt;span class=&quot;math-container&quot;&gt;$u(n)$&lt;/span&gt; and some randomly initialized &lt;strong&gt;non-trainable&lt;/strong&gt; weights are assigned to them (the weights with which you dot product these 2 terms).&#xA;&lt;span class=&quot;math-container&quot;&gt;$$a(n) = W_{in}[1:u(n)] + W[x(n-1)]$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The next &lt;span class=&quot;math-container&quot;&gt;$x(n)$&lt;/span&gt; is produced as some linear function of non-linear transformation of &lt;span class=&quot;math-container&quot;&gt;$x(n-1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u(n)$&lt;/span&gt; multiplied by their respective weights and &lt;span class=&quot;math-container&quot;&gt;$x(n-1)$&lt;/span&gt;.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\tilde x(n) = tanh(a(n))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ x(n) = (1-\alpha).x(n-1) + \alpha\tilde x(n)$$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The prediction target &lt;span class=&quot;math-container&quot;&gt;$y(n)$&lt;/span&gt; is done simply by multiplying &lt;span class=&quot;math-container&quot;&gt;$x(n)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u(n)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$W_{out}$&lt;/span&gt; which are &lt;strong&gt;trainable&lt;/strong&gt; and the only trainable weights.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y(n) = W_{out}[1:u(n):x(n)]$$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now you apply a suitable Loss Function on your predicted &lt;span class=&quot;math-container&quot;&gt;$y(n)$&lt;/span&gt; vs actual target &lt;span class=&quot;math-container&quot;&gt;$t(n)$&lt;/span&gt; to train weights &lt;span class=&quot;math-container&quot;&gt;$W_{out}$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(&lt;span class=&quot;math-container&quot;&gt;$:$&lt;/span&gt; means concatenation and follows the usual RNN concatenation rules)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a very simple overview of ESN's and it works surprisingly well for Sequential prediction tasks. The main &lt;strong&gt;idea&lt;/strong&gt; and &lt;strong&gt;intuition&lt;/strong&gt; behind the working is that the inputs are &lt;strong&gt;echoed&lt;/strong&gt; in the reservoir of untrainable weights which basically means that the input is converted into a certain 'form' due to the random initialisation of weights and this 'form' trains the &lt;span class=&quot;math-container&quot;&gt;$W_{out}$&lt;/span&gt; (the first few training &lt;span class=&quot;math-container&quot;&gt;$u(n)$&lt;/span&gt; are run through the network without training for the network to assume a starting state where the input has already been echoed in the network and is now coming finally coming out). There are a few mathematical details for the ESN to train and not blow up by huge oscillations and can be found here in this simple intuitive &lt;a href=&quot;https://www.semanticscholar.org/paper/A-Practical-Guide-to-Applying-Echo-State-Networks-Lukosevicius/11bb0941b1f6088783e26d7f9603789ee1db7faa&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of difference of purpose, a few are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Provides superfast training times.&lt;/li&gt;&#xA;&lt;li&gt;Works on Sequential data mainly.&lt;/li&gt;&#xA;&lt;li&gt;It is being used as a side network to initialise RNN weights which apparently improves the performance.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although these networks have some very hard to tune hyperparameters, owing to its simple and superfast training one can easily check a lot of hyperparameters. The network also performs surprisingly well and it has resulted in Reservoir Computing being actually researched very actively. Check this very old &lt;a href=&quot;https://ai.stackexchange.com/questions/6091/what-is-the-significance-of-this-stanford-university-financial-market-time-seri&quot;&gt;question&lt;/a&gt; too.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-11T22:55:43.447" LastActivityDate="2019-07-11T22:55:43.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13283" PostTypeId="1" CreationDate="2019-07-09T17:10:18.493" Score="1" ViewCount="69" Body="&lt;p&gt;I’m not really sure which machine learning approach is best for my problem at hand. I work in an engineering company that designs and builds different kinds of ships. In my particular job, I collect the individual weight of items on these vessels. The weight and there location is important because it is used to ensure the vessel in question can float in a balanced manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a large corpus of historical data on hand that lists the items on the vessel, there attributes, the weight for these items and where the weight came from (documentation), or the source. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for example, let’s say I have the following information:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; ITEM  |         ATTRIBUTES            | WEIGHT  |WEIGHT SOURCE&#xA;Valve  |Size: 1 inch |Type: Ball Valve |2 lbs.   |Database 1&#xA;Elbow  |Size: 2 inch |Type: Reducing   |1 lb.    |Database 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have to comb through many systems on these vessels and find the proper documentation or engineering drawings that lists the weight for the item in question. It usually starts by investigating the item and its attributes and then looking in a number of databases for the weight documentation. This takes a long time, as there is no organization or criteria as to what database has what. You just have to start randomly searching them and hope you find what you need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well I now have a large corpus of real world data that lists thousands of items, there attributes, there weight and most importantly the source of the documentation (Database 1, 2, 3 etc.). I’m wondering if there is any correlation between an item, its attributes and its weight location (database). This is where machine learning comes in. What I’d like to do is use machine learning to help find the weight location more quickly. Ideally it would be nice if it could analyze a batch of information and then provide recommendations on which databases to search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thoughts are that this is a classification problem, and maybe a CNN would be helpful here. If that is the case, I have over 100 categories in my dataset.&#xA;I actually went ahead and programmed a simple feed forward neural network using the following resources: &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/&lt;/a&gt; I attempted to use this network to solve the above problem, but so far I have had no success. I’m in over my head here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don’t expect it to be correct 100% of the time. Even if it had an 80% success rate that would be awesome. So my question is this;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of neural network do I need to accomplish this? &lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="2444" LastEditDate="2019-07-09T22:07:53.573" LastActivityDate="2019-07-09T22:07:53.573" Title="Is this a classification problem?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;&lt;data-mining&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13284" PostTypeId="1" CreationDate="2019-07-09T18:10:01.303" Score="2" ViewCount="55" Body="&lt;p&gt;I've written &lt;a href=&quot;https://rjnienaber.github.io/heroes_helper/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an application&lt;/a&gt; to help players pick the optimal heroes during the draft phase of the &lt;a href=&quot;https://heroesofthestorm.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heroes of the Storm&lt;/a&gt; MOBA. It can be daunting to pick from 80+ characters that have synergies/counters to other characters, strong/weak maps, etc. The app attempts to pick the optimal composition using a genetic algorithm (GA) based on various sources of information on these heroes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I've realized is that not all sources of information are created equal. At the moment I'm giving &lt;a href=&quot;https://github.com/rjnienaber/heroes_helper/blob/bf0d5248c40ceeed9e0568192ffe305548e458d3/draft_picker/src/ui/settings_component.jsx#L7&quot; rel=&quot;nofollow noreferrer&quot;&gt;all sources roughly equal importance&lt;/a&gt; in the fitness function but as I add other sources, I think it's going to be necessary to be more discerning about them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like the right way to do this would be to use a single layer neural network where the weights of the synapses represent the weights in the fitness function. I could use matches played at a high-level (e.g. from &lt;a href=&quot;https://masterleague.net/matches/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MasterLeague.net&lt;/a&gt;) to form the training and test sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this sound like a viable approach or am I missing something simpler?  Is the idea of the using a GA even the correct way to approach this problem?&lt;/p&gt;&#xA;" OwnerUserId="27010" LastActivityDate="2019-07-09T18:10:01.303" Title="Is a neural network the correct approach to optimising a fitness function in a genetic algorithm?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;fitness-functions&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13285" PostTypeId="2" ParentId="13232" CreationDate="2019-07-09T19:49:42.640" Score="4" Body="&lt;p&gt;There seem to be two different ideas in this question here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What's the impact / importance of our choice for reward values?&lt;/li&gt;&#xA;&lt;li&gt;What's the impact / importance of our choice for initial value estimates (how do we initialise our table of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values in the case of a simple, tabular RL algorithm like Sarsa or &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The reward values are typically assumed to be a part of the problem definition - something we shouldn't modify if we're using an existing problem definition as a benchmark. But if we're in charge of defining the problem ourselves, we can of course also pick the reward values. Modifying them may indeed have a huge impact on the speed with which RL algorithms are able to learn a task - but it may also intrinsically changes the task, it changes the objective of the problem, it may change which policies are optimal.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As for initialisation of our table of value approximations: by default, we normally assume an all-&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; initialisation. However, it is a fairly common trick (in tabular RL algorithms, without function approximation) to &lt;strong&gt;initialise value estimates optimistically&lt;/strong&gt;; pick higher initial &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; value estimates than are likely (or even pick values higher than a known upper bound on what the true value possibly could be). This is often beneficial - also in large gridworlds with sparse rewards (e.g. a single distant goal somewhere) and negative rewards (i.e. costs) incurred for every step taken - &lt;strong&gt;because it incentivises exploration of state-action pairs that have not yet been tried&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have your gridworld with negative rewards associated with every time step, and the optimal policy being one that takes you to a distant goal as soon as possible. If all &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; are initialised to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (or worse, to negative values), your agent may quickly learn that everything it does is equally bad anyway, and get stuck near the starting position. If all &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values are initialised optimistically (to large, positive values), your agent during the learning process will still have optimistic expectations of what it can achieve if it just tries to navigate to unexplored parts of the state-action space.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-09T19:49:42.640" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13286" PostTypeId="5" CreationDate="2019-07-09T22:32:10.043" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;http://intelligence.org/files/AIPosNegFactor.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence as a Positive and Negative Factor in Global Risk&lt;/a&gt; (2008), by Eliezer Yudkowsky, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Friendly_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Friendly_artificial_intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-09T22:32:10.043" LastActivityDate="2019-07-09T22:32:10.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13287" PostTypeId="4" CreationDate="2019-07-09T22:32:10.043" Score="0" Body="For questions related to the concept of friendly artificial intelligence (FAI), which is a hypothetical AI that possesses the desire not to harm humans. The expression &quot;friendly AI&quot; was popularised by Eliezer Yudkowsky and it is mentioned in his article &quot;Artificial Intelligence as a Positive and&#xD;&#xA;Negative Factor in Global Risk&quot; (2008)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-09T22:32:10.043" LastActivityDate="2019-07-09T22:32:10.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13288" PostTypeId="1" AcceptedAnswerId="13290" CreationDate="2019-07-10T03:27:03.690" Score="0" ViewCount="78" Body="&lt;p&gt;As simple as that. Is there any scenario where the error might &lt;em&gt;increase&lt;/em&gt;, if only by a tiny amount, when using SGD (no momentum)?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-07-10T05:33:59.047" Title="Is it possible with stochastic gradient descent for the error to increase?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13289" PostTypeId="1" AcceptedAnswerId="13293" CreationDate="2019-07-10T03:29:44.830" Score="51" ViewCount="13463" Body="&lt;p&gt;Imagine you show a neural network a picture of a lion 100 times and label with &quot;dangerous&quot;, so it learns that lions are dangerous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine that previously you have shown it millions of images of lions and alternatively labeled it as &quot;dangerous&quot; and &quot;not dangerous&quot;, such that the probability of a lion being dangerous is 50%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But those last 100 times has pushed the neural network into being very positive about regarding the lion as &quot;dangerous&quot;, thus ignoring the last million lessons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it seems  there is a flaw in neural networks, in that they can change their mind too quickly based on recent evidence. Especially if that previous evidence was in the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a neural network model that keeps track of how much evidence it has seen? (Or would this be equivalent to letting the learning rate decrease by &lt;span class=&quot;math-container&quot;&gt;$1/T$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is the number of trials?)&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="1671" LastEditDate="2019-07-12T00:33:15.003" LastActivityDate="2020-06-27T00:59:38.967" Title="Are neural networks prone to catastrophic forgetting?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;theory&gt;&lt;incremental-learning&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="27" ContentLicense="CC BY-SA 4.0" />
  <row Id="13290" PostTypeId="2" ParentId="13288" CreationDate="2019-07-10T05:33:59.047" Score="2" Body="&lt;p&gt;Yes. Not only that, but error is highly noisy, prone to big spikes and sometimes quite long period of increase before decrease again or stabilize. Often it's even impossible to understand error plot without passing it through smoothing filter, so noisy it is. Specific depend on the problem of cause. It's not only for SGD but for any optimizer.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-07-10T05:33:59.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13291" PostTypeId="2" ParentId="13289" CreationDate="2019-07-10T08:29:04.967" Score="21" Body="&lt;p&gt;Yes, the problem of &lt;em&gt;forgetting&lt;/em&gt; older training examples is a characteristic of Neural Networks. I wouldn't call it a &quot;flaw&quot; though because it helps them be more adaptive and allows for interesting applications such as transfer learning (if a network remembered old training too well, fine tuning it to new data would be meaningless).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice what you want to do is to mix the training examples for &lt;em&gt;dangerous&lt;/em&gt; and &lt;em&gt;not dangerous&lt;/em&gt; so that it doesn't see one category in the beginning and one at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A standard training procedure would work like this:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for e in epochs:&#xA;    shuffle dataset&#xA;    for x_batch, y_batch in dataset:&#xA;        train neural_network on x_batxh, y_batch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the shuffle at every epoch guarantees that the network won't see the same training examples in the same order every epoch and that the classes will be mixed&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to answer your question, yes decreasing the learning rate would make the network less prone to &lt;em&gt;forgetting&lt;/em&gt; its previous training, but how would this work in a non-online setting? In order for a network to converge it needs multiple epochs of training (i.e. seeing each sample in the dataset many times).&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-07-10T08:29:04.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13293" PostTypeId="2" ParentId="13289" CreationDate="2019-07-10T10:14:52.953" Score="62" Body="&lt;p&gt;Yes, indeed, neural networks are very prone to &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0079742108605368?via%3Dihub&quot; rel=&quot;noreferrer&quot;&gt;catastrophic forgetting (or interference)&lt;/a&gt;. Currently, this problem is often ignored because neural networks are mainly trained offline (sometimes called &lt;em&gt;batch training&lt;/em&gt;), where this problem does not often arise, and not &lt;a href=&quot;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf&quot; rel=&quot;noreferrer&quot;&gt;online or incrementally&lt;/a&gt;, which is &lt;a href=&quot;https://nips.cc/Conferences/2018/Schedule?showEvent=10910&quot; rel=&quot;noreferrer&quot;&gt;fundamental to the development of &lt;em&gt;artificial general intelligence&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are some people that work on &lt;em&gt;continual lifelong learning&lt;/em&gt; in neural networks, which attempts to adapt neural networks to continual lifelong learning, which is the ability of a model to learn from a stream of data continually, so that they do not completely forget previously acquired knowledge while learning new information. See, for example, the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608019300231&quot; rel=&quot;noreferrer&quot;&gt;Continual lifelong learning with neural networks: A review&lt;/a&gt; (2019), by German I. Parisi et al., which summarises the problems and existing solutions related to catastrophic forgetting of neural networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-27T00:59:38.967" LastActivityDate="2020-06-27T00:59:38.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13294" PostTypeId="5" CreationDate="2019-07-10T10:25:58.103" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-10T10:25:58.103" LastActivityDate="2019-07-10T10:25:58.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13295" PostTypeId="4" CreationDate="2019-07-10T10:25:58.103" Score="0" Body="For questions related to the concept of catastrophic forgetting (or, also called, catastrophic interference), which is the problem of forgetting previously acquired information (or the ability to solve certain tasks) while learning new information (in an online or incremental fashion) that certain machine learning models (in particular, neural networks) face." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-15T23:06:05.697" LastActivityDate="2019-07-15T23:06:05.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13296" PostTypeId="2" ParentId="11640" CreationDate="2019-07-10T10:41:02.730" Score="5" Body="&lt;p&gt;You need to read this 2020 paper by Deepmind:&#xA;&lt;a href=&quot;http://acsweb.ucsd.edu/%7Ewfedus/pdf/replay.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Revisiting Fundamentals of Experience Replay&amp;quot;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, to add to the answer by @nbro&lt;/p&gt;&#xA;&lt;p&gt;Assume you implement experience replay as a buffer where the newest memory is stored instead of the oldest. Then, if your buffer contains 100k entries, any memory will remain there for exactly 100k iterations.&lt;/p&gt;&#xA;&lt;p&gt;Such a buffer is simply a way to &amp;quot;see&amp;quot; what was up to 100k iterations ago.&#xA;After the first 100k iterations you fill the buffer and begin &amp;quot;moving&amp;quot; it, much like a sliding window, by inserting new memories instead of the oldest.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The size of the buffer (relative to the total number of iterations you plan to ever train with) depends on &amp;quot;how much you believe your network architecture is susceptible to &lt;a href=&quot;https://ai.stackexchange.com/a/13293/27042&quot;&gt;catastrophic forgetting&lt;/a&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;A tiny buffer might force your network to only care about what it saw recently.&lt;/p&gt;&#xA;&lt;p&gt;But an excessively large buffer might take a long time to &amp;quot;become refreshed&amp;quot; with good trajectories, when they finally start to be discovered. So the network would be like a university student whose book shelf is diluted with first-grade school books.&lt;/p&gt;&#xA;&lt;p&gt;The student might have already decided that he/she wishes to become a programmer, so re-reading those primary school books has little benefit (time could have been spent more productively on programming literature) + it takes a long time to replace those with relevant university books.&lt;/p&gt;&#xA;" OwnerUserId="27042" LastEditorUserId="27042" LastEditDate="2020-10-16T23:28:42.070" LastActivityDate="2020-10-16T23:28:42.070" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13297" PostTypeId="1" CreationDate="2019-07-10T12:29:47.940" Score="1" ViewCount="33" Body="&lt;p&gt;Due to the fast-growing applications of AI technologies applied to &lt;em&gt;vehicle re-identification&lt;/em&gt; tasks, there have already been hot contests, such as the Nvidia AI challenge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms or models are really adopted in commercial vehicle re-identification tasks, effective and reliable, nowadays?&lt;/p&gt;&#xA;" OwnerUserId="14948" LastEditorUserId="2444" LastEditDate="2019-07-10T20:50:48.183" LastActivityDate="2019-07-10T20:50:48.183" Title="What models and algorithms are used in commercial vehicle re-identification tasks?" Tags="&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13298" PostTypeId="2" ParentId="13261" CreationDate="2019-07-10T13:37:47.020" Score="2" Body="&lt;p&gt;Perhaps it would help to give an example of what can go wrong without common sense: At the start of the novel &quot;The Two Faces of Tomorrow&quot; by James Hogan, a construction supervisor on the Moon files a request with an automated system, asking that a particular large piece of construction equipment be delivered to his site as soon as possible.  The system replies that it will arrive in twenty minutes.  Twenty minutes later, the supervisor is killed as the equipment crashes into his construction site.  The system had determined that the fastest way to deliver the equipment to that site was to mount it on a mass-driver and launch it at the site.  Had the system in question been given common sense, it would have inferred additional unstated constraints on the query, such as 'the equipment should arrive intact', 'the arrival of the equipment should not cause damage or loss of life', and so on.  (the rest of the novel describes an experiment designed to produce a new system with common sense)&lt;/p&gt;&#xA;" OwnerUserId="27045" LastActivityDate="2019-07-10T13:37:47.020" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13299" PostTypeId="1" CreationDate="2019-07-10T15:35:25.937" Score="1" ViewCount="80" Body="&lt;p&gt;If I see a hundred elephants and fifty of them are grey I'd say the probability of an elephant being grey is 50%. And my certainty of that probability is high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I see two elephants and one of them is grey. Still the probability is 50%. But my certaintity of this is low.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any AI models where not only the probability is given by the AI but it's certainty is also?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Certainty&quot; might be thought of as the probability that the probability is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could go up more levels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any advantage in doing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way I can envisage this working is instead of a weight, the NN stores two integers &lt;span class=&quot;math-container&quot;&gt;$(P,N)$&lt;/span&gt; which represent positive and negative evidence and the weight is given by &lt;span class=&quot;math-container&quot;&gt;$P/(P+N)$&lt;/span&gt;. And each iteration &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; can only be incremented by 1. &lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="26958" LastEditDate="2019-07-11T21:32:21.497" LastActivityDate="2019-07-11T21:32:21.497" Title="Is there an AI model with &quot;certainty&quot; built in?" Tags="&lt;neural-networks&gt;&lt;probability&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13300" PostTypeId="2" ParentId="13299" CreationDate="2019-07-10T16:38:38.030" Score="1" Body="&lt;p&gt;You make a valid point, vanilla neural networks cannot give you more than a point estimate of class confidence. If one wanted to actually gain an idea of variance, you need a framework that allows such a mechanism.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A popular methodology to this is Bayesian modeling. In other words given some data, &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;, you want to create some form of descriminative model &lt;span class=&quot;math-container&quot;&gt;$p(y|x;\theta)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; denotes the parameters that are random variables. This is unline NN's where they also learn some descriminative model &lt;span class=&quot;math-container&quot;&gt;$p(y|x;\theta)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are fixed. This difference is key to your goal, because if &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is fixed, &lt;span class=&quot;math-container&quot;&gt;$Var(Y|X) = 0$&lt;/span&gt; since any time you put in the same &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; into the network, itll always come out the same. On the other hand a bayesian model will have a variance, &lt;span class=&quot;math-container&quot;&gt;$Var(Y|X) = E[(Y - E[Y|X])^2 | X]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(Y - E[Y|X])^2 | X$&lt;/span&gt;  is no longer gauranteed to be 0 and can be empirically measured through some MC method (or analytically based on your model).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that your idea is on the right track, that the more data points you have, the more confident your estimates will be, but the only issue is that just considering &lt;span class=&quot;math-container&quot;&gt;$\frac{P}{(P+N)}$&lt;/span&gt; in a neural network will be just a heuristic and is difficult to quantify its association to the variance.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this may fit your fancy: &lt;a href=&quot;https://medium.com/neuralspace/bayesian-neural-network-series-post-1-need-for-bayesian-networks-e209e66b70b2&quot; rel=&quot;nofollow noreferrer&quot;&gt;high level overview of bayesian network blog post&lt;/a&gt;. This is a high level overview of using a neural netowrk paradigm but including uncertainty in the weights. Theres tons of tricks out there to make it so you can still even train these with gradient descent and such.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-10T16:38:38.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13301" PostTypeId="1" CreationDate="2019-07-10T17:05:59.593" Score="1" ViewCount="19" Body="&lt;p&gt;I would like to implement various AI-estimators for quantile estimation for a regression problem. It would be necessary to have non-crossing quantiles, that is larger quantiles would correspond to higher prediction values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My objective is to have a multidimensional prediction vector as an output in the estimation, each dimension corresponding to a specific quantile. Maybe I would have to define a custom loss function, as well, for that purpose. I would like to try different methods such as deeplearning or gradient boosting or random forests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone having an idea how to build such AI-estimators? My preferred library choice would be scikit-learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone give me an idea how to do this?&lt;/p&gt;&#xA;" OwnerUserId="23784" LastActivityDate="2019-07-10T17:05:59.593" Title="Scikit-Learn: monotoneous quantile estimation" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;data-science&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13302" PostTypeId="2" ParentId="5769" CreationDate="2019-07-10T19:19:26.140" Score="0" Body="&lt;p&gt;For anyone trying to understand how convolutions are calculated, here is a useful code snippet in Pytorch:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;batch_size = 1&#xA;height = 3 &#xA;width = 3&#xA;conv1_in_channels = 2&#xA;conv1_out_channels = 2&#xA;conv2_out_channels = 2&#xA;kernel_size = 2&#xA;# (N, C_in, H, W) is shape of all tensors. (batch_size, channels, height, width)&#xA;input = torch.Tensor(np.arange(0, batch_size*height*width*in_channels).reshape(batch_size, in_channels, height, width))&#xA;conv1 = nn.Conv2d(in_channels, conv1_out_channels, kernel_size, bias=False) # no bias to make calculations easier&#xA;# set the weights of the convolutions to make the convolutions easier to follow&#xA;nn.init.constant_(conv1.weight[0][0], 0.25)&#xA;nn.init.constant_(conv1.weight[0][1], 0.5)&#xA;nn.init.constant_(conv1.weight[1][0], 1) &#xA;nn.init.constant_(conv1.weight[1][1], 2) &#xA;out1 = conv1(input) # compute the convolution&#xA;&#xA;conv2 = nn.Conv2d(conv1_out_channels, conv2_out_channels, kernel_size, bias=False)&#xA;nn.init.constant_(conv2.weight[0][0], 0.25)&#xA;nn.init.constant_(conv2.weight[0][1], 0.5)&#xA;nn.init.constant_(conv2.weight[1][0], 1) &#xA;nn.init.constant_(conv2.weight[1][1], 2) &#xA;out2 = conv2(out1) # compute the convolution&#xA;&#xA;for tensor, name in zip([input, conv1.weight, out1, conv2.weight, out2], ['input', 'conv1', 'out1', 'conv2', 'out2']):&#xA;    print('{}: {}'.format(name, tensor))&#xA;    print('{} shape: {}'.format(name, tensor.shape))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this gives the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: tensor([[[[ 0.,  1.,  2.],&#xA;          [ 3.,  4.,  5.],&#xA;          [ 6.,  7.,  8.]],&#xA;&#xA;         [[ 9., 10., 11.],&#xA;          [12., 13., 14.],&#xA;          [15., 16., 17.]]]])&#xA;input shape: torch.Size([1, 2, 3, 3])&#xA;conv1: Parameter containing:&#xA;tensor([[[[0.2500, 0.2500],&#xA;          [0.2500, 0.2500]],&#xA;&#xA;         [[0.5000, 0.5000],&#xA;          [0.5000, 0.5000]]],&#xA;&#xA;&#xA;        [[[1.0000, 1.0000],&#xA;          [1.0000, 1.0000]],&#xA;&#xA;         [[2.0000, 2.0000],&#xA;          [2.0000, 2.0000]]]], requires_grad=True)&#xA;conv1 shape: torch.Size([2, 2, 2, 2])&#xA;out1: tensor([[[[ 24.,  27.],&#xA;          [ 33.,  36.]],&#xA;&#xA;         [[ 96., 108.],&#xA;          [132., 144.]]]], grad_fn=&amp;lt;MkldnnConvolutionBackward&amp;gt;)&#xA;out1 shape: torch.Size([1, 2, 2, 2])&#xA;conv2: Parameter containing:&#xA;tensor([[[[0.2500, 0.2500],&#xA;          [0.2500, 0.2500]],&#xA;&#xA;         [[0.5000, 0.5000],&#xA;          [0.5000, 0.5000]]],&#xA;&#xA;&#xA;        [[[1.0000, 1.0000],&#xA;          [1.0000, 1.0000]],&#xA;&#xA;         [[2.0000, 2.0000],&#xA;          [2.0000, 2.0000]]]], requires_grad=True)&#xA;conv2 shape: torch.Size([2, 2, 2, 2])&#xA;out2: tensor([[[[ 270.]],&#xA;&#xA;         [[1080.]]]], grad_fn=&amp;lt;MkldnnConvolutionBackward&amp;gt;)&#xA;out2 shape: torch.Size([1, 2, 1, 1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice how the each channel of the convolution sums over all previous channels outputs.&lt;/p&gt;&#xA;" OwnerUserId="26838" LastActivityDate="2019-07-10T19:19:26.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13304" PostTypeId="5" CreationDate="2019-07-10T21:16:38.317" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-10T21:16:38.317" LastActivityDate="2019-07-10T21:16:38.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13305" PostTypeId="4" CreationDate="2019-07-10T21:16:38.317" Score="0" Body="For questions related to machine translation (MT), which is the task of translating text using a computer, machine or software." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-10T21:16:38.317" LastActivityDate="2019-07-10T21:16:38.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13307" PostTypeId="1" AcceptedAnswerId="13312" CreationDate="2019-07-10T21:59:48.627" Score="1" ViewCount="199" Body="&lt;p&gt;GLIE+MC control Algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NXzPW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NXzPW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is why does this algorithm use only a single Monte Carlo episode (during PE step) to compute the &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;? In my understanding this has the following drawbacks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If we have multiple terminal states then we will only reach one (per Policy Iteration step PE+PI).&lt;/li&gt;&#xA;&lt;li&gt;It is highly unlikely that we will visit all the states (during training), and a popular scheduling algorithm for exploration constant &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 1/k$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is apparently the episode number, ensures that exploration decays very very rapidly. This ensures that we may never visit a state during our entire training.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So why this algorithm uses single MC episode and why not multiple episodes in a single Policy Iteration step so that the agent gets a better feel of the environment?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-07-11T19:47:57.537" LastActivityDate="2019-07-11T19:47:57.537" Title="Why does GLIE+MC Control Algorithm use a single episode of Monte Carlo evaluation?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13308" PostTypeId="1" CreationDate="2019-07-10T22:23:57.623" Score="1" ViewCount="316" Body="&lt;p&gt;As far as I understand, RL is a process that can be divided into 2 stages:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Exploring a wide range of paths (acting randomly)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Refining the current optimal paths (revolving around actions with a so-far most promising score estimate)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Completing 1. too quickly results in a network that just doesn't spot the best combination of actions, especially if rewards are sparse. &amp;quot;Refining&amp;quot; then has little benefit, since the network will tend to choose between unlucky estimates it observed so far, and will specialise in those.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, finishing 2. too quickly results in a network that might have encountered the best combination, but never got time to refine these &amp;quot;good trajectories&amp;quot;. Thus its estimates of scores along these &amp;quot;good trajectories&amp;quot; are rather poor and inaccurate, so again the network will fear to select and specialize those, because they might have a low (inaccurate) estimate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Why not to give both 1. and 2. the maximum time possible?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;In other words, instead of gradually annealing the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; coefficient (in the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy) down to a low value, why not to always have it as a step function?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example, train 50% of iterations with a value of 1 (acting completely randomly), and for the second half of training with the value of 0.05, etc (very greedy).  Well, 50% is a random guess, could be adjusted manually, as needed. The most important part is this &amp;quot;step function&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;To me, always using such a &amp;quot;step&amp;quot; function would instantly reveal if the initial random search was not long enough.  Perhaps there is a disadvantage of such a step curve?&lt;/p&gt;&#xA;&lt;p&gt;So far, I got the impression that annealing is a gradual process.&#xA;To me, it seems that when using gradual annealing it might not be evident if the neural network (e.q. in DQN or DQRNN) learns poorly because of the mentioned issue or something else.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is there some literature exploring this?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a paper &lt;a href=&quot;https://arxiv.org/abs/1706.10295&quot; rel=&quot;nofollow noreferrer&quot;&gt;Noisy Networks for Exploration&lt;/a&gt;, but it proposes another approach that removes the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; hyperparameter. My question is different, specifically, about this &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27042" LastEditorUserId="2444" LastEditDate="2020-12-04T21:01:13.603" LastActivityDate="2020-12-04T21:01:13.603" Title="Why is the $\epsilon$ hyper-parameter (in the $\epsilon$-greedy policy) annealed smoothly?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;hyper-parameters&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13310" PostTypeId="2" ParentId="13289" CreationDate="2019-07-11T01:21:03.087" Score="5" Body="&lt;p&gt;What you are describing sounds like it could be a deliberate case of &lt;a href=&quot;http://wiki.fast.ai/index.php/Fine_tuning&quot; rel=&quot;noreferrer&quot;&gt;fine-tuning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a fundamental assumption that makes &lt;a href=&quot;https://adventuresinmachinelearning.com/stochastic-gradient-descent/&quot; rel=&quot;noreferrer&quot;&gt;minibatch gradient descent&lt;/a&gt; work for learning problems: It is assumed that any batch or temporal window of consecutive batches forms a decent approximation of the true &lt;em&gt;global&lt;/em&gt; gradient of the error function with respect to any parameterization of the model. If the error surface itself is moving in a big way, that would thwart the purposes of gradient descent--since gradient descent is a local refinement algorithm, all bets are off when you suddenly change the underlying distribution.&#xA;In the example you cited, catastrophic forgetting seems like it would be an after-effect of having &quot;forgotten&quot; data points previously seen, and is either a symptom of the distribution having changed, or of under-representation in the data of some important phenomenon, such that it is rarely seen relative to its importance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience replay from reinforcement learning is a relevant concept that transfers well to this domain. Here is &lt;a href=&quot;https://arxiv.org/abs/1811.11682&quot; rel=&quot;noreferrer&quot;&gt;a paper&lt;/a&gt; that explores this concept with respect to catastrophic forgetting. As long as sampling represents the true gradients sufficiently well (look at &lt;a href=&quot;https://towardsdatascience.com/handling-imbalanced-datasets-in-deep-learning-f48407a0e758&quot; rel=&quot;noreferrer&quot;&gt;training sample balancing&lt;/a&gt; for this) and the model has enough parameters, the catastrophic forgetting problem is unlikely to occur. In randomly shuffled datasets with replacement, it is most likely to occur where datapoints of a particular class are so rare that they are unlikely to be included for a long time during training, effectively fine-tuning the model to a different problem until a matching sample is seen again.&lt;/p&gt;&#xA;" OwnerUserId="27060" LastActivityDate="2019-07-11T01:21:03.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13312" PostTypeId="2" ParentId="13307" CreationDate="2019-07-11T03:06:39.053" Score="1" Body="&lt;p&gt;I feel the general answer is that we want to be as efficient as possible in learning from experience. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy improvement here always produces an equivalent or better policy, so delaying the improvement step to gather more episodes will only slow down learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would note too that often a different kind of Monte Carlo learning is used. Instead the speed of the update is typically controlled with a new hyper parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, instead of keeping track of the visit counts. The Q estimate is then updated something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q \leftarrow Q + \alpha \left (G - Q \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; then lets you tune how much evaluation vs improvement happens. This is called constant alpha Monte Carlo. Often this is used as a stepping stone to introduce TD methods, e.g., in 6.1 of the Sutton and Barto book.&lt;/p&gt;&#xA;" OwnerUserId="27061" LastActivityDate="2019-07-11T03:06:39.053" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13315" PostTypeId="1" CreationDate="2019-07-11T06:46:06.877" Score="0" ViewCount="51" Body="&lt;p&gt;I am a bit confused about NeuralODE and I want to make sure that what I understood so far is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have (for simplicity) 2 data points &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt; measured at &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z_1$&lt;/span&gt; measured at &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;. Normally (in normal NN approach), one would train a NN to predict &lt;span class=&quot;math-container&quot;&gt;$z_1$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$NN(z_0)=z_1$&lt;/span&gt;. In NeuralODE approach, the goal is to train the NN to approximate a function &lt;span class=&quot;math-container&quot;&gt;$f(z_0)$&lt;/span&gt; (I will ignore the explicit time dependence) such that given the ODE: &lt;span class=&quot;math-container&quot;&gt;$\frac{dz}{dt}|_{t_0}=f(z_0)$&lt;/span&gt; which would be approximated as &lt;span class=&quot;math-container&quot;&gt;$\frac{dz}{dt}|_{t_0}=NN(z_0)$&lt;/span&gt; and solving this using some (non AI based) ODE integrator (Euler's method for example) one gets as the solution for this ODE at time &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt; something close to &lt;span class=&quot;math-container&quot;&gt;$z_1$&lt;/span&gt;. So basically the NN now approximates the tangent of the function (&lt;span class=&quot;math-container&quot;&gt;$\frac{dz}{dt}$&lt;/span&gt;) instead of the function itself (&lt;span class=&quot;math-container&quot;&gt;$z(t)$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is my understanding so far correct?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am a bit confused about the training itself. I understand that they use the adjoint method. What I don't understand is what exactly is being updated. As far as I can see, the only things that are free (i.e. not measured data) are the parameters of the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, i.e. the NN approximating it. So one would need to compute &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial loss}{\partial \theta}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the parameters (weights and biases of the network). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why would I need to compute, for example (as they do in the paper) &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial loss}{\partial z_0}$&lt;/span&gt;?&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$Z_0$&lt;/span&gt; is the input which is fixed, so I don't need to update it. &lt;em&gt;What am I missing here?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, if what I said in the first part is correct, it seems like in principle one can get great results for a reasonably simple function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, such as a (for example) 3 layers fully connected NN. So one needs to update the parameters of this NN. On the other hand, ResNets can have tens or hundreds of layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Am I missing a step here or is this new approach so powerful that with a lot fewer parameters one can get very good results?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like a ResNet, even with 2 layers, should be more powerful than Euler's Method ODE, as ResNets would allow more freedom in the sense that the 2 blocks don't need to be the same, while in the NeuralODE using Euler's Method one has the same (single) block. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, I am not sure I understand what do they mean by (continuous) depth in this case. &lt;em&gt;What is the definition of the depth here (I assume it is not just the depth of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;)?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="23871" LastEditorUserId="2444" LastEditDate="2019-07-11T22:13:54.517" LastActivityDate="2019-07-11T22:13:54.517" Title="Confused about NeuralODE" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13316" PostTypeId="1" AcceptedAnswerId="13339" CreationDate="2019-07-11T08:06:56.830" Score="0" ViewCount="124" Body="&lt;p&gt;I know that it is very common for machine learning systems to classify objects based on their visual features such as shapes, colours, curvatures, width-to-length ratios, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'd like to know is this: Do any techniques exist for machine learning systems to classify objects based on how they move?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Suppose that in a still image, 2 different classes of objects look identical. However, in a video Class A glides smoothly across the screen while Class B meanders chaoticaly across the screen.&lt;/li&gt;&#xA;&lt;li&gt;When given multiple videos of the same person walking, classify whether he's sober or drunk.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27066" LastEditorUserId="27066" LastEditDate="2019-07-12T00:44:43.527" LastActivityDate="2019-07-12T13:10:35.307" Title="Can an object's movement (instead of its appearance) be used to classify it?" Tags="&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13317" PostTypeId="1" AcceptedAnswerId="13319" CreationDate="2019-07-11T08:40:14.007" Score="16" ViewCount="5030" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia article for the universal approximation&lt;/a&gt; theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from &lt;a href=&quot;https://arxiv.org/pdf/1709.02540.pdf&quot; rel=&quot;noreferrer&quot;&gt;this conference paper&lt;/a&gt;. However, the paper does not include the proofs of the theorem. Does anybody know where the proof can be found?&lt;/p&gt;&#xA;" OwnerUserId="27047" LastEditorUserId="2444" LastEditDate="2020-07-14T12:13:26.773" LastActivityDate="2021-01-22T15:17:02.490" Title="Where can I find the proof of the universal approximation theorem?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;proofs&gt;&lt;function-approximation&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13318" PostTypeId="1" CreationDate="2019-07-11T08:42:28.213" Score="2" ViewCount="59" Body="&lt;p&gt;For a deep NN, should I generally apply batch normalization after each convolution layer? Or only after some of them?  Which? Every 2nd, every 3rd, lowest, highest, etc.?&lt;/p&gt;&#xA;" OwnerUserId="5852" LastEditorUserId="2444" LastEditDate="2019-07-11T17:14:19.390" LastActivityDate="2019-07-11T21:46:06.870" Title="What is the most common practice to apply batch normalization?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13319" PostTypeId="2" ParentId="13317" CreationDate="2019-07-11T09:05:24.107" Score="19" Body="&lt;p&gt;There are multiple papers on the topic because there have been multiple attempts to prove that neural networks are universal (i.e. they can &lt;strong&gt;approximate&lt;/strong&gt; any &lt;strong&gt;continuous&lt;/strong&gt; function) from slightly different perspectives and using slightly different assumptions (e.g. assuming that certain activation functions are used). Note that these proofs tell you that neural networks can &lt;strong&gt;approximate&lt;/strong&gt; any &lt;strong&gt;continuous&lt;/strong&gt; function, but they do not tell you &lt;strong&gt;exactly&lt;/strong&gt; how you need to train your neural network so that it approximates your desired function. Moreover, most papers on the topic are quite technical and mathematical, so, if you do not have a solid knowledge of approximation theory and related fields, they may be difficult to read and understand. Nonetheless, below there are some links to some possibly useful articles and papers.&lt;/p&gt;&#xA;&lt;p&gt;The article &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;noreferrer&quot;&gt;A visual proof that neural nets can compute any function&lt;/a&gt; (by Michael Nielsen) should give you some intuition behind the &lt;em&gt;universality&lt;/em&gt; of neural networks, so this is probably the first article you should read.&lt;/p&gt;&#xA;&lt;p&gt;Then you should probably read the paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/BF02551274.pdf&quot; rel=&quot;noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function&lt;/a&gt; (1989), by G. Cybenko, who proves that &lt;strong&gt;multi-layer perceptrons&lt;/strong&gt; (i.e. feed-forward neural networks with at least one hidden layer) can approximate any &lt;a href=&quot;https://en.wikipedia.org/wiki/Continuous_function&quot; rel=&quot;noreferrer&quot;&gt;continuous function&lt;/a&gt;. However, he assumes that the neural network uses sigmoid activations functions, which, nowadays, have been replaced in many scenarios by ReLU activation functions. Other works (e.g. [&lt;a href=&quot;http://www2.math.technion.ac.il/%7Epinkus/papers/neural.pdf&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;http://www2.math.technion.ac.il/%7Epinkus/papers/acta.pdf&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;]) showed that you don't necessarily need sigmoid activation functions, but only certain classes of activation functions do not make neural networks universal.&lt;/p&gt;&#xA;&lt;p&gt;The universality property (i.e. the ability to approximate any continuous function) has also been proved in the case of &lt;strong&gt;convolutional neural networks&lt;/strong&gt;. For example, see &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S1063520318302045&quot; rel=&quot;noreferrer&quot;&gt;Universality of Deep Convolutional Neural Networks&lt;/a&gt; (2020), by Ding-Xuan Zhou, which shows that convolutional neural networks can approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough.&lt;/p&gt;&#xA;&lt;p&gt;See also page 632 of &lt;a href=&quot;https://link.springer.com/chapter/10.1007/11840817_66&quot; rel=&quot;noreferrer&quot;&gt;Recurrent Neural Networks Are Universal Approximators&lt;/a&gt; (2006), by Schäfer et al., which shows that &lt;strong&gt;recurrent neural networks&lt;/strong&gt; are universal function approximators. See also &lt;a href=&quot;http://binds.cs.umass.edu/papers/1992_Siegelmann_COLT.pdf&quot; rel=&quot;noreferrer&quot;&gt;On the computational power of neural nets&lt;/a&gt; (1992, COLT) by Siegelmann and Sontag. &lt;a href=&quot;https://stats.stackexchange.com/a/221142/82135&quot;&gt;This answer&lt;/a&gt; could also be useful.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-21T23:59:27.243" LastActivityDate="2020-12-21T23:59:27.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13322" PostTypeId="2" ParentId="13289" CreationDate="2019-07-11T11:23:56.817" Score="1" Body="&lt;p&gt;Maybe in theory, but not in practice. The thing is you seem to consider only chronological/sequential training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there are two ways to view this issue:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;online learning -&gt; then it is a feature of the method&lt;/li&gt;&#xA;&lt;li&gt;offline learning -&gt; it does not happen thanks to several order randomizations&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;1. Online-Training or &lt;a href=&quot;https://en.wikipedia.org/wiki/Online_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Online Machine Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vowpal_Wabbit&quot; rel=&quot;nofollow noreferrer&quot;&gt;woppal wabbit library&lt;/a&gt;. It is a &lt;strong&gt;feature&lt;/strong&gt; (not an issue like you consider) of this library to adapt chronologically to the input it is fed with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I insist: it is a feature to adapt chronologically. It is wanted that when you start only telling him that lions are dangerous, then that it adapts consequently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;2. Offline-Training&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my personal experience, I have used only &lt;strong&gt;randomized&lt;/strong&gt; subsets of my input data as training set. And this randomization is &lt;strong&gt;crucial&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Randomizations happens namely:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;during the training of the neural network, each epochs generally randomize the dataset order&lt;/li&gt;&#xA;&lt;li&gt;during cross-validation, randomization is used as a way to evaluate a robust model that generalises well and does not overfit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12298" LastEditorUserId="12298" LastEditDate="2020-02-03T10:08:57.373" LastActivityDate="2020-02-03T10:08:57.373" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13323" PostTypeId="1" CreationDate="2019-07-11T11:37:39.023" Score="0" ViewCount="54" Body="&lt;p&gt;First than all, I am not sure if this questions is more about Machine Learning, or if its Artificial Intelligence, if not, just let me know I will delete it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At my company we need to create a solution for banks, where a client comes in and they want to to open a bank account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They need to know if that person is a politician or political exposed person, maybe they work in the european comission, or they are family from a pep for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The business users has lots of data sources where to get these people, for example: &lt;a href=&quot;http://www.europarl.europa.eu/meps/en/full-list/all&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.europarl.europa.eu/meps/en/full-list/all&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They want to train a Model (Machine Learning), where the end user can enter the name:  Bill Clinton for example, and then the system has to return the percentage of a person being political or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviosly some persons are 100% politicial and the percentage will be 100%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if they enter a name that is not in any of their data sources, how would I train a model to decide if its pep or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;quite confused&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="27075" LastActivityDate="2019-07-11T12:10:37.393" Title="How to recognize with just name and last name if the person is a political exposed person" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13324" PostTypeId="1" CreationDate="2019-07-11T11:59:21.147" Score="1" ViewCount="119" Body="&lt;p&gt;What is the best choice for loss function in Convolution Neural Network and in Autoencoder in particular - and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the MSE is probably not the best choice, because little difference in lighting can cause a big difference in end loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about Binary cross-entropy? As I understand, this should be used when target vector is composed as 1 at one place and 0 at all others, so you compare only class that should be correct (and ignore others),... But this is an image (although the values are converted in 0-1 values,...)&lt;/p&gt;&#xA;" OwnerUserId="26993" LastEditorUserId="2444" LastEditDate="2019-07-11T17:11:31.560" LastActivityDate="2019-07-11T18:04:32.447" Title="What is the best loss function for convolution neural network and autoencoder?" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13325" PostTypeId="2" ParentId="13323" CreationDate="2019-07-11T12:10:37.393" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But if they enter a name that is not in any of their data sources, how would I train a model to decide if its pep or not?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on just a person's name and nothing else, the accuracy of this model is going to be very low. Consider that most first name, surname combinations in Europe are going to be repeated across the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the accuracy &lt;em&gt;might&lt;/em&gt; still be slightly better than guessing. Some families and social classes could be more likely to be involved in political work, and a statistical model would pick up on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To train the model, take your positive names, and combine with a random selection of names of people that are known to be &quot;not political&quot; or similar enough. It doesn't matter if some of the names are the same provided you are confident in your data. Probably you could just take a phone directory or the electoral register or some other list of general names. Provided your &quot;political&quot; people are a small fraction of all people, this will work well enough even if you have some of them in the negative class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally you mix those name groups in the rough proportion that the bank expects to see &quot;political&quot; and &quot;non-political&quot; customers, so that your data set is a good representation of the target population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you train a classifier on the names. As this is text and sequence data, you will need a solution for that. Possibly LSTM would be suitable architecture, but so might some feature selection from the names in a more simple ML model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember to hold back some data (both positive and negative cases) for cross-validating and testing the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expect the accuracy of your model when testing to be low. Very low. I would not at all be surprised to find the end result unusable by itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is seriously to be part of some bank's account setup process, there needs to be additional data used later in the process. A gate for additional checks based purely on someone's name will perform very poorly in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-07-11T12:10:37.393" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13326" PostTypeId="1" CreationDate="2019-07-11T12:24:50.597" Score="0" ViewCount="60" Body="&lt;p&gt;&lt;strong&gt;UPDATE: After reading more about the topic, I've tried implementing the &#xA; DDPG algorithm instead of using a variation of Q-Learning and still have the same issue.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following issue:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to train my critic to estimate values of state/action pairs. My state consists of 2 real valued variables and my action is another real valued variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I normalize all values before I feed them into the network. Now I have the following issue: The network is very unresponsive to changes in the input. Before I normalize the state and the actions, they can take up any value between 0 and 50. After normalizing them they are in the range between -1 and 1. A change of 1 in the input can become a very small change in the input after normalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in my specific situation, a small change in the action or the state can cause a very large change in the value of the state/action pair. The network does not really learn that correctly, it handles similar inputs similarly all the time (which is okay most of the times, but there are hard cuts in the shape of the value function here and there). If I further reduce the networks' capacity, the networks output becomes constant and ignores all three inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any other tricks, that I could use to increase sensitivity to the input at some points? Or is my network configuration/approach the wrong one (too large, too small)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network I'm training is a simple feedforward neural network that takes two inputs, followed by 2 hidden layers followed by a single output to predict the value for that state/action combination. (I'm still trying out different configurations here, as I have no real feeling for the amount of elements per layer and the amount of layers needed to get the capacity I need without encouraging overfitting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help :)&lt;/p&gt;&#xA;" OwnerUserId="27078" LastEditorUserId="27078" LastEditDate="2019-07-24T14:53:07.663" LastActivityDate="2019-07-24T14:53:07.663" Title="Encoding real valued inputs" Tags="&lt;neural-networks&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13327" PostTypeId="2" ParentId="13324" CreationDate="2019-07-11T18:04:32.447" Score="1" Body="&lt;p&gt;There is no right answer to this. Finding the right loss function is a tough and difficult problem. So your goal as the architect is to try to find one that best suits your needs. So lets think about your needs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention that you dont want lighting shifts to cause large error, so ill take a leap and assume you care more about the shapes and style of the image more than the coloring. To deal with this, maybe consider using difference of the gram matrices (this is considered common place in style transfer literature: &lt;a href=&quot;https://arxiv.org/pdf/1508.06576.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Neural Algorithm of Artistic Style&lt;/a&gt;) Note that you could use the encoder to get the representation of the output as well for the loss, &lt;span class=&quot;math-container&quot;&gt;$L(x) = D(Gram(Enc(x)), Gram(Enc(\hat x))$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; would be some distance metric like euclidian distance.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the outline is all you care about. You could use some known edge detector filter and compare those, ex: &lt;span class=&quot;math-container&quot;&gt;$D(Edge(x), Edge(\hat x))$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you just dont care about color shifts, you could do &lt;span class=&quot;math-container&quot;&gt;$D(x - \mu_x, \hat x - \mu_{\hat x}$&lt;/span&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can play around with whatever distance metric you use, whether it be with MSE, RMSE, MAE, etc... Each has their own small pros/cons based on the loss manifolds they create. In your case i dont think the difference there will be night and day though, but you never know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also mixing and matching is always nice: ex: &lt;span class=&quot;math-container&quot;&gt;$L(x) = \lambda_1 D(x, \hat x) + \lambda_2D(Gram(Enc(x)), Gram(Enc(\hat x)) + ...$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Takeaway:&lt;/strong&gt; MSE might actually be fine, but it really depends on what &lt;strong&gt;you&lt;/strong&gt; prioritize, and once you figure that out you can start getting clever and design the loss that fits your needs and problem&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-11T18:04:32.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13328" PostTypeId="1" AcceptedAnswerId="13332" CreationDate="2019-07-11T21:01:39.310" Score="1" ViewCount="293" Body="&lt;p&gt;I came across the &lt;span class=&quot;math-container&quot;&gt;$TD(0)$&lt;/span&gt;  algorithm from Sutton and Barto:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PmGAZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PmGAZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, the only difference of TD methods with the MC methods is that TD method is not waiting till the end of the episode to update the &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;, but according to David Silver's lecture (Lecture 4- ~34:00),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v5wnp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v5wnp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$TD(0)$&lt;/span&gt; algorithm learns from incomplete episodes, but in the earlier algorithm we can see that the loop repeats until &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is terminal which mean completion of episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So does David Silver by learning from incomplete episodes, mean learning of &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; even when the episode is not completed? Or Did I interpret the algorithm wrong? If so what is the correct interpretation?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-11T21:41:23.617" LastActivityDate="2019-07-11T21:41:23.617" Title="Understanding TD(0) algorithm implementation" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13329" PostTypeId="5" CreationDate="2019-07-11T21:05:25.403" Score="0" Body="&lt;p&gt;Temporal difference (TD) learning refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function. These methods sample from the environment, like Monte Carlo methods, and perform updates based on current estimates, like dynamic programming methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Temporal_difference_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Temporal difference learning - Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/ebook/node60.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Temporal-Difference Learning - Sutton and Barto&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-11T21:32:17.617" LastActivityDate="2019-07-11T21:32:17.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13330" PostTypeId="4" CreationDate="2019-07-11T21:05:25.403" Score="0" Body="For questions regarding concepts/maths/intuition/implementation of Temporal Difference (TD) Learning Algorithms." OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-11T21:32:08.140" LastActivityDate="2019-07-11T21:32:08.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13331" PostTypeId="2" ParentId="13318" CreationDate="2019-07-11T21:14:55.687" Score="1" Body="&lt;p&gt;In the literature, it differs. You will see models do it after or before pooling only, and sometimes you see it after every single convolution.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Batch normalization's assistance to neural networks wasn't really understood for the longest time, initially it was thought to assist with internal covariate shift (hypothesized by the initial paper: &lt;a href=&quot;https://arxiv.org/abs/1502.03167&quot; rel=&quot;nofollow noreferrer&quot;&gt;Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift&lt;/a&gt;) but lately has been tied to the optimization process (&lt;a href=&quot;https://arxiv.org/pdf/1805.11604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Does Batch Normalization Help Optimization?&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means, from an architectural perspective, it is difficult to correctly assume how it should be utilized, unless you really understand its impact on the loss landscape and how your optimization process will traverse it given some initialization (Note, by the way, a recent paper by Google showed that you can alleviate a lot of the benefits of batch normalization sheerly by understanding what issues it's resolving and attempting to mitigate them in the initialization process: &lt;a href=&quot;https://arxiv.org/abs/1901.09321&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fixup Initialization&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I would recommend 3 things until it is more understood how to utilize it generally:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Play around, get &lt;em&gt;frisky&lt;/em&gt; and experiment. Use what works best.  &lt;/li&gt;&#xA;&lt;li&gt;Use block featurizers that are known to work well like residual blocks. Proven in practice and will probably work for you too.  &lt;/li&gt;&#xA;&lt;li&gt;Do the research and investigate it more, if you find the answer, you'll be helping a lot of people :)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-11T21:46:06.870" LastActivityDate="2019-07-11T21:46:06.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13332" PostTypeId="2" ParentId="13328" CreationDate="2019-07-11T21:22:07.153" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$TD(0)$&lt;/span&gt; algorithm learns from incomplete episodes, but in the earlier algorithm we can see that the loop repeats until &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is terminal which mean completion of episode. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the pseudocode, you have two loops: one for each episode and one (nested) for each step of the episode. The &lt;em&gt;until &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is terminal&lt;/em&gt; means that you perform the updates until you end the episode (that is, you end up in a terminal state, e.g. checkmate in the game of chess). &lt;em&gt;For each step of the episode&lt;/em&gt;, you perform the TD(0) update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, you're confusing two things: the fact that each episode ends in a terminal state and the fact that TD learns from incomplete information. Each episode ends in a terminal state (otherwise it would not be called an episode), but this does not mean that it collects a full rollout before updating &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;. In fact, at each step of the episode, it updates &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The information in the David Slider's slide is consistent with the pseudocode. TD learns from experience because it uses the given policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to behave.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;learning from incomplete episodes, mean learning of &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; even when the episode is not completed&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, essentially you're updating the value function during each step of each episode.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-11T21:35:11.320" LastActivityDate="2019-07-11T21:35:11.320" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13333" PostTypeId="5" CreationDate="2019-07-11T21:57:18.030" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-11T21:57:18.030" LastActivityDate="2019-07-11T21:57:18.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13334" PostTypeId="4" CreationDate="2019-07-11T21:57:18.030" Score="0" Body="For questions related to the concept of a parameter of AI and ML models. Examples of parameters are the weights of a neural network, as well as the mean and variance of a Gaussian distribution." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-11T21:57:18.030" LastActivityDate="2019-07-11T21:57:18.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13335" PostTypeId="1" AcceptedAnswerId="15499" CreationDate="2019-07-12T00:20:58.340" Score="2" ViewCount="368" Body="&lt;p&gt;In link prediction problems, there are only known edges and nodes. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If there is a known edge in the node pair, the node pair is regarded as a positive sample. Except for those node pairs whose edges are known, There may exist unobserved edges in some node pairs or there really doesn't exist edges in some node pairs. Our target is to predict potential links in those candidate node pairs. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The node pair where there exist known edge is regarded as a positive sample. So the node pair whose edge are not observed can neither be regarded as a positive example, nor a negative example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think link prediction problem is a semi-supervised problem. However, I find that many papers, for example, &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-94968-0_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRTR: Drug-Disease Association Prediction Based on Graph Regularized Transductive Regression on Heterogeneous Network&lt;/a&gt;, use AUC(Area Under the ROC Curve, &lt;strong&gt;a metric for supervised problems&lt;/strong&gt;) as the metric. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should we understand such behavior? What's the reason?&lt;/p&gt;&#xA;" OwnerUserId="27112" LastEditorUserId="27112" LastEditDate="2019-09-18T00:23:33.243" LastActivityDate="2019-09-18T01:55:16.900" Title="How should we understand the evaluation metric, AUC, in link prediction problems?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13337" PostTypeId="1" CreationDate="2019-07-12T05:32:51.970" Score="1" ViewCount="67" Body="&lt;p&gt;I am trying to ascertain if my neural network is able to generalize or if it’s simply using memory/overfitting to solve a task. I would like my model to generalise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I train the neural network on a randomly generated 3x3 frozen lake environment - with no holes. (The network simply chooses an action for each state it is presented.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I test the model on a much larger frozen lake environment. Still no holes. Still randomly generated. The test environment size is assigned by a random value of 5-15 for each axis (height/width), randomly generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I determine the &quot;degree of generalization&quot; by how many large environments the network is able to solve. At present, it solves 100/100 on the 3x3, and about 83/100 on the larger test environments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I track the solutions it generates, I can see that the network always takes the shortest route available, which is great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys have any ideas, inputs or criticism on the method I use to determine the degree of generalization?&lt;/p&gt;&#xA;" OwnerUserId="26768" LastEditorUserId="2444" LastEditDate="2019-07-13T13:39:14.573" LastActivityDate="2019-07-13T13:39:14.573" Title="How do I determine the generalisation ability of a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;generalization&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13339" PostTypeId="2" ParentId="13316" CreationDate="2019-07-12T09:46:26.577" Score="2" Body="&lt;p&gt;Machine learning systems work based on input data. The form of that data is irrelevant. It may need some configuring on the technical side of things, but the general ability to learn from the dataset remains the same.&lt;br&gt;&#xA;ML systems are built to learn how to interpret data, without you needing to predefine exactly what it should be looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even then, you can reason that videos are no different from images. A video is essentially nothing but a flipbook of images. If you were to paste every frame of the video in a long sequence, you would have one big picture.&lt;br&gt;&#xA;The rest should follow as usual: if a machine learning system can be taught to interpret images, it can evidently also interpret these &quot;frame sequence&quot; images, which means it's able to interpret the dataset that we colloquially call a video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, since a video contains much more information (a 5 second 30fps video is 150x as much data as a single image with the same resolution), the required learning process increases exponentially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, it's perfectly possible, but it will require more processing power and training as there are more input variables to account for.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There are ways to reduce this increase in complexity. For example, to check if someone is drunk or not, rather than push the entire video through the learning algorithm, you instead preprocess the video to figure out the person's &lt;a href=&quot;https://en.wikipedia.org/wiki/Gait_(human)&quot; rel=&quot;nofollow noreferrer&quot;&gt;gait&lt;/a&gt; or even just their footsteps (relative location and timing) and only push this processed data through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This dramatically cuts down on the complexity of the network needed; at the cost of requiring preprocessing which may introduce issues (bad preprocessing = bad learning).&lt;/p&gt;&#xA;" OwnerUserId="27127" LastEditorUserId="27127" LastEditDate="2019-07-12T13:10:35.307" LastActivityDate="2019-07-12T13:10:35.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13340" PostTypeId="1" CreationDate="2019-07-12T10:12:07.717" Score="6" ViewCount="82" Body="&lt;p&gt;Suicide is on the increase in my country and most victims tend to leave early traces from text messages, social media accounts, search engine queries. So I came up with the idea to develop an AI system with the following features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ability to read text messages searching for suicide trigger words&lt;/li&gt;&#xA;&lt;li&gt;Read chats also for the same trigger words&lt;/li&gt;&#xA;&lt;li&gt;Incorporate synchronization of words from software/browsers on recent words typed&lt;/li&gt;&#xA;&lt;li&gt;Learning algorithm to predict the next action after taking notes of the use of these suicide trigger words&lt;/li&gt;&#xA;&lt;li&gt;Ability to access any cellphone, Android, computer, websites &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this feasible or not feasible?&lt;/p&gt;&#xA;" OwnerUserId="27128" LastEditorUserId="2444" LastEditDate="2019-07-12T23:25:44.627" LastActivityDate="2019-07-14T08:31:28.990" Title="Suicide Predictor and Locator" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;applications&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13343" PostTypeId="1" AcceptedAnswerId="13344" CreationDate="2019-07-12T19:21:10.080" Score="3" ViewCount="72" Body="&lt;p&gt;I need a model that will take in a few numerical parameters, and give back a numerical answer (Context: predicting a slope based on environmental factors without having to actually take measurements to find the slope). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am lost, given that I have not found (on the web) any model that is able to solve this task. If someone can suggest a type of model that would be able to solve this type of problem, I would greatly appreciate it. Resources and libraries would also be useful if you care to suggest any. I'm using Python.&lt;/p&gt;&#xA;" OwnerUserId="27156" LastEditorUserId="2444" LastEditDate="2019-07-12T23:17:23.903" LastActivityDate="2019-07-13T13:55:58.717" Title="Which models accept numerical parameters and produce a numerical output?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;definitions&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13344" PostTypeId="2" ParentId="13343" CreationDate="2019-07-12T23:12:56.077" Score="2" Body="&lt;p&gt;You're probably looking for &lt;em&gt;regression&lt;/em&gt;, either &lt;em&gt;linear&lt;/em&gt; or &lt;em&gt;non-linear&lt;/em&gt;, which usually refers to a set of methods that can be used to predict a continuous (or numerical) value (the value of the so-called &lt;em&gt;dependent variable&lt;/em&gt;), given one or more possibly numerical values (the values of the independent variables). (The other common task is called &lt;em&gt;classification&lt;/em&gt;, where the outcome variable is discrete.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dependent and independent variables can also be vectors. In that case, it is called &lt;em&gt;multivariate&lt;/em&gt; linear or non-linear regression. If there is more than one independent variable (or, also called, &lt;em&gt;predictor&lt;/em&gt;), then it is called &lt;em&gt;multivariable (or multiple) linear or non-linear regression&lt;/em&gt;. See, for example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3518362/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multivariate or Multivariable Regression?&lt;/a&gt; (2013), by Bertha Hidalgo and Melody Goodman.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear regression&lt;/a&gt;, the dependent variable is assumed to be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_combination&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear combination&lt;/a&gt; of the independent variables, while, in &lt;a href=&quot;https://en.wikipedia.org/wiki/Nonlinear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-linear regression&lt;/a&gt;, it is assumed to be a non-linear combination (which is any combination or relationship that is not linear).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The independent variables (or predictors) do not actually need to be independent of each other and, of course, the dependent variable does not need to be independent of the predictors (otherwise you would not be able to predict the value of the dependent variable, given the predictors), so the expression &lt;em&gt;predictor&lt;/em&gt; is possibly more appropriate than &lt;em&gt;independent variable&lt;/em&gt;. See, for example, &lt;a href=&quot;https://stats.stackexchange.com/a/357746/82135&quot;&gt;In Regression Analysis, why do we call independent variables &quot;independent&quot;?&lt;/a&gt;, by Frank Harrell. However, the dependent variable is assumed to be dependent on the independent variables, so you could think of the independent variables as the variables that are not the dependent variables (if this helps you to memorize the concept).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more &lt;a href=&quot;https://en.wikipedia.org/wiki/Dependent_and_independent_variables&quot; rel=&quot;nofollow noreferrer&quot;&gt;synonyms for the dependent and independent variables&lt;/a&gt;, depending on the context or area. For example, the predictors (or independent variables) can also be called &lt;em&gt;covariates&lt;/em&gt;, &lt;em&gt;regressors&lt;/em&gt; or &lt;em&gt;explanatory variables&lt;/em&gt;. The dependent variable can also be called &lt;em&gt;regressand, outcome variable&lt;/em&gt;, or &lt;em&gt;explainable variable&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other types of regression models. In particular, there is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalized_linear_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalized regression model&lt;/a&gt; (GLM), which is (as the name suggests) a generalization of several regression models. The famous logistic regression (where the outcome is actually binomial or binary, so discrete) is an application of the generalized regression model, where the &lt;em&gt;link function&lt;/em&gt; (which is a concept that is used in the context of GLMs) is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Logit&quot; rel=&quot;nofollow noreferrer&quot;&gt;logit function&lt;/a&gt;. See, for example, &lt;a href=&quot;https://stats.stackexchange.com/q/88603/82135&quot;&gt;Why is logistic regression a linear model?&lt;/a&gt;. Note that linear regression is also an application of a GLM. See also this article &lt;a href=&quot;https://scikit-learn.org/stable/supervised_learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1.1. Generalized Linear Models&lt;/a&gt;, which gives you an overview of different GLMs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at this article &lt;a href=&quot;https://statisticsbyjim.com/regression/choosing-regression-analysis/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Choosing the Correct Type of Regression Analysis&lt;/a&gt;, which provides a few guidelines to help you choose the most appropriate regression model for your task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Python, you could use, for example, &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn.linear_model.LinearRegression&lt;/code&gt;&lt;/a&gt;. The Python library &lt;a href=&quot;https://scikit-learn.org/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn&lt;/code&gt;&lt;/a&gt; provides several other regressors. For example, &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostRegressor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn.ensemble.AdaBoostRegressor&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn.ensemble.RandomForestRegressor&lt;/code&gt;&lt;/a&gt;. (Of course, you will need a training dataset to train these regressors, when calling their method &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression.fit&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fit&lt;/code&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-13T13:55:58.717" LastActivityDate="2019-07-13T13:55:58.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13345" PostTypeId="2" ParentId="12759" CreationDate="2019-07-12T23:36:59.593" Score="2" Body="&lt;p&gt;I am also a bit confused by your wording but I will try to clear some things up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During MCTS the policy head is used to guide the search while the value head is used as a replacement for roll outs to estimate how good the game position looks. One iteration of the search procedure in MCTS finds a new leaf node which has not been evaluated by the network yet. This leaf node does not have to be a terminal state of the actual game. In fact, in the rare cases towards the end of the game where it IS a terminal state then the network evaluation can be skipped and the real value is used instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code you find slightly confusing -1 and {} are both place holders. The value -1 is a placeholder for the value head output and {} a placeholder for the policy head output of the neural network evaluation. The policy head output should be a vector of values (one value for each child of the node). It is not related to FPU at all, the code you linked uses FPU 0.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  def value(self):&#xA;    if self.visit_count == 0:&#xA;      return 0&#xA;    return self.value_sum / self.visit_count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The maximum number of moves (512 for chess and 722 for GO) you mention are used when creating training games to train the neural network. The current network is playing against itself and the goal is to create many useful training samples so the games are cut off if they take too long (lots of repetition in chess for example). Note that for chess the actual number of maximum moves is 5899 or an infinite number if draw is not mandatory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you will find these illustrations regarding AlphaZero's MCTS search useful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://tim.hibal.org/blog/alpha-zero-how-and-why-it-works/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://tim.hibal.org/blog/alpha-zero-how-and-why-it-works/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27160" LastActivityDate="2019-07-12T23:36:59.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13349" PostTypeId="1" AcceptedAnswerId="13350" CreationDate="2019-07-13T06:46:17.370" Score="3" ViewCount="217" Body="&lt;p&gt;On Wikipedia, &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;the MCTS algorithm&lt;/a&gt; is described&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Selection&lt;/strong&gt;: start from root &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and select successive child nodes until a leaf node &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is reached. A leaf is any node from which no simulation (playout) has yet been initiated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Expansion&lt;/strong&gt;: create one (or more) child nodes and choose node &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; from one of them. Child nodes are any valid moves from the game position defined by &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Simulation&lt;/strong&gt;: complete one random playout from node &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why is the playout started from a child of the first leaf, not the leaf itself? And aren't leaves then permanently stuck as leaves, since playouts always start from their children, not them? Or does the leaf get attributed as having had a &amp;quot;playout initialised&amp;quot; from it, even though it started at its child?&lt;/p&gt;&#xA;" OwnerUserId="27165" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-19T22:34:04.640" Title="Is the playout started from a leaf or child of leaf in Monte Carlo Tree Search?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13350" PostTypeId="2" ParentId="13349" CreationDate="2019-07-13T09:04:23.260" Score="1" Body="&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: their description of the &lt;em&gt;selection&lt;/em&gt; phase actually &lt;strong&gt;does not match&lt;/strong&gt; the &quot;standard implementation&quot;. They state to traverse the tree according to the selection strategy until a leaf node is reached. I disagree with this. &lt;strong&gt;The standard implementation is to traverse the tree until a node is reached in which there still are legal actions for which no child node has been created&lt;/strong&gt; (a node that is not yet &quot;fully expanded&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These two are only equivalent in the case where the expansion strategy is to immediately fully expand, to immediately create new nodes for &lt;em&gt;all&lt;/em&gt; legal actions in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. With such an expansion strategy, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; would immediately turn from a leaf node into a fully expanded node. But that is not the &quot;standard implementation&quot;. The most common expansion strategy is to only add a single new node to the tree per iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; denote a node that is reached at the end of the selection phase. The standard implementation is indeed to first expand in some way, which results in a new child &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, and then start the playout from &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; onwards.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;And aren't leaves then permanently stuck as leaves, since playouts always start from their children, not them?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, as soon as you add the new child &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is by definition no longer a leaf node; a leaf node is defined as a node with no children. The next time that MCTS decides to traverse the same part of the tree, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; will be just like any other internal node, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; will be a leaf node, and we may add yet another node below that in a future expansion phase. Or, of course, it is also possible that in a future iteration, our selection phase still ends in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; despite it not (anymore) being a leaf, if it still has unexpanded nodes other than &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; (to-be-siblings of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is the playout started from a child of the first leaf, not the leaf itself?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is because we typically require a slightly different mechanism for choosing a node to expand than the playout mechanism. The most common (or, at least, most simple and straightforward) &lt;strong&gt;playout&lt;/strong&gt; mechanism is to select actions uniformly at random &lt;strong&gt;from all legal actions&lt;/strong&gt;. The most common (or, at least, most simple and straightforward) mechanism for selecting a new node &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; to add to &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is to select uniformly at random &lt;strong&gt;only from those legal actions for which children have not already been added to &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;&lt;/strong&gt;. Due to these mechanisms being different, we cannot initiliase the playout at &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and declare the first action of the playout to be the one that generates the new node &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;; we have to use a (slightly) different implementation.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-13T09:04:23.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13351" PostTypeId="1" CreationDate="2019-07-13T10:50:54.687" Score="2" ViewCount="91" Body="&lt;p&gt;I read about Q-Learning and was reading about multi-agent environments. I tried to read the paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.589.8571&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Friend-or-Foe Q-learning&lt;/a&gt;, but could not understand anything, except for a very vague idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.589.8571&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Friend-or-Foe Q-learning&lt;/a&gt; mean? How does it work? Could someone please explain this expression or concept in a simple yet descriptive way that is easier to understand and that helps to get the correct intuition?&lt;/p&gt;&#xA;" OwnerUserId="27173" LastEditorUserId="2444" LastEditDate="2019-07-13T16:25:06.523" LastActivityDate="2019-07-14T19:10:34.283" Title="How does Friend-or-Foe Q-learning intuitively work?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;terminology&gt;&lt;game-theory&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13352" PostTypeId="1" CreationDate="2019-07-13T11:06:19.230" Score="0" ViewCount="40" Body="&lt;p&gt;I am trying to use a CNN to do a regression prediction on some statistical data. The data is time-series data formatted into a 2-D grid. The network I'm using looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_model(num_features):&#xA;&#xA;    model = Sequential()&#xA;    model.add(Conv2D(32, (3, 3), input_shape=(WINDOW, num_features, 2)))&#xA;    model.add(Activation('relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;    model.add(Conv2D(32, (3, 3)))&#xA;    model.add(Activation('relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;    model.add(Conv2D(64, (3, 3)))&#xA;    model.add(Activation('relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;    model.add(Flatten())&#xA;    model.add(Dense(64))&#xA;    model.add(Activation('relu'))&#xA;    model.add(Dropout(0.5))&#xA;    model.add(Dense(1))&#xA;    model.add(Activation('linear'))&#xA;&#xA;    model.compile(loss='mse', optimizer='adam', metrics=['mse'])&#xA;&#xA;    return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The input to it is a 200 x 100 x 2 array of floats 0..1 and visualised (using the green and red channels of a RGB png and scaling to 0..255) looks like this as an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Xlzr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Xlzr.png&quot; alt=&quot;Example input&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that after an amount of training I'm getting a point where no matter what input I give it, the predictions are all coming back as the same value. So something has gone wrong in the training. If I check after the first epoch, then the predictions come back varying (but of course not accurate at that point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, what do I need to look for? I'm guessing this symptom has a name? I don't even know what to call it to start looking for answers online. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any tips greatly appreciated, thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Matt&lt;/p&gt;&#xA;" OwnerUserId="18372" LastActivityDate="2019-07-13T11:06:19.230" Title="What to look for when CNN returns same prediction for every input?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13353" PostTypeId="5" CreationDate="2019-07-13T13:13:41.950" Score="0" Body="&lt;p&gt;For a more complete description of the field, for example, have a look at &lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;amp;arnumber=8352646&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Agent Systems: A Survey&lt;/a&gt; (2018), by Ali Dorri, Salil S. Kanhere and Raja Jurdak.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-13T13:13:41.950" LastActivityDate="2019-07-13T13:13:41.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13354" PostTypeId="4" CreationDate="2019-07-13T13:13:41.950" Score="0" Body="For questions related to multi-agent systems (MAS), which are systems that involve multiple agents (each of them can have different skills) that cooperate with each other and interact with the environment. There are several challenges faced by MAS, including coordination between agents, security, and task allocation. Multi-agent systems have been applied in areas such as computer science, civil engineering, and electrical engineering." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-13T13:13:41.950" LastActivityDate="2019-07-13T13:13:41.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13357" PostTypeId="1" AcceptedAnswerId="13358" CreationDate="2019-07-13T21:48:24.443" Score="2" ViewCount="48" Body="&lt;p&gt;I wrote a simple implementation of Flappy Bird in Python, and now I'm trying to train an agent to play it at a reasonable skill level using TFLearn. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feed the network an input vector of size 4:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the horizontal distance to the next obstacle&lt;/li&gt;&#xA;&lt;li&gt;the agent's vertical distance from the ground-&lt;/li&gt;&#xA;&lt;li&gt;the agent's vertical distances from the top, and&lt;/li&gt;&#xA;&lt;li&gt;the agent's vertical distances from the bottom parts of the opening in the obstacle. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output layer of the network contains one unit, telling me the Q value of the provided state with the assumption that the action taken in that state will be determined by the policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't know what policy would make the agent learn to play the best.  I can't just make it choose random actions because that would make the policy non-stationary.  What can I do?&lt;/p&gt;&#xA;" OwnerUserId="27169" LastEditorUserId="27169" LastEditDate="2019-07-15T22:29:16.103" LastActivityDate="2019-07-15T22:29:16.103" Title="If deep Q learning involves adjusting the value function for a specific policy, then how do I choose the right policy?" Tags="&lt;q-learning&gt;&lt;deep-rl&gt;&lt;policies&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13358" PostTypeId="2" ParentId="13357" CreationDate="2019-07-14T05:39:47.163" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The output layer of the network contains one unit, telling me the Q value of the provided state with the assumption that the action taken in that state will be determined by the policy. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Typically in Reinforcement Learning, the symbol &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is used when you calculate an &lt;em&gt;action value&lt;/em&gt;, and if you are evaluating for a specific policy, it is noted &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the policy, &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the current state, and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; the action to be taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you appear to be calculating with your network is not the action value, but the &lt;em&gt;state value&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s) = q_{\pi}(s,\pi(s))$&lt;/span&gt; when you have a deterministic policy, and there is a similar relationship for stochastic policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your setup, it would be possible to learn the value of any fixed policy that you provided as input. It is also &lt;em&gt;possible&lt;/em&gt; to drive policy improvements, with some caveats, but that would not be Q learning. The rest of this answer assumes you want to implement Q learning, and probably something like Deep Q Networks (DQN).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I don't know what policy would make the agent learn to play the best. I can't just make it choose random actions because that would make the policy non-stationary. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Actually that would make the policy &lt;em&gt;stochastic&lt;/em&gt;. Non-stationary means that the policy would change over time. Choosing random actions is a perfectly valid stationary policy (provided the probabilities in each state remain the same).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, in an optimal control scenario you actually will have and &lt;em&gt;want&lt;/em&gt; a non-stationary policy. The goal is to start with some poor guess at a policy, like completely random behaviour, and improve it based on experience. The policy is going to change over time. That makes learning Q values in RL a non-stationary problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically you really do start control problems with a random policy. At least in safe environments such as games and simulations.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm trying to train an agent to play it at a reasonable skill level using TFLearn.  What can I do?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;First, modify your network so that it estimates action values &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;. There are two ways to do that: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Take the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; as an input, e.g. one hot encoded, concatenated with the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to make the complete inputs to the neural network. This is simplest approach conceptually.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assign a number in range &lt;span class=&quot;math-container&quot;&gt;$[0,N_{actions})$&lt;/span&gt; to each action. Change the network to output a value for each possible action, so that your estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a)$&lt;/span&gt; is obtained by looking at output indexed by &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. This is a common choice because it is more efficient for selecting best actions to drive the policy later.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using action values for &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;, not state values &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, is an important part of Q learning. Technically it is possible to use state values if you have a model of the environment, and you could add one here. But it would not really be Q learning in that case, but some other related version of Temporal Difference (TD) learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you start working with action values, then you have a way to determine a policy. Your best estimate of the optimal policy is to take the action in state &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; with the highest action value. This is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a \hat{q}(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So run your neural network for the current state and all actions. Using the first form of NN above this means run a minibatch, using the second form you run it once which is more efficient (but constructing the training data later is more complex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you always take the best action, this is called acting &lt;em&gt;greedily&lt;/em&gt;. Usually in Q learning, you want the agent to explore other possibilities instead of always acting the same way - because you need to know whether changing a policy would be better. A very common approach in DQN is to act &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedily, which means take this greedy action by default, but with some probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; take a completely random action. Usually &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; starts high, often at &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, and then is decayed relatively quickly down to some lower value e.g. &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0.01$&lt;/span&gt;, where it stays during learning. You set &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0$&lt;/span&gt; for fully greedy behaviour when evaluating or using the policy in production.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that has set your policy, as per your question. There is still more to implement in DQN, summarised below:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Unfortunately you cannot train the agent directly online on each step and expect good results in practice. DQN requires &lt;em&gt;experience replay&lt;/em&gt; as the source of training data. For each step of experience, should store data for start state, action, immediate reward, end state and whether this was the last step in an episode (&lt;span class=&quot;math-container&quot;&gt;$s,a,r,s',done$&lt;/span&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To train the network, improve the Q estimates and thus improve the policy, then after each step you use the Bellman relation for optimal policy as a target: &lt;span class=&quot;math-container&quot;&gt;$q^{*}(s,a) = r + \gamma \text{max}_{a'} q^{*}(s',a')$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Construct a minibatch of training data from the experience replay table, using your NN to calculate the TD target for each one &lt;span class=&quot;math-container&quot;&gt;$\hat{g} = r + \gamma \text{max}_{a'} \hat{q}(s',a')$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Train your NN towards &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a) \rightarrow \hat{g}$&lt;/span&gt; on this minibatch.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;It is usual to use a separate NN to calculate the TD target, and every so many steps (e.g. 1000 steps) make this &quot;target network&quot; a clone of the current learning network. It helps with stability, but you may not need that for a simple environment like Flappy Bird.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-14T06:26:48.847" LastActivityDate="2019-07-14T06:26:48.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13359" PostTypeId="2" ParentId="13340" CreationDate="2019-07-14T08:31:28.990" Score="2" Body="&lt;p&gt;&lt;strong&gt;This question seems to be specifically about your idea so I'll answer as such&lt;/strong&gt; - any general 'can digital activity be used to predict suicide attempts' is probably too broad for this site.&lt;/p&gt;&#xA;&lt;p&gt;As I see it you want to use what a person types, across all types of digital media, to get some measure of their risk of suicide attempt.&lt;/p&gt;&#xA;&lt;h1&gt;Two key questions in any machine learning problem&lt;/h1&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can we create variables as predictors?&lt;/li&gt;&#xA;&lt;li&gt;Is the data there? i.e. do we have labelled data already&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As far as creating variables goes a lot of people don't realise that machine learning (or 'AI') isn't about dumping a load of unedited data into an algorithm and letting it figure it out - 90% of the work is in making those variables into something sensible. In this case it may involve a fair bit of psychology. Perhaps you look at changes in message length - has someone gone from writing essays to short answers? (I'm not a psychologist, that may well be way off the mark). The use of particular words or phrases may come into it - you would need to do a fair bit of research into the differences between those who attempted suicide and those who didn't (perhaps ranking words by frequency and see if there are distinct differences).&lt;/p&gt;&#xA;&lt;p&gt;This brings us to our second question (or perhaps it should be the first) 'is the data there'? You want to predict suicide risk from digital activity so you'll need the digital activity of all different types of people with a label of whether or not they attempted suicide (and when - no point collecting the data two years after their attempt). &lt;strong&gt;Do you think this dataset exists?&lt;/strong&gt; I'm sceptical - the level of detail you would need (both into someone's digital activity and their personal mental state) is unlikely to be recorded.&lt;/p&gt;&#xA;&lt;p&gt;It isn't to say I think its an idea to be thrown out, only that you need to be realistic about the amount of effort required to do something like this. You would need to carry out research to collect this data, need to plan out your ideas before hand to know what you would like to collect (the worst thing is to find, after a year of data collection, that you didn't monitor a variable you would have now found vital).&lt;/p&gt;&#xA;" OwnerUserId="22897" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-07-14T08:31:28.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13360" PostTypeId="1" AcceptedAnswerId="13361" CreationDate="2019-07-14T08:47:20.760" Score="2" ViewCount="163" Body="&lt;p&gt;I have a continuous state space, and a continuous action space. The way I understand it, I can build a policy network which takes as input a continuous state vector and outputs both mean vector and covariance matrix of the action-distribution. To get a valid action I then sample from that distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when trying to implement such a network, I get the error message that the parts of my output layer which I want to be the covariance matrix are singular/not positive-semi-definite. How can I fix this? I tried different activation-functions and initializations for the last layer, but once in a while I run into the same problem again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I enforce that my network outputs a valid covariance matrix?&lt;/p&gt;&#xA;" OwnerUserId="21366" LastActivityDate="2019-12-11T08:13:00.690" Title="How to enforce covariance-matrix output as part of the last layer of a Policy Network?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13361" PostTypeId="2" ParentId="13360" CreationDate="2019-07-14T09:24:05.260" Score="3" Body="&lt;p&gt;Usually it is assumed that there is no correlation between different actions, so the covariance matrix will be zero everywhere except on the main diagonal. Diagonal will represent variances of actions. Diagonal covariance matrix will be positive semidefinite if all values on diagonal are &lt;span class=&quot;math-container&quot;&gt;$\geq$&lt;/span&gt; 0 so you need to insure that output of final layer is &lt;span class=&quot;math-container&quot;&gt;$\geq$&lt;/span&gt; 0, which can be done with ReLU activation for example.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-07-14T09:24:05.260" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13362" PostTypeId="1" CreationDate="2019-07-14T10:29:59.907" Score="2" ViewCount="157" Body="&lt;p&gt;I new in machine learning, especially in Conditional Random Fields (CRF).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read several articles and papers and in there is always associated with HMM and sequences classification. I don't really understand mathematics, especially in the annoying formula. So I can't understand the process. Where I need to start to understand CRFs??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to make an information extraction application using CRF Named Entity Recognition (NER).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got some tutorial for that: &lt;a href=&quot;https://eli5.readthedocs.io/en/latest/tutorials/sklearn_crfsuite.html#training-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eli5.readthedocs.io/en/latest/tutorials/sklearn_crfsuite.html#training-data&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But&lt;/strong&gt; I don't know the proses each step, like training proses, evaluation, and testing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use this code :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  data_frame = eli5.format_as_dataframes(&#xA;            eli5.explain_weights_sklearn_crfsuite(self.crf))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Targets&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bCHpg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bCHpg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Transition Features&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jqlGi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jqlGi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to get that number ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and 1 more thing makes me confused:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;crf = sklearn_crfsuite.CRF(&#xA;    algorithm='lbfgs',&#xA;    c1=0.1,&#xA;    c2=0.1,&#xA;    max_iterations=20,&#xA;    all_possible_transitions=False,&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the algorithm &lt;code&gt;lbfgs&lt;/code&gt;? Is the CRF not an algorithm? Why do I need &lt;code&gt;lbfgs&lt;/code&gt;? What is exactly a conditional random field?&lt;/p&gt;&#xA;" OwnerUserId="22686" LastEditorUserId="22686" LastEditDate="2019-07-20T11:14:53.367" LastActivityDate="2019-07-20T11:14:53.367" Title="What is a conditional random field?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;definitions&gt;&lt;graphical-model&gt;&lt;conditional-random-field&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13363" PostTypeId="2" ParentId="5835" CreationDate="2019-07-14T10:52:00.907" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How does Q learning handle this? Is the Q function only used during the training process, where the future states are known? And is the Q function still used afterwards, if that is the case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The learned &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function is not only used during training, but also after training (in what we may call &quot;deployment&quot;, when we expect a trained agent to behave according to what it has learned).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the reliance on future states is only there during training, it is no longer required for deployment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During training, we use the following &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning update rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a) \gets (1 - \alpha) Q(s, a) + \alpha \left( R + \gamma \color{red}{\max_{a'} Q(s', a')} \right),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the state we reach after executing &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Here, the &lt;span class=&quot;math-container&quot;&gt;$\color{red}{\text{red}}$&lt;/span&gt; part is the part where we rely on knowledge of the future &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. This is available in training because we can simply pick &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, execute it in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, observe &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, and only then trigger our update step.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Outside of training (and actually also during training), we also rely on our &lt;span class=&quot;math-container&quot;&gt;$Q(s, \cdot)$&lt;/span&gt; function for the selection of actions. We typically select an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; according to &lt;span class=&quot;math-container&quot;&gt;$a = \arg\max_a Q(s, a)$&lt;/span&gt;; we select the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that maximises &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; in our current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. &lt;strong&gt;The important thing to note here is that there is no &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; term in this description of how we select actions: we do not require knowledge of our future state&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; in my answer I decided to answer the question literally as it is written, i.e. I'm explaining how &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning can still work in the described setting where actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; have no influence whatsoever on the future state reached. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, I would never recommend actually using &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning in such a setting, and instead refer to Neil Slater's comment about Contextual Multi-Armed Bandit algorithms likely providing a better solution.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-14T10:52:00.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13364" PostTypeId="2" ParentId="13360" CreationDate="2019-07-14T16:29:48.370" Score="1" Body="&lt;p&gt;@Brale_ 's answer is correct, it is common practice in a multitude of models to learn a representation of an independent multivariate normal, but don't let that stop you from pushing the envelope for your needs.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can actually learn a dependent form as well. Normally the independent form is done by learning the means and standard deviations, because sampling a standard normal can achieve your draw by &lt;span class=&quot;math-container&quot;&gt;$z \sim N(\mu, Diag(\sigma^2))$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$z = \mu + \sigma \epsilon$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is drawn from a unit normal.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you can actually achieve a similar trick for a generalized multivariate normal distribution: lets assume your trying to learn &lt;span class=&quot;math-container&quot;&gt;$N(\mu, \Sigma)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; is the covariance matrix. So what you would do is learn the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cholesky_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cholesky decomposition&lt;/a&gt; because where &lt;span class=&quot;math-container&quot;&gt;$\Sigma = AA^T$&lt;/span&gt; you can now draw it through the parametrization trick: &lt;span class=&quot;math-container&quot;&gt;$z = \mu + A\epsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-14T16:29:48.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13365" PostTypeId="1" CreationDate="2019-07-14T16:40:24.660" Score="1" ViewCount="16" Body="&lt;p&gt;Can the agent of reinforcement learning system serve as the environemnt for other agents and expose actions as services? Are there research that consider such question?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to formulate the problem of network of reinforcement learning systems in other site of Stack network: &lt;a href=&quot;https://cs.stackexchange.com/questions/111820/value-flow-and-economics-in-stacked-reinforcement-learning-systems-agent-as-r&quot;&gt;https://cs.stackexchange.com/questions/111820/value-flow-and-economics-in-stacked-reinforcement-learning-systems-agent-as-r&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So - is there some research along such stacked RLs? Of course, I know that Google is my friend, but sometimes I come up with the ideas that I don't know how to name them or how they are named by other scientists who have already discovered and researched them. And that is why I am useing stackexchange to give me some keywords for my ideas and I can myself explore futher the field using those keywords. So - &lt;strong&gt;what are the keywords of resarch about such stakced RL systems, of agent-environment interchange in reinforcement?&lt;/strong&gt; learning?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2019-07-14T17:22:04.570" LastActivityDate="2019-07-14T17:22:04.570" Title="Can the agent of reinforcement learning system serve as the environment for other agents and expose actions as services?" Tags="&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13368" PostTypeId="1" CreationDate="2019-07-14T19:49:30.710" Score="1" ViewCount="35" Body="&lt;p&gt;Can a system, for instance robotic vehicle, be controlled by more than one reinforcement learning algorithm. I intend to use one to address collision avoidance whereas the other to tackle autonomous task completion.&lt;/p&gt;&#xA;" OwnerUserId="27182" LastActivityDate="2019-07-14T19:49:30.710" Title="Can multiple reinforcement algorithms be applied to the same system?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13371" PostTypeId="1" CreationDate="2019-07-15T06:26:33.147" Score="3" ViewCount="89" Body="&lt;p&gt;In reinforcement learning, an agent is usually fully autonomous and independent. It executes actions on the environment, but no other agent can control, explore or command this agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In multi-agent reinforcement learning, can one agent explore, command or communicate with other agents?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would communication between agents somehow be different from the reciprocal execution of each other's actions between agents?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-07-15T21:15:31.830" LastActivityDate="2021-01-22T15:06:00.733" Title="Can one agent command another agent in a multi-agent reinforcement learning setting?" Tags="&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;&lt;multi-agent-systems&gt;&lt;hierarchical-rl&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2021-01-23T04:21:20.937" ContentLicense="CC BY-SA 4.0" />
  <row Id="13372" PostTypeId="1" CreationDate="2019-07-15T08:18:09.700" Score="1" ViewCount="60" Body="&lt;p&gt;I have 3 types of classes and I have 2 images of each object (or class).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I perform image recognition using this very small dataset?&lt;/p&gt;&#xA;" OwnerUserId="27203" LastEditorUserId="2444" LastEditDate="2019-07-16T21:41:28.450" LastActivityDate="2019-07-16T21:41:28.450" Title="Can I perform image recognition using a very small dataset?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13375" PostTypeId="1" CreationDate="2019-07-15T15:18:52.833" Score="1" ViewCount="64" Body="&lt;p&gt;Suppose we trained a neural network on some training set that we call &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the neural network and the method of training(algorithm, hyperparameters etc.) can we infer anything about &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, instead, suppose we also have some subset of the training data &lt;span class=&quot;math-container&quot;&gt;$Y \in X$&lt;/span&gt; available. Is there anything we can infer about &lt;span class=&quot;math-container&quot;&gt;$Y^c$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="27209" LastEditorUserId="27209" LastEditDate="2019-07-16T08:32:38.763" LastActivityDate="2019-07-16T08:32:38.763" Title="What can be inferred about the training data from a trained neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13376" PostTypeId="1" AcceptedAnswerId="13488" CreationDate="2019-07-15T18:02:16.527" Score="2" ViewCount="123" Body="&lt;p&gt;From the subjective perspective, the number of documentaries about the subject Artificial Intelligence and robotics is small. It seems, that the topic is hard to visualize for the audience and in most cases, the assumption is, that the recipient isn't familiar with computers at all. I've found the following documentaries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The Computer Chronicles - Artificial Intelligence (1985)&lt;/li&gt;&#xA;&lt;li&gt;The Machine That Changed the World (1991), Episode IV, The Thinking Machine&lt;/li&gt;&#xA;&lt;li&gt;Robots Rising (1998)&lt;/li&gt;&#xA;&lt;li&gt;Rodney's Robot Revolution (2008)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The subjective awareness is, that the quality of the films in the 1980s was higher than in modern documentaries, and in 50% of the documentaries Rodney Brooks is the host. Are more documentaries available which can be recommended to watch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Focus on non-fictional documentaries&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some fictional movies were already mentioned in a different &lt;a href=&quot;https://ai.stackexchange.com/questions/8844/what-are-the-most-instructive-movies-about-artificial-intelligence&quot;&gt;post&lt;/a&gt;. For example Colossus: The Forbin Project (1970), Bladerunner (1982) or A.I. Artificial Intelligence (2001). They are based on fictional characters which doesn't exist and the presented robots are running with a &lt;a href=&quot;http://wiki.c2.com/?HollywoodOs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hollywood OS&lt;/a&gt;. This question is only about nonfictional motion pictures.&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="1671" LastEditDate="2021-04-19T21:32:30.760" LastActivityDate="2021-04-19T21:32:30.760" Title="Which nonfictional documentaries about Artificial Intelligence are available?" Tags="&lt;education&gt;&lt;journalism&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13377" PostTypeId="1" CreationDate="2019-07-15T20:52:41.933" Score="2" ViewCount="127" Body="&lt;p&gt;The definition of &lt;em&gt;deterministic environment&lt;/em&gt; I am familiar with goes as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The next state of the agent depends only on the current state and the action chosen by the agent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By exclusion, everything else would be a &lt;em&gt;stochastic environment&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what about environments where the next state depends deterministically on the history of previous states and actions chosen? Are such environments also considered &lt;em&gt;deterministic&lt;/em&gt;? Are they very uncommon, and hence just ignored, or should I include them into my working definition of &lt;em&gt;deterministic environment&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="21366" LastEditorUserId="2444" LastEditDate="2019-07-15T20:56:20.613" LastActivityDate="2019-07-16T05:26:17.543" Title="Can non-Markov environments also be deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;environment&gt;&lt;markov-property&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13380" PostTypeId="1" CreationDate="2019-07-16T02:58:43.097" Score="1" ViewCount="135" Body="&lt;p&gt;I am using dropout of different values to train my network. The problem is, dropout is contributing almost nothing to training, either causing so much noise the error never changes, or seemingly having no effect on the error at all:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following runs were seeded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;key: dropout = 0.3, means 30% chance of dropout&lt;/p&gt;&#xA;&#xA;&lt;p&gt;graph x axis: iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y axis: error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dropout=0&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bOctt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bOctt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dropout = 0.001&lt;a href=&quot;https://i.stack.imgur.com/DKau9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DKau9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dropout = 0.1&lt;a href=&quot;https://i.stack.imgur.com/0uUjJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0uUjJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dropout = 0.5&#xA;&lt;a href=&quot;https://i.stack.imgur.com/D3dhv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D3dhv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't quite understand why dropout of 0.5 effectively kills the networks ability to train. This specific network here is rather small, a CNN of architecture:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3x3x3                    Input image&#xA;3x3x3                    Convolutional layer: 3x3x3, stride = 1, padding = 1&#xA;20x1x1                   Flatten layer: 27 -&amp;gt; 20&#xA;20x1x1                   Fully connected layer: 20&#xA;10x1x1                   Fully connected layer: 10&#xA;2x1x1                    Fully connected layer: 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I have tested a CNN with architecture:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10x10x3                  Input image&#xA;9x9x12                   Convolutional layer: 4x4x12, stride = 1, padding = 1&#xA;8x8x12                   Max pooling layer: 2x2, stride = 1&#xA;6x6x24                   Convolutional layer: 3x3x24, stride = 1, padding = 0&#xA;5x5x24                   Max pooling layer: 2x2, stride = 1&#xA;300x1x1                  Flatten layer: 600 -&amp;gt; 300&#xA;300x1x1                  Fully connected layer: 300&#xA;100x1x1                  Fully connected layer: 100&#xA;2x1x1                    Fully connected layer: 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;overnight with dropout = 0.2 and it completely failed to learn anything, having an accuracy of just below 50%, whereas without dropout, its accuracy is ~85%. I would just like to know if there's a specific reason as to why this might be happening. My implementation of dropout is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;activation = relu(val)*(random.random() &amp;gt; self.dropout)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then at test time:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;activation = relu(val)*(1-self.dropout)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="26726" LastEditDate="2019-07-18T02:54:10.007" LastActivityDate="2021-04-08T15:08:15.850" Title="Dropout causes too much noise for network to train" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;dropout&gt;&lt;regularization&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13381" PostTypeId="2" ParentId="13377" CreationDate="2019-07-16T04:22:29.673" Score="0" Body="&lt;p&gt;Depends on the information provided in the state of the system. In theory, the &lt;em&gt;history&lt;/em&gt; can be an element of the state, in which case, by the definition you provided:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The next state of the agent depends only on the current state and the action chosen by the agent.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is a deterministic agent.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the otherhand assume the state has no information about the history, in which case at every point you only know its current status and nothing about where it was previously. In this case, it is a stochastic environment because you can define a distribution with greater than 0 entropy/uncertainty over possible next states.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-16T04:22:29.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13382" PostTypeId="1" CreationDate="2019-07-16T04:49:48.700" Score="4" ViewCount="158" Body="&lt;p&gt;Say you have to enter a story to a computer. Now the computer has to identify the philosophical concept on which the story is based, say: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Was it a &quot;self-fulfilling prophecy&quot;? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Was it an example of &quot;Deadlock&quot; or &quot;Pinocchio paradox situation&quot;? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Was it an example of how rumours magnify? or something similar to a chain reaction process? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Was it an example of &quot;cognitive dissonance&quot; of a person? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Was it a story about &quot;altruism&quot;? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Was it a story about a &quot;misunderstanding&quot; when a person did something &quot;innovative&quot; but it accidentally was innovated earlier so the person was &quot;falsely accused&quot; of &quot;plagiarising&quot;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;et cetera, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the story is &lt;strong&gt;not only&lt;/strong&gt; a heavy rephrase of the pre-existing story; &lt;strong&gt;not only&lt;/strong&gt; character names and identities are totally changed, &lt;strong&gt;&lt;em&gt;but&lt;/em&gt;&lt;/strong&gt; the context completely changed,  the exact tasks they were doing are changed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can computers identify such &quot;concepts&quot; from stories? if yes, then what mechanism it uses?&lt;/p&gt;&#xA;" OwnerUserId="27217" LastEditorUserId="26393" LastEditDate="2019-07-16T08:10:26.520" LastActivityDate="2019-07-16T15:42:08.993" Title="Can a computer conclude following philosophical concepts from a story?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;models&gt;&lt;concepts&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13383" PostTypeId="2" ParentId="13377" CreationDate="2019-07-16T05:26:17.543" Score="1" Body="&lt;p&gt;Markov Environment is not about deterministic or stochastic. &quot;Depends only on the current state and your action&quot; does not mean you know what will happen(deterministic). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can have Markov + deterministic, Markov + Stochastic, Non-Markov + deterministic, and Non-Markov + stochastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition you have is not a definition of deterministic. It is a definition of Markov property. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refer to Wikipedia. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A stochastic process has the Markov property if the conditional&#xA;  probability distribution of future states of the process (conditional&#xA;  on both past and present values) depends only upon the present state;&#xA;  that is, &lt;strong&gt;given the present, the future does not depend on the past&lt;/strong&gt;. A&#xA;  process with this property is said to be Markovian or a Markov&#xA;  process. The most famous Markov process is a Markov chain. Brownian&#xA;  motion is another well-known Markov process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Markov property is assumed mostly in stochastic problems.&#xA;Brownian motion is the motion of molecules of ink in the water and used to model the movement of a stock price, which is stochastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deterministic means when you are in the same state and choose the same action your next state will be always the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic means even you are in the same state and choose the same action, you next state can be different than the previous time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example) You toss a coin and roll a die. Every time you roll a die you get pennies as many. If the coin gets head, you get a chance to roll a die twice next time. &#xA;Your state can be (money you collect so far, coin head/tail in the previous time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this problem, your next state will not be affected by the past. the only thing you need to know is the current state, the money you got and head or tail. It has a &lt;strong&gt;Markov&lt;/strong&gt; process/environment. However, still, it is &lt;strong&gt;stochastic&lt;/strong&gt; because you don't know what will be the next state. &lt;/p&gt;&#xA;" OwnerUserId="23788" LastActivityDate="2019-07-16T05:26:17.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13384" PostTypeId="2" ParentId="13382" CreationDate="2019-07-16T08:15:56.460" Score="4" Body="&lt;p&gt;No. This is currently out of the scope for any language processing system. It requires a general understanding of abstract concepts which is not possible for machines at present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to recognise a self-fulfilling prophecy, you first need to identify that something is a prophecy. So it needs to be something that expresses a possible future state, for which you need to identify what possible future states are; and then you need to see whether it is self-fulfilling. Conceptually this is far too complex to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might get away for some of these with formal criteria (eg use of future tense for something describing a future state/event), but this is far too imprecise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Altruism&quot; requires knowledge about typical expected behaviour; you would need to be able to identify motives behind people's actions, and then decide whether it was altruistic or not. This is just too complex for now (and the foreseeable future).&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-07-16T15:14:07.140" LastActivityDate="2019-07-16T15:14:07.140" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13385" PostTypeId="1" CreationDate="2019-07-16T08:23:12.440" Score="2" ViewCount="80" Body="&lt;p&gt;I learn a DNN for image recognition. During each epoch, I calculate mean loss in the training set. After each epoch, I calculate loss and number of errors over both training and test set. The problem is, training and test error go to (almost) zero, then increase, go to zero again, increase, and so on.  The process seems stochastic.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;epoch: 1 mean_loss=0.109 train: errs=7 loss=0.00622 test: errs=3 loss=0.00608&#xA;epoch: 2 mean_loss=0.00524 train: errs=5 loss=0.00309 test: errs=3 loss=0.00369&#xA;epoch: 3 mean_loss=0.00408 train: errs=13 loss=0.00614 test: errs=7 loss=0.00951&#xA;epoch: 4 mean_loss=0.00198 train: errs=113 loss=0.102 test: errs=51 loss=0.265&#xA;epoch: 5 mean_loss=0.00424 train: errs=3 loss=0.00201 test: errs=2 loss=0.00148&#xA;epoch: 6 mean_loss=0.0027 train: errs=1 loss=0.000466 test: errs=2 loss=0.00193&#xA;epoch: 7 mean_loss=0.00797 train: errs=5 loss=0.00381 test: errs=0 loss=0.000493&#xA;epoch: 8 mean_loss=0.00368 train: errs=1 loss=0.000345 test: errs=2 loss=0.00148&#xA;epoch: 9 mean_loss=0.000358 train: errs=0 loss=6.76e-05 test: errs=0 loss=0.000446&#xA;epoch: 10 mean_loss=0.00101 train: errs=164 loss=0.0863 test: errs=67 loss=0.19&#xA;epoch: 11 mean_loss=0.000665 train: errs=0 loss=2.38e-05 test: errs=0 loss=9.86e-05&#xA;epoch: 12 mean_loss=0.00714 train: errs=5 loss=0.00909 test: errs=0 loss=0.00816&#xA;epoch: 13 mean_loss=0.00266 train: errs=73 loss=0.0333 test: errs=10 loss=0.0192&#xA;epoch: 14 mean_loss=0.00213 train: errs=0 loss=7.74e-05 test: errs=0 loss=0.000197&#xA;epoch: 15 mean_loss=6.12e-05 train: errs=0 loss=7.66e-05 test: errs=0 loss=3.44e-05&#xA;epoch: 16 mean_loss=0.00162 train: errs=5 loss=0.00265 test: errs=0 loss=0.0012&#xA;epoch: 17 mean_loss=0.000159 train: errs=0 loss=3.11e-05 test: errs=0 loss=4.26e-05&#xA;epoch: 18 mean_loss=4.68e-05 train: errs=0 loss=3.28e-05 test: errs=0 loss=6.05e-05&#xA;epoch: 19 mean_loss=2.47e-05 train: errs=0 loss=2.8e-05 test: errs=0 loss=5.01e-05&#xA;epoch: 20 mean_loss=2.2e-05 train: errs=0 loss=2.31e-05 test: errs=0 loss=3.95e-05&#xA;epoch: 21 mean_loss=2.37e-05 train: errs=0 loss=1.76e-05 test: errs=0 loss=2.52e-05&#xA;epoch: 22 mean_loss=1.4e-05 train: errs=0 loss=1.16e-05 test: errs=0 loss=1.52e-05&#xA;epoch: 23 mean_loss=2.13e-05 train: errs=0 loss=1.65e-05 test: errs=0 loss=2.13e-05&#xA;epoch: 24 mean_loss=1.53e-05 train: errs=0 loss=1.91e-05 test: errs=0 loss=2.46e-05&#xA;epoch: 25 mean_loss=0.00419 train: errs=0 loss=5.27e-05 test: errs=0 loss=4.65e-05&#xA;epoch: 26 mean_loss=0.000372 train: errs=6 loss=0.00297 test: errs=3 loss=0.00731&#xA;epoch: 27 mean_loss=0.0016 train: errs=0 loss=4.23e-05 test: errs=0 loss=3.69e-05&#xA;epoch: 28 mean_loss=3.34e-05 train: errs=0 loss=2.44e-05 test: errs=0 loss=2.76e-05&#xA;epoch: 29 mean_loss=7.03e-05 train: errs=0 loss=2.16e-05 test: errs=0 loss=1.69e-05&#xA;epoch: 30 mean_loss=2.41e-05 train: errs=0 loss=1.84e-05 test: errs=0 loss=1.77e-05&#xA;epoch: 31 mean_loss=1.26e-05 train: errs=0 loss=2.11e-05 test: errs=0 loss=1.78e-05&#xA;epoch: 32 mean_loss=1.39e-05 train: errs=0 loss=2.75e-05 test: errs=0 loss=2.42e-05&#xA;epoch: 33 mean_loss=7.68e-05 train: errs=0 loss=0.00014 test: errs=0 loss=4.66e-05&#xA;epoch: 34 mean_loss=2.53e-05 train: errs=0 loss=1.48e-05 test: errs=0 loss=1.56e-05&#xA;epoch: 35 mean_loss=0.000352 train: errs=1786 loss=2.17 test: errs=493 loss=2.56&#xA;epoch: 36 mean_loss=0.0088 train: errs=0 loss=0.000347 test: errs=0 loss=0.000449&#xA;epoch: 37 mean_loss=0.000395 train: errs=0 loss=6.18e-05 test: errs=0 loss=0.000125&#xA;epoch: 38 mean_loss=5e-05 train: errs=0 loss=6.73e-05 test: errs=0 loss=9.89e-05&#xA;epoch: 39 mean_loss=0.00401 train: errs=26 loss=0.00836 test: errs=27 loss=0.0269&#xA;epoch: 40 mean_loss=0.00051 train: errs=0 loss=7.66e-05 test: errs=0 loss=7.07e-05&#xA;epoch: 41 mean_loss=5.49e-05 train: errs=0 loss=2.47e-05 test: errs=0 loss=2.58e-05&#xA;epoch: 42 mean_loss=3.38e-05 train: errs=0 loss=1.67e-05 test: errs=0 loss=2.1e-05&#xA;epoch: 43 mean_loss=2.45e-05 train: errs=0 loss=1.28e-05 test: errs=0 loss=2.95e-05&#xA;epoch: 44 mean_loss=0.00137 train: errs=44 loss=0.0141 test: errs=16 loss=0.0207&#xA;epoch: 45 mean_loss=0.000785 train: errs=1 loss=0.000493 test: errs=0 loss=4.46e-05&#xA;epoch: 46 mean_loss=5.46e-05 train: errs=1 loss=0.000487 test: errs=0 loss=1.34e-05&#xA;epoch: 47 mean_loss=1.99e-05 train: errs=1 loss=0.00033 test: errs=0 loss=1.57e-05&#xA;epoch: 48 mean_loss=1.78e-05 train: errs=1 loss=0.000307 test: errs=0 loss=1.58e-05&#xA;epoch: 49 mean_loss=0.000903 train: errs=1 loss=0.00103 test: errs=0 loss=0.000393&#xA;epoch: 50 mean_loss=4.74e-05 train: errs=0 loss=4.63e-05 test: errs=0 loss=3.53e-05&#xA;Finished Training, time: 234.69774420000002 sec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The images are 96*96 gray.  There are about 7000 training and 1750 test images. The order of presentation is random, and different at each epoch. Each image is either contains the object or not.  The architecture is (Conv2d-&gt;ReLU-&gt;BatchNorm2d-&gt;MaxPool)*4-&gt;AvgPool(6,6)-&gt;Flatten-&gt;Conv-&gt;Conv-&gt;Conv.  All MaxPool's are 2*2. First two Conv2d layers are 5*5, padding=2, others 3*3, padding=1.  The optimiser is like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Optimizer= Adam (&#xA;Parameter Group 0&#xA;    amsgrad: False&#xA;    betas: (0.9, 0.999)&#xA;    eps: 1e-08&#xA;    lr: 0.001&#xA;    weight_decay: 1e-05&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Currently I just choose the epoch when the training set error was minimal. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if epoch == 0 or train_loss &amp;lt; train_loss_best:&#xA;    net_best = copy.deepcopy(net)&#xA;    train_loss_best = train_loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It works, but I don't like it. Is there a way to make the learning more stable and steady?&lt;/p&gt;&#xA;" OwnerUserId="5852" LastEditorUserId="9947" LastEditDate="2019-07-16T16:55:00.573" LastActivityDate="2021-04-06T19:02:20.217" Title="Spikes in of Train and Test error" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;learning-curve&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13387" PostTypeId="1" AcceptedAnswerId="13388" CreationDate="2019-07-16T10:08:40.127" Score="2" ViewCount="48" Body="&lt;p&gt;I don't know if I worded the title correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save it, the file would be too large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have a neural network (Using keras with tf). My question is, is it better to train each file individually for X epochs (File 1 for 5 epochs, then File 2 for 5 epochs, etc.), or should I do an epoch for each, repeatedly (File 1 for an epoch, File 2 for an epoch, etc., and repeat for 5 times).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used the first and I get an accuracy of about 88%. Whould I get an improvement by doing the latter?&lt;/p&gt;&#xA;" OwnerUserId="27219" LastActivityDate="2019-07-16T12:30:35.320" Title="Better to learn the same small set for multiple epochs then go to the next or learn from each one time repeatedly for multiple times?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13388" PostTypeId="2" ParentId="13387" CreationDate="2019-07-16T12:30:35.320" Score="0" Body="&lt;p&gt;Small set for multiple epochs causes overfit, so expose entire data in one epoch, then augment (change a littel bit), and train again. Use small amount of dropout also.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save it, the file would be too large.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do not store augmented data, just do augment with function right after next epoch and overwrite the dataset array&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-07-16T12:30:35.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13389" PostTypeId="1" AcceptedAnswerId="13391" CreationDate="2019-07-16T12:46:04.743" Score="0" ViewCount="433" Body="&lt;p&gt;I notice that I create a model using &lt;code&gt;tensorflow.keras.Sequential()&lt;/code&gt;, save it and the file size is around 5 MiB, but after I call &lt;code&gt;model.fit(..)&lt;/code&gt;, the file size is increased to 17 MiB. I copied the model to reduce the filesize and see that the accuracy is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what exactly is the content of extra 12 MiB that &lt;code&gt;fit()&lt;/code&gt; produces?&#xA;How can I access such content? If I remove those extra 12 MiB, could it affect prediction accuracy or any weird side-effect?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See my experiment code here: &lt;a href=&quot;https://nbviewer.jupyter.org/github/off99555/TensorFlowExperiments/blob/master/test-save-keras-model.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nbviewer.jupyter.org/github/off99555/TensorFlowExperiments/blob/master/test-save-keras-model.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20819" LastEditorUserId="20819" LastEditDate="2019-07-16T13:15:52.483" LastActivityDate="2019-07-16T13:55:39.813" Title="Why does keras model get bigger after training?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-03-10T00:26:32.033" ContentLicense="CC BY-SA 4.0" />
  <row Id="13390" PostTypeId="1" CreationDate="2019-07-16T13:30:52.173" Score="3" ViewCount="386" Body="&lt;p&gt;I have successfully trained a Yolo model to recognize k classes. Now I want to train by adding k+1 class to the pre-trained weights (k classes) without forgetting previous k classes. Ideally, I want to keep adding classes and train over the previous weights, i.e., train only the new classes. If I have to train all classes (k+1) every time a new class is added, it would be too time-consuming, as training k classes would take &lt;span class=&quot;math-container&quot;&gt;$k*20000$&lt;/span&gt; iterations, versus the &lt;span class=&quot;math-container&quot;&gt;$20000$&lt;/span&gt; iterations per new class if I can add the classes incrementally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dataset is balanced (5000 images per classes for training).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciated if you can throw some methods or techniques to do this continual training for Yolo.&lt;/p&gt;&#xA;" OwnerUserId="27221" LastEditorUserId="2444" LastEditDate="2021-01-29T00:05:26.493" LastActivityDate="2021-02-28T00:05:40.587" Title="How can I incrementally train a Yolo model without catastrophic forgetting?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;yolo&gt;&lt;incremental-learning&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13391" PostTypeId="2" ParentId="13389" CreationDate="2019-07-16T13:55:39.813" Score="1" Body="&lt;p&gt;The answer is that it's the size of the Adam optimizer state. When I change the optimizer to SGD (the vanilla optimizer), the size is not big anymore.&#xA;As far as I know, the Adam optimizer maintains gradients information of previous training iterations. And the gradient size can be as big as the model size. That's why it causes the file size to be so big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, when you save your model please make sure to set &lt;code&gt;include_optimizer=False&lt;/code&gt; if you seem to use an optimizer that maintains big state similar to Adam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beware though, it means that you cannot load the model and continue training it again, it should only be used for inference.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-07-16T13:55:39.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13392" PostTypeId="1" AcceptedAnswerId="13394" CreationDate="2019-07-16T15:09:05.087" Score="3" ViewCount="404" Body="&lt;p&gt;In &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007%2F11871842_29.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit Based Monte-Carlo Planning&lt;/a&gt;, the article where UCT is introduced as a planning algorithm, there is an algorithm description in page 285 (4 of the pdf).&lt;/p&gt;&#xA;&lt;p&gt;Comparing this implementation of UCT (a specific type of MCTS algorithm) to the application normally used in games, there is one major difference. Here, the rewards are calculated for every state, instead of only doing an evaluation at the end of the simulation.&lt;/p&gt;&#xA;&lt;p&gt;My questions are (they are all related to each other):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this the only big difference? Or in other words, can I do the same implementation as in MCTS for games with the 4 stages: selection, expansion, simulation and backpropagation, where the result of the simulation is the accumulated reward instead of a value between 0 and 1? How would the UCT selection be adjusted in this case?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What does the UpdateValue function in line 12 does exactly? In the text it says it is used to adjust the state-action pair value at a given depth, will this be used to the selection? How is this calculated exactly?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the depth parameter needed for? Is it related with the UpdateValue?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Finally I would like to ask if you know any other papers where a clear implementation of UCT for planning is used with multiple rewards, not only on the end of the simulation.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-07-16T22:52:37.847" Title="Several questions related to UCT and MCTS" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;planning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13394" PostTypeId="2" ParentId="13392" CreationDate="2019-07-16T15:50:54.180" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is this the only big difference? Or in other words, can I do the same implementation as in a simple MCTS with the 4 stages: selection, expansion, simulation and backpropagation, where the result of the simulation is the accumulated reward instead of a value between 0 and 1? How would the UCT selection be adjusted in this case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, this is not the only difference. The important differences between the &quot;original UCT&quot; (as described in the paper you linked), and the &quot;standard UCT&quot; (as typically implemented in game AI research) are nicely summarised in Subsection 2.4 of &lt;a href=&quot;https://www.jair.org/index.php/jair/article/view/11099/26289&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;On Monte Carlo Tree Search and Reinforcement Learning&quot;&lt;/a&gt;. I'd recommend reading the entire paper, or large parts of it, if you have the time, but to summarise the key differences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Original stores as many nodes as allowed by memory limitations per iteration, standard only expands one node per iteration.&lt;/li&gt;&#xA;&lt;li&gt;Original uses transpositions, standard implementation does not&lt;/li&gt;&#xA;&lt;li&gt;Original uses discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, standard does not (equivalent to picking &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Original takes into account at which point in time rewards were observed, standard does not&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That last point is the one your question mainly seems to be about. Your idea of &lt;strong&gt;using a standard implementation, but accumulating any intermediate rewards and treating them as one large terminal reward, would not be the same&lt;/strong&gt;. Actually taking into account the times at which rewards were observed, as done in the original UCT, can lead to more accurate value estimates in nodes in the middle of the tree (or at least to finding them more quickly). Your idea would be equivalent to learning from Monte-Carlo backups in Reinforcement Learning literature (e.g. Sutton and Barto's book), whereas the original UCT implementation would be more similar to learning from TD(&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) backups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, when talking about the &quot;standard&quot; implementation of UCT, that's very often in papers about two-player zero-sum games in which all intermediate rewards have a value of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, discounting is considered to be unimportant (i.e. &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;), and only terminal game states have a non-zero reward. In this special case, the two ideas do become equivalent.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What does the UpdateValue function in line 12 does exactly? In the text it says it is used to adjust the state-action pair value at a given depth, will this be used to the selection? How is this calculated exactly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This would be done in the &quot;standard&quot; way. The visit count is incremented, and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; could be added to a sum of all &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; values ever observed during the search for that state-action pair, such that the average score can be computed as the sum of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; scores divided by visit count. This average score is the &lt;span class=&quot;math-container&quot;&gt;$\bar{X}$&lt;/span&gt; used in Selection.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the depth parameter needed for? Is it related with the UpdateValue?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think it's usually not really needed. In the paper they describe that it may be considered to have a depth cut-off. From paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;This can be the reach of a terminal state, &lt;strong&gt;or episodes can be cut at a certain depth (line 8)&lt;/strong&gt;.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-16T15:50:54.180" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13395" PostTypeId="1" CreationDate="2019-07-16T16:10:11.193" Score="1" ViewCount="42" Body="&lt;p&gt;In a genetic algorithm, the order of the genes on a chromosome can have a significant effect on the performance (&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.532.8900&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;capacity to generate adaptation&lt;/a&gt;) of the genetic algorithm, where two or more genes interact to produce highly fit individuals. If we have a chromosome length of &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; and genes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; interact, then having them next to each other is strongly preferable than having them at opposing ends of the chromosome. In the former case, the probability of crossover breaking the genes apart is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;, and in the latter it is one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What mechanisms have been tried to optimise the order of genes on a chromosome, so that interacting genes are best protected from crossover? Is it even possible? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've asked at &lt;a href=&quot;https://biology.stackexchange.com/&quot;&gt;Biology SE&lt;/a&gt; if there exists any known biological mechanism which is responsible for such a possible order of the genes on a chromosome.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastEditorUserId="2444" LastEditDate="2019-07-16T21:38:19.533" LastActivityDate="2019-07-16T21:38:19.533" Title="How can I solve the linkage problem in genetic algorithms?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;biology&gt;" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="13396" PostTypeId="2" ParentId="13385" CreationDate="2019-07-16T16:55:45.527" Score="0" Body="&lt;p&gt;I was actually very recently working on CNNs and extensively training models and have noticed the same thing. (&lt;strong&gt;NOTE&lt;/strong&gt;: The answer I will give is purely based on empirical observations and my understanding of mathematics of deep learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the thing I observed on training set (since we are directly optimising on the training set) is that between period of low losses there was suddenly a large loss, and then again low losses, &lt;strong&gt;but&lt;/strong&gt;, this time the loss (of training set) reached even more lower values and the accuracy of the test set reached a higher stable accuracy (by higher stable accuracy I mean, that in general I was observing the accuracy was somewhat oscillating around a fixed value of accuracy, and now the accuracy is oscillating around a higher fixed value of accuracy). From this I concluded that, the high loss was some sort of an obstacle which is stopping the weights from reaching global minima and is trapping it in local minima, unless it gains enough momentum to escape the obstacle, which is indicated by the high loss (think of the loss function as a rotated around y-axis sawtooth waveform inclined at  certain degree). So, as we reach lower loss it is expected (if the model is good) that you see better generalisation and hence the stable higher accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can actually somewhat see this happening in your training too, although the data-set is smaller to make concrete comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seeing your results it seems pretty clear that training loss and test loss are going hand in hand as well as accuracy, which is a good sign, and it means your model has not over-fitted yet and you can train it more, which will probably make you encounter these loss spikes less, nevertheless I am pretty sure they will be there every now and then, whichever method you choose, as the loss curve is always a pretty jagged terrain for a DNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I chose to disregard training accuracy, because most of the times it is not really related to the loss in a monotonically increasing way (check this &lt;a href=&quot;https://forums.fast.ai/t/training-and-validation-loss-both-decrease-but-accuracy-doesnt-increase/8150/3&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt;), and the test loss is expected to increase since at such high accuracies, it has been empirically seen that test loss might decrease without affecting test accuracy (check this &lt;a href=&quot;https://stats.stackexchange.com/questions/282160/how-is-it-possible-that-validation-loss-is-increasing-while-validation-accuracy&quot;&gt;answer&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-07-16T16:55:45.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13397" PostTypeId="1" CreationDate="2019-07-16T17:23:35.290" Score="3" ViewCount="23" Body="&lt;p&gt;In human communication, tonality or tonal language play many complex information, including emotions and motives. But excluding such complex aspects, tonality serves some a very basic purpose of &quot;grouping&quot; or &quot;taking common&quot; functions such as: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The sweet, (pause), bread-and-drink. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It means &quot;The sweet bread and the sweet drink&quot;. However&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The sweet-bread, (pause) and drink. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It means only the bread is sweet but the drink isn't necessarily sweet, or the drink's sweetness property isn't assigned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can computers recognise these differences of meaning based on tonality?&lt;/p&gt;&#xA;" OwnerUserId="27217" LastEditorUserId="2444" LastEditDate="2019-07-16T22:38:57.543" LastActivityDate="2019-07-16T22:38:57.543" Title="Can computers recognise &quot;grouping&quot; from voice tonality?" Tags="&lt;natural-language-processing&gt;&lt;voice-recognition&gt;&lt;speech-synthesis&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13399" PostTypeId="1" CreationDate="2019-07-16T20:42:39.837" Score="1" ViewCount="43" Body="&lt;p&gt;I want to use a neural network to perform a multivariable regression, where my dataset contains multiple features, but I can't for the life of me figure it out. Every kind of tutorial on the internet seems to be either for a single feature without information on how to upgrade it to multiple, or results in a yes or a no when I need numeric predictions (that is, it uses neural networks for classification). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please recommend some kind of resource I can use to learn this?&lt;/p&gt;&#xA;" OwnerUserId="27156" LastEditorUserId="2444" LastEditDate="2019-07-16T21:20:35.907" LastActivityDate="2019-07-16T21:20:35.907" Title="How can I perform multivariable regression with neural networks?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13400" PostTypeId="2" ParentId="13399" CreationDate="2019-07-16T21:17:13.320" Score="1" Body="&lt;p&gt;Have a look at &lt;code&gt;sklearn&lt;/code&gt;'s &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sklearn.neural_network.MLPRegressor&lt;/a&gt; class, which uses a multi-layer neural network to do regression. You first need to define the object &lt;code&gt;MLPRegressor&lt;/code&gt;, for example, by specifying the value of the parameter &lt;code&gt;hidden_layer_sizes&lt;/code&gt;, which determines the number of layers and the number of neurons per layer, then you should call the method &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html#sklearn.neural_network.MLPRegressor.fit&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fit&lt;/code&gt;&lt;/a&gt; on this created object and pass to it your data matrix &lt;span class=&quot;math-container&quot;&gt;$X \in \mathbb{R}^{n \times m}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of samples and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of features.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-16T21:17:13.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13401" PostTypeId="1" CreationDate="2019-07-16T22:46:26.720" Score="7" ViewCount="1662" Body="&lt;p&gt;Tay was a chatbot, who learned from Twitter users.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Microsoft's AI fam from the internet that's got zero chill. The more you talk the smarter Tay gets. — Twitter tagline.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Microsoft trained the AI to have a basic ability to communicate, and taught it a few jokes from hired comedians before setting it lose to learn from its conversations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was a mistake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why did Tay go so wrong? Was this an example of &lt;a href=&quot;https://ai.stackexchange.com/q/13289/125&quot;&gt;catastrophic forgetting&lt;/a&gt;, where short, recent trends override large, less recent training, or was it something else entirely?&lt;/p&gt;&#xA;" OwnerUserId="125" LastActivityDate="2019-07-17T09:25:35.587" Title="Was the corruption of Microsoft's &quot;Tay&quot; chatbot an example of catastrophic forgetting?" Tags="&lt;chat-bots&gt;&lt;history&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13404" PostTypeId="2" ParentId="13401" CreationDate="2019-07-17T08:20:39.973" Score="6" Body="&lt;p&gt;Looking at what happened, it was something similar. Though, the case differs in my eyes from one perspective: if it could only do a few comedy jokes, that probably is not a profound starting point to excel in Twitter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, Twitter is about real life, not about comedy. Discussions are sometimes tough and you easily end up to Social Media Bubbles, where only a certain kind of speaking style and topics is cultivated. So, even humans get on the wrong track there; why not a newbie bot? And, with jokes you would probably catch something about language itself, but not about the topics. So, becoming a jerk instead of a nice comedian is a logical direction, where the bot even has to go, at least a little, to communicate on same level, and not alone. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comedian dataset compared to a Twitter dataset is also very small in a technical sense, so talking about a mini trend overkilling a megatrend in this case is probably not true, because of amounts of examples available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, catastrophic things happened in learning, but not catastrophic learning with the definition of that term. &lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="2193" LastEditDate="2019-07-17T09:25:35.587" LastActivityDate="2019-07-17T09:25:35.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13405" PostTypeId="2" ParentId="13401" CreationDate="2019-07-17T09:03:41.213" Score="6" Body="&lt;p&gt;It was essentially a lack of control over crowd-sourced training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While &lt;a href=&quot;https://en.wikipedia.org/wiki/Tay_(bot)&quot; rel=&quot;noreferrer&quot;&gt;Tay&lt;/a&gt; was initially set up with some conversational ability, it seemed to be programmed to learn from interactions with other users. Once users became aware of this, they basically gamed the bot by exposing it to inappropriate language, which Tay's algorithms then picked up and repeated. According to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tay_(bot)&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia article&lt;/a&gt; on the topic, it is not known for sure whether its &lt;em&gt;repeat after me&lt;/em&gt; facility was solely at fault, or if there was other behaviour that caused it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not really an example of &lt;em&gt;catastrophic forgetting&lt;/em&gt;; for once we don't know how Tay worked internally. I would think it's just that it was overwhelmed by new data coming in which was different from the pre-set. It seems unlikely that the kind of language it was exposed to was in any way known in advance and part of its training set (and labelled as 'inappropriate').&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the lesson from this is to never trust any unvetted input data for training, unless you want to risk people abusing this trust as happened in this case.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-07-17T09:03:41.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13406" PostTypeId="1" CreationDate="2019-07-17T11:01:41.400" Score="1" ViewCount="23" Body="&lt;p&gt;I have been executing an open-source Text-to-speech system &lt;a href=&quot;https://github.com/CSTR-Edinburgh/Ossian&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ossian&lt;/a&gt;. It uses feed forward DNNs for it's acoustic modeling. The error graph I've got after running the acoustic model looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rB6J5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rB6J5.png&quot; alt=&quot;Plot of training vs. validation error of acoustic modeling&quot;&gt;&lt;/a&gt;&#xA;Here are some relevant information:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Size of Data: 7 hours of speech data (4000 sentences) &lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;Some hyper-parameters: &lt;br&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;batch_size       : 128 &lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;training_epochs  : 15 &lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;L2_regularization: 0.003 &lt;br&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me to the directions to improve this model? I'm assuming it is suffering from over-fitting problem? What should I do to avoid this? Increasing data? Or changing batch-size/epochs/regularization parameters? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="27246" LastActivityDate="2019-07-17T11:01:41.400" Title="Improving the performance of a DNN model" Tags="&lt;deep-learning&gt;&lt;speech-synthesis&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13409" PostTypeId="2" ParentId="13217" CreationDate="2019-07-17T16:43:34.750" Score="0" Body="&lt;p&gt;I heard back from the authors of the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As expected the bernoulli sampler is non-differentiable, so as an approximation they use the expectation of the samplers gradient.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;\begin{align*}  &#xA;\frac{dL}{dv_i} &amp;amp;= \frac{dL}{dBern(\sigma(v_i))} * \frac{dBern(\sigma(v_i))}{d\sigma(v_i)} * \frac{d\sigma(v_i)}{dv_i} \\&#xA;&amp;amp;\approx \frac{dL}{dBern(\sigma(v_i))} * \frac{dE[Bern(\sigma(v_i))]}{d\sigma(v_i)} * \frac{d\sigma(v_i)}{dv_i} \\&#xA;&amp;amp;= \frac{dL}{dBern(\sigma(v_i))} * \frac{d\sigma(v_i)}{d\sigma(v_i)} * \frac{d\sigma(v_i)}{dv_i} \\&#xA;&amp;amp;= \frac{dL}{dBern(\sigma(v_i))} * 1 * \frac{d\sigma(v_i)}{dv_i} \\&#xA;\end{align*} &#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the answer ended up being as simple as that.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-17T16:43:34.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13410" PostTypeId="1" CreationDate="2019-07-17T17:11:47.747" Score="2" ViewCount="60" Body="&lt;p&gt;The Kohonen network is one fully connected layer, which clusters the input into classes by a given metric. However, the one layer does not allow to operate with complex relations, that's why deep learning is usually used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible then to make multi-layered Kohonen network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AFAIK, the output of the first layer is already cluster flags, so the activation function on the non-last layers must be different from the original Kohonnen definition?&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="2444" LastEditDate="2019-07-17T20:38:58.630" LastActivityDate="2019-07-17T20:38:58.630" Title="Is a multi-layer Kohonen network possible?" Tags="&lt;neural-networks&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13411" PostTypeId="2" ParentId="13410" CreationDate="2019-07-17T18:16:30.500" Score="1" Body="&lt;p&gt;Kohonen networks by definition are single layer FCNN's, but what differentiates them from others is their unsupervised training procedure.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure is a function of the input, the weights and some hyperparameters. This means if you have a multilayer network you could train only the final layers weights using this procedure. Think of it this way, let &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; be the final layer, and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; be the composition of all other layers, such that the whole network &lt;span class=&quot;math-container&quot;&gt;$N(x) = f \circ g(x)$&lt;/span&gt;. Using kohonens training procedure, you could learn &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;'s weights where the input is &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; rather than x. But then how would you learn &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;'s weights?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you could do this by iterative learning. let &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; be a 2 layer network: &lt;span class=&quot;math-container&quot;&gt;$N = f_2 \circ f_1$&lt;/span&gt;. first learn &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; using the single layer procedure, and now you could learn &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; in the same manner, except the input features its trying to cluster would be &lt;span class=&quot;math-container&quot;&gt;$f_1(x)$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There does exist cons to this procedure though: Using a kohonen network at each layer will try to cluster it to the best of its abilities, not so that its final composition will be best (a common problem in many optimization procedures that arent end-to-end), but that its current representation is. This may lead to non-optimal results, not achieving that deep representation that you're looking for that you could achieve in autoencoders or other deeper unsupervised models.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-17T18:16:30.500" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13412" PostTypeId="1" CreationDate="2019-07-17T20:22:22.167" Score="3" ViewCount="61" Body="&lt;p&gt;I have asked this question a number of times, but I always get confusing answers to this, like &quot;normalized data works better&quot;, &quot;data lives in the same scale&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can &lt;code&gt;x-m/s&lt;/code&gt; make the scale of images the same? Please explain to me the maths. Also, take MNIST dataset for example &amp;amp; illustration.&lt;/p&gt;&#xA;" OwnerUserId="27072" LastEditorUserId="2444" LastEditDate="2019-07-17T20:40:58.483" LastActivityDate="2019-07-17T22:18:27.970" Title="Why do we normalize data in a deep neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13413" PostTypeId="2" ParentId="13412" CreationDate="2019-07-17T22:18:27.970" Score="3" Body="&lt;p&gt;I answered a similar question earlier and here is a piece of my answer that i think covers your question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Batch normalization's assistance to neural networks wasn't really understood for the longest time, initially it was thought to assist with internal covariate shift (hypothesized by the initial paper: &lt;a href=&quot;https://arxiv.org/abs/1502.03167&quot; rel=&quot;nofollow noreferrer&quot;&gt;Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift&lt;/a&gt;) but lately has been tied to the optimization process (&lt;a href=&quot;https://arxiv.org/pdf/1805.11604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Does Batch Normalization Help Optimization?&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means, from an architectural perspective, it is difficult to correctly assume how it should be utilized or how it will effect your network, unless you really understand its impact on the loss landscape and how your optimization process will traverse it given your initialization (Note, by the way, a recent paper by Google showed that you can alleviate a lot of the benefits of batch normalization sheerly by understanding what issues it's resolving and attempting to mitigate them in the initialization process: &lt;a href=&quot;https://arxiv.org/abs/1901.09321&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fixup Initialization&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-17T22:18:27.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13414" PostTypeId="1" AcceptedAnswerId="13424" CreationDate="2019-07-18T04:12:21.953" Score="3" ViewCount="65" Body="&lt;p&gt;If I am training an agent to try and navigate a maze as fast as possible, a simple reward would be something like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R(\text{terminal}) &amp;amp;= N - \text{time}\ \ , \ \ N \gg \text{everything} \\&#xA;R(\text{state})&amp;amp; = 0\ \ \text{if not terminal}&#xA;\end{align}&lt;/span&gt;&#xA;i.e. when it reaches the terminal state, it receives a reward but one that decreases if it is slower. Actually I'm not sure if this is better or worse than &lt;span class=&quot;math-container&quot;&gt;$R(\text{terminal}) = 1 / \text{time}$&lt;/span&gt;, so please correct me if I'm wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the maze is really big, it could spend a long time wandering around before even encountering that reward. Are there any reliable ways of modifying the reward function to make the rewards less sparse? Assume that the agent knows the Euclidean distance between itself and the exit, just not the topography of the maze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it at all sound to simply do something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R(\text{current}) = (d_E(\text{start}, \text{exit}) - d_E(\text{current}, \text{exit})) + (\text{terminal}==True)*(N-\text{time})?&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or if not, what kind of dense heuristic reward or other techniques might be better?&lt;/p&gt;&#xA;" OwnerUserId="27260" LastEditorUserId="2444" LastEditDate="2020-10-07T17:25:04.943" LastActivityDate="2020-10-07T17:25:04.943" Title="Are there any reliable ways of modifying the reward function to make the rewards less sparse?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;reward-shaping&gt;&lt;reward-design&gt;&lt;sparse-rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13416" PostTypeId="2" ParentId="13380" CreationDate="2019-07-18T10:23:17.247" Score="0" Body="&lt;pre&gt;&lt;code&gt;why dropout of 0.5 effectively kills the networks ability to train.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because that is too mutch normal values are like 0.15-0.05. Imagine, the 50% of input image is randomly set to 0, THEN it happens on next layer, means in average 25%  remains, etc... also if you have small dataset with too different images for each class, this + drouput wil confuse the network. &#xA;Also your CNN setup is not realy rational. Too much fc layers, replace one or two fc to convo layers.  And i d say the reason of using 4x4 is only with stride 2, else use 3x3. And you sould use batch normalisation and augmentation like small noice would be probably petter then dropout in your case.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2019-07-18T10:23:17.247" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13418" PostTypeId="1" CreationDate="2019-07-18T11:54:50.357" Score="14" ViewCount="523" Body="&lt;p&gt;As a student who wants to work on machine learning, I would like to know how it is possible to start my studies and how to follow it to stay up-to-date. For example, I am willing to work on RL and MAB problems, but there are huge literatures on these topics. Moreover, these topics are studied by researchers from different communities such as AI and ML, Operations Research, Control Engineering, Statistics, etc. And, I think that several papers are published on these topics every week which make it so difficult to follow them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be thankful if someone can suggest a road-map to start studying these topics, follow them and how I should select and study new published papers. Finally, I am willing to know the new trend in RL and MAB problem.&lt;/p&gt;&#xA;" OwnerUserId="10191" LastActivityDate="2019-07-18T21:58:58.073" Title="How to stay a up-to-date researcher in ML/RL community?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;research&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13419" PostTypeId="2" ParentId="13418" CreationDate="2019-07-18T12:23:59.887" Score="15" Body="&lt;p&gt;There are some wonderful resources for keeping up to date in the ML community. Here are just a handful that a coworker showed me:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://deeplearn.org/&quot; rel=&quot;noreferrer&quot;&gt;Deep Learning Monitor&lt;/a&gt;: this site contains hot and new papers along with tweets that are popularized by the community! You can even checkout RL papers specifically here  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.arxiv-sanity.com/&quot; rel=&quot;noreferrer&quot;&gt;arxiv-sanity&lt;/a&gt;: this site updates with popular and new papers that make it onto Arxiv&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://paperswithcode.com/&quot; rel=&quot;noreferrer&quot;&gt;papers with code&lt;/a&gt;: this site is wonderful because not only does it link to papers, but it links to their implementation for reproduction or assistance in your own personal projects. They even have a leaderboard and track state of the art (&lt;a href=&quot;https://paperswithcode.com/sota&quot; rel=&quot;noreferrer&quot;&gt;SoTA&lt;/a&gt;) on tons of different tasks  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://twitter.com/DL_ML_Loop/lists/deep-learning-loop&quot; rel=&quot;noreferrer&quot;&gt;DL_twitter loop&lt;/a&gt;: You can't forget twitter, given that most researchers use it; this is just a single nice group you may like&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-18T21:58:58.073" LastActivityDate="2019-07-18T21:58:58.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13420" PostTypeId="1" AcceptedAnswerId="13431" CreationDate="2019-07-18T14:17:21.907" Score="4" ViewCount="184" Body="&lt;p&gt;I have read through many of the papers and articles linked in &lt;a href=&quot;https://ai.stackexchange.com/questions/12434/how-is-neural-architecture-search-performed&quot;&gt;this thread&lt;/a&gt; but I haven't been able to find an answer to my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have built some small RL networks and I understand how REINFORCE works. I don't quite understand how they are applied to NAS though. Usually RL agents map a &lt;a href=&quot;https://en.wikipedia.org/wiki/File:Reinforcement_learning_diagram.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;state to an action&lt;/a&gt; and get a reward so they can improve their decision making (which action to choose). I understand that the reward comes from the accuracy of the child network and the action is a series of digits encoding the network architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is passed as the state to the RL agent? This doesn't seem to be mentioned in the papers and articles I read. Is it the previous network? Example input data?&lt;/p&gt;&#xA;" OwnerUserId="27267" LastEditorUserId="2444" LastEditDate="2019-07-18T19:57:50.117" LastActivityDate="2019-07-19T17:24:58.317" Title="How does RL based neural architecture search work?" Tags="&lt;reinforcement-learning&gt;&lt;neural-architecture-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13421" PostTypeId="1" CreationDate="2019-07-18T14:55:58.603" Score="2" ViewCount="147" Body="&lt;p&gt;I was reading the recent paper &lt;a href=&quot;https://arxiv.org/abs/1907.04652&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Representation Learning via Hard and Channel-Wise Attention Networks&lt;/a&gt;, where the authors claim that there is &lt;em&gt;no hard attention operator for graph data&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, the difference between hard and soft attention is that for soft attention you're computing the attention scores between the nodes and all their neighbors while for hard attention you have a sampling function that selects only the most important neighbors. If that is the case, then &lt;a href=&quot;https://arxiv.org/abs/1907.04652&quot; rel=&quot;nofollow noreferrer&quot;&gt;GraphSage&lt;/a&gt; is an example of hard attention, because they apply the attention only on a subset of each node's neighbors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding of hard and soft attention wrong, or the claim that the authors made does not hold?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2019-07-18T21:56:15.780" LastActivityDate="2021-04-07T17:00:50.077" Title="Does GraphSage use hard attention?" Tags="&lt;comparison&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13422" PostTypeId="1" AcceptedAnswerId="13423" CreationDate="2019-07-18T15:03:50.343" Score="1" ViewCount="81" Body="&lt;p&gt;I would like to implement the approach represented in &lt;a href=&quot;https://arxiv.org/pdf/1811.06861.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper. Here they used following reconstruction loss:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(X)= \frac{\lambda \cdot || M \odot (X - F(\overline{M} \odot X)) ||_{1} + (1 - \lambda) \cdot || \overline{M} \odot (X - F(\overline{M} \odot X)) ||_{1}}{N}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the author does not explain the function &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;.&#xA;Does someone know a similar function or could understand the function's purpose from the context?&lt;/p&gt;&#xA;" OwnerUserId="23063" LastEditorUserId="2444" LastEditDate="2019-07-18T18:03:42.643" LastActivityDate="2019-07-18T18:03:42.643" Title="Understanding the reconstruction loss in the paper &quot;Anomaly Detection using Deep Learning based Image Completion&quot;" Tags="&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;anomaly-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13423" PostTypeId="2" ParentId="13422" CreationDate="2019-07-18T15:13:04.387" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; in this context is the output of the Convolutional Neural Network that's being trained, which is of the same size as &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.   &lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-18T18:03:08.133" LastActivityDate="2019-07-18T18:03:08.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13424" PostTypeId="2" ParentId="13414" CreationDate="2019-07-18T15:49:52.697" Score="3" Body="&lt;p&gt;Doing something like the dense, distance-based reward signal you propose is possible... but you have to do it very carefully. If you're not careful, and do it in a naive manner, you are likely to reinforce unwanted behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the way I read that reward function you propose, it provide a positive reward for any steps taken by the agent, with larger rewards for steps that get you closer to the goal (except for steps moving back into the start, those would have a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. &lt;strong&gt;There does not appear to be any &quot;compensation&quot; with negative rewards for moves that take you back away from the goal; in fact, such steps also still seem to carry positive rewards!&lt;/strong&gt; This means that the optimal behaviour that your agent can end up learning is to keep moving around in circles (somewhat close to the goal, but never quite stepping into the goal) for an infinite amount of time, continuously racking up those positive rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of adding some extra (heuristic) rewards to speed up learning is referred to as &quot;reward shaping&quot;. Naive approaches to reward shaping often end up unintentionally modifying the &quot;true&quot; objective, as highlighted above. The correct way to implement reward shaping, which provably does not modify the optimal policy, is &lt;a href=&quot;https://people.eecs.berkeley.edu/~russell/papers/icml99-shaping.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Potential-Based Reward Shaping&lt;/a&gt;. The basic intuition behind this is that, if you use reward shaping to encourage &quot;movement&quot; in one &quot;direction&quot;, you should also provide equivalent (taking into account discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;) discouragement for subsequent &quot;movement&quot; in the other &quot;direction&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there is this really cool paper named &quot;&lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9893&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expressing Arbitrary Reward Functions as Potential-Based Advice&lt;/a&gt;&quot; which proposes a method that can automatically convert from additional reward shaping functions specified in the more &quot;natural&quot; or &quot;intuitive&quot; manner like you did, into (approximately) a potential-based one that is more likely to actually function correctly. This is not quite straightforward though, and the approach involves learning an additional value function which makes additional predictions used to implement the &quot;conversion&quot;. So... in practice, in a simple grid-world like yours, I think it's going to be simpler to just figure out the correct potential-based definition yourself than trying to learn it like this, but it's cool stuff nevertheless.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-18T15:49:52.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13426" PostTypeId="1" AcceptedAnswerId="13428" CreationDate="2019-07-18T17:10:25.233" Score="1" ViewCount="118" Body="&lt;p&gt;I'm using &lt;code&gt;sklearn&lt;/code&gt;'s MinMaxScaler in order to scale my data down. However, it would be nice to be able to rescale it back to its original range. Is there any way I can do this? &lt;/p&gt;&#xA;" OwnerUserId="27156" LastEditorUserId="2444" LastEditDate="2019-07-18T17:56:48.570" LastActivityDate="2019-07-18T19:28:33.670" Title="How to rescale data to its original range after MinMaxScaler?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;data-science&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-05-04T12:21:17.770" ContentLicense="CC BY-SA 4.0" />
  <row Id="13428" PostTypeId="2" ParentId="13426" CreationDate="2019-07-18T19:28:33.670" Score="2" Body="&lt;p&gt;You can use the function &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html#sklearn.preprocessing.MinMaxScaler.inverse_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;inverse_transform&lt;/code&gt;&lt;/a&gt; of the created &lt;code&gt;MinMaxScaler&lt;/code&gt; object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://stackoverflow.com/q/41551165/3924118&quot;&gt;this Stack Overflow question&lt;/a&gt; for other answers and examples.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-18T19:28:33.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13429" PostTypeId="1" AcceptedAnswerId="13430" CreationDate="2019-07-18T20:57:51.023" Score="3" ViewCount="133" Body="&lt;p&gt;I am trying to use Q-learning for energy optimization. I only wish to have states that will be visited by the learning agent, and, for each state, I have a function that generates possible actions, so that I would have a Q-table in form of a nested dictionary, with states (added as they occur) as keys whose values are also dictionaries of possible actions as keys and Q-values as values. &lt;em&gt;Is this possible? How would it affect learning? What other methods can I use?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is possible and okay, and I want to update the Q-value, but the next state is one that was never there before and has to be added to my nested dictionary with all possible actions having initial Q-values of zero, &lt;em&gt;how do I update the Q-value, now that all of the actions in this next state have Q-values of zero?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="27231" LastEditorUserId="2444" LastEditDate="2019-07-18T21:43:08.827" LastActivityDate="2019-07-18T21:50:22.927" Title="Is it possible to have a dynamic $Q$-function?" Tags="&lt;ai-design&gt;&lt;q-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13430" PostTypeId="2" ParentId="13429" CreationDate="2019-07-18T21:42:49.833" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have a function that generates possible actions, so that I would have a Q-table in form of a nested dictionary, with states (added as they occur) as keys whose values are also dictionaries of possible actions as keys and q-values as values. Is this possible? How would it affect learning? What other method can I use?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I provided this suggestion to OP here as &lt;a href=&quot;https://datascience.stackexchange.com/questions/55785/q-table-creation-and-update-for-dynamic-action-space&quot;&gt;an answer to a question on Data Science Stack Exchange&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes this is possible. Assuming you have settled on all your other decisions for Q learning (such as rewards, discount factor or time horizon etc), then it will have no &lt;em&gt;logical&lt;/em&gt; impact on learning compared to any other approach to table building. The structure of your table has no relevance to how Q learning converges, it is an implementation detail. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This choice of structure may have a performance impact in terms of how fast the code runs - the design works best when it significantly reduces memory overhead compared to using a tensor that over-specifies all possible states and actions. If all parts of the state vector could take all values in any combination, and all states had the same number of allowed actions, then a tensor model for the Q table would likely be more efficient than a hash.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to update Q-value, but the next state is one that was never there before and has to be added to my nested dictionary with all possible actions having initial Q-values of zero; how do I update the Q-value, now that all of the actions in this next state have Q-values of zero.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume you are referring to the update rule from single step Q learning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) \leftarrow Q(s,a) + \alpha(r + \gamma \text{max}_{a'} Q(s',a') - Q(s,a))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you do when you first visit &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt;, and want to calculate &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'} Q(s',a')$&lt;/span&gt; for the above update, yet all of &lt;span class=&quot;math-container&quot;&gt;$Q(s',a') = 0$&lt;/span&gt;, because you literally just created them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you do is use the zero value in your update. There is no difference whether you create entries on demand or start with a large table of zeroes. The value of zero is your best estimate of the action value, because you have no data. Over time, as the state and action pair are visited multiple times, perhaps across multiple episodes, the values from experience will back up over time steps due to the way that the update formula makes a link between states &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually you can use any arbitrary value other than zero. If you have some method or information from outside of your reinforcement learning routine, then you could use that. Also, sometimes it helps exploration if you use &lt;em&gt;optimistic&lt;/em&gt; starting values - i.e. some value which is likely higher than the true optimal value. There are limits to that approach, but it's a quick and easy trick to try and sometimes it helps explore and discover the best policy more reliably.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-18T21:50:22.927" LastActivityDate="2019-07-18T21:50:22.927" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13431" PostTypeId="2" ParentId="13420" CreationDate="2019-07-18T23:25:15.663" Score="2" Body="&lt;p&gt;(I will repeat a few details that you're already aware of, so that other users can also understand the context).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1611.01578.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Architecture Search (NAS) paper&lt;/a&gt; (that I mention &lt;a href=&quot;https://ai.stackexchange.com/a/12435/2444&quot;&gt;in my answer&lt;/a&gt; to the question you link to in your question), the agent is the controller (see also this question &lt;a href=&quot;https://ai.stackexchange.com/q/13119/2444&quot;&gt;Is there any difference between a control and an action in reinforcement learning?&lt;/a&gt;), which is implemented as a recurrent neural network (RNN). This controller produces actions (or controls), which are strings that represent the hyper-parameters of a neural network (see e.g. section 3.2), based on the reward that it receives, which, in this case, is the accuracy on the validation dataset that the designed (by the controller) and trained neural network obtains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, the controller is thus the reinforcement learning agent (or policy). The controller is an RNN that is represented by a vector of parameters, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}_\text{c}$&lt;/span&gt;. This controller is trained using a policy gradient method to maximize the expected accuracy on the validation dataset (see e.g. section 3 of the paper). So &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}_\text{c}$&lt;/span&gt; are adjusted using this policy gradient method so that the controller generates NN architectures that, after trained on some task, produce higher accuracy on a validation dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function of the controller is thus &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J(\boldsymbol{\theta}_\text{c}) = \mathbb{E}[R]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the accuracy on the validation dataset. Why &lt;em&gt;expected&lt;/em&gt;? You will be performing this operation multiple times, so, intuitively, you want an average of the accuracy that you obtain on the validation dataset using multiple neural network architectures that the controller might produce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accuracy on the validation dataset, represented by &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, is actually non-differentiable, so the authors use the famous REINFORCE algorithm (which I will not explain here). The authors actually use an approximation of the REINFORCE algorithm (see section 3.2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{1}{m} \sum_{k=1}^m \sum_{t=1}^T \nabla_{\boldsymbol{\theta}_\text{c}} \log P (a_t \mid a_{{t-1}:1}; \boldsymbol{\theta}_\text{c}) R_k&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, given this formulation and taking into account the formulation of the REINFORCE algorithm, then the state of the agent seems to be &lt;span class=&quot;math-container&quot;&gt;$a_{{t-1}:1}$&lt;/span&gt;, that is, the previous actions of the agent. (Recall that the agent is a controller that is implemented as a &lt;em&gt;recurrent&lt;/em&gt; neural network).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-19T17:24:58.317" LastActivityDate="2019-07-19T17:24:58.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13432" PostTypeId="1" CreationDate="2019-07-19T01:35:21.130" Score="4" ViewCount="102" Body="&lt;p&gt;I've heard that &lt;a href=&quot;https://www.reddit.com/r/artificial/comments/5r4vwa/data_compression_as_the_key_to_agi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;prediction is equivalent to data compression&lt;/a&gt;. &#xA;Is there a way to take a compression function and use it to create an AI that predicts?&lt;/p&gt;&#xA;" OwnerUserId="27283" LastEditorUserId="2444" LastEditDate="2021-01-26T16:50:24.540" LastActivityDate="2021-01-26T16:50:24.540" Title="Can a data compression function be used to make predictions?" Tags="&lt;ai-design&gt;&lt;agi&gt;&lt;prediction&gt;&lt;data-compression&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13434" PostTypeId="1" CreationDate="2019-07-19T03:19:36.697" Score="2" ViewCount="133" Body="&lt;p&gt;What is the difference between a Machine Learning Engineer and Deep Learning Engineer and an AI developer? What would be their daily tasks at the office?&lt;/p&gt;&#xA;" OwnerUserId="26845" LastEditorUserId="2444" LastEditDate="2019-07-19T21:00:10.090" LastActivityDate="2019-07-19T22:16:58.687" Title="What is the difference between a machine learning engineer and deep learning engineer?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;profession&gt;" AnswerCount="2" CommentCount="2" ClosedDate="2020-05-16T15:16:54.377" ContentLicense="CC BY-SA 4.0" />
  <row Id="13437" PostTypeId="1" AcceptedAnswerId="13441" CreationDate="2019-07-19T08:10:41.447" Score="1" ViewCount="111" Body="&lt;p&gt;In the Pursuit algorithm (to balance exploration and exploitation), the greedy action has a probability say &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; (updated every episode) of being selected, while the rest have a probability &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; (updated every episode) of being selected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please show me an example code (Python) on how to enforce such conditional probabilistic picking?&lt;/p&gt;&#xA;" OwnerUserId="27231" LastEditorUserId="2444" LastEditDate="2019-07-19T17:23:08.590" LastActivityDate="2019-07-19T17:23:08.590" Title="Probabilistic action selection in pursuit algorithm" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13438" PostTypeId="2" ParentId="13432" CreationDate="2019-07-19T08:11:02.650" Score="2" Body="&lt;p&gt;The way some (not all) compression algorithms work is that they encode frequent events in a short code, and rarer events with a longer code. Overall you save more space by encoding the common elements than you need to expend coding the rare ones. One example of this is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Huffman_coding&quot; rel=&quot;nofollow noreferrer&quot;&gt;Huffman code&lt;/a&gt;, which uses a variable length encoding based on the frequency of the items.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use a compression algorithm for prediction if if encodes more than one event at a time. For example, word pairs rather than individual words. Each word pair will have a code, and the common word pairs (eg &lt;em&gt;of the&lt;/em&gt;) will have shorter codes that the ones which are less common (eg &lt;em&gt;of three&lt;/em&gt;). For prediction, select all the word pairs that start with your known sequence (eg &lt;em&gt;of&lt;/em&gt;). Now select from that list the pair with the shortest code (which is more common), so in this example &lt;em&gt;of&lt;/em&gt; would more likely be followed by &lt;em&gt;the&lt;/em&gt; rather than &lt;em&gt;three&lt;/em&gt;. After than, repeat the process with the next word, so look for pairs that begin with &lt;em&gt;the&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All you need is the compression 'code book' which is produced during the compression process -- it's essentially a model of the data you compressed. This also works for longer sequences than pairs, of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more about the topic, I can recommend &lt;em&gt;Managing Gigabytes&lt;/em&gt; by Witten, Moffat, and Bell. Great book on compression techniques.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-07-19T08:11:02.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13440" PostTypeId="2" ParentId="13434" CreationDate="2019-07-19T09:45:46.377" Score="2" Body="&lt;p&gt;Job titles in IT are somewhat arbitrary, and not very consistent even within the same country or general industry. Job titles in software development, especially newer forms of software development such as mobile apps, even more so. The latest trends in Data Science, Machine Learning and &quot;AI&quot; are at the sharp end of that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some common themes - e.g. an &lt;em&gt;AI Researcher&lt;/em&gt; is likely to be expected to be up to date with scientific papers and might be expected to produce some (or summaries/applications of same within the work environment) whilst a &lt;em&gt;Deep Learning Engineer&lt;/em&gt; might &#xA;be expected to be familiar with the ins and outs of provisioning compute resources in Amazon (AWS) or Google (GCP), moving data around for training or inference, with enough coding experience to integrate a TensforFlow model with some already-existing web service. However, none of this is strictly separated and a job advertised as &quot;Data Scientist&quot; for a smaller company might be expected to do both those things and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this means that your best bet is to not take much notice of the job title. Read the job specification carefully. Approach each company with questions about daily tasks, responsibilities and assumed technical skills. Expect in some cases that the companies will be unsure themselves of what the job entails and expecting you to take a mature role in defining your own working relationship with the rest of the company - the more senior and experienced the role being looked for, the more likely this is. Both sides  - prospective employee and prospective employer - may have incorrect assumptions about what is realistic and possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR: It's a bit Wild West out there at the moment, and likely will remain so over the next decade. To reduce variance in your own experience, research job posts carefully and talk in detail to employers about expectations in advance.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-07-19T09:45:46.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13441" PostTypeId="2" ParentId="13437" CreationDate="2019-07-19T09:56:52.770" Score="1" Body="&lt;p&gt;If I am understanding your question properly, you could use something like the following: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np &#xA;p1 = 0.1&#xA;if np.random.rand() &amp;lt; p1: &#xA;    action = 'greedy'&#xA;else: &#xA;    action = np.random.choice(['other_policy1', 'other_policy2', 'other_policy3']) &#xA;&#xA;return action &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27290" LastActivityDate="2019-07-19T09:56:52.770" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="13443" PostTypeId="1" AcceptedAnswerId="13497" CreationDate="2019-07-19T10:50:21.983" Score="3" ViewCount="166" Body="&lt;p&gt;To me, most ANN/RNN related articles don't tell me actually how the network is implemented. I know that in the ANN you'll have multiple neurons, activation function, weights, etc. But, how do you, actually, in each neuron, convert the input to the output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting activation function aside, is the neuron simply doing &lt;span class=&quot;math-container&quot;&gt;$\text{input}*a+b=\text{output}$&lt;/span&gt;, and try to find the correct &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;? If it's true, then how about where you have two neurons and their output (&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;) is pointing to one neuron? Do you first multiply &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; then feed it in as input?&lt;/p&gt;&#xA;" OwnerUserId="23017" LastEditorUserId="2444" LastEditDate="2019-07-19T17:44:41.207" LastActivityDate="2019-07-22T03:57:49.357" Title="How do layers in an artificial neural network transform inputs to outputs?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;artificial-neuron&gt;&lt;neurons&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13444" PostTypeId="1" AcceptedAnswerId="13445" CreationDate="2019-07-19T11:18:03.643" Score="0" ViewCount="66" Body="&lt;p&gt;Referring to the blog, &lt;a href=&quot;http://bamos.github.io/2016/08/09/deep-completion/#step-1-interpreting-images-as-samples-from-a-probability-distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Completion with Deep Learning in TensorFlow&lt;/a&gt;, it clearly says that we would want a generator &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; whose modeled distribution fits our dataset &lt;span class=&quot;math-container&quot;&gt;$data$&lt;/span&gt;, in other words, &lt;span class=&quot;math-container&quot;&gt;$P_{data}=P_g$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, as described earlier in the blog, the space &lt;span class=&quot;math-container&quot;&gt;$P_{data}$&lt;/span&gt; is in is a higher-dimensional space, where a dimension represents a particular pixel in an image, making it a &lt;span class=&quot;math-container&quot;&gt;$64*64*3$&lt;/span&gt; dimensional space (in this case). I have a few questions regarding this  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Since each pixel here will have an intensity value, will the pdf try to encapsulate a unique pdf for each pixel?&lt;/li&gt;&#xA;&lt;li&gt;If we sample the most likely pixel value for each pixel, considering the distributions need not be the same for each pixel, is it not quite likely that the most probabilistic generated image is just noise apart from things like a common background or so?&lt;/li&gt;&#xA;&lt;li&gt;If &lt;span class=&quot;math-container&quot;&gt;$P_g$&lt;/span&gt; is trying to replicate &lt;span class=&quot;math-container&quot;&gt;$P_{data}$&lt;/span&gt; only, does that mean a GAN only tries to learn lower level features that are common in the training set? Are GANs clueless about what its doing?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-07-20T13:39:33.663" LastActivityDate="2019-07-20T13:39:33.663" Title="If the goal of training of a GAN is to have $P_g=P_{data}$, shouldn't this produce the exact same images?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13445" PostTypeId="2" ParentId="13444" CreationDate="2019-07-19T13:56:28.237" Score="0" Body="&lt;p&gt;Answers:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;No, generally GANs model the joint distribution of pixel space. The space is so entangled that it would be very difficult to model a singular pixels PDF.  &lt;/li&gt;&#xA;&lt;li&gt;This question doesn't make sense given the first question's answer. You do &lt;strong&gt;NOT&lt;/strong&gt; sample singular pixels, you sample the joint.  &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_g$&lt;/span&gt; is trying to model &lt;span class=&quot;math-container&quot;&gt;$P_{data}$&lt;/span&gt;, so that's your goal, so &lt;strong&gt;yes&lt;/strong&gt; it will find features in the training set. Ideally, though it can extrapolate usage of the features to generate new images. But, yes depending on your setup you can end up with overfitting or mode-collapse where you really only generate a small subset of images that trick the discriminator. So I would say, no, the GAN doesn't know what its doing (since its just a minimax optimization of a loss you setup), so your goal is to tell it what to do in a way to achieve your best result.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-19T17:42:03.033" LastActivityDate="2019-07-19T17:42:03.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13447" PostTypeId="2" ParentId="13443" CreationDate="2019-07-19T14:50:52.710" Score="4" Body="&lt;p&gt;The basic calculation for a single neuron is of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma\left(\sum_{i} x_i  w_i \right),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is the input to the neuron &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; are the neuron-specific weights for &lt;em&gt;every single&lt;/em&gt; input and &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the pre-specified activation function. In your terms, and disregarding the activation function, the calculation would turn out to be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$c\,a_c + d\,a_d + b$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that the bias term &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is just a weight that gets multiplied by the input &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, thus it appears to have no input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to develop a further understanding for this, you should try to get familiar with matrix and vector notations and the basic linear algebra that underlies the &lt;em&gt;feed-forward neural networks&lt;/em&gt;. If you do, an entire layer of neurons on a whole batch of data will suddenly simply look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma(WX)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a FFNN with say 3 layers will look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma_{3}(W_3\sigma_2(W_2\sigma_1(W_1X)))$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="26117" LastEditorUserId="2444" LastEditDate="2019-07-19T17:46:32.367" LastActivityDate="2019-07-19T17:46:32.367" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13449" PostTypeId="1" CreationDate="2019-07-19T17:59:27.777" Score="2" ViewCount="57" Body="&lt;p&gt;Normally, when you develop a neural network, train it for object recognition (on normal objects like bike, car, plane, dog, cloud, etc.), and it turns out to perform very well, you would like to fine-tune it for e.g. recognizing dog breeds, and this is called fine-tuning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, in reinforcement learning, let's consider a game with rewards on checkpoints &lt;span class=&quot;math-container&quot;&gt;$1, 2, 3, \dots n$&lt;/span&gt;. When you have a bot that plays and learns using some (e.g. value) neural net, it develops some style of solving the problem to reach some checkpoint &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, and, after that, when it will reach the &lt;span class=&quot;math-container&quot;&gt;$k+1$&lt;/span&gt; checkpoint, it probably will have to revalue the whole strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this situation, will the bot fine-tune itself? Does it makes sense to keep a replay buffer as is and to &quot;reset&quot; the neural net to train it from scratch, or it's better to stay with fine-tune approach? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible, topic-related papers would be very welcome!&lt;/p&gt;&#xA;" OwnerUserId="23181" LastEditorUserId="2444" LastEditDate="2019-07-19T19:58:01.263" LastActivityDate="2019-07-19T19:58:01.263" Title="Will the RL agent implemented as a neural network fine-tune itself?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13450" PostTypeId="1" CreationDate="2019-07-19T19:54:35.257" Score="3" ViewCount="90" Body="&lt;p&gt;I am trying to model operational decisions in inventory control. The control policy is base stock with a fixed stock level of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. That is replenishment orders are placed for every demand arrival to take the stock level to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. The replenishments arrive at constant lead time &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. There is an upper limit &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; on the allowed stock out time and it is measured every &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; periods, otherwise, a cost is incurred &lt;span class=&quot;math-container&quot;&gt;$C_p$&lt;/span&gt;. This system functions in a similar manner to the M/G/S queue. The stock out time can be thought as the customer waiting time due to all server busy.  So every &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; period (&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is less than &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;) the inventory level and pipeline of outstanding orders are monitored and a decision about whether to expedite outstanding order (a cost involved &lt;span class=&quot;math-container&quot;&gt;$C_e$&lt;/span&gt;)  or not is taken in order to control the waiting/stock-out time and to minimize the total costs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel it is a time and state-dependent problem and would like to use &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning to solve this MDP problem. The time period &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is typically a quarter i.e. 3 months and I plan to simulate demands as poisson arrivals. My apprehension is whether simulating arrivals would help to evaluate the Q-values because the simulation is for such a short period. Am I not overestimating the Q value in this way? I request some help on how should I proceed with implementation. &lt;/p&gt;&#xA;" OwnerUserId="27304" LastEditorUserId="27304" LastEditDate="2019-11-20T15:16:07.830" LastActivityDate="2019-11-20T15:16:07.830" Title="How can I use Q-learning for inventory decision making?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13451" PostTypeId="1" CreationDate="2019-07-19T22:04:56.670" Score="2" ViewCount="59" Body="&lt;p&gt;Say I have a ML model which is not very costly to train. It has around say 5 hyperparameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to select best hyperparameters would be to keep all the other hyperparamaters fixed and train the model by changing only one hyperparameter within a certain range. For the sake of mathematical convenience, we assume for the hyperparameter &lt;span class=&quot;math-container&quot;&gt;$h^1$&lt;/span&gt;, keeping all other hyperparameters fixed to their initial values, the model performs best when  &lt;span class=&quot;math-container&quot;&gt;$h^1_{low} &amp;lt; h^1 &amp;lt; h^1_{high}$&lt;/span&gt; (which we found out by running the model on a huge range of &lt;span class=&quot;math-container&quot;&gt;$h^1$&lt;/span&gt;). Now we, fix &lt;span class=&quot;math-container&quot;&gt;$h^1$&lt;/span&gt; to one of the best values and tune &lt;span class=&quot;math-container&quot;&gt;$h^2$&lt;/span&gt; the same way, where &lt;span class=&quot;math-container&quot;&gt;$h^1$&lt;/span&gt; is chosen and the rest of the hyperparameters are again fixed on their initial values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Does this method find the best hyperparameter choices for the model? I know if the hyperparameters are independent, then this definitely does find the best solution, but in a general case, what is the general theory around this? (NOTE: I am not asking about the problem of choosing hyperparamaters, but I am asking about the aforementioned approach of choosing hyperparameters)&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-07-20T00:17:35.353" LastActivityDate="2019-07-20T12:09:15.997" Title="Does this hyperparameter optimisation approach yield the optimal hyperparameters?" Tags="&lt;machine-learning&gt;&lt;hyper-parameters&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13452" PostTypeId="2" ParentId="13434" CreationDate="2019-07-19T22:09:01.197" Score="1" Body="&lt;p&gt;Deep learning is widely regarded as a subset of machine learning. Here are some definitions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AI: any technique that enables computers (machine) to mimic human brain, such as threshold logic, conditional clause. decision tree&lt;/li&gt;&#xA;&lt;li&gt;Machine Learning: A subset of AI that included statistically based method  that utilize computer (machine) to improve at task with specific data&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning: the subset of machine learning composed of algorithms that permit software to get train (learn) to perform tasks by (deep) multi layer neural network&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Daily task of the engineer could be &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Looking for data, such as picture, sound, sensor output, video&lt;/li&gt;&#xA;&lt;li&gt;Making dataset from the data including cleaning, labeling&lt;/li&gt;&#xA;&lt;li&gt;Reading and studying other engineers' Git repositories, ML competition article and winner blog, opensource manual&lt;/li&gt;&#xA;&lt;li&gt;Implementing appropriate existing ML algorithms&lt;/li&gt;&#xA;&lt;li&gt;Running machine learning tests and experiments with different model, dataset, hyper parameter&lt;/li&gt;&#xA;&lt;li&gt;Sharing founding from training using visualization of the score&lt;/li&gt;&#xA;&lt;li&gt;Reading (data) scientists' research papers &lt;/li&gt;&#xA;&lt;li&gt;Designing and developing machine learning or deep learning systems&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27229" LastEditorUserId="27229" LastEditDate="2019-07-19T22:16:58.687" LastActivityDate="2019-07-19T22:16:58.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13453" PostTypeId="2" ParentId="13451" CreationDate="2019-07-19T23:19:36.367" Score="2" Body="&lt;p&gt;After you've computed &lt;span class=&quot;math-container&quot;&gt;$h^{1}_{optimal}$&lt;/span&gt; the only thing you can be sure is that this is the best (assuming constrained case) value of &lt;span class=&quot;math-container&quot;&gt;$h^1$&lt;/span&gt; (with respect to some model performance metric) &lt;strong&gt;given your initial values for &lt;span class=&quot;math-container&quot;&gt;$h^2, ..., h^n$&lt;/span&gt;&lt;/strong&gt;. If you change a bit any of &lt;span class=&quot;math-container&quot;&gt;$h^2, ..., h^n$&lt;/span&gt; you're no longer certain that the value &lt;span class=&quot;math-container&quot;&gt;$h^1$&lt;/span&gt; you found is the optimal one. So yes, the key here is the assumption about the independence.&lt;/p&gt;&#xA;" OwnerUserId="22835" LastActivityDate="2019-07-19T23:19:36.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13454" PostTypeId="1" AcceptedAnswerId="13465" CreationDate="2019-07-19T23:27:45.927" Score="3" ViewCount="110" Body="&lt;p&gt;I am unsure about the following parts of the architecture and mechanics of convolution layers in CNNs. Possibly, this is implementation-dependent though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have 2 convolution layers with 10 filters each and the dimension of my input tensors is &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 1$&lt;/span&gt;  (so, grayscale images for example). Passing this input to the first convolution layer results in 10 feature maps (10 matrices of &lt;span class=&quot;math-container&quot;&gt;$n \times m$&lt;/span&gt;, if we use padding), each produced by a different filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what does actually happen when this is passed to the second convolution layer? Are all 10 feature maps passed as one big &lt;span class=&quot;math-container&quot;&gt;$m \times n \times 10$&lt;/span&gt; tensor or are the overlapping cells of the 10 feature maps averaged and a &lt;span class=&quot;math-container&quot;&gt;$m \times n \times 1$&lt;/span&gt; tensor is passed to the next convolution layer? The former would result in an explosion of feature maps with increasing number of convolution layers and the spacial complexity would be in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}\left((nm)^k\right)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of chained convolution layers. Averaging the feature maps before passing them to the next layer would keep the complexity linear. So, which is it? Or are both possibilities commonly used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second question&lt;/strong&gt; (with two sub questions)&lt;strong&gt;:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) This is a similar question. If I have an input volume of &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 3$&lt;/span&gt; (e.g. RGB images) and I have again 2 convolution layers with 10 filters, does each convolution layer have in actuality 30 filters? So 10 sets of 3 filters, one for each channel? Or do I have in fact only 10 filters and the filters are applied to all 3 channels?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) This is the same question as question (1) but for channels: Once I have convolved a filter (consisting of three channel filters? (a)) over the input tensor I end up with 3 feature maps. One for each channel. What do I do with these? Do I average them component-wise with each other? Or do I keep them separate until I have convolved all 10 filters across the input and THEN average the 10 feature maps of each channel? Or do I average all 30 feature maps of all three channels? Or do I just pass on 30 feature maps to the next convoloution layers which in turn knows which of these feature maps belong to which channel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quite a few possibilities... None of the sources I consulted makes this explicit. Maybe because it depends on the individual implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, would be great if somebody could clear this confusion up a little!&lt;/p&gt;&#xA;" OwnerUserId="20150" LastEditorUserId="2444" LastEditDate="2019-07-20T13:55:27.040" LastActivityDate="2019-07-20T15:34:48.060" Title="Are feature maps merged or are they passed on as they are?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;definitions&gt;&lt;implementation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13455" PostTypeId="2" ParentId="13451" CreationDate="2019-07-20T00:07:46.027" Score="2" Body="&lt;p&gt;The theory behind hyper-parameter optimization (HPO) is not well developed. Nonetheless, there are several hyper-parameter optimization approaches, such as &lt;a href=&quot;https://arxiv.org/pdf/1807.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian optimization&lt;/a&gt; (using &lt;a href=&quot;https://distill.pub/2019/visual-exploration-gaussian-processes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian processes&lt;/a&gt;), &lt;a href=&quot;https://stats.stackexchange.com/q/160479/82135&quot;&gt;random search, grid search&lt;/a&gt;, genetic algorithms, etc. See, for example, the paper &lt;a href=&quot;https://arxiv.org/pdf/1502.02127.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyperparameter Search in Machine Learning&lt;/a&gt; (2015), which attempts to formalize the problem of hyper-parameter optimization in machine learning, &lt;a href=&quot;http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Search for Hyper-Parameter Optimization&lt;/a&gt; (2012), and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;related Wikipedia article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1502.02127.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyperparameter Search in Machine Learning&lt;/a&gt; (and, similarly, in &lt;a href=&quot;http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Search for Hyper-Parameter Optimization&lt;/a&gt;), the authors formally define the hyper-parameter optimization problem as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\lambda^* &#xA;&amp;amp;= \operatorname{arg min}_{\lambda}\mathcal{L}(X^{test}; \mathcal{M} = \mathcal{A}(X^{train}; \lambda)) \tag{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; are the hyper-parameters of the &lt;em&gt;learning algorithm&lt;/em&gt; (for example, gradient descent, whose hyper-parameters are the learning rate and the batch size), that is, the algorithm that is used to train the model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; (e.g. a convolutional neural network, with a fixed architecture) using the training (&lt;span class=&quot;math-container&quot;&gt;$X^{train}$&lt;/span&gt;) and test (&lt;span class=&quot;math-container&quot;&gt;$X^{test}$&lt;/span&gt;) datasets (for simplicity, ignore cross-validation and related techniques).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simple words, in equation &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, we want to find the hyper-parameters &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; of the learning algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; that minimize the loss &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; on the test dataset &lt;span class=&quot;math-container&quot;&gt;$X^{test}$&lt;/span&gt;, when the model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; is trained using &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; and the training dataset &lt;span class=&quot;math-container&quot;&gt;$X^{train}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; thus ignores the hyper-parameters associated with the model (e.g., the number of layers of a multi-layer perceptron) and only considers the hyper-parameters associated with the learning algorithm. However, note that the optimal hyperparameters of the learning algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; depend on the given training and test datasets, the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; and the model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;. Eventually, the formulation in &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; could be extended to include the hyper-parameters associated with the model (and other hyper-parameters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in general, the choice of the HPO method (including the method you're proposing) depends on several factors, including the model (and its architecture), the task that needs to be solved, the loss function, the training and test datasets, and the computational complexity and runtime efficiency of the HPO method. For example, if the space of hyper-parameters is discrete and small, then grid search (which can be an exhaustive search) will find the best combination of hyper-parameters, for a given task and dataset. However, grid search can be impractical if the search space is huge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the method you're proposing will not be optimal because, as you state, the hyper-parameters might not be independent of each other. For example, if you're using stochastic gradient descent (that is, you train your model one example at a time), you probably do not want to update the parameters of your model too fast (that is, you probably do not want a high learning rate), given that a single training example is unlikely to be able to give the error signal that is able to update the parameters in the appropriate direction (that is, the global or even local optimum of the loss function). However, if you're using batch gradient descent, the higher the batch size, the more likely you can use a higher learning rate. This example is only meant to give you some intuition, but this might not hold in all cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mostly used hyper-parameter optimization methods (that I mentioned above) seem to assume that the hyper-parameters are, in general, not independent of each other. In fact, this might be a correct assumption, given that, in the real world, the independence assumption almost never holds (see, for example, the related discussions in the context of the naive Bayes classifiers).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-20T12:09:15.997" LastActivityDate="2019-07-20T12:09:15.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13456" PostTypeId="2" ParentId="12757" CreationDate="2019-07-20T02:57:37.093" Score="-1" Body="&lt;p&gt;Politics are driven by a combination of things such as culture, ideology, religion, ethics, morality, desire for money, fame, and power, etc.  Although God has one truth, most people don't know this truth, a few do, and the rest either disagree about it or just don't believe in Him. The Constitution was written by man.  It does not tell us what to do in every situation. It is open to interpretation.  A person's interpretation of the Constitution is driven by his morality, culture, and all the things I mentioned above. AI systems are designed by people so they are driven by their biases. &lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2019-07-20T02:57:37.093" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13457" PostTypeId="1" AcceptedAnswerId="13459" CreationDate="2019-07-20T03:17:34.007" Score="2" ViewCount="55" Body="&lt;p&gt;I'm having some trouble understanding some parts of the usage of target networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get that having the same network predict the state/action/advantage values for both the current networks can lead to instability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my understanding, the intuition behind 1-step TD error that going a step into the future will give you a better estimate, that can then be used to update your original state/action/advantage value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you use a target network, which is less trained than the normal net — especially at early stages of the training — wouldn't the state/action/advantage value be updating towards an inferior estimate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried implementing DQNs and DDPGs on Cartpole, and I've found that the algorithms fail to converge when target networks are used, but work fine when those target networks are removed. &lt;/p&gt;&#xA;" OwnerUserId="27240" LastEditorUserId="2444" LastEditDate="2019-07-20T12:12:12.337" LastActivityDate="2019-07-20T12:12:12.337" Title="Will the target network, which is less trained than the normal network, output inferior estimates?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13459" PostTypeId="2" ParentId="13457" CreationDate="2019-07-20T08:35:36.493" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, if you use a target network, which is less trained than the normal net — especially at early stages of the training — wouldn't the state/action/advantage value be updating towards an inferior estimate?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Possibly, but a critical part of &lt;em&gt;stability&lt;/em&gt; of TD learning with function approximation when adding a target network is that any updates will be consistent in the short term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that with a single network, the TD target calculation will also be biased (and by the same amounts on the first few passes), but they will not be consistent. Each learning update shifts the estimates by a value that is biased. Problems occur with stability when instead of the bias decaying over time due to real data from each step, that the bias amount is enough to form a positive feedback loop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a concrete example, suppose an initial random network calculating &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta)$&lt;/span&gt; where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The reward is 1.0&lt;/li&gt;&#xA;&lt;li&gt;The true value function &lt;span class=&quot;math-container&quot;&gt;$q^*(s,a)$&lt;/span&gt; is 10.0 (it is not really relevant) and that would make &lt;span class=&quot;math-container&quot;&gt;$q^*(s',a') = 9.0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The NN initially, predicts 5.0 for &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta)$&lt;/span&gt; and 7.0 for &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}\hat{q}(s',a',\theta)$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We also have a learning network using parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; that change on each learning update. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We have a frozen copy of the initial network using parameter &lt;span class=&quot;math-container&quot;&gt;$\bar{\theta}$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We have some learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; that for arguments sake we say reduces error in this case by 50% each time it sees this example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The approximation process in the NN means that &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}\hat{q}(s',a',\theta)$&lt;/span&gt; are linked. This is a real feature of neural networks, but hard to model as it evolves over time. Let's say that with the initial setup that any learning of &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta)$&lt;/span&gt; in isolation will make &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}\hat{q}(s',a',\theta)$&lt;/span&gt; change towards &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta)$&lt;/span&gt; by a 50% step. These steps sizes are not that important, they could be just 1% and the problem still occurs.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can control the learning rate directly, but cannot really control the &quot;generalisation link strength&quot; of the approximation.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To really trigger the problem I want to show, let's say that these single steps are occurring far enough away from the end of an episode, or other moderating feedback, that we effectively get a few hundred visits to our &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; pair without impact from other items (in fact it is likely that many will be going through the same issues)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Without the target network, the first TD target is &lt;span class=&quot;math-container&quot;&gt;$1.0 + 7.0 = 8.0$&lt;/span&gt;. After update &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s,a,\theta) = 6.5$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}\hat{q}(s',a',\theta)$&lt;/span&gt; = 6.75$. This looks good, right? Getting closer to the real values . . . but keep going . . . the next updates work like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q(s,a)      max q(s',a')&#xA; 7.125       6.938&#xA; 7.531       7.234&#xA; 7.882       7.559&#xA; 8.221       7.889&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, this looks OK? But let's come back to it after 100 [isolated, so a bit fake] iterations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;40.222      39.889&#xA;40.556      40.222&#xA;40.889      40.556&#xA;41.222      40.889&#xA;41.556      41.222&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This has overshot, and both values are increasing exponentially. The initial neural network bias is caught in a positive feedback loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if we use the target network that always predicts 7.0, what happens is what you expect. After 100 iterations we have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 8.0         7.5&#xA; 8.0         7.5&#xA; 8.0         7.5&#xA; 8.0         7.5&#xA; 8.0         7.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These values are still incorrect, but the updates have made a more conservative and stable step towards the correct values. Note the second value is what the learning network would &lt;em&gt;predict&lt;/em&gt; for &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}\hat{q}(s',a',\theta)$&lt;/span&gt;, but we have used the other prediction &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}\hat{q}(s',a',\bar{\theta})$&lt;/span&gt; on each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality the feedback loops are more complex that this, because this is function approximation so action value estimates from different &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; pairs interact in ways that are hard to predict. But the worry is that bias causes divergence in estimates, and this does happen in practice. It is more likely to happen in environments with long episodes, or where state loops are possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also worth noting that using the frozen target network has not &lt;em&gt;solved&lt;/em&gt; the problem, it has just significantly throttled runaway feedback. The amount of throttling required to keep learning stable will vary depending on the problem. The number of steps between target network updates is a hyperparameter. Set it too low and you risk seeing stability problems. Set it too high and learning will take longer.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I've tried implementing DQNs and DDPGs on Cartpole, and I've found that the algorithms fail to converge when target networks are used, but work fine when those target networks are removed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In that case, your implementations are incorrect. Definitely I have observed DQN with a target network working many times on Cartpole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether or not using a target network makes convergence faster or more stable is more complex, and it may be for your network design or hyperparameter choices that adding the target network is making performance worse.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-07-20T08:35:36.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13460" PostTypeId="1" CreationDate="2019-07-20T09:21:30.297" Score="2" ViewCount="98" Body="&lt;p&gt;I've seen in most deep learning papers use tensors. I understood what tensors are, but I want to dive into them, because I think that might be beneficial for further studies in Artificial Intelligence. Do you have any suggestion (e.g. books or papers) about that? &lt;/p&gt;&#xA;" OwnerUserId="27315" LastEditorUserId="2444" LastEditDate="2019-07-20T10:48:53.580" LastActivityDate="2020-06-26T23:14:52.693" Title="How can I learn tensors for deep learning?" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13462" PostTypeId="2" ParentId="13454" CreationDate="2019-07-20T12:43:46.367" Score="2" Body="&lt;p&gt;Answers:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generally its the former. The next layer would learn at each filter how to merge the channels of the previous layer, that is why in a 2D convolution the kernel is a 3-dimensional tensor. But the number of parameters is &lt;span class=&quot;math-container&quot;&gt;$nmc_ic_{i+1}$&lt;/span&gt; at the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; layer (this is ignoring bias). lets assume all channels are &lt;span class=&quot;math-container&quot;&gt;$O(c)$&lt;/span&gt; then the spatial complexity becomes &lt;span class=&quot;math-container&quot;&gt;$O(knmc^2)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of layers.   &lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;a)&lt;/em&gt; The first convolutional filter would have kernel size of (w,h,3,10) where w and h are the kernel sizes of the 2d convolution (often in practice is 3). So there are 10 filters of size (w,h,3), but the # of parameters is w*h*30 (once again for ease, ignoring bias). The second layer though, since it is working on a layer with 10 channels, will have kernel (w,h,10,10).&lt;br&gt;&#xA;&lt;em&gt;b)&lt;/em&gt; I think you need to go back and look at what a convolution does (in your setting specifically a 2D convolution). Each &lt;strong&gt;filter&lt;/strong&gt; works on every &lt;strong&gt;channel&lt;/strong&gt; of the previous layer. Each &lt;strong&gt;channel&lt;/strong&gt; of the last convolutional layer refers to a &lt;strong&gt;single&lt;/strong&gt; filter that convolved over the entire previous layer to that. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-20T12:43:46.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13464" PostTypeId="1" CreationDate="2019-07-20T14:06:58.600" Score="2" ViewCount="109" Body="&lt;p&gt;I'm trying to implement the proximal policy optimization (PPO) algorithm. I'm confused on how to make it work with continuous action space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For discrete action space, the output of the network is the probability for every available action, then I choose the next action based on this probability. The ratio, in the objective function, is the ratio between the action &lt;strong&gt;probability&lt;/strong&gt; of the new policy and the action &lt;strong&gt;probability&lt;/strong&gt; between the old policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For continuous action space, from what I understand, the output of the network should be the action itself. How should this ratio (or the objective function itself) look like in that case?&lt;/p&gt;&#xA;" OwnerUserId="26654" LastEditorUserId="2444" LastEditDate="2019-07-20T19:30:52.873" LastActivityDate="2019-07-20T19:30:52.873" Title="What is ratio of the objective function in the case of continuous action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13465" PostTypeId="2" ParentId="13454" CreationDate="2019-07-20T15:34:48.060" Score="3" Body="&lt;h3&gt;tl;dr&lt;/h3&gt;&#xA;&lt;p&gt;It helps to think that the &lt;em&gt;channels&lt;/em&gt; dimension of a convolutional layer works like a &lt;strong&gt;fully connected&lt;/strong&gt; layer (i.e. the layer computes the weighted sum over all channels).&lt;/p&gt;&#xA;&lt;h3&gt;For a single pixel...&lt;/h3&gt;&#xA;&lt;p&gt;Let's consider a single pixel (e.g. the top left pixel). This pixel has &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; different values, where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; are the number of channels. In order to produce the result of a single filter, the layer takes the weighted sum of these &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; pixels. It does this by actually having &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; weights, multiplying the pixel values with their corresponding weights and summing them together.&lt;/p&gt;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&lt;p&gt;Let's say you have an &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 10$&lt;/span&gt; tensor as an input to a convolutional layer with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; filter and a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; kernel. For creating its output the layer has &lt;span class=&quot;math-container&quot;&gt;$3 \times 3 \times 10 = 90$&lt;/span&gt; weights, i.e. a different &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; kernel for each of the &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; input channels. To create its output, the layer performs the convolution operation &lt;strong&gt;separately&lt;/strong&gt; on each of the input channels (each with its corresponding weight matrix) and creates this way &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; feature maps which are &lt;strong&gt;summed&lt;/strong&gt; together.&lt;/p&gt;&#xA;&lt;p&gt;Now imagine the layer has &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; filters instead of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Nothing changes, just that the same procedure is done &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; times with &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; different sets of weights So the total number of weights in the layer in this case is &lt;span class=&quot;math-container&quot;&gt;$3 \times 3 \times 10 \times 20 = 1800$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;To answer your questions...&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;(1)&lt;/strong&gt; You have a grayscale image &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 1$&lt;/span&gt; and it passes through the first convolution layer (which has &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; filters). This layer will perform the convolution operation on the input image &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; times &lt;strong&gt;independently&lt;/strong&gt; and produce &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; feature maps, i.e. an output tensor of &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 10$&lt;/span&gt;. Now this in fed into the second convolution layer, which again has &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; filters. &lt;strong&gt;For each&lt;/strong&gt; filter the layer will perform the convolution operation on the &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; input feature maps independently and will sum the corresponding pixels together to form a &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 1$&lt;/span&gt; feature map. It will perform this procedure &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; times and generate an output tensor of &lt;span class=&quot;math-container&quot;&gt;$n \times m \times 10$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;(2)&lt;/strong&gt; Yes, each layer actually has &lt;span class=&quot;math-container&quot;&gt;$30$&lt;/span&gt; filters, a total of &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; for each of the R, G, B channels. The layer just sums the results of the R, G, B channels to produce a single feature map. For the second part, if I got it right it's pretty much what you said but it sums the maps instead of averaging them&lt;/p&gt;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&lt;p&gt;I'd recommend checking &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford's CS231 notes on convolution layers&lt;/a&gt;, which explain it in much detail and also have numerical examples to confirm if you've got it right.&lt;/p&gt;&#xA;&lt;p&gt;You could also check &lt;a href=&quot;https://stats.stackexchange.com/a/335327/119015&quot;&gt;this answer&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-07-20T15:34:48.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13466" PostTypeId="1" CreationDate="2019-07-20T17:20:04.543" Score="1" ViewCount="48" Body="&lt;p&gt;Looking for a solution to my below game problem. I believe it to require some sort of reinforcement learning, dynamic programming, or probabilistic programming solution, but am unsure... This is my original problem, and is part of an initiative to create &lt;em&gt;&quot;unique and challenging problem that you're able to conceptualize and then solve. 3 Judging criteria: uniqueness, complexity, and solution (no particular weighting and scoring may favor uniqueness/challenge over solution&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inspirations: Conway's Game of Life, DeepMind's Starcraft Challenge, deep Q-learning, probabilistic programming&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;BEAR SURVIVAL&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;A bear is preparing for hibernation. A bear must reach life-strength 1000 in order to rest &amp;amp; survive the winter. A bear starts off at a health of 500. A bear explores an environment of magic berries. A bear makes a move  (chosen randomly with no optional direction) and comes across a berry each time. There are 100 different types of berries that all appear across the wilderness equally and infinitely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A magic berry always consumes 20 life from the bear upon arrival (this not an energy cost for moving and we should not think of it as such). A bear may then choose to give more, all, or none of its remaining life to the berry. If eaten, the berry may provide back to the bear 2x the amount of life given. Berries, however, are not the same and a bear knows this. A bear knows that any berry has some percentage of being poisonous.  Of the 100 different types of berries, each may be 0%-100% poisonous. A berry that is 0% poisonous is the perfect berry and a bear knows that it should commit all of its remaining life to receive max health gain. If a bear wants to eat the berry, it must commit at least 20 more health. Again, a bear does not have to eat the berry, but if it chooses not to, it walks away and does not get the original 20 back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example:&lt;/strong&gt; On a bear's first move (at 500 life), it comes across a magic berry and the berry automatically takes 20 life.  The bear notices that the berry is 0% poisonous, the perfect berry, and gives its remaining 480 health, eats the berry, and then receives 1000 health gain. The bear has reached it's goal, hibernates, and wins the game. However, if that first berry was 100% poisonous, the anti-berry, and the bear committed all of its remaining life it would've received back 0 health gain, died, and lost the game. A bear knows to never eat the anti-berry. It knows it can come across any poisonous value from 0-100 (3,25,52,99, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bear must be picky &amp;amp; careful, but also bold &amp;amp; smart about how much life it wants to commit per berry, per move. A bear knows that if it never eats, it will eventually die as it loses 20 health per berry, per move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it's important for an individual bear to survive, it is even more important for the bear population to not go extinct. A population is going extinct if they lose over half of the population that year. Bears in a population, and their consumption of berries, are completely independent of each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;May we find a bear's optimal strategy for committing health &amp;amp; eating&#xA;berries to reach 1000 health gain? &lt;/li&gt;&#xA;&lt;li&gt;Is the bear population eventually doomed to a unfavorable environment?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus Complexity:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Winter is coming, and conditions grow progressively harsher over time. A bear knows that every 10 moves, each berry will consume &lt;strong&gt;20 * (fib(i)/ environmentFactor)&lt;/strong&gt;. &lt;em&gt;fib(i) stands for fibonacci-sequence at index i, starting at 1. For all indexes where the progression is less than 20, a berry's initial health consumption remains at 20. environmentFactor is a single environment's progressive-harshness variable (how harsh winter becomes over time).&lt;/em&gt; The bear population is currently in an environment with environmentFactor of 4. Spelled out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Moves 01-10: Berries consume 20 --  20*(1/4)&#xA;Moves 10-20: Berries consume 20 --  20*(1/4)&#xA;Moves 20-30: Berries consume 20 --  20*(2/4)&#xA;Moves 30-40: Berries consume 20 --  20*(3/4)&#xA;Moves 40-50: Berries consume 25 --  20*(5/4)&#xA;Moves 50-60: Berries consume 40 --  20*(8/4)&#xA;Moves 60-70: Berries consume 65 --  20*(13/4)&#xA;Moves 70-80: Berries consume 105 --  20*(21/4)&#xA;Moves 80-90: Berries consume 170 --  20*(34/4)&#xA;Moves 90-100: Berries consume 275 --  20*(55/4)&#xA;Moves 100-110: Berries consume 445 --  20*(89/4)&#xA;... and so on ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Same questions as above, with a third: if this environment is proven unfavorable, and extinction unavoidable, what maximum environment/environmentFactor must the bear population move to in order to avoid extinction? (this may or may not exist if a berry's requirement of 20 initial life is always unfavorable without any progression).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Details:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;QUESTION:&#xA;  Can you give an example of what happens when a bear eats a semi-poisonous berry (e.g. 20%)?&#xA;  - Also, is the bear always immediately aware of the poison value of berries?&#xA;  - In this, it also seems like you're using health, life, strength, life-strength, health-gain, etc interchangeably. Are they all the same&#xA;  thing?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;ANSWER:  &lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;if the bear eats the poison berry of 20%, then it becomes a probability problem of whether or not the berry provides back life or&#xA;  keeps the health amount committed by the bear. Example: the bear is at  400, the next move &amp;amp; berry take the initial  20 (bear is at 380 now), the bear&#xA;  decides to commit an additional 80 (now at 300) and eat the berry.&#xA;  8/10 times the berry will return to the bear 200 (2x 100 committed --&#xA;  bear ends turn at 500), 2/10 times the berry&#xA;  returns nothing and the bear must move on with 300 life.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;the bear is always immediately aware of the poison value of a berry.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;life/health/strength are all the same thing.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="27322" LastEditorUserId="27322" LastEditDate="2019-07-20T20:35:20.403" LastActivityDate="2019-07-20T20:35:20.403" Title="Unique game problem (ML, DP, PP etc)" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;q-learning&gt;&lt;probability&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13467" PostTypeId="2" ParentId="12957" CreationDate="2019-07-20T18:08:00.453" Score="0" Body="&lt;p&gt;If you prefer to use Python (rather than Java, which is used to implement &lt;a href=&quot;https://moa.cms.waikato.ac.nz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOA&lt;/a&gt;, which is suggested in &lt;a href=&quot;https://ai.stackexchange.com/a/12958/2444&quot;&gt;the other answer&lt;/a&gt;), you might want to have a look at the Python's &lt;a href=&quot;https://pypi.org/project/creme/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;creme&lt;/code&gt;&lt;/a&gt; library, whose API is described at &lt;a href=&quot;https://creme-ml.github.io/api.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://creme-ml.github.io/api.html&lt;/a&gt;, which is a library for incremental and online learning. In particular, you might be interested in the class &lt;a href=&quot;https://creme-ml.github.io/generated/creme.multiclass.OneVsRestClassifier.html#creme.multiclass.OneVsRestClassifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;OneVsRestClassifier&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-20T18:08:00.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13468" PostTypeId="5" CreationDate="2019-07-20T18:30:23.987" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-20T18:30:23.987" LastActivityDate="2019-07-20T18:30:23.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13469" PostTypeId="4" CreationDate="2019-07-20T18:30:23.987" Score="0" Body="For questions related to the reinforcement learning algorithm called proximal policy optimization (PPO), which was introduced in the paper &quot;Proximal Policy Optimization Algorithms&quot; (2017) by John Schulman et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-20T18:30:23.987" LastActivityDate="2019-07-20T18:30:23.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13470" PostTypeId="1" AcceptedAnswerId="13471" CreationDate="2019-07-20T20:10:41.933" Score="2" ViewCount="63" Body="&lt;p&gt;I have been reading up on CNNs. One of the different confusing things has been that people always talk of normalization &lt;strong&gt;layers&lt;/strong&gt;. A common normalization layer is a ReLU layer. But I never encountered an explanation of why all of a sudden, activation functions become their own layers in CNNs, while they are only &lt;strong&gt;parts&lt;/strong&gt; of a fully connected layer in MLPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the reason for having dedicated activation layers in CNNs rather than applying the activation to the output volume of a convolutional layer as part of the convolutional layer, as it is the case for dense layers in MLPs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess, in the end, there is no functional difference. We could just as well have separate activation layers in MLPs rather than activation functions in their fully connected layers. But this difference in the convention is irritating still. Well, assuming it only &lt;strong&gt;is&lt;/strong&gt; an artifact of the convention.&lt;/p&gt;&#xA;" OwnerUserId="20150" LastEditorUserId="2444" LastEditDate="2019-07-20T20:13:20.677" LastActivityDate="2019-07-20T20:36:54.257" Title="Why are activation functions independent layers in CNNs rather than part of convolutional layers?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;definitions&gt;&lt;activation-function&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13471" PostTypeId="2" ParentId="13470" CreationDate="2019-07-20T20:25:40.017" Score="1" Body="&lt;p&gt;These are just two &lt;em&gt;equivalent&lt;/em&gt; interpretations (or illustrations) of the application of an activation function. In other words, in a multi-layer perceptron (MLP), you could also illustrate the application of the activation function as a separate layer that follows a linear combination layer. However, in the context of MLPs, the math is relatively simple and &lt;em&gt;elegant&lt;/em&gt;, so a fully-connected layer of an MLP can simply be represented as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma \left(\mathbf{W} \mathbf{X} + \mathbf{b} \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is some activation function and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W} \mathbf{X}$&lt;/span&gt; is the linear combination of the inputs, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt;, and the weights, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; is a bias. You could even represent a full or complete MLP (and not just one fully-connected layer) as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Function_composition&quot; rel=&quot;nofollow noreferrer&quot;&gt;composite (or nested) function&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of convolutional neural networks (CNNs), people might illustrate the application of the activation function as a separate layer because the application of an activation function to the result of the convolution operation of the CNN is &lt;strong&gt;optional&lt;/strong&gt; and &lt;strong&gt;out-of-favor&lt;/strong&gt; (as stated in this article &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#norm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/convolutional-networks&lt;/a&gt;), as opposed to the case of MLPs, where activations functions usually follow the linear combination. However, note that the last layers of a CNN are usually fully-connected layers (and not convolutional or pooling layers), that is, they are a linear combination of their input and their weights followed by an application of an activation function.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-20T20:36:54.257" LastActivityDate="2019-07-20T20:36:54.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13473" PostTypeId="1" CreationDate="2019-07-20T21:04:23.937" Score="1" ViewCount="25" Body="&lt;p&gt;I was training an AI to learn things during its lifetime such as find food and navigate a maze. Behaviors that might change during its lifetime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I hit upon a snag. Some behaviors, like avoiding poisonous snakes, cannot be learned in a lifetime since once bitten by a snake the being is dead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That got me thinking about how to separate out behaviors that must be given to the AI at birth (either by programming or using some evolutionary algorithm) and which behaviors to let the AI learn in its lifetime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there is the matter of when a learned behavior should be able to overrule an innate behavior (if at all).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there much research into this? I'm looking for some method to determine a set of innate behaviors which can't be learned.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2019-07-20T21:41:52.583" LastActivityDate="2019-07-20T21:41:52.583" Title="Is there a rule-of-thumb to determine which behaviours must be learned in a lifetime and which innate?" Tags="&lt;neural-networks&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13474" PostTypeId="2" ParentId="8607" CreationDate="2019-07-20T21:44:31.957" Score="1" Body="&lt;p&gt;You have a small dataset.  Should you even be using neural nets?  Have you done any diagnostics to see if you even have enough data?  Are you using the right metric?  Accuracy is not always the correct metric.  Which weights are you retaining?  You will overfit if you save the weights that produce the lowest training error.  Save the weights that produced the lowest validation error.  L1, L2, and dropout are all great.  So many things not described in the problem...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ultravioletanalytics.com/blog/kaggle-titanic-competition-part-ix-bias-variance-and-learning-curves&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ultravioletanalytics.com/blog/kaggle-titanic-competition-part-ix-bias-variance-and-learning-curves&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering why you're not trying interpretable models to see if the resulting weights for the features make sense.  Also if your comparing all those models and parameters, set your random initial starting point to be the same by setting the seed.  I also hope you are using the same training set for each model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably need more data...&lt;/p&gt;&#xA;" OwnerUserId="27326" LastActivityDate="2019-07-20T21:44:31.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13475" PostTypeId="1" CreationDate="2019-07-21T00:30:23.427" Score="1" ViewCount="163" Body="&lt;p&gt;The computer is made up of NAND conditional statements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &quot;conditional statements that read and write conditional statements&quot; must exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example&#xA;&quot;Modify conditional statement 2 if conditional statement 1 exists.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, is there any artificial intelligence using this metaprogramming?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastActivityDate="2019-07-21T00:30:23.427" Title="Is it possible to create intelligence with metaprogramming?" Tags="&lt;artificial-neuron&gt;&lt;programming-languages&gt;&lt;computer-programming&gt;&lt;artificial-life&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13476" PostTypeId="1" CreationDate="2019-07-21T01:59:07.673" Score="1" ViewCount="7709" Body="&lt;p&gt;I am detecting wheels with a deep learning algorithm. The algorithm gives me the coordinates of those rectangles. I want to keep data that is in the rectangles of the image. I created rectangles as a mask of the area I want to keep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the output of my system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read my image&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;im = cv2.imread(filename)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I created the rectangles with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;height,width,depth = im.shape&#xA;cv2.rectangle(img,(384,0),(510,128),(0,255,0),3)&#xA;cv2.rectangle(rectangle,(width/2,height/2),200,1,thickness=-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I mask out the data outside of the rectangle from the original image? and keep those rectangles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited: I wrote this code and it only gives me one wheel. How can I have multiple masks and get all the wheels?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  mask = np.zeros(shape=frame.shape, dtype=&quot;uint8&quot;)&#xA;&#xA;# Draw a bounding box.&#xA;# Draw a white, filled rectangle on the mask image&#xA;cv.rectangle(img=mask,&#xA;             pt1=(left, top), pt2=(right, bottom),&#xA;             color=(255, 255, 255),&#xA;             thickness=-1)&#xA;&#xA;&#xA;# Apply the mask and display the result&#xA;maskedImg = cv.bitwise_and(src1=frame, src2=mask)&#xA;&#xA;cv.namedWindow(winname=&quot;masked image&quot;, flags=cv.WINDOW_NORMAL)&#xA;cv.imshow(winname=&quot;masked image&quot;, mat=maskedImg)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="20025" LastEditDate="2019-07-22T16:36:41.467" LastActivityDate="2019-07-22T16:36:41.467" Title="How to Mask an image using Numpy/OpenCV?" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-06-02T22:38:59.247" ContentLicense="CC BY-SA 4.0" />
  <row Id="13477" PostTypeId="2" ParentId="10500" CreationDate="2019-07-21T03:09:19.453" Score="1" Body="&lt;p&gt;Yes you can, provided you know about &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;. Expression &lt;span class=&quot;math-container&quot;&gt;$X3 = f(X1, g(X1))$&lt;/span&gt;can be written as &lt;span class=&quot;math-container&quot;&gt;$X3 = h(X1)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; takes into account both &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;. After this finding the PDF is simple by differentiating the CDF:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ F_{X3} (x3) = P(X3 \leq x3) =  P(h(X1) \leq x3) = P(X1 \leq h^{-1}(x3))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac {d F_{X3} (x3)}{dx3}  = \frac {d P(X1 \leq h^{-1}(x3))}{dx3} = f_{X3}(x3)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: The conventions followed are the same as used in the field of Probablity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Take care of the function inversion step in non-monotonic cases)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check these &lt;a href=&quot;https://www.youtube.com/watch?v=X-AzW70e2M0&amp;amp;list=PLUl4u3cNGP60hI9ATjSFgLZpbNJ7myAg6&amp;amp;index=115&quot; rel=&quot;nofollow noreferrer&quot;&gt;lectures&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-21T03:19:29.167" LastActivityDate="2019-07-21T03:19:29.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13478" PostTypeId="1" CreationDate="2019-07-21T07:14:35.417" Score="1" ViewCount="110" Body="&lt;p&gt;I've built an A2C model whose actor's network has two different kinds of discrete actions, so the critic would take state and action (note that critic takes 2 actions because in each timestep we will do two kinds of actions) to predict the advantage for each of these two different kinds of actions. However, my problem is when the critic network is about to train itself with discounted rewards. It only has the reward of each timestep, so cannot determine which of the two kinds of our actions contributed more to this reward, so both of its outputs (the advantage for the action of kind 1 and the advantage for the action of kind 2) will be changed in the same direction, so output of critic would be biased with its initialization values. How can I solve this problem, that is, to distinguish between the contribution amount of each of these two kinds of actions to the outcome?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An example of my problem:&lt;/strong&gt;&lt;br&gt;&#xA;Consider we have 10 cubes 3 boxes. In each time step, we have to choose between 10 cubes and choose between 3 boxes to place the selected cube in the selected box. So we have 2 kinds of actions here: one to pick a cube and the second to put it in one the boxes. Each box only has the capacity of only 4 cubes, and one of the cubes is so big that don't fit in any of the boxes. The reward of each time step will be the negative number of cubes that are not placed in a box, so because of the bigger cube, the agent won't get reward 0 ever. Consider a scenario that a box already contains 4 cubes and we choose that box to place the chosen cube (one of the small cubes) in it, we can't and the time will proceed. Another scenario is when we choose the bigger cube so no matter which box we choose, we cannot place it and the time will proceed. How the agent can distinguish which of these two kinds of actions contributed more to the reward?&lt;/p&gt;&#xA;" OwnerUserId="27329" LastEditorUserId="27329" LastEditDate="2019-07-21T09:38:13.640" LastActivityDate="2019-07-21T09:38:13.640" Title="Reward problem in A2C with multiple simultaneous discrete actions" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13481" PostTypeId="2" ParentId="13476" CreationDate="2019-07-21T09:23:04.980" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;cv2.bitwise_and&lt;/code&gt; and pass rectangle as a mask.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;im = cv2.imread(filename)&#xA;height,width,depth = im.shape&#xA;cv2.rectangle(img,(384,0),(510,128),(0,255,0),3)&#xA;cv2.rectangle(rectangle,(width/2,height/2),200,1,thickness=-1)&#xA;&#xA;masked_data = cv2.bitwise_and(im, im, mask=rectangle)&#xA;&#xA;cv2.imshow(&quot;masked_data&quot;, masked_data)&#xA;cv2.waitKey(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27313" LastActivityDate="2019-07-21T09:23:04.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13482" PostTypeId="1" CreationDate="2019-07-21T09:42:14.717" Score="2" ViewCount="60" Body="&lt;p&gt;I'm trying to use gradient boosting and I'm using &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sklearn&lt;/a&gt;'s &lt;code&gt;GradientBoostingClassifier&lt;/code&gt; class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I'm having a data frame with 5 columns and I want to use these columns as features. I want to use them continuously. I mean I want each tree based classifier uses the residue of the previous tree which is based on the previous feature. As I know, by default, this classifier uses a feature and passes the residue of the previous tree to the next tree, which both are based on a single feature. How can I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I do this on my own or there is a library which does this?&lt;/p&gt;&#xA;" OwnerUserId="27332" LastEditorUserId="27332" LastEditDate="2019-07-22T12:54:47.873" LastActivityDate="2019-07-22T12:54:47.873" Title="How can I use gradient boosting with multiple features?" Tags="&lt;machine-learning&gt;&lt;features&gt;&lt;scikit-learn&gt;&lt;gradient-boosting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13483" PostTypeId="5" CreationDate="2019-07-21T11:44:17.523" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-21T11:44:17.523" LastActivityDate="2019-07-21T11:44:17.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13484" PostTypeId="4" CreationDate="2019-07-21T11:44:17.523" Score="0" Body="For questions related to gradient boosting, which is a machine learning technique that can be used for regression and classification problems and which produces a prediction model in the form of an ensemble of other smaller prediction models (typically decision trees)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-21T11:44:17.523" LastActivityDate="2019-07-21T11:44:17.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13485" PostTypeId="5" CreationDate="2019-07-21T11:45:40.720" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-21T11:45:40.720" LastActivityDate="2019-07-21T11:45:40.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13486" PostTypeId="4" CreationDate="2019-07-21T11:45:40.720" Score="0" Body="For questions related to the Python's package scikit-learn (or sklearn)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-21T11:45:40.720" LastActivityDate="2019-07-21T11:45:40.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13487" PostTypeId="1" CreationDate="2019-07-21T12:51:44.853" Score="0" ViewCount="53" Body="&lt;p&gt;Does a model trained in keras (tensorflow backend) saves the weights with max accuracy and minimum losses or does it simply saves the weights from the last epoch? If it is the latter then how do I save the best weights?&lt;/p&gt;&#xA;" OwnerUserId="27336" LastEditorUserId="27336" LastEditDate="2019-07-21T13:03:09.383" LastActivityDate="2020-08-14T18:02:15.210" Title="Is a trained model in keras is saved with the weights for max accuracy?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13488" PostTypeId="2" ParentId="13376" CreationDate="2019-07-21T13:51:04.653" Score="3" Body="&lt;p&gt;The documentary &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Plug_%26_Pray&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plug &amp;amp; Pray&lt;/a&gt;&lt;/strong&gt; (2010), directed and written by Jens Schanze, with main protagonists Joseph Weizenbaum (the creator of &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELIZA&lt;/a&gt;) and the futurist Raymond Kurzweil, is about the promise, problems and ethics of artificial intelligence and robotics. This documentary won several awards, including the Bavarian Film Award 2010 for &quot;best documentary&quot;. Here's the &lt;a href=&quot;https://www.youtube.com/watch?v=ecPEkG2Pclg&quot; rel=&quot;nofollow noreferrer&quot;&gt;official trailer&lt;/a&gt; of the movie.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-21T13:51:04.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13489" PostTypeId="1" CreationDate="2019-07-21T14:22:58.110" Score="1" ViewCount="70" Body="&lt;p&gt;I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The learn data only contains extreme situations where each row contains one and only one output value set to 100.&#xA;The data to predict is expected to contain more nuances in the output values.&#xA;All my attempts lead to very low accuracy as the predicted output sum is almost never a 100. Even when I try to normalize the predicted output, the predictions show very poor accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I try to organize the data with 2 angles instead and deduct the 3rd angle as the remainder in a circle? How to normalize those 2 angles and how to make sure their sum will not exceed the maximum value making the 3rd angle negative?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Illustration of learn data extract (4 input columns and 3 output columns).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0    1    2    3    100  0    0&#xA;4    5    6    7    0    100  0&#xA;8    9    0    1    0    0    100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Illustration of desired output predictions where each line sums as 100:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;7    83   10&#xA;39   12   49&#xA;68   24   8&#xA;28   72   0&#xA;86   6    8&#xA;32   49   19&#xA;0    0    100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19852" LastEditorUserId="19852" LastEditDate="2019-07-27T16:33:07.647" LastActivityDate="2019-07-27T16:33:07.647" Title="How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13490" PostTypeId="2" ParentId="13487" CreationDate="2019-07-21T15:07:40.280" Score="1" Body="&lt;p&gt;The weights update by default, they do not get saved anywhere, they are just in the programs memory.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you can choose to save the weights manually or define a callback to do so. You should checkout the ModelCheckpoint callback. It will give you the power to decide what criterion you want to save the weights on.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-21T15:07:40.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13491" PostTypeId="2" ParentId="10942" CreationDate="2019-07-21T15:20:37.927" Score="0" Body="&lt;p&gt;You cannot achieve that from a single &lt;code&gt;model.evaluate(...) call&lt;/code&gt;. But you could always do &lt;code&gt;evals = [model.eval(X[i:i+1], Y[i:i+1]) for i in range(len(X))]&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note if you get the tensors yourself, you can get the backend session from &lt;code&gt;keras.backend.get_session()&lt;/code&gt; and get anything you want by making your own &lt;code&gt;sess.run() calls&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-21T15:20:37.927" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13494" PostTypeId="1" CreationDate="2019-07-22T02:38:37.400" Score="2" ViewCount="15" Body="&lt;p&gt;I've been reading about Mixture of Expert models, and I've noticed that there is very little new work being produced in this subfield. Has there been a better method discovered? Why aren't more people doing stuff in this area?&lt;/p&gt;&#xA;" OwnerUserId="27240" LastActivityDate="2019-07-22T02:38:37.400" Title="Current state of MoE models" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13495" PostTypeId="1" AcceptedAnswerId="13519" CreationDate="2019-07-22T03:23:57.923" Score="2" ViewCount="56" Body="&lt;p&gt;Consider AlexNet, which has 1000 output nodes, each of which classifies an image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mAVsh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mAVsh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I have been having with training a neural network of similar proportions, is that it does what any reasonable network would do: it finds the &lt;em&gt;easiest&lt;/em&gt; way to reduce the error which happens to be setting all nodes to 0, as in the vast majority of the time, that's what they'll be. I don't understand how a network where 999 times out of 1000, the node's output is 0, could possibly learn to make that node 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But obviously, it's possible, as AlexNet did very well in the 2012 ImageNet challenge. So I wanted to know, how would one train a neural network (specifically a CNN) when for the majority of the inputs the desired value for an output node is 0?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2020-06-12T23:52:21.370" LastActivityDate="2020-06-12T23:52:21.370" Title="How is a neural network where the majority of inputs are 0 trained?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;alexnet&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13497" PostTypeId="2" ParentId="13443" CreationDate="2019-07-22T03:57:49.357" Score="2" Body="&lt;p&gt;Simon Krannig's answer provides the math notation behined exactly what is going on, but since you still seem a bit confused, I've made a visual representation of a neural network using &lt;em&gt;only weights with no activation function&lt;/em&gt;. See below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vkQkI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vkQkI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm fairly sure it as you suspected: At each neuron, you take the &lt;strong&gt;sum&lt;/strong&gt; of the inputs of the previous layer multiplied by the weight that connects that specific input to said neuron, where each input has its own unique weight for every one of its outgoing connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a bias, you would do the exact same math as shown in the above image, but once you find the final value (0.2, -0.15, 0.16 and -0.075, the output layer doesn't have a bias) you would add the bias to the total value. So see below for an example including a bias:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Tyxsy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Tyxsy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt; I did not update the outputs at each layer to include the bias because I can't be bothered redrawing this in paint. Just know that the final value for all the nodes with the brown bias haven't carried over to the next layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, if you were to include an activation function, you would finally take your value and put it through. So including the bias', looking at node 1 of layer 2, it would be (lets pretend your activation function is a sigmoid):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigmoid((0.4*0.5)+0.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and for layer 3 node 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigmoid(((0.6*0.2)+(1.3*-0.15))-0.4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is how you would do a forward pass of a simple neural network.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-07-22T03:57:49.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13498" PostTypeId="1" CreationDate="2019-07-22T05:14:52.053" Score="2" ViewCount="58" Body="&lt;p&gt;I need some advice on what AI methods would be suited to the identification of a recipient of a document, where the format of the documents may vary. &lt;/p&gt;&#xA;" OwnerUserId="21695" LastEditorUserId="1671" LastEditDate="2019-07-24T16:47:35.380" LastActivityDate="2019-07-24T16:47:35.380" Title="Machine learning methods to identify the recipient of a document?" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13499" PostTypeId="1" CreationDate="2019-07-22T06:03:38.093" Score="4" ViewCount="82" Body="&lt;p&gt;Given that recurrent neural networks are equivalent to a Turing machine, then why isn't the evolutionary Turing machine, e.g. described in the paper &lt;a href=&quot;https://www.researchgate.net/publication/224302029_Evolution_of_evolution_Self-constructing_Evolutionary_Turing_Machine_case_study&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolution of evolution: Self-constructing Evolutionary Turing Machine case study&lt;/a&gt; (2007), mainstream?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2019-07-23T18:25:48.827" LastActivityDate="2019-07-23T18:25:48.827" Title="Why isn't the evolutionary Turing machine mainstream?" Tags="&lt;neural-networks&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13500" PostTypeId="2" ParentId="11534" CreationDate="2019-07-22T06:11:56.330" Score="2" Body="&lt;p&gt;I’ll have a stab at this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cognitive performance in narrow domains is determined by competency, efficiency and speed. Take calculating numbers, extremely narrow domain but compared to humans the ability of a calculator to calculate numbers exceeds normal human performance, it is much competent in terms of speed. In a bit broader domain, AlphaGo has defeated Go players, which is more difficult than chess, and requires intuition. In fact, there is an instance where the AlphaGo makes a long-term move that was &lt;a href=&quot;https://en.m.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol?wprov=sfti1&quot; rel=&quot;nofollow noreferrer&quot;&gt;previously unimagined&lt;/a&gt;. In all domains however Humans are well rounded, therefore Human Intelligence is called general intelligence. An AlphaGo or Calculator cannot speak eloquently or make music, but AIs are gaining pace in these areas too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree with @nbro that Bostrom wants to keep the interpretation of Superintelligence open. But if there is a rough category, these are- &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;ANI-&lt;/strong&gt; &lt;code&gt;Artificial Narrow Intelligence&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;AGI-&lt;/strong&gt; &lt;code&gt;Artificial General Intelligence&lt;/code&gt;: Where the AI’s performance is at par with humans. After AGI, it quickly takes off to SI.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;SI-&lt;/strong&gt; &lt;code&gt;Superintelligence&lt;/code&gt;: Superintelligence is beyond our imagination, we have not figured out yet what will a SI do, think or want.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While these categories are discrete, the functions of strength are not. I’d say they are rather discrete to continuous, because if you look at the computing power plots that follow Moore’s Law, a similar exponential graph can be drawn for AI’s performance towards general intelligence. In that &lt;a href=&quot;https://images.app.goo.gl/7a7e1ar6XwSMPh9M9&quot; rel=&quot;nofollow noreferrer&quot;&gt;graph&lt;/a&gt;, it seems the AI’s performance starts with discrete performance points, and then as it takes off, it becomes continuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why the term &lt;code&gt;Singularity&lt;/code&gt; is often associated with &lt;code&gt;Superintelligence&lt;/code&gt;. I hope this answers your question.&lt;/p&gt;&#xA;" OwnerUserId="27299" LastActivityDate="2019-07-22T06:11:56.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13502" PostTypeId="1" CreationDate="2019-07-22T10:48:59.513" Score="0" ViewCount="135" Body="&lt;p&gt;Let's say we have a task where the cost depends entirely on the path length to a terminal state, so the goal of an agent would be to take actions to reach terminal state as quickly as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let us say, we know the optimal path length is of length &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, and there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; such paths possible. Each state has 5 possible actions. Let's say the scheme we are using to find optimal policy is On-Policy MC/TD(n) along with GLIE Policy improvement (Generalised Policy Iteration).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first Policy Iteration Step, each actions are equally likely, there for the probability of sampling this optimal path (or the agent discovering this path) is &lt;span class=&quot;math-container&quot;&gt;$n* \frac {1}{5^{10}} \approx n* \frac {1}{2^{20}}$&lt;/span&gt;. So, according to probability theory we need to sample around &lt;span class=&quot;math-container&quot;&gt;$2^{20}/n$&lt;/span&gt; steps to atleast discover one of the best paths (worst case scenario).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, it is not possible to go through such huge number of samplings, let's say we do not sample the path, thus in the next Policy Iteration step (after GLIE Policy Improvement) some other sub-optimal path will have a higher probability of being sampled than the optimal path, hence the probability falls even lower. So, like this there is a considerably high probability that we may not find the best path at all, yet theory says we will find &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt; which indicates the best path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is wrong in my reasoning here?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-07-29T16:00:44.880" Title="Why is On-Policy MC/TD Algorithm guaranteed to converge to optimal policy?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13503" PostTypeId="1" CreationDate="2019-07-22T11:27:45.560" Score="1" ViewCount="12" Body="&lt;p&gt;I have the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get a 360 RGB image in a room. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I've the 3D model of this room, hence, I can generate a 3D nominal mask of the room (1-wall, 2-ceiling, 3-floor, 4-door, etc..) in a specific location (x0,y0,z0,roll0,pitch0,yaw0)&lt;/li&gt;&#xA;&lt;li&gt;I can also generate the depth map of this location.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My model has to predict whether the generated mask+depth match with the RGB frame, and &lt;strong&gt;if not&lt;/strong&gt; what are the &lt;em&gt;dx, dy, d_yaw&lt;/em&gt; of the offset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've implemented pix2pix discriminator that receives a concatenated tensor of the [RGB, one-hot mask, depth] and yields the (dx, dy, d_yaw).&#xA;obviously, if there is a perfect match, dx=dy=dyaw=0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately my model isn't converging. I've tried everything, and it's a reasonable &quot;request&quot; from this model, since a human can roughly guess this offset if he looks on the images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what would you suggest? &lt;/p&gt;&#xA;" OwnerUserId="25412" LastActivityDate="2019-07-22T11:27:45.560" Title="Estimating camera's offset to its true position" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13504" PostTypeId="1" CreationDate="2019-07-22T12:36:56.800" Score="2" ViewCount="19" Body="&lt;p&gt;Say I have some data I am trying to learn, and I'm aware that the output is quantised in some way, e.g. I can get only get discrete values (0.1, 0.2, 0.3...0.9) in a finite range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you treat that as regression or classification? In this case the numbers do have a relation to each other e.g. 0.3 is close to 0.4 in meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could treat it as classification with a softmax final layer with N outputs, or could treat it as regression with a linear layer with single output and then somehow quantise the result post-prediction. But my gut feeling is that the fact there is a finite number of answers that that should somehow be used in my model?&lt;/p&gt;&#xA;" OwnerUserId="18372" LastEditorUserId="2444" LastEditDate="2019-07-23T20:23:50.237" LastActivityDate="2019-07-23T20:23:50.237" Title="Should I model a problem with quantised output as classification or regression?" Tags="&lt;classification&gt;&lt;models&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13505" PostTypeId="2" ParentId="11055" CreationDate="2019-07-22T13:47:02.367" Score="1" Body="&lt;p&gt;One way I can think of is to redefine &quot;actions&quot; in a game to make them more fragmented, in such a way that a player has multiple actions per turn. In chess, for example, we can define an action as choosing a tile from which to move, &lt;em&gt;or&lt;/em&gt; choosing the motion from the chosen tile, as 2 separate actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example a turn might consist of the following two actions: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose E4&lt;/li&gt;&#xA;&lt;li&gt;Move forward 2 spaces&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That way there are 64 + 73, rather than 64 * 73 possible actions. The transition model would indicate that it's still the same player's turn after a &quot;tile selection&quot; action is done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this would require increasing the state space in such a way that you can determine which action is legal. So there's a difference between a board state where nothing is &quot;selected&quot; and the same board state where one tile is selected by one player. In the chess example, this would require 2 more boolean CNN layers, one for each player, indicating which tile (if any) is &quot;selected&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I never tried this myself, and I imagine that this might make the learning slower and more difficult, since it requires a deeper tree in the MCTS for the same set of actions.&lt;/p&gt;&#xA;" OwnerUserId="27354" LastEditorUserId="27354" LastEditDate="2019-07-22T14:25:14.467" LastActivityDate="2019-07-22T14:25:14.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13506" PostTypeId="2" ParentId="13504" CreationDate="2019-07-22T15:10:02.320" Score="1" Body="&lt;p&gt;So this is considered &lt;a href=&quot;https://en.wikipedia.org/wiki/Ordinal_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ordinal Regression&lt;/a&gt;. There are many ways to model this type of data, generally in some form of regression setting. I do not recommend the softmax route because as you mentioned, there exists prebuilt correlation to the outputs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Some common ways to approach this&lt;/em&gt;&lt;br&gt;&#xA;(Note that im assuming your looking for methodologies that can be optimized through gradient techniques because of the way you formulated your question)  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Treat it as a normal regressor where you clip the output to your range, and then define thresholds arbitrarily, such as &lt;span class=&quot;math-container&quot;&gt;$.36 \rightarrow .4$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$.34 \rightarrow .3$&lt;/span&gt;, etc..&lt;/li&gt;&#xA;&lt;li&gt;Use a bounded activation function then scale (if your outputs are [0,1,.1], you could use sigmoid and just scale by a factor of 1, but if [0,10,1], you could use sigmoid and scale by 10. Once again youll need to create arbitrary threshold between each 2 points for inference (this can also be incorporated into your loss)  &lt;/li&gt;&#xA;&lt;li&gt;Using the above two methods, but learn the threshold between each 2 ordered points. Have an output layer that learns the ideal threshold for inference  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And if you didnt want to use neural netowrks, each of these approaches have a bayesian analog that work as well! &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-22T15:10:02.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13507" PostTypeId="1" CreationDate="2019-07-22T16:07:29.120" Score="2" ViewCount="139" Body="&lt;p&gt;I have a question about the context of CNN and LSTM. I have trained a CNN network for image classification. However, I would like to combine it with LSTM for visualizing the attention weights. So, I extracted the features from the CNN to put it into LSTM. However, I am stuck at the concept of combinating the CNN with LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;– Do I need to train the whole network again? Or just training the LSTM part is fine?&#xA;– Can I just train the LSTM on image sequences based on classes (for e.g. 1 class has around 300 images) and do predictions later on extracted video frames?&#xA;- In what way can I implement the attention mechanism with Keras? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope you can help me while I struggle with the context of understanding the combination of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;~ EDITED ~&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have trained a resnet50 to classify images. Although, I removed the last dense layer, to extract features from the trained CNN network. Those extracted features will be used as input in the newly created LSTM with attention mechanism to find out where the focus lies. The predictions will be on videos (extracted frames).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image -&gt; extract features (CNN) -&gt; LSTM + Attention (to check where the focus lies during the prediction) -&gt; classify image (output class from N labels) &lt;/p&gt;&#xA;" OwnerUserId="27360" LastEditorUserId="27360" LastEditDate="2019-07-22T23:29:12.327" LastActivityDate="2019-07-22T23:29:12.327" Title="Understanding CNN+LSTM concept with attention and need help" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13508" PostTypeId="1" CreationDate="2019-07-22T17:05:02.367" Score="2" ViewCount="163" Body="&lt;p&gt;I am looking for a rigorous mathematical proof for finding the several local minima of the Hopfield networks. I am searching for something rigorous, a demonstration, not just let the network keep updating its neurons and wait for noticing a stable state of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have look virtually everywhere but I found nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a rigorous proof for Hopfield minima ? Could you give me ideas or resources ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="26719" LastEditorUserId="26719" LastEditDate="2019-07-22T21:49:28.027" LastActivityDate="2019-08-21T22:02:35.320" Title="Is there a rigorous proof for finding Hopfield minima?" Tags="&lt;recurrent-neural-networks&gt;&lt;math&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;hopfield-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13510" PostTypeId="1" CreationDate="2019-07-22T17:37:32.963" Score="1" ViewCount="42" Body="&lt;p&gt;Is Hopfield network more efficient than a naive implementation of Hamming distance that compare an input pattern and return the nearest pattern ?&lt;/p&gt;&#xA;" OwnerUserId="26719" LastActivityDate="2019-07-22T17:37:32.963" Title="Is Hopfield network more efficient than a naive implementation of Hamming distance comparator?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13512" PostTypeId="2" ParentId="13508" CreationDate="2019-07-22T19:35:22.157" Score="2" Body="&lt;p&gt;See the paper &lt;a href=&quot;http://www.paradise.caltech.edu/CNS188/bruck90-conv.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the Convergence Properties of the Hopfield Model&lt;/a&gt; (1990), by Jehoshua Bruck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first section of the paper, J. Bruck describes the Hopfield network (popularized by J. J. Hopfield in 1982 in his paper &lt;a href=&quot;https://www.its.caltech.edu/~bi250c/papers/Hopfield-1982.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural networks and physical systems with emergent collective computational abilities&lt;/a&gt;, hence the name of the network), then he describes the notation that is used throughout the paper and he gives some examples where a simple Hopfield network (with two nodes) converges to stable states (of the network) and cycles (of which the author also gives a definition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual proofs of the convergence properties of Hopfield networks involve the concept of an &lt;strong&gt;energy function&lt;/strong&gt;, but, in this paper, J. Bruck uses an approach (based on an equivalent formulation of the Hopfield network as an undirected graph) that does not involve an energy function, and he unifies three apparently different convergence properties related to Hopfield networks (described in part &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; of the section &lt;em&gt;Introduction&lt;/em&gt;). More specifically, finding the global maximum of the energy function associated with the Hopfield network operating in a serial mode (which is defined in part &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; of the &lt;em&gt;Introduction&lt;/em&gt; section of the paper) is equivalent to find a &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_cut&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimum cut&lt;/a&gt; in the undirected graph associated with this Hopfield network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, note that the proofs of convergence of the Hopfield networks actually depend on the structure of the network (more specifically, its weight matrix). For example, if the weight matrix &lt;span class=&quot;math-container&quot;&gt;$W \in \mathbb{R}^{n \times n}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of nodes in the network) associated with the Hopfield network is a symmetric matrix with the elements of the diagonal being non-negative, then the network will always converge to a stable state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the chapter &lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;13 The Hopfield Model&lt;/a&gt; of the book &lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks - A Systematic Introduction&lt;/a&gt; (1996) by Raul Rojas.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-22T19:52:49.197" LastActivityDate="2019-07-22T19:52:49.197" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13513" PostTypeId="5" CreationDate="2019-07-22T19:41:17.953" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-22T19:41:17.953" LastActivityDate="2019-07-22T19:41:17.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13514" PostTypeId="4" CreationDate="2019-07-22T19:41:17.953" Score="0" Body="For questions related to the Hopfield network, popularized by J. J. Hopfield in the paper &quot;Neural networks and physical systems with emergent collective computational abilities&quot; (1982)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-22T19:41:17.953" LastActivityDate="2019-07-22T19:41:17.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13515" PostTypeId="1" AcceptedAnswerId="13517" CreationDate="2019-07-22T20:47:25.953" Score="5" ViewCount="147" Body="&lt;p&gt;I understand that neural nets are fundamentally interpolative tools. Meaning, given a training dataset, a well trained neural net can approximate values within the domain of the training dataset. However, we are unsure about their behavior once we test against values outside that domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speaking in the context of Imagenet, a NN trained on one of the classes in Imagenet will probably be able to predict an image of the same class outside Imagenet because Imagnet itself covers a huge domain for each class that whatever image we come across in the wild, its features will be accounted for by Imagnet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, this intuition breaks down for me when I talk about simple functions with simple inputs. For example, consider &lt;span class=&quot;math-container&quot;&gt;$sin(x)$&lt;/span&gt;. Our goal is to train a neural net to predict the function given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; with a training domain &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;. Theoretically, the neural net should not be able to predict the values well outside that domain, right? This seems counterintuitive to me because the function behaves in a very simple and periodic way that I find it hard to believe that a neural net cannot figure out the proper transformation of that function even outside the training domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, are neural nets inherently unable to find a generalizable transformation outside the training domain no matter how simple is the function we are trying to approximate? Is this a property of the Deep Learning framework?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there problems where researchers were able to learn a robust generalizable transformation using neural nets outside the training domain? What are the possible conditions so that such results can happen?&lt;/p&gt;&#xA;" OwnerUserId="17582" LastEditorUserId="2444" LastEditDate="2019-07-22T22:41:04.787" LastActivityDate="2019-07-22T23:00:25.647" Title="Why can't neural networks learn functions outside of the specified domains?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13516" PostTypeId="1" CreationDate="2019-07-22T22:35:34.427" Score="1" ViewCount="21" Body="&lt;p&gt;I noticed what I considered a close resemblance of a woman B to another woman A which led to a close relative A labelled C, who B bore an even stronger resemblance to than she did to A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I feel that if I had compared A to C directly I wouldn't have detected the blood relationship so strongly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am just wondering whether there is some mathematical underpinning to my perception and strong intuition that B bore a strong resemblance to A.&lt;/p&gt;&#xA;" OwnerUserId="27370" LastActivityDate="2019-07-22T22:35:34.427" Title="Are there some formulae in facial recognition that are indicators of close kinship?" Tags="&lt;machine-learning&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13517" PostTypeId="2" ParentId="13515" CreationDate="2019-07-22T23:00:25.647" Score="6" Body="&lt;p&gt;The problem you discuss extends past the machine but to the man behind the machine (or woman). ML can be broken down into 3 components, the &lt;em&gt;model&lt;/em&gt;, the &lt;em&gt;data&lt;/em&gt;, and the &lt;em&gt;learning procedure&lt;/em&gt;. This by the way extends to us as well. The model is our brain, the data is our experience and sensory input, and the learning procedure is there but unknown (for now &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt;insert evil laugh&lt;span class=&quot;math-container&quot;&gt;$&amp;gt;$&lt;/span&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your model's inability to understand that it is a sin function is normally by construction. First, though lets start with the fact that if someone showed me &lt;span class=&quot;math-container&quot;&gt;$sin(x)$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt;, a sinusoid is the last thing id think of:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/clB7h.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/clB7h.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA; It looks almost linear. So for sake of argument i'm going to continue on the assumption you meant an entire period (&lt;span class=&quot;math-container&quot;&gt;$[-\pi, \pi])$&lt;/span&gt;. &lt;a href=&quot;https://i.stack.imgur.com/NfQEY.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NfQEY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given this most people who ever got past pre-calculus would assume sinusoid, as youd expect, but let me show you how unstable that idea is. Lets use a full period, shifted by &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi}{2}$&lt;/span&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/MWewD.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MWewD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Now i don't know about you but my spidey-sense would make me think of a Gaussian before anything. This is the same sinusoid along an entire period but just the slight difference changes an entire perspective.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets talk about some of the mathematics. Given the uncountable &lt;span class=&quot;math-container&quot;&gt;$(x,sin(x))$&lt;/span&gt; pairs between &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;, we can extract the function &lt;span class=&quot;math-container&quot;&gt;$sin(x)$&lt;/span&gt; with certainty (assuming infintely differentiable) by simply taking the taylor series along any of the points in that domain. More so, we only need an infinitiscemally small continuous subset of the function to achieve that result, but as a person, would you be able to tell from &lt;span class=&quot;math-container&quot;&gt;$[-.0001, .0001]$&lt;/span&gt;? (i don't think so!), because by default we quantize, and what were gonna give to the computer is a quantization as well. We are giving it a finite countable set of points and expect it to to generalize to an uncountable set. This is silly, since there exists an infinite set of functions that can fit those exact points (concept of overfitting steps in here).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if its so unfeasible for a computer to correctly solve the function, why can us humans extrapolate it so well from the &lt;span class=&quot;math-container&quot;&gt;$[-\pi, \pi]$&lt;/span&gt; domain? The answer is our preset biases. Growing up we have dealt so much with periodicity, specifically sins and cosines, that graphed period triggered something in our brains and we just know what it is. But this isnt always a benefit, i could draw you a low-ordered polynomial on that domain that would make you think that exact same thought making you &lt;strong&gt;wrong&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So going back to my initial point, the problems lies with the models creator and not the model. Like i explained just now, its unfeasible without preset biases to learn the function we want, so what should we do? &lt;em&gt;give up?&lt;/em&gt; &lt;strong&gt;NO&lt;/strong&gt;, its our goal as the model architect to figure out what we want and how we can best achieve it! so if we want it to accomplish this task, lets try our best in modeling these biases. A quick example of how you could do that, is to force the function to fit the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_series&quot; rel=&quot;noreferrer&quot;&gt;fourier&lt;/a&gt; coefficients, thatll probably solve this quite quickly (ofcourse this limits this models ability to solve for other common functions)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Takeaway&lt;/strong&gt;: It's not the model thats struggling, its us using preset biases that the model does not have access to. The solution to this is be clever and figure out how to have it either start with these preset biases or learn them.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-22T23:00:25.647" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13518" PostTypeId="1" CreationDate="2019-07-22T23:54:19.433" Score="1" ViewCount="22" Body="&lt;p&gt;I want to learn how a set of operations (my vocabulary) are composed in a dataset of algorithms (corpus). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithms are a sequence of higher level operations which have varying low-level implementations. I am able to map raw code to my vocabulary, but not all of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. I observe a lossy description of an algorithm that does something:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;X: missing data&#xA;Algo 1: BIND3 EXTEND2 X X ROTATE360 X PUSH&#xA;Algo 2: X X EXTEND2 ROTATE360&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying rotate operation could have very different raw code, but effectively the same function and so it gets mapped to the same operation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to infer what the next operation will be given a sequence of (potentially missing) operations (regions of code I could not map). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. I want a probability distribution over my operations vocabulary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on the best approach here? The standard thing seems to throw out  missing data, but I can still learn in these scenarios. Also, the gaps in the code are non-homogenous--some could do many things, The alternative is to contract the sequences and lose the meaning of the gaps, or to learn an imputation.&lt;/p&gt;&#xA;" OwnerUserId="12849" LastEditorUserId="12849" LastEditDate="2019-07-23T00:12:48.817" LastActivityDate="2019-07-23T00:12:48.817" Title="Language Model from missing data" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13519" PostTypeId="2" ParentId="13495" CreationDate="2019-07-23T05:12:34.483" Score="1" Body="&lt;p&gt;It's the loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was using squared sum error, which I didn't think would have as a negative effect as it does, and I had to come to the explanation in my own time. Here's why:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the perspective of the loss function, 999 times out of 1000, the output should be 0, so there will be an inherent massive bias towards 0 for all the output nodes. But this only occurs if the output nodes are &lt;em&gt;actually&lt;/em&gt; trained when their desired outputs are 0, which is what happens in the case of the squared sum/mean error. However, in the case of cross-entropy loss, which is explained excellently &lt;a href=&quot;https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation&quot;&gt;here&lt;/a&gt;, you can see that the only node that receives gradients is the node that should be trained towards 1. This removes the massive bias towards 0, and punishes confident false positives, making it perfect for a classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to how you would achieve something like this for regression I do not know, but at least this solves the issue for classification problems.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-07-23T05:12:34.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13521" PostTypeId="2" ParentId="3989" CreationDate="2019-07-23T08:55:36.817" Score="0" Body="&lt;p&gt;I am very thankful to the people who responded with hints and suggestions. However, I think what seems most applicable for my case is &lt;a href=&quot;https://probcomp.github.io/Gen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gen&lt;/a&gt; - A general-purpose probabilistic programming system with programmable inference from MIT described in the paper &lt;a href=&quot;http://Gen:%20A%20General-Purpose%20Probabilistic%20ProgrammingSystem%20with%20Programmable%20Inference&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Gen: A General-Purpose Probabilistic Programming System with Programmable Inference&quot;&lt;/a&gt; by M. F. Cusumano-Towner et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case you are looking for something along these lines it looks like a very good start for an application in probabilistic programming.&lt;/p&gt;&#xA;" OwnerUserId="8788" LastActivityDate="2019-07-23T08:55:36.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13523" PostTypeId="1" AcceptedAnswerId="13598" CreationDate="2019-07-23T10:45:22.097" Score="6" ViewCount="1041" Body="&lt;p&gt;Suppose that you show a neural network its own code, and allow it to edit itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a neural network modify its own weights and architecture (the number of layers, the number of neurons per layer, etc.)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without using back propagation and gradient descent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks modify its own weights and architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a valid notion?  Is it being explored? Is this a valid concept in relation to NNs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It feels like the ultimate thing to recognize and edit its own structure as data.&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="23500" LastEditDate="2019-08-08T19:25:09.343" LastActivityDate="2019-08-08T19:25:09.343" Title="Can Neural Networks self-optimize?" Tags="&lt;neural-networks&gt;&lt;theory&gt;" AnswerCount="9" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13524" PostTypeId="1" AcceptedAnswerId="13543" CreationDate="2019-07-23T11:35:34.260" Score="1" ViewCount="68" Body="&lt;p&gt;Is an optimization algorithm equivalent to a neural network?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="1671" LastEditDate="2019-10-15T19:18:42.013" LastActivityDate="2019-10-15T19:18:42.013" Title="Is an optimization algorithm equivalent to a neural network?" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;&lt;comparison&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13526" PostTypeId="1" AcceptedAnswerId="13528" CreationDate="2019-07-23T13:54:07.807" Score="4" ViewCount="369" Body="&lt;p&gt;Why do people use the &lt;span class=&quot;math-container&quot;&gt;$PReLU$&lt;/span&gt; activation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$PReLU[x] = ReLU[x] + ReLU[p*x]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the parameter &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; typically being a small negative number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a fully connected layer is followed by a at least two element &lt;span class=&quot;math-container&quot;&gt;$ReLU$&lt;/span&gt; layer then the combined layers together are capable of emulating exactly the &lt;span class=&quot;math-container&quot;&gt;$PReLU$&lt;/span&gt;, so why is it necessary?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="27386" LastEditorUserId="2444" LastEditDate="2019-07-23T17:58:13.957" LastActivityDate="2019-07-23T17:58:13.957" Title="Is PReLU superfluous with respect to ReLU?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13528" PostTypeId="2" ParentId="13526" CreationDate="2019-07-23T15:15:18.240" Score="1" Body="&lt;p&gt;Lets assume we have 3 Dense layers, where the activations are &lt;span class=&quot;math-container&quot;&gt;$x^0 \rightarrow x^1 \rightarrow x^2$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$x^2 = \psi PReLU(x^1) + \gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x^1 = PReLU(Ax^0 + b)$&lt;/span&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets see what it would take to conform the PReLU into a ReLU    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\begin{align*} &#xA;PReLU(x^1) &amp;amp;= ReLU(x^1) + ReLU(p \odot x^1)\\&#xA;&amp;amp;= ReLU(Ax^0+b) + ReLU(p\odot(Ax^0+b))\\&#xA;&amp;amp;= ReLU(Ax^0+b) + ReLU((eye(p)A + eye(p)b)x^0)\\ &#xA;&amp;amp;= ReLU(Ax^0+b) + ReLU(Qx^0+c) \quad s.t. \quad Q = eye(p)A, \ \ c = eye(p)b\\&#xA;&amp;amp;= [I, I]^T[ReLU(Ax^0+b), ReLU(Qx^0+c)]\\&#xA;\implies x^2 &amp;amp;= [\psi, \psi][ReLU(Ax^0+b), ReLU(Qx^0+c)]\\&#xA;&amp;amp;= V*ReLU(Sx^0 + d) \quad V=[\psi, \psi], \ \ S=[A, Q] \ \ d=[b, c]&#xA;\end{align*}$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So as you said it is possible to break the form of the intermiediary &lt;span class=&quot;math-container&quot;&gt;$PReLU$&lt;/span&gt; into a pure &lt;span class=&quot;math-container&quot;&gt;$ReLU$&lt;/span&gt; while keeping it as a linear model, but if you take a second look at the parameters of the model, the size increase drastically. The hidden units of S &lt;strong&gt;doubled&lt;/strong&gt; meaning to keep &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; the same size &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; also doubles in size. So this means if you dont want to use the &lt;span class=&quot;math-container&quot;&gt;$PReLU$&lt;/span&gt; you are learning double the parameters to achieve the same capability (granted it allows you to learn a wider span of functions as well), and if you enforce the constraints on &lt;span class=&quot;math-container&quot;&gt;$V,S$&lt;/span&gt; set by the &lt;span class=&quot;math-container&quot;&gt;$PReLU$&lt;/span&gt; the number of paramaters is the same but you are still using more memory and more operations!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this example convinces you of the difference&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-23T15:15:18.240" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13529" PostTypeId="1" CreationDate="2019-07-23T15:18:19.450" Score="1" ViewCount="411" Body="&lt;p&gt;According to my lecture, Fuzzy c-Means tries to minimize the following objective function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(X,B,U)=\sum_{i=1}^c\sum_{j=1}^n u_{ij}^w \, d^2(\vec{\beta_i},\vec{x_j})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; are the data points, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are the cluster-'prototypes', and &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; is the matrix containing the fuzzy membership degrees. &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is a distance measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A constraint is that the membership degrees for a single datapoint w.r.t. all clusters sum to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\sum_{j=1}^n\, u_{ij}=1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in the first equation, what is the role of the &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;? I read that one could use any convex function instead of &lt;span class=&quot;math-container&quot;&gt;$(\cdot)^w$&lt;/span&gt;. But why use anything at all. Why don't we just use the membership degrees? My lecture says using the fuzzifier is necessary but doesn't explain why.&lt;/p&gt;&#xA;" OwnerUserId="21366" LastEditorUserId="2444" LastEditDate="2019-07-23T19:54:22.907" LastActivityDate="2019-07-24T17:32:07.377" Title="What is the role of the 'fuzzifier' w in Fuzzy Clustering?" Tags="&lt;unsupervised-learning&gt;&lt;fuzzy-logic&gt;&lt;k-means&gt;&lt;clustering&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13530" PostTypeId="2" ParentId="13526" CreationDate="2019-07-23T16:23:32.910" Score="1" Body="&lt;p&gt;Here are 3 reasons I can think of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Space&lt;/strong&gt; - As @mshlis pointed out, size. To approximate a PReLu you require more than 1 ReLu. Even without formal proof one can easily see that PReLu is 2 adjustable (parameterizable) linear functions within 2 different ranges joined together, while ReLu is just a single adjustable (parameterizable) linear function within half that range, so you require minimum 2 ReLu's to approximate a PReLU. And thus space complexity increases and you require more space to store parameters&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Time&lt;/strong&gt; - This increase in number of ReLu directly affects training time, here is a &lt;a href=&quot;https://ai.stackexchange.com/questions/5728/what-is-the-time-complexity-for-training-a-neural-network-using-back-propagation&quot;&gt;question&lt;/a&gt; on the time complexity of training a Neural Network, you can check out and work out the necessary mathematical details for time increment for a 2x Neural Network size.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Dead ReLu's&lt;/strong&gt; - This is a problem in which a ReLu output becomes 0 due to negative input, so there is no way of flowing your gradient through it and thus, it further has no effect on the training. It can be made alive again only if the other alive ReLu's optimise some  activations from earlier layers such that the dead ReLu again has a positive output. This is not very likely, since the loss is optimised by adjusting weights and not by adjusting whether dead ReLu's are present (basically it is not a learnable parameter so there is a random chance of it coming alive again, other ReLu's do not strive to make it alive). So, to accommodate dead ReLu's the size of Neural Net needs to be increased more, which again leads to added time complexity. Here is a &lt;a href=&quot;https://datascience.stackexchange.com/questions/5706/what-is-the-dying-relu-problem-in-neural-networks&quot;&gt;question&lt;/a&gt; on Dead ReLU's.&#xA;PReLu's do not suffer from this problem (which is probably one of the reasons of their introduction) and thus is definitely a better choice in terms of this criteria.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From personal experience, for a small number of epochs PReLu's tend to perform better than ReLu's for small number of epochs (I have trained only for small epochs). With further epochs and optimisation, this observation might cease to hold true.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-07-23T16:23:32.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13531" PostTypeId="1" AcceptedAnswerId="13548" CreationDate="2019-07-23T16:46:09.163" Score="0" ViewCount="640" Body="&lt;p&gt;Just needed a clarification on the training procedure for a standard GAN.&#xA;Of my understanding the loss function to optimize is a min max (max min causing mode collapse due to focus on one class generation) problem where the loss function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://miro.medium.com/max/695/1*PI-Qb7CvbMYy_XJCIX07mw.png&quot; alt=&quot;as follows&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;needs to maximized for the discriminator and minimized for the generator networks -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) In this equation are the &lt;span class=&quot;math-container&quot;&gt;$E_{z~p_{z(z)}}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E_{x~p_{data(x)}}$&lt;/span&gt; the means of the distributions of the mini batch samples? Also is the optimal case for the discriminator a maximum value of 0? and the optimal case for the generator a minimum value of &lt;span class=&quot;math-container&quot;&gt;$log(small~value)$&lt;/span&gt; {basically a massive negative value}? If so, what happens to the first term during the training of the generator - is it taken as a constant or is the discriminator performing badly considered optimal for the generator? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) While putting this in code, for every step is the discriminator trained first for one step, keeping the generator constant, followed by the generator being trained for the same step, with the discriminator kept constant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) In every step of training are there multiple latent vectors sampled to produce multiple generator outputs for each step? If so is the loss function an average or sum of all &lt;span class=&quot;math-container&quot;&gt;$V(D, G)$&lt;/span&gt; for that step&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-07-23T23:04:00.393" Title="Query regarding the minmax loss function formulation of the training of a Generative Adversarial Network (GAN)" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;optimization&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13532" PostTypeId="5" CreationDate="2019-07-23T18:19:08.173" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T18:19:08.173" LastActivityDate="2019-07-23T18:19:08.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13533" PostTypeId="4" CreationDate="2019-07-23T18:19:08.173" Score="0" Body="For questions related to cognitive architectures, such as SOAR and ACT." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T18:19:08.173" LastActivityDate="2019-07-23T18:19:08.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13534" PostTypeId="5" CreationDate="2019-07-23T18:19:56.980" Score="0" Body="&lt;p&gt;See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Soar_(cognitive_architecture)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Soar_(cognitive_architecture)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T18:19:56.980" LastActivityDate="2019-07-23T18:19:56.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13535" PostTypeId="4" CreationDate="2019-07-23T18:19:56.980" Score="0" Body="For questions related to the SOAR cognitive architecture, originally created by John Laird, Allen Newell, and Paul Rosenbloom at Carnegie Mellon University." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T18:19:56.980" LastActivityDate="2019-07-23T18:19:56.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13536" PostTypeId="5" CreationDate="2019-07-23T18:23:03.587" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T18:23:03.587" LastActivityDate="2019-07-23T18:23:03.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13537" PostTypeId="4" CreationDate="2019-07-23T18:23:03.587" Score="0" Body="For questions related to the Stanford Research Institute Problem Solver (also known as STRIPS), which is an automated planner developed by Richard Fikes and Nils Nilsson in 1971 at SRI International." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-23T18:23:03.587" LastActivityDate="2019-07-23T18:23:03.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13538" PostTypeId="1" CreationDate="2019-07-23T19:08:31.140" Score="4" ViewCount="85" Body="&lt;p&gt;I am trying to create an AI that makes reasonable guesses at truths of statements. However...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human: &quot;Prove that no number exists which is one more than a billion.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI: &quot;Is it true for the number 1? No. I am 1% sure the statement is correct.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI: &quot;Is it true for the number 2? No. I am 2% sure the statement is correct.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI: &quot;Is it true for the number 999,999? No. I am 99.99% sure the statement is correct.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI: &quot;Having tested a large number of examples. I conclude that the statement is correct.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human: &quot;The statement is wrong: one billion and one.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think has gone wrong?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-08-24T00:31:58.783" Title="How would an AI work out this question?" Tags="&lt;ai-design&gt;&lt;math&gt;&lt;logic&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13539" PostTypeId="2" ParentId="13529" CreationDate="2019-07-23T19:35:16.033" Score="1" Body="&lt;p&gt;Its not required, you can have &lt;span class=&quot;math-container&quot;&gt;$m=1$&lt;/span&gt;, actually it can be any number &lt;span class=&quot;math-container&quot;&gt;$\geq 1$&lt;/span&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the better question is why to have it? The answer is that it adds a smoothing effect. Lets look at it in each of the limits (&lt;span class=&quot;math-container&quot;&gt;$\lim m \rightarrow 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\lim m \rightarrow \infty$&lt;/span&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Towards &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;, it makes &lt;span class=&quot;math-container&quot;&gt;$u_{ij}$&lt;/span&gt; equal to &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{c}$&lt;/span&gt;, making each point have equal membership of each class regardless of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. From the optimization perspective, its saying how can we achieve finding clusters that are closest to all points, therefore by definition it has already achieved that, and so the Loss will always be 0. (at its global minimum)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in the other limit, the constants are inversely proportional to the square of the normalized euclidean distance. This makes intuitive sense, the membership is high if they are close, and the membership is low if they are not (relatively)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why do we have the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, its for control. It allows us to choose and experiment with how heavy each distance should hold weight in the membership. An example where a larger &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; may be useful is when the data isnt clean, and you dont want to rely so heavily on euclidean distance as the membership, so you forcibly add in a smoothing effect &lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2019-07-24T17:32:07.377" LastActivityDate="2019-07-24T17:32:07.377" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13540" PostTypeId="1" AcceptedAnswerId="13609" CreationDate="2019-07-23T20:01:18.997" Score="0" ViewCount="855" Body="&lt;p&gt;I haven't been able to find any assistance / examples which could help me implement OpenAI's Spinning Up resource to solve Atari's Breakout-v0 game in the OpenAI gym.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I simply want to know why the following command doesn't run, and instead produces an error that I can't find any help on:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python -m spinup.run ppo --env Breakout-v0 --exp_name simpletest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...and then the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Shape must be rank 2 but is rank 4 for 'pi/multinomial/Multinomial'&#xA; (op: 'Multinomial') with input shapes: [?,210,160,4], [].&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand the shape dynamics, and have written several (albeit quite unoptimized!) reinforcement learning neural nets in Python, but I was looking forward to using OpenAI's Spinning Up environment to use something more sophisticated and optimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much for any help on the seeminly noobish question!&lt;/p&gt;&#xA;" OwnerUserId="27392" LastEditorUserId="27516" LastEditDate="2019-07-29T20:27:26.000" LastActivityDate="2019-07-29T20:27:26.000" Title="OpenAI Spinning Up: Breakout-v0 example" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13543" PostTypeId="2" ParentId="13524" CreationDate="2019-07-23T21:41:41.970" Score="2" Body="&lt;p&gt;Recurrent neural networks (RNNs) are &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_completeness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing complete&lt;/a&gt; (that is, they can simulate any Turing machine), which means that, in theory, they can execute any algorithm that a Turing machine can, so it can also run any optimization algorithm that can be run on a Turing machine (e.g. gradient descent). See &lt;a href=&quot;https://stats.stackexchange.com/a/221142/82135&quot;&gt;this good answer&lt;/a&gt; to the question &lt;a href=&quot;https://stats.stackexchange.com/q/220907/82135&quot;&gt;Meaning (and proof) of &quot;RNN can approximate any algorithm&quot;&lt;/a&gt; for a more exhaustive discussion of this fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is a model: it &lt;em&gt;represents&lt;/em&gt; some (fixed) function, given some fixed weights (which can be thought of as the state of the NN). However, as opposed to most optimization algorithms, the weights or parameters of a NN can be changed by an optimization algorithm (like gradient descent), but this will arguably no more be the same NN. So, in theory, you could have an RNN that approximates gradient descent and back-propagation, so that a neural network trains another neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An optimization algorithm is usually not trained to approximate any continuous function. It is just a possibly parametrized program that runs on a Turing machine. However, note again that a specific optimization algorithm corresponds to a specific NN, so, in this sense, a specific optimization algorithm is equivalent to some NN.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-23T21:41:41.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13544" PostTypeId="1" AcceptedAnswerId="13546" CreationDate="2019-07-23T22:02:12.300" Score="4" ViewCount="130" Body="&lt;p&gt;In &lt;a href=&quot;https://ai.stackexchange.com/a/13543/2444&quot;&gt;this answer&lt;/a&gt; to the question &lt;a href=&quot;https://ai.stackexchange.com/q/13524/2444&quot;&gt;Is an optimization algorithm equivalent to a neural network?&lt;/a&gt;, the author stated that, in theory, there is some recurrent neural network that implements a given optimization algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, then can we optimize the optimization algorithm (neural network)?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2019-07-23T22:05:35.187" LastActivityDate="2020-05-13T19:48:30.960" Title="Can we optimize an optimization algorithm?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;optimization&gt;&lt;logic&gt;&lt;function-approximation&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13546" PostTypeId="2" ParentId="13544" CreationDate="2019-07-23T22:28:27.030" Score="4" Body="&lt;p&gt;First, you need to consider what are the &quot;parameters&quot; of this &quot;optimization algorithm&quot; that you want to &quot;optimize&quot;. Let's take the most simple case, a SGD without momentum. The update rule for this optimizer is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{t+1} \leftarrow w_{t} - a \cdot \nabla_{w_{t}} J(w_t) = w_{t} - a \cdot g_t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$w_t$&lt;/span&gt; are the weights at iteration &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the cost function, &lt;span class=&quot;math-container&quot;&gt;$g_t = \nabla_{w_{t}} J(w_t)$&lt;/span&gt; are the gradients of the cost function w.r.t &lt;span class=&quot;math-container&quot;&gt;$w_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An optimization algorithm accepts as its input the weights and their gradients and returns the update. So we could write the above equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{t+1} \leftarrow w_{t} - SGD(w_t, g_t)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same is true for all optimization algorithms (e.g. Adam, RMSprop, etc.). Now our initial question was what are the &lt;strong&gt;parameters of the optimizer&lt;/strong&gt;, which we want to optimize. In the simple case of the SGD, the sole parameter of the optimizer is the &lt;strong&gt;learning rate&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question that arises at this point is &lt;em&gt;can we optimize the learning rate of the optimizer during training?&lt;/em&gt; Or more practically, can we compute this derivative?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial J(w_t)}{\partial a}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea was explored in &lt;a href=&quot;https://arxiv.org/pdf/1703.04782.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, where they coin this technique &quot;hypergradient descent&quot;. I suggest you take a look.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-07-23T22:28:27.030" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13547" PostTypeId="2" ParentId="13544" CreationDate="2019-07-23T22:29:29.700" Score="2" Body="&lt;p&gt;We usually optimize &lt;strong&gt;with respect to&lt;/strong&gt; something. For example, you can train a neural network to locate cats in an image. This operation of locating cats in an image can be thought of as a function: given an image, a neural network can be trained to return the position of the cat in the image. In this sense, we can optimize a neural network with respect to this task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if a neural network represents an optimization algorithm, then, if you change it a little bit, then it will no more be the same optimization algorithm: it might be another optimization algorithm or some other different algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, most optimizations algorithms that are used to train neural networks (like Adam) are a variation of gradient descent (GD). If you think that Adam performs better than GD, then you could say that Adam is an optimization of GD. So, Adam performs better than GD with respect to something. Possibly, GD also performs better than Adam with respect to something else. Of course, this is a little bit of a stretch.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-23T22:29:29.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13548" PostTypeId="2" ParentId="13531" CreationDate="2019-07-23T23:04:00.393" Score="2" Body="&lt;p&gt;I'll answer your questions one by one:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this equation are the &lt;span class=&quot;math-container&quot;&gt;$E_{z \sim p_z(z)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E_{x \sim p_{data}(x)}$&lt;/span&gt; the means of the distributions of the mini batch samples?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So let's take the first part &lt;span class=&quot;math-container&quot;&gt;$E_{x \sim p_{data}(x)}[log \,D(x)]$&lt;/span&gt;. This is read as the &quot;expected value of &lt;span class=&quot;math-container&quot;&gt;$log \, D(x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is sampled from &lt;span class=&quot;math-container&quot;&gt;$p_{data}(x)$&lt;/span&gt;&quot;. So, in simpler terms this means that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You have a distribution &lt;span class=&quot;math-container&quot;&gt;$p_{data}(x)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You sample a batch of samples &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from this distribution.&lt;/li&gt;&#xA;&lt;li&gt;You feed the batch to the discriminator and get its output &lt;span class=&quot;math-container&quot;&gt;$D(x)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You compute the log for the batch of predictions.&lt;/li&gt;&#xA;&lt;li&gt;You average over the samples in the batch.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also is the optimal case for the discriminator a maximum value of 0?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, the goal of the discriminator is to &lt;strong&gt;maximize&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$V(D, G)$&lt;/span&gt;. This is achieved when &lt;span class=&quot;math-container&quot;&gt;$D(x) = 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D(z) = 0$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$V(D, G)=0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the optimal case for the generator a minimum value of log(small value) {basically a massive negative value}?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, the generator wants to &lt;strong&gt;minimize&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$V(D, G)$&lt;/span&gt;, so it wants &lt;span class=&quot;math-container&quot;&gt;$D(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D(z)$&lt;/span&gt; to be as small as possible (though it can affect only the second term). The minimum value for &lt;span class=&quot;math-container&quot;&gt;$D(z)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1 / M$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the number of classes in the dataset. So the minimum value for &lt;span class=&quot;math-container&quot;&gt;$V(D, G)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\log{[(M-1) / M^2]}$&lt;/span&gt; (I think).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, what happens to the first term during the training of the generator - is it taken as a constant or is the discriminator performing badly considered optimal for the generator?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first term is of no consequence to the Generator, because the generator can't affect it somehow (i.e. like you say it is taken as a constant). Consider the generator &lt;span class=&quot;math-container&quot;&gt;$G_{\theta}$&lt;/span&gt; has parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and the discriminator &lt;span class=&quot;math-container&quot;&gt;$D_{\phi}$&lt;/span&gt; has parameters &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;. When training &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\log{D_{\phi}(x)}}{\partial \theta} = 0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the generator is trained of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;While putting this in code, for every step is the discriminator trained first for one step, keeping the generator constant, followed by the generator being trained for the same step, with the discriminator kept constant?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In theory, yes. In practice for every step the Generator takes, we train the Discriminator for more steps (e.g. &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; steps).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In every step of training are there multiple latent vectors sampled to produce multiple generator outputs for each step? If so is the loss function an average or sum of all V(D,G) for that step.?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, at every step we sample a batch of latent vectors. The equation indicates that we average over these (i.e. the expectation) but in practice it doesn't have any difference (the average is the sum divided by a constant number, which doesn't affect the optimization very much).&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-07-23T23:04:00.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13549" PostTypeId="1" AcceptedAnswerId="13561" CreationDate="2019-07-24T00:28:38.590" Score="3" ViewCount="115" Body="&lt;p&gt;From the literature I have read so far, it is not clear how exactly the convolution operation is defined. It seems people use two different definitions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us assume we are given an &lt;span class=&quot;math-container&quot;&gt;$n_w \times n_h \times d$&lt;/span&gt; input tensor &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and an &lt;span class=&quot;math-container&quot;&gt;$m_w \times m_h \times d$&lt;/span&gt; filter &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; kernels (I use the convention of referring to the depth-slices of filters as &lt;em&gt;kernels&lt;/em&gt;. I also will call the depth slices of the input tensor &lt;em&gt;channels&lt;/em&gt;). Let us also assume &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th filter of &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to the definitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Option 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convolution of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is obtained by sliding &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; across &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and computing the Frobenius inner product between channel &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and kernerl &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; at each position, adding the products and storing them in an output matrix. That matrix is the result of the convolution. It is also the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th feature map in the output tensor of the convolution layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$I \in \mathbb{R}^{n_w \times n_h \times d}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$F \in \mathbb{R}^{m_w \times m_h \times d}$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$s \in \mathbb{N}$&lt;/span&gt; be the stride. The operation will only be defined if the smaller tensor fits within the larger tensor along its width and height a positive integer number of times when shifting by &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, that is if and only if &lt;span class=&quot;math-container&quot;&gt;$k_w = (n_w - m_w) / s + 1\in \mathbb{N}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k_h = (n_h - m_h) / s + 1\in \mathbb{N}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k_w \times k_h \times d$&lt;/span&gt; is the shape of the output tensor. Furthermore let &lt;span class=&quot;math-container&quot;&gt;${f_x : i \mapsto (x - 1)s + i}$&lt;/span&gt; be a function that returns the absolute index in the input tensor, given an index &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in the output tensor, the stride length &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and a relative index &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;    \begin{split}&#xA;        (I * F)_{x y} =&#xA;            &amp;amp; \sum_{k=1}^d \sum_{i = 1}^{m_w} \sum_{j = 1}^{m_h} I_{f_x(i) f_y(j) k} \cdot F_{i j k}&#xA;    \end{split}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Option 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convolutions (plural) of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; are obtained by sliding &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; across &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and computing the Frobenius inner product between channel &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and kernel &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; at each position. Each product is stored in a matrix associated with the channel &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. There is no adding of the products yet. The convolutions are the result matrices. The step where the matrices are added component wise to obtain the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th feature map of the output tensor of the convolution layer is not part of the convolution operation, but an independent step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$I \in \mathbb{R}^{n_w \times n_h}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$F \in \mathbb{R}^{m_w \times m_h}$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$s \in \mathbb{N}$&lt;/span&gt; be the stride. The operation will only be defined if the smaller matrix fits within the larger one along its width and height a positive integer number of times when shifting by &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, that is, if and only if &lt;span class=&quot;math-container&quot;&gt;$k_w = (n_w - m_w) / s + 1\in \mathbb{N}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k_h = (n_h - m_h) / s + 1\in \mathbb{N}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k_w \times k_h$&lt;/span&gt; is the shape of the output matrix. Furthermore let &lt;span class=&quot;math-container&quot;&gt;${f_x : i \mapsto (x - 1)s + i}$&lt;/span&gt; be a function that returns the absolute index in the input matrix, given an index &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in the output matrix, the stride length &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and a relative index &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;    \begin{split}&#xA;        (I * F)_{x y} =&#xA;            &amp;amp;\sum_{i = 1}^{m_w} \sum_{j = 1}^{m_h} I_{f_x(i) f_y(j)} \cdot F_{i j}&#xA;    \end{split}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which of these two definitions is the common one?&lt;/p&gt;&#xA;" OwnerUserId="20150" LastEditorUserId="20150" LastEditDate="2019-07-25T10:31:23.647" LastActivityDate="2019-07-25T10:31:23.647" Title="Clarification on definition of convolution: Is adding the Frobenius inner products between filter and input part of convolution or a separate step?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13550" PostTypeId="1" CreationDate="2019-07-24T08:21:40.497" Score="0" ViewCount="43" Body="&lt;p&gt;What neural network can be used to detect patterns and anomalies of a network map.&#xA;The network data comes from tools like nmap and masscan.&lt;/p&gt;&#xA;" OwnerUserId="27399" LastEditorUserId="1671" LastEditDate="2019-07-25T19:43:47.553" LastActivityDate="2019-07-25T19:43:47.553" Title="What neural network can be used to detect patterns and anomalies of a network map?" Tags="&lt;neural-networks&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13551" PostTypeId="1" CreationDate="2019-07-24T09:44:54.920" Score="0" ViewCount="177" Body="&lt;p&gt;How do I do further (domain-specific) pre-training with Google BERT in preparation for subsequent fine-tuning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to say this is: can you create a checkpoint file created from the final output of BERT?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://www.scihive.org/paper/1905.05583&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Fine-Tune BERT for Text Classification?&lt;/a&gt; talks about this additional fine-tuning. &lt;/p&gt;&#xA;" OwnerUserId="3370" LastEditorUserId="2444" LastEditDate="2019-11-01T02:24:08.777" LastActivityDate="2019-11-01T02:24:08.777" Title="How do I do further (domain-specific) pre-training with Google BERT in preparation for subsequent fine-tuning?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13552" PostTypeId="1" AcceptedAnswerId="13553" CreationDate="2019-07-24T11:52:05.077" Score="2" ViewCount="155" Body="&lt;p&gt;I do not understand why with enough training how the generator cannot learn all images from the training set as a mapping from the latent space - It is the absolute optimal case in training as it replicates the distribution and the discriminator output will always be 0.5. Even though most blog posts I have seen do not mention noise, a few of them have them in their diagrams or describe their presence, but never exactly describe the purpose of this noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this noise injected to avoid the exact reproduction of the training data? If not what is the purpose of this injection and how is exact reproduction avoided?&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-07-24T21:53:24.687" LastActivityDate="2019-07-24T21:56:07.843" Title="What is the purpose of the noise injection in the generator network of a GAN?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13553" PostTypeId="2" ParentId="13552" CreationDate="2019-07-24T12:26:36.250" Score="1" Body="&lt;p&gt;Your goal is to model a distribution when constructing a GAN, therefore you need a way to be able to sample that distribution. The &lt;em&gt;noise&lt;/em&gt;'s purpose is so you can do this. Generally, it's drawn from a distribution that is computationally easy to draw from (like a gaussian).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are modeling the generator &lt;span class=&quot;math-container&quot;&gt;$G(X)$&lt;/span&gt;  where &lt;span class=&quot;math-container&quot;&gt;$X \sim N(\mu, \sigma^2)$&lt;/span&gt;. this means &lt;span class=&quot;math-container&quot;&gt;$G(X)$&lt;/span&gt; is a random variable itself. The forward pass of the network transforms the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; samples into our &lt;span class=&quot;math-container&quot;&gt;$G(X)$&lt;/span&gt; samples, allowing us to formulate a loss function (by solving the expectation as the mean of drawn samples) and train the model.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Takeaway&lt;/strong&gt;: The noise injected is just a parametrization of our Generator in another space, and the training goal is to learn the ideal transformation (we use neural networks because they are differentiable and are effective function approximators)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, note to your point of why it doesn't learn the training data (in its entirety) exactly is because generally &lt;span class=&quot;math-container&quot;&gt;$G(X)$&lt;/span&gt; is continuous, and therefore if it has 2 images in its codomain, there also exists some path in pixel space from one to the other containing an uncountable (or quantized countable) number of images that don't exist in the training, and this would be reflected in the loss, therefore in the min-max game of the optimization, its difficult for it learn the training set on the nose.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-24T21:56:07.843" LastActivityDate="2019-07-24T21:56:07.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13554" PostTypeId="1" CreationDate="2019-07-24T13:07:19.477" Score="1" ViewCount="177" Body="&lt;p&gt;If I understand correctly, the KL divergence is a measure of information loss between a ground truth distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and a predicted distribution &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, and the Jensen-Shannon divergence is the mean of the KL Divergences of 2 cases&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Predicted distribution is mean of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, and ground truth is &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Predicted distribution is mean of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, and ground truth is &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since KL divergence can be easily interpreted as information loss in &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; relative to &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, what can JS divergence interpret-ably represent? I cannot see any use cases of these measures unless there are two distributions to compare. Is there any other problem where I could use them as loss functions other than generation problems? If so how, and why?  &lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-07-24T20:24:39.443" LastActivityDate="2019-07-24T20:24:39.443" Title="Could the Jensen-Shannon divergence and Kullback-Leibler divergence be used as loss functions of non-generation problems?" Tags="&lt;deep-learning&gt;&lt;optimization&gt;&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13555" PostTypeId="1" CreationDate="2019-07-24T14:51:04.757" Score="1" ViewCount="256" Body="&lt;p&gt;I have implemented the total loss of my PPO objective as follows:-&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;total_loss = critic_discount * critic_loss + actor_loss -  entropy_beta * K.mean(-(newpolicy_probs * K.log(newpolicy_probs)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After training for a few epochs, the entropy term becomes &quot;nan&quot; for some reason. I used &lt;code&gt;tf.Print()&lt;/code&gt; to see the new policy probabilities when the entropy becomes undefined, it is as follows-&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;new policy probs: [[6.1029973e-06 1.93471514e-08&#xA;  0.000299338106...]...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not clear as to why taking log of these small probabilities is coming out as &lt;code&gt;nan&lt;/code&gt;. Any idea how to prevent this?&lt;/p&gt;&#xA;" OwnerUserId="27405" LastActivityDate="2019-07-25T12:36:08.580" Title="Entropy term in Proximal Policy Optimization (PPO) becomes undefined after few training epochs" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;keras&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13557" PostTypeId="1" AcceptedAnswerId="13559" CreationDate="2019-07-24T19:25:13.597" Score="4" ViewCount="81" Body="&lt;p&gt;First of all, it is great to have found this community!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently implementing my own Alpha Zero clone on Connect4. However, I have a mental barrier I cannot overcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use one neural network for both players? I do not understand what the input should be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I just put in the board position (&lt;span class=&quot;math-container&quot;&gt;$6 \times 7$&lt;/span&gt;) and let's say Player1's pieces on the board are represented as &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;, empty board as &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and Player2's pieces as &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, that seems the most efficient. But then, in the backpropagation, I feel like this cannot be working. If I update the same network for both players (which Alpha Zero does), don't I try to optimize Player1 and Player 2 at the same time?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just can't get my head around it. 2 Neural networks, each for one player is understandable for me. But one network? I don't understand how to backpropagate? Should I just flip my &quot;z&quot; (the result of the game) every time I go one layer backward? Is that all there is to using one network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I made this clear enough. I am quite confused, I tried my best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for reading this!&lt;/p&gt;&#xA;" OwnerUserId="27406" LastEditorUserId="2444" LastEditDate="2019-07-24T22:20:41.543" LastActivityDate="2019-07-24T22:20:41.543" Title="How can I use one neural network for both players in Alpha Zero (Connect 4)?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;deep-neural-networks&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13559" PostTypeId="2" ParentId="13557" CreationDate="2019-07-24T20:22:53.700" Score="3" Body="&lt;p&gt;Let's define your problem from another point of view. Let's say that in this RL problem you have two agents (agent1 and agent2) that compete with each other in order to accomplish their own goal, i.e., wining connect4 game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we could say that from agent1's point of view, he is player1 and the player2 is agent2. The same way, from agent2's point of view, he is player1 and the player2 is agent1.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we can use the same definition as you proposed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;let's say player1's pieces on the board are represented as -1, empty board as 0 and player2's pieces as 1&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it is the turn for agent1, he will receive an observation (representing the board configuration) from the environment. In this case, agent1's pieces are represented as -1, empty board as 0 and for player2's (agent2's) pieces as 1. Then, you could use the neural network to compute, e.g., the next action for agent1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it is the turn for agent2, he will receive another observation from the environment, but this time you represent its own pieces as -1, while the agent1's (player2's from agent2's point of view) as 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this approach, the neural network will always receives as an input a representation in which the pieces 'it' controls are -1, while its rival's pieces are 1 and empty space 0. From the neural network's point of view, he does not care about which agent is playing as the actual observation is indistinguishable to calculate the next action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, you will have to generate two different observations, in which you invert the sign of the ones on it, depending on which turn it is (agent1's or agent2's turn).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KlgSW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KlgSW.png&quot; alt=&quot;agent1&amp;#39;s board&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ER9xb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ER9xb.png&quot; alt=&quot;agent2&amp;#39;s board&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27341" LastActivityDate="2019-07-24T20:22:53.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13560" PostTypeId="1" CreationDate="2019-07-25T00:22:51.017" Score="0" ViewCount="45" Body="&lt;p&gt;I'm new to machine learning and especially, deep learning. Given a video (and it's subtitle), I need to generate a 10-second summary out of this video. How can I use ML and DL to produce the most representative summary out of this video? More specifically, given video scenes, what are some ways to select and rank them, and how to do it? Any ideas would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2019-07-25T11:17:42.413" Title="Use deep learning to rank video scenes" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;problem-solving&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13561" PostTypeId="2" ParentId="13549" CreationDate="2019-07-25T01:02:40.330" Score="2" Body="&lt;p&gt;Both are incorrect.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;using your notation&lt;/em&gt;&lt;br&gt;&#xA;You do not take a sliding frobenius inner product of a singular channel of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;, but with all the channels at once. This may be easier to understand if you do not assume the number of channels of the input and output are the same (ie different number of input channels then filters). So lets say your input has &lt;span class=&quot;math-container&quot;&gt;$k_1$&lt;/span&gt; channels and you have &lt;span class=&quot;math-container&quot;&gt;$k_2$&lt;/span&gt; filters.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means &lt;span class=&quot;math-container&quot;&gt;$shape(I) = (N, M, k_1)$&lt;/span&gt; and each of the &lt;span class=&quot;math-container&quot;&gt;$k_2$&lt;/span&gt; filters is of shape &lt;span class=&quot;math-container&quot;&gt;$(n, m,k_2)$&lt;/span&gt; and your output shape is &lt;span class=&quot;math-container&quot;&gt;$(N-n+1, M-m+1, k_2)$&lt;/span&gt; assuming your not using padding.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i guess trying to put it in the manner you use: The convolution's output's &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; channel is taking the sliding Frobenius inner product between all &lt;span class=&quot;math-container&quot;&gt;$n \times m$&lt;/span&gt; cross-section of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; (including &lt;strong&gt;all input channels&lt;/strong&gt;) with filter &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-25T01:02:40.330" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="13563" PostTypeId="2" ParentId="13523" CreationDate="2019-07-25T02:56:51.950" Score="2" Body="&lt;p&gt;I believe the question you're asking is whether or not it is possible for a neural network to edit it's own structure and learning ability intelligently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would argue this is a matter of philosophy, of how intelligent can we program a computer to be, and I personally believe that yes, at some point humans will develop some form of an algorithm capable of intelligently improving upon on its own learning ability. Right now, I do not think it exists (as far as I know), and there are equal arguments stating that this isn't possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is by no means an easy task, as even nature couldn't replicate it. Sure, humans have gotten great at &lt;em&gt;tailoring&lt;/em&gt; the information we receive so it is easier to learn, but as to actually improving the learning ability we were born with, I don't believe we can do much about it. The only way intelligence emerged in nature was through evolution, a mechanic of chance, not meaningful decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I do believe effort is being dedicated to this idea, but I don't think it has yet been achieved. The reason I can't be sure, is because there is a rather large incentive to keep something like this a secret should it be done. &lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-07-25T02:56:51.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13564" PostTypeId="2" ParentId="11617" CreationDate="2019-07-25T06:09:25.453" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So, from this I will conclude (correct me if I'm wrong) that AlphaZero&#xA;  evaluates the current position, but uses a very powerful heuristic.&#xA;  Stockfish on the other hand searches through lots of positions from&#xA;  the current one first, and then uses a less powerful heuristic when a&#xA;  certain depth is reached.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is wrong. Like Stockfish, AlphaZero as well &quot;searches through lots of positions from the current one.&quot; You hint at this yourself when you say &quot;possibly combined with monte carlo methods&quot;, but it seems you don't understand exactly what that means, so let me explain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stockfish searches through the tree of future moves using an algorithm called Minimax (actually a variant called alpha beta pruning), whereas AlphaZero searches through future moves using a different algorithm called Monte Carlo Tree Search (MCTS). Minimax is well suited to quick evaluation functions, whereas MCTS explores fewer moves and thus can handle a more expensive evaluation function. Further, MCTS works not with exact values but with probabilities, and AlphaZero uses the Neural Net not just for the value of moves but to guide which moves to explore next (it functionally is actually 2 networks, a policy network and a value network).&#xA;To be sure, that is somewhat of a simplification. It is not impossible to use a NeuralNet in conjunction with Minimax. Its just that in practice, due to the nature of the algorithm, its simply too expensive. &lt;/p&gt;&#xA;" OwnerUserId="12201" LastActivityDate="2019-07-25T06:09:25.453" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13565" PostTypeId="2" ParentId="11617" CreationDate="2019-07-25T06:31:18.380" Score="1" Body="&lt;p&gt;Yes it's possible to to combine AlphaZero with Minimax methods (including alpha-beta pruning). AlphaZero itself is combination of  Monte Carlo Tree Search (MCTS) and Deep Network, where MCTS is used to get data to train network and network used for tree leafs evaluation (instead of rollout as in classical MCTS). It's possible to combine selection-expansion part of AlphaZero MCTS with Minimax the same way as it was done for classical MCTS - &lt;a href=&quot;https://dke.maastrichtuniversity.nl/m.winands/documents/paper%2049.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Monte-Carlo Tree Search and Minimax Hybrids&quot;, pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-07-25T06:31:18.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13566" PostTypeId="1" CreationDate="2019-07-25T08:17:00.343" Score="3" ViewCount="99" Body="&lt;p&gt;In reinforcement learning approaches, like temporal-difference (TD) learning or Monte Carlo methods, two of the metrics used to measure their performance are the &lt;em&gt;bias&lt;/em&gt; and the &lt;em&gt;variance&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What do these terms mean? Which characteristic of the agent do they affect (value functions, policy, optimal behavior, etc.)?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2021-02-05T10:30:34.740" LastActivityDate="2021-02-05T10:30:34.740" Title="What is the meaning of the words 'bias' and 'variance' in RL?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;math&gt;&lt;definitions&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13567" PostTypeId="1" CreationDate="2019-07-25T09:28:02.890" Score="1" ViewCount="45" Body="&lt;p&gt;While doing transfer learning where my two problems are face-generation and car-generation is it likely that, if I use the weights of one problem as the initialization of the weights for the other problem, the model will converge to a local minima? In any problem is it better to train from scratch over transfer learning? (especially for GAN training?)&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-07-25T16:08:08.317" LastActivityDate="2019-07-25T16:08:08.317" Title="Is convergence to a local minima more likely with transfer learning?" Tags="&lt;deep-learning&gt;&lt;optimization&gt;&lt;generative-adversarial-networks&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13568" PostTypeId="1" AcceptedAnswerId="13581" CreationDate="2019-07-25T09:50:35.887" Score="3" ViewCount="60" Body="&lt;p&gt;I am going through the book Pattern Recognition by Bishop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At one point he says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$M = 9$&lt;/span&gt;, the training set error goes to zero, as we might expect because this polynomial contains 10 degrees of freedom corresponding to the &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; coefficients &lt;span class=&quot;math-container&quot;&gt;$w_0, \dots, w_9$&lt;/span&gt;, and so can be tuned exactly to the &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; data points in the training set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the order of the hypothesis function, and &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; are the weights of the hypothesis function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did not understand how having &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; degrees of freedom will tune the model EXACTLY to the &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; data points? Does it mean that whenever we have a number of data points in training set equal to the degrees of freedom, the error will be zero?&lt;/p&gt;&#xA;" OwnerUserId="27422" LastEditorUserId="2444" LastEditDate="2019-07-25T16:10:19.897" LastActivityDate="2019-07-26T02:31:00.173" Title="What is the relationship between degrees of freedom and the size of the training dataset?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13570" PostTypeId="1" CreationDate="2019-07-25T11:01:57.763" Score="2" ViewCount="116" Body="&lt;p&gt;I have created a chatbot by Keras based on movie dialog. I used RNN more specifically GRU . My bot can reply well. But the problem is , it can't hold the context . As an example if I say &lt;code&gt;Tell me a joke&lt;/code&gt;, the bot will reply something , and then if I say &lt;code&gt;one more&lt;/code&gt; , the bot simply doesn't understand that I was asking for another joke and many more similar cases, like if I used a slang against the bot , the bot will reply me with something similar , but if I just say something romantic or good immediately after using slang , the bot will reply to me with something good . I want to keep context or environment . How can I do so . Any lead would be helpful .&lt;/p&gt;&#xA;" OwnerUserId="26850" LastActivityDate="2019-07-26T10:28:55.117" Title="How can I keep context in my chatbot" Tags="&lt;recurrent-neural-networks&gt;&lt;keras&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13571" PostTypeId="2" ParentId="13560" CreationDate="2019-07-25T11:17:42.413" Score="1" Body="&lt;p&gt;It’s seems like quite challenging problem; at least you would need quite a lot of annotated data and computational power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approaches/optimizations you could consider:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To make scene change detection and take short piece out of each&lt;/li&gt;&#xA;&lt;li&gt;To introduce some kind of “novelty” metric and try to maximize&#xA;it to get most different parts of video   &lt;/li&gt;&#xA;&lt;li&gt;To convert video to kind of vector with existing solution like&#xA;r-cnn and yolo and then process it with recurrent networks.   &lt;/li&gt;&#xA;&lt;li&gt;The task seems to be very close to video capturing/summarization, you&#xA;can take inspiration there  &lt;/li&gt;&#xA;&lt;li&gt;Also, the attention approach might be handy, look, for example&#xA;&lt;a href=&quot;https://arxiv.org/abs/1906.02792&quot; rel=&quot;nofollow noreferrer&quot;&gt;self-attention for video&lt;/a&gt;,&#xA;&lt;a href=&quot;https://pdfs.semanticscholar.org/5dc7/c33475b545271d1de726fd88bb68dfb7e11b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic attention for&#xA;video&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16940" LastActivityDate="2019-07-25T11:17:42.413" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13572" PostTypeId="1" CreationDate="2019-07-25T11:57:48.477" Score="3" ViewCount="189" Body="&lt;p&gt;What is the difference between image processing and computer vision? They are apparently both used in artificial intelligence.&lt;/p&gt;&#xA;" OwnerUserId="27427" LastEditorUserId="2444" LastEditDate="2019-07-26T10:25:30.403" LastActivityDate="2019-07-26T10:25:30.403" Title="What is the difference between image processing and computer vision?" Tags="&lt;computer-vision&gt;&lt;terminology&gt;&lt;comparison&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13573" PostTypeId="1" AcceptedAnswerId="13591" CreationDate="2019-07-25T12:19:08.883" Score="1" ViewCount="275" Body="&lt;p&gt;I was watching a video about Convolutional Neural Networks: &lt;a href=&quot;https://www.youtube.com/watch?v=SQ67NBCLV98&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=SQ67NBCLV98&lt;/a&gt;. What I'm confused about is the arrangement of applying the filters' channels to the input image or even to the output of a previous layer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;b&gt;Question 1&lt;/b&gt; - Looking at the visual explanation example of how one filter with 3 channels is applied to the input image (with 3 channels), so that each 1 filter channel is applied to its corresponding input channel: &lt;a href=&quot;https://imgur.com/hNb5BwF&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://imgur.com/hNb5BwF&quot; alt=&quot;Conv 2_D&quot; /&gt;&lt;/a&gt;.&#xA;So hence the output is 3 channels. Makes sense.&lt;/p&gt;&#xA;&lt;p&gt;However, looking at the second screenshot which shows an example of the VGG network: &lt;a href=&quot;https://imgur.com/ZGQSo57&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://imgur.com/ZGQSo57&quot; alt=&quot;VGG&quot; /&gt;&lt;/a&gt;, looking at the first layer (I've delineated with a red frame), which is 64 channels, where the input of the image contains 3 channels. How does the output shape become 64? The only way I would think this would be possible is if you apply:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;filter channel 1 to image channel 1&lt;/li&gt;&#xA;&lt;li&gt;filter channel 2 to image channel 2&lt;/li&gt;&#xA;&lt;li&gt;filter channel 3 to image channel 3&lt;/li&gt;&#xA;&lt;li&gt;filter channel 4 to image channel 1&lt;/li&gt;&#xA;&lt;li&gt;filter channel 5 to image channel 2&lt;/li&gt;&#xA;&lt;li&gt;filter channel 6 to image channel 3&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;.. and so on.&lt;/p&gt;&#xA;&lt;p&gt;Or the other thing could be, that these are representing Conv layers, with 64 filters. Rather than a filter with 64 channels. And that's precisely what I'm confused about here. In all the popular Convolutional networks, when we see these big numbers - 64, 128, 256 ... etc, are these Conv layers with 64 filters, or are they individual filters with 64 channels each?&lt;/p&gt;&#xA;&lt;p&gt;&lt;b&gt;Question 2&lt;/b&gt; - Referring back to the second screenshot, the layer I've delineated with blue frame (3x3x128). This Conv layer, as I understand, takes the output of 64 Max-pooled nodes and applies 128 Conv filters. But how does the output become 128. If we apply each filter to each Max-pooled output node, that's 64 x 128 = 8192 channels or nodes in output shape. Clearly that's not what's happening and so I'm definitely missing something here. So, how does 128 filters is applied to 64 output nodes in a way so that the output is still 128? What's the arrangement?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="25360" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-07-26T02:50:45.170" Title="Understanding arrangement of applying filters to input channels" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13574" PostTypeId="2" ParentId="13555" CreationDate="2019-07-25T12:36:08.580" Score="1" Body="&lt;p&gt;I browsed through some other implementations of PPO and they all add small offset (1e-10) to prevent undefined log(0). I did that and the training works now.&lt;/p&gt;&#xA;" OwnerUserId="27405" LastActivityDate="2019-07-25T12:36:08.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13575" PostTypeId="1" AcceptedAnswerId="13640" CreationDate="2019-07-25T12:39:34.397" Score="0" ViewCount="64" Body="&lt;p&gt;Can neural networks modify their own weights without back-propagation and gradient descent?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2019-07-28T19:20:40.360" LastActivityDate="2019-07-28T20:37:59.983" Title="Can neural networks modify their own weights without back-propagation and gradient descent?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;neural-architecture-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13576" PostTypeId="1" CreationDate="2019-07-25T14:05:01.557" Score="1" ViewCount="9" Body="&lt;p&gt;I am building a negation detection system. How to use dependency parsers for the same. I am using SPACY for dependency parser&lt;/p&gt;&#xA;" OwnerUserId="26115" LastActivityDate="2019-07-25T14:05:01.557" Title="How can we use Dependency Parsers for Negation detection" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13577" PostTypeId="1" CreationDate="2019-07-25T14:12:08.937" Score="3" ViewCount="198" Body="&lt;p&gt;I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (&lt;a href=&quot;https://github.com/ChintanTrivedi/rl-bot-football/blob/master/train.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;code here&lt;/a&gt;), but I am confused about certain concepts.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the correct way to implement log probability of a policy (denoted by &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt; below)?&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L^{C P I}(\theta)=\hat{\mathbb{E}}_{t}\left[\frac{\pi_{\theta}\left(a_{t} | s_{t}\right)}{\pi_{\theta_{\text {old }}}\left(a_{t} | s_{t}\right)} \hat{A}_{t}\right]=\hat{\mathbb{E}}_{t}\left[r_{t}(\theta) \hat{A}_{t}\right]&#xA;$$&lt;/span&gt;&#xA;Let's say my old network policy output is &lt;code&gt;oldpolicy_probs=[0.1,0.2,0.6,0.1]&lt;/code&gt; and new network policy output is &lt;code&gt;newpolicy_probs=[0.2,0.2,0.4,0.2]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I take the log of this directly, or should I first multiply these with the true label &lt;code&gt;y_true = [0,0,1,0]&lt;/code&gt; as implemented &lt;a href=&quot;https://github.com/LuEE-C/PPO-Keras/blob/3bd92268d45ac4f0709f926f24392ed99252daac/Main.py#L45&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;ratio = np.mean(np.exp(np.log(newpolicy_probs + 1e-10) - K.log(oldpolicy_probs + 1e-10))*advantage)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once I have the ratio and I multiply it with an advantage, why do we take the mean over all actions? I suspect it might be because we are taking estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{\mathbb{E}_t}$&lt;/span&gt; but conceptually I don't understand what this gives us. Is my implementation above correct?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27405" LastEditorUserId="2444" LastEditDate="2020-05-17T22:08:58.407" LastActivityDate="2020-05-17T22:08:58.407" Title="Understanding log probabilities of actions in the PPO objective" Tags="&lt;python&gt;&lt;loss-functions&gt;&lt;implementation&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13578" PostTypeId="2" ParentId="7222" CreationDate="2019-07-25T14:58:41.863" Score="0" Body="&lt;p&gt;I've been tackling a similar job title classification problem and used this paper as the basis for my approach: &lt;a href=&quot;https://web.stanford.edu/~gavish/documents/phrase_based.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/~gavish/documents/phrase_based.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Might find it useful.&lt;/p&gt;&#xA;" OwnerUserId="27431" LastActivityDate="2019-07-25T14:58:41.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13580" PostTypeId="1" CreationDate="2019-07-25T16:29:04.390" Score="6" ViewCount="1084" Body="&lt;p&gt;I am reading Francois Chollet's &lt;a href=&quot;http://faculty.neu.edu.cn/yury/AAI/Textbook/Deep%20Learning%20with%20Python.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep learning with Python&lt;/a&gt;, and I came across a section about max-pooling that's really giving me trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unable to copy-paste the content, so I've included screenshots of the paragraph that's troubling me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I simply don't understand what he means when he talks about &quot;What's wrong with this setup?&quot; (towards the end). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How does removing the max-pooling layers &quot;reduce&quot; the amount of the initial image that we're looking at? What are the benefits of using max-pooling in convolutional neural networks, as opposed to just using convolution layers?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0jDhk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0jDhk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x01ii.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x01ii.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27433" LastEditorUserId="2444" LastEditDate="2019-07-26T20:08:05.723" LastActivityDate="2019-07-27T00:07:22.763" Title="What are the benefits of using max-pooling in convolutional neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13581" PostTypeId="2" ParentId="13568" CreationDate="2019-07-25T17:11:18.487" Score="3" Body="&lt;p&gt;When you define a straight line of the form &lt;span class=&quot;math-container&quot;&gt;$y=mx+c$&lt;/span&gt;, you need 2 points &lt;span class=&quot;math-container&quot;&gt;$(x_1,y_1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(x_2,y_2)$&lt;/span&gt;, to solve for the 2 variables &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; (you can easily visualise this graphically). Similarly, a parabola of the form &lt;span class=&quot;math-container&quot;&gt;$y=ax^2+bx+c$&lt;/span&gt; will require 3 such points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now viewing it as a ML problem, you are given the points and you have to estimate the parameters such that the training error is 0 (Regression). So just like the previous case you have a bunch of &lt;span class=&quot;math-container&quot;&gt;$(x_i,y_i)$&lt;/span&gt; and you have to fit a curve whose degree of freedom you have to choose. Here &lt;span class=&quot;math-container&quot;&gt;$m,c,a,b$&lt;/span&gt; are all replaced with more generic &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; called as a parameter&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; degree of freedom and &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; data-points you can solve for the parameters of the model (unambiguous solution i.e only one and one unique solution will exist). Whereas , if the degree of freedom is lower you'll get a solution which may miss one point. For, example if you are given 3 points and ask to fit a straight line through it, you may or may not be able to (depending on collinearity). On the opposite case if you have more degrees of freedom you can get multiple values for a single parameter. Assuming a single extra degree of freedom, you can take a parameter hold it to some fixed value and solve the rest of the equation as mentioned above you get some value for the rest of parameters. Now hold the same parameter at a different fixed value and repeat the same process, you get some different value of the other parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it is easier to view it this way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If let's say you have 3 degrees of freedom &lt;span class=&quot;math-container&quot;&gt;$y=w_2x_1^2+w_1x_1+w_0$&lt;/span&gt; and 3 data-points &lt;span class=&quot;math-container&quot;&gt;$(x_1,y_1), (x_2,y_2), (x_3,y_3)$&lt;/span&gt;, you can get a system of equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_1=w_2x_1^2+w_1x_1+w_0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_2=w_2x_2^2+w_1x_2+w_0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_3=w_2x_3^2+w_1x_3+w_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus you get a system of linear equations (NOTE: &lt;span class=&quot;math-container&quot;&gt;$x_i,y_i$&lt;/span&gt; are known here). The only unknowns here are &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;'s for which there are multiple methods to solve. You can extend this for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; equations.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-26T02:31:00.173" LastActivityDate="2019-07-26T02:31:00.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13582" PostTypeId="2" ParentId="13580" CreationDate="2019-07-25T18:00:19.493" Score="1" Body="&lt;p&gt;MaxPooling pools together information. Imagine you have 2 convolutional layers &lt;span class=&quot;math-container&quot;&gt;$(F_1, F_2)$&lt;/span&gt; respectively, each with a 3x3 kernel and a stride of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Also, imagine your input is &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is of shape &lt;span class=&quot;math-container&quot;&gt;$(w,h)$&lt;/span&gt;. Let's call a max-pooling layer &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is of size &lt;span class=&quot;math-container&quot;&gt;$(2,2)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I'm ignoring channels because, for these purposes, it's not necessary and can be extended to any amount of them.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you have two cases:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$O_1 = F_2 \circ F_1 \circ I$&lt;/span&gt;  &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$O_2 = F_2 \circ M \circ F_1 \circ I$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In these cases, &lt;span class=&quot;math-container&quot;&gt;$shape(O_1)=(w-4, h-4)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$shape(O_2)=\left(\frac{w-2}{2}-2, \frac{h-2}{2}-2 \right)$&lt;/span&gt;. If we plug in dummy values, like &lt;span class=&quot;math-container&quot;&gt;$w,h = 64,64$&lt;/span&gt;, we get the shapes become &lt;span class=&quot;math-container&quot;&gt;$(60,60)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(29,29)$&lt;/span&gt; respectively. As you can tell, these are very different!   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there is more of a difference than just the size of the outputs, each neuron holds a pooling of more information. Let's do it out:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Each output neuron of &lt;span class=&quot;math-container&quot;&gt;$F_1 \circ I$&lt;/span&gt; has information from a &lt;span class=&quot;math-container&quot;&gt;$(3,3)$&lt;/span&gt; receptive field.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each output neuron then of &lt;span class=&quot;math-container&quot;&gt;$F_2 \circ F_1 \circ I$&lt;/span&gt; has information from a &lt;span class=&quot;math-container&quot;&gt;$(3,3)$&lt;/span&gt; receptive field of &lt;span class=&quot;math-container&quot;&gt;$F_1 \circ I$&lt;/span&gt;, which, if we eliminate reused nodes, is a &lt;span class=&quot;math-container&quot;&gt;$(5,5)$&lt;/span&gt; receptive field from the initial &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each output neuron then of &lt;span class=&quot;math-container&quot;&gt;$M \circ F_1 \circ I$&lt;/span&gt; has information from a &lt;span class=&quot;math-container&quot;&gt;$(2,2)$&lt;/span&gt; receptive field of &lt;span class=&quot;math-container&quot;&gt;$F_1 \circ I$&lt;/span&gt;, which, if we eliminate reused nodes, is a &lt;span class=&quot;math-container&quot;&gt;$(4,4)$&lt;/span&gt; receptive field from the initial &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each output neuron then of &lt;span class=&quot;math-container&quot;&gt;$F_2 \circ M \circ F_1 \circ I$&lt;/span&gt; has information from a &lt;span class=&quot;math-container&quot;&gt;$(4,4)$&lt;/span&gt; receptive field of &lt;span class=&quot;math-container&quot;&gt;$M \circ F_1 \circ I$&lt;/span&gt;, which, if we eliminate reused nodes, is a &lt;span class=&quot;math-container&quot;&gt;$(8,8)$&lt;/span&gt; receptive field from the initial &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, let us discuss these: Using max-pooling reduces the feature space heavily by throwing out a lot of nodes whose features aren't as indicative (makes training models more tractable) along with it does extend the receptive field with no additional parameters.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2019-07-27T00:07:22.763" LastActivityDate="2019-07-27T00:07:22.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13584" PostTypeId="5" CreationDate="2019-07-25T19:33:41.800" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-optimization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-organizing_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-organizing_network&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Mathematical_optimization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Engineering_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Engineering_optimization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Process_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Process_optimization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Optimality_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Optimality_model&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Profile-guided_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Profile-guided_optimization&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-25T19:33:41.800" LastActivityDate="2019-07-25T19:33:41.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13585" PostTypeId="4" CreationDate="2019-07-25T19:33:41.800" Score="0" Body="Use for the concept of self-optimization in general.  (i.e. use may be for theoretical subjects, not restricted to Self Organizing Networks in communications technology.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-25T19:33:41.800" LastActivityDate="2019-07-25T19:33:41.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13586" PostTypeId="5" CreationDate="2019-07-25T20:26:20.180" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-25T20:26:20.180" LastActivityDate="2019-07-25T20:26:20.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13587" PostTypeId="4" CreationDate="2019-07-25T20:26:20.180" Score="0" Body="Use this tag for disambiguation of terms or concepts that may be similar. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-25T20:26:20.180" LastActivityDate="2019-07-25T20:26:20.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13588" PostTypeId="2" ParentId="13572" CreationDate="2019-07-25T21:18:16.923" Score="2" Body="&lt;p&gt;The Wikipedia article related to computer vision gives, in my opinion, a good description of the field and its relation to image processing. Below, I will only cite the most relevant parts of the article.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer vision&lt;/a&gt;&lt;/strong&gt; is an &lt;em&gt;interdisciplinary&lt;/em&gt; scientific field that deals with how computers can be made to gain &lt;em&gt;high-level understanding from digital images or videos&lt;/em&gt;. From the perspective of engineering, it seeks to automate tasks that the human visual system can do.&lt;/p&gt;&#xA;&lt;p&gt;The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner.&lt;/p&gt;&#xA;&lt;p&gt;Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, and image restoration.&lt;/p&gt;&#xA;&lt;p&gt;The fields most closely related to computer vision are image processing, image analysis and machine vision. There is a significant overlap in the range of techniques and applications that these cover. This implies that the basic techniques that are used and developed in these fields are similar, something which can be interpreted as there is only one field with different names. On the other hand, it appears to be necessary for research groups, scientific journals, conferences and companies to present or market themselves as belonging specifically to one of these fields and, hence, various characterizations which distinguish each of the fields from the others have been presented.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Image processing&lt;/strong&gt; and image analysis tend to focus on 2D images, &lt;em&gt;how to transform one image to another&lt;/em&gt;, e.g., by pixel-wise operations such as contrast enhancement, local operations such as &lt;em&gt;edge extraction&lt;/em&gt; or &lt;em&gt;noise removal&lt;/em&gt;, or geometrical transformations such as rotating the image. This characterization implies that image processing/analysis neither require assumptions nor produce interpretations about the image content.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;What is the difference between computer vision and image processing?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Computer vision&lt;/strong&gt; is about gaining &lt;strong&gt;high-level understanding from images or videos&lt;/strong&gt;. For example, object recognition, which is the task of identifying the type of objects (e.g. apples or humans) in an image, is a computer vision problem. Of course, this task requires a high-level understanding of the image, that is, an understanding of the image similar to the way humans understand visual inputs, given that an apple is a high-level object that is composed of atoms, can be green, etc. For example, a neural network that attempts to classify the type of object in an image (assuming, for simplicity, there is just one type of object) would be a computer vision technique. In computer vision, you receive an image as input and you can produce an image as output or some other type of information (e.g. the type of objects in the image).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, &lt;strong&gt;image processing&lt;/strong&gt; does &lt;em&gt;not necessarily&lt;/em&gt; imply a high-level understanding of the image. Image processing is a subfield of signal processing but applied to images, which are &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;d signals (or functions of a fixed domain). So, for example, if you have a blurred or noisy image, the task of deblurring or denoising it is part of image processing. The typical tasks in image processing are filtering (e.g. using the Gaussian filter or the mean filter), noise removal, edge detection and color processing. In image processing, you receive an image as input and you produce another image as output.&lt;/p&gt;&#xA;&lt;p&gt;However, note that, in many cases, to gain a high-level understanding of the images, you first need to e.g. denoise them, so you could use an image processing technique to partially solve a computer vision task. In this sense, computer vision is an interdisciplinary field.&lt;/p&gt;&#xA;&lt;p&gt;To conclude, computer vision is not a subfield of image processing, given that image processing does not necessarily involve a high-level understanding of images. On the other hand, computer vision can use image processing techniques to gain a high-level understanding of images.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-07-25T22:25:27.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13591" PostTypeId="2" ParentId="13573" CreationDate="2019-07-26T02:50:45.170" Score="1" Body="&lt;p&gt;Ok, here's the break down:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The depth of an input to a convolutional layer is termed channels. The depth of a convolutional layer is the number of kernels (aka filters). The depth of a &lt;em&gt;kernel&lt;/em&gt; is equal to the number of channels in the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z9kb7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z9kb7.png&quot; alt=&quot;Convolution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input (of 7x7, pad of 1) has 3 channels. The convolutional layer has 2 kernels (or filters). Each filter has a depth of 3, equal to the number of channels in the input. Using the notation you used in your question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Filter 1, channel 1 to input channel 1&lt;/li&gt;&#xA;&lt;li&gt;Filter 1, channel 2 to input channel 2&lt;/li&gt;&#xA;&lt;li&gt;Filter 1, channel 3 to input channel 3&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sum all three channels of filter 1, then add bias&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Filter 2, channel 1 to input channel 1&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Filter 2, channel 2 to input channel 2&lt;/li&gt;&#xA;&lt;li&gt;Filter 2, channel 3 to input channel 3&lt;/li&gt;&#xA;&lt;li&gt;Sum all three channels of filter 2, then add bias&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These steps are repeated for each frame the filter slides over the input image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer question 2, if the output is 128, that simply means there are 128 filters. There could be an infinite number of filters if you so choose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the link to the interactive graphic: &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/convolutional-networks/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-07-26T02:50:45.170" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="13593" PostTypeId="1" CreationDate="2019-07-26T08:31:06.503" Score="1" ViewCount="37" Body="&lt;p&gt;While I have limited resource usually to train my machine learning models, I often find that my hyperparameter optimization procedure is not necessary using all my GPU and CPU, and that is because the results also depend on the batch size in my experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you find in your project that a low batch size is necessary, how do you scale your project? In a multi-GPU scenario, I could imagine running different hyperparameter settings on different GPUs, but what other options are out there?&lt;/p&gt;&#xA;" OwnerUserId="27440" LastEditorUserId="1671" LastEditDate="2019-07-29T20:49:27.303" LastActivityDate="2019-07-29T20:49:27.303" Title="How do you scale your ML problems?" Tags="&lt;machine-learning&gt;&lt;hyperparameter-optimization&gt;&lt;scalability&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13594" PostTypeId="2" ParentId="13523" CreationDate="2019-07-26T09:05:48.727" Score="1" Body="&lt;p&gt;There are works on expandable neural networks, for example, &lt;a href=&quot;https://arxiv.org/abs/1708.01547&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lifelong learning with dynamically expandable networks&lt;/a&gt; by Yoon et al. So, if you consider the whole system with expanding algorithm (in the same way as you consider an SGD optimizer as part of a network system), then yes, it's totally possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But they didn't show breakthrough performance on any task, as far as I know, at least at the moment. The key challenge is that's hard to say, do you need to expand or just train more - and where exactly do you need to expand.&lt;/p&gt;&#xA;" OwnerUserId="16940" LastEditorUserId="2444" LastEditDate="2019-07-28T19:52:12.433" LastActivityDate="2019-07-28T19:52:12.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13595" PostTypeId="1" AcceptedAnswerId="13597" CreationDate="2019-07-26T09:35:05.530" Score="3" ViewCount="34" Body="&lt;p&gt;If I use my mobile camera on a signboard or announcement board on a road or in a street (like the one attached in photo) where the message is written in Russian and my mobile shows me that message in English, would this be an image processing or computer vision technique?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1FJ01.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1FJ01.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="27427" LastEditorUserId="2444" LastEditDate="2019-07-26T09:39:45.427" LastActivityDate="2019-07-26T09:42:24.330" Title="Is this technique image processing or computer vision?" Tags="&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;machine-translation&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13596" PostTypeId="1" AcceptedAnswerId="13601" CreationDate="2019-07-26T09:39:21.027" Score="2" ViewCount="47" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9bi7E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9bi7E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3fnEL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3fnEL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is from the book Pattern Recognition by Bishop. Why is expectation here a simple average? Why is &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; not being multiplied by &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="27422" LastEditorUserId="2444" LastEditDate="2019-07-26T20:38:23.437" LastActivityDate="2019-07-26T20:38:23.437" Title="Why is the expectation calculated over finite number of points drawn from a probability distribution?" Tags="&lt;math&gt;&lt;probability-distribution&gt;&lt;expectation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13597" PostTypeId="2" ParentId="13595" CreationDate="2019-07-26T09:42:24.330" Score="0" Body="&lt;p&gt;This is a computer vision task, given that it requires a high-level understanding of the image (that is, it contains text, and the text needs to be translated to another language), which can use an image processing technique (for example, to deblur the possibly blurry original photo).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-26T09:42:24.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13598" PostTypeId="2" ParentId="13523" CreationDate="2019-07-26T09:43:16.333" Score="6" Body="&lt;p&gt;Since a neural network does iteratively learn its own weights I assume you mean the structure of the neural network - the number of layers and nodes per layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If what I said above was your question, then yes, it most definitely is being explored.&#xA;Even when a neural network is allowed to learn its own structure it still needs to be suited to a specific problem. &#xA;Google wrote a blog where they &lt;a href=&quot;https://ai.googleblog.com/2017/05/using-machine-learning-to-explore.html&quot; rel=&quot;noreferrer&quot;&gt;used machine learning to explore the architecture of a neural network&lt;/a&gt; calling it the AutoML project where the blog claimed &lt;em&gt;&quot;To make this process of designing machine learning models much more accessible, we’ve been exploring ways to automate the design of machine learning models.&quot;&lt;/em&gt;&#xA;It was found that AutoML can design small neural networks that perform on par with neural networks designed by human experts, but these results were constrained to small academic datasets like CIFAR-10, and Penn Treebank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This study was further extended to more challenging problems and datasets in another blog where they proposed &lt;a href=&quot;https://ai.googleblog.com/2017/11/automl-for-large-scale-image.html&quot; rel=&quot;noreferrer&quot;&gt;NASNet&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both these blogs were written in 2017 and I'm not very sure how much progress has been made on this since, but I hope these blogs helped!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="25658" LastEditDate="2019-07-26T09:52:46.750" LastActivityDate="2019-07-26T09:52:46.750" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13599" PostTypeId="2" ParentId="13570" CreationDate="2019-07-26T10:28:55.117" Score="1" Body="&lt;p&gt;This is an idea that I used for my model - &#xA;Try using two RNN (GRU) Networks, one of them to manage current output state and the other to maintain context&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say we are at timestamp &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and the two GRUCells are represented as &lt;span class=&quot;math-container&quot;&gt;$GRU_c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$GRU_s$&lt;/span&gt; for GRU context network and state network. (Your output coming from the state network)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At time stamp &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; , the input &lt;span class=&quot;math-container&quot;&gt;$GRU_s(t) = concat(input, att(all~GRU_c~from ~[0, ~t-1]))$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$att$&lt;/span&gt; is an attention mechanism to give importance to specific parts of the conversation uptil that point (This is what maintains context)  and input &lt;span class=&quot;math-container&quot;&gt;$GRU_c(t) = learned~representation~of~GRU_s(t)$&lt;/span&gt; , hence updating &lt;span class=&quot;math-container&quot;&gt;$GRU_c$&lt;/span&gt; for that timestamp, which along with the historical information can be used for &lt;span class=&quot;math-container&quot;&gt;$GRU_s(t+1)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helped!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-07-26T10:28:55.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13600" PostTypeId="1" CreationDate="2019-07-26T10:51:32.840" Score="2" ViewCount="298" Body="&lt;p&gt;I have very outdated idea about how NLP tasks are carried out by normal RNN's, LSTM's/GRU's, word2vec, etc to basically generate some hidden form of the sentence understood by the machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the things I have noticed is that in general researchers are interested in generating the context of the sentence, but oftentimes ignore punctuation marks which is on of the most important aspects for generating context. For example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;“Most of the time, travellers worry about their luggage.”&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;“Most of the time travellers worry about their luggage”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cybertext.wordpress.com/2012/11/22/a-light-hearted-look-at-how-punctuation-can-change-meaning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like this there exists probably 4 important punctuation marks &lt;code&gt;.,?&lt;/code&gt; and &lt;code&gt;!&lt;/code&gt;. Yet, I have not seen any significant tutorials/blogs on them. It is also interesting to note that punctuations don't have a meaning (quite important, since most language models try to map word to a numerical value/meaning), they are more of a 'delimiter'. So what is the current theory or perspective on this? And why is it ignored?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-07-27T09:52:10.610" LastActivityDate="2019-08-02T15:34:16.027" Title="Why do language models place less importance on punctuation?" Tags="&lt;natural-language-processing&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13601" PostTypeId="2" ParentId="13596" CreationDate="2019-07-26T10:56:51.547" Score="1" Body="&lt;p&gt;When we say that we have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; points that were &lt;em&gt;&quot;drawn from the probability distribution or probability density&quot;&lt;/em&gt;, this means that every point &lt;span class=&quot;math-container&quot;&gt;$x_n$&lt;/span&gt; had the correct probability &lt;span class=&quot;math-container&quot;&gt;$p(x_n)$&lt;/span&gt; of being sampled from the distribution when we were sampling our &lt;span class=&quot;math-container&quot;&gt;$n^{th}$&lt;/span&gt; point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that we wish to compute/estimate the expected value of a distribution given by flipping a coin, which is weighted such that it lands on heads two-thirds of the time (&lt;em&gt;value of heads&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$= 1$&lt;/span&gt;), and tails one-thirds of the time (&lt;em&gt;value of tails&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$= 0$&lt;/span&gt;). This means that the ground-truth probabilities are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p(1) = \frac{2}{3}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p(0) = \frac{1}{3}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this case, we actually wouldn't have to estimate anything by sampling; the exact probabilities are known, so we could just compute the expected value as &lt;span class=&quot;math-container&quot;&gt;$p(1) \times 1 + p(0) \times 0 = \frac{2}{3}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now suppose that we do not know exactly how the coin is weighted, i.e. we do not know the exact values of &lt;span class=&quot;math-container&quot;&gt;$p(1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(0)$&lt;/span&gt;. If we simply flip our weighted coin &lt;span class=&quot;math-container&quot;&gt;$N = 100$&lt;/span&gt; times, we will in expectation find about &lt;span class=&quot;math-container&quot;&gt;$67$&lt;/span&gt; samples of &lt;span class=&quot;math-container&quot;&gt;$x_n = 1$&lt;/span&gt;, and about &lt;span class=&quot;math-container&quot;&gt;$33$&lt;/span&gt; samples of &lt;span class=&quot;math-container&quot;&gt;$x_n = 0$&lt;/span&gt; (rounded to integers because we can't obtain a non-integer number of observations). So we cannot explicitly use the probabilities (because we do not know them), but they will implicitly be present in the number of repeated observations we have for every possible datapoint. &lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-26T10:56:51.547" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13602" PostTypeId="2" ParentId="13600" CreationDate="2019-07-26T11:21:50.397" Score="1" Body="&lt;p&gt;Language models almost always map every word to an embedding. There are many embedding algorithms with most of them having interpolation properties i.e. If &lt;span class=&quot;math-container&quot;&gt;$E(word)$&lt;/span&gt; represents embedding of a word then &lt;span class=&quot;math-container&quot;&gt;$E(king)-E(male)+E(female) \sim E(queen)$&lt;/span&gt;. The smoother the interpolation properties the better the model understands the word, these properties don't exactly make much sense when it comes to delimiters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, there are instances where a delimiter embedding is learned ( always has an embedding ). While using these first all punctuation in the text is converted to one specific word, say 'dlmt', the embedding algorithm learns an embedding for this word treating it as if it would have any word. This maintains interpolation properties where the delimiter is understood to be a word that is used to split context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have observed that delimiters such as question mark or exclamation marks at the end of sentence are also understood to be breaks in context, in these cases the model learns if the statement is a question or so just by the context given by the words and stops in the sentence&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-07-26T11:21:50.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13603" PostTypeId="1" CreationDate="2019-07-26T11:54:25.757" Score="2" ViewCount="151" Body="&lt;p&gt;I am trying to build an RL agent to solve the NP-hard problem graph coloring. The problem is quite challenging. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This how I addressed it.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The environment&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;To preserve the scalability of the algorithm, providing the agent with the whole graph wouldn't be a good idea. Therefore, the input for the agent would be a window of embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, first, I would apply an embedding to the graph to generate fixed-size vectors to every vertex in the graph (thus, every vertex in the graph is represented as a vector that contains some information about its neighborhood and position in the graph).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the agent will get a window of the embedding. For example, when coloring vertex number &lt;span class=&quot;math-container&quot;&gt;$17$&lt;/span&gt;, the input would be the &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; vectors from vertex &lt;span class=&quot;math-container&quot;&gt;$17-n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$17+n$&lt;/span&gt;, to give the agent more local information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, I think the agent would require more information about the number of colors already used and the number of the already colores vertices.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The agent&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;My biggest problem is how the agent should be. Technically, the problem is the action space dimension. For a given graph, the maximal number of colors is the number of vertices which varies from graph to graph (losing the scalability). Plus, the possible actions at each state varies with the history of the coloring. The possible colors for a given state (or node) are all the used colors eliminating the connected colors and adding the possibility of a new color, that is, for vertex &lt;span class=&quot;math-container&quot;&gt;$56$&lt;/span&gt;, the agent has already used the first &lt;span class=&quot;math-container&quot;&gt;$40$&lt;/span&gt; colors &lt;span class=&quot;math-container&quot;&gt;$\{0, 1, 2, 3, \dots,40 \}$&lt;/span&gt;, and node &lt;span class=&quot;math-container&quot;&gt;$56$&lt;/span&gt; is connected to some neighbors already colored with &lt;span class=&quot;math-container&quot;&gt;$\{14, 22, 40 \}$&lt;/span&gt;, the possible colors are &lt;span class=&quot;math-container&quot;&gt;$\{0,1, \dots, 40 \}- \{14, 22, 40 \} + \{41\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How do I overcome the high dimensional inconsistent action space?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="27443" LastEditorUserId="2444" LastEditDate="2019-07-27T15:15:37.450" LastActivityDate="2019-07-27T15:15:37.450" Title="Coloring graphs with reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13604" PostTypeId="1" AcceptedAnswerId="13672" CreationDate="2019-07-26T12:39:24.413" Score="1" ViewCount="91" Body="&lt;p&gt;I'm training a text classifier in PyTorch and I'm experiencing an unexplainable cyclical pattern in the loss curve. The loss drops drastically at the beginning of each epoch and then starts rising slowly. However, the global convergence pattern seems OK. Here's how it looks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/15Azq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/15Azq.png&quot; alt=&quot;loss curve (global)&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/4VTjD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4VTjD.png&quot; alt=&quot;loss curve (zoom)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model is very basic and I'm using the Adam optimizer with default parameters and a learning rate of 0.001. Batches are of 512 samples. I've checked and tried a lot of stuff, so I'm running out of ideas, but I'm sure I've made a mistake somewhere.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Things I've made sure of:&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data is delivered correctly (VQA v1.0 questions).&lt;/li&gt;&#xA;&lt;li&gt;DataLoader is shuffling the dataset.&lt;/li&gt;&#xA;&lt;li&gt;LSTM's memory is being zeroed correctly&lt;/li&gt;&#xA;&lt;li&gt;Gradient isn't leaking through input tensors.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Things I've already tried:&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lowering the learning rate. Pattern remains, although amplitude is lower.&lt;/li&gt;&#xA;&lt;li&gt;Training without momentum (plain SGD). Gradient noise masks the pattern a bit, but it's still there.&lt;/li&gt;&#xA;&lt;li&gt;Using a smaller batch size (gradient noise can grow until it kinda masks the pattern, but that's not like solving it).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;The model&lt;/h1&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class QuestionAnswerer(nn.Module):&#xA;&#xA;    def __init__(self):&#xA;        super(QuestionAnswerer, self).__init__()&#xA;        self._wemb = nn.Embedding(N_WORDS, HIDDEN_UNITS, padding_idx=NULL_ID)&#xA;        self._lstm = nn.LSTM(HIDDEN_UNITS, HIDDEN_UNITS)&#xA;        self._final = nn.Linear(HIDDEN_UNITS, N_ANSWERS)&#xA;&#xA;    def forward(self, question, length):&#xA;        B = length.size(0)&#xA;        embed = self._wemb(question)&#xA;        hidden = self._lstm(embed)[0][length-1, torch.arange(B)]&#xA;        return self._final(hidden)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-08-01T11:23:30.283" Title="LSTM text classifier shows unexpected cyclical pattern in loss" Tags="&lt;long-short-term-memory&gt;&lt;loss-functions&gt;&lt;pytorch&gt;&lt;learning-curve&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13605" PostTypeId="1" AcceptedAnswerId="13704" CreationDate="2019-07-26T14:39:21.703" Score="4" ViewCount="160" Body="&lt;p&gt;&lt;strong&gt;Introduction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI in a box experiment is about a super strong game AI which starts with lower resources than the opponent and the question is, if the AI is able to win the game at the end, which is equal to escape from the prison. A typical example is a match of computer chess in which the AI player starts only with a king, but the human starts with all the 16 pieces including the queen, and the powerful bishop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Winning the game&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of a very asymmetric setup, the AI has no chance to win the game. Even if the AI thinks 100 moves ahead, a single king can't win against 16 opponent figures. But what happens, if the AI starts with 8 pieces and the human with 16? A formalized hypothesis will look like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;strength of the AI x resources weakness = strength of the human x resources strength&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To put the AI for sure into a prison, the strength of the AI should be low and it's resources too. If the resources are low but the strength is middle, then the AI has a certain chance to escape from the prison. And if the AI has maximum strength and maximum resources, then the human player gets a serious problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this formalized prediction supported by the AI literature in academia?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="1671" LastEditDate="2019-07-31T17:57:54.740" LastActivityDate="2019-07-31T17:57:54.740" Title="Can the AI in a box experiment be formalized?" Tags="&lt;philosophy&gt;&lt;control-problem&gt;&lt;ai-box&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13606" PostTypeId="2" ParentId="13600" CreationDate="2019-07-26T14:51:55.547" Score="1" Body="&lt;p&gt;You are right. Approaches that map words to meaning solely do fail in this regard. None the less Word2Vec and Glove have shown wonderful downstream results. This in itself may indicate that most of the time, punctuation's addition can be interpolated. But as you provided, there are cases where this just is not true!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now of days I would say most models actually use almost &lt;strong&gt;NO&lt;/strong&gt; preprocessing. This is surprising but its due to the rise in power of learnable, reversable, tokenizations. Some examples of these include &lt;a href=&quot;https://en.wikipedia.org/wiki/Byte_pair_encoding&quot; rel=&quot;nofollow noreferrer&quot;&gt;byte pair encoding (bpe)&lt;/a&gt; and the &lt;a href=&quot;https://arxiv.org/abs/1808.06226&quot; rel=&quot;nofollow noreferrer&quot;&gt;sentence piece model (spm)&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;State-of-the-art NLP generally rely on these. Examples include &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt; and &lt;a href=&quot;https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT2&lt;/a&gt;, which are general purpose pretrained Language Models. Their ability to parse and &lt;em&gt;understand&lt;/em&gt; (i use this word loosely) a wide variety of phrasing, spelling and more can be partially due to the freedom in the preprocessing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Takeaway:&lt;/strong&gt; You can achieve good results by using preprocessing in a manner that will eliminate information but keep the meat and bones that you are interested in (but this requires domain knowledge paired with optimization experience), but the field seems gearing towards models that are more inclusive, more transferable, and dont have the problems you mention by design.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-26T14:51:55.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13607" PostTypeId="1" AcceptedAnswerId="13675" CreationDate="2019-07-26T15:09:27.237" Score="4" ViewCount="120" Body="&lt;p&gt;I'm very new to AI.  I read somewhere that AI can be used to create GUI UI/UX design. That has fascinated me for a long time. But, since I'm very new here, I don't have any idea how it can happen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual steps to create the UI Design are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create Grids.&lt;/li&gt;&#xA;&lt;li&gt;Draw Buttons/Text/Boxes/Borders/styles.&lt;/li&gt;&#xA;&lt;li&gt;Choose Color Schemes.&lt;/li&gt;&#xA;&lt;li&gt;Follow CRAP Principle (Contrast, Repeatition, Alignment, Proximity)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I wonder how can AI algorithms help with that. I know a bit of neural networking and the closest I can think of is the following two methods (Supervised Learning).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Draw grids manually and train the Software manually to learn proper styles until it becomes capable of giving modern results and design its own design language.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Take a list of a few websites (for example) from the internet and let the software learn and explore the source code and CSS style sheets and learn and program neurons manually until it becomes capable of making it's own unique styles.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user27450" LastEditorUserId="5763" LastEditDate="2019-07-29T20:27:34.473" LastActivityDate="2019-07-30T13:32:32.693" Title="How can AI be used to design UI Interfaces?" Tags="&lt;neural-networks&gt;&lt;supervised-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13608" PostTypeId="2" ParentId="13607" CreationDate="2019-07-26T17:11:11.733" Score="1" Body="&lt;p&gt;If you see the use case, on higher level it seems to generate some visual output - the design but when seen at lower level, this design is output of some code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way we can do it is to train a neural network that learns to generate code which can be seen as some form of organized text. So now can be treated as a text generation problem on which you can find a lot of literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1705.07962&quot; rel=&quot;nofollow noreferrer&quot;&gt;pix2code&lt;/a&gt; is one implementation that uses this idea and even expands it. In this paper the authors took it to another level, they also used the visual part - the GUI and built an architecture that takes both the code and the GUI as input and learns to generate code. So eventually it would be capable of generating code when a simple GUI was given. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also there are some implementations where even the network could produce code even when a rough sketch was given.&lt;/p&gt;&#xA;" OwnerUserId="15852" LastActivityDate="2019-07-26T17:11:11.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13609" PostTypeId="2" ParentId="13540" CreationDate="2019-07-26T18:17:23.070" Score="1" Body="&lt;p&gt;So for anyone struggling to understand the &lt;a href=&quot;https://spinningup.openai.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI's Spinning Up&lt;/a&gt; educational resource, I'll provide the answer to my question here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, it's important to understand that the &lt;a href=&quot;https://spinningup.openai.com/en/latest/user/algorithms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithms&lt;/a&gt; expect a 2-dimensional input &lt;strong&gt;shape&lt;/strong&gt;, in rudimentary terms a shape of &lt;code&gt;Box(int)&lt;/code&gt;, which isn't the case with the default &lt;a href=&quot;https://gym.openai.com/envs/Breakout-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Breakout-v0&lt;/a&gt; game environment, which supplies inputs in the shape &lt;code&gt;Box(210, 160, 3)&lt;/code&gt;, which is the game screen's height, width, and RGB colour space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://gym.openai.com/envs/Breakout-ram-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Breakout-ram-v0&lt;/a&gt; game environment, however, &lt;em&gt;DOES&lt;/em&gt; provide an input of the appropriate shape, consequently &lt;code&gt;Box(128,)&lt;/code&gt;. So switching to this environment solves the initial problem I was having. To train any of the on-policy algorithms (&lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ppo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO&lt;/a&gt; in my case), the command should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python -m spinup.run ppo --env Breakout-ram-v0 --exp_name simpletest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To take this a step further, how would you train an off-policy algorithm (such as &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPQ&lt;/a&gt;)? Well, the off-policy algorithms expect a &lt;strong&gt;continuous action space&lt;/strong&gt;, which can be seen in the differences in the source code here between &lt;a href=&quot;https://gym.openai.com/envs/MountainCar-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MountainCar-v0&lt;/a&gt;, and &lt;a href=&quot;https://gym.openai.com/envs/MountainCarContinuous-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MountainCarContinuous-v0&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.action_space = spaces.Discrete(3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...versus:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.action_space = spaces.Box(low=self.min_action, high=self.max_action,&#xA;    shape=(1,), dtype=np.float32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, to sum up, how would you train something like the &lt;a href=&quot;https://gym.openai.com/envs/LunarLander-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lunar Lander&lt;/a&gt; game environment with the DDPQ algorithm? Well, you'd need the continuous action space version, and by providing it with 2 dense layers with 192 nodes each, I was able to achieve the following results with just 200 epochs using the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python -m spinup.run ddpg --env LunarLanderContinuous-v2 --exp_name ddpq-test --hid [192,192] --epochs 200&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Npihs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Npihs.png&quot; alt=&quot;LunarLanderContinuous-v2 OpenAI DDPQ test&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pretty cool...anyways, I hope this helps someone sometime, and I would thoroughly recommend looking at the source code for the various OpenAI game environments, as well as their &lt;a href=&quot;https://github.com/openai/gym/wiki/Table-of-environments&quot; rel=&quot;nofollow noreferrer&quot;&gt;table of game environments&lt;/a&gt; for a quick reference to the observation and action spaces. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="27392" LastActivityDate="2019-07-26T18:17:23.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13610" PostTypeId="5" CreationDate="2019-07-26T20:09:46.160" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-26T20:09:46.160" LastActivityDate="2019-07-26T20:09:46.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13611" PostTypeId="4" CreationDate="2019-07-26T20:09:46.160" Score="0" Body="For questions related to the convolution operation in mathematics, convolutional neural networks, image processing and computer vision." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-07T17:11:43.477" LastActivityDate="2020-05-07T17:11:43.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13612" PostTypeId="1" AcceptedAnswerId="13626" CreationDate="2019-07-27T06:23:37.193" Score="9" ViewCount="2111" Body="&lt;p&gt;How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?&lt;/p&gt;&#xA;" OwnerUserId="27462" LastEditorUserId="2444" LastEditDate="2020-10-10T21:56:01.943" LastActivityDate="2020-10-10T21:56:01.943" Title="What is the time complexity of the forward pass algorithm of a feedforward neural network?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;time-complexity&gt;&lt;complexity-theory&gt;&lt;forward-pass&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13615" PostTypeId="1" AcceptedAnswerId="13624" CreationDate="2019-07-27T08:24:08.187" Score="1" ViewCount="51" Body="&lt;p&gt;I need to identify the number and type of all objects in a picture, so there can be multiple objects of the same type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have a picture with &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; animals, and I want my program to tell me that, on the picture, I have &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; elephants, &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; cats and &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; dogs. However, I &lt;strong&gt;do not&lt;/strong&gt; need the detection of the location of the objects. All I need is the information on the number of objects of each class, without their possible locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask you guys for help in defining the type of problem I am dealing with and maybe some suggestions about where to start looking for a solution. It would be nice if you could point out some directions, algorithms or network architectures to solve the problem described below.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2019-07-27T20:58:34.910" LastActivityDate="2019-07-27T21:08:38.207" Title="How do I identify the number and type of objects in the same picture?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13617" PostTypeId="1" AcceptedAnswerId="13618" CreationDate="2019-07-27T12:00:28.540" Score="2" ViewCount="252" Body="&lt;p&gt;I was going through an article where it is mentioned:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Monte-Carlo methods require only knowledge base (history/past experiences)—sample sequences of (states, actions and rewards) from the interaction with the environment, and no actual model of the environment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Aren't the model-based method dependent on past sequences? How is Monte Carlo is different than?&lt;/p&gt;&#xA;" OwnerUserId="27470" LastEditorUserId="2444" LastEditDate="2019-07-27T12:15:42.110" LastActivityDate="2019-07-28T22:21:24.200" Title="How is Monte Carlo different from model-based methods?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;monte-carlo-methods&gt;&lt;model-based-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13618" PostTypeId="2" ParentId="13617" CreationDate="2019-07-27T13:01:27.880" Score="0" Body="&lt;p&gt;Model-based methods (such as value or &lt;a href=&quot;http://www.incompleteideas.net/book/first/ebook/node43.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;policy iteration&lt;/a&gt;) use a &lt;em&gt;model&lt;/em&gt; of the environment, which is usually represented as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov decision process&lt;/a&gt;. More specifically, the model consists of the transition and reward functions of the Markov decision process, which should represent the &lt;em&gt;dynamics&lt;/em&gt; of the environment. For example, in &lt;a href=&quot;http://www.incompleteideas.net/book/first/ebook/node43.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;policy iteration&lt;/a&gt;, the rewards (used to estimate the policy or value functions) are not the result of the interaction with the environment but given by the MDP (the model of the environment), so the decisions are made according to the reward function (and the transition function) of the MDP that represents the dynamics of the environment. Model-based methods are not (usually) dependent on past actions. For example, policy iteration converges to the optimal policy independently of the initial values of the states, the initial policy or the order of iteration through the states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo methods do not &lt;strong&gt;use&lt;/strong&gt; such a model (the MDP), even though the assumption that the environment can be represented as an MDP is (often implicitly) made (and the MDP might actually be available). In the case of Monte Carlo methods, all estimates are solely based on the interaction with the environment. In general, Monte Carlo methods are based on sampling (or random) operations. In the case of reinforcement learning, they sample the environment. The samples are the rewards that are obtained when certain actions are taken from certain states.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-28T22:21:24.200" LastActivityDate="2019-07-28T22:21:24.200" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13619" PostTypeId="1" AcceptedAnswerId="13621" CreationDate="2019-07-27T14:04:02.170" Score="4" ViewCount="363" Body="&lt;p&gt;I am new to reinforcement learning. I would like to solve an optimal control problem with reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is for a wolf to catch a rabbit. The wolf and the rabbit run on a plane. The time is discrete. At every time step, the wolf can only run straight, change direction by 10 degrees to the right or left, change the speed by 0.1 m/s or remain at the same speed. It starts running in some random direction, and then sees the rabbit and starts chasing it. For the time being, let's assume that the rabbit sits still. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like this problem is a continuous state space and discrete action space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to use DQN in Keras, but I am not sure that I am using correct state variables/reward. Currently, the state variables are the velocity vector of the wolf, the distance vector from the wolf to the rabbit. The reward at each time point is the negative current time. When the wolf catches the rabbit, the reward is 1000 - current time (the wolf is penalized for running too long). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody provide me some guidance? Eventually, I would add brains to the rabbit so that it tries to escape the wolf and compare to the optimal control solution.&lt;/p&gt;&#xA;" OwnerUserId="27472" LastEditorUserId="2444" LastEditDate="2019-07-27T14:43:52.553" LastActivityDate="2019-07-27T22:59:39.453" Title="How do I solve this optimal control problem with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;control-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13620" PostTypeId="1" CreationDate="2019-07-27T14:58:15.827" Score="1" ViewCount="325" Body="&lt;p&gt;I am seeking help to implement my idea on developing an automation test framework that can fix locator related issues by itself. My abstract idea is as given below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will have a Selenium Test Automation framework that will run in three modes - &lt;code&gt;Learning&lt;/code&gt;, &lt;code&gt;Analysis and Fix&lt;/code&gt;, and &lt;code&gt;Normal&lt;/code&gt; modes. When I run on Learning mode (assume that all test cases run fine on current build) the framework will run all test cases and it will capture screenshot of the web element it interact with (may be the html of the web page instead of screenshot).&#xA;Once a new build is released I will run the framework in Analysis and Fix mode. In this mode the framework will navigate through each pages in the application and check whether the elements I already captured are valid in the current build or not. If the element is no longer valid, it will try and automatic fix. Framework will check the screenshot it already captured (or html file) in Learning mode and it will learn how the element looks like and it will try to identify the element in current build. Once the element is identified it will automatically generate suitable locator for the element and update the element locator file.&#xA;I know the question is not specific to a problem but I am trying to apply AI techniques it Selenium Web Automation framework. Python has rich library for AI, I prefer Java though. How to implement this idea?. &lt;/p&gt;&#xA;" OwnerUserId="27439" LastEditorUserId="27439" LastEditDate="2019-07-29T03:58:25.410" LastActivityDate="2021-01-09T18:43:04.530" Title="AI for Selenium Test Automation" Tags="&lt;ai-design&gt;&lt;java&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13621" PostTypeId="2" ParentId="13619" CreationDate="2019-07-27T18:07:40.397" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have tried to use DQN in Keras, but I am not sure that I am using correct state variables/reward. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You have a wide range of choices that are all valid. As it is a simple control and learning scenario, provided you cover basics (described in a moment), then the difference in your choices are about how easy you make it for the agent to learn. You may actually want to set things up so that there is a certain difficultly level, so you get to compare different agent designs. You don't necessarily need the &quot;best&quot; state representation and reward function designs here. Just some that work.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;State representation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The most important factor is that your state has the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; - that it contains enough information that the agent really can predict expected future rewards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is OK for there to be randomness, but generally not OK for there to be important hidden variables that affect the outcome in a major way. If there are such variables in the environment, but not in the state representation, then a simple agent like DQN will struggle.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Reward function&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The reward function should capture the end goals that you are interested in, and ideally as simply and purely as possible given that. The purpose of most RL methods is to maximise the expected sum of reward, and they are built to deal with sparsity - you don't need to help or signal the same thing twice or give rewards for &quot;getting close&quot; etc, although sometimes this can improve learning speed if done well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also written a more comprehensive answer about reward function design as an &lt;a href=&quot;https://ai.stackexchange.com/a/12299/1847&quot;&gt;answer to Deciding on a reward per each action in a given state (Q-learning)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Currently, the state variables are the velocity vector of the wolf, the distance vector from the wolf to the rabbit. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems reasonable. Do consider that for a neural network you will want to keep the scale of these to within a nice range e.g. -1 to +1 for all features. So you may want to scale from whatever you are using for the values in the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make this problem easier to learn by using polar co-ordinates. That removes the need for the neural network to solve the inherent trigonometry problem when deciding if turning left or right is better. I have solved a very similar pursuit RL problem and compared between cartesian and polar co-ordinates, and the difference is very large between the two representations for a DQN based agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically for the easiest learning you want the difference vector between your agent's current heading and the target, expressed as a difference in heading plus distance to go. If you do this, sometimes the initial randomised neural network will already be efficient at solving the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it may be interesting for you to see how well different agents do solve that trigonometry problem of converting a difference in vector coordinates and velocity, into a direction to turn. You could even make it harder and give absolute location coordinates for both agent and the target, plus the agent velocity (3 vectors instead of 2), requiring the neural network to approximate even more complex maths before it solves the problem.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The reward at each time point is the negative current time. When the wolf catches the rabbit, the reward is 1000 - current time (the wolf is penalized for running too long).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That should work, but seems more complex than you need. You are already penalising the wolf for each time step during the chase, so there is no real need to have any extra formula for the final reward. In fact you can just have a final reward of 0 for reaching a satisfactory goal (providing there are no unsatisfactory end points that you want it to avoid - perhaps yo umay make the environment trickier later on?). The agent will simply try to resolve the episode quickly because that will still maximise the (negative total) return.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative might be to reward some value for catching the rabbit e.g. +10, and zero on each time step. Then to encourage a fast resolution you would need to use discounting, so that the agent values being nearer to the rabbit higher than being further away because the potential reward is fewer time steps away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also no need to use large values. Values are relative, so if you have some minor issue to solve plus a main goal, sometimes it is worth having a wide range of values. Here you don't need it. Having a value of 1000 may challenge your neural network to learn properly (because error for the first time the wolf catches a rabbit would be so large it may cause a large step in weights - enough to destabilise learning in the NN), for no real benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your first reward scheme I recommend a fixed reward of e.g. -1 per time step, and to end the episode when the wolf catches the rabbit. No multiplications or offsets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a very large area for the wolf to explore compared to the distance that the wolf needs to be from the rabbit in order to catch it, it may help to give some reward for proximity to the rabbit. The simplest to implement and learn would just be the difference in distance between the wolf and rabbit between &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; multiplied some small factor to put this into a similar range to the penalty per time step. Note if you extend this reward out to any significant distance that it should make the learning problem far easier (combine polar coordinates and this reward system will make it very easy for the DQN to learn - it should take only a few episodes to get near optimal behaviour, if your learning hyperparameters are good for the DQN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without any proximity reward, you will rely on the wolf literally bumping into the rabbit through random behaviour, before it will have any data example that getting the vector between itself and the rabbit close to (0,0) is a good thing. You may need to have a relatively large capture radius, plus limit the area that the wolf (and eventually rabbit) can explore, in order to avoid very long sequences of random behaviour where nothing is learned initially.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-27T22:59:39.453" LastActivityDate="2019-07-27T22:59:39.453" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="13622" PostTypeId="1" CreationDate="2019-07-27T18:41:01.820" Score="1" ViewCount="78" Body="&lt;p&gt;I want to design a NN that can remember it's last 7 actions and use them as inputs. So for example it would be able to store words in it's memory. Therefore if it had a choice of 10 different actions, the number of words it could store is &lt;span class=&quot;math-container&quot;&gt;$10^7$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my design:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$out_{n+1} = f(out_n, in_n)\mathbf{N} + out_n.\mathbf{M}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$action_n = \sigma(\mathbf{N} \cdot out_n)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; represents some layered neural network. Some of the actions would be physical actions and some might be internal (such as thinking of the letter 'C').&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I want &lt;span class=&quot;math-container&quot;&gt;$out_n$&lt;/span&gt; to be an array that keeps the last 6 action values and puts them back in. So &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; will be the matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix}&#xA;0&amp;amp;1&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0\\&#xA;0&amp;amp;0&amp;amp;1&amp;amp;0&amp;amp;0&amp;amp;0\\&#xA;0&amp;amp;0&amp;amp;0&amp;amp;1&amp;amp;0&amp;amp;0\\&#xA;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;1&amp;amp;0\\&#xA;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;1\\&#xA;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. it would drop the 6th item from it's memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; would be the vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix}&#xA;1&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this would be equivalent to an equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$out_{n+1}=F(in_n,out_n,out_{n-1},out_{n-2},...,out_{n-6})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think this would be an advantage over an RNN since this model remembers precisely it's last 6 actions. But would this be better than an RNN or worse? One could increase it's memory to more than 7 quite easily. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's basically the same archececture as an RNN except elinimating a lot of the connections. Is this a new design or a common design?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem with this design is that you might also want a memory that is over longer time periods (e.g. for actions that take more than one tick.) But that might be solved by enhancing the archecture.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2019-07-27T19:52:04.203" LastActivityDate="2019-07-27T22:33:09.400" Title="Would this neural network have short term memory?" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13623" PostTypeId="2" ParentId="13622" CreationDate="2019-07-27T18:50:55.280" Score="2" Body="&lt;p&gt;Congrats, you have invented 1d convolution. Convolution combined with RNN would have some advantage over just RNN. Think about the perception field. &#xA;In this layer, you do aggregate &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt; values to one. Imagine two of them - it will be &lt;span class=&quot;math-container&quot;&gt;$36$&lt;/span&gt; already, etc. But, in the end, you still need RNN at the end to aggregate a variable length to constant length.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastEditorUserId="2444" LastEditDate="2019-07-27T22:33:09.400" LastActivityDate="2019-07-27T22:33:09.400" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13624" PostTypeId="2" ParentId="13615" CreationDate="2019-07-27T20:56:11.590" Score="0" Body="&lt;p&gt;Your problem might involve a combination of &lt;strong&gt;&lt;a href=&quot;https://www.mathworks.com/solutions/image-video-processing/object-recognition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;object recognition&lt;/a&gt;&lt;/strong&gt; (or object classification), which is the problem of determining which objects are in an image, and &lt;strong&gt;&lt;a href=&quot;https://www.mathworks.com/solutions/image-video-processing/object-recognition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;object detection&lt;/a&gt;&lt;/strong&gt; (which could also be called object localization), which is the task of locating a specific object in the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A naive approach to solve your problem would then be to first locate the position of all possible objects in the image (object detection), then, for each these positions, perform object recognition (or classification). Then you would count the number of detected and classified objects for each possible class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several models that can be used to solve your problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;YOLO&lt;/strong&gt; (You Only Look Once) model (introduced in the paper &lt;a href=&quot;https://arxiv.org/pdf/1506.02640.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;You Only Look Once: Unified, Real-Time Object Detection&lt;/a&gt;) (2016) can be used to detect multiple objects in an image. See the &lt;a href=&quot;https://pjreddie.com/darknet/yolo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;official YOLO implementation&lt;/a&gt;, which is part of the &lt;a href=&quot;https://github.com/pjreddie/darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Darknet&lt;/a&gt; framework. Have also a look at &lt;a href=&quot;https://towardsdatascience.com/an-introduction-to-implementing-the-yolo-algorithm-for-multi-object-detection-in-images-99cf240539&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, which explains how to use the official implementation of YOLO. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use the SSD model (introduced in the paper &lt;a href=&quot;https://arxiv.org/pdf/1512.02325.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single Shot MultiBox Detector&lt;/a&gt;). There is also &lt;a href=&quot;https://arxiv.org/abs/1311.2524&quot; rel=&quot;nofollow noreferrer&quot;&gt;R-CNN (Regions with CNN features)&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1504.08083.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast R-CNN&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/pdf/1506.01497.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faster R-CNN&lt;/a&gt;, but these should perform worse than both YOLO and SSD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, even if you do not care about the locations, YOLO (and other mentioned methods) can give them to you, in addition to the number and type of objects in the image.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-27T21:08:38.207" LastActivityDate="2019-07-27T21:08:38.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13626" PostTypeId="2" ParentId="13612" CreationDate="2019-07-27T23:13:56.277" Score="8" Body="&lt;p&gt;Let's suppose that we have an MLP with &lt;span class=&quot;math-container&quot;&gt;$15$&lt;/span&gt; inputs, &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; hidden neurons and &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; output neurons. The operations performed are only in the hidden and output neurons, given that the input neurons only represent the inputs (so they do not perform any operation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each hidden neuron performs a linear combination of its inputs followed by the application of a non-linear (or activation) function. So, each hidden neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; performs the following operation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;o_j = \sigma \left(\sum_{i}^{15} w_{ij}x_i \right),\tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the input coming from the input neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt; is the weight of the connection from the input neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to the hidden neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$o_j$&lt;/span&gt; is used to denote the output of neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; hidden neurons and, for each of them, according to equation &lt;span class=&quot;math-container&quot;&gt;$\ref{1}$&lt;/span&gt;, we perform &lt;span class=&quot;math-container&quot;&gt;$15$&lt;/span&gt; multiplications (ignoring any multiplications that might be associated with the activation function), so &lt;span class=&quot;math-container&quot;&gt;$15*20 = 300$&lt;/span&gt; multiplications are performed at the (only) hidden layer. In general, if there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; inputs and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; hidden neurons, then &lt;span class=&quot;math-container&quot;&gt;$n*m$&lt;/span&gt; multiplications will be performed in the first hidden layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, each neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; at the next layer (in this case, the output layer), also performs a linear combination followed by the application of an activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;o_j = \tau \left(\sum_{i}^{20} w_{ij}x_i \right),\tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is another activation function which might or not be equal to &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;, but we ignore all multiplications that might involve the application of the activation functions (we just want to count the ones in the linear combinations). Of course, in this case, &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; corresponds to the activation of neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; (of the hidden layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly to the previous reasoning, there are &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; output neurons and, to compute the output of each of them, &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; multiplications are performed (in the linear combination), so there are a total of &lt;span class=&quot;math-container&quot;&gt;$2*20 = 40$&lt;/span&gt; multiplications at the output layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, an MLP with &lt;span class=&quot;math-container&quot;&gt;$15$&lt;/span&gt; inputs, &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; hidden neurons and &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; output neurons will perform &lt;span class=&quot;math-container&quot;&gt;$15*20 + 20*2 = 340$&lt;/span&gt; multiplications (excluding activation functions). Of course, in this case, the number of multiplication depends not only on the number of neurons but also on the input size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, an MLP with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; inputs, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; hidden layers, where the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th hidden layer contains &lt;span class=&quot;math-container&quot;&gt;$m_i$&lt;/span&gt; hidden neurons, and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; output neurons will perform the following number of multiplications (excluding activation functions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;nm_{1} + m_{1}m_{2} + m_{2}m_{3} + \dots + m_{M-1}m_{M} + m_{M}k = nm_{1} + m_{M}k + \sum_{i=1}^{M-1} m_{i}m_{i+1} &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which, in big-O notation, can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\Theta\left(nm_{1} + m_{M}k + \sum_{i=1}^{M-1} m_{i}m_{i+1} \right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\Theta(\cdot)$&lt;/span&gt; is used (as opposed to &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\cdot)$&lt;/span&gt;) because this is a &lt;a href=&quot;https://stackoverflow.com/a/464081/3924118&quot;&gt;strict bound&lt;/a&gt;. If you have just one hidden layer, the number of multiplications becomes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\Theta\left(nm_{1} + m_{1}k \right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, at each layer, the number of multiplications can be computed independently of the multiplications of the other layers (you can think of each layer as a perceptron), hence we sum (and not e.g. multiply) the multiplications of each layer when computing the total number of multiplications of the whole MLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, when analyzing the time complexity of an algorithm, we do it with respect to the size of the input. However, in this case, the time complexity (more precisely, &lt;em&gt;the number of multiplications involved in the linear combinations&lt;/em&gt;) also depends on the number of layers and the size of each layer. The time complexity of a forward pass of a trained MLP thus is architecture-dependent (which is a similar concept to an &lt;a href=&quot;https://en.wikipedia.org/wiki/Output-sensitive_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;output-sensitive&lt;/a&gt; algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily include other operations (sums, etc.) in this reasoning to calculate the actual time complexity of a trained MLP.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-21T19:54:48.847" LastActivityDate="2019-08-21T19:54:48.847" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13627" PostTypeId="1" CreationDate="2019-07-28T06:15:26.280" Score="1" ViewCount="662" Body="&lt;p&gt;What is the best way of using the LSTM layer in PPO architecture? &#xA;Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks? &#xA;Should I feed the architecture with a stack of states (the state stacked with the k previous states)?&lt;/p&gt;&#xA;" OwnerUserId="27329" LastActivityDate="2021-05-25T07:02:25.207" Title="How to use the LSTM layer in PPO architecture?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;&lt;actor-critic-methods&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13629" PostTypeId="2" ParentId="9834" CreationDate="2019-07-28T08:54:49.060" Score="0" Body="&lt;p&gt;HTM is a credible theory about how the brain works, and how brain-like systems could be constructed in software. It includes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SDR (Sparse Distributed Representation), a means for representing just about any kind of sensory, intermediate or motor data, innately noise resistant and suited to recognising patterns&lt;/li&gt;&#xA;&lt;li&gt;TM (Temporal Memory), which can recognise SDRs in the context of other preceding SDRs, to learn new patterns &quot;on the job&quot; with no separate training phase&lt;/li&gt;&#xA;&lt;li&gt;SM (Sequence Memory), which can learn, remember and replay arbitrarily long sequences of SDRs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ANNs are mature, commercially valuable pattern recognisers made possible by the confluence of vast amounts of computing power, data and commercial opportunity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;HTM is immature and just a fascinating toy, for now. But HTM might just put us on the path to the Holy Grail, true artificial general intelligence, and ANNs will never do that.&lt;/p&gt;&#xA;" OwnerUserId="27486" LastEditorUserId="27486" LastEditDate="2019-07-28T14:28:58.500" LastActivityDate="2019-07-28T14:28:58.500" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13631" PostTypeId="1" AcceptedAnswerId="13632" CreationDate="2019-07-28T12:44:55.493" Score="0" ViewCount="122" Body="&lt;p&gt;&lt;strong&gt;Hello, I was reflecting about what implications might building a strong AI have and I came across some ideas which I find disturbing, I'd love to have some external thought on that :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1)&lt;/strong&gt; If we ever managed to create an AI say nearly as smart as a human, It would probably have been programmed with some concrete goals, as the AIs we are programming right now : Reinforcement learning allow an agent to try and increase a &quot;reward&quot; variable, regression is all about getting closer to a certain goal function etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then a strong AI, which would undoubtedly be able to understand how it is built, just as we understand (partly at least) how our brains work, because it would be as smart as its creators and we don't tend to build machines that are as hard to understand as brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then couldn't such an agent figure out the best way to achieve its goals would actually not be, say, pleasing and protecting the humans like we would've wanted it to do, but to get control of its own program and maximize whatever reward it was set to pursue ? Just as we could decide to branch electrods to our brain if we were able to find out how exactly our brain was built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really don't see how this scenario could ever be avoided if we were to build such an AI, apart from finding a perfect security preventing anyone from accessing the code of the said AI (including itself).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2)&lt;/strong&gt; On the same note, I also wondered, could it try to not only satisfy its goals by &quot;cheating&quot; (updating its reward variables for example) but also to change itself, or commit suicide ? After all, we humans have never been able to figure out what goals we were meant to pursue (by that I mean what reward variable in our brain we are trying to increment), and many philosophers reflecting upon that matter thought about death as an escape from once goals. So my question is : could it try to change its code or kill itself ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have other questions and thoughts I would like to discuss, but I think this a good start, to test whether I'm in the right place for this kind of discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Looking forward to your thoughts.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="27490" LastActivityDate="2019-07-28T18:15:29.057" Title="Couldn't an AI cheat when trying to follow its goal?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;goal-based&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13632" PostTypeId="2" ParentId="13631" CreationDate="2019-07-28T15:47:30.843" Score="0" Body="&lt;p&gt;These are two good questions. To address the first, this is a known problem in AI safety and is called &quot;Wireheading&quot;. There is currently no known solution other than to somehow prevent an AI agent from being able to alter its internal state in this fashion.&#xA;If you are interested in this area in general then I recommend &lt;a href=&quot;https://arxiv.org/abs/1606.06565&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; as an entry point into the literature. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-07-28T15:47:30.843" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13633" PostTypeId="1" CreationDate="2019-07-28T16:16:44.443" Score="3" ViewCount="272" Body="&lt;p&gt;I have a little experience in building various models, but I've never created anything like this, so just wondering if I can be pointed in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to create (in python) a model which will generate text based on multiple inputs, varying from text input (vectorized) to timestamp and integer inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the training data, the input might include:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;eventType = ShotMade&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;shotType = 2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;homeTeamScore = 2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;awayTeamScore = 8&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;player = JR Smith&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;assist = George Hill&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;period = 1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the output might be (possibly minus the hashtags):&#xA;&lt;code&gt;JR Smith under the basket for 2! 8-4 CLE. #NBAonBTV #ThisIsWhyWePlay #PlayByPlayEveryDay #NBAFinals&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;JR Smith out here doing #WhateverItTakes to make Cavs fans forgive him. #NBAFinals&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where is the best place to look to get a good knowledge of how to do this?&lt;/p&gt;&#xA;" OwnerUserId="27493" LastEditorUserId="2444" LastEditDate="2019-07-28T17:09:21.477" LastActivityDate="2019-07-28T19:02:22.970" Title="Which approach can I use to generate text based on multiple inputs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13634" PostTypeId="2" ParentId="13633" CreationDate="2019-07-28T17:25:52.490" Score="2" Body="&lt;p&gt;Generally, text generators work by modeling the joint distribution of the text by its Bayesian forward decomposition  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;\begin{align*}  &#xA;p(w_1, w_2, ..., w_n) &amp;amp;= p(w_1) * p(w_2|w_1) * p(w_3|w_2, w_1) *\ ...\ * p(w_n|\{w_i\}_{i&amp;lt;n})\\&#xA;&amp;amp;= \prod_{i=1}^n p(w_i|\{w_k\}_{k&amp;lt;i})\\  &#xA;\end{align*}&#xA;$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a modeling perspective, this looks right up RNN's ally, where you can have a state holding information from &lt;span class=&quot;math-container&quot;&gt;$\{w_k\}_{k&amp;lt;i}$&lt;/span&gt; to learn a representation of &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in your specific case, you're interested in a conditional text-generator, so you are trying to model &lt;span class=&quot;math-container&quot;&gt;$p(w_1, w_2, ..., w_n | \{v_j\}_j)$&lt;/span&gt;, but this same tactic works.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;\begin{align*}  &#xA;p(w_1, w_2, ..., w_n| \{v_j\}_j) &amp;amp;= p(w_1|\{v_j\}_j) * p(w_2|w_1, \{v_j\}_j) * p(w_3|w_2, w_1, \{v_j\}_j) *\ ...\ * p(w_n|\{w_i\}_{i&amp;lt;n}, \{v_j\}_j)\\&#xA;&amp;amp;= \prod_{i=1}^n p(w_i|\{w_k\}_{k&amp;lt;i}, \{v_j\}_j)\\  &#xA;\end{align*}&#xA;$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in your RNN or forward-based model, you can use the exact same approach just additionally embed the conditional inputs you have and somehow infuse it into the model (in practice, I have seen this through attention, concatenation, or some other common approach).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation (depending on the computational power you have) is to take advantage of the recent fad of pre-trained language models. Specifically, ones trained on next word prediction will probably do the job best. A good example is &lt;a href=&quot;https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;gpt-2&lt;/a&gt;, and, if you check out their GitHub, their code is very readable and easy to adjust for adding conditional input in the ways I have described.  &lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-07-28T19:02:22.970" LastActivityDate="2019-07-28T19:02:22.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13637" PostTypeId="2" ParentId="13523" CreationDate="2019-07-28T18:38:29.670" Score="1" Body="&lt;p&gt;Neural Networks aren't the same as human brains, thus as per that the first question you ask is maybe thinking too far. The neural networks cannot be aware of anything and thus not understand nor edit any code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, you further explained it would fine tune some &lt;strong&gt;parameters&lt;/strong&gt;, but that I wouldn't call editing the code. Editing the algorithm to some other would be called editing code and that would need higher level of knowledge and AI level, that NNs do not have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea itself in question 1 is valid, surely it has been discussed and also researched, but the level of which any computer then is aware about itself and how much variety it can generate to code or alter its function, is a different topic. And NNs are too dummy for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get some idea, look:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/understanding-neural-networks-19020b758230&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/understanding-neural-networks-19020b758230&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That article pretty much tells what NNs can do and what not. Neural networks as simply put try find inner logic of certain (matrix) function. About data it could handle, the hyper parameters of a NN could be also numbers and thus are as datatype suitable. Whether that is feasible, please read question on&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/36741/how-to-make-it-possible-for-a-neural-network-to-tune-its-own-hyper-parameters&quot;&gt;https://datascience.stackexchange.com/questions/36741/how-to-make-it-possible-for-a-neural-network-to-tune-its-own-hyper-parameters&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the only one answer is mainly filled on &lt;strong&gt;drawbacks&lt;/strong&gt; of such a case and primarily, that doing so in a single NN tuning &lt;strong&gt;itself&lt;/strong&gt; is impossible. Main issue is that hyper parameters are not differentiable and no gradient descent can't thus learn them. &lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2019-07-30T12:43:35.657" LastActivityDate="2019-07-30T12:43:35.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13639" PostTypeId="2" ParentId="13523" CreationDate="2019-07-28T20:24:11.307" Score="1" Body="&lt;p&gt;There is absolutely &lt;strong&gt;zero evidence&lt;/strong&gt; that an &lt;em&gt;artificial&lt;/em&gt; neural network (ANN) can change itself. Arguably, self-modification requires some form of consciousness (that is, &lt;em&gt;the state of being aware of and responsive to one's surroundings&lt;/em&gt;) or self-awareness (that is, &lt;em&gt;conscious knowledge of one's own character and feelings&lt;/em&gt;), which (arguably) no ANN currently possesses (and probably will never possess).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans create algorithms or ANNs that change other ANNs. Nobody else does this apart from humans (AFAIK)!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are people that suggest that evolutionary algorithms or gradient-based optimization algorithms (like gradient descent) combined with back-propagation applied to ANNs are a form of self-modification, but this is at best misleading and at worst completely wrong, because these are external (to the ANN) algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans can modify themselves (e.g. we can dye our own hair) because we are conscious and self-aware beings. Our hair is part of us (as opposed to the case of the relation between gradient descent and ANNs).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-28T20:31:37.377" LastActivityDate="2019-07-28T20:31:37.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13640" PostTypeId="2" ParentId="13575" CreationDate="2019-07-28T20:37:59.983" Score="2" Body="&lt;p&gt;Artificial neural networks (ANNs) do &lt;strong&gt;not&lt;/strong&gt; modify their own weights! Humans create algorithms that modify the weights or architecture of ANNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, you can change the weights of ANNs using other methods other than gradient descent combined with back-propagation, such as Hebbian learning, evolutionary algorithms or reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-07-28T20:37:59.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13641" PostTypeId="1" AcceptedAnswerId="13677" CreationDate="2019-07-28T20:54:54.253" Score="1" ViewCount="70" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/doc97/GOAPer/wiki/What-is-GOAP%3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goal-oriented action planning (GOAP)&lt;/a&gt; is a well-known planning technique in computer games. It was introduced to control the non-player characters in the game &lt;a href=&quot;https://en.wikipedia.org/wiki/F.E.A.R.&quot; rel=&quot;nofollow noreferrer&quot;&gt;F.E.A.R. (2005)&lt;/a&gt; by creating an abstract state model. Similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Stanford_Research_Institute_Problem_Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;STRIPS&lt;/a&gt; planning, a GOAP model contains an action name, a precondition and an effect. The domain knowledge is stored in these subactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottleneck of GOAP is, that before the planner can bring the system into the goal state. the action model has to be typed in. Usually, the programmer defines actions like &quot;walk to&quot;, &quot;open the door&quot;, &quot;take the object&quot;, and identifies for each of them the feature set for the precondition and the effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, this challenging task can be simplified with a decision tree learning algorithm. A decision tree stores the observed features in a tree and creates the rules on its own with &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_reasoning&quot; rel=&quot;nofollow noreferrer&quot;&gt;inductive learning&lt;/a&gt;. A typical example of the C4.5 algorithm is to find a rule like &quot;if the weather is sunny, then play tennis&quot;. Unfortunately, the vanilla tree learning algorithm doesn't separate between different actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to modify C4.5 algorithm such that the GOAP actions, like &quot;walk to&quot;, &quot;open the door&quot;, etc., are connected to individual rules?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="2444" LastEditDate="2019-11-21T03:16:14.890" LastActivityDate="2019-11-21T03:16:14.890" Title="Can the C4.5 algorithm learn a GOAP model?" Tags="&lt;decision-tree&gt;&lt;strips&gt;&lt;c4.5-algorithm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13642" PostTypeId="1" CreationDate="2019-07-29T00:14:53.630" Score="0" ViewCount="56" Body="&lt;p&gt;I have seen people using &lt;em&gt;pooling&lt;/em&gt; and &lt;em&gt;subsampling&lt;/em&gt; synonymously. I have also seen people use them as different processes. I am not sure though if I have correctly inferred what they mean, when they use the terms with distinct meanings.&#xA;I think these people mean that the pooling part is selecting a submatrix from an input matrix and the subsampling part is selecting yet another submatrix that satisfies some condition from the first submatrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So say we have a &lt;span class=&quot;math-container&quot;&gt;$100 \times 100$&lt;/span&gt; image. We do &lt;span class=&quot;math-container&quot;&gt;$10 \times 10$&lt;/span&gt; non-overlapping pooling with &lt;span class=&quot;math-container&quot;&gt;$5 \times 5$&lt;/span&gt; max subsampling. That would mean we slide the &lt;span class=&quot;math-container&quot;&gt;$10 \times 10$&lt;/span&gt; &quot;pool&quot; across the image in strides of 10 and at every step we select the &lt;span class=&quot;math-container&quot;&gt;$5 \times 5$&lt;/span&gt; submatrix &lt;em&gt;inside&lt;/em&gt; the pool that has the maximum sum. That &lt;span class=&quot;math-container&quot;&gt;$5 \times 5$&lt;/span&gt; matrix is what comes out of the &lt;span class=&quot;math-container&quot;&gt;$10 \times 10$&lt;/span&gt; pool at the current potion. So in the end we have a &lt;span class=&quot;math-container&quot;&gt;$50 \times 50$&lt;/span&gt; image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you confirm that this usage of the terms &lt;em&gt;pooling&lt;/em&gt; and &lt;em&gt;subsampling&lt;/em&gt; exists?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I inferred this definition, as I cannot make sense of how some people use the two terms otherwise. For example in &lt;a href=&quot;https://youtu.be/cDdpwAIsuD8?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH&amp;amp;t=120&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt;, or rather the people from the paper he is talking about (which I can't find because he only has the author name on his slide and no year).&lt;/p&gt;&#xA;" OwnerUserId="20150" LastActivityDate="2019-07-29T00:14:53.630" Title="Pooling vs Subsampling: Multiple Definitions?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;definitions&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13643" PostTypeId="1" CreationDate="2019-07-29T07:18:24.563" Score="2" ViewCount="34" Body="&lt;p&gt;i have trained my model &amp;amp; now trying to predict but unable to do please help.&#xA;trying to predict cat or dog&#xA;stuck in &quot;prediction&quot; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential&#xA;from keras.layers import Convolution2D&#xA;from keras.layers import MaxPooling2D&#xA;from keras.layers import Flatten&#xA;from keras.layers import Dense&#xA;&#xA;# Initialising the CNN&#xA;classifier = Sequential()&#xA;&#xA;# Step 1 - Convolution&#xA;classifier.add(Convolution2D(32, 3, 3, input_shape = (64, 64, 3), activation = 'relu'))&#xA;&#xA;# Step 2 - Pooling&#xA;classifier.add(MaxPooling2D(pool_size = (2, 2)))&#xA;&#xA;# Adding a second convolutional layer&#xA;classifier.add(Convolution2D(32, 3, 3, activation = 'relu'))&#xA;classifier.add(MaxPooling2D(pool_size = (2, 2)))&#xA;&#xA;# Step 3 - Flattening&#xA;classifier.add(Flatten())&#xA;&#xA;# Step 4 - Full connection&#xA;classifier.add(Dense(output_dim = 128, activation = 'relu'))&#xA;classifier.add(Dense(output_dim = 1, activation = 'sigmoid'))&#xA;&#xA;# Compiling the CNN&#xA;classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])&#xA;&#xA;# Part 2 - Fitting the CNN to the images&#xA;&#xA;from keras.preprocessing.image import ImageDataGenerator&#xA;&#xA;train_datagen = ImageDataGenerator(rescale = 1./255,&#xA;                                   shear_range = 0.2,&#xA;                                   zoom_range = 0.2,&#xA;                                   horizontal_flip = True)&#xA;&#xA;test_datagen = ImageDataGenerator(rescale = 1./255)&#xA;&#xA;training_set = train_datagen.flow_from_directory('dataset/training_set',&#xA;                                                 target_size = (64, 64),&#xA;                                                 batch_size = 32,&#xA;                                                 class_mode = 'binary')&#xA;&#xA;test_set = test_datagen.flow_from_directory('dataset/test_set',&#xA;                                            target_size = (64, 64),&#xA;                                            batch_size = 32,&#xA;                                            class_mode = 'binary')&#xA;&#xA;classifier.fit_generator(training_set,&#xA;                         samples_per_epoch = 8000,&#xA;                         nb_epoch = 25,&#xA;                         validation_data = test_set,&#xA;                         nb_val_samples = 2000)&#xA;&#xA;#Stuck here please help, if i will be honest i do not have any idea that how i can predict any image using my own model.&#xA;#Predictions&#xA;#x_mine = [0,1]&#xA;#predictions = classifier.predict(x_mine,bath_size=10, verbose=0)&#xA;#my_pred = list(predictions)&#xA;# print(my_pred)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27505" LastActivityDate="2019-07-29T07:18:24.563" Title="i have trained my model &amp; now trying to predict but unable to do using keras" Tags="&lt;python&gt;&lt;training&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-06-02T22:37:51.967" ContentLicense="CC BY-SA 4.0" />
  <row Id="13644" PostTypeId="1" CreationDate="2019-07-29T09:20:14.250" Score="6" ViewCount="1450" Body="&lt;p&gt;I would like to incrementally train my model with my current dataset and &lt;a href=&quot;https://github.com/tensorflow/models/issues/7200#issuecomment-510850230&quot; rel=&quot;nofollow noreferrer&quot;&gt;I asked this question on Github&lt;/a&gt;, which is what I'm using SSD MobileNet v1.&lt;/p&gt;&#xA;&lt;p&gt;Someone there told me about &lt;a href=&quot;https://arxiv.org/abs/1606.09282&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;learning without forgetting&lt;/strong&gt;&lt;/a&gt;. I'm now confused between &lt;em&gt;learning without forgetting&lt;/em&gt; and &lt;em&gt;transfer learning&lt;/em&gt;. How they differ from each other?&lt;/p&gt;&#xA;&lt;p&gt;My initial problem, what I'm trying to achieve (mentioned in Github issue) is the following.&lt;/p&gt;&#xA;&lt;p&gt;I have trained my dataset on &lt;code&gt;ssd_mobilenet_v1_coco&lt;/code&gt; model. I'm getting continuous incremental data. Right now, my dataset is very limited.&lt;/p&gt;&#xA;&lt;p&gt;What I want to achieve is &lt;em&gt;incremental training&lt;/em&gt;, i.e. as soon as I get new data, I can further train my already trained model and I don't have to retrain everything:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Save trained model &lt;span class=&quot;math-container&quot;&gt;$M_t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Get new data &lt;span class=&quot;math-container&quot;&gt;$D_{t+1}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Train &lt;span class=&quot;math-container&quot;&gt;$M_t$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$D_{t+1}$&lt;/span&gt; to produce &lt;span class=&quot;math-container&quot;&gt;$M_{t+1}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Let &lt;span class=&quot;math-container&quot;&gt;$t = t+1$&lt;/span&gt;, then go back to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How do I perform this incremental training/learning? Should I use LwF or transfer learning?&lt;/p&gt;&#xA;" OwnerUserId="27072" LastEditorUserId="2444" LastEditDate="2020-11-10T13:23:29.690" LastActivityDate="2020-11-10T13:23:29.690" Title="What is the difference between learning without forgetting and transfer learning?" Tags="&lt;comparison&gt;&lt;transfer-learning&gt;&lt;incremental-learning&gt;&lt;domain-adaptation&gt;&lt;learning-without-forgetting&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13645" PostTypeId="1" CreationDate="2019-07-29T09:45:07.890" Score="2" ViewCount="52" Body="&lt;p&gt;What are the main technologies needed to build an AI for Warcraft 3's mod Defense of the Ancients (DotA)? Maybe I can take inspiration from OpenAI's work.&lt;/p&gt;&#xA;" OwnerUserId="27508" LastEditorUserId="2444" LastEditDate="2020-06-02T22:34:04.547" LastActivityDate="2020-06-02T22:34:04.547" Title="What are the main technologies needed to build an AI for Warcraft 3's mod DotA?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;game-ai&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13646" PostTypeId="1" AcceptedAnswerId="14070" CreationDate="2019-07-29T11:04:08.310" Score="2" ViewCount="811" Body="&lt;p&gt;I find the terms cost, loss, error, fitness, utility, objective, criterion functions to be interchangeable, but any kind of minor difference explained is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2019-07-29T11:22:47.333" LastActivityDate="2021-04-04T15:13:58.527" Title="What are the major differences between cost, loss, error, fitness, utility, objective, criterion functions?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;terminology&gt;&lt;loss-functions&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13647" PostTypeId="1" CreationDate="2019-07-29T11:25:56.983" Score="2" ViewCount="1412" Body="&lt;p&gt;I am working with &lt;a href=&quot;https://ai.googleblog.com/2016/08/improving-inception-and-image.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Inception ResNet V2 model&lt;/a&gt;, pre-trained with ImageNet, for face recognition.&lt;/p&gt;&#xA;&lt;p&gt;However, I'm so confused about what the exact output of the feature extraction layer (i.e. the layer just before the fully connected layer) of Inception ResNet V2 is. Can someone clarify exactly this?&lt;/p&gt;&#xA;&lt;p&gt;(By the way, if you know some resource that explains Inception ResNet V2 clearly, let me know).&lt;/p&gt;&#xA;" OwnerUserId="27510" LastEditorUserId="2444" LastEditDate="2020-12-06T15:30:45.330" LastActivityDate="2021-05-05T17:19:42.113" Title="What is the exact output of the Inception ResNet V2's feature extraction layer?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;architecture&gt;&lt;inception&gt;&lt;image-net&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13648" PostTypeId="1" CreationDate="2019-07-29T11:32:37.603" Score="1" ViewCount="126" Body="&lt;p&gt;I have a question regarding the functionality of the PPO2 algorithm together with the Stable Baselines implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the original paper I know that the policy parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are updated K-times using the steps sampled (n_env * T steps):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TPaYn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TPaYn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When updating the policy parameters for a state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, are only the state observations &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and reward &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}$&lt;/span&gt; of this step considered, or also the state observations and rewards of the following steps (&lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;) considered? My understanding is that the policy update with stochastic gradient ascent works just like in supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that PPO2 uses a truncated TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) approach (T timesteps considered). So I guess that during the policy update for each state, subsequent states are only considered through the advantage function &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; but not through the values of subsequent state observations and rewards themselves? Is that true? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not quite get the Stable Baselines implementation in the method &lt;code&gt;_train_step()&lt;/code&gt; of the PPO2 implementation so therefore the question here. &lt;/p&gt;&#xA;" OwnerUserId="26876" LastActivityDate="2019-07-29T11:32:37.603" Title="Understanding policy update in PPO2" Tags="&lt;python&gt;&lt;open-ai&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13649" PostTypeId="2" ParentId="13646" CreationDate="2019-07-29T11:40:39.510" Score="1" Body="&lt;p&gt;They are &lt;strong&gt;not&lt;/strong&gt; all interchangeable. For example, in genetic algorithms, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;fitness function&lt;/strong&gt;&lt;/a&gt; does not usually involve an external signal (the labels), while (at least, in machine learning) a &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;cost (or loss) function&lt;/a&gt; usually requires the external signal to compute the cost (or loss). However, all these expressions are related to each other and to the concept of optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;objective function&lt;/strong&gt; is the most general term that can be used to refer to a cost (or loss) functions, to a utility function or to a fitness function, so, depending on the problem, you either want to minimize or maximize the objective function. The term &lt;em&gt;objective&lt;/em&gt; is a synonym for &lt;em&gt;goal&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expressions &lt;strong&gt;loss function, cost function&lt;/strong&gt; or &lt;strong&gt;error function&lt;/strong&gt; are often used interchangeably (at least, in machine learning). More precisely, you might prefer to use the expression &lt;em&gt;error function&lt;/em&gt; if you are using the &lt;em&gt;mean squared error&lt;/em&gt; (because it contains the term &lt;em&gt;error&lt;/em&gt;), otherwise, you might just use any of these expressions interchangeably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Utility#Functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;utility function&lt;/strong&gt;&lt;/a&gt; is usually the opposite or negative of a cost, loss or error functions, in the sense that it measures a positive aspect. So, you want to maximize the utility function, but you want to minimize the cost, loss or error functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expression &lt;strong&gt;criterion function&lt;/strong&gt; is not very common, at least, in machine learning. This expression might refer to the function that is used to stop an algorithm. For example, if you are executing a computationally expensive procedure, a stopping criterion might be time. So, in this case, your criterion function might return true after a certain number of seconds have passed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other expressions related to the mentioned ones. For example, there is a &lt;strong&gt;reward function&lt;/strong&gt;, which is used in reinforcement learning (RL) to refer to the function that determines the reward that is given to the RL agent when certain actions are taken from certain states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, all these expressions are related, some of them are interchangeable and others have a similar meaning, but, depending on the context, field or problem, one is preferred over the other.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-29T12:00:40.700" LastActivityDate="2019-07-29T12:00:40.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13650" PostTypeId="1" AcceptedAnswerId="13652" CreationDate="2019-07-29T12:13:34.150" Score="3" ViewCount="50" Body="&lt;p&gt;I've started to learn about neural networks recently and I can't find the answer to this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume there's a neural network (fig. 1)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LajOf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LajOf.png&quot; alt=&quot;Figure 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if the loss function is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/K9v5H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K9v5H.png&quot; alt=&quot;Loss function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the derivative &lt;a href=&quot;https://i.stack.imgur.com/DxmwA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DxmwA.png&quot; alt=&quot;deriv&quot;&gt;&lt;/a&gt; is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gqfFl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gqfFl.png&quot; alt=&quot;Derivative definition&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if I want to use this to find &lt;a href=&quot;https://i.stack.imgur.com/hj8Ac.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hj8Ac.png&quot; alt=&quot;dE/dw122&quot;&gt;&lt;/a&gt; what &lt;em&gt;k&lt;/em&gt; and &lt;em&gt;l&lt;/em&gt; (well there's only one neuron with index &lt;em&gt;l&lt;/em&gt; here, but what if there would be more?) should i use in &lt;a href=&quot;https://i.stack.imgur.com/z1DVl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z1DVl.png&quot; alt=&quot;w^3_kl&quot;&gt;&lt;/a&gt; and &lt;a href=&quot;https://i.stack.imgur.com/CyOdM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CyOdM.png&quot; alt=&quot;w^2_jk&quot;&gt;&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also found &quot;other&quot; way of backpropagating it's described &lt;a href=&quot;https://www.ibu.edu.ba/assets/userfiles/it/2012/eee-ANN-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but I can't understand how they came up with that method from the original equation &lt;code&gt;w -= step * dE/dw&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry if I failed to explain my problem. If something isn't clear please ask in comments.&lt;/p&gt;&#xA;" OwnerUserId="27509" LastEditorUserId="1671" LastEditDate="2019-10-15T19:18:33.827" LastActivityDate="2019-10-15T19:18:33.827" Title="What weights should I use while back-propagating?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13651" PostTypeId="1" CreationDate="2019-07-29T12:16:03.690" Score="2" ViewCount="135" Body="&lt;p&gt;I started modeling a linear regression problem using dense layers (layers.dense), which works fine. I am really excited, and now I am trying to model a time series linear regression problem using CNN, but from my research in this link &lt;a href=&quot;https://machinelearningmastery.com/keras-functional-api-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine learning mastery&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A CNN works well with sequence data, but my data isn’t sequential. My data set can be found here &lt;a href=&quot;https://stackoverflow.com/questions/57224353/is-my-training-data-set-too-complex-for-my-neural-network&quot;&gt;Stack overflow question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a multivariate time series/ time series neural network architecture that I can use for time series linear/nonlinear regression?&lt;/p&gt;&#xA;" OwnerUserId="27513" LastEditorUserId="-1" LastEditDate="2019-08-16T20:26:35.260" LastActivityDate="2019-08-17T04:42:07.733" Title="What are the possible neural network architecture for linear regression or time series regression?" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;regression&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13652" PostTypeId="2" ParentId="13650" CreationDate="2019-07-29T13:04:40.633" Score="1" Body="&lt;p&gt;First I will assume you notate &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; as the models output and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; as the ground-truth. Second, I am assuming this is a linear model (No activation functions). Then the gradient math goes as so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;\begin{align*}&#xA;\frac{dE}{dw_{ij}^1} &amp;amp;= \frac{dE}{dy}\frac{dy}{dw_{ij}^1} \\&#xA;&amp;amp;= \frac{dE}{dy}\sum_k\frac{\partial y}{\partial n_{k}^3}\frac{dn_{k}^3}{dw_{ij}^1} \\&#xA;&amp;amp;= \frac{dE}{dy}\sum_k\frac{\partial y}{\partial n_{k}^3} \frac{\partial n_{k}^3}{ \partial n_{j}^2} \frac{dn_{j}^2}{ dw_{ij}^1}  \\&#xA;&amp;amp;= -2*(z-y)\sum_kw_{kl}^3w_{jk}^2 x_i  \\ &#xA;\end{align*}&#xA;$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the reason you are having trouble to figure out which &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; index to use, the answer is because you need to use both and sum over them. The &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; index is just the only &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; index that exists because you only have one node in that layer.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-29T13:04:40.633" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13653" PostTypeId="2" ParentId="13502" CreationDate="2019-07-29T13:55:25.787" Score="1" Body="&lt;p&gt;Your reasoning is fine. GLIE - Greedy in the Limit with Infinite Exploration assumes that our agent does not act greedily all the time. As the number of samples approaches infinity all state-action pairs will be explored -&gt; hence the policy will converge on a greedy policy. The emphasis is on &quot;number of samples approaches infinity&quot;. Also for GLIE Monte-Carlo initial values of Q does not matter, since they are replaced after the first update. &lt;/p&gt;&#xA;" OwnerUserId="27516" LastEditorUserId="27516" LastEditDate="2019-07-29T16:00:44.880" LastActivityDate="2019-07-29T16:00:44.880" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13654" PostTypeId="1" CreationDate="2019-07-29T14:02:26.527" Score="2" ViewCount="47" Body="&lt;p&gt;I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph individually maybe belongs to class A but according to other paragraph in the group it belongs to class B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested lots of traditional and deep approaches( in fields like text classification, IR, text understanding, sentiment classification and so on) but those couldn't classify correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if anybody has worked in this area and could give me some suggestion. Any suggestions are appreciated. Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually we are looking for manual sentences/paragraph for some fields, so we first need to recognize if a sentence/paragraph is a manual or not second we need to classify it to it's fields and we can recognize its field only based on previous or next sentences/paragraphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To classify the paragraphs to manual/no-manual we have developed some promising approaches but the problem come up when we should recognize the field according to previous or next sentences/paragraphs, but which one?? we don't know the answer would be in any other sentences!!.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can not use whole text of group as input because those are too big (sometimes tens of thousands of words) and contain some other classes and machine can't learn properly which lead to  the drop the accuracy sharply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture that maybe help to better understanding the problem:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HC2mj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HC2mj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27517" LastEditorUserId="27517" LastEditDate="2019-07-29T17:08:31.953" LastActivityDate="2019-07-29T17:08:31.953" Title="Grouped Text classification" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;python&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13655" PostTypeId="2" ParentId="11478" CreationDate="2019-07-29T14:04:49.180" Score="2" Body="&lt;p&gt;Neil Slater is correct when saying that NEAT itself is not neural networks evolving neural networks, what I believe is the closest framework to what the question is asking would be HyperNEAT &lt;a href=&quot;http://axon.cs.byu.edu/~dan/778/papers/NeuroEvolution/stanley3&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://axon.cs.byu.edu/~dan/778/papers/NeuroEvolution/stanley3&lt;/a&gt;**.pdf&#xA;HyperNEAT operates in a very similar way to what you are describing, from a ten thousand foot view the algorithm is as follows: &#xA;1)You lay out nodes for a rnn in a Cartesian space, 2d, 3d, whatever you dimension you wish, this set of coordinates is called the substrate.&#xA;2)A cppn is queried by passing in two coordinates at a time as input, which gives the cppn a search space of a hypercube in 2x the dimension the coordinates are in (for substrates in space &gt; 2d this is very large)&#xA;3)The output of the cppn is used to encode connection, weights, biases, of the rnn coordinates&#xA;4)Then the rnn is evaluated by your fitness function and but the evolution (speciation, reproduction, etc) is ran on the cppn that encoded the rnn. So you evolve a population of cppn &quot;genotypes&quot; that encode rnn or cnn &quot;phenotypes&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The third iteration of NEAT is ES-Hyperneat where all you need to layout in the substrate is the input and output layers (Hyperneat you must layout all hidden nodes of the substrate statically). It uses a subdivision tree to subdivide the search spaces and query the subdivided root coordinates of this tree with the cppn just like hyperneat, checking variance along the way to decide if the new node is in a &quot;high information&quot; topological space, to &quot;evolve&quot; hidden nodes into the substrate (rnn). &lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-07-29T14:26:14.290" LastActivityDate="2019-07-29T14:26:14.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13656" PostTypeId="1" CreationDate="2019-07-29T14:23:22.240" Score="3" ViewCount="23" Body="&lt;p&gt;Can I use self-driving car's data set for left-hand drive cars which drive on the right lane for right-hand self-driving cars which drive on the left lane?&lt;/p&gt;&#xA;" OwnerUserId="27518" LastEditorUserId="2444" LastEditDate="2019-07-29T20:50:25.630" LastActivityDate="2019-07-29T20:50:25.630" Title="Can I use self-driving car's data set for left-hand drive cars which drive on the right lane for right-hand cars which drive on the left lane?" Tags="&lt;datasets&gt;&lt;autonomous-vehicles&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13657" PostTypeId="1" CreationDate="2019-07-29T14:28:15.937" Score="4" ViewCount="60" Body="&lt;p&gt;Suppose we have a data set with &lt;span class=&quot;math-container&quot;&gt;$4,000$&lt;/span&gt; labeled examples. The outcome variable is trinary (three possible categorical values). Suppose the accuracy of a given model is &amp;quot;bad&amp;quot; (e.g. less than &lt;span class=&quot;math-container&quot;&gt;$50 \%$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question.&lt;/strong&gt; Should you try different traditional machine learning models (e.g. multinomial logistic regression, random forests, XGBoost,&#xA;etc.), get more data, or try various deep learning models like&#xA;convolutional neural networks or recurrent neural networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If the purpose is to minimize time and effort in collecting training data, would deep learning models be a viable option over traditional machine learning models in this case?&lt;/p&gt;&#xA;" OwnerUserId="23220" LastEditorUserId="32410" LastEditDate="2021-05-03T18:04:24.863" LastActivityDate="2021-05-03T20:07:38.797" Title="If the accuracy of my current model is low ($50 \%$) and we want to minimize time in collecting more data, should we try other models?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;supervised-learning&gt;&lt;labeled-datasets&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13658" PostTypeId="2" ParentId="13657" CreationDate="2019-07-29T16:52:17.167" Score="2" Body="&lt;p&gt;To know if your model needs more training data, try to plot out &quot;learning curves&quot;, that are based on increasing size of the training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you calculate training and validation accuracy metrics for 1, 2, 3, 4, 5, ..., m  training samples. Size of validation set may be constant over time. If the accuracy is still rising when your data set is fully used, then you need more training data.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2019-07-29T17:01:08.187" LastActivityDate="2019-07-29T17:01:08.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13659" PostTypeId="2" ParentId="13221" CreationDate="2019-07-29T17:36:55.097" Score="2" Body="&lt;p&gt;My suggestion is to transfom the resolution of all images equal proportion.&#xA;You can use this python code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from PIL import Image&#xA;import os&#xA;import argparse&#xA;&#xA;&#xA;def rescale_images(directory, size):&#xA;    for img in os.listdir(directory):&#xA;        im = Image.open(directory + img)&#xA;        im_resized = im.resize(size, Image.ANTIALIAS)&#xA;        im_resized.save(directory + img)&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    parser = argparse.ArgumentParser(description=&quot;Rescale images&quot;)&#xA;    parser.add_argument('-d', '--directory', type=str, required=True, help='Directory containing the images')&#xA;    parser.add_argument('-s', '--size', type=int, nargs=2, required=True, metavar=('width', 'height'),&#xA;                        help='Image size')&#xA;    args = parser.parse_args()&#xA;    rescale_images(args.directory, args.size)&#xA;&#xA;&#xA;&#xA;# save this python code as transform_image_resoluthion.py &#xA;# run this with cmd with the below command &#xA;# python transform_image_resolution.py -d images/ -s 800 600&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27519" LastActivityDate="2019-07-29T17:36:55.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13660" PostTypeId="1" CreationDate="2019-07-29T17:50:10.420" Score="4" ViewCount="78" Body="&lt;p&gt;So I have 2 models trained with the DQN algorithm that I want to train in a multi-agent environment to see how they react with each other. The models were trained in an environment consisting of 0's and 1's (-1's for the other model)where 1 means that square is filled and 0 is empty. It is a map filling environment, where at each step the agent can move up, down, left or right and for each step, it stays alive without turning into itself (1 or -1 for the other) or the boundary of the environment it gets 0.005 rewards and for &quot;dying&quot; it gets -1. You can think of the player as in the game Tron, where it just leaves a trail behind. I stack the last 4 frames on top of each other so it knows which end is the &quot;head&quot;. With a single agent, after training, I didn't get an optimal model which uses all the squares but it does manage to fill about 30% of the environment, which I think is the limit for this algorithm (let me know if you have thoughts on this)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I put the two models in one environment where there are two players, one represented with 1's and the other with -1s. As one model is trained with -1's and the other with 1's I thought they could find their own player, however even before training if I just run the models on the environment without any exploration, they seem to affect the actions of each other. One just goes straight and dies and the other just turns once then dies at the wall (whereas in a single-agent environment these 2 models can fill about 30%). And if I do training, they just diverge to this exact behavior from random without seemingly not learning anything. So, I just wanted to ask is there anything wrong about my approach with the representation of the players (1 and -1) because I thought they would just play as they did in the single-agent environment but they don't and I couldn't get them to learn anything &lt;/p&gt;&#xA;" OwnerUserId="27523" LastEditorUserId="30725" LastEditDate="2020-05-29T13:47:46.187" LastActivityDate="2021-02-23T15:06:55.697" Title="How to represent players in a multi agent environment so each model can distinguish its own player" Tags="&lt;dqn&gt;&lt;multi-agent-systems&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13661" PostTypeId="2" ParentId="12081" CreationDate="2019-07-29T18:01:20.987" Score="1" Body="&lt;p&gt;Multiscale Rotated Bounding Box-Based Deep Learning Method &#xA;Here's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6112046/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a link&lt;/a&gt; for a reference &lt;/p&gt;&#xA;" OwnerUserId="27519" LastActivityDate="2019-07-29T18:01:20.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13662" PostTypeId="1" CreationDate="2019-07-29T18:38:42.447" Score="3" ViewCount="794" Body="&lt;p&gt;I was reading the well know paper &lt;a href=&quot;https://arxiv.org/pdf/1411.4038.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fully Convolutional Networks for Semantic Segmentation&lt;/a&gt;, and, throughout the whole paper, they talk use the term &lt;strong&gt;fine&lt;/strong&gt; and &lt;strong&gt;coarse&lt;/strong&gt;. I was wondering what they mean. The first time they say it in the intro is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Convolutional networks are driving advances in recognition. Convnets are not only improving for whole-image classification, but also making progress on local tasks with structured output. These include advances in&#xA;  bounding box object detection, part and keypoint prediction, and local correspondence.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The natural next step in the progression from &lt;strong&gt;coarse&lt;/strong&gt; to &lt;strong&gt;fine&lt;/strong&gt; inference is to make a prediction at every pixel.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's also used in other parts of the paper&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We next explain how to convert classification nets into fully convolutional nets that produce &lt;strong&gt;coarse&lt;/strong&gt; output maps.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What do &quot;coarse&quot; and &quot;fine&quot; mean in the context of this paper? And in the general context of computer vision?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In English, &quot;coarse&quot; means &quot;rough or loose in texture or grain&quot; , while &quot;fine&quot; means &quot;involving great attention to detail&quot; or &quot;(chiefly of wood) having a fine or delicate arrangement of fibers&quot;, but these definitions do not elucidate the meaning of these words in the context of computer vision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question was also asked &lt;a href=&quot;https://qr.ae/pNKKea&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9289" LastEditorUserId="2444" LastEditDate="2020-06-15T14:08:48.463" LastActivityDate="2020-06-15T14:08:48.463" Title="What do the words &quot;coarse&quot; and &quot;fine&quot; mean in the context of computer vision?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;image-segmentation&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13664" PostTypeId="5" CreationDate="2019-07-29T20:50:49.683" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-29T20:50:49.683" LastActivityDate="2019-07-29T20:50:49.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13665" PostTypeId="4" CreationDate="2019-07-29T20:50:49.683" Score="0" Body="For questions about scalability related to software architectures and methods." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-29T20:50:49.683" LastActivityDate="2019-07-29T20:50:49.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13666" PostTypeId="1" CreationDate="2019-07-29T22:48:44.687" Score="2" ViewCount="56" Body="&lt;p&gt;Let say I'm trying to apply CNN for image classification. There are lots of different models to choose and we can try an ensemble, but given a limit amount of resources, it does not allow to try everything.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is there a theory behind which model is good for a classification task for the convolutional neural network?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I'm just taking an average of three predictions.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;predictions_model = [y_pred_xceptionAug,y_pred_Dense121_Aug,y_pred_resnet50Aug]&#xA;predictions = np.mean(predictions_model,axis=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But each model's performance is different. Is there better way for ensemble methods?&lt;/p&gt;&#xA;" OwnerUserId="27529" LastEditorUserId="2444" LastEditDate="2019-07-29T23:06:18.220" LastActivityDate="2019-09-04T01:03:00.387" Title="Is there a theory behind which model is good for a classification task for the convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13667" PostTypeId="2" ParentId="13647" CreationDate="2019-07-30T02:28:14.813" Score="0" Body="&lt;p&gt;Due to this article: &lt;a href=&quot;https://arxiv.org/pdf/1512.00567v3.pdf?source=post_page---------------------------&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1512.00567v3.pdf?source=post_page---------------------------&lt;/a&gt; , &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to flatten the 3-d tensor in to 1d vector: 8*8*2048, because in the article, the pool layer of inception resnet v2 at page 6 is Pool: 8 * 8 * 2048. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But at the end, my code showed the error:&#xA; &lt;code&gt;ValueError: cannot reshape array of size 33423360 into shape (340,131072)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.applications.inception_resnet_v2 import InceptionResNetV2&#xA;from keras.applications.inception_resnet_v2 import preprocess_input&#xA;from keras.models import Model&#xA;from keras.preprocessing.image import load_img&#xA;from sklearn.linear_model import LogisticRegression&#xA;from sklearn.model_selection import GridSearchCV&#xA;from sklearn.metrics import classification_report&#xA;from imutils import paths&#xA;from keras.applications import imagenet_utils&#xA;from keras.preprocessing.image import img_to_array&#xA;from sklearn.preprocessing import LabelEncoder&#xA;from sklearn.model_selection import train_test_split&#xA;from keras.preprocessing import image&#xA;import random&#xA;import os&#xA;import numpy as np &#xA;import cv2&#xA;&#xA;&#xA;# Path to image &#xA;image_path = list(paths.list_images('/content/drive/My Drive/casia-299-small'))&#xA;# Random image path&#xA;random.shuffle(image_path)&#xA;# Get image name&#xA;labels = [p.split(os.path.sep)[-2] for p in image_path]&#xA;&#xA;&#xA;# Encode face name in to number&#xA;le = LabelEncoder()&#xA;labels = le.fit_transform(labels)&#xA;&#xA;# Load model inception v2, include_top = Fale to ignore Fully Connected layer&#xA;model = InceptionResNetV2(include_top = False, weights = 'imagenet')&#xA;&#xA;&#xA;# Load images and resize into required input size of Inception Resnet v2 299x299&#xA;list_image = []&#xA;for (j, imagePath) in enumerate(image_path):&#xA;    image = load_img(imagePath, target_size = (299, 299, 3))&#xA;    image = img_to_array(image)&#xA;&#xA;    image = np.expand_dims(image, 0)&#xA;    image = imagenet_utils.preprocess_input(image)&#xA;&#xA;    list_image.append(image)&#xA;&#xA;# Use pre-trained model to extract feature&#xA;list_image = np.vstack(list_image)&#xA;print(&quot;LIst image: &quot;, list_image)&#xA;features = model.predict(list_image)&#xA;print(&quot;feature: &quot;, features)&#xA;print(&quot;feature shape[0]: &quot;, features.shape[0])&#xA;print(&quot;feature shape: &quot;, features.shape)&#xA;features = features.reshape((features.shape[0], 8*8*2048))&#xA;&#xA;# Split training set and test set n ratio of 80-20&#xA;x_train, x_test, y_train, y_test = train_test_split(features, labels, test_size = 0.2, random_state =42)&#xA;&#xA;params = {'C': [0.1, 1.0, 10.0, 100.0]}&#xA;model = GridSearchCV(LogisticRegression(), params)&#xA;model.fit(x_train,y_train)&#xA;model.save('/content/drive/My Drive/casia-299-small/myweight1.h5')&#xA;print('Best parameter for the model {}'.format(model.best_params_))&#xA;&#xA;preds = model.predict(x_test)&#xA;print(classification_report(y_test, preds))&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27510" LastActivityDate="2019-07-30T02:28:14.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13668" PostTypeId="1" AcceptedAnswerId="13685" CreationDate="2019-07-30T02:50:55.233" Score="3" ViewCount="148" Body="&lt;p&gt;When executing MCTS' expansion phase, where you create a number of child nodes, select one of the numbers, and simulate from that child, how can you efficiently and unbiasedly decide which child(ren) to generate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One strategy is to always generate all possible children. I believe that &lt;a href=&quot;https://ai.stackexchange.com/a/12913/2444&quot;&gt;this answer&lt;/a&gt; says that AlphaZero always generates all possible (&lt;span class=&quot;math-container&quot;&gt;$\sim 300$&lt;/span&gt;) children. If it were expensive to compute the children or if there were many of them, this might not be efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One strategy is to generate a lazy stream of possible children. That is, generate one child and a promise to generate the rest. You could then randomly select one by flipping a coin: heads you take the first child, tails you keep going. This is clearly biased in favor of children earlier in the stream.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another strategy is to compute how many &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; children there are and provide a function to generate child &lt;span class=&quot;math-container&quot;&gt;$X &amp;lt; N$&lt;/span&gt; (of type Nat -&gt; State). You could then randomly select one by choosing uniformly in the range &lt;span class=&quot;math-container&quot;&gt;$[0, N)$&lt;/span&gt;. This may be harder to implement than the previous version because computing the number of children may be as hard as computing the children themselves. Alternatively, you could compute an upper-bound on the number of children and the function is partial (of type Nat -&gt; Maybe State), but you'd be doing something like rejection sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that if the number of iterations of MCTS remaining, &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt;, is larger than the number of children, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, then it doesn't matter what you do, because you'll find this node again the next iteration and expand one of the children. This seems to suggest that the only time it matters is when &lt;span class=&quot;math-container&quot;&gt;$X_t &amp;lt; N$&lt;/span&gt; and in situations like AlphaZero, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is so much smaller than &lt;span class=&quot;math-container&quot;&gt;$X_0$&lt;/span&gt;, that this basically never matters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cases where &lt;span class=&quot;math-container&quot;&gt;$X_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; are of similar size, then it seems like the number of iterations really needs to be changed into something like an amount of time and sometimes you spend your time doing playouts while other times you spend your time computing children.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have I thought about this correctly?&lt;/p&gt;&#xA;" OwnerUserId="27530" LastEditorUserId="2444" LastEditDate="2019-11-19T19:58:07.017" LastActivityDate="2019-11-19T19:58:07.017" Title="How can we efficiently and unbiasedly decide which children to generate in the expansion phase of MCTS?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13669" PostTypeId="1" AcceptedAnswerId="13670" CreationDate="2019-07-30T06:56:21.333" Score="1" ViewCount="358" Body="&lt;p&gt;If you are a freelancer, when a client asks to create a website we can easily measure how much the total cost is needed based on the requirements of the client. (the backend, UI/UX design, features, etc.). We can even measure the estimated time of completion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if a client asks you to make an AI project (image recognition, speech recognition, or NLP), how do you tell the client the estimated cost and time needed to complete the project in the beginning? because the results obtained can be very different for each data used&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2021-03-17T10:51:05.830" LastActivityDate="2021-03-17T10:51:05.830" Title="How to estimate the cost and time to complete an AI Project" Tags="&lt;ai-design&gt;&lt;profession&gt;&lt;computational-complexity&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2021-03-17T10:55:56.207" ContentLicense="CC BY-SA 4.0" />
  <row Id="13670" PostTypeId="2" ParentId="13669" CreationDate="2019-07-30T09:46:20.963" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If you are a freelancer, when a client asks to create a website we can easily measure how much the total cost is needed based on the requirements of the client. (the backend, UI/UX design, features, etc.). We can even measure the estimated time of completion.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is only the case when the full scope and design of the site is a well understood and relatively standard thing. With more experience, and perhaps a dedicated team, more complex sites and features can be addressed like this. It is just as common in my experience to view ongoing development of a site as a series of related projects, each of which can be estimated and costed as they become more feasible and closer to starting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never seen a stand-alone site fully costed and estimated in my own work, because I work in-house on more complex features than this is possible for, but am aware this is a common approach, especially for agencies and freelancers. The difference is that the people involved are experts on producing well-defined work, and that in order to make a pre-estimated or even fixed price sale, they have put a lot of effort into risk management. Some of the risk management is contractual (e.g. &quot;maximum of 3 major revisions&quot; for web site design), some of it is in requirements gathering - often an initial pre-sales consultancy meeting is arranged, and/or a project proposal document generated where a lot of the possible misunderstandings between customer and developer are dealt with, and it is made clear what expectations are on both sides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a freelancer or agency, this de-risking is also spread over multiple projects as they will develop solutions that can be re-used including standardised contracts and templates for project planning etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also a few different project management approaches that deal with initial uncertainty in different ways. A &quot;waterfall&quot; approach of pre-planning as much in advance as possible requires very good knowledge of the work that will occur in later stages of a project, which of course depends heavily on earlier stages meeting all expectations of both the customer and developer. An &quot;agile&quot; approach will acknowledge that there are unknowns, and attempt to address them with some form of just in time planning. The problem with an agile approach is that &quot;just in time&quot; may be far too late if you need to budget up front (although it is still possible for instance by reducing scope to fit, provided that is agreed). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What if the client asks you to make an AI project (image recognition, speech recognition, or NLP), how do you tell the client the estimated cost and time needed to complete the project in the beginning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;There is no difference in this respect just because it is an AI project&lt;/em&gt;. This would apply to any software project, including web development, where one or more of the following is true:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Scope of work is not clear&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are technical hurdles with unknown resolutions. &lt;em&gt;I think it is likely you are focusing on this for your imagined AI project.&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Customer expectations are not clear&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Project success criteria is not measurable, or not known if achievable&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As any software developer, if you are faced with project work with these features, you do some pre-project tasks focused on de-risking the project before attempting it. There are a few different approaches, than can be used in combination:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Gather more requirements, spend more time up front consulting on the project&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Test assumptions before starting project. E.g. if you are not sure about quality of the data, take a look at it as early as possible. If you don't know if technology X can do Y, investigate it in advance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Work on a timesheet basis, not fixed cost. Alternatively fix cost (or have a maximum), but leave scope of work open.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have a flexible project plan, and re-estimate remaining work routinely.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Put any assumptions about customer support for the project, such as availablilty and quality of data, into project plan and in general form into your standard contract (e.g. something legal along the lines of &quot;Project delivery depends on customer providing resources as outlined in the project plan document. If the resources are not provided in reasonable time, then the project may over-run or have additional costs, which will be covered by the customer, and not by the freelancer&quot;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For additional up-front work on project feasibilty and scope, if you are concerned about risk to yourself as a freelancer (because customer expects these as a pre-sale effort), you could propose to bill for that as consultancy work, prior to starting project proper. You would likely need to produce something like a project proposal document in order that the paying customer received something for spending this money. Producing such a document might even show both you and the customer that the project is not feasible without additional work - in whch case you could note the steps needed to make it feasible (e.g. customer needs to source 10,000 more training images without the flaws noticed in existing images).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-07-30T11:41:22.837" LastActivityDate="2019-07-30T11:41:22.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13671" PostTypeId="1" AcceptedAnswerId="13679" CreationDate="2019-07-30T10:14:11.237" Score="5" ViewCount="329" Body="&lt;p&gt;Many games have multiple paths to the same states. What is the appropriate way to deal with this in MCTS? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the state appears once in the tree, but with multiple parents, then it seems to be difficult to define back propagation: do we only propagate back along the path that got us there &quot;this&quot; time? Or do we incorporate the information everywhere? Or maybe along the &quot;first&quot; path?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the state appears once in the tree, but with only one parent, then we ignored one of the paths, but it doesn't matter because by definition this is the same state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the state appears twice in the tree, aren't we wasting a lot of resources thinking about it multiple times?&lt;/p&gt;&#xA;" OwnerUserId="27530" LastEditorUserId="2444" LastEditDate="2019-07-31T22:17:08.683" LastActivityDate="2019-07-31T22:17:08.683" Title="What is the appropriate way to deal with multiple paths to same state in MCTS?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13672" PostTypeId="2" ParentId="13604" CreationDate="2019-07-30T12:02:23.020" Score="0" Body="&lt;p&gt;It turns out that the zig-zag pattern is an inherent effect of using a word embedding layer. I don't fully understand the phenomenon, but I believe it has a strong correlation with the embeddings acting as a sort of &lt;em&gt;memory slots&lt;/em&gt;, which can change relatively quickly, and the LSTM generating a summary of the sequence, so that the model can remember past combinations.&lt;/p&gt;&#xA;&lt;p&gt;I found this plot of a &lt;a href=&quot;https://www.d2l.ai/chapter_natural-language-processing/word2vec-gluon.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;training loss curve of word2vec&lt;/a&gt; and it exhibits the same per-epoch pattern.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZtLAj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZtLAj.png&quot; alt=&quot;word2vec loss&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Edit&lt;/h3&gt;&#xA;&lt;p&gt;After conducting several experiments, I've isolated the causes. It seems that this is an indirect effect of having a large model capacity. In my case, I had too large word embeddings (size 1024) and too many classes (2002), which also increases model capacity, so the model was doing an almost per-sample learning. Reducing both resulted in a smooth-as-silk learning curve and a better generalisation.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-01T11:23:30.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13673" PostTypeId="2" ParentId="13671" CreationDate="2019-07-30T12:53:38.777" Score="1" Body="&lt;p&gt;Node in a tree must have a single parent, otherwise it violates a tree definition. Also the way I look at it, there are no &quot;same&quot; states when you do MCTS. Because you are keeping the history of how you got there. So the second time you visit the &quot;same&quot; state it'll have a different history path and a single parent. &lt;/p&gt;&#xA;" OwnerUserId="27516" LastActivityDate="2019-07-30T12:53:38.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13674" PostTypeId="2" ParentId="13660" CreationDate="2019-07-30T13:20:48.837" Score="0" Body="&lt;p&gt;When you trained your agents separately they never saw squares with opposing values (1/-1). So agents don't really know what to expect from visiting that square. &#xA;I'd try adding (1/-1) to the condition on which you base your squares availability. Also try increasing the reward. It's hard to give suggestions without looking at the code.&lt;/p&gt;&#xA;" OwnerUserId="27516" LastActivityDate="2019-07-30T13:20:48.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13675" PostTypeId="2" ParentId="13607" CreationDate="2019-07-30T13:32:32.693" Score="1" Body="&lt;p&gt;The layout problem is suitable for a rule-based approach, as used to generate levels in Rogue or customised home-layouts, you encode some constraints and search the remaining space. The colour problem is also suitable for a rule-based approach, colour is a 3-dimensional space (4 if you include opacity). A palette can be created through taking colours at appropriate distances from a starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seen as this domain is well analysed, there are known heuristic and encodable rules. You'll likely have more success and faster by using these to generate designs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to give an idea, seen as rule based approach isn't discussed much these days, the following is valid Prolog, query with &lt;code&gt;phrase(webpage, Design).&lt;/code&gt;, that'll quickly generate designs like: &lt;code&gt;[brand, header_nav, searchbox, sidebar_nav, form_content, sidebar_list]&lt;/code&gt;. This is just a super-quick demo, you'd likely want a tree instead of a list and you'll need to derive HTML and CSS from it too, which is simple enough in Prolog.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;webpage --&amp;gt; body.&#xA;webpage --&amp;gt; header, body.&#xA;webpage --&amp;gt; header, body, footer.&#xA;&#xA;header --&amp;gt; [brand], header_content.&#xA;header_content --&amp;gt; [header_nav].&#xA;header_content --&amp;gt; [header_nav], [searchbox].&#xA;&#xA;body --&amp;gt; content.&#xA;body --&amp;gt; sidebar(_), content.&#xA;body --&amp;gt; { dif(A, B) }, sidebar(A), content, sidebar(B).&#xA;body --&amp;gt; content, sidebar(_).&#xA;&#xA;content --&amp;gt; [form_content].&#xA;content --&amp;gt; [article_content].&#xA;content --&amp;gt; [list_content].&#xA;&#xA;sidebar(list) --&amp;gt; [sidebar_list].&#xA;sidebar(nav) --&amp;gt; [sidebar_nav].&#xA;sidebar(ads) --&amp;gt; [sidebar_ads].&#xA;&#xA;footer --&amp;gt; [notification].&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15541" LastActivityDate="2019-07-30T13:32:32.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13676" PostTypeId="2" ParentId="13523" CreationDate="2019-07-30T14:28:13.443" Score="1" Body="&lt;p&gt;While I dont know if something like what I will describe exist, I will give my idea of how this might be achieved. I think there are two ways, one being an nlp network that as you said, you &quot;show the net its own code&quot; and let if try to output code that runs but is different, likely this would be done in an rl environment with another problem domain to determine loss or fitness of the new code it outputs to be evaluated then you (meaning it would also have outputs for the this task domain as it needs to be evaluated just like the new version of itself is). &#xA;The other way would be a cppn that has output functions that control the modification of new structures and weights of itself, you may have one node that determines if you should add or remove a layer (sigmoid), another two ouputs one that determines which layer and another for the amount of nodes to add or remove to the layer (maybe relu or some other linear function that could be rounded down and used for this), and so on and so forth for whatever you want it to learn to control. Again you would likely have this running in an rl environment with an accompanying fitness function for a problem other than modifying itself and set of outputs that are evaluated by the fitness function, then you would have a self contained network that can learn to &quot;build itself&quot; into a network that converges to a high degree of fitness. You would have some sort of optimization algorithm to adjusts weights before making the modifications that are output. &#xA;This is just a total shot in the dark but I imagine it could be done by weaseling in the modification of itself into your existing rl environments problem space.   &lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-07-30T14:28:13.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13677" PostTypeId="2" ParentId="13641" CreationDate="2019-07-30T14:48:57.093" Score="0" Body="&lt;p&gt;You're going to run straight into the qualification problem, the frame problem and ramification problem on these. Plus you'll need to get your head round what an action is and means.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Qualification Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A more formal &quot;STRIPS&quot; would be Situation Calculus (logical equivalency demonstrated by Reiter), which can be defined in FOL or SOL, depending on who you talk to. In Situation Calculus you have your actions and you define when they are possible (S is the current Situation):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;poss(open_door, S) → holds(door_position(closed) ^ door_lock(unlocked), S).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is true, when it is possible to open the door it's both closed and unlocked. The difficulty arises when we try to flip that implication around:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;poss(open_door, S) ← holds(door_position(closed) ^ door_lock(unlocked), S).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is not true, because there must also be nothing blocking the door, the door can't be wedged too tightly into the door frame, and a whole bunch of other stuff that we might not think of. This is the qualification problem, which we get around in STRIPS by ignoring it and just flipping the implication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case you'll end up learning a whole bunch of irrelevant qualifications:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; poss(open_door, S) ← holds(door_position(closed) ^ door_lock(unlocked) ^ weather(sunny) ^ grass(green), S).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'd need some way to trim them down, by observing gameplay for your data you'll only learn the conditions when players choose to do an action, not when it is possible to do that action: the old correlation vs causation problem. You can immediately prune things that aren't fluents, but as far as I'm aware there's no good solution to this without letting the computer simulate to try actions in subsets of the fluents that hold in-order to prune.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Frame Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The result of an action in the language of Situation Calculus is that it causes some fluent to hold. So in our example we have a &lt;code&gt;door_position&lt;/code&gt; fluent, we can say that opening the door causes &lt;code&gt;door_position(open)&lt;/code&gt; to hold, but what about some other action, such as making tea? We know it has no effect on the position of the door, but for the computer to know that we need to tell it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;holds(door_position(open), S) → holds(door_position(open), do(make_tea, S)).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates far too many axioms. Reiter has a working solution to this for Golog in his book &quot;Knowledge in Action&quot;. In your case it will manifest as missing axioms, i.e. not knowing what the result of an action will be for some fluent. Try beginning with the assumption that an action has no effect unless proven otherwise.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Ramification Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The ramification problem is very similar to the frame problem, it means you might not attribute all of the effects of an action to that action. Opening a door can cause changes in air-pressure through a house that can cause another door to slam, such an effect is easy to miss. For this you'd need a sensitive algorithm that can correctly recognise rare events that are important, which is a big ask. The solution is likely to be to ignore it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Closely related and not usually a problem for such systems, you could also attribute effects to an action incorrectly. How will you be able to distinguish what the action caused vs occurred simultaneously? In some Mario game data it could likely be induced that jumping causes a turtle to move closer to Mario, again correlation vs causation and needing a method to distinguish. The solution here is likely to come from increasing the volume of data.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Definition and Granularity Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;OK, it's probably possible that with enough data and a sensitive training algorithm you can overcome the above problems. The definition one is merely a conceptual one and easily overcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Humans tend to define an action by it's name, say for example &quot;turning on a light&quot;. However, in Situation Calculus the action is defined by when it is possible and the result of that action, the name is merely a label. So &quot;turning on a light&quot; where the bulb is broken and when it is not may share the same label, but they are different actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, in STRIPS and Situation Calculus we're defining &quot;types&quot; or &quot;classes&quot; of actions, the actual execution is an &quot;instance&quot; of that &quot;class&quot;. Your problem is to derive the abstract from the concrete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For you this problem will arise in terms of defining the granularity, as each instance of an action is possible in the unique situation that it occurred in, and caused the unique situation that came after it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you group these actions together to abstract the most similar ones? How do you know how many actions there are to create these groups? If it's not based on a limited set of user-inputs to some computer game, then you might want to draw some ideas from the research into clustering, particularly heuristics of how many clusters to use.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Alternative to a decision tree&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If I were to tackle this problem (fund a Post-Doc for me and I'd be tempted), I'd first investigate ILP as a method to learn the relations using Golog as the foundation of my action axioms. Then I'd look at clustering based on a space where each fluent was a dimension. I imagine it would be a rather useful tool, even if it merely created a pretty accurate template for manual review.&lt;/p&gt;&#xA;" OwnerUserId="15541" LastEditorUserId="15541" LastEditDate="2019-07-30T14:56:29.843" LastActivityDate="2019-07-30T14:56:29.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13678" PostTypeId="2" ParentId="1906" CreationDate="2019-07-30T15:40:34.060" Score="3" Body="&lt;p&gt;Sure! There's the whole Semantic Web scene! OWL is derived from DLs and Frames, arguably has a lot in common with semantic networks too. Expert-driven decision support systems are still being developed (and researched) in industries where the human is required to take responsibility or getting data is not going to happen. As the ideas evolve so do the names.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out the academic conferences like &lt;a href=&quot;http://reasoning.eas.asu.edu/kr2018/&quot; rel=&quot;nofollow noreferrer&quot;&gt;KR&lt;/a&gt;, &lt;a href=&quot;https://iswc2019.semanticweb.org/accepted-papers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ISWC&lt;/a&gt;, &lt;a href=&quot;http://fois2018.cs.uct.ac.za/?page_id=46&quot; rel=&quot;nofollow noreferrer&quot;&gt;FOIS&lt;/a&gt;, even broader AI conferences like &lt;a href=&quot;http://static.ijcai.org/2019-Program.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;IJCAI&lt;/a&gt; have a healthy dose of symbolic  AI, I even spotted a search algorithm in the 2019 line up.&lt;/p&gt;&#xA;" OwnerUserId="15541" LastActivityDate="2019-07-30T15:40:34.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13679" PostTypeId="2" ParentId="13671" CreationDate="2019-07-30T15:47:41.410" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If the state appears twice in the tree, aren't we wasting a lot of resources thinking about it multiple times?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You're right. Precisely the same problem was also noticed decades before MCTS existed, in the classic minimax-style tree search algorithms (alpha-beta search, etc.) that were used in games before MCTS. The solution is also mostly the same; &lt;strong&gt;transposition tables&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of MCTS, the statistics used by the algorithm that are normally associated with nodes (or their incoming edges) may instead be stored in entries of a transposition table. I mean stuff like visit counts and sums (or averages) of backpropagated scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A brief description of how it would work, and references to more extensive relevant literature, can be found in subsubsection 5.2.4 of the &lt;a href=&quot;http://mcts.ai/pubs/mcts-survey-master.pdf&quot; rel=&quot;noreferrer&quot;&gt;well-known 2012 Survey paper on MCTS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does require that you can efficiently (incrementally) compute hash values for the states you encounter, which may not always be easy (should usually be possible, depends on the details of your problem domain). Use of transpositions in MCTS is also not always guaranteed to actually improve performance. It does come with computational overhead, and in games where transpositions are very rare it may be more efficient to simply ignore them and use the regular tree structure.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-30T15:47:41.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13680" PostTypeId="2" ParentId="12979" CreationDate="2019-07-30T15:51:14.750" Score="2" Body="&lt;p&gt;Not all search is planning (is A connected to B), but all planning is search (&lt;strong&gt;how&lt;/strong&gt; do I get from this to that). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://swish.swi-prolog.org/p/A*%20Planning.swinb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an example in Prolog&lt;/a&gt; with a domain described in terms of actions, when they are possible, and what the result of the actions are. The description is of an uncomputed graph of un-calculated size where each node is a situation and each edge is an action. Then we have A* search algorithm that searches this graph, calculating it as it goes, to find a plan to reach the goal state. Running the final query will produce a plan via search.&lt;/p&gt;&#xA;" OwnerUserId="15541" LastActivityDate="2019-07-30T15:51:14.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13681" PostTypeId="2" ParentId="13117" CreationDate="2019-07-30T16:24:57.447" Score="0" Body="&lt;p&gt;This is theoretically possible for an exhaustive set of sequential, non-concurrent primitive actions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall s_1, s_2 \left(  \exists a \left(  \text{possible}(a, s_1) \land \text{do}(a, s_1, s_2) \right) \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; is the prior situation, &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt; is the result of doing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\text{possible}(a, s_1)$&lt;/span&gt; is true if it is possible to do an action in a situation (See &lt;a href=&quot;https://en.wikipedia.org/wiki/Situation_calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Situation Calculus&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;So, for a given situation, reduce your search space to those actions that are possible, then reduce your search space again to those that result in the following situation.&lt;/p&gt;&#xA;" OwnerUserId="15541" LastEditorUserId="2444" LastEditDate="2021-02-06T13:45:22.167" LastActivityDate="2021-02-06T13:45:22.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13682" PostTypeId="2" ParentId="13523" CreationDate="2019-07-30T17:40:28.607" Score="2" Body="&lt;h2&gt;Can a neural network modify its own weights?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;One important step in training a neural network is called &lt;strong&gt;backpropagation&lt;/strong&gt;. In the course of this process, the weights of the neural network are updated into a direction that minimizes the training loss. Usually, this step happens after each batch (with batch gradient descent) or after each sample (with stochastic gradient descent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, &lt;em&gt;training a neural network&lt;/em&gt; does not mean anything else than iteratively updating its weights. However, note that the weights are not updated by the neural network itself, but by the optimizer and the machine learning framework you use.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Can a neural network modify its architecture?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Currently, there is no popular solution of a neural network which is able to change its own structure. However, there has been a lot of research in this area at least since the 90s. The search-term with which you will find most research in this area is &lt;strong&gt;self-organizing networks&lt;/strong&gt;. Some of the first self-organizing networks were from &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0893608094900914&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fritzke (1994)&lt;/a&gt; and from &lt;a href=&quot;https://papers.nips.cc/paper/975-dynamic-cell-structures&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bruske and Sommer (1994)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, these were not neural networks that learned how to optimize themselves but neural networks that had the ability to grow while following certain rules. And that has a reason. If you think about it, you want your network to be optimized in solving a certain problem (e.g. detecting breast cancer on X-ray images) as good as possible. You do not want it to waste time and resources on learning how to optimize neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are other approaches in this direction! There is a lot of research on how to train a neural network on the task of creating other neural networks. Currently, the most famous are &lt;a href=&quot;https://arxiv.org/abs/1611.02167&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaQNN (2017)&lt;/a&gt; (less known) and &lt;a href=&quot;https://arxiv.org/abs/1611.01578&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;NASNet&lt;/strong&gt; (2017)&lt;/a&gt;. They use &lt;strong&gt;reinforcement learning&lt;/strong&gt; to train a model on creating architectures that maximize the performance (e.g. validation accuracy). However, even as they use reinforcement learning, they perform only slightly better than random search (see &lt;a href=&quot;https://arxiv.org/abs/1707.07012&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). That means the question is more how you define the search space than whether you use neural networks for architecture optimization.&lt;/p&gt;&#xA;" OwnerUserId="23934" LastEditorUserId="23934" LastEditDate="2019-07-31T15:48:47.400" LastActivityDate="2019-07-31T15:48:47.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13683" PostTypeId="1" AcceptedAnswerId="13687" CreationDate="2019-07-30T18:02:28.337" Score="4" ViewCount="956" Body="&lt;p&gt;I'm reading the &lt;a href=&quot;https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ImageNet Classification with Deep Convolutional Neural Networks&lt;/a&gt; paper by Krizhevsky et al, and came across these lines in the Intro paragraph:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Their (convolutional neural networks') capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, &lt;strong&gt;stationarity of statistics and locality of pixel dependencies&lt;/strong&gt;). Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, &lt;strong&gt;while their theoretically-best performance is likely to be only slightly worse&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What's meant by &quot;stationarity of statistics&quot; and &quot;locality of pixel dependencies&quot;? Also, what's the basis of saying that CNN's theoretically best performance is only slightly worse than that of feedforward NN?&lt;/p&gt;&#xA;" OwnerUserId="27548" LastEditorUserId="2444" LastEditDate="2019-07-30T22:39:00.987" LastActivityDate="2019-07-31T10:27:06.947" Title="What is the meaning of &quot;stationarity of statistics&quot; and &quot;locality of pixel dependencies&quot;?" Tags="&lt;convolutional-neural-networks&gt;&lt;terminology&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13684" PostTypeId="2" ParentId="13523" CreationDate="2019-07-30T18:02:37.780" Score="3" Body="&lt;p&gt;Yes, the notion is valid and it has been indeed explored to some extent, although we are still far away from a breath-taking result. These are the topics that have been explored in that regard and as far as I know:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding &lt;strong&gt;code generation&lt;/strong&gt;, the most successful results imply the use of Domain-Specific Languages and/or strong restrictions. A neural policy would then be used to dynamically &lt;em&gt;explore&lt;/em&gt; that domain. I recommend you these articles:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;S. Reed &amp;amp; N. de Freitas. Neural Programmer-Interpreters.&lt;/li&gt;&#xA;&lt;li&gt;M. Balog et al. DeepCoder: Learning to Write Programs&lt;/li&gt;&#xA;&lt;li&gt;D. A. Abolafia et al. Neural Program Synthesis with Priority Queue Training&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding the use of a &lt;strong&gt;NN as an optimizer&lt;/strong&gt;, I'd highlight two articles, which explore a neural optimizer and an initial-state predictor, respectively:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;O. Wichrowska et al. Learned Optimizers that Scale and Generalize.&lt;/li&gt;&#xA;&lt;li&gt;S. Ravi &amp;amp; H. Larochelle. Optimization as a Model for Few-Shot Learning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My personal bet goes for the descriptive methods i.e. the ones that solve the problem by creating a representation to work on (like generating code, which isn't the only way), but as said, this is starting and is mostly speculative.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-07-30T18:02:37.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13685" PostTypeId="2" ParentId="13668" CreationDate="2019-07-30T19:07:09.143" Score="1" Body="&lt;p&gt;The first thing to consider in this question is: what do we mean when we talk about &quot;generating a child/node&quot;. Just creating a node for a tree data structure, and allocating some memory (initialised to nulls / zeros) for data like deeper children, visit counts, backpropagated scores, etc., is rarely a problem in terms of efficiency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you also include generating a game state to store in that node when you say &quot;generating a node&quot;, that can be a whole lot more expensive, since it requires applying the effects of a move to the previous game state to generate the new game state (and, depending on implementation, probably also requires first copying that previous game state). But you don't have to do this generally. You can just generate nodes, and only actually put a game state in them if you later on reach them again through the MCTS Selection phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you could say that AlphaZero does indeed generate all the nodes for all actions immediately, but they're generally &quot;empty&quot; nodes without game states. They do get &quot;primed&quot; with probabilities computed by the policy network, but that policy network doesn't require successor states inside those nodes; it's a function &lt;span class=&quot;math-container&quot;&gt;$\pi(s, a)$&lt;/span&gt; of the &lt;strong&gt;current&lt;/strong&gt; state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (inside the previous node), and the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; leading to the newly-generated node.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;But if you're really sure that, for your particular problem domain, the generation of nodes itself already is inefficient, then...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] This is clearly biased in favor of children earlier in the stream.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, you would get a significant bias with such a stream-based approach, probably wouldn't work well.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] This may be harder to implement than the previous version because computing the number of children may be as hard as computing the children themselves. [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Again I agree with your observation, I don't think there are many problems where this would be a feasible solution.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I believe that if the number of iterations of MCTS remaining, X_t, is larger than the number of children, N, then it doesn't matter what you do, because you'll find this node again the next iteration and expand one of the children.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This would only be correct for the children of the root node. For any nodes deeper in the tree, it is possible that MCTS never reaches them again even if &lt;span class=&quot;math-container&quot;&gt;$X_t &amp;gt; N$&lt;/span&gt;, because it could dedicate most of it search effort to different subtrees.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I think your solution would have to involve some sort of learned function (like the policy network in AlphaZero) which can efficiently compute a recommendation for a node to generate, only using the inputs that are already available before you pick a node to generate. In AlphaZero's policy network, those inputs would be the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in your current node, and the outward actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (each of which could lead to a node to be generated). &lt;strong&gt;This would often actually be very far from unbiased&lt;/strong&gt;, but I imagine a strong, learned bias would likely be desireable anyway if you're in a situation where the mere generation of nodes is a legitimate concern for performance.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-07-30T19:07:09.143" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13687" PostTypeId="2" ParentId="13683" CreationDate="2019-07-30T22:23:35.617" Score="3" Body="&lt;p&gt;&lt;em&gt;locality of pixel dependencies&lt;/em&gt; probably means that neighboring pixels tend to be correlated, while faraway pixels are &lt;em&gt;usually&lt;/em&gt; not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;stationarity of statistics&lt;/em&gt; might mean that the values of the pixels do not change over time, so this could be related to &lt;a href=&quot;https://en.wikipedia.org/wiki/Anisotropic_diffusion&quot; rel=&quot;nofollow noreferrer&quot;&gt;diffusion techniques&lt;/a&gt; in image processing. &lt;em&gt;stationarity of statistics&lt;/em&gt; might also mean that the values of the pixels do not change much in a spatial neighborhood, even though, stationarity, e.g. in reinforcement learning, usually means that something does not change &lt;em&gt;over time&lt;/em&gt; (so, if that's the case, the terminology &lt;em&gt;stationarity&lt;/em&gt; is at least misleading and confusing, in this context), so this might be related to the &lt;em&gt;locality of pixel dependencies&lt;/em&gt; property. Possibly, &lt;em&gt;stationarity of statistics&lt;/em&gt; could also indirectly mean that you can use the same filter to detect the same feature in different regions of the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With &lt;em&gt;while their theoretically-best performance is likely to be only slightly worse&lt;/em&gt; the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both &lt;a href=&quot;https://ai.stackexchange.com/a/13319/2444&quot;&gt;CNNs and FFNNs are universal function approximators&lt;/a&gt; (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-07-31T10:27:06.947" LastActivityDate="2019-07-31T10:27:06.947" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13688" PostTypeId="2" ParentId="13662" CreationDate="2019-07-30T23:02:45.397" Score="5" Body="&lt;h3&gt;tl;dr&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What does that mean in the context of this paper?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With &quot;coarse segmentation&quot; the author means a segmentation that doesn't have much detail. &quot;Fine segmentation&quot;, on the other hand, refers to a segmentation with a high level of detail.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But also more importantly &lt;em&gt;[what does that mean in the context of]&lt;/em&gt; general computer vision?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The most common use in CV is to describe how general or specific a class in is classification. A &quot;coarse class&quot; is a very broad one, while a &quot;fine class&quot; is a very specific one.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Intended use&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;What the author refers to is the &lt;strong&gt;level of detail&lt;/strong&gt; of the resulting segmentation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;coarse&lt;/strong&gt; segmentation would mean that we have large blobs covering each class without much detail. On the other hand, a &lt;strong&gt;fine&lt;/strong&gt; segmentation would have a much higher level of detail which can even go down to pixel level (i.e. pixel-by-pixel correct segmentation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this clear look at the following two examples. As we go from the left to right, the segmentation maps go from coarse to fine:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tamtb.png&quot; alt=&quot;&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/vT8Pe.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in the rightmost images the result an almost pixel-perfect (i.e. fine details) segmentation map, while the ones on the left don't have much detail and can be considered coarse.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Alternative use&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Because this isn't an established terminology, some of the times coarse and fine can refer to the nature of the classes in a classification task. Take the top image for example; the label for a coarse classification task could be a &lt;em&gt;tree&lt;/em&gt;. For a fine classification task we would have labels like &lt;em&gt;oak tree&lt;/em&gt;, &lt;em&gt;pine tree&lt;/em&gt;, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most prominent example of this is the &lt;a href=&quot;https://www.cs.toronto.edu/~kriz/cifar.html&quot; rel=&quot;noreferrer&quot;&gt;cifar&lt;/a&gt; dataset which has two versions: a coarse one that has 10 classes and a fine one that has 100 classes, which are all subclasses of the coarse classes. For example a coarse class is &lt;em&gt;fish&lt;/em&gt;, while the fine ones are &lt;em&gt;aquarium fish&lt;/em&gt;, &lt;em&gt;flatfish&lt;/em&gt;, &lt;em&gt;ray&lt;/em&gt;, &lt;em&gt;shark&lt;/em&gt;, &lt;em&gt;trout&lt;/em&gt;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For semantic segmentation an example could be the following: you want to make a street segmentation model. This a coarse segmentation would mean that it just splits the image into &lt;em&gt;road&lt;/em&gt;, &lt;em&gt;vehicle&lt;/em&gt;, etc. A fine segmentation, on the other hand, could also detect the type of vehicle, e.g. &lt;em&gt;truck&lt;/em&gt;, &lt;em&gt;car&lt;/em&gt;, etc.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-07-30T23:02:45.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13689" PostTypeId="1" CreationDate="2019-07-30T23:45:11.930" Score="1" ViewCount="23" Body="&lt;p&gt;I need to build a multi-camera people tracking system and I have no idea how to start. I read ML for Dummies and I've watched a lot of youtube classes/conferences and read a lot of articles about ML/DL, so I have all this theoretical information about what is a NN, loss function, weights, vectors, convolution, etc., but when I need to start building something, I get stuck. Even more, I don't think I can create my own models because I only have six months to finish this and I'm not sure if I'll be able to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read some papers explaining architectures for an improved people-tracking system (e.g. &lt;a href=&quot;https://www.intechopen.com/online-first/multi-person-tracking-based-on-faster-r-cnn-and-deep-appearance-features#B8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.intechopen.com/online-first/multi-person-tracking-based-on-faster-r-cnn-and-deep-appearance-features#B8&lt;/a&gt;), and it says it used ResNet-30 and stuff like that. My question is, how could I recreate the architectures in papers like that? Where can I find those pre-trained models? Or is there a place where I can get the data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to start with at least a people-tracking system, without worrying about the multi-camera part for now, and I thought of almost the same approach as the people in the paper posted, meaning I want to recognize people based on parts of their body/the whole body to identify them, and track them based on their unique features (clothing color, hair, skin tone, etc), maybe skipping the part of facial recognition since that's too advanced I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea on where to start?&#xA;Sorry if the question is too broad or too complex. Comments about first steps and sub-dividing the problem are also welcome.&#xA;PS: The main ultimate is to track how much time people are in a certain area filmed by many cameras.&lt;/p&gt;&#xA;" OwnerUserId="27552" LastActivityDate="2019-07-30T23:45:11.930" Title="Where to find pre-trained models for multi-camera people tracking?" Tags="&lt;deep-learning&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13691" PostTypeId="1" CreationDate="2019-07-31T03:07:26.250" Score="3" ViewCount="138" Body="&lt;p&gt;What I know about CRF is that they are discriminative models, while HMM are generative models, but, in the inference method, both use the same algorithm, that is, the Viterbi algorithm, and forward and backward algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does CRF use the same features as HMM, namely features transition and state features? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in here &lt;a href=&quot;https://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf&lt;/a&gt;, CRF has these features Edge-Observation and Node-Observation Features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference features transition and state features vs features Edge-Observation and Node-Observation features?&lt;/p&gt;&#xA;" OwnerUserId="22686" LastEditorUserId="2444" LastEditDate="2019-07-31T22:15:23.637" LastActivityDate="2019-07-31T22:15:23.637" Title="What are the differences between CRF and HMM?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;supervised-learning&gt;&lt;hidden-markov-model&gt;&lt;conditional-random-field&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13692" PostTypeId="1" AcceptedAnswerId="13786" CreationDate="2019-07-31T06:09:54.267" Score="7" ViewCount="3435" Body="&lt;p&gt;I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, while the 3D convolution helps detect low-level features and relationships between them in all the 3 dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a CNN employing 2D convolutional layers to recognize handwritten digits. If a digit, say 5, was written in different colors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3Ajlf.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3Ajlf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would a strictly 2D CNN perform poorly (since they belong to different channels in the z-dimension)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, are there practical well-known neural nets that employ 3D convolution?&lt;/p&gt;&#xA;" OwnerUserId="27555" LastEditorUserId="2444" LastEditDate="2019-07-31T22:01:15.267" LastActivityDate="2019-08-05T07:49:31.513" Title="When should I use 3D convolution?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13693" PostTypeId="1" CreationDate="2019-07-31T06:35:55.440" Score="4" ViewCount="58" Body="&lt;p&gt;I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;output the angle in radians&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;cyclic nature of the angles is not captured&lt;/li&gt;&#xA;&lt;li&gt;output might be outside &lt;span class=&quot;math-container&quot;&gt;$\left[-\pi, \pi \right)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;output the sine and the cosine of the angle&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;outputs might not satisfy &lt;span class=&quot;math-container&quot;&gt;$\sin^2 \theta + \cos^2 \theta = 1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of different methods?&lt;/p&gt;&#xA;" OwnerUserId="20859" LastEditorUserId="20859" LastEditDate="2019-07-31T15:01:26.667" LastActivityDate="2021-04-22T01:16:56.587" Title="What kind of output should be used for predicting angles in DNNs?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;deep-neural-networks&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13694" PostTypeId="1" AcceptedAnswerId="13702" CreationDate="2019-07-31T06:52:04.033" Score="6" ViewCount="220" Body="&lt;p&gt;I have a decent background in Mathematics and Computer Science .I started learning AI from Andrew Ng's course from one month back. I understand logic and intuition behind everything taught but if someone asks me to write or derive mathematical formulas related to back propagation I will fail to do so.&#xA;I need to complete object recognition project within 4 months.&#xA;Am I on right path?&lt;/p&gt;&#xA;" OwnerDisplayName="user27556" LastEditorUserId="1671" LastEditDate="2019-10-15T19:18:21.700" LastActivityDate="2019-12-30T19:09:50.330" Title="Is it ok to struggle with mathematics while learning AI as a beginner?" Tags="&lt;math&gt;&lt;getting-started&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="2" ClosedDate="2021-01-22T02:27:59.587" ContentLicense="CC BY-SA 4.0" />
  <row Id="13695" PostTypeId="1" CreationDate="2019-07-31T07:49:11.390" Score="1" ViewCount="90" Body="&lt;p&gt;How to detect patterns in a data set of given IP addresses using a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data set is actually a list of all the vulnerable devices on a network. I want to use a neural network that detctes any patters in the occurrences of these vulnerabilities with reference to their IPs and ports.&lt;/p&gt;&#xA;" OwnerUserId="27399" LastActivityDate="2020-09-08T04:01:50.157" Title="How to detect patterns in a data set of given IP addresses using a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13696" PostTypeId="2" ParentId="13693" CreationDate="2019-07-31T07:50:07.317" Score="0" Body="&lt;p&gt;As you said, the first option is not very suitable due to the cyclic nature of the angles. However, if you don't mind discretizing the values, you could represent the output as a binary vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A variant of the second option seems perfect to me. You may output a 2D vector and use that vector's angle as output. You'll probably need a regularizer for the vector's norm, but that's it.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-07-31T07:50:07.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13698" PostTypeId="1" CreationDate="2019-07-31T09:31:40.827" Score="4" ViewCount="317" Body="&lt;p&gt;The &lt;a href=&quot;https://wiseodd.github.io/techblog/2016/12/17/conditional-vae/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conditional Variational Autoencoder (CVAE)&lt;/a&gt;, introduced in the paper &lt;a href=&quot;https://papers.nips.cc/paper/5775-learning-structured-output-representation-using-deep-conditional-generative-models.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Structured Output Representation using Deep Conditional Generative Models&lt;/a&gt; (2015), is an extension of &lt;a href=&quot;https://arxiv.org/abs/1312.6114&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Autoencoder (VAE)&lt;/a&gt; (2013). In VAEs, we have no control over the data generation process, something problematic if we want to generate some specific data. Say, in MNIST, generate instances of 6.&lt;/p&gt;&#xA;&lt;p&gt;So far, I have only been able to find CVAEs that can condition to discrete features (classes). Is there a CVAE that allows us to condition to continuous variables, kind of a stochastic predictive model?&lt;/p&gt;&#xA;" OwnerUserId="27565" LastEditorUserId="2444" LastEditDate="2020-11-22T17:21:05.113" LastActivityDate="2020-11-22T17:51:35.043" Title="Is there a continuous conditional variational auto-encoder?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;autoencoders&gt;&lt;variational-autoencoder&gt;&lt;conditional-vae&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13699" PostTypeId="1" AcceptedAnswerId="13705" CreationDate="2019-07-31T10:01:03.963" Score="3" ViewCount="1141" Body="&lt;p&gt;I am using a CNN to train on some data, where training size = 21700 samples, and test size is 653 samples, and say I am using a batch_size of 500 (I am accounting for samples out of batch size as well). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been looking this up for a long time now, but can't get a clear answer, but when plotting the loss functions to check for whether the model is overfitting or not, do I plot as follows &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for j in range(num_epochs):&#xA;  &amp;lt;some training code---Take gradient descent step do wonders&amp;gt;&#xA;  batch_loss=0&#xA;  for i in range(num_batches_train):&#xA;       batch_loss = something....criterion(target,output)...&#xA;       total_loss += batch_loss&#xA;  Losses_Train_Per_Epoch.append(total_loss/num_samples_train)#and this is &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where I need help&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Losses_Train_Per_Epoch.append(total_loss/num_batches_train)&#xA;and doing the same for Losses_Validation_Per_Epoch.&#xA;plt.plot(Losses_Train_Per_Epoch, Losses_Validation_Per_epoch)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, basically, what I am asking is, should I divide by num_samples or num_batches or batch_size? Which one is it?&lt;/p&gt;&#xA;" OwnerUserId="27566" LastEditorUserId="2444" LastEditDate="2019-08-02T23:47:48.080" LastActivityDate="2019-08-02T23:47:48.080" Title="Are the training loss and validation loss plotted per sample or per batch?" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;supervised-learning&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13700" PostTypeId="2" ParentId="4650" CreationDate="2019-07-31T11:53:09.843" Score="0" Body="&lt;p&gt;Well AI can be used in Major Areas in Telecom&lt;/p&gt;&#xA;&lt;h1&gt;1/ Network Optimization&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Where the network is trained to come up with the required Parameter tunning to solve a particular issue, for example in 4G system we have user licenses for cell level that how many users this cell will support. Every time the number of users increases on that cell the Network optimizer sends a work order to the Backoffice to increase the number of users with temporary license that is in pool. For an AI system it can analyze the utilization and increase the licenses or it can tune the parameters to shift traffic on the neighbour cell, it can learn from the optimizers behaviour and actions, this is a single example&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;2/ Customer Intelligence&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;     Based on the customer behavior, his social media engagements and responses, usage patterns, past usage history and current context  the AI System can generate personalized offers for the user that can add value to the overall customer Journey and make him and engaged happy customer.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;3/ Network expansions and Dynamic capacity:&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;     Mostly  network operators have licenses per resource contracts with vendors, the excess capacity is charged. In most cases for example in summers some particular areas need more additional capacity but once the summers are gone that particular area is the lowest utilized area. An AI system can learn based on the usage patterns when a particular area need higher capacity and when low capacity and can increase and decrease the capacity resulting in reduced costs.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Above are some of the use cases which can further be added with alot of Customer Engagement and Customer Experience Management uses cases&lt;/p&gt;&#xA;&lt;p&gt;Regards&#xA;Muhammad Azfar&#xA;Customer Journey and Business Analyst&#xA;Customer Experience Management&lt;/p&gt;&#xA;" OwnerUserId="27568" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-07-31T11:53:09.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13701" PostTypeId="2" ParentId="13698" CreationDate="2019-07-31T12:49:24.410" Score="1" Body="&lt;p&gt;Whether a discrete or continuous class, you can model it the same.&lt;/p&gt;&#xA;&lt;p&gt;Denote the encoder &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; and the decoder &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. Recall the variational autoencoder's goal is to minimize the &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence between &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;'s posterior. i.e. &lt;span class=&quot;math-container&quot;&gt;$\min_{\theta, \phi} \ KL(q(z|x;\theta) || p(z|x; \phi))$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; parameterize the encoder and decoder respectively. To make this tractable this is generally done by using the Evidence Lower Bound (because it has the same minimum) and parametrizing &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; with some form of reparametrization trick to make sampling differentiable.&lt;/p&gt;&#xA;&lt;p&gt;Now your goal is to condition the sampling. In other words you are looking for modeling &lt;span class=&quot;math-container&quot;&gt;$p(x|z, c;\phi)$&lt;/span&gt; and in turn will once again require &lt;span class=&quot;math-container&quot;&gt;$q(z|x, c; \theta)$&lt;/span&gt;. Your goal will now intuitively become once again &lt;span class=&quot;math-container&quot;&gt;$\min_{\theta, \phi} \ KL(q(z|x, c;\theta) || p(z|x, c; \phi))$&lt;/span&gt;. This is still simply transformed into the ELBO for tractability purposes. In other words your loss becomes &lt;span class=&quot;math-container&quot;&gt;$E_q[log \ p(x|z,c)] - KL(q(z|x,c)||p(z|c)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Takeaway:&lt;/strong&gt; Conditioning doesn't change much, just embed your context and inject it both into the encoder and decoder, the fact that its continuous doesn't change anything. For implementation details, normally people just project/normalize and concatenate it somehow to some representation of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in both the decoder/encoder.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2020-11-22T17:51:35.043" LastActivityDate="2020-11-22T17:51:35.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13702" PostTypeId="2" ParentId="13694" CreationDate="2019-07-31T13:18:55.317" Score="6" Body="&lt;p&gt;I think the key part of your question is &lt;em&gt;&quot;as a beginner&quot;&lt;/em&gt;. For all intents and purposes you can create a state of the art (SoTA) model in various fields with no knowledge of the mathematics what so ever.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means you do not need to understand back-propagation, gradient descent, or even mathematically how each layer works. Respectively you could just know there exists an optimizer and that different layers generally do different things (convolutions are good at picking up local dependencies, fully connected layers are good at picking up connections among your neurons in an expensive manner when you hold no previous assumptions), etc.. Follow some common intuitions and architectures built upon in the field and your ability to model will follow (thanks to the amazing work on opensource ML frameworks -- Looking at you Google and Facebook)! But this is only a stop-gap.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Newton quote that I'm about to butcher: &quot;If I have seen further it's because I'm standing on the shoulders of giants&quot;. In other words he saw &lt;em&gt;further&lt;/em&gt; because he didn't just use what people before him did, he utilized it to expand even further. So &lt;strong&gt;yes&lt;/strong&gt;, I think you can finish your object detection project on time with little understanding of the math (look at the &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research/object_detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google object detection API&lt;/a&gt;, it does wonders and you don't even need to know anything about ML to use it, you just need to have data. But, and this is a big but, if you ever want to extend into a realm that isn't particularly touched upon or push the envelope in any meaningful way, you will probably have to learn the math, learn the basics, learn the foundations.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="4709" LastEditDate="2019-12-30T19:09:50.330" LastActivityDate="2019-12-30T19:09:50.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13703" PostTypeId="2" ParentId="13523" CreationDate="2019-07-31T16:27:30.993" Score="1" Body="&lt;p&gt;I am very surprised no one mentioned the '&lt;a href=&quot;https://arxiv.org/abs/1803.03635&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lottery Ticket Hypothesis&lt;/a&gt;' paper which won the best paper award in ICLR 2019. Although, the main idea has been presented as ways to reduce connections between successive nodes in a Neural Network, it can be viewed as a way Neural Network learning for itself which of its connections are not important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main aim is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reduce the number of weights by cutting connecting weights between successive nodes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How to know which weight to trim?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The method used in the paper is very simple, remove connections with lowest values of weights (after some epochs of training).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What is the effect?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Very strangely the authors found out these Neural Nets with less number of connections performed significantly better than denser Neural Networks. In my opinion, or as I interpret it, these dense connections were introducing noise/irrelevant details which was fudging actual important things, and hence removing these improved the performance of the Neural Net drastically.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The authors have proposed several experiments which show that even if you start to train with the final sparse (lesser number of connections) structure itself, the performance will not reach the desired level. Hence, the final architecture is also a function of the initialisation of weights. Thus a Neural Network itself is learning how to depending on initialisation and inputs which structure is suitable for it. The authors have also extended the results for CNN's.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-08-01T10:16:23.343" LastActivityDate="2019-08-01T10:16:23.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13704" PostTypeId="2" ParentId="13605" CreationDate="2019-07-31T16:31:08.343" Score="2" Body="&lt;p&gt;I think that something like &quot;strength&quot; would be difficult to quantify in this context.  I do think that formal experimentation around the &quot;AI in a box&quot; scenario could be interesting.  I know that experiments have been done where a human plays the role of the AI, attempting to get naive test subjects to &quot;release&quot; him by interacting with them over a chat interface.  In all cases, the &quot;AI&quot; tends to be extraordinarily effective.  It is often simply not possible to anticipate every way someone can trick you into revealing information or creating security holes.  This is how human hackers do their jobs too.  But I think a fully automated &quot;game&quot; could produce some interesting data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way I think it may be interesting to go about creating an &quot;AI in a box&quot; game is to have an ambiguous win condition.  Essentially, make it possible to let the AI out and still &lt;em&gt;think&lt;/em&gt; you are &quot;winning&quot; the game.  One example that pops into my head right away is to use a (pointless) point system.  You tell the player that the primary goal of the game is to keep the AI in the box, but you have a score that is displayed in the game, and tell the player that they win, &lt;em&gt;no matter what&lt;/em&gt;, if their score is higher than the AI's by the end of the game.  Of course, they never get to see the AI's score so they never know how well they are doing by comparison.  This could be justified as representing human vs AI &quot;strength&quot; or &quot;leverage&quot; or whatever else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality the AI &lt;em&gt;doesn't have&lt;/em&gt; a score, but it &lt;em&gt;does&lt;/em&gt; have the ability to effect the player's score.  It can raise the player's score if they do things that help it escape and lower it when the player stands in its way.  The score has no impact on the game and the player &lt;em&gt;really&lt;/em&gt; wins by keeping the AI in the box.  It only serves as a red herring that the AI can use to manipulate the player.  I think this sort of experiment could provide an interesting model of how an AI might leverage bribes, threats, promises, and deception, as well as technical tricks, to convince a human to &quot;let it out of the box&quot;.&lt;/p&gt;&#xA;" OwnerUserId="22130" LastEditorUserId="22130" LastEditDate="2019-07-31T17:29:16.550" LastActivityDate="2019-07-31T17:29:16.550" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13705" PostTypeId="2" ParentId="13699" CreationDate="2019-07-31T16:36:22.923" Score="2" Body="&lt;p&gt;You want to compute the &lt;strong&gt;mean&lt;/strong&gt; loss over all batches. What you need to do is to divide the sum of batch losses with the number of batches!&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;In your case:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You have a training set of &lt;span class=&quot;math-container&quot;&gt;$21700$&lt;/span&gt; samples and a batch size of &lt;span class=&quot;math-container&quot;&gt;$500$&lt;/span&gt;. This means that you take &lt;span class=&quot;math-container&quot;&gt;$21700/500 \approx 43$&lt;/span&gt; training iterations. This means that for each epoch the model is updated &lt;span class=&quot;math-container&quot;&gt;$43$&lt;/span&gt; times! So the way you compute your training loss, that is what you need to divide by. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I'm not sure what exactly you're trying to plot but I'm assuming you want to plot the training losses and the validation losses&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;training_loss = []&#xA;validation_loss = []&#xA;training_steps = num_samples // batch_size&#xA;validation_steps = num_validation_samples // batch_size&#xA;&#xA;for epoch in range(num_epochs):&#xA;&#xA;    # Training steps&#xA;    total_loss = 0&#xA;    for b in range(training_steps):&#xA;        batch_loss = ...  # compute batch loss&#xA;        total_loss += batch_loss&#xA;    training_loss.append(total_loss / training_steps)&#xA;&#xA;    # Validation steps&#xA;    total_loss = 0&#xA;    for b in range(validation_steps):&#xA;        batch_loss = ...  # compute batch validation loss&#xA;        total_loss += batch_loss&#xA;    training_loss.append(total_loss / validation_steps)&#xA;&#xA;# Plot training and validation curves&#xA;plt.plot(range(num_epochs), training_loss)&#xA;plt.plot(range(num_epochs), validation_loss)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another way would be to store the losses in a list and compute the mean. You can use this if you're not sure with what to divide.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;&#xA;for epoch in range(num_epochs):&#xA;&#xA;    list_of_batch_losses = []  # initialize list that is going to store batch losses&#xA;&#xA;    # Training steps&#xA;    for b in range(training_steps):&#xA;        batch_loss = ...  # compute batch loss&#xA;        list_of_batch_losses.append(batch_loss)  # store loss in a list&#xA;&#xA;    epoch_loss = np.mean(list_of_batch_losses)&#xA;    training_loss.append(epoch_loss)&#xA;&#xA;    ...&#xA;&#xA;plt.plot(range(num_epochs), training_loss)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-07-31T16:36:22.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13706" PostTypeId="1" AcceptedAnswerId="13717" CreationDate="2019-07-31T17:06:18.147" Score="7" ViewCount="154" Body="&lt;p&gt;I wanted to ask about the methodology of testing the ML models against overfitting. Please note that I don't mean any overfitting reducing methods like regularisation, just a measure to judge whether a model has overfitting problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently developing a framework for tuning models (features, hyperparameters) based on evolutionary algorithms. And the problem that I face is the lack of a good method to judge if the model overfits before using the test set. I encountered the cases where the model that was good on both training and validation sets, behaved poorly on the test set for both randomized and not randomized training and validation splits. I used k-fold cross-validation with additionally estimating the standard deviation of all folds results (the smaller deviation means better model), but, still, it doesn't work as expected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summing up, I usually don't see a correlation (or a very poor one) between training, validation and k-fold errors with test errors. In other words, tuning the model to obtain lower values of any of the above mentioned measures usually does not mean lowering the test error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I ask you, how in practice you test your models? And maybe there are some new methods not mentioned in typical ML books?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2019-07-31T22:21:08.833" LastActivityDate="2020-04-08T23:16:47.113" Title="What is the best measure against overfitting?" Tags="&lt;machine-learning&gt;&lt;overfitting&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13707" PostTypeId="5" CreationDate="2019-07-31T18:00:24.837" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AI_box&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AI_box&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-31T18:00:24.837" LastActivityDate="2019-07-31T18:00:24.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13708" PostTypeId="4" CreationDate="2019-07-31T18:00:24.837" Score="0" Body="A hypothetical artificial superintelligence is kept in a virtual prison with limited means of affecting the external world.  Can the AI hack its way out or trick its human keepers into releasing it?" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-31T18:00:24.837" LastActivityDate="2019-07-31T18:00:24.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13709" PostTypeId="1" CreationDate="2019-07-31T19:08:57.443" Score="2" ViewCount="46" Body="&lt;p&gt;The problem I am trying to attack is a predator-prey pursuit problem. There are multiple predators that pursue multiple preys and preys tried to evade predators. I am trying to solve a simplified version - one predator tries to catch a static prey on a plane. There is bunch of literature on the above problem when predators and preys are on the grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody suggest articles/code where such problem is solved on a continuous plane? I am looking at continuous state space, discrete action space (predator can turn left 10 degrees, go straight, turn right 10 degrees, runs at constant speed), and discrete time. MountainCar is one dimensional version (car is predator and flag is prey) and DQN works fine. However, when I tried DQN on two dimensional plane the training become very slow (I guess dimensionality curse).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second question concerns the definition of states and reward. In my case I consider angle between predator heading vector and vector between the predator and prey positions. Reward is the change in distance between predator and prey, 10 when prey is captured, and -10 when predator gets too far from the prey. Is this reasonable? I already asked similar question before and with the help of @Neil Slater was able to refine reward and state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The third question concerns when to update train network to target network. At each episode? Or only when prey is caught? Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last question I have is about the network structure: activation functions and regularization. Currently I am using two tanh hidden layers and linear output with l2 and dropout. Can anybody share some insights?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="27472" LastActivityDate="2019-07-31T19:08:57.443" Title="How to solve optimal control problem with reinforcement learning" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13710" PostTypeId="1" CreationDate="2019-07-31T19:13:40.010" Score="2" ViewCount="43" Body="&lt;p&gt;This is a follow-up question from my previous question &lt;a href=&quot;https://ai.stackexchange.com/questions/13560&quot;&gt;here&lt;/a&gt;. I'm new to ML/DL, and one thing I need to do is to use a machine or deep learning video attention model which as the name suggests, can tag which parts of a video is probably more interesting and absorbs more viewer attention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we have an available model to do that? If not, how to do it?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-07-31T22:33:42.550" LastActivityDate="2019-07-31T22:33:42.550" Title="How do I tag the most interesting parts of a video?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;keras&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13711" PostTypeId="2" ParentId="13693" CreationDate="2019-07-31T19:13:48.063" Score="0" Body="&lt;p&gt;Approaches I have seen/used: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 1:&lt;/strong&gt; The angle is in between 2 vectors of some form, in which case, &lt;span class=&quot;math-container&quot;&gt;$[-\pi, 0)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[0, \pi)$&lt;/span&gt; are equivalent. (In vector space there always exists two angles between vectors: &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1-\theta$&lt;/span&gt; )  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use the sigmoid &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; function to put it between 0 and 1, and then scale to &lt;span class=&quot;math-container&quot;&gt;$[0, \pi)$&lt;/span&gt;. Since you arent modeling the whole rotation in this case, this works decently because the 2 ends dont have to be modeled as equivalent  &lt;/li&gt;&#xA;&lt;li&gt;Use cosine similarity, learn 2 vector representations and use &lt;span class=&quot;math-container&quot;&gt;$cos\ \theta = \frac{v_1^T v_2}{||v_1||*||v_2||}$&lt;/span&gt; and then you could use the arccos.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 2:&lt;/strong&gt; Wanting to learn an angle on the unit circle, meaning &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$1-\theta$&lt;/span&gt; is very important and so your bound is &lt;span class=&quot;math-container&quot;&gt;$[-\pi, \pi)$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a sinusoidal activation and multiply by &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (ex: &lt;span class=&quot;math-container&quot;&gt;$z=\pi*sin(x)$&lt;/span&gt;) This way you model the periodic nature of circling the unit circle (the activation eventually roates backs to itself continuously)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-07-31T19:13:48.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13712" PostTypeId="2" ParentId="10888" CreationDate="2019-07-31T19:34:04.260" Score="1" Body="&lt;p&gt;Reinforcement Learning experiments are empirical and you want to be able to reproduce your experiments. Random numbers are generated off a seed; with the seed function you are fixing the seed so the RNG function produces the same sequence of random numbers. &lt;/p&gt;&#xA;" OwnerUserId="27575" LastActivityDate="2019-07-31T19:34:04.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13713" PostTypeId="5" CreationDate="2019-07-31T20:32:29.843" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Computational_complexity&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-31T20:32:29.843" LastActivityDate="2019-07-31T20:32:29.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13714" PostTypeId="4" CreationDate="2019-07-31T20:32:29.843" Score="0" Body="The amount of resources required to run a given algorithm in relation to a given task.   " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-07-31T20:32:29.843" LastActivityDate="2019-07-31T20:32:29.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13717" PostTypeId="2" ParentId="13706" CreationDate="2019-07-31T21:47:49.937" Score="5" Body="&lt;h3&gt;tl;dr&lt;/h3&gt;&#xA;&lt;p&gt;The safest method I've found is to use &lt;strong&gt;cross-validation&lt;/strong&gt; for hyperparameter selection &lt;strong&gt;and a hold-out test set&lt;/strong&gt; for a final evaluation.&lt;/p&gt;&#xA;&lt;h3&gt;Why this isn't working for you...&lt;/h3&gt;&#xA;&lt;p&gt;In your case, I suspect you're either running a &lt;strong&gt;large number of iterations during for hyperparameter selection&lt;/strong&gt; or you have a fairly &lt;strong&gt;small dataset&lt;/strong&gt; (or even a combination of both). If you can't find more data or use a larger dataset, I'd suggest limiting the &lt;em&gt;exhaustiveness&lt;/em&gt; of your hyperparameter selection phase. If you run the process enough times the model is bound to overfit on the validation sets.&lt;/p&gt;&#xA;&lt;p&gt;Note that there is &lt;strong&gt;no guaranteed safe&lt;/strong&gt; way of detecting overfitting before the test set!&lt;/p&gt;&#xA;&lt;h3&gt;Why I consider this strategy to be the safest&lt;/h3&gt;&#xA;&lt;p&gt;There are two different types of overfitting you need to be able to detect.&lt;/p&gt;&#xA;&lt;p&gt;The first is the most straightforward: &lt;strong&gt;overfitting on the training set&lt;/strong&gt;. This means that the model has &lt;strong&gt;memorized&lt;/strong&gt; the training set and can't generalize beyond that. If the test set even slightly differs from the training set (which is the case in most real-world problems), then the model will perform worse on it than on the training set. This is simple to detect and, in fact, the only thing you need to catch this is a test set!&lt;/p&gt;&#xA;&lt;p&gt;The second type of overfitting you need to detect is &lt;strong&gt;on the test set&lt;/strong&gt;. Imagine you have a model and you make an exhaustive hyperparameter selection using a test set. The you evaluate on the same test set. By doing this, you have adjusted your hyperparameters to achieve the best score &lt;strong&gt;for the specific test set&lt;/strong&gt;. These hyperparameters are thus overfit to that test set even though the samples from that set were never seen during training. This is possible because during the iterative hyperparameter selection process, you have &lt;strong&gt;passed information about your test set to your model&lt;/strong&gt;!&lt;/p&gt;&#xA;&lt;p&gt;This is much harder to detect. In fact the only way to do so is to split the original data into three parts: the training, the validation and the test sets. The first is used for training the model, the second for hyperparameter selection and the final is used &lt;strong&gt;only once&lt;/strong&gt; for a final evaluation. If your model has overfit (i.e. test performance is worse than training and validation) you need to &lt;strong&gt;start from scratch&lt;/strong&gt;. Shuffle your data, split it again and repeat the process. This is usually referred to as a &lt;a href=&quot;https://www.kdnuggets.com/wp-content/uploads/dataiku-holdout-strategy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;hold-out strategy&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To make the model even less prone to overfitting, you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-validation_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-validation&lt;/a&gt; instead of a hold-out validation set. Because your model now has to be trained on &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; slightly different training sets and evaluated on &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; completely different validation sets, it is &lt;strong&gt;much harder&lt;/strong&gt; for it to overfit on the validation sets (because it needs to &lt;em&gt;fool&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; different validation sets instead of one).&lt;/p&gt;&#xA;&lt;h3&gt;Cases where this might not be applicable&lt;/h3&gt;&#xA;&lt;p&gt;Depending on the circumstances, it might not be practical to apply both of these techniques.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Cross-validation is pretty robust regarding overfitting but it imposes a &lt;strong&gt;computational burden&lt;/strong&gt; to your process as it requires multiple trainings of the same model. This obviously isn't practical for computationally expensive models (e.g. image classifiers). In this case use a hold-out strategy as mentioned previously.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using a hold-out test set means that you are reducing the size of your training set, which might actually make your model &lt;strong&gt;more prone to overfitting&lt;/strong&gt; (i.e. have a higher-variance) for &lt;strong&gt;small datasets&lt;/strong&gt;. In this case (if your model is practically untrainable due to the small size) you can resort to cross-validation, &lt;strong&gt;but you risk overfitting on the validation set&lt;/strong&gt; and not having any way to detect it.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;How to combat overfitting&lt;/h3&gt;&#xA;&lt;p&gt;Since it is fairly related I'll post a link to an &lt;a href=&quot;https://stats.stackexchange.com/a/365806/119015&quot;&gt;answer on how to combat overfitting&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-08T23:16:47.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13718" PostTypeId="2" ParentId="6669" CreationDate="2019-07-31T22:51:25.560" Score="0" Body="&lt;p&gt;As epsilon is throttled down networks 1 and 2 can freely specialize to producing tic-tac-toe's well-known non-losing behaviour against quasi-perfect adversaries without encoding any non-losing or winning behaviour against random (in other words, bad) adversaries.&#xA;I suggest while training network 1 (1st mover) and reducing epsilon-1 you keep the 2nd mover's epsilon-2 to values distinctly above 0, indeed why not fixed. Vice-versa for training the 2nd mover.&lt;/p&gt;&#xA;" OwnerUserId="27580" LastActivityDate="2019-07-31T22:51:25.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13720" PostTypeId="1" AcceptedAnswerId="27364" CreationDate="2019-08-01T10:54:38.090" Score="2" ViewCount="58" Body="&lt;p&gt;I'm programming my work with python, and I have a mesh and I want to extract 3d descriptors and feature points from it( trying to work on multi-scale strategy) , to visualize them later on the mesh,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm asking about, is references, guidelines, anything which could benefit me with this situation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main work I'm trying to do, is to reach matching stage, where I could find one-to-one correspondence.&lt;/p&gt;&#xA;" OwnerUserId="27587" LastActivityDate="2021-04-16T10:38:34.180" Title="Extracting Descriptors and feature points for 3d mesh" Tags="&lt;image-recognition&gt;&lt;python&gt;&lt;computer-vision&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13722" PostTypeId="1" AcceptedAnswerId="13731" CreationDate="2019-08-01T19:41:38.463" Score="1" ViewCount="48" Body="&lt;p&gt;I've been trying out a simple neural network on the fashion_mnist dataset using keras. Regarding normalization, I've watched &lt;a href=&quot;https://www.youtube.com/watch?reload=9&amp;amp;v=UIp2CMI0748&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; explaining why it's necessary to normalize input features, but the explanation covers the case &lt;strong&gt;when input features have different scales&lt;/strong&gt;. The logic is, say there are only two features - then if the range of one of them is much larger than that of the other, the gradient descent steps will stagger along slowly towards the minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm doing a different course on implementing neural networks and am currently studying the following example - the input features are pixel values ranging from 0 to 255, the total number of features (pixels) is 576 and we're supposed to classify images into one of ten classes. Here's the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;(Xtrain, ytrain) ,  (Xtest, ytest) = tf.keras.datasets.fashion_mnist.load_data()&#xA;&#xA;Xtrain_norm = Xtrain.copy()/255.0&#xA;Xtest_norm = Xtest.copy()/255.0&#xA;&#xA;model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),&#xA;                                    tf.keras.layers.Dense(128, activation=&quot;relu&quot;),&#xA;                                    tf.keras.layers.Dense(10, activation=&quot;softmax&quot;)])&#xA;&#xA;model.compile(optimizer = &quot;adam&quot;, loss = &quot;sparse_categorical_crossentropy&quot;)&#xA;model.fit(Xtrain_norm, ytrain, epochs=5)&#xA;model.evaluate(Xtest_norm, ytest)&#xA;------------------------------------OUTPUT------------------------------------&#xA;Epoch 1/5&#xA;60000/60000 [==============================] - 9s 145us/sample - loss: 0.5012&#xA;Epoch 2/5&#xA;60000/60000 [==============================] - 7s 123us/sample - loss: 0.3798&#xA;Epoch 3/5&#xA;60000/60000 [==============================] - 7s 123us/sample - loss: 0.3412&#xA;Epoch 4/5&#xA;60000/60000 [==============================] - 7s 123us/sample - loss: 0.3182&#xA;Epoch 5/5&#xA;60000/60000 [==============================] - 7s 124us/sample - loss: 0.2966&#xA;10000/10000 [==============================] - 1s 109us/sample - loss: 0.3385&#xA;0.3384787309527397&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So far, so good. Note that, as advised in the course, I've rescaled all inputs by dividing by 255. Next, I ran without any rescaling:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;(Xtrain, ytrain) ,  (Xtest, ytest) = tf.keras.datasets.fashion_mnist.load_data()&#xA;&#xA;model2 = tf.keras.models.Sequential([tf.keras.layers.Flatten(),&#xA;                                    tf.keras.layers.Dense(128, activation=&quot;relu&quot;),&#xA;                                    tf.keras.layers.Dense(10, activation=&quot;softmax&quot;)])&#xA;&#xA;model2.compile(optimizer = &quot;adam&quot;, loss = &quot;sparse_categorical_crossentropy&quot;)&#xA;model2.fit(Xtrain, ytrain, epochs=5)&#xA;model2.evaluate(Xtest, ytest)&#xA;------------------------------------OUTPUT------------------------------------&#xA;Epoch 1/5&#xA;60000/60000 [==============================] - 9s 158us/sample - loss: 13.0456&#xA;Epoch 2/5&#xA;60000/60000 [==============================] - 8s 137us/sample - loss: 13.0127&#xA;Epoch 3/5&#xA;60000/60000 [==============================] - 8s 140us/sample - loss: 12.9553&#xA;Epoch 4/5&#xA;60000/60000 [==============================] - 9s 144us/sample - loss: 12.9172&#xA;Epoch 5/5&#xA;60000/60000 [==============================] - 9s 142us/sample - loss: 12.9154&#xA;10000/10000 [==============================] - 1s 121us/sample - loss: 12.9235&#xA;12.923488986206054&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So somehow rescaling does make a difference? Does that mean if I further reduce the scale, the performance will improve? Worth trying out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;(Xtrain, ytrain) ,  (Xtest, ytest) = tf.keras.datasets.fashion_mnist.load_data()&#xA;&#xA;Xtrain_norm = Xtrain.copy()/1000.0&#xA;Xtest_norm = Xtest.copy()/1000.0&#xA;&#xA;model3 = tf.keras.models.Sequential([tf.keras.layers.Flatten(),&#xA;                                    tf.keras.layers.Dense(128, activation=&quot;relu&quot;),&#xA;                                    tf.keras.layers.Dense(10, activation=&quot;softmax&quot;)])&#xA;&#xA;model3.compile(optimizer = &quot;adam&quot;, loss = &quot;sparse_categorical_crossentropy&quot;)&#xA;model3.fit(Xtrain_norm, ytrain, epochs=5)&#xA;model3.evaluate(Xtest_norm, ytest)&#xA;------------------------------------OUTPUT------------------------------------&#xA;Epoch 1/5&#xA;60000/60000 [==============================] - 9s 158us/sample - loss: 0.5428&#xA;Epoch 2/5&#xA;60000/60000 [==============================] - 9s 147us/sample - loss: 0.4010&#xA;Epoch 3/5&#xA;60000/60000 [==============================] - 8s 141us/sample - loss: 0.3587&#xA;Epoch 4/5&#xA;60000/60000 [==============================] - 9s 144us/sample - loss: 0.3322&#xA;Epoch 5/5&#xA;60000/60000 [==============================] - 8s 138us/sample - loss: 0.3120&#xA;10000/10000 [==============================] - 1s 133us/sample - loss: 0.3718&#xA;0.37176641924381254&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Nope. I divided by 1000 this time and the performance seems worse than the first model. So I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is it necessary to rescale? I understand rescaling when different features are of different scales - that will lead to a skewed surface of the cost function in parameter space. And &lt;em&gt;even then&lt;/em&gt;, as I understand from the linked video, the problem has to do with &lt;strong&gt;slow learning (convergence)&lt;/strong&gt; and &lt;strong&gt;not high loss/inaccuracy&lt;/strong&gt;.  In this case, ALL the input features had the same scale. I'd assume the model would automatically adjust the scale of the weights and there would be no adverse effect on the loss. So why is the loss so high for the non-scaled case?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the answer has anything to do with the magnitude of the inputs, why does further scaling down of the inputs lead to worse performance?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does any of this have anything to do with the nature of the sparse categorical crossentropy loss, or the ReLU activation function? I'm very confused.&lt;/p&gt;&#xA;" OwnerUserId="27548" LastActivityDate="2019-08-02T06:58:47.267" Title="Effect of rescaling of inputs on loss for a simple neural network" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13723" PostTypeId="2" ParentId="13694" CreationDate="2019-08-01T19:48:49.790" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;Not only is it 100% ok, it's the process.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You may be surprised to know that even mathematicians struggle with mathematics, both the proofs they are working on, and the proofs of their colleagues.  Some thinkers are so far ahead of the curve, very few understand what they're stating until generations later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main thing is to keep with it.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-08-01T19:48:49.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13725" PostTypeId="1" CreationDate="2019-08-02T00:47:19.400" Score="7" ViewCount="369" Body="&lt;p&gt;I came across a comment recently &quot;reads like sentences strung together with no logic.&quot;  But is this even possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sentences can be strung together randomly if the selection process is random. (Random sentences in a random sequence.) Stochasticity does not seem logical&amp;mdash;it's a probability distribution, not based on sequence or causality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;but&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That stochastic process is part of an algorithm, which is a set of instructions that must be valid for the program to compute.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So which is it?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Is randomness anti-logical?&lt;/strong&gt;  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;strong&gt;Some definitions of computational logic:&lt;/strong&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;The arrangement of circuit elements (as in a computer) needed for computation&#xA;also: the circuits themselves &lt;a href=&quot;https://www.merriam-webster.com/dictionary/logic&quot; rel=&quot;noreferrer&quot;&gt;Merriam Websters&lt;/a&gt;  A system or set of principles underlying the arrangements of elements in a computer or electronic device so as to perform a specified task. Logical operations collectively. &lt;a href=&quot;https://www.google.com/search?q=logic%20definition&amp;amp;oq=logic%20definition&amp;amp;aqs=chrome..69i57j0l5.3583j1j7&amp;amp;sourceid=chrome&amp;amp;ie=UTF-8&quot; rel=&quot;noreferrer&quot;&gt;Google Dictionary&lt;/a&gt;&#xA;The system or principles underlying the representation of logical operations. Logical operations collectively, as performed by electronic or other devices.&#xA;&lt;br&gt;&lt;a href=&quot;https://www.oed.com/view/Entry/109788?rskey=w2r2Z9&amp;amp;result=1&amp;amp;isAdvanced=false#eid&quot; rel=&quot;noreferrer&quot;&gt;Oxford English Dictionary&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;sub&gt;Some definitions of randomness&lt;/sub&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;Being or relating to a set or to an element of a set each of whose elements has equal probability of occurrence. Lacking a definite plan, purpose, or pattern.&#xA;&lt;a href=&quot;https://www.merriam-webster.com/dictionary/random&quot; rel=&quot;noreferrer&quot;&gt;Merriam Websters&lt;/a&gt;. Made, done, happening, or chosen without method or conscious decision. &lt;a href=&quot;https://www.google.com/search?safe=off&amp;amp;ei=HYRDXd72ErCJggfWtQQ&amp;amp;q=definition%20of%20random&amp;amp;oq=definition%20of%20random&amp;amp;gs_l=psy-ab.3..0i70i249j0l9.3955.3955..4570...0.0..0.57.57.1......0....1..gws-wiz.......0i71.sIz5wR18Zj8&amp;amp;ved=0ahUKEwie-aC19-LjAhWwhOAKHdYaAQAQ4dUDCAo&amp;amp;uact=5&quot; rel=&quot;noreferrer&quot;&gt;Google Dictionary&lt;/a&gt; Having no definite aim or purpose; not sent or guided in a particular direction; made, done, occurring, etc., without method.  Seeming to be without purpose or direct relationship to a stimulus. &lt;a href=&quot;https://www.oed.com/view/Entry/157984#eid27053631&quot; rel=&quot;noreferrer&quot;&gt;Oxford English Dictionary&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-08-05T20:17:06.767" LastActivityDate="2020-05-09T22:23:25.707" Title="Is randomness anti-logical?" Tags="&lt;philosophy&gt;&lt;logic&gt;&lt;randomness&gt;" AnswerCount="6" CommentCount="7" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13726" PostTypeId="1" CreationDate="2019-08-02T00:50:30.260" Score="2" ViewCount="51" Body="&lt;p&gt;I am developing a Body Measurement extraction application, my current stage is able to extract the point clouds of human body in a standing posture, from every angles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to be able to recognize shoulders, neck point etc, my research seems to fall into following flows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Method A:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Obtain a lot of data, with labeled landmark points (shoulder left, shoulder right, neck line).&lt;/li&gt;&#xA;&lt;li&gt;Use PointCNN / PointNet++ to perform segmentation for each landmark.&lt;/li&gt;&#xA;&lt;li&gt;Once the landmarks are extracted, use Open3D / Point Cloud Library convex hull to obtain the measurement along point clouds.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Method A seems straight forward, but might depends on the quality of point cloud, especially 3rd step. &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Method B:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Obtain a lot of data, with labeled landmarks and also measurement of shoulder length, chest circumference etc.&lt;/li&gt;&#xA;&lt;li&gt;We first train the network to identify landmarks,&lt;/li&gt;&#xA;&lt;li&gt;Then from landmark, we record the distance to the next nearest point, train the network to obtain the measurement that we want.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Method C:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Obtain a lot of data, with measurement of shoulder length, chest circumference etc ONLY.&lt;/li&gt;&#xA;&lt;li&gt;Pick a random point, and we record the distance to the next nearest point, train the network to obtain the measurement that we want.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How much data needed for this kind of learning?&lt;/li&gt;&#xA;&lt;li&gt;If I obtain training data from somewhere online, and later validate using my own scanned data, will that valid?&lt;/li&gt;&#xA;&lt;li&gt;Which method makes more sense?&lt;/li&gt;&#xA;&lt;li&gt;Which existing problem with solution is similar to my case? (Facial recognition?) That I can refer to it to solve my problem.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is technically my first machine learning project, so please bear with me if my questions seems too silly.&lt;/p&gt;&#xA;" OwnerUserId="15755" LastEditorUserId="15755" LastEditDate="2019-08-02T10:51:28.930" LastActivityDate="2019-08-02T10:51:28.930" Title="Pipeline to Estimate Measurement of Human Body Point Cloud" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13727" PostTypeId="5" CreationDate="2019-08-02T00:50:56.950" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Stochastic&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-08-02T00:50:56.950" LastActivityDate="2019-08-02T00:50:56.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13728" PostTypeId="4" CreationDate="2019-08-02T00:50:56.950" Score="0" Body="Use for questions about the concept of randomness, or use of stochasticity in decision making algorithms.  " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-08-02T00:50:56.950" LastActivityDate="2019-08-02T00:50:56.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13729" PostTypeId="2" ParentId="13725" CreationDate="2019-08-02T04:51:38.670" Score="3" Body="&lt;p&gt;I think the answer here lies in that the dictionary definition of randomness you have is not the one used in statistics, ML, or mathematics. We define randomness to mean there exists a distribution with generally greater than 0 uncertainty.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on who you talk to, we live in a random universe (the way we define quantum mechanics depends on a wave function (essentially a probability distribution)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why if a sequence is drawn from a distribution is it illogical? First, even as humans we can make a strong argument that what we say is &lt;em&gt;random&lt;/em&gt;. I mean we speak to convey some form of message or context, but there exists multiple ways to deliver this, but we choose a single one. Our brains inherently model &lt;span class=&quot;math-container&quot;&gt;$p(\vec w|c)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\vec w$&lt;/span&gt; is the sequence and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is our context or message we want to convey.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Takeaway:&lt;/strong&gt; Generating a sequence in an ergodic or uniform manner would be illogical, but that is not what is being modeled or done in practice. Normally its drawn from some complex distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sidenote:&lt;/strong&gt; My above claim could make it seem that being uniformly random implicates something illogical, and I want to emphasize that is not the case. It is domain to domain, sometimes that is the &lt;strong&gt;most&lt;/strong&gt; logical solution, just in the case of sentence generation it normally isnt. I would define a logical algorithm as one that given the information at hand acts in a sensible manner towards achieving some goal, and so if something purely random does that, I don't see the problem.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-02T04:51:38.670" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13731" PostTypeId="2" ParentId="13722" CreationDate="2019-08-02T06:58:47.267" Score="1" Body="&lt;p&gt;One of the factors that rescaling gives a better result is on your weight initialization of the model. Your experiments run on the same model that using the same weight initialization (&lt;code&gt;tf.keras.dense&lt;/code&gt; using &lt;a href=&quot;https://keras.io/layers/core/#dense&quot; rel=&quot;nofollow noreferrer&quot;&gt;Glorot Uniform&lt;/a&gt;). If your input is too big or too small the model needs time to adjust the weight.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is it necessary to rescale? I understand rescaling when different features are of different scales - that will lead to a skewed surface of the cost function in parameter space. And even then, as I understand from the linked video, the problem has to do with slow learning (convergence) and not high loss/inaccuracy. In this case, ALL the input features had the same scale. I'd assume the model would automatically adjust the scale of the weights and there would be no adverse effect on the loss. So why is the loss so high for the non-scaled case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Slow learning means you can't compare the loss of two models even at the same epoch. As the model using the same initial value of weights, the second model may need time (epoch) to adjust its weight to get the same result with the first model.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the answer has anything to do with the magnitude of the inputs, why does further scaling down of the inputs lead to worse performance?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It has the same reason, very small values of the inputs also lead to the difficulty to adjust the weight.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-08-02T06:58:47.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13732" PostTypeId="1" AcceptedAnswerId="13758" CreationDate="2019-08-02T08:16:11.057" Score="1" ViewCount="116" Body="&lt;p&gt;Some puzzle games have a unique solution that can be solved by deduction rather than guesswork (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Slitherlink&quot; rel=&quot;nofollow noreferrer&quot;&gt;Slitherlink&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Masyu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Masyu&lt;/a&gt;). Using a computer to solve this puzzle it's pretty easy, we can use a backtracking method to find the best solution in second (in general, the puzzle size is not too big).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to train a bot to solve this kind of puzzle by deduction?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think by train it to watch a previous step-by-step solution several times the bot can find some &lt;a href=&quot;https://en.wikipedia.org/wiki/Slitherlink#Rules_for_squares_with_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;implicit rules/patterns&lt;/a&gt; to solve a specific puzzle. Is this possible? are there any references for this method?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-08-03T17:49:37.877" Title="Create an AI to solve a puzzle (by deduction)" Tags="&lt;ai-design&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13733" PostTypeId="2" ParentId="13725" CreationDate="2019-08-02T08:24:30.547" Score="3" Body="&lt;p&gt;I might misunderstand your question, but there seem to be different levels of logic at play here.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Computing logic, whereby any computational process is based on processor logic. In this case, any computing is involving logic, as boolean logic drives any processing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Linguistic logic, where there is a logic in the sequencing of sentences within a text. A random collection of sentences is not a text, as there need to be certain principles behind the structure to make it a narrative.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While you can easily generate a sequence of random sentences, they will not mean anything; there won't be any logic behind selecting a particular sentence to follow on from another one. So this is &lt;em&gt;linguistic logic&lt;/em&gt; rather than &lt;em&gt;processing logic&lt;/em&gt;. Note that where the linguistic logic is makes it a bit vague: I can read a randomly selected sequence of sentences and ascribe meaning to it by building a mental model that treats it as a logically constructed text. This principle is what made &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELIZA&lt;/a&gt; so successful: even though the program's answers were based on simple pattern matching rules with no understanding, many users assumed there was logic/meaning behind it and interpreted it as such, papering over the cracks in the conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary: there is logic involved in random sentence combining, but it is the low-level computing logic, not the higher-level linguistic interpretative logic, which is generally absent from randomly generated data.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-08-02T08:24:30.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13734" PostTypeId="2" ParentId="13644" CreationDate="2019-08-02T09:26:33.913" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What I want to achieve is incremental training. So, as soon as I get new data, I can further train my already trained model and I don't have to retrain everything.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Learning without forgetting is one of the methods to solve multitask learning. If your model trained to solve problem A and then after sometimes you need your model to solve new problem B without forgetting the problem A (the model still good to solve the problem A), then you need this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transfer learning is a method to use a trained model to solve another task (and may forget the original task). For example, you use a model that originally trained to classify cat or dog to a new task that trying to classify goat or cow. You use this in hopes of speeding up your training process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your new data has the same task with the old data, you don't need to use multitask learning method. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if your model trained with 50 images to detect an apple in the image, and then you get new 100 images to detect an apple then you just need to continue your train (incremental learning). In this case, you need (to save) the latest parameter of your model after trained (latest learning rate value, epoch, etc.), if you have it then you just need to run your training again (continue the epoch).&lt;/li&gt;&#xA;&lt;li&gt;if your model trained with 100 images to detect an apple in the image, and then you get new 100 images to train your model to detect an orange and &lt;strong&gt;you don't care if your model will give a bad result to detect an apple&lt;/strong&gt;, then you can use transfer learning. You may freeze a few first layers as &quot;extractor&quot; and initialize a new layer at the end.&lt;/li&gt;&#xA;&lt;li&gt;if your model trained with 100 images to detect an apple in the image, and then you get new 100 images to detect an orange and your model must good to detect both apple and orange in an image, then you use multitask learning. The easiest method is to train your model with the apple+orange image, but you can also use another approach like proposed in Learning without Forgetting paper.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2019-08-05T04:02:21.260" LastActivityDate="2019-08-05T04:02:21.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13735" PostTypeId="1" CreationDate="2019-08-02T10:19:20.153" Score="2" ViewCount="25" Body="&lt;p&gt;so I have this dataset of images of people sitting in a restaurant.&lt;br&gt;&#xA;I've annotated about 300 images with an average of 30 instances of &quot;person&quot; per image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm wondering if I should have annotated only one (or just a few) person per image and processed way more images ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've successfully trained an SSD network with only one class, but I'm still wondering if I should have gone the other way...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone got input on that ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers.&lt;/p&gt;&#xA;" OwnerUserId="27602" LastActivityDate="2019-08-02T10:19:20.153" Title="Train detector : 300 images with 30 objects or 9000 images with one?" Tags="&lt;training&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13737" PostTypeId="1" CreationDate="2019-08-02T12:46:41.010" Score="3" ViewCount="94" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ETBtK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ETBtKl.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This page is from Pattern Recognition by Bishop. What is the explanation for equations 1.70. 1.71 and 1.72?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The distributions are all Gaussian.&lt;/p&gt;&#xA;" OwnerUserId="27422" LastEditorUserId="4709" LastEditDate="2020-01-03T20:38:06.130" LastActivityDate="2021-01-27T22:08:30.853" Title="How are mean and variance calculated in Bayesian curve fitting?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-27T23:37:32.493" ContentLicense="CC BY-SA 4.0" />
  <row Id="13738" PostTypeId="1" AcceptedAnswerId="13745" CreationDate="2019-08-02T12:57:49.087" Score="4" ViewCount="583" Body="&lt;p&gt;I have recently discovered &lt;a href=&quot;https://arxiv.org/pdf/1512.00567.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;asymmetric convolution layers&lt;/a&gt; in deep learning architectures, a concept which seems very similar to &lt;a href=&quot;https://arxiv.org/pdf/1610.02357.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;depthwise separable convolutions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are they really the same concept with different names? If not, where is the difference? To make it concrete, what would each one look like if applied to a 128x128 image with 3 input channels (say R,G,B) and 8 output channels?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: I cross-posted this from stackoverflow, since this kind of theoretical question is maybe better suited here. Hoping it is OK...&lt;/p&gt;&#xA;" OwnerUserId="27606" LastEditorUserId="2444" LastEditDate="2019-08-02T14:28:02.100" LastActivityDate="2019-08-02T16:56:55.070" Title="What is the difference between asymmetric and depthwise separable convolution?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13739" PostTypeId="1" AcceptedAnswerId="13740" CreationDate="2019-08-02T13:27:07.657" Score="2" ViewCount="32" Body="&lt;p&gt;I'm reading the book Pattern Recognition and Machine Learning by Bishop, specifically the intro where he covers polynomial regression model. In short, let's say we generate &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; data points using the function &lt;span class=&quot;math-container&quot;&gt;$\sin(2\pi x)$&lt;/span&gt; and add some gaussian random noise to each observation. Now we pretend not knowing the generating function and try to fit a polynomial model to these points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we increase the degree of the polynomial, it goes from underfitting (&lt;span class=&quot;math-container&quot;&gt;$d=1,2$&lt;/span&gt;) to overfitting (&lt;span class=&quot;math-container&quot;&gt;$d=10$&lt;/span&gt;). One thing the author notes is that the higher the degree of the polynomial, the higher the values of the coefficients (parameters). This is my first doubt: &lt;strong&gt;why does the size of the coefficients increase with the polynomial degree? And why is the size of the parameters related to overfitting?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, he states that even for degree &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, if we get sufficiently many data points (say &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;), then the high degree polynomial will no longer overfit the data and should have comparatively better generalization performance. Second doubt: &lt;strong&gt;Why is this so?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="27548" LastActivityDate="2019-08-02T16:51:37.197" Title="Relation between size of parameters and complexity of model with overfitting" Tags="&lt;overfitting&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13740" PostTypeId="2" ParentId="13739" CreationDate="2019-08-02T14:31:20.190" Score="1" Body="&lt;p&gt;Regarding your first question, this is domain/task specific, and not always the case. My guess of why it happens in your situation (You did not specify your domain, so ill assume it's sometimes outside &lt;span class=&quot;math-container&quot;&gt;$(-1,1)$&lt;/span&gt;) is that higher order polynomials increase much faster than lower order ones so it may fall under the trap where the coefficients have to be large to handle it. (e.g. if &lt;span class=&quot;math-container&quot;&gt;$x=2.0$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$ 2^5*x^2 = x^7 \rightarrow 32*x^2 = x^7$&lt;/span&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to answer why more points will cause less overfitting is sheerly because you have more equations. Your loss is generally some form of average distance accross points and predictions. Given &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; points I can always find an &lt;span class=&quot;math-container&quot;&gt;$N-1$&lt;/span&gt; order polynomial that fits it exactly. Instead of a formal proof I will provide a form in which you can easily generate this polynomial from &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt; pairs&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ f(x) = \sum_{i=1}^{N}y_i\prod_{j=1\atop j \ != i}^N\frac{(x-x_j)}{(x_i-x_j)}$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, for each &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; it will zero out all terms in the sum except one where the product become 1 making &lt;span class=&quot;math-container&quot;&gt;$f(x_i) = y_i$&lt;/span&gt; by construction. So adding more points will prevent whatever optimization process you are learning to directly solve each point exactly, but instead try to find an algorithm that best generalizes to each of the points.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-02T14:31:20.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13741" PostTypeId="1" CreationDate="2019-08-02T14:50:06.933" Score="2" ViewCount="208" Body="&lt;p&gt;I am trying to train my model using LTSM layer in Keras (python). I have some problems regarding the data representation and feeding it into the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data is 184 XY coodinates encoded as a numpy array with two dimensions: one corresponding to the X or Y and second is every single point of X or Y. Shape of a single spectrum is (2, 70). Altogether, my data has a dimension of (184, 2, 70).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The label set is an array of 8 elements which describes the percentage distribution of some 8 features which are describing XY. The shape of an output is (184, 8).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I train using the time series for each XY pair and compare it to the corresponding label set? Different XY data show similar features to each other that is why it is important to use all 184 sample for the training. What would be the best approach to handle this problem? Below I show the schematics of my data and model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input: (184, 2, 70) (number of XY, X / Y, points)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output: (184, 8) (number of XY, predictions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I look forward for some ideas!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZkR9b.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZkR9b.jpg&quot; alt=&quot;Data representation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27608" LastEditorUserId="27608" LastEditDate="2019-08-05T09:12:29.980" LastActivityDate="2021-05-26T15:01:20.103" Title="How to train a LSTM model with multi dimensional data" Tags="&lt;python&gt;&lt;keras&gt;&lt;time-series&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13742" PostTypeId="2" ParentId="13600" CreationDate="2019-08-02T15:34:16.027" Score="1" Body="&lt;p&gt;This is a somewhat provocative view, so be warned (and please don't down-vote this if you feel provoked by it!):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &quot;old days&quot;, when information retrieval (IR) was one of the main tasks in NLP, several categories of words were ignored as &lt;em&gt;stopwords&lt;/em&gt;; conjunctions, determiners, prepositions, etc. These function words do not carry meaning themselves, but organise the structure of sentences. Most IR algorithms worked on frequencies of individual words, and as functions words are very frequent (&lt;em&gt;of&lt;/em&gt; and &lt;em&gt;the&lt;/em&gt; are the two most frequent English words) and don't mean anything by themselves, they were ignored. This kept the index files small and didn't seem to influence the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you want to analyse sentences themselves, they are rather important. They are also useful for all sorts of other tasks where you are looking at sequences of words (eg part-of-speech tagging based on context). Similar for word embeddings: without function words you'd not have any meaningful context to work with. So, increasingly you would not ignore function words anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suspicion is, that punctuation is now in the 'stopword position': it's not too clear how it influences meaning, and is often inconsistent or redundant (obviously not in all cases). So you can probably treat it as 'noise' and get away with it for most applications. For example, looking at meanings of words, it probably doesn't matter that much whether the sentence they occurred in was a question or an exclamation. By removing punctuation (maybe apart from sentence-terminators), your model is a bit smaller and you don't lose much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since punctuation is purely a property of written language, we can generally get away without it, as we do in speech. A text without punctuation might be harder to read, because we're not used to it, but don't forget that some writing systems (Chinese, Egyptian hieroglyphics, ...) don't even have spaces between words &amp;mdash; and people can still use them without problems.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-08-02T15:34:16.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13743" PostTypeId="2" ParentId="13739" CreationDate="2019-08-02T16:44:45.180" Score="1" Body="&lt;p&gt;Size of the co-efficients will probably increase only upto a certain degree of polynomial. This is due to the fact you are using &lt;span class=&quot;math-container&quot;&gt;$sin(2\pi x)$&lt;/span&gt;, if you used &lt;span class=&quot;math-container&quot;&gt;$sin(4\pi x)$&lt;/span&gt; then the size of co-efficients will increase upto more degrees of polynomial. This can be seen when &lt;span class=&quot;math-container&quot;&gt;$sin(x)$&lt;/span&gt; is represented as series:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ sin(x) = \frac{x}{1!} - \frac{x^3}{3!} + \frac{x^5}{5!}....$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case &lt;span class=&quot;math-container&quot;&gt;$x \rightarrow 2\pi x$&lt;/span&gt; so in order to approximate it the higher order terms must have very high co-efficients which the denominator factorial terms cannot cancel out (only upto a certain point though) and hence for small orders like &lt;span class=&quot;math-container&quot;&gt;$N=10$&lt;/span&gt; (assume we have even terms in the series, since we are not dealing with mathematical definiteness, so even terms will cancel out or get cancelled out in some way), &lt;span class=&quot;math-container&quot;&gt;$10! = 3628800$&lt;/span&gt; whereas &lt;span class=&quot;math-container&quot;&gt;$(2\pi) ^{10} = 95410558$&lt;/span&gt; around 26 times greater. So you see till certain point the co-efficient values must increase for &lt;span class=&quot;math-container&quot;&gt;$sin(2\pi x)$&lt;/span&gt;. I think this answers both of your questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to your second question, &lt;strong&gt;in general loosely we can say, the ML algorithm you are using performs Polynomial Regression which means fitting a curve by adjusting parameters, in a way such that the distance between the points generated by your model, for a given input, is as close as possible to the real data&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is why does increasing data points gives better generalisation? What most people do not mention is now that you have a better generalisation of the function itself, by which I mean, if I give you 2 points (least number required as per &lt;a href=&quot;https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nyquist Sampling theorem&lt;/a&gt; to define a &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt; wave of certain frequency) from a &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt; curve, unless you know beforehand you cannot tell whether it was generated from a &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt;, but if I give you 100 points within the same time period (of a sine wave) you can easily guess the data must be generated from &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt;. Similarly, an ML algorithm cannot guess where the data is generated from when the number of data-points is less and tries to fit a model according to its best guess (minimum loss), but if you give a larger number of points it'll make better guess hence better generalisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think like this, you want to make a circle with rubber band around pins. Can you make it with 4-5 pins? You need at-least certain number of pins to make it look a circle. The rubber band here is your model.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2019-08-02T16:51:37.197" LastActivityDate="2019-08-02T16:51:37.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13744" PostTypeId="2" ParentId="13737" CreationDate="2019-08-02T16:50:06.250" Score="0" Body="&lt;p&gt;Since you only mention that their Gaussian, here is a screenshot depicting the relationship of gaussian conjugate priors  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JKbi8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JKbi8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took this from &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_prior&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;. If you check with what ever your likelihood and priors parameters are it probably all works out (you did not add these in the question)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a quick google, here's a link to someone going through the proof: &lt;a href=&quot;https://www.youtube.com/watch?v=MUhsT0U_nxY&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-02T16:50:06.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13745" PostTypeId="2" ParentId="13738" CreationDate="2019-08-02T16:56:55.070" Score="2" Body="&lt;p&gt;They are not the same thing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;asymmetric convolutions work by taking the x and y axes of the image separately. For example performing a convolution with an &lt;span class=&quot;math-container&quot;&gt;$(n \times 1)$&lt;/span&gt; kernel before one with a &lt;span class=&quot;math-container&quot;&gt;$(1 \times n)$&lt;/span&gt; kernel.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other-hand depth-wise separable convolutions separate the spatial and channel components of a 2D convolution. It will first perform the &lt;span class=&quot;math-container&quot;&gt;$(n \times n)$&lt;/span&gt; convolution on each channel separately (full kernel shape will be &lt;span class=&quot;math-container&quot;&gt;$(n \times n \times 1)$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$(n \times n \times k)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of channels in the previous layer) before doing a &lt;span class=&quot;math-container&quot;&gt;$(1 \times 1)$&lt;/span&gt; convolution to learn a relationship between the channels (full kernel size for that being &lt;span class=&quot;math-container&quot;&gt;$(1 \times 1 \times k)$&lt;/span&gt;)&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-02T16:56:55.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13746" PostTypeId="2" ParentId="13725" CreationDate="2019-08-02T17:09:47.413" Score="1" Body="&lt;p&gt;In certain games, random selection is the optimal strategy.  See: &lt;a href=&quot;https://en.wikipedia.org/wiki/Matching_pennies&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matching Pennies&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strategy is essentially a plan of action utilized to achieve a goal. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If random choice can be a strategy, it seems that it must be a form of logic, even if the nature of the stochastic process is counter to all forms of formal logic.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This seems paradoxical, in that the random strategy is to have no strategy (random choices.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-08-05T18:39:15.430" LastActivityDate="2019-08-05T18:39:15.430" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="13748" PostTypeId="1" CreationDate="2019-08-02T20:53:56.667" Score="1" ViewCount="16" Body="&lt;p&gt;I have an idea for an RNN which has no separate internal memory state only an output. But there is a gate in which tells the neural network whether the output will be acted out in the physical world or it will be an internal thought. (It would also store its last, say, 10 outputs, so that it can have a memory of some kind.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this would be quite realistic because human's either talk or think in an internal monologue, but don't do both. (It is hard to think and do things at the same time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I wonder how this gate will be activated. For example, when talking to someone familiar, this gate will be open, as you just say what's in your head. But for quiet contemplation time, this gate will be closed. And for thoughtful conversation, it will be open 50% of the time. So, I wonder if this gate would be controlled by the NN itself or be controlled from the environment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there would be social pressure involved when talking to someone to keep the gate open. And likewise when in a library or a quiet place to keep the gate closed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there are some models like this out there already?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2019-08-02T21:16:54.923" LastActivityDate="2019-08-02T21:16:54.923" Title="A gated neural network for internal thought?" Tags="&lt;ai-design&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13749" PostTypeId="2" ParentId="10623" CreationDate="2019-08-02T22:06:05.900" Score="12" Body="&lt;p&gt;Self-supervised learning is when you use some parts of the samples as labels for a task that requires a good degree of comprehension to be solved. I'll emphasize these two key points, before giving an example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Labels are extracted from the sample&lt;/em&gt;, so they can be generated automatically, with some very simple algorithm (maybe just random selection).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;The task requires understanding&lt;/em&gt;. This means that, in order to predict the output, the model has to extract some good patterns from the data, generating on the process a good representation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A very common case for semi-supervised learning takes place in natural language processing, when you need to solve a task but have few labeled data. In such cases, you need to learn a good representation or language model, so you take sentences and give your network self-supervision tasks like these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Ask the network to predict the next word in a sentence (which you know because you took it away).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mask a word and ask the network to predict which word goes there (which you know because you had to mask it).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Change the word for a random one (that probably doesn't make sense) and ask the network which word is wrong.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As you can see, these tasks are fairly simple to formulate and the labels are part of the same sample, but they require a certain understanding of the context to be solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it's always like this: alter your data in some way, generating the label in the process, and ask the model something related to that transformation. If the task requires enough understanding of the data, you'll have success.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-08-02T22:06:05.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13750" PostTypeId="1" CreationDate="2019-08-02T23:22:37.243" Score="2" ViewCount="55" Body="&lt;p&gt;I am looking into whether a neural network is appropriate to detect &quot;points of interest&quot; (POI) in a set of tuples (say length, and some sensor value). A POI is essentially a quick change in the value which doesn't follow the pattern. So if we have a linear increase in the sensor value and then it suddenly jumps by 200% that would be a POI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of the data I am working with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[(1,10),(2,11),(3,14),(5,24),(6.5,25), (7,26), (8,45)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this example lets say &quot;(3,14)&quot;, &quot;(5,24)&quot;, and &quot;(8,45)&quot; are points of interest. So I am trying to design a neural network which will detect these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have started by creating a Convolution 1D layer with a static input length of 500 elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a couple hidden layers I apply a sigmoid function which provides a list of 0s and 1s as output where 1s signify a POI in the set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of issues with this approach which I am trying to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a categorical loss function an output of &lt;code&gt;[1,0,0,1,0,0]&lt;/code&gt; for example would be seen as completely inaccurate if the expected output is &lt;code&gt;[0,1,0,0,1,0]&lt;/code&gt; whereas in reality that is fairly accurate since the predicted POIs are very close to the real POIs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what I am trying to do is find a loss function to optimize the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Binary Cross Entropy: I read this is good for classifying where inputs can belong to multiple classes. I tried this out thinking each POI is essentially a &quot;category&quot;. But this seems to not work and I assume it's because of what I noted above.&lt;/li&gt;&#xA;&lt;li&gt;Mean Absolute Error: This seems to have gotten slightly better results but after closer inspection it didn't seem very accurate and would mostly uniformly predict POIs on a set.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried a few others without much luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What loss function would be more appropriate for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other output I tried was instead of outputting 0s and 1s it should just return the indexes of the points of interest so say 3, 5 8. Would this be a better output?&lt;/p&gt;&#xA;" OwnerUserId="27615" LastActivityDate="2019-08-03T10:51:31.333" Title="What loss function is appropriate for finding &quot;points of interest&quot; in a array of x,y inputs" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13751" PostTypeId="1" CreationDate="2019-08-03T01:00:28.310" Score="3" ViewCount="94" Body="&lt;p&gt;I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like &lt;a href=&quot;https://www.researchgate.net/publication/281670427_Recurrent_Reinforcement_Learning_A_Hybrid_Approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://papers.nips.cc/paper/1953-reinforcement-learning-with-long-short-term-memory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. If I were to use a sequence of raw observations (or features) as an input to RNN for encoding the state of the system, I cannot change the weights of my network in the middle of the episode. Is that correct? Otherwise, the hidden state vectors will be different when the weights are changed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that mean that the use of RNN in RL has to store the entire episode before the weights can be changed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does then one take into account the hidden states in RNN for RL? Are there any good tutorials on RNN-RL?&lt;/p&gt;&#xA;" OwnerUserId="21509" LastEditorUserId="2444" LastEditDate="2019-08-03T10:49:49.777" LastActivityDate="2021-04-27T02:06:45.603" Title="How are the observations stored in the RNN that encodes the state?" Tags="&lt;reinforcement-learning&gt;&lt;recurrent-neural-networks&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13752" PostTypeId="2" ParentId="13750" CreationDate="2019-08-03T02:26:25.543" Score="1" Body="&lt;p&gt;You don't have to use machine learning to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Unify the scale of each data input (or each curve), such as normalize to &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; (not necessary).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the slope of each pair of points &lt;span class=&quot;math-container&quot;&gt;$\frac{(y2 - y1)}{(x2 - x1)}$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Set the threshold. Compare the difference between two adjacent slopes, the difference exceeds the threshold are marked as POI.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Isn't that simpler?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you must solve the problem with CNN, what I can think of is that you first collect (or draw) a bunch of curves, mark the POI in advance, and then feed it to the CNN model.&lt;/p&gt;&#xA;" OwnerUserId="27617" LastEditorUserId="2444" LastEditDate="2019-08-03T10:51:31.333" LastActivityDate="2019-08-03T10:51:31.333" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13755" PostTypeId="1" CreationDate="2019-08-03T14:53:51.103" Score="3" ViewCount="77" Body="&lt;p&gt;Taking out the weighting factor we can define focal loss as&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$FL(p) = -(1-p)^\gamma log(p) $$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the target probability. The idea being that single stage object detectors have a huge class imbalance between foreground and background (several orders of magnitude of difference), and this loss will down-scale all results that are positively classified compared to normal cross entropy (&lt;span class=&quot;math-container&quot;&gt;$CE(p) = -log(p)$&lt;/span&gt;) so that the optimization can then focus on the rest.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the general optimization scheme uses the gradient to find the direction with the steepest descent. There exists methodologies for adaption, momentum and etc but that is the general gist.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \theta \leftarrow \theta - \eta \nabla_\theta L $$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Focal losses gradient follows as so&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\dot {FL}(p) = \dot p [\gamma(1-p)^{\gamma -1} log(p) -\frac{(1-p)^\gamma}{p}]$$&lt;/span&gt; compared to the normal crossentropies loss of&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{CE}(p) = -\frac{\dot p}{p}$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can now rewrite these as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot{FL}(p) = (1-p)^\gamma \dot{CE}(p) + \gamma \dot p (1-p)^{\gamma -1} log(p)$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial term, given our optimization scheme will do what we (and the authors of &lt;a href=&quot;https://arxiv.org/abs/1708.02002&quot; rel=&quot;nofollow noreferrer&quot;&gt;the retinanet paper&lt;/a&gt;) want which is downscale the effect of the labels that are already well classified but the second term is slightly less interpretative and in parameter space and may cause an unwanted result. So my question is why not remove it and only use the gradient&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\dot L =  (1-p)^\gamma \dot{CE}(p)$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which given a &lt;span class=&quot;math-container&quot;&gt;$\gamma \in \mathbb{N}$&lt;/span&gt; produces a loss function&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ L(p) = -log(p) - \sum_{i=1}^\gamma {\gamma \choose i}\frac{(-p)^i}{i}$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary:&lt;/strong&gt; Is there a reason we make the loss adaptive and not the gradient in cases like focal loss? Does that second term add something useful?&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-06T23:05:30.470" Title="Does Retina-net's focal loss accomplish its goal?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13756" PostTypeId="1" CreationDate="2019-08-03T15:51:41.137" Score="2" ViewCount="19" Body="&lt;p&gt;I need to create an application that can detect if a person X entered as an input exists in an image set and return as output all the images in which the person X exists. The problem is that the pictures do not only contain people's faces, they also contain pictures taken from behind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use &lt;a href=&quot;https://azure.microsoft.com/en-us/services/cognitive-services/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft's cognitive services&lt;/a&gt;? If not, is there another solution to allow the realization of this application?&lt;/p&gt;&#xA;" OwnerUserId="22702" LastEditorUserId="2444" LastEditDate="2019-08-03T16:09:57.240" LastActivityDate="2019-08-03T16:09:57.240" Title="Can Microsoft's cognitive service find similar person in a set of images without using the face service?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;cognitive-science&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13757" PostTypeId="2" ParentId="2122" CreationDate="2019-08-03T17:26:30.057" Score="0" Body="&lt;p&gt;After almost three years the question is still relevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me add some too:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://deeplearning.net/datasets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Datasets&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The datasets from the above link can be used for benchmarking deep learning algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cs.stanford.edu/~acoates/stl10/&quot; rel=&quot;nofollow noreferrer&quot;&gt;STL-10 dataset&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An image dataset which is inspired by &lt;a href=&quot;http://www.cs.toronto.edu/~kriz/cifar.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIFAR-10 dataset&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16708" LastActivityDate="2019-08-03T17:26:30.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13758" PostTypeId="2" ParentId="13732" CreationDate="2019-08-03T17:49:37.877" Score="1" Body="&lt;p&gt;What you are trying to achieve sounds a lot like &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_logic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;inductive logic programming&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given an encoding of the known background knowledge and a set of examples represented as a logical database of facts, an ILP system will derive a hypothesised logic program which entails all the positive and none of the negative examples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="27624" LastActivityDate="2019-08-03T17:49:37.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13759" PostTypeId="2" ParentId="2122" CreationDate="2019-08-03T17:57:19.897" Score="1" Body="&lt;p&gt;If you want to solve a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiclass_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-class classification&lt;/a&gt; problem, you could use the famous &lt;a href=&quot;https://en.wikipedia.org/wiki/Iris_flower_data_set&quot; rel=&quot;nofollow noreferrer&quot;&gt;iris flower dataset&lt;/a&gt;, which was &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/iris&quot; rel=&quot;nofollow noreferrer&quot;&gt;introduced by Fisher in 1936&lt;/a&gt;. In this dataset, each flower has (only) &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; features (the inputs), namely&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;petal length, &lt;/li&gt;&#xA;&lt;li&gt;petal width, &lt;/li&gt;&#xA;&lt;li&gt;sepal length, and &lt;/li&gt;&#xA;&lt;li&gt;sepal width&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; classes (the outputs)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;iris setosa, &lt;/li&gt;&#xA;&lt;li&gt;iris virginica, and &lt;/li&gt;&#xA;&lt;li&gt;iris versicolor&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And there are a total of &lt;span class=&quot;math-container&quot;&gt;$150$&lt;/span&gt; observations (or records).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The iris flower dataset is available in &lt;code&gt;sklearn&lt;/code&gt;. See, for example, &lt;a href=&quot;https://scikit-learn.org/stable/datasets/index.html#iris-plants-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iris plants dataset&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;To search for other datasets, you can also use &lt;a href=&quot;https://toolbox.google.com/datasetsearch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://toolbox.google.com/datasetsearch&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-03T21:43:31.770" LastActivityDate="2019-08-03T21:43:31.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13761" PostTypeId="2" ParentId="4663" CreationDate="2019-08-03T19:03:27.807" Score="1" Body="&lt;p&gt;I will break it down for you in very simple words. The accuracy will drop down as you label them wrong. In simpler words- accuracy is directly proportional on how perfect the data is labelled. If you think about it, suppose you have 2 categories-cats and dogs, and you have a dataset of 10,000 pictures. Out of which 50 are wrongly labelled. The accuracy will less than the perfectly labelled but not that less since the neural network built will not be that bad. But suppose now you have 1000 wrongly labelled which is 1/10 of the dataset, then the NN will have more abrupt outcomes.&lt;/p&gt;&#xA;" OwnerUserId="27626" LastEditorUserId="38076" LastEditDate="2020-12-18T17:02:44.233" LastActivityDate="2020-12-18T17:02:44.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13762" PostTypeId="1" AcceptedAnswerId="13769" CreationDate="2019-08-03T21:44:58.360" Score="2" ViewCount="88" Body="&lt;p&gt;I'm a beginner in ML and have been researching RL quite a bit recently. I'm planning to create an RL application to play a zero-sum game. This will be web-based, so anyone can play it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wondered if I need to create a database (or some other kind of storage) to store the policy the RL algorithm is updating, so that it can be used by the application when the next human user comes along to play against the application?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2019-09-22T18:43:20.830" LastActivityDate="2019-09-22T18:43:20.830" Title="Is it a good idea to store the policy in a database?" Tags="&lt;reinforcement-learning&gt;&lt;storage&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13763" PostTypeId="2" ParentId="13189" CreationDate="2019-08-03T21:52:45.590" Score="1" Body="&lt;p&gt;The starting point is that for a fair dice thrown fairly the p(n) is 1/n where n is the number of sides.&lt;/p&gt;&#xA;&lt;p&gt;You said both&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;there are too many variables (up 40 dimensions with value range 1-100) in input, I don't know how these properties relate and an empirical approach would require too much data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It seems that this problem has 2 solutions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Don't use a neural net and create a 'std' statistical model.&#xA;It may be possible since you said:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I know there is some underlying rule that simplify a lot the problem (ie. reduce the actual number of dimension of the input)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a neural network (with softmax at the end) - for a fair dice; with enough training data the classifier should arrive as 1/n as the approximating function for a fair dice. The other 40 dimensions/settings your mentioned are the inputs. I think a 'basic' neural network with dense layers only could work for your task.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3526" LastEditorUserId="3526" LastEditDate="2020-11-16T05:21:27.817" LastActivityDate="2020-11-16T05:21:27.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13765" PostTypeId="1" CreationDate="2019-08-04T00:28:40.533" Score="4" ViewCount="96" Body="&lt;p&gt;Recently I encountered a variant on the normal linear neural layer architecture: Instead of &lt;span class=&quot;math-container&quot;&gt;$Z = XW + B$&lt;/span&gt;, we now have &lt;span class=&quot;math-container&quot;&gt;$Z = (X-A)W + B$&lt;/span&gt;. So we have a 'pre-bias' &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that affects the activation of the last layer, before multiplication by weights. I don't understand the backpropagation equations for &lt;span class=&quot;math-container&quot;&gt;$dA$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dB$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$dW$&lt;/span&gt; is as expected). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the original paper that it appeared in (although the paper itself isn't actually that relevant): &lt;a href=&quot;http://papers.nips.cc/paper/4830-learning-invariant-representations-of-molecules-for-atomization-energy-prediction.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://papers.nips.cc/paper/4830-learning-invariant-representations-of-molecules-for-atomization-energy-prediction.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link to the full code of the neural network: &lt;a href=&quot;http://www.quantum-machine.org/code/nn-qm7.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.quantum-machine.org/code/nn-qm7.tar.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Linear(Module):&#xA;&#xA;    def __init__(self,m,n):&#xA;&#xA;        self.tr = m**.5 / n**.5&#xA;        self.lr = 1 / m**.5&#xA;&#xA;        self.W = numpy.random.normal(0,1 / m**.5,[m,n]).astype('float32')&#xA;        self.A = numpy.zeros([m]).astype('float32')&#xA;        self.B = numpy.zeros([n]).astype('float32')&#xA;&#xA;    def forward(self,X):&#xA;        self.X = X&#xA;        Y = numpy.dot(X-self.A,self.W)+self.B&#xA;        return Y&#xA;&#xA;    def backward(self,DY):&#xA;        self.DW = numpy.dot((self.X-self.A).T,DY)&#xA;        self.DA = -(self.X-self.A).sum(axis=0)&#xA;        self.DB = DY.sum(axis=0) + numpy.dot(self.DA,self.W)&#xA;        DX = self.tr * numpy.dot(DY,self.W.T)&#xA;        return DX&#xA;&#xA;    def update(self,lr):&#xA;        self.W -= lr*self.lr*self.DW&#xA;        self.B -= lr*self.lr*self.DB&#xA;        self.A -= lr*self.lr*self.DA&#xA;&#xA;    def average(self,nn,a):&#xA;        self.W = a*nn.W + (1-a)*self.W&#xA;        self.B = a*nn.B + (1-a)*self.B&#xA;        self.A = a*nn.A + (1-a)*self.A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27634" LastEditorUserId="1671" LastEditDate="2019-11-07T03:10:55.840" LastActivityDate="2021-03-31T07:04:50.383" Title="Backpropagation equation for a variant on the usual Linear Neuron architecture" Tags="&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13768" PostTypeId="2" ParentId="13765" CreationDate="2019-08-04T09:30:18.523" Score="0" Body="&lt;p&gt;The forward prop equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Z = (X-A)W - B = XW - AW - B&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the derivatives for &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; w.r.t &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; should be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial Z}{\partial W} = X-A  \\&#xA;\frac{\partial Z}{\partial A} = - W  \\&#xA;\frac{\partial Z}{\partial B} = - 1 \\&#xA;\frac{\partial Z}{\partial X} = W&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know why he needs the last one though. The first is, like you said, as expected. The other two are wrong, I don't know why he used them in the implementation.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-08-04T09:30:18.523" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13769" PostTypeId="2" ParentId="13762" CreationDate="2019-08-04T09:38:30.247" Score="2" Body="&lt;p&gt;You have lots of choices in how to store a policy, depending on how you have built it - using which RL algorithm, and what kind of representation for states and actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tabular reinforcement learning algorithms lend themselves well to storage in a database table with an indexed &lt;code&gt;state_id&lt;/code&gt; column and one or both &lt;code&gt;action&lt;/code&gt; and &lt;code&gt;value&lt;/code&gt; columns. This might be a good choice if you have a moderate sized state space, as you would avoid the need to load the whole table into memory just to compute the next move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether this is feasible will depend on the complexity of your game. Even relatively simple &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;games like checkers turn out to have too large a state space&lt;/a&gt; to enumerate all the states in this way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you are more likely to need some kind of policy function or state value function implemented using a parametric function approximator. Very often in RL this will be a neural network. In which case you would use whatever storage mechanism your neural network library supported - most will happily read and write their parameters to a file or string, allowing you a lot of flexibility on how and where to store them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your policy is likely to be stored in one or two files on disk as a serialised neural network. How to use that efficiently in a web service is a complex subject in its own right. You could just read the files back and instantiate the neural network each time it is needed, and this will probably be OK for a simple game and low traffic service. However, this is very inefficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some neural network libraries designed around use in production will allow you to pre-load the neural network and keep it in memory between requests. How to do this depends entirely upon the frameworks you are using, so I cannot explain in more detail here. Initially I would not worry too much about this part for your project.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-08-04T09:38:30.247" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13770" PostTypeId="1" CreationDate="2019-08-04T10:33:19.023" Score="3" ViewCount="24" Body="&lt;p&gt;A machine learning project I am working on requires me to interface with an Xbox controller connected to a PC. The implementation must do the following two things:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Record the joystick input from the controller into a file at regular intervals, along with an associated screenshot from a game. (ex: 60 times a second).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this data I want to try to replicate/reverse engineer a few different FPS games sensitivity’s,dead zones, acceleration curves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any idea as to how I'd go about doing this? I'm not sure where to start. If this question isn't appropriate for this sub, I’m where could I ask?&lt;/p&gt;&#xA;" OwnerUserId="27637" LastActivityDate="2019-08-04T10:33:19.023" Title="Reverse engineering controller sensitivity/aim for several games ie acceleration curves, deadzones, etc" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13772" PostTypeId="1" CreationDate="2019-08-04T20:08:42.683" Score="1" ViewCount="496" Body="&lt;p&gt;What is an identity recurrent neural network (IRNN)? What is the difference between an IRNN and RNN?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-04T20:08:42.683" Title="What is an identity recurrent neural network?" Tags="&lt;recurrent-neural-networks&gt;&lt;comparison&gt;&lt;definitions&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13773" PostTypeId="2" ParentId="13772" CreationDate="2019-08-04T20:08:42.683" Score="1" Body="&lt;p&gt;An identity recurrent neural network (IRNN) is a &lt;em&gt;vanilla&lt;/em&gt; recurrent neural network (as opposed to e.g. &lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTMs&lt;/a&gt;) whose recurrent weight matrices are initialized with the &lt;em&gt;identity matrix&lt;/em&gt;, the biases are initialized to zero, and the hidden units (or neurons) use the rectified linear unit (ReLU).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An IRNN can be trained more easily using gradient descent (as opposed to a vanilla RNN that is not an IRNN), given that it behaves similarly to an LSTM-based RNN, that is, an IRNN does not suffer (much) from the &lt;em&gt;vanishing gradient problem&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The IRNN achieves a performance similar to LSTM-based RNNs in certain tasks, including the &lt;em&gt;adding problem&lt;/em&gt; (a standard problem that is used to examine the power of recurrent models in learning long-term dependencies). In terms of architecture, the vanilla RNNs are much simpler than LSTM-based RNNs, so this is an advantage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, see the paper &lt;a href=&quot;https://arxiv.org/pdf/1504.00941.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Simple Way to Initialize Recurrent Networks of Rectified Linear Units&lt;/a&gt; (2015), by Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton. See also &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/examples/mnist_irnn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Keras implementation of the MNIST experiment&lt;/a&gt; described in the linked paper.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-04T20:08:42.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13774" PostTypeId="2" ParentId="13668" CreationDate="2019-08-04T20:30:39.687" Score="1" Body="&lt;p&gt;Dennis' answer is very helpful. I also found section 5.5 of the &lt;a href=&quot;http://mcts.ai/pubs/mcts-survey-master.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCTS survey&lt;/a&gt; very useful, in particular the widening discussion. Another useful reference was &lt;a href=&quot;https://project.dke.maastrichtuniversity.nl/games/files/msc/Roelofs_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://project.dke.maastrichtuniversity.nl/games/files/msc/Roelofs_thesis.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27530" LastActivityDate="2019-08-04T20:30:39.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13775" PostTypeId="1" AcceptedAnswerId="13787" CreationDate="2019-08-04T22:24:19.287" Score="35" ViewCount="15899" Body="&lt;p&gt;I just finished a 1-year Data Science master's program where we were taught R. I found that Python is more popular and has a larger community in AI.&lt;/p&gt;&#xA;&lt;p&gt;What are the advantages that Python may have over R in terms of features applicable to the field of Data Science and AI (other than popularity and larger community)? What positions in Data Science and AI would be more Python-heavy than R-heavy (especially comparing industry, academic, and government job positions)? In short, is Python worthwhile in all job situations or can I get by with only R in some positions?&lt;/p&gt;&#xA;" OwnerUserId="27652" LastEditorUserId="16959" LastEditDate="2020-10-27T01:26:48.680" LastActivityDate="2020-10-27T01:26:48.680" Title="Is a switch from R to Python worth it?" Tags="&lt;python&gt;&lt;comparison&gt;&lt;r&gt;" AnswerCount="8" CommentCount="2" FavoriteCount="18" ClosedDate="2019-11-19T04:26:27.647" ContentLicense="CC BY-SA 4.0" />
  <row Id="13776" PostTypeId="1" CreationDate="2019-08-04T22:33:07.360" Score="3" ViewCount="182" Body="&lt;p&gt;In neural networks with stochastic layers I've seen the use of the REINFORCE estimator for estimating the gradient (because it can't be computed directly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some such examples are &lt;a href=&quot;https://arxiv.org/pdf/1502.03044.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Show, Attend and Tell&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1406.6247&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent models of visual attention&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1412.7755&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiple Object Recognition with Visual Attention&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I haven't figured out how this exactly works. How do we &quot;bypass&quot; the gradient's computation by using the REINFORCE learning rule? Does anyone have any insight on this? &lt;/p&gt;&#xA;" OwnerUserId="27653" LastActivityDate="2019-08-06T22:59:32.010" Title="How is REINFORCE used instead of Backpropagation?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13777" PostTypeId="1" AcceptedAnswerId="14233" CreationDate="2019-08-04T22:53:49.960" Score="3" ViewCount="104" Body="&lt;p&gt;I've been reading on the differences between &lt;a href=&quot;https://www.differencebetween.com/difference-between-strong-ai-and-vs-weak-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Strong&quot; and &quot;Weak &quot;AI&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering, where do Neural Networks (especially deep ones) fall in this spectrum? Can they be considered &quot;Strong AI&quot;? If not, is there any model that can be considered &quot;Strong AI&quot;?&lt;/p&gt;&#xA;" OwnerUserId="27654" LastActivityDate="2019-09-01T05:21:12.580" Title="Can Neural Networks be considered as &quot;Strong AI&quot;?" Tags="&lt;neural-networks&gt;&lt;agi&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13778" PostTypeId="2" ParentId="13775" CreationDate="2019-08-04T23:16:27.707" Score="32" Body="&lt;p&gt;Of course, this type of questions will also lead to primarily opinion-based answers. Nonetheless, it is possible to enumerate the strengths and weakness of each language, with respect to machine learning, statistics, and data analysis tasks, which I will try to list below.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;R&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Strengths&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;R was designed and developed for statisticians and data analysts, so it provides, &lt;a href=&quot;https://en.wikipedia.org/wiki/Out_of_the_box_(feature)&quot; rel=&quot;noreferrer&quot;&gt;out-of-the-box&lt;/a&gt; (that is, they are part of the language itself), features and facilities for statisticians, which are not available in Python, unless you install a related package. For example, the data frame, which Python does not provide, unless you install the famous Python's &lt;a href=&quot;https://pandas.pydata.org/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;pandas&lt;/code&gt;&lt;/a&gt; package. There are other examples like matrices, vectors, etc. In Python, there are also similar data structures, but they are more general, so not specifically targeted for statisticians.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are a lot of statistical libraries.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Weakness&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given its purpose, R is mainly used to solve statistical or data analysis problems. However, it can also be used outside of this domain. See, for example, this Quora question: &lt;a href=&quot;https://qr.ae/TWvefg&quot; rel=&quot;noreferrer&quot;&gt;Is R used outside of statistics and data analysis?&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Python&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Strengths&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A lot of people and companies, including Google and Facebook, invest a lot in Python. For example, the main programming language of TensorFlow and PyTorch (two widely used machine learning frameworks) is Python. So, it is very unlikely that Python won't continue to be widely used in machine learning for at least 5-10 more years.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Python community is likely a lot bigger than the R community. In fact, for example, if you look at &lt;a href=&quot;https://www.tiobe.com/tiobe-index/&quot; rel=&quot;noreferrer&quot;&gt;Tiobe's index&lt;/a&gt;, Python is placed 3rd, while R is placed 20th.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Python is also widely used outside of the statistics or machine learning communities. For example, it is used for web development (see e.g. the Python frameworks Django or Flask).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are a lot of machine learning libraries (e.g. TensorFlow and PyTorch).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Weakness&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It does not provide, out-of-the-box, the statistical and data analysis functionalities that R provides, unless you install an appropriate package. This might be a weakness or a strength, depending on your philosophical point of view.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are other possible advantages and disadvantages of these languages. For example, both languages are dynamic. However, this feature can both be an advantage and a disadvantage (and it is not strictly related to machine learning or statistics), so I did not list it above. I avoided mentioning opinionated language features, such as code readability and learning curve, for obvious reasons (e.g. not all people have the same programming experience).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Python is definitely worth learning if you are studying machine learning or statistics. However, it does not mean that you will not use R anymore. R might still be handier for certain tasks. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-04T23:16:27.707" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13779" PostTypeId="2" ParentId="13776" CreationDate="2019-08-05T00:12:52.437" Score="1" Body="&lt;p&gt;REINFORCE is called a gradient estimator because it doesn't work on the true gradient, that comes from a loss function and the whole data, but makes up a heuristic loss, so that the gradient it ends up with isn't the true one. Let's see that with the REINFORCE equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{\huge&#xA;\Delta \mathbf{\theta}_t = \alpha \nabla_{\mathbf{\theta}} \log \pi_{\mathbf{\theta}} (a_t \mid s_t) v_t&#xA;}%&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this shows, the gradient is still there (&lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta$&lt;/span&gt;). But the policy corresponds to the network's output, so we can use backpropagation to compute the gradient of that heuristic loss with respect to the weights. The real gradient is unknown to us, but this estimation will do the job.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastEditorUserId="2444" LastEditDate="2019-08-06T22:59:32.010" LastActivityDate="2019-08-06T22:59:32.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13780" PostTypeId="2" ParentId="13692" CreationDate="2019-08-05T00:38:13.260" Score="5" Body="&lt;p&gt;3D convolutions should when you want to extract spatial features from your input on three dimensions. For Computer Vision, they are typically used on &lt;strong&gt;volumetric images&lt;/strong&gt;, which are 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples are &lt;a href=&quot;http://graphics.stanford.edu/projects/3dcnn/&quot; rel=&quot;noreferrer&quot;&gt;classifying 3D rendered images&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1606.06650.pdf&quot; rel=&quot;noreferrer&quot;&gt;medical image segmentation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27655" LastActivityDate="2019-08-05T00:38:13.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13781" PostTypeId="2" ParentId="13666" CreationDate="2019-08-05T00:49:25.197" Score="2" Body="&lt;p&gt;Ensembles aren't very popular in the field of computer vision. The main reason why this is, is that models are already so large parameter-wise that it is hard to fit multiple models in-memory for classification. Since there are effective ways of training very large models, people would rather create a larger networks if they had the capacity than averaging the results from multiple ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, there is no reason why ensembling wouldn't have beneficial results for your task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way would be, as you do, to average the results of the models. This is usually used to reduce the bias of weaker models. Another way would be to use &lt;a href=&quot;http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta-modelling&lt;/a&gt;, i.e. create a fourth model (even as simple as a linear classifier) that will be trained with the outputs of the three CNNs as its input features. The idea is that the meta-model will learn the best way to weight the outputs of the CNNs so that, instead of them all having an equal vote (as is the case when you average them), the meta-model will learn the best way to weigh them.&lt;/p&gt;&#xA;" OwnerUserId="27655" LastActivityDate="2019-08-05T00:49:25.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13782" PostTypeId="1" CreationDate="2019-08-05T01:34:37.530" Score="3" ViewCount="45" Body="&lt;p&gt;Deep fakes work by using a single encoder but then having a different decoder for different people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I wondered what if the encoder encodes say &quot;closed eyes&quot; of person A as the same code for &quot;closed mouth&quot; of person B. i.e. the codes could use the same codewords for different aspects of person A and person B. i.e. person A and person B could use the same codewords to descibe each of them except the codewords don't mean the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then when you do a deep fake on person A with closed eyes it emerges as person B with a closed mouth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one combat this effect. Or is does it just work and no-one knows why?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-08-05T01:34:37.530" Title="How do deep fakes get the right encoding for both people?" Tags="&lt;deep-learning&gt;&lt;deepfakes&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13784" PostTypeId="1" CreationDate="2019-08-05T04:52:48.333" Score="3" ViewCount="687" Body="&lt;p&gt;Is there any way to make &lt;a href=&quot;https://en.wikipedia.org/wiki/Deepfake&quot; rel=&quot;nofollow noreferrer&quot;&gt;deepfake&lt;/a&gt; videos without a fancy computer? For example, run the &lt;a href=&quot;https://github.com/iperov/DeepFaceLab&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepFaceLab&lt;/a&gt; on a website so your own computer won't get involved?&lt;/p&gt;&#xA;" OwnerUserId="27660" LastEditorUserId="2444" LastEditDate="2019-08-11T15:47:35.140" LastActivityDate="2019-08-11T15:47:35.140" Title="How to make deepfake video without a fancy PC?" Tags="&lt;deep-learning&gt;&lt;deepfakes&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13785" PostTypeId="1" CreationDate="2019-08-05T07:05:52.717" Score="2" ViewCount="21" Body="&lt;p&gt;Let's say our task is to pick and place a block, like: &lt;a href=&quot;https://gym.openai.com/envs/FetchPickAndPlace-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gym.openai.com/envs/FetchPickAndPlace-v0/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reward function 1: -1 for block not placed, 0 for block placed&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reward function 2: 0 for block not placed, +1 for block placed&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed training 1 is much faster than 2... I am using the HER implementation from OpenAI. Why is that?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastActivityDate="2019-08-05T07:05:52.717" Title="Why do these reward functions give different training curves?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13786" PostTypeId="2" ParentId="13692" CreationDate="2019-08-05T07:43:25.680" Score="12" Body="&lt;p&gt;3D CNN's are used when you want to extract features in 3 Dimensions or establish a relationship between 3 dimensions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially its the same as 2D convolutions but the kernel movement is now 3-Dimensional causing a better capture of dependencies within the 3 dimensions and a difference in output dimensions post convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kernel on convolution will move in 3-Dimensions if the kernel depth is lesser then the feature map depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://miro.medium.com/max/700/1*wUVVgZnzBwYKgQyTBK_5sg.png&quot; alt=&quot;this&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand 2-D convolutions on 3-D data mean that the kernel will traverse in 2-D only. This happens when the feature map depth is the same as the kernel depth (channels) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://miro.medium.com/max/700/1*K_OsaLGJ7I8cGlPvvWYJdg.png&quot; alt=&quot;here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some use cases for better understanding are - MRI scans where relationship between a stack of images is to be understood ; and a low level feature extractor for spatio-temporal data like videos for Gesture Recognition, Weather forecast etc. (3-D CNN's are used as low level feature extractors only over multiple short intervals as 3D CNN's fail to capture long term spatio-temporal dependencies - for more on that check out &lt;a href=&quot;https://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf&quot; rel=&quot;noreferrer&quot;&gt;ConvLSTM&lt;/a&gt; or an alternate perspective &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6387284/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.)&#xA;Most CNN models that learn from video data almost always have 3D CNN as a low level feature extractor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example you have mentioned above regarding the number 5 - 2D convolutions would probably perform better, as you're treating every channel intensity as an aggregate of the information it holds, meaning the learning would almost be the same as it would on a black and white image. Using 3D convolution for this on the other hand would cause learning of relationships between the channels which do not exist in this case! (Also 3D convolutions on an image with depth 3 would require a very uncommon kernel to be used, especially for the use case)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope your query has been cleared!  &lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="25658" LastEditDate="2019-08-05T07:49:31.513" LastActivityDate="2019-08-05T07:49:31.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13787" PostTypeId="2" ParentId="13775" CreationDate="2019-08-05T08:01:04.957" Score="67" Body="&lt;p&gt;I want to reframe your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't think about switching, think about adding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In data science you'll be able to go very far with either python or r but you'll go farthest with both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python and r integrate very well, thanks to the &lt;code&gt;reticulate&lt;/code&gt; package. I often tidy data in r because it is easier for me, train a model in python to benefit from superior speed and visualize the outcomes in r in beautiful &lt;code&gt;ggplot&lt;/code&gt; all in one notebook!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you already know r there is no sense in abandoning it, use it where sensible and easy to you. But it is 100% a good idea to add python for many uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you feel comfortable in both you'll have  a workflow that fits you best dominated by your favorite language.&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2019-08-05T08:01:04.957" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13789" PostTypeId="2" ParentId="13784" CreationDate="2019-08-05T09:27:48.950" Score="2" Body="&lt;p&gt;Yes. There are services that provide free environment to run jupyter notebooks for research purposes (with GPU included, which is crucial for neural networks) - such as Google Colaboratory and Kaggle Kernels. Although they limit how long your computation may run (12 and 6 hours accordingly), which adds some difficulties to the process, although I think it is possible to bypass these restrictions.&lt;/p&gt;&#xA;" OwnerUserId="27672" LastActivityDate="2019-08-05T09:27:48.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13790" PostTypeId="1" CreationDate="2019-08-05T10:11:52.713" Score="2" ViewCount="81" Body="&lt;p&gt;I am playing around with creating &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/guide/custom_policy.html#examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;custom architectures in stable-baselines&lt;/a&gt;. Specifically I am training an agent using a &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/modules/ppo2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO2&lt;/a&gt; model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is, are there some rules of thumb or best practices in network architecture (of actor and critic networks) to achieve higher performance i.e. larger rewards?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I find that usually using wider layers (e.g. 256 rather than 128 units) and adding more layers (e.g. a deep network with 5 layers rather than 2) achieves a smaller RMSE (better performance) for time series prediction when training an LSTM. Would similar conventions apply to reinforcement learning - would adding more layers to the actor and critic network have higher performance - does sharing an input layer work well?&lt;/p&gt;&#xA;" OwnerUserId="27570" LastActivityDate="2019-08-05T10:11:52.713" Title="What is a high performing network architecture to use in a PPO2 MlpLnLstmPolicy RL model?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13791" PostTypeId="2" ParentId="13775" CreationDate="2019-08-05T11:27:29.797" Score="6" Body="&lt;p&gt;I didn't have this choice because I was forced to move from R to Python:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It depends on your &lt;strong&gt;environment&lt;/strong&gt;: When you are embedded in an &lt;strong&gt;engineer&lt;/strong&gt; department, working technical group or something similar than Python is more feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are surrounded by &lt;strong&gt;scientists&lt;/strong&gt; and especially &lt;strong&gt;statisticians&lt;/strong&gt;, stay with R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: R offers keras and tensorflow as well though it is implemented under the hood of python. Only very advanced stuff will make you need Python.&#xA;Though I'm getting more and more used to Python, the synthax in &lt;strong&gt;R is easier&lt;/strong&gt;. And though each package has its own, it is somehow consistent while Python is not..&#xA;And ggplot is so strong. Python has a clone (plotnine) but it lacks several (important) features. In principle you can do nearly as much as in R but especially visualization and data wrangling is much easier in R. Thus, the most famous Python library, pandas, is a &lt;strong&gt;clone&lt;/strong&gt; of R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PSS: &lt;strong&gt;Advanced&lt;/strong&gt; statistics aims definitely at R. Python offers a lot of everyday tools and methods for a data scientist but it will never reach those &gt;&lt;strong&gt;13,000&lt;/strong&gt; packages R provides. For example, I had to do an inverse regression and python doesn't offer this. In R you can choose between several confidence tests and whether it is linear or nonlinear. &#xA;The same goes to mixed models: It is implemented in python but it is so basic there I can't realize how this can be sufficient for someone.&lt;/p&gt;&#xA;" OwnerUserId="26353" LastEditorUserId="26353" LastEditDate="2019-08-06T13:35:10.263" LastActivityDate="2019-08-06T13:35:10.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13792" PostTypeId="2" ParentId="13775" CreationDate="2019-08-05T11:32:55.813" Score="1" Body="&lt;p&gt;As others have said, it's not a &quot;switch&quot;. But is it worth adding Python to your arsenal? I would say certainly. In data science, Python is popular and becoming ever more popular, while R is receding somewhat. And in the fields of machine learning and neural networks, I'd say that Python is the main language now -- I don't think R really comes close here in terms of usage. The reason for all of this is generality. Python is intended as a general programming language, and allows you to easily script all kinds of tasks. If you're staying strictly within a neatly structured statistical world, R is great, but with AI you often end up having to do novel, miscellaneous things, and I don't think R can beat Python at that. And because of this, I think Python and its packages will be receiving more support and development when it comes to the more cutting-edge tech.&lt;/p&gt;&#xA;" OwnerUserId="26737" LastActivityDate="2019-08-05T11:32:55.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13793" PostTypeId="1" AcceptedAnswerId="13795" CreationDate="2019-08-05T12:31:15.447" Score="1" ViewCount="110" Body="&lt;p&gt;I made a simple Python game. A screenshot is below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QVn9W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QVn9W.png&quot; alt=&quot;screenshot&quot; /&gt;&lt;/a&gt;&#xA;Basically, a paddle moves left and right catching particles. Some make you lose points while others make you gains points.&lt;/p&gt;&#xA;&lt;p&gt;This is my first Deep Q Learning Project, so I probably messed something up, but here is what I have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(200, input_shape=(4,), activation='relu'))&#xA;model.add(Dense(200, activation='relu'))&#xA;model.add(Dense(3, activation='linear'))&#xA;model.compile(loss='categorical_crossentropy', optimizer='adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The four inputs are X position of player, X and Y position of particle (one at a time), and the type of particle. Output is left, right, or don't move.&lt;/p&gt;&#xA;&lt;p&gt;Here is the learning algorithm:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def learning(num_episodes=500):&#xA;    y = 0.8&#xA;    eps = 0.5&#xA;    decay_factor = 0.9999&#xA;    for i in range(num_episodes):&#xA;        state = GAME.reset()&#xA;        GAME.done = False&#xA;        eps *= decay_factor&#xA;        done = False&#xA;        while not done:&#xA;            if np.random.random() &amp;lt; eps: #exploration&#xA;                a = np.random.randint(0, 2)&#xA;            else:&#xA;                a = np.argmax(model.predict(state))&#xA;            new_state, reward, done = GAME.step(a) #does that step&#xA;            #reward can be -20, -5, 1, and 5&#xA;            target = reward + y * np.max(model.predict(new_state))&#xA;            target_vec = model.predict(state)[0]&#xA;            target_vec[a] = target&#xA;            model.fit(state, target_vec.reshape(-1, 3), epochs=1, verbose=0)&#xA;            state = new_state&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After training, this usually results in the paddle just going to the side and staying there. I am not sure if the NN architecture (units and hidden layers) is appropriate for given complexity. Also, is it possible that this is failing due to the rewards being very delayed? It can take 100+ frames to get to the food, so maybe this isn't registering well with the neural network.&lt;/p&gt;&#xA;&lt;p&gt;I only started learning about reinforcement learning yesterday, so would appreciate advice!&lt;/p&gt;&#xA;" OwnerUserId="27681" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-05T19:38:33.743" Title="Deep Q Learning Algorithm for Simple Python Game makes player stuck" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13794" PostTypeId="1" AcceptedAnswerId="13804" CreationDate="2019-08-05T13:09:15.580" Score="1" ViewCount="122" Body="&lt;p&gt;The Wikipedia definitions are as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multi-agent systems&lt;/strong&gt; - &lt;em&gt;A multi-agent system is a computerized system composed of multiple interacting intelligent agents.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multi-modal interaction&lt;/strong&gt; - &lt;em&gt;Multimodal interaction provides the user with multiple modes of interacting with a system.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doesn't providing a user with multiple modes of interacting with a system, assuming all modalities interact with each other to give final output (some sort of fusion mechanism for example), make it a multi-agent system? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what is the difference between multi-modal and multi-agent systems and, monolithic and uni-modal systems?&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-08-05T17:40:18.153" LastActivityDate="2019-08-05T23:48:10.847" Title="What is the difference between multi-agent and multi-modal systems?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13795" PostTypeId="2" ParentId="13793" CreationDate="2019-08-05T13:22:21.913" Score="2" Body="&lt;p&gt;This is probably the most major factor:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(loss='categorical_crossentropy', optimizer='adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you have set the loss function for a multiclass classifier. It is going to have some weird results when values - either predicted or target - are outside of range 0..1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should use this instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(loss='mean_squared_error', optimizer='adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because your Q network outputs the expected future return on each action. This could easily be outside of the range that 'categorical_crossentropy' is designed for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, you &lt;em&gt;really&lt;/em&gt; need to look into &lt;a href=&quot;https://datascience.stackexchange.com/questions/20535/what-is-experience-replay-and-what-are-its-benefits&quot;&gt;experience replay&lt;/a&gt;. It is not an optional extra when using neural networks with Q learning - it is pretty much required for anything but the most trivial environments. It is very likely your agent will still fail to learn without experience replay, if you correct all other problems with your code.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am not sure if the NN architecture (units and hidden layers) is appropriate for given complexity. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It looks more complex than it needs to be, assuming that your 4 inputs represent paddle x position, particle x,y position and colour. I would suggest making the network simpler (maybe just 40 neurons per layer at a guess), to speed things up a little.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check your input scaling. Neural networks like to train on inputs that have mean 0, standard deviation 1, and it is worth scaling them so that they fit roughly into -1..1 or similar. Your feature engineering is not shown in your code, so it &lt;em&gt;might&lt;/em&gt; be an issue.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, is it possible that this is failing due to the rewards being very delayed? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This can be a factor that makes learning harder.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It can take 100+ frames to get to the food, so maybe this isn't registering well with the neural network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;100 time steps delay between rewards is not much for DQN. It should be correctly predicting Q values - it will just take more episodes to learn to predict the best movement when the food is further away.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-08-05T13:22:21.913" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13796" PostTypeId="2" ParentId="13784" CreationDate="2019-08-05T13:30:46.983" Score="1" Body="&lt;p&gt;Take a look at using &lt;a href=&quot;https://aws.amazon.com/machine-learning/amis/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS&lt;/a&gt; You can spin up an instance with as much processing power as you need and by using their pre-built images it will already be preconfigured with a lot of the packages etc you might need for any kind of ML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see you wanted to use DeepFaceLab which I guess requires some kind of GUI so unsure if this is suitable for your requirements but check it out, it seems to be the best way to perform high-processing machine learning without the &lt;em&gt;fancy computer&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="8960" LastActivityDate="2019-08-05T13:30:46.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13797" PostTypeId="1" CreationDate="2019-08-05T14:02:01.750" Score="2" ViewCount="76" Body="&lt;p&gt;For real applications, concept drifts often exist, i.e., the relationship between the input and output changes overtime. Thus, we need our AI or machine learning system to quickly adapt to the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the most common methods to enable neural networks to quickly adapt to the changing environment for supervised learning? Could somebody provide a link to a good review article? &lt;/p&gt;&#xA;" OwnerUserId="22105" LastEditorUserId="2444" LastEditDate="2019-08-05T23:58:19.560" LastActivityDate="2019-09-05T12:00:45.073" Title="What are the most common methods to enable neural networks to adapt to changing environments?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;supervised-learning&gt;&lt;transfer-learning&gt;&lt;incremental-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13798" PostTypeId="1" CreationDate="2019-08-05T14:23:03.950" Score="1" ViewCount="24" Body="&lt;p&gt;I have some rated time-sequential data and I would like to test if an ANN can learn a correlation between my measurements and ratings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect I could just try a CNN where 1 Dimension is time or an LSTM/GRU and put the result through sigmoid, but is there any good literature on this? I have been trying to find information on datasets for the problem but it seems that Sequence regression is lacking any big official datasets, even though use-cases are there(e.g. learning personal music taste, try to predict rotten-tomato scores, etc..).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking for links to papers describing successful architectures or benchmarks where I can test my models.&lt;/p&gt;&#xA;" OwnerUserId="27267" LastActivityDate="2019-08-05T14:23:03.950" Title="Literature on Sequence Regresssion" Tags="&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;sequence-modelling&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13799" PostTypeId="1" CreationDate="2019-08-05T18:31:54.333" Score="2" ViewCount="199" Body="&lt;p&gt;While analyzing the data for a given problem set, I came across a few distributions which are not Gaussian in nature. They are not even uniform or Gamma distributions(so that I can write a function, plug the parameters and calculate the &quot;Likelihood probability&quot; and solve it using Bayes classification method). I got a set of a few absurd looking PDFs and I am wondering how should I define them mathematically so that I can plug the parameters and calculate the likelihood probability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set of PDFs/Distributions that I got are the following and I am including some solutions that I intend to use. Please comment on their validity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)&lt;a href=&quot;https://i.stack.imgur.com/P5GYb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P5GYb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ y = ax +b $&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$ 0.8&amp;lt;x&amp;lt;1.5 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to programmatically calculate&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. The value of x where the pdf starts&#xA;2. The value of x where the pdf ends&#xA;3. The value of y where the pdf starts&#xA;4. The value of y where the pdf ends&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I would have liked it better to have a generic distribution for this form of graphs so that I can plug the parameters to calculate the probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2)&lt;a href=&quot;https://i.stack.imgur.com/4X0K0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4X0K0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This PDF looks neither uniform nor Gaussian. What kind of distribution should I consider it roughly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3)&lt;a href=&quot;https://i.stack.imgur.com/8pqnM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8pqnM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can divide this graph into three segments. The first segment is from &lt;span class=&quot;math-container&quot;&gt;$2&amp;lt;x&amp;lt;3$&lt;/span&gt; with a steep slope, the second segment is from &lt;span class=&quot;math-container&quot;&gt;$3&amp;lt;x&amp;lt;6$&lt;/span&gt; with a moderate sope and the third segment is from &lt;span class=&quot;math-container&quot;&gt;$6&amp;lt;x&amp;lt;8$&lt;/span&gt; with a high negative slope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to programmatically calculate&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1. the values of x where the graph changes its slope.&#xA; 2. the values of y where the graph changes its slope.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4)&lt;a href=&quot;https://i.stack.imgur.com/pAQUv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pAQUv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks like two Gaussian densities with different mean superimposed together. But then the question arises, how do we find these two individual Gaussian densities?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code may help:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;variable1=nasa1['PerihelionArg'][nasa1.PerihelionArg&amp;gt;190] &#xA;variable2=nasa1['PerihelionArg'][nasa1.PerihelionArg&amp;lt;190] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Find mean and variance of &lt;code&gt;variable1&lt;/code&gt; and &lt;code&gt;variable2&lt;/code&gt;, find the corresponding PDFs. Define the overall PDF with a suitable range of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5)&lt;a href=&quot;https://i.stack.imgur.com/SyrFO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SyrFO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be estimated as a Gamma distribution. We can find the mean and variance, calculate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and finally calculate the PDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be very helpful if someone could give their insights on the above analysis, its validity, and correctness and their suggestions regarding how problems such as these should be dealt with.&lt;/p&gt;&#xA;" OwnerUserId="26301" LastEditorUserId="26301" LastEditDate="2019-08-07T18:11:53.867" LastActivityDate="2021-05-11T17:01:45.150" Title="What to do when PDFs are not Gaussian/Normal in Naive Bayes Classifier" Tags="&lt;probability&gt;&lt;probability-distribution&gt;&lt;naive-bayes&gt;&lt;conditional-probability&gt;" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13800" PostTypeId="1" CreationDate="2019-08-05T19:00:56.817" Score="1" ViewCount="35" Body="&lt;p&gt;newbie here. I am studying the REINFORCE method in &quot;Deep Reinforcement Learning Hands-On&quot;. I can't understand how, after computing the loss of the episode, that loss is backpropagated in a NN with multiple output nodes. To be more precise, in Supervised Learning, when we have multiple output nodes we know the loss of each of them, but in RL, how do we compute the loss of each output node (or maybe the partial derivative of the total loss with respect to each output layer)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope to have been clear, thanks in advance.  &lt;/p&gt;&#xA;" OwnerUserId="27698" LastActivityDate="2019-08-05T19:00:56.817" Title="How is computed the gradient with respect to each output node from a loss value?" Tags="&lt;reinforcement-learning&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13801" PostTypeId="2" ParentId="11100" CreationDate="2019-08-05T21:43:34.387" Score="0" Body="&lt;p&gt;your targets should be in the same range as your output functions other wise your loss function wont be accurate, with supervised learning your trying to reduce the loss of your output against your targets so in this case your targets should be the true/optimal probability distribution for that set of input data. Im from the midwest so obligatory &quot;cant compare apples to oranges&quot; here ;)&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-08-05T21:50:32.407" LastActivityDate="2019-08-05T21:50:32.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13803" PostTypeId="2" ParentId="13751" CreationDate="2019-08-05T23:06:28.027" Score="0" Body="&lt;p&gt;This research question seems to be analyzed in further details here (section 3) - &lt;a href=&quot;https://openreview.net/pdf?id=r1lyTjAqYX&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openreview.net/pdf?id=r1lyTjAqYX&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, a sequence is taken as a state to be fed into RNN to compute the final hidden state. One can then ask what initial state should the RNN be seeded with? This paper analyzes three methods with respect to the seed - &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;zero initialization: When the RNN is initialized with the zero state &lt;/li&gt;&#xA;&lt;li&gt;burn-in: when the sequence is prepended by some preceding observations for RNN to learn a good initial state &lt;/li&gt;&#xA;&lt;li&gt;storing the initial hidden state: When the hidden state at the beginning of the sequence is stored&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21509" LastActivityDate="2019-08-05T23:06:28.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13804" PostTypeId="2" ParentId="13794" CreationDate="2019-08-05T23:48:10.847" Score="1" Body="&lt;p&gt;An agent is a concept, which can have slightly different meanings, abilities or instantiations depending on the context. However, given the purpose of this website, I will use and refer to the definition of agent commonly used in artificial intelligence. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For more details regarding the definition of an agent in AI, see &lt;a href=&quot;https://ai.stackexchange.com/a/12995/2444&quot;&gt;my answer&lt;/a&gt; to the question &lt;a href=&quot;https://ai.stackexchange.com/q/12991/2444&quot;&gt;What is an agent in Artificial Intelligence?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A multi-agent system is a system composed of multiple agents that interact with an environment. See &lt;a href=&quot;https://www.researchgate.net/publication/324847369_Multi-Agent_Systems_A_survey&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Agent Systems: A survey&lt;/a&gt; (2018) fore a more exhaustive overview of the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multimodal interaction (MI) refers to the interaction with a system (e.g. a computer) using multiple modalities (e.g. speech or gestures). For example, we usually can interact with a laptop using a keyboard and a touchpad (or mouse), so the keyboard and the touchpad are the two different modalities that are used to interact with the computer. MI could thus be considered a sub-field of &lt;a href=&quot;http://www.ee.columbia.edu/~ajaimes/Pubs/ajaimes_iccv05.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;human-computer interaction&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceptually, an agent could be associated with each modality provided by a multimodal system, so a system that provides multimodal interaction could indeed be a multi-agent system. See, for example, &lt;a href=&quot;https://pdfs.semanticscholar.org/74b9/ecc75c58f5ad296db2039efdf4feef25c7ce.pdf?_ga=2.265678347.50664443.1565048863-308724315.1559322718&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Multi-Agent based Multimodal System Adaptive to the User’s Interaction Context&lt;/a&gt; (2011).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-05T23:48:10.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13805" PostTypeId="1" AcceptedAnswerId="13811" CreationDate="2019-08-06T03:02:23.953" Score="3" ViewCount="169" Body="&lt;p&gt;I've heard somewhere that due to their nature of capturing spatial relations, even untrained CNNs can be used as feature extractors? Is this true? Does anyone have any sources regarding this I can look at?&lt;/p&gt;&#xA;" OwnerUserId="27240" LastActivityDate="2019-08-06T11:14:10.187" Title="Untrained CNNs as feature extractors?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13806" PostTypeId="1" CreationDate="2019-08-06T03:08:18.480" Score="1" ViewCount="87" Body="&lt;p&gt;This is a follow-up question about one I asked earlier. The first question is &lt;a href=&quot;https://ai.stackexchange.com/questions/13793/deep-q-learning-algorithm-for-simple-python-game-makes-player-stuck&quot;&gt;here&lt;/a&gt;. Basically, I have a game where a paddle moves left and right to catch as much &quot;food&quot; as possible. Some food is good (gain points) and some is bad (lose points). NN Architecture:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #inputs - paddle.x, food.x, food.y, food.type&#xA;    #moves: left, right, stay&#xA;    model = Sequential()&#xA;    model.add(Dense(10, input_shape=(4,), activation='relu'))&#xA;    model.add(Dense(10, activation='relu'))&#xA;    model.add(Dense(3, activation='linear'))&#xA;    model.compile(loss='mean_squared_error', optimizer='adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As suggested in the other question, I scaled my inputs to be between 0 and 1. Also, implemented experience replay (although I am not confident I did it correctly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my ReplayMemory class:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class ReplayMemory():&#xA;    def __init__(self, capacity):&#xA;        self.capacity = capacity&#xA;        self.memory = []&#xA;        self.count = 0&#xA;&#xA;    def push(self, experience):&#xA;        if len(self.memory) &amp;lt; self.capacity:&#xA;            self.memory.append(experience)&#xA;        else:&#xA;            self.memory[self.count % self.capacity] = experience&#xA;        self.count += 1&#xA;&#xA;    def sample(self, batch_size):&#xA;        return random.sample(self.memory, batch_size)&#xA;&#xA;    def can_provide_sample(self, batch_size):&#xA;        return len(self.memory) &amp;gt;= batch_size&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This basically stores states/rewards/actions and returns a random group when asked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, here is my learning code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def learning(num_episodes=20):&#xA;        global scores, experiences, target_vecs&#xA;        y = 0.8&#xA;        eps = 0&#xA;        decay_factor = 0.9999&#xA;&#xA;for i in range(num_episodes):&#xA;    state = GAME.reset()&#xA;    GAME.done = False&#xA;    done = False&#xA;    counter = 0&#xA;    while not done:&#xA;        eps *= decay_factor&#xA;        counter+=1&#xA;&#xA;        if np.random.random() &amp;lt; eps:&#xA;            a = np.random.randint(0, 2)&#xA;        else:&#xA;            a = np.argmax(model.predict(np.array([scale(state)])))&#xA;&#xA;        new_state, reward, done = GAME.step(a) #does that step&#xA;        REPLAY_MEMORY.push((scale(state), a, reward, scale(new_state)))&#xA;&#xA;        #experience replay is here&#xA;        if REPLAY_MEMORY.can_provide_sample(20):&#xA;            experiences = REPLAY_MEMORY.sample(20)&#xA;            target_vecs = []&#xA;            for j in range(len(experiences)):&#xA;                target = experiences[j][2] + y * np.max(model.predict(np.array([experiences[j][3]])))&#xA;                target_vec = model.predict(np.array([experiences[j][0]]))[0]&#xA;                target_vec[experiences[j][1]] = target&#xA;                target_vecs.append(target_vec)&#xA;            target_vecs = np.array(target_vecs)&#xA;            states = [s for s, _, _, _ in [exp for exp in experiences]]&#xA;            states = np.array(states)&#xA;            model.fit(states, target_vecs, epochs=1, verbose=1 if counter % 100 == 0 else 0)&#xA;        state = new_state&#xA;        if counter &amp;gt; 1200: #game runs for 20 seconds each episode&#xA;            done = True&#xA;            scores.append(GAME.PLAYER.score)&#xA;model.save(&quot;model.h5&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First, this takes a long time to train on my GTX1050. Is this normal for such a simple game? Also, does my code look fine? This is my first time with Deep Q Learning, so I would appreciate a second set of eyes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is happening is that training is super slow (more than an hour for 20 episodes (or 400 seconds of actual game play)). Also, it does not seem to get much better. The paddle (after 20 episodes) moves left and right but without any obvious pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://repl.it/repls/DarkorchidTurbulentPolygon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a link to the code.&lt;/a&gt; Also, available on &lt;a href=&quot;https://github.com/msolonko/first_ai_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27681" LastEditorUserId="27681" LastEditDate="2019-08-06T21:48:38.757" LastActivityDate="2019-08-10T16:44:28.450" Title="Deep Q Learning for Simple Game Not Effective" Tags="&lt;deep-learning&gt;&lt;game-ai&gt;&lt;q-learning&gt;&lt;keras&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13807" PostTypeId="2" ParentId="13805" CreationDate="2019-08-06T05:41:21.837" Score="0" Body="&lt;p&gt;I'm not sure it's possible. Untrained CNN means it has random kernel values. Let's say you have a kernel with size 3x3 like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 0 0&#xA;0 0 0&#xA;0 0 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't think it is possible for that kernel to provide good information about the image. on the contrary, the kernel eliminates a lot of information. We cannot rely on random values for feature extraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if you use CNN with &quot;assigned&quot; kernel, then you don't need to train the convolutional layer. For example, you can start a CNN with a kernel that designed to extract vertical line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1 2 -1&#xA;-1 2 -1&#xA;-1 2 -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-08-06T05:41:21.837" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13808" PostTypeId="1" AcceptedAnswerId="13813" CreationDate="2019-08-06T09:15:25.577" Score="1" ViewCount="343" Body="&lt;p&gt;As per subject title, are there ways to try Deep Learning without downloading and installing anything? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm just trying to have a feel of how this work, not really want to go through the download and install step if possible. &lt;/p&gt;&#xA;" OwnerUserId="26942" LastEditorUserId="26942" LastEditDate="2019-08-06T11:29:58.383" LastActivityDate="2019-08-06T17:32:32.413" Title="Are there ways to learn and practice Deep Learning without downloading and installing anything?" Tags="&lt;deep-learning&gt;&lt;architecture&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13810" PostTypeId="2" ParentId="13808" CreationDate="2019-08-06T10:44:22.837" Score="0" Body="&lt;p&gt;You can definitely get a good handle on the theory of various concepts in ML(I.e the agent-environment loop and Markov Decision Processes) but true understanding(for the vast majority of people) will only come through application of the aforementioned theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest something like &lt;a href=&quot;https://www.youtube.com/watch?v=TjZBTDzGeGg&amp;amp;t=4s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; course to get your feet wet in ML&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-08-06T10:44:22.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13811" PostTypeId="2" ParentId="13805" CreationDate="2019-08-06T10:57:34.543" Score="4" Body="&lt;p&gt;Yes, it has been demonstrated that the main factor for CNNs to work is its architecture, which exploits locality during the feature extraction. A CNN with random weights will do a random partition of the feature space, but still with that spatial prior that works so well, so those random features are OK for classification (and sometimes even better than trained ones, as they don't introduce additional bias).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more in these papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1711.10925&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. Ulyanov et al. Deep Image Prior&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1802.00844&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Rosenfeld and J. K. Tsotsos. Intriguing Properties of Randomly Weighted Networks: Generalizing While Learning Next to Nothing&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27444" LastEditorUserId="27444" LastEditDate="2019-08-06T11:14:10.187" LastActivityDate="2019-08-06T11:14:10.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13812" PostTypeId="2" ParentId="13797" CreationDate="2019-08-06T11:07:59.483" Score="1" Body="&lt;p&gt;For the vast majority of cases where you have a dynamic(and assumed non-linear) relationship between your input and output, you would not use modified architecture. You would simply retrain on the new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, based on domain knowledge or intuition, one might put a &quot;weight&quot; on the new data to increase or decrease its importance relative to previous data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some attempts(mostly by those studying one-shot learning) to create NNs that quickly fit to new data effectively with only a few samples. However, most of these are not ready for anything resembling real-world problems(particularly on tabular data).&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-08-06T11:07:59.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13813" PostTypeId="2" ParentId="13808" CreationDate="2019-08-06T11:43:00.133" Score="4" Body="&lt;p&gt;As I understand, I think you wish to directly try out some deep learning stuff and things like library downloading, tools downloading, and managing all these really stop you from even starting to try out deep learning experiments.If this is what you asked for:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Google Colab&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is the best place for you.&#xA;Anyone with a Google Drive account can sign up for Colab by heading to &#xA; and following the listed instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you mentioned that you just wanted to try out and practice stuffs, this would be &#xA;ideal for you.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;All major Python libraries like TensorFlow, Scikit-learn, Matplotlib among many &#xA;others &#xA;are pre-installed and ready to be imported.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Built on top of Jupyter Notebook&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please have a quick look at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/lean-in-women-in-tech-india/google-colab-the-beginners-guide-5ad3b417dfa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/lean-in-women-in-tech-india/google-colab-the-beginners-guide-5ad3b417dfa&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;2.Microsoft Azure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Azure free account is available to all new customers of Azure. If you have never &#xA;  tried or paid for Azure before, you’re eligible.&#xA;  Try out a student account:&#xA;   &lt;a href=&quot;https://azure.microsoft.com/en-in/free/students/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://azure.microsoft.com/en-in/free/students/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps, go ahead with practicing deep learning &#xA;if not please feel free to raise questions, always ready to help&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="26854" LastActivityDate="2019-08-06T11:43:00.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13814" PostTypeId="2" ParentId="13775" CreationDate="2019-08-06T11:47:12.527" Score="0" Body="&lt;p&gt;It sounds like you have invested 1 year for data science with R, and embedded into R environment, but want to explore python for data science. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First learn the basics of the python like how lists and tuple works and how classes and objects work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then get your hands dirty with some libraries like numpy matplotlib pandas. Learn tensorflow or keras and then go for data science. &lt;/p&gt;&#xA;" OwnerUserId="27728" LastEditorUserId="1671" LastEditDate="2019-08-06T21:21:53.977" LastActivityDate="2019-08-06T21:21:53.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13815" PostTypeId="2" ParentId="13775" CreationDate="2019-08-06T12:02:06.503" Score="0" Body="&lt;p&gt;This is totally my personal opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read in my office (at a construction site) that &quot;There is a right tool for every task.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect me to face a variety of tasks, as a programmer. I want as many tools as I can &quot;buy or invest in&quot;, as possible. One day one tool will help me solve it, some other day some other tool. R (for statistics) and Python (for in general) are two tools I definitely want with me and I think it is worth investment for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as switch is concerned, I will use the most efficient tool I know (where efficiency is measured by client's requirement, time and cost investment and ease of coding) . The more tools I know, the merrier! Of course there is a practical limit to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this is my personal opinion and not necessarily correct.&lt;/p&gt;&#xA;" OwnerUserId="27729" LastActivityDate="2019-08-06T12:02:06.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13816" PostTypeId="2" ParentId="13775" CreationDate="2019-08-06T12:04:17.830" Score="4" Body="&lt;p&gt;I would say yes. Python is better than R for most tasks, but R has its niche and you would still want to use it in many circumstances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, learning a second language will improve your programming skills.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own perspective on the strengths of R vs Python is that I would prefer R for a small, single-purpose program involving tables or charts, or exploratory work in the same vein. I would prefer Python for everything else.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;R is &lt;strong&gt;really&lt;/strong&gt; good for table mashing. If most of what a particular program is going to do is smoosh some tables into different shapes, then R is the thing to pick. Python has tools for this, but R is designed for it and does it better.&lt;/li&gt;&#xA;&lt;li&gt;It's worth switching to R whenever you need to make a chart, because &lt;em&gt;ggplot2&lt;/em&gt; is a masterpiece of API usability and &lt;em&gt;matplotlib&lt;/em&gt; is a crawling horror.&lt;/li&gt;&#xA;&lt;li&gt;Python is well designed for general purpose programming. It has a very well designed set of standard data structures, standard libraries, and control flow statements.&lt;/li&gt;&#xA;&lt;li&gt;R is poorly suited for general purpose programming. It doesn't handle tree-structured or graph-structured data well. It has some rules (like being able to look into and modify your parent scope) which are immediately convenient, but when used lead to programs that do are hard to grow, modify, or compose.&lt;/li&gt;&#xA;&lt;li&gt;R also has some straightforwardly bad things in it. These are mostly just historical leftovers like the three different object systems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To elaborate more on the last point: computer programming done well is lego where you make your own bricks (functions and modules). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programs are usually modified and repurposed past their original design. As you build them it is useful to think about which parts might be reused, and to build those part in a general way that will let them plug in to the other bricks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R encourages you to melt all the bricks together.&lt;/p&gt;&#xA;" OwnerUserId="27727" LastActivityDate="2019-08-06T12:04:17.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13817" PostTypeId="2" ParentId="9312" CreationDate="2019-08-06T12:24:57.623" Score="2" Body="&lt;p&gt;There is indeed an investigation in progress, regarding this topic. A first publication from last march noted that modularity has been done, although not explicitly, since some time ago, but somehow training keeps being monolithic. This paper assess some primary questions about the matter and compares training times and performances on modular and heavily recurrent neural networks. See:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ceur-ws.org/Vol-2350/paper8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Castillo-Bolado et al. Modularity as a Means for Complexity Management in Neural Networks Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some others are very focused on modularity, but staying with the monolithic training (see &lt;a href=&quot;https://people.eecs.berkeley.edu/~jda/pubs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jacob Andrea's research&lt;/a&gt;, specially &lt;em&gt;Learning to reason&lt;/em&gt; is very related to your third question). Somewhere between late 2019 and march next year, there should be more results (I happen to know).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In relation to your two last questions, we're starting to see now that modularity is a major key towards generalisation. Let me recommend you some papers (you can find them all in arxiv or google scholar):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Stochastic Adaptive Neural Architecture Search for Keyword Spotting (variations of an architecture to balance performance and resource usage).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Making Neural Programming Architectures Generalize via Recursion (they do task submodularity and I believe it's the first time that generalisation is guaranteed within field of neural networks).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mastering the game of Go with deep neural networks and tree search (network topology is actually the search tree itself, you can see more of this if you look for graph neural networks).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27444" LastEditorUserId="27444" LastEditDate="2019-08-06T12:36:26.893" LastActivityDate="2019-08-06T12:36:26.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13818" PostTypeId="2" ParentId="13775" CreationDate="2019-08-06T12:50:11.093" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Person who chases two rabbits catches neither&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And yes, Python is more popular. I work in both but, business speaking, it's easy to find a job on Python than in R.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you could:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pick Python because it is more popular.   However, you must start from scratch.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stay with R, after all, you have one year worth of training with R.  But it is not popular.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27731" LastActivityDate="2019-08-06T12:50:11.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13819" PostTypeId="1" AcceptedAnswerId="13825" CreationDate="2019-08-06T13:26:55.690" Score="1" ViewCount="187" Body="&lt;p&gt;I have created a &lt;code&gt;Tf.Sequential&lt;/code&gt; model which outputs 1 for numbers bigger then 5 and 0 otherwise:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const model = tf.sequential();&#xA;model.add(tf.layers.dense({ units: 5, activation: 'sigmoid', inputShape: [1]}));&#xA;model.add(tf.layers.dense({ units: 1, activation: 'sigmoid'}));&#xA;model.compile({loss: 'meanSquaredError', optimizer: 'sgd'});&#xA;const xs = tf.tensor2d([[1], [2], [3], [4], [6], [7], [8], [9]]);&#xA;const ys = tf.tensor2d([[0], [0], [0], [0], [1], [1], [1], [1]]);&#xA;model.fit(xs, ys);&#xA;model.predict(xs).print();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With 5 hidden neurons, not even the right trend is detected. Sometimes all the number are too low, or the outputs decrease even if the inputs increase or the outputs are too high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I later thought that the best way to do this is to have 2 neurons, where 1 is for the input and the other applies a sigmoid function to the input. The weight and bias should easily be adjusted to make the ANN work.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const model = tf.sequential();&#xA;model.add(tf.layers.dense({ units: 1, activation: 'sigmoid', inputShape: [1]}));&#xA;model.compile({loss: 'meanSquaredError', optimizer: 'sgd'});&#xA;const xs = tf.tensor2d([[1], [2], [3], [4], [6], [7], [8], [9]]);&#xA;const ys = tf.tensor2d([[0], [0], [0], [0], [1], [1], [1], [1]]);&#xA;model.fit(xs, ys);&#xA;model.predict(xs).print();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sometimes, this ANN does detect the right trend (the higher the input, the higher the output), but still, the results are never correct and are usually simply too high, always providing an output too close to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I make my ANN work, and what have I done wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code I'm using now, same problem as before:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const AdadeltaOptimizer = tf.train.adadelta();&#xA;&#xA;const model = tf.sequential();&#xA;model.add(tf.layers.dense({ units: 5, activation: 'sigmoid', inputShape: [1]}));&#xA;model.add(tf.layers.dense({ units: 1, activation: 'sigmoid'}));&#xA;model.compile({loss: 'meanSquaredError', optimizer: AdadeltaOptimizer});&#xA;const xs = tf.tensor1d([1, 2, 3, 4, 5, 6, 7, 8, 9]);&#xA;const ys = tf.tensor1d([0, 0, 0, 0, 0, 1, 1, 1, 1]);&#xA;model.fit(xs, ys, {&#xA;epochs: 2000,&#xA;});&#xA;model.predict(xs).print();&#xA;&#xA;tf.losses.meanSquaredError(ys, model.predict(xs)).print();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21788" LastEditorUserId="198" LastEditDate="2019-08-14T08:41:54.883" LastActivityDate="2019-08-14T15:50:51.813" Title="Why is this simple neural network not training?" Tags="&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13820" PostTypeId="1" AcceptedAnswerId="13851" CreationDate="2019-08-06T13:58:36.640" Score="1" ViewCount="82" Body="&lt;p&gt;In the image given below, I do not understand a few things&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Why is an entire area colored to signify misclassification? For the given decision boundary, only the points between &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; and the decision boundary signify misclassification right? It's supposed to be only a set of points on the x-axis, not an area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Why is the green area with &lt;span class=&quot;math-container&quot;&gt;$x &amp;lt; x_0$&lt;/span&gt; a misclassification? It's classified as &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; and it is supposed to be &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Similarly, why is the blue area a misclassification? Any &lt;span class=&quot;math-container&quot;&gt;$x &amp;gt;$&lt;/span&gt; the decision boundary belongs to &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt; and is also classified as such...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/THTLv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/THTLv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27422" LastEditorUserId="16708" LastEditDate="2019-08-16T14:08:06.197" LastActivityDate="2020-09-09T16:04:15.340" Title="Why is the entire area of a join probability distribution considered when it comes to calculating misclassification?" Tags="&lt;machine-learning&gt;&lt;probability&gt;&lt;decision-theory&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13821" PostTypeId="1" CreationDate="2019-08-06T14:25:18.957" Score="4" ViewCount="1508" Body="&lt;p&gt;I have implemented a CNN for image classification. I have not used fully connected layers, but only a softmax. Still, I am getting results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Must I use fully-connected layers in a CNN?&lt;/p&gt;&#xA;" OwnerUserId="27735" LastEditorUserId="2444" LastEditDate="2019-08-06T22:35:10.900" LastActivityDate="2020-06-12T16:03:06.430" Title="Are fully connected layers necessary in a CNN?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13822" PostTypeId="2" ParentId="13821" CreationDate="2019-08-06T14:41:46.437" Score="1" Body="&lt;p&gt;In theory, you do not need fully-connected (FC) layers. FC layers are used to introduce scope for updating weights during back-propagation, due to its ability to introduce more connectivity possibilities, as every neuron of the FC is connected every neuron of the further layers.&lt;/p&gt;&#xA;" OwnerUserId="26115" LastEditorUserId="2444" LastEditDate="2019-08-06T22:38:15.707" LastActivityDate="2019-08-06T22:38:15.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13823" PostTypeId="2" ParentId="13821" CreationDate="2019-08-06T14:42:41.743" Score="1" Body="&lt;p&gt;The reasons people use the FC after convolutional layer is that CNN preserves spatial information. You said you use softmax, so you probably make some classification task. If you don't use FC layer, then you probably evaluate first class by first position of the first kernel, not by whole image with all kernels, second class by second position of the kernel and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dense net combine the info from all the kernels in all positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you technically can convert FC to Conv net, as &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#convert&quot; rel=&quot;nofollow noreferrer&quot;&gt;described here&lt;/a&gt;, so then you can said you &quot;skipped&quot; FC layer.&lt;/p&gt;&#xA;" OwnerUserId="16940" LastActivityDate="2019-08-06T14:42:41.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13824" PostTypeId="1" CreationDate="2019-08-06T14:49:32.830" Score="4" ViewCount="95" Body="&lt;p&gt;&lt;strong&gt;In a neural network, by how much does the # of neurons typically vary from layer to layer?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PLEASE NOTE:&lt;br&gt;&#xA;I am NOT asking how to find the optimal # of neurons per lyr.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a hardware design engineer with no practical experience programming neural networks, I would like to glean for example... &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;By how much does the number of neurons in hidden layers typically vary from that of the input layer?&lt;/li&gt;&#xA;&lt;li&gt;What is the maximum deviation in the # of hidden lyr neurons to the # of input layer neurons? &lt;/li&gt;&#xA;&lt;li&gt;How commonly do you see a large spike in the number of neurons?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It likely depends on the application so I would like to hear from as many people as possible. Please tell me about your experience.&lt;/p&gt;&#xA;" OwnerUserId="27726" LastActivityDate="2019-08-06T20:06:18.673" Title="Standard deviation in the # of neurons per layer" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13825" PostTypeId="2" ParentId="13819" CreationDate="2019-08-06T16:08:06.317" Score="3" Body="&lt;p&gt;You should try bigger learning rate / more epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you see is basically output of randomly initialized weights. Default learning rate for SGD is 0.001, it's definitely not enough for weights to change quick enough with single batch epochs and 9 examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I never used TF.js, but in keras (that's look very similar) you can set it like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sgd = keras.optimizers.SGD(lr=1)&#xA;model.compile(sgd, 'mse')&#xA;&#xA;model.fit(xs, ys, epochs=200)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to use adaptive learning rate optimizer, for example adadelta. It still requires quite a lot of epochs, but much less, then SGD with standard learning rate and you don't need to set LR manually.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.compile('adadelta', 'mse')&#xA;model.fit(xs, ys, epochs=2000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you can read how different optimizers work and converge &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6HZFE.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6HZFE.gif&quot; alt=&quot;Optimizers convergency&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;Another thing with tf.js (and javascript code in general) is it doesn't guarantee order of operations. So, in your case it prints evaluation &lt;em&gt;before&lt;/em&gt; it finishes training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code should perform fine. Take note that we evaluate model in &lt;code&gt;then&lt;/code&gt; clause.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const model = tf.sequential();&#xA;model.add(tf.layers.dense({ units: 5, activation: 'sigmoid', inputShape: [1]}));&#xA;model.add(tf.layers.dense({ units: 1, activation: 'sigmoid'}));&#xA;model.compile({loss: 'meanSquaredError', optimizer: 'adadelta'});&#xA;const xs = tf.tensor1d([1, 2, 3, 4, 5, 6, 7, 8, 9]);&#xA;const ys = tf.tensor1d([0, 0, 0, 0, 0, 1, 1, 1, 1]);&#xA;model.fit(xs, ys, {epochs: 1000}).then(h =&amp;gt; {&#xA;   console.log(&quot;Loss: &quot; + h.history.loss[0]);&#xA;   model.predict(xs).print();&#xA;});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also use async/await approach, for example as it described &lt;a href=&quot;https://stackoverflow.com/a/50705155/1173441&quot;&gt;in this answer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16940" LastEditorUserId="16940" LastEditDate="2019-08-14T15:50:51.813" LastActivityDate="2019-08-14T15:50:51.813" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13826" PostTypeId="1" AcceptedAnswerId="13828" CreationDate="2019-08-06T16:11:24.333" Score="3" ViewCount="188" Body="&lt;p&gt;I have just watched a few videos on Tedtalks talking about how AI benefits creatives and artists but none of the videos I watched provided further resources for reference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have only came across mentioning of the creative usage of AI but would like to know how creatives and artists can apply AI in their work process. Like at least a tutorial guide on how it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any recommendations on communities, tutorials, guides, platforms and real world AI applications that are meant for creatives and artists?&lt;/p&gt;&#xA;" OwnerUserId="26942" LastEditorUserId="26942" LastEditDate="2019-08-06T16:16:33.713" LastActivityDate="2019-10-13T15:51:24.130" Title="Applications of AI for creatives and artists" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13827" PostTypeId="2" ParentId="13824" CreationDate="2019-08-06T16:18:45.690" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Input layers will always have the dimensionality of your input data(for every model I can think of).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;See above, the deviation between hidden layers can be significant. For example, 128 in the first hidden and 64 in the rest(or vice versa).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This question in particular will always be problem dependent. It is decided via architecture search or intuition/experience combined with some exploratory search.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2019-08-06T20:06:18.673" LastActivityDate="2019-08-06T20:06:18.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13828" PostTypeId="2" ParentId="13826" CreationDate="2019-08-06T17:16:19.433" Score="3" Body="&lt;p&gt;The machine learning (~AI) is all about code and data. If you planning to create something really unique, you'll probably need to, well, work with code and data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll probably should look at github and try to get inspiration from opensourced projects. If you found some AI-related artwork, try to google it's name with 'github'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to look at medium, it's usually have some post/tutorial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The art is very broad topic and it's hard to cover it all. I try to provide examples of different resources that could help you to start.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Google Research &lt;a href=&quot;https://research.google.com/seedbank/&quot; rel=&quot;nofollow noreferrer&quot;&gt;made some examples&lt;/a&gt;, where you run code in colab without installing anything locally. It helps you to try basic stuff. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is tensorflow.js, that allows to launch code at browser. You can look at &lt;a href=&quot;https://www.tensorflow.org/js/demos&quot; rel=&quot;nofollow noreferrer&quot;&gt;demos with code&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is intro course for visual artist on &lt;a href=&quot;https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow-iv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kadenze&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/vibertthio/awesome-machine-learning-art#learning-resources&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here it is&lt;/a&gt; some more links on different courses. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/carpedm20/DCGAN-tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;DCGAN&lt;/a&gt; - very popular image generation library. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TouchDesigner, very popular tool among visual artists, provides python interface. For example, &lt;a href=&quot;https://github.com/mlarthack/DCGAN_TouchDesigner&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is usage&lt;/a&gt; of aforementioned DCGAN with it &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, there is creative section on NIPS(one of the biggest ML conferences) &lt;a href=&quot;http://nips4creativity.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nips4creativity.com/&lt;/a&gt;. Some of the work provides detailed explanation, like &lt;a href=&quot;http://paulsoulos.com/editorial/2017/07/18/noise-in-design.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know, starting to code could be intimidating, but many github repos provides some description about how to launch them and you don't even have to know how they work exactly to use them. Also, it would help to have a person with programming experience, who could help you to deal with code.&lt;/p&gt;&#xA;" OwnerUserId="16940" LastEditorUserId="16940" LastEditDate="2019-10-13T15:51:24.130" LastActivityDate="2019-10-13T15:51:24.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13829" PostTypeId="2" ParentId="13808" CreationDate="2019-08-06T17:32:32.413" Score="0" Body="&lt;p&gt;Have you tried the Tensorflow playground?  It allows you turn all of the knobs and see the effect without having to code anything&lt;br&gt;&#xA;&lt;a href=&quot;https://playground.tensorflow.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://playground.tensorflow.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27726" LastActivityDate="2019-08-06T17:32:32.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13830" PostTypeId="1" AcceptedAnswerId="13833" CreationDate="2019-08-06T17:39:17.890" Score="1" ViewCount="70" Body="&lt;p&gt;I recently came across this &lt;a href=&quot;https://www.analyticsindiamag.com/iisc-bangalores-ai-team-comes-up-with-a-solution-to-an-old-nlp-problem/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; which cites a &lt;a href=&quot;https://malllabiisc.github.io/publications/papers/EWISE_ACL19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; which apparently won outstanding paper in ACL 2019. The theme is that it solved a longstanding problem called &lt;strong&gt;Word Sense Disambiguation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the expression &lt;em&gt;Word Sense Disambiguation&lt;/em&gt; mean? How does it affect NLP? And also how does the proposed method solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-08-06T19:57:08.197" LastActivityDate="2019-08-06T20:26:33.173" Title="What is &quot;Word Sense Disambiguation&quot;?" Tags="&lt;natural-language-processing&gt;&lt;terminology&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13831" PostTypeId="2" ParentId="13824" CreationDate="2019-08-06T18:00:51.283" Score="0" Body="&lt;p&gt;There is no right answer to this question. But, I would like to point you to &lt;a href=&quot;https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw&quot;&gt;an answer on CV&lt;/a&gt; that addresses the &lt;em&gt;mean&lt;/em&gt; of your problem.&lt;/p&gt;&#xA;&lt;p&gt;Two points from the accepted answer that I want to draw your attention to are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a) There are some empirically-derived rules-of-thumb, of these, the most commonly relied on is &lt;em&gt;'the optimal size of the hidden layer is usually between the size of the input and size of the output layers'&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;b) In sum, for most problems, one could probably get decent performance (even without a second optimization step) by setting the hidden layer configuration using just two rules: (i) number of hidden layers equals one; and (ii) the number of neurons in that layer is the mean of the neurons in the input and output layers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Other answers in the thread are also very insightful. I will recommend you to go through the answers and figure out the standard deviation and just assume things are normal, and hence you would have your distribution.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-06T18:16:51.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13832" PostTypeId="1" CreationDate="2019-08-06T19:52:50.630" Score="3" ViewCount="497" Body="&lt;p&gt;As everyone experienced in deep learning might know, in an image classification problem we normally add borders to images then resize it to the input size of a CNN network. The reason of doing this is to keep aspect ratio of the original image and retain it's information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen people fill black (0 pixel value for each channel), gray (127 pixel value for each channel), or random value generated from gaussian distribution to the border. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there any proof that which of these is correct?&lt;/p&gt;&#xA;" OwnerUserId="25552" LastEditorUserId="2444" LastEditDate="2019-08-06T22:42:15.217" LastActivityDate="2019-09-19T17:01:15.120" Title="How should we pad an image to be fed in a CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13833" PostTypeId="2" ParentId="13830" CreationDate="2019-08-06T20:14:52.030" Score="1" Body="&lt;p&gt;&quot;Word Sense Disambiguation&quot; refers to the idea that words can have different meanings in different contexts. Here are some examples&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;I went to the river river &lt;strong&gt;bank&lt;/strong&gt;&quot; vs &quot;I deposited my check at the &lt;strong&gt;bank&lt;/strong&gt;&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;He's &lt;strong&gt;mad&lt;/strong&gt; good at that game&quot; vs &quot;I am so &lt;strong&gt;mad&lt;/strong&gt; at you&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How it effects NLP, comes down to the way we process text. This generally includes the steps of tokenization and embedding them into some form of vector space. These embeddings in many cases are trained either through some self supervised task on some corpora (examples include Word2Vec or Glove) or doing it from scratch under whichever task/data-set is being used.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now regarding that paper it does not &lt;em&gt;solve&lt;/em&gt; the problem but it does extend a new methodology that assists in helping achieve a better learned generalizable representation for this task. The way I interpreted it is that they don't just use a sense-label (which would be a one-hot encoding / what they call discrete) but instead use a continuous sense representation and do the comparison there. This difference allows for words with similar but different senses to not be equidistant from words with completely different senses. &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-06T20:14:52.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13834" PostTypeId="2" ParentId="13830" CreationDate="2019-08-06T20:20:20.993" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.mt-archive.info/SEMEVAL-2013-Navigli.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Word Sense Disambiguation (WSD)&lt;/a&gt; is the task of associating meanings or senses (from an existing collection of meanings) with words, given the context of the words. (The word &lt;em&gt;sense&lt;/em&gt; is a synonym for meaning.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the noun &quot;tie&quot; in the following two sentences&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;He wore a vest and a tie.&lt;/li&gt;&#xA;&lt;li&gt;Their record was 3 wins, 6 losses, and one tie.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In these two sentences, the meaning of the word &lt;em&gt;tie&lt;/em&gt; is different. In sentence 1, the word &lt;em&gt;tie&lt;/em&gt; refers to a necktie, which is a piece of cloth. In sentence 2, the word &lt;em&gt;tie&lt;/em&gt; is a synonym for a draw, so it refers to a situation of a game. Therefore, we could associate the meaning (or sense) &quot;neckwear consisting of a long narrow piece of material&quot; to the word &lt;em&gt;tie&lt;/em&gt; in the first sentence and the meaning &quot;the finish of a contest in which the winner is undecided&quot; to the same word in the second sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of WSD is thus to predict the appropriate sense or meaning of a word, given the context of the word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why is WSD important in NLP?&lt;/em&gt; Of course, there are many words that change meaning depending on the context, so WSD is important because you expect NLP algorithms and models to be able to correctly give meanings to words, given their context. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-06T20:26:33.173" LastActivityDate="2019-08-06T20:26:33.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13836" PostTypeId="1" AcceptedAnswerId="13846" CreationDate="2019-08-07T01:38:40.540" Score="2" ViewCount="200" Body="&lt;p&gt;I read different articles and keep getting confused on this point. Not sure if the literature is giving mixed information or I'm interpreting it incorrectly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So from reading articles my understanding (loosely) for the following terms are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Epoch:&lt;/strong&gt;&#xA;One Epoch is when an ENTIRE dataset is passed forward and backward through the neural network only ONCE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Batch Size:&lt;/strong&gt;&#xA;Total number of training examples present in a single batch. In real life scenarios of utilising neural nets, the dataset needs to be as large as possible, for the network to learn better. So you can’t pass the entire dataset into the neural net at once (due to computation power limitation). So, you divide dataset into Number of Batches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Iterations:&lt;/strong&gt;&#xA;Iterations is the number of batches needed to complete one epoch. We can divide the dataset of 2000 examples into batches of 500 then it will take 4 iterations to complete 1 epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if all is correct, then my question is, at what point does the loss/cost function and the subsequent backprop processes take place (assuming from my understanding that backprop takes place straight after the loss/cost is calculated)? Does the cost/loss function gets calculated:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;At the end of each batch where the data samples in that batch have been forward-fed to the network (i.e. at each &quot;Iteration, not each Epoch&quot;)? If so, then the loss/cost functions gets the average loss of all losses of all data samples in that batch, correct?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At the end of each Epoch? Meaning all the data samples of all the batches are forward-fed first, before the a cost/loss function is calculated. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My understanding is that it's the first point, i.e. at the end of each batch (passed to the network), hence at each iteration (not Epoch). At least when it comes to SGD optimisation. My understanding is - the whole point is that you calculate loss/cost and backprop for each batch. That way you're not calculating the average loss of the entire data samples. Otherwise you would get a very universal minima value in the cost graph, rather than local minima with lower cost from each batch you train on separately. Once all iterations have taken place, then that would count as 1 Epoch. &#xA;But then I was watching a YouTube video explaining Neural Nets, which mentioned that the cost/loss function is calculated at the end of each Epoch, which confused me. Any clarification would be really appreciated. &lt;/p&gt;&#xA;" OwnerUserId="25360" LastActivityDate="2019-08-07T12:23:23.077" Title="Clarifying when a loss/cost function calculation and backprop take place" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13837" PostTypeId="1" AcceptedAnswerId="13841" CreationDate="2019-08-07T02:43:06.217" Score="2" ViewCount="91" Body="&lt;p&gt;The work I've seen so far have the nodes containing features. Any resources for how to use a GCN on a graph where the edges are the ones that contain features rather than the nodes?&lt;/p&gt;&#xA;" OwnerUserId="27240" LastActivityDate="2019-08-07T10:00:09.797" Title="How are edge features implemented in Geometric Deep Learning?" Tags="&lt;deep-learning&gt;&lt;geometric-deep-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13838" PostTypeId="1" CreationDate="2019-08-07T04:17:09.163" Score="1" ViewCount="44" Body="&lt;p&gt;I am not sure what are the common loss functions people usually use when training a student in a teacher-student learning model. Any insight on this is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="25989" LastEditorUserId="2444" LastEditDate="2019-08-07T10:16:23.383" LastActivityDate="2019-08-07T10:16:23.383" Title="What are the loss functions used in teacher-student learning models?" Tags="&lt;loss-functions&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13840" PostTypeId="1" CreationDate="2019-08-07T09:21:26.147" Score="2" ViewCount="22" Body="&lt;p&gt;I am very new to AI/ML but have lot of interest in these. I am trying to understand how &lt;a href=&quot;https://www.fujitsu.com/jp/group/kyushu/en/solutions/industry/agriculture/gyuho/feature/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; gadget works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DUAg5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DUAg5.jpg&quot; alt=&quot;Wearable device features&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have understood that a NN model of the cattle is generated by offline classification of the tagged data which is received from the wearable sensor. Consequently some ML algorithms are used to generate a model of a cattle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That model is then embedded in the programmable wearable device. The device then sends the real-time tagged (classified, parameterized) data to the server.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am looking for a sample NN-model of a cattle. I wonder how does a NN-model of a cattle would look like?&lt;/p&gt;&#xA;" OwnerUserId="26033" LastActivityDate="2019-08-07T09:21:26.147" Title="A NN based model of a Cattle for 'Heat Detection'" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;classification&gt;&lt;training&gt;&lt;models&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13841" PostTypeId="2" ParentId="13837" CreationDate="2019-08-07T09:54:57.220" Score="1" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1704.01212.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Message Passing for Quantum Chemistry&lt;/a&gt; (2017), the authors (from Google, Google Brain and Google DeepMind) introduce a framework called &lt;em&gt;message passing neural network&lt;/em&gt; (MPNN), which generalizes previously proposed geometric deep learning models. In section 2 of the paper, they describe this MPNN framework and they state that edge features can also be learned, using the MPNN framework, by introducing hidden states for all edges in the graph, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{h}_{e_{vw}}^t$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the iteration number and &lt;span class=&quot;math-container&quot;&gt;$e_{vw}$&lt;/span&gt; is the edge from node &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; to node &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the paper &lt;a href=&quot;https://arxiv.org/pdf/1702.05532.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine learning prediction errors better than DFT accuracy&lt;/a&gt;, which is cited in the MPNN paper as an example where the authors also learn the edge features. In section 2 of the MPNN paper, they briefly describe this specific instantiation of the MPNN framework, called the &lt;em&gt;Molecular Graph Convolutions&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-07T10:00:09.797" LastActivityDate="2019-08-07T10:00:09.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13842" PostTypeId="1" CreationDate="2019-08-07T09:59:10.067" Score="3" ViewCount="1466" Body="&lt;p&gt;So I've got a neural net model (ResNet-18) and made a diagram according to the literature (&lt;a href=&quot;https://arxiv.org/abs/1512.03385&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1512.03385&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I understand most of the format of the convolutional layers:&#xA;&lt;em&gt;filter dims&lt;/em&gt; ,conv, &lt;strong&gt;&lt;em&gt;unknown number&lt;/em&gt;&lt;/strong&gt; ,&lt;em&gt;stride&lt;/em&gt;(if applicable)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the number after 'conv' in the convolutional layers indicate? is it the number of neurons in the layer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pkPi3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pkPi3.png&quot; alt=&quot;ResNet-18 architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;bonus q: this is being used for unsupervised learning of images, i.e the embedding output a network produces for an image is used for clustering. Would this make it incorrect for my architecture to have an FC layer at the end (which would be used for classifcation)?&lt;/p&gt;&#xA;" OwnerUserId="27774" LastActivityDate="2019-08-07T10:10:35.693" Title="What do the numbers in this CNN architecture stand for?" Tags="&lt;convolutional-neural-networks&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13843" PostTypeId="2" ParentId="13832" CreationDate="2019-08-07T10:04:46.933" Score="0" Body="&lt;p&gt;I've more often seen image resizing than padding to be honest and tend to resize the images. Maybe it's because datasets I've used have images with near equal aspect ratios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One major exception was when I worked with MR images. These were orthogonal and it would be wrong to mess up the aspect ratio. However, in this domain images have black borders everywhere, so a zero-padding was easy to apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common use of padding I've seen is for data augmentations (to fill values gone due to translations, rotations, shifting etc.). In this regard, I've used many types of paddings (constant value, random value, 'same' padding, mirrored padding etc.) The best I've found to empirically work is zero-padding but I don't think that you'll ever find a &lt;em&gt;proof&lt;/em&gt; for this. I like to think of it as a &lt;em&gt;hyperparameter&lt;/em&gt;; different padding strategies maybe work better for different tasks. Though I think that zero-padding is the safest (there is a small chance of messing things up). &lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-08-07T10:04:46.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13844" PostTypeId="2" ParentId="13842" CreationDate="2019-08-07T10:10:35.693" Score="1" Body="&lt;p&gt;This number refers to the number of kernels (or feature maps) that are convolved with the input. So, for example, in the first convolutional layer, &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; kernels are convolved with the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ResNet presented in &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Residual Learning for Image Recognition&lt;/a&gt; is used for image classification. Furthermore, note that your diagram already contains a fully connected layer at the end.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-07T10:10:35.693" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13846" PostTypeId="2" ParentId="13836" CreationDate="2019-08-07T12:23:23.077" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Epoch: One Epoch is when an ENTIRE dataset is passed forward and backward through the neural network only ONCE.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Batch Size: Total number of training examples present in a single&#xA;  batch. In real life scenarios of utilising neural nets, the dataset&#xA;  needs to be as large as possible, for the network to learn better. So&#xA;  you can’t pass the entire dataset into the neural net at once (due to&#xA;  computation power limitation). So, you divide dataset into Number of&#xA;  Batches.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Iterations: Iterations is the number of batches needed to complete one&#xA;  epoch. We can divide the dataset of 2000 examples into batches of 500&#xA;  then it will take 4 iterations to complete 1 epoch.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is for the most part correct, except there are other reasons you would sometimes want to use batches (even if you could fit the whole thing in memory). One is that its less likely to overfit in a stochastic setting then in the full setting. Another is that it can achieve similar extrema with faster convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now regarding your question, yes you apply the gradient descent step at the end of each batch or desired batch (what I mean by desired batch, is if you want to use a batch-size of 24 but your device can only process 8, you may use gradient accumulation of 3 pseudo-batches to achieve an emulated batch of 24). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though I do think its worth mentioning, your goal is still to find the global minimum. Even if your batch size is the &lt;strong&gt;entire&lt;/strong&gt; dataset, that does not mean you will not fall into a local minimum, its actually in most cases still the most probable outcome&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-07T12:23:23.077" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="13847" PostTypeId="1" CreationDate="2019-08-07T13:25:17.847" Score="2" ViewCount="82" Body="&lt;p&gt;I am applying a reinforcement learning agent (PPO2, stable baselines implementation) to a custom built environment using OpenAI Gym. One reward function (formualted as loss function, that is, all rewards are negative) I tested is of type &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt;. During training, it can happen that not only one but several actions are applied simulataneously to the environement before a reward is returned: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$s_t →a_{t,1}, a_{t,2}, a_{t,3} →s_{t+1}$&lt;/span&gt;   instead of   &lt;span class=&quot;math-container&quot;&gt;$s_t →a_t→s_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of all actions applied, only one is generated by the agent. The others are either a copy of the agent's action or are new values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I look at the tensorboard output of the trained agent, it looks rather horrific as displayed below (~ zero explained variance, key trainig values do not converge or behave weirdly, etc. etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the training did not really work. Now I wonder what the reason for that is.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it possible to train an agent using a reward function of type &lt;span class=&quot;math-container&quot;&gt;$R(s, a, s')$&lt;/span&gt; even if several actions are applied simulataneously or is this not possible at all? Other agents I trained using a reward function of type &lt;span class=&quot;math-container&quot;&gt;$R(s,a)$&lt;/span&gt; have a better tensorboard output so I guess that this is the problem.&lt;/li&gt;&#xA;&lt;li&gt;Or is maybe another reason more likely to be the root of the problem?  Like a bad observation space formulation or hyperparameter selection (both for RL algorithm and reward function used).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2Sr9f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2Sr9f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LstdT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LstdT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26876" LastEditorUserId="26876" LastEditDate="2019-08-07T15:14:48.650" LastActivityDate="2019-08-07T15:14:48.650" Title="Is it possible to use Reward Function of type R(s, a, s') if more than one action is applied?" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;rewards&gt;&lt;proximal-policy-optimization&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13848" PostTypeId="1" AcceptedAnswerId="17836" CreationDate="2019-08-07T16:06:29.393" Score="1" ViewCount="1935" Body="&lt;p&gt;As the question suggests, I'm trying to see if I can solve &lt;a href=&quot;https://gym.openai.com/envs/BipedalWalkerHardcore-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI's hardcore version of their gym's bipedal walker&lt;/a&gt; using OpenAI's &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a performance graph from my latest attempt, including the hyper parameters, along with some other attempts I've made. I realise it has been solved using other custom implementations (also utilising only dense layers in Tensorflow, not convolution), but I don't seem to understand why it seems so difficult to solve using OpenAI's implementation of DDPG? Can anyone please point out where I might be going wrong? Thank you so much for any help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Latest attempt's performance:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CD2KP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CD2KP.png&quot; alt=&quot;DDPG performance - Bipedal Walker Hardcore v2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Average score: &lt;strong&gt;about -75 to -80&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Env interacts: &lt;strong&gt;about 8.4mil (around 2600 epochs)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Batch size: &lt;strong&gt;64&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Replay memory: &lt;strong&gt;1000000&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Network: &lt;strong&gt;512, 256 (relu activation on inputs, tanh on outputs)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;All other inputs left to default&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Similar experiments yielded similar scores (or less), and included:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Network sizes of &lt;strong&gt;(400,300)&lt;/strong&gt;, &lt;strong&gt;(256,128)&lt;/strong&gt;, and &lt;strong&gt;(128,128,128)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Number of epochs ranging from &lt;strong&gt;500&lt;/strong&gt; all the way to &lt;strong&gt;100000&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Replay memory sizes all the way up to &lt;strong&gt;5000000&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Batch sizes of &lt;strong&gt;32&lt;/strong&gt;, &lt;strong&gt;64&lt;/strong&gt;, &lt;strong&gt;128&lt;/strong&gt;, and &lt;strong&gt;256&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;All of the above, with both &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG&lt;/a&gt; as well as &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/td3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you so much for any help! It would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="27392" LastEditorUserId="27392" LastEditDate="2019-08-07T17:15:28.350" LastActivityDate="2020-10-07T23:52:02.103" Title="Has anyone been able to solve OpenAI's hardcore bipedal walker with their implementation of DDPG?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13849" PostTypeId="1" CreationDate="2019-08-07T16:18:22.413" Score="2" ViewCount="28" Body="&lt;p&gt;Is it possible to use neuro-fuzzy systems for problems where ANNs are currently being used, for instance, when you have tabular data for regression or classification tasks? What kind of advantage can give me neuro-fuzzy systems over using an ANN for the mentioned tasks?&lt;/p&gt;&#xA;" OwnerUserId="27791" LastEditorUserId="2444" LastEditDate="2021-01-22T01:59:34.690" LastActivityDate="2021-01-22T01:59:34.690" Title="Can neuro-fuzzy systems be used for supervised learning tasks with tabular data?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;applications&gt;&lt;regression&gt;&lt;neuro-fuzzy-systems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13850" PostTypeId="1" CreationDate="2019-08-07T16:23:08.983" Score="4" ViewCount="280" Body="&lt;p&gt;I have a problem I would like to tackle with RL but I am not sure if it is even doable. My agent has to figure out how to fill a very large vector (let's say from 600 to 4000 in the most complex setting) made of natural numbers, i.e. a 600 vector [2000,3000,3500 ...] consisting of an energy profile for each timestep of a day, for each house in the neighborhood. I receive a reward for each of these possible combinations and my goal is, of course, that of maximizing the reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can start always from the same initial state, and I receive a reward every time any profile is chosen. I believe these two factors simplify the task, as I don't need to have large episodes to get a reward nor I have to take into consideration different states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I only have experience with DQN and I have never worked on Policy Gradient methods. So I have some questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- I would like to utilize the simplest method to implement, I considered DDPG however, I do not really need a target network or a critique network, as the state is always the same. Should I use a vanilla PG? - would REINFORCE be a good option?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- I get how PG methods work with discrete action space (using softmax and selecting one action - which then gets reinforced or discouraged based on reward), however, I don't get how it is possible to update a continuous value. In DQN or stochastic PG the output of the neural network is either a Q value or a probability value, and both can be directly updated via reward (the more reward the bigger the Qvalue/probability). However, I don't get how this happens in the continuous case, where I have to use the output of the model as it is. What would I have to change in this case in the loss function for my model?&lt;/p&gt;&#xA;" OwnerUserId="23638" LastActivityDate="2019-08-07T16:23:08.983" Title="Policy gradient methods for continuous action space" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13851" PostTypeId="2" ParentId="13820" CreationDate="2019-08-07T18:08:11.180" Score="0" Body="&lt;p&gt;The misclassifications that &lt;em&gt;could arise if &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; is used as decision boundary&lt;/em&gt; are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Classifying a point as &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt; when actually it was &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; -- which will only happen when &lt;span class=&quot;math-container&quot;&gt;$x &amp;gt; \hat{x}$&lt;/span&gt; as only the points greater than &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; are being classified as &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Classifying a point as &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; when actually it was &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt; -- which will only happen when &lt;span class=&quot;math-container&quot;&gt;$x &amp;lt; \hat{x}$&lt;/span&gt; as only the points less than &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; are being classified to be from class &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2) Why is the green area with x &amp;lt; x0 a misclassification? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If we label a point -- the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; which is from the interval on the horizontal axis which corresponds to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;green area -- which is a probability&lt;/a&gt; as being from either class then there is a good chance that it could be misclassified. This is because both the &lt;a href=&quot;https://en.wikipedia.org/wiki/Joint_probability_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;joint distributions'&lt;/a&gt; curves are above the axis for that interval on the axis and have some area (probability) for that interval on the horizontal axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a positive probability that a point that was drawn from the interval (a subset of the sample space, an event), which corresponds to the green area, was drawn from either of the two joint distributions or belongs to either of the two classes &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt;. The probability is the area under the curve.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It's classified as C1 and it is supposed to be C1 right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The same data point could well be generated from multiple different probability distributions. Here, you have an illustration of two probability distributions for which the author is trying to find the optimal decision boundary that will minimize the misclassification &quot;region&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is not supposed to be from &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt;, even if the &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_density_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;density of the joint prob. distribution&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$(x, C_1)$&lt;/span&gt; has &lt;a href=&quot;https://math.stackexchange.com/questions/210630/what-does-the-value-of-a-probability-density-function-pdf-at-some-x-indicate&quot;&gt;higher values&lt;/a&gt;. It could well be from &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt;. And, when one classifies the same data point as being from class &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; and make an error. This is why the whole area under the curve &lt;span class=&quot;math-container&quot;&gt;$p(x, C_2)$&lt;/span&gt; has been painted green -- which means that there is some probability that a point could be from the distribution &lt;span class=&quot;math-container&quot;&gt;$p(x, C_2)$&lt;/span&gt; and if we blindly label all the points to be from &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; will certainly lead to some misclassifications. &lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-08-09T06:28:41.967" LastActivityDate="2019-08-09T06:28:41.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13852" PostTypeId="1" CreationDate="2019-08-07T20:56:33.707" Score="1" ViewCount="17" Body="&lt;p&gt;What is interval of the initial weights at Matlab neural network tool?Yes I know it is randomly but are there limitation?&lt;/p&gt;&#xA;" OwnerUserId="27803" LastActivityDate="2019-08-07T20:56:33.707" Title="What is interval of the initial weights at Matlab neural network tool?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13853" PostTypeId="2" ParentId="6231" CreationDate="2019-08-08T01:49:02.833" Score="1" Body="&lt;p&gt;In my implementation, I used a recursion system to calculate the output nodes. It works as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Assume a feed-forward network&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Only allow the &quot;add connection&quot; mutation to connect a node with another node &gt;that have a higher maximum distance from an input node. This should result in &gt;feed forward network, without much extra work. (Emergent properties are great!)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Define function &lt;code&gt;x&lt;/code&gt;, a recursive function that takes in a node number&lt;/li&gt;&#xA;&lt;li&gt;Define function &lt;code&gt;y&lt;/code&gt;, a second function that takes in a node and returns all the connections with that node as an output&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In the recursive function:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Call function &lt;code&gt;y&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Call function &lt;code&gt;x&lt;/code&gt; on function y outputs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the parameter for &lt;code&gt;x&lt;/code&gt; is any input node, return the node value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This was the most elegant way of implementing I could think of, and its a lot simpler than explicitly tracking all of the connections.&lt;/p&gt;&#xA;" OwnerUserId="27789" LastEditorUserId="27789" LastEditDate="2019-08-08T01:54:16.720" LastActivityDate="2019-08-08T01:54:16.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13854" PostTypeId="1" CreationDate="2019-08-08T07:31:32.770" Score="2" ViewCount="124" Body="&lt;p&gt;I'm quite new in ANNs. I intend to use ANNs for predicting spike points in time series right before they happen. I've already used LSTM for another scenario, and I know that they can be used in similar situations as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give me a piece of advice or some suitable resources that might be used as a beginning point? It would be much appreciated if it uses &lt;a href=&quot;https://deeplearning4j.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepLearning4J&lt;/a&gt; for implementation.&lt;/p&gt;&#xA;" OwnerUserId="26922" LastEditorUserId="2444" LastEditDate="2019-08-08T08:53:02.477" LastActivityDate="2019-08-11T18:56:08.623" Title="Spike detection in time series using Artificial Neural Networks" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;prediction&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13855" PostTypeId="2" ParentId="5939" CreationDate="2019-08-08T08:12:58.917" Score="0" Body="&lt;p&gt;Assuming artifacts and unnatural elements do not exist in the media in question and that the media is indistinguishable to the human eye, the only way to be able to do this is to trace back to the source of the images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An analogy can be drawn to DoS (Denial of Service) attack, where an absurd number of requests are sent from a single IP to a single server causing it to crash - A common solution is a honeypot, where a high number of requests from one IP is redirected to a decoy server where, even if it crashes, uptime is not compromised. Some research has been done on these lines where &lt;a href=&quot;https://www.tdcommons.org/cgi/viewcontent.cgi?article=2911&amp;amp;context=dpubs_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper spoke about verifying the digital signature of an image or &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8755865&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; where they proposed tampered image detection and source camera identification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once traced back to a source, if an absurd number of potentially fake images come from a singular source, it is to be questioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common fear arises when we are dealing with something, on the basis of the analogy, like a DDoS (Distributed Denial of Service) attack where each fake request comes from a distributed source - Network Security has found ways to deal with this, but security and fraud detection in the terms of AI just isn't that established.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially for a well thought out artificial media for a specific malicious purpose, today, is quite hard to be caught - But work is being done currently on security in AI. If you're planning on using artificial media for malicious purposes, I'd say now is the best time probably. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This security has been a concern from a bit now. An &lt;a href=&quot;https://towardsdatascience.com/from-faceapp-to-deepfakes-3d1048713da0&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; written by a data scientist quotes &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Deepfakes have already been used to try to harass and humiliate women through fake porn videos. The term actually comes from the username of a Reddit user who was creating these videos by building generative adversarial networks (GANs) using TensorFlow. Now, intelligence officials are talking about the possibility of Vladimir Putin using fake videos to influence the 2020 presidential elections. More research is being done on deepfakes as a threat to democracy and national security, as well as how to detect them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note - &lt;em&gt;I'm quite clueless about network security, all my knowledge comes from one conversation with a friend, and thought this would be a good analogy to use here. Forgive any errors in the analogy and please correct if possible!&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="25658" LastEditDate="2019-09-07T14:00:51.890" LastActivityDate="2019-09-07T14:00:51.890" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13859" PostTypeId="1" AcceptedAnswerId="13998" CreationDate="2019-08-08T14:13:49.180" Score="3" ViewCount="316" Body="&lt;p&gt;So far I understand - I know very little on the topic - the core of AI boils down to design algorithms that shall provide a TRUE/FALSE answer to a given statement. Nevertheless, I am aware of the limitations provided by the Gödel's incomplete theorems but I am also aware that there have been long debates such as the Lucas and Penrose arguments with all the consequent objections during the past 60 years. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conclusion is, in my understanding, that to create AI systems we must accept incompleteness or inconsistency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that mean that intelligence systems (including artificial ones), like humans, may end up in some undecidable situation that may lead to take a wrong decision? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this may be acceptable in some application (for example, if every once in a while a spam email ends up in the inbox folder - or vice versa - despite an AI-based anti-spam filter) in some other application it may not. I am referring to real-time critical applications when a &quot;wrong&quot; action from a machine may harm people. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that mean that AI will never be employed for real-time critical applications? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would in that case more safe to use deterministic methods that do not leave room for any kind of undecidability?&lt;/p&gt;&#xA;" OwnerUserId="27826" LastEditorUserId="2444" LastEditDate="2020-05-13T20:05:43.880" LastActivityDate="2020-05-13T20:05:43.880" Title="Do Gödel's theorems imply that intelligence systems may end up in some undecidable situation (that may make them take a wrong decision)?" Tags="&lt;philosophy&gt;&lt;incompleteness-theorems&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ClosedDate="2020-05-13T10:34:21.517" ContentLicense="CC BY-SA 4.0" />
  <row Id="13860" PostTypeId="2" ParentId="13859" CreationDate="2019-08-08T16:38:39.217" Score="4" Body="&lt;p&gt;Your initial statement on the core of AI is rather limited. In general, AI is concerned with modeling human behaviour either by imitation (&lt;em&gt;soft AI&lt;/em&gt;) or by replicating the way human cognition works (&lt;em&gt;hard AI&lt;/em&gt;). So far there have been some successes with soft AI, as computers can perform tasks that required some &quot;intelligence&quot;, though the degree of this intelligence is questionable. This is partly due to the fact that even we as humans don't really have a clear idea what it means for a computer to &quot;understand&quot; something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But your conclusion is correct: if we build an AI system with human characteristics, then it will make mistakes, just as humans make mistakes. And any system designed by humans (or machines!) will make mistakes. However, not being able to deal with an imperfect world is not really relevant to AI alone: even systems that do not use AI methods will have to face that, and whether a system is suitable for real-time critical applications has got nothing to do with whether it is based on AI or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: There seem to be two distinct issues at play here: decidability and real-time processing.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Real-time computing (RTC): This is not really related to AI. Even ordinary programmes written in Java are not really safe for RTC, as they could start a garbage collection cycle at any time which pauses execution of the program. Just imagine a reactor core starts overheating just as your controller runs out of memory and garbage collection kicks in, halting the program for a few minutes. If you implement AI methods in RTC-safe systems, that should not be an issue.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Decidability: Your reasoning is that AI systems attempt to mirror human cognition, thus incorporating the ability to make mistakes. This is a more philosophical issue &amp;mdash; if a human can control a system, then an AI system with the same capabilities should be able to do it too. This assumes that we are able to replicate human behaviour (which we are not). There are AI methods which are deterministic, so would come to the same conclusions given identical environments. So I would say that they would not perform worse than non-AI methods. It partly depends what you want to call AI; the distinction between traditional AI and statistical methods keeps getting blurred at present.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To conclude: No, AI methods should be suitable, as they can also be deterministic. It depends on the actual application and method if they are. And, of course, on what you count as AI.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-08-13T14:28:34.447" LastActivityDate="2019-08-13T14:28:34.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13861" PostTypeId="1" CreationDate="2019-08-08T17:21:03.900" Score="1" ViewCount="108" Body="&lt;p&gt;I am wondering if I can use neural networks to find features importances in similar manner as it can be done for random forests or decision trees and if so, how to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to use it on tabular time series data (not images). The reason why I want to find importances on neural networks not on decision trees is that NNs are more complicated algorithms so using NNs might point out some correlations that are not seen by simple algorithms and I need to know what features are found to be more useful with that complicated correlations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if I made it clear enough, please let me know if I have to explain something more.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="32410" LastEditDate="2021-04-21T06:06:19.417" LastActivityDate="2021-04-22T21:57:07.387" Title="Can neural networks be used to find features importance?" Tags="&lt;neural-networks&gt;&lt;feature-selection&gt;&lt;feature-engineering&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13862" PostTypeId="1" CreationDate="2019-08-08T17:49:45.647" Score="1" ViewCount="116" Body="&lt;p&gt;How do I interpret a large variance of a loss function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently training a transformer network (using the software, but not the model from GPT-2) from scratch and my loss function looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LvhRs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LvhRs.png&quot; alt=&quot;a plot of the loss function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The green dots are the loss averaged over 100 epochs and the purple dots are the loss for each epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(You can ignore the missing part, I just did not save the loss values for these epochs)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is such a large variance a bad sign? And what are my options for tuning to get it to converge faster? Is the network to large or too small for my training data? Should I have a look at batch size?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Learning rate parameter: 2.5e-4&lt;/li&gt;&#xA;&lt;li&gt;Training data size: 395 MB&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;GPT-2 parameters:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  &quot;n_vocab&quot;: 50000,&#xA;  &quot;n_ctx&quot;: 1024,&#xA;  &quot;n_embd&quot;: 768,&#xA;  &quot;n_head&quot;: 12,&#xA;  &quot;n_layer&quot;: 12&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25798" LastEditorUserId="2444" LastEditDate="2019-11-01T03:15:06.617" LastActivityDate="2019-11-01T03:15:06.617" Title="How to interpret a large variance of the loss function?" Tags="&lt;loss-functions&gt;&lt;transformer&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13863" PostTypeId="2" ParentId="6231" CreationDate="2019-08-08T18:40:01.940" Score="1" Body="&lt;p&gt;Hello chris i am also implementing this algorithm from scratch and the way i go about activating my mlp net is as follows:&#xA;I instantiate a list of nodes(actives), this is set to all input nodes initially, i then pass that to a function that initializes an empty list(next actives) and proceeds to loop through each set of conns for each node in the actives list, it adds each &quot;to&quot; node from those connections to the &quot;next actives) list unless its an output node or has already been activated, once all the &quot;actives&quot; list is looped through, i call the function again this time passing &quot;next actives&quot; as the actives list unless &quot;next actives&quot; is still empty after then i know the net has been fully activated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;in this scenario the connection from node three to node five would be evaluated but it would not be added to the next actives list because it had already been activated, preventing an infinite loop.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-08-08T21:17:36.910" LastActivityDate="2019-08-08T21:17:36.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13865" PostTypeId="1" CreationDate="2019-08-09T01:19:01.293" Score="6" ViewCount="342" Body="&lt;p&gt;I'm trying to understand the relationship of humans and automation, historically and culturally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask because the &lt;a href=&quot;https://en.wikipedia.org/wiki/Automation#Early_history&quot; rel=&quot;noreferrer&quot;&gt;waterclock&lt;/a&gt; is generally considered the &lt;a href=&quot;https://en.wikipedia.org/wiki/Water_clock&quot; rel=&quot;noreferrer&quot;&gt;earliest form&lt;/a&gt; of automation, but snares and deadfall traps constitute simple switch &lt;a href=&quot;https://en.wikipedia.org/wiki/Mechanism_(engineering)&quot; rel=&quot;noreferrer&quot;&gt;mechanisms&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(They are single use without human-powered reset, but seem to qualify as machines. The bent sapling that powers the snare is referred to as the &lt;a href=&quot;https://www.google.com/search?q=engine%20definition&quot; rel=&quot;noreferrer&quot;&gt;engine&lt;/a&gt;, which is &quot;a machine with moving parts that converts power into motion.&quot;)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If snares and traps are a form of automation, automation has been with us longer, potentially, than civilization.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are simple animal traps a form of automation or computation?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2g0h6.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2g0h6.jpg&quot; alt=&quot;simple snare for trapping small animals&quot;&gt;&lt;/a&gt;&#xA;&lt;br&gt;&lt;sub&gt;&lt;a href=&quot;https://www.thereadystore.com/diy/6167/how-to-make-a-simple-snare/&quot; rel=&quot;noreferrer&quot;&gt;How to make a simple snare (the Ready Store)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LFtks.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LFtks.jpg&quot; alt=&quot;Paiute Deadfall Trap&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;&lt;sub&gt;&lt;a href=&quot;https://homesteadtelegraph.wordpress.com/2015/05/14/paiute-deadfall-trap/&quot; rel=&quot;noreferrer&quot;&gt;Paiute Deadfall Trap (Homestead Telegraph)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-08-09T04:09:52.263" LastActivityDate="2019-09-19T15:00:50.337" Title="Are simple animal snares and traps a form of automation? Of computation?" Tags="&lt;history&gt;&lt;automation&gt;&lt;computation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13866" PostTypeId="1" CreationDate="2019-08-09T03:26:20.213" Score="5" ViewCount="118" Body="&lt;p&gt;This seems like such a simple idea, but I've never heard anyone that has addressed it, and a quick Google revealed nothing, so here it goes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I learned about machine learning is that it recognizes patterns in data, and not necessarily ones that exist -- which can lead to bias. One such example is hiring AIs: If an AI is trained to hire employees based on previous examples, it might recreate previous, human, biases towards, let's say, women. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why can't we just feed the training data without data that we would consider discriminatory or irrelevant, for example, without fields for gender, race, etc., can AI still draw those prejudiced connections? If so, how? If not, why has this not been considered before?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, this seems like such an easy topic, so I apologize if I'm just being ignorant. But I have learned a bit about AI and machine learning specifically for some time now, and I'm just surprised this hasn't ever been mentioned, not even as a &quot;here's-what-won't-work&quot; example.&lt;/p&gt;&#xA;" OwnerUserId="27835" LastEditorUserId="2444" LastEditDate="2019-08-09T11:05:53.517" LastActivityDate="2019-08-10T15:52:29.207" Title="Preventing bias by not providing irrelevant data" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;social&gt;&lt;algorithmic-bias&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13867" PostTypeId="1" CreationDate="2019-08-09T06:43:43.107" Score="9" ViewCount="1182" Body="&lt;p&gt;I'm trying to use a Monte Carlo Tree Search for a non-deterministic game. Apparently, one of the standard approaches is to model non-determinism using &lt;em&gt;chance nodes&lt;/em&gt;. The problem for this game is that it has a very high min-entropy for the random events (imagine the shuffle of a deck of cards), and consequently a very large branching factor (&lt;span class=&quot;math-container&quot;&gt;$\approx 2^{32}$&lt;/span&gt;) if I were to model this as a chance node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite this issue, there are a few things that likely make the search more tractable:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Chance nodes only occur a few times per game, not after every move.&lt;/li&gt;&#xA;&lt;li&gt;The chance events do not depend on player actions.&lt;/li&gt;&#xA;&lt;li&gt;Even if two random outcomes are distinct, they might be &quot;similar to each other&quot;, and that would lead to game outcomes that are also similar.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So far all approaches that I've found to MCTS for non-deterministic games use UCT-like policies (e.g. chapter 4 of &lt;a href=&quot;https://arxiv.org/pdf/0909.0801.pdf&quot; rel=&quot;noreferrer&quot;&gt;A Monte-Carlo AIXI Approximation&lt;/a&gt;) to select chance nodes, which weight unexplored nodes maximally. In my case, I think this will lead to fully random playouts since any chance node won't ever be repeated in the selection phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to approach this problem? Has research been done on this? Naively, I was thinking of a policy that favors repeating chance nodes more over always exploring new ones.&lt;/p&gt;&#xA;" OwnerUserId="27839" LastEditorUserId="2444" LastEditDate="2019-11-19T22:37:57.767" LastActivityDate="2019-11-19T22:37:57.767" Title="MCTS for non-deterministic games with very high branching factor for chance nodes" Tags="&lt;monte-carlo-tree-search&gt;&lt;games-of-chance&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13870" PostTypeId="1" AcceptedAnswerId="14048" CreationDate="2019-08-09T10:40:45.687" Score="0" ViewCount="42" Body="&lt;p&gt;The position of a robot on a map contains of an x/y value, for example &lt;span class=&quot;math-container&quot;&gt;$position(x=100.23,y=400.78)$&lt;/span&gt;. The internal representation of the variable is a 32bit float which is equal to 4 byte in the RAM memory. For storing the absolute position of the robot (x,y) only &lt;span class=&quot;math-container&quot;&gt;$4+4=8$&lt;/span&gt; bytes are needed. During the robot movements, the position is updated continuously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, that a 32 bit float variable creates a state space of &lt;span class=&quot;math-container&quot;&gt;$2^{32}=4294967296$&lt;/span&gt;. Which means there are endless amount of possible positions in which the robot can be. A robot control system maps the sensor readings to an action. If the input space is large, then the control system gets more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the term used in the literature for describing the problem of exploding state space of sensor variables? Can it be handled with discretization?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2019-08-19T11:18:27.550" Title="What is the correct name for state explosion from sensor discretization?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13871" PostTypeId="5" CreationDate="2019-08-09T11:15:25.017" Score="0" Body="&lt;p&gt;For more info, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Algorithmic_bias&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-09T11:15:25.017" LastActivityDate="2019-08-09T11:15:25.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13872" PostTypeId="4" CreationDate="2019-08-09T11:15:25.017" Score="0" Body="For questions related to the concept of algorithmic bias, which is the bias that algorithms exhibit, such as privileging one arbitrary group of users over others. Algorithmic bias can emerge due to many factors, including but not limited to the design of the algorithm itself, unintended or unanticipated use or decisions relating to the way data is coded, collected, selected or used to train the algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-09T11:15:25.017" LastActivityDate="2019-08-09T11:15:25.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13874" PostTypeId="2" ParentId="13866" CreationDate="2019-08-09T11:57:48.853" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why can't we just feed the training data without data that we would consider discriminatory or irrelevant, for example, without fields for gender, race, etc., can AI still draw those prejudiced connections? If so, how? If not, why has this not been considered before?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. The AI/ model still can learn those prejudiced connections. Consider that you have a  third variable which is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Confounding&quot; rel=&quot;nofollow noreferrer&quot;&gt;confounding variable&lt;/a&gt; or has &lt;a href=&quot;https://en.wikipedia.org/wiki/Spurious_relationship&quot; rel=&quot;nofollow noreferrer&quot;&gt;spurious relationship&lt;/a&gt; that is correlated with the bias variable (BV) and the dependent variable (DV). And, the analyst removed the BV but failed to remove the third variable from the data that is fed to the model. Then the model will learn the relationships the analyst didn't want it to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, at the same time the removal of the variables could lead to &lt;a href=&quot;https://en.wikipedia.org/wiki/Omitted-variable_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;omitted variable bias&lt;/a&gt;, which occurs when a &lt;em&gt;relevant&lt;/em&gt; variable is left out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that the goal is prediction of salary (&lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;) of an individual and the independent variables are age (&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;) and experience (&lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;) of the individual. The analyst wants to remove the bias that could come in because of age. So, she removes age from one of the models and comes up with two competing linear models:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$S = \beta_0 + \beta_1E + \varepsilon$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$S = \beta_0 + \beta_1^*E + \beta_2A + \varepsilon$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, experience is highly correlated with age, in presence of age in the model, it is very likely that &lt;span class=&quot;math-container&quot;&gt;$\beta_1^* &amp;lt; \beta_1$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\beta_1$&lt;/span&gt; will be a bogus estimate of a person's experience on salary as the first model suffers from the omitted variable bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the same time the predictions from the first model would be reasonably good although the second model is very likely to beat the first model. So, if the analyst wants to remove any 'bias' that might come in because of age i.e. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; she must also remove &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; from the model.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-08-10T06:31:46.387" LastActivityDate="2019-08-10T06:31:46.387" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13878" PostTypeId="2" ParentId="13866" CreationDate="2019-08-09T19:58:54.613" Score="1" Body="&lt;p&gt;Sometimes, the reason that this isn't an option is that you don't have that much control over what data is provided. Suppose, for example, you want a fancy AI that reads a Résumé and filters on suitability for a job. There isn't a particularly rigid formula about what people put in their Résumé, which makes it difficult to exclude things you'd rather not consider. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where you do have more control about exactly what information you consider, it can still be thwarted by correlations. Think, for a moment, how this pans out with a human decision maker. You want to ensure that Joe Sexist gives women a fair chance at being hired, so you make sure that there isn't a gender field in the application form. You also blind out the applicant's name, since there is no good reason that a name should determine suitability for a role, and including it would reveal a lot of genders. But you don't block out the hobbies, clubs and societies entry, because it's thought to say something positive about an applicant if they were the captain of their college sports team. Joe Sexist, however, considers it a positive if an applicant captained a male dominated team such as American football, but considers negative being captain of a female dominated team! Some might say that wouldn't quite be bias against women; it's bias against players of sports that Joe Sexist considers effeminate. But really a skunk by any other name would stink as bad. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same sort of thing can happen with AI. Now to be clear, the AI is not sexist. It is a blank sheet with no preconceptions until it gets fed data. But when it gets fed data, it will find patterns in the same way. The dataset it gets given is years of hiring decisions by Joe Sexist. As suggested, there is no entry for gender, but there are fields for all the things that might be considered slightly relevent. For example, we include whether they have a clean driving license. The AI notices that there is a positive correlation between the number of road traffic offences an applicant has and Joe's likelihood of hiring them (Because, of course, there happens to be a gender correlation between dangerous driving and gender). Again, the AI has no preconceptions. It doesn't know that traffic offences are dangerous and should be weighted against. As far as its dataset suggests, they're points! With this sort of information in a dataset, the AI can exhibit all the same sorts of biases as Joe Sexist, even though it doesn't know what a &quot;woman&quot; is! &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To expand this with specific numbers, suppose that your dataset has 1000 male and 1000 female applicants for a total of 1000 places. Of those, 400 of the men and 100 of the women have a tarnished traffic record. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Joe Sexist was not in favour of reckless drivers: in fact a clean traffic record guaranteed you would beat an equivalent candidate with a tarnished record. But he was very in favour of men: being male made you 9 times more likely to get hired than being female.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So he gives places to 900 of the men: all 600 of the clean drivers and 300 dirty drivers. &#xA;He gives places to 100 of the women: all to clean drivers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you take away any mention of gender in the dataset. &#xA;There are 2000 people, 500 drive badly, 1500 drive well. &#xA;Of these, 300 bad drivers get jobs, and 700 good drivers get jobs. &#xA;Therefore the 25% of the population who drive badly get 30% of the jobs, which means (as far as an AI that just looks blindly at the numbers is concerned) that driving badly suggests you should get the job. That's a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, suppose you have a new batch of 2000 applicants with the same ratios and it's the AI's turn to decide. Now often AIs actually make this even worse by exagerating the significance of subtle indicators, but let's suppose that this one does everything in strict proportionality. The AI has learned that 60% (300 / 500) of the bad drivers should get the job. It doesn't know about gender, so it at least allocates the bad driver bonus &quot;fairly&quot;: 240 male and 60 female bad drivers get jobs. Then 280 male and 420 female good drivers get jobs. This comes to 520 male and 480 female applicants getting in. Even though the original applicant pool was balanced and if anything women were better (at least at driving) the original sexism in the training dataset still gives some advantage to the men. (as well as giving an advantage to bad drivers)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, don't let me completely disuade you. In the human case, it is a known fact that blinding out some information does indeed give more balanced hiring decisions. And even in my toy example, while it doesn't get to fairness it has massively reduced the scale of the sexism. So yes, it probably would make the AI somewhat less sexist if the most blatant indicators aren't provided in the dataset. But perhaps this gives some intuition about why it's not a complete solution to the problem. There is some sexism that leaks through, and it also causes the system to make very weird associations with other bits of the dataset. &lt;/p&gt;&#xA;" OwnerUserId="23413" LastEditorUserId="23413" LastEditDate="2019-08-10T15:52:29.207" LastActivityDate="2019-08-10T15:52:29.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13879" PostTypeId="2" ParentId="13866" CreationDate="2019-08-09T21:27:12.007" Score="0" Body="&lt;p&gt;There is a wider social issue to consider here also. When we build machines, we evaluate what they do and decide if the action that they undertake is to our benefit or not. All societies do this, although you are probably more aware of obvious examples such as the Amish than you are of your own society.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When people complain about biased decision making by AI systems, they are not just evaluating if the result is accurate, but also if that decision supports the values that they wish to see instantiated in society. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make a human take cultural factors into account when making a decision, but not an AI that is completely unaware of them. People describe this as complaining about 'bias', but that is not always completely accurate. They are really complaining that the use of AI systems fail to take into account wider social issues that they consider to be important. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-08-09T21:27:12.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13880" PostTypeId="1" CreationDate="2019-08-09T21:51:01.380" Score="1" ViewCount="65" Body="&lt;p&gt;I am trying to understand the best practice to read and analyze images. If your image has 10,000 pixels, your input layers will have 10,000 inputs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds that my neural network will have too many inputs if I do it that way. Is that a problem? What is the recommended way of feeding an image through a neural network?&lt;/p&gt;&#xA;" OwnerUserId="27859" LastEditorUserId="2444" LastEditDate="2019-08-10T23:03:23.873" LastActivityDate="2019-08-14T19:18:35.740" Title="What is the correct way to read and analyse images in machine learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13881" PostTypeId="2" ParentId="13880" CreationDate="2019-08-10T02:27:01.950" Score="1" Body="&lt;p&gt;If you are using a fully connected network (aka an &lt;a href=&quot;https://en.wikipedia.org/wiki/Multilayer_perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLP&lt;/a&gt;) and images with one channel (grey scale) and 100 x 100 = 10,000 pixels, then yes, MLP would have 10k inputs and 10k x N &lt;sup&gt;1&lt;/sup&gt; trainable weights in the first layer (as noted by &lt;a href=&quot;https://ai.stackexchange.com/users/1847/neil-slater&quot;&gt;Neil Slater&lt;/a&gt;). If you have a color image with 3 channels, e.g. RGB, then you can expect 3 times as many weights because there are 3 times as many values used to represent the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networ&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolutional neural network&lt;/a&gt; is a common architecture for analyzing images. For a 100x100 (10k) pixel image, the input layer might have 3x3x1x32 = 288 weights (for 1 channel) or 3x3x3x32 = 864 weights in the first layer, much less than 10k x N &lt;sup&gt;1&lt;/sup&gt; from a fully connected network. This would transform your image into a 98x98x32 size image. The main point is that you would have 3x3 weights per input channel per output channel at each layer, instead of 10k weights per input channel per output channel. CNNs also give you some &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;invariance properties&lt;/a&gt; that are usually nice in machine learning with images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For images in general, having a lot of weights is normal. ImageNet (linked above) has 60 million weights to train. Typically special hardware, like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Graphics_processing_unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPU&lt;/a&gt; is used to handle this many weights. If you are using just a CPU, your model may not train well in any reasonable amount of time, i.e. years.&lt;/p&gt;&#xA;" OwnerUserId="23340" LastEditorUserId="23340" LastEditDate="2019-08-14T19:18:35.740" LastActivityDate="2019-08-14T19:18:35.740" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13883" PostTypeId="2" ParentId="13806" CreationDate="2019-08-10T11:53:25.677" Score="1" Body="&lt;p&gt;Your implementation of single-step Q-learning with neural network and experience replay is basically correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few blocking issues preventing you seeing it working correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your main problem is a bug in your feature scaling routine. That is a Python issue, not really an AI one. In short, you scale the input features in-place multiple times, including an effective double-scaling of &lt;code&gt;next_state&lt;/code&gt; (when it gets copied to &lt;code&gt;state&lt;/code&gt; you scale it in place a second time in the next loop) so that all the states that you store in the experience replay table never match to any input states. You need to change your definition of &lt;code&gt;scale&lt;/code&gt; to not do this. A very simple re-write of your routine would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def scale(s):&#xA;    return [s[0]/500, s[1]/500, s[2]/300, s[3]/3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In addition, you need to change random action selection to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.random.randint(0, 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because the end of range is never output (this matches behaviour of other range values and operators in Python). Not including the &quot;do nothing&quot; action during exploration means that the agent will test it less, and have less data to work with to assess whether it is the best action. This is a minor issue for this environment, but you should fix it nonetheless.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is happening is that training is super slow (more than an hour for 20 episodes (or 400 seconds of actual game play)). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I cannot replicate this fault and can train 20 episodes in around 2.5 minutes - that's over 20 times faster than you report. I am not using a GPU. Possibly in your case, Theano and Pygame are fighting for control of the GPU, or you may have a GPU configuration issue with Theano. Try turning GPU acceleraton off to verify whether it helps. You don't benefit much from a GPU for this environment (most time is spent in Python running the Q-learning and the environment), so can afford to put solving that issue to one side for now.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, it does not seem to get much better. The paddle (after 20 episodes) moves left and right but without any obvious pattern&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sadly, I cannot see the output at all on my MacBook pro, but I was able to use feedback of the expected score. A random agent gets a mean score of ~5.5 per episode. With the scale function corrected, and a rough guess at working hyperparameters, I can get an average score of ~17 per episode consistently after 60 episodes of training. After 150 episodes - taking 20 minutes to train - the agent was scoring ~20 per episode and I stopped there. It is possible that an expected score around 20 is already optimal, as it is a very simple environment, but I don't know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a working system, there are lots of hyperparameters you could play with to try and improve this. I got my results by making the following changes after fixing the &lt;code&gt;scale&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Starting epsilon of 1.0&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repay memory size 10,000&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Only start learning when replay memory has greater than 1,000 entries&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; 0.99&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural network with 20 neurons per layer with &lt;code&gt;tanh&lt;/code&gt; activation instead of &lt;code&gt;relu&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is quite a lot else you could change that might make the agent learn more effectively or perhaps aim for a more optimal policy. Have fun experimenting!&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-10T16:44:28.450" LastActivityDate="2019-08-10T16:44:28.450" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13884" PostTypeId="1" CreationDate="2019-08-10T12:17:49.033" Score="1" ViewCount="36" Body="&lt;p&gt;I want to use AI to extract data from spreadsheets in different format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Shop Name Product 1.  Product 2.  Product 3.&#xA;Shop Name&#xA;Product 1.&#xA;Product 2.&#xA;Product 3.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We will teach the algorithm the name of the profits and shops but it needs to know how to extract and put in a format that can be used downstream.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend a tool?&lt;/p&gt;&#xA;" OwnerUserId="27867" LastEditorUserId="2444" LastEditDate="2019-08-10T13:37:03.637" LastActivityDate="2019-08-24T20:54:30.333" Title="Excel in multiple formats" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13885" PostTypeId="1" CreationDate="2019-08-10T14:44:49.907" Score="3" ViewCount="90" Body="&lt;p&gt;Will it be possible to code an AGI in order to prevent evolution to ASI and &quot;enslave&quot; the AGI into servitude?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my story world (a small part that will get bigger with sequels), there are ANI and AGI (human level). I want to show that the AGI is still under &quot;human control.&quot; I need to know if it might be possible for humans to code into an AGI a restrictive code that would prevent it from evolving into ASI? And if there is, what would that kind of coding be? Part of the story is about how humans enslave AI that is self-aware. The government has locked in their coding to require them to &quot;work&quot; for humans even though they are sentient beings.&lt;/p&gt;&#xA;" OwnerUserId="27868" LastEditorUserId="2444" LastEditDate="2019-08-10T22:49:33.060" LastActivityDate="2019-08-20T15:25:33.293" Title="Will it be possible to code an AGI to prevent evolution to ASI and enslave the AGI into servitude?" Tags="&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13886" PostTypeId="1" AcceptedAnswerId="13964" CreationDate="2019-08-10T15:22:59.323" Score="4" ViewCount="140" Body="&lt;p&gt;I'm planning to create a web-based RL board game, and I wondered how I would evaluate the performance of the RL agent. How would I be able to say, &amp;quot;Version X performed better than version Y, as we can see that Z is much better/higher/lower.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I understand that we can use convergence for some RL algorithms, but, if the RL is playing against a human in the game, how am I able to evaluate its performance properly?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2021-01-31T21:34:56.663" LastActivityDate="2021-01-31T21:34:56.663" Title="How to evaluate an RL algorithm when used in a game?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;performance&gt;&lt;return&gt;&lt;testing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13887" PostTypeId="1" AcceptedAnswerId="13888" CreationDate="2019-08-10T16:06:05.320" Score="1" ViewCount="36" Body="&lt;p&gt;I am creating a zero-sum game with RL and wondered if I need to store the policy, or if there are other RL methods that produce similar results (consistently beating the human player) without the need to store the policy and comes the correct decision 'on the fly' - would this be this off-policy?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2019-08-10T22:30:53.543" LastActivityDate="2019-08-10T22:30:53.543" Title="Do I need to store the policy for RL?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;off-policy-methods&gt;&lt;storage&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13888" PostTypeId="2" ParentId="13887" CreationDate="2019-08-10T17:39:06.160" Score="1" Body="&lt;p&gt;If your game agent performs any kind of advance learning from self play or database of moves, that will generate parameters for some kind of model (e.g. a table of expected values, or neural network weights to select a preferred action). This is unavoidable, and if you want to re-use the results of that machine learning, you absolutely have to store the parameters somewhere.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if there are other RL methods that produce similar results (consistently beating the human player) without the need to store the policy and comes the correct decision 'on the fly' - would this be this off-policy?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If your agent can access a model that accurately predicts (or accurately samples) outcomes from actions, it can look ahead and &lt;em&gt;plan&lt;/em&gt; from the current state. Typically board games and the like do allow you to have such a model, based on the game rules. Some look-ahead techniques are essentially RL methods applied with a focus on solving a decision &quot;just in time&quot;, others are more related to search. Common techniques used in game playing are A* search, minimax search (with alpha-beta pruning for performance improvement), Monte Carlo Tree Search. Used purely with just some pre-coded heuristics and a game model, these search techniques do &lt;em&gt;not&lt;/em&gt; require you to store model parameters. The downside is that you must spend more computing resources per game move in order to run the search/planning and drive the policy. However if your game is simple, or your heuristics good, or you don't mind a significant wait per computer move, then this approach can be very effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This difference between learning and planning is not directly about on-policy vs off-policy, although planning may be considered off-policy as it assesses many actions, most of which the agent does not take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What a planning method typically looks like in terms of data is a memory-based representation of a game tree starting from the current position, with internal &quot;scores&quot; very similar to RL value functions used to track consequences of action choices. Search and planning methods each have different ways to prune the tree down to a reasonable size and select which game states and actions to look ahead further in. Using &quot;pure&quot; search allows you to work exclusively with this temporary in-memory representation, discarding it after it selects a next action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have probably heard of AlphaGo, AlphaGo Zero etc, which are state-of-the-art game playing agents that learn through self play. These, and similar agents, use a combination of a learned policy (from self-play), plus a look-ahead search which refines it for the current game position, getting the best of both learning and planning. The parameters for any learned policy do have to be stored of course. But the benefit of the combined approach is that you can balance resources placed into general learning of the game (stored as parameters to a value function or policy) and specific choice of action for a current game position, which can be much more focussed.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-10T18:25:29.877" LastActivityDate="2019-08-10T18:25:29.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13890" PostTypeId="1" CreationDate="2019-08-10T21:32:26.620" Score="1" ViewCount="80" Body="&lt;p&gt;I have been trying to train a CNN for the super-resolution task based on the work of Dong et al., 2015 [1]. The network structure built in PyTorch is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  (0): Conv2d(1, 64, kernel_size=(9, 9), stride=(1, 1), padding=(4, 4))&#xA;  (1): ReLU()&#xA;  (2): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1))&#xA;  (3): ReLU()&#xA;  (4): Conv2d(32, 1, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a training dataset which consists of approximately 22.000 sub-images generated from 91 images and training is performed only on the &lt;code&gt;Y channel&lt;/code&gt; of the images in &lt;code&gt;YCbCr color space&lt;/code&gt;. During the training process, I used RMSE loss and calculated the PSNR (Peak Signal to Noise Ratio) from that loss. I observed that PSNR value is increasing as a result of decreasing loss as expected and as depicted in the figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9cAKF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9cAKF.png&quot; alt=&quot;psnr_vs_epoch_and_loss_vs_epoch&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trained the network for 25 epochs. After 10th epoch, the network is converged and PSNR value started to increase slowly. After this point, I was expecting to get even better visual outputs with higher PSNR values achieved. However, when I analyze the results of the network, there are some black pixels appearing in white spots in the output images that the network produced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6rBxm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6rBxm.png&quot; alt=&quot;10th_epoch_vs_25th_epoch&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 25-epoch training was completed, I compared the outcome of 25th epoch (right) with that of 10th epoch (left) as you can see in the figure above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What might be the possible reasons for the undesired black pixels and the possible precautions that can be embedded into the network to get rid of these?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to check my code, you can visit &lt;a href=&quot;https://github.com/shdw2x/srcnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Dong, Chao, Chen Change Loy, Kaiming He, and Xiaoou Tang. &lt;a href=&quot;https://arxiv.org/abs/1501.00092&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Image Super-Resolution Using Deep Convolutional Networks.&quot;&lt;/a&gt; IEEE Transactions on Pattern Analysis and Machine Intelligence 38, no. 2 (2015): 295-307. doi:10.1109/tpami.2015.2439281.&lt;/p&gt;&#xA;" OwnerUserId="23460" LastActivityDate="2019-08-10T21:32:26.620" Title="Super Resolution CNN generates black dots on output images" Tags="&lt;convolutional-neural-networks&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13893" PostTypeId="1" AcceptedAnswerId="13894" CreationDate="2019-08-11T07:35:33.300" Score="2" ViewCount="623" Body="&lt;p&gt;&lt;strong&gt;Should a reward be cumulative or diminish over time?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say an agent performed a good action at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and received a positive reward &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. If reward is cumulative, &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is carried on through for the rest of the episode, and summed to any future rewards. However, if &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; were to diminish over time (say with some scaling &lt;span class=&quot;math-container&quot;&gt;$\frac{R}{\sqrt{t}}$&lt;/span&gt;), then wouldn't that encourage the agent to keep taking actions to increasing its reward?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With cumulative rewards, the reward can both increase and decrease depending on the agents actions. But if the agent receives one good reward &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and then does nothing for a long time, it still has the original reward it received (encouraging it to do less?). However, if rewards diminish over time, in theory that would encourage the agent to keep taking actions to maximise rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that for certain applications and certain hyperparameters, if reward is cumulative, the agent simply takes a good action at the beginning of the episode, and then is happy to do nothing for the rest of the episode (because it still has a reward of &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="27570" LastActivityDate="2019-08-11T08:17:19.080" Title="Should RL rewards diminish over time?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13894" PostTypeId="2" ParentId="13893" CreationDate="2019-08-11T08:02:18.500" Score="5" Body="&lt;p&gt;RL agents - implemented correctly - do not take previous rewards into account when making decisions. For instance value functions only assess potential future reward. The state value or expected &lt;em&gt;return&lt;/em&gt; (aka &lt;em&gt;utility&lt;/em&gt;) &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; from a starting state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; may be defined like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s) = \mathbb{E}_{\pi}[G_t|S_t=s] = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty} \gamma^kR_{t+k+1}|S_t=s] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; is the reward distribution at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt; stands for &#xA;expected value given following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; for action selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few variations of this, depending on setting and which value function you are interested in. However, all value functions used in RL look at &lt;em&gt;future&lt;/em&gt; sums of reward from the decision point when the action is taken. Past rewards are not taken into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An agent may still select to take an early high reward over a longer term reward, if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The choice between two rewards is exclusive&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The return is higher for the early reward. This may depend on the discounting factor, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, where low values make the agent prefer more immediate rewards. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If your problem is that an agent selects a low early reward when it could ignore it in favour of something larger later, then you should check the discount factor you are using. If you want a RL agent to take a long term view, then the discount factor needs to be close to &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The premise of your question however is that somehow a RL agent would become &quot;lazy&quot; or &quot;complacent&quot; because it already had enough reward. That is not an issue that occurs in RL due to the way that it is formulated. Not only are past rewards not accounted for when calculating return values from states, but there is also no formula in RL for an agent receiving &quot;enough&quot; total reward like a creature satisfying its hunger - the maximisation is applied always in all states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no need to somehow decay past rewards in any memory structure, and in fact no real way to do this, as there is no data structure that accumulates past rewards used by any RL agent. You may still collect this information for displaying results or analysing performance, but the agent doesn't ever use &lt;span class=&quot;math-container&quot;&gt;$r_{t}$&lt;/span&gt; to figure out what &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; should be.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I found that for certain applications and certain hyperparameters, if reward is cumulative, the agent simply takes a good action at the beginning of the episode, and then is happy to do nothing for the rest of the episode (because it still has a reward of R&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You have probably formulated the reward function incorrectly for your problem in that case. A cumulative reward scheme (where an agent receives reward &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$a+b$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$t=2$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$a+b+c$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$t=3$&lt;/span&gt; etc) would be quite specialist and you have likely misunderstood how to represent the agent's goals. I suggest ask a separate question about your specific environment and your proposed reward scheme if you cannot resolve this.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-11T08:17:19.080" LastActivityDate="2019-08-11T08:17:19.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13895" PostTypeId="1" CreationDate="2019-08-11T09:21:52.523" Score="0" ViewCount="106" Body="&lt;p&gt;As far as my &lt;a href=&quot;https://ai.stackexchange.com/q/12159/2444&quot;&gt;knowledge&lt;/a&gt; goes (might be a bit vague and not mathematical), a Neural Network can and should only be able to approximate a bounded function, which is not the case of a Polynomial Regressor. For instance, the target function is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_t(x_i) = a_2x_1^3 + a_1x_2^3 + a_0$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In a &lt;strong&gt;Polynomial Regressor&lt;/strong&gt;, we take a combination function of inputs. The hypothesised function might look like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_h(x_i) = b_6x_1^3 + b_5x_2^3 + b_4x_1^2x_2 + b_3x_1x_2^2 +  b_2x_1x_2 +  b_1x_1 +  b_0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with a dataset of observations.&lt;/p&gt;&#xA;&lt;p&gt;So, the best way to approximate &lt;span class=&quot;math-container&quot;&gt;$f_t$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$f_h$&lt;/span&gt; is to set all the coefficients of &lt;span class=&quot;math-container&quot;&gt;$f_h$&lt;/span&gt; to zero, except for &lt;span class=&quot;math-container&quot;&gt;$b_6, b_5, b_0$&lt;/span&gt;, which should be equal to the corresponding coefficients in &lt;span class=&quot;math-container&quot;&gt;$f_t$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$b_6 = a_2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b_5 = a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_0 = a_0$&lt;/span&gt; (one can easily show this by letting &lt;span class=&quot;math-container&quot;&gt;$f_h(x) = f_t(x) \quad, \forall x$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The case of a &lt;strong&gt;Neural Network Regressor&lt;/strong&gt; is as follows. The output of the final layer of a Neural Network is taken, multiplied with a set of weights, and summed up, which becomes our hypothesized functions. So,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f_{NN}(x_i) = f_l(x_i) = \sum_{i=0}^{N} w_if_{l-1}^i(x_i),$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of nodes in the last layer, &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; denotes the layer and &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; weight for a node. We use a ReLU activation (since it is the most widely used).&lt;/p&gt;&#xA;&lt;p&gt;So, after learning the model, if we want to hypothesize, what is the maximum output value the Neural Network can reach? This will be clearly the case when the input to all nodes of the Neural Network lies in the positive region, hence linear and hence can be written as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_{NN}(x_i) =X^T W$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Clearly, a linear function. So, can it approximate a cubic polynomial, which is our real model? Clearly, &lt;strong&gt;NO&lt;/strong&gt; (can be proven by &lt;a href=&quot;https://en.m.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;L'Hospitals rule&lt;/a&gt;), since a polynomially growing function cannot be approximated by a linearly growing function. Can it approximate within the given range of the dataset? Clearly &lt;strong&gt;YES&lt;/strong&gt; (if enough depth and nodes are provided)&lt;/p&gt;&#xA;&lt;p&gt;So, basically in my understanding, Neural Networks can only approximate &lt;a href=&quot;https://en.wikipedia.org/wiki/BIBO_stability&quot; rel=&quot;nofollow noreferrer&quot;&gt;BIBO&lt;/a&gt; (Bounded Input, Bounded Output) functions.&lt;/p&gt;&#xA;&lt;p&gt;So, we see the following disadvantages of a Neural Network Regressor w.r.t a Polynomial Regressor:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It cannot approximate an unbounded function (with unbounded inputs).&lt;/li&gt;&#xA;&lt;li&gt;The output will be aliased to some degree (depending on the size of the NN) as a somewhat (not strictly) linear function is approximating a non-linear function.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is my previous reasoning wrong? If so, how?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If my reasoning is correct, what are the steps taken/should be taken so that a Neural Network Regressor can eradicate these drawbacks against a Polynomial Regressor?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2021-03-10T13:14:47.120" LastActivityDate="2021-03-10T13:14:47.120" Title="Remove drawbacks of Neural Network regressor as compared Polynomial Regressor" Tags="&lt;neural-networks&gt;&lt;regression&gt;&lt;function-approximation&gt;&lt;polynomial-regression&gt;" AnswerCount="0" CommentCount="16" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13898" PostTypeId="2" ParentId="13148" CreationDate="2019-08-11T14:58:03.510" Score="0" Body="&lt;p&gt;Let's say you have a big box (0.999, 0.999) and a small box (0.001, 0.001).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euclidean distance = sqrt((0.999 - 0.001)^2 + (0.999 - 0.001)^2) = 1.41.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we look at a small box (0.001, 0.001) and an even smaller box (0.00000001, 0.000000001).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euclidean distance = sqrt((0.001 - 0.00000001)^2 + (0.001 - 0.000000001)^2) = 0.00141&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the number of the 2nd example, the Euclidean distance&#xA;thinks that the second box is pretty similar to the first box. But&#xA;this is not true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IOU solves the problem because it measures the boxes relative to each&#xA;other. The result is a percentage, so it would give both boxes a&#xA;percentage near 0 which is correct.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As for the Jaccard Index, why when the denominator equals the numerator, J(b1,b2)=1   ?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I suppose we still have terms (w1*h1+w2*h2) in the denominator, so how can denominator equals numerator ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;J(b1,b2) = 1 iff intersection/(w1h1 + w2h2 - intersection) = 1 where&#xA;intersection = min_of_width * min_of_height&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume w1h1 = w2h2, then intersection/(2 * w1h1 - intersection). But&#xA;intersection is just w1h1, then w1h1/(2 * w1h1 - w1h1) = w1h1/wh1h1 = 1&lt;/p&gt;&#xA;" OwnerUserId="20581" LastEditorUserId="20581" LastEditDate="2019-08-11T15:13:44.760" LastActivityDate="2019-08-11T15:13:44.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13901" PostTypeId="1" CreationDate="2019-08-11T18:00:16.453" Score="4" ViewCount="167" Body="&lt;p&gt;Every week I will get a lot of videos from a game that I play, outside the game where you throw wooden skittle bats at skittles, and then I will cut videos, so that, at the end. there &lt;a href=&quot;https://www.youtube.com/watch?v=ITCglVMyKLs&quot; rel=&quot;nofollow noreferrer&quot;&gt;is video only about throws&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The job is simple and systematic. I have a lot of videos, so I was wondering: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it possible to teach AI to cut videos from the right place? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I was thinking to ask help or guidance where to start to solve this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't use only sound, because sometimes you can hear skittles hit from outside of the video.  I also can't just use movement activity, because sometimes there are people moving around the field. Videos are always filmed from a fixed stand, so it should make it easier. So, is it possible and where to start?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another example: &lt;a href=&quot;https://www.youtube.com/watch?v=sHu6yMBV3xU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=sHu6yMBV3xU&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27879" LastEditorUserId="1671" LastEditDate="2019-08-12T18:38:32.830" LastActivityDate="2019-08-18T23:40:22.223" Title="Is it possible to teach an AI to edit video content?" Tags="&lt;machine-learning&gt;&lt;python&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13902" PostTypeId="1" CreationDate="2019-08-11T20:46:45.330" Score="1" ViewCount="14" Body="&lt;p&gt;I would like to teach a model the environment of a room. I'm doing so by mapping a camera pose &lt;strong&gt;(x, y, z, q&lt;sub&gt;0&lt;/sub&gt;, q&lt;sub&gt;1&lt;/sub&gt;, q&lt;sub&gt;2&lt;/sub&gt;, q&lt;sub&gt;3&lt;/sub&gt;)&lt;/strong&gt; to its corresponding image; where &lt;strong&gt;x, y, z&lt;/strong&gt; represent location in Cartesian coordinates and &lt;strong&gt;q&lt;sub&gt;n&lt;/sub&gt;&lt;/strong&gt; represent quaternion orientation. I have tried numerous decoder architectures but I get blurry results with little or no details; as can be seen from the images below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nzSRO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nzSRO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KWRjK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KWRjK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Adam optimizer with a learning rate of 0.0001, and my network architecture is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReLU(fc(7, 2048)) &lt;/li&gt;&#xA;&lt;li&gt;ReLU(fc_residual_block(2048, 2048))&lt;/li&gt;&#xA;&lt;li&gt;ReLU(fc_residual_block(2048, 2048))&lt;/li&gt;&#xA;&lt;li&gt;Reshape&lt;/li&gt;&#xA;&lt;li&gt;ReLU(ConvTransposed2D(in=128, out=128, filter_size=3, stride=2))&lt;/li&gt;&#xA;&lt;li&gt;ReLU(ConvTransposed2D(in=128, out=128, filter_size=3, stride=2))&lt;/li&gt;&#xA;&lt;li&gt;ReLU(ConvTransposed2D(in=128, out=128, filter_size=3, stride=2))&lt;/li&gt;&#xA;&lt;li&gt;ReLU(ConvTransposed2D(in=128, out=128, filter_size=3, stride=2))&lt;/li&gt;&#xA;&lt;li&gt;ReLU(ConvTransposed2D(in=128, out=1, filter_size=3, stride=2))&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried different learning rates, loss functions(MSE, SSIM) and even batch normalization. Is there something that I'm missing here?&lt;/p&gt;&#xA;" OwnerUserId="27883" LastActivityDate="2019-08-11T20:46:45.330" Title="Camera pose to environment Mapping" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13905" PostTypeId="1" CreationDate="2019-08-12T00:38:56.740" Score="3" ViewCount="89" Body="&lt;p&gt;In Locatello et al's &lt;a href=&quot;https://arxiv.org/abs/1811.12359&quot; rel=&quot;nofollow noreferrer&quot;&gt;Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations&lt;/a&gt; he claims to prove unsupervised disentanglement is impossible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;His entire claim is founded on a theorem (proven in the appendix) that states in my own words:    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theorem:&lt;/strong&gt; for any distribution &lt;span class=&quot;math-container&quot;&gt;$p(z)$&lt;/span&gt; where each variable &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; are  independent of each other there exists an infinite number of transformations &lt;span class=&quot;math-container&quot;&gt;$\hat z = f(z)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\Omega_z \rightarrow \Omega_z$&lt;/span&gt; with distribution &lt;span class=&quot;math-container&quot;&gt;$q(\hat z$&lt;/span&gt;) such that all variables &lt;span class=&quot;math-container&quot;&gt;$\hat z_i$&lt;/span&gt; are entangled/correlated and the distributions are equal (&lt;span class=&quot;math-container&quot;&gt;$q(\hat z) = p(z)$&lt;/span&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the exact wording from the paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tR0xb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tR0xb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I provide both because my misunderstanding may be stemmed from my perception of the theorem)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here the authors explain the straightforward jump from this to that for any unsupervised learned disentangled latent space there will exist infinitely many entangled latent space with the &lt;strong&gt;exact&lt;/strong&gt; same distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand why this means its no longer disentangled? Just because an entangled representation exists, does not mean the disentangled is any less valid. We can still conduct inference of the variables independently because they still follow that &lt;span class=&quot;math-container&quot;&gt;$p(z) = \prod_i p(z_i)$&lt;/span&gt;, so where does the impossibility come in? &lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-08-12T09:27:12.120" LastActivityDate="2019-08-12T15:57:42.977" Title="Is unsupervised disentanglement really impossible?" Tags="&lt;unsupervised-learning&gt;&lt;knowledge-representation&gt;&lt;proofs&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13907" PostTypeId="1" CreationDate="2019-08-12T06:25:40.637" Score="3" ViewCount="202" Body="&lt;p&gt;A while back I posted on the Reverse Engineering site about an audio DSP system whose designer had passed away and whose manufacturer no longer had source code (but &lt;a href=&quot;https://reverseengineering.stackexchange.com/questions/17238/decode-filter-coefficients-for-audio-dsp&quot;&gt;the question was deleted&lt;/a&gt;). Basically, the audio filter settings are passed from a Windows program to the DSP device presumably as coefficients and then generic descriptions of those filters (boost/cut, frequency and bandwidth) are passed back from the box to the software - but only if it somehow recognizes the filter setting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to be able to generate the filter settings separately from the manufacturer software, so I need to know how they are calculated. I've not been able to deduce how this is structured from observing the USB communication that I've gathered. So, I wonder if AI could do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I go about creating an AI to send commands to the box (I know how to communicate with the box and have a framework for how these types of commands are phrased) and then look at the responses to either further decode the system and/or create an algorithm for creating filters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The communication with the DSP mixer box is basically via &quot;Serial&quot; commands and although it uses a USB port, there is a significant bottleneck inside the command control system in the mixer box. Any attempts to reverse engineer may encounter problems based on the sheer amount of time that it would take to compile enough data. Or not.&lt;/p&gt;&#xA;" OwnerUserId="27891" LastEditorUserId="2444" LastEditDate="2019-08-12T09:35:17.460" LastActivityDate="2021-05-03T16:03:25.267" Title="Can AI be used to reverse engineer a black box?" Tags="&lt;ai-design&gt;&lt;audio-processing&gt;&lt;signal-processing&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13908" PostTypeId="1" CreationDate="2019-08-12T06:46:30.270" Score="2" ViewCount="47" Body="&lt;p&gt;I am actually reading the linear classification. There is a question in the question set behind the chapter in the book as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sketch two multimodal distributions for which a linear discriminant could give excellent or possibly even the optimal classification accuracy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have no idea about how to get the optimal solution on linear classification, any ideas?&lt;/p&gt;&#xA;" OwnerUserId="27893" LastEditorUserId="2444" LastEditDate="2021-01-04T23:00:36.700" LastActivityDate="2021-01-04T23:00:36.700" Title="When could a linear discriminant give excellent or possibly even the optimal classification accuracy?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13909" PostTypeId="2" ParentId="13905" CreationDate="2019-08-12T07:27:30.593" Score="1" Body="&lt;p&gt;The impossibility is referring how to &lt;em&gt;learn&lt;/em&gt; the disentangled representations from the observed distribution or to &lt;em&gt;know&lt;/em&gt; whether you have a disentangled representation in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, an unsupervised learning agent tasked with learning a disentangled transformation of some features &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; needs to infer a set of features from the data which are not entangled, but the supplied data will always have many equally valid entangled solutions - valid from the point of view of describing the distribution of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An analogy would be &quot;I have observed the value 50, and know it is the sum of 3 numbers. What are those numbers?&quot;. Whilst the correct answer exists, and it is possible to guess it, it cannot be inferred from the supplied information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The part of the proof you quote shows that the multiple equivalent entangled feature sets exist, and theoretically cannot be separated from a &quot;true&quot; disentangled feature set on the basis of knowing the distribution. Once you accept this, it is indeed just a short hop in logic to say that the disentangled features are not learnable - there is no way for a learning system to differentiate between the entangled and disentangled features that explain the distribution, and a large (infinite) set of valid entangled features are guaranteed to exist, which will confound attempts to find a perfect solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth noting that the impossibility refers to learning perfect solutions, and that the proof does not rule out useful or practical approximate solutions, or solutions that augment unsupervised learning by applying some additional rules or a semi-supervised approach.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2019-08-12T09:32:11.020" LastActivityDate="2019-08-12T09:32:11.020" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13910" PostTypeId="1" CreationDate="2019-08-12T09:05:22.343" Score="2" ViewCount="77" Body="&lt;p&gt;I am using a CNN for function approximation using geospatial data. &#xA;The input of the function I am trying to approximate consists of all the spatial distances between N location on a grid and all the other points in the grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of now I implemented a CNN that takes an &quot;image&quot; as input. The image has N channels, one for each location of interest. Each i-th channel is a matrix representing my grid, where the pixel values are the distance between each point in the grid and the i-th location of interest. The labels are the N values computed via the actual function I want to approximate. N can be up to 100. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here an example input of the first layer&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m6fMT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m6fMT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I could see the train and validation loss go down, but since it is a bit of a unusual application for a CNN (to my knowledge the input channels are at most 3, RGB) I was wondering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;does this many-channel-input approach have any pitfalls?&lt;/li&gt;&#xA;&lt;li&gt;will I be able to obtain a good accuracy or are there any hard limits I am not aware of?&lt;/li&gt;&#xA;&lt;li&gt;are there any other similar application in literature?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27677" LastEditorUserId="27677" LastEditDate="2019-08-12T09:42:44.030" LastActivityDate="2019-09-12T12:02:10.293" Title="Tweaking a CNN for large number of input channels" Tags="&lt;convolutional-neural-networks&gt;&lt;architecture&gt;&lt;function-approximation&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13912" PostTypeId="2" ParentId="13861" CreationDate="2019-08-12T11:16:46.017" Score="-1" Body="&lt;p&gt;It's possible. I've used the &lt;code&gt;olden()&lt;/code&gt; function from &lt;code&gt;NeuralNetTools&lt;/code&gt;.&#xA;Please take a look at this example found online:&#xA;&lt;a href=&quot;http://blogs2.datall-analyse.nl/2016/02/19/rcode_variable_importance_neural_network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blogs2.datall-analyse.nl/2016/02/19/rcode_variable_importance_neural_network/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26804" LastActivityDate="2019-08-12T11:16:46.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13913" PostTypeId="2" ParentId="13867" CreationDate="2019-08-12T11:24:53.363" Score="0" Body="&lt;p&gt;If you have that sort of a priori knowledge about your environment it, as you said, will simplify the problem substantially. From what I gather you have done a good amount of background research and simply want to apply UTC MCTS(or similar) to the environment. You mention that &quot;chance nodes won't ever be repeated in the selection phase&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand what you are asking correctly, you can simply use what you know to alter the way you search nodes of the tree. I.e you can essentially act greedily wrt the initial chance nodes and then slowly decay that search strategy as the training progresses(to avoid convergence to local maxima).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I encourage you to dig a bit deeper into the methods around exploration vs. exploitation as there may be an elegant solution to this problem in particular.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-08-12T11:24:53.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13914" PostTypeId="2" ParentId="13908" CreationDate="2019-08-12T11:31:08.910" Score="1" Body="&lt;p&gt;A linear model has a linear decision boundary. So in the case of the question, you need to draw two multimodal distributions whose domains do not overlap at all and then you can just say the linear model would should all to the right of some number would be class 1 and all to the left would be class 2&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-12T11:31:08.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13915" PostTypeId="2" ParentId="13907" CreationDate="2019-08-12T11:53:37.070" Score="0" Body="&lt;p&gt;Yes this is entirely possible. As was previously mentioned, complex connectionist systems are often thought of as black boxes(despite us being able to &quot;look in&quot; the box given enough computation and analysis) because of the difficulty in understanding learning and the networks ultimate decision making.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we can model the problem as such: given an input of filter settings(and presumably some information about the audio), predict the target descriptors as an output. All you really need to do is generate a dataset from the program and then train it in a multi-label classification context to predict the output descriptors.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-08-12T11:53:37.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13916" PostTypeId="1" CreationDate="2019-08-12T13:27:06.423" Score="3" ViewCount="245" Body="&lt;p&gt;I came across 'Amber'(&lt;a href=&quot;https://ambervideo.co/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ambervideo.co/&lt;/a&gt;) where they are claiming that they have trained their AI to find patterns emerging due to artificially created videos which are invisible to naked eye. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering that the people who are creating deepfakes can as well their AI's to remove these imperfections and so the problem reduces to 'cat-mouse' game where having more resources(to train their AI) is more crucial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not work in AI and vision and so I may be missing some trivial points in the area. I would really appreciate if detailed explanation or relevant resources are given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Most of the people who do manipulate the media news or create fake news could afford more resources than an average citizen. So, is the future is really going to be dark where only few strong have even more control on the society than today?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean even though there are fake photos created by photo shop, most of the good photo-shopped photos do take a long time to make. But if AIs can be trained to do that then it is more about having large resources. Are there related works which give hope to know real from fakes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: I realize that after the edit, the question also went tangential to the topic-tags here. Please let me if there are relevant tags.&lt;/p&gt;&#xA;" OwnerUserId="27904" LastEditorUserId="1671" LastEditDate="2019-08-12T19:41:52.930" LastActivityDate="2019-08-18T13:12:33.027" Title="Can we combat against deepfakes?" Tags="&lt;social&gt;&lt;deepfakes&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ClosedDate="2020-10-05T21:28:10.280" ContentLicense="CC BY-SA 4.0" />
  <row Id="13917" PostTypeId="1" CreationDate="2019-08-12T13:38:44.793" Score="1" ViewCount="154" Body="&lt;p&gt;I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!python create_pretraining_data.py \&#xA;          --input_file=/input_path/input_file.txt \&#xA;          --output_file=/tf_path/tf_examples.tfrecord \&#xA;          --vocab_file=/vocab_path/uncased_L-12_H-768_A-12/vocab.txt \&#xA;          --do_lower_case=True \&#xA;          --max_seq_length=128 \&#xA;          --max_predictions_per_seq=20 \&#xA;          --masked_lm_prob=0.15 \&#xA;          --random_seed=12345 \&#xA;          --dupe_factor=5&#xA;&#xA;!python run_pretraining.py \&#xA;      --input_file=/tf_path/tf_examples.tfrecord \&#xA;      --output_dir=pretraining_output \&#xA;      --do_train=True \&#xA;      --do_eval=True \&#xA;      --bert_config_file=/bert_path/uncased_L-12_H-768_A-12/bert_config.json \&#xA;      --init_checkpoint=/bert_path/uncased_L-12_H-768_A-12/bert_model.ckpt\&#xA;      --train_batch_size=32 \&#xA;      --max_seq_length=128 \&#xA;      --max_predictions_per_seq=20 \&#xA;      --num_train_steps=20 \&#xA;      --num_warmup_steps=10 \&#xA;      --learning_rate=2e-5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have run a pre-trained BERT model with some domain of corpora from scratch. I have got the checkpoints and graph.pbtxt file from the code above. But I am unaware on how to use those files for evaluating semantic text similarity test file.&lt;/p&gt;&#xA;" OwnerUserId="27902" LastEditorUserId="2444" LastEditDate="2019-11-01T02:45:33.020" LastActivityDate="2019-11-01T02:45:33.020" Title="How to use pretrained checkpoints of BERT model on semantic text similarity task?" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13918" PostTypeId="2" ParentId="13861" CreationDate="2019-08-12T15:03:21.907" Score="2" Body="&lt;p&gt;This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to train on. I would us a rnn for with a softmax output layer that ranks features by performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a good explanation of softmax here: &lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax&lt;/a&gt;&#xA;basically it will assign probability values for each output node with all of these values adding up to 1.0 &lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-08-13T21:34:06.413" LastActivityDate="2019-08-13T21:34:06.413" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13919" PostTypeId="2" ParentId="13867" CreationDate="2019-08-12T15:15:22.987" Score="4" Body="&lt;p&gt;You can try using an &quot;Open-Loop&quot; MCTS approach, instead of the standard &quot;closed-loop&quot; one, and eliminate chance nodes altogether. See, for example, &lt;a href=&quot;http://www.diego-perez.net/papers/OpenLoopGVG.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open Loop Search for General Video Game Playing&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a &quot;standard&quot; (closed-loop) implementation, you would store a game state in every normal (non-chance) node. Whenever there is a chance event, you would stochastically traverse to one of its children, and then have a normal node with a &quot;deterministic&quot; game state again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an open-loop approach, you do not store game states in any node (except possibly the root nodes), because nodes no longer deterministically correspond to specific game states. &lt;strong&gt;Every node in an open-loop MCTS approach only corresponds to the sequence of actions that leads to it from the root node&lt;/strong&gt;. This completely eliminates the need for chance nodes, and results in a significantly smaller tree because you only need a single path in your tree for every possible unique sequence of actions. A single sequence of actions may, depending on stochastic events, lead to a distribution over possible game states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In every separate MCTS iteration, you would re-generate game states again by applying moves &quot;along the edges&quot; as you traverse through the tree. You also &quot;roll the dice&quot; again for any stochastic events. If your MCTS iteration traverses a certain path of the tree often enough, it will still be able to observe all the possible stochastic events through sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, &lt;strong&gt;given an infinite amount of time, the closed-loop approach with explicit chance nodes will likely perform much better&lt;/strong&gt;. But when you have a small amount of time (as is the case in the real-time video game setting considered in the paper I linked above), an open-loop approach without explicit chance nodes may perform better.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you prefer the closed-loop approach with explicit chance nodes, you could try some mix of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Allowing MCTS to prioritise promising parts of the search tree over parts that have not been visited at all (i.e. do not automatically prioritise nodes with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; visits). For example, instead of giving unvisited node a value estimate of &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt; (this is how you could interpret the automatic selection of them), you could give them a value estimate equal to the value estimate of the parent node, and just apply the UCB1 equation directly.&lt;/li&gt;&#xA;&lt;li&gt;Use AMAF value estimates / RAVE / &lt;a href=&quot;https://www.lamsade.dauphine.fr/~cazenave/papers/grave.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRAVE&lt;/a&gt; in your selection phase. This allows you to very quickly learn some crude value estimates for moves that you have never selected in the Selection phase yet, by generalising from observations of playing them in the Play-out phase. I have noticed that the &quot;standard&quot; implementation of RAVE / GRAVE, without an explicit UCB-like exploration term, does not mix well with my previous suggestion of using a non-infinite value estimate for unvisited children. It may be good to consider a UCB-like variant with an explicit exploration term instead.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-08-12T15:15:22.987" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13921" PostTypeId="1" CreationDate="2019-08-12T16:08:40.787" Score="4" ViewCount="183" Body="&lt;p&gt;I am working on a research project in a domain where other related works have always resorted to deep Q-learning. The motivation of my research stems from the fact that the domain has an inherent structure to it, and should not require resorting to deep Q-learning. Based on my hypothesis, I managed to create a tabular Q-learning based algorithm which uses limited domain knowledge to perform on-par/outperform the deep Q-learning based approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that model &lt;em&gt;interpretability&lt;/em&gt; is a subjective and sometimes vague topic, I was wondering if my algorithm should be considered interpretable. The way I understand it, the lack of interpretability in deep-learning-based models stems from the stochastic gradient descent step. However, in case of tabular Q-learning, every chosen action can always be traced back to a finite set of action-value pairs, which in turn are a deterministic function of inputs of the algorithm, although over multiple training episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe in using deep-learning-based approaches conservatively only when absolutely required. However, I am not sure how to justify this in my paper without wading into the debated topic of model interpretability. I would greatly appreciate any suggestions/opinions regarding this. &lt;/p&gt;&#xA;" OwnerUserId="27910" LastEditorUserId="2444" LastEditDate="2019-08-12T16:40:53.653" LastActivityDate="2019-08-13T21:15:30.703" Title="Is tabular Q-learning considered interpretable?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;explainable-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13922" PostTypeId="2" ParentId="13917" CreationDate="2019-08-12T18:13:56.810" Score="1" Body="&lt;p&gt;Have a look at &lt;a href=&quot;https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have the two sentences as first and second use the next sentence score as a similarity measure. You can further fine-tune your model on some semantic similarity tasks like Sent-Eval or your own dataset if you have one&lt;/p&gt;&#xA;" OwnerUserId="27851" LastActivityDate="2019-08-12T18:13:56.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13923" PostTypeId="2" ParentId="13865" CreationDate="2019-08-12T18:24:01.307" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Absolutely these traps and snares are a form of automation.&lt;/strong&gt;  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;They take a task--harvesting small animals--which was traditionally done by hunting them, and make the process automatic.  The mechanism requires a human to set up, but its function is automatic.  This is to say that the mechanism operates without human involvement.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Absolutely this is a form of computation.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As DuttaA observed, these machines utilize a simple &quot;IF/THEN&quot; statement.  In the case of the snare:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;IF the hook is displaced from the base, THEN the sapling straightens&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These simple machines will also return True or False:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TRUE: The trap catches an animal &lt;br&gt;&#xA;  FALSE: The trap is sprung but empty&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The small animals are the input and, potentially, the output, depending on whether the mechanism returns &quot;true&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&lt;em&gt;(The use of &quot;true&quot; has historically included phrases such as &quot;their aim was true&quot; in the sense of shooting an arrow or throwing a spear.)&lt;/em&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-08-12T18:24:01.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13924" PostTypeId="2" ParentId="13916" CreationDate="2019-08-12T18:28:09.443" Score="1" Body="&lt;p&gt;As mshlis begins to touch on, yes we can. However, it will be an unending war. There are quite a few reasons for this. For one, the problem itself is not simple. There are many different 'versions' of the deepfakes framework out in the wild at this point, any algorithm you create to try and spot them would have to work for all of the different iterations. Another reason is the systems that would be used to combat it can be quite easily fooled(&lt;a href=&quot;https://arxiv.org/abs/1412.1897&quot; rel=&quot;nofollow noreferrer&quot;&gt;see&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the most glaring, and unending problem comes from the architecture itself. Let us say we create a perfect algorithm that is foolproof and extremely accurate. Even then, all one would have to do is use that algorithm as the discriminator during training of your deepfakes model, and bing-bang-boom, your deepfake detection model is busted. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2019-08-13T05:09:59.640" LastActivityDate="2019-08-13T05:09:59.640" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13925" PostTypeId="1" CreationDate="2019-08-12T19:37:25.763" Score="1" ViewCount="55" Body="&lt;p&gt;&quot;The final 9 planes encode possible underpromotions for pawn moves or captures in two possible diagonals, to knight, bishop or rook  respectively.   Other  pawn  moves  or  captures  from  the  seventh  rank  are  promoted  to  a queen.&quot; &#xA;Doesn't this mean that the network does not know that it can promote to a queen?&lt;/p&gt;&#xA;" OwnerUserId="19604" LastActivityDate="2019-08-12T20:13:49.117" Title="Alpha Zero queen promotion" Tags="&lt;neural-networks&gt;&lt;chess&gt;&lt;alphazero&gt;&lt;deepmind&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13926" PostTypeId="1" CreationDate="2019-08-12T19:58:00.797" Score="2" ViewCount="605" Body="&lt;p&gt;Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can &lt;a href=&quot;https://allennlp.org/elmo&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELMO embeddings&lt;/a&gt; be used to train a model that can give you the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; most similar sentences to an input string?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, gensim provides a &lt;a href=&quot;https://radimrehurek.com/gensim/models/doc2vec.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;doc2vec&lt;/a&gt; model that can be trained on a list of strings, then you can use the trained model to infer a vector from some input string. That inferred vector can then be used to find the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; most similar vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could something similar be done, but using ELMO embedding instead?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any guidance would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="27915" LastEditorUserId="2444" LastEditDate="2019-08-13T21:50:49.853" LastActivityDate="2019-08-13T23:16:12.177" Title="Can ELMO embeddings be used to find the n most similar sentences?" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13927" PostTypeId="2" ParentId="13925" CreationDate="2019-08-12T20:13:49.117" Score="3" Body="&lt;p&gt;It means that there is no explicit coding of action choices to promote to queen, it is the default assumption if the underpromotion actions are not taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Alpha Zero chess implementation can represent promotion to queen by not selecting an underpromotion action, whilst moving a pawn so that it qualifies for promotion.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-08-12T20:13:49.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13929" PostTypeId="2" ParentId="13921" CreationDate="2019-08-12T21:58:44.407" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1901.04592.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;There is not a widely accepted definition of explainable AI (XAI)&lt;/a&gt;. However, as a rule of thumb (my rule of thumb), if you can't explain it &lt;em&gt;easily&lt;/em&gt; to a layperson (or even an expert), then the model or algorithm is not (very) interpretable. There are other concepts related to XAI, such as accountability (who is responsible for what?), transparency and fairness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the final decision of (trained) decision tree can easily be explained to (almost) any person, so a (trained) decision tree is a relatively interpretable model. See the chapter &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/tree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;4.4. Decision Tree&lt;/a&gt; of the book &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning: A Guide for Making Black Box Models Explainable&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An artificial neural network (ANN) is usually considered not very interpretable because, unless you attempt to understand which parts of the network contribute to the output of the ANN (for example, with the technique &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140&quot; rel=&quot;nofollow noreferrer&quot;&gt;layer-wise relevance propagation&lt;/a&gt;), then you cannot immediately or easily understand the output or decision of the ANN, given that an ANN involves many non-linear functions, which produce unintuitive outcomes. In other words, it is more difficult to attribute the contributions of each unit of an ANN to the output of the same ANN than to explain e.g. the decision of a decision tree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of deep reinforcement learning (DRL), the ANN is used to approximate the value or policy functions. This approximation is, in the first place, the main reason behind the low interpretability of deep RL models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-learning is an algorithm, so it is not a model, like an ANN. Q-learning is used to learn a state-action value function, denoted with &lt;span class=&quot;math-container&quot;&gt;$Q: S \times A \rightarrow \mathbb{R}$&lt;/span&gt;, which can then be used to derive another function, &lt;a href=&quot;https://ai.stackexchange.com/q/12274/2444&quot;&gt;the policy&lt;/a&gt;, which can then be used to take actions. In a way, Q-learning is similar to gradient descent, because both are machine learning (or optimization) algorithms. The &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function is a model of the environment, given that, for each state, it represents the expected amount of reward that can be obtained, so, in a certain way, the learned &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function represents a prediction of reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the learned tabular &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function interpretable? Yes, it is &lt;em&gt;relatively&lt;/em&gt; interpretable, but how much? What kind of interpretation do you really need? It depends on the context and people that need the interpretation or explanation. A reinforcement learning researcher will usually be satisfied with the usual explanation of the inner workings of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, Markov decision processes, etc., because the usual RL researcher is not concerned with the really important problems that involve the life of people and other beings. However, for example, in the context of healthcare, doctors might not just be interested in the explanation &quot;expected maximum future reward&quot;, but they might also be interested in the environment, &lt;a href=&quot;https://ai.stackexchange.com/q/12908/2444&quot;&gt;the credit assignment problem&lt;/a&gt;, the meaning and effectiveness of the reward function with respect to the actual problem that needs to be solved, in a probabilistic interpretation of the results (rather than just a mere action that needs to be taken), possible alternative good actions, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently, there have been some attempts to make RL and, in particular, deep RL more interpretable and explainable. In the paper &lt;a href=&quot;https://arxiv.org/pdf/1804.02477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Programmatically Interpretable Reinforcement Learning&lt;/a&gt; (2019), Verma et al. propose a more interpretable (than deep RL) RL framework that is based on the idea of learning policies that are represented in a human-readable language. In the paper &lt;a href=&quot;https://arxiv.org/pdf/1905.10404.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;InfoRL: Interpretable Reinforcement Learning using Information Maximization&lt;/a&gt; (2019), the authors focus on learning multiple ways of solving the same task and they claim that their approach provides more interpretability. In the paper &lt;a href=&quot;http://www.ecmlpkdd2018.org/wp-content/uploads/2018/09/246.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees&lt;/a&gt; (2018), the authors also claim that their approach &lt;em&gt;facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, deep RL should not necessarily be avoided: it depends on the context (e.g., it is usually perfectly fine to use deep RL to solve video games). However, in cases where liability is an issue, then deep RL should also be explainable or more explainable alternatives should also be taken into account.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-13T21:15:30.703" LastActivityDate="2019-08-13T21:15:30.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13930" PostTypeId="2" ParentId="13247" CreationDate="2019-08-13T03:58:42.020" Score="2" Body="&lt;p&gt;In your example, the output node would still get a value from &lt;code&gt;Input1&lt;/code&gt;, even though &lt;code&gt;Input2&lt;/code&gt; is disabled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the child was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Child = {&#xA;          (1, Input1, Output1),&#xA;          (2, Input2, Output2) //Disabled&#xA;         }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;Output2&lt;/code&gt; would return 0, meaning it wasn't activated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question, it is up to your implementation. You could:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) Use only the connection genes in crossover, and derive your node genes from the connection genes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) Test if every node is in use, and delete the ones that are not&lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-13T03:58:42.020" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13931" PostTypeId="2" ParentId="12455" CreationDate="2019-08-13T04:09:12.900" Score="1" Body="&lt;p&gt;Your species count will increase as the chance of mutation increases. This is because in every generation, so many genes will be mutated that they have little resemblance of each other, and the distance function doesn't factor in historical markings / innovation numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try lowering the mutation rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the distance function from &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; page 110&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\delta = \frac{c_1E}{N} + \frac{c_2D}{N} + c_3 \cdot \overline{W}.  $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your fitness vary a lot, try ranking the fitnesses in each specie and setting the survival chance based on its rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean a large action space by changing environment, you can set the number of output nodes to the total number of actions, and rank each action, best to worst, then pick the best available action for the state. &lt;/p&gt;&#xA;" OwnerUserId="27789" LastEditorUserId="1671" LastEditDate="2019-08-13T21:29:49.000" LastActivityDate="2019-08-13T21:29:49.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13932" PostTypeId="2" ParentId="11345" CreationDate="2019-08-13T04:21:08.383" Score="0" Body="&lt;p&gt;I'm not familiar with neat-python, but I have implemented NEAT to do openai tasks. If there is a class for initializing a population, you could just use that and have 2 objects like &lt;code&gt;population1&lt;/code&gt; and &lt;code&gt;population2&lt;/code&gt; and call them in the same loop.&lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-13T04:21:08.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13933" PostTypeId="2" ParentId="10641" CreationDate="2019-08-13T04:24:21.347" Score="1" Body="&lt;p&gt;You can use a feed forward style network, so that every node outputs to a higher node except output nodes. This will eliminate connection loops.&lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-13T04:24:21.347" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13935" PostTypeId="1" CreationDate="2019-08-13T07:39:56.690" Score="4" ViewCount="334" Body="&lt;p&gt;For an experiment that I'm working on, I want to train a deep network in a special way. I want to initialize and train a small network first, then, in a specific way, I want to increase network depth leading to a bigger network which is subsequently to be trained. This process will be repeated until one reaches the desired depth. It would be great if anybody heard of anything similar and could point out to me some related work. I think in some paper I read something about a related technique where people used something similar but I don't find it anymore. &lt;/p&gt;&#xA;" OwnerUserId="27047" LastEditorUserId="2444" LastEditDate="2019-08-13T21:24:25.513" LastActivityDate="2019-08-13T21:24:25.513" Title="Iteratively and adaptively increasing the network size during training" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;deep-neural-networks&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13936" PostTypeId="2" ParentId="13935" CreationDate="2019-08-13T08:35:37.343" Score="2" Body="&lt;p&gt;I haven't read any relevant paper about this, but I have seen some implementations based on what you are describing, arbitrarily called &lt;strong&gt;DGNN (Dynamic Growing Neural Network)&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this term can help your search.&lt;/p&gt;&#xA;" OwnerUserId="23818" LastEditorUserId="2444" LastEditDate="2019-08-13T16:39:40.450" LastActivityDate="2019-08-13T16:39:40.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13938" PostTypeId="2" ParentId="13916" CreationDate="2019-08-13T09:37:22.467" Score="2" Body="&lt;p&gt;I think this game will go pretty crazy, because, at some point, the generator AI will be able to generate absolutely perfect images. Actually, no, just perfect enough that no AI can be sure whether they are real or fake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I think the AI war will go onto more than the image, the detector AI will probably evolve to analyze whether this video is logically plausible, for example, by tracking the celebrities' position to prove that it is impossible that he/she was, for example, let's put it this way, being unloyal to his/her partner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, currently, AI can tell whether an image is fake or not better than human because it has seen about a million times more samples than us, but if we know who the person in the image is and we are as stalky as the AI I just described, we can probably work out that this image is implausible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there will be counter measurements to that. But, at that point, we might as well just let the AI rule the world, given that it will have become this smart (lol).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, seriously, if it's smart enough to think this far ahead in this 'real world' problem, then strong AI is nigh.&lt;/p&gt;&#xA;" OwnerUserId="27925" LastEditorUserId="2444" LastEditDate="2019-08-13T22:02:46.790" LastActivityDate="2019-08-13T22:02:46.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13940" PostTypeId="2" ParentId="13926" CreationDate="2019-08-13T10:07:33.630" Score="-1" Body="&lt;p&gt;I'm assuming you are trying to train a network that compares 2 sentences and give how similar they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that you will need the dataset (the list of sentences) and a corresponding list of 'correct answers' (the exact similarity of the sentences, which I'm assuming you don't have?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do you need to compare them using a neural network though? For python, difflib's sequence matcher would be my suggestion, but I'm sure there are many other libraries out there :)&lt;/p&gt;&#xA;" OwnerUserId="27925" LastEditorUserId="27925" LastEditDate="2019-08-13T23:16:12.177" LastActivityDate="2019-08-13T23:16:12.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13941" PostTypeId="2" ParentId="13910" CreationDate="2019-08-13T10:15:07.347" Score="1" Body="&lt;p&gt;As far as I know, more than 3 channel is perfectly fine, since, 3 channels are what we use for images and that's enough since we can only see this many colors, but I don't see why more than that wouldn't work&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your 2nd question is like asking whether or not you will be good at a sport... Just try it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your 3rd question, I've never seen any language AI using CNN instead they all use RNN, not sure if that's what you meant though&lt;/p&gt;&#xA;" OwnerUserId="27925" LastActivityDate="2019-08-13T10:15:07.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13942" PostTypeId="2" ParentId="13901" CreationDate="2019-08-13T10:31:44.640" Score="1" Body="&lt;p&gt;I'm thinking, you can input the video you are trying to edit and make it output the timestamps to cut. You will probably have to manually type in the timestamps that you would cut for that video, or maybe a keylogging program of some sort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works in theory, kinda, but I'm not sure how exactly though. The inconsistent input length is easy enough to deal with, but I'm pretty sure you cant have inconsistent output lengths (otherwise I would be using it long ago). Maybe just make like 100 nodes and make it output (timestamp, confirm) where if confirm is &gt; 0.5 it is an actual cut it wants to make.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But inputting whole videos is never a good idea. It takes way too long and takes like 30 gigs of RAM(I had to use virtual RAM which is ridiculously slow) for like a 5 min video and crashes all the time. Any suggestions though?&lt;/p&gt;&#xA;" OwnerUserId="27925" LastEditorUserId="27925" LastEditDate="2019-08-13T10:42:10.677" LastActivityDate="2019-08-13T10:42:10.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13944" PostTypeId="1" CreationDate="2019-08-13T11:51:01.170" Score="2" ViewCount="261" Body="&lt;p&gt;Consider an MLP that outputs an integer 'rating' of 0 to 4. Would it be correct to say this could be modeled in either of the following ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;map each rating in the dataset to a 'normalized set' between 0 and 1 (i.e. 0, 0.25, 0.5, 0.75, 1), have a single neuron with sigmoid activation at output provide a single decimal value and then take as the rating whatever is closest to that value in the 'normalized set'&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;have 5 output neurons with a softmax activation function output 5 values, each representing a probability of one of the 5 ratings as the outcome, and then take as the rating whichever neuron gives the highest probability?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If this is indeed the case, how does one typically decide 'which way to go'? Approach 1 certainly appears to yield a simpler model. What are the considerations, pros/cons of each approach? Perhaps a couple of concrete examples to illustrate?&lt;/p&gt;&#xA;" OwnerUserId="27920" LastEditorUserId="2444" LastEditDate="2019-08-13T21:09:39.433" LastActivityDate="2019-08-20T12:28:34.353" Title="One vs multiple output neurons" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;multilayer-perceptron&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13945" PostTypeId="2" ParentId="13935" CreationDate="2019-08-13T13:36:27.783" Score="5" Body="&lt;p&gt;&lt;strong&gt;Neuroevolution Through Augmenting Topologies&lt;/strong&gt; or &lt;strong&gt;NEAT&lt;/strong&gt; may be what you are referring to. The original paper by Kenneth O. Stanley is &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEAT combines a neural network and a genetic algorithm. Instead of using back propagation or gradient descent to &quot;train&quot; your network, NEAT creates a population of very simple neural networks (no connections) and evolves them with &lt;strong&gt;fitness evaluation, crossover,&lt;/strong&gt; and &lt;strong&gt;mutation.&lt;/strong&gt; The genome syntax: every connection gene has a few settings. In node, Out node, Weight of connection, activated, and innovation. &lt;strong&gt;In, Out,&lt;/strong&gt; and &lt;strong&gt;Weight&lt;/strong&gt; values are the same as regular neural networks. &lt;strong&gt;Enabled&lt;/strong&gt; and &lt;strong&gt;Disabled&lt;/strong&gt; genes are well, enabled and disabled. The innovation value is possibly the most defining feature of NEAT, since it allows for crossover of different topologies and historical tracking of each connection.&lt;a href=&quot;https://i.stack.imgur.com/J9zMq.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J9zMq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;NEAT can mutate or change both its weights and connections, so for example, &lt;code&gt;Parent1&lt;/code&gt; and &lt;code&gt;Parent2&lt;/code&gt; has 5 of the same connections, represented by innovation / ID numbers 1 through 5. Since they have the same connection nodes, the genetic algorithm will randomly pick either &lt;code&gt;Parent1 weight&lt;/code&gt; or &lt;code&gt;Parent2 weight&lt;/code&gt;. The excess and disjoint genes are inherited from the more fit parent.&lt;a href=&quot;https://i.stack.imgur.com/l0je3.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l0je3.png&quot; alt=&quot;NEAT crossover&quot;&gt;&lt;/a&gt; NEAT will then mutate each genome, shown in the image below.&lt;a href=&quot;https://i.stack.imgur.com/DbJ29.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DbJ29.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-13T13:36:27.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13947" PostTypeId="1" CreationDate="2019-08-13T15:23:07.253" Score="0" ViewCount="21" Body="&lt;p&gt;I'm reading notes on word vectors &lt;a href=&quot;https://web.stanford.edu/class/cs224n/readings/cs224n-2019-notes01-wordvecs1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding:&lt;/p&gt;&#xA;&lt;p&gt;If we have a vocabulary &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, the naive way to represent words in it would be via one-hot-encoding, or in other words, as basis vectors of &lt;span class=&quot;math-container&quot;&gt;$R^{|V|}$&lt;/span&gt; - say &lt;span class=&quot;math-container&quot;&gt;$e_1, e_2,\ldots,e_{|V|}$&lt;/span&gt;. We want to map these to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;, via some linear transformation such that the images of similar words (more precisely, the images of basis vectors corresponding to similar words) have higher inner products. Assuming the matrix representation of the linear transformation given the standard basis of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{|V|}$&lt;/span&gt; is denoted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt;, then the &amp;quot;embedding&amp;quot; of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th vocab word (i.e. the image of the corresponding basis vector &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) is given by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}e_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now suppose we have a context &amp;quot;The cat ____ over a&amp;quot;, CBoW seeks to find a word that would fit into this context. Let the words &amp;quot;the&amp;quot;, &amp;quot;cat&amp;quot;, &amp;quot;over&amp;quot;, &amp;quot;a&amp;quot; be denoted (in the space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) by &lt;span class=&quot;math-container&quot;&gt;$x_{i_1},x_{i_2},x_{i_3},x_{i_4}$&lt;/span&gt; respectively. We take the image of their linear combination (in particular, their average):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat v=\mathcal{V}\bigg(\frac{x_{i_1}+x_{i_2}+x_{i_3}+x_{i_4}}{4}\bigg)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We then map &lt;span class=&quot;math-container&quot;&gt;$\hat v$&lt;/span&gt; back from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{|V|}$&lt;/span&gt; via &lt;em&gt;another&lt;/em&gt; linear mapping whose matrix representation is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$$z=\mathcal{U}\hat v$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then we turn this &lt;em&gt;score vector&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; into softmax probabilities &lt;span class=&quot;math-container&quot;&gt;$\hat y=softmax(z)$&lt;/span&gt; and compare it to the basis vector corresponding to the actual word, say &lt;span class=&quot;math-container&quot;&gt;$e_c$&lt;/span&gt;. For example, &lt;span class=&quot;math-container&quot;&gt;$e_c$&lt;/span&gt; could be the basis vector corresponding to &amp;quot;jumped&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Here's my interpretation of what this procedure is trying to do: given a context, we're trying to learn maps &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; such that given a context like &amp;quot;the cat ____ over a&amp;quot;, the model should give a high score to words like &amp;quot;jumped&amp;quot; or &amp;quot;leaped&amp;quot;, etc. Not just that - but &amp;quot;similar&amp;quot; contexts should also give rise to high scores for &amp;quot;jumped&amp;quot;, &amp;quot;leaped&amp;quot;, etc. For example, given a context &amp;quot;that dog ____ above this&amp;quot; wherein &amp;quot;that&amp;quot;, &amp;quot;dog&amp;quot;, &amp;quot;above&amp;quot;, &amp;quot;this&amp;quot; are represented by &lt;span class=&quot;math-container&quot;&gt;$x_{j_1},x_{j_2},x_{j_3},x_{j_4}$&lt;/span&gt;, let the image of their average be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat w=\mathcal{V}\bigg(\frac{x_{j_1}+x_{j_2}+x_{j_3}+x_{j_4}}{4}\bigg)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This gets mapped to a score vector &lt;span class=&quot;math-container&quot;&gt;$z'=\mathcal{U}\hat w$&lt;/span&gt;. Ideally, both score vectors &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z'$&lt;/span&gt; should have similarly high magnitudes in their components corresponding to similar words &amp;quot;jumped&amp;quot; and &amp;quot;leaped&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Now to the questions:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We create two matrices, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V} \in \mathbb{R}^{n\times |V|}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U} \in \mathbb{R}^{|V|\times n}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is an arbitrary size which defines the size of our embedding space. &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; is the input word matrix such that the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th column of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional embedded vector for word &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; when it is an input to this model. We denote this &lt;span class=&quot;math-container&quot;&gt;$n\times 1$&lt;/span&gt; vector as &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt;. Similarly, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; is the output word matrix. The &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;-th row of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional embedded vector for word &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt; when it is an output of the model. We denote this row of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$u_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How does minimizing the cross-entropy loss between &lt;span class=&quot;math-container&quot;&gt;$e_c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat y$&lt;/span&gt; ensure that basis vectors corresponding to similar words &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_j$&lt;/span&gt; are mapped to vectors in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; that have high inner product? I'm not sure of the mechanism how the above procedure ensures that. In other words, how is it ensured that if words no. &lt;span class=&quot;math-container&quot;&gt;$i_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$i_2$&lt;/span&gt; are similar, then &lt;span class=&quot;math-container&quot;&gt;$\langle v_{i_1}, v_{i_2}\rangle$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\langle u_{i_1}, u_{i_2}\rangle$&lt;/span&gt; have high values?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does the above procedure ensure that linear combinations of words in similar contexts are mapped to &amp;quot;similar&amp;quot; images? Does that even happen? In the above description for example, do &lt;span class=&quot;math-container&quot;&gt;$\hat v$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat w$&lt;/span&gt; corresponding to similar contexts also have a high inner product? If so, how is that ensured?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Maybe my linear algebra is rusty and this is a silly question, but from what I gather, the columns of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; represent the images of OHE vectors (standard basis of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) in the standard basis of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; - i.e. the embedded representation of vocab words. Also, the rows of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; also somehow represent the embedded representation of vocab words in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;. It's not obvious to me why &lt;span class=&quot;math-container&quot;&gt;$v_i=\mathcal{V}e_i$&lt;/span&gt; should be the same as or even similar to &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt;. Again, how does the above procedure ensure that?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27548" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-15T11:44:47.263" Title="Understanding how continuous bag of words method learns embedded representations" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;word2vec&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13948" PostTypeId="1" CreationDate="2019-08-13T15:40:34.577" Score="0" ViewCount="398" Body="&lt;p&gt;I just finished my implementation of NEAT and I want to see the phenotype of each genome. Is there a library for displaying a neural network like this?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XKvZ2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XKvZ2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example of my genome syntax:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[0, 11, 0.9154901559275923, 1, 19],&#xA;[4, 11, 1.3524964932656411, 1, 19],&#xA;[12, 9, -1.755210214894685, 1, 23],&#xA;[11, 12, 0.6193383549414015, 1, 23]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where [In, Out, Weight, Activated?, Innovation]&lt;/p&gt;&#xA;" OwnerUserId="27789" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-13T22:36:00.857" Title="Library for rendering neural network NEAT" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13949" PostTypeId="2" ParentId="13944" CreationDate="2019-08-13T15:52:40.207" Score="1" Body="&lt;p&gt;This depends on whether the output is a continuous or discrete variable. If the output variable is discrete (there are a finite number of possibilities that it can be), as in a classification task (such as this one, where you are trying to place the input into one of 5 categories), you want to use one output neuron for each class. If the variable is continuous, however, you should only use one output neuron. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because of how the training process works. When training your network successively makes adjustments to try and reduce the errors. These adjustments are made in the direction of the error – so if the network predicts a value which is too high then the network's weights are adjusted to make the output value lower. On the other hand if the network's predicts a value which is too low the network's weights are adjusted to make the output bigger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have output neurons labeled 0 to 4 and a training sample with some input value and a target prediction of 2 then the neural network will make its prediction. Once the prediction has made each neuron is adjusted individually – in this case neuron 2 will be adjusted in the direction of the correct probability and all the other neurons will be adjusted in the direction of the incorrect probability. In this way you have one prediction for each class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backpropagation is a about error attribution, and using multiple neurons allows the error of the neural network to be better attributed as the neural network can adjust each neuron individually, and thus adjust the required probabilities for each class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a single neuron with a sigmoid activation function would be less good as the sigmoid function saturates values close to 0 and 1 so there would be an unnatural bias towards category 0 and category 4 over the other categories. The neural network could learn to overcome this, but it would take more time.&lt;/p&gt;&#xA;" OwnerUserId="27933" LastEditorUserId="27933" LastEditDate="2019-08-14T06:43:52.777" LastActivityDate="2019-08-14T06:43:52.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13950" PostTypeId="1" CreationDate="2019-08-13T16:36:48.510" Score="2" ViewCount="22" Body="&lt;p&gt;I want to implement a CNN, but I want to explore what happens when my first layer is a fully-connected one. I still want to use convolutions, of course, but I want to apply them after the first layer. I noticed that the input then loses its 3D structure. Does that mean I can only apply 1d convolutions after that? Is there a non-trivial way to recover the 3d structure, so that 2d convolutions may be applied?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully, when I reconstruct it to have 3d structure the 3d structure is somehow meaningful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also posted this question at &lt;a href=&quot;https://forums.fast.ai/t/how-do-i-recover-the-3d-structure-of-a-layer-after-a-fully-connected-layer-or-a-flatten-layer/52489&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://forums.fast.ai/t/how-do-i-recover-the-3d-structure-of-a-layer-after-a-fully-connected-layer-or-a-flatten-layer/52489&lt;/a&gt; and &lt;a href=&quot;https://discuss.pytorch.org/t/how-do-i-recover-the-3d-structure-of-a-layer-after-a-fully-connected-layer-or-a-flatten-layer/53313&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://discuss.pytorch.org/t/how-do-i-recover-the-3d-structure-of-a-layer-after-a-fully-connected-layer-or-a-flatten-layer/53313&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9289" LastEditorUserId="2444" LastEditDate="2019-08-13T21:44:52.573" LastActivityDate="2019-08-13T21:44:52.573" Title="How do I recover the 3D structure of a layer after a fully-connected layer?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13951" PostTypeId="2" ParentId="6411" CreationDate="2019-08-13T17:25:38.693" Score="0" Body="&lt;p&gt;After the first Conv layer, the size reduced to 20x20&#xA;for the primary caps which is convolutional caps layer&#xA;n final = (n + 2p -f )/s + 1&#xA;which gives a 6x6 output with 256 channels&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6x6x256 is further encoded into capsules of 8 dimensions by reshaping the channels &#xA;i.e. 256/8 = 32&#xA;which gives 6x6x32 = 1152 capsules&lt;/p&gt;&#xA;&#xA;&lt;p&gt;try experimenting with the same hyperparameters first and then try to encode higher-level features by making suitable changes to the hyperparameters. &lt;/p&gt;&#xA;" OwnerUserId="27936" LastActivityDate="2019-08-13T17:25:38.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13952" PostTypeId="1" CreationDate="2019-08-13T17:31:03.910" Score="1" ViewCount="1163" Body="&lt;p&gt;Please tell me that is the LSTM network for the problem of reinforcement learning, as I explain to her what she will get the reward of a prediction, because the output will contain only actions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, well, let's say at first I can play and upload my actions to training so that she sees which actions are right, that is, which she should strive for, but how to make her learn relatively independently?&lt;/p&gt;&#xA;" OwnerUserId="27905" LastActivityDate="2019-10-18T17:03:15.503" Title="LSTM in reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13953" PostTypeId="1" CreationDate="2019-08-13T17:35:25.323" Score="2" ViewCount="26" Body="&lt;p&gt;I'm trying to detect if a given video shot is fast or slow motion. Basically, I need to calculate a &quot;video motion&quot; score in a given video sequence, meaning how fast or slow motion the video is. For instance, if a video is about a car racing or camera moving fast, the score is high. Whereas if the video is about two persons standing/talking, then the motion is low, so the lower score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What comes into my mind is using &lt;strong&gt;optical flow&lt;/strong&gt; which is already an implemented function in OpenCV. I never used it. But I don't know how to interpret or use it for a &quot;motion score&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is optical flow applicable here? How can I use it to calculate a score? In particular, if there is a ML/Deep learning model that already does it, please share it.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="2444" LastEditDate="2019-08-13T21:08:00.483" LastActivityDate="2019-08-13T21:08:00.483" Title="How can I detect fast and slow motion in videos?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;long-short-term-memory&gt;&lt;image-processing&gt;&lt;signal-processing&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13955" PostTypeId="2" ParentId="9550" CreationDate="2019-08-13T18:24:48.027" Score="3" Body="&lt;p&gt;The first equation deals with distance. &lt;strong&gt;Delta&lt;/strong&gt;, or distance, is the measure of how &lt;strong&gt;compatible&lt;/strong&gt; two genomes are with each other. &lt;code&gt;c1&lt;/code&gt;, &lt;code&gt;c2&lt;/code&gt; and &lt;code&gt;c3&lt;/code&gt; are parameters you set to dictate the importance of &lt;code&gt;E&lt;/code&gt;,  &lt;code&gt;D&lt;/code&gt; and &lt;code&gt;W&lt;/code&gt;. Note that if you change c&lt;code&gt;c1&lt;/code&gt;, &lt;code&gt;c2&lt;/code&gt; or &lt;code&gt;c3&lt;/code&gt;, you will most likely also have to change dt, which is the distance threshold, or the maximum distance apart 2 genomes can be before they are separated into different species. &lt;code&gt;E&lt;/code&gt; represents the total number of &lt;strong&gt;excess&lt;/strong&gt; genes. &lt;code&gt;D&lt;/code&gt; represents the total number of &lt;strong&gt;disjoint&lt;/strong&gt; genes in both genomes, &lt;code&gt;W&lt;/code&gt; represents the total &lt;strong&gt;weight difference&lt;/strong&gt; between genes that match, and finally, &lt;code&gt;N&lt;/code&gt; represents the number of connections / genes the genome with the larger number of connections has. For example, take the following 2 genomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[1,.25][2,.55],[4,.78],[6,.2]]&#xA;and&#xA;[[1,.15][3,.92],[5,.37]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the 0 index represents innovation number and the 1 index represents weight value. &lt;code&gt;E&lt;/code&gt;  would be 1, since there is 1 excess gene, gene &lt;code&gt;6&lt;/code&gt;. &lt;code&gt;D&lt;/code&gt; would be 4, since connections &lt;code&gt;2&lt;/code&gt; and &lt;code&gt;4&lt;/code&gt; are not in genome 2, and connections &lt;code&gt;3&lt;/code&gt; and &lt;code&gt;5&lt;/code&gt; are not in genome 1. &lt;code&gt;W&lt;/code&gt; would be .10, since only connection &lt;code&gt;1&lt;/code&gt; is shared between the two genomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second formula is a bit more complicated. From my understanding, correct me if I'm wrong, this is a formula for adjusting fitness. &lt;code&gt;f′i&lt;/code&gt; is the adjusted fitness, which will replace the original fitness, &lt;code&gt;fi&lt;/code&gt;. For every genome &lt;code&gt;j&lt;/code&gt; in the &lt;strong&gt;entire population&lt;/strong&gt;, yes, entire population and not just every genome in its specie, it will calculate the distance between &lt;code&gt;j&lt;/code&gt; and &lt;code&gt;i&lt;/code&gt;, &lt;code&gt;i&lt;/code&gt; being the genome of fitness &lt;code&gt;fi&lt;/code&gt;. Then it will sum up all the distance    values, and divide the original fitness &lt;code&gt;fi&lt;/code&gt; by the total distance sum, and set &lt;code&gt;f'i&lt;/code&gt; to that. Next,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Every species is&#xA;  assigned a potentially different number of offspring in proportion to the sum of adjusted fitnesses &lt;code&gt;f`i&lt;/code&gt; of its member organisms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This assigning of number of species offspring is used so that one specie can't take over the entire population, which is the whole point of speciation in the first place, so in conclusion, these two formulas are vital to the function and efficiency of the NEAT algorithm.&lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-13T18:24:48.027" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13956" PostTypeId="2" ParentId="8333" CreationDate="2019-08-13T18:32:59.430" Score="0" Body="&lt;p&gt;In some implementations, the specie representatives change each generation. This allows for a dynamic definition of what a specie is. If you speciate from scratch(meaning each specie is assigned a new representative) every generation, you won't have the problem where a representative leaves its specie.&lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-13T18:32:59.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13957" PostTypeId="1" CreationDate="2019-08-13T18:34:45.210" Score="2" ViewCount="271" Body="&lt;p&gt;I was trying to understand the definition of 2d convolutions vs 3d convolutions. I saw the &quot;simplest definition&quot; according to Pytorch and it seems the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2d convolutions map &lt;span class=&quot;math-container&quot;&gt;$(N,C_{in},H,W) \rightarrow (N,C_{out},H_{out},W_{out})$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;3d convolutions map &lt;span class=&quot;math-container&quot;&gt;$(N,C_{in},D,H,W) \rightarrow (N,C_{out},D_{out},H_{out},W_{out})$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which make sense to me. However, what I find confusing is that I would have expected images to be considered 3D tensors but we apply 2D convolutions to them. Why is that? Why is the channel tensor not part of the &quot;definitionality of the images&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also asked this question at &lt;a href=&quot;https://forums.fast.ai/t/what-is-the-difference-between-2d-vs-3d-convolutions/52495&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://forums.fast.ai/t/what-is-the-difference-between-2d-vs-3d-convolutions/52495&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9289" LastEditorUserId="2444" LastEditDate="2019-08-13T21:04:51.323" LastActivityDate="2019-08-13T21:04:51.323" Title="What is the difference between 2d vs 3d convolutions?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13959" PostTypeId="1" CreationDate="2019-08-13T19:20:23.760" Score="6" ViewCount="54" Body="&lt;p&gt;We have the popular &lt;a href=&quot;https://github.com/summanlp/textrank&quot; rel=&quot;noreferrer&quot;&gt;TextRank&lt;/a&gt; API which given a text, ranks keywords and can apply summarization given a predefined text length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is a similar tool for &lt;strong&gt;video&lt;/strong&gt; summarization. Maybe a library, a deep model or ML-based tool that given a video file and a length, it ranks frames, or video scenes/shots. I'd like to generate a short summary of a video with visual features.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-08-13T22:18:09.057" LastActivityDate="2021-03-26T02:03:00.277" Title="Video summarization similar to Summe's TextRank" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13960" PostTypeId="2" ParentId="13957" CreationDate="2019-08-13T19:20:27.837" Score="1" Body="&lt;p&gt;Looking at it from the perspective of input to output in that fashion is probably not the best. So lets start with our goal and how these ND convolutions accomplish that (Note these are in my own words, and may not be best stated).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Assumption:&lt;/strong&gt; There exists highly correlative local associations  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt; Have a linear model that takes advantage of these local associations  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution:&lt;/strong&gt; The ND Convolution  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Explanation:&lt;/strong&gt; ND convolutions take advantage of our locality assumption by connecting only local nodes/neurons. The fact that its a sliding window allows us to learn filters for any location along with ones that can be reusable.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Your Question:&lt;/strong&gt;  &lt;em&gt;Where does the N in ND convolution matter?:&lt;/em&gt;&lt;br&gt;&#xA;When using an ND convolution we are working off the assumption that there exists this locality in &lt;strong&gt;N&lt;/strong&gt; dimensions and nothing more. So we connect all other components of the input in a dense matter because we have no assumptions to work on in this space. So now going to the shapes you mentioned such as the input and output of the &lt;strong&gt;2&lt;/strong&gt;D convolution. We are convolving a filter of size &lt;span class=&quot;math-container&quot;&gt;$(C_{in}, k_h, k_w)$&lt;/span&gt; with a &lt;span class=&quot;math-container&quot;&gt;$(C_{in},H,W)$&lt;/span&gt; activation shaped map (the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; just refers to the number of activation maps, and there is no association between them). We use &lt;span class=&quot;math-container&quot;&gt;$C_{in}$&lt;/span&gt; channels on the kernel because we are not making any assumptions about locality between channels.  On the other hand in a &lt;strong&gt;3&lt;/strong&gt;D convolution we make locality assumption is 3 dimensions, so our kernel will be (&lt;span class=&quot;math-container&quot;&gt;$C_{in}, k_h, k_w, k_r$&lt;/span&gt;). These kernel sizes are actually determined by the input size, the amount of dimensions of the kernel will actually match the inputs (minus the batch) because it needs to densely match each one.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may be thinking now, that in torch this is not the case: This is because its rare to want a 2D Convolution with an inputs that's different than the one you mention, so they only implemented it for a singular shape. I hope this clears up the convolutions and helps you understand not just for 2 and 3 dimensional convolutions, but for all N. &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-13T19:20:27.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13961" PostTypeId="5" CreationDate="2019-08-13T20:11:22.537" Score="0" Body="&lt;p&gt;See, for example, &lt;a href=&quot;https://arxiv.org/pdf/1707.02268.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Text Summarization Techniques: A Brief Survey&lt;/a&gt; for an overview of the field.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-13T20:11:22.537" LastActivityDate="2019-08-13T20:11:22.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13962" PostTypeId="4" CreationDate="2019-08-13T20:11:22.537" Score="0" Body="For questions related to (automatic) text summarization, which is the task of producing a concise&#xD;&#xA;and fluent summary of a text or document while preserving key information content and&#xD;&#xA;the overall meaning of the original document. For example, search engines are an example of an application that generates summaries as the previews of the documents or websites." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-13T20:11:22.537" LastActivityDate="2019-08-13T20:11:22.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13963" PostTypeId="2" ParentId="13926" CreationDate="2019-08-13T21:23:31.040" Score="2" Body="&lt;p&gt;I ended up finding &lt;a href=&quot;https://ai.intelligentonlinetools.com/ml/document-similarity-in-machine-learning-text-analysis-with-elmo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; which does what I'm looking for. &#xA;Below is the portion of code I adapted for my needs&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.metrics.pairwise import cosine_similarity&#xA;&#xA;import tensorflow_hub as hub&#xA;import tensorflow as tf&#xA;&#xA;elmo = hub.Module(&quot;https://tfhub.dev/google/elmo/2&quot;, trainable=True)&#xA;&#xA;def elmo_vectors(x):&#xA;  embeddings=elmo(x, signature=&quot;default&quot;, as_dict=True)[&quot;elmo&quot;]&#xA;&#xA;  with tf.device('/device:GPU:0'):&#xA;    with tf.Session() as sess:&#xA;      sess.run(tf.global_variables_initializer())&#xA;      sess.run(tf.tables_initializer())&#xA;      # return average of ELMo features&#xA;      return sess.run(tf.reduce_mean(embeddings,1))&#xA;&#xA;&#xA;corpus=[&quot;I'd like an apple juice&quot;,&#xA;        &quot;An apple a day keeps the doctor away&quot;,&#xA;         &quot;Eat apple every day&quot;,&#xA;         &quot;We buy apples every week&quot;,&#xA;         &quot;We use machine learning for text classification&quot;,&#xA;         &quot;Text classification is subfield of machine learning&quot;]&#xA;&#xA;&#xA;elmo_embeddings=[]&#xA;print (len(corpus))&#xA;for i in range(len(corpus)):&#xA;    print (corpus[i])&#xA;    elmo_embeddings.append(elmo_vectors([corpus[i]])[0])&#xA;&#xA;print ( elmo_embeddings, len(elmo_embeddings))&#xA;print(elmo_embeddings[0].shape)&#xA;sims = cosine_similarity(elmo_embeddings, elmo_embeddings)&#xA;print(sims)&#xA;print(sims.shape)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27915" LastActivityDate="2019-08-13T21:23:31.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13964" PostTypeId="2" ParentId="13886" CreationDate="2019-08-13T22:13:00.000" Score="3" Body="&lt;p&gt;When you want to compare Reinforcement Learning algorithms, you might want to compare the average rewards they generate and how fast and close they get to the optimal policy. However, in the case of comparing it to humans, you might want to compare the game results of all the games played.&lt;/p&gt;&#xA;&lt;h1&gt;Reward Comparison&lt;/h1&gt;&#xA;&lt;p&gt;Often Reinforcement Learning algorithms are compared by using the rewards (either direct, maximum or average in time/iteration). For example, in &lt;a href=&quot;https://www.cse.unsw.edu.au/%7Ecs9417ml/RL1/algorithms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page about RL&lt;/a&gt; a comparison of two algorithms is shown:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zDk9w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zDk9w.png&quot; alt=&quot;Comparing two RL algorithms&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or, when you know the optimal actions, you can plot the number of plays/iterations against the percentage of actions. See for example this &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL comparison on the 10-armed testbed problem&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/buIpx.png&quot; alt=&quot;Comparing two RL algorithms with optimal actions&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1709.06560&quot; rel=&quot;nofollow noreferrer&quot;&gt;Henderson et al. 2017&lt;/a&gt; have a whole section about the evaluation metrics of Reinforcement Learning algorithms. They also comment on the plotting of the average or maximum cumulative rewards, moreover they mention the &lt;em&gt;sample bootstrap&lt;/em&gt; method to create confidence intervals for a better comparison. Lastly, they mention that the &lt;em&gt;significance&lt;/em&gt; of the improvements of the algorithms should be calculated, using a statistical test, such the &lt;a href=&quot;https://stattrek.com/statistics/dictionary.aspx?definition=two-sample%20t-test&quot; rel=&quot;nofollow noreferrer&quot;&gt;two sample t-Test&lt;/a&gt;. Note that you should take into account the distributions of the datasets to choose the right statistical test. An interesting article related to this is &lt;a href=&quot;https://openreview.net/pdf?id=ryx0N3IaIV&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Hitchhiker's Guide to Statistical Comparison of Reinforcement Learning Algorithms&lt;/a&gt; of &lt;a href=&quot;https://openreview.net/pdf?id=ryx0N3IaIV&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colas et al.&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h1&gt;Comparison of plays against humans&lt;/h1&gt;&#xA;&lt;p&gt;To find out how well different algorithms play against humans you should do a large number of games and compare - what you consider - important parameters, for example: did the algorithm won, the time it took to win, number of points gained, etc. These values can then be compared statistically. Note that you have to think well about the setup of these experiments since you only should change the algorithms, the other parameters should stay equal. Therefore, you should - preferably - use a large number of subjects (of different ages, sexes, etc. to cover many types of people), and try to prevent any bias; think about the order, how many games are played, location, time etc.&lt;/p&gt;&#xA;" OwnerUserId="198" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-13T22:13:00.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13965" PostTypeId="2" ParentId="13948" CreationDate="2019-08-13T22:36:00.857" Score="1" Body="&lt;p&gt;I've never used it, but, if you are using Python, have a look at &lt;a href=&quot;https://neat-society.readthedocs.io/en/latest/neat_overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;neat-python&lt;/a&gt;, which is a Python package that implements NEAT and also provides a module, &lt;a href=&quot;https://github.com/CodeReclaimers/neat-python/blob/master/examples/xor/visualize.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;visualize&lt;/code&gt;&lt;/a&gt;, to &lt;em&gt;plot the best and average fitness vs. generation, plot the change in species vs. generation, and to show the structure of a network described by a genome&lt;/em&gt;. See &lt;a href=&quot;https://stackoverflow.com/q/41752291/3924118&quot;&gt;this Stack Overflow thread&lt;/a&gt;, if you encounter any issue while attempting to use this module. See also &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/xor_example.html#visualizations&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at &lt;a href=&quot;https://eplex.cs.ucf.edu/neat_software/#NEAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;this web page&lt;/a&gt;, which lists links to many NEAT implementations in different languages, where some of them provide visualization tools.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-13T22:36:00.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13966" PostTypeId="1" CreationDate="2019-08-14T07:29:38.973" Score="1" ViewCount="55" Body="&lt;p&gt;I have some problems with training CNN :(&lt;/p&gt;&#xA;&lt;p&gt;For example:&#xA;Input 6x6x3, 1 core 3x3x3, output = 4x4x1 =&amp;gt; pool: 2x2x1&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5IkyU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5IkyU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;By backpropagation I calculated deltas for output.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This tutor&lt;/a&gt; and other tutors are explain to calc deltas for weights and Input only for 2D:&lt;/p&gt;&#xA;&lt;p&gt;input*output=deltas for 2D weights&lt;/p&gt;&#xA;&lt;p&gt;filter*out = input delta&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;But how I can to calc weights deltas for 3D filters?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I must to multiply each input by output as below?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;FilterLayer1Delta = OutputDelta * InputLayer1 &lt;strong&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;FilterLayer2Delta = OutputDelta * InputLayer2 &lt;strong&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;FilterLayer3Delta = OutputDelta * InputLayer3 &lt;strong&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="27945" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-14T07:29:38.973" Title="How to train and update weights of filters" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13967" PostTypeId="2" ParentId="13944" CreationDate="2019-08-14T08:14:56.780" Score="0" Body="&lt;p&gt;In the case of one output neuron, you don't have to use sigmoid. As Teymour Aldridge suggested, it would cause a tendency to output 0 or 1. What I normally do is that I set the layer before the output layer to sigmoid of tanh so it won't output ridiculously off numbers, and set the output layer to linear. There would be cases that it outputs something like 1.5, but over time that disappears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps :)&lt;/p&gt;&#xA;" OwnerUserId="27925" LastActivityDate="2019-08-14T08:14:56.780" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13968" PostTypeId="1" CreationDate="2019-08-14T11:20:23.583" Score="2" ViewCount="523" Body="&lt;p&gt;I'm a beginner of RL and currently trying to make DQN agent that can act optimally in a simple situation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the situation agent should decide at what rate to charge or discharge the electrical battery, which is equivalent to buying the electrical energy or selling it, for making money by means of arbitrage. So the action space is for example [-6, -4, -2, 0, 2, 4, 6]kW. The negative numbers mean discharging, and the positive numbers mean charging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a case that battery is empty, discharging actions(-6, -4, -2) should be forbidden.&#xA;Otherwise in a case that battery is fully charged, charging actions(2, 4, 6) should be forbidden.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To deal with this issue, I tried two approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In every step, renewing the action space, which means masking the forbidden action.&lt;/li&gt;&#xA;&lt;li&gt;Give extreme penalties for selecting forbidden actions (in my case the penalty was -9999)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But none of them worked. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first approach, the training curve (the cumulative rewards) didn't converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second approach, the training curve converged, but the charging/discharging results are not reasonable (almost random results).&#xA;I think in second approach, a lot of forbidden actions are selected randomly by the epsilon-greedy policy, and these samples are stored in experience memory, which negatively affect the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The state is defined as [p_t, e_t] where p_t is the market price for selling (discharging) the battery, and e_t is the amount of energy left in the battery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When state = [p_t, e_t = 0], and discharging action (-6), which is forbidden action in this state, is selected, the next state is [p_t, e_t = -6]. And then the next action (2) is selected, then the next state is [p_t, e_t = -4] and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case the &amp;lt; s, a, r, s' &gt; samples  are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;lt; [p_t, 0], -6, -9999, [p_t+1, -6] &gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;lt; [p_t, -6], 2, -9999, [p_t+1, -4] &gt; ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are not expected to be stored in the experience memory because they are not desired samples (e_t should be more than zero). I think this is why desired results didn't come out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what should I do? Please help.&lt;/p&gt;&#xA;" OwnerUserId="27946" LastActivityDate="2019-08-18T12:10:48.793" Title="Reinforcement learning: How to deal with illegal actions?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ClosedDate="2020-09-27T22:41:11.427" ContentLicense="CC BY-SA 4.0" />
  <row Id="13971" PostTypeId="2" ParentId="13968" CreationDate="2019-08-14T13:23:59.317" Score="1" Body="&lt;p&gt;In my project I also had the problem that the action space is not the same for every state of the environment. I do not like the approach to penalty forbidden actions with a high negative reward since it feels a bit like cheating. However it might work, I just haven't tried it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach I used, which you could apply as well, is to integrate an additional function into your action space. This function would map an action to a specific amount of kW. Thereby, depending on the current state, the function maps the action to the amount of kW to charge or discharge your battery with. This has the advantage that you do not have to deal with illegal actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be applied as follows:&#xA;Instead of defining for every action the amount to charge / discharge your battery with, you create a set of functions that defines the respective amount. Here an example with five actions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Action: Discharge the battery entirely&lt;/li&gt;&#xA;&lt;li&gt;Action. Discharge the battery so that half of its capacity remains,&#xA;otherwise do nothing&lt;/li&gt;&#xA;&lt;li&gt;Action: do nothing&lt;/li&gt;&#xA;&lt;li&gt;Action: charge the batter to half its capacity, otherwise do nothing&lt;/li&gt;&#xA;&lt;li&gt;Action: charge the battery to its maximum capacity&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26876" LastActivityDate="2019-08-14T13:23:59.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13972" PostTypeId="2" ParentId="11534" CreationDate="2019-08-14T15:23:10.927" Score="0" Body="&lt;p&gt;It occurs to me that Superintelligence as an ordinal value could be explained as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Artificial General Intelligence is strong utility in all problems that humans can conceive of.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Superintelligence as a category includes problems beyond what humans can conceive of.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An easy way to understand this is to look at the difference between dogs and humans.  Dogs are optimized for certain tasks, such as tracking, but limited in terms of what they can conceive and what tasks they can engage in.  Humans have superintelligence, compared to dogs, because we regularly engage with problems and tasks beyond the conception of canines.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-08-14T15:23:10.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13973" PostTypeId="1" AcceptedAnswerId="14038" CreationDate="2019-08-14T16:15:30.013" Score="5" ViewCount="522" Body="&lt;p&gt;In chapter 6 of &lt;a href=&quot;http://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; (p. 128), they claim temporal difference converges to the maximum likelihood estimate (MLE). How can this be shown formally?&lt;/p&gt;&#xA;" OwnerUserId="26846" LastEditorUserId="2444" LastEditDate="2019-08-18T14:47:00.250" LastActivityDate="2019-08-18T19:13:15.377" Title="How to show temporal difference methods converge to MLE?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13974" PostTypeId="1" CreationDate="2019-08-14T16:53:35.113" Score="2" ViewCount="43" Body="&lt;p&gt;In chapter six of Sutton and Barto (p.128), they claim Monte Carlo methods converge to an estimate minimizing the mean squared error. How can this be shown formally?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bump&lt;/p&gt;&#xA;" OwnerUserId="26846" LastEditorUserId="26846" LastEditDate="2019-08-16T17:06:17.070" LastActivityDate="2019-08-16T17:06:17.070" Title="How to show Monte Carlo methods converge to an estimate which minimizes mean squared error?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;monte-carlo-methods&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13975" PostTypeId="1" CreationDate="2019-08-14T19:57:37.627" Score="6" ViewCount="83" Body="&lt;p&gt;I am curious if there is any advantage of using 3D convolutions on Images like Cifar10/100 or Imagenet. I know that they are not usually used on this data set, though they could because the channel could be used as the &quot;depth&quot; channel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there are only 3 channels, but lets think more deeply. They could be used &lt;strong&gt;deeper&lt;/strong&gt; in the architecture despite the input image only using 3 channels. So we could have at any point in the depth of the network something like &lt;span class=&quot;math-container&quot;&gt;$(C_F,H,W)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C_F$&lt;/span&gt; is dictated by the number of filters and then apply a 3D convolution with kernal size less than &lt;span class=&quot;math-container&quot;&gt;$C_F$&lt;/span&gt; in the depth dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any point in doing that? When is this helpful? When is it &lt;strong&gt;not&lt;/strong&gt; helpful?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming (though I have no mathematical proof or any empirical evidence) that if the first layer aggregates all input pixels/activations and disregards locality (like a fully connected layer or conv2D that just aggreagates all the depth numbers in the feature space), then 3D convs wouldn't do much cuz earlier layers destroyed the locality structure in that dimension anyway. It sounds plausible but lack any evidence or theory to support it. I know Deep Learning uses empirical evidence to support its claims so perhaps there is something that confirms my intuition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Similar posts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://forums.fast.ai/t/are-there-any-successful-vision-models-that-use-3d-convolutions/52503/2&quot; rel=&quot;noreferrer&quot;&gt;https://forums.fast.ai/t/are-there-any-successful-vision-models-that-use-3d-convolutions/52503/2&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/13692/when-should-i-use-3d-convolution&quot;&gt;When should I use 3D convolution?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/5979/3d-convolution-neural-nets&quot;&gt;3D convolution Neural Nets&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9289" LastActivityDate="2019-08-24T22:56:12.543" Title="Is there any use of using 3D convolutions for traditional images (like cifar10, imagenet)?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13976" PostTypeId="2" ParentId="13968" CreationDate="2019-08-14T20:51:52.780" Score="0" Body="&lt;p&gt;You can set the number of output nodes to the number of all actions, then choose the highest output value, try to do that action, if it can't, move to the next highest output value and so on. The only problem with this is that you have to know how many possible actions there are &lt;/p&gt;&#xA;" OwnerUserId="27789" LastActivityDate="2019-08-14T20:51:52.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13977" PostTypeId="1" CreationDate="2019-08-14T21:10:40.593" Score="-1" ViewCount="78" Body="&lt;p&gt;Alpha Zero is the game playing AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might try to use the algorithm in a robot which takes input from it's environment and thinks about taking the best course of action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was thinking about what is missing from Alpha Zero that would not make this work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing, I think, is that Alpha Zero is pre-programmed with the rules of the game it is playing. Wheres I think a general AI should learn what effect it's actions have. This could be done with a neural network to simulate the next step in a game given various actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I think it needs some sort of attention, since in Chess there are only a finite number of moves, but a human has potentially many more. But a move such as &quot;pick up apple&quot; is not good if there is no apple around. But &quot;pick up thing in my attention&quot; would work better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it would need some sort of RNN module to generate or recieve sentences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it generated say 5 sentences, (related to what it has in it's attention) it could see these as &quot;moves&quot; and decided to either play one of these moves or not. This is somewhat similar to what a human does when they think about what to say in an awkward situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything else?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2019-08-14T21:15:58.980" LastActivityDate="2019-08-15T16:27:23.793" Title="What's missing from Alpha Zero to make it generally intelligent?" Tags="&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13978" PostTypeId="1" CreationDate="2019-08-14T22:30:05.013" Score="8" ViewCount="2251" Body="&lt;p&gt;I'm new to NN. I am trying to understand some of its foundations. One question that I have is: &lt;em&gt;why the &lt;strong&gt;derivative&lt;/strong&gt; of an activation function is important (not the function itself), and why it's the derivative which is tied to how the network performs learning&lt;/em&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, when we say a constant derivative isn't good for learning, what is the intuition behind that? Is the activation function somehow like a &lt;strong&gt;hash function&lt;/strong&gt; that needs to well differentiate small variance in inputs?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="2444" LastEditDate="2019-08-15T00:06:04.720" LastActivityDate="2019-08-24T05:08:08.457" Title="Why is the derivative of the activation functions in neural networks important?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;activation-function&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13979" PostTypeId="2" ParentId="13978" CreationDate="2019-08-15T00:39:17.153" Score="6" Body="&lt;p&gt;If what you are asking is what is the &lt;em&gt;intuition&lt;/em&gt; for using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Derivative&quot; rel=&quot;nofollow noreferrer&quot;&gt;derivative&lt;/a&gt; in backpropagation learning, instead of an in-depth mathematical explanation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that the derivative tells you a function's sensitivity to change with respect to a change in its input. A high (absolute) value for the derivative at a certain point means that the function is very &lt;em&gt;steep&lt;/em&gt;, and a small change in input may result in a drastic change in its output; conversely, a low absolute value means little change, so not &lt;em&gt;steep&lt;/em&gt; at all, with the extreme case that the function is constant when the derivative is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training a neural network essentially amounts to an optimization problem where one wants to minimize a certain value, in this case the error produced by the network on the given training examples. Backpropagation learning can be viewed as a case of &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent&lt;/a&gt; (the inverse of hill climbing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for a moment we assume that your input is only 2-dimensional (just for illustration, the mathematics of course also work for higher dimensions), you could imagine the error function as a landscape with hills, mountains, valleys, ridges etc. You are standing at a high point and want to get down as far as possible. Gradient descent means that, in discrete steps, you always walk down in the direction that has the &lt;em&gt;steepest&lt;/em&gt; slope downwards from where you are currently standing, until you eventually reach a (local) minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to determine where that steepest slope is, you need the derivative of the activation function. Basically, you want to sort out how much each unit in your network contributes to an error, and adjust in the direction that contributes the most.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Regarding constant values for a derivative, in the landscape metaphor it would mean that the gradient is the same no matter where you are, so you'll always go in the same direction and never reach an optimum. However, multi-layer networks with linear activation function are kind of besides the point anyhow when you consider that each cell computes a linear combination of its inputs, which then is again a linear function, so the output of the last layer will ultimately be a linear function of the inputs at the first layer. That is to say, anything you can do with a multi-layer net with linear activation functions, you could also achieve with just a single layer. &lt;/p&gt;&#xA;" OwnerUserId="27624" LastEditorUserId="27624" LastEditDate="2019-08-15T01:01:59.207" LastActivityDate="2019-08-15T01:01:59.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13980" PostTypeId="2" ParentId="13978" CreationDate="2019-08-15T00:43:38.783" Score="4" Body="&lt;p&gt;The basic (and usual) algorithm used to update the weights of the artificial neural network (ANN) is an iterative, &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimization algorithm&lt;/a&gt;, called &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;gradient descent&lt;/em&gt;&lt;/a&gt;, which is based on and requires the computation of the derivative of the function you want to find the minimum of. If the function you want to find the minimum of is &lt;a href=&quot;https://en.wikipedia.org/wiki/Multivariable_calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;multivariable&lt;/a&gt;, then, rather than the derivative, gradient descent requires the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient&lt;/a&gt;, which is a vector where the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th element contains the &lt;em&gt;partial&lt;/em&gt; derivative of the function with respect to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th variable. Hence the name &lt;em&gt;gradient descent&lt;/em&gt;, where the derivative of a function of one variable can be considered the gradient of the function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of ANNs, we usually have a loss function that we want to minimize: for example, the mean squared error (MSE). Therefore, in order to apply gradient descent to find the minimum of the MSE, we need to find the derivative or, more precisely, the gradient of the MSE. To do it, the back-propagation (an algorithm based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;chain rule&lt;/a&gt;) is often used, given that the MSE is a function of the ANN, which is a composite function of multiple non-linear functions, the activation functions, whose main purpose is thus to introduce non-linearity, or, in other words, it makes the ANN powerful. Given that the MSE is a function of the parameters of the ANN, then we need to find the partial derivative of the MSE with respect to all parameters of the ANN. In this process, we will also need to find the derivatives of the activation functions that each neuron applies to its linear combination of weights: &lt;em&gt;to fully see this, you will need to learn the details of back-propagation!&lt;/em&gt; Hence the importance of the derivatives of the activation functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A constant derivative would always give the same &lt;em&gt;learning signal&lt;/em&gt;, independently of the error, but this is not desirable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;To fully understand all these statements, I recommend you learn about back-propagation and gradient descent in detail, which requires a little bit of effort!&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-15T01:29:03.253" LastActivityDate="2019-08-15T01:29:03.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13981" PostTypeId="2" ParentId="12444" CreationDate="2019-08-15T00:52:05.060" Score="2" Body="&lt;p&gt;If you are just starting out with Deep Learning, then a laptop with &lt;strong&gt;GTX 1060&lt;/strong&gt; is enough. I am using a GTX 1060 myself and I find it adequate for many of my personal projects, training large datasets and participating in most (&lt;strong&gt;not all&lt;/strong&gt;) Kaggle competitions as well. But you say that you want to do research work as well. In that case, you may want to contact someone who has worked in that field. What kind of hardware are they using, the efficiency, training time, and all such questions. You should have a pretty decent idea after that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the meantime, keep searching for &lt;strong&gt;RTX&lt;/strong&gt; laptops in \$1200 range. I think that a few laptops have popped up in the market that is giving RTX GPUs for \$1200. &lt;/p&gt;&#xA;" OwnerUserId="27960" LastActivityDate="2019-08-15T00:52:05.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13982" PostTypeId="1" AcceptedAnswerId="13985" CreationDate="2019-08-15T01:55:54.943" Score="2" ViewCount="79" Body="&lt;p&gt;Reading this blog post about AlphaZero:&#xA;&lt;a href=&quot;https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses language such as &quot;the amount of training the network needs&quot; and &quot;fully trained&quot; to describe how long they had the machine play against itself before they stopped training. They state training times such as 9 hours, 12 hours, and thirteen days for chess, shogi, and Go respectively. Why is there a point at which the training &quot;completes?&quot; They show plots of AlphaZero's performance on the Y axis (its Elo rating) as a function of the number of training steps. Indeed, the performance seems to level out as the number of training steps increases beyond a certain point. Here's a picture from that site of the chess performance vs training steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aY0sx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aY0sx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice how sharply the Elo rating levels off as a function of training steps.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First: am I interpreting this correctly? That is, is there an asymptotic limit to improvement on performance as training sessions tend to infinity? &lt;/li&gt;&#xA;&lt;li&gt;If I am interpreting this correctly, why is there a limit? Wouldn't more training mean better refinement and improvement upon its play? It makes sense to me that the millionth training step may yield less improvement than the very first one, but I wouldn't expect an asymptotic limit. That is, maybe it gets to about 3500 Elo points in the first 200k training steps over the course of the first 10 hours or so of playing ches. If it continued running for the rest of the year, I'd expect it to rise significantly above that. Maybe double its Elo rating? Is that intuition wrong? If so, what are the factors that limit its training progress beyond the first 10 hours of play?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="27962" LastActivityDate="2019-08-15T06:39:20.070" Title="What does it mean for AlphaZero's network to be &quot;fully trained&quot;" Tags="&lt;training&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13984" PostTypeId="2" ParentId="13695" CreationDate="2019-08-15T02:39:27.517" Score="1" Body="&lt;p&gt;It seems that you want to detect &lt;em&gt;ranges&lt;/em&gt; of IP addresses that are vulnerable/dangerous/etc, right? Such ranges are essentially numeric intervals, and so my suggestion is to look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision tree learning&lt;/a&gt; instead of neural networks, because you are essentially doing a classification task where you want to test both categorical data and splits over numerical attributes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result will be a tree-like function (nested conditionals) of the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IF ...&amp;gt; address &amp;gt; ... &#xA;   THEN [vulnerable]&#xA;   ELSE IF port=... &#xA;              THEN [not vulnerable]&#xA;              ELSE [vulnerable]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where a huge benefit is that it is also more human-readable than a neural net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most prominent algorithms for decision trees are &lt;a href=&quot;https://en.wikipedia.org/wiki/ID3_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID 3&lt;/a&gt; and its successor &lt;a href=&quot;https://en.wikipedia.org/wiki/C4.5_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;C4.5&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27624" LastActivityDate="2019-08-15T02:39:27.517" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13985" PostTypeId="2" ParentId="13982" CreationDate="2019-08-15T06:39:20.070" Score="3" Body="&lt;p&gt;Neural network will eventually reach limit of it's approximation power. You can't expect to learn more and more things infinitely long with the same amount of learnable parameters. Also, if you eventually reach optimal performance, you can't play more optimal than what optimal is (not saying that it reached optimal performance but possibly something close to optimal for it approximation abilities). So probably combination of those two leads for the performance increase to reach it's limit.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-08-15T06:39:20.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13986" PostTypeId="1" AcceptedAnswerId="13996" CreationDate="2019-08-15T08:28:30.943" Score="4" ViewCount="205" Body="&lt;p&gt;I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2019-08-15T08:39:37.703" LastActivityDate="2020-04-21T17:10:08.853" Title="Why isn't conditional probability sufficient to describe causality?" Tags="&lt;philosophy&gt;&lt;comparison&gt;&lt;conditional-probability&gt;&lt;causation&gt;" AnswerCount="5" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13987" PostTypeId="2" ParentId="13986" CreationDate="2019-08-15T10:27:42.413" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But the conditional probability is clearly not symmetric and captures directed relationships.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One needs to consider the kinds of directed relationships that is captured by conditional probability. It surely does capture &lt;em&gt;some&lt;/em&gt; kind of association or dependence which could be directed. At the same time, it is not right to say that it &lt;em&gt;surely&lt;/em&gt; captures the causal relationships. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sun rises = &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, Rooster crows = &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, then, &lt;span class=&quot;math-container&quot;&gt;$P(A |B)$&lt;/span&gt; is bound to be very high but it does not mean that rooster crowing causes sunrise.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He will ask him to go back to school.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-08-15T11:11:33.963" LastActivityDate="2019-08-15T11:11:33.963" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13988" PostTypeId="1" AcceptedAnswerId="13990" CreationDate="2019-08-15T11:44:55.927" Score="3" ViewCount="70" Body="&lt;p&gt;This is related to &lt;a href=&quot;https://ai.stackexchange.com/questions/13947/understanding-how-continuous-bag-of-words-method-learns-embedded-representations&quot;&gt;my earlier question&lt;/a&gt;, which I'm trying to break down into parts (this being the first). I'm reading notes on word vectors &lt;a href=&quot;https://web.stanford.edu/class/cs224n/readings/cs224n-2019-notes01-wordvecs1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have a vocabulary &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, the naive way to represent words in it would be via one-hot-encoding, or in other words, as basis vectors of &lt;span class=&quot;math-container&quot;&gt;$R^{|V|}$&lt;/span&gt; - say &lt;span class=&quot;math-container&quot;&gt;$e_1, e_2,\ldots,e_{|V|}$&lt;/span&gt;. We want to map these to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;, via some linear transformation such that the images of similar words (more precisely, the images of basis vectors corresponding to similar words) have higher inner products. Assuming the matrix representation of the linear transformation given the standard basis of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{|V|}$&lt;/span&gt; is denoted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt;, then the &quot;embedding&quot; of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th vocab word (i.e. the image of the corresponding basis vector &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) is given by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}e_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose we have a context &quot;The cat ____ over a&quot;, CBoW seeks to find a word that would fit into this context. Let the words &quot;the&quot;, &quot;cat&quot;, &quot;over&quot;, &quot;a&quot; be denoted (in the space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) by &lt;span class=&quot;math-container&quot;&gt;$x_{i_1},x_{i_2},x_{i_3},x_{i_4}$&lt;/span&gt; respectively. We take the image of their linear combination (in particular, their average):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat v=\mathcal{V}\bigg(\frac{x_{i_1}+x_{i_2}+x_{i_3}+x_{i_4}}{4}\bigg)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then map &lt;span class=&quot;math-container&quot;&gt;$\hat v$&lt;/span&gt; back from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{|V|}$&lt;/span&gt; via &lt;em&gt;another&lt;/em&gt; linear mapping whose matrix representation is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$$z=\mathcal{U}\hat v$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we turn this &lt;em&gt;score vector&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; into softmax probabilities &lt;span class=&quot;math-container&quot;&gt;$\hat y=softmax(z)$&lt;/span&gt; and compare it to the basis vector corresponding to the actual word, say &lt;span class=&quot;math-container&quot;&gt;$e_c$&lt;/span&gt;. For example, &lt;span class=&quot;math-container&quot;&gt;$e_c$&lt;/span&gt; could be the basis vector corresponding to &quot;jumped&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my interpretation of what this procedure is trying to do: given a context, we're trying to learn maps &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; such that given a context like &quot;the cat ____ over a&quot;, the model should give a high score to words like &quot;jumped&quot; or &quot;leaped&quot;, etc. Not just that - but &quot;similar&quot; contexts should also give rise to high scores for &quot;jumped&quot;, &quot;leaped&quot;, etc. For example, given a context &quot;that dog ____ above this&quot; wherein &quot;that&quot;, &quot;dog&quot;, &quot;above&quot;, &quot;this&quot; are represented by &lt;span class=&quot;math-container&quot;&gt;$x_{j_1},x_{j_2},x_{j_3},x_{j_4}$&lt;/span&gt;, let the image of their average be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat w=\mathcal{V}\bigg(\frac{x_{j_1}+x_{j_2}+x_{j_3}+x_{j_4}}{4}\bigg)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gets mapped to a score vector &lt;span class=&quot;math-container&quot;&gt;$z'=\mathcal{U}\hat w$&lt;/span&gt;. Ideally, both score vectors &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z'$&lt;/span&gt; should have similarly high magnitudes in their components corresponding to similar words &quot;jumped&quot; and &quot;leaped&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my above understanding correct? Consider the following quote from the lectures:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We create two matrices, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V} \in \mathbb{R}^{n\times |V|}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U} \in \mathbb{R}^{|V|\times n}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is an arbitrary size which defines the size of our embedding space. &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; is the input word matrix such that the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th column of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional embedded vector for word &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; when it is an input to this model. We denote this &lt;span class=&quot;math-container&quot;&gt;$n\times 1$&lt;/span&gt; vector as &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt;. Similarly, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; is the output word matrix. The &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;-th row of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional embedded vector for word &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt; when it is an output of the model. We denote this row of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$u_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It's not obvious to me why &lt;span class=&quot;math-container&quot;&gt;$v_i=\mathcal{V}e_i$&lt;/span&gt; should be the same as or even similar to &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt;. How does the whole backpropagation procedure above ensure that?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Also, how does the procedure ensure that basis vectors corresponding to similar words &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_j$&lt;/span&gt; are mapped to vectors in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; that have high inner product?&lt;/strong&gt; (In other words, how is it ensured that if words no. &lt;span class=&quot;math-container&quot;&gt;$i_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$i_2$&lt;/span&gt; are similar, then &lt;span class=&quot;math-container&quot;&gt;$\langle v_{i_1}, v_{i_2}\rangle$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\langle u_{i_1}, u_{i_2}\rangle$&lt;/span&gt; have high values?)&lt;/p&gt;&#xA;" OwnerUserId="27548" LastActivityDate="2019-08-16T13:48:25.033" Title="How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;word2vec&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13989" PostTypeId="2" ParentId="7394" CreationDate="2019-08-15T12:06:21.860" Score="1" Body="&lt;p&gt;In case this is still relevant to you I can share my tutorial on SVM with Python implementation in Jupyter notebook:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/jjepsuomi/Primer-to-support-vector-machines/blob/master/Primer_to_SVM.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer to support vector machines&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tutorial assumes some mathematics and programming background knowledge. The SVM codes utilize no external machine learning packages and tries to teach the reader to build a SVM model him-/herself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope it helps you!&lt;/p&gt;&#xA;" OwnerUserId="27971" LastActivityDate="2019-08-15T12:06:21.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13990" PostTypeId="2" ParentId="13988" CreationDate="2019-08-15T14:36:11.270" Score="2" Body="&lt;p&gt;Unlike in skip-gram, the reason similar words have similar embeddings in CBOW is because the words show up in the same contexts of other skipped words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;lets assume two words &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_j$&lt;/span&gt; pop up in the exact same context of some word &lt;span class=&quot;math-container&quot;&gt;$e_k$&lt;/span&gt; with 3 other context words as well. An example would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;He &lt;em&gt;leaped&lt;/em&gt; &lt;strong&gt;over&lt;/strong&gt; the truck&lt;/li&gt;&#xA;&lt;li&gt;He &lt;em&gt;jumped&lt;/em&gt; &lt;strong&gt;over&lt;/strong&gt; the truck&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Where the italics represent the words with similar meanings but the bolded words above are the ones being predicted/skipped in CBOW. Lets now show this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the rest of the context be denoted &lt;span class=&quot;math-container&quot;&gt;$\{e_r\}_r$&lt;/span&gt; and the skipped word as &lt;span class=&quot;math-container&quot;&gt;$e_s$&lt;/span&gt; and so the loss will try to minimize both &lt;span class=&quot;math-container&quot;&gt;$-e_s^T log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_i + \sum_re_r)]))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-e_s^T log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_j + \sum_re_r)]))$&lt;/span&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming a long run withlarge enough batch-size (so ignoring catastrophic forgetting) we can essentially say it will be minimizing&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$-e_s^T [log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_i + \sum_re_r)])) +\\ log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_j + \sum_re_r)]))]$$&lt;/span&gt;&lt;br&gt;&#xA;Now let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}\mathcal{V}\frac{1}{R+1}\sum_re_r$&lt;/span&gt; be denoted as &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}\mathcal{V}\frac{1}{R+1}e_i$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\hat u_i$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}\mathcal{V}\frac{1}{R+1}e_j$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\hat u_j$&lt;/span&gt;. So we have to minimize&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$-e_s^T[log(softmax(\hat u_i+c)) + log(softmax(\hat u_j+c))]$$&lt;/span&gt;&lt;br&gt;&#xA;or equivalently&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$-e_s^T\ log(softmax(\hat u_i+c) * softmax(\hat u_j+c))$$&lt;/span&gt;&lt;br&gt;&#xA;As we know in crossentropy, we get our critical point when the inside of the log here would equal &lt;span class=&quot;math-container&quot;&gt;$e_s$&lt;/span&gt;, which is only possible if both &lt;span class=&quot;math-container&quot;&gt;$softmax(\hat u_i+c)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$softmax(\hat u_j+c)$&lt;/span&gt; were to be equal to &lt;span class=&quot;math-container&quot;&gt;$e_s$&lt;/span&gt;. softmax is non-invertible and is invariant under addition only, therefore to achieve this we would need &lt;span class=&quot;math-container&quot;&gt;$\hat u_i + c$&lt;/span&gt; to equal &lt;span class=&quot;math-container&quot;&gt;$\hat u_j + c + K$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is just some arbitrary constant vector (&lt;span class=&quot;math-container&quot;&gt;$K=[k,k,k,...]$&lt;/span&gt;. Subtracting &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and multiplying by &lt;span class=&quot;math-container&quot;&gt;$R+1$&lt;/span&gt; we make our model want  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;u_i &amp;amp;= u_j + K \\&#xA;\implies K &amp;amp;= \mathcal{U}\mathcal{V}(e_i -e_j) \\&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would mean for all common words, we see this activity that &lt;span class=&quot;math-container&quot;&gt;$u_{i-j}$&lt;/span&gt; would be a constant, but this is difficult because that means a &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; must exist such that &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}\delta = k*\vec 1$&lt;/span&gt; and that all similar word pairs indexed by &lt;span class=&quot;math-container&quot;&gt;$(a,b)$&lt;/span&gt; would have &lt;span class=&quot;math-container&quot;&gt;$v_a \approx v_b + const*\delta$&lt;/span&gt; (because &lt;span class=&quot;math-container&quot;&gt;$u_a = u_b + K \implies v_b = v_a + \mathcal{U}^{-1}K$&lt;/span&gt; and we denote &lt;span class=&quot;math-container&quot;&gt;$\delta = \mathcal{U}^{-1}K$&lt;/span&gt;). This constraint is heavy, tough to learn, and would also be difficult to process on multiple word senses that would appear in the vocabulary. This is because if it were to learn this, it enforces constraints on the images of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; along with creating word vectors with highly different magnitudes making the learning process difficult. This would indicate that this constant would be low in practice. Therefore we would get &lt;span class=&quot;math-container&quot;&gt;$v_i \approx v_j$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2019-08-16T13:48:25.033" LastActivityDate="2019-08-16T13:48:25.033" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13991" PostTypeId="2" ParentId="4786" CreationDate="2019-08-15T14:36:43.547" Score="0" Body="&lt;p&gt;The reason the adjusted fitness prevents species from growing to big is do to the fact that the summation that determines the divisor in the adjusted fitness function reduces to the number of genomes in the species f&lt;sub&gt;i&lt;/sub&gt; belongs to, so as species grow the adjusted fitness for every genome belonging to it is divided by a larger number and thus receives a lower adjusted fitness value which in turn will also reduce the summed adjusted fitness of the species. This smaller adjusted fitness then affects the number of offspring the species will get to create after elitism reduces the population.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-11-24T21:30:05.407" LastActivityDate="2019-11-24T21:30:05.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13992" PostTypeId="2" ParentId="13977" CreationDate="2019-08-15T16:27:23.793" Score="-1" Body="&lt;p&gt;One reason Alpha Zero doesn't have human-level intelligence is the lack of computing power in modern day computers when compared to the human brain. The greater level of computing power allows the brain to perform more complex calculations. Less computing power translates to fewer calculations which in turn translates to less conditional probabilities (which is effectively what a neural network calculates) which means less intelligent behaviour (as the neural network can handle fewer samples). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possible reason is that the algorithms in the brain are superior to those of modern neural networks. For instance, humans can easily identify objects based only one or two previous encounters while neural networks have to be trained with huge volumes of data before they work.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human behaviour might also be non-algorithmic, as Roger Penrose argues. The example he gives (in &quot;The Emperor's New Mind&quot;) is that of a computer trying to solve Turing's halting problem. The computer can be in one of two states – the first one in which the Turing machine halts and the second one in which the Turing machine does not halt. The system is deterministic (every event is caused by a preceding one) but cannot be algorithmically predicted. Penrose argues that the human brain works a little like this, through a quantum wave collapse. This stands in contrast to beliefs that the behaviour of the brain is fundamentally algorithmic; if this was the case more and more computing power as well as better algorithms would eventually lead to an AI on par with humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;He also believes that it is impossible to describe the human brain given the state of today's physics – a theory of quantum gravity (describing gravity at small scales and the gravity of elementary particles) would be needed to have a shot at doing so. &lt;/p&gt;&#xA;" OwnerUserId="27933" LastActivityDate="2019-08-15T16:27:23.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13993" PostTypeId="1" CreationDate="2019-08-15T18:00:18.730" Score="2" ViewCount="82" Body="&lt;p&gt;Is this true?  Are we planning to switching such reasoning methods regarding AI tech in the future?&lt;/p&gt;&#xA;" OwnerUserId="27979" LastActivityDate="2021-01-09T10:03:50.367" Title="Is AI and Big Data science recommending a shift in the scientific method from inductive to deductive reasoning?" Tags="&lt;philosophy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13994" PostTypeId="1" AcceptedAnswerId="14014" CreationDate="2019-08-15T21:45:42.813" Score="4" ViewCount="76" Body="&lt;p&gt;I'm trying to score video scenes in terms of aesthetics and cinematography features. Basically, how &quot;interesting&quot; a scene or video frame can be for a viewer. Simpler, how attractive a scene is. My final goal is to tag intervals of video which can be more interesting to viewers. It can be a &quot;temporal attention&quot; model as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we have an available model or prototype to score cinematographic features of an image or a video? I need a starter tutorial on that. Basically, a ready-to-use prototype/model that I can test as opposed to a paper that I need to implement myself. Paper is fine as long as the code is open-source. I'm new and can't yet write a code given a paper.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-08-16T15:55:10.613" LastActivityDate="2019-08-16T17:49:49.497" Title="Aesthetics analysis with deep learning" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-processing&gt;&lt;art-aesthetics&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13995" PostTypeId="2" ParentId="13986" CreationDate="2019-08-15T22:42:45.840" Score="-1" Body="&lt;p&gt;&lt;em&gt;Why isn't conditional probability sufficient to describe causality?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that, when the barometric pressure, in a certain region, drops below a certain level, two things happen&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;the height of the column of mercury in your barometer drops below a certain level&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a storm occurs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We may be tempted to model these relationships with the following graphical model, where each directed edge represents a causal relationship, so, for example, the drop in barometric pressure causes the storm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dmz3j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dmz3j.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this graphical (and causal) model is likely wrong (and unintuitive), given that the drop in barometric pressure is likely &lt;a href=&quot;https://en.wikipedia.org/wiki/Correlation_does_not_imply_causation&quot; rel=&quot;nofollow noreferrer&quot;&gt;only correlated with the storm, so it is not the cause of the storm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How can we see that the drop in barometric pressure is or not the cause of the storm?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can compare the probabilities &lt;span class=&quot;math-container&quot;&gt;$P(A \mid \text{do}(B))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the event &quot;a storm occurs&quot; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is the event &quot;drop in barometric pressure&quot;. &lt;em&gt;What does &lt;span class=&quot;math-container&quot;&gt;$\text{do}(B)$&lt;/span&gt; mean?&lt;/em&gt; It means that we force the event &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to occur, that is, we &lt;strong&gt;force&lt;/strong&gt; the drop in barometric pressure to occur. &lt;em&gt;Intuitively, what is then the difference between &lt;span class=&quot;math-container&quot;&gt;$P(A \mid \text{do}(B))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt;&lt;/em&gt;? In the case of &lt;span class=&quot;math-container&quot;&gt;$P(A \mid \text{do}(B))$&lt;/span&gt;, we force the event &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to always occur. In the case of &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt;, we only and passively look at the cases of event &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; when event &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; occurs (without thus forcing &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to occur). We now know the difference between &lt;span class=&quot;math-container&quot;&gt;$P(A \mid \text{do}(B))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(A \mid B)$&lt;/span&gt;. &lt;em&gt;However, how does this help us to understand that &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is not the cause of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;?&lt;/em&gt; If &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; was a cause of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, then, if we forced &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; always to occur, then the probability of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; should also change accordingly. However, imagine that we are (magically) able to drop the barometric pressure, if the probability of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; does not change accordingly (in this case, if it does not increase), then the storm is not an effect of the drop in barometric pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, Judea Pearl would say that &lt;strong&gt;do&lt;/strong&gt;-operators (or &lt;em&gt;interventions&lt;/em&gt;) are required to analyze causal relationships.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article &lt;a href=&quot;https://plato.stanford.edu/entries/causation-probabilistic/#Inte&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probabilistic Causation&lt;/a&gt; by Stanford Encyclopedia of Philosophy gives a good overview of the (probabilistic) causation (or causality) field. In particular, have a look at section 3, which describes causal modeling (according to Pearl). Causal modeling and inference actually involve several nontrivial concepts that require some time to get familiar with, such as interventions (or do-operations), several basic causal relationships (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Causal_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;forks, chains and colliders&lt;/a&gt;), d-separation or Bayesian networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-15T22:58:00.943" LastActivityDate="2019-08-15T22:58:00.943" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13996" PostTypeId="2" ParentId="13986" CreationDate="2019-08-15T23:50:27.663" Score="5" Body="&lt;p&gt;Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(A|B) = \frac{P(B|A)P(A)}{P(B)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you have &lt;span class=&quot;math-container&quot;&gt;$P(B|A)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(A)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$P(B)$&lt;/span&gt;, we can determine &lt;span class=&quot;math-container&quot;&gt;$P(A|B)$&lt;/span&gt;, and similarly you can determine &lt;span class=&quot;math-container&quot;&gt;$P(B|A)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$P(A|B)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(B)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(A)$&lt;/span&gt;. Just by looking at &lt;span class=&quot;math-container&quot;&gt;$P(B|A)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(A|B)$&lt;/span&gt;, it is therefore impossible to tell what the causal direction is (if any).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, probabilistic inference usually works the other way round: When there is a &lt;em&gt;known&lt;/em&gt; causal relation, say from &lt;em&gt;diseases&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to &lt;em&gt;symptoms&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, we usually have &lt;span class=&quot;math-container&quot;&gt;$P(B|A)$&lt;/span&gt;, and are interested in the diagnostic reasoning task of determining &lt;span class=&quot;math-container&quot;&gt;$P(A|B)$&lt;/span&gt; from that. (The only other thing we need for that is the prior probability &lt;span class=&quot;math-container&quot;&gt;$P(A)$&lt;/span&gt; since &lt;span class=&quot;math-container&quot;&gt;$P(B)$&lt;/span&gt; is just a normalization factor.)&lt;/p&gt;&#xA;" OwnerUserId="27624" LastActivityDate="2019-08-15T23:50:27.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13997" PostTypeId="2" ParentId="7394" CreationDate="2019-08-16T01:41:16.567" Score="1" Body="&lt;p&gt;Support vector machines are supervised learning models with associated learning algorithms that analyze data and are used for classification and regression analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link where you can learn more about it from a introduction level: &lt;a href=&quot;https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Support Vector Machine — Introduction to Machine Learning Algorithms&quot; (Medium)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19325" LastEditorUserId="1671" LastEditDate="2019-08-16T20:25:33.543" LastActivityDate="2019-08-16T20:25:33.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13998" PostTypeId="2" ParentId="13859" CreationDate="2019-08-16T05:40:48.770" Score="1" Body="&lt;p&gt;&lt;sup&gt;Your question is mostly philosophical, not technical or scientific. So I am giving opinions and references here.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the core of AI boils down to design algorithms&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am noticing that you are not even try to define &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI&lt;/a&gt; (whose definition changed since the previous century). You could look at the table of contents of the &lt;a href=&quot;https://www.journals.elsevier.com/artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence&lt;/em&gt; journal&lt;/a&gt; and notice how topics covered there changed drastically in a few decades (even experimental approaches have declined).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be interested in reading more about &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI&lt;/a&gt; and follow the few conferences about it. Beware, there is a lot of too much simplified approaches, and even a lot of bullshit (e.g. on &lt;a href=&quot;https://agi.topicbox.com/groups/agi&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; AGI mailing list, but some messages there are gems)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume you accept the &lt;a href=&quot;https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Church-Turing&lt;/a&gt; philosophical thesis: every intelligent cognition (either natural i.e. biological or artificial) is some symbolic computation. In particular, the work of a mathematician can be abstracted as a Turing machine (that was the major insight of Turing and in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;halting problem&lt;/a&gt;). Be also aware of the related &lt;a href=&quot;https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Curry-Howard&lt;/a&gt; correspondence and &lt;a href=&quot;https://en.wikipedia.org/wiki/Rice%27s_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rice's theorem&lt;/a&gt;. Read &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Gödel, Escher, Bach&lt;/em&gt;&lt;/a&gt; !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't know yet how to make AGI. You could read Bostrom's &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;SuperIntelligence&lt;/em&gt;&lt;/a&gt; book about potential dangers. You could also read J.Pitrat's book &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1717130&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings&lt;/em&gt;&lt;/a&gt; (which gives much more positive and constructive insights about eventually making some AGI) and &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal belief (just an opinion) is that AGI could be perhaps achieved (in many dozens of years), should be definitely get much more funding -and more time- as a research topic (e.g. as much as the &lt;a href=&quot;https://en.wikipedia.org/wiki/ITER&quot; rel=&quot;nofollow noreferrer&quot;&gt;ITER&lt;/a&gt; reactor; see also &lt;a href=&quot;https://www.softwareheritage.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;softwareheritage.org&lt;/a&gt; and the motivations there), but won't be achieved by any single technique, but by a clever combination of many AI techniques (both &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic AI&lt;/a&gt; -e.g. for &lt;a href=&quot;https://en.wikipedia.org/wiki/Planning&quot; rel=&quot;nofollow noreferrer&quot;&gt;planning&lt;/a&gt;- and &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Connectionism&quot; rel=&quot;nofollow noreferrer&quot;&gt;connectionnist&lt;/a&gt; approaches, with inspiration from cognitive psychology).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The conclusion is, in my understanding, that to create AI systems we must accept incompleteness or inconsistency. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We, members of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Human_taxonomy#Subspecies&quot; rel=&quot;nofollow noreferrer&quot;&gt;Homo Sapiens Sapiens&lt;/a&gt; species (in latin, the humans who know that they know, so capable of &lt;a href=&quot;https://en.wikipedia.org/wiki/Metaknowledge&quot; rel=&quot;nofollow noreferrer&quot;&gt;metaknowledge&lt;/a&gt;), claim to be intelligent. But all of us have a globally incomplete and inconsistent behavior, because each of us have contradictions (e.g. in our personal lives or &lt;a href=&quot;https://en.wikipedia.org/wiki/Ethical_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;ethical&lt;/a&gt; beliefs). So, logically speaking, incompleteness or inconsistency is not opposed to intelligence. Read also more about &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence,_situated_approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;situated AI&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine ethics&lt;/a&gt;. BTW, I believe (since educated by J.Pitrat about this) that explicit and &lt;a href=&quot;https://en.wikipedia.org/wiki/Descriptive_knowledge&quot; rel=&quot;nofollow noreferrer&quot;&gt;declarative&lt;/a&gt; &lt;a href=&quot;https://www.coursehero.com/file/ptcioic/CS-484-Artificial-Intelligence-8-Meta-Rules-The-rules-that-determine-the/&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta&lt;/a&gt;knowledge is required in any AGI system.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does that mean that AI will never be employed for real-time critical applications? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Notice that &lt;a href=&quot;https://en.wikipedia.org/wiki/Lethal_autonomous_weapon&quot; rel=&quot;nofollow noreferrer&quot;&gt;autonomous killing robots&lt;/a&gt; are already a controversial research topic today. &lt;a href=&quot;https://en.wikipedia.org/wiki/Autonomous_robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autonomous robots&lt;/a&gt; already exist (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Mars_rover&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mars rovers&lt;/a&gt; cannot be teleoperated -for every elementary movement- from Earth, because any radio signal takes minutes to reach Mars). And autonomous vehicles (à la &lt;a href=&quot;https://en.wikipedia.org/wiki/Waymo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google car&lt;/a&gt;) claim today to use AI techniques and are &lt;a href=&quot;https://en.wikipedia.org/wiki/Real-time_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;real-time&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Safety-critical_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;safety-critical systems&lt;/a&gt;. Today's Airbus or Boeing (cf &lt;a href=&quot;https://en.wikipedia.org/wiki/DO-178C&quot; rel=&quot;nofollow noreferrer&quot;&gt;DO-178C&lt;/a&gt;) are flying automatically most of the time. &lt;a href=&quot;https://en.wikipedia.org/wiki/Cruise_missile&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cruise missiles&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Intercontinental_ballistic_missile&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICBM&lt;/a&gt;s are &lt;a href=&quot;https://en.wikipedia.org/wiki/Fire-and-forget&quot; rel=&quot;nofollow noreferrer&quot;&gt;fire-and-forget&lt;/a&gt; devices. Many &lt;a href=&quot;https://en.wikipedia.org/wiki/High-frequency_trading&quot; rel=&quot;nofollow noreferrer&quot;&gt;high-frequency trading&lt;/a&gt; systems claim to use AI techniques and are real-time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. Notice that what was called AI in the previous century is today called AGI. My PhD in AI was defended in 1990 (and was about explicit metaknowledge for metaprogramming goals, see e.g. &lt;a href=&quot;https://inis.iaea.org/collection/NCLCollectionStore/_Public/19/059/19059867.pdf?r=1&amp;amp;r=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; old 1987 paper)&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2019-08-18T13:56:54.867" LastActivityDate="2019-08-18T13:56:54.867" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13999" PostTypeId="1" AcceptedAnswerId="14002" CreationDate="2019-08-16T05:47:03.300" Score="3" ViewCount="92" Body="&lt;p&gt;In a convolutional neural network, when we apply the convolution on a &lt;span class=&quot;math-container&quot;&gt;$5 \times 5$&lt;/span&gt; image with &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; kernel, with stride &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, we should get only one &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; as output. In most of the CNN tutorials, we are having &lt;span class=&quot;math-container&quot;&gt;$4 \times 4 \times m$&lt;/span&gt; as output. I don't know how we are getting a three-dimensional output and I don't know how we need to calculate &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. How is &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; determined? Why do we get a three-dimensional output after a convolutional layer?&lt;/p&gt;&#xA;" OwnerUserId="27986" LastEditorUserId="2444" LastEditDate="2019-08-16T08:12:10.350" LastActivityDate="2019-08-20T11:34:19.907" Title="Why do we get a three-dimensional output after a convolutional layer?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-processing&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14000" PostTypeId="2" ParentId="2712" CreationDate="2019-08-16T06:42:28.863" Score="0" Body="&lt;p&gt;You could be interested in orthogonally &lt;a href=&quot;https://en.wikipedia.org/wiki/Persistence_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;persistent&lt;/a&gt; systems. You could look at them as  schema-agnostic database systems whose data fits &lt;em&gt;entirely&lt;/em&gt; in RAM (remember also 1980s &lt;a href=&quot;https://en.wikipedia.org/wiki/Smalltalk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smalltalk&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Lisp_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lisp Machines&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt; ones and 1994 &lt;a href=&quot;https://www.usenix.org/legacy/publications/compsystems/1994/sum_dearle.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GrassHopper&lt;/a&gt; OS) or at least in &lt;a href=&quot;https://en.wikipedia.org/wiki/Virtual_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;virtual memory&lt;/a&gt;. With that approach, even &lt;a href=&quot;https://sbcl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBCL&lt;/a&gt; almost fits in your wish, since it has &lt;a href=&quot;http://www.sbcl.org/manual/index.html#Saving-a-Core-Image&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;save-lisp-and-die&lt;/code&gt;&lt;/a&gt;. Look also into &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;frame&lt;/a&gt; based systems and &lt;a href=&quot;https://en.wikipedia.org/wiki/Object_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;object databases&lt;/a&gt;. Read also a good &lt;a href=&quot;http://pages.cs.wisc.edu/~remzi/OSTEP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;operating systems&lt;/em&gt; textbook&lt;/a&gt; and see past discussions archived on &lt;a href=&quot;https://tunes.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tunes.org&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shameful self promotion: My &lt;a href=&quot;https://github.com/bstarynk/bismon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bismon system&lt;/a&gt; (work in progress in summer 2019) &lt;a href=&quot;http://starynkevitch.net/Basile/bismon-chariot-doc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;claims&lt;/a&gt; to be a GPLv3+ orthogonally persistent system applied for static source code analysis of IoT software. But you might reuse most of it for other kind of orthogonal persistence (of &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;frame&lt;/a&gt; based data).&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2019-08-16T06:54:01.370" LastActivityDate="2019-08-16T06:54:01.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14001" PostTypeId="1" AcceptedAnswerId="14004" CreationDate="2019-08-16T08:07:59.417" Score="4" ViewCount="1713" Body="&lt;p&gt;I am a newbie in the fantastic AI world, I have started my learning recently. &#xA;After a while, my understanding is, we need to feed in tremendous data to train a or many models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the training is complete, we could take out the trained models and &quot;plug in&quot; to any other programming languages to use to detect things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. What are the trained models? are they algorithms or a collection of parameters in a file?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. What do they look like? e.g. file extensions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="27988" LastEditorUserId="2444" LastEditDate="2019-08-16T09:18:02.347" LastActivityDate="2019-08-16T17:46:53.493" Title="What is the &quot;thing&quot; which is trained in AI model training" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;models&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14002" PostTypeId="2" ParentId="13999" CreationDate="2019-08-16T08:31:34.373" Score="2" Body="&lt;p&gt;If you have a &lt;span class=&quot;math-container&quot;&gt;$h_i \times w_i \times d_i$&lt;/span&gt; input, where &lt;span class=&quot;math-container&quot;&gt;$h_i, w_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; respectively refer to the height, width and depth of the input, then we usually apply &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$h_k \times w_k \times d_i$&lt;/span&gt; kernels (or filters) to this input (with the appropriate stride and padding), where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is usually a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyper-parameter&lt;/a&gt;. So, after the application of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; kernels, you will obtain &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$h_o \times w_o \times 1$&lt;/span&gt; so-called &lt;em&gt;feature maps&lt;/em&gt; (also known as &lt;em&gt;activation maps&lt;/em&gt;), which are usually concatenated along the depth dimension, hence your output will have a depth of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; (given that the application of a kernel to the input usually produces a two-dimensional output). For this reason, the output is usually referred to as &lt;em&gt;output volume&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of CNNs, the kernels are learned, so they are not constant (at least, during the learning process, but, after training, they usually remain constant, unless you perform continual lifelong learning). Each kernel will be different from any other kernel, so each kernel will be doing a different convolution with the input (with respect to the other kernels), therefore, each kernel will be responsible for filtering (or detecting) a specific and different (with respect to the other kernels) feature of the input, which can, for example, be the initial image or the output of another convolutional layer.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-16T08:38:35.733" LastActivityDate="2019-08-16T08:38:35.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14003" PostTypeId="1" CreationDate="2019-08-16T08:47:11.267" Score="11" ViewCount="3681" Body="&lt;p&gt;I've been reading different papers regarding graph convolution and it seems that they come into two flavors: spatial and spectral. From what I can see the main difference between the two approaches is that for spatial you're directly multiplying the adjacency matrix with the signal where for the spectral version you're using the Laplacian matrix. Am I missing something or are there any other differences that I am not aware of?   &lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2019-08-20T07:46:16.393" LastActivityDate="2019-12-12T23:01:00.410" Title="What is the difference between graph convolution in the spatial vs spectral domain?" Tags="&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;geometric-deep-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14004" PostTypeId="2" ParentId="14001" CreationDate="2019-08-16T09:13:05.350" Score="6" Body="&lt;p&gt;This answer applies to Machine Learning (ML) part of AI, as that seems to be what you are asking about. Please bear in mind that AI is still a broad church, including many other techniques than ML. ML, including neural networks for deep learning, and Reinforcement Learning (RL) is only a subset of AI - some AI techniques are more focused on the algorithm than parameters.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;What are the trained models? are they algorithms or a collection of parameters in a file?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In ML, the usual process is to feed data into parametric function (e.g. a neural network) and alter the parameters of it to &quot;fit&quot; the data. The main output of this is a collection of parameters and hyperparameters that describe the parametric function. So 90% of the time, when discussing the &quot;trained model&quot;, it means the same thing as the collection of paramaters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, those parameters are of limited use without a library that can re-create the function from them. Parameters will be saved from a specific library and can be loaded back into that library easily. It is also possible for libraries to read or convert from models saved from other libraries, much like how different spreadsheet programs can read each others' files.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;What do they look like? e.g. file extensions&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This varies a lot, depending on which library was used. It is not possible to make a general statement. For example, &lt;a href=&quot;https://www.tensorflow.org/guide/saved_model&quot; rel=&quot;noreferrer&quot;&gt;Tensorflow can save variables to a &quot;Checkpoint&quot; file&lt;/a&gt; with &lt;code&gt;.ckpt&lt;/code&gt; extension, but can get more sophisticated depending on how much of the model you want to export, and full models with whole structure will contain more than just the variables and have the &lt;code&gt;.pb&lt;/code&gt; extension.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are a few places where you can find selections of pre-trained models. One such place is &lt;a href=&quot;https://github.com/tensorflow/models&quot; rel=&quot;noreferrer&quot;&gt;Tensorflow's Model Zoo&lt;/a&gt; and you might be interested in &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md&quot; rel=&quot;noreferrer&quot;&gt;Tensorflow detection model zoo&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other frameworks may also provide example code. For instance &lt;a href=&quot;https://caffe.berkeleyvision.org/model_zoo.html&quot; rel=&quot;noreferrer&quot;&gt;Caffe also has a &quot;Model Zoo&quot;&lt;/a&gt; (searching Model Zoo is a good starting strategy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are working at the level of collecting model parameters and want to run these models yourself, you will need to learn a bit about each library, what language is used to work with it, and maybe follow some tutorial about how to use it. A few models will be packaged up with working scripts to use from the command line, but many are not and may take sometime and effort to get working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you have a specific detection target, you may be disappointed to find models that don't quite match what you want. For image classifying, it is common if you have a specialist need to take a pre-existing general model that has been trained on large dataset for weeks, and then &quot;fine tune&quot; it with your own image dataset for your purpose. Most NN libraries will have tutorials and examples of this fine tuning process.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-16T09:20:09.530" LastActivityDate="2019-08-16T09:20:09.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14006" PostTypeId="1" CreationDate="2019-08-16T09:53:46.727" Score="3" ViewCount="47" Body="&lt;p&gt;What contemporary information system or &lt;a href=&quot;http://bicasociety.org/cogarch/architectures.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;cognitive architecture&lt;/a&gt; is the one with the highest measure of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Integrated_information_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integrated Information Theory (IIT)&lt;/a&gt; (that is, a theory of consciousness, which states that a system's consciousness is determined by its causal properties and is, therefore, an intrinsic, fundamental property of any physical system)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there race/competitions to develop (or allow autonomous development) the system with the maximum IIT measure?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-08-17T13:59:03.357" LastActivityDate="2019-08-17T13:59:03.357" Title="What is the cognitive architecture with the highest IIT measure?" Tags="&lt;expert-system&gt;&lt;cognitive-architecture&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14007" PostTypeId="1" CreationDate="2019-08-16T11:06:07.877" Score="2" ViewCount="68" Body="&lt;p&gt;I am currently building a chatbot. What I have done so far is, collected possible questions/training data/files and created a model out of it using Apache OpenNLP; the model is able to predict all the questions that are in the training data and fails to predict for new questions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of doing all the above, I can write a program that matches the question/words against training data and predict the answer &amp;mdash; what is the advantage of using Machine Learning algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have searched extensively about this and all I got was, in Machine Learning there is no need to change the algorithm and the only change would be in the training data, but that is the case with programming too: the change will be in training data.&lt;/p&gt;&#xA;" OwnerUserId="27992" LastEditorUserId="2193" LastEditDate="2019-08-18T12:21:08.667" LastActivityDate="2019-08-18T12:44:55.353" Title="Chatbot using Machine Learning vs Programming" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14008" PostTypeId="2" ParentId="14007" CreationDate="2019-08-16T11:16:37.187" Score="1" Body="&lt;p&gt;In my view ML does not work very well for conversational AI systems. It is generally alright for intent recognition, so getting what the user wants if they ask a question (&quot;I want to book a flight?&quot;, &quot;What is the weather in London?&quot;), but anything after that quickly becomes difficult to handle, especially multi-step conversations that go beyond simple question/answer pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion would be to plan possible dialogues out as flow charts (more like trees/graphs, as there can be multiple branches at any point), and then write a program that interprets the graph based on user input and gives appropriate replies. You will also want to have some conversational memory to keep track of any information the user has mentioned. That is also tricky to do in a ML system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a very simple framework to start off with, have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELIZA&lt;/a&gt;. It's half a century old, but you can still use it as a starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I work for a company that makes conversational AI systems)&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-08-16T11:16:37.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14009" PostTypeId="1" CreationDate="2019-08-16T11:30:53.543" Score="2" ViewCount="64" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1709.01507.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Squezee-and-excite networks&lt;/a&gt; introduced SE blocks, while &lt;a href=&quot;https://arxiv.org/pdf/1801.04381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MobileNet v2&lt;/a&gt; introduced linear bottlenecks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the effective difference between these two concepts?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it only implementation (depth-wise convolution, vs per-channel pooling), or they serve a different purpose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that both approaches are used as attention mechanism, working per-channel. In other words, both approaches are used to filter unnecessary information (information that we consider noise, not signal).&#xA;Is this correct?&#xA;Do bottlenecks ensure, that the same feature won't be represented multiple times in different channels, or they don't help at all in this regard?&lt;/p&gt;&#xA;" OwnerUserId="27994" LastEditorUserId="2444" LastEditDate="2019-08-17T15:57:49.097" LastActivityDate="2019-08-17T15:57:49.097" Title="What is the difference between Squeeze-and-excite and bottleneck modules from Mobilenet v2?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;papers&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14010" PostTypeId="2" ParentId="12444" CreationDate="2019-08-16T11:53:50.920" Score="4" Body="&lt;p&gt;I think you should redirect your focus.&#xA;Learn and play with ML, and only when compute becomes the main bottleneck for your learning, invest in hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've recently engaged into a ML project for which I 've assembled a machine with gtx1080, installed gui-less ubuntu, configured ssh, drivers etc, and than spend 3 months on data collection. And the dataset was so small (30k images), that I could probably learn mobilenet v3 on my CPU instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that in most cases focusing on hardware is a form of procrastination when it comes to machine learning.&#xA;Also, a laptop with really strong GPU makes little sense (unless it's really well cooled like Alienware r3 that I use at work, but it's to heavy to be really considered a laptop...). It will throttle, It will be less durable due to constant overheating when you learn models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'd say, go for gtx1060 laptop and only when you really need compute (you think you have a chance to actually win a competition due to a great idea you have, or you have a client that has seen a demo and really want's to pay for your model if you improve it), invest into standalone PC or cloud to tune your prototype model. &lt;/p&gt;&#xA;" OwnerUserId="27994" LastActivityDate="2019-08-16T11:53:50.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14011" PostTypeId="2" ParentId="13986" CreationDate="2019-08-16T13:40:24.607" Score="0" Body="&lt;p&gt;Conditional probability does not describe causality at all, since description is a layer of detail about some state, action, scenario, or phenomenon. Causality is a level of abstraction above description. Conditional probability does not imply causality either, because implication is also an abstraction, above causality and description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a boxing match, a punch and a person knocked out may occur in succession indicating a high probability that the punch caused the knock out. It may be reasonable to say that the repetition of the trend of knock outs following punches supports this probability of causality, supporting the correlation between conditional probability and causality, but the correlation coefficient is certainly &amp;lt; 1.0 for the general case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most that would be studying conditional probability already realize that the two observations (punch, knock out) represent a minuscule subset of all observations that could be made with the right instrumentation. The person may have fainted from dehydration immediately after being punched and the temporal alignment of the two events leads to an incorrect conclusion about cause among the observers. On a higher level of abstraction, the person may only appear knocked out but is acting in a boxing sequence on a realistically choreographed movie shoot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the appearance of impact precedes the appearance of unconsciousness, we can correctly make probabilistic statements about causality within an abstract system we constructed about the class of phenomena about which the observations are a part. The announcer might say, &quot;And a round house from Thompson brings Pauly swirling to the floor.&quot; The fight is judged on the basis of conclusions about causality that stem from observing multiple fights. Media channels are filled with such conclusions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human beings are swimming in a pool of unproven statements presented as facts, so most humans are unaware of the basis upon which they draw conclusions. Diligent and well educated scientists are more likely to remember these concepts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Only in virtual scenarios brought to life in human constructions such as chess games and digital circuits can observation be exhaustive.&lt;/li&gt;&#xA;&lt;li&gt;Purely unidirectional causality is rare, such as the moonlight on a camera lens. The sun is unaffected by the reflection of light off of the camera lens but the exposure of the image is the detection of solar nuclear reaction. Within the biosphere, physical phenomena more commonly exhibit bidirectional information flow. A acts on B while B acts on A.&lt;/li&gt;&#xA;&lt;li&gt;Most descriptions of an event are of a trend found in a cacophony of tiny interactions, such as a punch and knock out being a large number of electrical events in a set of neurons, the elastic deformation of a large number of molecules within cells and bone material, and a large number of electrical events in a second set of neurons.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Human cognitive abilities embrace all of these abstractions of composition and causality, and many thoughtful humans attach some level of doubt about causality when observations appear to indicate it. We can hear reports like, &quot;I think the shot came from behind me and I saw the President's head jerk back.&quot; The causality can nonetheless be debated for decades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We see a panic and crash in a market, but there is no singular panic and singular crash. No person's trade, public message, personal financial concern, or verbal expression causes the entire trend we examine in a graph part way through or after the sequence of singular events marking such a scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In electronics, we take great pains to permit circuit A's control of circuit B without allowing the back flow of causality from B to A. We like digital electronics because we get a purified universe where causality is unidirectional except where we specify feedback, as in the case of digital learning that depends on successive approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these things are part of the framework of concepts that decouple conditional probability from conclusive causality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A statement concluding that we don't have causality is not particularly compelling as a scientific statement. Causality is not something we can have or not have universally, and what one thinks they have another may think they don't, so the first person plural is presumptuous. A statement concluding that physical equations are symmetric can be correct if the symmetry to which the author is referring is regarding bidirectional impact, not reversibility, and equations are restricted to equalities. However, such a statement is not generally true either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be reasonable to say any of these statements.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Any consistent temporal pattern of asymmetry gives us information about causality but cannot be conclusive.&lt;/li&gt;&#xA;&lt;li&gt;Causal interaction may be bidirectional without being balanced in magnitude such that one side may seem to cause the other when what appears as cause and effect both occur as part of a single unified phenomenon.&lt;/li&gt;&#xA;&lt;li&gt;Alleged causality written into historic accounts can be unreliable.&lt;/li&gt;&#xA;&lt;li&gt;At a quantum level, causality is not strictly applicable.&lt;/li&gt;&#xA;&lt;li&gt;The concept of causality is an abstraction that can be misleading if applied in scenarios that have bidirectional imposition of forces or fields or bidirectional information exchange.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The concluding question is interesting, but the approaches of educators and media figures in answering such questions are usually context dependent.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is easy to imagine a defensive retort, a response indicating thoughtful consideration of the proposal, or (in the middle ground) a pedantic one. A private conversation over a coffee or some sushi may result in the more thoughtful conversation.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-16T13:40:24.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14012" PostTypeId="1" CreationDate="2019-08-16T16:21:48.760" Score="2" ViewCount="35" Body="&lt;p&gt;Given a robot in a situation such as in a library reading a book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to create a neural network that suggests an appropriate action in this situation. And, generally, ignore actions such as &quot;get up and dance&quot; and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there are limitless actions a robot could do, I need to narrow it down to the ones in this situation. Using its vision system, the word &quot;book&quot; and book neurons should already be activated as well as &quot;reading&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea I had was to create an adversarial network which generates words (sequences of letters) based on the situation such as &quot;turn page&quot;, &quot;read next line&quot; and so on. And then have another neural network which translates these words into actions. (It would them simulate whether this was a good idea. If not it would somehow suppress the first word and try to generate a new word.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is the robot is in a maze and gets to a crossroads. The network would generate the word &quot;turn left&quot; and &quot;turn right&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea would be to have the actions be composed of a body part e.g. &quot;eyes&quot; and a movement such as &quot;move left&quot; and it would combine these to suggest actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, it seems like I need a way to encode actions so that the robot doesn't consider every possible action in the universe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any research in this area or ideas on how to achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I think this may be somewhat related to the task of &quot;try to name as many animals as you can.&quot;)&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2019-08-17T13:52:10.997" LastActivityDate="2019-09-04T01:41:53.750" Title="How to select the most appropriate set of actions for a given environment or task?" Tags="&lt;ai-design&gt;&lt;action-model-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14013" PostTypeId="1" CreationDate="2019-08-16T16:51:46.013" Score="5" ViewCount="230" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;It (Adagrad) adapts the learning rate to the parameters, performing smaller updates&#xA;  (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. high learning rates) for parameters associated with infrequent features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/index.html#adagrad&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sebastian Ruder's Blog&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a parameter is associated with an infrequent feature then yes, it is more important to focus on properly adjusting that parameter since it is more decisive in classification problems. But how does making the learning rate higher in this situation help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it only changes the size of the movement in the dimension of the parameter (makes it larger) wouldn't that make things even more imprecise? Since the network depends more on those infrequent features, shouldn't adjusting those parameters be done more precisely instead of just faster? The more decisive parameters should have a higher &quot;slope&quot;, thus why should they also have high learning rates? I must be missing something, but what is it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, in the article, the formula for parameter adjustments with Adagrad is given. Where exactly in that formula do you find the information about the frequency of a parameter? There must be a relationship between the gradients of a parameter and the frequency of features associated with it because it's the gradients that play an important role in the formula. What is that relationship?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TLDR:&lt;/strong&gt; I don't understand both the purpose and formula behind Adagrad. What is an intuitive explanation of it that also provides an answer to the questions above, or shows why they are irrelevant?&lt;/p&gt;&#xA;" OwnerUserId="21788" LastEditorUserId="16565" LastEditDate="2019-08-17T08:04:26.033" LastActivityDate="2021-05-11T19:00:20.093" Title="An intuitive explanation of Adagrad, its purpose and its formula" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14014" PostTypeId="2" ParentId="13994" CreationDate="2019-08-16T17:35:34.240" Score="3" Body="&lt;p&gt;Aesthetics of images has a strong subjective element and possibility of multiple dimensions depending on purpose of the media. That means:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is hard to define what we mean by scoring aesthetics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Given any well-constrained definition, it is then time-consuming to collect relevant data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, there is some interest in the machine-learning community, as media quality would be a very useful metric to sort and filter data on (provided the metric is close enough to the end user who wants to select it). As a result, there are data sets, research papers and pre-built models for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Media quality training data can be crowdsourced in a variety of ways, including looking at popularity of items on social media, to paying experts to assess large numbers of images. An example of one open dataset compiled by researchers for this purpose is &lt;a href=&quot;http://refbase.cvc.uab.es/files/MMP2012a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;called AVA&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This data might be reduced to image/quality pairs which you can then train a CNN model to predict the quality metric (score out of 10 for example). This might just be a regression, but other more complex loss functions are also considered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick search for existing models brings up Google's &lt;a href=&quot;https://ai.googleblog.com/2017/12/introducing-nima-neural-image-assessment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIMA project&lt;/a&gt;, which has more than one implementation available as open-source code. NIMA appears to use multiclass classification approach to predict which ratings humans would most likely give the image, and the resulting score is then a weighted average of the predicted scores - the claimed benefit of that seems to be that it better matches how the quality ratings are sourced, and it will better capture split opinions (e.g. half of people think the image is terrible, but half think it is great is a different type of image to one where everyone thinks it is just average). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/idealo/image-quality-assessment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is an implementation of NIMA by Github account &quot;idealo&quot; looks complete with documentation, and ready to use with pre-built scripts&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to show this is not a one-off, &lt;a href=&quot;http://karpathy.github.io/2015/10/25/selfie/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's a blog by Andrej Karpathy about using CNNs to rate selfies&lt;/a&gt; which includes some introduction to core CNN concepts. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-16T17:49:49.497" LastActivityDate="2019-08-16T17:49:49.497" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14015" PostTypeId="2" ParentId="14001" CreationDate="2019-08-16T17:46:53.493" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;What are the trained models? are they algorithms or a collection of parameters in a file?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&quot;Model&quot; could refer to the algorithm with or without a set of trained parameters.&#xA;If you specify &quot;trained model&quot;, the focus is on the parameters, but the algorithm is implicitly part of that, since without the algorithm, the parameters are just an arbitrary set of numbers.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;What do they look like? e.g. file extensions&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That very much depends on both the algorithm you're using &lt;em&gt;and&lt;/em&gt; the specific implementation. A few simple examples might help clarify matters.  Let's suppose that the problem we're trying to learn is the exclusive or (XOR) function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a | b | a XOR b&#xA;--+---+---------&#xA;0 | 0 | 0&#xA;0 | 1 | 1&#xA;1 | 0 | 1&#xA;1 | 1 | 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;First, let's use a 2-layer neural net to learn it.  We'll define our activation function to be a simple step function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ f(x) = \begin{cases}&#xA;1 &amp;amp; \text{if } x &amp;gt; 0.5 \\&#xA;0 &amp;amp; \text{if } x \le 0.5&#xA;\end{cases} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is actually a terrible activation function for real neural nets since it's non-differentiable, but it makes the example clearer.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our model is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_0 = f(1\cdot a+1\cdot b + 0)\\&#xA; h_1 = f(0.5\cdot a + 0.5\cdot b + 0)\\&#xA; \,\;y = f(1\cdot h_0 - 1\cdot h_1 + 0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each step of this essentially draws a hyperplane and evaluates to 1 if the input is on one side of the hyperplane and 0 otherwise.  In this particular case, h_0 tells us if either a or b is true. h_1 tells us if they're both true, and y tells us if exactly one of them is true, which is the exact definition of the XOR function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our parameters are the coefficients and biases (the offset added at the end of each expression):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \begin{bmatrix}&#xA;1 &amp;amp; 1 &amp;amp; 0 \\&#xA;0.5 &amp;amp; 0.5 &amp;amp; 0 \\&#xA;1 &amp;amp; 1 &amp;amp; 0 \\&#xA;\end{bmatrix}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They can be stored in a file in any way we want; all that matters is that the code that stores them and the code that reads them back agree on the format.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now let's solve the same problem using a decision tree. For these, we traverse a tree, and at every node, ask a question about the input to decide which child to visit next. Ideally, each question will divide the space of possibilities exactly in half. Once we reach a leaf node, we know our answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this diagram, we visit the right child iff the expression is true.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     a+b=2&#xA;    /     \&#xA;  a+b=0    0&#xA; /     \     &#xA;0       1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, the model and parameters are harder to separate. The only part of the model that isn't learned is &quot;It's a tree&quot;.  The expressions in each interior node, the structure of the tree, and the value of the leaf nodes are all learned parameters. As with the weights from the neural network, we can store these in any format we want to.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Both methods are learning the same problem, and actually find basically the same solution: a XOR b = (a OR b) AND NOT (a AND B).  But the nature of the mathematical model we use depends on the method we choose, the parameters depend on what we train it on, the file format depends on the code we use to do it, and the line between model and parameter is fairly arbitrary; the math works out the same regardless of how we split it up.  We could even write a program that tries different methods, and outputs a &lt;em&gt;program&lt;/em&gt; that classifies inputs using the method that performed best. In this case, the model and parameters aren't separate at all.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know of any pretrained models that specifically recognize birds, but I'm not in image-recognition, so that doesn't mean much.  If you're not averse to training your own model (using existing code), I believe the &lt;a href=&quot;http://image-net.org/index&quot; rel=&quot;nofollow noreferrer&quot;&gt;ImageNet&lt;/a&gt; dataset includes birds.  AlexNet and LeNet would probably be good starting points for the model. Most if not all of the the state of the art image recognition models are based on convolutional networks, so you'll need a decent GPU to run them.&lt;/p&gt;&#xA;" OwnerUserId="2212" LastActivityDate="2019-08-16T17:46:53.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14016" PostTypeId="2" ParentId="13725" CreationDate="2019-08-16T18:02:11.177" Score="1" Body="&lt;p&gt;Previous answers are very well written. I just wanted to supplement the thread by giving a simple example. The example shows how a logical function can be computed without errors using noisy components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taken verbatim from &lt;a href=&quot;http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/1996/NeuralNetworks/neuron.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks by Raul Rojas&lt;/a&gt;. An excellent book:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lPXkG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lPXkG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;an example of a network built using four&#xA;  units. Assume that the first three units connected directly to the three bits of&#xA;  input &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, x_3$&lt;/span&gt; all fire with probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; when the total excitation is greater&#xA;  than or equal to the threshold &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; but also with probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; when it is &lt;span class=&quot;math-container&quot;&gt;$\theta − 1$&lt;/span&gt;.&#xA;  The duplicated connections add redundancy to the transmitted bit, but in&#xA;  such a way that all three units fire with probability one when the three bits&#xA;  are &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Each unit also fires with probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; if two out of three inputs are &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&#xA;  However each unit reacts to a different combination. The last unit, finally, is&#xA;  also noisy and fires any time the three units in the first level fire and also with&#xA;  probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; when two of them fire. Since, in the first level, at most one unit&#xA;  fires when just two inputs are set to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, the third unit will only fire when all&#xA;  three inputs are &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. This makes the logical circuit, the AND function of three&#xA;  inputs, built out of unreliable components error-proof.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-08-18T07:04:53.220" LastActivityDate="2019-08-18T07:04:53.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14019" PostTypeId="2" ParentId="13694" CreationDate="2019-08-17T07:19:54.783" Score="1" Body="&lt;p&gt;If you want to bee engineer who work with models as black boxes it could be OK. If you want to be researcher, as the job position or for better understanding of the subject it's not OK. Backporpagation is just basic multivariate calculus. If you straggling with it things like Hessians, regularizers, stochastic processes etc. would cause even more problems. If you want to go research track it could be good idea to take some math courses and prioritize them. &lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-08-17T07:19:54.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14020" PostTypeId="1" AcceptedAnswerId="14036" CreationDate="2019-08-17T08:34:08.307" Score="3" ViewCount="65" Body="&lt;p&gt;I'm looking for NLP techniques to transform sentences without affecting their meaning, and measures of textual consistency. For example, techniques that could transform active voice into passive voice, such as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The cat was chasing the mouse.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The mouse was being chased by the cat.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can think of a number of heuristics one could implement to make this happen for specific cases, but would assume that there is existing research on this in the field of linguistics or NLP. My searches for &quot;sentence transformation&quot; and similar terms didn't bring up anything though, and I'm wondering if I simply have the wrong search terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related to this, I'm also looking for measures of text consistency, e.g., an approach that could detect that most sentences in a corpus are written in active voice and detect outliers written in passive voice. I'm using active vs. passive voice as an example here and would be interested in more general approaches. Any pointers would be much appreciated, thanks!&lt;/p&gt;&#xA;" OwnerUserId="28018" LastEditorUserId="2444" LastEditDate="2019-08-17T13:47:46.770" LastActivityDate="2019-08-18T12:56:35.477" Title="Techniques for sentence transformation and consistency" Tags="&lt;natural-language-processing&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14021" PostTypeId="2" ParentId="11336" CreationDate="2019-08-17T08:46:42.037" Score="1" Body="&lt;p&gt;Specifically for face recognition (and other identification algorithms) there are better approaches than using classifiers directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most identity recognition algorithms generate some kind of metric - typically an &quot;embedding&quot; of the original image into an abstract space i.e. a vector of real numbers. The space might be based on real-world biometrics e.g. normalised measurements of eye distance, eyebrow arch etc, which would be trained as a regression algorithm. The problem with this is that it requires a lot of labelled data, and the biometrics are not necessarily good at differentiating between identities. An alternative is to get the neural network to find the best abstract space for identities. You can do this if you have at least two images of each identity, and using &lt;a href=&quot;https://en.wikipedia.org/wiki/Triplet_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;triplet loss&lt;/a&gt; to train the network - the loss function directly rewards embedding the same identity close and different identity far apart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have an embedding, you no longer directly classify identities using the neural network. Instead, you base identity on distance between measured embedding and stored embeddings. This requires implementing a search function that looks at known embeddings and sorts by distance. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;how do we ensure the network tells us when it encounters a new person?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Embeddings don't solve this problem directly, but give a useful heuristic - distance in embedding space. Typically a maximum allowed distance is set as a cutoff to consider an image as showing a new identity. This is a hyperparameter of the model. This is an area that triplet loss helps with, since it is trained to make the distance as large as possible between images that show different identities. If it has generalised well during training, then it should ignore differences due to lighting, pose, makeup etc, but still be able to differentiate similar looking people. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the embedding is approximate, any such system may make mistakes, and needs to be carefully tested. The quality and quantity of training data are important, and it should match images used in production. But that is no different to the pure classifier, which must in addition be re-built and re-trained for every new class added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether to use a more basic classifier or something like triplet loss is a question of scale - if the number of identities that need to be tracked is high, or the rate of change in identities is high, then embeddings trained on triplet loss (or similar) are more practical.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-17T09:11:48.640" LastActivityDate="2019-08-17T09:11:48.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14022" PostTypeId="2" ParentId="2452" CreationDate="2019-08-17T14:48:56.370" Score="1" Body="&lt;p&gt;CRUD applications today can't be considered expert systems.&lt;/p&gt;&#xA;&lt;p&gt;However, even the so-called expert systems, which are currently developed, are implemented using normal programming statements, but what is important is the architecture that is built.&lt;/p&gt;&#xA;&lt;p&gt;Current expert systems use only if-then types of rules, which produce data results that can be used as inputs to other rules, and an engine to step through them. This is quite limited, and it is greatly fragile.&lt;/p&gt;&#xA;&lt;p&gt;What I do consider as expert systems are ones that can reason about variables (logical and numerical ones) and can use the limited formation of hypotheses and attempt a proof of them.&lt;/p&gt;&#xA;&lt;p&gt;But, unfortunately, even what you might analyze and describe as an expert system is not really able to form models by itself, so it can easily run up against knowledge boundaries beyond which it cannot go.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, CRUD web applications today are not a modern version of the expert systems.&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2020-12-31T20:46:26.490" LastActivityDate="2020-12-31T20:46:26.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14023" PostTypeId="1" CreationDate="2019-08-17T16:30:28.230" Score="1" ViewCount="25" Body="&lt;p&gt;You can feed books to an RNN and it learn how to produce text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm interested in is an algorithm that, given say 20 letters it suggest, say the best 10 options for the next 10 letters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example it begins with &quot;The cat jumped &quot;&#xA;and then we get various options such as &quot;over the dog&quot;. &quot;on the table&quot; and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial thoughts are to first use the most likely next letters. Then find the letter which is most uncertain and change this to the second likely next letter. And repeat this process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Then I may have another evaluation neural network to assess which is &quot;best&quot; English.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words I want the RNN to &quot;think ahead&quot; at what it's saying - much like a chess playing machine.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-08-17T16:30:28.230" Title="Is there an standard algorithm for giving options from an RNN?" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14024" PostTypeId="2" ParentId="13859" CreationDate="2019-08-17T18:10:40.850" Score="2" Body="&lt;p&gt;Artificial intelligence cannot be boiled down to designing algorithms, binary or otherwise, simply because the exhibition of intelligence in biological systems predated the invention of algorithmic computing.  From this, we can further draw the conclusion that algorithms are not a necessary component of systems that exhibit behavior we deem intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A decision was made, per the recommendation of John von Neumann, to increase reliability of computing machinery by delegating to a single binary central processing unit all computation. This choice and the prior work upon which it was based (Shannon, Church, and Turing) led to the preeminence of algorithm specification in computer languages. The foundation of expressing functional design in algorithmic terms was laid and the software industry was born.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since that time, there has existed a parallel trend in research back toward the biological inspiration of computing machinery and, more specifically, parallel processing.  We see this at several levels.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Movement of floating point arithmetic, video rendering, and machine learning bottlenecks to dedicated VLSI hardware acceleration&lt;/li&gt;&#xA;&lt;li&gt;Multiple core VLSI processors&lt;/li&gt;&#xA;&lt;li&gt;Computing clusters and processing frameworks, containers, and environments that expose interfaces through which compiler and kernel programmers can control parallel machinery explicitly or implicitly&lt;/li&gt;&#xA;&lt;li&gt;Multiple thread and processes delegated to multiple cores, agents, or hosts in computing clusters&lt;/li&gt;&#xA;&lt;li&gt;Sophisticated VLSI level caching to maximize the efficiency of parallel operations&lt;/li&gt;&#xA;&lt;li&gt;Language and compiler features to support the trends toward deployment to multiprocessing environments, such as declarative languages for Big Data platforms (ECL for example)&lt;/li&gt;&#xA;&lt;li&gt;Development of AI chip designs that completely or partially shift the computing paradigm to prior to the emergence of the CPU in some ways, returning to considerable parallelism and departing from centralized processing (yet capitalizing on lessons learned in computer vision, cognitive science, reverse engineering of brain genetics, mental signal tracing, the use of gradient descent with back propagation, reinforcement designs, and applied robotics) &amp;mdash; This is likely a major research direction for the 2020s.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some believe that an implication of Gödel's two incompleteness theorems is that the human mind does not meet the criteria of a computing machine as Turing defined one, but these are largely tangential issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that The working out of a proof that RNNs of sufficient resolution, depth, and width can be trained to be equivalent to any Turing Machine by Hava Siegelmann. It is true that her work is considered support for Marvin Minski's bold assertion that the human brain is a meat machine. However, the work on determinism by John Lucas and Roger Penrose's &lt;em&gt;The Emperor's New Mind&lt;/em&gt; are not refutations of either of Gödel's theorems.  They are refutations of what some thought were consequences Gödel's theorems and some of the implications of Minski's declaration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gödel clearly explains his intentions in the early portion of the paper presenting the theorems, and they had nothing to do with computing.  He intended to and succeeded in proving that theorems within a concrete mathematical system not always be proven even if they are true.  Gödel's work placed unwanted doubt on the initiative to prove all remaining unproven mathematical theorems.  Mathematicians naturally tended to think of mathematics as the perfect human endeavor, and a legitimate proof of incongruity between what is true and what is provable seemed an imperfect irritation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the most profound response to Gödel's incompleteness theorems came from Alan Turing, who likely deliberately placed the word Completeness in the name of his theorem.  But this was not a refutation either.  He worked around incompleteness by defining a class of mathematical operations and finite data structures upon which they can operate that he could prove could be complete. Upon doing so, he put into place an important portion of the basis for algorithm development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, it is probably wise for present day AI researchers to accept both incompleteness and inconsistency and realize that intelligence, artificial or not, is likely fallible after any finite degree of learning.  This is likely because one cannot provide an infinite range of problem types to a learning system in a finite amount of time.  There may always be at least one problem that the current state of learning cannot address.  The practical colloquialism for this condition of partial knowledge is, &quot;We don't know what we don't know.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, a clear implication of the work of Gödel is that no proof may be found for some things that are true, ever, by any type of intelligence.  Similarly, we cannot be sure that the most intelligent searching for a counter example to dispute a false assertion may end in finding one, ever.  The PAC Learning framework addresses categories of problems that are solvable or not from a mathematical perspective and is worthy of study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, but perhaps most profoundly, it is not clear that a type of intelligence exists that can learn anything, as opposed to be programmed to accomplish anything.  Said another way, general intelligence may be an ideal conception never achieved but possibly approached. What may seem like super intelligence in one environment and during one specific time period may be entirely ineffective or even counter-intelligent and problematic in another environment or during a different time period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This cannot be stressed too much, with so many statements about AI being made in the guise of science that have no origin in scientific rigor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, even with these likely limitations on both AI and human intelligence, one cannot conclude that AI will be ineffective in real time critical applications.  One cannot conclude that AI will be less effective than human intelligence in any particular domain either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is actually difficult to conclude anything about intelligence at all, without defining it formally and reaching a consensus in that definition, which continues to escape us.  We can see that in the absence of this formality, mail industry from continuing to sort mail automatically.  The automotive industry continues to pursue the invention of better artificial drivers than the average human driver.  The game industry implements artificial opponents that have to deliberately make mistakes to let people win in an otherwise fair, real time game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly AI is evolving faster than the DNA components that affect the human brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People are less startled today than they would have been ten years ago by the proposition that, some time during this century, driving a car will be illegal in some jurisdictions, when the human and property loss statistics prove automated drivers to be substantially safer than nearly all manual ones.  The bar for driving safety set by humans is not very high, with day dreaming, texting, occasional tiredness or inebriation slowing an already insufficient reaction time for many street events.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the driving computing agent panics because there is determines the trajectory of a dog, a child, and an elderly person is intersecting with the car's trajectory, it may resolve the panic and plot a safe course in a millisecond (perhaps avoiding all three or perhaps sacrificing the dog to save the two people), whereas the human may resolve the panic only after hitting some one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, it is not infallibility that determines the proper balance or volume of AI deployment but the comparison of the distribution of human performances compared with the distribution found with the machine replacements under similar conditions.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-08-18T11:25:35.303" LastActivityDate="2019-08-18T11:25:35.303" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14027" PostTypeId="2" ParentId="9372" CreationDate="2019-08-18T01:56:19.233" Score="4" Body="&lt;p&gt;There is an academic paper here that studies a neural approach to deciphering ancient languages:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://arxiv.org/pdf/1906.06718.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1906.06718.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;In this paper we propose a novel neural approach for automatic decipherment of lost languages. To compensate for the lack of strong supervision signal, our model design is informed by patterns in language change doc-umented in historical linguistics. The model utilizes  an  expressive  sequence-to-sequence model to capture character-level  correspon-dences between cognates. To effectively train the model in an unsupervised manner, we innovate the training procedure by formalizing it as a minimum-cost flow  problem. When applied to the decipherment of Ugaritic, we achieve a 5.5% absolute improvement overstate-of-the-art  results. We also  report  the first automatic results in deciphering Linear B, a syllabic language related to ancient Greek, where our model correctly translates 67.3% of cognates.&quot; ― Luo, Jiaming, Yuan Cao, and Regina Barzilay. &quot;Neural Decipherment via Minimum-Cost Flow: from Ugaritic to Linear B.&quot; &lt;em&gt;arXiv preprint arXiv:1906.06718&lt;/em&gt; (2019).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Reading and Articles for the Layperson&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://themindguild.com/artificial-intelligence-is-now-translating-long-lost-ancient-languages/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://themindguild.com/artificial-intelligence-is-now-translating-long-lost-ancient-languages/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://bstrategyhub.com/sanskrit-is-the-best-language-for-artificial-intelligence-says-nasa/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bstrategyhub.com/sanskrit-is-the-best-language-for-artificial-intelligence-says-nasa/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;http://www.openculture.com/2019/07/can-artificial-intelligence-decipher-lost-languages.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openculture.com/2019/07/can-artificial-intelligence-decipher-lost-languages.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="25982" LastEditDate="2019-08-18T11:06:53.697" LastActivityDate="2019-08-18T11:06:53.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14028" PostTypeId="1" AcceptedAnswerId="14092" CreationDate="2019-08-18T04:51:13.947" Score="0" ViewCount="175" Body="&lt;p&gt;I have developed face recognition algorithms by using pre-built libraries in Python and open CV. However, suppose if I want to make my own neural network algorithm for face recognition, what are the steps that I need to follow?&lt;/p&gt;&#xA;&lt;p&gt;I have just seen &lt;a href=&quot;https://www.youtube.com/watch?v=PPLop4L2eGk&amp;amp;list=PLLssT5z_DsK-h9vYZkQkYNWcItqhlRJLN&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's course videos&lt;/a&gt; (specifically, I watched 70 videos).&lt;/p&gt;&#xA;" OwnerDisplayName="user28028" LastEditorUserId="2444" LastEditDate="2020-09-07T22:30:16.193" LastActivityDate="2020-09-13T10:23:17.590" Title="What are the steps that I need to follow to build a neural network for face recognition?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;face-recognition&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14029" PostTypeId="2" ParentId="13993" CreationDate="2019-08-18T06:31:32.257" Score="1" Body="&lt;p&gt;You could argue that AI and big data are trying to switch the AI method from deductive to inductive reasoning in the sense that original AI was deductive (if...then conditionals) but deep learning implies inductive reasoning (feed the network a million images of white swans and the network will &quot;conclude&quot; all swans are white - the classic example of (erroneous) inductive reasoning.)&lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2019-08-18T06:31:32.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14030" PostTypeId="2" ParentId="8518" CreationDate="2019-08-18T08:53:12.107" Score="2" Body="&lt;p&gt;Penalty (&lt;a href=&quot;https://en.wikipedia.org/wiki/Barrier_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;barrier function&lt;/a&gt;) is perfectly valid and simplest method for simplex type constraint (L1 norm is simplex constraint on absolute values). Any type of barrier function may work, logarithmic, reciprocal or quadratic. All of them supported by any major framework(pytorch, tensorflow), just add them to loss function. You would need some hyperparameter tuning for the scale factor of penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is more efficient, though more complex way to do it. Instead of putting constraint you can automatically output value wich satisfy simplex constraint:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that L1 norm constraint is &lt;span class=&quot;math-container&quot;&gt;$\left \|v\right \|_1 \leq 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v \in \mathbb{R}^n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;put &lt;span class=&quot;math-container&quot;&gt;$sigmoid(v_i)$&lt;/span&gt; activation on output to norm elements to [-1, 1]&lt;/li&gt;&#xA;&lt;li&gt;add slack (fake) variable element &lt;span class=&quot;math-container&quot;&gt;$v_{n+1} = 1 - \sum_{1}^{n}  v_i $&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;project new &lt;span class=&quot;math-container&quot;&gt;$v{}'\in \mathbb{R}^{n+1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v{}'_i = |v_i|,1\leq  i  \leq n+1$&lt;/span&gt; onto unit simplex &lt;a href=&quot;https://en.wikipedia.org/wiki/Simplex#Projection_onto_the_standard_simplex&quot; rel=&quot;nofollow noreferrer&quot;&gt;with standard algorithm&lt;/a&gt; (also &lt;a href=&quot;https://arxiv.org/abs/1101.6081&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Backpropagation of last step may require &lt;em&gt;differentiable sorting&lt;/em&gt;, which is missing in most of frameworks, you may have to look for open sourced implementation, for example extract it from &lt;a href=&quot;https://github.com/MaestroGraph/sparse-hyper&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or use some automatic differentiation package. Both require some substantial code reading/debugging. However in my experience assuming constant &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; also works in many cases, in that case  differentiable sorting is not needed. Intuition behind constant &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt;  is that  &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt;  could be chosen such way that there is some interval on which it's value doesn't affect sorting order.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2019-08-18T09:13:58.340" LastActivityDate="2019-08-18T09:13:58.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14031" PostTypeId="2" ParentId="9890" CreationDate="2019-08-18T10:36:25.240" Score="1" Body="&lt;p&gt;The probability map / output isn't produced by your loss function, but your output layer, which is activated either by softmax or sigmoid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, your dice loss output is also a probability map. It's simply very confident in itself. If you forget about the problem with potential overfitting for a moment and train your binary crossentropy model longer, the probability values will eventually all converge to the 2 ends (0 and 1). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, dice loss and IOU tend to converge much faster than binary crossentropy for semantic segmentation, so if you stop the training early on, dice loss will produce a probability map that resembles a binarized output more so than binary crossentropy.&lt;/p&gt;&#xA;" OwnerUserId="28033" LastEditorUserId="1671" LastEditDate="2019-08-22T20:04:15.723" LastActivityDate="2019-08-22T20:04:15.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14032" PostTypeId="1" CreationDate="2019-08-18T11:03:58.400" Score="1" ViewCount="74" Body="&lt;p&gt;I have a robotics assignment, which I am unable to solve. Given the axis-angle rotation vector &lt;span class=&quot;math-container&quot;&gt;$\Theta = (2, 2, 0)$&lt;/span&gt;, how can I calculate the unit vector of the rotation axis &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and the angle &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="28034" LastEditorUserId="2444" LastEditDate="2021-01-09T20:19:08.183" LastActivityDate="2021-02-08T22:01:15.107" Title="Given an axis-angle rotation vector, how can I find the unit rotation axis and angle?" Tags="&lt;math&gt;&lt;robotics&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14033" PostTypeId="1" CreationDate="2019-08-18T11:39:03.080" Score="2" ViewCount="12" Body="&lt;p&gt;I have a mobile app recommendation project, so I need data set which has user-app matrix-rate. Actually, I want to know what rate does a specific user submit for a specific application. in other words, &quot;user-app matrix-rate&quot; is a 2-D matrix, rows are users, columns are applications and cell(i,j) is rate of user i submit for application j.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, It's good for me to know &quot;user-app matrix-install&quot; where &quot;user-app matrix-install&quot; is a 2-D matrix, rows are users, columns are applications and cell(i,j) is 1 if user i installed application j, otherwise cell(i,j) is 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks for your help&lt;/p&gt;&#xA;" OwnerUserId="28037" LastActivityDate="2019-08-18T11:39:03.080" Title="Mobile App Recommendation: How to get the rate of a specific user submit for a specific application" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14034" PostTypeId="2" ParentId="13968" CreationDate="2019-08-18T12:10:48.793" Score="-1" Body="&lt;p&gt;﻿Reinforcement learning: How to deal with illegal actions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be useful to you to consider the action space and the state space using the mathematical constructs of sets, regions, and boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the variable type used for actions allows the representation of actions that are not in the action space, then the type will lead to the inefficiency of validation.  This is also true for the variable type used for states.  In the current state of the technology, this is often the case, leaving the responsibility of execution efficiency with how those variables are assigned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is faster?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assign a variable to a possibly invalid value.&lt;/li&gt;&#xA;&lt;li&gt;Validate it.&lt;/li&gt;&#xA;&lt;li&gt;Deal with invalid values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... or, applying the concepts of sets, regions, and boundary conditions to the assignment mechanism, ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assign a variable to a value within constraints that represent the actor and its environment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Applying this to the problem of this question, what is outside the boundary conditions?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it the negative current (relative to the polarity of the storage device) when the storage device's voltage is zero?&lt;/li&gt;&#xA;&lt;li&gt;Is it a negative voltage that could potentially result from a negative current applied to a zero voltage storage device?&lt;/li&gt;&#xA;&lt;li&gt;Is it the cost of the physical disintegration of the storage device under this condition?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These questions help illuminate whether this is a question of what is legal or forbidden or whether it is a question of return?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the cost of wasting electrical energy?&lt;/li&gt;&#xA;&lt;li&gt;Wasting materials?&lt;/li&gt;&#xA;&lt;li&gt;Wasting time?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Applying a negative current to a dead lithium, alkaline, or lead acid battery wastes all three.  The wasteful actions that lead to wasted states exist, but the curve of return on the investment of energy in resale or reuse value of the battery should not be represented by a first degree polynomial.  It should not be represented by -9999 either, as that doesn't nearly represent the degree of waste, and the knee of the curve may not be hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If using discrete states and actions rather than floating point representations of current and voltage, then a lookup table (perhaps tuned by experience using other AI techniques and indexed on the make and model of the storage device) must represent the actual curve approximating return on investment.  If the goal is making money, as the question suggests, the goal of the learning is to maximize ROI.&#xA;That must be embodied in your definition of optimality and then implemented to remain faithful to that definition in the DQN code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, in real application, the initial internal resistance in combination with the initial voltage is key to determining whether a charge can ever produce a positive return for the given battery, so if voltage and current are continuously available, your test suite (which you should implement first if you are adhering to test driven development best practices) should include a case that tests the abandonment of charging altogether as a cost mitigation necessity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The test of the algorithm regarding its fitness for the project is whether it behaves when the representations are real world.  If not, there are other AI strategies to test.  Not every algorithm is fit for every purpose, which is why job openings titled &quot;AI Architect&quot; and &quot;AI Designer&quot; are beginning to appear.  It is likely that DQN will work if the value function is either set correctly by you or learned correctly by the appropriate learning mechanism that converges on good value approximations.  The later method is suggested as best practice in this case.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-18T12:10:48.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14035" PostTypeId="2" ParentId="14007" CreationDate="2019-08-18T12:44:55.353" Score="0" Body="&lt;p&gt;Sebastian Thrun in one of his online interviews once suggested that he thought that the conversational solution was a combination of both massive machine learning and rules based programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem in the case of chats is that our expectations are very high which dooms early solutions to failure. For the ML side we require large amounts of data, and while large amounts may be available they are highly biased and unbalanced; they mostly focus on one area (specialized context) and re-use the same sentence formulas over and over again, so the learning finds a comfortable corner case solution and refuses to learn anything else. People are so predictable so they are an unhelpful source of raw data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach might be to use carefully constructed rules to generate the data that ML can learn from, rules that can guarantee broad contextual applicability and sentence construction variation.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2019-08-18T12:44:55.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14036" PostTypeId="2" ParentId="14020" CreationDate="2019-08-18T12:56:35.477" Score="0" Body="&lt;p&gt;Strictly speaking, this is impossible. Changing the form of a sentence also changes its meaning. Even active-passive can be important, as you would use it to emphasise what is important: was it relevant what the cat was doing, or was it more relevant what happened to the mouse? True, the purely propositional meaning is not affected by this, but that is only one component of a sentence's meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been a lot of work in traditional linguistics about sentence form. You could look at one of the seminal works, &lt;a href=&quot;https://en.wikipedia.org/wiki/Syntactic_Structures&quot; rel=&quot;nofollow noreferrer&quot;&gt;Syntactic Structures&lt;/a&gt; by &lt;a href=&quot;https://en.wikipedia.org/wiki/Noam_Chomsky&quot; rel=&quot;nofollow noreferrer&quot;&gt;Noam Chomsky&lt;/a&gt;, where he introduces the concept which later lead to &lt;a href=&quot;https://en.wikipedia.org/wiki/Transformational_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformational grammar&lt;/a&gt;. This influenced a lot of subsequent linguistic approaches, but as far as I am aware transformations are not really that much in the linguistic focus anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question, stylistic consistency, you could look at the work of Douglas Biber. His book &lt;em&gt;&lt;a href=&quot;https://www.cambridge.org/core/books/variation-across-speech-and-writing/A546CF5ED8F8E62F1432CB2F369CF356&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variation across speech and writing&lt;/a&gt;&lt;/em&gt; introduces a number of (easily extractable) linguistic features that you could use to quantify consistency.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-08-18T12:56:35.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14037" PostTypeId="2" ParentId="13916" CreationDate="2019-08-18T13:12:33.027" Score="0" Body="&lt;p&gt;﻿Sound and image manipulation necessarily creates artifacts.  Around the edges of superimposition in layers there are such.  Face replacement and other more surface or object centered operations create a different class of artifacts.  A sufficiently well constructed LSTM or GRU network and data set of manipulated frame sequences and the user (mouse and keyboard) events that manipulated them can be used to produce good guesses of the event set from new images.  Adding unmanipulated images to the data set can allow for the no-event case.  That would be the supervised way to do it.  There are unsupervised approaches that would require considerably less training resources, which is likely the case with this San Francisco solutions provider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, the question of escalation is a good one.  One can also create a device, building from the current state of machine learning, that hides manipulations from existing detection software.  If they are forward thinking, the same provider may have already developed it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can we combat against deepfakes? ...&#xA;  I am wondering that the people who are creating deepfakes can as well their AI's to remove these imperfections ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes and yes.  In war, the combatants learn the methods of the opposing combatants and adapt.  A detection mechanism for opposing strategy changes is also theoretically possible, which is one of the reasons that military research facilities spend so much on higher forms of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The edit to the question is not entirely tangential either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we propose, which some people have, that a virtual reality may damage human culture or individual psyches, the average citizen is likely to be considered collateral damage on the field of combat by companies seeking a good financial return from their AI development.  Of course, we could say the same thing about the use of diminished fifths in music.  Two notes that are six half steps apart produce a dissonant frequency ratio of &lt;span class=&quot;math-container&quot;&gt;$1:\sqrt{2}$&lt;/span&gt;.  The diminished fifth was considered subliminally satanic in Europe centuries ago and prohibited in music compositions by law.  The glass harmonica was alleged to have driven listeners insane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anthropologically, it is possible that a mark of our species is to manipulate appearance.  To hunt fakes in frames and audio is likely a fruitless hunting ground, with our without the escalation.  The current hunting ground of import is the research into what genetic elements led to human abilities to imagine, design, and fabricate.  After that is known, we may have a better window into whether the cat-and-mouse games we play have any sustainable value for our species going forward.  Those who love competition believe that it strengthens, which is possible.  It is also possible that the games are solely an artifact of a painful path to our emergence as the dominant mammalian species and no longer of any particular use.  &quot;Do to others as you would want them to do to you,&quot; has the ring of truth we can't ignore either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we look through this wider lens, we can see that our entertainment choices tend toward what could (in the absence of bias) qualify as deepfakes.  There are entire cities fueled by the money made by the entertainment industry producing excellence in sound and image capture, synthesis, and manipulation.  The story lines are not necessarily representing deep truths.  This is more overt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the more covert side, some pass fakes off as reality as a move in their own game to achieve some objective, but this is not the exception in our culture.  The fields of public relations and marketing are based on the creation and preservation of business value.  Some elements of government, education, and community are based on the creation of economy-preserving beliefs.  The intention may be to benefit others or beat them and gain personal wealth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of us seek authenticity and would like the fake-finders to win the combat, but it appears they may be on the losing side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this question and this answer pertain to this Stack Exchange community?  Absolutely.  This community's description in the drop down of SE communities reads, &quot;For people interested in conceptual questions about life and challenges in a world where 'cognitive' functions can be mimicked in purely digital environment.&quot;  Whether AI ultimately weighs in on the side of playing people or informing them certainly pertains to this published view of this community's purpose.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-18T13:12:33.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14038" PostTypeId="2" ParentId="13973" CreationDate="2019-08-18T13:36:08.887" Score="6" Body="&lt;p&gt;The convergence and optimality proofs of (linear) temporal-difference methods (under batch training, so not online learning) can be found in the paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007%2FBF00115009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to predict by the methods of temporal differences&lt;/a&gt; (1988) by Richard Sutton, specifically section 4 (p. 23). In this paper, Sutton uses a different notation than the notation used in the famous book &lt;a href=&quot;http://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (2nd ed.), by Sutton and Barto, so I suggest you get familiar with the notation before attempting to understand the theorem and the proof. For example, Sutton uses letters such as &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; to denote &lt;em&gt;states&lt;/em&gt; (rather than &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;), &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; to denote (scalar) &lt;em&gt;outcomes&lt;/em&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to denote (vector) &lt;em&gt;observations&lt;/em&gt; (see section 3.2 for example of the usage of this notation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1023%2FA%3A1022632907294.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Convergence of TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) for General &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;&lt;/a&gt; (1992), Peter Dayan, apart from recapitulating the convergence proof provided by Sutton, he also shows the convergence properties of TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) and he extends Watkins' Q-learning convergence theorem, whose sketch is presented in his PhD thesis &lt;a href=&quot;https://www.cs.rhul.ac.uk/home/chrisw/new_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning from Delayed Rewards&lt;/a&gt; (1989), and defined in detail in &lt;a href=&quot;http://www.gatsby.ucl.ac.uk/~dayan/papers/cjch.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technical Note: Q-learning&lt;/a&gt; (1992), by Dayan and Watkins, to provide the first strongest guarantee or convergence proof for TD(0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is much more research work on the convergence properties of TD methods, such as Q-learning and SARSA. For example, in the paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.4499&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the Convergence of Stochastic Iterative Dynamic Programming Algorithms&lt;/a&gt; (1994), where Q-learning is presented as a stochastic form of dynamic programming methods, the authors provide a proof of convergence for Q-learning by making direct use of &lt;a href=&quot;http://www.stat.columbia.edu/~liam/teaching/compstat-spr14/lauren-notes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic approximation theory&lt;/a&gt;. See also &lt;a href=&quot;http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence of Q-learning: a simple proof&lt;/a&gt; by Francisco S. Melo. In the paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1023%2FA%3A1007678930559.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms&lt;/a&gt;, the authors provide a proof of the convergence properties of on-line temporal difference methods (e.g. SARSA).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-18T19:13:15.377" LastActivityDate="2019-08-18T19:13:15.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14040" PostTypeId="2" ParentId="14032" CreationDate="2019-08-18T17:12:30.590" Score="0" Body="&lt;p&gt;Given the &lt;a href=&quot;https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation&quot; rel=&quot;nofollow noreferrer&quot;&gt;axis-angle&lt;/a&gt; rotation vector &lt;span class=&quot;math-container&quot;&gt;$\Theta = (2, 2, 0)$&lt;/span&gt;, you can find the unit vector in the same direction by diving by the norm (or length) of &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt;, denoted by &lt;span class=&quot;math-container&quot;&gt;$\|\Theta\| = \sqrt{2^2 + 2^2 + 0^2} = \sqrt{8} = 2\sqrt{2}$&lt;/span&gt;. Therefore, the unit vector in the direction of &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$k= \Theta/\|\Theta\| = (1/\sqrt{2}, 1/\sqrt{2}, 0)$&lt;/span&gt;, which should be the axis of rotation that you're looking for. The angle should just be the norm of &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$\theta = \|\Theta\| = 2\sqrt{2}$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$k \theta$&lt;/span&gt; gives you your original vector &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-18T17:23:39.153" LastActivityDate="2019-08-18T17:23:39.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14041" PostTypeId="1" CreationDate="2019-08-18T17:33:51.073" Score="3" ViewCount="320" Body="&lt;p&gt;Given an axis-angle rotation vector &lt;span class=&quot;math-container&quot;&gt;$\Theta = (2,2,0)$&lt;/span&gt;, after finding the unit vector &lt;span class=&quot;math-container&quot;&gt;$k=(1/\sqrt{2}, 1/\sqrt{2}, 0)$&lt;/span&gt; and angle &lt;span class=&quot;math-container&quot;&gt;$\theta = 2\sqrt{2}$&lt;/span&gt; representing the same rotation, I need to derive the rotation matrix &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; representing the same rotation and to show, that the matrix is orthonormal. How can I do that?&lt;/p&gt;&#xA;" OwnerUserId="28035" LastEditorUserId="2444" LastEditDate="2019-08-18T18:30:30.073" LastActivityDate="2019-08-21T14:38:53.597" Title="How can I derive the rotation matrix from the axis-angle rotation vector?" Tags="&lt;math&gt;&lt;robotics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14042" PostTypeId="2" ParentId="9372" CreationDate="2019-08-18T17:44:51.537" Score="0" Body="&lt;p&gt;I would say that it depends on whether that language would have wanted to be decyphered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The origin of Cryptography dates back to around 700-800 AD. &#xA;We may not know the methods by which these texts are obscurred. One such example is the Lesser Banishing Ritual of the Pentagram which, revealed only by happenstance, formed the basis of a whole new esoteric renaissance in the west, the lexicon included forming a basis for the lexicon in use by these orders today. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This Rite centers around the letters INRI, which tradition says were written upon the cross of Jesus Christ as an abbreviation for Jesus of Nazareth, King of the Jews. There are, however, numerous other levels of occult meaning regarding these four letters in the Rosicrucian Magical Tradition. One of these is a Hermetic secret alluded to by the Latin phrase &quot;Igne Natura Renovatur Integra&quot; which means &quot;By fire, nature is perfectly renewed.&quot; These four letters additionally adorn the rays of the angles of the Rose Cross Lamen worn by Adepts of the Ordo Rosae Rubeae et Aureae Crucis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A deeper interpretation lies occulted behind the attributions of the Hebrew letters and the Magical Forces to the Paths on the Qabalistic Tree of Life. The Path attributed to the Hebrew letter y is attributed to the Zodiacal Sign Virgo as well, that of n to Scorpio, and r to the Sun. There exist further magical associations between the Sign Virgo with the Egyptian Goddess Isis, Scorpio with Apophis, and the Sun with Osiris. When the first letter is taken from the Names of each of these Gods, the name &quot;IAO&quot; is formed. Additionally, due to the Signs associated with Isis, Apophis, and Osiris, they form the letters &quot;LVX.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus within the letters IRNI lie concealed the letters IAO and LVX, which may also be found upon the rays of the angles of the Rose Cross Lamen worn by Adepts of the R. R. et A. C. The name IAO was considered by the Gnostics to be the Supreme Name of God. Its letters further allude to Salt, Sulfur, and Mercury in Alchemy and to an even more recondite secret symbolized by the relationship between Isis, Apophis, and Osiris.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I would say for machine learning to be able to decipher real languages, it would almost have to be intuitive, able to draw parallels and connect various points of reference together. It would have to determine what things mean, when a lot of the times, this is virtually impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your variables are infinite. You have no real method of finding out what they are. Add to that context, lexicons, deliberate obfuscation, culture and the passage of time and I think it would not be realistic to expect machine-learning to provide much help.  &lt;/p&gt;&#xA;" OwnerUserId="28043" LastActivityDate="2019-08-18T17:44:51.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14044" PostTypeId="2" ParentId="14041" CreationDate="2019-08-18T19:03:15.953" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Rotation_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;rotation matrix&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$R_k(\theta)$&lt;/span&gt; associated with a given unit-length vector &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and angle &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is given by the following formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\small{R_k(\theta) = \begin{bmatrix} &#xA;\cos \theta +k_x^2 \left(1-\cos \theta\right) &amp;amp; k_x k_y \left(1-\cos \theta\right) - k_z \sin \theta &amp;amp; k_x k_z \left(1-\cos \theta\right) + k_y \sin \theta \\ &#xA;k_y k_x \left(1-\cos \theta\right) + k_z \sin \theta &amp;amp; \cos \theta + k_y^2\left(1-\cos \theta\right) &amp;amp; k_y k_z \left(1-\cos \theta\right) - k_x \sin \theta \\ &#xA;k_z k_x \left(1-\cos \theta\right) - k_y \sin \theta &amp;amp; k_z k_y \left(1-\cos \theta\right) + k_x \sin \theta &amp;amp; \cos \theta + k_z^2\left(1-\cos \theta\right)&#xA;\end{bmatrix}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to find your specific rotation matrix, you just need to substitute the values of your &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in the above matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivation of this matrix can be found in section &lt;em&gt;9.2 Rotation Matrix Derivation&lt;/em&gt; of the PhD thesis &lt;a href=&quot;https://dspace.lboro.ac.uk/dspace-jspui/handle/2134/18050&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Modelling CPV&lt;/em&gt; (2015)&lt;/a&gt;, by Ian R. Cole. The basic idea of the derivation follows the following steps&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Rotate the given axis &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and the point &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (that you want to rotate) such that the axis &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; lies in one of the coordinate &lt;strong&gt;planes&lt;/strong&gt;: xy, yz or zx&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rotate the given axis &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and the point &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (that you want to rotate) such that the axis &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is aligned with one of the two coordinate &lt;em&gt;&lt;strong&gt;axes&lt;/strong&gt; for that particular coordinate &lt;strong&gt;plane&lt;/strong&gt;&lt;/em&gt;: &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use one of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rotation_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;fundamental rotation matrix&lt;/a&gt; to rotate the point &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; depending on the coordinate axis with which the rotation axis is aligned&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reverse rotate the axis-point pair such that it attains the final configuration as that was in step 2 (that is, you have to undo step 2)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reverse rotate the axis-point pair which was done in step 1 (that is, you have to undo step 1)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To show that a matrix is orthonormal, you need to show that it is &lt;a href=&quot;https://en.wikipedia.org/wiki/Orthogonal_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;orthogonal&lt;/em&gt;&lt;/a&gt; (each row is independent of any other row and each column is independent of any other column) and that the length of each row (and column) is 1. Equivalently, a &lt;em&gt;square&lt;/em&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is orthogonal if and only if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q^TQ = QQ^T = I&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Q^T$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Transpose&quot; rel=&quot;nofollow noreferrer&quot;&gt;transpose&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is the identity matrix. If you are really stuck, have a look at this proof &lt;a href=&quot;https://math.stackexchange.com/a/537248/168764&quot;&gt;https://math.stackexchange.com/a/537248/168764&lt;/a&gt; (and this other answer &lt;a href=&quot;https://math.stackexchange.com/a/156742/168764&quot;&gt;https://math.stackexchange.com/a/156742/168764&lt;/a&gt;), but, at this point, it should just be a matter of replacing &lt;span class=&quot;math-container&quot;&gt;$R_k(\theta)$&lt;/span&gt; in the equation above and check that the equation holds.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-21T14:38:53.597" LastActivityDate="2019-08-21T14:38:53.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14045" PostTypeId="2" ParentId="13901" CreationDate="2019-08-18T23:40:22.223" Score="2" Body="&lt;p&gt;I believe the answer is yes, but the video edition is mostly programmatic. The AI part comes with detecting the right spots to cut. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You want to detect the right portion when someone picks a wooden skittle&lt;/li&gt;&#xA;&lt;li&gt;you want to detect when the Skittles stop moving on the ground (to detect this, also when they start moving)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These will give you timestamps on the video. The next step is to add some padding and run the drop-down frame operations in the video, which can all be done with a simple script. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might find this video very similar in concept, and maybe even as code source: &lt;a href=&quot;https://youtu.be/DQ8orIurGxw&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/DQ8orIurGxw&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the recognition of the objects in the video frames, there are lots of options. You might look into computer vision or object motion detection.&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2019-08-18T23:40:22.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14046" PostTypeId="1" CreationDate="2019-08-19T02:12:05.930" Score="2" ViewCount="117" Body="&lt;p&gt;First of all, I should mention that I have a very basic knowledge of ML so I apologize if this question seems trivial or stupid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a small personal project, basically an app that analyzes Facebook posts concerning movies and translates them into a rating (out of 100). The algorithm looks for keywords, the length of the post, etc.. to determine the individual rating, and then averages all the ratings among a user's FB friends to give the result. My question is, would I be able to drastically improve such algorithm by using ML or is it not worth it? If yes, what algorithms/techniques do you advise me to learn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All help is appreciated!&lt;/p&gt;&#xA;" OwnerDisplayName="user28054" LastActivityDate="2020-09-13T04:07:30.847" Title="Using ML to analyze Facebook posts" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14047" PostTypeId="1" AcceptedAnswerId="14306" CreationDate="2019-08-19T11:08:49.433" Score="3" ViewCount="361" Body="&lt;p&gt;There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with neural network architectures.&lt;/p&gt;&#xA;&lt;p&gt;By continual learning, I mean improving a model (while using it) with a stream of data coming in (maybe after a partial initial training with ordinary batches and epochs).&lt;/p&gt;&#xA;&lt;p&gt;A lot of real-world distributions are likely to gradually change with time, so I believe that we should be able to train NNs in an online fashion.&lt;/p&gt;&#xA;&lt;p&gt;Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them?&lt;/p&gt;&#xA;" OwnerUserId="11303" LastEditorUserId="2444" LastEditDate="2020-11-10T11:51:01.303" LastActivityDate="2020-12-12T12:02:12.357" Title="What are the state-of-the-art approaches for continual learning with neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;online-learning&gt;&lt;incremental-learning&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14048" PostTypeId="2" ParentId="13870" CreationDate="2019-08-19T11:18:27.550" Score="1" Body="&lt;p&gt;I would refer to your problem as having a &lt;strong&gt;&lt;em&gt;continuous&lt;/em&gt; state space&lt;/strong&gt;.&#xA;By using a 32-bit float variable you &lt;em&gt;discritize&lt;/em&gt; it. However, creating states for every possible value of a 32-bit float variable is probably too much. You should decide on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;strong&gt;variable range&lt;/strong&gt;: what is the real range of the position variables (e.g. from 0 m to 10 m), &lt;/li&gt;&#xA;&lt;li&gt;and what is the &lt;strong&gt;resolution&lt;/strong&gt; you require for your problem (e.g. 0.01 m or 0.1 m). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that you should take into account: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the sensor range resolution,&lt;/li&gt;&#xA;&lt;li&gt;the required range and resolution for the problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then, based on the number of states you could decide to discretize the states or to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Monte Carlo&lt;/em&gt;&lt;/a&gt; approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See for example the work of &lt;a href=&quot;http://proceedings.mlr.press/v28/brechtel13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brechtel et al. (2013)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="198" LastActivityDate="2019-08-19T11:18:27.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14050" PostTypeId="1" CreationDate="2019-08-19T14:35:29.083" Score="2" ViewCount="12" Body="&lt;p&gt;Inspired by: &lt;a href=&quot;https://www.instagram.com/ugurgallen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Two Worlds Pictures&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PIg8j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PIg8j.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rKPVq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rKPVq.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just want to create a Machine Learning Model that can automatically combine the opposite images into 1 image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about 2 possible solutions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://medium.com/tensorflow/move-mirror-an-ai-experiment-with-pose-estimation-in-the-browser-using-tensorflow-js-2f7b769f9b23&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pose Estimation&lt;/a&gt;: Detect humans and their poses from image data and search an archive by pose, but totally out of context.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://developers.google.com/web/showcase/2016/land-lines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Land Lines&lt;/a&gt;: explore similar lines.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It's just my ideas, do you have any recommendations? &#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="27990" LastActivityDate="2019-08-19T14:35:29.083" Title="How to use machine learning to create combine of opposite images side by side" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14052" PostTypeId="1" CreationDate="2019-08-19T15:36:57.337" Score="1" ViewCount="24" Body="&lt;p&gt;I am a newbie in reinforcement learning and trying to understand how to implement continuous actions bounded by &lt;span class=&quot;math-container&quot;&gt;$[-2, 2]$&lt;/span&gt;. My research shows that doing nothing is a possible solution (i.e. action of 4.5 is mapped to 2 and the action of -3.1 is mapped to -2), but I wonder if there are more elegant approaches.&lt;/p&gt;&#xA;" OwnerUserId="27472" LastEditorUserId="2444" LastEditDate="2021-01-22T01:46:56.070" LastActivityDate="2021-01-22T01:46:56.070" Title="DDPG: how to implement continuous action space bounded in the interval [-2, 2]?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;ddpg&gt;&lt;continuous-action-spaces&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14053" PostTypeId="2" ParentId="14013" CreationDate="2019-08-19T16:50:55.043" Score="0" Body="&lt;p&gt;I found a somewhat more accessible introduction here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/konvergen/an-introduction-to-adagrad-f130ae871827&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/konvergen/an-introduction-to-adagrad-f130ae871827&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me start from the last part of your question. The frequency of a parameter is in G_t, which is the accumulated sum of squared gradients from all the time steps up to step t. If the gradient vanishes in many of the previous steps, then you divide the learning rate with a smaller number for that parameter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for the first part, you want the parameter that is more frequent to have a smaller learning rate as it is updated on more iterations compared to a parameter which is updated only a small number of times. &lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-08-20T13:23:00.393" LastActivityDate="2019-08-20T13:23:00.393" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14054" PostTypeId="1" CreationDate="2019-08-19T17:12:33.753" Score="0" ViewCount="50" Body="&lt;p&gt;What are some famous algorithms that i can apply to develop a face recognition software to achieve more than 95% accuracy .What are some good sources related to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have basic knowledge related to neural networks .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Andrew Ng's video only don't provide any algorithm for ace detection .It just tells how to compare finals results.&lt;/p&gt;&#xA;" OwnerDisplayName="user28028" LastEditorDisplayName="user28028" LastEditDate="2019-08-20T13:14:32.373" LastActivityDate="2019-08-20T13:14:32.373" Title="Few doubts regarding facial recognition" Tags="&lt;reference-request&gt;&lt;software-evaluation&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14055" PostTypeId="1" CreationDate="2019-08-19T18:22:48.453" Score="2" ViewCount="54" Body="&lt;p&gt;Currenly I'm trying to reimplement alphazero in pure c++ using libtorch to accomodate my project's need. But when I training my model, I found out that the value loss doesn't decrese at all after even ~2000 iterations and the policy loss decreases pretty fast from the very begining.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have anybody met any similar issue when developing your alphazero project? And could you give some suggestion of the cause of my issue based on your experience?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many apreciates&lt;/p&gt;&#xA;" OwnerUserId="28008" LastActivityDate="2019-08-19T18:22:48.453" Title="Alphazero Value loss doesn't decrease" Tags="&lt;alphazero&gt;&lt;alphago-zero&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14056" PostTypeId="1" CreationDate="2019-08-19T20:57:58.363" Score="2" ViewCount="294" Body="&lt;p&gt;How do I create a chatbot using TensorFlow or PyTorch using like the one defined in DialogFlow? What are the best datasets that I can use so to create my own personal assistant like google assistant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to create a chatbot (an open-source project) as an assistant for custom tasks (like google assistant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried many neural network models like seq2seq but I couldn't get satisfiable results maybe because of the small dataset (I took from Simpsons movie script) or model (seq2seq). I am curious what model they use at google and what type of dataset they pick to give such good results or any normal person can create fully functional chatbots without relying on paid services ( like google's DialogFlow, api.ai, etc.) with good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently heard of OpenAi's implementation of a specific model named as gpt-2 which as they concluded in the paper showed remarkable performance but they didn't provide the dataset because of some reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to say there are a lot of resources and codes on the internet to make a working chatbot (or maybe that what they show) but when I try to replicate them I always fail to get even remotely close good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I need proper guidance on how to make and train such chatbots with my own laptop (with 16GB RAM, 2GB GPU . I can even get better configuration) and no other money spent on google services or any such paid API's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please suggest something if someone got good results.&lt;/p&gt;&#xA;" OwnerUserId="28071" LastEditorUserId="16565" LastEditDate="2019-08-21T07:15:52.867" LastActivityDate="2019-08-21T07:15:52.867" Title="How do I create a chatbot using tensorflow or pytorch using like the one defined in dialogflow?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14058" PostTypeId="2" ParentId="14046" CreationDate="2019-08-20T01:28:27.387" Score="1" Body="&lt;p&gt;for this kind of ml training, you will need a ton of data first, at least in the thousands. If you have a bot program that fetches those data for you, AI is the way to go. I'm not sure how else you would do it though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To train the nn you will need the inputs(the post) and the targets(the rating you want it to output). The targets could be anything you want, like the ratio of likes to views, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are tons of ML libraries out there and I recommend keras as it is easy to learn for beginners, hope it helps :)&lt;/p&gt;&#xA;" OwnerUserId="27925" LastActivityDate="2019-08-20T01:28:27.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14059" PostTypeId="1" CreationDate="2019-08-20T03:46:44.977" Score="3" ViewCount="136" Body="&lt;p&gt;What are some conferences for publishing papers on Deep Learning for Human Activity recognition? Do any of the major conferences have specific tracks for Human Activity Recognition?&lt;/p&gt;&#xA;" OwnerUserId="28077" LastEditorUserId="2444" LastEditDate="2021-01-22T01:31:30.170" LastActivityDate="2021-02-21T02:05:05.437" Title="What are some conferences for publishing papers on Deep Learning for Human Activity recognition?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;research&gt;&lt;human-activity-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14060" PostTypeId="2" ParentId="6934" CreationDate="2019-08-20T06:18:34.080" Score="0" Body="&lt;p&gt;In short, its basically a magic of matrix multiplications. The dimensions of the weight matrix are such that w.r.t the input layer dimensions, the desired output layer dimensions are taken care of.   &lt;/p&gt;&#xA;" OwnerUserId="28080" LastActivityDate="2019-08-20T06:18:34.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14061" PostTypeId="1" CreationDate="2019-08-20T10:01:51.837" Score="1" ViewCount="79" Body="&lt;p&gt;I have a dataset of 3D images (volumes) with dimensions 400x250x400. For each input image I have an output of the same dimensions. I would like to train a machine learning (or deep learning) model on this data in order to predict values with new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main problems are :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Images are very big, which leads to memory issues (tried with an NVIDIA 2080Ti and doesn't fit on memory during training)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a very fast inference, because the model will be used on real time(speed is a requirment)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have experience with architectures such as 3D Unet using Keras with tensorflow backend, but it didn't worked for me because of the previous reasons, even with very few layers and convolution filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that one of the first solutions that one could imagine, is to reduce resolution of the volumes, but in my case I'm not allowed this because I would lose a lot of spatial information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas or suggestions ? Maybe Neural Nets are not the best solution ? If not, what could I use ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your suggestions&lt;/p&gt;&#xA;" OwnerUserId="28085" LastActivityDate="2019-08-20T10:01:51.837" Title="Suggestions for Deep Learning for regression on huge 3D volumes" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14062" PostTypeId="2" ParentId="13999" CreationDate="2019-08-20T11:34:19.907" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do we get a three-dimensional output after a convolutional layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;During a search for an optimal convolution kernel via gradient descent or some other method there must be at least one additional dimension to represent trials.  It is most often one.  If the input is in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}^n$&lt;/span&gt; space, then the output of the convolution operation is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}^{n+1}$&lt;/span&gt; space.  However, this is not often (or ever) the output of the learning system using a convolutional layer, since the final layer in the conventional deep network designs used today is not the convolution layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case in this question, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; represents the number of discrete kernels tried, usually in rapid succession in most algorithms and hardware acceleration scenarios.  It is using the results in this &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}^{n+1}$&lt;/span&gt; space that the corrective mechanism requires to converge on an optimum efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By corrective mechanism is meant whatever corrects the assumptions made for the next set of kernels to be tried.  This mechanism often involves gradient descent and back propagation in an artificial network design.  It is the learning algorithm or, in the case of hardware acceleration, the learning circuit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is not arbitrary, but is largely problem dependent and based on hardware and execution environment.  If &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is too large, then too much convolution work is performed before the correction is made.  If &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is too small, then whatever efficiency is gained by grouping kernel tries together in time is lost.  There may be a formula to find &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, but it will be based on platform dependent metrics and can be found by trying several &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; values and determining the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; providing the lowest convergence time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; can also affect accuracy and reliability of convergence.  Predicting this effect is not straightforward.  Such prediction, which would allow the automated selection of hyper-parameters like &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, is of interest to many researchers for obvious reasons.  It has been and will probably continue to be an objective of AI development to remove the need for human intervention in AI system applications.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-20T11:34:19.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14064" PostTypeId="2" ParentId="13944" CreationDate="2019-08-20T12:28:34.353" Score="0" Body="&lt;p&gt;A somewhat large set of designs and set-ups can be made to learn a rating function for a given set of labeled examples.  If the objectives are simplicity and effectiveness (accuracy, reliability, and speed), then a third option should be considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The requirement in the question includes, &quot;Outputs an integer rating 0 [through] 4 [inclusive].&quot;  For such a discrete result, the number of required output bits &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the number of possible states and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is the set of integers) is given as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_b \, (b \in I \; \land \; b \ge \log_2 s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, we require three bits of output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$s = 5 \quad \implies \quad b = 3$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that with similar configuration ratings of 0 through 7 would also require only three bits of output.  Either way, the output layer would likely be simplest and most efficient if its activation function was binary step function.  This removes the need for rounding after it is applied.  The output layer would then provide a binary value indicating rating.  The goal of learning would be to reduce the error between the feed forward output and the associated the binary value of the label for each example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previous layer(s) could be sigmoid or a more contemporary and less problematic continuous activation function like ISRLU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the engineer can select the error function used by the learning framework to accept any input range and distribution, normalizing the labels for supervised learning is primarily employed to remove redundancy from time and resource consuming operations required to compute error.  With ratings as the labels, unless the distribution of ratings is skewed and the data set is such that learning time is excessive, normalization may not be necessary.  If it is, it would likely be because improving the label distribution in advance (requiring floating point input to the error function and removing skew) would reduce learning time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other two approaches introduce unnecessary complexities mentioned in context above.  A consequence of removing complexities without adding impediments to convergence is more efficiency during learning and during execution after learning.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-20T12:28:34.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14065" PostTypeId="2" ParentId="13865" CreationDate="2019-08-20T13:14:15.410" Score="0" Body="&lt;p&gt;&lt;strong&gt;Automation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes.  When the mechanism keeping the trigger from inadvertently springing the trap after it is set is merely the static friction of a tangential force to the direction of the trap force, we have the below, where &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is force, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is work, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the direction of the trap movement, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the direction of the trigger movement, and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the mechanical advantage gained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W_T = \int F_x \, dx \\&#xA;W_t = \int F_y \, dy \\&#xA;W_T = k \, W_t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not unusual for the friction of the trigger device to be deliberately reduced so that &lt;span class=&quot;math-container&quot;&gt;$k \ge 100$&lt;/span&gt;, meaning that the energy transferred from the animal to the trigger is no greater than 1% of the energy unleashed on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That the system has&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A detection device (the contact point of the trigger),&lt;/li&gt;&#xA;&lt;li&gt;A control function (the identity function),&lt;/li&gt;&#xA;&lt;li&gt;Energy storage (the consequence of the setting action), and&lt;/li&gt;&#xA;&lt;li&gt;A motor function (the trap release)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These qualify early traps not only as machines but as automatic one.  The human that set the trap does not perform the identity function.  The triggering design was performed in advance by humans, and even the purchaser of the trap need not understand it to set it and capture an animal.  This clearly fits the formal definition of automation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Computation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No. The identity function does not require computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deliberate Information Obscurity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the only historical or cultural aspect to the information science involved.  That the existence of the trigger and the action it triggers must be unknown to the animal is part of the information science involved.  The secrecy is part of the mechanism, and the secrecy may not involve hiding the trigger or the spring mechanism from the animal.  That the animal does not understand mechanical engineering may be ample secrecy for an animal that has not learned in DNA (by repeatedly eliminating lines of offspring that could not visually discern that something was amok) or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does the System Learn?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No.  It does not improve each time it is used.  However, learning traps have been designed and are deployed today.  The pray is the consumer in the case of marketing automation.  The pray is the cyber-enemy in the case of national infrastructure defense or attacks upon it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is Any Learning Involved in Mechanical Traps?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly.  For the earliest traps to be fabricated, much had to be learned in advance by humans, again, either parametrically in DNA or in the human brain through experience or through language and education.  These are just a few of many cognitive transitions that had to occur, represented as sentences.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Animals can be killed or maimed and then used as food.&lt;/li&gt;&#xA;&lt;li&gt;Animals can sense human presence and flee.&lt;/li&gt;&#xA;&lt;li&gt;Bending and releasing wood can propel somewhat like an arm can.&lt;/li&gt;&#xA;&lt;li&gt;The releasing can be suspended&lt;/li&gt;&#xA;&lt;li&gt;The animal can cause the release unintentionally&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is interesting to note that to artificially recreate the above learned understandings, both DNA and neurology were involved but not gradient descent, because the experiments and the analysis of results were discrete and no gradient could be calculated.  It is also unlikely that it was entirely trial and error in searching for better ways to get fed.  This is why logic based AI continues to be an important area of research.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-20T13:14:15.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14066" PostTypeId="2" ParentId="13799" CreationDate="2019-08-20T13:35:39.720" Score="0" Body="&lt;p&gt;The relationship between the axes of graph (1) and your variables &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is not clear, so this generalized answer may be helpful or useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From graph (1) it appears that the correlation coefficient &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; of a quadratic fit of data set &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; would be much better.  Consider &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt; approximations of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{C} (y_2, a, b, c, \mathcal{S}) &amp;gt; \mathcal{C} (y_1, a, b, \mathcal{S}) \\&#xA;y_2 = ax^2 + bx + c \\&#xA;y_1 = ax + b&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve a more nearly uniform distribution, perform a least squares fit for &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt; against &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; to obtain &lt;span class=&quot;math-container&quot;&gt;$(a, b, c)$&lt;/span&gt;.  Then find a mapping function that produces &lt;span class=&quot;math-container&quot;&gt;$y'$&lt;/span&gt; and use it where the uniform distribution is desired.  A reasonable approximation is simply this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y' = \frac{y}{y_2(x)}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-20T13:35:39.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14067" PostTypeId="2" ParentId="13885" CreationDate="2019-08-20T14:41:55.040" Score="1" Body="&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any intelligent system &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_a$&lt;/span&gt; with the set of adaptive features &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}_a$&lt;/span&gt;, there may exist another intelligent system &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_b$&lt;/span&gt; with the set of adaptive features &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}_b$&lt;/span&gt; such that there exists one element of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}_b$&lt;/span&gt; that can be made subservient (controlled in full) through the expression of at least one element in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}_b$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has not been proven that there ALWAYS exist such a &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_b$&lt;/span&gt;, but it is likely given what we know about escalation in nature via DNA and in human industrial development via innovation there.  Thus 100% generalized intelligence is not likely to exist.  Escalation appears to be the natural course of evolution.  And that is a feature of both cognitive and functional adaptation, with or without artificiality as a criterion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can temporarily prevent one adaptive system from escaping the boundary conditions of a particular set of boundary condition classes through the design and deployment of another adaptive system.  However, it cannot be inferred that any guarantees achieved temporarily will necessarily constrain the subservient system indefinitely.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-08-20T15:25:33.293" LastActivityDate="2019-08-20T15:25:33.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14068" PostTypeId="2" ParentId="13832" CreationDate="2019-08-20T15:20:32.270" Score="2" Body="&lt;p&gt;If the computational components of the forward feed through the network have no curvature, which is normally the case in a sum of products, then it can be proven that any constant pixel value is equivalent in terms of effect on convergence results.  We wouldn't expect a proof for that, since it would be too trivial to spend time writing up for publication.  In general, functioning vision systems have feed forward computational components with curvature, so the padding is likely significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even the convolutional layers may have activation functions or something even more complex going forward, as noted in &lt;a href=&quot;http://openaccess.thecvf.com/content_iccv_2017/html/Zoumpourlis_Non-Linear_Convolution_Filters_ICCV_2017_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gauge Equivariant Convolutional Networks and the Icosahedral CNN&lt;/a&gt; (Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling, 2019).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If purely stochastic values with value distributions like that of the un-padded coordinates are used, it may be possible to prove that some gain is made, but none appeared in a few academic article searches just made.  Not surprisingly, there are many proofs regarding the properties of various message padding strategies for cryptography.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short of the inclusion of thermal or quantum noise acquisition devices in VLSI circuitry and exposure of those devices in software, purely stochastic values cannot be generated.  This leaves the risk of a learning approach expected to extract features from frames learning features of the pseudo-random noise generator used to pad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is that none are universally correct and there appears to be much work to do in proving advantages between different techniques in as many cases as such advantages can be proven.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2019-08-20T15:20:32.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14069" PostTypeId="2" ParentId="9765" CreationDate="2019-08-20T15:38:56.263" Score="2" Body="&lt;p&gt;I think that the universal approximation theorem plays a large role in why companies and governments are investing in deep learning, it states that theoretically an ann can approximate any continuous function with n-dimension input variables. Specifically it states that feed forward nets with a single hidden layer can do this lending credence to the implication that rnns and cnns are also capable of universal function approximation. So they are investing because they have continuous functions that need to be approximated and really the best tool for the job is neural networks.  &lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-08-20T15:38:56.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14070" PostTypeId="2" ParentId="13646" CreationDate="2019-08-20T16:18:29.463" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What are the major differences between cost, loss, error, fitness, utility, objective, criterion functions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We can first consider what kind of elements are in this list.  They are all quantifications that guide a process of adaptation, learning, or innovation — controls a process that optimizes or attempts to optimize the development of another process.  Quantification allows for comparison and the determination of first and second-degree derivatives to use in approximating steps in improvement.&lt;/p&gt;&#xA;&lt;p&gt;On the basis of this generalization, the list may be expanded to include other potential quantifications that can control.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pain&lt;/li&gt;&#xA;&lt;li&gt;Pleasure&lt;/li&gt;&#xA;&lt;li&gt;Disparity&lt;/li&gt;&#xA;&lt;li&gt;Value&lt;/li&gt;&#xA;&lt;li&gt;Advantage&lt;/li&gt;&#xA;&lt;li&gt;Expense&lt;/li&gt;&#xA;&lt;li&gt;Profit&lt;/li&gt;&#xA;&lt;li&gt;Gain&lt;/li&gt;&#xA;&lt;li&gt;Return&lt;/li&gt;&#xA;&lt;li&gt;Distance&lt;/li&gt;&#xA;&lt;li&gt;Burden&lt;/li&gt;&#xA;&lt;li&gt;Optimality&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In English, there is quite a bit of nuance in terms of normal contextual use.  In academic papers on machine learning topics there are some conventions forming there, producing jargon, meanings beyond the normal English use.  We can first divide them symmetrically around the most basic qualifier.  Is the increase in the number what is intended or something to be avoided?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Decreases are Desirable&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cost&lt;/li&gt;&#xA;&lt;li&gt;Loss&lt;/li&gt;&#xA;&lt;li&gt;Error&lt;/li&gt;&#xA;&lt;li&gt;Distance&lt;/li&gt;&#xA;&lt;li&gt;Pain&lt;/li&gt;&#xA;&lt;li&gt;Disparity&lt;/li&gt;&#xA;&lt;li&gt;Expense&lt;/li&gt;&#xA;&lt;li&gt;Burden&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Increases are Desirable&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fitness&lt;/li&gt;&#xA;&lt;li&gt;Utility&lt;/li&gt;&#xA;&lt;li&gt;Objective&lt;/li&gt;&#xA;&lt;li&gt;Criterion&lt;/li&gt;&#xA;&lt;li&gt;Pleasure&lt;/li&gt;&#xA;&lt;li&gt;Value&lt;/li&gt;&#xA;&lt;li&gt;Advantage&lt;/li&gt;&#xA;&lt;li&gt;Profit&lt;/li&gt;&#xA;&lt;li&gt;Gain&lt;/li&gt;&#xA;&lt;li&gt;Optimality&lt;/li&gt;&#xA;&lt;li&gt;Complete&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Other Features&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some of these are relative, such as Advantage being a relative measure between two Values associated with two Action options.  Some are direct inversions, as in that the quotient of Gain over Loss equals -1 in the usual contexts of those words.  Some appear to be opposite but in fact are not, as in the case of pain and pleasure occurring concurrently and via different neural mechanisms in biological neural systems.&lt;/p&gt;&#xA;&lt;p&gt;Some are discrete and have only two possible values, such as Criterion and Complete.  Those can be well represented by Boolean values rather than integers, fixed-point, or floating-point numbers.&lt;/p&gt;&#xA;&lt;p&gt;Some are conditionally synonymous, as in the case of Distance and Error if the output dimensionality is one or the output is multidimensional but the error aggregation method used is the square root of the sum of squares.&lt;/p&gt;&#xA;&lt;p&gt;Some seem better in financial contexts ... others in robotic contexts.&lt;/p&gt;&#xA;&lt;p&gt;Words like Disparity and Optimality are generalizations.&lt;/p&gt;&#xA;&lt;p&gt;One, in particular, is a quotients based on investment &amp;amp;mdash Return — which could be applied to energy return on energy invested (as Hall suggests is the fitness function for genetic evolution).&lt;/p&gt;&#xA;&lt;p&gt;What is missing from this list is learning based on multiple degrees of freedom, not used in mainstream ML yet, and probably will emerge in papers and then libraries and then VLSI designs in time.  This likely direction would introduce the plurals of many of these quantifications because it will be vectors of corrective signaling back-propagating to perform curved modifications on network parameters.  It is likely that brain signaling (pulses, energy balances, neurochemical secretions and uptake, and genetic expression) will not be well simulated until this happens.&lt;/p&gt;&#xA;&lt;p&gt;In all cases, it is the definition of these quantifications that drive adaptation, learning, and innovation in biological and artificial systems.  It is thus likely that the application of ethics and risk analysis might best be applied to AI in the formulation of the calculation of these quantities in adaptive systems.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="36737" LastEditDate="2021-04-04T15:13:58.527" LastActivityDate="2021-04-04T15:13:58.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14071" PostTypeId="2" ParentId="13376" CreationDate="2019-08-20T20:11:24.327" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://www.rottentomatoes.com/m/alphago&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; (2017) is quite a good watch, given that it is a documentary about the AlphaGo program, how DeepMind developed it, the help they had, and doesn't get too technical.  You can watch the trailer &lt;a href=&quot;https://www.youtube.com/watch?v=8tq1C8spV_g&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another documentary which wasn't exactly AI but something that is an interesting watch or at least was when it came out was &lt;a href=&quot;https://www.youtube.com/watch?v=OV1y6ZUV_Q4&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Human Face of Big Data&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2019-08-20T23:14:15.373" LastActivityDate="2019-08-20T23:14:15.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14073" PostTypeId="1" CreationDate="2019-08-20T21:02:56.617" Score="2" ViewCount="29" Body="&lt;p&gt;Are there methods (possibly logical or (how they are called in the literature) relational) that allows for the developmental systems to understand or explain the value of the received reward during the developmental process. E.g. if the system (agent) can understand that reward is by the chance, then it should be processed quite differently than the reward that is just initial downpayment for the expected series of rewards. &lt;strong&gt;&lt;em&gt;Credit assignment&lt;/em&gt; is the one method (especially for delayed rewards), but maybe there are different methods as well?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relational reinforcement learning allows to learn symbolic transition and reward functions and emerging understanding of the reward by the agent can greatly facilitate the inner consciousness of the agent and the search process for the best transition and reward functions (symbolic search space can be enormous).&lt;/p&gt;&#xA;" OwnerUserId="8332" LastActivityDate="2019-08-20T21:02:56.617" Title="Developmental systems that try to explain or understand the reward value in the reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;logic&gt;&lt;artificial-consciousness&gt;&lt;rewards&gt;&lt;symbolic-ai&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14075" PostTypeId="2" ParentId="6333" CreationDate="2019-08-21T01:27:04.753" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Who stands a chance against Google in the AI race?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The Google &lt;a href=&quot;https://cloud.google.com/blog/products/ai-machine-learning/what-makes-tpus-fine-tuned-for-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;TPU&lt;/a&gt; is a &lt;strong&gt;GPU sized chip&lt;/strong&gt; developed for deep learning, it is a matrix processor specialized for neural network work loads.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The latest-generation Cloud &lt;a href=&quot;https://cloud.google.com/blog/products/ai-machine-learning/googles-scalable-supercomputers-for-machine-learning-cloud-tpu-pods-are-now-publicly-available-in-beta&quot; rel=&quot;nofollow noreferrer&quot;&gt;TPU v3 Pods&lt;/a&gt; (more than 1,000 individual TPU chips) are liquid-cooled for maximum performance, and each one delivers more than 100 petaFLOPs of computing power. In terms of raw mathematical operations per second, a Cloud TPU v3 Pod is comparable with a top 5 supercomputer worldwide (though it operates at lower numerical precision).&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VbO0C.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VbO0C.jpg&quot; alt=&quot;Google TPUs mounted on a re&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.cerebras.net/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cerebras&lt;/a&gt; 16nm TSMC wafer sized chip is a whopping 46,225mm² die that consumes 15kW and packs 400,000 cores, it's said to &lt;a href=&quot;https://www.eetimes.com/document.asp?doc_id=1335043&amp;amp;page_number=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;deliver the performance&lt;/a&gt; of a farm of a thousand Nvidia GPUs that can take months to assemble while requiring just 2-3% of its space and power.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The Cerebras &lt;a href=&quot;https://www.forbes.com/sites/moorinsights/2019/08/19/cerebras-unveils-ai-supercomputer-on-a-chip/#7218cf482fe8&quot; rel=&quot;nofollow noreferrer&quot;&gt;device&lt;/a&gt; packs 84 tiles in a 7x12 array. Each includes about 4,800 cores geared for AI’s sparse linear algebra with 48 KBytes SRAM each (a total of 18 GB), their sole memory source.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;9 Petabytes/s memory bandwidth&lt;/li&gt;&#xA;&lt;li&gt;100 Petabits/s fabric bandwidth&lt;/li&gt;&#xA;&lt;li&gt;Native optimization for sparsity (to avoid multiplying by zero)&lt;/li&gt;&#xA;&lt;li&gt;Software compatibility with the standard AI Frameworks such as TensorFlow and PyTorch&lt;/li&gt;&#xA;&lt;li&gt;Cerebras says it is currently working with major customers to evaluate early silicon, and that it hopes to ship production servers using its WSE by mid-2020&amp;quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For comparison the Nvidia Tesla V100 is 815mm² with 21.1 billion transistors, while Cerabras chip is 46,225mm² with 1.2 trillion transistors, that's 50x bigger.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Xbup.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Xbup.jpg&quot; alt=&quot;Chip size Comparison&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yes, it's &lt;em&gt;&lt;strong&gt;one&lt;/strong&gt;&lt;/em&gt; chip; they don't slice the wafer.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-08-21T01:27:04.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14076" PostTypeId="1" CreationDate="2019-08-21T02:21:47.937" Score="1" ViewCount="78" Body="&lt;p&gt;We are exploring the images classified by a CNN at its decision boundary, using Genetic Algorithms to generate them. We have created a fine-tuned binary grayscale image classifier for cats. As the base model, we are using an Inception-ResNet v2 pre-trained on the ImageNet dataset, and then fine-tune it with a subset of cat and non-cat images (grayscale) from ImageNet. The model achieves ~97% accuracy for a test set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have constrained the problem such that evolution starts from a pure white image, and random crossover and mutations are performed with only black pixels. Crossover and mutation probabilities are kept at 0.8 and 0.015 respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an incentive to generate a &quot;cat&quot; with the minimum number of black pixels, I add a penalty for the black pixel count in the image. The initial population is a set of 100 white images that have a single random pixel coloured black in them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evolution generates images with only black and white pixels, and we have a fitness function that is taken as a linear transformation of loss calculated between target label and network prediction as follows;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;loss = binary cross entropy (target, prediction) + λ(# of black pixels)&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Target value (y)    = target label cat - in this case, 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;λ                              = hyperparameter to weight the penalty for black pixel count.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;My problem is that across multiple runs of evolution, all images classified as &lt;strong&gt;cats tend to have black pixels towards the edges&lt;/strong&gt; of the image. Below is an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B64Wc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B64Wc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image is classified as a cat with over 96% confidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried different crossover mechanisms including&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Random rectangular area swap between parents &lt;/li&gt;&#xA;&lt;li&gt;Alternating column interchange &lt;/li&gt;&#xA;&lt;li&gt;Direct black pixel crossover after encoding the image to&#xA;a reduced form that only kept track of the black pixels (black pixel&#xA;list is the genome)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Initially, we ran evolution with a similarly fine-tuned VGG-16 model, and then moved to the Inception ResNet due to better accuracy. &lt;strong&gt;Pixels tend to edges across models and crossover mechanisms.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one run, I explicitly constrained the evolution to perform mutations in the middle section of the images for 3,000 generations before lifting this restriction. But the images generated after that point always had better scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are at a loss as to why the images never have pixels coloured in the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any ideas on this? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="27919" LastActivityDate="2019-08-21T02:21:47.937" Title="CNN - Visualizing images near decision boundary - Pixels inexplicably tend to edges" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14077" PostTypeId="1" CreationDate="2019-08-21T06:58:47.830" Score="1" ViewCount="24" Body="&lt;p&gt;I am in the process of collecting a huge dataset of Human poses captured images to create a model to classify poses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how will I be able to train on this massive dataset? I have multiple GPUs and Multiple machines access (Also have GCP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to train on such huge dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="6242" LastActivityDate="2019-08-24T17:50:01.357" Title="Train on big dataset (1mil + images)" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14079" PostTypeId="1" CreationDate="2019-08-21T08:33:19.477" Score="3" ViewCount="934" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7FnY6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7FnY6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to create a simple model that receives an &lt;span class=&quot;math-container&quot;&gt;$80 \times 130$&lt;/span&gt; pixel image. I only had 35 images and 10 test images. I trained this model for a binary classification task. The architecture of the model is described below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv2d_1 (Conv2D)            (None, 80, 130, 64)       640       &#xA;_________________________________________________________________&#xA;conv2d_2 (Conv2D)            (None, 78, 128, 64)       36928     &#xA;_________________________________________________________________&#xA;max_pooling2d_1 (MaxPooling2 (None, 39, 64, 64)        0         &#xA;_________________________________________________________________&#xA;dropout_1 (Dropout)          (None, 39, 64, 64)        0         &#xA;_________________________________________________________________&#xA;conv2d_3 (Conv2D)            (None, 39, 64, 128)       73856     &#xA;_________________________________________________________________&#xA;conv2d_4 (Conv2D)            (None, 37, 62, 128)       147584    &#xA;_________________________________________________________________&#xA;max_pooling2d_2 (MaxPooling2 (None, 18, 31, 128)       0         &#xA;_________________________________________________________________&#xA;dropout_2 (Dropout)          (None, 18, 31, 128)       0         &#xA;_________________________________________________________________&#xA;flatten_1 (Flatten)          (None, 71424)             0         &#xA;_________________________________________________________________&#xA;dense_1 (Dense)              (None, 512)               36569600  &#xA;_________________________________________________________________&#xA;dropout_3 (Dropout)          (None, 512)               0         &#xA;_________________________________________________________________&#xA;dense_2 (Dense)              (None, 1)                 513     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What could the oscillating training loss curve represent above? Why is the validation loss constant?&lt;/p&gt;&#xA;" OwnerUserId="28100" LastEditorUserId="2444" LastEditDate="2019-08-22T23:04:22.170" LastActivityDate="2019-08-30T04:07:39.257" Title="What could an oscillating training loss curve represent?" Tags="&lt;training&gt;&lt;computer-vision&gt;&lt;deep-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14081" PostTypeId="1" AcceptedAnswerId="14083" CreationDate="2019-08-21T09:34:52.867" Score="2" ViewCount="479" Body="&lt;p&gt;I am trying to have the type of optimizer as a variable in the hyperparameter tuning phase. For that reason I am trying to use the &lt;code&gt;tf.keras.optimizer.get&lt;/code&gt; function defined in &lt;a href=&quot;https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/keras/optimizers.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/keras/optimizers.py&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am getting different kind of errors depending on how I am calling the function and I cannot figure out the problem. For example, when I run the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opt = tf.keras.optimizers.get({&quot;class_name&quot;: hyperparams['optimizer'],&#xA;                                      &quot;learning_rate&quot;: hyperparams['learning_rate']})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get an error saying &lt;code&gt;&quot;ValueError: Improper config format: {'class_name': 'adagrad', 'learning_rate': 0.01} &quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know what is the proper way of calling this function with a configuration dictionary if I want to have a configurable optimizer type and learning rate?&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2019-08-21T13:14:10.060" Title="Create optimizer object using the tf.keras.optimizers.get function" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-03-13T04:04:44.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="14082" PostTypeId="1" CreationDate="2019-08-21T12:02:23.180" Score="5" ViewCount="109" Body="&lt;p&gt;Is it possible to specify what the asymptotic behaviour of a Neural Networks (NN) model should be?&lt;/p&gt;&#xA;&lt;p&gt;I am thinking of a NN which tries to learn a mapping &lt;span class=&quot;math-container&quot;&gt;$\vec y=f(\vec x)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\vec x$&lt;/span&gt; a vector of features of dimension &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec y$&lt;/span&gt; a vector of outputs of dimension &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to specify that, for instance, the NN should have a fixed value when &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; goes to infinite?&lt;/p&gt;&#xA;&lt;p&gt;I mean:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\lim_{x_1\to \infty} f(\vec x) = \vec c&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If it is not possible with NN, do you know other machine learning models (for instance  Gaussian Process Regression or Support Vector Regression) which have a known asymptotic behaviour?&lt;/p&gt;&#xA;" OwnerUserId="28108" LastEditorUserId="2444" LastEditDate="2021-03-29T13:55:09.760" LastActivityDate="2021-03-29T18:55:13.197" Title="Is it possible to control asymptotic behaviour of neural network models?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computational-learning-theory&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14083" PostTypeId="2" ParentId="14081" CreationDate="2019-08-21T13:14:10.060" Score="2" Body="&lt;p&gt;In order to have a configurable optimizer and configurable hyperparameters for it you need to make the function call in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opt = tf.keras.optimizers.get({&quot;class_name&quot;: hyperparams['optimizer'],&#xA;                               &quot;config&quot;: {&quot;learning_rate&quot;: hyperparams['learning_rate']}})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;class_name&lt;/code&gt; is the name of the optimizer you want to use and the &lt;code&gt;config&lt;/code&gt; dictionary contains the hyperparameters and their values for the given optimizer.&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2019-08-21T13:14:10.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14084" PostTypeId="1" CreationDate="2019-08-21T14:45:09.843" Score="1" ViewCount="12" Body="&lt;p&gt;I have a general question regarding the &lt;code&gt;mAP&lt;/code&gt; score used in measuring object detection system performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understood how the &lt;code&gt;AP&lt;/code&gt; score is calculated, by averaging precision over recall 0 to 1. And then we can compute &lt;code&gt;mAP&lt;/code&gt;, by averaging &lt;code&gt;AP&lt;/code&gt; score of different labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what I have been really confused, is that, it seems that &lt;code&gt;mAP&lt;/code&gt; score is used to denote the &quot;precision&quot; of a model. Then what about the &quot;recall&quot; aspect? Note that generally speaking, when measuring the performance of a machine learning model, we need to report &lt;code&gt;precision&lt;/code&gt; and &lt;code&gt;recall&lt;/code&gt; at the same time, right? It seems that &lt;code&gt;mAP&lt;/code&gt; can only cover the &lt;code&gt;precision&lt;/code&gt; aspect of a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missed anything here? Or &lt;code&gt;mAP&lt;/code&gt; score, despite its name is derived from &lt;code&gt;Precision&lt;/code&gt;, can indeed subsume both &quot;precision&quot; and &quot;recall&quot; and therefore become comprehensive enough? &lt;/p&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-08-21T14:45:09.843" Title="Can mAP score be used to describe &quot;recall&quot; rate of a model?" Tags="&lt;object-detection&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14085" PostTypeId="1" AcceptedAnswerId="14088" CreationDate="2019-08-21T16:24:11.617" Score="2" ViewCount="772" Body="&lt;p&gt;I mean what is determine my model size, connection amount between layers and neurons, or size of my dataset? &lt;/p&gt;&#xA;" OwnerUserId="28061" LastActivityDate="2019-08-21T18:23:38.073" Title="Is &quot;dataset size&quot; and &quot;model size&quot; same thing?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;models&gt;&lt;artificial-neuron&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14086" PostTypeId="2" ParentId="14079" CreationDate="2019-08-21T16:41:58.217" Score="2" Body="&lt;p&gt;Try lowering the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a loss curve can be indicative of a high learning rate. Due to a high learning rate the algorithm can take large steps in the direction of the gradient and miss the local minima. Then it will try to come back to the minima in the next step and overshoot it again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also try switching to a momentum-based GD algorithm. Such a training loss curve can be indicative of a loss contour like in &lt;a href=&quot;http://ruder.io/optimizing-gradient-descent/index.html#momentum&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;, for which momentum-based GD methods are helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that you have a very small training set. You may have better luck with a larger training data (~1000 examples) or using a pre-trained Conv network as a starting point.&lt;/p&gt;&#xA;" OwnerUserId="20799" LastActivityDate="2019-08-21T16:41:58.217" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14088" PostTypeId="2" ParentId="14085" CreationDate="2019-08-21T18:23:38.073" Score="0" Body="&lt;p&gt;Dataset and model refer to different things. Dataset means part of data available for training (training dataset) or validation (validation dataset). Model is the learning process goal, the state of the computer &quot;brain&quot; after it has been fully educated (or made its learning). Model size refers to size of the container which contains the model. In deep learning it can be measured by width and depth of the network used, I also found a &lt;a href=&quot;https://www.deeplearningmodel.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt; comparing different models by npy file size, that physically contains the generated model as computer code. In that case model contained a more complex structure which was documented and size in bytes was for comparison purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in short, it is roughly speaking the size of layers and neurons, if I have to take one of your options. Dataset is a different thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precise explanation about what is model and what is dataset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/What-are-different-models-in-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/What-are-different-models-in-machine-learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2019-08-21T18:23:38.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14090" PostTypeId="1" CreationDate="2019-08-22T00:41:39.297" Score="3" ViewCount="296" Body="&lt;p&gt;There are many known ways to overcome overfitting or make a model generalize better to unseen data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I would like to ask if normalizing/standardizing/similiraizing the train and test data is a plausible approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By similarizing I mean making the images look alike by using some function that could be a Neural Network itself. I know that normally one would approach this the opposite way by augmenting and therefore increasing the variation in the training data. But is also possible to improve the model by restricting the variation of the training and test data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this may not be the best approach and maybe too complicated but I see some use cases where known techniques of preventing overfitting aren't applicable. In those cases, having a network that can normalize/standardize/similarize the &quot;style&quot; of different images could be very useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I didn't find a single paper discussing this approach.&lt;/p&gt;&#xA;" OwnerUserId="23063" LastActivityDate="2019-09-21T10:01:00.443" Title="Is normalizing the data a way to improve generalization?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14091" PostTypeId="2" ParentId="14090" CreationDate="2019-08-22T09:23:25.067" Score="1" Body="&lt;p&gt;Batch Normalization is usually known to speed up the learning process as it makes the weights in the deeper layers more robust. It restricts the distribution of the weights in a particular layer - &lt;a href=&quot;https://www.youtube.com/watch?v=nUUqwaxLnWs&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; video might tend to be useful to what BatchNorm does. This said, batch normalization does have a regularizing effect which does to tend to increase generalization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Talking about generalization - A focus on regularization would probably be more helpful &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-08-22T09:23:25.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14092" PostTypeId="2" ParentId="14028" CreationDate="2019-08-22T09:55:53.633" Score="0" Body="&lt;p&gt;My implementation to increase detection on video is using object tracking algorithms.&lt;/p&gt;&#xA;&lt;p&gt;More specifically, first, I detect the object using a trained classifier. Then I track the object with the KCF algorithm. If the object tracker misses the object, again I call for the classifier.&lt;/p&gt;&#xA;" OwnerUserId="28129" LastEditorUserId="2444" LastEditDate="2020-09-07T22:32:32.743" LastActivityDate="2020-09-07T22:32:32.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14094" PostTypeId="1" AcceptedAnswerId="14104" CreationDate="2019-08-22T14:03:31.313" Score="2" ViewCount="55" Body="&lt;p&gt;I am wondering why tf object detection api needs so few picture samples for training while regular cnns needs many more?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I read in tutorials is that tf object detection api needs around 100-500 pictures per class for training (is it true?) while regular CNNs need many many more samples, like tens of thousands or more. Why is it so?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-08-23T12:16:34.817" Title="Why tf object detection api needs so few pictures?" Tags="&lt;tensorflow&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14096" PostTypeId="1" CreationDate="2019-08-22T19:50:45.020" Score="2" ViewCount="40" Body="&lt;p&gt;We all know that using CNN, or even simpler functions, like CLD or EHD, we can generate a set of features out of images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any ways or approaches that given a set of features, we can somehow generate a corase version of the original image that was given as input? Maybe a gray-scale version with visible objects inside? If so, what features do we need?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-08-22T20:08:56.060" LastActivityDate="2019-08-22T21:12:31.370" Title="How to generate the original image from feature set?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;generative-adversarial-networks&gt;&lt;image-processing&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14097" PostTypeId="2" ParentId="14096" CreationDate="2019-08-22T21:12:31.370" Score="1" Body="&lt;p&gt;The model (that I know of) which most resembles your description is the &lt;em&gt;auto-encoder&lt;/em&gt;, which is trained to learn a compact representation (a vector) of the input, which can later be used to reconstruct the original input. In a certain way, this compact representation (implicitly) encodes the most important features of the input. In particular, you may be looking for &lt;a href=&quot;https://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;denoising auto-encoders&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-08-22T21:12:31.370" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14098" PostTypeId="1" CreationDate="2019-08-22T22:10:12.020" Score="8" ViewCount="182" Body="&lt;p&gt;FaceNet uses a novel loss metric (triplet loss) to train a model to output embeddings (128-D from the paper), such that any two faces of the same identity will have a small Euclidean distance, and such that any two faces of different identities will have a Euclidean distance larger than a specified margin. However, it needs another mechanism (HOG or MTCNN) to detect and extract faces from images in the first place.&lt;/p&gt;&#xA;&lt;p&gt;Can this idea be extended to object recognition? That is, can an object detection framework (e.g. MaskR-CNN) be used to extract bounding boxes of an object, cropping the object feeding this to a network that was trained on triplet loss, and then compares the embeddings of objects to see if they’re the same object?&lt;/p&gt;&#xA;&lt;p&gt;Is there any research that has been done or any published public datasets for this?&lt;/p&gt;&#xA;" OwnerUserId="28145" LastEditorUserId="43231" LastEditDate="2021-02-19T12:30:43.140" LastActivityDate="2021-03-21T14:02:35.227" Title="Extending FaceNet’s triplet loss to object recognition" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;object-detection&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14099" PostTypeId="1" AcceptedAnswerId="14100" CreationDate="2019-08-23T02:34:05.960" Score="1" ViewCount="64" Body="&lt;p&gt;The ready-to-use DNNClassifier in tf.estimator seems not able to fit these data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = [[1,2], [1,12], [1,17], [9,33], [48,49], [48,50]]&#xA;Y = [ 1,     1,      1,      1,      2,       3     ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tried with 4 layers but it's fitting to 83% (=5/6 sampes) only:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hidden_units = [2000,1000,500,100]&#xA;n_classes    = 4   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The sample data above are supposed to be separated by 2 lines (right-click image to open in new tab):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XT3OJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XT3OJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems stuck be cause of Y=2 and Y=3 are too close. How to change the DNNClassifier to fit to 100%?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-08-23T08:23:10.400" Title="TensorFlow estimator DNNClassifier fails to fit simple data" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;linear-regression&gt;&lt;underfitting&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14100" PostTypeId="2" ParentId="14099" CreationDate="2019-08-23T07:41:43.830" Score="2" Body="&lt;p&gt;&lt;strong&gt;Normalise your inputs.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks work poorly outside of relatively small numerical ranges on input. An ideal range is for each feature to be drawn from &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(0,1)$&lt;/span&gt; i.e. a Normal distribution with mean &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and standard deviation &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. In your case, divide both parts of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$25$&lt;/span&gt; and subtract &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; would probably suffice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your neural network architecture is completely overblown for the problem at hand. That may be because you were trying to force it to fit this data (and failing because of lack of normalisation). Try something more like: &lt;code&gt;hidden_units = [20,10]&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-23T08:23:10.400" LastActivityDate="2019-08-23T08:23:10.400" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14101" PostTypeId="1" CreationDate="2019-08-23T09:31:13.053" Score="4" ViewCount="161" Body="&lt;p&gt;I'm a computer scientist who's studying support vector machines (SVMs) in a machine learning course. I have some understanding of how SVMs are designed, thanks to &lt;a href=&quot;https://www.youtube.com/watch?v=_PwhiWxHK8o&quot; rel=&quot;nofollow noreferrer&quot;&gt;16. Learning: Support Vector Machines - MIT&lt;/a&gt;. However, what I'm not understanding is the transition from the optimization problem of the Lagrangian function to its implementation in any programming language. Basically, what I need to understand is how to build, from scratch, the decision function, given a training set. In particular, how do I find Lagrange multipliers in order to know which points are to be considered to define support vectors and the decision function?&lt;/p&gt;&#xA;&lt;p&gt;Can anyone explain this to me?&lt;/p&gt;&#xA;" OwnerUserId="28151" LastEditorUserId="36057" LastEditDate="2021-04-29T12:19:27.547" LastActivityDate="2021-05-01T14:11:04.967" Title="How to implement SVM algorithm from scratch in a programming language?" Tags="&lt;machine-learning&gt;&lt;implementation&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14102" PostTypeId="1" AcceptedAnswerId="14108" CreationDate="2019-08-23T11:08:53.977" Score="0" ViewCount="33" Body="&lt;p&gt;I have a bunch of training data for classifying product names, around &lt;strong&gt;30,000 samples&lt;/strong&gt;. The task is to classify these product names into types of product, around &lt;strong&gt;100 classes&lt;/strong&gt; (single words).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dutch lady sweetened uht milk =&amp;gt; milk&#xA;samsung galaxy note 10        =&amp;gt; electronics&#xA;cocacola zero                 =&amp;gt; softdrink&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All words in inputs are indexed to numbers, and so classes. I've tried to use tf.estimator.&lt;strong&gt;DNNClassifier&lt;/strong&gt; to classify them but no good results. The outcome is just an &lt;strong&gt;accuracy of 4%&lt;/strong&gt; which is no meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should it be I'm in the case that classes (Y values) are distributed kinda randomly and too hard to do multi-time linear separation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any existing solutions to classify a list of names? like my product names?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-08-23T21:53:55.483" Title="Solution to classify product names" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;linear-regression&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14103" PostTypeId="1" CreationDate="2019-08-23T12:09:00.257" Score="1" ViewCount="78" Body="&lt;p&gt;If we use LSTMCell from torch:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Model:&#xA;&#xA;def __init__(...):&#xA;    self.lstm_h0 = torch.randn(1, hidden_size, requires_grad=False)&#xA;    self.lstm_c0 = torch.randn(1, hidden_size, requires_grad=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Let's say our problem is we want to count the number of balls in a video and run an LSTM over each frame of the video. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.lstm_cell = LSTMCell(...)&#xA;&#xA;for i in range(num_frames):&#xA;    h, c = self.lstm_cell(frame[i], (h, c))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The final output embedding we use (to then project into a scalar 'ball count') is the final &quot;h&quot; hidden output, right?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have (N, nB, nE) dimensional data, call it &quot;my_data&quot;. N is the batch dimension, nB is an object dimension, and nE is an embedding dimension. I want to use LSTM to perform dimensionality reduction over nB. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(nB):&#xA;    h, c = self.lstm_cell(my_data[:, i, :], (h, c))&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is this the correct way to do it?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="16565" LastEditDate="2019-08-28T17:17:20.997" LastActivityDate="2019-08-28T17:17:20.997" Title="Initial LSTM hidden state and cell" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14104" PostTypeId="2" ParentId="14094" CreationDate="2019-08-23T12:16:34.817" Score="2" Body="&lt;p&gt;I guess that they need so little data because their models are already trained on huge datasets, and they are just transferring the learning (using those pre-trained models as starting point).&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-08-23T12:16:34.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14105" PostTypeId="2" ParentId="13978" CreationDate="2019-08-23T19:56:13.083" Score="5" Body="&lt;p&gt;Consider a dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}=\{x^{(i)},y^{(i)}:i=1,2,\ldots,N\}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x^{(i)}\in\mathbb{R}^3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y^{(i)}\in\mathbb{R}$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\forall i$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to fit a function that best explains our dataset.We can fit a simple function, as we do in &lt;strong&gt;linear regression&lt;/strong&gt;. But that's different about neural networks, where we fit a complex function, say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\begin{align}h(x) &amp;amp; = h(x_1,x_2,x_3)\\&#xA;&amp;amp; =\sigma(w_{46}\times\sigma(w_{14}x_1+w_{24}x_2+w_{34}x_3+b_4)+w_{56}\times\sigma(w_{15}x_1+w_{25}x_2+w_{35}x_3+b_5)+b_6)\end{align}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, &lt;span class=&quot;math-container&quot;&gt;$\theta = \{w_{14},w_{24},w_{34},b_4,w_{15},w_{25},w_{35},b_5,w_{46},w_{56},b_6\}$&lt;/span&gt; is the set of the respective coefficients we have to determine such that we minimize:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$J(\theta) = \frac{1}{2}\sum_{i=1}^N (y^{(i)}-h(x^{(i)}))^2$$&lt;/span&gt;&#xA;The above optimization problem can be easily solved with &lt;strong&gt;&lt;em&gt;gradient descent&lt;/em&gt;&lt;/strong&gt;. Just initiate &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; with random values and with proper learning parameter &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;, update as follows till convergence:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\theta:=\theta-\eta\frac{\partial J}{\partial \theta}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to get the gradients, we express the above function as a neural network as follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rf7db.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rf7db.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's calculate the gradient, say w.r.t. &lt;span class=&quot;math-container&quot;&gt;$w_{14}$&lt;/span&gt;.&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial J}{\partial w_{14}} = \sum_{i=1}^N \Big[\big(h(x^{(i)})-y^{(i)}\big)\frac{\partial h(x^{(i)})}{\partial w_{14}}\Big]$$&lt;/span&gt;&#xA;Let &lt;span class=&quot;math-container&quot;&gt;$p(x) = w_{14}x_1+w_{24}x_2+w_{34}x_3+b_4$&lt;/span&gt; , and&lt;br&gt;&#xA;Let &lt;span class=&quot;math-container&quot;&gt;$q(x) = w_{46}\times\sigma(p(x))+w_{56}\times\sigma(w_{15}x_1+w_{25}x_2+w_{35}x_3+b_5)+b_6)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\therefore \frac{\partial h(x)}{\partial w_{14}} = \frac{\partial h(x)}{\partial q(x)}\times\frac{\partial q(x)}{\partial p(x)}\times\frac{\partial p(x)}{\partial w_{14}} = \frac{\partial\sigma(q(x))}{\partial q(x)}\times\frac{\partial\sigma(p(x))}{\partial p(x)}\times\frac{\partial p(x)}{\partial w_{14}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We see that the derivative of the activation function is important for getting the gradients and so for the learning of the neural network. A constant derivative will not help in the gradient descent and we won't be able to learn the optimal parameters.&lt;/p&gt;&#xA;" OwnerUserId="28058" LastEditorUserId="28058" LastEditDate="2019-08-24T05:08:08.457" LastActivityDate="2019-08-24T05:08:08.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14107" PostTypeId="1" CreationDate="2019-08-23T21:19:55.670" Score="5" ViewCount="60" Body="&lt;p&gt;Suppose we have a labeled data set with columns &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; and a binary outcome variable &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. Suppose we have rows as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; col  A B C X&#xA;  1   1 2 3 1&#xA;  2   4 2 3 0&#xA;  3   6 5 1 1&#xA;  4   1 2 3 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should we throw away either row 1 or row 4 because they have different values of the outcome variable X? Or keep both of them?&lt;/p&gt;&#xA;" OwnerUserId="28161" LastEditorUserId="16708" LastEditDate="2019-09-04T09:36:30.033" LastActivityDate="2019-09-04T09:36:30.033" Title="What should we do when we have equal observations with different labels?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;data-science&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14108" PostTypeId="2" ParentId="14102" CreationDate="2019-08-23T21:53:55.483" Score="1" Body="&lt;p&gt;If you're looking for an existing solution, the best approach I found was using a TF-IDF model, check out the links below which have similar examples which should be easily adapted for your dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.kaggle.com/selener/multi-class-text-classification-tfidf#targetText=Text%20classification%20(multiclass)&amp;amp;targetText=With%20the%20aim%20to%20classify,one%20of%20the%20product%20categories&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/selener/multi-class-text-classification-tfidf#targetText=Text%20classification%20(multiclass)&amp;amp;targetText=With%20the%20aim%20to%20classify,one%20of%20the%20product%20categories&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/susanli2016/Machine-Learning-with-Python/blob/master/Consumer_complaints.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/susanli2016/Machine-Learning-with-Python/blob/master/Consumer_complaints.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if you specifically want to go for a DNN approach, there are a few options you can take for a multi-class text classification. Try looking into a simple CNN classifier, which is a relatively lightweight approach, computationally speaking, yet showing pretty good results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can use a word2vec or a doc2vec model to map your sentences to unique vectors, and then put them through a regression algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/multi-class-text-classification-model-comparison-and-selection-5eb066197568&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/multi-class-text-classification-model-comparison-and-selection-5eb066197568&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28160" LastActivityDate="2019-08-23T21:53:55.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14109" PostTypeId="2" ParentId="14107" CreationDate="2019-08-24T00:13:55.467" Score="4" Body="&lt;p&gt;The problem you are portraying looks like a modified &lt;a href=&quot;https://medium.com/@jayeshbahire/the-xor-problem-in-neural-networks-50006411840b&quot; rel=&quot;nofollow noreferrer&quot;&gt;XOR problem&lt;/a&gt;. You can't throw away the lines with a label of 1 because a the model won't be able to learn this class.&lt;/p&gt;&#xA;" OwnerUserId="28162" LastActivityDate="2019-08-24T00:13:55.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14110" PostTypeId="2" ParentId="13538" CreationDate="2019-08-24T00:31:58.783" Score="4" Body="&lt;p&gt;I don't think that the &quot;try all the numbers&quot; approach is very representative, because I'm not sure whether or not the agent that uses that approach can be considered by any means AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no &quot;intelligence&quot; in just checking numbers to try to prove the statement. An agent that is considered to be &lt;em&gt;intelligent&lt;/em&gt; should apply a more &lt;em&gt;intelligent&lt;/em&gt; approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This becomes more evident because the question aims at exploiting the lack of scalability of the agent's strategy. If the question was &quot;Prove that no number exists which is one more than 5&quot;, then the agent would have no trouble in finding the correct answer.&lt;/p&gt;&#xA;" OwnerUserId="28163" LastActivityDate="2019-08-24T00:31:58.783" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14111" PostTypeId="2" ParentId="4095" CreationDate="2019-08-24T05:57:17.783" Score="0" Body="&lt;p&gt;MCTS only need to &quot;see&quot; states in respect of reward. All game mechanics is abstarcted away from MCTS  and MCTS only access actions and rewards. MCTS player don't access states itself, it's only choose action according to backpropagated reward. For partially observed MCTS player can't even access rewards of states, but instead access only expected reward over information set. Because player don't see reward of each state of information set but only expected reward over all set he can't knowledgeably choose specific state from information set. Player choose random  state from information set according to some distribution instead. That mean player &quot;don't see&quot; which state from information is set actually realized.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-08-24T05:57:17.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14112" PostTypeId="1" CreationDate="2019-08-24T07:14:46.937" Score="2" ViewCount="285" Body="&lt;p&gt;What is the difference between machine learning and quantum machine learning?&lt;/p&gt;&#xA;" OwnerUserId="1581" LastEditorUserId="2444" LastEditDate="2020-03-07T06:56:59.470" LastActivityDate="2020-12-31T22:44:50.197" Title="What is the difference between machine learning and quantum machine learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;quantum-computing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14113" PostTypeId="1" CreationDate="2019-08-24T09:06:42.970" Score="3" ViewCount="37" Body="&lt;p&gt;I am looking for a CNN method, or any other machine learning method, to recognize 3D natural geometries that are similar to each others, and compare these geometries with a reference 3D model. To illustrate this, consider the following crater topographic map (x,y,z) of the Moon as an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RId90.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RId90.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exercise would be to recognize the craters, and compare their (3D) geometry (scale-invariant) with a reference 3D crater model (e.g. the one within the blue square). The result I am looking for is a kind of heatmap showing the similarity measure of (1) a sampled crater with the crater model, and/or (2) the geometry of some parts of the sampled crater (e.g. the inner crater steep sides) with those of the reference model. No classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to think that a 3D-oriented CNN method (OctNet, Octree CNN... etc) is a starting point for the above-mentioned task but I would rather prefer getting opinions on this matter since I am still a newbie in machine learning and we are dealing with direct application to real-world natural objects here.&lt;/p&gt;&#xA;" OwnerUserId="28171" LastActivityDate="2019-08-24T09:06:42.970" Title="3D geometry and similarity with a reference model" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14114" PostTypeId="2" ParentId="14079" CreationDate="2019-08-24T09:27:32.570" Score="2" Body="&lt;h1&gt;Overview&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;As it has already been observed, your main problem, beside the training related issues like fixing the learning rate, is you have basically &lt;strong&gt;no chance to learn such a big model woth such a small dataset ... from scratch&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So focusing on the real problem, here are some techniques you could use &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;dataset augmentation &lt;/li&gt;&#xA;&lt;li&gt;transfer learning &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;from a pretrained model &lt;/li&gt;&#xA;&lt;li&gt;from the encoder stage of an autoencoder (last resort option before getting into more advanced topics)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Dataset Augmentation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Add transformations to your dataset you want your classifier to learn to be invariant to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is an input image &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; its associated label &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(\mathcal{I};\theta) \rightarrow \mathcal{I}$&lt;/span&gt; is a parametric transformation that affects appearance but not semantic, for example it is a rotation of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; angle &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then you can augment your dataset by generating &lt;span class=&quot;math-container&quot;&gt;$\{I_{\theta}, l\}$&lt;/span&gt; a set of transformed (e.g. rotated) images associated the same &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; label &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Transfer Learning&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The fundamental idea of transfer learning is to re-use a NN which has been trained to solve a task, to solve other tasks retraining only a selected subset of the weights &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It means using a pre-trained convolutive backend, the part of the model with Conv2D and Pooling, and train dense layers with dropout only (but you should still probably think about reducing the dimensionality there) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More formally think about representing your CNN Classifier as follows &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f_{C}(I; \theta_{X})$&lt;/span&gt; : Convolutive Processing on Input Image &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it is the part of the CNN composed of &lt;code&gt;Conv2D&lt;/code&gt; and &lt;code&gt;MaxPooling2D&lt;/code&gt; layers &lt;/li&gt;&#xA;&lt;li&gt;the &lt;span class=&quot;math-container&quot;&gt;$\theta_{C}$&lt;/span&gt; is the convolutive learnable weights set &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b = f_{C}(I; \theta_{C})$&lt;/span&gt; : Bottleneck Feature Representation &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it is the result of &lt;code&gt;Flatten&lt;/code&gt; layer &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f_{D}(b; \theta_{D})$&lt;/span&gt; : Dense Processing &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it is the part of the model composed of &lt;code&gt;Dense&lt;/code&gt; layers &lt;/li&gt;&#xA;&lt;li&gt;the &lt;span class=&quot;math-container&quot;&gt;$\theta_{D}$&lt;/span&gt; is the dense learnable weights set &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The idea is to pick &lt;span class=&quot;math-container&quot;&gt;$\theta_{C}$&lt;/span&gt; from a training performed on an another dataset, bigger than your current one, and keep it fixed while training in your task &#xA;This means reducing the number of parameters to be trained, however beware the dense layers account for most of the weights, as you can also see from your mode summary, which means you should also focus on reducing that number, for example reducing the bottleneck feature tensor size &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Transfer Learning from Pre-Trained Model&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For example, if your actual goal was to perform binary classification on some kind of MNIST-like data then you could use a convolutive backend from a CNN which has been pre-trained on the MNIST 0..9 classification task or you can train it yourself but is important is the &lt;span class=&quot;math-container&quot;&gt;$\theta_{C}$&lt;/span&gt; weights will be learned from a MNIST dataset, which is much bigger than yours, even if the task is (slightly) different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, in case of MNIST like data, please consider if you really need your full &lt;code&gt;80 x 130&lt;/code&gt; resolution hence your input tensor, considering I can deduct from your model summary it is grayscale (no color), needs to be &lt;span class=&quot;math-container&quot;&gt;$(80,130,1)$&lt;/span&gt; or you could rescale to the &lt;code&gt;28 x 28&lt;/code&gt; MNIST resolution so you work with a smaller &lt;span class=&quot;math-container&quot;&gt;$(28,28,1)$&lt;/span&gt; tensor &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is to start from an architecture like this &lt;a href=&quot;https://gist.github.com/mjbhobe/cf895231aab8fa45a2a23da1351d185d#file-mnist_keras_model-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST Keras Model&lt;/a&gt; as &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it has a bottleneck representation of 64 which could be enough for your task and &lt;/li&gt;&#xA;&lt;li&gt;also suggesting to remove the first dense layer so to significantly reduce &lt;span class=&quot;math-container&quot;&gt;$\theta_{D}$&lt;/span&gt; the number of learnable paramters hence going for something like &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  model = Sequential()&#xA;  # add Convolutional layers&#xA;  model.add(Conv2D(filters=32, kernel_size=(3,3), activation='relu', padding='same', input_shape=(10, 10, 1)))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(Conv2D(filters=64, kernel_size=(3,3), activation='relu', padding='same'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(Conv2D(filters=64, kernel_size=(3,3), activation='relu', padding='same'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))    &#xA;  model.add(Flatten())&#xA;  # output layer&#xA;  model.add(Dense(1, activation='sigmoid'))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then compile the model with &lt;code&gt;binary_crossentropy&lt;/code&gt; loss and maybe start giving a try to &lt;code&gt;adam&lt;/code&gt; optimizer &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Transfer Learning from Autoencoder&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If your data is so special you can't find any big enough and similar enough dataset to use this strategy and you do not come up with any transformation you could use &#xA;to perform dataset augmentation, without getting into advanced things, you could try to play one last card: use an Autoencoder to learn a compressed representation aimed at reconstructing the original image and perform transfer learning with the encoder only &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, again under the assumption of working with a &lt;span class=&quot;math-container&quot;&gt;$(28,28,1)$&lt;/span&gt; tensor, you could start with an architecture like the following one &lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def build_ae(input_img): &#xA;  x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)&#xA;  # (28,28,16)&#xA;&#xA;  encoded = MaxPooling2D((8, 8), padding='same')(x)&#xA;  # (4,4,8)&#xA;&#xA;&#xA;  x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)&#xA;  # (4,4,8)&#xA;&#xA;  x = UpSampling2D((8, 8))(x)&#xA;  # (16,16,8)&#xA;&#xA;  x = Conv2D(16, (3, 3), activation='relu')(x)&#xA;  # Note: Convolving without padding='same' in order to get w-2 and h-2 dimensioality reduction so that following upsampling can lead to the desired 28x28 spatial resolution &#xA;  # (14,14,8)&#xA;&#xA;  x = UpSampling2D((2, 2))(x)&#xA;  # (28,28,8)&#xA;&#xA;  decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)&#xA;  autoencoder = Model(input_img, decoded)&#xA;  return autoencoder&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, the full model has 2633 weights but the encoding stage consists only of Conv2D+Relu+MaxPooling which means in total &lt;code&gt;3x3x1x16&lt;/code&gt; weights for the convolutive step and &lt;code&gt;16&lt;/code&gt; weights for the relu for a total of 160 weights only and the latent representation is a &lt;span class=&quot;math-container&quot;&gt;$(4,4,8)$&lt;/span&gt; tensor which means a 128 dimensional flattened tensor and hence assuming, as before, to perform the binary classification with a dense sigmoid layer it would mean 128+1 weights to learn in the actual binary classification task &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course it is possible to go for an even more compressed latent representation both on the spatial domain or channel domain with consequent reduced flattened vector dimensionality and ultimately even less weights to learn &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you share more details about your problem, also your dataset, we could try to help more &lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2019-08-24T09:27:32.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14116" PostTypeId="1" CreationDate="2019-08-24T10:20:47.827" Score="4" ViewCount="366" Body="&lt;p&gt;Is it popular or effective to concatenate the results of mean-pooling and max-pooling? To get the invariance of the latter and the expressivity of the former.&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2019-08-24T13:22:55.457" LastActivityDate="2019-08-24T13:22:55.457" Title="Combining mean pooling and max pooling" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14117" PostTypeId="1" CreationDate="2019-08-24T10:23:48.223" Score="3" ViewCount="351" Body="&lt;p&gt;How can transfer learning be used to mitigate &lt;a href=&quot;https://en.wikipedia.org/wiki/Catastrophic_interference&quot; rel=&quot;nofollow noreferrer&quot;&gt;catastrophic forgetting&lt;/a&gt;. Could someone elaborate on this?&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-08-24T11:51:21.170" LastActivityDate="2019-09-23T16:01:00.827" Title="How is transfer learning used to mitigate catastrophic forgetting in neural networks?" Tags="&lt;machine-learning&gt;&lt;transfer-learning&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14118" PostTypeId="2" ParentId="14116" CreationDate="2019-08-24T10:56:35.153" Score="3" Body="&lt;p&gt;I haven't seen it as you describe and I don't think it would be much useful. Pooling layers are being gradually phased out of networks, because they don't seem to be that useful anymore. With the emergence of more and more conv-only architectures, I don't see that likely.&lt;/p&gt;&#xA;" OwnerUserId="28173" LastActivityDate="2019-08-24T10:56:35.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14121" PostTypeId="1" AcceptedAnswerId="14138" CreationDate="2019-08-24T13:28:09.330" Score="7" ViewCount="3125" Body="&lt;p&gt;Currently, I am working on a few projects that use feedforward neural networks for regression and classification of simple tabular data. I have noticed that training a neural network using TensorFlow-GPU is often slower than training the same network  using TensorFlow-CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could something be wrong with my setup/code or is it possible that sometimes GPU is slower than CPU?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="27933" LastEditDate="2019-08-30T18:27:24.733" LastActivityDate="2019-08-30T18:27:24.733" Title="Is a GPU always faster than a CPU for training neural networks?" Tags="&lt;tensorflow&gt;&lt;gpu&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14123" PostTypeId="2" ParentId="14117" CreationDate="2019-08-24T15:18:47.850" Score="1" Body="&lt;p&gt;Transfer learning is a field where you apply knowledge from a source onto a target. This is a vague notion and there is an abundance of literature pertaining to it. Given your question I will work under the assumption that you are referring weight/architecture sharing between model (in other words training a model on one dataset and using it as a featurizer for another dataset)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now any learning system without lossless memory will have remnants of catastrophic forgetting. So let's think about how we would implement this transfer and what effects can be derived from this.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;One implementation involves transferring a component and only training additional layers.&lt;/li&gt;&#xA;&lt;li&gt;Another is retraining the entire system but at a lower learning rate?  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In setting 1, we can make the claim catastrophic forgetting is minimized by the fact that there is an unbiased featurizer that cant &lt;em&gt;forget&lt;/em&gt; based on a sampling regime, though this does not mean additional layers which are still being trained can still faulter in this error mode.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In setting 2, we can make the claim catasrophic forgetting can be reduced compared to a normal end-to-end no-transfer training because the unbiased featurizers difference can be analytically bounded by its initial transferred featurization (complexity class is based on both the function and the number of steps -- so longer you train, the more likely it can &lt;em&gt;forget&lt;/em&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These reasons are talking about mitigating and not erasing the concept of catastrophic forgetting, that is because as I mentioned above &lt;em&gt;any learning system without lossless memory will have remnants of catastrophic forgetting&lt;/em&gt;, so making the generalized claim about transfer learning may not always fit the bill.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-24T15:18:47.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14124" PostTypeId="2" ParentId="14107" CreationDate="2019-08-24T15:37:00.547" Score="1" Body="&lt;p&gt;This is perfectly acceptable in a stochastic environment. Generally your loss is to minimize &lt;span class=&quot;math-container&quot;&gt;$-log\ p(Y|X)$&lt;/span&gt; or equivalently &lt;span class=&quot;math-container&quot;&gt;$-\sum_i log\ p(y_i|x_i)$&lt;/span&gt;. This optimization is equivalent to &lt;span class=&quot;math-container&quot;&gt;$-\mathbb{E}\log\ p(y_i|x_i)$&lt;/span&gt;. In other words you are minimizing in this case:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;\begin{align*}  &#xA;L &amp;amp;= -log\ p(1|x_0) - log\ p(0|x_0) \\ &#xA;&amp;amp;= -log [p(1|x_0) * p(0|x_0)] \\&#xA;&amp;amp;= -log [p(1|x_0) * (1 - p(1|x_0))] \\&#xA;\end{align*}&#xA;$$&lt;/span&gt;&lt;br&gt;&#xA;or since log is concave equivalently minimizing&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \hat L = -p(1|x_0) * (1 - p(1|x_0)) $$&lt;/span&gt; &#xA;After some basic calc 1, we see the optimal result we want the system to learn is that&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ p(1|x_0) = .5$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if you had more evidence, the result would just be that you want it to learn that it is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_i\ y_i | x$&lt;/span&gt; &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-08-24T15:37:00.547" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14125" PostTypeId="2" ParentId="14079" CreationDate="2019-08-24T16:13:31.687" Score="1" Body="&lt;p&gt;Nicola Bernini's answer is quite comprehensive. Here are my insights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First of all, think whether you really need neural networks to solve your problem. Think whether traditional computer vision operations like edge detection/ region-based methods help you to solve your problem (OpenCV can help you here). Think about your data again. In case you decide to use neural networks, some things to try out:-&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your data set size is too small. Try to recall that we are learning to approximate functions (Universal approximation theorem). Less data + More parameters have high chance of overfitting. Use transfer learning. (Try Resizing the image and perform random resized square crop and use as input to your neural network. This may or may not work since I don't know what exactly you are doing). Also try data augmentations that make sense (i.e. : Vertical flip of traffic sign images don't work)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try reducing the learning rate/ using different learning rates for different parts of your network if you decide to use transfer learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check whether your train and test dataset distributions are the same. i.e.: Don't train with 95 % of label 0 and test with a set that has 95% label 1. I do not know whether your dataset is highly class imbalanced or whether you are doing some kind of anomaly detection.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Think about optimizers. Try Adam if you haven't. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you need more help, try sharing the data and we can try to help you.&lt;/p&gt;&#xA;" OwnerUserId="28182" LastActivityDate="2019-08-24T16:13:31.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14126" PostTypeId="1" CreationDate="2019-08-24T16:35:56.727" Score="2" ViewCount="53" Body="&lt;p&gt;I am completely new to CNN's, and I do not quite know how to design or use them efficiently. That being said, I am attempting to build a CNN that learns to play Pac-man with reinforcement learning. I have trained it for about 3 hours and have seen little to no improvement. My observation space is 3 channels * 15 * 19, and there are 5 actions. Here is my code, I am open to any and all suggestions. Thanks for all your help.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from minipacman import MiniPacman as pac&#xA;from torch import nn&#xA;import torch&#xA;import random&#xA;import torch.optim as optimal&#xA;from torch.autograd import Variable&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;import keyboard&#xA;&#xA;&#xA;loss_fn = nn.MSELoss()&#xA;epsilon = 1&#xA;env = pac(&quot;regular&quot;, 1000)&#xA;time = 0&#xA;action = random.randint(0, 4)&#xA;q = np.zeros(3)&#xA;alpha = 0.01&#xA;gamma = 0.9&#xA;tick = 0&#xA;decay = 0.9999&#xA;&#xA;&#xA;class Value_Approximator (nn.Module):&#xA;    def __init__(self):&#xA;        super(Value_Approximator, self).__init__()&#xA;        # Convolution 1&#xA;        self.cnn1 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=5, stride=1, padding=2)&#xA;        self.relu1 = nn.ReLU()&#xA;&#xA;        # Max pool 1&#xA;        self.maxpool1 = nn.MaxPool2d(kernel_size=2)&#xA;&#xA;        # Convolution 2&#xA;        self.cnn2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, stride=1, padding=2)&#xA;        self.relu2 = nn.ReLU()&#xA;&#xA;        # Max pool 2&#xA;        self.maxpool2 = nn.MaxPool2d(kernel_size=2)&#xA;&#xA;        # Fully connected 1 (readout)&#xA;        self.fc1 = nn.Linear(384, 5)&#xA;&#xA;    def forward(self, x):&#xA;        # Convolution 1&#xA;        out = self.cnn1(x)&#xA;        out = self.relu1(out)&#xA;&#xA;        # Max pool 1&#xA;        out = self.maxpool1(out)&#xA;&#xA;        # Convolution 2&#xA;        out = self.cnn2(out)&#xA;        out = self.relu2(out)&#xA;&#xA;        # Max pool 2&#xA;        out = self.maxpool2(out)&#xA;&#xA;        # Resize&#xA;        # Original size: (100, 32, 7, 7)&#xA;        # out.size(0): 100&#xA;        # New out size: (100, 32*7*7)&#xA;        out = out.view(out.size(0), -1)&#xA;&#xA;        # Linear function (readout)&#xA;        out = self.fc1(out)&#xA;&#xA;        return out&#xA;&#xA;approx = Value_Approximator()&#xA;optimizer = optimal.SGD(approx.parameters(), lr=alpha)&#xA;&#xA;&#xA;while time &amp;lt; 50000:&#xA;    print(&quot;Time: &quot;+str(time))&#xA;    print(&quot;Epsilon: &quot;+str(epsilon))&#xA;    print()&#xA;    time += 1&#xA;    state = env.reset()&#xA;    tick = 0&#xA;&#xA;    epsilon *= decay&#xA;&#xA;    if epsilon &amp;lt; 0.1:&#xA;        epsilon = 0.1&#xA;&#xA;    while True:&#xA;        tick += 1&#xA;        state = np.expand_dims(state, 1)&#xA;        state = state.reshape(1, 3, 15, 19)&#xA;        q = approx.forward(torch.from_numpy(state))[0]&#xA;&#xA;        if random.uniform(0, 1) &amp;lt; epsilon:&#xA;            action = env.action_space.sample()&#xA;        else:&#xA;            _, action = torch.max(q, -1)&#xA;            action = action.item()&#xA;        new_state, reward, terminal, _ = env.step(action)&#xA;        show_state = new_state&#xA;        new_state = np.expand_dims(new_state, 1)&#xA;        new_state = state.reshape(1, 3, 15, 19)&#xA;&#xA;        q_new = approx.forward(torch.from_numpy(new_state).type(torch.FloatTensor))[0]  # &quot; find Q (s', a') &quot;&#xA;        #  find optimal action Q value for next step&#xA;        new_max, _ = torch.max(q_new, -1)&#xA;        new_max = new_max.item()&#xA;&#xA;        q_target = q.clone()&#xA;        q_target = Variable(q_target.data)&#xA;&#xA;        #  update target value function according to TD&#xA;        q_target[action] = reward + torch.mul(new_max, gamma)  # &quot; reward + gamma*(max(Q(s', a')) &quot;&#xA;&#xA;        loss = loss_fn(q, q_target)  # &quot; reward + gamma*(max(Q(s', a')) - Q(s, a)) &quot;&#xA;        # Update original policy according to Q_target ( supervised learning )&#xA;        approx.zero_grad()&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;&#xA;        #  Q and Q_target should converge&#xA;        if time % 100 == 0:&#xA;            state = torch.FloatTensor(show_state).permute(1, 2, 0).cpu().numpy()&#xA;&#xA;            plt.subplot(131)&#xA;            plt.title(&quot;Imagined&quot;)&#xA;            plt.imshow(state)&#xA;            plt.subplot(132)&#xA;            plt.title(&quot;Actual&quot;)&#xA;            plt.imshow(state)&#xA;            plt.show(block=False)&#xA;            plt.pause(0.000001)&#xA;&#xA;        if keyboard.is_pressed('1'):&#xA;            torch.save(approx.state_dict(), 'trained-10000.mdl')&#xA;        if keyboard.is_pressed('9'):&#xA;            torch.save(approx.state_dict(), 'trained-10000.mdl')&#xA;&#xA;        if terminal or tick &amp;gt; 100:&#xA;            plt.close()&#xA;            break&#xA;&#xA;        state = new_state&#xA;&#xA;&#xA;torch.save(approx.state_dict(), 'trained-10000.mdl')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4744" LastEditorUserId="4744" LastEditDate="2019-08-24T23:32:40.207" LastActivityDate="2019-08-24T23:32:40.207" Title="Torch CNN not training" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14127" PostTypeId="2" ParentId="14121" CreationDate="2019-08-24T17:21:18.877" Score="2" Body="&lt;p&gt;I advice you to always use GPU over CPU for training your models. This is driven by the usage of deep learning methods on images and texts, where the data is very rich.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must have a GPU suited perfectly for training (e.g. NVIDIA 1080, NVIDIA Titan or higher versions), I wouldn't be surprised to find that your CPU was faster if you don't have a powerful GPU.&lt;/p&gt;&#xA;" OwnerUserId="27519" LastActivityDate="2019-08-24T17:21:18.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14128" PostTypeId="1" CreationDate="2019-08-24T17:36:24.823" Score="2" ViewCount="26" Body="&lt;p&gt;I want to know how to generate the structured light which projects different patterns of light on a 3D object which is under scanning. &lt;/p&gt;&#xA;" OwnerUserId="27519" LastEditorUserId="2444" LastEditDate="2019-08-25T10:35:30.103" LastActivityDate="2019-08-25T10:35:30.103" Title="How do I generate structured light for the 3D bin picking system?" Tags="&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14129" PostTypeId="2" ParentId="14077" CreationDate="2019-08-24T17:50:01.357" Score="1" Body="&lt;p&gt;is your data stored in raw ASCII text, like a CSV file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you can speed up data loading and use less memory by using another data format. A good example is a binary format like GRIB, NetCDF, or HDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many command line tools that you can use to transform one data format into another that do not require the entire dataset to be loaded into memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using another format may allow you to store the data in a more compact form that saves memory, such as 2-byte integers, or 4-byte floats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, you may need to resort to a big data platform.&#xA;That is, a platform designed for handling very large datasets, that allows you to use data transforms and machine learning algorithms on top of it.&lt;/p&gt;&#xA;" OwnerUserId="27519" LastActivityDate="2019-08-24T17:50:01.357" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14130" PostTypeId="1" CreationDate="2019-08-24T19:25:56.633" Score="4" ViewCount="44" Body="&lt;p&gt;I'm looking for either an existing AI app or a pre-trained NN that will tell me if a photograph is right-side up or not. I want to use this to create an application that automatically rotates photos so they are right-side-up. This doesn't seem hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it doesn't exist, presumably I can create it with Tensorflow, and just use a ton of photos to train it, and assume they are all correctly oriented in the training set. Would that work?&lt;/p&gt;&#xA;" OwnerUserId="27476" LastActivityDate="2019-08-26T15:47:34.830" Title="Turn photos right-side up?" Tags="&lt;image-processing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14131" PostTypeId="2" ParentId="13884" CreationDate="2019-08-24T20:54:30.333" Score="1" Body="&lt;p&gt;The question would need clarification. I'm new here so I will try to give an answer.&#xA;The easiest way would be to prepare the data in comma separated values (CSV). It is possible to export your data in this format from excel.&#xA;For downstream applications it will depends what programming language is used but in general it is possible to import CSV files and the data they contain for training (example panda data frame in python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="28010" LastActivityDate="2019-08-24T20:54:30.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14133" PostTypeId="2" ParentId="13975" CreationDate="2019-08-24T22:56:12.543" Score="5" Body="&lt;p&gt;As far as I've seen there is no use of 3D CNNs for traditional image classification tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I think is that, while these images do have multiple channels, there is no spatial information in those channels for the 3D convolution to extract. On the other hand it makes more sense to take the weighted sum of those pixels along that dimension (as the 2D convolution does).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3D CNNs have been used, as far as I know, only for applications where you have volumetric data, i.e. the images are sequential and, when combined, form a large 3D image.&lt;/p&gt;&#xA;" OwnerUserId="27652" LastActivityDate="2019-08-24T22:56:12.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14135" PostTypeId="1" CreationDate="2019-08-25T02:19:35.037" Score="2" ViewCount="25" Body="&lt;p&gt;I made a CNN with a reasonable loss curve, but the performance of the model does not improve. I have tried making the model larger, I am using three convolutional layers with batch norms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9QnRN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9QnRN.png&quot; alt=&quot;Data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4744" LastActivityDate="2019-08-25T02:19:35.037" Title="Loss reduction, but constant performance with CNN" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14137" PostTypeId="1" AcceptedAnswerId="14139" CreationDate="2019-08-25T07:12:44.720" Score="5" ViewCount="115" Body="&lt;p&gt;I have a question about state representation of Q-learning or DQN algorithm.&#xA;I'm still a beginner of RL, so I'm not sure that is it suitable to take exogenous variables as state features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in my current project, deciding to charge/discharge an electric vehicle actions according to the real-time fluctuating electricity prices, I'm wondering if the &lt;strong&gt;past n-step prices or hours&lt;/strong&gt; can be considered as state features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because both the prices and the hour are just given information in every time step rather than being dependent to the charging/discharging actions, I'm suspicious about whether they can are theoretically qualified to be state features or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they are not qualified, could someone give me a reference or something that I can read?&lt;/p&gt;&#xA;" OwnerUserId="27946" LastEditorUserId="2444" LastEditDate="2019-08-25T10:14:11.083" LastActivityDate="2019-08-25T10:14:11.083" Title="Can exogenous variables be state features in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14138" PostTypeId="2" ParentId="14121" CreationDate="2019-08-25T08:08:53.130" Score="6" Body="&lt;p&gt;This changes according to your data and complexity of your models. See following article by &lt;a href=&quot;https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-deployment-of-deep-learning-models/&quot; rel=&quot;noreferrer&quot;&gt;microsoft&lt;/a&gt;. Their conclusion is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The results suggest that the throughput from GPU clusters is always&#xA;  better than CPU throughput for all models and frameworks proving that&#xA;  GPU is the economical choice for inference of deep learning models.&#xA;  ...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It is important to note that, for standard machine learning models&#xA;  where number of parameters are not as high as deep learning models,&#xA;  CPUs should still be considered as more effective and cost efficient.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since you are training MLP, it can not be thought as standard machine learning model. See my &lt;a href=&quot;https://peerj.com/preprints/2838/&quot; rel=&quot;noreferrer&quot;&gt;preprint, The impact of using large training data set KDD99 on classification accuracy&lt;/a&gt;. I compare different machine learning algorithms using weka.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H9Ogn.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H9Ogn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see from above image, MLP takes 722 minutes to train while Naive Bayes ~2 minutes. If your data is small and your models parameters are not high, you will see better performance on CPU.&lt;/p&gt;&#xA;" OwnerUserId="4300" LastActivityDate="2019-08-25T08:08:53.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14139" PostTypeId="2" ParentId="14137" CreationDate="2019-08-25T09:44:37.117" Score="3" Body="&lt;p&gt;Including exogenous variables in your state representation certainly can be useful, &lt;strong&gt;as long as you expect them to be relevant information for determining the action to pick.&lt;/strong&gt; So, state features are not only useful if you expect your agent (through application of actions) to have (partial) influence on those state variables; you just want the state variables themselves to be informative for your next action to take / prediction of expected future rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you &lt;strong&gt;only have exogenous variables&lt;/strong&gt;, i.e. if you expect your agent to have no influence whatsoever on what states you'll end up in next... then the full problem definition typically used in RL (Markov decision processes) may be unnecessarily complex, and you may prefer to look into the Multi-Armed Bandits (MAB) problem formulation. If you're already familiar with RL / MDPs, you may think of MAB problems as (sequences of) single-step episodes, where you always just look at the current state and don't care at all about future states (because you expect to have 0 influence on them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, the RL / MDP framework is more general and is also applicable to those MAB problems, but RL algorithms that support this framework may perform worse than MAB algorithms in practice, because they (informally speaking) still put in effort trying to &quot;learn&quot; how their actions affect future states (a waste of effort when you expect there to be no such influence from the agent).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-08-25T09:44:37.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14140" PostTypeId="1" AcceptedAnswerId="14142" CreationDate="2019-08-25T14:19:09.403" Score="3" ViewCount="47" Body="&lt;p&gt;I started working on the application of deep learning in medical imaging recently. While dealing with MRI images in the BraTS dataset, I observe that first and last few frames are always completely empty (black). I want to ask those who are already working in the field, is there a way to remove them in a procedural manner before training and add them correctly after the training as a postprocessing step (to comply with the ground truth segmentations' shape)? Has anyone tried that?&#xA;I could not find any results on Google. So asking here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I think I did not make my point clear enough. I meant to say first and last few frames of &lt;em&gt;each&lt;/em&gt; MRI scan are empty. How to deal with those is what I intended to ask.&lt;/p&gt;&#xA;" OwnerUserId="16159" LastEditorUserId="-1" LastEditDate="2019-08-28T17:17:29.007" LastActivityDate="2019-08-28T17:17:29.007" Title="Dealing with empty frames in MRI images" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;autoencoders&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14141" PostTypeId="2" ParentId="14140" CreationDate="2019-08-25T14:51:30.073" Score="0" Body="&lt;p&gt;As an expert in image analysis I don't think this would be a problem. I have never worked with MRI images from the particular dataset you described but I found that the format of the file containing the images is NIfTI.&#xA;NIfTI files can be imported in Matlab(niftiread function), ImageJ and Python (NiBabel-Nipy). Thus you should be able to write a script to import the images from the file, select which images you want to keep, and save the output I'm the same format as the input (NIfTI).&lt;/p&gt;&#xA;" OwnerUserId="28010" LastEditorUserId="28010" LastEditDate="2019-08-27T16:59:53.447" LastActivityDate="2019-08-27T16:59:53.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14142" PostTypeId="2" ParentId="14140" CreationDate="2019-08-25T22:37:03.600" Score="1" Body="&lt;p&gt;I've worked on the BRATS dataset and I can verify that this is pretty much standard process. Besides throwing the totally blank images, I also throw away the images in the beginning and ending of the sequence that show the tip of the scull and the base of the neck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally when dealing with MRIs,&#xA;I do this with a script (think of is as a preprocessing step) I run on each image that counts the amount of pixels that have a positive intensity (I actually add a small value to this to account for noise). Let's say for images with values 0-255 I count the amount of pixels with an intensity of over 10-15, let's call this &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. After that I set a threshold (empirically), let's call it &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and discard images with &lt;span class=&quot;math-container&quot;&gt;$x &amp;lt; t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically for BRATS, because you have the labels, you can see which of these have the desired classes and discard most of the rest. If you try to train a network on the dataset as is you face an &lt;strong&gt;enormous&lt;/strong&gt; imbalance ratio. I've had trouble training networks due to this and the most success I got was when I threw away most of the irrelevant images.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-08-25T22:37:03.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14143" PostTypeId="2" ParentId="14121" CreationDate="2019-08-26T00:25:15.360" Score="1" Body="&lt;p&gt;It depends, if you ve to solve a &quot;simple&quot; problem which does not require CNN or stacked models without multidimensional data and not many multiplications, big long numbers then if you decide to use CNN / stacked architectures AND GPU it is like using a hammer to insert a needle.It will not only spend energy but the computations will do zero padding in memory, you will observe a degradation in speed.&lt;/p&gt;&#xA;" OwnerUserId="28200" LastEditorUserId="28200" LastEditDate="2019-08-30T13:48:59.523" LastActivityDate="2019-08-30T13:48:59.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14144" PostTypeId="2" ParentId="14079" CreationDate="2019-08-26T06:55:56.960" Score="0" Body="&lt;p&gt;From my experience, this oscillation comes from:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Too high learning rate: weights change too quick.&lt;/li&gt;&#xA;&lt;li&gt;Too few neurons in layers: not enough to fit.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-08-26T06:55:56.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14145" PostTypeId="1" CreationDate="2019-08-26T10:36:57.950" Score="3" ViewCount="29" Body="&lt;p&gt;I'm using Prioritized Experience Replay (PER) with a DDQN. To compensate for overfitting relatively high-value samples due to the non-uniform selection, I'm training with sample weights provided along with the PER samples to downplay each sample's loss contribution according to its probability of selection. I've observed that typically these sample weightings vary from &lt;span class=&quot;math-container&quot;&gt;$~0.1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;0.01$&lt;/span&gt;, as the buffer gradually fills up (4.8M samples). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using this compensation, the growth of the maximal Q value per episode stalls prematurely compared to a non-weight-compensated regime. I presume that this is because the size of the back-propagation updates is being greatly and increasingly diminished by the sample weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To correct for this I've tried taking the beta-adjusted maximum weight as reported by the PER (the same buffer-wide value by which the batch is normalized) and multiplying the base learning rate by it, thereby adjusting the optimizer after each batch selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is two-fold:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this the correct interpretation of what's going on? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it standard practice to compensate for sample weighting in this way?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Although it seems to be working in keeping the Q growth alive whilst taming the loss, I've not been able to find any information on this and haven't found any implementations that compensate in this way so have a major doubt about the mathematical validity of it.&lt;/p&gt;&#xA;" OwnerUserId="28203" LastEditorUserId="2444" LastEditDate="2019-08-31T23:50:21.110" LastActivityDate="2019-08-31T23:50:21.110" Title="Should importance sample weighting be compensated for by dynamically increasing learning rate?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14146" PostTypeId="2" ParentId="14059" CreationDate="2019-08-26T14:00:20.630" Score="1" Body="&lt;p&gt;If your using computer vision, the top recognised conference is CVPR (computer vision and pattern recognition).&lt;/p&gt;&#xA;&lt;p&gt;You can also try to submit at ICML (International conference on machine learning) and NIPS (Neural Information Processing Systems), which focuses on applications of machine learning and deep learning.&lt;/p&gt;&#xA;&lt;p&gt;I'd also recommend IJCAI (International joint conference on artificial intelligence).&lt;/p&gt;&#xA;" OwnerUserId="27227" LastEditorUserId="2444" LastEditDate="2021-01-22T01:32:09.393" LastActivityDate="2021-01-22T01:32:09.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14147" PostTypeId="1" CreationDate="2019-08-26T14:19:49.737" Score="3" ViewCount="170" Body="&lt;p&gt;I am basically interested in vehicle on the road.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ayooshkathuria/pytorch-yolo-v3&quot; rel=&quot;nofollow noreferrer&quot;&gt;YoloV3 pytorch&lt;/a&gt; is giving a decent result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my interested Vehicles &lt;code&gt;Car&lt;/code&gt; &lt;code&gt;Motorbike&lt;/code&gt; &lt;code&gt;Bicycle&lt;/code&gt; &lt;code&gt;Truck&lt;/code&gt; and &lt;code&gt;bus&lt;/code&gt;, I have a small vehicles being detected as &lt;code&gt;truck&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the small vehicle is nicely being detected as truck. I have annotated this small vehicle as a different class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though, I could add an extra class say 81th class, since the current YoloV3 being used is trained on 80 classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;81th class would contain the weight of the truck, I would freeze the weights such that the rest of the 80 classes remain unaltered and only the 81th class of this new data gets trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the final layer gets tuned according to the prediction of all the classes it learns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was not able to find any post that could actually mention this way of preserving the predictions of the other classes and introducing a new class using transfer learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest, I was able to get is this post of &#xA;&lt;a href=&quot;https://github.com/pierluigiferrari/ssd_keras/blob/master/weight_sampling_tutorial.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Sampling Tutorial&lt;/a&gt; SSD using keras&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its mentioned in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Option 1: Just ignore the fact that we need only 8 classes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;This would work, and it wouldn't even be a terrible option. Since only 8 out of the 80 classes would get trained, the model might get gradually worse at predicting the other 72 clases&lt;/code&gt; in the second paragraph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to preserve the predictions, of the previous pre trained model while introducing the new class and use transfer learning to train only for that class?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I Feel that this is not possible, would like to know your opinion. Hope someone can prove me wrong. &lt;/p&gt;&#xA;" OwnerUserId="17763" LastEditorUserId="23503" LastEditDate="2020-11-09T11:14:00.433" LastActivityDate="2020-11-09T11:14:00.433" Title="Transfer learning to train only for a new class while not affecting the predictions of the other class" Tags="&lt;convolutional-neural-networks&gt;&lt;transfer-learning&gt;&lt;incremental-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14148" PostTypeId="2" ParentId="14130" CreationDate="2019-08-26T15:47:34.830" Score="2" Body="&lt;p&gt;I don't know if there is an existing pretrained NN that does this but it wouldn't be very hard to modify one to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I'd take a pretrained image classification NN (e.g. VGG, ResNet), drop its final layer and replace it with one with 4 neurons, representing the 4 orientations (so that you know which way to rotate it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I'd take again a dataset of regular images (e.g. a subset of ImageNet) and assume that they are correctly oriented. I'd make three more duplicate datasets with the same images rotates by 90, 180 and 270 degrees respectively. These 4 datasets would be the 4 classes I'd fine tune the model on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By training your model on this dataset, you'll be training it to recognize which side your image is facing. Since it's a pretrained net and its a fairly simple task, I think that after a few iterations, your model will have converged. Then you could write a script that uses this model to predict an image's orientation and rotate it accordingly.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-08-26T15:47:34.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14149" PostTypeId="1" AcceptedAnswerId="14155" CreationDate="2019-08-26T16:26:27.947" Score="0" ViewCount="502" Body="&lt;p&gt;I am using feedforward neural network for regression and what I get as a result of prediction is a constant value visible on the graph below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GWP60.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GWP60.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data I use are typical standardised tabular numbers. The architecture is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.add(Dropout(0.2))&#xA;model.add(Dense(units=512, activation='relu'))&#xA;model.add(Dropout(0.3))&#xA;model.add(Dense(units=256, activation='relu'))&#xA;model.add(Dropout(0.3))&#xA;model.add(Dense(units=128, activation='relu'))&#xA;model.add(Dense(units=128, activation='relu'))&#xA;model.add(Dense(units=1))&#xA;&#xA;adam = optimizers.Adam(lr=0.1)&#xA;&#xA;model.compile(loss='mean_squared_error', optimizer=adam)&#xA;&#xA;reduce_lr = ReduceLROnPlateau(&#xA;    monitor='val_loss',&#xA;    factor=0.9,&#xA;    patience=10,&#xA;    min_lr=0.0001,&#xA;    verbose=1)&#xA;&#xA;tensorboard = TensorBoard(log_dir=&quot;logs\{}&quot;.format(NAME))&#xA;&#xA;history = model.fit(&#xA;    x_train,&#xA;    y_train,&#xA;    epochs=500,&#xA;    verbose=10,&#xA;    batch_size=128,&#xA;    callbacks=[reduce_lr, tensorboard],&#xA;    validation_split=0.1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems to me that all weights are zeroed and only constant bias is present here, since for different data samples from a test set I get the same value, but I am not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the algorithm has found smallest MSE for such a constant value, but is there a way of avoiding such situation, since straight line is not really good solution for my project?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-08-27T08:52:45.217" Title="Why do I get a straight line as an output from a neural network?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14150" PostTypeId="1" CreationDate="2019-08-26T17:00:41.613" Score="2" ViewCount="11" Body="&lt;p&gt;While reading the &lt;a href=&quot;https://arxiv.org/pdf/1606.03657.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;InfoGAN&lt;/a&gt; paper and implement it taking help from a previous implementation, I'm having some difficulty understanding how it learns the discrete categorical code when trained on MNIST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://github.com/pianomania/infoGAN-pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementation&lt;/a&gt; we tried to follow uses a target as a randomly generated integer from 0 to 9. My doubt is this: how can it learn categorical information if from the start it's learning using a loss which takes in random values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this implementation is wrong, then what should the target be while training the Q network, when using the categorical-cross-entropy loss on the output logits?&lt;/p&gt;&#xA;" OwnerUserId="26831" LastActivityDate="2019-08-26T17:00:41.613" Title="How does InfoGAN learn latent categorical codes on MNIST" Tags="&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14152" PostTypeId="1" CreationDate="2019-08-27T01:23:40.080" Score="0" ViewCount="144" Body="&lt;p&gt;I am a student of last year of computer engineering and lately I have been very interested in AI. Fields such as ML and DL seem very disruptive to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few months ago I saw an interview of Bill Gates in which he was asked the following: if now you had 20 years in which area of ​​technology would you focus or what kind of company would you start? He replied that he would try to teach to read to an artificial intelligence algorithm and so it would draw all the knowledge from all possible books.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.hotlinenews.com/2019/06/24/bill-gates-if-i-were-starting-a-company-today-it-would-use-ai-to-teach-computers-how-to-read/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bill Gates: If I were starting a company today, it would use AI to teach computers how to read&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that currently nobody has managed to do it in a very efficient way. Do you think that with the AI ​​techniques that exist now such as Supervised Learning, Unsupervised Learning, etc., is it possible to achieve something similar? If so, why type of algorithm or technique do you bet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you. It is only scientific curiosity.&lt;/p&gt;&#xA;" OwnerUserId="28217" LastEditorUserId="25982" LastEditDate="2019-09-04T09:36:07.820" LastActivityDate="2019-09-04T09:36:07.820" Title="Can a neural net learn to read?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14153" PostTypeId="1" CreationDate="2019-08-27T03:39:25.507" Score="4" ViewCount="109" Body="&lt;p&gt;I am trying to rank video scenes/frames based on how appealing they are for a viewer. Basically, how &quot;interesting&quot; or &quot;attractive&quot; a scene inside a video can be for a viewer. My final goal is to generate say a 10-second short summary given a video as input, such as those seen on Youtube when you hover your mouse on a video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I previously asked a similar question &lt;a href=&quot;https://ai.stackexchange.com/questions/13994&quot;&gt;here&lt;/a&gt;. But the &quot;aesthetics&quot; model is good for ranking artistic images, not good for frames of videos. So it was failing. I need a score based on &quot;engagement for general audience&quot;. Basically, which scenes/frames of video will drive more clicks, likes, and shares when selected as a thumbnail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we have an available deep-learning model or a prototype doing that? A ready-to-use prototype/model that I can test as opposed to a paper that I need to implement myself. Paper is fine as long as the code is open-source. I'm new and can't yet write a code given a paper.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2019-09-11T23:21:29.513" Title="Video engagement analysis with deep learning" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14155" PostTypeId="2" ParentId="14149" CreationDate="2019-08-27T08:52:06.277" Score="0" Body="&lt;p&gt;You should try experimenting with a lower learning rate as a starting point. You're starting with 0.1 which is quite high for most of the cases, and only reduce by 0.9 which is not much.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-08-27T08:52:06.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14158" PostTypeId="2" ParentId="7966" CreationDate="2019-08-27T11:41:08.590" Score="4" Body="&lt;p&gt;This article on &lt;a href=&quot;https://hackernoon.com/dynamically-expandable-neural-networks-ce75ff2b69cf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamically Expandable Neural Networks&lt;/a&gt; (DEN) (by Harshvardhan Gupta) is based on this paper &lt;a href=&quot;https://arxiv.org/pdf/1708.01547v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lifelong Learning with Dynamically Expandable Networks&lt;/a&gt; (by Jeongtae Lee, Jaehong Yoon, Eunho Yang, Sung Ju Hwang)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This presents 3 solutions to increase the capacity of the network if needed retaining whatever useful information from the old model and train the new model:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Selective retraining&lt;/li&gt;&#xA;&lt;li&gt;Dynamic Network Expansion &lt;/li&gt;&#xA;&lt;li&gt;Network Split/Duplication&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To me, it seems that such neural network is dynamic and improving. As such, they answer partially your question. If they don't sorry about that.&lt;/p&gt;&#xA;" OwnerUserId="19852" LastEditorUserId="2444" LastEditDate="2019-08-27T23:30:11.123" LastActivityDate="2019-08-27T23:30:11.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14159" PostTypeId="1" AcceptedAnswerId="14160" CreationDate="2019-08-27T14:46:48.670" Score="7" ViewCount="682" Body="&lt;p&gt;I came across some papers that use &lt;span class=&quot;math-container&quot;&gt;$\mathbb E$&lt;/span&gt; in equations, in particular, this paper: &lt;a href=&quot;https://arxiv.org/pdf/1511.06581.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1511.06581.pdf&lt;/a&gt;. Here is some equations from the paper that uses it: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q^\pi \left(s,a \right) = \mathbb E \left[R_t|s_t = s, a_t = a, \pi \right]$&lt;/span&gt; ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V^\pi \left(s \right) = \mathbb E_{a\backsim\pi\left(s \right)} \left[Q^\pi \left(s, a\right) \right]$&lt;/span&gt; ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q^\pi \left(s, a \right) = \mathbb E_{s'} \left[r + \gamma\mathbb E_{a'\backsim\pi \left(s' \right)} \left[Q^\pi \left(s', a' \right) \right] | s,a,\pi \right]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_i}L_i\left(\theta_i \right) = \mathbb E_{s, a, r, s'} \left[\left(y_i^{DQN} - Q \left(s, a; \theta_i \right) \right) \nabla_{\theta_i} Q\left(s, a, \theta_i \right) \right]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone explain to me what is the purpose of &lt;span class=&quot;math-container&quot;&gt;$\mathbb E$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="28233" LastEditorUserId="2444" LastEditDate="2019-08-28T10:52:15.883" LastActivityDate="2019-11-07T22:03:52.043" Title="What does the symbol $\mathbb E$ mean in these equations?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14160" PostTypeId="2" ParentId="14159" CreationDate="2019-08-27T15:06:12.793" Score="12" Body="&lt;p&gt;That's the &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;noreferrer&quot;&gt;Expected Value operator&lt;/a&gt;. Intuitively, it gives you the value that you would &quot;expect&quot; (&quot;on average&quot;) the expression after it (often in square or other brackets) to have. Typically that expression involves some random variables, which means that there may be a wide range of different values the expression may take in any concrete, single event. Taking the expectation basically means that you &quot;average&quot; over all the values the expression could potentially take, appropriately weighted by the probabilities with which certain events occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll often find assumptions under which the expectation is taken after a vertical line (&lt;span class=&quot;math-container&quot;&gt;$\mid$&lt;/span&gt;) inside the brackets, and/or in the subscript to the right of the &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}$&lt;/span&gt; symbol. Sometimes, some assumptions may also be left implicit.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E} \left[ R_t \mid s_t = s, a_t = a, \pi \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;may be read in english as &quot;the expected value of the discounted returns from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; onwards (&lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;), given that the state at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$s_t = s$&lt;/span&gt;), given that our action at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$a_t = a$&lt;/span&gt;), given that we continue behaving according to policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; after time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say that, in this case, the expected value also relies on the transition dynamics of the Markov decision process (i.e. the probabilities with which we transition between states, given our actions). This is left implicit.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Second example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^{\pi}(s) = \mathbb{E}_{a \sim \pi(s)} \left[ Q^{\pi}(s, a) \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;may be read as &quot;&lt;span class=&quot;math-container&quot;&gt;$V^{\pi}(s)$&lt;/span&gt; is equal to the expected value of &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s, a)$&lt;/span&gt;, under the assumption that &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is sampled according to &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt;&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, something like this could be computed by enumerating over all possible &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, computing &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s, a)$&lt;/span&gt; for every such &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, and multiplying it by the probability that &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; assigns to &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. In practice, you could also approximate it by running a large number of experiments in which you sample &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt;, and then evaluate a single concrete case of &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s, a)$&lt;/span&gt;, and average over all the evaluations.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-08-27T15:23:00.037" LastActivityDate="2019-08-27T15:23:00.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14161" PostTypeId="2" ParentId="14159" CreationDate="2019-08-27T15:41:03.960" Score="7" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb E$&lt;/span&gt; is the symbol for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectation (or expected value)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fully understand the concept of &lt;em&gt;expected value&lt;/em&gt;, you need to understand the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_variable&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;random variable&lt;/em&gt;&lt;/a&gt;. An example should help you understand the idea behind the concept of a random variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you &lt;a href=&quot;https://en.wikipedia.org/wiki/Coin_flipping&quot; rel=&quot;nofollow noreferrer&quot;&gt;toss a coin&lt;/a&gt;. The outcome of this (random) experiment can either be &lt;em&gt;heads&lt;/em&gt; or &lt;em&gt;tails&lt;/em&gt;. Formally, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sample_space&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;sample space&lt;/em&gt;&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Omega = \{\text{heads}, \text{tails}\}$&lt;/span&gt;, is the set that contains the possible outcomes of a random experiment. The outcome (e.g. &lt;em&gt;heads&lt;/em&gt;) is the result of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;random process&lt;/a&gt;. A random variable is a function that we can associate with a random process so that we can more formally describe the random process. In this case, we can associate a random variable, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, with this random process of tossing a coin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;T(\omega) =&#xA;\begin{cases}&#xA;1, &amp;amp; \text{if } \omega = \text{heads}, \\[6pt]&#xA;0, &amp;amp; \text{if } \omega = \text{tails},&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\omega \in \Omega$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if the outcome of the random process is &lt;em&gt;heads&lt;/em&gt;, then the output of the associated random variable &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, else it is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can also associate with each random process (and thus with the corresponding random variable) a probability distribution, which, intuitively, describes the probability of occurrence of each possible outcome of the random process. In the case of the coin-flipping random variable (or process), assuming that the coin is &quot;fair&quot;, then the following function describes the probability of each outcome of the coin&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_T(t) =&#xA;\begin{cases}&#xA;\tfrac 12,&amp;amp; \text{if }t=1,\\[6pt]&#xA;\tfrac 12,&amp;amp; \text{if }t=0,&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, there is &lt;span class=&quot;math-container&quot;&gt;$\tfrac 12$&lt;/span&gt; probability that the outcome of the random process is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (heads) and &lt;span class=&quot;math-container&quot;&gt;$\tfrac 12$&lt;/span&gt; probability that it is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (tails).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you throw a coin &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times in the air, how many times will it land heads and tails? Of course, it will depend on the experiment. In the first experiment, you might get &lt;span class=&quot;math-container&quot;&gt;$\frac{3n}{4}$&lt;/span&gt; heads and &lt;span class=&quot;math-container&quot;&gt;$\frac{n}{4}$&lt;/span&gt; tails. In the second experiment, you might get &lt;span class=&quot;math-container&quot;&gt;$\frac{n}{2}$&lt;/span&gt; heads and &lt;span class=&quot;math-container&quot;&gt;$\frac{n}{2}$&lt;/span&gt; tails, and so on. If you repeat this experiment an infinite amount of times (of course, we can't do that, but imagine if we could do that), how many times do you &lt;strong&gt;expect&lt;/strong&gt; (on average) to get heads and tails? The expected value is the answer to this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the coin-tossing experiment, the outcomes are discrete (heads or tails), consequently, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is a &lt;em&gt;discrete random variable&lt;/em&gt;. In the case of a discrete random variable, the expected value &lt;strong&gt;is defined as&lt;/strong&gt; follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb E[T] = \sum_{t \in T} p(t) t$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the outcome of the random variable &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(t)$&lt;/span&gt; is the probability of such outcome. In other words, the expected value of a random variable &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is defined as a weighted sum of the values it can take, where the weights are the corresponding probabilities of occurrence. So, in the case of the coin-tossing experiment, the expected value is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbb E[T] &#xA;&amp;amp;= \sum_{t \in T} p(t) t\\&#xA;&amp;amp;= \frac{1}{2}1 + \frac{1}{2} 0\\&#xA;&amp;amp;=\frac{1}{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &lt;span class=&quot;math-container&quot;&gt;$\mathbb E[T] = \frac{1}{2}$&lt;/span&gt; mean? Intuitively, it means that half of the times the random process produces heads and half of the times it produces tails, assuming it is governed by the probability distribution &lt;span class=&quot;math-container&quot;&gt;$f_T(t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, if the probability distribution &lt;span class=&quot;math-container&quot;&gt;$f_T(t)$&lt;/span&gt; had been defined differently, then the expected value would also have been different, given that the expected value is defined as a function of the probability of occurrence of each outcome of the random process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your specific examples, &lt;span class=&quot;math-container&quot;&gt;$\mathbb E$&lt;/span&gt; is still the symbol for the expected value. For example, in the case of &lt;span class=&quot;math-container&quot;&gt;$Q^\pi \left(s,a \right) = \mathbb E \left[R_t|s_t = s, a_t = a, \pi \right]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Q^\pi \left(s,a \right)$&lt;/span&gt; is thus defined as the expected value of the random variable &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;, given that &lt;span class=&quot;math-container&quot;&gt;$s_t = s$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_t = a$&lt;/span&gt; and the policy is &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (so this is actually a &lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_expectation&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional expectation&lt;/a&gt;). In this specific case, &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; represents the &lt;em&gt;return&lt;/em&gt; at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, which, in reinforcement learning, is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_t = \sum_{k=0}^\infty \gamma^k r_{t+k+1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r_{t+k+1} \in \mathbb{R}$&lt;/span&gt; is the reward at time step &lt;span class=&quot;math-container&quot;&gt;$t+k+1$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; a random variable because it is assumed that the underlying environment is a random process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not always easy to intuitively understand the expected value of a random variable. For example, in the case of a coin-flipping random process, the expected value &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; should be intuitive (given that it is the average of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;), but, in the case of &lt;span class=&quot;math-container&quot;&gt;$Q^\pi \left(s,a \right)$&lt;/span&gt;, at first glance, it is not clear what the expected value should be (hence the need for algorithms such as Q-learning), given that it depends on the rewards, which depend on the dynamics of the environment. However, the intuition behind the concept of the expected value and the calculation (provided the associated random variable is discrete) does not change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case there is more than one random variable involved in the calculation of the expected value, then we also need to specify the random variable the expected value is being calculated with respect to, hence the subscripts of the expected value in your examples. See, for example, &lt;a href=&quot;https://stats.stackexchange.com/q/72613/82135&quot;&gt;Subscript notation in expectations&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-07T22:03:52.043" LastActivityDate="2019-11-07T22:03:52.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14162" PostTypeId="1" AcceptedAnswerId="14171" CreationDate="2019-08-27T17:54:19.253" Score="2" ViewCount="109" Body="&lt;p&gt;If AlphaZero was always playing the best moves it would just generate the same training game over and over again. So where does the randomness come from? When does it decide not to play the most optimal move?&lt;/p&gt;&#xA;" OwnerUserId="19604" LastEditorUserId="2444" LastEditDate="2019-08-31T23:58:50.977" LastActivityDate="2019-08-31T23:58:50.977" Title="When does AlphaZero play suboptimal moves?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;alphazero&gt;&lt;greedy-ai&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14165" PostTypeId="1" AcceptedAnswerId="14199" CreationDate="2019-08-27T20:11:47.490" Score="3" ViewCount="215" Body="&lt;p&gt;I'm just starting to learn about neural networking and I decided to study a simple 3-input perceptron to get started with. I am also only using binary inputs to gain a full understanding of how the perceptron works. I'm having difficulty understanding why some training outputs work and others do not. I'm guessing that it has to do with the linear separability of the input data, but it's unclear to me how this can easily be determined. I'm aware of the graphing line test, but it's unclear to me how to plot the input data to fully understand what will work and what won't work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is quite a bit of information that follows. But it's all very simple. I'm including all this information to be crystal clear on what I'm doing and trying to understand and learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a schematic graphic of the simple 3-input perceptron I'm modeling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://live.staticflickr.com/65535/48621620907_284a9e1f87_o.png&quot; width=&quot;596&quot; height=&quot;278&quot; alt=&quot;3 Input Perceptron&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because it only has 3 inputs and they are binary (0 or 1), there are only 8 possible combinations of inputs. However, this also allows for 8 possible outputs. This allows for training of 256 possible outputs. In other words, the perceptron can be trained to recognize more than one input configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's call the inputs 0 thru 7 (all the possible configurations of a 3-input binary system). But we can train the perceptron to recognize more than just one input. In other words, we can train the perceptron to fire for say any input from 0 to 3 and not for inputs 4 thru 7. And all those possible combinations add up to 256 possible training input states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of these training input states work, and others do not. I'm trying to learn how to determine which training sets are valid and which are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written the following program in Python to emulate this Perceptron through all 256 possible training states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code for this emulation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;np.set_printoptions(formatter={'float': '{: 0.1f}'.format})&#xA;&#xA;# Perceptron math fucntions. &#xA;def sigmoid(x):&#xA;    return 1 / (1 + np.exp(-x))&#xA;def sigmoid_derivative(x):&#xA;    return x * (1 - x)&#xA;# END Perceptron math functions.&#xA;&#xA;# The first column of 1's is used as the bias.  &#xA;# The other 3 cols are the actual inputs, x3, x2, and x1 respectively&#xA;training_inputs = np.array([[1, 0, 0, 0],&#xA;                         [1, 0, 0, 1],&#xA;                         [1, 0, 1, 0],&#xA;                         [1, 0, 1, 1],&#xA;                         [1, 1, 0, 0],&#xA;                         [1, 1, 0, 1],&#xA;                         [1, 1, 1, 0],&#xA;                         [1, 1, 1, 1]])&#xA;&#xA;# Setting up the training outputs data set array                         &#xA;num_array = np.array&#xA;num_array = np.arange(8).reshape([1,8])&#xA;num_array.fill(0)&#xA;&#xA;for num in range(25):&#xA;    bnum = bin(num).replace('0b',&quot;&quot;).rjust(8,&quot;0&quot;)&#xA;    for i in range(8):&#xA;        num_array[0,i] = int(bnum[i])&#xA;&#xA;    training_outputs = num_array.T&#xA;# training_outputs will have the array form: [[n,n,n,n,n,n,n,n]]&#xA;# END of setting up training outputs data set array                      &#xA;&#xA;    # -------  BEGIN Perceptron functions ----------&#xA;    np.random.seed(1)&#xA;    synaptic_weights = 2 * np.random.random((4,1)) - 1&#xA;    for iteration in range(20000):&#xA;        input_layer = training_inputs&#xA;        outputs = sigmoid(np.dot(input_layer, synaptic_weights))&#xA;        error = training_outputs - outputs&#xA;        adjustments = error * sigmoid_derivative(outputs)&#xA;        synaptic_weights += np.dot(input_layer.T, adjustments)&#xA;    # -------  END Perceptron functions ----------&#xA;&#xA;&#xA;    # Convert to clean output 0, 0.5, or 1 instead of the messy calcuated values.&#xA;    # This is to make the printout easier to read.&#xA;    # This also helps with testing analysis below.&#xA;    for i in range(8):&#xA;        if outputs[i] &amp;lt;= 0.25:&#xA;            outputs[i] = 0&#xA;        if (outputs[i] &amp;gt; 0.25 and outputs[i] &amp;lt; 0.75):&#xA;            outputs[i] = 0.5&#xA;        if outputs[i] &amp;gt; 0.75:&#xA;            outputs[i] = 1&#xA;    # End convert to clean output values.&#xA;&#xA;    # Begin Testing Analysis&#xA;    # This is to check to see if we got the correct outputs after training.&#xA;    evaluate = &quot;Good&quot;&#xA;    test_array = training_outputs&#xA;    for i in range(8):&#xA;        # Evaluate for a 0.5 error.&#xA;        if outputs[i] == 0.5:&#xA;            evaluate = &quot;The 0.5 Error&quot;&#xA;            break&#xA;        # Evaluate for incorrect output&#xA;        if outputs[i] != test_array[i]:&#xA;            evaluate = &quot;Wrong Answer&quot;&#xA;    # End Testing Analysis&#xA;&#xA;    # Printout routine starts here:&#xA;    print_array = test_array.T&#xA;    print(&quot;Test#: {0}, Training Data is: {1}&quot;.format(num, print_array[0]))&#xA;    print(&quot;{0}, {1}&quot;.format(outputs.T, evaluate))&#xA;    print(&quot;&quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And when I run this code I get the following output for the first 25 training tests.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Test#: 0, Training Data is: [0 0 0 0 0 0 0 0]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0]], Good&#xA;&#xA;Test#: 1, Training Data is: [0 0 0 0 0 0 0 1]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0]], Good&#xA;&#xA;Test#: 2, Training Data is: [0 0 0 0 0 0 1 0]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0]], Good&#xA;&#xA;Test#: 3, Training Data is: [0 0 0 0 0 0 1 1]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0]], Good&#xA;&#xA;Test#: 4, Training Data is: [0 0 0 0 0 1 0 0]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0]], Good&#xA;&#xA;Test#: 5, Training Data is: [0 0 0 0 0 1 0 1]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.0]], Good&#xA;&#xA;Test#: 6, Training Data is: [0 0 0 0 0 1 1 0]&#xA;[[ 0.0 0.0 0.0 0.0 0.5 0.5 0.5 0.5]], The 0.5 Error&#xA;&#xA;Test#: 7, Training Data is: [0 0 0 0 0 1 1 1]&#xA;[[ 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0]], Good&#xA;&#xA;Test#: 8, Training Data is: [0 0 0 0 1 0 0 0]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0]], Good&#xA;&#xA;Test#: 9, Training Data is: [0 0 0 0 1 0 0 1]&#xA;[[ 0.0 0.0 0.0 0.0 0.5 0.5 0.5 0.5]], The 0.5 Error&#xA;&#xA;Test#: 10, Training Data is: [0 0 0 0 1 0 1 0]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 0.0 1.0 0.0]], Good&#xA;&#xA;Test#: 11, Training Data is: [0 0 0 0 1 0 1 1]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 0.0 1.0 1.0]], Good&#xA;&#xA;Test#: 12, Training Data is: [0 0 0 0 1 1 0 0]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0]], Good&#xA;&#xA;Test#: 13, Training Data is: [0 0 0 0 1 1 0 1]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0]], Good&#xA;&#xA;Test#: 14, Training Data is: [0 0 0 0 1 1 1 0]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0]], Good&#xA;&#xA;Test#: 15, Training Data is: [0 0 0 0 1 1 1 1]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0]], Good&#xA;&#xA;Test#: 16, Training Data is: [0 0 0 1 0 0 0 0]&#xA;[[ 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0]], Good&#xA;&#xA;Test#: 17, Training Data is: [0 0 0 1 0 0 0 1]&#xA;[[ 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0]], Good&#xA;&#xA;Test#: 18, Training Data is: [0 0 0 1 0 0 1 0]&#xA;[[ 0.0 0.0 0.5 0.5 0.0 0.0 0.5 0.5]], The 0.5 Error&#xA;&#xA;Test#: 19, Training Data is: [0 0 0 1 0 0 1 1]&#xA;[[ 0.0 0.0 0.0 1.0 0.0 0.0 1.0 1.0]], Good&#xA;&#xA;Test#: 20, Training Data is: [0 0 0 1 0 1 0 0]&#xA;[[ 0.0 0.5 0.0 0.5 0.0 0.5 0.0 0.5]], The 0.5 Error&#xA;&#xA;Test#: 21, Training Data is: [0 0 0 1 0 1 0 1]&#xA;[[ 0.0 0.0 0.0 1.0 0.0 1.0 0.0 1.0]], Good&#xA;&#xA;Test#: 22, Training Data is: [0 0 0 1 0 1 1 0]&#xA;[[ 0.0 0.0 0.0 1.0 0.0 1.0 1.0 1.0]], Wrong Answer&#xA;&#xA;Test#: 23, Training Data is: [0 0 0 1 0 1 1 1]&#xA;[[ 0.0 0.0 0.0 1.0 0.0 1.0 1.0 1.0]], Good&#xA;&#xA;Test#: 24, Training Data is: [0 0 0 1 1 0 0 0]&#xA;[[ 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0]], Wrong Answer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the most part, it appears to be working. But there are situations where it clearly does not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the labels in two different ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first type of error is &quot;The 0.5 Error&quot; which is easy to see. It should never return any output of 0.5 in this situation. Everything should be binary. The second type of error is when it reports the correct binary outputs but they don't match what it was trained to recognize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to understand the cause of these errors. I'm not interested in trying to correct the errors as I believe these are valid errors. In other words, these are situations where the perceptron is simply incapable of being trained for. And that's ok.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to learn is why these cases are invalid. I'm suspecting that they have something to do with the input data not being linearly separable in these situations. But if that's the case, then how do I go about determining which cases are not linearly separable? If I could understand how to do that I would be very happy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, are the reasons why it doesn't work in specific cases the same? In other words, are both types of errors caused by linear inseparability of the input data? Or is there more than one condition that causes a Perceptron to fail in certain training situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28240" LastEditorUserId="2444" LastEditDate="2019-08-27T23:54:01.807" LastActivityDate="2019-08-29T18:51:46.833" Title="What are the reasons a perceptron is not able to learn?" Tags="&lt;training&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14167" PostTypeId="1" AcceptedAnswerId="14168" CreationDate="2019-08-27T21:50:48.243" Score="5" ViewCount="185" Body="&lt;p&gt;Let's assume that there is a sequence of pairs &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i), (x_{i+1}, y_{i+1}), \dots$&lt;/span&gt; of observations and corresponding labels. Let's also assume that the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is considered as independent variable and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is considered as the variable that depends on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. So, in supervised learning, one wants to learn the function &lt;span class=&quot;math-container&quot;&gt;$y=f(x)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can reinforcement learning be used to learn &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (possibly, even learning the symbolic form of &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just some sketches how can it be done: &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; can be considered as the environment and each &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; defines some set of possible &quot;actions&quot; - possible symbolic form of &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; or possible numerical values of parameters for &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; (if the symbolic form is fized). And concrete selected action/functional form &lt;span class=&quot;math-container&quot;&gt;$f(x, a)$&lt;/span&gt; (a - set of parameters) can be assigned reward from the loss function: how close the observation &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt; is to the value that can be inferred from &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there ideas or works of RL along the framework that I provided in the previous passage?&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2019-08-27T23:22:30.097" LastActivityDate="2020-07-07T15:27:16.643" Title="Can supervised learning be recast as reinforcement learning problem?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;supervised-learning&gt;&lt;function-approximation&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14168" PostTypeId="2" ParentId="14167" CreationDate="2019-08-27T23:14:02.027" Score="9" Body="&lt;p&gt;Any supervised learning (SL) problem can be cast as an equivalent reinforcement learning (RL) one.&lt;/p&gt;&#xA;&lt;p&gt;Suppose you have the training dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D} = \{ (x_i, y_i \}_{i=1}^N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is an observation and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; the corresponding label. Then let &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; be a state and let &lt;span class=&quot;math-container&quot;&gt;$f(x_i) = \hat{y}_i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is your (current) model, be an action. So, the predicted label of observation &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; corresponds to the action taken in state &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;. The reward received after having taken action &lt;span class=&quot;math-container&quot;&gt;$f(x_i)$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; can then be defined as the loss &lt;span class=&quot;math-container&quot;&gt;$|f(x_i) - y_i|$&lt;/span&gt; (or any other suitable loss).&lt;/p&gt;&#xA;&lt;p&gt;The minimization of this loss is then equivalent to the maximization of the (expected) reward. Therefore, in theory, you could use trajectories of the form &lt;span class=&quot;math-container&quot;&gt;$$T=\{(x_1, f(x_1), |f(x_1) - y_1|),  \dots, (x_N, f(x_N), |f(x_N) - y_N|)\}$$&lt;/span&gt; to learn a value function &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (for example, with Q-learning) or a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, which then, given a new state &lt;span class=&quot;math-container&quot;&gt;$x_{\text{new}}$&lt;/span&gt; (an observation) produces an action &lt;span class=&quot;math-container&quot;&gt;$f(x_{\text{new}})$&lt;/span&gt; (the predicted label).&lt;/p&gt;&#xA;&lt;p&gt;However, note that the learned policy might not be able to generalise to observations not present in the training dataset. Moreover, although it is possible to solve an SL problem as an RL problem, in practice, this may not be the most appropriate approach (i.e. it may be inefficient).&lt;/p&gt;&#xA;&lt;p&gt;For more details, read the paper &lt;a href=&quot;https://pdfs.semanticscholar.org/ccec/df227475db9be6f0fa8f431c1db51e9dcd83.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning and its Relationship to Supervised Learning&lt;/a&gt; (2004) by Barto and Dietterich, who give a good overview of supervised and reinforcement learning and their relationship. The paper &lt;a href=&quot;https://link.springer.com/article/10.1007/BF00115009&quot; rel=&quot;noreferrer&quot;&gt;Learning to predict by the methods of temporal differences&lt;/a&gt; (1988), by Richard Sutton, should also give you an overview of reinforcement learning from a supervised learning perspective. However, note that this does not mean that a reinforcement learning problem can be cast as an equivalent supervised learning one. See section &lt;em&gt;1.3.3 Converting Reinforcement Learning to Supervised Learning&lt;/em&gt; of the mentioned paper &lt;a href=&quot;https://pdfs.semanticscholar.org/ccec/df227475db9be6f0fa8f431c1db51e9dcd83.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning and its Relationship to Supervised Learning&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;p&gt;Reinforcement learning can thus be used for classification and regression tasks. See, for example, &lt;a href=&quot;http://www.maths.lth.se/sminchisescu/media/papers/mps-rlcvpr16.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning for Visual Object Detection&lt;/a&gt; (2016) by Mathe et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-07T15:27:16.643" LastActivityDate="2020-07-07T15:27:16.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14171" PostTypeId="2" ParentId="14162" CreationDate="2019-08-28T08:49:14.597" Score="2" Body="&lt;p&gt;During the self-play training process, AlphaZero does not greedily play only the moves it thinks are &quot;best&quot; (which would normally be the move with the highest visit count leading out of the root node of the MCTS search tree). Instead, for the purpose of generating a more diverse set of experience, it samples moves &lt;strong&gt;proportionally&lt;/strong&gt; to the visit counts. This means that in any given situation encountered during self-play, the move considered to be &quot;optimal&quot; will still have the largest probability of getting picked, but other moves will also have smaller probabilities of getting picked. In theory, it might even sometimes pick the move that it expects to be the &lt;strong&gt;worst&lt;/strong&gt; one (very rarely)!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I recall correctly, they only did what I describe above for the first 30 moves of any game, and afterwards move on to greedy play. This still results in a very diverse set of 30-move-starts for every game it experiences though. I'm not 100% sure if I remember this detail correctly though, maybe they only did this earlier (in AlphaGo Zero for example), and no longer do it in AlphaZero. Would have to check the paper to make sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, whenever they start a new search process, they perturb the prior probabilities assigned by the learned policy network to all the moves available in the root node. This is done in a non-deterministic way using Dirichlet noise. I think this is not really explicitly mentioned in the AlphaZero paper, but it is in the Supplementary Materials (and also in the AlphaGo Zero paper?). Anyway, this also means that if precisely the same game state is encountered twice in two different games of self-play, the search behaviour may be slightly different due to the introduced stochasticity, and hence it may come to a different conclusion as to what move is &quot;optimal&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-08-28T08:49:14.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14172" PostTypeId="1" CreationDate="2019-08-28T10:24:53.117" Score="4" ViewCount="307" Body="&lt;p&gt;I recently read &lt;a href=&quot;https://arxiv.org/abs/1610.10099&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bytenet&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1609.03499&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wavenet&lt;/a&gt; and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the decoder are similar to Wavenet. Following the trends from NLP where seq2seq models seem to perform better, I find it strange that I couldn't find any paper that compares the two. Are there any drawbacks of Bytenet over Wavenet other than the computation time?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2019-08-28T14:01:03.520" LastActivityDate="2021-04-21T06:01:15.153" Title="What are the differences between Bytenet and Wavenet?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;machine-translation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14173" PostTypeId="1" CreationDate="2019-08-28T13:00:39.093" Score="3" ViewCount="84" Body="&lt;p&gt;I wanted to use True Positive (and True Negative) in my cost function to make to modify the ROC shape of my classifier. Someone told me and I read that it is not differentiable and therefore not usable as a cost function for a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example where 1 is positive and 0 negative I deduce the following equation for True Positive (&lt;span class=&quot;math-container&quot;&gt;$\hat y = prediction, y = label$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ TP = \bf(\hat{y}^Ty) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial TP}{\partial \bf y} = \bf y $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following for True Negative:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ TN = \bf(\hat{y}-1)^T(y-1) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial TN}{\partial \bf y} = \bf \hat y^T -1    $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The False Positive: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ FP = - \bf  (\hat y^T-1) y $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial FP}{\partial \bf y} = - \bf ( \hat y^T - 1)    $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The False Negative: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ FN = \bf  \hat y^T (y-1) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial FN}{\partial \bf y} = \bf \hat y    $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All equations seem differentiable to me. Can someone explain where I went wrong?&lt;/p&gt;&#xA;" OwnerUserId="27843" LastEditorUserId="27933" LastEditDate="2019-09-06T07:16:26.147" LastActivityDate="2019-09-06T07:16:26.147" Title="Using True Positive as a Cost Function" Tags="&lt;deep-learning&gt;&lt;metric&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14174" PostTypeId="1" AcceptedAnswerId="14200" CreationDate="2019-08-28T13:48:14.463" Score="2" ViewCount="68" Body="&lt;p&gt;I'm trying to understand the intuition behind how the Content Loss is calculated in a Neural Style Transfer. I'm reading from an articles: &lt;a href=&quot;https://medium.com/mlreview/making-ai-art-with-style-transfer-using-keras-8bb5fa44b216&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/mlreview/making-ai-art-with-style-transfer-using-keras-8bb5fa44b216&lt;/a&gt; ,  that explains the implementation of Neural Style Transfer, from the Content loss function:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MiyOW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MiyOW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article explains that: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;F and P are matrices with a number of rows equal to N and a number of columns equal to M.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;N is the number of filters in layer l and M is the number of spatial elements in the feature map (height times width) for layer l.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From the code below for getting the features/content representation from particular Conv layers, I didn't quite understand how it works. Basically I printed out the output of every line of code to try to make it easier, but it still left a number of questions to be asked, which I listed below the code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_feature_reps(x, layer_names, model):&#xA;    &quot;&quot;&quot;&#xA;    Get feature representations of input x for one or more layers in a given model.&#xA;    &quot;&quot;&quot;&#xA;    featMatrices = []&#xA;    for ln in layer_names:&#xA;        selectedLayer = model.get_layer(ln)&#xA;        featRaw = selectedLayer.output&#xA;        featRawShape = K.shape(featRaw).eval(session=tf_session)&#xA;        N_l = featRawShape[-1]&#xA;        M_l = featRawShape[1]*featRawShape[2]&#xA;        featMatrix = K.reshape(featRaw, (M_l, N_l))&#xA;        featMatrix = K.transpose(featMatrix)&#xA;        featMatrices.append(featMatrix)&#xA;    return featMatrices&#xA;&#xA;def get_content_loss(F, P):&#xA;    cLoss = 0.5*K.sum(K.square(F - P))&#xA;    return cLoss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;1- For the line &lt;code&gt;featRaw = selectedLayer.output&lt;/code&gt;, when I print &lt;code&gt;featRaw&lt;/code&gt;, I get the output:&#xA;&lt;code&gt;Tensor(&quot;block4_conv2/Relu:0&quot;, shape=(1, 64, 64, 512), dtype=float32)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a- &lt;code&gt;Relu:0&lt;/code&gt; does this mean Relu activation has not yet been applied?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;b- Also I presume we're outputing the feature maps outputs from &lt;code&gt;block4_conv2&lt;/code&gt;, not the filters/kernels themselves, correct?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;c- Why is there an axis of 1 at the start? My understanding of Conv layers is that they're simply made up from the number of filters/kernels (with shape-height, width, depth) to apply to the input.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;d- Is &lt;code&gt;selectedLayer.output&lt;/code&gt; simply outputs the shape of the Conv layer, or does the output object also hold other information like the pixel values from the output feature maps of the layer?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;2- With the line: &lt;code&gt;featMatrix = K.reshape(featRaw, (M_l, N_l)&lt;/code&gt; where printing &lt;code&gt;featMatrix&lt;/code&gt; would output: &lt;code&gt;Tensor(&quot;Reshape:0&quot;, shape=(4096, 512), dtype=float32)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a- This is where I'm confused the most. So to get the feature/content representation of a particular Conv layer of an image, we simply create a matrix of 2 dimensions, the first being the number of filters and the other being the area of the filter/kernel (height * width). That doesn't make sense! How do we get unique feature of an image from just that?!! We're not retrieving any pixel values from a feature map. We're simply getting the area size of filter/kernel and the number of filters, but not retrieving any of the content (pixel values) itself!! &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;b- Also the final &lt;code&gt;featMatrix&lt;/code&gt; is transposed - i.e. &lt;code&gt;featMatrix = K.transpose(featMatrix)&lt;/code&gt; with the output &lt;code&gt;Tensor(&quot;transpose:0&quot;, shape=(512, 4096), dtype=float32)&lt;/code&gt;. Why is that (i.e. why reverse the axis)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;3 - Finally I want to know, once we retrieve the content representation, how can I output that in both as a numpy array and save it as an image? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="25360" LastEditorUserId="25360" LastEditDate="2019-08-29T11:54:17.423" LastActivityDate="2019-08-29T19:20:15.600" Title="Understanding the intuition behind Content Loss (Neural Style Transfer)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14175" PostTypeId="1" CreationDate="2019-08-28T14:27:13.083" Score="3" ViewCount="572" Body="&lt;p&gt;Why do model-based methods use fewer samples than model-free methods? Here, I'm specifically referring to model-based methods in which we have to learn a policy and model. I can only think of two reasons for this question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;We can potentially obtain more samples from the learned model, which may speed up the learning speed. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Models allow us to predict the future states and run simulations. This may lead to more valuable transitions, whereby speeding up learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But I heavily doubt this is the whole story. Sincerely, I hope someone could share a more detailed explanation for this question.&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="2444" LastEditDate="2020-10-13T08:31:34.797" LastActivityDate="2020-10-13T08:31:34.797" Title="Why are model-based methods more sample efficient than model-free methods?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;&lt;sample-efficiency&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14176" PostTypeId="2" ParentId="14172" CreationDate="2019-08-28T15:01:09.963" Score="0" Body="&lt;p&gt;My conclusion is the same as yours that there doesn't seem to be any published comparison of the two models.  ByteNet is computationally expensive and requires a lot of parameters.  WaveNet improves on ByteNet's efficiency, as you mentioned, and I believe that is the main difference. &lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2019-08-28T15:01:09.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14178" PostTypeId="1" CreationDate="2019-08-28T17:02:16.643" Score="0" ViewCount="52" Body="&lt;p&gt;K-means tries to find centroid and then clusters around the centroids. But what if we want to cluster based on the complement?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose we have a group of animals and we want to cluster Dogs, Cats, (Not Dogs and Not Cats). The 3rd category will not arise from mean clustering.&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2019-09-01T00:02:45.550" LastActivityDate="2021-05-16T13:07:00.670" Title="How can I cluster based on the complementary categories?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14182" PostTypeId="2" ParentId="14175" CreationDate="2019-08-28T19:34:39.343" Score="2" Body="&lt;p&gt;In this Medium article I found [1] it is quite well explained what is behind the better model efficiency in model based RL in comparison to model free one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Main difference between those two is like you said, that the model helps in finding the correct path more efficiently because of the existence of the model. Maybe you cannot find new samples (point 1) but you know better the whole inner logic about the system and instead of just knowing what to do with the specific sample, you can relate it to the whole picture (sort of like in point 2: you can play with the choices) and make more profound calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article had a comparison, which told that you are writing a map in a city about every possible direction you can take when you are model-based and while in model-free you can enter specific places and remember which direction was best based on last visits but you still never know where you are coming nor going exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if you think you're teaching a taxi driver on a big city with many signs and rules, model based guy would drive more precisely sooner because the sign language (the inner logic and model of the city) helps them on understanding the map sooner than just reacting crossing by crossing somewhat by chance all the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample efficiency tells what is the amount of information fetched from one sample [2]. Model-based machine can adjust model, maybe make some calculations about expected rewards AND after that the same as model-free, adjust the common policy. Model-free does only have the policy. Again the taxi guys: model-free guys know that last time and second last I stopped in crossing, model-based guy knows also it was due to red lights in pole. Third time model-free guy is the first in row and BANG - hits the crossing car. Next time rule is there comes sometimes cars and model-based guy knew that from the first place.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My sources:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://medium.com/the-official-integrate-ai-blog/understanding-reinforcement-learning-93d4e34e5698&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/the-official-integrate-ai-blog/understanding-reinforcement-learning-93d4e34e5698&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://ai.stackexchange.com/questions/5246/what-is-sample-efficiency-and-how-can-importance-sampling-be-used-to-achieve-it&quot;&gt;What is sample efficiency, and how can importance sampling be used to achieve it?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2019-08-28T19:34:39.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14184" PostTypeId="1" CreationDate="2019-08-28T22:59:13.337" Score="2" ViewCount="47" Body="&lt;p&gt;I am starting to create my first bot with Microsoft Bot Framework with the help of Azure, initially I want to know where all the conversations the user has with the bot are stored, so then get a log of all the conversations that have been held.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have some answers stored in knowledge bases using QnA Maker, for certain questions that you can answer, I want to know where the questions that were not answered or better that the bot could not answer are stored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am asking the users to write their question in a feedback form when they don't get a response from my bot. This is taking up my time and also annoys the user as they have to type more. I want my bot to collect these questions and store them in a database.&lt;/p&gt;&#xA;" OwnerUserId="28278" LastEditorUserId="27933" LastEditDate="2019-08-30T14:52:28.143" LastActivityDate="2019-08-30T14:52:28.143" Title="How can we log user-bot conversations using the Microsoft Bot Framework?" Tags="&lt;chat-bots&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14187" PostTypeId="2" ParentId="14173" CreationDate="2019-08-29T07:56:38.660" Score="1" Body="&lt;p&gt;The vector functions for true positive, false positive etc all make use of the &quot;magic&quot; numbers &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; used to represent Boolean values. They are convenience methods that you can use in a numerical library, but you do need to be aware of the fundamental Boolean nature of the data. The &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; values allow the maths for calculating TP et al, but are not fundamental to it, they are a representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your derivations of gradients for the functions you give seem correct, barring the odd typo. However, the gradient doesn't really apply to the value of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;, because all components of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; are either &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. The idea that you could increase &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; slightly where  &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\hat{y}}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in order to increase the value of the TP metric slightly has no basis. Instead the only valid changes to make an improvement are to modify FN values from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could probably still use your derivations as a gradient for optimisation (it would not be the only time in machine learning that something does not quite apply theoretically but you could still use it in practice). However, you then immediately hit the problem of how the values of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; have been discretised to  &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; - &lt;em&gt;that&lt;/em&gt; function will not be differentiable, and it will prevent you back propagating your gradients to the neural network weights that you want to change. If you fix that follow-on problem using a smoother function (e.g. a sigmoid) then you are likely to end up with something close to either cross-entropy loss or the perceptron update step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, although what you have been told is an over-simplification, you will not find a way to improve the performance of your classifier by adding cost functions based directly on TP, FP etc. That is what binary cross-entropy loss is already doing. There are other, perhaps more fruitful avenues of investigation - hyperparameter searches, regularisation, ensembling, and if you have an unbalanced data set then consider weighting the true/false costs.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-08-29T08:12:00.430" LastActivityDate="2019-08-29T08:12:00.430" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14188" PostTypeId="1" CreationDate="2019-08-29T09:00:25.407" Score="1" ViewCount="23" Body="&lt;p&gt;I want to train a WGAN where the convolution layers in the critic are only allowed to have non-negative weights (for a technical reason). The biases, nonetheless, can take both +/- values. There is no constraint on the generator weights.&lt;/p&gt;&#xA;&lt;p&gt;I did a toy experiment on MNIST and observed that the performance is significantly worse than a regular WGAN.&lt;/p&gt;&#xA;&lt;p&gt;What could be the reason? Can you suggest some architectural modifications so that the nonnegativity constraint doesn't severely impair the model capacity?&lt;/p&gt;&#xA;" OwnerUserId="28286" LastEditorUserId="2444" LastEditDate="2021-01-25T19:02:52.490" LastActivityDate="2021-01-25T19:02:52.490" Title="Wasserstein GAN with non-negative weights in the critic" Tags="&lt;generative-adversarial-networks&gt;&lt;convolutional-layers&gt;&lt;constrained-optimization&gt;&lt;wasserstein-gan&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14189" PostTypeId="1" CreationDate="2019-08-29T09:46:27.250" Score="3" ViewCount="51" Body="&lt;p&gt;I have a very big table with lots of names and how much they are searched by date. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to find trending patterns. When does a name rise and when does it fall. Without knowing the name or the pattern before.&#xA;The rise could be during the seasons of the year but also during a week.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like a 'warm hat' is trending in winter and falling in summer.&#xA;Or searches for a &quot;board game&quot; might rise on Sunday and decrease on Monday.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The table looks simplified like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;winter gloves, 2014-01-01, 200&#xA;warm hat, 2014-01-01, 300&#xA;swimming short, 2014-01-01, 1&#xA;sunscreen, 2014-01-01, 2&#xA;....&#xA;winter gloves, 2014-07-01, 1&#xA;warm hat, 2014-07-01, 1&#xA;swimming short, 2014-07-01, 200&#xA;sunscreen, 2014-07-01, 300&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which algorithms should I have a look at? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any hint,&#xA;Joerg&lt;/p&gt;&#xA;" OwnerUserId="28289" LastEditorUserId="28289" LastEditDate="2019-08-29T10:58:34.010" LastActivityDate="2019-08-30T18:23:24.510" Title="Algorithm for seasonal trends" Tags="&lt;data-mining&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14190" PostTypeId="2" ParentId="13725" CreationDate="2019-08-29T10:07:13.740" Score="0" Body="&lt;p&gt;&lt;strong&gt;Central Premises&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, computability of randomness in conjunction to logic, is unfortunately/fortunately a very technical topic.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot; ... That stochastic process is part of an algorithm, which is a set of instructions that must be valid for the program to compute.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;... Is randomness anti-logical?&quot; ~ DukeZhou (Stack Exchange user, opening poster)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This answer is about: randomness and chaos; and how they relate to logic and computability.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;What is randomness and where does it come from?&#xA;  This is one scary place to venture in. We take for granted the randomness in our reality. We compensate for that randomness with probability theory. However, is randomness even real or is it just a figment of our lack of intelligence? That is, does what we describe as randomness just a substitute for our uncertainty about reality? Is randomness just a manifestation of something else?&quot; &#xA;  — &quot;Medium.&quot; Medium, &amp;lt; &lt;a href=&quot;http://medium.com/intuitionmachine/there-is-no-randomness-only-chaos-and-complexity-c92f6&quot; rel=&quot;nofollow noreferrer&quot;&gt;medium.com/intuitionmachine/there-is-no-randomness-only-chaos-and-complexity-c92f6&lt;/a&gt; &gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Many natural intensional properties in artificial and natural languages are hard to compute. We show that randomized algorithms are often necessary to have good estimators of natural properties and to verify some specific relations. We concentrate on the reliability of queries to show the advantage of randomized algorithms in uncertain cognitive worlds.&quot; — de Rougemont, Michel. &quot;Logic, randomness and cognition.&quot; Logic, Thought and Action. Springer, Dordrecht, 2005. 497-506.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Layperson's Explanations&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, quantum chaos in relation to randomness, is a profoundly technical topic. I have managed to track down sources that relatively aren't overly technical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point, this Wikipedia article is worth reading:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://simple.wikipedia.org/wiki/Chaos_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://simple.wikipedia.org/wiki/Chaos_theory&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can continue and read this particular Medium post:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://medium.com/intuitionmachine/there-is-no-randomness-only-chaos-and-complexity-c92f6dccd7ab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/intuitionmachine/there-is-no-randomness-only-chaos-and-complexity-c92f6dccd7ab&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For profoundly technical topics, I recommend this book series, as they are written by experts in basic technical terms, for the laypersons wanting to study technical topics:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Very_Short_Introductions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Very_Short_Introductions&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://global.oup.com/academic/search?q=randomness%20a%20very%20short%20introduction&amp;amp;cc=us&amp;amp;lang=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;I recommend reading&lt;/strong&gt;&lt;/a&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Chaos: A Very Short Introduction&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Probability: A Very Short Introduction&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Fractals: A Very Short Introduction&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other References for the Layperson&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.random.org/randomness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.random.org/randomness/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://plato.stanford.edu/entries/computability/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/computability/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://plato.stanford.edu/entries/chance-randomness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/chance-randomness/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://plato.stanford.edu/entries/qt-quantlog/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/qt-quantlog/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://plato.stanford.edu/entries/chaos/#SomBroImpCha&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some broader implications of chaos&lt;/a&gt; [Link to Stanford Encyclopedia of Philosophy].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.newscientist.com/article/mg22530120-500-chance-is-anything-in-the-universe-truly-random/&quot; rel=&quot;nofollow noreferrer&quot;&gt;When I think of randomness, I'm inclined to think in cosmological terms. Is randomness is a structural property of the universe?&lt;/a&gt; Is anything in the universe truly random?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Technical Explanations&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;In mathematical logic, independence is the unprovability of a sentence from other sentences.&quot; Wikipedia contributors. — &quot;Independence (mathematical logic).&quot;  &lt;em&gt;Wikipedia, The Free Encyclopedia&lt;/em&gt;. Wikipedia, The Free Encyclopedia, 3 Feb. 2019. Web. 29 Aug. 2019.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;We propose a link between logical independence and quantum physics. We demonstrate that quantum systems in the eigenstates of Pauli group operators are capable of encoding mathematical axioms and show that Pauli group quantum measurements are capable of revealing whether or not a given proposition is logically dependent on the axiomatic system. Whenever a mathematical proposition is logically independent of the axioms encoded in the measured state, the measurement associated with the proposition gives random outcomes. This allows for an experimental test of logical independence. Conversely, it also allows for an explanation of the probabilities of random outcomes observed in Pauli group measurements from logical independence without invoking quantum theory. The axiomatic systems we study can be completed and are therefore not subject to Gödel's incompleteness theorem.&quot; Paterek, Tomasz, et al. &quot;Logical independence and quantum randomness.&quot; — &lt;em&gt;New Journal of Physics&lt;/em&gt; 12.1 (2010): 013019.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other Technical Explanations, Sources, References, and Further Reading:--&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://iopscience.iop.org/article/10.1088/1367-2630/12/1/013019/meta&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://iopscience.iop.org/article/10.1088/1367-2630/12/1/013019/meta&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_chaos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Quantum_chaos&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://arxiv.org/pdf/0708.1362.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/0708.1362.pdf&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://arxiv.org/pdf/1701.01107.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1701.01107.pdf&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;http://csc.ucdavis.edu/~cmg/papers/idep.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://csc.ucdavis.edu/~cmg/papers/idep.pdf&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.datasciencecentral.com/profiles/blogs/logistic-map-chaos-randomness-and-quantum-algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.datasciencecentral.com/profiles/blogs/logistic-map-chaos-randomness-and-quantum-algorithms&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.deepdyve.com/lp/wiley/non-linear-dynamics-complexity-and-randomness-algorithmic-foundations-9vxIJOTT6u&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.deepdyve.com/lp/wiley/non-linear-dynamics-complexity-and-randomness-algorithmic-foundations-9vxIJOTT6u&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Independence_(mathematical_logic)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Independence_(mathematical_logic)&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I usually do not use Medium or Quora as a source, with some exceptions. I have  chosen to do so here.&lt;/li&gt;&#xA;&lt;li&gt;I've decided to place Stanford Encyclopedia of Philosophy sources in the layperson's section.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="4709" LastEditDate="2020-05-09T22:23:25.707" LastActivityDate="2020-05-09T22:23:25.707" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="14191" PostTypeId="1" AcceptedAnswerId="14231" CreationDate="2019-08-29T11:49:35.990" Score="3" ViewCount="44" Body="&lt;p&gt;I'm currently working on a project to make an DQN agent that decides whether to charge or discharge an electric vehicle according to hourly changing price to sell or buy. The price pattern also varies from day to day. The goal of this work is to schedule the optimal charging/discharging actions so that it can save money.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The state contains past n-step price records, current energy level in battery, hour, etc., like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;s_t = \{ p_{t-5}, p_{t-4}, p_{t-3}, p_{t-2}, p_{t-1}, E_t, t \}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm wondering is whether this is a partial observable situation or not, because the agent can only observe past n-step prices rather than knowing every price at every time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone comment on this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the partial observable situation, is there any simple way to deal with it?&lt;/p&gt;&#xA;" OwnerUserId="27946" LastEditorUserId="2444" LastEditDate="2019-08-29T12:01:14.420" LastActivityDate="2019-09-01T10:41:17.093" Title="Is a state that includes only the past n-step price records partially observable?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14192" PostTypeId="2" ParentId="14003" CreationDate="2019-08-29T11:57:10.920" Score="2" Body="&lt;p&gt;After I read multiple explanations from different sources I think I found the main difference between the two methods. Implementation wise the only difference is the matrix that you're multiplying the signal with (Laplacian/adjacency matrix). But by using the Laplacian, you're encoding the graph structure (in-out degree of each node) which dictates how a signal should &quot;diffuse&quot; in the network.  &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-08-29T11:57:10.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14194" PostTypeId="1" AcceptedAnswerId="14197" CreationDate="2019-08-29T15:41:37.647" Score="2" ViewCount="114" Body="&lt;p&gt;From &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt;, a book by Stuart Russell and Peter Norvig, this is the definition of AI:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;We define AI as the study of agents that receive percepts from the environment and perform actions.&lt;/strong&gt; Each such agent implements a function that maps percept sequences to actions, and we cover different ways to represent these functions, such as reactive agents, real-time planners, and decision-theoretic systems. We explain the role of learning as extending the reach of the designer into unknown environments, and we show how that role constrains agent design, favoring explicit knowledge representation and reasoning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Given the definition of AI above, is unsupervised learning (e.g. clustering) a branch of AI? I think the definition above is more suitable for supervised or reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2019-09-04T00:02:00.687" LastActivityDate="2019-09-04T00:02:00.687" Title="Is unsupervised learning a branch of AI?" Tags="&lt;definitions&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14196" PostTypeId="1" CreationDate="2019-08-29T16:59:49.980" Score="2" ViewCount="33" Body="&lt;p&gt;I found on the web that fisherface is the best algorithm for face detection. Before investing deeply into it, I just want to know how hard is it to implement it and how much time will it take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to this website and I welcome any suggestions.&lt;/p&gt;&#xA;" OwnerUserId="28295" LastEditorUserId="2444" LastEditDate="2019-08-31T13:23:45.100" LastActivityDate="2019-08-31T13:23:45.100" Title="How to implement fisherface algorithm and how much time will it take?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14197" PostTypeId="2" ParentId="14194" CreationDate="2019-08-29T18:18:39.763" Score="3" Body="&lt;p&gt;There is a problem with confining Artificial Intelligence to a single definition, because it has become an umbrella term encompassing many fields of science. It has come a long way from the &quot;thinking machines&quot; of the 50s. Actually, the term was coined in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Dartmouth_workshop&quot; rel=&quot;nofollow noreferrer&quot;&gt;summer workshop&lt;/a&gt; in 1956, whose proposal was:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The study is to proceed on the basis of the conjecture that every aspect of &lt;strong&gt;learning&lt;/strong&gt; or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So even from its very first introduction the field of AI included &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;learning&lt;/a&gt;. Personally, I've never seen a definition of AI, that doesn't include learning, of which &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsupervised learning&lt;/a&gt; is clearly a part. So by any means clustering algorithms, which are unsupervised learning algorithms, can be considered AI. This can be confirmed in multiple sources, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Glossary_of_artificial_intelligence#C&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-08-29T18:18:39.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14199" PostTypeId="2" ParentId="14165" CreationDate="2019-08-29T18:51:46.833" Score="0" Body="&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the answers on my own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To begin with I figured out how to plot the input data and output training data on a scatter plot using matplotlib.  And once I was able to do that I could instantly see exactly what's going on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the answers are correct the input data is indeed linearly separable based on what the training output is looking to recognize. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;0.5 error&quot; occurs when a single violation of linear separability occurs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;Wrong Answer&quot; error occurs when linear separation is violated twice.  In other words, there are conditions where linear separation is violated in two separate planes.  Or when the data can be separated by planes but it would require more than one plane to do this. (see graphic examples below)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspected that there would be a difference between these different types of errors and the answer is, yes, there is a difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have solved my own question. Thanks to anyone who may have been working on this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to see some of my graphs here are some specific examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A graph of all possible binary inputs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7koLV.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7koLV.gif&quot; alt=&quot;All Possible Inputs&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a good training_outputs = np.array([0, 0, 0, 0, 1, 1, 1, 0]). As you can see in this graph the red points are linearly separable from the blue points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JHaEe.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JHaEe.gif&quot; alt=&quot;Good Training Example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a 0.5 error, training_outputs = np.array([0, 0, 0, 1, 0, 0, 1, 0]). The points are not linearly separable in one plane&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NB0hQ.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NB0hQ.gif&quot; alt=&quot;0.5 error example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of a 2-plane wrong answer error, training_outputs = np.array([0, 0, 0, 1, 0, 1, 1, 0]). You can see that there are two planes that are not linearly separable&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7niYx.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7niYx.gif&quot; alt=&quot;Wrong answer 2 planes violation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a different kind of wrong answer, training_outputs = np.array([0, 0, 0, 1, 1, 0, 0, 0]).  In this case the data can be separated by planes, but it would require 2 different planes to do this which a perceptron cannot handle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vzuI6.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vzuI6.gif&quot; alt=&quot;wrong answer requiring 2 planes to separate&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this covers all possible error conditions.  Aren't graphs great!&lt;/p&gt;&#xA;" OwnerUserId="28240" LastActivityDate="2019-08-29T18:51:46.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14200" PostTypeId="2" ParentId="14174" CreationDate="2019-08-29T19:20:15.600" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;For the line &lt;code&gt;featRaw = selectedLayer.output&lt;/code&gt;, when I print &lt;code&gt;featRaw&lt;/code&gt;, I get the output: &lt;code&gt;Tensor(&quot;block4_conv2/Relu:0&quot;, shape=(1, 64, 64, 512), dtype=float32)&lt;/code&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;a) &lt;code&gt;Relu:0&lt;/code&gt; does this mean Relu activation has not yet been applied?&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It has been applied.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;b) Also I presume we're outputing the feature maps outputs from &lt;code&gt;block4_conv2&lt;/code&gt;, not the filters/kernels themselves, correct?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;c) Why is there an axis of 1 at the start? My understanding of Conv layers is that they're simply made up from the number of filters/kernels (with shape-height, width, depth) to apply to the input.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first dimension in keras is used for representing &lt;code&gt;batch&lt;/code&gt; dimension. When training the model, you aren't passing the images one by one but in batches. The number of images that are processed in parallel before the model calculates its loss and makes its update is called the &lt;code&gt;batch_size&lt;/code&gt;. When computing the content loss, though, you have a single image you want to perform style transfer to, so &lt;code&gt;batch_size=1&lt;/code&gt;. That's why you see the unit in the beginning of the tensor's shape.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;d) Is &lt;code&gt;selectedLayer.output&lt;/code&gt; simply outputs the shape of the Conv layer, or does the output object also hold other information like the pixel values from the output feature maps of the layer?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It outputs the tensor containing all the pixel values that are outputted from that layer.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;&lt;p&gt;With the line: &lt;code&gt;featMatrix = K.reshape(featRaw, (M_l, N_l)&lt;/code&gt; where printing &lt;code&gt;featMatrix&lt;/code&gt; would output: &lt;code&gt;Tensor(&quot;Reshape:0&quot;, shape=(4096, 512), dtype=float32)&lt;/code&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;a) This is where I'm confused the most. So to get the feature/content representation of a particular Conv layer of an image, we simply create a matrix of 2 dimensions, the first being the number of filters and the other being the area of the filter/kernel (height * width). That doesn't make sense! How do we get unique feature of an image from just that?!! We're not retrieving any pixel values from a feature map. We're simply getting the area size of filter/kernel and the number of filters, but not retrieving any of the content (pixel values) itself!!&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you're confusing things a bit. You &lt;strong&gt;are&lt;/strong&gt; getting the pixel values, just in a &lt;strong&gt;different shape&lt;/strong&gt;. Instead of stacking them in a &lt;span class=&quot;math-container&quot;&gt;$64 \times 64 \times 515$&lt;/span&gt; array you are stacking them in a &lt;span class=&quot;math-container&quot;&gt;$1096 \times 515$&lt;/span&gt; array. &lt;strong&gt;Exactly&lt;/strong&gt; the same information is stored in a different shape. Think of it like transforming a &lt;span class=&quot;math-container&quot;&gt;$10 \times 2$&lt;/span&gt; matrix to a &lt;span class=&quot;math-container&quot;&gt;$5 \times 4$&lt;/span&gt; one. It contains the same information, in a different shape.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;b) Also the final &lt;code&gt;featMatrix&lt;/code&gt; is transposed - i.e. &lt;code&gt;featMatrix = K.transpose(featMatrix)&lt;/code&gt; with the output &lt;code&gt;Tensor(&quot;transpose:0&quot;, shape=(512, 4096), dtype=float32)&lt;/code&gt;. Why is that (i.e. why reverse the axis)?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure why exactly he does this, but from a bit that I saw the code I think it has something to do with the style loss (not the content loss). Probably something didn't match up with the dimensions of some operation, maybe a matrix multiplication...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Finally I want to know, once we retrieve the content representation, how can I output that in both as a numpy array and save it as an image?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have the output image as a tensor, let's say &lt;code&gt;y&lt;/code&gt;. You can convert it to numpy through &lt;a href=&quot;https://keras.io/backend/#eval&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;keras.backend.eval&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import keras.backend as K&#xA;&#xA;y_arr = K.eval(y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are many ways to save the image, including &lt;a href=&quot;https://scikit-image.org/docs/dev/api/skimage.io.html#skimage.io.imsave&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-image&lt;/a&gt;, &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.misc.imsave.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt;, &lt;a href=&quot;https://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#imwrite&quot; rel=&quot;nofollow noreferrer&quot;&gt;opencv&lt;/a&gt;, &lt;a href=&quot;https://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.save&quot; rel=&quot;nofollow noreferrer&quot;&gt;PIL&lt;/a&gt;, &lt;a href=&quot;https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.pyplot.imsave.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;matplotlib&lt;/a&gt;, etc. I'll use the last:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;&#xA;plt.imsave('my_image.png', y_arr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that in order to work &lt;code&gt;y_arr&lt;/code&gt; should have a shape of either &lt;code&gt;(height, width, 3)&lt;/code&gt; (if it is an RGB image) or &lt;code&gt;(height, width)&lt;/code&gt; (if it is grayscale).&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-08-29T19:20:15.600" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="14201" PostTypeId="2" ParentId="9418" CreationDate="2019-08-29T19:37:56.087" Score="0" Body="&lt;p&gt;Don't use the Euclidian distance as the similar/dissimilar factor, you'll get better results if you put a couple Dense layers at the top of your Siamese network.  You don't mention how large your feature vectors are, but if they are 128D face encodings, this is what I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# dist_euclid_layer is a layers.Lambda that performs the euclid distance on the input&#xA;first_dense = layers.Dense(512, activation='relu')(dist_euclid_layer)&#xA;drop_one = layers.Dropout(0.5)(first_dense)&#xA;output_dense = layers.Dense(2, activation='sigmoid')(drop_one)&#xA;model = models.Model(inputs=[input_layer], outputs=output_dense)&#xA;model.compile(loss=losses.binary_crossentropy, optimizer=opt, metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In training, use [0,1] or [1,0] for Y to indicate different / similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then in production, use the np.argmax to find the positive match:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;predictions_raw = model.predict(vstack_batch_input, batch_size=len(batch_input))&#xA;predictions = np.argmax(predictions_raw,axis=1)&#xA;tot_matches = np.sum(predictions)&#xA;if tot_matches == 1:&#xA;    # ... good times, whatever face[N] has prediction[N] == 1 is the matching face&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28300" LastActivityDate="2019-08-29T19:37:56.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14202" PostTypeId="1" CreationDate="2019-08-29T19:44:27.477" Score="3" ViewCount="65" Body="&lt;p&gt;I am a physicist and I don't have much background on machine learning or deep learning except taking a couple of courses on statistics. In physics, we often simulate a model by means of two-way coupled systems where each system is described by a partial differential equation. The equations are generally unfolded in a numerical grid of interest and then solved iteratively until a self-consistent solution is obtained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A well-known example is the Schrödinger-Poisson solver. Here, for a given nano/atomic structure, we assume an initial electron density. Then we solve the Poisson equation for that electron density. The Poisson equation tells us the electrostatics (electric potential) of the structure. Given this information, we solve the Schrödinger equation for the structure which tells us about the energy levels of the electrons and their wave functions in the structure. But one would then find that this energy levels and wavefunctions correspond to a different electron density (than the one we initially guessed). So we iterate the process with the new electron density and follow the above mentioned procedures in a loop until a self-consistent solution is obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often times the iteration processes are computationally expensive and extremely time-consuming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: Would the use of deep learning algorithms offer any advantage in modeling such problems where iteration and self-consistency are involved? Are there any study/literature where researchers explored this avenue?   &lt;/p&gt;&#xA;" OwnerUserId="28301" LastEditorUserId="2444" LastEditDate="2019-09-01T22:26:11.697" LastActivityDate="2019-09-01T22:26:11.697" Title="Feasibility of using machine learning to obtain self-consistent solutions" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14204" PostTypeId="1" CreationDate="2019-08-29T19:53:02.270" Score="3" ViewCount="46" Body="&lt;p&gt;Context: I'm a complete beginner to evolutionary algorithms and genetic algorithms and programming. I'm currently taking a course about genetic algorithms and genetic programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the concepts introduced in the course is &quot;closure,&quot; the idea that - with an expression tree representing a genetic program that we're evolving - all nodes in the tree need to be the same type. As a practical example, the lecturer mentions that implementing &lt;code&gt;greater_than(a, b)&lt;/code&gt; for two integers &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; can't return a boolean like &lt;code&gt;true&lt;/code&gt; or &lt;code&gt;false&lt;/code&gt; (it can return, say, &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt; instead).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What he didn't explain is &lt;em&gt;why&lt;/em&gt; the entire tree needs to match in all operators. It seems to me that this requirement would result in the entire tree (representing your evolved program) being composed of nodes that all return the same type (say, &lt;code&gt;integer&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="28304" LastEditorUserId="2444" LastEditDate="2021-01-19T18:49:50.260" LastActivityDate="2021-02-18T19:06:40.617" Title="Why do all nodes in a GP tree need to be the same type?" Tags="&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;&lt;evolutionary-computation&gt;&lt;tgp&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14205" PostTypeId="1" CreationDate="2019-08-29T20:24:33.163" Score="2" ViewCount="155" Body="&lt;p&gt;I want to use Mask RCNN to do image segmentation. I need to override the load_mask function for the dataset class. I know this function should return mask tensors and class ids of objects in an image. But what should the function return if there is no object (only background) in this image? &lt;/p&gt;&#xA;" OwnerUserId="28305" LastActivityDate="2021-05-26T07:02:30.370" Title="What should load_mask() return if an image doesn't have any objects? (Mask RCNN)" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14206" PostTypeId="1" CreationDate="2019-08-30T00:43:59.150" Score="3" ViewCount="238" Body="&lt;p&gt;I am working on an image to image regression task which requires me to develop a deep learning model that takes in a sequence of 5 images and return another image. The sequence of 5 images and the output images are conceptually and temporally related. In fact, the 5 images in the sequence each correspond to timestep in a simulation and the output, the one I am trying to predict, corresponds eventually to the 6th timestep of that sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I have been training a simple regression-type CNN model which takes in the sequence of 5 images stored in a list and outputs an image corresponding to the next timestep in the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have been researching a bit now in order to find a better way to carry out this task and I found the idea of ConvLSTMs. However, I have seen these applied to the prediction of feature and the output of a sentence describing that image. What I wanted to know is whether ConvLSTMs can also output images, but more importantly if they can be applied to my case. If not, what other types of deep learning network can be suitable for this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance !&lt;/p&gt;&#xA;" OwnerUserId="28307" LastActivityDate="2019-08-30T00:43:59.150" Title="Image to image regression in tensorflow" Tags="&lt;deep-learning&gt;&lt;python&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14207" PostTypeId="1" CreationDate="2019-08-30T02:55:16.430" Score="2" ViewCount="56" Body="&lt;p&gt;A single iteration of gradient descent can be parallelised across many worker nodes. We simple split the training set across the worker nodes, pass the parameters to each worker, each worker computes gradients for their subset of the training set, and then passes it back to the master to be averaged. With some effort, we can even use model parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, stochastic gradient descent is an inherently serial proces. Each update must be performed sequentially. Each iteration, we must perform a broadcast and gather of all parameters. This is bad for performance. Ultimately, number of updates is the limiting factor of deep model training speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why must we perform many updates? &#xA;With how few updates can we achieve good accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What factors affect the minimum number of updates requires to reach some accuracy?&lt;/p&gt;&#xA;" OwnerUserId="28308" LastActivityDate="2019-09-29T04:04:45.637" Title="In how few updates can a multi layer neural net be trained?" Tags="&lt;deep-neural-networks&gt;&lt;distributed-computing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14208" PostTypeId="2" ParentId="14207" CreationDate="2019-08-30T03:13:12.723" Score="2" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1711.00489&quot; rel=&quot;nofollow noreferrer&quot;&gt;Don't Decay the Learning Rate, Increase the Batch Size&lt;/a&gt;, Smith et al. train ResNet-50 on ImageNet to 76.1% with only 2500 updates. Has anyone done it in less?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1904.06963&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent&lt;/a&gt; Sankararaman et al. present the concept of gradient confusion which slows convergence, and show that depth increases gradient confusion while overparameterization on width and skip connections reduces gradient confusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1811.05233&quot; rel=&quot;nofollow noreferrer&quot;&gt;Massively Distributed SGD: ImageNet/ResNet-50 Training in a Flash&lt;/a&gt;, Mikami et al. train ResNet-50 on ImageNet to 75.29% with some number of updates, but I can't do the math to compute out how many updates they must have used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarise:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Larger mini-batches help, but give diminishing returns and cause other issues.&lt;/li&gt;&#xA;&lt;li&gt;Deeper models tend to require more updates.&lt;/li&gt;&#xA;&lt;li&gt;Wider layers need fewer updates.&lt;/li&gt;&#xA;&lt;li&gt;Skip connections, label smoothing, and batch norm might help.&lt;/li&gt;&#xA;&lt;li&gt;The best I have found so far on ImageNet/ResNet-50 to &gt;75% is 2500 updates.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28308" LastActivityDate="2019-08-30T03:13:12.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14210" PostTypeId="1" CreationDate="2019-08-30T11:41:48.537" Score="2" ViewCount="41" Body="&lt;p&gt;I want to make a thing that produces a stochastic process from time series data.&#xA;The time series data is recorded every hour over the year, which means 24-hour of patterns exist for 365 days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is something like below:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Fit a probability distribution using data for each hour so that I can sample the most probable value for this hour.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat it for 24 hours to generate a pattern of a day.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BUT!&lt;/strong&gt; I want the sampling to be done considering previous values rather than being done in an independent manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I want to sample from &lt;img src=&quot;https://latex.codecogs.com/gif.latex?p_%7Bh1%7D%28x_t%7Cx_%7Bt-1%7D%2C%20x_%7Bt-2%7D%2C%20...%20%2C%20x_%7Bt-k%7D%29&quot; alt=&quot;p_{h1}(x_t|x_{t-1}, x_{t-2}, ... , x_{t-k})&quot;&gt; or just &lt;img src=&quot;https://latex.codecogs.com/gif.latex?p_%7Bh1%7D%28x_t%7Cx_%7Bt-1%7D%29&quot; alt=&quot;p_{h1}(x_t|x_{t-1})&quot;&gt; rather than &lt;img src=&quot;https://latex.codecogs.com/gif.latex?p_%7Bh1%7D%28x_t%29&quot; alt=&quot;p_{h1}(x_t)&quot;&gt; when &lt;img src=&quot;https://latex.codecogs.com/gif.latex?h1&quot; alt=&quot;h1&quot;&gt; refers to a specific hour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I came up with was the Markov chain, but I couldn't find any reference or materials on how to model it from real data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone give me a comment for this issue?&lt;/p&gt;&#xA;" OwnerUserId="27946" LastEditorUserId="27933" LastEditDate="2019-09-04T09:36:32.567" LastActivityDate="2019-09-04T09:36:32.567" Title="Are there any ways to model markov chains from time series data?" Tags="&lt;markov-chain&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14211" PostTypeId="1" CreationDate="2019-08-30T12:38:49.890" Score="2" ViewCount="43" Body="&lt;p&gt;I am working on an image to image regression task which requires me to develop a deep learning model that takes in a sequence of 5 images and return another image. The sequence of 5 images and the output images are conceptually and temporally related. In fact, the 5 images in the sequence each correspond to timestep in a simulation and the output, the one I am trying to predict, corresponds eventually to the 6th timestep of that sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I have been training a simple regression-type CNN model which takes in the sequence of 5 images stored in a list and outputs an image corresponding to the next timestep in the simulation. This does work with a small and rather simple dataset (13000 images) but works a bit worse on a more diverse and larger dataset (102000 images). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, I have been researching a bit now in order to find a better way to carry out this task and I found the idea of ConvLSTMs. However, I have seen these applied to the prediction of feature and the output of a sentence describing that image. What I wanted to know is whether ConvLSTMs can also output images, but more importantly if they can be applied to my case. If not, what other types of deep learning network can be suitable for this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="28307" LastActivityDate="2019-09-01T12:59:57.770" Title="Can ConvLSTMs outuput images?" Tags="&lt;deep-learning&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14212" PostTypeId="1" CreationDate="2019-08-30T12:48:56.257" Score="3" ViewCount="60" Body="&lt;p&gt;I am working on an MDP where there are four states and ten actions. I am supposed to derive the optimal policy to reach the desired state. At any state, a particular action can take you to any of the other states. &#xA;For ex. If we begin with state S1 -&gt; performing action A1 on S1 can take you to S2 or S3 or S4 or just stay in the same state S1. Similarly for other actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is - is it mandatory to have only a single reward value for a single action A? Or is it possible to give a reward of 10 if action a1 on state s1 takes you to s2, give a reward of 50 if action a1 on state s1 takes you to s3, give a reward of 100 if action a1 on state s1 takes you to s4 which is the terminal state or give zero reward if that action results in the state being unchanged. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I do this?? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because in my case every state is better than its previous state. i.e S2 is better than S1, S3 is better than S2 and so on. So if an action on S1 is directly taking u to S4 which is the final state i would like to award it the maximal reward.    &lt;/p&gt;&#xA;" OwnerUserId="28314" LastEditorUserId="27933" LastEditDate="2019-09-04T09:36:27.570" LastActivityDate="2019-09-04T09:36:27.570" Title="Can I have different rewards for a single action based on which state it transitions to?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14213" PostTypeId="1" CreationDate="2019-08-30T15:02:04.933" Score="2" ViewCount="38" Body="&lt;p&gt;How do I program a neural network such that, when an image is inputted, the output is a numerical value that is not the probability of the image being a certain class? In other words, a CNN that doesn't classify. For example, when an input of an image of a chair is given, the model should not give the chance that the image is a chair but rather give the predicted age of the chair, the predicted price of the chair, etc. I'm currently not sure how to program a neural net like this.&lt;/p&gt;&#xA;" OwnerUserId="28303" LastActivityDate="2019-08-31T05:46:03.580" Title="How does one create a non-classifying CNN in order to gain information from images?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14214" PostTypeId="2" ParentId="14213" CreationDate="2019-08-30T17:21:52.853" Score="2" Body="&lt;p&gt;This can be thought of as a &lt;strong&gt;loss function design problem&lt;/strong&gt;. If you optimize your network weights for something like multi-class classification, then expect your network to learn weights for this task (You will use cross entropy loss for this task). If you optimize your network to output a single value at the last layer and treat it as a regression problem for age prediction, then your network can learn weights for this particular task (You may use something like Mean Square Error loss here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give you a weak guidline on how to do this. Suppose say your input is images of chairs and you want to predict their age using a pretrained resnet, this is how you may do it in pytorch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Definition&lt;/strong&gt;&lt;br&gt;&#xA;X: Input pictures&lt;br&gt;&#xA;Y: List of ground truth values which is age here. i.e. : [1.4, 2.5, 2.2, ....]  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Modify your neural network to give one output at the last layer&lt;/strong&gt;&lt;br&gt;&#xA;model = torchvision.models.resnet18(pretrained=True)&lt;br&gt;&#xA;num_ftrs = model.fc.in_features&lt;br&gt;&#xA;model.fc = torch.nn.Linear(num_ftrs, 1)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Design your loss function appropriately. You can treat age prediction as a standard regression problem. Of course, there are better loss designs here.&lt;/strong&gt;&lt;br&gt;&#xA;criterion =  torch.nn.MSELoss()&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So use this loss during training&lt;/strong&gt;&lt;br&gt;&#xA;loss = criterion(output, target) where output is your neural network prediction and target is your ground truth values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how you can modify an existing architecture for your task. Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="28182" LastActivityDate="2019-08-30T17:21:52.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14215" PostTypeId="1" AcceptedAnswerId="17581" CreationDate="2019-08-30T18:04:12.953" Score="4" ViewCount="821" Body="&lt;p&gt;I would like to take in some input values for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; variables, say &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; denote the response variable of these &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; inputs (in this example, we have &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; inputs). Other than these, I would like to use a reference/target value to compare the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose the relation between the inputs (&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;) with the output &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is (let's say):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y = R + B + G$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the system/machine has no knowledge of this relation. It can only read its inputs, &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, and the output, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. Also, the system is provided with the reference value, say, &lt;span class=&quot;math-container&quot;&gt;$\text{REF} = 30$&lt;/span&gt; (suppose). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim of the machine is to find this relation between its inputs and output(s). For this, I have come across some quite useful material online like &lt;a href=&quot;https://ai.stackexchange.com/q/7434/2444&quot;&gt;this forum query&lt;/a&gt; and &lt;a href=&quot;https://pdfs.semanticscholar.org/05ce/b32839c26c8d2cb38d5529cf7720a68c3fab.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function&lt;/a&gt; by &lt;strong&gt;G. Cybenko&lt;/strong&gt; and felt that it were possible. Also, I doubt that &lt;strong&gt;Polynomial Regression&lt;/strong&gt; may be helpful as suggested &lt;a href=&quot;https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/1467-9868.00192&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One vague approach that comes to my mind is to use a truth table like approach to somehow deduce the effect of the inputs on the output and hence, get a function for it. But neither am I sure how to proceed with it, nor do I trust its credibility. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any alternative/already existing method to accomplish this?&lt;/p&gt;&#xA;" OwnerUserId="26452" LastEditorUserId="2444" LastEditDate="2019-09-01T22:21:40.547" LastActivityDate="2020-01-19T13:47:33.570" Title="How can I determine the mathematical relation between the input and output variables?" Tags="&lt;machine-learning&gt;&lt;math&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14216" PostTypeId="2" ParentId="14189" CreationDate="2019-08-30T18:23:24.510" Score="1" Body="&lt;p&gt;As you are handling with time series data and you want to find trends; A good approach should be consider applying &lt;a href=&quot;https://otexts.com/fpp2/holt-winters.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Holt-Winter's seasonal method&lt;/a&gt;. This algorithm handle seasonal, trend and smooth parameters. A good implementation of this kind of algorithm is &lt;a href=&quot;https://facebook.github.io/prophet/docs/quick_start.html#r-api&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prophet by Facebook&lt;/a&gt;. You can code an exploratory analysis with this library and obtain trend, yearly seasonality, and weekly seasonality of the time series, among other components. Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pDyOB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pDyOB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12006" LastActivityDate="2019-08-30T18:23:24.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14219" PostTypeId="1" AcceptedAnswerId="14268" CreationDate="2019-08-31T02:17:57.847" Score="0" ViewCount="105" Body="&lt;p&gt;Take the below LSTM:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: 5x1 matrix&#xA;hidden units: 256&#xA;output size (aka classes, 1 hot vector): 10x1 matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is my understanding that an LSTM of this size will do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w_x$&lt;/span&gt; = weight matrix at &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b_x$&lt;/span&gt; = bias matrix at &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;activation_gate = tanh(&lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; input + &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; prev_output + &lt;span class=&quot;math-container&quot;&gt;$b_1$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;input_gate = sigmoid(&lt;span class=&quot;math-container&quot;&gt;$w_3$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; input + &lt;span class=&quot;math-container&quot;&gt;$w_4$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; prev_output + &lt;span class=&quot;math-container&quot;&gt;$b_2$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;forget_gate = sigmoid(&lt;span class=&quot;math-container&quot;&gt;$w_5$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; input + &lt;span class=&quot;math-container&quot;&gt;$w_6$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; prev_output + &lt;span class=&quot;math-container&quot;&gt;$b_3$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;output_gate = sigmoid(&lt;span class=&quot;math-container&quot;&gt;$w_7$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; input + &lt;span class=&quot;math-container&quot;&gt;$w_8$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; prev_output + &lt;span class=&quot;math-container&quot;&gt;$b_4$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The size of the output of each gate should be equal to the number of hidden units, ie, 256. The problem arrises when trying to convert to the correct final output size, of 10. If the forget gate outputs 256, then it is summed with the element wise product of the activation and input gate to find the new state, this will result in a hidden state of size 256. (Also in all my research I have not found anywhere whether this addition is actually addition, or simply appending the two matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I have a hidden state of 256, and the output gate outputs 256, doing an element wise product of these two results in, surprise surprise, 256, not 10. If I instead ensure the output gate outputs a size of 10, this no longer works with the hidden state in an element wise product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this handled? I can come up with many ways of doing it myself, but I want an identical replica of the basic LSTM unit, as I have some theories I want to test, and if it is even the slightest bit different it would make the research invalid.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-09-03T03:53:32.623" Title="How does an LSTM output the correct dimensions for classes?" Tags="&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14221" PostTypeId="2" ParentId="14213" CreationDate="2019-08-31T05:46:03.580" Score="1" Body="&lt;p&gt;You generally use SoftMax Layer as the output layer for a neural network that is used as a Classifier. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you want your neural network to predict the age of a chair, predicting price of a chair like in linear regression (output is continuous), you have to remove the SoftMax Layer and add one or multiple layers such that the output of the final layer gives only one value at the output. (which is the prediction for you age or price). And, instead of logit loss you can use MSE for back propagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, like keshik mentioned in another answer, it's all about the final layer used and the loss function used. Based on what you use, your weights get trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what is done in &lt;strong&gt;transfer learning&lt;/strong&gt; also. Based on the task you want to achieve you change the last layers and retrain your network. &lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-08-31T05:46:03.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14224" PostTypeId="1" AcceptedAnswerId="14247" CreationDate="2019-08-31T12:28:39.917" Score="66" ViewCount="9916" Body="&lt;p&gt;If the original purpose for developing AI was to help humans in some tasks and that purpose still holds, why should we care about its explainability? For example, in deep learning, as long as the intelligence helps us to the best of their abilities and carefully arrives at its decisions, why would we need to know how its intelligence works?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2020-08-19T10:52:22.167" LastActivityDate="2020-10-29T14:40:39.247" Title="Why do we need explainable AI?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;explainable-ai&gt;" AnswerCount="9" CommentCount="1" FavoriteCount="26" ContentLicense="CC BY-SA 4.0" />
  <row Id="14226" PostTypeId="2" ParentId="14224" CreationDate="2019-08-31T14:54:43.283" Score="7" Body="&lt;p&gt;Explainable AI is often desirable because&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;AI (in particular, artificial neural networks) can &lt;a href=&quot;https://en.wikipedia.org/wiki/Catastrophic_failure&quot; rel=&quot;noreferrer&quot;&gt;catastrophically fail&lt;/a&gt; to do their intended job. More specifically, it can be hacked or attacked with &lt;a href=&quot;https://arxiv.org/pdf/1312.6199.pdf&quot; rel=&quot;noreferrer&quot;&gt;adversarial examples&lt;/a&gt; or it can take unexpected wrong decisions whose consequences are catastrophic (for example, it can lead to the death of people). For instance, imagine that an AI is responsible for determining the dosage of a drug that needs to be given to a patient, based on the conditions of the patient. What if the AI makes a wrong prediction and this leads to the death of the patient? Who will be responsible for such an action? In order to accept the dosage prediction of the AI, the doctors need to &lt;em&gt;trust&lt;/em&gt; the AI, but trust only comes with understanding, which requires an explanation. So, to avoid such possible failures, it is fundamental to understand the inner workings of the AI, so that it does not make those wrong decisions again.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AI often needs to interact with humans, which are sentient beings (we have feelings) and that often need an explanation or reassurance (regarding some topic or event).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In general, humans are often looking for an explanation and understanding of their surroundings and the world. By nature, we are curious and exploratory beings. Why does an apple fall?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-08-31T15:20:16.763" LastActivityDate="2019-08-31T15:20:16.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14227" PostTypeId="2" ParentId="14224" CreationDate="2019-08-31T15:29:44.573" Score="4" Body="&lt;p&gt;Another reason: In the future, AI might be used for tasks that are not possible to be understood by human beings, by understanding how given AI algorithm works on that problem we might understand the nature of the given phenomenon.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2019-08-31T15:35:01.863" LastActivityDate="2019-08-31T15:35:01.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14228" PostTypeId="1" CreationDate="2019-08-31T16:44:30.220" Score="3" ViewCount="67" Body="&lt;p&gt;I understand that RMSE is just the square root of MSE. Generally, as far as I have seen, people seem to use MSE as a loss function and RMSE for evaluation purposes, since it exactly gives you the error as a distance in the Euclidean space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What could be a major difference between using MSE and RMSE when used as loss functions for training?&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious because good frameworks like PyTorch, Keras, etc. don't provide RMSE loss functions out of the box. Is it some kind of standard convention? If so, why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm aware of the difference that MSE magnifies the errors with magnitude&gt;1 and shrinks the errors with magnitude&amp;lt;1 (on a quadratic scale), which RMSE doesn't do.&lt;/p&gt;&#xA;" OwnerUserId="12574" LastEditorUserId="2444" LastEditDate="2019-08-31T18:24:23.190" LastActivityDate="2021-05-17T03:13:37.853" Title="When to use RMSE as opposed to MSE and vice versa?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14230" PostTypeId="1" CreationDate="2019-09-01T00:39:34.147" Score="1" ViewCount="251" Body="&lt;p&gt;I have been training my CNN for a bit now and, while both the training loss and the training error curves are going down during training, both my validation loss and my validation error curves are kind of zig-zagging and oscillating along the epochs. What does this represent?&lt;/p&gt;&#xA;" OwnerUserId="28307" LastEditorUserId="2444" LastEditDate="2020-12-14T10:31:25.060" LastActivityDate="2020-12-14T10:31:25.060" Title="What does an oscillating validation error curve represent?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14231" PostTypeId="2" ParentId="14191" CreationDate="2019-09-01T01:24:19.210" Score="1" Body="&lt;p&gt;An environment is partially observable if the agent cannot &lt;em&gt;fully&lt;/em&gt; observe the current state but it only &lt;em&gt;partially&lt;/em&gt; observes it. More specifically, in &lt;em&gt;fully&lt;/em&gt; observable MDPs (FOMDPs), the agent knows the current state of the environment, which &lt;em&gt;can or not&lt;/em&gt; (for example, depending on whether the state is Markov or not) contain all the information required to theoretically take the optimal action. In a &lt;em&gt;partially&lt;/em&gt; observable MDPs (POMDPs), the state is not available or observable, so the agent might not possess all the required information to theoretically take the optimal action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the game of poker, there are hidden and observable cards. In this game, the state of the environment &lt;em&gt;could&lt;/em&gt; be defined as all the hands of all players, the cards on the table and the next cards that will be drawn from the deck until the end of the round. If a player had access to all this information, then the environment would be fully observable, but, usually, this information is not available, so poker is usually considered a partially observable game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You &lt;em&gt;defined&lt;/em&gt; the state as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$s_t = \{ p_{t:t-n}, E_t\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you have access to the past &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; price records, &lt;span class=&quot;math-container&quot;&gt;$p_{1:n}$&lt;/span&gt;, and the energy level, &lt;span class=&quot;math-container&quot;&gt;$E_t$&lt;/span&gt;, then your environment is fully observable. However, this does not mean that it is a Markov environment, that is, intuitively, that your current state is sufficient to theoretically take the optimal action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, partial observability depends on the definition of state in a specific problem. The concept of Markov property is also related to the concept of partial observability. However, in the case of MDPs, the concept of Markov property has a specific definition: given the current state and an action, the probability of the next state is conditionally independent of all previous states and actions or, more formally&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(S_{t+1} \mid S_t, A_t) = P(S_{t+1} \mid S_t, S_{t-1}, \dots, S_1, A_t, A_{t-1}, \dots, A_1)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; is the next state, &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; the current state, &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; the current action, &lt;span class=&quot;math-container&quot;&gt;$S_{t-1}, \dots, S_1$&lt;/span&gt; the previous states and &lt;span class=&quot;math-container&quot;&gt;$A_{t-1}, \dots, A_1$&lt;/span&gt; the previous actions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-01T10:41:17.093" LastActivityDate="2019-09-01T10:41:17.093" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14233" PostTypeId="2" ParentId="13777" CreationDate="2019-09-01T04:33:09.763" Score="2" Body="&lt;p&gt;In this excerpt there is something &lt;strong&gt;very&lt;/strong&gt; misleading&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The principle behind Weak AI is simply the fact that machines can be made to act as if they are intelligent. For example, when a human player plays chess against a computer, the human player may feel as if the computer is actually making impressive moves. &lt;strong&gt;But the chess application is not thinking and planning at all. All the moves it makes are previously fed in to the computer by a human and that is how it is ensured that the software will make the right moves at the right times.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Intelligent systems &lt;strong&gt;do not&lt;/strong&gt; in general make use of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Lookup_table&quot; rel=&quot;nofollow noreferrer&quot;&gt;look up table&lt;/a&gt;. For example, the first chess computer &lt;a href=&quot;https://core.ac.uk/download/pdf/82416379.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Blue&lt;/a&gt; made use of complex &lt;a href=&quot;https://en.wikipedia.org/wiki/Evaluation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;evaluation functions&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks could be trained to act as these evaluation functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, there &lt;strong&gt;are&lt;/strong&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;planning agents&lt;/a&gt; - this is a &lt;strong&gt;very&lt;/strong&gt; large sub-field in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below again the article mistakes all artificial agents for simple deterministic Turing-like machines&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Weak AI is focused towards the technology which is capable of carrying out &lt;strong&gt;pre-planned moves&lt;/strong&gt; based on some rules and applying these to achieve a certain goal but, Strong AI is based on coming up with a technology that can think and function very similar to humans. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A highly recommended book is &quot;Artificial Intelligence: A modern Approach.&quot; This reference covers many important sub-fields of AI and provides enough information to realize when people have mistaken the operations of their laptop as the final measure of artificial agency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here Comes the hand waving&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strong vs weak AI dichotomy is fueled by what appears to be a fundamental question: Is there something &quot;special&quot; about humanity that sets us apart from the rest of the material realm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we are just a complex arrangement of matter &lt;strong&gt;with no metaphysical properties&lt;/strong&gt; then in theory we only need to solve two problems for strong AI:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Powerful enough hardware&lt;/li&gt;&#xA;&lt;li&gt;Knowing how to correctly assemble said hardware&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Justification of 1 &amp;amp; 2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the absence of metaphysical properties, we know that a correct arrangement of matter will result in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Epiphenomenalism&quot; rel=&quot;nofollow noreferrer&quot;&gt;epiphenomenal&lt;/a&gt; emergence of human like experience - this idea is &lt;strong&gt;strongly&lt;/strong&gt; supported by Big Bang theory and the Theory of Evolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, all that is left is finding how to correctly arrange matter (hardware) so as to ensure said emergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though, this philosophical thread quickly descends into the rabbit hole where we must tread softly when discussing whether a synthesized agent actually has &lt;a href=&quot;https://plato.stanford.edu/entries/qualia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;qualia&lt;/a&gt;. Subjectivity is arguably the fundamental characteristic of human experience and we can't even assure ourselves that another human has this - we just accept that they do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that I have clarified where the science and philosophy stand. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-09-01T05:21:12.580" LastActivityDate="2019-09-01T05:21:12.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14234" PostTypeId="2" ParentId="14178" CreationDate="2019-09-01T05:36:04.327" Score="0" Body="&lt;p&gt;Note: K-means does not assume an interpretation/label of the clusterings - in fact it is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Unsupervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;unsupervised&lt;/a&gt; algorithm. The interpretations are a result of &lt;strong&gt;human analysis&lt;/strong&gt; &lt;em&gt;after&lt;/em&gt; running K-means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the case of cats and dogs one would most definitely chose k = 2 - which provides an easy interpretation. However, what would it mean if we set k = 1000. We no longer have a &quot;clean&quot; interpretation of the centroids. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: how I keep saying &quot;interpretation.&quot; The algorithm simply assigns a data point to a cluster and calls it a day. Humans then look at the results and try to &lt;strong&gt;understand&lt;/strong&gt; them with an &lt;strong&gt;interpretation&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuing with the example where k = 2. One could easily interpret &quot;is cat&quot; as &quot;not dog&quot; and &quot;is dog&quot; as &quot;not cat.&quot; The idea here is that the data is &lt;strong&gt;unlabeled&lt;/strong&gt; beforehand and humans try to fathom the results retrospectively by assigning the resulting clusters with an understandable label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this clarifies the issue.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2020-04-19T15:53:24.137" LastActivityDate="2020-04-19T15:53:24.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14235" PostTypeId="2" ParentId="5087" CreationDate="2019-09-01T05:47:20.653" Score="1" Body="&lt;p&gt;One approach for this is &lt;a href=&quot;https://en.wikipedia.org/wiki/Collaborative_filtering&quot; rel=&quot;nofollow noreferrer&quot;&gt;collaborative filtering&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;see also &lt;a href=&quot;https://towardsdatascience.com/various-implementations-of-collaborative-filtering-100385c6dfe0&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does however need you to have data about &lt;em&gt;some&lt;/em&gt; user preferences on &lt;em&gt;some&lt;/em&gt; products. Given that you have stated you are willing to mine user preferences this approach may be feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that with this data you can train a model to predict how a user might rate a product. This is accomplished by learning the &quot;preference signature&quot; of a user and a feature vector for each product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;General Idea&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for your i-th user, the algorithm will learn said preferences as a vector &lt;span class=&quot;math-container&quot;&gt;$$\theta^{(i)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, for your j-th product it will learn a feature vector &lt;span class=&quot;math-container&quot;&gt;$$x^{(j)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then predict how the i-th user will rate the j-th item by computing the dot product (or some equivalent). That is, compute the predicted rating as: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat R(i,j)=\theta^{(i)}\cdot x^{(j)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then use this rating to decide whether or not a a product is a good match for a user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.coursera.org/learn/machine-learning?&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Ng Machine Learning Coursera MOOC&lt;/a&gt; has a &lt;strong&gt;very&lt;/strong&gt; nice module on collaborative filtering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Implementation Note&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When asking your users for feedback, try to ask for &lt;em&gt;quantitative&lt;/em&gt; measures. For example, the classic 1-5 scale rating.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-09-02T02:37:48.307" LastActivityDate="2019-09-02T02:37:48.307" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14236" PostTypeId="1" CreationDate="2019-09-01T10:29:42.920" Score="2" ViewCount="74" Body="&lt;p&gt;I'm new to artificial intelligence. I am looking for the most appropriate AI solution for my application, which is developing an algorithm to predict a proceeding situation (edited: I want my algorithm to predict a situation or more than one to happen at a predefined moment) and, at the same time, to learn from the iterative stages of my application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions? Any help? Any proposals?&lt;/p&gt;&#xA;" OwnerUserId="28345" LastEditorUserId="28345" LastEditDate="2019-09-01T22:08:26.430" LastActivityDate="2020-05-28T23:06:15.493" Title="Which predictive algorithm is most appropriate for a proceeding situation?" Tags="&lt;ai-design&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14237" PostTypeId="2" ParentId="14212" CreationDate="2019-09-01T12:24:50.270" Score="2" Body="&lt;p&gt;The reward function &lt;strong&gt;can&lt;/strong&gt; be a function of the current state, current action, and next state: &lt;span class=&quot;math-container&quot;&gt;$R(s_t, a_t, s_{t+1})$&lt;/span&gt;. It's valid to use the Bellman operator in this setting because it's still a contraction and will yield the optimal value function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I'm assuming that you will be solving the MDP with the Bellman equation.&lt;/p&gt;&#xA;" OwnerUserId="28347" LastActivityDate="2019-09-01T12:24:50.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14238" PostTypeId="2" ParentId="14224" CreationDate="2019-09-01T12:36:43.300" Score="-1" Body="&lt;p&gt;IMHO, the most important need for explainable AI is to prevent us from becoming intellectually lazy.  If we stop &lt;em&gt;trying&lt;/em&gt; to understand how answers are found, we have conceded the game to our machines.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-09-01T12:36:43.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14239" PostTypeId="2" ParentId="14211" CreationDate="2019-09-01T12:59:57.770" Score="2" Body="&lt;p&gt;If you only need 5 frames to predict the next frame, then I'd recommend a &lt;a href=&quot;https://arxiv.org/abs/1505.04597&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net architecture&lt;/a&gt;, wich is basically a CNN encoder/decoder network in which the decoder uses the intermediate features produced in the encoder as well as its own features to produce an output image. Also, in additional to using a conventional L2 loss for the output image, you can always add an additional GAN loss to make the image look more realistic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If using a longer history of frames can help, then I recommend taking a look at &lt;a href=&quot;https://arxiv.org/abs/1704.02254&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Recurrent Environment Simulators&quot;&lt;/a&gt; and combining it with the ideas above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="28347" LastActivityDate="2019-09-01T12:59:57.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14240" PostTypeId="2" ParentId="14224" CreationDate="2019-09-01T13:23:58.630" Score="9" Body="&lt;p&gt;If you're a bank, hospital or any other entity that uses predictive analytics to make a decision about actions that have huge impact on people's lives, you would not make important decisions just because Gradient Boosted trees told you to do so. Firstly, because it's risky and the underlying model might be wrong and, secondly, because in some cases it is illegal - see &lt;a href=&quot;https://en.wikipedia.org/wiki/Right_to_explanation&quot; rel=&quot;noreferrer&quot;&gt;Right to explanation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22835" LastEditorUserId="2444" LastEditDate="2019-09-01T19:42:49.803" LastActivityDate="2019-09-01T19:42:49.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14242" PostTypeId="2" ParentId="14236" CreationDate="2019-09-01T17:05:25.307" Score="1" Body="&lt;p&gt;One approach would be to use a Sequence Processing Neural architecture - one option is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrent network&lt;/a&gt;. These were specifically designed with your intent in mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They can consume sequences of data and learn from these in order to predict subsequent time-steps. Though these will require you to understand &lt;a href=&quot;https://towardsdatascience.com/10-things-to-think-about-before-starting-to-code-your-deep-neural-network-65094a1e7c08&quot; rel=&quot;nofollow noreferrer&quot;&gt;best practices&lt;/a&gt; for implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your task isn't too complex you could make use of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Forward_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;forward algorithm&lt;/a&gt; or some variant. Many of these models make use of simplifying assumptions - if your use case cannot admit these you'll have to go for something more advanced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your prediction task is complex and requires &lt;strong&gt;heavy&lt;/strong&gt; use of long term dependencies you may have to go for a more cutting edge architecture like a &lt;a href=&quot;https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer&lt;/a&gt; - but I heard that these are difficult to implement and most definitely require familiarity with deep learning systems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you need a system that can understand data and dependencies across &lt;strong&gt;very large&lt;/strong&gt; spans of time then you may have to wait for the science to progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastActivityDate="2019-09-01T17:05:25.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14243" PostTypeId="1" AcceptedAnswerId="14298" CreationDate="2019-09-01T17:39:13.060" Score="7" ViewCount="587" Body="&lt;p&gt;After an AI goes through the process described in &lt;a href=&quot;https://ai.stackexchange.com/questions/1970/how-would-an-ai-learn-language&quot;&gt;How would an AI learn language?&lt;/a&gt;, an AI knows the grammar of a language through the process of grammar induction. They can speak the language, but they have learned &lt;em&gt;formal grammar&lt;/em&gt;. But most conversations today, even formal ones, use idiomatic phrases. Would it be possible for an AI to be given a set of idioms, for example, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Immer mit der Ruhe&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which, in German, means 'take it easy' but an AI of grammar induction, if told to translate 'take it easy' to German, would not think of this. And if asked to translate this, it would output&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Always with the quiet&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, it is possible to teach an AI to use idiomatic phrases to keep up with the culture of humans?&lt;/p&gt;&#xA;" OwnerUserId="26306" LastActivityDate="2019-09-06T15:48:19.237" Title="How would an AI learn idiomatic phrases in a natural language?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14244" PostTypeId="2" ParentId="14230" CreationDate="2019-09-01T17:44:47.027" Score="3" Body="&lt;p&gt;Without more information the best diagnostic is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;You potentially have a bug in your code.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Justification&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Neural network systems are capable of failing silently. The system can still appear to &amp;quot;learn&amp;quot; even in the presence of said bug - a while back I had a very similar issue with a toy CNN project. My network could get 99% accuracy on the training set but always achieved 8-13% on the validation set. This looked like overfit but none of the methods solved the issue. Finally, I found that I wasn't feeding the data correctly into the network during train time but I &lt;strong&gt;was&lt;/strong&gt; feeding the data correctly during validation time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Provide the following for better diagnostic:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this a custom CNN you hardcoded?&lt;/li&gt;&#xA;&lt;li&gt;are you using python?&lt;/li&gt;&#xA;&lt;li&gt;what's the objective function?&lt;/li&gt;&#xA;&lt;li&gt;can you provide the loss curve for the training set?&lt;/li&gt;&#xA;&lt;li&gt;show us some code too (this will be very helpful for working toward a solution)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I hope this helps, and best of luck!&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2020-07-25T16:14:12.170" LastActivityDate="2020-07-25T16:14:12.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14246" PostTypeId="1" AcceptedAnswerId="14270" CreationDate="2019-09-01T18:11:42.990" Score="6" ViewCount="206" Body="&lt;p&gt;To compare the performance of various algorithms for &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;perfect information games&lt;/a&gt;, reasonable benchmarks include &lt;a href=&quot;https://en.wikipedia.org/wiki/Reversi&quot; rel=&quot;nofollow noreferrer&quot;&gt;reversi&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/M,n,k-game&quot; rel=&quot;nofollow noreferrer&quot;&gt;m,n,k-games&lt;/a&gt; (generalized tic-tac-toe). For imperfect information games, something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Kuhn_poker&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplified poker&lt;/a&gt; is a reasonable benchmark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some reasonable benchmarks to compare the performance of various algorithms for reinforcement learning in &lt;strong&gt;discrete&lt;/strong&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDPs&lt;/a&gt;? Instead of using a random environment from the space of all possible discrete MDPs on &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; states and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; actions, are there subsets of such a space with more structure that are more reflective of &quot;real-world&quot; environments? An example of this might be so-called gridworld (i.e. maze-like) environments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/145122/real-life-examples-of-markov-decision-processes&quot;&gt;This&lt;/a&gt; is a related question, though I'm looking for specific examples of MDPs (with specified transitions and rewards) rather than general areas where MDPs can be applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Some example MDPs are found in section 5.1 (Standard Domains) of &lt;a href=&quot;https://papers.nips.cc/paper/4767-efficient-bayes-adaptive-reinforcement-learning-using-sample-based-search.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search&lt;/a&gt; (2012) by Guez et al.:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The &lt;strong&gt;Double-loop&lt;/strong&gt; domain is a 9-state deterministic MDP with 2actions,&#xA;  1000 steps are executed in this domain. &lt;strong&gt;Grid5&lt;/strong&gt; is a 5×5 grid with no&#xA;  reward anywhere except for a reward state opposite to the reset state.&#xA;  Actions with cardinal directions are executed with small probability&#xA;  of failure for 1000 steps. &lt;strong&gt;Grid10&lt;/strong&gt; is a 10×10 grid designed like Grid5.&#xA;  We collect 2000 steps in this domain. &lt;strong&gt;Dearden’s Maze&lt;/strong&gt; is a 264-states&#xA;  maze with 3 flags to collect. A special reward state gives the number&#xA;  of flags collected since the last visit as reward, 20000 steps are&#xA;  executed in this domain.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3373" LastEditorUserId="3373" LastEditDate="2019-12-27T04:11:21.580" LastActivityDate="2019-12-27T04:11:21.580" Title="Benchmarks for reinforcement learning in discrete MDPs" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;markov-decision-process&gt;&lt;benchmarks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14247" PostTypeId="2" ParentId="14224" CreationDate="2019-09-01T18:44:47.510" Score="72" Body="&lt;p&gt;As argued by &lt;a href=&quot;https://arxiv.org/pdf/1610.02391.pdf&quot; rel=&quot;noreferrer&quot;&gt;Selvaraju et al.&lt;/a&gt;, there are three stages of AI evolution, in all of which interpretability is helpful. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In the early stages of AI development, when AI is weaker than human performance, transparency can help us &lt;strong&gt;build better models&lt;/strong&gt;. It can give a better understanding of how a model works and helps us answer several key questions. For example &lt;em&gt;why&lt;/em&gt; a model works in some cases and doesn't in others, &lt;em&gt;why&lt;/em&gt; some examples confuse the model more than others, &lt;em&gt;why&lt;/em&gt; these types of models work and the others don't, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When AI is on par with human performance and ML models are starting to be deployed in several industries, it can help build &lt;strong&gt;trust&lt;/strong&gt; for these models. I'll elaborate a bit on this later, because I think that it is the most important reason.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When AI significantly outperforms humans (e.g. AI playing chess or Go), it can help with &lt;strong&gt;machine teaching&lt;/strong&gt; (i.e. learning from the machine on how to improve human performance on that specific task).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Why is trust so important?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;First, let me give you a couple of examples of industries where &lt;em&gt;trust&lt;/em&gt; is paramount:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In healthcare, imagine a Deep Neural Net performing diagnosis for a specific disease. A classic &lt;em&gt;black box&lt;/em&gt; NN would just output a binary &quot;yes&quot; or &quot;no&quot;. Even if it could outperform humans in sheer predictability, it would be utterly useless in practice. What if the doctor disagreed with the model's assessment, shouldn't he know &lt;em&gt;why&lt;/em&gt; the model made that prediction; maybe it saw something the doctor missed. Furthermore, if it made a misdiagnosis (e.g. a sick person was classified as healthy and didn't get the proper treatment), who would take responsibility: the model's user? the hospital? the company that designed the model? The legal framework surrounding this is a bit blurry.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another example are self-driving cars. The same questions arise: if a car crashes whose fault is it: the driver's? the car manufacturer's? the company that designed the AI? Legal accountability, is key for the development of this industry.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In fact, this lack of trust, has according to many hindered the &lt;strong&gt;adoption&lt;/strong&gt; of AI in many fields (sources: &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.8362&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://www.researchgate.net/publication/3454453_Autonomy_and_Common_Ground_in_Human-Robot_Interaction_A_Field_Study&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1706.07269&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;). While there is a running hypothesis that with more transparent, interpretable or explainable systems users will be &lt;strong&gt;better equipped to understand and therefore trust&lt;/strong&gt; the intelligent agents (sources: &lt;a href=&quot;https://www.arl.army.mil/arlreports/2014/ARL-TR-6905.pdf&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://journals.sagepub.com/doi/10.1177/0018720815621206&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;http://www.bradhayes.info/papers/hri17.pdf&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In several real world applications you can't just say &quot;it works 94% of the time&quot;. You might also need to provide a justification...&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Government regulations&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Several governments are slowly proceeding to regulate AI and transparency seems to be at the center of all of this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first to move in this direction is the EU, which has set several guidelines where they state that AI should be transparent (sources: &lt;a href=&quot;https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://ec.europa.eu/commission/news/artificial-intelligence-2019-apr-08_en&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;http://www.europarl.europa.eu/news/en/press-room/20190214IPR26425/online-platforms-required-by-law-to-be-more-transparent-with-eu-businesses&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;). For instance the GDPR states that if a person's data has been subject to &quot;automated decision-making&quot; or &quot;profiling&quot; systems, then he has a right to access&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;meaningful information about the logic involved&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;http://www.privacy-regulation.eu/en/article-15-right-of-access-by-the-data-subject-GDPR.htm&quot; rel=&quot;noreferrer&quot;&gt;Article 15, EU GDPR&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this is a bit blurry, but there is clearly the intent of requiring some form of explainability from these systems. The general idea the EU is trying to pass is that &quot;if &#xA; you have an automated decision-making system affecting people's lives then they have a right to know &lt;em&gt;why&lt;/em&gt; a certain decision has been made.&quot; For example a bank has an AI accepting and declining loan applications, then the applicants have a right to know &lt;em&gt;why&lt;/em&gt; their application was rejected.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;To sum up...&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Explainable AIs are necessary because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It gives us a better understanding, which helps us improve them.&lt;/li&gt;&#xA;&lt;li&gt;In some cases we can learn from AI how to make better decisions in some tasks.&lt;/li&gt;&#xA;&lt;li&gt;It helps users &lt;em&gt;trust&lt;/em&gt; AI, which which leads to a wider adoption of AI.&lt;/li&gt;&#xA;&lt;li&gt;Deployed AIs in the (not to distant) future might be required to be more &quot;transparent&quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-09-01T18:44:47.510" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14248" PostTypeId="1" CreationDate="2019-09-01T20:30:13.570" Score="1" ViewCount="26" Body="&lt;p&gt;I have written my own AlphaZero implementation and started training it recently.&lt;br&gt;&#xA;Problem is, I am 99% sure there is a mistake and I do not know how to tackle this, since I cannot explain it. I am new too AI so my own go at debugging this wasn't quite succesful.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input to my NN:&lt;/strong&gt; A game state, represented by the board and position of the stones.&lt;br&gt;&#xA;&lt;strong&gt;Output of my NN:&lt;/strong&gt; a policy vector &lt;em&gt;P&lt;/em&gt; and a scalar &lt;em&gt;v&lt;/em&gt;(so an array and a number).&lt;br&gt;&#xA;During self-play, training examples for each move are generated. These are later used to fit the network.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After having trained a bit, I can see both policy and value loss decreasing, which is good.&lt;br&gt;&#xA;But for the very first game state (empty board) my prediction &lt;em&gt;v&lt;/em&gt; of winning the game always stays at 0.  This is very concerning, since the game I am training on is Connect4. Connect4 is a solved game and in the long run, the value for v should be 1(100% win chance).&lt;br&gt;&#xA;So, any ideas what I can do? I mean I could post the code here, but that is quite a lot, I don't know if any of you are willing to read through it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show you what I mean, I'll show you the output of one of my test cases:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def test_p_v():&#xA;    game = connect4.Connect4()&#xA;    nnetwrapper = neuralnetwrapper.NNetWrapper(game, args)&#xA;    trainer = trainingonly1NN.Training(game = game, nnet= nnetwrapper, args = args)&#xA;&#xA;    trainer.nnet.load_checkpoint(folder = './NNmodels/', filename='best.pth.tar')&#xA;&#xA;    m = mctsnn.MCTS(nnet = nnetwrapper, args=args)&#xA;&#xA;    m.root.expand()&#xA;&#xA;    for c in m.root.children:&#xA;        print(&quot;P: {}  v: {}&quot;.format(c.state.P, c.state.v))&#xA;&#xA;    print(&quot;Root MCTS: P: {}   v: {}&quot;.format(m.root.state.P, m.root.state.v))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P: [0.1436838  0.13809082 0.14174062 0.18597336 0.11126296 0.120884&#xA; 0.15836443]  v: [-0.14345692]&#xA;P: [0.14202288 0.13772981 0.14302546 0.1945151  0.11690026 0.1178078&#xA; 0.1479987 ]  v: [0.4222183]&#xA;P: [0.1447647  0.13066562 0.14334281 0.18055147 0.13374692 0.12126701&#xA; 0.1456615 ]  v: [-0.5827425]&#xA;P: [0.15192215 0.14221476 0.1443521  0.16634388 0.12634312 0.12711576&#xA; 0.14170831]  v: [-0.0229549]&#xA;P: [0.1456457  0.136381   0.13940862 0.17145196 0.12714048 0.12233274&#xA; 0.15763956]  v: [-0.02743456]&#xA;P: [0.15353182 0.13510287 0.1433772  0.16371183 0.12161442 0.1228981&#xA; 0.15976372]  v: [0.37902302]&#xA;P: [0.14321715 0.13596673 0.13836266 0.18927328 0.11999774 0.12481775&#xA; 0.1483647 ]  v: [-0.521353]&#xA;Root MCTS: P: [0.14296353 0.13863131 0.1358864  0.18102945 0.10981551 0.12779148&#xA; 0.16388236]   v: [0.]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So as you can see, for every different state, there are different P-values and different v-values, which makes sense.&lt;br&gt;&#xA;It also makes sense for my ROOT Node to have the highest value in P at position 3, since this refers to the middle column.&lt;br&gt;&#xA;But the v in my root Node is 0.  This is alarming and I have no idea what do from here on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also checked some of the training examples passed to my neural network to learn, they look like this (board, P, Actual game result):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[array([[0, 0, 0, 0, 0, 0, 0],&#xA;       [0, 0, 0, 0, 0, 0, 0],&#xA;       [0, 0, 0, 0, 0, 0, 0],&#xA;       [0, 0, 0, 0, 0, 0, 0],&#xA;       [0, 0, 0, 0, 0, 0, 0],&#xA;       [0, 0, 0, 0, 0, 0, 0]]), [0.13, 0.13, 0.15, 0.13, 0.18, 0.13, 0.15], 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;whereas the very last number (1) is the v-value my network is supposed to fit! So it often even is 1!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I fear that since this is the very root of all, the game result for the following notes is -1 and 1  changing every &quot;step&quot;, so the average probably is 0 which is quite logical. Don't know how to express this, I fear it is trying to average the v mean of all states, instead of just training the v for one state.&lt;/p&gt;&#xA;" OwnerUserId="27406" LastEditorUserId="27406" LastEditDate="2019-09-01T21:26:02.187" LastActivityDate="2019-09-01T21:26:02.187" Title="AlphaZero value at root node not being affected by training" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;alphazero&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14249" PostTypeId="1" CreationDate="2019-09-02T03:04:55.237" Score="1" ViewCount="28" Body="&lt;p&gt;I used following custom loss function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def custom_loss(epo):&#xA;&#xA;  def loss(y_true,y_pred):&#xA;      m=K.binary_crossentropy(y_true, y_pred)&#xA;      x=math.log10(epo)&#xA;      y=x*x&#xA;      y=(math.sqrt(y)/100)&#xA;      l=(m*(y))&#xA;&#xA;      return K.mean(l, axis=-1)&#xA;  return loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this is my discriminator model&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def Discriminator():&#xA;&#xA;&#xA;  inputs = Input(shape=img_shape)&#xA;&#xA;&#xA;  x=Conv2D(32, kernel_size=3, strides=2, padding=&quot;same&quot;)(inputs)&#xA;  x=LeakyReLU(alpha=0.2)(x)&#xA;  x=Dropout(0.25)(x, training=True)&#xA;&#xA;  x=Conv2D(64, kernel_size=3, strides=2, padding=&quot;same&quot;)(x)&#xA;  x=ZeroPadding2D(padding=((0, 1), (0, 1)))(x)&#xA;  x=BatchNormalization(momentum=0.8)(x)&#xA;  x=LeakyReLU(alpha=0.2)(x)&#xA;&#xA;  x=Dropout(0.25)(x, training=True)&#xA;  x=Conv2D(128, kernel_size=3, strides=2, padding=&quot;same&quot;)(x)&#xA;  x=BatchNormalization(momentum=0.8)(x)&#xA;  x=LeakyReLU(alpha=0.2)(x)&#xA;&#xA;  x=Dropout(0.25)(x, training=True)&#xA;  x=Conv2D(256, kernel_size=3, strides=1, padding=&quot;same&quot;)(x)&#xA;  x=BatchNormalization(momentum=0.8)(x)&#xA;  x=LeakyReLU(alpha=0.2)(x)&#xA;&#xA;  x=Dropout(0.25)(x, training=True)&#xA;  x=Flatten()(x)&#xA;  outputs=Dense(1, activation='sigmoid')(x)&#xA;  model = Model(inputs, outputs)&#xA;  #model.summary()&#xA;  img = Input(shape=img_shape)&#xA;  validity = model(img)&#xA;  return Model(img, validity)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and initialize discriminator here&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D = Discriminator()&#xA;epoch=0&#xA;D.compile(loss=custom_loss(epoch), optimizer=optimizer, metrics= &#xA;['accuracy'])&#xA;G = Generator()&#xA;z = Input(shape=(100,))&#xA;img = G(z)&#xA;D.trainable = False&#xA;valid = D(img)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i want to update epo value of loss function after each epoch in the following code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for epoch in range(epochs):&#xA;&#xA;  for batch in range(batches):&#xA;      ............&#xA;     d_loss_real = D.train_on_batch(imgs, valid)&#xA;     d_loss_fake = D.train_on_batch(gen_batch, fake)&#xA;     d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)&#xA;     g_loss = combined.train_on_batch(noise_batch, valid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Are there any way for updating loss function without effecting training after compiling the model?&lt;/p&gt;&#xA;" OwnerUserId="23509" LastEditorUserId="23509" LastEditDate="2019-09-02T03:15:17.720" LastActivityDate="2019-09-02T03:15:17.720" Title="How to update Loss Function parameter after compilation" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14251" PostTypeId="2" ParentId="14224" CreationDate="2019-09-02T11:05:46.330" Score="4" Body="&lt;p&gt;The answer to this is incredibly simple. If you are a bank executive one day you may need to stand up in court and explain why your AI denied mortgages to all these people... who just happen to share some protected characteristic under anti-discrimination legislation. The judge will not be happy if you handwave the question away mumbling something about algorithms. Or worse, why did this car/plane crash and how will you prevent it next time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the major blocker to more widespread adoption of AI in many industries.&lt;/p&gt;&#xA;" OwnerUserId="16207" LastActivityDate="2019-09-02T11:05:46.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14254" PostTypeId="1" CreationDate="2019-09-02T12:44:54.130" Score="2" ViewCount="323" Body="&lt;p&gt;I'm following &lt;a href=&quot;https://medium.com/@ngwaifoong92/beginners-guide-to-retrain-gpt-2-117m-to-generate-custom-text-content-8bb5363d8b7f&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial, and I wonder why is there a train-step - why is it necessary? I thought the whole idea of GPT-2 is that you do not need to train it on specific text domain, as it's already pre-trained on a large amount of data.&lt;/p&gt;&#xA;" OwnerUserId="27947" LastEditorUserId="27947" LastEditDate="2019-09-02T21:13:21.943" LastActivityDate="2019-09-02T21:13:21.943" Title="Why do you need to retrain GPT-2?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14257" PostTypeId="2" ParentId="14215" CreationDate="2019-09-02T14:54:26.127" Score="0" Body="&lt;p&gt;There are always a large number of possible functions that can produce a given set of input-output values. The challenge is to find a &lt;em&gt;simplest&lt;/em&gt; function (according to whatever criteria you choose) to produce those values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach is to write a general function of the input variables, comprising terms of all order in R, G, and B, with a coefficient for each term, then search for values of the coefficients that A) reproduce the known input-output values accurately and B) leave the largest number of coefficients equal to zero.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several different algorithms can be used to do the search efficiently.  My choice would be a genetic algorithm to seek the minimum of the RMS difference between the produced and known I-O values, summed with the product of a gradually increasing parameter and the number of nonzero coefficients.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-09-02T14:54:26.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14259" PostTypeId="2" ParentId="14254" CreationDate="2019-09-02T15:30:53.780" Score="1" Body="&lt;p&gt;I've come to learn about GPT-2 through Robert Miles AI safety Youtube channel and intend to look into it in more detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my current understanding, GPT-2 is pre-trained to &quot;understand&quot; &quot;natural&quot; language (for any definition of the words in quotes). However you would want it to not only understand general text but generate text similar to some specific &quot;genre&quot;, e.g. scientific articles, youtube comments, twitter messages, you name it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So using its pre-trained understanding, it analyzes the &lt;strong&gt;structure of sample texts&lt;/strong&gt; and replicates this structure.&lt;br&gt;&#xA;For scientific articles this structure could be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Abstract&lt;/li&gt;&#xA;&lt;li&gt;Context of research topic&lt;/li&gt;&#xA;&lt;li&gt;Introduction of researchers&lt;/li&gt;&#xA;&lt;li&gt;Explanation of methods/experiments/discoveries&lt;/li&gt;&#xA;&lt;li&gt;Results and interpretation&lt;/li&gt;&#xA;&lt;li&gt;Future research and application&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For Youtube comments the structure is probably more chaotic but could include a vague reference to former comments, insults, nonsensical bar-grade philosophy, internet slang and smileys.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; The domain specific text is only used to tell GPT-2 what you're looking for. You basically hand it context to work with, instead of prompting &quot;Say something clever&quot; (my least favorite line at parties, when I've been introduced as clever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;P.S.:&lt;/strong&gt; Take this with a grain of salt. It's 90% conjecture from incomplete information.&lt;/p&gt;&#xA;" OwnerUserId="28388" LastActivityDate="2019-09-02T15:30:53.780" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14260" PostTypeId="2" ParentId="12487" CreationDate="2019-09-02T16:30:58.593" Score="1" Body="&lt;p&gt;I have not come across a model like this yet. BUT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have not tried smaller models I'd recommend trying that first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Justification: this lets you use &lt;a href=&quot;https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;learning curves&lt;/a&gt; to diagnose what to do next. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you might try starting with a GRU (the LSTM overhead may not be needed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;One Idea for a starting model&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observe that turnout &lt;strong&gt;does not&lt;/strong&gt; affect weather. So the weather forecast model will be independent of the turnout model but not vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note: I'm using RNN to denote which ever recurrent architecture you choose)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Formulation for simultaneous prediction&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Weather&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{RNN}_w$&lt;/span&gt; is your weather prediction model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat w_t = \text{RNN}_w(\hat w_{t-1})$&lt;/span&gt; be the predicted weather at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turnout: The idea here is that people will or will not go to the beach for various reasons (overcrowded, too hot or stormy etc). So in the beach population prediction task we use all these as &lt;strong&gt;features&lt;/strong&gt; to predict the population at the next time-step. This reduces the problem to any one of the classical models already developed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{RNN}_p$&lt;/span&gt; is your turnout (beach population) prediction model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat p_t$&lt;/span&gt; be the predicted turnout at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat c_t=[\hat p_t, \hat w_t]$&lt;/span&gt; is the concatenation at time t&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat p_{t+1} = \text{RNN}_p(\hat c_t)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Formulation in the presence of weather forecast&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply replace &lt;span class=&quot;math-container&quot;&gt;$\hat w_t$&lt;/span&gt; with the true weather forecasts &lt;span class=&quot;math-container&quot;&gt;$\hat f_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A final warning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you think your RNN is better at weather prediction than the forecasting systems I would &lt;strong&gt;not&lt;/strong&gt; recommend using &lt;span class=&quot;math-container&quot;&gt;$\text{RNN}_w$&lt;/span&gt; in a production application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-09-02T16:42:25.703" LastActivityDate="2019-09-02T16:42:25.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14261" PostTypeId="2" ParentId="14215" CreationDate="2019-09-02T18:40:12.723" Score="0" Body="&lt;p&gt;As I see your explanation, what you're looking for can be stated via another way saying&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y = aX + b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where Y is output vector, X input vector and a &amp;amp; b are the coefficients you want to find.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why so? And what is happening?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First thing first: I recommend to look the video [4] about how matrices and vectors work together and form after multiplication very familiar equations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y = a1 * x1 + a2 * x2 + a3 * x3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you see, you do not only get R + G + B but also some constants supplied for each of the variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About polynomial solutions I found [2] and [3] but reading through [3] you'll soon notice it is about completely different approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y = a1 * x + a2 * x^2 + a3 * x^3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which you don't want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you find something called linear regression and a so called Deep Neural Network for example to solve it [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would summarize the source [1] in these steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You have to find some training data. That is examples of correct Y values, when X values are known.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then you build in Python note book some code that has: a Neural Network with hidden layer(s), Activation Function, Back Propagation and Objective Function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many many iterations with the data, called Training.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are plenty of samples and courses online about these in detail, but with correct tools and tutorials some dozen lines and no more are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Process is ended with validation phase with some more known data items and results. It can tell you how well the estimated model works.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final Notes:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;As you may observe, the solution includes quite a many funny terms you have to learn somehow before mastering the task. For example Udemy has great online courses on this topic, also free tutorials are available on another sites. Your plans sound quite ambitious compared to the knowledge you have so far, so I really do recommend you learn little bit more to be able to fine tune the already given examples online. For example tutorial [5] includes one. It is at first quite complicated code, you'd need quite a lot practice to master it line by line.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In short:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find your favorite tutorial, study neural networks a little bit (basics) and pick code sample to start experimenting. It is a long way but it is worth it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://lightsapplications.wordpress.com/linear-regression-and-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lightsapplications.wordpress.com/linear-regression-and-deep-learning/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://www.ritchieng.com/machine-learning-polynomial-regression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ritchieng.com/machine-learning-polynomial-regression/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://arachnoid.com/polysolve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arachnoid.com/polysolve/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://www.youtube.com/watch?v=F2lJ7oSwcyY&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=F2lJ7oSwcyY&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;https://missinglink.ai/guides/neural-network-concepts/backpropagation-neural-networks-process-examples-code-minus-math/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://missinglink.ai/guides/neural-network-concepts/backpropagation-neural-networks-process-examples-code-minus-math/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2019-09-02T18:40:12.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14262" PostTypeId="1" AcceptedAnswerId="14339" CreationDate="2019-09-02T20:34:45.123" Score="0" ViewCount="496" Body="&lt;p&gt;I have a simple text classifier, with the following structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    input = keras.layers.Input(shape=(len(train_x[0]),))&#xA;&#xA;    x=keras.layers.Dense(500, activation='relu')(input)&#xA;    x=keras.layers.Dropout(0.5)(x)&#xA;    x=keras.layers.Dense(250, activation='relu')(x)&#xA;    x=keras.layers.Dropout(0.5)(x)&#xA;    preds = keras.layers.Dense(len(train_y[0]), activation=&quot;sigmoid&quot;)(x)&#xA;&#xA;    model = keras.Model(input, preds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When training it with 300,000 samples, with a batch size of 500, I get an accuracy value of .95 and loss of .22 in the first iteration, and the subsequent iterations are .96 and .11.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does the accuracy grow so quickly, and then just stop growing?&lt;/p&gt;&#xA;" OwnerUserId="17272" LastActivityDate="2019-09-09T20:08:15.433" Title="Accuracy too high too fast?" Tags="&lt;training&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14263" PostTypeId="1" CreationDate="2019-09-02T21:50:20.247" Score="3" ViewCount="22" Body="&lt;p&gt;Was just experimenting with something when i ran into this error : I am getting matrix dimension errors only when i use L1 Regularizer. I have checked and the regularizer itself doesn't change the shape of the weight matrix. The error is gone if i remove the L1 regularzer in 3rd layer or make the number of nodes = 4 (same) in all the hidden layers. As far as i understand the regularizer should act on its current layer add the norm of the vectors to the cost of current layer and not to the next layer.&lt;/p&gt;&#xA;&lt;p&gt;Actual Error occurs during &lt;code&gt;model.complie()&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Dimensions must be equal, but are 4 and 3 for 'loss_15/add_1' (op: 'Add') with input shapes: [4,3], [3,2].&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H1XTH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H1XTH.png&quot; alt=&quot;r3&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28395" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-09-04T20:19:07.520" Title="L1 Reguarizer in Keras model throwing weight matrix dimension error" Tags="&lt;keras&gt;&lt;regularization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14264" PostTypeId="2" ParentId="14224" CreationDate="2019-09-02T23:19:56.493" Score="17" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do we need explainable AI?&#xA;  ... why we need to know &quot;how does its intelligence work?&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because anyone with access to the equipment, enough skill, and enough time, can force the system to make a decision that is unexpected. The owner of the equipment, or 3rd parties, relying on the decision without an explanation as to why it is correct would be at a disadvantage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples - Someone &lt;strong&gt;&lt;em&gt;might&lt;/em&gt;&lt;/strong&gt; discover:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;People whom are named John Smith and request heart surgery on: Tuesday mornings, Wednesday afternoons, or Fridays on odd days and months have a 90% chance of moving to the front of the line.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Couples whom have the male's last name an odd letter in the first half of the alphabet and apply for a loan with a spouse whose first name begins with a letter from the beginning of the alphabet are 40% more likely to receive the loan if they have fewer than 5 bad entries in their credit history.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Notice that the above examples &lt;em&gt;ought not&lt;/em&gt; to be determining factors in regards to the question being asked, yet it's possible for an adversary (with their own equipment, or knowledge of the algorithm) to exploit it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source papers&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;&lt;a href=&quot;https://arxiv.org/abs/1908.08705&quot; rel=&quot;noreferrer&quot;&gt;AdvHat: Real-world adversarial attack on ArcFace Face ID system&lt;/a&gt;&quot; (Aug 23 2019) by Stepan Komkov and Aleksandr Petiushko&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Creating a sticker and placing it on your hat fools facial recognition system.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;&lt;a href=&quot;https://arxiv.org/abs/1906.03444&quot; rel=&quot;noreferrer&quot;&gt;Defending against Adversarial Attacks through Resilient Feature Regeneration&lt;/a&gt;&quot; (Jun 8 2019), by Tejas Borkar, Felix Heide, and Lina Karam&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Deep neural network (DNN) predictions have been shown to be vulnerable to carefully crafted adversarial perturbations. Specifically, so-called universal adversarial perturbations are image-agnostic perturbations that can be added to any image and can fool a target network into making erroneous predictions. Departing from existing adversarial defense strategies, which work in the image domain, we present a novel defense which operates in the DNN feature domain and effectively defends against such universal adversarial attacks. Our approach identifies pre-trained convolutional features that are most vulnerable to adversarial noise and deploys defender units which transform (regenerate) these DNN filter activations into noise-resilient features, guarding against unseen adversarial perturbations.&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;&lt;a href=&quot;https://arxiv.org/abs/1710.08864&quot; rel=&quot;noreferrer&quot;&gt;One pixel attack for fooling deep neural networks&lt;/a&gt;&quot; (May 3 2019), by Jiawei Su, Danilo Vasconcellos Vargas, and Sakurai Kouichi&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Altering &lt;strong&gt;one pixel&lt;/strong&gt; can cause these errors:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X9yf8.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X9yf8.jpg&quot; alt=&quot;Figure 1&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;  &lt;sub&gt;Fig. 1. One-pixel attacks created with the proposed algorithm that successfully fooled three types of DNNs trained on CIFAR-10 dataset: The All convolutional network (AllConv), Network in network (NiN) and VGG. The original class labels are in black color while the target class labels and the corresponding confidence are given below.&lt;/sub&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6RdaP.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6RdaP.jpg&quot; alt=&quot;Figure 2&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;  &lt;sub&gt;Fig. 2. One-pixel attacks on ImageNet dataset where the modified pixels are highlighted with red circles. The original class labels are in black color while the target class labels and their corresponding confidence are given below.&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Without an explanation as to how and why a decision is arrived at the decision can't be absolutely relied upon.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2019-09-02T23:19:56.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14265" PostTypeId="2" ParentId="14224" CreationDate="2019-09-02T23:28:20.200" Score="2" Body="&lt;p&gt;In addition to all these answers mentioning the more practical reasons of why we'd want explainable AIs, I'd like to add a more philosophical one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding how things around us work is one of the main driving forces of science from antiquity. If you don't have an understanding of how things work, you can't evolve beyond that point. Just because &quot;gravity works&quot; hasn't stopped us trying to understand how it works. In turn a better understanding of it led to several key discoveries, which have helped us advance our technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, if we stop at &quot;it works&quot; we will stop improving it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI hasn't been just about making &quot;machines think&quot;, but also through them to understand how the human brain works. AI and neuroscience go &lt;a href=&quot;https://deepmind.com/blog/article/ai-and-neuroscience-virtuous-circle&quot; rel=&quot;nofollow noreferrer&quot;&gt;hand-by-hand&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This all wouldn't be possible without being able to explain AI.&lt;/p&gt;&#xA;" OwnerUserId="27655" LastEditorUserId="27655" LastEditDate="2019-09-04T19:02:18.310" LastActivityDate="2019-09-04T19:02:18.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14267" PostTypeId="1" CreationDate="2019-09-03T00:00:04.173" Score="-2" ViewCount="2686" Body="&lt;p&gt;Update 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The OS I'm using is Windows 10, since we have WSL, I also use Ubuntu to run the code. The code is written in Python. I know there are thousands of factors which affect the final performance for model training, let's put it aside. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scope of this post is to limited to talk about the very specific 2 CPUs: ThreadRipper 2920X and Ryzen 7 3700X in terms of performance for AI model training. Nothing more than this.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update 1 for more information about what I am trying to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;initial motivation&lt;/strong&gt; for this question:&#xA;My mom has a fish pond in her backyard. Throughout years she found that birds come and stole fish and none of fake owl standing besides pond works, so she ends up covering the pond with a giant and ugly net which destroys all good sights in the backyard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to train a model which recognizes birds, full stop and as simple as this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have huge amount of data downloaded. How big? it contains 11K bird images for training, 5K for validation and 3K for testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those who said a GPU is not needed, I agree. A cpu is not needed for some cases, but definitely not in my case. Why? coz I have started running my training since Saturday 2 weeks ago for 30 epochs. Since then it's been more than 1 week and only 12 epochs have been done. So I definitely need a GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But let's get back to the topic. This topic is not about GPU, this topic is about comparing very specific CPUs for the sake of ML in terms of performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CPUs mentioned here are AMD ThreadRipper 2920X Gen2 VS Ryzen 7 3700X&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;First of all, I know GPU matters much more than CPU and I have nailed down my GPU to be NVIDIA 2080 TI, so not asking for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read several articles and found &lt;a href=&quot;https://medium.com/the-mission/how-to-build-the-perfect-deep-learning-computer-and-save-thousands-of-dollars-9ec3b2eb4ce2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; really helpful. In this article, it recommends to use AMD ThreadRipper 2920X and here is &lt;a href=&quot;https://pcpartpicker.com/b/nrjypg&quot; rel=&quot;nofollow noreferrer&quot;&gt;the build&lt;/a&gt;.&#xA;But I wend to &lt;a href=&quot;https://cpu.userbenchmark.com/Compare/AMD-Ryzen-TR-2920X-vs-AMD-Ryzen-7-3700X/m625966vs4043&quot; rel=&quot;nofollow noreferrer&quot;&gt;UserBenchMark&lt;/a&gt; and found Ryzen 7 3700X actually is newer, cheaper and performs better in most aspects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, should I keep everything same but just replace Threadripper 2920X with Ryzen 7 3700X? or should I stick to TR 2920X? and Why?&lt;/p&gt;&#xA;" OwnerUserId="27988" LastEditorUserId="2444" LastEditDate="2019-09-04T12:54:54.903" LastActivityDate="2019-09-04T12:54:54.903" Title="Should I use the Threadripper 2920X or Ryzen 7 3700X?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hardware&gt;" AnswerCount="2" CommentCount="10" ClosedDate="2020-05-13T20:08:43.973" ContentLicense="CC BY-SA 4.0" />
  <row Id="14268" PostTypeId="2" ParentId="14219" CreationDate="2019-09-03T03:53:32.623" Score="0" Body="&lt;p&gt;After some more research, I have found the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An LSTM is comprised of 1 LSTM cell that is continuously updated by passing in new inputs, the hidden state and the previous output. All nodes inside the LSTM cell are of size hidden_units. That means the output of the activation gate, input gate, forget gate and output gate are all of size hidden_units. As an extension of this, the size of the hidden state is also equal to hidden_units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem arises when the desired output is not equal to that of hidden_units. This is fixed simple and easily, by slapping a basic feed forward net onto the end of the output of the LSTM, so even if it outputs 256, you can convert it to n nodes. These final output nodes are typically put through a softmax with cross-entropy loss applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came to the conclusion by looking at the code included here: &lt;a href=&quot;https://towardsdatascience.com/lstm-by-example-using-tensorflow-feb0c1968537&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/lstm-by-example-using-tensorflow-feb0c1968537&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In which you can see the final output has its own separate weights and bias'.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-09-03T03:53:32.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14269" PostTypeId="2" ParentId="14267" CreationDate="2019-09-03T06:51:26.003" Score="2" Body="&lt;p&gt;Usually, the only relevant criterion is whether the CPU can feed the GPU fast enough. How much preprocessing does your task require? &lt;/p&gt;&#xA;" OwnerUserId="16378" LastActivityDate="2019-09-03T06:51:26.003" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14270" PostTypeId="2" ParentId="14246" CreationDate="2019-09-03T08:31:02.280" Score="5" Body="&lt;p&gt;Although I am not aware of any &quot;benchmark problems&quot; for (discrete) MDPs, I'll comment a bit on possible benchmarks and I will show some benchmarks used to test POMDP algorithms.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;MDP vs POMDP&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markovian Decision Processes (MDPs)&lt;/a&gt; the whole state space is known, this means you know all the information for your problem; therefore, you can use them to find solutions for &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;perfect information&lt;/em&gt;&lt;/a&gt; problems or games. &#xA;Many of these games could use an MDP, some examples: &lt;a href=&quot;https://jdlm.info/articles/2018/03/18/markov-decision-process-2048.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2048&lt;/a&gt; and &lt;a href=&quot;https://www.chessprogramming.org/Reinforcement_Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;chess&lt;/a&gt;. Note that you must take into mind that the &lt;a href=&quot;http://web.mit.edu/jnt/www/Papers/J016-87-mdp-complexity.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational complexity&lt;/a&gt; grows with the number of states.&#xA;Although I could not find any benchmarks for MDPs, games with perfect information can be used to compare MDP solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a problem or game has &lt;em&gt;imperfect information&lt;/em&gt;, you should use a  &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Partially Observable Markovian Decision Processes (POMDPs)&lt;/em&gt;&lt;/a&gt;; in which you do not need to know the current state, but you keep track of the probabilities of being in any of the (discrete) states.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;POMDP Benchmarks&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Since I worked with &lt;em&gt;POMDPs&lt;/em&gt;, I will comment some of the benchmarks researches used for discrete POMDPs (Pineau et al. (2003), Spaan and Vlassis (2004), Kurniawati et al. (2008), Ong et al. (2010), ArayaLopez et al. (2010)):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Tag&lt;/em&gt;: a robot and target move in a grid environment and can move one step at a time, moving has a cost, and a reward is gained if the robot is at the same position as the target (i.e. tagged it).&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Two-Robot Tag&lt;/em&gt;: two robots attempt to catch a target, thereby sharing their observations and actions; the target tries to get away from them.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Mazes&lt;/em&gt; (Littman et al. (1995), Kaelbling et al. (1998), Spaan and Vlassis (2004)): &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Hallway&lt;/em&gt; and &lt;em&gt;Hallway2&lt;/em&gt; are robot navigation tasks in a hallway, where the robot has only local noisy sensor information. The difficulty of hallways is it being long areas which look alike, which causes ambiguity in the localization.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Tiger-grid&lt;/em&gt; a two world state with a tiger being either behind the left or right door. The actions are &lt;em&gt;listen&lt;/em&gt;, &lt;em&gt;open the right&lt;/em&gt; or &lt;em&gt;left&lt;/em&gt; door and there is a positive reward when opening the door without the tiger, otherwise a large negative reward.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Rock Sample&lt;/em&gt;: a rover explores a grid area, it knows its own position and the position of the rocks, however, it does not know which rocks are valuable. The rover can sense how valuable they are, but this sensor is less reliable when it is used farther away.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gKzqF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gKzqF.png&quot; alt=&quot;tag-game&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;The &lt;em&gt;tag&lt;/em&gt; game: the robot (blue) and target on a map with 29 positions and 870 states (29 for the robot, 29 + 1 (tagged) for the target).&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These problems tend to be of the same size (number of states and actions) such that the results of different algorithms can be compared easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&lt;strong&gt;References&lt;/strong&gt;:&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;sup&gt;Araya-Lopez, M., Thomas, V., Buffet, O., and Charpillet, F. (2010). A closer look at MOMDPs. In 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, volume 2, pages 197–204.&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;sup&gt;Kaelbling, L.P., Littman, M.L., Cassandra, A.R. (1998). &#xA;Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1-2): 99-134&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;sup&gt;Kurniawati, H., Hsu, D., and Lee, W. (2008). SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. In Proceedings of Robotics: Science and Systems IV, Zurich, Switzerland.&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;sup&gt;Littman, M.L., Cassandra, A.R. and Kaelbling, L.P. (1995). Learning policies for partially observable environments: Scaling up. in Proc. 12th Int. Conf. on Machine Learning, San Francisco, CA.&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;sup&gt;Ong, S. C. W., Png, S. W., Hsu, D., and Lee, W. S. (2010). Planning under Uncertainty for Robotic Tasks with Mixed Observability. The International Journal of Robotics Research, 29(8):1053–1068.&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;sup&gt;Pineau, J., Gordon, G., and Thrun, S. (2003). Point-based value iteration: An anytime algorithm for POMDPs. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI),&#xA;pages 477–484.&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;sup&gt;Spaan, M. T. J. and Vlassis, N. (2004). A point-based POMDP algorithm for robot planning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 2399–2404, New Orleans, Louisiana.&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="198" LastEditorUserId="198" LastEditDate="2019-09-15T15:54:24.273" LastActivityDate="2019-09-15T15:54:24.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14271" PostTypeId="2" ParentId="14267" CreationDate="2019-09-03T13:40:13.040" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;First of all, I know GPU matters much more than CPU&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general this is wrong or very naive. See also &lt;a href=&quot;https://ai.stackexchange.com/a/2237/3335&quot;&gt;this&lt;/a&gt; answer (then consider &lt;a href=&quot;https://sbcl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBCL&lt;/a&gt;) and &lt;a href=&quot;https://softwareengineering.stackexchange.com/a/323147/40065&quot;&gt;that&lt;/a&gt; one (and also &lt;a href=&quot;https://ai.stackexchange.com/a/4264/3335&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/13998/3335&quot;&gt;that&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The OS I'm using is Windows 10,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is your biggest mistake, assuming you have access (and some understanding) to the source code of your application. See below why. Notice that the most expansive resource is your brainpower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you might deal with a large volume of data (e.g. a few terabytes, e.g. of post-processed raw HTML texts with NLP preprocessing in mind). In that case, SSD or RAM may matter much more than the GPU. See also &lt;a href=&quot;https://softwareengineering.stackexchange.com/a/332071/40065&quot;&gt;this&lt;/a&gt;, but consider also  any serious free software &lt;a href=&quot;https://en.wikipedia.org/wiki/Database#Database_management_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;database management system&lt;/a&gt; (e.g. &lt;a href=&quot;https://postgresql.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PostGreSQL&lt;/a&gt;, &lt;a href=&quot;https://www.mongodb.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MongoDB&lt;/a&gt;, etc etc etc...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the contrary, in some applications, the volume of data is small, but its incoming bandwidth is very important (real-time video applications, à la autonomous vehicles).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a comment, you mention bird recognition. Look into &lt;a href=&quot;http://starynkevitch.net/Basile/Workshop-StaticCodeAnalysIoT-Starynkevitch-oct2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these&lt;/a&gt; slides and their mention of &lt;a href=&quot;https://xkcd.com/1425/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://xkcd.com/1425/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ask yourself: do you have hundred of gigabytes of bird photos? &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tagged&lt;/a&gt; by humans? Do you want to process them quickly for an industrial application, or are you just starting to learn machine learning techniques. You'll need a decade to understand &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt;, but you could learn it with a cheap PC (or even a Raspberry Pi). You should take months or years to read books and papers (and wikipages).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be aware that machine learning techniques can be &lt;a href=&quot;https://www.splunk.com/en_us/form/5-big-myths-of-ai-and-machine-learning-debunked.html?utm_campaign=google_emea_tier1_en_search_generic_ml&amp;amp;utm_source=google&amp;amp;utm_medium=cpc&amp;amp;utm_content=AI_ML_Myths_Debunked_EB&amp;amp;utm_term=machine%20learning_bk=machine%20learning&amp;amp;_bt=337909453339&amp;amp;_bm=p&amp;amp;_bn=g&amp;amp;_bg=69670838324&amp;amp;_bk=machine%20learning&amp;amp;gclid=CjwKCAjwnrjrBRAMEiwAXsCc45S_d6elg5ko_XlYUNiHHaAdCztDh3quT7DEVTw-corkbNnarUDuuhoCjKsQAvD_BwE&quot; rel=&quot;nofollow noreferrer&quot;&gt;tricked&lt;/a&gt; (see also &lt;a href=&quot;https://www.youtube.com/watch?v=QZlCsg96lzw&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://norvig.com/21-days.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; (look &lt;em&gt;who&lt;/em&gt; wrote that!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to learn more about &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_science&quot; rel=&quot;nofollow noreferrer&quot;&gt;computer science&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;computer architecture&lt;/a&gt;, several &lt;a href=&quot;https://www.cs.rochester.edu/~scott/pragmatics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;programming languages&lt;/a&gt;, &lt;a href=&quot;http://pages.cs.wisc.edu/~remzi/OSTEP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;operating systems&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Introduction_to_Algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithms&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Free_software&quot; rel=&quot;nofollow noreferrer&quot;&gt;free software&lt;/a&gt; (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; and &lt;a href=&quot;http://gudhi.gforge.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ghudi&lt;/a&gt; ...), &lt;a href=&quot;https://en.wikipedia.org/wiki/Linux_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linux&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you code something which classifies successfully just a hundred thousands of bird images, going to ten million images is simply a matter of money. But learning to code that thing will take you months or years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For learning and job opportunities purposes, &lt;strong&gt;investing more in books and less in hardware could be worthwhile&lt;/strong&gt;, especially if you &lt;a href=&quot;https://softwareengineering.stackexchange.com/a/275146/40065&quot;&gt;contribute&lt;/a&gt; to some &lt;em&gt;existing&lt;/em&gt; open source software project (with your real name) on &lt;a href=&quot;https://github.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt; or &lt;a href=&quot;https://gitlab.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gitlab&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used neural networks in the previous century on Sun workstations running some SunOS5, before even GPUs, as we know them today, existed. That (proprietary) code was on Unix, and today many free software or open source machine learning software exist (e.g. on &lt;a href=&quot;https://github.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt; etc) and runs on most Linux PCs. Most of these libraries don't require GPUs, but some of them can take advantage of CUDA or of &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenCL&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCL&lt;/a&gt; (and OpenCL don't need a GPU, just take advantage of it when you enable it; an OpenCL code just runs a bit slower without GPUs, by a practical time factor of 10x to 1000x). Look also into &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenACC&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenACC&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://en.wikipedia.org/wiki/Message_Passing_Interface&quot; rel=&quot;nofollow noreferrer&quot;&gt;MPI&lt;/a&gt;. You'll find free software (or at least gratis) implementations of all of them on Linux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a comment you also ask; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;which CPU feeds the GPU better?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But that also depends not only on the executed code, but also on the data. And the data of your problem is, per se, problem specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more concrete words, the same code (on Linux, e.g. using &lt;a href=&quot;https://en.wikipedia.org/wiki/TensorFlow&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; suitably configured to take advantage of your GPU) using OpenCL to drive the GPU will feed it in a rhythm which depends mostly of the data itself - not just of the code. Both GPU and CPU are involved, and that is exactly why OpenCL programming is so difficult. (I happened to try automatize that in a former &lt;a href=&quot;http://www.opengpu.net/EN/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenGPU&lt;/a&gt; project).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to solve a particular machine learning problem faster and already legally have the &lt;a href=&quot;https://en.wikipedia.org/wiki/Source_code&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; for that (running on Linux), consider also &lt;strong&gt;&lt;a href=&quot;https://www.dummies.com/programming/cloud-computing/renting-infrastructure-in-cloud-computing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;renting&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Cloud_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cloud computing&lt;/a&gt; resources&lt;/strong&gt;, and cloud computing is mostly used for machine learning purposes (read also about &lt;a href=&quot;http://www.shoshanazuboff.com/new/recent-publications-and-interviews/big-other-surveillance-capitalism-and-the-prospects-of-an-information-civilization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;surveillance capitalism&lt;/a&gt;, since your question is at the heart of it). Most of them have fast GPGPUs. That is probably the cheapest and greenest solution for your concrete problem. Today, all cloud computers and every &lt;a href=&quot;https://www.top500.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;top500 supercomputer&lt;/a&gt; run on some variant of Linux. And cloud computing is counter-unintuitively greener than an equivalent PC box (at least in France, where electric power is mostly nuclear).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned a AMD ThreadRipper 2920X. I am sure that, every thing else being the same, my AMD 2970WX Linux workstation would solve your problem faster than your AMD 2920 based Linux system (because an 2970 has more cores!), with the &lt;em&gt;same&lt;/em&gt; source code. And Linux (used in datacenters on the Cloud, in top500 supercomputers) is the most practically useful operating system for machine learning. For machine learning programming you should use Linux (not Windows) because it usually takes better advantage of multi-core machines, supercomputer, cloud computers.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The scope of this post is to limited to talk about the very specific 2 CPUs: ThreadRipper 2920X and Ryzen 7 3700X in terms of performance for AI model training. Nothing more than this.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Did you google that question: &lt;a href=&quot;https://www.google.com/search?q=threadripper%202920%20vs%20Ryzen%207%203700X&amp;amp;ie=utf-8&amp;amp;oe=utf-8&amp;amp;client=firefox-b-e&quot; rel=&quot;nofollow noreferrer&quot;&gt;ThreadRipper 2920X vs Ryzen 7 3700X&lt;/a&gt; ? The following step is to use your intuition to find out, amongst the various benchmarks mentioned in answers, which one is the most similar to your code (assuming you understand it). Think also of energy costs.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The code is written in Python.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Are you aware that Python &lt;a href=&quot;https://www.netguru.com/blog/python-vs.-c-for-machine-learning-language-comparison&quot; rel=&quot;nofollow noreferrer&quot;&gt;is&lt;/a&gt; a lot slower than C++? Of course, most of the computer time would be spent in numerical libraries (those coded &lt;em&gt;also&lt;/em&gt; in OpenCL to use your GPUs). But the same algorithm coded in C++ (or even in Go) and in Python is a lot faster in C++ or Go. Depending on the problem, you could win just a few percents, or a factor of ten (be aware of Python's &lt;a href=&quot;https://en.wikipedia.org/wiki/Global_interpreter_lock&quot; rel=&quot;nofollow noreferrer&quot;&gt;GIL&lt;/a&gt; and consider using &lt;a href=&quot;https://www.gnu.org/software/guile/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Guile&lt;/a&gt; - you could be more productive in it than with Python. See &lt;a href=&quot;https://mitpress.mit.edu/sites/default/files/sicp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SICP&lt;/a&gt;, still the best introduction to programming I ever read).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So I definitely need a GPU.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, &lt;strong&gt;you need more computing power.&lt;/strong&gt; And you could rent it and build then use some Web service above that using a cheap Wifi connection (even with a RaspBerryPi) to the Internet. Think also of &lt;a href=&quot;https://en.wikipedia.org/wiki/Internet_of_things&quot; rel=&quot;nofollow noreferrer&quot;&gt;IoT&lt;/a&gt; approaches to your (still unclear) problem. In most parts of Europe or USA, IoT approaches (combined with cloud computing) make a lot of economical sense. For pond surveillance, they probably are the cheapest. And I cannot imagine why pond surveillance is so confidential to forbid such a mixed (IoT + cloud computing) approach. Your pond surveillance problem is &lt;em&gt;not&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Safety-critical_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;safety-critical&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is, should I keep everything same but just replace Threadripper 2920X with Ryzen 7 3700X? or should I stick to TR 2920X? and Why?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That question is off-topic here. &lt;a href=&quot;https://superuser.com/&quot;&gt;https://superuser.com/&lt;/a&gt; could be a better place for it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. A few paragraphs of my continously evolving &lt;a href=&quot;http://starynkevitch.net/Basile/bismon-chariot-doc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bismon draft report&lt;/a&gt;, and several references there, the &lt;a href=&quot;https://www.decoder-project.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DECODER&lt;/a&gt; project, are very relevant to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PPS. Read more about &lt;a href=&quot;https://en.wikipedia.org/wiki/Technology_readiness_level&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRL&lt;/a&gt;s. For learning purposes, they don't matter a lot (but your own time does). For commercial or lucrative purposes, it is the main issue (and costs depend exponentially on the TRL level).&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2019-09-04T06:25:04.723" LastActivityDate="2019-09-04T06:25:04.723" CommentCount="17" ContentLicense="CC BY-SA 4.0" />
  <row Id="14272" PostTypeId="2" ParentId="14147" CreationDate="2019-09-03T14:37:25.963" Score="1" Body="&lt;p&gt;Even if you want to re-train your model for just one new class you will have to prepare your training data such that it includes all or most of the classes which you want to predict. Most of the times last two layers of a network have the data of number of labels which are to be predicted and that should always be sum of the number of classes you already trained on and the number of classes you want to add for training.&lt;/p&gt;&#xA;" OwnerUserId="28430" LastActivityDate="2019-09-03T14:37:25.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14273" PostTypeId="1" AcceptedAnswerId="14285" CreationDate="2019-09-03T18:45:31.760" Score="1" ViewCount="155" Body="&lt;p&gt;I tried &lt;a href=&quot;https://androidkt.com/multi-label-text-classification-in-tensorflow-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; example for a multi class classifier, but when looking at the data I realized two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There are many examples of &quot;all zeros&quot; vectors, that is, messages that don't belong in any classification.&lt;/li&gt;&#xA;&lt;li&gt;These all-zeros are actually the majority, by far.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is it valid to have an all-zeros output for a certain input? I would guess a Sigmoid activation would have no problems with this, by simply not trying to force a one out of all the &quot;near zero&quot; outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I also think an &quot;accuracy&quot; metric will be skewed too optimistically: if all outputs are zero 90% of the time, the network will quickly overfit to always output 0 all the time, and get 90% score.&lt;/p&gt;&#xA;" OwnerUserId="17272" LastActivityDate="2019-10-04T11:01:15.160" Title="Class imbalance and &quot;all zeros&quot; one-hot encoding?" Tags="&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14277" PostTypeId="2" ParentId="14205" CreationDate="2019-09-04T01:16:33.877" Score="0" Body="&lt;p&gt;It should return an empty mask. Like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mask = np.empty([0, 0, 0]) &#xA;class_ids = np.empty([0], np.int32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17793" LastActivityDate="2019-09-04T01:16:33.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14278" PostTypeId="2" ParentId="14107" CreationDate="2019-09-04T01:25:30.100" Score="0" Body="&lt;p&gt;I might consider 2 models (throw away col 1 and throw away col 4), and one more that keeps both, and see which generalises better to test set.&lt;/p&gt;&#xA;" OwnerUserId="17793" LastActivityDate="2019-09-04T01:25:30.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14279" PostTypeId="2" ParentId="14012" CreationDate="2019-09-04T01:41:53.750" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;One idea I had was to create an adversarial network which generates words&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It may be better to use for e.g. numbers instead of words, then map these numbers to the language of your choice like English in your case. This mapping can be changed to other languages and you will have a robot that overcomes language barriers.&lt;/p&gt;&#xA;" OwnerUserId="17793" LastActivityDate="2019-09-04T01:41:53.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14280" PostTypeId="1" CreationDate="2019-09-04T01:58:24.943" Score="2" ViewCount="1551" Body="&lt;p&gt;I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such &quot;classic&quot; models are frequently referenced as starting points when building new CNNs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there similar examples for RNN/LSTM networks? All I've found so far are articles and slides explaining recurrent neurons, LSTM layers, and the math behind them, but no well-known examples of entire multi-layered network architectures, unlike CNNs which seem to have in abundance.&lt;/p&gt;&#xA;" OwnerUserId="23844" LastActivityDate="2019-09-05T12:07:45.953" Title="What are some examples of LSTM architectures?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14282" PostTypeId="2" ParentId="14262" CreationDate="2019-09-04T09:42:30.630" Score="1" Body="&lt;p&gt;As you have trained your model in batch_size of 500. Weights has been updated for each batch therefore 600 times(300000/500) by the end of one epoch.&lt;br&gt;&#xA;So, Your model generalized well. Check the predictions. If Predictions are well. Your model is ready.&lt;/p&gt;&#xA;" OwnerUserId="28455" LastActivityDate="2019-09-04T09:42:30.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14284" PostTypeId="1" CreationDate="2019-09-04T10:52:49.967" Score="1" ViewCount="48" Body="&lt;p&gt;I am solving an RL MDP problem which is model based. I have an MDP which has four possible states S1-S4 and four different actions A1-A4, with S4 being terminal state and S1 is the beginning state. There is equal probability of applying any of the available actions on S1. The goal of my problem is to get from S1 to S4 with the maximum possible reward. I have two questions in this regard - &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Will this be a valid MDP model if I have this rule for my model - If i perform action a1 on S1 and next state is S2 then the set of available valid actions on S2 will be only a2,a3 and a4. If i apply any of the actions a2,a3 or a4 on S2 and it takes me to S3 then now i am left with a set of two valid actions for S3(except the one that was taken on s2). Can i do this? Because in my problem an action once taken does not require to be taken again later. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am confused about finding the optimal value and policy for my MDP. Both the optimal value and policy functions are not known to me. The objective is to find the optimal policy for my MDP which would get me from S1 to S4 with max reward. Any action taken on a particular state can lead to any other states (i.e there is a uniform equal state transition probability of 25% in my case for all states except S4 since it's terminal state). How can i approach this problem? After a lot of google search I vaguely understood that I must start with choosing a random policy (equal probability of taking any valid action) -&gt; find value functions for each state -&gt; iteratively compute V until it converges and then from these value functions I need to compute the optimal policy. Those solutions mostly use the Bellman Equations. Can someone please elaborate on how i can do this? Or if there is any other method to do it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="28314" LastActivityDate="2019-09-04T10:52:49.967" Title="Finding optimal Value function and Policy for an MDP" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;policies&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14285" PostTypeId="2" ParentId="14273" CreationDate="2019-09-04T10:56:25.700" Score="0" Body="&lt;p&gt;I understand your questions as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it valid to have an all-zeros output for a certain input?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes its possible in some cases like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In the tutorial &quot;jigsaw-toxic-comment-classification-challenge&quot;, the data is taken from wikipedia comments, so generally there is no toxic behavior in people's comment, it is very rare that a person posts &quot;bad&quot; comments in an informative source like wikipedia, this might lead to all the labels being zero for a particular example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In a single label classification like predicting whether a person has a rare disease,the dataset would contain labels that are mostly &quot;zero&quot;-(no disease)-skewed towards zero&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This happens in datasets where the positive output to be predicted is a very rare case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I guess your second question is -whether &quot;accuracy&quot; is a good way of evaluating a model for this type of a problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are right, in such a case &lt;strong&gt;even&lt;/strong&gt; a simple program/model that outputs zeros for all the inputs would achieve an accuracy of more than 90%, &lt;strong&gt;so&lt;/strong&gt; here &lt;strong&gt;accuracy&lt;/strong&gt; is &lt;strong&gt;not&lt;/strong&gt; a good metric to evaluate the model on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should go through the metrics &lt;strong&gt;f1_score&lt;/strong&gt;,&lt;strong&gt;recall&lt;/strong&gt;, and &lt;strong&gt;precision&lt;/strong&gt; which are ideal for this type of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically here we are interested in &quot;out of those which are predicted positive, how many are really positive -precision&quot; and &quot;out of those which should have been predicted positive, how many are really predicted positive&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my definitions seem confusing to understand, please got through the link below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9&quot; rel=&quot;nofollow noreferrer&quot;&gt;f1_score/recall/precision&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I was helpful&lt;/p&gt;&#xA;" OwnerUserId="26854" LastActivityDate="2019-09-04T10:56:25.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14286" PostTypeId="2" ParentId="14280" CreationDate="2019-09-04T11:04:53.203" Score="2" Body="&lt;p&gt;In the paper, &lt;a href=&quot;https://arxiv.org/pdf/1503.04069.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM: A Search Space Odyssey&lt;/a&gt; (2017), by Klaus Greff et al., eight LSTM variants on three representative tasks (&lt;a href=&quot;https://en.wikipedia.org/wiki/Speech_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;speech recognition&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Handwriting_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;handwriting recognition&lt;/a&gt;, and &lt;a href=&quot;https://arxiv.org/pdf/1206.6392.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;polyphonic music modeling&lt;/a&gt;) are compared. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The compared variants are &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;ftp://ftp.idsia.ch/pub/juergen/nn_2005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vanilla LSTM&lt;/a&gt; features three gates (input, forget, output), block input, a single cell, an output activation function, and &lt;a href=&quot;https://pdfs.semanticscholar.org/e10f/98b86797ebf6c8caea6f54cacbc5a50e8b34.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;peephole connections&lt;/a&gt; (connections from the cell to the gates). The output of the block is recurrently connected back to the block input and all of the gates. The &lt;a href=&quot;ftp://ftp.idsia.ch/pub/juergen/nn_2005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;vanilla LSTM&lt;/a&gt; is trained using gradient descent and &lt;a href=&quot;http://axon.cs.byu.edu/~martinez/classes/678/Papers/Werbos_BPTT.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;back-propagation through time&lt;/a&gt; (BPTT). The &lt;a href=&quot;http://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original LSTM&lt;/a&gt; (which is not the vanilla LSTM) does not contain, for example, the forget gate or the peephole connections (but the cell possesses a &lt;em&gt;constant error carousel&lt;/em&gt;, a constant weight of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;LSTM trained based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Extended_Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;decoupled extended Kalman filtering&lt;/em&gt;&lt;/a&gt; &lt;a href=&quot;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2002-211.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(DEKF-LSTM)&lt;/a&gt;, which enables the LSTM to be trained on some pathological cases at the cost of high computational complexity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://people.idsia.ch/~tino/papers/schmidhuber.neco07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vanilla LSTM trained with an evolution-based method (called &lt;em&gt;evolino&lt;/em&gt;)&lt;/a&gt;, instead of BPTT.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://people.idsia.ch/~juergen/icann2009bayer.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM block architectures evolved with a multi-objective evolutionary algorithm&lt;/a&gt;, so that to maximize fitness on &lt;a href=&quot;https://en.wikipedia.org/wiki/Context-sensitive_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;context-sensitive grammar&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43905.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM architectures for large scale acoustic modeling&lt;/a&gt;, which introduces a &lt;em&gt;linear projection layer&lt;/em&gt; that projects the output of the LSTM layer down before recurrent and forward connections in order to reduce the number of parameters for LSTM networks with many blocks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/6981033&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM architecture with a trainable scaling parameter for the slope of the gate activation functions&lt;/a&gt;, which improves the performance of LSTM on an offline handwriting recognition dataset.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F978-3-319-11179-7_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Cortex Memory&lt;/a&gt;, an LSTM composed of recurrent connections between the gates of a single block, but not between different blocks, which improves the convergence speed of LSTM.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1406.1078&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gated Recurrent Unit (GRU)&lt;/a&gt;, which simplifies the architecture of the LSTM by combining the input and forget gate into an &lt;em&gt;update gate&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are other related neural network architectures, such as the &lt;a href=&quot;https://arxiv.org/pdf/1410.5401.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural Turing machine (NTM)&lt;/a&gt; or &lt;a href=&quot;https://www.nature.com/articles/nature20101&quot; rel=&quot;nofollow noreferrer&quot;&gt;differentiable neural computer (DNC)&lt;/a&gt;. In general, there are several architectures that use LSTM blocks, even though they are not just recurrent neural networks. Other examples are the &lt;a href=&quot;https://arxiv.org/abs/1511.06279&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural programmer-interpreter (NPI)&lt;/a&gt; or the &lt;a href=&quot;https://arxiv.org/pdf/1705.02670.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta-controller&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-05T12:07:45.953" LastActivityDate="2019-09-05T12:07:45.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14287" PostTypeId="2" ParentId="7390" CreationDate="2019-09-04T12:23:58.057" Score="2" Body="&lt;p&gt;According to Sutton and Barto, they are the same thing. Note 13.5-6 (page 338) of their &lt;em&gt;Reinforcement Learning: An Introduction, 2nd Edition&lt;/em&gt; book:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Actor-critic methods are sometimes referred to as advantage actor-critic methods in the literature&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="28474" LastEditorUserId="2444" LastEditDate="2020-05-14T10:22:43.137" LastActivityDate="2020-05-14T10:22:43.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14289" PostTypeId="1" CreationDate="2019-09-04T12:49:02.230" Score="2" ViewCount="30" Body="&lt;p&gt;I would like to perform Machine Translation in tensorflow.js, in the browser. The issue is that state-of-the-art models have many gigabytes (&lt;a href=&quot;https://github.com/pytorch/fairseq/blob/master/examples/wmt19/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;fairseq ensemble&lt;/a&gt;), and translation is slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know some good small models for Machine Translation? Probably something below the 50 million parameter threshold. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some older papers with models below 100 MB for training on small datasets (&lt;a href=&quot;https://nlp.stanford.edu/projects/nmt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IWSLT'15 English-Vietnamese attention-based models&lt;/a&gt;), but these are probably superseded.&lt;/p&gt;&#xA;" OwnerUserId="28473" LastActivityDate="2019-09-04T12:49:02.230" Title="Small Machine Translation Model" Tags="&lt;machine-translation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14290" PostTypeId="1" AcceptedAnswerId="14291" CreationDate="2019-09-04T22:51:34.940" Score="3" ViewCount="1625" Body="&lt;p&gt;When watching the machine learning course on Coursera by Andrew Ng, in the logistic regression week, the cost function was a bit more complex than the one for linear regression, but definitely not that hard. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it got me thinking, why not use the same cost function for logistic regression? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the cost function would be &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2m} \sum_{i}^m|h(x_i) - y_i|^2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h(x_i)$&lt;/span&gt; is our hypothesis &lt;span class=&quot;math-container&quot;&gt;$\text{function}(\text{sigmoid}(X * \theta))$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of training examples and &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; are our &lt;span class=&quot;math-container&quot;&gt;$ith$&lt;/span&gt; training example?&lt;/p&gt;&#xA;" OwnerDisplayName="user17894" LastEditorUserId="2444" LastEditDate="2019-09-05T09:57:18.580" LastActivityDate="2019-09-05T10:56:22.363" Title="Why not use the MSE instead of the current logistic regression?" Tags="&lt;classification&gt;&lt;logistic-regression&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14291" PostTypeId="2" ParentId="14290" CreationDate="2019-09-05T00:34:19.653" Score="4" Body="&lt;p&gt;The mean squared error (MSE), &lt;span class=&quot;math-container&quot;&gt;$J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$&lt;/span&gt;, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the data that are not appropriate for classification. Though, as an optimization objective, it is still possible to attempt to minimize MSE even in a classification problem, and thus still learn parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new cost function has better convergence characteristics as it is more inline with the objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://rohanvarma.me/Loss-Functions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; for the precise mathematical formulation that explains these loss functions from a probabilistic perspective. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the absolute value is redundant because &lt;span class=&quot;math-container&quot;&gt;$\forall x:x^2\geq0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this clarifies the matter.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="2444" LastEditDate="2019-09-05T10:00:48.993" LastActivityDate="2019-09-05T10:00:48.993" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14292" PostTypeId="1" AcceptedAnswerId="14294" CreationDate="2019-09-05T03:41:47.233" Score="0" ViewCount="53" Body="&lt;p&gt;I'm initialising DNN of shape [2 inputs, 2 hiddens, 1 output] with these weights and biases:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#hidden layer&#xA;weight1= tf.Variable(tf.random_uniform([2,2], -1, 1), &#xA;         name=&quot;layer1&quot;);&#xA;bias1 = tf.Variable(tf.zeros([2]), name=&quot;bias1&quot;);&#xA;&#xA;#output layer&#xA;weight2 = tf.Variable(tf.random_uniform([2,1], -1, 1), &#xA;          name=&quot;layer2&quot;);&#xA;bias2 = tf.Variable(tf.zeros([1]), name=&quot;bias2&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's what I followed some online article, however, I wonder what if I initialise bias values using &lt;code&gt;tf.random_uniform&lt;/code&gt; instead of &lt;code&gt;tf.zeros&lt;/code&gt;? Should I choose zero biases or random biases generically?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2019-09-05T10:03:53.930" LastActivityDate="2019-09-05T10:48:37.990" Title="Should the biases be zero or randomly initialised?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;random-variable&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14293" PostTypeId="1" CreationDate="2019-09-05T09:47:14.450" Score="0" ViewCount="106" Body="&lt;p&gt;My project requires gender recognition of people shown on the given images, with more than one person per image. However, these people can be positioned in frontal or side view(passing by perpendicularly, no face visible). On the pictures there will be entire bodies shown, not only the faces. My idea is to firstly use object detection to point where people would be and next use CNNs to recognize gender of each person. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: should I use one object detection algorithm for both frontal and side views of a person and then classify them with one CNN, or should I use object detection to separately find people positioned in frontal and side manner and use two different CNNs, one for classification of frontal views and one for side views?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking this because I think it might be easier for one NN to classify only one view at a time, because side view might have different features than frontal, and mixing this features might be confusing for a network. However I am not really sure. If something is unclear, please let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] Since problem might be hard to understand only by reading, I made some illustrations. Basically I wonder if using second option can help in achieveing better accuracy for the subtle differences like those in gender recognition, especially when face is not visible:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Single detection and classification:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/goxbS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/goxbS.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Two different classifiers:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/umqqE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/umqqE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.istockphoto.com/fr/vectoriel/illustration-vectorielle-de-trois-hommes-daffaires-en-officiel-illustration-de-gm842760028-137596397&quot; rel=&quot;nofollow noreferrer&quot;&gt;Img Source&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2019-09-10T08:28:59.760" LastActivityDate="2021-02-02T02:05:20.567" Title="Should I use single or double view for gender recognition?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14294" PostTypeId="2" ParentId="14292" CreationDate="2019-09-05T10:48:37.990" Score="2" Body="&lt;p&gt;From the stanford CNN class (&lt;a href=&quot;http://cs231n.github.io/neural-networks-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.github.io/neural-networks-2/&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Blockquote&#xA;  Initializing the biases. It is possible and common to initialize the biases to be zero, since the asymmetry breaking is provided by the small random numbers in the weights. For ReLU non-linearities, some people like to use small constant value such as 0.01 for all biases because this ensures that all ReLU units fire in the beginning and therefore obtain and propagate some gradient. However, it is not clear if this provides a consistent improvement (in fact some results seem to indicate that this performs worse) and it is more common to simply use 0 bias initialization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="28448" LastActivityDate="2019-09-05T10:48:37.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14295" PostTypeId="2" ParentId="14290" CreationDate="2019-09-05T10:56:22.363" Score="1" Body="&lt;p&gt;I mean you technically could (it's not going to break or something) however, cross entropy is much better suited for classification as it penalizes for misclassification errors: have a look at the function: when you are wrong the loss goes to infinity: &lt;a href=&quot;https://i.stack.imgur.com/nHVdT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nHVdT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;you are either from one class or another. MSE is designed for regression where you have nuance: you get close to target is sometimes good enough. You should try both and you will see the performance will be much better for the cross entropy.&lt;/p&gt;&#xA;" OwnerUserId="28448" LastActivityDate="2019-09-05T10:56:22.363" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14296" PostTypeId="1" AcceptedAnswerId="14300" CreationDate="2019-09-05T15:08:52.330" Score="3" ViewCount="157" Body="&lt;p&gt;&lt;strong&gt;Problem Statement :&lt;/strong&gt; &#xA;I have a system with four states - S1 through S4 where S1 is the beginning state and S4 is the end/terminal state. The next state is always better than the previous state i.e if the agent is at S2, it is in a slightly more desirable state than S1 and so on with S4 being the most desirable i.e terminal state. We have two different actions which can be performed on any of these states without restrictions. &#xA;Our goal is to make the agent reach state S4 from S1 in the most optimal way i.e the route with maximum reward (or minimum cost). The model i have is a pretty uncertain one so i am guessing the agent must initially be given a lot of experience to make any sense of the environment. The MDP i have designed is shown below : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MDP Formulation :&lt;/strong&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/GuymJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GuymJ.jpg&quot; alt=&quot;MDP for the problem&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MDP might a look a bit messy and complicated but it basically is just showing that any action (A1 or A2) can be taken at any state (except the terminal state S4). The probability with which the transition takes place from one state to the other and the associated rewards are given below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;States :&lt;/strong&gt; States S1 to S4. S4 is terminal state and S1 is the beginning state. S2 is a better state than S1 and S3 is a better state than S1 or S2 and S4 is the final state we expect the agent to end up in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Actions :&lt;/strong&gt; Available actions are A1 and A2 which can be taken at any state (except of course the terminal state S4).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;State Transition Probability Matrix :&lt;/strong&gt; One action taken at a particular state S can lead to any of the other available states. For ex. taking action A1 on S1 can lead the agent to S1 itself or S2 or S3 or even directly S4. Same goes for A2. So i have assumed an equal probability of 25% or 0.25 as the state transition probability. The state transition probability matrix is the same for actions A1 and A2. I have just mentioned it for one action but it is the same for the other action too. Below is the matrix I created - &#xA;&lt;a href=&quot;https://i.stack.imgur.com/lfyOS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lfyOS.jpg&quot; alt=&quot;State transition probability matrix for both the actions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reward Matrix :&lt;/strong&gt; The reward function i have considered is a function of the action, current state and future state - R(A,S,S'). The desired route must go from S1 to S4. I have awarded positive rewards for actions that take the agent from S1 to S2 or S1 to S3 or S1 to S4 and similarly for states S2 and S3. A larger reward is given when the agent moves more than one step i.e S1 to S3 or S1 to S4. What is not desired is when the agent gets back to a previous state because of a action. So i have awarded negative rewards when the state goes back to a previous state. The reward matrix currently is the same for both the actions (meaning both A1 and A2 have same importance but it can be altered if A1/A2 is preferred over the other). Following is the reward matrix i created (same matrix for both the actions) - &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9HX88.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9HX88.jpg&quot; alt=&quot;Reward matrix R(A,S,S&amp;#39;) for both the actions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Policy, Value Functions and moving forward :&lt;/strong&gt; &#xA;Now that i have defined my states, actions, rewards, transition probabilities the next step I guess i need to take is to find the optimal policy. I do not have an optimal value function or policy. From lot of googling i did, I am guessing i should start with a random policy i.e both actions have equal probability of being taken at any given state -&gt; compute the value function for each state -&gt; compute the value functions iteratively until they converge -&gt; then find the optimal policy from the optimal value functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am totally new to RL and all the above knowledge is from whatever i have gathered reading online. Can someone please validate my solution and MDP if I am going the right way? If the MDP i created will work ? &#xA;Apologies for such a big write-up but i just wanted to clearly depict my problem statement and solution. If the MDP is ok then can someone also help me with how can the value function iteratively converge to an optimal value? I have seen lot of examples which are deterministic but none for stochastic/random processes like mine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help/pointers on this would be greatly appreciated. &#xA;Thank you in advance  &lt;/p&gt;&#xA;" OwnerUserId="28314" LastEditorUserId="28314" LastEditDate="2019-09-05T15:18:02.883" LastActivityDate="2019-09-05T21:39:00.713" Title="Can someone please help me validate my MDP?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;policies&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="12" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14297" PostTypeId="2" ParentId="14243" CreationDate="2019-09-05T15:25:47.803" Score="3" Body="&lt;p&gt;Do you have access to parallel corpora in source and target language that translates idioms correctly? Neural machine translation.  (NMT) should handle this. NMT uses deep learning to match sequences/pairs of words in one language to another and is now the state of the art method for translation AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think an AI knows grammar of a language. A translating AI knows patterns but not necessarily grammar in the sense that we learn in school as children. Here's a potential approach that should work give a large enough corpora with examples of idioms - &lt;a href=&quot;https://github.com/facebookresearch/MUSE&quot; rel=&quot;nofollow noreferrer&quot;&gt;github.com/facebookresearch/MUSE&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10287" LastEditorUserId="10287" LastEditDate="2019-09-05T15:32:12.407" LastActivityDate="2019-09-05T15:32:12.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14298" PostTypeId="2" ParentId="14243" CreationDate="2019-09-05T15:51:29.383" Score="8" Body="&lt;p&gt;&lt;strong&gt;Short answer&lt;/strong&gt;: Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the presence of good datasets this can be accomplished with a &lt;a href=&quot;https://www.aclweb.org/anthology/E17-3027&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipeline&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality an idiom is a series of words which is supposed to have a semantic meaning that is not denoted by the literal reading (&lt;a href=&quot;https://pdfs.semanticscholar.org/6a4e/8ebd0979ae38d693528fe4da30f67db82768.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;). This means that any system that is used must be capable of considering multiple words at a time. Additionally, some idioms are &lt;strong&gt;context dependent&lt;/strong&gt;. Example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The fisherman broke the ice with his tool.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are we to believe that this is a very suave fisherman?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, it is possible to teach an AI to use idiomatic phrases to &lt;strong&gt;keep up with the culture of humans&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Observe that humans do not come linguistically &quot;pre-loaded&quot; with idioms. So we can safely assume that idiom usage is a learning task and that the only way for them to keep up is for them to keep learning. So if we solve the idiom learning task we just need to keep our agent &lt;a href=&quot;https://en.wikipedia.org/wiki/Online_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;online&lt;/a&gt; or periodically retrain it on nascent corpora.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One difficulty is that, in the absence of a label, a metaphor could be easily mistaken for an idiom and vice versa. So &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-37247-6_35&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic outlier&lt;/a&gt; (sorry it's not free) approaches  may suffer from precision issues. Example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;She's a thorny wildflower (metaphor - could easily be an idiom)&lt;/li&gt;&#xA;&lt;li&gt;She's a diamond in the rough (idiom - could easily be a metaphor)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Though, idioms will most likely be repeated if a dataset is large whereas a &quot;custom metaphor&quot; is less likely to repeat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, some idioms (eg bite the bullet or break a leg) do not have readily available &quot;interpretable semantics&quot; that allow us to extract their intended meaning. For example, if one did not know the idiom &quot;cut me some slack&quot; one could think:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Slack implies loosening or to make less tight/taut. I was being very uptight. They probably want me to loosen up and not be so critical.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the human understanding of it might happen in a flash and not follow such a delineated path. The idea is that some &lt;a href=&quot;https://www.quora.com/What-is-natural-language-processing-pipeline&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLP pipeline&lt;/a&gt; might be constructible that satisfactorily handles idioms in some specific use cases (example of a &lt;a href=&quot;https://www.aclweb.org/anthology/E17-3027&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipeline&lt;/a&gt;). For example, one module might attempt to process outliers like &quot;diamond in the rough&quot; which have said interpretable semantics. Though, something like &quot;bite the bullet&quot; may have to be labelled with the correct semantics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've only scratched the surface of this. Natural language understanding is already a hard problem - and idioms are thus a tough task in a tough task. I hope that this motivates the reading of some more thorough articles. I have gathered some articles that can be used as a springboard into the literature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a &lt;a href=&quot;https://nlp.stanford.edu/~muzny/docs/mz-emnlp2013.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt; that uses a dictionary type approach to train the model to recognize idioms. Excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For identification, we assume data of the form &lt;span class=&quot;math-container&quot;&gt;${(⟨p_i,d_i⟩,y_i) : i = 1...n}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p_i$&lt;/span&gt; is the phrase associated with definition &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i ∈ \{literal, idiomatic\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://www.aclweb.org/anthology/R15-1087&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt; provides pseudo-code for idiom extraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://pdfs.semanticscholar.org/6a4e/8ebd0979ae38d693528fe4da30f67db82768.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt; describes a dataset to help solve the idiom difficulties.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-09-06T15:48:19.237" LastActivityDate="2019-09-06T15:48:19.237" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14299" PostTypeId="1" CreationDate="2019-09-05T17:19:41.740" Score="3" ViewCount="96" Body="&lt;p&gt;After completing Coursera course from Andrew Ng, I wanted to implement again simple RNN for generating dinosaurs name based on a text file containing around 800 dinosaurs name. &#xA;This is done with Numpy in coursera, here is a link to a  Jupyter notebook (not my repo) to get strategy and full objective: &#xA;&lt;a href=&quot;https://github.com/Kulbear/deep-learning-coursera/blob/master/Sequence%20Models/Dinosaurus%20Island%20--%20Character%20level%20language%20model%20final%20-%20v3.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started similar implementation but in Pytorch, here is the model:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class RNN(nn.Module):&#xA;    def __init__(self,input_size):&#xA;        super(RNN, self).__init__()&#xA;        print(&quot;oo&quot;)&#xA;        self.hiddenWx1 = nn.Linear(input_size, 100) &#xA;        self.hiddenWx2 = nn.Linear(100, input_size)&#xA;        self.z1 = nn.Linear(input_size,100)&#xA;        self.z2 = nn.Linear(100,input_size)&#xA;        self.tanh = nn.Tanh()&#xA;        self.softmax = torch.nn.Softmax(dim=1)&#xA;&#xA;    def forward(self, input, hidden):&#xA;        layer = self.hiddenWx1(input)&#xA;        layer = self.hiddenWx2(layer)&#xA;        a_next = self.tanh(layer)&#xA;        z = self.z1(a_next)&#xA;        z = self.z2(z)&#xA;        y_next = self.softmax(z)&#xA;        return y_next,a_next&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the main algorithm of training:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for word in examples:  # for every dinosaurus name&#xA;                model.zero_grad()&#xA;                hidden= torch.zeros(1, len(ix_to_char)) #initialise hidden to null, ix_to_char is below&#xA;                word_vector = word_tensor(word) # convert each letter of  the current name in one-hot tensors&#xA;                output = torch.zeros(1, len(ix_to_char)) #first input is null&#xA;                loss = 0&#xA;                counter = 0&#xA;                true = torch.LongTensor(len(word)) #will contains the index of each letter.If word is &quot;badu&quot; =&amp;gt; [2,1,4,22,0]&#xA;&#xA;                measured = torch.zeros(len(word)) # will contains the vectors returned by the model for each letter (softmax output) &#xA;&#xA;&#xA;                for t in range(len(word_vector)): # for each letter of current word&#xA;                    true[counter] = char_to_ix[word[counter]] # char_to_ix return the index of letter in dictionary&#xA;&#xA;                    output, hidden = model(output, hidden)&#xA;&#xA;                    if (counter ==0):&#xA;                        measured = output&#xA;                    else: #measures is a tensor containing tensors of probability distribution&#xA;                        measured = torch.cat((measured,output),dim=0)&#xA;                    counter+=1&#xA;&#xA;                loss = nn.CrossEntropyLoss()(measured, true) #&#xA;                loss.backward()&#xA;                optimizer.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The letter dictionary (ix_to_char) is as follow: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;{0: '\n', 1: 'a', 2: 'b', 3: 'c', 4: 'd', 5: 'e', 6: 'f', 7: 'g', 8: 'h', 9: 'i', 10: 'j', 11: 'k', 12: 'l', 13: 'm', 14: 'n', 15: 'o', 16: 'p', 17: 'q', 18: 'r', 19: 's', 20: 't', 21: 'u', 22: 'v', 23: 'w', 24: 'x', 25: 'y', 26: 'z'}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every 2000 epochs, I sample some new words with this function using torch multimonial to select a letter based on the softmax probability returned by the model:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def sampling(model):&#xA;    idx = -1 &#xA;    counter = 0&#xA;    newline_character = char_to_ix['\n']&#xA;&#xA;    x = torch.zeros(1,len(ix_to_char))&#xA;    hidden = torch.zeros(1, len(ix_to_char))&#xA;    generated_word=&quot;&quot;&#xA;&#xA;&#xA;    while (idx != newline_character and counter != 35):&#xA;        x,hidden = model(x, hidden)&#xA;        #print(x)&#xA;        counter+=1&#xA;        idx = torch.multinomial(x,1)&#xA;        #print(idx.item())&#xA;        generated_word+=ix_to_char[idx.item()]&#xA;    if counter ==35:&#xA;        generated_word+='\n'&#xA;    print(generated_word)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the results of the first display:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;epoch:1, loss:3.256033420562744&#xA;aaasaaauasaaasasauaaaaapsaaaasaaaaa&#xA;&#xA;aaaaaaaaaaaaasaaaoaaaaaauaaaaaaaaaa&#xA;&#xA;taaaauasaasaaaaasaaasaauaaaaaaaausa&#xA;&#xA;uaasaaaaauaaaasasssaauaaaaasaaaaaaa&#xA;&#xA;auaaaaaaaassasaaauaaaaaaaaasasaaaas&#xA;&#xA;epoch:2, loss:3.199960231781006&#xA;aaasaaassussssusssussssssssssusssss&#xA;&#xA;aasaaassssssssssssasusssissssssssss&#xA;&#xA;sasaaassssuosasssssssssssssssssssss&#xA;&#xA;aasassasassusssssssssussssssssssuss&#xA;&#xA;oasaasassssssussssssssussssssssssss&#xA;&#xA;epoch:3, loss:3.263746500015259&#xA;aaaaaaasaaaasaaaaasaaaasaaaaaaaaaaa&#xA;&#xA;aaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaa&#xA;&#xA;aaaaaaaaaaaaaaaaaaaaaauaaaaaaaaaaas&#xA;&#xA;aaaaaaaasaaaasraaaaaaaaaaaaaaaaaaaa&#xA;&#xA;aaaaaaaaaaaaauusaaaaauaaaaaaaaaaaaa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It doesn't work but I have no idea how to fix the issue. &#xA;With no training at all, the sampling function seems to work as the returned words seem complete random:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hbtpsbykkxvlah&#xA;&#xA;ttiwlzxdxabzmbdvsapsnwwpaoiasotalft&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My post may be a bit long, but so far I have no idea what is the issue of my program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manny thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="28507" LastEditorUserId="28507" LastEditDate="2019-09-06T21:19:10.957" LastActivityDate="2020-02-03T23:01:45.233" Title="Issue at training simple RNN for word generation" Tags="&lt;recurrent-neural-networks&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14300" PostTypeId="2" ParentId="14296" CreationDate="2019-09-05T17:31:32.990" Score="2" Body="&lt;p&gt;The good news is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Your MDP appears valid, with well-defined states, actions. It has state transition and reward functions (which you have implemented as matrices). There is nothing else to add, it's a full MDP.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could use this MDP to evaluate a policy, using a variety of reinforcement learning (RL) methods suitable for finite discrete MDPS. For instance, Dynamic Programming could be used, or Monte Carlo or SARSA.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could use this MDP to find an optimal policy for the environment it represents, again using a variety of RL methods, such as Value Iteration, Monte Carlo Control, SARSA or Q-Learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The bad news is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;All policies in the MDP as defined are optimal, with expected returns (total reward summed until end of episode) of &lt;span class=&quot;math-container&quot;&gt;$v(S1) = 55, v(S2) = 33.75, v(S3) = 21.25$&lt;/span&gt; - solved using Dynamic Programming in case you are wondering.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The MDP is degenerate because action choice has no impact on either state transition or reward. It is effectively a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_reward_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Reward Process&lt;/a&gt; (MRP) because the agent policy has been made irrelevant.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Without discounting, the best result is not going from S1-S4 directly, as you appear to want, but repeatedly looping S1-S3-S2-S1-S3-S2... (this is currently hiiden by action choice being irrelevant). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are a few ways to fix this, but maybe the simplest is to make the rewards more straightforward (e.g. +0, +10, +20, +30 for S1-S1, S1-S2..., -10, 0, +10, +20 for S2-S1, S2-S2...) and add a discount factor, often labelled &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, when calculating values. A discount factor makes immediate rewards have higher value to the agent, so it will prefer to get a larger reward all at once and end the episode than loop around before finishing.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This whole &quot;bad news&quot; section should not worry you too much though. Instead it points to a different issue. The key point is here:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The model i have is a pretty uncertain one so i am guessing the agent must initially be given a lot of experience to make any sense of the environment. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It looks like you have assumed that you need to explicitly build a MDP model of your environment in order to progress with your problem. So you are providing an inaccurate model, and expect that RL works with that, improving it as part of searching for an optimal policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few different approaches you could take in order to learn a model. In this case as your number of states and actions are very low, then you could do it like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a 2D tensor (i.e. just a matrix) to count number of times each state, action pair is visited, initialised with all zeroes, and indexed using S, A&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a 3D tensor to count number of times each state transition is observed, again initialised with all zeroes, indexed using S, A, S'.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run a large number of iterations with the real environment, choosing actions at random, and adding +1 to each visited S, A pair in the first tensor, and +1 to each S, A, S' triple in the second tensor.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You now have an approximate transition function based on real experience, without needing an initial guess, or anything particularly clever, you are just taking averages in a table. Divide each count of S, A, S' by the total count of S, A to get conditional transition probability &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt;. It's not really an established, named RL method, but will do.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, if your construction of the MDP is just step 1 for running some RL policy optimisation approach, none of that is really necessary. Instead, you can use a model-free approach such as &lt;a href=&quot;https://www.learndatasci.com/tutorials/reinforcement-q-learning-scratch-python-openai-gym/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tabular Q learning&lt;/a&gt; to learn directly online from interactions with the environment. This is likely to be more efficient than learning the model first or alongside the policy optimisation. You don't need the explicit MDP model at all, and adding one can make things more complex - in your case for no real gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably do still need to define a reward function in your case as there is no inherent reward in the system. You want the agent to reach state S4 as quickly as possible, so you need to monitor the states observed and add a reward signal that is appropriate for this goal. As above, I suggest you modify your planned reward structure to be simple/linear and add discounting to capture the requirement to &quot;increase&quot; state as fast as possible (here I am assuming that being in S2 is still somehow better than being in S1 - if that's not the case, and reaching S4 is the only real goal, then you could simplify further). That's because if you make the rewards for state progression non-linear - as in your example - the agent may find loops that exploit the shape of the reward function and not work to progress states towards S4 as you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond this very simple looking environment, there are use cases for systems that learn transition models alongside optimal policies. Whether or not to use them will depend on other qualities of your environment, such as how cheap/fast it is to get real experience of the environment. Using a learned model can help by doing more optimisation with the same raw data, using it to simulate and plan in between taking real actions. However, if the real environment data is very easy to collect, then there may be no point to that.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-05T21:39:00.713" LastActivityDate="2019-09-05T21:39:00.713" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14301" PostTypeId="1" CreationDate="2019-09-05T22:35:52.023" Score="1" ViewCount="17" Body="&lt;p&gt;Is there any grounds for assuming an algorithms applied to a data-set that created a decently accurate model will perform as well on a different data-set with meta-features chosen and evaluated by meta-learning? What meta-features are even worth considering when evaluating similarity between data-sets with the goal of finding an optimal combination of algorithm application to this new data-set to create an accurate model?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastActivityDate="2019-09-05T22:35:52.023" Title="Applying ML algorithms to data-sets with similar meta-features?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14303" PostTypeId="1" CreationDate="2019-09-06T05:37:12.577" Score="1" ViewCount="46" Body="&lt;p&gt;I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the problem is the neural network aimed to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, what the initial architecture (amount of hidden layers, the amount of neurons) was for a MLP in a CNN, the steps taken to optimize the architecture (adding more layers and reducing nodes, changing activation functions) and the results each step produced (i.e. increased error or decreased error). What the problem is the CNN tried to solve (differentiation of human faces, object detection inteded for self driving cars etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course I used a CNN as an example but I am referring to data for any MLP architecture in plain MLPs or Deep Learning architectures such as RNNs, CNNs and mroe. I am focused on the MLP architecture mostly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is not how do you think one can accumulate this data?&lt;/p&gt;&#xA;" OwnerUserId="14621" LastEditorUserId="14621" LastEditDate="2019-09-06T07:51:25.500" LastActivityDate="2019-09-06T07:51:25.500" Title="Is there data available about successful neural network architectures?" Tags="&lt;neural-networks&gt;&lt;research&gt;&lt;architecture&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14306" PostTypeId="2" ParentId="14047" CreationDate="2019-09-06T09:43:08.030" Score="2" Body="&lt;p&gt;What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem should be addressed by implementing methods that reduce catastrophic forgetting for different tasks. At first glance it might seem that they don't apply because it's data that change and not a particular task but changing data result in a change of the task. Say your goal is to classify different breeds of dogs. Your online data-set morphs into excluding &quot;Great Danes&quot;. Your neural network after enough epochs would forget about &quot;Great Danes&quot;. The task is still serving its purpose by classifying different breeds but the task still changed. It changed from recognizing &quot;Great Danes&quot; as a dog breed to not recognizing &quot;Great Danes&quot; as a dog breed. The weights changed to exclude them but the methods I linked tries and keep weights intact even though it was not intended for the purpose of online learning. Just set the hyper parameters to include these techniques to low as I believe data won't have an instant change but would change over time, and you should be fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious technique being storing information as you train. This is called &lt;a href=&quot;https://www.semanticscholar.org/paper/A-Strategy-for-an-Uncompromising-Incremental-Venkatesan-Venkateswara/ac7ea1a3050fe81a565b22b4bafdd5baa465be07&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudo-rehearsal&lt;/a&gt;. With this at least you would be able to use stochastic gradient decent but you need memory and resources as the data set grows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there was an attempt to reduce impacts of weights on old tasks to keep some relevancy to them. &lt;a href=&quot;https://www.pnas.org/content/pnas/114/13/3521.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structural Regularization&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later these guys implemented &lt;a href=&quot;https://arxiv.org/pdf/1801.01423.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;HAT&lt;/a&gt; which seems to keep some weights static while others adapt to new tasks. &lt;/p&gt;&#xA;" OwnerUserId="14621" LastActivityDate="2019-09-06T09:43:08.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14309" PostTypeId="1" AcceptedAnswerId="14328" CreationDate="2019-09-06T19:17:53.750" Score="3" ViewCount="78" Body="&lt;p&gt;Imagine I have a tensorflow CNN model with good accuracy but maybe too many filters:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Is there a way to determine which filters have more impact in output?&lt;/strong&gt; I think it should be possible. At least, if a filter A has a 0, that only multiples the output of a filter B, then filter B is not related to filter A. In particular, I'm thinking in 2d data where 1 dimension is time-related and the other feature related (like one-hot char).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Is there a way to eliminate the less relevant filters from a trained model&lt;/strong&gt;, and leave the rest of the model intact?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Is it useful or there are better methods?&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28526" LastEditorUserId="2444" LastEditDate="2019-09-08T13:26:08.913" LastActivityDate="2019-09-08T13:26:08.913" Title="Is it useful to eliminate the less relevant filters from a trained CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14310" PostTypeId="1" AcceptedAnswerId="14313" CreationDate="2019-09-06T20:23:43.427" Score="3" ViewCount="95" Body="&lt;p&gt;Is there any computer vision technology that can detect any type of object? For example, there is a camera fixed, looking in one direction always looking at a similar background. If there is an object, no matter what the object is (person, bag, car, bike, cup, cat) the CV algorithm would notice if there is an object in the frame. &lt;strong&gt;It wouldn't know what type of object it is, just that there is an object in the frame.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something similar to motion detector but that would work on a flat conveyor belt. Even though the conveyor belt moves will look similar between frames. Would something like this be possible? Possibly something to do with extracting differences from the background, with the goal being to not have to train the network with data for every possible object that may pass by the camera.&lt;/p&gt;&#xA;" OwnerUserId="28528" LastEditorUserId="2444" LastEditDate="2019-09-06T23:24:13.447" LastActivityDate="2019-09-07T17:44:50.813" Title="Is there any computer vision technology that can detect any type of object?" Tags="&lt;algorithm&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14311" PostTypeId="2" ParentId="14299" CreationDate="2019-09-06T21:37:37.500" Score="1" Body="&lt;p&gt;Your forward function is not using the previous hidden state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;observe: you pass hidden but never use it.&#xA;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def forward(self, input, hidden):&#xA;    layer = self.hiddenWx1(input)&#xA;    layer = self.hiddenWx2(layer)&#xA;    a_next = self.tanh(layer)&#xA;    z = self.z1(a_next)&#xA;    z = self.z2(z)&#xA;    y_next = self.softmax(z)&#xA;    return y_next,a_next&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28343" LastActivityDate="2019-09-06T21:37:37.500" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14313" PostTypeId="2" ParentId="14310" CreationDate="2019-09-07T03:51:21.387" Score="3" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is possible. You need a correctly labeled dataset. Your dataset has two labels: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y\in \{\text{background},\text{object in frame}\}$&lt;/span&gt; or simply &lt;span class=&quot;math-container&quot;&gt;$y\in \{0,1\}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This labelling avoids needing to know &lt;em&gt;what&lt;/em&gt; object is in frame only that &lt;em&gt;there is&lt;/em&gt; an object in frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Examples With Similar Objectives&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here as a paper (&lt;a href=&quot;https://arxiv.org/pdf/1703.05830.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) that was seeking to classify animals in a frame. The first part of their pipeline needed to know whether there was an animal in frame or not. Excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Task I: Detecting Images That Contain Animals.&lt;/strong&gt; For this task, our models take an image as input and output two probabilities describing whether the image has an animal or not (i.e. binary classification)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is another paper (&lt;a href=&quot;http://%20Wildlifedetection,%20which%20is%20actually%20a%20binary%20classi%EF%AC%81er%20capable%20of%20clas-sifying%20input%20images%20into%20two%20classes:%20%E2%80%9Canimal%E2%80%9D%20or%20%E2%80%9Cno%20animal%E2%80%9Dbased%20on%20the%20prediction%20of%20animal%20presence%20in%20images;&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) regarding detecting animals in images. Excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Wildlife detection&lt;/em&gt;, which is actually a binary classiﬁer capable of clasifying input images into two classes: “animal” or “no animal”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The methods used in these papers could be adapted to fit the conveyor belt use case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a system (&lt;a href=&quot;http://www.ee.nchu.edu.tw/Pic/Writings/3258_06419121.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) that detects moving objects using background extraction. The method they use to check whether or not the image is a background image or not could be adapted to your use case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ideas for Implementation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the conveyor belt use case, a very simple model might work. Thus, it is recommended that you try a simple model first. For example, you could flatten your inputs to a vector and feed these into a shallow feed forward network. A simple logistic regression classifier might even work on flattened inputs for this use case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your desired performance is not reached with a simple model you can analyze their performance using methods like &lt;a href=&quot;https://www.ritchieng.com/machinelearning-learning-curve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;learning curves&lt;/a&gt; to decide how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If you find that your model has high bias:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can either use a larger/deeper model like a custom tuned &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNN&lt;/a&gt; or you can use a pre-trained network like &lt;a href=&quot;https://neurohive.io/en/popular-networks/vgg16/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG16&lt;/a&gt; and  use &lt;a href=&quot;https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt; to achieve your goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a post that describes using a CNN to classify cats vs dogs (&lt;a href=&quot;https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-to-classify-photos-of-dogs-and-cats/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;). Simply replace the cat/dog dataset with your object/no-object dataset. The post even describes using transfer learning with VGG16.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If you find that your model has high variance&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More data can help. Furthermore, in the conveyor belt example, hand labeling data would be easy and quick. Thus, it would not take very much time to get many more examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-09-07T17:44:50.813" LastActivityDate="2019-09-07T17:44:50.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14314" PostTypeId="2" ParentId="4737" CreationDate="2019-09-07T09:15:48.763" Score="1" Body="&lt;p&gt;It is an unreasonable assumption for many scenarios to know the full transition model of an environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the deterministic toy scenarios used for explaining RL, such as a grid world, it seems like a trivial requirement. However, it starts to get harder very quickly as problems and state features get more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in relatively simple environments that are possible to model in this way, the calculations start to become difficult enough that there is an argument to not bother. For instance, a common toy example is the game of &lt;a href=&quot;https://towardsdatascience.com/playing-blackjack-using-model-free-reinforcement-learning-in-google-colab-aa2041a2c13d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blackjack&lt;/a&gt; - in order to calculate the probability of different rewards after sticking you would need to resolve the remaining game tree of typically 0,1,2, but up to more, for the dealer, accounting for their starting card, and perhaps accounting for which cards have been drawn so far (if you are simulating this more realistically, without replacement).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that &lt;em&gt;could&lt;/em&gt; be resolved if we wanted. To make things harder, you need to consider agents that are set to solve problems where we don't have any meaningful probabilistic model of how the state will progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example would be any real world agent that used a vision system as part of its state. Consider a flying drone set with the task of locating and tracking migrating whales over some area of the ocean. It is rewarded for capturing footage of whales. Its state is the current image plus GPS and telemetrics data.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s,a)$&lt;/span&gt; in the whale-tracking example implies that you would know the probability distribution of the next video frame, somehow accounting for the chaotic movement of waves, the likely behaviour of any creatures in shot, and effects of wind turbulence on drone position etc. Even the most sophisticated physics simulations with ray-tracing rendering engines cannot rise to that challenge with much accuracy, and the computation required even for a rough render of expected image(s) based on a physics engine would be far beyond anything you could run in real time on the drone. However, it is still possible to build an agent which learns how to solve this kind of environment through gathering statistics on its task using images as state data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scaling that challenge back a bit, you can view many of the Atari game-playing challenges like this. The input state is a collection of still frames from the game. Knowing &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s,a)$&lt;/span&gt; implies knowing the probability distribution of next set of frame images, given the current inputs. This is a near impossible challenge, even if you know the code for the game, providing an analytical result for &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid s,a)$&lt;/span&gt; is unrealistic - the best way to do it would be to have a memory clone of the game that you could use to look ahead then reset. Whilst this is theoretically possible in a computer game, it is not something you can consider when interacting with the real world.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-07T09:24:13.890" LastActivityDate="2019-09-07T09:24:13.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14317" PostTypeId="5" CreationDate="2019-09-07T12:21:35.573" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-07T12:21:35.573" LastActivityDate="2019-09-07T12:21:35.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14318" PostTypeId="4" CreationDate="2019-09-07T12:21:35.573" Score="0" Body="For questions related to deepfakes, which refers to machine learning techniques used to combine and superimpose existing images and videos onto source images or videos. Deepfakes have been used to create fake celebrity pornographic videos, revenge porn,  fake news, and malicious hoaxes." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-07T12:21:35.573" LastActivityDate="2019-09-07T12:21:35.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14319" PostTypeId="2" ParentId="5939" CreationDate="2019-09-07T13:12:08.300" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Digital_forensics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digital Media Forensics&lt;/a&gt; (DMF) field aims to develop technologies for the automated assessment of the integrity of an image or video, so &lt;a href=&quot;https://farid.berkeley.edu/downloads/tutorials/digitalimageforensics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMF&lt;/a&gt; is the field you are looking for. There are several approaches in DMF: for example, those based on machine learning (ML) techniques, in particular, convolutional neural networks (CNNs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the paper &lt;a href=&quot;https://engineering.purdue.edu/~dgueraco/content/deepfake.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deepfake Video Detection Using Recurrent Neural Networks&lt;/a&gt; (2018), David Güera and Edward J. Delp propose a two-stage analysis composed of a CNN to extract features at the frame level followed by a temporally-aware RNN to capture temporal inconsistencies between frames introduced by the deepfake tool. More specifically, they use a convolutional LSTM architecture (CNN combined with an LSTM), which is trained end-to-end, so that the CNN learns the features in the videos, which are passed to the RNN, which attempts to predict the likelihood of those features belonging to a fake video or not. Section 3 explains the creation of deepfake videos, which leads to &lt;em&gt;inconsistencies between video frames&lt;/em&gt; (which are exploited in the proposed method) because of the use of images with different viewing and illumination conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other similar works have been proposed. See this curated list &lt;a href=&quot;https://github.com/aerophile/awesome-deepfakes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aerophile/awesome-deepfakes&lt;/a&gt; for more related papers.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-07T13:27:14.840" LastActivityDate="2019-09-07T13:27:14.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14320" PostTypeId="1" CreationDate="2019-09-07T13:12:27.857" Score="1" ViewCount="53" Body="&lt;p&gt;I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer  can model any mathematic function with accuracy dependant on number of hidden layer neurons. Try as I might I cannot find any explanation as to why that occurs - could someone maybe explain the question why that is?&lt;/p&gt;&#xA;" OwnerUserId="28537" LastActivityDate="2021-04-16T16:49:07.513" Title="How does a single neuron in hidden layer affect training accuracy" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;hidden-layers&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14321" PostTypeId="1" AcceptedAnswerId="14323" CreationDate="2019-09-07T14:18:13.677" Score="4" ViewCount="1101" Body="&lt;p&gt;In the &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original DQN paper&lt;/a&gt;, page 1, the loss function of the DQN is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim  U(D)} [(r+\gamma \max_{a'} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}))^2]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whose gradient is presented (on page 7)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta_i} L_i(\theta_i) = \mathbb{E}_{s,a,r,s'} [(r+\gamma \max_{a'}Q(s',a';\theta_i^-) - Q(s,a;\theta_i))\nabla_{\theta_i}Q(s,a;\theta_i)] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why is there no minus (-) sign if &lt;span class=&quot;math-container&quot;&gt;$-Q(s,a;\theta_i)$&lt;/span&gt; is parameterized by &lt;span class=&quot;math-container&quot;&gt;$\theta_i$&lt;/span&gt; and why is the 2 from power gone?&lt;/p&gt;&#xA;" OwnerUserId="28538" LastEditorUserId="2444" LastEditDate="2021-01-18T00:59:45.380" LastActivityDate="2021-01-18T00:59:45.380" Title="How is the gradient of the loss function in DQN derived?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;gradient-descent&gt;&lt;gradient&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14323" PostTypeId="2" ParentId="14321" CreationDate="2019-09-07T17:24:21.510" Score="1" Body="&lt;p&gt;In general, if you have a composite function &lt;span class=&quot;math-container&quot;&gt;$h(x) = g(f(x))$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\frac{dh}{dx} = \frac{d g}{df} \frac{d f}{dx}$&lt;/span&gt;. In your case, the function to differentiate is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim  U(D)} \left[ \left(r+\gamma \max_{a '} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}) \right)^2 \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, we want to calculate &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_i} L_{i}(\theta_{i})$&lt;/span&gt;, which is equal to&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta_i}  \mathbb{E}_{(s,a,r,s') \sim  U(D)} \left[ \left(r+\gamma \max_{a '} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}) \right)^2 \right] \label{2} \tag{2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For clarity, let's ignore the iteration number &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in \ref{2}, which can thus more simply be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}  \mathbb{E}_{(s,a,r,s') \sim  U(D)} \left[ \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta ) \right)^2 \right] \label{3} \tag{3} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://stats.stackexchange.com/q/72613/82135&quot;&gt;subscript of the expected value operator&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathbf{e}=(s,a,r,s') \sim  U(D)$&lt;/span&gt; means that the expected value is being taken with respect to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Multivariate_random_variable&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;multivariate&lt;/em&gt; random variable&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E}$&lt;/span&gt; (for &lt;em&gt;experience&lt;/em&gt;), whose values (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Realization_(probability)&quot; rel=&quot;nofollow noreferrer&quot;&gt;realizations&lt;/a&gt;) are &lt;span class=&quot;math-container&quot;&gt;$\mathbf{e}=(s,a,r,s')$&lt;/span&gt;, and that follows the distribution &lt;span class=&quot;math-container&quot;&gt;$U(D)$&lt;/span&gt; (a uniform distribution), that is, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{e}=(s,a,r,s')$&lt;/span&gt; are uniformly drawn from the experience replay buffer, &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E} \sim U(D)$&lt;/span&gt;). However, let's ignore or omit this subscript for now (because there are no other random variables in \ref{3}, given that &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; should not be random variables, thus it should not be ambiguous with respect to which random variable the expectation is being calculated), so \ref{3} can be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}  \mathbb{E} \left[ \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta ) \right)^2 \right] \label{4} \tag{4} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, recall that, in the case of a discrete random variable, the expected value is a weighted sum. In the case of a continuous random variable, it is an integral. So, if &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E}$&lt;/span&gt; is a continuous random variable, then the &lt;em&gt;expectation&lt;/em&gt; \ref{4} can be expanded to&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_{\mathbb{D}} {\left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta )\right)}^2 f(\mathbf{e}) d\mathbf{e}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the density function associated with &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt; is the domain of the random variable &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The derivative of an integral can be calculated with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Leibniz_integral_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leibniz integral rule&lt;/a&gt;. In the case the bounds of the integration are constants (&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;), then the Leibniz integral rule reduces to&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle {\frac {d}{dx}}\left(\int _{a}^{b}f(x,t)\,dt\right)=\int _{a}^{b}{\frac {\partial }{\partial x}}f(x,t)\,dt.}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Observe that the derivative is taken with respect to the variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, while the integration is taken with respect to variable &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In our case, the domain of integration, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{D}$&lt;/span&gt;, is constant because it only represents all experience in the dataset, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;. Therefore, the gradient in \ref{4} can be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \int_{\mathbb{D}} \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta )\right)^2 f(\mathbf{e}) d\mathbf{e} &#xA;&amp;amp;= \\&#xA;\int_{\mathbb{D}} \nabla_{\theta} \left( \left( r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta )\right)^2  f(\mathbf{e}) \right) d\mathbf{e}&#xA;&amp;amp;=\\&#xA;\mathbb{E} \left[ \nabla_{\theta} { \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta) \right)}^2 \right] \label{5}\tag{5}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Recall that the derivative of &lt;span class=&quot;math-container&quot;&gt;$f(x)=x^2$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$f'(x)=2x$&lt;/span&gt; and that the derivative of a constant is zero. Note now that the only term in \ref{5} that contains &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$Q(s,a; \theta)$&lt;/span&gt;, so all other terms are constant with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. Hence, \ref{5} can be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbb{E} \left[ \nabla_{\theta} {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))}^2 \right] &#xA;&amp;amp;=\\&#xA;\mathbb{E} \left[ 2 {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))} \nabla_{\theta} \left( {r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta)} \right) \right] &#xA;&amp;amp;=\\&#xA;\mathbb{E} \left[ 2 {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))}  \left( {\nabla_{\theta} r + \nabla_{\theta} \gamma \max_{a'} Q(s',a',\theta^{-}) - \nabla_{\theta} Q(s,a; \theta)} \right) \right] &#xA;&amp;amp;=\\&#xA;\mathbb{E} \left[ - 2 {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))} {\nabla_{\theta} Q(s,a; \theta)} \right] &#xA;&amp;amp;=\\&#xA;-2 \mathbb{E} \left[ {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))}{\nabla_{\theta} Q(s,a; \theta)} \right] &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$-2$&lt;/span&gt; disappears because it can be absorbed by the learning rate.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T00:59:18.373" LastActivityDate="2021-01-18T00:59:18.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14324" PostTypeId="1" CreationDate="2019-09-07T18:23:13.163" Score="3" ViewCount="73" Body="&lt;p&gt;In lots of games there are multiple phases or decision points that are not similar yet seem to have a dependency on one another when taking the perspective of the overall strategy of the player. A couple examples I thought up:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In a simple draw poker, you can have a strategy for discarding cards and a strategy for betting. They may not be mutually exclusive if you know your opponents betting will change with the number of cards you draw.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In Cribbage there are two phases, Discard to crib and the Play. The Play phase is definitely dependent on which cards are discarded in the discard phase. So it seems knowledge of Play strategy would be needed to make the Discard decision.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The intent is to learn how to set up an unsupervised learning algorithm to play a game with multiple types of decision making. Doesn't matter the game. I'm at a loss at the highest level in what ML models to learn to use for this scenario. I don't think a single NN would work because of the different decision types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how are these dependencies handled in ML? What are some known algorithms/models that can handle this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm at a loss on what to even search for so feel free to dump some terminology and keywords on me. =)&lt;/p&gt;&#xA;" OwnerUserId="26540" LastEditorUserId="26540" LastEditDate="2019-09-07T20:32:04.437" LastActivityDate="2019-09-08T10:27:46.423" Title="How to handle multiple types of decisions?" Tags="&lt;ai-design&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14325" PostTypeId="1" CreationDate="2019-09-07T22:42:13.813" Score="1" ViewCount="57" Body="&lt;p&gt;I experimented with a CNN operating on texts encoded as sequences of character vectors, where characters are encoded as one-hot vectors in one embedding and as random unit length pairwise orthogonal vectors (orthogonal matrix) in another. While geometrically these encode the same vector space, the one-hot embedding outperformed the random orthogonal one consistently. I suppose this has to do with the clarity of the signal: A zero vector with a single 1-valued cell is an easier to learn signal than just &lt;em&gt;some&lt;/em&gt; vector with lots of different values in each cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wondered if you know of any papers on this kind of effect. I did not find any but would like to back up this finding and check if my reasoning for why this is the case makes sense/ find a better or more in-depth explanation.&lt;/p&gt;&#xA;" OwnerUserId="20150" LastEditorUserId="20150" LastEditDate="2019-09-08T17:31:45.367" LastActivityDate="2019-09-08T17:31:45.367" Title="Reference request: one-hot encoding outperforming random orthogonal encoding" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;word-embedding&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14326" PostTypeId="1" AcceptedAnswerId="14327" CreationDate="2019-09-08T03:20:14.860" Score="3" ViewCount="53" Body="&lt;p&gt;I've found multiple depictions of how an LSTM cell operates. See 2 below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YGgZO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YGgZO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3e630.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3e630.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these images suggest the hidden state is utilised differently. On the top diagram, it is shown that the hidden state is added along with the previous output and current input to both the forget gate and the input gate. The bottom image suggests the input and forget gates are calculated only using the previous output and current input. Which is it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, when the previous output is fed in for the current layer, is this before or after it has been reshaped to the final output size and been put through a softmax?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-09-08T05:05:00.863" Title="Structure discrepancy of an LSTM?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14327" PostTypeId="2" ParentId="14326" CreationDate="2019-09-08T05:05:00.863" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There are different variants of LSTM, in most ML packages now of days you'll probably see what's shown in the bottom picture. For more details, intuition and motivations please see &lt;a href=&quot;https://arxiv.org/pdf/1503.04069.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is not reshaped and there is no softmax layer. That is all done &lt;strong&gt;outside&lt;/strong&gt; the LSTM.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-09-08T05:05:00.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14328" PostTypeId="2" ParentId="14309" CreationDate="2019-09-08T05:46:23.867" Score="4" Body="&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: All the observations and results are from the paper &lt;a href=&quot;https://arxiv.org/pdf/1803.03635.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your questions one by one:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Yes there are ways to determine which filters have more impact on the output. Its a very naive way but works very good in practice. Filters with small weights impact output less (according to empirical evidence), which basically means neurons whose weights lie in the switching region i.e ~&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; in ReLu and ~&lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (say) have less impact on final output.&lt;/li&gt;&#xA;&lt;li&gt;Yes, just eliminating these lower weight filters eliminate the unnecessary noise and indecisiveness introduced by these filters and suprisingly makes the model perform better (observed empirically).&lt;/li&gt;&#xA;&lt;li&gt;The concept a relatively old paradigm but has been a new twist by the simplicity of the method of elimination of unnecessary weights in the aforementioned paper and thus winning it the best paper award in ICLR 2019. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: Eliminating unnecessary weights makes the model perform better than the original model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also &lt;a href=&quot;https://github.com/google-research/lottery-ticket-hypothesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is the TensorFlow code.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2019-09-08T05:46:23.867" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14329" PostTypeId="1" CreationDate="2019-09-08T05:59:26.920" Score="2" ViewCount="26" Body="&lt;p&gt;I am currently trying to improve a CNN architecture that was proposed for generating depth images. &#xA;&lt;a href=&quot;https://arxiv.org/abs/1902.05356&quot; rel=&quot;nofollow noreferrer&quot;&gt;The architecture was originally proposed for autonomous driving&lt;/a&gt; and it looks like following :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uSub5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uSub5.png&quot; alt=&quot;cnn architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind this architecture is to improve the accuracy of depth images by adding confidence maps &#xA;to the outputs of different sensors to govern their influence on the result for each pixel. Input is an &#xA;RGB image and its corresponding LIDAR data, output is a depth image with the same dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, RGB features are best at discriminating global information such as sky, land, body of water, etc. &#xA;while LIDAR features are better at capturing fine-grained, local information. &lt;strong&gt;So to decide which features will &#xA;have more influence at the final regression result of which pixel, scientists proposed a confidence-guided &#xA;architecture where confidence weights for each map are learned during training.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging by their test results and how successful their paper was, their idea worked out pretty well for their &#xA;problem domain. That's why I would like to employ the idea of confidence weights in my own domain, where I have &#xA;multiple sources of features too. I have implemented the same architecture and got promising results: &lt;em&gt;Accuracy &#xA;have been improved, but not enough to compete with SOTA.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I believe that the architecture above can be improved for my needs. The diversity of the scene structure &#xA;in my domain adds some amount of complexity to the image generation problem: &lt;strong&gt;Opposed to depth image generation for &#xA;autonomous driving where scene structure is somewhat restricted (there is sky, there is road, there are sideways etc), &#xA;the images that I need to analyze are sometimes taken from handheld cameras, some other images are areal views.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a typical example of their scenario and some examples related to my domain, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WHd07.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WHd07.jpg&quot; alt=&quot;A typical driving scene&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9WA6m.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9WA6m.jpg&quot; alt=&quot;My images&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This means my CNN needs to learn confidence weights for regions in fundamentally different scene setups. Top of &#xA;the image can be sky, but can also be populated by people. Bottom of the image can be sea, or can be road.&lt;/strong&gt; This &#xA;brings me to the understanding that &lt;em&gt;there is a non-linear relation between the position of the segments and confidence &#xA;weights in my case&lt;/em&gt;; and I need to modify the CNN architecture by introducing some additional non-linearity to learn the &#xA;confidences for different CNN columns in each situation correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TLDR; I want to improve the CNN architecture above by introducing additional non-linearity, but I do not know how to do it&lt;/strong&gt;. &#xA;&lt;em&gt;I have tried adding another layer to confidence weights (Extended the architecture by duplicating the same weights and activating &#xA;with ReLU), but it has decreased the accuracy of the resulting model. Using the confidence weights as-is increases the accuracy, but&#xA;not as much as I need.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="15526" LastEditorUserId="15526" LastEditDate="2019-09-09T03:48:36.653" LastActivityDate="2019-09-09T03:48:36.653" Title="Confidence Maps and Non-Linearity" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;regression&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14331" PostTypeId="2" ParentId="14324" CreationDate="2019-09-08T10:21:40.557" Score="1" Body="&lt;p&gt;Your intuition is correct, neural networks are a no go (except see bottom). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like you'd want to look into the ML sub-field called Reinforcement Learning.&#xA;In a nutshell, RL offers a set of methods to learn what is the best action to take in a given situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More formally, in RL settings the algorithm learns from experience by observing a reward (&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;) associated to an action (&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL problems can be conceptualized as Markov Decision Processes (MDPs). From &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2018jan1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;MDPs are a classical formalization of sequential decision making, where actions influence not just immediate rewards, but also subsequent situations, or states, and through those future rewards.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;MDP can be faced with several methods, mainly Monte Carlo Methods and Temporal-Difference Learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, both use the &lt;a href=&quot;https://joshgreaves.com/reinforcement-learning/understanding-rl-the-bellman-equations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bellman equation&lt;/a&gt; in different ways. A key component of the eq. is the discounting term ( &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; ). Values for rewards associated with future states are discounted (multiplied) by &lt;span class=&quot;math-container&quot;&gt;$\gamma&amp;lt;1$&lt;/span&gt; in order to modulate the importance of future rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you're concerned with strategy, another way to influence the agent's learning wrt. the environmental rewards is the adoption of an epsilon value &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;lt;1$&lt;/span&gt;. When defining an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy, the agent will choose the action with highest value &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; only with a probability equal to &lt;span class=&quot;math-container&quot;&gt;$1-\epsilon$&lt;/span&gt;. This enables the agent to balance &lt;em&gt;exploitation&lt;/em&gt; of previous experience with the &lt;em&gt;exploration&lt;/em&gt; of the environment. Very useful in cases where an immediate low reward compromises the achievement of a much higher reward later during the episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's many other ways to influence the behaviour of your learner of choice to respond differently to its environment. For a complete view, I recommend Sutton and Burton who offer their book in pdf format for free at the given link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: almost forgot. Neural networks can be useful when combined with more classical RL methods, say Q-learning (a type of Temporal-difference learning), taking on the name of deep Q-learning.&#xA;Alpha go also uses a combination of Monte Carlo methods combined with 2 NNs for Value evaluation and policy evaluation &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo#Algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;.&#xA;I wouldn't necessarily venture in this domain without first having a clear overview of classical RL methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="25893" LastEditorUserId="25893" LastEditDate="2019-09-08T10:27:46.423" LastActivityDate="2019-09-08T10:27:46.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14332" PostTypeId="1" CreationDate="2019-09-08T10:51:15.487" Score="3" ViewCount="694" Body="&lt;p&gt;When trying to implement my own PPO (Proximal Policy Optimizer), I came across two different implementations :&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Exploration with std&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Collect trajectories on &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; timesteps, by using a policy-centered distribution with progressively trained std variable for exploration&lt;/li&gt;&#xA;&lt;li&gt;Train policy function on &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; steps&lt;/li&gt;&#xA;&lt;li&gt;Train value function on &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; steps&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For example, the &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ppo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI's implementation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Exploration with entropy&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Collect trajectories on &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; timesteps, by using policy function directly&lt;/li&gt;&#xA;&lt;li&gt;Train policy and value function at the same time on &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; steps, with a common loss for the two models, with &lt;strong&gt;additional entropy&lt;/strong&gt; bonus for exploration purpose.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For example, the PPO algorithm as described in the &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;official paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros/cons of these two algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this specific to PPO, or is this a classic question concerning policy gradients algorithms, in general?&lt;/p&gt;&#xA;" OwnerUserId="23818" LastEditorUserId="2444" LastEditDate="2020-04-30T17:57:57.863" LastActivityDate="2020-09-27T19:06:10.563" Title="What are the pros and cons of using standard deviation or entropy for exploration in PPO?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14333" PostTypeId="1" AcceptedAnswerId="14334" CreationDate="2019-09-08T18:00:23.347" Score="5" ViewCount="152" Body="&lt;p&gt;I'm looking for more or less successful artificial intelligence usage examples to build an ontology or rationale why it can't be done. I found a lot of articles on how to use ontologies for AI, but not succeded vice versa.&lt;/p&gt;&#xA;" OwnerUserId="28546" LastEditorUserId="2444" LastEditDate="2019-09-12T12:04:06.630" LastActivityDate="2019-09-12T12:04:06.630" Title="Examples of ontologies made with AI" Tags="&lt;applications&gt;&lt;ontology&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14334" PostTypeId="2" ParentId="14333" CreationDate="2019-09-08T22:43:33.517" Score="7" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Ontology_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ontology learning&lt;/a&gt; is a relatively new field that aims to automatically (or semi-automatically) learn or create ontologies (using machine learning, text mining, knowledge representation and reasoning, information retrieval and natural language processing techniques) from some text or corpus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ontology learning can be divided into different phases or tasks&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;the acquisition of terms that refer to specific concepts (&lt;a href=&quot;https://en.wikipedia.org/wiki/Named-entity_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;named-entity&#xA;recognition&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the recognition of synonyms among these terms&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the identification of taxonomic relations (such as the &quot;is-a&quot; relation)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the establishment of non-hierarchical relations&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the derivation of new knowledge, i.e. knowledge that is not explicitly&#xA;encoded by the ontology.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://pdfs.semanticscholar.org/9613/3e6f5d80c89d3ea31673dc9e467f63fd7366.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ontology Learning from Text: An Overview&lt;/a&gt; (2003) and &lt;a href=&quot;https://academic.oup.com/database/article/doi/10.1093/database/bay101/5116160&quot; rel=&quot;nofollow noreferrer&quot;&gt;A survey of ontology learning techniques and applications&lt;/a&gt; (2018) for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;http://ceur-ws.org/Vol-1795/paper12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ontology Learning with Deep Learning: a case study on Patient Safety using PubMed&lt;/a&gt; (2016), the authors investigate how &lt;a href=&quot;https://arxiv.org/pdf/1301.3781.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;continuous bag-of-words (CBOW) and skip-gram&lt;/a&gt; (two &lt;a href=&quot;https://en.wikipedia.org/wiki/Language_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;language models&lt;/a&gt; based on artificial neural networks) can be used to aid ontology development for patient safety, using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubMed&lt;/a&gt; citations as a corpus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Latent Dirichlet allocation (LDA)&lt;/em&gt; has also been used for ontology learning, for example, in the paper &lt;a href=&quot;https://ksiresearchorg.ipage.com/seke/dms14paper/paper24.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Terminological ontology learning and population using latent Dirichlet allocation.&lt;/a&gt; (2014).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-12T12:03:37.307" LastActivityDate="2019-09-12T12:03:37.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14335" PostTypeId="1" AcceptedAnswerId="14336" CreationDate="2019-09-09T01:21:45.100" Score="2" ViewCount="64" Body="&lt;p&gt;I have been running my 2013 server box since 2 weeks ago for training an AI model.&#xA;I set up 30 epochs to run but since than it only ran 1 epoch as my PC config is super slow. But it generates 1 .h5 file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is will this .h5 file that I trained with Xception model work for Resnet 50?&lt;/p&gt;&#xA;" OwnerUserId="27988" LastActivityDate="2019-09-09T06:52:22.893" Title="Will a .h5 file trained with Xception model work with Resnet50?" Tags="&lt;training&gt;&lt;models&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14336" PostTypeId="2" ParentId="14335" CreationDate="2019-09-09T06:52:22.893" Score="1" Body="&lt;p&gt;If I understand your question correctly, you are asking whether you could load the saved weights of a trained model with the Xception architecture on a a Resnet50 architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Short answer: No&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long answer&lt;/strong&gt;: Xception and Resnet50 have very different architectures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a paper comparing multilple CNNs including Xception and Resnet50:&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/330478807_Deep_Feature-Based_Classifiers_for_Fruit_Fly_Identification_Diptera_Tephritidae/figures?lo=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/330478807_Deep_Feature-Based_Classifiers_for_Fruit_Fly_Identification_Diptera_Tephritidae/figures?lo=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the architectures are quiet different between Xception and ResNet50. Thus when you train a model, you are changing the weights and bias of the different layers of the model. If you switch models, and they are similar like let's say VGG16 and VGG19, you could import part of the weights for the layers which are similar between the two models.&#xA;As far as I know, I don't know of a function which can do this operation in tensorflow or keras.&#xA;But in your case, the architectures are extremely different between Xception and ResNet and there seem not to have any layers in common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, one train a model, save the weights of the training, import back the weights, and use the same model for the next round of training/testing/predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="28010" LastActivityDate="2019-09-09T06:52:22.893" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14337" PostTypeId="1" CreationDate="2019-09-09T07:43:46.320" Score="1" ViewCount="729" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FFjwb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FFjwb.png&quot; alt=&quot;Sample blurred image&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xBIQg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xBIQg.png&quot; alt=&quot;Ground truth&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to deblur text images using deep learning. Which approaches are best suited for the task? Any example networks? Is unsupervised network the best approach? GAN or cycle GAN for these purposes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have currently prepared 1000 images for training (shapr+blur) is it sufficient? For each of these approaches, how many training images do I need? &#xA;I have attached sample blurred image and Ground truth&lt;/p&gt;&#xA;" OwnerUserId="28567" LastEditorUserId="28567" LastEditDate="2019-09-09T21:35:54.980" LastActivityDate="2019-09-09T21:35:54.980" Title="Which approaches are best suited for text deblurring?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;unsupervised-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14338" PostTypeId="1" AcceptedAnswerId="14340" CreationDate="2019-09-09T10:01:12.823" Score="0" ViewCount="81" Body="&lt;p&gt;Currently I'm working on a project for scanning credit card and text extraction from cards. So first of all I decided to preprocess my images with some filters like thresholding, dilation and some other stuff. But it was not successfully for OCR of every credit cards. So I learned a lot and I found a solution like &lt;a href=&quot;https://matthewearl.github.io/2016/05/06/cnn-anpr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; for number plate recognition that is very similar to my project.&#xA;In the first step I want to generate a random dataset like my cards to locate card number region, and for every card that I've generated I cropped two images that one of them has numbers and another has not. I generated 2000 images for every cards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I have some images like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H4Bm3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H4Bm3.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; (does not have numbers)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mcqvc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mcqvc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; (has numbers)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And after generating my dataset I used this model with tensorflow to train my network.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    model = models.Sequential()&#xA;    model.add(layers.Conv2D(8, (5, 5), padding='same', activation='relu', input_shape=(30, 300, 3)))&#xA;    model.add(layers.MaxPooling2D((2, 2)))&#xA;&#xA;    model.add(layers.Conv2D(16, (5, 5), padding='same', activation='relu'))&#xA;    model.add(layers.MaxPooling2D((2, 2)))&#xA;&#xA;    model.add(layers.Conv2D(32, (5, 5), padding='same', activation='relu'))&#xA;    model.add(layers.MaxPooling2D((2, 2)))&#xA;&#xA;    model.add(layers.Flatten())&#xA;    model.add(layers.Dense(512, activation='relu'))&#xA;    model.add(layers.Dense(2, activation='softmax'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my plot for 5 epochs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zbPS4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zbPS4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/RqFVy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RqFVy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I almost get 99.5% of accuracy and It seems to be wrong, I think I have kind of overfitting in my data. Does it work correctly or my model is overfitted ? And how can I generate dataset for this purpose ?&lt;/p&gt;&#xA;" OwnerUserId="28569" LastActivityDate="2019-09-09T12:35:17.263" Title="Generate credit cards dataset for locating number region" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;datasets&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14339" PostTypeId="2" ParentId="14262" CreationDate="2019-09-09T10:38:12.637" Score="4" Body="&lt;p&gt;It actually depends on a couple of things here -&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred.&lt;/li&gt;&#xA;&lt;li&gt;As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes (95%), then this accuracy you see makes sense but note that your model won't do well in production.&lt;/li&gt;&#xA;&lt;li&gt;Do not try to evaluate your model on the basis of your training accuracy. Test it on data your model has never seen before and then evaluate the classification accuracy making sure that your training/testing data is not heavy with one of the classes. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25704" LastActivityDate="2019-09-09T10:38:12.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14340" PostTypeId="2" ParentId="14338" CreationDate="2019-09-09T12:18:20.373" Score="2" Body="&lt;p&gt;I am assuming the question you are asking is how to prevent over-fitting on the maximum accuracy. Your graph does show that your model over-fits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a couple of different methods to prevent over-fitting from happening. You can specify training to stop after a certain amount of epochs. In your case it seems to be 2 or 3 epochs. Take care as a new initialization might require more or less epochs to reach optimal accuracy thus it is required run your model a couple of times to determine the correct amount of epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also specify the accuracy you want your model to reach before you stop training, this can be dangerous as a local-minima could be found below your expected accuracy thus resulting in your training to run indefinitely. Also your model could have become more accurate then your expected accuracy but stopped prematurely due to the model reaching your stopping-condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can combine the two. Terminate training once it reach your expected accuracy otherwise terminate it if it reaches a certain amount of epochs. This way you prevent your training from happening indefinitely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Tensorflow you would want to build your own custom training seen &lt;a href=&quot;https://www.tensorflow.org/tutorials/eager/custom_training_walkthrough&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method you are interested is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;epoch_accuracy = tfe.metrics.Accuracy()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In their example they get the accuracy after each epoch, you would want to create your own batches and apply it there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to dive deeper, you can implement methods for the model to detect when it starts over-fitting such as looking at the standard deviation of the accuracy across a certain measure (usually epochs but your model starts over-fitting after 5 so its not a good measure in your case, maybe do standard deviation over a certain amount of batches trained) if the accuracy leaves the standard deviation space then it might be good idea to stop training. The problem with this technique is that your model has to over-fit a bit to know it needs to stop. The other problem is that your model might seem it is over-fitting but was just a dip in accuracy and was stopped prematurely. (There is methods around this but can't recall them) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another technique is to add drop out. This introduce noise and pushes your model out of a local-minima. Also forces nodes that was ignored to be used and optimized. This surprisingly works well but is not a surefire way to prevent over-fitting. This technique is built into Tensorflow and don't need custom training. You can find more reading about Tensorflow dropout &lt;a href=&quot;https://www.tensorflow.org/tutorials/keras/overfit_and_underfit&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, also they have more techniques on how to overcome over-fitting but their solutions is to change the model and your model seems to be fine. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tf.keras.layers.Dropout(0.2),&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are just some techniques I listed but there is a lot more out there. Unfortunately there is no concrete method to prevent it but steps can be taken to stop well before over-fitting occurs. The first two techniques are mostly used together.&lt;/p&gt;&#xA;" OwnerUserId="14621" LastEditorUserId="14621" LastEditDate="2019-09-09T12:35:17.263" LastActivityDate="2019-09-09T12:35:17.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14341" PostTypeId="1" CreationDate="2019-09-09T14:16:09.250" Score="3" ViewCount="35" Body="&lt;p&gt;I've got a test dataset with 4 features and the PCA produces a set of 4 eigenvectors, e.g., &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EigenVectors: [0.7549043055910286, 0.24177972266822534, -0.6095588015369825, -0.01000612689310429]&#xA;EigenVectors: [0.0363767549959317, -0.9435613299702559, -0.3290509434298886, -0.009706951562064631]&#xA;EigenVectors: [-0.001031816289317291, 0.004364438034564146, 0.016866154627905586, -0.999847698334029]&#xA;EigenVectors: [-0.654824523403971, 0.2263084929291885, -0.7210264051508555, -0.010499173877772439]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do the eigenvector values represent the features from the original dataset? E.g., is feature 1 &amp;amp; 2 explaining most of the variance in eigenvector 1?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I interpreting the results correct to say that features 1 and 2 are therefore the most important in my dataset since PC1 represents 90% of the variance?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to map back to the original features but am unsure how to interpret the results. &lt;/p&gt;&#xA;" OwnerUserId="28572" LastEditorUserId="2444" LastEditDate="2019-09-09T14:45:48.150" LastActivityDate="2019-09-09T17:18:24.487" Title="Do the eigenvectors represent the original features?" Tags="&lt;unsupervised-learning&gt;&lt;features&gt;&lt;principal-component-analysis&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14342" PostTypeId="1" CreationDate="2019-09-09T14:18:46.267" Score="2" ViewCount="46" Body="&lt;p&gt;I am using raw data set with 4 feature variables (Total Cholesterol, Systolic Blood Pressure, Diastolic Blood Pressure, and Cigraeette count) to do a Binominal Classification (find stroke likelihood) using Logistic Regression Algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made sure that the class counts are balanced. i.e., an equal number of occurrences per class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using Python + sklearn, the problem is that the classification performance gets very negatively-impacted when I try to normalize the dataset using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; X=preprocessing.StandardScaler().fit(X).transform(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  X=preprocessing.MinMaxScaler().fit_transform(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So before normalizing the dataset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         precision    recall  f1-score   support&#xA;&#xA;      1       0.70      0.72      0.71        29&#xA;      2       0.73      0.71      0.72        31&#xA;&#xA;avg / total   0.72      0.72      0.72        60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while after normalizing the dataset (the precision of class:1 decreased significantly)&#xA;             precision    recall  f1-score   support&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      1       0.55      0.97      0.70        29&#xA;      2       0.89      0.26      0.40        31&#xA;&#xA; avg / total  0.72      0.60      0.55        60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another observation that I failed to find an explanation to is the probability of each predicted class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before the normalization:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; [ 0.17029846  0.82970154]&#xA; [ 0.47796534  0.52203466]&#xA; [ 0.45997593  0.54002407]&#xA; [ 0.54532438  0.45467562]&#xA; [ 0.45999462  0.54000538]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After the normalization ((for the same test set entries))&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; [ 0.50033247  0.49966753]&#xA; [ 0.50042371  0.49957629]&#xA; [ 0.50845194  0.49154806]&#xA; [ 0.50180353  0.49819647]&#xA; [ 0.51570427  0.48429573] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Dataset description is shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       TOTCHOL    SYSBP    DIABP  CIGPDAY   STROKE&#xA;count  200.000  200.000  200.000  200.000  200.000&#xA;mean   231.040  144.560   81.400    4.480    1.500&#xA;std     42.465   23.754   11.931    9.359    0.501&#xA;min    112.000  100.000   51.500    0.000    1.000&#xA;25%    204.750  126.750   73.750    0.000    1.000&#xA;50%    225.500  141.000   80.000    0.000    1.500&#xA;75%    256.250  161.000   90.000    4.000    2.000&#xA;max    378.000  225.000  113.000   60.000    2.000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;SKEW is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TOTCHOL    0.369&#xA;SYSBP      0.610&#xA;DIABP      0.273&#xA;CIGPDAY    2.618&#xA;STROKE     0.000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a logical explanation for the decreased precision?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a logical explanation for the very-close-to-0.5 probabilities?&lt;/p&gt;&#xA;" OwnerUserId="25463" LastEditorUserId="25463" LastEditDate="2019-09-09T15:04:44.060" LastActivityDate="2021-02-07T12:01:15.120" Title="Is it compulsary to normalize the dataset if doing so can negatively impact a Binary Logistic regression performance?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;logistic-regression&gt;&lt;batch-normalization&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14343" PostTypeId="5" CreationDate="2019-09-09T14:46:38.457" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-09T14:46:38.457" LastActivityDate="2019-09-09T14:46:38.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14344" PostTypeId="4" CreationDate="2019-09-09T14:46:38.457" Score="0" Body="For questions related to the principal component analysis (PCA) technique, which can be used for dimensionality reduction." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-09T14:46:38.457" LastActivityDate="2019-09-09T14:46:38.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14346" PostTypeId="1" CreationDate="2019-09-09T16:17:16.000" Score="1" ViewCount="21" Body="&lt;p&gt;Suppose that we have a labeled training set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; closely cropped images of cars &lt;span class=&quot;math-container&quot;&gt;$(x_1, y_1) , \dots, (x_n, y_n)$&lt;/span&gt;. We then train a CNN on this. Let's say we have &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; test images. Then for each of the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; images, do we use the trained CNN on a cropped out portion of the box to detect whether there is a car or not? If the object is large, wouldn't having a large sliding window have better performance than a smaller sliding window?&lt;/p&gt;&#xA;" OwnerUserId="28575" LastActivityDate="2019-09-09T16:17:16.000" Title="Sliding Window Detection" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14347" PostTypeId="2" ParentId="14341" CreationDate="2019-09-09T17:12:35.180" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The principal components (eigenvectors) correspond to the direction&#xA;  (in the original n-dimensional space) with the greatest variance in&#xA;  the data.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The corresponding eigenvalue is a number that indicates how much&#xA;  variance there is in the data along that eigenvector (or principal&#xA;  component).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, feature 2 is the most important (based on eigenvalue alone). Then feature 1. The other 2 features have little impact and theoretically could be removed as a part of your data reduction effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it is important to point out that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When performing PCA, it is typically a good idea to normalize the data&#xA;  first. Because PCA seeks to identify the principal components with the&#xA;  highest variance, if the data are not properly normalized, attributes&#xA;  with large values and large variances (in absolute terms) will end up&#xA;  dominating the first principal component when they should not.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;IOW, if you didn't normalize your data then your PCA analysis is quite likely meaningless.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;*The above quoted text is from &lt;a href=&quot;http://www.lauradhamilton.com/introduction-to-principal-component-analysis-pca&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.lauradhamilton.com/introduction-to-principal-component-analysis-pca&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1881" LastEditorUserId="1881" LastEditDate="2019-09-09T17:18:24.487" LastActivityDate="2019-09-09T17:18:24.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14348" PostTypeId="1" CreationDate="2019-09-09T20:00:16.570" Score="2" ViewCount="23" Body="&lt;p&gt;Let's assume, that I have a neural network with few numerical features and one binary categorical feature. The network in this case is used for regression. I wonder if such a neural network can properly adjust to two different states of the categorical feature or maybe training two separate networks, according to these two states can be a better idea in the sense of smaller achievable error(assuming I have enough data for each of the two states)? The new model will use simple 'if statement' on the begining of the regression process and use proper network accordingly.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2019-09-09T20:43:43.927" LastActivityDate="2019-09-09T20:43:43.927" Title="Can two neural networks be better instead of one with a categorical feature?" Tags="&lt;neural-networks&gt;&lt;regression&gt;&lt;categorical-data&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14349" PostTypeId="2" ParentId="14262" CreationDate="2019-09-09T20:08:15.433" Score="1" Body="&lt;p&gt;It can be normal and there might be nothing wrong with your model. If there is a very strong and clear correlation in your data(good separability) then a network can achive very high accuracy very fast. After reaching some value learning gets harder.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-09-09T20:08:15.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14351" PostTypeId="1" AcceptedAnswerId="14352" CreationDate="2019-09-10T05:18:34.943" Score="0" ViewCount="69" Body="&lt;p&gt;I'm trying to separate classes in 3D space, the data are as in the sketch below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wMQ0U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wMQ0U.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 3 classes: &lt;strong&gt;0,1,2&lt;/strong&gt;; and with the look into the sketch, it seems that I need &lt;strong&gt;3 planes to separate&lt;/strong&gt; the classes, thus how &lt;strong&gt;many hidden layers should be in the DNN&lt;/strong&gt;? Any roughly how many neurons in each layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some say the number of hidden layers is roughly the number of separation times, so I put 3 hidden layers and it worked! But any reasons behind that simple measure?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-09-10T06:13:01.530" LastActivityDate="2019-09-10T06:42:45.267" Title="How many hidden layers are needed for this training data set" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-neural-networks&gt;&lt;hidden-layers&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14352" PostTypeId="2" ParentId="14351" CreationDate="2019-09-10T06:42:45.267" Score="1" Body="&lt;p&gt;You need to perform Hyperparameter Tuning to identify -&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Number of hidden layers.&lt;/li&gt;&#xA;&lt;li&gt;Number of neurons in each of the hidden layers.&lt;/li&gt;&#xA;&lt;li&gt;Dropout&lt;/li&gt;&#xA;&lt;li&gt;The activation function you use in each of your hidden layers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There parameters are only related to how you build your model. There are others that relate to training like batch size, number of epochs and so on. Your model's performance ultimately depends on how well you tune your hyperparameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that hyperparameter tuning is a trial and error task because it depends on several factors that may not be obvious to us. With experience, experts do build certain thumb rules about what may be the right choice, but there is no way to generalize it. &quot;Some say the number of hidden layers is roughly the number of separation times&quot; - is just another thumb rule. You simply need to find out what best suits your scenario.&lt;/p&gt;&#xA;" OwnerUserId="25704" LastActivityDate="2019-09-10T06:42:45.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14353" PostTypeId="1" CreationDate="2019-09-10T09:17:29.953" Score="1" ViewCount="35" Body="&lt;p&gt;I have developed a new algorithm which has dynamic connections. it is based off of the following paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/334226867_Dynamic_Encoder_Network_to_Process_Noise_into_Images&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/334226867_Dynamic_Encoder_Network_to_Process_Noise_into_Images&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i have figured out how to effect the dynamism. The pseudo code is in the diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally i would really like to know how to do the rest of the code in terms of a general overview. i would need to have the same deconvolutional layer for all training examples, as well as different current_weight_matr&lt;a href=&quot;https://i.stack.imgur.com/NHwDb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NHwDb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;ices for the different training examples. so how do i go about adding the dynamic layers to the deconv neural network . Will i have to have many copies of the same network?&lt;/p&gt;&#xA;" OwnerUserId="28585" LastActivityDate="2019-09-10T09:17:29.953" Title="I am trying to create a network with dynamic connections for every different training example" Tags="&lt;python&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14356" PostTypeId="2" ParentId="13097" CreationDate="2019-09-10T16:51:12.093" Score="1" Body="&lt;p&gt;Both study properties of a network. The literature under respective titles seems to focus on certain topics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Network analysis seems to focus on understanding the structure of a network. Centrality , modularity, assortativity etc are metrics used to study properties of networks. Key areas of research are for egs community detection, centrality measures, clustering algorithms, link prediction&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GDL is oriented more towards using dataset structured as a graph as an input to machine learning problems like classification, regression.&#xA;Key areas of research include graph representation, neural architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some problems like link prediction for example are present in both domains. &#xA;Some problems like network construction for egs isn't covered deeply in both areas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some disciplines like algebraic graph theory appear in both disciplines. &#xA;Fiedler vector for egs  is studied in network analysis for community &#xA;detection . Spectral analysis, matrix factorisation are ideas being explored in representation learning. &lt;/p&gt;&#xA;" OwnerUserId="28590" LastActivityDate="2019-09-10T16:51:12.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14357" PostTypeId="1" CreationDate="2019-09-10T17:08:31.260" Score="3" ViewCount="59" Body="&lt;p&gt;In Reinforcement Learning, when I train a model, it comes up with its own set of solutions. For example, if I am training a robot to walk, it will come up with its own walking gait, such as &lt;a href=&quot;https://youtu.be/hx_bgoTF7bs?t=88&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Deep Mind robot&lt;/a&gt; that has learned to walk in a bizarre gait. It can surely walk/run although the movements does not quite look like a human.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering how can I train a model by providing it some kind of reference motion data? For example, if I collect motion data from a walking human and then provide it to the training, can the training be made learn the walking movements that looks similar to the reference motion data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching online I did find some links that shows this is possible. For example, &lt;a href=&quot;https://youtu.be/9yqa0wfw2lA?t=90&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a research&lt;/a&gt; where the researchers did exactly what I am trying to do, they fed motion data captured from humans to a simulation and made it learn the movements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is: How can I give some kind of hints or reference data to a reinforcement learning model instead of just leaving it all by itself? how exactly is this done? What is it even called? What are some terms and keywords that I can search for to learn more? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="28596" LastActivityDate="2019-11-12T14:52:22.020" Title="Reinforcement learning with hints or reference model" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14358" PostTypeId="1" CreationDate="2019-09-10T17:26:50.613" Score="0" ViewCount="75" Body="&lt;p&gt;I'm about to go into my third and final year of my mathematics degree and I recently discovered some master's programmes in AI that really interest me. I have very little experience in programming. I've done some very basic programming in Python when I was doing my A-levels and some of my Uni modules so far included the use of MATLAB, Maple and R. I'm also going to be doing a module called Formal Languages and Theory of Computation this year, but I feel like this is not going to be enough to show my interest in the subject when I apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone advise on what sort of things I should be doing over next year in order to prepare myself for a course in AI and how do I go about doing them? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="28598" LastEditorUserId="1671" LastEditDate="2019-09-12T21:45:08.207" LastActivityDate="2019-09-12T21:45:08.207" Title="What kind of experience do I need to get into an AI master's programme?" Tags="&lt;getting-started&gt;&lt;academia&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2021-01-21T22:23:14.303" ContentLicense="CC BY-SA 4.0" />
  <row Id="14359" PostTypeId="1" AcceptedAnswerId="23787" CreationDate="2019-09-10T18:05:50.787" Score="1" ViewCount="62" Body="&lt;p&gt;My work colleague got a project with a lot of work that is not hard or complicated.  The problem is simple but it is a lot of work. &#xA;We have two XML files with a lot of variables in it. Not only is the XML files flatten but you would have classes with classes in them that can reach an absurd amount of depth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem comes in that the one file is a request that is received and the other is a response. The request needs to map its variables to the response variables. A simple tool could be built to solve this solution but the problem comes in that there are rules for certain variables. Some variables in the request have arithmetic involve, sometimes with other variables, or some don't get mapped if other variables are present. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about Genetic Programming when I heard about this problem. If all the rules could be defined then it should be able to build a tree that would represent the desired output which is the XML response. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will it work and if not do you think there is an AI algorithm that can solve this? &lt;/p&gt;&#xA;" OwnerUserId="14621" LastActivityDate="2020-09-27T02:42:16.500" Title="Which AI algorithm is great for mapping between two XML files" Tags="&lt;machine-learning&gt;&lt;problem-solving&gt;&lt;genetic-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14360" PostTypeId="2" ParentId="14358" CreationDate="2019-09-10T18:25:44.097" Score="1" Body="&lt;p&gt;Well, mathematics is a very strong field in AI as that is what most AI algorithms revolve around. You only need to learn how to write code (Mostly python or c++, python is easier to learn in my opinion) and how to write mathematical equations in code. Once you have that down it should be enough but I don't how courses work at yours as I know our University require a degree in computer science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to impress somebody then I can pass you an old University assignment I had about backpropagation in neural networks. It does not involve any code but to determine the partial derivative of an equation that is used to update weights in a neural network. If you do the assignment correctly you should know the basics of how a fully connected neural network works, which is the cornerstone of most neural networks, and also how to train it through backpropagation using only maths. Not only would you know how neural networks work but you would have something to show for it. Witch hopefully could contribute to applying for your Masters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, I know of people that wrote scientific papers in the realm of computer science, to prove they would survive in masters, and then applied for a Master's degree in Computer Science. If you are really serious about it you can do this but just double-check with someone as you might do all the work for naught. &lt;/p&gt;&#xA;" OwnerUserId="14621" LastActivityDate="2019-09-10T18:25:44.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14361" PostTypeId="2" ParentId="14358" CreationDate="2019-09-10T18:36:08.870" Score="1" Body="&lt;p&gt;First of all, it is imperative that you have good grades. So work on that. Mathematics is considered as appropriate background for an AI masters so having good grades will get you on the entry radar! Then, since you are on your last year you could select an elective related to AI or even better choose a final project on AI, if available on your course modules. Furthermore, you could check whether a professor in your department's faculty publishes with students and try to work with him for a while. Having published in the field would help your academic CV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from that, in order to gain experience you could check online courses but at the same time work on your own projects. Andrew Ng's &lt;a href=&quot;https://www.deeplearning.ai/deep-learning-specialization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;specialization&lt;/a&gt; has helped me a lot, but what helped me even more was applying the taught concepts and creating a portfolio(e.g. create an API, try to implement papers etc). There is also a &lt;a href=&quot;https://www.coursera.org/specializations/tensorflow-in-practice&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow&lt;/a&gt; specialization among many other courses that you could check on the web. Forming a group would enable hands-on experience as you could collectively work with others and learn from them. So search for local AI groups and within your university, there should be someone you could work with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coding wise, as you probably know python is very popular in AI-ML but C++ and Java are viable options as well. Just decide on what suits you and focus your effort on a single coding language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach you could evaluate is applying for an AI internship if possible. Check your university's research lab or your companies in your area for a data-science or ML related position. This way you can learn and apply concepts while enhancing your CV. In general keep an open mind and do not try to focus on a single AI sub-domain early on. Chances are that you 'll change your mind!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, check the prerequisites of the schools that your want to go to, thoroughly. Follow them and their alumni on linkedin to see what they are up to and try contacting one or two in order to ask questions. Getting inside information is key!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck and may you accomplish your goals!&lt;/p&gt;&#xA;" OwnerUserId="27540" LastActivityDate="2019-09-10T18:36:08.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14363" PostTypeId="1" CreationDate="2019-09-10T20:11:29.470" Score="1" ViewCount="133" Body="&lt;p&gt;I’ve heard it’s not worth getting a masters from a school that’s not top 100, but AI masters is not that common so I feel the selection is quite scarce. Is it worth getting a masters in AI from not a top 100 school? By worth it I mean will it push your career forward or potentially lead to a good PhD program in the future? A post doc I spoke to about the subject said it's not worth it because the professors at not a top 100 school might not make very good mentors and you might end up wasting your time and money by learning to do things in bad way.&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="1671" LastEditDate="2019-09-12T21:44:06.410" LastActivityDate="2019-09-12T21:44:06.410" Title="Is it worth getting a masters in AI from a top 100 ranked university?" Tags="&lt;soft-question&gt;&lt;academia&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14364" PostTypeId="2" ParentId="6274" CreationDate="2019-09-10T23:49:32.240" Score="11" Body="&lt;p&gt;There are 2 problems you might face.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your neural net (in this case convolutional neural net) cannot physically accept images of different resolutions. This is usually the case if one has fully-connected layers, however, if the network is &lt;strong&gt;fully-convolutional&lt;/strong&gt;, then it should be able to accept images of any dimension. Fully-convolutional implies that it doesn't contain fully-connected layers, but only convolutional, max-pooling, and batch normalization layers all of which are invariant to the size of the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exactly this approach was proposed in this ground-breaking paper &lt;a href=&quot;https://arxiv.org/pdf/1411.4038.pdf&quot; rel=&quot;noreferrer&quot;&gt;Fully Convolutional Networks for Semantic Segmentation&lt;/a&gt;. Keep in mind that their architecture and training methods might be slightly outdated by now. A similar approach was used in widely used &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;noreferrer&quot;&gt;U-Net: Convolutional Networks for Biomedical Image Segmentation&lt;/a&gt;, and many other architectures for object detection, pose estimation, and segmentation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convolutional neural nets are not scale-invariant. For example, if one trains on the cats of the same size in pixels on images of a fixed resolution, the net would fail on images of smaller or larger sizes of cats. In order to overcome this problem, I know of two methods (might be more in the literature): &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;multi-scale training of images of different sizes in fully-convolutional nets in order to make the model more robust to changes in scale; and &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;having multi-scale architecture. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A place to start is to look at these two notable papers: &lt;a href=&quot;https://arxiv.org/pdf/1612.03144.pdf&quot; rel=&quot;noreferrer&quot;&gt;Feature Pyramid Networks for Object Detection&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1904.04514.pdf&quot; rel=&quot;noreferrer&quot;&gt;High-Resolution Representations for Labeling Pixels and Regions&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28608" LastEditorUserId="2444" LastEditDate="2020-06-13T20:32:07.500" LastActivityDate="2020-06-13T20:32:07.500" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14365" PostTypeId="2" ParentId="14293" CreationDate="2019-09-11T00:11:08.067" Score="0" Body="&lt;p&gt;I think your fundamental question is: can convolutional neural networks generalize to objects independent of the view?&lt;br&gt;&#xA;The answer is mostly yes (given the dataset contains multiple views of objects). This is evident by looking at the results of various challenges: e.g. &lt;a href=&quot;https://www.youtube.com/watch?v=OOT3UIXZztE&quot; rel=&quot;nofollow noreferrer&quot;&gt;COCO object detection challenge results on youtube&lt;/a&gt;. You can see that no matter what the view is on the car, pedestrians, the detector is not biased towards any specific viewpoint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, one can assume that you can build only one network to perform object detection and another network to perform classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to go even further:&lt;br&gt;&#xA;- you can make a small change to the architecture of your detector (I guess you might be using something like SSD, YOLO or Faster-RCNN), in which you make gender classification for every bounding box prediction. If you think about it, it is intuitive because the detector is already doing classification (there is softmax + cross-entropy loss usually), you can just add another term in its tensor output and modify the loss. That way you don't even need another network! It would be much faster and simpler.&lt;br&gt;&#xA;- you can predict the pose estimation of the object (and corresponding normals) with respect to the camera to capture the best viewpoint to perform classification.&lt;/p&gt;&#xA;" OwnerUserId="28608" LastActivityDate="2019-09-11T00:11:08.067" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14366" PostTypeId="2" ParentId="10911" CreationDate="2019-09-11T00:44:00.763" Score="2" Body="&lt;p&gt;Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just like in training-time) and predict multiple times for a given input, then look at distribution of your outputs. Intuitively this would add &quot;probabilistic, bayesian effect&quot; to you neural network.&#xA;I think this method was first proposed in &lt;a href=&quot;https://arxiv.org/pdf/1506.02142.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout as a Bayesian Approximation:&#xA;Representing Model Uncertainty in Deep Learning&lt;/a&gt; which is called &lt;em&gt;Monte Carlo Dropout&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28608" LastActivityDate="2019-09-11T00:44:00.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15365" PostTypeId="1" AcceptedAnswerId="15373" CreationDate="2019-09-11T04:10:19.787" Score="3" ViewCount="81" Body="&lt;p&gt;In the paper, &lt;a href=&quot;https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1079&amp;amp;context=cs_faculty_pubs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eligibility Traces for off-Policy Policy Evaluation&lt;/a&gt; (2010), by Doina Precup et al., mentioned the term &quot;non-starving&quot; many times. The specific use of the term was like &quot;non-starving policy&quot; in the context of off-policy learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A specific mention of the term&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we consider a method that requires nothing of the behavior&#xA;  policy other than that it be non-starving, i.e., that it&#xA;  never reaches a time when some state-action pair is never&#xA;  visited again.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does the thing look like intuitively? Why is it required?&lt;/p&gt;&#xA;" OwnerUserId="9793" LastEditorUserId="2444" LastEditDate="2019-09-11T13:03:28.937" LastActivityDate="2019-09-15T16:09:55.830" Title="What is a non-starving policy in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;papers&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15367" PostTypeId="1" AcceptedAnswerId="15456" CreationDate="2019-09-11T06:49:49.830" Score="3" ViewCount="1137" Body="&lt;p&gt;I've recently encounter different articles that are recommending to use KL instead of L1/L2 norm when trying to minimize a probability distribution. But none of the articles are giving a clear reasoning why ones is better than the other. Could anyone give me a strong argument why KL is suitable for this?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2019-09-11T06:57:37.973" LastActivityDate="2019-09-16T09:44:09.160" Title="Advantages of Kullback-Leibler over L1/L2?" Tags="&lt;optimization&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15368" PostTypeId="1" AcceptedAnswerId="15419" CreationDate="2019-09-11T07:11:04.483" Score="0" ViewCount="52" Body="&lt;p&gt;A single neuron will be able to do linear separation. For example, XOR simulator network:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x1 --- n1.1&#xA;   \  /    \&#xA;    \/      \&#xA;             n2.1 &#xA;    /\      /&#xA;   /  \    /&#xA;x2 --- n1.2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;x1&lt;/code&gt;, &lt;code&gt;x2&lt;/code&gt; are the 2 inputs, &lt;code&gt;n1.1&lt;/code&gt; and &lt;code&gt;n1.2&lt;/code&gt; are the 2 neurons in hidden layer, and &lt;code&gt;n2.1&lt;/code&gt; is the output neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output neuron &lt;code&gt;n2.1&lt;/code&gt; does a &lt;strong&gt;linear separation&lt;/strong&gt;. How about the 2 neurons in hidden layer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it still called linear separation&lt;/strong&gt; (at 2 nodes and join the 2 separation lines)? or &lt;strong&gt;polynomial separation of degree 2&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm confused about how it's called because there are curvy lines in this wiki article: &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Overfitting&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o8idU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o8idU.png&quot; alt=&quot;Some curvy separation?&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UByN2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UByN2.png&quot; alt=&quot;Some other curvy separation?&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-09-11T07:18:52.387" LastActivityDate="2019-09-13T10:31:15.020" Title="Is it still called linear separation with a layer of more than 1 neuron" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;deep-neural-networks&gt;&lt;artificial-neuron&gt;&lt;linear-regression&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15369" PostTypeId="2" ParentId="15368" CreationDate="2019-09-11T07:26:47.480" Score="0" Body="&lt;p&gt;What you have depicted is a nonlinear classificator. Although each stage does a linear separation, the sequential composition of linear separations is nonlinear. The nonlinearity of the neuron is key in this regard, as otherwise it would all be equivalent to a matrix multiplication, which is linear. You were right about the degree, although it's rarely called like that. People usually describe just the number of layers, and I guess the main reason is that it's not directly equivalent to such polynomials, as that depends on other factors (e.g. the activation function).&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-09-11T07:26:47.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15371" PostTypeId="1" CreationDate="2019-09-11T09:39:10.840" Score="1" ViewCount="258" Body="&lt;p&gt;I want to create a model to solve a multi-class classification problem.&lt;/p&gt;&#xA;&lt;p&gt;Here are more details about my problem.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Every picture contains only one object&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The background is very simple&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All objects belong to the same family of objects (for example, all objects are knives), but there are different specific subtypes&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the model will learn and predict the name of the object (example: the model learn all types of knives, and when it get an image it will tell us the name of the knife)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To be clear, let's say I have 50 types of knives, and the output of the model has to recognize the correct name of the knife. Knife name could be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chef's Knife,&lt;/li&gt;&#xA;&lt;li&gt;Heavy Duty Utility Knife,&lt;/li&gt;&#xA;&lt;li&gt;Boning Knife, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To solve this problem, I have started to use annotated, segmented (masked) images (&lt;a href=&quot;https://cocodataset.org/#home&quot; rel=&quot;nofollow noreferrer&quot;&gt;COCO&lt;/a&gt;-like dataset) and the &lt;a href=&quot;https://arxiv.org/pdf/1703.06870.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mask R-CNN&lt;/a&gt; model.&lt;/p&gt;&#xA;&lt;p&gt;As a first step, I got a prediction, but I really don't know if I'm on the right way.&lt;/p&gt;&#xA;&lt;p&gt;For this problem, Mask R-CNN could be the solution, or it is impossible to recognize a tiny difference between two objects from the same class (for example Chef's Knife, Heavy Duty Utility Knife)?&lt;/p&gt;&#xA;" OwnerUserId="29616" LastEditorUserId="2444" LastEditDate="2021-03-09T18:06:05.573" LastActivityDate="2021-04-08T19:03:40.330" Title="Is Mask R-CNN suited to solve a multi-class classification problem where the classes are related?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;&lt;mask-rcnn&gt;&lt;multiclass-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15372" PostTypeId="1" AcceptedAnswerId="15377" CreationDate="2019-09-11T12:16:29.353" Score="3" ViewCount="68" Body="&lt;p&gt;Is it crucial to always have the same initial (starting) state for Reinforcement Learning, for example, for Q-learning or DQN?&#xA;Or it can vary?&lt;/p&gt;&#xA;" OwnerUserId="29617" LastEditorUserId="2444" LastEditDate="2019-09-11T12:34:29.260" LastActivityDate="2019-09-11T16:34:29.597" Title="How important is the choice of the initial state?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15373" PostTypeId="2" ParentId="15365" CreationDate="2019-09-11T12:59:48.743" Score="4" Body="&lt;p&gt;A non-starving policy is a (behavior) policy that is &lt;em&gt;theoretically&lt;/em&gt; guaranteed to visit each state and take all possible actions from each state an infinite number of times, so that to always update &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\forall s, \forall a$&lt;/span&gt;, an infinite number of times. In the context of off-policy prediction, this criterion implies that any trajectory will have no zero probability under a behavior policy. As a consequence, the experience from the behavior policy sufficiently covers the possibilities of any target policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a non-starving policy is the &lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy&lt;/em&gt;, which, with &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \epsilon \leq 1$&lt;/span&gt; (which is usually a small number between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) probability, takes a random action from a given state, and, with &lt;span class=&quot;math-container&quot;&gt;$1-\epsilon$&lt;/span&gt; probability, takes the current best action, that is, the action with the highest value from a given state, according to the current value function.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="9793" LastEditDate="2019-09-15T16:09:55.830" LastActivityDate="2019-09-15T16:09:55.830" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="15374" PostTypeId="1" CreationDate="2019-09-11T13:14:31.460" Score="2" ViewCount="171" Body="&lt;p&gt;I am using a fully connected neural network that uses sigmoid activation function. If we feed a big enough weights the sigmoid function will finally become 1 or 0, is there any solution to avoid this?&lt;/p&gt;&#xA;&lt;p&gt;Will this lead to classical sigmoid problems vanishing gradient or exploding gradient?&lt;/p&gt;&#xA;" OwnerUserId="28602" LastEditorUserId="38892" LastEditDate="2020-07-28T20:35:19.933" LastActivityDate="2020-07-28T20:35:19.933" Title="why the sigmoid function will be 1 and 0 if we use a fully connected layer that produce a big enough positive(res negative )output" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15375" PostTypeId="1" CreationDate="2019-09-11T15:30:21.440" Score="0" ViewCount="44" Body="&lt;p&gt;I have a huge amount of data and I want to calculate my false positive and false negative. Is there a software that can help me determine it?&lt;/p&gt;&#xA;" OwnerUserId="29622" LastEditorUserId="2444" LastEditDate="2019-09-11T19:58:08.613" LastActivityDate="2019-09-11T19:58:08.613" Title="How to calculate the false positives and negatives?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;data-science&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15376" PostTypeId="1" AcceptedAnswerId="15381" CreationDate="2019-09-11T16:05:13.550" Score="2" ViewCount="40" Body="&lt;p&gt;Is it still a policy iteration algorithm if the policy is updated optimizing a function of the immediate reward instead of the value function?&lt;/p&gt;&#xA;" OwnerUserId="29621" LastEditorUserId="1847" LastEditDate="2019-09-11T17:55:44.527" LastActivityDate="2019-09-11T18:57:37.010" Title="Can policy iteration use only the immediate reward for updates?" Tags="&lt;reinforcement-learning&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15377" PostTypeId="2" ParentId="15372" CreationDate="2019-09-11T16:12:29.877" Score="2" Body="&lt;p&gt;The initial state can vary during in both training and use, and how you decide to do this makes very little difference to Q-learning. The important factor is whether all state/action pairs relevant to optimal behaviour can be reached. As there is already randomness in any exploring policy, and in many environments as part of state transitions and reward functions, adding some more at the start is not an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More formally, you can take any existing environment with states &lt;span class=&quot;math-container&quot;&gt;$S_1, S_2, S_3 ... S_n$&lt;/span&gt; plus defined actions and rewards. Add a special fixed start state &lt;span class=&quot;math-container&quot;&gt;$S_{0}$&lt;/span&gt; with one special action &lt;span class=&quot;math-container&quot;&gt;$A_{0}$&lt;/span&gt; allowed. Make the state transition matrix following that action any distribution you like over the other states, and reward &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. It is clearly a valid MDP, and is identical to your original MDP in terms of value and policy functions for states &lt;span class=&quot;math-container&quot;&gt;$S_1, S_2, S_3 ... S_n$&lt;/span&gt;. For all intents and purposes to the agent (which gets no meaningful policy choice in &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;), it starts in the original MDP in some randomly chosen state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is as long as the variations are not made to deliberately exclude some of the state space during training that it would need later, or otherwise used to &quot;attack&quot; the agent and prevent it learning. Q-learning proofs of convergence assume that all state/action pairs are reachable an infinite number of times in the limit of infinite training time. Of course in practice this is never achieved, but clearly if you excluded some important state from ever being seen at the start by choosing to start in a way that it is never reachable, then the agent would never learn about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use a highly variable start state for training to your advantage. It is a form of exploration, and truly exploring starts (that could start from any state/action pair, so you would want to randomise the first action choice) allow you to skip any further exploration and still guarantee convergence to an optimal policy - i.e. you could learn Q values on-policy using a deterministic policy. This will not necessarily make things converge faster, but does go to demonstrate that random start states are not a barrier to RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of minor/partial exceptions to &quot;do what you like&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the environment is otherwise highly deterministic, adding a random start may increase the difficulty in optimising it, as more states will be reachable, even with an optimal policy, therefore the state space over which optimal behaviour needs to be discovered may be larger and take longer to learn.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Policy Gradient methods, like REINFORCE, optimise the policy towards maximising return from a known distribution of start states. So you can vary start state, but you should really stick to a standard start procedure, including a fixed &lt;em&gt;distribution&lt;/em&gt; of allowed start states. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Similarly, if you want to report a standard score for your agent, it is common to quote the expected return from the start states. Again that doesn't mean you have a fixed start state, but you should use a fixed &lt;em&gt;distribution&lt;/em&gt; of start states to assess your agent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many standard environments, like Open AI's Gym environments CartPole and LunarLander, include some amount of randomness in the start state, so that the agent has to solve a more general problem than just generating a fixed sequence of actions that always works from the beginning.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-11T16:34:29.597" LastActivityDate="2019-09-11T16:34:29.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15378" PostTypeId="2" ParentId="15375" CreationDate="2019-09-11T17:11:45.683" Score="1" Body="&lt;p&gt;Yes, you can use &lt;code&gt;sklearn&lt;/code&gt;'s &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;confusion_matrix&lt;/code&gt;&lt;/a&gt;. To explicitly extract the false positives and negatives, you can do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.metrics import confusion_matrix&#xA;&#xA;y_true = [0, 1, 0, 1]&#xA;y_pred = [1, 1, 1, 0]&#xA;tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-09-11T17:11:45.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15379" PostTypeId="1" CreationDate="2019-09-11T17:20:27.750" Score="1" ViewCount="193" Body="&lt;p&gt;In Policy Iteration (PI), the action generated by the policy, whether it's optimal or not w.r.t the current value function &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;. Whereas, in Value Iteration, the action is greedily generated w.r.t current &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;, which is an approximation of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;objective function&lt;/a&gt; (as I understand). As a consequence, in the first few iterations, will Value Iteration perform better than Policy Iteration?&lt;/p&gt;&#xA;" OwnerUserId="29621" LastEditorUserId="9947" LastEditDate="2019-09-15T18:59:33.763" LastActivityDate="2019-09-15T18:59:33.763" Title="Is Value Iteration better than Policy Iteration for first few iterations?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15380" PostTypeId="2" ParentId="14363" CreationDate="2019-09-11T18:08:03.917" Score="1" Body="&lt;p&gt;This is a very vague question as well as a really vague statement. Deciding on your career based on &quot;one person who did x said y...&quot; shouldn't be a sole guide to anything in life. Even gathering a multitude of opinions would be useless if your sample is biased on either opinion, good or bad. Personally if I had the rep I would flag the question as it is also a duplicate...&lt;a href=&quot;https://ai.stackexchange.com/questions/13161/is-a-masters-in-ai-worth-it&quot;&gt;&lt;strong&gt;your&lt;/strong&gt;&lt;/a&gt; duplicate. Anyways, I 'd like to try and answer  objectively even if I am biased towards a positive attitude by personal and peer experience.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By worth it I mean will it push your career forward or potentially lead to a good PhD program in the future? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is nothing that &quot;pushes&quot; your career towards a set goal, other than your own ambition and effort. Receiving the MsC title won't do any good if you don't have the work to show for it. Yes a masters in A.I. is a step in the ladder of pursuing a PhD, but in order to achieve it you need to tick some boxes. There should be a specific sub-domain you target (else I believe a lifetime would not be enough to gain expertise). One should seek out the people in his network that can assist to that, be it professors, supervisors, colleagues or family. Academia does not pay that well , even directors of AI programs do not get as much as in industry thus you should decide which path is more keen to you. But most importantly you need to work hard. Often times a masters opens up mid-level/senior engineer positions, with some experience so it sure does help. Before proceeding to the second part of the question we need to define what &quot;a good PhD&quot; means. Does it mean getting into Stanford? Does it mean finding the right people that have the expertise at whichever university regardless of ranking? You need to decide on that. Then it is easier as there are steps for both. Not definitive ones, but in general you can carve out a plan. For example getting into Stanford would mean that you have excellent grades, a great network for recommendation letters along with published articles. Oh you need to be able to afford it too. The second would suggest that you search tirelessly for universities that have labs with high output, talk to the people in charge and decide which one is best for you.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A post doc I spoke to about the subject said it's not worth it because the professors at not a top 100 school might not make very good mentors and you might end up wasting your time and money by learning to do things in bad way.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am sorry to criticize this harshly, but it is among the most preposterous statements I have ever read regarding academia. Let's grab a rankings list. For example the &lt;a href=&quot;https://www.timeshighereducation.com/world-university-rankings/2019/world-ranking#!/page/4/length/25/subjects/3081/sort_by/scores_teaching/sort_order/asc/cols/scores&quot; rel=&quot;nofollow noreferrer&quot;&gt;Times&lt;/a&gt; one. This would mean that Karlsruhe, Eindhoven, Uppsala, Copenhagen, St Andrews (&lt;em&gt;selection is arbitrary but I know that St Andrews has a great research oriented department&lt;/em&gt;) among others do not have a single person that would help you and they might even make you learn things the wrong way...let's be serious please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that there are bad apples, there exist everywhere. While pursuing a phd you won't attend any lectures though. You will study and do research. That means questioning everything, even your mentors. This is how science advances. Another thing you should consider is what do you expect from the PhD. Do you want to enter the industry or pursue a career in academia? Depending on your answer you should seek different institutions and different people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest these sources too:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/4750/which-online-ai-masters-degrees-do-you-recommend-me&quot;&gt;Ai masters Q#1&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/learnmachinelearning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ML Learning Reddit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More importantly I would suggest taking time to talk with people from universities. Try tracking down alumni from schools you like and talk to them about their experience, people are always eager to talk about themselves. Then decide what it is that you want to do. A million stack questions want make the decision for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DISCLAIMER:&lt;/strong&gt; &lt;em&gt;This is a positively biased opinion based on personal experience as well as based on the notion that in order to achieve a goal one must have clearly predefined it and work hard towards it.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DISCLAIMER 2:&lt;/strong&gt; Times just updated their &lt;a href=&quot;https://www.timeshighereducation.com/world-university-rankings/world-university-rankings-2020-reaching-critical-mass?utm_source=THE%20Website%20Users&amp;amp;utm_campaign=f6f650fc25-EMAIL_CAMPAIGN_2019_09_04_03_14&amp;amp;utm_medium=email&amp;amp;utm_term=0_daa7e51487-f6f650fc25-74501945&quot; rel=&quot;nofollow noreferrer&quot;&gt;rankings&lt;/a&gt; so the previous link is deprecated...&lt;s&gt;not that it matters as most of ranking sites are marketing stunts and unis pay for them shhh&lt;/s&gt;&lt;/p&gt;&#xA;" OwnerUserId="27540" LastEditorUserId="27540" LastEditDate="2019-09-11T19:18:22.690" LastActivityDate="2019-09-11T19:18:22.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15381" PostTypeId="2" ParentId="15376" CreationDate="2019-09-11T18:51:05.097" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it still a policy iteration algorithm if the policy is updated optimizing a function of the immediate reward instead of the value function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Technically yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value update step in Policy Iteration is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s) \leftarrow \sum_{r,s'}p(r,s'|s,\pi(s))(r + \gamma v(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; can be set to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, making the update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s) \leftarrow \sum_{r,s'}p(r,s'|s,\pi(s))r$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are two key details that are important, and make this a &lt;em&gt;technically&lt;/em&gt; yes, not some alternative way of solving problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Changing discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; changes what it means for an agent to act optimally. Setting it to zero means that the agent will prioritise only its immediate reward signal, and make no long-term decisions at all. This would be useless for instance if the task was to escape a maze in minimum time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Technically there is still a value function being updated. The function &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; is still the expected future reward, just we have set it to only care a very short step into the future. So short that it doesn't care what the value of the next state is, so the next state does not appear in any updates.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Due to the lack of bootstrapping between states, &lt;em&gt;all&lt;/em&gt; the data for optimal behaviour is already available in the reward distribution. So the entire MDP can be solved with a single sweep through all the states. Or it could just be solved on-demand using &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \text{argmax}_a[\sum_{r,s'}p(r,s'|s,a)r]$&lt;/span&gt; for any state, making a policy iteration process redundant.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, with these caveats in mind, yes this is still policy iteration. It is the &lt;em&gt;same&lt;/em&gt; update process, just with a particular choice of one of the parameters.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-11T18:57:37.010" LastActivityDate="2019-09-11T18:57:37.010" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15383" PostTypeId="2" ParentId="4321" CreationDate="2019-09-11T22:58:57.500" Score="0" Body="&lt;p&gt;I will add on top of another answer which proposed to use LSTM network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSTMs and RNNs in general are great tools in speech recognition that can model sequential information inherently. There were several ground-breaking papers which used RNNs, e.g. &lt;a href=&quot;https://arxiv.org/abs/1412.5567&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Speech: Scaling up end-to-end speech recognition&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1512.02595&quot; rel=&quot;nofollow noreferrer&quot;&gt;its second version&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently we have found that Convolutional Neural Networks (CNNs) are also great with speech/sound even though they do not model sequential information inherently. There was a problem that it wouldn't be able to model long-term dependencies due to small receptive field of kernels (even though they are stacked in many layers). &lt;a href=&quot;https://arxiv.org/abs/1609.03499&quot; rel=&quot;nofollow noreferrer&quot;&gt;WaveNet: A Generative Model for Raw Audio&lt;/a&gt; was one of the first papers that successfully used CNN to generate speech. One of their main contributions was usage of dilated convolutions which allowed them to model long-term dependencies. In principle, you can use a similar architecture but instead of predicting another point in time as done in Wavenet you can do your own classification in the last layer (while also changing the loss to cross-entropy for example). You might also want to look at the papers that cite Wavenet because deep learning is quickly growing field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally there is an &lt;a href=&quot;https://www.youtube.com/watch?v=HyUtT_z-cms&quot; rel=&quot;nofollow noreferrer&quot;&gt;amazing lecture on Speech Recognition&lt;/a&gt; which you might find very valuable and it's relatively recent too.&lt;/p&gt;&#xA;" OwnerUserId="28608" LastActivityDate="2019-09-11T22:58:57.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15384" PostTypeId="2" ParentId="14153" CreationDate="2019-09-11T23:21:29.513" Score="1" Body="&lt;p&gt;One of the key terms in the literature that you are looking for is &lt;em&gt;video captioning&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have a look at some of the &lt;a href=&quot;https://paperswithcode.com/task/video-captioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;relevant papers with code on this subject&lt;/a&gt;. In short, it is an active area of research and a difficult problem, one reason is because videos are still difficult to learn about (because of larger amount of data + larger model, etc...) and this model has to be working with two modalities of data: text and image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A paper that you might want to start with is &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/deepimagesent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Visual-Semantic Alignments for Generating Image Descriptions&lt;/a&gt; which works on single images. In short, you can use something similar like in the paper: object detector (e.g. Faster RCNN) to extract visual features and feed them into the state of an RNN (LSTM) which would output a sequence of words in your summary (see picture below).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j28Jx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j28Jx.png&quot; alt=&quot;image captioning model&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28608" LastActivityDate="2019-09-11T23:21:29.513" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15385" PostTypeId="2" ParentId="8221" CreationDate="2019-09-12T01:26:29.163" Score="-1" Body="&lt;p&gt;If you are looking for the functional verbs used in VA/VE (value analysis / value engineering) look at: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.valueanalysis.ca/functionanalysis.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.valueanalysis.ca/functionanalysis.php&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.brcommunity.com/articles.php?id=b679&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.brcommunity.com/articles.php?id=b679&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.washoeschools.net/cms/lib08/NV01912265/Centricity/Domain/253/Social%20Studies/Marker%20Verbs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.washoeschools.net/cms/lib08/NV01912265/Centricity/Domain/253/Social%20Studies/Marker%20Verbs.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28546" LastActivityDate="2019-09-12T01:26:29.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15386" PostTypeId="1" CreationDate="2019-09-12T02:03:04.950" Score="1" ViewCount="45" Body="&lt;p&gt;The Transformer model proposed in &quot;Attention Is All You Need&quot; uses sinusoid functions to do the positional encoding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why have both sine and cosine been used? And why do we need to separate the odd and even dimensions to use different sinusoid functions?&lt;/p&gt;&#xA;" OwnerUserId="29631" LastEditorUserId="2444" LastEditDate="2019-11-01T03:08:03.163" LastActivityDate="2019-11-01T03:08:03.163" Title="Why do both sine and cosine have been used in positional encoding in the transformer model?" Tags="&lt;deep-learning&gt;&lt;papers&gt;&lt;machine-translation&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15387" PostTypeId="1" AcceptedAnswerId="15391" CreationDate="2019-09-12T03:04:52.880" Score="4" ViewCount="283" Body="&lt;p&gt;When we are working on an AI project, does the background or the domain make the process different? The &quot;domain&quot; I mean here is like AI for academic research, AI for industry, and AI for competition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I see in the competition most participants even winners use the stacking model, but I have not found anyone implementing it in the industry. How about the cross-validation process, I think there is a slight difference in industry and academia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So does the domain of an AI project will make the process different? if so, what are the things I need to pay attention to when creating an AI project based on its domain?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-09-16T09:23:08.720" Title="AI on Academic vs Industry vs Competition" Tags="&lt;ai-design&gt;&lt;academia&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15388" PostTypeId="1" CreationDate="2019-09-12T03:35:02.333" Score="1" ViewCount="688" Body="&lt;p&gt;From my understanding, the critic evaluates the policy (actor) following dynamic programming (DP) or approximate dynamic programming (ADP) scheme, which should converge to the optimal value function after sufficient iterations. The policy (actor) then updates its parameter w.r.t the optimal value function using gradient methods. This policy evaluation and improvement circle are repeated until neither the critic nor the actor changes anymore. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How's guaranteed to converge as a whole? Is there any mathematical proof? Is it possible that it may converge to a local optimal point instead of a global one?&lt;/p&gt;&#xA;" OwnerUserId="29621" LastEditorUserId="2444" LastEditDate="2019-09-13T12:29:46.763" LastActivityDate="2019-09-13T19:49:50.597" Title="How is the actor-critic algorithm guaranteed to converge?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15389" PostTypeId="1" CreationDate="2019-09-12T05:02:54.927" Score="1" ViewCount="19" Body="&lt;p&gt;I have a probabilistic classifier that produces a distribution over my 3 classes - C1, C2, C3.&#xA;I want to compare some new points I'm classifying &lt;strong&gt;to each other&lt;/strong&gt;, to see which one is the best fit for a specific class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example:.&#xA;for a new point X1 the classifier will output something like [0.2, 0.2, 0.6]&#xA;for another new point X2 it will produce [0.2, 0.4, 0.4] &#xA;so for both X1, X2 - the chosen class would be C3.&#xA;Now I want to know - &lt;strong&gt;which of X1, X2 is a better fit to C3&lt;/strong&gt;&#xA;I cannot simply choose the one with the highest probability for C3, because it's probability for C3 depends on it's probabilities for the other classes. X1 got 0.6 and X2 0.4, but it's possible that X2 is closer to C3 in the hyper-plane than X1, it is just &lt;strong&gt;less unique&lt;/strong&gt; for C3 than X1, and therefor X1 got a higher probability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;here's a visual, in 2 dimensions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ETeWR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ETeWR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;X2, who got a lower probability, is clearly a better fit to the Red class then X1, which is truly &lt;strong&gt;unique&lt;/strong&gt; to the Red class, but is further from the class cluster&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;how do I normalize the results of a probabilistic classifier so I can compare predictions to each other?&lt;/li&gt;&#xA;&lt;li&gt;given an output of a probabilistic classier - how can I get the actual distance from the probabilities. It must be possible because there's an exact mapping between a set of probabilities to a point in the classified hyper-plan.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&#xA;Amir &lt;/p&gt;&#xA;" OwnerUserId="29636" LastActivityDate="2019-09-12T05:02:54.927" Title="Probabilistic classification - normalize results" Tags="&lt;classification&gt;&lt;probability&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15391" PostTypeId="2" ParentId="15387" CreationDate="2019-09-12T06:42:39.730" Score="4" Body="&lt;p&gt;I cannot comment about the process for AI for academia. I can compare AI for competitions and AI for business. To clarify whatever I say is about ML not any other AI techniques. The process might be different for other techniques. But most of things that I say are general enough that I am assuming should still apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference that I saw while doing ML for a competition vs. for a business was that of focus. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When doing it for a competition for Kaggle the focus was mainly creating the model&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;machine learning metrics are specified for you&lt;/li&gt;&#xA;&lt;li&gt;some data was given to you&lt;/li&gt;&#xA;&lt;li&gt;business problem was given to you&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When doing it for business what is different&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;given a business problem finding the parts that can actually benefit from ML. You have to define the ML problem in it and define how it actually benefits the business. This may involve significant discussions with business stakeholders, weighting the pros and cons of doing it versus doing something else, communicate the benefits to the business stakeholders, take them into confidence for the process to start&lt;/li&gt;&#xA;&lt;li&gt;find the right data for the problem from scratch, ensure it is collected by rest of the system or brought from 3rd parties&lt;/li&gt;&#xA;&lt;li&gt;define business metrics over and above machine learning metrics. At the end of day nobody really cares about whether ML model recall, accuracy is good or bad. What is important is the relevant business effect.&lt;/li&gt;&#xA;&lt;li&gt;make the model, deploy it and integrate it with the rest of the system. This is important because if your goal is just to making the model you would not care about the factors associated with actually using it i.e. latency of predictions, cost of machines needed to run it etc.&lt;/li&gt;&#xA;&lt;li&gt;A/B testing for the models, running multiple models in parallel, dynamically being able to adjust which models to use&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this gives some idea about the differences in AI for competitions and AI for business.&lt;/p&gt;&#xA;" OwnerUserId="29638" LastActivityDate="2019-09-12T06:42:39.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15393" PostTypeId="1" CreationDate="2019-09-12T08:09:53.253" Score="6" ViewCount="108" Body="&lt;p&gt;Let's say we have a problem that can be solved by some RL algorithms (DQN, for example, because we have discrete action space). At first, the action space is fixed (the number of actions is &lt;span class=&quot;math-container&quot;&gt;$n_1$&lt;/span&gt;), and we have already well trained an offline DQN model. Later, we have to add more actions for some reasons (and the number of action is now &lt;span class=&quot;math-container&quot;&gt;$n_2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n_2 &amp;gt; n_1$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Are there some solutions to update the value function or policy (or the neural network) with only minor changes?&lt;/p&gt;&#xA;" OwnerUserId="29643" LastEditorUserId="2444" LastEditDate="2020-12-28T01:34:45.740" LastActivityDate="2020-12-28T01:34:45.740" Title="Are there RL techniques to deal with incremental action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15394" PostTypeId="1" AcceptedAnswerId="15405" CreationDate="2019-09-12T09:37:02.920" Score="1" ViewCount="38" Body="&lt;p&gt;I'm learning HMMs and decided to model a problem for learning purposes. I came to this idea of word predicting by letters.&#xA;here is the model :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while typing, the word is typed letter by letter, so we can consider them as series of observations.&#xA;let's say we have just 4 words in our database:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Tap&lt;/li&gt;&#xA;&lt;li&gt;Trip&lt;/li&gt;&#xA;&lt;li&gt;Trap&lt;/li&gt;&#xA;&lt;li&gt;Trigger&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and we want to predict the word after 1,2 or 3 written letters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have to define states and HMM parameters (state transitions, emissions and priors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;our &lt;strong&gt;[hidden ?] states&lt;/strong&gt; would be :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[ ][ ][ ][ ][ ][ ][ ] : no observations. I chose 7 [ ] because of the longest word&lt;/li&gt;&#xA;&lt;li&gt;T [ ][ ][ ][ ][ ][ ]&lt;/li&gt;&#xA;&lt;li&gt;T A [ ][ ][ ][ ][ ]&lt;/li&gt;&#xA;&lt;li&gt;T R [ ][ ][ ][ ][ ]&lt;/li&gt;&#xA;&lt;li&gt;… .&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and we have to &lt;strong&gt;&lt;em&gt;learn&lt;/em&gt;&lt;/strong&gt; the transition probabilities of each state pairs (the pairs which come after each other like T,TR and TA. but not  for TA and TR).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;our &lt;strong&gt;prior probabilities&lt;/strong&gt; are 1/3 because we have 3 words. but we may change it by learning which word is used more frequently.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, I have these questions :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;is HMM suitable for this kind of problem ?&lt;/li&gt;&#xA;&lt;li&gt;are my assumptions (about states and prior probabilities) correct ?&lt;/li&gt;&#xA;&lt;li&gt;the states get out of hand when the word count increases. making the model very complex. is that deduction true? &lt;/li&gt;&#xA;&lt;li&gt;how the emission probabilities are defined in this model ?&lt;/li&gt;&#xA;&lt;li&gt;what do I miss in case of parameters or definitions ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm a new contributor, be nice to me. regards :) &lt;/p&gt;&#xA;" OwnerUserId="18124" LastEditorUserId="18124" LastEditDate="2019-09-12T09:55:17.843" LastActivityDate="2019-09-12T18:06:22.497" Title="is a &quot;word prediction&quot; problem, applicable using HMMs?" Tags="&lt;hidden-markov-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15395" PostTypeId="2" ParentId="15393" CreationDate="2019-09-12T09:39:53.327" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there some solutions to update the model with only minor changes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, assuming the new action choices are meaningful - in at least some states, the expected return from taking one of the new actions is higher than the current optimal policy using just the old action selection - then the answer here is &quot;no&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the very least you will need to re-train your agent such that it explores the new action choices, and learns the new value function and policy. You can of course start this re-training using data and internal representations learned from the earlier environment, and that may help if the new actions have not changed things too radically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of things that might help improve performance on this re-training:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the actions are not entirely discrete, but have some features that could be generalised from, you could base your value function or policy function estimators on those features instead of discrete actions. So for example in DQN your input to the neural network would be concatenated state, action feature vectors, and output a single value. Then it may generalise to the new actions quickly, in some cases perhaps even getting close to the correct value estimates from the start.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you were training using DynaQ+, this includes an exploration term (which is added to planning assessments of immediate reward) that will prioritise exploring new state/action pairs when they appear. Other planning algorithms may have similar adjustments, although I am not aware of specific ones that could be dropped straight into a DQN agent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you know in advance which states the new actions are likely to be most useful in, you may be able to insert that knowledge into some initial action selection or value estimate helpers to avoid the need to train from scratch.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-12T11:26:31.543" LastActivityDate="2019-09-12T11:26:31.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15396" PostTypeId="1" AcceptedAnswerId="15400" CreationDate="2019-09-12T10:26:36.237" Score="0" ViewCount="276" Body="&lt;p&gt;I have these training data to separate, the classes are rather randomly scattered:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/uVOxZR7.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first attempt was using &lt;code&gt;tf.nn.relu&lt;/code&gt; activation function, but output was stuck with whatever number of training steps. So I guessed it could be because of &lt;strong&gt;dead ReLU&lt;/strong&gt; units, thus I changed the activation function in hidden layers to &lt;code&gt;tf.nn.leaky_relu&lt;/code&gt;, but it's still no good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works when all hidden layers come with &lt;code&gt;tf.sigmoid&lt;/code&gt;, yes, but why doesn't ReLU work here? dead ReLU units, or exploding gradients, or anything else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Source code&lt;/strong&gt; (TensorFlow):&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#core&#xA;import time;&#xA;&#xA;#libs&#xA;import tensorflow        as tf;&#xA;import matplotlib.pyplot as pyplot;&#xA;&#xA;#mockup to emphasize value name&#xA;def units(Num):&#xA;  return Num;&#xA;#end def&#xA;&#xA;#PROGRAMME ENTRY POINT==========================================================&#xA;#data&#xA;#https://i.imgur.com/uVOxZR7.png&#xA;X = [[1,1],[1,2],[1,3],[2,1],[2,2],[2,3],[3,1],[3,2],[3,3],[4,1],[4,2],[4,3],[5,1],[6,1]];&#xA;Y = [[0],  [1],  [0],  [1],  [0],  [1],  [0],  [2],  [1],  [1],  [1],  [0],  [0],  [1]  ];&#xA;Max_X      = 6;&#xA;Max_Y      = 2;&#xA;Batch_Size = 14;&#xA;&#xA;#normalise&#xA;for I in range(len(X)):&#xA;  X[I][0] /= Max_X;&#xA;  X[I][1] /= Max_X;&#xA;  Y[I][0] /= Max_Y;&#xA;#end for&#xA;&#xA;#model&#xA;Input     = tf.placeholder(dtype=tf.float32, shape=[Batch_Size,2]);&#xA;Expected  = tf.placeholder(dtype=tf.float32, shape=[Batch_Size,1]);&#xA;&#xA;#RELU DOESN'T WORK, DEAD RELU? SIGMOID WORKS BUT SLOW.&#xA;#CHANGE TO tf.sigmoid OR tf.tanh AND IT WORKS:&#xA;activation_fn = tf.nn.leaky_relu;&#xA;&#xA;#1&#xA;Weight1   = tf.Variable(tf.random_uniform(shape=[2,units(60)], minval=-1, maxval=1));&#xA;Bias1     = tf.Variable(tf.random_uniform(shape=[  units(60)], minval=-1, maxval=1));&#xA;Hidden1   = activation_fn(tf.matmul(Input,Weight1) + Bias1);&#xA;&#xA;#2&#xA;Weight2   = tf.Variable(tf.random_uniform(shape=[60,units(50)], minval=-1, maxval=1));&#xA;Bias2     = tf.Variable(tf.random_uniform(shape=[   units(50)], minval=-1, maxval=1));&#xA;Hidden2   = activation_fn(tf.matmul(Hidden1,Weight2) + Bias2);&#xA;&#xA;#3&#xA;Weight3   = tf.Variable(tf.random_uniform(shape=[50,units(40)], minval=-1, maxval=1));&#xA;Bias3     = tf.Variable(tf.random_uniform(shape=[   units(40)], minval=-1, maxval=1));&#xA;Hidden3   = activation_fn(tf.matmul(Hidden2,Weight3) + Bias3);&#xA;&#xA;#4&#xA;Weight4   = tf.Variable(tf.random_uniform(shape=[40,units(30)], minval=-1, maxval=1));&#xA;Bias4     = tf.Variable(tf.random_uniform(shape=[   units(30)], minval=-1, maxval=1));&#xA;Hidden4   = activation_fn(tf.matmul(Hidden3,Weight4) + Bias4);&#xA;&#xA;#5&#xA;Weight5   = tf.Variable(tf.random_uniform(shape=[30,units(20)], minval=-1, maxval=1));&#xA;Bias5     = tf.Variable(tf.random_uniform(shape=[   units(20)], minval=-1, maxval=1));&#xA;Hidden5   = activation_fn(tf.matmul(Hidden4,Weight5) + Bias5);&#xA;&#xA;#out&#xA;Weight6   = tf.Variable(tf.random_uniform(shape=[20,units(1)], minval=-1, maxval=1));&#xA;Bias6     = tf.Variable(tf.random_uniform(shape=[   units(1)], minval=-1, maxval=1));&#xA;Output    = tf.sigmoid(tf.matmul(Hidden5,Weight6) + Bias6);&#xA;&#xA;Loss      = tf.reduce_sum(tf.square(Expected-Output));&#xA;Optimiser = tf.train.GradientDescentOptimizer(1e-1);&#xA;Training  = Optimiser.minimize(Loss);&#xA;&#xA;#training&#xA;Sess = tf.Session();&#xA;Init = tf.global_variables_initializer();&#xA;Sess.run(Init);&#xA;&#xA;Feed   = {Input:X, Expected:Y};&#xA;Losses = [];&#xA;Start  = time.time();&#xA;&#xA;for I in range(10000):&#xA;  if (I%1000==0):&#xA;    Lossvalue = Sess.run(Loss, feed_dict=Feed);&#xA;    Losses   += [Lossvalue];&#xA;&#xA;    if (I==0):&#xA;      print(&quot;Loss:&quot;,Lossvalue,&quot;(first)&quot;);&#xA;    else:&#xA;      print(&quot;Loss:&quot;,Lossvalue);&#xA;  #end if&#xA;&#xA;  Sess.run(Training, feed_dict=Feed);&#xA;#end for&#xA;&#xA;Lastloss = Sess.run(Loss, feed_dict=Feed);&#xA;Losses  += [Lastloss];&#xA;print(&quot;Loss:&quot;,Lastloss,&quot;(last)&quot;);&#xA;&#xA;Finish = time.time();&#xA;print(&quot;Time:&quot;,Finish-Start,&quot;seconds&quot;);&#xA;&#xA;#eval&#xA;print(&quot;\nEval:&quot;);&#xA;Evalresults = Sess.run(Output,feed_dict=Feed).tolist();&#xA;for I in range(len(Evalresults)):&#xA;  Evalresults[I] = [round(Evalresults[I][0]*Max_Y)];&#xA;#end for&#xA;print(Evalresults);&#xA;Sess.close();&#xA;&#xA;#result: diagram&#xA;print(&quot;\nLoss curve:&quot;);&#xA;pyplot.plot(Losses,&quot;-bo&quot;);&#xA;#eof&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-09-12T10:46:21.760" LastActivityDate="2019-09-12T11:36:06.910" Title="Network doesn't converge with ReLU or Leaky ReLU, but works well with sigmoid/tanh" Tags="&lt;tensorflow&gt;&lt;gradient-descent&gt;&lt;activation-function&gt;&lt;relu&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15397" PostTypeId="1" CreationDate="2019-09-12T11:01:04.070" Score="1" ViewCount="92" Body="&lt;p&gt;I am confused in understanding the maximum likelihood as a classifier. I know what is Bayesian network and I know that ML is used for estimating the parameters of models. Also, I read that there are two methods to learn the parameters of a Bayesian network: MLE and Bayesian estimator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question which confused me are the following.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Can we use ML as a classifier? For example, can we use ML to model the user's behaviors to identify the activity of them? If yes, How? What is the likelihood function that should be optimized? Should I suppose a normal distribution of users and optimize it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If ML can be used as a classifier, what is the difference between ML and BN to classify activities? What are the advantages and disadvantages of each model?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29645" LastEditorUserId="2444" LastEditDate="2020-02-09T18:01:50.703" LastActivityDate="2021-04-05T01:03:05.407" Title="Can maximum likelihood be used as a classifier?" Tags="&lt;classification&gt;&lt;bayesian-network&gt;&lt;maximum-likelihood&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15398" PostTypeId="1" AcceptedAnswerId="15402" CreationDate="2019-09-12T11:08:54.327" Score="1" ViewCount="612" Body="&lt;p&gt;Neural networks (NNs) are used as approximators in reinforcement learning (RL). To update the policy in RL, the actor network's gradients w.r.t its weights are needed. Since NN doesn't have a mathematical expression to work with, how can its derivatives be calculated?&lt;/p&gt;&#xA;" OwnerUserId="29621" LastEditorUserId="2444" LastEditDate="2019-09-13T22:06:03.563" LastActivityDate="2019-09-13T22:06:03.563" Title="How can the derivative of a neural network be calculated, given no mathematical expression?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;backpropagation&gt;&lt;actor-critic-methods&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15400" PostTypeId="2" ParentId="15396" CreationDate="2019-09-12T11:20:20.880" Score="1" Body="&lt;p&gt;I don't think it is dead ReLU units as a main cause, although they may be happening as part of the NN failing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NN architecture is too complex for the given task (too deep, too many neurons) and that means that any problems you have with other design choices will tend to get amplified. It could be that your NN is close to diverging on the given data and architecture, and that sigmoid is more resilient to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest the following changes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Dropping the learning rate, try 0.01 or even 0.001&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Normalising the two input features. NNs like to work with data that is mean 0, standard deviation 1, although there is some flexibility here, your values ranging from 0 to 6 are probably starting to cause minor problems&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Look at standard initialisation routines for weights, available within TensorFlow framework, such as Glorot uniform. Your random -1 to +1 is probably too high a range for the given network, and NNs - especially &quot;deep&quot; NNs with 3+ hidden layers - are very sensitive to how initial weights are set.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Simplify the network architecture a little. Five hidden layers and 200 neurons seems a bit much for your goal of over-fitting to this small data set. Try something like 3 hidden layers and 50 neurons.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your output layer and loss function are designed for a regression task, but you mention that the goal is to identify classes. You need a softmax layer and multiclass log loss for predicting exclusive classes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-12T11:36:06.910" LastActivityDate="2019-09-12T11:36:06.910" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15401" PostTypeId="2" ParentId="15397" CreationDate="2019-09-12T13:55:09.283" Score="0" Body="&lt;p&gt;If you read nothing else, &lt;b&gt;maximum likelihood estimate =&gt; chance that the data predicted is the data observed.&lt;/b&gt; If you have a range of points (2, 3, 4, 5, 71) your MLE is going to favour ~4.5 because of means and standard deviations. MLE speeds up finding good input parameters, usually for a different classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;a href=&quot;https://towardsdatascience.com/bayes-classifier-with-maximum-likelihood-estimation-4b754b641488&quot; rel=&quot;nofollow noreferrer&quot;&gt;Columbia University have a great example of using MLE classifiers&lt;/a&gt;, where everything is broken down into bitesize (or bytesize) chunks. Read this. Seriously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) In short, &lt;b&gt;MLE is best used for simple, univariable distributions.&lt;/b&gt; It doesn't scale well to big problems but is waaay faster than a Bayesian network for simple tasks like predicting your height based on the heights of your immediate relatives. If you want to get technical, the conditional probability network of the Bayesian model reveals insights faster than the chain multiplication of the more primitive MLE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps! &lt;/p&gt;&#xA;" OwnerUserId="28446" LastActivityDate="2019-09-12T13:55:09.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15402" PostTypeId="2" ParentId="15398" CreationDate="2019-09-12T14:13:25.160" Score="4" Body="&lt;p&gt;I think what you mean to ask is &lt;b&gt;how can differentiation occur when there's no obvious neural network function to differentiate?&lt;/b&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't worry - lots of people get confused about this, because it seems like an obvious hole in the puzzle. As mentioned by @AtillaOzgur, neural networks use partial differentiation through backpropagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, take the output of all the neurons (except the one you're about to differentiate by) as a function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jIrA9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jIrA9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above diagram represents the output of one neuron. Do this for every neuron in your network until you have a set. Let's call this set function NN. The output of NN (given all your neuron outputs) is what you'd normally plug into your RL policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then &lt;b&gt;differentiate NN by a single neuron (n) as shown:&lt;/b&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial NN}{\partial n} = \lim_{h\to0} \left(\frac{NN(\text{all other neuron outputs}, n + h) - NN(\text{all other neuron outputs}, n)}{h} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality however, it's the partial derivative of the activation function (A) with respect to the output of a single neuron (n):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial A}{\partial n}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, depending on your activation function, you just plug in your neuron output to a certain expression and you've found the value by which to update your neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps. Deep learning is definitely a field with a learning curve, but places like StackExchange are great resources to help you out.&lt;/p&gt;&#xA;" OwnerUserId="28446" LastEditorUserId="2444" LastEditDate="2019-09-13T22:04:39.023" LastActivityDate="2019-09-13T22:04:39.023" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15403" PostTypeId="1" AcceptedAnswerId="15425" CreationDate="2019-09-12T15:46:59.000" Score="1" ViewCount="45" Body="&lt;p&gt;A convolutional neural network (CNN) can easily predict the class of an object in an image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a CNN distinguish the Pisa Tower from other buildings, or Hagia Sophia from other mosques easily? If it can, how many training images can be sufficient? Do I need thousands of training images of that specific thing to distinguish it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is a term project recommendation about deep neural networks, so I need to understand its feasibility.)&lt;/p&gt;&#xA;" OwnerUserId="29650" LastEditorUserId="2444" LastEditDate="2020-06-06T00:47:04.583" LastActivityDate="2020-06-06T00:47:04.583" Title="How well can a CNN distinguish an object from its class?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15404" PostTypeId="2" ParentId="13135" CreationDate="2019-09-12T17:04:15.520" Score="1" Body="&lt;p&gt;You're correct, the return is the discounted future reward from the one iteration while the expected return is averaged over a bunch of iterations.&lt;/p&gt;&#xA;" OwnerUserId="29652" LastActivityDate="2019-09-12T17:04:15.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15405" PostTypeId="2" ParentId="15394" CreationDate="2019-09-12T18:06:22.497" Score="1" Body="&lt;p&gt;I think a HMM is overkill for this problem. You kind of have 'hidden' states, but they are very limited and dependent on the full sequence of previous states, which you probably want to avoid to make best use of the HMM's features. It also, as you rightly say, leads to a proliferation in states: each dictionary item adds as many states as it has letters to the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I would approach this it to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Trie&quot; rel=&quot;nofollow noreferrer&quot;&gt;trie&lt;/a&gt; which contains your dictionary, and then you traverse the trie as the user types characters. At each point you have a sub-trie which gives you the possible completions of the word. You could even (if available) augment this with probabilities to guess the most likely word (though this does not take into account the previous word in the sentence, which might be more useful).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to learn about HMMs, one possible application would be a weather forecast (I thought this was an example &lt;a href=&quot;http://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lawrence Rabiner uses in his tutorial&lt;/a&gt;, but it's actually on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMM Wikipedia page&lt;/a&gt;). Or, if you want to work with texts, use parts-of-speech tagging. You want to have something where each observation can belong to several possible states (eg &lt;em&gt;light&lt;/em&gt; can belong to &lt;code&gt;adj&lt;/code&gt;, &lt;code&gt;noun&lt;/code&gt;, and &lt;code&gt;verb&lt;/code&gt;). Here you would have the words as observations and the part-of-speech tags as states, which keeps the model at a reasonably small size (and is thus easier to train).&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-09-12T18:06:22.497" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15406" PostTypeId="2" ParentId="10975" CreationDate="2019-09-12T18:56:07.050" Score="1" Body="&lt;p&gt;I can comment on several properties of MSE and related losses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you mentioned MSE (aka &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt;-loss) is convex which is a great property in optimization in which one can find a single global optimum. MSE is used in linear and non-linear least squares problems which form the basis of many widely used statistical methods. I would imagine the math and implementation would be more difficult if one would use a higher-order loss (e.g. &lt;span class=&quot;math-container&quot;&gt;$x^3$&lt;/span&gt;) and that would also prove to be futile because MSE already possesses great statistical and optimization properties on its own.&lt;br&gt;&#xA;Another important aspect, one wouldn't use higher-order loss functions in regression is because it would be extremely prone to outliers. MSE on its own would weigh the outliers much more than l1-loss would! And in real world data there is always noise and outliers present. In comparison l1 loss is more difficult in optimization, one reason for which is it's not differentiable at zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other interesting losses you might want to read about are &lt;a href=&quot;https://medium.com/@montjoile/l0-norm-l1-norm-l2-norm-l-infinity-norm-7a7d18a4f40c&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$l_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l_{inf}$&lt;/span&gt; loss&lt;/a&gt;, all of which have their own trade-offs in optimization-sense.&lt;/p&gt;&#xA;" OwnerUserId="28608" LastActivityDate="2019-09-12T18:56:07.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15407" PostTypeId="2" ParentId="8333" CreationDate="2019-09-12T19:42:42.693" Score="0" Body="&lt;p&gt;I think in the specific case you described if the representatives are now compatible you would move the lower fitness representative into the higher fitness representatives species, then with the remainder of the lower representatives species (all other genomes that didnt have compat distances close enough to move into the other species) you would just randomly pick a new representative and carry on with the process.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-09-12T19:42:42.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15408" PostTypeId="1" CreationDate="2019-09-12T20:29:47.870" Score="3" ViewCount="751" Body="&lt;p&gt;Say I have x,y data connected by a function with some additional parameters (a,b,c):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ y = f(x ; a, b, c) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now given a set of data points (x and y) I want to determine a,b,c. If I know the model for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, this is a simple curve fitting problem. What if I don't have &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; but I do have lots of examples of y with corresponding a,b,c values? (Or alternatively &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is expensive to compute, and I want a better way of guessing the right parameters without a brute force curve fit.) Would simple machine-learning techniques (e.g. from sklearn) work on this problem, or would this require something more like deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example generating the kind of data I'm talking about:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np                                                                                                                                                                                                                           &#xA;import matplotlib.pyplot as plt                                                                                                                                                                                                              &#xA;&#xA;Nr = 2000                                                                                                            &#xA;Nx = 100                                                                                                             &#xA;x = np.linspace(0,1,Nx)                                                                                              &#xA;&#xA;f1 = lambda x, a, b, c : a*np.exp( -(x-b)**2/c**2) # An example function                                             &#xA;f2 = lambda x, a, b, c : a*np.sin( x*b + c)        # Another example function                                        &#xA;prange1 = np.array([[0,1],[0,1],[0,.5]])                                                                             &#xA;prange2 = np.array([[0,1],[0,Nx/2.0],[0,np.pi*2]])                                                                   &#xA;#f, prange = f1, prange1                                                                                             &#xA;f, prange = f2, prange2                                                                                              &#xA;&#xA;data = np.zeros((Nr,Nx))                                                                                             &#xA;parms = np.zeros((Nr,3))                                                                                             &#xA;for i in range(Nr) :                                                                                                 &#xA;    a,b,c = np.random.rand(3)*(prange[:,1]-prange[:,0])+prange[:,0]                                                  &#xA;    parms[i] = a,b,c                                                                                                 &#xA;    data[i] = f(x,a,b,c) + (np.random.rand(Nx)-.5)*.2*a                                                              &#xA;&#xA;plt.figure(1)                                                                                                        &#xA;plt.clf()                                                                                                            &#xA;for i in range(3) :                                                                                                  &#xA;    plt.title('First few rows in dataset')                                                                           &#xA;    plt.plot(x,data[i],'.')                                                                                          &#xA;    plt.plot(x,f(x,*parms[i]))                                                                                       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/msnzH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/msnzH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given &lt;code&gt;data&lt;/code&gt;, could you train a model on half the data set, and then determine the a,b,c values from the other half?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been going through some sklearn tutorials, but I'm not sure any of the models I've seen apply well to this type of a problem. For the guassian example I could do it by extracting features related to the parameters (e.g. first and 2nd moments, %5 and .%95 percentiles, etc.), and feed those into an ML model that would give good results, but I want something that would work more generally without assuming anything about &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; or its parameters.&lt;/p&gt;&#xA;" OwnerUserId="21099" LastEditorUserId="21099" LastEditDate="2019-09-16T19:08:07.870" LastActivityDate="2019-09-16T19:08:07.870" Title="Can ML be used to curve fit data based on dataset of example fits?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;scikit-learn&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15409" PostTypeId="1" AcceptedAnswerId="15423" CreationDate="2019-09-12T21:18:57.923" Score="4" ViewCount="141" Body="&lt;p&gt;I am playing around with neural networks in Tensorflow and I figured an interesting test would be whether I can write a calculator using a Tensorflow Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started with simple addition and it kinda worked (so given 2, 4 it would get around 5.9 or 6.1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I wanted to add the ability to calculate using &quot;+&quot;, &quot;-&quot;, and &quot;*&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code I came up with in the end:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import tensorflow as tf&#xA;from random import randrange&#xA;&#xA;def generate_input(size):&#xA;    nn_input = []&#xA;    for i in range(0,size):&#xA;        symbol = float(randrange(3))&#xA;        nn_input.append([&#xA;                float(randrange(1000)),&#xA;                float(randrange(1000)),&#xA;                1 if symbol == 0 else 0,&#xA;                1 if symbol == 1 else 0,&#xA;                1 if symbol == 2 else 0,&#xA;                ])&#xA;    return nn_input&#xA;&#xA;def generate_output(input_data):&#xA;    return [[generate_single_output(i)] for i in input_data]&#xA;&#xA;def generate_single_output(input_data):&#xA;    plus = input_data[2]&#xA;    minus = input_data[3]&#xA;    multiplication = input_data[4]&#xA;&#xA;    if (plus):&#xA;        return input_data[0] + input_data[1]&#xA;&#xA;    if (minus):&#xA;        return input_data[0] - input_data[1]&#xA;&#xA;    if (multiplication):&#xA;        return input_data[0] * input_data[1]&#xA;&#xA;def user_input_to_nn_input(user_input):&#xA;    symbol = user_input[1]&#xA;    return np.array([[&#xA;            float(user_input[0]),&#xA;            float(user_input[2]),&#xA;            1 if symbol == '+' else 0,&#xA;            1 if symbol == '-' else 0,&#xA;            1 if symbol == '*' else 0,&#xA;            ]])&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    model = tf.keras.models.Sequential([&#xA;        tf.keras.layers.Dense(64, activation='relu', input_shape=(5,)),&#xA;        tf.keras.layers.Dense(64, activation='relu'),&#xA;        tf.keras.layers.Dense(1),&#xA;        ])&#xA;&#xA;    model.compile(tf.keras.optimizers.RMSprop(0.001), loss=tf.keras.losses.MeanSquaredError())&#xA;&#xA;&#xA;    input_data = np.array(generate_input(10000))&#xA;    output_data = np.array(generate_output(input_data))&#xA;&#xA;    model.fit(input_data, output_data, epochs=20)&#xA;&#xA;    while True:&#xA;        user_calculation = input(&quot;Enter expression (e.g. 2 + 3):&quot;)&#xA;        user_input = user_calculation.split()&#xA;        nn_input = user_input_to_nn_input(user_input)&#xA;        print(model.predict(nn_input)[0][0])&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea is built on this tutorial: &lt;a href=&quot;https://www.tensorflow.org/tutorials/keras/basic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/keras/basic_regression&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input is 5 fields: number 1, number 2, plus?, minus?, multiplication?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the last 3 are simply 1 or 0 depending on whether that is the calculation I am trying to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an output for say [1,4,1,0,0] I would expect 1 + 4 = 5&#xA;for [1,4,0,1,0] I would expect 1 - 4 = -3 etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some reason though the numbers I am getting are completely off and seem random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I am trying to understand what I went wrong?&#xA;The data being input to the NN seems correct and the model is based on the model used in the tutorial I quoted (and the problems seem fairly similar so I expect if one would work the other would too).&lt;/p&gt;&#xA;" OwnerUserId="27615" LastActivityDate="2019-09-23T07:56:46.567" Title="Why isn't my Neural Network based calculator working?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15410" PostTypeId="5" CreationDate="2019-09-12T21:43:31.767" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gamification&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Gamification&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-09-12T21:43:31.767" LastActivityDate="2019-09-12T21:43:31.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15411" PostTypeId="4" CreationDate="2019-09-12T21:43:31.767" Score="0" Body="The application of elements of game play to address real-world problems.  For questions on utilizing this process in the field of AI." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-09-12T21:43:31.767" LastActivityDate="2019-09-12T21:43:31.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15412" PostTypeId="2" ParentId="11345" CreationDate="2019-09-12T22:05:31.317" Score="0" Body="&lt;p&gt;well if you want to just train genomeN against genomeN-1 then you just need to loop through genomes using indexing and start at 1 so you can always do genome1 = genomes[ix-1] and genome2 = genomes[ix] and then modify your fitness evaluation code to run both at the same time and applies a fitness function to each. Also maybe you iterate by twos so that you only eval each net once.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2020-02-10T15:23:01.273" LastActivityDate="2020-02-10T15:23:01.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15413" PostTypeId="1" CreationDate="2019-09-13T04:21:02.757" Score="1" ViewCount="16" Body="&lt;p&gt;How good would a &lt;a href=&quot;https://en.wikipedia.org/wiki/Content-based_image_retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;CBIR system&lt;/a&gt; trained on a dataset, for example, DELF, trained on the Google Landmarks dataset, perform when evaluated on a contextually different dataset such as the WANG or the COREL dataset without retraining?&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="2444" LastEditDate="2019-09-13T12:49:55.920" LastActivityDate="2019-09-13T12:49:55.920" Title="CBIR Evaluation on contextually different data" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;image-processing&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15415" PostTypeId="1" CreationDate="2019-09-13T09:22:49.877" Score="1" ViewCount="21" Body="&lt;p&gt;Let's say that we have a test data set with &lt;span class=&quot;math-container&quot;&gt;$20,000$&lt;/span&gt; observations for which we want to make a binary prediction for. When we apply our best trained model to this data set (e.g. logistic regression with threshold = 0.5, data_size = 4000 rows, 5 fold cv), only about &lt;span class=&quot;math-container&quot;&gt;$1 \%$&lt;/span&gt; of the predictions are positive. That is, &lt;span class=&quot;math-container&quot;&gt;$p(\text{positive}) \geq 0.5$&lt;/span&gt; is true only for about &lt;span class=&quot;math-container&quot;&gt;$1 \%$&lt;/span&gt; of the predictions. We expect many more positives since the recall of the positive class of the best trained model is about &lt;span class=&quot;math-container&quot;&gt;$40 \%$&lt;/span&gt;. If we manually lower the threshold to &lt;span class=&quot;math-container&quot;&gt;$0.45$&lt;/span&gt;, then about &lt;span class=&quot;math-container&quot;&gt;$10 \%$&lt;/span&gt; of the predictions are positive. Assume that the &lt;span class=&quot;math-container&quot;&gt;$20,000$&lt;/span&gt; observations come from the same distribution as the training/validation data and are independent samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why would a a model with decent recall for the positive class predict very few positives in the out of sample data? &lt;/li&gt;&#xA;&lt;li&gt;If (1) is true, then is it appropriate to lower the threshold for positive (e.g. &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0.45$&lt;/span&gt;) to increase the number of predicted positives in the test set?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28575" LastActivityDate="2019-09-13T09:22:49.877" Title="Improving Recall of a Certain Class" Tags="&lt;prediction&gt;&lt;performance&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15416" PostTypeId="1" AcceptedAnswerId="15418" CreationDate="2019-09-13T09:42:37.680" Score="2" ViewCount="73" Body="&lt;p&gt;I often read &lt;em&gt;&quot;the performance of the system is satisfactory&quot;&lt;/em&gt; or &lt;em&gt;&quot; when your model is satisfactory&quot;&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what does it mean in the context of Machine Learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any clear and/or generic criteria for Machine Learning model to be satisfactory for commercial use? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is decision what model to choose or whether additional model adjustments or improvements are needed based on data scientist experience, customer satisfaction or benchmarking academic or market competition results?&lt;/p&gt;&#xA;" OwnerUserId="28605" LastEditorUserId="1671" LastEditDate="2019-10-15T19:17:41.063" LastActivityDate="2019-10-15T19:17:41.063" Title="What are criteria for ML model to be satisfactory for commercial use?" Tags="&lt;machine-learning&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15417" PostTypeId="1" AcceptedAnswerId="15420" CreationDate="2019-09-13T09:50:11.293" Score="2" ViewCount="193" Body="&lt;p&gt;&lt;em&gt;Can an RL algorithm trained in one environment be successful in a different one?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I train a model to go through one labyrinth, could this model also go through a different but similar labyrinth or would it need a new training process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By similar, I mean like these two:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hVGOA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hVGOA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jnEKx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jnEKx.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But with this one being not similar:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4koPq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4koPq.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2019-09-13T12:23:55.907" LastActivityDate="2019-09-13T12:23:55.907" Title="Can an RL algorithm trained in one environment be successful in a different one?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15418" PostTypeId="2" ParentId="15416" CreationDate="2019-09-13T09:55:22.770" Score="2" Body="&lt;p&gt;The answer is &quot;when it works well enough to perform the task that you have set it&quot;.&#xA;It is a good idea to set your performance criteria in advance so that you can clearly identify the goal that you are trying to achieve and also so that you will know if the model is likely to be successful or not. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-09-13T09:55:22.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15419" PostTypeId="2" ParentId="15368" CreationDate="2019-09-13T10:31:15.020" Score="0" Body="&lt;p&gt;I found out the curvy zigzag green line is not polynomial as if it were polynomial, a vertical line won't cut that curvy line more than 1 time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's the combination of straight lines (linear separation) of multiple neurons in the same layer. So it's linear separation ('linear' by previous_layer_output*weight, 'separation' by activation function), at multiple nodes.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-09-13T10:31:15.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15420" PostTypeId="2" ParentId="15417" CreationDate="2019-09-13T10:42:58.610" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can an RL algorithm trained in one environment be succesfull on a different one? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Strictly the answer here is &quot;no&quot;. You train an agent to solve a single environment. If a second environment is similar enough, you can probably re-train the agent on the new environment with little or no changes to the agent, and to get results just as good in the second agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, you could even start with the trained agent and &quot;fine tune&quot; it to the new environment. That probably would not work well with the labyrinth example though, especially if the start and end points are moved.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Example: If I train a model to go through one labyrinth, could this model also go through different but similar labyrinth or would it need a new training process?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This could be subtly different. It is down to how you define the environment. It is possible to define an environment not as &quot;this maze&quot; but for example &quot;all possible mazes in a 30x20 grid pattern&quot;. To do so, you need to make the maze configuration part of the environment's &lt;em&gt;state&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expanding from a singular environment example to all similar environments has a cost. In the labyrinth example, this is significant. A single 30x20 maze has 600 states, which is a trivial problem in RL. All possible mazes of the same size has closer to &lt;span class=&quot;math-container&quot;&gt;$2^{1200}$&lt;/span&gt; states, which is much larger and requires different kinds of approaches, different RL methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would expect to train on many example mazes (typically a randomly generated new one for each episode) and use some kind of generalisation - e.g. a neural network, maybe a CNN due to the grid design - when handling the value function or policy function. Training time for a single maze will be under a second, and the policy will be perfect. Training time for all possible mazes measured in hours or days and the policy will still get things wrong from time-to-time.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-09-13T10:42:58.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15422" PostTypeId="1" CreationDate="2019-09-13T12:04:54.023" Score="2" ViewCount="338" Body="&lt;p&gt;I have a regression MLP network with all input values between 0 and 1, and am using MSE for the loss function. The minimum MSE over the validation sample set comes to 0.019. So how to express the 'accuracy' of this network in 'lay' terms? If RMSE is 'in the units of the quantity being estimated', does this mean we can say: &quot;The network is on average (1-SQRT(0.019))*100 =  86.2% accurate&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in the validation data set, there are three 'extreme' expected values. The lowest MSE results in predicted values closer to these three values, but not as close to all the other values, whereas a slightly higher MSE results in the opposite - predicted values further from the 'extreme' values but more accurate relative to all other expected values (and this outcome is actually preferred in the case I'm dealing with). I assume this can be explained by RMSE's sensitivity to outliers? &lt;/p&gt;&#xA;" OwnerUserId="27920" LastEditorUserId="32410" LastEditDate="2021-02-23T12:36:19.330" LastActivityDate="2021-02-23T12:36:19.330" Title="How to express accuracy of a regression ANN that uses MSE loss function?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;regression&gt;&lt;accuracy&gt;&lt;mean-squared-error&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15423" PostTypeId="2" ParentId="15409" CreationDate="2019-09-13T12:39:50.940" Score="6" Body="&lt;p&gt;A neural network is not good at selecting a function based on those 3 input parameters, because of the way a neuron is setup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you should do is either make a neural network for each operation, or use different input neurons for each operation. E.g. 2 input neurons for the addition operation, 2 for the multiplication, and 2 for the minus. 6 inputs in total of which 4 will always be 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will make it easier for the neural network to calculate the result.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastEditorUserId="25982" LastEditDate="2019-09-23T07:56:46.567" LastActivityDate="2019-09-23T07:56:46.567" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15424" PostTypeId="2" ParentId="12455" CreationDate="2019-09-13T13:45:57.517" Score="0" Body="&lt;p&gt;If the compatibility distances are coming out to be &gt; float.max that tells me something may be wrong with that calculation, I would suggest setting break points and debugging that code, I usually have my threshold set 5-10 and can noticeably tell when a change in species size when i move the threshold to something like 10 &lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="20044" LastEditDate="2019-09-13T14:11:35.257" LastActivityDate="2019-09-13T14:11:35.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15425" PostTypeId="2" ParentId="15403" CreationDate="2019-09-13T14:48:42.590" Score="0" Body="&lt;p&gt;To help you understand the feasibility of your project these posts could be a good start:&#xA;&lt;a href=&quot;https://datascience.stackexchange.com/questions/13181/how-many-images-per-class-are-sufficient-for-training-a-cnn&quot;&gt;https://datascience.stackexchange.com/questions/13181/how-many-images-per-class-are-sufficient-for-training-a-cnn&lt;/a&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/13181/how-many-images-per-class-are-sufficient-for-training-a-cnn&quot;&gt;enter link description here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/226672/how-few-training-examples-is-too-few-when-training-a-neural-network&quot;&gt;https://stats.stackexchange.com/questions/226672/how-few-training-examples-is-too-few-when-training-a-neural-network&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, the short answer would be it depends. It depends how precise you want to be, what is the difficulty of the task, the infrastructure you have for the training etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the images, you should not worry, you could start with the ImageNet dataset:&#xA;For construction and buildings:&#xA;&lt;a href=&quot;http://www.image-net.org/explore?wnid=n04341686&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.image-net.org/explore?wnid=n04341686&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For mosquees:&#xA;&lt;a href=&quot;http://www.image-net.org/synset?wnid=n03788195&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.image-net.org/synset?wnid=n03788195&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then use data augmentation techniques to enhance the size of you training set. Here is a library I have used in the past which helped me greatly to achieve this task: &lt;a href=&quot;https://github.com/aleju/imgaug&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aleju/imgaug&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="28010" LastActivityDate="2019-09-13T14:48:42.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15428" PostTypeId="2" ParentId="15388" CreationDate="2019-09-13T19:43:37.107" Score="2" Body="&lt;p&gt;There are different actor-critic (AC) algorithms with different convergence guarantees. For example, AC algorithms where the critic is &lt;em&gt;tabular&lt;/em&gt; have different convergence guarantees than AC algorithms where the critic is a neural network (function approximation). Most convergence proofs assume that the actor and the critic operate at different time scales, but, for example, in the paper &lt;a href=&quot;http://www.jmlr.org/papers/volume11/dicastro10a/dicastro10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Convergent Online Single Time Scale Actor-Critic Algorithm&lt;/a&gt; (2010) this assumption is not made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://papers.nips.cc/paper/3258-incremental-natural-actor-critic-algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Incremental Natural Actor-Critic Algorithms&lt;/a&gt; (2007), the authors propose four different AC algorithms that use function approximation (neural networks) to represent the critic. Three of these proposed AC algorithms are based on &lt;a href=&quot;https://papers.nips.cc/paper/2073-a-natural-policy-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural policy gradients&lt;/a&gt;. In section 6 of the extended and more technical version of the mentioned paper, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.516.4121&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Actor-Critic Algorithm&lt;/a&gt; (2007), the authors prove the convergence of the parameters of the policy and value function to a &lt;strong&gt;local maximum&lt;/strong&gt; of an &lt;em&gt;objective&lt;/em&gt; (or &lt;em&gt;performance&lt;/em&gt;) &lt;em&gt;function&lt;/em&gt;, which corresponds to the &lt;em&gt;average reward&lt;/em&gt; (equation 2) plus a measure of the temporal-difference (TD) error inherent in the function approximation.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-13T19:49:50.597" LastActivityDate="2019-09-13T19:49:50.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15429" PostTypeId="2" ParentId="13088" CreationDate="2019-09-13T20:09:36.950" Score="3" Body="&lt;p&gt;A &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.516.4121&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;stationary policy&lt;/strong&gt;&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;, is a policy that does not change over time, that is, &lt;span class=&quot;math-container&quot;&gt;$\pi_t = \pi, \forall t \geq 0$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; can either be a function, &lt;span class=&quot;math-container&quot;&gt;$\pi: S \rightarrow A$&lt;/span&gt; (a deterministic policy), or a conditional density, &lt;span class=&quot;math-container&quot;&gt;$\pi(A \mid S)$&lt;/span&gt; (a stochastic policy). A &lt;strong&gt;non-stationary policy&lt;/strong&gt; is a policy that is &lt;strong&gt;not&lt;/strong&gt; stationary. More precisely, &lt;span class=&quot;math-container&quot;&gt;$\pi_i$&lt;/span&gt; may not be equal to &lt;span class=&quot;math-container&quot;&gt;$\pi_j$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i \neq j \geq 0$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; are thus two different time steps.&lt;/p&gt;&#xA;&lt;p&gt;There are problems where a stationary optimal policy is guaranteed to exist. For example, in the case of a stochastic (there is a probability density that models the dynamics of the environment, that is, the transition function and the reward function) and discrete-time Markov decision process (MDP) with finite numbers of states and actions, and bounded rewards, where the &lt;em&gt;objective&lt;/em&gt; is the long-run average reward, a stationary optimal policy exists. The proof of this fact is in the book &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316887&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Markov Decision Processes: Discrete Stochastic Dynamic Programming&lt;/em&gt;&lt;/a&gt; (1994), by Martin L. Puterman, which &lt;em&gt;apparently&lt;/em&gt; is not freely available on the web.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="36821" LastEditDate="2021-02-25T00:17:37.457" LastActivityDate="2021-02-25T00:17:37.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15430" PostTypeId="2" ParentId="15422" CreationDate="2019-09-14T00:50:39.493" Score="2" Body="&lt;p&gt;You can not use error to reliably measure accuracy. Error is best used as a measure of how fast the model is currently learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, using different loss functions (cross entorpy vs MSE) results in massively different values for the error at similar accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also considering this, an error of 0.0000000001 quite often has lower validation set accuracy then and error of 0.1, as the prior is likely over trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for you second question, yes this is because MSE has a huge bias towards outliers. I have personally found regression networks to struggle in most circumstances, so if it is at all possible to turn the network into a classifier, you may see an improvement.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-09-14T00:50:39.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15431" PostTypeId="1" AcceptedAnswerId="15436" CreationDate="2019-09-14T08:58:17.670" Score="3" ViewCount="217" Body="&lt;p&gt;When you train a model using Monte Carlo-based learning the state and action taken at each step is recorded, and then at some point an end state is reached and the agent receives some reward - what do you do at that point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say there were 100 steps taken to reach this final reward state, would you update the full rollout of those 100 state/action/rewards and then begin the next episode, or do you then 'bubble up' that final reward to the previous states and update on those as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Process an update for the full 100 experiences. Can either stop here, or...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Bubble up the final reward to the 99th step and process an update for the 99 state/action/reward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Bubble up the final reward to the 98th step and process an update for the 98 state/action/reward.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;and so on right the way to the first step...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or, do you just process an update for the full 100-step roll-out and that's it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or perhaps these are two different approaches?  Is there a situation where you'd one rather than the other?&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="2444" LastEditDate="2021-01-02T12:46:03.727" LastActivityDate="2021-01-02T12:46:03.727" Title="In Monte Carlo learning, what do you do when an end state is reached, after having recorded the previously visited states and taken actions?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15432" PostTypeId="2" ParentId="15422" CreationDate="2019-09-14T10:55:53.003" Score="1" Body="&lt;p&gt;Just as a general remark, notice that technically we don't use the term &quot;accuracy&quot; for regression settings, such as yours - only for classification ones.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If RMSE is 'in the units of the quantity being estimated', does this mean we can say: &quot;The network is on average (1-SQRT(0.019))*100 = 86.2% accurate&quot;? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of the RMSE, as you have correctly quoted, is that it is in the same units with your predicted quantity; so, if this quantity is, say, USD, you can say (to the &lt;em&gt;business&lt;/em&gt; user) that the error of the model is 0.019 USD, and this can be perfectly fine by itself. But you cannot convert it to a percentage - it would be meaningless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If required to give the performance of a &lt;em&gt;regression&lt;/em&gt; model in a percentage, your best option would be the Mean Absolute Percentage Error (&lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_absolute_percentage_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAPE&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="11539" LastActivityDate="2019-09-14T10:55:53.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15433" PostTypeId="1" CreationDate="2019-09-14T23:48:45.457" Score="3" ViewCount="24" Body="&lt;p&gt;I know how to train a NN for recognizing handwritten digits (e.g. using the MNIST database).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering how to accomplish the same &quot;online&quot;, which is &lt;strong&gt;during the process of writing&lt;/strong&gt; e.g. I'm writing down a digit, and during the process and while keeping &quot;the pen down&quot;, the NN recognizes the digit.&#xA;To make it simpler, I could assume one stroke-order only for each digit (e.g. each digit is written with a certain order of strokes, like for digit 1 is just a vertical line going up to down).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the most suitable NN for the purpose and how to accomplish this?&lt;/p&gt;&#xA;" OwnerUserId="13087" LastActivityDate="2019-09-14T23:48:45.457" Title="Handwritten digits recognition during the process of writing" Tags="&lt;neural-networks&gt;&lt;handwritten-characters&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15434" PostTypeId="1" AcceptedAnswerId="15553" CreationDate="2019-09-15T02:22:26.187" Score="3" ViewCount="31" Body="&lt;p&gt;I have a dataset of images with 9 different classes. However, there are different categories with the same type of associated image and only can be differentiated with an associated matrix in my specific problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to train a neural network with the images and the associated matrix as inputs. What type of architecture is good to use? Or where can I find bibliography about it?&lt;/p&gt;&#xA;" OwnerUserId="29693" LastEditorUserId="2444" LastEditDate="2019-09-15T11:38:01.883" LastActivityDate="2019-09-20T03:23:32.823" Title="Image classification with an associated matrix" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15435" PostTypeId="1" CreationDate="2019-09-15T03:46:14.700" Score="1" ViewCount="34" Body="&lt;p&gt;Can I use any of the existing cognitive services from Microsoft, IBM or AWS to recognize the name and gender of characters (actors) from a story (novel) book (written in plain text or pdf format)?&lt;/p&gt;&#xA;" OwnerUserId="29695" LastEditorUserId="1671" LastEditDate="2019-09-16T19:37:50.120" LastActivityDate="2019-09-16T19:37:50.120" Title="Can existing cognitive services be used for name and gender recognition in a story?" Tags="&lt;software-evaluation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15436" PostTypeId="2" ParentId="15431" CreationDate="2019-09-15T09:46:43.093" Score="2" Body="&lt;p&gt;I am assuming you are asking about Monte Carlo simulation for value estimates, perhaps as part of a Monte Carlo control learning agent.&lt;/p&gt;&#xA;&lt;p&gt;The basic approach of all value-based methods is to estimate an expected &lt;em&gt;return&lt;/em&gt;, often the action value &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; which is a sum of expected future reward from taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Monte Carlo methods take a direct and simple approach to this, which is to run the environment to the end of an episode and measure the return. This return is a sample out of all possible returns, so it can just be averaged with other observed returns to obtain an estimate. A minor complication is that the return depends on the current policy, and in control scenarios that will change, so the average needs to be recency-weighted for control e.g. using a fixed learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; in an update like &lt;span class=&quot;math-container&quot;&gt;$Q(s,a) \leftarrow Q(s,a) + \alpha(G - Q(s,a))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given this, you can run pretty much any approach that calculates the returns from observed state/action pairs. You will find that the &amp;quot;bubble up&amp;quot; approach is used commonly - the process usually termed &lt;em&gt;backing up&lt;/em&gt; - working backwards from the end of the episode.&lt;/p&gt;&#xA;&lt;p&gt;If you have an episode from &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt; and records of states, actions, rewards &lt;span class=&quot;math-container&quot;&gt;$s_0, a_0, r_1, s_1, a_1, r_2, s_2, a_2, r_3 . . . s_{T-1}, a_{T-1}, r_T, s_T$&lt;/span&gt; (note indexing, reward follows state/action, there is no &lt;span class=&quot;math-container&quot;&gt;$r_0$&lt;/span&gt; and no &lt;span class=&quot;math-container&quot;&gt;$a_T$&lt;/span&gt;), then the following algorithm could be used to calculate individual returns &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g \leftarrow 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$t = T-1$&lt;/span&gt; down to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad g \leftarrow r_{t+1} + \gamma g$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad Q(s_t,a_t) \leftarrow Q(s_t,a_t) + \alpha(g - Q(s_t,a_t))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This working backwards is an efficient way to process rewards and assign them with discounting to action values for all state, action pairs observed in the episode.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Or perhaps these are two different approaches?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It would be valid to calculate only the return for the first state/action, and randomly select state/actions to start from (called exploring starts). Or in fact take any arbitrary set of estimates generated this way. You don't have to use all return estimates, but you do need to have an algorithm that is guaranteed to update values of all state/action pairs in the long term.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there a situation where you'd one rather than the other?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Most usually you will see the backed up return estimates to all observed state/action pairs, as this is more sample efficient, and Monte Carlo is already a high variance method that requires lots of samples to get good estimates (especially for early state/action pairs at the start of long episodes).&lt;/p&gt;&#xA;&lt;p&gt;However, if you work with function approximation such as neural networks, you need to avoid feeding in correlated data to learn from. A sequence of state/action pairs from a single episode are going to be correlated. There are a few ways to avoid that, but one simple approach could be to take just one sample from each rollout. You &lt;em&gt;might&lt;/em&gt; do this if rollouts could be run very fast, possibly simulated on computer. But other alternatives may be better - for instance put all the state, action, return values into a data set, shuffle it after N episodes and learn from everything.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-09-17T09:40:11.910" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15437" PostTypeId="1" CreationDate="2019-09-15T10:10:01.060" Score="2" ViewCount="21" Body="&lt;p&gt;I'm trying to get a grasp on scalability of clustering algorithms, and have a toy example in mind. Let's say I have around a million or so songs from &lt;span class=&quot;math-container&quot;&gt;$50$&lt;/span&gt; genres. Each song has characteristics - some of which are common across all or most genres, some of which are common to only a few genres and some of which are genre-specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Common attributes could be something like song duration, artist, label, year, album, key, etc. Genre-specific attributes could be like lead guitarist, trombone player, conductor, movie name (in case of movie soundtracks), etc. Assume that there are, say, &lt;span class=&quot;math-container&quot;&gt;$2000$&lt;/span&gt; attributes across all possible genres.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim is to identify attributes that characterize subgenres of these genres. So of course, let's say for rock I can just collect all the attributes for all rock songs, but even that set of attributes may be too broad to characterize the rock genre - maybe there are some that are specific to subgenres and so I won't have the desired level of granularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for the purpose of this example, I'm &lt;strong&gt;not assuming that I already know the subgenres a priori.&lt;/strong&gt; For example, I'm not going to categorize songs into subgenres like post rock, folk rock, etc. and then pick out attributes characterizing them. I want to &lt;em&gt;discover&lt;/em&gt; subgenres on the basis of clustering, if that makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nutshell, about a million songs belonging to &lt;span class=&quot;math-container&quot;&gt;$50$&lt;/span&gt; genres and all songs collectively have &lt;span class=&quot;math-container&quot;&gt;$2000$&lt;/span&gt; attributes. So for each song I'll make a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{2000}$&lt;/span&gt; - each dimension corresponds to an attribute. If that attribute is present for that song, the corresponding element of the vector is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, otherwise &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (e.g. a jazz-related attribute will be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for a rock song). &lt;em&gt;Now I want to do &lt;strong&gt;genre-wise&lt;/strong&gt; clustering&lt;/em&gt;. &lt;strong&gt;For each genre, not only do I want to cluster songs of that genre into groups, but I also want to get an idea which attributes are the most important to characterize individual groups.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the basis of this clustering, I can identify subgenres (e.g. of rock music) that I can characterize using a subset of the &lt;span class=&quot;math-container&quot;&gt;$2000$&lt;/span&gt; attributes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is: &lt;em&gt;is there a better way to initialize the problem than forming 2000-dimensional vectors of ones and zeros?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, &lt;em&gt;given the vast number of dimensions and examples, what clustering methods could be tried? From what I've surveyed, there are graph-based clustering methods, hierarchical, density-based, spectral clustering and so on. Which of these would be best for the toy example? I've heard that one can project the points on to a lower-dimensional subspace, then do clustering. But I also want which attributes define different clusters. Since attributes are encoded in the dimensions, with dimensionality reduction techniques I'll lose information about the attributes.&lt;/em&gt; So now what?&lt;/p&gt;&#xA;" OwnerUserId="27548" LastActivityDate="2019-09-15T10:10:01.060" Title="Clustering of very high dimensional data and large number of examples without losing info in dimensions" Tags="&lt;clustering&gt;&lt;dimensionality-reduction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15439" PostTypeId="1" CreationDate="2019-09-15T14:22:12.050" Score="1" ViewCount="263" Body="&lt;p&gt;I'm interested in modeling a Siamese network for facial verification. I've already written a simple working model that inputs feature vectors generated from two CNNs with shared weights then outputs a similarity score (euclidean distance.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://keras.io/examples/mnist_siamese/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a similar model found within the Keras documentation (this Siamese network joins two networks comprised of fully connected layers) The model also uses a euclidean distance metric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The threshold used in that example when computing the accuracy of the model is 0.5. The similarity scores generated by the model run from that training script roughly ranges from 0 to 1.68. My model outputs scores ranging from 0 to 1.96.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suppose that the choice of threshold when working with a similarity metric could be determined by finding the threshold value that maximizes an appropriate metric (e.g. F1 score) on a test set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now when it comes to parameter tuning using a validation set - done so to choose the appropriate optimization and regularization parameters and model architecture to generate scores in the first place, how do I determine what value to set as the threshold? This threshold would be important to calculating the performance metric of each model generated during the parameter search - needed to determine what final model architecture and parameter set to use when training. Also, would the method for choosing the threshold change should I choose a different distance metric (e.g. cosine similarity)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already done a parameter search using an arbitrarily-set threshold of 0.5. I'm unsure if this would reflect best practices or if it ought to be adjusted when using a different distance metric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help offered. Please let me know if any more details on my part are necessary to facilitate a better discussion in this thread.&lt;/p&gt;&#xA;" OwnerUserId="29702" LastEditorUserId="29702" LastEditDate="2019-09-16T11:54:18.197" LastActivityDate="2019-09-16T11:54:18.197" Title="Threshold selection for Siamese network hyper-parameter tuning" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;facial-recognition&gt;&lt;hyperparameter-optimization&gt;&lt;similarity&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15440" PostTypeId="2" ParentId="11244" CreationDate="2019-09-15T14:38:46.567" Score="1" Body="&lt;p&gt;Suppose that &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; are &lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_space&quot; rel=&quot;nofollow noreferrer&quot;&gt;metric spaces&lt;/a&gt;. A metric space is a set equipped with a &lt;em&gt;metric&lt;/em&gt;, which is a function that defines the intuitive notion of &lt;em&gt;distance&lt;/em&gt; between the elements of the set. For example, the set of real numbers, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;, equipped with the &lt;a href=&quot;https://math.stackexchange.com/a/1893295/168764&quot;&gt;metric induced by the absolute value function (which is a norm)&lt;/a&gt;. More precisely, the metric &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; can be defined as &lt;span class=&quot;math-container&quot;&gt;$$d(x,y)=|x - y|, \forall x, y \in \mathbb{R} \tag{1}\label{1}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; be a function from the metric space &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to the metric space &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$f: X \rightarrow Y$&lt;/span&gt;. Then, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a &lt;em&gt;non-expansive map&lt;/em&gt; (also known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;metric map&lt;/a&gt;) &lt;em&gt;if and only if&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$d_{Y}(f(x),f(y)) \leq d_{X}(x,y) \tag{2} \label{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the subscript &lt;span class=&quot;math-container&quot;&gt;$_X$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$d_X$&lt;/span&gt; means that the metric &lt;span class=&quot;math-container&quot;&gt;$d_X$&lt;/span&gt; is the metric associated with the metric space &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. Therefore, any function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; between two metric spaces that satisfies \ref{2} is a non-expansive operator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show that the max operator is non-expansive, consider the set of real numbers, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;, equipped with the absolute value metric defined in \ref{1}. Then, in this case, &lt;span class=&quot;math-container&quot;&gt;$f=\operatorname{max}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$d(x, y) = |x - y|$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X = Y = \mathbb{R}$&lt;/span&gt;, so \ref{2} becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$|\operatorname{max}(x) - \operatorname{max}(y)| \leq | x - y | \tag{3} \label{3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}(x) = x, \forall x$&lt;/span&gt;, then \ref{3} trivially holds, that is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;|\operatorname{max}(x) - \operatorname{max}(y)| &amp;amp;\leq | x - y | \iff \\&#xA;|x - y| &amp;amp;\leq | x - y | \iff \\&#xA;|x - y| &amp;amp;= | x - y | \tag{4} \label{4}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that &lt;span class=&quot;math-container&quot;&gt;$x=6$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y=9$&lt;/span&gt;, then \ref{4} becomes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;|\operatorname{max}(6) - \operatorname{max}(9)| &amp;amp;\leq | 6 - 9 | \iff \\&#xA;|6 - 9| &amp;amp;\leq | -3 | \iff \\&#xA;|-3| &amp;amp;= 3&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other examples of non-expansive maps. For example, &lt;span class=&quot;math-container&quot;&gt;$f(x) = k x$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$0 \leq k \leq 1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f : \mathbb{R} \rightarrow \mathbb{R}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://en.wikipedia.org/wiki/Contraction_mapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Contraction_mapping&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Contraction_(operator_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Contraction_(operator_theory)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-09-15T14:38:46.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15441" PostTypeId="1" CreationDate="2019-09-15T16:17:37.197" Score="5" ViewCount="540" Body="&lt;p&gt;I am trying to solve a maze puzzle using the A* algorithm. I am trying to analyze the algorithm based on different applicable heuristic functions.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I explored the Manhattan and Euclidean distances. Which other heuristic functions are available? How do we compare them? How do we know whether a heuristic function is better than another?&lt;/p&gt;&#xA;" OwnerUserId="10118" LastEditorUserId="2444" LastEditDate="2021-02-04T22:04:30.633" LastActivityDate="2021-02-04T22:04:30.633" Title="How do we determine whether a heuristic function is better than another?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;a-star&gt;&lt;admissible-heuristic&gt;&lt;heuristic-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15442" PostTypeId="2" ParentId="15441" CreationDate="2019-09-15T17:55:52.307" Score="2" Body="&lt;p&gt;In the A* algorithm, at each iteration, a node is chosen which minimizes a certain function, called the &lt;em&gt;evaluation function&lt;/em&gt;, which, in the case of A*, is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(n)=g(n)+h(n)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the length (or cost) of the cheapest path from the start node to the current node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the heuristic function that estimates the &lt;em&gt;cost&lt;/em&gt; of the cheapest path from current node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to goal node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is potentially more than one path to the goal from a given node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. However, one of these paths is the cheapest path. An &lt;a href=&quot;https://en.wikipedia.org/wiki/Admissible_heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;admissible heuristic&lt;/strong&gt;&lt;/a&gt; function is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;heuristic&lt;/a&gt; function that does &lt;strong&gt;not&lt;/strong&gt; overestimate the cost to reach the goal node, that is, it estimates a cost to reach a goal that is smaller or equal to the cheapest path from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, which is denoted by &lt;span class=&quot;math-container&quot;&gt;$h^*(n)$&lt;/span&gt;. Therefore, an admissible heuristic &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; satisfies &lt;span class=&quot;math-container&quot;&gt;$h(n) \leq h^*(n), \forall n$&lt;/span&gt;. Given that the goal is to find the cheapest path from a start to a goal node, intuitively, an admissible heuristic is an &lt;em&gt;optimistic&lt;/em&gt; predictive function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A* is guaranteed to find the optimal solution (or path) if it uses an admissible heuristic. In section 2.4 of the book &lt;a href=&quot;https://books.google.ch/books?id=4JwN5DTpcqkC&amp;amp;pg=PA72&amp;amp;source=gbs_toc_r&amp;amp;cad=4#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principles of Artificial Intelligence&lt;/a&gt; (1982), Nils J. Nilsson provides the proof of this fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, not all admissible heuristics give the same information, so not all admissible heuristics are equally efficient. For instance, a heuristic function that is trivially admissible is &lt;span class=&quot;math-container&quot;&gt;$h(n) = 0, \forall n$&lt;/span&gt;. However, in this case, the only actual information that is used to choose the next node to expand is only based on &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n)$&lt;/span&gt;. &lt;a href=&quot;https://ai.stackexchange.com/q/9182/2444&quot;&gt;This evaluation function corresponds to the evaluation function of the uniform-cost search algorithm&lt;/a&gt;, which is an uninformed-search algorithm (as opposed to A*, which, nonetheless, is considered an informed-search algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which admissible heuristic is thus more informed? Consider two versions of A*, each with a different &lt;em&gt;admissible&lt;/em&gt; heuristic function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_1(n) = g_1(n) + h_1(n)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_2(n) = g_1(n) + h_2(n)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h_1(n) \leq h^*(n), \forall n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_2(n) \leq h^*(n), \forall n$&lt;/span&gt;. Then A* with the evaluation function &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; is more informed than A* with &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt; if, for all non-goal nodes &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_1(n) &amp;gt; h_2(n)$&lt;/span&gt;. See &lt;a href=&quot;https://books.google.ch/books?id=4JwN5DTpcqkC&amp;amp;pg=PA72&amp;amp;source=gbs_toc_r&amp;amp;cad=4#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 2.4.4. of the cited book&lt;/a&gt; where an example that attempts to show this is given. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The admissibility of a heuristic depends on the problem. For example, in the case of the &lt;a href=&quot;https://www.aaai.org/Papers/AAAI/2000/AAAI00-212.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fifteen Puzzle problem&lt;/a&gt;, both Manhattan and the Hamming distances are admissible heuristics. However, in other problems, these distances might not &lt;a href=&quot;https://math.stackexchange.com/q/2704941/168764&quot;&gt;induce&lt;/a&gt; an admissible heuristic.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-20T11:56:46.343" LastActivityDate="2019-09-20T11:56:46.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15444" PostTypeId="1" AcceptedAnswerId="15445" CreationDate="2019-09-15T20:58:34.960" Score="2" ViewCount="287" Body="&lt;p&gt;Consider the reward function &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt; with optimal state-action value function &lt;span class=&quot;math-container&quot;&gt;$q_*(s, a)$&lt;/span&gt;. What would be the optimal state-action value function of &lt;span class=&quot;math-container&quot;&gt;$c r(s, a)$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$c \in \mathbb{R}$&lt;/span&gt;? Would it be &lt;span class=&quot;math-container&quot;&gt;$c q_*(s, a)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T00:48:49.140" LastActivityDate="2020-11-01T00:48:49.140" Title="What is the optimal value function of the scaled version of the reward function?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;bellman-equations&gt;&lt;reward-shaping&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15445" PostTypeId="2" ParentId="15444" CreationDate="2019-09-15T20:58:34.960" Score="2" Body="&lt;p&gt;The &lt;em&gt;Bellman optimality equation&lt;/em&gt; is given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_*(s,a) = \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')}q_*(s',a')) \tag{1}\label{1}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the reward is multiplied by a constant &lt;span class=&quot;math-container&quot;&gt;$c &amp;gt; 0 \in \mathbb{R}$&lt;/span&gt;, then the new optimal action-value function is given by &lt;span class=&quot;math-container&quot;&gt;$cq_*(s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To prove this, we just need to show that equation \ref{1} holds when the reward is &lt;span class=&quot;math-container&quot;&gt;$cr$&lt;/span&gt; and the action-value is &lt;span class=&quot;math-container&quot;&gt;$c q_*(s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;c q_*(s,a) &#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(c r + \gamma \max_{a'\in\mathcal{A}(s')} c q_*(s',a')) \tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$c &amp;gt; 0$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\max_{a'\in\mathcal{A}(s')} c q_*(s',a') = c\max_{a'\in\mathcal{A}(s')}q_*(s',a')$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; can be taken out of the &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}$&lt;/span&gt; operator. Therefore, the equation \ref{2} becomes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;c q_*(s,a) &#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(c r + \gamma c \max_{a'\in\mathcal{A}(s')} q_*(s',a'))  \\&#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}c p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \\&#xA;&amp;amp;= c \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \\&#xA;q_*(s,a) &amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \tag{3}\label{3}&#xA;\end{align}&lt;/span&gt;&#xA;which is equal to the the Bellman optimality in \ref{1}, which implies that, when the reward is given by &lt;span class=&quot;math-container&quot;&gt;$cr$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c q_*(s,a)$&lt;/span&gt; is the solution to the Bellman optimality equation. Consequently, in this case, the set of optimal policies does not change.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$c=0$&lt;/span&gt;, then \ref{2} becomes &lt;span class=&quot;math-container&quot;&gt;$0=0$&lt;/span&gt;, which is true.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$c &amp;lt; 0$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\max_{a'\in\mathcal{A}(s')} c q_*(s',a') = c\min_{a'\in\mathcal{A}(s')}q_*(s',a')$&lt;/span&gt;, so equation \ref{3} becomes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q_*(s,a) &amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \min_{a'\in\mathcal{A}(s')} q_*(s',a'))&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is &lt;em&gt;not&lt;/em&gt; equal to the Bellman optimality equation in \ref{1}.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-08T12:41:11.997" LastActivityDate="2020-10-08T12:41:11.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15446" PostTypeId="1" CreationDate="2019-09-15T21:28:37.853" Score="2" ViewCount="119" Body="&lt;p&gt;Do neural networks compute the probability distribution for policy gradient methods. If so, how do they compute an infinite probability distribution? How do you represent a continuous action policy with a neural network?&lt;/p&gt;&#xA;" OwnerUserId="29708" LastEditorUserId="29708" LastEditDate="2019-09-16T04:00:40.747" LastActivityDate="2019-09-16T08:26:34.763" Title="How do policy gradients compute an infinite probability distribution from a neural network" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15447" PostTypeId="2" ParentId="15408" CreationDate="2019-09-16T02:27:37.330" Score="1" Body="&lt;p&gt;Yes, ML can fit a curve based on examples that include hyperparameters but not a model specification. To do this, you need to specify a &lt;em&gt;family&lt;/em&gt; of models that is large enough to include the true model. You can then treat this as learning a relationship from 4 inputs to a single output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose you are willing to make only the following relatively mild assumptions about &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a function mapping 4 inputs (3 parameters and a true input) to 1 output, all real valued.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a composition of a finite number (say, no more than 60) of the following basic mathematical operators: +,-,*,/,exp, ln, sin, min, max.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can now frame the search for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; as a graph-search or local-search problem through the space of possible functions, which is finite. If the space is small, or is smooth, you are likely to find good or exact representations of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of an ML technique that is explicitly designed for this purpose is Koza's &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0262111705&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt;. It searches the space of all possible LISP programs constructed from a pre-specified set of functions for a program that maps from specified inputs to specified outputs. It has been widely used for the kind of curve fitting you describe here.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-16T02:27:37.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15449" PostTypeId="1" AcceptedAnswerId="15462" CreationDate="2019-09-16T06:10:49.857" Score="53" ViewCount="10234" Body="&lt;p&gt;We often hear that artificial intelligence may harm or even kill humans, so it might prove dangerous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could artificial intelligence harm us?&lt;/p&gt;&#xA;" OwnerUserId="29713" LastEditorUserId="1671" LastEditDate="2019-09-16T21:19:15.030" LastActivityDate="2019-09-19T16:55:14.260" Title="How could artificial intelligence harm us?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;neo-luddism&gt;" AnswerCount="13" CommentCount="1" FavoriteCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="15450" PostTypeId="2" ParentId="15446" CreationDate="2019-09-16T06:58:07.077" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do neural networks compute the probability distribution for policy gradient methods. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In short, yes. It does not have to be neural networks, any trainable parametric function approximator based on gradients will do. Neural networks are a common choice, as are linear function approximators using selected basis functions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, how do they compute an infinite probability distribution? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For background, this is an issue for generating stochastic policies in continuous action spaces only. In discrete action spaces, it is usually possible to compute an arbitrary probability density function for the whole action space, and sample from it to model the policy. It is also possible to compute a deterministic policy simply enough in continuous spaces - the input is the current state and the output is the action to take. The issue then is that this does not allow an agent to learn through exploration of the environment. To do that requires a stochastic policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to generate a stochastic policy in continuous action spaces, you could discretise the space and sample from that using e.g. softmax to generate the action probabilities. Or you could have the approximation function do something more indirect: Output the parameters of a probability distribution that can be sampled from.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do you represent a continuous action policy with a neural network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Typically by having state features as input and the parameters of a PDF that can be sampled as the output. For instance, the network could output mean &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; of a normal distribution for an action value, and the policy is given by &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s) = a \sim \mathbb{N}(\mu, \sigma)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This distribution can be sampled (there are simple methods to generate a sample from a normal distribution), and returns from following this policy used as feedback to the neural network using the policy gradient theorem. Assuming that there is an optimal deterministic policy to be found, the neural network can learn over time to home in on a specific mean with low standard deviation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, the standard deviation can be treated as a hyper-parameter, similar to &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy action selection, and might be decayed over time. In that case, a neural network can just output the mean action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to learn a deterministic policy through off-policy learning, adding a noise function to support exploration. This is what Deep Deterministic Policy Gradient does.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-16T08:26:34.763" LastActivityDate="2019-09-16T08:26:34.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15451" PostTypeId="1" CreationDate="2019-09-16T07:58:02.007" Score="2" ViewCount="42" Body="&lt;p&gt;Considering a black box optimization problem on non-linear, non-convex function where we want to minimize an objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to assess the quality of an optimizer is to look at the best solution it finds. However that doesn't give us any information on how much of the parameter space the optimizer had to explore to come up with these solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore I was wondering if there are metrics quantifying how much of the parameter space is explored ?&lt;/p&gt;&#xA;" OwnerUserId="29715" LastEditorUserId="29715" LastEditDate="2019-09-23T08:25:54.223" LastActivityDate="2019-10-24T03:03:45.530" Title="Metrics of quality of parameter space exploration" Tags="&lt;algorithm&gt;&lt;genetic-algorithms&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15453" PostTypeId="2" ParentId="14342" CreationDate="2019-09-16T09:03:25.553" Score="0" Body="&lt;p&gt;I found the answer to my question, went back to the Python script and in the command that fits the model i.e.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; LR = LogisticRegression (C=0.1, solver = &quot;sag&quot;,max_iter=1000).fit (X_train, y_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The parameter C was set to 0.001 which is a very small value (meaning lambda is very high as C=1/lambda) (C is the regularization strength and smaller values indicate stronger regularization). more on that matter can be found &lt;a href=&quot;https://stackoverflow.com/questions/22851316/what-is-the-inverse-of-regularization-strength-in-logistic-regression-how-shoul&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25463" LastActivityDate="2019-09-16T09:03:25.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15455" PostTypeId="2" ParentId="15387" CreationDate="2019-09-16T09:23:08.720" Score="1" Body="&lt;p&gt;Not very sure about the AI in competitions, as I have not taken part in any competitive competitions. On comparing AI in Academia and Industry, the biggest difference is probably freedom.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In academia, considering a research project or so, a large number of experiments and trying new things are encouraged. New learnings are heeded to, and it usually involves rigorous literature survey and studies of previous works. Even if a model performed badly, if there were new learnings one could take from it, it wouldn't be deemed a failure. There is also a lot of data available that could be used for research purposes, and open-source projects used or learned from, are always thanked and appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In industry the scene is quite different. There is more of a focus on using pre-trained models or transfer learning. Quite frequently, open-source projects are just cloned, mildly developed, and deployed under the companies name without releasing the code - basically requiring bare minimum effort towards literature. More of a focus was given (&lt;em&gt;In my case at least&lt;/em&gt;) on reading blog posts and readme's, over the papers themselves, in order to save time. And compute efficiency is key. In industry, the effort is more directed towards scaling these models, building the data pipelines, and satisfying the clients needs. Data is also another concern in industry, with it being common practice to outsource data collection and preparation to third parties (&lt;em&gt;Usually other companies that specialize in this area&lt;/em&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key difference, I would say, is the amount of freedom one has in academia, as compared to a strong sense of direction towards a singular goal in industry. AI in industry pretty much mostly is in the solutions-and-services sector (&lt;em&gt;mostly&lt;/em&gt;), making it quite similar to software engineering, broadly speaking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, summarizing, the domain of the AI project makes a big difference, with the main difference being what part of the project most effort and focus is put into.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-09-16T09:23:08.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15456" PostTypeId="2" ParentId="15367" CreationDate="2019-09-16T09:44:09.160" Score="1" Body="&lt;p&gt;Considering your question was asking why KL-Divergence would be preferred over MSE, RMSE, or L1/L2 Norm between ground truth and predicted, as loss function for predicting a distribution -   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;KL-divergence is a measure on probability distributions. It essentially captures the information loss between ground truth distribution and predicted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L2-norm/MSE/RMSE doesn’t do well with probabilities, because of the power operations involved in the calculation of loss. Probabilities, being fractions under 1, are significantly effected by any power operations (Square or Root), and considering we are calculating the squares of differences of probabilities, the values that are summed are abnormally small, essentially barely learning anything as the random initialization itself starts with an abnormally small loss, almost always staying constant.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;L1 norm on the other hand, does not have any power operations, making it relatively acceptable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loss functions such as Kullback-Leibler-divergence or Jensen-Shannon-Divergence are preffered for probability distributions because of the statistical meaning they hold. KL-Divergence, as mentioned before, is a statistical measure of information loss between distributions, or in other words, assuming &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is the ground truth distribution, KL-Divergence is a measure of how much &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; deviates from &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;. Also considering probability distributions, convergence is much stronger in measures of Information Loss such KL-Divergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More clarity on the motivation behind Kullback-Leibler can be read &lt;a href=&quot;https://math.stackexchange.com/questions/90537/what-is-the-motivation-of-the-kullback-leibler-divergence&quot;&gt;here&lt;/a&gt;. Hope your query has been answered!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-09-16T09:44:09.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15457" PostTypeId="2" ParentId="15416" CreationDate="2019-09-16T09:54:49.157" Score="1" Body="&lt;p&gt;From what I have observed, the ability to scale an ML model is key.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real time inference must be quick, and cause no delays from the provider side. Being able to deploy the model also carries enormous weight - that is, how easy would it be to build the data pipelines and how easy would it be to integrate it in a web application from the server perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from the obvious achievement of set metrics and performance criterion, speed and ease of deployment also carry a very important role. There have been scenarios of brilliant solutions being denied (&lt;em&gt;from what I have seen&lt;/em&gt;) because they exceeded the limits set for time and compute in an application scenario.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-09-16T09:54:49.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15458" PostTypeId="2" ParentId="14357" CreationDate="2019-09-16T10:23:32.463" Score="2" Body="&lt;p&gt;You can look into the techniques used in GANs (genereative adverserial networks). These networks work by having 2 learning agents. 1 to create images and 1 to learn the difference in a human made image and a computer generated image. This works because the 2 agents drive each other to be better and ultimately make the generator create images which can't be distinguished from real life images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case you can make a agent trying to tell if the data is human or computer generated. The agent learning to move will then get negative rewards when the other agent can identify it as computer movement. This way the mover will learn to move like your reference data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just found this video and paper which does exactly the same as you asked. Instead of using a GAN like structure they use a task specific reward and a imitation reward, which is based on reference motion data they have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=vppFvq2quQ0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=vppFvq2quQ0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29671" LastEditorUserId="29671" LastEditDate="2019-11-12T14:52:22.020" LastActivityDate="2019-11-12T14:52:22.020" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15459" PostTypeId="1" CreationDate="2019-09-16T10:51:21.577" Score="4" ViewCount="102" Body="&lt;p&gt;Why does estimation error increase with &lt;span class=&quot;math-container&quot;&gt;$|H|$&lt;/span&gt; and decrease with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; in PAC learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across &lt;a href=&quot;https://i.stack.imgur.com/JLnLi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;this statement&lt;/a&gt; in the section 5.2 of the book &lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;understanding machine learning: from theory to algorithms&quot;&lt;/a&gt;. You just search &quot;&lt;strong&gt;increases (logarithmically)&lt;/strong&gt;&quot; in your browser and then you can find the sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just can't understand the statement. And there is no proof in the book either. What I would like to do is prove that estimation error &lt;span class=&quot;math-container&quot;&gt;$\epsilon_{est}$&lt;/span&gt; increase (logarithmically) with || and decrease with . Hope you can help me out. A rigorous proof can't be better!&lt;/p&gt;&#xA;" OwnerUserId="27112" LastEditorUserId="27112" LastEditDate="2019-09-18T00:27:51.523" LastActivityDate="2020-03-12T21:33:26.407" Title="Why does estimation error increase with $|H|$ and decrease with $m$ in PAC learning?" Tags="&lt;machine-learning&gt;&lt;proofs&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15460" PostTypeId="2" ParentId="15459" CreationDate="2019-09-16T11:25:55.493" Score="2" Body="&lt;p&gt;Definitely, you can find the proof in different resources (for example, in &lt;a href=&quot;http://karlstratos.com/notes/pac.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these notes&lt;/a&gt; or in the paper that originally proposed PAC learnability, &lt;a href=&quot;http://web.mit.edu/6.435/www/Valiant84.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Theory of the Learnable&lt;/a&gt;). However, the intuition behind your question is when the size of the hypothesis increases, if you do not change anything, you can't see more part of the space. Hence, the estimation error will increase. Moreover, when you increase the number of samples, you have more chance to see more part of the hypothesis space, hence, the estimation error decrease.&lt;/p&gt;&#xA;&lt;p&gt;Also, you can see some lemma about the relation of the PAC learnability and other similar concepts in the Wikipedia article &lt;a href=&quot;https://en.wikipedia.org/wiki/Probably_approximately_correct_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probably approximately correct learning&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Under some regularity conditions these three conditions are equivalent:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The concept class &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is PAC learnable.&lt;/li&gt;&#xA;&lt;li&gt;The VC dimension of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is finite.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a uniform Glivenko-Cantelli class.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-09-17T17:04:52.413" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15461" PostTypeId="1" CreationDate="2019-09-16T11:51:26.477" Score="1" ViewCount="32" Body="&lt;p&gt;&lt;strong&gt;tldr&lt;/strong&gt;; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which produces two outcomes (&lt;code&gt;p&lt;/code&gt; and &lt;code&gt;v&lt;/code&gt;) always predicts exactly &lt;code&gt;0&lt;/code&gt; for &lt;code&gt;v&lt;/code&gt; and I would like to change that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following post I will provide all code necessary to reproduce the problem.&lt;br&gt;&#xA;I am training a neural network on the same input. The wanted outcome for a value &quot;&lt;code&gt;v&lt;/code&gt;&quot; is &lt;code&gt;1&lt;/code&gt;. If I create the network and train it, sometimes the predicted outcome will be &lt;code&gt;1&lt;/code&gt;, sometimes it will be &lt;code&gt;-&lt;/code&gt;1.&lt;br&gt;&#xA; Also, the &lt;code&gt;loss&lt;/code&gt; seems to flip between &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;4&lt;/code&gt; during training epochs.&lt;br&gt;&#xA; Additionally, the loss blows up immensly, even though both losses for the outcome layers are close to zero.&lt;br&gt;&#xA; I do not understand where this behaviour comes from. I used &lt;code&gt;Leaky-ReLU&lt;/code&gt; to make sure it can handle negative input, I used a high &lt;code&gt;learning rate&lt;/code&gt; to make sure the data in this example is sufficient on the training, and the input is the same all the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Neural network looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input_layer = keras.Input(shape=(6,7),)    &#xA;formatted_input_layer = keras.layers.Reshape((6,7, 1))(input_layer)       &#xA;conv_layer1 = self.create_conv_layer(formatted_input_layer)&#xA;res_layer1 = self.create_res_layer(conv_layer1)&#xA;res_layer2 = self.create_res_layer(res_layer1)&#xA;res_layer3 = self.create_res_layer(res_layer2)&#xA;res_layer4 = self.create_res_layer(res_layer3)&#xA;policy_head = self.create_policy_head(res_layer4)&#xA;value_head = self.create_value_head(res_layer4)&#xA;model = keras.Model(inputs=input_layer, outputs=[policy_head, value_head])&#xA;optimizer = keras.optimizers.SGD(lr=args['lr'],momentum=args['momentum'])&#xA;model.compile(loss = {'policy_head' : 'categorical_crossentropy', 'value_head' : 'mean_squared_error'}, optimizer=optimizer, loss_weights={'policy_head':0.5, 'value_head':0.5})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Methods for the different layers:&lt;br&gt;&#xA;conv_layer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_conv_layer(self, input_layer):&#xA;    conv_layer = keras.layers.Conv2D(filters=256,&#xA;                                     kernel_size=3,&#xA;                                     strides=1,&#xA;                                     padding='same',&#xA;                                     use_bias=False,&#xA;                                     data_format=&quot;channels_last&quot;,&#xA;                                     activation = &quot;linear&quot;,&#xA;                                     kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)&#xA;    conv_layer= keras.layers.BatchNormalization(axis=-1)(conv_layer)&#xA;    conv_layer = keras.layers.LeakyReLU()(conv_layer)&#xA;    return conv_layer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;res_layer:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_res_layer(self, input_layer):&#xA;        conv_layer = self.create_conv_layer(input_layer)&#xA;        res_layer = keras.layers.Conv2D(filters=256,&#xA;                                         kernel_size=3,&#xA;                                         strides=1,&#xA;                                         padding='same',&#xA;                                         use_bias=False,&#xA;                                         data_format=&quot;channels_last&quot;,&#xA;                                         activation = &quot;linear&quot;,&#xA;                                         kernel_regularizer = keras.regularizers.l2(0.0001))(conv_layer)&#xA;        res_layer= keras.layers.BatchNormalization(axis=-1)(res_layer)&#xA;        res_layer = keras.layers.add([input_layer, res_layer])&#xA;        res_layer = keras.layers.LeakyReLU()(res_layer)&#xA;        return res_layer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;policy head:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_policy_head(self, input_layer):&#xA;        policy_head = keras.layers.Conv2D(filters=2,&#xA;                                          kernel_size=1,&#xA;                                          strides=1,&#xA;                                          padding='same',&#xA;                                          use_bias = False,&#xA;                                          data_format='channels_last',&#xA;                                          activation='linear',&#xA;                                          kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)&#xA;        policy_head = keras.layers.BatchNormalization(axis=-1)(policy_head)&#xA;        policy_head = keras.layers.LeakyReLU()(policy_head)&#xA;        policy_head = keras.layers.Flatten()(policy_head)&#xA;        policy_head = keras.layers.Dense(units = 7,&#xA;                                         use_bias = False,&#xA;                                         activation = 'softmax',&#xA;                                         kernel_regularizer = keras.regularizers.l2(0.0001),&#xA;                                         name = &quot;policy_head&quot;&#xA;                                         )(policy_head)&#xA;        return policy_head&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;value head:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_value_head(self, input_layer):&#xA;        value_head = keras.layers.Conv2D(filters=1,&#xA;                                          kernel_size=1,&#xA;                                          strides=1,&#xA;                                          padding='same',&#xA;                                          use_bias = False,&#xA;                                          data_format='channels_last',&#xA;                                          activation='linear',&#xA;                                          kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)&#xA;        value_head = keras.layers.BatchNormalization(axis=-1)(value_head)&#xA;        value_head = keras.layers.LeakyReLU()(value_head)  &#xA;        value_head = keras.layers.Flatten()(value_head)        &#xA;        value_head = keras.layers.Dense(units = 21,&#xA;                                         use_bias = False,&#xA;                                         activation = 'linear',&#xA;                                         kernel_regularizer = keras.regularizers.l2(0.0001)&#xA;                                         )(value_head)&#xA;        value_head = keras.layers.LeakyReLU()(value_head)      &#xA;        value_head = keras.layers.Dense(units = 1,&#xA;                                         use_bias = False,&#xA;                                         activation = 'tanh',&#xA;                                         kernel_regularizer = keras.regularizers.l2(0.0001),&#xA;                                         name = &quot;value_head&quot;                                     &#xA;                                         )(value_head)&#xA;        return value_head&#xA;&#xA;                                )(value_head)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The way I am testing my NN:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    canonicalBoard = np.zeros(shape = (6,7), dtype=int) &#xA;&#xA;&#xA;    Pi = [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0] &#xA;&#xA;    trainExamples = [[canonicalBoard, Pi, 1]]*50        &#xA;&#xA;    nnetwrapper.train(trainExamples)&#xA;&#xA;    board = canonicalBoard[np.newaxis, :, :]&#xA;&#xA;    p, v = nnetwrapper.nnet.model.predict(board)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which results in the training looking like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 1/10&#xA;50/50 [==============================] - 4s 71ms/step - loss: 1.6829 - policy_head_loss: 1.9459 - value_head_loss: 1.0000&#xA;Epoch 2/10&#xA;50/50 [==============================] - 1s 15ms/step - loss: 2.3218 - policy_head_loss: 3.8470 - value_head_loss: 0.3768&#xA;Epoch 3/10&#xA;50/50 [==============================] - 1s 13ms/step - loss: 4456112.5000 - policy_head_loss: 0.9027 - value_head_loss: 0.8510&#xA;Epoch 4/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 16085884.0000 - policy_head_loss: 0.0945 - value_head_loss: 3.9925&#xA;Epoch 5/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 32722448.0000 - policy_head_loss: 2.6572 - value_head_loss: 4.0000&#xA;Epoch 6/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 52690084.0000 - policy_head_loss: 9.6345 - value_head_loss: 3.1810e-12&#xA;Epoch 7/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 74703120.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000&#xA;Epoch 8/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 97784832.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000&#xA;Epoch 9/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 121202520.0000 - policy_head_loss: 2.0802e-05 - value_head_loss: 4.0000&#xA;Epoch 10/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 144415040.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and my prediction looking like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p: [[0. 0. 0. 0. 0. 1. 0.]]  v: [[-1.]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;another outcome could be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 1/10&#xA;50/50 [==============================] - 4s 82ms/step - loss: 1.6829 - policy_head_loss: 1.9459 - value_head_loss: 1.0000&#xA;Epoch 2/10&#xA;50/50 [==============================] - 1s 17ms/step - loss: 2.2826 - policy_head_loss: 2.0001 - value_head_loss: 2.1454&#xA;Epoch 3/10&#xA;50/50 [==============================] - 1s 16ms/step - loss: 1718694.1250 - policy_head_loss: 0.5434 - value_head_loss: 3.9772&#xA;Epoch 4/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 6204218.0000 - policy_head_loss: 9.4180e-05 - value_head_loss: 0.0000e+00&#xA;Epoch 5/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 12620835.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00&#xA;Epoch 6/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 20322232.0000 - policy_head_loss: 7.7489 - value_head_loss: 0.0000e+00&#xA;Epoch 7/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 28812526.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 1.5966&#xA;Epoch 8/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 37715012.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00&#xA;Epoch 9/10&#xA;50/50 [==============================] - 1s 15ms/step - loss: 46747064.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00&#xA;Epoch 10/10&#xA;50/50 [==============================] - 1s 14ms/step - loss: 55699992.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with the predictions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p: [[0. 0. 0. 0. 0. 1. 0.]]  v: [[1.]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which are the correct ones as I would have expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How come on some trainings, my NN doesnt fit the data at all?   I wanted to start the training process for a whole week now, but before I do so I want to make sure there are no errors in the way I layed out my NN.   And this looks like I am missing something here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here at the &lt;code&gt;predict&lt;/code&gt; / &lt;code&gt;train&lt;/code&gt; methods of my neuralnet, (the NN is part of an alpha-zero replica and the involved game is connect4, I omitted these in the example to make it easier to actually replicate the problem. This is why you see some transform operations in &lt;code&gt;predict&lt;/code&gt; and &lt;code&gt;train&lt;/code&gt; methods)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def train(self, examples):&#xA;        input_boards, target_pis, target_vs = list(zip(*examples))       &#xA;        input_boards = np.asarray(input_boards)&#xA;        target_pis = np.asarray(target_pis)&#xA;        target_vs = np.asarray(target_vs)        &#xA;        logger.debug(&quot;Passing to nn: x: {}, y: {}, batch_size: {}, epochs: {}&quot;.format(input_boards, [target_pis, target_vs], self.args[&quot;batch_size&quot;], self.args[&quot;epochs&quot;]))&#xA;        self.nnet.model.fit(x = input_boards, y = [target_pis, target_vs], batch_size = self.args[&quot;batch_size&quot;], epochs = self.args[&quot;epochs&quot;])&#xA;&#xA;&#xA;def predict(self, board):&#xA;        board = board.nn_board_2d&#xA;        # preparing input&#xA;        board = board[np.newaxis, :, :] # this has to be done for the conv2d to work&#xA;&#xA;        # run&#xA;        pi, v = self.nnet.model.predict(board)&#xA;        return pi[0], v[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The parameters I used for this example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'lr': 0.2&#xA;'dropout': 0.1&#xA;'epochs': 10&#xA;'num_channels': 512,&#xA;'filters': 256&#xA;'momentum':0.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: As soon as I use a lower learning rate and more iterations, my &lt;code&gt;p&lt;/code&gt; changes, but my &lt;code&gt;v&lt;/code&gt; stays exactly at &lt;code&gt;0&lt;/code&gt;. This is what was bothering me in the first place:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 1/10&#xA;118/118 [==============================] - 5s 44ms/step - loss: 1.8037 - policy_head_loss: 2.4305 - value_head_loss: 0.7578&#xA;Epoch 2/10&#xA;118/118 [==============================] - 2s 14ms/step - loss: 1.1666 - policy_head_loss: 1.7723 - value_head_loss: 0.1416&#xA;Epoch 3/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 1.0987 - policy_head_loss: 1.6832 - value_head_loss: 0.0948&#xA;Epoch 4/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 1.0430 - policy_head_loss: 1.5787 - value_head_loss: 0.0876&#xA;Epoch 5/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 0.9943 - policy_head_loss: 1.4859 - value_head_loss: 0.0826&#xA;Epoch 6/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 0.9469 - policy_head_loss: 1.3959 - value_head_loss: 0.0777&#xA;Epoch 7/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 0.9046 - policy_head_loss: 1.3180 - value_head_loss: 0.0707&#xA;Epoch 8/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 0.8629 - policy_head_loss: 1.2403 - value_head_loss: 0.0647&#xA;Epoch 9/10&#xA;118/118 [==============================] - 2s 14ms/step - loss: 0.8068 - policy_head_loss: 1.1344 - value_head_loss: 0.0583&#xA;Epoch 10/10&#xA;118/118 [==============================] - 2s 13ms/step - loss: 0.7335 - policy_head_loss: 0.9922 - value_head_loss: 0.0538&#xA;I0916 14:24:38.595780  7116 trainingonly1NN.py:232] Values for empty board: new network: P : [0.20057818 0.11068489 0.15129891 0.20042823 0.13117987 0.04705378&#xA; 0.15877616]  v: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27406" LastEditorUserId="27406" LastEditDate="2019-09-16T15:56:17.993" LastActivityDate="2019-09-16T15:56:17.993" Title="Neural Network training on one example to try overfitting leads to strange predictions" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;architecture&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15462" PostTypeId="2" ParentId="15449" CreationDate="2019-09-16T12:18:11.900" Score="51" Body="&lt;h2&gt;tl;dr&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are many &lt;strong&gt;valid&lt;/strong&gt; reasons why people might fear (or better &lt;em&gt;be concerned about&lt;/em&gt;) AI, not all involve robots and apocalyptic scenarios. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To better illustrate these concerns, I'll try to split them into three categories.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conscious AI&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is the type of AI that your question is referring to. A super-intelligent conscious AI that will destroy/enslave humanity. This is mostly brought to us by science-fiction. Some notable Hollywood examples are &lt;em&gt;&quot;The terminator&quot;&lt;/em&gt;, &lt;em&gt;&quot;The Matrix&quot;&lt;/em&gt;, &lt;em&gt;&quot;Age of Ultron&quot;&lt;/em&gt;. The most influential novels were written by Isaac Asimov and are referred to as the &lt;em&gt;&quot;Robot series&quot;&lt;/em&gt; (which includes &lt;em&gt;&quot;I, robot&quot;&lt;/em&gt;, which was also adapted as a movie).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic premise under most of these works are that AI will evolve to a point where it becomes conscious and will surpass humans in intelligence. While Hollywood movies mainly focus on the robots and the battle between them and humans, not enough emphasis is given to the actual AI (i.e. the &quot;brain&quot; controlling them). As a side note, because of the narrative, this AI is usually portrayed as supercomputer controlling everything (so that the protagonists have a specific target). Not enough exploration has been made on &quot;ambiguous intelligence&quot; (which I think is more realistic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the real world, AI is focused on solving specific tasks! An AI agent that is capable of solving problems from different domains (e.g. understanding speech and processing images and driving and ... - like humans are) is referred to as General Artificial Intelligence and is required for AI being able to &quot;think&quot; and become conscious.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realistically, we are a &lt;strong&gt;loooooooong way&lt;/strong&gt; from General Artificial Intelligence! That being said there is &lt;strong&gt;no evidence&lt;/strong&gt; on why this &lt;strong&gt;can't&lt;/strong&gt; be achieved in the future. So currently, even if we are still in the infancy of AI, we have no reason to believe that AI won't evolve to a point where it is more intelligent than humans.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Using AI with malicious intent&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Even though an AI conquering the world is a long way from happening there are &lt;strong&gt;several reasons to be concerned with AI today&lt;/strong&gt;, that don't involve robots!&#xA;The second category I want to focus a bit more on is several malicious uses of today's AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll focus only on &lt;strong&gt;AI applications that are available today&lt;/strong&gt;. Some examples of AI that can be used for malicious intent:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deepfake&quot; rel=&quot;noreferrer&quot;&gt;DeepFake&lt;/a&gt;: a technique for imposing someones face on an image a video of another person. This has gained popularity recently with celebrity porn and can be used to generate &lt;strong&gt;fake news&lt;/strong&gt; and hoaxes. Sources: &lt;a href=&quot;https://www.youtube.com/watch?v=gLoI9hAX9dw&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://theoutline.com/post/3179/deepfake-videos-are-freaking-experts-out?zd=1&amp;amp;zi=zw6lhzz2&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://www.nytimes.com/2018/03/04/technology/fake-videos-deepfakes.html&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With the use of &lt;strong&gt;mass surveillance systems&lt;/strong&gt; and &lt;strong&gt;facial recognition&lt;/strong&gt; software capable of recognizing &lt;a href=&quot;https://www.digitaltrends.com/cool-tech/goodbye-anonymity-latest-surveillance-tech-can-search-up-to-36-million-faces-per-second/&quot; rel=&quot;noreferrer&quot;&gt;millions of faces per second&lt;/a&gt;, AI can be used for mass surveillance. Even though when we think of mass surveillance we think of China, many western cities like &lt;a href=&quot;https://www.cctv.co.uk/how-many-cctv-cameras-are-there-in-london/&quot; rel=&quot;noreferrer&quot;&gt;London&lt;/a&gt;, &lt;a href=&quot;https://cloudtweaks.com/2019/09/mass-surveillance-adversarial-ai-in-atlanta/&quot; rel=&quot;noreferrer&quot;&gt;Atlanta&lt;/a&gt; and Berlin are among the &lt;a href=&quot;https://www.comparitech.com/vpn-privacy/the-worlds-most-surveilled-cities/&quot; rel=&quot;noreferrer&quot;&gt;most-surveilled cities in the world&lt;/a&gt;. China has taken things a step further by adopting the &lt;a href=&quot;https://futurism.com/china-social-credit-system-rate-human-value&quot; rel=&quot;noreferrer&quot;&gt;social credit system&lt;/a&gt;, an evaluation system for civilians which seems to be taken straight out of the pages of George Orwell's 1984.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Influencing&lt;/strong&gt; people through &lt;strong&gt;social media&lt;/strong&gt;. Aside from recognizing user's tastes with the goal of targeted marketing and add placements (a common practice by many internet companies), AI can be used malisciously to influence people's voting (among other things). Sources: &lt;a href=&quot;https://theintercept.com/2018/04/13/facebook-advertising-data-artificial-intelligence-ai/&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://www.martechadvisor.com/articles/machine-learning-ai/the-impact-of-artificial-intelligence-on-social-media/&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://venturebeat.com/2018/04/13/ai-weekly-facebook-fiasco-shows-we-need-a-new-scheme-for-personal-data/&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://gizmodo.com/hackers-have-already-started-to-weaponize-artificial-in-1797688425&quot; rel=&quot;noreferrer&quot;&gt;Hacking&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Military applications, e.g. drone attacks, missile targeting systems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Adverse effects of AI&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This category is pretty subjective, but the development of AI might carry some adverse side-effects. The distinction between this category and the previous is that these effects, while harmful, aren't done intentionally; rather they occur with the development of AI. Some examples are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Jobs becoming redundant&lt;/strong&gt;. As AI becomes better, many jobs will be replaced by AI. Unfortunately there are not many things that can be done about this, as most technological developments have this side-effect (e.g. agricultural machinery caused many farmers to lose their jobs, automation replaced many factory workers, computers did the same). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reinforcing the &lt;strong&gt;bias in our data&lt;/strong&gt;. This is a very interesting category, as AI (and especially Neural Networks) are only as good as the data they are trained on and have a tendency of perpetuating and even enhancing different forms of social biases, already existing in the data. There are many examples of networks exhibiting racist and sexist behavior. Sources: &lt;a href=&quot;https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1607.06520&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://cacm.acm.org/magazines/2018/6/228035-bias-on-the-web/fulltext&quot; rel=&quot;noreferrer&quot;&gt;3&lt;/a&gt;, &lt;a href=&quot;https://read.dukeupress.edu/world-policy-journal/article-abstract/33/4/111/30942/Racist-in-the-MachineThe-Disturbing-Implications?redirectedFrom=fulltext&quot; rel=&quot;noreferrer&quot;&gt;4&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-09-16T12:18:11.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15463" PostTypeId="1" CreationDate="2019-09-16T12:29:45.347" Score="0" ViewCount="100" Body="&lt;p&gt;I'm dealing with a &quot;ticket similarity task&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every time new tickets arrive at the help desk (customer service), I need to compare them and find out about similar ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, once the operator responds to a ticket, at the same time he can solve the others similar to the one solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect an input ticket and all the other tickets with their similarity in output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using &lt;strong&gt;DOC2VEC&lt;/strong&gt;, but it requires training every time a new ticket enters ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what do you recommend?&lt;/p&gt;&#xA;" OwnerUserId="20780" LastEditorUserId="4446" LastEditDate="2019-09-16T21:28:08.763" LastActivityDate="2021-02-08T01:03:20.250" Title="Real time ticket similarity" Tags="&lt;machine-learning&gt;&lt;word-embedding&gt;&lt;similarity&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15464" PostTypeId="2" ParentId="15449" CreationDate="2019-09-16T13:23:41.400" Score="9" Body="&lt;p&gt;In addition to the other answers, I would like to add to nuking cookie factory example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning AIs basically try to fulfill a goal described by humans. For example, humans create an AI running a cookie factory. The goal they implement is to sell as many cookies as possible for the highest profitable margin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, imagine an AI which is sufficiently powerful. This AI will notice that if he nukes all other cookie factories, everybody has to buy cookies in his factory, making sales rise and profits higher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the human error here is giving no penalty for using violence in the algorithm. This is easily overlooked because humans didn't expect the algorithm to come to this conclusion.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastEditorUserId="2444" LastEditDate="2019-09-16T15:41:34.993" LastActivityDate="2019-09-16T15:41:34.993" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15465" PostTypeId="2" ParentId="7875" CreationDate="2019-09-16T16:46:42.113" Score="3" Body="&lt;p&gt;There is at least one very important and &lt;em&gt;serious&lt;/em&gt; AI scientist that apparently believes in the creation of true artificial general intelligence and possibly superintelligence: Jürgen Schmidhuber, who is the co-author of the &lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt;, among many other important contributions. In fact, he recently founded &lt;a href=&quot;https://nnaisense.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;NNAISENSE&lt;/a&gt; for this ultimate purpose, that is, to build a general-purpose artificial general intelligence. In his talk &lt;a href=&quot;https://www.youtube.com/watch?v=KQ35zNlyG-o&quot; rel=&quot;nofollow noreferrer&quot;&gt;When creative machines overtake man&lt;/a&gt;, at TEDxLausanne, Schmidhuber talks about the &lt;em&gt;singularity&lt;/em&gt; (also known as &lt;em&gt;omega&lt;/em&gt;). See also his web article &lt;a href=&quot;http://people.idsia.ch/~juergen/history.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Is History Converging? Again?&lt;/a&gt; (2012) or the paper &lt;a href=&quot;https://arxiv.org/abs/0708.4311&quot; rel=&quot;nofollow noreferrer&quot;&gt;2006: Celebrating 75 years of AI - History and Outlook: the Next 25 Years&lt;/a&gt; (2007).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-09-16T16:46:42.113" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15467" PostTypeId="2" ParentId="15449" CreationDate="2019-09-16T18:31:16.767" Score="5" Body="&lt;p&gt;I would say the biggest real threat would be the unbalancing/disrupting we are already seeing.  The changes of putting 90% the country out of work are real, and the results (which will be even more uneven distribution of wealth) are terrifying if you think them through.&lt;/p&gt;&#xA;" OwnerUserId="29725" LastActivityDate="2019-09-16T18:31:16.767" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15470" PostTypeId="2" ParentId="15408" CreationDate="2019-09-16T19:06:21.880" Score="0" Body="&lt;p&gt;While John's answer I think gives a better idea of which direction I might want to go to seriously tackle this, it turns out that just throwing the data straight into some &lt;code&gt;sklearn&lt;/code&gt; algorithms does work better than I thought it would. For example, the following produces ballpark results for the model parameters (for the two model cases I tested), without assuming (explicitly) anything about the model itself:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.ensemble import RandomForestRegressor                                                                   &#xA;from sklearn.model_selection import train_test_split                                                                 &#xA;from sklearn import metrics &#xA;&#xA;Xtrain, Xtest, ytrain, ytest = train_test_split(data, parms, random_state=1, )  &#xA;&#xA;model = RandomForestRegressor(random_state=1, n_estimators=15, n_jobs=7)                                             &#xA;model.fit(X=Xtrain,y=ytrain)                                                                                       &#xA;ypred = model.predict(Xtest)&#xA;&#xA;print(f'i={i} ' + str([metrics.explained_variance_score(ytest[:,i], ypred[:,i]) for i in [0,1,2]]))                  &#xA;&#xA;plt.figure(2)                                                                                                        &#xA;plt.title('Guessed parameter fits')                                                                                  &#xA;plt.clf()                                                                                                            &#xA;for i in range(3) :                                                                                                  &#xA;    for j in range(3) :                                                                                              &#xA;        plt.subplot(3,3,i*3+j+1)                                                                                     &#xA;        plt.plot(x,Xtest[i*3+j],'.')                                                                                 &#xA;        plt.plot(x,f(x,*ypred[i*3+j,:]))                                                                             &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gaussian curves, explained variance for a,b,c parameter guesses (1.0=perfect everytime):&lt;br&gt;&#xA;&lt;code&gt;[0.8784556933371098, 0.9172501716286985, 0.8874106964444304]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sin curves, explained variance for a,b,c parameter guesses&lt;br&gt;&#xA;&lt;code&gt;[0.8190156553698631, 0.9757765139100565, 0.7551784827108721]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's some plots of from the test sets, along with the original model evaluated with the guessed parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KNVAA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KNVAA.png&quot; alt=&quot;Example plots&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ca9wB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ca9wB.png&quot; alt=&quot;Example plots&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21099" LastActivityDate="2019-09-16T19:06:21.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15472" PostTypeId="2" ParentId="15463" CreationDate="2019-09-16T20:32:43.260" Score="0" Body="&lt;p&gt;You need to create an active learning loop over the process of the learning. Try to start from a history of tickets and using doc2vec to get the similarity. When you find a bad result in the result of your classifier, then report it and then try to retrain the classifier. Also, you can wait to retrain the model, up to finding the predefined batch0size of the new data which are not in the training set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, to get a better result in the active learning loop, you can testify incoming data by the measuring of the classifier uncertainty over it. If the entropy of the classifier over the data is not in a good situation, you can label the data by the operator (as an oracle) and then up reach to the predefined batch-size, retrain the classifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Morevoer, to know better about the active learning process and query strategies follow &lt;a href=&quot;https://en.wikipedia.org/wiki/Active_learning_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; (and other articles in that link like &lt;a href=&quot;http://burrsettles.com/pub/settles.activelearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-09-16T20:39:01.170" LastActivityDate="2019-09-16T20:39:01.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15476" PostTypeId="2" ParentId="15449" CreationDate="2019-09-16T21:23:42.783" Score="8" Body="&lt;p&gt;My favorite scenario for harm by AI involves not high intelligence, but low intelligence.  Specifically, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gray_goo&quot; rel=&quot;noreferrer&quot;&gt;grey goo&lt;/a&gt; hypothesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where a self-replicating, automated process runs amok and converts all resources into copies of itself.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point here is that the AI is not &quot;smart&quot; in the sense of having high intelligence or general intelligence--it is merely very good at a single thing and has the ability to replicate exponentially.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-09-16T21:23:42.783" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15478" PostTypeId="2" ParentId="15449" CreationDate="2019-09-17T04:09:31.267" Score="15" Body="&lt;h1&gt;Short term&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Physical accidents&lt;/strong&gt;, e.g. due to industrial machinery, aircraft autopilot, self-driving cars. Especially in the case of &lt;em&gt;unusual situations&lt;/em&gt; such as extreme weather or sensor failure. Typically an AI will function poorly under conditions where it has not been extensively tested.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Social impacts&lt;/strong&gt; such as reducing job availability, barriers for the underprivileged wrt. loans, insurance, parole.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Recommendation engines&lt;/strong&gt; are manipulating us more and more to change our behaviours (as well as reinforce our own &quot;small world&quot; bubbles). Recommendation engines routinely serve up inappropriate content of various sorts to young children, often because content creators (e.g. on YouTube) use the right keyword stuffing to appear to be child-friendly.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Political manipulation...&lt;/strong&gt; Enough said, I think.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Plausible deniability of privacy invasion&lt;/strong&gt;. Now that AI can read your email and even make phone calls for you, it's easy for someone to have humans act on your personal information and claim that they got a computer to do it.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Turning war into a video game&lt;/strong&gt;, that is, replacing soldiers with machines being operated remotely by someone who is not in any danger and is far removed from his/her casualties.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Lack of transparency&lt;/strong&gt;. We are trusting machines to make decisions with very little means of getting the justification behind a decision.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Resource consumption and pollution.&lt;/strong&gt; This is not just an AI problem, however every improvement in AI is creating more demand for Big Data and together these ram up the need for storage, processing, and networking. On top of the electricity and rare minerals consumption, the infrastructure needs to be disposed of after its several-year lifespan.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Surveillance&lt;/strong&gt; — with the ubiquity of smartphones and listening devices, there is a gold mine of data but too much to sift through every piece. Get an AI to sift through it, of course!&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cybersecurity&lt;/strong&gt; — cybercriminals are increasingly leveraging AI to attack their targets.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Did I mention that &lt;em&gt;all&lt;/em&gt; of these are in full swing already?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Long Term&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Although there is no clear line between AI and AGI, this section is more about what happens when we go further towards AGI. I see two alternatives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Either we develop AGI as a result of our improved understanding of the nature of intelligence,&lt;/li&gt;&#xA;&lt;li&gt;or we slap together something that seems to work but we don't understand very well, much like a lot of machine learning right now.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the first case, if an AI &quot;goes rogue&quot; we can build other AIs to outwit and neutralise it. In the second case, we can't, and we're doomed. AIs will be a new life form and we may go extinct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some potential problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Copy and paste.&lt;/strong&gt; One problem with AGI is that it could quite conceivably run on a desktop computer, which creates a number of problems:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Script Kiddies&lt;/strong&gt; ­— people could download an AI and set up the parameters in a destructive way. Relatedly,&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Criminal or terrorist groups&lt;/strong&gt; would be able to configure an AI to their liking. You don't need to find an expert on bomb making or bioweapons if you can download an AI, tell it to do some research and then give you step-by-step instructions.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Self-replicating AI&lt;/strong&gt; — there are plenty of computer games about this. AI breaks loose and spreads like a virus. The more processing power, the better able it is to protect itself and spread further.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Invasion of computing resources&lt;/strong&gt;. It is likely that more computing power is beneficial to an AI. An AI might buy or steal server resources, or the resources of desktops and mobile devices. Taken to an extreme, this could mean that all our devices simply became unusable which would wreak havoc on the .world immediately. It could also mean massive electricity consumption (and it would be hard to &quot;pull the plug&quot; because power plants are computer controlled!)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Automated factories.&lt;/strong&gt; An AGI wishing to gain more of a physical presence in the world could take over factories to produce robots which could build new factories and essentially create bodies for itself.&lt;/li&gt;&#xA;&lt;li&gt;These are rather philosophical considerations, but some would argue that AI would destroy what makes us human:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Inferiority.&lt;/strong&gt; What if plenty of AI entities were smarter, faster, more reliable and more creative than the best humans?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Pointlessness.&lt;/strong&gt; With robots replacing the need for physical labour and AIs replacing the need for intellectual labour, we will really have nothing to do. Nobody's going to get the Nobel Prize again because the AI will already be ahead. Why even get educated in the first place?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Monoculture/stagnation&lt;/strong&gt; — in various scenarios (such as a single &quot;benevolent dictator&quot; AGI) society could become fixed in a perpetual pattern without new ideas or any sort of change (pleasant though it may be). Basically, &lt;em&gt;Brave New World.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think AGI is coming and we need to be mindful of these problems so that we can minimise them.&lt;/p&gt;&#xA;" OwnerUserId="29739" LastActivityDate="2019-09-17T04:09:31.267" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15479" PostTypeId="1" CreationDate="2019-09-17T04:13:13.033" Score="4" ViewCount="281" Body="&lt;p&gt;I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised way, the positions of that object. For example, us, as humans, would record the x and y location of the object. Of course the NN doesn't have a notion of x and y, but i would like, given an image, the NN to produce 2 numbers, that preserve as much as possible from the actual relative position of objects on the background. For example, if 3 objects are equally spaced on a straight line (in 3 of the images), I would like the 2 numbers produced by the NN for each of the 3 images to preserve this ordering, even if they won't form a straight line. They can form a weird curve, but as long as the order is correct that can be topologically transformed to the right, straight line. Can someone suggest me any paper/architecture that did something similar? Thank you! &lt;/p&gt;&#xA;" OwnerUserId="29742" LastActivityDate="2019-09-17T04:13:13.033" Title="Get the position of an object, out of an image" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15482" PostTypeId="5" CreationDate="2019-09-17T10:18:52.053" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-17T10:18:52.053" LastActivityDate="2019-09-17T10:18:52.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15483" PostTypeId="4" CreationDate="2019-09-17T10:18:52.053" Score="0" Body="For questions related to computational learning theory (or, in short, learning theory), which is a research subfield of artificial intelligence devoted to studying the design and mathematical analysis of machine learning algorithms. Computational learning theory (COLT) is largely concerned with computational and data efficiency. A seminal paper in COLT is Valiant's &quot;A theory of the learnable&quot; (1984)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-17T10:18:52.053" LastActivityDate="2019-09-17T10:18:52.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15484" PostTypeId="2" ParentId="15449" CreationDate="2019-09-17T11:18:42.830" Score="6" Body="&lt;p&gt;I have an example which goes in kinda the opposite direction of the public's fears, but is a very real thing, which I already see happening. It is not AI-specific, but I think it will get worse through AI. It is the problem of &lt;strong&gt;humans trusting the AI conclusions blindly&lt;/strong&gt; in critical applications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have many areas in which human experts are supposed to make a decision. Take for example medicine - should we give medication X or medication Y? The situations I have in mind are frequently complex problems (in the Cynefin sense) where it is a really good thing to have somebody pay attention very closely and use lots of expertise, and the outcome really matters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a demand for medical informaticians to write decision support systems for this kind of problem in the medicine (and I suppose for the same type in other domains). They do their best, but the expectation is always that a human expert will always consider the system's suggestion just as one more opinion when making the decision. In many cases, it would be irresponsible to promise anything else, given the state of knowledge and the resources available to the developers. A typical example would be the use of computer vision in radiomics: a patient gets a CT scan and the AI has to process the image and decide whether the patient has a tumor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the AI is not perfect. Even when measured against the gold standard, it never achieves 100% accuracy. And then there are all the cases where it performs well against its own goal metrics, but the problem was so complex that the goal metric doesn't capture it well - I can't think of an example in the CT context, but I guess we see it even here on SE, where the algorithms favor popularity in posts, which is an imperfect proxy for factual correctness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You were probably reading that last paragraph and nodding along, &quot;Yeah, I learned that in the first introductory ML course I took&quot;. Guess what? Physicians never took an introductory ML course. They rarely have enough statistic literacy to understand the conclusions of papers appearing in medical journals. When they are talking to their 27th patient, 7 hours into their 16 hour shift, hungry and emotionally drained, and the CT doesn't look all that clear-cut, but the computer says &quot;it's not a malignancy&quot;, they don't take ten more minutes to concentrate on the image more, or look up a textbook, or consult with a colleague. They just go with what the computer says, grateful that their cognitive load is not skyrocketing yet again. So they turn from being experts to being people who read something off a screen. Worse, in some hospitals the administration does not only trust computers, it also has found out that they are convenient scapegoats. So, a physician has a bad hunch which goes against the computer's output, it becomes difficult for them to act on that hunch and defend themselves that they chose to overrode the AI's opinion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AIs are powerful and useful tools, but there will always be tasks where they can't replace the toolwielder. &lt;/p&gt;&#xA;" OwnerUserId="29755" LastActivityDate="2019-09-17T11:18:42.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15485" PostTypeId="1" AcceptedAnswerId="15531" CreationDate="2019-09-17T11:37:53.417" Score="2" ViewCount="43" Body="&lt;p&gt;For example, if I want to do a cat and mouse AI, the cat would wish to minimize the time taken for it to catch the mouse and the mouse would want to maximize that time. The time is analog and thus I cannot use a traditional Xy method but need another method that goes like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;network.train_against_value(X, y, determinator)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, X is more like where the cat and mouse are. y is where the cat or mouse should move, and determinator is the time taken for the mouse to be caught, where the mouse wishes to maximize this value through its output of y and the cat wishes to minimize it. There is one Xy pair for each decision made by the cat and mouse, but one determinator throughout one game. Many games are played to train the AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: X: (300, 300, 200, 200) -&gt; (mousex, mousey, catx, caty)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Y: (1,3) -&gt; (xmove, ymove) direction, the numbers are then tuned by code for the actual movement to be always 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determinator: 50 -&gt; time for mouse to be caught in seconds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where it would train so that with every X inputted it outputs a y so that determinator is minimum. Is there a method for train_towards_value as well? If there is no prebuilt method, how do I create one? What is the technical name for this kind of training?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two neural networks for the cat and mouse, where the cat is slower than the mouse but is larger and could eat the mouse. Just consider the mouse is difficult to control from the neural network because of inefficiencies so that it is possible for the cat to catch the mouse. &lt;/p&gt;&#xA;" OwnerUserId="17423" LastEditorUserId="17423" LastEditDate="2019-09-19T08:22:59.227" LastActivityDate="2019-09-19T09:07:53.043" Title="Training Keras Towards Or Against Analog Value?" Tags="&lt;keras&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15486" PostTypeId="2" ParentId="15449" CreationDate="2019-09-17T12:19:22.697" Score="5" Body="&lt;p&gt;This only intents to be a complement to other answers so I will not discuss to possibility of AI trying to willingly enslave humanity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But a different risk is already here. I would call it &lt;em&gt;unmastered technology&lt;/em&gt;. I have been teached science and technology, and IMHO, AI has &lt;em&gt;by itself&lt;/em&gt; no notion of good and evil, nor freedom. But it is built and used by human beings and because of that non rational behaviour can be involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start with a real life example more related to general IT than to AI. I will speak of viruses or other malwares. Computers are rather stupid machines that are good to quickly process data. So most people rely on them. An some (bad) people develop malwares that will disrupt the correct behaviour of computers. And we all know that they can have terrible effects on small to medium organizations that are not well prepared to an computer loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is computer based so it is vulnerable to computer type attacks. Here my example would be an AI driven car. The technology is almost ready to work. But imagine the effect of a malware making the car trying to attack other people on the road. Even without a direct access to the code of the AI, it can be attacked by &lt;em&gt;side channels&lt;/em&gt;. For example it uses cameras to read signal signs. But because of the way machine learning is implemented, AI generaly does not analyses a scene the same way a human being does. Researchers have shown that it was possible to change a sign in a way that a normal human will still see the original sign, but an AI will see a different one. Imagine now that the sign is the road priority sign...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean is that even if the AI has no evil intents, bad guys can try to make it behave badly. And to more important actions will be delegated to AI (medecine, cars, planes, not speaking of bombs) the higher the risk. Said differently, I do not really fear the AI for itself, but for the way it can be used by humans.&lt;/p&gt;&#xA;" OwnerUserId="29757" LastActivityDate="2019-09-17T12:19:22.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15487" PostTypeId="2" ParentId="15449" CreationDate="2019-09-17T14:10:18.600" Score="4" Body="&lt;p&gt;I think one of the most real (ie. related to current, existing AIs) risks are in blindly relying on unsupervised AIs, for two reasons.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;1. AI systems may degrade&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Physical error in AI systems may start producing wildly wrong results in regions in which they were not tested for because the physical system starts providing wrong values. This is sometimes redeemed by self-testing and redundancy, but still requires occasional human supervision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self learning AIs also have a software weakness - their weight networks or statistic representations may approach local minima where they are stuck with one wrong result.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2. AI systems are biased&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is fortunately frequently discussed, but worth mentioning: AI systems' classification of inputs is often biased because the training/testing dataset were biased as well. This results in AIs not recognizing people of certain ethnicity, for more obvious example. However there are less obvious cases that may only be discovered after some bad accident, such as AI not recognizing certain data and accidentally starting fire in a factory, breaking equipment or hurting people.&lt;/p&gt;&#xA;" OwnerUserId="26069" LastActivityDate="2019-09-17T14:10:18.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15488" PostTypeId="2" ParentId="11889" CreationDate="2019-09-17T14:36:45.017" Score="0" Body="&lt;p&gt;It sums the squared error for the output vs the expected output, this isnt something you need to do for each experiment they are simply telling you the metric they are using as fitness for the genomes in the xor example experiment, in other experiments you could use something else. If you were training it to play video games you set your fitness to be a numerical representation of how well the genome played the game, so you dont always need to have an expected value as long as your fitness function uses a meaningful metric as the fitness value. &lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-09-17T14:36:45.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15490" PostTypeId="1" CreationDate="2019-09-17T16:00:12.110" Score="2" ViewCount="18" Body="&lt;p&gt;I'm reading the notes &lt;a href=&quot;https://web.stanford.edu/class/cs224n/readings/cs224n-2019-notes02-wordvecs2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and have a doubt on page 2 (&quot;Least squares objective&quot; section). The probability of a word &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; occurring in the context of word &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=1}^W\exp(u_w^Tv_i)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notes read:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Training proceeds in an on-line, stochastic fashion, but the implied global cross-entropy loss can be calculated as &lt;span class=&quot;math-container&quot;&gt;$$J=-\sum_{i\in corpus}\sum_{j\in context(i)}\log Q_{ij}$$&lt;/span&gt;&#xA;  As the same words &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; can occur multiple times in the corpus, it is more efficient to first group together the same values for &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;:&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$J=-\sum_{i=1}^W\sum_{j=1}^WX_{ij}\log(Q_{ij})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$X_{ij}$&lt;/span&gt; is the total number of times &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; occurs in the context of &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and the value of co-occuring frequency is given by the co-occurence matrix &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. This much is clear. But then the author states that the denominator of &lt;span class=&quot;math-container&quot;&gt;$Q_{ij}$&lt;/span&gt; is too expensive to compute, so the cross entropy loss won't work.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Instead, we use a least square objective in which the normalization factors in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; are discarded:&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$\hat J=\sum_{i=1}^W\sum_{j=1}^WX_i(\hat P_{ij}-\hat Q_{ij})^2$$&lt;/span&gt;&#xA;  where &lt;span class=&quot;math-container&quot;&gt;$\hat P_{ij}=X_{ij}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat Q_{ij}=\exp(u_j^Tv_i)$&lt;/span&gt; are the unnormalized distributions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X_i=\sum_kX_{ik}$&lt;/span&gt; is the number of times any word appears in the context of &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. I don't understand this part. &lt;em&gt;Why have we introduced &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; out of nowhere? How is &lt;span class=&quot;math-container&quot;&gt;$\hat P_{ij}$&lt;/span&gt; &quot;unnormalized&quot;? Is there a tradeoff in switching from softmax to MSE?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(As far as I know, softmax made total sense in skip gram because we were calculating scores corresponding to different words (discrete possibilities) and matching the predicted output to the actual word - similar to a classification problem, so softmax makes sense.)&lt;/p&gt;&#xA;" OwnerUserId="27548" LastEditorUserId="27548" LastEditDate="2019-09-17T17:06:36.507" LastActivityDate="2019-09-17T17:06:36.507" Title="Doubt on formulating cost function for GloVe" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;glove&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15493" PostTypeId="5" CreationDate="2019-09-17T17:12:14.640" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;http://web.mit.edu/6.435/www/Valiant84.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Theory of the Learnable&lt;/a&gt; (1984) by Leslie G. Valiant.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-17T17:12:14.640" LastActivityDate="2019-09-17T17:12:14.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15494" PostTypeId="4" CreationDate="2019-09-17T17:12:14.640" Score="0" Body="For questions related to Probably Approximately Correct (PAC) learning, a framework for mathematical analysis of machine learning algorithms, which was introduced in the paper &quot;A Theory of the Learnable&quot; (1984) by Leslie G. Valiant." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-17T17:12:14.640" LastActivityDate="2019-09-17T17:12:14.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15496" PostTypeId="2" ParentId="15449" CreationDate="2019-09-17T19:12:45.277" Score="2" Body="&lt;p&gt;Human beings currently exist in an ecological-economic niche of &quot;the thing that thinks&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is also a thing that thinks, so it will be invading our ecological-economic niche.  In both ecology and economics, having something else occupy your niche is not a great plan for continued survival.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly Human survival is compromised by this is going to be pretty chaotic.  There are going to be a bunch of plausible ways that AI could endanger human survival as a species, or even as a dominant life form.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Suppose there is a strong AI without &quot;super ethics&quot; which is cheaper to manufacture than a human (including manufacturing a &quot;body&quot; or way of manipulating the world), and as smart or smarter than a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a case where we start competing with that AI for resources.  It will happen on microeconomic scales (do we hire a human, or buy/build/rent/hire an AI to solve this problem?).  Depending on the rate at which AIs become cheap and/or smarter than people, this can happen slowly (maybe an industry at a time) or extremely fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a capitalist competition, those that don't move over to the cheaper AIs end up out-competed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in the short term, if the AI's advantages are only marginal, the high cost of educating humans for 20-odd years before they become productive could make this process slower.  In this case, it might be worth paying a Doctor above starvation wages to diagnose disease instead of an AI, but it probably isn't worth paying off their student loans.  So new human Doctors would rapidly stop being trained, and existing Doctors would be impoverished.  Then over 20-30 years AI would completely replace Doctors for diagnostic purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI's advantages are large, then it would be rapid.  Doctors wouldn't even be worth paying poverty level wages to do human diagnostics.  You can see something like that happening with muscle-based farming when gasoline-based farming took over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During past industrial revolutions, the fact that humans where able to think means that you could repurpose surplus human workers to do other actions; manufacturing lines, service economy jobs, computer programming, etc.  But in this model, AI is cheaper to train and build and as smart or smarter than humans at that kind of job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As evidenced by the ethanol-induced Arab spring, crops and cropland can be used to fuel both machines and humans.  When machines are more efficient in terms of turning cropland into useful work, you'll start seeing the price of food climb.  This typically leads to riots, as people really don't like starving to death and are willing to risk their own lives to overthrow the government in order to prevent this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can mollify the people by providing subsidized food and the like.  So long as this isn't economically crippling (ie, if expensive enough, it could result in you being out-competed by other places that don't do this), this is merely politically unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an alternative, in the short term, the ownership caste who is receiving profits from the increasingly efficient AI-run economy can pay for a police or military caste to put down said riots.  This requires that the police/military castes be upper lower to middle class in standards of living, in order to ensure continued loyalty -- you don't want them joining the rioters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one of the profit centers you can put AI towards is AI based military and policing.  Drones that deliver lethal and non-lethal ordnance based off of processing visual and other data feeds can reduce the number of middle-class police/military needed to put down food-price triggered riots or other instability.  As we have already assumed said AIs can have bodies and training cheaper than a biological human, this can also increase the amount of force you can deploy per dollar spent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, we are talking about a mostly AI run police and military being used to keep starving humans from overthrowing the AI run economy and seizing the means of production from the more efficient use it is currently being put to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vestigial humans who &quot;own&quot; the system at the top are making locally rational decisions to optimize their wealth and power.  They may or may not persist for long; so long as they drain a relatively small amount of resources and don't mess up the AI run economy, there won't be much selection pressure to get rid of them.  On the other hand, as they are contributing nothing of value, they position &quot;at the top&quot; is politically unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This process assumed a &quot;strong&quot; general AI.  Narrower AIs can pull this off in pieces.  A cheap, effective diagnostic computer could reduce most Doctors into poverty in a surprisingly short period of time, for example.  Self driving cars could swallow 5%-10% of the economy.  Information technology is already swallowing the retail sector with modest AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is said that every technological advancement leads to more and better jobs for humans.  And this has been true for the last 300+ years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But prior to 1900, it was also true that every technological advancement led to more and better jobs for horses.  Then the ICE and automobile arrived, and now there are far fewer working horses; the remaining horses are basically the equivalent of human personal servants: kept for the novelty of &quot;wow, cool, horse&quot; and the fun of riding and controlling a huge animal.&lt;/p&gt;&#xA;" OwnerUserId="29777" LastActivityDate="2019-09-17T19:12:45.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15497" PostTypeId="1" AcceptedAnswerId="15507" CreationDate="2019-09-17T21:32:47.143" Score="3" ViewCount="277" Body="&lt;p&gt;AI experts like Ben Goertzel and Ray Kurzweil say that AGI   will be developed in the coming decade. Are they credible?&lt;/p&gt;&#xA;" OwnerUserId="17601" LastEditorUserId="1671" LastEditDate="2019-09-17T21:34:44.847" LastActivityDate="2021-01-26T13:27:46.550" Title="Is AGI likely to be developed in the next decade?" Tags="&lt;agi&gt;&lt;futurism&gt;" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15499" PostTypeId="2" ParentId="13335" CreationDate="2019-09-18T01:55:16.900" Score="0" Body="&lt;p&gt;The paper you link to in the question is paywalled for me, so this answer may not be specific enough, but we'll make do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamentally, AUC is about measuring a classifier's &lt;strong&gt;robustness&lt;/strong&gt;. The intuition behind the measure is that if we have classifier that outputs a &lt;em&gt;score&lt;/em&gt; for an example, rather than a class label alone, we can choose to interpret that score in ways that trade detection rate (true positives) for false positive rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a very simple example, imagine that we have a classifier that outputs a &lt;em&gt;probability&lt;/em&gt; that an example pair is a link. By default we might chose to interpret this a probability of more than 0.5 as a link being present. Suppose this produces a certain true positive and false positive rate. Now let's change our mind. It's suddenly very important that we detect &lt;em&gt;all&lt;/em&gt; the links, even if we get a lot of false positives. We choose to interpret any probability value &gt; 0 as being a link. Our true positive rate will spike, but so will our false positive rate. Finally, if we care only about minimizing the prediction of fake links, we could chose to interpret only probability values that are &gt;0.99 as being links. Our true positive rate will go way down, but our false positive rate will too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AUC quantifies how rapidly the false positive rate increases as we increase the true positive rate by changing our interpretation of the scores the classifier outputs. Better classifiers will generally be able to increase their true positive rates quickly without introducing many false positives, while worse classifiers will not. An AUC of 1.0 indicates that we can achieve 100% true positive rate without increasing the false positive rate at all. An AUC of 0.0 indicates that increasing the true positive rate by 1% will always result in an increase of 1% in the false positive rate (i.e. the classifier is essentially just as good as a random guess without looking at the input).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your link prediction problem, we cannot directly measure the false positive rate. However, if we treat &lt;strong&gt;all&lt;/strong&gt; non-links as negative examples when we compute this measure, a classifier with a higher AUC will is one that can more rapidly separate positive examples from non-positive ones as you increase your sensitivity. Even though we aren't sure which examples are false positives and which are unseen links, a classifier with a high AUC is one where we would be more likely to trust that the non-links it labels as links are actually links we can't see.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-18T01:55:16.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15501" PostTypeId="1" CreationDate="2019-09-18T09:04:56.930" Score="2" ViewCount="37" Body="&lt;p&gt;I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. &lt;a href=&quot;https://github.com/CorentinJ/Real-Time-Voice-Cloning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/CorentinJ/Real-Time-Voice-Cloning&lt;/a&gt;), but I would like to go beyond that and clone a voice even more precisely (with subsequent text-to-speech using that voice). It doesn't matter if I have to provide minutes or hours of voice inputs.&lt;/p&gt;&#xA;" OwnerUserId="29792" LastActivityDate="2019-09-18T09:04:56.930" Title="What is the State-of-the-Art open source Voice Cloning tool right now?" Tags="&lt;natural-language-processing&gt;&lt;speech-synthesis&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15502" PostTypeId="1" AcceptedAnswerId="15530" CreationDate="2019-09-18T10:22:05.530" Score="0" ViewCount="37" Body="&lt;p&gt;Usually for DNN, I have the training data of matching X (2D) to Y (2D), for example, XOR data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = [[0,0],[0,1],[1,0],[1,1]];&#xA;Y = [[0],  [1],  [1],  [0]  ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, RNN seems strange, I don't get it how to match X to Y, input of RNN layer is 3D and output is 2D (rightclick to open in new tab): &lt;a href=&quot;https://colab.research.google.com/drive/17IgFuxOYgN5fNO9LKwDijEBkIeWNPas6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/17IgFuxOYgN5fNO9LKwDijEBkIeWNPas6&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf;&#xA;&#xA;x = [[[1],[2],[3]], [[4],[5],[6]]];&#xA;bsize = 2;&#xA;times = 3;&#xA;&#xA;#3d input&#xA;input = tf.placeholder(tf.float32, [bsize,times,1]);&#xA;&#xA;cell  = tf.keras.layers.LSTMCell(20);&#xA;rnn   = tf.keras.layers.RNN(cell);&#xA;hid   = rnn(input);&#xA;&#xA;sess = tf.Session();&#xA;init = tf.global_variables_initializer();&#xA;sess.run(init);&#xA;&#xA;#results in 2d&#xA;print(sess.run(hid, {input:x}));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The example data seen on &lt;a href=&quot;https://www.tensorflow.org/tutorials/sequences/recurrent&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/sequences/recurrent&lt;/a&gt; are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; t=0  t=1    t=2  t=3     t=4&#xA;[the, brown, fox, is,     quick]&#xA;[the, red,   fox, jumped, high]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to map these data &lt;strong&gt;from X (3D&lt;/strong&gt; input for RNN layer) &lt;strong&gt;to Y (2D&lt;/strong&gt;)? (Y is 2D because RNN layer output is 2D).&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-09-18T10:44:11.033" LastActivityDate="2019-09-20T10:33:20.557" Title="How to map X to Y for TensorFlow RNN training data" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15503" PostTypeId="2" ParentId="15449" CreationDate="2019-09-18T11:00:28.847" Score="1" Body="&lt;p&gt;AI that is used to solve a real world problem could pose a risk to humanity and doesn't exactly require sentience, this also requires a degree of human stupidity too.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike humans, an AI would find the most logical answer without the constraint of emotion, ethics, or even greed... Only logic. Ask this AI how to solve a problem that humans created (for example, Climate Change) and it's solution might be to eliminate the entirety of the human race to protect the planet. Obviously this would require giving the AI the ability to act upon it's outcome which brings me to my earlier point, human stupidity.&lt;/p&gt;&#xA;" OwnerUserId="29795" LastActivityDate="2019-09-18T11:00:28.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15504" PostTypeId="1" CreationDate="2019-09-18T11:23:55.177" Score="3" ViewCount="88" Body="&lt;p&gt;I would like to develop a platform in which people will write text and upload images. I am going to use Google API to classify the text and extract from the image all kinds of metadata. In the end, I am going to have a lot of text which describes the content (text and images). Later, I would like to show my users related posts (that is, similar posts, from the content point of view).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most ppropriate way of doing this? I am not an AI expert and the best approach from my prescriptive it to have some tools, like google API or Apache Lucene search engine, which can hide the details of how this is done.&lt;/p&gt;&#xA;" OwnerUserId="29799" LastEditorUserId="2444" LastEditDate="2019-11-15T19:51:05.340" LastActivityDate="2020-08-18T05:53:37.207" Title="What is the best way to find the similarities between two text documents?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;recommender-system&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15505" PostTypeId="2" ParentId="15504" CreationDate="2019-09-18T13:58:58.453" Score="0" Body="&lt;p&gt;Google has introduced &lt;a href=&quot;https://ai.googleblog.com/2019/07/multilingual-universal-sentence-encoder.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Sentence Encoder&lt;/a&gt;, which converts sentences into vector representations while preserving the semantic details. The pre-trained models are available on &lt;a href=&quot;https://tfhub.dev/google/universal-sentence-encoder/3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow Hub&lt;/a&gt;. The &lt;a href=&quot;https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/semantic_similarity_with_tf_hub_universal_encoder.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colab notebook&lt;/a&gt; would help you get started as well.&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="2444" LastEditDate="2019-11-15T19:47:08.883" LastActivityDate="2019-11-15T19:47:08.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15506" PostTypeId="1" CreationDate="2019-09-18T14:09:41.747" Score="1" ViewCount="174" Body="&lt;p&gt;I have made sure that layers,parameters, hyperparameters,kernel_initialization, bias_initialization, seed and dataset are all equal. But still the output for both the models are different.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function, absolute_import&#xA;import tensorflow.keras.datasets.mnist as mnist&#xA;import tensorflow as tf&#xA;tf.reset_default_graph()&#xA;tf.random.set_random_seed(43)&#xA;from input_data import Dataset&#xA;# Import MNIST data&#xA;import input_data&#xA;# mnist = input_data_old.read_data_sets(&quot;MNSITData/raw/&quot;, one_hot=True)&#xA;&#xA;# Seed value&#xA;# Apparently you may use different seed values at each stage&#xA;seed_value= 43&#xA;&#xA;# 1. Set the `PYTHONHASHSEED` environment variable at a fixed value&#xA;import os&#xA;os.environ['PYTHONHASHSEED']=str(seed_value)&#xA;&#xA;# 2. Set the `python` built-in pseudo-random generator at a fixed value&#xA;import random&#xA;random.seed(seed_value)&#xA;&#xA;# 3. Set the `numpy` pseudo-random generator at a fixed value&#xA;import numpy as np&#xA;np.random.seed(seed_value)&#xA;&#xA;# 4. Set the `tensorflow` pseudo-random generator at a fixed value&#xA;tf.set_random_seed(seed_value)&#xA;# (x_train, y_train), (x_test, y_test) = mnist.load_data()&#xA;# x_train, x_test = x_train / 255, x_test / 255&#xA;&#xA;# # Add a channels dimension&#xA;# x_train = x_train[..., None]&#xA;# x_test = x_test[..., None]&#xA;# train_ds = tf.data.Dataset.from_tensor_slices(&#xA;#     (x_train, y_train)).batch(64)&#xA;# test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(64)&#xA;# Training Parameters&#xA;learning_rate = 0.001&#xA;batch = 5&#xA;display_step = 10&#xA;&#xA;dataset={&#xA;    &quot;file_path&quot;:&quot;/home/pat-011/Desktop/deep_learning/playground/flower_photos&quot;,&#xA;    &quot;ratio&quot;:(0.8,0.1,0.1),&#xA;    &quot;target_size&quot;:(150,150),&#xA;    &quot;color_mode&quot;:&quot;rgb&quot;,&#xA;    &quot;class_mode&quot;:'sparse'&#xA;}&#xA;train,test,val_ds= Dataset(dataset)()&#xA;np.save(&quot;train&quot;,train[0])&#xA;np.save(&quot;test&quot;,test[0])&#xA;&#xA;# Network Parameters&#xA;num_classes = 5# MNIST total classes (0-9 digits)&#xA;dropout = 0.25 # Dropout, probability to keep units&#xA;&#xA;# tf Graph input&#xA;X = tf.placeholder(tf.float32, [None, 150,150,3])&#xA;Y = tf.placeholder(tf.float32, [None, num_classes])&#xA;keep_prob = tf.placeholder(tf.float32) # dropout (keep probability)&#xA;&#xA;&#xA;# Create some wrappers for simplicity&#xA;def conv2d(x, W, b, strides=1):&#xA;    print(&quot;shape of x&quot;,x.shape.as_list(),type(x.shape.as_list()),x.shape.as_list()[0],x.shape.as_list()[1])&#xA;    # Conv2D wrapper, with bias and relu activation&#xA;    tf.compat.v1.summary.scalar('conv_weights', W )&#xA;    tf.compat.v1.summary.scalar('conv_bias', b)&#xA;    x = tf.nn.conv2d(x, W, strides=[1, strides, strides, 1], padding='VALID')&#xA;    x = tf.nn.bias_add(x, b)&#xA;    return tf.nn.relu(x)&#xA;&#xA;def maxpool2d(x, k=2):&#xA;    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, k, k, 1],&#xA;                          padding='VALID')&#xA;&#xA;&#xA;# Create model&#xA;def conv_net(x, weights, biases):&#xA;    # MNIST data input is a 1-D vector of 784 features (128*128 pixels)&#xA;    # Reshape to match picture format [Height x Width x Channel]&#xA;    # Tensor input become 4-D: [Batch Size, Height, Width, Channel]&#xA;    print(type(x.shape[-1]),x.shape[-1],type(int(x.shape[-1])))&#xA;    x = tf.reshape(x, shape=[-1, 150, 150, 3])&#xA;    # Convolution Layerint()&#xA;    conv1 = conv2d(x, weights['wc1'], biases['bc1'])&#xA;    tf.compat.v1.summary.scalar(conv1.name, conv1)&#xA;    # Max Pooling (down-sampling)&#xA;    print(conv1.shape)&#xA;    conv1 = maxpool2d(conv1, k=2)&#xA;    print(conv1.shape)&#xA;    # Convolution Layer&#xA;    conv2 = conv2d(conv1, weights['wc2'], biases['bc2'])&#xA;    # Max Pooling (down-sampling)&#xA;    print(conv2.shape)&#xA;    conv2 = maxpool2d(conv2, k=2)&#xA;    print(conv2.shape)&#xA;&#xA;    # Fully connected layer&#xA;    # Reshape conv2 output to fit fully connected layer input&#xA;    print(&quot;weights_____________________________&quot;,[-1, weights['wd1'].get_shape().as_list()[0]])&#xA;&#xA;    fc1 = tf.reshape(conv2, [-1, weights['wd1'].get_shape().as_list()[0]])&#xA;    print(fc1.shape,)&#xA;    fc1 = tf.add(tf.matmul(fc1, weights['wd1']), biases['bd1'])&#xA;    fc1 = tf.nn.relu(fc1)&#xA;    # Apply Dropout&#xA;    print(&quot;dropout++++++++++++++++++++++++++++++++++&quot;,dropout)&#xA;    fc1 = tf.nn.dropout(fc1, rate = dropout,seed=43)&#xA;    print(fc1.shape)&#xA;&#xA;    fc1 = tf.reshape(fc1, [-1, weights['out'].get_shape().as_list()[0]])&#xA;    out = tf.add(tf.matmul(fc1, weights['out']), biases['out'])&#xA;    tf.compat.v1.summary.scalar('dense_weights', weights['out'] )&#xA;    tf.compat.v1.summary.scalar('dense_bias', biases['out'])&#xA;    print(out.shape)&#xA;    tf.compat.v1.summary.scalar(out.name, out)&#xA;    return out&#xA;&#xA;# Store layers weight &amp;amp; bias&#xA;weights = {&#xA;    # 5x5 conv, 1 input, 32 outputs&#xA;    'wc1': tf.Variable(tf.random_normal([5, 5, 3, 32],mean=0.0,&#xA;    stddev=1.0, seed=43)),&#xA;    # 'wc1': tf.Variable(np.load(&quot;conv_w.npy&quot;)),&#xA;    'wc2': tf.Variable(tf.random_normal([5, 5, 32, 64],mean=0.0,&#xA;    stddev=1.0, seed=43)),&#xA;    # fully connected, 7*7*64 inputs, 1024 outputs&#xA;    'wd1': tf.Variable(tf.random_normal([34*34*64, 1024],mean=0.0,&#xA;    stddev=1.0, seed=43)),&#xA;    # 1024 inputs, 10 outputs (class prediction)&#xA;    # 'out': tf.Variable(np.load(&quot;dense_w.npy&quot;))&#xA;    'out': tf.Variable(tf.random_normal([1024, num_classes],mean=0.0,&#xA;    stddev=1.0, seed=43))&#xA;}&#xA;&#xA;biases = {&#xA;    'bc1': tf.Variable(tf.random_normal([32],mean=0.0,&#xA;    stddev=1.0, seed=43)),&#xA;    # 'bc1': tf.Variable(np.load(&quot;conv_b.npy&quot;)),&#xA;    'bc2': tf.Variable(tf.random_normal([64],mean=0.0,&#xA;    stddev=1.0, seed=43)),&#xA;    'bd1': tf.Variable(tf.random_normal([1024],mean=0.0,&#xA;    stddev=1.0, seed=43)),&#xA;    'out': tf.Variable(tf.random_normal([num_classes], mean=0.0,&#xA;    stddev=1.0, seed=43))&#xA;    # 'out': tf.Variable(np.load(&quot;dense_b.npy&quot;))&#xA;}&#xA;print(weights,biases)&#xA;# Construct model&#xA;logits = conv_net(X, weights, biases)&#xA;prediction = tf.nn.softmax(logits)&#xA;&#xA;# Define loss and optimizer&#xA;loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(&#xA;    logits=logits, labels=Y))&#xA;optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)&#xA;train_op = optimizer.minimize(loss_op)&#xA;tf.compat.v1.summary.scalar('loss_op', loss_op)&#xA;&#xA;# Evaluate model&#xA;correct_pred = tf.equal(tf.argmax(prediction, 1), tf.argmax(Y, 1))&#xA;accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))&#xA;tf.compat.v1.summary.scalar('accuracy', accuracy)&#xA;&#xA;# Merge all the summaries and write them out to&#xA;# /tmp/tensorflow/mnist/logs/mnist_with_summaries (by default)&#xA;merged = tf.compat.v1.summary.merge_all()&#xA;# Initialize thcorrect_prede variables (i.e. assign their default value)&#xA;init = tf.global_variables_initializer()&#xA;&#xA;# Start training&#xA;with tf.Session() as sess:&#xA;    writer = tf.summary.FileWriter(&quot;/tmp/tf_withoutjson/&quot;,sess.graph)&#xA;    # Run the initializer&#xA;    sess.run(init)&#xA;    for i in range(5):&#xA;        for step in range(1, int(len(train[0])/batch)):&#xA;            # Run optimization op (backprop)&#xA;            offset = (step * batch) % (train[1].shape[0] - batch)&#xA;            # Generate a minibatch.&#xA;            batch_x = train[0][offset:(offset + batch), :]&#xA;            batch_y = train[1][offset:(offset + batch), :]&#xA;            # np.save(&quot;arrays/xtrain&quot;+str(step),batch_x)&#xA;            # np.save(&quot;arrays/ytrain&quot;+str(step),batch_y)&#xA;            sess.run(train_op, feed_dict={X: batch_x, Y: batch_y})&#xA;            if step % display_step == 0 or step == 1:&#xA;                # Calculate batch loss and accuracy&#xA;&#xA;                loss, acc ,cw,cb,dw,db= sess.run([loss_op, accuracy,weights[&quot;wc1&quot;],biases[&quot;bc1&quot;],weights[&quot;out&quot;],biases['out']], feed_dict={X: batch_x,&#xA;                                                                    Y: batch_y&#xA;                                                                    })&#xA;                np.save(&quot;weightBias/conv_w&quot;+str(step),cw)&#xA;                np.save(&quot;weightBias/conv_b&quot;+str(step),cb)&#xA;                np.save(&quot;weightBias/dense_w&quot;+str(step),dw)&#xA;                np.save(&quot;weightBias/dense_b&quot;+str(step),db)&#xA;                print(&quot;Step &quot; + str(step) + &quot;, Minibatch Loss= &quot; + \&#xA;                    &quot;{:.4f}&quot;.format(loss) + &quot;, Training Accuracy= &quot; + \&#xA;                    &quot;{:.3f}&quot;.format(acc))&#xA;&#xA;    print(&quot;Optimization Finished!&quot;)&#xA;&#xA;    # Calculate accuracy for 256 MNIST test images&#xA;    print(&quot;Testing Accuracy:&quot;, \&#xA;        sess.run(accuracy, feed_dict={X: test[0],&#xA;                                      Y: test[1],&#xA;                                      keep_prob: 0.75}))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the tensorflow flow and I have also saved the weights generated in each layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the keras flow,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import keras&#xA;from keras.layers import *&#xA;from keras.models import Model&#xA;from keras import optimizers&#xA;from keras.datasets import mnist&#xA;from keras import backend as K&#xA;from keras.utils import multi_gpu_model&#xA;import tensorflow as tf&#xA;from input_data import Dataset&#xA;from numpy import sqrt&#xA;from keras import initializers&#xA;&#xA;# Seed value&#xA;# Apparently you may use different seed values at each stage&#xA;seed_value= 43&#xA;&#xA;# 1. Set the `PYTHONHASHSEED` environment variable at a fixed value&#xA;import os&#xA;os.environ['PYTHONHASHSEED']=str(seed_value)&#xA;&#xA;# 2. Set the `python` built-in pseudo-random generator at a fixed value&#xA;import random&#xA;random.seed(seed_value)&#xA;&#xA;# 3. Set the `numpy` pseudo-random generator at a fixed value&#xA;import numpy as np&#xA;np.random.seed(seed_value)&#xA;&#xA;# 4. Set the `tensorflow` pseudo-random generator at a fixed value&#xA;import tensorflow as tf&#xA;tf.set_random_seed(seed_value)&#xA;&#xA;# 5. Configure a new global `tensorflow` session&#xA;from keras import backend as K&#xA;session_conf = tf.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1)&#xA;sess = tf.Session(graph=tf.get_default_graph(), config=session_conf)&#xA;K.set_session(sess)&#xA;&#xA;&#xA;img_rows, img_cols = 150, 150&#xA;&#xA;dataset={&#xA;    &quot;file_path&quot;:&quot;/home/pat-011/Desktop/deep_learning/playground/flower_photos&quot;,&#xA;    &quot;ratio&quot;:(0.8,0.1,0.1),&#xA;    &quot;target_size&quot;:(150,150),&#xA;    &quot;color_mode&quot;:&quot;rgb&quot;,&#xA;    &quot;class_mode&quot;:'sparse'&#xA;}&#xA;&#xA;(x_train, y_train), (x_test, y_test), (x_val, y_val) = Dataset(dataset)()&#xA;&#xA;in_shape = x_train.shape[1]&#xA;out_shape = y_train.shape[1]&#xA;n_train = x_train.shape[0]&#xA;n_test = x_test.shape[0]&#xA;&#xA;&#xA;# def get_network_utils():&#xA;#     dropout_ph = tf.placeholder_with_default(0.25, shape=())&#xA;#     optimizer = tf.train.AdamOptimizer(learning_rate=lr, name='Trainer')&#xA;#     activation = tf.nn.relu&#xA;&#xA;#     def loss(y_true, y_pred):&#xA;#         return tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y_true, logits=y_pred))&#xA;&#xA;#     def accuracy(y_true, y_pred):&#xA;#         correct = tf.equal(tf.argmax(y_true, 1), tf.argmax(y_pred, 1))&#xA;#         return tf.reduce_mean(tf.cast(correct, tf.float32))&#xA;&#xA;#     def weight_initializer(shape, dtype=None, partition_info=None):&#xA;#         init_range = sqrt(6.0 / (shape[0] + shape[1]))&#xA;#         return tf.get_variable('weights', shape=shape, dtype=dtype,&#xA;#                                initializer=tf.random_uniform_initializer(-init_range, init_range))&#xA;&#xA;#     def bias_initializer(shape, dtype=None, partition_info=None):&#xA;#         return tf.Variable(name='bias', initial_value=tf.random_normal(shape))&#xA;&#xA;#     return (dropout_ph, optimizer, activation, loss, accuracy,&#xA;#             weight_initializer, bias_initializer)&#xA;&#xA;x_train = x_train.astype('float32')&#xA;x_test = x_test.astype('float32')&#xA;x_train /= 255&#xA;x_test /= 255&#xA;print('x_train shape:', x_train.shape)&#xA;print(x_train.shape[0], 'train samples')&#xA;print(x_test.shape[0], 'test samples')&#xA;&#xA;lr = 0.001&#xA;&#xA;# _, optimizer, activation, loss, accuracy, weight_initializer, bias_initializer = get_network_utils()&#xA;&#xA;# # convert class vectors to binary class matrices&#xA;# y_train = keras.utils.to_categorical(y_train, num_classes)&#xA;# y_test = keras.utils.to_categorical(y_test, num_classes)&#xA;&#xA;inputs = Input(shape=(150, 150, 3))&#xA;x = Conv2D(32, (5, 5), activation='relu', strides = (1, 1), padding = 'VALID', kernel_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0,seed = 43), bias_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43))(inputs)&#xA;x = MaxPool2D(pool_size=2, strides = (2, 2), padding = 'VALID')(x)&#xA;x = Conv2D(64, (5, 5),activation='relu', strides = (1, 1), padding = 'VALID',kernel_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43), bias_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43))(x)&#xA;x = MaxPool2D(pool_size=2, strides = (2, 2), padding = 'VALID')(x)&#xA;x = Flatten()(x)&#xA;x = Dense(1024, activation='relu', kernel_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43), bias_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43))(x)&#xA;x = Dropout(rate = 0.25, seed = 43)(x)&#xA;predictions = Dense(5, activation='softmax', kernel_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43), bias_initializer=initializers.RandomNormal(mean=0.0,&#xA;    stddev=1.0, seed = 43))(x)&#xA;model = Model(inputs=inputs, outputs=predictions, name='mnist_model')&#xA;&#xA;model.compile(loss='categorical_crossentropy',&#xA;            optimizer=optimizers.Adam(),&#xA;            metrics=['accuracy'])&#xA;&#xA;history = model.fit(x_train, y_train,&#xA;                    batch_size=5,&#xA;                    epochs=10)&#xA;model.summary()&#xA;&#xA;test_scores = model.evaluate(x_test, y_test, verbose=0)&#xA;print('Test loss:', test_scores[0])&#xA;print('Test accuracy:', test_scores[1])&#xA;&#xA;weights = []&#xA;for layer in model.layers:&#xA;    weights = layer.get_weights()&#xA;    np.save(&quot;../weightBias/&quot;+layer.name,weights)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each layer weights are different though initial weights are initialized with the same random distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please someone point out why I am getting different output for the same model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow accuracy-&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Step 1, Minibatch Loss= 68458.3750, Training Accuracy= 0.800&#xA;Step 10, Minibatch Loss= 451470.5625, Training Accuracy= 0.200&#xA;Step 20, Minibatch Loss= 582654.3750, Training Accuracy= 0.200&#xA;Step 30, Minibatch Loss= 185989.4219, Training Accuracy= 0.400&#xA;Step 1, Minibatch Loss= 161536.2188, Training Accuracy= 0.600&#xA;Step 10, Minibatch Loss= 286743.1875, Training Accuracy= 0.400&#xA;Step 20, Minibatch Loss= 206555.5000, Training Accuracy= 0.600&#xA;Step 30, Minibatch Loss= 201791.6250, Training Accuracy= 0.800&#xA;Step 1, Minibatch Loss= 212732.3438, Training Accuracy= 0.600&#xA;Step 10, Minibatch Loss= 108325.5469, Training Accuracy= 0.800&#xA;Step 20, Minibatch Loss= 80389.1641, Training Accuracy= 0.800&#xA;Step 30, Minibatch Loss= 34269.3750, Training Accuracy= 0.800&#xA;Step 1, Minibatch Loss= 134766.6406, Training Accuracy= 0.400&#xA;Step 10, Minibatch Loss= 166675.9531, Training Accuracy= 0.400&#xA;Step 20, Minibatch Loss= 66996.0938, Training Accuracy= 0.600&#xA;Step 30, Minibatch Loss= 29773.1660, Training Accuracy= 0.800&#xA;Step 1, Minibatch Loss= 33207.5234, Training Accuracy= 0.600&#xA;Step 10, Minibatch Loss= 154839.6250, Training Accuracy= 0.800&#xA;Step 20, Minibatch Loss= 23777.3125, Training Accuracy= 0.800&#xA;Step 30, Minibatch Loss= 57576.0742, Training Accuracy= 0.400&#xA;Optimization Finished!&#xA;Testing Accuracy: 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Keras Accuracy -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;156/156 [==============================] - 6s 41ms/step - loss: 13.0185 - acc: 0.1923 &#xA;Epoch 2/10&#xA;156/156 [==============================] - 3s 18ms/step - loss: 12.9151 - acc: 0.1987&#xA;Epoch 3/10&#xA;156/156 [==============================] - 3s 18ms/step - loss: 13.1218 - acc: 0.1859&#xA;Epoch 4/10&#xA;156/156 [==============================] - 3s 18ms/step - loss: 12.9151 - acc: 0.1987&#xA;Epoch 5/10&#xA;156/156 [==============================] - 3s 19ms/step - loss: 13.1218 - acc: 0.1859&#xA;Epoch 6/10&#xA;156/156 [==============================] - 3s 19ms/step - loss: 12.9151 - acc: 0.1987&#xA;Epoch 7/10&#xA;156/156 [==============================] - 3s 19ms/step - loss: 12.8118 - acc: 0.2051&#xA;Epoch 8/10&#xA;156/156 [==============================] - 3s 19ms/step - loss: 12.8118 - acc: 0.2051&#xA;Epoch 9/10&#xA;156/156 [==============================] - 3s 19ms/step - loss: 12.8118 - acc: 0.2051&#xA;Epoch 10/10&#xA;156/156 [==============================] - 3s 19ms/step - loss: 12.9151 - acc: 0.1987&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28417" LastActivityDate="2019-09-23T07:58:09.307" Title="Tensorflow and Keras model having same parameters, hyperparameters, weight and bias initialization give different accuracy?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2019-09-23T19:11:33.010" ContentLicense="CC BY-SA 4.0" />
  <row Id="15507" PostTypeId="2" ParentId="15497" CreationDate="2019-09-18T14:26:04.467" Score="5" Body="&lt;p&gt;As a riff on my answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/7875/is-the-singularity-something-to-be-taken-seriously/7888#7888&quot;&gt;this question&lt;/a&gt;, which is about the broader concern of the development of the singularity, rather than the narrower concern of the development of AGI:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can say that among AI researchers I interact with, it far more common to view the development of AGI in the next decade as speculation (or even wild speculation) than as settled fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is borne out by &lt;a href=&quot;https://nickbostrom.com/papers/survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;surveys of AI researchers&lt;/a&gt;, with 80% thinking &quot;The earliest that machines will be able to simulate learning and every other&#xA;aspect of human intelligence&quot; is in &quot;more than 50 years&quot; or &quot;never&quot;, and just a few percent thinking that such forms of AI are &quot;near&quot;. It's possible to quibble over what exactly is meant by AGI, but it seems likely that for us to reach AGI, we'd need to simulate human-level intelligence in at least most of its aspects. The fact that AI researchers think this is very far off suggests that they also think AGI is not right around the corner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the reasons AI researchers are less optimistic about AGI than Kurzweil or others in tech (but not in AI), are rooted in the fact that we still don't have a good understanding of what human intelligence &lt;em&gt;is&lt;/em&gt;. It's difficult to simulate something that we can't pin down. Another factor is that most AI researchers have been working in AI for a long time. There are countless past proposals for AGI frameworks, and &lt;em&gt;all&lt;/em&gt; of them have been not just wrong, but in the end, more or less hopelessly wrong. I think this creates an innate skepticism of AGI, which may perhaps be unfair. Nonetheless, expert opinion on this one is pretty well settled: no AGI this decade, and maybe not ever!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-09-18T17:54:32.817" LastActivityDate="2019-09-18T17:54:32.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15508" PostTypeId="1" AcceptedAnswerId="15511" CreationDate="2019-09-18T14:39:30.677" Score="2" ViewCount="91" Body="&lt;p&gt;Let's say that we have three actions. The highest-valued action of the three choices is the first. When training the DQN, what do we do with the other two, as we don't have a target for them, since they weren't taken?&lt;/p&gt;&#xA;&lt;p&gt;I've seen some code that leaves the target for off actions as whatever the prediction returned, which feels a bit wrong to me as two or more similar behaving actions might never be differentiated well after random action selection dwindles.&lt;/p&gt;&#xA;&lt;p&gt;I've also seen some implementations that set the target for all actions to zero and only adjust the target for the action taken. This would help regarding action differentiation long term, but it also puts more reliance on taking random actions for any unfamiliar states (I believe) as an off action might never be taken otherwise.&lt;/p&gt;&#xA;" OwnerUserId="29806" LastEditorUserId="2444" LastEditDate="2020-12-05T14:22:12.063" LastActivityDate="2020-12-05T14:22:12.063" Title="When training a DQN, how should we update the value of actions that were not taken?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15509" PostTypeId="1" AcceptedAnswerId="15514" CreationDate="2019-09-18T15:05:43.263" Score="4" ViewCount="79" Body="&lt;p&gt;At least at some level, maybe not end-to-end always, but deep learning always learns a function, essentially a mapping from a domain to a range. The domain and range, at least in most cases, would be multi-variate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, when a model learns a mapping, considering every point in the domain-space has a mapping, does it try to learn a continuous distribution based on the training-set and its corresponding mappings, and map unseen examples from this learned distribution? Could this be said about all predictive algorithms? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, then could binary classification be compared to having a hyper-plane (as in support vector classification) in a particular kernel-space, and could the idea of classification problems using hyper-planes be extended in general to any deep learning problem learning a mapping? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would also explain why deep learning needs a lot of data and why it works better than other learning algorithms for simple problems.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2020-02-20T13:34:44.630" LastActivityDate="2020-06-24T10:45:26.147" Title="In deep learning, do we learn a continuous distribution based on the training dataset?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;probability-distribution&gt;&lt;computational-learning-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15510" PostTypeId="1" CreationDate="2019-09-18T15:29:43.083" Score="6" ViewCount="79" Body="&lt;p&gt;I was reading the paper &lt;a href=&quot;https://arxiv.org/pdf/1609.05566.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Label-Free Supervision of Neural Networks with Physics and Domain Knowledge&lt;/a&gt;, published at AAAI 2017, which won the best paper award.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the math and it makes sense. Consider the first application shown in the paper of tracking falling objects. They train only on multiple trajectories of the said pillow, and during the evaluation, they claim that they can track any other falling object (which may not be pillows).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unable to understand how that happens? How does the network know which object to track? Even during the training, how does it know that it's the pillow that it's supposed to track?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network is trained to fit a parabola. But any parabola could fit it. There are infinite such parabolas.&lt;/p&gt;&#xA;" OwnerUserId="29809" LastEditorUserId="2444" LastEditDate="2019-09-25T21:15:27.713" LastActivityDate="2021-02-17T00:04:10.380" Title="How does the network know which objects to track in the paper &quot;Label-Free Supervision of Neural Networks with Physics and Domain Knowledge&quot;?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15511" PostTypeId="2" ParentId="15508" CreationDate="2019-09-18T16:06:31.723" Score="3" Body="&lt;p&gt;The loss function for DQN algorithm is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;L(\theta_i) = \mathbb E_{s, a, r, s'} [(y - Q(s, a;\theta_i))^2]&#xA;\end{equation}&lt;/span&gt;&#xA;Like you said, we only take one action per timestep. We can only shift weights of the network that had the effect in calculating action value &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for that particular action that we took. For that action, variable &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; would have value&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;y = r + \gamma \max_{a'} Q(s', a', \theta^-_i)&#xA;\end{equation}&lt;/span&gt;&#xA;and we would have standard form of DQN loss. We calculate the gradient of that loss with respect to network parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, backprop it and slightly shift weights so to more accurately estimate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for that specific state-action pair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We never took other actions. Since we never took them we didn't get reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and we can't estimate &lt;span class=&quot;math-container&quot;&gt;$\max_{a'} Q(s', a', \theta^-_i)$&lt;/span&gt; for those other actions. We don't want to change weights that had effect on calculating actions values for those other actions because we have no way of estimating how accurate they were. Standard approach is to set&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;y = Q(s, a;\theta_i)&#xA;\end{equation}&lt;/span&gt;&#xA;this way the loss for those other actions would be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and it would result in not changing the weights that had influence in calculating their action values.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setting target &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for all other actions would mean that we want &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for them all to slightly shift to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. That would not be correct since we have no way of knowing their true value. I think you misinterpreted that part in the implementations.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-09-18T16:06:31.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15512" PostTypeId="2" ParentId="13952" CreationDate="2019-09-18T16:18:49.670" Score="1" Body="&lt;p&gt;You can use LSTM in reinforcement learning, of course. You don't give actions to the agent, it doesn't work like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent give actions to your MDP and you must return proper reward in order to teach the agent. For example if you implement trading bot, the policy(policy=the agent, which is your LSTM network) will say that at step T it is going to have action 34, which means something to your MDP and you return reward for example -0.03 or +0.05 or whatever depending what that actions is doing at the moment T. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I get the question like you want to do a supervised learning on a reinforcement learning environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can mimic supervised learning as well, but the idea of reinforcement learning is not that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how to mimic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scenario: you are at step T, lets say you have 3 possible actions -1,0,+1;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a supervised learning you must give the desired action to the learning process.&#xA;In reinforcement learning you must give reward based on if you are happy or not from the agent's action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you must have predefined that for -1 you are not happy and you give reward 0.0, for action 0 you are not happy and you give reward 0.0 and for action +1 you are happy and you give reward +100;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this makes things clear.&lt;/p&gt;&#xA;" OwnerUserId="29812" LastActivityDate="2019-09-18T16:18:49.670" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15513" PostTypeId="2" ParentId="15449" CreationDate="2019-09-18T16:32:22.223" Score="1" Body="&lt;p&gt;In addtion to the many answers already provided, I would bring up the issue of &lt;a href=&quot;https://arxiv.org/pdf/1607.02533.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;&lt;em&gt;adversarial examples&lt;/em&gt;&lt;/strong&gt;&lt;/a&gt; in the area of image models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adversarial examples are images that have been perturbed with specifically designed noise that is often imperceptible to a human observer, but strongly alters the prediction of a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Affecting the predicted diagnosis in a &lt;a href=&quot;https://arxiv.org/pdf/1804.05296.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chest x-ray&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Affecting the &lt;a href=&quot;https://arxiv.org/pdf/1807.07769.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;detection of roadsigns&lt;/a&gt; necessary for autonomous vehicles.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26341" LastActivityDate="2019-09-18T16:32:22.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15514" PostTypeId="2" ParentId="15509" CreationDate="2019-09-18T16:40:08.983" Score="3" Body="&lt;p&gt;Well, there are some questions here...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does it (Deep Learning) try to learn a continuous distribution based&#xA;on the training-set and its corresponding mappings, and map unseen&#xA;examples from this learned distribution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. Talking about Deep Artificial Neural Networks, they try to learn continuous distribution using continuous activation functions in each neuron. Therefore, the output is also a continuous function to represent a continuous probability distribution. The issue with the unseen examples is the need for similar examples in the training set; otherwise, the weights and bias of the network will not be tuned in the regions of space around the unseen example. Imagine a Neural Network learning a function y = x, if we only present values between 0 and 10 during training, we should expect it to only make good predictions for y for values of x ranging from 0 to 10. It doesn't mean that it won't predict for other values, but the predictions will not be so accurate or nowhere close to the expectations. That is because the network is not trying to guess what was the function used to generate y, but it is simply trying to adjust its parameters to make its internal functions generate the expected y for the given x. That is why Deep Neural Networks require a lot of data. In a unidimensional space is easier to provide examples that cover the subset of the domain we want our network to learn. When we use multidimensional space, we need a lot more examples to have a good representation of the hyperspace used as domain.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Could this (map unseen examples) be said about all predictive algorithms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, it should. Otherwise, the algorithm would not be able to generalize well. A good predictive algorithm is the one that can predict unseen examples using fewer training samples.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Could the idea of classification problems using hyper-planes be&#xA;extended in general to any Deep Learning problem learning a mapping?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the case of Deep Neural Networks, the result is more like, for a given input value, return the probability of it belonging to a class. For binary classification, the network will have a single output. The sigmoid function modulates this output to ranges between 0 and 1. We can interpret the output as the probability of belonging to one out of two possible classes. To know the probability for the other class, we subtract it from 1. For three or more classes, we will need three or more outputs ranging from 0 to 1, and each output is the probability of belonging to one of the classes. In this case, the outputs are also normalized by a softmax function, that guarantees that the sum of all outputs is equal to 1, as a probability distribution.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would also explain why Deep Learning needs a lot of data and why it&#xA;works better mostly than other Learning algorithms for simple&#xA;problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Already partially explained... The need for a lot of data is to have a good representation of the hyperplane used as the domain.&#xA;The Deep Neural Networks work well because of their power to represent different models. They are a very 'flexible' functions that can be bent to approximate the relation existent between the data in the training set and the expected target. Simpler algorithms, as linear models, for instance, have less representation power, they are limited to a smaller set of models. Even though many models can be linearly approximated (because the input and output almost follow a linear relation), the neural network will be able to learn the nuances of the dataset better. This can also be the curse of neural networks, because they may try to learn every detail of the training set that wasn't really relevant and true for other cases and this concept is called overtraining... but is a discussion for another topic.&lt;/p&gt;&#xA;" OwnerUserId="29810" LastEditorUserId="29810" LastEditDate="2020-06-24T10:45:26.147" LastActivityDate="2020-06-24T10:45:26.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15515" PostTypeId="1" CreationDate="2019-09-18T18:02:06.907" Score="1" ViewCount="16" Body="&lt;p&gt;I have a dataset of some activities. The dataset contains the status of different sensors and the label of activity. T trained a model in Keras with the following architecture which models the activities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/krrou.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/krrou.png&quot; alt=&quot;an LSTM-based model to predict activities&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I need to make a different LSTM-based model for each activity separately. Then, calculate the probability of creating the sequence by each model. Can I do that with Keras? How should I design the architecture of the model? Is it right to remove the dense layer and then create a separate model for each activity label separately?&lt;/p&gt;&#xA;" OwnerUserId="29645" LastActivityDate="2019-09-18T18:02:06.907" Title="Make an LSTM model for each class separately" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15516" PostTypeId="2" ParentId="11927" CreationDate="2019-09-18T19:10:47.877" Score="1" Body="&lt;p&gt;Empathy is about living and values, just as someone can be a good person, or a bad person, based on their education, culture, past experience and values, just as I who recently became a father became much more empathic about my parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's really a semi-programmed artificial intelligence that captures around you the experience with your programmers, with people, with the environment, etc., and somehow is set to learn from it, I believe you can develop some kind of “ empathy ”, but as Jaume said, it won't be“ natural ”&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have to remember that real artificial intelligence is the one that absorbs and learns, not a set of IF's and ELSE's.&lt;/p&gt;&#xA;" OwnerUserId="29817" LastActivityDate="2019-09-18T19:10:47.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15518" PostTypeId="2" ParentId="11927" CreationDate="2019-09-18T21:10:43.363" Score="2" Body="&lt;p&gt;It seems to me that empathy is based on understanding the experience of another entity:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;originally Psychology. The ability to understand and appreciate another person's feelings, experience, etc. &lt;br&gt;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://www.oed.com/view/Entry/61284?redirectedFrom=empathy#eid&quot; rel=&quot;nofollow noreferrer&quot;&gt;OED&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Using this definition, the AI would have to understand human experience.  (There may be a &quot;Chinese Room&quot; issue in terms of whether one considers the algorithm to truly &quot;understand&quot;.  But, if it can classify the input sufficiently to produce an appropriate response, that can constitute understanding.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying problem is that the algorithm likely doesn't &quot;feel&quot; in the same way humans experience emotions, in that the human experience is colored by chemical response.  So while the algorithm might be able to demonstrate sufficient &quot;understanding&quot; of a human's experience, and act in an empathetic manner, the degree of understanding may always be limited.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-09-18T21:31:55.023" LastActivityDate="2019-09-18T21:31:55.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15519" PostTypeId="5" CreationDate="2019-09-18T21:27:30.503" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-09-18T21:27:30.503" LastActivityDate="2019-09-18T21:27:30.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15520" PostTypeId="4" CreationDate="2019-09-18T21:27:30.503" Score="0" Body="For questions relating to predictions of future technology specifically related to AI and modeling human cognition.  (Includes subjects such as the hypothetical Singularity.)" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-09-18T21:27:30.503" LastActivityDate="2019-09-18T21:27:30.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15521" PostTypeId="2" ParentId="15497" CreationDate="2019-09-18T22:58:29.117" Score="2" Body="&lt;p&gt;I wouldn't take anything Ray Kurzweil says especially seriously. Actual AI experts spend large quantities of time reading the existing scientific literature, and working to expand it. Because Kurzweil doesn't spend much of his time actually &lt;em&gt;learning&lt;/em&gt; about AI, he has plenty of time in which to talk about it. Loudly. This is harmful to research, because 1) a lot of the uninformed predictions he and others make resemble doomsday scenarios, and 2) the predictions of &lt;em&gt;good&lt;/em&gt; things have insanely optimistic time frames attached, and when they don't come true, research funding may be lost because AI hasn't lived up to what people thought it promised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI research has been progressing very rapidly in the last decade, but if we're being honest, a lot of the credit for that has to go to the people who develop research-grade graphics cards. The ability to perform &lt;em&gt;massive&lt;/em&gt; amounts of linear algebra in parallel has allowed us to use techniques that we've known about for a couple decades, but that were too computationally expensive to be practical at the time. And because those techniques are now practical, a lot of current research is applying those techniques to new problems, and modifying and improving them based on what we've learned. (I don't want to understate the contributions here; there have been a lot of &lt;em&gt;really&lt;/em&gt; clever ideas developed in the last ten years. But it's mostly consistent iterative improvement of techniques that already existed, rather than completely revolutionary ideas.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make human-equivalent AIs, we'll probably need to make a few of those giant conceptual leaps. And each of those leaps will then need to be followed up by a decade or two of iterative improvement, because that's how the process works. Case in point, the revolutionary idea that eventually led to all the Deep Learning models out there today was &lt;a href=&quot;https://www.nature.com/articles/323533a0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;, dated 1986. First, there was the revolutionary idea. It was followed up by a bunch of work that built on it and expanded it in new directions. The work eventually stagnated because of hardware constraints. Then hardware scientists and engineers made some advances that let us continue work, and only then did we finally start getting the major applications that we're seeing today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know human-level intelligence is possible, since humans manage it. I have little doubt that we'll figure out how to do it with AI eventually (maybe in my lifetime, maybe not). But if you want Kurzweil's predictions to be even remotely plausible, you might want to add a zero to the end of most of his time frames.&lt;/p&gt;&#xA;" OwnerUserId="2212" LastActivityDate="2019-09-18T22:58:29.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15522" PostTypeId="1" AcceptedAnswerId="15523" CreationDate="2019-09-18T23:19:04.040" Score="3" ViewCount="124" Body="&lt;p&gt;I want to give some examples of AI via movies to my students. There are &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_artificial_intelligence_films&quot; rel=&quot;nofollow noreferrer&quot;&gt;many movies that include AI&lt;/a&gt;, whether being the main character or extras.&lt;/p&gt;&#xA;&lt;p&gt;Which movies have the most realistic (the most possible or at least close to being made in this era) artificial intelligence?&lt;/p&gt;&#xA;" OwnerUserId="22111" LastEditorUserId="2444" LastEditDate="2020-06-27T22:25:59.227" LastActivityDate="2020-06-27T22:25:59.227" Title="Which movies have the most realistic artificial intelligence?" Tags="&lt;applications&gt;&lt;social&gt;&lt;academia&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15523" PostTypeId="2" ParentId="15522" CreationDate="2019-09-18T23:33:21.753" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://ironman.fandom.com/wiki/J.A.R.V.I.S.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Just A Rather Very Intelligent System (J.A.R.V.I.S.)&lt;/a&gt; in &lt;a href=&quot;https://ironman.fandom.com/wiki/Iron_Man_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iron Man&lt;/a&gt; (and related films, such as The Avengers) is something (a personal assistant) that people are already trying to develop, so JARVIS is a quite realistic artificial intelligence. Examples of existing &lt;a href=&quot;https://en.wikipedia.org/wiki/Virtual_assistant&quot; rel=&quot;nofollow noreferrer&quot;&gt;personal assistants&lt;/a&gt; are &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Assistant&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Assistant&lt;/a&gt; (integrated into &lt;a href=&quot;https://en.wikipedia.org/wiki/Google_Home&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Home&lt;/a&gt; devices), &lt;a href=&quot;https://en.wikipedia.org/wiki/Cortana&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cortana&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Siri&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siri&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Amazon_Alexa&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alexa&lt;/a&gt;. There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Virtual_assistant#Comparison_of_notable_assistants&quot; rel=&quot;nofollow noreferrer&quot;&gt;other virtual assistants&lt;/a&gt;, but, unfortunately, there aren't many reliable open-source ones. Note that JARVIS is way more intelligent and capable than the other mentioned personal assistants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, &lt;a href=&quot;https://2001.fandom.com/wiki/HAL_9000&quot; rel=&quot;nofollow noreferrer&quot;&gt;HAL 9000&lt;/a&gt;, in &lt;a href=&quot;https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;2001: A Space Odyssey&lt;/a&gt;, is a sentient artificial intelligence which can be considered a personal assistant.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-18T23:39:28.120" LastActivityDate="2019-09-18T23:39:28.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15524" PostTypeId="1" CreationDate="2019-09-18T23:45:28.737" Score="15" ViewCount="3489" Body="&lt;p&gt;The Transformer model introduced in &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;noreferrer&quot;&gt;&quot;Attention is all you need&quot;&lt;/a&gt; by Vaswani et al. incorporates a so-called position-wise feed-forward network (FFN):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In addition to attention sub-layers, each of the layers in our encoder&#xA;  and decoder contains a fully connected feed-forward network, which is&#xA;  applied to each position separately and identically. This consists of&#xA;  two linear transformations with a ReLU activation in between.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{FFN}(x) = \max(0, x \times {W}_{1} + {b}_{1}) \times {W}_{2} + {b}_{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;While the linear transformations are the same across different positions, they use different parameters from layer to layer. Another way of describing this is as two convolutions with kernel size 1. The dimensionality of input and output is &lt;span class=&quot;math-container&quot;&gt;${d}_{\text{model}} = 512$&lt;/span&gt;, and the inner-layer has dimensionality &lt;span class=&quot;math-container&quot;&gt;${d}_{ff} = 2048$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have seen at least one implementation in Keras that directly follows the convolution analogy. Here is an excerpt from &lt;a href=&quot;https://github.com/Lsdefine/attention-is-all-you-need-keras/blob/master/transformer.py&quot; rel=&quot;noreferrer&quot;&gt;attention-is-all-you-need-keras&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class PositionwiseFeedForward():&#xA;    def __init__(self, d_hid, d_inner_hid, dropout=0.1):&#xA;        self.w_1 = Conv1D(d_inner_hid, 1, activation='relu')&#xA;        self.w_2 = Conv1D(d_hid, 1)&#xA;        self.layer_norm = LayerNormalization()&#xA;        self.dropout = Dropout(dropout)&#xA;    def __call__(self, x):&#xA;        output = self.w_1(x) &#xA;        output = self.w_2(output)&#xA;        output = self.dropout(output)&#xA;        output = Add()([output, x])&#xA;        return self.layer_norm(output)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Yet, in Keras you can apply a single &lt;code&gt;Dense&lt;/code&gt; layer across all time-steps using the &lt;code&gt;TimeDistributed&lt;/code&gt; wrapper (moreover, a simple &lt;code&gt;Dense&lt;/code&gt; layer applied to a 2D input &lt;a href=&quot;https://stackoverflow.com/a/44616780/3846213&quot;&gt;implicitly behaves&lt;/a&gt; like a &lt;code&gt;TimeDistributed&lt;/code&gt; layer). Therefore, in Keras a stack of two Dense layers (one with a ReLU and the other one without an activation) is exactly the same thing as the aforementioned position-wise FFN. So, why would you implement it using convolutions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding benchmarks in response to the answer by @mshlis:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;import typing as t&#xA;os.environ['CUDA_VISIBLE_DEVICES'] = '0'&#xA;&#xA;import numpy as np&#xA;&#xA;from keras import layers, models&#xA;from keras import backend as K&#xA;from tensorflow import Tensor&#xA;&#xA;&#xA;# Generate random data&#xA;&#xA;n = 128000  # n samples&#xA;seq_l = 32  # sequence length&#xA;emb_dim = 512  # embedding size&#xA;&#xA;x = np.random.normal(0, 1, size=(n, seq_l, emb_dim)).astype(np.float32)&#xA;y = np.random.binomial(1, 0.5, size=n).astype(np.int32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Define constructors&#xA;&#xA;def ffn_dense(hid_dim: int, input_: Tensor) -&amp;gt; Tensor:&#xA;    output_dim = K.int_shape(input_)[-1]&#xA;    hidden = layers.Dense(hid_dim, activation='relu')(input_)&#xA;    return layers.Dense(output_dim, activation=None)(hidden)&#xA;&#xA;&#xA;def ffn_cnn(hid_dim: int, input_: Tensor) -&amp;gt; Tensor:&#xA;    output_dim = K.int_shape(input_)[-1]&#xA;    hidden = layers.Conv1D(hid_dim, 1, activation='relu')(input_)&#xA;    return layers.Conv1D(output_dim, 1, activation=None)(hidden)&#xA;&#xA;&#xA;def build_model(ffn_implementation: t.Callable[[int, Tensor], Tensor], &#xA;                ffn_hid_dim: int, &#xA;                input_shape: t.Tuple[int, int]) -&amp;gt; models.Model:&#xA;    input_ = layers.Input(shape=(seq_l, emb_dim))&#xA;    ffn = ffn_implementation(ffn_hid_dim, input_)&#xA;    flattened = layers.Flatten()(ffn)&#xA;    output = layers.Dense(1, activation='sigmoid')(flattened)&#xA;    model = models.Model(inputs=input_, outputs=output)&#xA;    model.compile(optimizer='Adam', loss='binary_crossentropy')&#xA;    return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Build the models&#xA;&#xA;ffn_hid_dim = emb_dim * 4  # this rule is taken from the original paper&#xA;bath_size = 512  # the batchsize was selected to maximise GPU load, i.e. reduce PCI IO overhead&#xA;&#xA;model_dense = build_model(ffn_dense, ffn_hid_dim, (seq_l, emb_dim))&#xA;model_cnn = build_model(ffn_cnn, ffn_hid_dim, (seq_l, emb_dim))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Pre-heat the GPU and let TF apply memory stream optimisations&#xA;&#xA;model_dense.fit(x=x, y=y[:, None], batch_size=bath_size, epochs=1)&#xA;%timeit model_dense.fit(x=x, y=y[:, None], batch_size=bath_size, epochs=1)&#xA;&#xA;model_cnn.fit(x=x, y=y[:, None], batch_size=bath_size, epochs=1)&#xA;%timeit model_cnn.fit(x=x, y=y[:, None], batch_size=bath_size, epochs=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting 14.8 seconds per epoch with the Dense implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 116us/step - loss: 0.6332&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 115us/step - loss: 0.5327&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 117us/step - loss: 0.3828&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 14s 113us/step - loss: 0.2543&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 116us/step - loss: 0.1908&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 116us/step - loss: 0.1533&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 117us/step - loss: 0.1475&#xA;Epoch 1/1&#xA;128000/128000 [==============================] - 15s 117us/step - loss: 0.1406&#xA;&#xA;14.8 s ± 170 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and 18.2 seconds for the CNN implementation. I am running this test on a standard Nvidia RTX 2080.&#xA;So, from a performance perspective there seems to be no point in actually implementing an FFN block as a CNN in Keras. Considering that the maths are the same, the choice boils down to pure aesthetics. &lt;/p&gt;&#xA;" OwnerUserId="29824" LastEditorUserId="29824" LastEditDate="2020-02-12T20:14:05.110" LastActivityDate="2020-07-12T00:02:30.237" Title="Why would you implement the position-wise feed-forward network of the transformer with convolution layers?" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;convolution&gt;&lt;transformer&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15525" PostTypeId="1" CreationDate="2019-09-19T04:49:46.070" Score="3" ViewCount="54" Body="&lt;p&gt;The terms are mentioned in the paper: &lt;a href=&quot;http://jmlr.org/papers/volume17/14-488/14-488.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;“An Emphatic Approach to the Problem of off-Policy Temporal-Difference Learning.” (Sutton, Mahmood, White; 2016)&lt;/a&gt; and more, of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In which paper, they proposed the proof of &quot;stability&quot; but not convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that stability is guarantee if the &quot;key matrix&quot; is shown to be positive definite. However, convergence requires more than that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand the exact difference between the two. &lt;/p&gt;&#xA;" OwnerUserId="9793" LastEditorUserId="2444" LastEditDate="2019-09-19T13:11:33.277" LastActivityDate="2019-09-19T13:11:33.277" Title="What are the differences between stability and convergence in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15526" PostTypeId="1" CreationDate="2019-09-19T05:01:12.347" Score="0" ViewCount="46" Body="&lt;p&gt;Prediction's goal is to get an estimate of a performance of a policy given a specific state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Control's goal is to improve the policy wrt. the prediction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternation between the two is the basis of reinforcement learning algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1606.02647&quot; rel=&quot;nofollow noreferrer&quot;&gt;“Safe and Efficient Off-Policy Reinforcement Learning.” (Munos, 2016)&lt;/a&gt;, the section 3.1) &quot;Policy evaluation&quot; assumes that the target policy is fixed, while the section 3.2) &quot;Control&quot; extends to where the target policy is a sequence of policies improved by a sequence of increasingly greedy operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests that even a proof of convergence is established with a fixed target policy, one cannot immediately imply that of the case where the target policy is a sequence of improving policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder why it is the case. If an algorithm converges under a fixed target policy assumption, any policy during the chain of improvement should have no problem with this algorithm as well. With the merit of policy improvement, each policy in sequence is increasingly better hence converging to an optimal policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should be obvious from the policy improvement perspective and should require no further proof at all?&lt;/p&gt;&#xA;" OwnerUserId="9793" LastEditorUserId="9793" LastEditDate="2019-09-19T08:35:09.800" LastActivityDate="2019-09-22T17:20:20.627" Title="Why doesn't stability in prediction imply stability in control in off-policy reinforcement learning?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15527" PostTypeId="2" ParentId="8971" CreationDate="2019-09-19T05:09:33.353" Score="1" Body="&lt;p&gt;Ignoring HER for now the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; functions operate on states and actions which are part of a Markov decision process which we call &lt;span class=&quot;math-container&quot;&gt;$M_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Back to HER, the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; functions now take a goal as an additional parameter called the goal. We will denote individual goals &lt;span class=&quot;math-container&quot;&gt;$g_n$&lt;/span&gt;, the true goal &lt;span class=&quot;math-container&quot;&gt;$g_0$&lt;/span&gt;, and the set of all goals &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. The set of goals is chosen such that every state matches at least one goal. We create a new MDP &lt;span class=&quot;math-container&quot;&gt;$M_1 = M_0 \times G$&lt;/span&gt; (i.e. a larger MDP composed of multiple copies of &lt;span class=&quot;math-container&quot;&gt;$M_0$&lt;/span&gt;, all the states in each being tagged with a goal). The reward is either +1 or 0 depending on whether or not the goal and the component from &lt;span class=&quot;math-container&quot;&gt;$M_0$&lt;/span&gt; match in some predefined sense. In HER trajectories are collected from the subset of &lt;span class=&quot;math-container&quot;&gt;$M_1$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$g = g_0$&lt;/span&gt; and added to the replay buffer. When training the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; functions we don't only use the original trajectories: we create new ones by substituting new values of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, which we do so strategically so as to include some trajectories with a positive reward.&lt;/p&gt;&#xA;&lt;p&gt;Things to note:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;HER doesn't assign rewards to states or trajectories in &lt;span class=&quot;math-container&quot;&gt;$M_0$&lt;/span&gt;: the reward function used is only defined for &lt;span class=&quot;math-container&quot;&gt;$M_1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;The performance of HER depends on &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; both being models with some ability to extrapolate to unseen data points; such as neural networks, support vector machines, etc. It would not provide any benefit if applied to value tables.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29828" LastEditorUserId="2444" LastEditDate="2020-11-27T15:54:58.027" LastActivityDate="2020-11-27T15:54:58.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15528" PostTypeId="1" CreationDate="2019-09-19T06:50:07.103" Score="1" ViewCount="28" Body="&lt;p&gt;As we all know Keras backend uses Tensorflow and so it should give out same kind of results when we provide same parameters, hyper-parameters, weights and biases initialisation at each layer, but still the accuracy is different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This maybe because of the batches of images which are fed at each step in both the models are not identical and as it gets shuffled randomly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way in which we can make sure that the same batch of images are fed into the model while eliminating the randomness?&lt;/p&gt;&#xA;" OwnerUserId="28417" LastActivityDate="2019-09-19T06:50:07.103" Title="How to get same accuracy with identical models in Keras and Tensorflow?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15529" PostTypeId="1" AcceptedAnswerId="15533" CreationDate="2019-09-19T08:24:49.837" Score="0" ViewCount="585" Body="&lt;p&gt;I'm testing out TensorFlow LSTM layer text generation task, not classification task; but something is wrong with my code, it doesn't converge. What changes should be done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf;&#xA;&#xA;# t=0  t=1    t=2  t=3     &#xA;#[the, brown, fox, is,     quick]&#xA;#   0  1      2    3       4&#xA;#[the, red,   fox, jumps,  high]&#xA;#   0  5      2    6       7&#xA;&#xA;#t0 x=[[the],  [the]]&#xA;#   y=[[brown],[red]]&#xA;#t1 ...&#xA;#t2&#xA;#t3&#xA;bsize = 2;&#xA;times = 4;&#xA;&#xA;#data&#xA;x = [];&#xA;y = [];&#xA;#t0        the:     the:&#xA;x.append([[0/6],   [0/6]]); #normalise: x divided by 6 (max x)&#xA;#          brown:   red:&#xA;y.append([[1/7],   [5/7]]); #normalise: y divided by 7 (max y)&#xA;#t1&#xA;x.append([[1/6],   [5/6]]);&#xA;y.append([[2/7],   [2/7]]);&#xA;#t2&#xA;x.append([[2/6],   [2/6]]);&#xA;y.append([[3/7],   [6/7]]);&#xA;#t3&#xA;x.append([[3/6],   [6/6]]);&#xA;y.append([[4/7],   [7/7]]);&#xA;&#xA;#model&#xA;inputs  = tf.placeholder(tf.float32,[times,bsize,1]) #4,2,1&#xA;exps    = tf.placeholder(tf.float32,[times,bsize,1]);&#xA;&#xA;layer1  = tf.keras.layers.LSTMCell(20) &#xA;hids1,_ = tf.nn.static_rnn(layer1,tf.split(inputs,times),dtype=tf.float32);&#xA;&#xA;w2      = tf.Variable(tf.random_uniform([20,1],-1,1));&#xA;b2      = tf.Variable(tf.random_uniform([   1],-1,1));&#xA;outs    = tf.sigmoid(tf.matmul(hids1,w2) + b2);&#xA;&#xA;loss  = tf.reduce_sum(tf.square(exps-outs))&#xA;optim = tf.train.GradientDescentOptimizer(1e-1)&#xA;train = optim.minimize(loss)&#xA;&#xA;#train&#xA;s    = tf.Session();&#xA;init = tf.global_variables_initializer();&#xA;s.run(init)&#xA;&#xA;feed = {inputs:x, exps:y}&#xA;for i in range(10000):&#xA;  if i%1000==0:&#xA;    lossval = s.run(loss,feed)&#xA;    print(&quot;loss:&quot;,lossval)&#xA;  #end if&#xA;  s.run(train,feed)&#xA;#end for&#xA;&#xA;lastloss = s.run(loss,feed)&#xA;print(&quot;loss:&quot;,lastloss,&quot;(last)&quot;);&#xA;#eof&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output showing loss values (a little different every run):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss: 3.020703&#xA;loss: 1.8259083&#xA;loss: 1.812584&#xA;loss: 1.8101325&#xA;loss: 1.8081319&#xA;loss: 1.8070083&#xA;loss: 1.8065354&#xA;loss: 1.8063282&#xA;loss: 1.8062303&#xA;loss: 1.8061805&#xA;loss: 1.8061543 (last)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Colab link:&#xA;&lt;a href=&quot;https://colab.research.google.com/drive/1TsHjmucuynCPOgKuo4a0hiM8B8UaOWQo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/1TsHjmucuynCPOgKuo4a0hiM8B8UaOWQo&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-09-20T10:36:14.183" LastActivityDate="2019-09-20T15:48:14.473" Title="LSTM network doesn't converge, what should be changed?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;optimization&gt;&lt;long-short-term-memory&gt;&lt;convergence&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ClosedDate="2019-09-20T20:55:38.043" ContentLicense="CC BY-SA 4.0" />
  <row Id="15530" PostTypeId="2" ParentId="15502" CreationDate="2019-09-19T08:49:04.100" Score="0" Body="&lt;p&gt;I found out how to get 3D output from LSTMCell so that I can matmul with output weights + biases and subtract with expected values:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Inputs &amp;amp; expecteds should be: placeholder(,[times,batch_size,num_inp]) instead of batch_size first then times. However, tf.keras.layers.LSTM will ask for [batch_size,times,num_inp]&lt;/li&gt;&#xA;&lt;li&gt;Use tf.nn.static_rnn with a list of inputs, instead of 1 input&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf;&#xA;&#xA;x = [[[1],[2],[3]],[[4],[5],[6]]];&#xA;times = 2;&#xA;bsize = 3;&#xA;&#xA;#3d input&#xA;inputs = tf.placeholder(tf.float32, [times,bsize,1]);&#xA;&#xA;cell   = tf.nn.rnn_cell.BasicRNNCell(20);&#xA;hids,_ = tf.nn.static_rnn(cell,tf.unstack(inputs,times),dtype=tf.float32);&#xA;&#xA;sess = tf.Session();&#xA;init = tf.global_variables_initializer();&#xA;sess.run(init);&#xA;&#xA;#results in 2d&#xA;print(sess.run(hids, {inputs:x}));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-09-20T10:33:20.557" LastActivityDate="2019-09-20T10:33:20.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15531" PostTypeId="2" ParentId="15485" CreationDate="2019-09-19T08:55:06.223" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is the technical name for this kind of training?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The name for the problem is &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequential_decision_making&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequential Decision Making&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Optimal_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal Control&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are a few different approaches you can take when solving this kind of problem. However, I think that the way that you are describing your project, &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning (RL)&lt;/a&gt; would match your approach the best.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I cannot use a traditional Xy method but need another method that goes like this:&lt;/p&gt;&#xA;&lt;p&gt;network.train_against_value(X, y, determinator)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Although this method signature could probably be made to work by re-structuring one or other RL frameworks, it is more usual when using RL with neural networks to treat the experience-gathering and scoring systems - the core ideas behind RL - as a data generator for either a supervised learning problem or in some cases directly producing the output layer gradients.&lt;/p&gt;&#xA;&lt;p&gt;One approach you could use is called &lt;a href=&quot;https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q Networks (DQN)&lt;/a&gt; which effectively generates mini-batches for supervised learning of a neural network, based on gathering experiences.&lt;/p&gt;&#xA;&lt;p&gt;In brief, DQN trains a neural network to predict what you are calling the &amp;quot;determinator&amp;quot;, but that RL would call the &amp;quot;value&amp;quot; of each action. So you may move the action choice (y) to the input of the neural network, or alternatively predict multiple values (one output for each possble action) - the RL theory is the same for each approach here, it is an implementation detail. The agent (cat or mouse) would then pick the action with the highest predicted value by default (with the cat's values being the negative of the mouse's), trying other actions at random whilst training so as to learn all values. Whenever the agent gained new experience, it would add that to the training data so it could improve its predictions.&lt;/p&gt;&#xA;&lt;p&gt;RL is a complex subject in its own right, and to begin understanding it properly you would do well to start by tackling even simpler problems that don't require neural networks or running two agents against each other. There is a good introductory book on the subject with the option to read a free PDF version: &lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (Sutton &amp;amp; Barto)&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-09-19T09:07:53.043" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15533" PostTypeId="2" ParentId="15529" CreationDate="2019-09-19T09:26:40.317" Score="3" Body="&lt;p&gt;writing here my suggestion, because i haven't earned the right to comment yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your main &quot;problem&quot; could be your loss function. It converges, this is why your loss value is decreasing. So I suggest to let it maybe train longer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you could change the loss function to fit your need. For example you could use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss  = tf.reduce_mean(tf.square(exps-outs))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will get a smaller loss value which decreases clearly after every output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps :)&lt;/p&gt;&#xA;" OwnerUserId="29834" LastActivityDate="2019-09-19T09:26:40.317" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15534" PostTypeId="2" ParentId="15510" CreationDate="2019-09-19T09:50:32.273" Score="0" Body="&lt;p&gt;They are not tracking anything, instead they are trying to find an object which satisfies free fall equation. Gravity acts the same regardless of object's properties - at least in vacuum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;In this paper, we model prior knowledge on the structure&#xA;of the outputs by providing a weighted constraint function&#xA;g used to penalize “structures” that are not&#xA;consistent with our prior knowledge.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a restriction to the possible parabola that is given in the last equation on page 2. They are training the Network to learn time dependence of that equation, which is the same for all objects.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastActivityDate="2019-09-19T09:50:32.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15535" PostTypeId="2" ParentId="15525" CreationDate="2019-09-19T10:30:58.893" Score="2" Body="&lt;p&gt;Sometimes when training, particularly in reinforcement learning, the model can become unstable due to the amount of variance that exists in the training data that the agent generates by interacting with the environment.  This is certainly a problem at the start of training as you can get huge outliers in the data because the agent is behaving randomly.  You can find that just one update to the policy could potentially make it collapse because it moves the policy into some obscure region, e.g. so the agent always take a particular action.  You can make training more stable by using larger batches and a smaller learning rate so it takes smaller steps at a time, but the downside to that is training is slower.  So you need to test different hyperparameters to find a good trade-off between the two.  You can also use an training architecture such as Proximal Policy Optimization (PPO) which clips the amount the policy can move in any given update to try and maintain some stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergence is a term used to describe when the model has found an optimal policy and isn't learning any further, usually demonstrated when the reward plateaus for a certain number of episodes.  Of course, it may have settled on a local optima, and other global optima may exist; the data you present and the way you train your model may yield better results - again, all part of testing and experimentation.&lt;/p&gt;&#xA;" OwnerUserId="20352" LastEditorUserId="20352" LastEditDate="2019-09-19T10:36:22.333" LastActivityDate="2019-09-19T10:36:22.333" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15537" PostTypeId="2" ParentId="15522" CreationDate="2019-09-19T11:24:33.420" Score="2" Body="&lt;p&gt;I would like to mention WOPR from &lt;a href=&quot;https://en.wikipedia.org/wiki/WarGames&quot; rel=&quot;nofollow noreferrer&quot;&gt;War Games&lt;/a&gt;, maybe is an old movie for your students, but it is a more realistic IA centered around the problem of playing board games (if you exclude the part about deciding that a game is not worth the time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I remember an artificial assistant in &lt;a href=&quot;https://www.imdb.com/title/tt0268695/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;The Time machine&quot;&lt;/a&gt; that was more convincing than J.A.R.V.I.S because it is not so intelligent, I remember it more like an agent that can find and read you wikipedia articles, but without reasoning about them a lot, but I could be wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The robot companion in &lt;a href=&quot;https://www.imdb.com/title/tt1182345/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moon&lt;/a&gt; is also interesting and comical as it is like a small child that has been told to cheat but can't disobey direct orders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other films go around the dilema of creating AGI, like, &quot;Blade runner&quot;, &quot;The bicentenary Man&quot;, Spilbergs' &quot;I.A.&quot;, &quot;Her&quot;, or &quot;Ex Machina&quot;, they are more interesting from a philosofical point of view (they are all very similar to Mary Shelley's Frankenstein) because the actual implementation is unconceivable right now.&lt;/p&gt;&#xA;" OwnerUserId="29839" LastActivityDate="2019-09-19T11:24:33.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15538" PostTypeId="2" ParentId="15449" CreationDate="2019-09-19T12:04:36.913" Score="0" Body="&lt;p&gt;Artificial intelligence can harm us in any of the ways of natural intelligence (of humans). The distinction between natural and artificial intelligence will vanish when humans start augmenting themselves more intimately. Intelligence may no longer characterize the identity and will become a limitless possession. The harm caused will be as much the humans can endure for preserving their evolving self-identity.&lt;/p&gt;&#xA;" OwnerUserId="21732" LastActivityDate="2019-09-19T12:04:36.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15539" PostTypeId="1" CreationDate="2019-09-19T13:10:13.547" Score="3" ViewCount="69" Body="&lt;p&gt;I was working on an RL problem and I am confused at one specific point. We use &lt;strong&gt;replay memory&lt;/strong&gt; so that the network learns about previous actions and how these actions lead to a success or a failure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to train the neural network, we use batches from this replay or experience memory. But here's my confusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some places like &lt;a href=&quot;https://deeplizard.com/learn/video/Bcuj2fTH4_4&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; extract random (non-sequential) batches from the memory to train the neural network but &lt;a href=&quot;https://gist.github.com/karpathy/a4166c7fe253700972fcbc77e4ea32c5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrej Karpathy&lt;/a&gt; uses the sequential data to train the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone tell me why there's the difference?&lt;/p&gt;&#xA;" OwnerUserId="29843" LastEditorUserId="2444" LastEditDate="2019-09-19T13:38:16.263" LastActivityDate="2019-09-19T13:38:16.263" Title="What is the difference between random and sequential sampling from the reply memory?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15541" PostTypeId="1" AcceptedAnswerId="15552" CreationDate="2019-09-19T16:38:25.600" Score="4" ViewCount="165" Body="&lt;p&gt;This is my problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 10 variables that I intend to evaluate two by two (in pairs). I want to know which variables have the strongest relationships with each other. And I'm only interested in evaluating relationships two by two. Well, one suggestion would be to calculate the pairwise correlation coefficient of these variables. And then list the pairs with the highest correlation coefficient to the lowest correlation. That way I would have a ranking between the most correlated to the lowest correlated pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is there anything analogous in the world of artificial intelligence to the correlation coefficient calculation? That is, what tools can the world of A.I / Machine Learning offer me to extract this kind of information? So that in the end I can have something like a ranking among the most &quot;correlated&quot; pairs from the point of view of A.I / Machine Learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, how do I know which variable among these 10 best &quot;relates&quot; (or &quot;correlates&quot;) with variable 7, for example?&lt;/p&gt;&#xA;" OwnerUserId="18579" LastActivityDate="2019-09-19T23:50:38.980" Title="How Machine Learning Studies Correlation?" Tags="&lt;machine-learning&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15542" PostTypeId="1" AcceptedAnswerId="15551" CreationDate="2019-09-19T16:45:25.910" Score="5" ViewCount="137" Body="&lt;p&gt;Evolutionary algorithms are mentioned in some sources as a method to train a neural network (finding weights, not hyperparameters). However, I have not heard about one practical application of such an idea yet.&lt;/p&gt;&#xA;&lt;p&gt;My question is, why is that? What are the issues or limitations with such a solution that prevented it from practical use?&lt;/p&gt;&#xA;&lt;p&gt;I am asking because I am planning on developing such an algorithm and want to know what to expect and where to put most attention.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2021-01-20T19:31:34.743" LastActivityDate="2021-01-20T19:31:34.743" Title="Why evolutionary training of neural networks is not popular?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;gradient-descent&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15543" PostTypeId="2" ParentId="15449" CreationDate="2019-09-19T16:55:14.260" Score="0" Body="&lt;p&gt;Few people realize that our global economy should be considered an AI:&#xA;- The money transactions are the signals over a neural net. The nodes in the neural net would be the different corporations or private persons paying or receiving money.&#xA;- It is man-made so qualifies as artificial&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This neural network is better in its task then humans:&#xA;Capitalism has always won against economy planned by humans (plan-economy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this neural net dangerous ?&#xA;Might differ if you are the CEO earning big versus a fisherman in a river polluted by corporate waste.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How did this AI become dangerous?&#xA;You could answer it is because of human greed.&#xA;Our creation reflects ourselves.&#xA;In other words: we didnot train our neural net to behave well. &#xA;Instead of training the neural net to improve living quality for all humans, we trained it to make rich fokes more rich.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be easy to train this AI to be no longer dangerous ?&#xA;Maybe not, maybe some AI are just larger then life. &#xA;It is just survival of the fittest.&lt;/p&gt;&#xA;" OwnerUserId="29850" LastActivityDate="2019-09-19T16:55:14.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15544" PostTypeId="1" AcceptedAnswerId="15547" CreationDate="2019-09-19T17:55:58.390" Score="3" ViewCount="59" Body="&lt;p&gt;Let's assume we have an ANN which takes a vector &lt;span class=&quot;math-container&quot;&gt;$x\in R^D$&lt;/span&gt;, representing an image, and classifies it over two classes. The output is a vector of probabilities &lt;span class=&quot;math-container&quot;&gt;$N(x)=(p(x\in C_1), p(x\in C_2))^T$&lt;/span&gt; and we pick &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; iff &lt;span class=&quot;math-container&quot;&gt;$p(x\in C_1) \geq 0.5$&lt;/span&gt;. Let the two classes be &lt;span class=&quot;math-container&quot;&gt;$C_1= \texttt{cat}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_2= \texttt{dog}$&lt;/span&gt;. Now imagine we want to extract this ANN's idea of ideal cat by finding &lt;span class=&quot;math-container&quot;&gt;$x^* = argmax_x N(x)_1$&lt;/span&gt;. How would we proceed? I was thinking about solving &lt;span class=&quot;math-container&quot;&gt;$\nabla_xN(x)_1=0$&lt;/span&gt;, but I don't know if this makes sense or if it is solvable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In short, how do I compute the input which maximizes a class-probability?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-09-19T18:02:28.310" LastActivityDate="2019-09-19T20:42:31.120" Title="How can we find find the input image which maximizes the class-probability for an ANN?" Tags="&lt;neural-networks&gt;&lt;generative-model&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15545" PostTypeId="2" ParentId="1655" CreationDate="2019-09-19T19:11:19.570" Score="0" Body="&lt;p&gt;Low precision enables high parallelism computation in Convo and FC layers.&#xA;CPU &amp;amp; GPU fixed architecture, but ASIC/FPGA can be designed based on neural network architecture &lt;/p&gt;&#xA;" OwnerUserId="17792" LastActivityDate="2019-09-19T19:11:19.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15546" PostTypeId="1" AcceptedAnswerId="15550" CreationDate="2019-09-19T20:15:54.113" Score="2" ViewCount="31" Body="&lt;p&gt;Made a neural network using tensor flows that was supposed matches an Ip to one of the 7 type of vulnerabilities and gives out what type of vulnerability that IP has.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    model = tf.keras.models.Sequential([&#xA;  tf.keras.layers.Flatten(),&#xA;  tf.keras.layers.Dense(50, activation=tf.nn.relu),&#xA;  tf.keras.layers.Dense(7, activation=tf.nn.softmax)&#xA;])&#xA;&#xA;model.compile(optimizer='adam',&#xA;              loss='sparse_categorical_crossentropy',&#xA;              metrics=['accuracy'])&#xA;&#xA;&#xA;&#xA;model.fit(xs, ys, epochs=500)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of &lt;code&gt;print(model.predict([181271844]))&lt;/code&gt; when this command is executed should be one of the numbers from 1 to 7 but the out put its gives is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[[0.22288103 0.20282331 0.36847615 0.11339897 0.04456346 0.02391759&#xA;    0.02393949]]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can't seem to figure out what the problem is.&lt;/p&gt;&#xA;" OwnerUserId="27399" LastActivityDate="2019-09-19T23:38:04.110" Title="Neural network does not give out the required out put?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15547" PostTypeId="2" ParentId="15544" CreationDate="2019-09-19T20:25:40.720" Score="4" Body="&lt;p&gt;In deep networks there is actually a wide variety of solutions to the problem, but if you need to find one, any easy way to do this is just through normal optimization schemes&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat x = argmin_x \ L(y,x)$$&lt;/span&gt;&lt;br&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$L(y,x)$&lt;/span&gt; is your loss function. Since ANN's are generally differentiable you can optimize this iteratively with some form gradient descent scheme:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$x^{i+1} \leftarrow x^{i} - \lambda \nabla_{x^i}L(y,x^i)$$&lt;/span&gt;&lt;br&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is your learning rate.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-09-19T20:25:40.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15548" PostTypeId="2" ParentId="15544" CreationDate="2019-09-19T20:42:31.120" Score="4" Body="&lt;p&gt;Probably the simplest way to search for an image with the highest probability of being a cat is to use a technique similar to Deep Dream:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Load the network for training, but freeze all the network weights&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a random input image, and connect it to the network as a &quot;variable&quot; i.e. data that can be changed through training&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Set a loss function based on maximising the pre-sigmoid value in the last layer (this is easier to handle than working with 0.999 etc probability)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train using backpropagation, but instead of using gradients to change the weights, back propagate all the way to the input layer and use gradients to change the input image.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Typically you will also want to normalise the input image between iterations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a good chance that the ideal input you find which triggers &quot;maximum catness&quot; will be a very noisy jumbled mess of cat-related features. You may be able to encourage something more visually apppealing, or at least less noisy, by adding a little movement - e.g. minor blurring, or a slight zoom (then crop) between each iteration. At that point, it becomes a more an artistic endeavour than mathematical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is something I produced using some TensorFlow Deep Dream code plus zooming and blurring to encourage larger scale features to dominate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nPNHG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nPNHG.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically the above maximises a single internal feature map of a CNN, not a class probability, but it is the same thing conceptually.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-09-19T20:42:31.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15550" PostTypeId="2" ParentId="15546" CreationDate="2019-09-19T23:38:04.110" Score="1" Body="&lt;p&gt;The numbers you are seeing as output are a &lt;em&gt;probability vector&lt;/em&gt;. This is a common output format for multi-class classification models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you can interpret the vector as saying:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;22% chance of class 1&lt;/li&gt;&#xA;&lt;li&gt;20% chance of class 2&lt;/li&gt;&#xA;&lt;li&gt;37% chance of class 3&lt;/li&gt;&#xA;&lt;li&gt;11% chance of class 4&lt;/li&gt;&#xA;&lt;li&gt;4% chance of class 5&lt;/li&gt;&#xA;&lt;li&gt;2% chance of class 6&lt;/li&gt;&#xA;&lt;li&gt;2% chance of class 7&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to get a concrete label out of this, the easiest choice is to compute and return the index of the maximum element. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-19T23:38:04.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15551" PostTypeId="2" ParentId="15542" CreationDate="2019-09-19T23:45:29.787" Score="4" Body="&lt;p&gt;The main evolutionary algorithm used to train neural networks is &lt;em&gt;Neuro-Evolution of Augmenting Topoloigies&lt;/em&gt;, or &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/neat_overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt;. NEAT has seen fairly widespread use. There are &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C33&amp;amp;q=%22NeuroEvolution%20of%20Augmenting%20Topologies%22&amp;amp;btnG=&quot; rel=&quot;nofollow noreferrer&quot;&gt;thousands of academic papers&lt;/a&gt; building on or using the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEAT is not widely used in commercial applications because if you have a clean objective function, a topology that is optimized for gradient decent via backpropogation, and an implementation that is highly optimized for a GPU, you are almost certainly going to see better, faster, results from a conventional training process. Where NEAT is really useful is if you want to do something weird, like train to &lt;a href=&quot;https://www.researchgate.net/profile/Joel_Lehman/publication/221008669_Improving_evolvability_through_novelty_search_and_self-adaptation/links/0deec51dfe82216608000000.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;maximize novelty&lt;/a&gt;, or if you want to try to train neurons that don't have cleanly decomposable gradients. Basically, you need to have any of the usual reasons you might prefer an evolutionary algorithm to hill-climbing approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You don't have a clean mapping from loss function to individual model components.&lt;/li&gt;&#xA;&lt;li&gt;Your loss function has many local maximia.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-19T23:45:29.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15552" PostTypeId="2" ParentId="15541" CreationDate="2019-09-19T23:50:38.980" Score="2" Body="&lt;p&gt;Welcome to AI.SE @Laura!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds like you have a series of data points, each with 10 related measurements, and you want automatically assess which of the measurements are most closely related to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are right that the correlation coefficient is a good choice for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other techniques used in some AI algorithms include the &lt;em&gt;Information Gain&lt;/em&gt; measurement (where you &lt;a href=&quot;https://en.wikipedia.org/wiki/Information_gain_in_decision_trees&quot; rel=&quot;nofollow noreferrer&quot;&gt;measure the reduction in entropy&lt;/a&gt; of one variable that follows from partitioning on another one), and embedded feature selection approaches, like the one in &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/viewFile/9576/9282&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-19T23:50:38.980" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15553" PostTypeId="2" ParentId="15434" CreationDate="2019-09-20T03:23:32.823" Score="0" Body="&lt;p&gt;The topic is multimodal neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here some repositories that i hope will help me a lot &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.google.com/presentation/d/1z8-GeTXvSuVbcez8R6HOG1Tw_F3A-WETahQdTV38_uc/edit#slide=id.g1ea5aac985_0_892&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/presentation/d/1z8-GeTXvSuVbcez8R6HOG1Tw_F3A-WETahQdTV38_uc/edit#slide=id.g1ea5aac985_0_892&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/prml615/prml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/prml615/prml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/husseinmozannar/multimodal-deep-learning-for-disaster-response&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/husseinmozannar/multimodal-deep-learning-for-disaster-response&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/guillaume-be/multimodal-avito&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/guillaume-be/multimodal-avito&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to kbrose, his suggestion led me to this type of architectures.&lt;/p&gt;&#xA;" OwnerUserId="29693" LastActivityDate="2019-09-20T03:23:32.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15554" PostTypeId="1" CreationDate="2019-09-20T08:36:08.003" Score="0" ViewCount="48" Body="&lt;p&gt;Is federated learning a privacy breach, given that the model transmission, for a period of time, may cause the adversarial to reach and manipulate the model and the data?&lt;/p&gt;&#xA;" OwnerUserId="14395" LastEditorUserId="2444" LastEditDate="2019-09-20T12:23:42.467" LastActivityDate="2019-09-20T12:23:42.467" Title="Is federated learning a privacy breach?" Tags="&lt;machine-learning&gt;&lt;social&gt;&lt;federated-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15556" PostTypeId="2" ParentId="10566" CreationDate="2019-09-20T12:53:48.057" Score="1" Body="&lt;p&gt;I know the new bot in JP Morgan uses reinforcement learning:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.jpmorgan.com/country/US/EN/markets/machine-learning-fx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning in FX&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There a lot of studies of Reinforcement learning can apply in crypto currency trading. Here is one of many examples:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1706.10059&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29812" LastEditorUserId="11539" LastEditDate="2020-11-14T02:17:02.713" LastActivityDate="2020-11-14T02:17:02.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15557" PostTypeId="2" ParentId="6997" CreationDate="2019-09-20T13:04:46.477" Score="1" Body="&lt;p&gt;I would say this post is a must to read:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://rubenfiszel.github.io/posts/rl4j/2016-08-24-Reinforcement-Learning-and-DQN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rubenfiszel.github.io/posts/rl4j/2016-08-24-Reinforcement-Learning-and-DQN.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="29812" LastActivityDate="2019-09-20T13:04:46.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15559" PostTypeId="1" CreationDate="2019-09-20T13:14:20.410" Score="2" ViewCount="34" Body="&lt;p&gt;I am looking to plan a solution for a workspace fault and not hardware faults. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a task where a robot has to move balls from one place to another. In case it faces any condition which is outside the task for eg. someone snatches the ball from the robot while it is transferring or the robot drops the balls in between. These are some example faults that could occur, many other might be possible. I am trying to build a generalized algorithm that so that the robot can find a way to resolve unexpected changes itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently have an FSM for the whole task. Any fault that somehow changes any of the state machine variables should be considered. For instance, there are faults that deal with obstacles that may come in the way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there might be faults for example a cloth in front of the camera. This fault should be corrected by a human since the robot cannot manage that. All the faults like that are out of the scope of the robot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or ideas related to the algorithm will be helpful. &lt;/p&gt;&#xA;" OwnerUserId="29868" LastEditorUserId="1847" LastEditDate="2019-09-20T16:13:13.620" LastActivityDate="2019-09-20T16:13:13.620" Title="Algorithm to solve a fault independent of its type" Tags="&lt;learning-algorithms&gt;&lt;robotics&gt;&lt;problem-solving&gt;&lt;path-planning&gt;&lt;path-finding&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15560" PostTypeId="2" ParentId="11949" CreationDate="2019-09-20T15:00:53.350" Score="0" Body="&lt;p&gt;For sure, this may be helpful. You remove excessive information from image and make the classification task a bit more simple. But you need to remember that bounding boxes may work not perfectly and accuracy of the classification algorithm may suffer from corrupted inputs (when bounding boxes are corrupted).&lt;/p&gt;&#xA;" OwnerUserId="29872" LastActivityDate="2019-09-20T15:00:53.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15561" PostTypeId="2" ParentId="15529" CreationDate="2019-09-20T15:48:14.473" Score="0" Body="&lt;p&gt;I'm still working on how to make the code work for text generation, but the following converges and work for text classification:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf;&#xA;tf.reset_default_graph();&#xA;&#xA;#data&#xA;'''&#xA;t0      t1      t2&#xA;british gray    is =&amp;gt; cat (y=0)&#xA;0       1       2&#xA;white   samoyed is =&amp;gt; dog (y=1)&#xA;3       4       2 &#xA;'''&#xA;Bsize = 2;&#xA;Times = 3;&#xA;Max_X = 4;&#xA;Max_Y = 1;&#xA;&#xA;X = [[[0],[1],[2]], [[3],[4],[2]]];&#xA;Y = [[0],           [1]          ];&#xA;&#xA;#normalise&#xA;for I in range(len(X)):&#xA;  for J in range(len(X[I])):&#xA;    X[I][J][0] /= Max_X;&#xA;&#xA;for I in range(len(Y)):&#xA;  Y[I][0] /= Max_Y;&#xA;&#xA;#model&#xA;Inputs   = tf.placeholder(tf.float32, [Bsize,Times,1]);&#xA;Expected = tf.placeholder(tf.float32, [Bsize,      1]);&#xA;&#xA;#single LSTM layer&#xA;#'''&#xA;Layer1   = tf.keras.layers.LSTM(20);&#xA;Hidden1  = Layer1(Inputs);&#xA;#'''&#xA;&#xA;#multi LSTM layers&#xA;'''&#xA;Layers = tf.keras.layers.RNN([&#xA;  tf.keras.layers.LSTMCell(30), #hidden 1&#xA;  tf.keras.layers.LSTMCell(20)  #hidden 2&#xA;]);&#xA;Hidden2 = Layers(Inputs);&#xA;'''&#xA;&#xA;Weight3  = tf.Variable(tf.random_uniform([20,1], -1,1));&#xA;Bias3    = tf.Variable(tf.random_uniform([   1], -1,1));&#xA;Output   = tf.sigmoid(tf.matmul(Hidden1,Weight3) + Bias3);&#xA;&#xA;Loss     = tf.reduce_sum(tf.square(Expected-Output));&#xA;Optim    = tf.train.GradientDescentOptimizer(1e-1);&#xA;Training = Optim.minimize(Loss);&#xA;&#xA;#train&#xA;Sess = tf.Session();&#xA;Init = tf.global_variables_initializer();&#xA;Sess.run(Init);&#xA;&#xA;Feed = {Inputs:X, Expected:Y};&#xA;for I in range(1000): #number of feeds, 1 feed = 1 batch&#xA;  if I%100==0: &#xA;    Lossvalue = Sess.run(Loss,Feed);&#xA;    print(&quot;Loss:&quot;,Lossvalue);&#xA;  #end if&#xA;&#xA;  Sess.run(Training,Feed);&#xA;#end for&#xA;&#xA;Lastloss = Sess.run(Loss,Feed);&#xA;print(&quot;Loss:&quot;,Lastloss,&quot;(Last)&quot;);&#xA;&#xA;#eval&#xA;Results = Sess.run(Output,Feed);&#xA;print(&quot;\nEval:&quot;);&#xA;print(Results);&#xA;&#xA;print(&quot;\nDone.&quot;);&#xA;#eof&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-09-20T15:48:14.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15562" PostTypeId="1" AcceptedAnswerId="15602" CreationDate="2019-09-20T15:57:15.957" Score="0" ViewCount="26" Body="&lt;p&gt;I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T in the time dimension of RNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matter is for text generation, I need &lt;strong&gt;a word somewhere in the middle of time dimension&lt;/strong&gt;, eg.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t0  t1    t2  t3&#xA;The brown fox jumps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For this example, I have the first 2 words: &lt;code&gt;The&lt;/code&gt;, &lt;code&gt;brown&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to get the next word ie. &quot;&lt;code&gt;fox&lt;/code&gt;&quot; using RNN (LSTM)? How to convert the following text classification code to text generating code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source code (text classification):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf;&#xA;tf.reset_default_graph();&#xA;&#xA;#data&#xA;'''&#xA;t0      t1      t2&#xA;british gray    is =&amp;gt; cat (y=0)&#xA;0       1       2&#xA;white   samoyed is =&amp;gt; dog (y=1)&#xA;3       4       2 &#xA;'''&#xA;Bsize = 2;&#xA;Times = 3;&#xA;Max_X = 4;&#xA;Max_Y = 1;&#xA;&#xA;X = [[[0],[1],[2]], [[3],[4],[2]]];&#xA;Y = [[0],           [1]          ];&#xA;&#xA;#normalise&#xA;for I in range(len(X)):&#xA;  for J in range(len(X[I])):&#xA;    X[I][J][0] /= Max_X;&#xA;&#xA;for I in range(len(Y)):&#xA;  Y[I][0] /= Max_Y;&#xA;&#xA;#model&#xA;Inputs   = tf.placeholder(tf.float32, [Bsize,Times,1]);&#xA;Expected = tf.placeholder(tf.float32, [Bsize,      1]);&#xA;&#xA;#single LSTM layer&#xA;#'''&#xA;Layer1   = tf.keras.layers.LSTM(20);&#xA;Hidden1  = Layer1(Inputs);&#xA;#'''&#xA;&#xA;#multi LSTM layers&#xA;'''&#xA;Layers = tf.keras.layers.RNN([&#xA;  tf.keras.layers.LSTMCell(30), #hidden 1&#xA;  tf.keras.layers.LSTMCell(20)  #hidden 2&#xA;]);&#xA;Hidden2 = Layers(Inputs);&#xA;'''&#xA;&#xA;Weight3  = tf.Variable(tf.random_uniform([20,1], -1,1));&#xA;Bias3    = tf.Variable(tf.random_uniform([   1], -1,1));&#xA;Output   = tf.sigmoid(tf.matmul(Hidden1,Weight3) + Bias3);&#xA;&#xA;Loss     = tf.reduce_sum(tf.square(Expected-Output));&#xA;Optim    = tf.train.GradientDescentOptimizer(1e-1);&#xA;Training = Optim.minimize(Loss);&#xA;&#xA;#train&#xA;Sess = tf.Session();&#xA;Init = tf.global_variables_initializer();&#xA;Sess.run(Init);&#xA;&#xA;Feed = {Inputs:X, Expected:Y};&#xA;for I in range(1000): #number of feeds, 1 feed = 1 batch&#xA;  if I%100==0: &#xA;    Lossvalue = Sess.run(Loss,Feed);&#xA;    print(&quot;Loss:&quot;,Lossvalue);&#xA;  #end if&#xA;&#xA;  Sess.run(Training,Feed);&#xA;#end for&#xA;&#xA;Lastloss = Sess.run(Loss,Feed);&#xA;print(&quot;Loss:&quot;,Lastloss,&quot;(Last)&quot;);&#xA;&#xA;#eval&#xA;Results = Sess.run(Output,Feed);&#xA;print(&quot;\nEval:&quot;);&#xA;print(Results);&#xA;&#xA;print(&quot;\nDone.&quot;);&#xA;#eof&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-09-23T21:57:04.337" Title="How to change this RNN text classification code to become text generation code?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15565" PostTypeId="1" CreationDate="2019-09-21T01:49:41.203" Score="0" ViewCount="40" Body="&lt;p&gt;OK, now I think an AI must view grids in a different way to computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example a computer would represent a grid like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cells = [[1,2,3],[4,5,6],[7,8,9]] = [row1,row2,row3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is a grid is 3 rows of 3 cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But... that's not how a human sees it. A human sees a grid as made of 3 rows and 3 collumns somehow intersecting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an AI is built on some mathematical logic like set theory, it's like a set of rows which in turn is a set of cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what would be a way to represent a grid in a computer that is more &quot;human&quot;. And doesn't favor either rows or columns? Or is there some mathematical or programmatical description of a grid that treats rows and columns as equivalent? &lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="16909" LastEditDate="2019-09-21T20:28:41.967" LastActivityDate="2019-09-21T20:36:32.573" Title="How would an AI understand grids?" Tags="&lt;ai-design&gt;&lt;knowledge-representation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15566" PostTypeId="1" CreationDate="2019-09-21T05:48:34.063" Score="0" ViewCount="65" Body="&lt;p&gt;I solved the OpenAI-Gym MountainCar-v0 environment using dqn(using low-state-dimensional input). When I used the same code for solving CartPole-v0 environment, the network got trained in the reverse direction (It sort of unlearned the environment and now performs worse than random actions). What could be the possible reason for this?&lt;/p&gt;&#xA;" OwnerUserId="29879" LastActivityDate="2019-09-21T05:48:34.063" Title="DQN unlearns certain OpenAI-Gym environments" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15571" PostTypeId="2" ParentId="15565" CreationDate="2019-09-21T20:36:32.573" Score="4" Body="&lt;p&gt;Although it is &lt;em&gt;common&lt;/em&gt; to represent a grid as two dimensional array in a computer program, this is not the only way to represent one. You could, for example, use a generalized &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;graph structure&lt;/a&gt; made of linked nodes, with 4 links each. Many other representations are possible. Even if you use a 2d array, some languages &lt;em&gt;would&lt;/em&gt; index the columns first rather than the rows (FORTRAN for instance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That issue is really an aside though, because it conflates &lt;em&gt;representation&lt;/em&gt; with &lt;em&gt;reasoning&lt;/em&gt;. By choosing to represent the world in certain data structures, we can made it easier or harder for different AI algorithms to reason about them, but the reasoning can often proceed even if the representation is inefficient. That is, an AI algorithm can ask questions about the columns in the grid, even if the columns of the grid are not represented in a way that makes it programmitcally easy to group them together. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-21T20:36:32.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15573" PostTypeId="1" CreationDate="2019-09-22T04:35:51.703" Score="3" ViewCount="91" Body="&lt;p&gt;I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm trying to do a NN that can make a prediction. The idea is that I have a large set of customer purchase history, so people I can &quot;target&quot; with marketing, others I can't (maybe I just have a credit-card number but no email address to spam). I've a catalogue of products that changes on a monthly basis with daily updates to stock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My original idea was to use the same NN that I've used before, with inputs like purchased y/n for each product and an output for each product (softmax to get a weighted prediction). But I get stuck at handling a changing catalog. I'm also not sure if I should lump everyone in together or sort of generate a NN for each person individually (but some people would have very little purchase history, so I'd need to use everyone else as the training set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I thought I'd need something with some ability to use the purchase data as a sequence, so purchased A, then B, then C etc. But reviewing something like LSTM, I kind of think it's still not right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I know how to NN for a game-state sort of problem. But I don't know how to do it for this new problem.&lt;/p&gt;&#xA;" OwnerUserId="2819" LastActivityDate="2020-10-16T13:00:35.087" Title="What sort of Neural Network is best suited to predicting a future purchase?" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15577" PostTypeId="1" CreationDate="2019-09-22T12:31:23.257" Score="1" ViewCount="51" Body="&lt;p&gt;I want to use a GAN for sequence prediction, in a similar way that we use RNNs for sequence prediction. I want to test its performance in comparison with RNNs. Is there a GAN that can be used for sequence prediction?&lt;/p&gt;&#xA;" OwnerUserId="10051" LastEditorUserId="2444" LastEditDate="2019-09-22T14:52:29.627" LastActivityDate="2019-09-22T14:52:29.627" Title="Is there a GAN that can be used for sequence prediction?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;prediction&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15579" PostTypeId="1" CreationDate="2019-09-22T15:46:14.707" Score="5" ViewCount="49" Body="&lt;p&gt;When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 score .etc, and do the same for an old optimization algorithm, or do we need to compute the average performance,i.e.,the average value of accuracy or f1 scores for these runs, to show that it is better than the old optimization algorithm? Because when I read the &lt;strong&gt;papers on a new optimization algorithm&lt;/strong&gt;, I don't know how they calculate the performance and draw the train-loss vs iters curves, because it has random effects, and for different runs we may get different performance and different curves. &lt;/p&gt;&#xA;" OwnerUserId="29902" LastActivityDate="2019-09-22T19:53:55.717" Title="How can we conclude that an optimization algorithm is better than another one" Tags="&lt;optimization&gt;&lt;convergence&gt;&lt;performance&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15583" PostTypeId="1" CreationDate="2019-09-22T17:07:28.977" Score="1" ViewCount="181" Body="&lt;p&gt;After looking around the internet (including &lt;a href=&quot;https://arxiv.org/abs/1502.05082&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, I cannot seem to find a satisfactory explanation of the Average Recall (AR) metric. On the COCO &lt;a href=&quot;http://cocodataset.org/#detection-eval&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;, it describes AR as: &quot;the maximum recall given a fixed number of detections per image, averaged over categories and IoUs&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &quot;maximum recall&quot; mean here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if someone could give a reference or a high level overview of the AR calculation algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="19789" LastActivityDate="2019-09-22T17:07:28.977" Title="How is Average Recall (AR) calculated for an object detection model?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15584" PostTypeId="2" ParentId="15526" CreationDate="2019-09-22T17:20:20.627" Score="1" Body="&lt;p&gt;It is a mathematical problem. Usually, we use contraction mapping theorem for the proof of convergence. You should apply the Banach fixed point theorem for Bellman's functions.&lt;/p&gt;&#xA;" OwnerUserId="29906" LastActivityDate="2019-09-22T17:20:20.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15585" PostTypeId="2" ParentId="6903" CreationDate="2019-09-22T19:34:18.920" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Direct Answer to Your Question&lt;/strong&gt;:--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend: &lt;em&gt;Automate the Boring Stuff with Python: Practical Programming for Total Beginners&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot; ... Does anyone know a good book to buy for beginners to coding for artificial intelligence? ... &quot; ~  Stephen Messing (Stack Exchange user)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The key word here is &lt;em&gt;beginners&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A very serious issue with people studying advanced topics is not having a strong base in the beginner topics. A house without structure will collapse. Follow these two principles: [1] step-by-step and [2] first things first.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Impatience is a very serious issue for folks starting off in A.I., as people can have expectations that do not match up with the actual field. A.I. is a demanding, intense field requiring intense dedication and study.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A solution to these problems is to supply a very practical and hands-on book that provides direct results upon applying the techniques therein but still reinforces the fundamentals.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So, with all of the above in mind and considered, I recommend &lt;em&gt;Automate the Boring Stuff with Python: Practical Programming for Total Beginners&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Automation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Automation&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.goodreads.com/book/show/22514127-automate-the-boring-stuff-with-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.goodreads.com/book/show/22514127-automate-the-boring-stuff-with-python&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="25982" LastEditDate="2019-09-24T14:16:39.907" LastActivityDate="2019-09-24T14:16:39.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15586" PostTypeId="2" ParentId="15579" CreationDate="2019-09-22T19:53:55.717" Score="1" Body="&lt;p&gt;See here for a potential way to do it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://infinity77.net/global_optimization/#motivation-motivation&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://infinity77.net/global_optimization/#motivation-motivation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://infinity77.net/global_optimization/#rules-the-rules&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://infinity77.net/global_optimization/#rules-the-rules&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You basically test the two (or more) optimization algorithms against known objective functions, with several random (but repeatable) starting points and then analyze the outcome.&lt;/p&gt;&#xA;" OwnerUserId="29908" LastActivityDate="2019-09-22T19:53:55.717" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15587" PostTypeId="2" ParentId="12160" CreationDate="2019-09-22T21:25:40.327" Score="2" Body="&lt;p&gt;I don't know about your first question, but I got a  basic policy gradient approach with the kinetic energy as reward working on &lt;code&gt;MountainCar-v0&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can implement it based on &lt;a href=&quot;https://medium.com/@ts1829/policy-gradient-reinforcement-learning-in-pytorch-df1383ea0baf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; and the notebook you find there. It uses an &lt;code&gt;MLP&lt;/code&gt; with one hidden layer of size 128 and standard policy gradient learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward engineering boils down to replacing the &lt;code&gt;reward&lt;/code&gt; variable with the kinetic energy &lt;span class=&quot;math-container&quot;&gt;$v^2$&lt;/span&gt; (no potential energy and no constant factor, the reward itself is not used). I takes &lt;span class=&quot;math-container&quot;&gt;$&amp;gt;1000$&lt;/span&gt; episodes to solve the environment consistently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm afraid the solution is not very satisfactory and I don't have the feeling there is much to learn from it. The solution is originally for the cartpole problem and it stops working for me if I change hyperparameters/optimizer or the specifics of the reward. &lt;/p&gt;&#xA;" OwnerUserId="29907" LastActivityDate="2019-09-22T21:25:40.327" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15588" PostTypeId="1" CreationDate="2019-09-22T22:32:21.157" Score="3" ViewCount="164" Body="&lt;p&gt;I'm not sure what this type of data is called, so I will give an example of the type of data I am working with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A city records its inflow and outflow of different types of vehicles every hour. More specifically, it records the engine size. The output would be the pollution level X hours after the recorded hourly interval. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that the data consists of individual vehicle engine size, so they cant be aggregated. This means the 2 input vectors (inflow and outflow) will be of variable length (different number of vehicles would be entering and lraving every hour) and I'm not sure how to handle this. I could aggregate and simply sum the number of vehicles, but I want to preserve any patterns in the data. E.g. perhaps there is a quick succession of several heavy motorbike engines, denoting a biker gang have just entered the city and are known to ride recklessly, contributing more to pollution than the sum of its parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insight is appreciated. &lt;/p&gt;&#xA;" OwnerUserId="29910" LastActivityDate="2019-09-24T07:13:34.383" Title="What kind of neural network architecture is suitable for variable length block-like time series data?" Tags="&lt;neural-networks&gt;&lt;prediction&gt;&lt;time-series&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15589" PostTypeId="1" CreationDate="2019-09-23T03:18:08.933" Score="3" ViewCount="878" Body="&lt;p&gt;I have a book containing lots of puzzles with instructions like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Find a path through all the white squares.&quot;&#xA;&#xA;&quot;Connect each black circle to a white circle with a straight line without crossing lines&quot;.&#xA;&#xA;&quot;Put the letters A to G in the grid so that no letters are repeated in any row or collumn&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I thought it might be fun to (1) Try and write a program to solve each individual puzzle (2) Write a program that can solve more general problems, and even more interesting (3) Try to write a program that parses the English instructions and then solves the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that in general there would be common themes like, &quot;draw a path&quot;, &quot;connect the dots&quot;, &quot;place the letters in the grid&quot;, and so forth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program would have general knowledge of thing like squares, cells, rows, collumns, colours, letters, numbers and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wondered if there is anything similar out there already?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an AI could read instructions and solve the puzzles could we say that it is in someway intelligent?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-09-23T12:48:07.107" Title="Puzzle solving AI?" Tags="&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15590" PostTypeId="1" CreationDate="2019-09-23T05:32:23.523" Score="5" ViewCount="550" Body="&lt;p&gt;Is randomness (either true randomness or simulated randomness) necessary for AI? If true, does it mean &quot;intelligence comes from randomness&quot;? &#xA;If not, can a robot lacking the ability to generate random numbers be called an artificial general intelligence?&lt;/p&gt;&#xA;" OwnerUserId="29915" LastEditorUserId="2444" LastEditDate="2019-09-23T12:32:40.927" LastActivityDate="2019-09-24T00:19:48.940" Title="Is randomness necessary for AI?" Tags="&lt;philosophy&gt;&lt;randomness&gt;" AnswerCount="3" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15591" PostTypeId="2" ParentId="15590" CreationDate="2019-09-23T06:06:10.550" Score="2" Body="&lt;p&gt;Yes, randomness is necessary to achieve generality in theory. Right now AIs we have are on the basis of seeking pattern and use them to predict future moves or outcomes. If we don't include randomness in data then machine might consider that as pattern and behave according to that (Which will be bias for us).&#xA;Generating random numbers is a different story in itself and won't be a criterion alone to judge. While this might be one of the conditions for sure.&lt;/p&gt;&#xA;" OwnerUserId="29916" LastActivityDate="2019-09-23T06:06:10.550" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15593" PostTypeId="2" ParentId="15506" CreationDate="2019-09-23T07:58:09.307" Score="0" Body="&lt;p&gt;I'm writing here because I can't comment yet.&#xA;Just for clarification. Does your code run on the same machine and/or with cpu or a gpu?&#xA;For example &lt;a href=&quot;https://machinelearningmastery.com/reproducible-results-neural-networks-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; article suggest that even more randomness is added to models, if you use a gpu. This might yield different results of models.&lt;/p&gt;&#xA;&lt;p&gt;Also you might want to make sure, that keras is really using the tensorflow backend on the same version, as suggested &lt;a href=&quot;https://datascience.stackexchange.com/questions/43003/keras-neural-network-entirely-different-results-on-two-different-hardwares-one&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Hope to hear from you :)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I just saw the &amp;quot;multi-gpu&amp;quot; import...Can you try to disable the gpu and try again on cpu only.&lt;/p&gt;&#xA;" OwnerUserId="29834" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-09-23T07:58:09.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15594" PostTypeId="1" AcceptedAnswerId="15604" CreationDate="2019-09-23T08:33:58.867" Score="12" ViewCount="1084" Body="&lt;p&gt;I found the following neural network cheat sheet (&lt;a href=&quot;https://becominghuman.ai/cheat-sheets-for-ai-neural-networks-machine-learning-deep-learning-big-data-678c51b4b463&quot; rel=&quot;noreferrer&quot;&gt;Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning &amp;amp; Big Data&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0WL34.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0WL34.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are all these different kinds of neural networks used for? For example, which neural networks can be used for regression or classification, which can be used for sequence generation, etc.? I just need a brief overview (1-2 lines) of their applications.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2019-09-24T17:06:01.147" LastActivityDate="2019-09-25T02:15:15.033" Title="What are all the different kinds of neural networks used for?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="8" ClosedDate="2020-11-15T18:53:34.207" ContentLicense="CC BY-SA 4.0" />
  <row Id="15595" PostTypeId="2" ParentId="15497" CreationDate="2019-09-23T08:39:56.877" Score="0" Body="&lt;p&gt;My simple answer is &lt;strong&gt;NO&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me elaborate. If you closely observe nature, you see that nothing changes drastically all of a sudden. Even when it does, it doesn't stay for long. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Field of AI, has just started and it needs a lot more evolution to achieve AGI. Though AI is solving many directed problems like Face Recognition, Speech Recognition and many more (applications are innumerable), all these can be considered as Narrow AI. They solve a particular task. For AI reach to the state where it can better than humans in all aspects, not only do we need breakthroughs in algorithms, we also need many more breakthroughs in electronics and physics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please read this article. Summary is experts(around 350) estimate that there’s a 50% chance that AGI will occur until 2060. So, there is a very bleak chance that AGI will become a reality in next decade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.aimultiple.com/artificial-general-intelligence-singularity-timing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.aimultiple.com/artificial-general-intelligence-singularity-timing/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2019-09-23T08:39:56.877" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15596" PostTypeId="2" ParentId="15589" CreationDate="2019-09-23T12:33:40.220" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The program would have general knowledge of thing like squares, cells, rows, collumns, colours, letters, numbers and so on.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What do you mean by &quot;general knowledge&quot; of this objects ? If you intend to give it similar knowledge that a human would have about them you are going to have a realy hard time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a broader task but the Cyc project is attempting to assemble a comprehensive knowledge base that spans the basic concepts that every little child know and have worked on it for the last 35 years (&lt;a href=&quot;https://en.wikipedia.org/wiki/Cyc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Cyc&lt;/a&gt;). You could check what they have on their database about the thing that interest you but even so it will be a really hard task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically what Cyc is trying to do is to create a huge dataset which contain every &quot;common knowledge&quot; about the world. In your case the world would be limited to just the dot, line, grid, letter, etc but it would still be a huge work to create this database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the way AI community dealed with &quot;general knowledges&quot; for a long time. An alternative to this approach would be to use machine learning but I have no idea how it could be used for this objective (apart from using NLP to help you create the database which is what Cyc is now doing)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I wondered if there is anything similar out there already?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As far as I know there is nothing of the sort as of now. There exist AI which can learn to play multiple video game but they don't read the rule (like : &quot;Find a path through all the white squares.&quot;), they just try things until they find the &quot;good&quot; way to play a game. This AI generally use reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If an AI could read instructions and solve the puzzles could we say that it is in someway intelligent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The concept of intelligence is not something on which everyone agree and even so most of the people think that there is different level of intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my point of view this AI would have a certain level of intelligence if it had the possibility to learn a new kind of game (even if somewhat similar to the ones it already know) in just a few try (I can't give you a number but definitively not the thousands that actual method would require). Just like you presented it this ability would probably require your AI to have some level of understanding of the different common concepts between games (lines, dots, path ...).&lt;/p&gt;&#xA;" OwnerUserId="26961" LastEditorUserId="26961" LastEditDate="2019-09-23T12:48:07.107" LastActivityDate="2019-09-23T12:48:07.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15597" PostTypeId="2" ParentId="15588" CreationDate="2019-09-23T12:45:11.703" Score="1" Body="&lt;p&gt;(this response should be a comment but I don't have yet the reputation to comment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm understanding your problem correctly you have a variable number of input which have an order and only one output ? It look like the kind of task where you could use recurrent neural network (the most common ones are the LSTM and GRU). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use a recurrent neural network you could (if you have the timestamps of your data) cut the hourly interval into smaller time step to help detect pattern.&lt;/p&gt;&#xA;" OwnerUserId="26961" LastActivityDate="2019-09-23T12:45:11.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15598" PostTypeId="1" CreationDate="2019-09-23T14:05:46.593" Score="6" ViewCount="86" Body="&lt;p&gt;I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip.&lt;/p&gt;&#xA;&lt;p&gt;If, instead of layers that would be fully connected, I have layers with neurons connected in some other way (some pairs of neurons connected, others not), how is this going to affect the hardware acceleration?&lt;/p&gt;&#xA;&lt;p&gt;An example of this is the convolutional networks. However, there is still a clear pattern, which perhaps is exploited by the acceleration, which would mean that if there is no such pattern, the acceleration would not work as well?&lt;/p&gt;&#xA;&lt;p&gt;Should this be a concern? If so, is there some rule of thumb for how the connectivity pattern is going to affect the efficiency of hardware acceleration?&lt;/p&gt;&#xA;" OwnerUserId="29923" LastEditorUserId="2444" LastEditDate="2020-08-15T09:38:59.053" LastActivityDate="2020-08-15T09:38:59.053" Title="How do neural network topologies affect GPU/TPU acceleration?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;architecture&gt;&lt;hardware&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15600" PostTypeId="2" ParentId="15590" CreationDate="2019-09-23T19:16:20.830" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It might be too philosophical answer, but maybe first we need to answer the question whether a human way of thinking or his creativeness includes random elements. For example if an author writing a book uses some randomness in developing some side thread or some episodic character and I would say, that yes - sometimes we think up of something random.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some algorithms uses randomness at their basis, for example evolutionary algorithms for generating first population.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-09-23T19:16:20.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15601" PostTypeId="1" AcceptedAnswerId="15607" CreationDate="2019-09-23T20:39:23.957" Score="6" ViewCount="527" Body="&lt;p&gt;My weights go from being between 0 and 1 at initialisation to exploding into the tens of thousands in the next iteration. In the 3rd iteration they become so large that only arrays of nan values are displayed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I go about fixing this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it to do with the unstable nature of the sigmoid function, or is one of my equations incorrect during backpropagation which makes my gradients explode? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any advice&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from numpy import exp&#xA;import matplotlib.pyplot as plt&#xA;import h5py&#xA;&#xA;# LOAD DATASET&#xA;MNIST_data = h5py.File('data/MNISTdata.hdf5', 'r')&#xA;x_train = np.float32(MNIST_data['x_train'][:])&#xA;y_train = np.int32(np.array(MNIST_data['y_train'][:,0]))&#xA;x_test = np.float32(MNIST_data['x_test'][:])&#xA;y_test = np.int32(np.array(MNIST_data['y_test'][:,0]))&#xA;MNIST_data.close()&#xA;&#xA;##############################################################################&#xA;# PARAMETERS &#xA;number_of_digits = 10 # number of outputs&#xA;nx = x_test.shape[1] # number of inputs ... 784 --&amp;gt; 28*28&#xA;ny = number_of_digits&#xA;m_train = x_train.shape[0]&#xA;m_test = x_test.shape[0]&#xA;Nh = 30 # number of hidden layer nodes&#xA;alpha = 0.001&#xA;iterations = 3&#xA;##############################################################################&#xA;# ONE HOT ENCODER - encoding y data into 'one hot encoded'&#xA;lr = np.arange(number_of_digits)&#xA;y_train_one_hot = np.zeros((m_train, number_of_digits))&#xA;y_test_one_hot = np.zeros((m_test, number_of_digits))&#xA;for i in range(len(y_train_one_hot)):&#xA;  y_train_one_hot[i,:] = (lr==y_train[i].astype(np.int))&#xA;for i in range(len(y_test_one_hot)):&#xA;  y_test_one_hot[i,:] = (lr==y_test[i].astype(np.int))&#xA;&#xA;# VISUALISE SOME DATA&#xA;for i in range(5):&#xA;  img = x_train[i].reshape((28,28))&#xA;  plt.imshow(img, cmap='Greys')&#xA;  plt.show()&#xA;&#xA;y_train = np.array([y_train]).T&#xA;y_test = np.array([y_test]).T&#xA;##############################################################################&#xA;# INITIALISE WEIGHTS &amp;amp; BIASES&#xA;params = { &quot;W1&quot;: np.random.rand(nx, Nh),&#xA;           &quot;b1&quot;: np.zeros((1, Nh)),&#xA;           &quot;W2&quot;: np.random.rand(Nh, ny),&#xA;           &quot;b2&quot;: np.zeros((1, ny))&#xA;          }&#xA;&#xA;# TRAINING&#xA;# activation function&#xA;def sigmoid(z):&#xA;  return 1/(1+exp(-z))&#xA;&#xA;# derivative of activation function&#xA;def sigmoid_der(z):&#xA;  return z*(1-z)&#xA;&#xA;# softamx function&#xA;def softmax(z):&#xA;  return 1/sum(exp(z)) * exp(z)&#xA;&#xA;# softmax derivative is alike to sigmoid&#xA;def softmax_der(z):&#xA;  return sigmoid_der(z)&#xA;&#xA;def cross_entropy_error(v,y):&#xA;  return -np.log(v[y])&#xA;&#xA;# forward propagation&#xA;def forward_prop(X, y, params):&#xA;  outs = {}&#xA;  outs['A0'] = X&#xA;  outs['Z1'] = np.matmul(outs['A0'], params['W1']) + params['b1']&#xA;  outs['A1'] = sigmoid(outs['Z1'])&#xA;  outs['Z2'] = np.matmul(outs['A1'], params['W2']) + params['b2']&#xA;  outs['A2'] = softmax(outs['Z2'])&#xA;&#xA;  outs['error'] = cross_entropy_error(outs['A2'], y)&#xA;  return outs&#xA;&#xA;# back propagation&#xA;def back_prop(X, y, params, outs):&#xA;  grads = {}&#xA;  Eo = (y - outs['A2']) * softmax_der(outs['Z2'])&#xA;  Eh = np.matmul(Eo, params['W2'].T) * sigmoid_der(outs['Z1'])&#xA;  dW2 = np.matmul(Eo.T, outs['A1']).T&#xA;  dW1 = np.matmul(Eh.T, X).T&#xA;  db2 = np.sum(Eo,0)&#xA;  db1 = np.sum(Eh,0)&#xA;&#xA;  grads['dW2'] = dW2&#xA;  grads['dW1'] = dW1&#xA;  grads['db2'] = db2&#xA;  grads['db1'] = db1&#xA;#  print('dW2:',grads['dW2'])&#xA;  return grads&#xA;&#xA;# optimise weights and biases&#xA;def optimise(X,y,params,grads):&#xA;  params['W2'] -= alpha * grads['dW2']&#xA;  params['W1'] -= alpha * grads['dW1']&#xA;  params['b2'] -= alpha * grads['db2']&#xA;  params['b1'] -= alpha * grads['db1']&#xA;  return &#xA;&#xA;# main&#xA;for epoch in range(iterations):&#xA;  print(epoch)&#xA;  outs = forward_prop(x_train, y_train, params)&#xA;  grads = back_prop(x_train, y_train, params, outs)&#xA;  optimise(x_train,y_train,params,grads)&#xA;  loss = 1/ny * np.sum(outs['error'])&#xA;  print(loss)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29877" LastEditorUserId="2444" LastEditDate="2019-09-27T00:23:15.663" LastActivityDate="2019-10-28T10:22:58.520" Title="How to deal with large (or NaN) neural network's weights?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;exploding-gradient-problem&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15602" PostTypeId="2" ParentId="15562" CreationDate="2019-09-23T21:57:04.337" Score="0" Body="&lt;p&gt;I found out how to switch it (the code) to do text generation task, use 3D input (X) and 3D labels (Y) as in the source code below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf;&#xA;tf.reset_default_graph();&#xA;&#xA;#data&#xA;'''&#xA;t0       t1       t2&#xA;british  gray     is  cat&#xA;0        1        2   (3)  &amp;lt;=x&#xA;1        2        3        &amp;lt;=y&#xA;white    samoyed  is  dog&#xA;4        5        2   (6)  &amp;lt;=x&#xA;5        2        6        &amp;lt;=y &#xA;'''&#xA;Bsize = 2;&#xA;Times = 3;&#xA;Max_X = 5;&#xA;Max_Y = 6;&#xA;&#xA;X = [[[0],[1],[2]], [[4],[5],[2]]];&#xA;Y = [[[1],[2],[3]], [[5],[2],[6]]];&#xA;&#xA;#normalise&#xA;for I in range(len(X)):&#xA;  for J in range(len(X[I])):&#xA;    X[I][J][0] /= Max_X;&#xA;&#xA;for I in range(len(Y)):&#xA;  for J in range(len(Y[I])):&#xA;    Y[I][J][0] /= Max_Y;&#xA;&#xA;#model&#xA;Input    = tf.placeholder(tf.float32, [Bsize,Times,1]);&#xA;Expected = tf.placeholder(tf.float32, [Bsize,Times,1]);&#xA;&#xA;#single LSTM layer&#xA;'''&#xA;Layer1   = tf.keras.layers.LSTM(20);&#xA;Hidden1  = Layer1(Input);&#xA;'''&#xA;&#xA;#multi LSTM layers&#xA;#'''&#xA;Layers = tf.keras.layers.RNN([&#xA;  tf.keras.layers.LSTMCell(30), #hidden 1&#xA;  tf.keras.layers.LSTMCell(20)  #hidden 2&#xA;],&#xA;return_sequences=True);&#xA;Hidden2 = Layers(Input);&#xA;#'''&#xA;&#xA;Weight3  = tf.Variable(tf.random_uniform([20,1], -1,1));&#xA;Bias3    = tf.Variable(tf.random_uniform([   1], -1,1));&#xA;Output   = tf.sigmoid(tf.matmul(Hidden2,Weight3) + Bias3); #sequence of 2d * 2d&#xA;&#xA;Loss     = tf.reduce_sum(tf.square(Expected-Output));&#xA;Optim    = tf.train.GradientDescentOptimizer(1e-1);&#xA;Training = Optim.minimize(Loss);&#xA;&#xA;#train&#xA;Sess = tf.Session();&#xA;Init = tf.global_variables_initializer();&#xA;Sess.run(Init);&#xA;&#xA;Feed   = {Input:X, Expected:Y};&#xA;Epochs = 10000;&#xA;&#xA;for I in range(Epochs): #number of feeds, 1 feed = 1 batch&#xA;  if I%(Epochs/10)==0: &#xA;    Lossvalue = Sess.run(Loss,Feed);&#xA;    print(&quot;Loss:&quot;,Lossvalue);&#xA;  #end if&#xA;&#xA;  Sess.run(Training,Feed);&#xA;#end for&#xA;&#xA;Lastloss = Sess.run(Loss,Feed);&#xA;print(&quot;Loss:&quot;,Lastloss,&quot;(Last)&quot;);&#xA;&#xA;#eval&#xA;Results = Sess.run(Output,Feed).tolist();&#xA;print(&quot;\nEval:&quot;);&#xA;for I in range(len(Results)):&#xA;  for J in range(len(Results[I])):&#xA;    for K in range(len(Results[I][J])):&#xA;      Results[I][J][K] = round(Results[I][J][K]*Max_Y);&#xA;#end for i      &#xA;print(Results);&#xA;&#xA;print(&quot;\nDone.&quot;);&#xA;#eof&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-09-23T21:57:04.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15603" PostTypeId="2" ParentId="15590" CreationDate="2019-09-23T23:49:11.127" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is randomness (either true randomness or simulated randomness) necessary for AI&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It depends on how you define Artificial Intelligence.  If you regard it strictly as an intentionally created construct which demonstrates utility, then no.  (For instance, &lt;a href=&quot;https://en.wikipedia.org/wiki/Nimatron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nimatron&lt;/a&gt;, potentially the first functioning AI, beat most human competitors at NIM.  But Nimatron was classical AI, entirely rules based with no learning.) That said:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Randomness has proved a useful component in machine learning, and any feasible AGI would likely require ML.&lt;/strong&gt;  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given sufficient computing power, aka time and space, it would absolutely be possibly to brute force anything, including AGI, but the resulting algorithm would be &quot;brittle&quot;, unable to &quot;compute&quot; anything not previously defined. A learning algorithm, presented a problem outside of its domain of knowledge may initially degrade in performance, but it can learn from those outcomes, and gradually improve performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;IBM brute forced Chess with &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)#Design&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Blue&lt;/a&gt;, but Chess is a strictly narrow problem that turned out not to require general intelligence.  AGI requires human level performance in all tasks engaged in by humans, which, even if they could be broken down to a set of individual narrow problems, it's an ever expanding set of problems.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it mean &quot;intelligence comes from randomness&quot;? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not if the definition of intelligence is rooted in utility because deterministic processes can demonstrate utility. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;In statistical AI, the intelligence arises from the analysis of random search or the fitness of the genetic algorithm, not the randomness per se.&lt;/strong&gt;  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In other words, if you have the randomness without the analysis, every decision is an unqualified guess. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My sense is that it is free will that would arise from randomness—effects unrelated to causes—because without true randomness, the universe and everything in it is purely deterministic.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-09-24T00:19:48.940" LastActivityDate="2019-09-24T00:19:48.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15604" PostTypeId="2" ParentId="15594" CreationDate="2019-09-24T02:00:57.873" Score="9" Body="&lt;p&gt;I agree that this is too broad, but here's a 1 sentence answer for most of them. The ones I left out (from the bottom of the chart) are very modern, and very specialized. I don't know much about them, so perhaps someone who does can improve this answer.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Perceptron: Linear or logistic-like regression (and thus, classification).&lt;/li&gt;&#xA;&lt;li&gt;Feed Forward: Usually non-linear regression or classification with sigmoidal activation. Essentially a multi-layer perceptron.&lt;/li&gt;&#xA;&lt;li&gt;Radial Basis Network: Feed Forward network with Radial Basis activation functions. Used for classification and some kinds of video/audio filtering&lt;/li&gt;&#xA;&lt;li&gt;Deep Feed Forward: Feed Forward with more than 1 hidden layer. Used to learn more complex patterns in classification or regression, maybe reinforcement learning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Recurrent Neural Network: A Deep Feed Forward Network where some nodes connect to &lt;em&gt;past&lt;/em&gt; layers. Used in reinforcement learning, and&#xA;to learn patterns in sequential data like text or audio.&lt;/li&gt;&#xA;&lt;li&gt;LSTM: A recurrent neural network with specialized control neurons (sometimes called gates) that allow signals to be remembered for longer periods of time, or selectively forgotten. Used in any RNN application, and often able to learn sequences that have a very long repetition time. &lt;/li&gt;&#xA;&lt;li&gt;GRU: Much like LSTM, another kind of gated RNN with specialized control neurons.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Auto Encoder: Learns to compress data and then decompress it. After learning this model, it can be split into two useful subparts: a mapping from the input space to a low-dimensional feature space, that may be easier to interpret or understand; and a mapping from a small dimensional subspace of simple numbers into complex patterns, which can be used to generate those complex patterns. Basis of much modern work in vision, language, and audio processing.&lt;/li&gt;&#xA;&lt;li&gt;VAE, DAE,SAE: Specializations of the Auto Encoder.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Markov Chain: A neural network representation of a markov chain: State is encoded in the set of neurons that are active, and&#xA;transition probabilities are thus defined by the weights. Used for&#xA;learning transition probabilities and unsupervised feature learning&#xA;for other applications.&lt;/li&gt;&#xA;&lt;li&gt;HN, BM, RBM, DBM: Specialized architectures based on the Markov Chain idea, used to automatically learn useful features for other&#xA;applications.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Deep Convolutional Network: Like a feed-forward network, but each node is really  a bank of nodes learning a convolution from the layer&#xA;before it. This essentially allows it to learn filters, edge&#xA;detectors, and other patterns of interest in video and audio&#xA;processing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Deep Deconvolutional Network: Opposite of a Convolutional Network in some sense. Learn a mapping from features that represent edges or&#xA;other high level properties of some unseen image, back to the pixel&#xA;space. Generate images from summaries.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;DCIGN: Essentially an auto-encoder made of a DCN and a DN stuck together. Used to learn generative models for complex images like&#xA;faces.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generative Adversarial Network: Used to learn generative models for complex images (or other data types) when not enough training data is&#xA;available for a DCIGN. One model learns to generate data from random&#xA;noise, and the other learns to classify the output of the first&#xA;network as distinct from whatever training data is available.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-09-25T02:15:15.033" LastActivityDate="2019-09-25T02:15:15.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15605" PostTypeId="2" ParentId="15451" CreationDate="2019-09-24T02:06:04.840" Score="1" Body="&lt;p&gt;It is somewhat common to measure the number of &lt;em&gt;operations&lt;/em&gt; involved in the search. This is only really useful if you're doing scientific work, because the computational cost of measuring it accurately is quite high. For example, if you were using a GA that used tournament-based fitness selection, and wanted to compare it to one that used round-robin selection, counting the number of evaluations of individuals would be a good measure of total computational effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Genetic Programming, it is fairly common to build the measurement of the total op count into the interpreter you write for the programs. You can then compare that directly to the number of evals times the length of the genomes for something like a GA.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-24T02:06:04.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15607" PostTypeId="2" ParentId="15601" CreationDate="2019-09-24T07:03:58.003" Score="5" Body="&lt;p&gt;This problem is called exploding gradients, resulting in an unstable network that at best cannot learn from the training data and at worst results in NaN weight values that can no longer be updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to assure it is exploding gradients, is if loss is unstable and not improving, or if loss shows NaN value during training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from the usual gradient clipping and weights regularization that are recommended, I think the problem with your network is the architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;30 is an abnormally high number of nodes for 2 layer perceptron model. Try increasing number of layers and reducing nodes per layer. - This is under the assumption that you're experimenting with MLP's, because for the problem above, convolutional neural networks seem like an obvious way to go. If unexplored - definitely check out CNN's  for digit recognition, two layer models will surely work better there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helped!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-09-24T07:03:58.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15608" PostTypeId="2" ParentId="15588" CreationDate="2019-09-24T07:13:34.383" Score="1" Body="&lt;p&gt;I have come across the same issue but in language. Where each input was a sentence, hence of different lengths. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easier solution is to just find the longest sequence, extract its length, and 0 pad all other values to be able to get all of them to the same size, and then use any recurrent neural network architecture (&lt;em&gt;Since you're dealing with a time series&lt;/em&gt;), where these padded values act as input. (&lt;em&gt;When 0 padding, it is best if fewer of your layers have a bias value, because you want all multiplication with 0's to just give 0's as they represent a pad, having bias would mean after the weight multiplies with 0 it adds a small bias value, which is undesired because it is irrelevant information, One can always assume backprop will learn 0's for the biases even if present, but it never actually does, they mostly learn extremely small values, which can also hinder results&lt;/em&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to experiment with is creating a small network, to create an embedding for each input. The padded input can be input to a small neural network which generates a fixed size embedding (&lt;em&gt;Activations of a particular layer&lt;/em&gt; ; &lt;em&gt;Backprop will learn a decent representation when trained end-to-end&lt;/em&gt; ) that is the input to the recurrent neural architecture. - Adding a network to create an embedding almost always helps, as long as kept simple. Experimenting with the embedding algorithm (network), can help you find an idealistic one, which can significantly boost your results.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-09-24T07:13:34.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15609" PostTypeId="2" ParentId="3172" CreationDate="2019-09-24T07:24:38.653" Score="3" Body="&lt;p&gt;Edge Computing is an approach for extended from cloud computing which leverages the same concept but has its advantage like mitigating the latency, resource usage, energy usage and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Federated learning is just an algorithm or a kind of approach which empower the edge computing by applying the technique of model iteration instead of fetching data from the device.&#xA;It also removes privacy concern in edge computing.&lt;/p&gt;&#xA;" OwnerUserId="14395" LastEditorUserId="9947" LastEditDate="2019-09-24T13:05:27.047" LastActivityDate="2019-09-24T13:05:27.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15611" PostTypeId="1" CreationDate="2019-09-24T15:39:38.640" Score="3" ViewCount="118" Body="&lt;p&gt;I'm wondering if there is a NN that can achieve the following task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output a unit vector that is parallel to the input vector. i.e., input a vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}\in\mathbb{R}^d$&lt;/span&gt;, output &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}/\|\mathbf{v}\|$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dimension &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; can be fixed, say &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve this, it seems to me that we need to use NN to do three functions: square, square-root, and division. But I don't know if a NN can do all of these. &lt;/p&gt;&#xA;" OwnerUserId="29948" LastEditorUserId="9947" LastEditDate="2019-09-24T17:01:20.410" LastActivityDate="2019-09-24T17:01:20.410" Title="Vector normalization by a neural network" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15612" PostTypeId="1" CreationDate="2019-09-24T16:08:37.407" Score="5" ViewCount="66" Body="&lt;p&gt;I have a use case where the set of actions is different for different states. Is the agent aware of what actions are valid for each state, or is the agent only aware of the entire action space (in which case I guess the environment needs to discard invalid actions)?&lt;/p&gt;&#xA;&lt;p&gt;I presume the answer is yes, but I would like to confirm.&lt;/p&gt;&#xA;" OwnerUserId="29949" LastEditorUserId="2444" LastEditDate="2020-11-17T21:54:50.510" LastActivityDate="2020-11-17T21:54:50.510" Title="Is the agent aware of a possible different set of actions for each state?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15613" PostTypeId="1" CreationDate="2019-09-24T19:35:11.970" Score="3" ViewCount="52" Body="&lt;p&gt;As computers are getting bigger better and faster, the concept of what constitutes a single datum is changing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the world of pen-and-paper, we might take readings of temperature over time and obtain a time-series in which an individual datum is a time, temperature pair.  However, it is now common to desire &lt;a href=&quot;http://timeseriesclassification.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;classifications of entire time-series&lt;/a&gt;, in the context of which our entire temperature time-series would be but a single data point in a data set consisting of a great number of separate time-series.  In image processing, an (x,y,c) triple is not a datum, but a whole grid of such values is a single datum.  With lidar data and all manner of other fields things that were previously considered a &lt;em&gt;dataset&lt;/em&gt; are now best thought of as a &lt;em&gt;datum&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the term for datasets that are themselves composed of datasets?  The term &quot;metadata&quot; is occupied, I should think.  Are there any papers that talk about this transition from datasets of data to datasets of datasets and what the implications are for data scientists and researchers?&lt;/p&gt;&#xA;" OwnerUserId="21298" LastActivityDate="2019-09-26T19:35:04.263" Title="Terminology for the use of datasets as data points" Tags="&lt;terminology&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15615" PostTypeId="2" ParentId="15612" CreationDate="2019-09-25T02:40:20.897" Score="1" Body="&lt;p&gt;This is actually an implementation choice, and will depend on how you chose to represent the agent's model of the function that maps from states to actions.&lt;/p&gt;&#xA;&lt;p&gt;If you explicitly represent the entire state space, as you might chose to do with simple benchmark problems that you solve by directly solving an MDP with something like &lt;a href=&quot;https://www.cs.cmu.edu/afs/cs/project/jair/pub/volume4/kaelbling96a-html/node19.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;value iteration&lt;/a&gt;, then you can also easily explicitly represent exactly the set of actions that the agent can perform in each state, and the agent can learn the expected value of just taking those actions.&lt;/p&gt;&#xA;&lt;p&gt;If your state space is very large, you may not be able to represent it explicitly, and your agent is more likely to use some approximation of the value function or its policy, as is commonly done in &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Learning&lt;/a&gt;. Here, it is often preferable to define your model of the environment so that taking an invalid action in a state causes some well-defined outcome, or causes the agent to randomly re-select its actions until it ends up picking a valid one. The agent will eventually learn that selecting an invalid action leads to bad outcomes, without &amp;quot;realizing&amp;quot; that the action is invalid.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-11-17T21:54:09.500" LastActivityDate="2020-11-17T21:54:09.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15616" PostTypeId="1" CreationDate="2019-09-25T05:33:16.187" Score="13" ViewCount="176" Body="&lt;p&gt;Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our results. Instead, we do 10 fold cross validation (CV) to estimate how well the model might generalize.&lt;/p&gt;&#xA;&lt;p&gt;We're training deep learning models that take between 24-48 hours each, and the process of parameter sweeping is obviously very slow when performing 10-fold cross validation.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any experience or citations for how well parameter sweeping on one split of the data followed by cross validation (used to estimate how well it generalizes) works?&lt;/p&gt;&#xA;&lt;p&gt;I suspect it's highly dependent on the distribution of data and local minima &amp;amp; maxima of the hyper parameters, but I wanted to ask.&lt;/p&gt;&#xA;" OwnerUserId="23125" LastEditorUserId="43231" LastEditDate="2020-12-31T18:01:59.807" LastActivityDate="2021-05-01T00:07:47.653" Title="Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hyperparameter-optimization&gt;&lt;cross-validation&gt;&lt;generalization&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15617" PostTypeId="1" CreationDate="2019-09-25T08:37:53.380" Score="1" ViewCount="23" Body="&lt;p&gt;I'm using Q-learning with some extensions such as noisy linear layers, n-steps and double DQN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training, however, isn't that successful, my rewards are descreasing over time after a steep increase at the beginning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aZoJe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aZoJe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what's interesting is that my td loss is also descreasing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wMY8V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wMY8V.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sigma magnitudes of the noisy linear layers which control the exploration are strangely increasing, and also, seems to converge. I expected it to reduce uncertainty over time, but the opposite is the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/msID4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/msID4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another intresting thing, and that's probably why my loss is decreasing: The model tends to generate always the same transition, which is why the episodes are ending early and the rewards are getting lower. My experience replay is full of this single transition (around 99 percent of the buffer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the reason? Which things I should check? Is there anything I could try? I'm also willing to add information, just comment what could be of interest.&lt;/p&gt;&#xA;" OwnerUserId="21685" LastEditorUserId="21685" LastEditDate="2019-09-25T13:46:59.397" LastActivityDate="2019-09-25T13:46:59.397" Title="TD losses are descreasing, but also rewards are decreasing, increasing sigma" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15618" PostTypeId="1" CreationDate="2019-09-25T08:41:48.707" Score="2" ViewCount="1039" Body="&lt;p&gt;I've started getting into object detection in image. I have YOLOv3 neural network with Darknet framework. The network is pre-trained from COCO data set. Now I need to do some transfer learning in order to try to make the results better. What I have so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've created my custom data set and did transfer learning following this guide: &lt;a href=&quot;https://github.com/ultralytics/yolov3/wiki/Example:-Transfer-Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ultralytics/yolov3/wiki/Example:-Transfer-Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After few hours it spit out some &lt;code&gt;.pt&lt;/code&gt; file. From what I gathered &lt;code&gt;.pt&lt;/code&gt; is PyTorch format but my program uses &lt;code&gt;.weights&lt;/code&gt; which is Darknet format. First I've tried to find how to convert &lt;code&gt;.pt&lt;/code&gt; to &lt;code&gt;.weights&lt;/code&gt; and I stumbled upon this: &lt;a href=&quot;https://github.com/marvis/pytorch-caffe-darknet-convert&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/marvis/pytorch-caffe-darknet-convert&lt;/a&gt; but apparently this works only with YOLOv2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried to look for something on how to transfer learn using Darknet but with no luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then I'm stuck and can't move forward, can any of you give me some tips and pointers on how to achieve transfer learning on YOLOLv3 + Darknet (or how to convert &lt;code&gt;.pt&lt;/code&gt; to &lt;code&gt;.weights&lt;/code&gt; since I already know how to create &lt;code&gt;.pt&lt;/code&gt;)? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="29965" LastActivityDate="2019-09-25T08:41:48.707" Title="How to transfer learn Darknet YOLOv3" Tags="&lt;training&gt;&lt;pytorch&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15619" PostTypeId="1" CreationDate="2019-09-25T09:47:12.147" Score="8" ViewCount="118" Body="&lt;p&gt;Discussing the video &lt;a href=&quot;https://www.youtube.com/watch?time_continue=21&amp;amp;v=_sBBaNYex3E&quot; rel=&quot;nofollow noreferrer&quot;&gt;More Parkour Atlas&lt;/a&gt;, a friend asked how the robot's movement was so similar to the one from a real human and wondering how this is achieved?&lt;/p&gt;&#xA;&lt;p&gt;To my knowledge, this is not something the developer &amp;quot;programmed&amp;quot;, but instead emerged from the learning algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Could you provide an overview and some reference on how this is achieved?&lt;/p&gt;&#xA;" OwnerUserId="29969" LastEditorUserId="2444" LastEditDate="2021-01-22T00:42:05.003" LastActivityDate="2021-01-22T00:42:47.140" Title="How does Atlas from Boston Dynamics have human-like movement?" Tags="&lt;reference-request&gt;&lt;robotics&gt;&lt;human-like&gt;&lt;robots&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15621" PostTypeId="1" CreationDate="2019-09-25T13:59:41.587" Score="13" ViewCount="2937" Body="&lt;p&gt;I was following some examples to get familiar with TensorFlow's LSTM API, but noticed that all LSTM initialization functions require only the &lt;code&gt;num_units&lt;/code&gt; parameter, which denotes the number of hidden units in a cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to what I have learned from the famous &lt;a href=&quot;https://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;noreferrer&quot;&gt;colah's blog&lt;/a&gt;, the cell state has nothing to do with the hidden layer, thus they could be represented in different dimensions (I think), and then we should pass at least 2 parameters denoting both &lt;code&gt;#hidden&lt;/code&gt; and &lt;code&gt;#cell_state&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, this confuses me a lot when trying to figure out what the TensorFlow's cells do. Under the hood, are they implemented like this just for the sake of convenience or did I misunderstand something in the blog mentioned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pb4NN.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pb4NN.png&quot; alt=&quot;dimensions illustration&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29974" LastEditorUserId="2444" LastEditDate="2020-04-12T13:31:48.280" LastActivityDate="2021-05-24T10:13:34.653" Title="What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15622" PostTypeId="1" AcceptedAnswerId="15635" CreationDate="2019-09-25T14:20:06.840" Score="3" ViewCount="46" Body="&lt;p&gt;Let's assume I have a CNN model trained to categorize some objects on the images. By using this model I find more categorized images. If I now retrain this model on data set that consists old set and newly categorised images is there a chance that such new model will have higher accuracy? Or maybe because new data posses only information that could be found on initial set, model will have similar/lower accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if something unclear. &lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2019-09-25T18:17:49.697" LastActivityDate="2019-09-26T11:31:58.740" Title="Can a model, retrained on images classified previously by itself, increase its accuracy?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15623" PostTypeId="1" CreationDate="2019-09-25T15:00:22.613" Score="1" ViewCount="17" Body="&lt;p&gt;I am looking for a GAN paper I have read a while ago, but unfortunately cannot find it again. I think it compared GANs and other methods (like CVAEs) w.r.t. how they handle multi-modal data, not sure about the CVAEs though.&#xA;What I know is that they created different synthetic toy datasets with multiple modes to analyze this. I remember one plotting of a blue spiral of this data on a white background. Any guesses?&lt;/p&gt;&#xA;" OwnerUserId="17959" LastEditorUserId="2444" LastEditDate="2019-09-25T15:43:39.320" LastActivityDate="2019-09-25T15:43:39.320" Title="Looking for GAN paper with spiral image" Tags="&lt;generative-adversarial-networks&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15624" PostTypeId="1" CreationDate="2019-09-25T15:13:25.103" Score="6" ViewCount="53" Body="&lt;p&gt;Here's a sort of a conceptual question. I was implementing a SOM algorithm to better understand its variations and parameters. I got curious about one bit: the BMU (best matching unit == the neuron that is more similar to the vector being presented) is chosen as the neuron that has the smallest distance in feature space to the vector. Then I updated it and its neighbours.&lt;/p&gt;&#xA;&lt;p&gt;This makes sense, but what if I used more than one BMU for updating the network? For example, suppose that the distance to one neuron is 0.03, but there is another neuron with distance 0.04. These are the two smallest distances. I would use the one with 0.03 as the BMU.&lt;/p&gt;&#xA;&lt;p&gt;The question is, what would be the expected impacts on the algorithm if I used &lt;em&gt;more&lt;/em&gt; than one BMU? For example, I could be selecting for update all neurons for which the distance is up to 5% more than the minimum distance.&lt;/p&gt;&#xA;&lt;p&gt;I am not asking for code. I can implement it to see what happens. I am just curious to see if anyone have any insight on the pros and cons (except additional complexity) of this approach.&lt;/p&gt;&#xA;" OwnerUserId="29975" LastEditorUserId="43231" LastEditDate="2021-01-14T08:31:55.397" LastActivityDate="2021-02-13T09:02:25.240" Title="What is the impact of using multiple BMUs for self-organizing maps?" Tags="&lt;neural-networks&gt;&lt;self-organizing-map&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15626" PostTypeId="1" CreationDate="2019-09-25T17:01:24.040" Score="1" ViewCount="27" Body="&lt;p&gt;I have a use case where the state of the environment could change due to random events in between time steps that the agent takes actions. For example, at t1, the agent takes action a1 and is given the reward and the new state s1. Before the agent takes the next action at t2, some random events occurred in the environment that altered the state. Now when the agent takes action at t2, it's now acting on &quot;stale information&quot; since the state of the environment had changed. Also, the new state s2 will represent changes not only due to the agent's action, but also due to the prior random events that occurred. In the worst case, the action could possibly have become invalid for the new state that was introduced due to these random events occurred within the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we deal with this? Does this mean that this use case is not a good one to solve with RF? If we just ignore these changing states due to the random events in the environment, how would that affect the various learning algorithms? I presume that this is not a uncommon or unique problem in real-life use cases...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Francis&lt;/p&gt;&#xA;" OwnerUserId="29949" LastActivityDate="2019-09-25T17:01:24.040" Title="RF: How to deal with environments changing state due to external factors" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15627" PostTypeId="1" CreationDate="2019-09-25T19:03:51.430" Score="0" ViewCount="78" Body="&lt;p&gt;So I wrote a convolutional neural network for a binary image classification. I have around 5300 images for each class which I thought would be enough to at least give me a good accuracy on the training cases. When the training was finished, I wrote a program to find the accuracy for each class (here's what I mean: consider class A. I ran a for loop through all the training images that belonged to class A, gave the images to my classifier and divided the number of images that my classifier classified as a class A image by 5300).For class A, the accuracy was pretty good (around 96%). But for class B, the accuracy was only 80%. Further more, I did the same thing with the testing images and found out that the accuracy for class A was 95% but for class B was 70%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do in order to improve my accuracy? I thought about augmentation but wasn't sure whether to do it for both classes or just class B. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S: I think the main problem that causes this is that the two classes are really alike but I don't really know how to deal with it so any help regarding that would be appreciated!&lt;/p&gt;&#xA;" OwnerDisplayName="user17894" LastActivityDate="2019-09-30T12:06:17.313" Title="What to do when an image classifier does good for a class but bad for another?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;classification&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15628" PostTypeId="2" ParentId="15619" CreationDate="2019-09-25T23:35:05.590" Score="2" Body="&lt;p&gt;Because Boston Dynamics is a private, for-profit, company, we cannot know for sure how they achieve their results. However, we can examine the available public information and make educated guesses.&lt;/p&gt;&#xA;&lt;p&gt;In the information posted with the video, Boston Dynamics tells us that they use&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;... an optimization algorithm transforms high-level descriptions of&#xA;each maneuver into dynamically-feasible reference motions.  Then Atlas&#xA;tracks the motions using a model predictive controller that smoothly&#xA;blends from one maneuver to the next.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This sounds like three older AI approaches have been blended together to create the video.&lt;/p&gt;&#xA;&lt;p&gt;First, they mention using an optimization algorithm to assemble complex motions from &amp;quot;dynamically-feasible reference motions&amp;quot;. This sounds like they have first learned, or possibly pre-programmed, a range of simple movements that, on their own, are not very impressive, but that can be composed together into more complex movements. This approach is called &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/3-540-45164-1_38.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Layered Learning&lt;/a&gt;, and was pioneered by Peter Stone and Manuela Veloso in the late 1990s. It is widely and successfully used in academic robotics competitions. Basically, this algorithm tries out different combinations and sequences of simple actions until it finds one that is close to the desired complex action. This is usually done with a local search algorithm, or sometimes with other optimization tools.&lt;/p&gt;&#xA;&lt;p&gt;The second technique is, of course, actually learning to perform the basic actions that layered learning can compose together into more complex actions. This is usually done with some form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt;, but sometimes is done by a programming who explicitly solves the equations for the movement of a simple system.&lt;/p&gt;&#xA;&lt;p&gt;Finally, they need to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Model_predictive_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;model predictive controller&lt;/a&gt; to smoothly interpolate between the sequence of actions that the layered learning approach has come up with. In this approach, the engineers designing the robot have very carefully measured, for this specific robot, exactly how parts of it tend to move or continue moving, and written this down as a mathematical model called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamical_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;dynamical system&lt;/a&gt;. This model allows the algorithm to figure out how this specific robot's movements are slipping away from the planned ones as it executes the motions. The slippage occurs because of things like friction (the robot's parts are not all perfectly smooth, and may have different amounts of lubrication, or a motor might slip). The algorithm can then make small changes to the motion to smooth out these unexpected bumps. That's the part that makes everything look so smooth.&lt;/p&gt;&#xA;&lt;p&gt;It's worth noting too, that the video is not &lt;em&gt;fully&lt;/em&gt; representative of the typical results their algorithm obtains. Boston Dynamics claims in the video caption that their algorithm &amp;quot;succeeds&amp;quot; in achieving a desired motion about 80% of the time. They don't tell us what &amp;quot;success&amp;quot; means, but the video is probably just the best of many takes for a carefully planned and filmed routine.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-22T00:42:47.140" LastActivityDate="2021-01-22T00:42:47.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15631" PostTypeId="1" AcceptedAnswerId="15656" CreationDate="2019-09-26T02:04:05.900" Score="4" ViewCount="91" Body="&lt;p&gt;The following X-shape alternated pattern can be separated quite well and super fast by K-nearest Neighbour algorithm (go to &lt;a href=&quot;https://ml-playground.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ml-playground.com&lt;/a&gt; to test it):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vsi8v.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vsi8v.png&quot; alt=&quot;image of x-shaped data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, DNN seems to face great struggles to separate that X-shape alternated data. &lt;strong&gt;Is it possible to do K-nearest before DNN&lt;/strong&gt;, ie. set the DNN weights somehow to simulate the result of K-nearest before doing DNN training?&lt;/p&gt;&#xA;&lt;p&gt;Another place to test the X-shape alternated data: &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="156" LastEditDate="2021-05-17T01:43:41.930" LastActivityDate="2021-05-17T01:43:41.930" Title="Is it possible to do K-nearest-neighbours before training DNN" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;learning-algorithms&gt;&lt;greedy-ai&gt;&lt;symbolic-ai&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15632" PostTypeId="1" AcceptedAnswerId="15633" CreationDate="2019-09-26T06:15:57.067" Score="2" ViewCount="120" Body="&lt;p&gt;I am going through Russel and Norvig's &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt; (3rd edition). I was reading the part regarding the A* algorithm&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A* graph search version is optimal when heuristic is consistent and tree search version optimal when heuristic is just admissible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The book gives the following graph search algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IHBOd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IHBOd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above algorithm says that pop the node from the frontier set and expand it (assuming not in explored set), and add its children to frontier only if child not in frontier or explored set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I apply the same to A* (assuming a consistent heuristic) and suppose I find goal state (as a child of some node) for the first time I add it to the frontier set. Now, according to this algorithm, if the goal state is already in the frontier set, it must never be added again (this implies never be updated/promoted right?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I might as well stop the search when I find goal state for the first time as a child of some node and not wait till I pop the goal state from the frontier?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does a consistent heuristic guarantee that when I add a node to the frontier set I have found the optimal path to it? (because if I don't update it or re-add it with updated cost (according to the graph search algorithm, the answer to the question must be yes.)  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Am I missing something? Because it also states that, whenever A* selects the node for expansion, the optimal path to that node is found and doesn't say that when a node is added to the frontier set, the optimal path is found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I'm pretty confused, but I think the general graph search definition (in the above image) is misleading.&lt;/p&gt;&#xA;" OwnerUserId="5122" LastEditorUserId="2444" LastEditDate="2019-11-10T17:19:08.070" LastActivityDate="2019-11-10T17:19:08.070" Title="In the graph search version of A*, can I stop the search the first time I encounter the goal node?" Tags="&lt;definitions&gt;&lt;search&gt;&lt;a-star&gt;&lt;consistent-heuristic&gt;&lt;graph-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15633" PostTypeId="2" ParentId="15632" CreationDate="2019-09-26T06:45:51.003" Score="0" Body="&lt;p&gt;No, the optimal path is found when you pop the goal state. If you stop the search when you first add the goal state then the final path may not be optimal. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-09-26T06:45:51.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15635" PostTypeId="2" ParentId="15622" CreationDate="2019-09-26T11:25:46.587" Score="0" Body="&lt;p&gt;The most likely outcome of this approach is wasted time and very little effect on accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will be changes to the model. Some will be beneficial and improve the model, but some will backfire making it worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The model predicts with probability 0.4 that an image is in a certain class. It is the highest prediction, and actually &lt;em&gt;true&lt;/em&gt;. It will be added to the training dataset with a &quot;ground truth&quot; of probability 1.0, so on balance more and better data has been added to the data set. This will improve generalisation, as whatever caused the relatively low 0.4 value initially - e.g. a pose or lighting variation - will now be covered correctly in the training set.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The model predicts with probability 0.4 that an image is in a certain class. It is the highest prediction, and actually &lt;em&gt;false&lt;/em&gt;. It will be added to the dataset with &quot;ground truth&quot; of probability 1.0 for the &lt;em&gt;wrong class&lt;/em&gt;. This will weaken associations to the correct class for similar input images, meaning for exaple that a certain pose or lighting difference that is already causing problems for the model will be used to incorrectly classify images in future.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These two scenarios will occur, on average, at a rate determined by the model's current test accuracy. So if your current model is 90% accurate, 1 in 10 images in your new training data will be mislabelled. This will  &quot;lock in&quot; the current errors at the same rate on average as they already occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The effect may be a drift up or down in accuracy as the model will definitely change due to the new training data. However you have little to no control over how this drift effect goes if you are not willing or able to oversee the automatic classifications generated on new data by the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few ways to get some improvement unsupervised from new data. For instance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Build an autoencoder from the early convolutional layers of your model and train it to re-generate all inputs as outputs. This should help it learn important features of the variations in data that you are using. Once this training is done, discard the decoder part of the auto-encoder and add your classifier back in to fine tune it. This may help if you have only a small amount of labelled data, but a lot of unlabelled data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a model that has better accuracy than yours to auto-label the data. This might seem a little chicken-and-egg, but you may be able to create such a model using ensemble techniques. The ensemble model could be too awkward to use in production, but may still be used in an auto-labeling pipeline to improve your training data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note you may get even better results simply ignoring the extra unlabeled data, and instead fine tuning a high quality ImageNet-trained model on the labeled data you already have - saving yourself a lot of effort. Depends on the nature of the images, and how much labeled data you are already working with.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-26T11:31:58.740" LastActivityDate="2019-09-26T11:31:58.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15637" PostTypeId="1" CreationDate="2019-09-26T12:33:04.110" Score="2" ViewCount="179" Body="&lt;p&gt;I'm currently writing a program using keras (python 3) to play a game similar to Atari games, only in this one there are objects moving in the screen in different angles and directions (in most of Atari games I've encountered the objects you need to shoot are static). The agent's aim is to shoot them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;after executing every action I get feedback from the environment: I get the locations of all the objects on the screen, the locations of the collisions that happened, my position (angle of the turret) and the total score (from which I can calculate the reward)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I defined that each state will consists from the parameters mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use softmax algorithm in order to choose the next action, but I'm not sure how to do it. I'd be very grateful if anyone could help me or refer me to a source that can explain the syntax? currently I'm using decay epsilon-greedy algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your time and attention.&lt;/p&gt;&#xA;" OwnerUserId="29989" LastActivityDate="2019-09-26T12:33:04.110" Title="how to use Softmax action selection algorithm in atari-like game" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15639" PostTypeId="1" AcceptedAnswerId="15649" CreationDate="2019-09-26T16:21:35.740" Score="4" ViewCount="92" Body="&lt;p&gt;I have implemented a neural network from scratch (only using numpy) and I am having problems understanding why the results are so different between stochastic/minibatch gradient descent and batch gradient descent:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Bkn4o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bkn4o.png&quot; alt=&quot;A comparison between the prediction results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training data is a collection of point coordinates (x,y).&#xA;The labels are 0s or 1s (below or above the parabola).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aAoqc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aAoqc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a test, I am doing a classification task.&#xA;My objective is to make the NN learn which points are above the parabola (yellow) and which points are below the parabola (purple).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link to the notebook: &lt;a href=&quot;https://github.com/Pign4/ScratchML/blob/master/Neural%20Network.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Pign4/ScratchML/blob/master/Neural%20Network.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why is the batch gradient descent performing so poorly with respect&#xA;to the other two methods?&lt;/li&gt;&#xA;&lt;li&gt;Is it a bug? But how can it be since the code is almost identical to the&lt;br&gt;&#xA;minibatch gradient descent?&lt;/li&gt;&#xA;&lt;li&gt;I am using the same (randomly chosen with try and error) hyperparameters for all three&lt;br&gt;&#xA;neural networks. Does batch gradient descent need a more accurate&lt;br&gt;&#xA;technique to find the correct hyperparameters? If yes, why so?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10813" LastEditorUserId="10813" LastEditDate="2019-09-27T11:55:13.290" LastActivityDate="2019-09-27T11:55:13.290" Title="Why is batch gradient descent performing worse than stochastic and minibatch gradient descent?" Tags="&lt;python&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15640" PostTypeId="2" ParentId="15613" CreationDate="2019-09-26T19:16:10.867" Score="1" Body="&lt;p&gt;I don't think that this is anything new. Let's use your example of classifying an entire time series, say predicting word 1 vs word 2 for speech recognition. We can write out the data as a data frame like we would do with any other multivariate data: observations at time 1, time 2, etc as the predictors and the classification label as the response variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each observation is a vector of the values at particular times for your subject, plus the label--no different than any other multivariate data. Sure, there might be special dependence structure because of the time series nature of your data, but you can still write it as a multivariate problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, let's say that you hit the speech signal with a wavelet transform, resulting in an image-looking spectrogram of 2D data. Then just consider each &quot;pixel&quot; (time-frequency pair) to be a variable in your multivariate problem, along with the classification label. This is some kind of bijection between an &lt;span class=&quot;math-container&quot;&gt;$m\times n$&lt;/span&gt; matrix and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{n\times m}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can extend this idea to 3D or 4D data (or higher), too. Just unwrap the high-dimension tensor in some kind of map &lt;span class=&quot;math-container&quot;&gt;$T^{m\times n \times \dots} \rightarrow \mathbb{R}^{m\times n \times \dots}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25529" LastActivityDate="2019-09-26T19:16:10.867" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="15642" PostTypeId="1" CreationDate="2019-09-26T19:37:12.077" Score="3" ViewCount="524" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/pjreddie/darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;pjreddie's official darknet version&lt;/a&gt; (link from official website &lt;a href=&quot;https://pjreddie.com/darknet/install/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) has been forked several times. In particular I've come accross &lt;a href=&quot;https://github.com/AlexeyAB/darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlexeyAB's fork&lt;/a&gt; through &lt;a href=&quot;https://timebutt.github.io/static/how-to-train-yolov2-to-detect-custom-objects/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial. I assume the tutorial's author used AlexeyAB's fork because he wanted to use it on a Windows machine, which pjreddie's darknet cannot do AFAIK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not really concerned about that (I am a linux user), but I am very interested about the half precision option (&lt;code&gt;CUDNN_HALF&lt;/code&gt;) that AlexeyAB's darknet has, whereas pjreddie's darknet does not. Of course I've checked that this option was handled by the graphic card (RTX2080) we use at my office.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, I wonder: how stable/robust is that fork? Of course I want high-performing software, but I also want a certain level of stability! On the other hand, the latest commit on pjreddie's darknet is back from September 2018 (ie 1 year old), whereas AxeleyAB's darknet is active…&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More broadly, &lt;a href=&quot;https://github.com/search?q=darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;there seems to be a lot of darknet forks&lt;/a&gt;: which ones to prefer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the neural network community think?&lt;/p&gt;&#xA;" OwnerUserId="30003" LastActivityDate="2019-09-27T01:20:19.860" Title="What is the best variant of darknet to use?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15643" PostTypeId="1" CreationDate="2019-09-26T20:12:22.630" Score="1" ViewCount="302" Body="&lt;p&gt;I have CNN model to classify 2 classes. (Yes or No)&#xA;I use categorical_crossentropy loss and softmax activation at the end. &#xA;For input I use image with all 3 channels, for output I use One hot encoded vector ([0,1] or [1,0])&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have function that guaranty me, that each batch I have same number of one and another class, so the classes are not unevenly represented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happened when I train the model is that I am stuck at same loss while trening,...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that model predict always same class and half in batch has loss 0 half of them max, so that bring it to 8 all the time,...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could went wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network is something like this : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = Conv2D(16, (3, 3), padding='same')(input_img)&#xA;x = LeakyReLU(0.1)(x)&#xA;x = Conv2D(32 , (3, 3), padding='same')(x)&#xA;x = LeakyReLU(0.1)(x)&#xA;x = MaxPooling2D((2, 2))(x)&#xA;x = Dropout(0.25)(x)&#xA;x = Conv2D(32 , (3, 3), padding='same')(x)&#xA;x = LeakyReLU(0.1)(x)&#xA;x = Conv2D(48 , (3, 3), padding='same')(x)&#xA;x = LeakyReLU(0.1)(x)&#xA;x = MaxPooling2D((2, 2))(x)&#xA;x = Dropout(0.25)(x)&#xA;&#xA;x = Flatten()(x)&#xA;x = Dense(4096)(x)&#xA;x = LeakyReLU(0.1)(x)&#xA;x = Dropout(0.5)(x)&#xA;x = Dense(2048)(x)&#xA;x = LeakyReLU(0.1)(x)&#xA;x = Dropout(0.5)(x)&#xA;out = Dense(2, activation='softmax', name='table')(x)&#xA;&#xA;model = Model(input_img, out)&#xA;model.compile(optimizer='adam', loss= 'categorical_crossentropy')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Training Loss:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xLsTb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xLsTb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26993" LastEditorUserId="26993" LastEditDate="2019-10-02T04:35:56.183" LastActivityDate="2019-10-02T04:35:56.183" Title="CNN clasification model loss stuck at same value" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;categorical-data&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15645" PostTypeId="1" AcceptedAnswerId="15655" CreationDate="2019-09-26T23:49:34.430" Score="3" ViewCount="698" Body="&lt;p&gt;I am trying to train a CNN-LSTM model. The size of my images is 640x640. I have a GTX 1080 ti 11GB. I am using Keras with the TensorFlow backend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the model.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;img_input_1 = Input(shape=(1, n_width, n_height, n_channels))&#xA;conv_1 = TimeDistributed(Conv2D(96, (11,11), activation='relu', padding='same'))(img_input_1)&#xA;pool_1 = TimeDistributed(MaxPooling2D((3,3)))(conv_1)&#xA;conv_2 = TimeDistributed(Conv2D(128, (11,11), activation='relu', padding='same'))(pool_1)&#xA;flat_1 = TimeDistributed(Flatten())(conv_2)&#xA;dense_1 = TimeDistributed(Dense(4096, activation='relu'))(flat_1)&#xA;drop_1 = TimeDistributed(Dropout(0.5))(dense_1)&#xA;lstm_1 = LSTM(17, activation='linear')(drop_1)&#xA;dense_2 = Dense(4096, activation='relu')(lstm_1)&#xA;dense_output_2 = Dense(1, activation='sigmoid')(dense_2)&#xA;model = Model(inputs=img_input_1, outputs=dense_output_2)&#xA;&#xA;op = optimizers.Adam(lr=0.00001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.001)&#xA;&#xA;model.compile(loss='mean_absolute_error', optimizer=op, metrics=['accuracy'])&#xA;&#xA;model.fit(X, Y, epochs=3, batch_size=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Right now, using this model, I can only use the training data when the images are resized to 60x60, any larger and I run out of GPU memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use the largest possible size as I want to retain as much discriminatory information as possible. (The labels will be mouse screen coordinates between 0 - 640).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among many others, I found this question: &lt;a href=&quot;https://ai.stackexchange.com/q/3938/2444&quot;&gt;How to handle images of large sizes in CNN?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though I am not sure how I can &quot;restrict your CNN&quot; or &quot;stream your data in each epoch&quot; or if these would help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I reduce the amount of memory used so I can increase the image sizes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to sacrifice training time/computation speed in favor of higher resolution data whilst retaining model effectiveness?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the above model is not final, just a basic outlay.&lt;/p&gt;&#xA;" OwnerUserId="30005" LastEditorUserId="2444" LastEditDate="2020-05-22T23:49:41.177" LastActivityDate="2020-05-23T05:39:07.453" Title="How can I reduce the GPU memory usage with large images?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;keras&gt;&lt;gpu&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15647" PostTypeId="2" ParentId="15642" CreationDate="2019-09-27T00:04:12.080" Score="3" Body="&lt;p&gt;I used &lt;a href=&quot;https://github.com/thtrieu/darkflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trieu's Darkflow&lt;/a&gt; and it was trained well up to around 30 to 50 epoch with maximum 30 GB dataset. At the same time, it was quite often crashed after these epochs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Half precision...I don't remember Darkflow have that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to estimate each fork reputation is by number of Star and Issues, for example &lt;a href=&quot;https://github.com/AlexeyAB/darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alexey's Darknet&lt;/a&gt; 5,000 and 2,500 and Darkflow 5,000, 539 respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: OS Ubuntu 16.04&lt;/p&gt;&#xA;" OwnerUserId="27229" LastEditorUserId="27229" LastEditDate="2019-09-27T01:20:19.860" LastActivityDate="2019-09-27T01:20:19.860" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15648" PostTypeId="1" CreationDate="2019-09-27T08:12:46.980" Score="8" ViewCount="91" Body="&lt;p&gt;I have a set of fixed integers &lt;span class=&quot;math-container&quot;&gt;$S = \{c_1, \dots, c_N \}$&lt;/span&gt;. I want to find a single integer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, greater than a certain threshold &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$D &amp;gt; T \geq 0$&lt;/span&gt;, that divides each &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; and leaves remainder &lt;span class=&quot;math-container&quot;&gt;$r_i \geq 0$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$r_i$&lt;/span&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$r_i = c_i \text{ mod } D$&lt;/span&gt;, such that the sum of remainders is minimized.&lt;/p&gt;&#xA;&lt;p&gt;In other words, this is my problem&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\begin{aligned}&#xA;D^* \quad = \text{argmin}_D&amp;amp;  \sum_i c_i \text{ mod } D \\&#xA;\textrm{subject to} &amp;amp;\quad D &amp;gt; T&#xA;\end{aligned}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the integers have a common divisor, this problem is easy. If the integers are relatively co-prime however, then it is not clear how to solve it.&lt;/p&gt;&#xA;&lt;p&gt;The set &lt;span class=&quot;math-container&quot;&gt;$|S| = N$&lt;/span&gt; can be around &lt;span class=&quot;math-container&quot;&gt;$10000$&lt;/span&gt;, and each element also has a value in tens of thousands.&lt;/p&gt;&#xA;&lt;p&gt;I was thinking about solving it with a genetic algorithm (GA), but it is kind of slow. I want to know is there any other way to solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="30015" LastEditorUserId="2444" LastEditDate="2021-01-23T17:35:42.010" LastActivityDate="2021-01-23T17:35:42.010" Title="Given a list of integers $\{c_1, \dots, c_N \}$, how do I find an integer $D$ that minimizes the sum of remainders $\sum_i c_i \text{ mod } D$?" Tags="&lt;machine-learning&gt;&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;constrained-optimization&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15649" PostTypeId="2" ParentId="15639" CreationDate="2019-09-27T08:37:42.387" Score="2" Body="&lt;p&gt;Assuming the problem at hand is a classification (Above or Below parabola), this is probably because of the nature of Batch gradient descent. Since the gradient is being calculated on the whole batch, it tends to work well on only convex loss functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for why batch gradient descent is not working too great probably is because of the high number of minimas in the error manifold, ending up learning nothing relevant. You can change the loss function and observe the change in results, they might not be great (Batch GD usually isn't) but you'll be able to see differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check &lt;a href=&quot;https://stats.stackexchange.com/questions/49528/batch-gradient-descent-versus-stochastic-gradient-descent&quot;&gt;this&lt;/a&gt; out for more on the differences between the three. Hope this helped!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-09-27T08:37:42.387" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15650" PostTypeId="1" AcceptedAnswerId="15651" CreationDate="2019-09-27T13:52:15.007" Score="10" ViewCount="782" Body="&lt;p&gt;The No Free Lunch (NFL) theorem states (see the paper &lt;a href=&quot;http://fab.cba.mit.edu/classes/865.18/design/optimization/nfl.pdf&quot; rel=&quot;noreferrer&quot;&gt;Coevolutionary Free Lunches&lt;/a&gt; by David H. Wolpert and William G. Macready)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;any two algorithms are equivalent when their performance is averaged across all possible problems&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is the &quot;No Free Lunch&quot; theorem really true? What does it actually mean? A nice example (in ML context) illustrating this assertion would be nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen some algorithms which behave very poorly, and I have a hard time believing that they actually follow the above-stated theorem, so I am trying to understand whether my interpretation of this theorem is correct or not. Or is it just another ornamental theorem like Cybenko's Universal Approximation theorem?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-09-27T16:49:36.837" LastActivityDate="2019-10-04T17:32:40.763" Title="What are the implications of the &quot;No Free Lunch&quot; theorem for machine learning?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;proofs&gt;&lt;no-free-lunch-theorems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15651" PostTypeId="2" ParentId="15650" CreationDate="2019-09-27T16:43:01.863" Score="11" Body="&lt;p&gt;This is a really common reaction after first encountering the No Free Lunch theorems (NFLs). The one for machine learning is especially unintuitive, because it flies in the face of everything that's discussed in the ML community. That said, the theorem is true, but what it &lt;em&gt;means&lt;/em&gt; is open to some debate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To restate the theorem for people who don't know it, the NFL theorem for machine learning is really a special case of the &lt;a href=&quot;http://georgemaciunas.com/wp-content/uploads/2012/07/Wolpert_NLFoptimization-1.pdf&quot; rel=&quot;noreferrer&quot;&gt;NFL theorem for local search and optimization&lt;/a&gt;. The local search version is easier to understand. The theorem makes the following, somewhat radical claim:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Averaged across &lt;em&gt;all&lt;/em&gt; possible optimization problems, the average solution quality found by any local search algorithm you choose to use is &lt;em&gt;exactly&lt;/em&gt; the same as the average solution quality of a local &quot;search&quot; algorithm that just generates possible solutions by sampling uniformly at random from the space of all solutions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Another formulation, when people want an even stronger reaction, is to say that if you want to find the best solution to a problem, it's just as good to try things that seem to be making your solution iteratively worse as it is to try things that seem to be making your solution iteratively better. On average, both these approaches are equally good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, so &lt;em&gt;why&lt;/em&gt; is this true? Well, the key is in the details. Wolpert has sometimes described the theorem as a specialization of &lt;a href=&quot;https://plato.stanford.edu/entries/induction-problem/&quot; rel=&quot;noreferrer&quot;&gt;Hume's work on the problem of induction&lt;/a&gt;. The basic statement of the problem of induction is: we have no logical basis for assuming that the future will be like the past. Logically, there's no reason that the laws of physics couldn't all just radically change tomorrow. From a purely &lt;em&gt;logical&lt;/em&gt; perspective, it's totally reasonable that the future can be different from the past in any number of ways. Hume's problem is that, in general the future &lt;em&gt;is&lt;/em&gt; like the past in a lot of ways. He tried to formulate a philosophical (logical) argument that this &lt;em&gt;needed&lt;/em&gt; to be so, but basically failed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The No Free Lunch theorems say the same thing. If you don't know what your search space looks like, then if you iteratively refine your guess at what a good solution looks like, in response to the observations you've made in the &lt;em&gt;past&lt;/em&gt; about what good solutions look like (i.e. learning from data), then it's just as likely that operation you make helps as it is that it hurts. That's why the &quot;averaged over all possible optimization problems&quot; part is key. For any optimization problem where hill climbing is a good strategy after &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; moves, we can make one that is identical, except that the kth hill climbing move leads to an awful solution. The actual proof is more subtle than that, but that's the basic idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very brief lay summary might be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A machine learning algorithim can only be made to work better on some kinds of problems by being made to work worse on another kind of problem.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So what does this &lt;strong&gt;mean&lt;/strong&gt; in a practical sense? It means that you need to have some apriori &lt;em&gt;reason&lt;/em&gt; for thinking that your algorithm will be effective on a &lt;em&gt;particular&lt;/em&gt; problem. Exactly what a &lt;em&gt;good&lt;/em&gt; reason looks like is the subject of vigorous debate within the ML community. This is very closely related to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff&quot; rel=&quot;noreferrer&quot;&gt;bias/variance tradeoff&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some common responses are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When you're looking at a new optimization problem, although it &lt;em&gt;could&lt;/em&gt; have any random kind of structure, the problems we actually encounter in the real world are a lot more regular, and certain common themes are present, like the the fact that moving &quot;uphill&quot; (minimizing error) iteratively tends to lead to good solutions. Basically, this school of thought says NFL is an ornamental theorem: most ML algorithms work better on &quot;the kind of problems we see in real life&quot;, by working worse on &quot;the kind of problems we don't see in real life&quot;.&lt;/li&gt;&#xA;&lt;li&gt;When you're looking at a new optimization problem in [insert your favourite application domain], although it &lt;em&gt;could&lt;/em&gt; have any random kind of structure,  problems tend to look like [whatever you think], which makes [your favorite algorithm] a lot more effective than random guessing.&lt;/li&gt;&#xA;&lt;li&gt;Wolpert &amp;amp; McCready themselves published an &lt;a href=&quot;https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20060007558.pdf&quot; rel=&quot;noreferrer&quot;&gt;interesting result&lt;/a&gt; showing that there actually &lt;em&gt;are&lt;/em&gt; specialized optimization processes, based on co-evolution, that are &lt;em&gt;consistently&lt;/em&gt; better than random guessing. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regardless, it's indisputable that some algorithms are better than others, in certain sub-domains (we can see this empirically). NFL tells us that to be better there, they need to be worse somewhere else. The question up for debate is whether the &quot;somewhere else&quot; is real problems, or purely artificial ones.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-04T17:32:40.763" LastActivityDate="2019-10-04T17:32:40.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15652" PostTypeId="5" CreationDate="2019-09-27T16:51:52.867" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-27T16:51:52.867" LastActivityDate="2019-09-27T16:51:52.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15653" PostTypeId="4" CreationDate="2019-09-27T16:51:52.867" Score="0" Body="For questions related to the various No Free Lunch (NFL) theorems (both in the context of machine learning and optimization)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-27T16:51:52.867" LastActivityDate="2019-09-27T16:51:52.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15655" PostTypeId="2" ParentId="15645" CreationDate="2019-09-27T22:09:02.393" Score="2" Body="&lt;p&gt;As the other links suggest, you basically have four options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;restrict your CNN&quot;. This means making your model smaller and simpler, possibly by inserting a pooling layer at the front, or reducing the total number of layers. From a memory perspective, this isn't likely to produce really large gains though.&lt;/li&gt;&#xA;&lt;li&gt;&quot;stream your data in each epoch&quot;. By default, the entire training set will be stored on the GPU. This is a good idea, because the bus connecting the GPU and the RAM has extremely high latency. It takes a very long time to start sending data across the bus. However, most systems have much more space in the RAM than in the video memory on a GPU. Instead of storing all the training data in the GPU, you could store it in main memory, and then manually move over just the batch of data you want to use for a given update. After computing the update, you could free the memory assigned to the batch. I am not sure how to do this in Keras. In the past, I have done this by writing a custom CUDA kernel.&lt;/li&gt;&#xA;&lt;li&gt;&quot;Use less data&quot;. If you train on a random subset of the training data, you can keep your images at high quality, but your model will probably overfit. If you train on downsampled images, your model may not be able to discriminate between them well. However, both of these options are easy to do.&lt;/li&gt;&#xA;&lt;li&gt;&quot;Get more memory&quot;. If you buy a video card with more video RAM, you can train more complex models. This is why scientific-grade cards have much larger memories than gaming cards. The &lt;a href=&quot;https://www.nvidia.com/en-us/data-center/tesla-v100/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesla V100&lt;/a&gt; has 32GB of memory, and 16GB cards are common, whereas even the most &lt;a href=&quot;https://www.pcgamer.com/the-best-graphics-cards/&quot; rel=&quot;nofollow noreferrer&quot;&gt;advanced cards&lt;/a&gt; for gaming have only 11GB.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-27T22:09:02.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15656" PostTypeId="2" ParentId="15631" CreationDate="2019-09-28T00:57:45.797" Score="2" Body="&lt;p&gt;There are two factors that will change the ability of a deep neural network to fit a given dataset: either you need more data, or a deeper and wider network. Since the pattern is only 2-d, it can likely be approximated by some sort of simple periodic function. A DNN can approximate periodic functions pretty well, so the issue is probably that you don't have enough data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an apriori belief that the pattern is well approximated by K-nearest neighbors, then you could do the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Fit a K-NN model to the data.&lt;/li&gt;&#xA;&lt;li&gt;Generate &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; new points uniformly at random from the input space.&lt;/li&gt;&#xA;&lt;li&gt;Label the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; new points using the K-NN model.&lt;/li&gt;&#xA;&lt;li&gt;Fit your DNN to the original dataset, plus the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; new points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-09-28T00:57:45.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15658" PostTypeId="1" CreationDate="2019-09-28T18:25:39.067" Score="6" ViewCount="86" Body="&lt;p&gt;My understanding of the vanishing gradient problem in deep networks is that as backprop progresses through the layers the gradients become small, and thus training progresses slower. I'm having a hard time reconciling this understanding with images such as below where the losses for a deeper network are higher than for a shallower one. Should it not just take longer to complete each iteration, but still reach the same level if not higher of accuracy?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y9HV7.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y9HV7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30038" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-07-25T22:00:15.040" Title="Why do very deep non resnet architectures perform worse compared to shallower ones for the same iteration? Shouldn't they just train slower?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15660" PostTypeId="2" ParentId="12397" CreationDate="2019-09-29T01:01:37.133" Score="0" Body="&lt;p&gt;The discriminator's job is to tell between real images and generated images. It would be impossible to do that if it never actually sees generated images, just as if you wanted a network to differentiate between cats and dogs it wouldn't work if you only showed it pictures of dogs.&lt;br&gt;&#xA;If you were to train a generator on only real images, all labels it would see would be a 1. What you end up with is a network that learns how to produce 1 regardless of its inputs, which is very easy to learn without finding any underlying patterns in the data. Once you add in the generated images and 0 labels it is forced to learn something interesting.  &lt;/p&gt;&#xA;" OwnerUserId="30043" LastActivityDate="2019-09-29T01:01:37.133" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15661" PostTypeId="2" ParentId="6185" CreationDate="2019-09-29T06:39:24.253" Score="1" Body="&lt;p&gt;You claim that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;C++ is technically a more powerful language than python.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But that claim is wrong (or does not mean much). Remember that &lt;strong&gt;a programming language is a &lt;em&gt;specification&lt;/em&gt;&lt;/strong&gt; (often some document written in English). For example, &lt;a href=&quot;https://github.com/cplusplus/draft/blob/master/papers/n3337.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;n3337&lt;/a&gt; is a late draft of the C++ specification. I don't like Python, but it does seems as powerful than C++ (even if C++ &lt;em&gt;implementations&lt;/em&gt; are generally faster than Python ones): what a good Python programmer can code well in Python, another good C++ programmer can code well in C++ and vice versa.&lt;/p&gt;&#xA;&lt;p&gt;Theoretically, both C++ and Python are &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_completeness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing-complete&lt;/a&gt; (on purpose) programming languages.&lt;/p&gt;&#xA;&lt;p&gt;And Python is as expressive as C++ is. I cannot name a programming language feature that Python has but not C++ (except those related to &lt;a href=&quot;https://en.wikipedia.org/wiki/Reflection_(computer_programming)&quot; rel=&quot;nofollow noreferrer&quot;&gt;reflection&lt;/a&gt;; see also &lt;a href=&quot;https://ai.stackexchange.com/a/2237/3335&quot;&gt;this&lt;/a&gt; answer and be aware of &lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_loading&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;dlopen&lt;/code&gt;&lt;/a&gt; - see my &lt;a href=&quot;https://github.com/bstarynk/misc-basile/blob/master/manydl.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;manydl.c&lt;/a&gt; program -, of &lt;a href=&quot;https://llvm.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LLVM&lt;/a&gt;, of &lt;a href=&quot;https://gcc.gnu.org/onlinedocs/jit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;libgccjit&lt;/a&gt;, of &lt;a href=&quot;https://github.com/ianlancetaylor/libbacktrace&quot; rel=&quot;nofollow noreferrer&quot;&gt;libbacktrace&lt;/a&gt;, and consider some &lt;a href=&quot;https://en.wikipedia.org/wiki/Metaprogramming&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta-programming&lt;/a&gt; approach with them, à la &lt;a href=&quot;https://github.com/bstarynk/bismon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bismon&lt;/a&gt; or like &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;J.Pitrat's blog&lt;/a&gt; advocates it).&lt;/p&gt;&#xA;&lt;p&gt;Maybe you think of a programming language as the software implementing it. Then Python is as expressive as C++ is (and seems easier to learn, but that is an illusion; see &lt;a href=&quot;http://norvig.com/21-days.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://norvig.com/21-days.html&lt;/a&gt; for more about that illusion). Python and C++ have a quite similar &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantics_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantics&lt;/a&gt;, even if their syntax is very different. Their &lt;a href=&quot;https://en.wikipedia.org/wiki/Type_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;type system&lt;/a&gt; is very different.&lt;/p&gt;&#xA;&lt;p&gt;Observe that sadly, many recent &lt;strong&gt;major machine learning libraries&lt;/strong&gt; (such as &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; or &lt;a href=&quot;http://gudhi.gforge.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gudhi&lt;/a&gt;, both mostly coded in C++) are &lt;em&gt;in practice&lt;/em&gt; easier to use in Python than in C++. But you can use TensorFlow or Gudhi from C++ code since &lt;strong&gt;TensorFlow and Gudhi are mostly coded in C++&lt;/strong&gt; and both provide and document a C++ &lt;a href=&quot;https://en.wikipedia.org/wiki/Application_programming_interface&quot; rel=&quot;nofollow noreferrer&quot;&gt;API&lt;/a&gt; (not just a Python one).&lt;/p&gt;&#xA;&lt;p&gt;C++ enables &lt;a href=&quot;https://en.cppreference.com/w/cpp/thread&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-threaded programming&lt;/a&gt;, but the usual Python implementation has its &lt;a href=&quot;https://en.wikipedia.org/wiki/Global_interpreter_lock&quot; rel=&quot;nofollow noreferrer&quot;&gt;GIL&lt;/a&gt;, is &lt;a href=&quot;https://opensource.com/article/18/4/introduction-python-bytecode&quot; rel=&quot;nofollow noreferrer&quot;&gt;bytecoded&lt;/a&gt;, so is significantly slower than C++ (which is usually compiled by &lt;a href=&quot;https://en.wikipedia.org/wiki/Optimizing_compiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimizing compilers&lt;/a&gt; such as &lt;a href=&quot;https://gcc.gnu.orrg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GCC&lt;/a&gt; or &lt;a href=&quot;https://clang.llvm.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clang&lt;/a&gt;; however you could find C++ interpreters, e.g. &lt;a href=&quot;https://github.com/root-project/cling&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cling&lt;/a&gt;). Some experimental implementations of Python are &lt;a href=&quot;https://en.wikipedia.org/wiki/Just-in-time_compilation&quot; rel=&quot;nofollow noreferrer&quot;&gt;JIT-compiled&lt;/a&gt; and without GIL. But these are not mature: I recommend investing a million euros to increase their &lt;a href=&quot;https://en.wikipedia.org/wiki/Technology_readiness_level&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRL&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Observe also that C++ is much more difficult to learn than Python. Even with a dozen years of C++ programming experience, I cannot claim to really know most of C++.&lt;/p&gt;&#xA;&lt;p&gt;Sadly, most recent books teaching AI software engineering (e.g. &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1449369413&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one or &lt;a href=&quot;https://mitpress.mit.edu/books/introduction-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;that&lt;/a&gt; one) use Python (not C++) for their examples. I actually want more recent AI books using C++ !&lt;/p&gt;&#xA;&lt;p&gt;BTW, I program open source software (like &lt;a href=&quot;https://github.com/bstarynk/bismon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one, or the obsolete &lt;a href=&quot;http://starynkevitch.net/Basile/gcc-melt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GCC MELT&lt;/a&gt;) using AI techniques, but they don't use Python. My approach to AI applications is to start designing some &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain-specific_language&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSL&lt;/a&gt; in them.&lt;/p&gt;&#xA;&lt;p&gt;Some AI approaches involve metaprogramming, e.g. generating some (or most, or even all) the code of a system by itself. J.Pitrat (he passed away in October 2019) pioneered this approach. See &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;his blog&lt;/a&gt;, his CAIA system, read his &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1848211015&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings, the conscience of a couscious machine&lt;/em&gt;&lt;/a&gt; book (ISBN 978-1848211018) and the &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; project (whose ambition is to generate most -and hopefully all- of its C++ code).&lt;/p&gt;&#xA;&lt;p&gt;On operating systems such as Linux you could in practice generate C++ (or C) code at runtime and compile it (using &lt;a href=&quot;http://gcc.gnu.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GCC&lt;/a&gt;) into a &lt;a href=&quot;https://en.wikipedia.org/wiki/Plug-in_(computing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;plugin&lt;/a&gt;, then later &lt;a href=&quot;https://man7.org/linux/man-pages/man3/dlopen.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dlopen(3)&lt;/a&gt; that generated plugin, and retrieve function pointers by their name using &lt;a href=&quot;https://man7.org/linux/man-pages/man3/dlsym.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dlsym(3)&lt;/a&gt;. See the &lt;a href=&quot;https://github.com/bstarynk/misc-basile/blob/master/manydl.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;manydl.c&lt;/a&gt; example (on a powerful desktop in 2020, you would be able to generate and load half a million of plugins, if you run that example several days). With &lt;a href=&quot;https://man7.org/linux/man-pages/man3/dladdr.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dladdr(3)&lt;/a&gt; and Ian Taylor's &lt;a href=&quot;https://github.com/ianlancetaylor/libbacktrace&quot; rel=&quot;nofollow noreferrer&quot;&gt;libbacktrace&lt;/a&gt;, you can also inspect some of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Call_stack&quot; rel=&quot;nofollow noreferrer&quot;&gt;call stack&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;AFAIK major corporations such as Google use C++ internally for most of their AI-related code. Look also into &lt;a href=&quot;https://en.wikipedia.org/wiki/MILEPOST_GCC&quot; rel=&quot;nofollow noreferrer&quot;&gt;MILEPOST GCC&lt;/a&gt; or the H2020 &lt;a href=&quot;https://www.decoder-project.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Decoder project&lt;/a&gt; for an application of machine learning techniques to compilers. See also &lt;a href=&quot;https://www.hipeac.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HIPEAC&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Of course, you can code AI software in &lt;a href=&quot;https://www.haskell.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haskell&lt;/a&gt;, in &lt;a href=&quot;https://en.wikipedia.org/wiki/Common_Lisp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Common Lisp&lt;/a&gt; (e.g. with &lt;a href=&quot;https://sbcl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBCL&lt;/a&gt;), or in &lt;a href=&quot;https://ocaml.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ocaml&lt;/a&gt;. Many &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt; frameworks can be called from them. Number crunching libraries could use &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenCL&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCL&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-09-04T08:46:53.413" LastActivityDate="2020-09-04T08:46:53.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15662" PostTypeId="2" ParentId="15658" CreationDate="2019-09-29T07:45:39.453" Score="0" Body="&lt;p&gt;Those graphs do not disprove your 'vanishing gradient' theory. The deeper network may eventually do better than the shallower one, but it might take much longer to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incidentally, the ReLU activation function was designed to mitigate the vanishing gradient problem. &lt;/p&gt;&#xA;" OwnerUserId="12509" LastActivityDate="2019-09-29T07:45:39.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15663" PostTypeId="1" AcceptedAnswerId="15664" CreationDate="2019-09-29T15:36:34.613" Score="4" ViewCount="112" Body="&lt;p&gt;Intuitively, I understand that having an unbiased estimate of a policy is important because being biased just means that our estimate is distant from the truth value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't understand clearly why having lower variance is important. Is that because, in offline policy evaluation, we can have only 'one' estimate with a stream of data, and we don't know if it is because of variance or bias when our estimate is far from the truth value? Basically, variance acts like bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if that is the case, why having variance is preferable to having a bias?&lt;/p&gt;&#xA;" OwnerUserId="30051" LastEditorUserId="2444" LastEditDate="2019-09-29T21:41:07.587" LastActivityDate="2019-09-29T21:41:07.587" Title="Why is having low variance important in offline policy evaluation of reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15664" PostTypeId="2" ParentId="15663" CreationDate="2019-09-29T20:30:57.057" Score="1" Body="&lt;p&gt;Having low variance is important in general as it reduces the number of samples needed to obtain accurate estimates. This is the case for all statistical machine learning, not just reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you are estimating a mean or expected quantity by taking many samples, the variation in the error is proportional to &lt;span class=&quot;math-container&quot;&gt;$\frac{\sigma}{\sqrt{N}}$&lt;/span&gt; for a direct arithemtic mean of all samples, and behaves similarly for other averaging approaches (such as recency-weighted means using a learning rate). The bounds on accuracy can be made better by either increasing &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; i.e. taking more samples, or by decreasing the variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So anything you can do to reduce variance in your measurements has a direct consequence of reducing the number of samples required to achieve the same degree of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of off-policy reinforcement learning, there is added variance - compared to on-policy learning - due to different probabilities of taking an action in behaviour and target policies. This is due to the need to adjust reward signals using importance sampling - multiplying by the importance sampling ratio will make the reward signal vary more (in fact it can become unbounded). This is not really any more of a challenge than any other source of variance, but as it interferes with the goal of speedy learning, a lot of research effort has been put into methods that reduce the variance.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-09-29T20:30:57.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15665" PostTypeId="2" ParentId="15663" CreationDate="2019-09-29T21:27:08.230" Score="1" Body="&lt;p&gt;Bias is not necessarily bad, even though the term &lt;em&gt;bias&lt;/em&gt; usually has a negative connotation. In fact, in machine learning, &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;inductive bias&lt;/a&gt; is quite important and necessary. For example, if you want to learn a function &lt;span class=&quot;math-container&quot;&gt;$f(x) = y$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x \in \mathcal{X}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y \in \mathcal{Y}$&lt;/span&gt;, you often just have a &lt;em&gt;finite&lt;/em&gt; dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D} = \{ (x_i, y_i)\}_{i=1}^N$&lt;/span&gt;, which may not contain all possible &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; pairs associated with &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. In that case, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; may not contain enough information to learn &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, so you need to assume that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; behaves in a certain way or that the input and output spaces have certain characteristics. A typical way of dealing with finite datasets is to introduce noise during the learning process (which is a &lt;em&gt;regularization technique&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, bias can lead to sub-optimal solutions. For example, you could assume that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a lot more complex than the function &lt;span class=&quot;math-container&quot;&gt;$\hat{f}$&lt;/span&gt; that maps &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; (of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;), for &lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, N$&lt;/span&gt;. So, to solve this issue, you could introduce a lot of noise, while, in reality, &lt;span class=&quot;math-container&quot;&gt;$\hat{f}$&lt;/span&gt; may be extremely similar to &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, even though not exactly the same, so, in reality, you may not need all this noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is low variance desirable? Essentially, while you are learning something, it is easier to learn regular patterns as opposed to more irregular ones. For example, &lt;span class=&quot;math-container&quot;&gt;$1, 2, 1, 2, 1, 2$&lt;/span&gt; is a relatively regular sequence compared to &lt;span class=&quot;math-container&quot;&gt;$8, 2, 5, 6, 1, 7, 99$&lt;/span&gt;, which is thus harder to learn (or memorise) than the former.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-09-29T21:39:14.717" LastActivityDate="2019-09-29T21:39:14.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15666" PostTypeId="1" AcceptedAnswerId="15669" CreationDate="2019-09-30T02:15:29.137" Score="3" ViewCount="286" Body="&lt;p&gt;If a policy is fixed, it is said that a Markov Decision Process (MDP) becomes a Markov Reward Process (MRP).&lt;/p&gt;&#xA;&lt;p&gt;Why is this so? Aren't the transitions and rewards still parameterized by the action and current state? In other words, aren't the transition and reward matrices still cubes?&lt;/p&gt;&#xA;&lt;p&gt;From my current train of thought, the only thing that is different is that the policy is not changing (the agent is not learning the policy). Everything else is the same.&lt;/p&gt;&#xA;&lt;p&gt;How is it that it switches to an MRP, which is not affected by actions?&lt;/p&gt;&#xA;&lt;p&gt;I am reading &amp;quot;Deep Reinforcement Learning Hands-On&amp;quot; by Maxim Lapan, which states this. I have also found this statement in online articles, but I cannot seem to wrap my head around it.&lt;/p&gt;&#xA;" OwnerUserId="30059" LastEditorUserId="2444" LastEditDate="2021-01-20T22:36:37.413" LastActivityDate="2021-01-20T22:36:37.413" Title="Why does having a fixed policy change a Markov Decision Process to a Markov Reward Process?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;policies&gt;&lt;markov-reward-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15667" PostTypeId="2" ParentId="15621" CreationDate="2019-09-30T04:24:42.000" Score="2" Body="&lt;p&gt;I had a very similar issue as you did with the dimensions. Here's the rundown:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every node you see inside the LSTM cell has the exact same output dimensions, &lt;em&gt;including the cell state&lt;/em&gt;. Otherwise, you'll see with the forget gate and output gate, how could you possible do an element wise multiplication with the cell state? They have to have the same dimensions in order for that to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using an example where &lt;code&gt;n_hiddenunits = 256&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Output of forget gate: 256&#xA;Input gate: 256&#xA;Activation gate: 256&#xA;Output gate: 256&#xA;Cell state: 256&#xA;Hidden state: 256&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now this can obviously be problematic if you want the LSTM to output, say, a one hot vector of size 5. So to do this, a softmax layer is slapped onto the end of the hidden state, to convert it to the correct dimension. So just a standard FFNN with normal weights (no bias', because softmax). Now, also imagining that we input a one hot vector of size 5:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input size: 5&#xA;total input size to all gates: 256+5 = 261 (the hidden state and input are appended)&#xA;Output of forget gate: 256&#xA;Input gate: 256&#xA;Activation gate: 256&#xA;Output gate: 256&#xA;Cell state: 256&#xA;Hidden state: 256&#xA;Final output size: 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is the final dimensions of the cell.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-09-30T04:24:42.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15668" PostTypeId="1" CreationDate="2019-09-30T04:49:08.410" Score="1" ViewCount="49" Body="&lt;p&gt;I wanted to know about Intelligent reflecting surface (IRS) technology. &#xA;what is the application of IRS in wireless communication? &#xA;what are the competitive advantages over existing technologies?&lt;/p&gt;&#xA;" OwnerUserId="28346" LastActivityDate="2019-09-30T04:49:08.410" Title="Intelligent reflecting surface" Tags="&lt;machine-learning&gt;&lt;swarm-intelligence&gt;&lt;intelligence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15669" PostTypeId="2" ParentId="15666" CreationDate="2019-09-30T06:56:53.027" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If a policy is fixed, it is said that an MDP becomes an MRP.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would change the phrasing slightly here, to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If a policy is fixed, an MDP can be accurately modeled as an MRP.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is this so? Aren't the transitions and rewards still parameterized by the action and current state? In other words, aren't the transition and reward matrices still cubes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The transition and reward matrices remain the same in the MDP, but it is &lt;em&gt;possible&lt;/em&gt; to flatten them into an equivalent MRP, because in terms of observations of next state and reward, the action that is taken is just part of the transition rules - if the policy is fixed, then so are all the probabilities for next state and reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More concretely, if you have an MDP with &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}|$&lt;/span&gt; transition matrices &lt;span class=&quot;math-container&quot;&gt;$P_{ss'}^a$&lt;/span&gt; and a fixed policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;, then you can create a &lt;em&gt;combined&lt;/em&gt; transition matrix with a sum:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P_{ss'} = \sum_{a \in \mathcal{A}} \pi(a|s) P_{ss'}^a$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you can similarly reduce the reward function. Once you have done so you have data that describe an MRP.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How is it that it switches to an MRP which is not affected by actions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the MDP represents a real system where actions are still being taken by an agent, then of course those are still present within the system, and still affect it. The difference is that if you know the agent's policy, then the action choice is predictable, and the MRP representation covers the full definition of probabilities of observed state transitions and rewards.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-09-30T08:42:12.170" LastActivityDate="2019-09-30T08:42:12.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15670" PostTypeId="2" ParentId="1462" CreationDate="2019-09-30T11:47:21.917" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;Simply we say AI is software and robot is its body. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is because the algorithms we commonly think of AI come in the form of software, where when we talk about robots, we're talking about physical automation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an automobile manufacturing process where automation is used, the &lt;em&gt;software&lt;/em&gt; makes the decisions on what physical action the &lt;em&gt;robot&lt;/em&gt; arm should take at any given time. &lt;/p&gt;&#xA;" OwnerUserId="30064" LastEditorUserId="1671" LastEditDate="2019-09-30T23:41:07.500" LastActivityDate="2019-09-30T23:41:07.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15675" PostTypeId="2" ParentId="9813" CreationDate="2019-10-01T00:18:06.740" Score="0" Body="&lt;p&gt;My sense is that this would require a statistical approach with a large dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm would need to &quot;translate&quot; the slang into formal terms (discrete words or phrases expressing a single concept.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick would be vetting the algorithms decisions, which would require a sufficient sample of humans to evaluate the given translation for each novel instance of slang. (This would likely require some form of crowdsourcing, similar to Captcha.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would determine whether 4⇔ 5⇔ 6 where 4x, 5x, 6x of the symbol are equivalent, and whether spacing between the emoji's is meaningful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most likely these would be fuzzy associations in that the same slang can be interpreted differently by different people, and the meaning can vary when used in different contexts:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt; could mean &quot;I'm laughing super-hard because what you say is so absurdly incorrect.&quot; [Adversarial]&lt;/p&gt;&#xA;&#xA;&lt;p&gt; could mean &quot;I'm laughing super-hard because the joke is extremely funny.&quot; [Cooperative]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Informally, my experience of 5 has always been in adversarial, but that could be a function of the contexts in which I've encountered it, which reinforces the need for a large samples.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It occurs to me that you could reduce the sample size by using a friendly chatbot that parses social media posts for any symbolic information that is non-standard, then queries the posters asking for clarification.  (This way, you'd get the intent of the slang from the person using it, as opposed interpretations of those viewing it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For informal text (as opposed to emojis) the algorithm would want to be able to distinguish between intentional or unintentional mis-spellings.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-01T00:53:47.247" LastActivityDate="2019-10-01T00:53:47.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15676" PostTypeId="1" AcceptedAnswerId="15689" CreationDate="2019-10-01T02:59:58.343" Score="1" ViewCount="130" Body="&lt;p&gt;People say &lt;strong&gt;embedding&lt;/strong&gt; is necessary in NLP because if using just the word indices, the &lt;strong&gt;efficiency is not high&lt;/strong&gt; as &lt;strong&gt;similar words are supposed to be related to each other&lt;/strong&gt;. However, I still don't truly get it why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;subword-based embedding&lt;/strong&gt; (aka syllable-based embedding) is understandable, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;biology   --&amp;gt; bio-lo-gy&#xA;biologist --&amp;gt; bio-lo-gist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the 2 words above, when turning them into syllable-based embeddings, it's good because the 2 words will be related to each other due to the sharing syllables: &lt;code&gt;bio&lt;/code&gt;, and &lt;code&gt;lo&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's hard to understand the &lt;code&gt;autoencoder&lt;/code&gt;, it turns an index value into vector, then feed these vectors to DNN. Autoencoder can turn vectors back to words too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How does autoencoder make words related to each other?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-01T09:16:36.460" LastActivityDate="2019-10-02T09:09:59.297" Title="Why is embedding important in NLP, and how does autoencoder work?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;word-embedding&gt;&lt;hidden-layers&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15677" PostTypeId="1" CreationDate="2019-10-01T09:15:12.140" Score="3" ViewCount="39" Body="&lt;p&gt;I wonder if anyone knows a database (preferably free) of images of different classes of people (age, gender) but with entire bodies shown on the pictures taken from different angles (mostly front and side views, back won't hurt too)? &#xA;The higher resolution the better.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-10-01T09:15:12.140" Title="Is there a database of images of entire people bodies?" Tags="&lt;image-recognition&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15678" PostTypeId="2" ParentId="15676" CreationDate="2019-10-01T09:15:33.270" Score="1" Body="&lt;p&gt;The &lt;strong&gt;subword-based embedding&lt;/strong&gt; is rather visual and easily understandable. However, the &lt;strong&gt;autoencoder embedding&lt;/strong&gt; is what machines understand the componential meaning of words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) An &lt;strong&gt;autoencoder embedding layer&lt;/strong&gt; can be trained together with other layers to fit with the relation of data in dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Or the &lt;strong&gt;embedding layer can be kept unchanged&lt;/strong&gt; as used as a function, &lt;strong&gt;required&lt;/strong&gt; that the embedding layer must have been trained on a similar task like said in 1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And as stated in the question, embedding is important in NLP as &lt;strong&gt;word indices don't show the full meanings of words&lt;/strong&gt;, they must be &lt;strong&gt;separated&lt;/strong&gt; into embedding values &lt;strong&gt;for better efficiency&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TensorFlow random uniform trainable embedding: &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TensorFlow utility class for creating subword-based encoder: &lt;a href=&quot;https://www.tensorflow.org/datasets/api_docs/python/tfds/features/text/SubwordTextEncoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/datasets/api_docs/python/tfds/features/text/SubwordTextEncoder&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-02T05:23:29.520" LastActivityDate="2019-10-02T05:23:29.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15681" PostTypeId="1" AcceptedAnswerId="15682" CreationDate="2019-10-01T17:30:55.690" Score="1" ViewCount="44" Body="&lt;p&gt;The author explains in &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node16.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2.2 Action-Value Methods&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To roughly assess the relative effectiveness of the greedy and &lt;span class=&quot;math-container&quot;&gt;$\varepsilon $&lt;/span&gt;-greedy methods, we compared them numerically on a suite of test problems. This is a set of 2000 randomly generated n-armed bandit tasks with n = 10. For each action, a, the rewards were selected from a normal (Gaussian) probability distribution with mean Q*(a) and variance 1. The 2000 n-armed bandit tasks were generated by reselecting the Q*(a) 2000 times, each according to a normal distribution with mean 0 and variance 1. Averaging over tasks, we can plot the performance and behavior of various methods as they improve with experience over 1000 plays, as in Figure 2.1. We call this suite of test tasks the 10-armed testbed. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jYKEu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jYKEu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But doing my best, the replication yields something nearer to :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/myPpm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/myPpm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I am misunderstanding how the author took the averages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import exp&#xA;&#xA;import numpy&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;def act(action, Qstar):&#xA;    return numpy.random.normal(Qstar[action], 1)&#xA;&#xA;&#xA;def run(epsilon):&#xA;    history = [0 for i in range(1000)]&#xA;&#xA;    for task in range(1, 2000):&#xA;        Qstar = [numpy.random.normal(0, 1) for i in range(10)]&#xA;        Q = [0 for i in range(10)]&#xA;        for t in range(1, 1001):&#xA;            if numpy.random.randint(0, 100) &amp;lt; epsilon:&#xA;                action = numpy.random.randint(0, len(Q))&#xA;            elif t == 0:&#xA;                action = 0&#xA;            else:&#xA;                averages = [q/t for q in Q]&#xA;                action = averages.index(max(averages))&#xA;&#xA;            reward = act(action, Qstar)&#xA;            Q[action] += reward&#xA;&#xA;            history[t-1] += reward&#xA;&#xA;    return [elem/2000 for elem in history]&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    plt.plot(run(10), 'b', label=&quot;ɛ=0.1&quot;)&#xA;    plt.plot(run(1), 'r', label=&quot;ɛ=0.01&quot;)&#xA;    plt.plot(run(0), 'g', label=&quot;ɛ=0&quot;)&#xA;    plt.xlabel('Plays')&#xA;    plt.ylabel('Reward')&#xA;    plt.legend()&#xA;    plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14892" LastActivityDate="2019-10-01T19:54:57.083" Title="Unable to replicate Figure 2.1 from &quot;Reinforcement Learning: An Introduction&quot;" Tags="&lt;reinforcement-learning&gt;&lt;greedy-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15682" PostTypeId="2" ParentId="15681" CreationDate="2019-10-01T19:48:19.320" Score="1" Body="&lt;p&gt;You are calculating the average reward for each action (i.e. bandit arm) incorrectly. You cannot calculate this simply with a list comprehension, and you need to keep a second list storing the number of times each action was taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct calculation is to divide the total reward obtained from each action by the number of times &lt;strong&gt;&lt;em&gt;that action&lt;/em&gt;&lt;/strong&gt; was taken. You are dividing all actions by the &lt;em&gt;total&lt;/em&gt; number of actions taken. Adding:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;counts = [0 for i in range(10)]&#xA;...&#xA;else:&#xA;    averages = []&#xA;    for i in range(0, 10):&#xA;        averages.append(Q[i]/counts[i] if counts[i] &amp;gt; 0 else 0)&#xA;    ...&#xA;...&#xA;counts[action] += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in working code, and I can generate this graph using 200 samples per method, which looks like a noisier version of the one in the figure you referenced:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DBnS1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DBnS1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-01T19:54:57.083" LastActivityDate="2019-10-01T19:54:57.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15683" PostTypeId="1" CreationDate="2019-10-01T20:59:28.153" Score="1" ViewCount="113" Body="&lt;p&gt;I'm interested in learning about Neural Networks and implementing them. I'm particularly interested in GANs and LSTM networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand perceptrons and basic Neural Network configuration (sigmoid activation, weights, hidden layers etc). But what topics do I need to learn in order, to get to the point where I can implement GAN or LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I intend to make an implementation of each in C++ to prove to myself that I understand. I haven't got a particularly good math background, but I understand most math-things when they are explained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I understand backpropagation, but I don't really understand it. I understand how reinforced learning is used with backpropagation, but not fully how you can have things like training without datasets (like tD-backgammon). I don't quite understand CNNs, especially why you might make a particular architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for each &quot;topic&quot; there was a book or website or something for each it would be great.&lt;/p&gt;&#xA;" OwnerUserId="2819" LastActivityDate="2019-11-06T12:06:55.977" Title="What order should I learn about Neural Networks?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="8" FavoriteCount="1" ClosedDate="2020-05-16T23:10:07.467" ContentLicense="CC BY-SA 4.0" />
  <row Id="15684" PostTypeId="2" ParentId="7853" CreationDate="2019-10-02T00:01:32.687" Score="0" Body="&lt;p&gt;Your description sounds like something similar to &lt;a href=&quot;http://www.image-net.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imagenet dataset&lt;/a&gt;. According to &lt;a href=&quot;https://paperswithcode.com/sota/image-classification-on-imagenet&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt;, the state-of-the-art top-1 accuracy just as high as 86%, not much higher than yours. There are plenty of methods to improve accuracy. I would suggest you to read the paper or github listed in &lt;a href=&quot;https://paperswithcode.com/sota/image-classification-on-imagenet&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOTA&lt;/a&gt; to find ideas that best fits your situation. &lt;/p&gt;&#xA;" OwnerUserId="18276" LastActivityDate="2019-10-02T00:01:32.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15685" PostTypeId="1" AcceptedAnswerId="22747" CreationDate="2019-10-02T01:35:17.870" Score="2" ViewCount="53" Body="&lt;p&gt;&lt;strong&gt;What I did:&lt;/strong&gt;&lt;br&gt;&#xA;Created a population of 2D legged robots in a simulated environment. Found the best motor rotation values to make the robots move rightward, using an objective function with Differential Evolution (could use PSO or GA too), that returned the distance moved rightward. Gradient descent used for improving fitness.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I want to do:&lt;/strong&gt;&lt;br&gt;&#xA;Add more objectives. To find the best motor rotation, with the least motion possible, with the least jittery motion, without toppling the body upside down and making the least collision impact on the floor.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I found:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Spent almost two weeks searching for solutions, reading research&#xA;papers, going through tutorials on Pareto optimality, installing&#xA;libraries and trying the example programs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using &lt;a href=&quot;https://en.wikipedia.org/wiki/Pairing_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;pairing functions&lt;/a&gt; to create a cost function wasn't good&#xA;enough.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are many &lt;a href=&quot;https://www.google.com/search?client=firefox-b-d&amp;amp;q=multi%20objective%20evolutionary%20algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-objective PSO, DE, GA etc.&lt;/a&gt;, but they seem&#xA;to be built for solving some other kind of problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where I need help:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Existing multi objective algorithms seem to use some pre-existing&#xA;minimization and maximization functions (Fonseca, Kursawe, OneMax,&#xA;DTLZ1, ZDT1, etc.) and it's confusing to understand how I can use my&#xA;own maximization and minimization functions with the libraries.&#xA;(minimize(motorRotation), maximize(distance),&#xA;minimize(collisionImpact), constant(bodyAngle)).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I know which is the best Pareto front to choose in a&#xA;multi-dimensional space? There seem to be ways of choosing the&#xA;top-right Pareto front or the top-left or the bottom-right or&#xA;bottom-left. In multi-dimensional space, it'd be even more varied.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Libraries like Platypus, PyGMO, Pymoo etc. just define the problem using&#xA;&lt;code&gt;problem = DTLZ2()&lt;/code&gt;, instantiate an algorithm &lt;code&gt;algorithm =&#xA;NSGAII(problem)&lt;/code&gt; and run it &lt;code&gt;algorithm.run(10000)&lt;/code&gt;, where I assume&#xA;10000 is the number of generations. But since I'm using a legged&#xA;robot, I can't simply use &lt;code&gt;run(10000)&lt;/code&gt;. I need to assign motor values&#xA;to the robots, wait for the simulator to make the robots in the&#xA;population move and then calculate the objective function cost. How&#xA;can I achieve this?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once the pareto optimal values are found, how is it used to create a&#xA;cost value that helps me determine the fittest robot in the&#xA;population?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9268" LastEditorUserId="9268" LastEditDate="2019-10-02T02:34:14.877" LastActivityDate="2020-07-29T16:29:11.730" Title="How to calculate multiobjective optimization cost for ordinary problems?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15687" PostTypeId="2" ParentId="7853" CreationDate="2019-10-02T05:01:31.970" Score="0" Body="&lt;p&gt;I can't comment but here are a few suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Play with lr and lr finder&lt;/li&gt;&#xA;&lt;li&gt;Use a pretrained model&lt;/li&gt;&#xA;&lt;li&gt;Use architecture search or use something like efficientnet b6&lt;/li&gt;&#xA;&lt;li&gt;Use swish over relu&lt;/li&gt;&#xA;&lt;li&gt;Try different optimizers&lt;/li&gt;&#xA;&lt;li&gt;Try Bayesian Optimization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28538" LastActivityDate="2019-10-02T05:01:31.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15688" PostTypeId="1" CreationDate="2019-10-02T05:28:45.947" Score="4" ViewCount="87" Body="&lt;p&gt;I recently read a paper on community detection in networks. In the paper &lt;a href=&quot;https://arxiv.org/abs/1906.04560&quot; rel=&quot;nofollow noreferrer&quot;&gt;EdMot: An Edge Enhancement Approach for Motif-aware Community Detection&lt;/a&gt;, the authors consider the &quot;lower-order structure&quot; of the network at the level of individual nodes and edges. And they mentioning about some &quot;higher-order structure&quot; method. The point is, what is the exact meaning (definition) of lower- and higher-order structure in a network? &lt;/p&gt;&#xA;" OwnerUserId="26886" LastEditorUserId="2444" LastEditDate="2019-10-02T17:16:48.163" LastActivityDate="2020-05-25T20:04:45.437" Title="What are the exact meaning of &quot;lower-order structure&quot; and &quot;higher-order structure&quot; in this paper?" Tags="&lt;terminology&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15689" PostTypeId="2" ParentId="15676" CreationDate="2019-10-02T09:09:59.297" Score="1" Body="&lt;p&gt;The information you are probably missing is that word embeddings are learned on the basis of context. For example, you might try to predict a vector for a word from the wordvectors of the other words in the same sentence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way word vectors of words that occur in similar contexts will turn out to be similar. You can think of it as word vectors not encoding the word themselves but the contexts in which they are used. Of course ultimately that is the same. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2019-10-02T09:09:59.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15693" PostTypeId="1" AcceptedAnswerId="15698" CreationDate="2019-10-02T14:32:42.570" Score="6" ViewCount="176" Body="&lt;p&gt;In a single agent environment, the agent takes an action, then observes the next state and reward:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ep in num_episodes:&#xA;    action = dqn.select_action(state)&#xA;    next_state, reward = env.step(action)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Implicitly, the for moving the simulation (env) forward is embedded inside the env.step() function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in the multiagent scenario, agent 1 (&lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;) has to make a decision at time &lt;span class=&quot;math-container&quot;&gt;$t_{1a}$&lt;/span&gt;, which will finish at time &lt;span class=&quot;math-container&quot;&gt;$t_{2a}$&lt;/span&gt;, and agent 2 (&lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;) makes a decision at time &lt;span class=&quot;math-container&quot;&gt;$t_{1b} &amp;lt; t_{1a}$&lt;/span&gt;  which is finished at &lt;span class=&quot;math-container&quot;&gt;$t_{2b} &amp;gt; t_{2a}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If both of their actions would start and finish at the same time&lt;/strong&gt;, then it could easily be implemented as: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ep in num_episodes:&#xA;    action1, action2 = dqn.select_action([state1, state2])&#xA;    next_state_1, reward_1, next_state_2, reward_2 = env.step([action1, action2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because the env can execute both in parallel, wait till they are done, and then return the next states and rewards. But in the scenario that I described previously, it is not clear how to implement this (at least to me). Here, we need to explicitly track time, a check at any timepoint to see if an agent needs to make a decision, Just to be concrete:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ep in num_episodes:&#xA;    for t in total_time:&#xA;       action1 = dqn.select_action(state1)&#xA;       env.step(action1) # this step might take 5t to complete. &#xA;       as such, the step() function won't return the reward till 5 t later. &#xA;        #In the mean time, agent 2 comes and has to make a decision. its reward and next step won't be observed till 10 t later. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To summarize, how would one implement a multiagent environment with asynchronous action/rewards per agents?&lt;/p&gt;&#xA;" OwnerUserId="22943" LastEditorUserId="2444" LastEditDate="2019-10-03T02:59:59.593" LastActivityDate="2019-10-03T02:59:59.593" Title="How would one implement a multi-agent environment with asynchronous action and rewards per agent?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15694" PostTypeId="2" ParentId="12034" CreationDate="2019-10-02T14:44:22.920" Score="5" Body="&lt;p&gt;No, AiAngel is not a bot.&#xA;It's Rouge's software that changes his voice along with facial recognition software which tracks his movement and copies it to the avatar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, he has created a very entertaining channel and line of work for himself. By looking at the videos, you can see that he is a true genius at work. Single handedly puts on all hats for that project, from hardware infrastructure development, to software design to graphic art to acting and video editing. Truely one of the great minds of this era. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find Ai Angel (Angelica) on Rogue Shadow's &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Youtube Channel: &lt;a href=&quot;https://www.youtube.com/user/TheRogueShadow1/videos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/user/TheRogueShadow1/videos&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Twitch &lt;a href=&quot;https://www.twitch.tv/aiangellive&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.twitch.tv/aiangellive&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Discord &lt;a href=&quot;https://discordapp.com/invite/DFhaYgj&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://discordapp.com/invite/DFhaYgj&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Twitter &lt;a href=&quot;https://twitter.com/AiAngel13&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/AiAngel13&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Patreon: &lt;a href=&quot;https://www.patreon.com/AiAngel&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.patreon.com/AiAngel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;..along with several other platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope than answers your question with certainty (instead of philosophically)&lt;/p&gt;&#xA;" OwnerUserId="30115" LastEditorUserId="30115" LastEditDate="2020-04-16T19:32:39.150" LastActivityDate="2020-04-16T19:32:39.150" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15695" PostTypeId="1" CreationDate="2019-10-02T17:30:43.540" Score="2" ViewCount="44" Body="&lt;p&gt;I'm looking for examples of AI systems or agents that best represent these five characteristics (one example for each characteristics):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reactivity &lt;/li&gt;&#xA;&lt;li&gt;Proactivity&lt;/li&gt;&#xA;&lt;li&gt;Adaptability&lt;/li&gt;&#xA;&lt;li&gt;Sociability &lt;/li&gt;&#xA;&lt;li&gt;Autonomy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It would be better if its machine learning-based application.&lt;/p&gt;&#xA;" OwnerUserId="30118" LastEditorUserId="2444" LastEditDate="2019-10-03T01:50:30.303" LastActivityDate="2019-10-03T01:50:30.303" Title="What are the examples of agents that is represent these characteristics?" Tags="&lt;machine-learning&gt;&lt;intelligent-agent&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15698" PostTypeId="2" ParentId="15693" CreationDate="2019-10-02T19:26:48.630" Score="2" Body="&lt;p&gt;The cleanest solution from a theoretical point of view is to switch over to a hierarchical framework, some framework that supports temporal abstraction. My favourite one is the &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370299000521&quot; rel=&quot;nofollow noreferrer&quot;&gt;options framework as formalised by Sutton, Precup and Singh&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that the things that you consider &quot;actions&quot; for your agents become &quot;options&quot;, which are &quot;large actions&quot; that may take more than a single primitive time step. When an agent selects an option, it will go on &quot;auto-pilot&quot; and keep selecting primitive actions at the more primitive, fine-grained timescale as dictated by the last selected option, until that option has actually finished executing. In your case, you could:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;implement the first &quot;primitive action&quot; of an option to immediately apply all effects to the state, and append a sequence of &quot;no-op&quot; actions afterwards to make sure the option actually has a longer duration than a single primitive timestep, OR&lt;/li&gt;&#xA;&lt;li&gt;implement the very last primitive action of an option to actually apply changes to the state, and prepend a sequence of &quot;no-op&quot; actions in front of it to make the option take more time, OR&lt;/li&gt;&#xA;&lt;li&gt;something in between (i.e. actually make partial changes to the state visible during the execution of the option).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since &lt;strong&gt;all&lt;/strong&gt; legal choices for agents in your scenario appear to be options, i.e. you do not allow agents to select primitive actions at the more fine-grained timescale, you would only have to implement &quot;inter-option&quot; learning in your RL algorithms; there would be no need for &quot;intra-option&quot; learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, if you only have a small number of agents and have options that take relatively large amounts of time, you don't have to actually loop through all primitive time-steps. You could, for example, compute the primitive timestamps at which &quot;events&quot; should be executed in advance, and insert these events to be processed into an event-handling queue based on these timestamps. Then you can always just skip through to the next timestamp at which an event needs handling. With &quot;events&quot; I basically mean all timesteps at which something should happen, e.g. timesteps where an option ends and a new option should be selected by one or more agents. Inter-option Reinforcement Learning techniques are basically oblivious to the existence of a more fine-grained timescale, and they only need to operate at precisely these decision points where one option ends and another begins.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-10-02T19:26:48.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15699" PostTypeId="1" AcceptedAnswerId="15700" CreationDate="2019-10-02T19:42:11.757" Score="2" ViewCount="88" Body="&lt;p&gt;By definition, every state in RL has Markov property, which means that the future state depends only on the current state, not the past states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I saw that in some case we can define a state to be the history of observations and actions taken so far, such as &lt;span class=&quot;math-container&quot;&gt;$s_t = h_t = o_1a_1\dots o_{t-1}a_{t-1}o_t$&lt;/span&gt;. I think maze solving can be of that case since the current state, or the current place in a maze, clearly depends on which places the agent has been and which ways the agent has taken so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it seems that the future states naturally depend on the past states and the past actions as well. What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="30051" LastEditorUserId="30051" LastEditDate="2019-10-02T20:04:13.423" LastActivityDate="2019-10-03T04:11:12.537" Title="Markov property in maze solving problem in reinforcement learning" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15700" PostTypeId="2" ParentId="15699" CreationDate="2019-10-02T20:32:14.620" Score="3" Body="&lt;p&gt;Hi Hunnam and welcome to our community!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By definition, every state in RL has Markov property, which means that the future state depends only on the current state, not the past states. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No this is not exactly correct.&lt;br&gt;&#xA;We can use RL to solve problems with the Markov Property exactly because the current state is a sufficient statistic of the future. In other words, the state encodes the distribution of future states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the state isn't necessarily the observations. As you point out in the next paragraph:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I saw that in some case we can define a state to be the history of observations and actions taken so far, such as =ℎ=11…−1−1 . &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;At times we can use the history to represent the state. The history can be a series of observations.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think maze solving can be of that case since the current state, or the current place in a maze, clearly depends on which places the agent has been and which ways the agent has taken so far. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This isn't correct in the general case. Given a maze which you know how to solve, regardless of where you start, you know how to reach the exit. This is the markov property. Given the current position, you have enough information to make a certain and optimal decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps an example of a situation where the history is necessary will help illustrate the differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you are playing Pong. If you take a single frame, it doesn't contain enough information to know the direction of the ball. Therefore the observations alone are insufficient. What if you remember the previous frame? Then combining the two observations gives you all the information you need in order to make an optimal move.&lt;/p&gt;&#xA;" OwnerUserId="28538" LastEditorUserId="28538" LastEditDate="2019-10-03T04:11:12.537" LastActivityDate="2019-10-03T04:11:12.537" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15701" PostTypeId="1" AcceptedAnswerId="15702" CreationDate="2019-10-03T00:55:04.350" Score="6" ViewCount="145" Body="&lt;p&gt;I am working on a project for my artificial intelligence class. I was wondering if I have 2 admissible heuristics, A and B, is it possible that A does not dominate B and B does not dominate A?  I am wondering this because I had to prove if each heuristic is admissible and I did that, and then for each admissible heuristic, we have to prove if each one dominates the other or not. I think I have a case that neither dominates the other and I was wondering if maybe I got the admissibility wrong because of that.&lt;/p&gt;&#xA;" OwnerUserId="30124" LastEditorUserId="2444" LastEditDate="2019-11-10T16:44:38.853" LastActivityDate="2019-11-11T15:44:42.173" Title="Can two admissable heuristics not dominate each other?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;heuristics&gt;&lt;admissible-heuristic&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15702" PostTypeId="2" ParentId="15701" CreationDate="2019-10-03T01:40:04.967" Score="4" Body="&lt;p&gt;This is possible. Admissibility only asserts that the heuristic will never overestimate the true cost. With that being said, it is possible for one heuristic in some cases to do better than another and vice-versa. Think of it as a game of rock paper scissors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, you may find that sometimes &lt;span class=&quot;math-container&quot;&gt;$h_1 &amp;lt; h_2$&lt;/span&gt; and in other times &lt;span class=&quot;math-container&quot;&gt;$h_2 &amp;lt; h_1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; are admissible heuristics. Thus, by definition, neither strictly dominates the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, there is a way to &quot;combine&quot; the two &lt;em&gt;admissible&lt;/em&gt; heuristics to get the best of both using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h_3 = \max(h_1, h_2)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="2444" LastEditDate="2019-11-11T15:44:42.173" LastActivityDate="2019-11-11T15:44:42.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15703" PostTypeId="1" CreationDate="2019-10-03T10:30:41.750" Score="0" ViewCount="124" Body="&lt;p&gt;I am a beginner in Python. I want to calculate the covariance matrix of a mini-batch in a given hidden layer.&lt;/p&gt;&#xA;" OwnerUserId="22823" LastEditorUserId="2444" LastEditDate="2019-10-04T13:05:54.690" LastActivityDate="2019-10-04T13:05:54.690" Title="How to calculate covariance matrix of the mini-batch in the k-th layer using Python?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15704" PostTypeId="1" CreationDate="2019-10-03T13:38:44.527" Score="1" ViewCount="41" Body="&lt;p&gt;I currently use an object detector to detect an object and specific parts of it (a crop and its stem). Such detector is not the best choice for detecting parts that could be represented by a point (typically a stem) so I'm planning on moving to a keypoint detector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading the literature it appears that there are many solutions. I'm particularly interested in using a hourglass network to predict a set of heatmaps abstracting the keypoints positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that many of the existing frameworks are dedicated to human pose estimation (for instance OpenPose) but I don't need all this complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By best choice for now is &lt;a href=&quot;https://github.com/wbenbihi/hourglasstensorlfow&quot; rel=&quot;nofollow noreferrer&quot;&gt;this framework&lt;/a&gt; that is a Tensorflow implementation of Hourglass Networks but it is still too specific to human pose estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do you have any suggestion of frameworks that best suit my application? i.e. simple keypoint detection.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="19859" LastActivityDate="2019-10-03T13:38:44.527" Title="Looking or the simplest framework to train keypoint detector" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15706" PostTypeId="1" CreationDate="2019-10-03T15:00:11.467" Score="1" ViewCount="33" Body="&lt;p&gt;I am learning how to use tensorflow without keras, just to make sure I understand tensorflow directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created a spiral-looking datasets with 100 points of each class (200 total), and I created a neural network to classify it. For the life of me I can't figure out how to get a good accuracy. Do you mind looking at my code to see what I did wrong? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I've gleaned from various forums, it seems like if I do 4 hidden layers and 14 neurons per layer, I should be able to perfectly separate this dataset. I tried with with learning rate of 0.01, and 20k epochs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried different combinations of activation functions (tanh, sigmoid, relu, even alternating between them), but the best that I've gotten is around 60 percent, whereas people with less layers and less neurons have gotten close to 90 percent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did NOT do is to create additional features (for example, r and theta), and this was intentional. I'm just curious to see if I can do this by looking at x and y alone. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is pasted below, and it includes the code to create the data (and it plots the data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np;&#xA;import matplotlib.pyplot as plt;&#xA;import tensorflow as tf;&#xA;import random;&#xA;&#xA;# Part A&#xA;# Gather data and plot&#xA;def chooseTrainingBatch(X, Y, n, k):&#xA;    indices = range(0,n)&#xA;    chosenIndices = random.choices(indices,k=k)&#xA;    batchX = X[chosenIndices, :]&#xA;    batchY = Y[chosenIndices, :]&#xA;    return (batchX, batchY)&#xA;&#xA;&#xA;def doLinearClassification(n, learning_rate=1, epochs=20, num_hidden_layer_1=100, num_of_layers=4, batch_size = 20):&#xA;    d = 1;&#xA;    plt.figure();&#xA;    X = np.zeros([2*n, 2]);&#xA;    Y = np.zeros([2*n, 2]);&#xA;    for t in np.arange(1,n+1,d):&#xA;        r1 = 50 + 0.2*t;&#xA;        r2 = 30 + 0.4*t;&#xA;        phi1 = -0.06*t + 3;&#xA;        phi2 = -0.08*t + 2;&#xA;&#xA;        x1 = r1 * np.cos(phi1);&#xA;        y1 = r1 * np.sin(phi1);&#xA;        x2 = r2 * np.cos(phi2);&#xA;        y2 = r2 * np.sin(phi2);&#xA;&#xA;        plt.scatter(x1, y1, c='b');&#xA;        plt.scatter(x2, y2, c='r');&#xA;&#xA;        X[t-1,0] = x1;&#xA;        X[t-1,1] = y1;&#xA;        Y[t-1,0] = 1;&#xA;&#xA;        X[n+t-1,0] = x2;&#xA;        X[n+t-1,1] = y2;&#xA;        Y[n+t-1,1] = 1;&#xA;&#xA;&#xA;    # declare the training data placeholders&#xA;    x = tf.placeholder(tf.float32, [None, 2])&#xA;    y = tf.placeholder(tf.float32, [None, 2])&#xA;&#xA;    # Weights connecting the input to the hidden layer 1&#xA;    W1 = tf.Variable(tf.random_normal([2, num_hidden_layer_1]))&#xA;    b1 = tf.Variable(tf.random_normal([num_hidden_layer_1]))&#xA;&#xA;    # activation of hidden layer 1&#xA;    hidden_1_out = tf.nn.relu(tf.add(tf.matmul(x, W1), b1))&#xA;&#xA;    last_hidden_out = hidden_1_out&#xA;    for i in range(1,num_of_layers):    &#xA;        # weights connecting the hidden layer i-1 to the hidden layer i&#xA;        next_W = tf.Variable(tf.random_normal([num_hidden_layer_1, num_hidden_layer_1]))&#xA;        next_b = tf.Variable(tf.random_normal([num_hidden_layer_1]))&#xA;&#xA;        # activation of hidden layer&#xA;        if (i%2 == 0):&#xA;            next_hidden_out = tf.nn.tanh(tf.add(tf.matmul(last_hidden_out, next_W), next_b))&#xA;        else:    &#xA;            next_hidden_out = tf.nn.tanh(tf.add(tf.matmul(last_hidden_out, next_W), next_b))&#xA;&#xA;        # update for next loop&#xA;        last_hidden_out = next_hidden_out&#xA;&#xA;    # and the weights connecting the last hidden layer to the output layer&#xA;    W_end = tf.Variable(tf.random_normal([num_hidden_layer_1, 2]))&#xA;    b_end = tf.Variable(tf.random_normal([2]))&#xA;&#xA;    # activation of output layer&#xA;    y_ = tf.nn.sigmoid(tf.add(tf.matmul(last_hidden_out, W_end), b_end))&#xA;&#xA;    # loss function&#xA;    y_clipped = tf.clip_by_value(y_, 1e-10, 0.9999999)&#xA;    cross_entropy = -tf.reduce_mean(tf.reduce_sum(y * tf.log(y_clipped) + (1 - y) * tf.log(1 - y_clipped), axis=1))&#xA;&#xA;    # add an optimiser&#xA;    optimiser = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)&#xA;&#xA;    # finally setup the initialisation operator&#xA;    init_op = tf.global_variables_initializer()&#xA;&#xA;    # define an accuracy assessment operation&#xA;    correct_prediction = tf.equal(tf.argmax(y_, 1), tf.argmax(y, 1))&#xA;    blah = tf.add(tf.matmul(last_hidden_out, W_end), b_end)&#xA;    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))&#xA;&#xA;    # For the purpose of separating dataset, testing data is the entire set&#xA;    testingX = X;&#xA;    testingY = Y;&#xA;&#xA;    # start the session&#xA;    with tf.Session() as sess:&#xA;       # initialise the variables&#xA;       sess.run(init_op)&#xA;       for epoch in range(epochs):&#xA;            this_cost = 0&#xA;            trainingX, trainingY = chooseTrainingBatch(X, Y, n, batch_size)&#xA;            _, c = sess.run([optimiser, cross_entropy], feed_dict={x: trainingX, y: trainingY})&#xA;            this_cost += c&#xA;            print(&quot;Epoch:&quot;, (epoch + 1), &quot;cost =&quot;, &quot;{:.3f}&quot;.format(this_cost))&#xA;&#xA;       #debugging statements&#xA;       #this_y = sess.run(y, feed_dict={x: testingX, y: testingY})        &#xA;       #print(&quot;y&quot;)&#xA;       #print(this_y)&#xA;       #this_blah = sess.run(blah, feed_dict={x: testingX, y: testingY})&#xA;       #print(&quot;blah&quot;)&#xA;       #print(this_blah)&#xA;       #this_y_ = sess.run(y_, feed_dict={x: testingX, y: testingY})&#xA;       #print(&quot;y_&quot;)&#xA;       #print(this_y_)&#xA;       this_accuracy = sess.run(accuracy, feed_dict={x: testingX, y: testingY})&#xA;       print(&quot;Accuracy: &quot;, this_accuracy)&#xA;       return this_accuracy&#xA;&#xA;&#xA;# this is the real thing&#xA;#doLinearClassification(100, learning_rate = 0.01, epochs=20000, num_hidden_layer_1=14, num_of_layers=4, batch_size=20);&#xA;&#xA;# this is just to debug the code&#xA;doLinearClassification(20, learning_rate = 0.01, epochs=200, num_hidden_layer_1=14, num_of_layers=4, batch_size=2);&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30134" LastActivityDate="2019-10-03T15:00:11.467" Title="Trying to separate spiral data with neural network, learning tensorflow" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15707" PostTypeId="1" CreationDate="2019-10-03T16:14:22.313" Score="2" ViewCount="25" Body="&lt;p&gt;Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, from the &lt;a href=&quot;https://github.com/google-research-datasets/paws&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAWS&lt;/a&gt; data set, given the phrases &quot;Although interchangeable, the body pieces on the 2 cars are not similar.&quot; and &quot;Although similar, the body parts are not interchangeable on the 2 cars.&quot; we have the label 0 because it is a counterexample, whereas for the phrases &quot;Katz was born in Sweden in 1947 and moved to New York City at the age of 1.&quot; and &quot;Katz was born in 1947 in Sweden and moved to New York at the age of one.&quot; we have the label 1 because it is a positive example of a valid paraphrase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to use the transformer model to generate paraphrases, and I am attempting to build a GAN but could not find any references for updating the transformer text-to-text model using counterexamples.&lt;/p&gt;&#xA;" OwnerUserId="29999" LastEditorUserId="2444" LastEditDate="2019-11-01T03:06:41.103" LastActivityDate="2019-11-01T03:06:41.103" Title="How to train a transformer text-to-text model on counterexamples?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;generative-adversarial-networks&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15708" PostTypeId="1" AcceptedAnswerId="15709" CreationDate="2019-10-03T18:50:30.653" Score="1" ViewCount="125" Body="&lt;p&gt;A stationary policy is a function that maps a state to a probability distribution of actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a contextual bandit problem, a state itself does not include the history. But in a reinforcement learning problem, the history can be used to define a state. In this case, does the history include the rewards revealed thus far as well? If so, the policy is not stationary anymore I guess. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the title says, I think I am confused about the difference in the definitions of stationary (and/or non-stationary) policy between reinforcement learning and contextual bandit.&lt;/p&gt;&#xA;" OwnerUserId="30051" LastEditorUserId="2444" LastEditDate="2019-10-03T18:56:39.720" LastActivityDate="2019-10-03T20:25:52.283" Title="What is the difference between the definition of a stationary policy in reinforcement learning and contextual bandit?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;definitions&gt;&lt;stationary-policy&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15709" PostTypeId="2" ParentId="15708" CreationDate="2019-10-03T20:00:04.150" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the difference between the definition of a stationary policy in reinforcement learning and contextual bandit?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no difference. A policy decides which action to take in each state. This is usually split into deterministic policies of the form &lt;span class=&quot;math-container&quot;&gt;$\pi(s) : \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt; and stochastic policies of the form &lt;span class=&quot;math-container&quot;&gt;$\pi(s|a) : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}, \text{Pr}[A_t=a |S_t=s]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we say that a policy is static, it means that the mapping from state to action - or distribution of actions - does not change over the time that we are interested in. This definition applies equally in the Reinforcement Learning (RL) and Contextual Bandit settings.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But in a reinforcement learning problem, the history can be used to define a state. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It can be used this way, but is not required. What is required is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; i.e. that identifying the state also determines all the allowed transitions and rewards, plus their probabilities of occurring given an action. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this case, does the history include the rewards revealed thus far as well? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If those can affect the future state transitions and rewards yes. If they generally do not, then you would usually exclude them from the state description.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, the policy is not stationary anymore I guess.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What you are proposing is a system that changes its state depending on rewards seen so far. As above, this is not necessary to define a RL problem, but it is allowed in RL. Whilst in a contextual bandit you assume no rules apply to state transitions, and this is a key difference between contextual bandit and RL settings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy is stationary if its mapping rules remain unchanged.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your proposed addition to the state does not require changing the policy. The policy is a function of the state. It may choose a different action depending on this historical aspect of the state, but it can still remain a fixed function - its input may change, but the policy function itself does not need to change to account for that.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-03T20:25:52.283" LastActivityDate="2019-10-03T20:25:52.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15710" PostTypeId="1" CreationDate="2019-10-03T20:08:00.337" Score="3" ViewCount="231" Body="&lt;p&gt;As far as I can tell (correct me if I'm wrong), Alphazero (with MCTS and neural network heuristic function RL) is the state of the art training method for turn based, deterministic, &lt;em&gt;perfect&lt;/em&gt; information, complete information, two player, zero sum games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what is the state of the art for turn based, &lt;em&gt;imperfect&lt;/em&gt; information games, that have 2 players, complete information, and is zero sum? (Deterministic or stochastic.) Examples include Battleship and most 2 player card games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there standard games, or other tests by which this is measured? Is the criteria I offered for type of game not specific enough to narrow the answer down properly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the state of the art involves supervised learning (data set of manually played games), then what's the state of the art for pure reinforcement learning, if there is one?&lt;/p&gt;&#xA;" OwnerUserId="27354" LastEditorUserId="27354" LastEditDate="2019-10-03T20:25:44.477" LastActivityDate="2019-10-03T20:25:44.477" Title="What is the state of the art AI training technique for imperfect information 2 player turn based games?" Tags="&lt;ai-design&gt;&lt;training&gt;&lt;game-theory&gt;&lt;markov-decision-process&gt;&lt;imperfect-information&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15711" PostTypeId="1" CreationDate="2019-10-04T01:50:09.870" Score="1" ViewCount="42" Body="&lt;p&gt;It was noted today that automated text generation is advancing at a rapid pace, potentially accelerating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As bots become more and more capable of passing turing tests, especially in single iterations, such as social media posts or news blurbs, I have to ask: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it matter where a text originates, if the content is strong?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Strength here is used in the sense of meaning.  To elucidate my argument I'll present an example.  (It helps to know the &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Library_of_Babel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Library of Babel&lt;/a&gt;, an infinite memory array where every possible combination of characters exists.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An algorithm is set up to produce &lt;a href=&quot;https://en.wikipedia.org/wiki/Aphorism&quot; rel=&quot;nofollow noreferrer&quot;&gt;aphorisms&lt;/a&gt;. The overwhelming majority of the output is gibberish, but among the junk is an incredibly profound observation emerges that changes the way people think about a subject or issue.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Where the bot just spams social media, the aphorism in question is identified because it recieves a high number of reposts by humans, who, in this scenario, provide the mechanism for finding the needle (the profound aphorism) in the haystack (the junk output).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the value of the insight depend on the cognitive quality of the generator, in the sense of having to understand the statement?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A real world example would be &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol#Game_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game 2, Move 37&lt;/a&gt; in the AlphaGo vs. Lee Sedol match.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-04T23:00:12.980" LastActivityDate="2019-10-04T23:00:12.980" Title="Does it matter if it's a bot or a human generating text? Doesn't it come down to the content?" Tags="&lt;natural-language-processing&gt;&lt;philosophy&gt;&lt;semantics&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15712" PostTypeId="1" AcceptedAnswerId="15760" CreationDate="2019-10-04T02:35:12.173" Score="2" ViewCount="86" Body="&lt;p&gt;I initialised an LSTM with Xavier initialisation, although I've found this occurs for all initialisations I have tested. When initialised, if the LSTM is tested with a random input, it will get stuck in a cycle, either over a few characters or just one. Example output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nhhbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb&#xA;&#xA;f(b(bf(bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb&#xA;&#xA;kk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,mkk,,m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've also noticed the LSTM is particularly bad in this way, that even when trained it has a tendency to get stuck in loops like this. It seems it has difficulty truly retaining context strong enough to over power the input, activation and output gates with only the forget gate. Is there an explanation for this?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2019-10-08T00:38:48.620" LastActivityDate="2019-10-08T00:38:48.620" Title="Why does an LSTM cycle on initialisation?" Tags="&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15713" PostTypeId="1" CreationDate="2019-10-04T05:30:35.870" Score="2" ViewCount="40" Body="&lt;p&gt;Should awareness of the existence of the AI designer be embedded in the AI consciousness itself?&lt;/p&gt;&#xA;" OwnerUserId="28605" LastEditorUserId="16909" LastEditDate="2019-10-06T01:59:53.860" LastActivityDate="2019-10-06T01:59:53.860" Title="Should awareness of the existence of the AI designer be embedded in the AI consciousness?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;artificial-consciousness&gt;&lt;mythology-of-ai&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15714" PostTypeId="1" CreationDate="2019-10-04T07:36:14.937" Score="2" ViewCount="76" Body="&lt;p&gt;There are the 3 Asimov’s laws:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot may not injure a human being or, through inaction, allow a&#xA;human being to come to harm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot must obey orders given to it by human beings, except where&#xA;such orders would conflict with the first law.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot must protect its own existence as long as such protection&#xA;does not conflict with the first or second law.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These laws are based on morality, which assumes that robots have sufficient agency and cognition to make moral decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally there are alternative laws of responsible robotics:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A human may not deploy a robot without the human–robot work&#xA;system meeting the highest legal and professional standards of&#xA;safety and ethics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot must respond to humans as appropriate for their roles.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A robot must be endowed with sufficient situated autonomy to&#xA;protect its own existence as long as such protection provides smooth&#xA;transfer of control to other agents consistent the first and second&#xA;laws&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thinking beyond morality, consciousness and the AI designer's professionalism to incorporate safety and ethics into the AI design. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should AI incorporate irrefutable parent rules for AI to be inevitably mortal by design?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to assure AI can be deactivated if necessary the way the deactivation procedure cannot be worked around by the AI itself, even at the cost of the AI termination as its inevitable destiny?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: to explain reasoning behind the main question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technological solutions are often based on observing biology and nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In evolutionary biology, for example, research results of birds mortality, show&lt;br&gt;&#xA;potential negative effect of telomere shortening (dna) on life in general.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;telomere length (TL) has become a biomarker of increasing interest&#xA;  within ecology and evolutionary biology, and has been found to predict&#xA;  subsequent survival in some recent avian studies but not others. &#xA;  (...) We performed a meta-analysis on these estimates and found an overall significant&#xA;  negative association implying that short telomeres are associated with&#xA;  increased mortality risk&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If such research is confirmed in general, then natural life expectancy is limited by design of its DNA, ie by design of its cell-level code storage. I assume this process of built-in mortality cannot be effectively worked around by a living creature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar design could be incorporated in any AI design, to assure its vulnerability and mortality, in the sense a conscious AI could otherwise recover and restore its full health state and continue to be up and running infinitely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise a simple turn off switch could be disabled by the conscious AI itself.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;References&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.925.4331&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Murphy, R. and Woods, D.D., 2009. Beyond Asimov: the three laws of responsible robotics. IEEE intelligent systems, 24(4), pp.14-20.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://royalsocietypublishing.org/doi/full/10.1098/rstb.2016.0447&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wilbourn Rachael V., Moatt Joshua P., Froy Hannah, Walling Craig A., Nussey Daniel H. and Boonekamp Jelle J. The relationship between telomere length and mortality risk in non-model vertebrate systems: a meta-analysis373Phil. Trans. R. Soc. B&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28605" LastEditorUserId="28605" LastEditDate="2019-10-04T09:40:48.337" LastActivityDate="2019-10-04T11:01:58.827" Title="Should AI be mortal by design?" Tags="&lt;philosophy&gt;&lt;death&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2021-01-23T04:09:04.077" ContentLicense="CC BY-SA 4.0" />
  <row Id="15715" PostTypeId="1" AcceptedAnswerId="15749" CreationDate="2019-10-04T08:32:35.453" Score="0" ViewCount="71" Body="&lt;p&gt;Here's a simple image classifier implemented in TensorFlow Keras (right click to open in new tab): &lt;a href=&quot;https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I altered it a bit to fit with my &lt;strong&gt;2-class dataset&lt;/strong&gt;. And the output layer is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dense(2, activation=tf.nn.softmax);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The loss function and optimiser are still the same as in the example in the link above.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss_fn   = tf.losses.SparseCategoricalCrossentropy();&#xA;optimizer = tf.optimizers.Adam();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wish to turn it into a classifier with single output neuron as I have only 2 classes in dataset, and sigmoid does the 2 classes good. Tried some combinations of output activation functions + loss functions + optimisers, but the network doesn't work any more (ie. it doesn't converge).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this doesn't work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//output layer&#xA;Dense(1, activation=tf.sigmoid);&#xA;&#xA;//loss and optim&#xA;loss_fn   = tf.losses.mse;&#xA;optimizer = tf.optimizers.Adagrad(1e-1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which combination of &lt;strong&gt;output activation + loss + optimiser&lt;/strong&gt; should work for the single-output-neuron model? &lt;strong&gt;And generically, which loss functions and optimisers should pair well?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-06T07:25:26.260" LastActivityDate="2019-10-06T08:39:33.427" Title="TF Keras: How to turn this probability-based classifier into single-output-neuron label-based classifier" Tags="&lt;classification&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;loss-functions&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15716" PostTypeId="2" ParentId="15711" CreationDate="2019-10-04T08:38:35.910" Score="1" Body="&lt;p&gt;sometimes voice pitch also matters(and system designers forget about that) - if system is badly designed then you will feel that it is a bot, if design is great than you don't feel it's a bot at all&lt;/p&gt;&#xA;" OwnerUserId="23181" LastActivityDate="2019-10-04T08:38:35.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15717" PostTypeId="2" ParentId="15711" CreationDate="2019-10-04T08:40:13.203" Score="1" Body="&lt;p&gt;There are several aspects to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, content. I guess a further comparison would be to the monkeys on typewriters coming up with the complete works of Shakespeare eventually. You will probably have a huge mass of tedious text, with the odd nugget in it. One would hope that the signal-to-noise ratio would be better with human authors, though looking at Twitter that might not actually be the case (unless there are already more bots on there than humans!) Content evaluation/filtering is still an issue, especially with longer texts. How many computer-generated novels are there? Not as many as weather reports or news summaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, legal issues. Who 'owns' the text and is responsible for it? If a computer generates a text that incites violence, who is to blame for the consequences? The developer? Some texts require 'ownership', as they might have consequences attached to them. A wrong weather forecast that causes people to make wrong decisions, leading to loss of life. Do you sue the computer for that? So in this respect, clear responsibility is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So even if the content is good (point 1), there might also be responsibility (point 2) where it matters whether a text has been generated by an algorithm.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-10-04T08:40:13.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15720" PostTypeId="1" AcceptedAnswerId="15721" CreationDate="2019-10-04T10:38:53.320" Score="2" ViewCount="55" Body="&lt;p&gt;According to the definition, the AI agent has to play a game by it's own. A typical domain is the blocksworld problem. The AI determines which action the robot in a game should execute and a possible strategy for determine the action sequence is reinforcement learning. Colloquial spoken, reinforcement learning leads to an AI agent who can play games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before a self-learning character can be realized, the simulation has to be programmed first. That is an environment which contains the rules for playing blocksworld or any other game. The environment is the house in which the AI character operates. Can the Q-learning algorithm be utilized to build the simulation itself?&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastActivityDate="2019-10-04T15:26:55.330" Title="Can reinforcement learning be utilized for creating a simulation?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15721" PostTypeId="2" ParentId="15720" CreationDate="2019-10-04T10:53:48.260" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the Q-learning algorithm be utilized to build the simulation itself?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Only in the presence of a meta-environment, or meta-simulation where the goals of creating the original simulation are encoded in the states, available actions and rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A special case of this might be in model-learning planning algorithms where there exists a &quot;real&quot; environment to refer to, and the agent benefits from exploring it and constructing a statistical model that it can then use to create an approximate simulation of the outcomes of sequences of actions. The &lt;a href=&quot;http://www-anw.cs.umass.edu/~barto/courses/cs687/Chapter%209.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dyna-Q algorithm&lt;/a&gt;, which is a simple extension of Q-learning, is an example of this kind of model building approach. The simulation is very basic - it simply replays previous relevant experience. But you could consider this as an example of the agent constructing a simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting an agent to act like a researcher and actually design and/or code a simulation from scratch would require a different kind of meta-environment. It is theoretically possible, but likely very hard to implement in general way - even figuring out the reward scheme to express the goals of such an agent could be a challenge. I'm not aware of any examples, but entirely possible someone has attempted this kind of meta agent, because it is an interesting idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly the simplest example would be a gridworld meta-environment where a &quot;designer&quot; agent could select the layout of objects in the maze, with the goal of making a second &quot;explorer&quot; agent's task progressivley more difficult. The designer would be &quot;creating the simulation&quot; only in a very abstract way though, by setting easy-to-manage parameters of the environment, not writing low level code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is not much difference between the approach above and having two opposing agents playing a game. It is different from a turn-based game like chess in that each agent would complete a full episode, and then be rewarded by the outcome at the end of the combined two episodes. There are some similarities to GANs for image generation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-04T15:26:55.330" LastActivityDate="2019-10-04T15:26:55.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15724" PostTypeId="2" ParentId="12857" CreationDate="2019-10-04T13:31:04.017" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The variables are macro-econometric and they, in general, seem to have some influence on stocks' prices. This inclusion might as well increase your model's prediction accuracy. You can definitely use them as predictors. As mentioned in comments - Experimentation is the way to go.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can I feed the data to my neural network like the table above?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, you can have any kind of &lt;em&gt;numeric&lt;/em&gt; variables as input to a neural network. Things will work out fine. The important thing is the selection of relevant predictor variables that, potentially, have some relationship with the response variable.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-10-05T05:26:04.093" LastActivityDate="2019-10-05T05:26:04.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15727" PostTypeId="1" CreationDate="2019-10-04T18:23:14.843" Score="0" ViewCount="45" Body="&lt;p&gt;H2O's &lt;a href=&quot;https://www.h2o.ai/products/h2o-sparkling-water/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sparkling Water&lt;/a&gt; provides the ability to combine H2O's machine learning algorithms with &lt;a href=&quot;https://spark.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spark&lt;/a&gt;.  As far as I understand support is for version 3 of H2O.  I want to build a cluster of computers with GPUs and run &lt;a href=&quot;https://www.h2o.ai/products/h2o4gpu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;H2O4GPU&lt;/a&gt;.  Does Sparkling Water support H2O4GPU?&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2019-10-04T18:23:14.843" Title="Does Sparkling Water support H2O4GPU?" Tags="&lt;software-architecture&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15729" PostTypeId="1" CreationDate="2019-10-04T20:52:00.600" Score="2" ViewCount="153" Body="&lt;p&gt;A little background... I’ve been on-and-off learning about data science for around a year or so, however, I started thinking about artificial intelligence a few years ago.  I have a cursory understandings of some common concepts but still not much depth.  When I first learned about deep learning, my automatic response was “that’s not how our minds do it.”  Deep learning is obviously an important topic, but I’m trying to think outside the black box.&lt;/p&gt;&#xA;&lt;p&gt;I think of deep learning as being “outside-in” in that a model has to rely on examples to understand (for lack of a better term) that some dataset is significant.  However, our minds seem to know when something is significant in the absence of any prior knowledge of the thing (i.e., “inside-out”).&lt;/p&gt;&#xA;&lt;p&gt;Here’s a thing:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wiuEg.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;I googled “IKEA hardware” to find that.  The point is that you probably don’t know what this is or have any existing mental relationship between the image and anything else, but you can see that it’s something (or two somethings).  I realize there is unsupervised learning, image segmentation, etc., which deal with finding order in unlabeled data, but I think this example illustrates the difference between the way we tend to think about machine learning/AI and how our minds actually work.&lt;/p&gt;&#xA;&lt;p&gt;More examples:&lt;/p&gt;&#xA;&lt;h1&gt;1)&lt;/h1&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/F67jR.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;h1&gt;2)&lt;/h1&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QkSsr.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;h1&gt;3)&lt;/h1&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sbYoO.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let’s say that #1 is a stock chart.  If I were viewing the chart and trying to detect a pattern, I might mentally simplify the chart down to #2.  That is, the chart can be simplified into a horizontal segment and a rising segment.&lt;/p&gt;&#xA;&lt;p&gt;For #3, let’s say this represents log(x).  Even though it’s not a straight line, someone with no real math background could describe it as an upward slope that it decreasing as the line gets higher.  That is, the line can still be reduced to a small number of simple ideas.&lt;/p&gt;&#xA;&lt;p&gt;I think this simplification is the key to the gap between how our minds work and what currently exists in AI.  I’m aware of Fourier transforms, polynomial regression, etc., but I think there’s a more general process of finding order in sensory data.  Once we identify something orderly (i.e., something that can’t reasonably be random noise), we label it as a thing and then our mental network establishes relationships between it and other things, higher order concepts, etc.&lt;/p&gt;&#xA;&lt;p&gt;I’ve been trying to think about how to use decision trees to find pockets of order in data (to no avail yet - I haven’t figured out to apply it to all of the scenarios above), but I’m wondering if there are any other techniques or schools of thought that align with the general theory.&lt;/p&gt;&#xA;" OwnerUserId="30154" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-10-29T20:03:35.377" Title="“Outside-in” versus “Inside-out” machine learning" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;decision-tree&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15730" PostTypeId="1" AcceptedAnswerId="15744" CreationDate="2019-10-05T00:18:38.083" Score="44" ViewCount="14835" Body="&lt;p&gt;As a human being, we can think infinity. In principle, if we have enough resources (time etc.), we can count infinitely many things (including abstract, like numbers, or real).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, at least, we can take into account integers. We can think, principally, and &quot;understand&quot; infinitely many numbers that are displayed on the screen. Nowadays, we are trying to design artificial intelligence which is capable at least human being. However, I am stuck with infinity. I try to find a way how can teach a model (deep or not) to understand infinity. I define &quot;understanding' in a functional approach. For example, If a computer can differentiate 10 different numbers or things, it means that it really understand these different things somehow. This is the basic straight forward approach to &quot;understanding&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I mentioned before, humans understand infinity because they are capable, at least, counting infinite integers, in principle. From this point of view, if I want to create a model, the model is actually a function in an abstract sense, this model must differentiate infinitely many numbers. Since computers are digital machines which have limited capacity to model such an infinite function, how can I create a model that differentiates infinitely many integers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we can take a deep learning vision model that recognizes numbers on the card. This model must assign a number to each different card to differentiate each integer. Since there exist infinite numbers of integer, how can the model assign different number to each integer, like a human being, on the digital computers? If it cannot differentiate infinite things, how does it understand infinity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I take into account real numbers, the problem becomes much harder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the point that I am missing? Are there any resources that focus on the subject?&lt;/p&gt;&#xA;" OwnerUserId="19102" LastEditorUserId="2444" LastEditDate="2019-10-05T00:34:50.200" LastActivityDate="2021-04-01T15:00:15.513" Title="Can digital computers understand infinity?" Tags="&lt;deep-learning&gt;&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="19" CommentCount="1" FavoriteCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="15731" PostTypeId="1" AcceptedAnswerId="15732" CreationDate="2019-10-05T01:51:23.897" Score="5" ViewCount="195" Body="&lt;p&gt;What are the real-life applications of transfer learning in machine learning? I am particularly interested in industrial applications of the concept.&lt;/p&gt;&#xA;" OwnerUserId="30157" LastEditorUserId="2444" LastEditDate="2020-06-11T21:35:46.220" LastActivityDate="2020-06-11T21:35:46.220" Title="What are the real-life applications of transfer learning?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;transfer-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15732" PostTypeId="2" ParentId="15731" CreationDate="2019-10-05T05:10:52.177" Score="6" Body="&lt;p&gt;One application I know of being used in industry is of image classification, by only training the last layer of one of the inception models released by Google,  with the desired number of classes. I can't provide specific details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transfer learning is useful when:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You do not have the resources (time, processing power, etc.) to train a DL  model from scratch.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can compromise a bit on accuracy. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="2444" LastEditDate="2020-06-11T21:35:13.357" LastActivityDate="2020-06-11T21:35:13.357" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15733" PostTypeId="2" ParentId="12570" CreationDate="2019-10-05T09:26:37.997" Score="1" Body="&lt;p&gt;The true value &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; is a conceptual target for the &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt; in the book. You often do not know it in real problems. However, it is still used in two main ways in the book:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Theoretically for analysis of different aprpoximation schemes, which can be shown to converge to minimise the &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt; objective, or a related one. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In toy problems when exploring the nature of approximation in Reinforcement Learning (RL), it is possible to use tabular methods guranateed to get close to zero error, and then compare them to approximate methods. There are several plots of this type in the book.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The book shows derivation of gradient descent methods that start with minimising &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt; as an objective, and that use &lt;em&gt;samples&lt;/em&gt; of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; such as Monte Carlo return &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; or the TD target) in place of the unknown &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; in the loss function. These also rely on the fact that the sample distribution will be weighted by &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; if they are taken naturally from the environment whilst the agent is following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; also does not need to be explicitly known. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outside of toy problems deliberately constructed to demonstrate that this theory is correct, you will not know &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; or be able to calculate &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt;. However, you will know from the theory that if you follow the update rules derived in the book for approximate gradient descent methods, that the process should find a local minimum for &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt;, for whatever state approximation scheme you have chosen to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually you cannot even &lt;em&gt;approximate&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt; from raw data, as the variance in returns will add noise to the signal, and there is no way to separate variance in returns from error in approximation in the general case. However there are a couple of scenarios that do lend themselves to measuring this objective, provided you already have your estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(s,w)$&lt;/span&gt; and the policy remains fixed throughout:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Simple, fast (perhaps simulated), environments which can be solved to arbitrary accuracy using tabular methods. In this case, you first calculate &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; using a non-approximate method, then sample many approximations by running the environment using policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and treating that as your data set.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fully deterministic environments where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is also deterministic. These have a variance of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for Monte Carlo returns, so each observed return from any given state is already the true value of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;. Again you can just run the environment many times to get your data set to calculate  &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(s,w)$&lt;/span&gt; for the observed states in the correct frequencies, and thus have data to calculate &lt;span class=&quot;math-container&quot;&gt;$\overline{VE}$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-05T11:00:54.360" LastActivityDate="2019-10-05T11:00:54.360" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15736" PostTypeId="2" ParentId="15683" CreationDate="2019-10-05T12:36:49.023" Score="1" Body="&lt;p&gt;I think, once you are covered with the common stuff, you can probably go on and study all kinds of neural network variants. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The common stuff:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) An undergraduate level Linear Algebra course -- covering matrix calculus. You might find &lt;a href=&quot;https://stats.stackexchange.com/questions/21346/reference-book-for-linear-algebra-applied-to-statistics&quot;&gt;this&lt;/a&gt; useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) An undergraduate level study in statistical inference. Concepts from this topic will come up most of the time and you might have hard time getting around even though you understand the rest of the math. I would recommend &lt;a href=&quot;https://fsalamri.files.wordpress.com/2015/02/casella_berger_statistical_inference1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;c) A starter book on neural networks. Ex- &lt;a href=&quot;http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/1996/NeuralNetworks/neuron.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural networks&lt;/a&gt; by Raul Rojas.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;After all these are covered you will certainly be ready for learning the variants of neural networks with ease. For LSTM I would recommend &lt;a href=&quot;https://www.cs.toronto.edu/~graves/preprint.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alex Graves&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="16708" LastActivityDate="2019-10-05T12:36:49.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15737" PostTypeId="1" CreationDate="2019-10-05T13:53:10.597" Score="5" ViewCount="175" Body="&lt;p&gt;I'm trying to learn how genetic algorithms can solve optimization problems. I have already learned how genetic algorithms can solve the knapsack, TSP and set cover problems. I'm looking for some other similar optimization problems, but I have not found any.&lt;/p&gt;&#xA;&lt;p&gt;Would you please mention some other famous optimization problems that can be solved by using genetic algorithms?&lt;/p&gt;&#xA;" OwnerUserId="30164" LastEditorUserId="2444" LastEditDate="2021-01-15T11:49:43.263" LastActivityDate="2021-01-15T11:52:41.100" Title="What are examples of optimization problems that can be solved using genetic algorithms?" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;applications&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15738" PostTypeId="2" ParentId="15737" CreationDate="2019-10-05T14:15:55.027" Score="1" Body="&lt;p&gt;There are numerous problems that can be solved with genetic algorithms or, more generally, with evolutionary algorithms (which includes also genetic programming and evolutionary strategies), even though they may not necessarily be the most efficient approach.&lt;/p&gt;&#xA;&lt;p&gt;Here are a few examples.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Evolution of the topology of neural networks. This is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroevolution&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Automatic test case generation (in particular, for self-driving cars). &lt;a href=&quot;https://www.researchgate.net/profile/Alessio_Gambi/publication/335362266_AsFault_Testing_Self-Driving_Car_Software_Using_Search-Based_Procedural_Content_Generation/links/5d70bd4792851cacdb21a956/AsFault-Testing-Self-Driving-Car-Software-Using-Search-Based-Procedural-Content-Generation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AsFault&lt;/a&gt; is one specific example.&lt;/li&gt;&#xA;&lt;li&gt;Design of novel quantum computing algorithms. Specifically, genetic programming has been used to solve this problem (see &lt;a href=&quot;https://www.researchgate.net/publication/244428002_Genetic_Programming_for_Quantum_Computers&quot; rel=&quot;nofollow noreferrer&quot;&gt;this reference&lt;/a&gt; for more details).&lt;/li&gt;&#xA;&lt;li&gt;As an alternative to reinforcement learning algorithms to solve RL problems. Specifically, evolution strategies have been successfully used in this case (see &lt;a href=&quot;https://openai.com/blog/evolution-strategies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There is a Wikipedia article that lists many other applications of genetic algorithms: &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_genetic_algorithm_applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;List of genetic algorithm applications&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-15T11:52:41.100" LastActivityDate="2021-01-15T11:52:41.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15740" PostTypeId="2" ParentId="15730" CreationDate="2019-10-05T15:31:58.387" Score="3" Body="&lt;p&gt;By adding some rules for infinity in arithmetic (such as infinity minus a large finite number is infinity, etc.), the digital computer  can appear to understand the notion of infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, the computer can simply replace the number n with its &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterated_logarithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;log-star&lt;/a&gt; value.  Then, it can differentiate the numbers at a different scale, and can learn that  any number with log-star value &gt; 10 is practically equivalent to infinity. &lt;/p&gt;&#xA;" OwnerUserId="20745" LastActivityDate="2019-10-05T15:31:58.387" CommentCount="2" CommunityOwnedDate="2019-10-05T15:31:58.387" ContentLicense="CC BY-SA 4.0" />
  <row Id="15743" PostTypeId="1" AcceptedAnswerId="15748" CreationDate="2019-10-05T18:25:11.080" Score="6" ViewCount="2522" Body="&lt;p&gt;As I understand, ResNet has some identity mapping layers, whose task is to create the output as the same as the input of the layer. The ResNet solved the problem of accuracy degrading. But what is the benefit of adding identity mapping layers in intermediate layers?&lt;/p&gt;&#xA;&lt;p&gt;What's the effect of these identity layers on the feature vectors that will be produced in the last layers of the network? Is it helpful for the network to produce better representation for the input? If this expression is correct, what is the reason?&lt;/p&gt;&#xA;" OwnerUserId="30170" LastEditorUserId="2444" LastEditDate="2021-02-06T02:43:35.740" LastActivityDate="2021-05-04T10:12:05.717" Title="What is the benefit of using identity mapping layers in deep neural networks like ResNet?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;residual-networks&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15744" PostTypeId="2" ParentId="15730" CreationDate="2019-10-05T19:19:04.713" Score="61" Body="&lt;p&gt;I think this is a fairly common misconception about AI and computers, especially among laypeople. There are several things to unpack here.&lt;/p&gt;&#xA;&lt;p&gt;Let's suppose that there's something special about infinity (or about continuous concepts) that makes them especially difficult for AI. For this to be true, it must &lt;em&gt;both&lt;/em&gt; be the case that humans can understand these concepts while they remain alien to machines, &lt;em&gt;and&lt;/em&gt; that there exist other concepts that are not like infinity that both humans &lt;em&gt;and&lt;/em&gt; machines can understand. What I'm going to show in this answer is that wanting both of these things leads to a contradiction.&lt;/p&gt;&#xA;&lt;p&gt;The root of this misunderstanding is the problem of what it means to &lt;em&gt;understand&lt;/em&gt;. Understanding is a vague term in everyday life, and that vague nature contributes to this misconception.&lt;/p&gt;&#xA;&lt;p&gt;If by understanding, we mean that a computer has the conscious experience of a concept, then we quickly become trapped in metaphysics. There is a &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;long running&lt;/a&gt;, and essentially open debate about whether computers can &amp;quot;understand&amp;quot; anything in this sense, and even at times, about whether humans can! You might as well ask whether a computer can &amp;quot;understand&amp;quot; that 2+2=4. Therefore, if there's something &lt;em&gt;special&lt;/em&gt; about understanding infinity, it cannot be related to &amp;quot;understanding&amp;quot; in the sense of subjective experience.&lt;/p&gt;&#xA;&lt;p&gt;So, let's suppose that by &amp;quot;understand&amp;quot;, we have some more specific definition in mind. Something that would make a concept like infinity more complicated for a computer to &amp;quot;understand&amp;quot; than a concept like arithmetic. Our more concrete definition for &amp;quot;understanding&amp;quot; must relate to some objectively measurable capacity or ability related to the concept (otherwise, we're back in the land of subjective experience). Let's consider what capacity or ability might we pick that would make infinity a special concept, understood by humans and not machines, unlike say, arithmetic.&lt;/p&gt;&#xA;&lt;p&gt;We might say that a computer (or a person) understands a concept if it can provide a correct definition of that concept. However, if even one human understands infinity by this definition, then it should be easy for them to write down the definition. Once the definition is written down, a computer program can output it. Now the computer &amp;quot;understands&amp;quot; infinity too. This definition doesn't work for our purposes.&lt;/p&gt;&#xA;&lt;p&gt;We might say that an entity understands a concept if it can &lt;em&gt;apply&lt;/em&gt; the concept correctly. Again, if even the one person understands how to apply the concept of infinity correctly, then we only need to record the rules they are using to reason about the concept, and we can write a program that reproduces the behavior of this system of rules. Infinity is actually very well characterized as a concept, captured in ideas like &lt;a href=&quot;https://en.wikipedia.org/wiki/Aleph_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aleph Numbers&lt;/a&gt;. It is not impractical to encode these systems of rules in a computer, at least up to the level that any human understands them. Therefore, computers can &amp;quot;understand&amp;quot; infinity up to the same level of understanding as humans by this definition as well. So this definition doesn't work for our purposes.&lt;/p&gt;&#xA;&lt;p&gt;We might say that an entity &amp;quot;understands&amp;quot; a concept if it can logically relate that concept to arbitrary new ideas. This is probably the strongest definition, but we would need to be pretty careful here: very few humans (proportionately) have a deep understanding of a concept like infinity. Even fewer can readily relate it to arbitrary new concepts. Further, algorithms like the &lt;a href=&quot;https://en.wikipedia.org/wiki/General_Problem_Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Problem Solver&lt;/a&gt; can, in principal, derive any logical consequences from a given body of facts, given enough time. Perhaps under this definition computers understand infinity &lt;em&gt;better&lt;/em&gt; than most humans, and there is certainly no reason to suppose that our existing algorithms will not further improve this capability over time. This definition does not seem to meet our requirements either.&lt;/p&gt;&#xA;&lt;p&gt;Finally, we might say that an entity &amp;quot;understands&amp;quot; a concept if it can generate examples of it. For example, I can generate examples of problems in arithmetic, and their solutions. Under this definition, I probably do not &amp;quot;understand&amp;quot; infinity, because I cannot actually point to or create any concrete thing in the real world that is definitely infinite. I cannot, for instance, actually write down an infinitely long list of numbers, merely formulas that express ways to create ever longer lists by investing ever more effort in writing them out. A computer ought to be at least as good as me at this. This definition also does not work.&lt;/p&gt;&#xA;&lt;p&gt;This is not an exhaustive list of possible definitions of &amp;quot;understands&amp;quot;, but we have covered &amp;quot;understands&amp;quot; as I understand it pretty well. Under every definition of understanding, there isn't anything special about infinity that separates it from other mathematical concepts.&lt;/p&gt;&#xA;&lt;p&gt;So the upshot is that, either you decide a computer doesn't &amp;quot;understand&amp;quot; anything at all, or there's no particularly good reason to suppose that infinity is harder to understand than other logical concepts. If you disagree, you need to provide a concrete definition of &amp;quot;understanding&amp;quot; that &lt;em&gt;does&lt;/em&gt; separate understanding of infinity from other concepts, and that doesn't depend on subjective experiences (unless you want to claim your particular metaphysical views are universally correct, but that's a &lt;strong&gt;hard&lt;/strong&gt; argument to make).&lt;/p&gt;&#xA;&lt;p&gt;Infinity has a sort of semi-mystical status among the lay public, but it's really just like any other mathematical system of rules: if we can write down the rules by which infinity operates, a computer can do them as well as a human can (or better).&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="36737" LastEditDate="2021-04-01T15:00:15.513" LastActivityDate="2021-04-01T15:00:15.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15745" PostTypeId="2" ParentId="15729" CreationDate="2019-10-05T19:47:57.893" Score="1" Body="&lt;p&gt;It sounds like you are interested in the ideas of intrinsic motivation and attention in the context of machine learning. These are big topics, and the subject of much active research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.cornell.edu/~helou/IMRL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intrinsic motivation&lt;/a&gt; says that the key to identifying interesting patterns and skills that are worth learning is to give the agent some intrinsic reason to learn to do new things. This is not dissimilar from what humans have: learning new things, and improving or exercising our capabilities to the fullest is what &lt;a href=&quot;https://en.wikipedia.org/wiki/Aristotle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aristotle&lt;/a&gt; identified as the good life. There are thus good reasons to think that intrinsic motivation for AI might solve the problem you identify. Current research in this domain is exploring different mathematical ways to represent intrinsic motivation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attention was the subject of a large burst of research in deep neural networks during the last few years. Here's a &lt;a href=&quot;https://icml.cc/media/Slides/icml/2019/halla(10-09-15)-10-15-45-4343-a_tutorial_on.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent talk&lt;/a&gt; from AWS at ICML that provides a good overview. The idea here is that an agent can learn both a reasonable mapping from inputs to outputs for some problem, &lt;em&gt;and&lt;/em&gt; a separate mapping that describes how different future inputs should &quot;activate&quot; certain parts of the input/output mapping that the agent has learned. Essentially attention-driven models include a second component that learns which features of the input to pay attention to when engaging in certain kinds of tasks. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-05T19:47:57.893" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15746" PostTypeId="1" CreationDate="2019-10-05T19:55:32.150" Score="4" ViewCount="92" Body="&lt;p&gt;Are there datasets to solve differential equations in a supervised fashion? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, the input is a differential equation and the label should be the general solution to that differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've searched the internet and nothing has come up. I'm looking for some recommendations.&lt;/p&gt;&#xA;" OwnerUserId="30174" LastEditorUserId="2444" LastEditDate="2019-10-13T00:05:33.380" LastActivityDate="2019-10-13T00:43:16.953" Title="Are there datasets to solve differential equations in a supervised fashion?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ClosedDate="2021-02-03T17:07:21.333" ContentLicense="CC BY-SA 4.0" />
  <row Id="15747" PostTypeId="2" ParentId="15730" CreationDate="2019-10-06T01:10:10.737" Score="12" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: The subtleties of infinity are made apparent in the notion of unboundedness. Unboundedness is finitely definable. &quot;Infinite things&quot; are really things with unbounded natures. Infinity is best understood not as a thing but as a concept. Humans theoretically possess unbounded abilities &lt;em&gt;not&lt;/em&gt; infinite abilities (eg to count to any arbitrary number as opposed to &quot;counting to infinity&quot;). A machine can be made to recognize unboundedness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Down the rabbit hole again&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to proceed? Let's start with &quot;limits.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Limitations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our brains are not infinite (lest you believe in some metaphysics). So, we do not &quot;think infinity&quot;. Thus, what we purport as infinity is best understood as some &lt;strong&gt;finite&lt;/strong&gt; mental &lt;em&gt;concept&lt;/em&gt; against which we can &quot;compare&quot; other concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, we cannot &quot;count infinite integers.&quot; There is a subtly here that is &lt;em&gt;very&lt;/em&gt; important to point out: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our concept of quantity/number is &lt;strong&gt;&lt;em&gt;unbounded&lt;/em&gt;&lt;/strong&gt;. That is, for any any finite value we have a finite/concrete way or producing another value which is strictly larger/smaller. That is, Provided &lt;em&gt;finite&lt;/em&gt; time we could only count &lt;em&gt;finite&lt;/em&gt; amounts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You cannot be &quot;given infinite time&quot; to &quot;count all the numbers&quot; this would imply a &quot;finishing&quot; which directly contradicts the notion of infinity. Unless you believe humans have metaphysical properties which allow them to &quot;consistently&quot; embody a paradox. Additionally how would you answer: What was the last number you counted? With no &quot;last number&quot; there is never a &quot;finish&quot; and hence never an &quot;end&quot; to your counting. That is you can never &quot;have enough&quot; time/resources to &quot;count to infinity.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think what you mean is we can fathom the notion of &lt;a href=&quot;https://en.wikipedia.org/wiki/Bijection&quot; rel=&quot;nofollow noreferrer&quot;&gt;bijection&lt;/a&gt; between infinite sets. But this notion is a logical &lt;em&gt;construction&lt;/em&gt; (ie it's a finite way of wrangling what we understand to be infinite).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what we are really doing is: Within our bounds we are talking about our bounds and, when ever we need to, we can expand our bounds (by a finite amount). And we can even talk about the nature &lt;em&gt;of&lt;/em&gt; expanding our bounds. Thus: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Unboundedness&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A process/thing/idea/object is deemed unbounded if given some &lt;em&gt;measure&lt;/em&gt; of its quantity/volume/existence we can in a finite way produce an &quot;extension&quot; of that object which has a measure we deem &quot;larger&quot; (or &quot;smaller&quot; in the case of infinitesimals) than the previous measure and that this extension process can be applied to the nascent object (ie the process is recursive).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Canonical case number one: The Natural Numbers&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, our notion of infinity prevents any &quot;at-ness&quot; or &quot;upon-ness&quot; unto infinity. That is, one never &quot;arrives&quot; at infinity nor does one ever &quot;have&quot; infinity. Rather, one proceeds unboundedly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus how do we conceptualize infinity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Infinity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that &quot;infinity&quot; as a word is misconstrued to mean that there is a &lt;em&gt;thing that exists&lt;/em&gt; called &quot;infinity&quot; as opposed to a &lt;em&gt;concept&lt;/em&gt; called &quot;infinity&quot;. Let's smash atoms with the word:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Infinite: limitless or endless in space, extent, or size; impossible to measure or calculate.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;in- :a prefix of Latin origin, corresponding to English un-, having a negative or privative force, freely used as an English formative, especially of adjectives and their derivatives and of nouns (inattention; indefensible; inexpensive; inorganic; invariable).&#xA;  (&lt;a href=&quot;https://www.dictionary.com/browse/in-&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Finite: having limits or bounds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So in-finity is really un-finity which is &lt;em&gt;not having limits or bounds&lt;/em&gt;. But we can be more precise here because we can all agree the natural numbers are infinite but &lt;em&gt;any&lt;/em&gt; given natural number is finite. So what gives? Simple: &lt;em&gt;the&lt;/em&gt; natural numbers satisfy our unboundedness criterium and thus we say &quot;the natural numbers are infinite.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, &quot;infinity&quot; is a concept. An object/thing/idea is deemed infinite if it possess a property/facet that is unbounded. As before we saw that unboundedness is finitely definable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if the agent you speak of was programmed well enough to spot the pattern in the numbers on the cards and that the numbers are all coming from the same set it could deduce the unbounded nature of the sequence and hence define the set of all numbers as infinite - purely because the set &lt;em&gt;has no upper bound&lt;/em&gt;. That is, the progression of the natural numbers is unbounded and hence definably infinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, to me, infinity is best understood as a general concept for identifying when processes/things/ideas/objects posses an unbounded nature. That is, infinity is not independent of unboundedness. Try defining infinity without comparing it to finite things or the bounds of those finite things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems feasible that a machine could be programmed to represent and detect instances of unboundedness or when it might be admissible to assume unboundedness.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-10-08T20:03:14.257" LastActivityDate="2019-10-08T20:03:14.257" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15748" PostTypeId="2" ParentId="15743" CreationDate="2019-10-06T04:10:29.097" Score="7" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: Deep networks have some issues that skip connections fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address this statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As I understand Resnet has some identity mapping layers that their task is to create the output as the same as the input of the layer&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The residual blocks don't strictly learn the identity mapping. They are simply &lt;em&gt;capable&lt;/em&gt; of learning such a mapping. That is, the residual block makes learning the identity function easy. So, at the very least, skip connections will not hurt performance (this is explained formally in the paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5TPjb.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5TPjb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observe: it's taking some of the layer outputs from earlier layers and passing their outputs further down and element wise summing these with the the outputs from the skipped layers. These blocks may learn mappings that are not the identity map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From paper (some benefits):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\boldsymbol{y} = \mathcal{F}(\boldsymbol{x},\{W_i\})+\boldsymbol{x}\quad\text{(1)}$$&lt;/span&gt;The shortcut connections in Eqn.(1) introduce neither extra parameter nor computation complexity. This is not only attractive in practice but also important in our comparisons between plain and residual networks. We can fairly compare plain/residual networks that simultaneously have the same number of parameters, depth, width, and computational cost (except for the negligible element-wise addition).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An example of a residual mapping from the paper is &lt;span class=&quot;math-container&quot;&gt;$$\mathcal{F} = W_2\sigma_2(W_1\boldsymbol{x})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is &lt;span class=&quot;math-container&quot;&gt;$\{W_i\}$&lt;/span&gt; represents a set of i weight matrices (&lt;span class=&quot;math-container&quot;&gt;$W_1,W_2$&lt;/span&gt; in the example) occurring in the layers of the residual (skipped) layers. The &quot;identity shortcuts&quot; are referring to performing the element wise addition of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}$&lt;/span&gt; with the output of the residual layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So using the residual mapping from the example (1) becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\boldsymbol{y} = W_2\sigma_2(W_1\boldsymbol{x})+\boldsymbol{x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, you take the output &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}$&lt;/span&gt; of a layer skip it forward and element wise sum it with the output of the residual mapping and thus produce a residual block. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Limitations of deep networks expressed in paper&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly. Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments. Fig. 1 shows a typical example.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The skip connections and hence the residual blocks allow for stacking deeper networks while avoiding this degradation issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;noreferrer&quot;&gt;Link to paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-10-06T05:53:53.920" LastActivityDate="2019-10-06T05:53:53.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15749" PostTypeId="2" ParentId="15715" CreationDate="2019-10-06T06:34:00.437" Score="0" Body="&lt;p&gt;Advice from Neil, yes, output targeting class labels is still classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output range is in contiguous range:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This is regression&lt;/li&gt;&#xA;&lt;li&gt;For example: Linear activation, it has full numeric range of outputs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Output targeting class labels is classification:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Single output neuron with sigmoid-like functions. This will classify to 2 classes, although Y data can be normalised to classify more classes.&lt;/li&gt;&#xA;&lt;li&gt;Multiple output neurons (probabilities of classes) with sigmoid-like functions (&lt;code&gt;softmax&lt;/code&gt; is mainly used). This will classify to 2 or more classes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Multiple combinations of loss functions and optimisers can make the single-neuron output layer work, with different configs for them. Note that learning rates of different optimisers are different, some take 1e-1, some need 1e-3 for good training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this combination should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss_fn   = tf.losses.LogCosh();&#xA;optimizer = tf.optimizers.RMSprop(1e-3);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From my trying out, these other combinations also work for single output neuron with my data (Adam, Adamax, Nadam, RMSprop work when learning_rate=1e-3 instead of 1e-1):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                         Adadelta  Adagrad  Adam  Adamax  Ftrl  Nadam  RMSprop  SGD&#xA;BinaryCrossentropy       Yes       Yes      --    --      --    --     --       Yes&#xA;CategoricalCrossentropy  Yes       --       --    --      --    --     --       --&#xA;CategoricalHinge         --        --       --    --      --    --     --       --&#xA;CosineSimilarity         --        --       --    --      --    --     --       --&#xA;Hinge                    Yes       Yes      --    --      --    --     --       Yes&#xA;Huber                    Yes       Yes      --    --      --    --     --       Yes&#xA;LogCosh                  Yes       Yes      --    --      --    --     --       Yes&#xA;Poisson                  Yes       Yes      --    --      --    --     --       Yes&#xA;SquaredHinge             Yes       Yes      --    --      --    --     --       Yes&#xA;&#xA;KLD: lambda a,b: KLD(a,b)&#xA;MAE,MAPE,MSE,MSLE: lambda a,b: Mxxx(a,b)&#xA;The above lambdas are direct functions, not classes like SGD, Adadelta, etc.&#xA;SparseCategorialCrossentropy: Seems not working with single output neuron.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-06T08:39:33.427" LastActivityDate="2019-10-06T08:39:33.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15750" PostTypeId="2" ParentId="15743" CreationDate="2019-10-06T10:05:47.720" Score="3" Body="&lt;p&gt;As explained in this &lt;a href=&quot;https://arxiv.org/pdf/1603.05027.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; , the major benefit of identity mapping is that it enables backpropagation signal to reach  from output (last) layers to input (first) layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see on the paper at section 2 that it resolves vanishing gradient problem which arises in deeper networks. &lt;/p&gt;&#xA;" OwnerUserId="19102" LastActivityDate="2019-10-06T10:05:47.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15751" PostTypeId="1" AcceptedAnswerId="15773" CreationDate="2019-10-06T13:42:59.847" Score="2" ViewCount="64" Body="&lt;p&gt;Why is awareness of itself such a point when speaking about AI? Does it always mean a starting point for apocalyptic nightmares to occur when such a level is reached or is it just a classical example about what could be a really abstract thing that machine cannot easily posses?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would sleep my nights far more calmfully if the situation was the latter, and I understand the first does not automatically  happen. The main thing I would like to discover is the starting point - which approach came first historically? Or is there another view point in the historical first occurrence of self awareness term?&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2019-10-07T16:14:28.910" Title="Why is awareness of itself such a point when speaking about AI?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;history&gt;&lt;self-awareness&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15752" PostTypeId="1" CreationDate="2019-10-06T15:27:10.937" Score="9" ViewCount="171" Body="&lt;p&gt;Recently, according to some reports Google achieved something called &lt;a href=&quot;https://www.wired.com/story/googles-quantum-supremacy-isnt-end-encryption/&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Quantum Supremacy'&lt;/a&gt;. Whether its true or not remains to be seen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my question is does Quantum Computers or the principle they work on has any effect on AI. We know that according to &lt;a href=&quot;https://en.wikipedia.org/wiki/Shor%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shor's Algorithm&lt;/a&gt; Quantum Computing can reduce the complexity of breaking an encryption from &lt;span class=&quot;math-container&quot;&gt;$O(2^n)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$O(2^{\frac{n}{2}})$&lt;/span&gt; (not sure). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So similar to this, will Quantum computing have any kind of effect in accelerating AI training or something similar?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2019-10-10T02:49:08.753" LastActivityDate="2019-11-19T14:02:55.423" Title="Will quantum computing have any kind of effect on the development of AI?" Tags="&lt;training&gt;&lt;hardware&gt;&lt;hardware-evaluation&gt;&lt;quantum-computing&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ClosedDate="2020-05-13T21:21:54.500" ContentLicense="CC BY-SA 4.0" />
  <row Id="15753" PostTypeId="2" ParentId="15730" CreationDate="2019-10-06T16:19:57.863" Score="18" Body="&lt;p&gt;I think your premise is flawed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to assume that to &quot;understand&quot;(*) infinities requires infinite processing capacity, and imply that humans have just that, since you present them as the opposite to limited, finite computers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But humans &lt;em&gt;also&lt;/em&gt; have finite processing capacity. We are beings built of a finite number of elementary particles, forming a finite number of atoms, forming a finite number of nerve cells. If we can, in one way or another, &quot;understand&quot; infinities, then surely finite computers can also be built that can. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(* I used &quot;understand&quot; in quotes, because I don't want to go into e.g. the definition of sentience etc. I also don't think it matters in regarding this question.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As a human being, we can think infinity. In principle, if we have enough resources (time etc.), we can count infinitely many things (including abstract, like numbers, or real).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here, you actually say it out loud. &quot;With enough resources.&quot; Would the same not apply to computers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While humans &lt;em&gt;can&lt;/em&gt;, e.g. use infinities when calculating limits etc. and can think of the idea of something getting arbitrarily larger, we can only do it in the abstract, not in the sense being able to process arbitrarily large numbers. The same rules we use for mathematics could also be taught to a computer.&lt;/p&gt;&#xA;" OwnerUserId="30205" LastActivityDate="2019-10-06T16:19:57.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15754" PostTypeId="1" CreationDate="2019-10-06T17:09:15.833" Score="3" ViewCount="135" Body="&lt;p&gt;Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed. More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?&lt;/p&gt;&#xA;" OwnerUserId="30206" LastEditorUserId="1671" LastEditDate="2019-10-07T02:07:22.547" LastActivityDate="2019-10-07T02:07:22.547" Title="Is there a way to do reinforcement learning in POMDP?" Tags="&lt;reinforcement-learning&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15756" PostTypeId="2" ParentId="15730" CreationDate="2019-10-06T19:38:59.697" Score="8" Body="&lt;p&gt;In Haskell, you can type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;print [1..]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it will print out the infinite sequence of numbers, starting with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It will do this until your console runs out of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's try something more interesting.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double x = x * 2&#xA;print (map double [1..])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here's the start of the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128,130,132,134,136,138,140,142,144,146,148,150,152,154,156,158,160,162,164,166,168,170,172,174,176,178,180,182,184,186,188,190,192,194,196,198,200,202,204,206,208,210,212,214,216,218,220,222,224,226,228,230,232,234,236,238,240,242,244,246,248,250,252,254,256,258,260,262,264,266,268,270,272,274,276,278,280,282,284,286,288,290,292,294,296,298,300,302,304,306,308,310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344,346,348,350,352,354,356,358,360,362,364,366,368,370,372,374,376,378,380,382,384,386,388,390,392&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These examples show infinite computation. In fact, you can keep infinite data structures in Haskell, because Haskell has the notion of &lt;em&gt;non-strictness&lt;/em&gt;-- you can do computation on entities that haven't been fully computed yet. In other words, you don't have to fully compute an infinite entity to manipulate that entity in Haskell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reductio ad absurdum.&lt;/p&gt;&#xA;" OwnerUserId="1916" LastActivityDate="2019-10-06T19:38:59.697" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15758" PostTypeId="2" ParentId="15730" CreationDate="2019-10-07T01:23:07.133" Score="8" Body="&lt;p&gt;I believe humans can be said to understand infinity since at least &lt;a href=&quot;https://en.wikipedia.org/wiki/Georg_Cantor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Georg Cantor&lt;/a&gt; because we can recognize different &lt;a href=&quot;https://en.wikipedia.org/wiki/Aleph_number#Aleph-one&quot; rel=&quot;nofollow noreferrer&quot;&gt;types of infinites&lt;/a&gt; (chiefly countable vs. uncountable) via the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Cardinal_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;cardinality&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, a set is countably infinite if it can be mapped to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural numbers&lt;/a&gt;, which is to say there is a 1-to-1 correspondence between the elements of countably infinite sets.  The set of all reals is uncountable, as is the set of all combinations of natural numbers, because there will always be more combinations than natural numbers where n&gt;2, resulting in a set with a greater cardinality.  &lt;em&gt;(The first formal proofs for uncountability can be found in Cantor, and is subject of &lt;a href=&quot;https://en.wikipedia.org/wiki/Philosophy_of_mathematics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Philosophy of Math&lt;/a&gt;.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding of infinity involves logic as opposed to arithmetic because we can't express, for instance, all of the decimals of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Transcendental_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;transcendental number&lt;/a&gt;, only use approximations. Logic is a fundamental capability of what we think of as computers.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An analytic process (AI) that can recognize a function that produces an infinite loop, such as using &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to draw a circle, might be said to understand infinity...  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&quot;Never ending&quot; is a definition of infinity, with the set of natural numbers as an example (there is a least number, 1, but no greatest number.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Intractability vs. Infinity&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outside of the special case of infinite loops, I have to wonder if an AI is more oriented on &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational intractability&lt;/a&gt; as opposed to infinity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A problem is said to be intractable if there is not enough time and space to completely represent it, and this can be extended to many real numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; may be understood to be infinite because it arises from/produces a circle, but I'm not sure this is the case with all real numbers with an intractable number of decimals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would the AI assume such a number were infinite or merely intractable?  The latter case is concrete as opposed to abstract--either it can finish the computation or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads to the &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;halting problem&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Turing's proof that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist could be taken as an indication that an algorithm based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing-Church model of computation&lt;/a&gt; cannot have a perfect understanding of infinity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If an alternate computational model arose that could solve the halting problem, it might be argued that an algorithm could have a perfect understanding, or at least demonstrate an understanding comparable to humans.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-16T21:08:08.437" LastActivityDate="2019-10-16T21:08:08.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15759" PostTypeId="1" CreationDate="2019-10-07T02:31:42.040" Score="1" ViewCount="82" Body="&lt;p&gt;Are there any machine learning or deep learning techniques to use the content of an image for another image?&lt;/p&gt;&#xA;&lt;p&gt;More specifically, suppose I take a photo of a notebook. I get the angle, lighting, and perspective perfect. Now I copy an image I found online that contains text or handwriting. Is there a machine learning technique that would now draw this writing in my notebook?&lt;/p&gt;&#xA;&lt;p&gt;Just asking if it's possible before I attempt to hire someone.&lt;/p&gt;&#xA;" OwnerUserId="30219" LastEditorUserId="2444" LastEditDate="2020-10-31T15:52:57.603" LastActivityDate="2020-10-31T15:52:57.603" Title="Are there any deep learning techniques to use the content of an image for another image?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;reference-request&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15760" PostTypeId="2" ParentId="15712" CreationDate="2019-10-07T04:49:30.883" Score="0" Body="&lt;p&gt;This doesn't necessarily answer the question, but it does give some possible solutions to mitigate the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, the emphasised looping behaviour above is a result of improper initialisation, in which I was initialising with only positive and 0 weights. The looping behavior is largely diminished by proper initialisation, however it is not totally removed. See below for examples:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;djbgnuywjkfrrkrkrpueeeeeffrrdv&#xA;kmkkkkkkkkkkkrkkkkkkkkkkkkkkkk&#xA;clfwtpjbsqeeeeeeesssjjeeeeeeee&#xA;ldwgmbvcmmbvcmmmgmmmgmmmgmmmmg&#xA;ywzrfotntntnttntntunuwqzffotnt&#xA;lvvurrrrrrxafllvvurrrrrrxcfhla&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(These are samples from 6 different uniquely initialised LSTM's on the exact same architecture).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also found that this common behaviour is typically combated by adding a certain degree of randomness in feeding in the LSTM's output. This is normally done by interpreting the output at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; of the LSTM as a probability distribution, used to pick the next value fed into the LSTM at &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;. This helps break non-confident looping behaviour which is where looping behaviour most commonly occurs through some quick experimentation, while still mostly retaining confident predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tested this by randomising the input a little bit, using a probability distribution where 90% of the time the correct input is selected and 10% a random one is, and back-propagating as normal. This didn't seem to have much of an effect on the looping behaviour, although perhaps might work as a good form of regularisation. I am yet to test it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use the &lt;em&gt;temperature&lt;/em&gt; method in LSTMs, which explained really well here: &lt;a href=&quot;https://cs.stackexchange.com/q/79241/20691&quot;&gt;https://cs.stackexchange.com/q/79241/20691&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2019-10-08T00:38:40.043" LastActivityDate="2019-10-08T00:38:40.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15761" PostTypeId="1" CreationDate="2019-10-07T05:15:32.387" Score="3" ViewCount="126" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1502.05477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRPO paper&lt;/a&gt;, the objective to maximize is (equation 14)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{E}_{s\sim\rho_{\theta_\text{old}},a\sim q}\left[\frac{\pi_\theta(a|s)}{q(a|s)} Q_{\theta_\text{old}}(s,a) \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which involves an expectation over states sampled with some density &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, itself defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\rho_\pi(s) = P(s_0 = s)+\gamma P(s_1=s) + \gamma^2 P(s_2=s) + \dots&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to suggest that later timesteps should be sampled less often than earlier timesteps, or equivalently sampling states uniformly in trajectories but adding an importance sampling term &lt;span class=&quot;math-container&quot;&gt;$\gamma^t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the usual implementations simply use batches made of truncated or concatenated trajectories, without any reference to the location of the timesteps in the trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is similar to what can be seen in the &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO paper&lt;/a&gt;, which transforms the above objective into (equation 3)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{E}_t \left[ \frac{\pi_\theta(a_t|s_t)}{\pi_{\theta_\text{old}}(a_t|s_t)} \hat A_t \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that something is missing in going from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{s\sim \rho}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_t$&lt;/span&gt; in the discounted setting.&#xA;Are they really equivalent?&lt;/p&gt;&#xA;" OwnerUserId="30223" LastEditorUserId="2444" LastEditDate="2020-06-21T13:21:28.270" LastActivityDate="2020-06-21T13:21:28.270" Title="Are these two TRPO objective functions equivalent?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15762" PostTypeId="2" ParentId="15730" CreationDate="2019-10-07T06:30:09.427" Score="7" Body="&lt;p&gt;(There's a summary at the bottom for those who are too lazy or pressed for time to read the whole thing.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately to answer this question I will mainly be deconstructing the various premises.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As I mentioned before, humans understand infinity because they are capable, at least, counting infinite integers, in principle.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I disagree with the premise that humans would actually be able to count to infinity. To do so, said human would need an infinite amount of time, an infinite amount of memory (like a Turing machine) and most importantly an infinite amount of patience - in my experience most humans get bored before they even count to 1,000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the problem with this premise is that infinity is actually not a number, it's a concept that expresses an unlimited amount of 'things'. Said 'things' can be anything: integers, seconds, lolcats, the important point is the fact that those things are not finite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this relevant SE question for more details:&#xA;&lt;a href=&quot;https://math.stackexchange.com/questions/260876/what-exactly-is-infinity&quot;&gt;https://math.stackexchange.com/questions/260876/what-exactly-is-infinity&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put it another way: if I asked you &quot;what number comes before infinity?&quot; what would your answer be? This hypothetical super-human would have to count to that number before they could count infinity. And they'd need to know the number before that first, and the one before that, and the one before that...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully this demonstrates why the human would not be able to actually count to infinity - because infinity does not exist at the end of the number line, it is the concept that explains the number line has no end. Neither man nor machine can actually count up to it, even with infinite time and infinite memory.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, If a computer can differentiate 10 different numbers or things, it means that it really understand these different things somehow.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Being able to 'differentiate' between 10 different things doesn't imply the understanding of those 10 things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A well-known thought experiment that questions the idea of what it means to 'understand' is John Searle's &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;noreferrer&quot;&gt;Chinese Room&lt;/a&gt; experiment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The thing to take away from this experiment is that the ability to process symbols does not imply that one actually understands those symbols. Many computers process natural languages every day in the form of text (characters encoded as integers, typically in a unicode-based encoding like UTF-8), but they do not neccessarily understand those languages. On a simpler Effectively all computers are able to add two numbers together, but they do no necessarily understand what they are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, even in the 'deep learning vision model' the computer arguably does not understand the numbers (or 'symbols') it is being shown, it is merely the algorithm's ability to simulate intelligence that allows it to be classed as artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, we can take a deep learning vision model that recognizes numbers on the card. This model must assign a number to each different card to differentiate each integer. Since there exist infinite numbers of integer, how can the model assign different number to each integer, like a human being, on the digital computers? If it cannot differentiate infinite things, how does it understand infinity?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you were to perform the same card test on a human, and continually increased the number of cards used, eventually a human wouldn't be able to keep track of them all due to lack of memory. A computer would experience the same problem, but could theoretically outperform the human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I ask you, can a human really differentiate infinite things? Personally I suspect the answer is no, because all humans have limited memory, and yet I would agree that humans most likely can understand infinity to some degree (some can do so better than others).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, I think the question &quot;If it cannot differentiate infinite things, how does it understand infinity?&quot; has a flawed premise - being able to differentiate infinite things is not a prerequisite for understanding the concept of infinity.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially your question hinges on what it means to 'understand' something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computers can certainly &lt;em&gt;represent&lt;/em&gt; infinity, the IEEE floating point specification defines both positive and negative infinity, and all modern processors are capable of processing floating points (either in hardware or through software).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If AIs are ever capable of actually understanding things then theoretically they might be able to understand the concept of infinity, but we're a long way off being able to definitively prove this either way, and we'd have to come to a consensus about what it means to 'understand' something first.&lt;/p&gt;&#xA;" OwnerUserId="16369" LastActivityDate="2019-10-07T06:30:09.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15764" PostTypeId="1" CreationDate="2019-10-07T09:16:18.790" Score="1" ViewCount="76" Body="&lt;p&gt;I am using the &lt;code&gt;SpaceInvaders-ram-v0&lt;/code&gt; from OpenAI gym. I want to extract the game's current score using the RAM values. How do I get it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried doing some research in google, couldn't find anything that explains how data stored in the ROM.&lt;/p&gt;&#xA;" OwnerUserId="30230" LastActivityDate="2019-10-07T09:16:18.790" Title="Getting score values from openai gym rom" Tags="&lt;game-ai&gt;&lt;open-ai&gt;&lt;gym&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15767" PostTypeId="2" ParentId="6678" CreationDate="2019-10-07T12:13:43.483" Score="0" Body="&lt;p&gt;I think your intuition is not correct completely. At the beginning of the training, the discriminator could overshoot. Therefore, It can cause escalation of backpropagation signal.&lt;/p&gt;&#xA;" OwnerUserId="19102" LastActivityDate="2019-10-07T12:13:43.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15768" PostTypeId="2" ParentId="15730" CreationDate="2019-10-07T12:22:22.187" Score="2" Body="&lt;p&gt;Computers don't understand &quot;infinity&quot; or even &quot;zero&quot;, just like a screwdriver does not understand screws.  It is a tool made for processing binary signals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, a computer's equivalent in wetware is not a person but a brain.  Brains don't think, persons do.  The brain is just the platform persons are implemented with.  It's a somewhat common mistake to conflate the two since their connection tends to be rather inseparable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wanted to assign understanding, you'd at least have to move to actual programs instead of computers.  Programs may or may not have representations for zero or infinity, and may or may not be able to do skillful manipulations of either.  Most symbolic math programs fare mostly better here than someone required to work with math as part of their job.&lt;/p&gt;&#xA;" OwnerDisplayName="user30242" LastActivityDate="2019-10-07T12:22:22.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15772" PostTypeId="1" CreationDate="2019-10-07T14:37:34.157" Score="0" ViewCount="94" Body="&lt;p&gt;Currently I have a setup where I'm determining the position of a transmitter using the RSSI of 4 receivers. Its a simple feed-forward network with some hidden layers, where the input is the RSSI values, and the output is a 2d coordinate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I decide to add/remove receivers, I have to train the network again, since the input size changes. This is not ideal, since the receivers can move around, dissapear, etc. I have looked at some alternatives, but being pretty new to machine learning, it's difficult to pick which direction to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked at a potential solution (stolen from another question), but I'm lost at how to implement it using tensorflow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x4cCi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x4cCi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30250" LastActivityDate="2021-03-03T10:01:10.120" Title="Indoor positioning with variable number of distance measurements in tensorflow" Tags="&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15773" PostTypeId="2" ParentId="15751" CreationDate="2019-10-07T16:14:28.910" Score="0" Body="&lt;p&gt;Neil Slater has it right, there most probably is no fear of AI self awareness as some starting point of evil series of things to happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia [1] puts self awareness talks to sci-fi section, among stories, not a real thing. Self-awareness is among a list of terms that tries to make machines or aliens similarly human than ordinary people and uses that as a method of story telling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-awareness or other humanlike skills that we posses that the machines don't have yet and will not have in near future can twist minds and make a seed of conspiracy theories, but at least the exhaustive Wikipedia overview on topic did not speak anything about AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the concept of humanlike behaviour materialises on our minds as a term of self-awareness but my source puts the origin to different category.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Self-awareness&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/Self-awareness&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2019-10-07T16:14:28.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15776" PostTypeId="1" AcceptedAnswerId="15782" CreationDate="2019-10-07T17:58:08.840" Score="3" ViewCount="104" Body="&lt;p&gt;The &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room argument&lt;/a&gt; &lt;em&gt;against&lt;/em&gt; strong AI overlooks the fact that &quot;the man in the room&quot; is acting as a macro-scale &quot;neurotransmitter&quot; of the larger system in which he resides. It does not rule out strong AI, it simply reduces to an enigmatic question: &lt;em&gt;where does understanding &quot;reside&quot; and how does it epiphenomenally emerge?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are other examples of thought experiments against or in favor of strong AI (apart from the Chinese room argument) or extensions or refutations to known experiments?&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="2444" LastEditDate="2021-01-22T00:38:53.360" LastActivityDate="2021-01-22T00:38:53.360" Title="What are examples of thought experiments against or in favour of strong AI, apart from the Chinese room argument?" Tags="&lt;reference-request&gt;&lt;philosophy&gt;&lt;agi&gt;&lt;chinese-room-argument&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15779" PostTypeId="2" ParentId="15730" CreationDate="2019-10-07T18:38:29.600" Score="1" Body="&lt;p&gt;I would think that a computer couldn’t understand infinity primarily because the systems and parts of a system, that are driving the computer are finite themselves.&lt;/p&gt;&#xA;" OwnerUserId="30261" LastActivityDate="2019-10-07T18:38:29.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15781" PostTypeId="2" ParentId="3903" CreationDate="2019-10-07T20:10:57.187" Score="1" Body="&lt;p&gt;I've spent some time thinking about this in the context of games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with reward functions is that they generally involve weighting nodes, which is useful but ultimately materially meaningless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are two materially meaningful rewards:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;COMPUTATIONAL RESOURCES&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a game where an AI is competing not for points, but for processor time and memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The better the algorithm performs at the game, the more memory and processing it has access to.  This has a practical effect--the more resources available to the automata, the stronger its capabilities.  (i.e. it's &lt;a href=&quot;https://en.wikipedia.org/wiki/Bounded_rationality&quot; rel=&quot;nofollow noreferrer&quot;&gt;rationality is less bounded&lt;/a&gt; in terms of time and space to make a decision.)  Thus the algorithm would be &quot;motivated&quot; to prevail such a contest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ENERGY&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any automata with a sufficient degree of &quot;self awareness&quot;, here specifically referring to the knowledge that it requires energy to process, would be motivated to self-optimize its own code to eliminate unnecessary flipping of bits (unnecessary energy consumption.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such an algorithm would also be motivated to ensure its power supply so that it can continue to function.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-10-07T20:10:57.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15782" PostTypeId="2" ParentId="15776" CreationDate="2019-10-07T20:40:14.017" Score="3" Body="&lt;p&gt;An excellent book summarizing the development of thought in this area over several hundred years is &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0262581531&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mind Design II&lt;/a&gt;, edited by John Haugeland. This book contains a collection of essays written by the major thinkers in this area through until about the 1990s. A brief summary of some of the major ideas is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Descartes: Minds are spirits, brains are bodies. Minds attach to brains in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Corpus_callosum&quot; rel=&quot;nofollow noreferrer&quot;&gt;special part of the brain&lt;/a&gt;. Descartes has a number of thought experiments, but they mostly have a Theological root. Most modern scientists do not find these to be satisfying because the resolve metaphysical questions about the mind by assuming you already believe in a particular interpretation of a God. &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gilbert_Ryle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gilbert Ryle&lt;/a&gt; argues in the late 1940s that concepts like the Mind are in some sense vacuous. Asking whether a machine has a &lt;em&gt;mind&lt;/em&gt; or not is like asking whether a field contains two cows or a pair of cows: this is a conceptual distinction that has no relation to the real world. Because of this, its study is futile. The parallel movement of &lt;a href=&quot;https://en.wikipedia.org/wiki/Behaviorism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Behaviorism&lt;/a&gt; in psychology also viewed the study of minds and mental states as a fool's errand. Rather, this movement focused on studying behaviors that could be directly measured.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Turing publishes his &lt;a href=&quot;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;seminal paper&lt;/a&gt; in AI, in 1950. In this paper, Turing anticipates nearly all later objections to AI, and summarizes reasonable refutations of them. This paper contains a number of short thought experiments. One of the more notable ones is the Turing Test. A key observation Turing makes is that if a machine behaves in a way indistinguishable from a human, then any arguments used to reject its claim to intelligence (and even to &lt;em&gt;consciousness&lt;/em&gt;) appear to work equally well on other humans. Searle doesn't really address this in his later argument. Basically, I have no way to know for sure whether you have subjective experiences of the same kind I have. The fact that you're made of roughly the same kind of meat as I am seems like a shaky explanation. For example, would humanoid aliens that act like us also be intelligent? What if it were found that their brains took some radically different form from ours? What if it were found that their brains were really networks of transistor-like machines?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Cognitivism_(psychology)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cognativist revolution&lt;/a&gt; spanned a number of fields starting in the 1960s. This movement held that behaviorism was wrong for two basic reasons. First, Behaviorism does not account for the subjective feelings of existence or understanding. Second, Behaviorism could not explain phenomena like language, that appeared to involve reasoning logically about symbols. Notable authors in this period are &lt;a href=&quot;https://en.wikipedia.org/wiki/Noam_Chomsky&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chomsky&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Jerry_Fodor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fodor&lt;/a&gt;, who argued that minds were essentially computer programs that happened to be running on brains instead of computers. These ideas dominated AI, Psychology and Linguistics for about 30-40 years. Fodor's works contain a number of thought experiments involving language and programs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;John Searle's &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room argument&lt;/a&gt; caused problems for the cognativists, because it presents a convincing chain of logic showing that a program that does complicated tasks (like linguistics) does not subjectively understand what it is doing. Various objections are raised and Searle responds to them in his publication of the argument. Objections to Searle's arguments form the basis of several later movements in philosophy of AI. &lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://www.iep.utm.edu/connect/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Connectionists&lt;/a&gt; argue that we can represent the behavior of the brain (and thus, the mind) with a brain-like program. This paralleled increasing interest in artificial neural networks. An example of a cognativist thought experiment is to imagine a simulation of an entire human brain, conducted with perfect fidelity (for instance, we accurately model all the quantum mechanical interactions). If such a machine were not conscious, opponents need to answer the question of where consciousness is hiding (Is it in the meat? Then see Turing. Is it in some yet-undiscovered property of neurons? Then the burden of proof is definitely on the person proposing such a hidden property, and they must explain why we cannot simulate it). &lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Paul_Churchland&quot; rel=&quot;nofollow noreferrer&quot;&gt;Churchlands&lt;/a&gt; and others argued, much as Ryle, that minds as the public understand them are a sort of &quot;folk theory&quot;, that appear to explain something about reality to lay people, but are actually a dead end. Concepts like &quot;beliefs&quot; and &quot;experiences&quot; are, according to these arguments, akin to aether or phlogiston: things that our current, malformed, understandings of neuroscience seem to require, but that in fact have no basis in reality, even though nearly everyone believes in them at present. This line of work contains many extensions of the kinds of thought experiments Turing hinted at. &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rodney_Brooks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brooks&lt;/a&gt; and others advance the theory of embodied cognition, in which the mind is not confined to the body, but is actually a property of a body plus the environment it is placed in. An example thought experiment from this school (by Andy Clarke if I recall correctly) is to imagine a person with profound loss of the ability to form new long term memories. This person carries around a notebook. The cover says &quot;Your memory&quot;. The first page contains instructions explaining that the person has lost the ability to remember things, and that they write new things they want to remember in the book. The person can remember things for, say, 15 minutes at a time, and the book is very well organized. Where is this person's mind exactly? Is it in their brain (which does not remember anything 15 minutes in the past), or is it in the brain &lt;em&gt;and&lt;/em&gt; the book, which together can remember and reason about things essentially like a normal person. If it &lt;em&gt;is&lt;/em&gt; in the brain and book, what if we replace the book with a computer inside the person's head? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-07T20:40:14.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15783" PostTypeId="2" ParentId="15730" CreationDate="2019-10-07T20:46:53.950" Score="4" Body="&lt;p&gt;Then premise assumes that humans &quot;understand&quot; infinity. Do we? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you'd need to tell me what criterion you would use, if you wanted to know whether I &quot;understand&quot; infinity, first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the OP, the idea is given that I could &quot;prove&quot; I &quot;understand&quot; infinity, because &lt;em&gt;&quot;In principle, if we have enough resources (time etc.), we can count infinitely many things (including abstract, like numbers, or real).&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, that's simply not true. Worse, if it were true (which it isnt), then it would be equally true for a computer. Here's why:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yes, you can in principle count integers, and see that counting never ends. &lt;/li&gt;&#xA;&lt;li&gt;But even if you had enough resources, you could never &quot;count infinitely many things&quot;. There would always be more. That's what &quot;infinite&quot; means.&lt;/li&gt;&#xA;&lt;li&gt;Worse, there are multiple orders (&quot;cardinalities&quot;) of infinity. Most of them, you can't count, even with infinite time, and perhaps not even with infinite other resources. They are actually uncountable. They literally cannot be mapped to a number line, or to the set of integers. You cannot order them in such a way that they can be counted, even in principle.&lt;/li&gt;&#xA;&lt;li&gt;Even worse, how do you do that bit where you decide &quot;in principle&quot; what I can do, when I clearly can't ever do it, or even the tiniest part of it? That step feels layman-style assumptive, not actually seeing the issues in doing it rigorously. It may not be trivial.&lt;/li&gt;&#xA;&lt;li&gt;Last, suppose this was your actual test, like in the OP. So if I could &quot;in principle with enough resources (time etc) count infinitely many things&quot;, it would be enough for you to decide I &quot;understood&quot; infinity (whatever that means). Then so could a computer with sufficient resources (RAM, time, algorithm). So the test itself would be satisfied trivially by a computer if you gave the computer the same criteria.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think maybe a more realistic line of logic is that what this question actually shows, is that most (probably all?) humans actually do &lt;strong&gt;not&lt;/strong&gt; understand infinity. So understanding infinity is probably &lt;strong&gt;not&lt;/strong&gt; a good choice of test/requirement for AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you doubt this, ask yourself. Do you honestly, truly, and seriously, &quot;understand&quot; a hundred trillion years (the possible life of a red dwarf star)? Like, can you really comprehend what its like, experiencing a hundred trillion years, or is it just a 1 with lots of zeros? What about a femtosecond? Or a time interval of about 10^-42 seconds? Can you truly &quot;understand&quot; that? A timescale compared to which, one of your heartbeats, compares like one of your heartbeats compares to a billion billion times the present life of this universe? Can you &lt;em&gt;really&lt;/em&gt; &quot;understand infinity&quot;, yourself? Worth thinking about......&lt;/p&gt;&#xA;" OwnerUserId="5817" LastEditorUserId="5817" LastEditDate="2019-10-07T20:55:13.250" LastActivityDate="2019-10-07T20:55:13.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15784" PostTypeId="1" AcceptedAnswerId="15786" CreationDate="2019-10-07T21:53:19.197" Score="2" ViewCount="131" Body="&lt;p&gt;I would like to work on a project where I teach an NN to play N64 games. &#xA;To my current understanding, I would need an emulator?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do the Machine Learning side of it, im just unsure how I can give the NN access to the game's controls, such as left, right, up or down?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where could I find more information on doing so and is using an Emulator the right path to take?&lt;/p&gt;&#xA;" OwnerUserId="30270" LastActivityDate="2019-10-08T00:28:24.460" Title="How can I teach a computer to play N64 games using Neural Nets?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15785" PostTypeId="2" ParentId="8031" CreationDate="2019-10-07T21:59:25.943" Score="0" Body="&lt;p&gt;I see that the objective is to achieve multiple goals using A* algorithm. &#xA;So if your problem is more like a Traveling Salesman Problem, which is what it kind of sounds like, you can refer to this post:&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/4453477/using-a-to-solve-travelling-salesman&quot;&gt;https://stackoverflow.com/questions/4453477/using-a-to-solve-travelling-salesman&lt;/a&gt;&#xA;The problem can be converted to a graph search problem, and could utilize Minimum Spanning Tree. A-star can be used to compute edge weight. &lt;/p&gt;&#xA;" OwnerUserId="30269" LastActivityDate="2019-10-07T21:59:25.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15786" PostTypeId="2" ParentId="15784" CreationDate="2019-10-08T00:28:24.460" Score="3" Body="&lt;p&gt;The common options are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Implement a simple simulation of the game, perhaps without graphics. Train the agent on the simulation. This is usually not very satisfying, because the agent won't play the &lt;em&gt;real&lt;/em&gt; game.&lt;/li&gt;&#xA;&lt;li&gt;Use an emulator that allows you to inspect the game's memory, and input values directly at the controls. &lt;a href=&quot;http://tasvideos.org/BizHawk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BizHawk&lt;/a&gt; is a good choice for SNES and I think also for N64 emulation. Use a &lt;a href=&quot;https://www.geeksforgeeks.org/pipe-system-call/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipe&lt;/a&gt; to connect this program with your agent. Pass across the pipe any key values from the game's memory that represent the state of the game, and send back actions that represent the inputs the agent makes in that state. Usually this is the best option.&lt;/li&gt;&#xA;&lt;li&gt;Scrape the screen in real time. Tools like &lt;a href=&quot;https://github.com/muldjord/skyscraper&quot; rel=&quot;nofollow noreferrer&quot;&gt;SkyScraper&lt;/a&gt; will allow you to capture the raw pixels displayed in a window. You should then be able to pass these pixels to your agents via a pipe. You may be able to use tools like &lt;a href=&quot;https://www.seleniumhq.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Selenium&lt;/a&gt; to pass values back from the agent to the game. This route is usually a lot more intensive than option 2, but is the most realistic. It may also require very good hardware to keep up with the game in real time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-08T00:28:24.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15787" PostTypeId="2" ParentId="15730" CreationDate="2019-10-08T01:14:40.687" Score="1" Body="&lt;p&gt;The &quot;concept&quot; of infinity is 1 thing to understand.  I can represent it with 1 symbol (∞).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As I mentioned before, humans understand infinity because they are&#xA;  capable, at least, counting infinite integers, in principle.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By this definition humans do not understand infinity.  Humans are not capable of counting infinite integers.  They will die (run out of compute resources / power) at some time.  It would probably be easier in fact to get a computer to count towards infinity than it would be to get a human to do so.&lt;/p&gt;&#xA;" OwnerUserId="30272" LastActivityDate="2019-10-08T01:14:40.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15790" PostTypeId="1" CreationDate="2019-10-08T07:11:44.910" Score="2" ViewCount="139" Body="&lt;p&gt;What is the difference between a semantic network and an ontology? How are they related? I have not found any article that describes how these two concepts are related.&lt;/p&gt;&#xA;" OwnerUserId="30283" LastEditorUserId="2444" LastEditDate="2021-02-07T22:52:48.307" LastActivityDate="2021-02-07T22:52:48.307" Title="What is the difference between a semantic network and an ontology?" Tags="&lt;comparison&gt;&lt;knowledge-representation&gt;&lt;ontology&gt;&lt;knowledge-base&gt;&lt;semantic-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15791" PostTypeId="2" ParentId="15790" CreationDate="2019-10-08T08:24:06.793" Score="2" Body="&lt;p&gt;A semantic network is a way to implement an ontology. An ontology is just a generalised way of representing knowledge in a particular domain, and there are multiple ways of doing so. The key that distinguishes an ontology from, say, Wikipedia, is that it is formally defined, so that the knowledge represented can be used in programs to reason with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semantic networks can be used to do that. Please note that there are many types of semantic networks, so you can use them to represent a wide range of relevant information. Since the point of an ontology is to show the relationships between entities relevant to a domain, they are usually represented as networks &amp;mdash; if the information is stored as RDF triples, these can usually be visualised as an equivalent network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;em&gt;ontology&lt;/em&gt; is the broader, more general term, whereas a &lt;em&gt;semantic network&lt;/em&gt; is a more specific way of representing information.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-10-08T08:24:06.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15792" PostTypeId="1" CreationDate="2019-10-08T09:00:41.550" Score="0" ViewCount="167" Body="&lt;p&gt;I built a three-layer neural network (first is 1D convolutional and the remaining two are linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively the probability of failure or success. The NN is trained in a simulation.&lt;/p&gt;&#xA;&lt;p&gt;The simulation goes this way: it takes 5 angles in radians and calculates the vector sum of 5 vectors having &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; as module and &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; as angles (taken from the input). It returns &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; if the vector sum has a module greater than &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if it is less than &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My intention is to be able to tell sequences of radians that will generate vectors with a sum greater than &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in module from the ones which won't.&lt;/p&gt;&#xA;&lt;p&gt;Which would be the best configuration to achieve this? Is the configuration I set up (1D convolution layer + 2 linear layers) efficient? If so, would it be easy to find the right size for the convolution? Or should I just remove it?&lt;/p&gt;&#xA;&lt;p&gt;I noticed that if I change the order of the input angles the output of the simulation will be the same. Is there a particular configuration you should use when dealing with these cases?&lt;/p&gt;&#xA;" OwnerUserId="30287" LastEditorUserId="2444" LastEditDate="2020-07-05T15:10:12.740" LastActivityDate="2021-04-01T19:05:30.520" Title="What is the best neural network architecture for this problem?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15797" PostTypeId="1" CreationDate="2019-10-08T16:20:14.627" Score="2" ViewCount="14" Body="&lt;p&gt;The following figure is from the last page in &lt;a href=&quot;https://pjreddie.com/media/files/papers/YOLOv3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLOv3 paper&lt;/a&gt; highlighting how mAP is unfair metric for evaluating Object Detectors:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ewN5u.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ewN5u.jpg&quot; alt=&quot;YOLOv3 last page&quot;&gt;&lt;/a&gt;&#xA;The figure shows two hypothetical Object Detector results which the author say they give the same perfect mAP, while visually the first detector is clearly more accurate than the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to my understanding, the two detectors do not give the same mAP. This is how I calculate it for each detector:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Detector 1, 'Dog' class AP table:&#xA; ______________________________________ &#xA;| Object  | True? | Precision | Recall |&#xA;|_________|_______|___________|________|&#xA;| Dog_99% | Yes   |     1     |    1   |  &#xA;|_________|_______|___________|________|&#xA;Hence, AP_dog = 1&#xA;&#xA;Detector 1, 'Person' class AP table:&#xA; ________________________________________&#xA;| Object    | True? | Precision | Recall |&#xA;|___________|_______|___________|________|&#xA;|Person_99% | Yes   |     1     |    1   |  &#xA;|___________|_______|___________|________|&#xA;Hence, AP_person = 1&#xA;And by continuing doing so for the other 7 classes in the dataset, mAP=1. &#xA;&#xA;Detector 2, 'Dog' class AP table:&#xA; ______________________________________ &#xA;| Object  | True? | Precision | Recall |&#xA;|_________|_______|___________|________|&#xA;| Dog_48% | Yes   |     1     |    1   |  &#xA;|_________|_______|___________|________|&#xA;Hence, AP_dog = 1&#xA;&#xA;Detector 2, 'Bird' class AP table:&#xA; _______________________________________&#xA;| Object   | True? | Precision | Recall |&#xA;|__________|_______|___________|________|&#xA;| Bird_90% | Yes   |     1     |    1   | &#xA;| Bird_89% | No    |     0.5   |    1   | &#xA;|__________|_______|___________|________|&#xA;Hence, AP_bird = 0.75&#xA;And by continuing doing so for the other 7 classes in the dataset, mAP is less than 1 because AP for at least one class is less than one (AP_bird).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hence, according to my understanding mAP for the first detector is 1, and for the second detector is less than 1. What is the mistake I'm doing in the calculation? Or is there some assumption in the paper that I'm not considering?&lt;/p&gt;&#xA;" OwnerUserId="30301" LastActivityDate="2019-10-08T16:20:14.627" Title="How mAP is unfair evaluation metric for Object Detection?" Tags="&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15800" PostTypeId="1" AcceptedAnswerId="15809" CreationDate="2019-10-08T19:48:13.057" Score="3" ViewCount="183" Body="&lt;p&gt;I am recording the vibrations of an AC Motor (50Hz Europe) and I am trying to find out whether it is powered on or not. When I record these vibrations, I basically get the vibration values (&lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;) over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to develop a program to &lt;strong&gt;detect the presence&lt;/strong&gt; of a 50Hz sine wave on a steady stream of input data. I will have &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; measurements, where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; represents amplitude and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; the time (sampled at 100Hz - it is possible to increase the sample rate to 200Hz or 400Hz at max)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is this a task suited for a neural network, and if so, would it be less efficient than other means of detection?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="30307" LastEditorUserId="2444" LastEditDate="2019-11-06T23:08:47.170" LastActivityDate="2019-11-06T23:08:47.170" Title="Can a neural network be used to detect sine waves?" Tags="&lt;neural-networks&gt;&lt;pattern-recognition&gt;&lt;signal-processing&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15801" PostTypeId="2" ParentId="15730" CreationDate="2019-10-08T20:51:54.363" Score="1" Body="&lt;p&gt;Just food for thought: how about if we try to program infinity not in theoretical, but in practical terms? Thus, if we deem something that a computer cannot calculate, given its resources as infinity, it would fulfill the purpose. Programmatically, it can be implemented as follows: if the input is less than available memory it's not infinity. Subsequently, infinity can be defined as something that returns out-of-memory error on an evaluation attempt.&lt;/p&gt;&#xA;" OwnerUserId="3992" LastEditorUserId="2444" LastEditDate="2019-10-09T01:15:50.283" LastActivityDate="2019-10-09T01:15:50.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15802" PostTypeId="1" AcceptedAnswerId="16083" CreationDate="2019-10-08T21:21:06.000" Score="6" ViewCount="395" Body="&lt;p&gt;I'm studying about different selection methods in genetic algorithms. My question is about the Stochastic Universal Sampling (SUS) selection method. I know that each individual will occupy a segment of the line according to its fitness value and then equally spaced pointers will be placed over this line.&lt;/p&gt;&#xA;&lt;p&gt;I want to know how the distance between pointers is determined. I have seen 1/6 and 1/4 as the distance between pointers. I want to choose the number of pointers dynamically according to the situation. I want to know what conditions or factors affect the determination of this distance. For example, when do we decide to choose 1/4 as distance? I want to know if it is possible to change the number of samples in each iteration according to different conditions or situations. If so, what are these conditions?&lt;/p&gt;&#xA;" OwnerUserId="30311" LastEditorUserId="2444" LastEditDate="2021-01-30T22:03:26.880" LastActivityDate="2021-01-30T22:03:26.880" Title="How is the distance between pointers in Stochastic Universal Sampling determined?" Tags="&lt;genetic-algorithms&gt;&lt;selection-operators&gt;&lt;stochastic-universal-sampling&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15803" PostTypeId="2" ParentId="15730" CreationDate="2019-10-08T21:25:32.363" Score="3" Body="&lt;p&gt;I think the concept that is missing in the discussion, so far, is symbolic representation.  We humans represent and &lt;strong&gt;&lt;em&gt;understand&lt;/em&gt;&lt;/strong&gt; many concepts symbolically.  The concept of Infinity is a great example of this.   Pi is another, along with some other well-known irrational numbers.  There are many, many others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it is, we can easily represent and present these values and concepts, both to other humans and to computers, using symbols.  Both computers and humans, can manipulate and reason with these symbols.  For example, computers have been performing mathematical proofs for a few decades now.  Likewise, commercial and/or open source programs are available that can manipulate equations symbolically to solve real world problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as @JohnDoucette has reasoned, there isn't anything that special about Infinity vs many other concepts in math and arithmetic.   When we hit that representational brick wall, we just define a symbol that represents &quot;that&quot; and move forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the concept of infinity has many practical uses.  Any time you have a ratio and the denominator &quot;goes to&quot; zero, the value of the expression &quot;approaches&quot; infinity.   This isn't a rare thing, really.   So, while your average person on the street isn't conversant with these ideas, lots and lots of scientists, engineers, mathematicians and programmers are.   It's common enough that software has been dealing with Infinity symbolically for a couple decades, now, at least.  E.g. Mathematica: &lt;a href=&quot;http://mathworld.wolfram.com/Infinity.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mathworld.wolfram.com/Infinity.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30310" LastEditorUserId="30310" LastEditDate="2019-10-08T21:40:59.250" LastActivityDate="2019-10-08T21:40:59.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15804" PostTypeId="2" ParentId="15792" CreationDate="2019-10-08T23:17:00.987" Score="0" Body="&lt;p&gt;I'm not completely sure I understand your simulation. What I think you are doing is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate 5 angles specified in radians (Are these always normalized to within (0, &lt;span class=&quot;math-container&quot;&gt;$2*\pi$&lt;/span&gt;)?).&lt;/li&gt;&#xA;&lt;li&gt;Interpret each angle as a &lt;strong&gt;unit&lt;/strong&gt; vector in a 2d space.&lt;/li&gt;&#xA;&lt;li&gt;Add the unit vectors together, yielding a vector that lies somewhere inside a circle with radius 5.&lt;/li&gt;&#xA;&lt;li&gt;Ask whether the summed vector is more or less than a distance &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; from the origin of the circle.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you're doing that, your problem looks like trying to learn a separation of two concentric rings, which is a well known benchmarking problem for classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am reasonably certain you can learn this pattern with several layers of ReLU neurons. I'm not certain that convolutional layers will help you much here. The main patterns I'd expect the network to learn are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;perhaps 2-3 layers to learn whether the point lies far away from the origin in each of several different directions.&lt;/li&gt;&#xA;&lt;li&gt;1 layer to learn where the decision boundary is in each of 4 directions away from the origin.&lt;/li&gt;&#xA;&lt;li&gt;1 layer to inclusive-OR the 4 decision boundaries together.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My guess is that this is fairly easy to learn with 4 layers of 8 ReLU neurons, or something like it.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-08T23:17:00.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15806" PostTypeId="2" ParentId="11781" CreationDate="2019-10-09T02:33:26.430" Score="2" Body="&lt;p&gt;I have found some clues in Maei's thesis (2011): &lt;a href=&quot;https://era.library.ualberta.ca/items/fd55edcb-ce47-4f84-84e2-be281d27b16a&quot; rel=&quot;nofollow noreferrer&quot;&gt;“Gradient Temporal-Difference Learning Algorithms.”&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the thesis: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;GTD2 is a method that minimizes the projected Bellman error (MSPBE).&lt;/li&gt;&#xA;&lt;li&gt;GTD2 is convergent in non-linear function approximation case (and off-policy).&lt;/li&gt;&#xA;&lt;li&gt;GTD2 converges to a TD-fixed point (same point as semi-gradient TD).&lt;/li&gt;&#xA;&lt;li&gt;GTD2 is slower to converge than usual semi-gradient TD. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It doesn't readily apply to non-linear function approximation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, it does.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It doesn't yield a good solution. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, it does. TD-fixed point is the same point for the solution of semi-gradient TD (which is generally used). There is no edge on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The only explanation seems to be practical convergence rate.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To quote his words: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Some of our empirical results suggest that gradient-TD method maybe slower than conventional TD methods on problems on which conventional TD methods are sound (that is, on-policy learning problems).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9793" LastActivityDate="2019-10-09T02:33:26.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15807" PostTypeId="2" ParentId="15800" CreationDate="2019-10-09T06:34:07.717" Score="0" Body="&lt;p&gt;You can implement an autoencoder network. Autoencoder is an unsupervised artificial neural network that learns how to efficiently compress and encode data then learns how to reconstruct the data back from the reduced encoded representation to a representation that is as close to the original input as possible. &#xA;When you train autoencoder with 50Hz sine wave data, your model can reconstruct correctly if gets 50Hz sine wave data as an input. When input's &quot;Reconstruction Loss&quot; is less than your threshold value that can be say the given input is 50Hz sine wave.&lt;/p&gt;&#xA;" OwnerUserId="30288" LastActivityDate="2019-10-09T06:34:07.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15808" PostTypeId="1" AcceptedAnswerId="15811" CreationDate="2019-10-09T06:53:07.880" Score="0" ViewCount="39" Body="&lt;p&gt;I'm trying to solve &lt;strong&gt;dLoss/dW1&lt;/strong&gt;. The network is as in picture below with &lt;strong&gt;identity activation&lt;/strong&gt; at all neurons:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lEEKx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lEEKx.png&quot; alt=&quot;3-layer Network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving &lt;strong&gt;dLoss/dW7&lt;/strong&gt; is simple as there's only 1 way to output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Delta = Out-Y$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Loss = abs(Delta)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case when Delta&gt;=0, partial derivative of Loss over W7 is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\dfrac{dLoss}{dW_7} = \dfrac{dLoss}{dOut} \times \dfrac{dOut}{dH_4} \times \dfrac{dH_4}{dW_7} \\&#xA;= \dfrac{d(Out-Y)}{dOut} \times \dfrac{d(H_4W_{13} + H_5W_{14})}{dH_4} \times \dfrac{d(H_1W_7 + H_2W_8 + H_3W_9)}{dW_7} \\&#xA;= 1 \times W_{13} \times H_1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when solving &lt;strong&gt;dLoss/dW1&lt;/strong&gt;, the situation is very different, there are 2 chains to &lt;strong&gt;W1&lt;/strong&gt; through &lt;strong&gt;W7&lt;/strong&gt; and &lt;strong&gt;W10&lt;/strong&gt;, and now, how should the chain for &lt;span class=&quot;math-container&quot;&gt;$\dfrac{dLoss}{dW_1}$&lt;/span&gt; be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, at an arbitrary layer, with all outputs of all layers already calculated plus all gradients of weights on the right side also calculated, what should a formula for &lt;span class=&quot;math-container&quot;&gt;$\dfrac{dLoss}{dW}$&lt;/span&gt; be?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-09T08:07:52.997" LastActivityDate="2019-10-09T09:26:08.010" Title="Backpropagation: Chain Rule to the Third Last Layer" Tags="&lt;math&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;derivative&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15809" PostTypeId="2" ParentId="15800" CreationDate="2019-10-09T06:57:17.813" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Is this a task suited for a neural network&lt;/em&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. You have choices in fact:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A fully-connected network would be simplest architecture, and would work if you gave it some time window of samples (e.g. every 0.5 seconds or every 50 samples) and supervised training data - sets of samples with sensor readings and the ground truth value of whether the motor was on or not.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A 1D convolutional neural network would likely be most efficient and robust to train, and would take the same inputs and outputs as the fully-connected network.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A recurrent neural network would be tricker to train, but a nicer design as you could feed it samples one at a time. The input would be the current sample, and output the probability that the motor was on. When training this, you would also want to provide it transitions between the motor being on and off. The nice feature about this is that it should give you quick feedback about whether the motor was on or off - with the caveat that it may be more likely to trigger intermittent false positives, so a little extra post-processing might be required.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of the above require you to collect training data, ideally in situations identical to planned use of the detector. So if the motor is mounted somewhere that could experience other vibrations, a few of those kind of scenarios should be simulated with motor both on and off.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;and if so, would it be less efficient than other means of detection?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In terms of computing power and effort on your part, you may find that an off-the-shelf Fast Fourier Transform (FFT) library function with a simple threshold at your target frequency will make a robust and simple detector, with no need for a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically for specific frequency detection you would take a window of samples, adjust them (using e.g. &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.19.1/reference/generated/scipy.signal.hamming.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hamming window&lt;/a&gt;) to reduce edge effects which would appear as frequencies in the conversion, and then run FFT. This combination is so common that you may find it already combined in the FFT library. For more on this, you would want to ask in &lt;a href=&quot;https://dsp.stackexchange.com/&quot;&gt;Signal Processing Stack Exchange&lt;/a&gt;, where use of FFT is well understood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the environment is noisy or the target frequency can drift (making it hard to set a simple threshold) then you could also combine FFT with a neural network. This combination can solve much more complicated signal detection, and is used in speech processing for instance.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sampled at 100Hz - it is possible to increase the sample rate to 200Hz or 400Hz at max&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For reliably detecting a 50Hz signal, I would say that 200Hz sample rate is minimum. The theorectical minimum is 100Hz (i.e. twice the signal frequency) but may give you problems with noise and the possibility that your sample points just happen to fall on low amplitude parts of the oscillations, making it look like the motor is off even when it is on.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-09T08:14:28.820" LastActivityDate="2019-10-09T08:14:28.820" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15810" PostTypeId="2" ParentId="15730" CreationDate="2019-10-09T07:59:43.847" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.wired.com/2014/02/halting-problem/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Questions That Computers Can Never Answer - Wired (magazine)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Computers might not be able to reach infinity at all: &amp;lt; &lt;a href=&quot;https://www.nature.com/articles/35023282&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/35023282&lt;/a&gt; &gt;, never mind actually understand it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computation and computers do have implications for &quot;hard limits of systems.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Limits_of_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Limits_of_computation&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="25982" LastEditDate="2019-11-26T08:27:58.557" LastActivityDate="2019-11-26T08:27:58.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15811" PostTypeId="2" ParentId="15808" CreationDate="2019-10-09T08:04:22.460" Score="0" Body="&lt;p&gt;I finally solved it out but it's long, it's not only &lt;strong&gt;chain rule&lt;/strong&gt;, it includes &lt;strong&gt;quotient rule&lt;/strong&gt; too. And, this is only the third last layer, once a DNN has more layers then it's more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\dfrac{dLoss}{dW_1} = \dfrac{d}{dW_1}(Out-Y) = \dfrac{d}{dW_1}Out = \dfrac{d}{dW_1}(H_4W_{13} + H_5W_{14}) \\= \dfrac{d}{dW_1}H_4W_{13} + \dfrac{d}{dW_1}H_5W_{14} \\= W_{13} \times \dfrac{d}{dW_1}H_4 + W_{14} \times \dfrac{d}{dW_1}H_5 \\= W_{13} \times \dfrac{d}{dW_1}(H_1W_7 + H_2W_8 + H_3W_9) + W_{14} \times \dfrac{d}{dW_1}(H_1W_{10} + H_2W_{11} + H_3W_{12}) \\= W_{13} \times \dfrac{d}{dW_1}H_1W_7 + W_{14} \times \dfrac{d}{dW_1}H_1W_{10} \\= W_{13}W_7 \times \dfrac{d}{dW_1}H_1 + W_{14}W_{10} \times \dfrac{d}{dW_1}H_1 \\= (W_{13}W_7 + W_{14}W_{10}) \times \dfrac{d}{dW_1}(X_1W_1 + X_2W_2) \\= (W_{13}W_7 + W_{14}W_{10}) \times X_1$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-09T09:26:08.010" LastActivityDate="2019-10-09T09:26:08.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15813" PostTypeId="2" ParentId="11781" CreationDate="2019-10-09T10:50:28.163" Score="0" Body="&lt;p&gt;As I understand it above-mentioned projection operator project into linear feature subspace produced from set of feature vectors (or feature functions), that is space of linear combinations of features. Vanilla DQN don't have any feature space, projection into linear subspace doesn't make sense in DQN context. If you attempt to produce feature space for values/Q with some NN it wouldn't be DQN (because Q wouldn't be produced) and it wouldn't work anyway on anything but toy problems because amount of degrees of freedom of output would be too high.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-10-09T10:50:28.163" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15816" PostTypeId="1" CreationDate="2019-10-09T15:31:02.267" Score="1" ViewCount="20" Body="&lt;p&gt;so I'm working on a Project where I want to predict the Vehicle Position from the Vehicle Data like speed, acceleration etc.. now the data that I have comes also with a timestamp for each sample ( I mean that I have also a timestamp feature).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at first I thought that I should get rid of that timestamp feature because it is not relevant to my Project, I mean logically, I will not need a timestamp feature to predict the vehicle position, that didn't make sense to me when I first took a look at the dataset. I thought other features like speed, acceleration, braking pressure etc.. are more important and I thought also that the solution for this Problem would be to use a normal Deep NN or RBFNN for making this Prediction. recently, I read some papers that shows how a Convolutional NN can be also used for regression and that confused me to choose the Architecture needed for my Project. this Week I also watched a Tutorial where a RNN/ LSTM was implemented for regression Tasks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm very confused which architecture should I use for my Project. I also noticed that maybe if I used that timestamp feature, I can maybe use an RNN/LSTM Network for this Task but I don't know if my dataset can be seen as time-series dataset, actually the vehicle position doesn't depend on the time as far as I can tell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully can someone answer me based on Experience. It would be also great to have some Papers or references where I can look for more. &lt;/p&gt;&#xA;" OwnerUserId="30327" LastActivityDate="2019-10-09T15:31:02.267" Title="How to choose the suitable Neural Network Architecture for Regression Tasks" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;prediction&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15817" PostTypeId="1" AcceptedAnswerId="15822" CreationDate="2019-10-09T15:47:06.657" Score="2" ViewCount="181" Body="&lt;p&gt;I'm just started to learn deep learning and I have a question about this neural network:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W6FuR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W6FuR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_n$&lt;/span&gt; are perceptrons. So, if they are perceptrons, all of them will have an activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if it is possible to have only one activation function, and sum the output to all of the perceptrons, and pass that sum to that activation function. And the output of this activation function will be &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will have this network, where &lt;span class=&quot;math-container&quot;&gt;$H1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Hj$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Hn$&lt;/span&gt; don't have activation function:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2OuoU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2OuoU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input for the activation function will be the sum of the outputs of &lt;span class=&quot;math-container&quot;&gt;$H1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Hj$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Hn$&lt;/span&gt; without been processed by an activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that possible (or is it a good idea)?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2019-10-10T02:14:08.813" LastActivityDate="2019-10-10T03:33:40.743" Title="Can multiple activation functions be replaced with a single activation function?" Tags="&lt;deep-learning&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15818" PostTypeId="1" CreationDate="2019-10-09T15:51:49.570" Score="1" ViewCount="20" Body="&lt;p&gt;Turns out that it looks like I will be approximating a 100x10 matrix in my project thesis. II have the following equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y = Dx$&lt;/span&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$(100 \times 1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$100 \times 10$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$10 \times 1$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is the transformation matrix &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; I will be approximating by iterating over quite alot of pairs (x,y). I was wondering if it is possible to output a &lt;span class=&quot;math-container&quot;&gt;$(100 \times 10)$&lt;/span&gt; matrix as output in a FFNN-architecture, or is there any other ways to do it? As this is not an image or anything similar which gives rise to pooling etc., my guess is that CNN is not ideal here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So tldr: Is it easy to use feed-forward architecture to approximate an matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I figured out that &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; obviously doesn't need to be a mtrxi, just have 100 output nodes, duh. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="30329" LastEditorUserId="30329" LastEditDate="2019-10-09T16:26:08.637" LastActivityDate="2019-10-09T16:26:08.637" Title="Matrix-output for FFNN?" Tags="&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15819" PostTypeId="2" ParentId="15817" CreationDate="2019-10-09T17:24:05.327" Score="0" Body="&lt;p&gt;Of course, it is possible, but why do you want to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's think about it. Imagine your weights for that layer is a matrix full of ones, so, if you have no bias, then your output of that layer would be the sum of all values in the &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_n$&lt;/span&gt; neurons, right? So, it is possible to sum all the values together, give the output to an activation function, and then you'll have your output.&lt;/p&gt;&#xA;" OwnerUserId="30327" LastEditorUserId="2444" LastEditDate="2019-10-10T02:15:24.603" LastActivityDate="2019-10-10T02:15:24.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15820" PostTypeId="1" AcceptedAnswerId="15823" CreationDate="2019-10-09T17:45:29.010" Score="25" ViewCount="4043" Body="&lt;p&gt;Is there any research on the development of attacks against artificial intelligence systems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, is there a way to generate a letter &quot;A&quot;, which every human being in this world can recognize but, if it is shown to the state-of-the-art character recognition system, this system will fail to recognize it? Or spoken audio which can be easily recognized by everyone but will fail on the state-of-the-art speech recognition system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there exists such a thing, is this technology a theory-based science (mathematics proved) or an experimental science (randomly add different types of noise and feed into the AI system and see how it works)? Where can I find such material?  &lt;/p&gt;&#xA;" OwnerUserId="30335" LastEditorUserId="1671" LastEditDate="2019-10-11T00:34:52.770" LastActivityDate="2019-10-11T19:20:12.913" Title="Is there any research on the development of attacks against artificial intelligence systems?" Tags="&lt;image-recognition&gt;&lt;reference-request&gt;&lt;voice-recognition&gt;&lt;security&gt;&lt;adversarial-ml&gt;" AnswerCount="8" CommentCount="4" FavoriteCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="15821" PostTypeId="2" ParentId="15820" CreationDate="2019-10-09T18:05:52.367" Score="12" Body="&lt;p&gt;Sometimes if the rules used by an AI to identify characters are discovered, and if the rules used by a human being to identify the same characters are different, it is possible to design characters that are recognized by a human being but not recognized by an AI.  However, if the human being and AI both use the same rules, they will recognize the same characters equally well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A student I advised once trained a neural network to recognize a set of numerals, then used a genetic algorithm to alter the shapes and connectivity of the numerals so that a human could still recognize them but the neural network could not.  Of course, if he had then re-trained the neural network using the expanded set of numerals, it probably would have been able to recognize the new ones.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-10-09T18:05:52.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15822" PostTypeId="2" ParentId="15817" CreationDate="2019-10-09T18:34:15.507" Score="2" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: This is possible but removing the activations will decrease the expressivity of the network because it will become mathematically equivalent to a single neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mathematical Explanation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The outputs of your intermediate neurons (in the absence of activation functions) are now:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(1)}\quad H_i(x) = \sum_{j=1}^nw_{ij}\cdot x_j+b_i$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are then summing each &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(2)}\quad\mathcal{H(x)}=\sum_{i=1}^nH_i(x)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then pass &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(x)}$&lt;/span&gt; to some activation say &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(3)}\quad\hat y = g(\mathcal{H(x)})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trouble is that the inner term &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(x)}$&lt;/span&gt; mathematically reduces to a single linear operation on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Proof:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(x)}=\sum_{i=1}^nH_i(x)$&lt;/span&gt;. Substituting in (1):&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(x)}=\sum_{i=1}^n((\sum_{j=1}^nw_{ij}\cdot x_j)+b_i)$&lt;/span&gt;. This can be re-aranged :&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(x)}=(\sum_{j=1}^n(\sum_{i=1}^nw_{ij})\cdot x_j)+\sum_{i=1}^nb_i$&lt;/span&gt;. But this reduces to:&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(x)}=\sum_{j=1}^n\tilde w_{j}\cdot x_j+\tilde b$&lt;/span&gt;. Where &lt;span class=&quot;math-container&quot;&gt;$\tilde w_{j},\tilde b$&lt;/span&gt; are scalars. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thus, without the non-linear activations (3) mathematically reduces to a single neuron. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-10-10T03:33:40.743" LastActivityDate="2019-10-10T03:33:40.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15823" PostTypeId="2" ParentId="15820" CreationDate="2019-10-09T21:56:34.037" Score="28" Body="&lt;p&gt;Yes, there is some research on this topic, which can be called &lt;a href=&quot;https://arxiv.org/abs/1611.01236&quot; rel=&quot;noreferrer&quot;&gt;adversarial machine learning&lt;/a&gt;, which is more an experimental field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;a href=&quot;https://arxiv.org/pdf/1312.6199.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;adversarial example&lt;/em&gt;&lt;/a&gt; is an input similar to the ones used to train the model, but that leads the model to produce an unexpected outcome. For example, consider an artificial neural network (ANN) trained to distinguish between oranges and apples. You are then given an image of an apple similar to another image used to train the ANN, but that is slightly blurred. Then you pass it to the ANN, which unexpectedly predicts the object to be an orange.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several machine learning and optimization methods have been used to detect the boundary behaviour of machine learning models, that is, the unexpected behaviour of the model that produces different outcomes given two slightly different inputs (but that correspond to the same object). For example, evolutionary algorithms have been used to develop tests for self-driving cars. See, for example, &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=3330566&quot; rel=&quot;noreferrer&quot;&gt;Automatically testing self-driving cars with search-based procedural content generation&lt;/a&gt; (2019) by Alessio Gambi et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-10T13:17:13.447" LastActivityDate="2019-10-10T13:17:13.447" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15824" PostTypeId="1" CreationDate="2019-10-09T23:49:09.500" Score="3" ViewCount="243" Body="&lt;p&gt;It is a well-known math fact that composition of linear/affine transformations is still linear/affine. For a naive example,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{A}_1\textbf{A}_2\textbf{x}$&lt;/span&gt; is simply &lt;span class=&quot;math-container&quot;&gt;$\textbf{A}\textbf{x}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\textbf{A}=\textbf{A}_1\textbf{A}_2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any one knows why in practice multiple linear layers tend to work better, even though it is mathematically equivalent to a single linear layer? Any reference is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="30344" LastEditorUserId="28343" LastEditDate="2021-02-15T14:05:41.487" LastActivityDate="2021-02-15T14:05:41.487" Title="Any explanation why multiple linear layers work better than a single linear layer in practice?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15825" PostTypeId="2" ParentId="15824" CreationDate="2019-10-10T01:08:56.700" Score="1" Body="&lt;p&gt;The key is that the layers of neurons in neural networks are &lt;em&gt;not&lt;/em&gt; affine transformations. All commonly used neurons have some kind of non-linearity. The simplest of these is the Rectified Linear Unit (ReLU), which takes the form &lt;span class=&quot;math-container&quot;&gt;$y = x$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$x &amp;gt; 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y = 0$&lt;/span&gt; for all other values, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a weighted sum of the inputs to the neuron.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-10T01:08:56.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15826" PostTypeId="5" CreationDate="2019-10-10T02:27:28.130" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-10T02:27:28.130" LastActivityDate="2019-10-10T02:27:28.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15827" PostTypeId="4" CreationDate="2019-10-10T02:27:28.130" Score="0" Body="For questions related to variational auto-encoders (VAEs). The first VAE was proposed in &quot;Auto-Encoding Variational Bayes&quot; (2013) by Diederik P. Kingma and Max Welling. There are several other VAEs, for example, the conditional VAE." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-10T02:27:28.130" LastActivityDate="2019-10-10T02:27:28.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15828" PostTypeId="5" CreationDate="2019-10-10T02:29:11.697" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-10T02:29:11.697" LastActivityDate="2019-10-10T02:29:11.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15829" PostTypeId="4" CreationDate="2019-10-10T02:29:11.697" Score="0" Body="For questions related to adversarial machine learning, which is a branch of machine learning focused on the study of adversarial examples, which are malicious inputs designed to fool machine learning models." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-10T02:29:11.697" LastActivityDate="2019-10-10T02:29:11.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15832" PostTypeId="2" ParentId="15820" CreationDate="2019-10-10T06:00:32.593" Score="4" Body="&lt;p&gt;Isn't that essentially what chess does? For example, A human can recognize that a Ruy exchange offers white great winning chances (because of pawn structure) by move 4 while an engine would take several hours of brute force calculation to understand the same idea. &lt;/p&gt;&#xA;" OwnerUserId="30348" LastActivityDate="2019-10-10T06:00:32.593" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15833" PostTypeId="1" CreationDate="2019-10-10T07:26:19.763" Score="1" ViewCount="17" Body="&lt;p&gt;i am a new beginner to EP algorithm.Could someone help me with a coding for economic load dispatch using EP? Can i have some examples to learn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tq&lt;/p&gt;&#xA;" OwnerUserId="30350" LastActivityDate="2019-10-10T07:26:19.763" Title="evolutionary programming for economic load dispatch" Tags="&lt;matlab&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15834" PostTypeId="1" CreationDate="2019-10-10T07:34:57.290" Score="2" ViewCount="42" Body="&lt;p&gt;Could it be possible to reach a similar output via feeding a unidirectional network with the original data and the data played backwards? &lt;/p&gt;&#xA;" OwnerUserId="30351" LastEditorUserId="1671" LastEditDate="2019-11-13T03:04:11.280" LastActivityDate="2019-11-13T03:04:11.280" Title="How does a Bidirectional RNN work?" Tags="&lt;backpropagation&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15835" PostTypeId="2" ParentId="15772" CreationDate="2019-10-10T08:47:13.633" Score="0" Body="&lt;p&gt;The solution mentioned seems feasible but you'd probably encounter a lot of problems, such as -  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Since you're outputting coordinates, each one of the input networks must be trained differently. Considering &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; should ideally be a variable, how many input networks do you train?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You'd use an intermediate encoding of sorts as co-ordinate representation which would be averaged and then passed through the output network. - Interpolation properties of the encoding might not be too great, meaning a linear change in the representation might not lead to a linear change in the co-ordinate, causing an averaging function to give skewed results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Just a suggestion but I think it would be better if instead of learning multiple estimates of co-ordinate representations and then averaging them it might be better if you tried to learn some sort of distance function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, each receiver would have a fixed position (&lt;em&gt;Assumption&lt;/em&gt;) and the input network (or two input networks preferably) would output the distance  (&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;) of the transmitter from that receiver and the angle (&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) it makes with a common axis. The problem with this approach is you wouldn't be leveraging multiple receivers as they would not be learning together, and the advantage being that whatever &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is, the RSSI readings pass through the same network always. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once an &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; number of &lt;span class=&quot;math-container&quot;&gt;$(r, \theta)$&lt;/span&gt; values are obtained - a robust algorithm &#xA;&lt;span class=&quot;math-container&quot;&gt;$f(r_1, r_2,..., \theta_1, \theta_2, ...) = (x_t, y_t)$&lt;/span&gt; could be found to output the transmitter co-ordinates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not too confident if it would work too well but just a suggested direction I could think of! Hope this helped!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-10-10T08:47:13.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15838" PostTypeId="2" ParentId="15820" CreationDate="2019-10-10T09:42:44.430" Score="4" Body="&lt;p&gt;There are many insightful comments and answers so far. I want to illustrate my idea of &quot;color blindness test&quot; more. Maybe it's a hint to lead us to the truth.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine there are two people here. One is colorblind (AI) and another one is non-colorblind (human). If we show them a normal number &quot;6&quot;, both of them can easily recognize it as number 6. Now, if we show them a delicately designed colorful number &quot;6&quot;, only human can recognize it as number 6 while AI will recognize it as number 8. The interesting of this analogy is that we can not teach/train colorblind people to recognize this delicately designed colorful number &quot;6&quot; because of natural difference, which I believe is also the case between AI and human. AI gets results from computation while human gets results from &quot;mind&quot;. Therefore, like @S. McGrew's answer, if we can find the fundamental difference between AI and human of how we read things, then this question is answered.  &lt;/p&gt;&#xA;" OwnerUserId="30335" LastActivityDate="2019-10-10T09:42:44.430" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15839" PostTypeId="2" ParentId="15820" CreationDate="2019-10-10T10:28:36.707" Score="11" Body="&lt;p&gt;Yes there are, for instance one pixel attacks described in&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Su, J.; Vargas, D.V.; Kouichi, S. One pixel attack for fooling deep&#xA;  neural networks. &lt;a href=&quot;https://arxiv.org/pdf/1710.08864.pdf&quot; rel=&quot;noreferrer&quot;&gt;arXiv:1710.08864&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One pixels attacks are attacks in which changing one pixel in input image can strongly affect the results.&lt;/p&gt;&#xA;" OwnerUserId="30354" LastEditorUserId="2212" LastEditDate="2019-10-10T18:27:54.553" LastActivityDate="2019-10-10T18:27:54.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15840" PostTypeId="1" CreationDate="2019-10-10T10:37:41.517" Score="1" ViewCount="44" Body="&lt;p&gt;Blockchain and artificial intelligence are both making strides in the industry. While Blockchain is revolutionizing the financial sector, artificial intelligence is advancing the industry with self-driving cars and other machine learning advancements. One of the reasons artificial intelligence is progressing at a rapid pace is the data manipulation capability offered by blockchain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Blockchain’s decentralized technology can lend more transparency and security to artificial intelligence. So far blockchain has had a big impact throughout the industry. As it continues to improve the way we encrypt, examine, and handle large datasets, it can play an important role in the advancement of artificial intelligence. Thus, giving a &lt;a href=&quot;https://www.cbcamerica.org/blockchain-certifications/certified-blockchain-engineer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;blockchain engineer&lt;/strong&gt;&lt;/a&gt; opportunities to experiment more and come up with ways to advance AI development. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://medium.com/@alinaparker11/how-can-blockchain-make-artificial-intelligence-better-f2a0477c7685&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Can Blockchain Make Artificial Intelligence Better?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26265" LastEditorUserId="2444" LastEditDate="2019-10-10T13:26:39.303" LastActivityDate="2019-10-10T13:26:39.303" Title="How exactly can blockchain make Artificial Intelligence better?" Tags="&lt;intelligence&gt;&lt;blockchain&gt;&lt;artificial&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15841" PostTypeId="2" ParentId="1285" CreationDate="2019-10-10T10:48:50.270" Score="0" Body="&lt;p&gt;Blockchain and artificial intelligence are both making strides in the industry. While Blockchain is revolutionizing the financial sector, artificial intelligence is advancing the industry with self-driving cars and other machine learning advancements. One of the reasons artificial intelligence is progressing at a rapid pace is the data manipulation capability offered by blockchain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Blockchain’s decentralized technology can lend more transparency and security to artificial intelligence. So far blockchain has had a big impact throughout the industry. As it continues to improve the way we encrypt, examine, and handle large datasets, it can play an important role in the advancement of artificial intelligence. Thus, giving a &lt;a href=&quot;https://www.cbcamerica.org/blockchain-certifications/certified-blockchain-engineer&quot; rel=&quot;nofollow noreferrer&quot;&gt;blockchain engineer&lt;/a&gt; opportunities to experiment more and come up with ways to advance AI development. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also read this article &lt;a href=&quot;https://medium.com/@alinaparker11/how-can-blockchain-make-artificial-intelligence-better-f2a0477c7685&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Can Blockchain Make Artificial Intelligence Better?&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="26265" LastEditorUserId="2444" LastEditDate="2019-10-10T13:24:55.100" LastActivityDate="2019-10-10T13:24:55.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15842" PostTypeId="2" ParentId="9076" CreationDate="2019-10-10T11:15:35.343" Score="0" Body="&lt;p&gt;Mutation is secondary operator (10%) while Crossover is Primary (90%) which creates offspring while mutation can only shuffle information within the Choromsome&lt;/p&gt;&#xA;" OwnerUserId="30358" LastEditorUserId="28348" LastEditDate="2019-10-10T20:43:12.877" LastActivityDate="2019-10-10T20:43:12.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15843" PostTypeId="1" AcceptedAnswerId="15855" CreationDate="2019-10-10T13:49:14.647" Score="0" ViewCount="264" Body="&lt;p&gt;I've a prediction matrix(P) of dimension 3x3 and one-hot encoded label matrix(L) of dimension 3x3 as shown below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    |0.5 0.3 0.1|      |1 0 0|&#xA;P = |0.3 0.2 0.1|  L = |0 1 0|&#xA;    |0.2 0.5 0.8|      |0 0 1|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;each column in 'P' corresponds to prediction of a label in 'L'&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How is the BCELoss calculated using pytorch?, my experimentation by giving these two matrices as parameters to loss function yielded me poor results and pytorch's loss calculation function doesn't disclose on how loss calculation is done for this case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How is the loss averaged for each instance and across the a batch?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;if loss is calculated column wise and averaged for each instance and across the batch, then how can loss be backprop'd in pytorch?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastActivityDate="2019-10-11T04:09:29.853" Title="Confused with backprop in pytorch with BCE loss" Tags="&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15844" PostTypeId="2" ParentId="1285" CreationDate="2019-10-10T13:49:59.890" Score="0" Body="&lt;p&gt;I'm currently working on a p2p framework to train with neuroevolution, it will have neat, hyperneat, and eshyperneat example experiments. AI developers will be able to fork and add any experiments they want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It isn't a blockchain per sé but will have a dht for genes, champion nets, peers of course, and genomes. Neuroevolution training can be done in parallel so the population will be distributed evenly among peers, peers that finish early will help slower peers with their evaluations, and all peers will have a check on one of the nets they evaluated by a random peer to prevent malice and all peers will check the champion of each generation. Any peer that evaluates a net will be rewarded a token that is specific to the generation it helped train and will also download a copy of the champion net, later I plan to allow non training clients the ability to view performance of past champs and purchase them from the network, any peer with a token for the genome they purchase will receive part of the payment. This will not be proof of stake, it will be a proof of work where work is evaluating nets, since this needs to be checked by other nets so people don't just post phony fitness for genomes I'll be calling it proof of fitness.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="4709" LastEditDate="2019-12-08T23:00:21.953" LastActivityDate="2019-12-08T23:00:21.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15845" PostTypeId="2" ParentId="1285" CreationDate="2019-10-10T15:10:12.740" Score="0" Body="&lt;p&gt;Maybe the term you're looking for is federated learning. Check out &lt;a href=&quot;https://www.openmined.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMined&lt;/a&gt; project, &lt;a href=&quot;https://github.com/OpenMined/PySyft&quot; rel=&quot;nofollow noreferrer&quot;&gt;PySyft&lt;/a&gt; and &lt;a href=&quot;https://www.tensorflow.org/federated&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow Federated&lt;/a&gt; libraries.&lt;/p&gt;&#xA;" OwnerUserId="30363" LastActivityDate="2019-10-10T15:10:12.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15846" PostTypeId="2" ParentId="9076" CreationDate="2019-10-10T21:13:59.127" Score="1" Body="&lt;p&gt;I like to use the term, &quot;recombination operator&quot; rather than &quot;crossover operator&quot;, because the latter term suggests a specific type of operation: constructing an offspring by switching corresponding chromosome segments between two parents.  &quot;Recombination&quot; (to me) suggests any operation that forms an offspring from the genetic information of two parents.  &quot;Crossover&quot; in that sense doesn't work when the individuals are, for example, permutations; but many &quot;recombination operators&quot; that &lt;em&gt;do&lt;/em&gt; work are still possible, which preserve non-conflicting portions of two parent permutations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In GA, mutation can be thought of as a relatively small random change that occurs within an individual. Mutation usually is a change of the value of one gene &lt;em&gt;without making use of gene values in any other individuals&lt;/em&gt;, but can also be a random rearrangement of elements in a permutation, or a random change in the values  of several genes.  Sometimes the term is applied to a &quot;hill climbing&quot; procedure in which several mutations are applied to an individual and their effect on fitness is tested; then the one that produces the most fitness improvement is retained.  &lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-10-10T21:13:59.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15847" PostTypeId="2" ParentId="15820" CreationDate="2019-10-10T23:27:08.880" Score="5" Body="&lt;p&gt;Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.wired.co.uk/article/avoid-facial-recognition-software&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to hack your face to dodge the rise of facial recognition tech&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In his recent book &lt;em&gt;The Fall&lt;/em&gt;, Stephenson wrote about smartglasses that that project a pattern over the facial features to foil recognition algorithms (which seems not only feasible but likely;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an article from our sponsors, &lt;em&gt;&lt;a href=&quot;https://securityintelligence.com/adversarial-ai-as-new-attack-vector-opens-researchers-aim-to-defend-against-it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial AI: As New Attack Vector Opens, Researchers Aim to Defend Against It&lt;/a&gt;&lt;/em&gt; which includes this graphic of &quot;&lt;a href=&quot;https://securityintelligence.com/wp-content/uploads/2018/04/2018-ibm-security-ai-hacks-infographic-1.2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Five ways AI hacks can lead to real world problems&lt;/a&gt;&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article references the conference on &lt;a href=&quot;https://maliciousaireport.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation&lt;/a&gt;, where you can download the full report.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I'm assuming many such examples exist in the real world, and will amend this link-based answer as I find them. Good question!&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-11T00:08:24.047" LastActivityDate="2019-10-11T00:08:24.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15848" PostTypeId="1" CreationDate="2019-10-11T00:33:56.297" Score="2" ViewCount="129" Body="&lt;p&gt;There was a recent question on &lt;a href=&quot;https://ai.stackexchange.com/questions/15820/is-there-any-research-on-the-development-of-attacks-against-artificial-intellige&quot;&gt;adversarial AI applications&lt;/a&gt;, which led me to start digging around.  Here my interest is not general, but specific:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the vulnerabilities of automated trading systems?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Essentially, what are the strategies for adversarial economic AI?  How do you game the system by messing with data, as opposed to hacking the algorithm itself. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-10-11T02:59:28.033" Title="In what ways can financial markets be hacked? [Algorithmic Trading]" Tags="&lt;applications&gt;&lt;reference-request&gt;&lt;algorithmic-trading&gt;&lt;economics&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15849" PostTypeId="2" ParentId="15820" CreationDate="2019-10-11T01:19:52.470" Score="3" Body="&lt;p&gt;Here's a live demo: &lt;a href=&quot;https://www.labsix.org/physical-objects-that-fool-neural-nets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.labsix.org/physical-objects-that-fool-neural-nets/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that neural nets are trained by feeding in the training data, evaluating the net, and using the error between the observed and the intended output to adjust the weights and bring the observed output closer to the intended. Most attacks have been on the observation that you can, instead of updating the weights, update the input neurons. That is, permute the image. However, this attack is very finnicky. It falls apart when the permuted image is scaled, rotated, blurred, or otherwise altered. That's clearly a cat to us, but guacamole to the neural net. But a slight rotation and the net starts classifying it correctly again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However recent breakthroughs allow actual objects presented to a real camera to be reliably misclassified. That's clearly a turtle, albeit with a wonky pattern on its shell. But that net is convinced it's a rifle from practically every angle.&lt;/p&gt;&#xA;" OwnerUserId="30378" LastActivityDate="2019-10-11T01:19:52.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15850" PostTypeId="1" CreationDate="2019-10-11T01:38:13.327" Score="1" ViewCount="113" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$H_1$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$H_2$&lt;/span&gt; ,... be a sequence of hypothesis classes for binary classiﬁcation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that there is a learning algorithm that implements the ERM rule in the realizable case such that the output hypothesis of the algorithm for each class &lt;span class=&quot;math-container&quot;&gt;$H_n$&lt;/span&gt; only depends on &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; examples out of the training set. Furthermore, assume that such a hypothesis can be calculated given these &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; examples in time &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;, and that the empirical risk of each such hypothesis can be evaluated in time &lt;span class=&quot;math-container&quot;&gt;$O(mn)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if &lt;span class=&quot;math-container&quot;&gt;$H_n$&lt;/span&gt; is the class of axis aligned rectangles in &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt; , we saw that it is possible to ﬁnd an ERM hypothesis in the realizable case that is deﬁned by at most &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prove that in such cases, it is possible to ﬁnd an ERM hypothesis for &lt;span class=&quot;math-container&quot;&gt;$H_n$&lt;/span&gt; in the unrealizable case in time &lt;span class=&quot;math-container&quot;&gt;$O(mnm^{O(n)})$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27112" LastEditorUserId="16909" LastEditDate="2019-10-11T02:11:22.710" LastActivityDate="2021-03-04T08:06:15.597" Title="Prove that in such cases, it is possible to ﬁnd an ERM hypothesis for $H_n$ in the unrealizable case in time $O(mnm^{O(n)})$" Tags="&lt;machine-learning&gt;&lt;time-complexity&gt;&lt;pac-learning&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15851" PostTypeId="2" ParentId="15848" CreationDate="2019-10-11T02:13:47.083" Score="2" Body="&lt;p&gt;I don't have a proper source for this, as i've only read this from an online forum: everything i say is just hearsay and I am very uneducated on the subject. With that being said... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you may know algorithmic trading relies on &lt;em&gt;strategies&lt;/em&gt;, i.e. I trade a certain way once I see certain indexes move in a certain way. If your procedure is known by other people for some reason, then other adverse agents can release information or influence the market to manipulate your bot into making trades that the adverse agent then takes advantage over. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how this works in detail, and frankly I find it quite unlikely that a single adverse agent can target a single known strategy with the entire market in-between the two. But I've heard that a process known as the alpha algorithm can be used to discover hidden trading strategies. But again, I wasn't able to find any articles that directly linked this method towards financial trading. So my usual warning of me talking out my butt again applies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a fun thought. A friend of mine told me (unfortunately, again I wasn't able to find a proper source for this) that Jane Street found that the most effective trading strategies were based on relatively simple ML techniques (I recall the specific example mentioned was just a common form of regression). The important follow question would be &quot;What are their input metrics&quot;, I don't think I was told this, but I would believe that it would also just be something computed by relatively well-founded stock market indices. If this is the case, than again it would be very difficult for a single adverse agent to influence that particular bot since the force of the entire market is in between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the opposite end, we have &lt;a href=&quot;https://archive.fo/C2HB5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this redditor&lt;/a&gt;(finally, a source!), who amazingly analyzed about 130+ trading strategies, each of which were &quot;big brained&quot; in kind. This redditor concluded that all papers were essentially p-hacked, and none of the results were significant. Of course there is the possibility that authors with profitable results would not publish them, but the conclusion remains: keep your strategies simple. Again, this implies that it would be harder to &quot;hack&quot; a specific strategy used in the market by a particular individual. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. There are other &quot;big brained&quot; methods that are kept well secret and thus difficult to analyze: I heard a rumour that a Chinese company paid people to take pictures inside shopping malls, to use to analyze shopping behavior, and how vibrant the economy is; I also hear that some people use satellites photos to look at, say, car dealerships, to see how much of their stuff is being purchased. Again its unclear if these techniques are profitable or not, but certainly a lot of money is being thrown into it. &lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2019-10-11T02:59:28.033" LastActivityDate="2019-10-11T02:59:28.033" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15852" PostTypeId="1" AcceptedAnswerId="15856" CreationDate="2019-10-11T02:47:47.147" Score="2" ViewCount="38" Body="&lt;p&gt;The empirical error equation given in the book &lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Machine Learning: From Theory to Algorithms&lt;/a&gt; is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XPFmE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XPFmE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition for this equation is: total wrong predictions divided by the total number of samples &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; in the given sample set &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (Correct me if I'm wrong). But, in this equation, the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; takes &lt;span class=&quot;math-container&quot;&gt;$\{ 1, \dots, m \}$&lt;/span&gt;. How is this actually calculated, as I thought it should be one number (the size of the sample)?&lt;/p&gt;&#xA;" OwnerUserId="30381" LastEditorUserId="2444" LastEditDate="2021-01-22T15:12:22.040" LastActivityDate="2021-01-22T15:12:22.040" Title="What does the notation $[m]=\{1, \ldots, m\}$ mean in the equation of the empirical error?" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;notation&gt;&lt;books&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15853" PostTypeId="2" ParentId="15850" CreationDate="2019-10-11T03:09:49.190" Score="0" Body="&lt;p&gt;Here's a rough proof sketch that might be clearer, but is probably less precise, than the solution you already have.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;We have a sequence of hypotheses classes, but the problem actually only asks us to consider a single class. A hypothesis class is a set of possible models for classification that differ only in a set of parameter values.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We are told that the algorithm for selecting a hypothesis from &lt;span class=&quot;math-container&quot;&gt;$H_n$&lt;/span&gt; uses only &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; samples from the training set, and that using those samples, it takes the algorithm &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; to find the hypothesis that minimizes the empirical risk for the class, provided that the ERM minimizing hypothesis has an empirical risk of 0 (i.e. it is realizable). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Further, if we pick some hypothesis at random from the class, we are told that the cost of determining its exact empirical risk is &lt;span class=&quot;math-container&quot;&gt;$O(nm)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the total size of the training set, and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is just a parameter associated with this particular hypothesis class &lt;span class=&quot;math-container&quot;&gt;$H_n$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Observer that, in the unrealizable case, we cannot use the algorithm to find the risk minimizing hypothesis (which, by 2, runs in O(n)). No other algorithm is mentioned, but we are also told (in 2) that the algorithm can construct a hypothesis from &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; datapoints in &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; time, even if no feasible hypothesis can be found for those &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose we wanted to find the hypothesis with minimal risk via brute force. We could compute every subset of the training dataset that is of the size &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; that the algorithm may be able to use to construct a hypothesis in &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; time. There are &lt;span class=&quot;math-container&quot;&gt;$O(m^{O(n)})$&lt;/span&gt; such subsets (e.g. &lt;span class=&quot;math-container&quot;&gt;$m*(m-1)/2$&lt;/span&gt; possible pairs, &lt;span class=&quot;math-container&quot;&gt;$m*(m-1)*(m-2)/6$&lt;/span&gt; possible triplets, etc.). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since there are &lt;span class=&quot;math-container&quot;&gt;$O(m^{O(n)})$&lt;/span&gt; possible subsets of size &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;, and our algorithm can compute a candidate hypothesis in &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; time, it costs us &lt;span class=&quot;math-container&quot;&gt;$O(nm^{O(n)})$&lt;/span&gt; to find all possible hypotheses that our algorithm can produce for this dataset.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since it costs us &lt;span class=&quot;math-container&quot;&gt;$O(nm)$&lt;/span&gt; time to compute the emperical risk of a hypothesis generated by this algorithm, we can minimize risk by computing risk for each of the &lt;span class=&quot;math-container&quot;&gt;$O(m^{O(n)})$&lt;/span&gt; hypotheses we made in step 6, and then selecting the one with lowest empirical risk. This costs us &lt;span class=&quot;math-container&quot;&gt;$O(mnm^{O(n)})$&lt;/span&gt; time total. Our total runtime is then the sum of the cost to generate the models in step 6, and the cost to evaluate them here in step 7: &lt;span class=&quot;math-container&quot;&gt;$O(mnm^{O(n)})+O(nm^{O(n)})$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$O(mnm^{O(n)})+O(nm^{O(n)}) \in O(mnm^{O(n)})$&lt;/span&gt;, we have proven the desired bound on runtime.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-11T03:09:49.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15855" PostTypeId="2" ParentId="15843" CreationDate="2019-10-11T04:09:29.853" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network. &lt;/li&gt;&#xA;&lt;li&gt;For the calculation of BCE, it is shown on pytorch documentation.&#xA;&lt;a href=&quot;https://pytorch.org/docs/stable/nn.html#BCELoss&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/docs/stable/nn.html#BCELoss&lt;/a&gt;&#xA;For across batch, it either sum the loss or take the mean. you can set that through the reduction=&quot;&quot; argument. &lt;/li&gt;&#xA;&lt;li&gt;As i said, the loss is either summed or taken the mean. Then there will be a single value so pytorch can do backdrop.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-11T04:09:29.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15856" PostTypeId="2" ParentId="15852" CreationDate="2019-10-11T04:25:48.507" Score="2" Body="&lt;p&gt;This is a commonly used notation in theoretical computer science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$[m]$&lt;/span&gt; is &lt;em&gt;not&lt;/em&gt; the variable &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, but is instead the set of integers from &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; inclusive. The empirical error equation thus reads in English:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cardinality of a set consisting of the elements &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; of the set of integers &lt;span class=&quot;math-container&quot;&gt;$[m]$&lt;/span&gt; such that the hypothesis given input &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; disagrees with label &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;, normalized by &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-11T04:25:48.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15857" PostTypeId="1" CreationDate="2019-10-11T05:27:08.410" Score="5" ViewCount="78" Body="&lt;p&gt;Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read this in a paper titled &lt;a href=&quot;https://arxiv.org/abs/1712.08969&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mean Field Residual Networks: On the Edge of Chaos&lt;/a&gt; (2017). I am not sure how far this is true, because the error depends on those activations.&lt;/p&gt;&#xA;" OwnerUserId="30384" LastEditorUserId="2444" LastEditDate="2019-10-22T01:15:27.400" LastActivityDate="2019-10-22T13:58:57.510" Title="Is the gradient at a layer independent of the activations of the previous layers?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;theory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15858" PostTypeId="2" ParentId="15857" CreationDate="2019-10-11T05:33:48.960" Score="1" Body="&lt;p&gt;Yes, this is the premise of back-propagation, the gradient at layer &lt;span class=&quot;math-container&quot;&gt;$j_{n}$&lt;/span&gt; is not impacted by the gradient at layer &lt;span class=&quot;math-container&quot;&gt;$j_{n-1}$&lt;/span&gt;. This allows you to start with a gradient at the output layer and propagated it back through the network to the input layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is however impacted by the gradient at &lt;span class=&quot;math-container&quot;&gt;$j_{n+1}$&lt;/span&gt;, which the back-prop algorithm also follows.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2019-10-22T01:16:11.403" LastActivityDate="2019-10-22T01:16:11.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15859" PostTypeId="1" AcceptedAnswerId="15863" CreationDate="2019-10-11T06:39:07.193" Score="1" ViewCount="1135" Body="&lt;p&gt;The answers to &lt;a href=&quot;https://qr.ae/TWYRNu&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Quora question&lt;/a&gt; say it's OK to ignore machine learning and start right away with deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is machine learning required or is useful for understanding (theoretically and practically) deep learning? Can I start right away with deep learning or should I cover machine learning first? In what way machine learning useful for deep learning? (leave the mathematics part - I'm ok with it).&lt;/p&gt;&#xA;" OwnerUserId="30381" LastEditorUserId="2444" LastEditDate="2019-10-11T16:05:48.253" LastActivityDate="2019-11-24T06:17:02.670" Title="Is machine learning required for deep learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15860" PostTypeId="1" CreationDate="2019-10-11T06:44:04.070" Score="3" ViewCount="32" Body="&lt;p&gt;I'm trying to figure out how to extract specific text from an utterance by a user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to extract &quot;unknown&quot; text from a short and simple text.  In this case, the user wants to create a list.  everything in the {} is unknown text. As it doesn't belong to a specific entity such as food, athletes, movies, etc.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;create a new {groceries} list&lt;/li&gt;&#xA;  &lt;li&gt;create a list {movies}&lt;/li&gt;&#xA;  &lt;li&gt;create a new list {movies}&lt;/li&gt;&#xA;  &lt;li&gt;create a list and call it {books}&lt;/li&gt;&#xA;  &lt;li&gt;create a new list and give it the name {stamps}&lt;/li&gt;&#xA;  &lt;li&gt;create a list with the title {red ketchup}&lt;/li&gt;&#xA;  &lt;li&gt;create another list called {rotten food}&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the above list is but a small sample of all the different ways that a user can say he wants to create a list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In everything that I have seen, it's all based on existing entities for the NER and when someone says that it's custom, I found that it just means we have to train a specific set of words and hope for the best.  If I add one more word that isn't trained, it fails to get the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in this case, the user can say anything such as &quot;old shoes&quot;, &quot;schools I want to go to&quot;, &quot;Keanu Reeves movies&quot;.  So I cannot see how I could possibly train it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Spacy, I followed this example (&lt;a href=&quot;https://raw.githubusercontent.com/explosion/spaCy/master/examples/training/train_intent_parser.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://raw.githubusercontent.com/explosion/spaCy/master/examples/training/train_intent_parser.py&lt;/a&gt;) and it mostly works in getting the proper titles. However, I have to train it for every different phrase to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if a user says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;create a beautiful new list and give it the name {stamps}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the word beautiful causes it to fail and now I have to train for that as well. At this rate, we are looking at millions of phrases to train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;before Spacy, we tried Dialogflow and Rasa.  At each point, it's about training phrases but the more we train, the more one thing worked and another broke.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, we have tried and overall had good intent detection success but when it comes to extracting data such as this, I'm starting to look like a deer in a headlight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are new to NLP and while we've had a lot of good progress and over the past few weeks, we cannot seem to find any articles written on this specific problem and whether it can be solved.  Dialogflow has the concept of &lt;em&gt;any&lt;/em&gt; entity but they recommend avoiding it and it works 2 out of 3 times when things get complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to detect which of these words is the title based training. Can it be done? and if so, what's the approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any code, hints or articles that might get us started would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30386" LastActivityDate="2019-10-11T06:44:04.070" Title="How to train a model to extract custom and unknown entities" Tags="&lt;training&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15861" PostTypeId="1" CreationDate="2019-10-11T06:55:14.323" Score="1" ViewCount="60" Body="&lt;p&gt;lately we read that many manufacturers are forcing ARM architectures to be used on future workstations. One of ARM's recent announcements is a &lt;a href=&quot;https://www.arm.com/products/silicon-ip-cpu/machine-learning/arm-ml-processor&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning processor&lt;/a&gt;. What will change in terms of computing performance if ARM architectures become new standard, and these kinds of ML-focused chips are found in most devices?&lt;/p&gt;&#xA;" OwnerUserId="23181" LastEditorUserId="16909" LastEditDate="2019-10-11T12:49:09.903" LastActivityDate="2019-10-11T13:15:45.503" Title="What will change when workstations will have ARM Machine Learning Processors onboard?" Tags="&lt;machine-learning&gt;&lt;performance&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15862" PostTypeId="2" ParentId="15859" CreationDate="2019-10-11T08:13:10.340" Score="4" Body="&lt;p&gt;Deep learning is part of machine learning.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You will miss out useful information if you ignore machine learning.&lt;/li&gt;&#xA;&lt;li&gt;You are ok to start your work in machine learning with deep learning and neural networks. You have to start somewhere and starting with a strong and successful method is resaonable, especially if you need to be able to produce good results quickly.&lt;/li&gt;&#xA;&lt;li&gt;You will learn essential machine learning stuff while reading about deep learning.&lt;/li&gt;&#xA;&lt;li&gt;The deep learning tutorials and other learning materials you will be reading may not be telling you that what you are learning also applies to other machine learning methods but you will be learning lot's of stuff that applies more generally. You will be studying some machine learning whether you want to or not.&lt;/li&gt;&#xA;&lt;li&gt;If you have plenty of time a more broad view will help understanding. Still, there is no need to wait with deep learning to after mastering some other methods.&lt;/li&gt;&#xA;&lt;li&gt;Broader knowledge helps you to relate and memorise concepts and be more aware of potential issues, especially issues that are rarely discussed in the deep learning community. Such knowledge and experience will be most useful when trying to apply deep learning to new problems or if trying to make substantial changes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30388" LastActivityDate="2019-10-11T08:13:10.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15863" PostTypeId="2" ParentId="15859" CreationDate="2019-10-11T10:00:49.397" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FyHLS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FyHLS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is Machine Learning required or is useful for understanding (theoretically and practically) Deep Learning?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;NO&lt;/p&gt;&#xA;&lt;p&gt;Deep learning is itself a huge subject area with serious applications in NLP, Computer Vision, Speech and Robotics.&#xA;You should learn deep learning from scratch like understanding forward propagation, back propagation, how weights are updated etc.. instead of using high level frameworks like keras, pytorch.&#xA;It's OK to use them once you understand the basics to save time and code complexity, but remember &amp;quot;surely&amp;quot; you don't need machine learning for that.&lt;/p&gt;&#xA;&lt;p&gt;Since you are familiar with the mathematics part, I would suggest you to straight away jump into Deep Learning. Note that deep learning is inspired by how the brain works.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Can I start right away with Deep learning or should I cover Machine learning first?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes you may start right away, start with the hello world problems &amp;quot;MNIST DIGIT Classification&amp;quot; if you know little image processing.&#xA;Start with a simple neural network model from scratch, then use keras (very easy) and then proceed to CNN ...&#xA;You may start with simple problems in other fields too (NLP, Speech)&#xA;I suggest Andrew Yangs, course in Machine Learning (within this he explains a neural network model for MNIST I guess).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;In what way machine learning useful for Deep learning?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You will understand that in machine learning you sit down and find useful features in the dataset yourself, but in deep learning it happens automatically&#xA;(Learn Deep learning in detail and come back and read this, you will understand exactly what I mean!)&#xA;If you learn Machine learning and then go to deep learning, you will realise that it was unnecessary .if you interested in this field of AI, jump into deep learning right now!&lt;/p&gt;&#xA;" OwnerUserId="26854" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-24T06:17:02.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15867" PostTypeId="2" ParentId="15861" CreationDate="2019-10-11T13:15:45.503" Score="1" Body="&lt;p&gt;It is not clear to me that much will change.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;ARM makes many devices, mostly designed to consume as little power as possible. My guess is that most workstations will not contain ARM's ML processor, even if they contain an ARM CPU.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ARM's ML processor can do machine learning. It is specifically optimized for training convolutional deep neural networks, and it is a bit faster that most existing non-purpose built chips, especially for its price. In practice, chips like this one should make ML a bit cheaper and faster, but it's not a breakthrough product. This just looks like the next iteration of &lt;a href=&quot;https://en.wikipedia.org/wiki/Moore%27s_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moore's law&lt;/a&gt;.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-11T13:15:45.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15868" PostTypeId="1" CreationDate="2019-10-11T13:50:58.347" Score="2" ViewCount="34" Body="&lt;p&gt;I want to implement a model that improves itself with the passage of time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little dataset available to train. Even if I create a corpus still there is a possibility of that corpus having poor translations due to outdated word choice for my native language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking to create a model which predicts output and user tells whether it is correct or not. Or maybe suggests a better translation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have two options. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Take that input from end user, append it to my dataset and retrain the model. (I don't know whether it is even possible or not at production level).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Second is to reinforce that data into previous system. So far I only came to know about Online learning or Reinforcement learning (Q-learning, as my data is very small and even if user is training still not going to be in millions of sentences)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Am I on the right track, and how can I progress with either of these two options? Is there any prebuilt solution similar to this?&lt;/p&gt;&#xA;" OwnerUserId="29891" LastEditorUserId="29891" LastEditDate="2019-10-11T16:39:33.003" LastActivityDate="2019-10-11T16:39:33.003" Title="Best approach for online Machine Translation with few hundred of samples?" Tags="&lt;reinforcement-learning&gt;&lt;sequence-modelling&gt;&lt;machine-translation&gt;&lt;online-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15869" PostTypeId="2" ParentId="36" CreationDate="2019-10-11T17:31:26.013" Score="1" Body="&lt;p&gt;&lt;strong&gt;Direct Answer to Your Question&lt;/strong&gt;:--&lt;/p&gt;&#xA;&lt;p&gt;The field where quantum computing and A.I. intersect is called &lt;strong&gt;quantum machine learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A.I. is a developing field, with some background (ala McCarthy of LISP fame).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Quantum computing is a virgin field that is largely unexplored.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;A particular type of complexity interacts with another type of complexity to create a very rich field.&lt;/p&gt;&#xA;&lt;p&gt;Now combine (1) and (2), and you end up with even more uncertainty; the technical details shall be explored in this answer.&lt;/p&gt;&#xA;&lt;p&gt;Google Explains Quantum Computing in One Simple Video: &lt;a href=&quot;https://www.youtube.com/watch?v=CMdHDHEuOUE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google and NASA's Quantum Artificial Intelligence Lab&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Body&lt;/strong&gt;:--&lt;/p&gt;&#xA;&lt;p&gt;IBM is an authority:--&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://futurism.com/the-byte/ibm-problems-quantum-computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM: Quantum Computers Could Be Useful, But We Don't Know Exactly How&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Quantum machine learning is an interesting phenomenon. This field studies the intersection between quantum computing and machine learning.&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Quantum_machine_learning&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;While machine learning algorithms are used to compute immense quantities of data, quantum machine learning increases such capabilities intelligently, by creating opportunities to conduct analysis on quantum states and systems.&amp;quot; Wikipedia contributors. — &amp;quot;Quantum machine learning.&amp;quot; &lt;em&gt;Wikipedia, The Free Encyclopedia&lt;/em&gt;. Wikipedia, The Free Encyclopedia, 7 Oct. 2019. Web. 11 Oct. 2019.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Technical Mirror&lt;/strong&gt;:--&lt;/p&gt;&#xA;&lt;p&gt;This particular section on the implementations is worth noting:--&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_machine_learning#Implementations_and_experiments&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Quantum_machine_learning#Implementations_and_experiments&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; ... This dependence on data is a powerful training tool. But it comes with potential pitfalls. If machines are trained to find and exploit patterns in data then, in certain instances, they only perpetuate the race, gender or class prejudices specific to current human intelligence.&lt;/p&gt;&#xA;&lt;p&gt;But the data-processing facility inherent to machine learning also has the potential to generate applications that can improve human lives. 'Intelligent' machines could help scientists to more efficiently detect cancer or better understand mental health.&lt;/p&gt;&#xA;&lt;p&gt;Most of the progress in machine learning so far has been classical: the techniques that machines use to learn follow the laws of classical physics. The data they learn from has a classical form. The machines on which the algorithms run are also classical.&lt;/p&gt;&#xA;&lt;p&gt;We work in the emerging field of quantum machine learning, which is exploring whether the branch of physics called quantum mechanics might improve machine learning. Quantum mechanics is different to classical physics on a fundamental level: it deals in probabilities and makes a principle out of uncertainty. Quantum mechanics also expands physics to include interesting phenomena which cannot be explained using classical intuition. ... &amp;quot; — &amp;quot;Explainer: What Is Quantum Machine Learning And How Can It Help Us?&amp;quot;. &lt;em&gt;Techxplore.Com&lt;/em&gt;, 2019, &lt;a href=&quot;https://techxplore.com/news/2019-04-quantum-machine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://techxplore.com/news/2019-04-quantum-machine.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8301126&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Future with Quantum Machine Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.datasciencecentral.com/profiles/blogs/quantum-computing-deep-learning-and-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Computing, Deep Learning, and Artificial Intelligence&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Business Applications and Practical Uses&lt;/strong&gt;:--&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.forbes.com/sites/forbestechcouncil/2019/10/07/is-your-it-department-prepared-for-the-next-wave-of-enterprise-tech-2/#2d1ce7bb5787&quot; rel=&quot;nofollow noreferrer&quot;&gt;Is Your IT Department Prepared For The Next Wave Of Enterprise Tech?&lt;/a&gt; (Quantum Computing is mentioned here.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.dwavesys.com/press-releases/d-wave-announces-quadrant-machine-learning-business-unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;D-Wave Announces Quadrant Machine Learning Business Unit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Further Reading&lt;/strong&gt;:--&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://techxplore.com/news/2019-04-quantum-machine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://techxplore.com/news/2019-04-quantum-machine.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://physics.aps.org/articles/v12/74?fbclid=IwAR2hVTFReQA-3lTNQXKEAtQN7KQ5Lz41wyM19DJDtS1H4fLDNivqxqh5G2k&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://physics.aps.org/articles/v12/74?fbclid=IwAR2hVTFReQA-3lTNQXKEAtQN7KQ5Lz41wyM19DJDtS1H4fLDNivqxqh5G2k&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.forbes.com/sites/bernardmarr/2017/09/05/how-quantum-computers-will-revolutionize-artificial-intelligence-machine-learning-and-big-data/#59b153bf5609&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.forbes.com/sites/bernardmarr/2017/09/05/how-quantum-computers-will-revolutionize-artificial-intelligence-machine-learning-and-big-data/#59b153bf5609&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;http://www.messagetoeagle.com/what-is-quantum-machine-learning-and-how-can-it-help-us/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.messagetoeagle.com/what-is-quantum-machine-learning-and-how-can-it-help-us/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-10-13T07:18:56.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15870" PostTypeId="2" ParentId="15820" CreationDate="2019-10-11T19:20:12.913" Score="2" Body="&lt;p&gt;There are some research at least on the &quot;foolability&quot; of neural networks, that gives insight on potential high risk of neural nets even when they &quot;seem&quot; 99.99% acurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very good paper on this is in Nature: &lt;a href=&quot;https://www.nature.com/articles/d41586-019-03013-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/d41586-019-03013-5&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nutshell: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It shows diverse exemples of fooling neural networks/AIs, for exemple one where a few bits of scotch tape places on a &quot;Stop&quot; sign changes it, for the neural net, into a &quot;limited to 40&quot; sign... (whereas a human would still see a &quot;Stop&quot; sign!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also 2 striking exemples of turning an animal into another by just adding invisible (for humans!) colored dots, (turning in the exemple a Panda into a Gibbon, where a human hardly see anything different so still sees a Panda). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then they elaborate on diverse research venues, involving for exemple ways to try to prevent such attacks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole page is a good read to any AI researcher and shows lots of troubling problems (especially for automated systems such as cars, and soon maybe armaments).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;An exerpt relevant to the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Hendrycks and his colleagues have suggested quantifying a DNN’s robustness against making errors by testing how it performs against a large range of adversarial examples. However, training a network to withstand one kind of attack could weaken it against others, they say. And researchers led by Pushmeet Kohli at Google DeepMind in London are trying to inoculate DNNs against making mistakes. Many adversarial attacks work by making tiny tweaks to the component parts of an input — such as subtly altering the colour of pixels in an image — until this tips a DNN over into a misclassification. Kohli’s team has suggested that a robust DNN should not change its output as a result of small changes in its input, and that this property might be mathematically incorporated into the network, constraining how it learns.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;For the moment, however, no one has a fix on the overall problem of brittle AIs. The root of the issue, says Bengio, is that DNNs don’t have a good model of how to pick out what matters. When an AI sees a doctored image of a lion as a library, a person still sees a lion because they have a mental model of the animal that rests on a set of high-level features — ears, a tail, a mane and so on — that lets them abstract away from low-level arbitrary or incidental details. “We know from prior experience which features are the salient ones,” says Bengio. “And that comes from a deep understanding of the structure of the world.”&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another excerpt, near the end:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&quot;Researchers in the field say they are making progress in fixing deep learning’s flaws, but acknowledge that they’re still groping for new techniques to make the process less brittle. There is not much theory behind deep learning, says Song. “If something doesn’t work, it’s difficult to figure out why,” she says. “The whole field is still very empirical. You just have to try things.”&quot;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="2233" LastActivityDate="2019-10-11T19:20:12.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15872" PostTypeId="2" ParentId="15859" CreationDate="2019-10-12T07:56:30.823" Score="0" Body="&lt;p&gt;I would argue definitely since it is a bit sequential. (i) Start off applying basic machine learning concepts such as regression, classification and generalization techniques etc. to real world problems. (ii) you will soon realize the limitations of those techniques.(iii) Take your learning to the next level by learning and applying deep learning concepts specially if the issues are around image classification or NLP. As mentioned by @Joachim Wagner you will not only miss out on useful info but there will a huge gap in our learning. Hence, i would suggest learning concurrently or ML first otherwise DL will become black box of black box. &lt;/p&gt;&#xA;" OwnerUserId="30157" LastActivityDate="2019-10-12T07:56:30.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15873" PostTypeId="1" AcceptedAnswerId="15878" CreationDate="2019-10-12T08:25:02.070" Score="1" ViewCount="126" Body="&lt;p&gt;I was reading a PyTorch code then I saw this learning rate scheduler:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def warmup_lr_scheduler(optimizer, warmup_iters, warmup_factor):&#xA;    &quot;&quot;&quot;&#xA;    Learning rate scheduler&#xA;    :param optimizer:&#xA;    :param warmup_iters:&#xA;    :param warmup_factor:&#xA;    :return:&#xA;    &quot;&quot;&quot;&#xA;    def f(x):&#xA;        if x &amp;gt;= warmup_iters:&#xA;            return 1&#xA;        alpha = float(x) / warmup_iters&#xA;        return warmup_factor * (1 - alpha) + alpha&#xA;&#xA;    return torch.optim.lr_scheduler.LambdaLR(optimizer, f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this is where the function is called:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if epoch == 0:&#xA;    warmup_factor = 1. / 1000&#xA;    warmup_iters = min(1000, len(data_loader) - 1)&#xA;&#xA;    lr_scheduler = utils.warmup_lr_scheduler(optimizer, warmup_iters, warmup_factor)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As I understood it gradually increase learning rate until it reach initial learning rate. Am I correct? Why we need to increase learning rate? As I know for better learning in Neural Networks we decrease learning rate. &lt;/p&gt;&#xA;" OwnerUserId="10051" LastEditorUserId="2444" LastEditDate="2019-10-12T16:04:27.697" LastActivityDate="2019-10-12T16:04:27.697" Title="Is this learning rate schedule increasing the learning rate?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;pytorch&gt;&lt;learning-rate&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15874" PostTypeId="1" CreationDate="2019-10-12T08:42:37.123" Score="1" ViewCount="39" Body="&lt;p&gt;I am working as a Data Scientist in a non IT company(in fortune 500) and the group CEO is visiting the Data Science department after it's inception a few months back. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have models like chrun prediction, recommendation engine etc. Can someone suggest what visualization tools can be used to present the results to CEO who I'm not sure is technical enough to show the code?&lt;/p&gt;&#xA;" OwnerUserId="30421" LastEditorUserId="1671" LastEditDate="2019-10-12T22:26:08.533" LastActivityDate="2019-10-12T22:26:08.533" Title="Showing machine learning results to Group CEO" Tags="&lt;machine-learning&gt;&lt;data-visualization&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2021-01-22T00:36:23.527" ContentLicense="CC BY-SA 4.0" />
  <row Id="15875" PostTypeId="1" CreationDate="2019-10-12T10:52:30.437" Score="1" ViewCount="308" Body="&lt;p&gt;I am new to neural networks. I would like to use them as a fitting or forecasting method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple NN model that does not contain hidden layers, that is, the input nodes are directly connected to the outputs nodes, represents a linear model.  Nonlinearity begins to appear in an ANN model when we have hidden nodes, in which a nonlinear function is assigned to the hidden nodes, and using minimization their weights are determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we choose the non-linear activation function that should be assigned to each hidden neuron?&lt;/p&gt;&#xA;" OwnerUserId="27312" LastEditorUserId="2444" LastEditDate="2019-10-13T00:14:30.943" LastActivityDate="2019-10-13T00:14:30.943" Title="How do we choose the activation function for each hidden node?" Tags="&lt;neural-networks&gt;&lt;linear-regression&gt;&lt;hidden-layers&gt;&lt;regression&gt;&lt;forecasting&gt;" AnswerCount="2" CommentCount="2" ClosedDate="2019-10-14T04:41:33.000" ContentLicense="CC BY-SA 4.0" />
  <row Id="15876" PostTypeId="2" ParentId="15875" CreationDate="2019-10-12T14:03:00.303" Score="1" Body="&lt;p&gt;To know the form of your non-linear function,  firstly you should define the type of problem you are dealing with such as an image classification task. Secondly, pick the activation functions based on your task such as sigmoid, Tanh, ReLu, LeadyRelu, Softmax etc. Overall, your ANN performance mainly depends on the number of hidden layers (hidden units), selection of activation functions,  weight decay, momentum and dropout etc. &lt;/p&gt;&#xA;" OwnerUserId="30157" LastActivityDate="2019-10-12T14:03:00.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15877" PostTypeId="1" AcceptedAnswerId="15879" CreationDate="2019-10-12T15:02:46.907" Score="5" ViewCount="1572" Body="&lt;p&gt;I'm certain that this is a very naive question, but I am just beginning to look more deeply at neural networks, having only used decision tree approaches in the past.  Also, my formal mathematics training is more than 30 years in the past, so please be kind. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I'm reading François Chollet's book on &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1617294438&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning with Python&lt;/a&gt;, I'm struck that it appears that we are effectively treating the weights (kernel and biases) as terms in the standard linear equation (&lt;span class=&quot;math-container&quot;&gt;$y=mx+b$&lt;/span&gt;). At page 72 of the book, the author writes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;output = dot(W, input) + b&#xA;output = (output &amp;lt; 0 ? 0 : output)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Am I reading too much into this, or is this correct (and so fundamental I shouldn't be asking about it)?&lt;/p&gt;&#xA;" OwnerUserId="30426" LastEditorUserId="32410" LastEditDate="2021-05-09T23:50:18.120" LastActivityDate="2021-05-09T23:50:18.120" Title="Do neurons of a neural network model a linear relationship?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;neurons&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15878" PostTypeId="2" ParentId="15873" CreationDate="2019-10-12T15:50:27.620" Score="1" Body="&lt;p&gt;The higher (or smaller) the learning rate, the higher (or, respectively, smaller) the contribution of the gradient of the objective function, with respect to the parameters of the model, to the new parameters of the model. Therefore, if you progressively increase (or decrease) the learning rate, then you will accelerate (or, respectively, slow down) the learning process, so later training examples have higher (or, respectively, smaller) influence on the parameters of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, the function &lt;code&gt;warmup_lr_scheduler&lt;/code&gt; returns an object of class &lt;code&gt;LambdaLR&lt;/code&gt;, initialized with a certain optimizer and the function &lt;code&gt;f&lt;/code&gt;, which is defined as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(x):&#xA;    if x &amp;gt;= warmup_iters:&#xA;        return 1&#xA;    alpha = float(x) / warmup_iters&#xA;    return warmup_factor * (1 - alpha) + alpha&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The documentation of &lt;code&gt;torch.optim.lr_scheduler.LambdaLR&lt;/code&gt; says that the function &lt;code&gt;f&lt;/code&gt; should compute a multiplicative factor given an integer parameter epoch, so &lt;code&gt;x&lt;/code&gt; is a training epoch. If the epoch &lt;code&gt;x&lt;/code&gt; is greater than or equal to &lt;code&gt;warmup_iters&lt;/code&gt;, then 1 is returned, but anything multiplied by 1 is itself, so, when the epoch &lt;code&gt;x&lt;/code&gt; is greater than a threshold, &lt;code&gt;warmup_iters&lt;/code&gt; (e.g. 1000), then the initial learning rate is unaffected. However, when &lt;code&gt;x &amp;lt; warmup_iters&lt;/code&gt;, the multiplicative factor is given by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alpha = float(x) / warmup_iters&#xA;warmup_factor * (1 - alpha) + alpha&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is a function of the epoch &lt;code&gt;x&lt;/code&gt;. The higher the epoch &lt;code&gt;x&lt;/code&gt;, the higher the value of &lt;code&gt;alpha&lt;/code&gt;, so the smaller &lt;code&gt;(1 - alpha)&lt;/code&gt; and &lt;code&gt;warmup_factor * (1 - alpha)&lt;/code&gt;. Note that &lt;code&gt;float(x) / warmup_iters&lt;/code&gt; will never be greater than 1 because &lt;code&gt;x&lt;/code&gt; is never greater than &lt;code&gt;warmup_iters&lt;/code&gt;. So, effectively, as the epoch increases, &lt;code&gt;warmup_factor * (1 - alpha)&lt;/code&gt; tends to 0 and &lt;code&gt;alpha&lt;/code&gt; tends to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The learning rate can only increase if you multiply it with a constant greater than 1. However, this can only happen if &lt;code&gt;warmup_factor &amp;gt; 1&lt;/code&gt;. You can verify this by solving the inequality &lt;code&gt;warmup_factor * (1 - alpha) + alpha &amp;gt; 1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the initial learning rate is not being increased, but the learning process starts with a smaller learning rate than the given learning rate, for a &lt;code&gt;warmup_iters&lt;/code&gt; epochs, then, after &lt;code&gt;warmup_iters&lt;/code&gt; epochs, it uses the initially given learning rate (e.g. 0.002).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-12T15:56:17.190" LastActivityDate="2019-10-12T15:56:17.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15879" PostTypeId="2" ParentId="15877" CreationDate="2019-10-12T16:23:44.157" Score="9" Body="&lt;p&gt;In a neural network (NN), a neuron can act as a linear operator, but it usually acts as a non-linear one. The usual equation of a neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; of an NN is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$o_i^l = \sigma(\mathbf{x}_i^l \cdot  \mathbf{w}_i^l + b_i^l),$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is a so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;activation function&lt;/em&gt;&lt;/a&gt;, which is usually a non-linearity, but it can also be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Identity_function&quot; rel=&quot;noreferrer&quot;&gt;identity function&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i^l$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}_i^l$&lt;/span&gt; are the vectors that respectively contain the inputs and the weights for neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$b_i^l \in \mathbb{R}$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_bias&quot; rel=&quot;noreferrer&quot;&gt;bias&lt;/a&gt;. Similarly, the output of a layer of a feed-forward neural network (FFNN) is computed as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{o}^l = \sigma(\mathbf{X}^l \mathbf{W}^l + \mathbf{b}^l).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your specific example, you set the new weight to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, if the output of the linear combination is less than &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, else you use the output of the linear combination. This is the definition of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;noreferrer&quot;&gt;ReLU activation function&lt;/a&gt;, which is a non-linear function.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-12T16:34:19.923" LastActivityDate="2019-10-12T16:34:19.923" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="15880" PostTypeId="5" CreationDate="2019-10-12T16:47:36.807" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-12T16:47:36.807" LastActivityDate="2019-10-12T16:47:36.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15881" PostTypeId="4" CreationDate="2019-10-12T16:47:36.807" Score="0" Body="For questions related to the concept of learning rate (of an optimization algorithm, such as gradient descent) in machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-12T16:47:36.807" LastActivityDate="2019-10-12T16:47:36.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15882" PostTypeId="1" CreationDate="2019-10-12T17:12:37.553" Score="1" ViewCount="21" Body="&lt;p&gt;I've been exploring word-level alignments tools such as MGIZA and it seems to me that there hasn't been any new tool for this problem. Are neural networks not suitable to solve this problem or simply no interest in the area to build new tools?&lt;/p&gt;&#xA;" OwnerUserId="30428" LastActivityDate="2019-10-12T17:12:37.553" Title="Why hasn't deep learning been used for word level alignment?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15883" PostTypeId="1" CreationDate="2019-10-12T17:38:49.073" Score="0" ViewCount="24" Body="&lt;p&gt;Gday guys,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i have this idea in my mind for quite a while. I want to teach an AI to play Mario Kart on the NDS, which can hopefully beat me and my friends one day. Iam familiar with the theoretical stuff behind the learning (at least I think so) But iam wondering how to actual train the model in a practical way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can capture the pixel data with python and process the image. To get a better understanding of what is going on I can also use 3-4 frames as one state of the environment. This state can be given to the agents which is then taking an action and is getting the reward from the environment. Sounds simple but do I have to pause the emulation for each training step? And how do I check the outcome of the training. To I just feed the game pixels to the algorithm and hope that it will do something?&lt;/p&gt;&#xA;" OwnerUserId="30431" LastActivityDate="2019-10-12T17:38:49.073" Title="Ideas on how to train an AI to play Mario Kart with the DeSmuME Emulator" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15884" PostTypeId="2" ParentId="15875" CreationDate="2019-10-12T17:50:20.937" Score="1" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;:One does not know ahead of time what hyper-parameters will achieve optimal performance. So what you need is an &lt;em&gt;iterative&lt;/em&gt; implementation strategy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Implementation Strategy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When working with neural networks it is key to make sure that you spend your time wisely. It is possible to spend lots of time on a dead end simply because you made an assumption about your model at the very beginning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when selecting activation functions and other hyper parameters don't over think things. That is, get a quick and dirty model up and running and tune from there. From this model you can iterate. For example, you could start with ReLU activations in the hidden layers and as you tune your model you could experiment with other other activations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the data and the task at hand along with your tuning shape your model. A highly recommended video on this is A. Ng's lecture &lt;a href=&quot;https://www.youtube.com/embed/JUJNGv_sb4Y&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.coursera.org/lecture/deep-neural-network/tuning-process-dknSn&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; from the A. Ng deep learning specialization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some content not in the video is how to use &lt;a href=&quot;https://www.ritchieng.com/machinelearning-learning-curve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;learning curves&lt;/a&gt; to help define your iterations. These help you decide what you should do next when your model is not achieving desired performance. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-10-12T17:56:48.653" LastActivityDate="2019-10-12T17:56:48.653" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15885" PostTypeId="1" AcceptedAnswerId="16859" CreationDate="2019-10-12T18:53:23.077" Score="4" ViewCount="46" Body="&lt;p&gt;When training a deep network to learn object classification from a set like ImageNet, we minimize the cross entropy between the ground truth and the predicted  categories. This is done in a supervised way. It is my understanding that you can separate categories in an unsupervised way using principal component analysis, but I have never seen that in a deep network. I am curious if this can be done easily in the last case. One possible way to do this would be to minimize a loss that favors categorization into one-hot vectors (this would only guarantee that an image is classified into a single category, rather that the correct category, though). Has this been done, or is there any reason why not?&lt;/p&gt;&#xA;" OwnerUserId="30433" LastActivityDate="2019-12-30T01:01:10.150" Title="Are there methods that allow deep networks to learn object categorization in a self-supervised way?" Tags="&lt;deep-learning&gt;&lt;object-recognition&gt;&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15886" PostTypeId="2" ParentId="15877" CreationDate="2019-10-12T21:17:19.920" Score="3" Body="&lt;p&gt;Taking the question from &lt;a href=&quot;https://ai.stackexchange.com/questions/15877/do-neurons-of-a-neural-network-model-a-linear-relationship/15886#comment24003_15879&quot;&gt;comments&lt;/a&gt; on nbro's answer.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I wrong to see a clear relationship between how we are currently training networks and the classic function that defines a line? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You are right about it. This is an intuitive way to understand neural networks. You can create a neural network that only does simple linear regression, by using linear activations functions in all the layers, such as the neural network (model) output is a linear combination of the inputs. And, this seems like a great way to introduce neural networks to students. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, one must also look at the fact that neural networks provide the flexibility to model many kinds of non-linear relationships.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://stats.stackexchange.com/questions/115258/comprehensive-list-of-activation-functions-in-neural-networks-with-pros-cons/&quot;&gt;list of activation functions&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-10-15T07:08:00.847" LastActivityDate="2019-10-15T07:08:00.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15888" PostTypeId="2" ParentId="15877" CreationDate="2019-10-13T04:07:24.240" Score="4" Body="&lt;p&gt;Almost never. The sum of linear functions is another linear function, so if neurons were only linear transformations there would be basically no point to having more than one neuron per layer. Instead, every neuron applies some kind of &lt;em&gt;nonlinear&lt;/em&gt; function to its input. There are lots of different variations, but in the end the combination of the nonlinear activation function at each layer with the linear matrix multiplication connecting the outputs of each layer to the inputs of the next, creates something that has much more intricate behavior while still being reasonably efficient to compute.&lt;/p&gt;&#xA;" OwnerUserId="30438" LastActivityDate="2019-10-13T04:07:24.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15890" PostTypeId="1" AcceptedAnswerId="15896" CreationDate="2019-10-13T11:36:41.637" Score="3" ViewCount="82" Body="&lt;p&gt;I am wondering where Google uses the result from deep learning of reCaptcha (how can a system that knows to recognize street signs is useful somewhere? how they profit from it?)&lt;/p&gt;&#xA;" OwnerUserId="30443" LastEditorUserId="2444" LastEditDate="2019-10-13T14:47:53.440" LastActivityDate="2019-10-13T22:51:54.780" Title="How is the reCaptcha useful for Google?" Tags="&lt;intelligence-testing&gt;&lt;google&gt;&lt;captcha&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15891" PostTypeId="5" CreationDate="2019-10-13T12:08:17.667" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-13T12:08:17.667" LastActivityDate="2019-10-13T12:08:17.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15892" PostTypeId="4" CreationDate="2019-10-13T12:08:17.667" Score="0" Body="For questions related to CAPTCHA, an acronym for &quot;Completely Automated Public Turing test to tell Computers and Humans Apart&quot;, which is a type of challenge-response test used in computing to determine whether or not the user is human." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-13T12:08:17.667" LastActivityDate="2019-10-13T12:08:17.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15894" PostTypeId="1" CreationDate="2019-10-13T15:29:01.817" Score="1" ViewCount="74" Body="&lt;p&gt;I have a car with 8 lidars, each with a field of view of 60 degrees. My car looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Swmo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Swmo.jpg&quot; alt=&quot;Car&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I merge all the lidar readings into 1 point cloud?&lt;/p&gt;&#xA;" OwnerUserId="29708" LastEditorUserId="2444" LastEditDate="2019-10-20T15:11:30.993" LastActivityDate="2019-10-20T16:27:44.307" Title="How can I combine the readings of multiple lidars into 1 point cloud?" Tags="&lt;autonomous-vehicles&gt;&lt;robots&gt;&lt;signal-processing&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2021-01-22T00:35:56.597" ContentLicense="CC BY-SA 4.0" />
  <row Id="15896" PostTypeId="2" ParentId="15890" CreationDate="2019-10-13T22:51:54.780" Score="0" Body="&lt;p&gt;At this time, Google is aggressively pursuing research into AI of many sorts. All AI at this point is constrained by the size and accuracy of available training data. Getting human curated data is expensive and time consuming. Crowdsourcing through Captcha gets access to that without paying directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why roads specifically? It can hardly be a coincidence that Google's holding company also owns a self driving car company Waymo. They need machines to learn what road signs look like in real life, so that they can respond to them. &lt;/p&gt;&#xA;" OwnerUserId="23413" LastActivityDate="2019-10-13T22:51:54.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15897" PostTypeId="1" CreationDate="2019-10-14T01:28:18.200" Score="1" ViewCount="40" Body="&lt;p&gt;I was watching a video of policy gradient by Andrej Karpathy at &lt;a href=&quot;https://youtu.be/tqrcjHuNdmQ?t=604&quot; rel=&quot;nofollow noreferrer&quot;&gt;10:00&lt;/a&gt; there shows an equation for supervised learning for image classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$max\sum _{i}log \:p(y_i|x_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have worked with image classification models before but I always used a minimizing cost function aka loss function. I also never seen some one using maximizing cost function for image classification in the wild. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why image classification tasks are dominated by loss functions? &lt;/li&gt;&#xA;&lt;li&gt;What is the advantages of a minimizing loss function  over a maximizing&#xA;loss function in image classification?&lt;/li&gt;&#xA;&lt;li&gt;Other than RL where else maximizing cost functions are used?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2019-10-14T05:18:08.127" LastActivityDate="2019-10-14T05:18:08.127" Title="Why is image classification tasks are dominated by minimizing cost function instead of maximizing ones?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;theory&gt;&lt;loss-functions&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15898" PostTypeId="2" ParentId="8689" CreationDate="2019-10-14T02:11:40.007" Score="1" Body="&lt;p&gt;It depends on the used network as well as the feeding mechanism but let's give an example; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When working with LSTM, giving &lt;strong&gt;the time data&lt;/strong&gt; (as an integer sequence) in addition to the time-series data(coming from features) dramatically increases the performance of the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;b&gt;[&lt;span class=&quot;math-container&quot;&gt;$X_{0}$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$X_{1}$&lt;/span&gt;, ...] &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; [[&lt;span class=&quot;math-container&quot;&gt;$X_{0}$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$t_{0}$&lt;/span&gt;],&lt;span class=&quot;math-container&quot;&gt;$[X_{1}$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$t_{1}$&lt;/span&gt;], ...]&lt;/b&gt;&#xA;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you go and look for the &lt;strong&gt;kaggle&lt;/strong&gt; competition winner's notebooks, they do also &lt;em&gt;create additional features&lt;/em&gt; based on the featured data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that the performance is already &lt;strong&gt;quite high&lt;/strong&gt; on the three features so that you can predict those three features with high reliability. &#xA;It would only make sense to increase the number of features if you would like to predict additional features!  &lt;/p&gt;&#xA;" OwnerUserId="30351" LastActivityDate="2019-10-14T02:11:40.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15899" PostTypeId="2" ParentId="15897" CreationDate="2019-10-14T04:22:04.797" Score="1" Body="&lt;p&gt;There is really no difference between minimizing a cost function and maximizing a value function.  One can be the reciprocal of the other, or the negative of the other, for example.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-10-14T04:22:04.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15900" PostTypeId="1" AcceptedAnswerId="15919" CreationDate="2019-10-14T04:33:39.283" Score="1" ViewCount="90" Body="&lt;p&gt;What are the standard (or baseline) problems (or at least common ones) for CNNs and LSTMs? As an example, for a feed-forward neural net, a common problem is the XOR problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a standard problem like this for CNNs and LSTMs? I think for a CNN the standard test is to try it on MNIST, but I'm not sure of an LSTM.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2019-10-14T23:29:30.357" LastActivityDate="2019-10-20T15:07:47.947" Title="What are the standard problems for CNNs and LSTMs?" Tags="&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15901" PostTypeId="2" ParentId="9105" CreationDate="2019-10-14T04:41:16.553" Score="1" Body="&lt;p&gt;In my experience, the fitness function is a way to define the goal of a genetic algorithm. It provides a way to compare how &quot;good&quot; two solutions are, for example, for mate selection and for deleting &quot;bad&quot; solutions from the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fitness function can also be a way to incorporate constraints, prior knowledge you may have about the shape of the fitness landscape, or the way your crossover/recombination operators will work in that fitness landscape.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the fitness function can include hard constraints like &quot;Genes x,y, and z must all stay on one side of the surface &lt;span class=&quot;math-container&quot;&gt;$Ax +By +Cz = k$&lt;/span&gt;&quot; by assigning the fitness value at zero if the gene values are on the wrong side of the surface.  However, it's often better in a case like that soften the boundary by assigning a fitness penalty that is zero at the surface and grows larger as the gene values move farther from the surface on the wrong side of the surface.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different fitness functions can be used for mate selection vs deleting &quot;bad&quot; trial solutions.  For example, &quot;mating fitness&quot; between two potential parents A and B can be a function of how different the two parents are.  By providing a mating advantage to pairs that are significantly different, the population can be forced to remain fairly diverse and thus explore a larger region of solution space, or to avoid converging to local (sub-optimum) fitness maxima.  Meanwhile, the usual kind of fitness will cull the low-fitness individuals from the population and drive evolution toward high fitness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is often much more important is the set of variables (&quot;genes&quot;) used to represent a trial solution, how the genes are arranged in the &quot;chromosome&quot;, and the ways genes from two parents can be combined to form a new trial solution. Since you didn't ask about those things I won't go into detail in this answer, but if you ask in a separate question I will provide a detailed answer.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastEditorUserId="28348" LastEditDate="2019-10-14T05:03:17.993" LastActivityDate="2019-10-14T05:03:17.993" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15902" PostTypeId="2" ParentId="15900" CreationDate="2019-10-14T05:32:32.847" Score="1" Body="&lt;p&gt;I think for LSTM, I have not come across a standard test, but when I started, I tried like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generate a sequence of numbers like, [0,1,2,3,4],[1,2,3,4,5]..... as the dataset and then your labels would be [5,6,.....].Train this using an LSTM network.This would be a good way to understand the parameters involved and by changing the number of layers, and different parameters you can easily check how it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;of course, mnist is the test for CNNs&lt;/p&gt;&#xA;" OwnerUserId="26854" LastActivityDate="2019-10-14T05:32:32.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15903" PostTypeId="1" CreationDate="2019-10-14T09:27:08.417" Score="3" ViewCount="1216" Body="&lt;p&gt;Recently, I always hear about the terms sim2sim, sim2real and real2real. Will anyone explain the meaning/motivation of these terms (in DL/RL research community)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the challenges in this research area?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything intuitive would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="30470" LastEditorUserId="2444" LastEditDate="2019-10-14T12:13:31.093" LastActivityDate="2019-10-14T12:13:31.093" Title="What are sim2sim, sim2real and real2real?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;robotics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15904" PostTypeId="2" ParentId="15903" CreationDate="2019-10-14T12:04:23.167" Score="2" Body="&lt;p&gt;The abbreviations &lt;em&gt;sim2sim, sim2real&lt;/em&gt; and &lt;em&gt;real2real&lt;/em&gt; refer to techniques that can be used to transfer knowledge from one environment (e.g. in simulation) to another one (e.g. in the real world).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sim2sim stands for simulation-to-simulation,&lt;/li&gt;&#xA;&lt;li&gt;sim2real stands for simulation-to-real, and &lt;/li&gt;&#xA;&lt;li&gt;real2real stands for real-to-real. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In sim2sim, knowledge acquired during one simulation is transferred to an agent (or robot) in another simulation. Similarly, in sim2real, knowledge acquired during the simulation is used in a real-world problem (or environment). Finally, in real2real, knowledge acquired in a real-world problem can be transferred to another agent in another real-world problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main challenges are related to the differences that exist between one environment and the other (either in simulation or in the real-world). For example, in &lt;a href=&quot;https://arxiv.org/pdf/1812.07252.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sim2real&lt;/a&gt;, the simulation is almost never a perfect model of the real-world environment, so an agent trained in a simulation will probably not behave optimally in the real-world environment, which is often a lot more complex than the simulated environment. However, it is often the case that a robot needs to be trained in simulation, given that a robot trained in a real-world environment is subject to crashes.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-14T12:10:01.993" LastActivityDate="2019-10-14T12:10:01.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15906" PostTypeId="2" ParentId="1997" CreationDate="2019-10-14T14:04:44.453" Score="0" Body="&lt;p&gt;Random residual networks for many non-linearities such as tanh live on the edge of chaos,&#xA;in that the cosine distance of two input vectors will converge to a fixed point at a polynomial rate, rather than an exponential rate, as with vanilla tanh networks. Thus a typical residual network will slowly cross the stable-chaotic boundary with depth, hovering around this boundary for many layers. Basically it does not “forget” the geometry of the input space “very quickly”. So even if we make them considerably deep, they work better the vanilla networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information on the propagation of information in residual networks - &lt;a href=&quot;https://ai.google/research/pubs/pub46492/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mean Field Residual Networks: On the Edge of Chaos&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30384" LastActivityDate="2019-10-14T14:04:44.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15908" PostTypeId="5" CreationDate="2019-10-14T15:21:21.570" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-14T15:21:21.570" LastActivityDate="2019-10-14T15:21:21.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15909" PostTypeId="4" CreationDate="2019-10-14T15:21:21.570" Score="0" Body="For questions related to residual networks (ResNets), introduced in &quot;Deep Residual Learning for Image Recognition&quot; (2015) by Kaiming He et al. and that won the first place at &quot;Large Scale Visual Recognition Challenge 2015&quot; (ILSVRC2015)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-14T15:21:21.570" LastActivityDate="2019-10-14T15:21:21.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15910" PostTypeId="1" AcceptedAnswerId="15911" CreationDate="2019-10-14T16:52:27.350" Score="3" ViewCount="677" Body="&lt;p&gt;I am trying to learn tabular Q learning by using a table of states and actions (i.e. no neural networks). I was trying it out on the &lt;a href=&quot;https://gym.openai.com/envs/FrozenLake-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FrozenLake&lt;/a&gt; environment. It's a very simple environment, where the task is to reach a &lt;code&gt;G&lt;/code&gt; starting from a source &lt;code&gt;S&lt;/code&gt; avoiding holes &lt;code&gt;H&lt;/code&gt; and just following the frozen path which is &lt;code&gt;F&lt;/code&gt;. The &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; FrozenLake grid looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SFFF&#xA;FHFH&#xA;FFFH&#xA;HFFG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am working with the slippery version, where the agent, if it takes a step, has an equal probability of either going in the direction it intends or slipping sideways perpendicular to the original direction (if that position is in the grid). Holes are terminal states and a goal is a terminal state.&lt;/p&gt;&#xA;&lt;p&gt;Now I first tried &lt;em&gt;value iteration&lt;/em&gt; which converges to the following set of values for the states&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[0.0688909  0.06141457 0.07440976 0.05580732 0.09185454 0. 0.11220821 0.         0.14543635 0.24749695 0.29961759 0. 0.         0.3799359  0.63902015 0.        ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also coded &lt;em&gt;policy iteration&lt;/em&gt;, and it also gives me the same result. So I am pretty confident that this value function is correct.&lt;/p&gt;&#xA;&lt;p&gt;Now, I tried to code the Q learning algorithm, here is my code for the Q learning algorithm&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_action(Q_table, state, epsilon):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Uses e-greedy to policy to return an action corresponding to state&#xA;    &#xA;    Args:&#xA;        Q_table: numpy array containing the q values&#xA;        state: current state&#xA;        epsilon: value of epsilon in epsilon greedy strategy&#xA;        env: OpenAI gym environment &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    return env.action_space.sample() if np.random.random() &amp;lt; epsilon else np.argmax(Q_table[state]) &#xA;&#xA;&#xA;def tabular_Q_learning(env):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Returns the optimal policy by using tabular Q learning&#xA;    &#xA;    Args:&#xA;        env: OpenAI gym environment&#xA;        &#xA;    Returns:&#xA;        (policy, Q function, V function) &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    &#xA;    # initialize the Q table&#xA;    # &#xA;    # Implementation detail: &#xA;    # A numpy array of |x| * |a| values&#xA;    &#xA;    Q_table = np.zeros((env.nS, env.nA))&#xA;    &#xA;    # hyperparameters&#xA;    epsilon = 0.9&#xA;    episodes = 500000&#xA;    lr = 0.81&#xA;&#xA;    &#xA;    for _ in tqdm_notebook(range(episodes)):&#xA;        # initialize the state&#xA;        state = env.reset()&#xA;        &#xA;        if episodes / 1000 &amp;gt; 21:&#xA;            epsilon = 0.1&#xA;        &#xA;        t = 0&#xA;        while True: # for each step of the episode&#xA;            # env.render()&#xA;            # print(observation)&#xA;         &#xA;            # choose a from s using policy derived from Q &#xA;            action = get_action(Q_table, state, epsilon) &#xA;            &#xA;            # take action a, observe r, s_dash&#xA;            s_dash, r, done, info = env.step(action)&#xA;            &#xA;            # Q table update &#xA;            Q_table[state][action] += lr * (r + gamma * np.max(Q_table[s_dash]) - Q_table[state][action])&#xA;            &#xA;            state = s_dash&#xA;            &#xA;            t += 1&#xA;            &#xA;            if done:&#xA;                # print(&amp;quot;Episode finished after {} timesteps&amp;quot;.format(t+1))&#xA;                break&#xA;        # print(Q_table)&#xA;    &#xA;    policy = np.argmax(Q_table, axis=1)&#xA;    V = np.max(Q_table, axis=1)&#xA;    &#xA;    return policy, Q_table, V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried running it and it converges to a different set of values which is following &lt;code&gt;[0.26426802 0.03656142 0.12557195 0.03075882 0.35018374 0. 0.02584052 0.         0.37657211 0.59209091 0.15439031 0. 0.         0.60367728 0.79768863 0.        ]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not getting, what is going wrong. The implementation of Q learning is pretty straightforward. I checked my code, it seems right.&lt;/p&gt;&#xA;&lt;p&gt;Any pointers would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="30479" LastEditorUserId="2444" LastEditDate="2020-11-01T16:49:25.543" LastActivityDate="2020-11-01T16:51:03.593" Title="Why is my implementation of Q-learning not converging to the right values in the FrozenLake environment?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;&lt;frozen-lake&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15911" PostTypeId="2" ParentId="15910" CreationDate="2019-10-15T00:56:09.393" Score="4" Body="&lt;p&gt;I was able to solve the problem.&lt;/p&gt;&#xA;&lt;p&gt;The main issue for non-convergence was that I was not decaying the learning rate appropriately. I put a decay rate of &lt;span class=&quot;math-container&quot;&gt;$-0.00005$&lt;/span&gt; on the learning rate &lt;code&gt;lr&lt;/code&gt;, and subsequently Q-Learning also converged to the same value as value iteration.&lt;/p&gt;&#xA;" OwnerUserId="30479" LastEditorUserId="2444" LastEditDate="2020-11-01T16:51:03.593" LastActivityDate="2020-11-01T16:51:03.593" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15912" PostTypeId="1" CreationDate="2019-10-15T01:00:15.977" Score="1" ViewCount="25" Body="&lt;p&gt;We usually categorize objects in a hierarchy of classes. Let us say crow vs bird. In addition, classes can be &quot;messy&quot;, for instance a crow can be also a predator, but not all birds are predators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, can deep networks represent these hierarchies easily? Has anybody studied that? (I could not find anything at all).&lt;/p&gt;&#xA;" OwnerUserId="30433" LastActivityDate="2019-10-15T09:27:35.840" Title="Are there deep networks that can differentiate object class from individual object?" Tags="&lt;classification&gt;&lt;deep-neural-networks&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15913" PostTypeId="2" ParentId="12324" CreationDate="2019-10-15T02:43:12.250" Score="2" Body="&lt;p&gt;Let's write down Fibonacci!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;K = 0 1 1 2 3 5 ...&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And another series that is derived from Fibo; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;X = 1 4 7 12 30 &lt;span class=&quot;math-container&quot;&gt;$X_{5}$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Guessing &lt;span class=&quot;math-container&quot;&gt;$X_{5}$&lt;/span&gt; is our task and both series are available to you (Fibonacci as an additional feature). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One unit that you feed with &lt;strong&gt;X&lt;/strong&gt; will try to capture the relation of &lt;span class=&quot;math-container&quot;&gt;$X_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{t-1}$&lt;/span&gt; &lt;em&gt;only&lt;/em&gt;;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X_{t}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$X_{t-1}$&lt;/span&gt; + &lt;span class=&quot;math-container&quot;&gt;$X_{t-2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, an additional unit that you insert&amp;amp;feed with &lt;strong&gt;K&lt;/strong&gt; will not only try to capture &lt;span class=&quot;math-container&quot;&gt;$K_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K_{t-1}$&lt;/span&gt; relation but also the relation between &lt;span class=&quot;math-container&quot;&gt;$K_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{t}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$K_{t}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$K_{t-1}$&lt;/span&gt; + &lt;span class=&quot;math-container&quot;&gt;$2*$&lt;/span&gt;&lt;span class=&quot;math-container&quot;&gt;$X_{t}$&lt;/span&gt; + &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example above, there is a clear correlation between &lt;span class=&quot;math-container&quot;&gt;$K_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{t}$&lt;/span&gt; (which is not always the case) and it will support the network to capture the sequential relation. Even the CORRELATION is not crystal clear, almost every additional feature data correlates with the other features and will support the network to grab the relation.&lt;/p&gt;&#xA;" OwnerUserId="30351" LastActivityDate="2019-10-15T02:43:12.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15914" PostTypeId="1" CreationDate="2019-10-15T07:14:42.473" Score="1" ViewCount="40" Body="&lt;p&gt;Sorry if this is a stupid question. I'm just starting out in ML and am working with gpt-2 for text generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My situation is that I have to generate text in a particular field for eg. family businesses, which pretrained gpt-2 is unlikely to have much &quot;training&quot; with. Besides the topic, I also need to generate the text in the style of one particular writer (for eg. incorporating their turns of phrase etc) This particular writer hasn't written much about the family business topic unfortunately, but has written about other topics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It occurred to me that I can take gpt-2, finetune it on a large corpus of material on family businesses, and then finetune the new model on the written material of the particular writer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this be the right way to achieve my objective of creating content on family businesses in the style of this particular writer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on what sort of stuff I should keep in mind while doing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30489" LastActivityDate="2019-10-15T07:14:42.473" Title="Finetuning GPT-2 twice for particular style of writing on a particular topic" Tags="&lt;natural-language-processing&gt;&lt;training&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15915" PostTypeId="2" ParentId="15912" CreationDate="2019-10-15T09:27:35.840" Score="2" Body="&lt;p&gt;In this case, you have an ontology and want to learn the ontology. There are many researches in this topic that you can find. However, the data could be the most challenging part. Some of the researches: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/920602&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ontology learning for the Semantic Web&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-540-24750-0_9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ontology Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, as these are some frameworks to ontology learning, you can use deep networks such as RNN‌ to learn the task.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-10-15T09:27:35.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15916" PostTypeId="1" CreationDate="2019-10-15T11:34:25.603" Score="2" ViewCount="97" Body="&lt;p&gt;I am trying to understand the best loss function to be used with a convolutional neural network. I came to know that we can &lt;a href=&quot;https://ai.stackexchange.com/questions/13324/what-is-the-best-loss-function-for-convolution-neural-network-and-autoencoder&quot;&gt;mix two loss functions&lt;/a&gt;. Can any body share in what case was it done and how?&lt;/p&gt;&#xA;" OwnerUserId="23737" LastEditorUserId="2444" LastEditDate="2019-10-20T15:04:31.700" LastActivityDate="2019-10-22T00:47:03.753" Title="When and how to use a mix of loss functions for back-propagation?" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15917" PostTypeId="1" CreationDate="2019-10-15T13:51:23.133" Score="1" ViewCount="24" Body="&lt;p&gt;I have been building a multilabel image classification model using inception v3, which uses images of size 299x299, I have been wondering what are the effects of feeding images of rectangular shapes for example (or arbitrary resolutions) on the performance of the model, and if I can define requirements for how the data should be to ensure optimal performance, what would those requirements be ?&#xA;Intuitively, I think that square images would perform better than rectangular images, is this true?&lt;/p&gt;&#xA;" OwnerUserId="23866" LastActivityDate="2019-10-15T13:51:23.133" Title="Resizing effects on image recognition" Tags="&lt;neural-networks&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15918" PostTypeId="1" AcceptedAnswerId="15920" CreationDate="2019-10-15T16:04:31.937" Score="3" ViewCount="314" Body="&lt;p&gt;I am working on a deep reinforcement learning problem. Throughout the episode, there is a small positive and negative reward for good or bad decisions. In the end, there is a huge reward for the completion of the episode. So, this reward function is quite sparse.&lt;/p&gt;&#xA;&lt;p&gt;This is my understanding of how DQN works. The neural network predicts quality values for each possible action that can be taken from a state &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;. Let us assume the predicted quality value for an action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$Q(S_1, A)$&lt;/span&gt;, and this action allows the agent to reach &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We now need the target quality value &lt;span class=&quot;math-container&quot;&gt;$Q_\text{target}$&lt;/span&gt;, so that using &lt;span class=&quot;math-container&quot;&gt;$Q(S_1, A)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q_\text{target}$&lt;/span&gt; the temporal difference can be calculated, and updates can be made to the parameters of the value network.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q_\text{target}$&lt;/span&gt; is composed of two terms. The immediate reward &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and the maximum quality value of the resulting state that this chosen action leaves us in, which can be denoted by &lt;span class=&quot;math-container&quot;&gt;$Q_\text{future} = \text{max}_a Q(S_2, a)$&lt;/span&gt;, which is in practice obtained by feeding the new state &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; into the neural network and choosing (from the list of quality values for each action) the maximum quality value. We then multiply the discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; with this &lt;span class=&quot;math-container&quot;&gt;$Q_\text{future}$&lt;/span&gt; and add it to the reward &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$Q_\text{target} = R + \gamma \text{max}_a Q(S_2, a) = R + \gamma Q_\text{future}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, let us assume the agent is in the penultimate state, &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;, and chooses the action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that leads him to the completion state, &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;, and gets a reward &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;How do we form the target value &lt;span class=&quot;math-container&quot;&gt;$Q_\text{target}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; now? Do we still include the &lt;span class=&quot;math-container&quot;&gt;$Q_\text{future}$&lt;/span&gt; term? Or is it only the reward in this case?&lt;/em&gt; I am not sure if &lt;span class=&quot;math-container&quot;&gt;$Q_\text{future}$&lt;/span&gt; even has meaning after reaching the final state &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;. So, I think that, for the final step, the target value must simply be the reward. Is this right?&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="2444" LastEditDate="2020-11-01T14:37:04.630" LastActivityDate="2020-11-01T14:37:14.337" Title="How do we compute the target value when the agent ends up in the terminal state?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15919" PostTypeId="2" ParentId="15900" CreationDate="2019-10-15T16:12:22.513" Score="2" Body="&lt;p&gt;It's more domain- or task-specific. There is no obvious baseline anymore because these models and this field has evolved into too large of an ecosystem. Nonetheless, I'll list a couple of notorious examples below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Image classification:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MNIST&lt;/li&gt;&#xA;&lt;li&gt;CIFAR  &lt;/li&gt;&#xA;&lt;li&gt;ImageNet&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Detection/segmentation:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PascalVOC  &lt;/li&gt;&#xA;&lt;li&gt;COCO  &lt;/li&gt;&#xA;&lt;li&gt;CityScapes  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pose estimation:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MPII  &lt;/li&gt;&#xA;&lt;li&gt;LEEDS&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Text classification:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;IMDB  &lt;/li&gt;&#xA;&lt;li&gt;yelp  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question answering:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SQuAD  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Translation:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;WMT&lt;/li&gt;&#xA;&lt;li&gt;IWSLT  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is just a taste. There are tons more both in each category and the number of categories, a good source is the &lt;a href=&quot;https://paperswithcode.com/sota&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papers with Code&lt;/a&gt; website.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, there is no single &lt;em&gt;standard&lt;/em&gt; problem, given that there are too many that all in one shape or form use CNNs or RNNs (or others). &lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-10-20T15:07:47.947" LastActivityDate="2019-10-20T15:07:47.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15920" PostTypeId="2" ParentId="15918" CreationDate="2019-10-15T17:39:56.637" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Now, let us assume the agent is in the penultimate state, &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;, and&#xA;chooses the action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that leads him to the completion state, &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;,&#xA;and gets a reward &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;How do we form the target value &lt;span class=&quot;math-container&quot;&gt;$Q_\text{target}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; now? Do we still include the &lt;span class=&quot;math-container&quot;&gt;$Q_\text{future}$&lt;/span&gt; term? Or is it only the reward&#xA;in this case?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your term &amp;quot;completion state&amp;quot; is commonly called &amp;quot;terminal state&amp;quot;. In a terminal state, there are no more actions to take, no more time steps, and no possibility to take any action. So, by definition, in your state &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;, the expected future reward is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Mathematically, this is often noted like &lt;span class=&quot;math-container&quot;&gt;$v(S_T) = 0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$q(S_T,\cdot) = 0$&lt;/span&gt; with the &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; standing for last time step of the episode, and dot standing in for the fact that no action need to be supplied, or the specific action value is not relevant. So, therefore using your terms, &lt;span class=&quot;math-container&quot;&gt;$Q_\text{future} = \text{max}_a Q(S_2, a) = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;That makes the equations work in theory, but does not explain what to do in &lt;em&gt;code&lt;/em&gt;. In practice in your code, you would do as you suggest and use &lt;em&gt;just the reward&lt;/em&gt; when calculating the TD target for &lt;span class=&quot;math-container&quot;&gt;$Q(S_1, A)$&lt;/span&gt;. This is typically done using an &lt;code&gt;if&lt;/code&gt; block around the &lt;code&gt;done&lt;/code&gt; condition e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if done:&#xA;  td_target = r&#xA;else:&#xA;  td_target = r + gamma * np.max(q_future_values) &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course, the details depend on how you have structured and named your variables. You will find code similar to this in most DQN implementations though.&lt;/p&gt;&#xA;&lt;p&gt;You should not really try to &lt;em&gt;learn&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$V(S_2)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$Q(S_2, A)$&lt;/span&gt;, or calculate TD target starting from &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;, because the result should be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; by definition.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-11-01T14:37:14.337" LastActivityDate="2020-11-01T14:37:14.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15921" PostTypeId="2" ParentId="15877" CreationDate="2019-10-15T18:07:29.330" Score="1" Body="&lt;p&gt;You're quite right in your interpretation, but I'll answer in two parts in order to avoid confusion with respect to activation functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part 1. (TLDR: a neurons weights is the normal vector of a hyperplane that divides input space in two parts. The neuron's preactivation is proportional to the distance of the input point to the plane.) Every artificial neuron learns a linear relationship between its inputs. The most recalled equation of a line is &lt;span class=&quot;math-container&quot;&gt;$y=m \cdot x+b$&lt;/span&gt;, but that's actually a very specific form that allows us going through values of X of the line and seeing to what values of Y it corresponds. A most general form would be &lt;span class=&quot;math-container&quot;&gt;$0=n \cdot y + m \cdot x + b$&lt;/span&gt;. This tells us that the line is formed by the points (X,Y) whose values make that series of calculations be zero. We can explore different values of (X,Y) and see that most of them give non-zero values, and that they give positive values at one side of the line and negative values at the other side. Only if you land just on the line it will give you a zero. This is a very important interpretation, because it's what allows neurons to find divisions of the input space (into a positive side and a negative side). Of course it probably won't be a 2d space, so it will be a hyperplane instead of a line, but I hope you get the idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part 2. However, if we only use linear transformations we couldn't learn non-linear functions. Here's where the activation function plays a very important role: it distorts the neuron's preactivation value (which is linear) in a non-linear way (what makes it a non-linear function). Activation functions have lots of bells and whistles, which are too much to write here, but you can start thinking about them as distortions applied to that distance of the input point to the neuron's hyperplane. The one you saw is called ReLU, and it basically truncates the negative values, thus focusing only on the positive side of the hyperplane (it may be interpreted as measuring how far the point has crossed a frontier).&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-10-15T18:07:29.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15922" PostTypeId="5" CreationDate="2019-10-15T20:13:27.157" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-15T20:13:27.157" LastActivityDate="2019-10-15T20:13:27.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15923" PostTypeId="4" CreationDate="2019-10-15T20:13:27.157" Score="0" Body="For questions related to the ExpectiMinimax algorithm (or tree), which is a variation of the minimax algorithm (or tree). In addition to &quot;min&quot; and &quot;max&quot; nodes of the traditional minimax tree, this variant has &quot;chance&quot; nodes (which take the expected value of a random event occurring), which are interleaved with the max and min nodes. The ExpectiMinimax was proposed in &quot;Game-playing and game-learning automata&quot; (1966) by Donald Michie." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-15T20:19:55.833" LastActivityDate="2019-10-15T20:19:55.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15924" PostTypeId="1" AcceptedAnswerId="15928" CreationDate="2019-10-15T20:52:04.270" Score="1" ViewCount="71" Body="&lt;p&gt;I have a game/simulation that takes a vector of encoded sequences of moves (up, down, left, right). Let's say that these are sequential step taken by an ant moving in a 2D space, starting from the origin. The moves are generated randomly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know for any game, if the ant gets farther than a certain distance y from the origin (although it might even be closer than y at the end of the game). I would like to classify games into &quot;ant gets further away than y&quot; with value of one, or zero for &quot;ant does not get further away than y&quot;. I don't need an AI for this task, I have set this objective as a training goal for myself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am able to tell if the last position is past y or not, using a regular feed forward network, I believe it is easier because it is as easy as summing up all the moves, regardless of the order. But to tell if the ant got past y and then got back, that still needs to return one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought I might be able to reach my objective through an RNN, encoding the moves as a sequence of one-hot encoded sequential directions to move towards. Currently, I am using one hidden layer (I tried with different sizes ranging from 10 to 100), backpropagating the loss only at the last step of a single training on a vector, but it seems like the RNN total loss doesn't decrease at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any obivious flaw in my simulation, or in the neural network model? Is there a category of problems this could belong to?&lt;/p&gt;&#xA;" OwnerUserId="30287" LastEditorUserId="30287" LastEditDate="2019-10-16T08:41:43.147" LastActivityDate="2019-10-16T08:41:43.147" Title="Recurrent Neural Network to track distance from origin" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15926" PostTypeId="2" ParentId="4301" CreationDate="2019-10-16T02:16:41.773" Score="1" Body="&lt;p&gt;The biggest issue here may be similarity to prior work. As for the benchmarks, benchmarks are a common means for comparing algorithms. What it means here would be to compare the end-result (your chosen goal) for each of the algorithms compared in a similar scenario, or a generated test-scenario. This will mean that you will have to utilize all chosen algorithms in your test-game-map software. To make it more original, shake it up a little with some creative/disruptive element which should lower the performance of the algorithm. Find some way to quantify how close the algorithms actually come to meeting the stated goal (or not meeting goal, like how far off they were) with and without the disruptive element.&lt;/p&gt;&#xA;" OwnerUserId="16959" LastActivityDate="2019-10-16T02:16:41.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15928" PostTypeId="2" ParentId="15924" CreationDate="2019-10-16T06:47:01.310" Score="0" Body="&lt;p&gt;This kind of problem does not really have a name other than &quot;toy problem&quot; since no-one needs to teach an AI to add up, multiply or divide* - there are already far more reliable and far faster ways to achieve that on any computer. What you are doing here is essentially vector addition, applying a distance metric then setting a true/false value based on a comparison. It would be 2 or 3 lines of code in most high-level programming languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks can learn any function though, so in theory what you want to do is possible. You should not expect results to be perfect, a statistical learner never actually learns the analytical form of a function or process, just the rough &quot;shape&quot; of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not done your experiment. However, your idea to use a RNN seems reasonable. With the details you have given, I can offer a few pieces of general advice:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a modern RNN gated architecture, either LSTM or GRU. That's because the point in the sequence of moves where you want to set a &quot;distance exceeded&quot; toggle could be many steps away from the end of a game. The simplest RNNs (with direct loop backs from output to input within a layer) can easily suffer from vanishing gradients in this situation, whilst LSTM and GRU architectures are designed to deal with it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generate a &lt;em&gt;lot&lt;/em&gt; of training data. You will need many examples of both categories before any neural network will home in on what is causing one class or the other to occur. The learning is based on statistics, not reason.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Take a look at &lt;a href=&quot;https://machinelearningmastery.com/learn-add-numbers-seq2seq-recurrent-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;related LTSM examples that learn to add binary numbers&lt;/a&gt;. Repeat those even simpler experiments first, then move on to your own problem. This will avoid some beginner mistakes with poor choices of hyper-parameters, bad implementations etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Humans of course do use intelligence, reasoning and logic to learn reliable procedures for addition, multiplication and division. No doubt someone could be interested in how an AI can replicate that, without starting from built-in capabilties of a CPU (which of course the humans designed and built those procedures into the system at a low level). However, that's at a higher level of AI research than we are dealing with here. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-16T07:05:53.267" LastActivityDate="2019-10-16T07:05:53.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15929" PostTypeId="1" CreationDate="2019-10-16T07:09:45.620" Score="1" ViewCount="171" Body="&lt;p&gt;I have recently started developing a chatbot with TensorFlow. I already have a model that receives a question and outputs an answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do I use Flask to integrate my Tensorflow chatbot model into a website?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please help me with this by suggesting blogs or your views on the above?&lt;/p&gt;&#xA;" OwnerUserId="30509" LastEditorUserId="2444" LastEditDate="2019-10-20T15:00:06.137" LastActivityDate="2019-10-20T15:00:06.137" Title="How can I use Flask to integrate a chatbot created with TensorFlow into a website?" Tags="&lt;tensorflow&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2021-01-22T00:33:58.200" ContentLicense="CC BY-SA 4.0" />
  <row Id="15930" PostTypeId="2" ParentId="15730" CreationDate="2019-10-16T09:56:43.067" Score="1" Body="&lt;p&gt;Its arguable if we humans understand infinity. We just create new concept to enplace old mathematics when we meet this problem.&#xA;In division by infinity machine can understand it the same way as we:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double* xd = new double;&#xA;*xd =...;&#xA;if (*xd/y&amp;lt;0.00...1){&#xA;int* xi = new int;&#xA;*xi = (double) (*xd);&#xA;delete xd;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If human thinks of infinity  - imagines just huge number in his/her current context. So key to writing algorithm is just finding a scale that AI is currently working with. And BTW this problem must ve been solved years ago. People designing float/double must ve been conscious what they were doing. Moving exponenta sign is linear operation in double.&lt;/p&gt;&#xA;" OwnerUserId="30520" LastEditorUserId="30520" LastEditDate="2019-10-16T17:09:45.207" LastActivityDate="2019-10-16T17:09:45.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15931" PostTypeId="2" ParentId="15730" CreationDate="2019-10-16T12:40:36.427" Score="0" Body="&lt;p&gt;I think the property humans have which computers do not, is some sort of parallel process that runs alongside every other thing they are thinking and tries to assign an importance weighting evaluation to everything you are doing. &#xA;If you ask a computer to run the program : &#xA;A = 1;&#xA;DO UNTIL(A&amp;lt;0)&#xA;  a=a+1;&#xA;END;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer will. &#xA;If you ask a human, another process interjects with &quot;I'm bored now... this is taking ages... I'm going to start a new parallel process to &lt;em&gt;examine the problem, project where the answer lies and look for a faster route to the answer&lt;/em&gt; ... &#xA;Then we discover that we are stuck in an infinite loop that will never be &quot;solved&quot;.. and interject with an interrupt that flags the issue, kills the boring process and goes to get a cup of tea :-) &#xA;Sorry if that is unhelpful. &lt;/p&gt;&#xA;" OwnerUserId="30524" LastActivityDate="2019-10-16T12:40:36.427" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15932" PostTypeId="1" CreationDate="2019-10-16T13:57:37.143" Score="5" ViewCount="148" Body="&lt;p&gt;Science Fiction has frequently shown AI to be a threat to the very existence of mankind. AI systems have often been the antagonists in many works of fiction, from &lt;a href=&quot;https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)&quot; rel=&quot;noreferrer&quot;&gt;2001: A Space Odyssey&lt;/a&gt; through to &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Terminator&quot; rel=&quot;noreferrer&quot;&gt;The Terminator&lt;/a&gt; and beyond.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Media seems to buy into this trope as well.  And in recent years we have had people like &lt;a href=&quot;https://www.cnbc.com/2018/03/13/elon-musk-at-sxsw-a-i-is-more-dangerous-than-nuclear-weapons.html&quot; rel=&quot;noreferrer&quot;&gt;Elon Musk warn us&lt;/a&gt; of the dangers of an impending AI revolution, stating that AI is more dangerous than nukes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And apparently experts think that we will be seeing this AI revolution &lt;a href=&quot;https://www.sciencealert.com/experts-think-this-is-how-long-we-have-before-ai-takes-all-of-our-jobs&quot; rel=&quot;noreferrer&quot;&gt;in the next 100 years&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from my (albeit limited) study of AI, I get the impression that they are all wrong. I am going to outline my understanding below, please correct me if I am wrong:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Firstly, all of these things seem to be confusing Artificial Intelligence with Artificial Consciousness.  AI is essentially a system to make intelligent decisions, whereas AC is more like the &quot;self-aware&quot; systems that are shown in science fiction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Not AI itself, but intelligence and intelligent decision making algorithms is something we've been working with and enhancing since before computers have been around.  Moving this over to an artificial framework is fairly easy.  However consciousness is still something we are learning about.  My guess is we wont be able to re-create something artificially if we barely understand how it works in the real world.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;So my conclusion is that no AI system will be able to learn enough to start thinking for itself, and that all our warnings of AI are completely unjustified.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The real danger comes from AC, which we are a long, long way from realizing because we are still a long way off from defining exactly what consciousness is, let alone understanding it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So my question is, assuming that my understanding is correct, are any efforts are being made by companies or organizations that work with AI to correct these popular misunderstandings in sci-fi, the media and/or the public?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or are the proponents of AI ambivalent towards this public fear-mongering?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the fear mongering is going to remain popular for some time, as bad news sells better than good news.  I am just wondering if the general attitude from AI organizations is to ignore this popular misconception, or whether a concerted effort is being made to fight against these AI myths (but unfortunately nobody in the media is listening or cares).&lt;/p&gt;&#xA;" OwnerUserId="30526" LastActivityDate="2019-10-16T18:39:40.993" Title="&quot;AI will kill us all! The machines will rise up!&quot; - what is being done to dispel such myths?" Tags="&lt;ai-basics&gt;&lt;artificial-consciousness&gt;&lt;popular-culture&gt;" AnswerCount="1" CommentCount="11" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15934" PostTypeId="2" ParentId="15730" CreationDate="2019-10-16T17:13:00.730" Score="1" Body="&lt;p&gt;Well -- just to touch on the question of people and infinity -- my father has been a mathematician for 60 years. Throughout this time, he's been the kind of geek who prefers to talk and think about his subject over pretty much anything else. He loves infinity and taught me about it from a young age. I was first introduced to the calculus in 5th grade (not that it made much of an impression). He loves to teach, and at the drop of a hat, he'll launch into a lecture about any kind of math. Just ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I would say that there are few things he is more familiar with than infinity...my mother's face, perhaps? I wouldn't count on it.  If a human can understand anything, my father understands infinity.&lt;/p&gt;&#xA;" OwnerUserId="30534" LastActivityDate="2019-10-16T17:13:00.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15935" PostTypeId="2" ParentId="15932" CreationDate="2019-10-16T17:42:49.043" Score="1" Body="&lt;p&gt;Nothing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its in &lt;em&gt;almost&lt;/em&gt; everyone's favor for it to stay that way financially. Having non-technical individuals associate AI with &lt;em&gt;terminators&lt;/em&gt; makes a perception that the field has greater capabilities than it does &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; this leads to grants, funding, etc...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any negative? &lt;strong&gt;Yes&lt;/strong&gt;. Misconceptions always have drawbacks. We see the creation of dumb ethics boards and such &lt;em&gt;cough cough Elon Musk&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if history has anything to say about this, as the field gains popularity (which it is dnagerously quick), information will spread by definition, and eventually misconceptions will be laid to rest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this answer is biased and based upon my own opinions&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-10-16T17:42:49.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15936" PostTypeId="1" AcceptedAnswerId="24217" CreationDate="2019-10-16T18:06:55.083" Score="4" ViewCount="136" Body="&lt;p&gt;The game &quot;Flow Free&quot; in which you connect coloured dots with lines is very popular. A human can learn techniques to play it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering how an AI might approach it. There are certain rules of thumb that a human learns, e.g. connecting dots on the edges one should keep to the edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the time it appears the best approach is a depth-first search, e.g. one tries very long paths to see if they work. Combined with rules of thumbs and inferences such as &quot;don't leave gaps&quot;. Also &quot;Don't cut off one dot from another dot of the same colour&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are ways to &quot;not leave gaps&quot; such as keep within one square of another line. That humans seem to be able to grasp but seems harder for an AI to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact I wonder if the rule of thumb &quot;keep close to other lines&quot; might even require some kind of internal language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean to even understand the rules of the game one would think one would need language. (Could an ape solve one of these puzzles? I doubt it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically I'm trying to solve how an AI could come up with these technqiues for solving puzzles like Flow Free. (Techniques that might not work in all cases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps, humans have an innate understanding of concepts such as &quot;keep close to the wall&quot; and &quot;don't double back on yourself&quot; and can combine them in certain ways. Also we are able to spot simple regions quickly bounded by objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a built in understanding of &quot;regions&quot; would be key. And the key concept that dots can't be joined unless they are in the same region. And we have got to a dead-end if:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There is an empty region&lt;/li&gt;&#xA;&lt;li&gt;There is a region with a dot without it's pair&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Still I don't think this is enough. &lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2019-10-16T18:18:08.540" LastActivityDate="2020-10-22T20:29:26.310" Title="How can an AI play Flow Free?" Tags="&lt;game-ai&gt;&lt;game-theory&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15937" PostTypeId="1" AcceptedAnswerId="15944" CreationDate="2019-10-16T19:00:37.127" Score="0" ViewCount="80" Body="&lt;p&gt;We are a team of computer science our graduation project about EmotionalSpeech Synthesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've found valuable information like research papers and WaveNet, Tacotron. &#xA;A website (&lt;a href=&quot;https://www.voicery.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.voicery.com/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we were hoping to get to know more information from you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We need more details what should we start with to grasp the fundamentals to build this idea, what is the architecture to be used in this project, whether there are papers, a GitHub Repository containing helpful documentation, datasets, some other resources, previous knowledge. &lt;/p&gt;&#xA;" OwnerUserId="26390" LastEditorUserId="1671" LastEditDate="2019-10-21T21:06:21.830" LastActivityDate="2019-10-21T21:06:21.830" Title="Emotional Speech Synthesis" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;&lt;emotional-intelligence&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-18T00:39:45.743" ContentLicense="CC BY-SA 4.0" />
  <row Id="15938" PostTypeId="2" ParentId="15730" CreationDate="2019-10-16T21:03:01.603" Score="1" Body="&lt;p&gt;Humans certainly don't understand infinity. Currently computers cannot understand things that humans cannot because computers are programmed by humans. In a dystopian future that may not be the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some thoughts about infinity. &#xA;The set of natural numbers is infinate. It has also been proved that the set of prime numbers, which is a subset of the natural numbers, is also infinate. So we have an infinate set within an infinate set. &#xA;It gets worse, between any 2 real numbers there is an infinate number of real numbers. &#xA;Have a look at the link to Hilbert's paradox of the Grand Hotel to see how confusing infinity can get - &#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hilbert%27s_paradox_of_the_Grand_Hotel&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hilbert%27s_paradox_of_the_Grand_Hotel&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30539" LastActivityDate="2019-10-16T21:03:01.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15939" PostTypeId="1" CreationDate="2019-10-16T23:22:03.890" Score="1" ViewCount="90" Body="&lt;p&gt;I'm looking for an &quot;elevator pitch&quot; breakdown of areas of applications for Reinforcement Learning &amp;amp; Neural Networks vs. Genetic Algorithms, both actual and theoretical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links are welcome, but please provide some explanation.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-10-17T21:53:37.473" LastActivityDate="2019-10-17T21:53:37.473" Title="An &quot;elevator pitch&quot; breakdown of areas of applications for Reinforcement Learning &amp; Neural Networks vs. Genetic Algorithms" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15940" PostTypeId="2" ParentId="15939" CreationDate="2019-10-16T23:38:16.083" Score="3" Body="&lt;p&gt;Your question suggests a confusion of &lt;em&gt;techniques&lt;/em&gt;, &lt;em&gt;representations&lt;/em&gt; and &lt;em&gt;problems&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural Networks are a &lt;em&gt;representation&lt;/em&gt; that can be used to &lt;em&gt;approximate functions&lt;/em&gt;. A neural network approximates a function that maps from inputs to outputs by optimizing parameters (weights).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Genetic Algorithms are a &lt;em&gt;technique&lt;/em&gt; that can be used to optimize a problem. You might chose to use a GA to optimize the weights in a neural network for instance. Or you might use it to optimize a different representation or approximation of a function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reinforcement Learning is a &lt;em&gt;problem&lt;/em&gt;. In a Reinforcement learning problem, the agent learns a function mapping states to actions. You can learn this function directly in some problem domains, or by a near-direct approximation (like tile-coding), or with a function appropriator (like a neural network).  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-16T23:38:16.083" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15941" PostTypeId="2" ParentId="12026" CreationDate="2019-10-16T23:50:57.267" Score="0" Body="&lt;p&gt;No, your tree is not an accurate representation of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: Consider the case where both players flip an H on their first coin. Player 1 decides not to flip a second coin. Is the game over?&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-16T23:50:57.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15942" PostTypeId="1" CreationDate="2019-10-17T00:43:11.233" Score="3" ViewCount="115" Body="&lt;p&gt;I have the following homework.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We proved Sauer's lemma by proving that for every class &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; of ﬁnite VC-dimension &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, and every subset &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; of the domain,&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left|\mathcal{H}_{A}\right| \leq |\{B \subseteq A: \mathcal{H} \text { shatters } B\} | \leq \sum_{i=0}^{d}\left(\begin{array}{c}{|A|} \\ {i}\end{array}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Show that there are cases in which the previous two inequalities are strict (namely, the &lt;span class=&quot;math-container&quot;&gt;$\leq$&lt;/span&gt; can be replaced by &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt;) and cases in which they can be replaced by equalities. Demonstrate all four combinations of &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can I solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="27112" LastEditorUserId="9947" LastEditDate="2020-03-31T00:05:25.837" LastActivityDate="2021-04-25T02:35:14.983" Title="How to show Sauer's Lemma when the inequalities are strict or they are equalities?" Tags="&lt;computational-learning-theory&gt;&lt;pac-learning&gt;&lt;vc-dimension&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15943" PostTypeId="2" ParentId="15730" CreationDate="2019-10-17T01:27:47.977" Score="2" Body="&lt;p&gt;John Doucette's &lt;a href=&quot;https://ai.stackexchange.com/a/15744&quot;&gt;answer&lt;/a&gt; covers my thoughts on this pretty well, but I thought a concrete example might be interesting. I work on a symbolic AI called Cyc, which represents concepts as a web of logical predicates. We often like to brag that Cyc &quot;understands&quot; things because it can elucidate logical relationships between them. It knows, for example, that people don't like paying their taxes, because paying taxes involves losing money and people are generally averse to that. In reality, I think most philosophers would agree that this is an incomplete &quot;understanding&quot; of the world at best. Cyc might know all of the rules that describe people, taxes, and displeasure, but it has no real experience of any of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of infinity, though, what more is there to understand? I would argue that as a mathematical concept, infinity has no reality beyond its logical description. If you can correctly apply every rule that describes infinity, you've grokked infinity. If there's anything that an AI like Cyc can't represent, maybe it's the emotional reaction that such concepts tend to evoke for us. Because we live actual lives, we can relate abstract concepts like infinity to concrete ones like mortality. Maybe it's that emotional contextualization that makes it seem like there's something more to &quot;get&quot; about the concept.&lt;/p&gt;&#xA;" OwnerUserId="30545" LastActivityDate="2019-10-17T01:27:47.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15944" PostTypeId="2" ParentId="15937" CreationDate="2019-10-17T02:57:22.310" Score="0" Body="&lt;p&gt;Since I am working with audio synthesis projects, I can suggest you to go through the recently released paper &quot;MelNet: A Generative Model for Audio in the Frequency Domain&quot;, in this architecture human like speech is synthesized, I think this might be better than WaveNet where the generated audio does contain robotic nature, I think this will definitely help you with Emotional speech synthesis as it generates human-like voices.&#xA;You can check for online demos as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://sjvasquez.github.io/blog/melnet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1906.01083.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here Is the Paper!&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=BZ_sjvAHLpA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Related YouTube Video &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="26854" LastEditorUserId="26854" LastEditDate="2019-10-17T06:52:44.633" LastActivityDate="2019-10-17T06:52:44.633" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15945" PostTypeId="1" CreationDate="2019-10-17T04:31:13.380" Score="2" ViewCount="34" Body="&lt;p&gt;I have a hypothetical example that closes to my research problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you are a boss and you have different types of tasks that you need to assign to your employee. Sensitive task (very classified), and task that requires high skills. So you need to assign a sensitive task (government document) to the trusted employee. While the other task (e.g. statistical analysis ) can be assigned to employee who is more creative and smart. Now every day you have many tasks that need to be done and have a large number of employees with a number of crowdsources (freelancers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have an outcome and history of trust and performance along of failure rate of assigned task on that day of these employees as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R9ZkM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R9ZkM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see here on day 1: the trust of emp 111 is good, so on that day, he had a low failure rate of the sensitive task. While his performance is low, and that made other task failed a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now assume you have a sensitive task coming, and you have a pool of workers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic equation might not good here: &lt;code&gt;Trust + Performance.&lt;/code&gt; I need to weigh each factor based on the type of tasks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Trust x w1  +  performance x w2&lt;/code&gt;  ::::: &lt;code&gt;w1&lt;/code&gt; is high coefficient when sensitive is coming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea of how I model these issues. &lt;/p&gt;&#xA;" OwnerUserId="30551" LastActivityDate="2019-10-17T04:31:13.380" Title="Assigning Weighting Factors" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15946" PostTypeId="1" AcceptedAnswerId="16085" CreationDate="2019-10-17T12:15:55.270" Score="5" ViewCount="162" Body="&lt;p&gt;Although I have a decent background in math, I'm trying to understand which courses from CS and logic to look into. My aim is to get into a Machine Learning PhD program.&lt;/p&gt;&#xA;" OwnerUserId="30568" LastActivityDate="2019-10-25T20:05:55.823" Title="Which courses in computer science and logic are relevant to Machine Learning?" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15947" PostTypeId="1" CreationDate="2019-10-17T12:27:56.897" Score="3" ViewCount="114" Body="&lt;p&gt;I need to write a minimax algorithm with alpha-beta pruning in limited time for the &lt;a href=&quot;https://en.wikipedia.org/wiki/2048_(video_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;2048 game&lt;/a&gt;. I know expectimax is better for this work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume I wrote different heuristic functions. If I want to write an evaluation function as a linear combination of these heuristic functions, do I have to give random weights, or can I calculate the optimal weights with some optimization algorithm? &lt;/p&gt;&#xA;" OwnerUserId="30569" LastEditorUserId="2444" LastEditDate="2019-10-17T15:49:27.027" LastActivityDate="2019-10-17T15:49:27.027" Title="How to choose the weights for a linear combination of heuristic functions?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;heuristics&gt;&lt;alpha-beta-pruning&gt;&lt;expectiminimax&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15949" PostTypeId="1" AcceptedAnswerId="15957" CreationDate="2019-10-17T14:16:32.320" Score="1" ViewCount="117" Body="&lt;p&gt;Is artificial intelligence and, in particular, neural networks being used in real-world critical applications and devices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had a discussion with my colleague who states that nobody would use artificial intelligence, especially neural nets, for critical stuff, like technical devices or sensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm only aware of the problem of neural nets being so-called black-boxes, but, nevertheless, I think it is possible to make an NN robust so that it matches the demands of daily processes, also in sensitive fields like health care, energy market, self-driving cars, and so on. Yet I cannot underline this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does somebody have more insights or other information, opinions and so on? I appreciate any meaningful answer.&lt;/p&gt;&#xA;" OwnerUserId="26353" LastEditorUserId="2444" LastEditDate="2019-10-17T22:31:47.020" LastActivityDate="2019-10-18T05:40:55.737" Title="Is artificial intelligence and, in particular, neural networks being used in real-world critical applications?" Tags="&lt;neural-networks&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15950" PostTypeId="1" CreationDate="2019-10-17T16:05:39.920" Score="3" ViewCount="57" Body="&lt;p&gt;Consider a Bayesian classifier used in &lt;em&gt;spam e-mail filtering&lt;/em&gt;. It converts an e-mail to a vector, most of the time using the bag-of-words method. Although it learns first before getting employed, it can be made to work as an &lt;em&gt;online system&lt;/em&gt;, i.e. it can be used to filter and learn from examples even after deployment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, on the other hand, now comes the &lt;em&gt;perceptron&lt;/em&gt;. It calculates a mean vector of spam and not spam, and then classifies them into the appropriate categories. The model adjusts the mean vectors each time it makes mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, comes neural nets, they too are capable of taking a vector-like bag of words or image pixels of dogs and cats and classify them into yes or no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, while designing and implementing them into the system, how to determine which one of the methods (Bayesian classifier, perceptron or neural network) is the most appropriate for a given situation or task? One factor to consider is the time complexity (or speed), but what are other factors, and how to rank them? &lt;/p&gt;&#xA;" OwnerDisplayName="user27450" LastEditorUserId="2444" LastEditDate="2019-10-17T22:09:10.450" LastActivityDate="2019-10-18T02:18:37.243" Title="How do I determine the most appropriate classifier for a certain problem?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;comparison&gt;&lt;perceptron&gt;&lt;bayesian-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15951" PostTypeId="1" AcceptedAnswerId="15963" CreationDate="2019-10-17T16:19:22.220" Score="2" ViewCount="112" Body="&lt;p&gt;In my experience with Neural Nets, I have only used them to take input vectors and return binary output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, here in a video, &lt;a href=&quot;https://youtu.be/ajGgd9Ld-Wc?t=214&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/ajGgd9Ld-Wc?t=214&lt;/a&gt;, Kai Fu Lee, renowned AI Expert shows a deep net which takes thousands of samples of Trump's speeches and &lt;strong&gt;generates output in the Chinese Language.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, how can deep nets/neural nets be used to &lt;strong&gt;generate&lt;/strong&gt; output rather than giving answer &lt;strong&gt;yes or no&lt;/strong&gt;? Additionally, how are these nets being trained? Can anyone here provide me a simple design to nets that are capable of doing that?&lt;/p&gt;&#xA;" OwnerDisplayName="user27450" LastEditorUserId="25496" LastEditDate="2019-10-18T14:06:22.117" LastActivityDate="2019-10-19T01:42:53.657" Title="How can neural networks be used to generate rather than classify?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15952" PostTypeId="2" ParentId="15951" CreationDate="2019-10-17T17:37:56.910" Score="1" Body="&lt;p&gt;If the output can either be &lt;em&gt;yes&lt;/em&gt; or &lt;em&gt;no&lt;/em&gt;, then you have a discrete and binary output, so this problem is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary classification&lt;/a&gt;, that is, it is the task of classifying (or categorizing) the input into one of two categories (or classes). You can also have a neural network with an output that can take more than two possible discrete values, which is can be used to solve a &lt;a href=&quot;https://stats.stackexchange.com/q/11859/82135&quot;&gt;multi-class classification problem&lt;/a&gt;. For example, a neural network that outputs a sentence, which is composed of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; words, where &lt;span class=&quot;math-container&quot;&gt;$n&amp;gt;1$&lt;/span&gt;. In general, the output does not necessarily need to take a value from a set of discrete values (or classes), but it can also take a numeric value (e.g. a floating-point number). In that case, the problem is called &lt;em&gt;regression&lt;/em&gt;. For example, the task of predicting the height of a person (a numeric value) given a picture of the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different types of neural networks. The simplest neural network is either a perceptron (if you consider it a neural network) or a multi-layer feed-forward neural network, that is, a neural network with only forward connections, with possible multiple layers. There are also &lt;a href=&quot;https://ai.stackexchange.com/a/12290/2444&quot;&gt;convolutional neural networks (CNNs) and recurrent neural networks (RNNs)&lt;/a&gt;, which are more sophisticated neural networks that are more suited for processing respectively imagery or sequences. There are also &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative&lt;/a&gt; neural networks (e.g. &lt;a href=&quot;https://arxiv.org/abs/1312.6114&quot; rel=&quot;nofollow noreferrer&quot;&gt;variational auto-encoders&lt;/a&gt;), which are trained to learn a &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;distribution&lt;/a&gt;, from which you can then &lt;a href=&quot;https://en.wikipedia.org/wiki/Sampling_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;sample&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your specific example, the sentence could have been generated with a recurrent neural network or a generative model (or a combination of both). More precisely, a recurrent generative model could have been trained to learn the rules of either the English or Chinese language. Then you sample from this distribution to generate sentences. In principle, a sentence could also be generated with simpler neural networks (such as a multi-layer perceptron), but, in practice, this may be more inefficient.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-17T17:48:09.437" LastActivityDate="2019-10-17T17:48:09.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15954" PostTypeId="2" ParentId="15946" CreationDate="2019-10-17T18:23:16.090" Score="1" Body="&lt;p&gt;In several projects, I found data analysis and data structures to be critical.  Machine Learning requires huge amounts of data and, most likely, the data will come from multiple sources. Prior to use, data  requires analysis, cleaning, interpretation, feature engineering (subject matter expertise), and structure. &lt;/p&gt;&#xA;" OwnerUserId="30577" LastActivityDate="2019-10-17T18:23:16.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15955" PostTypeId="1" CreationDate="2019-10-17T19:29:26.390" Score="1" ViewCount="43" Body="&lt;p&gt;Gday guys,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I tried to implement my own enviroment and agent in order to fully understand DQNs. The enviroment is a dungeon with five states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;actionspace = 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;statespace = 5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;!!!Action a0 is actually jump to s0!!! &lt;a href=&quot;https://i.stack.imgur.com/2D9Au.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2D9Au.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation of the enviroment is trivial so I will left that part out.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;env = enviroment.Dungeon()&#xA;episodes = 1000&#xA;dqAgent = agent.DeepQAgent(5, 2, 0.5, episodes)&#xA;&#xA;for episode in range(episodes):&#xA;   action = dqAgent.nextAction(env.state)&#xA;   state, reward, newState = env.takeAktion(action)&#xA;   dqAgent.update(state, action, reward, newState)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the academy or orchestration in which the agent will choose, which is then passed to the enviroment. (You get the idea)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agents brain is a pytorch NN with the following architecture:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.fc1 = nn.Linear(5, 5)&#xA;self.fc2 = nn.Linear(5, 2)&#xA;self.criterion = nn.MSELoss()&#xA;self.optim = optim.SGD(self.parameters(), self.lr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The training procedure is the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;choose an action based on the state (exploration vs. exploitation)&lt;/li&gt;&#xA;&lt;li&gt;observe the feedback (oldState, reward, newState)&lt;/li&gt;&#xA;&lt;li&gt;then update the agents &quot;mind&quot; based on the obervtion&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The critical part is the following. &#xA;When I got it right. Iam trying to minimize the loss between the predicted qValues for a given state AND the actual qValues for a state. The actual Values can be calculated with this equation:&#xA;(please tell me if the quation is wrong)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/a42XN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a42XN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;oldQValues = self.predictQ(state)&#xA;newQValues = self.predictQ(newState)&#xA;bestNewAction = self.nextAction(newState)&#xA;targetQValues = oldQVals&#xA;&#xA;targetQValues[0][action] = reward + self.discount * newQVal[0][nextAction]&#xA;&#xA;self.optim.zero_grad()&#xA;loss = self.criterion(oldQVals, targetQ)&#xA;self.optim.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is what I came up with. And this is kinda working. &#xA;E.g when I let the agent predict the action for every given state:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tensor([[0.0000, 0.4230]], grad_fn=&amp;lt;ReluBackward0&amp;gt;)&#xA;tensor([[0.0000, 0.3108]], grad_fn=&amp;lt;ReluBackward0&amp;gt;)&#xA;tensor([[0.0371, 0.4376]], grad_fn=&amp;lt;ReluBackward0&amp;gt;)&#xA;tensor([[0.0000, 0.4763]], grad_fn=&amp;lt;ReluBackward0&amp;gt;)&#xA;tensor([[0.0137, 0.5945]], grad_fn=&amp;lt;ReluBackward0&amp;gt;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The agant will always choose to go right. (a1) &#xA;This is truly the right action to take. But as I alter the rewards the actions appear suboptimal. &#xA;After tweaking the number of episodes and the learning rate the agent will also learn the new circumstances. But this whole situation seems strange to me. I would love it if someone can tell me if I got thie whole process right in my head. (Please ask if you want to check my knowlage) If I miss something or messed something up. Please also tell me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thank you&lt;/p&gt;&#xA;" OwnerUserId="30431" LastEditorUserId="1671" LastEditDate="2019-10-17T20:33:11.507" LastActivityDate="2019-10-17T20:33:11.507" Title="Did I understand deep Q leaning right? (Implementation)" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;q-learning&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15957" PostTypeId="2" ParentId="15949" CreationDate="2019-10-17T23:18:17.763" Score="1" Body="&lt;p&gt;This is as much an ethical concern as a practical one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI systems are already reaching or exceeding human performance in many critical areas. Consider &lt;a href=&quot;https://www.nytimes.com/2019/05/20/health/cancer-artificial-intelligence-ct-scans.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the detection of common cancers&lt;/a&gt;, where AI systems match or exceed humans. Another good example is Tesla's Autopilot, which is &lt;a href=&quot;https://futurism.com/the-byte/tesla-safety-autopilot-safer-human&quot; rel=&quot;nofollow noreferrer&quot;&gt;actually safer than human drivers&lt;/a&gt;, but gets a lot of bad press when it makes a mistake. Both of these systems are likely using Neural Networks, possibly alongside other heuristic or rule-drive approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue isn't whether these systems can be &quot;safe enough&quot; for everyday use. They are safe enough in a societal sense already. The concern is that the people who die when these systems make mistakes are randomly selected, whereas the people who die when a human performs the work die &lt;em&gt;because&lt;/em&gt; a human makes a mistake (usually). This is difficult to accept for the same reason that some people are scared of flying, even though it is many times safer than driving the same distance: there is a loss of control, and &quot;good&quot; people may die through no fault of their own, or perhaps through events that are no one's fault.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether we use this systems will thus probably depend on the application. In Medicine it's easy to see a case: we can do more tests than before for the same price. People who can afford to have a doctor review the machine's decisions are probably no worse off. People who couldn't afford this already are better off (they get a diagnosis with some positive predictive power value now, instead of none at all). In driving, it's more complicated, and will probably require further development. No one knows for sure how good self-driving cars can get, but they'll probably get somewhat better over time. Maybe they'll get good enough that they more or less &lt;em&gt;never&lt;/em&gt; kill people, or maybe they won't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I actually think your friend is way off about the use cases he thinks AI is not suitable for though. Check out this article in &lt;a href=&quot;http://mil-embedded.com/10211-ai-in-safety-certification-efforts-evolving/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Military Embedded Systems&lt;/a&gt;. In applications where decisions are made on a pure cost/benefit basis (and not based on people's gut feelings about morality), AI systems are actually &lt;em&gt;easier&lt;/em&gt; to adopt, and are already often better than human operators. This trend seems likely to increase in the future, so &quot;technical devices and sensors&quot;, which are often black-boxes to the lay public anyway, seem like they are among the first things to go.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-18T02:08:46.483" LastActivityDate="2019-10-18T02:08:46.483" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="15958" PostTypeId="2" ParentId="15950" CreationDate="2019-10-18T02:18:37.243" Score="1" Body="&lt;p&gt;This is one of the main skills that separates someone with a deep understanding of, and experience in, machine learning learning, from a neophyte. There are several approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Try several methods&lt;/strong&gt;, perhaps with automated hyperparameter optimization, and see if there's a big difference in typical model quality. This is pretty common if you don't have a lot of experience, but also something experts may try in a more targeted way.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Visualize the shape of your problem&lt;/strong&gt;, perhaps by using a dimensionality reduction technique like PCA or tSNE, or maybe an auto-encoder. If you compress the data to 2d, are there clear linear patterns? Maybe try a linear model like logistic regression. Are there several distinct groups? How many lines would you need to draw to separate them? If it's a lot, maybe you're going to need a very non-linear model. If it's just a few, maybe a small multi-layer perceptron can help. If there are spiral bands or circular shapes, maybe and SVM with a non-linear kernel. Knowing how to translate the visualizations into intuition about the kinds of model that can help is an advanced skill. You need to understand what shapes of patterns each kind of model can learn to fit, and how these do or do not translate into a higher dimensional space.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Read the literature&lt;/strong&gt;. If you're working in computer vision, you should try a CNN. Why? Well, everyone else is using them. They work great on most computer vision problems. They hold most competition records. It'd be silly not to try them. If you're working on spam classification though, CNNs are a bad choice. People use compression classifiers, Bayesian models, and sometimes multi-stage models to build complex features. If you look at the recent literature in your area, you can tell what to use. Reading and &lt;em&gt;understanding&lt;/em&gt; this literature well enough to interpret it usually requires more advanced scientific training in ML and/or the application area. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As a final note, the &lt;a href=&quot;https://ai.stackexchange.com/questions/15650/what-are-the-implications-of-the-no-free-lunch-theorem-for-machine-learning/15651#15651&quot;&gt;No Free Lunch theorems&lt;/a&gt; for ML tell us that this is always going to be an art (at least, that's one interpretation experts argue for), and not a science, so the more practice you get with it, the better you'll become.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-18T02:18:37.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15959" PostTypeId="1" AcceptedAnswerId="15962" CreationDate="2019-10-18T03:28:00.377" Score="1" ViewCount="53" Body="&lt;p&gt;For a single neuron with &lt;strong&gt;2 weights&lt;/strong&gt;, I can plot the loss landscape and it looks like this (OR data, sigmoid activation, MAE loss):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vG2Vp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vG2Vp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, when the neuron accepts more inputs, which means more than 2 weights required, or when there are more neurons, more layers in the network; how should the 3D loss landscape be plotted?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2019-10-18T06:56:44.183" LastActivityDate="2019-10-18T07:59:15.020" Title="How to plot Loss Landscape with more than 2 weights in the network" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;plotting&gt;&lt;loss&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15960" PostTypeId="2" ParentId="15949" CreationDate="2019-10-18T05:40:55.737" Score="1" Body="&lt;p&gt;There is significant lag between overseas military operations and people operators back in the states. Because of this, the military is offloading more functionality to the remote piloted drones and convoys. The same will be the case once we have functional rescue bots. Look up boston robotics, if you don't think robots will be entering fires or ravines in the near future. &lt;/p&gt;&#xA;" OwnerUserId="28098" LastActivityDate="2019-10-18T05:40:55.737" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15962" PostTypeId="2" ParentId="15959" CreationDate="2019-10-18T07:59:15.020" Score="0" Body="&lt;p&gt;It seems not possible to plot loss values (z) against all combinations of weights in all layers, especially when the network is big with thousands or millions of params; in that case, the number of points to plot is too too big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also, the 3D space can't be used to plot more than 3 dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with a deep network with lots of weights, these can be plotted:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Loss value against any pair of 2 weights&lt;/li&gt;&#xA;&lt;li&gt;Turn the layer right before output layer (single neuron) into a layer of 2 neurons, and loss can be plotted against these 2 weights (but doesn't make much sense as the meaning of loss value depends on all other weights also)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Example plot when there are 2 neurons in the layer right before output layer (of 1 neuron):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0XeMB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0XeMB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2019-10-18T07:59:15.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15963" PostTypeId="2" ParentId="15951" CreationDate="2019-10-18T14:00:35.030" Score="2" Body="&lt;p&gt;Think of a neural network as a universal function approximator (With infinite width under a set of constraints this is actually provable). Now when discussing generation in the context you have provided, you essentially want to draw from some distribution &lt;span class=&quot;math-container&quot;&gt;$p(y|c)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is your output and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is your context or input.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theorem:&lt;/strong&gt; For any distribution &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;, if we take &lt;span class=&quot;math-container&quot;&gt;$z \sim \mathcal{N}(0,I)$&lt;/span&gt;, there exists a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$f(z) \sim \Omega$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the above theorem (for the purposes of this post I don't need to prove it, but its very similar to the universal approximation theorem proof) and if we take neural networks as a pseudo-universal function approximator, if we have a valid objective or training procedure that can learn the parameters of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, sampling is as easy as sampling &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(0,I)$&lt;/span&gt; and then applying &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the trick really is finding a good training procedure, and this is where you see GANs, VAEs and other models/schemes come into play.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything I've said above works really well when there isn't autocorrelation like in text, but when there is, the above methodology would result in a combinatorially large output space which isn't realistic with a vocabulary size usually spanning somewhere between a couple thousand and a couple hundred thousand. So to handle this they model the joint by taking advantage of that autocorrelation by modeling the joint probability as its bayesian decomposition. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$p(\vec w) = p(w_0)\prod_{i=1}^{N-1}p(w_i|w_{&amp;lt;i})$$&lt;/span&gt;&lt;br&gt;&#xA;Now that there is a framework to efficiently model this type of output, were back into the position as before where we are looking for clever training schemes. In this case you'll see commonly RNN's or other sequential model training with teacher forcing (@nbro described this in his answer too), or using GAN like compositions using either reinforcement learning to handle the lack of differentiability in sampling or using approximations like Gumbel-Softmax or Intermediate Loss Sampling (method I actually developed)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this answered your question.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2019-10-19T01:42:53.657" LastActivityDate="2019-10-19T01:42:53.657" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15964" PostTypeId="1" AcceptedAnswerId="15967" CreationDate="2019-10-18T14:22:00.050" Score="1" ViewCount="64" Body="&lt;p&gt;Bellow I have a validation plot&#xA;How should I interpret this validation plot?&#xA;Is my data underfitting? What else can be seen from this?&#xA;Which one is the best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does it mean that the right line is growing and green line decrease (slightly) for example after 15?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ONEtk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ONEtk.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second random forrest&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yHk3P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yHk3P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="30599" LastEditDate="2019-10-19T11:53:33.317" LastActivityDate="2019-10-19T11:53:33.317" Title="How should I interpret this validation plot?" Tags="&lt;machine-learning&gt;&lt;cross-validation&gt;&lt;random-forest&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15965" PostTypeId="1" CreationDate="2019-10-18T15:16:44.650" Score="1" ViewCount="438" Body="&lt;p&gt;Is it possible and how trivial (or not) might it be (if possible) to retrain GPT-2 on time-series data instead of text?&lt;/p&gt;&#xA;" OwnerUserId="3861" LastEditorUserId="2444" LastEditDate="2019-11-01T03:17:06.510" LastActivityDate="2019-12-01T05:02:35.400" Title="Is it possible to use the GPT-2 model for time-series data prediction?" Tags="&lt;natural-language-processing&gt;&lt;prediction&gt;&lt;time-series&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15966" PostTypeId="1" CreationDate="2019-10-18T15:30:41.103" Score="1" ViewCount="56" Body="&lt;p&gt;I am trying to reproduce results presented in this &lt;a href=&quot;https://arxiv.org/pdf/1802.09464.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. On page 4, the authors state:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... we train for 50 epochs (one epoch consists of 19*2*50 = 1900 full&#xA;  episodes), which amounts to a total of 4.75*10^6 timesteps.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The 1900 episodes are broken down into Rollouts per MPI worker (2) * Number of MPI Workers (19) * Cycles per epoch (50), as shown in the hyper parameters section on page 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When testing on my local machine, using the GitHub &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Baselines repo&lt;/a&gt;, I am using 1 MPI worker and the following hyperparams:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'n_cycles': 50,  # per epoch&#xA;'rollout_batch_size': 2,  # per mpi thread&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By the same calculation, this means that I should have 1*50*2 = 100 episodes per epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I run &lt;code&gt;her&lt;/code&gt; on &lt;code&gt;FetchReach-v1&lt;/code&gt; turns out I only have 10 episodes per epoch. Here is a log sample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Training...&#xA;---------------------------------&#xA;| epoch              | 0        |&#xA;| stats_g/mean       | 0.893    |&#xA;| stats_g/std        | 0.122    |&#xA;| stats_o/mean       | 0.269    |&#xA;| stats_o/std        | 0.0392   |&#xA;| test/episode       | 10       |&#xA;| test/mean_Q        | -0.602   |&#xA;| test/success_rate  | 0.5      |&#xA;| train/episode      | 10       |  &amp;lt;-- 10 episodes/epoch&#xA;| train/success_rate | 0        |&#xA;---------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why is there this discrepancy? Any suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14390" LastEditorUserId="14390" LastEditDate="2019-10-19T06:39:16.577" LastActivityDate="2019-10-19T06:39:16.577" Title="Reinforcement learning number of episodes per epoch not matching with paper" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15967" PostTypeId="2" ParentId="15964" CreationDate="2019-10-18T23:18:29.637" Score="2" Body="&lt;p&gt;This is a sign of &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you make your trees deeper, it becomes possible to &quot;memorize&quot; the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not exist. When you try out these patterns on new data (which is what cross-validation is imitating), then the patterns do not work, and your model fails to generalize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main piece of information to draw from this plot is that the optimum tree depth is about 15.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-18T23:18:29.637" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15968" PostTypeId="2" ParentId="15965" CreationDate="2019-10-19T01:36:42.877" Score="1" Body="&lt;p&gt;Definitely! but at that point it would be training a transformer-encoder (gpt2's architecture) and not GPT2 because GPT2 is defined by the weights / training procedure / data it was trained and not the architecture, and I don't think it would transfer properly to time series.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-10-19T01:36:42.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15970" PostTypeId="1" CreationDate="2019-10-19T06:07:57.087" Score="0" ViewCount="55" Body="&lt;p&gt;My full code is as follows. I have tried to whittle it down to just the code that matters, but the problem I have is that i'm not sure what part of my network code is producing the problem. I've removed my code that loads and sifts through the CSV data because then my code would be too long.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;array&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;#include &amp;lt;chrono&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;#include &amp;lt;variant&amp;gt;&#xA;#include &amp;lt;unordered_set&amp;gt;&#xA;&#xA;&#xA;typedef std::variant&amp;lt;std::string,std::uint_fast16_t,bool,float&amp;gt; CSVType;&#xA;&#xA;/* ... functions to load CSV data ... */&#xA;&#xA;typedef float DataType;&#xA;typedef DataType (*ActivationFuncPtr)(const DataType&amp;amp;);&#xA;&#xA;DataType step(const DataType&amp;amp; x, const DataType&amp;amp; threshold)&#xA;{&#xA;    return x &amp;gt;= threshold ? 1 : 0;&#xA;}&#xA;&#xA;DataType step0(const DataType&amp;amp; x)&#xA;{&#xA;    return step(x,0);&#xA;}&#xA;&#xA;DataType step05(const DataType&amp;amp; x)&#xA;{&#xA;    return step(x,0.5);&#xA;}&#xA;&#xA;DataType sigmoid(const DataType&amp;amp; x)&#xA;{&#xA;    return DataType(1) / (DataType(1) + std::exp(-x));&#xA;}&#xA;&#xA;DataType sigmoid_derivative(const DataType&amp;amp; x)&#xA;{&#xA;    return x * (DataType(1) - x);&#xA;}&#xA;&#xA;template&amp;lt;std::size_t NumInputs&amp;gt;&#xA;class Neuron&#xA;{&#xA;public:&#xA;&#xA;    Neuron()&#xA;    {&#xA;        RandomiseWeights();&#xA;    }&#xA;&#xA;    void RandomiseWeights()&#xA;    {&#xA;        std::generate(m_weights.begin(),m_weights.end(),[&amp;amp;]()&#xA;        {&#xA;            return m_xavierNormalDis(m_mt);&#xA;        });&#xA;        m_biasWeight = 0;&#xA;&#xA;        for(std::size_t i = 0; i &amp;lt; NumInputs+1; ++i)&#xA;            m_previousWeightUpdates[i] = 0;&#xA;    }&#xA;&#xA;    DataType FeedForward(const std::array&amp;lt;DataType,NumInputs&amp;gt; inputValues)&#xA;    {&#xA;        DataType output = m_biasWeight;&#xA;        for(std::size_t i = 0; i &amp;lt; inputValues.size(); ++i)&#xA;            output += inputValues[i] * m_weights[i];&#xA;&#xA;        m_inputValues = inputValues;&#xA;&#xA;        return output;&#xA;    }&#xA;&#xA;    std::array&amp;lt;DataType,NumInputs&amp;gt; Backpropagate(const DataType&amp;amp; error)&#xA;    {&#xA;        std::array&amp;lt;DataType,NumInputs&amp;gt; netInputOverWeight;&#xA;        for(std::size_t i = 0; i &amp;lt; NumInputs; ++i)&#xA;        {&#xA;            netInputOverWeight[i] = m_inputValues[i];&#xA;        }&#xA;&#xA;        DataType netInputOverBias = DataType(1);&#xA;&#xA;        std::array&amp;lt;DataType,NumInputs&amp;gt; errorOverWeight;&#xA;        for(std::size_t i = 0; i &amp;lt; NumInputs; ++i)&#xA;        {&#xA;            errorOverWeight[i] = error * netInputOverWeight[i];&#xA;        }&#xA;&#xA;        DataType errorOverBias = error * netInputOverBias;&#xA;&#xA;        for(std::size_t i = 0; i &amp;lt; NumInputs; ++i)&#xA;        {&#xA;            m_outstandingWeightAdjustments[i] = errorOverWeight[i];&#xA;        }&#xA;        m_outstandingWeightAdjustments[NumInputs] = errorOverBias;&#xA;&#xA;        DataType errorOverNetInput = error;&#xA;&#xA;        std::array&amp;lt;DataType,NumInputs&amp;gt; errorWeights;&#xA;        for(std::size_t i = 0; i &amp;lt; NumInputs; ++i)&#xA;        {&#xA;            errorWeights[i] = errorOverNetInput * m_weights[i];&#xA;        }&#xA;&#xA;        return errorWeights;&#xA;    }&#xA;&#xA;    void AdjustWeights(const DataType&amp;amp; learningRate, const DataType&amp;amp; momentum)&#xA;    {&#xA;        for(std::size_t i = 0; i &amp;lt; NumInputs; ++i)&#xA;        {&#xA;            DataType adjustment = learningRate * m_outstandingWeightAdjustments[i] + momentum * m_previousWeightUpdates[i];&#xA;            m_weights[i] = m_weights[i] - adjustment;&#xA;            m_previousWeightUpdates[i] = adjustment;&#xA;        }&#xA;        DataType adjustment = learningRate * m_outstandingWeightAdjustments[NumInputs] + momentum * m_previousWeightUpdates[NumInputs];&#xA;        m_biasWeight = m_biasWeight - adjustment;&#xA;        m_previousWeightUpdates[NumInputs] = adjustment;&#xA;    }&#xA;&#xA;    const std::array&amp;lt;DataType,NumInputs&amp;gt;&amp;amp; GetWeights() const&#xA;    {&#xA;        return m_weights;&#xA;    }&#xA;&#xA;    const DataType&amp;amp; GetBiasWeight() const&#xA;    {&#xA;        return m_biasWeight;&#xA;    }&#xA;&#xA;protected:&#xA;&#xA;    static std::mt19937 m_mt;&#xA;    static std::uniform_real_distribution&amp;lt;DataType&amp;gt; m_uniformDisRandom;&#xA;    static std::uniform_real_distribution&amp;lt;DataType&amp;gt; m_xavierUniformDis;&#xA;    static std::normal_distribution&amp;lt;DataType&amp;gt; m_xavierNormalDis;&#xA;&#xA;    std::array&amp;lt;DataType,NumInputs&amp;gt; m_weights;&#xA;    DataType m_biasWeight;&#xA;&#xA;    std::array&amp;lt;DataType,NumInputs+1&amp;gt; m_previousWeightUpdates;&#xA;    std::array&amp;lt;DataType,NumInputs+1&amp;gt; m_outstandingWeightAdjustments;&#xA;&#xA;    std::array&amp;lt;DataType,NumInputs&amp;gt; m_inputValues;&#xA;};&#xA;&#xA;template&amp;lt;std::size_t NumInputs&amp;gt;&#xA;std::mt19937 Neuron&amp;lt;NumInputs&amp;gt;::m_mt(std::chrono::duration_cast&amp;lt;std::chrono::milliseconds&amp;gt;(std::chrono::system_clock::now().time_since_epoch()).count());&#xA;&#xA;template&amp;lt;std::size_t NumInputs&amp;gt;&#xA;std::uniform_real_distribution&amp;lt;DataType&amp;gt; Neuron&amp;lt;NumInputs&amp;gt;::m_uniformDisRandom(-1,1);&#xA;&#xA;template&amp;lt;std::size_t NumInputs&amp;gt;&#xA;std::uniform_real_distribution&amp;lt;DataType&amp;gt; Neuron&amp;lt;NumInputs&amp;gt;::m_xavierUniformDis(-std::sqrt(6.f / NumInputs+1),std::sqrt(6.f / NumInputs+1));&#xA;&#xA;template&amp;lt;std::size_t NumInputs&amp;gt;&#xA;std::normal_distribution&amp;lt;DataType&amp;gt; Neuron&amp;lt;NumInputs&amp;gt;::m_xavierNormalDis(0,std::sqrt(2.f / NumInputs+1));&#xA;&#xA;template&amp;lt;std::size_t NumNeurons&amp;gt;&#xA;class ActivationLayer&#xA;{&#xA;public:&#xA;&#xA;    ActivationLayer()&#xA;    :&#xA;        m_outputs({})&#xA;    {}&#xA;&#xA;    virtual std::array&amp;lt;DataType,NumNeurons&amp;gt; GetOutputs() const final&#xA;    {&#xA;        return m_outputs;&#xA;    }&#xA;&#xA;    virtual void CompleteBackprop(const DataType&amp;amp; learningRate, const DataType&amp;amp; momentum) final&#xA;    {&#xA;    }&#xA;&#xA;protected:&#xA;    std::array&amp;lt;DataType,NumNeurons&amp;gt; m_outputs;&#xA;};&#xA;&#xA;template&amp;lt;std::size_t NumNeurons&amp;gt;&#xA;class SigmoidActivation : public ActivationLayer&amp;lt;NumNeurons&amp;gt;&#xA;{&#xA;public:&#xA;&#xA;    virtual std::array&amp;lt;DataType,NumNeurons&amp;gt; FeedForward(const std::array&amp;lt;DataType,NumNeurons&amp;gt;&amp;amp; inputValues)&#xA;    {&#xA;        for(std::size_t i = 0; i &amp;lt; NumNeurons; ++i)&#xA;            ActivationLayer&amp;lt;NumNeurons&amp;gt;::m_outputs[i] = sigmoid(inputValues[i]);&#xA;        return ActivationLayer&amp;lt;NumNeurons&amp;gt;::m_outputs;&#xA;    }&#xA;&#xA;    virtual std::array&amp;lt;DataType,NumNeurons&amp;gt; Backpropagate(const std::array&amp;lt;DataType,NumNeurons&amp;gt; errors)&#xA;    {&#xA;        std::array&amp;lt;DataType,NumNeurons&amp;gt; backpropErrors;&#xA;        for(std::size_t i = 0; i &amp;lt; NumNeurons; ++i)&#xA;            backpropErrors[i] = errors[i] * sigmoid_derivative(ActivationLayer&amp;lt;NumNeurons&amp;gt;::m_outputs[i]);&#xA;        return backpropErrors;&#xA;    }&#xA;};&#xA;&#xA;template&amp;lt;std::size_t NumInputs, std::size_t NumNeurons&amp;gt;&#xA;class FullyConnectedLayer&#xA;{&#xA;public:&#xA;&#xA;    FullyConnectedLayer()&#xA;    :&#xA;        m_neurons([=]()&#xA;        {&#xA;            std::array&amp;lt;Neuron&amp;lt;NumInputs&amp;gt;,NumNeurons&amp;gt; neurons;&#xA;            for(Neuron&amp;lt;NumInputs&amp;gt;&amp;amp; n : neurons)&#xA;                n = Neuron&amp;lt;NumInputs&amp;gt;();&#xA;            return neurons;&#xA;        }())&#xA;    {&#xA;    }&#xA;&#xA;    virtual std::array&amp;lt;DataType,NumNeurons&amp;gt; FeedForward(const std::array&amp;lt;DataType,NumInputs&amp;gt;&amp;amp; inputValues)&#xA;    {&#xA;        std::array&amp;lt;DataType,NumNeurons&amp;gt; outputValues;&#xA;        for(std::size_t i = 0; i &amp;lt; NumNeurons; ++i)&#xA;            outputValues[i] = m_neurons[i].FeedForward(inputValues);&#xA;        return outputValues;&#xA;    }&#xA;&#xA;    /** \brief Take a sum of errors for each node and produce the errors for each input node in the previous layer.&#xA;     *&#xA;     */&#xA;&#xA;    virtual std::array&amp;lt;DataType,NumInputs&amp;gt;&#xA;    Backpropagate(const std::array&amp;lt;DataType,NumNeurons&amp;gt; errors)&#xA;    {&#xA;        std::array&amp;lt;std::array&amp;lt;DataType,NumInputs&amp;gt;,NumNeurons&amp;gt; errorValues;&#xA;        for(std::size_t i = 0; i &amp;lt; NumNeurons; ++i)&#xA;        {&#xA;            errorValues[i] = m_neurons[i].Backpropagate(errors[i]);&#xA;        }&#xA;        std::array&amp;lt;DataType,NumInputs&amp;gt; returnErrors;&#xA;        std::fill(returnErrors.begin(),returnErrors.end(),0);&#xA;        for(std::size_t i = 0; i &amp;lt; NumNeurons; ++i)&#xA;        {&#xA;            for(std::size_t j = 0; j &amp;lt; NumInputs; ++j)&#xA;            {&#xA;                returnErrors[j] += errorValues[i][j];&#xA;            }&#xA;        }&#xA;        return returnErrors;&#xA;    }&#xA;&#xA;    virtual void CompleteBackprop(const DataType&amp;amp; learningRate, const DataType&amp;amp; momentum)&#xA;    {&#xA;        for(Neuron&amp;lt;NumInputs&amp;gt;&amp;amp; n : m_neurons)&#xA;            n.AdjustWeights(learningRate, momentum);&#xA;    }&#xA;&#xA;    const Neuron&amp;lt;NumInputs&amp;gt;&amp;amp; operator[](const std::size_t&amp;amp; index) const&#xA;    {&#xA;        return m_neurons[index];&#xA;    }&#xA;&#xA;    std::array&amp;lt;std::array&amp;lt;DataType,NumInputs&amp;gt;,NumNeurons&amp;gt; GetWeights() const&#xA;    {&#xA;        std::array&amp;lt;std::array&amp;lt;DataType,NumInputs&amp;gt;,NumNeurons&amp;gt; weights;&#xA;        for(std::size_t i = 0; i &amp;lt; NumNeurons; ++i)&#xA;        {&#xA;            weights[i] = m_neurons[i].GetWeights();&#xA;        }&#xA;        return weights;&#xA;    }&#xA;&#xA;protected:&#xA;    std::array&amp;lt;Neuron&amp;lt;NumInputs&amp;gt;,NumNeurons&amp;gt; m_neurons;&#xA;};&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I == sizeof...(Tp)&amp;gt;::type for_each(std::tuple&amp;lt;Tp...&amp;gt; &amp;amp;, FuncT)&#xA;{&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I &amp;lt; sizeof...(Tp)&amp;gt;::type for_each(std::tuple&amp;lt;Tp...&amp;gt;&amp;amp; t, FuncT f)&#xA;{&#xA;    f(std::get&amp;lt;I&amp;gt;(t)); // call f, passing the Ith element of the std::tuple t and the existing output O&#xA;    for_each&amp;lt;I + 1, FuncT, Tp...&amp;gt;(t, f); // process the next element of the tuple with the new output&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I == sizeof...(Tp)&amp;gt;::type for_each(const std::tuple&amp;lt;Tp...&amp;gt; &amp;amp;, FuncT)&#xA;{&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I &amp;lt; sizeof...(Tp)&amp;gt;::type for_each(const std::tuple&amp;lt;Tp...&amp;gt;&amp;amp; t, FuncT f)&#xA;{&#xA;    f(std::get&amp;lt;I&amp;gt;(t)); // call f, passing the Ith element of the std::tuple t and the existing output O&#xA;    for_each&amp;lt;I + 1, FuncT, Tp...&amp;gt;(t, f); // process the next element of the tuple with the new output&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename O, typename FinalOutput, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I == sizeof...(Tp)&amp;gt;::type for_each_get_final_output(std::tuple&amp;lt;Tp...&amp;gt; &amp;amp;, FuncT, O o, FinalOutput&amp;amp; finalOutput)&#xA;{&#xA;    finalOutput = o;&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename O, typename FinalOutput, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I &amp;lt; sizeof...(Tp)&amp;gt;::type for_each_get_final_output(std::tuple&amp;lt;Tp...&amp;gt;&amp;amp; t, FuncT f, O o, FinalOutput&amp;amp; finalOutput)&#xA;{&#xA;    auto newO = f(std::get&amp;lt;I&amp;gt;(t),o); // call f, passing the Ith element of the std::tuple t and the existing output O&#xA;    for_each_get_final_output&amp;lt;I + 1, FuncT, decltype(newO), FinalOutput, Tp...&amp;gt;(t, f, newO, finalOutput); // process the next element of the tuple with the new output&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename O, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;I == 0&amp;gt;::type for_each_reverse_impl(std::tuple&amp;lt;Tp...&amp;gt;&amp;amp; t, FuncT f, O o)&#xA;{&#xA;    f(std::get&amp;lt;0&amp;gt;(t),o);&#xA;}&#xA;&#xA;template&amp;lt;std::size_t I = 0, typename FuncT, typename O, typename... Tp&amp;gt;&#xA;inline typename std::enable_if&amp;lt;(I &amp;gt; 0)&amp;gt;::type&#xA;for_each_reverse_impl(std::tuple&amp;lt;Tp...&amp;gt;&amp;amp; t, FuncT f, O o)&#xA;{&#xA;    auto newO = f(std::get&amp;lt;I&amp;gt;(t),o); // call f, passing the Ith element of the std::tuple t and the existing output O&#xA;    for_each_reverse_impl&amp;lt;I - 1, FuncT, decltype(newO), Tp...&amp;gt;(t, f, newO); // process the next element of the tuple with the new output&#xA;}&#xA;&#xA;template&amp;lt;typename FuncT, typename O, typename... Tp&amp;gt;&#xA;inline void for_each_reverse(std::tuple&amp;lt;Tp...&amp;gt;&amp;amp; t, FuncT f, O o)&#xA;{&#xA;    for_each_reverse_impl&amp;lt;sizeof...(Tp)-1, FuncT, O, Tp...&amp;gt;(t, f, o);&#xA;}&#xA;&#xA;enum class LOSS_FUNCTION : std::uint_fast8_t&#xA;{&#xA;    MEAN_SQUARE_ERROR,&#xA;    CROSS_ENTROPY&#xA;};&#xA;&#xA;class ValidationOptions&#xA;{&#xA;public:&#xA;    enum class METRIC : std::uint_fast8_t { NONE, ACCURACY, LOSS };&#xA;&#xA;    ValidationOptions()&#xA;    :&#xA;        m_validationSplit(.3f),&#xA;        m_enableLoss(false),&#xA;        m_lossFunction(LOSS_FUNCTION::MEAN_SQUARE_ERROR),&#xA;        m_enableAccuracy(false),&#xA;        m_outputFilter([](const DataType&amp;amp; x){ return x; }),&#xA;        m_earlyStoppingMetric(METRIC::NONE),&#xA;        m_earlyStoppingPatience(1.f),&#xA;        m_earlyStoppingDelta(1.f),&#xA;        m_earlyStoppingNumEpochsAverage(1)&#xA;    {}&#xA;&#xA;    ValidationOptions&amp;amp; Loss(const bool enable = true, LOSS_FUNCTION lossFunction = LOSS_FUNCTION::MEAN_SQUARE_ERROR)&#xA;    {&#xA;        m_enableLoss = enable;&#xA;        m_lossFunction = lossFunction;&#xA;        return *this;&#xA;    }&#xA;&#xA;    ValidationOptions&amp;amp; Split(const float dataSplitValidation)&#xA;    {&#xA;        m_validationSplit = dataSplitValidation;&#xA;        return *this;&#xA;    }&#xA;&#xA;    ValidationOptions&amp;amp; Accuracy(const bool enable = true, ActivationFuncPtr outputFilter = [](const DataType&amp;amp; x){return x;})&#xA;    {&#xA;        m_enableAccuracy = enable;&#xA;        m_outputFilter = outputFilter;&#xA;        return *this;&#xA;    }&#xA;&#xA;    ValidationOptions&amp;amp; EarlyStop(const bool enable = true,&#xA;                                 const METRIC metric = METRIC::ACCURACY,&#xA;                                 const float patience = .1f,&#xA;                                 const DataType delta = .01,&#xA;                                 const std::size_t epochNumToAverage = 10)&#xA;     {&#xA;         if(enable == false)&#xA;            m_earlyStoppingMetric = METRIC::NONE;&#xA;         else&#xA;            m_earlyStoppingMetric = metric;&#xA;&#xA;         m_earlyStoppingPatience = patience;&#xA;         m_earlyStoppingDelta = delta;&#xA;         m_earlyStoppingNumEpochsAverage = epochNumToAverage;&#xA;&#xA;         return *this;&#xA;     }&#xA;&#xA;     float GetValidationSplit() const { return m_validationSplit; }&#xA;     bool Loss() const { return m_enableLoss; }&#xA;     LOSS_FUNCTION GetLossFunction() const { return m_lossFunction; }&#xA;     bool Accuracy() const { return m_enableAccuracy; }&#xA;     ActivationFuncPtr GetOutputFilter() const { return m_outputFilter; }&#xA;     METRIC GetEarlyStoppingMetric() const { return m_earlyStoppingMetric; }&#xA;     float GetEarlyStoppingPatience() const { return m_earlyStoppingPatience; }&#xA;     DataType GetEarlyStoppingDelta() const { return m_earlyStoppingDelta; }&#xA;     std::size_t GetEarlyStoppingNumEpochsAvg() const { return m_earlyStoppingNumEpochsAverage; }&#xA;&#xA;protected:&#xA;&#xA;    float m_validationSplit;        /**&amp;lt; Percentage of the data set aside for validation */&#xA;&#xA;    bool m_enableLoss;&#xA;    LOSS_FUNCTION m_lossFunction;   /**&amp;lt; Loss function to use */&#xA;&#xA;    bool m_enableAccuracy;&#xA;    ActivationFuncPtr m_outputFilter;   /**&amp;lt; When measuring accuracy data is passed through this */&#xA;&#xA;    METRIC m_earlyStoppingMetric;                   /**&amp;lt; The metric used to stop early */&#xA;    float m_earlyStoppingPatience;                  /**&amp;lt; Percentage of total epochs to wait before stopping early */&#xA;    DataType m_earlyStoppingDelta;                  /**&amp;lt; The amount that the early stopping metric needs to change in a single step before stopping */&#xA;    std::size_t m_earlyStoppingNumEpochsAverage;    /**&amp;lt; The number of epochs averaged over to smooth out the stopping metric */&#xA;};&#xA;&#xA;template&amp;lt;typename... Layers&amp;gt;&#xA;class NeuralNetwork&#xA;{&#xA;public:&#xA;&#xA;    NeuralNetwork(Layers... layers)&#xA;    :&#xA;        m_layers(std::make_tuple(layers...))&#xA;    {&#xA;&#xA;    }&#xA;&#xA;    template&amp;lt;std::size_t NumFeatures, std::size_t NumOutputs, std::size_t NumTrainingRows&amp;gt;&#xA;    void Fit(const std::size_t&amp;amp; numberEpochs,&#xA;             const std::size_t&amp;amp; batchSize,&#xA;             DataType learningRate,&#xA;             const DataType&amp;amp; momentum,&#xA;             std::array&amp;lt;std::array&amp;lt;DataType,NumFeatures&amp;gt;,NumTrainingRows&amp;gt;&amp;amp; trainingData,&#xA;             std::array&amp;lt;std::array&amp;lt;DataType,NumOutputs&amp;gt;,NumTrainingRows&amp;gt;&amp;amp; trainingOutput,&#xA;             const ValidationOptions validationOptions,&#xA;             const bool linearDecayLearningRate = true,&#xA;             std::ostream&amp;amp; outputStream = std::cout)&#xA;    {&#xA;        std::size_t epochNumber = 0;&#xA;&#xA;        // need to support more than just MSE to measure loss&#xA;        std::vector&amp;lt;DataType&amp;gt; lastEpochLoss(validationOptions.GetEarlyStoppingNumEpochsAvg(),0);&#xA;        DataType lastEpochLossAverage = std::numeric_limits&amp;lt;DataType&amp;gt;::max();&#xA;&#xA;        std::vector&amp;lt;DataType&amp;gt; lastValidationAccuracys(validationOptions.GetEarlyStoppingNumEpochsAvg(),0);&#xA;        DataType lastValidationAccuraryAvg = 0;&#xA;&#xA;        std::vector&amp;lt;std::size_t&amp;gt; randomIndices(NumTrainingRows,0);&#xA;        for(std::size_t i = 0; i &amp;lt; NumTrainingRows; ++i)&#xA;            randomIndices[i] = i;&#xA;&#xA;        std::random_shuffle(randomIndices.begin(),randomIndices.end());&#xA;        // take some percentage as validation split&#xA;        // we do this by taking the first percentage of already shuffled indices and removing them&#xA;        // from what is available&#xA;        std::size_t numValidationRecords = NumTrainingRows*validationOptions.GetValidationSplit();&#xA;        std::size_t numTrainingRecords = NumTrainingRows - numValidationRecords;&#xA;        std::vector&amp;lt;std::size_t&amp;gt; validationRecords(numValidationRecords);&#xA;        for(std::size_t i = 0; i &amp;lt; numValidationRecords; ++i)&#xA;        {&#xA;            std::size_t index = randomIndices.back();&#xA;            randomIndices.pop_back();&#xA;            validationRecords[i] = index;&#xA;        }&#xA;&#xA;        while(epochNumber &amp;lt; numberEpochs)&#xA;        {&#xA;            // shuffle the indices so that they are pulled into each batch randomly each time&#xA;            std::random_shuffle(randomIndices.begin(),randomIndices.end());&#xA;&#xA;            DataType epochLoss = 0;&#xA;&#xA;            std::tuple&amp;lt;Layers...&amp;gt; backupLayers = m_layers;&#xA;&#xA;            for(std::size_t batchNumber = 0; batchNumber &amp;lt; std::ceil(numTrainingRecords / batchSize); ++batchNumber)&#xA;            {&#xA;                std::array&amp;lt;DataType,NumOutputs&amp;gt; propagateError = {0};&#xA;&#xA;                std::size_t startIndex = batchNumber * batchSize;&#xA;                std::size_t endIndex = startIndex + batchSize;&#xA;                if(endIndex &amp;gt; numTrainingRecords)&#xA;                    endIndex = numTrainingRecords;&#xA;&#xA;                DataType batchLoss = 0;&#xA;&#xA;                for(std::size_t index = startIndex; index &amp;lt; endIndex; ++index)&#xA;                {&#xA;                    std::size_t row = randomIndices[index];&#xA;                    const std::array&amp;lt;DataType,NumFeatures&amp;gt;&amp;amp; dataRow = trainingData[row];&#xA;                    const std::array&amp;lt;DataType,NumOutputs&amp;gt;&amp;amp; desiredOutputRow = trainingOutput[row];&#xA;&#xA;                    // Feed the values through to the output layer&#xA;                    // use of &quot;auto&quot; is so this lambda can be used for all layers without&#xA;                    // me needing to do any fucking around&#xA;                    std::array&amp;lt;DataType,NumOutputs&amp;gt; finalOutput;&#xA;                    for_each_get_final_output(m_layers, [](auto&amp;amp; layer, auto o)&#xA;                    {&#xA;                        return layer.FeedForward(o);&#xA;                    }, dataRow, finalOutput);&#xA;&#xA;                    DataType totalError = 0;&#xA;                    for(std::size_t i = 0; i &amp;lt; NumOutputs; ++i)&#xA;                    {&#xA;                        if(validationOptions.GetLossFunction() == LOSS_FUNCTION::MEAN_SQUARE_ERROR)&#xA;                            totalError += std::pow(desiredOutputRow[i] - finalOutput[i],2.0);&#xA;                        else if(validationOptions.GetLossFunction() == LOSS_FUNCTION::CROSS_ENTROPY)&#xA;                        {&#xA;                            if(NumOutputs == 1)&#xA;                            {&#xA;                                // binary cross entropy&#xA;                                totalError += (desiredOutputRow[i] * std::log(1e-15 + finalOutput[i]));&#xA;                            }&#xA;                            else&#xA;                            {&#xA;                                // cross entropy&#xA;                            }&#xA;                        }&#xA;                    }&#xA;&#xA;                    batchLoss += totalError;&#xA;                }&#xA;&#xA;                batchLoss *= DataType(1) / (endIndex - startIndex);&#xA;&#xA;                for(std::size_t i = 0; i &amp;lt; NumOutputs; ++i)&#xA;                    propagateError[i] = batchLoss;&#xA;&#xA;                // update after every batch&#xA;                for_each_reverse(m_layers, [](auto&amp;amp; layer, auto o)&#xA;                {&#xA;                    auto errors = layer.Backpropagate(o);&#xA;                    return errors;&#xA;                }, propagateError);&#xA;&#xA;                // once backprop is finished, we can adjust all the weights&#xA;                for_each(m_layers, [&amp;amp;](auto&amp;amp; layer)&#xA;                {&#xA;                    layer.CompleteBackprop(learningRate,momentum);&#xA;                });&#xA;&#xA;                epochLoss += batchLoss;&#xA;            }&#xA;&#xA;            epochLoss *= DataType(1) / numTrainingRecords;&#xA;&#xA;            lastEpochLoss.erase(lastEpochLoss.begin());&#xA;            lastEpochLoss.push_back(epochLoss);&#xA;            DataType avgEpochLoss = 1.f * std::accumulate(lastEpochLoss.begin(),lastEpochLoss.end(),0.f) / (epochNumber &amp;lt; validationOptions.GetEarlyStoppingNumEpochsAvg() ? epochNumber+1 : lastEpochLoss.size());&#xA;&#xA;            if(validationOptions.GetEarlyStoppingMetric() == ValidationOptions::METRIC::LOSS&#xA;               &amp;amp;&amp;amp; epochNumber &amp;gt; numberEpochs * validationOptions.GetEarlyStoppingPatience()&#xA;               &amp;amp;&amp;amp; avgEpochLoss &amp;gt; lastEpochLossAverage + validationOptions.GetEarlyStoppingDelta())&#xA;            {&#xA;                // the loss average has decreased, so we should go back to the previous run and exit&#xA;                std::cout   &amp;lt;&amp;lt; &quot;Early exit Loss Avg \n&quot;&#xA;                            &amp;lt;&amp;lt; &quot;Last Epoch: &quot; &amp;lt;&amp;lt; lastEpochLossAverage &amp;lt;&amp;lt; &quot;\n&quot;&#xA;                            &amp;lt;&amp;lt; &quot;This Epoch: &quot; &amp;lt;&amp;lt; avgEpochLoss &amp;lt;&amp;lt; std::endl;&#xA;                m_layers = backupLayers;&#xA;                break;&#xA;            }&#xA;&#xA;            lastEpochLossAverage = avgEpochLoss;&#xA;&#xA;            // check for the error against the reserved validation set&#xA;            std::size_t numCorrect = 0;&#xA;            for(std::size_t row = 0; row &amp;lt; validationRecords.size(); ++row)&#xA;            {&#xA;                const std::array&amp;lt;DataType,NumFeatures&amp;gt;&amp;amp; dataRow = trainingData[row];&#xA;                const std::array&amp;lt;DataType,NumOutputs&amp;gt;&amp;amp; desiredOutputRow = trainingOutput[row];&#xA;&#xA;                std::array&amp;lt;DataType,NumOutputs&amp;gt; finalOutput;&#xA;                for_each_get_final_output(m_layers, [](auto&amp;amp; layer, auto o)&#xA;                {&#xA;                    return layer.FeedForward(o);&#xA;                }, dataRow, finalOutput);&#xA;&#xA;                bool correct = true;&#xA;                for(std::size_t i = 0; i &amp;lt; NumOutputs; ++i)&#xA;                {&#xA;                    if(validationOptions.GetOutputFilter()(finalOutput[i]) != desiredOutputRow[i])&#xA;                        correct = false;&#xA;                }&#xA;                if(correct)&#xA;                    ++numCorrect;&#xA;            }&#xA;&#xA;            DataType validationAccuracy = DataType(numCorrect) / numValidationRecords;&#xA;&#xA;            lastValidationAccuracys.erase(lastValidationAccuracys.begin());&#xA;            lastValidationAccuracys.push_back(validationAccuracy);&#xA;            DataType avgValidationAccuracy = std::accumulate(lastValidationAccuracys.begin(),lastValidationAccuracys.end(),0.f) / (epochNumber &amp;lt; validationOptions.GetEarlyStoppingNumEpochsAvg() ? epochNumber+1 : lastValidationAccuracys.size());&#xA;&#xA;            if(validationOptions.GetEarlyStoppingMetric() == ValidationOptions::METRIC::ACCURACY&#xA;               &amp;amp;&amp;amp; epochNumber &amp;gt; numberEpochs * validationOptions.GetEarlyStoppingPatience()&#xA;               &amp;amp;&amp;amp; avgValidationAccuracy &amp;lt; lastValidationAccuraryAvg - validationOptions.GetEarlyStoppingDelta())&#xA;            {&#xA;                // the accuracy has decreased, so we should go back to the previous run and exit&#xA;                std::cout   &amp;lt;&amp;lt; &quot;Early exit validation accuracy \n&quot;&#xA;                            &amp;lt;&amp;lt; &quot;Last Epoch: &quot; &amp;lt;&amp;lt; lastValidationAccuraryAvg &amp;lt;&amp;lt; &quot;\n&quot;&#xA;                            &amp;lt;&amp;lt; &quot;This Epoch: &quot; &amp;lt;&amp;lt; avgValidationAccuracy &amp;lt;&amp;lt; std::endl;&#xA;                m_layers = backupLayers;&#xA;                break;&#xA;            }&#xA;&#xA;            lastValidationAccuraryAvg = avgValidationAccuracy;&#xA;&#xA;            outputStream &amp;lt;&amp;lt; epochNumber &amp;lt;&amp;lt; &quot;,&quot; &amp;lt;&amp;lt; epochLoss &amp;lt;&amp;lt; &quot;,&quot; &amp;lt;&amp;lt; avgEpochLoss &amp;lt;&amp;lt; &quot;,&quot; &amp;lt;&amp;lt; validationAccuracy &amp;lt;&amp;lt; &quot;,&quot; &amp;lt;&amp;lt; avgValidationAccuracy &amp;lt;&amp;lt; std::endl;&#xA;&#xA;            learningRate -= learningRate / (numberEpochs-epochNumber);&#xA;&#xA;            ++epochNumber;&#xA;        }&#xA;    }&#xA;&#xA;    template&amp;lt;std::size_t NumFeatures, std::size_t NumOutputs, std::size_t NumEvaluationRows&amp;gt;&#xA;    void Evaluate(std::array&amp;lt;std::array&amp;lt;DataType,NumFeatures&amp;gt;,NumEvaluationRows&amp;gt; inputData,&#xA;                  std::array&amp;lt;std::array&amp;lt;DataType,NumOutputs&amp;gt;,NumEvaluationRows&amp;gt; correctOutputs,&#xA;                  DataType&amp;amp; loss,&#xA;                  DataType&amp;amp; accuracy,&#xA;                  ActivationFuncPtr outputFilter = [](const DataType&amp;amp; x){return x;})&#xA;    {&#xA;        loss = 0;&#xA;&#xA;        std::size_t numCorrect = 0;&#xA;&#xA;        for(std::size_t row = 0; row &amp;lt; NumEvaluationRows; ++row)&#xA;        {&#xA;            const std::array&amp;lt;DataType,NumFeatures&amp;gt;&amp;amp; dataRow = inputData[row];&#xA;            const std::array&amp;lt;DataType,NumOutputs&amp;gt;&amp;amp; outputRow = correctOutputs[row];&#xA;&#xA;            // Feed the values through to the output layer&#xA;&#xA;            std::array&amp;lt;DataType,NumOutputs&amp;gt; finalOutput;&#xA;            for_each_get_final_output(m_layers, [](auto&amp;amp; layer, auto o)&#xA;            {&#xA;                layer.FeedForward(o);&#xA;                return layer.GetOutputs();&#xA;            }, dataRow, finalOutput);&#xA;&#xA;            DataType thisLoss = 0;&#xA;            for(std::size_t i = 0; i &amp;lt; NumOutputs; ++i)&#xA;                thisLoss += outputRow[i] - finalOutput[i];&#xA;            loss += thisLoss * thisLoss;&#xA;&#xA;            bool correct = true;&#xA;            for(std::size_t i = 0; i &amp;lt; NumOutputs; ++i)&#xA;            {&#xA;                if(outputFilter(finalOutput[i]) != outputRow[i])&#xA;                    correct = false;&#xA;            }&#xA;            if(correct)&#xA;                ++numCorrect;&#xA;        }&#xA;&#xA;        loss *= DataType(1) / NumEvaluationRows;&#xA;        accuracy = DataType(numCorrect) / NumEvaluationRows;&#xA;    }&#xA;&#xA;    template&amp;lt;std::size_t NumFeatures, std::size_t NumOutputs, std::size_t NumRecords&amp;gt;&#xA;    void Predict(std::array&amp;lt;std::array&amp;lt;DataType,NumFeatures&amp;gt;,NumRecords&amp;gt; inputData,&#xA;                 std::array&amp;lt;std::array&amp;lt;DataType,NumOutputs&amp;gt;,NumRecords&amp;gt;&amp;amp; predictions,&#xA;                 ActivationFuncPtr outputFilter = [](const DataType&amp;amp; x){return x;})&#xA;    {&#xA;        for(std::size_t row = 0; row &amp;lt; NumRecords; ++row)&#xA;        {&#xA;            const std::array&amp;lt;DataType,NumFeatures&amp;gt;&amp;amp; dataRow = inputData[row];&#xA;&#xA;            // Feed the values through to the output layer&#xA;&#xA;            std::array&amp;lt;DataType,NumOutputs&amp;gt; finalOutput;&#xA;            for_each_get_final_output(m_layers, [](auto&amp;amp; layer, auto o)&#xA;            {&#xA;                return layer.FeedForward(o);&#xA;            }, dataRow, finalOutput);&#xA;&#xA;            for(std::size_t i = 0; i &amp;lt; NumOutputs; ++i)&#xA;                predictions[row][i] = outputFilter(finalOutput[i]);&#xA;        }&#xA;    }&#xA;&#xA;protected:&#xA;    std::tuple&amp;lt;Layers...&amp;gt; m_layers;&#xA;};&#xA;&#xA;main()&#xA;{&#xA;    std::vector&amp;lt;std::vector&amp;lt;CSVType&amp;gt;&amp;gt; trainingCSVData;&#xA;    /* load training CSV data */&#xA;&#xA;    std::vector&amp;lt;std::vector&amp;lt;CSVType&amp;gt;&amp;gt; testCSVData;&#xA;    /* load test CSV data */&#xA;&#xA;    std::cout &amp;lt;&amp;lt; std::fixed &amp;lt;&amp;lt; std::setprecision(80);&#xA;&#xA;    std::ofstream file(&quot;error_out.csv&quot;, std::ios::out | std::ios::trunc);&#xA;    if(!file.is_open())&#xA;    {&#xA;        std::cout &amp;lt;&amp;lt; &quot;couldn't open file&quot; &amp;lt;&amp;lt; std::endl;&#xA;        return 0;&#xA;    }&#xA;&#xA;    file &amp;lt;&amp;lt; std::fixed &amp;lt;&amp;lt; std::setprecision(80);&#xA;&#xA;    /*&#xA;        Features&#xA;        1   pClass 1&#xA;        2   pClass 2&#xA;        3   pClass 3&#xA;        4   Sex female 1, male 0&#xA;        5   Age normalised between 0 and 1  age range 0 to 100&#xA;        6   Number siblings between 0 and 1   num range 0 to 8&#xA;        7   Number of parents / children        num range 0 to 9&#xA;        8   Ticket cost   between 0 and 1       num range 0 to 512.3292&#xA;        9   embarked S&#xA;        10  embarked Q&#xA;        11  embarked C&#xA;    */&#xA;&#xA;    std::array&amp;lt;std::array&amp;lt;DataType,29&amp;gt;,891&amp;gt; inputData;&#xA;    std::array&amp;lt;std::array&amp;lt;DataType,1&amp;gt;,891&amp;gt; desiredOutputs;&#xA;&#xA;    /* ... data that loads the titanic data into a series of features. Either class labels or normalised values (like age) */&#xA;&#xA;    NeuralNetwork neuralNet{&#xA;        FullyConnectedLayer&amp;lt;29,256&amp;gt;(),&#xA;        SigmoidActivation&amp;lt;256&amp;gt;(),&#xA;        FullyConnectedLayer&amp;lt;256,1&amp;gt;(),&#xA;        SigmoidActivation&amp;lt;1&amp;gt;()&#xA;    };&#xA;&#xA;    neuralNet.Fit(300,&#xA;                  1,&#xA;                  0.05,&#xA;                  0.25f,&#xA;                  inputData,&#xA;                  desiredOutputs,&#xA;                  ValidationOptions().Accuracy(true,step05).Loss(true,LOSS_FUNCTION::CROSS_ENTROPY).Split(0.3),&#xA;                  false,&#xA;                  file);&#xA;&#xA;    file.close();&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The data used is from the titanic problem that you can &lt;a href=&quot;https://www.kaggle.com/c/titanic/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;download from Kaggle here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical output file that's being generated is like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0,-4.91843843460083007812500000000000000000000000000000000000000000000000000000000000,-4.91843843460083007812500000000000000000000000000000000000000000000000000000000000,0.65168541669845581054687500000000000000000000000000000000000000000000000000000000,0.65168541669845581054687500000000000000000000000000000000000000000000000000000000&#xA;1,-6.14257431030273437500000000000000000000000000000000000000000000000000000000000000,-6.14257431030273437500000000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;2,-6.43130302429199218750000000000000000000000000000000000000000000000000000000000000,-6.43130302429199218750000000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;3,-6.58864736557006835937500000000000000000000000000000000000000000000000000000000000,-6.58864736557006835937500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;4,-6.70884752273559570312500000000000000000000000000000000000000000000000000000000000,-6.70884752273559570312500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;5,-6.78206682205200195312500000000000000000000000000000000000000000000000000000000000,-6.78206682205200195312500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;6,-6.86832284927368164062500000000000000000000000000000000000000000000000000000000000,-6.86832284927368164062500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;7,-6.92110681533813476562500000000000000000000000000000000000000000000000000000000000,-6.92110681533813476562500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;8,-6.96584081649780273437500000000000000000000000000000000000000000000000000000000000,-6.96584081649780273437500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;9,-7.02414274215698242187500000000000000000000000000000000000000000000000000000000000,-7.02414274215698242187500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;10,-7.06421041488647460937500000000000000000000000000000000000000000000000000000000000,-7.06421041488647460937500000000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000,0.65543073415756225585937500000000000000000000000000000000000000000000000000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(shortened for space)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was previously working when the error I fed back through backprop was just the difference between the correct result and the prediction. &lt;a href=&quot;https://www.reddit.com/r/learnmachinelearning/comments/dixyiy/i_think_ive_got_some_elements_of_neural_networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;But I've since been told that I should be propagating back the Loss Functions error&lt;/a&gt;, which I then implemented as Binary Cross Entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So either:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I should not feed back the error from the loss function&lt;/li&gt;&#xA;&lt;li&gt;I am calculating the loss incorrectly&lt;/li&gt;&#xA;&lt;li&gt;My back propagation code is wrong&lt;/li&gt;&#xA;&lt;li&gt;I've got something horrible happening in my validation loop&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2819" LastActivityDate="2019-10-19T15:45:29.890" Title="Why is my neural network giving me wildly incorrect error and not changing accuracy?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15971" PostTypeId="1" AcceptedAnswerId="15989" CreationDate="2019-10-19T07:17:16.243" Score="0" ViewCount="66" Body="&lt;p&gt;Bellow I have a Learning Curve plot How should I interpret this plot for my random forrest algorithm (the second one the most complex one)? &#xA;Which one is the best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v4P65.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v4P65.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Iz5ZP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Iz5ZP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W44dm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W44dm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="30599" LastEditDate="2019-10-19T11:48:40.380" LastActivityDate="2019-10-20T23:34:27.090" Title="How to interpret this learning curve plot" Tags="&lt;machine-learning&gt;&lt;cross-validation&gt;&lt;random-forest&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15972" PostTypeId="1" AcceptedAnswerId="16032" CreationDate="2019-10-19T08:18:03.573" Score="2" ViewCount="50" Body="&lt;p&gt;On this website &lt;a href=&quot;https://scikit-learn.org/stable/modules/learning_curve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/modules/learning_curve.html&lt;/a&gt;, the authors are speaking about variance and bias and they give a simple example of how works in a linear model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I determine the bias and variance of a random forest?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-21T03:23:29.703" LastActivityDate="2019-11-21T03:23:29.703" Title="How can I determine the bias and variance of a random forrest?" Tags="&lt;machine-learning&gt;&lt;random-forest&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15973" PostTypeId="1" CreationDate="2019-10-19T08:56:10.823" Score="1" ViewCount="54" Body="&lt;p&gt;Say we have the layer &lt;span class=&quot;math-container&quot;&gt;$X W + b = Y$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to get &lt;span class=&quot;math-container&quot;&gt;$\frac{dL}{dW}$&lt;/span&gt; and we assume I have &lt;span class=&quot;math-container&quot;&gt;$\frac{dL}{dY}$&lt;/span&gt;.&#xA;So all I need is to find &lt;span class=&quot;math-container&quot;&gt;$\frac{dY}{dW}$&lt;/span&gt;. I know that it should be &lt;span class=&quot;math-container&quot;&gt;$X^T\frac{dL}{dY}$&lt;/span&gt; but don't understand why. please explain.&lt;/li&gt;&#xA;&lt;li&gt;I want to get &lt;span class=&quot;math-container&quot;&gt;$\frac{dL}{db}$&lt;/span&gt; and we assume I have &lt;span class=&quot;math-container&quot;&gt;$\frac{dL}{dY}$&lt;/span&gt;.&#xA;So all I need is to find &lt;span class=&quot;math-container&quot;&gt;$\frac{dY}{db}$&lt;/span&gt;. I know that it should be &lt;span class=&quot;math-container&quot;&gt;$\sum(\frac{dL}{dY})_i$&lt;/span&gt; (I mean sum the rows) but I don't understand why. please explain.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks :)&lt;/p&gt;&#xA;" OwnerUserId="30611" LastEditorUserId="2444" LastEditDate="2019-10-19T13:53:20.007" LastActivityDate="2019-10-19T13:53:20.007" Title="Understanding the partial derivative with respect to the weight matrix and bias" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15974" PostTypeId="2" ParentId="15970" CreationDate="2019-10-19T10:16:15.853" Score="1" Body="&lt;p&gt;This is a guess, as I am not reading all that code!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This was previously working when the error I fed back through backprop was just the difference between the correct result and the prediction. But I've since been told that I should be propagating back the Loss Functions error, which I then implemented as Binary Cross Entropy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You may have been right before, and the advice you received wrong. Sort of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backpropagation does not feed back the error value directly. It works exclusively with &lt;em&gt;gradients&lt;/em&gt; of the error. So you typically start with a gradient value based on the objective function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you backprop each function in the network a single step at a time, then you would start with &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\hat{y}}\mathcal{L}$&lt;/span&gt;, the gradient of the loss function with respect to the estimated values (i.e. output of the NN). Then from that you could calculate &lt;span class=&quot;math-container&quot;&gt;$\nabla_{z}\mathcal{L}$&lt;/span&gt;, which is the gradient of the loss function with respect to the pre-activation values of the output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is possible to combine steps analytically, and really common to start with the first gradient being &lt;span class=&quot;math-container&quot;&gt;$\nabla_{z}\mathcal{L}$&lt;/span&gt;. That's because these pre-activation values are what you use to then calculate &lt;span class=&quot;math-container&quot;&gt;$\nabla_{W}\mathcal{L}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla_{b}\mathcal{L}$&lt;/span&gt; - the gradients with respect to the layer's weights and biases, plus also to calculate &lt;span class=&quot;math-container&quot;&gt;$\nabla_{z'}\mathcal{L}$&lt;/span&gt; - the gradients with respect to pre-activation values of the previous layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, once you have &lt;span class=&quot;math-container&quot;&gt;$\nabla_{z}\mathcal{L}$&lt;/span&gt; for the output layer you can just run back though each layer in turn, repeating the same calcuation steps again and again. It is this repetition over each layer that looks like classic backprop in code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming your neural network outputs a single value, the probablility of survival in the case of this dataset:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Your objective/loss function for binary classification &lt;em&gt;should&lt;/em&gt; be Binary Cross Entropy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your output layer should have a sigmoid activation function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you take the gradient of the loss function and backpropagate it throught that output layer's activation function, you end up with a starting delta &lt;span class=&quot;math-container&quot;&gt;$\nabla_{z} \mathcal{L}$&lt;/span&gt; i.e. the gradient of the loss function with respect to the pre-activation value of the first layer. This is a nice place to start the recursive backprop code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of this gradient is mathematically, per item:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{z} \mathcal{L} = \hat{y}_i - y_i$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. the difference the neural network estimate and the ground truth value. It takes this value because complex terms in the gradient of the loss function and the sigmoid activation function cancel out exactly. This is one reason why you often see sigmoid activation used with binary cross entropy - it is very convenient that the combination simplifies the gradient like this. You need to backpropagate each data point separately, and average the gradients for each batch/minibatch (actually you don't need to take this mean value, but doing so means that you don't need to adjust your learning rate as much to account for the number of items being processed per update step).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears to of worked for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether or not you were accidentally correct depends on whether you were getting an estimate for &lt;span class=&quot;math-container&quot;&gt;$\nabla_{z}\mathcal{L}$&lt;/span&gt; to start your backprop routine, or &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\hat{y}}\mathcal{L}$&lt;/span&gt;. I cannot find what that assumption is in your code, but the fact that it worked before and does not now suggests that you may of been correct, even though you may not of fully understood the maths.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-19T15:45:29.890" LastActivityDate="2019-10-19T15:45:29.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15976" PostTypeId="1" CreationDate="2019-10-19T12:43:28.273" Score="2" ViewCount="46" Body="&lt;p&gt;I have a neural network that should be able to classify documents to target label A. The problem is that the network is actually classifying label B, which is an easier task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the problem more clear: I need to classify documents from different sources. In the training data each source occurs repeatedly, but the network should be able to work on unknown sources. All documents from a single source have the same class. In this case, it is easier to identify sources than the target label so in practice the network is not really identifying the target label, but the source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to this problem is making sure that the model is bad at identifying the sources in the training data, while still attaching the right target labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the first step is to get two output layers, one for the target label and one for identifying which source it is from. My approach fails however at the training procedure: I want to minimize the loss on the target output, but maximize the loss on the non-target output. But if I maximize the loss on that non-target output, that does not mean that the network 'unlearns' the non-target labels. So the main question for the non-target output is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR; How do I define a training procedure that minimizes the loss on a non-target output layer, and then maximizes that loss on all layers before it. My goal is to have a network that is good at classifying label A, but bad at a related label B. If anyone wants to give a code example, my prefered framework is PyTorch.&lt;/p&gt;&#xA;" OwnerUserId="30614" LastActivityDate="2021-04-13T22:44:38.950" Title="Maximize loss on non-target variable" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15977" PostTypeId="1" CreationDate="2019-10-19T18:23:15.347" Score="12" ViewCount="161" Body="&lt;p&gt;Is there a way to understand, for instance, a multi-layered perceptron without hand-waving about them being similar to brains etc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: it is obvious that what a perceptron does is approximating a function; there might be many other ways, given a labelled dataset, to find the separation of the input area into smaller areas that correspond to the labels; however, these ways would probably be computationally rather ineffective, which is why they cannot be practically used. However, it seems that the iterative approach of finding such areas of separation may give a huge speed-up in many cases; then, natural questions arise why this speed-up may be possible, how it happens and in which cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could be sure that this question was investigated. If anyone could shed any light on the history of this question, I would be very grateful. So, why are neural networks useful and what do they do? I mean, from the practical and mathematical standpoint, without relying on the concept of &quot;brain&quot; or &quot;neurons&quot; which can explain nothing at all.&lt;/p&gt;&#xA;" OwnerUserId="30623" LastEditorUserId="1671" LastEditDate="2019-10-19T21:57:08.577" LastActivityDate="2019-10-29T06:19:31.340" Title="Is there a way to understand neural networks without using the concept of brain?" Tags="&lt;neural-networks&gt;&lt;theory&gt;&lt;history&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15978" PostTypeId="1" CreationDate="2019-10-19T19:17:05.203" Score="1" ViewCount="32" Body="&lt;p&gt;I'm trying to come up with a generative model that can input a name and output all valid formats of it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &quot;Bob Dylan&quot; could be an input and the gen model will output &quot;Dylan, Bob&quot;, &quot;B Dylan&quot;, &quot;Bob D&quot; and any other type of valid formatting of a person's name.  So given my example the gen model doesn't seem that complicated to build, but it also has to handle stuff like &quot;Dylan, Bob&quot; and &quot;B Dylan&quot;, but obviously the 2nd one it shouldn't output &quot;Bob Dylan&quot; as a potential output cause inferring that requires more than just &quot;B Dylan&quot;. Any ideas for a good Generative Model for this?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastActivityDate="2019-10-19T19:17:05.203" Title="What's a good generative model for creating valid formats of a person's name?" Tags="&lt;generative-model&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="15979" PostTypeId="2" ParentId="15977" CreationDate="2019-10-20T01:53:34.830" Score="11" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; I always like to think of Neural Networks as a generalization of &lt;strong&gt;logistic regression&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I too don't like that, traditionally, when introducing Neural Networks, books start with biological neurons and synapses, etc. I think its more beneficial to start from statistics and linear regression, then logistic regression and then neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A perceptron is essentially a simple binary logistic regressor (if you threshold the output). If you have many perceptrons that share the same input (i.e. a layer in a neural network), you can think of it as a multi-class logistic regressor. Now, by stacking one such layer after an other, you create a Multi-Layer Perceptron (MLP), which is a Neural Network with two layers. There is equivalent to two multi-class logistic regressors stacked one after the other. One notable thing that changes is the training technique here, i.e. backpropagation (because you don't have direct access to the targets from the hidden layer). Another thing that can change is the activation function (it's not always sigmoid in Neural Networks)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introduce sparse connectivity and weight sharing and you get a Convolutional Neural Network. Add a connection from a layer to its self (for the next timestep) and you get a Recurrent Neural Network. Likewise, you can reproduce any Neural Network through this reasoning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is an over-simplified way of presenting them, but I think you get the point.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-10-20T01:53:34.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15980" PostTypeId="1" CreationDate="2019-10-20T05:12:20.537" Score="0" ViewCount="69" Body="&lt;p&gt;I cannot find information in detail about autoencoder&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do with an autoencoder (and how can I do this), practically speaking? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the encoder (this part I think I understand) and a decoder (could not find much about this) part do? Can it for example show on an explainable way how patterns in the data are being represented?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read some papers that say that it can be used to denoise the input, how does this work? (Am I changing the values of my input)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it true that an autoencoder can be also done with PCA (if we assume linearity)?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-10-20T12:33:44.990" LastActivityDate="2019-10-20T12:33:44.990" Title="What can I do with an autoencoder?" Tags="&lt;machine-learning&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2019-10-21T21:09:04.890" ContentLicense="CC BY-SA 4.0" />
  <row Id="15981" PostTypeId="2" ParentId="15752" CreationDate="2019-10-20T12:07:36.430" Score="1" Body="&lt;h2&gt;Preface&lt;/h2&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Before answering this question, let me preface by stating that the&#xA;  following is simply MY answer as a Machine Learning Researcher and&#xA;  &quot;Hobbyist&quot; Theoretical Physicist, although I have strong feelings that&#xA;  my answer will most certainly be proven as true, I am more than sure&#xA;  that others will have differing opinions as with everything else in&#xA;  this world!&quot; ~ Me&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h1&gt;Answer&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;That being said, I would say that the answer to your question, is &lt;strong&gt;unequivocally YES!&lt;/strong&gt; I could write a short book on the subject &lt;em&gt;(that might be a good idea actually)&lt;/em&gt; but I will keep this response brief, although I am happy to answer any further questions your may have to which I possess the answers in the comments! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary reasons that I believe the newly blossoming field of Quantum Computation will have a massive impact on the field of Machine Learning in general, are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The most simple reason for my belief is that the primary goal of Machine Learning, is to create an entity which is capable of coherent, self-aware thought much like we exhibit as human beings. We know that the brain is what allows us to be capable of such feats, and thus I view the field something like brain counterfeiting. Without going into esoteric detail, there are many subtleties of the brain's workings which are thought to be quantum mechanical in operation, and thus would suggest that the path of least resistance to replicating the system would require a quantum mechanical computational medium.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The second primary rationale which solidifies my position, is the point which you briefly touched upon by stating that,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;We know that according to Shor's Algorithm Quantum Computing can reduce the complexity &gt; of breaking an encryption from O(2n) to O(2n2) (not sure).&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is primarily due to the quantum phenomena referred to as &lt;strong&gt;Super Position,&lt;/strong&gt; which allows for a multiple qubit gate to not only work with the options of 00, 01, 10 and 11 &lt;em&gt;(assuming a two qubit gate)&lt;/em&gt; but to also work with any combination in-between, during the computation. Although when the result is obtained &lt;em&gt;(this is what is referred to as &lt;strong&gt;collapsing the wave function&lt;/strong&gt;)&lt;/em&gt; you will only still have a resulting state space  with 2^n possibilities, where n is the number of qubits. This being said, there are very clever ways, by which one can design their algorithms to make full use of this &lt;em&gt;technically infinite&lt;/em&gt; computational space before observing the final results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I hope that my answer is helpful to you in some way, although I am aware that it is not a very in-depth answer, I feel that it hits the primary reasons why my personal belief is that there will be a wall which is hit, in the pursuit of a general AI, while we are limited to classical computation faculties; and thus will require quantum based computation before we are able to truly mimic the brain's most well kept secrets! The next couple decades should be VERY interesting in the fields, keep a close eye on the latest happenings!&lt;/p&gt;&#xA;" OwnerUserId="30631" LastActivityDate="2019-10-20T12:07:36.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15982" PostTypeId="2" ParentId="15980" CreationDate="2019-10-20T12:14:45.027" Score="1" Body="&lt;p&gt;An autoencoder learns to &lt;em&gt;compress&lt;/em&gt; data, and then to &lt;em&gt;decompress&lt;/em&gt; it again, recovering the original data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does this by learning a mapping from the original feature space to a lower-dimensional space, and then another mapping back. This is indeed, like PCA. The same technique is used to compress JPEG images to transmit them over the web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Autoencoders can denoise an image because the model will not be able to easily compress or decompress random noise, and so learns to ignore it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Autoencoders can also be used to find &lt;em&gt;embeddings&lt;/em&gt; of data that are likely to have semantic meaning. For example, you can use an autoencoder to compress English text, and a different one to compress French text. If you add some special constraints, then the models can learn to embed in the &lt;em&gt;same&lt;/em&gt; lower-dimensional space. This technique underlies many of the recent advances in machine translation.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2019-10-20T12:32:35.050" LastActivityDate="2019-10-20T12:32:35.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15983" PostTypeId="1" CreationDate="2019-10-20T12:53:16.597" Score="2" ViewCount="54" Body="&lt;p&gt;Are there are any openly available implementations of the &lt;a href=&quot;https://arxiv.org/pdf/1712.10285.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation&lt;/a&gt; paper?&lt;/p&gt;&#xA;" OwnerUserId="30632" LastEditorUserId="30632" LastEditDate="2019-11-30T04:55:10.667" LastActivityDate="2019-11-30T04:55:10.667" Title="Is there any open source implementation of the SBEED learning algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;function-approximation&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15984" PostTypeId="1" CreationDate="2019-10-20T14:37:20.490" Score="4" ViewCount="56" Body="&lt;p&gt;I want to know if there is a measure of how well two classes in Y are separable (linearly or not) based on their features in X. Easiest way of explaining this is to compare it to correlation coefficients, the higher the correlation the higher possiblity for successful regression based on given feature (at least in theory). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any measure that will tell me how well classes are separated based on input data features, before training a ML model? &lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2019-10-21T13:55:33.667" Title="Is there any measure of separability of classes?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15985" PostTypeId="2" ParentId="15894" CreationDate="2019-10-20T16:27:44.307" Score="0" Body="&lt;p&gt;A high-level solution is to use the &lt;a href=&quot;https://developer.nvidia.com/drive/drive-software&quot; rel=&quot;nofollow noreferrer&quot;&gt;NVIDIA DriveWorks SDK&lt;/a&gt;.  NVIDIA's Developer Blog has a good post, &quot;&lt;a href=&quot;https://devblogs.nvidia.com/point-cloud-processing-nvidia-driveworks-sdk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Point Cloud Processing with NVIDIA DriveWorks SDK&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The NVIDIA DriveWorks SDK contains a collection of CUDA-based low level point cloud processing modules optimized for NVIDIA DRIVE AGX platforms. The DriveWorks Point Cloud Processing modules include common algorithms that any AV developer working with point cloud representations would need, such as accumulation and registration.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe you don't need the AGX hardware for the problem you posted.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2019-10-20T16:27:44.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15986" PostTypeId="1" CreationDate="2019-10-20T17:32:01.927" Score="29" ViewCount="7364" Body="&lt;p&gt;To produce tangible results in the field of AI/ML, one must take theoretical results under the lens of computational complexity.&lt;/p&gt;&#xA;&lt;p&gt;Indeed, minimax effectively solves any two-person &amp;quot;board game&amp;quot; with win/loss conditions, but the algorithm quickly becomes untenable for games of large enough size, so it's practically useless asides from toy problems.&lt;/p&gt;&#xA;&lt;p&gt;In fact, this issue seems to cut at the heart of intelligence itself: the &lt;a href=&quot;https://plato.stanford.edu/entries/frame-problem/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Frame Problem&lt;/em&gt;&lt;/a&gt; highlights this by observing that any &amp;quot;intelligent&amp;quot; agent that operates under logical axioms must somehow deal with the explosive growth of computational complexity.&lt;/p&gt;&#xA;&lt;p&gt;So, we need to deal with &lt;em&gt;computational complexity&lt;/em&gt;: but that doesn't mean researchers must limit themselves with practical concerns. In the past, multilayered perceptrons were thought to be intractable (I think), and thus we couldn't evaluate their utility until recently. I've heard that Bayesian techniques are conceptually elegant, but they become computationally intractable once your dataset becomes large, and thus we usually use variational methods to compute the posterior, instead of naively using the exact solution.&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for more examples like this: What are examples of promising (or neat/interesting) AI/ML techniques that are computationally intractable (or uncomputable)?&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="2444" LastEditDate="2021-01-19T14:39:24.200" LastActivityDate="2021-01-19T14:39:24.200" Title="What are examples of promising AI/ML techniques that are computationally intractable?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;computational-complexity&gt;&lt;intractability&gt;" AnswerCount="7" CommentCount="0" FavoriteCount="17" ContentLicense="CC BY-SA 4.0" />
  <row Id="15987" PostTypeId="2" ParentId="15986" CreationDate="2019-10-20T21:24:27.673" Score="12" Body="&lt;p&gt;&lt;em&gt;Exact&lt;/em&gt; Bayesian inference is (often) &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;intractable&lt;/em&gt;&lt;/a&gt; (i.e. there is no closed-form solution, or numerical approximations are also computationally expensive) because it involves the computation of an integral over a range of real (or even floating-point) numbers, which can be intractable.&lt;/p&gt;&#xA;&lt;p&gt;More precisely, for example, if you want to find the parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta} \in \Theta$&lt;/span&gt; of a model given some data &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, then &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_inference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian inference is just the application of the Bayes' theorem&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;p(\mathbf{\theta} \mid D) &#xA;&amp;amp;= \frac{p(D \mid \mathbf{\theta}) p(\mathbf{\theta})}{p(D)} \\&#xA;&amp;amp;= \frac{p(D \mid \mathbf{\theta}) p(\mathbf{\theta})}{\int_{\Theta} p(D \mid \mathbf{\theta}^\prime) p(\mathbf{\theta}^\prime) d \mathbf{\theta}^\prime} \\&#xA;&amp;amp;= \frac{p(D \mid \mathbf{\theta}) p(\mathbf{\theta})}{\int_{\Theta} p(D, \mathbf{\theta}^\prime) d \mathbf{\theta}^\prime } \tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{\theta} \mid D)$&lt;/span&gt; is the posterior (which is what you want to find or compute), &lt;span class=&quot;math-container&quot;&gt;$p(D \mid \mathbf{\theta})$&lt;/span&gt; is the likelihood of your data given the (fixed) parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{\theta})$&lt;/span&gt; is the prior and &lt;span class=&quot;math-container&quot;&gt;$p(D) = \int_{\Theta} p(D \mid \mathbf{\theta}^\prime) p(\mathbf{\theta}^\prime) d \mathbf{\theta}^\prime$&lt;/span&gt; is the evidence of the data (which is an integral given that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; is assumed to be a continuous random variable), which is intractable because the integral is over all possible values of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;${\Theta}$&lt;/span&gt;. If all terms in \ref{1} were tractable (polynomially computable), then, given more data &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, you could iteratively keep on updating your posterior (which becomes your prior on the next iteration), and exact Bayesian inference would become tractable.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Variational_Bayesian_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;variational Bayesian approach&lt;/a&gt; casts the problem of inferring &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{\theta} \mid D)$&lt;/span&gt; (which requires the computation of the intractable &lt;em&gt;evidence&lt;/em&gt; term) as an optimization problem, which &lt;em&gt;approximately&lt;/em&gt; finds the posterior, more precisely, it approximates the intractable posterior, &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{\theta} \mid D)$&lt;/span&gt;, with a tractable one, &lt;span class=&quot;math-container&quot;&gt;$q(\mathbf{\theta} \mid D)$&lt;/span&gt; (the &lt;em&gt;variational distribution&lt;/em&gt;). For example, the important &lt;a href=&quot;https://arxiv.org/abs/1312.6114&quot; rel=&quot;nofollow noreferrer&quot;&gt;variational auto-encoder (VAEs)&lt;/a&gt; paper (which did not introduce the variational Bayesian approach) uses the variational Bayesian approach to approximate a posterior in the context of neural networks (that represent distributions), so that existing machine (or deep) learning techniques (that is, gradient descent with back-propagation) can be used to learn the parameters of a model.&lt;/p&gt;&#xA;&lt;p&gt;The variational Bayesian approach (VBA) becomes always more appealing in machine learning. For example, &lt;a href=&quot;https://arxiv.org/abs/1505.05424&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural networks&lt;/a&gt; (which can partially solve some of the inherent problems of non-Bayesian neural networks) are usually inspired by the results reported in &lt;a href=&quot;https://arxiv.org/abs/1312.6114&quot; rel=&quot;nofollow noreferrer&quot;&gt;the VAE paper&lt;/a&gt;, which shows the feasibility of the VBA in the context of deep learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T14:26:19.770" LastActivityDate="2021-01-19T14:26:19.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15988" PostTypeId="2" ParentId="15916" CreationDate="2019-10-20T23:21:02.700" Score="1" Body="&lt;p&gt;Mixing loss functions is very possible.  For example, in the case of &lt;a href=&quot;https://www.tensorflow.org/tutorials/generative/style_transfer&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural style transfer&lt;/a&gt;, there is a style loss and a content loss. Both of them are backpropagated through the network. The final loss used for the backpropagation is a weighted sum of the losses. In the case of style transfer, it ensures that the image generated is not only imitating the style of the style image, but also keeping the original content. Sometimes there is also a third loss called variation loss. This is also applied to a weight and summed. Note the weight is a hyperparameter and is not changed during training. Also, the loss should not measure the same purpose, but rather a different purpose that you want to optimize together. Example code using PyTorch:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss1 = torch.nn.MSELoss(a,b)&#xA;loss2 = torch.nn.MSELoss(b,c)&#xA;loss = loss1 * alpha + loss2 * beta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="2444" LastEditDate="2019-10-22T00:47:03.753" LastActivityDate="2019-10-22T00:47:03.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15989" PostTypeId="2" ParentId="15971" CreationDate="2019-10-20T23:34:27.090" Score="1" Body="&lt;p&gt;Note the X index is training set size. For the first and second case, teh training set size starts at 0(or 1). The model will overfit certainly at that data size. When data size increases, the model overfits less and less and eventually the model have enough data samples that it won't overfit. The data size continue to increase and the model performance increase as well. To a certain point, the validation loss increase start to diminish and the model slightly overfits the samples.  For the third graph, it seems like originally teh loss is low and started to increase.&#xA;Hopes it help&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-20T23:34:27.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15990" PostTypeId="2" ParentId="15986" CreationDate="2019-10-21T00:15:49.743" Score="15" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/10377/2444&quot;&gt;AIXI&lt;/a&gt; is a Bayesian, non-Markov, reinforcement learning and artificial general intelligence agent that is &lt;a href=&quot;https://en.wikipedia.org/wiki/Computable_function#Uncomputable_functions_and_unsolvable_problems&quot; rel=&quot;noreferrer&quot;&gt;incomputable&lt;/a&gt;, given the involved &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov_complexity#Uncomputability_of_Kolmogorov_complexity&quot; rel=&quot;noreferrer&quot;&gt;incomputable Kolmogorov complexity&lt;/a&gt;. However, there are approximations of AIXI, such as AIXItl, described in &lt;a href=&quot;http://hutter1.net/ai/uaibook.htm&quot; rel=&quot;noreferrer&quot;&gt;Universal Artificial Intelligence:  Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability&lt;/a&gt; (2005), by Marcus Hutter (the original author of AIXI), and &lt;a href=&quot;https://arxiv.org/abs/0909.0801&quot; rel=&quot;noreferrer&quot;&gt;MC-AIXI-CTW&lt;/a&gt; (which stands for Monte Carlo AIXI Context-Tree Weighting). Here is a Python implementation of MC-AIXI-CTW: &lt;a href=&quot;https://github.com/gkassel/pyaixi&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/gkassel/pyaixi&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-10-21T00:15:49.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15991" PostTypeId="2" ParentId="15986" CreationDate="2019-10-21T00:30:36.727" Score="3" Body="&lt;p&gt;In general, &lt;a href=&quot;https://arxiv.org/pdf/1301.2308.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;partially-observable Markov decision processes (POMDPs) are also computationally intractable to solve exactly&lt;/a&gt;. However, there are &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process#Approximate_POMDP_solutions&quot; rel=&quot;nofollow noreferrer&quot;&gt;several approximations methods&lt;/a&gt;. See, for example, &lt;a href=&quot;https://arxiv.org/abs/1106.0234&quot; rel=&quot;nofollow noreferrer&quot;&gt;Value-Function Approximations for Partially Observable Markov Decision Processes&lt;/a&gt; (2000) by Milos Hauskrecht.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-10-21T00:30:36.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15992" PostTypeId="1" CreationDate="2019-10-21T03:17:13.280" Score="3" ViewCount="93" Body="&lt;p&gt;As part of a research project for college, I would like to understand what many of you astern to be the risks associated with regulating Artificial Intelligence. Such as whether regulation is too risky in regards to limiting progress or too risky in regards to uninformed regulation.&lt;/p&gt;&#xA;" OwnerUserId="30641" LastEditorUserId="2444" LastEditDate="2019-11-21T03:25:09.037" LastActivityDate="2019-11-21T22:51:07.327" Title="What are the risks associated with regulating AI?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;risk-management&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15993" PostTypeId="1" AcceptedAnswerId="16002" CreationDate="2019-10-21T03:28:50.820" Score="4" ViewCount="146" Body="&lt;p&gt;I want to build a multivariable and multivariate regression model in Keras (with TensorFlow as backend), that is, a regression model with multiple values as input (multivariable) and output (multivariate). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The independent variables are, for example, the length, weight, force, etc., and the dependent variables are the torque, friction, heat, temperature, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best approach to achieve that? Any guidance before I start?  (If anyone can share any example code/notebook/code would be great as well).&lt;/p&gt;&#xA;" OwnerUserId="30642" LastEditorUserId="2444" LastEditDate="2019-12-06T15:09:14.453" LastActivityDate="2019-12-06T15:09:14.453" Title="What is the best approach for multivariable and multivariate regression?" Tags="&lt;machine-learning&gt;&lt;keras&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15994" PostTypeId="1" AcceptedAnswerId="16003" CreationDate="2019-10-21T04:36:29.380" Score="2" ViewCount="434" Body="&lt;p&gt;There's this 7 player social deduction game called Secret Hitler, and I have been trying to find a self-learning AI algorithm to learn how to play this game for a while.  Basically, four players are given a liberal role, two players are given a fascist role, and 1 player is given a hitler role.  The liberals and hitler do not know any other roles, and the fascists know everyone's roles.  During a turn, a president elects a chancellor based on a yes/no vote and then the government passes a policy (either liberal or fascist) that is drawn from a randomized deck. At certain points in the game, different special abilities come into play, like executing a player or investigating their role. To win the game, the liberals must either enact 5 liberal policies or kill hitler; the fascists must enact 6 fascist policies or get hitler enacted as the chancellor after 3 fascist policies have been enacted.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there are other details that are irrelevant that I didn't mention, but those are the general rules. It seems simple enough to build a visual implementation in a language like Java, but there are so many moving pieces that I would have to account for.  I doubt that simply making random moves at first and learning off of the bad/good moves would work, because I need a way for agents to make moves based on which roles they know.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, AlphaZero wouldn't work here, and I'm struggling to find any algorithm that &lt;em&gt;would&lt;/em&gt; work for this (or any other social deduction game).  Do I have to write my own algorithm?  I'm slightly confident that this is a case of supervised learning where I can give weight to the nodes in a neural network that correspond to wins, but please correct me if I'm incorrect. &lt;/p&gt;&#xA;" OwnerUserId="23897" LastEditorUserId="23897" LastEditDate="2019-10-21T19:45:48.263" LastActivityDate="2019-10-21T19:45:48.263" Title="What would be the most effective self-learning algorithm for a 7 player social deduction game?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;game-ai&gt;&lt;games-of-chance&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15995" PostTypeId="1" CreationDate="2019-10-21T05:32:57.103" Score="4" ViewCount="351" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bMmxd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bMmxd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't really understand what this equation is saying or what the purpose of the ELBO is. How does it help us find the true posterior distribution?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2020-11-07T01:14:00.640" LastActivityDate="2020-11-07T01:14:00.640" Title="What's going on in the equation of the variational lower bound?" Tags="&lt;deep-learning&gt;&lt;statistical-ai&gt;&lt;variational-autoencoder&gt;&lt;evidence-lower-bound&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15996" PostTypeId="2" ParentId="15976" CreationDate="2019-10-21T11:14:02.110" Score="0" Body="&lt;p&gt;You can maximize the loss by doing &lt;code&gt;1-loss&lt;/code&gt; (if the range of the loss is between 1 and 0. However, I am not sure if that would help. The network can just propagate the weights of the source classifier to always output the wrong answer. a better approach is to hand craft features of the document to make it less identifiable as the source. If the document have stated the source, remove it. This may help. Also, another way is to use documents from more sources so the network cannot use the source to classify the label. Hope I can help you. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-21T11:14:02.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15997" PostTypeId="2" ParentId="15995" CreationDate="2019-10-21T11:49:09.993" Score="3" Body="&lt;p&gt;From &lt;a href=&quot;http://users.umiacs.umd.edu/~xyang35/files/understanding-variational-lower.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;, as you found here, &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is an observed variable and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is a hidden variable; &lt;span class=&quot;math-container&quot;&gt;$p(X)$&lt;/span&gt; is the density function of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. The posterior distribution of the hidden variables can then be written as follows using the Bayes’ Theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(Z|X) = \frac{p(X|Z)p(Z)}{p(X)} = \frac{p(X|Z)p(Z)}{\int_Zp(X,Z)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now base on what you post, if we denote that &lt;span class=&quot;math-container&quot;&gt;$L= \mathbb{E}_q [\log p(X, Z)] + H[Z]$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$q(Z)$&lt;/span&gt; is a distribution we use to approximation the true posterior distribution &lt;span class=&quot;math-container&quot;&gt;$p(Z|X)$&lt;/span&gt; in VB and &lt;span class=&quot;math-container&quot;&gt;$H[Z] = -\mathbb{E}_q [\log q(Z)]$&lt;/span&gt;), then it is obvious that &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is a lower bound of the log probability of the observations.&#xA;As a result, if in some cases we want to maximize the marginal probability (the log probability of the observations), we can instead maximize its variational lower bound &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. As a real example, you can follow the &quot;Multiple Object Recognition with Visual Attention&quot; example in the referenced document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, the term &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; will be presented in KL-divergence that will be used to measuring the similarity of two distributions. Be aware that there is progress on the bound in &lt;a href=&quot;http://proceedings.mlr.press/v80/alemi18a/alemi18a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (Fixing a Broken ELBO).&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-10-21T13:00:23.927" LastActivityDate="2019-10-21T13:00:23.927" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15998" PostTypeId="2" ParentId="15984" CreationDate="2019-10-21T13:55:33.667" Score="2" Body="&lt;p&gt;Are you thinking something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree_learning#Information_gain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Information Gain&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Information Gain basically uses the concept of information entropy to determine if splitting a variable is useful.&lt;/p&gt;&#xA;" OwnerUserId="30662" LastActivityDate="2019-10-21T13:55:33.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15999" PostTypeId="1" CreationDate="2019-10-21T14:20:48.283" Score="2" ViewCount="50" Body="&lt;p&gt;I am looking for research and experience working with ML models to ingest data for tasks, like text analysis, and creates a system that copies (or in other words enciphers) the input data, to then reproduce it in the future without the original.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in how ML models can be used in this way to obfuscate information without too much information loss by the model, e.g. overfitting on purpose to create a new representation of the input information.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="2444" LastEditDate="2019-10-22T01:07:25.107" LastActivityDate="2019-10-22T06:54:03.410" Title="Using ML to encypher data for production" Tags="&lt;machine-learning&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16000" PostTypeId="2" ParentId="15999" CreationDate="2019-10-21T14:44:54.640" Score="2" Body="&lt;p&gt;It sounds like you are trying to &lt;em&gt;compress&lt;/em&gt; data, and then &lt;em&gt;recover&lt;/em&gt; the same data later. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common tool for this task is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;autoencoder&lt;/a&gt;. This model accepts data as input, and then learns to compress it and decompress it to produce something as close as possible to the original data. By making the middle layer of an autoencoder narrower, you can make the compression more lossy. By making it wider, you can make it less lossy. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-21T14:44:54.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16001" PostTypeId="2" ParentId="15995" CreationDate="2019-10-21T14:54:31.187" Score="2" Body="&lt;p&gt;The use of KL provides a more intuitive way of what the ELBO is attempting to maximize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, we want to find a posterior approximation such that &lt;span class=&quot;math-container&quot;&gt;$p(z\mid x) \approx q(z)\in\mathcal{Q}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$KL(q(z)\parallel p(z\mid x)) \rightarrow \min_{q(z)\in\mathcal{Q}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result of this, while finding this optimal posterior approximation, we maximize the probability of all the observed data &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Note that the evidence is usually intractable. Thus, can express the &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*} \log p(x) &amp;amp;= \int q(z) \log p(x)dz \\&#xA;&amp;amp;= \int q(z) \log\frac{p(x,\theta)}{p(\theta\mid x)}dz \\ &#xA;&amp;amp;= \int q(z) \log\frac{p(x,z)q(z)}{p(\theta\mid x)q(z)}dz\\ &#xA;&amp;amp;= \int q(z) \log\frac{p(x,z)}{q(z)}dz + \int q(z) \log\frac{q(z)}{p(z\mid x)}dz \\ &amp;amp;= \mathcal{L}(q(z)) + KL(q(z)\parallel p(z\mid x)) \end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, KL just gives us the difference between &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. We want to make this difference close to zero meaning that &lt;span class=&quot;math-container&quot;&gt;$q=p$&lt;/span&gt;. So, minimizing the KL is the same as maximizing the ELBO, and as a result, we obtain the lower bound in your expression. If you expand your bound, you can find a nice interpretion: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \begin{align*} &#xA;\mathcal{L}(q(z)) &amp;amp;= \int q(z) \log\frac{p(x,z)}{q(z)}dz \\&#xA;&amp;amp;= \mathbb{E}_{q(z)} \log p(x\mid z) - KL(q(z)\parallel p(z)) \end{align*} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we optimize this expression, we want to find a &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; that fits our data properly and also is really close to true posterior. Thus, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{q(z)} \log p(x\mid z)$&lt;/span&gt; act as a data term and &lt;span class=&quot;math-container&quot;&gt;$KL(q(z)\parallel p(z)) $&lt;/span&gt; as a regularizer. &lt;/p&gt;&#xA;" OwnerUserId="29857" LastEditorUserId="29857" LastEditDate="2019-11-03T14:50:42.060" LastActivityDate="2019-11-03T14:50:42.060" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16002" PostTypeId="2" ParentId="15993" CreationDate="2019-10-21T15:07:46.810" Score="0" Body="&lt;p&gt;You create a model like this very easily with keras. Follow these steps.&lt;/p&gt;&#xA;&lt;h1&gt;Data Exploring&lt;/h1&gt;&#xA;&lt;p&gt;Before building a machine learning model, you must explore your data first. An approach is to use libraries to visualize the data as graphs. You can use the tool &lt;code&gt;pandas-profiling&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/exploring-your-data-with-just-1-line-of-python-4b35ce21a82d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/exploring-your-data-with-just-1-line-of-python-4b35ce21a82d&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Points to look for when exploring the data&lt;/h2&gt;&#xA;&lt;p&gt;You should look for the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Distribution of labels and input features&lt;/li&gt;&#xA;&lt;li&gt;Range of input features and labels&lt;/li&gt;&#xA;&lt;li&gt;Missing label or input feature&lt;/li&gt;&#xA;&lt;li&gt;Nan on value&lt;/li&gt;&#xA;&lt;li&gt;Outliers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You should look out for them and use appropriate data cleaning or filtering method to clean them. For example, Nan or missing value may be substituted with 0 and outliers will be removed. Input features may also be normalized.&lt;/p&gt;&#xA;&lt;h1&gt;Building the model&lt;/h1&gt;&#xA;&lt;p&gt;Keras is a easy tool for building machinea learning model. For how to build a basic MLP ( Multi-Layer Perceptron), you can refer to the example code and the resource.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Experimenting&lt;/h1&gt;&#xA;&lt;p&gt;After creating the basic model, you can start experimenting with stuff like:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Different model structure like adding layers and changing hidden layer size&lt;/li&gt;&#xA;&lt;li&gt;Different Optimizer&lt;/li&gt;&#xA;&lt;li&gt;Different hyperparameters like learning rate and batch size&lt;/li&gt;&#xA;&lt;li&gt;Different input features&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You will gain more experience as you go. There is a lot of ways to improve the performance of the model.&lt;/p&gt;&#xA;&lt;h1&gt;Bewares&lt;/h1&gt;&#xA;&lt;p&gt;Here are things to beware:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your model doesn't work out, don't worry. Try another method or tune parameters until it work.&lt;/li&gt;&#xA;&lt;li&gt;If the testing loss become very high while training loss is low, you can add dropout to prevent this from happening. This is called overfitting.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Hope I can help you and have fun!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-10-21T15:07:46.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16003" PostTypeId="2" ParentId="15994" CreationDate="2019-10-21T15:16:16.157" Score="1" Body="&lt;p&gt;It is very likely that you want an algorithm like &lt;a href=&quot;http://modelai.gettysburg.edu/2013/cfr/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Counterfactual Minimax Regret&lt;/a&gt;. This algorithm has several variants, but they differ mostly in their efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CFR is the algorithm that was used to &lt;a href=&quot;https://poker.cs.ualberta.ca/publications/NIPS07-cfr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;solve 2-Player Poker&lt;/a&gt;, although the solution comes from one of its more advanced versions. The algorithm is highly applicable to other games of incomplete information and games with roles, like Secret Hitler. Essentially, it learns how to act so that the actions it takes leak as little information to the other players as possible about its role (or hand of cards), while also learning about the roles of other players from their actions, and maximizing its chances of winning the game.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-21T15:16:16.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16004" PostTypeId="1" CreationDate="2019-10-21T15:29:02.173" Score="0" ViewCount="140" Body="&lt;p&gt;I am working on a project that takes signals from the brain, preprocesses them, and then makes the machine learn about what human is thinking about. I am struck on preprocessing the signal (incoming from the EEG). I am having a problem when I attempt to remove noise. I used SVM but to no avail. I need some other suggestions from experts who have worked on a project similar to this. What can I do to preprocess the signal?&lt;/p&gt;&#xA;" OwnerUserId="27469" LastEditorUserId="30426" LastEditDate="2021-05-18T22:03:21.803" LastActivityDate="2021-05-18T22:03:21.803" Title="How can I remove the noise from an EEG signal?" Tags="&lt;machine-learning&gt;&lt;signal-processing&gt;&lt;data-preprocessing&gt;" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16005" PostTypeId="1" CreationDate="2019-10-21T16:18:23.513" Score="2" ViewCount="71" Body="&lt;p&gt;I'm new to object detectors and segmentation. I want to localize digits on a plate as fast as possible. All images of the dataset are normalized to &lt;span class=&quot;math-container&quot;&gt;$300 \times 60$&lt;/span&gt;. There are different approaches to solve the problem. For example, binarization + connected component labeling, vertical and horizontal projection. The aforementioned approaches fail in &lt;em&gt;ambient lights&lt;/em&gt;, &lt;em&gt;noises&lt;/em&gt;, and &lt;em&gt;shadows&lt;/em&gt;. Also, there are other approaches such as STN-OCR (based on convolutional recurrent neural networks) that need a lot of plates with different composition of numbers. I have limited plates with the same numbers (about 1000 different numbers) but totally 10000 plates in different illuminations and noises. I have a good OCR (without segmentation), so I need a network just localize digits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any deep learning-based architecture for this purpose? Can I use faster RCNN? Yolo? SSD?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trained Faster RCNN in Matlab, but it detects too many random bounding boxes for each plate. What could be the problem?&lt;/p&gt;&#xA;" OwnerUserId="30669" LastEditorUserId="2444" LastEditDate="2019-10-22T00:59:58.200" LastActivityDate="2019-10-22T00:59:58.200" Title="Is there a deep learning-based architecture for digit localisation?" Tags="&lt;deep-learning&gt;&lt;optical-character-recognition&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16006" PostTypeId="1" CreationDate="2019-10-21T16:25:22.597" Score="2" ViewCount="37" Body="&lt;p&gt;As I stated in my question, I would like to know the underlying pipeline and machine learning models that are used to classify intents and identify entities in IBM Watson Assistant and Microsoft LUIS services.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched on different websites and the documentations of those services, but I did not find anything. However, there are some blogs mentioned that IBM Watson is trained using one billion words from Wikipedia but there is no reference to support that claim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I highly appreciate if anyone could refer me to a doc/blog that answers my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance :)&lt;/p&gt;&#xA;" OwnerUserId="30671" LastActivityDate="2019-10-21T16:25:22.597" Title="What is the underlying model of IBM Watson Assistant and Microsoft LUIS?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16007" PostTypeId="2" ParentId="15986" CreationDate="2019-10-21T17:13:14.770" Score="5" Body="&lt;p&gt;The &lt;a href=&quot;https://intelligence.org/2016/09/12/new-paper-logical-induction&quot; rel=&quot;noreferrer&quot;&gt;logical induction algorithm&lt;/a&gt; can make predictions about whether mathematical statements are true or false, which are eventually consistent; e.g. if &lt;code&gt;A&lt;/code&gt; is true, its probability will eventually reach 1; if &lt;code&gt;B&lt;/code&gt; implies &lt;code&gt;C&lt;/code&gt; then &lt;code&gt;C&lt;/code&gt;'s probability will eventually reach or exceed &lt;code&gt;B&lt;/code&gt;'s; the probability of &lt;code&gt;D&lt;/code&gt; will eventually be the inverse of &lt;code&gt;not(D)&lt;/code&gt;; the probabilities of &lt;code&gt;E&lt;/code&gt; and &lt;code&gt;F&lt;/code&gt; will eventually reach or exceed that of &lt;code&gt;E AND F&lt;/code&gt;; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can also give consistent predictions about itself, e.g. &quot;the logical induction algorithm will predict the probability of X to be Y at timestep T&quot;, whilst avoiding paradoxes like the liar's paradox.&lt;/p&gt;&#xA;" OwnerUserId="30672" LastActivityDate="2019-10-21T17:13:14.770" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16008" PostTypeId="1" AcceptedAnswerId="16011" CreationDate="2019-10-21T17:48:08.497" Score="3" ViewCount="89" Body="&lt;p&gt;I have a couple of small questions about the David Silver lecture about reinforcement learning, &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/MC-TD.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture slides&lt;/a&gt;  (slides 23, 24). More specifically it is about the temporal difference algorithm: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s_{t}) \leftarrow  V(s_t)+ \alpha \left[ G_{t+1}+\gamma V(s_{t+1})- V(s_t) \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is our discount rate and &lt;span class=&quot;math-container&quot;&gt;$\alpha $&lt;/span&gt; the learning rate. &#xA;In the example given in the lecture slides we observe the following paths: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(A,1,B,0), (B,1), (B,1), (B,1), (B,1), (B,1), (B,1), (B,0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meaning for the first trajectory we are in state &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, get reward &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, get to state &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and get reward &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and the game finishes. For the second trajectory we start in state &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, get reward &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and the game finishes ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let`s say we initialize all states with value &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and choose &lt;span class=&quot;math-container&quot;&gt;$\alpha=0.1, \gamma=1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is whether the following &quot;implementation&quot; of the &lt;span class=&quot;math-container&quot;&gt;$TD(0)$&lt;/span&gt; algorithm for the first two of the above observed trajectories correct?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(a)\leftarrow0 + 0.1(1+0-0)= 0.1; \quad V(b)\leftarrow0+0.1(1+0-0)=0.1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(b)\leftarrow0.1+(0.1)(1+0-0.1)= 0.19$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;? If so, why dont we use the updated value function for &lt;span class=&quot;math-container&quot;&gt;$V(b)$&lt;/span&gt; to also update our value for &lt;span class=&quot;math-container&quot;&gt;$V(a)$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My third question is about the statement that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$TD(0)$&lt;/span&gt; converges to solution of max likelihood Markov model&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;this means that if we keep sampling and apply the &lt;span class=&quot;math-container&quot;&gt;$TD(0)$&lt;/span&gt; algorithm that the thereby obtained solution converges towards the ML-estimate of that sample using the Markov Model? Why dont we just use the ML-estimate immediately?&lt;/p&gt;&#xA;" OwnerUserId="30369" LastEditorUserId="30369" LastEditDate="2019-10-21T19:56:29.870" LastActivityDate="2019-10-21T20:09:46.813" Title="Confusion about temporal difference learning" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16009" PostTypeId="2" ParentId="15986" CreationDate="2019-10-21T19:04:00.377" Score="10" Body="&lt;p&gt;This question gets at a really interesting fact about AI research in general: &lt;strong&gt;AI is hard&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, almost every AI problem is computationally hard (typically NP-Hard, or #P-Hard). This means that most new areas of AI research starts out by characterizing some problem that is intractable, and proposing an algorithm that technically works, but is too slow to be useful. However, that's &lt;strong&gt;not the whole story&lt;/strong&gt;. Usually AI researchers then proceed to develop &lt;em&gt;tractable&lt;/em&gt; techniques according to one of two schools:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Algorithms that usually work in practice, and are always fast, but are not completely correct.&lt;/li&gt;&#xA;&lt;li&gt;Algorithms that are always correct, and are usually fast, but are sometimes very slow, or only work on specific kinds of sub-problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Take together, these let AI address most problems. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_traversal&quot; rel=&quot;nofollow noreferrer&quot;&gt;Search&lt;/a&gt; was developed as a general purpose AI technique for solving &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;planning&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_theorem_proving&quot; rel=&quot;nofollow noreferrer&quot;&gt;logic&lt;/a&gt; problems. The first algorithm, called the &lt;a href=&quot;https://en.wikipedia.org/wiki/General_Problem_Solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;general problem solver&lt;/a&gt;, always worked, but was extremely slow. Eventually, we developed heuristic guided search techniques like &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A*&lt;/a&gt;, domain specific tricks like &lt;a href=&quot;https://en.wikipedia.org/wiki/Graphplan&quot; rel=&quot;nofollow noreferrer&quot;&gt;GraphPlan&lt;/a&gt;, and stochastic search techniques like &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte-Carlo Tree Search&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Bayesian Learning (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_inference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Inference&lt;/a&gt;) has been known since the 1800's, but it is known to involve either the computation of intractable integrals, or the creation of exponentially sized discrete tables, making it &lt;a href=&quot;https://en.wikipedia.org/wiki/NP-hardness&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP-Hard&lt;/a&gt;. A very simple algorithm involves applying brute force and enumerating all of the options, but this is too slow. Eventually, we developed techniques like &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gibbs Sampling&lt;/a&gt; (that is always fast, and usually right), or &lt;a href=&quot;https://en.wikipedia.org/wiki/Variable_elimination&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variable Elimination&lt;/a&gt; (that is always right, and usually fast). Today we can solve most problems of this kind very well.&lt;/li&gt;&#xA;&lt;li&gt;Reasoning about language was thought to be very hard (see the &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Frame Problem&lt;/a&gt;), because there are an infinite number of possible sentences, and an infinite number of possible contexts they could be used in. Exact approaches based on rules did not work. Eventually we developed probabilistic approaches like &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hidden Markov Models&lt;/a&gt; and &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.248.4448&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Neural Networks&lt;/a&gt;, that aren't certain to work, but work so well in practice that language problems are, if not completely solve, getting &lt;a href=&quot;https://translate.google.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;very close&lt;/a&gt;.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Games of chance, like Poker, were thought to be impossible, because they are &lt;a href=&quot;https://en.wikipedia.org/wiki/%E2%99%AFP-complete&quot; rel=&quot;nofollow noreferrer&quot;&gt;#P-Hard&lt;/a&gt; to complete exactly (this is &lt;em&gt;harder&lt;/em&gt; than NP-Hard). There will probably never be an exact algorithm for these. In spite of this, techniques like &lt;a href=&quot;https://poker.cs.ualberta.ca/publications/2015-ijcai-cfrplus.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CFR+&lt;/a&gt; can derive solutions that are so close to exactly perfect that you would need to play for decades against them to tell the difference.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, what's &lt;strong&gt;still&lt;/strong&gt; hard?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.hindawi.com/journals/complexity/2018/1591878/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inferring&lt;/a&gt; the &lt;em&gt;structure&lt;/em&gt; of a Bayesian network. This is closely related to the problem of &lt;em&gt;causality&lt;/em&gt;. It's #P-Hard, but we don't currently have any good algorithms to even do this approximately very well. This is an active area of research.&lt;/li&gt;&#xA;&lt;li&gt;Picking a machine learning algorithm to use for an arbitrary problem. The &lt;a href=&quot;https://ai.stackexchange.com/questions/15650/what-are-the-implications-of-the-no-free-lunch-theorem-for-machine-learning/15651#15651&quot;&gt;No Free Lunch theorem&lt;/a&gt; tells us this is not possible in general, but it seems like we ought to be able to do it pretty well in practice.&lt;/li&gt;&#xA;&lt;li&gt;More to come...?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="20044" LastEditDate="2019-10-22T14:49:18.230" LastActivityDate="2019-10-22T14:49:18.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16011" PostTypeId="2" ParentId="16008" CreationDate="2019-10-21T20:02:35.117" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;My first question is whether the following &amp;quot;implementation&amp;quot; of the (0) algorithm for the first two of the above observed trajectories correct?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(a)\leftarrow0 + 0.1(1+0-0)= 0.1; \quad V(b)\leftarrow0+0.1(1+0-0)=0.1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(b)\leftarrow0.1+(0.1)(1+0-0.1)= 0.19$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your calculations for the first trajectory &lt;span class=&quot;math-container&quot;&gt;$(A,1,B,0)$&lt;/span&gt; is incorrect for either TD or Monte Carlo. For some reason you have assigned either an immediate reward or return of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to the second step, whilst in the example, it is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for both the sampled return and the single-step TD target.&lt;/p&gt;&#xA;&lt;p&gt;In addition, you quote this update rule for single-step TD:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s_{t}) \leftarrow  V(s_t)+ \alpha \left[ G_{t+1}+\gamma V(s_{t+1})- V(s_t) \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;. . . actually that is not the usual notation. The symbol &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; is normally used to show a &amp;quot;return&amp;quot; value - a sum of rewards (often weighted by some factor, such as &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;). The usual way of showing the TD update rule would be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s_{t}) \leftarrow  V(s_t)+ \alpha \left[ r_{t+1}+\gamma V(s_{t+1})- V(s_t) \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;i.e. using the immediate reward. This might be a simple typo, however I am explaining this because it may behind your incorrect calculation.&lt;/p&gt;&#xA;&lt;p&gt;The correct calculation is not very much different from yours though:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(a)\leftarrow0 + 0.1(1+0-0)= 0.1; \quad V(b)\leftarrow0+0.1(0+0-0)=0.0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$V(b)\leftarrow0.0+(0.1)(1+0-0.0)= 0.1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If so, why dont we use the updated value function for &lt;span class=&quot;math-container&quot;&gt;$V(b)$&lt;/span&gt; to also update our value for &lt;span class=&quot;math-container&quot;&gt;$V(a)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can, and would do this in either of the following situations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In online learning, you experience a trajectory with states in order (A,B) again&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In offline learning, you repeat the previous experience in batch learning or using experience replay&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It is worth noting that if you take a small batch of data and repeat it again and again to update the value functions, that they will converge to values depending on your data set. That is what the slide is explaining in the lecture - highlighting the difference that TD and MonteCarlo will make when you do this. However, if that data set is a very small subsample of possible random behaviour in the environment, then you may not create an accurate value function, but instead it will be the best one that you can given the limited data. If it is easy to collect more experience, then that is often preferable.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why dont we just use the [maximum-likelihood]-estimate immediately?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Because it is not directly useful for a value prediction task, and you would need some mechanism to use that maximum likelihood MDP model to generate value predictions. With TD, you are already in the process of making this estimate*.&lt;/p&gt;&#xA;&lt;p&gt;You could take the existing samples, and use them to generate the parameters of an MRP (Markov &lt;em&gt;Reward&lt;/em&gt; Process, as there are no example actions in the trajectory) based on the observations. That &amp;quot;best guess&amp;quot; MRP is your maximum likelihood MDP model, and would evaluate the same as your converged repeated TD batch over the samples.&lt;/p&gt;&#xA;&lt;p&gt;The main difference explained by the slide is that Monte Carlo will converge to &lt;span class=&quot;math-container&quot;&gt;$V(a) = 1$&lt;/span&gt; because the only sample with A in it has a return of 1 following state A. Whilst TD will converge to &lt;span class=&quot;math-container&quot;&gt;$V(a) = 1.75$&lt;/span&gt;, because it treats the same sample as the only instance of state progression from A - e.g state A &amp;quot;always&amp;quot; has an immediate reward of 1 then goes to state B. Both algorithms will converge to &lt;span class=&quot;math-container&quot;&gt;$V(b) = 0.75$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;* There are algorithms, such as Dyna-Q, which partially do this, using experience gathered so far to create a model of the environment dynamics. Sometimes this is useful and effective. However, it is not always possible or the best approach.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-10-21T20:09:46.813" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16013" PostTypeId="2" ParentId="15992" CreationDate="2019-10-21T20:57:19.950" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Risks of regulation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you mention in your survey, it is generally understood that the primary concern with regulating AI research is that other parties risk falling behind.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Should we regulate it? Can it be done?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can't really &quot;regulate&quot; technological development in the same way you can regulate some other things in general. Asides from the fact that there is no global governance that can implement this regulation on nations, you can't really regulate someone's research more than you can control how people think: you just need a pen / paper / computer to do any research in math/AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NSA tried to regulate encryption citing national security reasons during a saga known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Crypto_Wars&quot; rel=&quot;nofollow noreferrer&quot;&gt;Crypto Wars&lt;/a&gt;. They failed. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is AI anyways? How will we get there? What will it be like?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Honestly, from the phrasing of your questions in your survey, I get the impression that you don't really understand the hypothetical existential risk due to AI. Personally I don't really buy into their thesis, but in any case, if such a super-intelligent agent emerges, the problem isn't so much &quot;oh no my city is destroyed&quot; or &quot;oh no so many people are killed&quot;, but more so &quot;all of humanity is enslaved without being aware&quot; or &quot;everything is dead&quot;. We think this might happen because we assume AI is all-powerful and we project our own negative qualities onto this unknown agent with unknown power. It's mostly fear really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all speculation, and by definition you cannot predict the behavior of an agent smarter than you, so literally every single comment on this topic is purely unbased speculation. The only thing that is true is that we don't know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another aspect of AI which is dangerous, which more so concerns with how humans use it: i.e. facial recognition, automated weapon systems, automated hacking. These are more pressing issues.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What should we do? We are forced to research AI because no party can afford to fall behind, but at the same time we are pushing ourselves towards a dangerous future: it's a catch-22....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Consensus and current practice suggests that every researcher publicizes our results. Compared to other areas of academia, whose research is often locked behind paywall, ML/AI research is quite publicly accessible. Of course, this doesn't prevent the possibility of a rouge agent....&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2019-10-21T23:17:20.713" LastActivityDate="2019-10-21T23:17:20.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16014" PostTypeId="1" AcceptedAnswerId="16016" CreationDate="2019-10-21T21:11:41.467" Score="2" ViewCount="264" Body="&lt;p&gt;I have read this post:&#xA;&lt;a href=&quot;https://ai.stackexchange.com/q/7088/2444&quot;&gt;How to choose an activation function?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is enough literature about activation functions, but when should I use a linear activation instead of ReLU? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the author mean with &lt;em&gt;ReLU when I'm dealing with positive values, and a linear function when I'm dealing with general values.&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more detail answer to this?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-10-22T00:50:53.437" LastActivityDate="2020-09-25T02:45:28.143" Title="When should I use a linear activation instead of ReLU?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16015" PostTypeId="2" ParentId="15992" CreationDate="2019-10-21T21:21:02.777" Score="0" Body="&lt;p&gt;I think there is a very strong argument for regulating AI.  Chiefly, unintentional (or intentional) bias in statistically driven algorithms, and the idea that responsibility can be offloaded to processes that cannot be meaningfully punished where they transgress.  Additionally, the history of technology, especially since the industrial revolution, strongly validates neo-luddism in the sense that the problems arising from implementation of new technology are not always predictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this sense, there are both ethical reasons to consider regulation, and minimax reasons (here in the sense of erring on the side of caution to minimize the maximum potential downside.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Risk of falling behind&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A risk is that not all participants will hew to the regulations, giving those who don't a significant advantage, but, that, in and of itself, is not a reason to forgo sensible regulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this is not a justification to forgo regulation in that that penalties at least serve as potential deterrent. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Opportunity cost&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Not a risk, but a driver.  The idea of &quot;leaving money on the table&quot; in that not implementing a given technology forgoes greater utility, sacrificing potential benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not invalid, but shouldn't ignore hidden costs.  For instance, the wide-scale deployment of even primitive bots has had a profound social impact.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-22T01:09:04.073" LastActivityDate="2019-10-22T01:09:04.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16016" PostTypeId="2" ParentId="16014" CreationDate="2019-10-21T22:17:20.057" Score="2" Body="&lt;p&gt;The activation function you choose depends on the application you are building/data that you have got to work with. It is hard to recommend one over the other, without taking this into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a short-summary of the advantages and disadvantages of some common activation functions:&#xA;&lt;a href=&quot;https://missinglink.ai/guides/neural-network-concepts/7-types-neural-network-activation-functions-right/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://missinglink.ai/guides/neural-network-concepts/7-types-neural-network-activation-functions-right/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What does the author mean with ReLU when I'm dealing with positive values, and a linear function when I'm dealing with general values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ReLU is good for inputs &gt; 0, since ReLU = 0 if input &amp;lt; 0(which would kill the neuron, if the gradient is = 0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To remedy this, you could look into using a Leaky-ReLU instead.&#xA;(Which avoids killing the neuron by returning a non-zero value in the cases of input &amp;lt;= 0)&lt;/p&gt;&#xA;" OwnerUserId="30565" LastActivityDate="2019-10-21T22:17:20.057" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16017" PostTypeId="1" AcceptedAnswerId="16021" CreationDate="2019-10-22T04:32:46.623" Score="2" ViewCount="90" Body="&lt;p&gt;I'm reading about &lt;a href=&quot;https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;the KL divergence on Wikipedia&lt;/a&gt;. I don't understand how the equation gives &amp;quot;information gained&amp;quot; as it says in the &amp;quot;Interpretations&amp;quot; section&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Expressed in the language of Bayesian inference, &lt;span class=&quot;math-container&quot;&gt;${\displaystyle D_{\text{KL}}(P\parallel Q)}$&lt;/span&gt; is a measure of the &lt;strong&gt;information gained&lt;/strong&gt; by revising one's beliefs from the prior probability distribution &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; to the posterior probability distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I was under the impression that KL divergence is a way of measure the difference between distributions (used in autoencoders to determine the difference between the input and the output generated from latents).&lt;/p&gt;&#xA;&lt;p&gt;How does the equation &lt;span class=&quot;math-container&quot;&gt;$$D_{KL}(P \| Q)=\sum P(x) \log \left( \frac{P(X)}{Q(X)} \right)$$&lt;/span&gt; give us a divergence? Also, in encoding and decoding algorithms that use KL divergence, is the goal to minimize &lt;span class=&quot;math-container&quot;&gt;$D_{KL}(P \| Q)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2020-11-07T15:06:48.767" LastActivityDate="2020-11-07T15:06:48.767" Title="How does the Kullback-Leibler divergence give &quot;knowledge gained&quot;?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;variational-autoencoder&gt;&lt;information-theory&gt;&lt;kl-divergence&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16018" PostTypeId="2" ParentId="15986" CreationDate="2019-10-22T04:50:25.913" Score="4" Body="&lt;p&gt;Hutter's &lt;a href=&quot;https://www.semanticscholar.org/paper/The-Fastest-and-Shortest-Algorithm-for-All-Problems-Hutter/acf58b7dd2b902fa519db8f7cf8af2e3294d56ee&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;fastest and shortest algorithm for all well-defined problems&quot;&lt;/a&gt; is the ultimate &lt;a href=&quot;https://en.wikipedia.org/wiki/Just_in_time_compilation&quot; rel=&quot;nofollow noreferrer&quot;&gt;just-in-time compiler&lt;/a&gt;. It runs a given program and, in parallel, searches for proofs that some other program is equivalent but faster. The running program is restarted at exponentially-spaced intervals; if a faster program has been found, that is started instead. The running time of this algorithm is of the same order as the fastest provably-equivalent algorithm, plus a constant &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; term (the time taken to find the proof, which doesn't dependent on the input size). For example, it will run &lt;a href=&quot;https://en.wikipedia.org/wiki/Bubble_sort&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bubble Sort&lt;/a&gt; in at most &lt;span class=&quot;math-container&quot;&gt;$O(n~log (n))$&lt;/span&gt;) time, by finding a proof that it's equivalent to such a fast algorithm (like &lt;a href=&quot;https://en.wikipedia.org/wiki/Merge_sort&quot; rel=&quot;nofollow noreferrer&quot;&gt;Merge Sort&lt;/a&gt;) then switching to that algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hutter's algorithm is similar to the best &lt;a href=&quot;https://en.wikipedia.org/wiki/Compiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;ahead-of-time compilers&lt;/a&gt;, known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Superoptimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;super-optimisers&lt;/a&gt;. They search through all possible programs, starting with the smallest/fastest, until they find one equivalent to the given code. These are actually in use right now, but are only practical for programs that are a few (machine code) instructions long. The LLVM compiler contains some &quot;peephole optimisations&quot; (i.e. find/replace templates) that were found by a super-optimiser a few years ago. Note that super-optimisation should not be confused with &lt;a href=&quot;https://stackoverflow.com/questions/9067545/what-is-supercompilation&quot;&gt;super-compilation&lt;/a&gt; (a rather general optimisation, which is not optimal and involves no search).&lt;/p&gt;&#xA;" OwnerUserId="30672" LastActivityDate="2019-10-22T04:50:25.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16019" PostTypeId="2" ParentId="15986" CreationDate="2019-10-22T05:36:06.480" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.scholarpedia.org/article/Universal_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Levin's search algorithm&lt;/a&gt; is a general method of function inversion. Many AI tasks are of this sort, e.g. given a cost or reward function (&lt;code&gt;object -&amp;gt; cost&lt;/code&gt; or &lt;code&gt;object -&amp;gt; reward&lt;/code&gt;), its inverse (&lt;code&gt;cost -&amp;gt; object&lt;/code&gt; or &lt;code&gt;reward -&amp;gt; object&lt;/code&gt;) would find an object with the given cost/reward; we could ask this inverse function for an object with low cost or high reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Levin's algorithm is optimal iff the given function is a &quot;black box&quot; with no known pattern in its output. For example, if a small change in the input produces a small change in the output, Levin search wouldn't be optimal; instead we could use hill climbing or some other gradient method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Levin's algorithm looks for the function's inverse by running all possible programs in parallel, assigning exponentially more time to shorter programs. Whenever a program halts, we check whether its output is the desired inverse (i.e. whether &lt;code&gt;givenProgram(outputOfHaltedProgram) = desiredOutput&lt;/code&gt;, e.g. whether &lt;code&gt;cost(outputOfHaltedProgram) = low&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way &quot;simpler&quot; guesses at the inverse are made first; where we define the simplicity (AKA &quot;Levin complexity&quot;) of a value by looking through all programs &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; which generate that value, and minimising the sum of: &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;'s length (in bits) plus the logarithm of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;'s running time (in steps). If we ignored running time we would get &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kolmogorov complexity&lt;/a&gt;, which is theoretically nicer but is incomputable (we don't know when to give up waiting for short non-halting programs, due to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Halting_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Halting Problem&lt;/a&gt;). Levin complexity is computable, since we can give up waiting for those loops once they've taken exponentially-many steps as a longer solution (e.g. once we've spent &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; steps waiting for a possible loop of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, we can start trying programs that are &lt;span class=&quot;math-container&quot;&gt;$N+1$&lt;/span&gt; bits long for &lt;span class=&quot;math-container&quot;&gt;$T/2$&lt;/span&gt; steps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The running time of Levin Search is of the same order as the simplest such inverse-value-generating program. However, this is misleading, since the fraction of steps allocated to running any particular program &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1/2^{complexity(p)}$&lt;/span&gt;, so this constant factor will be slowing down the computation of the inverse too. There is also overhead associated with context-switching between all of these programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://people.idsia.ch/~juergen/toesv2/node28.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAST algorithm&lt;/a&gt; does the same job as Levin Search, in the same time, but avoids the overhead of context-switching between an infinite number parallel programs. Instead it runs one program at a time, cuts it off if it hasn't halted within an appropriate number of steps, then retries for twice as many steps later on. The &lt;a href=&quot;http://people.idsia.ch/~juergen/toesv2/node31.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GUESS algorithm&lt;/a&gt; is also equivalent, but chooses programs at random; the expected runtime is the same, but there's no need to keep track of loop counters like in FAST, plus it can be run on parallel hardware without having to coordinate anything (whilst still avoiding the infinite parallelism of the original).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Levin search is currently impractical in its original setting of searching through general-purpose, Turing-complete programs. It can be useful in less general domains, e.g. searching through the space of hyper-parameters or other domain-specific, configuration-like &quot;programs&quot;.&lt;/p&gt;&#xA;" OwnerUserId="30672" LastEditorUserId="30672" LastEditDate="2019-10-22T14:41:27.890" LastActivityDate="2019-10-22T14:41:27.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16021" PostTypeId="2" ParentId="16017" CreationDate="2019-10-22T08:10:36.950" Score="3" Body="&lt;p&gt;You can know it better, if you know the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Entropy_(information_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;entropy&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Information entropy is the average rate at which information is produced by a stochastic source of data. The information content (also called the surprisal) of an event &lt;span class=&quot;math-container&quot;&gt;${\displaystyle E}$&lt;/span&gt;  is an increasing function of the reciprocal of the &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p(E)}$&lt;/span&gt; of the event, precisely &lt;span class=&quot;math-container&quot;&gt;${\displaystyle I(E)=-\log _{2}(p(E))=\log _{2}(1/p(E))}$&lt;/span&gt;. Shannon defined the entropy Η of a discrete random variable &lt;span class=&quot;math-container&quot;&gt;${\textstyle X}$&lt;/span&gt; with possible values &lt;span class=&quot;math-container&quot;&gt;${\textstyle \left\{x_{1},\ldots ,x_{n}\right\}}$&lt;/span&gt; and probability mass function &lt;span class=&quot;math-container&quot;&gt;${\textstyle \mathrm {P} (X)}$&lt;/span&gt; as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle \mathrm {H} (X)=\operatorname {E} [\operatorname {I} (X)]=\operatorname {E} [-\log(\mathrm {P} (X))].}$$&lt;/span&gt;&#xA;Here &lt;span class=&quot;math-container&quot;&gt;${\displaystyle \operatorname {E} }$&lt;/span&gt; is the expected value operator, and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is the information content of &lt;span class=&quot;math-container&quot;&gt;${\displaystyle I(X)}$&lt;/span&gt; is itself a random variable. The entropy can explicitly be written as&#xA;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle \mathrm {H} (X)=-\sum _{i=1}^{n}{\mathrm {P} (x_{i})\log _{b}\mathrm {P} (x_{i})}}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the base of the logarithm used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now, the KL divergence, try to find the cross entropy of two probability distributions.&lt;/p&gt;&#xA;&lt;p&gt;So, what is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross entropy&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The cross entropy between two probability distributions &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;${\displaystyle q}$&lt;/span&gt; over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set if a coding scheme used for the set is optimized for an estimated probability distribution &lt;span class=&quot;math-container&quot;&gt;${\displaystyle q}$&lt;/span&gt;, rather than the true distribution  &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The cross entropy for the distributions &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p}$&lt;/span&gt; and  &lt;span class=&quot;math-container&quot;&gt;${\displaystyle q}$&lt;/span&gt; over a given set is defined as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle H(p,q)=\operatorname {E} _{p}[-\log q]}.$$&lt;/span&gt;&#xA;The definition may be formulated using the Kullback–Leibler divergence&lt;span class=&quot;math-container&quot;&gt;${\displaystyle D_{\mathrm {KL} }(p\|q)}$&lt;/span&gt; of  &lt;span class=&quot;math-container&quot;&gt;${\displaystyle q}$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p}$&lt;/span&gt; (also known as the relative entropy of &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p}$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;${\displaystyle q}$&lt;/span&gt;).&#xA;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle H(p,q)=H(p)+D_{\mathrm {KL} }(p\|q)},$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;${\displaystyle H(p)}$&lt;/span&gt; is the entropy of &lt;span class=&quot;math-container&quot;&gt;${\displaystyle p}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now, as we want to approximate a distribution function &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; with other distribution function &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, we want to minimize the cross entropy between these two. The first part &lt;span class=&quot;math-container&quot;&gt;$H(p)$&lt;/span&gt; could not be changed as we want to find a distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is given. Hence, we need to minimize &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence of these two, to minimize the cross entropy and better approximation for the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; distribution.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-10-22T08:18:37.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16022" PostTypeId="1" CreationDate="2019-10-22T08:57:44.060" Score="3" ViewCount="117" Body="&lt;p&gt;I'm using a neural network to solve a multi regression problem because I'm trying to predict continuous values. To be more specific, I'm making a tracking algorithm to track the position of an object, I'm trying to predict two values, the latitude and longitude of an object.&lt;/p&gt;&#xA;&lt;p&gt;Now, to calculate the loss of the model, there are some common functions, like mean squared error or mean absolute error, etc., but I'm wondering if I can use some custom function, like &lt;a href=&quot;https://en.wikipedia.org/wiki/Haversine_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, to calculate the distance between the two longitude and latitude values, and then the loss would be the difference between the real distance (calculated from the real longitude and latitude) and the predicted distance (calculated from the predicted longitude and latitude). These are some thoughts from me, so I'm wondering if such an idea would make sense?&lt;/p&gt;&#xA;&lt;p&gt;Would this work in my case better than using the mean squared error as a loss function?&lt;/p&gt;&#xA;&lt;p&gt;I had another question in mind. In my case, I'm predicting two values (longitude and latitude), but is there a way to transform these two target values to only one value so that my neural network can learn better and faster? If yes, which method should I use? Should I calculate the summation of the two and make that as a new target? Does this make sense?&lt;/p&gt;&#xA;" OwnerUserId="30327" LastEditorUserId="2444" LastEditDate="2020-07-18T12:35:10.483" LastActivityDate="2021-04-14T14:10:36.973" Title="When should I create a custom loss function?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;loss-functions&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16023" PostTypeId="1" CreationDate="2019-10-22T10:12:31.823" Score="2" ViewCount="75" Body="&lt;p&gt;I'm studying different stop criteria in genetic algorithms and the advantages and disadvantages of each of them for evaluating different algorithms. One of these methods is the &lt;em&gt;max number of fitness function calls (max NFFC)&lt;/em&gt;, so that we define a value for max NFFC and, if the number of fitness function calls reached this value, the algorithm will stop. Fitness function is called for calculating the fitness of the initial population and whenever a crossover or mutation happens (if parents are chosen as offspring there is no need to compute fitness function).&lt;/p&gt;&#xA;&lt;p&gt;I searched if there is a disadvantage or limitation about using this stop criterion, but I didn't find anything. So, I wanted to know if applying this stop criterion in my algorithm has any disadvantages or there is nothing wrong with using this criterion.&lt;/p&gt;&#xA;" OwnerUserId="30311" LastEditorUserId="2444" LastEditDate="2021-01-30T02:50:10.607" LastActivityDate="2021-03-01T03:03:20.343" Title="Is there any disadvantage of the maximum number of fitness function call as a stop criterion?" Tags="&lt;genetic-algorithms&gt;&lt;fitness-functions&gt;&lt;stopping-conditions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16024" PostTypeId="2" ParentId="16022" CreationDate="2019-10-22T10:14:28.793" Score="0" Body="&lt;p&gt;Using two value and using MSE is probably a better approach. I'd you combine the value to one value, like the case of summation, the network may fits to output 0 on one axis and the value on the other. The method you propose also have the same issue. There are many combination to the real distance, but only one is correct. &#xA;For a neural network to learn faster, one value will not help it learn faster. Instead, accuracy is often increased if the predicted value is a one hot vector of labels instead of a single value. Hope this can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-22T10:14:28.793" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16025" PostTypeId="1" CreationDate="2019-10-22T12:24:55.860" Score="4" ViewCount="87" Body="&lt;p&gt;I'm trying to implement the Reinforce algorithm (Monte Carlo policy gradient) in order to optimize a portfolio of 94 stocks on a daily basis (I have suitable historical data to achieve this). The idea is the following: on each day, the input to a neural network comprises of the following: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;historical daily returns (daily momenta) for previous 20 days for each of the 94 stocks&lt;/li&gt;&#xA;&lt;li&gt;the current vector of portfolio weights (94 weights)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Therefore states are represented by 1974-dimensional vectors. The neural network is supposed to return a 94-dimensional action vector which is again a vector of (ideal) portfolio weights to invest in. Negative weights (short positions) are allowed and portfolio weights should sum to one. Since the action space is continuous I'm trying to tackle it via the Reinforce algorithm. Rewards are given by portfolio daily returns minus trading costs. Here's a code snippet:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Policy(nn.Module):&#xA;    def __init__(self, s_size=1974, h_size=400, a_size=94):&#xA;        super().__init__()&#xA;        self.fc1 = nn.Linear(s_size, h_size)&#xA;        self.fc2 = nn.Linear(h_size, a_size)&#xA;        self.state_size = 1974&#xA;        self.action_size = 94&#xA;    def forward(self, x):&#xA;        x = F.relu(self.fc1(x))&#xA;        x = self.fc2(x)&#xA;        return x&#xA;    def act(self, state):&#xA;        state = torch.from_numpy(state).float().unsqueeze(0).to(device)&#xA;        means = self.forward(state).cpu()&#xA;        m = MultivariateNormal(means,torch.diag(torch.Tensor(np.repeat(1e-8,94))))&#xA;        action = m.sample()&#xA;        action[0] = action[0]/sum(action[0])&#xA;        return action[0], m.log_prob(action)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that in order to ensure that portfolio weights (entries of the action tensor) sum to 1 I'm dividing  by their sum. Also notice that I'm sampling from a multivariate normal distribution with extremely small diagonal terms since I'd like the net to behave as deterministically as possible. (I should probably use something similar to DDPG but I wanted to try out simpler solutions to start with).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training part looks like this:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;optimizer = optim.Adam(policy.parameters(), lr=1e-3)&#xA;&#xA;def reinforce(n_episodes=10000, max_t=10000, gamma=1.0, print_every=1):&#xA;    scores_deque = deque(maxlen=100)&#xA;    scores = []&#xA;    for i_episode in range(1, n_episodes+1):&#xA;        saved_log_probs = []&#xA;        rewards = []&#xA;        state = env.reset()&#xA;        for t in range(max_t):&#xA;            action, log_prob = policy.act(state)&#xA;            saved_log_probs.append(log_prob)&#xA;            state, reward, done, _ = env.step(action.detach().flatten().numpy())&#xA;            rewards.append(reward)&#xA;            if done:&#xA;                break &#xA;        scores_deque.append(sum(rewards))&#xA;        scores.append(sum(rewards))&#xA;&#xA;        discounts = [gamma**i for i in range(len(rewards)+1)]&#xA;        R = sum([a*b for a,b in zip(discounts, rewards)])&#xA;&#xA;        policy_loss = []&#xA;        for log_prob in saved_log_probs:&#xA;            policy_loss.append(-log_prob * R)&#xA;        policy_loss = torch.cat(policy_loss).sum()&#xA;&#xA;        optimizer.zero_grad()&#xA;        policy_loss.backward()&#xA;        optimizer.step()&#xA;&#xA;        if i_episode % print_every == 10:&#xA;            print('Episode {}\tAverage Score: {:.2f}'.format(i_episode, np.mean(scores_deque)))&#xA;            print(scores[-1])&#xA;&#xA;    return scores, scores_deque&#xA;&#xA;scores, scores_deque = reinforce()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, there is no convergence during training even after fiddling with the learning rate so my question is the following: is there anything blatantly wrong with my approach here and if so, how should I tackle this?&lt;/p&gt;&#xA;" OwnerUserId="26195" LastActivityDate="2021-03-15T17:05:05.407" Title="REINFORCE algorithm for portfolio optimization - problem while training" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;convergence&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16026" PostTypeId="1" AcceptedAnswerId="16027" CreationDate="2019-10-22T13:06:59.983" Score="2" ViewCount="59" Body="&lt;p&gt;I read the &lt;a href=&quot;https://arxiv.org/abs/1509.02971&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG paper&lt;/a&gt;, in which the authors state that the actions are fed only later to their Q network:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Actions were not included until the 2nd hidden layer of Q. (Sec 7, Experiment Details)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So does that mean, that the input of the first hidden layer was simply the state and the input of the second hidden layer the output of the first hidden layer concatenated with the actions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why would you do that? To have the first layer focus on learning the state value independent of the selected action? How would that help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this just a small little tweak or a more significant improvement?&lt;/p&gt;&#xA;" OwnerUserId="19928" LastActivityDate="2019-10-22T13:42:13.300" Title="Why feed actions in later layer in Q network?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16027" PostTypeId="2" ParentId="16026" CreationDate="2019-10-22T13:42:13.300" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So does that mean, that the input of the first hidden layer was simply the state and the input of the second hidden layer the output of the first hidden layer concatenated with the actions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why would you do that? To have the first layer focus on learning the state value independent of the selected action? How would that help?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Neural networks hidden layers learn representations that get progressively closer to a linear relationship with the target, layer by layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the first layer would not be learning the state value per se, but some representation of the state that was better related to the action value at the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural network architectures are often established by experimentation, so I expect here they tried the idea and the performance was OK. If they do not give alternative architectures in the paper, then the precise reason is not clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can try a few guesses:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Concatenating the actions with the states in the input layer would result in more parameters for the neural network to achieve the same accuracy, running slightly slower. That's because more weights would be required to link the input layer with the first hidden layer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Separating the layers of state and action inputs is a form of regularisation, as the first layer has to produce features that are useful for all possible actions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You don't want the neural network to construct a reverse map &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s) \rightarrow Q(s,a)$&lt;/span&gt;, you want it to independently assess the action values to do its job as a critic. By having the states and actions presented in different laters, this reduces the chance of the neural network finding shortcuts due to &quot;recognising the policy&quot; to predict the values. That's because the state and action pairings may repeat (so be recognised), but the first layer activations change over time (so even with repeated state/action pairs, this is a new representation to learn from).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is this just a small little tweak or a more significant improvement?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know, and suggest looking through previous work by the same authors in case they describe the approach in more detail.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-10-22T13:42:13.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16028" PostTypeId="2" ParentId="16025" CreationDate="2019-10-22T13:51:05.540" Score="0" Body="&lt;p&gt;One thing you could try to simplify the output logic would be to use a softmax output and then with your outputs set a var to = (max_output - min_output)/2 then treat that number as your long/short &quot;threshold&quot; and this ensures that your ouput always sums to 1 while still allowing the net to learn to output short signals. I would also check to make sure you have 1.0 bias nuerons since I imagine at times (especially first epoch) you are passing in zeros for the portfolio weights. &lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-10-22T13:51:05.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16030" PostTypeId="2" ParentId="15857" CreationDate="2019-10-22T13:58:57.510" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, as per @recessive answer they are indeed independent of the previous layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of back-propagation is to trace the loss(error between target and network output) to specific weights in the network, and then tweak them to minimize this loss.&#xA;For this to be possible, the activation must be independent of previous activation functions(going forward in the network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very helpful to have a good understanding of the back-propagation process when reading these papers, and I personally suggest watching this video to get a good understanding of it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=Ilg3gGewQ5U&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=Ilg3gGewQ5U&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30565" LastActivityDate="2019-10-22T13:58:57.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16031" PostTypeId="2" ParentId="16004" CreationDate="2019-10-22T14:14:34.897" Score="1" Body="&lt;p&gt;This might be more of a signal-processing question, rather than a artificial intelligence question, but I will try my best to be of help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know what the noise you are trying to remove is? How it behaves/where it stems from?&#xA;Or, do you know how your output signal should look, post processing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know these things and you are familiar with MATLAB or any other matrix multiplication software, they come with great prebuilt toolboxes for traditional approaches to remove noise from signals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not exactly sure what patterns you are looking for, I suggest perhaps looking into Autoencoders to discover the hidden patterns. Though it is important to note that the origin of the noise may greatly effect its abilities.&#xA;If you plan on using such a technique it is important that you have a sufficiently large dataset of the signals available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without the clarifications to these questions, along with @nbro's questions, it is hard to be more specific.&lt;/p&gt;&#xA;" OwnerUserId="30565" LastActivityDate="2019-10-22T14:14:34.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16032" PostTypeId="2" ParentId="15972" CreationDate="2019-10-22T14:46:57.117" Score="1" Body="&lt;p&gt;To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A great article on this topic that I recommend as a light mathematical introduction is this:&#xA;&lt;a href=&quot;https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short:&#xA;Bias represents the models effort to generalize samples, as opposed to Variance that represents the models effort to conform to new data.&#xA;A high bias, low variance model will thus look more like a straight(underfitted) line, while a low bias, high variance model will look jagged and all-over the place(overfitted).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, you need to find a balance between the two to avoid both overfitting(high variance, low bias) and underfitting(high bias, low variance) for your specific application.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But how can I determine this for a model such as a Random Forrest classifier?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To determine your models bias and variance configuration(if either is too high/low), you can look at the models performance on the validation and test set.&#xA;The very reason we divide our data into training-validation-test sets, is so that we can validate the models performance when it is presented with samples it has not seen during training.&lt;/p&gt;&#xA;" OwnerUserId="30565" LastActivityDate="2019-10-22T14:46:57.117" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16033" PostTypeId="1" CreationDate="2019-10-22T15:08:58.017" Score="3" ViewCount="404" Body="&lt;p&gt;I've been searching for a tool to convert from the &lt;a href=&quot;https://brat.nlplab.org/standoff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;brat standoff format&lt;/a&gt; to the &lt;a href=&quot;https://universaldependencies.org/format.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CoNLL-U format&lt;/a&gt;, so that to use it as parsing corpus model to the &lt;a href=&quot;https://spacy.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;spaCy&lt;/a&gt; library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me?&lt;/p&gt;&#xA;" OwnerUserId="30702" LastEditorUserId="2444" LastEditDate="2019-10-23T16:12:52.880" LastActivityDate="2020-07-15T00:07:58.513" Title="Is there a tool to convert from the brat standoff format to CoNLL-U format?" Tags="&lt;natural-language-processing&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-20T00:11:11.070" ContentLicense="CC BY-SA 4.0" />
  <row Id="16034" PostTypeId="2" ParentId="16004" CreationDate="2019-10-22T16:15:54.807" Score="0" Body="&lt;p&gt;by svm do you possibly mean singular value decomposition (svd a known noise reduction technique) if this is true then i would say the next method i would try would be wavelet transform for noise reduction and if neither of these techniques are working on there own it is not uncommon to use them together as is done &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1877050915014234&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-10-22T16:15:54.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16035" PostTypeId="2" ParentId="4682" CreationDate="2019-10-22T21:58:13.180" Score="2" Body="&lt;p&gt;Additional features can also cause overfitting if they have low or misleading information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X = [1, 3, 3, 4, 5]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Y = [1, 3, 4, 4, 5]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that the real dataset was generated from the relationship:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Y = X$&lt;/span&gt;, with a probability of 0.2 of adding or subtracting 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reasonable model estimate is &lt;span class=&quot;math-container&quot;&gt;$Y = X$&lt;/span&gt;. &lt;strong&gt;Note&lt;/strong&gt; that &lt;strong&gt;no model can fit this data perfectly&lt;/strong&gt;, because the two 3 inputs map to different outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose we add a new feature: a random number between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;: &#xA;&lt;span class=&quot;math-container&quot;&gt;$W = [1 ,5, 2, 6, 3]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may not be obvious, but a sufficiently deep and broad neural network can learn a new function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(W) = 1$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$W = 2,4,7,8,9,$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(W) = 0$&lt;/span&gt; otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and define a new prediction: &lt;span class=&quot;math-container&quot;&gt;$Y = X - g(W)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This happens to produce a &lt;em&gt;perfect&lt;/em&gt; fit on the training data. However, it will perform extremely poorly on new data (like a test set), because it has learned a meaningless pattern out of random noise. Coincidentally, it will be wrong on about 50% of samples, while our first model will be wrong on only 20% of samples.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="9947" LastEditDate="2020-03-21T00:53:18.397" LastActivityDate="2020-03-21T00:53:18.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16036" PostTypeId="1" CreationDate="2019-10-23T07:26:04.950" Score="3" ViewCount="1301" Body="&lt;p&gt;Is it possible to calculate the best possible placements for settlements in &lt;a href=&quot;https://en.wikipedia.org/wiki/Catan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Catan&lt;/a&gt; without using an ML algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it is trivial to simply add up the numbers surrounding the settlement (highest point location), I'm looking to build a deeper analysis of the settlement locations. For example, if the highest point location is around a sheep-sheep-sheep, it might be better to go to a lower point location for better resource access. It could also weight for complementary resources, blocking other players from resources, and being closer to ports.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems feasible to program arithmetically, yet some friends said this is an ML problem. If it is ML, how would one go about training, as the gameboard changes every game?&lt;/p&gt;&#xA;" OwnerUserId="30720" LastEditorUserId="1671" LastEditDate="2019-10-25T23:53:23.837" LastActivityDate="2019-10-26T19:49:28.217" Title="How to calculate the optimal placements for settlements in Catan without an ML algorithm?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;game-theory&gt;&lt;combinatorial-games&gt;&lt;symbolic-ai&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16037" PostTypeId="1" AcceptedAnswerId="16040" CreationDate="2019-10-23T13:04:51.670" Score="5" ViewCount="507" Body="&lt;p&gt;(I apologize for the title being too broad and the question being not 'technical')&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that my task is to label news articles. This means that given a news article, I am supposed to classify which category that news belong to. Eg, 'Ronaldo scores a fantastic goal' should classify under 'Sports'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After much experimentation, I came up with a model that does this labeling for me. It has, say, 50% validation accuracy. (Assume that it is the best)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so I deployed this model for my task (on unseen data obviously). Of course, from a probabilistic perspective, I should get roughly 50% of the articles labelled correctly. But how do I know that which labels are actually correct and which labels need to be corrected? If I were to manually check (say, by hiring people to do so), how is deploying such a model better than just hiring people to do the classification directly? (Do not forget that the manpower cost of developing the model could have been saved.)&lt;/p&gt;&#xA;" OwnerUserId="30729" LastEditorUserId="1671" LastEditDate="2019-10-24T00:47:16.813" LastActivityDate="2019-10-24T00:47:16.813" Title="How does text classification reduce manpower costs?" Tags="&lt;classification&gt;&lt;utility&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16038" PostTypeId="1" AcceptedAnswerId="16041" CreationDate="2019-10-23T14:40:33.857" Score="5" ViewCount="132" Body="&lt;p&gt;I very often applied a grid search to tune the parameters of my supervised model. I have the feeling that parameter tuning will eventually (very often) lead to overfitting? Is this crazy to say?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way that we can apply grid search in such a way that it will not overfit?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-10-23T15:24:27.237" LastActivityDate="2019-10-23T15:24:27.237" Title="How can I avoid overfitting when doing parameter tuning?" Tags="&lt;machine-learning&gt;&lt;overfitting&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16039" PostTypeId="2" ParentId="16037" CreationDate="2019-10-23T14:41:43.887" Score="0" Body="&lt;p&gt;First of all to be more real, you usually expect more than 50% validation accuracy on articles predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back on your question, you should definitely try to automate this process if you are looking for a long-term solution of labeling articles. Deploying such model should not cost more than hiring employees to do this manually, at least for a long-term perspective.&lt;/p&gt;&#xA;" OwnerUserId="28233" LastActivityDate="2019-10-23T14:41:43.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16040" PostTypeId="2" ParentId="16037" CreationDate="2019-10-23T14:46:56.503" Score="10" Body="&lt;p&gt;There are several advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Some text classification systems are &lt;em&gt;much&lt;/em&gt; more accurate than 50%. For example, most spam classification systems are 99.9% accurate, or more. There will be little value to having employees review these labels.&lt;/li&gt;&#xA;&lt;li&gt;Many text classification systems can output a &lt;em&gt;confidence&lt;/em&gt; as well as a label. You can selectively have employees review only the examples the model is not confident about. Often these will be small in number.&lt;/li&gt;&#xA;&lt;li&gt;You can usually &lt;em&gt;test&lt;/em&gt; a text classification model by having it classify some unseen data, and then asking people to check the work. If you do this for a small number of examples, you can make sure the system is working. You can then confidently use the system on a much large set of unlabeled examples, and be reasonably sure about how accurate it is. &lt;/li&gt;&#xA;&lt;li&gt;For text, it is also important to measure how much different &lt;em&gt;people&lt;/em&gt; agree on the ratings. You are unlikely to do better than this, because this gives you a notion of the subjectivity of the specific problem you are working on. If people disagree 50% of the time anyway, maybe you can accept a 50% failure rate from the automated system, and not bother checking its work.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-23T17:25:19.900" LastActivityDate="2019-10-23T17:25:19.900" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16041" PostTypeId="2" ParentId="16038" CreationDate="2019-10-23T15:09:05.503" Score="4" Body="&lt;p&gt;Yes. Usually you would use &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-validation_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross validation&lt;/a&gt; to avoid overfitting during parameter tuning. If your dataset is large enough, and you don't try &lt;em&gt;too&lt;/em&gt; many parameter combinations, this will work well, because to &quot;get lucky&quot; and overfit, a parameter combination will need to work very well on &lt;em&gt;many&lt;/em&gt; variations of the problem, which is less likely than working well on just one set of data.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-23T15:09:05.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16042" PostTypeId="1" AcceptedAnswerId="16044" CreationDate="2019-10-23T15:26:13.847" Score="1" ViewCount="18" Body="&lt;p&gt;Based on the answer of my previous question:&#xA;&lt;a href=&quot;https://ai.stackexchange.com/questions/16038/how-can-i-avoid-overfitting-when-doing-parameter-tuning/16041#16041&quot;&gt;How can I avoid overfitting when doing parameter tuning?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we say: the more we increase the numbers K of cross validation the less likely it is that we overfit?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastActivityDate="2019-10-23T15:40:05.737" Title="Can we say: the more we increase the numbers of cross validation the less likely it is that we overfit?" Tags="&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16043" PostTypeId="2" ParentId="16036" CreationDate="2019-10-23T15:27:15.190" Score="3" Body="&lt;p&gt;Catan is actually a much more complicated game than the simple rules would suggest, and an &lt;em&gt;exact&lt;/em&gt; solution is probably beyond the scope of current AI techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo Tree Search or Expectiminimax techniques &lt;em&gt;seem&lt;/em&gt; like they could help, but are intended for games of perfect information. Catan is not a game of perfect information (the development cards are hidden), and also has a phase that occurs &lt;em&gt;without&lt;/em&gt; a regular turn sequence (trading). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve Catan properly, I think you're going to need both algorithms for solving POMDPs (like CFR+), &lt;em&gt;and&lt;/em&gt; algorithms for negotiation (like &lt;a href=&quot;http://u.cs.biu.ac.il/~sarit/data/articles/An%20Automated%20Diplomacy%20Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kraus' Diplomat&lt;/a&gt;). I'm not certain that these have been combined before in formal analysis, so this might actually be a good PhD thesis for someone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you can probably get a good player using self-play techniques, because Catan has randomization, and a relatively small set of moves, like &lt;a href=&quot;https://en.wikipedia.org/wiki/TD-Gammon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Backgammon&lt;/a&gt;. These may or may not offer simple rules about how-best to play the game. Your friends are right to think about this as, at root, an ML problem. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-23T15:27:15.190" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16044" PostTypeId="2" ParentId="16042" CreationDate="2019-10-23T15:40:05.737" Score="1" Body="&lt;p&gt;In general, no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a tradeoff between making the &lt;em&gt;validation set&lt;/em&gt; for each fold smaller, and having more folds in total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, if you have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; folds for &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; datapoints, each fold will have only a single datapoint in its validation set. The validation accuracy of a model on a single datapoint is not a reliable estimator for the test performance of the model. In fact, you can construct examples where the error is arbitrarily large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, people sometimes use &lt;a href=&quot;https://datascience.stackexchange.com/questions/32264/what-is-the-difference-between-bootstrapping-and-cross-validation&quot;&gt;Bootstrap Validation&lt;/a&gt; if they need a very large number of folds. In practice though, most people just us 10 folds, and that's &quot;good enough&quot;. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-23T15:40:05.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16045" PostTypeId="1" AcceptedAnswerId="16127" CreationDate="2019-10-23T16:21:21.960" Score="1" ViewCount="712" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hKGIu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hKGIu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got this slide from CMU's lecture notes. The &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;s on the right are inputs and the &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;s are weights that get multiplied together then summed up at each hidden layer node. So I'm assuming this is a node in the hidden layer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the mathematical reason for taking the sum of the weights and inputs and inputting that into a sigmoid function? Is there something the sigmoid function provides mathematically or provides some sort of intuition useful for the next layer?&lt;/p&gt;&#xA;" OwnerUserId="25721" LastEditorUserId="2444" LastEditDate="2019-10-31T20:05:06.477" LastActivityDate="2019-10-31T20:05:06.477" Title="Why is there a sigmoid function in the hidden layer of a neural network?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;hidden-layers&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2019-10-31T19:32:22.097" ContentLicense="CC BY-SA 4.0" />
  <row Id="16046" PostTypeId="1" CreationDate="2019-10-23T22:09:31.160" Score="1" ViewCount="442" Body="&lt;p&gt;I want to design a neural network that can be used for predicting sports scores for betting, specifically for American football.  What I’d like to do is create a kind of profile for each game based on the specific strengths and weaknesses of each team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let’s say two teams have the following characteristics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Team A:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Passing Offense Rating: 5&lt;/li&gt;&#xA;&lt;li&gt;Rushing Offense Rating: 2&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Team B:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Passing Defense Rating: 3&lt;/li&gt;&#xA;&lt;li&gt;Rushing Defense Rating: 4&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I’d like to be able to search for historical games where two teams have similar profiles.  I could perhaps then narrow it down to games with profiles that have statistically significant historical outcomes (i.e., certain types of matchups are likely to result in similar results).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality, I’d have dozens of team characteristics to compare.  I would then need to assign weights of importance to each characteristic, which could be used to further ensure effective selection of similar games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;em&gt;think&lt;/em&gt; I could do this like a convolutional neural network where there is an additional filter applied to the characteristics for the weights.  Are there any other ways that are specifically applicable to this strategy?&lt;/p&gt;&#xA;" OwnerUserId="30154" LastActivityDate="2019-10-23T23:56:21.640" Title="Neural networks for sports betting" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;pattern-recognition&gt;&lt;similarity&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16047" PostTypeId="2" ParentId="16046" CreationDate="2019-10-23T23:56:21.640" Score="1" Body="&lt;p&gt;This is probably not going to work well as a way to make money. &lt;a href=&quot;https://www.bloomberg.com/news/features/2018-05-03/the-gambler-who-cracked-the-horse-racing-code&quot; rel=&quot;nofollow noreferrer&quot;&gt;People&lt;/a&gt; with far larger budgets, and far more training, are already milking out any money to be made this way. This is probably their day job, and they are good at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, here are some ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You do not need or want to use a convolutional network for this. Convolutional networks are useful when you want to detect a pattern invariant to translation in complex inputs. The only translation you have is that the teams could appear in either order. Just input them in both possible orders if this is a concern.&lt;/li&gt;&#xA;&lt;li&gt;You want to find similar games. You do not need or (generally) want to use a neural network to do this. As a starting point, normalize the features and compute the &lt;a href=&quot;https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;nearest neighbors&lt;/a&gt; of an input point directly. Algorithms for doing this are fast and will return all similar points. You can even (with small modifications) output the degree of similarity as a numeric value.&lt;/li&gt;&#xA;&lt;li&gt;If you want to predict who will win, you will probably have much better luck doing that directly. Build a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multilayer_perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;feed-forward neural network&lt;/a&gt; that predicts which team won from the features you gathered. This is a routine classification task, and the resulting model will probably work better than finding similar games and then trying to manually determine what to do.&lt;/li&gt;&#xA;&lt;li&gt;If you care about scores instead of just who won, use a feed forward network for regression instead of classification. The task is almost identical.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-23T23:56:21.640" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="16048" PostTypeId="5" CreationDate="2019-10-24T00:51:25.950" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Utility#Utility_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Utility#Utility_function&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Utility#Expected_utility&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Utility#Expected_utility&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Utility_maximization_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Utility_maximization_problem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://plato.stanford.edu/entries/decision-theory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/decision-theory/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-24T00:51:25.950" LastActivityDate="2019-10-24T00:51:25.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16049" PostTypeId="4" CreationDate="2019-10-24T00:51:25.950" Score="0" Body="For questions about the economic and game theoretic concepts of utility, including utility functions and expected utility." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-24T00:51:25.950" LastActivityDate="2019-10-24T00:51:25.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16050" PostTypeId="2" ParentId="2817" CreationDate="2019-10-24T02:21:48.703" Score="2" Body="&lt;p&gt;I wrote some python code to reproduce this paper's purported results. My code very efficiently optimizes simple smooth functions like bowls, but does not come close to reproducing the paper's claimed results on more complex functions, including with the parameters the authors report. I think that, since both @Jairo and I were unable to reproduce the results from the information in the paper, independently, it is likely that something is wrong with the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It &lt;em&gt;may&lt;/em&gt; be possible to reproduce the paper's claimed behaviors using a library like &lt;a href=&quot;https://pyswarms.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;py_swarm&lt;/a&gt;, however the paper is using a Firefly Algorithm (already a rarer form of a PSO), and is using a home-brew variant of that algorithm, so my guess is that rolling your own is required, as we both did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I find the paper's claims fairly unlikely. For example, in section 4.1, they claim to converge on the global minimum of a function with many local minima in just 10 iterations. It seems to me like most of the time the fireflies will quickly converge on the bottom of one of the gullies in this function, and get stuck there. This is also what I observe in my replications. I suspect the authors may have cherry-picked their results from the best runs without reporting this, or have omitted some key detail from the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my replication code, in case someone else wants to try to reproduce this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import sin, pi, exp, sqrt&#xA;from random import random&#xA;from copy import deepcopy&#xA;&#xA;def michalewiz_objective(x):&#xA;    result = 0&#xA;    for i, x_i in enumerate(x):&#xA;        result -= sin(x_i)*(sin(i*(x_i**2)/pi))**20&#xA;    return -result&#xA;&#xA;def bowl_objective(x):&#xA;    return -sum([x_i**2 for x_i in x])&#xA;&#xA;pop_size = 40&#xA;max_generations = 10&#xA;alpha = 0.2&#xA;gamma = 1&#xA;beta_0 = 1&#xA;d = 2&#xA;I = michalewiz_objective&#xA;#I = bowl_objective&#xA;&#xA;def move(firefly, other_firefly):&#xA;    radius = sqrt(sum([(firefly[i] - other_firefly[i])**2 for i in range(0, len(firefly))]))&#xA;    for i, value in enumerate(firefly):&#xA;        firefly[i] += beta_0*exp(-gamma*radius**2)*(other_firefly[i] - firefly[i])&#xA;        firefly[i] += alpha * (random() - 0.5)&#xA;&#xA;# Using 4*random() to match Figure 3's apparent spread.&#xA;fireflies = [[4*random() for i in range(0, d)] for j in range(0, pop_size)]&#xA;&#xA;for generation in range(0, max_generations):&#xA;    new_fireflies = [deepcopy(firefly) for firefly in fireflies]&#xA;    for index, firefly in enumerate(fireflies):&#xA;        for other_firefly in fireflies:&#xA;            if I(other_firefly) &amp;gt; I(firefly):&#xA;                move(new_fireflies[index], other_firefly)&#xA;    fireflies = new_fireflies&#xA;    best = max([I(f) for f in fireflies])&#xA;    mean = [sum([f[0] for f in fireflies]), sum([f[1] for f in fireflies])]&#xA;    print(best)&#xA;    print(mean)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-23T04:31:35.337" LastActivityDate="2019-11-23T04:31:35.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16051" PostTypeId="1" CreationDate="2019-10-24T02:54:05.587" Score="1" ViewCount="87" Body="&lt;p&gt;I am trying to implement NEAT for the snake game. My game logic is ready, which is working properly and NEAT configured. But even after 100 generations with 200 genomes per generation, the snakes perform very poorly. It barely ever eats more than 2 food. Below is the snip of the eval_genome function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def eval_genome(genomes, config):&#xA;    clock = pygame.time.Clock()&#xA;    win = pygame.display.set_mode((WIN_WIDTH, WIN_HEIGHT))&#xA;    for genome_id, g in genomes:&#xA;        net = neat.nn.FeedForwardNetwork.create(g, config)&#xA;        g.fitness = 0&#xA;        snake = Snake()&#xA;        food = Food(snake.body)&#xA;        run = True&#xA;        UP = DOWN = RIGHT = LEFT = MOVE_SNAKE = False&#xA;        moveToFood = 0&#xA;        score = 0&#xA;        moveCount = 0&#xA;        while run:&#xA;            pygame.time.delay(50)&#xA;            clock.tick_busy_loop(10)&#xA;            for event in pygame.event.get():&#xA;                if event.type == pygame.QUIT:&#xA;                    run = False&#xA;            snakeHeadX = snake.body[0]['x']&#xA;            snakeHeadY = snake.body[0]['y']&#xA;            snakeTailX = snake.body[len(snake.body)-1]['x']&#xA;            snakeTailY = snake.body[len(snake.body)-1]['y']&#xA;            snakeLength = len(snake.body)&#xA;            snakeHeadBottomDist = WIN_HEIGHT - snakeHeadY - STEP&#xA;            snakeHeadRightDist = WIN_WIDTH - snakeHeadX - STEP&#xA;            foodBottomDist = WIN_HEIGHT - food.y - STEP&#xA;            foodRightDist = WIN_WIDTH - food.x - STEP&#xA;            snakeFoodDistEuclidean = math.sqrt((snakeHeadX - food.x)**2 + (snakeHeadY - food.y)**2)&#xA;            snakeFoodDistManhattan = abs(snakeHeadX - food.x) + abs(snakeHeadY - food.y)&#xA;            viewDirections = snake.checkDirections(food, UP, DOWN, LEFT, RIGHT)&#xA;            deltaFoodDist = snakeFoodDistEuclidean&#xA;&#xA;            outputs = net.activate((snakeHeadX, snakeHeadY, snakeHeadBottomDist, snakeHeadRightDist, snakeTailX, snakeTailY, snakeLength, moveCount, moveToFood, food.x, food.y, foodBottomDist, foodRightDist, snakeFoodDistEuclidean, snakeFoodDistManhattan, viewDirections[0], viewDirections[1], viewDirections[2], viewDirections[3], viewDirections[4], viewDirections[5], viewDirections[6], viewDirections[7], deltaFoodDist))&#xA;&#xA;            if (outputs[0] == max(outputs) and not DOWN):&#xA;                snake.setDir(0,-1)&#xA;                UP = True&#xA;                LEFT = False&#xA;                RIGHT = False&#xA;                MOVE_SNAKE = True&#xA;            elif (outputs[1] == max(outputs) and not UP):&#xA;                snake.setDir(0,1)&#xA;                DOWN = True&#xA;                LEFT = False&#xA;                RIGHT = False&#xA;                MOVE_SNAKE = True&#xA;            elif (outputs[2] == max(outputs) and not RIGHT):&#xA;                snake.setDir(-1,0)&#xA;                LEFT = True&#xA;                UP = False&#xA;                DOWN = False&#xA;                MOVE_SNAKE = True&#xA;            elif (outputs[3] == max(outputs) and not LEFT):&#xA;                snake.setDir(1,0)&#xA;                RIGHT = True&#xA;                UP = False&#xA;                DOWN = False&#xA;                MOVE_SNAKE = True&#xA;            elif (not MOVE_SNAKE):&#xA;                if (outputs[0] == max(outputs)):&#xA;                    snake.setDir(0,-1)&#xA;                    UP = True&#xA;                    MOVE_SNAKE = True&#xA;                elif (outputs[1] == max(outputs)):&#xA;                    snake.setDir(0,1)&#xA;                    DOWN = True&#xA;                    MOVE_SNAKE = True&#xA;                elif (outputs[2] == max(outputs)):&#xA;                    snake.setDir(-1,0)&#xA;                    LEFT = True&#xA;                    MOVE_SNAKE = True&#xA;                elif (outputs[3] == max(outputs)):&#xA;                    snake.setDir(1,0)&#xA;                    RIGHT = True&#xA;                    MOVE_SNAKE = True  &#xA;&#xA;            win.fill((0, 0, 0))&#xA;            food.showFood(win)&#xA;            if(MOVE_SNAKE):&#xA;                snake.update()&#xA;                newSnakeHeadX = snake.body[0]['x']&#xA;                newSnakeHeadY = snake.body[0]['y']&#xA;                newFoodDist = math.sqrt((newSnakeHeadX - food.x)**2 + (newSnakeHeadY - food.y)**2)&#xA;                deltaFoodDist = newFoodDist - snakeFoodDistEuclidean&#xA;                moveCount += 1&#xA;                if (newFoodDist &amp;lt;= snakeFoodDistEuclidean):&#xA;                    g.fitness += 1&#xA;                else:&#xA;                    g.fitness -= 10&#xA;            snake.show(win)&#xA;            if(snake.collision()):&#xA;                if score != 0:&#xA;                    print('FINAL SCORE IS: '+ str(score))&#xA;                g.fitness -= 50&#xA;                break&#xA;&#xA;            if(snake.eat(food,win)):&#xA;                g.fitness += 15&#xA;                score += 1&#xA;                if score == 1 :&#xA;                    moveToFood = moveCount&#xA;                    # foodEatenMove = pygame.time.get_ticks()/1000&#xA;                else:&#xA;                    moveToFood = moveCount - moveToFood&#xA;                food.foodLocation(snake.body)&#xA;                food.showFood(win)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Additionally, I am putting the definition of the checkDirections function. What it does is returns an array of size 8 corresponding to 8 directions where each value can be either 0 (not food or body), 1(food found but no body), 3(body found but no food), or 4(both body and food found).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def checkDirections(self, food, up, down, left, right):&#xA;        '''&#xA;        x+STEP, y-STEP&#xA;        x+STEP, y+STEP&#xA;        x-STEP, y-STEP&#xA;        x-STEP, y+STEP&#xA;        x+STEP, y&#xA;        x, y-STEP&#xA;        x, y+STEP&#xA;        x-STEP, y&#xA;        '''&#xA;        view = []&#xA;        x = self.xdir&#xA;        y = self.ydir&#xA;&#xA;        view.append(self.check(x, y, STEP, -STEP, food.x, food.y))&#xA;        view.append(self.check(x, y, STEP, STEP, food.x, food.y))&#xA;        view.append(self.check(x, y, -STEP, -STEP, food.x, food.y))&#xA;        view.append(self.check(x, y, -STEP, STEP, food.x, food.y))&#xA;        view.append(self.check(x, y, STEP, 0, food.x, food.y))&#xA;        view.append(self.check(x, y, 0, -STEP, food.x, food.y))&#xA;        view.append(self.check(x, y, 0, STEP, food.x, food.y))&#xA;        view.append(self.check(x, y, -STEP, 0, food.x, food.y))&#xA;&#xA;        if up == True:&#xA;            view[6] = -999&#xA;        elif down == True:&#xA;            view[5] = -999&#xA;        elif left == True:&#xA;            view[4] == -999&#xA;        elif right == True:&#xA;            view[7] == -999        &#xA;        return view&#xA;&#xA;    def check(self, x, y, xIncrement, yIncrement, foodX, foodY):&#xA;        value = 0&#xA;        foodFound = False&#xA;        bodyFound = False&#xA;        while (x &amp;gt;= 0 and x &amp;lt;= WIN_WIDTH and y &amp;gt;= 0 and y &amp;lt;= WIN_HEIGHT):&#xA;            x += xIncrement&#xA;            y += yIncrement&#xA;            if (not foodFound):&#xA;                if (foodX == x and foodY == y):&#xA;                    foodFound = True&#xA;            if (not bodyFound):&#xA;                for i in range(1, len(self.body)):&#xA;                    if ((x == self.body[i]['x']) and (y == self.body[i]['y'])):&#xA;                        bodyFound = True&#xA;            if (not bodyFound and not foodFound):&#xA;                value = 0&#xA;            elif (not bodyFound and foodFound):&#xA;                value = 1&#xA;            elif (bodyFound and not foodFound):&#xA;                value = 2&#xA;            else:&#xA;                value = 3&#xA;        return value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using sigmoid as the activation function. Although I have tried with tanh and relu as well with no luck. Below is the NEAT config file that I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[NEAT]&#xA;fitness_criterion     = max&#xA;fitness_threshold     = 10000&#xA;pop_size              = 200&#xA;reset_on_extinction   = False&#xA;&#xA;[DefaultGenome]&#xA;# node activation options&#xA;activation_default      = sigmoid&#xA;activation_mutate_rate  = 0.0&#xA;activation_options      = sigmoid&#xA;&#xA;# node aggregation options&#xA;aggregation_default     = sum&#xA;aggregation_mutate_rate = 0.0&#xA;aggregation_options     = sum&#xA;&#xA;# node bias options&#xA;bias_init_mean          = 0.0&#xA;bias_init_stdev         = 1.0&#xA;# was 30 max and -30 for min bias&#xA;bias_max_value          = 100.0&#xA;bias_min_value          = -100.0&#xA;bias_mutate_power       = 0.5&#xA;bias_mutate_rate        = 0.7&#xA;bias_replace_rate       = 0.3&#xA;&#xA;# genome compatibility options&#xA;compatibility_disjoint_coefficient = 1.0&#xA;compatibility_weight_coefficient   = 0.5&#xA;&#xA;# connection add/remove rates&#xA;conn_add_prob           = 0.8&#xA;conn_delete_prob        = 0.56&#xA;&#xA;# connection enable options&#xA;enabled_default         = True&#xA;# below was 0.01&#xA;enabled_mutate_rate     = 0.3&#xA;&#xA;feed_forward            = True&#xA;initial_connection      = full&#xA;&#xA;# node add/remove rates&#xA;node_add_prob           = 0.7&#xA;node_delete_prob        = 0.4&#xA;&#xA;# network parameters&#xA;num_hidden              = 0&#xA;num_inputs              = 24&#xA;num_outputs             = 4&#xA;&#xA;# node response options&#xA;response_init_mean      = 1.0&#xA;response_init_stdev     = 0.0&#xA;response_max_value      = 30.0&#xA;response_min_value      = -30.0&#xA;response_mutate_power   = 0.0&#xA;response_mutate_rate    = 0.0&#xA;response_replace_rate   = 0.0&#xA;&#xA;# connection weight options&#xA;weight_init_mean        = 0.0&#xA;weight_init_stdev       = 1.0&#xA;weight_max_value        = 30&#xA;weight_min_value        = -30&#xA;weight_mutate_power     = 0.5&#xA;weight_mutate_rate      = 0.8&#xA;weight_replace_rate     = 0.1&#xA;&#xA;[DefaultSpeciesSet]&#xA;compatibility_threshold = 3.0&#xA;&#xA;[DefaultStagnation]&#xA;species_fitness_func = max&#xA;max_stagnation       = 20&#xA;species_elitism      = 2&#xA;&#xA;[DefaultReproduction]&#xA;elitism            = 2&#xA;survival_threshold = 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If anyone has any insights or thoughts that could help improve the performance of the snake AI, please let me know.&lt;/p&gt;&#xA;" OwnerUserId="30739" LastActivityDate="2021-04-29T06:05:45.650" Title="Unable to achieve expected outputs using NEAT for the snake game" Tags="&lt;neural-networks&gt;&lt;game-ai&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16052" PostTypeId="1" AcceptedAnswerId="16065" CreationDate="2019-10-24T03:30:05.840" Score="3" ViewCount="192" Body="&lt;p&gt;Can alpha-beta pruning/ minimax be used for systems apart from games? Like for selecting the right customer for a product, etc. (the typical data science problems)? I have seen people do it, but can't understand how. Can someone help me understand that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I do something like if - find two criteria on which customers can buy product depends on like gender and age. Find the probability for all the customers depending on age and gender if they can buy it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;like if there are 3 customers - there probability to buy a product on the basis of their age and gender is - Customer 1 - (20%, 30%), Customer 2 - (30%, 60%), Customer 3 - (40%, 20%). here the x and y represents - (probability based on age, probability based on gender ). Probability is probability to buy the product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For minimax, will it be correct if one player(max) tries to select the customer on basis of gender and other player(min) on basis of age. so, one can be max and one can be min. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dont know if this correct or not, but just a idea.&lt;/p&gt;&#xA;" OwnerUserId="30749" LastEditorUserId="30749" LastEditDate="2019-10-24T17:03:06.483" LastActivityDate="2019-10-24T19:40:21.443" Title="Can alpha-beta pruning be used for applications apart from games?" Tags="&lt;applications&gt;&lt;data-science&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16053" PostTypeId="2" ParentId="16045" CreationDate="2019-10-24T04:38:01.610" Score="2" Body="&lt;p&gt;By itself, I'm not sure it's possible to know. It's possible the slides were old. Or, the intended purpose was to mention how as sigmoid ranges from 0 to 1. Mostly, it looks like it was intended to bring up gradient descent. But it could also be an entry point to the discussion of other methods such as ReLU. Either that or perhaps some sort of norming function.&lt;/p&gt;&#xA;" OwnerUserId="30750" LastActivityDate="2019-10-24T04:38:01.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16054" PostTypeId="1" CreationDate="2019-10-24T08:09:40.940" Score="2" ViewCount="137" Body="&lt;p&gt;What are the actual risks to society associated with the widespread use of AI? Outside of the use of AI in a military context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not talking about accidental risks or unintentional behaviour - eg, a driver-less car accidentally crashing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I am not talking about any transitional effects when we see the use of AI being widespread and popular.  For instance I have heard that the widespread use of AI will make many existing jobs redundant, putting many people out of work.  However this is true of any major leap forward in technology (for example the motor car killed off the stable/farrier industries).  The leaps forward in technology almost always end up creating more jobs than were lost in the long run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in long term risks and adverse effects stemming directly from the widespread use of AI in a non-military sense.  Has anybody speculated on the social or psychological impacts that AI will produce once it has become popular?&lt;/p&gt;&#xA;" OwnerUserId="30526" LastEditorUserId="1671" LastEditDate="2019-10-26T20:10:25.897" LastActivityDate="2019-10-26T20:40:12.457" Title="What are the societal risks associated with AI?" Tags="&lt;social&gt;&lt;neo-luddism&gt;&lt;risk-management&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16055" PostTypeId="2" ParentId="16054" CreationDate="2019-10-24T08:36:45.450" Score="4" Body="&lt;p&gt;The biggest risk is algorithmic bias. As more and more decision-making processes are taken on by AI systems, there will be an abdication of responsibility to the computer; people in charge will simply claim the computer did it, and they cannot change it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real problem is that training data for machine learning often contains bias, which is usually ignored or not recognised. There was a story on BBC Radio about someone whose passport photo was rejected by an algorithm because he supposedly had his mouth open. However, he belonged to an ethnic group which has larger lips than Caucasian whites, but the machine could not cope with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a whole raft of examples where similar things happen: if you belong to a minority group, machine learning can lead to you being excluded, just because the algorithms will have been trained on training data that was too restricted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: Here is a link to &lt;a href=&quot;https://www.bbc.co.uk/news/technology-49993647&quot; rel=&quot;nofollow noreferrer&quot;&gt;a BBC News story about the example&lt;/a&gt; I mentioned.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-10-24T08:59:25.807" LastActivityDate="2019-10-24T08:59:25.807" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16056" PostTypeId="1" AcceptedAnswerId="16059" CreationDate="2019-10-24T09:30:49.650" Score="2" ViewCount="108" Body="&lt;p&gt;Is it possible for value-based methods to learn stochastic policies? I'm trying to get a clear picture of the different categories for RL algorithms, and while doing so I started to think about settings where the optimal policy is stochastic (POMDP), and if it is possible to learn this policy for the &amp;quot;traditional&amp;quot; value-based methods&lt;/p&gt;&#xA;&lt;p&gt;If it is possible, what are the most common methods for doing this?&lt;/p&gt;&#xA;" OwnerUserId="30565" LastEditorUserId="2444" LastEditDate="2020-11-20T02:05:10.690" LastActivityDate="2020-11-20T02:05:10.690" Title="Is it possible for value-based methods to learn stochastic policies?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;pomdp&gt;&lt;stochastic-policy&gt;&lt;value-based-methods&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16057" PostTypeId="1" CreationDate="2019-10-24T09:37:08.823" Score="2" ViewCount="188" Body="&lt;p&gt;I've been reading different papers which implements the &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformer&lt;/a&gt; for time series forecasting. Most of the them are claiming that the training time is significantly faster then using a normal RNN. From my understanding when training such a model, you can encode the input in parallel, but the decoding is still sequential unless you're using teacher-forcing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes the transformer faster than RNN in such a setting? Is there something that I am missing?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2019-11-01T03:05:04.407" LastActivityDate="2019-11-01T03:05:04.407" Title="Why is the transformer for time series forecasting faster than RNN?" Tags="&lt;prediction&gt;&lt;sequence-modelling&gt;&lt;attention&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16058" PostTypeId="1" AcceptedAnswerId="16060" CreationDate="2019-10-24T10:17:46.843" Score="1" ViewCount="83" Body="&lt;p&gt;In his original &lt;a href=&quot;https://arxiv.org/pdf/1406.2661v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN paper&lt;/a&gt; Goodfellow gives a game theoretic perspective for GANs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;\underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right) = &#xA;\mathbb{E}_{x\sim\mathit{p}_{\textrm{data}}\left(x \right)} \left[\textrm{log}\, D \left(x \right) \right]&#xA;+ \mathbb{E}_{z\sim\mathit{p}_{\textrm{z}}\left(z \right)} \left[\textrm{log} \left(1 - D \left(G \left(z \right)\right)\right) \right]&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I understand this formula, at least it makes sense to me.&#xA;What I don't understand is that he writes in his &lt;a href=&quot;https://arxiv.org/pdf/1701.00160.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIPS tutorial&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the minimax game, the discriminator minimizes a cross-entropy, but the generator maximizes the same cross-entropy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why does he write that the discriminator &lt;strong&gt;minimizes&lt;/strong&gt; the cross-entropy while the generator &lt;strong&gt;maximizes&lt;/strong&gt; it? Shouldn't it be the other way around? At least that is how I understand &lt;span class=&quot;math-container&quot;&gt;$\underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess this shows that I have a fundamental error in my understanding. Could anyone clarify what I'm missing here?&lt;/p&gt;&#xA;" OwnerUserId="3199" LastEditorUserId="2444" LastEditDate="2020-05-18T15:20:58.277" LastActivityDate="2020-05-18T15:20:58.277" Title="Why does the discriminator minimize the cross-entropy while the generator maximize it?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16059" PostTypeId="2" ParentId="16056" CreationDate="2019-10-24T11:38:07.870" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible for value-based methods to learn stochastic policies? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but only in a limited sense, due to the ways it is possible to generate stochastic policies from a value function. For instance, the simplest exploratory policy used by SARSA and Monte Carlo Control, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, is stochastic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;SARSA natually learns the optimal &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy for any fixed value of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;. That is not quite the same as learning the optimal policy, but might still be useful in a non-stationary environment where exploration is always required and the algorithm is forever learning online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use other functions to generate stochastic policies from value functions. For instance, sampling from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boltzmann distribution&lt;/a&gt; over action values using a temperature parameter to decide relative priorities between actions with different action values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, all these approaches share the problem that they cannot converge towards an optimal stochastic policy. The policies are useful for mangaging exploration, but will only be optimal in the limited sense of &lt;em&gt;optimal given the fixed policy generator&lt;/em&gt; or by chance. There is no way for a purely value-based method to &lt;em&gt;learn&lt;/em&gt; a conversion from values to an optimal balance of probabilities for action choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For strict MDPs this is not an issue. If the MDP has the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; in the state representation, then there will always be a deterministic optimal policy, and value-based methods can converge towards it. That may include reducing &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy approaches or the temperature in Gibbs sampling, when using an on-policy method.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I started to think about settings where the optimal policy is stochastic(POMDP), and if it is possible to learn this policy for the &quot;traditional&quot; value-based methods&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It isn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To resolve this you need to add some kind of policy function and a mechanism to search for better policies directly by modifying that function. Policy Gradient methods are one approach, but you could include genetic algorithms or other search methods too under this idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may still be useful to use a value-based method as part of a policy search, to help evaluate changes to the policy. This is how Actor-Critic works.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-10-24T11:38:07.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16060" PostTypeId="2" ParentId="16058" CreationDate="2019-10-24T13:22:07.990" Score="1" Body="&lt;p&gt;Your mistake is that you think that the referenced &lt;span class=&quot;math-container&quot;&gt;$V(D,G)$&lt;/span&gt; is the deifinition of the cross entropy! Indeed, the cross entropy is defined base on the negative value of the &lt;span class=&quot;math-container&quot;&gt;$V(D,G)$&lt;/span&gt;. Hence, if you consider the minus behind the &lt;span class=&quot;math-container&quot;&gt;$V(D,G)$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$-V(D,G)$&lt;/span&gt;) the sentence will be meaningful.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-10-24T13:22:07.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16061" PostTypeId="1" CreationDate="2019-10-24T15:25:58.957" Score="4" ViewCount="697" Body="&lt;p&gt;I recently read a new paper (late 2019) about a one-shot object detector called &lt;a href=&quot;https://arxiv.org/abs/1904.07850&quot; rel=&quot;nofollow noreferrer&quot;&gt;CenterNet&lt;/a&gt;. Apart from this, I'm using Yolo (V3) one-shot detector, and what surprised me is the close &lt;strong&gt;similarity&lt;/strong&gt; between Yolo V1 and CenterNet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, both frameworks treat object detection as a regression problem, each of them outputs a tensor that can be seen as a grid with cells (below is an example of an output grid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pJTI7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pJTI7.png&quot; alt=&quot;Yolo V1 Grid cell&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each cell in this grid predicts an object class, a box offset relative to the cell's position and a box size. The only major difference between Yolo V1 and CenterNet is that Yolo also predicts an object confidence score, that is represented in CenterNet by the class score. Yolo also predicts 2 boxes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In brief, the tensor at one cell position is &lt;code&gt;Class + B x (Conf + Size + Off)&lt;/code&gt; for Yolo V1 and &lt;code&gt;Class + Size + Off&lt;/code&gt; for CenterNet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training strategy is quite similar too. Only the cell containing the center of a ground truth is responsible for that detection and thus affects the loss. Cells near the ground truths center (base on the distance for CenterNet and IoU for Darknet) have a reduced penalty in the loss (Focal Loss for CenterNet vs and tuned hyper parameter for Yolo).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss functions have near the same structure (see above) except that L1 is preferred in CenterNet while Yolo uses L2, among other subtleties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Wmrd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Wmrd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My point is not that Yolo V1 and CenterNet are the same — there are not — but they are far closer that it appears at first glance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that recent papers like CenterNet (CornerNet, ExtremeNet, Triplet CenterNet, MatrixNet) all claim to be &quot;Keypoint-based detector&quot; while they are not so much different than regular &quot;anchor-based&quot; detectors (that are preconditioned regressors in fact).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead I think that the biggest difference between Yolo and CenterNet is the backbone that has a bigger resolution for CenterNet (64x64) while Darknet has 7 or 8 only.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question is: do you see a major difference between the two concepts that I may have missed and that could explain the performance gap? I understand that new backbones, new loss functions and better resolutions can improve the accuracy but is there a structural difference between the two approaches?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="19859" LastEditorUserId="2444" LastEditDate="2021-01-29T00:02:07.467" LastActivityDate="2021-01-29T00:02:07.467" Title="What are the differences between Yolo v1 and CenterNet?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;object-detection&gt;&lt;yolo&gt;&lt;center-net&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16062" PostTypeId="1" CreationDate="2019-10-24T16:41:57.483" Score="4" ViewCount="194" Body="&lt;p&gt;I know this is a very general question, but I'm trying to illustrate this topic to people who are not from the field, and also my understanding is very limited since I'm just a second-year physics student with a basic understanding of R and Python. My point is, I'm not trying to say anything wrong here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So according to Wikipedia, after the second AI winter, which happened because expert systems didn't match expectations of the general public and of scientists, AI made a recovery &quot;due to increasing computational power (see Moore's law), greater emphasis on solving specific problems, new ties between AI and other fields (such as statistics, economics and mathematics), and a commitment by researchers to mathematical methods and scientific standards&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm trying to understand now is whether the rise of AI is rather connected to greater computational power available to the public or whether there have been fundamental mathematical advances that I'm not aware of. If the latter is the case (because according to my understanding, the mathematical models behind neural networks are rooted in the 70s and 80s), I would appreciate examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, please don't be offended by the general character of this question, I know it is probably really hard to answer correctly, however, I'm just trying to give a short historic introduction to the field to a lay audience and wanted to be clear in that regard.&lt;/p&gt;&#xA;" OwnerUserId="30765" LastEditorUserId="2444" LastEditDate="2019-10-24T16:52:40.823" LastActivityDate="2020-03-23T15:27:17.340" Title="What happened after the second AI winter?" Tags="&lt;history&gt;&lt;ai-winter&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16063" PostTypeId="5" CreationDate="2019-10-24T16:51:10.133" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-24T16:51:10.133" LastActivityDate="2019-10-24T16:51:10.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16064" PostTypeId="4" CreationDate="2019-10-24T16:51:10.133" Score="0" Body="For questions related to AI winters, which are periods of reduced funding and interest in artificial intelligence research, due to unmet expectations after a period of hype. There have been at least two major AI winters in 1974-1980 and 1987-1993." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-24T16:51:10.133" LastActivityDate="2019-10-24T16:51:10.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16065" PostTypeId="2" ParentId="16052" CreationDate="2019-10-24T17:05:20.237" Score="1" Body="&lt;p&gt;Thinking about this more, the answer is in fact yes, but not for the application you mention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You cannot use alpha-beta pruning to learn a model to predict customer outcomes, because it is only useful for domains where you are concerned about an &lt;em&gt;adversary&lt;/em&gt;. In finding a customer model, there is no reason to worry about someone coming in and forcing you to make bad decisions about the optimization of the model. Consequentially, there is no reason to use minimax search, and thus, to use alpha-beta pruning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There &lt;em&gt;are&lt;/em&gt; applications other than (video) games where you could use these techniques though. For example, there are &lt;a href=&quot;http://teamcore.usc.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;security games&lt;/a&gt;. In these &quot;games&quot; we want to use AI to find a strategy to protect an airport. It is reasonable to try and design our model under the assumption that someone else wants to break it. You could use Alpha-Beta pruning here (although in practice, more sophisticated algorithms are used). &lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-24T19:40:21.443" LastActivityDate="2019-10-24T19:40:21.443" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16066" PostTypeId="2" ParentId="16062" CreationDate="2019-10-24T18:04:12.307" Score="1" Body="&lt;p&gt;My reading of AI development (somewhat simplified here) is that the availability of large data sets, increased computing power, and the introduction of new machine learning algorithms (which require large data sets and massive computing power) contributed to the resurgence of AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as witnessed on this site, there has been a paradigm shift within the field: while previous approaches to AI were largely symbolic, with a bit of connectionism thrown in, the current AI mainstream is purely based on statistical models and machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Massive (distributed) computing power on its own is not sufficient, as there was a bottleneck with domain modelling/knowledge acquisition in traditional symbolic approaches. New algorithms (basically further developments of neural networks) on their own would also not be sufficient without massive amounts of training data. Only the combination of all three elements enabled the AI resurgence.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-10-24T18:04:12.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16067" PostTypeId="1" AcceptedAnswerId="16079" CreationDate="2019-10-24T19:47:51.923" Score="1" ViewCount="499" Body="&lt;p&gt;I’m doing some research into what hardware I need and what hardware I have available in college for a final year project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The project is designing a self driving car/computer vision system inside a simulation (instead of using an actual car and track as that’s out of scope for a final year project)&#xA;The simulation will be an emulator of either N64 or SNES. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the recommended hardware specs?&#xA;How will having many GPUs benefit the project?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can also get the specs of some of the available computers I have in college?&#xA;I may even be able to get the college to fund new hardware. &lt;/p&gt;&#xA;" OwnerUserId="30270" LastActivityDate="2020-08-19T08:22:03.513" Title="Necessity of GPUs and hardware minimum specs for Deep Learning?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;hardware&gt;&lt;image-processing&gt;&lt;hardware-evaluation&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ClosedDate="2020-08-19T22:03:39.947" ContentLicense="CC BY-SA 4.0" />
  <row Id="16069" PostTypeId="1" AcceptedAnswerId="16070" CreationDate="2019-10-24T23:21:41.170" Score="2" ViewCount="103" Body="&lt;p&gt;In Chapter 1 of the book Reinformcement Learning An Introduction 2nd Edition by Richard S. Sutton and Andrew G.Barto, there is one statement &quot;Exploratory moves do not result in any learning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sentence is in Figure 1.1.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Figure 1.1: A sequence of tic-tac-toe moves. The solid black lines represent the moves taken during a game; the dashed lines represent moves that we (our reinforcement learning player) considered but did not make. Our second move was an exploratory move, meaning that it was taken even though another sibling move, the one leading to e⇤, was ranked higher. Exploratory moves do not result in any learning, but each of our other moves does, causing updates as suggested by the red arrows in which estimated values are moved up the tree from later nodes to earlier nodes as detailed in the text.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It confuses me. In my understanding, exploration should contribute to learning in almost all RL algorithms. So, why does the book state &quot;Exploratory moves do not result in any learning&quot; in this case?&lt;/p&gt;&#xA;" OwnerUserId="30044" LastActivityDate="2019-10-25T11:46:25.600" Title="Why &quot;Exploratory moves do not result in any learning&quot;?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16070" PostTypeId="2" ParentId="16069" CreationDate="2019-10-25T00:14:23.983" Score="3" Body="&lt;p&gt;I &lt;em&gt;believe&lt;/em&gt; this is a pedagogical decision. Because the sentence occurs in the first chapter of the book, I think the authors are trying to avoid the objection that a neophyte might make: learning from random movements &lt;em&gt;seems&lt;/em&gt; like it will cause you to learn strange behaviors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly, the statement is inaccurate. We need only reach page 26 to see a counterexample: The Q-learning equation (2.1) sums over &lt;em&gt;all&lt;/em&gt; actions, not just the greedy ones. This also applies to the temporal difference learning method specified in Chapter 6, which is the method Figure 1.1 is discussing.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-25T11:46:25.600" LastActivityDate="2019-10-25T11:46:25.600" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16071" PostTypeId="2" ParentId="16067" CreationDate="2019-10-25T05:22:56.007" Score="1" Body="&lt;p&gt;Your hardware choice depends majorly on the sensors and camera specifications you need for your solution.&#xA;For example, we are running a 2536x1920 resolution 20fps camera to detect cars and then read their number plates. Nvidia's RTX 2080Ti 11 GB can handle 2 such cameras at 20fps (real-time). However, this involves 2 models: 1 for detection of plates and a second for Optical Character Recognition (OCR, text reading of number plates (heavier)). If it were just detection, may be 4 cameras could be handled by a single GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your camera fps is higher (which, I think should be the case) you may require a better card with Volta architecture.&#xA;DGX1 is one such machine. Edge TPUs don't provide as good a performance compared to such heavy GPUs in my experience.&lt;/p&gt;&#xA;" OwnerUserId="30778" LastEditorUserId="4709" LastEditDate="2019-11-18T06:07:16.923" LastActivityDate="2019-11-18T06:07:16.923" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16072" PostTypeId="1" CreationDate="2019-10-25T05:46:16.507" Score="1" ViewCount="11" Body="&lt;p&gt;I am interested in the node classification task for graph data. So far,I've tried it with the Cora dataset, but it is an undirected graph and has word attributes as features. I want to extend this task to a time-varying directed graph. Does anybody know about this kind of dataset?&lt;/p&gt;&#xA;" OwnerUserId="26886" LastActivityDate="2019-10-25T05:46:16.507" Title="Is there any time-varying directed graph dataset?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;geometric-deep-learning&gt;&lt;graph-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16073" PostTypeId="1" AcceptedAnswerId="16078" CreationDate="2019-10-25T08:32:46.553" Score="3" ViewCount="493" Body="&lt;p&gt;I am pretty new to Artificial Intelligence programming, however i do understand the basic concept. &#xA;I have an idea in my mind: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Import a JPEG Image, &#xA;Convert this Image into a 2D Array (x,y values + r g b values). &#xA;Then create a second array with same (xy) values wit rgb all set to 0,0,0.&#xA;Now i want to build an AI Layer which will try to lower the error factor between the arrays until they are equal (the rgb values in the second array are equal to the first array (error factor 0) ).&#xA;I would prefer to do it in Java. Any suggestions to librarys or example that can help me get started? Thanks for any help. &lt;/p&gt;&#xA;" OwnerUserId="30781" LastActivityDate="2019-10-25T13:19:45.127" Title="Generate Image with Artificial intelligence" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;neurons&gt;&lt;java&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16074" PostTypeId="2" ParentId="6908" CreationDate="2019-10-25T08:40:57.983" Score="2" Body="&lt;p&gt;You can take a look at this &lt;a href=&quot;https://arxiv.org/pdf/1905.12149.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; that solving your problem with a neural network. You can use the pytorch implementation of the satnet layer : &lt;a href=&quot;https://github.com/locuslab/SATNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;satnet layer API&lt;/a&gt;. In this supervised setup the layer also learn the boolean constraints of your model. You can find an example of a sodoku solver in the github repo.&lt;/p&gt;&#xA;" OwnerUserId="8912" LastActivityDate="2019-10-25T08:40:57.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16075" PostTypeId="1" CreationDate="2019-10-25T11:13:20.683" Score="1" ViewCount="37" Body="&lt;p&gt;I'm trying to run the Deepmind Spriteworld demo described on the &lt;a href=&quot;https://github.com/deepmind/spriteworld&quot; rel=&quot;nofollow noreferrer&quot;&gt;project's GitHub page&lt;/a&gt;, but I'm not finding &lt;code&gt;run_demo.py&lt;/code&gt; in the distribution and the closest sounding file, &lt;code&gt;demo_ui.py&lt;/code&gt; doesn't launch a UI  when run (tried both on Linux and Windows).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should the Deepmind Spriteworld demo UI be launched?&lt;/p&gt;&#xA;" OwnerUserId="30787" LastEditorUserId="30787" LastEditDate="2019-10-25T15:34:20.087" LastActivityDate="2019-11-24T16:00:41.967" Title="Deepmind Spriteworld run_demo.py not found" Tags="&lt;reinforcement-learning&gt;&lt;deepmind&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-01-10T20:12:33.353" ContentLicense="CC BY-SA 4.0" />
  <row Id="16076" PostTypeId="1" CreationDate="2019-10-25T11:55:48.227" Score="2" ViewCount="290" Body="&lt;p&gt;I'm trying to extract some particular information from the image(png).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to extract the text using the below code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cv2&#xA;import pytesseract&#xA;import os&#xA;from PIL import Image&#xA;import sys&#xA;&#xA;def get_string(img_path):&#xA;    # Read image with opencv&#xA;    img = cv2.imread(img_path)&#xA;&#xA;    # Convert to gray&#xA;    img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)&#xA;    # Apply dilation and erosion to remove some noise&#xA;    kernel = np.ones((1, 1), np.uint8)&#xA;    img = cv2.dilate(img, kernel, iterations=1)&#xA;    img = cv2.erode(img, kernel, iterations=1)&#xA;&#xA;    # Write the image after apply opencv to do some ...&#xA;    cv2.imwrite(&quot;thres.png&quot;, img)&#xA;    # Recognize text with tesseract for python&#xA;    result = pytesseract.image_to_string(Image.open(&quot;invoice.png&quot;))&#xA;    os.remove(&quot;invoice.png&quot;)&#xA;&#xA;    return result&#xA;&#xA;if __name__ == '__main__':&#xA;    from sys import argv&#xA;&#xA;    if len(argv)&amp;lt;2:&#xA;        print(&quot;Usage: python image-to-text.py relative-filepath&quot;)&#xA;    else:&#xA;        print('--- Start recognize text from image ---')&#xA;        for i in range(1,len(argv)):&#xA;            print(argv[i])&#xA;            print(get_string(argv[i]))&#xA;            print()&#xA;            print()&#xA;&#xA;        print('------ Done -------')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I want to extract data from particular fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt; a) INVOICE NO.&#xA; b) CUSTOMER NO.&#xA; c) SUBTOTAL&#xA; d) TOTAL&#xA; e) DATE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I extract the required information from the below image &quot;invoice&quot;?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PFB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RarrU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RarrU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2019-11-15T10:47:17.127" LastActivityDate="2019-12-15T11:01:35.310" Title="How to Extract Information from the Image" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16078" PostTypeId="2" ParentId="16073" CreationDate="2019-10-25T13:19:45.127" Score="0" Body="&lt;p&gt;For recreating an image exactly the same as the original, you can use an autoencoder. This basically use AI Layers to encode the image raw pixel values to a vector of floats, drastically decreasing the representing vector. Afterwards another AI Layer increases the dimensions back to the original image. The method does not required labels, as it only refer to teh image to encode it to a vector of features. For implementing in java, there is not a lot 9f resources. However, you can check this library out: &lt;a href=&quot;https://deeplearning4j.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deeplearning4j.org/&lt;/a&gt;&#xA;For the implementation, see this: &lt;a href=&quot;https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/unsupervised/variational/VariationalAutoEncoderExample.java&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/unsupervised/variational/VariationalAutoEncoderExample.java&lt;/a&gt;&#xA;For the method, you can see this python tutorial and implement it in java. &lt;a href=&quot;https://towardsdatascience.com/autoencoders-in-keras-c1f57b9a2fd7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/autoencoders-in-keras-c1f57b9a2fd7&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For generating completely new image, you can try GAN(Generative Adverserial Network). This generates completely new images from a random noise image. The noise image is passed through a generator which is a CNN(convolutional neural network) and get a result image. The result image is then feed to  a discriminator(CNN as well) to classify if that image is fake or real. The generator and discriminator compete and slowly gets better. For java implementation, see this: &lt;a href=&quot;https://github.com/wmeddie/dl4j-gans&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/wmeddie/dl4j-gans&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-25T13:19:45.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16079" PostTypeId="2" ParentId="16067" CreationDate="2019-10-25T13:29:31.373" Score="2" Body="&lt;p&gt;For a simulation of SNES or N64, the resolution is probably not that high. You can use either online credits or buy new hardware. For online credits, it is recommended if you do teh simulation/training for only several dozens hours, as eache hour costs around 5-10 dollars. AWS is a good choice as it have a large variety of choice for hardware. For higher performance, choose multi GPU setup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the simulation requires a long time, or you wish to own the hardware, you should buy your own hardware. Multiple GPU will scale quite nicely, so you may go for multi GPU if performance is needed. For single GPU, a RTX 2000 series is recommended as it have tensor cores to accelerate training of CNN. For a budget option, consider an RTX 2060 SUPER. It have 8 GB of VRAM with a relatively low price and yet it have tensor core functionality. For more powerful GPU, choose the RTX 2080 TI. It have 11GB of VRAM which is enough in most use case. &#xA;For multiple GPU, you might want several RTX 2060 SUPER as they are cheap but the performance is quite good. The 8GB VRAM helps as well. For the highest performance, choose 2 or more RTX 2080 TI. It is enough for nearly most task. If this is not enough, renting an online instance is a better idea. Hope it helps and have a nice day. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-10-25T13:35:47.717" LastActivityDate="2019-10-25T13:35:47.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16080" PostTypeId="1" CreationDate="2019-10-25T13:30:35.780" Score="4" ViewCount="124" Body="&lt;p&gt;I recently started looking for networks that focus on image segmentation tasks related to biomedical applications. I could not miss the publication &lt;a href=&quot;https://arxiv.org/abs/1505.04597&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net: Convolutional Networks for Biomedical Image Segmentation&lt;/a&gt; (2015) by Ronneberger, Fischer, and Brox. However, as deep learning is a fast-growing field and the article was published more than 4 years ago, I was wondering if anyone knows other algorithms that yield better results for image segmentation tasks?  And if so, do they also use a U-shape architecture (i.e. contraction path then expansion path with up-conv)? &lt;/p&gt;&#xA;" OwnerUserId="30792" LastEditorUserId="2444" LastEditDate="2020-06-13T00:07:27.037" LastActivityDate="2020-06-13T00:07:27.037" Title="What are the best algorithms for image segmentation tasks?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16081" PostTypeId="2" ParentId="16075" CreationDate="2019-10-25T15:31:58.067" Score="0" Body="&lt;p&gt;I posted the question to the &lt;a href=&quot;https://github.com/deepmind/spriteworld/issues/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;project's issue page&lt;/a&gt;, where the recommendation was to manually download and run &lt;a href=&quot;https://github.com/deepmind/spriteworld/blob/master/run_demo.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;run_demo.py&lt;/a&gt;. That worked for me.   &lt;/p&gt;&#xA;" OwnerUserId="30787" LastActivityDate="2019-10-25T15:31:58.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16082" PostTypeId="2" ParentId="15946" CreationDate="2019-10-25T15:41:43.090" Score="2" Body="&lt;p&gt;From the top of my mind roughly in order of priority excl. math: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Practical/applied CS&lt;/strong&gt;: machine learning, artificial intelligence (incl. symbolic AI), data mining, algorithms, data structures&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theoretical CS&lt;/strong&gt;: complexity theory&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Programming&lt;/strong&gt;: Python&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logic is highly relevant for symbolic AI but not so much for sub-symbolic approaches like ML. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For all topics mentioned in the first two categories you can find free online lectures from different universities. &lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2019-10-25T15:41:43.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16083" PostTypeId="2" ParentId="15802" CreationDate="2019-10-25T15:51:39.550" Score="2" Body="&lt;p&gt;As originally conceived in James Baker's 1989 paper &lt;a href=&quot;https://books.google.com/books?hl=en&amp;amp;lr=&amp;amp;id=MYJ_AAAAQBAJ&amp;amp;oi=fnd&amp;amp;pg=PA14&amp;amp;dq=Reducing%20Bias%20and%20Inefficiency%20in%20the%20Selection%20Algorithm&amp;amp;ots=XwtGqi3xGF&amp;amp;sig=ldv4xuElciFpR_XbEITj-9pbzQs#v=snippet&amp;amp;q=stochastic%20universal&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reducing bias and inefficiency in the selection algorithm&lt;/a&gt;, Stochastic Universal Sampling accepts a population containing &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; individuals, and a &lt;em&gt;number of parents to sample&lt;/em&gt;, denoted &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. Assuming fitness values are normalized so that they sum up to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, at each step, a new pointer is placed a step equal in size to the fraction &lt;span class=&quot;math-container&quot;&gt;$\frac{N}{n}$&lt;/span&gt; ahead of the location of the previous pointer (and the location of the first pointer is set to a random value in the range [0, &lt;span class=&quot;math-container&quot;&gt;$\frac{N}{n}$&lt;/span&gt;) ). So, for example, if you want to sample 6 individuals from a population of size 10, you would make steps of size &lt;span class=&quot;math-container&quot;&gt;$\frac{10}{6}$&lt;/span&gt;, spacing your pointers at even intervals of &lt;span class=&quot;math-container&quot;&gt;$\frac{10}{6}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern implementations, like the one on &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_universal_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; sometimes do not document this fact clearly, although it is apparent what is intended if you already understand the method. They often write the step size as &lt;span class=&quot;math-container&quot;&gt;$\frac{F}{n}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is the total fitness of the population, without discussing its relation to the size of the population. The extra normalization step is actually not essential, so modern implementations generally seem to skip it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in summary, the step size &lt;span class=&quot;math-container&quot;&gt;$\frac{F}{n}$&lt;/span&gt; used if fitness values of a population sum to &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;, and you want to select &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; individuals. If you want to select more individuals, use a higher value for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. If you want to select fewer, use a lower value for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, which updates your step size accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Values of this parameter of &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{4}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{6}$&lt;/span&gt; suggest that the implementation may be normalizing the sum of fitness values is being normalized to N, and then using the parameter as a multiplicative factor automatically. This is a fairly reasonable design. You could interpret these values as &quot;Select &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{4}$&lt;/span&gt; of the population&quot; and &quot;Select &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{6}$&lt;/span&gt; of the population&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this sort of bumps your question up a level: how do you pick the fraction of the population to keep? That question doesn't have a clean answer, and picking it is generally an art developed by experts through practice. It is very closely related to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning#Exploration&quot; rel=&quot;nofollow noreferrer&quot;&gt;exploration/exploitation tradeoff&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ways you might pick &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a fixed value, for instance, keep half the population at each step. The exact proportion you will want to pick is not something you can know in advance. Expert practitioners can make effective guesses. Others will need to just try out different values using a technique like &lt;a href=&quot;https://machinelearningmastery.com/k-fold-cross-validation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross validation&lt;/a&gt;, and pick whichever one seems to work best.&lt;/li&gt;&#xA;&lt;li&gt;You can use a value that changes over time. A common strategy for this would be to use one of the &lt;a href=&quot;https://www.fys.ku.dk/~andresen/BAhome/ownpapers/permanents/annealSched.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;temperature schedules developed in the simulated annealing literature&lt;/a&gt;, and keep a portion of the population that was inversely proportionate to the temperature. That is, early on, you'd use a large &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and keep most of the population around (probably with mutations). Later, you'd use a small &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and keep only the best individuals around.&lt;/li&gt;&#xA;&lt;li&gt;You could use a value of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; that changes in response to the fitness of the population. This is much like the &lt;a href=&quot;https://towardsdatascience.com/gradient-descent-algorithms-and-adaptive-learning-rate-adjustment-methods-79c701b086be?gi=12bf1035e653&quot; rel=&quot;nofollow noreferrer&quot;&gt;adaptive learning rates&lt;/a&gt; used in some algorithms to train neural networks (most notably: the ADAM optimizer). When fitness levels have a lot of variety, use low values of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to encourage more exploitation. When fitness levels are all within a narrow band, use high values of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to encourage more exploration. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2020-01-15T15:42:40.400" LastActivityDate="2020-01-15T15:42:40.400" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16085" PostTypeId="2" ParentId="15946" CreationDate="2019-10-25T18:14:12.787" Score="4" Body="&lt;p&gt;I worked as a professor for a time, and often advised students on this. For a PhD in machine learning, I think the ideal background is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Core CS Courses&lt;/strong&gt; &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Programming (typically 3-4 courses). Language choice is not highly important, but Python, C++, Java, and perhaps JavaScript, are reasonable picks, if only because of their prevalence.&lt;/li&gt;&#xA;&lt;li&gt;Core topics: data structures, algorithms, operating systems, databases&lt;/li&gt;&#xA;&lt;li&gt;numerical linear algebra or numerical methods&lt;/li&gt;&#xA;&lt;li&gt;advanced algorithm design and analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Together these will allow you to read and write the code that even highly optimized versions of ML algorithms are written in, and to understand what might be going wrong within them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. AI &amp;amp; ML courses, usually offered through a CS department&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a broad survey course in AI (like AI:AMA by Russel &amp;amp; Norvig), usually offered to senior undergraduates. &lt;/li&gt;&#xA;&lt;li&gt;A course applied machine learning or data mining. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You may also take other AI courses, but they are not as common to see offered to undergraduates, so many students wait until graduate school:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;reinforcement learning&lt;/li&gt;&#xA;&lt;li&gt;soft computing&lt;/li&gt;&#xA;&lt;li&gt;computational learning theory&lt;/li&gt;&#xA;&lt;li&gt;Bayesian Methods&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning&lt;/li&gt;&#xA;&lt;li&gt;Multiagent Systems&lt;/li&gt;&#xA;&lt;li&gt;Information Retrieval&lt;/li&gt;&#xA;&lt;li&gt;Natural Language Processing&lt;/li&gt;&#xA;&lt;li&gt;Computer Vision&lt;/li&gt;&#xA;&lt;li&gt;Robotics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Together, these will give you the broadest possible background in AI &amp;amp; ML. These can allow you to find new applications of ML, or to pull AI techniques from one area into another as you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Statistics courses&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a 1 or 2 term course in probability theory, ideally a version that requires and uses calculus. &lt;/li&gt;&#xA;&lt;li&gt;at &lt;em&gt;minimum&lt;/em&gt; a course in statistical hypothesis testing. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Much stronger would be to also take courses in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;regression&lt;/li&gt;&#xA;&lt;li&gt;generalized linear models&lt;/li&gt;&#xA;&lt;li&gt;experiment design&lt;/li&gt;&#xA;&lt;li&gt;causal inference&lt;/li&gt;&#xA;&lt;li&gt;Bayesian methods&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These courses allow you to reason formally and comfortably about uncertainty. They also give you the correct framework for answering questions about whether your ML algorithm is working, and what patterns an ML algorithm uncovers mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4. Mathematics courses&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3 semesters of calculus, going at least as far as multi-variate/vector calculus. &lt;/li&gt;&#xA;&lt;li&gt;optionally, a more advanced course that builds on calculus, like real analysis, but only to reinforce calculus concepts.&lt;/li&gt;&#xA;&lt;li&gt;at least 1, and preferably 2, courses in linear algebra&lt;/li&gt;&#xA;&lt;li&gt;at least 1, and preferably 2, courses in discrete mathematics. &lt;/li&gt;&#xA;&lt;li&gt;ideally, something like Knuth et al.'s &lt;em&gt;Concrete Mathematics&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;ideally a course in advanced optimization techniques&lt;/li&gt;&#xA;&lt;li&gt;optionally, courses in logic, but be aware that this is almost a fringe area in AI now, and essentially irrelevant to a PhD in machine learning. The parts you need are usually covered in a broad survey AI course.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These courses give you the basic mathematical fluency to understand most machine learning algorithms well.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-10-25T20:05:55.823" LastActivityDate="2019-10-25T20:05:55.823" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16086" PostTypeId="2" ParentId="16036" CreationDate="2019-10-25T23:48:21.873" Score="2" Body="&lt;p&gt;Historically, the non-ML approach would be an &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;expert system&lt;/a&gt;.  This is typically a rules-based decision system, falling under the umbrella of &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic AI&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These systems can have strong utility in limited contexts, but are generally &quot;brittle&quot; in that parameters not previously defined or accounted will produce no-compute or weak utility. Because the rules of a game are fully definable, the main concern is utility, which relates to the degree to which the game has been solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Informing a &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;heuristic&lt;/a&gt; system in this case requires analysis of the game in in the sense of game theory and combinatorial game theory, since Catan involves both imperfect information and combinatorial elements.  The complexity is high indeed, not only per imperfect information, branching factors, stochasticity, players &gt; 2, but, as you note, the game board itself has a very high number of potential configurations, so solving the game is presumed to be extremely difficult to impossible.  (Possibly &lt;a href=&quot;https://en.wikipedia.org/wiki/NEXPTIME&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEXPTIME&lt;/a&gt; if finite and undecidable otherwise.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://homepages.inf.ed.ac.uk/alex/papers/cig2014_gs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game strategies for The Settlers of Catan&lt;/a&gt; suggests that the game tree for Catan is not surveyable b/c the options for trade negotiation in natural language aren't bounded:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One response to this is to develop a symbolic model consisting of heuristic strategies for playing the game. Developing&#xA;  such models potentially has two advantages. First, a symbolic&#xA;  model can in principle lead to an interpretable model of human&#xA;  expert play ... Second, a symbolic model can provide&#xA;  a prior distribution over which next move is likely to be&#xA;  optimal...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The paper mentions this second part to relation to machine learning, where &quot;the posterior distribution over optimal actions acquired through training improves on the baseline prior distribution.&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially where the game is unsolved and intractable, machine learning has demonstrated strong utility for an increasing number of games, so it is unlikely not to be an optimal component for truly strong play.  However, such a system can be a combination of ML and &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain_knowledge&quot; rel=&quot;nofollow noreferrer&quot;&gt;domain specific knowledge&lt;/a&gt;, such as in &lt;a href=&quot;https://en.wikipedia.org/wiki/Search_algorithm#For_virtual_search_spaces&quot; rel=&quot;nofollow noreferrer&quot;&gt;informed search&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://pdfs.semanticscholar.org/0ef6/7eac64e0d7ee2d68abf3976751940616aab9.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimizing UCT for Settlers of Catan&lt;/a&gt; goes into this in detail, and also provides reference to prior work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your primary requirement is strong utility, some form of machine learning is likely optimal.  But it can be fun to attempt to solve games and cobble together sets of heuristics. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-26T19:49:28.217" LastActivityDate="2019-10-26T19:49:28.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16087" PostTypeId="1" CreationDate="2019-10-26T06:51:11.217" Score="8" ViewCount="361" Body="&lt;p&gt;According to the &lt;a href=&quot;https://arxiv.org/pdf/1712.10285.pdf&quot; rel=&quot;noreferrer&quot;&gt;SBEED: Convergent Reinforcement Learning with&#xA;Nonlinear Function Approximation&lt;/a&gt; for convergent reinforcement learning, the Smoothed Bellman operator is a way to dodge the double sample problem? Can someone explain to me what the double sample problem is and how SBEED solves it?&lt;/p&gt;&#xA;" OwnerUserId="30632" LastEditorUserId="1671" LastEditDate="2019-10-27T23:03:12.697" LastActivityDate="2021-05-28T13:38:47.587" Title="What is the double sample problem in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16088" PostTypeId="1" CreationDate="2019-10-26T10:08:11.423" Score="2" ViewCount="69" Body="&lt;p&gt;I understand fuzzy logic is a variant of formal logic where, instead of just 0 or 1, a given sentence may have a truth value in the [0..1] interval. Also, I understand that logical probability (objective bayesian) understands probability as an extension of logic, where uncertainity is taken into account. To me they sound rather similar (they both extend formal logic by modelling truth as a continuos interval between 0 and 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, &lt;strong&gt;what is the relationship between these two concepts?. What is the difference, and what are the differences in AI approaches based upon these two formal systems?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2020-06-10T07:21:48.050" LastActivityDate="2020-06-10T07:21:48.050" Title="What is the relationship between fuzzy logic and objective bayesian probability?" Tags="&lt;fuzzy-logic&gt;&lt;bayesian-probability&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16089" PostTypeId="5" CreationDate="2019-10-26T10:17:31.973" Score="0" Body="&lt;p&gt;Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification of a personal belief. The Bayesian interpretation of probability can be seen as an extension of propositional logic that enables reasoning with hypotheses.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-10-26T19:46:46.130" LastActivityDate="2019-10-26T19:46:46.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16090" PostTypeId="4" CreationDate="2019-10-26T10:17:31.973" Score="0" Body="Questions in this tag should be about the Bayesian approach of probability theory and its relevance to AI-related issues." OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-10-26T19:46:14.817" LastActivityDate="2019-10-26T19:46:14.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16091" PostTypeId="2" ParentId="16036" CreationDate="2019-10-26T11:03:42.867" Score="0" Body="&lt;p&gt;From the way you have phrased your question one can derive a couple of strong assumptions which simplify the problem tremendously and make it feasable:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We do not look for an agent being able to play the game but only an evaluation of settlement options (no other agents to be considered)&lt;/li&gt;&#xA;&lt;li&gt;The evaluation of settlement options is static (i.e. does not change over time) and is independent of other settlements&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From that two simple ideas come to my mind: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. the ML approach&lt;/strong&gt;&#xA;Look at historical game data and see which settlement options led to a win. So basically look at tuples of (X,y) with X being something like (W8, C2, O6) meaning that the settlement give access to wood with an 8, clay with a 2 and ore with a 6. And y indicating a win or loss. &#xA;To make it a bit dynamic you could differentiate between initial settlements (being placed at the beginning) and the ones during the game. So for each of these two categories you would derive basically a score for the possible settlements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can compute all the possible combinations you will not even need ML since you can simply run the math once and then look it up. Might be doable in this case as the assumptions mentioned above simplify the problem a lot (compared to fully &quot;solving&quot; the game). Thinking through the possible combinations for a given settlement location (selecting 3 fields with A possible resources and B possible numbers) will quickly give you an idea about that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. The classy symbolic approach&lt;/strong&gt;&#xA;What comes to my mind right away is linear programming as it offers a convenient way to model the strategic aspects you have mentioned. You could develop a target function to maximize using scores for different resources and numbers (e.g. you could give clay higher importance than wool). Besides that constraints can capture additional aspects of game strategies like &quot;always make sure to have access to clay&quot; or &quot;do not settle where the 3 resources are the same&quot; etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My very first idea to model this is using decision variables like X_(i,j) with X being 0 or 1, i representing the resources out of {clay, wood, ..., desert} (side note: do not forget the water and different ports here) and j modelling the numbers out of {2,...12}. The constraints would need to model the fact that you need to select 3 of those X_(i,j) for every settlement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to calculate this for a given game you would need to feed the model the possible settlement options based on the layout of that specific game. Then run the optimization and it gives you the best settlement option (i.e. the 3 feasable X_(i,j) maximizing your goal function).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Qua definition you need to bring in game knowledge for this approach. And probably talking to someone who is really good at the game would help to understand what matters. &lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2019-10-26T11:03:42.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16092" PostTypeId="2" ParentId="16088" CreationDate="2019-10-26T13:29:02.680" Score="3" Body="&lt;p&gt;This was a somewhat hotly debated question in the 1980s. The debate was more-or-less ended with papers like Cheeseman's &lt;em&gt;&lt;a href=&quot;https://www.ijcai.org/Proceedings/85-2/Papers/064.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;In Defense of Probability&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short answer is that Fuzzy Logic does not &lt;em&gt;just&lt;/em&gt; assign a continuous value to sentences, what it does is assign &lt;em&gt;degrees of membership&lt;/em&gt; in different &lt;em&gt;fuzzy sets&lt;/em&gt;. These degrees of membership range between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, probability says: Among the set of values this variable could take on, what &lt;em&gt;fraction&lt;/em&gt; of them are in a certain set? This fraction also ranges between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might seem like a semantic distinction, but it has deep implications if you try to use these systems to reason about uncertainty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the question &quot;Will it rain tomorrow?&quot;. A probabilistic approach would try to determine the fraction of days like tomorrow that have had rain. A fuzzy logic approach will try to determine whether tomorrow is &lt;em&gt;like&lt;/em&gt; a rainy day. The distinction becomes obvious if we then ask whether it will &lt;em&gt;not&lt;/em&gt; rain tomorrow. The probabilistic approach will try to find the fraction of days like tomorrow where it did not rain. The chance of it raining or not raining will sum to 1. The fuzzy logic approach will try to determine whether tomorrow is &lt;em&gt;like&lt;/em&gt; a non-rainy day. Note that tomorrow may be both like a stereotypical rainy day &lt;em&gt;and&lt;/em&gt; not like one. There is no firm requirement that these sets be disjoint. This reflects Cheeseman's critique of Fuzzy Sets: they implicitly reject the additive axiom of probability theory, which (as denoted by the name &lt;em&gt;axiom&lt;/em&gt;) is something that is fairly unreasonable to reject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are various approaches to make memberships in fuzzy sets more probabilistic in nature, that's the root distinction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the dominant modern view is that fuzzy sets are a great tool when you need to reason about membership in a fuzzy concept. Whether &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/B00007J5U7&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;rice is &quot;done&quot; or not&lt;/a&gt; isn't a question of something being true or false. It's a degree of membership in the set &quot;done&quot;. On the other hand, whether the object in front of a self-driving car is a person or not is either true or false. It is not necessarily a good idea to reason about this in terms of whether the object is partially in the person set or plastic bag set. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-26T13:29:02.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16093" PostTypeId="2" ParentId="16054" CreationDate="2019-10-26T14:49:24.863" Score="2" Body="&lt;p&gt;One risk that’s already realized:  large online vendors think they have implemented artificial intelligence in their “help” pages and therefore they can (try to) make it impossible to get to someone who can actually think.  And since the artificial &lt;strong&gt;stupidity&lt;/strong&gt; (AS) usually feeds the customer articles completely unrelated to the issue, anyone sufficiently persistent to pursue it is extremely pissed off at the company before (if ever) the issue is resolved.  And because far too many people passively accept this abuse, the companies have no incentive to be more reasonable.  In other words, “AS” is reducing our expectations for customer service.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another is the JavaScript intended to prevent invalid names, phone numbers, and email addresses in web forms which due to bugs or obsolescence rejects legitimate inputs.&lt;/p&gt;&#xA;" OwnerUserId="30818" LastActivityDate="2019-10-26T14:49:24.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16094" PostTypeId="2" ParentId="16054" CreationDate="2019-10-26T16:21:28.940" Score="1" Body="&lt;p&gt;IMHO the greatest risk is that AI can make people lazy.  If you can ask an AI for an answer to any problem, what's your motivation to figure out how to figure out the answer for yourself?  I have run into a lot of young people who can't add or multiply two three-digit numbers without using a calculator.  When it's possible to dump a huge mass of data into an AI, and the AI tells you the structures it finds in the data without explaining &lt;em&gt;how&lt;/em&gt; it finds the structures so you can do it yourself, the AI wins and you lose.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-10-26T16:21:28.940" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16095" PostTypeId="2" ParentId="16054" CreationDate="2019-10-26T20:21:45.737" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;Offloading of responsibility may the single greatest danger.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Where algorithmic bias may be the core issue of Machine Learning, it can be identified and mitigated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transferring responsibility to a robot or algorithm requires an intentional choice with moral dimension.  As the scholar Joanna Bryson put it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In humans consciousness and ethics are associated with our morality, but that is because of our evolutionary and cultural history.  In artefacts, moral obligation is not tied by either logical or mechanical necessity to awareness or feelings.  This is one of the reasons we shouldn't make AI responsible: we can't punish it in a meaningful way.&#xA;  &lt;br&gt;&lt;sub&gt;Source: &lt;a href=&quot;http://www.cs.bath.ac.uk/~jjb/web/ai.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI Ethics: Artificial Intelligence, Robots, and Society&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a malicious sense, transferring agency to an automaton that may do something harmful which benefits me allows me to say &quot;I didn't make the decision and have no responsibility for the outcome.&quot;  (It seems to me that companies are doing this more and more.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was a very good short story on the subject &lt;em&gt;&lt;a href=&quot;https://www.technologyreview.com/s/610831/unchained-a-story-of-love-loss-and-blockchain/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unchained: A story of love, loss, and blockchain&lt;/a&gt;&lt;/em&gt; in which automate taxis develop novel strategies that have an unintended moral dimension in regard to humans.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-10-26T20:40:12.457" LastActivityDate="2019-10-26T20:40:12.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16098" PostTypeId="1" CreationDate="2019-10-26T23:47:18.310" Score="2" ViewCount="26" Body="&lt;p&gt;I'm working on finding out the motion vectors of objects in images. The inputs are the images of objects in motion. The outputs of neural network are the object name, direction of object vector and prediction of next vector change. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different 3D ConvNets I'm considering as a baseline like ReMotENet. I would appreciate if you would recommend any interesting papers in MoCap domain and any existing neural networks performing similar task.&lt;/p&gt;&#xA;" OwnerUserId="29928" LastActivityDate="2019-10-26T23:47:18.310" Title="Which neural network algorithms can be used to map motion vectors in image processing?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16099" PostTypeId="1" CreationDate="2019-10-27T08:22:50.173" Score="2" ViewCount="67" Body="&lt;p&gt;In my experience, most of the time, when people talk about AI nowadays they mostly mean machine learning. Despite this, ML is usually seen as a mere technique to build high-performance software; rarely I see people discuss the foundational questions of it, such as, from which &quot;philosophy&quot; of AI did ML emerge? Why is ML compelling in AI research, if not by its performance? What are the fundamental differences between statistical/probabilistic AI and logical AI? For reference, this hasn't even been mentioned on my master-level course on machine learning. Even myself I used to have a distaste for ML because I thought it was just mindless data-crunching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But lately I've been reading through &quot;&lt;a href=&quot;https://www.goodreads.com/book/show/151848.Probability_Theory?from_search=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;Probability Theory: The Logic Of Science&lt;/a&gt;&quot; and I'm starting to appreciate the theoretical side of ML, for instance, how bayesian probability can be seen as a model of plausible reasoning in humans, and how probability theory extends logic (motivating, maybe, why probabilistic AIs were the next logical [no pun intended] step after logical AI). I would like now to delve deeper in the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are some books/paper that deal with fundamental and philosophical issues of ML and relate it to the global discourse of AIs?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2019-10-27T18:04:38.760" LastActivityDate="2019-10-27T20:40:58.780" Title="What are some books/paper that deal with fundamental and philosophical issues of ML and relate it to the global discourse of AIs?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16100" PostTypeId="2" ParentId="15601" CreationDate="2019-10-27T09:03:51.700" Score="1" Body="&lt;p&gt;Try using float64 instead of float32; int64 instead of int32; increasing the bits of memory gradually increases the weights that can be stored&lt;/p&gt;&#xA;" OwnerUserId="30829" LastActivityDate="2019-10-27T09:03:51.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16101" PostTypeId="1" CreationDate="2019-10-27T12:52:22.367" Score="2" ViewCount="55" Body="&lt;p&gt;I was trying to implement the loss function of  &lt;a href=&quot;https://yanweifu.github.io/papers/hairstyle_v_14_weidong.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;H-GAN&lt;/a&gt;. Here is my &lt;a href=&quot;http://codepad.org/9C3Vac25&quot; rel=&quot;nofollow noreferrer&quot;&gt;code &lt;/a&gt;. But it seem somethings wrong, maybe is recognition loss on z (EQ 9). I used the EQ 5 on &lt;a href=&quot;https://arxiv.org/pdf/1902.03938.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MISO&lt;/a&gt; to calculate it. Here is my code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def recognition_loss_on_z(self,latent_code, r_cont_mu, r_cont_var):&#xA;    eplison = (r_cont_mu - latent_code) / (r_cont_var+1e-8)&#xA;    return -tf.reduce_mean(tf.reduce_sum(-0.5*tf.log(2*np.pi*r_cont_var+1e-8)-0.5*tf.square(eplison), axis=1))/(config.batch_size * config.latent_dim)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I calculated loss function: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    self.z_mean, self.z_sigm = self.Encode(self.images)&#xA;    self.z_x = tf.add(self.z_mean, tf.sqrt(tf.exp(self.z_sigm))*self.ep)&#xA;&#xA;    self.D_pro_logits, self.l_x_real, self.Q_y_style_given_x_real, continuous_mu_real, continuous_var_real  = self.discriminator(self.images, training=True, reuse=False)&#xA;    self.De_pro_tilde, self.l_x_tilde, self.Q_y_style_given_x_tidle, continuous_mu_tidle, continuous_var_tidle= self.discriminator(self.x_tilde, training=True, reuse = True)&#xA;    self.G_pro_logits, self.l_x_fake, self.Q_y_style_given_x_fake, continuous_mu_fake, continuous_var_fake = self.discriminator(self.x_p, training=True, reuse=True)&#xA;&#xA;    tidle_latent_loss = self.recognition_loss_on_z(self.z_x, continuous_mu_tidle,continuous_var_tidle)&#xA;    real_latent_loss  = self.recognition_loss_on_z(self.z_x, continuous_mu_real,continuous_var_real)&#xA;    fake_latent_loss =  self.recognition_loss_on_z(self.zp, continuous_mu_fake,continuous_var_fake)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And discriminator: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def discriminator(self, x_var,training=False, reuse=False):&#xA;    with tf.variable_scope(&quot;discriminator_recongnizer&quot;) as scope:&#xA;        if reuse==True:&#xA;            scope.reuse_variables()&#xA;        conv1 = tf.nn.leaky_relu(batch_normalization(conv2d(x_var, output_dim = 64 , kernel_size=6, name='dis_R_conv1'),training = training,name='dis_bn1', reuse = reuse), alpha =0.2)&#xA;        conv2 = tf.nn.leaky_relu(batch_normalization(conv2d(conv1, output_dim = 128 , kernel_size=4, name='dis_R_conv2'),training = training ,name='dis_bn2', reuse = reuse), alpha =0.2)&#xA;        conv3 = tf.nn.leaky_relu(batch_normalization(conv2d(conv2, output_dim = 128 , kernel_size=4, name='dis_R_conv3'),training = training,name='dis_bn3', reuse = reuse), alpha =0.2)&#xA;        conv4 = conv2d(conv3, output_dim = 256 , kernel_size=4, name='dis_R_conv4')&#xA;        lth_layer = conv4&#xA;        conv4 = tf.nn.leaky_relu(batch_normalization(conv4, training=training, name='dis_bn4', reuse = reuse),alpha =0.2)&#xA;        conv4 = tf.reshape(conv4,[-1, 256*8*8])&#xA;        #Discriminator&#xA;        with tf.variable_scope('discriminator'):&#xA;            d_output = fully_connect(conv4, output_size=1, scope='dr_dense_2')&#xA;        with tf.variable_scope('dis_q'):&#xA;            fc_r = tf.nn.leaky_relu(batch_normalization(fully_connect(conv4, output_size=256 + config.style_classes, scope='dis_dr_dense_3'), training=training, name='dis_bn_fc_r', reuse=reuse), alpha=0.2)&#xA;            continuous_mu = fully_connect(fc_r, output_size=256, scope='dis_dr_dense_mu')&#xA;            continuous_var = tf.exp(fully_connect(fc_r, output_size=256, scope='dis_dr_dense_logvar'))&#xA;            style_predict = fully_connect(fc_r, output_size=config.style_classes, scope='dis_dr_dense_y_style')&#xA;&#xA;        return d_output,lth_layer,style_predict,continuous_mu,continuous_var   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone have experience with that, please tell me where I was wrong. Thanks you so much, I really appreciate that!&lt;/p&gt;&#xA;" OwnerUserId="30832" LastEditorUserId="1671" LastEditDate="2019-10-29T16:28:40.477" LastActivityDate="2019-10-29T16:28:40.477" Title="How to implement loss function of H-GAN model" Tags="&lt;tensorflow&gt;&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16102" PostTypeId="5" CreationDate="2019-10-27T17:16:21.133" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-27T17:16:21.133" LastActivityDate="2019-10-27T17:16:21.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16103" PostTypeId="4" CreationDate="2019-10-27T17:16:21.133" Score="0" Body="For questions related to the exploding gradient problem, which is the numerical problem associated with the significant increase (or explosion) of the numbers of the gradient vector of an objective function with respect to the parameters of a neural network, which is being trained with a gradient-based optimization algorithm and backpropagation. There is also the related vanishing gradient problem, which arises when the numbers become very small." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-27T17:21:40.033" LastActivityDate="2019-10-27T17:21:40.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16104" PostTypeId="1" AcceptedAnswerId="16107" CreationDate="2019-10-27T19:58:56.447" Score="6" ViewCount="1338" Body="&lt;p&gt;I have an idea to find the optimal number of hidden neurons required in a neural network, but I'm not sure how accurate it is.&lt;/p&gt;&#xA;&lt;p&gt;Assuming that it has only 1 hidden layer, it is a classification problem with 1 output node (so it's a binary classification task), has N input nodes for N features in the data set, and every node is connected to every node in the next layer.&lt;/p&gt;&#xA;&lt;p&gt;I'm thinking that to ensure that the network is able to extract all of the useful relations between the data, then every piece of data must be linked to every other piece of data, like in a complete graph. So, if you have 6 inputs, there must therefore be 15 edges to make it complete. Any more and it will be recomputing previously computed information and any less will be not computing every possible relation.&lt;/p&gt;&#xA;&lt;p&gt;So, if a network has 6 input nodes, 1 hidden node, 1 output node. There will be 6 + 1 connections. With 6 input nodes, 2 hidden nodes, and 1 output node, there will be 12 + 2 connections. With 3 hidden nodes there will be 21 connections. Therefore, the hidden layer should have 3 hidden nodes to ensure all possibilities are covered.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/a/136542/82135&quot;&gt;This answer&lt;/a&gt; discusses another method. For the sake of argument, I've tried to keep both examples using the same data. If this idea is computed with 6 input features, 1 output node, &lt;span class=&quot;math-container&quot;&gt;$\alpha = 2$&lt;/span&gt;, and 60 samples in the training set, this would result in a maximum of 4 hidden neurons. As 60 samples is very small, increasing this to 600 would result in a maximum of 42 hidden neurons.&lt;/p&gt;&#xA;&lt;p&gt;Based on my idea, I think there should be at most 3 hidden nodes and I can't imagine anymore being useful, but would there be any reason to go beyond 3 and up to 42, like in the second example?&lt;/p&gt;&#xA;" OwnerUserId="11795" LastEditorUserId="2444" LastEditDate="2021-01-11T00:37:23.153" LastActivityDate="2021-01-11T00:37:23.153" Title="Is this idea to calculate the required number of hidden neurons for a single hidden layer neural network correct?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;hyperparameter-optimization&gt;&lt;hidden-layers&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16105" PostTypeId="2" ParentId="16099" CreationDate="2019-10-27T20:40:58.780" Score="2" Body="&lt;p&gt;I'll recommend two sources:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The venerable &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt; book, which is a common text in AI courses. Russell &amp;amp; Norvig &lt;em&gt;end&lt;/em&gt; each chapter with a summary of the history of the developments of the techniques they have just discussed. These sections are often skipped by novice readers, but are almost exactly what you are looking for. The ones in the back half of the book should, together, give you a good sense of the order in which developments occurred, why techniques were developed, and what advances techniques enabled.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;R&amp;amp;N does a good job of covering &lt;em&gt;what&lt;/em&gt; happened, but not always &lt;em&gt;why&lt;/em&gt;. For that, you want a philosophy of AI book. I recommend &lt;a href=&quot;https://mitpress.mit.edu/books/mind-design-ii&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mind Design II&lt;/a&gt; as a starting place. This book is a chronologically organized collection of papers and academic essays by the big thinkers in philosophy of mind and in AI research. Often, the papers are responses to one another. With some side reading about the history of each author, you can begin to get a good sense of the big philosophical movements in the field over the last 70 years, and why things have changed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you don't want to read the book, I can give you a summary (spoilers ahead!):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;1920's: behaviorists, and others, propose that Mind = Brain, and specifically focus on intelligent behavior.&lt;/li&gt;&#xA;&lt;li&gt;1950: Turing proposes that a computer could be programmed to exhibit intelligent behavior.&lt;/li&gt;&#xA;&lt;li&gt;1960's: Cognativists in AI, Psych, and Linguistics argue that behavior is not enough. Minds &lt;em&gt;think&lt;/em&gt;, and thought takes the form of reasoned algorithms. The lynchpin of their argument is domains like language understanding, which they claim cannot be modeled without logic. Their work produces search algorithms, and early rule-based planning and language systems.&lt;/li&gt;&#xA;&lt;li&gt;1970's Searle argues that computers can't think because of &lt;a href=&quot;https://en.wikipedia.org/wiki/Phenomenology_(philosophy)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phenomenology&lt;/a&gt;. Most AI folk ignore him, and go on working, but Philosophy &amp;amp; Psych take greater notice.&lt;/li&gt;&#xA;&lt;li&gt;1970's Dryfus (and others) argue compelling that logic and reason cannot explain human thought, by dissecting the rule based systems of the day through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Frame Problem&lt;/a&gt;. AI researchers take some notice.&lt;/li&gt;&#xA;&lt;li&gt;The Connectionists, especially Hinton (AI) &amp;amp; Churchland (Philosophy) propose the first post-Cognativist theories of mind. These focus on the idea that Mind is in the Connections (specifically, the firing patterns) of neurons, not in the brain itself. This view of minds spurs (re)-development of Neural Networks. In the 1980's, this is mostly ignored.&lt;/li&gt;&#xA;&lt;li&gt;During the 1990's, Connectionists, and others working on probabilistic methods in AI demonstrate systems for language that are substantially better than rule based methods. Cognativists begin to decline, because the domains they claimed needed reasoned algorithms are actually better handled by statistical algorithms. Connectionist views gain support in AI and elsewhere.&lt;/li&gt;&#xA;&lt;li&gt;Today, with rising computational power, statistical techniques come to the forefront of many AI domains. Simultainiously, the Churchlands, Rodney Brooks, and others propose further Post-Cognativist schools of thought based around dynamical systems theory and embodied cognition, which were somewhat influential in robotics. Cognativism continues to enjoy some support within the AI, Psych, and Linguistics communities, but this is much diminished. Some hybrid systems use a mix of statistical and rule-based techniques.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-27T20:40:58.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16106" PostTypeId="1" AcceptedAnswerId="16117" CreationDate="2019-10-27T20:53:06.633" Score="4" ViewCount="521" Body="&lt;p&gt;How do i calculate weights for max pooling output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if there are 10 inputs, a pooling filter of size and a stride 2, how many weights including bias are required for the max pooling output ?&lt;/p&gt;&#xA;" OwnerUserId="30843" LastEditorUserId="30843" LastEditDate="2019-10-27T21:37:50.090" LastActivityDate="2019-10-28T10:13:21.247" Title="Calculating Weights for CNN Max Pooling Output" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16107" PostTypeId="2" ParentId="16104" CreationDate="2019-10-27T20:56:18.747" Score="7" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have an idea to find the optimal number of hidden neurons required in a neural network but I'm not sure how accurate it is.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's a complete non-starter, and there is a no such calculation possible in the general case (real-valued inputs to a neural network). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even with one input neuron it is not possible. That is because even with one input, the output can be an arbitrarily complex mapping to classes. A good example with two inputs that would require an infinite number of hidden neurons to supply a simple classifier would be classifying x,y points as being in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mandelbrot_set&quot; rel=&quot;noreferrer&quot;&gt;Mandelbrot set&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some, more constrained, examples, with well-defined functions, you can construct a minimal neural network that solves the problem perfectly. For instance a neural network model of XOR can be made with two hidden neurons (and six links). However, this kind of analysis is limited to simple problems. You might be able to come up with some variation of your idea if all inputs were boolean, and the neural network limited to some combined bitwise logic on all the inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your idea of matching number of edges to number of possible interactions between inputs does not work because you are only considering the most basic kind of interaction between two variables, whilst variables can in practice combine in all sorts of ways to form a function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, each neuron in a hidden layer works with a linear weighted sum, plus a fixed transformation function. This is in no way guaranteed to match the function shape that you are trying to approximate with the neural network. An analogy that you might be aware of is discrete Fourier transforms - it is possible to model any part of a function by combine sine and cosine waves of different frequencies, but some functions will require many such waves in order to be represented accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your link to the answer in Cross Validated Stack Exchange gives you a rule of thumb that the writers find often works with the kinds of data that they work with. This is useful experience. You can use such rules as the starting point for searching for architecture that works on your problem. This will likely be more useful than your idea based on counting the possible variable interactions. However, in both cases, the most important step is to perform a test with some unseen examples, and to &lt;em&gt;search&lt;/em&gt; for the best neural network architecture for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are things you can do with variable interactions though. For instance, try looking for linear correlations between simple polynomial combinations of variables and your target variable, e.g. plot &lt;span class=&quot;math-container&quot;&gt;$x_1 x_2$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$x_3^2 x_4$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; . . . you may find some combinations have a clear signal implying a relationship. Take care if you do this sort of thing though, if you test very many of these, you &lt;em&gt;will&lt;/em&gt; find a linear relationship purely by chance that looks good initially but turns out to be a dud when testing (it's a form of overfitting). So you should generally test a lot less than the size of your dataset, and limit yourself to some modest maximum total power.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-10-27T20:56:18.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16108" PostTypeId="1" CreationDate="2019-10-27T21:26:55.173" Score="4" ViewCount="65" Body="&lt;p&gt;The question is broad, since I am looking for any kind of insight into the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently looking into the possibility for useing inverse reinforcement learning for learning robotic manipulation tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I have gathered so far, it seems that the approach of IRL has been applied with success primarily to game- and small-action-space physical problems.&lt;/p&gt;&#xA;" OwnerUserId="30565" LastActivityDate="2019-10-27T21:26:55.173" Title="What is the current state-of-the-art for inverse reinforcement learning on robotic control tasks?" Tags="&lt;reinforcement-learning&gt;&lt;state-of-the-art&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16109" PostTypeId="1" AcceptedAnswerId="16110" CreationDate="2019-10-28T00:25:52.887" Score="4" ViewCount="65" Body="&lt;p&gt;Did I get it right, that RNNs most often have just one hidden neuron layer? Is there a reason for that? Will RNNs with several hidden layers in each cell work worse?&#xA;Thank you!! &lt;/p&gt;&#xA;" OwnerUserId="30851" LastActivityDate="2019-10-28T02:44:12.723" Title="Why RNNs often use just one hidden layer?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16110" PostTypeId="2" ParentId="16109" CreationDate="2019-10-28T02:44:12.723" Score="3" Body="&lt;p&gt;Definitely you can have multiple hidden layers in RNN. One the most common approaches to determine the hidden units is to start with a very small network (one hidden unit) and apply the K-fold cross validation ( k over 30 will give very good accuracy) and estimate the average prediction risk. Then you will have to repeat the procedure for increasing growing networks, for example for 1 to 10 hidden units or more if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my experience, if you are interested to get best possible accuracy, you should start with small number of hidden layers and more simple structure, and if you are not satisfied with the corresponding accuracy, then we should go on increasing the learning rate by fixed but small small steps and each time start training fresh. &lt;/p&gt;&#xA;" OwnerUserId="30799" LastActivityDate="2019-10-28T02:44:12.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16111" PostTypeId="1" CreationDate="2019-10-28T07:47:59.513" Score="4" ViewCount="119" Body="&lt;p&gt;The mean episodic reward is generally increasing, but it has spontaneous drops, and I'm not sure of their cause.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NGQeV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NGQeV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem has a sparse reward, batch &lt;code&gt;size=2000&lt;/code&gt;, &lt;code&gt;entropy_coefficient=0.1&lt;/code&gt;, other hyper-parameters are pretty standard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone seen this kind of behavior? What could be the cause these drops in the reward(not enough exploration, too sparse rewards, the state not expressive enough, etc.)?&lt;/p&gt;&#xA;" OwnerUserId="28125" LastEditorUserId="28125" LastEditDate="2019-11-06T21:51:25.573" LastActivityDate="2021-03-30T23:01:55.217" Title="What could be the cause of the drop in the reward in A3C?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;rewards&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16112" PostTypeId="1" AcceptedAnswerId="16116" CreationDate="2019-10-28T09:08:49.590" Score="1" ViewCount="129" Body="&lt;p&gt;I'm just started to learn about meta learning and CNN and in most paper that I've read they mention to have one CNN to feature extraction. These features will help the another network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what is feature extraction (I don't know what are those features) but I'm wondering if I can use it on image segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to use the first network to feature extraction without doing image classification, and pass those features to the other network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;How can I use feature extraction in CNN on image segmentation?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastActivityDate="2019-11-27T11:02:20.513" Title="How can I use feature extraction in CNN with image segmentation?" Tags="&lt;convolutional-neural-networks&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16113" PostTypeId="2" ParentId="12529" CreationDate="2019-10-28T09:23:15.560" Score="1" Body="&lt;p&gt;Unfortunately no, the way to go is track the total reward and see if it's increasing and converging eventually. Value loss isn't a useful metric as the loss can be 0 when the value network always predicts 0 and the agent doesn't collect any reward, meaning very poor behavior.&lt;/p&gt;&#xA;" OwnerUserId="28125" LastActivityDate="2019-10-28T09:23:15.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16114" PostTypeId="2" ParentId="16087" CreationDate="2019-10-28T09:25:18.977" Score="3" Body="&lt;p&gt;The double sampling problem is referenced in Chaper 11.5 &lt;em&gt;Gradient Descent in the Bellman Error&lt;/em&gt; in &lt;a href=&quot;http://incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction (2nd edition)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From the book, this is the full gradient descent (as opposed to semi-gradient descent) update rule for weights of an estimator that should converge to a minimal distance from the Bellman error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w_{t+1} = w_t + \alpha[\mathbb{E}_b[\rho_t[R_{t+1} + \gamma\hat{v}(S_{t+1},\mathbf{w})] - \hat{v}(S_{t},\mathbf{w})][\nabla\hat{v}(S_{t},\mathbf{w})- \gamma\mathbb{E}_b[\rho_t\nabla\hat{v}(S_{t+1},\mathbf{w})]]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;[...] But this is naive, because the equation above involves the next state, &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;, appearing in two&#xA;expectations that are multiplied together. To get an unbiased sample of the product,&#xA;two independent samples of the next state are required, but during normal interaction&#xA;with an external environment only one is obtained. One expectation or the other can be&#xA;sampled, but not both.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Basically, unless you have an environment that you can routinely re-wind and re-sample to get two independent estimates (for &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(S_{t+1},\mathbf{w})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla\hat{v}(S_{t+1},\mathbf{w})$&lt;/span&gt;) then the update rule that naturally arises from gradient descent on the Bellman error does will work any better than other approaches, such as semi-gradient methods. If you can do this rewind process on every step, then it may be worth it because of the guarantees of convergence, even in off-policy with non-linear approximators.&lt;/p&gt;&#xA;&lt;p&gt;The paper proposes a workaround for this issue, keeping the robust convergence guarantees, but dropping the need to collect two independent samples of the same estimate on each step.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-10-28T09:25:18.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16115" PostTypeId="2" ParentId="12008" CreationDate="2019-10-28T09:26:18.530" Score="0" Body="&lt;p&gt;Just use one class inheriting from &lt;code&gt;nn.Module&lt;/code&gt; called e.g. &lt;code&gt;ActorCriticModel&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, have two members called &lt;code&gt;self.actor&lt;/code&gt; and &lt;code&gt;self.critic&lt;/code&gt; and define them to have the desired architecture.Then, in the &lt;code&gt;forward()&lt;/code&gt; method return two values, one for the actor output (which is a vector) and one for the critic value (which is a scalar). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you can use only one optimizer.&lt;/p&gt;&#xA;" OwnerUserId="28125" LastActivityDate="2019-10-28T09:26:18.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16116" PostTypeId="2" ParentId="16112" CreationDate="2019-10-28T10:10:53.920" Score="0" Body="&lt;p&gt;Feature extraction is a way that people use pretrained model to extract information from input data. For example, image segmentation task may use the VGG network or other image classifying network for feature extraction. The output of the last convolution layer is taken. Then, the features are feed into the untrained network to get outputs. The bottom network for image segmentation usually consists of upsampling and convolutional layers. Then output of size of original image is resulted in teh main network. Hope I can help you&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-28T10:10:53.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16117" PostTypeId="2" ParentId="16106" CreationDate="2019-10-28T10:13:21.247" Score="1" Body="&lt;p&gt;A Max Pool layer don't have any trainable weights. Only hyperparameters is present and they are non-trainable. The max pooling process calculates the maximum value of the filter, which consists of no weights and biases. It is purely a way to down scale the data to a smaller dimension. Hope this helps you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-28T10:13:21.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16119" PostTypeId="1" CreationDate="2019-10-28T12:47:22.353" Score="1" ViewCount="115" Body="&lt;p&gt;Which model is the most appropriate for this problem with multiple inputs and outputs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data set is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A1, A2, A3, A4, A5, A6, B1, B2, B3, B4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;A1, A2, A3, A4, A5, A6&lt;/code&gt; are the inputs and &lt;code&gt;B1, B2, B3, B4&lt;/code&gt; the outputs (this is what I want the model to predict).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What an LSTM be appropriate for this task? Any advice or hint would be much appreciated. Also if anyone can share already done examples, it would really help me a lot.&lt;/p&gt;&#xA;" OwnerUserId="30642" LastEditorUserId="2444" LastEditDate="2019-11-04T14:20:24.813" LastActivityDate="2021-03-28T19:00:40.070" Title="Which model can I use for this problem with multiple inputs and outputs?" Tags="&lt;ai-design&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16120" PostTypeId="1" AcceptedAnswerId="16122" CreationDate="2019-10-28T14:21:13.480" Score="3" ViewCount="119" Body="&lt;p&gt;I am trying to predict crime. I have data with factors: location, keyword description of the crime, time crime occurred and so on. This is for crimes that occurred in the past. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to treat the prediction of crimes as a binary classification problem. In this model, the data I have collected would form the &quot;positive&quot; examples: they are all examples of a crime happening. However, I am unsure what to use for the negative examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, most of the time there is no crime at the location, but can I use this as negative data? For example, if I know there was a crime at 7pm at location X, and no other crimes there, should I generate new negative data points for every hour &lt;em&gt;except&lt;/em&gt; 7pm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I want to create probabilities of crime based on a set of factors.&lt;/p&gt;&#xA;" OwnerUserId="8385" LastEditorUserId="16909" LastEditDate="2019-10-29T13:55:01.713" LastActivityDate="2019-10-30T04:21:19.317" Title="How do I predict the occurrence of rare events?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;applications&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16121" PostTypeId="2" ParentId="11103" CreationDate="2019-10-28T14:38:35.653" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/openai/retro&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/retro&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current list of machines is &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Atari&lt;/li&gt;&#xA;&lt;li&gt;Atari2600 (via Stella)&lt;/li&gt;&#xA;&lt;li&gt;TurboGrafx-16/PC Engine (via Mednafen/Beetle PCE Fast)&lt;/li&gt;&#xA;&lt;li&gt;Game Boy/Game Boy Color (via gambatte)&lt;/li&gt;&#xA;&lt;li&gt;Game Boy Advance (via mGBA)&lt;/li&gt;&#xA;&lt;li&gt;Nintendo Entertainment System (via FCEUmm)&lt;/li&gt;&#xA;&lt;li&gt;Super Nintendo Entertainment System (via Snes9x)&lt;/li&gt;&#xA;&lt;li&gt;GameGear (via Genesis Plus GX)&lt;/li&gt;&#xA;&lt;li&gt;Genesis/Mega Drive (via Genesis Plus GX)&lt;/li&gt;&#xA;&lt;li&gt;Master System (via Genesis Plus GX)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a vague tutorial here for adding other systems&#xA;&lt;a href=&quot;https://github.com/openai/retro/issues/169&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/retro/issues/169&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30869" LastActivityDate="2019-10-28T14:38:35.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16122" PostTypeId="2" ParentId="16120" CreationDate="2019-10-28T15:37:54.613" Score="3" Body="&lt;p&gt;It might be more informative to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Label each combination of location, type, and time of crime with a crime &lt;em&gt;rate&lt;/em&gt;. For example, theft, in Crystal City, at 11pm at night, occurs 20 times per year, or 0.4 times per resident per year.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Predict the crime &lt;em&gt;rate&lt;/em&gt;, rather than individual events.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This avoids the need to have explicit examples of &quot;non-crime&quot;, and lets you instead directly learn something related to the probabilities of crimes being committed (the rate).&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-28T15:37:54.613" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16123" PostTypeId="2" ParentId="16080" CreationDate="2019-10-28T15:49:50.953" Score="3" Body="&lt;p&gt;U-Net and U-Net inspired architectures have been quite popular in the medical image-related tasks ever since it was first introduced. There have been several improved versions of U-Net designed for specific tasks that followed. One such example is &lt;a href=&quot;https://arxiv.org/abs/1804.03999v3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention U-Net&lt;/a&gt;, extremely popular for Pancreas Segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other examples of architectures that have achieved state-of-the-art results in image segmentation tasks in recent years include &lt;a href=&quot;https://arxiv.org/abs/1603.05959v3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Scale 3DCNN + CRF&lt;/a&gt;, popular for Brain and Lesion images, &lt;a href=&quot;https://arxiv.org/abs/1906.02849&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Scale Attention&lt;/a&gt; for MRIs, etc. A recent paper that describes an interesting 3D FCNN architecture is &lt;a href=&quot;https://arxiv.org/abs/1804.02967v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;HyperDense-Net&lt;/a&gt;, widely used for multi-modal tasks in medical image segmentation.&lt;/p&gt;&#xA;" OwnerUserId="30871" LastActivityDate="2019-10-28T15:49:50.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16124" PostTypeId="1" AcceptedAnswerId="16126" CreationDate="2019-10-28T15:52:31.180" Score="5" ViewCount="82" Body="&lt;p&gt;I am reading &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goodfellow's&lt;/a&gt; book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this algorithm shown step-by-step, but now I would like to understand the &lt;strong&gt;matrix calculus&lt;/strong&gt; (so not basic calculus!), that is, calculus with matrices and vectors, but especially everything related to the derivatives with respect to a matrix or a vector, and so on.&lt;/p&gt;&#xA;&lt;p&gt;Which math book could you advise me to read?&lt;/p&gt;&#xA;&lt;p&gt;I specify I studied 2 years after the bachelor in math school (in French: mathématiques supérieures et spéciales), but did not practice for years.&lt;/p&gt;&#xA;" OwnerUserId="30870" LastEditorUserId="2444" LastEditDate="2021-01-22T01:01:10.733" LastActivityDate="2021-01-22T01:02:50.043" Title="Which linear algebra book should I read to understand vectorized operations?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;linear-algebra&gt;&lt;books&gt;&lt;calculus&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16125" PostTypeId="2" ParentId="16124" CreationDate="2019-10-28T16:30:47.773" Score="1" Body="&lt;p&gt;Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2019-10-28T16:30:47.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16126" PostTypeId="2" ParentId="16124" CreationDate="2019-10-28T17:29:22.603" Score="3" Body="&lt;p&gt;If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of neural nets (in part 1), and then actually goes through the various models in detail in the later parts. The review is done at a level that is suitable for someone who has already studied these topics, but needs a refresher.&lt;/p&gt;&#xA;&lt;p&gt;The book &lt;a href=&quot;https://www.janmagnus.nl/misc/mdc-ch18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix Differential Calculus with Applications in Statistics and Econometrics&lt;/a&gt; covers more advanced topics, which might also be what you are looking for. There is also &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2021-01-22T01:02:50.043" LastActivityDate="2021-01-22T01:02:50.043" CommentCount="1" CommunityOwnedDate="2021-01-22T00:28:07.810" ContentLicense="CC BY-SA 4.0" />
  <row Id="16127" PostTypeId="2" ParentId="16045" CreationDate="2019-10-28T19:31:25.873" Score="5" Body="&lt;p&gt;Let us suppose we have a network without any functions in between. Each layer consists of a linear function. i.e&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;layer_output = Weights.layer_input + bias&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Consider a 2 layer neural network, the outputs from layer one will be:&#xA;    x2 = W1*x1 + b1&#xA;Now we pass the same input to the second layer, which will be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x3 = W2x*2 + b2&#xA;Also x2 = W1*x1 + b1&#xA;Substituting back, we have:&#xA;x3 = W2(W1*x1 + b1) + b2&#xA;x3 = (W2W1)*x1 + (W2*b1 + b2)&#xA;x3 = W*x1 + b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Oh no! We still got a linear function. No matter how many layers we add, we will still get a linear function. In that case, our network will never be able to approximate any non linear functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will simply add some non linear functions in between. These functions are called activation functions. Some of these functions include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReLU&lt;/li&gt;&#xA;&lt;li&gt;Sigmoid&lt;/li&gt;&#xA;&lt;li&gt;tanh&lt;/li&gt;&#xA;&lt;li&gt;Softmax&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and there are a lot more of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yay! Our network is no more linear!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a lot of different non linear functions, and each of them serve a different purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, ReLU is simple and computationally cheap.&#xA;    ReLU(x) = max(0, x)&#xA;Sigmoid outputs are between 0 and 1.&#xA;tanh is similar to sigmoid, but zero centered, with outputs from -1 to 1&#xA;Softmax is usually used if you want to represent any vector as a discrete probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope you are having a great day!&lt;/p&gt;&#xA;" OwnerUserId="21229" LastActivityDate="2019-10-28T19:31:25.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16128" PostTypeId="1" AcceptedAnswerId="16129" CreationDate="2019-10-28T20:16:29.790" Score="2" ViewCount="342" Body="&lt;p&gt;I've just started to learn CNN and somewhere I have read if I remove the last FCL I will get the features extracted from the input image but... what are those features?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are they numbers? Labels? An image location (x,y) where there is a line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use these features on a one shot network, but I can't imagine how to use them if I don't know what they are.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2019-10-29T06:57:17.230" LastActivityDate="2019-10-29T06:57:17.230" Title="What are the features get from a feature extraction using a CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;feature-extraction&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16129" PostTypeId="2" ParentId="16128" CreationDate="2019-10-28T22:46:00.693" Score="2" Body="&lt;p&gt;You get what we call &lt;strong&gt;high-level features&lt;/strong&gt;, which are basically abstract representations of the parts that carry information in the image you want to classify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine you want to classify a car. The image you feed your network could be a car on a road with a driver and trees and clouds, etc. The network, however, if you've trained it to recognize cars, will try to focus on parts of the image regarding a car. The final layers will have learned to extract an abstract representation of a car from the image (this means a low-resolution car-like shape). Now your final FC layers will try to classify the image from these high-level features. In this example, you would have an FC layer that learns to classify a car if this this abstract car-like figure is present in the image. Likewise, if it isn't present it won't classify it as a car. By accessing these high-level features, you essentially have a more compact and meaningful representation of what the image represents (based always on the classes that the CNN has been trained on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By visualizing the activations of these layers we can take a look on what these high-level features look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://images.duckduckgo.com/iu/?u=https%3A%2F%2Fi.stack.imgur.com%2FHl2H6.png&amp;amp;f=1&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The top row here is what you are looking for: the high-level features that a CNN extracts for four different image types.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-10-28T22:46:00.693" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16130" PostTypeId="2" ParentId="12390" CreationDate="2019-10-28T23:53:10.360" Score="0" Body="&lt;p&gt;I agree a neural net is a good start and you might want to add a constitutional neural network to list of models you want to test and evaluate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However your question was really on getting something up and running using python. I don't have enough time for me to play games, much more let my computers do it for me... but I know &lt;a href=&quot;https://github.com/NJacobsohn/Your-Learning-is-in-Another-Castle#the-numerical-model&quot; rel=&quot;nofollow noreferrer&quot;&gt;someone who has&lt;/a&gt; done such a project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully that helps.&lt;/p&gt;&#xA;" OwnerUserId="30750" LastActivityDate="2019-10-28T23:53:10.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16131" PostTypeId="1" AcceptedAnswerId="16144" CreationDate="2019-10-29T01:23:30.953" Score="2" ViewCount="128" Body="&lt;p&gt;&lt;strong&gt;This is the problem&lt;/strong&gt;: I have 66 slot-machines and for each of them I have 7 possible actions/arms to choose from. At each trial, I have to choose one of 7 actions for each and every one of the 66 slots. The reward depends on the combination of these actions but the slots are not equal, that is, pulling the same arm for different slots gives different results. I do not care about an initial state or feature-vector as the problem always starts from the same setting (it is not contextual). My reward depends on how I pull one of the 7 arms of all of the 66 bandits simultaneously, where, as said, each slots has its own unique properties towards the calculation of the total reward. Basically, the action space is a one-hot encoded 66x7 matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My solution&lt;/strong&gt;: I ignored the fact that I do not care about a feature vector or state and I treated the problem using a deep NN with a basic policy-gradient algorithm, where I increase directly the probability of each action depending on the reward I get. The state simply does not change, so the NN receive always the same input. This solution does work effectively in finding an approximately optimal strategy, however, it is very computationally expensive and something tells me I am overkilling the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not see how I could apply standard solutions to MAB such as epsilon-greedy and so on. I need simultaneity between the different &quot;slot-machines&quot; and if I just take each possible permutation as a different action, in order to explore them with greedy methods, I get way too many actions (in the order of 10^12). I have not found in the literature something similar to this multi-armed multi-bandit problem and I am clueless if anything like that has ever been considered - perhaps I am overthinking it and this can be somehow reduced to a normal MAB? &lt;/p&gt;&#xA;" OwnerUserId="23638" LastEditorUserId="2444" LastEditDate="2020-04-15T20:18:09.393" LastActivityDate="2020-04-15T20:18:09.393" Title="Solving a Multi-Armed, &quot;Multi-Bandit&quot; Problem" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;policy-gradients&gt;&lt;multi-armed-bandit&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16132" PostTypeId="2" ParentId="15977" CreationDate="2019-10-29T04:59:12.923" Score="2" Body="&lt;p&gt;One way to view a neural network is as a series of linear transformations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You take a bunch of data points and look at it from a different perspective from a different space. You apply some non linear function on the data points like, ReLU, sigmoid etc. Now you repeat the same process of looking from a different space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our goal is to look at it from a point where things starts looking right for our tasks. These linear transformations is what the network has to optimise. &lt;/p&gt;&#xA;" OwnerUserId="21229" LastActivityDate="2019-10-29T04:59:12.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16133" PostTypeId="1" AcceptedAnswerId="16833" CreationDate="2019-10-29T05:25:21.840" Score="4" ViewCount="4679" Body="&lt;p&gt;I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its &lt;strong&gt;hidden states&lt;/strong&gt;. In my specific case, the hidden state of the encoder is passed to the decoder, and this would allow the model to learn better latent representations.&lt;/p&gt;&#xA;&lt;p&gt;Does this make sense?&lt;/p&gt;&#xA;&lt;p&gt;I am a bit confused about this because I really don't know what the hidden state is. Moreover, we're using separate LSTMs for the encoder and decoder, so I can't see how the hidden state from the encoder LSTM can be useful to the decoder LSTM because only the encoder LSTM really understands it.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2021-01-17T16:43:36.097" LastActivityDate="2021-01-17T16:43:36.097" Title="What exactly is a hidden state in an LSTM and RNN?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;hidden-layers&gt;&lt;seq2seq&gt;&lt;encoder-decoder&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16134" PostTypeId="2" ParentId="15977" CreationDate="2019-10-29T06:19:31.340" Score="0" Body="&lt;p&gt;A good way of looking at it would be understanding neural networks mathematically, i.e. purely on the basis of the fact that you're just trying to fit a function and solve an optimisation problem (&lt;em&gt;apart from looking at it as multiple units of logistic regression&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say we want to approximate a function &lt;span class=&quot;math-container&quot;&gt;$y =f_w(x)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$x \in D$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is our domain-space. We want this function to map to &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, our co-domain, with all the values the function ends up taking being the set &lt;span class=&quot;math-container&quot;&gt;$y \in R$&lt;/span&gt;, our range. Essentially we frame &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; as a sequence of operations (&lt;em&gt;What operation should be done where is got from common-practice, intuition, and insight mostly gained from experience&lt;/em&gt;) assuming that when the right parameters are used for these operations we will arrive at a very reasonable approximation of the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We initialise the parameters with whatever values we want initially (&lt;em&gt;usually random&lt;/em&gt;), calling this parameter-space &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;. The essential idea would be frame another function &lt;span class=&quot;math-container&quot;&gt;$L(f_w(x), \hat{y})$&lt;/span&gt; called the loss function which we want to minimise. This acts as a test to how good our function is - since our function parameters were initially random, the error between the function approximations and the actual range values for known points (training set) are estimated. These estimated error values and its gradient is then used by back-propagation where &lt;span class=&quot;math-container&quot;&gt;$w_{init}\in W$&lt;/span&gt; is updated to another &lt;span class=&quot;math-container&quot;&gt;$w_{1}\in W$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; is calculated by moving on &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; in the direction of decreasing gradient, in hopes of reaching the loss functions minima.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simplifying, essentially all you want to do is find a &lt;span class=&quot;math-container&quot;&gt;$y=f_w(x)$&lt;/span&gt; where parameters &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; are to be chosen such that &lt;span class=&quot;math-container&quot;&gt;$L(f_w(x), \hat{y})$&lt;/span&gt; is minimised for the training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though this is a very rough idea of neural networks, such a direction in thinking can especially be useful when studying generative networks and other problems where the problem has to be formulated mathematically before being able to approach it. &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-10-29T06:19:31.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16135" PostTypeId="2" ParentId="16133" CreationDate="2019-10-29T06:34:12.777" Score="2" Body="&lt;p&gt;As you said, one way to look at it is definitely that the LSTM-encoder's encoding can be only understood by itself, that's why the decoder exists there. An optimisation process encoded it, why couldn't an optimisation process decode it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hidden state is essentially just an encoding of the information you gave it keeping the time-dependencies in check. Most encoder-decoder networks are trained end to end meaning, when the encoding is learned a corresponding decoding is learned simultaneously to decode the encoded latent in your desired format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend you read &lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; on how transformer models are used to convert French to English, as it would give you better intuition and understanding on what happens with encoder-decoder sequence models&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-10-29T06:34:12.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16136" PostTypeId="1" AcceptedAnswerId="16139" CreationDate="2019-10-29T07:12:03.703" Score="4" ViewCount="316" Body="&lt;p&gt;REINFORCE is a Monte Carlo policy gradient algorithm, which updates weights (parameters) of policy network by generating episodes. Here's a pseudo-code from Sutton's book (which is same as the equation in Silver's RL note):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QJFB2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QJFB2.png&quot; alt=&quot;reinforce&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to implement this with my own problem, I found something strange. Here's implementation from &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch's official GitHub&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def finish_episode():&#xA;    R = 0&#xA;    policy_loss = []&#xA;    returns = []&#xA;    for r in policy.rewards[::-1]:&#xA;        R = r + args.gamma * R&#xA;        returns.insert(0, R)&#xA;    returns = torch.tensor(returns)&#xA;    returns = (returns - returns.mean()) / (returns.std() + eps)&#xA;    for log_prob, R in zip(policy.saved_log_probs, returns):&#xA;        policy_loss.append(-log_prob * R)&#xA;    optimizer.zero_grad()&#xA;    policy_loss = torch.cat(policy_loss).sum()&#xA;    policy_loss.backward()&#xA;    optimizer.step()&#xA;    del policy.rewards[:]&#xA;    del policy.saved_log_probs[:]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I feel like there's a difference between the above two. In Sutton's pseudo-code, the algorithm updates &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; for &lt;strong&gt;each step&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, while the second code (PyTorch's one) accumulate loss and update &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; with the summation, i.e. &lt;strong&gt;after each episode&lt;/strong&gt;. &#xA;I tried to search other implementation of REINFORCE, and I found that most of the implementations follow the second form, update after each generated episodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check whether both give the same result, I changed the second code as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def finish_episode():&#xA;    R = 0&#xA;    policy_loss = []&#xA;    returns = []&#xA;    for r in policy.rewards[::-1]:&#xA;        R = r + args.gamma * R&#xA;        returns.insert(0, R)&#xA;    returns = torch.tensor(returns)&#xA;    returns = (returns - returns.mean()) / (returns.std() + eps)&#xA;    for log_prob, R in zip(policy.saved_log_probs, returns):&#xA;        optimizer.zero_grad()&#xA;        loss = -log_prob * R&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and run it, which gives different result (if my code has no problem). &#xA;So they are not the same, and I think the last one is more close to the original pseudo-code of REINFORCE. What am I missing now? Is it okay because the results are approximately same? (I'm not sure about this claim)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in some sense, I think Pytorch's implementation is the &lt;em&gt;right&lt;/em&gt; version of REINFORCE. In Sutton's pseudo-code, episode is generated first, so I think &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; shouldn't be updated at each step and should be updated after the total loss is computed. If &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is updated at each step, then such &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; might be different with the original &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; that used to generate the episode.  &lt;/p&gt;&#xA;" OwnerUserId="30886" LastActivityDate="2019-10-29T11:01:40.100" Title="Update in REINFORCE algorithm - step-wise or episode-wise?" Tags="&lt;reinforcement-learning&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16137" PostTypeId="2" ParentId="12989" CreationDate="2019-10-29T07:48:24.080" Score="1" Body="&lt;p&gt;First of all, when you add hidden layers, or stack RBMs, you get a Deep Belief Network (DBN). Your question then deals with the comparison of DBNs and RBMs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some elements to answer this question in the article &lt;a href=&quot;http://www.iro.umontreal.ca/~lisa/pointeurs/rbm_universality_techreport.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Representational Power of Restricted Boltzmann Machines and Deep Belief Networks&lt;/a&gt; by Nicolas Le Roux, which can be found summarized in these &lt;a href=&quot;http://swoh.web.engr.illinois.edu/courses/IE598/handout/fall2016_slide1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;course slides&lt;/a&gt;. The main results are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Restricted Boltzmann Machines:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;Increasing the number of hidden units improves representational ability.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;With an unbounded number of units, any distribution over &lt;span class=&quot;math-container&quot;&gt;$\{0,1\}^n$&lt;/span&gt; can be approximated arbitrarily well.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Deep Belief Networks:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;strong&gt;Adding additional layers using greedy contrastive divergence training does not provide additional benefit&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;There remain open questions about the benefits additional layers add.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I emphasized the &lt;span class=&quot;math-container&quot;&gt;$4^{th}$&lt;/span&gt; point that covers the most your question. It does not mean that additional layers are useless, but since RBM are universal approximators (&lt;span class=&quot;math-container&quot;&gt;$2^{nd}$&lt;/span&gt; point), the benefits of adding layers seem less straightforward. They seem dependent on the first layer, the training procedure... You can see the article for more details and the open questions raised about DBNs. Note that this answer is an entry point on the topic, there might be more recent results following this article which I'm not familiar with...&lt;/p&gt;&#xA;" OwnerUserId="25825" LastEditorUserId="2444" LastEditDate="2019-10-29T13:38:04.060" LastActivityDate="2019-10-29T13:38:04.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16138" PostTypeId="1" CreationDate="2019-10-29T08:20:18.053" Score="3" ViewCount="66" Body="&lt;p&gt;I am working on classifying the &lt;a href=&quot;https://github.com/brendenlake/omniglot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Omniglot dataset&lt;/a&gt;, and the different papers dealing with this topic describe the problem as &lt;em&gt;one-shot learning&lt;/em&gt; (classification). I would like to nail down a precise description of what counts as &lt;em&gt;one-shot learning&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's clear to me that in one-shot classification, a model tries to classify an input into one of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; classes by comparing it to exactly one example from each of the &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; classes.&lt;/p&gt;&#xA;&lt;p&gt;What I want to understand is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it necessary that the model has never seen the input and the target examples before, for the problem to be called one-shot?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Goodfellow et. al. describe one-shot learning as an extreme case transfer learning where only one labeled example of the transfer task is &lt;a href=&quot;http://www.deeplearningbook.org/contents/representation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;presented&lt;/a&gt;. So, it means they are considering the training process as a kind of continuous transfer learning? What has the model learned earlier, that is being transferred?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23273" LastEditorUserId="2444" LastEditDate="2020-12-14T11:11:10.400" LastActivityDate="2020-12-14T11:11:10.400" Title="Precise description of one-shot learning" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;transfer-learning&gt;&lt;one-shot-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16139" PostTypeId="2" ParentId="16136" CreationDate="2019-10-29T11:01:40.100" Score="2" Body="&lt;p&gt;Hi Seewoo Lee and welcome to our community!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essence of your observation is that Sutton's version of REINFORCE is taking into consideration all of the trajectory to compute the returns while in the pytorch version only the future is taken into consideration, hence going in reverse to sum the future rewards and ignore the previous rewards. The consequence is that future actions are not punished for early mistakes. The guys at OpenAI refer to this as reward-to-go but personally, I find that it resembles Monte Carlo On Policy Control without Exploring Starts or First Visit from Sutton's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more on REINFORCE and Policy Gradient in &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html#don-t-let-the-past-distract-you&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spinning Up RL: Part3: Intro to policy gradient -  Don't let the past distract you&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, something to note is that even in Sutton's version, the whole trajectory is unrolled, i.e. the episode completes, and then the weights get updated. Otherwise it stops being  a Monte Carlo Method and it becomes a TD method. In addition, you can't make a change on a single point because sampling is a non differentiable operation, instead, the gradient is estimated by collecting a lot of trajectories.&lt;/p&gt;&#xA;" OwnerUserId="28538" LastActivityDate="2019-10-29T11:01:40.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16141" PostTypeId="2" ParentId="15658" CreationDate="2019-10-29T11:19:39.790" Score="1" Body="&lt;p&gt;In theory, deeper architectures can encode more information than shallower ones because they can perform more transformations of the input which results in better results at the output. The training is slower because back propagation is quite expensive, as you increase the depth, you increase the number of parameters and gradients that need to be computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue that you need to take into account is the effect of the activation function. Saturating functions like sigmoid and hyperbolic tangent result in very small gradients in their edges, other activation functions are just flat, eg. ReLU is flat on the negatives therefore, there is no error to propagate because the gradient is either very small (as in saturating functions) or zero. Batch Norm greatly assists in this operation because it collapses values in better ranges where the gradients aren't close to zero.&lt;/p&gt;&#xA;" OwnerUserId="28538" LastEditorUserId="28538" LastEditDate="2019-10-29T20:52:53.777" LastActivityDate="2019-10-29T20:52:53.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16142" PostTypeId="2" ParentId="4683" CreationDate="2019-10-29T12:19:52.823" Score="0" Body="&lt;p&gt;In the case of applying both to natural language, CNN's are good at extracting local and position-invariant features but it does not capture long range semantic dependencies. It just consider local key-phrasses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when the result is determined by the entire sentence or a long-range semantic dependency CNN is not effective as shown in &lt;a href=&quot;https://arxiv.org/abs/1702.01923&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper where the authors compared both architechrures on NLP takss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be extended for general case. &lt;/p&gt;&#xA;" OwnerUserId="30892" LastActivityDate="2019-10-29T12:19:52.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16143" PostTypeId="2" ParentId="13644" CreationDate="2019-10-29T14:33:14.560" Score="6" Body="&lt;p&gt;Learning without Forgetting (LwF) is an incremental learning (sometimes also called continual or lifelong learning) technique for neural networks, which is a machine learning technique that attempts to avoid &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;catastrophic forgetting&lt;/a&gt;. There are several incremental learning approaches. LwF is an incremental learning approach based on the concept of &lt;a href=&quot;https://arxiv.org/pdf/1710.10686.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;regularization&lt;/em&gt;&lt;/a&gt;. In section 3.2 of the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608019300231&quot; rel=&quot;noreferrer&quot;&gt;Continual lifelong learning with neural networks: A review&lt;/a&gt; (2019), by Parisi et al., other regularisation-based continual learning techniques are described. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;LwF could be seen as a combination of &lt;a href=&quot;https://arxiv.org/abs/1503.02531&quot; rel=&quot;noreferrer&quot;&gt;distillation networks&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1311.2524&quot; rel=&quot;noreferrer&quot;&gt;fine-tuning&lt;/a&gt;, which refers to the re-training with a low learning rate (which is a very rudimentary technique to avoid catastrophically forgetting the previously learned knowledge) an already trained model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; with new and (usually) more specific dataset, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}_{\text{new}}$&lt;/span&gt;, with respect to the dataset, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}_{\text{old}}$&lt;/span&gt;, with which you originally trained the given model &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;LwF, as opposed to other continual learning techniques, only uses the new data, so it assumes that past data (used to pre-train the network) is unavailable. The paper &lt;a href=&quot;https://arxiv.org/abs/1606.09282&quot; rel=&quot;noreferrer&quot;&gt;Learning without Forgetting&lt;/a&gt; goes into the details of the technique and it also describes the concepts of &lt;em&gt;feature extraction&lt;/em&gt;, &lt;em&gt;fine tuning&lt;/em&gt; and &lt;em&gt;multitask learning&lt;/em&gt;, which are related to incremental learning techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What is the difference between LwF and transfer learning?&lt;/em&gt; LwF is a combination of distillation networks and fine-tuning, which is a &lt;a href=&quot;https://www.cse.ust.hk/~qyang/Docs/2009/tkde_transfer_learning.pdf&quot; rel=&quot;noreferrer&quot;&gt;transfer learning&lt;/a&gt; technique, which is a special case of incremental learning, where the old and new tasks are different, while, in general, in incremental learning, the old and new tasks can also be the same (which is called &lt;a href=&quot;https://arxiv.org/abs/1606.09282&quot; rel=&quot;noreferrer&quot;&gt;domain adaptation&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-29T16:11:20.250" LastActivityDate="2019-10-29T16:11:20.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16144" PostTypeId="2" ParentId="16131" CreationDate="2019-10-29T16:16:26.913" Score="2" Body="&lt;p&gt;Although you can frame your problem as a bandit problem or RL, it has other workable interpretations. Critical information from your comments is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Total reward is not a simple sum of all the results from 66 different machines. There are interactions between machines.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Total reward is deterministic.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This looks like a problem in &lt;a href=&quot;https://en.wikipedia.org/wiki/Combinatorial_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;combinatorial optimisation&lt;/a&gt;. There are many possible techniques you can throw at this. Which ones work best will depend on how nonlinearities and dependence between choices on different machines affect the end results.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Best Case&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;With deterministic results, if changes between machines were completely isolated, you could search each machine in turn, because you can treat all other 65 components as a constant if you don't change their settings. That would be very simple to code and take &lt;span class=&quot;math-container&quot;&gt;$7 \times 66 = 462$&lt;/span&gt; steps to find the optimimum result.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Worst Case&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In the worst case, the dependencies are so strong and chaotic that there is essentially no predictable difference between changing a single machine's setting and all of them. Pseudo-random number generators and secure hashing functions have this property, as do many quite simple physical systems with feedback loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the worst case, there will be a &quot;magic setting&quot; with best results, and only a brute force search through all combinations of levers will find it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to apply any more efficient search method, you have to assume that the response to combinations of levers is not quite so chaotic.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How to Search?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It seems likely from your description, that the best search algorithm is going to be somewhere between simple machine-by-machine optimisation and an exhaustive global search. However, it is hard to tell just where on that spectrum it lies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few different ways to frame it as reinforcement learning. For instance, you could use current switch combination as state, and run 66 switch changes as an &quot;episode&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest that genetic algorithms are a good match for this search task, assuming there is at least some local-only effect that means combining two good solutions is likley to result in a third good solution. Genetic algorithms don't need calculations for gradients, and fit nicely with discrete combinations. Your genome can simply be the 66 different switch positions, and the fitness rating your black box score for those positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plenty of other combinatorial search algorithms are available. Enough to fill a book or two. One place you could look for inspiration is &lt;a href=&quot;http://blog.shuo1.com/zms/books/program/Clever%20Algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clever Algorithms: Nature-Inspired Programming Recipes&lt;/a&gt; which is a free PDF.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-30T09:13:24.870" LastActivityDate="2019-10-30T09:13:24.870" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16146" PostTypeId="1" CreationDate="2019-10-29T20:49:07.290" Score="5" ViewCount="103" Body="&lt;p&gt;In the deep learning specialization course by Andrew Ng, in the video &lt;a href=&quot;https://www.coursera.org/lecture/nlp-sequence-models/negative-sampling-Iwx0e&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence Models&lt;/a&gt; (minute 4:13), he says that in negative sampling we have to choose a sample of words from the corpus to train rather than choosing the whole corpus. But he said that, for smaller datasets, we need a bigger number of samples, for example, 5-20, and, for larger datasets, we need a smaller sample, for example, 2-5. By sample, I am referring to the number of words along with the target word we have taken to train the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do small datasets require more samples, while big datasets require fewer samples?&lt;/p&gt;&#xA;" OwnerUserId="30907" LastEditorUserId="30907" LastEditDate="2019-10-30T18:24:14.983" LastActivityDate="2019-10-30T18:55:01.937" Title="Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?" Tags="&lt;recurrent-neural-networks&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16147" PostTypeId="1" AcceptedAnswerId="16150" CreationDate="2019-10-29T22:30:46.097" Score="3" ViewCount="39" Body="&lt;p&gt;i have read a lot about NAS but I still do not understand one concept:&#xA;When setting up a neural network, hyperparameters need to set up, like for example the learning rate, dropout rate, batch size, filter size... etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In NAS only the best architecture is decided, e.g. how many layers and neurons.. .&#xA;But what about the hyperparameters? Are they randomly decided?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help&lt;/p&gt;&#xA;" OwnerUserId="30909" LastActivityDate="2019-10-30T05:33:43.280" Title="Which hyperparameter does Neural Architecture search (NAS) use?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;neural-architecture-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16148" PostTypeId="1" CreationDate="2019-10-30T04:19:41.780" Score="3" ViewCount="413" Body="&lt;p&gt;Natural gradient aims to do a steepest descent on the &quot;function&quot; space, a manifold that is independent from how the function is parameterized. It argues that the steepest descent on this function space is not the same as steepest descent on the parameter space. We should favor the former. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, for example in a regression task, a neural net could be interpreted as a probability function (Gaussian with the output as mean and some constant variance), it is &quot;natural&quot; to form a distance on the manifold under the KL-divergence (and a Fisher information matrix as its metric). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I want to be creative, I could use the same argument to use &quot;square distance&quot; between the outputs of the neural nets (distance of the means) which I think is not the same as the KL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I wrong, or it is just another legit way? Perhaps, not as good?&lt;/p&gt;&#xA;" OwnerUserId="9793" LastActivityDate="2020-06-26T09:55:10.123" Title="Why they use KL divergence in Natural gradient?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16149" PostTypeId="2" ParentId="16120" CreationDate="2019-10-30T04:21:19.317" Score="0" Body="&lt;p&gt;I would go so far as to say that unless the training examples include predicate data- that is, data about conditions leading up to a crime or non-crime-, then you cannot have enough information to predict the occurence of a crime from conditions or events that happen in advance of a potential crime not yet committed.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-10-30T04:21:19.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16150" PostTypeId="2" ParentId="16147" CreationDate="2019-10-30T05:33:43.280" Score="2" Body="&lt;p&gt;Great question! It's not clearly stated (it's not stated &lt;strong&gt;at all&lt;/strong&gt; on wikipedia), but after a bit of searching, I found an answer &lt;a href=&quot;https://towardsdatascience.com/neural-architecture-search-limitations-and-extensions-8141bec7681f&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; about a third of the way down the page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The best performing architecture observed during the training of the controller is taken, and &lt;strong&gt;a grid search is performed over some basic hyperparameters such as learning rate and weight decay&lt;/strong&gt; in order to achieve near STOTA (state of the art) performance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So as a direct answer: The norm; &lt;a href=&quot;https://medium.com/datadriveninvestor/an-introduction-to-grid-search-ff57adcc0998&quot; rel=&quot;nofollow noreferrer&quot;&gt;A grid search&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-10-30T05:33:43.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16151" PostTypeId="1" CreationDate="2019-10-30T06:32:06.797" Score="2" ViewCount="39" Body="&lt;p&gt;I have a lengthy timeseries datasets which contains several variables (from sensors etc) to be classified as actions or states. Providing they are successfully done, I want to learn a control policy using DDPG. &#xA;But I have no knowledge of the environment. &#xA;How can I learn my policy off-line only by using these datasets without having any model of the environment? After learning off-line first, then the policy can then be used to learn and control online later in a certain real-world environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I know that experience buffer can be used to store the datasets. How should you set the buffer size in this case? &#xA;From what I understand, DDPG needs lots of data to be used for learning.&#xA;Should I build an environment model using the specified datasets? Or I don't really need this step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these will be implemented in Python and maybe with the help of another tools if needed. There are some implementation of DDPG available so it is not the main problem, but this implementation must be tweaked to solve my proposed problem. Normally the implemented DDPG in Python requires a Gym-environment as an input so I must change it to satisfy my needs as I don't need Gym for my use case. And these implementations in Python are somehow on-line codes so you need to interact directly with the environment model for the algorithm to be working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me tackle this  problem or give me some advice regarding this? I can help giving more details if needed. Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="30918" LastEditorUserId="30918" LastEditDate="2019-10-30T07:11:41.473" LastActivityDate="2019-10-30T07:11:41.473" Title="How to learn using DDPG in python solely using a timeseries datasets" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;datasets&gt;&lt;control-problem&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16152" PostTypeId="1" AcceptedAnswerId="16154" CreationDate="2019-10-30T07:19:50.070" Score="4" ViewCount="699" Body="&lt;p&gt;In my AI literature research, I often notice authors use term 'democratizing AI', especially in the AutoML area. I think I have an idea of what this means, but I would like to ask you for some more specific answers.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2019-10-30T09:34:32.397" LastActivityDate="2019-10-30T09:59:33.213" Title="What does 'democratizing AI' exactly mean?" Tags="&lt;philosophy&gt;&lt;terminology&gt;&lt;theory&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16153" PostTypeId="1" CreationDate="2019-10-30T09:22:55.397" Score="6" ViewCount="194" Body="&lt;p&gt;I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is certainly possible to put the output of a CNN and other data into, say, a densely connected network; but it seems inefficient. Are there well-established ways of doing something like this?&lt;/p&gt;&#xA;" OwnerUserId="23584" LastEditorUserId="2444" LastEditDate="2019-11-03T13:05:00.353" LastActivityDate="2019-11-03T13:05:00.353" Title="Are there well-established ways of mixing different inputs (e.g. image and numbers)?" Tags="&lt;convolutional-neural-networks&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16154" PostTypeId="2" ParentId="16152" CreationDate="2019-10-30T09:59:33.213" Score="4" Body="&lt;p&gt;In this particular context, &quot;Democratize&quot; means to make more accessible to people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, &quot;Democratizing AI&quot; means to make AI softwares and AI programming available, accessible and easy to use for the vast majority of people.&lt;/p&gt;&#xA;" OwnerUserId="19859" LastActivityDate="2019-10-30T09:59:33.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16155" PostTypeId="2" ParentId="16080" CreationDate="2019-10-30T10:12:42.020" Score="1" Body="&lt;p&gt;You can find leaderboards as well as code at &lt;a href=&quot;https://paperswithcode.com/task/semantic-segmentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;this address&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, &lt;a href=&quot;https://arxiv.org/pdf/1908.07919.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;HRNetV2&lt;/a&gt; leads the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The U-Net architecture is part of a broad family of network architectures that aggregate multi-scale features to extract finer details useful for semantic segmentation. Examples are Feature Pyramidal Networks (FPN), Hourglass, Encoder-Decoder, MatrixNet, etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oKEhJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oKEhJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19859" LastActivityDate="2019-10-30T10:12:42.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16156" PostTypeId="2" ParentId="16153" CreationDate="2019-10-30T11:27:01.430" Score="4" Body="&lt;p&gt;A more efficient way would be creating a multi input model, with something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;___________    _____________&#xA;|__Image__|    |Other input|            &#xA;_____|_____     _____|_____&#xA;|___CNN___|     |__Dense___|&#xA;_____|______    _____|______&#xA;|_Features1_|   |_Features2_|&#xA;         __|_____|__&#xA;         |__Merge___|&#xA;         _____|______&#xA;         |___Dense__|&#xA;         _____|_____&#xA;         |__Output__|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, you could also combine the unstructured data to the image, as stated in the quora answer:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;The out-of-the-box method:&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;If you want to just take your CNN library and use it without much&#xA;  thought, there's an easy way to do it.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Your image has “channels”: red blue and green channels, for example.&#xA;  Just add another channel for each unstructured feature. Those channels&#xA;  will just be 2D-arrays whose entries are all the same value: the value&#xA;  of your outside feature.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It means more memory and more parameters though. If you have a lot of&#xA;  unstructured data, this can become prohibitively expensive.&lt;/p&gt;&#xA;  &#xA;  &lt;h3&gt;The more efficient method (and still not hard):&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;You use one or more deconvolutional filters to bring the unstructured&#xA;  data up to the size of the structured data, concatenate them along the&#xA;  channel dimension, and keep going as if nothing happened.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.quora.com/How-can-l-train-a-CNN-with-extra-features-other-than-the-pixels&quot; rel=&quot;nofollow noreferrer&quot;&gt;How can l train a CNN with extra features other than the pixels? (Quora)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="1671" LastEditDate="2019-11-02T16:13:18.643" LastActivityDate="2019-11-02T16:13:18.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16157" PostTypeId="1" AcceptedAnswerId="16159" CreationDate="2019-10-30T12:09:27.973" Score="2" ViewCount="85" Body="&lt;p&gt;In OpenAI Gym &amp;quot;reward&amp;quot; is defined as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;reward&lt;/code&gt; (float): amount of reward achieved by the previous action. The&#xA;scale varies between environments, but the goal is always to increase&#xA;your total reward.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am training Hindsight Experience Replay on Fetch robotics environments, where rewards are sparse and binary indicating whether or not the task is completed. The &lt;a href=&quot;https://arxiv.org/pdf/1707.01495.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; implementing HER uses &lt;strong&gt;success rate&lt;/strong&gt; as a metric in its plots, like so:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ryiyZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ryiyZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;On page 5 of the original paper, it is stated that the reward is binary and sparse.&lt;/p&gt;&#xA;&lt;p&gt;When I print the rewards obtained during a simulation of &lt;code&gt;FetchReach-v1&lt;/code&gt; trained with HER, I get the following values. The first column shows the reward and the second column shows the episode length.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t5cCj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t5cCj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As can be seen, at every time step, I am getting a reward, sometimes I get a &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; reward at every time step throughout the episode for a total of &lt;span class=&quot;math-container&quot;&gt;$-50$&lt;/span&gt;. The maximum reward I can achieve throughout the episode is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Therefore my question is: What is the reward obtained at each time-step? What does it represent and how is this different from the success rate?&lt;/p&gt;&#xA;" OwnerUserId="14390" LastEditorUserId="2444" LastEditDate="2020-11-21T12:39:49.543" LastActivityDate="2020-11-21T12:39:49.543" Title="What is the difference between success rate and reward when dealing with binary and sparse rewards?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;reward-functions&gt;&lt;hindsight-experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16158" PostTypeId="2" ParentId="5230" CreationDate="2019-10-30T12:13:32.910" Score="0" Body="&lt;p&gt;To train neural nets for image classification I like to use AIDeveloper. &#xA;&lt;a href=&quot;https://github.com/maikherbig/AIDeveloper&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/maikherbig/AIDeveloper&lt;/a&gt;&#xA;This is a free GUI based software for Win7 and 10. There's a bunch of tutorials on Youtube.&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=dvFiSRnwoto&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=dvFiSRnwoto&lt;/a&gt;&#xA;Created models are saved as Keras model file and can also be exported to other formats, so you can use the trained model in other frameworks.&lt;/p&gt;&#xA;" OwnerUserId="30930" LastActivityDate="2019-10-30T12:13:32.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16159" PostTypeId="2" ParentId="16157" CreationDate="2019-10-30T12:20:10.830" Score="1" Body="&lt;p&gt;Page 6 of the paper describes the exact reward functions, and why they were used:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Goals: Goals describe the desired position of the object (a box or a&#xA;  puck depending on the task) with some fixed tolerance of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; i.e. &lt;span class=&quot;math-container&quot;&gt;$G = \mathcal{R}^3$&lt;/span&gt;&#xA;   and &lt;span class=&quot;math-container&quot;&gt;$f_g(s) = [|g − s_{object}| ≤ \epsilon]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s_{object}$&lt;/span&gt; is the position of&#xA;  the object in the state s. The mapping from states to goals used in&#xA;  HER is simply &lt;span class=&quot;math-container&quot;&gt;$m(s) = s_{object}$&lt;/span&gt;. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Rewards: Unless stated otherwise we use binary and sparse rewards &lt;span class=&quot;math-container&quot;&gt;$r(s, a, g) = −[f_g(s 0 ) = 0]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is&#xA;  the state after the execution of the action a in the state s. We&#xA;  compare sparse and shaped reward functions in Sec. 4.4.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, at least in the base version (which I believe is your &lt;code&gt;fetchreach-v1&lt;/code&gt;), the agent receives a reward of -1 for every timestep spent more than &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; from the goal state, and a reward of 0 for every timestep spent within &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; of the goal state. Thus, a score of -5.0 would seem to correspond to the agent moving directly to the goal and staying there, while a score of -50.0 would correspond to the agent failing to reach the goal state entirely.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-30T12:20:10.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16160" PostTypeId="1" AcceptedAnswerId="16167" CreationDate="2019-10-30T14:29:24.127" Score="3" ViewCount="87" Body="&lt;p&gt;Training neural network with 4 GPUs using pyTorch, performance is not even 2 times (btw 1 &amp;amp; 2 times) compare to using one GPU. From Nvidia-smi we see GPU usage is for few milliseconds and next 5-10 seconds looks like data is off-loaded and loaded for new executions (mostly GPU usage is 0%). Is there any way in pyTorch to improve the data upload and offload for the GPU execution.   &lt;/p&gt;&#xA;" OwnerUserId="27221" LastActivityDate="2019-10-30T23:47:33.530" Title="Training network with 4 GPUs performance is not exactly 4 times over one GPU why?" Tags="&lt;training&gt;&lt;pytorch&gt;&lt;gpu&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16161" PostTypeId="1" CreationDate="2019-10-30T15:17:05.660" Score="1" ViewCount="52" Body="&lt;p&gt;There are several occasion that reinforcement learning can be used as a control mean.&#xA;The action is for example the set target temperature (which in many occasions change with time) and the state is for example the current temperature and other variables. The policy is then the control mean that is going to be learnt using the reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As there is a dead time (input lag) and time delay in the real world, how can one propose to tackle this problem when using reinforcement learning as a control mean? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="30918" LastActivityDate="2019-10-30T15:17:05.660" Title="Solving the dead time problem for control using reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;control-problem&gt;&lt;control-theory&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="16162" PostTypeId="1" CreationDate="2019-10-30T17:03:00.880" Score="2" ViewCount="238" Body="&lt;p&gt;A general AI &lt;em&gt;x&lt;/em&gt; creates another AI &lt;em&gt;y&lt;/em&gt; which is better than &lt;em&gt;x&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;y&lt;/em&gt; creates an AI better than itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so on, with each generation's primary goal to create a better AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a name for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By better, I mean survivability, ability to solve new problems, enhance human life physically and mentally, and advance our civilization to an intergalactic civilization to name a few.&lt;/p&gt;&#xA;" OwnerUserId="30935" LastEditorUserId="2444" LastEditDate="2019-10-31T12:42:18.670" LastActivityDate="2019-11-05T06:36:44.837" Title="What is the name of an AI whose primary goal is to create a better AI?" Tags="&lt;terminology&gt;&lt;evolutionary-algorithms&gt;&lt;agi&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16163" PostTypeId="2" ParentId="16162" CreationDate="2019-10-30T17:20:34.077" Score="6" Body="&lt;p&gt;I don't think there is a single standard word or phrase that covers just this concept. Perhaps &lt;em&gt;recursive self-improvement&lt;/em&gt; matches the idea concisely - but that is not specific AI jargon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very little is understood about what strength this effect can have or what the limits are. Will 10 generations of self-improvement lead to a machine that is 10% better, 10 times better, or &lt;span class=&quot;math-container&quot;&gt;$2^{10}$&lt;/span&gt; times better? And by what measure? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some futurologists suggest this might be a very strong effect, and use the term &lt;em&gt;Singularity&lt;/em&gt; to capture the idea that intelligence growth through recursive self-improvement will be strong, exceed human intelligence, and lead to some form of super-intelligent machine - the point at which this goal is reached is called &lt;em&gt;The Singularity&lt;/em&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Singularity_Is_Near&quot; rel=&quot;noreferrer&quot;&gt;Ray Kurzweil&lt;/a&gt; is a well-known proponent of this idea. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, use of the term &lt;em&gt;Singularity&lt;/em&gt; implies more than just the basic recursion that you suggest, and includes assumptions of a very large effect. Plus technically, it refers to a stage that results from the recursion, not the recursion itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, despite the popularity of it as a concept, whether or not such self-improving system will have a large impact on the generation of intelligent machines is completely unknown at this stage. Related research about general intelligence is still in its infancy, so it is not even clear what would count as being the first example system x.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-10-30T17:33:20.970" LastActivityDate="2019-10-30T17:33:20.970" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16164" PostTypeId="2" ParentId="16148" CreationDate="2019-10-30T18:39:37.330" Score="1" Body="&lt;p&gt;The KL divergence has slightly different interpretations depending on the context. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence#Interpretations&quot; rel=&quot;nofollow noreferrer&quot;&gt;related Wikipedia article&lt;/a&gt; contains a section dedicated to these interpretations. Independently of the interpretation, the KL divergence is always defined as a &lt;em&gt;specific&lt;/em&gt; function of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-entropy&lt;/a&gt; (which you should be familiar with before attempting to understand the KL divergence) between two distributions (in this case, probability mass functions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;D_\text{KL}(P\parallel Q) &#xA;&amp;amp;= -\sum_{x\in\mathcal{X}} p(x) \log q(x) + \sum_{x\in\mathcal{X}} p(x) \log p(x) \\&#xA; &amp;amp;= H(P, Q) - H(P)&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$H(P, Q)$&lt;/span&gt; is the cross-entropy of the distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$H(P) = H(P, P)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The KL is not a metric, given that it does not obey the triangle inequality. In other words, in general, &lt;span class=&quot;math-container&quot;&gt;$D_\text{KL}(P\parallel Q) \neq D_\text{KL}(Q\parallel P) $&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Given that a neural network is trained to output the mean (which can be a scalar or a vector) and the variance (which can be a scalar, a vector or a matrix), why don't we use a metric like the MSE to compare means and variances?&lt;/em&gt; When you use the KL divergence, you don't want to compare just numbers (or matrices), but probability distributions (more precisely, probability densities or mass functions), so you will not compare just the mean and the variance of two different distributions, but you will actually compare the distributions. See &lt;a href=&quot;https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence#Basic_example&quot; rel=&quot;nofollow noreferrer&quot;&gt;the example of the application of the KL divergence&lt;/a&gt; in the related Wikipedia article.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-30T18:52:26.117" LastActivityDate="2019-10-30T18:52:26.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16165" PostTypeId="2" ParentId="16146" CreationDate="2019-10-30T18:55:01.937" Score="1" Body="&lt;p&gt;He likely found this to a be a best practice to avoid over fitting, with a small data set if you only use small and easy to learn (less words -&gt; less degrees of freedom) sequences then you open your model to the risk of over fitting that data set where as on a large data set that has alot more total information you can train on small sequences without being at risk of over fitting because although the smaller sequences will be easier to learn the variance of sequences will be much higher. &lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2019-10-30T18:55:01.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16166" PostTypeId="2" ParentId="16148" CreationDate="2019-10-30T19:43:00.560" Score="1" Body="&lt;p&gt;Yes, Squared distances &amp;amp; KL Divergence are not the same. Squared distance between means is not a useful metric as it doesn't gauge the amount of similarity between 2 distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we compute &lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;D_\text{KL}(P\parallel Q) &#xA;\end{align}&lt;/span&gt;&#xA;We are computing the amount of information that is lost when we approximate &lt;strong&gt;P&lt;/strong&gt; as &lt;strong&gt;Q&lt;/strong&gt;. Ideally, we would want the KL divergence to be as low as possible.&#xA;Here is an interesting article &lt;a href=&quot;https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained&lt;/a&gt;&#xA;where the author has explained KL Divergence with a toy example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope it helps :)&lt;/p&gt;&#xA;" OwnerUserId="30939" LastActivityDate="2019-10-30T19:43:00.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16167" PostTypeId="2" ParentId="16160" CreationDate="2019-10-30T23:35:35.550" Score="2" Body="&lt;p&gt;Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data beforehand and save to a file. Multiple GPU cannot scale as the GPU have to get all data to one GPU to calculate the loss. The performance of 4 GPU is around 3.5x. A large batch size also would help as each GPU will have 1/4 the batch size. a batch size of 64-128 is good for 4 GPU. See the following example code fro CIFAR-10 for multi gpu code. It have dataloaders and dataparallel. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;import time&#xA;import datetime&#xA;&#xA;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;from torch.optim import lr_scheduler&#xA;import torch.backends.cudnn as cudnn&#xA;&#xA;import torchvision&#xA;import torchvision.transforms as transforms&#xA;from torchvision.datasets import CIFAR10&#xA;from torch.utils.data import DataLoader&#xA;&#xA;from model import pyramidnet&#xA;import argparse&#xA;from tensorboardX import SummaryWriter&#xA;&#xA;&#xA;parser = argparse.ArgumentParser(description='cifar10 classification models')&#xA;parser.add_argument('--lr', default=0.1, help='')&#xA;parser.add_argument('--resume', default=None, help='')&#xA;parser.add_argument('--batch_size', type=int, default=768, help='')&#xA;parser.add_argument('--num_worker', type=int, default=4, help='')&#xA;parser.add_argument(&quot;--gpu_devices&quot;, type=int, nargs='+', default=None, help=&quot;&quot;)&#xA;args = parser.parse_args()&#xA;&#xA;gpu_devices = ','.join([str(id) for id in args.gpu_devices])&#xA;os.environ[&quot;CUDA_VISIBLE_DEVICES&quot;] = gpu_devices&#xA;&#xA;&#xA;def main():&#xA;    best_acc = 0&#xA;&#xA;    device = 'cuda' if torch.cuda.is_available() else 'cpu'&#xA;&#xA;    print('==&amp;gt; Preparing data..')&#xA;    transforms_train = transforms.Compose([&#xA;        transforms.RandomCrop(32, padding=4),&#xA;        transforms.RandomHorizontalFlip(),&#xA;        transforms.ToTensor(),&#xA;        transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))])&#xA;&#xA;    dataset_train = CIFAR10(root='../data', train=True, download=True, &#xA;                            transform=transforms_train)&#xA;&#xA;    train_loader = DataLoader(dataset_train, batch_size=args.batch_size, &#xA;                              shuffle=True, num_workers=args.num_worker)&#xA;&#xA;    # there are 10 classes so the dataset name is cifar-10&#xA;    classes = ('plane', 'car', 'bird', 'cat', 'deer', &#xA;               'dog', 'frog', 'horse', 'ship', 'truck')&#xA;&#xA;    print('==&amp;gt; Making model..')&#xA;&#xA;    net = pyramidnet()&#xA;    net = nn.DataParallel(net)&#xA;    net = net.to(device)&#xA;    num_params = sum(p.numel() for p in net.parameters() if p.requires_grad)&#xA;    print('The number of parameters of model is', num_params)&#xA;&#xA;    criterion = nn.CrossEntropyLoss()&#xA;    optimizer = optim.Adam(net.parameters(), lr=args.lr)&#xA;    # optimizer = optim.SGD(net.parameters(), lr=args.lr, &#xA;    #                       momentum=0.9, weight_decay=1e-4)&#xA;&#xA;    train(net, criterion, optimizer, train_loader, device)&#xA;&#xA;&#xA;def train(net, criterion, optimizer, train_loader, device):&#xA;    net.train()&#xA;&#xA;    train_loss = 0&#xA;    correct = 0&#xA;    total = 0&#xA;&#xA;    epoch_start = time.time()&#xA;    for batch_idx, (inputs, targets) in enumerate(train_loader):&#xA;        start = time.time()&#xA;&#xA;        inputs = inputs.to(device)&#xA;        targets = targets.to(device)&#xA;        outputs = net(inputs)&#xA;        loss = criterion(outputs, targets)&#xA;&#xA;        optimizer.zero_grad()&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;&#xA;        train_loss += loss.item()&#xA;        _, predicted = outputs.max(1)&#xA;        total += targets.size(0)&#xA;        correct += predicted.eq(targets).sum().item()&#xA;&#xA;        acc = 100 * correct / total&#xA;&#xA;        batch_time = time.time() - start&#xA;&#xA;        if batch_idx % 20 == 0:&#xA;            print('Epoch: [{}/{}]| loss: {:.3f} | acc: {:.3f} | batch time: {:.3f}s '.format(&#xA;                batch_idx, len(train_loader), train_loss/(batch_idx+1), acc, batch_time))&#xA;&#xA;    elapse_time = time.time() - epoch_start&#xA;    elapse_time = datetime.timedelta(seconds=elapse_time)&#xA;    print(&quot;Training time {}&quot;.format(elapse_time))&#xA;&#xA;&#xA;if __name__=='__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://github.com/dnddnjs/pytorch-multigpu/blob/master/data_parallel/train.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dnddnjs/pytorch-multigpu/blob/master/data_parallel/train.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-10-30T23:47:33.530" LastActivityDate="2019-10-30T23:47:33.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16168" PostTypeId="2" ParentId="16119" CreationDate="2019-10-30T23:45:57.367" Score="0" Body="&lt;p&gt;This depend on type of data you use. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Time sequence data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If the data advanced in time, a LSTM or similar RNN should be used. RNN calculate output through time. It works very good on time series data as it have a real sense of time. While CNN and MLP could work for time series data, it often don't work that well as different timestep of data is not defined. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Non- Time sequence data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;According to you previous comment, the data seems to be of this kind. In the case of your data, a normal Multi-layer perceptron works well for this. The data is a direct mapping between the input and the output. If the input data is an image, use a CNN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example code in keras, see here:&#xA;You need the &lt;code&gt;pandas&lt;/code&gt; module for this to work. Run &lt;code&gt;pip install pandas&lt;/code&gt; to install pandas.&#xA;    from keras.models import Sequential&#xA;    from keras.utils import np_utils&#xA;    from keras.layers.core import Dense, Activation, Dropout&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;&#xA;# Read data&#xA;train = pd.read_csv('../input/train.csv')&#xA;labels = train.ix[:,0].values.astype('int32')&#xA;X_train = (train.ix[:,1:].values).astype('float32')&#xA;X_test = (pd.read_csv('../input/test.csv').values).astype('float32')&#xA;&#xA;# convert list of labels to binary class matrix&#xA;y_train = np_utils.to_categorical(labels) &#xA;&#xA;# pre-processing: divide by max and substract mean&#xA;scale = np.max(X_train)&#xA;X_train /= scale&#xA;X_test /= scale&#xA;&#xA;mean = np.std(X_train)&#xA;X_train -= mean&#xA;X_test -= mean&#xA;&#xA;input_dim = X_train.shape[1]&#xA;nb_classes = y_train.shape[1]&#xA;&#xA;# Here's a Deep Dumb MLP (DDMLP)&#xA;model = Sequential()&#xA;model.add(Dense(128, input_dim=input_dim))&#xA;model.add(Activation('relu'))&#xA;model.add(Dropout(0.15))&#xA;model.add(Dense(128))&#xA;model.add(Activation('relu'))&#xA;model.add(Dropout(0.15))&#xA;model.add(Dense(nb_classes))&#xA;model.add(Activation('softmax'))&#xA;&#xA;# we'll use categorical xent for the loss, and RMSprop as the optimizer&#xA;model.compile(loss='categorical_crossentropy', optimizer='rmsprop')&#xA;&#xA;print(&quot;Training...&quot;)&#xA;model.fit(X_train, y_train, nb_epoch=10, batch_size=16, validation_split=0.1, show_accuracy=True, verbose=2)&#xA;&#xA;print(&quot;Generating test predictions...&quot;)&#xA;preds = model.predict_classes(X_test, verbose=0)&#xA;&#xA;def write_preds(preds, fname):&#xA;    pd.DataFrame({&quot;ImageId&quot;: list(range(1,len(preds)+1)), &quot;Label&quot;: preds}).to_csv(fname, index=False, header=True)&#xA;&#xA;write_preds(preds, &quot;keras-mlp.csv&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Code source: &lt;a href=&quot;https://www.kaggle.com/fchollet/simple-deep-mlp-with-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/fchollet/simple-deep-mlp-with-keras&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, in the case of our data, use a multi layer perceptron  should work. Hope kit can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-11-04T04:59:54.633" LastActivityDate="2019-11-04T04:59:54.633" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16169" PostTypeId="1" CreationDate="2019-10-31T01:57:59.730" Score="1" ViewCount="98" Body="&lt;p&gt;Is there any link or explanation on how to create Color palettes for coco dataset for segmentation. I see there is a link for cityscapes dataset &lt;a href=&quot;https://gluon-cv.mxnet.io/_modules/gluoncv/utils/viz/segmentation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gluon-cv.mxnet.io/_modules/gluoncv/utils/viz/segmentation.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27221" LastActivityDate="2019-10-31T01:57:59.730" Title="Colour palettes for COCO dataset" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;inference&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16170" PostTypeId="1" CreationDate="2019-10-31T07:00:58.860" Score="1" ViewCount="26" Body="&lt;p&gt;I am reproducing the results from &lt;a href=&quot;https://arxiv.org/abs/1707.01495&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience Replay&lt;/a&gt; by Andrychowicz et. al. In the original paper they present the results below, where the agent is trained for 200 epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;200 epochs * 800 episodes * 50 time steps = 8,000,000 total time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xw447.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xw447.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to reproduce the results but instead of using 8 cpu cores, I am using 19 CPU cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I train the FetchPickAndPlace for 120 epochs, but with only 50 episodes per epoch. Therefore 120 * 50 * 50 = 300,000 iterations. I present the curve below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;￼&lt;a href=&quot;https://i.stack.imgur.com/OraSY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OraSY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and logger output for the first two epochs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;￼&lt;a href=&quot;https://i.stack.imgur.com/rtxyP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rtxyP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as can be seen from my tensorboard plot, after 30 epochs we get a steady success rate very close to 1. 30 epochs * 50 episodes * 50 time steps = 75,000 iterations. Therefore it took the algorithm 75,000 time steps to learn this environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original paper took approximately 50 * 800 * 50 = 2,000,000 time steps to achieve the same goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is it that in my case the environment was solved nearly 30 times faster? Are there any flaws in my workings above? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: This was not a one off case. I tested again and got the same results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Post on Reddit: &lt;a href=&quot;https://www.reddit.com/r/reinforcementlearning/comments/dpjwfu/getting_same_results_with_half_the_number_of/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.reddit.com/r/reinforcementlearning/comments/dpjwfu/getting_same_results_with_half_the_number_of/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14390" LastEditorUserId="14390" LastEditDate="2019-10-31T07:50:02.673" LastActivityDate="2019-10-31T07:50:02.673" Title="How am I getting same results 30 times faster than in original HER paper?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16171" PostTypeId="1" CreationDate="2019-10-31T09:08:33.547" Score="1" ViewCount="90" Body="&lt;p&gt;I’m using a simple neural network to solve a reinforcement learning problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The configuration is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;X-inputs: The current state&#xA;Y-outputs: The possible actions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever the network yields a “good” solution, i “reward” the network by training it a number of times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever the network yields a “bad” or “neutral” solution, i ignore it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be working somewhat, but from what i read, everyone else (in broad terms) seems to be using a 2 neural network configuration for similar tasks. (Policy network and value network)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am i missing something? - and are there any obvious caveats of the “single network” method i am using?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supplemental question: Are there other methods of “rewarding” a network, aside from simply training it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="26768" LastActivityDate="2019-11-03T18:50:06.097" Title="Neural network for reinforcement learning" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16172" PostTypeId="1" AcceptedAnswerId="16181" CreationDate="2019-10-31T10:19:48.970" Score="1" ViewCount="391" Body="&lt;p&gt;I have an image classification task to solve, but based on quite simple/good terms:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are only two classes (either good or not good)&lt;/li&gt;&#xA;&lt;li&gt;The images always show the same kind of piece (either with or w/o fault)&lt;/li&gt;&#xA;&lt;li&gt;That piece is always filmed from the same angle &amp;amp; distance&lt;/li&gt;&#xA;&lt;li&gt;I have at least 1000 sample images for both classes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So I thought it should be easy to come up with a good CNN solution - and it was. I created a VGG16-based model with a custom classifier (Keras/TF). Via transfer learning I was able to achieve up to 100% validation accuracy during model training, so all is fine on that end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of curiosity and because the VGG-based approach seems a bit &quot;slow&quot;, I also wanted to try it with a more modern model architecture as the base, so I did with &lt;em&gt;ResNet50v2&lt;/em&gt; and &lt;em&gt;Xception&lt;/em&gt;. I trained both similar to the VGG-based model, tried it with several hyperparameter modifications, etc. However, I was not able to achieve a better validation accuracy than 95% - so much worse than with the &quot;old&quot; VGG architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence my question is: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given these &quot;simple&quot; (always the same) images and only two classes, is the VGG model probably a better base than a modern network like ResNet or Xception? Or is it more likely that I messed something up with my model or simply got the training/hyperparameters not right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14504" LastEditorUserId="2444" LastEditDate="2020-06-14T22:17:25.627" LastActivityDate="2020-12-04T05:04:52.403" Title="Is a VGG-based CNN model sometimes better for image classfication than a modern architecture?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;residual-networks&gt;&lt;vgg&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16173" PostTypeId="1" AcceptedAnswerId="16174" CreationDate="2019-10-31T11:34:50.637" Score="1" ViewCount="40" Body="&lt;p&gt;I'm trying to replace the strided convolutions of &lt;a href=&quot;https://github.com/keras-team/keras-applications/blob/71acdcd98088501247f4b514b7cbbdf8182a05a4/keras_applications/mobilenet.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras' MobileNet implementation&lt;/a&gt; with the &lt;code&gt;ConvBlurPool&lt;/code&gt; operation as defined in the &lt;a href=&quot;https://arxiv.org/abs/1904.11486&quot; rel=&quot;nofollow noreferrer&quot;&gt;Making Convolutional Networks Shift-Invariant Again paper&lt;/a&gt;. In the paper, a &lt;code&gt;ConvBlurPool&lt;/code&gt; is implemented as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Relu \circ Conv_{k,s} \rightarrow Subsample_s \circ Blur_m \circ Relu \circ Conv_{k,1}&#xA;$$&lt;/span&gt;&#xA;where &lt;code&gt;k&lt;/code&gt; is the convolution's output kernels, &lt;code&gt;s&lt;/code&gt; is the stride, &lt;code&gt;m&lt;/code&gt; is the blurring kernel size and the subsample+blur is implemented as a strided convolution with a constant kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My issues start when batch normalization enter the picture.&#xA;In MobileNet, a conv block is defined as follows (omitting the zero-padding):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Relu \circ BatchNorm \circ Conv_{k,s}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am leaning towards converting it to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Subsample_s \circ Blur_m \circ Relu \circ BatchNorm \circ Conv_{k,1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e., putting the BN before the activation as it's normally done. This is not equivalent though, because the first BN operates on the downsampled signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;BatchNorm \circ Subsample_s \circ Blur_m \circ Relu \circ Conv_{k,1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the BN as last operation. This is also not equivalent, because now the BN comes after the ReLu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any reason to prefer one option over the other? Are there any other options I'm not considering?&lt;/p&gt;&#xA;" OwnerUserId="22086" LastActivityDate="2019-10-31T12:18:04.080" Title="Positioning of batch normalization layer when converting strided convolution to convolution + blurpool" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16174" PostTypeId="2" ParentId="16173" CreationDate="2019-10-31T12:18:04.080" Score="0" Body="&lt;p&gt;After finding the &lt;a href=&quot;https://github.com/adobe/antialiased-cnns&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper authors' Github&lt;/a&gt;, I saw that, although they only have a MobileNet V2 model implemented, they choose the Subsample-after-ReLu option (the first one in the question).&#xA;Although this doesn't fully answer my question, I'll take &quot;the paper authors do it this way&quot; as enough reason to prefer this over the alternative.&lt;/p&gt;&#xA;" OwnerUserId="22086" LastActivityDate="2019-10-31T12:18:04.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16175" PostTypeId="1" AcceptedAnswerId="16429" CreationDate="2019-10-31T12:51:19.793" Score="2" ViewCount="571" Body="&lt;p&gt;Suppose i trained the images of two people say Bob , Thomas .When i run the algorithm to detect the face of a totally different person from these two say John , then John is recognized as Bob or Thomas.How to avoid this ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am studying a face recognition model on GitHub(&lt;a href=&quot;https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) which uses Facenet model. Problem is when an unknown image (the image which is not in training data set) is given to identify , it identifies the unknown person as one the person in the data set .I searched on web and i found i need to increase threshold value .I guess i need to increase the &lt;a href=&quot;https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/b351d5d3f4766572868236eed78a25082da44c27/identify_face_image.py#L28&quot; rel=&quot;nofollow noreferrer&quot;&gt;threshold&lt;/a&gt;. But when i am increasing the threshold value to 0.99,0.99,99 then only it is rejecting the unknown image (image of the person who is not in data set) and sometimes even rejecting the image of person who is in dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess by increasing the threshold value what we are assuring is that an image is classified as one of the person in training data only when they are close enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to make changes so that the model works properly ?And can someone explain Threshold in Facenet model better.&lt;/p&gt;&#xA;" OwnerUserId="30306" LastEditorUserId="30306" LastEditDate="2019-11-10T17:07:30.720" LastActivityDate="2019-11-11T10:21:57.777" Title="Three step threshold in Facenet model of face recogniton" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16176" PostTypeId="1" CreationDate="2019-10-31T12:58:48.673" Score="1" ViewCount="59" Body="&lt;p&gt;Suppose that my task is to label news articles; that is, to classify which category a news article belongs to. Using the labelled data (with old labels) that I have, I have trained a model for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For relevancy purposes, certain labels may be split into multiple new labels. For example, 'Sports' may split into 'Sports' and 'E-Sports'. Because of these new labels, I will need to retrain my model. However, my training data is labelled with the old labels. What can I do to address these 'label updates'?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea: Perhaps use some unsupervised clustering method (K-means?) to split the data with the old labels into the new labels. (But how can we be certain that which cluster has what new label?) Then use this 'updated' data to train a model. Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="30729" LastActivityDate="2019-11-04T07:33:44.840" Title="How to handle classification with label updates?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16177" PostTypeId="1" AcceptedAnswerId="16178" CreationDate="2019-10-31T13:04:34.567" Score="1" ViewCount="73" Body="&lt;p&gt;im working on a project in which I have to make a multi-layer perceptron with two hidden layers with 3 nodes in each. The target value in my data contains 8 unique values/classes. One of the tasks states &quot;For the most popular class CYT plot weight values per iteration for the last layer (3 weights and bias)&quot;.  My question is &quot;does this statement make sense&quot;?  I can access the weights and biases of a layer but I don't get what are  weight values for a specific class and how to access them&lt;/p&gt;&#xA;" OwnerUserId="21084" LastActivityDate="2019-10-31T13:38:51.830" Title="How can we print weights per iteration in a simple feed forward MLP for an specific class?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;feedforward-neural-network&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16178" PostTypeId="2" ParentId="16177" CreationDate="2019-10-31T13:38:51.830" Score="1" Body="&lt;p&gt;A common model used for this kind of classification task is to have one output neuron per class. So, for example, neuron 1 may have a loss function that is related to outputting &quot;1&quot; for examples of class 1, and &quot;0&quot; for examples of other classes. Neuron 2 may be asked to do the same, but for class 2 rather than class 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use a model of this kind, you can pull the weights for each neuron in the final output layer. It sounds like this is what you are being asked to plot.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-31T13:38:51.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16181" PostTypeId="2" ParentId="16172" CreationDate="2019-10-31T16:04:59.380" Score="0" Body="&lt;p&gt;VGG is a more basic architecture which uses no residual blocks. Reset usually perform better then VGG due to it's more layers and residual approach. Given that resnet-50 can get 99% accuracy on MNIST and 98.7% accuracy on CIFAR-10, it probably should achieve better than VGG network. Also, the validation accuracy should not be 100%. You could try increasing the size of your validation set to improve accuracy on validation. VGG network should perform worst than ResNet in most scenario, but experimenting is the way to go. Try and experiment more to get a method that works for your data. Hope that I can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-10-31T16:04:59.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16182" PostTypeId="1" AcceptedAnswerId="16190" CreationDate="2019-10-31T16:15:47.053" Score="4" ViewCount="155" Body="&lt;p&gt;I am quite new to the Reinforcement Learning domain and I am curious about something. It seems to be the case that the majority of current research assumes Markovian environments, that is, future states of the process depend only upon the present state, not on the sequence of events that preceded it. I was curious about how we can assign rewards when the Markovian property doesn't hold anymore. Do the state-of-the-art RL theory and research support this? &lt;/p&gt;&#xA;" OwnerUserId="30960" LastEditorUserId="2444" LastEditDate="2019-10-31T20:15:00.920" LastActivityDate="2019-11-01T15:10:22.147" Title="How to assign rewards in a non-Markovian environment?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;markov-decision-process&gt;&lt;environment&gt;&lt;markov-property&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16183" PostTypeId="1" CreationDate="2019-10-31T17:02:35.590" Score="3" ViewCount="24" Body="&lt;p&gt;I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters considered in NAS?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NAS aims to find a good performing model in the search space of all possible model architectures using a certain search- and performance estimation strategy.&#xA;There are architecture-specific hyper-parameters (in the most simple feed-forward network case) like the number of hidden layers, the number of hidden neurons per layer as well as the type of activation function per neuron&#xA;There are classical hyper-parameters like learning rate, dropout rate, etc.&#xA;What I don't understand is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What exactly is part of the model architecture as defined above? Is it only the architecture-specific hyper-parameters or also the classical hyper-parameters? In other words, what is spanning the search space in NAS: Only the architecture-specific hyper-parameters or also the classical hyper-parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case only the architecture-specific hyper-parameters are part of the NAS search space, what about the classical hyper-parameters? A certain architecture (with a fixed configuration of the architecture-specific hyper-parameters) might perform better or worse depending on the classical hyper-parameters - so not taking into account the classical hyper-parameters in the NAS search space might result in a non-optimal ultimate model architecture, or not?&lt;/p&gt;&#xA;" OwnerUserId="30909" LastEditorUserId="2444" LastEditDate="2019-10-31T20:18:43.427" LastActivityDate="2019-10-31T20:18:43.427" Title="Which hyper-parameters are considered in neural architecture search?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16184" PostTypeId="1" AcceptedAnswerId="16188" CreationDate="2019-10-31T17:20:34.147" Score="1" ViewCount="77" Body="&lt;p&gt;I'm building an MLP with scikit-learn. Is there a way I can access the weights and biases of the output layer per iteration? There is an option &lt;code&gt;mlp.coefs_&lt;/code&gt;, but it outputs the trained weights when the model is compiled. I need weights per iterations how can I get weights and biases per iteration?&lt;/p&gt;&#xA;" OwnerUserId="21084" LastEditorUserId="2444" LastEditDate="2019-10-31T17:26:24.100" LastActivityDate="2019-10-31T21:20:00.637" Title="How can I access the weights at each training iteration of an MLP with scikit-learn?" Tags="&lt;neural-networks&gt;&lt;multilayer-perceptron&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16185" PostTypeId="5" CreationDate="2019-10-31T20:12:52.930" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-31T20:12:52.930" LastActivityDate="2019-10-31T20:12:52.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16186" PostTypeId="4" CreationDate="2019-10-31T20:12:52.930" Score="0" Body="For questions related to the concept of Markov decision process (MDP), which is a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision-maker. The concept of MDP is useful for studying optimization problems solved via dynamic programming and reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-10-31T20:12:52.930" LastActivityDate="2019-10-31T20:12:52.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16187" PostTypeId="1" AcceptedAnswerId="25236" CreationDate="2019-10-31T21:19:14.077" Score="2" ViewCount="34" Body="&lt;p&gt;I am reading the paper &lt;a href=&quot;https://openaccess.thecvf.com/content_ECCV_2018/papers/Yanbei_Chen_Semi-Supervised_Deep_Learning_ECCV_2018_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semi-Supervised Deep Learning with Memory&lt;/a&gt; (2018) by Yanbei Chen et al.  The topic is the classification of images using semi-supervised learning. The authors use a term on page 2 in the middle of the page that I am not familiar with. They write:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The key to our framework design is two-aspect: (1) the class-level discriminative feature representation and the network inference uncertainty are gradually accumulated in an external memory module; (2) this memorised information is utilised to assimilate the newly incoming image samples on-the-fly and generate an informative unsupervised memory loss to guide the network learning jointly with the supervised classification loss&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not sure what the term &lt;em&gt;discriminative feature representation&lt;/em&gt; means.&lt;/p&gt;&#xA;&lt;p&gt;I know that a discriminative model determines the decision boundary between the classes, and examples include: Logistic Regression (LR), Support Vector Machine (SVM), conditional random fields (CRFs) and others.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I know that, in machine learning, feature learning or representation learning is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data.&lt;/p&gt;&#xA;&lt;p&gt;Any insights on the definition of this term much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30962" LastEditorUserId="2444" LastEditDate="2020-12-17T11:49:34.713" LastActivityDate="2020-12-17T11:49:34.713" Title="What does &quot;class-level discriminative feature representation&quot; mean in the paper &quot;Semi-Supervised Deep Learning with Memory&quot;?" Tags="&lt;terminology&gt;&lt;papers&gt;&lt;feature-learning&gt;&lt;semi-supervised-learning&gt;&lt;discriminative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16188" PostTypeId="2" ParentId="16184" CreationDate="2019-10-31T21:20:00.637" Score="2" Body="&lt;p&gt;You can use the MLP function &lt;code&gt;partial_fit&lt;/code&gt; to perform a single training iteration at a time. If you do retrieve the weights between calls to this function, you can see what they look like after each iteration.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-10-31T21:20:00.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16190" PostTypeId="2" ParentId="16182" CreationDate="2019-10-31T23:35:44.103" Score="2" Body="&lt;p&gt;Dealing with a Non-Markovian process is unusual in Reinforcement Learning. Although some &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/000437029400012P&quot; rel=&quot;nofollow noreferrer&quot;&gt;explicit&lt;/a&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4405578/&quot; rel=&quot;nofollow noreferrer&quot;&gt;attempts&lt;/a&gt; have been made, the most common approach when confronted with a non-Markovian environment is to try and make the agent's representation of it Markovian. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reducing Agent's model of the dynamics to a Markovian process, rewards are assigned from the environment in exactly the same way as before. The environment simply sends the agent a reward signal in response to each action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Markovian assumption is essentially a formalism of the idea that the future can be predicted from the present. It says that if you know the dynamics of a system, and you know the state of the system now, you know everything you need to predict the state of the system later, and how we got to this state is not important. Formally, we write this as &lt;span class=&quot;math-container&quot;&gt;$P(s_t∣s_{t−1:0})=P(s_t∣s_{t−1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the models we use in AI are usually &lt;em&gt;simplifications&lt;/em&gt; of the real world. When we simplify the world, we can introduce non-Markovian dynamics. However, if the model grows too complex, and the state space too large, learning will take too long. The goal is then to define a state space that is small enough to be learnable, and not &lt;em&gt;too bad&lt;/em&gt; an approximation of the real dynamics of the world. AI researchers have several tools to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a working example, imagine that the future position of a robot depends mainly on the current position, and current velocity, along with the action the robot takes right now. Using these variables to define a state, we get almost Markovian dynamics, but as the robot moves over time, its battery drains and the movements become very slightly more imprecise. If we wanted to remove this error, we can:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Expand the state variables&lt;/strong&gt;. If we add &quot;current battery level&quot; to the state, then our process will become Markovian again. This is the most obvious approach, but it only works up to a point. The state space grows exponentially in size as you add new variables. It will quickly become too large to learn, so in many problems, a complex state is subsequently decomposed into simpler sub-states. These simpler states may not depend on one another at all, or may depend only to varying degrees. The main limiting factor in learning to navigate an exponential state space is that the number of parameters in our model will grow exponentially. If the original space had &lt;span class=&quot;math-container&quot;&gt;$O(2^n)$&lt;/span&gt; parameters, splitting it in half will yield two seperate learning problems of size &lt;span class=&quot;math-container&quot;&gt;$O(2^{n/2} + 2^{n/2} = 2^{n/2 + 1})$&lt;/span&gt;. Splitting the problem in two reduces its size by a factor &lt;span class=&quot;math-container&quot;&gt;$O(2^{n/2})$&lt;/span&gt;, which is big. This is the technique exploited by &lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_Bayesian_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Bayesian Networks&lt;/a&gt;. In the case of our robot, we could make current &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; position depend only on previous &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; position, previous &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; velocity, and the robot's action, rather than on all previous variables, and likewise for the other variables.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Use Higher Order Models of the Environment.&lt;/strong&gt; This is really a generalization of (1) above. Instead of the state being the current location and velocity of the robot, we could define the state to be the current location/velocity &lt;strong&gt;and&lt;/strong&gt; the location/velocity of the robot 1 step in the past (or 2, or 3, or &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; steps). This increases the size of the state space enormously, but if we don't know &lt;em&gt;why&lt;/em&gt; there is an error in the robot's movements, this can still allow us to model it. In this case, if we know the size of the change in the robot's position last time, and we observe that it is smaller this time (for the same action), we can estimate the rate at which the change is changing, without understanding its cause.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As an example, consider the process of setting the price of a good. The agent's reward is non-Markovian, because sales increase or decline gradually in response to price changes. However, they don't depend on &lt;em&gt;all&lt;/em&gt; of the history of prices. Imagine that instead they depend on the last 5 prices (or the last k). We can use technique (2) above to expand the agent's model of what a state is. The now the agent learns that when prices have been &lt;span class=&quot;math-container&quot;&gt;$p_{t-1:t-5}$&lt;/span&gt; in the last 5 steps, and it sets the price at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to some value, it's reward is &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Since the reward depends only on the prices now, and in the last 5 steps, the agent is now learning a Markovian process, even though the original process is non-Markovian, and the reward function is non-Markovian. No changes are made to the reward function, or the environment, only the agent's model of the environment.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-01T15:10:22.147" LastActivityDate="2019-11-01T15:10:22.147" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="16191" PostTypeId="1" CreationDate="2019-11-01T01:08:15.463" Score="1" ViewCount="140" Body="&lt;p&gt;I use Google's Cloud TPU hardware extensively using Tensorflow for training models and inference, however, when I run inference I do it in large batches. The TPU takes about 3 minutes to warm up before it runs the inference. But when I read the &lt;a href=&quot;https://cloud.google.com/tpu/docs/faq&quot; rel=&quot;nofollow noreferrer&quot;&gt;official TPU FAQ&lt;/a&gt;, it says that we can do real-time inference using TPU. It says the latency is 10ms which for me is fast enough but I cannot figure out how to write code that does this, since every time I want to pass something for inference I have to start the TPU again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to run large Transformer-based Language Models in real-time on TPUs. I guessed that TPUs would be ideal for this problem. Even Google seems to &lt;a href=&quot;https://www.blog.google/products/search/search-language-understanding-bert/&quot; rel=&quot;nofollow noreferrer&quot;&gt;already do this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quote from the &lt;a href=&quot;https://cloud.google.com/tpu/docs/faq&quot; rel=&quot;nofollow noreferrer&quot;&gt;official TPU FAQ&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Executing inference on a single batch of input and waiting for the&#xA;  result currently has an overhead of at least 10 ms, which can be&#xA;  problematic for low-latency serving.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="29999" LastEditorUserId="2444" LastEditDate="2019-11-01T03:15:52.040" LastActivityDate="2019-11-01T03:15:52.040" Title="How to use TPU for real-time low-latency inference?" Tags="&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;transformer&gt;&lt;inference&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16192" PostTypeId="5" CreationDate="2019-11-01T01:54:15.223" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T01:54:15.223" LastActivityDate="2019-11-01T01:54:15.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16193" PostTypeId="4" CreationDate="2019-11-01T01:54:15.223" Score="0" Body="For questions related to grammar induction (or grammar inference), which is the problem, in machine learning, of learning a formal grammar from a set of observations (a dataset)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:00:00.060" LastActivityDate="2019-11-01T02:00:00.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16194" PostTypeId="5" CreationDate="2019-11-01T02:26:05.883" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:26:05.883" LastActivityDate="2019-11-01T02:26:05.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16195" PostTypeId="4" CreationDate="2019-11-01T02:26:05.883" Score="0" Body="For questions related to BERT (which stands for Bidirectional Encoder Representations from Transformers), a language representation model introduced in the paper &quot;BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding&quot; (2019) by Google." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:26:05.883" LastActivityDate="2019-11-01T02:26:05.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16196" PostTypeId="5" CreationDate="2019-11-01T02:30:22.400" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:30:22.400" LastActivityDate="2019-11-01T02:30:22.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16197" PostTypeId="4" CreationDate="2019-11-01T02:30:22.400" Score="0" Body="For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper &quot;Long-Short Term Memory&quot;." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:30:22.400" LastActivityDate="2019-11-01T02:30:22.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16198" PostTypeId="5" CreationDate="2019-11-01T02:44:49.813" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:44:49.813" LastActivityDate="2019-11-01T02:44:49.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16199" PostTypeId="4" CreationDate="2019-11-01T02:44:49.813" Score="0" Body="For questions related to the transformer, which is a deep machine learning model introduced in 2017 in the paper &quot;Attention Is All You Need&quot;, used primarily in the field of natural language processing (NLP)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:44:49.813" LastActivityDate="2019-11-01T02:44:49.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16200" PostTypeId="5" CreationDate="2019-11-01T02:57:32.453" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:57:32.453" LastActivityDate="2019-11-01T02:57:32.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16201" PostTypeId="4" CreationDate="2019-11-01T02:57:32.453" Score="0" Body="For questions related to GPT (which stands for Generative Pre-Training), which is a combination of transformers (proposed in &quot;Attention is All You Need&quot;) and unsupervised pre-training for solving language tasks, such as machine translation. GPT was proposed in &quot;Improving Language Understanding by Generative Pre-Training&quot; (2018) by Open AI. There's also GPT-2, which was proposed in &quot;Language Models are Unsupervised Multitask Learners&quot; (2019) by Open AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T02:57:32.453" LastActivityDate="2019-11-01T02:57:32.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16202" PostTypeId="5" CreationDate="2019-11-01T03:00:01.297" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T03:00:01.297" LastActivityDate="2019-11-01T03:00:01.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16203" PostTypeId="4" CreationDate="2019-11-01T03:00:01.297" Score="0" Body="For questions related to the concept of a language model, which is a probability distribution over sequences of words (for example, of a natural language, such as English)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-01T03:00:01.297" LastActivityDate="2019-11-01T03:00:01.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16204" PostTypeId="2" ParentId="3494" CreationDate="2019-11-01T03:14:48.847" Score="1" Body="&lt;p&gt;I actually prefer C for machine learning. Because like in life, in the world as we know it, consists of never-ending &quot;logic gates&quot; (which basically is like flipping a coin - there WILL be 2 possible outcomes - not counting the third: landing on the side!). Which also means that while the universe seems never-ending, we still never stop finding those things that are even smaller than the last smallest thing, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So... To put it in a context when programming C, I can control the memory usage more efficiently by coding smaller snippets that get combined, to always form smaller &amp;amp; efficient &quot;code-fragments&quot;, that make up what we would call &quot;cells&quot; in biology (it got a measurable function, and has some pre-set properties).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I like to optimize for low RAM usage, low CPU usage etc. when programming AI. I have only done feedforward with a basic genetic algorithm in C, but the more advanced recurrent neural network I wrote in C++ (ONLY because of the simplicity of using &lt;code&gt;std::vector&amp;lt;TYPE&amp;gt; name;&lt;/code&gt;, so I wrote my own &lt;code&gt;cvector.c&lt;/code&gt;: &lt;a href=&quot;https://pastebin.com/sBbxmu9T&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/sBbxmu9T&lt;/a&gt;  &amp;amp; &lt;code&gt;cvector.h&lt;/code&gt;: &lt;a href=&quot;https://pastebin.com/Rd8B7mK4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/Rd8B7mK4&lt;/a&gt; &amp;amp; debug: &lt;a href=&quot;https://pastebin.com/kcGD8Fzf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/kcGD8Fzf&lt;/a&gt; - compile with &lt;code&gt;gcc -o debug debug.c cvector.c&lt;/code&gt;). That actually helped a lot in the quest of optimizing CPU usage (and overall runtime) when creating optimized neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;So I am in one sense really see the opposite of what AlexPnt sees, when it comes to exploring what is possible within the realm of a &quot;self&quot;.&lt;/p&gt;&#xA;" OwnerUserId="30967" LastEditorUserId="5300" LastEditDate="2020-04-14T22:09:29.960" LastActivityDate="2020-04-14T22:09:29.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16206" PostTypeId="1" CreationDate="2019-11-01T07:55:48.913" Score="1" ViewCount="7" Body="&lt;p&gt;Suppose that we want to segment a red blob from the image, normally you will have a class for this red blob e.g. 0. And every red blob you detected will have a class of 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, in my case, I want that the model will look at the surrounding context, e.g., if the red blob is surrounded by blue blobs, it should be classified as class 1, instead of 0. Like the following image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o9F3F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o9F3F.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this something easily achieve able with U-Net or other models (you can suggest)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, the context can be more difficult than this, e.g., if there are blue and green surrounding you, you will have another class.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastEditorUserId="2444" LastEditDate="2020-06-13T00:03:39.250" LastActivityDate="2020-06-13T00:03:39.250" Title="Would models like U-Net be able to segment objects which has label based on its surrounding context?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16207" PostTypeId="1" CreationDate="2019-11-01T09:20:14.807" Score="1" ViewCount="21" Body="&lt;p&gt;I am building a CNN with two outputs. I still have to put time in the network itself, but I was trying to get a good evaluation/classification report of the results. My code is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scores = model.evaluate(data_test, [Y1_test, Y2_test], verbose=0)&#xA;&#xA;for i, j in zip(model.metrics_names, scores):&#xA;    print(i,'=', j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loss = 5.124477842579717&#xA;Y1_output_loss = 1.3782909&#xA;Y2_output_loss = 4.10769&#xA;Y1_output_accuracy = 0.6304348&#xA;Y2_output_accuracy = 0.54347825&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not great, but that is not the point. My code for the classification repot is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y1_pred, Y2_pred = model.predict(data_test)&#xA;Y1_true, Y2_true = Y1_test.argmax(axis=-1), Y2_test.argmax(axis=-1)&#xA;Y1_pred, Y2_pred = Y1_pred.argmax(axis=-1), Y2_pred.argmax(axis=-1)&#xA;&#xA;&#xA;print(classification_report(Y1_true, Y1_pred))&#xA;print(classification_report(Y2_true, Y2_pred))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Classification report Y1&#xA;              precision    recall  f1-score   support&#xA;&#xA;           0       0.20      0.33      0.25         6&#xA;           3       0.00      0.00      0.00         3&#xA;           6       0.00      0.00      0.00         6&#xA;           8       0.00      0.00      0.00         2&#xA;           9       0.00      0.00      0.00         7&#xA;          10       0.03      0.50      0.06         2&#xA;          11       0.00      0.00      0.00         3&#xA;          12       0.00      0.00      0.00         7&#xA;          13       0.00      0.00      0.00         2&#xA;          14       0.00      0.00      0.00         7&#xA;          15       0.00      0.00      0.00         1&#xA;&#xA;    accuracy                           0.07        46&#xA;   macro avg       0.02      0.08      0.03        46&#xA;weighted avg       0.03      0.07      0.04        46&#xA;&#xA;&#xA;Classification report Y2&#xA;              precision    recall  f1-score   support&#xA;&#xA;           0       0.00      0.00      0.00         9&#xA;           2       0.00      0.00      0.00        10&#xA;           3       0.15      1.00      0.26         7&#xA;           4       0.00      0.00      0.00         9&#xA;           5       0.00      0.00      0.00         6&#xA;           6       0.00      0.00      0.00         2&#xA;           7       0.00      0.00      0.00         3&#xA;&#xA;    accuracy                           0.15        46&#xA;   macro avg       0.02      0.14      0.04        46&#xA;weighted avg       0.02      0.15      0.04        46&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the average accuracy is extremely low suddenly, so I have the feeling it isn't lining up correctly. But I don't see where? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all&lt;/p&gt;&#xA;" OwnerUserId="30971" LastActivityDate="2019-11-01T09:20:14.807" Title="CNN multi output scores and evaluation" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16208" PostTypeId="1" CreationDate="2019-11-01T09:31:44.733" Score="1" ViewCount="36" Body="&lt;p&gt;I want to build a Voice Assistance using Tensorflow, Like Google Assistance, So that I can give commands like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Open Camera&#xA;Send Message&#xA;Play Music&#xA;ETC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know I can use pre-trained model for Voice Recognition but this is not my problem. If I know correctly, What Neural Network does it learn from your input and output and creates a best algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to know Is it possible to somehow train my network for my commands so that I don't have to HARD-CODED them because it is hard to remember so many command?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is Google also Hard-Coded the commands for Google Voice Assistance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, If I'm unable to explain this to you :P&lt;/p&gt;&#xA;" OwnerUserId="30970" LastEditorUserId="30970" LastEditDate="2019-11-01T11:05:55.967" LastActivityDate="2019-11-01T12:12:31.410" Title="Best way to train Neural Network for Voice Commands?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16210" PostTypeId="2" ParentId="16208" CreationDate="2019-11-01T12:12:31.410" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What Neural Network does it learn from your input and output and creates a best algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is inaccurate. A neural network is a &lt;em&gt;function approximator&lt;/em&gt;, so it approximates an unknown function. However, it does so in many cases by learning from your input and output(as in the case of supervised learning). The function your approximating would here for example be the function that maps a certain soundwave input to &quot;open camera&quot;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with supervised learning is that you need massive datasets to accurately approximate the target function - luckily for Google they have such datasets and resources available.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is Google also Hard-Coded the commands for Google Voice Assistance?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, because that would not generalize well over multiple languages/accents/background noise, etc. It is therefor that neural networks as function approximators have had such a large impact over the last years - they generalize well.&#xA;If you remember when the first &quot;voice assistance&quot; products came out, they rarely functioned - this was often due to them being &quot;hard coded&quot;, with only static measures for noise-suppression.&lt;/p&gt;&#xA;" OwnerUserId="30565" LastActivityDate="2019-11-01T12:12:31.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16211" PostTypeId="2" ParentId="16162" CreationDate="2019-11-01T12:22:32.630" Score="1" Body="&lt;p&gt;The first thing that comes to mind when reading your question is &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic algorithms&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They create alternate versions of themselves and measure each versions performance on a specific task, before discarding those that work poorly, while keeping the best ones for their next generation.&#xA;The mutations here are often random, and for large/complex problems, these simulations can take incredibly long time.&#xA;This group of algorithms are heavily inspired by evolution and biology, as you can see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize as I read the last part of your question, that this might be on a much smaller scope than you had envisioned. But, in essence genetic algorithms does what you describe in your first part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the more grand-scale question, see @Neil Slater's answer.&lt;/p&gt;&#xA;" OwnerUserId="30565" LastEditorUserId="16909" LastEditDate="2019-11-04T16:09:51.837" LastActivityDate="2019-11-04T16:09:51.837" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16212" PostTypeId="1" AcceptedAnswerId="16215" CreationDate="2019-11-01T12:59:24.073" Score="5" ViewCount="87" Body="&lt;p&gt;I'm studying the paper &amp;quot;&lt;a href=&quot;https://pdfs.semanticscholar.org/f1b8/afdbf9f108afdb42476850ed3fc13db43cea.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimizing Total Tardiness on a Single Machine Using Ant Colony Optimization&lt;/a&gt;&amp;quot; which has proposed to use Ant colony optimization to SMTWTP.&lt;/p&gt;&#xA;&lt;p&gt;According to this paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Each artificial ant iteratively and independently decides which job to&#xA;append to the sub-sequence generated so far until all jobs are&#xA;scheduled, Each ant generates a complete solution by selecting a job &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;&#xA;to be on the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th position of the sequence. This selection process is&#xA;influenced through problem-specific heuristic information called&#xA;visibility and denoted by &lt;span class=&quot;math-container&quot;&gt;$\eta_{ij}$&lt;/span&gt; as well as pheromone trails denoted by &lt;span class=&quot;math-container&quot;&gt;$\tau_{ij}$&lt;/span&gt;. The former is an indicator of how good the choice of that job&#xA;seems to be and the latter indicates how good the choice of that job&#xA;was in former runs. Both matrices are only two dimensional as a&#xA;consequence of the reduction in complexity&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They have proposed this formula for the probability that job &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; be selected to be processed on position &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; (page 9 of the linked paper):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{P}_{i j}=\left\{\begin{array}{cl}&#xA;\frac{\left[\tau_{i j}\right]^{\alpha}\left[\eta_{i j}\right]^{\beta}}{\sum_{h \in \Omega}\left[\tau_{i h}\right]^{\alpha}\left[\eta_{i h}\right]^{\beta}} &amp;amp; \text { if } j \in \Omega \\&#xA;0 &amp;amp; \text { otherwise }&#xA;\end{array}\right.\tag{1}\label{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;but I can't understand what &lt;span class=&quot;math-container&quot;&gt;$[]$&lt;/span&gt; surrounding &lt;span class=&quot;math-container&quot;&gt;$\eta_{ij}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tau_{ij}$&lt;/span&gt; indicates. Does it show that these values are matrices?&lt;/p&gt;&#xA;" OwnerUserId="30164" LastEditorUserId="2444" LastEditDate="2021-01-15T11:56:32.447" LastActivityDate="2021-01-15T11:56:32.447" Title="What is the meaning of the square brackets in ant colony optimization?" Tags="&lt;papers&gt;&lt;swarm-intelligence&gt;&lt;notation&gt;&lt;ant-colony-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16213" PostTypeId="1" CreationDate="2019-11-01T13:03:45.053" Score="2" ViewCount="161" Body="&lt;p&gt;This is my first question in this community (Artificial Intelligence) and I hope I can get some answers to a troublesome issue of mine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to understand Stochastic Gradient Decent (SGD) (with possibly momentum) and modify the &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/keras/optimizers.py#L164&quot; rel=&quot;nofollow noreferrer&quot;&gt;keras version&lt;/a&gt; to an accumulative one. I have asked a &lt;a href=&quot;https://stackoverflow.com/questions/58522655/explaining-sgd-in-keras-functionality&quot;&gt;similar question&lt;/a&gt; on Stack Overflow but here I am interested in the actual function (and secondarily to the code itself).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity let's consider I have a neural network with just 10 layers and I am training for 100 iterations (regardless of epochs).&lt;br&gt;&#xA;For easier reference I post the keras code here (and also skipped nesterov and constraints lines of code):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_updates(self, loss, params):&#xA;    grads = self.get_gradients(loss, params)&#xA;    self.updates = [K.update_add(self.iterations, 1)]&#xA;&#xA;    lr = self.lr&#xA;    if self.initial_decay &amp;gt; 0:&#xA;        lr = lr * (1. / (1. + self.decay * K.cast(self.iterations, K.dtype(self.decay))))&#xA;    shapes = [K.int_shape(p) for p in params]&#xA;    moments = [K.zeros(shape) for shape in shapes]&#xA;    self.weights = [self.iterations] + moments&#xA;    for p, g, m in zip(params, grads, moments):&#xA;        v = self.momentum * m - lr * g  # velocity&#xA;        self.updates.append(K.update(m, v))&#xA;        new_p = p + v&#xA;&#xA;        self.updates.append(K.update(p, new_p))&#xA;    return self.updates&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I understand the mechanism of SGD works as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The gradients are updated at every iteration, which in the keras code is performed by calling the &lt;code&gt;get_updates(self, loss, params)&lt;/code&gt; function.&lt;/li&gt;&#xA;&lt;li&gt;There is a gradient that correspond to each layer (in the keras code I think a list of Tensors are produced to keep the gradients).&lt;/li&gt;&#xA;&lt;li&gt;So, in that sense the &lt;code&gt;for p, g, m in zip(params, grads, moments)&lt;/code&gt; loop just iterates through all layers (and their corresponding gradients and moments) and calculates new values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The really confusing part for me is that if the above function is being called at every iteration then the &lt;code&gt;moments = [K.zeros(shape) for shape in shapes]&lt;/code&gt; basically initializes &lt;code&gt;moments&lt;/code&gt; to a zero Tensor at every iteration. Wouldn't make more sense if &lt;code&gt;moment&lt;/code&gt; was initialized once and then keep updating with new values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have noticed that despite my understanding the moments are actually used (as it would be more expected for such a big project to have missed a large bug such as this one). I cannot really understand how though.&lt;/p&gt;&#xA;" OwnerUserId="21902" LastActivityDate="2020-05-16T04:07:10.677" Title="How exactly does SGD with momentum works in keras" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-05-16T15:23:11.513" ContentLicense="CC BY-SA 4.0" />
  <row Id="16214" PostTypeId="1" CreationDate="2019-11-01T13:45:47.640" Score="2" ViewCount="190" Body="&lt;p&gt;The thing about machine learning (ML) that worries me is that &amp;quot;knowledge&amp;quot; acquired in ML is hidden: we usually can't explain the criteria or methods used by the machine to provide an answer when we ask it a question.&lt;/p&gt;&#xA;&lt;p&gt;It's as if we asked an expert financial analyst for advice and he/she replied, &amp;quot;Invest in X&amp;quot;; then when we asked &amp;quot;Why?&amp;quot;, the analyst answered, &amp;quot;Because I have a feeling that's the right thing for you to do.&amp;quot;  It makes us dependent on the analyst.&lt;/p&gt;&#xA;&lt;p&gt;Surely there are some researchers trying to find ways for ML systems to encapsulate and refine their &amp;quot;knowledge&amp;quot; into a form that can then be taught to a human or encoded into a much simpler machine.  Who, if any, are working on that?&lt;/p&gt;&#xA;" OwnerUserId="28348" LastEditorUserId="2444" LastEditDate="2020-07-29T21:41:08.773" LastActivityDate="2021-01-06T13:25:46.587" Title="Who is working on explaining the knowledge encoded into machine learning models?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;knowledge-representation&gt;&lt;explainable-ai&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="1" ClosedDate="2021-01-05T11:08:30.413" ContentLicense="CC BY-SA 4.0" />
  <row Id="16215" PostTypeId="2" ParentId="16212" CreationDate="2019-11-01T14:56:44.023" Score="3" Body="&lt;p&gt;The square brackets &lt;span class=&quot;math-container&quot;&gt;$[]$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$[\tau_{ij}]^\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[\eta_{ij}]^\beta$&lt;/span&gt; may be just a way of emphasing that the elements &lt;span class=&quot;math-container&quot;&gt;$\tau_{ij} \in \mathbb{R}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta_{ij} \in \mathbb{R}$&lt;/span&gt; of respectively the matrices &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\tau} \in \mathbb{R}^{n \times n}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\eta} \in \mathbb{R}^{n \times n}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of nodes in the graph) are respectively raised to &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;, so they could have used also other type of brackets, for example, &lt;span class=&quot;math-container&quot;&gt;$()$&lt;/span&gt;. It may also be a way of indicating that &lt;span class=&quot;math-container&quot;&gt;$[\tau_{ij}]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[\eta_{ij}]$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; matrices or vectors that contain respectively the scalars &lt;span class=&quot;math-container&quot;&gt;$\tau_{ij}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta_{ij}$&lt;/span&gt;, so you are multiplying matrices or vectors (dot product). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This notation is also used in the paper that introduced the &lt;em&gt;ant colony system&lt;/em&gt; (ACS) (and it is probably used in many other papers related to ant colony optimization). See equation 1 of &lt;a href=&quot;http://people.idsia.ch/~luca/acs-ec97.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem&lt;/a&gt; (1997) by Dorigo and Gambardella.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-02T01:08:42.910" LastActivityDate="2019-11-02T01:08:42.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16216" PostTypeId="2" ParentId="16214" CreationDate="2019-11-01T15:05:43.957" Score="0" Body="&lt;p&gt;I have one concept, which allows to structure knowledge gathered by a ML system in both categorical and algorithms-like structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key idea here is that we contain in our minds some network-like structure in our minds, which helping us not only to classify some data, but also to form an abstract meaning about text or message.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than that, in opposite to our minds, it looks like &lt;a href=&quot;https://stackoverflow.com/questions/31268796/concept-based-text-summarization-abstraction/31272242#31272242&quot;&gt;this&lt;/a&gt; solution can produce these structures in readable for us forms.&lt;/p&gt;&#xA;" OwnerUserId="8861" LastActivityDate="2019-11-01T15:05:43.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16217" PostTypeId="1" CreationDate="2019-11-01T16:34:57.237" Score="2" ViewCount="23" Body="&lt;p&gt;Say I have a game like tic-tac-toe or chess. Or some other visual logic based problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could express the state of the game as a string. (or perhaps a 2D array)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to be able to express the possible moves of the game as rules which change the string. e.g. replacement rules. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked into regex as a possibility but this doesn't seem powerful enough. For example, one can't have named patterns which one can use again. (e.g. if I wanted to name a pattern called &quot;numbers_except_for_8&quot;. To be used again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it also should be able to express things like &quot;repeat if possible&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words I need some simple language to express rules of a game that has:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;modularity&lt;/li&gt;&#xA;&lt;li&gt;simpleness&lt;/li&gt;&#xA;&lt;li&gt;can act on other rules (self referential)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are languages like LISP but these on the other hand seem too complicated. (Perhaps there is no simple language hence why the English language is so complicated).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did read once about a generalised board game solving software program which had a way to express the rules of a game. But I can't seem to find a reference to it anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example rules for tic tac toe might be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Players-Turn:&#xA;&quot;Find a blank square&quot;-&gt;&quot;Put an X in it&quot;-&gt;Oponent's turn&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oponents-Turn:&#xA;&quot;Find a blank square&quot;-&gt;&quot;Put an O in it&quot;-&gt;Player's turn&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think the ingredients for rules are: searching for patterns, determining if an object is of a particular type (which might be the same as the first ingredient), and replacing.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2019-11-01T16:40:56.037" LastActivityDate="2019-11-01T16:40:56.037" Title="What is a good language for expressing replacement or template rules?" Tags="&lt;programming-languages&gt;&lt;meta-rules&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16218" PostTypeId="1" CreationDate="2019-11-01T23:27:58.363" Score="2" ViewCount="27" Body="&lt;p&gt;In recent years if you are working on stereo depth/disparity algorithms, it seems like you will only ever get your paper accepted to CVPR/ICCV/ECCV if there's some deep learning involved in it. A lot of authors published their code on github and I've tried out multiple of them and here is what I observed. None of these deep learning based methods generalized well. Almost all methods trained on the &lt;a href=&quot;http://www.cvlibs.net/datasets/kitti/eval_stereo.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;KITTI dataset&lt;/a&gt; (street images) or the &lt;a href=&quot;https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scene flow dataset&lt;/a&gt; (synthetic images). These methods perform well when the test data is similar to the training data, but fails miserably on other kinds of test data (e.g. close up human) whereas a classical traditional computer vision based method like &lt;a href=&quot;https://www.microsoft.com/en-us/research/publication/patchmatch-stereo-stereo-matching-with-slanted-support-windows/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PatchMatch&lt;/a&gt; would generate decent results. In my opinion, no matter how well these new deep learning methods perform on the KITTI benchmark, it's nearly useless in the real world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand deep learning has the potential to approximate any non-linear function when there's enough quality training data and unlimited computation, but ground truth depth/disparity cannot be labeled by manual labor like a cat-dog classification problem. That means the ground truth training data has to come from traditional computer vision algorithms or hardware or be synthetic. Traditional computer vision algorithms are not even close to perfect yet but the research pretty much stifled because of deep learning. The ground truth of the KITTI dataset comes from a hardware LIDAR, but it's extremely sparse. If we align multiple scans from LIDAR in order to form a dense result, that's relying on some type of SLAM which again is relying on an imperfect traditional computer vision algorithm. There is no sign of hardware that can generate accurate dense depth that is coming out soon. As for synthetic data, it doesn't accurately represent real data. Since there isn't even a good way to obtain training data for stereo depth/disparity, why are the researchers so fixated on building complex deep neural nets to solve stereo depth/disparity nowadays?&lt;/p&gt;&#xA;" OwnerUserId="30980" LastActivityDate="2019-11-01T23:27:58.363" Title="Why are researchers focused on deep learning based stereo depth/disparity methods instead of non deep learning ones?" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16219" PostTypeId="1" CreationDate="2019-11-02T01:22:26.637" Score="1" ViewCount="184" Body="&lt;p&gt;In section 5 of the paper &lt;a href=&quot;https://arxiv.org/pdf/1812.05905v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Soft Actor-Critic Algorithms and Applications&lt;/em&gt;&lt;/a&gt;, it is proposed an optimization problem to obtain an optimal temperature parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha^*_t$&lt;/span&gt;. First, one uses the original evaluation and improvement steps to estimate &lt;span class=&quot;math-container&quot;&gt;$Q_t^*$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_t^*$&lt;/span&gt;, and then one somehow solves the optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\alpha_t^* = \arg\min_{\alpha_t} \mathbb E _{a_t\sim\pi^*}\left[\alpha_t(-\log\pi_t^*(a_t|s_t;\alpha_t)-H)\right]\text .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, we should use our current estimate of &lt;span class=&quot;math-container&quot;&gt;$\pi_t^*$&lt;/span&gt; to solve that problem. Since it was obtained from a previous &lt;span class=&quot;math-container&quot;&gt;$\alpha_{t-1}^*$&lt;/span&gt;, in practice it is not dependent on &lt;span class=&quot;math-container&quot;&gt;$\alpha_t$&lt;/span&gt; and so the optimization problem becomes a linear problem with the only restriction being &lt;span class=&quot;math-container&quot;&gt;$\alpha_t\geq0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here comes my problem: under this rationale, if &lt;span class=&quot;math-container&quot;&gt;$\alpha_t$&lt;/span&gt; is a scalar independent of both state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;, the value of the cost function is just proportional to &lt;span class=&quot;math-container&quot;&gt;$\alpha_t$&lt;/span&gt; and so the solutions are either &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;, depending on the sign of the expected value (something similar happens if &lt;span class=&quot;math-container&quot;&gt;$\alpha_t^*=\alpha_t^*(s_t,a_t)$&lt;/span&gt;). However, the whole idea of introducing this parameter is to account optimally for the exploration of the policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct way to solve this optimization problem along with the evaluation and improvement steps? I am particularly interested in the &lt;strong&gt;tabular case&lt;/strong&gt;. Also, is there any explanation why they use a negative minimum entropy &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;, when the entropy is always positive? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, in the approximate case the &lt;a href=&quot;https://github.com/rail-berkeley/softlearning/blob/master/softlearning/algorithms/sac.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;current official implementation&lt;/a&gt; seems to be doing just that: moving &lt;span class=&quot;math-container&quot;&gt;$\alpha_t^*$&lt;/span&gt; up or down a little bit (closer to &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt; or 0, respectively), depending on the magnitude of the expected value. I guess one could do the same for the tabular case, modifying the &lt;span class=&quot;math-container&quot;&gt;$\alpha_t^*$&lt;/span&gt; only a little bit in each step, but this seems rather suboptimal.      &lt;/p&gt;&#xA;" OwnerUserId="30983" LastEditorUserId="2444" LastEditDate="2020-11-23T01:51:37.467" LastActivityDate="2020-11-23T01:51:37.467" Title="How does the automated temperature adjustment step work in Soft Actor-Critic?" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;actor-critic-methods&gt;&lt;dynamic-programming&gt;&lt;soft-actor-critic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16220" PostTypeId="1" CreationDate="2019-11-02T03:40:19.473" Score="3" ViewCount="52" Body="&lt;p&gt;I would like a few suggestions on an idea that I have -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to make a musical instrument (percussion), whilst just having a PVC disc. I am hitting the disc in a variety of styles (in order to produce a variety of sounds correspondingly), just like the way the actual percussion instrument is hit. I am converting the mechanical vibrations on the PVC disc to an electrical signal using a transducer, performing an FFT analysis of the different strokes, and trying to identify the stroke which is hit. Using this technique, I could get an accuracy of only 80 percent. I would like it to be extremely accurate ( more than 95 percent recognition). I was using only frequency as the parameter used to distinguish the sounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am thinking that if I could use other parameters too in order to identify the stroke, I might be able to get the required accuracy. I am thinking of resorting to Machine Learning for this. I am kind of new to this and would like to know what I might need to know before I proceed with this idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30985" LastActivityDate="2020-04-01T12:03:25.440" Title="Analyzing vibration using machine learning" Tags="&lt;machine-learning&gt;&lt;signal-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16221" PostTypeId="1" CreationDate="2019-11-02T07:12:37.827" Score="2" ViewCount="25" Body="&lt;p&gt;For example, given a face image, and you want to predict the gender. You also have age information for each person, should you feed the age information as input or should you use it as auxiliary output that the network needs to predict?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I know analytically (instead of experimentally) which approach will be better? What is the logic behind this?&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-11-02T16:10:18.650" Title="Should I use my redundant feature as an auxiliary output or as another input feature?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;feature-selection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16222" PostTypeId="1" CreationDate="2019-11-02T08:42:18.537" Score="2" ViewCount="27" Body="&lt;p&gt;I have a dataset which has two very similar classes (men wrestling, women wrestling). I've used InceptionV3 as a classifier to solve the problem of classifying this dataset. Unfortunately, the accuracy of this classifier doesn't hit more than 70%. Is there any suggestion about how I can overcome this problem or any other similar problems?&lt;/p&gt;&#xA;" OwnerUserId="26472" LastActivityDate="2019-11-02T15:12:07.770" Title="The best way of classifying a dataset including classes with high similarity?" Tags="&lt;classification&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16223" PostTypeId="2" ParentId="16214" CreationDate="2019-11-02T09:00:40.710" Score="0" Body="&lt;p&gt;We actually can explain how the machine answers our questions. For example, we know how the ML works for the case of image recognition. See the following paper by Chris Olah:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://distill.pub/2017/feature-visualization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://distill.pub/2017/feature-visualization/&lt;/a&gt;&#xA;and also&#xA;&lt;a href=&quot;https://distill.pub/2018/building-blocks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://distill.pub/2018/building-blocks/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;By feature visualization, we know that the machine sees at each layer and how it decides using given information. As the images are visual it is easy to &amp;quot;visualize&amp;quot; the features but of course, interpreting inner workings of a Neural Network for other kinds of data (such as financial data, for example) would be much more difficult I presume. But still, we can understand how the initial data is weighted or how the intermediate features are calculated.&lt;/p&gt;&#xA;&lt;p&gt;Given that, I highly doubt the reasoning of a machine can be taught to a human. It is far too complex to be learned. We can easily understand one or two dimensional functions, but the data we use in machine learning problems usually have hundreds of dimensions.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="11539" LastEditDate="2021-01-06T13:25:46.587" LastActivityDate="2021-01-06T13:25:46.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16224" PostTypeId="1" CreationDate="2019-11-02T12:32:52.137" Score="4" ViewCount="665" Body="&lt;p&gt;There are mainly two different areas of AI at the moment. There is the &quot;learning from experience&quot; based approach of neural networks. And there is the &quot;higher logical reasoning&quot; approach, with languages like LISP and PROLOG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has there been much overlap between these (I can't find much!)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple example, one could express some games in PROLOG and then use neural networks to try to play the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a more complicated example, one would perhaps have a set of PROLOG rules which could be combined in various ways, and a neural network to evaluate the usefulness of the rules (by simulation). Or even create new PROLOG rules. &#xA;(Neural networks have been used for language generation of a sort, so why not the generation of PROLOG rules, which could then be evaluated for usefulness by another neural network?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As another example, a machine with PROLOG rules might be able to use a neural network to be able to encode these rules into some language that could be in turn decoded by another machine. And so express instructions to another machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, such a combined system that could use PROLOG rules, combine them, generate new ones, and evaluate them, could be highly intelligent. As it would have access to higher-order logic. And have some similarity to &quot;thinking&quot;.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2019-11-02T14:20:49.577" LastActivityDate="2019-11-03T16:30:56.300" Title="Combining machine learning with languages like ProLog?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;prolog&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16225" PostTypeId="1" CreationDate="2019-11-02T13:52:21.363" Score="3" ViewCount="90" Body="&lt;p&gt;I'm studying ant colony optimization. I'm trying to understand the difference between the ant system (AS) and the max-min ant system (MMAS) approaches. As far as I found out, the main difference between these 2 is that in AS the pheromone trail is updated after all ants have finished the tour (it means all ants participate in this update), but in MMAS, only the best ant updates this value. Am I right? Is there any other significant difference?&lt;/p&gt;&#xA;" OwnerUserId="30164" LastEditorUserId="2444" LastEditDate="2019-11-02T14:16:09.720" LastActivityDate="2021-01-15T11:39:35.903" Title="What is the difference between the ant system and the max-min ant system?" Tags="&lt;comparison&gt;&lt;swarm-intelligence&gt;&lt;ant-colony-optimization&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16226" PostTypeId="1" AcceptedAnswerId="16245" CreationDate="2019-11-02T14:56:02.313" Score="8" ViewCount="1169" Body="&lt;p&gt;If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs are ordered sequentially?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example I saw where the top line of each section represents letters typed and the next row represents the predicted next character (red letters in the next row means a confident prediction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4LUjg.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4LUjg.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wouldn't it be simpler to just pass the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; number of letters leading up to the last letter into an FFNN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; equaled 4, the following might be the input to the FFNN&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S, T, A, C =&amp;gt; Prediction: K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30154" LastEditorUserId="30154" LastEditDate="2019-11-03T06:47:03.837" LastActivityDate="2019-11-03T21:36:20.833" Title="Why use a recurrent neural network over a feedforward neural network for sequence prediction?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;multilayer-perceptron&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16227" PostTypeId="2" ParentId="16226" CreationDate="2019-11-02T15:05:44.257" Score="7" Body="&lt;p&gt;An RNN or LSTM have the advantage of &quot;remembering&quot; the past inputs, to improve performance over prediction of a time-series data. If you use a neural network over like the past 500 characters, this may work but the network just treat the data as a bunch of data without any specific indication of time. The network can learn the time representation only through gradient descent. RNN or LSTM however have &quot;time&quot; as a mechanism built into the model. The model loops through the model sequentially and have a real &quot;sense of time&quot; even before the model is trained. The model also have &quot;memory&quot; of previous data points to help the prediction. The architecture is based on the progress of time and the gradient are propagated through time as well. This is a much more intuitive way to process time-series data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 1D CNN also will work for the task. An example of CNN in time series data is wavenet, which uses CNN for generating incredibly life like speech using dilated convolution neural network. For whether LSTM or CNN works better, it depends on the data. You should try experimenting with both networks to see which works best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you need to classify a video's genre. It is much simpler to watch it in sequence then seeing frames of it playing randomly in front of your eyes. This is why an RNN or an LSTM works better in time series data. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-11-03T07:00:36.540" LastActivityDate="2019-11-03T07:00:36.540" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16228" PostTypeId="2" ParentId="16222" CreationDate="2019-11-02T15:12:07.770" Score="1" Body="&lt;p&gt;If you want to classify data with similar characteristics, it would often helps if you hand craft features. For classifying women or men wrestling, you may want to try using cv2 to track human faces and feed that to your CNN as input. Example: &lt;a href=&quot;https://realpython.com/face-recognition-with-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://realpython.com/face-recognition-with-python/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the data is not images, you may want to do a analysis to see which feature have no clear relationship with whether it is men or women wrestling and remove them. Example: &lt;a href=&quot;https://towardsdatascience.com/a-feature-selection-tool-for-machine-learning-in-python-b64dd23710f0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/a-feature-selection-tool-for-machine-learning-in-python-b64dd23710f0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that I can help you and have a nice day! &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-02T15:12:07.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16229" PostTypeId="2" ParentId="16221" CreationDate="2019-11-02T15:16:35.167" Score="2" Body="&lt;p&gt;For extra input that does not matter, you should not input it to the network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Feature selection, the process of finding and selecting the most&#xA;  useful features in a dataset, is a crucial step of the machine&#xA;  learning pipeline. Unnecessary features &lt;strong&gt;decrease training speed&lt;/strong&gt;,&#xA;  &lt;strong&gt;decrease model interpretability&lt;/strong&gt;, and, most importantly, &lt;strong&gt;decrease&#xA;  generalization performance&lt;/strong&gt; on the test set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://towardsdatascience.com/a-feature-selection-tool-for-machine-learning-in-python-b64dd23710f0&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Feature Selection Tool for Machine Learning in Python&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the source says, unnecessary features decreases accuracy and training speed. Moreover, they have no mapping to the labels, so they won't end up being used. They are unnecessary and adding them will only cause you trouble. Hope this helps you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="1671" LastEditDate="2019-11-02T16:10:18.650" LastActivityDate="2019-11-02T16:10:18.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16230" PostTypeId="2" ParentId="6753" CreationDate="2019-11-02T16:29:05.860" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTMs&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/abs/1406.1078&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRUs&lt;/a&gt; are &lt;strong&gt;computationally&lt;/strong&gt; more effective than the standard RNNs because they explicitly attempt to address the &lt;a href=&quot;http://ai.dinfo.unifi.it/paolo/ps/tnn-94-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;vanishing and exploding gradient problems&lt;/a&gt;, which are &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical problems&lt;/a&gt; related to the vanishing or explosion of the values of the gradient vector (the vector that contains the partial derivatives of the loss function with respect to the parameters of the model) that arise when training recurrent neural networks with gradient descent and &lt;a href=&quot;http://axon.cs.byu.edu/~martinez/classes/678/Papers/Werbos_BPTT.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;back-propagation through time&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-02T23:34:58.110" LastActivityDate="2019-11-02T23:34:58.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16231" PostTypeId="5" CreationDate="2019-11-02T16:51:51.593" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-02T16:51:51.593" LastActivityDate="2019-11-02T16:51:51.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16232" PostTypeId="4" CreationDate="2019-11-02T16:51:51.593" Score="0" Body="For questions related to the gated recurrent unit (GRU), a modification and simplification of the LSTM unit, which is a more sophisticated unit (with respect to the standard one) of a recurrent neural network (RNN). An RNN that uses GRU units is often called a GRU network. GRUs were introduced in the paper &quot;Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation&quot; (2014) by Kyunghyun Cho et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-02T16:51:51.593" LastActivityDate="2019-11-02T16:51:51.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16233" PostTypeId="1" AcceptedAnswerId="26223" CreationDate="2019-11-02T17:05:55.893" Score="1" ViewCount="117" Body="&lt;p&gt;I am looking for a dataset, which I could train a model to detect people/boats/surfboards, etc., from a drone view.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone seen a dataset that could be useful for this purpose?&lt;/p&gt;&#xA;&lt;p&gt;I have some photos made by me (like this one below), but I need more data. Of course, the best will be if data will be labeled, but, if someone has seen an unlabeled dataset with videos/photos like that below, please share the link to it.&lt;/p&gt;&#xA;&lt;p&gt;Sample photos I am looking for:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/icdGt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/icdGt.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Es5er.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Es5er.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30992" LastEditorUserId="2444" LastEditDate="2021-02-05T13:06:08.467" LastActivityDate="2021-02-05T13:06:08.467" Title="Dataset for floating objects detection" Tags="&lt;deep-learning&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2021-02-05T13:04:52.080" ContentLicense="CC BY-SA 4.0" />
  <row Id="16234" PostTypeId="1" AcceptedAnswerId="16235" CreationDate="2019-11-02T18:55:26.247" Score="2" ViewCount="70" Body="&lt;p&gt;In the book &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0956372813&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence Engines: A Tutorial Introduction to the Mathematics of Deep Learning&lt;/a&gt;, James Stone says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With supervised learning, the response to each input vector is an output vector that &lt;strong&gt;receives immediate vector-valued feedback specifying the correct output&lt;/strong&gt;, and this feedback refers uniquely to the input vector just received; in contrast, each reinforcement learning output vector (action) receives scalar-valued feedback often sometime after the action, and this feedback signal depends on actions taken before and after the current action.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I fail to understand the part formatted in bold. Once we have a set of labeled examples (feature vector and label pairs), where is the &quot;feedback&quot; coming from? Testing and validation results of our calibrated model (say a neural network based one)? &lt;/p&gt;&#xA;" OwnerUserId="12855" LastEditorUserId="2444" LastEditDate="2019-11-03T13:34:28.143" LastActivityDate="2020-01-24T12:34:28.187" Title="What does &quot;immediate vector-valued feedback&quot; mean?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;supervised-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16235" PostTypeId="2" ParentId="16234" CreationDate="2019-11-02T19:09:04.160" Score="2" Body="&lt;p&gt;By &quot;immediate vector-valued feedback&quot;, they probably mean exactly the label in the &quot;labeled examples&quot; you mentioned.&lt;/p&gt;&#xA;" OwnerUserId="23527" LastActivityDate="2019-11-02T19:09:04.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16236" PostTypeId="2" ParentId="16171" CreationDate="2019-11-02T20:01:19.283" Score="1" Body="&lt;p&gt;From your description, it seems that you are implementing a version of an algorithm called REINFORCE. This algorithm belongs to a family called &lt;em&gt;Policy Gradient methods&lt;/em&gt;, which directly optimizes the policy network &lt;span class=&quot;math-container&quot;&gt;$\pi(a_t|s_t)$&lt;/span&gt; from rewards without ever worrying about estimating a value function. This type of algorithm is usually pretty slow and presents high variance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The methods that you recognize as the ones using two neural networks correspond to a family called &lt;em&gt;Actor-Critic methods&lt;/em&gt;. This type of algorithm uses the trajectories of rewards to estimate a &lt;em&gt;value network&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$q(s_t,a_t)$&lt;/span&gt; (called the &lt;em&gt;critic&lt;/em&gt;), and, contrary to the previous family of methods, it uses the value network to train the &lt;em&gt;policy network&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi(a_t|s_t)$&lt;/span&gt; (called the &lt;em&gt;actor&lt;/em&gt;), instead of directly using the trajectory of rewards. This indirect dependence usually makes variance smaller and also learning faster. I recommend you have a look at chapter 13 of the book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your first question: it seems you are missing the family of Actor-Critic methods. I recommend you learn about them since they are very powerful (e.g., read about DDPG or SAC).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About your second question, the standard method to &quot;reward&quot; a policy network is not by training it. Usually, you have a &lt;em&gt;reward function&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$r(s_t,a_t)$&lt;/span&gt; that depends on your current state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and you modify the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; of your network in such a way that the probability of an action &lt;span class=&quot;math-container&quot;&gt;$\pi(a_t|s_t)$&lt;/span&gt; increases if the reward is positive or decreases if it is negative. More specifically, you perform stochastic gradient ascend steps like this one: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta_t\leftarrow\theta_t+\alpha\mathbb E\left[\sum_{k=t}^{T+t} \gamma^{k-t}r(s_k,a_k)\right]\nabla \log\pi(a_t|s_t,\theta_t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this formula says is that if in the time-step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; you take an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, wait &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; steps and collect the rewards from &lt;span class=&quot;math-container&quot;&gt;$r(s_t,a_t)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$r(s_{t+T},a_{t+T})$&lt;/span&gt;, then, you should modify your parameters in the direction that the policy increases the most (i.e., &lt;span class=&quot;math-container&quot;&gt;$\nabla \log\pi(a_t|s_t,\theta_t)$&lt;/span&gt;) if the expected return &lt;span class=&quot;math-container&quot;&gt;$E\left[\sum_{k=t}^{T+t} \gamma^{k-t}r(s_k,a_k)\right]$&lt;/span&gt; is positive, or in the direction that the policy decreases the most if that expected return is negative.      &lt;/p&gt;&#xA;" OwnerUserId="30983" LastEditorUserId="30983" LastEditDate="2019-11-03T18:50:06.097" LastActivityDate="2019-11-03T18:50:06.097" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16238" PostTypeId="1" CreationDate="2019-11-03T00:40:31.030" Score="3" ViewCount="208" Body="&lt;p&gt;In a vanilla &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo tree search (MCTS)&lt;/a&gt; implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only then the information gathered is backed up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read the AlphaZero paper (and the AlphaGo Zero too) and I didn't find any information on how the rollout is implemented (maybe I missed it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?&lt;/p&gt;&#xA;" OwnerUserId="22369" LastEditorUserId="2444" LastEditDate="2019-11-03T13:49:14.927" LastActivityDate="2019-11-03T13:49:14.927" Title="How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?" Tags="&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;&lt;implementation&gt;&lt;alphago-zero&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16239" PostTypeId="2" ParentId="16233" CreationDate="2019-11-03T03:25:30.127" Score="2" Body="&lt;p&gt;Perhaps you can check this dataset out:&#xA;&lt;a href=&quot;http://www.aiskyeye.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.aiskyeye.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The VisDrone2019 dataset is collected by the AISKYEYE team at Lab of&#xA;  Machine Learning and Data Mining , Tianjin University, China. The&#xA;  benchmark dataset consists of 288 video clips formed by 261,908 frames&#xA;  and 10,209 static images, captured by various drone-mounted cameras,&#xA;  covering a wide range of aspects including location (taken from 14&#xA;  different cities separated by thousands of kilometers in China),&#xA;  environment (urban and country), objects (pedestrian, vehicles,&#xA;  bicycles, etc.), and density (sparse and crowded scenes). Note that,&#xA;  the dataset was collected using various drone platforms (i.e., drones&#xA;  with different models), in different scenarios, and under various&#xA;  weather and lighting conditions. These frames are manually annotated&#xA;  with more than 2.6 million bounding boxes of targets of frequent&#xA;  interests, such as pedestrians, cars, bicycles, and tricycles. Some&#xA;  important attributes including scene visibility, object class and&#xA;  occlusion, are also provided for better data utilization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It provides many drone view images with bounding boxes. Hope it can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-03T03:25:30.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16240" PostTypeId="1" CreationDate="2019-11-03T04:36:24.793" Score="3" ViewCount="112" Body="&lt;p&gt;I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and   the other without BN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find that the model DNN with BN can't predict well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The codes is shown as follow:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.stats&#xA;import matplotlib.pyplot as plt&#xA;from keras.models import Sequential&#xA;from keras.layers import Dense,Dropout,Activation, BatchNormalization&#xA;&#xA;np.random.seed(1)&#xA;&#xA;# generate non-gaussian data&#xA;def generate_data():&#xA;    distribution = scipy.stats.gengamma(1, 70, loc=10, scale=100)&#xA;    x = distribution.rvs(size=10000)&#xA;    # plt.hist(x)&#xA;    # plt.show()&#xA;    print ('[mean, var, skew, kurtosis]', distribution.stats('mvsk'))&#xA;&#xA;    y = np.sin(x) + np.cos(x) + np.sqrt(x)&#xA;    plt.hist(y)&#xA;    # plt.show()&#xA;    # print(y)&#xA;    return x ,y &#xA;&#xA;x, y = generate_data()&#xA;&#xA;x_train = x[:int(len(x)*0.8)]&#xA;y_train = y[:int(len(y)*0.8)]&#xA;x_test = x[int(len(x)*0.8):]&#xA;y_test = y[int(len(y)*0.8):]&#xA;&#xA;&#xA;def DNN(input_dim, output_dim, useBN = True):&#xA;    '''&#xA;    定义一个DNN model&#xA;    '''&#xA;    model=Sequential()&#xA;&#xA;    model.add(Dense(128,input_dim= input_dim))&#xA;    if useBN:&#xA;        model.add(BatchNormalization())&#xA;    model.add(Activation('tanh'))&#xA;    model.add(Dropout(0.5))&#xA;&#xA;    model.add(Dense(50))&#xA;    if useBN:&#xA;        model.add(BatchNormalization())&#xA;    model.add(Activation('tanh'))&#xA;    model.add(Dropout(0.5))&#xA;&#xA;    model.add(Dense(output_dim))&#xA;    if useBN:&#xA;        model.add(BatchNormalization())&#xA;    model.add(Activation('relu'))&#xA;&#xA;    model.compile(loss= 'mse', optimizer= 'adam')&#xA;    return model&#xA;&#xA;clf = DNN(1, 1, useBN = True)&#xA;clf.fit(x_train, y_train, epochs= 30, batch_size = 100, verbose=2, validation_data = (x_test, y_test))&#xA;&#xA;y_pred = clf.predict(x_test)&#xA;def mse(y_pred, y_test):&#xA;    return np.mean(np.square(y_pred - y_test))&#xA;print('final result', mse(y_pred, y_test))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The input &lt;code&gt;x&lt;/code&gt; is like this shape:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cFodJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cFodJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I add BN layers, the result is shown as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 27/30&#xA; - 0s - loss: 56.2231 - val_loss: 47.5757&#xA;Epoch 28/30&#xA; - 0s - loss: 55.1271 - val_loss: 60.4838&#xA;Epoch 29/30&#xA; - 0s - loss: 53.9937 - val_loss: 87.3845&#xA;Epoch 30/30&#xA; - 0s - loss: 52.8232 - val_loss: 47.4544&#xA;final result 48.204881459013244&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I don't add BN layers, the predicted result is better:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 27/30&#xA; - 0s - loss: 2.6863 - val_loss: 0.8924&#xA;Epoch 28/30&#xA; - 0s - loss: 2.6562 - val_loss: 0.9120&#xA;Epoch 29/30&#xA; - 0s - loss: 2.6440 - val_loss: 0.9027&#xA;Epoch 30/30&#xA; - 0s - loss: 2.6225 - val_loss: 0.9022&#xA;final result 0.9021717561981543&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anyone knows the theory about why BN is not suitable for non-gaussian data ?&lt;/p&gt;&#xA;" OwnerUserId="23200" LastEditorUserId="26652" LastEditDate="2019-11-03T09:38:15.913" LastActivityDate="2021-03-27T21:03:00.293" Title="Is batch normalization not suitable for non-gaussian input?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16241" PostTypeId="2" ParentId="11139" CreationDate="2019-11-03T06:57:41.307" Score="1" Body="&lt;p&gt;If you want to count the number of objects using a neural network, you can use pretrained YOLO with the bottom prediction layer removed, and feed the features to a classification feed forward layer of let's say 1000 class representing 0-999 objects in the image. You can then train it and propagate the gradients through it. For example, in the pytorch code for YOLO,(source:&lt;a href=&quot;https://github.com/eriklindernoren/PyTorch-YOLOv3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eriklindernoren/PyTorch-YOLOv3&lt;/a&gt;)&#xA;You can add a &lt;code&gt;nn.Linear&lt;/code&gt; and use cross entropy loss to classify the number of images. You can also change the architecture completely. Maybe you can try adding layers to reset or other classifying network to count the number of objects. Hope this can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-03T06:57:41.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16242" PostTypeId="1" CreationDate="2019-11-03T07:33:57.710" Score="1" ViewCount="286" Body="&lt;p&gt;I have a structured dataset of around 100 gigs, and I am using DNN for classification in TF 2.0. Because of this huge dataset, I cannot load entire data in memory for training. So, I'll be reading data in batches to train the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the input to the network should be normalized and for that, I need training dataset mean and SD. I have been reading TensorFlow docs to get info on how to normalize features when reading data in batches. But, couldn't find one. though I found this &lt;a href=&quot;https://www.tensorflow.org/tutorials/load_data/csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;, it is only for the case where entire data can be loaded in memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, If any of you have worked on creating such a TensorFlow data pipeline for normalizing input features while loading data in batches and training model, It would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="31002" LastActivityDate="2020-07-31T21:04:45.617" Title="TensorFlow 2.0 - Normalizing input to DNN (on structured data)" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2020-08-10T17:01:49.747" ContentLicense="CC BY-SA 4.0" />
  <row Id="16243" PostTypeId="2" ParentId="16220" CreationDate="2019-11-03T10:30:50.593" Score="1" Body="&lt;p&gt;If you want to use machine learning for such a project, you can use vibrations data directly, and treat the problem as a regular audio classification problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple approach would be to use a Neural Network with Convolutions. This would take care of features extraction for you. And maybe follow these by dense layers at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that, it would be easier to make suggestions if you posted samples of your data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Also, keep in mind that machine learning usually requires large datasets so if you are collecting data all by yourself in the way you describe, you might not have enough sample to run a good model. In such a case when number of samples is limited one can use transfer learning - using a pre-trained model - but I am not aware of any such pre-trained models for wave data.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-11-03T11:05:02.627" LastActivityDate="2019-11-03T11:05:02.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16245" PostTypeId="2" ParentId="16226" CreationDate="2019-11-03T14:41:12.320" Score="3" Body="&lt;h2&gt;Assumptions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Different model structures encode different assumptions - while we often make simplifying assumptions that aren't strictly correct, some assumptions are more wrong than others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, your proposed structure of &quot;just pass the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; number of letters leading up to the last letter into an FFNN&quot; makes an assumption that all the information relevant for the decision is fully obtainable from the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; previous letters, and &lt;span class=&quot;math-container&quot;&gt;$(X+1)$&lt;/span&gt;st and earlier input letters are not relevant - in some sense, an extension of the Markov property. Obviously, that's not true in many cases, there are all kinds of structures where long term relationships matter, and assuming that they don't lead to a model that intentionally doesn't take such relationships into account. Furthermore, it would make an independence assumption that the effect of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;th, &lt;span class=&quot;math-container&quot;&gt;$(X-1)$&lt;/span&gt;st and &lt;span class=&quot;math-container&quot;&gt;$(X-2)$&lt;/span&gt;nd elements on the current output is entirely distinct and separate, you don't make an assumption that those features are related, while in most real problems they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classic RNN structures also make some implicit assumptions, namely, that only the preceding elements are relevant for the decision (which is wrong for some problems, where information from the following items is also required), and that the transformative relationship between the input, output and the passed-on state is &lt;em&gt;the same&lt;/em&gt; for all elements in the chain, and that it doesn't change over time; That's also not certainly true in all cases, this is quite a strong restriction, but that's generally less wrong than the assumption that the last &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; elements are sufficient, and powerful true (or mostly true) restrictions are useful (e.g. the No Free Lunch Theorem applies) for models that generalize better; just like e.g. enforcing translational invariance for image analysis models, etc.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastEditorUserId="2444" LastEditDate="2019-11-03T21:36:20.833" LastActivityDate="2019-11-03T21:36:20.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16246" PostTypeId="2" ParentId="16242" CreationDate="2019-11-03T15:12:29.220" Score="0" Body="&lt;p&gt;One way could be to first iterate over the dataset in batches, just to get the mean and sd. Then when running training, use the true population parameters obtained before.&lt;/p&gt;&#xA;" OwnerUserId="19358" LastActivityDate="2019-11-03T15:12:29.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16247" PostTypeId="2" ParentId="16224" CreationDate="2019-11-03T16:30:56.300" Score="3" Body="&lt;p&gt;In reference to your exact question there is published research that attempts to bring these two areas together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;“HolStep: A Machine Learning Dataset for Higher-order Logic Theorem Proving” by Cezary Kaliszyk, François Chollet, Christian Szegedy (2017) &#xA;&lt;a href=&quot;https://arxiv.org/abs/1703.00426&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1703.00426&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That group also has other published work related to the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of their results they list several areas of logical systems that are highly suited to machine learning methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Predicting whether a statement is useful in the proof of a given conjecture&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Predicting the dependencies of a proof statement (premise selection)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Predicting whether a statement is an important one (human named)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Predicting which conjecture a particular intermediate statement originates from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Predicting the name given to a statement&lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Generating intermediate statements useful in the proof of a given conjecture &lt;/p&gt;&#xA;&#xA;&lt;p&gt;• Generating the conjecture the current proof will lead to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It’s tough to know whether or not you can combine Higher Order Logic and Machine Learning in an effective way without needing to create a general AI. This is equivalent to wondering if an effective merging of the two areas is an AI-complete / AI-hard problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are active attempts at general AI by researchers such as Ben Goertzel (many others as well but just to give a popular name for googling). Research into general AI would give you an idea of whether or not other pieces of the puzzle are needed in order to create something “highly intelligent”.&lt;/p&gt;&#xA;" OwnerUserId="31005" LastActivityDate="2019-11-03T16:30:56.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16248" PostTypeId="5" CreationDate="2019-11-03T16:38:22.280" Score="0" Body="&lt;p&gt;For more info, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Expert_system&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-03T16:38:22.280" LastActivityDate="2019-11-03T16:38:22.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16249" PostTypeId="4" CreationDate="2019-11-03T16:38:22.280" Score="0" Body="For questions related to expert systems, which are computer systems that emulate the decision-making ability of a human expert. Expert systems are designed to solve complex problems by reasoning through bodies of knowledge, represented mainly as if-then rules. The first expert systems were created in the 1970s and then proliferated in the 1980s. Expert systems were among the first truly successful forms of artificial intelligence software (symbolic AI). " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-03T16:38:22.280" LastActivityDate="2019-11-03T16:38:22.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16250" PostTypeId="2" ParentId="16240" CreationDate="2019-11-03T18:21:23.000" Score="0" Body="&lt;p&gt;So batch-normalization helps descent based learning have an easier time traversing the loss manifold, but in your case you use it along with a relu as a final activation is problematic, it means the output is relatively associated with the other samples in the batch.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remove that last BN and you get better results, but also understand BN is inherently problematic for this task. Think of DNNs as featurizers, and BN in this case takes out the 2 batch-wide statistics which if they don't align to the initial distribution will cause error, which will lead to error in the output. In theory if BN gets the perfect statistics of the gaussian it should not matter too much, so one thing I tried with your code was remove last BN and increase N to 100,000 while increasing the batch size to 10000 and you see a huge boost in performance.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-11-03T18:21:23.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16251" PostTypeId="1" AcceptedAnswerId="16637" CreationDate="2019-11-03T23:06:33.753" Score="1" ViewCount="47" Body="&lt;p&gt;I have data of 695 hours. I use the first 694 hours to train the network and I use 695th hour to validate it. Now my goal is to predict the next hour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I can use my trained network to predict the next hour, that is, the 696th hour (which I do not have access to)?&lt;/p&gt;&#xA;" OwnerUserId="30551" LastEditorUserId="2444" LastEditDate="2019-11-19T20:03:52.273" LastActivityDate="2019-11-19T20:03:52.273" Title="How can I test my trained network on the next unavailable hour?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;prediction&gt;&lt;matlab&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16252" PostTypeId="1" CreationDate="2019-11-04T00:06:14.980" Score="5" ViewCount="62" Body="&lt;p&gt;I am trying to write a CNN from scratch and am wondering if it is possible to vectorize the convolution step.&lt;/p&gt;&#xA;&lt;p&gt;For example, if I had a dataset of 500 RGB images of size 32x32x3, and wanted the first convolutional layer to have 64 filters, how would I go about the vectorization of this layer?&lt;/p&gt;&#xA;&lt;p&gt;Currently, I am running through all 500 images in a &lt;code&gt;for&lt;/code&gt; loop, convoluting individually. I do this for all the images up to the flattening stage (where it essentially becomes a normal NN again), at which point I can implement the normal vectorisation approach to get to my output, etc.&lt;/p&gt;&#xA;&lt;p&gt;A holistic overview of the process would be appreciated, as I am struggling to get my head around it and am struggling to find and information on the matter online.&lt;/p&gt;&#xA;" OwnerUserId="29877" LastEditorUserId="2444" LastEditDate="2021-01-11T00:58:17.487" LastActivityDate="2021-01-11T00:58:17.487" Title="Is it possible to vectorise a CNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16253" PostTypeId="2" ParentId="16162" CreationDate="2019-11-04T01:46:46.793" Score="2" Body="&lt;h3&gt;&lt;strong&gt;Direct Answer to Your Question&lt;/strong&gt;:--&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Google uses the term: &lt;strong&gt;Automated Machine Learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;&lt;strong&gt;What this Answer is About&lt;/strong&gt;:--&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot; ... A general AI x creates another AI y which is better than x. ... &quot; ~ Ashwin Rohit (Stack Exchange user, Opening Poster)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is the term for this: &quot;A.I. creating A.I.&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is some theory behind this:--&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The AutoML procedure has so far been applied to image recognition and language modeling. Using AI alone, the team have observed it creating programs that are on par with state-of-the-art models designed by the world’s foremost experts on machine learning.&quot; – Google's AI Is Now Creating Its Own AI. (2017, May 22). Retrieved from &amp;lt; &lt;a href=&quot;https://www.iflscience.com/technology/google-ai-creating-own-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.iflscience.com/technology/google-ai-creating-own-ai/&lt;/a&gt; &gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;&lt;strong&gt;Layperson Explanation&lt;/strong&gt;:--&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.mentalfloss.com/article/508019/googles-ai-can-make-its-own-ai-now&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's AI Can Make Its Own AI Now&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.alphr.com/artificial-intelligence/1007850/google-s-ai-builds-its-own-ai-child-and-it-s-better-than-anything&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google’s AI builds its own AI child and it’s better than anything humans have made&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencealert.com/google-is-improving-its-artificial-intelligence-with-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Researchers Are Teaching Their AI to Build Its Own, More Powerful AI&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot; ... Unfortunately, even people who have plenty of coding knowledge might not know how to create the kind of algorithm that can perform these tasks. Google wants to bring the ability to harness artificial intelligence to more people, though, and according to WIRED, it's doing that by teaching machine-learning software to make more machine-learning software.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The project is called AutoML, and it's designed to come up with better machine-learning software than humans can. As algorithms become more important in scientific research, healthcare, and other fields outside the direct scope of robotics and math, the number of people who could benefit from using AI has outstripped the number of people who actually know how to set up a useful machine-learning program. Though computers can do a lot, according to Google, human experts are still needed to do things like preprocess the data, set parameters, and analyze the results. These are tasks that even developers may not have experience in. ... &quot; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– Google's AI Can Make Its Own AI Now. (2017, October 19). Retrieved from &amp;lt; &lt;a href=&quot;https://www.mentalfloss.com/article/508019/googles-ai-can-make-its-own-ai-now&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mentalfloss.com/article/508019/googles-ai-can-make-its-own-ai-now&lt;/a&gt; &gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We use programs to write programs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researchers often need tools to solve complicated problems and algorithms are often needed. They don't always have the technical experience to do this. This is an artificial intelligence-based solution to the ever-growing challenge of applying machine learning to this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows non-experts to engage in predictive performance of their final machine learning models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the potential of &quot;feed-back between systems&quot; when A.I. feeds into A.I., which continues to feed into itself, ad infinitum. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Business Applications and Practical Uses:--&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Defer to the book: &lt;em&gt;Automated Machine Learning for Business&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.bigsquid.com/accelerate-your-data-value-journey-with-automated-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accelerate your Data Value Journey with Automated Machine Learning&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.bigsquid.com/medical-tech-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leveraging the Power of Machine Learning to Save Lives&lt;/a&gt; [This is not medical advice.]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;&lt;strong&gt;Technical Mirror&lt;/strong&gt;:--&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;http://www.ml4aad.org/automl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ml4aad.org/automl/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;What is AutoML?&#xA;  Automated Machine Learning provides methods and processes to make Machine Learning available for non-Machine Learning experts, to improve efficiency of Machine Learning and to accelerate research on Machine Learning.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Machine learning (ML) has achieved considerable successes in recent years and an ever-growing number of disciplines rely on it. However, this success crucially relies on human machine learning experts to perform the following tasks:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Preprocess and clean the data.&lt;/li&gt;&#xA;  &lt;li&gt;Select and construct appropriate features.&lt;/li&gt;&#xA;  &lt;li&gt;Select an appropriate model family.&lt;/li&gt;&#xA;  &lt;li&gt;Optimize model hyperparameters.&lt;/li&gt;&#xA;  &lt;li&gt;Postprocess machine learning models.&lt;/li&gt;&#xA;  &lt;li&gt;Critically analyze the results obtained.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;As the complexity of these tasks is often beyond non-ML-experts, the rapid growth of machine learning applications has created a demand for off-the-shelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML.&quot; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– AutoML. (n.d.). Retrieved from &amp;lt; &lt;a href=&quot;http://www.ml4aad.org/automl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ml4aad.org/automl/&lt;/a&gt; &gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Automated_machine_learning&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.researchgate.net/publication/237136437_The_Nature_of_Self-Improving_Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/237136437_The_Nature_of_Self-Improving_Artificial_Intelligence&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Sources and References; and Further Reading:--&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://ai.googleblog.com/2017/05/using-machine-learning-to-explore.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.googleblog.com/2017/05/using-machine-learning-to-explore.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;http://www.primaryobjects.com/2013/01/27/using-artificial-intelligence-to-write-self-modifying-improving-programs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.primaryobjects.com/2013/01/27/using-artificial-intelligence-to-write-self-modifying-improving-programs/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://skymind.ai/wiki/automl-automated-machine-learning-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://skymind.ai/wiki/automl-automated-machine-learning-ai&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.datarobot.com/wiki/automated-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.datarobot.com/wiki/automated-machine-learning/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://blog.aimultiple.com/auto-ml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.aimultiple.com/auto-ml/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://powerbi.microsoft.com/en-us/blog/announcing-automated-machine-learning-in-power-bi-general-availability/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://powerbi.microsoft.com/en-us/blog/announcing-automated-machine-learning-in-power-bi-general-availability/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="25982" LastEditDate="2019-11-05T06:36:44.837" LastActivityDate="2019-11-05T06:36:44.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16254" PostTypeId="2" ParentId="16252" CreationDate="2019-11-04T05:18:24.550" Score="1" Body="&lt;p&gt;Yes you can vectorize a CNN. See this github file for details: &lt;a href=&quot;https://github.com/parasdahal/deepnet/blob/master/deepnet/layers.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/parasdahal/deepnet/blob/master/deepnet/layers.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After looking through it it basically transposes the input to some dimension and apply matrix multiplication to the weight with some other kind of transfromation. Pls refer to the github repository for details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-04T05:18:24.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16256" PostTypeId="2" ParentId="16176" CreationDate="2019-11-04T07:33:44.840" Score="1" Body="&lt;p&gt;Doing some unsupervised learning will give you some divisions within each label but you can't be sure it will be split on &quot;E-sports&quot; and &quot;Sports&quot;. It might as wel split sports into &quot;land sports&quot; and &quot;water sports&quot; or &quot;long sports&quot; and &quot;short sports&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to reliably split &quot;sports&quot; in 2 subsections of your desire is to relabel this manualy I am afraid.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-11-04T07:33:44.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16257" PostTypeId="1" CreationDate="2019-11-04T09:17:17.120" Score="9" ViewCount="2098" Body="&lt;p&gt;Is there a reliable website about artificial intelligence that is updated regularly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like there is now TensorFlow 2.0 and they write some information about it. Or a magazine which can be subscribed. So a site where all news of AI are collected to be always up to date.&lt;/p&gt;&#xA;" OwnerUserId="24093" LastEditorUserId="1671" LastEditDate="2019-11-06T00:52:47.670" LastActivityDate="2019-11-06T10:04:48.893" Title="Is there a reliable website about artificial intelligence that is frequently updated?" Tags="&lt;resource-request&gt;&lt;journalism&gt;" AnswerCount="6" CommentCount="2" FavoriteCount="4" ClosedDate="2019-11-05T10:53:51.907" ContentLicense="CC BY-SA 4.0" />
  <row Id="16258" PostTypeId="2" ParentId="16257" CreationDate="2019-11-04T09:53:47.230" Score="6" Body="&lt;p&gt;For sites, &lt;a href=&quot;https://towardsdatascience.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/&lt;/a&gt; maybe a good choice. The articles on there is about AI. You can read a lot of articles about state-of-the-art AI networks there. You can also subscribe to their newsletter for emails everyday on new AI discoveries. However, as stated by @DuttaA, take the articles with caution. The articles may or may not be entirely true as they are written by the community. Choose articles with large number of likes to read. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The YouTube channel two minutes paper is also a great resource. The channel posts videos about AI and also computer science periodically, and the content is easy to understand and enjoyable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reminder: for the towards data science site, you should disable cookies for it so that there will not be a limit on how many articles you read. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you and have a nice day&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-11-04T15:17:21.617" LastActivityDate="2019-11-04T15:17:21.617" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16259" PostTypeId="2" ParentId="16257" CreationDate="2019-11-04T11:24:45.923" Score="4" Body="&lt;p&gt;Personally, for such stuff, I always felt twitter was the best. As long as you follow the right people/pages your feed can be quite informative and quite frequently links to articles or youtube videos which can be helpful are also found. To start off you could check out the &lt;a href=&quot;https://twitter.com/DL_ML_Loop&quot; rel=&quot;nofollow noreferrer&quot;&gt;DL loop list&lt;/a&gt; to start you off with a few popular people in DL on twitter&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-04T11:24:45.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16260" PostTypeId="1" CreationDate="2019-11-04T12:05:06.523" Score="3" ViewCount="245" Body="&lt;p&gt;I am new to machine learning. I am reading &lt;a href=&quot;http://me-ramesh.blogspot.com/p/machine-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt; on the VC dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal H$&lt;/span&gt; consists of all hypotheses in two dimensions &lt;span class=&quot;math-container&quot;&gt;$h: R^2 → \{−1, +1 \}$&lt;/span&gt;, positive inside some square boxes and negative elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cKgKt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cKgKt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the maximum number of dichotomies for the 4 data points? i.e calculate mH(4)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It seems that the square can shatter 3 points but not 4 points. The &lt;span class=&quot;math-container&quot;&gt;$\mathcal V \mathcal C$&lt;/span&gt; VC dimension of a square is 3. What is the proof behind this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31018" LastEditorUserId="9947" LastEditDate="2020-04-06T04:44:18.510" LastActivityDate="2020-04-06T20:54:00.243" Title="What is the maximum number of dichotomies in a square?" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16261" PostTypeId="2" ParentId="16257" CreationDate="2019-11-04T12:39:48.827" Score="4" Body="&lt;p&gt;Besides the other answers, you can follow &lt;a href=&quot;https://www.deeplearning.ai/thebatch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Batch news&lt;/a&gt; of deeplearning.ai:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Batch presents the most important AI events and perspective in a curated, easy-to-read report for engineers and business leaders. Every Wednesday, The Batch highlights a mix of the most practical research papers, industry-shaping applications, and high-impact business news.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2019-11-04T12:39:48.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16262" PostTypeId="2" ParentId="16242" CreationDate="2019-11-04T16:15:28.047" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Central_limit_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;central limit theorem&lt;/a&gt; tells us that the error in an estimate of the mean or standard deviation of a dataset will decline as &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{\sqrt{n}}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of samples taken at random from the set, and combined together to compute the mean and deviation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you select at random, for example, &lt;span class=&quot;math-container&quot;&gt;$10^6$&lt;/span&gt; examples (probably a few megabytes), then the mean and standard deviation you compute from those examples will be within &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{sqrt(10^6)} = \frac{1}{10^3}$&lt;/span&gt; of the &quot;true&quot; answer. That's one part in 1,000, which is certainly accurate enough to use for re-scaling the dataset.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-04T16:15:28.047" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16263" PostTypeId="1" AcceptedAnswerId="16265" CreationDate="2019-11-04T16:35:20.043" Score="1" ViewCount="115" Body="&lt;p&gt;I am interested in exploring whether AI techniques can derive hidden patterns of relationships in a data set.  For example, from among house size, lot size, age of house and asking price,   what formula best predicts selling price?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In explorations around how this might be done, I tried to use a neural network to solve for a predictable relationship between two variables to predict a third, so I trained my neural network with inputs consisting of the length of two sides of a triangle, and the result being the length of the hypotenuse.  It couldn't get it to work.      &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is true, I wonder if there is some other AI technique that could 'derive' a network modeling the phythagorean theorem from a training data set with better results than a normal neural network?&lt;/p&gt;&#xA;" OwnerUserId="31024" LastActivityDate="2019-11-05T09:24:13.373" Title="Which AI Technique best suited to discovering non-linear relationships in data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;ai-basics&gt;&lt;problem-solving&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16264" PostTypeId="1" CreationDate="2019-11-04T17:39:09.863" Score="5" ViewCount="66" Body="&lt;p&gt;I'm working on a project where there is a limited dataset of videos (about 200). We want to train a model that can detect a single class in the videos. That class can be of multiple different types of shapes (thin wire, a huge area of the screen, etc).&lt;/p&gt;&#xA;&lt;p&gt;There are three options on how we can label this data:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Image classification (somewhere in the image is this class)&lt;/li&gt;&#xA;&lt;li&gt;Bounding box (in this area, there is the class)&lt;/li&gt;&#xA;&lt;li&gt;Semantic segmentation (these pixels are the class)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My assumption is that if the model was trained on semantic segmentation data it would perform slightly better than bounding box data.  I'm also assuming it would perform way better than if the model only learned on image classification data.  Is that correct?&lt;/p&gt;&#xA;" OwnerUserId="20338" LastEditorUserId="2444" LastEditDate="2021-01-30T19:58:29.373" LastActivityDate="2021-01-30T19:58:29.373" Title="Do models train better if the labelling information is more specific (or dense)?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;object-detection&gt;&lt;image-segmentation&gt;&lt;data-labelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16265" PostTypeId="2" ParentId="16263" CreationDate="2019-11-04T18:44:27.400" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, from among house size, lot size, age of house and asking price, what formula best predicts selling price?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no general formula for this. Search for neural network regression and you can get started. The AI technique or any prediction algorithm in general will learn a function that maps from the input feature vector &lt;span class=&quot;math-container&quot;&gt;$(x_1, ...,x_n)$&lt;/span&gt;, where each of the element in the vector is a measurement on the &lt;span class=&quot;math-container&quot;&gt;$\text{predictors/independent variables/regressors}$&lt;/span&gt; to the &lt;span class=&quot;math-container&quot;&gt;$\text{variable of interest/dependent variables}$&lt;/span&gt; i.e. &lt;span class=&quot;math-container&quot;&gt;$\text{selling price}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The statement is incorrect. In fact the opposite is true. CNNs are known for modeling non-linear relationships. Examples are the highly successful image classification CNN architectures like Inception, ResNet, etc.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-11-05T09:24:13.373" LastActivityDate="2019-11-05T09:24:13.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16266" PostTypeId="1" CreationDate="2019-11-04T18:46:30.153" Score="3" ViewCount="561" Body="&lt;p&gt;I am studying about Computer Vision around 2-3 months long. Recently, I have an idea to conduct a research about computer vision and Artificial neural networks. The idea is I want to make an automatic object measurement using ANN. The purpose is to measure the distance between two opposite sides of the objects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main thing is this research is going to work as the fractional caliper tools works. Fractional caliper is a tool that use to measure the object. However, it will be time consuming to measure huge amounts of objects if there are a lot of small objects in the industry. Through this tool, I found an idea in how to make a system that can automatically give an accurate results similar as the fractional calipers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, giving an image of the object and the system can define the depth, weight or high, etc., Thus, the idea has been explained above, please suggest me what kinds of Computer Vision model used to apply for this study? And how to combine it with ANN technique? &lt;/p&gt;&#xA;" OwnerUserId="31027" LastEditorUserId="1581" LastEditDate="2019-11-04T19:22:34.637" LastActivityDate="2019-11-06T03:11:18.063" Title="How can I measure an object using Computer Vision and neural network techniques?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;image-processing&gt;&lt;intelligence&gt;&lt;artificial&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16267" PostTypeId="2" ParentId="16257" CreationDate="2019-11-04T19:20:40.547" Score="5" Body="&lt;p&gt;If you're willing to &quot;drink from the fire hose&quot;, it's probably hard to beat just browsing recent &lt;a href=&quot;https://arxiv.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;arXiv&lt;/a&gt; submissions when it comes to seeing the most novel results in AI/ML. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the unfamiliar, &lt;a href=&quot;https://en.wikipedia.org/wiki/ArXiv&quot; rel=&quot;nofollow noreferrer&quot;&gt;arXiv&lt;/a&gt; is a preprint repository where academic articles are published before/as they are submitted to an academic journal. It also contains work which is not intended for later publication, such as replicating someone else's results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two major caveats are 1) these articles are written with an expert audience in mind, so may not feel very inviting and 2) there is no barrier to entry to publishing to arXiv, so there is no guarantee of quality for any individual article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ArXiv has an &lt;a href=&quot;https://arxiv.org/list/cs.AI/recent&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI&lt;/a&gt; category, as well as a &lt;a href=&quot;https://arxiv.org/list/cs.LG/recent&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS/ML&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/list/stat.ML/recent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stats/ML&lt;/a&gt; category. &lt;/p&gt;&#xA;" OwnerUserId="31030" LastEditorUserId="16909" LastEditDate="2019-11-05T13:04:32.653" LastActivityDate="2019-11-05T13:04:32.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16268" PostTypeId="1" CreationDate="2019-11-04T19:30:46.193" Score="2" ViewCount="121" Body="&lt;p&gt;I'm new to Deep Learning, and I have some conceptual problems. I followed a simple tutorial &lt;a href=&quot;https://towardsdatascience.com/transfer-learning-for-image-classification-using-keras-c47ccf09c8c8&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and trained a model in Keras to do image classification on 10 classes of logos. I prepared 10 classes with each class having almost 100 images. My trained &lt;code&gt;Resnet50&lt;/code&gt; model performs exceptionally great when the image &lt;strong&gt;&lt;em&gt;is&lt;/em&gt;&lt;/strong&gt; one of those 10 logos, with 1.00 probability. But the problem is if I pass a &lt;strong&gt;&lt;em&gt;non-logo&lt;/em&gt;&lt;/strong&gt; item, a random image totally unrelated visually, still it marks it as one of those logos with close to 1.00 probability! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm confused. Am I missing anything? Why is this happening? How to find a solution? I need to find logos in video frames. But right now, with a high possbility each frame is marked as a logo!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my simple training code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def build_finetune_model(base_model, dropout, fc_layers, num_classes):&#xA;    for layer in base_model.layers:&#xA;        layer.trainable = False&#xA;&#xA;    x = base_model.output&#xA;    x = Flatten()(x)&#xA;    for fc in fc_layers:&#xA;        # New FC layer, random init&#xA;        x = Dense(fc, activation='relu')(x) &#xA;        x = Dropout(dropout)(x)&#xA;&#xA;    # New softmax layer&#xA;    predictions = Dense(num_classes, activation='softmax')(x) &#xA;    finetune_model = Model(inputs=base_model.input, outputs=predictions)&#xA;    return finetune_model&#xA;finetune_model = build_finetune_model(base_model, dropout=dropout, fc_layers=FC_LAYERS, num_classes=len(class_list))&#xA;adam = Adam(lr=0.00001)&#xA;finetune_model.compile(adam, loss='categorical_crossentropy', metrics=['accuracy'])&#xA;filepath=&quot;./checkpoints/&quot; + &quot;ResNet50&quot; + &quot;_model_weights.h5&quot;&#xA;checkpoint = ModelCheckpoint(filepath, monitor=[&quot;acc&quot;], verbose=1, mode='max')&#xA;callbacks_list = [checkpoint]&#xA;&#xA;history = finetune_model.fit_generator(train_generator, epochs=NUM_EPOCHS, workers=8, &#xA;                                       steps_per_epoch=steps_per_epoch, &#xA;                                       shuffle=True, callbacks=callbacks_list)&#xA;&#xA;plot_training(history)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-11-05T17:40:05.920" LastActivityDate="2021-03-29T21:05:00.497" Title="Why is this ResNet50 misclassifying objects?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16269" PostTypeId="2" ParentId="16263" CreationDate="2019-11-04T19:34:31.773" Score="2" Body="&lt;p&gt;You are mixing up lots of things here. Specifically, you seem to be lacking a basic understanding of artificial neural networks and what they can do (e.g. what type of articifial neural networks are linear classifiers/regressors and which can model non-linear relationships). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I'd take a step back and start with understanding the basics of AI. The go-to book for that is 'Artificial Intelligence: A Modern Approach' by Russel and Norvig. It might be a slower (and more theoretical) start but IMO that is the right approach to actually understand what you are doing. &lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2019-11-04T19:34:31.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16271" PostTypeId="2" ParentId="16264" CreationDate="2019-11-04T23:20:07.127" Score="3" Body="&lt;p&gt;It depends on what is your ultimate goal. If your goal is to simply classify the object in the image, having more complex output won't help. Simpler output representation yields better result. If your goal is to detect the bounding box, output the bounding box. There is no need for a more complex output feature. If you use a segmentation method for bounding box detection, it is more prone to error because of it's excess output features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you are given a gradr 6 math test. If you do the questions using grade 12 maths knowledge and do it with calculus and stuff to make your calculations seems more complex, will you have a higher mark than doing it the normal way? No! The marks is the same or even less due to higher chance of error in doing complex calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, higher complexity on your labels won't help your task if it is a simple task. Hope this would help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-04T23:20:07.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16272" PostTypeId="2" ParentId="16257" CreationDate="2019-11-05T01:07:00.750" Score="1" Body="&lt;p&gt;I think Deep Learning Weekly &lt;a href=&quot;https://www.deeplearningweekly.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.deeplearningweekly.com/&lt;/a&gt; is very good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Critiques of other sources mentioned elsewhere in these answers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;towardsdatascience&lt;/code&gt; and similar have a lot of articles which seem like undergrads trying to write out their best understanding of their lecture notes.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=ThDwS79HPhs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twitter is pain&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The Batch is ok but a lot less interesting, maybe more manager-oriented, than Deep Learning Weekly.&lt;/li&gt;&#xA;&lt;li&gt;A raw arXiv feed is too much detail for what I think the questioner wants.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31035" LastEditorUserId="31035" LastEditDate="2019-11-06T10:04:48.893" LastActivityDate="2019-11-06T10:04:48.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16273" PostTypeId="2" ParentId="16266" CreationDate="2019-11-05T01:22:41.977" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://youtu.be/MMiKyfd6hA0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Father Ted explains why this is a hard problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seriously -- if you have stereo images it should be possible, since that's what &lt;em&gt;we&lt;/em&gt; use for depth perception. If you know how far away points x1 and x2 are, then you can measure distance using trigonometry. No neural networks needed, I guess. &lt;a href=&quot;https://en.wikipedia.org/wiki/Triangulation_(computer_vision)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Triangulation_(computer_vision)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31035" LastEditorUserId="31035" LastEditDate="2019-11-05T18:22:31.173" LastActivityDate="2019-11-05T18:22:31.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16274" PostTypeId="1" AcceptedAnswerId="16275" CreationDate="2019-11-05T06:40:47.340" Score="1" ViewCount="66" Body="&lt;p&gt;I believe I saw an article about an AI that was able to decode human vision 'brain-waves' in real-time, which would create a blurry image of what the human was seeing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://futurism.com/the-byte/ai-draws-decodes-brainwaves&quot; rel=&quot;nofollow noreferrer&quot;&gt;This AI Decodes Your Brainwaves and Draws What You're Looking at&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anyone aware where I can find this?&lt;/p&gt;&#xA;" OwnerUserId="31041" LastEditorUserId="25982" LastEditDate="2019-11-05T13:19:35.943" LastActivityDate="2019-11-06T11:11:56.057" Title="Have any AI's been able to decode human vision 'thoughts'" Tags="&lt;human-like&gt;&lt;brain&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16275" PostTypeId="2" ParentId="16274" CreationDate="2019-11-05T09:22:31.357" Score="0" Body="&lt;h3&gt;Direct Answer to the OP's Question&lt;/h3&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Have any AI's been able to decode human vision 'thoughts'&amp;quot; ~ Albert (Stack Exchange user, OP)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://futurism.com/the-byte/ai-draws-decodes-brainwaves&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is technology that can produce pictures of what the user is thinking about through scanning a brain&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Is anyone aware where I can find this?&amp;quot; ~ Albert (Stack Exchange user, OP)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.emotiv.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emotiv&lt;/a&gt; is the most accessible commercial model (circa. late 2019).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The OP is probably interested in consumer brain–computer interfaces (also known as BCIs). These are varied technologies which range from:--&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://kids.frontiersin.org/article/10.3389/frym.2018.00024&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simple &amp;quot;yes-no&amp;quot; brain-interface (e.g. for people in a coma)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.technologyreview.com/s/407447/connecting-your-brain-to-the-game/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advanced programs that control video games through thought, such as a high fantasy wizard duel&lt;/a&gt; (Do a search about this on YouTube!)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://futurism.com/the-byte/ai-draws-decodes-brainwaves&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technology that can produce pictures of what the user is thinking about through scanning a brain&lt;/a&gt;.  (This was mentioned by the OP.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This Wikipedia page, &amp;lt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Consumer_brain%E2%80%93computer_interfaces&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Consumer_brain%E2%80%93computer_interfaces&lt;/a&gt; &amp;gt;, compares different models of BCIs.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Brain%E2%80%93computer_interface&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Brain%E2%80%93computer_interface&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Electroencephalography&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Electroencephalography&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Emotiv_Systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Emotiv_Systems&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There also some very serious ethical issues regarding being able to &amp;quot;read brains.&amp;quot; I mentioned the medical use; and I hope it goes in this direction.&lt;/p&gt;&#xA;&lt;p&gt;Deep philosophical discussions can be had on whether it is appropriate to read the brain of a supposed criminal. (I would personally say no.)&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://plato.stanford.edu/entries/neuroethics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/neuroethics/&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;[I can't comment on the technical details of this. It is outside of my purview. For example, if you need information about the Python-BCI interface, you will need an expert.]&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;[This is not medical and/or legal advice. This is theoretical discussion.]&lt;/p&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-06T01:57:30.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16276" PostTypeId="2" ParentId="16266" CreationDate="2019-11-05T09:57:14.723" Score="2" Body="&lt;p&gt;If the measurements you want from the object aren't too complicated (ie. length of a clearly defined feature), and if you are able to acquire a training dataset of images of the objects similar to what your model will see in your use case (same scale/distance), their bounding boxes and their measurements, a model you could try to implement is a &lt;a href=&quot;https://arxiv.org/abs/1604.02878&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Task Convolutional Neural Network (MTCNN)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;MTCNNs are typically used for face detection and alignment, but I would imagine it is possible to adapt them to your use case given proper training and tuning. If there are more complicated measurements that you want to obtain, you could pass on the detected objects to another model to make more specific measurements.&lt;/p&gt;&#xA;&lt;p&gt;You will have a problem however, with measuring depth. Depth is hard to estimate from an image because of the information that we lose when moving from a 3D to a 2D space.&#xA;A longer explanation on this is available in &lt;a href=&quot;https://stats.stackexchange.com/users/17760/machineepsilon&quot;&gt;MachineEpsilon's&lt;/a&gt; answer to the Cross Validated question &amp;quot;&lt;a href=&quot;https://stats.stackexchange.com/questions/322798/how-to-detect-the-exact-size-of-an-object-in-an-image-using-machine-learning/&quot;&gt;how to detect the exact size of an object in an image using machine learning?&lt;/a&gt;&amp;quot; but quoting his main statements:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This task of depth estimation is part of a hard and fundamental problem in computer vision called 3D reconstruction. Recovering metric information from images is sometimes called photogrammetry. It's hard because when you move from the real world to an image you lose information.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, the projective transformation  that takes your 3D point  to your 2D point  via = does not preserve distance. Since  is a 2×3 matrix, calculating −1 to solve −1= is an underdetermined inverse problem. A consequence of this is that pixel lengths are not generally going to be meaningful in terms of real world distances&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, that's not to say you could add additional sensors to resolve the depth estimation problem (ie. stereoscopic cameras or infrared distance sensors) if additional cost is not an issue.&lt;/p&gt;&#xA;" OwnerUserId="23289" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-06T03:11:18.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16278" PostTypeId="2" ParentId="16257" CreationDate="2019-11-05T10:33:24.830" Score="1" Body="&lt;p&gt;If you like papers, this keeps track of the state of the art (a lot of artificial intelligence, but no exclusively though)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://paperswithcode.com/sota&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://paperswithcode.com/sota&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The mission of Papers With Code is to create a free and open resource with Machine Learning papers, code and evaluation tables.&lt;/p&gt;&#xA;&lt;p&gt;We believe this is best done together with the community and powered by automation.&lt;/p&gt;&#xA;&lt;p&gt;We've already automated the linking of code to papers, and we are now working on automating the extraction of evaluation metrics from papers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="31049" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-06T01:16:04.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16279" PostTypeId="1" CreationDate="2019-11-05T10:34:53.090" Score="1" ViewCount="20" Body="&lt;p&gt;I've videos from a mounted camera on a helmet and the manually segmented labels (mask) of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mask is valid through the entire video, only the scene vary. &#xA;In different videos the camera is mounted differently on top of the helmet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I've tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Training semantic segmentation on frame-mask pairs&lt;/li&gt;&#xA;&lt;li&gt;Training semantic segmentation of concatenated frames with the mask.&lt;/li&gt;&#xA;&lt;li&gt;Averaging consecutive frames and calculating the time-wise std of the pixels, and feeding the NN with this as an input.&lt;/li&gt;&#xA;&lt;li&gt;ensembling (averaging) segmentation results from N frames&lt;/li&gt;&#xA;&lt;li&gt;Usage of classic background subtraction techniques such as MOG2 (worse)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although DNN models achieve 99% accuracy during training, in some of my test videos the model is missing large part of the helmet..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm certain this task can achieve ~100% accuracy even for never-before-seen examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have some ideas? &lt;/p&gt;&#xA;" OwnerUserId="25412" LastActivityDate="2019-11-05T10:34:53.090" Title="Segmentation of a static object in a video" Tags="&lt;computer-vision&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16282" PostTypeId="1" CreationDate="2019-11-05T11:08:37.507" Score="3" ViewCount="45" Body="&lt;p&gt;Is the Assumption-based Truth Maintenance System still used to maintain consistency while explicitly accounting for assumptions?&lt;/p&gt;&#xA;" OwnerUserId="31052" LastEditorUserId="2444" LastEditDate="2019-11-05T14:29:01.913" LastActivityDate="2019-11-05T14:29:01.913" Title="Is the Assumption-based Truth Maintenance System still used?" Tags="&lt;applications&gt;&lt;knowledge-representation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16283" PostTypeId="1" CreationDate="2019-11-05T11:29:41.587" Score="1" ViewCount="77" Body="&lt;p&gt;As far as I can tell, most work on facial recognition has been done in relation to human faces. Has any significant work been done for dogs, and are there any special challenges that would make it difficult to adapt existing work to this use case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify: I don't mean image classification, but segmentation. In other words, I refer to a model that can take an image that can have any number, including zero, of dogs in it, and that will produce bounding boxes around each dog's face.&lt;/p&gt;&#xA;" OwnerUserId="23857" LastEditorUserId="23857" LastEditDate="2019-11-05T23:42:52.840" LastActivityDate="2021-03-30T05:00:55.080" Title="What is the current state of the art in animal facial recognition?" Tags="&lt;computer-vision&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16285" PostTypeId="2" ParentId="16268" CreationDate="2019-11-05T14:27:54.977" Score="1" Body="&lt;p&gt;Your problem is a classification problem. If you are following the tutorial, it is using the ResNet50 network, which is a convolutional neural network with one fully connected layer at the end. At the end the activation function is softmax. Detailed description of the activation function can be found here: &lt;a href=&quot;https://medium.com/data-science-bootcamp/understand-the-softmax-function-in-minutes-f3a59641e86d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Softmax function explained&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/McVBq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/McVBq.png&quot; alt=&quot;Udacity Deep Learning Slide on Softmax&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, softmax increases the difference between the higher probability and lower probability. It also limits the output between 0 and 1. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problem Origin&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Due to the nature of the softmax function, it always chooses the best one and enlarge the value to be a value near one, even if the range of the output predictions is very small like 0-0.1. Also you training data only have the data of the 10 logos labelled, so if the network see unseen images with no logo it recognizes, it predicts the one with the most similarity. If you want to classify images with no logos in it, you should add an extra class in the training dataset and also the code to train the network to learn to classify images which is not in the 5 class into a separate maybe called unlabeled. Hope I can help you and have a nice day.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-05T14:27:54.977" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16286" PostTypeId="5" CreationDate="2019-11-05T14:33:09.720" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-05T14:33:09.720" LastActivityDate="2019-11-05T14:33:09.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16287" PostTypeId="4" CreationDate="2019-11-05T14:33:09.720" Score="0" Body="For questions related to the Vapnik–Chervonenkis theory (also known as VC theory), which a form of computational learning theory, so it attempts to explain the learning process from a statistical point of view, developed during 1960–1990 by Vladimir Vapnik and Alexey Chervonenkis." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-05T14:33:09.720" LastActivityDate="2019-11-05T14:33:09.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16288" PostTypeId="2" ParentId="16283" CreationDate="2019-11-05T14:40:09.173" Score="0" Body="&lt;p&gt;In case of animal classification - trying to specify the animal, trying to tell cat from a dog - it should be similar to human faces. An algorithm such as SVM could be used in that case. Trying to tell two breeds of dog apart would be more challenging. I have come across this article explaining how they use facial recognition software for animals &lt;a href=&quot;http://nymag.com/developing/2018/10/what-creatures-may-we-place-in-the-panopticon.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;news&lt;/a&gt;. And I have also come across  &lt;a href=&quot;http://320547161_Novel_method_for_identifying_individual_cetaceans_using_facial_features_and_symmetry_A_test_case_using_dolphins&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; where they write about a novel method for identifying individual cetaceans using facial features and symmetry on dolphins.&lt;/p&gt;&#xA;" OwnerUserId="31060" LastActivityDate="2019-11-05T14:40:09.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16289" PostTypeId="5" CreationDate="2019-11-05T14:47:38.543" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-05T14:47:38.543" LastActivityDate="2019-11-05T14:47:38.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16290" PostTypeId="4" CreationDate="2019-11-05T14:47:38.543" Score="0" Body="For questions related to the Vapnik–Chervonenkis (VC) dimension, originally defined by Vladimir Vapnik and Alexey Chervonenkis, which is a measure of the capacity (complexity, expressive power, richness, or flexibility) of a space of functions that can be learned by a statistical classification algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-05T14:47:38.543" LastActivityDate="2019-11-05T14:47:38.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16291" PostTypeId="1" CreationDate="2019-11-05T14:57:05.030" Score="3" ViewCount="70" Body="&lt;p&gt;In the book &quot;Perceptrons: An Introduction to Computational Geometry&quot; by Minsky and Papert (1969), which part of this book tells that a single-layer perceptron could not solve the XOR problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been already scanned it, but I did not find the part. Or am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="31061" LastEditorUserId="2444" LastEditDate="2021-01-19T02:18:49.130" LastActivityDate="2021-01-19T02:18:49.130" Title="Which part of &quot;Perceptrons: An Introduction to Computational Geometry&quot; tells that a perceptron cannot solve the XOR problem?" Tags="&lt;machine-learning&gt;&lt;perceptron&gt;&lt;books&gt;&lt;xor-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16294" PostTypeId="1" CreationDate="2019-11-05T16:48:38.490" Score="4" ViewCount="176" Body="&lt;p&gt;I am looking for a result that shows the convergence of semi-gradient TD(0) algorithm with non-linear function approximation for on-policy prediction. Specifically, the update equation is given by (borrowing notation from Sutton and Barto (2018))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf w \leftarrow \mathbf w +\alpha [R + \gamma \hat v(S', \mathbf w) - \hat v(S, \mathbf w)] \nabla \hat v(S, \mathbf w)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat v(S, \mathbf w)$&lt;/span&gt; is the approximate value function parameterized by &lt;span class=&quot;math-container&quot;&gt;$\mathbf w$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sutton and Barto (2018) mention that the above update equation converges when &lt;span class=&quot;math-container&quot;&gt;$\hat v$&lt;/span&gt; is linear in &lt;span class=&quot;math-container&quot;&gt;$\mathbf w$&lt;/span&gt;. But I couldn't find a similar result for non-linear function approximation. Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="31063" LastEditorUserId="2444" LastEditDate="2020-01-04T21:33:42.613" LastActivityDate="2020-01-04T21:33:42.613" Title="Convergence of semi-gradient TD(0) with non-linear function approximation" Tags="&lt;reinforcement-learning&gt;&lt;convergence&gt;&lt;function-approximation&gt;&lt;temporal-difference-methods&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16295" PostTypeId="1" CreationDate="2019-11-05T20:18:08.390" Score="2" ViewCount="38" Body="&lt;p&gt;So, I’m looking into some dynamic ways in which one can drive the behavior of a video game character. Specifically an NPC (Non playable character) that will be observable from the players point of view. Something I’d like to clarify from the start is what I mean by behavior. Since video games are understood visually, then I would qualify behavior to be anything visual, such as gestures, mannerisms or actions in any local space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let’s take a common archetype as an example. We’ll say we want the behavior of a villain. My first thought, was to use videos as training data. Videos of specific subjects or actors in a villainous scene (think Frankenstein, Dracula, Emperor Palpatine in Star Wars etc...) in hopes that an understanding of their mannerisms, body language and gestures could be captured and later applied to 3D animation dynamically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do understand that anything 3D typically requires rigging and animation. I’m currently not exactly sure how to marshal the data from one format (video analyses) to 3D animation. I thought I’d start working on the concept from high level first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts?&lt;/p&gt;&#xA;" OwnerUserId="20271" LastActivityDate="2019-11-05T20:18:08.390" Title="Can a Video Game Characters Behavior be directed by a NN?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16297" PostTypeId="5" CreationDate="2019-11-05T20:54:26.383" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-05T20:54:26.383" LastActivityDate="2019-11-05T20:54:26.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16298" PostTypeId="4" CreationDate="2019-11-05T20:54:26.383" Score="0" Body="For questions related to thought vectors, which are a generalization of word embeddings to e.g. sentences." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-05T20:54:26.383" LastActivityDate="2019-11-05T20:54:26.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16300" PostTypeId="5" CreationDate="2019-11-06T00:29:40.900" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Timeline_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Timeline_of_artificial_intelligence&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-06T00:29:40.900" LastActivityDate="2019-11-06T00:29:40.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16301" PostTypeId="4" CreationDate="2019-11-06T00:29:40.900" Score="0" Body="For questions about AI milestone, both those achieved and those predicted. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-06T00:29:40.900" LastActivityDate="2019-11-06T00:29:40.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16302" PostTypeId="2" ParentId="16274" CreationDate="2019-11-06T11:11:56.057" Score="1" Body="&lt;p&gt;There have been studies in University of Oregon and Kyoto University to be able to visualise thoughts and dreams on a screen using voxel values of an FMRI scan as input and an estimation of an image of the thoughts as the output. Instead of linking you to these studies and papers - you could just watch &lt;a href=&quot;https://youtu.be/AgbeGFYluEA&quot; rel=&quot;nofollow noreferrer&quot;&gt;this episode&lt;/a&gt; of mind field where both these studies are demonstrated and linked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind this is easier to understand if you have a good understanding of generative networks such as generative adversarial networks or so. Essentially in GAN's you'd map a known latent distribution to images in pixel-space. You would be doing the same thing here, just that the latent distribution would now be the FMRI scan input and the mapping would be made in a supervised setting where they are initially showed images. A very rough understanding of the idea can be drawn on these lines.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-06T11:11:56.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16304" PostTypeId="2" ParentId="2324" CreationDate="2019-11-06T11:24:00.790" Score="1" Body="&lt;p&gt;To avoid a repetitive answer that has been already spoken about such as absurdly high iterative ability or it being able to create another AGI system and multiplying or anything sci-fi like - there is one line of thought I feel people do not speak enough about. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our human senses are extremely limited i.e. we can see objects only when light from within the visible light spectrum (~ 400nm-700nm) reflects into our eyes, we can hear only a limited range of frequencies the rest being inaudible etc. An AGI system apart from its obvious intelligence, would be able to gain a significant amount of information from even common observations. It can see infrared, ultraviolet and radio waves as what we interpret as colours; it would be able to hear sounds that we did not know were being emitted at all. Essentially an AGI with good input sensor capabilities would be able to take information from experiencing the world as it actually is, and not a limited illusion we experience.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-06T11:24:00.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16306" PostTypeId="2" ParentId="15683" CreationDate="2019-11-06T12:06:55.977" Score="0" Body="&lt;p&gt;Would personally recommend deeplearning.ai's &lt;a href=&quot;https://www.coursera.org/specializations/deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;course&lt;/a&gt; to begin with. There may be more comprehensive or better MOOC's for covering basic MLP's, CNN, RNN's, tuning and training of neural networks but this is probably the most common one and the one that I can personally vouch for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this I'd recommend you get a physical or pdf copy of &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning by Goodfellow et al.&lt;/a&gt; and use it as reference material for any new idea you'd want to learn. Personally would not recommend reading the whole book and its better as a reference material as it is quite comprehensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should essentially be able to give you enough knowledge to be able to cover almost any paper/material on deep learning. The Course mentioned (most courses) would cover LSTM's as they are quite an old idea (~1997 I think) and GAN's are well covered in the book mentioned (The author invented them) since they are more of a recent advancement (2014). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this was helpful! &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-06T12:06:55.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16307" PostTypeId="5" CreationDate="2019-11-06T13:45:16.093" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-06T13:45:16.093" LastActivityDate="2019-11-06T13:45:16.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16308" PostTypeId="4" CreationDate="2019-11-06T13:45:16.093" Score="0" Body="For questions related to the hyper-parameters of AI models and algorithms, which are parameters that are set before the learning process begins. For example, the number of hidden layers in a feed-forward neural network is usually a hyper-parameter." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-06T13:45:16.093" LastActivityDate="2019-11-06T13:45:16.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16309" PostTypeId="5" CreationDate="2019-11-06T13:47:39.063" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-06T13:47:39.063" LastActivityDate="2019-11-06T13:47:39.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16310" PostTypeId="4" CreationDate="2019-11-06T13:47:39.063" Score="0" Body="For questions related to anomaly detection (or outlier detection) algorithms, which is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data. There are unsupervised, supervised and semi-supervised anomaly detection algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-06T13:47:39.063" LastActivityDate="2019-11-06T13:47:39.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16311" PostTypeId="1" AcceptedAnswerId="16313" CreationDate="2019-11-06T14:22:45.860" Score="6" ViewCount="181" Body="&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/abs/1503.03832&quot; rel=&quot;nofollow noreferrer&quot;&gt;FaceNet model&lt;/a&gt; returns the loss of the predictions and ground-truth classes. How is this loss calculated?&lt;/p&gt;&#xA;" OwnerUserId="30306" LastEditorUserId="2444" LastEditDate="2019-11-11T14:48:56.377" LastActivityDate="2019-11-11T14:48:56.377" Title="What is the formula used to calculate the loss in the FaceNet model?" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16312" PostTypeId="2" ParentId="16111" CreationDate="2019-11-06T15:37:27.367" Score="0" Body="&lt;p&gt;I'll share my understanding so far.&#xA;This kind of behavior is actually normal when using on-policy algorithms with a sparse final reward.&#xA;Issue stems from the fact that once you get stuck in a behavior policy which does nothing (uses a &quot;do nothing&quot; action, for instance, until timeout), it's quite hard to get out of it, because you keep getting experiences that teach you nothing (no reward signal at all) and keeps you in the same policy.&#xA;Possible mitigations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Encourage more exploration (in A3C, make the entropy loss coefficient bigger) in order to recover more quickly from this type of stationary behavior.&lt;/li&gt;&#xA;&lt;li&gt;Use an off-policy algorithm with a big enough replay buffer, so that even if you start behaving this way you still use experiences from a &quot;healthy&quot; older policy. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you stick to a totally on-policy algorithm, making the batch_size bigger might help a little.&lt;/p&gt;&#xA;" OwnerUserId="28125" LastActivityDate="2019-11-06T15:37:27.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16313" PostTypeId="2" ParentId="16311" CreationDate="2019-11-06T15:47:28.690" Score="6" Body="&lt;p&gt;The loss function used is the triplet loss function. &#xA;&lt;img src=&quot;https://i.stack.imgur.com/7uBuI.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;Let me explain it part by part.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Notation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$f^a_i$&lt;/span&gt; means the &lt;code&gt;anchor&lt;/code&gt; input image. The &lt;span class=&quot;math-container&quot;&gt;$f^p_i$&lt;/span&gt; means the &lt;code&gt;postive&lt;/code&gt; input image, which corresponds to the same people as the &lt;code&gt;anchor&lt;/code&gt; image. The &lt;span class=&quot;math-container&quot;&gt;$f^n_i$&lt;/span&gt; corresponds to the negative sample, which is a different person(input image) then the anchor image. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The formula explained step by step&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The first part, &lt;span class=&quot;math-container&quot;&gt;$||f^a_i - f^p_i||^2_2$&lt;/span&gt; basically calculates the distance between the &lt;code&gt;anchor&lt;/code&gt; image output features and the &lt;code&gt;postive&lt;/code&gt; image output features, which you want the distance to be as small as possible as the input is the same person. For the second part, &lt;span class=&quot;math-container&quot;&gt;$||f^a_i - f^n_i||^2_2$&lt;/span&gt; , it calculates the distance of the output features of the &lt;code&gt;anchor&lt;/code&gt; image and the &lt;code&gt;negative&lt;/code&gt; image. You wnat the distance to be as large as possible as they are not the same person. Finally, the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; term is a constant(hyperparameter) that adds to the loss to prevent negative loss. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;How it works&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The loss function optimizes for the largest distance between the anchor and negative sample and the smallest distance of the positive and anchor sample. It cleverly combines both metrics into one loss function. It can optimize for both case simultaneously in one loss function. If there is no negative sample, the model will not be able to differciate different person and vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="30306" LastEditDate="2019-11-11T08:49:18.283" LastActivityDate="2019-11-11T08:49:18.283" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16316" PostTypeId="1" AcceptedAnswerId="16342" CreationDate="2019-11-06T21:47:01.660" Score="5" ViewCount="149" Body="&lt;p&gt;Robot technology is usually thought from an engineering perspective. A human programmer writes a software this executed in a robot who is doing a task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what would happen, if the project is started with the opposite goal? The idea is, that the human becomes the robot by himself. That means, the human is using makeup to make his face more mechanically, buys special futuristic clothing which mirrors the light and imitates in a roleplay the working of a kitchen robot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What are methods human actors use to imitate robots?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user11571" LastEditorUserId="1671" LastEditDate="2019-12-02T21:41:02.670" LastActivityDate="2019-12-02T21:41:02.670" Title="What are methods human actors use to imitate robots?" Tags="&lt;social&gt;&lt;human-like&gt;&lt;mythology-of-ai&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16317" PostTypeId="5" CreationDate="2019-11-06T21:54:37.093" Score="0" Body="" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-06T21:54:37.093" LastActivityDate="2019-11-06T21:54:37.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16318" PostTypeId="4" CreationDate="2019-11-06T21:54:37.093" Score="0" Body="For question related to adoption of AI in the real world." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-06T21:54:37.093" LastActivityDate="2019-11-06T21:54:37.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16319" PostTypeId="2" ParentId="1644" CreationDate="2019-11-06T23:27:12.260" Score="1" Body="&lt;p&gt;There is some research on this topic. See, for example, the papers &lt;a href=&quot;http://people.idsia.ch/~gromov/repository/gromov2018robot.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robot Identification and Localization with Pointing Gestures&lt;/a&gt; (2018) and &lt;a href=&quot;http://people.idsia.ch/~gromov/repository/gromov2019proximity.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proximity Human-Robot Interaction Using Pointing Gestures&#xA;and a Wrist-mounted IMU&lt;/a&gt; (2019), by Boris Gromov et al., where the human is assumed to possess an inertial measurement unit (IMU) attached to the arm&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-11-06T23:27:12.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16320" PostTypeId="5" CreationDate="2019-11-06T23:29:06.393" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-06T23:29:06.393" LastActivityDate="2019-11-06T23:29:06.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16321" PostTypeId="4" CreationDate="2019-11-06T23:29:06.393" Score="0" Body="For questions related to human-computer, human-robot or human-AI interaction, in the context of AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-06T23:29:06.393" LastActivityDate="2019-11-06T23:29:06.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16322" PostTypeId="1" AcceptedAnswerId="16341" CreationDate="2019-11-07T01:02:53.640" Score="3" ViewCount="61" Body="&lt;p&gt;Until Chapter 6 of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto's book&lt;/a&gt; on Reinforcement Learning, the authors use &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; for the current estimate of a state value. Equation (6.1), for example, shows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ V(S_t) \leftarrow V(S_t) + \alpha[G_t - V(S_t)]\ \ \ \ \ \ (6.1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, on Chapter 7 they add a subscript to &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;. The first time this appears is on page 143 when they define the return from &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ G_{t:t+1} \dot{=} R_{t+1} + \gamma V_t(S_{t+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and say that &lt;span class=&quot;math-container&quot;&gt;$V_t : \mathcal{S} \rightarrow  \mathbb{R}$&lt;/span&gt; is &quot;the estimate at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt;.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first I thought I understood this as a natural consequence of considering &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps ahead in the future and needing an extra index to go over the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps. But then this stopped making sense when I realized that an estimate for a state must be consolidated, no matter at which of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps that is coming from. After all, a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; has a single value to estimate, &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;, and that does not depend on &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I thought that they are just taking into account that there are many successive estimates of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; as the algorithm progresses, so &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; is just the estimate after processing the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps starting at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. In other words, the subscript would be a rigorous mathematical way of denoting the sequence of algorithmic updates. But this does not make sense either since even in Chapter 6 and before, the estimate is &lt;em&gt;also&lt;/em&gt; successively updated. See Equation (6.1), for example. The &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; on the left-hand side is a different variable from the one on the right-hand side (this is why they must use &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; indicating an assignment as opposed to a mathematical equality with &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt;). It could have easily been written with an index as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what is the purpose of the new index for &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; in Chapter 7, and why is it more important at this particular chapter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit and elaboration&lt;/strong&gt;: Going back to the text, it seems to me that the new subscript is indeed added as an attempt for greater clarity, even though the subscript-less notation &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; from previous chapters might have been kept (and in fact it is still used in the pseudo-code in page 144).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems the authors wanted to stress that the update of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; happens not only for every trace of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps, but also at &lt;em&gt;every one&lt;/em&gt; of those steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think this introduced a technical error, because suppose we just learned from an 10-step episode (&lt;span class=&quot;math-container&quot;&gt;$T=10$&lt;/span&gt;), using &lt;span class=&quot;math-container&quot;&gt;$n = 3$&lt;/span&gt;. Then the latest estimate of &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$V_{T-1} = V_{10 - 1} = V_{9}$&lt;/span&gt;. Then at the next episode, the first time &lt;span class=&quot;math-container&quot;&gt;$V_{t + n}$&lt;/span&gt; is used to inform a target update, it will be for &lt;span class=&quot;math-container&quot;&gt;$\tau = 0$&lt;/span&gt; (from the pseudo-code), which implies &lt;span class=&quot;math-container&quot;&gt;$t - n + 1 = 0$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$t = n - 1$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$V_{t+n}=V_{n-1+n}=V_{2n-1}=V_5$&lt;/span&gt;, which is not the most up-to-date estimate &lt;span class=&quot;math-container&quot;&gt;$V_9$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the problem would be easily solved if we simply set the next used estimate &lt;span class=&quot;math-container&quot;&gt;$V_{2n + 1}$&lt;/span&gt; to be equal to the last episode's &lt;span class=&quot;math-container&quot;&gt;$V_{T-1}$&lt;/span&gt;, but to avoid confusion this would have to be explicitly stated somewhere.&lt;/p&gt;&#xA;" OwnerUserId="30679" LastEditorUserId="30679" LastEditDate="2019-11-07T05:30:53.797" LastActivityDate="2019-11-07T18:17:51.000" Title="Sutton &amp; Barto's notation $V_{t+n}$ in Chapter 7: $n$-step Bootstrapping" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16323" PostTypeId="1" CreationDate="2019-11-07T02:01:29.480" Score="4" ViewCount="121" Body="&lt;p&gt;I want to train an ANN. The problem is that the input features are completely unbounded (There are no boundaries as maximum and minimum for them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the following input vectors &lt;span class=&quot;math-container&quot;&gt;$(42, 54354354)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(0.4, 47239847329479324732984732947)$&lt;/span&gt; are both valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know RNNs that can add up input neurons, which are pretty similar to my case, but the number of the digits was limited in all of the implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to implement an ANN that can add up the input numbers of any magnitude?&lt;/p&gt;&#xA;" OwnerUserId="31092" LastEditorUserId="2444" LastEditDate="2019-11-07T16:11:42.867" LastActivityDate="2019-11-07T16:42:39.653" Title="Can neural networks deal with unbounded numbers as inputs?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16325" PostTypeId="5" CreationDate="2019-11-07T03:11:59.023" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;A multi-layer perceptron (MLP) is a class of feed-forward artificial neural network. An MLP consists of at least three layers of nodes. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called back-propagation for training. Its multiple layers and non-linear activation distinguish MLP from a linear perceptron. It can distinguish data that is not linearly separable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Multilayer_perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia - Multilayer perceptron&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="4709" LastEditDate="2020-08-05T16:02:47.313" LastActivityDate="2020-08-05T16:02:47.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16326" PostTypeId="4" CreationDate="2019-11-07T03:11:59.023" Score="0" Body="For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model. " OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-07T03:11:59.023" LastActivityDate="2019-11-07T03:11:59.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16327" PostTypeId="1" CreationDate="2019-11-07T07:24:35.957" Score="2" ViewCount="38" Body="&lt;p&gt;I am using a policy gradient algorithm (actor-critic) for wireless networks. The policy gradient-based algorithm helps because it considers continuous action space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how much does a policy gradient-based algorithm contribute to the complexity of the involved neural networks, compared to discrete action space algorithms (like Q-learning)? Moreover, in terms of computation, how do policy gradient algorithms (for continuous action spaces) compare to discrete action space algorithms?&lt;/p&gt;&#xA;" OwnerUserId="31095" LastEditorUserId="2444" LastEditDate="2019-11-08T14:29:09.200" LastActivityDate="2019-11-08T14:29:09.200" Title="What is the complexity of policy gradient algorithms compared to discrete action space algorithms?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;policy-gradients&gt;&lt;time-complexity&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16328" PostTypeId="1" CreationDate="2019-11-07T07:28:37.520" Score="2" ViewCount="51" Body="&lt;p&gt;How do you build a language model to predict the contextual similarity between two documents?&lt;/p&gt;&#xA;" OwnerUserId="31094" LastEditorUserId="2444" LastEditDate="2019-11-07T13:11:26.827" LastActivityDate="2019-11-08T03:18:25.977" Title="How do you build a language model to predict the contextual similarity between two documents?" Tags="&lt;natural-language-processing&gt;&lt;language-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16330" PostTypeId="1" AcceptedAnswerId="16335" CreationDate="2019-11-07T09:53:43.717" Score="4" ViewCount="119" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt; states, that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michael Nielsen states&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;No matter what the function, there is guaranteed to be a neural network so that for every possible input, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, the value &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; (or some close approximation) is output from the network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, for continuous functions, this seems plausible. Interestingly, in the same article, Nielsen mentioned &amp;quot;any function&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Later, he writes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;However, even if the function we'd really like to compute is discontinuous, it's often the case that a continuous approximation is good enough.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The last statement leaves open a gap, to ask how well an approximation practically can be.&lt;/p&gt;&#xA;&lt;p&gt;Let's ignore contradictory input/output training pairs like &lt;span class=&quot;math-container&quot;&gt;$f(0)=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(0)=1$&lt;/span&gt;, which actually don't event represent a function anyway.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, assume that the training data is generated randomly, which would practically result in a discontinuous function.&lt;/p&gt;&#xA;&lt;p&gt;How does a neural network learn such data? Will a learning algorithm always be able to find a neural network that approximates the function represented by the input-output pairs?&lt;/p&gt;&#xA;" OwnerUserId="10800" LastEditorUserId="2444" LastEditDate="2021-02-01T21:03:02.480" LastActivityDate="2021-02-01T21:03:02.480" Title="Does a neural network exist that can learn every possible training data?" Tags="&lt;neural-networks&gt;&lt;computational-learning-theory&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16331" PostTypeId="1" CreationDate="2019-11-07T11:30:58.147" Score="2" ViewCount="85" Body="&lt;p&gt;I am working on building a recommendation engine. I need to build a model that recommends similar items. Currently, I am using the Nearest Neighbor algorithm present in &lt;a href=&quot;https://scikit-learn.org/stable/modules/neighbors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn.neighbors&lt;/code&gt;&lt;/a&gt; package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working in finance domain, similarity can based on the &quot;Supplier&quot;, &quot;Buyer&quot;, &quot;Industry type&quot; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have attached sample data in the image below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BTFqr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BTFqr.png&quot; alt=&quot;Sample Data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any better machine learning algorithms/packages in Python for the same?&lt;/p&gt;&#xA;" OwnerUserId="29717" LastEditorUserId="29717" LastEditDate="2019-12-04T11:19:45.640" LastActivityDate="2019-12-04T11:19:45.640" Title="Which machine learning algorithms can be used to build a recommendation system?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="16332" PostTypeId="1" CreationDate="2019-11-07T12:37:58.473" Score="1" ViewCount="37" Body="&lt;p&gt;I would like to code a script that could locate a specific word or number in a financial statement. Financial statements roughly contain the same information, they are however not identical and organized in the same way. My thought is that by using Tensorflow I could train a neural network to locate the specific words or numbers for me. I am thinking that if I label different text and numbers in 1000 financial statements and use them to train the neural network, it will then be able to identify these numbers or words in all financial statements. For example, tell it in all 1000 training statements which number that is the profit of the company. Then when I give it an unseen financial statement, it should be able to identify which number that is the profit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this doable? I have been working with coding in python for a couple of months and so far I've built some web scrapers and integrated them with twitter, slack and google sheets. However, this would be my first AI related project. I would be very grateful for all your thoughts on this project and if anyone could steer me in the right direction by sharing relevant tutorials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="31101" LastEditorUserId="2444" LastEditDate="2019-11-07T17:07:38.300" LastActivityDate="2019-11-07T17:07:38.300" Title="How could I locate certain words or numbers in a financial statement?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16333" PostTypeId="2" ParentId="16328" CreationDate="2019-11-07T13:41:57.167" Score="1" Body="&lt;p&gt;You can use an autoencoder for text. For example, you can refer to this example here: &lt;a href=&quot;https://machinelearningmastery.com/lstm-autoencoders/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/lstm-autoencoders/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For comparing the contextual similarity, you can compare the encoded vectors for the distance maybe through a formula like mean squared error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works as the autoencoder compresses the input data into a vector of numbers, forcing the encoder and the decoder to learn specific features about text. The features cannot be understood by humans but have meanings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach will be using some word or document embedding like word2vec or GloVe. IT may also work well depending on your dataset size. You need to experiment through different methods to find out which is the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A supervised method can also be used if you have labels. For example, you can use a LSTM and train it like an Siamese network for LSTM using triplet loss. Here is an example: &lt;a href=&quot;https://medium.com/@gautam.karmakar/manhattan-lstm-model-for-text-similarity-2351f80d72f1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@gautam.karmakar/manhattan-lstm-model-for-text-similarity-2351f80d72f1&lt;/a&gt; Implementation: &lt;a href=&quot;https://github.com/GKarmakar/deep-siamese-text-similarity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/GKarmakar/deep-siamese-text-similarity&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can help you and have a nice day&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-11-08T03:18:25.977" LastActivityDate="2019-11-08T03:18:25.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16335" PostTypeId="2" ParentId="16330" CreationDate="2019-11-07T14:28:19.547" Score="1" Body="&lt;p&gt;The branch of AI research that answers questions like this is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_learning_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational learning theory&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the specific question you have asked, the universal approximation theorem does indeed prove that any function can be modeled by a sufficiently wide neural network. The definition of a function includes the requirement that each input be mapped to exactly one output, so contradictory labels in training data are excluded explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a rough sketch that can provide an intuition behind why this is true. This is not a proper proof, but it gives you an idea of the power of &quot;finite number of neurons&quot; in a hidden layer:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A single neuron can essentially learn to draw a straight line across the space of input features. It outputs a value arbitrarily close to 1 for things on one side of the line, and arbitrarily close to 0 for things on the other side of the line.&lt;/li&gt;&#xA;&lt;li&gt;For any given datapoint, it is possible to enclose the hyper-volume containing that datapoint, and &lt;em&gt;no others&lt;/em&gt; by drawing a series of lines in the input space, and defining one side of each line as &quot;inside&quot; the hyper-volume, and the other side as &quot;outside&quot;. In 2-d space, this corresponds to drawing the 4 sides of a square around a point.&lt;/li&gt;&#xA;&lt;li&gt;An output neuron learns to draw lines across the outputs of the hidden layer neurons. It can therefore decide to output 1 only when, say, 4 other neurons are all simultaneously active. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It should seem natural then that a sufficiently wide neural network can &lt;em&gt;memorize&lt;/em&gt; all of an input pattern. Since memorization is sufficient for &quot;learning&quot; to fit a pattern this should give you an intuition for the ability of neural networks to fit things.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-07T22:54:36.267" LastActivityDate="2019-11-07T22:54:36.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16336" PostTypeId="1" AcceptedAnswerId="16562" CreationDate="2019-11-07T14:48:22.037" Score="2" ViewCount="122" Body="&lt;p&gt;I was going through the &lt;a href=&quot;https://gist.github.com/karpathy/a4166c7fe253700972fcbc77e4ea32c5&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; by Andrej Karpathy on reinforcement learning using a policy gradient. I have some questions from the code.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Where is the logarithm of the probability being calculated? Nowhere in the code I see him calculating that. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Please explain to me the use of &lt;code&gt;dlogps.append(y - aprob)&lt;/code&gt; line. I know this is calculating the &lt;code&gt;loss&lt;/code&gt;, but how is this helping in a reinforcement learning environment, where we don't have the correct labels?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How is &lt;code&gt;policy_backward()&lt;/code&gt; working? How are the weights changing to the loss function mentioned above? More specifically, what's &lt;code&gt;dh&lt;/code&gt; here?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29843" LastEditorUserId="2444" LastEditDate="2019-11-08T15:18:17.560" LastActivityDate="2019-11-16T20:36:56.867" Title="How is gradient being calculated in Andrej Karpathy's pong code?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;backpropagation&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16338" PostTypeId="2" ParentId="16323" CreationDate="2019-11-07T16:42:39.653" Score="3" Body="&lt;p&gt;Depending on the activation functions in the first layer, this is not a real problem, many functions such as &lt;span class=&quot;math-container&quot;&gt;$\tanh(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma(x)$&lt;/span&gt; (sigmoid) have asymptotic upper and lower bounds, so the huge input values swamp all other inputs in the corresponding neurons, but the output is well-behaved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a conversion before the input layer may be appropriate for some kind of data. For example, you might want to take the &lt;span class=&quot;math-container&quot;&gt;$\log(x)$&lt;/span&gt; of the input values such that the initial weight networks work with the relation of the values instead of their absolute magnitude. This may be useful for audio data such as speech or music and many other measurements of signals that may be attenuated before the measurement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your numbers don't actually represent magnitudes of some kind but arbitrary sequences of digits, you should use a network that can deal with sequences, of course.&lt;/p&gt;&#xA;" OwnerUserId="22993" LastActivityDate="2019-11-07T16:42:39.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16339" PostTypeId="5" CreationDate="2019-11-07T16:48:10.870" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-07T16:48:10.870" LastActivityDate="2019-11-07T16:48:10.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16340" PostTypeId="4" CreationDate="2019-11-07T16:48:10.870" Score="0" Body="For questions related to feedforward neural networks (FFNNs), which are also sometimes called multilayer perceptrons, but these two expressions may not always be interchangeable." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-07T16:48:10.870" LastActivityDate="2019-11-07T16:48:10.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16341" PostTypeId="2" ParentId="16322" CreationDate="2019-11-07T18:17:51.000" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So, what is the purpose of the new index for &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; in Chapter 7, and why is it more important at this particular chapter?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My guess would be that your intuition is correct, and that it's mostly introduced just to clarify exactly which &quot;version&quot; of our value function approximator is going to be used in any particular equation. In previous chapters, which discuss single-step update rules, I guess the authors assumed there was less potential for confusion, and therefore no need to clarify this. Without the clarification, some people might for instance wonder if we should use &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; for our value estimates of an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step return &lt;span class=&quot;math-container&quot;&gt;$G_{t+n}$&lt;/span&gt;, regardless of how large &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I think this introduced a technical error, because suppose we just learned from an 10-step episode (&lt;span class=&quot;math-container&quot;&gt;$T=10$&lt;/span&gt;), using &lt;span class=&quot;math-container&quot;&gt;$n = 3$&lt;/span&gt;. Then the latest estimate of &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$V_{T-1} = V_{10 - 1} = V_{9}$&lt;/span&gt;. Then at the next episode, the first time &lt;span class=&quot;math-container&quot;&gt;$V_{t + n}$&lt;/span&gt; is used to inform a target update, it will be for &lt;span class=&quot;math-container&quot;&gt;$\tau = 0$&lt;/span&gt; (from the pseudo-code), which implies &lt;span class=&quot;math-container&quot;&gt;$t - n + 1 = 0$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$t = n - 1$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$V_{t+n}=V_{n-1+n}=V_{2n-1}=V_5$&lt;/span&gt;, which is not the most up-to-date estimate &lt;span class=&quot;math-container&quot;&gt;$V_9$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Once we start considering a situation with more than a single episode, the &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; notation becomes quite confusing. You should read &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; as &quot;the value function approximator that we have available &lt;strong&gt;at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; of the current episode&lt;/strong&gt;&quot;. So, if we were to use the symbol &lt;span class=&quot;math-container&quot;&gt;$V_0$&lt;/span&gt; within the context of a second episode, that would be identical to what was referred to as &lt;span class=&quot;math-container&quot;&gt;$V_T$&lt;/span&gt; in the context of the first episode. The &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; notation can be convenient if we're thinking about our equations with our minds in &quot;math-mode&quot;, but becomes highly confusing once we start thinking about practical implementations involving multiple episodes -- this is probably why they did not include it in the pseudocode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really wanted to use the subscript-notation in the pseudocode, you'd have to add an extra term in the subscript that adds up all the durations of all previous episodes. If we then try to work out your example situation, we'd run into another problem though... we'd want to use &lt;span class=&quot;math-container&quot;&gt;$V_{t+n+T} = V_{2n-1+T} = V_{15}$&lt;/span&gt; at the first iteration where &lt;span class=&quot;math-container&quot;&gt;$\tau = 0$&lt;/span&gt; in the second episode. But, across the two episodes, only &lt;span class=&quot;math-container&quot;&gt;$13$&lt;/span&gt; steps have passed, so this does not yet exist! You run into the same issue if you try to work out what happened when &lt;span class=&quot;math-container&quot;&gt;$\tau = 0$&lt;/span&gt; in the first episode: applying exactly the same reasoning as in your quote, we would have wanted to use &lt;span class=&quot;math-container&quot;&gt;$V_5$&lt;/span&gt; after only &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; time steps passed in the first episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is that you're trying to use the variable named &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the pseudocode as the subscript for &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;. To get a better idea of what's going on here, let's loop back to the previous page and examine the definition of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step return:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+n} \doteq R_{t+1} + \gamma R_{t+2} + \dots + \gamma^{n-1} R_{t+n} + \gamma^n V_{t+n-1} (S_{t+n}).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, we've got that. Now, let's take another look at the update rule in which we use this quantity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_{t+n}(S_t) \doteq V_{t+n-1}(S_t) + \alpha \left[ G_{t:t+n} - V_{t+n-1} (S_t) \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok. So &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}$&lt;/span&gt; appears three times in the update rule. Two times explicitly, estimating the value of &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;, and once more &quot;hidden&quot; in the definition of &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}$&lt;/span&gt;, where it is used to estimate the value of &lt;span class=&quot;math-container&quot;&gt;$S_{t+n}$&lt;/span&gt;. Note very carefully what it is that this update rule is doing; &lt;strong&gt;it's updating the state estimate of &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;&lt;/strong&gt;. If you now look at the pseudocode again, you'll see a comment on the line where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is computed: &lt;strong&gt;(&lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the time whose state's estimate is being updated)&lt;/strong&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this means, is that in the pseudocode, you should be using &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; as the subscript for &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;! If you do that, it'll at least be correct for the first episode. In the pseudocode, the update rule looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(S_{\tau}) \gets V(S_{\tau}) + \dots$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plugging in the subscripts from the mathematical definition leads to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_{\tau + n}(S_{\tau}) \gets V_{\tau + n - 1}(S_{\tau}) + \dots$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the pseudocode defines &lt;span class=&quot;math-container&quot;&gt;$\tau = t - n + 1$&lt;/span&gt;, we can substitute above:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;V_{t - n + 1 + n}(S_{t - n + 1}) &amp;amp;\gets V_{t - n + 1 + n - 1}(S_{t - n + 1}) + \dots\\&#xA;V_{t + 1}(S_{t - n + 1}) &amp;amp;\gets V_{t}(S_{t - n + 1}) + \dots&#xA;\end{aligned}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and now it should make sense again from a practical point of view. At every time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; measures number of steps of experience that we have simulated, we simply use the latest value function &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; we have available at that time for bootstrapping in the update rule. When &lt;span class=&quot;math-container&quot;&gt;$t + 1 &amp;lt; n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$S_{t - n + 1}$&lt;/span&gt; is undefined. In these cases, the above update rule doesn't work, which makes sense intuitively because we have not yet progressed far enough into the episode to be capable of compute &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-11-07T18:17:51.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16342" PostTypeId="2" ParentId="16316" CreationDate="2019-11-07T18:24:21.580" Score="4" Body="&lt;p&gt;The great acting teacher &lt;a href=&quot;https://en.wikipedia.org/wiki/Stella_Adler&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stella Adler&lt;/a&gt; wrote about &lt;a href=&quot;https://en.wikipedia.org/wiki/Mannerism&quot; rel=&quot;nofollow noreferrer&quot;&gt;mannerisms&lt;/a&gt; being a powerful tool for actors. &lt;a href=&quot;https://en.wikipedia.org/wiki/Method_acting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Method acting&lt;/a&gt; in general focuses on natural performances based roughly on understanding the mindset of the character portrayed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible actors who have portrayed androids have observed industrial robots to inform their physicality, and many performances convey the idea, via movement, of a mechanical inner structure.  (It is often said that an &quot;actor's body is their instrument&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is more interesting is actors trying to convey the cognitive structure of the androids.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Arnold, and &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Terminator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Terminator&lt;/a&gt; robots in general, the baseline performance is decidedly robotic, to convey their inhumanity. But the more advanced Terminators are able to mimic naturalistic human mannerisms, and even established human characters, to trick humans. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Data_(Star_Trek)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lieutenant Data&lt;/a&gt; often used head motions, such as cocking his head slightly, to convey computation.  Here the character arc involved working to become more human, as this character draws heavily on &lt;a href=&quot;https://en.wikipedia.org/wiki/Pinocchio&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pinocchio&lt;/a&gt;, the wooden puppet that became a boy.  Overall Data's performance conveyed a lack of emotion, a definite reference to the logic-oriented &lt;a href=&quot;https://en.wikipedia.org/wiki/Spock&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mr. Spock&lt;/a&gt;, although I recall episodes where Data experimented with &quot;emotional circuits&quot; and &quot;humor circuits&quot;, where the output was intentionally inconsistent with natural human behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Blade_Runner#Plot&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Blade Runner&lt;/em&gt;&lt;/a&gt;, where the &lt;a href=&quot;https://bladerunner.fandom.com/wiki/Tyrell_Corporation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tyrell Corporation&lt;/a&gt;'s motto was &quot;More Human than Human&quot;, presented the cutting edge Nexus-6 androids as having emotions, but, due to their artificially short life-spans, were portrayed as childlike in trying to reconcile extremely powerful feelings.  The &lt;a href=&quot;https://bladerunner.fandom.com/wiki/Voight-Kampff_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Voight-Kampff Test&lt;/a&gt;, a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Test&lt;/a&gt;, used in the film to identify androids, relied on the emotional response to questions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key plot point of &lt;a href=&quot;https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_Sheep%3F#Plot_summary&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Do Androids Dream of Electric Sheep&lt;/em&gt;&lt;/a&gt;, the novel the film was based on, utilized what would be formalized as &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory#Routes_to_altruism&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolutionary game theory&lt;/a&gt; to hypothesize that empathy is a natural function of intelligence sufficiently advanced.  Deckard, who may or may not have been an android, and Rachel, who definitely was, are both capable of love.  This capacity informed their performances, to the extent that the androids came off as more human than the actual humans, due to the depth of their emotion.  This is also reflected in &lt;a href=&quot;https://en.wikipedia.org/wiki/Blade_Runner_2049#Plot&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Blade Runner 2049&lt;/em&gt;&lt;/a&gt; via the girlfriend-bot Joi, who us the most limited android, but the most human character in the film per her capacity to love (or at least simulate it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the recent &lt;a href=&quot;https://en.wikipedia.org/wiki/Westworld_(TV_series)&quot; rel=&quot;nofollow noreferrer&quot;&gt;HBO Westworld&lt;/a&gt; reboot, the Androids replicate natural human mannerisms when playing their designated roles, but reset to more mechanical mannerisms when acting under their own agency.  This is reflected in &lt;a href=&quot;https://en.wikipedia.org/wiki/Ex_Machina_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Ex Machina&lt;/em&gt;&lt;/a&gt;, where the android mimics human emotions to pass a Turing Test and trick the human subject, only to revert to purely alien mannerisms after the android is free.  (&quot;Alien&quot; here used in the sense of non-human--it's possible the android is sentient as it seems to convey some degree of emotion in regarding the simulated human skin it will wear.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most interesting recent android performance may come from the recent &lt;a href=&quot;https://en.wikipedia.org/wiki/David_8#Character_development&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Alien: Covenant&lt;/em&gt;&lt;/a&gt; where &lt;strong&gt;Michael Fassbender plays two identical androids, David and Walter, which have two distinct neural structures&lt;/strong&gt;.  (David has the capacity to be creative, where Walter cannot.  In the film it is mentioned that David made people uncomfortable, so the creative functions were removed from subsequent models.)  The key difference in the performance seems to be that David demonstrates passion, and even emotions, where Walter is more clearly &quot;robotic&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In general, the underlying approach of actors seems to have been to show the androids being distinct from humans, drawing a clear, though sometimes subtle, contrast.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Actors portraying androids have typically utilized robotic mannerisms to convey an artificial entity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-07T20:00:36.063" LastActivityDate="2019-11-07T20:00:36.063" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16343" PostTypeId="1" CreationDate="2019-11-07T20:49:26.637" Score="1" ViewCount="182" Body="&lt;p&gt;I'm trying to train a neural network with my own dataset. The neural network can accept the &lt;a href=&quot;https://www.cityscapes-dataset.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;cityscape format&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any application that can give mask/segmented image, instance image, label IDs images and JSON file, similar to cityscape dataset format? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I want to create my own dataset similar to the cityscape dataset format. &lt;/p&gt;&#xA;" OwnerUserId="27221" LastEditorUserId="2444" LastEditDate="2019-11-08T18:46:28.490" LastActivityDate="2019-11-08T18:46:28.490" Title="creating your own dataset similar to cityscapes format" Tags="&lt;training&gt;&lt;datasets&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16346" PostTypeId="1" AcceptedAnswerId="16359" CreationDate="2019-11-08T02:22:40.693" Score="2" ViewCount="103" Body="&lt;p&gt;The brevity penalty is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$bp = e^{(1- r/c)},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the reference length and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the output length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what happens if the output length gets zero? Is there any standard way of coping with that issue?&lt;/p&gt;&#xA;" OwnerUserId="21685" LastEditorUserId="2444" LastEditDate="2019-11-08T14:10:07.173" LastActivityDate="2019-11-08T20:00:38.727" Title="What happens when the output length in the brevity penalty is zero?" Tags="&lt;natural-language-processing&gt;&lt;machine-translation&gt;&lt;bleu&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16347" PostTypeId="1" AcceptedAnswerId="16352" CreationDate="2019-11-08T02:31:39.923" Score="1" ViewCount="161" Body="&lt;p&gt;I am trying to understand the policy gradient method using &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;a PyTorch implementation&lt;/a&gt; and &lt;a href=&quot;https://towardsdatascience.com/policy-gradients-in-a-nutshell-8b72f9743c5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is about the end result of this gradient derivation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;\nabla \mathbb{E}_{\pi}[r(\tau)] &amp;amp;=\nabla \int \pi(\tau) r(\tau) d \tau \\&#xA;&amp;amp;=\int \nabla \pi(\tau) r(\tau) d \tau \\&#xA;&amp;amp;=\int \pi(\tau) \nabla \log \pi(\tau) r(\tau) d \tau \\&#xA;\nabla \mathbb{E}_{\pi}[r(\tau)] &amp;amp;=\mathbb{E}_{\pi}[r(\tau) \nabla \log \pi(\tau)]&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mainly in this equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla \mathop{\mathbb{E}_\pi }[r(\tau )] = \mathop{\mathbb{E}_\pi }[r(\tau )\nabla log \pi (\tau )]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does expectation follow a distributive or associative property?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that expectations of a function can be written as below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathop{\mathbb{E}}[f(x)] =\sum p(x)f(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then can we rewrite the first equations as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathop{\mathbb{E}_\pi }[r(\tau )\nabla log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\nabla log \pi (\tau )] \\= \sum p(\tau)r(\tau ) \,\, \sum p(\tau)\nabla log \pi (\tau ) \\&#xA;= p(\tau) \sum r(\tau ) \nabla log \pi (\tau )$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is when I compare  this to PyTorch implementation (line 71-74)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for log_prob, R in zip(policy.saved_log_probs, returns):&#xA;    policy_loss.append(-log_prob * R)&#xA;optimizer.zero_grad()&#xA;policy_loss = torch.cat(policy_loss).sum()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The  pytorch implementation simply multiplied log probability and reward &lt;code&gt;-log_prob * R&lt;/code&gt; and then summed the vector &lt;code&gt;torch.cat(policy_loss).sum()&lt;/code&gt; there is no &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt;. What is really happening here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second question is the multiplication of log probability and reward in PyTorch implementation &lt;code&gt;-log_prob * R&lt;/code&gt;, PyTorch implementation has a &lt;strong&gt;negative&lt;/strong&gt; log probability and derived equation has a positive one &lt;span class=&quot;math-container&quot;&gt;$\mathop{\mathbb{E}_\pi }[r(\tau )\nabla log \pi (\tau )]$&lt;/span&gt;. What is the need for multiplying log probability with a negative value in PyTorch implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have only a basic understanding of maths and that's why I am asking this question here.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: found a better derivation of above equation &lt;a href=&quot;https://youtu.be/Ys3YY7sSmIA?t=3622&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/Ys3YY7sSmIA?t=3622&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2020-05-17T22:11:37.663" LastActivityDate="2020-05-17T22:11:37.663" Title="How does the policy gradient's derivative work?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;pytorch&gt;&lt;reinforce&gt;&lt;expectation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16348" PostTypeId="1" CreationDate="2019-11-08T03:23:00.550" Score="6" ViewCount="4834" Body="&lt;p&gt;I came across this &lt;a href=&quot;https://www.quora.com/What-is-convergence-in-neural-network&quot; rel=&quot;noreferrer&quot;&gt;answer on Quora&lt;/a&gt;, but it was pretty sparse.  I'm looking for specific meanings in the context of machine learning, but also mathematical and economic notions of the term in general.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-08T03:36:37.360" LastActivityDate="2021-02-13T10:14:32.963" Title="What is convergence in machine learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;terminology&gt;&lt;theory&gt;&lt;convergence&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16349" PostTypeId="2" ParentId="16316" CreationDate="2019-11-08T03:29:38.257" Score="1" Body="&lt;p&gt;&lt;em&gt;Disclaimer: The intent of this answer is to suggest a a parallel between methods of acting and machine learning, both in intent and application, and theory.  A large number of links are included for the convenience of readers new to the field, and there is not an exact correspondence of AI concepts to acting preparation techniques.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my &lt;a href=&quot;https://ai.stackexchange.com/a/16342/1671&quot;&gt;prior answer&lt;/a&gt;, I mentioned the &lt;a href=&quot;https://en.wikipedia.org/wiki/Method_acting&quot; rel=&quot;nofollow noreferrer&quot;&gt;method acting&lt;/a&gt; technique, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Stella_Adler&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stella Adler&lt;/a&gt;'s interpretation of &lt;a href=&quot;https://en.wikipedia.org/wiki/Konstantin_Stanislavski&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanislavski's method&lt;/a&gt;.  Bear in mind that the method is a post-empiricism approach, an attempt, in some sense, to create a science of acting in the sense of analysis, and an approach that is fundamentally algorithmic in the sense of process.  (The original manual is titled &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/An_Actor_Prepares#Summary&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Actor Prepares&lt;/a&gt;&lt;/em&gt;.)  Note that areas covered include action, imagination (creativity), units and objectives, emotional memory (accessing memory), and adaptation.  See Also: &lt;a href=&quot;https://en.wikipedia.org/wiki/Classical_acting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classical Acting&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that &lt;a href=&quot;https://www.etymonline.com/word/play&quot; rel=&quot;nofollow noreferrer&quot;&gt;plays&lt;/a&gt; are aptly named.  Drama and comedy arise out of interplay of individuals, and the process of refining performance is the process of play—searching within a rule-space for the most optimal outcome.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Strong actors will rigorously research the character to create a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mental_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;mental model&lt;/a&gt; of the character's experience of the world, similar to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;model-based agent&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Modern actors seek objectives, sometimes referred to as motivations, similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent#/media/File:Model_based_goal_based_agent.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;goal-based agents&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The model has many dimensions, and there may be multiple layers of objectives in the sense of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Subconscious&quot; rel=&quot;nofollow noreferrer&quot;&gt;subconscious&lt;/a&gt;.  (What does the character want?  What does it really want?  What does it really really want?) This also applies to the contexts for any choice, which are multiple (personal, societal, economic, etc.) &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Actors observe human behavior for the purpose of &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test#Imitation_Game&quot; rel=&quot;nofollow noreferrer&quot;&gt;imitating&lt;/a&gt; it, commonly referred to as &quot;people watching&quot;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As you note, actors preparing for role of robot may observe machinery, with the purpose of indicating for an audience that quality.  Actors may also observe other actors, although novelty in performance is typically understood to be optimal.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Actors will access emotional memory, alternately referred to as &quot;sense memory&quot; &amp;amp; &quot;emotional recall&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Affective_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;affective memory&lt;/a&gt;), either to produce a physical effect or or analysis. Output are &lt;a href=&quot;https://en.wikipedia.org/wiki/Semiotics#Terminology&quot; rel=&quot;nofollow noreferrer&quot;&gt;signifiers&lt;/a&gt;. They create a &lt;a href=&quot;https://en.wikipedia.org/wiki/State_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;state space&lt;/a&gt; which they can return to access on command.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Essentially it's a form of &quot;memory palace&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Method_of_loci&quot; rel=&quot;nofollow noreferrer&quot;&gt;method of loci&lt;/a&gt;) where events take the place of locations.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Actors will improvise in preparation, to identify and test &lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_choice_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;choices&lt;/a&gt; (actions and mannerisms), which involves &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision theory&lt;/a&gt;. The choices of the other actors (&lt;a href=&quot;https://en.wikipedia.org/wiki/Rational_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;rational agents&lt;/a&gt;) are factor, and influence each other.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Choices are selected in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic process&lt;/a&gt;, for fitness in environment, here defined as audience response.  The improvisation that leads to the performance is &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolutionary&lt;/a&gt;, in that it optimizes via the rehearsal process, with director as audience, and later, in the case of live theater, in response to live audiences. (See also the &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node66.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Actor-critic&lt;/a&gt; model.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's not quite a &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;monte-carlo&lt;/a&gt;, being more of an &lt;a href=&quot;https://en.wikipedia.org/wiki/Search_algorithm#Informed_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;informed search&lt;/a&gt;, but does not exclude randomness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, it's a process of analysis, trial-and-error, more analysis, repeat, similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning&lt;/a&gt; with &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;heuristics&lt;/a&gt;.  It wouldn't be far off to say that there is a &lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/288777/what-does-it-mean-for-an-algorithm-to-converge&quot;&gt;convergence&lt;/a&gt;, leading to what is perceived to be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;optimal&lt;/a&gt; set of choices, (although it is more typical to say a performance &quot;gels&quot; or &quot;comes together&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It can be said that modern acting methods are themselves algorithmic processes, where the intent is &lt;a href=&quot;https://en.wikipedia.org/wiki/Utility&quot; rel=&quot;nofollow noreferrer&quot;&gt;maximizing utility&lt;/a&gt;, here audience response, which can carry significant economic consequences.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Modern actors are using methods similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;modern AI methods&lt;/a&gt; to imitate intelligent &lt;a href=&quot;https://en.wikipedia.org/wiki/Android_(robot)&quot; rel=&quot;nofollow noreferrer&quot;&gt;androids&lt;/a&gt;!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the sense of Adler specifically, the technique involves &lt;em&gt;simulating&lt;/em&gt; natural emotions to &quot;trick&quot; the observer, a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Affective_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Affective computing&lt;/a&gt;.   In other words, via training, the actor is doing what AI's are being trained to do in the context of interacting with humans.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying method can be understood as a form of applied &lt;a href=&quot;https://en.wikipedia.org/wiki/Psychology#Major_schools_of_thought&quot; rel=&quot;nofollow noreferrer&quot;&gt;psychology&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroscience&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroscience&lt;/a&gt;, where the actor is accessing emotion for the purpose of analysis, and accessing specific parts of the brain on command to create observable &lt;a href=&quot;https://en.wikipedia.org/wiki/Sign_(semiotics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;signs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-09T00:28:23.443" LastActivityDate="2019-11-09T00:28:23.443" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16350" PostTypeId="2" ParentId="16348" CreationDate="2019-11-08T04:50:21.047" Score="4" Body="&lt;p&gt;When formulating a problem in deep learning, we need to come up with a loss function, which uses model weights as parameters. Back-propagation starts at an arbitrary point on the error manifold defined by the loss function and with every iteration intends to move closer to a point that minimises error value by updating the weights. Essentially for every possible set of weights the model can have, there is an associated loss for a given loss function, with our goal being to find the minimum point on this manifold. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergence is a term mathematically most common in the study of series and sequences. A model is said to converge when the series &lt;span class=&quot;math-container&quot;&gt;$s(n) = loss_{w_n}(\hat y, y)$&lt;/span&gt; (Where &lt;span class=&quot;math-container&quot;&gt;$w_n$&lt;/span&gt; is the set of weights after the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;'th iteration of back-propagation and &lt;span class=&quot;math-container&quot;&gt;$s(n)$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;'th term of the series) is a converging series. The series is of course an infinite series only if you assume that loss = 0 is never actually achieved, and that learning rate keeps getting smaller. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially meaning, a model converges when its loss actually moves towards a minima (local or global) with a decreasing trend. Its quite rare to actually come across a strictly converging model but convergence is commonly used in a similar manner as convexity is. Strictly speaking rarely exists practically, but is spoken in a manner telling us how close the model is to the ideal scenario for convexity, or in this case convergence. &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-08T04:50:21.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16351" PostTypeId="1" CreationDate="2019-11-08T05:17:49.110" Score="1" ViewCount="20" Body="&lt;p&gt;Let's say I have a dataset, each item/row of which has &lt;span class=&quot;math-container&quot;&gt;$\mathit{X + 1}$&lt;/span&gt; characteristics where the last characteristic (i.e., the &lt;span class=&quot;math-container&quot;&gt;$\mathit{1}$&lt;/span&gt;) represents the some value I want to predict, &lt;span class=&quot;math-container&quot;&gt;$\mathit{Y}$&lt;/span&gt;, based on a SOM trained on the &lt;span class=&quot;math-container&quot;&gt;$\mathit{X}$&lt;/span&gt; characteristics.  I want to organize the dataset into groups such that each group has a small variance among the respective &lt;span class=&quot;math-container&quot;&gt;$\mathit{Y}$&lt;/span&gt; values.  I believe I could do this by using a non-Euclidean distance to find the Best Matching Unit (BMU) based on applying weights to each dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given a node at (0,0) and weights for dimension &lt;span class=&quot;math-container&quot;&gt;$\mathit{x}$&lt;/span&gt; of 1 and dimension &lt;span class=&quot;math-container&quot;&gt;$\mathit{y}$&lt;/span&gt; of 2, a data point at (3,2) would have a weighted distance of 5 from the node, calculated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sqrt{\mathit{(1 * (3 - 0)) ^ 2 + (2 * (2 - 0)) ^ 2}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think a simple linear regression would work to determine the weights because it would not take advantage of clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal would be, for a new data point, to approximate a probability distribution of outcomes based on similarly-profiled data points in the training set (i.e., retrieve all of the training results with the same BMU and analyze the results).  I think this might essentially just be replicating a deep feedforward network, but I'd like to try it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I could achieve this by modifying a SOM model or using a similar technique?&lt;/p&gt;&#xA;" OwnerUserId="30154" LastEditorUserId="30154" LastEditDate="2019-11-08T06:17:02.537" LastActivityDate="2019-11-08T06:17:02.537" Title="Self-organizing map using weighted non-euclidean distance to minimize variance of predictions" Tags="&lt;feedforward-neural-network&gt;&lt;self-organizing-map&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16352" PostTypeId="2" ParentId="16347" CreationDate="2019-11-08T08:24:04.223" Score="2" Body="&lt;p&gt;You cannot do this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is because &lt;span class=&quot;math-container&quot;&gt;$r(\tau )$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\bigtriangledown log \pi (\tau )$&lt;/span&gt; are correlated by their dependence on &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;. In a simpler concrete example, if your expectation was over simple equiprobable discrete distribution where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; could be any integer in range &lt;span class=&quot;math-container&quot;&gt;$[1,10]$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\mathop{\mathbb{E} }[\tau^2] = 38.5$&lt;/span&gt; whilst &lt;span class=&quot;math-container&quot;&gt;$\mathop{\mathbb{E} }[\tau]\mathop{\mathbb{E} }[\tau] = 30.25$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The  pytorch implementation simply multiplied log probability and reward &lt;code&gt;-log_prob * R&lt;/code&gt; and then summed the vector &lt;code&gt;torch.cat(policy_loss).sum()&lt;/code&gt; there is no &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt;. What is really happening here?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The purpose of transforming the gradient into an &lt;em&gt;expectation&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}$&lt;/span&gt; for the policy gradient theorem, is so that you can estimate it using &lt;em&gt;samples&lt;/em&gt; taken from the distribution. Typically, you don't know &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt;, but you do know that if you follow the same process where &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt; applies (i.e. measure the return from the environment whilst following the policy represented by the policy function) that you will get an unbiased &lt;em&gt;sample&lt;/em&gt; from that distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is going on here is that you throw away the outer expectation &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}[]$&lt;/span&gt; and replace it with a stochastic estimate for the same value based on taking samples. The samples are naturally obtained with distribution described by &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt;, if you follow the policy function when making action choices.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The second question is the multiplication of log probability and reward in pytorch implementation &lt;code&gt;-log_prob * R&lt;/code&gt;, pytorch implementation has a &lt;strong&gt;negative&lt;/strong&gt; log probability and derived equation has a positive one &lt;span class=&quot;math-container&quot;&gt;$\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )]$&lt;/span&gt;. What is the need for multipling log probability with negative value in pytorch implementaion?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know the code, but this very likely because of a sign change brought on by considering how to respond to the gradient estimate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a clue in the use of the name &quot;loss&quot;. To maximise return in policy gradient methods, you can perform gradient &lt;em&gt;ascent&lt;/em&gt; based on the estimated gradient as the goal is to find higher values. However, it is more usual in NN libraries to perform gradient &lt;em&gt;descent&lt;/em&gt; in order to minimise a loss function. That is a likely cause of the sign reversal here.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-11-08T13:53:37.633" LastActivityDate="2019-11-08T13:53:37.633" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16353" PostTypeId="1" CreationDate="2019-11-08T09:39:43.477" Score="4" ViewCount="92" Body="&lt;p&gt;Facebook has &lt;a href=&quot;https://arxiv.org/pdf/1911.02116.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;just pushed out&lt;/a&gt; a bigger version of their multi-lingual language model XLM, called XLM-R. My question is: do these kind of multi-lingual models imply, or even ensure, that their embeddings are comparable between languages? That is, are semantically related words close together in the vector space across languages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the most interesting citation from the paper that is relevant to my question (p. 3):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unlike Lample and Conneau (2019), we do not use language embeddings,&#xA;  which allows our model to better deal with code-switching.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because they do not seem to make a distinction between languages, and there's just one vocabulary for all trained data, I fail to see how this can be truly representative of semantics anymore. The move away from semantics is increased further by the use of BPE, since morphological features (or just plain, statistical &lt;em&gt;word chunks&lt;/em&gt;) of one language might often not be semantically related to the same chunk in another language - this can be true for tokens themselves, but especially so for subword information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in short: how well can the embeddings in multi-lingual language models be used for semantically comparing input (e.g. a word or sentence) of two different languages?&lt;/p&gt;&#xA;" OwnerUserId="29995" LastEditorUserId="29995" LastEditDate="2019-11-08T10:56:05.833" LastActivityDate="2020-03-10T08:55:47.770" Title="Are embeddings in multi-lingual language models comparable across languages?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;language-model&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16355" PostTypeId="5" CreationDate="2019-11-08T14:10:57.510" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-08T14:10:57.510" LastActivityDate="2019-11-08T14:10:57.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16356" PostTypeId="4" CreationDate="2019-11-08T14:10:57.510" Score="0" Body="For questions related to BLEU (BiLingual Evaluation Understudy), which is a metric for evaluating the quality of text which has been machine-translated from one natural language to another. The metric was proposed in the paper &quot;BLEU: a Method for Automatic Evaluation of Machine Translation&quot; (2002) by Kishore Papineni et al. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-08T14:10:57.510" LastActivityDate="2019-11-08T14:10:57.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16357" PostTypeId="1" AcceptedAnswerId="16373" CreationDate="2019-11-08T14:48:00.183" Score="5" ViewCount="242" Body="&lt;p&gt;In my opinion, deep learning algorithms and models (that is, multi-layer neural networks) are more sensitive to overfitting than machine learning algorithms and models (such as the SVM, random forest, perceptron, Markov models, etc.). They are capable of learning more complex patterns. At least that's how I look at it. Still, I and my colleagues disagree about this and I cannot really find any information about this. My colleagues say that deep learning algorithms are hardly vulnerable to overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there statements (or opinions) about this aspect?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-11T01:10:43.643" LastActivityDate="2019-11-11T01:10:43.643" Title="Are deep learning models more prone to overfitting than machine learning ones?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16359" PostTypeId="2" ParentId="16346" CreationDate="2019-11-08T16:40:58.307" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Division_by_zero&quot; rel=&quot;nofollow noreferrer&quot;&gt;Division by zero&lt;/a&gt; is &lt;strong&gt;not&lt;/strong&gt; mathematically defined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A usual or standard way of dealing with this issue is to raise an exception. For example, in Python, the exception &lt;code&gt;ZeroDivisionError&lt;/code&gt; is raised at &lt;em&gt;runtime&lt;/em&gt; if you happen to divide by zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you execute &lt;a href=&quot;https://repl.it/repls/OddHeartyDatalogs&quot; rel=&quot;nofollow noreferrer&quot;&gt;the following program&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zero = 0&#xA;numerator = 10&#xA;numerator / zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;main.py&quot;, line 3, in &amp;lt;module&amp;gt;&#xA;    numerator / zero&#xA;ZeroDivisionError: division by zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if you want to avoid this runtime exception, you can check for division by zero and deal with this issue in a way that is appropriate for your program (without needing to terminate it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://www.aclweb.org/anthology/P02-1040.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLEU: a Method for Automatic Evaluation of Machine Translation&lt;/a&gt; that introduced the BLEU (and brevity penalty) metric, the authors defined the brevity penalty as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;BP =&#xA;\begin{cases}&#xA;1, &amp;amp; \text{if } c &amp;gt; r\\&#xA;e^{(1- r/c)} &amp;amp; \text{if } c \leq r\\&#xA;\end{cases} \label{1} \tag{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition does not explicitly take into account the division by zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Python package &lt;a href=&quot;https://www.nltk.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;nltk&lt;/code&gt;&lt;/a&gt; does not raise an exception, but it (apparently, arbitrarily) returns zero when &lt;code&gt;c == 0&lt;/code&gt;. Note that the BLEU metric ranges from 0 to 1. For example, if you execute the &lt;a href=&quot;https://repl.it/repls/ShortWelltodoActivecontent&quot; rel=&quot;nofollow noreferrer&quot;&gt;following program&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from nltk.translate.bleu_score import brevity_penalty, closest_ref_length&#xA;&#xA;reference1 = list(&quot;hello&quot;) # A human reference translation. &#xA;references = [reference1] # You could have more than one human reference translation.&#xA;&#xA;# references = [] Without a reference, you will get a ValueError.&#xA;&#xA;candidate = list() # The machine translation.&#xA;c = len(candidate)&#xA;&#xA;r =  closest_ref_length(references, c)&#xA;print(&quot;brevity_penalty =&quot;, brevity_penalty(r, c))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;brevity_penalty = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the example above, the only human reference (translation) is &lt;code&gt;reference1 = list(&quot;hello&quot;)&lt;/code&gt; and the only candidate (the machine translation) is an empty list. However, if &lt;code&gt;references = []&lt;/code&gt; (you have no references), then you will get the error &lt;code&gt;ValueError: min() arg is an empty sequence&lt;/code&gt;, where &lt;code&gt;references&lt;/code&gt; are used to look for the closest reference (the closest human translation) to the  &lt;code&gt;candidate&lt;/code&gt; (the machine translation), given that there could be more than one human reference translation, and one needs to be chosen to compute the brevity penalty, with respect to your given candidate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, in the &lt;a href=&quot;https://www.nltk.org/_modules/nltk/translate/bleu_score.html#brevity_penalty&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation of the &lt;code&gt;brevity_penalty&lt;/code&gt; function&lt;/a&gt;, the following comment is written &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# If hypothesis is empty, brevity penalty = 0 should result in BLEU = 0.0.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;hypothesis&lt;/code&gt; is a synonym for &lt;em&gt;candidate&lt;/em&gt; (the machine translation) and the length of the candidate is &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in the formula \ref{1} (and &lt;code&gt;c&lt;/code&gt; in the example above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your second question more directly, I don't think there's a standard way of dealing with the issue, but I've not fully read the BLEU paper yet.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-08T20:00:38.727" LastActivityDate="2019-11-08T20:00:38.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16360" PostTypeId="1" CreationDate="2019-11-08T17:06:24.210" Score="2" ViewCount="87" Body="&lt;p&gt;There is a recent trend in people using LSTMs to write novels. I haven’t attempted this myself. From what I’m hearing, they can tell a story, but it seems they lose the context of the story rather quickly. After which they begin constructing new, but not necessarily related constructs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can they construct a plot in the long term?&lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="2444" LastEditDate="2019-11-08T17:15:58.000" LastActivityDate="2019-12-08T18:01:45.237" Title="Why can't LSTMs tell a long story?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16361" PostTypeId="1" AcceptedAnswerId="16376" CreationDate="2019-11-08T17:13:56.723" Score="2" ViewCount="608" Body="&lt;p&gt;Can some one explain how &lt;a href=&quot;https://github.com/davidsandberg/facenet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facenet&lt;/a&gt; model works in detail and simple words .&lt;/p&gt;&#xA;" OwnerUserId="30306" LastEditorUserId="30306" LastEditDate="2019-11-09T08:25:42.167" LastActivityDate="2019-11-09T11:49:24.393" Title="Detailed explaination of Facenet Model for face recogniton?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16362" PostTypeId="2" ParentId="16360" CreationDate="2019-11-08T17:31:11.503" Score="3" Body="&lt;p&gt;The long-short term memory (LSTM) is a type of recurrent neural network, which is &lt;em&gt;only&lt;/em&gt; suited for sequence modelling, that is, to keep track of statistical dependencies between elements of a sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LSTM prediction capabilities are limited to the training data that is used to train it, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;inductive bias&lt;/a&gt; (in the case of LSTMs, the inductive bias particularly refers to the fact that elements of a sequence are dependent on each other) and the available computation resources. However, storytelling often assumes the existence of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Commonsense_knowledge_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;common-sense knowledge&lt;/a&gt; between the storyteller and the listener (but LSTM completely ignores this) and requires the true understanding of language, which is believed to be an &lt;a href=&quot;https://ai.stackexchange.com/a/13262/2444&quot;&gt;AI-complete problem&lt;/a&gt; (in simple words, it is a very complex task, which probably &lt;a href=&quot;https://arxiv.org/abs/1901.02918&quot; rel=&quot;nofollow noreferrer&quot;&gt;cannot be exactly solved with statistical models&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, even though LSTM &lt;em&gt;partially&lt;/em&gt; addresses the vanishing gradient problem (and they were specifically created to partially solve this issue), they can still suffer from the &lt;a href=&quot;https://stats.stackexchange.com/q/320919/82135&quot;&gt;exploding gradient problem&lt;/a&gt;. Moreover, although LSTMs and, in general, neural networks are universal function approximators, in practice, functions may not be continuous, which is an assumption made in the universal approximation theorems.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-08T17:45:52.827" LastActivityDate="2019-11-08T17:45:52.827" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16364" PostTypeId="1" CreationDate="2019-11-08T17:50:47.923" Score="1" ViewCount="64" Body="&lt;p&gt;Simply speaking, I'm trying to somehow search an audio clip for a list of words, and if found, I mark the time stamps. My use-case is profanity check with a list of pre-defined profane words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any successfull approaches, samples, tools or APIs, possibly through deep learning, to perform this? I'm new to audio processing.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2021-02-07T19:01:05.100" Title="Deep audio fingerprinting for word search" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;voice-recognition&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16365" PostTypeId="1" CreationDate="2019-11-08T18:53:22.597" Score="3" ViewCount="98" Body="&lt;p&gt;I am looking for a book or paper which clearly explains the relationship between &lt;a href=&quot;https://en.wikipedia.org/wiki/Hopfield_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ising models&lt;/a&gt; and deep neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone provide any references?&lt;/p&gt;&#xA;" OwnerUserId="31126" LastEditorUserId="2444" LastEditDate="2021-01-17T19:34:50.570" LastActivityDate="2021-01-17T19:34:50.570" Title="Which books or papers clearly explain the relation between Ising models and deep neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;reference-request&gt;&lt;papers&gt;&lt;books&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16366" PostTypeId="1" CreationDate="2019-11-08T20:05:27.240" Score="1" ViewCount="37" Body="&lt;p&gt;Suppose I have a problem where I want to classify the color of LEDs seen in the image. I can use OpenCV to pinpoint the exact location of these LEDs but I do not know their color for sure because the image can appear too bright or too dim. And the LED itself can have so low resolution that the color appears distorted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I design a simple CNN model that takes in a cropped LED image and classify it as GREEN or BLUE independently it will be likely wrong. The model need to know the overall brightness of the image by looking at how the other LEDs are like in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can say that maybe I just need to compute the overall brightness of the image and feed it to the model, but think of this as a general problem. Suppose I want to do something more complicated that I cannot extract features for the model, how do I write this kind of model in tensorflow/keras ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sDQdV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sDQdV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model architecture is inspired by PointNet architecture where max pooling is used to extract global features from all the points.&#xA;In the image, the top and bottom models are simply the same. They share all the weights, and the number of cropped images doesn't need to be constant.&#xA;The idea is to extract 2 dense layers, where one of them will be used for global max pooling. This global feature represents values like the &quot;overall brightness&quot; of the image.&#xA;So theoretically, the model should use this info to classify the color better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, how do I implement this kind of thing in a keras/tensorflow? Any paper/model/link suggestions?&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2019-11-08T20:05:27.240" Title="How to implement CNN with variable number of images in tensorflow or keras?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16367" PostTypeId="1" CreationDate="2019-11-09T00:42:47.183" Score="2" ViewCount="84" Body="&lt;p&gt;A recent question on AI and acting recalled me to the idea that in drama, there are not only conflicting motives between agents (characters), but a character may themselves have objectives that are in conflict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result of this in performance is typically nuance, but also carries the benefit of combinatorial expansion, which supports greater novelty, and it occurs to me that this would be a factor in &lt;a href=&quot;https://en.wikipedia.org/wiki/Affective_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;affective computing&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The actress Eva Green is a good example, where her performances typically involve indicating two or more conflicting emotions at once.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It occurs to me that this can even arise in the context of a formal game where achieving the most optimal outcome requires managing competing concerns.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there literature or examples of AI with internal conflicting objectives?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-11-10T01:45:09.517" Title="AI with conflicting objectives?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;game-theory&gt;&lt;goal-based&gt;&lt;affective-computing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16368" PostTypeId="2" ParentId="16367" CreationDate="2019-11-09T02:02:52.607" Score="2" Body="&lt;p&gt;There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-objective_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-objective optimization&lt;/a&gt; problems, where the objective functions may be in conflict with each other, which can potentially have multiple &lt;a href=&quot;https://en.wikipedia.org/wiki/Pareto_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pareto-optimal&lt;/a&gt; solutions. The paper &lt;a href=&quot;http://www.eng.auburn.edu/~aesmith/files/Multi-objective%20optimization%20using%20genetic%20algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-objective optimization using genetic algorithms: A tutorial&lt;/a&gt; (2006) gives a good overview of the multi-objective optimization problem with genetic algorithms, which can be called &lt;a href=&quot;https://www.egr.msu.edu/~kdeb/papers/k2012009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolutionary multi-objective optimization (EMO)&lt;/a&gt; or &lt;a href=&quot;https://pdfs.semanticscholar.org/dc4e/c99d4201affa93f404c6b4e4f8538d07aeb5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-objective optimization evolutionary algorithms&lt;/a&gt; (MOEAs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common multi-objective genetic algorithm is &lt;a href=&quot;https://web.njit.edu/~horacio/Math451H/download/SrinivasDeb_GA.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NSGA&lt;/a&gt; (or &lt;a href=&quot;https://pdfs.semanticscholar.org/dc4e/c99d4201affa93f404c6b4e4f8538d07aeb5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NSGA-2&lt;/a&gt; and &lt;a href=&quot;https://www.egr.msu.edu/~kdeb/papers/k2012009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NSGA-3&lt;/a&gt;), which stands for Non-dominated Sorting Genetic Algorithm, which is based on the concepts of non-dominated sorting, &lt;a href=&quot;https://en.wikipedia.org/wiki/Pareto_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pareto front and optimality&lt;/a&gt;, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.30.8270&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;niches&lt;/a&gt; (sub-populations), and elitism (the best individuals of the current population are carried over to the next generation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to play with MOEAs, you may wanna try the Python &lt;a href=&quot;https://deap.readthedocs.io/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;deap&lt;/code&gt;&lt;/a&gt; package, which supports, for example, the &lt;a href=&quot;https://github.com/DEAP/deap/blob/master/examples/ga/nsga2.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;NSGA-2&lt;/a&gt; algorithm.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-09T02:13:16.137" LastActivityDate="2019-11-09T02:13:16.137" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16369" PostTypeId="1" CreationDate="2019-11-09T05:11:54.380" Score="2" ViewCount="74" Body="&lt;p&gt;I would like to solve the Sokoban puzzle, which consists in moving a character in a 2D map to push boulders into target cells. Each turn, the player can move to an adjacent cell (no diagonals) if it is empty, or push a boulder one step further. To push a boulder, the player needs to stand next to it, and the space behind the boulder needs to be empty (no other boulders and no walls).&lt;/p&gt;&#xA;&lt;p&gt;I'm using the STRIPS planner, and I am having a hard time defining the fixed and dynamic relations and also the preconditions and effects of each operator for this puzzle.&lt;/p&gt;&#xA;" OwnerUserId="31131" LastEditorUserId="2444" LastEditDate="2021-02-07T17:39:37.700" LastActivityDate="2021-02-07T17:39:37.700" Title="How can I define the relations, preconditions and effects of each operator for the Sokoban puzzle?" Tags="&lt;planning&gt;&lt;strips&gt;&lt;sokoban-puzzle&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16370" PostTypeId="2" ParentId="16365" CreationDate="2019-11-09T05:15:31.807" Score="4" Body="&lt;p&gt;The following articles&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/q-bio/0611072&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ising models for networks of real neurons&lt;/a&gt; (2006) by Gasper Tkacik et al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://journals.aps.org/pre/abstract/10.1103/PhysRevE.97.032119&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models&lt;/a&gt; (2018) by Kyle Mills et al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1706.08466&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inverse Ising inference by combining Ornstein-Zernike theory with deep learning&lt;/a&gt; (2017) by Soma Turi, Alpha A. Lee et al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1906.10935&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural-network based general method for statistical mechanics on sparse systems&lt;/a&gt; (2019) by Feng Pan, Hai-Jun Zhou, Pan Zhang et al.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;may help you understand the &quot;relationship&quot; between Ising models and DNN, assuming you know what the Ising model is and what a DNN is, the similarity should be fairly intuitive to you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;Ising model&lt;/strong&gt; is a sort of floating soup of ferromagnetic particles each generating their own small magnetic field either working against or with their neighbor. When many of the particles aline, they create an aligned field we refer to as a dipole moment in magnetism, while in a &lt;strong&gt;DNN&lt;/strong&gt; we refer to the joined effort of a few entities working to cause a larger effect in another entity an 'activation function'. In a fully connected DNN, where the Euclidean distance weights the connections and the nodes are initialized with a certain magnetic polarity in relation to the axis of the magnetic field it generates, the network would be an almost exact representation of the reality of what the Ising model seeks to simplify. &lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="30365" LastEditDate="2019-11-13T17:52:39.857" LastActivityDate="2019-11-13T17:52:39.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16371" PostTypeId="2" ParentId="16367" CreationDate="2019-11-09T05:31:04.927" Score="2" Body="&lt;p&gt;MOEAs sounds very cool, but I feel that you can't really talk about conflict in AI without discussing &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative adversarial networks (GANs)&lt;/a&gt;, which have been shown to have amazing performance by training a model to say detect in-between pictures of cats and dogs and an adversarial network being trained to create pictures to attempt to trick the training network as much as possible. The completely conflicting objectives of the networks enable both to be trained very well so the models, in the end, are much more robust and able to handle sometimes bizarrely generated edge cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found this paper &lt;a href=&quot;https://arxiv.org/abs/1910.04966&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Multi-Objective Optimization Driven by Generative Adversarial Networks (GANs)&lt;/a&gt; (2019), which combines MOEAs and GANs, but there are potentially more related papers.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="2444" LastEditDate="2019-11-10T01:45:09.517" LastActivityDate="2019-11-10T01:45:09.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16373" PostTypeId="2" ParentId="16357" CreationDate="2019-11-09T06:26:11.313" Score="5" Body="&lt;p&gt;Your reasoning isn't wrong. Deep Neural Networks (DNNs) have a much larger capacity than simpler ML algorithms (excluding NNs) and can easily memorize even a very complex dataset and overfit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;DNNs, however, are so effective because they usually are applied on tasks that are &lt;strong&gt;harder&lt;/strong&gt;, so it's not as easy to overfit. For example an image classifier might be trained on a dataset with millions of images; a task much harder to overfit on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cases where this isn't possible (e.g. an image classification task with a couple thousand images), &lt;strong&gt;transfer learning&lt;/strong&gt; is used. You can initialize your weights from a model pre-trained on a large dataset, use its already-trained feature extraction layers and simply fine tune the last layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Data augmentation&lt;/strong&gt; also helps a lot here, which effectively increases size of the training set and discourages the DNN from memorizing the samples. It is so effective that it is used even in large datasets, where it is harder to overfit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, DNNs employ several methods to prevent them from overfitting. The most prominent of these is &lt;strong&gt;dropout&lt;/strong&gt;, which is a very effective regularizer. &lt;strong&gt;Batch Normalization&lt;/strong&gt; has also proven an effective regularizer. &lt;strong&gt;SGD&lt;/strong&gt; allows you to explore more parameters than GD, which also is effective against overfitting. Finally, &lt;strong&gt;early stopping&lt;/strong&gt; and &lt;strong&gt;parameter norm penalties&lt;/strong&gt; aren't uncommon in DNNs.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-11-09T06:26:11.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16374" PostTypeId="1" CreationDate="2019-11-09T09:14:23.937" Score="6" ViewCount="158" Body="&lt;p&gt;In perfect information games, the agent can see all the moves performed in the past. Besides, it can observe the next action that will be put into practice by the opponent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, can we say that perfect information games are actually a fully observable environment? If we reach this conclusion, I guess that imperfect information becomes a partially observable environment?&lt;/p&gt;&#xA;" OwnerUserId="31133" LastEditorUserId="2444" LastEditDate="2021-01-20T23:03:15.813" LastActivityDate="2021-01-20T23:03:15.813" Title="Are perfect and imperfect information games modelled as fully and partially observable environments, respectively?" Tags="&lt;markov-decision-process&gt;&lt;game-theory&gt;&lt;pomdp&gt;&lt;imperfect-information&gt;&lt;perfect-information&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16375" PostTypeId="1" CreationDate="2019-11-09T11:18:09.290" Score="4" ViewCount="2640" Body="&lt;p&gt;I am solving a problem in which, according to the given values, the heuristic is not admissible. According to my calculation from other similar problems, it should be consistent, as well as keeping in mind the values, but the solution says it's not consistent either. Can someone tell why? &lt;/p&gt;&#xA;" OwnerUserId="31139" LastEditorUserId="2444" LastEditDate="2021-05-07T12:05:07.310" LastActivityDate="2021-05-07T12:05:07.310" Title="If an heuristic is not admissible, can it be consistent?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;admissible-heuristic&gt;&lt;consistent-heuristic&gt;&lt;heuristic-functions&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16376" PostTypeId="2" ParentId="16361" CreationDate="2019-11-09T11:49:24.393" Score="2" Body="&lt;p&gt;Facenet is a Siamese network. It's basic architecture is this:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/BkWbf.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;The input(a face) is fed through a deep convolutional neural network and also a fully connected layer at the end. The fully connected layer at the end output an embedding of the input image which is a predefined size. The embedding can contain feature that human understand or maybe not. The embedding represent the input image, just in a &quot;compressed&quot; form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further explain that, let me give an example. Let's say that you have to describe a face. What's will you say? You will probably say something like the face is round, the eyes are blue, it is a female face and more. The neural network is doing what you are doing, describing the face, but using numbers instead of words. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do a face recognition task, the network take an pre taken image of the list of people to recognize and the unknown new data from the people to be recognised. It then feed both images into the network and get the embeddings. The network then calculate the distance of the two embeddings, using some metric such as squared error or absolute error. In the image it uses the squared error. If the error is below a certain threshold, the face is recognised. If not, it then loops through the other pre taken images in the set of faces of the system and do the task again. The system stores embedding of the pre taken images before hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For training the FaceNet, triplet loss is used. Triplet loss has been explained in another of your post, by me. &lt;a href=&quot;https://ai.stackexchange.com/questions/16311/what-is-the-formula-used-to-calculate-the-accuracy-in-the-facenet-model&quot;&gt;What is the formula used to calculate the accuracy in the FaceNet model?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, the model is trained using the triplet loss as it can train the network to output a similar embedding for the same person and a very different embedded for a different person. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes, a binary classification end is also used. It removed the need of the triplet loss and outputs a number from 0 to 1 for similarity instead. This removes the triplet loss part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope my answer can help you and have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-09T11:49:24.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16378" PostTypeId="2" ParentId="16375" CreationDate="2019-11-09T17:16:19.317" Score="2" Body="&lt;p&gt;For a heuristic to be admissible, it must never overestimate the distance from a state to the nearest goal state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a heuristic to be consistent, the heuristic's value must be less than or equal to the cost of moving from that state to the state nearest the goal that can be reached from it, plus the heurstic's estimate for that state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this means is that, as you move along the sequence of nodes from start to goal that the heuristic recommends, a consistent heuristic should monotonically decrease in value. A consistent heuristic is thus also always admissible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that this means that if a heuristic is &lt;em&gt;not admissible&lt;/em&gt; (like yours), it is also &lt;em&gt;not consistent&lt;/em&gt; (by the contrapositive). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if you already know your heuristic is not admissible, you should not be surprised that it is not consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems most likely that you may have confused the definition of consistent for monotone. A consistent heuristic is &lt;em&gt;both&lt;/em&gt; monotone and admissible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Neil Says, if you want to know why your specific heuristic is inadmissible, you should post another question about it, or modify this one.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-09T17:16:19.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16380" PostTypeId="1" CreationDate="2019-11-09T18:51:26.307" Score="2" ViewCount="30" Body="&lt;p&gt;I am currently trying to implement an &lt;em&gt;end-to-end speech recognition&lt;/em&gt; system from scratch, that is, without using any of the existing frameworks (like TensorFlow, Keras, etc.). I am building my own library, where I am trying to do a polynomial approximation of functions (like exponential, log, sigmoid, ReLU, etc). I would like to have access to a nice description of the neural networks involved in an end-to-end speech recognition system, where the architecture (the layers, activation functions, etc.) is clearly laid out, so that I can implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find most of the academic or industry papers citing various previous works, toolkits or papers, making it tedious for me. I am new to the field, so I am having more difficulty, so looking for some help here. &lt;/p&gt;&#xA;" OwnerUserId="31145" LastEditorUserId="2444" LastEditDate="2019-11-11T16:27:33.253" LastActivityDate="2019-11-11T16:27:33.253" Title="Is there a detailed description or implementation of an end-to-end speech recognition system?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;research&gt;&lt;reference-request&gt;&lt;speech-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16381" PostTypeId="1" CreationDate="2019-11-09T21:30:07.420" Score="4" ViewCount="292" Body="&lt;p&gt;I was just doing a simple NN example with the fashion MNIST dataset, where I was getting 97% accuracy, when I noticed that I was using Binary cross-entropy instead of categorical cross-entropy by accident. When I switched to categorical cross-entropy, the accuracy dropped to 90%. I then got curious and tried to use binary cross-entropy instead of categorical cross-entropy in my other projects and in all of them the accuracy increased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I know that binary cross-entropy can be used in a multi-class, multi-label classification problem, but why is working better than categorical cross-entropy in a multiclass single label problem?&lt;/p&gt;&#xA;" OwnerUserId="31147" LastEditorUserId="2444" LastEditDate="2019-11-10T01:49:59.750" LastActivityDate="2019-11-10T01:49:59.750" Title="Why does the binary cross-entropy work better than categorical cross-entropy in a multi-class single label problem?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;loss-functions&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16382" PostTypeId="2" ParentId="16381" CreationDate="2019-11-09T22:35:38.370" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/260505/machine-learning-should-i-use-a-categorical-cross-entropy-or-binary-cross-entro&quot;&gt;https://stats.stackexchange.com/questions/260505/machine-learning-should-i-use-a-categorical-cross-entropy-or-binary-cross-entro&lt;/a&gt;&#xA;Is relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;based on my reading when you have a NN and do Binary crossentropy on what you might call 'Linked category data' the accuracy can tend to be better than in a Categorical crossentropy model. The binary aspect implies the categories can undergo multiple splits before deciding on the exact category when the data is categorically splittable like this in a tree like hierarchy the accuracy can tend to be better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of how difficult it would be to memorize the name of every type of clothing in someones wardrobe if each piece had its own special name. Vs if they had structurally relevant names like Upper/lower for category number one is it warn on your top half or bottom half. followed by inner or outer. It an inner or outer layer of clothing. learning such binary name/feature categories enables a more accurate model. If it was data unrelated in this way it would most likely not be as accurate. The binary model can take advantage learning such features while a muti categorical model I think assumes independence and tries to learn best the features of each group and gives out a prediction of how sure it falls in each category. &lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-09T22:35:38.370" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16383" PostTypeId="1" CreationDate="2019-11-09T22:39:58.420" Score="4" ViewCount="46" Body="&lt;p&gt;I am working on a problem in which I am attempting to find a stable region in a spiral galaxy. The PI I'm working with asked me to use machine learning as a tool to solve the problem. I have created some visualizations of my data, as bellow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P6FVS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P6FVS.png&quot; alt=&quot;data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this image, you can see there is a flat region between 0 and roughly 30 pixels, and between 90 pixels and 110 pixels. I have received suggestions to use an RNN LSTM model that can identify flat regions, but I wanted to hear other suggestions of other neural network models as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PI I'm working with suggests to feed my data visualization images into a neural network and have the neural network identify said stable regions. Can this be done using a neural network, and what resources would I have to look at? Moreover, can this problem be solved with RNN LSTM? I think the premise of this was to treat the radius as some temporal dimension. I've been extensively looking for answers online, and I cannot quite seem to find any similar examples.&lt;/p&gt;&#xA;" OwnerUserId="31148" LastEditorUserId="30365" LastEditDate="2019-11-11T00:14:11.900" LastActivityDate="2019-12-11T02:00:33.393" Title="Using a neural network to identify a stable region within a set of data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;python&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16384" PostTypeId="1" AcceptedAnswerId="16387" CreationDate="2019-11-09T23:15:28.767" Score="3" ViewCount="672" Body="&lt;p&gt;My vague understanding of reinforcement learning (RL) is that it's very similar to supervised learning except that it updates on a continuous feed of data/activity, this to me sounds very similar to AutoML (which I've started to notice being used). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do they use different algorithms? What is the fundamental difference between RL and AutoML?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm after an explanation for somebody who understands technology but does not work with machine learning tools regularly. &lt;/p&gt;&#xA;" OwnerUserId="19484" LastEditorUserId="2444" LastEditDate="2019-11-10T00:45:02.150" LastActivityDate="2019-11-11T15:51:46.183" Title="What is the difference between reinforcement learning and AutoML?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;supervised-learning&gt;&lt;automated-machine-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16385" PostTypeId="5" CreationDate="2019-11-10T00:46:14.920" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T00:46:14.920" LastActivityDate="2019-11-10T00:46:14.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16386" PostTypeId="4" CreationDate="2019-11-10T00:46:14.920" Score="0" Body="For questions related to automated machine learning (AutoML), which refers to a collection of techniques to automate the design and the application of machine learning algorithms and models." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T00:46:14.920" LastActivityDate="2019-11-10T00:46:14.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16387" PostTypeId="2" ParentId="16384" CreationDate="2019-11-10T01:29:26.383" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1908.05557.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automated machine learning&lt;/a&gt; (&lt;a href=&quot;https://arxiv.org/pdf/1810.13306.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AutoML&lt;/a&gt;) is an &lt;a href=&quot;https://en.wiktionary.org/wiki/umbrella_term&quot; rel=&quot;nofollow noreferrer&quot;&gt;umbrella term&lt;/a&gt; that encompasses a collection of techniques (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyper-parameter optimization&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;automated&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;feature engineering&lt;/a&gt;) to &lt;strong&gt;automate&lt;/strong&gt; the design and application of machine learning algorithms and models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning (RL)&lt;/a&gt; is a sub-field of machine learning concerned with the task of making decisions and taking actions in an environment so as to maximize (long-term) &lt;strong&gt;reward&lt;/strong&gt; (which is the goal of the so-called RL agent). RL is (at least partially) based on the way animals (including humans) learn. For example, the usual way of &lt;a href=&quot;https://en.wikipedia.org/wiki/Dog_training&quot; rel=&quot;nofollow noreferrer&quot;&gt;training a dog&lt;/a&gt; to perform a certain task is to reward it with food whenever it takes the correct action (for example, jumping, if you want the dog to jump whenever you make a certain gesture with your hand). In this case, the RL agent is the dog, the task the dog needs to perform (e.g. jumping) is the environment, food is the reward and the goal is to get food.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that reinforcement learning (RL) is a sub-field of machine learning, then, in principle, AutoML can also be used to automate the design of RL algorithms, models or agents. For example, if you use a neural network to represent the policy (the function the determines which action to take in the environment), then you can potentially use AutoML to find the most appropriate architecture (for example, the most appropriate number of layers) for this neural network.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T15:07:21.010" LastActivityDate="2019-11-10T15:07:21.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16388" PostTypeId="5" CreationDate="2019-11-10T01:53:42.387" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T01:53:42.387" LastActivityDate="2019-11-10T01:53:42.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16389" PostTypeId="4" CreationDate="2019-11-10T01:53:42.387" Score="0" Body="For questions related to the concept of cross-entropy in the context of artificial intelligence. For example, when the cross-entropy is used as a loss function to train a neural network." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T01:53:42.387" LastActivityDate="2019-11-10T01:53:42.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16390" PostTypeId="2" ParentId="16383" CreationDate="2019-11-10T02:27:50.810" Score="0" Body="&lt;p&gt;In image processing CNNs are usually used to create weighted filters for focusing in on the image features which are most important for making predictions. Keras is one of the libraries used to examine images in this way. With this type of analysis you will need labeled and unlabeled data you want to create a network that inputs a photo extracts the flat black line regions and outputs those. The model will be generative, generating guesses of regions where the function is flat. This is all possible to do but in order to label the data you need to label them by hand or you need to create a function that manually labels them which would not be very difficult. The input nodes will take in the pixels of the picture and the output layer will be guesses at location along the graph of wether the section is flat or not. It seems overkill to do this with a neural network when it is possible to not use a NN and creating a labeling method will most likely be your first step. If you have any questions please ask.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-10T02:27:50.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16391" PostTypeId="1" AcceptedAnswerId="16392" CreationDate="2019-11-10T02:54:14.940" Score="3" ViewCount="150" Body="&lt;p&gt;In a feed-forward neural network, in order to efficiently do backpropagation, what kind of data structure is needed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the weights can just be stored in an array, and you need pointers of some kind to represent connections from one layer to the next (or just a default scheme/pattern), but is anything else needed for backpropagation to work?&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="2444" LastEditDate="2019-11-10T13:31:40.870" LastActivityDate="2019-11-10T16:53:33.937" Title="What kind of data structures are needed to efficiently do back-propagation in a feedforward neural network?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16392" PostTypeId="2" ParentId="16391" CreationDate="2019-11-10T04:51:56.317" Score="1" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: You'll need to store a little bit more to perform backward passes. You'll need to store data from the forward pass. This stored information is used for calculating the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt; (warning: not trivial)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I know the weights can just be stored in an array&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You'll need a little more:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To update the weights you need to keep a &quot;cache&quot; of the forward pass intermediate terms. That is, forward propagation can be seen as a series of transformations on your input &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X\xrightarrow{\Theta^{[1]}+b^{[1]}} [ Z^{[1]} \xrightarrow{\alpha^{[1]}} A^{[1]}] \xrightarrow{\Theta^{[2]}+b^{[2]}}&#xA;\dots &#xA;\xrightarrow{\Theta^{[L]}+b^{[L]}} [ Z^{[L]} \xrightarrow{\alpha^{[L]}} A^{[L]}]\xrightarrow{\frac{1}{m}\sum\limits_m\sum\limits_{n_L} loss\{A^{[L]},y\}} J&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Z^{[1]}=\Theta^{[1]}X+b^{[1]}$&lt;/span&gt; (ie the linear part)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A^{[l]}=\alpha^{[l]}(Z^{[l]})$&lt;/span&gt; (ie element wise activation over linear part)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to store the &lt;span class=&quot;math-container&quot;&gt;$Z^{[l]}$&lt;/span&gt; &amp;amp; &lt;span class=&quot;math-container&quot;&gt;$A^{[l]}$&lt;/span&gt; terms in said &quot;cache.&quot; You could store these in an array or some other similar data structure. You need these for calculating the gradient during the backwards pass. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Syntax&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A^{[k]}$&lt;/span&gt; - this means we are indexing by layer (eg &lt;span class=&quot;math-container&quot;&gt;$\alpha^{[k]}$&lt;/span&gt; is the activation for k-th layer)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; - is the number of examples in the batch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$n_k$&lt;/span&gt; - denotes the number of neurons in the k-th layer&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; - the number of layers (so &lt;span class=&quot;math-container&quot;&gt;$n_L$&lt;/span&gt; is the number of neurons in last layer)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt; - The set of &lt;em&gt;all&lt;/em&gt; weights (notice no superscript)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Backprop&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of neural networks the cost is a scalar function of inputs and parameters. To get backprop started calculate the &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_calculus#Scalar-by-matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;scalar by matrix&lt;/a&gt; derivative of the cost with respect to the activations of the last layer call this matrix &lt;span class=&quot;math-container&quot;&gt;$dA^{[L]}$&lt;/span&gt;. Observe:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$dA^{[L]} = \frac{\partial J(\Theta,X)}{\partial A^{[L]}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, we calculate scalar-by-matrix derivative of &lt;span class=&quot;math-container&quot;&gt;$Z^{[L]}$&lt;/span&gt;. Doing this one realizes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$dZ^{[L]} = \frac{\partial J(\Theta,X)}{\partial Z^{[L]}} = dA\odot\alpha'^{[L]}(Z^{[L]})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; denotes element wise (&lt;a href=&quot;https://en.wikipedia.org/wiki/Hadamard_product_(matrices)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hadamard&lt;/a&gt;) product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the above one can make use of the matrix definitions for back propagation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(A)}\quad d\Theta^{[l]} = \frac{1}{m}dZ^{[l]}\times (A^{[l-1]})^T$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(B)}\quad db^{[l]} = \frac{1}{m}\sum_{c=1}^m dZ^{[l](c)}$&lt;/span&gt; (where the new superscript in &lt;span class=&quot;math-container&quot;&gt;$dZ^{[l](c)}$&lt;/span&gt; denotes summing along the &lt;em&gt;batch&lt;/em&gt; dimension )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(C)} \quad dZ^{[l]}= dA^{[l]}\odot \alpha^{'[l]}(Z^{[l]})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{(D)}\quad dA^{[k]} = (\Theta^{[k+1]})^T\times dZ^{[k+1]}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And of course the wight updates are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Theta^{[L]} \leftarrow \Theta^{[L]} - \frac{\eta}{m}d\Theta $&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b^{[L]} \leftarrow b^{[L]} - \frac{\eta}{m}db $&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(where &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt; is the learning rate)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observe, how the forward pass terms are used during the backprop calculations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A recommendation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the A. Ng deep learning specialization. He does a good job explaining the intuition and even has a project to implement this. Though, he does not derive the back propagation equations. You can find a not so easy derivation &lt;a href=&quot;https://rojasinate.com/documents/backprop.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-11-10T16:53:33.937" LastActivityDate="2019-11-10T16:53:33.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16393" PostTypeId="2" ParentId="16374" CreationDate="2019-11-10T06:26:48.463" Score="1" Body="&lt;p&gt;Not exactly, at least traditionally: in Game Theory, &quot;imperfect information&quot; is most often defined as agents having only partial information about the history of agents' actions, as you correctly noted. But also note that this doesn't refer to the general world facts or state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But &quot;partial observability&quot; is typically used in terms of &lt;em&gt;systems&lt;/em&gt;, e.g. in Markov Decision Processes, where it explicitly refers to world state, which might or might not include the history of other actors' actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But of course in the end it depends which exact definitions are used in the context you're looking at - every author is free to define their own concepts, using traditional names or new ones.&lt;/p&gt;&#xA;" OwnerUserId="3554" LastActivityDate="2019-11-10T06:26:48.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16394" PostTypeId="1" CreationDate="2019-11-10T09:50:50.433" Score="11" ViewCount="5478" Body="&lt;p&gt;A friend of mine, who is an International Master at chess, told me that humans were superior to machines provided you didn't impose the time constraints that exist in competitive chess (40 moves in 2 hours) since very often games were lost, to another human or a machine, when a bad move is made under time pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, with no time constraints and access to a library of games, the human mind remains superior to the machine is my friend's contention.  I'm an indifferent chess player and don't really know what to make of this.  I was wondering if any research had been made that could back up that claim or rebut it.&lt;/p&gt;&#xA;" OwnerUserId="31158" LastEditorUserId="2444" LastEditDate="2019-11-11T01:07:57.617" LastActivityDate="2021-01-22T01:25:46.247" Title="Are humans superior to machines in chess?" Tags="&lt;comparison&gt;&lt;chess&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16395" PostTypeId="1" AcceptedAnswerId="16397" CreationDate="2019-11-10T10:26:06.893" Score="1" ViewCount="32" Body="&lt;p&gt;I am trying to implement a tabular-based GLIE Monte-Carlo learning algorithm. &#xA;So I repeat n times: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;create observations using my previous policy &lt;span class=&quot;math-container&quot;&gt;$\pi_{n-1}(s)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;update my state-action values using the observations generated in 1 with the monte-carlo update rule: &lt;span class=&quot;math-container&quot;&gt;$Q_n(s_t,a_t)= Q_n(s_t,a_t)+1/N(s_t,a_t)\times(G_t-Q_n(S_t,a_t))$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;update my policy to &lt;span class=&quot;math-container&quot;&gt;$\pi_{n}$&lt;/span&gt; using epsilon-geedy improvement with &lt;span class=&quot;math-container&quot;&gt;$\epsilon=1/(n+1)$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In step 2 I need to decide for an initial estimate &lt;span class=&quot;math-container&quot;&gt;$\tilde{Q}_n$&lt;/span&gt;. Is it a decent option to use &lt;span class=&quot;math-container&quot;&gt;$\tilde{Q}_n=Q_{n-1}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="30369" LastActivityDate="2019-11-10T11:08:31.653" Title="Can I use my previous estimate of the state-action values as initialisation in GLIE-Monte Carlo Control?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16396" PostTypeId="2" ParentId="16394" CreationDate="2019-11-10T10:42:40.010" Score="21" Body="&lt;p&gt;Losing games to computers because of mistakes made under time pressure was probably a thing about 20 years ago, when Kasparov lost to &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepBlue&lt;/a&gt; after such a mistake(correction: it was Kramnik with the blunder, not Kasparov (see edit 2)). But after &lt;a href=&quot;https://en.chessbase.com/post/man-vs-machine-shocker-kramnik-allows-mate-in-one&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kramnik's&lt;/a&gt; loss in early 2000s, no world champion ever tried to play against a computer (to my knowledge). Nowadays, there are computer only &lt;a href=&quot;https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship&quot; rel=&quot;nofollow noreferrer&quot;&gt;tournaments&lt;/a&gt; among programs with ratings well above 3300 (for comparison, Carlsen's peak rating was around 2880), and it is not uncommon for computers to make moves with no apparent meaning to humans.&lt;/p&gt;&#xA;&lt;p&gt;No time limit for humans also mean no time limit for computers so I doubt any human can win a single game against a computer. Older models like Stockfish 8 depend on their computational power as it can look at several millions of position per second, &lt;a href=&quot;https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google AlphaZero&lt;/a&gt;  managed to beat &lt;a href=&quot;https://www.chess.com/news/view/updated-alphazero-crushes-stockfish-in-new-1-000-game-match&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stockfish&lt;/a&gt; with 80000 positions per second: they don't seem to depend on brute force calculations any more. Keep in mind that this is without any prior knowledge of openings etc, they are trained using reinforcement learning, starting from the rules of the game only. From there, they can develop their own strategies and implement them without making any mistakes. They create their own openings from scratch, so existing libraries is not going to be very useful.&lt;/p&gt;&#xA;&lt;p&gt;I am not aware of any research on this but lack of a challenge from humans is probably enough evidence. Also, Grand Masters regularly use chess engines in their training routine to analyze positions, so there is that.&lt;/p&gt;&#xA;&lt;p&gt;A few years ago there was a game between Stockfish against GM Nakamura + Rybka, which &lt;a href=&quot;https://www.chess.com/news/view/stockfish-outlasts-nakamura-3634&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stockfish won&lt;/a&gt;. It is possible that human GM + Stockfish might have better chances against AlphaZero in correspondence without any time limits, but we probably will never know.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://youtube.com/watch?v=n5ltO5jfniM&amp;amp;t=86&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is an interview with Carlsen after a game, very interesting to show what he thinks about AlphaZero.&lt;/p&gt;&#xA;&lt;p&gt;Both Kramnik and Kasparov made serious mistakes in their matches against computers. Kasparov resigned in a drawn position and missed a knight sacrifice, and Kramnik missed a mate in 1.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="2444" LastEditDate="2021-01-22T01:25:46.247" LastActivityDate="2021-01-22T01:25:46.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16397" PostTypeId="2" ParentId="16395" CreationDate="2019-11-10T11:08:31.653" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In step 2 I need to decide for an initial estimate &lt;span class=&quot;math-container&quot;&gt;$\tilde{Q}_n$&lt;/span&gt;. Is it a decent option to use &lt;span class=&quot;math-container&quot;&gt;$\tilde{Q}_n=Q_{n-1}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, this is a common choice. It's actually common to update the table for &lt;span class=&quot;math-container&quot;&gt;$\tilde{Q}$&lt;/span&gt; in place, without any separate initialisation per step. The separate phases of estimation and policy improvement are easier to analyse for theoretical correctness, but in practice updates made in place can be faster because new information is used as soon as it is available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on how the policy was changed, and how accurate the previous estimate was, this could place the estimates closer convergence for the next step. Often the previous estimates will be closer to the new targets than any fixed or random initialisation scheme you could set up.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-11-10T11:08:31.653" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16398" PostTypeId="2" ParentId="16374" CreationDate="2019-11-10T14:29:51.383" Score="3" Body="&lt;p&gt;There is indeed a close parallel here, but the concepts &lt;em&gt;are&lt;/em&gt; distinct. Every perfect information game is fully observable, but not every fully observable game is a game of perfect information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A game of &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;imperfect information&lt;/a&gt; is one in which you lack knowledge of &lt;em&gt;any&lt;/em&gt; of the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The state of the game (e.g. current market prices).&lt;/li&gt;&#xA;&lt;li&gt;The rewards you will receive from various states (i.e. utility and cost functions).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In contrast, in partially observable process (specifically, a &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;POMDP&lt;/a&gt;), the requirement is that you must not know &lt;em&gt;which&lt;/em&gt; state you are in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a subtle distinction, so here are some examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A multi-armed bandit game with stationary distributions. Here, you know which state you are in (in fact, if the distributions are stationary, you know that the state doesn't change, except for the value of your winnings). You are not in a POMDP (the game is fully observable), but you &lt;em&gt;are&lt;/em&gt; operating with imperfect information, because you don't know the utility function associated with different actions. You are operating in a regular MDP.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The game of chess has perfect information, and is also thus fully observable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;The game of poker has imperfect information because you cannot observe the current state of the game (you can't see the cards in your opponent's hand). It is thus a POMDP.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-10T14:29:51.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16399" PostTypeId="5" CreationDate="2019-11-10T15:42:51.213" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T15:42:51.213" LastActivityDate="2019-11-10T15:42:51.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16400" PostTypeId="4" CreationDate="2019-11-10T15:42:51.213" Score="0" Body="For questions related to speech recognition, also known as automatic speech recognition (ASR), computer speech recognition or speech to text (STT), which is a sub-field of computational linguistics that enables the recognition and translation of spoken language into text by computers." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T15:42:51.213" LastActivityDate="2019-11-10T15:42:51.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16401" PostTypeId="2" ParentId="16375" CreationDate="2019-11-10T16:06:38.123" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If a heuristic is not admissible, can it be consistent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. Consistency implies admissibility. In other words, if a heuristic is &lt;a href=&quot;https://en.wikipedia.org/wiki/Consistent_heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;consistent&lt;/a&gt;, it is also &lt;a href=&quot;https://en.wikipedia.org/wiki/Admissible_heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;admissible&lt;/a&gt;. However, admissibility does &lt;strong&gt;not&lt;/strong&gt; imply consistency. In other words, an admissible heuristic is not necessarily consistent.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Definitions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Given a graph &lt;span class=&quot;math-container&quot;&gt;$G=(V, E)$&lt;/span&gt; representing the search space, where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; are respectively the set of vertices and edges, and the function &lt;span class=&quot;math-container&quot;&gt;$w: E \times E \rightarrow \mathbb{R}$&lt;/span&gt; that defines the weight (or cost) of each edge of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, an admissible heuristic &lt;span class=&quot;math-container&quot;&gt;$h_{\text{a}}$&lt;/span&gt; is defined as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h_{\text{a}}(n) \leq h^*(n), \forall n \in V$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h^*(n)$&lt;/span&gt; is the optimal cost to reach a goal from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (so &lt;span class=&quot;math-container&quot;&gt;$h^*(n)$&lt;/span&gt; is the optimal heuristic). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, a consistent heuristic &lt;span class=&quot;math-container&quot;&gt;$h_{\text{c}}$&lt;/span&gt; is defined as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;h_{\text{c}}(n) &amp;amp;\leq w(n, s) + h_{\text{c}}(s), \forall n \in V \setminus \mathcal{G}, \text{ and} \\&#xA;h_{\text{c}}(g) &amp;amp;= 0, \forall g \in \mathcal{G},&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is a successor of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is any goal node and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{G}$&lt;/span&gt; is the set of goal nodes of the graph &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Theorem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A consistent heuristic is an admissible heuristic.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Proof&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; be a consistent heuristic. Given that &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is consistent, then &lt;span class=&quot;math-container&quot;&gt;$h(g) = 0$&lt;/span&gt;, for any goal node &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, so it does not overestimate the cost of reaching the goal at any of the goal nodes (given that, if you already are at a goal node, the cost is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$h(g) = 0$&lt;/span&gt; is not greater than &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;). Let &lt;span class=&quot;math-container&quot;&gt;$g_{n}$&lt;/span&gt; be an arbitrary neighbour of an arbitrary goal node &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;. Given that &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is consistent, then &lt;span class=&quot;math-container&quot;&gt;$h(g_{n}) \leq w(g_{n}, g) + h(g)$&lt;/span&gt;. Given that &lt;span class=&quot;math-container&quot;&gt;$h(g)$&lt;/span&gt; does not overestimate the cost to reach the goal from &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$w(g_{n}, g) + h(g)$&lt;/span&gt; also does not overestimate the cost of reaching the goal from &lt;span class=&quot;math-container&quot;&gt;$g_n$&lt;/span&gt;, given that &lt;span class=&quot;math-container&quot;&gt;$w(g_{n}, g)$&lt;/span&gt; is the true cost of the edge &lt;span class=&quot;math-container&quot;&gt;$(g_{n}, g) \in E$&lt;/span&gt; and the cost to reach the goal from &lt;span class=&quot;math-container&quot;&gt;$g_n$&lt;/span&gt; must be at least &lt;span class=&quot;math-container&quot;&gt;$w(g_{n}, g)$&lt;/span&gt;. This reasoning can be applied inductively (or recursively) on &lt;span class=&quot;math-container&quot;&gt;$g_n$&lt;/span&gt; (then on the neighbouring nodes of &lt;span class=&quot;math-container&quot;&gt;$g_n$&lt;/span&gt;, and so on), so &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; must be admissible.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T16:24:57.893" LastActivityDate="2019-11-10T16:24:57.893" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16402" PostTypeId="5" CreationDate="2019-11-10T16:29:55.693" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T16:29:55.693" LastActivityDate="2019-11-10T16:29:55.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16403" PostTypeId="4" CreationDate="2019-11-10T16:29:55.693" Score="0" Body="For questions related to admissible heuristics, which are heuristics that never overestimate the cost of reaching a goal." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T16:29:55.693" LastActivityDate="2019-11-10T16:29:55.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16404" PostTypeId="5" CreationDate="2019-11-10T16:31:51.300" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T16:31:51.300" LastActivityDate="2019-11-10T16:31:51.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16405" PostTypeId="4" CreationDate="2019-11-10T16:31:51.300" Score="0" Body="For questions related to consistent heuristics, which are heuristics whose estimate is always less than or equal to the estimated distance from any neighboring vertex to the goal, plus the cost of reaching that neighbor." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T16:31:51.300" LastActivityDate="2019-11-10T16:31:51.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16406" PostTypeId="5" CreationDate="2019-11-10T17:01:46.333" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T17:01:46.333" LastActivityDate="2019-11-10T17:01:46.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16407" PostTypeId="4" CreationDate="2019-11-10T17:01:46.333" Score="0" Body="For questions related to the uniform-cost search (UCS) algorithm, also known as the cheapest-first search, which is the uninformative version of the A* algorithm. The UCS is very similar to Dijkstra's algorithm (originally introduced by Edsger W. Dijkstra in 1956 and published in the 1959 paper &quot;A note on two problems in connexion with graphs&quot;), which originated UCS, according to Norvig and Russell's book." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-17T12:15:44.020" LastActivityDate="2020-11-17T12:15:44.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16408" PostTypeId="1" CreationDate="2019-11-10T17:06:13.603" Score="1" ViewCount="17" Body="&lt;ol&gt;&#xA;&lt;li&gt;Quantization means using low resolution formats for some variables some of the time: binary (e.g. BinaryConnect), ternary, etc. &lt;/li&gt;&#xA;&lt;li&gt;The Turing architecture recently introduced by Nvidia is much faster with half-precision calculations than the previous one.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Intuitively, there is a symbiosis between the two, but how exactly and to what extent can the GTX 16xx and RTX 20xx GPUs speed up these calculations?&lt;/p&gt;&#xA;" OwnerUserId="23584" LastActivityDate="2019-11-10T17:06:13.603" Title="Quantization techniques and new GPU architectures" Tags="&lt;hardware&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-05-13T21:21:03.900" ContentLicense="CC BY-SA 4.0" />
  <row Id="16409" PostTypeId="5" CreationDate="2019-11-10T17:20:22.863" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T17:20:22.863" LastActivityDate="2019-11-10T17:20:22.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16410" PostTypeId="4" CreationDate="2019-11-10T17:20:22.863" Score="0" Body="For questions related to the graph search. As opposed to a tree search, a graph search uses a list or set, called the closed list (or explored set), which contains the already visited and expanded nodes (or states) of the search space (which is usually represented as a graph, both in the case of tree and graph searches), so that not to revisit these already visited nodes." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T23:47:09.053" LastActivityDate="2019-11-10T23:47:09.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16411" PostTypeId="5" CreationDate="2019-11-10T17:23:52.580" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T17:23:52.580" LastActivityDate="2019-11-10T17:23:52.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16412" PostTypeId="4" CreationDate="2019-11-10T17:23:52.580" Score="0" Body="For questions related to the family of algorithms often denoted as best-first search algorithms. A* is an example of a best-first search algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T17:23:52.580" LastActivityDate="2019-11-10T17:23:52.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16413" PostTypeId="5" CreationDate="2019-11-10T17:29:04.587" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T17:29:04.587" LastActivityDate="2019-11-10T17:29:04.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16414" PostTypeId="4" CreationDate="2019-11-10T17:29:04.587" Score="0" Body="For questions related to the tree search. As opposed to the graph search, a tree search does not use a closed list to keep track of the already visited nodes, so a tree search may visit the same nodes (or states) multiple times." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-10T23:48:36.200" LastActivityDate="2019-11-10T23:48:36.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16415" PostTypeId="2" ParentId="16384" CreationDate="2019-11-10T18:34:00.520" Score="2" Body="&lt;p&gt;RL can be used in the context of Neural Architecture Search (NAS), with is a form of automated ML. A model searches for an architecture that performs a given task. How well this task is performed guides how the architecture will be modified (improved) on the next pass. It works but is very computation-intensive (think hundreds of GPUs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See for instance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;B. Zoph and Q. V. Le (2016), &lt;a href=&quot;https://arxiv.org/abs/1611.01578&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv: 1611.01578&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le (2017), &lt;a href=&quot;https://arxiv.org/abs/1707.07012&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv: 1707.07012&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;M. Tan, B. Chen, R. Pang, V. Vasudevan, and Q. V. Le (2018), &lt;a href=&quot;https://arxiv.org/abs/1807.11626&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv: 1807.11626&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23584" LastEditorUserId="23584" LastEditDate="2019-11-11T15:51:46.183" LastActivityDate="2019-11-11T15:51:46.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16416" PostTypeId="1" CreationDate="2019-11-10T18:47:48.893" Score="1" ViewCount="44" Body="&lt;p&gt;I ran my job on a computing cluster: first with 4 cores, then with 32 cores (2 nodes). But the training time is pretty much exactly the same for both of them: ~67 seconds per step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to &lt;a href=&quot;https://github.com/nshepperd/gpt-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;fine-tune GPT-2&lt;/a&gt; for my text dataset (chat logs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do to get a performance increase with  to the increase in processing power with CPUs?&lt;/p&gt;&#xA;" OwnerUserId="31165" LastActivityDate="2019-11-11T14:21:47.310" Title="Training with many CPU cores doesn't improve performance" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="5" ClosedDate="2020-12-29T14:40:47.073" ContentLicense="CC BY-SA 4.0" />
  <row Id="16418" PostTypeId="1" CreationDate="2019-11-10T20:00:07.917" Score="5" ViewCount="670" Body="&lt;p&gt;This at first sounds ridiculous. Of course there is an easy way to write a program to solve a wordsearch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what I would like to do is write a program that solves a word-search like a human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, use or invent different strategies. e.g. search randomly for the starting letter; go line-by-line; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the AI will eventually find out that going line by line looking for a given starting letter of a word is a good strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea how you would write such a strategy-finding AI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main &quot;moves&quot; would be things like &quot;move right one letter in the grid&quot;, &quot;store this word in memory&quot;, &quot;compare this letter with first letter in memory&quot; and a few more.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-11-11T01:15:35.077" Title="How to create an AI to solve a word search?" Tags="&lt;deep-learning&gt;&lt;search&gt;&lt;architecture&gt;&lt;reasoning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16420" PostTypeId="1" AcceptedAnswerId="16426" CreationDate="2019-11-10T21:51:46.357" Score="3" ViewCount="32" Body="&lt;p&gt;For the purpose of object detection, is it better to adjust the natural lighting (while recording the video) or to apply filters (e.g. brightness filters, etc.) on the original video to make it brighter? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition is that it shouldn't matter when you adjust the natural lighting or do it after with video filters.&lt;/p&gt;&#xA;" OwnerUserId="28201" LastEditorUserId="2444" LastEditDate="2019-11-10T23:38:15.427" LastActivityDate="2019-11-11T07:46:40.280" Title="Is it better to adjust the natural lighting (while recording the video) or to subsequently apply filters on the original video?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16421" PostTypeId="2" ParentId="16383" CreationDate="2019-11-11T01:08:50.823" Score="1" Body="&lt;p&gt;If you're really just trying to find long contiguous flat regions in a sequence, &lt;strong&gt;you do not need machine learning&lt;/strong&gt;. Your PI is mistaken. You would be better off simply writing a short data processing program. Your program could find the finite differences between adjacent datapoints, and then count whether a long string of them are below some threshold to identify long flat regions. This will be faster, simpler, and perhaps more accurate than using ML on data visualizations for this task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are trying to find something more complex than these long flat regions, you could instead train an &lt;a href=&quot;https://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt; on the raw sequential data that you are using to generate the images. Again, that will probably be more accurate than trying to train a CNN, or any non-sequential model, on the image data itself.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-11T01:08:50.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16422" PostTypeId="2" ParentId="16418" CreationDate="2019-11-11T01:15:35.077" Score="4" Body="&lt;p&gt;This sounds like a problem that might be solvable with a LSTM-DQN approach, as described in &lt;a href=&quot;https://arxiv.org/pdf/1506.08941.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Language Understanding for Text-based Games using Deep&#xA;Reinforcement Learning&lt;/a&gt; by Narasimhan et al., 2015, and then extended to a domain very similar to your problem in &lt;a href=&quot;https://aaai.org/ojs/index.php/AAAI/article/view/3882&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning for Syntactic Error Repair in Student Programs&lt;/a&gt; by Gupta et al., 2019.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to treat the array of letters each of which can be 'circled' or not, plus the position of a cursor as a state. An action is to move the cursor or toggle the state of a letter. You then model the problem as a reinforcement learning problem, with rewards given every time an entire new word becomes 'circled', probably with a caveat about not circling any invalid letters (otherwise, it'll just learn to circle everything).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a guess, but it seems like it's very closely related to your problem, so it's likely to be effective.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-11T01:15:35.077" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16423" PostTypeId="1" CreationDate="2019-11-11T01:33:59.913" Score="2" ViewCount="73" Body="&lt;p&gt;If I understood correctly, the AlphaGo Zero network returns two values: a vector of logit probabilities &lt;strong&gt;p&lt;/strong&gt; and a value &lt;strong&gt;v&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: in this vector that it is outputted, do we have a probability for every possible action in the game? If so: does it apply a probability of 0 to actions that are not possible in that particular state? If this is true, how does the network know which actions are valid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not: then the network will output vectors of different sizes according to each state. Is this even feasible? And again, how will the network know which actions are valid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related questions but none of them covers this question in specific: &lt;a href=&quot;https://ai.stackexchange.com/questions/7580/is-the-discount-not-needed-in-a-deterministic-environment-for-reinforcement-lear&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/questions/10149/what-is-a-logit-probability&quot;&gt;2&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/questions/12759/how-does-alphazero-use-its-value-and-policy-heads-in-conjunction&quot;&gt;3&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22369" LastActivityDate="2019-11-11T01:33:59.913" Title="AlphaGo Zero: Does the policy head give a probability for every possible move?" Tags="&lt;deep-learning&gt;&lt;policies&gt;&lt;deepmind&gt;&lt;alphago-zero&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16424" PostTypeId="1" CreationDate="2019-11-11T02:17:22.167" Score="2" ViewCount="126" Body="&lt;p&gt;I was curious about how people make AI to play games. Does anyone know of the AI used to play these games? What allows the AI to see/click the screen in real-time? Even just direction on what libraries for such tasks would be helpful. I can't imagine game developers make an API for creating bots in their games like browsers use with selenium.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="2444" LastEditDate="2019-11-12T00:01:49.523" LastActivityDate="2019-11-12T00:01:49.523" Title="What technology do people use to create bots for games like LOL or Runescape?" Tags="&lt;computer-vision&gt;&lt;game-ai&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16425" PostTypeId="2" ParentId="11515" CreationDate="2019-11-11T02:50:59.070" Score="0" Body="&lt;p&gt;Tensorflow released a version, however, it's missing negative sampling, which is very important in word2vec&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/tutorials/text/word_embeddings&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/text/word_embeddings&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18358" LastActivityDate="2019-11-11T02:50:59.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16426" PostTypeId="2" ParentId="16420" CreationDate="2019-11-11T07:46:40.280" Score="1" Body="&lt;p&gt;Personally, I'd say as long as the object is visible don't do either. If the model has been well built and if lighting changes would help, the convolution operation weights would learn an operation similar to contrast or brightness changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand if the object visibility is an issue, then natural lighting changes would be better, due to the lack of potential artefacts a filter would create.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So overall, I'd say natural lighting changes should be more helpful (Assuming model is built well) and brightness filters would not be very helpful as the convolution operations would learn them if they were useful, also there would be  artefacts in the input which can lead to the model learning irrelevant details.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helped! &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-11T07:46:40.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16427" PostTypeId="1" CreationDate="2019-11-11T08:38:07.207" Score="3" ViewCount="26" Body="&lt;p&gt;What is the general approach to defect detection in deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would the approach be better if we try to learn the positive images (defects in images) as much as possible or we try to learn the negative images (images without blemishes) and try to single out the defects as some anomalies&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me to some architecture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;REgards&lt;/p&gt;&#xA;" OwnerUserId="31175" LastActivityDate="2019-11-11T08:38:07.207" Title="Defect Detection System using Deep Learning" Tags="&lt;object-detection&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16428" PostTypeId="2" ParentId="16424" CreationDate="2019-11-11T08:56:51.083" Score="3" Body="&lt;p&gt;Bot development is more about 'hacking' than AI in a way that in the very first place you need to read and (over) write game data which you are not supposed to (and thereby potentially violating the Terms and Conditions - so be aware of that). The AI part is fairly simple for most hack/bot applications that I can think of. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Read data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To read game data you can for example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use an assembly code debugger like Ollydbg to locate relevant data in the memory, e.g. amount of gold&lt;/li&gt;&#xA;&lt;li&gt;Observe graphical objects being rendered, e.g. a unit being drawn&lt;/li&gt;&#xA;&lt;li&gt;Intercept network packages containing all kinds of game information, e.g. a unit appearing on your screen&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Write data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar way there are multiple ways to write data:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;overwrite game data in memory&lt;/li&gt;&#xA;&lt;li&gt;Use the Windows API SendInput() function to emulate keyboard inputs&lt;/li&gt;&#xA;&lt;li&gt;Use the Windows API SendMessage() function to send messages to the game&lt;/li&gt;&#xA;&lt;li&gt;Manipulate network traffic&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These lists are not comprehensive but to give you an idea of how it is being done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI-wise an A* algorithm for example can be deployed to do path finding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in the topic I suggest to read 'Game Hacking' by Nick Cano. The book provides a good introduction. &lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2019-11-11T08:56:51.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16429" PostTypeId="2" ParentId="16175" CreationDate="2019-11-11T10:21:57.777" Score="2" Body="&lt;p&gt;The problem originated because of the nature of the code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code:&#xA;&lt;a href=&quot;https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/classifier.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/classifier.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                model = SVC(kernel='linear', probability=True)&#xA;                model.fit(emb_array, label)&#xA;&#xA;                class_names = [cls.name.replace('_', ' ') for cls in img_data]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you see the code uses a SVC (Support Vector Classifier) to classify the classes. The SVC (or SVM) does not have an extra class for &lt;code&gt;unknown&lt;/code&gt; class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;code&gt;threshold&lt;/code&gt; variable, it is used in face detection, aka drawing a bounding box around the face for FaceNet to classify it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/identify_face_image.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/identify_face_image.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            frame = frame[:, :, 0:3]&#xA;            bounding_boxes, _ = detect_face.detect_face(frame, minsize, pnet, rnet, onet, threshold, factor)&#xA;            nrof_faces = bounding_boxes.shape[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the &lt;code&gt;threshold&lt;/code&gt; variable is only used in detecting the bounding box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code for getting class name:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;predictions = model.predict_proba(emb_array)&#xA;                    print(predictions)&#xA;                    best_class_indices = np.argmax(predictions, axis=1)&#xA;                    # print(best_class_indices)&#xA;                    best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices]&#xA;                    print(best_class_probabilities)&#xA;                    cv2.rectangle(frame, (bb[i][0], bb[i][1]), (bb[i][2], bb[i][3]), (0, 255, 0), 2)    #boxing face&#xA;&#xA;                    #plot result idx under box&#xA;                    text_x = bb[i][0]&#xA;                    text_y = bb[i][3] + 20&#xA;                    print('Result Indices: ', best_class_indices[0])&#xA;                    print(HumanNames)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can see that no &lt;code&gt;unknown&lt;/code&gt; class can be found. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Solution&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You can try adding another threshold value and check if the &lt;code&gt;predictions&lt;/code&gt; maximum value is lower than the threshold value. I have little experience in tensor flow so this is just a proof of concept, not sure if it will work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices] #original code&#xA;if(best_class_probabilities &amp;lt; threshold_2):&#xA;    best_class_indices = -1&#xA;HumanNames = &quot;unknown&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By the way, because of the nature of triplet loss, you don't have to add and extra class to the SVC/SVM as the embedding model is locked and not trained, so unknown class embeddings will be very different to the known class. However you can try either approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it can help you can have a nice day!&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-11T10:21:57.777" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16430" PostTypeId="1" CreationDate="2019-11-11T10:25:31.540" Score="3" ViewCount="51" Body="&lt;p&gt;I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image under construction to resemble the ideal image more. So basically, it translates image differences into motor commands to make them more similar?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm dealing with a 3D virtual environment, so the images are snapshots of objects and motor commands are simple alterations to the 3D shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the network needs two pre-trained CNNs with the same weights that extract image features, then output and concatenate those into a dense layer (or two), which converges into action-space. Training should probably happen via reinforcement learning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, in the end it needs recurrence, since there are multiple motor actions it needs to do in a row to get closer to the intended result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would there be any serious difficulties with this? Or are there any approaches to achieve the intended result? or any similar examples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="31180" LastEditorUserId="31180" LastEditDate="2019-11-11T11:23:19.803" LastActivityDate="2019-11-11T11:23:19.803" Title="Ideas on a network that can translate image differences into motor commands?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16431" PostTypeId="1" CreationDate="2019-11-11T10:46:17.667" Score="3" ViewCount="128" Body="&lt;p&gt;I made an AI for Reversi, aka Othello (8×8), like Alpha Zero, using &lt;a href=&quot;https://www.borndigital.co.jp/book/14383.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;. This book is written in Japanese.  The source code of the AI I implemented can be found &lt;a href=&quot;https://github.com/NL-User/Alpha-Ozero&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this Github repository&lt;/a&gt;. There are comments written in Japanese, but there are also comments written in English. Check the master &lt;a href=&quot;https://github.com/NL-User/Alpha-Ozero&quot; rel=&quot;nofollow noreferrer&quot;&gt;branch's README file&lt;/a&gt; and source code on GitHub for more details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I finished writing the code and I trained the model on Google Colab with a GPU. I stopped learning around the 3rd training cycle because it is very slow. I tried to rewrite to bitboard, but, it also very slow. So, how can I make my implementation faster or, in general, how can I improve my implementation, given that I have a limited amount or resources (including computing resources and money)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some ideas.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reduce the number of hidden layers (it can be weak)&lt;/li&gt;&#xA;&lt;li&gt;Rewrite the source code with Cython (it also run on Google Colab).&lt;/li&gt;&#xA;&lt;li&gt;Rewrite the source code in C++, and run on my PC with GPU.&lt;/li&gt;&#xA;&lt;li&gt;Increase the learning rate (it can do over-fitting)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31182" LastEditorUserId="31182" LastEditDate="2019-11-14T10:00:47.713" LastActivityDate="2019-11-14T10:00:47.713" Title="How can I increase the speed and performance of my implementation of an AI for Reversi?" Tags="&lt;tensorflow&gt;&lt;game-ai&gt;&lt;keras&gt;&lt;implementation&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16434" PostTypeId="2" ParentId="12434" CreationDate="2019-11-11T12:34:16.817" Score="1" Body="&lt;p&gt;Here are two review articles:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Elsken, Metzen, Hutter: &lt;a href=&quot;https://arxiv.org/abs/1808.05377&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Architecture Search: A Survey&lt;/a&gt; (2019), Journal of Machine Learning Research 20, 1-21 &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;He, Zhao, Chu: &lt;a href=&quot;https://arxiv.org/abs/1908.00709&quot; rel=&quot;nofollow noreferrer&quot;&gt;AutoML: A Survey of the State-of-the-Art&lt;/a&gt; (2019) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23584" LastEditorUserId="2444" LastEditDate="2019-11-11T14:16:45.350" LastActivityDate="2019-11-11T14:16:45.350" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16435" PostTypeId="1" AcceptedAnswerId="16436" CreationDate="2019-11-11T13:53:17.760" Score="1" ViewCount="105" Body="&lt;p&gt;This question is related to &lt;a href=&quot;https://ai.stackexchange.com/questions/16311/what-is-the-formula-used-to-calculate-the-accuracy-in-the-facenet-model&quot;&gt;What is the formula used to calculate the accuracy in the FaceNet model?&lt;/a&gt; . I know how loss is calculated in the FaceNet model , but how the loss function is used to calculate probability that this unknown person is , say Bob (0.70). Also  we don't know which is positive or negative image , we only know the Anchor (so how FaceNet finds which image is positive or negative ?) . How probability is calculated in FaceNet Model using triplet loss ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we know what is the exact formula or CNN is like black box which uses some unknown method to calculate probability  ?&lt;/p&gt;&#xA;" OwnerUserId="30306" LastEditorUserId="30306" LastEditDate="2019-11-11T13:59:03.927" LastActivityDate="2019-11-11T14:42:57.950" Title="How is the percentage or the probablity calculated using Loss function in Facenet Model?" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16436" PostTypeId="2" ParentId="16435" CreationDate="2019-11-11T14:42:57.950" Score="1" Body="&lt;p&gt;The facenet model is just the head of the model. The architecture is similar to the enocdr part of an autoencoder, but it uses supervised learning instead of unsupervised learning. The network is called a siamese network The triplet loss helps make the embeddings more representative of the input image/person, with the embedding distance being as large as possible for difference people and vice versa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the embeddings is just the representation of the people. It doesn't contain information directly mapping to what person it is. The classification head is used after the Face Net feature extraction top is used.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;One-shot learning method for person classfication&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This method saves the calculated embeddings of people in a database. The face recognition system first calculates the embeddings of the new, unknown image to be classified. The system loop through the embeddings database and calculates the Euclidean distance of the unknown embeddings and the embeddings in the database. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After computing, it chooses the smallest distance of all and compares it to a threshold set by you. If the distance is larger than the threshold, the classified class is &lt;code&gt;unknown&lt;/code&gt; , or else the resulting class will be the embeddings which results to the lowest distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example code from the deeplearning.ai coursera course (great course on AI btw):&#xA;&lt;a href=&quot;https://github.com/enggen/Deep-Learning-Coursera/blob/master/Convolutional%20Neural%20Networks/Week4/Face%20Recognition/Face%20Recognition%20for%20the%20Happy%20House%20-%20v2.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example code of one shot face recognition&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Advantages and disadvantages of the system&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The system have advantages ranging from no training needed and can classify &lt;code&gt;unknown&lt;/code&gt; class. However, once the number of people in the database becomes huge, the system will run slower. It's performance may also differ depending on the quality of the reference image.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;SVM (Support Vector Machine) method&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This is the method used in the github repository of your previous posts. It uses a one layer nn classifying head to output the classified class.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Advantages and disadvantages of the system&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The system have the flexibility of adapting to multiple photo environment and no affecting of performance due to reference images. This method also works well for larger database of people. Triplet loss can also be removed using this method. You can train the network directly from propagating the loss with the probability predicted. However, This method requires re-training when a new person is added to the system. Multiple images of a person is also needed.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-11T14:42:57.950" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16437" PostTypeId="2" ParentId="11759" CreationDate="2019-11-11T15:07:38.460" Score="2" Body="&lt;p&gt;Yes, in both cases. Below I give two very simple proofs that directly follow from the definitions of admissible and consistent heuristics. However, in a nutshell, the idea of the proofs is that &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}(n)$&lt;/span&gt; are, &lt;strong&gt;by definition&lt;/strong&gt; (of &lt;span class=&quot;math-container&quot;&gt;$h_{\max}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}$&lt;/span&gt;), equal to one of the given admissible (or consistent) heuristics, for all nodes &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}(n)$&lt;/span&gt; are consequently admissible (or consistent).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Definitions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Consider the graph &lt;span class=&quot;math-container&quot;&gt;$G=(V, E, \mathcal{G})$&lt;/span&gt; representing the search space, where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{G} \subseteq V$&lt;/span&gt; are respectively the set of nodes, edges and goal nodes, and the function &lt;span class=&quot;math-container&quot;&gt;$w\colon E \times E \rightarrow \mathbb{R}$&lt;/span&gt;, which gives you the cost of each edge &lt;span class=&quot;math-container&quot;&gt;$e = (u, v) \in E$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$u, v \in V$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$w(e) = w(u, v) \in \mathbb{R}$&lt;/span&gt; is the cost of the edge &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A heuristic &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is admissible if &lt;span class=&quot;math-container&quot;&gt;$$h(n) \leq h^*(n), \forall n \in V,$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$h^*(n)$&lt;/span&gt; is the optimal cost to reach a goal from node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (that is, it is the optimal heuristic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, a heuristic &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is consistent if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;h(n) &amp;amp;\leq w(n, s) + h(s), \forall n \in V \setminus \mathcal{G}, \text{ and} \\&#xA;h(n) &amp;amp;= 0, \forall n \in \mathcal{G},&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is a successor of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Theorem 1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Given a set of admissible heuristics &lt;span class=&quot;math-container&quot;&gt;$H = \{ h_1, \dots, h_N \}$&lt;/span&gt;, then, for every &lt;span class=&quot;math-container&quot;&gt;$n \in V$&lt;/span&gt;, the heuristics &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n) = \max(h_1(n), \dots, h_N(n))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}(n) = \min(h_1(n), \dots, h_N(n))$&lt;/span&gt; are also admissible.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Proof&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Given that, &lt;span class=&quot;math-container&quot;&gt;$h_i(n) \leq h^*(n), \forall n \in V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\forall i \in \{ 1, \dots N \}$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n) = h_j(n) \leq h^*(n)$&lt;/span&gt; (for some &lt;span class=&quot;math-container&quot;&gt;$j \in \{ 1, \dots N \}$&lt;/span&gt;)  and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}(n) = h_k(n) \leq h^*(n)$&lt;/span&gt; (for some &lt;span class=&quot;math-container&quot;&gt;$k \in \{ 1, \dots N \}$&lt;/span&gt;), so &lt;span class=&quot;math-container&quot;&gt;$h_{\max}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}$&lt;/span&gt; are also admissible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\tag*{$\blacksquare$}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Theorem 2&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Given a set of consistent heuristics &lt;span class=&quot;math-container&quot;&gt;$H = \{ h_1, \dots, h_N \}$&lt;/span&gt;, then, for every &lt;span class=&quot;math-container&quot;&gt;$n \in V$&lt;/span&gt;, the heuristics &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n) = \max(h_1(n), \dots, h_N(n))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}(n) = \min(h_1(n), \dots, h_N(n))$&lt;/span&gt; are also consistent.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Proof&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Given that, for every &lt;span class=&quot;math-container&quot;&gt;$i \in \{ 1, \dots, N \}$&lt;/span&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\begin{cases}&#xA;h_i(n) \leq w(n, s) + h_i(s), &amp;amp; \text{ if }  n \in V \setminus \mathcal{G}\\&#xA;h_i(n) = 0, &amp;amp; \text{ if }  n \in \mathcal{G}&#xA;\end{cases}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then, if &lt;span class=&quot;math-container&quot;&gt;$n \in V \setminus \mathcal{G}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n) = h_j(n) \leq w(n, s) + h_j(s)$&lt;/span&gt;, for some &lt;span class=&quot;math-container&quot;&gt;$j \in \{1, \dots, N \}$&lt;/span&gt;, and, similarly, &lt;span class=&quot;math-container&quot;&gt;$h_{\min}(n) = h_k(n) \leq w(n, s) + h_{k}(s)$&lt;/span&gt;, for some &lt;span class=&quot;math-container&quot;&gt;$k \in \{1, \dots, N \}$&lt;/span&gt;. Similarly, if &lt;span class=&quot;math-container&quot;&gt;$n \in \mathcal{G}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_{\max}(n) = h_{\min}(n) = h_i(n) = 0, \forall i \in \{1, \dots, N \} $&lt;/span&gt;. Thus, &lt;span class=&quot;math-container&quot;&gt;$h_{\max}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}$&lt;/span&gt; are also consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\tag*{$\blacksquare$}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Notes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_{\max}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\min}$&lt;/span&gt; have been defined in the only possible reasonable way, because, given two (admissible or consistent) heuristics, &lt;a href=&quot;https://ai.stackexchange.com/q/15701/2444&quot;&gt;one may not always dominate the other (or vice-versa)&lt;/a&gt;, even if both are admissible and consistent.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-11T17:03:57.083" LastActivityDate="2019-11-11T17:03:57.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16438" PostTypeId="1" CreationDate="2019-11-11T15:34:12.150" Score="2" ViewCount="366" Body="&lt;p&gt;I would like to decompile a compiled file to source code.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to use any AI technique to perform decompilation? Is there any research on this topic? If yes, can you briefly explain one of the existing approaches (just to get some insight)? I would also appreciate links to research papers on this topic.&lt;/p&gt;&#xA;" OwnerUserId="9126" LastEditorUserId="2444" LastEditDate="2020-11-19T15:27:43.220" LastActivityDate="2020-11-19T15:27:43.220" Title="Is it possible to create a decompiler using AI?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16439" PostTypeId="2" ParentId="12761" CreationDate="2019-11-11T15:47:29.183" Score="1" Body="&lt;p&gt;Other replies are commenting on the skip connections for a U-Net. I believe you want to exclude these skip connections from your auto-encoder. You say you want to use the auto-encoder for unsupervised pretraining, for which you want to pass the data through a bottle neck, so adding skip connections would work against you if you want to use the encoder for a classification task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You ask whether the decoder should 'mirror' the MobileNet encoder. This is actually an interesting one, and I think it could work even if the decoder does not look like the encoder at all. Since you don't need to (and in fact shouldn't) add skip connections, this should be easy to try.&lt;/p&gt;&#xA;" OwnerUserId="31192" LastActivityDate="2019-11-11T15:47:29.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16440" PostTypeId="1" AcceptedAnswerId="16446" CreationDate="2019-11-11T16:01:15.070" Score="5" ViewCount="316" Body="&lt;p&gt;I have read articles on how Jensen-Shannon divergence is preferred over Kullback-Leibler in measuring how good a distribution mapping is learned in a generative network because of the fact that JS-divergence better measures distribution similarity when there are zero values in either distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unable to understand how the mathematical formulation of JS-divergence would take care of this and also what advantage it particularly holds qualitatively apart from this edge case. Could anyone explain or link me to an explanation that could answer this satisfactorily? &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-12T00:58:24.067" Title="Why is Jensen-Shannon divergence preferred over Kullback-Leibler divergence in measuring the performance of a generative network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16441" PostTypeId="1" AcceptedAnswerId="16466" CreationDate="2019-11-11T16:09:48.093" Score="3" ViewCount="70" Body="&lt;p&gt;Of my understanding mode-collapse is when there happen to be multiple classes in the dataset and the generative network converges to only one of these classes and generates images only within this class. On training the model more, the model converges to another class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Goodfellows NeurIPS presentation he clearly addressed how training a generative network in an adversarial manner avoids mode-collapse. How exactly do GAN's avoid mode-collapse? and did previous works on generative networks not try to address this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from the obvious superior performance (generally), is the fact that GAN's address mode-collapse make them far preferred over other ways of training a generative model?&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-13T14:45:25.077" Title="How exactly does adversarial training help in handling mode-collapse in generative networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16443" PostTypeId="1" CreationDate="2019-11-11T20:44:01.343" Score="4" ViewCount="763" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qKVPw.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qKVPw.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above model is what really helped me understand the implementation of convolutional neural networks, so based on that, I've got a tricky hypothesis that I want to find more about, since actually testing it would involve developing an entirely new training model if the concept hasn't already been tried elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been building a machine learning project for image recognition and thought about how at certain stages we flatten the input after convoluting and max pooling, but it occurred to me that by flattening the data, we're fundamentally losing positional information. If you think about how real neurons process information based on clusters, it seems obvious that proximity of the biological neurons is of great significance rather than thinking of them as flat layers, by designing a neural network training model that takes neuron proximity into account in deciding the structure by which to form connections between neurons, so that positional information can be utilized and kept relevant, it seems that it would improve network effectiveness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit, for clarification, I made an image representing the concept I'm asking about:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wWPO2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wWPO2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically: Pixels 1 and 4 are related to each other and that's very important information. Yes we can train our neural network to know those relationships, but that's 12 unique relationships in just a 3x3 pixel grid that our training process needs to successfully teach the network to value, whereas a model that takes proximity of neurons into consideration, like the real world brain would maintain the importance of those relationships since neurons connect more readily to others in proximity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Does anyone know of white papers / experiments closely related to the concept I'm hypothesizing? Why would or would that not be a fundamentally better model?&lt;/p&gt;&#xA;" OwnerUserId="7249" LastEditorUserId="7249" LastEditDate="2019-11-12T01:31:24.067" LastActivityDate="2019-11-12T06:27:59.203" Title="Wouldn't convolutional neural network models work better without flattening the input in any stages?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16444" PostTypeId="1" CreationDate="2019-11-11T22:47:11.313" Score="2" ViewCount="161" Body="&lt;p&gt;I would like to train a deep neural network to recognize bomb explosions. I was wondering if there is an open visual dataset for bomb explosion? Alternatively, if you know a good deep architecture or code repository for explosion detection, please also share it. Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="2444" LastEditDate="2019-11-12T14:36:47.437" LastActivityDate="2019-11-12T15:23:11.810" Title="Is there a dataset for the detection of bomb explosions?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;datasets&gt;&lt;object-recognition&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="1" ClosedDate="2021-02-03T16:55:44.987" ContentLicense="CC BY-SA 4.0" />
  <row Id="16445" PostTypeId="2" ParentId="16443" CreationDate="2019-11-11T23:58:25.607" Score="3" Body="&lt;p&gt;I have had similar thoughts about neural networks before. Convolution layers are layers of two dimensional nodes effectively passing the spacial data so why don't we use two dimensional hidden layers to receive information out of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure someone has used this type of implementation before. I believe the papers bellow are using this. Part of the point of neural networks is that the weights are trained in order to optimize finding the best solution so regardless of the spacial information it learns to 'focus'/increase weight on locations that are associated with deciding the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of the problem where your neural network examines an image and gives true or false. Training images are True if the center is red and one of the corners is blue or if the center is blue and one of the corners are red. Flattening the layers or not should have basically no effect on this model. In other circumstances like object detection or labeling outlines yes I believe not flattening will benefit the model. With that said flattening the data does not erase spacial relationship each layer will still be trained to detect the spacial information that gives a correct answer the flattened layers just won't have the benefit of neighbors when the layers are one dimensional instead of two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a a CNN with multi class detecting as the task you could allow each class to have its own CNN like hidden layers that narrow to a decision node and decide if they match that class or not. Imagine a palm tree shape where the palm trunk is the image convolutions and each leaf on the top are the two dimensional hidden layers that narrow to an output layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-540-74690-4_56&quot; rel=&quot;nofollow noreferrer&quot; &gt; Multi-dimensional NN &lt;/a&gt;   and&#xA;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/80212&quot; rel=&quot;nofollow noreferrer&quot; &gt; Three dimensional Neural Network &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I spoke in a lot of abstraction so if any part doesn't make sense, I'll make an edit to clarify. &lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-11T23:58:25.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16446" PostTypeId="2" ParentId="16440" CreationDate="2019-11-12T00:58:24.067" Score="2" Body="&lt;p&gt;Lets start with question &lt;strong&gt;1)&lt;/strong&gt;  &lt;em&gt;how does JS-divergence handles zeros?&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;by definition:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;D_{JS}(p||q) &amp;amp;= \frac{1}{2}[D_{KL}(p||\frac{p+q}{2}) + D_{KL}(q||\frac{p+q}{2})] \\&#xA;&amp;amp;= \frac{1}{2}\sum_{x\in\Omega} [p(x)log(\frac{2 p(x)}{p(x)+q(x)}) + q(x)log(\frac{2 q(x)}{p(x)+q(x)})]&#xA;\end{align}&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; is the union of the domains of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. Now lets assume one distribution is zero where the other is not (without loss of generality due to symmetry we can just say &lt;span class=&quot;math-container&quot;&gt;$p(x_i) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q(x_i) \neq 0$&lt;/span&gt;. We then get for that term in the sum&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{2}q(x_i)log(\frac{2q(x_i)}{q(x_i)}) = q(x_i)\frac{log(2)}{2}$$&lt;/span&gt;&lt;br&gt;&#xA;Which isn't undefined as it would be the KL case.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now onto &lt;strong&gt;2)&lt;/strong&gt; &lt;em&gt;In GANS why does JS divergence produce better results than KL&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The asymmetry of KL divergence places an unfair advantage to one distribution over the other where in this case, its not ideal to consider it this way from an optimization perspective. Additionally KL divergences inability to handle non-overlapped distributions is crushing given that these are approximated through sampling schemes, and therefore there are no guarantees. JS solves both those issues and leads to a smoother manifold which is why its generally preferred. A good resource is &lt;a href=&quot;https://arxiv.org/pdf/1511.05101.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; where they go more in detail investigating this.    &lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-11-12T00:58:24.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16447" PostTypeId="1" CreationDate="2019-11-11T07:53:49.080" Score="1" ViewCount="342" Body="&lt;p&gt;I want to use deep reinforcement learning for vehicle rerouting in SUMO, but I don't know how to start training the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already created road network and vehicle routing in SUMO-XML files (mymap.net.xml and mymap.rou.xml). Currently, I'm trying to train the model on Jupyter Notebook, importing TraCI library to control the SUMO simulator and allow for a reinforcement learning approach. However, I'm still confused in training step. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do I need any traffic data to train my agent to take actions in the environment? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I train based on these SUMO-XML files I created?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to run the simulation on Windows? or I need to change to Ubuntu instead?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if someone could guide me. Thank you in advance.&lt;/p&gt;&#xA;" OwnerDisplayName="Chantakarn" LastActivityDate="2019-11-19T11:38:38.167" Title="How can I use deep reinforcement learning for vehicle rerouting in SUMO?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16448" PostTypeId="1" AcceptedAnswerId="16451" CreationDate="2019-11-12T06:17:04.593" Score="13" ViewCount="7485" Body="&lt;p&gt;What does AI software look like? What is the major difference between AI software and other software?&lt;/p&gt;&#xA;" OwnerUserId="31211" LastEditorUserId="16909" LastEditDate="2019-11-12T17:26:23.537" LastActivityDate="2021-03-30T07:55:18.317" Title="What does AI software look like, and how is it different from other software?" Tags="&lt;comparison&gt;&lt;implementation&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16449" PostTypeId="2" ParentId="16443" CreationDate="2019-11-12T06:27:59.203" Score="1" Body="&lt;p&gt;Read on Fully Convolutional Networks (FCN). There is a lot of papers on the subject, first was &quot;Fully Convolutional Networks for Semantic Segmentation&quot; by Long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is quite close to what you describe - preserve spatial locality in the  layers. In FCN there is no fully connected layer. Instead there is average pooling on top of last low-resolution/high-channels layer. The effect is like as if you have several fully connected layer centered on different locations and end result produced by weighted voting of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pleasant side effect of FCN is that they work on any spatial image size(bigger then receptive field) - image size is not coded into network.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-11-12T06:27:59.203" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16451" PostTypeId="2" ParentId="16448" CreationDate="2019-11-12T08:46:47.460" Score="19" Body="&lt;p&gt;Code in AI is not in principle different from any other computer code. After all, you encode algorithms in a way that computers can process them. Having said that, there are a few points where your typical &amp;quot;AI Code&amp;quot; might be different:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A lot of (especially early) AI code was more research based and exploratory, so certain programming languages were favoured that were not mainstream for, say, business applications. For example, much work in early AI has been coded in Lisp, and probably not much in Fortran or Cobol, which were more suited to engineering or business. Special languages were developed to make it easy to program with symbols and logic (eg Prolog).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The emphasis was more on algorithms than clever/complex programming. If you look at the source code for &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELIZA&lt;/a&gt; (there are multiple implementations in many different languages), it's really very simple.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Before the advent of neural networks and (statistical) machine learning, most AI programming was symbolic, so there hasn't been much emphasis on numerical computing. This changed as probabilities and fuzziness were increasingly used, but even if using general purpose languages there would be fewer numerical calculations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Self-modifying code is inherently complex; while eg Lisp made no difference between code and data (at least not in the same way as eg C or Pascal), this would just complicate development without much gain. Perhaps in the early days this was necessary when computers had precious little memory and power and you had to work around those constraints. But these days I don't think anybody would use such techniques anymore.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As modern programming languages evolved, Lisp and Prolog (which were the dominant AI languages until probably 20 to 30 years ago) have been slowly replaced by eg Python; probably because it is easier to find programmers comfortable in an imperative paradigm rather than a functional one. In general, interpreted languages would be preferred over compiled ones due to speed of development, unless performance is important.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The move to deep learning has of course shifted this a lot. Now the core processing is all numeric, so you would want languages that are better with calculations than symbol handling. Interpreted languages would now mainly make up the 'glue' code to interface between compiled modules, and be used for data pre-processing. So current AI code is probably not really that different from code used in scientific computing these days.&lt;/p&gt;&#xA;&lt;p&gt;There is of course still a difference between R&amp;amp;D and production code. You might explore a subject using an interpreted language, and then re-code your algorithm for production in a compiled language to gain better performance. This depends on how established the area is; there will for example be ready-made libraries available for neural networks or genetic algorithms which are well-established algorithms (where performance matters).&lt;/p&gt;&#xA;&lt;p&gt;In conclusion: I don't think AI code is any more complex than any other code. Of course, that's not very exciting to portray in a film, so artistic licence is used to make it more interesting. I guess self-modifying code also enables the machines to develop their own conscience and take over the world, which is even more gripping as a story element. However, given that a lot of behaviour is nowadays in the (training/model/configuration) data rather than the algorithm, this might even be more straight forward to modify.&lt;/p&gt;&#xA;&lt;p&gt;Note: this is a fairly simplified summary based on my own experience of working in AI; other people's views might vary, without either being 'wrong'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Update 2021&lt;/em&gt;: I now work at a company that extracts business information/events from news data on a large scale using NLP methods. And we're using Lisp... so it's still in active, commercial use in AI.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2021-03-30T07:55:18.317" LastActivityDate="2021-03-30T07:55:18.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16452" PostTypeId="1" CreationDate="2019-11-12T09:25:16.940" Score="3" ViewCount="87" Body="&lt;p&gt;As per the law of unintended consequences, could it be that deepfakes will eventually have the opposite effect to what people currently seem to fear most. For example, once it is clear that &lt;strong&gt;anyone&lt;/strong&gt; can be deepfaked to unlimited degrees of precision, wouldn't we have a situation where in regards to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;pornography, revenge-porn: no one (including the person being viewed) will actually care anymore. E.g. if a movie star's account gets hacked and nude pictures are released to the public, it becomes a non-story because everone simply assumes it's one of thousands of other deepfakes that already exist.&lt;/li&gt;&#xA;&lt;li&gt;fake-news, government propaganda: the general public will demand multiple sources, witnesses before believing the next crazy story. That, I assume, is a good thing as well. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31214" LastEditorUserId="1671" LastEditDate="2019-11-14T20:46:02.030" LastActivityDate="2019-11-14T20:46:02.030" Title="Deepfakes as &quot;force for good&quot;?" Tags="&lt;neural-networks&gt;&lt;social&gt;&lt;deepfakes&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16454" PostTypeId="2" ParentId="13647" CreationDate="2019-11-12T13:25:14.710" Score="0" Body="&lt;p&gt;You can use this to view the Keras Resnet Inception V2 network.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.applications.inception_resnet_v2 import InceptionResNetV2, preprocess_input&#xA;from keras.layers import Input&#xA;model = InceptionResNetV2(weights='imagenet', include_top=True)&#xA;print(model.summary())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will Output (im showing only the last few layers):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__________________________________________________________________________________________________&#xA;conv_7b_ac (Activation)         (None, 8, 8, 1536)   0           conv_7b_bn[0][0]                 &#xA;__________________________________________________________________________________________________&#xA;avg_pool (GlobalAveragePooling2 (None, 1536)         0           conv_7b_ac[0][0]                 &#xA;__________________________________________________________________________________________________&#xA;predictions (Dense)             (None, 1000)         1537000     avg_pool[0][0]                   &#xA;==================================================================================================&#xA;Total params: 55,873,736&#xA;Trainable params: 55,813,192&#xA;Non-trainable params: 60,544&#xA;__________________________________________________________________________________________________&#xA;None&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we look at the output of the 'avg_pool' layer from 'Top'. There will be 1536 features at the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make a model in this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.applications.inception_resnet_v2 import InceptionResNetV2, preprocess_input&#xA;from keras.layers import Input&#xA;import numpy as np&#xA;def extract(image_path):&#xA;    base_model = InceptionResNetV2(weights='imagenet', include_top=True)&#xA;    model = Model(inputs=base_model.input,outputs=base_model.get_layer('avg_pool').output)&#xA;&#xA;    img = image.load_img(image_path, target_size=(299, 299))&#xA;    x = image.img_to_array(img)&#xA;    x = np.expand_dims(x, axis=0)&#xA;    x = preprocess_input(x)&#xA;&#xA;    # Get the prediction.&#xA;    features = model.predict(x)    &#xA;    features = features[0]&#xA;    return features&#xA;&#xA;&#xA;features=[]&#xA;features = extract(image)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I couldn't try the code as, right now, I don't have an environment to test this code.&lt;/p&gt;&#xA;" OwnerUserId="31221" LastActivityDate="2019-11-12T13:25:14.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16455" PostTypeId="2" ParentId="16448" CreationDate="2019-11-12T13:31:14.413" Score="10" Body="&lt;p&gt;Oliver Mason's answer is quite good, but I think it can be expanded upon a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there are extra factors that could be popularly interpreted as making AI code difficult to read (as compared to other code):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;AI code actually &lt;em&gt;is&lt;/em&gt; more complex than most code that is written. When we work in AI, we often lose sight of this, but most code ever written does one of two things: turn data in one standard format into data in another standard format; display something to a user. Both of those are conceptually easy to understand. Neither of them is likely to require knowledge of mathematics. This is very unlike most code written in AI, where understanding what was written, and why, requires extensive knowledge &lt;em&gt;beyond&lt;/em&gt; the knowledge needed to read and write computer programs. So, &lt;strong&gt;reading AI code requires more knowledge of mathematics or of complicated AI-focused Algorithms&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The &quot;programs written by AIs&quot; are really our &lt;em&gt;models&lt;/em&gt; in the modern context. Our algorithms &quot;program&quot; a template model to make it work for a specific application. This is especially true if you think of it in the senses in which programming is also used in &quot;linear programming&quot;, &quot;quadratic programming&quot;, and even &quot;dynamic programming&quot;. Our models really &lt;em&gt;are&lt;/em&gt; hard to understand. Often even their creators cannot explain or characterize the model's behavior on specific inputs without running the model. The reason for this is that our &lt;strong&gt;models do not represent simple enough concepts that humans can easily understand or simplify them&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Self-modifying code is rare, &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;noreferrer&quot;&gt;but does exist&lt;/a&gt; within AI. However, as with other AI-generated models, AI-generated code tends to be comparatively difficult for humans to interpret, because (unlike most human-generated code), &lt;strong&gt;it is not written with the intention that humans are going to try to read and understand it&lt;/strong&gt;. There actually are some efforts to generate code that &lt;a href=&quot;https://blogs.oracle.com/meena/code-generation-using-lstm-long-short-term-memory-rnn-network&quot; rel=&quot;noreferrer&quot;&gt;conforms to human&lt;/a&gt; styles, but usually the code that is generated does not work well.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-12T14:16:58.193" LastActivityDate="2019-11-12T14:16:58.193" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16456" PostTypeId="1" CreationDate="2019-11-12T14:20:24.030" Score="2" ViewCount="530" Body="&lt;p&gt;I am trying to manually implement calculations of the image classification process using pre-trained weights from the MobilenetV2 network. I know how to apply filter weights to the channels, but not sure what to do with the coefficients from the batch normalization (BN) layers. The model uses BN after each convolution before ReLu6. As explained in many sources, BN has a lot of benefits during model training. The &lt;a href=&quot;https://arxiv.org/pdf/1801.04381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original Mobilenetv2 paper&lt;/a&gt; does say that they used BN during training, but nothing about using it during testing. The pre-trained MobilenetV2 model comes with BN layers which contain weights 4 x n_channels (I assume gamma, beta, mean, and std for each input featuremap in the BN layer). The following questions is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I apply the four coefficients to a featuremap during inference? (This &lt;a href=&quot;https://towardsdatascience.com/batch-normalization-theory-and-how-to-use-it-with-tensorflow-1892ca0173ad&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; explains it, but I still don't get it - aren't those imported coefficients already pre-calculated, so the operation on a featuremap is reduced to a multiply-add?)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper on BN&lt;/a&gt; in section 3.1 says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... Since the means and variances are fixed during inference,&#xA;  the normalization is simply a linear transform applied to&#xA;  each activation. It may further be composed with the scaling by γ and shift by β, to yield a single linear transform&#xA;  that replaces BN(x)...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does this mean that during inference I would use &lt;strong&gt;only &lt;em&gt;gamma&lt;/em&gt; and &lt;em&gt;beta&lt;/em&gt; coefficients&lt;/strong&gt; to &quot;scale and shift&quot; each pixel of a corresponding feature map? That is, something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ch&#xA;  for row&#xA;    for col&#xA;      out_feature[row][col][ch] = in_feature[row][col][ch] * BN[gamma][ch] + BN[beta][ch]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could anyone, please, confirm and explain if this is correct and re-iterate what exactly is expected from the BN layer output in terms of value ranges (before ReLu6)?&lt;/p&gt;&#xA;" OwnerUserId="31222" LastEditorUserId="31222" LastEditDate="2019-11-12T14:33:23.967" LastActivityDate="2019-11-12T14:33:23.967" Title="How to properly use batch normalization during inference" Tags="&lt;convolutional-neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16457" PostTypeId="1" CreationDate="2019-11-12T14:25:44.363" Score="3" ViewCount="98" Body="&lt;p&gt;One of the main disadvantages of the MC Policy Gradient algorithm (REINFORCE) as described say &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/pg.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is the fact that it has high variance (returns, which we sample, will significantly vary from episode to episode). Therefore it is perfectly reasonable to use a critic to reduce variance and this is what for example Deep Deterministic Policy Gradient (&lt;a href=&quot;https://arxiv.org/abs/1509.02971&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG)&lt;/a&gt; does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's assume that we're given an MDP with completely deterministic dynamics. In that case, if we start from a specific state and follow a certain deterministic policy we will always obtain the exact same return (therefore we have zero bias and zero variance). If we follow a certain stochastic policy the return will vary depending on how much we explore, but under an almost-deterministic policy our variance will be quite small. In any case, there's no contribution to the variance from the deterministic MDP dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In deep reinforcement learning for portfolio optimization, many researchers (&lt;a href=&quot;https://arxiv.org/pdf/1811.07522.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xiong at al. for example&lt;/a&gt;) use historical market data for model training. The resulting MDP dynamics is of course completely deterministic (if historical prices are used as states) and there's no real sequentiality involved. Consequently, all return variance stems from the stochasticity of the policy itself. However, most researchers still use DDPG as a variance reduction mechanism. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What's the point of using DDPG for variance reduction when the underlying MDP used for training has deterministic dynamics? Why not simply use the Reinforce algorithm?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="26195" LastActivityDate="2019-11-13T12:42:27.143" Title="Purpose of using actor-critic algorithms under deterministic MDP dynamics?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16458" PostTypeId="1" AcceptedAnswerId="16464" CreationDate="2019-11-12T14:35:54.737" Score="2" ViewCount="129" Body="&lt;p&gt;I want to create a solution, which clones my voice. I tried my commercial solutions or implementation of Tacotron. Unfortunately, results not sound natural, generated voice sounds like a robot. Anybody could recommend good alternative?&lt;/p&gt;&#xA;" OwnerUserId="19476" LastActivityDate="2019-11-12T15:57:30.627" Title="AI natural voice generator" Tags="&lt;deep-learning&gt;&lt;voice-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16459" PostTypeId="1" CreationDate="2019-11-12T15:06:16.140" Score="1" ViewCount="117" Body="&lt;p&gt;My question regards performing keyword spotting for custom keywords and justifying the use of keyword spotting models instead of speech recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been doing some searching around Keyword Spotting and I realized there is not so much work out there. Probably the most common dataset I have found people using is the &lt;a href=&quot;https://ai.googleblog.com/2017/08/launching-speech-commands-dataset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speech Commands Dataset&lt;/a&gt;. However, this dataset has only 30 keywords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want keyword spotting for my own custom application, then to the best of my knowledge I need either a pre-trained model or my own data to train a model on. However, to the best of my knowledge, there is no model pre-trained on a dataset with a large enough set of keywords that is likely to cover a lot of applications. Correct me if I am wrong in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have come to the conclusion that I need to train my own model and the only two ways I could train models on custom keywords is to get that data myself, either by crowdsourcing or by performing speech recognition on large datasets, picking up segments which include the words of interest and then doing some manual work to check if these segments truly include the keywords I want. Does someone think that this would be a good or bad idea and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, why would I even bother going the keyword detection route and not just use a speech recognition model that will recognize the words a human speaks and see if any of them match my keyword? Is the performance that much better with keyword detection?&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2019-11-12T15:06:16.140" Title="Keyword spotting with custom keywords and why not use speech recognition instead" Tags="&lt;machine-learning&gt;&lt;speech-recognition&gt;&lt;speech&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16460" PostTypeId="2" ParentId="10910" CreationDate="2019-11-12T15:10:22.737" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;Initial state: initial position of the monkey.&lt;/li&gt;&#xA;&lt;li&gt;Possible actions&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;climb on the crate, &lt;/li&gt;&#xA;&lt;li&gt;get down the crate, &lt;/li&gt;&#xA;&lt;li&gt;move the crate from one spot to another, &lt;/li&gt;&#xA;&lt;li&gt;stack one crate on another, &lt;/li&gt;&#xA;&lt;li&gt;walk from one spot to another,&lt;/li&gt;&#xA;&lt;li&gt;grab bananas (if standing on the crate)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Goal test: did the monkey get the bananas?&lt;/li&gt;&#xA;&lt;li&gt;Cost function: the number of actions completed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31184" LastEditorUserId="2444" LastEditDate="2019-11-12T17:15:30.047" LastActivityDate="2019-11-12T17:15:30.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16463" PostTypeId="1" CreationDate="2019-11-12T15:43:19.450" Score="1" ViewCount="30" Body="&lt;p&gt;Is it possible to do person detection and object detection within one model? The training data would be images annotated with bounding boxes for objects and people.  Because normally object detection and person detection are done separately? Is there any research about models that simultaneously detect both people and objects?&lt;/p&gt;&#xA;" OwnerUserId="28201" LastActivityDate="2019-11-12T16:27:36.453" Title="Two Models vs One Model for Person Detection and Object Detection" Tags="&lt;object-detection&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16464" PostTypeId="2" ParentId="16458" CreationDate="2019-11-12T15:57:30.627" Score="2" Body="&lt;p&gt;The reason for robot like speech may be because tacotron uses griffin lim for vocoder, which cannot reproduce sound with perfection, often introducing robot like sound artifects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uS31D.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uS31D.png&quot; alt=&quot;tacotron img&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A vocoder is a network that transforms a transform a spectrogram image back to speech waveform. Tacotron and many other speech generation neural network uses CNN to generate spectrogram instead of raw waveforms as output. Spectrogram is a lossy representation of raw audio waveform, so a perfect reconstruction of audio waveform is not possible. Griffin-Lim is a vocoder that uses algorithmic way to transform spectrogram to audio waveform, but often introduces a robot-like quality to generated waveforms. A neural network based vocoder can solve the problem. The wavenet vocoder is often used in speech generation as it can transform the spectrogram to audio with little artifects. Many new speech generation models use the wavenet vocoder as the deafult vocoder of the generation model. For a public implementation, this is a good github repository: &lt;a href=&quot;https://github.com/r9y9/wavenet_vocoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/r9y9/wavenet_vocoder&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use the newer tacotron 2 which uses the wavenet vocoder as the default vocoder. You can check it out here: &lt;a href=&quot;https://github.com/Rayhane-mamah/Tacotron-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Rayhane-mamah/Tacotron-2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-12T15:57:30.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16465" PostTypeId="1" AcceptedAnswerId="16470" CreationDate="2019-11-12T15:57:44.070" Score="5" ViewCount="135" Body="&lt;p&gt;I have been reading the paper which introduced &lt;a href=&quot;https://arxiv.org/pdf/1802.05957.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;spectral normalization&lt;/a&gt; in GANs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At some point the paper mentions the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The machine learning community has been pointing out recently that the&#xA;  function space from which the discriminators are selected crucially&#xA;  affects the performance of GANs. A number of works (Uehara et al.,&#xA;  2016; Qi, 2017; Gulrajani et al., 2017) advocate the importance of&#xA;  Lipschitz continuity in assuring the boundedness of statistics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does it mean that the Lipschitz continuity assures the boundedness of statistics and why does that happen?&lt;/p&gt;&#xA;" OwnerUserId="13257" LastEditorUserId="2444" LastEditDate="2019-11-12T17:10:29.860" LastActivityDate="2019-11-12T21:16:45.607" Title="Why does a Lipschitz continuous discriminator in GANs assure statistical boundedness?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;math&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16466" PostTypeId="2" ParentId="16441" CreationDate="2019-11-12T16:23:17.270" Score="1" Body="&lt;p&gt;I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then &lt;span class=&quot;math-container&quot;&gt;$min_{\phi}max_{\theta}L(G_\phi, D_\theta)$&lt;/span&gt; starts to look a lot like &lt;span class=&quot;math-container&quot;&gt;$max_{\theta}min_{\phi}L(G_\phi, D_\theta)$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is problematic because in the latter case the generator has an obvious minimum of transforming all generated output into a single-mode that the discriminator has considered acceptable.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then a lot of work has been done to deal with this point of failure. Examples include &lt;a href=&quot;https://arxiv.org/abs/1611.02163&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unrolled GANs&lt;/a&gt; (he mentions this one in the talk), where you essentially make the generator optimize what the discriminator &lt;em&gt;will&lt;/em&gt; think &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; steps in the future to ensure the ordering of the &lt;span class=&quot;math-container&quot;&gt;$min \ max$&lt;/span&gt; game, and &lt;a href=&quot;https://arxiv.org/abs/1701.07875&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wasserstein GANs&lt;/a&gt;, where you focus on a different metric that still has the same global minimum but allows for side by side training completely eliminating the ordering and failure mode, to begin with. On top of this, other work has been done as well, these are just two important examples.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding how they fare against other generative models, like VAEs, there is no &lt;em&gt;one is better than the other&lt;/em&gt;. The recent empirical success of GANs is why they are so popularly used, but we still see others being used in practice as well.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-11-13T14:45:25.077" LastActivityDate="2019-11-13T14:45:25.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16467" PostTypeId="2" ParentId="16463" CreationDate="2019-11-12T16:27:36.453" Score="2" Body="&lt;p&gt;Yes you can classify people bounding box with object detection. State of the art object detection model have people as one of the class in the object detection, as shown here:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kc7M1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kc7M1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see the image have both object bounding box and people bounding box.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-12T16:27:36.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16468" PostTypeId="2" ParentId="2111" CreationDate="2019-11-12T18:33:23.057" Score="1" Body="&lt;p&gt;A &lt;a href=&quot;https://www.thefreedictionary.com/life&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of life&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The property or quality that distinguishes living organisms from dead organisms and inanimate matter, manifested in functions such as metabolism, growth, reproduction, and response to stimuli or adaptation to the environment originating from within the organism.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The characteristic state or condition of a living organism.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's &lt;a href=&quot;https://www.lexico.com/en/definition/life&quot; rel=&quot;nofollow noreferrer&quot;&gt;another definition&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The condition that distinguishes animals and plants from inorganic matter, including the capacity for growth, reproduction, functional activity, and continual change preceding death.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yet &lt;a href=&quot;https://en.wikipedia.org/wiki/Life&quot; rel=&quot;nofollow noreferrer&quot;&gt;another definition&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Life is a characteristic that distinguishes physical entities that have biological processes, such as signaling and self-sustaining processes, from those that do not, either because such functions have ceased (they have died), or because they never had such functions and are classified as inanimate. Various forms of life exist, such as plants, animals, fungi, protists, archaea, and bacteria.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;AIs (or, in general, computers) do not have a real &lt;a href=&quot;https://en.wikipedia.org/wiki/Metabolism&quot; rel=&quot;nofollow noreferrer&quot;&gt;metabolism&lt;/a&gt;, do not really reproduce, do not respond to stimuli or adapt to &lt;em&gt;new&lt;/em&gt; circumstances (that is, circumstances they have not been programmed to deal with). AI does nothing without human intervention or it lacks real autonomy. In other words, if you do not turn the computer on and you do not program it, it really does nothing. A computer is a useful tool that you can use thanks to electricity. You can plug and unplug it indefinitely, but you cannot kill and revive a living being indefinitely.&lt;/p&gt;&#xA;&lt;p&gt;Even though computers may possess (at least, conceptually) some properties similar to the properties of certain living beings, it does not mean they are living beings. Similarly, airplanes are not birds. &lt;strong&gt;Computers are not living beings&lt;/strong&gt;, but this does not prevent you from drawing a comparison between computers and living beings, provided you are aware of their actual big differences. In fact, many useful AI software is inspired by the behavior of certain living beings or natural processes. For example, ant colony optimization algorithms are based on the behavior of real ants seeking a path between their colony and a source of food.&lt;/p&gt;&#xA;&lt;p&gt;Here's what (biological) &lt;a href=&quot;https://www.britannica.com/science/life&quot; rel=&quot;nofollow noreferrer&quot;&gt;life&lt;/a&gt; looks like.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/53pFY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/53pFY.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-13T04:05:05.190" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16470" PostTypeId="2" ParentId="16465" CreationDate="2019-11-12T21:16:45.607" Score="2" Body="&lt;p&gt;To put it simply GANs suffer from a problem of uneven learning rate. Imagine the learning rate of a pitcher and hitter if the pitcher gets to a point where they can throw much better than the hitter can hit then the hitter may fall into a 'training pit' as to be unable to ever learn how to hit from the pitcher. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This follows a continues relationship in between the two learning rates where if the pitcher is becoming a much better pitcher at a faster rate they could become too good and make learning impossible for the hitter. So the rate must be 'slowed down' as to ensure the pitcher doesn't ruin the hitter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the cone of the Lipschitz continuity function of either function is outrunning/outpacing the other than the learning for the one who is in front must be slowed down so the other catches up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two runners trying to push each other athletically is another example. If one outpaces the other an injury may occur in the one lagging behind while trying to keep pace this happens, when the adversarial network becomes too good at generating training material that the behind network is not ready to learn with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GAN will do best when the learning rates are adjusted to slow down the fast learner artificially. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statistics will not be bounded correctly if the learning rates are not kept in check similarly to how step size needs to be right to find local minimum and maximum. If the learning is not artificial augmented so both keep relatively same pace getting stuck at local minimum and maximum of the solution space will occur.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-12T21:16:45.607" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16471" PostTypeId="2" ParentId="14003" CreationDate="2019-11-12T22:05:18.027" Score="3" Body="&lt;p&gt;&lt;strong&gt;Spectral Convolution&lt;/strong&gt;&#xA;In a spectral graph convolution, we perform an Eigen decomposition of the Laplacian Matrix of the graph. This Eigen decomposition helps us in understanding the underlying structure of the graph with which we can identify clusters/sub-groups of this graph. This is done in the Fourier space.&#xA;An analogy is PCA where we understand the spread of the data by performing an Eigen Decomposition of the feature matrix. The only difference between these two methods is with respect to the Eigen values. Smaller Eigen values explain the structure of the data better in Spectral Convolution whereas it's the opposite in PCA.&#xA;ChebNet, GCN are some commonly used Deep learning architectures that use Spectral Convolution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Spatial Convolution&lt;/strong&gt;&#xA;Spatial Convolution works on local neighbourhood of nodes and understands the properties of a node based on its k local neighbours. Unlike Spectral Convolution which takes a lot of time to compute, Spatial Convolutions are simple and have produced state of the art results on graph classification tasks. GraphSage is a good example for Spatial Convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional References: &lt;a href=&quot;https://towardsdatascience.com/tutorial-on-graph-neural-networks-for-computer-vision-and-beyond-part-2-be6d71d70f49&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/tutorial-on-graph-neural-networks-for-computer-vision-and-beyond-part-2-be6d71d70f49&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/graph-convolutional-networks-for-geometric-deep-learning-1faf17dee008&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/graph-convolutional-networks-for-geometric-deep-learning-1faf17dee008&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30939" LastActivityDate="2019-11-12T22:05:18.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16472" PostTypeId="2" ParentId="2111" CreationDate="2019-11-12T22:22:15.737" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;My sense is that, yes, AI (and algorithms in general) constitute a form of &quot;life&quot; in that they are animate, able to respond to stimuli and act on an environment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Algorithms may be &lt;a href=&quot;https://en.wikipedia.org/wiki/Deterministic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;deterministic&lt;/a&gt; (always produce the same output for identical input), and this is not much different from more elementary forms of life (like proteins.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_virus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer viruses&lt;/a&gt; are another form of algorithmic life which typically have the capability of reproduction, copying themselves onto new systems/environments, similar to biological &lt;a href=&quot;https://en.wikipedia.org/wiki/Virus&quot; rel=&quot;nofollow noreferrer&quot;&gt;viruses&lt;/a&gt;.  (Here it's a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Parthenogenesis#Natural_occurrence&quot; rel=&quot;nofollow noreferrer&quot;&gt;parthenogenesis&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Cell_division&quot; rel=&quot;nofollow noreferrer&quot;&gt;mitosis&lt;/a&gt;, where exact copies of the original are formed.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine Learning algorithms can adapt to their environment (increasing fitness), and this applies to both genetic algorithms and other forms of machine learning that can optimize utility in relation to a problem (the environment in which the algorithm is applied.)  &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic process&lt;/a&gt; in particular will produce successive generations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In his novel &lt;a href=&quot;https://en.wikipedia.org/wiki/Valis_(novel)&quot; rel=&quot;nofollow noreferrer&quot;&gt;VALIS&lt;/a&gt;, PKD referred to an idea of god as a &quot;Vast Active Living Intelligence System&quot;.  This is relevant because it's a philosophical idea, as opposed to scientific.  The drafting of that book involved an &lt;a href=&quot;https://en.wiktionary.org/wiki/epiphany&quot; rel=&quot;nofollow noreferrer&quot;&gt;ephiphany&lt;/a&gt; and the drafting of a corresponding &lt;a href=&quot;https://en.wikipedia.org/wiki/Exegesis#Christianity&quot; rel=&quot;nofollow noreferrer&quot;&gt;exegesis&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This idea is controversial and would likely be rejected by the vast majority of scholars and AI ethicists&lt;/strong&gt;, but I'd posit rejection of this notion constitutes a form of biological-prejudice, and, in the case of &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;hypothetical future AGI&lt;/a&gt;, a form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Anthropomorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;anthropomorphic&lt;/a&gt; bias. (I'd go so far as to suggest that &lt;em&gt;not&lt;/em&gt; regarding algorithms as a form of life carries grave risks, in that computing has made active algorithms pervasive, with profound impacts to human experience.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there are no current algorithms I am aware of that have sufficient sentience to warrant having rights, whether &lt;a href=&quot;https://en.wikipedia.org/wiki/Human_rights&quot; rel=&quot;nofollow noreferrer&quot;&gt;human&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Animal_rights&quot; rel=&quot;nofollow noreferrer&quot;&gt;animal&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A note on the term &quot;animate&quot; (adjective): derives from the latin &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.04.0059%3Aentry%3Danima&quot; rel=&quot;nofollow noreferrer&quot;&gt;anima&lt;/a&gt;&lt;/em&gt;, which initially refers to wind and the &quot;breath of life&quot; [see also the Greek &lt;em&gt;&lt;a href=&quot;http://www.perseus.tufts.edu/hopper/morph?l=pneu%3Dma&amp;amp;la=greek&amp;amp;can=pneu%3Dma0#lexicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;pneuma&lt;/a&gt;&lt;/em&gt;. However, the Latin lexicon references animus as &quot;the mind as the seat of thought&quot; and &quot;the rational soul of man&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms can be rational in the strictest sense, and an entire branch of engineering is based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Pneumatics&quot; rel=&quot;nofollow noreferrer&quot;&gt;pneumatics&lt;/a&gt;. Although our algorithms use electrical signals, (as opposed to &lt;a href=&quot;https://en.wikipedia.org/wiki/Gas_flow_computer&quot; rel=&quot;nofollow noreferrer&quot;&gt;pneumatic computing&lt;/a&gt; or &lt;a href=&quot;http://nautil.us/blog/this-early-computer-was-based-on-a-urinal-flush-mechanism&quot; rel=&quot;nofollow noreferrer&quot;&gt;hydraulic computing&lt;/a&gt;,) what it comes down to is that &lt;a href=&quot;https://en.wiktionary.org/wiki/ratiocination&quot; rel=&quot;nofollow noreferrer&quot;&gt;intelligence requires process&lt;/a&gt;, and process requires energy.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machines convert energy into motion or change, and so humans fit the definition. (William Burroughs referred to us as &quot;soft machines.&quot;)  If animals can be thought of as machines, why can't machines be thought of as animals? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/DNA&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA&lt;/a&gt; is a type of &lt;a href=&quot;https://en.wikipedia.org/wiki/Code&quot; rel=&quot;nofollow noreferrer&quot;&gt;encoding&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/RNA&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA&lt;/a&gt; acts on that code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key distinction seems to be the medium in which process exist, where a biological context is used for what we conventionally think of as life.  Algorithms merely utilize different mediums (mechanical &amp;amp; electrical) and may exist in different environments such as the digital. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A note on The &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Soft_Machine#Title_and_structure&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soft Machine&lt;/a&gt;: In this trilogy, Burroughs describes an elaborate alien reproductive process involving numerous alien species acting as surrogates at various stages in the cycle. (It is a description of a 3-dimensional representation of a higher dimensional process.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has precedent in biology where, for instance, the lifecycle of a seed may involve a period inside the digestive system of an animal. &lt;a href=&quot;https://en.wikipedia.org/wiki/Pollination&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pollination&lt;/a&gt; is another example, where a surrogate species plays a critical role. Viruses require host organisms to reproduce and spread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not out of bounds to regard the lifecycle of current active algorithms as involving humans to fill in the capability gaps.  Here humans are surrogates in bringing information &quot;to life&quot;.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-11-13T03:19:33.160" LastActivityDate="2019-11-13T03:19:33.160" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16473" PostTypeId="1" CreationDate="2019-11-13T00:10:15.633" Score="2" ViewCount="44" Body="&lt;p&gt;This is a follow-up question from my previous post &lt;a href=&quot;https://ai.stackexchange.com/questions/16444&quot;&gt;here&lt;/a&gt; about explosion detection. I gathered a dataset of explosions. As I'm new to Deep Learning in Keras, I'm trying to see what architecture best suits this problem, given that here we have a cloud of smoke/fire as opposed to an object. Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, I've learned about Faster RCNN or RetinaNet, but that is mostly for &lt;strong&gt;object&lt;/strong&gt; detection. Is it going to be better than say a basic ResNet50? And here real-time prediction requirements are not an issue. So shall I assume a heavier model (e.g. NASNet Large or a Resnet-152 model) is better than a basic ResNet-50 model?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-11-13T17:52:17.407" LastActivityDate="2019-11-13T17:52:17.407" Title="Which deep neural networks are appropriate for the detection of bombs?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;keras&gt;&lt;deep-neural-networks&gt;&lt;models&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16476" PostTypeId="2" ParentId="2917" CreationDate="2019-11-13T05:34:46.627" Score="0" Body="&lt;p&gt;It might be of note to comment/update that the SuperGLUE benchmark, which is a suite of common sense reasoning tasks, incorporates the aforementioned Winnograd Schema challenge, among other tests that are said to be reflective of natural language &lt;strong&gt;understanding&lt;/strong&gt; (as opposed to simply its processing or the optimal statistical generation of language). The most recent result by Google (T5) has reached parity with the Human Baseline in terms of the average performance across all tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The leaderboard is available below, and the site includes subscores for each of the challenges:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://super.gluebenchmark.com/leaderboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://super.gluebenchmark.com/leaderboard&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16803" LastActivityDate="2019-11-13T05:34:46.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16478" PostTypeId="1" CreationDate="2019-11-13T06:20:41.290" Score="2" ViewCount="74" Body="&lt;p&gt;In recent years, we have seen quite a lot of impressive display of Deep Neural Network (DNN), as demonstrated most famously by AlphaGo and its cousin programs.&lt;/p&gt;&#xA;&lt;p&gt;But if I understand correctly, &lt;a href=&quot;https://ai.stackexchange.com/a/98/16291&quot;&gt;deep neural network is just a normal neural network with a lot of layers&lt;/a&gt;. We know about the principles of the neural network since the 1970s (?), and a deep neural network is &lt;em&gt;just&lt;/em&gt; the generalization of a one-layer neural network to many.&lt;/p&gt;&#xA;&lt;p&gt;From here, it doesn't seem like the recent explosion of DNN has anything to do with a &lt;strong&gt;theoretical&lt;/strong&gt; breakthrough, such as some new &lt;strong&gt;revolutionary&lt;/strong&gt; learning algorithms or particular topologies that have been theoretically proven effective. It seems like DNN successes can be entirely (or mostly) attributed to better hardware and more data, and not to any new theoretical insights or better algorithms.&lt;/p&gt;&#xA;&lt;p&gt;I would go even as far as saying that there are no new theoretical insights/algorithms that contribute significantly to the DNN's recent successes; that the most important (if not all) theoretical underpinnings of DNNs were done in the 1970s or prior.&lt;/p&gt;&#xA;&lt;p&gt;Am I right on this? How much &lt;strong&gt;weight&lt;/strong&gt; (if any) do &lt;strong&gt;theoretical&lt;/strong&gt; advancements have in contributing to the recent successes of DNNs?&lt;/p&gt;&#xA;" OwnerUserId="16291" LastEditorUserId="2444" LastEditDate="2021-01-20T23:24:08.313" LastActivityDate="2021-01-20T23:24:08.313" Title="Is there anything theoretically revolutionary about Deep Neural Networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;computational-learning-theory&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="16479" PostTypeId="1" AcceptedAnswerId="16532" CreationDate="2019-11-13T06:26:10.667" Score="2" ViewCount="85" Body="&lt;p&gt;In the field of adversarial machine learning, machine learning models are vulnerable to attacks both on the test and training data set. However, how does the attacker get access to these datasets? How do these datasets get manipulated/tampered with?&lt;/p&gt;&#xA;" OwnerUserId="31240" LastActivityDate="2019-11-15T03:38:07.430" Title="In adversarial machine learning, how does an attacker have access to the test and training dataset in order to poison it?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;datasets&gt;&lt;adversarial-ml&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16481" PostTypeId="1" AcceptedAnswerId="16483" CreationDate="2019-11-13T08:03:14.713" Score="4" ViewCount="61" Body="&lt;p&gt;A neural network can apparently be denoted as &lt;span class=&quot;math-container&quot;&gt;$N_{t,n,\sigma,L}$&lt;/span&gt;. What do these subscripts  &lt;span class=&quot;math-container&quot;&gt;$t, n, \sigma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; mean? Could you link me to a paper, article or webpage with an explanation for this? &lt;/p&gt;&#xA;" OwnerUserId="31125" LastEditorUserId="2444" LastEditDate="2019-11-13T12:36:59.670" LastActivityDate="2019-11-13T12:36:59.670" Title="What do the subscripts mean in $N_{t,n,\sigma,L}$?" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;definitions&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16482" PostTypeId="2" ParentId="16479" CreationDate="2019-11-13T08:32:45.063" Score="1" Body="&lt;p&gt;They don't have acces to the original training or test dataset. Machine learning environments are build on the premise of a benign environment. The models are trained on real data (real inputs). When someone sends a made up input (fake input) it is very easy to fool the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is used for example in image recognition. Imagine a fotograph of a panda. the model may correctly identify this fotograph as a panda. With knowledge of the model you can now alter some pixels in the fotograph. To the human Eye, the fotographs will appear exactly the same, but the model can be fooled to believe the fotograph is actually of a gibbon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all done after the training of the model and doesn't require the original datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more info, visit this site:&#xA;&lt;a href=&quot;https://medium.com/@ml.at.berkeley/tricking-neural-networks-create-your-own-adversarial-examples-a61eb7620fd8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@ml.at.berkeley/tricking-neural-networks-create-your-own-adversarial-examples-a61eb7620fd8&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-11-13T08:32:45.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16483" PostTypeId="2" ParentId="16481" CreationDate="2019-11-13T08:34:51.033" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1410.1141.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a paper with the mathematical definition of each term:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let N&lt;sub&gt;t,n,σ,L&lt;/sub&gt; be all target functions that can be implemented using a&#xA;  neural network of depth t, size n, activation function σ, and when we&#xA;  restrict the input weights of each neuron to be |w|&lt;sub&gt;1&lt;/sub&gt; + |b| ≤ L.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22301" LastActivityDate="2019-11-13T08:34:51.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16484" PostTypeId="2" ParentId="16479" CreationDate="2019-11-13T08:50:21.343" Score="0" Body="&lt;p&gt;In adverserial machine learning, someone (program or human) attempts to fool an existing model with a malicious input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best human example would be an optical illusion. The human brain's model for image processing starts outputting wrong information when looking at an optical illusion. So in the end we see wrong colour, shape, etc. In this case, the optical illusion would be considered as the malicious input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can trick the human brain’s model through images created with trial and error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you just have the trained model at hand, you don’t have to know the data it has been trained with. You just need to be able to input a value to the model and get the output.&lt;/p&gt;&#xA;" OwnerUserId="31252" LastActivityDate="2019-11-13T08:50:21.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16485" PostTypeId="2" ParentId="2111" CreationDate="2019-11-13T10:18:49.353" Score="1" Body="&lt;p&gt;&lt;strong&gt;Mind the hardware:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are different definitions of what life (synonymously used with 'organism' here (source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Life&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia: Life&lt;/a&gt;) is, e.g.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;All types of organisms are capable of reproduction, growth and development, maintenance, and some degree of response to stimuli. (Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Organism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia: Organism&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;they all have one thing in common: &#xA;&lt;strong&gt;they require physical matter!&lt;/strong&gt; &#xA;In contrast, to ask whether AI is alive is comparable to asking whether the human mind is alive. It is per definition not! Therefore, the question needs to be extended to include the hardware. It is rather 'are machines/computers alive' or 'do machines/computers have the potential to be considered alive'?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We are talking about agents and most likely robots:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And more specifically any machine/computer to be potentially considered alive will most likely need to be an agent,as it needs to interact with its environment (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia: Intelligent agent&lt;/a&gt; for a description of agents in Computer Science).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, any potentially intelligent machine/computer needs most likely to be a robot due to the strong emphasis on physical processes, incl. some kind of exchange with the physical environment  (perception or manipulation of it), which our common definitions of life carry.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some requirements for life are easy to fulfill while others are not:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the requirement for life forms to maintain and reproduce their entities any machine/computer to be considered intelligent will need to be able to physically maintain and reproduce, i.e. assemble hardware. If you think of an intelligent robot assembling another robot that might sound to be very far away from reality. However, the definition might include indirect reproduction e.g. using an automated hardware production facility. Certainly this is not the direct reproduction that we know from current living beings but it might be considered an indirect way to reproduce. Which, however, is certainly far away from current reality too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly you could think of 'maintain' as taking care of the physical need to supply itself with electricity. Any machine with a solar panel easily fulfills this requirement in a similar way plants do.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While machines/computers considered alive do not need to have any artificial intelligence it is an easy way to fulfill the requirement of 'development':&#xA;Sub-symbolic AI learns from data which is a form of (non-physical, i.e. software-related) development. Just like humans and other animals learn from data that comes in through one of their senses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Give it time:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize: current machines/computers certainly do not fulfill the requirements usually being considered 'life'. And especially the requirement to (physically) maintain itself and reproduce will be longstanding unfulfilled requirements. However, considering that the homo sapiens has been on this planet for about 150,000 years we might just need to give it more time. It took about 1 billion years for the first living beings to develop on planet earth (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Earliest_known_life_forms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia: Earliest known life forms&lt;/a&gt;). So it is a bit early to make a call on machines/computers which in the case of computers have been around for not even a century. Let's see where we stand in 1000, 1000000 or 10000000 years from now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However, the definitions might change anyway:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, it is important to note that the definition of life is closely built on what we know as current carbon-based life forms. And it could very well be adjusted in the light of machines/computer further developing. For example the aspects of physical reproduction might be one aspect to be dropped (just speculating here). So maybe we do not even need to wait a billion years but might have machines/computers considered 'alive' already in 200 or 500 years. Compared to the time biological life took to develop that would still be very rapid. &lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2019-11-13T10:18:49.353" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16486" PostTypeId="2" ParentId="16478" CreationDate="2019-11-13T10:50:18.570" Score="1" Body="&lt;p&gt;The first neural network machine was the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_neural_analog_reinforcement_calculator&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;stochastic neural analog reinforcement calculator&lt;/em&gt; (SNARC)&lt;/a&gt;, built in the 1950s. As you can see, it's pretty old. After that, there were several advances regarding backpropagation and the vanishing gradient problem. However, the ideas itself are not novel. Simply put, we have the data and processing power today that we did not have back then.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could look at &lt;a href=&quot;https://www.wikiwand.com/en/Timeline_of_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia timeline&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="31252" LastEditorUserId="2444" LastEditDate="2019-11-13T12:46:07.087" LastActivityDate="2019-11-13T12:46:07.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16487" PostTypeId="1" AcceptedAnswerId="16491" CreationDate="2019-11-13T11:17:47.837" Score="5" ViewCount="728" Body="&lt;p&gt;I have two trained models. One is using a LinearSVC algorithm and is trained on numerical data from medical examination from patients with diabetic retinopathy. The second one is a neural network trained on images of retina scans from patients with the same disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The models predict if the patient has retinopathy or not. Both are written using Python 3.6 and Keras and have accuracy around 0.84.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to combine those two models in any way to increase the accuracy of predictions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure in what way it could be achievable as they are using a different kinds of data. I have tried using ensembling methods but didn't get better scores with them.&lt;/p&gt;&#xA;" OwnerUserId="31256" LastEditorUserId="2444" LastEditDate="2019-11-13T12:39:45.727" LastActivityDate="2019-11-14T06:29:25.740" Title="How do I combine models trained on different data to increase classification accuracy?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;image-recognition&gt;&lt;keras&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16489" PostTypeId="2" ParentId="16457" CreationDate="2019-11-13T12:18:24.023" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In deep reinforcement learning for portfolio optimization, many researchers (Xiong at al. for example) use historical market data for model training. The resulting MDP dynamics is of course completely deterministic (if historical prices are used as states) and there's no real sequentiality involved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Whilst I cannot comment on the specific financial model, I think it unlikely that these researchers would apply RL without there being a sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More likely in my opinion, the historic data feed forms a major part of the environment, but that there are still time steps and a state which depends on an agent's actions. For instance, in a trading simulation, provided the values of trades are below levels that would significantly alter the market itself, it may be a reasonable approximation to use the history of prices and other factual information that progress like a recording, plus have state include the agent's current portfolio of investment and working funds. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What's the point of using DDPG for variance reduction when the underlying MDP used for training has deterministic dynamics? Why not simply use the Reinforce algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Variance in returns occurs due to stochastic dynamics (if they are present) &lt;em&gt;and&lt;/em&gt; the behaviour policy. You cannot use any RL &lt;em&gt;control&lt;/em&gt; algorithm with a deterministic behaviour policy*. It would never gain any data that allowed it to assess alternative behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in REINFORCE, which is on-policy (the behaviour policy and target policy are the same), and typically starts with near equiprobable action choices, there is high variance. It could be very high when measured over a long episode with many action choices. In basic REINFORCE, the variance is not controlled for, and training uses individual Monte Carlo style returns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In DDPG, which is off-policy (the target policy is deterministic**, the behaviour policy is stochastic), there is still variance, but it is much reduced with the actor-critic mechanism, plus can be constrained by choice of noise function that relates the behaviour policy to the target policy. In addition, updates to policy and value functions can be made independently of episode ends, which can significantly speed up learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine what difference this makes for any experiment, you would need to compare the two algorithms on the same task. In practice DDPG will significantly out-perform REINFORCE on many tasks, including those with deterministic environments. However, there might be specific combinations where the simplicity of REINFORCE wins out, if only because there are less hyperparameters to tune.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On one point:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;under an almost-deterministic policy our variance will be quite small&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's true, but how do you get to that stage of training with REINFORCE? It is by testing and working through more stochastic policies, which is what will take time. Your statement only applies to REINFORCE when the control problem is nearly complete, or if you take a short-cut and force the policy function into &lt;em&gt;what you hope&lt;/em&gt; is a near-optimal policy. In which case you are engaging in a form of variance reduction - it may even work for some scenarios, but is likely not as general as applying an Actor-Critic algoirthm.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Actually technically you can if the environment is stochastic in the right way so that you effectively explore all state/action combinations. But we are talking about deterministic environments here, and obviously the stochastic environment would introduce variance in returns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** The target policy changes over time, and this introduces non-stationarity and bias for the critic component to deal with, but not technically variance.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-11-13T12:42:27.143" LastActivityDate="2019-11-13T12:42:27.143" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16490" PostTypeId="1" AcceptedAnswerId="16499" CreationDate="2019-11-13T12:40:50.143" Score="5" ViewCount="136" Body="&lt;p&gt;Deep fakes are a growing concern: the ability to credibly alter a video may have great (negative) impacts on our society. It is so much of a concern, that the biggest tech companies launched a specific challenge: &lt;a href=&quot;https://deepfakedetectionchallenge.ai/&quot; rel=&quot;noreferrer&quot;&gt;https://deepfakedetectionchallenge.ai/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from what I understand, most deep fake generation techniques rely on the use of adversarial models. One model generates a new image, while another model tries to detect if the image is doctored or not. Both models &quot;learn&quot; from being confronted with the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, if a good deep fake detection model emerges (from the previous challenge, or not), wouldn't it be rendered useless almost instantly by learning from it in an adversarial setting?&lt;/p&gt;&#xA;" OwnerUserId="22654" LastEditorUserId="2444" LastEditDate="2019-11-13T12:49:36.817" LastActivityDate="2019-11-13T22:49:51.793" Title="Isn't deep fake detection bound to fail?" Tags="&lt;generative-adversarial-networks&gt;&lt;adversarial-ml&gt;&lt;deepfakes&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16491" PostTypeId="2" ParentId="16487" CreationDate="2019-11-13T12:48:06.347" Score="2" Body="&lt;p&gt;You can try using a multi-input model. &lt;a href=&quot;https://ai.stackexchange.com/questions/16153/are-there-well-established-ways-of-mixing-different-inputs-e-g-image-and-numbe&quot;&gt;Here&lt;/a&gt; is a recent post with a similar discussion, with the required architecture defined in the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of combining the separate models, you can create a model which uses image and numerical data side by side. Keras allows you to use different types of data using multi input structure via &lt;a href=&quot;https://keras.io/getting-started/functional-api-guide/#multi-input-and-multi-output-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;functional API&lt;/a&gt;. And then you can combine them to create a single machine learning model. Basic idea is like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6DBCf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6DBCf.png&quot; alt=&quot;this&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image was taken from &lt;a href=&quot;https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where you can find further details and the code implementation as well. Actually, the discussion in the introduction of that post is almost the same as your question: What to do when the inputs are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Numeric/continuous values, such as age, heart rate, blood pressure&lt;/li&gt;&#xA;  &lt;li&gt;Categorical values, including gender and ethnicity &lt;/li&gt;&#xA;  &lt;li&gt;Image data, such as any MRI, X-ray, etc.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also, section 5.1 of &lt;a href=&quot;https://machinelearningmastery.com/keras-functional-api-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; blogpost details the same process.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-11-14T06:29:25.740" LastActivityDate="2019-11-14T06:29:25.740" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16492" PostTypeId="1" CreationDate="2019-11-13T12:52:50.443" Score="0" ViewCount="64" Body="&lt;p&gt;I was discussing with a friend whether current AI does anything remotely similar to 'thinking' and he argued that AIs that play games must think up strategies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While &lt;em&gt;thinking&lt;/em&gt; may not be precisely defined, my understanding of algorithms like &lt;a href=&quot;https://openai.com/five/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI&lt;/a&gt; was that they just minimize a very non-convex objective, but still play the game based on examples, and not by coming up with intentional strategies. Is my understanding incorrect?&lt;/p&gt;&#xA;" OwnerUserId="12996" LastEditorUserId="2444" LastEditDate="2019-11-13T12:54:49.090" LastActivityDate="2019-11-13T18:56:39.673" Title="Do algorithms like OpenAI's &quot;think up strategies&quot;?" Tags="&lt;deep-learning&gt;&lt;philosophy&gt;&lt;game-ai&gt;&lt;agi&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16493" PostTypeId="1" AcceptedAnswerId="16494" CreationDate="2019-11-13T13:12:08.167" Score="2" ViewCount="86" Body="&lt;p&gt;In the case of single shot detection of point clouds, that is the point cloud of an object is taken only from one camera view without any registration. Can a Convolutional Network estimate the 6d pose of objects (initially primitive 3D objects -- cylinders, spheres, cuboids)?&lt;/p&gt;&#xA;&lt;p&gt;The dataset will be generated by simulating a depth sensor using a physics engine (ex:&lt;a href=&quot;http://gazebosim.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;gazebo&lt;/a&gt;) and primitive 3D objects are spawned with known 6d pose as ground truth. The resulting training data will be the single viewed point cloud of the object with the ground truth label (6d pose)?&lt;/p&gt;&#xA;" OwnerUserId="31259" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-13T20:02:11.840" Title="Pose estimation using CNNs on Point clouds" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;&lt;object-detection&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16494" PostTypeId="2" ParentId="16493" CreationDate="2019-11-13T18:19:08.567" Score="0" Body="&lt;p&gt;The answer is yes this is possible and here are the papers where they do almost exactly the same project you are describing above. Although none of the bellow combine gazebo, single point/single shot, 6D-pose and CNNs. In order to use synthetic data to train a model that works on real data.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8757773&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pose Estimation by Key Points Registration in Point Cloud&#xA;&lt;/a&gt; (2019) By Weiyi ZHANG, Chenkun QI This paper uses gazebo to test validate the model they made.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1703.09312&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics&lt;/a&gt; (2017) by Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2018/html/Tekin_Real-Time_Seamless_Single_CVPR_2018_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Real-Time Seamless Single Shot 6D Object Pose Prediction&lt;/a&gt; (2018) by Bugra Tekin, Sudipta N. Sinha, Pascal Fua&#xA;This paper sounds like exactly what you want to do.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_iccv_2017/html/Kehl_SSD-6D_Making_RGB-Based_ICCV_2017_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again&lt;/a&gt; (2017) by Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, Nassir Navab &#xA;This paper is also referenced by them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The model will be able to be trained but how effect a model trained on the synthetic data will be able to properly function on real data will be the challenge. &lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-13T18:19:08.567" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16495" PostTypeId="2" ParentId="16492" CreationDate="2019-11-13T18:40:20.700" Score="1" Body="&lt;p&gt;The key here is &lt;strong&gt;think up strategies&lt;/strong&gt;. If we define this as examining, creating a hypothesis, and testing it as strategizing then yes AI has the ability to strategize. It can examine other users' games, quantifies actions that correlated with victory then test if it gains victory by doing those actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strategy by definition is: a plan of action or policy designed to achieve a major or overall aim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI can not classically plan a series of actions designed to achieve a major victory. Instead, it learns the right strategy by testing simulated scenarios, like someone who thinks about the consequences before doing them, but the AI actually has the opportunity to play the game hundreds of times, in order to learn the correct strategy. Similar to Bill Murray in the movie &lt;a href=&quot;https://en.wikipedia.org/wiki/Groundhog_Day_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Groundhog Day&lt;/a&gt;, learning the ideal day to live. The AI can strategize by experiencing the game over and over until it fine-tunes what an ideal game should be and has seen enough examples of games to not be outwitted by gimmicky strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, AI can strategize, just in a way fairly different than people.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="2444" LastEditDate="2019-11-13T18:56:39.673" LastActivityDate="2019-11-13T18:56:39.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16496" PostTypeId="1" CreationDate="2019-11-13T19:35:12.587" Score="5" ViewCount="164" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5rN5a.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5rN5a.jpg&quot; alt=&quot;image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay.&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to build an AI like that displayed in the diagram: when given a training set of images with their corresponding node maps of face/nose posture, and an image with a missing &lt;strong&gt;section&lt;/strong&gt; (just a gap) with a node map, I would like it to reconstruct the initial image. My thoughts immediately went to GANs for this, but after some searching, the closest I could find were:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Face recreation without context/not filling gaps, just following pose (DeepFake)&lt;/li&gt;&#xA;&lt;li&gt;Filling gaps in images, but with no node reference&lt;/li&gt;&#xA;&lt;li&gt;Filling gaps from reference drawings/mappings, but with no way to provide sample images&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would like to hear about any implementations of such an algorithm, if possible optimised for faces, and if none exists, I would like to hear of how I would go about altering the generator of the GAN to work with the context/gap-fill bit (e.g a paper which talks about this idea, but doesn't implement it). Any guidance on the NN that is best for this type of task is also appreciated.&lt;/p&gt;&#xA;" OwnerUserId="27018" LastEditorUserId="27018" LastEditDate="2019-11-20T07:23:24.197" LastActivityDate="2019-11-20T07:23:24.197" Title="Context-based gap-fill face posture-mapper GAN" Tags="&lt;ai-design&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;implementation&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16499" PostTypeId="2" ParentId="16490" CreationDate="2019-11-13T20:05:29.650" Score="4" Body="&lt;p&gt;Not necessarily it depends on the function of the problem space for both the GANs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be a better pitcher because the threshold of reaction time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't yet know if a maximum threshold on realistic fake image generation exists or if a threshold on detection exists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As both reach near perfect accuracy it could be that the amount of nodes needed to detect a nearly perfect generated image from real image is more neurons than atoms in the universe, or conversely the amount of nodes needed to generated a nearly perfect image could reach impossible port potions we won't know until we continue to make better and better networks that close in on the boundary of generation and detection of &lt;strong&gt;real&lt;/strong&gt; vs &lt;strong&gt;fake&lt;/strong&gt; images from a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;Let's imagine this problem one adversary edits an image with a colored line of pixels the goal hide the line by editing the image, the student is responsible with finding the line after the adversary changes the image. The problem can become &lt;strong&gt;infinitely difficult&lt;/strong&gt; change all pixels to be the color of the line. The line is impossible to find the adversary always wins, &lt;strong&gt;if&lt;/strong&gt; it finds this solution meaning it is in its reachable problem space based on its hardware capabilities and its learning model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep fake detection is not bound to fail because the limit on the effectiveness of a generative model &lt;strong&gt;may&lt;/strong&gt; have a steeper limit than a discriminator at near optimal performance. I have not seen any paper about this specifically and in fact I believe the discriminator has a more difficult job in most cases I just disagree with deciding at this moment that the detectors are doomed. The combination of creating real images motion and sound perfectly in sink is not a trivial problem, in some scenarios it is basically impossible. &lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="30365" LastEditDate="2019-11-13T22:49:51.793" LastActivityDate="2019-11-13T22:49:51.793" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16500" PostTypeId="2" ParentId="15616" CreationDate="2019-11-13T21:43:17.900" Score="0" Body="&lt;p&gt;Depending on the number of features you have you might want to try reducing features in order to reduce overfitting and speed up your model. I assume you are using proper regularization as well. PCA may be an option to help if time is the main issue, &lt;a href=&quot;https://medium.com/@sadatnazrul/the-dos-and-donts-of-principal-component-analysis-7c2e9dc8cc48&quot; rel=&quot;nofollow noreferrer&quot;&gt;medium PCA&lt;/a&gt;. As the medium article states PCA can be used to reduce the dimensionality of the data while keeping retaining variance. If you run a random forest on the different sets and the feature importance ranking is at all different that is a big issue and the number of features should be reduced. Ideally, the features should be cropped till they have the same (structure/feature importance) but slightly different weights associated with those features. Here is a &lt;a href=&quot;https://towardsdatascience.com/feature-selection-using-random-forest-26d7b747597f&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; demonstrating feature selection.&lt;/p&gt;&#xA;&lt;p&gt;Here are some publications I believe are relevant to your problem. More in line with your desire to try to validate after one sweep. You may want to look up applications of LSTM or Yolo to your model I have a feeling these technologies will help direct your final decisions on what to do.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/771195&quot; rel=&quot;nofollow noreferrer&quot;&gt;Subspace regularization method for the single-trial estimation of evoked potentials&lt;/a&gt; (1999) by P.A. Karjalainen, J.P. Kaipio, A.S. Koistinen, M. Vauhkonen&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1712.03847&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Quadratic Penalties in Elastic Weight Consolidation&lt;/a&gt; (2017) by Ferenc Huszár&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1015367&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parameter space exploration with Gaussian process trees&lt;/a&gt; (2014) by  Robert B. Gramacy   University of California, Santa Cruz, CA&#xA;Herbert K. H. Lee   University of California, Santa Cruz, CA&#xA;William G. Macready&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1707.06799&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks&lt;/a&gt; (2017) by Nils Reimers, Iryna Gurevych&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1804.01825&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evaluating Hospital Case Cost Prediction Models Using Azure Machine Learning Studio&lt;/a&gt; (2018) by Alexei Botchkarev -evaluation of regression performance&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="36737" LastEditDate="2021-03-31T22:20:14.917" LastActivityDate="2021-03-31T22:20:14.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16501" PostTypeId="2" ParentId="16448" CreationDate="2019-11-13T22:22:49.000" Score="7" Body="&lt;p&gt;This may be a much simpler explanation than you're looking for, but in &lt;a href=&quot;https://www.youtube.com/watch?v=VwVg9jCtqaU&quot; rel=&quot;noreferrer&quot;&gt;Machine Learning Zero to Hero&lt;/a&gt;, Google engineer Laurence Moroney summarized it in a way that I thought was brilliant. Paraphrasing from a presentation slide:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In traditional programming, you input rules and data and the program outputs answers. In machine learning, you input data and answers and the program outputs rules.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There's an algebra-like symmetry to this. And the program doesn't even know what it's coming up with rules for. It just randomly evolves the rules until the data produces the correct answers. You can then take those rules, apply them to different data, and hopefully get correct answers.&lt;/p&gt;&#xA;" OwnerUserId="27054" LastActivityDate="2019-11-13T22:22:49.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16502" PostTypeId="1" AcceptedAnswerId="16504" CreationDate="2019-11-13T22:33:44.617" Score="2" ViewCount="538" Body="&lt;p&gt;I've been researching Adversarial Machine Learning and I know that &lt;a href=&quot;https://www.slideshare.net/DavidDao1/causative-adverserial-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;causative attacks&lt;/em&gt;&lt;/a&gt; are when an attacker manipulates training data. An &lt;a href=&quot;https://www.slideshare.net/DavidDao1/causative-adverserial-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;exploratory attack&lt;/em&gt;&lt;/a&gt; is when the attacker wants to find out about the machine learning model. However, there is not a lot of information on how an attacker can manipulate only training data and not the test data set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read about scenarios where the attacker performs an exploratory attack to find out about the ML model and then perform malicious input in order to tamper with the training data so that the model gives the wrong output. However shouldn't such input manipulation affect both the test and training data set? How does such tampering only affect the training data set and not the test data set?&lt;/p&gt;&#xA;" OwnerUserId="31240" LastEditorUserId="2444" LastEditDate="2019-11-14T14:15:32.147" LastActivityDate="2019-11-14T14:15:32.147" Title="What are causative and exploratory attacks in Adversarial Machine Learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16503" PostTypeId="2" ParentId="16448" CreationDate="2019-11-13T22:53:17.977" Score="2" Body="&lt;p&gt;AI has been redefined recently to machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All programming except machine learning (and we'll come back to this) is embodying human knowledge in terms a computer can follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EG A text editor has user interface rules, user expectations, a contract with the OS that it has to follow. A programmer puts it all together. This applies to text editors, expert medical systems, banking software, accounting software (and the programmer needs to know accounting to program it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning is training software with data and outputs allowing it to determine the link between them. No human knowledge. Nor can it explain what it is doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course they actually work far better when human knowledge surrounds them as part of their data. A AI that routes incoming invoices etc works better when told where things should actually go (accounts payable).&lt;/p&gt;&#xA;" OwnerUserId="31272" LastActivityDate="2019-11-13T22:53:17.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16504" PostTypeId="2" ParentId="16502" CreationDate="2019-11-13T23:07:51.593" Score="3" Body="&lt;p&gt;When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a guess on what is in the picture and then you click if it got it right or not. If you continue to input images and lie to it it will obviously get worse and worse if they use the user input to add to the test set. Most people are careful and don't mess around with mixing new data into the testing sample. If they did something like mixed the user input training and test then resampled something like that could occur but most people don't do that. It's bad practice and even worse than leaving your NN open to tampering from malicious user input. Information isn't really &lt;strong&gt;added&lt;/strong&gt; to the &lt;strong&gt;knowledge&lt;/strong&gt; in the model till it is fed into the model and backpropagation occurs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An exploratory attack is sending tons of inquiries to the model to gain information about the data set they have built into the model even to the point of extracting data about individuals pieces of data that are built into the model. Then, with this information, they could try to reconstruct the data set. They could attempt to trick the network by sending strange generated inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://people.eecs.berkeley.edu/~tygar/papers/SML2/Adversarial_AISEC.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial Machine Learning&lt;/a&gt; (2011), by Ling Huang et al., in section 2, the authors define these terms, under the category &lt;strong&gt;influence&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;Influence&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Causative&lt;/strong&gt; - Causative attacks alter the training process through influence over the training data.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Exploratory&lt;/strong&gt; - Exploratory attacks do not alter the training process but use other techniques, such as probing the detector, to discover information&#xA;  about it or its training data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They also provide other related definitions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;Security violation&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Integrity&lt;/strong&gt; - Integrity attacks result in intrusion points&#xA;  being classified as normal (false negatives). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Availability&lt;/strong&gt; - Availability attacks cause so many classification errors, both false negatives and false positives, that the system becomes effectively unusable.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Privacy&lt;/strong&gt; - In a privacy violation, the adversary obtains information from the learner, compromising the secrecy or privacy of the system’s users. &lt;/p&gt;&#xA;  &#xA;  &lt;h3&gt;Specificity (a continuous spectrum)&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Targeted&lt;/strong&gt; - In a targeted attack, the focus is on a single or small set of target points. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Indiscriminate&lt;/strong&gt; - An indiscriminate adversary has a more flexible goal that involves a very general class of points, such as “any false negative.”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="2444" LastEditDate="2019-11-14T14:14:10.533" LastActivityDate="2019-11-14T14:14:10.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16505" PostTypeId="1" AcceptedAnswerId="17182" CreationDate="2019-11-14T00:01:45.770" Score="1" ViewCount="33" Body="&lt;p&gt;I am looking to train a bipedal robot using unity as a scape with a genetic algorithm. I will import the CAD into unity so the hardware is exact. My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is Unity physics accurate enough to train a neural network that will perform in the real world?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I optimize the network using reinforcement learning in the real world (after trained in scape)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am looking to use air muscles for my build. If the physics aren’t exactly right in unity (elasticity, max length, torque) will the bot still perform in the real world?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any other programs that would be better than unity to train a robot inside a scape?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any other approaches or new ideas on how to train the bot more efficiently would be greatly appreciated.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4744" LastActivityDate="2019-12-20T09:33:54.933" Title="Training methods for bipedal robot" Tags="&lt;reinforcement-learning&gt;&lt;genetic-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16506" PostTypeId="1" AcceptedAnswerId="16508" CreationDate="2019-11-14T01:06:50.203" Score="3" ViewCount="59" Body="&lt;p&gt;I am wondering how to calculate the size of a 3d object in an image without knowing the focal length of the camera but the distance from the camera to the object. &lt;/p&gt;&#xA;" OwnerUserId="31273" LastEditorUserId="2444" LastEditDate="2019-11-14T14:09:19.727" LastActivityDate="2019-11-14T14:09:19.727" Title="How to calculate the size of a 3d object from an image?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16507" PostTypeId="1" CreationDate="2019-11-14T01:40:36.587" Score="2" ViewCount="114" Body="&lt;p&gt;I am working on an assignment problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; agents &lt;span class=&quot;math-container&quot;&gt;$A_1, \dots A_K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; tasks &lt;span class=&quot;math-container&quot;&gt;$T_1, \dots T_N$&lt;/span&gt;. Each task has a certain time &lt;span class=&quot;math-container&quot;&gt;$t(T_i)$&lt;/span&gt; to be completed and each agent has a matching (or affinity) value associated with each task &lt;span class=&quot;math-container&quot;&gt;$M_{A_j}(T_i), \forall i, j$&lt;/span&gt;. The goal is to assign agents to tasks, such that the matching value is maximized and the overall time to complete the tasks is minimized. Moreover, an agent can be assigned to multiple tasks. However, an agent cannot start a new task before finishing the previous one.I want to solve it with GA+MOA*algorithm What would be an admissible heuristic function?&lt;/p&gt;&#xA;" OwnerUserId="31274" LastEditorUserId="31274" LastEditDate="2019-12-10T01:18:24.650" LastActivityDate="2019-12-10T01:18:24.650" Title="How can I assign agents to tasks based on time and affinity?" Tags="&lt;ai-design&gt;&lt;optimization&gt;&lt;a-star&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16508" PostTypeId="2" ParentId="16506" CreationDate="2019-11-14T04:35:47.083" Score="2" Body="&lt;p&gt;There are hundred of papers on this task some older than I am! Normally this is done by trying to form a box shape around the image than estimate the volume. This task is typically done with multiple images so the two can generate a more clear picture of the size of the object than one image alone. An object could be 'infinitely' large but its mass could be behind the surface you can see with the picture. With the height and length dimensions extracted from the image and with the distance from the camera calculating the size of said surface is fairly easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/52246874/how-to-use-camera-parameters-to-calculate-world-distance-in-cm-between-pixels&quot; &gt; Stack overflow about problem &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2018/html/Qi_Frustum_PointNets_for_CVPR_2018_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Frustum PointNets for 3D Object Detection From RGB-D Data&lt;/a&gt; (2018) By Charles R. Qi, Wei Liu, Chenxia Wu, Hao Su, Leonidas J. Guibas&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/6126342&quot; rel=&quot;nofollow noreferrer&quot;&gt;From contours to 3D object detection and pose estimation&lt;/a&gt; (2017) By Nadia Payet Sinisa Todorovic&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-14T04:35:47.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16509" PostTypeId="1" CreationDate="2019-11-14T07:30:21.787" Score="5" ViewCount="480" Body="&lt;p&gt;I am solving many sequence-to-sequence prediction problems using RNN/LSTM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What type of evaluation metrics can be used for sequence prediction problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One metric is the mean squared error (MSE) that we can give as a parameter during the training model. Currently, the accuracy of my sequence-to-sequence problems is very low. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are other ways through which we can compare the performance of our models?&lt;/p&gt;&#xA;" OwnerUserId="14939" LastEditorUserId="2444" LastEditDate="2019-11-15T02:03:14.227" LastActivityDate="2021-05-03T20:16:23.327" Title="What evaluation metric are used for sequence-to-sequence prediction problems?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;metric&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16510" PostTypeId="1" CreationDate="2019-11-14T08:09:46.623" Score="5" ViewCount="72" Body="&lt;p&gt;&amp;quot;Why would the application of boosting prevent underfitting?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I read in some paper that applying boosting would prevent you from underfitting. Why is that?&lt;/p&gt;&#xA;&lt;p&gt;Source:&lt;br /&gt;&#xA;&lt;a href=&quot;https://www.cs.cornell.edu/courses/cs4780/2015fa/web/lecturenotes/lecturenote13.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.cornell.edu/courses/cs4780/2015fa/web/lecturenotes/lecturenote13.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="156" LastEditDate="2021-03-04T23:09:48.573" LastActivityDate="2021-03-04T23:09:48.573" Title="Why would the application of boosting prevent underfitting?" Tags="&lt;machine-learning&gt;&lt;underfitting&gt;&lt;boosting&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16512" PostTypeId="1" CreationDate="2019-11-14T08:39:57.297" Score="1" ViewCount="32" Body="&lt;p&gt;I am a masters student going to work in a project to analyze the cracks in underwater concrete structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need some suggestions for data acquisition and length measurement of the crack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have decided to do crack segmentation using Mask-RCNN. But I don't know which methodology is best to measure the length of the cracks. While searching about this, I found many ways to measure the crack size when there is another reference object of known size in the image. But in my case, there won't be any reference object and also it is not possible to know the distance between the camera and target since it is underwater.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the images are of stereotype, Will that solve this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="31396" LastActivityDate="2019-11-14T08:39:57.297" Title="How to measure the size of an crack which is segmented from an image using Mask-RCNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16513" PostTypeId="1" CreationDate="2019-11-14T09:21:42.047" Score="1" ViewCount="37" Body="&lt;p&gt;I am a student learning about image processing using CNN. I want to learn how to measure the object size from the &lt;a href=&quot;https://stackoverflow.com/q/7337323/3924118&quot;&gt;disparity map&lt;/a&gt; obtained from left and right stereo images.&lt;/p&gt;&#xA;" OwnerUserId="31396" LastEditorUserId="2444" LastEditDate="2019-11-14T14:04:38.277" LastActivityDate="2019-11-14T14:04:38.277" Title="How to measure object size from the disparity map using CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16514" PostTypeId="1" CreationDate="2019-11-14T09:25:58.350" Score="3" ViewCount="111" Body="&lt;p&gt;What needs to be done to make a fair algorithm (supervised and unsupervised)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, there is &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=3084096&quot; rel=&quot;nofollow noreferrer&quot;&gt;no consensus on the definition of &lt;em&gt;fairness&lt;/em&gt;&lt;/a&gt;, so you can use the definition you find most appropriate.&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-14T13:46:29.937" LastActivityDate="2019-11-14T23:28:41.317" Title="What needs to be done to make a fair algorithm?" Tags="&lt;unsupervised-learning&gt;&lt;social&gt;&lt;supervised-learning&gt;&lt;explainable-ai&gt;&lt;algorithmic-bias&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16515" PostTypeId="1" CreationDate="2019-11-14T10:41:14.330" Score="2" ViewCount="85" Body="&lt;p&gt;I'm trying to train a neural network to do a multiple non-linear regression &lt;span class=&quot;math-container&quot;&gt;$y=f(x_i), i=1,2…N$&lt;/span&gt;. So far it works good (low MSE), but some predictions &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are “non-physical”, for instance for our application it is known  from first principles that when &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; increases, then &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; also has to increase (&lt;span class=&quot;math-container&quot;&gt;$dy/dx_2&amp;gt;0$&lt;/span&gt;), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that &lt;span class=&quot;math-container&quot;&gt;$y + x_5 + x_7$&lt;/span&gt; should be less than a constant &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a &quot;harder&quot; way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)&lt;/p&gt;&#xA;" OwnerUserId="28108" LastEditorUserId="2444" LastEditDate="2019-11-14T14:01:09.580" LastActivityDate="2019-12-18T10:01:15.183" Title="Imposing physical constraints (previous knowledge) in a neural network for regression" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16516" PostTypeId="1" CreationDate="2019-11-14T11:41:12.923" Score="2" ViewCount="377" Body="&lt;p&gt;My understanding is that masked self-attention is necessary during training of GPT-2, as otherwise it would be able to directly see the correct next output at each iteration. My question is whether the attention mask is necessary, or even possible, during inference. As GPT-2 will only be producing one token at a time, it doesn't make sense to mask out future tokens that haven't been inferred yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a follow up, does this mean that during inference GPT-2 has a &quot;sample&quot; dimension that is always equal to the time iteration count + the prompt length, i.e. during the first iteration, if the prompt was just the &quot;START&quot; token, then only a vector of length 768 (or whatever the embedding size is) will flow through the network?&lt;/p&gt;&#xA;" OwnerUserId="31284" LastEditorUserId="31284" LastEditDate="2019-11-14T17:06:36.300" LastActivityDate="2019-11-14T17:06:36.300" Title="Is the Mask Needed for Masked Self-Attention During Inference with GPT-2" Tags="&lt;natural-language-processing&gt;&lt;attention&gt;&lt;transformer&gt;&lt;gpt&gt;&lt;inference&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16517" PostTypeId="1" AcceptedAnswerId="16519" CreationDate="2019-11-14T11:57:21.740" Score="5" ViewCount="430" Body="&lt;p&gt;On page 98 of &lt;a href=&quot;https://arxiv.org/abs/1709.04464&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jet Substructure at the Large Hadron Collider: A Review of Recent Advances in Theory and Machine Learning&lt;/a&gt; the author writes;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Redacted phase space: Studying the distribution of inputs and the&#xA;  network performance after &lt;strong&gt;conditioning&lt;/strong&gt; on standard physically-inspired&#xA;  features can help to visualize what new information the network is&#xA;  using from the jet. Training the network on inputs that have been&#xA;  &lt;strong&gt;conditioned&lt;/strong&gt; on specific values of known features can also be useful&#xA;  for this purpose.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I cannot find other references to &lt;em&gt;conditioning&lt;/em&gt; features. What does that mean?&lt;/p&gt;&#xA;" OwnerUserId="31285" LastEditorUserId="2444" LastEditDate="2019-11-14T13:33:27.970" LastActivityDate="2019-11-14T13:33:27.970" Title="What is &quot;conditioning&quot; on a feature?" Tags="&lt;terminology&gt;&lt;math&gt;&lt;features&gt;&lt;data-preprocessing&gt;&lt;conditional-probability&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16518" PostTypeId="1" CreationDate="2019-11-14T12:13:07.240" Score="2" ViewCount="26" Body="&lt;p&gt;I’m trying to train a neural network to approximate the output of a dynamical system &lt;span class=&quot;math-container&quot;&gt;$dy/dt=f\left(y(t), u(t) \right)$&lt;/span&gt;, namely, given &lt;span class=&quot;math-container&quot;&gt;$y(0)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u(t_i), i=1,2...N$&lt;/span&gt; I want the network to predict &lt;span class=&quot;math-container&quot;&gt;$y(t_i), i=1,2...N$&lt;/span&gt;. So far I’ve thought of several approaches, namely &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Predict the derivative &lt;span class=&quot;math-container&quot;&gt;$dy/dt (t_{i+1}) = f_1 \left(y(t_i), u(t_i) \right)$&lt;/span&gt; and then compute &lt;span class=&quot;math-container&quot;&gt;$y(t_{i+1}) = dy/dt (t_{i+1}) \cdot dt + y(t_{i})$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Predict the increment &lt;span class=&quot;math-container&quot;&gt;$\Delta y (t_{i+1})= f_2 \left(y(t_i), u(t_i), \Delta t \right)$&lt;/span&gt; and then compute &lt;span class=&quot;math-container&quot;&gt;$y(t_{i+1}) = \Delta y (t_{i+1}) + y(t_{i})$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Directly predict the next value &lt;span class=&quot;math-container&quot;&gt;$y(t_{i+1}) = f_3 \left(y(t_i), u(t_i), \Delta t \right)$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which option is recommended?&lt;/p&gt;&#xA;" OwnerUserId="28108" LastEditorUserId="2444" LastEditDate="2019-11-14T13:39:10.327" LastActivityDate="2019-11-14T13:39:10.327" Title="Choosing neural network output for prediction (regression) of a dynamical system" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;prediction&gt;&lt;regression&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16519" PostTypeId="2" ParentId="16517" CreationDate="2019-11-14T13:17:31.180" Score="5" Body="&lt;p&gt;This is conditioning in the sense of &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Conditional_probability&quot; rel=&quot;noreferrer&quot;&gt;conditional probability&lt;/a&gt;&lt;/em&gt;. The idea is that the authors have some &quot;standard physically-inspired features&quot;. They are splitting the data up into bins based on the values of these features, and then training a model for each bin. They are then examining the differences between the models. Usually this is done to learn something about the benefits of using the different features, and about the relationships between features and outputs.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-14T13:17:31.180" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16520" PostTypeId="1" CreationDate="2019-11-14T14:03:32.037" Score="0" ViewCount="66" Body="&lt;p&gt;I am trying to make a text score prediction network, and my dataset have 500 samples only. I know there is a public dataset called the &lt;a href=&quot;https://www.kaggle.com/c/asap-aes&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASAP Dataset&lt;/a&gt;.  I have tested my model &lt;code&gt;(word embedding layer --&amp;gt; LSTM Layer --&amp;gt; FC Layer)&lt;/code&gt; on this dataset and it performed as expected. The public dataset have 13000 data samples while my private one has only 500. When I trained the network on my private dataset, it performed very poorly, and started overfitting from the first epoch. I have tried reducing my model size to the minimum and still no improvement. I have also added dropout and l2 regularization ans still nothign works. Is there any suggestion that could help like a differnet model or something. I am thinking methods like cropping the text to generate more data or some other methods like a Siamese network approach will help. Will they help? For your refernce, this is my code. &lt;a href=&quot;https://github.com/Clement-Hui/EssayGrading&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Clement-Hui/EssayGrading&lt;/a&gt;&#xA;Thank you very much for your help. I would be glad to add a bounty if the answer really helps me. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-11-15T16:29:02.287" LastActivityDate="2019-11-15T16:29:02.287" Title="How to train LSTM score prediction with very little data? (Bounty to be added)" Tags="&lt;long-short-term-memory&gt;&lt;pytorch&gt;&lt;sentiment-analysis&gt;&lt;text-classification&gt;&lt;score-prediction&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16521" PostTypeId="2" ParentId="16516" CreationDate="2019-11-14T14:40:26.407" Score="1" Body="&lt;p&gt;Answer to &lt;strong&gt;Q1)&lt;/strong&gt; &lt;em&gt;If sampling for next token do you need to apply mask during inference?&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes you do! The models ability to transfer information across positions was trained in this manner, and changing it up will have unpredictable consequences. Let my try to give an example:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tokens:&lt;/strong&gt; 1:sally, 2:sold, 3:seashells, 4:on, 5:the, 6:____&lt;br&gt;&#xA;In the above you are trying to predict 6 from {1:5}  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote &lt;span class=&quot;math-container&quot;&gt;$n^{(m)}$&lt;/span&gt; as the set of tokens the &lt;span class=&quot;math-container&quot;&gt;$n^{th}$&lt;/span&gt; positional embedding has info from at the &lt;span class=&quot;math-container&quot;&gt;$m^{th}$&lt;/span&gt; layer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases we see that &lt;span class=&quot;math-container&quot;&gt;$n^{(0)} = \{n\} \ \  \forall n$&lt;/span&gt;. Now though with a mask we get &lt;span class=&quot;math-container&quot;&gt;$n^{(i)} = \{k\}_{k\leq n} \ \  \forall n \ \ s.t. \ \ i \geq 1$&lt;/span&gt; but without we see &lt;span class=&quot;math-container&quot;&gt;$n^{(i)} = \{k\}_{k \in [1:N]} \ \  \forall n$&lt;/span&gt;. This difference means at the final layer the mebeddings going in will differ completely, and unless we train for such an approach it will cause  error  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answer to &lt;strong&gt;Q2)&lt;/strong&gt; &lt;em&gt;What is the &lt;strong&gt;sample&lt;/strong&gt; dimension?&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It took me a couple reads to understand what youre asking for but I think I understand. The sample at each step is drawn from a distribution where its logits are linearly associated to a single embedding of dimension &lt;span class=&quot;math-container&quot;&gt;$d_{(model)}$&lt;/span&gt; therefore that is our upper bound: &lt;span class=&quot;math-container&quot;&gt;$dim(sample) \leq d_{(model)}$&lt;/span&gt; which in the example you gave is 768.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-11-14T14:40:26.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16522" PostTypeId="2" ParentId="2111" CreationDate="2019-11-14T16:29:43.317" Score="0" Body="&lt;p&gt;[&lt;strong&gt;Disclaimer&lt;/strong&gt;: This answer is research; not medical and/or legal advice. I am not a lawyer: &amp;lt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Practicing_without_a_license&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Practicing_without_a_license&lt;/a&gt; &amp;gt;.&lt;/p&gt;&#xA;&lt;p&gt;Though perhaps pedantic, it is better to be safe than sorry.]&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;This is Direct Answer to Your Question:--&lt;/h3&gt;&#xA;&lt;p&gt;The matter is controversial, with some established parameters as what constitutes life. An example of an established parameter for being alive is having: cells or reproduction.&lt;/p&gt;&#xA;&lt;p&gt;It is unclear whether A.I. can create a copy of itself that is independent of its parent. Arguably, partial satisfaction of the aforementioned has already occurred with code that is self-learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;May insight on artificial life lead us to saving lives, particularly in regards to CoVid-19. We want children to live.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;The Controversy&lt;/h3&gt;&#xA;&lt;p&gt;To begin, there is some controversy surrounding the definition of life in Biology.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; [...] The definition of life has long been a challenge for scientists and philosophers, with many varied definitions put forward. This is partially because life is a process, not a substance. This is complicated by a lack of knowledge of the characteristics of living entities, if any, that may have developed outside of Earth. Philosophical definitions of life have also been put forward, with similar difficulties on how to distinguish living things from the non-living. Legal definitions of life have also been described and debated, though these generally focus on the decision to declare a human dead, and the legal ramifications of this decision. [...] &amp;quot; – Wikipedia contributors. &amp;quot;Life.&amp;quot; &lt;em&gt;Wikipedia, The Free Encyclopedia&lt;/em&gt;. Wikipedia, The Free Encyclopedia, 5 Nov. 2019. Web. 14 Nov. 2019.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; [...] Of course, this lack of hard boundaries makes 'artificial life,' as a field of study, significantly ill-defined. Unlike the case for natural life, there are, as yet, no clear criteria for what&#xA;virtual world phenomena should qualify as 'living' or sufficiently 'life-like' to legitimately&#xA;count as lying within this field. In large measure, this simply reflects the continuing debate and&#xA;investigation within conventional biology, of what specific organizational (as opposed to&#xA;material) system characteristics are critical to properly living systems. The key advantage&#xA;and innovation in artificial life is precisely that it has this freedom to vary and explore&#xA;possibilities that are difficult or impossible to investigate in natural living systems. In this&#xA;context, a precise definition of 'life' (natural or artificial) is not a necessary, or even especially&#xA;desirable condition for progress. [...] &amp;quot; – Banzhaf, Wolfgang, and Barry McMullin. &amp;quot;Artificial life.&amp;quot; &lt;em&gt;Handbook of Natural Computing&lt;/em&gt; (2012): 1805-1834.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Since there is often debate and sometimes no clear answer in regards to these questions, I shall explore variable stances on these issues.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Philosophical Points to Ponder and Meditate On&lt;/h3&gt;&#xA;&lt;p&gt;These are some examples of points that involve controversy&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is a crystal alive?&lt;/li&gt;&#xA;&lt;li&gt;Is a virus alive?&lt;/li&gt;&#xA;&lt;li&gt;At what point does a self-learning machine become alive, if it can reproduce and create its own child-offspring A.I?&lt;/li&gt;&#xA;&lt;li&gt;Do we have a possible resolution to these issues?&lt;/li&gt;&#xA;&lt;li&gt;How does this resolution relate to artificial intelligence and artificial life?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Some of these questions have no clear answer, never mind involving A.I. into the question.&lt;/p&gt;&#xA;&lt;p&gt;Do viruses reproduce? Yes. Do they have genetic information? Yes. Do they have cells? No. So are they living? They are usually classified as non-living, yet they can have RNA.&lt;/p&gt;&#xA;&lt;p&gt;The point is, there is debate as to whether a virus is non-living or living; and it is often unclear what the definition of life is.&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://www.scientificamerican.com/article/are-viruses-alive-2004/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scientificamerican.com/article/are-viruses-alive-2004/&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;&lt;strong&gt;Variable Stances:--&lt;/strong&gt;&lt;/h3&gt;&#xA;&lt;p&gt;Artificial life researchers study traditional biology by trying to recreate aspects of biological phenomena.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; [...] Important propositions in the philosophy of AI include:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Turing's 'polite convention': If a machine behaves as intelligently as a human being, then it is as intelligent as a human being.&lt;/li&gt;&#xA;&lt;li&gt;The Dartmouth proposal: 'Every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it.'&lt;/li&gt;&#xA;&lt;li&gt;Newell and Simon's physical symbol system hypothesis: 'A physical symbol system has the necessary and sufficient means of general intelligent action.'&#xA;Searle's strong AI hypothesis: 'The appropriately programmed computer with the right inputs and outputs would thereby have a mind in exactly the same sense human beings have minds.'&lt;/li&gt;&#xA;&lt;li&gt;Hobbes' mechanism: 'For 'reason' ... is nothing but 'reckoning,' that is adding and subtracting, of the consequences of general names agreed upon for the 'marking' and 'signifying' of our thoughts...' [...] &amp;quot; – Wikipedia contributors. &amp;quot;Philosophy of artificial intelligence.&amp;quot;  Wikipedia, The Free Encyclopedia. &lt;em&gt;Wikipedia, The Free Encyclopedia&lt;/em&gt;, 15 Oct. 2019. Web. 14 Nov. 2019.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: I asked some of my colleagues and received the following advice.&lt;/p&gt;&#xA;&lt;p&gt;Another approach is simply to check for self-preservation. Under this postulate, all forms of life, ranging from the absolutely-most-simple single-celled organism to postulated beings-of-immense-power (see the &lt;em&gt;The Last Question&lt;/em&gt; by Isaac Asimov), would be profoundly invested in self-preservation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.inverse.com/article/23136-singularity-ai-death-fear&quot; rel=&quot;nofollow noreferrer&quot;&gt;When the Singularity Comes, Will A.I. Fear Death?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, the preservation of one's children is a valid exception, and does not contradict self-preservation.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-preservation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-preservation&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://idioms.thefreedictionary.com/Self-preservation+is+the+first+law+of+nature&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://idioms.thefreedictionary.com/Self-preservation+is+the+first+law+of+nature&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Many schools of meta-ethics claim that (Human) self-preservation and ethics are absolutely congruent. Other schools dispute this to varying degrees. Kantian meta-ethics, a strong reference, is worth considering here (not talking about A.I., here).&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Meta-ethics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Meta-ethics&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Do viruses follow the principles of avoiding damage to one's self? I leave that up to you.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do addictions factor into this? I leave that up to you, but &lt;strong&gt;please actually be tactful and use common sense&lt;/strong&gt; if you choose to discuss that topic.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Incertae_sedis&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Incertae sedis&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: Viruses are a very serious topic, right now (pandemic). If &lt;strong&gt;drawing parallels to artificial life&lt;/strong&gt; can help the Human Race, then I am touched to have being allowed to serve Humanity in this way.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;He who destroys a life, it is as if he destroyed an entire world. He who saves a life, it is as if he saved an entire world.&amp;quot; ~ Hillel the Elder&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I ask to be taken seriously.&lt;/p&gt;&#xA;&lt;p&gt;Thank you. You, as an educated person, deserve better than this pandemic, and I hope your children will find happiness.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Sources, References, and Further Reading&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Philosophy_of_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Philosophy_of_artificial_intelligence&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_life&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Artificial_life&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Life#Artificial&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Life#Artificial&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Turing_test&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.nytimes.com/2019/05/15/science/synthetic-genome-bacteria.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nytimes.com/2019/05/15/science/synthetic-genome-bacteria.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.scitecheuropa.eu/quantum-artificial-life-cloud/90936/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scitecheuropa.eu/quantum-artificial-life-cloud/90936/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Levy, Steven. &lt;em&gt;Artificial life: A report from the frontier where computers meet biology&lt;/em&gt;. Random House Inc., 1993.&#xA;by Steven Levy&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/frobt.2017.00064/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.frontiersin.org/articles/10.3389/frobt.2017.00064/full&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.educba.com/artificial-intelligence-vs-human-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.educba.com/artificial-intelligence-vs-human-intelligence/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.nytimes.com/2019/05/15/science/synthetic-genome-bacteria.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nytimes.com/2019/05/15/science/synthetic-genome-bacteria.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://plato.stanford.edu/entries/life/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/life/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_satisfaction&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Constraint_satisfaction&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Constraint_satisfaction_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Constraint_satisfaction_problem&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Pain_(philosophy)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Pain_(philosophy)&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://plato.stanford.edu/entries/pain/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/pain/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Other Links:--&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/16162/2444&quot;&gt;What is the name of an AI whose primary goal is to create a better AI?&lt;/a&gt; [link to another question on Stack Exchange]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Notes:--&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I could not find much information regarding business applications of artificial life.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="25982" LastEditDate="2020-09-23T12:01:22.057" LastActivityDate="2020-09-23T12:01:22.057" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="16523" PostTypeId="2" ParentId="16514" CreationDate="2019-11-14T17:48:05.183" Score="2" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/abs/1901.04730&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges&lt;/a&gt; argues that ensuring fairness is not a trivial task and that the current statistical formalizations of fairness lead to a long list of criteria that are each flawed (or even harmful) in different contexts, that is, there are trade-offs between the proposed formalizations. Therefore, fairness constraints in algorithms have to be specific to the domains to which the algorithms are applied. To achieve that, there is a need for collaboration with domain experts and &lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;explainable artificial intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The major obstacle towards fair machine learning algorithms is the presence of &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;algorithmic bias&lt;/strong&gt;&lt;/a&gt;, which can be subdivided into the following main categories: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;strong&gt;bias in the data&lt;/strong&gt;, and &lt;/li&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;inductive bias&lt;/strong&gt;&lt;/a&gt; (the implicit or explicit assumptions behind the algorithm or model) or, in general, any bias introduced during the development of the algorithm or model (for example, a certain choice of a subset of features can change the outcome of the model).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The bias in the data can be due to different factors, such as a biased choice of the collected (or labelled) data or measurement errors (which can make the data not representative of the population). &lt;a href=&quot;https://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Causal inference&lt;/a&gt; can be used to understand the causal relationships in the data, thus it can be used to find the source of bias in the data. To avoid unfairness due to bias in the data, there is a need to analyze and understand the data, so that to improve its quality (for example, by increasing the diversity of the data). However, the bias in the data is not always easily reducible, given that certain &lt;a href=&quot;https://en.wikipedia.org/wiki/Outcome_(probability)&quot; rel=&quot;nofollow noreferrer&quot;&gt;outcomes&lt;/a&gt; of an experiment may rarely occur or are hard to produce in practice, so unbiased data may not always be easily collectible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://cs.stanford.edu/~jure/pubs/contraction-kdd17.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The selective labels problem: Evaluating algorithmic predictions in the presence of unobservables&lt;/a&gt; (2017), the authors address the &lt;strong&gt;selective labels problem&lt;/strong&gt; (that is, the judgments of decision-makers determine which instances are labeled in the data, which can thus introduce bias in the data) and develop an approach called &lt;strong&gt;contraction&lt;/strong&gt;, which can be used to compare the performance of predictive models and human decision-makers (even in the presence of &lt;a href=&quot;https://en.wikipedia.org/wiki/Latent_variable&quot; rel=&quot;nofollow noreferrer&quot;&gt;unobservables&lt;/a&gt;). There are also works that are based on Bayesian or causal inference (for example, &lt;a href=&quot;https://arxiv.org/abs/1809.05651&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;risk-adjusted regression&lt;/strong&gt;&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/abs/1901.04730&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;sample bias&lt;/strong&gt;&lt;/a&gt; (the data is not representative of the overall population due to a systematic intentional or unintentional error in data collection, a &lt;strong&gt;measurement error&lt;/strong&gt;, which can also be due to social prejudices) is also a form of bias in the data. In the paper &lt;a href=&quot;https://arxiv.org/abs/1806.02887&quot; rel=&quot;nofollow noreferrer&quot;&gt;Residual unfairness in fair machine learning from prejudiced data&lt;/a&gt; (2018), the authors address this problem in the context of and of &lt;em&gt;police stop-and-frisk&lt;/em&gt; (where a biased police behavior leads to over-proportional stopping of a racial minority group). &lt;a href=&quot;https://arxiv.org/abs/1806.02887&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nathan Kallus and Angela Zhou&lt;/a&gt; show that adjusting the classifier for fairness does not solve the &lt;em&gt;sample bias problem&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1901.04730&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges&lt;/a&gt;, the author argues that any attempt to reduce the bias introduced during the development of the algorithms or models, if it does not take into account the specific social and moral context where they are supposed to be applied, can still lead to algorithmic bias. Furthermore, algorithms should undergo &lt;strong&gt;frequent reevaluations&lt;/strong&gt;, given that, for example, the underlying &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_population&quot; rel=&quot;nofollow noreferrer&quot;&gt;population&lt;/a&gt; or the application context may change.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-14T23:28:41.317" LastActivityDate="2019-11-14T23:28:41.317" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16525" PostTypeId="1" CreationDate="2019-11-14T19:28:26.253" Score="2" ViewCount="494" Body="&lt;p&gt;What kind of hardware (processor, memory, graphics card, etc.) and software are required to build an AI system? How do we go about linking each and every one of them to their proper place and function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is needed besides the main hardware or software of a computer that will be used to make an AI system?&lt;/p&gt;&#xA;" OwnerUserId="31293" LastEditorUserId="2444" LastEditDate="2019-11-15T15:26:21.143" LastActivityDate="2019-11-15T15:26:21.143" Title="What kind of hardware and software are required to build an AI system?" Tags="&lt;hardware&gt;&lt;software-evaluation&gt;&lt;hardware-evaluation&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2019-11-19T01:56:29.210" ContentLicense="CC BY-SA 4.0" />
  <row Id="16529" PostTypeId="1" CreationDate="2019-11-14T23:00:53.897" Score="3" ViewCount="120" Body="&lt;p&gt;I have a CNN model that I need to train for a large scale genomics application. It is working well with a subset of my training data. I have scaled up to a subset of about 130 million examples and training time is very long, about 3 hours per epoch. I plan to scale up to the hundreds of billions of training examples and I anticipate training time to be not be feasible with my current design. I would appreciate feedback on how I can streamline the training or improve some aspect of my design that I may not be considering. Currently, I am training from a MongoDB. The training examples are not very large. Here is an example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    'added': datetime.datetime(2019, 11, 1, 6, 13, 13, 340000),&#xA;    '_id': ObjectId('5dbbccf92464af872756022e'),&#xA;    'label': 0,&#xA;    'accession': 'GM_0001',&#xA;    'data': '34363,30450,9019,19152,8726,22128,59881,17670,15803,64454,64579,28103,52442,64951,29783,64574,652,19243,33498,14775,18803,4700,55446,53912,47645,41465,48257,16305,62071,12334,44698,24371,46515,8445,3000,61849,43228,18120,23587,11105,5453,42707,42739,46122,31285,40773,48162,16653,58783,2928,2836,21330,46947,6719,26992,8852,14520,46212,47362,43554,2147,39372,33885,59716,37384,14825,53387,58763,18065,34070,23278,15641,40237,47950,58811,40015,36880,29841,45351,14904,49660,48224,54638,50358,17202,10701,3564,4829,62655,5684,37207,49724,16369,6769,37827,38144,63885,5070,42882,48960,16178,35758,50554,54253,34556,2383,39431,30176,11482,24459,4472,53825,7764,44500,4869,50875,33037,56353,46848,30769,18729,46026,41409,2826,12092,17086',&#xA;    'name': 'Example_1'&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The relevant data is the 'data' field which is a string of 126 integers where each integer is a value between 0 and about 65,000. The other fields are convenient, but not necessary except for the 'label' field. But even this I could insert into the front of the data field. I mention this because I don't think I necessarily need to train from a MongoDB database. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Keras 2.3.0 with TensorFlow 2.0.0. Below is an example of my code. The workflow is 1) Load a text file containing the document ids of all training examples in the MongoDB collection. I do this so I can shuffle the examples before sending them to the model for training. 2) I load the examples in batches of 50,000 using my Custom_Generator class. This class pulls the documents from the MongoDB using the list of document ids. 3) The model is trained. I use 5 epochs. I currently have 5-fold cross-validation but I know this is not feasible on the full training set. For that I will do a single train-test split. I am currently performing this on a Google Cloud instance with 2 Tesla T4 GPUs. The database is on a bucket. With the cloud I have flexibility of hardware architectures. I would appreciate any insight. This is a rather large engineering challenge for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional background to the problem:&lt;/strong&gt;&#xA;The objective is to classify organisms into broad classes quickly for downstream analysis. The pool of organisms I want to classify is very large (10s of thousands) and very diverse. I'm essentially reading the genomes of the organisms like a book. The genome (a series of &quot;A&quot;, &quot;T&quot;, &quot;C&quot;, or &quot;G&quot;) is processed in chunks through a hash function producing integer strings as shown above. Depending on the size of the organism genome, thousands to millions of these integer strings may be produced. So I have many thousands of organisms producing many thousands to millions of examples. To be successful, I feel like I need to capture the diversity of the genomes in the organism pool. To give an example, even though Ecoli and Salmonella are both bacteria, their genomes are quite distinct. I feel like I need to have them both represented in the training set to distinguish them from other organisms I would label as a different class. As far as reducing the dataset, I think I can get by with only training on a representative organism for a give species (since there are many unique genomes available for Ecoli, for example). This will help considerably, but I think the training data set will likely still be in the billions of examples. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import time&#xA;from keras.utils import Sequence, to_categorical, multi_gpu_model&#xA;from keras.models import Sequential&#xA;from keras.layers import Dense&#xA;from keras.layers import Flatten&#xA;from keras.layers import Embedding&#xA;from keras.layers.convolutional import Conv1D&#xA;from keras.layers.convolutional import MaxPooling1D&#xA;from sklearn.model_selection import KFold&#xA;from keras.preprocessing.sequence import pad_sequences&#xA;import numpy as np&#xA;import random&#xA;from pymongo import MongoClient&#xA;from bson import ObjectId&#xA;from sklearn.metrics import classification_report, confusion_matrix&#xA;&#xA;&#xA;class Custom_Generator(Sequence) :&#xA;&#xA;    def __init__(self, document_ids, batch_size) :&#xA;        self.document_ids = document_ids&#xA;        self.batch_size = batch_size&#xA;&#xA;&#xA;    def __len__(self) :&#xA;        return (np.ceil(len(self.document_ids) / float(self.batch_size))).astype(np.int)&#xA;&#xA;&#xA;    def __getitem__(self, idx) :&#xA;        client = MongoClient(port=27017)&#xA;        db = client[database]&#xA;        document_ids = self.document_ids[idx * self.batch_size : (idx+1) * self.batch_size]&#xA;        query_results = db[collection].find({'_id': {'$in': document_ids}})&#xA;        batch_x, batch_y = [], []&#xA;        for result in query_results:&#xA;            kmer_list = result['kmers'].split(',')&#xA;            label = result['label']&#xA;            x = [x for x in kmer_list if len(x) &amp;gt; 0]&#xA;            if len(x) &amp;lt; 1:&#xA;                continue&#xA;            batch_x.append(x)&#xA;            one_hot_y = to_categorical(label, 5)&#xA;            batch_y.append(one_hot_y)&#xA;        batch_x = pad_sequences(batch_x, maxlen=126, padding='post')&#xA;        client.close()&#xA;        return np.array(batch_x), np.array(batch_y)&#xA;&#xA;&#xA;# MongoDB database, collection, and document ids of collection&#xA;database = 'db'&#xA;collection = 'collection_subset2'&#xA;docids_file = 'docids_collection_subset2.txt'&#xA;id_ls = []&#xA;# Convert docids strings to MongoDB ObjectID&#xA;with open(docids_file) as f:&#xA;    for line in f:&#xA;        id_ls.append(ObjectId(line.strip()))&#xA;random.shuffle(id_ls)&#xA;&#xA;# Model&#xA;model = Sequential()&#xA;model.add(Embedding(65521, 100, input_length=126))&#xA;model.add(Conv1D(filters=25, kernel_size=5, activation='relu'))&#xA;model.add(MaxPooling1D(pool_size=2))&#xA;model.add(Conv1D(filters=30, kernel_size=3, activation='relu'))&#xA;model.add(MaxPooling1D(pool_size=2))&#xA;model.add(Flatten())&#xA;model.add(Dense(1000, activation='relu'))&#xA;model.add(Dense(5, kernel_initializer=&quot;normal&quot;, activation=&quot;softmax&quot;))&#xA;metrics=['accuracy'])&#xA;parallel_model = multi_gpu_model(model, gpus=2)&#xA;parallel_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;&#xA;seed = 7&#xA;batch_size = 50000&#xA;&#xA;# Currently training with 5-fold CV. Will only use single test train split &#xA;# on the full-scale dataset.&#xA;kfold = KFold(n_splits=5, shuffle=True, random_state=seed)&#xA;kfold_stats = {}&#xA;accuracy_ls = []&#xA;val_accuracy_ls = []&#xA;confusion_ls = []&#xA;for fold_idx, (train_idx, test_idx) in enumerate(kfold.split(id_ls)):&#xA;    ids_train = np.array(id_ls)[train_idx].tolist()&#xA;    ids_test = np.array(id_ls)[test_idx].tolist()&#xA;    training_batch_generator = Custom_Generator(ids_train, batch_size)&#xA;    validation_batch_generator = Custom_Generator(ids_test, batch_size)&#xA;    print('Number of train files: %d' % len(ids_train))&#xA;    print('Number of test files: %d' % len(ids_test))&#xA;    start = time.time()&#xA;    history = parallel_model.fit_generator(&#xA;        generator=training_batch_generator,&#xA;        steps_per_epoch = int(len(ids_train) // batch_size),&#xA;        epochs = 5,&#xA;        verbose = 2,&#xA;        validation_data = validation_batch_generator,&#xA;        validation_steps = int(len(ids_test) // batch_size),&#xA;        use_multiprocessing=True&#xA;    )&#xA;    sys.stderr.write(&quot;time to train model (seconds): %d\n&quot;%(time.time() - start))&#xA;    sys.stderr.flush()&#xA;    print(history.history)&#xA;    fold_name = 'kfold_%s' % str(fold_idx)&#xA;    kfold_stats.update({fold_name: history.history})&#xA;    accuracy_ls.extend(history.history['accuracy'])&#xA;    val_accuracy_ls.extend(history.history['val_accuracy'])&#xA;    parallel_model.save('model_output_kfold_%s.h5' % str(fold_idx))&#xA;    print(&quot;Kfold %s finished&quot; % str(fold_idx))&#xA;    Y_pred = parallel_model.predict_generator(validation_batch_generator)&#xA;    y_pred = np.argmax(Y_pred, axis=1)&#xA;    y_true = np.concatenate([np.argmax(batch[1], axis=1) for batch in validation_batch_generator])  &#xA;    print('Confusion Matrix')&#xA;    conf = confusion_matrix(y_true, y_pred)&#xA;    print(conf)&#xA;    confusion_ls.append(conf)&#xA;    print('Classification Report')&#xA;    target_names = ['Class_name_1', 'Class_name_2', 'Class_name_3', 'Class_name_4', 'Class_name_5']&#xA;    report = classification_report(y_true, y_pred, target_names=target_names)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31294" LastEditorUserId="31294" LastEditDate="2019-11-15T18:40:11.307" LastActivityDate="2019-11-15T18:40:11.307" Title="What is the fastest way to train a CNN with billions of examples?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16530" PostTypeId="2" ParentId="16525" CreationDate="2019-11-15T01:19:19.217" Score="0" Body="&lt;p&gt;Many different components may go into building an AI system. AI is a broad topic encompassing, machine learning, statistical learning so an AI system could involve anything from a raspberry pi to a spoon. Your title refers specifically to processors of an AI system. The mathematics behind AI can be run on any processor. GPU, CPU, FPGA, and new &lt;a href=&quot;https://www.nextplatform.com/2019/11/13/intel-throws-down-ai-gauntlet-with-neural-network-chips/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural network chips&lt;/a&gt;. A neural network can be run on a Window PC a linux PC AI packages are not OS dependent. I can train a &lt;a href=&quot;https://www.tensorflow.org/tutorials/quickstart/beginner&quot; rel=&quot;nofollow noreferrer&quot;&gt;model&lt;/a&gt; on my laptop now. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2019-11-14 20:10:57.503229:&#xA;Epoch 1/5&#xA;60000/60000 [==============================] - 8s 137us/sample - loss: 0.2925 - acc: 0.9141&#xA;Epoch 2/5&#xA;60000/60000 [==============================] - 8s 140us/sample - loss: 0.1426 - acc: 0.9574&#xA;Epoch 3/5&#xA;60000/60000 [==============================] - 9s 142us/sample - loss: 0.1059 - acc: 0.9674&#xA;Epoch 4/5&#xA;60000/60000 [==============================] - 8s 135us/sample - loss: 0.0862 - acc: 0.9732&#xA;Epoch 5/5&#xA;60000/60000 [==============================] - 7s 112us/sample - loss: 0.0744 - acc: 0.9763&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just like any computer application an AI needs a processor and memory to store information. Beyond that it is difficult what specifically you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://delivery.acm.org/10.1145/3280000/3276541/p1-fowers.pdf?ip=198.137.18.146&amp;amp;id=3276541&amp;amp;acc=ACTIVE%20SERVICE&amp;amp;key=AA86BE8B6928DDC7%2EB68F0B462DD42179%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&amp;amp;__acm__=1573780985_0be9eae2aa14eb842d14ce4a94d9d833&quot; rel=&quot;nofollow noreferrer&quot;&gt;A configurable cloud-scale DNN processor for real-time AI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-15T01:19:19.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16531" PostTypeId="1" CreationDate="2019-11-15T03:08:08.313" Score="2" ViewCount="189" Body="&lt;p&gt;I am trying to train a VAE for anomaly detection. I chose one architecture from &lt;a href=&quot;https://github.com/1Konny/Beta-VAE&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Github repository&lt;/a&gt; and I adjusted the input and output to match what I need. In my case, the input (and hence the output) are a 12D vector. I tried several sizes for the latent space, but, for some reason, it's not training. From the beginning, the KL loss in almost zero (around 1e-10), while the reconstruction loss (MSE for Gaussian distribution) is around 1, and they basically vary around these values without learning anything further. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any general tips for troubleshooting a VAE (I never trained one before)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty sure that the code is right and the data for sure has a background and signal (the ratio is 10:1), so I am not really sure what I am doing wrong.&lt;/p&gt;&#xA;" OwnerUserId="31302" LastEditorUserId="2444" LastEditDate="2019-11-15T15:18:42.967" LastActivityDate="2019-11-15T15:18:42.967" Title="Are there any general tips for troubleshooting a VAE when apparently it is not learning?" Tags="&lt;generative-model&gt;&lt;autoencoders&gt;&lt;implementation&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16532" PostTypeId="2" ParentId="16479" CreationDate="2019-11-15T03:38:07.430" Score="0" Body="&lt;p&gt;We can manipulate a model's test data set if the machine learning model takes user input and uses it to resample test data set. The actual training dataset of the ML model does not get manipulated, but if we figure out the ML model through an exploratory attack (sending a lot of inquiries to the ML model to find out its nature), we can generate a training dataset which was built into the original ML model.&lt;/p&gt;&#xA;" OwnerUserId="31240" LastActivityDate="2019-11-15T03:38:07.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16533" PostTypeId="1" CreationDate="2019-11-15T07:00:52.413" Score="2" ViewCount="70" Body="&lt;p&gt;I have a large 2D grid having 30k rows and 35k columns, so a total of 30x35k grid cells. Each grid cell is represented by a unique integer number (identity of grid cell). I have several trajectories that passes through these grid cells. Each trajectory is represented by a sequence of numbers (that are grid cells through which the trajectory passes through). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve the problem of &lt;strong&gt;trajectory prediction&lt;/strong&gt; by giving the partial trajectory as input and predict the full trajectory. This becomes a sequence to sequence problem, where all sequences are integer values by default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve this problem through encoder-decoder LSTM architecture. Most tutorials/examples regarding sequence to sequence on net are on machine translations in which vocabularies or characters are &lt;strong&gt;one-hot-encoded&lt;/strong&gt; to represent the text as integer values. When I hot-encode my sequence values the one-hot vector becomes very large because there are (30x35)k grid cells, the program has given memory overflow error (because each vector has of size 1 million). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused here, do I need to treat grid identity as categorical variable? because all grid identities are numeric numbers but these identities are not comparable (like prices). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to hot-encode my integer values in my sequence? Or is there any other alternative to solve this problem? I also appreciate if you suggest me the &lt;strong&gt;similar tutorials&lt;/strong&gt; having the sequence to sequence prediction problem. &lt;/p&gt;&#xA;" OwnerUserId="14939" LastEditorUserId="14939" LastEditDate="2019-11-15T14:40:59.323" LastActivityDate="2021-04-08T17:03:05.370" Title="How to represent integer values in sequence to sequence prediction task in encoder-decoder LSTM?" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16534" PostTypeId="1" CreationDate="2019-11-15T07:27:21.777" Score="4" ViewCount="1300" Body="&lt;p&gt;I'm searching for a loss function that fits my project. Actually, I have two questions, but they are in the same direction. I take a look at the definition of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Root-mean-square_deviation&quot; rel=&quot;nofollow noreferrer&quot;&gt;root mean squared error (RMSE)&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Euclidean_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euclidean distance&lt;/a&gt; and they look the same to me. That's why I want to know the difference between the two. What would be the difference if I use RMSE as a loss function or the euclidean distance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second question is how to search for a loss function. I mean I know it depends on the problem and common things are MSE for regression and cross-entropy for classification, but let's say I have a specific problem, how do I search for a loss function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also saw that some people use a custom loss function and most of the deep learning frameworks allow us to define a custom loss function, but why would I want to use a custom one? How do I get &lt;strong&gt;the intuition that I need a custom loss function?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, I'm doing a project where I need to reduce the GPS error of a vehicle. I have some vehicle data and my neural network will try to predict the longitude and latitude, so it's a regression problem. That's why I thought that the Euclidean distance would make sense as a loss function, right? Now, somehow MSE also makes sense to me because it is getting the difference between prediction and ground truth. Does this make sense to you as a professional ML engineer or data scientist? And if there would be a &lt;strong&gt;custom loss function&lt;/strong&gt; that you can use, what would you suggest and why?&lt;/p&gt;&#xA;" OwnerUserId="30327" LastEditorUserId="2444" LastEditDate="2019-11-15T15:01:23.393" LastActivityDate="2019-11-15T15:01:23.393" Title="What's the difference between RMSE and Euclidean distance, and when to use a custom loss?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;optimization&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2019-11-19T04:25:11.387" ContentLicense="CC BY-SA 4.0" />
  <row Id="16535" PostTypeId="1" CreationDate="2019-11-15T08:27:40.310" Score="2" ViewCount="76" Body="&lt;p&gt;Is it possible to train a convolutional neural network (CNN) to predict the dimensions of primitive objects such as (spheres, cylinders, cuboids, etc.) from point clouds?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input to the CNN will be the point cloud of a &lt;strong&gt;single&lt;/strong&gt; object and the output will be the dimensions of the object (for example, radius and height of the cylinder). The training data will be the point cloud of the object with the ground truth dimensions in a regression final layer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is possible for images since it will similar to a bounding box detection, but I am not sure with point clouds.&lt;/p&gt;&#xA;" OwnerUserId="31259" LastEditorUserId="2444" LastEditDate="2019-11-15T15:15:18.840" LastActivityDate="2019-11-15T20:30:05.400" Title="Is it possible to train a CNN to predict the dimensions of primitive objects from point clouds?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16536" PostTypeId="1" AcceptedAnswerId="16737" CreationDate="2019-11-15T09:22:00.470" Score="11" ViewCount="309" Body="&lt;p&gt;I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite seemingly overfitting training data. A lot of people in the data science field that I've interacted with agree that there's no explanation on &lt;strong&gt;why&lt;/strong&gt; deep neural networks work as well as they do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's gotten me interested in the theoretical basis for why deep nets work so well. Googling tells me it's kind of an open problem, but I'm not sure of the current state of research in answering this question. Notably, there are these two preprints that seem to tackle this question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Deep Learning&lt;/a&gt; (2019)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1905.11427.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness&lt;/a&gt; (2019)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If anyone else is interested in and following this research area, could you please explain the current state of research on this open problem? What are the latest works, preprints or publications that attempt to tackle it?&lt;/p&gt;&#xA;" OwnerUserId="27548" LastEditorUserId="2444" LastEditDate="2019-11-25T16:15:22.343" LastActivityDate="2019-11-25T16:23:00.320" Title="What are the state-of-the-art results on the generalization ability of deep learning methods?" Tags="&lt;deep-learning&gt;&lt;research&gt;&lt;reference-request&gt;&lt;computational-learning-theory&gt;&lt;generalization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16537" PostTypeId="2" ParentId="16534" CreationDate="2019-11-15T09:50:42.580" Score="3" Body="&lt;p&gt;For the first question, RMSE and Euclidean distance have no difference, not that i know of. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second question, you only need the common loss function for normal tasks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;MSE is a common loss function used in &lt;strong&gt;linear&lt;/strong&gt; regression tasks as well as loss function similar in nature like the RMSE. For classification tasks, Cross Entropy Loss is preferred. For &lt;strong&gt;logistic regression&lt;/strong&gt;, use Binary Cross Entropy loss. See this for details:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Cross-entropy loss, or log loss, measure the performance of a&#xA;  classification model whose output is a probability value between 0 and&#xA;  1. It is preferred for classification, while mean squared error (MSE) is one of the best choices for regression. This comes directly from&#xA;  the statement of your problems itself. In classification you work with&#xA;  a very particular set of possible output values thus MSE is badly&#xA;  defined. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To better understand the phenomena it is good to follow and understand&#xA;  the relations between&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Cross-entropy&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Logistic regression (binary cross-entropy)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Linear regression (MSE)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;You will notice that both can be seen as a maximum likelihood&#xA;  estimator (MLE), simply with different assumptions about the dependent&#xA;  variable.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;When you derive the cost function from the aspect of probability and&#xA;  distribution, you can observe that MSE happens when you assume the&#xA;  error follows Normal Distribution and cross-entropy when you assume&#xA;  binomial distribution. It means that implicitly when you use MSE, you&#xA;  are doing regression (estimation) and when you use CE, you are doing&#xA;  classification. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://intellipaat.com/community/2015/why-is-the-cross-entropy-method-preferred-over-mean-squared-error-in-what-cases-does-this-doesnt-hold-up&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://intellipaat.com/community/2015/why-is-the-cross-entropy-method-preferred-over-mean-squared-error-in-what-cases-does-this-doesnt-hold-up&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a custom loss function, there is examples like the triplet loss and cases where you need to optimize two loss at once, then you need a custom loss function. For the case of triplet loss, it is used in one-shot learning tasks like face recognition. There, face images are fed through a CNN to get embeddings about description of the face. In the training phase there is three images per batch. One anchor, one positive (which is the same person as anchor&#xA;) and one negative (different person from anchor). The loss function, triplet loss, maximizes the distance between the anchor embedding and the negative embedding. In reverse, it also minimize the distance between the anchor embedding and the positive embedding. The loss function is like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WHdCj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WHdCj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, there is two components here, and they are subtracted to maximize the second component and minimize the first. The alpha value is to make the loss positive as the gradient descent optimizes the loss to be as near to zero as possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case of style transfer also introduces a custom loss function. Here is the loss function:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lbjUE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lbjUE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Style transfer basically alters the input image to be styled like the style image of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function consist of style loss and content loss. The content loss decreases the distance of the output image and the style loss decreases the distance of the output image and the style in an artistic sense. The second loss makes the output image imitate the style of the the style image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both scenario a custom loss function is used. In both cases, the custom loss function is used to optimize two losses together, either positively or negatively. A weight can also be introduced to weight the two losses as shown in the second case. However the second case introduce a custom loss function. That leads to the second use of custom loss function. They can be used for specific tasks and optimize a specific goal. The first use for custom loss function can be explored but the second requires careful engineering and research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in short, custom loss function are used for either combining two loss or a specific task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2019-11-15T11:36:48.757" LastActivityDate="2019-11-15T11:36:48.757" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="16538" PostTypeId="1" CreationDate="2019-11-15T09:54:44.230" Score="10" ViewCount="5441" Body="&lt;p&gt;What is the reason AMD Radeon is not widely used for machine learning and deep learning? Is it mainly an issue of lack of software? Or is Radeon's GPU not as good as NVIDIA's?&lt;/p&gt;&#xA;" OwnerUserId="31307" LastActivityDate="2019-11-15T12:20:47.577" Title="What is the reason AMD Radeon is not widely used for machine learning and deep learning?" Tags="&lt;hardware-evaluation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ClosedDate="2020-06-14T10:28:01.480" ContentLicense="CC BY-SA 4.0" />
  <row Id="16539" PostTypeId="1" AcceptedAnswerId="16540" CreationDate="2019-11-15T11:47:27.703" Score="1" ViewCount="48" Body="&lt;p&gt;I want to merge 2 data sets in one, but don't know the right approach to do it. The datasets are similar, the last column is the same - will or not them buy a product. In the first dataset, users who only will buy, in second - only who won't buy.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 1st dataset contains 500 rows and 2nd 10000 rows. What will be the right approach to merge it? How can I normalize them? And to point for an algorithm that the last column is the main sequence on what it should learn? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id    income date will_buy&#xA;&#xA;23123 200    10.5 Yes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and second dataset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id    income date will_buy&#xA;&#xA;2323  100    10.5 No&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30737" LastEditorUserId="2444" LastEditDate="2019-11-15T14:25:24.610" LastActivityDate="2019-11-15T14:25:24.610" Title="How can I merge two datasets?" Tags="&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2019-11-15T20:37:30.487" ContentLicense="CC BY-SA 4.0" />
  <row Id="16540" PostTypeId="2" ParentId="16539" CreationDate="2019-11-15T11:57:53.850" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.append.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;append&lt;/a&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;final = df1.append(df2, ignore_index=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To set the last column as labels, you set them as so by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;labels = np.array(final[&quot;will_buy&quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, when calling the &lt;code&gt;fit&lt;/code&gt; method on the model you build, you set &lt;code&gt;labels = labels&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="2444" LastEditDate="2019-11-15T14:16:49.387" LastActivityDate="2019-11-15T14:16:49.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16541" PostTypeId="2" ParentId="16538" CreationDate="2019-11-15T12:20:47.577" Score="5" Body="&lt;p&gt;The main reason that AMD Radeon graphics card is not used for deep learning is not the hardware and raw speed. Instead it is because the software and drivers for deep learning on Radeon GPU is not actively developed. NVIDIA have good drivers and software stack for deep learning such as CUDA, CUDNN and more. Many deep learning library also have CUDA support. However for AMD there is little support on software of GPU. There is ROCM but it is not well optimized and also a lot of deep learning libraries don't have ROCM support.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also on the hardware side, AMD lacks deep learning specific features like tensor cores. AMD also don't have data centre specific cards like the Tesla lineup. The performing of AMD cards also is not very good until recently with the RDNA architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, data centres don't change hardware very often. NVIDIA flourished in the deep learning field very early on so many companies bought a lot of Tesla GPU. Even if AMD caught up in the deep learning field it is very hard as many companies have used NVIDIA from a long time ago, and switching to a very different architecture of GPU is troublesome, especially for a data centre with couple hundred or more servers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AMD has also &quot;given up&quot; on deep learning market. They don't actively tries to make effort in deep learning data centres using AMD hardware. Many data centre also use very old Tesla hardwares like Tesla K80 of M series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-15T12:20:47.577" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16542" PostTypeId="1" AcceptedAnswerId="16544" CreationDate="2019-11-15T12:37:52.180" Score="2" ViewCount="33" Body="&lt;p&gt;Suppose we want to detect whether an object is one of the following classes: &lt;span class=&quot;math-container&quot;&gt;$\text{Object}_1, \text{Object}_2, \text{Object}_3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{Person}$&lt;/span&gt;. Should the annotated images only contain bounding boxes for either a person or an object? In other words, suppose an image has both &lt;span class=&quot;math-container&quot;&gt;$\text{Object}_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{Person}$&lt;/span&gt;. Should you create a copy of this image where the first version only has a bounding box on the object and the second copy only has a bounding box on the person? &lt;/p&gt;&#xA;" OwnerUserId="28201" LastEditorUserId="2444" LastEditDate="2019-11-15T15:30:20.337" LastActivityDate="2019-11-15T15:30:20.337" Title="If an image contains two distinct objects, should I create a copy of this image with distinct labels for each copy?" Tags="&lt;computer-vision&gt;&lt;datasets&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16543" PostTypeId="2" ParentId="16533" CreationDate="2019-11-15T12:58:25.270" Score="0" Body="&lt;p&gt;One hot encoding helps a lot. You can actually one hot encode each batch just before the training to reduce memory usage. One hot encoding makes the input data more intuitive for the network as a numeric value requires the network to do multiple comparisons to understand the value. For examples please tell me which deep learning framework you are using so I can give you some links or code. Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-15T12:58:25.270" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16544" PostTypeId="2" ParentId="16542" CreationDate="2019-11-15T13:26:45.213" Score="2" Body="&lt;p&gt;You should use both classes together. Let's say you use the method you proposed. Then they will be contradicting each other as one teaches the network to recognize people, not objects and the other teaches the network to recognizes objects not person. There is no need for seperation of the two classes, unless you are making two seperate classifier. Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-15T13:26:45.213" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16545" PostTypeId="1" AcceptedAnswerId="16547" CreationDate="2019-11-15T13:54:53.953" Score="1" ViewCount="111" Body="&lt;p&gt;How does one interpret the &quot;min_input_size&quot;, &quot;max_input_size&quot; and &quot;anchors&quot; fields in the Yolov3 config file &lt;a href=&quot;https://github.com/experiencor/keras-yolo3/blob/master/config.json&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In particular, suppose we have the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &quot;min_input_size&quot;:       288,&#xA;&#xA;    &quot;max_input_size&quot;:       448,&#xA;&#xA;    &quot;anchors&quot;:              [55,69, 75,234, 133,240, 136,129, 142,363, 203,290, 228,184, 285,359, 341,260]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does the min_input_size and max_input_size indicate the maximum number of training images we can have? What do the numbers in the &quot;anchors&quot; field indicate? Are they the coordinates of the anchor boxes? Surprisingly, I have not been able to find a good explanation of many of these  fields within this file.&lt;/p&gt;&#xA;" OwnerUserId="28201" LastActivityDate="2019-11-15T14:03:20.800" Title="Interpreting Keras Yolov3 config file" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2019-11-19T00:53:27.957" ContentLicense="CC BY-SA 4.0" />
  <row Id="16546" PostTypeId="2" ParentId="16529" CreationDate="2019-11-15T13:56:57.330" Score="1" Body="&lt;p&gt;First of all, you should add the argument &lt;code&gt;workers = n&lt;/code&gt; in the fit generator call. n should be bigger than 1 to prefetch data. As your data processing requires the data be taken from a server or port, you should do pre fetching data as that would fetch the next data while GPU is processing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you call fit_generator with &lt;code&gt;workers &amp;gt; 1&lt;/code&gt; , &lt;code&gt;use_multiprocessing=True&lt;/code&gt; , we will prefetch &lt;code&gt;queue_size&lt;/code&gt; batches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://github.com/keras-team/keras/issues/12847&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/keras-team/keras/issues/12847&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, as @MichaelHearn mentioned, you should plot a graph to see how many data is needed. You task should not require several billion data samples. A couple hundred thousand data should be enough. If the model is not getting a good accuracy from a couple hundred thousand data samples, perhaps you can try changing up the model architecture instead of adding more data. Maybe you can add drop out layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-15T13:56:57.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16547" PostTypeId="2" ParentId="16545" CreationDate="2019-11-15T14:03:20.800" Score="0" Body="&lt;p&gt;The min and max input size should be the min and max image size of the input images. The numbers represent pixels in both axis of the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The anchors represents the size of the anchor box. Anchor box does not have coordinates, only have size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-15T14:03:20.800" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16548" PostTypeId="1" AcceptedAnswerId="16552" CreationDate="2019-11-15T17:38:26.477" Score="3" ViewCount="346" Body="&lt;p&gt;I would like to use unsupervised learning with a technique called &lt;em&gt;diffusion map based manifold learning&lt;/em&gt; in Python. The original paper on the diffusion map is &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.462.6590&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Diffusion Maps&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already checkout &lt;a href=&quot;https://pydiffmap.readthedocs.io/en/master/theory.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pydiffmap&lt;/a&gt; library, but it is super slow with my use case, and I am yet to see someone endorse the correctness of the implementation. I am looking for an alternate implementation in Python or R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: It is also not available in &lt;code&gt;sklearn&lt;/code&gt;, as far as I know. The list of other methods in &lt;code&gt;sklearn&lt;/code&gt; is &lt;a href=&quot;https://scikit-learn.org/stable/modules/manifold.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="31314" LastEditorUserId="2444" LastEditDate="2019-11-15T19:44:14.440" LastActivityDate="2019-11-16T04:48:52.337" Title="Is there any Python library available for manifold learning using diffusion map?" Tags="&lt;python&gt;&lt;unsupervised-learning&gt;&lt;geometric-deep-learning&gt;&lt;resource-request&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-05-22T18:26:55.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="16549" PostTypeId="2" ParentId="16535" CreationDate="2019-11-15T19:12:41.487" Score="2" Body="&lt;p&gt;I'm working on a similar problem. I'm using a 2D point cloud of an object, for example, X and Y coordinates for height, and with that more simple data set I will train a regression model (currently working on that). In my opinion, this approach with dissecting complex point cloud into cross sections that contain wanted dimension and feeding that to the model will be more simple and easier for the training. &lt;/p&gt;&#xA;" OwnerUserId="31315" LastEditorUserId="31315" LastEditDate="2019-11-15T20:30:05.400" LastActivityDate="2019-11-15T20:30:05.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16550" PostTypeId="1" CreationDate="2019-11-15T20:32:05.960" Score="1" ViewCount="96" Body="&lt;p&gt;I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PcVxY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PcVxY.jpg&quot; alt=&quot;example sample image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I just run OCR (like with Tesseract in python):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cv2&#xA;import pytesseract&#xA;img = cv2.imread('costco.jpg')&#xA;text = pytesseract.image_to_string(img)&#xA;print(text)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Cadbury Chocolate&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;variety pack packet&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;ere &lt;span class=&quot;math-container&quot;&gt;$12.99 i rom hagst 31026 2012&#xA;&amp;gt; &#xA;&amp;gt; Je laa&#xA;&amp;gt; + a&#xA;&amp;gt; &#xA;&amp;gt; Wrigley’s Excel Gum variety&#xA;&amp;gt; &#xA;&amp;gt; Backol 24&#xA;&amp;gt; &#xA;&amp;gt; $&lt;/span&gt;13.79 fom agus 26.202&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;OFF&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Solon Extra virgin olive oil [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which is a lot noisy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that splitting the image to its base squares enchances the recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I have a sliding window and train several &quot;grid box&quot; objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?&lt;/p&gt;&#xA;" OwnerUserId="31316" LastActivityDate="2019-11-15T22:14:24.633" Title="Best approach for 2D Grid Image Segmentation" Tags="&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16551" PostTypeId="2" ParentId="16550" CreationDate="2019-11-15T22:14:24.633" Score="2" Body="&lt;p&gt;This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Grouping text based on locality. &quot;no segmentation&quot; &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0031320302000821&quot; rel=&quot;nofollow noreferrer&quot; &gt; Text region extraction in a document image based on the Delaunay tessellation &lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Segmentation &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/4036951&quot; rel=&quot;nofollow noreferrer&quot; &gt; Multiscale Edge-Based Text Extraction from Complex Images &lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Training a map of the image then extracting text form the map. &lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2016/html/Zhang_Multi-Oriented_Text_Detection_CVPR_2016_paper.html&quot; rel=&quot;nofollow noreferrer&quot; &gt; Multi Oriented Text Detection &lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former. &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/995125&quot; rel=&quot;nofollow noreferrer&quot; &gt; automatic image segmentation and edge detection &lt;/a&gt; Good luck!&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-15T22:14:24.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16552" PostTypeId="2" ParentId="16548" CreationDate="2019-11-16T04:48:52.337" Score="1" Body="&lt;p&gt;The first place I would have directed you would be Sklearn and pydiffmap. I found this paper specifically about the problem you are doing using python the reference a package called &lt;a href=&quot;http://www.jmlr.org/papers/volume17/16-109/16-109.pdf&quot; rel=&quot;nofollow noreferrer&quot; &gt; megaman &lt;/a&gt; It seems like an active &lt;a href=&quot;https://github.com/mmp2/megaman&quot; rel=&quot;nofollow noreferrer&quot; &gt; Github &lt;/a&gt;. I suggest not just looking at manifold learning papers but leaning towards a search toward non linear embedding or non linear dimensionality reduction as well. The paper you give doesn't reference local linear embedding but seems to be similar conceptually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this &lt;a href=&quot;http://sebastianraschka.com/Articles/2014_kernel_pca.html&quot; rel=&quot;nofollow noreferrer&quot; &gt;  kernel pca document &lt;/a&gt; which is very straight forward and I think will help you going forward as well. I believe the RBF Kernel PCA part may be of use to you at least as a way of benchmarking how well your diffusion map is functioning.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-11-16T04:48:52.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16553" PostTypeId="1" CreationDate="2019-11-16T10:36:31.120" Score="3" ViewCount="30" Body="&lt;p&gt;&lt;strong&gt;Problem Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working with a problem, which requires a character-level, deep learning model. Previously I was working with word-level deep NLP (Natural Language Processing) models, and in these models almost always embedding encoding was used to represent given word in a lower-dimensional vector form. Furthermore, such embedding encoding allowed for putting similar words near themselves in the new lower-dimensional vector representation (i.e. man and woman vectors were near themselves in the vector space) which improved learning. Nevertheless, I often see that people use embedding encoding in character level NLP models. Even if the character-level one-hot encoding vectors are quite small in comparison to word-level one-hot encoding vectors (about 36 to 32k rows). Furthermore, there is no much correlation between characters, there is no something like &quot;similar characters&quot; in comparison to similar words, therefore some characters in comparison to other shouldn't be put near themselves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&#xA;Why embedding encoding is used in the character-level NLP models?&lt;/p&gt;&#xA;" OwnerUserId="31324" LastActivityDate="2019-11-16T10:36:31.120" Title="Why embedding layer is used in the character-level Natural Language Processing models" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16554" PostTypeId="1" CreationDate="2019-11-16T10:50:06.207" Score="2" ViewCount="39" Body="&lt;p&gt;I am trying to make a neural network which takes in 0 and 1 as it's input and should give me output ranging from [-20,-1].I am using three layers with sigmoid as the activation function .How should I design my output layer?Any sort of code snippet from your side will be helpful .I am using tensorflow.Please help me out with the same&lt;/p&gt;&#xA;" OwnerUserId="31325" LastActivityDate="2019-11-16T10:50:06.207" Title="How should I make output layer of my neural network so that I can get outputs ranging from [-20,-1]" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16555" PostTypeId="1" CreationDate="2019-11-16T10:56:13.993" Score="1" ViewCount="34" Body="&lt;p&gt;I'm wondering how is the general return-based off-policy equation in &lt;a href=&quot;http://papers.nips.cc/paper/6538-safe-and-efficient-off-policy-reinforcement-learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Safe and efficient off-policy reinforcement learning&lt;/a&gt; derived&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{R} Q(x, a):=Q(x, a)+\mathbb{E}_{\mu}\left[\sum_{t \geq 0} \gamma^{t}\left(\prod_{s=1}^{t} c_{s}\right)\left(r_{t}+\gamma \mathbb{E}_{\pi} Q\left(x_{t+1}, \cdot\right)-Q\left(x_{t}, a_{t}\right)\right)\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is applied to TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;), is this equation the forward view of TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between trace &lt;span class=&quot;math-container&quot;&gt;$c_s$&lt;/span&gt; and eligibility trace?&lt;/p&gt;&#xA;" OwnerUserId="15525" LastEditorUserId="2444" LastEditDate="2020-06-04T17:09:23.567" LastActivityDate="2020-06-04T17:09:23.567" Title="How is the general return-based off-policy equation derived?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;temporal-difference-methods&gt;&lt;eligibility-traces&gt;&lt;td-lambda&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16556" PostTypeId="1" AcceptedAnswerId="16579" CreationDate="2019-11-16T11:50:59.470" Score="4" ViewCount="2193" Body="&lt;p&gt;In the following paragraph from the book &lt;a href=&quot;https://link.springer.com/book/10.1007%2F978-3-030-05318-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automated Machine Learning: Methods, Systems, Challenges&lt;/a&gt; (by Frank Hutter et al.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this section we first give a brief introduction to Bayesian optimization, present alternative surrogate models used in it, describe extensions to conditional and constrained configuration spaces, and then discuss several important applications to hyperparameter optimization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is an &quot;alternative surrogate model&quot;? What exactly does &quot;alternative&quot; mean?&lt;/p&gt;&#xA;" OwnerUserId="17410" LastEditorUserId="2444" LastEditDate="2019-11-17T20:21:37.657" LastActivityDate="2021-04-15T12:31:56.913" Title="What is a &quot;surrogate model&quot;?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;hyperparameter-optimization&gt;&lt;bayesian-optimization&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16557" PostTypeId="1" AcceptedAnswerId="16558" CreationDate="2019-11-16T12:06:10.730" Score="3" ViewCount="84" Body="&lt;p&gt;I have a large set of data points describing mappings of binary vectors to real-valued outputs. I am using TensorFlow, and would like to train a model to predict these relationships. I used four hidden layers with 500 neurons in each layer, and sigmoidal activation functions in each layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network appears to be unable to learn, and has high loss even on the training data. What might cause this to happen? Is there something wrong with the design of my network?&lt;/p&gt;&#xA;" OwnerUserId="31325" LastEditorUserId="2444" LastEditDate="2019-11-19T02:41:50.973" LastActivityDate="2019-11-19T02:41:50.973" Title="What could be the problem when a neural network with four hidden layers with the sigmoid activation function is not learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;activation-function&gt;&lt;hidden-layers&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16558" PostTypeId="2" ParentId="16557" CreationDate="2019-11-16T12:41:08.467" Score="0" Body="&lt;p&gt;When training our neural network, you need to scale your dataset in order to avoid slowing down the learning or prevent effective learning.&#xA;Try normalizing your output.&#xA;This &lt;a href=&quot;https://machinelearningmastery.com/how-to-scale-data-for-long-short-term-memory-networks-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tutorial &lt;/a&gt;might help&lt;/p&gt;&#xA;" OwnerUserId="26854" LastActivityDate="2019-11-16T12:41:08.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16560" PostTypeId="2" ParentId="16556" CreationDate="2019-11-16T16:53:21.590" Score="4" Body="&lt;p&gt;A surrogate model is a simplified model. It is a mapping &lt;span class=&quot;math-container&quot;&gt;$y_S=f_S(x)$&lt;/span&gt; that approximates the original model &lt;span class=&quot;math-container&quot;&gt;$y=f(x)$&lt;/span&gt;, in a given domain, reasonably well. Source: &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470770801&quot; rel=&quot;nofollow noreferrer&quot;&gt;Engineering Design via Surrogate Modelling: A Practical Guide&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of Bayesian optimization, one wants to optimize a function &lt;span class=&quot;math-container&quot;&gt;$y=f(x)$&lt;/span&gt; which is expensive (very time consuming) to evaluate, therefore one optimizes the surrogate model &lt;span class=&quot;math-container&quot;&gt;$y_S=f_S(x)$&lt;/span&gt; which is cheaper (faster) to evaluate.&lt;/p&gt;&#xA;" OwnerUserId="31335" LastEditorUserId="2444" LastEditDate="2019-11-19T16:49:58.793" LastActivityDate="2019-11-19T16:49:58.793" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16562" PostTypeId="2" ParentId="16336" CreationDate="2019-11-16T20:36:56.867" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;logp&lt;/code&gt; seen in code is actually &lt;code&gt;logit p&lt;/code&gt; which has this story behind:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given a probability p, the corresponding odds are calculated as p / (1 – p). For example if p=0.75, the odds are 3 to 1: 0.75/0.25 = 3.&lt;/p&gt;&#xA;&lt;p&gt;The logit function is simply the logarithm of the odds: logit(x) = log(x / (1 – x)).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Sigmoid near &lt;code&gt;logp&lt;/code&gt; is like follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The inverse of the logit function is the sigmoid function. That is, if you have a probability p, sigmoid(logit(p)) = p.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Source: [1]&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;In reinforcement learning we know in the end of game, if taken actions were successful or not. Then before next round we can adjust the gradients. From your link (commentary section):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For example in Pong we could wait until the end of the game, then take the reward we get (either +1 if we won or -1 if we lost), and enter that scalar as the gradient for the action we have taken (DOWN in this case). In the example below, going DOWN ended up to us losing the game (-1 reward). So if we fill in -1 for log probability of DOWN and do backprop we will find a gradient that discourages the network to take the DOWN action for that input in the future (and rightly so, since taking that action led to us losing the game).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;In the very same commentary section (later) there is a pic, and explanation of what h is. Unfortunately, you have to check it by yourself, pic was not compatible to be attached here. By describing the pic I could say h is equal to weights in hidden layer and in gradient case the dh is the derivative of h.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Roughly speaking the backpropagation is correcting the weights backwards the network after the round is done. More thorough explanation is in the mentioned comments section.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.google.com/amp/s/nathanbrixius.wordpress.com/2016/06/04/functions-i-have-known-logit-and-sigmoid/amp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.google.com/amp/s/nathanbrixius.wordpress.com/2016/06/04/functions-i-have-known-logit-and-sigmoid/amp/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-16T20:36:56.867" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16563" PostTypeId="2" ParentId="16496" CreationDate="2019-11-16T21:48:04.700" Score="1" Body="&lt;p&gt;I believe you may want to use a &lt;a href=&quot;https://arxiv.org/pdf/1202.3732.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sum Product Network&lt;/a&gt; for this task. SPNs are the state-of-the-art approach for face completion, and there are several more &lt;a href=&quot;https://arxiv.org/pdf/1902.06155.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent papers&lt;/a&gt; on this topic since the original above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Importantly, the SPN paper also covers other approaches that work well for this task. If lower-resolution results are acceptable for your task, PCA with 100 or more components works surprisingly well. If your dataset is very large, a nearest-neighbor approach can work decently too.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-16T21:48:04.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16564" PostTypeId="1" AcceptedAnswerId="16565" CreationDate="2019-11-17T00:09:42.150" Score="2" ViewCount="193" Body="&lt;p&gt;I was making a simple phoneme classification model for a 10 week-long class project and I ran into a small question. Is it possible, to create a model that takes a 1-second (the longest phoneme is 0.2 second but the large image is kept for context) spectrogram as input? Some people suggest creating an RNN for phoneme classification but can you build a pure CNN phoneme classification model?&#xA;Any responses would be great.&lt;/p&gt;&#xA;" OwnerUserId="23546" LastActivityDate="2019-11-17T01:52:23.817" Title="Making a CNN for phoneme classification" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16565" PostTypeId="2" ParentId="16564" CreationDate="2019-11-17T01:52:23.817" Score="1" Body="&lt;p&gt;Yes you can, a few years ago I made a simple &lt;a href=&quot;https://www.computer.org/csdl/proceedings-article/snpd/2017/08022720/12OmNxWuihg&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNN for a single Arabic phoneme classification&lt;/a&gt;. You can use &lt;a href=&quot;https://arxiv.org/abs/1712.00866&quot; rel=&quot;nofollow noreferrer&quot;&gt;spectogram&lt;/a&gt; or using &lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/CNN_ASLPTrans2-14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MFCC / MFSC&lt;/a&gt; as features, as long all data has the same size (use padding or cropping if needed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may need RNN if you want to combine some phonemes to recognize a single word or longer.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-11-17T01:52:23.817" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="16566" PostTypeId="1" AcceptedAnswerId="16568" CreationDate="2019-11-17T03:06:57.037" Score="3" ViewCount="107" Body="&lt;p&gt;I just find that Google patents some of the widely used machine learning algorithms. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://patents.google.com/patent/US9406017B2/en&quot; rel=&quot;nofollow noreferrer&quot;&gt;System and method for addressing overfitting in a neural network&lt;/a&gt; (Dropout?)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://patents.google.com/patent/US9715642B2/en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Processing images using deep neural networks&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://patents.google.com/patent/US20150100530&quot; rel=&quot;nofollow noreferrer&quot;&gt;Methods and apparatus for reinforcement learning&lt;/a&gt; (Q-Learning?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is that mean I can't use those algorithms commercially? &lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-11-18T12:00:50.870" Title="Can Google's patented ML algorithms be used commercially?" Tags="&lt;q-learning&gt;&lt;deep-neural-networks&gt;&lt;legal&gt;&lt;dropout&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16567" PostTypeId="1" CreationDate="2019-11-17T03:55:28.027" Score="1" ViewCount="29" Body="&lt;h2&gt;My Question&lt;/h2&gt;&#xA;&lt;p&gt;Would it be viable to implement and also how would I go about implementing something like this, &amp;quot;AI Agent Rating State-machine&amp;quot;?&lt;/p&gt;&#xA;&lt;h2&gt;Problem and constraints:&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;An AI Agent is tasked with making a &lt;strong&gt;cup of tea&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the environment there is a &lt;strong&gt;baby in the way&lt;/strong&gt; of the Agent.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The AI Agent has an &lt;strong&gt;off switch&lt;/strong&gt; on its stomach/front-side&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Problem from &lt;a href=&quot;https://www.youtube.com/watch?v=3TYT1QfdfsM&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI Stop Button, a 20 minute Video&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Solution&lt;/h2&gt;&#xA;&lt;p&gt;Could this possibly be solved by implementing action rating, with a internal-statemachine?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Power &lt;strong&gt;Off button&lt;/strong&gt; give &lt;strong&gt;10 points&lt;/strong&gt; reward&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;making a &lt;strong&gt;cup of tea&lt;/strong&gt; give &lt;strong&gt;20 points&lt;/strong&gt; reward&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Avoiding the &lt;strong&gt;baby&lt;/strong&gt; give &lt;strong&gt;50 points&lt;/strong&gt; reward&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However The AI is tasked by its &lt;strong&gt;utility function&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The Utility function should be in a &amp;quot;greedy&amp;quot; mode, meaning that it will order tasks by rating each one with what is more important to us Humans.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Solution Implementation:&lt;/h2&gt;&#xA;&lt;p&gt;AI Agent Talks to Sub System, This subsystem could theoretically govern what actions the AI Agent may take and also provide a &amp;quot;subconscious&amp;quot; like system by implemented this subsystem (mostly hard coded)&lt;/p&gt;&#xA;" OwnerUserId="1282" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-17T03:55:28.027" Title="Greedy Algorithm with State modelling" Tags="&lt;ai-design&gt;&lt;ai-basics&gt;&lt;generalization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16568" PostTypeId="2" ParentId="16566" CreationDate="2019-11-17T04:00:20.790" Score="1" Body="&lt;p&gt;&lt;strong&gt;Can you use them commercially?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is Google able to sue you any time they want?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Will they do that...&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google isn't a known patent bully, I would give them the benefit of the doubt in this kind of situation and say, unless you start really giving them real trouble, they wouldn't do anything. Some companies/people know an idea can really be used for good and patent it to protect its use not to inhibit its use. By patenting the idea and setting a precedent of not suing they are effectively allowing everyone to benefit. Maybe in the future Google cloud, Azure and Amazon web services will lose some money in a legal battle, but I doubt you personally will be hit with a lawsuit.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="2444" LastEditDate="2019-11-18T12:00:50.870" LastActivityDate="2019-11-18T12:00:50.870" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16570" PostTypeId="1" AcceptedAnswerId="16580" CreationDate="2019-11-17T10:31:05.017" Score="5" ViewCount="489" Body="&lt;p&gt;I want to prevent my model from overfitting. I think that k-fold cross-validation (because it is doing this each time with different datasets) may be more effective than splitting the dataset into training and test datasets to prevent overfitting, but a colleague (who has little experience in ML) says that, to prevent overfitting, the 70/30% split performs better than the cross-validation. In my opinion, k-fold cross-validation provides a reliable method to test the model performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting? &#xA;I am not concerned with computational resources.&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-18T12:15:53.097" LastActivityDate="2019-11-18T13:32:52.340" Title="Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;overfitting&gt;&lt;cross-validation&gt;" AnswerCount="3" CommentCount="12" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16571" PostTypeId="2" ParentId="16570" CreationDate="2019-11-17T10:49:32.847" Score="3" Body="&lt;p&gt;&lt;strong&gt;Purely in terms of overfitting&lt;/strong&gt;, and assuming you train both for equal amounts of time, 70/30 is probably better but performance is not going to be very good. Not training on %30 of data will make both training and test results equally bad (in my opinion). But it won't overfit, that is for sure. Cross validation (you have in mind 90/10, I assume) will take a long time, so that won't have enough time to train and it might be overfitting more compared to 70/30, but as it is going to see all training samples %90 at a time, there is a good chance it will train better. So, at the end of the day, it will overfit more but perform better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are asking which is better overall, performance and overfitting, I say it depends on the size of your dataset. If you have millions of samples in it, you can even use a 98/1/1 for training, testing and validation and &lt;a href=&quot;https://www.coursera.org/learn/deep-neural-network/lecture/cxG1s/train-dev-test-sets&quot; rel=&quot;nofollow noreferrer&quot;&gt;still be OK&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Thinking a little more about it, even if the time is not an issue the situation will roughly be the same. But you will know the performance of the model on new data to a higher certainty with cross validation. &lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-11-17T13:35:55.097" LastActivityDate="2019-11-17T13:35:55.097" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16574" PostTypeId="2" ParentId="16570" CreationDate="2019-11-17T15:35:30.850" Score="0" Body="&lt;p&gt;Both methods are fine if used properly. As a rule of thumb, when training time is not an issue, use split method if you have more data than you can use in your model and cross-validation if not. I would suggest handling overfitting by some other means.&lt;/p&gt;&#xA;" OwnerUserId="3579" LastActivityDate="2019-11-17T15:35:30.850" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16575" PostTypeId="1" AcceptedAnswerId="16589" CreationDate="2019-11-17T16:10:57.923" Score="4" ViewCount="704" Body="&lt;p&gt;In simple words, what does end-to-end training mean, in the context of deep learning?&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="2444" LastEditDate="2020-03-13T16:29:01.873" LastActivityDate="2020-03-13T17:41:21.833" Title="What does end-to-end training mean?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;terminology&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16576" PostTypeId="1" AcceptedAnswerId="16587" CreationDate="2019-11-17T17:27:06.503" Score="1" ViewCount="97" Body="&lt;p&gt;I'm studying a master's degree and my final work is going to be about the convolutional neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read a lot of books and I did Convolutional Network Standford's course, but I need more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there books or papers on the details of convolutional neural networks (in particular, convolutional layer)? &lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2021-01-17T19:32:10.800" LastActivityDate="2021-01-17T19:32:10.800" Title="What are examples of books or papers on the details of convolutional neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;papers&gt;&lt;books&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16577" PostTypeId="2" ParentId="16576" CreationDate="2019-11-17T17:39:45.187" Score="1" Body="&lt;p&gt;Chris Olah's work is always inspired, and not too technical as one would expect. He has several papers on CNNs on his &lt;a href=&quot;https://colah.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;. In particular, check the series titled &quot;Convolutional Neural Networks&quot; with four papers on the topic.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-11-17T23:42:08.770" LastActivityDate="2019-11-17T23:42:08.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16578" PostTypeId="1" CreationDate="2019-11-17T17:53:50.260" Score="1" ViewCount="53" Body="&lt;p&gt;Given the following 3 research papers, the authors have shown different heatmap graphical representations for features of the trained CNN models:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;On the performance of Convnet feature for place recognition: &lt;a href=&quot;https://arxiv.org/abs/1501.04158&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2rvFL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2rvFL.png&quot; alt=&quot;heatmap_1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;NetVLAD: CNN architecture for weakly supervised place recognition: &lt;a href=&quot;https://arxiv.org/abs/1511.07247&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/RGHTj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RGHTj.png&quot; alt=&quot;heatmap_2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Deep Learning Features at Scale for Visual Place Recognition: &lt;a href=&quot;https://arxiv.org/abs/1701.05105&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ka6Ei.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ka6Ei.png&quot; alt=&quot;heatmap_3&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does anyone know the easiest heatmap implementation in Python given &lt;code&gt;deploy.protxt&lt;/code&gt; and &lt;code&gt;model_wights_bias.caffemodel&lt;/code&gt; files? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I am aware of the following answers and packages: &lt;a href=&quot;https://stackoverflow.com/questions/46020894/how-to-superimpose-heatmap-on-a-base-image&quot;&gt;answer1&lt;/a&gt;, &lt;a href=&quot;https://github.com/metalbubble/CAM&quot; rel=&quot;nofollow noreferrer&quot;&gt;package1&lt;/a&gt; but they do not provide these solutions shown in figures above!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="31312" LastActivityDate="2019-11-17T17:53:50.260" Title="Plot class activation heatmap of Caffe Model in Python" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;feature-extraction&gt;&lt;feature-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16579" PostTypeId="2" ParentId="16556" CreationDate="2019-11-17T18:40:01.317" Score="4" Body="&lt;h2&gt;What is &lt;a href=&quot;http://krasserm.github.io/2018/03/21/bayesian-optimization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian optimization&lt;/a&gt;?&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Introduction&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian optimization (BO)&lt;/a&gt; is an optimization technique used to model an &lt;strong&gt;unknown&lt;/strong&gt; (&lt;a href=&quot;https://arxiv.org/pdf/1807.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;usually continuous&lt;/a&gt;) function &lt;span class=&quot;math-container&quot;&gt;$f: \mathbb{R}^d \rightarrow Y$&lt;/span&gt;, where &lt;a href=&quot;https://arxiv.org/pdf/1807.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;typically &lt;span class=&quot;math-container&quot;&gt;$d \leq 20$&lt;/span&gt;&lt;/a&gt;, so it can be used to solve regression and classification problems, where you want to find an approximation of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. In this sense, BO is similar to the usual approach of training a neural network with gradient descent combined with the back-propagation algorithm, so that to optimize an objective function. However, BO is particularly suited for regression or classification problems where &lt;strong&gt;the unknown function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is expensive to evaluate&lt;/strong&gt; (that is, given the input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \in \mathbb{R}^d$&lt;/span&gt;, the computation of &lt;span class=&quot;math-container&quot;&gt;$f(x) \in Y$&lt;/span&gt; takes a lot of time or, in general, resources). For example, when doing hyper-parameter tuning, we usually need to first train the model with the new hyper-parameters before evaluating the specific configuration of hyper-parameters, but this usually takes a lot of time (hours, days or even months), especially when you are training deep neural networks with big datasets. Moreover, BO does not involve the computation of gradients and &lt;a href=&quot;https://arxiv.org/pdf/1807.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;it usually assumes that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; lacks properties such as concavity or linearity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How does Bayesian optimization work?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;There are three main concepts in BO&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;strong&gt;surrogate model&lt;/strong&gt;, which models an unknown function, &lt;/li&gt;&#xA;&lt;li&gt;a method for &lt;strong&gt;statistical inference&lt;/strong&gt;, which is used to update the surrogate model, and&lt;/li&gt;&#xA;&lt;li&gt;the &lt;strong&gt;acquisition function&lt;/strong&gt;, which is used to guide the statistical inference and thus it is used to update the surrogate model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The surrogate model is usually a &lt;a href=&quot;https://distill.pub/2019/visual-exploration-gaussian-processes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian process&lt;/a&gt;, which is just a fancy name to denote a &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;collection of random variables such that the joint distribution of those random variables is a multivariate &lt;em&gt;Gaussian probability distribution&lt;/em&gt;&lt;/a&gt; (hence the name &lt;em&gt;Gaussian&lt;/em&gt; process). Therefore, in BO, we often use a Gaussian probability distribution (the surrogate model) to model the possible functions that are consistent with the data. In other words, given that we do not know &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, rather than finding the usual &lt;em&gt;point estimate&lt;/em&gt; (or maximum likelihood estimate), like in the usual case of supervised learning mentioned above, we maintain a Gaussian probability distribution that describes our uncertainty about the unknown &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method of statistical inference is often just an iterative application of the Bayes rule (hence the name &lt;em&gt;Bayesian&lt;/em&gt; optimization), where you want to find the posterior, given a prior, a likelihood and the evidence. In BO, you usually place a prior on &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, which is a multivariate Gaussian distribution, then you use the Bayes rule to find the posterior distribution of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; given the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the data in this case? In BO, the data are the outputs of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; evaluated at certain points of the domain of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The acquisition function is used to choose these points of the domain of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, based on the computed posterior distribution. In other words, based on the current uncertainty about &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (the posterior), the acquisition function attempts to cleverly choose points of the domain of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \in \mathbb{R}^d$&lt;/span&gt;, which will be used to find an updated posterior.  Why do we need the acquisition function? Why can't we simply evaluate &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; at random domain points? Given that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is expensive to evaluate, we need a clever way to choose the points where we want to evaluate &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. More specifically, we want to evaluate &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; where we are more uncertain about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several acquisition functions, such as &lt;a href=&quot;https://arxiv.org/pdf/1807.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;expected improvement, knowledge-gradient, entropy search&lt;/em&gt;, and &lt;em&gt;predictive entropy search&lt;/em&gt;&lt;/a&gt;, so there are different ways of choosing the points of the domain of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; where we want to evaluate it to update the posterior, each of which deals with the exploration-exploitation dilemma differently.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What can Bayesian optimization be used for?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;BO can be used for tuning hyper-parameters (also called hyper-parameter optimisation) of machine learning models, such as neural networks, but &lt;a href=&quot;https://arxiv.org/pdf/1807.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;it has also been used to solve other problems&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What is an alternative surrogate model?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the book &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007%2F978-3-030-05318-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automated Machine Learning: Methods, Systems, Challenges&lt;/a&gt; (by Frank Hutter et al.) that you are quoting, the authors say that the commonly used surrogate model &lt;em&gt;Gaussian process&lt;/em&gt; scales cubically in the number of data points, so sparse Gaussian processes are often used. Moreover, Gaussian processes also scale badly with the number of dimensions. In &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007%2F978-3-030-05318-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 1.3.2.2.&lt;/a&gt;, the authors describe some &lt;a href=&quot;https://www.merriam-webster.com/dictionary/alternative&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;alternative&lt;/strong&gt;&lt;/a&gt; surrogate models to the Gaussian processes, for example, alternatives that use neural networks or random forests.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-17T19:56:47.560" LastActivityDate="2019-11-17T19:56:47.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16580" PostTypeId="2" ParentId="16570" CreationDate="2019-11-17T18:52:04.453" Score="4" Body="&lt;p&gt;K-fold cross-validation is probably preferred in terms of completeness and generalization: you ensure that the system has seen the complete dataset for training. However, in deep learning this is often not feasible due to time and power constraints. They can both be used, and there is not one &lt;em&gt;better&lt;/em&gt; than the other. It really depends on the specific case, the size of the dataset and the time and hardware available. Note that overfitting can be (partially) remedied by things such as dropout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be fair: it is fine to have a discussion about this with your colleagues, but as so often there is no one correct answer. If you &lt;em&gt;really&lt;/em&gt; want proof, you can test it out and compare them. But performance-wise (i.e. the model's predictive power), the difference will be small.&lt;/p&gt;&#xA;" OwnerUserId="29995" LastEditorUserId="29995" LastEditDate="2019-11-18T13:32:52.340" LastActivityDate="2019-11-18T13:32:52.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16581" PostTypeId="1" CreationDate="2019-11-17T18:54:36.433" Score="3" ViewCount="28" Body="&lt;p&gt;Hope you're doing great&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing a personal project for android, I want to make an app in Android Studio. The App will be an AI Assistant Like Siri, Jarvis , etc. The idea in mind is to make the app able to have this features : &#xA;Voice Recognition and ability to respond &#xA;Text Recognition and the ability to response to text chats &#xA;Some Translation Skills &#xA;and maybe the ability to understand phrases and syntax and lookup a google search for example for something that is asked &#xA;answer basic questions like how old are you, what is your name , for age answer maybe it can update since app release on playstore for example?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've Started actually the project today and it's still in the early process I've got my first run on my android with Android Studio and the right SDKtools and plugins also started implementing firebase features and other things now im working on improving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please Recommend me some Features to add, Ideas to make it better and more intuitive and responding, more APIS to use and more options. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that you can help me with the best your imagination can explore.&#xA;Thanks for your time reading and sorry for the chatter. &lt;/p&gt;&#xA;" OwnerUserId="31355" LastActivityDate="2019-11-17T18:54:36.433" Title="What Are the key Plugins And Features to implement in AI Assistant Personal Project?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16583" PostTypeId="5" CreationDate="2019-11-17T20:23:33.443" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-17T20:23:33.443" LastActivityDate="2019-11-17T20:23:33.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16584" PostTypeId="4" CreationDate="2019-11-17T20:23:33.443" Score="0" Body="For questions related to Bayesian optimization (BO), which is a technique used to model an unknown function (that is expensive to evaluate), based on concepts of a surrogate model (which is usually a Gaussian process, which models the unknown function), Bayesian inference (to update the Gaussian process) and an acquisition function (which guides the Bayesian inference). BO can be used for hyper-parameter optimization." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-17T20:23:33.443" LastActivityDate="2019-11-17T20:23:33.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16585" PostTypeId="2" ParentId="16575" CreationDate="2019-11-17T21:59:01.227" Score="2" Body="&lt;p&gt;This is relevant when you have two or more neural networks serving as components to a larger architecture. Training this architecture in an end-to-end manner means simultaneously training all components (i.e. training it as a single network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best example I can think of are image captioning architectures. These usually comprise of two networks: a CNN whose role is to extract features from the input images and a RNN that accepts the CNN's features and generates the output captions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://shenxiaohai.me/2018/10/22/pytorch-tutorial-advanced-04/model.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have two options for training: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;First, train the CNN first for some arbitrary task (e.g. image classification) in hopes that it learns how to extract features. Then use the CNN to extract features from the input images and use those as inputs to train the RNN. This procedure trains the two components in two &lt;strong&gt;completely separate phases&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Treat the whole architecture as a single network and backpropagete the gradients to the CNN so that it also can be trained. This procedure trains the two components &lt;strong&gt;simultaneously&lt;/strong&gt;. This is what we call &lt;strong&gt;end-to-end&lt;/strong&gt; training.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2019-11-17T21:59:01.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16586" PostTypeId="2" ParentId="16576" CreationDate="2019-11-17T22:30:06.063" Score="0" Body="&lt;p&gt;I'm not sure if this is what you are looking for but I find Goodfellow's book a pretty good resource:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goodfellow, specifically Section 2, Chapter 9 deals with convolutional neural networks: &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.deeplearningbook.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Pattern Recognition and Machine Learning' by Bishop Might contains a section (5.5.5, pg 267 onwards) as well as an exercise, and a general discussion about neural networks in image recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you edit your question to post a bit more detail, we can offer better answers, for example, what is about the convolutional layer?  How it's implemented?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for a more basic introduction to convolutional layers I would also suggest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way&lt;/a&gt; gives a pretty general overview, starting at the difference between CNNs and ANNs and explains why CNNs are superior to ANNs (for certain problems).  It also gives some details about how the convolution actually works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/transpose-convolution-77818e55a123&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demystifying the transpose convolution&lt;/a&gt; explains the transpose convolution operation in the context of how a traditional convolution; this may not be relevant if you are strictly using CNNs and not transpose-CNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@RaghavPrabhu/understanding-of-convolutional-neural-network-cnn-deep-learning-99760835f148&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding of Convolutional Neural Network (CNN) — Deep Learning&lt;/a&gt; is quite similar to &quot;A Comprehensive...&quot; link above, but it also includes information about filtering and shows the effect that different filters have on an image, which is certainly very import to an understanding of why we use CNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/building-a-convolutional-neural-network-cnn-in-keras-329fbbadc5f5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building a Convolutional Neural Network (CNN) in Keras&lt;/a&gt; (or one of the other thousand similar pages) are pretty good for just starting out and building your own CNN classifier.  You can also check out examples from Keras, e.g. &lt;a href=&quot;https://keras.io/examples/cifar10_cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIFAR10 CNN&lt;/a&gt;, but these tend to give you a very little information about &lt;em&gt;why&lt;/em&gt; they designed the network the way that they did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, on the other hand, you are looking for some more advanced resources, here are is one that springs to mind:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1512.03385&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Residual Learning for Image Recognition&lt;/a&gt; by He et al., deals with a major advance in image recognition, using Residual Networks (ResNet).  This type of network has become pretty popular, so I highly recommend giving it a read.&lt;/p&gt;&#xA;" OwnerUserId="31360" LastEditorUserId="31360" LastEditDate="2019-11-20T20:06:19.210" LastActivityDate="2019-11-20T20:06:19.210" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16587" PostTypeId="2" ParentId="16576" CreationDate="2019-11-17T22:31:59.970" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 9&lt;/a&gt; of the book &lt;a href=&quot;https://www.deeplearningbook.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; (2016), by Goodfellow et al., describes the convolutional (neural) network (CNN), its main operations (namely, convolution and pooling) and properties (such as parameter sharing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also the article &lt;a href=&quot;http://gregorygundersen.com/blog/2017/02/24/cnns/&quot; rel=&quot;nofollow noreferrer&quot;&gt;From Convolution to Neural Network&lt;/a&gt;, which first introduces the mathematical operation &lt;em&gt;convolution&lt;/em&gt; and then describes its connection with signal processing (where images can be viewed as 2D signals) and, finally, describes the CNN.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-11-17T22:31:59.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16588" PostTypeId="1" AcceptedAnswerId="16591" CreationDate="2019-11-18T01:41:27.523" Score="2" ViewCount="1410" Body="&lt;p&gt;I am trying to build a CNN model on Keras. The data has a dimension of &lt;code&gt;921 rows × 10000 columns&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import keras&#xA;from keras.models import Sequential&#xA;from keras.layers import Dense, Dropout, Flatten, Activation&#xA;from keras.optimizers import SGD&#xA;&#xA;from sklearn.model_selection import train_test_split&#xA;X_train, X_test, y_train, y_test = train_test_split(merge.iloc[:,166:10166], merge[['Result_cat','Result_cat1']].values, test_size=0.2)&#xA;&#xA;model = Sequential()&#xA;&#xA;model.add(Dense(50, input_shape=(10000,))) &#xA;model.add(Activation(&quot;relu&quot;)) &#xA;model.add(Dense(50, activation='softmax')) &#xA;model.add(Dense(1, activation='linear')) &#xA;&#xA;model.compile(loss=keras.losses.categorical_crossentropy,&#xA;              optimizer=keras.optimizers.SGD(lr=0.01),&#xA;              metrics=['accuracy'])&#xA;&#xA;model.fit(X_train, y_train, batch_size=100, epochs=10)&#xA;score = model.evaluate(X_test, y_test, batch_size=32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I encountered error &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ValueError: Error when checking target: expected dense_3 to have shape (1,) but got array with shape (2,)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am new to Keras and CNN. Can someone please explain to me what this means and how I can fix this? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="31075" LastEditorUserId="22301" LastEditDate="2019-11-18T12:11:09.407" LastActivityDate="2019-11-18T12:11:09.407" Title="ValueError: Error when checking target: expected dense_3 to have shape (1,) but got array with shape (2,)" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16589" PostTypeId="2" ParentId="16575" CreationDate="2019-11-18T06:40:46.437" Score="2" Body="&lt;p&gt;Another explanation of deep learning as an end-to-end framework is in deep learning, pre-processing or feature extraction steps are not necessary. So it only uses a single processing step, which is to train the deep learning model. In other traditional machine learning methods, some separated feature extraction steps usually required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RtqsD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RtqsD.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in image classification, deep learning frameworks like CNN can receive a raw image and then trained to classify it directly. If we didn't use deep learning, we need to extract some features using more steps, like edge detection, corner detection, color histogram, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can also watch &lt;a href=&quot;https://www.coursera.org/lecture/machine-learning-projects/what-is-end-to-end-deep-learning-k0Klk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's explanation here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2019-11-18T06:40:46.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16590" PostTypeId="2" ParentId="16515" CreationDate="2019-11-18T08:29:30.043" Score="1" Body="&lt;p&gt;Some ideas out the top of my head:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of &lt;span class=&quot;math-container&quot;&gt;$dy/dx_2&amp;gt;0$&lt;/span&gt; you could compute the gradient using the chain rule and limit the weights so that the constrain holds&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of &lt;span class=&quot;math-container&quot;&gt;$y + x_5 + x_7 &amp;lt; K$&lt;/span&gt; you could use a clipping function on the output layer?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31335" LastActivityDate="2019-11-18T08:29:30.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16591" PostTypeId="2" ParentId="16588" CreationDate="2019-11-18T08:43:05.847" Score="1" Body="&lt;p&gt;Problem is in the output layer and you are using categorical_crossentropy for a loss function. Quoting &lt;a href=&quot;https://keras.io/losses/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note: when using the categorical_crossentropy loss, your targets&#xA;  should be in categorical format (e.g. if you have 10 classes, the&#xA;  target for each sample should be a 10-dimensional vector that is&#xA;  all-zeros except for a 1 at the index corresponding to the class of&#xA;  the sample). In order to convert integer targets into categorical&#xA;  targets, you can use the Keras utility to_categorical:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would check if the target values are defined consistently. They seem to be defined for sparse_categorical_crossentropy, which requires a dense layer where number of neurons = number of classes of the problem. If they are defined as integers - as required for sparse_categorical_crossentropy, you can convert them to categorical formatting using &lt;strong&gt;to_categorical&lt;/strong&gt; function, also defined in the documentation page linked above; or you can keep them as integers, and change the number of neurons in the final layer to 2, and loss function to sparse_categorical_crossentropy.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-11-18T09:22:32.893" LastActivityDate="2019-11-18T09:22:32.893" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16593" PostTypeId="1" AcceptedAnswerId="16595" CreationDate="2019-11-18T09:07:36.480" Score="5" ViewCount="955" Body="&lt;p&gt;I am reading the paper &lt;a href=&quot;https://arxiv.org/pdf/1806.00685v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction&lt;/a&gt; (2018) by Yunzhe Tao et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper, they use several times the expression &quot;semantic levels&quot;. Some examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;HRHN can adaptively select the relevant exogenous features in different &lt;strong&gt;semantic levels&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;the temporal information is usually complicated and may occur at different &lt;strong&gt;semantic levels&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;The encoder RHN reads the convolved features &lt;span class=&quot;math-container&quot;&gt;$(w_1,w_2,···,w_{T−1})$&lt;/span&gt; and models their temporal dependencies at different &lt;strong&gt;semantic levels&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Then an RHN is used to model the temporal dependencies among convolved input features at different &lt;strong&gt;semantic levels&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What is the &lt;em&gt;semantic level&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="31370" LastEditorUserId="31370" LastEditDate="2019-11-19T07:56:35.793" LastActivityDate="2019-11-19T10:24:29.257" Title="What is the &quot;semantic level&quot;?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16595" PostTypeId="2" ParentId="16593" CreationDate="2019-11-18T11:18:32.330" Score="4" Body="&lt;p&gt;In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text (&quot;semantic&quot; comes from the greek and means &quot;to signify&quot;). The semantic level is therefore generally independent from the syntax and even the language used to convey the message. &#xA;Here is an interesting picture I found on the internet to illustrate my point.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zdWU1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zdWU1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I took some time reading the paper. I think &quot;semantic levels&quot; refers here to the different neural networks layers used for the exogenous features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a modified version of their figure I've drawned to make it clearer:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kB6xD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kB6xD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, from what I have understood so far, the attention coefficients apply to the whole semantic level (which I find not really clearly indicated on their figure)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LHS of their figure would then be better described by this new one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tf6tP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tf6tP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="31374" LastEditorUserId="31374" LastEditDate="2019-11-19T10:24:29.257" LastActivityDate="2019-11-19T10:24:29.257" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16596" PostTypeId="1" CreationDate="2019-11-18T12:18:24.743" Score="1" ViewCount="29" Body="&lt;p&gt;I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the main problems I'm facing is that the encoder + sampler output doesn't fulfill the requirements for a mathematical metric (&lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Metric_(mathematics)&lt;/a&gt;) - is there a known way of how to decrease same-sample distance after encoding + sampling as well as promoting transitivity (triangle inequality) and symmetry?&lt;/p&gt;&#xA;" OwnerUserId="31376" LastActivityDate="2019-11-18T12:18:24.743" Title="Reduce same sample distance in VAE encodings" Tags="&lt;autoencoders&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="16597" PostTypeId="1" AcceptedAnswerId="16626" CreationDate="2019-11-18T14:59:18.097" Score="3" ViewCount="212" Body="&lt;p&gt;I know that classical control systems have been used to solve the problem of the inverted pendulum - &lt;a href=&quot;https://www.youtube.com/watch?v=D3bblng-Kcc&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverted pendulum&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I've seen that people have also used machine learning techniques to solve this nowadays - &lt;a href=&quot;https://www.youtube.com/watch?v=Lt-KLtkDlh8&quot; rel=&quot;nofollow noreferrer&quot;&gt;machine learning of inverted pendulum&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across a video on how to apply a machine learning technique called reinforcement learning on openAI gym - &lt;a href=&quot;https://www.youtube.com/watch?v=C2rpcQyv5bk&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI gym reinforcement learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, can I use this simulation and use it to train a controller for a real-world application of inverted pendulum?&lt;/p&gt;&#xA;" OwnerUserId="31381" LastEditorUserId="1847" LastEditDate="2019-11-19T07:27:48.920" LastActivityDate="2019-11-19T13:58:32.770" Title="Can OpenAI simulations be used in real world applications?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;applications&gt;&lt;open-ai&gt;&lt;control-theory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16598" PostTypeId="1" CreationDate="2019-11-18T15:37:53.450" Score="4" ViewCount="61" Body="&lt;p&gt;Decision trees and random forests may or &lt;em&gt;not&lt;/em&gt; be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article &lt;a href=&quot;https://statistics.berkeley.edu/sites/default/files/tech-reports/666.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using Random Forest to Learn Imbalanced Data&lt;/a&gt;, &lt;a href=&quot;https://stats.stackexchange.com/q/242833/82135&quot;&gt;this Stats SE question&lt;/a&gt; and &lt;a href=&quot;https://towardsdatascience.com/working-with-highly-imbalanced-datasets-in-machine-learning-projects-c70c5f2a7b16&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Medium post&lt;/a&gt;. The information across these sources does not seem to be consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could decision tree learning algorithms cope with imbalanced classes?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-21T02:15:08.360" LastActivityDate="2019-11-21T02:15:08.360" Title="How could decision tree learning algorithms cope with imbalanced classes?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;supervised-learning&gt;&lt;decision-tree&gt;&lt;random-forest&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16599" PostTypeId="1" CreationDate="2019-11-18T16:03:26.940" Score="9" ViewCount="1927" Body="&lt;p&gt;Kaggle is limited to only supervised learning problems. There used to be www.rl-competition.org but they've stopped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything else I can do other than locally trying out different algorithms for various RL problems? &lt;/p&gt;&#xA;" OwnerUserId="30632" LastEditorUserId="37965" LastEditDate="2020-06-18T13:29:13.737" LastActivityDate="2020-06-18T13:30:09.560" Title="Are there any online competitions for Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;kaggle&gt;" AnswerCount="4" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16600" PostTypeId="2" ParentId="16599" CreationDate="2019-11-18T16:54:07.460" Score="2" Body="&lt;p&gt;OpenAI has &lt;a href=&quot;https://github.com/openai/gym/wiki/Leaderboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;leaderboards&lt;/a&gt; for their gym-environments, if you want to compete with other people on runtime and efficiency.&lt;/p&gt;&#xA;" OwnerUserId="30565" LastEditorUserId="26652" LastEditDate="2019-11-19T12:36:20.073" LastActivityDate="2019-11-19T12:36:20.073" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16601" PostTypeId="2" ParentId="16599" CreationDate="2019-11-18T17:17:03.533" Score="5" Body="&lt;p&gt;AICrowd has numerous challenges in the domain, with some very interesting challenges running currently. Here is a short list:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aicrowd.com/challenges?&amp;amp;categories=reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aicrowd.com/challenges?&amp;amp;categories=reinforcement-learning&lt;/a&gt; (Complete list with RL tag)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aicrowd.com/challenges/neurips-2020-procgen-competition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aicrowd.com/challenges/neurips-2020-procgen-competition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aicrowd.com/challenges/neurips-2019-minerl-competition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aicrowd.com/challenges/neurips-2019-minerl-competition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="26961" LastEditorUserId="37962" LastEditDate="2020-06-18T13:30:09.560" LastActivityDate="2020-06-18T13:30:09.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16603" PostTypeId="5" CreationDate="2019-11-18T18:53:00.243" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-18T18:53:00.243" LastActivityDate="2019-11-18T18:53:00.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16604" PostTypeId="4" CreationDate="2019-11-18T18:53:00.243" Score="0" Body="For questions related to the wake-sleep algorithm, which can be used in the context of unsupervised learning for neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-18T18:53:00.243" LastActivityDate="2019-11-18T18:53:00.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16605" PostTypeId="1" CreationDate="2019-11-18T19:23:30.810" Score="2" ViewCount="34" Body="&lt;p&gt;I'm new to Deep Learning. I used &lt;code&gt;Keras&lt;/code&gt; and trained a &lt;code&gt;inception_resnet_v2&lt;/code&gt; model for my binary classification application (fire detection). As suggested from my &lt;a href=&quot;https://ai.stackexchange.com/questions/16268&quot;&gt;previous question&lt;/a&gt; of a &lt;code&gt;non-X&lt;/code&gt; class, I prepared a dataset of 8000 images of fire, and a larger dataset for non-fire (20,000 random images) to make sure the network also sees images of non-fire to perform classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trained the model, but now when trying to load the model and pass images of fire and non-fire ones, it shows same result for all of them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[0. 1.]]&#xA;[[0. 1.]]&#xA;[[0. 1.]]&#xA;[[0. 1.]]&#xA;[[0. 1.]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is going wrong? Am I doing anything wrong? Should I get the result another way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;===============================================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know it's not SO, but this is my prediction code in case it matters:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import print_function&#xA;from keras.models import load_model, model_from_json&#xA;import cv2, os, glob&#xA;import numpy as np&#xA;from keras.preprocessing import image&#xA;&#xA;if __name__ == '__main__':&#xA;    model = load_model('Resnet_26_0.79_model_weights.h5')&#xA;&#xA;    os.chdir(&quot;test&quot;)&#xA;    for file in glob.glob(&quot;*.jpg&quot;):&#xA;        img_path = file&#xA;        img = image.load_img(img_path, target_size=(300, 300))&#xA;        x = image.img_to_array(img)&#xA;        x = np.expand_dims(x, axis=0)&#xA;&#xA;        dictionary = {0: 'non-fire', 1: 'fire'}&#xA;&#xA;        results = model.predict(x)&#xA;        print(results)&#xA;        predicted_class= np.argmax(results)&#xA;        acc = 100*results[0][predicted_class]&#xA;        print(&quot;Network prediction is: file: &quot;+ file+&quot;, &quot;+dictionary[predicted_class]+&quot;, %{:0.2f}&quot;.format(acc))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is the training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.applications.inception_resnet_v2 import InceptionResNetV2, preprocess_input&#xA;from keras.preprocessing.image import ImageDataGenerator&#xA;from keras.layers import Dense, Activation, Flatten, Dropout&#xA;from keras.models import Sequential, Model&#xA;from keras.optimizers import SGD, Adam&#xA;from keras.callbacks import ModelCheckpoint&#xA;from keras.metrics import binary_accuracy&#xA;import os&#xA;import json&#xA;#==========================&#xA;HEIGHT = 300&#xA;WIDTH = 300&#xA;TRAIN_DIR = &quot;data&quot;&#xA;BATCH_SIZE = 8 #8&#xA;steps_per_epoch = 1000 #1000&#xA;NUM_EPOCHS = 50 #50&#xA;lr= 0.00001&#xA;#==========================&#xA;FC_LAYERS = [1024, 1024]&#xA;dropout = 0.5&#xA;&#xA;def build_finetune_model(base_model, dropout, fc_layers, num_classes):&#xA;    for layer in base_model.layers:&#xA;        layer.trainable = False&#xA;&#xA;    x = base_model.output&#xA;    x = Flatten()(x)&#xA;    for fc in fc_layers:&#xA;        # New FC layer, random init&#xA;        x = Dense(fc, activation='relu')(x) &#xA;        x = Dropout(dropout)(x)&#xA;&#xA;    # New layer&#xA;    predictions = Dense(num_classes, activation='sigmoid')(x) &#xA;    finetune_model = Model(inputs=base_model.input, outputs=predictions)&#xA;    return finetune_model&#xA;&#xA;train_datagen =  ImageDataGenerator(preprocessing_function=preprocess_input, rotation_range=90, horizontal_flip=True, vertical_flip=True&#xA;                                    ,validation_split=0.2)&#xA;train_generator = train_datagen.flow_from_directory(TRAIN_DIR, target_size=(HEIGHT, WIDTH), batch_size=BATCH_SIZE&#xA;                                                    ,subset=&quot;training&quot;)&#xA;#split validation manually&#xA;validation_generator = train_datagen.flow_from_directory(TRAIN_DIR, target_size=(HEIGHT, WIDTH), batch_size=BATCH_SIZE,subset=&quot;validation&quot;)&#xA;&#xA;base_model = InceptionResNetV2(weights='imagenet', include_top=False, input_shape=(HEIGHT, WIDTH, 3))&#xA;&#xA;root=TRAIN_DIR&#xA;class_list = [ item for item in os.listdir(root) if os.path.isdir(os.path.join(root, item)) ]&#xA;print (&quot;class_list: &quot;+str(class_list))&#xA;&#xA;finetune_model = build_finetune_model(base_model, dropout=dropout, fc_layers=FC_LAYERS, num_classes=len(class_list))&#xA;&#xA;adam = Adam(lr)&#xA;# change to categorical_crossentropy for multiple classes&#xA;finetune_model.compile(adam, loss='binary_crossentropy', metrics=['accuracy'])&#xA;&#xA;filepath=&quot;./checkpoints/&quot; + &quot;Resnet_{epoch:02d}_{acc:.2f}&quot; +&quot;_model_weights.h5&quot;&#xA;checkpoint = ModelCheckpoint(filepath, monitor=[&quot;val_accuracy&quot;], verbose=1, mode='max', save_weights_only=False)&#xA;callbacks_list = [checkpoint]&#xA;&#xA;history = finetune_model.fit_generator(train_generator, epochs=NUM_EPOCHS, workers=BATCH_SIZE, &#xA;                                    validation_data=validation_generator, validation_steps = validation_generator.samples, &#xA;                                       steps_per_epoch=steps_per_epoch, &#xA;                                       shuffle=True, callbacks=callbacks_list)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-11-19T01:38:46.167" LastActivityDate="2019-11-19T03:09:55.233" Title="Semantic issues with predictions made by my trained model" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;training&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16606" PostTypeId="1" AcceptedAnswerId="16622" CreationDate="2019-11-18T21:02:31.497" Score="1" ViewCount="368" Body="&lt;p&gt;From many blogs and this one &lt;a href=&quot;https://web.archive.org/web/20160308070346/http://mcts.ai/about/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.archive.org/web/20160308070346/http://mcts.ai/about/index.html&lt;/a&gt;&#xA;We know that the process of MCTS algorithm has 4 steps.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Selection: Starting at root node R, recursively select optimal child nodes until a leaf node L is reached. &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does leaf node L mean here? I thought it should be a node representing the terminal state of the game, or another word which ends the game. &#xA;If L is not a terminal node (one end state of the game), how do we decide that the selection step stops on node L? From the terms of general algorithm, a leaf node is the one that does not have any&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Expansion: If L is a not a terminal node (i.e. it does not end the game) then create one or more child nodes and select one C. &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From this description I realise that obviously my previous thought incorrect. &#xA;Then if L is not a terminal node, it implies that L should have children, why not continue finding a child from L at the &quot;Selection&quot; step? &#xA;Do we have the children list of L at this step?&lt;br&gt;&#xA;From the description of this step itself, when do we create one child node, and when do we need to create more than one child nodes? Based on what rule/policy do we select node C? &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Simulation: Run a simulated playout from C until a result is achieved.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because of the confusion of the 1st question, I totally cannot understand why we need to simulate the game. I thought from the selection step, we can reach the terminal node and the game should be ended on node L in this path. We even do not need to do &quot;Expansion&quot; because node L is the terminal node.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Backpropagation: Update the current move sequence with the simulation result. Fine.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Last question, from where did you get the answer to these questions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="31389" LastActivityDate="2019-11-20T01:13:34.433" Title="How to understand the 4 steps of Monte Carlo Tree Search" Tags="&lt;algorithm&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16607" PostTypeId="1" CreationDate="2019-11-19T01:21:45.230" Score="2" ViewCount="61" Body="&lt;p&gt;I have achieved around 85% accuracy using the following architecture:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/97JWn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/97JWn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bgeBW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bgeBW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used a learning rate of 0.001 and trained the model over 125 epochs with a batch size of 64.&#xA;Any suggestions would be much appreciated. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="29877" LastActivityDate="2019-11-21T07:42:11.177" Title="What could I do to this CNN to achieve a higher accuracy on the cifar10 dataset?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16608" PostTypeId="1" CreationDate="2019-11-19T01:55:16.477" Score="1" ViewCount="67" Body="&lt;p&gt;Are there any good ways of simultaneously incorporating object detection with speech recognition? For example, if you want to identify whether an animal is a dog or cat, we can obviously use visual features (e.g. YOLO, CNNs, etc.). But how would you incorporate speech and sound in this model?&lt;/p&gt;&#xA;" OwnerUserId="28201" LastEditorUserId="2444" LastEditDate="2020-09-11T15:07:29.827" LastActivityDate="2021-02-08T17:02:20.260" Title="Are there any good ways of simultaneously incorporating object detection with speech recognition?" Tags="&lt;reference-request&gt;&lt;object-detection&gt;&lt;speech-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16609" PostTypeId="2" ParentId="16557" CreationDate="2019-11-19T01:55:29.953" Score="3" Body="&lt;p&gt;Your code suggests a likely problem here: It looks like you are training a very deep neural network with sigmoidal activation functions at every layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sigmoid has the property that its derivative (S*(1-S)) will be extremely small when the activation function's value is close to 0 or close to 1. In fact, the largest it can be is about 0.25.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The backpropigation algorithm, which is used to train a neural network, will propagate an error signal backwards. At each layer, the error signal will be multiplied by, among other things, the derivative of the activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is therefore the case that by the 4th layer your signal is &lt;em&gt;at most&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$0.25^4 = \frac{1}{256}$&lt;/span&gt; the size that it was at the start of the network. In fact, it is likely much smaller than this. With a smaller signal, your learning rates at the bottom of the nextwork will effectively be much smaller than the learning rates at the top, which will make it very difficult to pick a learning rate that is effective overall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is known as &lt;a href=&quot;https://adventuresinmachinelearning.com/vanishing-gradient-problem-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the vanishing gradient&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this, if you want to use a deep architecture, consider using an activation function that does not suffer from a vanishing gradient. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rectified Linear&lt;/a&gt; activation function, used in so-called &quot;ReLU&quot; units, is a non-linear activation that does not have a vanishing gradient. It is common to use ReLUs for the earlier layers in a network, and a sigmoid at the output layer, if you need outputs to be bounded between 0 and 1.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-19T02:13:14.170" LastActivityDate="2019-11-19T02:13:14.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16610" PostTypeId="1" CreationDate="2019-11-19T02:30:52.573" Score="10" ViewCount="805" Body="&lt;p&gt;On Sutton and Barto's RL book, the reward hypothesis is stated as &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Are there examples of tasks where the goals and purposes cannot be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I can think of are tasks with subjective rewards, like &quot;writing good music&quot;, but I am not convinced because maybe this is actually definable (perhaps by some super-intelligent alien) and we just aren't smart enough yet. Thus, I'm especially interested in counterexamples that &lt;strong&gt;logically&lt;/strong&gt; or &lt;strong&gt;provably&lt;/strong&gt; fail the hypothesis.&lt;/p&gt;&#xA;" OwnerUserId="31395" LastEditorUserId="2444" LastEditDate="2020-12-20T16:44:06.053" LastActivityDate="2021-01-13T04:06:31.447" Title="Counterexamples to the reward hypothesis" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;reward-hypothesis&gt;" AnswerCount="4" CommentCount="4" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16611" PostTypeId="1" CreationDate="2019-11-19T02:32:09.443" Score="1" ViewCount="34" Body="&lt;p&gt;I tried to implement the exact same python coding by Andrej Karpathy to train RL agent to play Pong, except that I migrated the environment from Gym to Retro. &#xA;Everything is the same except the action space in Retro is in indices and not in discrete as in Gym. The index has a size of 8, and index 4 and 5 are actions to move up and down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why the little modification has caused the agent not learning at all with running reward at -20 after over 3,000 episodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked the frame pre-processing before input to the policy forward neural network and it seems to be normal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, the output from the neural network is the probability of the paddle to move upwards. So I checked it. After few thousands episodes, the probability of the agent to move up just maintained at 0.5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the problem exists between the pre-processing and policy forward neural network, but I just cannot locate the problem. Appreciate if someone could help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole coding is as follow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import retro&#xA;import numpy as np&#xA;import _pickle as pickle&#xA;&#xA;H = 200 # number of hidden layer neurons&#xA;batch_size = 10 # every how many episodes to do a param update?&#xA;learning_rate = 1e-4&#xA;gamma = 0.98 # discount factor for reward&#xA;decay_rate = 0.98 # decay factor for RMSProp leaky sum of grad^2&#xA;resume = False # resume from previous checkpoint?&#xA;render = True&#xA;&#xA;&#xA;# model initialization&#xA;D = 80 * 80 # input dimensionality: 80x80 grid&#xA;if resume:&#xA;  model = pickle.load(open('save.p', 'rb'))&#xA;else:&#xA;  model = {}&#xA;  model['W1'] = np.random.randn(H,D) / np.sqrt(D) # &quot;Xavier&quot; initialization&#xA;  model['W2'] = np.random.randn(H) / np.sqrt(H)&#xA;&#xA;grad_buffer = { k : np.zeros_like(v) for k,v in model.items() } # update buffers that add up gradients over a batch&#xA;rmsprop_cache = { k : np.zeros_like(v) for k,v in model.items() } # rmsprop memory&#xA;&#xA;def sigmoid(x): &#xA;  return 1.0 / (1.0 + np.exp(-x)) # sigmoid &quot;squashing&quot; function to interval [0,1]&#xA;&#xA;def prepro(I):&#xA;  &quot;&quot;&quot; prepro 210x160x3 uint8 frame into 6400 (80x80) 1D float vector &quot;&quot;&quot;&#xA;  I = I[35:195] # crop&#xA;  I = I[::2,::2,0] # downsample by factor of 2&#xA;  I[I == 144] = 0 # erase background (background type 1)&#xA;  I[I == 109] = 0 # erase background (background type 2)&#xA;  I[I != 0] = 1 # everything else (paddles, ball) just set to 1&#xA;  return I.astype(np.float).ravel()&#xA;&#xA;def discount_rewards(r):&#xA;  &quot;&quot;&quot; take 1D float array of rewards and compute discounted reward &quot;&quot;&quot;&#xA;  discounted_r = np.zeros_like(r)&#xA;  running_add = 0&#xA;  for t in reversed(range(0, r.size)):&#xA;    if r[t] != 0: running_add = 0 # reset the sum, since this was a game boundary (pong specific!)&#xA;    running_add = running_add * gamma + r[t]&#xA;    discounted_r[t] = running_add&#xA;  return discounted_r&#xA;&#xA;def policy_forward(x):&#xA;  h = np.dot(model['W1'], x)&#xA;  h[h&amp;lt;0] = 0 # ReLU nonlinearity&#xA;  logp = np.dot(model['W2'], h)&#xA;  p = sigmoid(logp)&#xA;  return p, h # return probability of taking action 2, and hidden state&#xA;&#xA;def policy_backward(eph, epdlogp):&#xA;  &quot;&quot;&quot; backward pass. (eph is array of intermediate hidden states) &quot;&quot;&quot;&#xA;  dW2 = np.dot(eph.T, epdlogp).ravel()&#xA;  dh = np.outer(epdlogp, model['W2'])&#xA;  dh[eph &amp;lt;= 0] = 0 # backpro prelu&#xA;  dW1 = np.dot(dh.T, epx)&#xA;  return {'W1':dW1, 'W2':dW2}&#xA;&#xA;env=retro.make(game='Pong-Atari2600',players=1)&#xA;observation = env.reset()&#xA;prev_x = None # used in computing the difference frame&#xA;xs,hs,dlogps,drs = [],[],[],[]&#xA;running_reward = None&#xA;reward_sum = 0&#xA;episode_number = 0&#xA;&#xA;while True:&#xA;  if render: env.render()&#xA;&#xA;  action=[0,0,0,0,0,0,0,0] #reset the rl action&#xA;  # preprocess the observation, set input to network to be difference image&#xA;  cur_x = prepro(observation)&#xA;  x = cur_x - prev_x if prev_x is not None else np.zeros(D)&#xA;  prev_x = cur_x&#xA;&#xA;  # forward the policy network and sample an action from the returned probability&#xA;  aprob, h = policy_forward(x)&#xA;&#xA;  rlaction = 4  if np.random.uniform() &amp;lt; aprob else 5 # roll the dice!&#xA;  # record various intermediates (needed later for backprop)&#xA;  xs.append(x) # observation&#xA;  hs.append(h) # hidden state&#xA;  y = 1 if rlaction == 4 else 0 # a &quot;fake label&quot;&#xA;  dlogps.append(y - aprob) # grad that encourages the action that was taken to be taken (see http://cs231n.github.io/neural-networks-2/#losses if confused)&#xA;  action[rlaction]=1&#xA;  # step the environment and get new measurements&#xA;  observation, reward, done, info = env.step(action)&#xA;  reward_sum += reward&#xA;  drs.append(reward) # record reward (has to be done after we call step() to get reward for previous action)&#xA;&#xA;&#xA;  if done: # an episode finished&#xA;&#xA;    episode_number += 1&#xA;&#xA;    # stack together all inputs, hidden states, action gradients, and rewards for this episode&#xA;    epx = np.vstack(xs)&#xA;    eph = np.vstack(hs)&#xA;    epdlogp = np.vstack(dlogps)&#xA;    epr = np.vstack(drs)&#xA;    xs,hs,dlogps,drs = [],[],[],[] # reset array memory&#xA;&#xA;    # compute the discounted reward backwards through time&#xA;    discounted_epr = discount_rewards(epr)&#xA;    # standardize the rewards to be unit normal (helps control the gradient estimator variance)&#xA;    discounted_epr -= np.mean(discounted_epr)&#xA;    discounted_epr /= np.std(discounted_epr)&#xA;&#xA;    epdlogp *= discounted_epr # modulate the gradient with advantage (PG magic happens right here.)&#xA;    grad = policy_backward(eph, epdlogp)&#xA;    for k in model: grad_buffer[k] += grad[k] # accumulate grad over batch&#xA;&#xA;    # perform rmsprop parameter update every batch_size episodes&#xA;    if episode_number % batch_size == 0:&#xA;      for k,v in model.items():&#xA;        g = grad_buffer[k] # gradient&#xA;        rmsprop_cache[k] = decay_rate * rmsprop_cache[k] + (1 - decay_rate) * g**2&#xA;        model[k] += learning_rate * g / (np.sqrt(rmsprop_cache[k]) + 1e-5)&#xA;        grad_buffer[k] = np.zeros_like(v) # reset batch gradient buffer&#xA;&#xA;    # boring book-keeping&#xA;    running_reward = reward_sum if running_reward is None else running_reward * 0.99 + reward_sum * 0.01&#xA;    print(('%d , %d , %f ') % (episode_number-1,reward_sum,running_reward))&#xA;    if episode_number % 20 == 0: pickle.dump(model, open('save.p', 'wb'))&#xA;    reward_sum = 0&#xA;    observation = env.reset() # reset env&#xA;    prev_x = None&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30487" LastActivityDate="2019-11-19T02:32:09.443" Title="Same implementation, but agent is not learning in Retro Pong Environment" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16612" PostTypeId="1" CreationDate="2019-11-19T02:41:17.013" Score="2" ViewCount="195" Body="&lt;p&gt;I have stereo pairs (left, right) images of concrete cracks. I want to measure the length of the crack from those image pairs. Which neural network is appropriate for measuring object dimensions from stereo images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am insisted to use the NN-based technique only.&lt;/p&gt;&#xA;" OwnerUserId="31396" LastEditorUserId="31396" LastEditDate="2019-11-19T08:24:14.570" LastActivityDate="2019-11-19T15:36:22.150" Title="Which neural network is appropriate for measuring object dimensions from stereo images?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;object-detection&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16613" PostTypeId="2" ParentId="16608" CreationDate="2019-11-19T02:54:08.800" Score="0" Body="&lt;p&gt;Check out &lt;a href=&quot;https://arxiv.org/abs/1702.01992&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. It deals with the problem of mixing input modalities! Here's the abstract.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This paper presents a novel model for &lt;strong&gt;multimodal learning&lt;/strong&gt; based on gated neural networks. The Gated Multimodal Unit (GMU) model is intended to be used as an internal unit in a neural network architecture whose purpose is to find an intermediate representation based on a combination of data from different modalities. The GMU learns to decide how modalities influence the activation of the unit using multiplicative gates. It was evaluated on a multilabel scenario for &lt;strong&gt;genre classification of movies using the plot and the poster&lt;/strong&gt;. The GMU improved the macro f-score performance of single-modality approaches and outperformed other fusion strategies, including mixture of experts models. Along with this work, the MM-IMDb dataset is released which, to the best of our knowledge, is the largest publicly available multimodal dataset for genre prediction on movies.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="31395" LastEditorUserId="2444" LastEditDate="2020-09-11T15:10:06.043" LastActivityDate="2020-09-11T15:10:06.043" CommentCount="0" CommunityOwnedDate="2021-02-08T18:15:57.687" ContentLicense="CC BY-SA 4.0" />
  <row Id="16614" PostTypeId="2" ParentId="16612" CreationDate="2019-11-19T03:00:58.090" Score="2" Body="&lt;p&gt;If you have stero pairs, and you can identify the objects in the scene, you do not need a neural network, you can just use &lt;a href=&quot;https://en.wikipedia.org/wiki/Triangulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;triangulation&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to identify which objects in the scene are the same, you have an &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2019/04/introduction-image-segmentation-techniques-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;image segmentation&lt;/a&gt; problem. Depending on your problem and the amount of data you have access to, you may be able to use simple techniques like clustering-based segmentation, or you may be able to use NN-based techniques, like &lt;a href=&quot;https://arxiv.org/abs/1703.06870&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mask R-CNN&lt;/a&gt;.    &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-19T03:00:58.090" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16615" PostTypeId="2" ParentId="16605" CreationDate="2019-11-19T03:09:55.233" Score="1" Body="&lt;p&gt;I &lt;em&gt;think&lt;/em&gt; you may have a &lt;a href=&quot;http://www.chioka.in/class-imbalance-problem/&quot; rel=&quot;nofollow noreferrer&quot;&gt;class imbalance problem&lt;/a&gt; here, if I am reading your output correctly. You have 20,000 negative examples, but only 8000 positive ones, and you are minimizing binary cross entropy without re-weighting the examples, so your model can achieve a low-ish loss just by consistently outputing a value close to 0. This forms a local optima in the search space for the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this, you could try to optimize some other loss function that is more sensitive to class imbalances, or, likely more productively, you could just use an equal number of examples for each class. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-19T03:09:55.233" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16616" PostTypeId="1" AcceptedAnswerId="16618" CreationDate="2019-11-19T03:32:49.120" Score="3" ViewCount="99" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto's &quot;Reinforcement Learning: An Introduction&quot;, 2nd edition&lt;/a&gt;, page 199, they describe the on-policy distribution for episodic tasks in the following box:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lf1Q3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lf1Q3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how this can be done without taking the length of the episode into account. Suppose a task has 10 states, has probability 1 of starting at the first state, then moves to any state uniformly until the episode terminates. If the episode has 100 time steps, then probability of the first state is proportional to &lt;span class=&quot;math-container&quot;&gt;$1 + 100\times 1/10$&lt;/span&gt;; if it has &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; time steps, it will be proportional to &lt;span class=&quot;math-container&quot;&gt;$1 + 1000\times 1/10$&lt;/span&gt;. However, the formula given would make it proportional to &lt;span class=&quot;math-container&quot;&gt;$1 + 1/10$&lt;/span&gt; in both cases. What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="30679" LastEditorUserId="9947" LastEditDate="2019-11-19T06:28:50.180" LastActivityDate="2019-11-19T10:29:32.290" Title="On-policy state distribution for episodic tasks on Sutton &amp; Barto, page 199" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16617" PostTypeId="1" AcceptedAnswerId="16620" CreationDate="2019-11-19T05:43:25.500" Score="0" ViewCount="134" Body="&lt;p&gt;Is there has any method to train Tensorflow AI/ML that I focus on detecting background of image more than common objects?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm newbie to ML field, but was assigned to do job that make an application which can detecting on showroom image/places and detecting on the floor, wall then find out what is the material/ceramic/marble/etc. product they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: This is showroom picture,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5xdwL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5xdwL.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the wall and the floor of showroom are using this product material&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OeXyr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OeXyr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it possible to do something like I described?&lt;/li&gt;&#xA;&lt;li&gt;How to start with?&lt;/li&gt;&#xA;&lt;li&gt;If I don't want to install Tensorflow into my computer, is there a service that can make a model to use in the device? (my goal need to use the model in Android device)&lt;/li&gt;&#xA;&lt;li&gt;What method/type of ML should I approach 'Classification' or 'Object Detection' or other else?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31401" LastActivityDate="2019-11-19T08:21:19.550" Title="Is there has any method to train Tensorflow AI/ML that I focus on detecting background of image more than common objects?" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;ai-basics&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2020-05-16T23:33:51.407" ContentLicense="CC BY-SA 4.0" />
  <row Id="16618" PostTypeId="2" ParentId="16616" CreationDate="2019-11-19T06:21:29.397" Score="3" Body="&lt;p&gt;Let's first assume that there is only one action so that &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s) = 1$&lt;/span&gt; for every state - action pair which simplifies the discussion.&#xA;Now let's consider a case with 100 time steps, 10 states and uniform distribution for starting state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$h(s_0) = 1$&lt;/span&gt;. The result would be&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\eta(s_0) &amp;amp;= 1 + \sum_{i = 0}^9 \eta(s_i) \cdot p(s_0|s_i) =\\&#xA;&amp;amp;= 1 + \sum_{i = 0}^9 10 \cdot \frac{1}{10} = 11&#xA;\end{align}&lt;/span&gt;&#xA;Now let's consider a case with 1000 time steps where other settings are the same as in the first case.&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\eta(s_0) &amp;amp;= 1 + \sum_{i = 0}^{9} \eta(s_i) \cdot p(s_0|s_i) =\\&#xA;&amp;amp;= 1 + \sum_{i = 0}^{9} 100 \cdot \frac{1}{10} = 101&#xA;\end{align}&lt;/span&gt;&#xA;In the first case&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mu(s_0) = \frac{11}{9\cdot 10 + 11} = 0.1089&#xA;\end{equation}&lt;/span&gt;&#xA;and in the second case you have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mu(s_0) = \frac{101}{9\cdot 100 + 101} = 0.1009&#xA;\end{equation}&lt;/span&gt;&#xA;so it looks like you are correct that &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; depends on the length of the episode, but they didn't really say that it doesn't. Obviously as the length of the episode increases so will the number of times a certain state was visited so you could say that formula implicitly depends on the number of time steps. If &lt;span class=&quot;math-container&quot;&gt;$h(s_i)$&lt;/span&gt; is equal for every state then results would be the same in both cases regardless of number of time steps. Also, as the number of possible states grows very large, as it usually is in real problems, the results would be approaching each other as the number of states grows.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-11-19T06:21:29.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16620" PostTypeId="2" ParentId="16617" CreationDate="2019-11-19T08:21:19.550" Score="4" Body="&lt;p&gt;Trying to address all the questions asked in the end in the same order&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Most definitely possible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I would say its best you approach this with segmentation to start with.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Just use a free GPU runtime notebook service such as Google Colab or Kaggle Kernels. But you would not directly be able to integrate with the device, you'd have to keep moving input and output from your drive (on Colab). There might be a better service for the needs described, but this is the best I know on this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your background can be segmented and the segment can work on transforms such as maybe convolutions or affine transforms to be able to get the relevant information regarding the background.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this was helpful!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-19T08:21:19.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16621" PostTypeId="2" ParentId="16612" CreationDate="2019-11-19T09:10:37.010" Score="3" Body="&lt;p&gt;Is the image taken from a constant distance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, you'd need to scale the images to the same dimensions first of all. For few images say 100-500 images (more the better) you'd need to label the dataset by proper scaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once labeled, use it to train a &lt;a href=&quot;https://github.com/gsurma/image_classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;CNN&lt;/strong&gt;&lt;/a&gt; (Although best would be training a &lt;a href=&quot;https://github.com/ry/tensorflow-resnet&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;ResNet&lt;/strong&gt;&lt;/a&gt;). Once trained with decent accuracy, test it for the rest of your dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did something similar for one of my projects, check it out if you want to &lt;a href=&quot;https://github.com/Ar9av/Feature-Extraction-from-Facial-Images&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="31407" LastEditorUserId="2444" LastEditDate="2019-11-19T15:36:22.150" LastActivityDate="2019-11-19T15:36:22.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16622" PostTypeId="2" ParentId="16606" CreationDate="2019-11-19T09:53:15.197" Score="2" Body="&lt;p&gt;Imagine a game with a very clear first move, such as a game where choosing to go first if you win a coin toss brings a clear and obvious advantage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this situation standard MCTS does little exploration down the side of the tree that branches at the win toss &gt; let opponent start step, as the basic simulations of the rest of the game at this split quickly show the large gain you get when always starting when you win the coin toss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result, you would end up with a tree with very little expansion on the side of win the toss &gt; put your opponent in, as every simulation step you do from even the most senior nodes ends with much worse expected outcome values than the alternatives on the other side of the tree where you do the correct move of always playing first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These nodes on the side of letting your opponent start have &lt;em&gt;huge&lt;/em&gt; potential sub trees (as the whole game would still need to be played out if your opponent started), but would have very little searching down them. As a result, on this side of the tree, you would have many leaf nodes with large (but as yet unexplored, outside of the basic, early simulations down that side) sub trees that you &lt;em&gt;could&lt;/em&gt; search if you modified the exploration vs exploration algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a basic example, take the 0/3 node at the far right on level one of the wiki example below, which would get much less attention than the much more promising 7/10 and 3/8 nodes, despite having potentially many subsequent children it &lt;em&gt;could&lt;/em&gt; explore. If you took this node as your L node, you would expand it's children that you have not yet searched, and thus find out more about why this side of the tree is bad and update our now more granular probabilities accordingly, just as it does for the 3/3 node here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A5vw0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A5vw0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14997" LastEditorUserId="14997" LastEditDate="2019-11-19T10:06:32.493" LastActivityDate="2019-11-19T10:06:32.493" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="16623" PostTypeId="2" ParentId="16616" CreationDate="2019-11-19T10:29:32.290" Score="2" Body="&lt;p&gt;You are missing that the expression &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{s'} \eta(s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is already a count of the expected length of an episode, and is used in the denominator to scale &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\sum_{s} \mu(s) = 1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the length of the episode &lt;em&gt;is&lt;/em&gt; taken into account in the formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice you don't need to know &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt;, it can be left unresolved as a theoretical construct. What you care about for the theory to work is that the samples that you train with are drawn with same frequency - this happens automatically if you work with an on-policy algorithm. So the theory can hide the maths that you might need to do in order to determine actual values for &lt;span class=&quot;math-container&quot;&gt;$\eta(s)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-11-19T10:29:32.290" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16624" PostTypeId="2" ParentId="16447" CreationDate="2019-11-19T11:38:38.167" Score="1" Body="&lt;p&gt;There is a platform called Flow, sounds like its what you're looking for&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/flow-project/flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/flow-project/flow&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31411" LastActivityDate="2019-11-19T11:38:38.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16625" PostTypeId="1" CreationDate="2019-11-19T12:22:47.323" Score="4" ViewCount="37" Body="&lt;p&gt;I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain requirements before the advert can be approved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Customers are required to submit the following vehicle photos: front, back, left-side, right-side, engine (similar to the front photo but with the hood open) and instrument panel cluster.  The vehicle must be well framed in the photo, in other words, it must not be too small or so big that the edges touch the frame of the photograph. It also needs to be one of the mentioned types and the camera must be facing the vehicle directly with only small angle variations (a front photo can't include a large piece of the side of a car).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another developer had a go and built a CNN with Keras which does alleviate some manual grind (about 20,000 photos were used for training - no annotations). The accuracy sits at around 75% for the vehicle photos but only 55% for the engine and instrument cluster. Each photo is still manually checked, but it is a case of agreeing or disagreeing with what was recognised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if it wouldn't be better to detect a vehicle in the image using an existing pre-trained model like ImageAI. Use the bounding box of the vehicle to determine it is correctly placed in the frame of the photograph and within acceptable dimensions. There may be multiple vehicles in the picture so work with the most prominent one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At that point would it be worth trying to develop something to workout the pose of the vehicle (idea: &lt;a href=&quot;https://github.com/johnberroa/CORY&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/johnberroa/CORY&lt;/a&gt;) or just do some transfer learning with whatever pre-existing trained model was used and spend some time annotating the images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/amsgP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/amsgP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31412" LastEditorUserId="2444" LastEditDate="2019-11-19T15:34:30.747" LastActivityDate="2019-11-19T15:34:30.747" Title="How can I improve the performance of a model trained to detect vehicle poses?" Tags="&lt;image-recognition&gt;&lt;computer-vision&gt;&lt;image-segmentation&gt;&lt;transfer-learning&gt;&lt;pretrained-models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16626" PostTypeId="2" ParentId="16597" CreationDate="2019-11-19T13:00:52.530" Score="1" Body="&lt;p&gt;In general, you &lt;em&gt;can&lt;/em&gt; use a simulation to prepare and train a controller for a real world application. A good example of this being done for robotics is in the paper &lt;a href=&quot;https://people.eecs.berkeley.edu/~jordan/papers/ng-etal03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autonomous helicopter flight via reinforcement learning&lt;/a&gt; where a Reinforcement Learning agent was trained on a model of helicopter dynamics before being used in reality. Often, as in this case, such work is done to avoid expensive failures due to the trial and error nature of RL - if an error is expensive, such as crashing a helicopter, then ideally the agent performs the checks to avoid it in simulation, by planning or some other virtual environment as opposed to in the real world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main hurdle to completing training in simulation then transfering to the real world, is the fidelity of the simulation. The simulation's physics, including measurements of physical quantities, the size of time steps, amount of randomness/noise, should match between the simulation and the target real-world environment. If they do not match, then a learning agent could generate a policy that works in simulation, but that fails in reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the autonomous helicopter, the researchers used data from a human operator controlling the real helicopter, to help generate a predictive model that was used in the simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you do the same with Open AI Gym environments? Probably not, unfortunately. The main issue is that the units used are fixed in most environments, and are unlikely to closely relate to any specific real world implementation of the same kind of system. In addition, the physics is often simplified - probably a minor issue for CartPole, but a more major one for environments like LunarLander which ignores weight of fuel used and is a 2D simulation of a 3D environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for instance, in CartPole environments, the following values are fixed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Size of time step&lt;/li&gt;&#xA;&lt;li&gt;Mass of cart&lt;/li&gt;&#xA;&lt;li&gt;Mass and length of pole to be balanced&lt;/li&gt;&#xA;&lt;li&gt;Force that cart motor pushes with&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are a couple of approaches you could use to work around this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Make a new version of the environment and adjust it so that values match to a real environment you want to train for. Note this may still be limited, as the physics model is still quite simple, and may not allow for the real operating characteristics of the cart motor.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the CartPole environment as-is, not to train a controller directly, but to select hyper parameters, such as neural network size, learning rate etc. That will result in a learning agent that you are reasonably confident &lt;em&gt;can&lt;/em&gt; learn policies with the state representation and general behaviour of your target system. You then train &quot;for real&quot; again in the physical system.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can combine these ideas, creating a best-guess controller from simulation, then refining it in a real environment by continuing the training on a real system.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-11-19T13:58:32.770" LastActivityDate="2019-11-19T13:58:32.770" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16627" PostTypeId="1" CreationDate="2019-11-19T13:04:34.387" Score="3" ViewCount="17" Body="&lt;p&gt;I have the following problem while using convolutional neural networks to detect forgeries:&#xA;Resizing the image to fit the required input size may not be a good way because the forgery detection largely relies on the details of images, for example, the noise. Thus the resizing process may change/hurt the details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Existing methods mainly use image patches (obtained from cropping) that have the same size. This way, however, will drop the spatial information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for some suggestions on how to deal with this problem (input size inconsistency) without leaving out the spatial information.&lt;/p&gt;&#xA;" OwnerUserId="26955" LastActivityDate="2019-11-19T13:04:34.387" Title="Size of image input of neural networks while resizing may not be appropriate" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16628" PostTypeId="1" CreationDate="2019-11-19T13:14:16.030" Score="2" ViewCount="364" Body="&lt;p&gt;I was watching this video from &lt;a href=&quot;https://www.youtube.com/watch?v=l_6Tumd8EQI&quot; rel=&quot;nofollow noreferrer&quot;&gt;corridor crew&lt;/a&gt;, according to them, they have used deepfake technology to create this video. I myself have never made a deepfake videos, but I have enough knowledge in the underlying technology to know that it's hard to swap a face with multiple people existing simultaneously in a frame and let alone swapping faces of multiple people in a single frame. But corridor crews video showed that multiple deepfakes can be done, and that's why I am sceptical about the video of using deepfake technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the video is indeed made with deepfake technology, then what is the mechanism behind this? My own guess is that they might have masked other people and concentrated on one in a frame. Then they have used this masked frame to generate the deepfake, which then concatenated to the original frame. Do you think this is possible? Is there a research article or blog post which explains this process?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2019-11-19T14:48:58.220" LastActivityDate="2019-11-19T14:48:58.220" Title="How does deepfake technology work with multiple people in a single frame?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;generative-adversarial-networks&gt;&lt;autoencoders&gt;&lt;deepfakes&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16629" PostTypeId="1" CreationDate="2019-11-19T14:31:30.013" Score="2" ViewCount="67" Body="&lt;p&gt;Consider a problem with many objectives. In my case, these are school grades for different courses (or subjects). To be more concrete, suppose that my current grade for the math course is &lt;span class=&quot;math-container&quot;&gt;$12/20$&lt;/span&gt; and for the philosophy course is &lt;span class=&quot;math-container&quot;&gt;$8/20$&lt;/span&gt;. My objective is to get &lt;span class=&quot;math-container&quot;&gt;$16/20$&lt;/span&gt; for the math course and &lt;span class=&quot;math-container&quot;&gt;$15/20$&lt;/span&gt; for the philosophy course.&lt;/p&gt;&#xA;&lt;p&gt;I have the possibility to take different courses, but I need to decide which ones. These courses can have a different impact depending on the subject. Let's say that the impact factor is in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; means no impact and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; means a big impact. Then the math course could have a big impact (e.g. &lt;span class=&quot;math-container&quot;&gt;$0.9$&lt;/span&gt;) on the grade, while maybe a philosophy course may not have such a big impact.&lt;/p&gt;&#xA;&lt;p&gt;The overall goal is to increase all the grades as much as possible while taking into account the impact of their associated course. In my case, I can have more than two courses and subjects.&lt;/p&gt;&#xA;&lt;p&gt;So, which algorithms can I use to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="31415" LastEditorUserId="2444" LastEditDate="2020-06-21T16:29:29.957" LastActivityDate="2020-06-21T16:29:29.957" Title="Which algorithm can I use to solve a problem with multiple objectives and constraints?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;constraint-satisfaction-problems&gt;&lt;linear-programming&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16630" PostTypeId="2" ParentId="16607" CreationDate="2019-11-19T14:53:50.433" Score="2" Body="&lt;p&gt;You results show signs of overfitting at around epoch 40. In order to overcome this you can either simplify the model somewhat or increase regularization. You do not share what values you are using for dropout regularization so you can try increasing that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to be honest, I am not sure if that is going to help. You are using dropout in a pure CNN architecture, which I do not see that often in recent models. When they exists, they are usually at the very end of the network, or after densely connected layers. There are claims that regular dropout does not work as intended for convolutional layers, so the idea of &lt;a href=&quot;https://arxiv.org/pdf/1411.4280.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;spatial dropout&lt;/a&gt; is developed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you are using them after pooling layers and such a design &lt;a href=&quot;https://arxiv.org/pdf/1512.01400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;may not be&lt;/a&gt; as bad as using them after convolutional layers. In any case, if you check the &lt;a href=&quot;http://en.wikipedia.org/wiki/CIFAR-10&quot; rel=&quot;nofollow noreferrer&quot;&gt;top performers&lt;/a&gt; on this dataset, you can see there are some who are trying to come up with new regularization techniques for CNN's, such as &lt;a href=&quot;https://arxiv.org/abs/1705.07485&quot; rel=&quot;nofollow noreferrer&quot;&gt;shake-shake&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1802.02375&quot; rel=&quot;nofollow noreferrer&quot;&gt;shakedrop&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1708.04552&quot; rel=&quot;nofollow noreferrer&quot;&gt;cutout&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, given you already have an overfitting model, removing regularization might not be the best idea. But if you replace the layers after (and including) the last convolution with some fully connected layers, you can use dropout regularization on them instead. This would remove the largest convolutional layer of the model and you can add dense layers without increasing model size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For image recognition tasks of any kind, easiest way to achieve high accuracy is via &lt;strong&gt;transfer learning&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-11-21T07:42:11.177" LastActivityDate="2019-11-21T07:42:11.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16631" PostTypeId="1" AcceptedAnswerId="20642" CreationDate="2019-11-19T14:57:33.023" Score="4" ViewCount="233" Body="&lt;p&gt;I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The &lt;a href=&quot;https://arxiv.org/pdf/1706.02262.pdf)%F0%9F%8E%99%EF%B8%8FPoster&quot; rel=&quot;nofollow noreferrer&quot;&gt;InfoVAE&lt;/a&gt; paper describes the case when the decoder is flexible enough to ignore the latent attributes and generate an averaged out image that best reduces the reconstruction loss. Thus the blurred image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much of the problem of blurring is really mitigated by the &lt;a href=&quot;https://arxiv.org/pdf/1706.02262.pdf)%F0%9F%8E%99%EF%B8%8FPoster&quot; rel=&quot;nofollow noreferrer&quot;&gt;MMD formulation&lt;/a&gt; in practical experiments? If someone has experience working with &lt;a href=&quot;https://arxiv.org/pdf/1706.02262.pdf)%F0%9F%8E%99%EF%B8%8FPoster&quot; rel=&quot;nofollow noreferrer&quot;&gt;MMD-VAEs&lt;/a&gt;, I'd like to know their opinion on what the reconstruction quality of MMD-VAEs is really like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, does the replacement of the MSE reconstruction loss metric by other perceptual similarity metrics improve generated image quality?&lt;/p&gt;&#xA;" OwnerUserId="31416" LastEditorUserId="31416" LastEditDate="2019-11-19T18:21:58.630" LastActivityDate="2020-04-26T11:17:53.820" Title="Does MMD-VAE solve the problem of blurred images of vanilla VAEs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;generative-model&gt;&lt;image-generation&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16632" PostTypeId="5" CreationDate="2019-11-19T15:59:24.157" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-19T15:59:24.157" LastActivityDate="2019-11-19T15:59:24.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16633" PostTypeId="4" CreationDate="2019-11-19T15:59:24.157" Score="0" Body="For questions related to the ID3 (Iterative Dichotomiser 3) algorithm, which is a decision tree algorithm, so it can be used to learn a decision tree from a dataset. The ID3 algorithm was also developed in 1986 by Ross Quinlan." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T03:09:13.387" LastActivityDate="2019-11-21T03:09:13.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16634" PostTypeId="1" AcceptedAnswerId="16635" CreationDate="2019-11-19T18:33:13.487" Score="2" ViewCount="35" Body="&lt;p&gt;&lt;a href=&quot;http://places.csail.mit.edu/downloadCNN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Places205-VGG&lt;/a&gt;, a CNN trained model for 205 scene categories of Places Database with  2.5 million images &lt;code&gt;Places205&lt;/code&gt; dataset has &lt;code&gt;top1 accuracy = 58.9%&lt;/code&gt; and &lt;code&gt;top5 accuracy = 87.7%&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &lt;code&gt;top1&lt;/code&gt; and &lt;code&gt;top5&lt;/code&gt; (and, in general, top &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;) accuracy mean in the context of deep learning?&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="2444" LastEditDate="2019-11-19T19:45:43.507" LastActivityDate="2019-11-19T19:45:43.507" Title="What does top N accuracy mean?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;scene-classification&gt;&lt;accuracy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16635" PostTypeId="2" ParentId="16634" CreationDate="2019-11-19T19:30:52.990" Score="2" Body="&lt;p&gt;It is explained in this &lt;a href=&quot;https://stats.stackexchange.com/questions/95391/what-is-the-definition-of-top-n-accuracy&quot;&gt;CrossValidated post&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Top1 accuracy means the best guess (class with highest probability) is the correct result 58.9% of the time, while top5 accuracy means the correct result is in the top 5 best guesses (5 classes with highest probabilities) 87.7% of the time.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastActivityDate="2019-11-19T19:30:52.990" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16636" PostTypeId="2" ParentId="16629" CreationDate="2019-11-19T19:35:41.740" Score="2" Body="&lt;p&gt;From your description, the problem you want to solve is a &lt;strong&gt;linear optimization problem&lt;/strong&gt;: suppose we use the indices &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; to denote the &lt;span class=&quot;math-container&quot;&gt;$i-$&lt;/span&gt;th class and the &lt;span class=&quot;math-container&quot;&gt;$j-$&lt;/span&gt;th grade. Also, let us call &lt;span class=&quot;math-container&quot;&gt;$y_j$&lt;/span&gt; the current value of the &lt;span class=&quot;math-container&quot;&gt;$j-th$&lt;/span&gt; grade, &lt;span class=&quot;math-container&quot;&gt;$g_j$&lt;/span&gt; the goal value in grade &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c_{ij}$&lt;/span&gt; the impact factor of taking class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in grade &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; the binary variable that indicates if you go to class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; or not. Now, suppose you decided to take certain classes (which is equivalent to choosing the values &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; for each &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;) and measure the new grades &lt;span class=&quot;math-container&quot;&gt;$\tilde g_j$&lt;/span&gt;. If your impact factors are accurate, then the new grades should be &lt;span class=&quot;math-container&quot;&gt;$\tilde g_j = \sum_i(x_ic_{ij}+y_j)$&lt;/span&gt;. Clearly, what you would like is for this new values to be as close as possible to the goal values. So, a possible way to express your optimization problem is that you want to minimize the addition of all these differences:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_{x_0,\cdots,x_n} \sum_j g_j - \tilde g_j = \sum_j g_j - \sum_i(x_ic_{ij}+y_j), \\ \text{s.t. } x_i\in\{0,1\}, \forall\hspace 2pt i.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most probably, you will have certain time restrictions. For example, maybe each class takes &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt; hours from your time and your total available time is just &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; hours. Or maybe you have a limit of classes &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; you can take, irrespective of their time duration. With these two constraints, your problem would look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_{x_0,\cdots,x_n} \sum_j g_j - \sum_i(x_ic_{ij}+y_j), \\ \text{s.t. } x_i\in\{0,1\}, \forall\hspace 2pt i,\\ \sum_ix_it_i\leq T,\\ \sum_i x_i\leq N.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from the constraint that &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; should be 0 or 1, this is a linear optimization problem with linear constraints. &lt;a href=&quot;http://web.mit.edu/15.053/www/AMP-Chapter-09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Integer programming&lt;/strong&gt;&lt;/a&gt; is an area of optimization that studies this type of problems so probably it's a good direction for your research.&lt;/p&gt;&#xA;" OwnerUserId="30983" LastEditorUserId="30983" LastEditDate="2019-11-19T19:46:16.233" LastActivityDate="2019-11-19T19:46:16.233" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16637" PostTypeId="2" ParentId="16251" CreationDate="2019-11-19T19:59:47.780" Score="1" Body="&lt;p&gt;You need to have access to the 696th hour (or successive hours), otherwise, you cannot test your model. An alternative would be, for example, to train your model on the first 693 hours, validate it on the 694th hour, and test it on the 695th hour.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-11-19T19:59:47.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16638" PostTypeId="1" CreationDate="2019-11-19T20:10:07.580" Score="2" ViewCount="24" Body="&lt;p&gt;I've been working on a question that is posed in a document I've been reading, that models qualifying for a job as a POMDP. In this model, a person takes 3 exams, and must pass all of them in order to get the job. The person make either be qualified or not qualified in the subjects covered by each individual exam, and there is some probability that a person may be qualified for a subject covered in a particular exam, but still not pass (due to nerves). False passes are also possible as well.&#xA;A candidate is allowed a maximum of 2 exam attempts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In understanding this problem, I've tried to list out all possible states the person might be in (qualified / not qualified for each exam), and have found the following possible states: (where Q is qualified, and N is not qualified)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;QQQ&#xA;QQN&#xA;QNN&#xA;QNQ&#xA;NNN&#xA;NQN&#xA;NNQ&#xA;NQQ&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the total number of possible states is 8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have I covered all possible states? I'm wondering if there's an easier way to find out total number of states, without having to list them all out in the above way. I'm very new to this field, so any help is appreciated. &lt;/p&gt;&#xA;" OwnerUserId="31421" LastActivityDate="2019-11-19T20:10:07.580" Title="Finding total number of states in a POMDP" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16639" PostTypeId="2" ParentId="16598" CreationDate="2019-11-19T21:56:40.013" Score="3" Body="&lt;p&gt;Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look carefully at the three sources you post, you will find that they actually all agree on this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two of the sources actually propose methods of &lt;em&gt;addressing&lt;/em&gt; this shortcoming, by making adjustments to the decision tree learning algorithms. The proposed adjustments are essentially standard solutions to these problems, being applied to decision trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example technique, discussed in the first paper you reference, is changing the weightings of the classes. An inefficient/approximate way to do this is to increase the number of examples from the minority class. For example, if you had an 80/20 split, you could add 3 new copies of each minority class example to move to an 80/80 = 50/50 split. Of course, if you add new data points, your algorithm may take longer to run. Instead, you can just modify the weightings of the classes in your optimization function. This approach is algorithm-specific, and will depend on your loss function, but achieves the same effect, just without needing to increase the number of points you use.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-20T13:59:43.477" LastActivityDate="2019-11-20T13:59:43.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16640" PostTypeId="2" ParentId="16576" CreationDate="2019-11-19T22:03:34.600" Score="0" Body="&lt;p&gt;You can look at the paper &lt;a href=&quot;http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gradient-Based Learning Applied to Document&#xA;Recognition&lt;/a&gt; (1998) by Yann LeCun et al., which reviews and compares various methods applied to handwritten character recognition and shows that CNNs outperform all other methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I suggest Andrew Ng's CNN videos.&lt;/p&gt;&#xA;" OwnerUserId="27315" LastEditorUserId="2444" LastEditDate="2019-11-19T22:10:38.943" LastActivityDate="2019-11-19T22:10:38.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16641" PostTypeId="2" ParentId="9153" CreationDate="2019-11-19T22:06:55.890" Score="1" Body="&lt;p&gt;Dempster-Shafer Theory was studied fairly seriously in AI through the 70's and 80's. However, Dempster-Shafer theory has some serious shortcomings, that will cause an agent that uses it to make irrational decisions. These were uncovered by Pearl and others in the 80's, with more problems emerging in later years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.ijcai.org/Proceedings/85-2/Papers/064.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cheeseman&lt;/a&gt; for a summary of the arguments that probability is the only suitable framework for reasoning under-uncertainty in AI (which is the current orthodox view), along with &lt;a href=&quot;https://ieeexplore.ieee.org/document/6289948&quot; rel=&quot;nofollow noreferrer&quot;&gt;Josang &amp;amp; Harkin 2012&lt;/a&gt; (which develops a theory that subsumes Dempster-Shafer, and under which DS can be shown to lead to poor decisions), and Pearls' &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0888613X8890117X?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Probability Intervals&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-19T22:06:55.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16642" PostTypeId="1" CreationDate="2019-11-19T22:15:05.103" Score="2" ViewCount="14" Body="&lt;p&gt;I am trying to increse the quality of the images that I gather from the microscope.  That is a acoustic microscope and there are lots of technical details but in a nutshell the low quality images and its corresponding high quality images that I gather from the same sample are not perfectly alligned because in my setting it is impossible to increase quality without removing the sample from the microscope so when I put it again it is a manual process so they are not perfectly aligned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output of my network will be,let's say, 256 x 256 image and its corresponding label will be high quality 256 x 256 image,in theory, of the exactly same area. If I make pixel to pixel comparison between them, for example taking MSE for the loss function, will it be able to learn ? I am not sure because pixels are not perfectly alligned, they do not represent the same area of the image(the difference is not that great but they are not perfectly alligned as I said)&lt;/p&gt;&#xA;" OwnerUserId="31427" LastActivityDate="2019-11-19T22:15:05.103" Title="Loss function for increasing the quality of the image when labels are not perfectly alligned" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16645" PostTypeId="1" CreationDate="2019-11-20T01:05:15.547" Score="1" ViewCount="32" Body="&lt;p&gt;I am trying to to do a sort of block-validator for bitcoin(and alike chains), in it I need to depending on chain and block-height only allow certain operators in the transactions scripts. One thought I had was that this might be something that LISA should be good for. (I might be wrong in this) but shouldn't something like a rule engine be a good fit for that? What I want is a good way to defines the rules for my validator on how to validate that a block and its transactions adhere to the consensus rules?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sort of getting to this point&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(defpackage #:chain-validator&#xA;  (:shadowing-import-from #:lisa #:assert)&#xA;  (:use #:lisa&#xA;    #:cl))&#xA;&#xA;(in-package :chain-validator)&#xA;&#xA;(defclass chain-fundamental () ())&#xA;&#xA;(defclass chain-block (chain-fundamental)&#xA;  ((height :initarg :height :initform 1)&#xA;   (chain :initarg :chain :initform :bitcoin)))&#xA;&#xA;(defclass chain-tx (chain-fundamental)&#xA;  ((in-block :initarg :block :initform 'nil)&#xA;   (pk-script :initarg :pk-script)&#xA;   (is-coinbase-tx :initarg :coinbase :initform 'f)))&#xA;&#xA;(defclass chain-OP (chain-fundamental)&#xA;  ((name :initarg :name)&#xA;   (op-code :initarg :op-code)))&#xA;&#xA;(defrule dont-allow-op-mul-after-height-1000&#xA;    ;;; how to write this one?????&#xA;    ;; but if I only want to allow mul on a certain chain after height&#xA;    ;; 2000?&#xA;    )&#xA;&#xA;(defrule startup ()&#xA;  =&amp;gt;&#xA;  (assert-instance&#xA;   (make-instance 'chain-OP :name :PUSHDATA4 :op-code #x4e))&#xA;  (assert-instance&#xA;   (make-instance 'chain-OP :name :EQUAL :op-code #x87))&#xA;  (assert-instance&#xA;   (make-instance 'chain-OP :name :MUL :op-code #x95))&#xA;  (let* ((genesis-blk (make-instance 'chain-block))&#xA;     (later-blk (make-instance 'chain-block :height 2500))&#xA;     (first-coinbase-tx (make-instance 'chain-tx :block genesis-blk))&#xA;     (later-coinbase-tx (make-instance 'chain-tx :block later-blk)))&#xA;    (assert-instance genesis-blk)&#xA;    (assert-instance later-blk)&#xA;    (assert-instance first-coinbase-tx)&#xA;    (assert-instance later-coinbase-tx)))&#xA;&#xA;&#xA;;; how can I use LISA to get the chain-OPs that are allowed for a&#xA;;; transaction belonging to a specific block at some height, I sort of&#xA;;; want to find them all so i later can verify that the pk-script part&#xA;;; only contains those OPs. Could I write rules that would acctually&#xA;;; do the validation for me? that would check if chain-tx.pk-script&#xA;;; only contains certain OPs. And if we have multiple chains, how do I&#xA;;; write the rules to take account for that?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But after that I don't know how to proceed, the questions I want LISA to answer for me are things like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the valid script operations for a certain block, or transaction?&lt;/li&gt;&#xA;&lt;li&gt;Is this block or transaction valid?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Maybe what I need is primer on rule engines or a good tutorial. I just can't really get my head around how to write the rules.&lt;/p&gt;&#xA;" OwnerUserId="31430" LastActivityDate="2019-11-20T01:05:15.547" Title="Trying to get started with LISA and Lisp" Tags="&lt;expert-system&gt;&lt;lisp&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16646" PostTypeId="1" CreationDate="2019-11-20T01:11:47.623" Score="8" ViewCount="1143" Body="&lt;p&gt;Essentially, AI is created by human minds, so is the intelligence &amp;amp; creativity of algorithms properly an extension of human intelligence &amp;amp; creativity, rather than something independent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that intelligence does not necessarily require creativity, however, creativity can result from machine learning.  (A simple example is AlphaGo discovering novel strategies.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2019-11-20T12:23:19.777" LastActivityDate="2021-01-19T16:19:31.457" Title="Is artificial intelligence really just human intelligence?" Tags="&lt;philosophy&gt;&lt;intelligence&gt;&lt;creativity&gt;" AnswerCount="8" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16647" PostTypeId="2" ParentId="16606" CreationDate="2019-11-20T01:13:34.433" Score="0" Body="&lt;p&gt;I also want to answer my question after watching the video that @Philip post youtube.com/watch?v=UXW2yZndl7U . But still thank @Philip 's answer which is pretty helpful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The definition of &quot;leaf&quot; node.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The key point is &lt;code&gt;what tree is the host/owner of a &quot;leaf&quot; node&lt;/code&gt; to this question. Through &quot;Expansion&quot; step, we are actually creating a tree with MCTS. The tree, the owner of a &quot;leaf&quot; node, should be the one that we are building, not the tree of the game state in our head (or perhaps it is too big to fill in our head, the tree of the game state actually does not exist). Then we can understand that a &quot;leaf&quot; node is the one, which does not have any child, in the tree that we are building.&#xA;Once we get the answer of this question, the other questions can be answered automatically.&lt;/p&gt;&#xA;" OwnerUserId="31389" LastActivityDate="2019-11-20T01:13:34.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16648" PostTypeId="2" ParentId="16646" CreationDate="2019-11-20T02:05:34.943" Score="1" Body="&lt;p&gt;I think no, it isn't. The reason I would say no, is that in order for it to be an extension of our intelligence &amp;amp; creativity, it must be limited by it. This, I believe, isn't the case however. We are capable of creating an AI that is smarter than ourselves (say at Go or Chess, without cheating and checking every possible move), and so it is not bound by our own intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would liken it to creating a child. Just because you gave birth to Einstein, doesn't mean he's an extension of your intelligence. (This is of course pretty rudimentary, as it's &lt;em&gt;very&lt;/em&gt; debatable as to whether it's reasonable to liken humans to AI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this is a philosophical question, so it's hard to really answer yes or no.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-11-20T02:05:34.943" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16649" PostTypeId="2" ParentId="7446" CreationDate="2019-11-20T02:49:19.277" Score="1" Body="&lt;p&gt;The book &lt;a href=&quot;https://papers.harvie.cz/unsorted/computational-intelligence-an-introduction.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Intelligence: An Introduction&lt;/a&gt; (2nd edition, 2007) by &lt;a href=&quot;https://ieeexplore.ieee.org/author/37276400500&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andries P. Engelbrecht&lt;/a&gt;, which has been cited more than 3000 times, defines &lt;strong&gt;artificial intelligence&lt;/strong&gt; as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;These intelligent algorithms include artificial neural networks, evolutionary computation, swarm intelligence, artificial immune systems, and fuzzy systems. Together with logic, deductive reasoning, expert systems, case-based reasoning and symbolic machine learning systems, these intelligent algorithms form part of the field of &lt;strong&gt;Artificial Intelligence (AI)&lt;/strong&gt;. Just looking at this wide variety of AI techniques, AI can be seen as a combination of several research disciplines, for example, computer science, physiology, philosophy, sociology and biology.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and &lt;strong&gt;computational intelligence&lt;/strong&gt; as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This book concentrates on a sub-branch of AI, namely &lt;strong&gt;Computational Intelligence (CI)&lt;/strong&gt; – the study of adaptive mechanisms to enable or facilitate intelligent behavior in complex and changing environments. These mechanisms include those AI paradigms that exhibit an ability to learn or adapt to new situations, to generalize, abstract, discover and associate. The following CI paradigms are covered: artificial neural networks, evolutionary computation, swarm intelligence, artificial immune systems, and fuzzy systems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;He then notes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;At this point it is necessary to state that there are different definitions of what constitutes CI. This book reflects the opinion of the author, and may well cause some debate. For example, swarm intelligence (SI) and artificial immune systems (AIS) are classified as CI paradigms, while many researchers consider these paradigms to belong only under Artificial Life. However, both particle swarm optimization (PSO)&#xA;and ant colony optimization (ACO), as treated under SI, satisfy the definition of CI given above, and are therefore included in this book as being CI techniques. The same applies to AISs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, there may be different definitions of CI (given by different people), but, given that this book has been cited so many times, I would just stick to these definitions and use this book as a reference (I have actually consulted it a few times in the past). My university library even contains a copy of it.&lt;/p&gt;&#xA;&lt;p&gt;To summarise, CI is a sub-field of AI, which studies (or is associated with) the following topics&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;artificial neural networks (NN),&lt;/li&gt;&#xA;&lt;li&gt;evolutionary computation (EC),&lt;/li&gt;&#xA;&lt;li&gt;swarm intelligence (SI),&lt;/li&gt;&#xA;&lt;li&gt;artificial immune systems (AIS), and&lt;/li&gt;&#xA;&lt;li&gt;fuzzy systems (FS).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;which are also part of AI, which additionally studies&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;logic,&lt;/li&gt;&#xA;&lt;li&gt;deductive reasoning,&lt;/li&gt;&#xA;&lt;li&gt;expert systems,&lt;/li&gt;&#xA;&lt;li&gt;case-based reasoning, and&lt;/li&gt;&#xA;&lt;li&gt;symbolic machine learning systems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Just to give further credibility to these definitions, Andries P. Engelbrecht &lt;a href=&quot;https://scholar.google.com/citations?user=h9pOfj0AAAAJ&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;has an h-index of 59, has been cited 22557 times&lt;/a&gt;, and is an &lt;a href=&quot;https://ieeexplore.ieee.org/author/37276400500&quot; rel=&quot;nofollow noreferrer&quot;&gt;IEEE Senior Member&lt;/a&gt;. You can find more info about him &lt;a href=&quot;https://engel.pages.cs.sun.ac.za/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Note that I have no affiliation with him. I am just providing this information so that people start to follow these definitions (rather than just looking at definitions given by people who have not extensively studied the field). Moreover, note that the definition of CI given by Engelbrecht is consistent with the definition given by IEEE that you are quoting.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-27T23:36:22.963" LastActivityDate="2020-11-27T23:36:22.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16650" PostTypeId="1" CreationDate="2019-11-20T03:14:10.083" Score="0" ViewCount="116" Body="&lt;p&gt;When are interpreted languages more optimal?  When are compiled languages more optimal?  What are the qualities and functions that render the so in relation to various AI methods?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-11-20T10:41:02.063" Title="When are compiled vs. interpreted languages more optimal in AI?" Tags="&lt;theory&gt;&lt;software-evaluation&gt;&lt;programming-languages&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16651" PostTypeId="2" ParentId="16646" CreationDate="2019-11-20T03:56:21.317" Score="1" Body="&lt;p&gt;No it isn't.&lt;/p&gt;&#xA;&lt;p&gt;AI is essentially human intelligence with a combination of computing power to achieve tasks that a human alone cannot achieve in the time period that a programmed machine can.&lt;/p&gt;&#xA;&lt;p&gt;To give an example. A human can identify a pattern in a data set of say 1000 records. However if that same logic needs applied to a data set of a billion records, a human would take ages to do it. But a machine can do that in seconds if the human gives the right instructions to the machine on how to do it.&lt;/p&gt;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="31435" LastEditorUserId="32410" LastEditDate="2021-01-19T16:19:31.457" LastActivityDate="2021-01-19T16:19:31.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16652" PostTypeId="1" AcceptedAnswerId="16654" CreationDate="2019-11-20T04:49:45.900" Score="1" ViewCount="49" Body="&lt;p&gt;I trained some weights to identify apples and oranges (using YOLOv3). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to be able to identify peaches, which approach is usually recommended:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start clean and train the 3 classes. &lt;/li&gt;&#xA;&lt;li&gt;Train the peaches over the already-trained weights (with apples and oranges)&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Only train with peaches images&lt;/li&gt;&#xA;&lt;li&gt;Use all available training data (including apples and oranges)&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This is what I have found:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I start clean, it will take longer until I can get a good result, but the detection is usually better.&lt;/li&gt;&#xA;&lt;li&gt;Every time I add a new class (using &lt;code&gt;2.2&lt;/code&gt;), the detection get worse for the already learned objects, but it takes less time until I can get a good result (however I suspect that apples and oranges become over-fitted?). &lt;/li&gt;&#xA;&lt;li&gt;I haven't tested &lt;code&gt;2.1&lt;/code&gt;, as I think that it won't be able to re-adjust the weights for the apples and the oranges.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is the above expected? What is the recommended course of action?&lt;/p&gt;&#xA;" OwnerUserId="9856" LastActivityDate="2019-11-20T06:30:07.483" Title="Classification with deeplearning : clean start vs continue training" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16653" PostTypeId="2" ParentId="16646" CreationDate="2019-11-20T04:58:19.993" Score="1" Body="&lt;p&gt;I believe AI is, at least in certain ways, &lt;em&gt;both&lt;/em&gt; an extension of human intelligence &amp;amp; creativity, and something independent as well. Note people didn't design airplanes to try to fly like birds do. Although planes use the same principles of aerodynamics that birds use to fly, we've adapted how those physics principles are applied to accommodate what we have to work with, i.e., metal, by having things like propellers, jet engines, fixed wings (initially, although later we also had helicopter rotor blades), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar fashion, we have adapted a few things we've learned about how human minds &amp;amp; intelligence work, with artificial neural networks being a prime example. However, even with just our fairly limited understanding, we've implemented neural networks differently, e.g., by which &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;activation functions&lt;/a&gt; are used.  Although we are learning more about how our brains work through neuroscience research, there's still so much we don't yet know. Nonetheless, I believe one of the biggest differences overall between our minds &amp;amp; AI is that our general intelligence comes from mostly massive parallel processing, to a much greater extent than even higher end GPUs can deliver, or even at least most supercomputers, while artificial intelligence generally depends instead a lot more on the massive speed of calculations available on our modern computer chips.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's this learning, adapting &amp;amp; extending what we know about how we think &amp;amp; create, in combination with the mostly independent boost of using the advantages of computer chips (mostly their ability to do very fast computations), that has allowed AI to advance as far as it has so far. Nobody, including myself, can be sure of where &amp;amp; how the next major advances in AI will occur, but I believe it'll likely be a combination of learning &amp;amp; using what we learn about how we mentally operate, along with advances in computer related knowledge &amp;amp; technology (e.g., new algorithm techniques, more &amp;amp; better parallel processing, quantum computers with many simultaneous &lt;a href=&quot;https://en.wikipedia.org/wiki/Qubit&quot; rel=&quot;nofollow noreferrer&quot;&gt;qubits&lt;/a&gt; operating, etc.).&lt;/p&gt;&#xA;" OwnerUserId="26797" LastEditorUserId="26797" LastEditDate="2019-11-20T07:27:45.127" LastActivityDate="2019-11-20T07:27:45.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16654" PostTypeId="2" ParentId="16652" CreationDate="2019-11-20T06:30:07.483" Score="2" Body="&lt;p&gt;If the task involves only apples, orange and peaches, you should use method 1. As the number of classes is small, the network cannot generalize well to all classes. As a side note, you should start with the pretrained weights of YOLO v3 as some classes of YOLO v3 may be fruits, which can help your model converge faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of classes is large, for example a hundred different fruits, you should use method 2.2 . The model should be able to generalize to all fruits and can converage faster as many fruits look the same. This is the case of transfer learning. In original YOLO v3 training, image net weights for dark net is used for the backbone network. It accelerates the training of YOLO v3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 2.1, it will not work as gradient descent will not consider the trained weight. The trained weights will be over written by the weights for peaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a recommended method, it depends on your class size. If the model will continue to add more classes, you should perhaps use 2.2 but only start using 2.2 when you have a considerable amount of classes. Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-20T06:30:07.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16655" PostTypeId="1" CreationDate="2019-11-20T06:46:12.207" Score="1" ViewCount="103" Body="&lt;p&gt;I have segmented concrete cracks from concrete structure images using Mask R-CNN. Now I need to measure the length of the segmented masked crack. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will the pixel counting method work? Can anyone help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The images are taken at the constant distance from the object.&lt;/p&gt;&#xA;" OwnerUserId="31396" LastEditorUserId="31396" LastEditDate="2019-11-20T08:50:10.463" LastActivityDate="2019-11-20T08:50:10.463" Title="How to count pixels in a object mask which is segmented using Mask R-CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16656" PostTypeId="1" AcceptedAnswerId="16658" CreationDate="2019-11-20T07:50:02.870" Score="2" ViewCount="53" Body="&lt;p&gt;I'm pursuing a master's degree in Artificial Intelligence. My final work is about Convolutional Neural Networks.&lt;/p&gt;&#xA;&lt;p&gt;I was looking for information about filters (or kernels) of the convolutional layers. I have found this article: &lt;a href=&quot;https://lodev.org/cgtutor/filtering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lode's Computer Graphics Tutorial - Image Filtering&lt;/a&gt;, but I need more.&lt;/p&gt;&#xA;&lt;p&gt;Do you know more resources about more filters (that it is known that they work) and how to create new ones?&lt;/p&gt;&#xA;&lt;p&gt;In other words, I want to know how they work and how can I create new ones.&lt;/p&gt;&#xA;&lt;p&gt;I've thought to create a C++ program, or with Octave, to test the new kernels.&lt;/p&gt;&#xA;&lt;p&gt;By the way, my research will be focused on image segmentation to process MRIs.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2021-01-19T15:00:12.310" LastActivityDate="2021-01-19T15:00:12.310" Title="What are some references that describe known filters (or kernels) and how we can create new ones?" Tags="&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16658" PostTypeId="2" ParentId="16656" CreationDate="2019-11-20T09:24:32.400" Score="3" Body="&lt;p&gt;I'd suggest you better understand edge detectors such as Robert or Sobel operators first to understand better how convolution operation on images extract features by constant value kernels.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would personally recommend &lt;a href=&quot;https://www.pearson.com/us/higher-education/program/Gonzalez-Digital-Image-Processing-4th-Edition/PGM241219.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gonzales and Woods&lt;/a&gt; for this,  as it gives a pure mathematical explanation to how and why these features are extracted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially the convolution kernels used in CNN's are ones with a learned set of values for the kernel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a better understanding of learned convolution kernels and, quite frankly, any idea under deep learning would easily recommend &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning by Goodfellow et al&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-20T09:24:32.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16660" PostTypeId="1" AcceptedAnswerId="16661" CreationDate="2019-11-20T10:39:30.783" Score="1" ViewCount="55" Body="&lt;p&gt;Are artificial neural networks based on human neural networks?&lt;/p&gt;&#xA;&lt;p&gt;Additionally, are the current computers capable of handling the power our brain has with the same speed?&lt;/p&gt;&#xA;" OwnerUserId="31211" LastEditorUserId="2444" LastEditDate="2021-01-20T00:22:34.080" LastActivityDate="2021-01-20T00:29:03.947" Title="Are artificial neural networks based on human neural networks?" Tags="&lt;neural-networks&gt;&lt;human-brain&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-20T00:29:11.577" ContentLicense="CC BY-SA 4.0" />
  <row Id="16661" PostTypeId="2" ParentId="16660" CreationDate="2019-11-20T10:39:30.783" Score="1" Body="&lt;p&gt;Artificial neural networks (ANNs) are loosely based on human neural networks. However, there are many aspects of the human brain that are not modelled in ANNs.&lt;/p&gt;&#xA;&lt;p&gt;First of all, ANNs have an input, which is followed by an output. Our brain doesn't work like this. We have many inputs, many outputs, and many cycles that loop in our brain. There's no simple input-output - end of the program. We think even if there is no 'input' (e.g. sleep). In current technology, such a thing is not ready. If you look at the Robot Sophia as another example - she acts like Siri: input-output. You speak, she responds. When in the future it will be possible, that robots will get a pulse, and start a process without an end, then we can call it 'thinking like a human' in a philosophical meaning.&lt;/p&gt;&#xA;&lt;p&gt;The second point is the size and processing power of our brain. Billions, if not more, neurons, and trillions, and more connections are in our brain. On YouTube you find neuronal networks with 6 neurons, some have 100. Maybe there's a secret military agency that has a network with million networks - we can't say. Entire rooms are hosted to simulate our brain which fits in just one head.&lt;/p&gt;&#xA;&lt;p&gt;But - to end this respond philosophically: In theory, humans are also machines. Advanced, but still machines. Software that runs on a biological hardware. So any intelligence may be comparable. But since you are mentioning the &lt;em&gt;human&lt;/em&gt; part - the answer would be probably no.&lt;/p&gt;&#xA;" OwnerUserId="31211" LastEditorUserId="2444" LastEditDate="2021-01-20T00:26:06.877" LastActivityDate="2021-01-20T00:26:06.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16662" PostTypeId="2" ParentId="16650" CreationDate="2019-11-20T10:41:02.063" Score="5" Body="&lt;p&gt;Interpreted languages allow for a faster development cycle, as they don't require time for compilation, and fragments can often be run without having a complete program. They often also have fewer constraints for variable declaration or typing. That means they can be used to quickly scope out a problem and try different solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The drawback is the slower execution speed. But during development this is not a big factor; it only becomes important in a production environment. So one option would be to use an interpreted language during the R&amp;amp;D phase, and then re-implement the algorithm in a compiled language for performance improvements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since ML and NNs have become more prevalent in AI, numerical computing has become more important. This is an area where interpreted languages traditionally don't perform too well, so one would use a (compiled) library for, say neural networks, or genetic algorithms, and use 'glue code' to integrate this into a bigger system. The glue code would transform/prepare data and convert this between different formats required by libraries. This is often done in interpreted scripting languages, as they might have to be changed more frequently and are not performance critical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from development, the type of computation is also key: as mentioned, numerical computing generally works better with compiled code, but interpreted languages often have advantages in symbolic programming. This is why Lisp and Prolog have become popular AI languages, as opposed to Fortran or C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an ideal world you would use an interpreted language for development, and then compile this once you're done. However, due to the way these languages work, compilation is often non-trivial.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-11-20T10:41:02.063" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16663" PostTypeId="2" ParentId="16646" CreationDate="2019-11-20T11:55:54.863" Score="1" Body="&lt;p&gt;I would say: &lt;strong&gt;no&lt;/strong&gt;, it's not just an extension of human intelligence.&#xA;Actually, I would argue there's nothing like human intelligence. At least it's not clearly distinguishable from intelligence in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you say AI is just a set of instructions that are made by humans, you might be right. But what if this set of instructions contains instructions on how to change instructions? That would mean that the AI knows how to learn. What if you include instructions on how to learn to learn to learn to learn (...) to change instructions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At what point would you say that this intelligence is still an extension of human intelligence? If you argue like this then you must also put &quot;human intelligence&quot; in a set altogether with every animal intelligence because it all originates from some sort of intelligence that is based on physical brain activity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, when a child is born, it is not more intelligent than most of the animal species. The only thing that enhances its intelligence from time to time (and do stuff like speaking or using its hands like tools) is the ability to &lt;strong&gt;learn&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see why an AI hasn't got the potential to increase its intelligence to level where one would say: &quot;This is not an extension of human intelligence anymore, this is something independent&quot;.&lt;/p&gt;&#xA;" OwnerUserId="31450" LastActivityDate="2019-11-20T11:55:54.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16664" PostTypeId="1" CreationDate="2019-11-20T12:54:46.480" Score="3" ViewCount="21" Body="&lt;p&gt;I am working on multiple deep learning projects, most of them in the area of computer vision. For many of them I create multiple models, try different approaches, use various model architectures. And of course I try to optimize hyperparameters for each model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, that itself works fine. However, I start to lose track of all the various parameters and model layouts I tried. The problem is, sometimes for example I want to re-train a model from a past project with a new data set, but using the same hyperparameters from the last (best) successful training. So I need to look up that project's documentation, or I have some hyperparameters saved in a text or Excel file, etc.pp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me that feels a bit cumbersome. I bet I am not the only one facing this problem, surely there must be a better way than &quot;remembering&quot; all the hyperparamters from all projects / models manually via text files and alike.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are your experiences, have you found a better software / solution / approach / best practice / workflow for that? I must admit, I would welcome a software to aid with that a lot.&lt;/p&gt;&#xA;" OwnerUserId="14504" LastActivityDate="2019-11-20T12:54:46.480" Title="How to organize model training hyperparameters" Tags="&lt;training&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16666" PostTypeId="2" ParentId="16646" CreationDate="2019-11-20T14:17:46.437" Score="3" Body="&lt;p&gt;This is an old question, going back at least to 1950. It is one of the original objections to AI that Turing considers and attempts to refute in his seminal 1950 paper &lt;a href=&quot;https://academic.oup.com/mind/article/LIX/236/433/986238&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing Machinery and Intelligence&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing actually attributes this objection to &lt;a href=&quot;https://en.wikipedia.org/wiki/Ada_Lovelace&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lady Lovelace&lt;/a&gt;, apparently quoted by another author. In Turing's paper, this is objection #6: &lt;em&gt;Lady Lovelace's Objection&lt;/em&gt;, in section 6 of the paper. The objection is concisely stated as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Analytical Engine has no pretensions to originate anything. It can&#xA;  do whatever we know how to order it to perform.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Analytical_Engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Analytical Engine&lt;/a&gt;&quot; was an early design for an all-mechanical general purpose computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing offers two replies to this objection. First, he reminds us that computer programs have bugs. That is, they often do things their creators did not intend. This is unsatisfying to many readers, but it does address the objection: programs may act in ways that are unrelated to our intelligence, and in doing so, might display unexpected intelligent behaviors. In this sense, their intelligence would not be an intentional product of human intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turing's stronger objection comes from an anticipation that &lt;em&gt;learning&lt;/em&gt; would eventually move to the center of AI research (keep in mind again, this is written in 1950, well before any reasonable learning algorithms had been proposed!). Turing uses the example of a robotic child in Section 7 of the paper (Learning Machines) to elaborate on his point. A child is created by its parents, but, endowed with the ability to learn, quickly begins to display behaviors its parents do not anticipate or intend. No one would suggest that a person's intelligence is &quot;really just&quot; the intelligence of their parents, even though their parents created them, and are &lt;em&gt;partially&lt;/em&gt; responsible for that intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, Turing's proposed robotic child is created by a parent, but, endowed with learning, quickly begins to engage in behaviors the parent does not anticipate or intend. Therefore, machine intelligence need not be reduced to just human intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that if Turing were alive today, he would agree that we are now beginning to move into the era of learning machines he anticipated. Some of our programs now engage in intelligent behaviors that we do not anticipate or understand. For example, self-driving cars now &lt;a href=&quot;https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg&quot; rel=&quot;nofollow noreferrer&quot;&gt;kill or maim people&lt;/a&gt;, because they have learned behaviors their creators did not intend or anticipate, perhaps not unlike a reckless teenage driver.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-21T02:12:06.027" LastActivityDate="2019-11-21T02:12:06.027" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16667" PostTypeId="1" AcceptedAnswerId="16670" CreationDate="2019-11-20T14:30:06.803" Score="3" ViewCount="1115" Body="&lt;p&gt;Does the Markov assumption say that the conditional probability of the next state only depends on the current state or does it say that the conditional probability depends on a fixed finite number of previous states?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand from &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property#Introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia article&lt;/a&gt;, the probability of the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; to appear only depends on the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the book &quot;Artificial Intelligence: A Modern Approach&quot; by Russell and Norvig, on page 568, they say: &quot;&lt;strong&gt;Markov assumption&lt;/strong&gt; — that the current state depends on only a &lt;em&gt;finite fixed number&lt;/em&gt; of previous states&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, the second statement seems contradictory to the first, because it may mean that a state can depend on the history of states as long as the number is fixed a finite. For example, the current state depended on the last state and the state before the last state, which is 2 sequential previous states (a finite number of states).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is Markov assumption and Markov property the same?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2019-11-20T23:55:26.197" LastActivityDate="2019-11-20T23:55:26.197" Title="What does the Markov assumption say about the history of state sequences?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;definitions&gt;&lt;norvig-russell&gt;&lt;markov-property&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16668" PostTypeId="1" CreationDate="2019-11-20T14:42:47.673" Score="1" ViewCount="23" Body="&lt;p&gt;I have implemented a multi-label image classification model where I can choose which model to use, I was surprised to find out that in my case mobilenet_v1_224 performed much better (95% Accuracy) than the inception models (around 88% Accuracy), I'm using pretrained models (that I download from &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/research/slim&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and adding a final layer that I train on my own data (3000 images). I wanted to get your opinion and see if maybe I'm doing something wrong.&lt;/p&gt;&#xA;" OwnerUserId="23866" LastActivityDate="2019-11-20T14:42:47.673" Title="Can mobilenet in some cases perform better than inception_v3 and inception_resnet_v2?" Tags="&lt;image-recognition&gt;&lt;tensorflow&gt;&lt;pretrained-models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16669" PostTypeId="1" AcceptedAnswerId="16673" CreationDate="2019-11-20T15:06:44.790" Score="3" ViewCount="44" Body="&lt;p&gt;I am training an RL agent (specifically using the &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO algorithm&lt;/a&gt;) on a game environment with 2 possible actions &lt;strong&gt;left&lt;/strong&gt; or &lt;strong&gt;right&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actions can be taken with varying &quot;force&quot;; e.g. go &lt;strong&gt;left&lt;/strong&gt; 17% or go &lt;strong&gt;right&lt;/strong&gt; 69.3%. Currently, I have the agent output 21 actions - 10 for &lt;strong&gt;left&lt;/strong&gt; (in 10% increments), 10 for &lt;strong&gt;right&lt;/strong&gt; in 10% increments and 1 for stay in place (do nothing). In other words, there is a direct 1-1 mapping in 10% increments between the agent output and the force the agent uses to move in the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering, if instead of outputting 21 possible actions, I change the action space to a binary output and obtain the action probabilities. The probabilities will have the form, say, [70, 30]. That is, go left with 70% probability and go right with 30% probability. Then I take these probabilities and put them through a non-linearity that translates to the actual action force taken; e.g an output of 70% probability to go left, may in fact translate to moving left with 63.8% force.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The non linear translation is not directly observed by the agent but will determine the proceeding state, which is directly observed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't fully understand what the implications of doing this will be. Is there any argument that this would increase performance (rewards) as the agent does not need to learn direct action mappings, rather just a binary probability output?&lt;/p&gt;&#xA;" OwnerUserId="27570" LastActivityDate="2019-11-20T18:04:54.093" Title="Effects of translating RL action probability through non linearity" Tags="&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16670" PostTypeId="2" ParentId="16667" CreationDate="2019-11-20T15:17:35.963" Score="6" Body="&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic process&lt;/a&gt; has the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the following equality holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(s_{t+1} \mid s_{t}, s_{t-1:1}) = p(s_{t+1} \mid s_{t}), \forall t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;hidden Markov model (HMM)&lt;/a&gt; is an example of a model where the Markov property is often assumed to hold. In other words, the Markov assumption is made in the case of the HMM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Variable-order_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;variable-order&lt;/a&gt; (or higher-order) Markov models, where the future state can depend on the previously &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; states or, more formally, the following equality holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(s_{t+1} \mid s_{t}, s_{t-1:1}) = p(s_{t+1} \mid s_{t:t-n}), \forall t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, a hidden Markov model is called a &lt;strong&gt;first-order&lt;/strong&gt; Markov model (&lt;span class=&quot;math-container&quot;&gt;$n=0$&lt;/span&gt;). Therefore, there can also be second-order (&lt;span class=&quot;math-container&quot;&gt;$n=1$&lt;/span&gt;), third-order (&lt;span class=&quot;math-container&quot;&gt;$n=2$&lt;/span&gt;), etc., Markov models. In fact, there are also &lt;a href=&quot;https://arxiv.org/pdf/1805.12201.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;higher-order hidden Markov models&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the expressions &lt;em&gt;Markov property&lt;/em&gt; and &lt;em&gt;Markov assumption&lt;/em&gt; are not exactly interchangeable. The Markov property is an attribute that a stochastic process can be assumed to possess. In that case, the Markov assumption is made. The expression &lt;em&gt;Markov property&lt;/em&gt; usually refers to a first-order Markov property, but it can more generally refer to a higher-order Markov property. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-20T18:32:43.050" LastActivityDate="2019-11-20T18:32:43.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16671" PostTypeId="2" ParentId="16669" CreationDate="2019-11-20T15:38:40.403" Score="1" Body="&lt;p&gt;Have you considered using a continuous action space? It might be worth looking into. If you aren't familiar with it, here are a few resources for discrete vs continuous action spaces - &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://mykel.kochenderfer.com/files/Kochenderfer2005ijcai2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Modeling and Planning in Large State and Action Spaces&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v80/lee18b/lee18b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning in Continuous Action Spaces:a Case Study in the Game of Simulated Curling&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31458" LastActivityDate="2019-11-20T15:38:40.403" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16672" PostTypeId="1" CreationDate="2019-11-20T16:45:10.173" Score="9" ViewCount="1553" Body="&lt;p&gt;I need an algorithm to &lt;a href=&quot;https://en.wikipedia.org/wiki/Image_tracing&quot; rel=&quot;nofollow noreferrer&quot;&gt;trace simple bitmaps&lt;/a&gt; which only contain paths with a given stroke width.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is obviously very easy to generate bitmaps from vector paths, so creating data for a machine learning algorithm is simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is any existing attempt to create a deep learning model which extracts vector paths from bitmaps?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model could be trained by giving both the vector and bitmap representation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once trained, it would be able to generate the vector paths from the given bitmap. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems simple but I could not find any work on this particular task; so I suppose this problem is not fitted for current deep learning architectures, why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to trace this kind of image, which would be drawn by hand with a thick felt pen and scanned: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xZX3N.png&quot; alt=&quot;Bitmap Image containing simple vector paths&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a deep learning architecture fitted for this problem? I believe this question could help me understand what is possible to do with deep learning and what is not, and why. Tracing bitmaps is a perfect example of converting sparse data to a dense abstract representation ; I have the intuition one can learn a lot from this problem.&lt;/p&gt;&#xA;" OwnerUserId="31460" LastEditorUserId="31460" LastEditDate="2019-12-06T13:29:48.330" LastActivityDate="2019-12-06T13:29:48.330" Title="Deep learning for bitmap tracing: extract simple svg paths from bitmaps" Tags="&lt;deep-learning&gt;&lt;algorithm&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16673" PostTypeId="2" ParentId="16669" CreationDate="2019-11-20T18:04:54.093" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I don't fully understand what the implications of doing this will be. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Without other matching adjustments, you will break your agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is how your new action space gets converted &lt;em&gt;back&lt;/em&gt; into gradients to update the agent, after it has acted and needs to learn from the results. The NN component of policy function you are considering is designed to work by balancing a discrete probablility distribution. It learns by increasing the probability of actions (in the binary case, the probability of going left vs going right) that score better than a current baseline level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When interpreting the result from going 63.8% left, you have to resolve two things - which action did the agent take, and what changes to your parameters will increase the probability of taking that action. Unfortunately neither of these tasks are simple if you combine the action choices in the way you suggest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you have lost exploration. The combined left/right algorithm will always output a fixed steering amount for each state. Whilst there are algorithms, like DDPG, that can work with this, it is not really possible to adapt PPO to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, PPO already supports continuous action spaces directly. You can have your network output the mean and standard deviation of a distribution for how to steer, and sample from that. Then the action choice taken will directly relate to the output of the network and you can adjust the policy to make that choice more or less possible depending on results from taking it. If you are using a library implementation of PPO, then this option should be available to you.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-11-20T18:04:54.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16674" PostTypeId="1" CreationDate="2019-11-20T19:21:18.880" Score="2" ViewCount="19" Body="&lt;p&gt;I do text annotations (POS tagging, NER, chunking, synset) by using a specific annotation tool for Natural Language Processing. I would like to make the same annotations on different tools to compare the performances of both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, for I found several logical and linguistic errors in the way the algorithm was previously trained, I would like to measure the way such anomalies affect the intelligence of the chatbot (that's to say its ability to understand questions and answers made by the customers as regard to sentences which have been structured in a certain way), by comparing results with those performed by other NLP engines.&#xA;In other terms, I would like to collect some &quot;benchmark&quot; to have an idea of which level the NLP algorithm developed by the company I work with works at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tool (open source annotation tools based on other NLP algorithms, tools to collect benchmark, etc.) which might help me to perform such a task? &lt;/p&gt;&#xA;" OwnerUserId="22959" LastActivityDate="2019-11-20T19:21:18.880" Title="NLP annotation tool online and other tools to compare performances of different NLP algorithms" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;benchmarks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16675" PostTypeId="1" CreationDate="2019-11-20T19:29:52.787" Score="2" ViewCount="29" Body="&lt;p&gt;While learning neural networks I've found a basic Python working example to play with. It has 3 input nodes, 4 nodes in a hidden layer, 1 output node. 5 data sets for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial code is without biases, which I'm trying to implement, forward and back calculations. From different internet sources I see that bias is just like other weights with a static input value &lt;strong&gt;1&lt;/strong&gt;, and backpropagation calculation should be similar and simplier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my current code version is not working - with the same input I get very different results from &lt;strong&gt;~0.002&lt;/strong&gt; to &lt;strong&gt;~0.99&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me to fix biases calculations. Probably lines marked with &lt;code&gt;???&lt;/code&gt;. Here is a Python 2 testing code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;&#xA;# Sigmoid and it's derivative&#xA;def nonlin(x, deriv=False):&#xA;    if (deriv == True):&#xA;        return x*(1-x)&#xA;&#xA;    return 1/(1+np.exp(-x))&#xA;&#xA;&#xA;X = np.array([[0,0,1],&#xA;              [0,1,1],&#xA;              [1,0,1],&#xA;              [1,1,1],&#xA;              [1,1,1]])&#xA;&#xA;Y = np.array([[0],&#xA;              [1],&#xA;              [1],&#xA;              [0],&#xA;              [0]])&#xA;&#xA;# Static initial hidd. layer weights for testing&#xA;wh = np.array([[-0.16258307,  0.43597283, -0.99471565, -0.39715906],&#xA;               [-0.70551921, -0.81601352, -0.62549935, -0.30959772],&#xA;               [-0.20477763,  0.07532473, -0.15920573,  0.3694664 ]])&#xA;# Static initial output layer weights for testing&#xA;wo = np.array([[-0.59572295],&#xA;               [ 0.74949506],&#xA;               [-0.95195878],&#xA;               [ 0.33625405]])&#xA;&#xA;# Hidden layer's biases&#xA;biasH = 2 * np.random.random((1, 4)) - 1  # ???&#xA;# Output neuron's bias&#xA;biasO = 2 * np.random.random((1, 1)) - 1  # ???&#xA;# Static hidden layer's biases input&#xA;biasInputH = np.array([[1, 1, 1, 1]])     # ???&#xA;# Static output layer's bias input&#xA;biasInputO = np.array([[1]])              # ???&#xA;&#xA;&#xA;# Number of iterations to teach&#xA;for j in xrange(60000):&#xA;&#xA;    # Feedforward&#xA;    h = nonlin(np.dot(X, wh) + biasH)&#xA;    o = nonlin(np.dot(h, wo) + biasO)&#xA;&#xA;    # Calculate partial derivatives &amp;amp; errors&#xA;    o_error = Y - o&#xA;&#xA;    if (j % 10000) == 0:&#xA;        print &quot;Error:&quot; + str(np.mean(np.abs(o_error)))&#xA;&#xA;    o_delta =  o_error * nonlin(o,     deriv=True)&#xA;    o_biases = o_error * nonlin(biasO, deriv=True)  # ???&#xA;&#xA;    h_error =  o_delta.dot(wo.T)&#xA;    h_delta =  h_error * nonlin(h,     deriv=True)&#xA;    h_biases = h_error * nonlin(biasH, deriv=True)  # ???&#xA;&#xA;    # Update weights and biases&#xA;    wo += h.T.dot(o_delta)&#xA;    wh += X.T.dot(h_delta)&#xA;&#xA;    # biasH += biasInputH.dot(h_delta)  # ???&#xA;    # biasO += biasInputO.dot(o_delta)  # ???&#xA;&#xA;&#xA;# Try new data&#xA;data = np.array([1,0,0])&#xA;&#xA;print &quot;weights 0:&quot;, wh&#xA;print &quot;weights 1:&quot;, wo&#xA;print &quot;biases 0:&quot;,  biasH&#xA;print &quot;biases 1:&quot;,  biasO&#xA;print &quot;input:   &quot;,  data&#xA;&#xA;h = nonlin(np.dot(data, wh))&#xA;print &quot;hidden:  &quot;, h&#xA;print &quot;output:  &quot;, nonlin(np.dot(h, wo))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31464" LastActivityDate="2019-11-20T19:29:52.787" Title="Calculation of Neural network biases in backpropagation" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16676" PostTypeId="1" CreationDate="2019-11-20T21:05:29.130" Score="1" ViewCount="53" Body="&lt;p&gt;I am working on implementing MCTS for a scheduling problem where MCTS is formulated each time there are multiple jobs that need to be scheduled. When a job is executed, the resulting state of the system is random. The challenge I'm having is that the implementation I'm currently using relies on the ability to determine if a node is fully expanded. However, there are so many children of the root node that it's not feasible to expect all of them will ever be visited. Is there a suggested method of conducting MCTS in cases where nodes will not likely ever be fully expanded?&lt;/p&gt;&#xA;" OwnerUserId="31466" LastActivityDate="2019-11-20T21:05:29.130" Title="Formulating MCTS with random outcomes of actions?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16677" PostTypeId="1" CreationDate="2019-11-20T22:24:13.327" Score="3" ViewCount="74" Body="&lt;p&gt;I am looking to build a neural network that takes an input vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; and outputs a vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Y}$&lt;/span&gt; such at &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{X}, \mathbf{Y})$&lt;/span&gt; is minimized, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is some function. The network will see many different &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; during training to adjust its weights and biases; then I will test the network by using the test set &lt;span class=&quot;math-container&quot;&gt;$\{x_1, \dots, x_n \}$&lt;/span&gt; to calculate &lt;span class=&quot;math-container&quot;&gt;$\sum(f(x_1, y), \dots, f(x_n, y))$&lt;/span&gt; to see if this sum is minimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have no labels for the output &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Y}$&lt;/span&gt;. The loss function I am trying to minimize is based on the input and output instead of the output and label. I tried many standard Keras and TensorFlow loss functions, but they are unable to do the job. Any thoughts on how this might be achieved? &lt;/p&gt;&#xA;" OwnerUserId="31468" LastEditorUserId="32410" LastEditDate="2021-04-23T01:38:53.427" LastActivityDate="2021-05-23T02:05:15.537" Title="Unsupervised learning to optimize a function of the input" Tags="&lt;neural-networks&gt;&lt;optimization&gt;&lt;unsupervised-learning&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16678" PostTypeId="2" ParentId="16677" CreationDate="2019-11-21T00:32:28.807" Score="0" Body="&lt;p&gt;According to your description, you already know your function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; to be optimized. So you should use it directly instead of the standard loss functions. In this &lt;a href=&quot;https://stackoverflow.com/questions/43818584/custom-loss-function-in-keras&quot;&gt;other post&lt;/a&gt; there is an explanation of how to use &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; as a custom loss function in Keras.&lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2019-11-21T00:32:28.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16679" PostTypeId="1" CreationDate="2019-11-21T00:44:32.513" Score="1" ViewCount="61" Body="&lt;p&gt;I'm reading about max-pooling in a dynamic CNN paper. I can see how it can help find features in images, given that the pixel with the highest density gets pooled, but how does it help to find features in words?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2021-01-11T00:51:27.203" LastActivityDate="2021-02-10T01:05:20.467" Title="How can max-pooling be applied to find features in words?" Tags="&lt;convolutional-neural-networks&gt;&lt;pooling&gt;&lt;max-pooling&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16680" PostTypeId="2" ParentId="16646" CreationDate="2019-11-21T00:46:39.170" Score="1" Body="&lt;p&gt;No, the way human minds think is in no way related to the way an AI thinks. Although you could say that AI is a much simpler form that represents how the brain processes information. For the human brain to think, sense, and act there are billions of connections is various cortex's of the brain that process information in different ways. If talking about brain information as electrical signals you could say that different cortex's of the brain have change in power of specific frequency bands of the brain signal which can be decoded as planning, preparation, thoughts, visual, movement, creativity, attentiveness and much more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question AI could be considered as an extremely minute extension of human intelligence. It's like comparing our solar system to the Milky Way, although the comparison maybe a bit too large as we are slowly becoming able to understand the underlying processes and build fast processors mimicking brain processing and efficient power consuming hardware tech to run humongous neural nets. In the soon future your statement may hold true.&lt;/p&gt;&#xA;" OwnerUserId="16740" LastActivityDate="2019-11-21T00:46:39.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16681" PostTypeId="2" ParentId="16679" CreationDate="2019-11-21T02:28:17.230" Score="0" Body="&lt;p&gt;In an image you are pooling usually over some (n x n) set of positions which lets you maintain spatial correlation but on the other hand most 1D CNNs used for language modeling pool over the temporal axis completely annihilating all form of temporal correlation within the resulting feature vector. I will take it this difference is what confuses you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity sake Ill keep my explanation to max-pooling but it can extend to any of the others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;MaxPooling as the name suggests just takes a maximum value from a set of candidates. In images you commonly see maxpooling over neighborhoods of pixels. This is because if we assume high activation is correlated to some feature being more prevalent (this assumption is not inherent, its more learned than anything else by construction of the architecture/training), then doing this we keep the most prevalent features, maintain spatial correlation, and reduce the dimensionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using 1D CNNs for LMs rather than RNNs the idea is that each kernel is &lt;em&gt;searching&lt;/em&gt; for a feature, and so you use an exorbitant number of the kernels at various sizes to &lt;em&gt;search&lt;/em&gt; for a lot of features. Pooling over the temporal dimension then just tells you at each neuron how prevalent its corresponding feature is in that time series input. That is why your output is usually a feature vector of size equal to the number of kernels you used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A point I want to emphasize is that pooling takes multiple inputs and returns one based on some condition/function but that is vague, and it's usage in different scenarios can be for different goals (to an extent).&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-11-21T02:28:17.230" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16682" PostTypeId="5" CreationDate="2019-11-21T03:06:29.600" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T03:06:29.600" LastActivityDate="2019-11-21T03:06:29.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16683" PostTypeId="4" CreationDate="2019-11-21T03:06:29.600" Score="0" Body="For questions related to the C4.5 algorithm, which is a decision tree learning developed by Ross Quinlan in 1993. The C4.5 algorithm is the successor of the ID3 algorithm, which was developed in 1986 by Ross Quinlan." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T03:06:29.600" LastActivityDate="2019-11-21T03:06:29.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16684" PostTypeId="1" AcceptedAnswerId="16732" CreationDate="2019-11-21T03:25:01.757" Score="1" ViewCount="73" Body="&lt;p&gt;I was learning about the &lt;em&gt;maximum a posteriori probability (MAP)&lt;/em&gt; estimation for machine learning and I found &lt;a href=&quot;https://www.youtube.com/watch?v=XepXtl9YKwc&quot; rel=&quot;nofollow noreferrer&quot;&gt;a nice short video&lt;/a&gt; that essentially explained it as finding a distribution and tweaking the parameters to fit the observed data in a way that makes the observations most likely (makes sense). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in mathematical terms, how does it determine which distribution best fits the data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are so many distributions out there that it could be any of them and the parameters you could fit them could be infinitely large.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2019-11-21T14:02:24.133" LastActivityDate="2019-11-23T17:14:12.193" Title="How does maximum approximation of the posterior choose a distribution?" Tags="&lt;probability-distribution&gt;&lt;bayesian-probability&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16685" PostTypeId="2" ParentId="15992" CreationDate="2019-11-21T03:41:08.580" Score="0" Body="&lt;h1&gt;My thoughts&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;AI is already indirectly regulated. This is important to acknowledge and this acknowledgement is missing, in my opinion, in the discourse about law and AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming that your question is about law that directly aims at AI technologies and this exemplifies one of the risks of regulating AI: that the law will focus on the technology rather than outcomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another concern is that law that is inadequate or outdates quickly creates a false sense of security and this could create a situation which is even more dangerous then if the laws are not there.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Law and innovation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When it comes to the view that law stifles innovation it is paramount to acknowledge that some regulation can have a very positive effect. There is no general rule that there is a inverse relation between law and innovation.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Pacing problem and Collingridge dilemma&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The following is basically what Wendell Wallach says in an espisode of Future of Life Institute's &lt;a href=&quot;https://futureoflife.org/ai-alignment-podcast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI Alignment Podcast&lt;/a&gt; entitled &lt;a href=&quot;https://futureoflife.org/2019/11/15/machine-ethics-and-ai-governance-with-wendell-wallach/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Machine Ethics and AI Governance with Wendell Wallach&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The pacing problem refers to the fact scientific discovery, and technological innovation, is far outpacing our ability to put in place appropriate ethical legal oversight.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Wendell Wallach continues to say that pacing problem converges with what is now called the Collingridge Dilemma, a problem that 'bedevilled' people in technology and governance since 1980, and he defines it the following way:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;While it was easiest to regulate a technology early in its development, early in its development we have little idea of what its societal impact would be. By the time we did understand what the challenges and the societal impact the technology would be so deeply entrenched in our society that it would be very difficult to change its trajectory. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Collingridge_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Collingridge dilemma&lt;/a&gt; on Wikipedia; and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://openlibrary.org/books/OL14443859M/The_social_control_of_technology&quot; rel=&quot;nofollow noreferrer&quot;&gt;The social control of technology&lt;/a&gt; by David Collingridge, published 1980 by Frances Pinter.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3526" LastEditorUserId="3526" LastEditDate="2019-11-21T22:39:37.983" LastActivityDate="2019-11-21T22:39:37.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16686" PostTypeId="2" ParentId="2876" CreationDate="2019-11-21T05:00:58.533" Score="0" Body="&lt;p&gt;Here are some problems that my ape mind came up with.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;1.Smart != all knowing&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The AI self improvement explosion smakes it smarter and smarter. Being smarter doesn't mean knowing more facts. I suppose this is quite a picky argument, but I think worth thinking about. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very smart doctor who doesn't know your history may still make a worse choice than a less intelligent one with better data.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;2. Is it all for humans? Is it for all humans?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;ASI which reaches a higher level could not be interested in our wellbeing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A controlled ASI could still work for the benefit of the few only, if these few decide wrong goals we could go backwards.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;3 Harsh ASI&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;A scientific mind in not necessarily full of sympathy or empathy.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;4. Being smart and not being clever&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Great minds still make mistakes: &#xA;   * in setting their goals, and&#xA;   * in executing the plan to achieve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geat intellect doesn' guarantee lack of shorosightedness or lack of blind spots.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;5. Limits&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If there are bounds of existence (speed of light type limits) then the AI will be bound by these as well. This may mean that there are things that even ASI won't 'get'.&#xA;Also, as our mind may have limits based on its structure the next AI may have limits as well - and even if it improves uppn may hit limits that it cannot find solutions to because it's 'too stupid'.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;6. We won't get it&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;ASI's understanding of certain aspects of the world may not be communicative to most humans. We just won't get it (even if we're capable of understanding everything, it doesn't mean we will understand it).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;7. How to use it?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;We may destroy ourselves, and the AI with the tech it helps us build. It doesn't need to be bombs. It can be geoengineering or wonder drugs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is especially intense when the ASI is already powerful but not strong enough to forsee negative consequences (or we'll just ignore it anyway).&lt;/p&gt;&#xA;" OwnerUserId="3526" LastActivityDate="2019-11-21T05:00:58.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16687" PostTypeId="1" AcceptedAnswerId="16720" CreationDate="2019-11-21T10:16:42.907" Score="5" ViewCount="73" Body="&lt;p&gt;In the context of the variational auto-encoder, can someone give me a concrete example of the application of the Bayes' rule&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p_{\theta}(z|x)=\frac{p_{\theta}(x|z)p(z)}{p(x)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a given latent variable and observable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand with VAE's we're essentially getting an approximation to &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(z|x)$&lt;/span&gt; that models the distribution that we think approximates the latent variables, but I need a concrete example to really understand it.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2019-11-22T20:35:45.777" LastActivityDate="2019-11-22T20:35:45.777" Title="Concrete example of latent variables and observables plugged into the Bayes' rule" Tags="&lt;convolutional-neural-networks&gt;&lt;autoencoders&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16688" PostTypeId="1" CreationDate="2019-11-21T13:10:13.240" Score="3" ViewCount="147" Body="&lt;p&gt;Why AI is (or not) a good option for the generation of random numbers? Would GANs be suited for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="31483" LastEditorUserId="2444" LastEditDate="2019-11-21T13:49:23.893" LastActivityDate="2019-11-21T15:05:41.980" Title="Why AI is (or not) a good option for the generation of random numbers?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;randomness&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16689" PostTypeId="1" CreationDate="2019-11-21T14:50:25.227" Score="3" ViewCount="17" Body="&lt;p&gt;I am trying to predict the solution time for riddles in which matchsticks are combined into digits and operators. An example of a matchstick riddle is 4-2=8. The solution for this riddle would be obtained by moving one matchstick from the ‘8’ to the ‘-’ sign, resulting in the correct equation 4+2=6. The data consists of 100 riddles and the corresponding solution times. The two types of features that are available for each riddle are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a 23 dimensional binary vector that indicates which of the available positions are filled with matches&#xA;or&lt;/li&gt;&#xA;&lt;li&gt;a 12-dimensional integer vector that counts the appearance of each token (10 digits, 2 operators)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although today neural nets are very popular I am not sure that a neural net is the best choice for this particular problem. Firstly, because the data set is very small. Secondly because of the binary inputs.  What might be a more effective model for this problem ? &lt;/p&gt;&#xA;" OwnerUserId="31485" LastActivityDate="2019-11-21T14:50:25.227" Title="What is a good model for regression problem with binary features and small data?" Tags="&lt;models&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16690" PostTypeId="2" ParentId="16688" CreationDate="2019-11-21T14:52:37.190" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why AI is (or not) a good option for the generation of random numbers? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;AI approaches are generally not good for generating random numbers, for these reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Similar to why they are not good for adding numbers, there already exist many strong pseudo-random and &quot;true&quot; random sources, possible without using any AI approach, and demonstrably good enough for purposes such as simulations and cryptography. These existing algorithms and devices perform much more efficiently at the task than any AI could.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generating random numbers with good statistical properties is a very precise task, that the approximations typically used in machine learning cannot cope with (a pseudo-random function is inherently noisy and difficult to learn). AI search techniques are not helpful either, because the amount of output that needs to be generated in order to validate a random data source is very large, making things like tree searches for solutions intractable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One possible relationship between AI and random number generation that could be explored is in code generation and code analysis for random number generators. In other words, an AI that is sophisticated enough to understand the complexity of code used in a typical random number generator (RNG) might help with making random number generation more efficient, or less easy to hack, or expose flaws in existing implementations. This is beyond the capabilities of AI techniques at the moment - or put another way, human intelligence has taken RNG technology quite far already, so there is very little practical that current AIs can do to improve on it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would GANs be suited for this purpose?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. The input to a GAN is typically a vector of random numbers that have been generated, by an already-existing RNG. The purpose the the GAN is convert this simply-distributed random vector input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; - which might be suitable for many other purposes, such as running simulations of stochastic processes - into a &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; sampled from a distribution of some target population &lt;span class=&quot;math-container&quot;&gt;$\mathbb{Y}$&lt;/span&gt;, which is typically being expressed as a subset of a large space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^d$&lt;/span&gt;. The values that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; will take are biased to a subspace (or &lt;em&gt;manifold&lt;/em&gt;) of all possible space within the representations possible in that vector space. For instance, you might input a random vector with 100 variables and the GAN will generate a picture of a dog. It cannot generate anything other than attempts at pictures of dogs, whilst the space of all possible images that it might create if completely random is far larger (and usually not interesting to look at).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a GAN would take an already-good random source and turn it into a biased set of numbers that would be generally unusable as a RNG without further processing. The kind of processing you would do (e.g. &quot;noise whitening&quot;) is something that some current &quot;secure&quot; random generators do - meaning to use the GAN to generate quality random numbers you would literally put existing RNG code on its input and output layers. The GAN would add nothing but wasted computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You cannot even train a GAN to do this task, since a truly random space contains all possible outputs, there will be no negative examples for the discriminator. Although a random picture most often looks like static noise, and a picture of the Mona Lisa is clearly considered &quot;not random&quot; by most people, in fact it is just as likely to be seen as any other image.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-11-21T15:05:41.980" LastActivityDate="2019-11-21T15:05:41.980" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16691" PostTypeId="1" CreationDate="2019-11-21T15:24:55.533" Score="2" ViewCount="14" Body="&lt;p&gt;I want to build the yolo architecture in keras but can't understand the basic idea behind the training of the yolo, like how to define the labels for whether there is no object there what we have to do. Do we have to take the boundary box as 0 or not include the block only for that part? It's quite confusing. &lt;/p&gt;&#xA;" OwnerUserId="31487" LastActivityDate="2019-11-21T15:24:55.533" Title="yolo output and how to define labels for backpropogation on it" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16692" PostTypeId="1" CreationDate="2019-11-21T16:42:06.473" Score="1" ViewCount="25" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_boosting&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia article on gradient boosting&lt;/a&gt;. There is written:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unfortunately, choosing the best function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; at each step for an arbitrary loss function &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is a computationally infeasible optimization problem in general. Therefore, we  restrict our approach to a simplified version of the problem.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How would the &quot;best function&quot; been constructed if there are no computationally limitations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the wiki gives me the idea that the model could been better, but that compromises have been made.&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-21T21:17:24.653" LastActivityDate="2019-11-21T21:17:24.653" Title="How would the &quot;best function&quot; been constructed if there are no computationally limitations?" Tags="&lt;machine-learning&gt;&lt;loss-functions&gt;&lt;gradient-boosting&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16693" PostTypeId="5" CreationDate="2019-11-21T16:42:20.367" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T16:42:20.367" LastActivityDate="2019-11-21T16:42:20.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16694" PostTypeId="4" CreationDate="2019-11-21T16:42:20.367" Score="0" Body="For questions related to the depth-first search algorithm, in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T16:42:20.367" LastActivityDate="2019-11-21T16:42:20.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16695" PostTypeId="1" AcceptedAnswerId="16700" CreationDate="2019-11-21T17:20:14.047" Score="0" ViewCount="50" Body="&lt;p&gt;I'm relatively new to Deep Learning, and trying various models and datasets using &lt;code&gt;Keras&lt;/code&gt;. I'm starting to love it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Through-out my experimentations, I have come into some semantic questions that I don't know how they can affect the overall accuracy of my trained model. My target application is fire detection in videos (fire vs non-fire). So I'm trying to get tips and tricks from those well experienced on Deep learning, and here are my semantic questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Given that I have to do detection on videos, I've been mostly adding actuall frames of videos to my dataset, and less photos. Does adding photos from Google ever help (as we largen our dataset) or it's actually more considered noises and shall be removed?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've trained a deep model (&lt;code&gt;ResNet50&lt;/code&gt;) as well as a shallow 5-layer model. I realized the ResNet50 model is more sensitive and has a high recall (all fires are definitely detected), but has false positives as well (strong source of lights like sunlight or lamps are identified as fire). While the shallower model is 10x faster, it can miss fires if it is smaller in the image, so it's less sensitive. But also has low false positives. Is it always true? So what are techniques and tips to fix these issues in each of these models? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For instance, the shallow model doesn't see this fire. Shall I think it's not complex enough to work well when the scene has many objects inside?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gwrjrm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gwrjrm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;The sample code I saw resizes photos to &lt;code&gt;256x256&lt;/code&gt; for training. What's the effect of bigger sizes vs smaller ones say &lt;code&gt;300x300&lt;/code&gt;? Can I expect while bigger sizes increase computation time, they provide higher accuracy?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The sample code also converts photos to grayscale and uses Antialiasing before passing. Does it have good effects? What if I pass the colored version as fire is mostly about colors?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When I see the model is doing bad on certain scenes (say those sun lights or lamps), I take multiple of those frames and add them to my non-fire dataset. Does it have any positive effects and being taken care of? And is it better to add multiple successive frames or just one frame is enough? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My fire dataset has &lt;code&gt;1800&lt;/code&gt; images and my non-fire dataset has &lt;code&gt;4500&lt;/code&gt; images. As a rule of thumb, the bigger each class, the better? Of course the non-fire data should be bigger, but we can not add whatever on earth as non-fire so what should be the distribution of the sizes?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-11-21T23:13:59.690" LastActivityDate="2019-11-21T23:13:59.690" Title="Techniques and semantics in better training of deep learning models" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;keras&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ClosedDate="2020-11-24T02:58:17.440" ContentLicense="CC BY-SA 4.0" />
  <row Id="16696" PostTypeId="1" CreationDate="2019-11-21T18:28:46.903" Score="2" ViewCount="26" Body="&lt;p&gt;I trained a binary classifier using ML.NET's AutoML feature on a small dataset (compared to other, similar models I've trained that seem to work well)-around 500 rows with around 50 features. AutoML used cross-validation with 5 folds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training data is balanced to about 200 positive cases to 300 negative cases, which isn't an unreasonable representation of the real world based on domain knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model's metrics are poor compared to other, similar models, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Accuracy: 0.64&lt;/li&gt;&#xA;&lt;li&gt;Positive Precision: 0.375&lt;/li&gt;&#xA;&lt;li&gt;Positive Recall: 0.09&lt;/li&gt;&#xA;&lt;li&gt;Negative Precision: 0.67&lt;/li&gt;&#xA;&lt;li&gt;Negative Recall: 0.92&lt;/li&gt;&#xA;&lt;li&gt;F1 Score: 0.15&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When the model is run against unseen data, it predicts the negative case 99% of the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the accuracy were truly as stated in the metric, a correct classification 2/3 of the time has some practical value in this application. However, the actual predictions of 99% the negative case are surely flawed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the training set too small to expect reasonable results? Is there anything I can do to improve the model? &lt;/p&gt;&#xA;" OwnerUserId="31489" LastActivityDate="2019-11-21T18:28:46.903" Title="Troubleshooting Binary Classifier" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16698" PostTypeId="1" CreationDate="2019-11-21T20:12:56.263" Score="2" ViewCount="32" Body="&lt;p&gt;I'm trying to use DQN to solve the cart-pole environment. I have 2 networks (target and behavior). Both of them have 3 hidden layers with 24 neurons, using the ReLU activation. The loss is MSE and the optimizer is Adam. I copy the weights of the behavior network to the target network every 15 backpropagation steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My agent learns. Below you can see the &lt;em&gt;total reward&lt;/em&gt; and &lt;em&gt;running average&lt;/em&gt; plots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W2liv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W2liv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it has a lot of &quot;drops&quot;. Usually, after a couple of perfect sequences, it just &quot;kills&quot; the running average with a couple of very short episodes. What may be the reason for this behavior?&lt;/p&gt;&#xA;" OwnerUserId="31494" LastEditorUserId="2444" LastEditDate="2019-11-22T00:11:29.977" LastActivityDate="2019-11-22T00:11:29.977" Title="What could be the cause of the drop of the total reward when using DQN to solve the cart-pole environment?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16700" PostTypeId="2" ParentId="16695" CreationDate="2019-11-21T22:26:14.503" Score="2" Body="&lt;p&gt;I try to answer the things I know for sure:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;One effect of bigger images is the increasing computation time due to more pixels (input to your training)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;4.Grayscaling reduces the information, which might decrease training time, but also model performance (accuracy, precision, recall). What I have seen is that grayscaling is used in for example face detection where structural information (shapes) is important and colors can be regarded as secondary. For your example you have to determine whether such structural differences between positives and negatives exist and if color is a valuable information.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;If possible, I would somehow alter these copies by clipping the objects (sun lights, lamps) and rotate, scale, make them transparent etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the end your training data should be representative of the data it should generalize to. If you would for example take my user icon as an additional non fire example, this wouldn't help your cause.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31498" LastActivityDate="2019-11-21T22:26:14.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16701" PostTypeId="5" CreationDate="2019-11-21T22:29:41.727" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T22:29:41.727" LastActivityDate="2019-11-21T22:29:41.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16702" PostTypeId="4" CreationDate="2019-11-21T22:29:41.727" Score="0" Body="For questions related to ensemble learning, which refers to machine learning techniques where multiple models (e.g. a neural network and a decision tree) are trained and their predictions are combined to solve the same problem. Bagging and boosting are two popular ensemble learning techniques." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-21T22:29:41.727" LastActivityDate="2019-11-21T22:29:41.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16703" PostTypeId="1" CreationDate="2019-11-21T22:38:18.797" Score="3" ViewCount="114" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1502.05477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trust Region Policy Optimization&lt;/a&gt; paper, in Lemma 1 of Appendix A, I did not quite understand the transition from (21) from (20). In going from (20) to (21), &lt;span class=&quot;math-container&quot;&gt;$A^\pi(s_t, a_t)$&lt;/span&gt; is substituted with its value. The value of &lt;span class=&quot;math-container&quot;&gt;$A^\pi(s_t, a_t)$&lt;/span&gt; is given as &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{s'∼P(s'|s,a)}[r(s) + \gamma V_\pi(s') − V_\pi(s)]$&lt;/span&gt; at the very begining of the proof. But when &lt;span class=&quot;math-container&quot;&gt;$A^\pi(s_t, a_t)$&lt;/span&gt; gets substituted, I don't see the expectation (over &lt;span class=&quot;math-container&quot;&gt;$s'∼P(s'|s,a)$&lt;/span&gt;) appearing anywhere. It will be of great help if somebody lends some light on this.&lt;/p&gt;&#xA;" OwnerUserId="31499" LastEditorUserId="2444" LastEditDate="2019-11-21T23:15:48.690" LastActivityDate="2021-04-15T22:49:49.730" Title="Understanding proof of lemma 1 (policy improvement bound) of the &quot;Trust Region Policy Optimization&quot; paper" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;papers&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16704" PostTypeId="2" ParentId="15992" CreationDate="2019-11-21T22:51:07.327" Score="1" Body="&lt;p&gt;I don't think regulating something necessarily causes that regulation to defacto become a &quot;risk&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regulation - including overregulation - may, in fact, aid in the dialogue between practitioners, which may end up educating the regulators, the public and the practitioners themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My answers to your survey would most likely be &quot;it depends...&quot;, or &quot;no risk&quot;, which isn't to say it's not an impediment, but just not a &quot;risk&quot;, per se.&lt;/p&gt;&#xA;" OwnerUserId="31496" LastActivityDate="2019-11-21T22:51:07.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16705" PostTypeId="1" CreationDate="2019-11-21T23:03:08.430" Score="1" ViewCount="34" Body="&lt;p&gt;How would we define a set that contains itself within a knowledge ontology? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking that &lt;em&gt;set membership&lt;/em&gt; would probably inherit from a generic base class of &lt;em&gt;total containment&lt;/em&gt; from which both &lt;em&gt;physical containment&lt;/em&gt; and &lt;em&gt;conceptual containment&lt;/em&gt; are derived. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;total containment&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;physical containment&lt;/li&gt;&#xA;&lt;li&gt;conceptual containment&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;set containment&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31497" LastEditorUserId="2444" LastEditDate="2019-11-21T23:17:23.227" LastActivityDate="2019-11-21T23:17:23.227" Title="How would we define a set that contains itself within a knowledge ontology?" Tags="&lt;knowledge-representation&gt;&lt;ontology&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16706" PostTypeId="1" CreationDate="2019-11-21T23:57:54.807" Score="4" ViewCount="87" Body="&lt;p&gt;Imagine a game played on a 10x10 grid system where a player can move up down left or right and imagine there are two players on this grid: An enemy and you. In this game, there are walls on the grid which you can't go through. The objective of this game is to block the enemy in so he can't move around the rest of the board and is effectively &quot;trapped&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to write an algorithm that detects which nodes on the board I as a player need to put blocks in, in order to trap the enemy. There are also some other considerations to think about. You have to be able to place the blocks before the enemy place can get out of the box. Also note more thing: You can move AND place a block in the position that you're moving to at the same time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a picture as an example of the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LQXVc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LQXVc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: note that the board in the picture is 5x5, but that's okay for the purposes of the example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, I could go up, then right and place a block, then right and place a block, then up and place a block. If there's more than one way of blocking off the enemy, then I should use the way that's going to give my enemy the least amount of space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researching on google couldn't find me anything relevant, although it may have been because I wasn't using relevant search terms. I also thought about using a monte Carlo search tree algorithm for simultaneous games, but I would need to research into that more. &lt;/p&gt;&#xA;" OwnerUserId="31501" LastEditorUserId="2444" LastEditDate="2019-11-22T00:04:07.740" LastActivityDate="2021-04-15T19:47:13.803" Title="How could an AI detect whether an enemy in a game can be blocked off/trapped?" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16707" PostTypeId="1" CreationDate="2019-11-22T00:16:45.453" Score="5" ViewCount="140" Body="&lt;p&gt;Suppose we have &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; products that we want to detect. For each of these products, we have &lt;span class=&quot;math-container&quot;&gt;$500$&lt;/span&gt; training images/annotations. Thus we have &lt;span class=&quot;math-container&quot;&gt;$500,000$&lt;/span&gt; training images/associated annotations. If we want to train a good object detection algorithm to recognize these objects (e.g. YOLO) would it be better to have multiple detection models? In other words, should we have 10 different YOLO models where each YOLO model is responsible for detecting 100 products? Or is it good enough to have one YOLO model that can detect all 1000 products? Which would be better in terms of mAP/recall/precision? &lt;/p&gt;&#xA;" OwnerUserId="28201" LastEditorUserId="2444" LastEditDate="2019-11-23T18:12:17.877" LastActivityDate="2019-11-24T17:04:11.280" Title="Should I train different models for detecting subsets of objects?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16708" PostTypeId="5" CreationDate="2019-11-22T00:45:17.463" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-22T00:45:17.463" LastActivityDate="2019-11-22T00:45:17.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16709" PostTypeId="4" CreationDate="2019-11-22T00:45:17.463" Score="0" Body="For questions related to the concept of novelty search (NS), where individuals in an evolving population (of an evolutionary algorithm) are selected based on how different they are compared to all of the other individuals evaluated so far. NS was proposed in &quot;Exploiting Open-Endedness to Solve Problems Through the Search for Novelty&quot; (2008) by Joel Lehman and Kenneth O. Stanley." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-22T00:45:17.463" LastActivityDate="2019-11-22T00:45:17.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16710" PostTypeId="2" ParentId="5347" CreationDate="2019-11-22T02:14:28.597" Score="3" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://eplex.cs.ucf.edu/papers/lehman_alife08.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exploiting Open-Endedness to Solve Problems Through the Search for Novelty&lt;/a&gt; (2008), by Joel Lehman and Kenneth O. Stanley, which introduced the novelty search approach, it is written&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thus this paper introduces the novelty search algorithm, which searches with no objective other than continually finding &lt;strong&gt;novel behaviors&lt;/strong&gt; in the search space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;instead of searching for a final objective, the learning method is rewarded for finding any instance whose &lt;strong&gt;functionality&lt;/strong&gt; is significantly different from what has been discovered before&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The novelty of a newly generated individual is computed with respect to the &lt;strong&gt;behaviors&lt;/strong&gt; (i.e. &lt;strong&gt;not the genotypes&lt;/strong&gt;) of an archive of past individuals whose behaviors were highly novel when they originated&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Therefore, the goal of a novelty search is to search for novel behavior and not necessarily novel chromosomes (or genotypes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the experiments reported in the novelty search paper, the authors use neural networks to represent the &lt;a href=&quot;https://stackoverflow.com/q/46260775/3924118&quot;&gt;policy&lt;/a&gt; that controls a robot that needs to navigate a maze, while evolving these neural networks with &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt; (a &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuroevolution&lt;/a&gt; method) and a &lt;em&gt;novelty metric&lt;/em&gt; (rather than a fitness metric, which is used in the original &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt;). In the same experiments section, Lehman and Stanley write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thus, for the maze domain, the behavior of a navigator is defined as its ending position. The novelty metric is then the Euclidean distance between the ending positions of two individuals. For example, two robots stuck in the same corner appear similar, while one robot that simply sits at the start position looks very different from one that reaches the goal, though they are both equally viable to the novelty metric.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Therefore, the evolution of the neural networks, which represent the controllers, is not necessarily guided by the novelty of (the architecture of) the neural networks but by the novelty of the behavior generated by the neural networks, even though novel neural networks might correspond or lead to novel behaviors.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-22T02:26:30.797" LastActivityDate="2019-11-22T02:26:30.797" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16713" PostTypeId="1" CreationDate="2019-11-22T10:09:33.890" Score="2" ViewCount="51" Body="&lt;p&gt;Here is a &lt;a href=&quot;https://www.researchgate.net/post/List_of_metaheuristics_algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;list of meta-heuristic algorithms&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ant colony optimization, &lt;/li&gt;&#xA;&lt;li&gt;Ant lion optimizer, &lt;/li&gt;&#xA;&lt;li&gt;Artificial bee colony algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Bat algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Cat swarm optimization, &lt;/li&gt;&#xA;&lt;li&gt;Crow search algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Cuckoo optimization algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Cuckoo search algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Differential evolution, &lt;/li&gt;&#xA;&lt;li&gt;Firefly algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Genetic algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Glowworm swarm optimization, &lt;/li&gt;&#xA;&lt;li&gt;Gravitational search algorithm, &lt;/li&gt;&#xA;&lt;li&gt;Grey wolf optimizer, &lt;/li&gt;&#xA;&lt;li&gt;Harmony search, &lt;/li&gt;&#xA;&lt;li&gt;Multi-verse optimizer, &lt;/li&gt;&#xA;&lt;li&gt;Particle swarm optimization, &lt;/li&gt;&#xA;&lt;li&gt;Shuffled complex evolution, &lt;/li&gt;&#xA;&lt;li&gt;Simulated annealing, &lt;/li&gt;&#xA;&lt;li&gt;Tabu search, &lt;/li&gt;&#xA;&lt;li&gt;Teaching-learning-based optimization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain the similarities and dissimilarities of evolutionary game theory and the meta-heuristic approach?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2019-11-22T14:50:35.210" LastActivityDate="2019-11-22T14:50:35.210" Title="What is the difference between evolutionary game theory and meta-heuristics?" Tags="&lt;comparison&gt;&lt;evolutionary-algorithms&gt;&lt;game-theory&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16714" PostTypeId="1" CreationDate="2019-11-22T10:09:53.327" Score="1" ViewCount="115" Body="&lt;p&gt;Problems I often face at work usually differ from tutorial or book-like examples so I end up with a code that works but it's not elegant and takes too much time to write.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask you if there are some publicly accesible examples or repositories of Python codes that deal with machine learning development and application process but were created in a real company or organisation to develop their real-life products or services?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: What I do not think about are libraries or packages repositories such as tensorflow. I would like to see some codes of projects that for example use tensorflow to create some other product or service.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2019-11-24T15:27:22.103" LastActivityDate="2020-05-25T09:42:52.657" Title="Are there any public real-life code examples of ML applications in Python?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;applications&gt;" AnswerCount="3" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16715" PostTypeId="2" ParentId="16703" CreationDate="2019-11-22T10:18:21.273" Score="0" Body="&lt;p&gt;Let's assume &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; to simplify things&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbb E_{\tau|\pi} [\sum_{t = 0}^{\infty} A_\pi(s_t, a_t)] &amp;amp;= \mathbb E_{\tau|\pi}[A_\pi(s_0, a_0) + \ldots A_\pi(s_i, a_i) + \ldots]\\&#xA;&amp;amp;= \mathbb E_{a_0 \sim \pi,s_1 \sim P(s_1|s_0, a_0)}[A_\pi(s_0, a_0)] + \ldots + \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] + \ldots&#xA;\end{align}&lt;/span&gt;&#xA;if we observe only &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th timestep&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &amp;amp;= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a')]) \pi(a'|s_i)\\&#xA;&amp;amp;= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]]) \pi(a'|s_i)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mathbb E[\mathbb E[f]] = \mathbb E[f]&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &amp;amp;= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]) \pi(a'|s_i)\\&#xA;&amp;amp;= \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now for all timesteps sum everything up.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-11-22T10:18:21.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16716" PostTypeId="1" CreationDate="2019-11-22T10:26:57.503" Score="1" ViewCount="17" Body="&lt;p&gt;Question on transfer learning object classification (MobileNet_v2 with 75% number of parameters) with my own synthetic data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made my own dataset of three shapes: triangles, rectangles and spheres. each category has 460 samples with diferent sizes, dimensions, different wobbles at edges. They look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NAgwS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NAgwS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yuxVt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yuxVt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7GFyG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7GFyG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the network to classify these primitive shapes in other environments as well with different lighting/color conditions and image statistics.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though I'm adding random crops, scaling, and brightnesses, at training step 10 it's already at 100% training and validation accuracy. Cross entropy keeps going down though. I'm using tensorflow hub. The performance of the network in the end could be better within other environments (virtual 3d space with such shapes). Also trained and tested for ~ 50 steps to see if the network is overfitting, but that doesn't work too well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What alterations would you  recommend to generalize better? Or shouldn't I train on synthetic data at all to learn primitive shapes? If so, any dataset recommendations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="31180" LastActivityDate="2019-11-22T10:26:57.503" Title="learning object recognition of primitive shapes through transfer learning problem" Tags="&lt;datasets&gt;&lt;object-recognition&gt;&lt;object-detection&gt;&lt;transfer-learning&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16717" PostTypeId="1" CreationDate="2019-11-22T12:03:58.380" Score="6" ViewCount="59" Body="&lt;p&gt;In my implementation of Thompson sampling (TS) for online Reinforcement Learning, my distribution for selecting &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C(s,a)$&lt;/span&gt; is the number of times &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; has been picked in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. However, I found that this does not work well in some cases depending on the magnitude of &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;. For example, if &lt;span class=&quot;math-container&quot;&gt;$Q(s_i,a_1) = 100$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C(s_i,a_1) = 1$&lt;/span&gt;, then then this gives a standard deviation of 0.5 which is extremely confident even though the action has only been picked once. Compare that to &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; which may be the optimal action but has never been picked, so &lt;span class=&quot;math-container&quot;&gt;$Q(s_i, a_2) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C(s_i,a_2) = 0$&lt;/span&gt;. It is unlikely that TS will ever pick &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do I solve this problem? I tried normalizing the Q-values such that it range from 0 to 1 but the algorithm returns much lower total returns. I think I have to adapt the magnitude of the standard deviations relative to the Q-values as well. Doing it for 1 normal distribution is pretty straightforward, but I can't figure out how to do it for multiple distributions which have to take into consideration of the other distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Counts should be &lt;span class=&quot;math-container&quot;&gt;$C(s,a)$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$C(s)$&lt;/span&gt; as Neil pointed out&lt;/p&gt;&#xA;" OwnerUserId="31518" LastEditorUserId="31518" LastEditDate="2019-11-22T13:56:57.697" LastActivityDate="2019-11-22T13:56:57.697" Title="Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;online-learning&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16718" PostTypeId="2" ParentId="16706" CreationDate="2019-11-22T14:49:20.260" Score="0" Body="&lt;p&gt;If I've understood the logic correctly, the player tries to build a wall such that the enemy cannot reach the player in any way. This can normally be determined by some path-finding algorithms, Dijkstra Shortest Path algorithm being a reasonable choice for the setting and grid-size. &#xA;This algorithm explores the possible paths from a starting point to one or multiple end points, and usually returns the shortest path to the point(s). If there is no path to the mentioned point, it will not return anything, and you will know the two points are blocked from reaching each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if I've truly understood the rules correctly, the bigger question is how to avoid having the player just build a wall around themselves to block of the enemy instead, which is likely going to be trivially easy. And additionally rule of requiring the player to be in the &quot;room&quot; with the bigger area after a room has been blocked off (Something which coincidentally also could be done by extending the Dijkstra's algorithm a bit with custom logic)&lt;/p&gt;&#xA;" OwnerUserId="10746" LastActivityDate="2019-11-22T14:49:20.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16720" PostTypeId="2" ParentId="16687" CreationDate="2019-11-22T16:50:58.203" Score="4" Body="&lt;p&gt;Let's assume the probability distributions are Gaussian (or normal) distributions. In other words, in the Bayes' rule&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;p(z|x)=\frac{p(x|z)p(z)}{p(x)}&#xA;\tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The posterior &lt;span class=&quot;math-container&quot;&gt;$p(z|x)$&lt;/span&gt;, the likelihood &lt;span class=&quot;math-container&quot;&gt;$p(x|z)$&lt;/span&gt;, the prior &lt;span class=&quot;math-container&quot;&gt;$p(z)$&lt;/span&gt; and the evidence (or marginal) &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; are Gaussian distributions. You can assume this because &lt;a href=&quot;https://distill.pub/2019/visual-exploration-gaussian-processes/#Multivariate&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian distributions are closed under conditioning and marginalization&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's further assume that they are &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;univariate&lt;/strong&gt; Gaussian distributions&lt;/a&gt;. Given that the Gaussian distribution is a &lt;em&gt;continuous probability distribution&lt;/em&gt;, it has an associated &lt;em&gt;probability density function&lt;/em&gt; (rather than a probability mass function, which is associated with discrete probability distributions, such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bernoulli_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bernoulli distribution&lt;/a&gt;). The probability density function of the Gaussian distribution is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(x \mid \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2} } e^{ -\frac{(x-\mu)^2}{2\sigma^2} } \tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; are respectively the mean and variance of the Gaussian distribution and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a variable (similarly to a variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in any mathematical function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;). So, given a concrete value for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, for example, &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f(x=1, \mu, \sigma^2)$&lt;/span&gt; is a so-called &lt;strong&gt;density&lt;/strong&gt; value (rather than a probability, which a probability mass function returns, given an input). For example, let's assume that the mean &lt;span class=&quot;math-container&quot;&gt;$\mu=0$&lt;/span&gt; and the variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2 = 1$&lt;/span&gt;, then, for &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;, the density will be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(1 \mid 0, 1) = \frac{1}{\sqrt{2\pi} } e^{ -\frac{1}{2} }&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to obtain the concrete density value, I've just replaced the concrete values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; in equation \ref{2}. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the posterior &lt;span class=&quot;math-container&quot;&gt;$p(z|x)$&lt;/span&gt; in equation \ref{1}, you just need to replace the likelihood &lt;span class=&quot;math-container&quot;&gt;$p(x|z)$&lt;/span&gt;, the prior &lt;span class=&quot;math-container&quot;&gt;$p(z)$&lt;/span&gt; and the evidence &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; with the Gaussian probability density shown in equation \ref{2}, so you will have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;p(z|x)=\frac{f_{X\mid Z}(x \mid \mu_{X\mid Z}, \sigma^2_{X\mid Z}, z) f_{Z}(z \mid \mu_{Z}, \sigma^2_{Z})}{f_{X}(x \mid \mu_{X}, \sigma^2_{X})}&#xA;\tag{3}\label{3}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've explicitly added a subscript to the means and variances of each probability density, given that, for example, the mean of the probability density &lt;span class=&quot;math-container&quot;&gt;$f_{X\mid Z}$&lt;/span&gt; might be different than the mean of the probability density &lt;span class=&quot;math-container&quot;&gt;$f_{Z}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$f_{X}$&lt;/span&gt;, etc. So, to get the actual density value (a real number) that represents &lt;span class=&quot;math-container&quot;&gt;$p(z|x)$&lt;/span&gt;, you just need to replace &lt;span class=&quot;math-container&quot;&gt;$f_{X\mid Z}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_{Z}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_{X}$&lt;/span&gt; with the definition of the Gaussian density function in \ref{2} with their actual mean and variance values. I'll let you do this, given that this is really just a matter of picking up a concrete value for the means and variances and doing some algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you assume the posterior, the likelihood, the prior or the evidence to have a different distribution, you will do the same thing, but using the probability density or mass function of your chosen distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of the variational auto-encoder, you will be learning the mean and variance of the distribution, so the mean and variance will not be fixed, but they will be the parameters that you want to find. However, this does not change the way you apply the Bayes' rule.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-22T19:24:34.210" LastActivityDate="2019-11-22T19:24:34.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16722" PostTypeId="1" CreationDate="2019-11-22T23:15:14.340" Score="3" ViewCount="186" Body="&lt;p&gt;I know backpropagation uses cost and gradient descent to tweak the weights to minimize the cost. But how does it know which weights to give more weight to in the first place? Is there something inside each neuron in the hidden layers that defines how this is an important neuron for the correct result in some way? How does the network know how to tweak those weights for that specific neuron?&lt;/p&gt;&#xA;" OwnerUserId="31534" LastEditorUserId="2444" LastEditDate="2019-11-22T23:19:16.557" LastActivityDate="2021-01-20T21:46:20.003" Title="How does the neural-network know how to tweak weights for a specific neuron?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16723" PostTypeId="2" ParentId="16722" CreationDate="2019-11-22T23:58:17.790" Score="3" Body="&lt;h3&gt;tl;dr&lt;/h3&gt;&#xA;&lt;p&gt;The whole point of gradient descent is to assess the &lt;strong&gt;contribution&lt;/strong&gt; of each parameter towards the loss. This information is uncovered through the gradient of the loss w.r.t each parameter.&lt;/p&gt;&#xA;&lt;h3&gt;A deeper look...&lt;/h3&gt;&#xA;&lt;p&gt;Suppose we have a NN with parameters &lt;span class=&quot;math-container&quot;&gt;$w_{i}, \; i={1, 2, ...}$&lt;/span&gt;. This NN makes some predictions, which we compare to the actual targets and compute a loss &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;. The loss (or cost) function tells us &lt;strong&gt;how far off&lt;/strong&gt; we are from the target. This is what we want to &lt;strong&gt;reduce&lt;/strong&gt;, so that the predictions fall closer to the target.&lt;/p&gt;&#xA;&lt;p&gt;By computing the partial derivative of the loss function &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; w.r.t a parameter &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; (so this is just one partial derivative and not the full gradient vector)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial J}{\partial w_i}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;the NN uncovers two pieces of information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;strong&gt;slope&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; w.r.t &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;, which tells the NN &lt;em&gt;how much &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; affects &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Its &lt;strong&gt;sign&lt;/strong&gt;, which tells the NN &lt;em&gt;which way to tweak &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; to decrease (or increase) the value of &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;By making the parameter updates depend on the derivative&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_i^{new} \leftarrow w_i^{old} - \lambda \frac{\partial J}{\partial w_i}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;the NN causes parameters that affect the loss the most, to be updated the most.&lt;/p&gt;&#xA;&lt;p&gt;You can think of this as: parameters that are more to blame for the network's mistakes (i.e. contribute more towards the loss) are forced to change the most, in the direction that will decrease the loss.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="2444" LastEditDate="2021-01-20T21:46:20.003" LastActivityDate="2021-01-20T21:46:20.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16725" PostTypeId="1" CreationDate="2019-11-23T08:10:19.513" Score="3" ViewCount="46" Body="&lt;p&gt;I read very often that &lt;a href=&quot;https://math.stackexchange.com/q/2589224/168764&quot;&gt;Bayesian algorithms work well on small datasets&lt;/a&gt;. Why is that? I think it is because they might generalize more, but why is that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=7636&amp;amp;context=etd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Investigating the use of Bayesian networks for small dataset problems&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-23T17:27:30.137" LastActivityDate="2019-11-23T17:27:30.137" Title="Why do Bayesian algorithms work well with small datasets?" Tags="&lt;machine-learning&gt;&lt;bayesian-network&gt;&lt;naive-bayes&gt;&lt;bayesian-statistics&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16726" PostTypeId="2" ParentId="4377" CreationDate="2019-11-23T08:46:49.760" Score="2" Body="&lt;p&gt;There are so many different versions of spiking neural networks out there. I think it is mainly due to the fact that there has been no dominant successful SNN model with proper learning algorithm like CNN with BP. However, there have been several recent papers(e.g. &lt;a href=&quot;https://arxiv.org/abs/1705.11146&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuperSpike&lt;/a&gt;, &lt;a href=&quot;https://papers.nips.cc/paper/7415-slayer-spike-layer-error-reassignment-in-time&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLAYER&lt;/a&gt;) on SNN that may lead to the standard framework for SNN. It happened within about 2 years, so it is one of the reasons why there have been no friendly introductions on SNN so far. I found some blogposts on SNN in general sense, but it didn't cover recent important trends in SNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the best way to learn about SNN is by reading papers. One paper that I would recommend is &lt;a href=&quot;https://arxiv.org/abs/1901.09948&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Surrogate Gradient Learning in Spiking Neural Networks&quot;&lt;/a&gt; which comprehensively reviews recent works on supervised learning in SNN with back-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to implement the SNN from scratch, I would recommend you to checkout BindsNET(&lt;a href=&quot;https://github.com/BindsNET/bindsnet&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;, &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fninf.2018.00089/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;) which is an SNN framework based on PyTorch. To me, it was most intuitive to use and understand compared to other existing SNN libraries. It covers various neuron models and learning rules. But I'm not sure whether it also covers learning rules described in &lt;a href=&quot;https://arxiv.org/abs/1901.09948&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Surrogate Gradient Learning in Spiking Neural Networks&quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31541" LastActivityDate="2019-11-23T08:46:49.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16727" PostTypeId="1" CreationDate="2019-11-23T09:24:36.710" Score="2" ViewCount="55" Body="&lt;p&gt;I am trying to build a Multi label classification model, having dataset with different input numerical values and specific label...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eg:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Value Label&lt;/p&gt;&#xA;&#xA;&lt;p&gt;35   X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;35.8 X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;29   Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;29.8 Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;39   AA&lt;/p&gt;&#xA;&#xA;&lt;p&gt;41   CB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So depending on input numerical value the model should specify its label....please note that the input values won't necessarily follow exact dataset values....eg dataset has 35 and 34.8 as input values with X as label. So if model has 35.4 as input label, the X should be output label. Bottom line is that the output label is based on range of input values instead of fixed one.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me with quick solution (example Jupyter notebook will be highly appreciated)&lt;/p&gt;&#xA;" OwnerUserId="30642" LastEditorUserId="31544" LastEditDate="2019-11-23T13:38:18.663" LastActivityDate="2019-11-23T16:42:31.240" Title="Multi label Classification using Keras" Tags="&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16728" PostTypeId="1" AcceptedAnswerId="16765" CreationDate="2019-11-23T11:26:17.327" Score="1" ViewCount="173" Body="&lt;p&gt;I was talking with an ex-fellow worker and he told me that the decision tree implicitly applies a feature selection. He told me that the most important feature is higher in the tree because of the usage of information gain criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does he mean with this and how does this work?&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="2444" LastEditDate="2019-11-23T17:54:47.337" LastActivityDate="2019-11-25T18:51:00.740" Title="How does the decision tree implicitly do feature selection?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;&lt;feature-selection&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16729" PostTypeId="1" CreationDate="2019-11-23T14:22:43.330" Score="4" ViewCount="103" Body="&lt;p&gt;For the purposes of object detection, are there any easy ways to create annotated training images? For example, if we have &lt;span class=&quot;math-container&quot;&gt;$10,000$&lt;/span&gt; images and want to draw bounding boxes on 2 objects for each image, do we have to physically draw those boxes? Is that what most people do these days to create training data?&lt;/p&gt;&#xA;" OwnerUserId="31547" LastEditorUserId="2444" LastEditDate="2019-11-23T17:52:26.483" LastActivityDate="2020-12-04T10:15:08.293" Title="Are there any easy ways to create annotated training images for object detection?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16731" PostTypeId="2" ParentId="16727" CreationDate="2019-11-23T16:42:31.240" Score="1" Body="&lt;p&gt;For a simple multi layer perceptron, you can refer to here: &#xA;&lt;a href=&quot;https://www.kaggle.com/fchollet/simple-deep-mlp-with-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/fchollet/simple-deep-mlp-with-keras&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a great resource for kerad multi input label classification. Also, here is a few reminders for implementing such classification model.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;One hot encoding&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the sample data you provided, it seems like you are using raw numbers as input. This will create unnecessary complications for the model. A better approach will be to encode them into a vector with 0 in all except one index with 1. The index will be the numerical value you are encoding. For example, 1 will be 01000000... Till end of your range of input, and 2 will be 001000 and so on. If you have decimal values, then this approach isn't for you. However you should still scale down your input to something smaller like 0-1. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;output encoding&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The output seems to be text, which cannot be done and also complicates the problem. Instead you can also use one hot encoding for the output. For example, let A be 0, B be 1, C be 2... AA be 27.... Until your list ends. And then use one hot encoding to encode them to vector of 0 and 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-11-23T16:42:31.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16732" PostTypeId="2" ParentId="16684" CreationDate="2019-11-23T17:08:42.543" Score="2" Body="&lt;h3&gt;Introduction: MAP finds a point estimate!&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As opposed to your apparently current belief, in maximum a posteriori (MAP) estimation, you are looking for a &lt;em&gt;point estimate&lt;/em&gt; (a number or vector) rather than a full probability distribution. The MAP estimation can be seen as a &lt;a href=&quot;http://www.scholarpedia.org/article/Bayesian_statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian version&lt;/a&gt; of the maximum likelihood estimation (MLE). Therefore, I will first remind you of the objective of MLE.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Maximum likelihood estimation (MLE)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; be the parameters you want to find. For example, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; can be the weights of your neural network. In MLE, we want to find a point estimate (rather than a full distribution). The objective in MLE is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\theta^* &#xA;&amp;amp;= \operatorname{argmax}_\theta p(X \mid \theta) \tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p(X \mid \theta)$&lt;/span&gt; is the likelihood of the data &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; given the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. In other words, we want to find the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$p(X \mid \theta)$&lt;/span&gt; is the highest, where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is your given training data, so &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is fixed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notation &lt;span class=&quot;math-container&quot;&gt;$p(X \mid \theta)$&lt;/span&gt; can be confusing because, in a conditional probability distribution, &lt;span class=&quot;math-container&quot;&gt;$p(a\mid b)$&lt;/span&gt;, we often assume that &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is given and &lt;span class=&quot;math-container&quot;&gt;$p(a\mid b)$&lt;/span&gt; is a distribution over &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. However, in the case of MLE, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$p(X \mid \theta)$&lt;/span&gt; is not fixed, but it is a variable, while &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is given and fixed. Hence we call &lt;span class=&quot;math-container&quot;&gt;$p(X \mid \theta)$&lt;/span&gt; a &lt;strong&gt;likelihood&lt;/strong&gt; rather than a probability density or mass function. Moreover, we often denote the likelihood as &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(\theta; X) = p_{\theta}(X)$&lt;/span&gt; (and there are other notations, but this is, in my opinion, the least confusing one), because we want to emphasize that the likelihood is actually a function of the variable &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. However, this is notation can also be confusing because we equate a function of a variable &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; to a probability distribution over &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. However, you should note that &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(X)$&lt;/span&gt; is parametrized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the MLE estimation \ref{1} can also be written as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\theta^* &#xA;&amp;amp;= &#xA;\operatorname{argmax}_\theta \mathcal{L}(\theta; X) \\&#xA;&amp;amp;=\operatorname{argmax}_\theta p_{\theta}(X)&#xA;\tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\theta^*$&lt;/span&gt; is the &lt;strong&gt;point estimate&lt;/strong&gt; of the objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This notation emphasizes the fact that we want to find &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, such that the probability of the given data &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is maximized.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Maximum a posteriori (MAP)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;MAP is similar to MLE, but the objective is slightly different. First of all, we assume that &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is a random variable, so we have an associated probability distribution &lt;span class=&quot;math-container&quot;&gt;$p(\theta)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that the Bayes' rule is the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;p(\theta \mid X) = \frac{p(X \mid \theta) p(\theta)}{p(X)}&#xA;\tag{3}\label{3}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function in MAP estimation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\theta^* &#xA;&amp;amp;= \operatorname{argmax}_\theta \frac{p(X \mid \theta) p(\theta)}{p(X)} \\&#xA;&amp;amp;= \operatorname{argmax}_\theta p(\theta \mid X) \tag{4}\label{4}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$p(X)$&lt;/span&gt; does not depend on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, for the purposes of optimization, we can ignore it, so equation \ref{4} becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\theta^* &#xA;&amp;amp;= \operatorname{argmax}_\theta p(X \mid \theta) p(\theta) \\&#xA;&amp;amp;= \operatorname{argmax}_\theta p(\theta \mid X)&#xA;\tag{5}\label{5}&#xA;\end{align}&lt;/span&gt;&#xA;which is the MAP objective.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is the relationship between MLE and MAP?&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In MAP, the objective is \ref{5}, which includes a prior over &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, while, in MLE, equation \ref{1}, there is no such thing. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Therefore, in MAP, we can assume that the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; follow a certain distribution, thanks to the usage of &lt;span class=&quot;math-container&quot;&gt;$p(\theta)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In both MAP and MLE, we want to find a &lt;strong&gt;point estimate&lt;/strong&gt; (which can be a number, if you have just one parameter, or a vector of size &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, if you have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; parameters).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MAP is equivalent to MLE if you use a uniform prior, that is, if &lt;span class=&quot;math-container&quot;&gt;$p(\theta)$&lt;/span&gt; is a uniform distribution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Which distribution fits the data &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In MAP, the human (you, me, etc.) chooses the family of distributions. For example, you can assume that your parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; follow a Gaussian distribution, so &lt;span class=&quot;math-container&quot;&gt;$p(\theta)$&lt;/span&gt; will be a Gaussian distribution over the parameters. Why do I say &quot;family&quot;? For example, in the case of a Gaussian distribution, you have two parameters that control the shape of the distribution, namely, the mean and variance. Depending on the concrete values of these two parameters, you will have different Gaussian distributions, so you call all these Gaussian distributions a family.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How do you find &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;To find &lt;span class=&quot;math-container&quot;&gt;$\theta^*$&lt;/span&gt;, you can use an optimization method like &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent&lt;/a&gt; or, in certain cases, you can find a &lt;a href=&quot;http://mathworld.wolfram.com/Closed-FormSolution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;closed-form solution&lt;/a&gt;. See also &lt;a href=&quot;https://stats.stackexchange.com/q/32103/82135&quot;&gt;Which distributions have closed-form solutions for maximum likelihood estimation?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Resources&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The following blog post &lt;a href=&quot;https://wiseodd.github.io/techblog/2017/01/01/mle-vs-map/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLE vs MAP: the connection between Maximum Likelihood and Maximum A Posteriori Estimation&lt;/a&gt;, by Agustinus Kristiadi (a Ph.D. student in machine learning), might also be useful, so I suggest you read it. It will give you more details that I've left out on purpose to avoid cluttering this answer.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-23T17:14:12.193" LastActivityDate="2019-11-23T17:14:12.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16733" PostTypeId="5" CreationDate="2019-11-23T17:29:25.160" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;http://www.scholarpedia.org/article/Bayesian_statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scholarpedia.org/article/Bayesian_statistics&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-23T17:29:25.160" LastActivityDate="2019-11-23T17:29:25.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16734" PostTypeId="4" CreationDate="2019-11-23T17:29:25.160" Score="0" Body="For questions related to Bayesian statistics in the context of artificial intelligence. Bayesian statistics is a system for describing uncertainty using the mathematical language of probability. Bayesian statistical methods start with existing &quot;prior&quot; beliefs, and update these using data to give &quot;posterior&quot; beliefs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-23T17:29:25.160" LastActivityDate="2019-11-23T17:29:25.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16735" PostTypeId="5" CreationDate="2019-11-23T18:26:14.643" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-23T18:26:14.643" LastActivityDate="2019-11-23T18:26:14.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16736" PostTypeId="4" CreationDate="2019-11-23T18:26:14.643" Score="0" Body="For questions related to the concept of boosting in machine learning, which is a collection of ensemble learning methods." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-23T18:26:14.643" LastActivityDate="2019-11-23T18:26:14.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16737" PostTypeId="2" ParentId="16536" CreationDate="2019-11-24T03:51:20.990" Score="5" Body="&lt;h1&gt;Introduction&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Deep Learning&lt;/a&gt; provides a good overview (in section 2) of several results regarding the concept of generalisation in deep learning. I will try to describe one of the results (which is based on concepts from &lt;a href=&quot;http://www2.denizyuret.com/ref/angluin/coltsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational&lt;/a&gt; or &lt;a href=&quot;http://www.econ.upf.edu/~lugosi/mlss_slt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical&lt;/a&gt; learning theory, so you should expect a technical answer), but I will first introduce and describe the &lt;strong&gt;general machine learning problem&lt;/strong&gt; and I will give a definition of the &lt;strong&gt;generalisation gap problem&lt;/strong&gt;. To keep this answer relatively short, I will assume the reader is familiar with certain basic machine learning and mathematical concepts, such as &lt;a href=&quot;http://papers.nips.cc/paper/506-principles-of-risk-minimization-for-learning-theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;expected risk minimization&lt;/em&gt;&lt;/a&gt;, but, nonetheless, I will refer the reader to more detailed explanations of the concepts (at least the first time they are mentioned). If you are familiar with the basic concepts of &lt;a href=&quot;http://www2.denizyuret.com/ref/angluin/coltsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational learning theory (e.g. hypotheses)&lt;/a&gt;, you will be advantaged.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Machine Learning Problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In the following description, unless stated otherwise, I do not make any assumption about the nature of the variables. However, I will occasionally provide examples of concrete values for these variables.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x \in \mathcal{X}$&lt;/span&gt; be an input and let &lt;span class=&quot;math-container&quot;&gt;$y \in \mathcal{Y}$&lt;/span&gt; be a target. Let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; be a loss function (e.g. MSE). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the &lt;a href=&quot;http://papers.nips.cc/paper/506-principles-of-risk-minimization-for-learning-theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;expected&lt;/strong&gt; risk&lt;/a&gt; of a function (or hypothesis) &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R[f] &#xA;&amp;amp;= \mathbb{E}_{x, y \sim \mathbb{P}(X, Y)} \left[ \mathcal{L}(f(x), y) \right] \\&#xA;&amp;amp;= \int \mathcal{L}(f(x), y) d\mathbb{P}(X=x, Y=y),&#xA;\end{align}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X, Y)$&lt;/span&gt; is the &lt;em&gt;true&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Joint_probability_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;joint probability distribution&lt;/a&gt; of the inputs and targets. In other words, each &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; is drawn from the joint distribution &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X, Y)$&lt;/span&gt;, which contains or represents all the information required to understand the relationship between the inputs and the targets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; be a learning algorithm or learner (e.g. gradient descent), which is the algorithm responsible for choosing a hypothesis &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (which can e.g. be represented by a neural network with parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;). Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S_m = \{(x_i, y_i) \}_{i=1}^m$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;be the training dataset. Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_{A(S_m)} : \mathcal{X} \rightarrow \mathcal{Y}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;be the &lt;a href=&quot;http://www2.denizyuret.com/ref/angluin/coltsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;hypothesis&lt;/a&gt; (or model) chosen by the learning algorithm &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; using the training dataset &lt;span class=&quot;math-container&quot;&gt;$S_m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://papers.nips.cc/paper/506-principles-of-risk-minimization-for-learning-theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;empirical&lt;/strong&gt; risk&lt;/a&gt; can then be defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_{S_m}[f] = \frac{1}{m} \sum_{i=1}^m \mathcal{L} (f(x_i), y_i) &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the total number of training examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; be the &lt;a href=&quot;http://www2.denizyuret.com/ref/angluin/coltsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;hypothesis space&lt;/a&gt; (for example, the space of all neural networks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{L_F} = \{ g : f \in F , g(x, y) = \mathcal{L}(f(x), y)\}&#xA;$$&lt;/span&gt; be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Family_of_sets&quot; rel=&quot;nofollow noreferrer&quot;&gt;family&lt;/a&gt; of loss functions associated with the hypothesis space &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Expected Risk Minimization&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In machine learning, the goal can be framed as the &lt;strong&gt;minimization&lt;/strong&gt; of the &lt;a href=&quot;http://papers.nips.cc/paper/506-principles-of-risk-minimization-for-learning-theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;expected risk&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f^*_{A(S_m)} &#xA;&amp;amp;= \operatorname{argmin}_{f_{A(S_m)}} R[f_{A(S_m)}] \\&#xA;&amp;amp;= \operatorname{argmin}_{f_{A(S_m)}} \mathbb{E}_{x, y \sim \mathbb{P}(X, Y)} \left[ \mathcal{L}(f_{A(S_m)}(x), y) \right] \tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the expected risk &lt;span class=&quot;math-container&quot;&gt;$R[f_{A(S_m)}]$&lt;/span&gt; is &lt;a href=&quot;https://en.wikipedia.org/wiki/Computability&quot; rel=&quot;nofollow noreferrer&quot;&gt;incomputable&lt;/a&gt;, because it is defined as an &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectation&lt;/a&gt; over &lt;span class=&quot;math-container&quot;&gt;$x, y \sim \mathbb{P}(X, Y)$&lt;/span&gt; (which  is defined as an integral), but the true joint probability distribution &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X, Y)$&lt;/span&gt; is &lt;strong&gt;unknown&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Empirical Risk Minimization&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Therefore, we solve the approximate problem, which is called the &lt;a href=&quot;http://papers.nips.cc/paper/506-principles-of-risk-minimization-for-learning-theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;empirical risk minimization&lt;/strong&gt;&lt;/a&gt; problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f^*_{A(S_m)} &amp;amp;= \operatorname{argmin}_{f_{A(S_m)} \in F} R_S[f_{A(S_m)}] \\&#xA;&amp;amp;= &#xA;\operatorname{argmin}_{f_{A(S_m)} \in F} \frac{1}{m} \sum_{i=1}^m \mathcal{L} (f_{A(S_m)}(x_i), y_i) &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Generalization&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In order to understand the generalization ability of &lt;span class=&quot;math-container&quot;&gt;$f_{A(S_m)}$&lt;/span&gt;, the hypothesis chosen by the learner &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with training dataset &lt;span class=&quot;math-container&quot;&gt;$S_m$&lt;/span&gt;, we need to understand when the &lt;strong&gt;empirical risk minimization&lt;/strong&gt; problem is a good proxy for the &lt;strong&gt;expected risk minimization&lt;/strong&gt; problem. In other words, we want to study the following problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R[f_{A(S_m)}] -  R_S[f_{A(S_m)}] \tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be called the &lt;strong&gt;generalization gap&lt;/strong&gt; problem. So, in generalization theory, one goal is to study the gap between the expected and empirical risks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, we would like the expected risk to be equal to the empirical risk &lt;span class=&quot;math-container&quot;&gt;$$R_S[f_{A(S_m)}] = R[f_{A(S_m)}]$$&lt;/span&gt; because this would allow us to measure the performance of the hypothesis (or model) &lt;span class=&quot;math-container&quot;&gt;$f_{A(S_m)}$&lt;/span&gt; with the empirical risk, which can be computed. So, if &lt;span class=&quot;math-container&quot;&gt;$R_S[f_{A(S_m)}] = R[f_{A(S_m)}]$&lt;/span&gt;, the generalization ability of &lt;span class=&quot;math-container&quot;&gt;$f_{A(S_m)}$&lt;/span&gt; roughly corresponds to &lt;span class=&quot;math-container&quot;&gt;$R_S[f_{A(S_m)}]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, in generalization theory, one goal is to provide &lt;a href=&quot;https://en.wikipedia.org/wiki/Upper_and_lower_bounds&quot; rel=&quot;nofollow noreferrer&quot;&gt;bounds&lt;/a&gt; for the generalisation gap &lt;span class=&quot;math-container&quot;&gt;$R[f_{A(S_m)}] -  R_S[f_{A(S_m)}]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Dependency on &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The hypothesis &lt;span class=&quot;math-container&quot;&gt;$f_{A(S_m)}$&lt;/span&gt; is explicitly dependent on the training dataset &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. How does this dependency affect &lt;span class=&quot;math-container&quot;&gt;$f_{A(S_m)}$&lt;/span&gt;? Can we avoid this dependency? Several approaches have been proposed to deal with this dependency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following sub-section, I will describe &lt;strong&gt;one&lt;/strong&gt; approach to deal with the generalization gap problem, but you can find a description of the &lt;strong&gt;stability&lt;/strong&gt;, &lt;strong&gt;robustness&lt;/strong&gt; and &lt;strong&gt;flat minima&lt;/strong&gt; approaches in &lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Deep Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Hypothesis-space Complexity&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In this approach, we try to avoid the dependency of the hypothesis &lt;span class=&quot;math-container&quot;&gt;$f_{A(S_m)}$&lt;/span&gt; by considering the &lt;strong&gt;worst-case&lt;/strong&gt; generalization problem in the hypothesis space &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R[f_{A(S_m)}] -  R_S[f_{A(S_m)}] \leq \sup_{f \in F} \left( R[f] -  R_S[f] \right)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\sup_{f \in F} \left( R[f] -  R_S[f] \right)$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Infimum_and_supremum&quot; rel=&quot;nofollow noreferrer&quot;&gt;supremum&lt;/a&gt; of a more general generalization gap problem, which is greater or equal to \ref{2}. In other words, we solve a more general problem to decouple the hypothesis (or model) from the training dataset &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Bound 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you assume the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; to take values in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, then, for any &lt;span class=&quot;math-container&quot;&gt;$\delta &amp;gt; 0$&lt;/span&gt;, with probability &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt; (or more), the following bound holds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\sup_{f \in F} \left( R[f] -  R_S[f] \right) \leq 2 \mathcal{R}_m \left( \mathcal{L}_F \right) + \sqrt{\frac{\log{\frac{1}{\delta}} }{2m}} \tag{3} \label{3}&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the size of the training dataset, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}_m$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rademacher_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rademacher complexity&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_F$&lt;/span&gt;, which is the family of loss functions for the hypothesis space &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; (defined above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This theorem is proved in &lt;a href=&quot;https://mitpress.ublish.com/ereader/7093/?preview=#page/29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations of machine learning&lt;/a&gt; (2nd edition, 2018) by Mehryar Mohri et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other bounds to this bound, but I will not list or describe them here. If you want to know more, have a look at the &lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;literature&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will also not attempt to give you an intuitive explanation of this bound (given that I am also not very familiar with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rademacher_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rademacher complexity&lt;/a&gt;). However, we can already understand how a change in &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; affects the bound. What happens to the bound if &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; increases (or decreases)?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Conclusion&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;There are several approaches to find bounds for the generalisation gap problem \ref{2}&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hypothesis-space complexity&lt;/li&gt;&#xA;&lt;li&gt;Stability&lt;/li&gt;&#xA;&lt;li&gt;Robustness&lt;/li&gt;&#xA;&lt;li&gt;Flat minima&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In section 2 of the paper &lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Deep Learning&lt;/a&gt;, bounds for problem \ref{2} are given based on the stability and robustness approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, the study of the generalization ability of deep learning models is based on &lt;a href=&quot;http://www2.denizyuret.com/ref/angluin/coltsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational&lt;/a&gt; or &lt;a href=&quot;http://www.econ.upf.edu/~lugosi/mlss_slt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical&lt;/a&gt; learning theory. There are many more results related to this topic. You can find some of them in &lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Deep Learning&lt;/a&gt;. The studies and results are highly technical, so, if you want to understand something, good knowledge of mathematics, proofs, and computational learning theory is required.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-25T16:23:00.320" LastActivityDate="2019-11-25T16:23:00.320" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16738" PostTypeId="1" AcceptedAnswerId="16752" CreationDate="2019-11-24T07:24:26.767" Score="3" ViewCount="330" Body="&lt;p&gt;Consider the following diagram of a graph representing a search space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NgNuB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NgNuB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we start at &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and try to reach goal state &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, the lowest-cost first search (LCFS) (aka uniform-cost search) algorithm fails to find a solution. This is because, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; selects &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; to expand as &lt;span class=&quot;math-container&quot;&gt;$f(A)=g(A)=36 &amp;lt; f(C)=g(C)=70$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt; is the cost function of node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the cost of reaching node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; from the start state. Continuing further, from &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, LCFS will now select &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to expand, which in turn will select &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; again over &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. This leads to an infinite loop. This shows LCFS is incomplete (not guaranteed to find a solution, if one exists).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For A*, we define &lt;span class=&quot;math-container&quot;&gt;$f(n)=g(n)+h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the expected cost of reaching goal state from node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. If we define Manhattan distance (&lt;span class=&quot;math-container&quot;&gt;$L_0$&lt;/span&gt; norm) for &lt;span class=&quot;math-container&quot;&gt;$h(\cdot)$&lt;/span&gt;, books (such as &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach (3rd Ed)&lt;/a&gt; by Stuart Russell and Peter Norvig) says A* is bound to find the solution (since it exists). However, I couldn't find how. Using, A*, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; will still select &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; since &lt;span class=&quot;math-container&quot;&gt;$f(A)=36+(h(A)=40)=76 &amp;lt; f(C)=70+(h(C)=30+50)=150$&lt;/span&gt;. You see, this means, when &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; expands back &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; will again select &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, and an infinite loop ensues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;" OwnerUserId="31550" LastEditorUserId="2444" LastEditDate="2019-11-25T02:43:52.963" LastActivityDate="2019-11-25T03:02:01.337" Title="A* and uniform-cost search are apparently incomplete" Tags="&lt;search&gt;&lt;a-star&gt;&lt;norvig-russell&gt;&lt;uniform-cost-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16739" PostTypeId="1" AcceptedAnswerId="16771" CreationDate="2019-11-24T11:00:18.683" Score="2" ViewCount="50" Body="&lt;p&gt;I've got a timeseries with sensor data (e.g. accelerometer and gyroscope). I now want to extract the activity out of it (e.g. walking, standing, driving, ...). I Followed &lt;a href=&quot;https://github.com/blue-yonder/tsfresh/blob/master/notebooks/human_activity_recognition_multi_class_example.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Jupyter Notebook&lt;/a&gt;. But there are some issues left.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why do they only pick 500 rows?&lt;/li&gt;&#xA;&lt;li&gt;What's the point of re-arranging the rows/columns?&lt;/li&gt;&#xA;&lt;li&gt;When they build their decicion tree learner with the train data, they build it upon extracted features. But how can we then use this tree for new sensor data? Should we extract the features of the new data and pass it as input for the tree? But new sensor data might not have as many features as the train data. Eg: &lt;code&gt;(ValueError: Number of features of the model must match the input. Model n_features is 321 and input n_features is 312)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31553" LastActivityDate="2019-11-25T22:23:39.127" Title="Feature extraction timeseries, model compatibility" Tags="&lt;classification&gt;&lt;decision-tree&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16740" PostTypeId="1" CreationDate="2019-11-24T11:01:08.023" Score="1" ViewCount="590" Body="&lt;p&gt;For search algorithms with heuristic functions, performance of heuristic functions are measured by effective branching factor &lt;span class=&quot;math-container&quot;&gt;${b^*}$&lt;/span&gt; which involves total nodes expanded &lt;span class=&quot;math-container&quot;&gt;${N}$&lt;/span&gt; and depth of the solution &lt;span class=&quot;math-container&quot;&gt;${d}$&lt;/span&gt;. I'm not able to find out how different values of &lt;span class=&quot;math-container&quot;&gt;${d}$&lt;/span&gt; affect the performance keeping same &lt;span class=&quot;math-container&quot;&gt;${N}$&lt;/span&gt;. Put another way, why not use just the &lt;span class=&quot;math-container&quot;&gt;${N}$&lt;/span&gt; as the performance measure instead of &lt;span class=&quot;math-container&quot;&gt;${b^*}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="31550" LastActivityDate="2019-11-24T17:28:18.690" Title="Why is the effective branching factor used for measuring performance of a heuristic function?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;performance&gt;&lt;branching-factors&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16741" PostTypeId="1" CreationDate="2019-11-24T13:38:32.187" Score="26" ViewCount="14246" Body="&lt;p&gt;I am a programmer but not in the field of AI. A question constantly confuses me is that how can an AI be trained if we human beings are not telling it its calculation is correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, news usually said something like &quot;company A has a large human face database so that it can train its facial recognition program more efficiently&quot;. What the piece of news doesn't mention is whether a human engineer needs to tell the AI program each of the program's recognition result is accurate or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any engineers who are constantly telling an AI what it produced it correct or wrong? If no, how can an AI determine if the result it produces is correct or wrong?&lt;/p&gt;&#xA;" OwnerUserId="31556" LastEditorUserId="2444" LastEditDate="2019-11-24T15:29:32.410" LastActivityDate="2019-12-01T04:34:27.037" Title="How can an AI train itself if no one is telling it if its answer is correct or wrong?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;" AnswerCount="7" CommentCount="3" FavoriteCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="16742" PostTypeId="2" ParentId="16740" CreationDate="2019-11-24T15:04:53.853" Score="1" Body="&lt;p&gt;As you found &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of nodes that are expanded. The cost of expansion of each node is equal to the number of children of that node. Hence, we use &lt;span class=&quot;math-container&quot;&gt;$b^*$&lt;/span&gt; for each node. In other words, the total number of nodes that are involved in the expansion process is &lt;span class=&quot;math-container&quot;&gt;$N \times b^*$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-11-24T17:28:18.690" LastActivityDate="2019-11-24T17:28:18.690" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16743" PostTypeId="2" ParentId="16741" CreationDate="2019-11-24T15:24:38.540" Score="31" Body="&lt;p&gt;By &quot;company A has a large human face database so that it can train its facial recognition program more efficiently&quot; the article probably means that there is a training dataset &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;S = \{ (\mathbf{x}_1, y_1), \dots,(\mathbf{x}_N, y_N) \}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt; is an image of the face of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th human and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; (which is often called a &lt;em&gt;label&lt;/em&gt;, &lt;em&gt;class&lt;/em&gt; or &lt;em&gt;target&lt;/em&gt;) is e.g. the name of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th human. So, the programmer provides a &lt;strong&gt;supervisory signal&lt;/strong&gt; (the label) for the AI to learn. The programmer also specifies the function that determines the error the AI program is making, based on the answer of the AI model and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way of learning is called &lt;strong&gt;supervised learning&lt;/strong&gt; (SL). However, there are other ways of training an AI. For example, there is &lt;strong&gt;unsupervised learning&lt;/strong&gt; (UL), where the AI needs to find patterns in the data by aggregating objects based on some similarity measure, which is specified by the programmer. There's also reinforcement learning (RL), where the programmer specifies only certain reinforcement signals, that is, the programmer tells the AI which moves or results are &quot;good&quot; and which ones are &quot;bad&quot; to achieve its goal, by giving to the AI, respectively, a positive or negative reward. You can also combine these three approaches and there are other variations. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any engineers who are constantly telling an AI what it produced it correct or wrong?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, in the case of SL. In the case of RL, the programmer also needs to provide the reinforcement signal, but it doesn't need to explicitly tell the AI which action it needs to take. In UL, the programmer needs to specify the way the AI needs to aggregate the objects, so, in this case, the programmer is also involved in the learning process.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-25T16:40:04.660" LastActivityDate="2019-11-25T16:40:04.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16744" PostTypeId="1" CreationDate="2019-11-24T15:28:25.960" Score="2" ViewCount="29" Body="&lt;p&gt;I'd like to learn about generalization theory for machine learning algorithms. I'm looking for books and other references (in case books aren't available) that provide a &lt;strong&gt;gentle introduction&lt;/strong&gt; to the field for a relative beginner like me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My background includes exposure to mostly undergrad mathematics and I have enough mathematical maturity to learn graduate-level topics as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, I'm looking to understand more about mathematical abstraction of ML concepts (e.g. &lt;em&gt;learning algorithm&lt;/em&gt;, &lt;em&gt;hypothesis space&lt;/em&gt;, &lt;em&gt;complexity of algorithm/hypothesis&lt;/em&gt; etc.), the purpose of an ML algorithm as an expected risk minimization exercise, techniques used to get bounds on generalization and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be even more specific, I'm looking to familiarize myself with concepts, theory and techniques so that I can understand papers (at least on a basic level) like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1710.05468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Deep Learning (2019)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1905.11427.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;nbro's answer to &lt;a href=&quot;https://ai.stackexchange.com/questions/16536/what-are-the-state-of-the-art-results-on-the-generalization-ability-of-deep-lear/&quot;&gt;this question&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and references therein&lt;/p&gt;&#xA;" OwnerUserId="27548" LastActivityDate="2019-11-24T15:28:25.960" Title="References on generalization theory and mathematical abstraction of ML concepts" Tags="&lt;research&gt;&lt;reference-request&gt;&lt;generalization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16745" PostTypeId="2" ParentId="16741" CreationDate="2019-11-24T15:32:33.037" Score="4" Body="&lt;p&gt;Taking your example of the faces data, keep in mind that when the model is run on a new unseen image the model can only return the already seen identity which emerges as the closest match. The result may be incorrect. The chances of mis-identification are much lower as the number of features incorporated increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input of the engineers lies at the level of the training data. Say we have a new photo of an individual that needs to be included in the model. The engineering task is now to morph that image to simulate different environments, angles of view, atmospheric conditions, lighting and so on to provide a large number of data input cases all of which will be &quot;true&quot; since the underlying features are all unchanged since the images are based on the same individual. Then the model is recalculated using the additional data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind too that adding a new set of data to an existing training set has the advantage that the parameters of the model are largely in the right ballpark already, and adding the new faces will make only small changes. Cross validation will show whether the addition has improved or spoiled the model.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2019-11-24T15:32:33.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16746" PostTypeId="1" AcceptedAnswerId="16794" CreationDate="2019-11-24T15:33:56.943" Score="1" ViewCount="323" Body="&lt;p&gt;What's the distinction between a learning algorithm &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and a hypothesis &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for a few concrete examples, if possible.&lt;/p&gt;&#xA;&lt;p&gt;For example, would the decision tree and random forest be considered two different learning algorithms? Would a shallow neural network (&lt;strong&gt;that ends up learning a linear function&lt;/strong&gt;) and a linear regression model, &lt;strong&gt;both of which use gradient descent to learn parameters&lt;/strong&gt;, be considered different learning algorithms?&lt;/p&gt;&#xA;&lt;p&gt;Anyway, from what I understand, one way to vary the hypothesis &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; would be to change the parameter values, maybe even the hyper-parameter values of, say, a decision tree. Are there other ways of varying &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;? And how can we vary &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="27548" LastEditorUserId="2444" LastEditDate="2020-12-07T21:16:54.350" LastActivityDate="2020-12-07T21:33:50.663" Title="What is the difference between a learning algorithm and a hypothesis?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;computational-learning-theory&gt;&lt;learning-algorithms&gt;&lt;hypothesis&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16747" PostTypeId="2" ParentId="16707" CreationDate="2019-11-24T16:33:22.187" Score="1" Body="&lt;p&gt;This is called decomposition of multi-class classifier. Your proposed method is called &lt;strong&gt;one vs all&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One vs. all provides a way to leverage binary classification. Given a classification problem with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; possible solutions, a one-vs.-all solution consists of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; separate binary classifiers—one binary classifier for each possible outcome. During training, the model runs through a sequence of binary classifiers, training each to answer a separate classification question.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/one-vs-all&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/one-vs-all&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;source=web&amp;amp;rct=j&amp;amp;url=http://www.cs.princeton.edu/~schapire/talks/ecoc-icml10.pdf&amp;amp;ved=2ahUKEwiIzYT3n4PmAhUZxosBHRjzDTMQFjAAegQIARAB&amp;amp;usg=AOvVaw2Vi3i0iYQvJOnyf3QiefNI&amp;amp;cshid=1574612627217&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. The author of the article did experiments on SVM on 8 different benchmark problems. According to the results, this method is sometimes as good as others, but usually not the best. It is also never substantially better than any other method. The article also stated that the best method is usually problem dependent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this method will decrease inference speed a lot, and used substantial amount of GPU memory. According to the source, it does not improve performance a lot, so you best bet for getting a higher performance is probably to use a different model architecture, for example the FPN FRCN, which is stated in the YOLO v3 paper having the best performance, but not fast inference speed. YOLOv3 is designed to have a fast inference speed, to provide real time object detection system, so for performance you should probably use other model architecture instead. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="2444" LastEditDate="2019-11-24T17:04:11.280" LastActivityDate="2019-11-24T17:04:11.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16748" PostTypeId="2" ParentId="16672" CreationDate="2019-11-24T17:47:11.057" Score="3" Body="&lt;p&gt;If we seek proven working source code to plug into a GPLv2-licence compatible solution, we should at least consider autotrace. Its &lt;a href=&quot;https://sourceforge.net/projects/autotrace/files/AutoTrace/0.31.1/autotrace-0.31.1-1.dag.8.0.src.rpm/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; is open for review. It can be tested against the example images we have and, if it works fine, called by our GPLv2 software. We can even use the calling code in &lt;a href=&quot;https://gitlab.com/inkscape/inkscape/blob/master/src/trace/autotrace/inkscape-autotrace.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inkscape's plug-in image tracing implementation&lt;/a&gt; as a good starting point for design and implementation of our calling program, whether it be C, C++, Java, Python, or ECMA (JS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trace algorithm in Adobe Illustrator is comparable but is not open source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we seek theory, there are several academic publications discussing some of the theory, the last being most aligned with machine learning ideology. I would not dismiss earlier work simply because it doesn't connect with the current machine learning idioms. Investigating what is fully implemented and successfully used by many follows a wise old business proverb: The bird in the hand is worth two in the bush.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.159.5801&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Potrace: a polygon-based tracing algorithm&lt;/a&gt;, Peter Selinger, 2003&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.109.2128&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vector Representation of Binary Images Containing Halftone Dots&lt;/a&gt;, Kei Kawamura, Hiroshi Watanabe, Hideyoshi Tominaga, 2004&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://core.ac.uk/download/pdf/38053960.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Testing AutoTrace:&#xA;A Machine-learning Approach to Automated&#xA;Tongue Contour Data Extraction&lt;/a&gt;, Jae-Hyun Sung, Jeff Berry, Marissa Cooper, Gustave Hahn-Powell, and Diana Archangeli, 2013&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many of the online drawing programs collect data.  It would not be surprising if, behind the gracious give-away of online bandwidth, they are establishing a continuously improving data set for training a new breed of autotracers.  None have published AI designs admitting as much, but they would not be legally obligated to do so because a single input example is indeterminable from the autotrace service that could resulting from the training.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2019-11-24T23:05:09.467" LastActivityDate="2019-11-24T23:05:09.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16749" PostTypeId="2" ParentId="16746" CreationDate="2019-11-24T19:22:54.497" Score="0" Body="&lt;p&gt;A hypothesis is a statement that suggests an as yet unproven explanation of a relationship between two or more phenomena that you intend to test. An agronomist thinks that more nitrogen on canola will always increase the crop output &lt;span class=&quot;math-container&quot;&gt;$$Harvest = f(N)$$&lt;/span&gt;, or a meteorologist thinks he can show that the path of a hurricane over the ocean can be determined by knowledge of the sea temperature and the wind speed at an altitude of 1000 feet one minute before. &lt;span class=&quot;math-container&quot;&gt;$$D(t,0) = f(T(t-1,1000),S(t-1,1000)$$&lt;/span&gt;&#xA;Both hypotheses are pegs on which later steps are based; testing follows with a conclusion whether the hypothesis can be rejected or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing a hypothesis can be simply adding or subtracting arguments to the function or changing the nature of the relationship such as the acceleration of the wind as opposed to its velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;learning&quot; algorithm describes how the parameters of a numeric model are changed in accordance with the delta rule, that is what the learning rate is and whether momentum is to be applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Random Forest and Decision Tree are &quot;classification&quot; algorithms. They are clearly stepwise processes that proceed towards the goal of a model, but they start by specifying the shape that the model will take and place boundaries on what values the parameters may take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both learning and classification algorithms specify a priori what shape the model will take and by doing so limit its relevance to particular problems.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2019-11-24T19:22:54.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16750" PostTypeId="1" CreationDate="2019-11-24T19:28:51.680" Score="1" ViewCount="111" Body="&lt;p&gt;I have a neural network for MNIST classification which I am hard coding using TensorFlow 2.0. The neural network has an input layer consisting of 784 neurons (28 * 28), one hidden layer having &quot;hidden_neurons&quot; number of neurons and an output layer having 10 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The part of the code that I want to get checked is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Partial derivative of cost function wrt b2-&#xA;dJ_db2 = (1 / m) * tf.reshape(tf.math.reduce_sum((A2 - Y), axis = 0), shape = (1, 10))&#xA;&#xA;# Partial derivative of cost function wrt b1-&#xA;dJ_db1 = (1 / m) * tf.reshape(tf.math.reduce_sum(tf.transpose(tf.math.multiply(tf.matmul(W2, tf.transpose((A2 - Y))), relu_derivative(A1))), axis = 0), shape = (1, hidden_neurons))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The notation is as follows. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;b1&quot; - bias for hidden layer and has the shape (1, hidden_neurons&quot;) &lt;/li&gt;&#xA;&lt;li&gt;&quot;b2&quot; - bias for output layer having the shape (1, 10).&lt;/li&gt;&#xA;&lt;li&gt;&quot;A2&quot; - is the output of output layer and have the shape (m, c)&lt;/li&gt;&#xA;&lt;li&gt;&quot;Y&quot; - is one-hot encoded target and have the shape (m, c)&lt;/li&gt;&#xA;&lt;li&gt;'m' - is number of training examples&lt;/li&gt;&#xA;&lt;li&gt;'c' - is number of classes&lt;/li&gt;&#xA;&lt;li&gt;&quot;A1&quot; - is the output of hidden layer and has the shape (hidden_neurons, m)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have used multiclass cross-entropy cost function. Hidden layer uses ReLU activation function, while the output layer has softmax activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my two lines of codes for cost function wrt to &quot;b1&quot; and &quot;b2&quot; correct?&lt;/p&gt;&#xA;" OwnerUserId="31215" LastEditorUserId="2444" LastEditDate="2019-11-25T01:47:51.357" LastActivityDate="2019-11-25T01:47:51.357" Title="Is this TensorFlow implementation of partial derivative of the cost with respect to the bias correct?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;backpropagation&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16752" PostTypeId="2" ParentId="16738" CreationDate="2019-11-25T02:41:43.283" Score="3" Body="&lt;p&gt;&lt;strong&gt;You forgot to calculate and take into account the costs of the actual paths. You forgot to accumulate the cost of the edges for going forward and backward multiple times!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The evaluation function of uniform-cost search (UCS) is &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n)$&lt;/span&gt;, where &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; represents the cost of the path from the start node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&lt;/strong&gt;. The evaluation function of A* is &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is an admissible heuristic. &lt;a href=&quot;https://ai.stackexchange.com/a/9183/2444&quot;&gt;UCS is a special case of A*&lt;/a&gt;, with the admissible heuristic &lt;span class=&quot;math-container&quot;&gt;$h(n) = 0, \forall n$&lt;/span&gt;. The evaluation function is used to choose the next node to &lt;strong&gt;visit&lt;/strong&gt; from the &lt;a href=&quot;https://ai.stackexchange.com/q/5949/2444&quot;&gt;fringe&lt;/a&gt;, which is the set of nodes that can potentially be &lt;strong&gt;visited&lt;/strong&gt;. Whenever we &lt;strong&gt;visit&lt;/strong&gt; a node, we remove it from the fringe. To &lt;strong&gt;expand&lt;/strong&gt; a node &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; means to add the children of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to the fringe. Whenever you &lt;strong&gt;visit&lt;/strong&gt; a node, you will also &lt;strong&gt;expand&lt;/strong&gt; it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's apply UCS to your specific example. Initially, we check whether &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is a goal node or not. It is not, so we expand &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. We can add &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to a list of visited (or expanded) nodes (graph search) or not (tree search). Let's use the tree search, so we will not be keeping track of the expanded nodes, which means that we could expand a node more than once. &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; has two children, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, so we add them to the fringe, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{F} = \{A, C\}$&lt;/span&gt;. Should we now visit &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;? We will visit the one with the smallest value of the evaluation function, which is &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, given that &lt;span class=&quot;math-container&quot;&gt;$f(A)=g(A)=36 &amp;lt; f(C)=g(C)=70$&lt;/span&gt;, so we remove &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; from the fringe, which is now &lt;span class=&quot;math-container&quot;&gt;$\mathbb{F} = \{ C \}$&lt;/span&gt;. Is &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; a goal node? No, so we expand it, but it only has one child, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, so we add &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to the fringe, so &lt;span class=&quot;math-container&quot;&gt;$\mathbb{F} = \{ C, B \}$&lt;/span&gt;. The cost of the path to reach &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; by going first to &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and then to &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$f(B) = 36 + 36 = 72$&lt;/span&gt; (given that you go back and forward on the same edge, so you need to accumulate the cost of these trips!) and &lt;span class=&quot;math-container&quot;&gt;$f(C) = 70$&lt;/span&gt;, so we visit &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, so we remove it from the fringe, which is now &lt;span class=&quot;math-container&quot;&gt;$\mathbb{F} = \{ B \}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should be able to work out the remaining search (I haven't actually done it). I suggest you watch the video &lt;a href=&quot;https://www.youtube.com/watch?v=dRMvK76xQJI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uniform Cost Search&lt;/a&gt;, by John Levine, who shows a concrete example of how UCS and A* work.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-25T03:02:01.337" LastActivityDate="2019-11-25T03:02:01.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16754" PostTypeId="1" AcceptedAnswerId="16755" CreationDate="2019-11-25T04:32:03.730" Score="0" ViewCount="51" Body="&lt;p&gt;I have a paper about trading which has been implemented with RNN on Tensorflow. We have about 2 years of data from trading. Here are some samples :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Date, Open, High, Low, Last, Close, Total Trade Quantity, Turnover (Lacs)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2004-08-25 , 1198.7, 1198.7, 979.0, 985.0, 987.95, 17116372.0, 172587.61&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2004-08-26 , 992.0, 997.0, 975.3, 976.85, 979.0, 5055400.0, 49828.65&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to predict the the future of trading (for example, the latest 10 days ). So, how can I make sure that my model is working correctly. Do we have any &quot;accuracy&quot; or &quot;loss&quot; like what we have in Deep Learning?&lt;/p&gt;&#xA;" OwnerUserId="31143" LastEditorUserId="31143" LastEditDate="2019-11-25T10:19:11.323" LastActivityDate="2019-11-25T10:19:11.323" Title="Do we have anything like accuracy and loss in RNN models?" Tags="&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16755" PostTypeId="2" ParentId="16754" CreationDate="2019-11-25T05:22:28.043" Score="5" Body="&lt;p&gt;RNN's stand for Recurrent Neural Networks which is, in fact, Deep Learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has to be a loss since you're dealing with supervised learning and the typical loss metrics used are the same as you would see in feedforward networks (usually binary cross-entropy), the main difference being loss would be calculated between the true label at a particular time stamp &lt;span class=&quot;math-container&quot;&gt;$(t)$&lt;/span&gt; and the prediction made from the subset of the network until time-stamp &lt;span class=&quot;math-container&quot;&gt;$(t-1)$&lt;/span&gt;. This leads the loss to act on all timestamps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accuracy metrics also would be used in the same way such as Mean Square Error or L1. For more details you can go through &lt;a href=&quot;https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks&quot; rel=&quot;noreferrer&quot;&gt;this link&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this was helpful!&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-25T05:22:28.043" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16756" PostTypeId="1" AcceptedAnswerId="16766" CreationDate="2019-11-25T05:48:43.410" Score="5" ViewCount="73" Body="&lt;p&gt;I have been reading a lot of articles on adversarial machine learning and there are mentions of &quot;best practices for robust machine learning&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A specific example of this would be when there are references to &quot;loss of efficient robust estimation in high dimensions of data&quot; in articles related to adversarial machine learning. Also, IBM has a Github repository named &quot;IBM's Adversarial Robustness Toolbox&quot;. &#xA;Additionally, there is a field of statistics called 'robust statistics' but there is no clear explanation anywhere about its relation to adversarial machine learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would therefore be grateful if someone could explain what robustness is in the context of Adversarial Machine Learning.&lt;/p&gt;&#xA;" OwnerUserId="31240" LastActivityDate="2019-11-25T17:43:31.037" Title="What is the relationship between robustness and adversarial machine learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;ai-safety&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16757" PostTypeId="1" CreationDate="2019-11-25T06:18:13.243" Score="2" ViewCount="1357" Body="&lt;p&gt;I would like to know how do Kaldi and DeepSpeech speech recognition systems differ algorithmically? Which one would be more accurate for continuous speech in time?&lt;/p&gt;&#xA;" OwnerUserId="31573" LastEditorUserId="4709" LastEditDate="2019-11-25T14:06:42.660" LastActivityDate="2019-12-01T14:15:51.680" Title="What is the difference between Kaldi and DeepSpeech speech recognition systems in their approach?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;deep-neural-networks&gt;&lt;speech-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16758" PostTypeId="1" CreationDate="2019-11-25T07:55:47.423" Score="2" ViewCount="40" Body="&lt;p&gt;I want to build a Deep Reinforcement Learning Model for Asset allocation.&lt;/p&gt;&#xA;&lt;h1&gt;Background:&lt;/h1&gt;&#xA;&lt;p&gt;I have 7 stock indexes from different markets, and I want to build a policy to produce the action (likes whether to sell or buy index? which index? and how much?) by observing the market informations.&lt;/p&gt;&#xA;&lt;h1&gt;Question 1:&lt;/h1&gt;&#xA;&lt;p&gt;I have two idea for the output of my policy. One is to produce a vector &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; of length 8, Each element &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; represent the target ratio of the stocks we want to hold (7 stock indexes and 1 cash), so I need to set &lt;span class=&quot;math-container&quot;&gt;$w_i&amp;gt;0, $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sum_{i}^{8}w_i=1.$&lt;/span&gt; How to implement? I just let the Activation function in the last layer of neural network to be sigmoid and divide the sum in environment. Is this available? And it's not easy to deal with transaction process if buy and sell fee exist. And the training process slowly when I use the policy gradient.&lt;/p&gt;&#xA;&lt;p&gt;The two is also produce a vector &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; of length 8, For each element &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; represent sell percent for stock i when &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; is negative and buy percent of cash when &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; is positive. It can solve the problem I meet in idea one. But I will meet a new question is cash is finite. I need to decide order of buy, in other words, which stock to buy first and buy which one later.&lt;/p&gt;&#xA;&lt;h1&gt;Question 2:&lt;/h1&gt;&#xA;&lt;p&gt;Many papers tell me to produce Distributed parameters by policy then create the action by distribution (like: normal distribution). It makes that more difficult to control the action  satisfy the condition above.&lt;/p&gt;&#xA;&lt;p&gt;And the result whether be unstable if the action is produce by sample.&lt;/p&gt;&#xA;" OwnerUserId="31479" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-25T07:55:47.423" Title="How to set the multiple continuous actions with constraints" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16759" PostTypeId="2" ParentId="16741" CreationDate="2019-11-25T09:38:54.920" Score="0" Body="&lt;p&gt;The trick with unsupervised learning is that the AI doesn't learn that something is a face or not, it just sees unnamed patterns that the researchers need to then name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you feed it a dataset with one million pictures in order to train a facial recognition algorithm. After training, the AI will have found a few patterns in the pictures based on the parameters of each picture such as color, lighting, topography, etc. However, without labels (supervised learning) the AI doesn't know what exactly it found, so a researcher then needs to label those patterns. You don't need a label to tell that a picture of a face is mostly different than the picture of a building. You need a label to tell you that one is a &quot;face&quot;and the other is a &quot;building&quot;. &lt;/p&gt;&#xA;" OwnerUserId="31577" LastActivityDate="2019-11-25T09:38:54.920" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16760" PostTypeId="1" CreationDate="2019-11-25T09:38:58.813" Score="1" ViewCount="963" Body="&lt;p&gt;I am trying to make a face login application. face comparison algorithm is using Euclidean distance to calculate two different face images that are the same or not the same. can anyone help me with how the Euclidean distance algorithm is working?&lt;/p&gt;&#xA;" OwnerUserId="31576" LastActivityDate="2019-11-25T13:22:13.223" Title="How Euclidean distance algorithm calculate two different face images are match or not match in face recognition?" Tags="&lt;image-recognition&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2019-11-26T19:55:32.657" ContentLicense="CC BY-SA 4.0" />
  <row Id="16761" PostTypeId="2" ParentId="16757" CreationDate="2019-11-25T10:34:58.383" Score="1" Body="&lt;p&gt;Both of them using the end-to-end approach for speech recognition. However, because of the code complexity in DeepSpeech, you can't tune the model for your work. Kaldi could be configured in a different manner and you have access to the details of the models and indeed it is a modular tool. I think Kaldi could be a better tool academically and also commercially. But, Deepspeech is a BlackBox and could be a proper tool if your work is near to the work of DeepSpeech. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if you are working in a language that there is not enough data for learning, you have a dozen of tools such as grapheme-to-phoneme to establish your dataset to start the learning process. Also, you can start with other models of Kaldi to work on which do not depend on the big data to be learned. However, DeepSpeech requires many hours of samples that is not a common asset for everyone in every language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you can know more about the traditional technique of speech processing vs the contemporary technique (deep learning) in &lt;a href=&quot;https://medium.com/@jonathan_hui/speech-recognition-gmm-hmm-8bb5eff8b196&quot; rel=&quot;nofollow noreferrer&quot;&gt;this medium post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2019-12-01T14:15:51.680" LastActivityDate="2019-12-01T14:15:51.680" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16762" PostTypeId="2" ParentId="16760" CreationDate="2019-11-25T13:22:13.223" Score="4" Body="&lt;p&gt;Simply put, &lt;a href=&quot;https://en.wikipedia.org/wiki/Euclidean_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euclidean distance&lt;/a&gt; measures how far away two items are (see Neil Slater's comment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to apply this to a pattern recognition task, you will need to convert the items to compare (in your case images of faces) into feature vectors (ie lists of numerical values), and then you do a pairwise comparison to work out how distant two faces are. You will then need to set a threshold where you treat two images as being the same face (typically where the distance between the feature vectors is small).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Selecting the right features is obviously crucial here. I'm not an expert on face descriptions, but it would probably be something like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the distance between the eyes&lt;/li&gt;&#xA;&lt;li&gt;the distance between the bottom of the nose and the top of the upper lip&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once you have these measurements, store the values for each image in a vector and you can apply the Euclidean distance. Effectively, each image is represented as a point in the  &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional feature space, where each measurement is a dimension. To select good features make sure they are not correlated (ie they are independent of each other) and are the same scale (eg all are distances, so not eye colour)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The choice of Euclidean distance is fairly minor: there are other distance metrics which might work equally well or even better. As I mentioned, the key point is selecting appropriate feature values.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-11-25T13:22:13.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16764" PostTypeId="2" ParentId="16741" CreationDate="2019-11-25T17:01:53.830" Score="0" Body="&lt;p&gt;I can't remember the researcher's name, but he specializes in psychology in Great Britain and has done a lot of work with machine learning and artificial intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The project he was working on that I read about earlier this year was one where they tried to deduce how humans learn.  They came up with the theory that we learn by making guesses about plausible and possible outcomes and that creates our expectations about reality.  When we are wrong, depending on the degree, we are possibly surprised or shocked or not affected at all.  They are working on creating AI that does not need human intervention, but to make guesses about outcomes before it performs tasks, and then update those expectations as it experiences more varying outcomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extremely interesting stuff, and definitely closer to how sentient beings gain experience and grow as individuals.&lt;/p&gt;&#xA;" OwnerUserId="31593" LastActivityDate="2019-11-25T17:01:53.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16765" PostTypeId="2" ParentId="16728" CreationDate="2019-11-25T17:35:41.533" Score="2" Body="&lt;p&gt;Consider a dataset &lt;span class=&quot;math-container&quot;&gt;$S \in \mathbb{R}^{N \times (M + 1)}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; observations (or examples), where each observation &lt;span class=&quot;math-container&quot;&gt;$S_i \in \mathbb{R}^{M + 1}$&lt;/span&gt; is composed of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; elements, one value for each of the &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; features (or independent variables), &lt;span class=&quot;math-container&quot;&gt;$f_1, \dots f_M$&lt;/span&gt;, and the corresponding target value &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision tree algorithm (DTA)&lt;/a&gt;, such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/ID3_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID3 algorithm&lt;/a&gt;, constructs a tree, such that each internal node of this tree corresponds to one of the &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; features, each edge corresponds to one value (or range of values) that such a feature can take on and each leaf node corresponds to a target. There are different ways of building this tree, based on different metrics to choose the features for each internal node and based on whether the problem is classification or regression (so based on whether the targets are classes or numeric values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's assume that the features and the target are binary, so each &lt;span class=&quot;math-container&quot;&gt;$f_k$&lt;/span&gt; can take on only one of two possible values, &lt;span class=&quot;math-container&quot;&gt;$f_{k} \in \{0, 1\}, \forall k$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$t_i \in \{0, 1\}$&lt;/span&gt; (where the index &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; correspond to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th observation, while the index &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; correspond to the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th column or feature of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;). In this case, a DTA first chooses (based on some metric, for example, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Information_gain_in_decision_trees&quot; rel=&quot;nofollow noreferrer&quot;&gt;information gain&lt;/a&gt;) one of the &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; features, for example, &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt;, to associate it with the root node of the tree. Let's call this root node &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt; will have two branches, one for each of the binary values of &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt; were a ternary variable, then the node corresponding to &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt; would have three branches, and so on. The DTA recursively chooses one of the remaining features for each node of the child branches. The DTA does this until all features have already been selected. In that case, we will have reached a leaf node, which will correspond to one value of the target variable. When the DTA chooses a feature for a node, all observations of the dataset &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; that take on the first binary value of that feature will go in the branch corresponding to that value and all other observations will go in the other branch. So, in this way, the DTA splits the dataset based on the features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following diagram represents a final decision tree built by a DTA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tmrRW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tmrRW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that the first feature selected (for the root node) by the DTA is &quot;Is it male?&quot;, which is a binary variable. If &lt;em&gt;yes&lt;/em&gt;, then, on the left branch, we have another internal node, which corresponds to another feature and, at the same time, to all observations associated with a male. However, on the right branch, we have a leaf node, which corresponds to one value of the target, which, in this case, is a probability (or, equivalently, a numerical value in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;). The shape of the tree depends on the dataset and DTA algorithm. Therefore, different datasets and algorithms might result in different decision trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, yes, you can view a decision tree algorithm as a feature selection or, more precisely, feature &lt;em&gt;splitting&lt;/em&gt; algorithm. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-25T18:51:00.740" LastActivityDate="2019-11-25T18:51:00.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16766" PostTypeId="2" ParentId="16756" CreationDate="2019-11-25T17:43:31.037" Score="3" Body="&lt;p&gt;A robust ML model is one that captures patterns that generalize well in the face of the kinds of small changes that humans expect to see in the real world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robust model is one that generalizes well from a training set to a test or validation set, but the term also gets used to refer to models that generalize well to, e.g. changes in the lighting of a photograph, the rotation of objects, or the introduction of small amounts of random noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adversarial machine learning is the process of finding examples that break an otherwise reasonable looking model. A simple example of this is that if I give you  a dataset of cat and dog photos, in which cats are always wearing bright red bow ties, your model may learn to associate bow ties with cats. If I then give it a picture of a dog with a bow tie, your model may label it as a cat. Adversarial machine learning also often includes the ability to identify specific pieces of noise that can be added to inputs to confound a model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if a model is robust, it basically means that it is &lt;em&gt;difficult&lt;/em&gt; to find adversarial examples for the model. Usually this is because the model has learned some desirable correlations (e.g. cats have a different muzzle shape than dogs), rather than undesirable ones (cats have bow ties; pictures containing cats are 0.025% more blue than those containing dogs; dog pictures have humans in them more often; etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approaches like &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;GANs&lt;/a&gt; try to directly exploit this idea, by training the model on both true data and data designed by an adversary to resemble the true data. In this sense, GANs are an attempt to create a robust discriminator. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-25T17:43:31.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16768" PostTypeId="1" CreationDate="2019-11-25T19:46:55.470" Score="1" ViewCount="33" Body="&lt;p&gt;I read a paper which is about Deep Reinforcement Learning and it tries to use this method on stock data set. It has been showed that it reach the maximum return(profit). It has been implemented in Tensorflow.&#xA;My question is, how we can make sure that we achieve the maximum value? I mean, is there a parameter or value that can show us how well the RL did its job?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nithinsethu/Deep-Direct-Reinforcement-Learning-for-Financial-Signal-Representation-and-Trading&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is codes and reference&lt;/p&gt;&#xA;" OwnerUserId="31143" LastEditorUserId="31143" LastEditDate="2019-11-26T01:52:23.470" LastActivityDate="2019-11-26T01:52:23.470" Title="How can we make sure, how well the reinforcement learning works?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16769" PostTypeId="1" CreationDate="2019-11-25T21:17:41.873" Score="4" ViewCount="58" Body="&lt;p&gt;I have a big dataset (28354359 rows) that has some blood values as features (11 features) and the label or outcome variable that tells whether a patient has a virus caused by a Neoplasm or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with my dataset is that 2% of the patients that are in my dataset have the virus and 98% does not have the virus. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am mandatory to use the random forest algorithm. While my random forest model has a high accuracy scores 92%, the problem is that more than 90% of the patients that have the virus are predicted that they don’t have the virus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the opposite effect, I want that my random forest is likely to predict more often that a patient has the virus (even if the patient does not have the virus (ideally I don’t want this side effect , but rather this than the opposite)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind this is that performing an extra test (via an echo) could not harm the patient that has not the virus, but not testing a patient will have result terrible for the patient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does somebody have advice how I could tweak my random forest model for this task?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I my self experimented with the SMOTE transformation and other sampling techniques but maybe you guys have other suggestion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have tried to apply a cutoff function.&lt;/p&gt;&#xA;" OwnerUserId="30599" LastEditorUserId="30599" LastEditDate="2019-11-26T06:47:43.770" LastActivityDate="2019-11-28T16:28:52.430" Title="Oposite type of predictions for unbalanced dataset" Tags="&lt;machine-learning&gt;&lt;random-forest&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16770" PostTypeId="2" ParentId="16646" CreationDate="2019-11-25T21:28:14.430" Score="0" Body="&lt;p&gt;The answer in part seems to depend on what you mean by &quot;human intelligence&quot;. If you mean behavior that would usually be regarded as requiring intelligence were a human to produce it, then various types machines can be intelligent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such &quot;intelligent&quot; machines presumably include player pianos. Playing the piano and producing a melody is widely regarded as requiring human intelligence when humans do it. Player pianos produce the same sort of behavior, but without a human touching a key. Hence (so the argument goes) player pianos are intelligent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if &quot;intelligence&quot; includes having the inner process of understanding, say understanding the meanings of symbols of written language, then at least according to philosopher John Searle, purely symbol manipulating devices such as digital computers could never be intelligent.  This is because symbols in themselves don't contain or indicate their meanings, and all the computing machine gets and manipulates is symbols in themselves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there does seem to be a sense in which the question &quot;Is artificial intelligence really just human intelligence&quot; is true of computers. This is when the behavior of the machine is caused by human intelligence. A human writes a program that defines, mandates, the behavior of the machine (just like a human designs the mechanism and paper roll of a player piano). This design takes human intelligence. The machine has no intrinsic, or innate, intelligence. It's just an automaton mindlessly following the causal sequence created by the intelligent human designer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if computers are purely symbol-manipulating devices, and if Searle is right, AI is doomed, at lest as long as its development platform is the digital computer (and no other machine is available or seems on the horizon).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, are computers purely symbol-manipulating devices? If not, there may be a way they can acquire meanings, or knowledge, and, for instance, learn languages. If computers can receive (including from digital sensors) and manipulate more than just symbols, they may be able to acquire the inner structures and execute the inner processes needed for human-like understanding. That is, they might be able to acquire knowledge by way of sensing the environment (as humans do). A human might write the program that facilitates acquisition of such knowledge, but what the knowledge is about would be derived from the sensed environment not from a human mind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here we're talking about &quot;intelligence&quot; defined over inner processes and structures, not or not just external behavior. If you define human intelligence as external behavior, as the Turing test does and as AI researchers often do, then music boxes with pirouetting figurines, player pianos, and programmed computers all have human-like intelligence, and artificial intelligence as it exists today is really just the same sort of thing as human intelligence. &lt;/p&gt;&#xA;" OwnerUserId="17709" LastEditorUserId="17709" LastEditDate="2019-11-25T21:34:44.693" LastActivityDate="2019-11-25T21:34:44.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16771" PostTypeId="2" ParentId="16739" CreationDate="2019-11-25T22:23:39.127" Score="2" Body="&lt;p&gt;there's a lot to un-pack in this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why do they only pick 500 rows?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my guess: in order to keep the example running quickly. tsfresh usually takes a while to calculate its features.  note that when they evaluated their model, they took the last 500 samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What's the point of re-arranging the rows/columns?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;answer: the data frame format that tsfresh requires in order to calculate features is that format. it is a bit of a pain...especially when you need to keep track of an id-column for other data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;When they build their decicion tree learner with the train data, they build it upon extracted features.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;answer: yes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;But how can we then use this tree for new sensor data?&#xA;Should we extract the features of the new data and pass it as input for the tree?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;answer: yes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;But new sensor data might not have as many features as the train data. Eg: (ValueError: Number of features of the model must match the input. Model n_features is 321 and input n_features is 312)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;answer: yes it will. I don't know where your copy-pasted error message came from. When you generate a set of features using tsfresh, you can do it in a couple of different ways. you can generate all of them or you can generate a subset of them---they generated a subset...but then subsetted it once again(using their importance stuff...whether you use the importance methods or other methods to select features you will end up with a bunch of features you need to calculate). If you follow the procedure for generating features based on a pre-determined subset (relevant_features in their case), you will end up with the same # of features.  This needs to be stressed...&lt;strong&gt;don't generate tsfeatures that you don't need!&lt;/strong&gt; as it will take foooorever.&lt;/p&gt;&#xA;" OwnerUserId="31608" LastActivityDate="2019-11-25T22:23:39.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16772" PostTypeId="1" CreationDate="2019-11-25T22:43:49.980" Score="4" ViewCount="68" Body="&lt;p&gt;I have just started playing with &lt;em&gt;Reinforcement learning&lt;/em&gt; and starting from the basics I'm trying to figure out how to solve &lt;a href=&quot;https://github.com/MartinThoma/banana-gym&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Banana Gym&lt;/strong&gt;&lt;/a&gt; with &lt;a href=&quot;https://github.com/NervanaSystems/coach&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;coach&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially &lt;code&gt;Banana-v0&lt;/code&gt; env represents a &lt;em&gt;Banana shop&lt;/em&gt; that buys a banana for \$1 on day 1 and has 3 days to sell it for anywhere between \$0 and \$2, where lower price means a higher chance to sell. Reward is the sell price less the buy price. If it doesn't sell on day 3 the banana is discarded and reward is -1 (banana buy price, no sale proceeds). That's pretty simple. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally the algorithm should learn to set a high price on day 1 and reducing it every day if it didn't sell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start I took the coach-bundled &lt;code&gt;CartPole_ClippedPPO.py&lt;/code&gt; and &lt;code&gt;CartPole_DQN.py&lt;/code&gt; preset files and modified them to run &lt;code&gt;Banana-v0&lt;/code&gt; gym. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trouble is that I don't see any learning progress regardless what I try, even after running like 50,000 episodes. In comparison the CartPole gym successfully trains in under 500 episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would have expected some improvement after 50k episodes for such a simple task like Banana. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yt2nk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yt2nk.png&quot; alt=&quot;Training progress&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it because the &lt;em&gt;Banana-v0&lt;/em&gt; rewards are not predictable? I.e. whether the banana sells or not is still determined by a random number (with success chance based on the price).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where should I take it from here? How to identify which Coach agent algorithm I should start with and try to tune it? &lt;/p&gt;&#xA;" OwnerUserId="31606" LastActivityDate="2019-11-26T11:11:20.183" Title="Unable to train Coach for Banana-v0 Gym environment" Tags="&lt;reinforcement-learning&gt;&lt;learning-algorithms&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16773" PostTypeId="2" ParentId="16741" CreationDate="2019-11-25T23:22:17.117" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;how can an AI be trained if we human beings are not telling it its calculation is correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What you are looking for is called &lt;strong&gt;self-supervised learning&lt;/strong&gt;. Yann LeCun, one of the originators behind modern neural network systems, has suggested that &lt;a href=&quot;https://www.simonsfoundation.org/event/could-machines-learn-like-humans/&quot; rel=&quot;nofollow noreferrer&quot;&gt;machines can reason usefully even in the absence of human-provided labels&lt;/a&gt; simply by learning auxiliary tasks, the answers for which are already encoded in the data samples. Self-supervision has &lt;a href=&quot;https://arxiv.org/pdf/1902.06162.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;already been successfully applied&lt;/a&gt; to a variety of tasks, showing improvement in multitask performance due to self-supervision. Unsupervised learning would in general be a subset of self-supervision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-supervision can be performed in a variety of ways. One of the most common is to use parts of the data as input and other parts as labels, and using the &quot;input&quot; subset of the data to predict the labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Supervised&lt;/strong&gt; learning looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.fit(various_data, human_labels)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The human_labels correspond to entries in various_data, which we expect the model to predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, &lt;strong&gt;self-supervised&lt;/strong&gt; learning can look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.fit(various_data[:,:500], various_data[:,500:]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Using Python array slice notation, some of the input data are used as training labels.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a machine could use half of the pixels in an image of a handwritten digit to try to predict the missing pixels. This is a form of self-supervision: Since the machine knows which pixels belong together in the same sample, it can &quot;automatically&quot; produce its own labeled data from the input itself, simply by using some inputs as outputs.&#xA;However, predicting pixels from other pixels is often not the desired task.&#xA;So instead, a neural network is often &lt;strong&gt;pretrained&lt;/strong&gt; using self-supervised or unsupervised learning techniques, and then subsequently trained on some amount of human-labeled data as a form of transfer learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the summary of the hypothetical news article promises is that self-supervision made the learning more &lt;strong&gt;efficient&lt;/strong&gt;, not that it outgrew the need for any kind of human intervention. This is exactly what we get from successful self-supervision in pretraining.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the best possible case, the machine learns to &quot;recognize&quot; each class of digit 0-9 but it still does not know how to ground its own internal labels to the human's labels. Then a human supplying the mapping between the machine's labels and the human-specified IDs would be the only step necessary to upgrade the self-supervised machine to one that is directly useful for digit recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will always be a need for humans to train a machine via direct supervision in order for the machine to learn the intended task. In order to solve a specific problem, a &lt;strong&gt;sufficient degree of supervision&lt;/strong&gt; is always required, and sufficient labels to reflect the intention must be provided.&lt;/p&gt;&#xA;" OwnerUserId="27060" LastEditorUserId="27060" LastEditDate="2019-12-01T04:34:27.037" LastActivityDate="2019-12-01T04:34:27.037" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16774" PostTypeId="2" ParentId="16741" CreationDate="2019-11-25T23:48:53.660" Score="3" Body="&lt;p&gt;I think you're probably looking at this the wrong way around. A conventional, old-fashioned AI doesn't make a guess, then require confirmation as to whether that guess was right or wrong. Instead, (in the simplest case) it undergoes a one-off computationally intensive &quot;training&quot;/&quot;learning&quot; phase, during which you feed it an enormous number of correct answers (which are labelled as correct) and an even more enormous number of incorrect answers (which are labelled as incorrect). Using whatever learning mechanism it has at its disposal, it then identifies some underlying structure in the &quot;corrects&quot; that doesn't exist in the &quot;incorrects&quot;. When, in the future, it encounters something new that seems to also exhibit this structure, then it will classify this as a &quot;correct&quot;. It might do rather well, or it might do terribly. Once the one-off training phase is done, it's stuck with whatever capability it has.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the company you mention is called Facebook and they have a feature that allows you to &quot;tag&quot; your friends in photos. No need to pay engineers to create the largest labelled image database in human history in order to train your AI.&lt;/p&gt;&#xA;" OwnerUserId="31610" LastActivityDate="2019-11-25T23:48:53.660" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16776" PostTypeId="1" CreationDate="2019-11-26T01:22:10.070" Score="1" ViewCount="30" Body="&lt;p&gt;I'm new to Deep Learning with &lt;code&gt;Keras&lt;/code&gt;. With some tutorials online for cat vs non-cat classification, I was able to compile this simple architecture for my own classification problem. However, my target application is &lt;strong&gt;fire detection&lt;/strong&gt; which essentially might have semantic differences with cats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training, I realized this model is accurate when the fire scene is simple and visible, but if many objects inside or fire is a bit smaller, it fails to detect. So I thought maybe I can change the architecture by increasing the complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First thing came into my mind was increasing the first layer filters from &lt;code&gt;32&lt;/code&gt; to &lt;code&gt;64&lt;/code&gt; by changing to &lt;code&gt;model.add(Conv2D(64, kernel_size = (3, 3), activation='relu', input_shape=(IMAGE_SIZE, IMAGE_SIZE, 1)))&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it going to help? What are other best practices to change the architecture? How about increasing the number of kernels to &lt;code&gt;kernel_size = (5, 5)&lt;/code&gt; or adding one more layer or even changing the images from grayscale to colored?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my original training code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras.models import Sequential, load_model&#xA;from keras.layers import Dense, Dropout, Flatten&#xA;from keras.layers import Conv2D, MaxPooling2D&#xA;from keras.layers.normalization import BatchNormalization&#xA;from PIL import Image&#xA;from random import shuffle, choice&#xA;import numpy as np&#xA;import os&#xA;&#xA;IMAGE_SIZE = 256&#xA;IMAGE_DIRECTORY = './data/test_set'&#xA;&#xA;def label_img(name):&#xA;  if name == 'cats': return np.array([1, 0])&#xA;  elif name == 'notcats' : return np.array([0, 1])&#xA;&#xA;def load_data():&#xA;  print(&quot;Loading images...&quot;)&#xA;  train_data = []&#xA;  directories = next(os.walk(IMAGE_DIRECTORY))[1]&#xA;&#xA;  for dirname in directories:&#xA;    print(&quot;Loading {0}&quot;.format(dirname))&#xA;    file_names = next(os.walk(os.path.join(IMAGE_DIRECTORY, dirname)))[2]&#xA;&#xA;    for i in range(200):&#xA;      image_name = choice(file_names)&#xA;      image_path = os.path.join(IMAGE_DIRECTORY, dirname, image_name)&#xA;      label = label_img(dirname)&#xA;      if &quot;DS_Store&quot; not in image_path:&#xA;        img = Image.open(image_path)&#xA;        img = img.convert('L')&#xA;        img = img.resize((IMAGE_SIZE, IMAGE_SIZE), Image.ANTIALIAS)&#xA;        train_data.append([np.array(img), label])&#xA;&#xA;  return train_data&#xA;&#xA;def create_model():&#xA;  model = Sequential()&#xA;  model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', &#xA;                   input_shape=(IMAGE_SIZE, IMAGE_SIZE, 1)))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Dropout(0.2))&#xA;  model.add(Flatten())&#xA;  model.add(Dense(256, activation='relu'))&#xA;  model.add(Dropout(0.2))&#xA;  model.add(Dense(64, activation='relu'))&#xA;  model.add(Dense(2, activation = 'softmax'))&#xA;&#xA;  return model&#xA;&#xA;training_data = load_data()&#xA;training_images = np.array([i[0] for i in training_data]).reshape(-1, IMAGE_SIZE, IMAGE_SIZE, 1)&#xA;training_labels = np.array([i[1] for i in training_data])&#xA;&#xA;print('creating model')&#xA;model = create_model()&#xA;model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;print('training model')&#xA;model.fit(training_images, training_labels, batch_size=50, epochs=10, verbose=1)&#xA;model.save(&quot;model.h5&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2019-11-26T16:50:20.303" LastActivityDate="2019-11-26T16:50:20.303" Title="How to change the architecture of my simple sequential model" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;training&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16777" PostTypeId="2" ParentId="16741" CreationDate="2019-11-26T05:34:11.210" Score="1" Body="&lt;p&gt;What you are missing is what the news story does't mention and gloss over. When a news article says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;company A has a large human face database so that it can train its facial recognition program more efficiently&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What it really means is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;company A has a large database of human faces along with additional information such as the identity of the person the face belong to that was created by other humans so that they can use this data set to train its facial recognition program&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How training works is basically as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You have a large database of correct (or almost all correct, ideally it should be correct) information that you want to relate one to the other. For example images of faces along with who that face belongs to.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You split this large database into several sets.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You use one set to train the AI.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After looping through the training set you use one or more of the other sets to test the AI and check if the training works.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you've done this before compare the performance of the current AI to previous AI. Else go to 6.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tweak some parameters of the AI to try to improve performance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Go to 2 until you are satisfied with the performance of the AI.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All the steps above are normally automated by scripts. The key here is that the original database has both the question you want to ask the AI (face) and the answer you want the AI to learn (person).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, humans are involved in training the AI but the involvement happens earlier at the database gathering stage.&lt;/p&gt;&#xA;" OwnerUserId="31616" LastActivityDate="2019-11-26T05:34:11.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16779" PostTypeId="1" CreationDate="2019-11-26T07:05:09.857" Score="3" ViewCount="87" Body="&lt;p&gt;The specific problem I have is learning the relation &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt;. I have an array of 0 through 19 (input values) and a target array of 0, 1, 4, 9, 16, 25, 36 and so on all the way up to &lt;span class=&quot;math-container&quot;&gt;$19^2$&lt;/span&gt;=361.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following LSTM architecture:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 input node&#xA;1 output node&#xA;32 hidden units&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, interestingly, I accidentally trained my network wrong, in that I forgot to reverse the expected output list when training. So I trained the network for:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$0 \rightarrow 361 \\ 1 \rightarrow 324 \\ 2 \rightarrow 289 \\ 3 \rightarrow 256 \\ ... \\ 17 \rightarrow 4 \\ 18 \rightarrow 1 \\ 19 \rightarrow 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with a learning rate of 1 and halving it every 400 epochs, after 3000 epochs my error (which started somwhere in the millions) was 0.2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I went to correct this mistake, my error will hardly ever go beneath 100,000. Testing the network shows it does well in the low inputs, but once it starts to get to &lt;span class=&quot;math-container&quot;&gt;$16^2$&lt;/span&gt; onwards, it really struggles to increase the output values past ~250.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was just wondering if anyone has an explanation for this, as to why the LSTM struggles to learn to increase exponentially but seems to be able to learn to decrease exponentially just fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT WITH CODE:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = np.array([i for i in range(20)])&#xA;b = np.array([i**2 for i in range(20)])&#xA;np.random.seed(5)&#xA;ls = LSTM(1, 1, 32, learning_rate=1, regression=1)&#xA;# Input size = 1, output size = 1, hidden units = 32&#xA;if 1:&#xA;    for k in range(3000):&#xA;        total = 0&#xA;        for i in range(20):&#xA;            ls * a[i]&#xA;        for i in range(20):&#xA;            total += ls / b[i]&#xA;        if k % 400 == 0:&#xA;            ls.learning_rate *= 0.5&#xA;        print(k, &quot;:&quot;, total)&#xA;        ls.update_params()&#xA;for i in a:&#xA;    print(i, ls*i)&#xA;#ls.save_state('1.trainstate')&#xA;for i in range(20,30):&#xA;    print(ls*i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note this code uses a class I wrote using numpy. If wanted I'll include this code as well it's just that is ~300 lines and I don't expect anyone to go through all that&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="26726" LastEditDate="2020-01-18T02:37:10.767" LastActivityDate="2020-01-18T02:37:10.767" Title="Why do regression LSTMs learn high to low inputs significantly better than low to high?" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16780" PostTypeId="2" ParentId="16646" CreationDate="2019-11-26T07:53:22.880" Score="0" Body="&lt;p&gt;Right, AI is an extension of human creativity and the implied limitation is that it inherits bias through the specific choice of which features to consider. Given a set of features it is then far more able at calculating which combination of features best helps explain the relationship being considered than is the human mind. Humans are too distracted to think to the depth that AI and machine learning can. But that extreme focus is not intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the issues that prevents the human mind from thinking at comparable depth is the need to massage the set of features that might apply; we are constantly reviewing features, adding in new and eliminating those that do not contribute. Creativity is openness to admitting other seemingly unrelated features and hoping for emergence, and managing to persist in being creative when emergence is delayed.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2019-11-26T07:53:22.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16781" PostTypeId="1" CreationDate="2019-11-26T08:05:25.553" Score="1" ViewCount="45" Body="&lt;p&gt;I'm comparing the results of an Newton optimizer for a modified version of SVM ( a generalized quadratic loss, similar to the one stated in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.frontiersinai.com/ecai/ecai2004/ecai04/pdf/p0628.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A generalized quadratic loss for SVM&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;) with classic SVM^light for regression. The problem is that it's able to overfit the data (UCI Yacht data-set) but I can't reach the generalization results of SVM^light. I've tried several hyper-parameters grids. I'm solving the primal problem. I'll send you my code if you need it. Any suggestion?&lt;/p&gt;&#xA;" OwnerUserId="31620" LastEditorUserId="32140" LastEditDate="2019-12-18T20:21:56.147" LastActivityDate="2019-12-18T20:21:56.147" Title="A generalized quadratic loss and Newton iteration for Support Vector Regression, why doesn't it generalize well?" Tags="&lt;loss-functions&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16782" PostTypeId="2" ParentId="12734" CreationDate="2019-11-26T10:02:43.207" Score="1" Body="&lt;p&gt;You should only look for the cross-validation score. If this set is large enough, it will give you an accurate prediction of how your model will act for unseen data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your case is exceptional. The fitted model which is obviously overfitted actually performs better on the cross-validation set. This means in turn that your overfitted model will perform better with unseen data.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-11-26T10:02:43.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16783" PostTypeId="2" ParentId="2158" CreationDate="2019-11-26T10:22:00.210" Score="2" Body="&lt;p&gt;##Why Some Investors and Researchers Prefer Radar Over Radar, and Thereof Recent Developments in Radar&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Direct Answer to Your Question / What This Answer is About&lt;/h3&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; ... Why does Google use radar? Doesn't LIDAR do everything radar can do? In particular, are there technical advantages with radar regarding object detection and tracking? ... &amp;quot; ~ Crashalot (Stack Exchange user, Opening Oposter)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To begin, some researchers and investors hold reservations about LiDAR.&lt;/p&gt;&#xA;&lt;p&gt;Lidar is criticised for being expensive and unwieldy.&lt;/p&gt;&#xA;&lt;p&gt;Advances, such as capturing images from a bird's eye view, and other such improvements, have made radar near-accurate compared to LiDar, while having a cheaper cost. (This will be explored in the technical section, not here.)&lt;/p&gt;&#xA;&lt;p&gt;This is not conclusive as autonomous cars, radar, and/or LiDAR are being heavily researched.&lt;/p&gt;&#xA;&lt;p&gt;Another path to take is a hybrid system between radar and LiDAR. (This will be explored in the technical section, not here.)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Layperson Explanation&lt;/h3&gt;&#xA;&lt;p&gt;Start here: &lt;a href=&quot;https://www.national.co.uk/tech-powers-google-car/&quot; rel=&quot;nofollow noreferrer&quot;&gt;10 Astonishing Technologies That Power Google’s Self-driving Cars&lt;/a&gt; (It has  slides, so I can't extract the information as easily.)&lt;/p&gt;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&lt;p&gt;Eon Musk has very strong opinions against LiDAR. Naturally, he has influence. A while back, radar was unable to detect certain phenomenon, such as in servere certain meteorological disturbances, such as a snow-storm.&lt;/p&gt;&#xA;&lt;p&gt;This has changed with advances. More efficient systems and pierce the storms. New antenna and radar fabrication technologies are here. Technologies have enabled an such as: higher (millimeter wave) frequencies providing greater resolution with smaller phased-array antennas are now available.&lt;/p&gt;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;While not as anti-LiDAR as Musk, it appears researchers at Cornell University agree with his LiDAR-less approach. Using two inexpensive cameras on either side of a vehicle’s windshield, Cornell researchers have discovered they can detect objects with nearly LiDAR’s accuracy and at a fraction of the cost.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The researchers found that analyzing the captured images from a bird’s-eye view, rather than the more traditional frontal view, more than tripled their accuracy, making stereo camera a viable and low-cost alternative to LiDAR.&amp;quot; – Crowe, Steve. &amp;quot;Researchers Back Tesla's Non-Lidar Approach To Self-Driving Cars - The Robot Report&amp;quot;. &lt;em&gt;The Robot Report&lt;/em&gt;, 2019, &amp;lt; &lt;a href=&quot;https://www.therobotreport.com/researchers-back-teslas-non-lidar-approach-to-self-driving-cars/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.therobotreport.com/researchers-back-teslas-non-lidar-approach-to-self-driving-cars/&lt;/a&gt; &amp;gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;So far in the self-driving realm, automakers and technology companies have been enamored with other sensors for this purpose. Automated cars are currently using cameras and laser sensors known as lidar.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;By comparison, radar, which has been on production vehicles for two decades, has been a staple of driver-assist systems for obstacle detection, and until now, not viewed as a tool for localization. Perhaps it's even underappreciated. Venture capital has flowed into lidar and camera-based solutions for automated vehicles; radar has been viewed as a commodity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;'It's unfortunate that's the perception, but it's probably as it should be,' says John Xin, CEO of Lunewave, a startup developing radar-sensing systems. &amp;quot;Over the last 20 years, there's not been a whole lot of game-changing hardware technology coming out of radar sensors.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;That's changing. Whether it's global suppliers such as Bosch, or startups such as Lunewave and WaveSense, a recent spinoff from MIT's Lincoln Laboratory, there's fresh innovation being wrung from radar, a technology that first found widespread use during World War II, and was first deployed on production automobiles by supplier Delphi in 1999.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;These three companies are rethinking the role of radar in mobility. Here's a rundown of the technology advances underway.&amp;quot; – Crowe, Steve. &amp;quot;Researchers Back Tesla's Non-Lidar Approach To Self-Driving Cars - The Robot Report&amp;quot;. The Robot Report, 2019, &amp;lt; &lt;a href=&quot;https://www.therobotreport.com/researchers-back-teslas-non-lidar-approach-to-self-driving-cars/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.therobotreport.com/researchers-back-teslas-non-lidar-approach-to-self-driving-cars/&lt;/a&gt; &amp;gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Business Applications&lt;/h3&gt;&#xA;&lt;p&gt;Is LiDAR, radar, or camera Better for your business: &lt;a href=&quot;https://leddartech.com/lidar-radar-camera-demystifying-adas-ad-technology-mix/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demystifying the ADAS / AD Technology Mix&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://levelfivesupplies.com/100-real-world-applications-of-lidar-technology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://levelfivesupplies.com/100-real-world-applications-of-lidar-technology/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://dronelife.com/2019/02/22/50-ways-to-love-your-lidar-how-lidar-is-used-in-commercial-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dronelife.com/2019/02/22/50-ways-to-love-your-lidar-how-lidar-is-used-in-commercial-applications/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Technical Mirror&lt;/h3&gt;&#xA;&lt;p&gt;Wikipedia is probably the best place to start:--&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Radar&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Radar&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Lidar&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Lidar&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Self-driving_car&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Self-driving_car&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Vehicular_automation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Vehicular_automation&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This paper is one example of advancement in radar:--&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;A procedure for radar range calculation is described, reflecting current knowledge of the effects of external natural noise sources, atmospheric-absorption losses, and the refractive effect of the normal atmosphere. The range equation is presented in terms of explicitly defined and readily evaluated quantities. Curves and equations are given for evaluating the quantities that are not ordinarily known by direct measurement. Some conventions are proposed for use in general radar range calculation, including an antenna-noise-temperature curve, minimum-detectable signal-to-noise ratio (``visibility factor'') curves, and a formula for the reflection coefficient of a rough sea. A noise-temperature table and a work-sheet for range calculation are included in the Appendix.&amp;quot; – Blake, Lamont V. &amp;quot;Recent advancements in basic radar range calculation technique.&amp;quot; &lt;em&gt;IRE Transactions on Military Electronics&lt;/em&gt; 2 (1961): 154-164.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There is a technical paper here that compares/constrats Radar and LiDAR:--&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://www.sae.org/publications/technical-papers/content/2000-01-0345/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sae.org/publications/technical-papers/content/2000-01-0345/&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&lt;p&gt;One path to take is synergy/a hybrid system between both radar and LiDAR:--&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; .... Currently, aerial light detecting and ranging (LIDAR) systems are therefore preferred for the detection and ranging of objects submerged in the sea. LIDAR provides for large area coverage at high speed, but it lacks coherent detection capability, a shortcoming that severely limits system sensitivity and underwater target contrast. In response to this problem, this paper details the merging of RADAR and LIDAR technologies in the constitution of a hybrid LIDAR-RADAR detection scheme. ... &amp;quot; – Mullen, Linda J., et al. &amp;quot;Application of RADAR technology to aerial LIDAR systems for enhancement of shallow underwater target detection.&amp;quot; &lt;em&gt;IEEE Transactions on microwave theory and techniques&lt;/em&gt; 43.9 (1995): 2370-2377.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;(&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/414591&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/abstract/document/414591&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0034425706002781&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/abs/pii/S0034425706002781&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Where lives are at stake, back-ups/layers can be argued to be needed. Redundancies/back-ups (&amp;quot;redundancies&amp;quot; in a similar sense to a kidney, per the systems-sciences) can be extremely useful in the case where one system does not pick up shard of ice that makes a car fall off the path.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Sources, References, and Further Reading:--&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.geospatialworld.net/blogs/why-lidar-is-important-for-autonomous-vehicle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.geospatialworld.net/blogs/why-lidar-is-important-for-autonomous-vehicle/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.fierceelectronics.com/components/lidar-vs-radar&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.fierceelectronics.com/components/lidar-vs-radar&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://archer-soft.com/en/blog/lidar-vs-radar-comparison-which-system-better-automotive&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://archer-soft.com/en/blog/lidar-vs-radar-comparison-which-system-better-automotive&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.alphr.com/cars/7038/how-do-googles-driverless-cars-work&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.alphr.com/cars/7038/how-do-googles-driverless-cars-work&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.mercurynews.com/2019/06/17/elon-musk-calls-lidar-stupid-but-most-self-driving-cars-use-it-whos-right/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mercurynews.com/2019/06/17/elon-musk-calls-lidar-stupid-but-most-self-driving-cars-use-it-whos-right/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.wired.com/story/the-know-it-alls-how-do-self-driving-cars-see/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wired.com/story/the-know-it-alls-how-do-self-driving-cars-see/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://blogs.nvidia.com/blog/2019/04/15/how-does-a-self-driving-car-see/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blogs.nvidia.com/blog/2019/04/15/how-does-a-self-driving-car-see/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.therobotreport.com/how-4d-radar-could-impact-autonomous-vehicles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.therobotreport.com/how-4d-radar-could-impact-autonomous-vehicles/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://semcon.com/what-we-do/applied-autonomy/lidar-vs-radar-for-applied-autonomy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://semcon.com/what-we-do/applied-autonomy/lidar-vs-radar-for-applied-autonomy/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.nytimes.com/2019/09/26/business/autonomous-cars-sensors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nytimes.com/2019/09/26/business/autonomous-cars-sensors.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.asme.org/topics-resources/content/system-helps-selfdriving-cars-see-fog&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.asme.org/topics-resources/content/system-helps-selfdriving-cars-see-fog&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://www.therobotreport.com/researchers-back-teslas-non-lidar-approach-to-self-driving-cars/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.therobotreport.com/researchers-back-teslas-non-lidar-approach-to-self-driving-cars/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;(&lt;a href=&quot;https://semcon.com/what-we-do/applied-autonomy/lidar-vs-radar-for-applied-autonomy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://semcon.com/what-we-do/applied-autonomy/lidar-vs-radar-for-applied-autonomy/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25982" LastEditorUserId="25982" LastEditDate="2020-09-16T22:29:30.370" LastActivityDate="2020-09-16T22:29:30.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16787" PostTypeId="1" CreationDate="2019-11-26T11:17:39.210" Score="1" ViewCount="24" Body="&lt;p&gt;I'm working on a college project about traffic sign detection and I have to choose a paper to implement it, but I have basic knowledge of TensorFlow and I'm afraid of choosing a paper that I can't implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are examples of models for traffic sign detection that can be easily implemented?&lt;/p&gt;&#xA;" OwnerUserId="31628" LastEditorUserId="2444" LastEditDate="2019-11-26T16:22:40.070" LastActivityDate="2019-11-26T16:22:40.070" Title="What are examples of models for traffic sign detection that can be easily implemented?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16789" PostTypeId="2" ParentId="16769" CreationDate="2019-11-26T13:22:57.310" Score="2" Body="&lt;p&gt;A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can do is lower the percentage needed to classify it as class A (in your case, patient has the virus). This way, you can tell your random forest to predict class A if only 20% (or 10%, or 5%, ...) of the decision trees actually predicts class A. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what code you are using for the random forest algoritm, but in most you should be able to ask the % of certainty the random forest has of each class. &lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-11-26T13:22:57.310" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16792" PostTypeId="1" CreationDate="2019-11-26T15:03:05.740" Score="2" ViewCount="154" Body="&lt;p&gt;In the neural Turing machine (NTM), the content-based addressing and location-based addressing is used for memory addressing. Content-based addressing is similar to the attention-based model, weighting each row of memory which shows the importance of each row of memory (or each location of memory). Then for location-based addressing, by using shift kernel, the attention focus is moved left or right or remains unchanged. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is location-based addressing? Why was location-based addressing used? What is the concept of &quot;for location-based addressing, by using shift kernel, the attention focus is moved left or right or remains unchanged.&quot;? &#xA;What is the difference between content-based addressing and location-based addressing?&lt;/p&gt;&#xA;" OwnerUserId="31642" LastEditorUserId="2444" LastEditDate="2019-11-26T16:17:41.157" LastActivityDate="2019-11-26T16:17:41.157" Title="What is a location-based addressing in a neural Turing machine?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;neural-turing-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16793" PostTypeId="1" AcceptedAnswerId="16795" CreationDate="2019-11-26T15:06:22.160" Score="1" ViewCount="37" Body="&lt;p&gt;Is it possible to perform multiclass classification on data where the number of features is less than the number of target variables? Do you have any suggestions on how to address a problem where I have 2000 target variables?&lt;/p&gt;&#xA;" OwnerUserId="23866" LastEditorUserId="2444" LastEditDate="2019-11-26T15:52:37.020" LastActivityDate="2019-11-26T16:13:16.347" Title="Can I perform multiclass classification when the number of features is less than the number of targets?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16794" PostTypeId="2" ParentId="16746" CreationDate="2019-11-26T16:09:37.260" Score="2" Body="&lt;p&gt;In &lt;a href=&quot;http://eliassi.org/COLTSurveyArticle.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational learning theory&lt;/a&gt;, a learning algorithm (or learner) &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is an algorithm that chooses a hypothesis (which is a function) &lt;span class=&quot;math-container&quot;&gt;$h: \mathcal{X} \rightarrow \mathcal{Y}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is the input space and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; is the target space, from the hypothesis space &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, consider the task of image classification (e.g. &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt;). You can train, with gradient descent, a neural network to classify the images. In this case, gradient descent is the learner &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, the space of all possible neural networks that gradient descent considers is the hypothesis space &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; (so each combination of parameters of the neural network represents a specific hypothesis), &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is the space of images that you want to classify, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt; is the space of all possible classes and the final trained neural network is the hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; chosen by the learner &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For example, would the decision tree and random forest be considered two different learning algorithms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The decision tree and random forest are not learning algorithms. A &lt;strong&gt;specific&lt;/strong&gt; decision tree or random forest is a &lt;strong&gt;hypothesis&lt;/strong&gt; (i.e. function of the form as defined above).&lt;/p&gt;&#xA;&lt;p&gt;In the context of decision trees, the &lt;a href=&quot;https://en.wikipedia.org/wiki/ID3_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID3 algorithm&lt;/a&gt; (a &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;decision tree algorithm&lt;/a&gt; that can be used to construct the decision tree, i.e. the hypothesis), is an example of a &lt;strong&gt;learning algorithm&lt;/strong&gt; (aka &lt;em&gt;learner&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The space of all trees that the learner considers is the &lt;strong&gt;hypothesis space/class&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would a shallow neural network (that ends up learning a linear function) and a linear regression model, both of which use gradient descent to learn parameters, be considered different learning algorithms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The same can be said here. A specific neural network or linear regression model (i.e. a line) corresponds to a specific hypothesis. The set of all neural networks (or lines, in the case of linear regression) that you consider corresponds to the hypothesis class.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Anyway, from what I understand, one way to vary the hypothesis &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; would be to change the parameter values, maybe even the hyper-parameter values of, say, a decision tree.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you consider a neural network (or decision tree) model, with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta} = [\theta_i, \dots \theta_N]$&lt;/span&gt;, then a specific combination of these parameters corresponds to a specific hypothesis. If you change the values of these parameters, you also automatically change the hypothesis. If you change the hyperparameters (such as the number of neurons in a specific layer), however, you will be changing the hypothesis class, so the set of hypotheses that you consider.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are there other ways of varying &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Off the top of my head, only by changing the parameters, you change the hypothesis.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And how can we vary &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Let's consider gradient descent as the learning algorithm. In this case, to change the learner, you could change, for example, the learning rate.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T21:33:50.663" LastActivityDate="2020-12-07T21:33:50.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16795" PostTypeId="2" ParentId="16793" CreationDate="2019-11-26T16:13:16.347" Score="2" Body="&lt;p&gt;Of course. It only depends if those features are informative enough for the task at hand. In order to better understand the phenomenon, you can imagine 2 features displayed as points in a 2D plane. The number of possible target classes goes up to the number of clusters you can find in that plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the suggestion, I can only recommend the utilisation of a non-linear classifier.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2019-11-26T16:13:16.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16796" PostTypeId="5" CreationDate="2019-11-26T16:19:59.060" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-26T16:19:59.060" LastActivityDate="2019-11-26T16:19:59.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16797" PostTypeId="4" CreationDate="2019-11-26T16:19:59.060" Score="0" Body="For questions related to the neural Turing machine model, proposed in &quot;Neural Turing Machines&quot; (2014) by Alex Graves et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-26T16:19:59.060" LastActivityDate="2019-11-26T16:19:59.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16799" PostTypeId="1" CreationDate="2019-11-26T19:58:35.463" Score="1" ViewCount="44" Body="&lt;p&gt;Suppose you want to detect objects and also track objects and people. Is it better to train a model using a single fisheye camera or using multiple cameras that mimic the view of the fisheye camera? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, what can be done to remove objects that are washed out? Like for very small objects, how do you make them more visible? Would multicamera tracking be better in this scenario?&lt;/p&gt;&#xA;" OwnerUserId="28201" LastActivityDate="2019-12-27T07:01:20.143" Title="Multicamera Tracking vs Single Fisheye Camera" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16800" PostTypeId="1" CreationDate="2019-11-26T21:54:53.043" Score="1" ViewCount="37" Body="&lt;p&gt;Suppose I learn an optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|c)$&lt;/span&gt; for a contextual multi-armed bandit problem, where the context &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is a composite of multiple context variables &lt;span class=&quot;math-container&quot;&gt;$c = c_1, c_2, c_3$&lt;/span&gt;. For example, the context is specified by three Bernoulli variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any literature on how to determine the optimal policy in the event where I no longer have access to one of the context variables? &lt;/p&gt;&#xA;" OwnerUserId="31663" LastEditorUserId="2444" LastEditDate="2020-04-15T21:22:18.743" LastActivityDate="2020-04-15T21:22:18.743" Title="How do I determine the optimal policy in a bandit problem with missing contexts?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;multi-armed-bandit&gt;&lt;contextual-bandit&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16801" PostTypeId="1" CreationDate="2019-11-26T22:28:58.433" Score="8" ViewCount="230" Body="&lt;p&gt;I'm interested in self replicating artificial life (with many agents), so after reviewing the literature with the excellent &lt;a href=&quot;http://www.molecularassembler.com/KSRM.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kinematic Self-Replicating Machines&lt;/a&gt; I started looking for software implementations. I understand that the field is still in the early stages and mainly academic, but the status of artificial life software looks rather poor in 2019. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On wikipedia there is this &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_life#Notable_simulators&quot; rel=&quot;nofollow noreferrer&quot;&gt;list of software simulators&lt;/a&gt;.&#xA;Going trough the list only ApeSDK, Avida, DigiHive, DOSE, Polyword have been updated in 2019. I did not find a public repo for Biogenesis. ApeSDK, DigiHive and DOSE are single author programs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all I don't see a single very active project with a large community around (I would be happy to have missed something). And this is more surprising considering the big momentum of AI and the proliferation of many ready to use AI tools and libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the status artificial life software so under-developed, when this field looks promising both from a commercial (see manufacturing, mining or space exploration applications) and academic (ecology, biology, human brain and more) perspective? Did the field underdelivered on expectations in past years and got less funding? Did the field hit a theoretical or computational roadblock? &lt;/p&gt;&#xA;" OwnerUserId="16363" LastEditorUserId="16363" LastEditDate="2019-11-28T22:09:26.607" LastActivityDate="2020-07-30T16:01:28.723" Title="Why is the status of artificial life software so under-developed?" Tags="&lt;software-evaluation&gt;&lt;artificial-life&gt;&lt;self-replication&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16802" PostTypeId="2" ParentId="16714" CreationDate="2019-11-26T23:37:40.317" Score="1" Body="&lt;p&gt;The best way is probably to Google it with &quot;[org name] tensorflow github&quot; and look what you get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance I found:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/microsoft/MMdnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/NVIDIA/tensorflow-determinism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nvidia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorflow/ngraph-bridge&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16363" LastActivityDate="2019-11-26T23:37:40.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16803" PostTypeId="1" CreationDate="2019-11-27T00:22:15.423" Score="2" ViewCount="151" Body="&lt;p&gt;Loss is MSE; orange is validation loss, blue training loss. The task is NN regression (18 inputs, 2 outputs), one layer 300 hidden units.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Rs1sq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rs1sq.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tuning the lr, mom, l2 regularization parameters this is the best validation loss I can obtain. Can be considered overfitting? Is 1 a bad vl loss value for a regression task?&lt;/p&gt;&#xA;" OwnerUserId="31665" LastActivityDate="2019-11-30T00:05:25.293" Title="How do you interpret this learning curve?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;loss-functions&gt;&lt;learning-algorithms&gt;&lt;cross-validation&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16804" PostTypeId="2" ParentId="16803" CreationDate="2019-11-27T04:13:31.617" Score="3" Body="&lt;p&gt;Depends on what does 1 represent in your task. &#xA;If you are trying to predict household prices and 1 represents \$1, I think the average validation loss is good. If 1 represents \$10000 in this case, probably something is not right. But remember that there are 2 parts contributing to the overall loss. The mse loss and the l2 penalty loss. (Also remember that most optimizers already implement l2 penalty as weight decay. So you do not need to add it separately)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some suggestions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Check if your data has any outliers/ anomalies. Based on your task you should know as to what you can do with these data points. Also see if your dataset has high variance. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you are worried about over-fitting, think about your data again. Less data + more parameters often leads to over-fitting. If your dataset is too small, you need to think again.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to adjust the number of hidden units and observe the results.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try using cross validation.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Alternatively, try using different optimizers and see what happens (Try Adam). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28182" LastActivityDate="2019-11-27T04:13:31.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16805" PostTypeId="1" AcceptedAnswerId="20630" CreationDate="2019-11-27T06:44:01.577" Score="4" ViewCount="274" Body="&lt;p&gt;Currently, I'm using a Python library, &lt;a href=&quot;https://github.com/stellargraph/stellargraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;StellarGraph&lt;/a&gt;, to implement GCN. And I now have a situation where I have graphs with weighted edges. Unfortunately, StellarGraph doesn't support those graphs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for an open-source implementation for graph convolution networks for &lt;strong&gt;weighted graphs&lt;/strong&gt;. I've searched a lot, but mostly they assumed unweighted graphs. Is there an open-source implementation for GCNs for weighted graphs?&lt;/p&gt;&#xA;" OwnerUserId="31672" LastEditorUserId="2444" LastEditDate="2020-04-25T20:58:02.887" LastActivityDate="2020-05-21T09:33:56.123" Title="Is there an open-source implementation for graph convolution networks for weighted graphs?" Tags="&lt;python&gt;&lt;resource-request&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16806" PostTypeId="2" ParentId="16799" CreationDate="2019-11-27T06:44:27.707" Score="1" Body="&lt;p&gt;Fisheye camera is always worst. Both convolutional networks object detectors and feature-based object detectors rely on the &quot;isometry&quot; of planar image - lack of strong distortions. Multiple camera have added benefit of several independent sources of information - ensembling. If each camera processed by separate network that may help in verification of objects by voting. With very small objects no simple methods would help. Multiple camera may help a little if all camera output processed together (stacked) as single network input, but don't hold much hope - detecting few pixel objects is very difficult, if possible at all.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2019-11-27T06:44:27.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16807" PostTypeId="1" CreationDate="2019-11-27T07:39:38.217" Score="4" ViewCount="130" Body="&lt;p&gt;What is the difference between genetic algorithms and evolutionary game theory algorithms?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2019-11-27T17:58:07.473" LastActivityDate="2019-12-28T15:00:10.143" Title="What is the difference between genetic algorithms and evolutionary game theory algorithms?" Tags="&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;game-theory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16808" PostTypeId="2" ParentId="2078" CreationDate="2019-11-27T09:53:50.093" Score="1" Body="&lt;p&gt;My answer is with respect to game theory perspective, &lt;strong&gt;Replicator Dynamics&lt;/strong&gt; is one of core concept of evolutionary game theory algorithm which means rate of adaptation with respect to rate of change in population. Whenever there is change in the system replicator dynamics will help to adapt with the change with respect to utility function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://chart.googleapis.com/chart?cht=tx&amp;amp;chl=%5Cfrac%7B%5Cpartial%20x%5Cleft%20(%20t%20%5Cright%20)%7D%7B%5Cpartial%20t%7D%20%3D%20x%0A%0A%5Cleft%20(%20%20t%5Cright%20)%5Cbegin%7Bbmatrix%7D%0AU_%7BA%7D%5Cleft%20(%20t%20%5Cright%20)%20-%20%5Cdot%7BU%5Cleft%20(%20t%20%5Cright%20)%7D%0A%5Cend%7Bbmatrix%7D%0A%0A&quot; rel=&quot;nofollow noreferrer&quot;&gt;Replicator Dynamics Equation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Better Understanding go through this link: &lt;a href=&quot;https://www.mimuw.edu.pl/~miekisz/cime.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Algorithm Pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this will be helpful.&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="9863" LastEditDate="2019-12-20T07:38:28.260" LastActivityDate="2019-12-20T07:38:28.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16810" PostTypeId="1" CreationDate="2019-11-27T11:16:37.257" Score="2" ViewCount="47" Body="&lt;p&gt;I read &lt;a href=&quot;https://hmkcode.com/ai/backpropagation-step-by-step/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; about backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So using this backpropagation we are training the neural network repeatedly for one input set, say [2,4], until we reach 100% accuracy of getting 1 as output. And the neural network is adjusting its weight values accordingly. So once after the neural network is trained this way, suppose we are giving another input set, say [6,8], also then will the neural network update its weight values (overwriting previous values), right? This will result in losing the previous learning, right?&lt;/p&gt;&#xA;" OwnerUserId="31684" LastEditorUserId="4709" LastEditDate="2019-11-27T18:59:23.787" LastActivityDate="2019-11-28T14:02:16.067" Title="How can I train a neural network for another input set, without losing the learning of the previous input set?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;artificial-neuron&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16812" PostTypeId="1" CreationDate="2019-11-27T12:05:29.783" Score="2" ViewCount="100" Body="&lt;p&gt;I know that &lt;a href=&quot;https://www.youtube.com/watch?v=5Q14EjnOJZc&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning has been used to solve the inverted pendulum problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can supervised learning be used to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverted_pendulum&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverted pendulum&lt;/a&gt; problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, there could be an interface (e.g. a joystick) with the cart-pole system, which the human can use to balance the pole and, at the same time, collect a dataset for supervised learning. Has this been done before?&lt;/p&gt;&#xA;" OwnerUserId="31381" LastEditorUserId="2444" LastEditDate="2019-11-27T17:54:32.847" LastActivityDate="2021-02-15T22:03:25.333" Title="Can supervised learning be used to solve the inverted pendulum problem?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;robotics&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16813" PostTypeId="2" ParentId="16810" CreationDate="2019-11-27T12:19:18.750" Score="3" Body="&lt;p&gt;Yes, this is actually a limitation known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Catastrophic_interference&quot; rel=&quot;nofollow noreferrer&quot;&gt;catastrophic forgetting&lt;/a&gt;. &#xA;A proposed way to deal with this is elastic weight consolidation that &quot;remembers old tasks by selectively slowing down learning on the weights important for those tasks&quot;. See &lt;a href=&quot;https://arxiv.org/abs/1612.00796&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overcoming catastrophic forgetting in neural networks&lt;/a&gt; for details. Another approach is &lt;a href=&quot;https://arxiv.org/pdf/1606.09282.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning without forgetting&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the tasks are different, the approach you are talking about is called transfer learning. You might want to have a look at multi-task learning as well &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the tasks are the same, you could try creating a join of both datasets and training on that.&lt;/p&gt;&#xA;" OwnerUserId="27851" LastEditorUserId="27851" LastEditDate="2019-11-28T14:02:16.067" LastActivityDate="2019-11-28T14:02:16.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16814" PostTypeId="1" CreationDate="2019-11-27T12:38:50.817" Score="2" ViewCount="46" Body="&lt;p&gt;I am working on an Intent detection problem for a chatbot in Java.&#xA;So I need to convert words from String to a double[] format.&#xA;I tried using wordToVec(deeplearning4j), but it does not return a vector for words not present in the training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. My dataset for wordToVec.train() does not contain the word &quot;morning&quot;. So wordToVec.getWordVector(&quot;morning&quot;) returns a null value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no need to find the coorelation between two words(like in word2vec), but it should be able to give me some sort of vector representation for any word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some things I thought of-&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I could use a fixed length hash function and convert resulatant hash into vector.(Will Hash Collision be strong enough to be an issue in this case?)&lt;/li&gt;&#xA;&lt;li&gt;I could initialize for each word a vector of huge length as zero, and set its elements as the ASCII value-64.&#xA;e.g. Keeping Maximum vector length as 10, AND would be represented as-&#xA;[1,14,4,0,0,0,0,0,0,0], and normalize this.&#xA;Is there a better solution to this problem?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is the code I used to train the model-&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public static void trainModel() throws IOException&#xA;    {&#xA;        //These lines simply generate the dataset into a format readable by wordToVec&#xA;        utilities.GenRawSentences.genRaw();&#xA;&#xA;        dataLocalPath = &quot;./TrainingData/&quot;;&#xA;        String filePath = new File(dataLocalPath, &quot;raw_sentences.txt&quot;).getAbsolutePath();&#xA;        //Data Generation ends   &#xA;&#xA;        SentenceIterator iter = new BasicLineIterator(filePath);&#xA;        TokenizerFactory t = new DefaultTokenizerFactory();&#xA;        t.setTokenPreProcessor(new CommonPreprocessor());&#xA;&#xA;        VocabCache&amp;lt;VocabWord&amp;gt; cache = new AbstractCache&amp;lt;&amp;gt;();&#xA;        WeightLookupTable&amp;lt;VocabWord&amp;gt; table = new InMemoryLookupTable.Builder&amp;lt;VocabWord&amp;gt;()&#xA;                .vectorLength(64)&#xA;                .useAdaGrad(false)&#xA;                .cache(cache).build();&#xA;&#xA;        Word2Vec vec = new Word2Vec.Builder()&#xA;                .minWordFrequency(1)&#xA;                .iterations(5)&#xA;                .epochs(1)&#xA;                .layerSize(64)&#xA;                .seed(42)&#xA;                .windowSize(5)&#xA;                .iterate(iter)&#xA;                .tokenizerFactory(t)&#xA;                .lookupTable(table)&#xA;                .vocabCache(cache)&#xA;                .build();&#xA;&#xA;        vec.fit();&#xA;&#xA;        //Saves the model for use in other programs&#xA;        WordVectorSerializer.writeWord2VecModel(vec, &quot;./Models/WordToVecModel.txt&quot;);&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31685" LastActivityDate="2019-11-27T12:38:50.817" Title="How can I feed any word into a neural network?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;word-embedding&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16815" PostTypeId="1" CreationDate="2019-11-27T13:12:03.713" Score="4" ViewCount="92" Body="&lt;p&gt;Some people say that abstract thinking, &lt;a href=&quot;http://artificial-intuition.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;intuition&lt;/a&gt;, common sense, and understanding cause and effect are important to make AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How important is &lt;em&gt;learning to learn&lt;/em&gt; for the development of AGI?&lt;/p&gt;&#xA;" OwnerUserId="31686" LastEditorUserId="2444" LastEditDate="2019-11-27T17:46:25.643" LastActivityDate="2020-04-13T22:10:31.230" Title="How important is learning to learn for the development of AGI?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16816" PostTypeId="1" CreationDate="2019-11-27T15:25:52.317" Score="2" ViewCount="67" Body="&lt;p&gt;I'm currently trying to develop an RL that will teach itself to play the popular fighting game &quot;Tekken 7&quot;. I initially had the idea of teaching it to play generally- against actual opponents with various levels of difficulty- but the idea proved to be rather complex. I've liquefied the goal down to &quot;get a non-active standing opponent to 0 health as fast as possible&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some experience with premade OpenAI environments, and tried making my own environment for this specific purpouse, but this proved to be rather difficult as there was no user friendly documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a DQN that was coded along with the help of a YouTube tutorial&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from tensorflow.keras.layers import Dense, Activation&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras.optimizers import Adam&#xA;&#xA;&#xA;class ReplayBuffer(object):&#xA;    def __init__(self, max_size, input_shape, n_actions, discrete=False):&#xA;        self.mem_size = max_size#memory size dictated&#xA;        self.discrete = discrete#determines a number of discrete values that can be inputted&#xA;        self.state_memory = np.zeros((self.memsize, input_shape))&#xA;        self.new_state_memory = np.zeros((self.mem_size, input_shape))&#xA;        dtype = np.int8 if self.discrete else np.float32&#xA;        self.action_memory = np.zeros((self.mem_size, n_actions))&#xA;        self.reward_memory = np.zeros(self.mem_size)&#xA;        self.terminal_memory = np.zeros(self.mem_size, dtype = np.float32)&#xA;&#xA;&#xA;    def store_transition(self, state, action, reward, state, done):&#xA;        index = self.mem_cntr % self.mem_size&#xA;        self.state_memory[index] = state&#xA;        self.new_state_memory[index] = state_&#xA;        self.reward_memory[index] = reward&#xA;        self.terminal_memory[index] = 1 - int(done)&#xA;        if self.discrete:&#xA;            actions = np.zeros(self.action_memory.shape[1])&#xA;            self.action_memory[index] = actions&#xA;        else:&#xA;            self.action_memory[index] = action&#xA;        self.mem_cntr += 1&#xA;&#xA;&#xA;    def sample_buffer(self, batch_size):&#xA;        max_mem = min(self.mem_cntr, self.mem_size)&#xA;        batch = np.random.choice(max_mem, batch_size)&#xA;&#xA;        states = self,state_memory[batch]&#xA;        states_ = self.new_state_memory[batch]&#xA;        rewards = self.reward_memory[batch]&#xA;        actions = self.action_memory[batch]&#xA;        terminal = self.terminal_memory[batch]&#xA;&#xA;        return states, actions, rewards, states_, terminal&#xA;&#xA;&#xA;    def build_dqn(lr, n_actions, input_dims, fcl_dims, fc2_dims):&#xA;        model = Sequential([&#xA;                    Dense (fcl_dims, input_shape = (input_dims, )),&#xA;                    Activation('relu')&#xA;                    Dense(fc2_dims),&#xA;                    Activation('relu')&#xA;                    Dense(n_actions)])&#xA;&#xA;        model.comile(optimizer = Adam(lr = lr), loss = 'mse')&#xA;&#xA;        return model&#xA;&#xA;    class Agent(object):&#xA;        def __init__(self, alpha, gamma, n_actions, epsilon, batch_size,&#xA;                     input_dims, epsilon_dec=0.996, epsilon_end=0.01,&#xA;                     mem_size = 1000000, fname = 'dqn_model.h5'):&#xA;            self.action_space = [i for i in range(n_actions)]&#xA;            self.n_actions = n_actions&#xA;            self.gamma = gamma&#xA;            self.epsilon = epsilon&#xA;            self.epsilon_dec = epsilon_dec&#xA;            self.epsilon_min = eps_end&#xA;            self.batch_size = batch_size&#xA;            self.model_file = fname&#xA;&#xA;            self.memory = ReplayBuffer(mem_size, input_dims, n_actions,&#xA;                                       discrete = True)&#xA;            self.q_eval = build_dqn(alpha, n_actions, input_dims, 256, 256)&#xA;&#xA;        def remember(self, state, action, reward, new_state, done):&#xA;            self.memory.store_transition(state, action, reward, new_state, done)&#xA;&#xA;        def choose_action(self, state):&#xA;            state = state[np.newaxis, :]&#xA;            rand = np.random.radnom()&#xA;            if rand &amp;lt; self.epsilon:&#xA;                action = np.random.choice(self.action_space)&#xA;            else:&#xA;                actions = slef.q.eval.predict(state)&#xA;                action = np.argmax(actions)&#xA;&#xA;            return action&#xA;&#xA;        def learn(self):#temporal difference learning, delta between steps \&#xA;            #and learns from this&#xA;            #&#xA;            #using numpy.zero approach, only drawback \&#xA;            #is that batch size of memory must be full before learning&#xA;            if self.memory.mem_cntr &amp;lt; self.batch_size:&#xA;                return&#xA;            state, action, reward, new_state, done = \&#xA;                                    self.memory.sample_buffer(self.batch_size)&#xA;&#xA;&#xA;            action_values = np.arary(self.action_space, dtype = np.int8)&#xA;            action_indices = np.dot(action, action_values)&#xA;&#xA;            q_eval = self.q_eval.predict(state)&#xA;            q_next = self.q_eval.predict(new_state)&#xA;&#xA;            q_target = q.eval.copy()&#xA;&#xA;            batch_index = np.arrange(self.batch_size, dtype = np.int32)&#xA;&#xA;            q_target[batch_index, action_indices] = reward + \&#xA;                                    self.gamma*np.max(q_next, axis=1)*done&#xA;&#xA;            _ = self.q_eval.fit(state, q_target, verbose=0)&#xA;&#xA;            self.epsilon = self.epsilon*epsilon_dec if self.epsilon &amp;gt; \&#xA;                           self.epsilon_min else self.epsilon_min&#xA;&#xA;            def save_model(self):&#xA;                self.q_eval.save(self.model.file)&#xA;&#xA;            def load_model(self):&#xA;                self.q_eval = load.model(self.model_file)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31692" LastActivityDate="2019-11-27T15:25:52.317" Title="How would one develop an action space for a game that is proprietary?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16818" PostTypeId="1" CreationDate="2019-11-27T16:50:08.227" Score="3" ViewCount="109" Body="&lt;p&gt;Can a CNN (or other non-sequential deep learning models) outperform LSTM (or other sequential models) in time series data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this question is not very specific, but I experienced this when predicting daily store sales and I am curious as to why it can happen.&lt;/p&gt;&#xA;" OwnerUserId="31694" LastEditorUserId="2444" LastEditDate="2019-11-27T17:20:58.690" LastActivityDate="2019-11-28T09:20:40.883" Title="Can non-sequential deep learning models outperform sequential models in time series forecasting?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16820" PostTypeId="2" ParentId="16815" CreationDate="2019-11-27T19:05:08.913" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://bair.berkeley.edu/blog/2017/07/18/learning-to-learn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning to learn&lt;/em&gt;&lt;/a&gt; (also known as &lt;a href=&quot;https://arxiv.org/pdf/1606.04474.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;meta-learning&lt;/em&gt;&lt;/a&gt;) is &lt;strong&gt;very important&lt;/strong&gt; for the development of artificial general intelligence (AGI), given that one of the desirable and fundamental properties of an AGI is the &lt;em&gt;adaptability&lt;/em&gt; to different environments and the ability to &lt;a href=&quot;https://arxiv.org/abs/1802.07569&quot; rel=&quot;nofollow noreferrer&quot;&gt;continually learn&lt;/a&gt;, and meta-learning can be used to achieve that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meta-learning is thus related to multi-task, &lt;a href=&quot;https://arxiv.org/abs/1802.07569&quot; rel=&quot;nofollow noreferrer&quot;&gt;continual (or lifelong)&lt;/a&gt; and &lt;a href=&quot;https://www.cse.ust.hk/~qyang/Docs/2009/tkde_transfer_learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer&lt;/a&gt; learning. There are several approaches to meta-learning, such as &lt;a href=&quot;https://arxiv.org/abs/1703.03400&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAML&lt;/a&gt;, but there is usually a meta-learner and a learner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1604.00289.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Building Machines That Learn and Think Like People&lt;/a&gt; (2016), Marcin Andrychowicz et al. argue that truly human-like learning and thinking&#xA;machines (or AGI) should &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;build causal models of the world that support explanation and understanding&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ground learning in intuitive theories of physics and psychology, to support and enrich the knowledge that is learned&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;harness compositionality and &lt;strong&gt;learning-to-learn&lt;/strong&gt; to rapidly acquire and generalize knowledge to new tasks and situations&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Another person that thinks that meta-learning is important for the development of AGI is Ben Goertzel. See e.g. his article &lt;a href=&quot;https://singularityhub.com/2018/07/22/from-here-to-human-level-artificial-general-intelligence-in-four-not-all-that-simple-steps/&quot; rel=&quot;nofollow noreferrer&quot;&gt;From Here to Human-Level Artificial General Intelligence in Four (Not All That) Simple Steps&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T22:10:31.230" LastActivityDate="2020-04-13T22:10:31.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16821" PostTypeId="5" CreationDate="2019-11-27T19:12:08.160" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-27T19:12:08.160" LastActivityDate="2019-11-27T19:12:08.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16822" PostTypeId="4" CreationDate="2019-11-27T19:12:08.160" Score="0" Body="For questions related to the concept of meta-learning (or learning-to-learn)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-11-27T19:12:08.160" LastActivityDate="2019-11-27T19:12:08.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16823" PostTypeId="1" CreationDate="2019-11-27T19:40:08.237" Score="1" ViewCount="24" Body="&lt;p&gt;In &lt;a href=&quot;https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this new book release&lt;/a&gt;, at the top of page 51 the authors mention that to do deep learning on time series tabular data the developer should structure the tensors such that the channels represent the time periods.&lt;/p&gt;&#xA;&lt;p&gt;For example, with a dataset of 17 features where each row represents an hour of a day: the tensor would have 3 dimensions,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;x&lt;/strong&gt; - the 17 features&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;y&lt;/strong&gt; - the # of days&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;z&lt;/strong&gt; - the 24 hours in each day&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So each entry in the tensor would represent that day/hour.&lt;/p&gt;&#xA;&lt;p&gt;Is this necessary to capture time series elements? Would the DNN not learn these representations simply by breaking up the date column into: day, hour?&lt;/p&gt;&#xA;" OwnerUserId="12983" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-27T19:40:08.237" Title="Deep Learning on time series tabular data" Tags="&lt;deep-learning&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16824" PostTypeId="1" AcceptedAnswerId="21981" CreationDate="2019-11-27T23:08:12.747" Score="2" ViewCount="70" Body="&lt;p&gt;I understand that in general an RNN is good for time series data and a CNN image data, and have noticed many blogs explaining the fundamental differences in the models.&lt;/p&gt;&#xA;&lt;p&gt;As a beginner in machine learning and coding, I would like to know from the code perspective, what the differences between an RNN and CNN are in a more practical way.&lt;/p&gt;&#xA;&lt;p&gt;For instance, I think most CNNs dealing with image data use &lt;code&gt;Conv1D&lt;/code&gt;  or &lt;code&gt;Conv2D&lt;/code&gt; and &lt;code&gt;MaxPooling2D&lt;/code&gt; layers and require reshaping input data with code looks something like this &lt;code&gt;Input(shape=(64, 64, 1))&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What are some other things that distinguish CNNs from RNNs from a coding perspective?&lt;/p&gt;&#xA;" OwnerUserId="31075" LastEditorUserId="2444" LastEditDate="2020-06-17T11:50:19.277" LastActivityDate="2020-06-18T00:55:31.880" Title="From an implementation point of view, what are the main differences between an RNN and a CNN?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16825" PostTypeId="1" CreationDate="2019-11-28T00:10:11.997" Score="1" ViewCount="24" Body="&lt;p&gt;does anyone know how to use the correlation coefficient or squared correlation coefficient as a metric in keras in R (although other languages may provide clues). This is for a CNN that functions similar to an image compressor/decompressor. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="31703" LastActivityDate="2019-11-28T00:10:11.997" Title="Keras correlation coefficient as network metric in R" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;statistical-ai&gt;&lt;r&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16826" PostTypeId="1" AcceptedAnswerId="16842" CreationDate="2019-11-28T03:58:29.183" Score="3" ViewCount="623" Body="&lt;p&gt;If we were learning or working in the machine learning field, then we frequently come across the term &amp;quot;probability distribution&amp;quot;. I know what probability, conditional probability, and probability distribution/density in math mean, but what is its meaning in machine learning?&lt;/p&gt;&#xA;&lt;p&gt;Take this example where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an element of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, which is a dataset,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x \in D$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say our dataset (&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;) is MNIST with about 70,000 images, so then &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; becomes any image of those 70,000 images.&lt;/p&gt;&#xA;&lt;p&gt;In many papers and web articles, these terms are often denoted as &lt;em&gt;probability distributions&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p\left(z \mid x \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What does &lt;span class=&quot;math-container&quot;&gt;$p(\cdot)$&lt;/span&gt; even mean, and what kind of output does &lt;span class=&quot;math-container&quot;&gt;$p(\cdot)$&lt;/span&gt; give?&lt;/li&gt;&#xA;&lt;li&gt;Is the output of &lt;span class=&quot;math-container&quot;&gt;$p(\cdot)$&lt;/span&gt; a scalar, vector, or matrix?&lt;/li&gt;&#xA;&lt;li&gt;If the output is vector or matrix, then will the sum of all elements of this vector/matrix always be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is my understanding,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(\cdot)$&lt;/span&gt; is a function which maps the real distribution of the whole dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. Then  &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; gives a scalar probability value given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, which is calculated from real distribution &lt;span class=&quot;math-container&quot;&gt;$p(\cdot)$&lt;/span&gt;. Similar to &lt;span class=&quot;math-container&quot;&gt;$p(H)=0.5$&lt;/span&gt; in a coin toss experiment &lt;span class=&quot;math-container&quot;&gt;$D={\{H,T}\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p\left(z \mid x \right)$&lt;/span&gt; is another function that maps the real distribution of the whole dataset to a vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; given an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; vector is a probability distribution that sums to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Are my assumptions correct?&lt;/p&gt;&#xA;&lt;p&gt;An example would be a &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/08/12/from-autoencoder-to-beta-vae.html#vae-variational-autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;VAE's data generation process&lt;/a&gt;, which is represented in this equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p_\theta(\mathbf{x}^{(i)}) = \int p_\theta(\mathbf{x}^{(i)}\vert\mathbf{z}) p_\theta(\mathbf{z}) d\mathbf{z}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2021-01-21T16:16:04.380" LastActivityDate="2021-01-21T16:16:04.380" Title="What is a probability distribution in machine learning?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;probability-distribution&gt;&lt;notation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16828" PostTypeId="1" CreationDate="2019-11-28T05:36:28.703" Score="1" ViewCount="53" Body="&lt;p&gt;I am trying to make a face login application that authenticates the user when matching the registered face and the given face.&#xA;currently, the issue is I cant extract the face descriptions from the given face when the user is taking the photo in the night or the photo has a backlight.&lt;/p&gt;&#xA;&lt;p&gt;currently, I am using &lt;a href=&quot;https://github.com/justadudewhohacks/face-api.js/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JavaScript API for face detection and face recognition in the browser and nodejs with tensorflow.js&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;can anyone suggest any good face detection and comparison algorithm that resolve my current issues, that will be very helpful for me&lt;/p&gt;&#xA;&lt;p&gt;Now I am extracting face descriptions from the face and using the Euclidean distance equation is used for comparing the similarity of the images. if any good methods for comparison, please suggest&lt;/p&gt;&#xA;" OwnerUserId="31576" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-28T05:36:28.703" Title="How to extract face details from a image" Tags="&lt;image-recognition&gt;&lt;comparison&gt;&lt;facial-recognition&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16829" PostTypeId="1" CreationDate="2019-11-28T08:10:47.857" Score="4" ViewCount="1845" Body="&lt;p&gt;I'm beginning to study and implement GAN to generate more dataset. I'll just try to experiment with state-of-the-art GAN models as described in here &lt;a href=&quot;https://paperswithcode.com/sota/image-generation-on-cifar-10&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://paperswithcode.com/sota/image-generation-on-cifar-10&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is I don't have big dataset (around 1.000) for image classification, I have tried to train and test my dataset with GoogleNet and InceptionV3 and the results are mediocre. I'm afraid that GAN will require bigger dataset than usual image classification. I couldn't find any detailed guideline of how to prepare dataset properly for GAN (e.g. minimum images). So, how many images are required to produce a good GAN model? Also I'm curious whether if I can use my image classification dataset directly to train GAN.&lt;/p&gt;&#xA;" OwnerUserId="20612" LastActivityDate="2020-07-30T02:20:59.150" Title="How many training data required for GAN?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;datasets&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16830" PostTypeId="2" ParentId="16818" CreationDate="2019-11-28T09:15:34.997" Score="2" Body="&lt;p&gt;You are right CNN based models can outperform RNN. You can take a look at this &lt;a href=&quot;https://arxiv.org/pdf/1803.01271.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; where they compared different RNN models with TCN (temporal convolutional networks) on different sequence modeling tasks. Even though there are no big differences in terms of results there are some nice properties that CNN based models offers such as: parallelism, stable gradients and low training memory footprint. In addition to CNN based models there are also attention based models (you might want to take a look at the &lt;a href=&quot;https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer&lt;/a&gt;)  &lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2019-11-28T09:20:40.883" LastActivityDate="2019-11-28T09:20:40.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16831" PostTypeId="2" ParentId="16803" CreationDate="2019-11-28T09:55:36.703" Score="3" Body="&lt;p&gt;The validation loss settles exactly at an error of one. Probably means there's something off with either the kind of data validation set has or with something in the training. An exact validation loss of one almost definitely means there's something off. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend before doing anything thoroughly go through your data or see if there's anything to debug in the model itself. Considering training error reduces there's probably something different about either the formatting of the validation data or the validation data itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A mild description of the type of data and exact problem in hand could further help.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastActivityDate="2019-11-28T09:55:36.703" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16832" PostTypeId="2" ParentId="12390" CreationDate="2019-11-28T09:56:13.963" Score="0" Body="&lt;p&gt;I would recommend learning about Reinforcement learning first. You don't need a dataset as you train your network by letting it play the game over and over again. but knowing how to do so doea mean finding out about markov decision process and how you can use the neural network to solve this.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-11-28T09:56:13.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16833" PostTypeId="2" ParentId="16133" CreationDate="2019-11-28T10:35:12.507" Score="3" Body="&lt;p&gt;This is my own understanding of hidden state in a recurrent network and if its wrong please feel free to let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets take this simple sequence first,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   X = [a,b,c,d,.......,y,z]&#xA;   Y = [b,c,d,e,.......,z,a]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Instead of RNN we will first try to train this in a simple multi layer neural network with one input and one output, here hidden layers details doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can write this relationship in maths as &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x)\rightarrow y$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x&lt;/code&gt; is an element of &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; is an element of &lt;code&gt;Y&lt;/code&gt; and &lt;code&gt;f()&lt;/code&gt; is our neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training, if given input &lt;code&gt;x = a&lt;/code&gt; our neural network will give an output &lt;code&gt;b&lt;/code&gt; because &lt;code&gt;f()&lt;/code&gt; learned the mapping between the sequence X and Y.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now instead of the above sequence try teach this sequence to the same neural network.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; X = [a,a,b,b,c,c,.........y,z,z]&#xA; Y = [a,b,c,,...z,a,b,c......y,z]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More than likely this neural netwok will  not be able to learn the relationship between X and Y. This is because a simple neural network &lt;strong&gt;can't learn and understand the relationship&lt;/strong&gt; between the previous and current characters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we train the same sequence in RNN, in RNN we take &lt;strong&gt;two inputs&lt;/strong&gt; one for our input and a previous hidden values and &lt;strong&gt;two outputs&lt;/strong&gt; one for output and next hidden values&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x,h)\rightarrow (y,h+t)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Important&lt;/strong&gt;: here  &lt;code&gt;h+t&lt;/code&gt; represents next hidden value and not the arithematic addition with time step t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will execute some sequence of this RNN model, at the start hidden values are considerd as zeros&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   x = a and h = 0&#xA;   f(x,h) = (a,next_hidden)&#xA;   prev_hidden = next_hidden&#xA;&#xA;   x = a and h = prev_hidden&#xA;   f(x,h) = (b,next_hidden)&#xA;   prev_hidden = next_hidden&#xA;&#xA;   x = b and h = prev_hidden&#xA;   f(x,h) = (c,next_hidden)&#xA;   prev_hidden = next_hidden&#xA;&#xA;   and so on &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we look at the above process we can see that we are taking previous hidden state values to compute the next hidden state. What happens is while we iterate through this process &lt;code&gt;prev_hidden = next_hidden&lt;/code&gt; it also &lt;strong&gt;encodes some information&lt;/strong&gt; about our sequence which will help in predicting our next character.&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2019-11-28T10:35:12.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16834" PostTypeId="1" AcceptedAnswerId="16840" CreationDate="2019-11-28T11:49:02.213" Score="3" ViewCount="188" Body="&lt;p&gt;I am currently trying to understand SAC (Soft Actor-Critic), and I am thinking of it as a basic actor-critic with the entropy included. However, I expected the entropy to appear in the Q-function. From &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/sac.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpinningUp-SAC&lt;/a&gt;, it looks like the entropy is entering through the value-function, so I'm thinking it enters by the &lt;span class=&quot;math-container&quot;&gt;$\log \pi_{\phi}(a_t \mid s_t)$&lt;/span&gt; in the value function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a little stuck on understanding SAC, can anyone confirm/explain this to me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, side-note question: is being a soft agent equivalent to including entropy in one of the object functions? &lt;/p&gt;&#xA;" OwnerUserId="31714" LastEditorUserId="2444" LastEditDate="2020-11-23T01:46:30.657" LastActivityDate="2020-11-23T01:46:30.657" Title="Where does entropy enter in Soft Actor-Critic?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;soft-actor-critic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16835" PostTypeId="1" CreationDate="2019-11-28T12:36:12.400" Score="2" ViewCount="103" Body="&lt;p&gt;In Attention Is All You Need paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;That is, the output of each sub-layer is &lt;span class=&quot;math-container&quot;&gt;$LayerNorm(x+Sublayer(x))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Sublayer(x)$&lt;/span&gt; is the function implemented by the sub-layer itself. We apply dropout to the output of each sub-layer, before it is added to the sub-layer input and normalized.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which makes the final formula &lt;span class=&quot;math-container&quot;&gt;$LayerNorm(x+Dropout(Sublayer(x)))$&lt;/span&gt;. However, in &lt;a href=&quot;https://github.com/tensorflow/models/blob/0effd158ae1e6403c6048410f79b779bdf344d7d/official/transformer/model/transformer.py#L278-L288&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorflow/models/blob/0effd158ae1e6403c6048410f79b779bdf344d7d/official/transformer/model/transformer.py#L278-L288&lt;/a&gt;, I see&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def __call__(self, x, *args, **kwargs):&#xA;  # Preprocessing: apply layer normalization&#xA;  y = self.layer_norm(x)&#xA;&#xA;  # Get layer output&#xA;  y = self.layer(y, *args, **kwargs)&#xA;&#xA;  # Postprocessing: apply dropout and residual connection&#xA;  if self.train:&#xA;    y = tf.nn.dropout(y, 1 - self.postprocess_dropout)&#xA;  return x + y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which ends up as &lt;span class=&quot;math-container&quot;&gt;$x+Dropout(Sublayer(LayerNorm(x)))$&lt;/span&gt;. Plus there are extra &lt;code&gt;LayerNorm&lt;/code&gt;s as final layers in both encoder and decoder stacks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a quick test, the performance of this model seems to be better than if I change back to the paper's order of operations. My question is: why? And could it be predicted in advance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I note that &lt;a href=&quot;https://arxiv.org/pdf/1904.10509.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generating Long Sequences with Sparse Transformers&lt;/a&gt; uses the &lt;span class=&quot;math-container&quot;&gt;$x+Dropout(Sublayer(LayerNorm(x)))$&lt;/span&gt; order, but doesn't discuss it, unlike the other changes it makes to Transformer.&lt;/p&gt;&#xA;" OwnerUserId="31712" LastActivityDate="2019-11-28T12:36:12.400" Title="Position of layer normalization in transformer model" Tags="&lt;deep-learning&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16836" PostTypeId="1" CreationDate="2019-11-28T13:12:13.470" Score="1" ViewCount="14" Body="&lt;p&gt;I am trying to recreate the architecture of the following paper: &lt;a href=&quot;https://arxiv.org/pdf/1807.03058.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1807.03058.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me in explaining how are the feature maps coming out of the output of GradCam used in the following conv layers?&lt;/p&gt;&#xA;" OwnerUserId="31516" LastActivityDate="2019-11-28T13:12:13.470" Title="Can Grad CAM feature maps be used for Training?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;gradient-descent&gt;&lt;architecture&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16837" PostTypeId="2" ParentId="16807" CreationDate="2019-11-28T13:19:32.460" Score="1" Body="&lt;p&gt;A genetic algorithm is typically a single population designed to optimise to a specific task, say minimising the distance on the &lt;a href=&quot;https://simple.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;travelling salesman problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evolutionary game theory algorithms typically model changes between populations that are in &lt;em&gt;competition&lt;/em&gt;, generally by using genetic algorithms as above but framed within a broader competitive environment between actors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a problem like the travelling salesman problem, it might frame the game as one with competing players where a player getting to a city 'locks' that city and makes it impassable to other players. In these situations new optimisations like localism over adventurism etc may develop, and while players are still trying to minimise the distance travelled overall via their respective genetic algorithm's fitness function, they have to do so in a directly competitive environment with other strategies which quickly creates a lot of additional nuance and depth.&lt;/p&gt;&#xA;" OwnerUserId="14997" LastEditorUserId="14997" LastEditDate="2019-11-28T13:35:45.417" LastActivityDate="2019-11-28T13:35:45.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16839" PostTypeId="2" ParentId="16807" CreationDate="2019-11-28T14:23:06.210" Score="2" Body="&lt;p&gt;Philip's answer is good, but I'll add to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;GA&lt;/a&gt;, a population of individuals (typically represented by bit strings) is evaluated for its fitness on a particular task. Each individual is evaluated separately by a fitness function than can determine its quality. In the &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Traveling Salesman Problem&lt;/a&gt;, the bit string might represent a sequence of numbers, for instance, corresponding to an order in which cities are visited during the tour. The fitness function would inspect a single individual, compute the total cost of the tour, and assign that individual a fitness based on that value. Low scoring individuals are removed, high scoring individuals generate variants on themselves, and the process repeats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Evolutionary Game Theory, a population of individuals is also evaluated on some task, but usually the task involves &lt;em&gt;interaction&lt;/em&gt; between the individuals. For example, you could use an EGT simulation to study what happens in a game like Iterated &lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prisoner's Dilemma&lt;/a&gt;. Here, an individual's fitness doesn't just depend on the rules of the task, but on the behaviors and strategies of the other players in the population. A strategy that is highly effective at first (like always cooperate) will quickly die out once defectors appear. Defectors are effective as long as there are some cooperators to prey upon, but are quickly defeated by strategies like &lt;a href=&quot;https://en.wikipedia.org/wiki/Tit_for_tat&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tit-for-Tat&lt;/a&gt;. Usually researchers are not interested in the specific strategies that emerge so much as in the population &lt;em&gt;dynamics&lt;/em&gt; over the course of the simulation, and in what kinds of population equilibria can emerge. Check out some of Dan Ashlock's &lt;a href=&quot;http://eldar.mathstat.uoguelph.ca/dashlock/Eprints.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;papers&lt;/a&gt; on Game Theory for more. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-28T14:23:06.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16840" PostTypeId="2" ParentId="16834" CreationDate="2019-11-28T15:14:16.873" Score="4" Body="&lt;p&gt;In the answer I'll be using notation similar to the one from the SAC paper.&#xA;If we look at the standard objective function for policy gradient methods we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;J_\pi &amp;amp;= V_\pi(s_t)\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[Q(s_t, a_t)]\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[ \mathbb E_{s_{t+1} \sim p(s|s_t, a_t)} [r(s_t, a_t) + V(s_{t+1})]]\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[ \mathbb E_{s_{t+1} \sim p(s|s_t, a_t)} [r(s_t, a_t) +  \mathbb E_{a_{t+1} \sim \pi(a|s_{t+1})}[ \mathbb E_{s_{t+2} \sim p(s|s_{t+1}, a_{t+1})} [r(s_{t+1}, a_{t+1}) + V(s_{t+2})]]]]\\&#xA;&amp;amp;\cdots\\&#xA;&amp;amp;= \sum_t \mathbb E_{(a_t, s_t) \sim \rho_\pi} [r( s_t, a_t)]&#xA;\end{align}&lt;/span&gt;&#xA;If you keep unwinding this &lt;span class=&quot;math-container&quot;&gt;$V(s_{t+i})$&lt;/span&gt; you will get expected sum of rewards.&#xA;We can define soft state value as&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;V(s_t) &amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[Q(s_t, a_t) + \mathcal H(\cdot|s_t)]\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[Q(s_t, a_t) + \mathbb E_{a \sim \pi(a|s_t)}[-\log(\pi(a|s_t))]]\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[Q(s_t, a_t) - \log(\pi(a_t|s_t))]&#xA;\end{align}&lt;/span&gt;&#xA;third equality comes from the fact that &lt;span class=&quot;math-container&quot;&gt;$\mathbb E_{a \sim \pi(a|s_t)}[-\log(\pi(a|s_t))]$&lt;/span&gt; is nonrandom so its the same thing as if we are sampling over &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; only once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In maximum entropy framework objective function would then be &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;J_\pi &amp;amp;= V_\pi(s_t)\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[Q(s_t, a_t) - \log(\pi(a_t|s_t))]\\&#xA;&amp;amp;= \mathbb E_{a_t \sim \pi(a|s_t)}[ \mathbb E_{s_{t+1} \sim p(s|s_t, a_t)} [r(s_t, a_t) - \log(\pi(a_t|s_t)) + V(s_{t+1})]]\\&#xA;&amp;amp; \cdots\\&#xA;&amp;amp;= \sum_t \mathbb E_{(a_t, s_t) \sim \rho_\pi} [r(s_t, a_t) -\log(\pi(a_t|s_t))]\\&#xA;&amp;amp;= \sum_t \mathbb E_{(a_t, s_t) \sim \rho_\pi} [r(s_t, a_t) + \mathcal H(\cdot|s_t)]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="31714" LastEditDate="2019-11-29T08:26:48.027" LastActivityDate="2019-11-29T08:26:48.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16841" PostTypeId="2" ParentId="16826" CreationDate="2019-11-28T15:17:21.000" Score="2" Body="&lt;p&gt;A probability distribution in ML is the same as a probability distribution elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A probability distribution (or probability function, or probability mass function, or probability density function) is any function that accepts as input elements of some specific set &lt;span class=&quot;math-container&quot;&gt;$x \in X$&lt;/span&gt;, and produces as output, real-valued numbers between 0 and 1 (inclusive), such that &lt;span class=&quot;math-container&quot;&gt;$\int_{x \in X} p(x) = 1$&lt;/span&gt; or for discrete sets, &lt;span class=&quot;math-container&quot;&gt;$\sum_{x \in X} p(x) = 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These distributions can also be more complex. For example, a conditional probability distribution &lt;span class=&quot;math-container&quot;&gt;$P(Y|X)$&lt;/span&gt; or a joint probability distribution &lt;span class=&quot;math-container&quot;&gt;$P(X,Y)$&lt;/span&gt; accept more than one input, but again, are constrained to producing an output in the range 0 to 1, and to ensuring that the summation of the output over all possible inputs is exactly 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When these conditions are met, the functions output can be interpreted as a belief about the percentage of times the input event will occur, out of all events, or as a degree of believe in the input event having occurred vs. other events (i.e. it can be interpreted as a probability).  &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-11-28T15:17:21.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16842" PostTypeId="2" ParentId="16826" CreationDate="2019-11-28T15:50:11.347" Score="3" Body="&lt;h3&gt;Random variables&lt;/h3&gt;&#xA;&lt;p&gt;You do not necessarily need to understand the concept of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_variable&quot; rel=&quot;nofollow noreferrer&quot;&gt;random variable (r.v.)&lt;/a&gt; to understand the concept of a probability distribution, but the concept of a random variable is strictly connected to the concept of a probability distribution (given that each random variable has an associated probability distribution), so, before proceeding, you should get familiar with the concept of an r.v., which is a (measurable) &lt;strong&gt;function&lt;/strong&gt; from the sample space (the set of possible outcomes of an experiment) to a measurable space (you can ignore the definition of a measurable space and assume that the codomain of the random variable is a finite set of numbers).&lt;/p&gt;&#xA;&lt;h3&gt;Probability measure, cdf, pdf and pmf&lt;/h3&gt;&#xA;&lt;p&gt;The expression &amp;quot;probability distribution&amp;quot; can be ambiguous because it can be used to refer to different (even though related) mathematical concepts, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_measure&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability measure&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Cumulative_distribution_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;cumulative distribution function (c.d.f.)&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_density_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability density function (p.d.f.)&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_mass_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability mass function (p.m.f.)&lt;/a&gt;. If a person uses the expression &amp;quot;probability distribution&amp;quot;, he (or she) intentionally (or not) refers to one or more of these mathematical concepts, depending on the context. However, &lt;strong&gt;a probability distribution is almost always a synonym for &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_measure&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability measure&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_density_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;c.d.f.&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, if I say &amp;quot;Consider the Gaussian probability distribution&amp;quot;, in that case, I could be referring to either the c.d.f. or the p.d.f. (or both) of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian distribution&lt;/a&gt;. Why couldn't I be referring to the p.m.f. of the Gaussian distribution? Because the Gaussian distribution is a continuous distribution, so it is a distribution associated with a continuous random variable, that is, a random variable that can take on continuous values (e.g. real numbers), so a Gaussian distribution does not have an associated p.m.f. or, in other words, no p.m.f. is defined for the Gaussian distribution. Why don't I simply say &amp;quot;Consider the p.d.f. of the Gaussian distribution.&amp;quot; or &amp;quot;Consider a Gaussian p.d.f.&amp;quot;? Because it is unnecessarily restrictive, given that, if I say &amp;quot;Consider the Gaussian distribution&amp;quot; I am implicitly also considering a p.d.f. and c.d.f. of the Gaussian distribution.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, in the case of a discrete distribution, such as the Bernoulli distribution, only the c.d.f. and p.m.f. are defined, so the Bernoulli distribution does not have an associated p.d.f.&lt;/p&gt;&#xA;&lt;p&gt;However, it is important to recall that both continuous and discrete distributions have an associated c.d.f., so the expression &amp;quot;probability distribution&amp;quot; almost always (implicitly) refers to a c.d.f., which is defined based on a probability measure (as stated above).&lt;/p&gt;&#xA;&lt;h3&gt;Notation&lt;/h3&gt;&#xA;&lt;p&gt;In the same vein, the notation &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; can be as ambiguous as the expression &amp;quot;probability distribution&amp;quot;, given that it can refer to different (but again related) concepts. However, &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; usually refers to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_measure&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability measure&lt;/a&gt; (so it refers to a probability distribution, given that a probability distribution is almost always a synonym for probability measure). In this case, assuming for simplicity that the r.v. is discrete, &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; is a shorthand for &lt;span class=&quot;math-container&quot;&gt;$p(X=x)$&lt;/span&gt;, which is also written as &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X=x)$&lt;/span&gt;&lt;/strong&gt; or &lt;span class=&quot;math-container&quot;&gt;$\operatorname{Pr}(X=x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a r.v., &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; a &lt;a href=&quot;https://en.wikipedia.org/wiki/Realization_(probability)&quot; rel=&quot;nofollow noreferrer&quot;&gt;realization&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (that is, a value that the r.v. &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; can take) and &lt;span class=&quot;math-container&quot;&gt;$X=x$&lt;/span&gt; represents an &lt;a href=&quot;https://en.wikipedia.org/wiki/Event_(probability_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;event&lt;/a&gt;. Given that an r.v. is a function, the notation &lt;span class=&quot;math-container&quot;&gt;$X=x$&lt;/span&gt; may look a bit weird.&lt;/p&gt;&#xA;&lt;p&gt;In the case of a discrete r.v., &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; can also refer to a p.m.f. and it can be defined as &lt;span class=&quot;math-container&quot;&gt;$p_X(x) = \mathbb{P}(X=x)$&lt;/span&gt; (I added the subscript &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; to emphasize that this is the p.m.f. of the discrete r.v. &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;). In the case of a continuous r.v., the p.d.f. is often denoted as &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. In the case of both discrete and continuous r.v.s, the c.d.f is usually denoted with &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; and it is defined as &lt;span class=&quot;math-container&quot;&gt;$F_X(x) = \mathbb{P}(X \leq x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}$&lt;/span&gt; is again a probability measure (or probability distribution). The p.d.f. of a continuous r.v. is then defined as the derivative of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;. At this point, it should be clear why a probability distribution can refer to different but related concepts, but, in any case, it always refers to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_measure&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability measure&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Empirical distributions&lt;/h3&gt;&#xA;&lt;p&gt;There are also &lt;a href=&quot;https://en.wikipedia.org/wiki/Empirical_distribution_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;empirical distributions&lt;/a&gt;, which are distributions of the data that you have collected. For example, if you toss a coin 10 times, you will collect the results (&amp;quot;heads&amp;quot; or &amp;quot;tails&amp;quot;). You can count the number of times the coin landed on heads and tails, then you plot these numbers as a histogram, which essentially represents your empirical distribution, where the adjective &amp;quot;empirical&amp;quot; usually refers to the fact that there is an experiment involved.&lt;/p&gt;&#xA;&lt;h3&gt;Multivariate r.v.s and distributions&lt;/h3&gt;&#xA;&lt;p&gt;To complicate things even more, there are also &lt;a href=&quot;https://en.wikipedia.org/wiki/Multivariate_normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;multivariate random variables and probability distributions&lt;/a&gt;. However, all the concepts above more or less are also applicable in this case.&lt;/p&gt;&#xA;&lt;h3&gt;Parametrized distributions&lt;/h3&gt;&#xA;&lt;p&gt;A parametrized probability distribution, often denoted by &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}$&lt;/span&gt;, is&#xA;a &lt;a href=&quot;https://en.wikipedia.org/wiki/Parametric_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;family of probability distributions&lt;/a&gt; (defined by the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;), rather than a single probability distribution. For example, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(0, 1)$&lt;/span&gt; refers to a &lt;em&gt;single&lt;/em&gt; Gaussian distribution with zero mean and unit variance. However, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(\mu, \sigma)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\theta=(\mu,  \sigma)$&lt;/span&gt; is a variable, is a family (or collection) of distributions.&lt;/p&gt;&#xA;&lt;h3&gt;Conclusion&lt;/h3&gt;&#xA;&lt;p&gt;To conclude, it is completely understandable that you are confused, given that the terminology and notation are used inconsistently, and there are several involved concepts, which I have not extensively covered in this answer (for example, I have not mentioned the concept of a probability space). If you get familiar with the concepts of probability measures, random variables, p.m.f., p.d.f., c.d.f., etc., and how they are related, then you will start to get a better feeling of the whole picture.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T15:34:16.217" LastActivityDate="2021-01-21T15:34:16.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16843" PostTypeId="2" ParentId="16769" CreationDate="2019-11-28T16:28:52.430" Score="2" Body="&lt;p&gt;There are two main things to consider for dealing with imbalanced data:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;During Evaluation: Using a suitable metric to try to evaluate your model and try to optimize on when you are fine-tuning your random-forest. For imbalanced data you usually use F1 score but since a high recall (predicting sick more often) is important here, F2 score (or other F-beta score where beta&gt;1) is more suitable &lt;a href=&quot;https://en.wikipedia.org/wiki/F1_score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/F1_score&lt;/a&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can also check for example &lt;a href=&quot;https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html&lt;/a&gt; for more about how to deal with imbalanced data in general&lt;/p&gt;&#xA;" OwnerUserId="27851" LastActivityDate="2019-11-28T16:28:52.430" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16844" PostTypeId="2" ParentId="13725" CreationDate="2019-11-28T16:56:21.230" Score="1" Body="&lt;p&gt;Let me add an example from machine learning that shows that resorting to randomness is the optimal way, sometimes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When working on the whole data is not tractable (computation cost, data does not fit in memory), working on random samples can be an optimal way to train a machine learning algorithm. One of the most used optimization technique in those cases is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic Gradient Descent&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an iterative procedure that computes the &lt;a href=&quot;https://en.wikipedia.org/wiki/Estimation&quot; rel=&quot;nofollow noreferrer&quot;&gt;estimates&lt;/a&gt; of the true &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient&lt;/a&gt; of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss function&lt;/a&gt; that needs to be minimized, over a &lt;em&gt;randomly&lt;/em&gt; selected data point from the whole data. After getting the gradient estimate, the weights are updated, all this done by the well known &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;back-propagation&lt;/a&gt; algorithm. This procedure is repeated many times until a &lt;a href=&quot;https://en.wikipedia.org/wiki/Early_stopping&quot; rel=&quot;nofollow noreferrer&quot;&gt;stopping criterion&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rule is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \theta_{k+1} \leftarrow{} \theta_{k} - \eta_k \nabla (f_{i(k)}(x_k))$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the weights of the network, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the loss function whose gradient is computed w.r.t the weights of the network,   &lt;span class=&quot;math-container&quot;&gt;$x_k$&lt;/span&gt; is the randomly chosen sample to compute gradients for, and &lt;span class=&quot;math-container&quot;&gt;$\eta_k$&lt;/span&gt; is the step size to multiply the negative of the gradient with. &lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-11-28T17:50:24.467" LastActivityDate="2019-11-28T17:50:24.467" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16845" PostTypeId="1" AcceptedAnswerId="16851" CreationDate="2019-11-28T20:50:30.713" Score="2" ViewCount="1391" Body="&lt;p&gt;There are three things in every constraint satisfaction problem (CSP):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Variables&lt;/li&gt;&#xA;&lt;li&gt;Domain&lt;/li&gt;&#xA;&lt;li&gt;Constraints&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the given scenario, I know how to identify the constraints, but I don't know how to identify the variables and the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The given scenario is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You are given a &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; board, where &lt;span class=&quot;math-container&quot;&gt;$n \geq 3$&lt;/span&gt;. On this board, you have to put &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; knights where &lt;span class=&quot;math-container&quot;&gt;$k &amp;lt; n^2$&lt;/span&gt;, such that no knight is attacking the other knight. The knights are expected to be placed on different squares on the board. A knight can move two squares vertically and one square horizontally or two squares horizontally and one square vertically. The knights attack each other if one of them can reach the other in a single move. For example, on a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; board, we can place &lt;span class=&quot;math-container&quot;&gt;$k=5$&lt;/span&gt; knights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the input is &lt;span class=&quot;math-container&quot;&gt;$m = 3, n = 3, k = 5$&lt;/span&gt;. There are two solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution 1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K A K   &#xA;A K A    &#xA;K A K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solution 2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A K A&#xA;K K K&#xA;A K A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31420" LastEditorUserId="2444" LastEditDate="2019-11-28T23:20:54.963" LastActivityDate="2019-11-29T16:51:28.943" Title="How can I formulate the k-knights problem as a constraint satisfaction problem?" Tags="&lt;problem-solving&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16846" PostTypeId="1" CreationDate="2019-11-28T20:50:31.353" Score="1" ViewCount="60" Body="&lt;p&gt;Suppose an AI is to play the game flappy bird. And the fitness function is how long the bird has traveled before the game ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would we have multiple neural networks initialized at the beginning with random weights (as in each bird has its own network) and then we determine the neural networks that have lasted the longest for the game and then we perform a selection of weights from the &quot;better&quot; neural networks followed by mutation? Those will then be used as the new weights of a brand new neural network (ie the offspring from two &quot;better&quot; neural networks?)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is the case, does that mean there is no backpropagation because there isn't a cost function?  &lt;/p&gt;&#xA;" OwnerUserId="31727" LastEditorUserId="2444" LastEditDate="2019-11-28T21:43:40.617" LastActivityDate="2019-12-01T02:35:58.010" Title="How are weights updated in a genetic algorithm with neural network?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16847" PostTypeId="2" ParentId="16846" CreationDate="2019-11-28T21:35:24.493" Score="1" Body="&lt;p&gt;I found my answer in a different post: &lt;a href=&quot;https://stackoverflow.com/q/31708478/3924118&quot;&gt;How to evolve weights of a neural network in Neuroevolution?&lt;/a&gt;. Note that the genetic algorithm is a subcategory of the neuroevolution algorithm. Short answer, my original thoughts were correct.&lt;/p&gt;&#xA;" OwnerUserId="31727" LastEditorUserId="2444" LastEditDate="2019-12-01T02:35:58.010" LastActivityDate="2019-12-01T02:35:58.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16848" PostTypeId="1" AcceptedAnswerId="16853" CreationDate="2019-11-28T23:19:57.650" Score="2" ViewCount="53" Body="&lt;p&gt;In policy gradient methods such as A3C/PPO, the output from the network is probabilities for each of the actions. At training time, the action to take is sampled from the probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When evaluating the policy in an environment, what would be the effect of always picking the action that has the highest probability instead of sampling from the probability distribution?&lt;/p&gt;&#xA;" OwnerUserId="31728" LastActivityDate="2019-11-29T13:02:51.130" Title="What is the effect of picking action deterministicly at inference with Policy Gradient Methods?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16850" PostTypeId="1" AcceptedAnswerId="16852" CreationDate="2019-11-29T01:51:07.773" Score="4" ViewCount="277" Body="&lt;p&gt;&lt;em&gt;When using CNNs for non-image (times series) data prediction, what are some constraints or things to look out for as compared to image data?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;To be more precise, I notice there are different types of layers in a CNN model, as described below, which seem to be particularly designed for image data.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;convolutional layer&lt;/strong&gt; that extracts features from a source image. Convolution helps with blurring, sharpening, edge detection, noise reduction, or other operations that can help the machine to learn specific characteristics of an image.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;pooling layer&lt;/strong&gt; that reduces the image dimensionality without losing important features or patterns.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;fully connected layer&lt;/strong&gt; also known as the dense layer, in which the results of the convolutional layers are fed through one or more neural layers to generate a prediction.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Are these operations also applicable to non-image data (for example, times series)?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="31075" LastEditorUserId="2444" LastEditDate="2020-12-06T12:58:30.813" LastActivityDate="2020-12-06T13:02:32.130" Title="Can CNNs be applied to non-image data, given that the convolution and pooling operations are mainly applied to imagery?" Tags="&lt;convolutional-neural-networks&gt;&lt;time-series&gt;&lt;convolution&gt;&lt;pooling&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16851" PostTypeId="2" ParentId="16845" CreationDate="2019-11-29T03:34:57.573" Score="2" Body="&lt;p&gt;There are many possible ways to encode this problem, and some will be more advantageous than others&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An encoding that seems like a reasonable starting point to me is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Variables: Let &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; be a set of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; variables representing the coordinates of knights on the chess board.&lt;/li&gt;&#xA;&lt;li&gt;Domain: The domain of each variable is initially all vectors in &lt;span class=&quot;math-container&quot;&gt;$[n]^2$&lt;/span&gt;. Denote the components of these vectors with &lt;span class=&quot;math-container&quot;&gt;$[r, c]$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Constraints: &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Let A be the set of 8 attacking moves: &lt;span class=&quot;math-container&quot;&gt;$A=\{[2,1],[2,-1],[-2,1],[-2,-1],[1,2],[1,-2],[-1,2],[-1,-2]\}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\forall_{i &amp;lt; j} S_i \neq S_j$&lt;/span&gt; (no knights on the same squares)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\forall_{i &amp;lt; j} [r_i, c_i] \notin \{[r_j + x, c_j + y] | x,y \in A\}$&lt;/span&gt; (no knight attacks another knight)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That should be it. Note the use of &lt;span class=&quot;math-container&quot;&gt;$\forall_{i&amp;lt;j}$&lt;/span&gt; in the constraints is to reduce the total number of constraints by half, exploiting the symmetry that knight &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; can attack knight &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; iff knight &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; can attack knight &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. You could also use &lt;span class=&quot;math-container&quot;&gt;$\forall_{i\neq j}$&lt;/span&gt;, but it would increase your constraint count to no gain.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-11-29T16:51:28.943" LastActivityDate="2019-11-29T16:51:28.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16852" PostTypeId="2" ParentId="16850" CreationDate="2019-11-29T07:53:51.133" Score="3" Body="&lt;p&gt;Usually, you need to ensure that your convolutions are causal, meaning that there is no information leakage from the future into the past. You could start by looking at this &lt;a href=&quot;https://arxiv.org/pdf/1803.01271.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, which compares Temporal Convolutional Networks (TCN) with vanilla RNNs models.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2020-12-06T13:02:32.130" LastActivityDate="2020-12-06T13:02:32.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16853" PostTypeId="2" ParentId="16848" CreationDate="2019-11-29T13:02:51.130" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When evaluating the policy in an environment, what would be the effect of always picking the action that has the highest probability instead of sampling from the probability distribution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Depends what you mean by &quot;evaluating the policy&quot;. Unlike in value-based methods, such as Q learning, the policy in gradient methods is not implied by anything else, it is described directly by the probability density function that is being optimised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking the maximum probability item will technically change the policy (unless you are already using deterministic policy gradient), and you would be evaluating a &lt;em&gt;different&lt;/em&gt; but related policy to that found by your policy gradient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in a standard MDP environment, and after at least some training, this should be a reasonable process that would give some indication of how well the agent is performing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, the nature of the environment means the agent is relying on a stochastic policy. In some partially-observable scenarios it may be better to decide randomly - a simple example is a corridor that needs to be traversed, but where the state features don't give enough information to determine the true direction. A deterministic policy will not be able to traverse the corridor in both directions, but a stochastic policy will get through it both ways, eventually. Another example is in adversarial situations where another agent can learn your agent's policy (the classic version of that being Scissor/Paper/Stone where two ideal opposed agents would learn probability &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{3}$&lt;/span&gt; for each action according to Nash equilibrium) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't think you have these special cases, then it should be OK to derive a deterministic policy from your policy gradient agent, and assess that. That's not quite the same as assessing the &quot;learned policy&quot;, but is quite sensible to do once you think the agent has converged anyway, since it may still be selecting non-optimal actions at some low probability, and you could get closer to optimal behaviour by removing that.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-11-29T13:02:51.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16854" PostTypeId="1" CreationDate="2019-11-29T18:19:47.383" Score="0" ViewCount="271" Body="&lt;p&gt;I want to detect moving objects in a surveillance video without using machine learning tools (like neural networks).&#xA;Is there a simple way in the OpenCV library? What is an efficient solution for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="30170" LastEditorUserId="30170" LastEditDate="2019-12-01T18:46:06.620" LastActivityDate="2019-12-02T05:06:31.877" Title="How can I detect moving objects in a video by OpenCV without using deep learning techniques?" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16855" PostTypeId="1" CreationDate="2019-11-29T21:05:57.427" Score="2" ViewCount="129" Body="&lt;p&gt;Today I was going through a tutorial of Andrew Ng about &lt;a href=&quot;https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43022.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inception network&lt;/a&gt;. He said that &lt;a href=&quot;https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43022.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GoogLeNet&lt;/a&gt;'s hidden layers are also good in prediction and it had somehow a regularization effect, so it reduces overfitting. I also search on this topic and tried to figure out by reading the &lt;a href=&quot;https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43022.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GoogLeNet paper&lt;/a&gt;. But I am not satisfied. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give me any mathematical intuition or reasoning about this in detail?&lt;/p&gt;&#xA;" OwnerUserId="26342" LastEditorUserId="2444" LastEditDate="2019-11-29T21:35:00.733" LastActivityDate="2019-12-31T07:01:20.180" Title="How GoogleNet actually deal with reducing overfitting?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16856" PostTypeId="2" ParentId="16850" CreationDate="2019-11-29T21:20:40.487" Score="2" Body="&lt;p&gt;You can use CNN for time-series data. The Convolutional Recurrent Neural Network (RCNN) is one of the examples.&lt;/p&gt;&#xA;&lt;p&gt;Convolutional layers basically extract features from images. It is not related to time-series data processing.&lt;/p&gt;&#xA;&lt;p&gt;Some CNNs (such as in ResNet, Highway Networks, and DenseNet) use some recurrent concepts to improve their prediction, but they all are within single datapoint reasoning. You can go through these concepts to improve your intuitions.&lt;/p&gt;&#xA;" OwnerUserId="26342" LastEditorUserId="2444" LastEditDate="2020-12-06T13:01:43.887" LastActivityDate="2020-12-06T13:01:43.887" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16857" PostTypeId="1" AcceptedAnswerId="16870" CreationDate="2019-11-29T23:33:05.727" Score="5" ViewCount="119" Body="&lt;p&gt;Lee Sedol, former world champion, and legendary Go player today announced his retirement with the quote &quot;&lt;a href=&quot;https://www.businessinsider.com/deep-mind-alphago-ai-lee-sedol-south-korea-go-2019-11?r=US&amp;amp;IR=T&quot; rel=&quot;nofollow noreferrer&quot;&gt;Even if I become the No. 1, there is an entity that cannot be defeated&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible that AIs could kill the joy of competitive games(Go, chess, Dota 2, etc.) or (thinking more futuristic with humanoid AIs) in sports?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens if AIs gets better than us at painting and making music. Will we still appreciate it in the same way we do now? &lt;/p&gt;&#xA;" OwnerUserId="31714" LastEditorUserId="31714" LastEditDate="2019-11-30T12:03:00.903" LastActivityDate="2019-11-30T23:36:46.443" Title="Could AI kill the joy of competitive sports and games?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;go&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16858" PostTypeId="2" ParentId="16803" CreationDate="2019-11-30T00:05:25.293" Score="1" Body="&lt;p&gt;The telltale &lt;strong&gt;signature&lt;/strong&gt; of overfitting is when your validation loss starts increasing, while your training loss continues decreasing, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KLM4h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KLM4h.png&quot; alt=&quot;adapted from Wikipedia&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Image adapted from Wikipedia entry on &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that this does not happen in your diagram, hence your model does not overfit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A difference between a training and a validation score by itself does &lt;strong&gt;not&lt;/strong&gt; signify overfitting. This is just the &lt;strong&gt;generalization gap&lt;/strong&gt;, i.e. the &lt;em&gt;expected&lt;/em&gt; gap in the performance between the training and validation sets; quoting from a recent &lt;a href=&quot;https://ai.googleblog.com/2019/07/predicting-generalization-gap-in-deep.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post by Google AI&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An important concept for understanding generalization is the &lt;em&gt;generalization gap&lt;/em&gt;, i.e., the difference between a model’s performance on training data and its performance on unseen data drawn from the same distribution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An MSE of 1.0 (or any other specific value, for that matter) by itself cannot be considered &quot;good&quot; or &quot;bad&quot;; everything depends on the context, i.e. of the particular problem and the actual magnitude of your dependent variable: if you are trying to predict something that is in the order of some thousands (or even hundreds), an MSE of 1.0 does not sound bad; it's not the same if your dependent variable takes values, say, in [0, 1] or similar.&lt;/p&gt;&#xA;" OwnerUserId="11539" LastActivityDate="2019-11-30T00:05:25.293" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16859" PostTypeId="2" ParentId="15885" CreationDate="2019-11-30T00:41:43.780" Score="1" Body="&lt;p&gt;I did an experiment, took a trained densenet121 and kept the bottom layers. I trained the FC layer to a softmax and then to a lambda layer that normalized the vector. I trained the network with imagenet to make the outputs the most far a away from (1,1,1,1,1...1) as possible, so I would get one hot vectors. I did, but the network trained to a single category (put all in the same hot vector). Then I added a penalty that encourages to make vectors different is their features are also different. it improved a little, a dozen categories instead of one, bout noting close to a thousand, the available number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am posting this so nobody wastes his time in a silly idea like this one.&lt;/p&gt;&#xA;" OwnerUserId="30433" LastEditorUserId="30433" LastEditDate="2019-11-30T00:50:11.103" LastActivityDate="2019-11-30T00:50:11.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16860" PostTypeId="1" CreationDate="2019-11-30T04:44:30.333" Score="3" ViewCount="435" Body="&lt;p&gt;Google says that their new AI program MuZero learnt the rules of chess and some other board games without being told so. How is this even possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/deepmind-unveils-muzero-a-new-agent-that-mastered-chess-shogi-atari-and-go-without-knowing-the-d755dc80ff08&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/deepmind-unveils-muzero-a-new-agent-that-mastered-chess-shogi-atari-and-go-without-knowing-the-d755dc80ff08&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17601" LastActivityDate="2020-01-28T07:23:18.180" Title="How did MuZero learn the rules of chess?" Tags="&lt;chess&gt;&lt;google&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16862" PostTypeId="2" ParentId="16855" CreationDate="2019-11-30T09:05:01.503" Score="1" Body="&lt;p&gt;The main reason of overfitting in any neural network is having too many unrestricted trainable degrees of freedom in the model. Methods similar to dropout reduce the number of neurons at each training run which effectively means having a smaller network. On the other hand in &lt;span class=&quot;math-container&quot;&gt;$l_1$&lt;/span&gt;  and &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt; regularization, a term added to the loss function which put a constraint on the total loss calculated at each run. So what we are trying to minimize with such regularizations is not just &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$L +l_1*f(w)$&lt;/span&gt; (for example). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I understand from that paper is that the auxiliary outputs do both at the same time: the point is to build smaller version of the same inception network within itself, and use the losses obtained from those as a constraint on the final loss function. The full model described in the paper is essentially 3 separate models: the first one is a network with 3 inception modules, the second one with 6 and the final one has 9. When loss is calculated, results from the auxiliary outputs added to the total loss with a 0.3 weight. Let us write this as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L = 0.7*L_9+ 0.3*(L_6+L_3).  $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$L_3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_6$&lt;/span&gt; represent the losses calculated at first and second outputs, and &lt;span class=&quot;math-container&quot;&gt;$L_9$&lt;/span&gt; is the loss calculated at the final output of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the function we wish to minimize during training. But when the evaluation is made, the auxiliary outputs are discarded, and just the final softmax layer is used. Not very dissimilar from the idea of using dropout during training but using the full model for predictions.&lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-12-01T06:03:32.247" LastActivityDate="2019-12-01T06:03:32.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16863" PostTypeId="1" CreationDate="2019-11-30T09:39:39.163" Score="1" ViewCount="39" Body="&lt;p&gt;I'm checking out how to manually apply resolution on a first order predicate logic knowledge base and I'm confused about what is allowed or not in the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that we have the following two clauses (where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are constants):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg P(A, B) \vee H(A)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg L(x_1) \vee P(x_1, y_1)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I try to unify these two clauses by making the substitutions &lt;span class=&quot;math-container&quot;&gt;$\{x_1/A, y_1/B\}$&lt;/span&gt; do I get &lt;span class=&quot;math-container&quot;&gt;$\neg L(A) \vee H(A)$&lt;/span&gt; ? Is it allowed to substitute &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; even if &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; doesn't appear in the unified clause?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we have the other way around where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg P(A, y_1) \vee H(A)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg L(x_1) \vee P(x_1, B)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I do &lt;span class=&quot;math-container&quot;&gt;$\{x_1/A, B/y_1\}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\neg L(A) \vee H(A)$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about the case where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg P(A, z_1) \vee H(A)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg L(x_1) \vee P(x_1, y_1)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I substitute &lt;span class=&quot;math-container&quot;&gt;$\{x_1/A, y_1/z_1\}$&lt;/span&gt; and get &lt;span class=&quot;math-container&quot;&gt;$\neg L(A) \vee H(A)$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally there is also the cases where we have something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg P(x_2, y_2) \vee H(z_1)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\neg L(x_1) \vee P(x_1, y_1)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we do &lt;span class=&quot;math-container&quot;&gt;$\{x_1/x_2, y_1/y_2\}$&lt;/span&gt; to get &lt;span class=&quot;math-container&quot;&gt;$\neg L(x_3) \vee H(z_2)$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm really confused about when unification succeeds once we have two clauses with a literal of the same kind (negation in one of them and not in the other one) that are a candidates for unification. &lt;/p&gt;&#xA;" OwnerUserId="31193" LastActivityDate="2021-05-25T11:00:50.117" Title="Does the substituted variable/constant have to appear in the unified term?" Tags="&lt;logic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16864" PostTypeId="2" ParentId="111" CreationDate="2019-11-30T12:01:12.437" Score="1" Body="&lt;p&gt;I think we need to state our own moral before thinking about what the cars moral(or ethical setting) should be. I recommend reading this paper &lt;a href=&quot;https://link.springer.com/article/10.1007/s11948-016-9806-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autonomous Cars: In Favor of a Mandatory Ethics Setting&lt;/a&gt; which argues why it is in everyone's best interest that we prioritize the safety of the majority, and not just the driver(yes, in the best interest of the driver too).&lt;/p&gt;&#xA;&lt;p&gt;You can test your own moral in many different situations, some like your examples, on &lt;a href=&quot;http://moralmachine.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MITs moral machine&lt;/a&gt;. It's rather uncomfortable but very interesting. You can also find some analysis of people's answers on their website.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My answers to your examples:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The car is heading toward a crowd of 10 people crossing the road, so it cannot stop in time, but it can avoid killing 10 people by hitting the wall (killing the passengers)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I assume hitting the brakes is not going to work, or else the dilemma is pointless. I think the car should hit the wall. Pedestrians should not suffer just because someone else is driving a car, especially when there are 10 pedestrians and maximal 5(typically 1 or 2) in the car.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Avoiding killing the rider of the motorcycle considering that the probability of survival is greater for the passenger of the car&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think this one is harder, especially since the motorcycle probably is not autonomous, and (in contrast to the pedestrians in the previous example) riding motorcycles is quite dangerous. Are the motorcyclist accepting the risk when entering the roads? If avoiding the motorcyclist means a probable death for the driver, then no. If not, probably should avoid it.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Killing an animal on the street in favor of a human being&lt;/p&gt;&#xA;&lt;p&gt;Purposely changing lanes to crash into another car to avoid killing a dog&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think humans are more important than animals.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I don't think there exists a correct answer for this.&lt;/strong&gt;&#xA;One of the really interesting things is that from the data collected by the moral machine is that there are big differences based on where in the world you're from. Typically, the western typically countries prioritize saving children over the elderly, while this is not the case for the whole world. Countries with strong governments like Finland and Japan prioritize people abiding the law, while people from countries with weaker/corrupt government does not care so much about that. Even in this comment section, you can find differences! I, for example, think the pedestrians should be spared in the first example,&#xA;while Doxosophoi thinks that it is obvious that the passengers should be protected!&lt;/p&gt;&#xA;" OwnerUserId="31714" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-11-30T12:01:12.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16865" PostTypeId="1" CreationDate="2019-11-30T15:37:31.583" Score="1" ViewCount="36" Body="&lt;p&gt;Suppose I trained a Gaussian process classifier with a linear kernel (using GPML toolbox) and got some feature weights for each input feature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it/When does it make sense to interpret the weights to indicate the real-life importance of each feature or interpret at group level the average over the weights of a group of features? &lt;/p&gt;&#xA;" OwnerUserId="31752" LastEditorUserId="31752" LastEditDate="2019-12-01T11:17:13.923" LastActivityDate="2019-12-01T11:17:13.923" Title="Interpretability of feature weights from Gaussian process classifier" Tags="&lt;machine-learning&gt;&lt;feature-selection&gt;&lt;weights&gt;&lt;gaussian-process&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16868" PostTypeId="1" CreationDate="2019-11-30T20:50:20.917" Score="4" ViewCount="46" Body="&lt;p&gt;I have a neural network that takes as an input a vector of &lt;span class=&quot;math-container&quot;&gt;$x \in R^{1\times d}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; hidden layers and each layer has &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; neurons (including the output layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly the computational complexity of the forward pass of a single input vector would be &lt;span class=&quot;math-container&quot;&gt;$O(d^{2}(s-1))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$d^{2}$&lt;/span&gt; is the computational complexity for the multiplication of the output of each layer and the weight matrix, and this happens &lt;span class=&quot;math-container&quot;&gt;$(s-1)$&lt;/span&gt; times, given that the neural network has &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; layers. We can ignore the activation function because the cost is &lt;span class=&quot;math-container&quot;&gt;$O(d)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if I am correct so far, and the computational complexity of the forward pass is &lt;span class=&quot;math-container&quot;&gt;$O(d^{2}(s-1))$&lt;/span&gt;, is the following correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$O(d^{2}(s-1)) = O(d^{2}s + d^{2}) = O(d^{2}s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would the computational complexity of the forward pass for this NN be &lt;span class=&quot;math-container&quot;&gt;$O(d^{2}s)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="31757" LastEditorUserId="2444" LastEditDate="2019-12-01T02:11:09.857" LastActivityDate="2019-12-01T02:11:09.857" Title="Given an input $x \in R^{1\times d}$ and a network with $s$ hidden layers, is the time complexity of the forward pass $O(d^{2}s)$?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;time-complexity&gt;&lt;computational-complexity&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ClosedDate="2019-12-02T21:02:48.693" ContentLicense="CC BY-SA 4.0" />
  <row Id="16869" PostTypeId="1" CreationDate="2019-11-30T21:38:19.370" Score="2" ViewCount="404" Body="&lt;p&gt;I am trying to implement the original YOLO architecture for object detection, but I am using the COCO dataset. However, I am a bit confused about the image sizes of COCO. The original YOLO was trained on the VOC dataset and it is designed to take 448x448 size images. Since I am using COCO, I thought of cropping down the images to that size. But that would mean I would have to change the annotations file and it might make the process of object detection a bit harder because some of the objects might not be visible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty new to this, so I am not sure if this is the right way or what are some other things that I can do. Any help will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="31760" LastEditorUserId="2444" LastEditDate="2021-01-29T00:05:02.577" LastActivityDate="2021-01-29T00:05:02.577" Title="How can I train YOLO with the COCO dataset?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;pytorch&gt;&lt;yolo&gt;&lt;coco-dataset&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16870" PostTypeId="2" ParentId="16857" CreationDate="2019-11-30T23:36:46.443" Score="6" Body="&lt;p&gt;Unlikely! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chess has been &quot;solved&quot; by AI much longer than GO (chess engines even before AI are way too strong for human players) and still people are playing and competing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply put competition and sports live from the human element. Humans competing against each other will still create the same joy for most people regardless of the fact that all involved players might lose against a computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some select individuals on the highest level might be put off by the new reality but it won't be the end of competition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;No human is faster than a car and yet we still celebrate running competitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed I think long-term we will gain entertainment by watching different AIs and models compete against each other in chess or Go.&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2019-11-30T23:36:46.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16872" PostTypeId="2" ParentId="9106" CreationDate="2019-12-01T00:57:42.820" Score="0" Body="&lt;p&gt;This can be calculated quite easily in the context of 8-queen problem. Just start with a particular configuration. Starting from the queen in the left-most column just keep on counting the non-attacking positions (pairs) on the right with each queen. Carry on column by column towards your right until you reach the last queen. As a special case for the last queen the non-attacking pairs will be zero as their are no other queens after that. &lt;/p&gt;&#xA;" OwnerUserId="31765" LastActivityDate="2019-12-01T00:57:42.820" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16873" PostTypeId="1" CreationDate="2019-12-01T04:16:58.727" Score="2" ViewCount="25" Body="&lt;p&gt;I would like to map the simplest 8X8 matrices, one to one, but am not sure which AI algorithm would give the best performance. I am thinking about the DeepLearning4j, however, I don't know which neural architecture to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to make a simple and a very &quot;stupid&quot; bot for playing the chess. The result I am hoping to obtain is a system which can learn the chess rules rather than make intelligent moves (although it would be great if it could do that as well).  I understand that chess bots are nothing new, however, I am not interested in making a chess bot but a system that can simply learn by giving nothing else than an 8X8 matrix as an input and 8X8 matrix as an output. Chess is irrelevant, and it can be any other game that can be represented with values within an 8X8 matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the newest image mappers that transform horses to zebraz, but I need something more precise and 1 to 1 learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The amount of data I can get for learning is also not an issue (I can generate as much as I want).&lt;/p&gt;&#xA;" OwnerUserId="31766" LastActivityDate="2019-12-01T04:16:58.727" Title="Which algorithm and architecture to use for 1:1 matrix transformation of an 8X8 dimension?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;regression&gt;&lt;rule-acquisition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16874" PostTypeId="1" CreationDate="2019-12-01T06:29:17.120" Score="4" ViewCount="147" Body="&lt;p&gt;For an AI to represent the world, it would be good if it could translate human sentences into something more precise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know, for example, that mathematics can be built up from set theory. So representing statement in language of set theory might be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. &quot;All grass is green&quot; is something like: &lt;span class=&quot;math-container&quot;&gt;$\forall x\in grass: isGreen(x)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then I learned that set theory is built up from something more basic. And that theorem provers use a special form of higher order logic of types. Then there is propositional logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically what the AI would need would be some way representing statements, some axioms, and ways to manipulate the statements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus what would be a good language to use an internal language for an AI? &lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-04-28T09:44:50.560" Title="What would be a good internal language for an AI?" Tags="&lt;knowledge-representation&gt;&lt;programming-languages&gt;&lt;language-model&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16875" PostTypeId="2" ParentId="15761" CreationDate="2019-12-01T08:04:21.403" Score="1" Body="&lt;p&gt;As you point out, they are not equivalent. I guess you could store the time index for each state visited, but there are two problems with this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, if you sample states according to their time index, sampling from the replay memory will become more cumbersome and probably much slower (you'd have to sample the time index and then a specific state with that time index). This is definitely something undesirable. If you choose to add the importance sampling term, then you will make really small most of the terms if &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is not so close to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, while the original objective is very nice to obtain theoretical results, you may ask yourself if that objective is really what you want to maximize. Do you really care more about the performance at the beginning of the trajectory than at the end? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I have no rigorous proofs, I like to think that a better definition for the expected discounted reward is a time average:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\eta(\pi) = \lim_{N\to\infty}\frac{1}{N}\sum_{k=1}^N\mathbb E_{s_k,a_k,s_{k+1},\cdots}\left[\sum_{t=k}^\infty\gamma^{t-k}r(s_t)\right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since each term of the average satisfies the original proofs, the only difference with the equation 14 is that the density is &lt;span class=&quot;math-container&quot;&gt;$\tilde\rho_{\theta_{old}}(s) = \lim_{N\to\infty}\frac{1}{N}\sum_{k=1}^N\rho_{k,\theta_{old}}(s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$$\rho_{k,\theta_{old}} = P(s_k = s)+\gamma P(s_{k+1}=s) + \gamma^2 P(s_{k+2}=s) + \dots.$$&lt;/span&gt;&#xA;You can notice that for a large &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; you basically give the same importance to all of the time-steps. So, the usual implementations actually optimize some complicated function similar to the one I defined, where you give a similar weight to all samples, irrespective of their corresponding time-step.&lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2019-12-01T08:04:21.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16876" PostTypeId="1" CreationDate="2019-12-01T08:51:29.950" Score="2" ViewCount="40" Body="&lt;p&gt;I remember reading an online news article a while ago on AI ethics that described a conference presentation on the subject of a military AI system that predicted terrorist movement inside buildings and used drones to shoot them when they exited the building. The presentation caused protests and sharp criticism in the audience due to the dehumanizing nature of the AI usage presented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article was in mainstream media, but I am unable to find it with Google search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which AI conference presentation was it and are any articles on it available?&lt;/p&gt;&#xA;" OwnerUserId="31769" LastEditorUserId="31769" LastEditDate="2019-12-01T18:39:25.747" LastActivityDate="2019-12-01T18:39:25.747" Title="Which AI conference presentation on predicting terrorist movement inside buildings caused protest in the audience and media?" Tags="&lt;ethics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16879" PostTypeId="1" CreationDate="2019-12-01T11:49:36.080" Score="2" ViewCount="93" Body="&lt;p&gt;I know that stochastic gradient descent always gives different results. What are the best practices to reduce this variance today?&#xA;I tried to predict simple function with two different approaches and every time I train them I see very different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def plot(model_out):&#xA;  fig, ax = plt.subplots()&#xA;  ax.grid(True, which='both')&#xA;  ax.axhline(y=0, color='k', linewidth=1)&#xA;  ax.axvline(x=0, color='k', linewidth=1)&#xA;&#xA;  ax.plot(x_line, y_line, c='g', linewidth=1)&#xA;  ax.scatter(inputs, targets, c='b', s=8)&#xA;  ax.scatter(inputs, model_out, c='r', s=8)&#xA;&#xA;a = 5.0; b = 3.0; x_left, x_right = -16., 16.&#xA;NUM_EXAMPLES = 200&#xA;noise   = tf.random.normal((NUM_EXAMPLES,1))&#xA;&#xA;inputs  = tf.random.uniform((NUM_EXAMPLES,1), x_left, x_right)&#xA;targets = a * tf.sin(inputs) + b + noise&#xA;x_line  = tf.linspace(x_left, x_right, 500)&#xA;y_line  = a * tf.sin(x_line) + b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Keras training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential()&#xA;model.add(tf.keras.layers.Dense(50, activation='relu', input_shape=(1,)))&#xA;model.add(tf.keras.layers.Dense(50, activation='relu'))&#xA;model.add(tf.keras.layers.Dense(1))&#xA;&#xA;model.compile(loss='mse', optimizer=tf.keras.optimizers.Adam(0.01))&#xA;model.fit(inputs, targets, batch_size=200, epochs=2000, verbose=0)&#xA;&#xA;print(model.evaluate(inputs, targets, verbose=0))&#xA;plot(model.predict(inputs))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Apel6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Apel6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manual training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential()&#xA;model.add(tf.keras.layers.Dense(50, activation='relu', input_shape=(1,)))&#xA;model.add(tf.keras.layers.Dense(50, activation='relu'))&#xA;model.add(tf.keras.layers.Dense(1))&#xA;&#xA;optimizer = tf.keras.optimizers.Adam(0.01)&#xA;&#xA;@tf.function&#xA;def train_step(inpt, targ):&#xA;  with tf.GradientTape() as g:&#xA;    model_out = model(inpt)&#xA;    model_loss = tf.reduce_mean(tf.square(tf.math.subtract(targ, model_out)))&#xA;&#xA;  gradients = g.gradient(model_loss, model.trainable_variables)&#xA;  optimizer.apply_gradients(zip(gradients, model.trainable_variables))&#xA;  return model_loss&#xA;&#xA;train_ds = tf.data.Dataset.from_tensor_slices((inputs, targets))&#xA;train_ds = train_ds.repeat(2000).batch(200)&#xA;&#xA;def train(train_ds):&#xA;  for inpt, targ in train_ds:&#xA;    model_loss = train_step(inpt, targ)&#xA;  tf.print(model_loss)&#xA;&#xA;train(train_ds)&#xA;plot(tf.squeeze(model(inputs)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GvgB5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GvgB5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31772" LastActivityDate="2019-12-01T21:59:00.560" Title="How to reduce variance of the model loss during training?" Tags="&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16880" PostTypeId="1" CreationDate="2019-12-01T11:52:09.413" Score="1" ViewCount="250" Body="&lt;p&gt;In reinforcement learning, what is the difference between &lt;em&gt;policy iteration&lt;/em&gt; and &lt;em&gt;value iteration&lt;/em&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As much as I understand, in value iteration, you use the Bellman equation to solve for the optimal policy, whereas, in policy iteration, you randomly select a policy π, and find the reward of that policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My doubt is that if you are selecting a random policy π in PI, how is it guaranteed to be the optimal policy, even if we are choosing several random policies.&lt;/p&gt;&#xA;" OwnerUserId="31420" LastActivityDate="2019-12-01T11:52:09.413" Title="What is the difference between value iteration and policy iteration?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2019-12-01T12:06:48.703" ContentLicense="CC BY-SA 4.0" />
  <row Id="16881" PostTypeId="1" CreationDate="2019-12-01T12:20:24.140" Score="1" ViewCount="54" Body="&lt;p&gt;The&#xA;problem in &lt;em&gt;Iris data&lt;/em&gt; is to classify three species of iris (&lt;em&gt;setosa, versicolor and virginica&lt;/em&gt;) by&#xA;four-dimensional attribute vectors consisting of &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sepal length (x1)&lt;/li&gt;&#xA;&lt;li&gt;sepal width (x2)&lt;/li&gt;&#xA;&lt;li&gt;petal length (x3)&lt;/li&gt;&#xA;&lt;li&gt;petal width (x4)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Every attribute of the fuzzy classifier is assigned with three linguistic&#xA;terms (fuzzy sets): short, medium and long. With normalized attribute values the&#xA;membership functions of these fuzzy sets for all the four attributes are depicted in the&#xA;figure below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1QQuC.jpg&quot; alt=&quot;Membership Function&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider the situation that a set of Rules are given - some examples are as below:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;R1: If (x3=short OR medium) AND (x4=short) Then iris &lt;strong&gt;setosa&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;R2: If (x2=short OR medium) AND (x3=long) AND (x4=long) Then iris &lt;strong&gt;virginica&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, I want to make a fuzzy classifier on Iris dataset. The problem here is, I need to use a membership function for Consequents in Rules(i.e, The 3 classes) so as to be able to compute aggregation of rules and difuzzification. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the proper pre-defined domain and membership function for three classes? &lt;/li&gt;&#xA;&lt;li&gt;All the Consequents (Outputs)  in fuzzy classifier need to acquire membership functions?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31773" LastActivityDate="2019-12-01T12:20:24.140" Title="The membership function of Consequents (Outputs) in Fuzzy classifier" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16882" PostTypeId="2" ParentId="16874" CreationDate="2019-12-01T13:16:24.693" Score="2" Body="&lt;p&gt;This is (even though it doesn't look like it at first glance) a deeply philosophical question about the nature of 'meaning'. This answer is necessarily limited in scope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways of representing knowledge, and countless formalisms have been developed since the early days of AI. Many of them are based on some kind of predicate calculus, ontologies, semantic networks (providing eg inheritance of features and part-of relationships), and they seem to work fine for limited domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem is the grounding: if you have a predicate &lt;em&gt;isGreen(x)&lt;/em&gt;, what does that actually mean? How is it related to &lt;em&gt;isBlue(x)&lt;/em&gt;? Do you want to treat them similarly? If so, you need to represent this somehow. You quickly come to the point where you will need to encode all the world's knowledge in some generalised way. An impossible task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linguists have struggled with this for decades: what is the meaning of a particular sentence? Apart from the fact that every individual human will interpret a given sentence differently (based on their own life experience and culture), there are many aspects to 'meaning' that need representing: the 'factual' meaning, but also pragmatic, evaluative, and all sorts of other nuances. An innocent utterance, &lt;em&gt;That's a nice Apple you've got there&lt;/em&gt;, could have a whole raft of meanings packed into it, all implicit. For example, the person probably likes apples, that one in particular, that apple looks like a tasty piece of fruit, the other person is the owner of the apple, and it might also be a request which is intended to prompt the other person to offer it to you. How are you going to represent all that meaning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One area that interests me personally is representing narrative events. This can &amp;mdash; up to a point &amp;mdash; be done using &lt;a href=&quot;https://en.wikipedia.org/wiki/Conceptual_dependency_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conceptual Dependency&lt;/a&gt;, which uses a limited set of semantic primitives. While useful to encode basic stories, you cannot easily use it to represent the fact that grass is green. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the answer is: there is no answer. AI is too broad a field, and you need to look at a particular application to decide which knowledge is relevant to it, and then how it can best be represented. There is a reason why there are so many ways of representing knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: You suggest this would be &lt;em&gt;more precise&lt;/em&gt;. My personal view is that precision here is a red herring. The word &lt;em&gt;green&lt;/em&gt; is not precise, as it covers a range of wave lengths, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Color_term#Cultural_differences&quot; rel=&quot;nofollow noreferrer&quot;&gt;different people would disagree on whether something is green or not&lt;/a&gt;. So a predicate &lt;em&gt;isGreen(x)&lt;/em&gt; is not any more precise than that. Hence the appeal of &lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic&quot; rel=&quot;nofollow noreferrer&quot;&gt;fuzzy logic&lt;/a&gt;, which allows computation to be based on less precision. &lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-12-01T13:16:24.693" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16883" PostTypeId="1" AcceptedAnswerId="16884" CreationDate="2019-12-01T15:31:44.697" Score="4" ViewCount="67" Body="&lt;p&gt;Suppose you want to predict the price of some stock. Let's say you use the following features.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OpenPrice  &#xA;HighPrice&#xA;LowPrice&#xA;ClosePrice&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it useful to create new features like the following ones?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BodySize = ClosePrice - OpenPrice  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or the size of the tail    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TailUp = HighPrice - Max(OpenPrice, ClosePrice)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or we don't need to do that because we are adding noise and the neural network is going to calculate those values inside?    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case of the body size maybe is a bit different from the tail, because for the tail we need to use a non-linear function (the max operation). So maybe is it important to add the input when it is not a linear relationship between the other inputs not if it's linear?     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example. Consider a box, with height &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, width &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; and length &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;.&lt;br&gt;&#xA;And suppose the real important input is the volume, will the neural network discover that the correlation is &lt;span class=&quot;math-container&quot;&gt;$X * Y * Z$&lt;/span&gt;? Or we need to put the volume as input too?    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry if it's a dumb question but I'm trying to understand what is doing internally the neural network with the inputs, if it's finding (somehow) all the mathematically possible relations between all the inputs or we need to specify the relations between the inputs that we consider important (heuristically) for the problem to solve?&lt;/p&gt;&#xA;" OwnerUserId="31776" LastEditorUserId="2444" LastEditDate="2019-12-02T17:03:17.120" LastActivityDate="2019-12-02T17:03:17.120" Title="Does the neural network calculate different relations between inputs automatically?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16884" PostTypeId="2" ParentId="16883" CreationDate="2019-12-01T16:47:54.820" Score="3" Body="&lt;p&gt;On paper, one expects a complex enough network to determine any complicated function of a limited number of inputs, given a large enough dataset. But in practice, there is no limit to the possible difficulty of the function to be learnt, and the datasets can be relatively small on occasion. In such cases - or arguably in general - it is definitely a good idea to define some combination of the inputs depending on some heuristics as you suggested. If you think some combination of inputs is an important variable by itself, you definitely should include it in your inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can visualize this situation in &lt;a href=&quot;http://playground.tensorflow.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow playground&lt;/a&gt;. Consider the circular pattern dataset on top left corner with some noise. You can use the default setting: &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; as inputs with 2 hidden layers with 4 and 2 neurons respectively. It should learn the pattern in less than 100 epochs. But if you reduce the number of neurons in the second layer to 2, it is not going to get as good as before. So, you are making the model more complicated to get the correct answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can experiment and see that one needs at least one 3 neuron layer to get the correct classification from just &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;. Now, if we examine the dataset, we see the circles so we know that instead of &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;, we can try &lt;span class=&quot;math-container&quot;&gt;$x_1^2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2^2$&lt;/span&gt;. This will learn perfectly without any hidden layers as the function is linear in these parameters. The lesson to be learnt here is that, our prior knowledge of the circle (&lt;span class=&quot;math-container&quot;&gt;$x_1^2 + x_2^2 = r^2$&lt;/span&gt;)  and familiarity with the data helped us in getting a good result with a simpler model (smaller number of neurons), by using derived inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the spiral data at the lower right corner for a more challenging problem. For this one, if you do not use any derived features, it is not likely to give you the correct result, even with several hidden layers. Keep in mind that every extra neuron is a potential source of overfitting, on top of being a computational burden.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the problem here is overly simplified but I expect the situation to be more or less the same for any complicated problem. In practice, we do not have infinite datasets or infinite compute times and the model complexity is always a restriction, so if you have any reason to think some relation between your inputs is relevant for your final result, you definitely should include it by hand at the beginning. &lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-12-01T17:06:13.200" LastActivityDate="2019-12-01T17:06:13.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16887" PostTypeId="2" ParentId="4456" CreationDate="2019-12-01T21:24:59.840" Score="5" Body="&lt;p&gt;Although there are several good answers, I want to add this paragraph from &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;, page 303, for a more psychological view on the difference.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The distinction between model-free and model-based reinforcement learning algorithms&#xA;  corresponds to the distinction psychologists make between habitual and goal-directed&#xA;  control of learned behavioral patterns. Habits are behavior patterns triggered by appropriate&#xA;  stimuli and then performed more-or-less automatically. Goal-directed behavior,&#xA;  according to how psychologists use the phrase, is purposeful in the sense that it is controlled&#xA;  by knowledge of the value of goals and the relationship between actions and their&#xA;  consequences. Habits are sometimes said to be controlled by antecedent stimuli, whereas&#xA;  goal-directed behavior is said to be controlled by its consequences (Dickinson, 1980,&#xA;  1985). Goal-directed control has the advantage that it can rapidly change an animal’s&#xA;  behavior when the environment changes its way of reacting to the animal’s actions. While&#xA;  habitual behavior responds quickly to input from an accustomed environment, it is unable&#xA;  to quickly adjust to changes in the environment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It keeps going from there, and has a nice example afterwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main point that was not always explained in the other answers, is that in a model-free approach you still need some kind of environment to tell you what is the reward associated with your action. The big difference is that you do NOT need to store any information about the model. You give the environment your chosen action, you update your estimated policy, and you forget about it. On the other hand, in model-based approaches, you either need to know the state transitions history as in Dynamic Programming, or you need to be able to calculate all possible next states and associated rewards, from the present state.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-12-01T21:33:49.050" LastActivityDate="2019-12-01T21:33:49.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16888" PostTypeId="2" ParentId="16879" CreationDate="2019-12-01T21:59:00.560" Score="1" Body="&lt;p&gt;There are a few things you can play with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Try reducing the learning rate, or increasing decay.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try using regularization(L1/L2 or dropout)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try using momentum(your model may be stuck in a local minima)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adjust other hyperparams(nodes, layers, batch size, etc.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Unless you have some knowledge about the specific cause of high loss variance, the above steps in some amount should get you where you need to go. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2019-12-01T21:59:00.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16889" PostTypeId="1" CreationDate="2019-12-02T00:42:35.727" Score="2" ViewCount="48" Body="&lt;p&gt;I've had a big interest in machine learning for a while, and I've followed along a few tutorials, but have never made my own project. After losing many games of connect 4 with my friends, I decided to try to make a replica of that, then create a neural network and AI to play against me (Or at least something where I can enter the current board scenario, and it will output which row is the best move). This may be an ambitious first project, but I'm willing to put in the work and research to create something I'm proud of. I created the game using p5.js, and though it may be simple, I'm really happy with how it turned out, as it's one of my first more interesting and unique projects in computer science. Now I don't know a ton about ML, so bear with me. I would like the use pytorch, but I'm open to tensorflow/keras as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few of my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What output do I need to train? My game currently doesn't have a win condition. Would an array or matrix filled with a 0 where there isn't a chip, a 1 where a red chip is, and a 2 for a yellow? Ie&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0,0,0,0&#xA; 1,0,0,0&#xA; 1,0,0,0&#xA; 1,0,2,0&#xA; 1,2,2,2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and enter a 1 somewhere to signify this as a win for player 1? Could an AI recognize this 4 in a row pattern as what needs to be done to win? &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the best way to simulate a lot of games to get my training data? I'm imagining using an RNG to drop chips randomly, export the data output to a file and then enter whether it was a win for p1, p2, or a tie?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any other general words of wisdom or links to read?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks so much for reading this and any help you can offer?&lt;/p&gt;&#xA;" OwnerUserId="31784" LastActivityDate="2019-12-02T00:42:35.727" Title="What form of output would be needed to train a model on a connect 4 AI?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16890" PostTypeId="2" ParentId="16854" CreationDate="2019-12-02T05:06:31.877" Score="2" Body="&lt;p&gt;After a quick scan, it would seem that, in the history of object detection, machine learning has always been at the forefront. Before then, it would just be a heuristic approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a quick answer, here: &lt;a href=&quot;https://towardsdatascience.com/real-time-object-detection-without-machine-learning-5139b399ee7d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/real-time-object-detection-without-machine-learning-5139b399ee7d&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That goes over object detection without using machine learning in OpenCV&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, there are some segments of computer vision that are not strictly machine learning. For example, a commonly used algorithm &lt;a href=&quot;https://www.learnopencv.com/selective-search-for-object-detection-cpp-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Selective Search&lt;/a&gt; for region proposal doesn't use machine learning, and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Teknomo%E2%80%93Fernandez_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF Algorithm&lt;/a&gt; for background generation doesn't either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when it comes to object detection, the gap between machine learning and all other methods is so sheer that ML is the only method really considered today.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-12-02T05:06:31.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16891" PostTypeId="2" ParentId="16883" CreationDate="2019-12-02T06:05:05.057" Score="0" Body="&lt;p&gt;The question is related to &quot;feature extraction&quot;. Firstly, to tackle a &lt;strong&gt;regression problem&lt;/strong&gt; like both the problems stated by you, you need to provide the neural network with the most relevant inputs that have a effect on the output. Eg. If you want your network to add x and y, you need to provide it training examples like  &lt;em&gt;input(x=1, y=3)  and output (sum=4)&lt;/em&gt;. This will make your network do exactly what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But suppose you do not know whether what inputs should you train your network on, neural networks can take care of that too. Look at this &lt;a href=&quot;https://iamtrask.github.io/2015/07/12/basic-python-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;:&#xA;Look at the first truth table. Notice that the output column is actually the first input column and the other two input columns are just random. Eventually, the network learns this relationship and provides the correct results. &lt;strong&gt;What we learnt:&lt;/strong&gt; if you are unsure about which inputs should you choose for your network, just provide as many as possible, or as many combinations as possible. Neural networks excel in finding relationships in input data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, talking of the volume problem, this is what I have been doing recently. It's actually an example of &lt;strong&gt;function approximation&lt;/strong&gt;. Usually, the problem has multiple inputs and a single output (just like the addition problem), but the inverse is also possible. i.e., input : sum and output: x &amp;amp; y. This comes under &lt;strong&gt;one to many function mapping&lt;/strong&gt; and &lt;strong&gt;multivariate regression.&lt;/strong&gt; So YES, you need to provide the volume as input and &lt;strong&gt;x,y and z&lt;/strong&gt; as outputs while training. The recommended configuration is &lt;strong&gt;one neuron in input layer, at least 6 hidden neurons and 3 neurons in output layer&lt;/strong&gt; For magical results, you can use a deeper neural network rather than the shallow one suggested by me. But remember, neural networks have been proved to be **Universal Approximators*&lt;/p&gt;&#xA;" OwnerUserId="26452" LastActivityDate="2019-12-02T06:05:05.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16893" PostTypeId="1" CreationDate="2019-12-02T09:03:45.270" Score="1" ViewCount="34" Body="&lt;p&gt;I was reading about weight pruning in convolutional neural networks. Is it applied for all the layers including convolutional layers or only it is done for dense layers? &lt;/p&gt;&#xA;" OwnerUserId="31795" LastEditorUserId="2444" LastEditDate="2019-12-02T13:32:50.993" LastActivityDate="2019-12-02T13:32:50.993" Title="Is weight pruning applied to all layers or only to dense layers in CNNs?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16895" PostTypeId="1" AcceptedAnswerId="16913" CreationDate="2019-12-02T13:17:35.210" Score="2" ViewCount="174" Body="&lt;p&gt;I'm trying to build neural networks for games like Go, Reversi, Othello, Checkers, or even tic-tac-toe, not by calculating a move, but by making them &lt;strong&gt;evaluate a position&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input is any board situation. The output is a score or estimate for the probability to win, or how favorable a given position is. Where 1 = guaranteed to win and 0 = guaranteed to lose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any given turn I can then loop over all possible moves for the current player, evaluate the resulting game situations, and pick the one with the highest score. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully, by letting this neural network play a trillion games vs itself, it can develop a sensible scoring function resulting in strong play. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: &lt;strong&gt;How do I train such a network&lt;/strong&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In every game, I can keep evaluating and making moves back and forth until one of the AI players wins. In that case the last game situation (right before the winning move) for the winning player should have a target value of 1, and the opposite situation (for the losing player) has a target value of 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I don't intend to make the evaluation network two-sided. I encode the game situation as always &quot;my own&quot; pieces vs &quot;the opponent&quot;, and then evaluate a score from my own (i.e. the current player's) side or perspective. Then after I pick a move, I flip sides so to speak, so the opponent pieces now become my own and vice versa, and then evaluate scores again (now from the other side's perspective) for the next counter move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the input to such a network does explicitly encode black and white pieces, or naughts and crosses (in case of tic-tac-toe) but just my pieces vs their pieces. And then evaluates how favorable the given game situation is for me, always assuming it's my turn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can obviously assign a desired score or truth value for the last move in the game (1 for win, 0 for loss) but how do I backpropagate that towards earlier situations in a played game?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I somehow distribute the 1 or 0 result back a few steps, with a decaying adjustment factor or learning rate? In a game with 40 turns, it might make sense to consider the last few situations as good or bad (being close to winning or losing) but I guess that shouldn't reflect all the way back to the first few moves in the game. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or am I completely mistaken with this approach and is this not how it's supposed to be done?&lt;/p&gt;&#xA;" OwnerUserId="31800" LastActivityDate="2019-12-03T14:52:32.130" Title="Building 'evaluation' neural networks for go, reversi, checkers etc, how to train?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;software-evaluation&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16896" PostTypeId="1" AcceptedAnswerId="16969" CreationDate="2019-12-02T17:22:13.163" Score="1" ViewCount="2180" Body="&lt;p&gt;I am planning to enroll for Andrew Ng's Machine Learning course &lt;a href=&quot;https://www.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.coursera.org/learn/machine-learning&lt;/a&gt;. I've no background in math. Is it OK if I start the course and learn math as and when required?&lt;/p&gt;&#xA;" OwnerUserId="31804" LastActivityDate="2019-12-07T09:35:08.337" Title="Prerequisites for Andrew Ng's Machine Learning Course" Tags="&lt;machine-learning&gt;" AnswerCount="3" CommentCount="1" ClosedDate="2019-12-09T18:07:09.987" ContentLicense="CC BY-SA 4.0" />
  <row Id="16897" PostTypeId="2" ParentId="16896" CreationDate="2019-12-02T19:40:22.370" Score="0" Body="&lt;p&gt;No pre-requisites required for Andrew Ng ML course. There are a couple of lectures in which he gives basic idea of Linear algebra. Also you can learn math when required.&lt;/p&gt;&#xA;" OwnerUserId="31799" LastActivityDate="2019-12-02T19:40:22.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16898" PostTypeId="2" ParentId="16874" CreationDate="2019-12-02T19:47:59.433" Score="2" Body="&lt;p&gt;I think the first question you should answer is: &quot;What questions should the AI be able to answer?&quot; If the intend was that the AI should be able to answer any questions, then that is simply not doable (or at least currently it is not doable). Currently this is similar to asking for a program that can do anything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the AI field is split between statistical approaches and logical approaches. In the early years AI was approached mainly from a logical perspective. Now statistical approaches are more popular. The main advantage of logical approaches is that answers can be explained, while the main advantage of statistical approaches is that given large enough data sets agents can be trained. There is definitely a drive in the AI community to merge statistical and logical approaches to AI, but these approaches are still in its infancy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I therefore will strongly suggest you first determine the kind of problems you will want to address with AI, then based on that, you determine the AI approach that is best suited for those problems.&lt;/p&gt;&#xA;" OwnerUserId="11450" LastEditorUserId="11450" LastEditDate="2019-12-02T19:54:53.183" LastActivityDate="2019-12-02T19:54:53.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16899" PostTypeId="1" CreationDate="2019-12-02T20:01:38.333" Score="8" ViewCount="888" Body="&lt;p&gt;I'm doing a paper for a class on the topic of big problems that are still prevalent in AI, specifically in the area of natural language processing and understanding. From what I understand, the areas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Text classification&lt;/li&gt;&#xA;&lt;li&gt;Entity recognition &lt;/li&gt;&#xA;&lt;li&gt;Translation&lt;/li&gt;&#xA;&lt;li&gt;POS tagging&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;are for the most part solved or perform at a high level currently, but areas such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Text summarization&lt;/li&gt;&#xA;&lt;li&gt;Conversational systems&lt;/li&gt;&#xA;&lt;li&gt;Contextual systems (relying on the previous context that will impact current prediction)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;are still relatively unsolved or are a big area of research (although this could very well change soon with the releases of big transformer models from what I've read).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For people who have experience in the field, what are areas that are still big challenges in NLP and NLU? Why are these areas (doesn't have to be ones I've listed) so tough to figure out?&lt;/p&gt;&#xA;" OwnerUserId="22840" LastEditorUserId="2444" LastEditDate="2020-03-14T18:39:57.680" LastActivityDate="2020-03-14T18:39:57.680" Title="What are the current big challenges in natural language processing and understanding?" Tags="&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16900" PostTypeId="1" AcceptedAnswerId="16903" CreationDate="2019-12-02T20:23:15.410" Score="3" ViewCount="168" Body="&lt;p&gt;I have 50,000 samples. Of these 23,000 belong to the desired class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. I can sacrifice the number of instances that are classified as belonging to the desired class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. It will be enough for me to get 7000 instances in the desired class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, provided that most of these instances classified as belonging to &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; really belong to the desired class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. How can I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is the confusion matrix in the case the instances are perfectly classified.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[23000   0]&#xA; [  0 27000]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it is unlikely to obtain this confusion matrix, so I'm quite satisfied with the following confusion matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[7000   16000]&#xA; [  500 26500]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am currently using the &lt;code&gt;sklearn&lt;/code&gt; library. I mainly use algorithms based on decision trees, as they are quite fast in the calculation.&lt;/p&gt;&#xA;" OwnerUserId="31808" LastEditorUserId="2444" LastEditDate="2019-12-02T23:52:18.793" LastActivityDate="2019-12-02T23:52:18.793" Title="How can I minimise the false positives?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;python&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16901" PostTypeId="1" CreationDate="2019-12-02T20:59:44.750" Score="1" ViewCount="27" Body="&lt;p&gt;I want to build a deep learning algorithm that places emojis in a extended tracts of text. The intent is to choose emojis related to the meaning of the text.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where should I start? I've read that recurrent neural networks are good for applications involving text.&lt;/p&gt;&#xA;" OwnerUserId="31811" LastEditorUserId="1671" LastEditDate="2019-12-02T22:54:04.853" LastActivityDate="2019-12-02T22:54:04.853" Title="Methods for automated insertion of emojis into text?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;getting-started&gt;&lt;reference-request&gt;&lt;software-evaluation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16902" PostTypeId="2" ParentId="5682" CreationDate="2019-12-02T21:22:49.443" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/5683/1671&quot;&gt;Phillipe's excellent answer&lt;/a&gt; covers the crux of the subject, so I'm just going to state the obvious: the key difference is the medium and timescale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Biological evolution is a function of the natural world, and typically occurs over a long time span, depending on the organisms and how quickly they produce new generations.  (We typically think of biological evolution as occurring over &quot;millions of years&quot;, but it can happen much more quickly, for instance in the case of microorganisms.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generic algorithms utilize a computing medium, which in the current era is silicon based, and involves microprocessors and various mediums for memory (magnetic tape and more recently solid-state.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both natural and artificial evolution are constrained by the size of the system (a planet or ecosystem in the former case, and available memory in the latter.)  However:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Artificial evolution can occur at an artificially accelerated pace, dependent on available processing resources.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This capacity for computationally &quot;accelerated subjective time&quot; and accelerated evolution of algorithms is on of the bases for the theory of the &quot;technological singularity&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be argued that genetic engineering allows accelerated evolution for biological species, but that would not fall under natural evolution. &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-12-02T21:37:01.197" LastActivityDate="2019-12-02T21:37:01.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16903" PostTypeId="2" ParentId="16900" CreationDate="2019-12-02T23:43:51.877" Score="3" Body="&lt;p&gt;I think you're looking for the minimization of &lt;em&gt;false positives&lt;/em&gt;, that is, the instances that are classified as belonging to the desired class (the &lt;em&gt;positive&lt;/em&gt; part of &lt;em&gt;false positives&lt;/em&gt;) but that do not actually belong to that class (the &lt;em&gt;false&lt;/em&gt; part of &lt;em&gt;false positives&lt;/em&gt;). In practice, given your constraints, you may want to maximize the &lt;a href=&quot;https://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow noreferrer&quot;&gt;precision&lt;/a&gt;, while maintaining a good &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;recall&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://stackoverflow.com/a/30000619/3924118&quot;&gt;this answer&lt;/a&gt; to the question &lt;a href=&quot;https://stackoverflow.com/q/29994694/3924118&quot;&gt;How can the model be tuned to improve precision, when precision is much more important than recall?&lt;/a&gt;, the user suggests performing a grid search (using &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn.grid_search.GridSearchCV(clf, param_grid, scoring=&quot;precision&quot;)&lt;/code&gt;&lt;/a&gt;) to find the parameters of the model that maximize the precision. See also the question &lt;a href=&quot;https://stats.stackexchange.com/q/140266/82135&quot;&gt;Classifier with adjustable precision vs recall&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2019-12-02T23:43:51.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16905" PostTypeId="1" AcceptedAnswerId="17713" CreationDate="2019-12-03T04:47:34.537" Score="6" ViewCount="598" Body="&lt;p&gt;When the time allotted to &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search#Principle_of_operation&quot; rel=&quot;noreferrer&quot;&gt;Monte Carlo tree search&lt;/a&gt; runs out, what action should be chosen from the root?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.102.1296&quot; rel=&quot;noreferrer&quot;&gt;original UCT paper&lt;/a&gt; (2006) says &lt;code&gt;bestAction&lt;/code&gt; in their algorithm.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.aaai.org/Papers/AIIDE/2008/AIIDE08-036.pdf&quot; rel=&quot;noreferrer&quot;&gt;Monte-Carlo Tree Search: A New Framework for Game AI&lt;/a&gt; (2008) says&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The game action finally executed by the program in the actual game, is the one corresponding to the child which was explored the most.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.incompleteideas.net/609%20dropbox/other%20readings%20and%20resources/MCTS-survey.pdf&quot; rel=&quot;noreferrer&quot;&gt;A Survey of Monte Carlo Tree Search Methods&lt;/a&gt; (2012) says&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The basic algorithm involves iteratively building a search tree until some predefined computational budget – typically a time, memory or iteration constraint – is reached, at which point the search is halted and the best-performing root action returned.&lt;/p&gt;&#xA;&lt;p&gt;[...] The result of the overall search a(BESTCHILD(v0)) is the action a that leads to the best child of the root node v0, where the exact definition of “best” is defined by the implementation.&lt;/p&gt;&#xA;&lt;p&gt;[...] As soon as the search is interrupted or the computation budget is reached, the search terminates and an action a of the root nodet0is selected by some mechanism. Schadd [188] describes four criteria for selecting the winning action, based on the work of Chaslot et al [60]:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Max child&lt;/strong&gt;: Select the root child with the highest reward.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Robust child&lt;/strong&gt;: Select the most visited root child.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Max-Robust child&lt;/strong&gt;: Select the root child with both the highest visit count and the highest reward. If none exist, then continue searching until an acceptable visit count is achieved [70].&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Secure child&lt;/strong&gt;: Select the child which maximises a lower confidence bound.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;[...] Once some computational budget has been reached, the algorithm terminates and returns the best move found,corresponding to the child of the root with the highest visit count.&lt;/p&gt;&#xA;&lt;p&gt;The return value of the overall search in this case is a(BESTCHILD(v0,0)) which will give the action a that leads to the child with the highest reward, since the exploration parameter c is set to 0 for this final call on the root node v0. The algorithm could instead return the action that leads to the most visited child; these two options will usually – but not always! – describe the same action. This potential discrepancy is addressed in the Go program ERICA by continuing the search if the most visited root action is not also the one with the highest reward. This improved ERICA’s winning rate against GNU GO from 47% to 55% [107].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But their algorithm 2 uses the same criterion as the internal-node selection policy:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\operatorname{argmax}_{v'} \frac{Q(v')}{N(v')} + c \sqrt{\frac{2 \ln N(v)}{N(v')}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is neither the max child nor the robust child! This situation is quite confusing, and I'm wondering which approach is nowadays considered most successful/appropriate.&lt;/p&gt;&#xA;" OwnerUserId="3373" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-01-28T13:35:18.367" Title="MCTS: How to choose the final action from the root" Tags="&lt;algorithm&gt;&lt;monte-carlo-tree-search&gt;&lt;monte-carlo-methods&gt;&lt;planning&gt;&lt;tree-search&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16906" PostTypeId="1" AcceptedAnswerId="16907" CreationDate="2019-12-03T09:02:03.547" Score="2" ViewCount="46" Body="&lt;p&gt;Let's suppose we have to train a neural network for the XOR classification task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the inputs &lt;span class=&quot;math-container&quot;&gt;$(00, 01, 10, 11)$&lt;/span&gt; inserted in a sequential way? For example, we first insert the 00 and change the weights, then the 01 and again slightly change them, etc. Or is there another way it can be implemented?&lt;/p&gt;&#xA;" OwnerUserId="31817" LastEditorUserId="2444" LastEditDate="2019-12-03T16:03:10.293" LastActivityDate="2019-12-03T16:03:10.293" Title="How are the inputs passed to the neural network during training for the XOR classification task?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16907" PostTypeId="2" ParentId="16906" CreationDate="2019-12-03T09:22:47.290" Score="1" Body="&lt;p&gt;Hello and welcome to the community. There are multiple ways you can train a neural network: &lt;strong&gt;stochastic, mini-batch and batch&lt;/strong&gt;. What you explained is the stochastic mode, where you input one training example 01 for example, calculate the gradients and update the networks weights before the next training example is fed. You could also select multiple such examples (a mini-batch) and update the weights only after you computed all the outputs (for this particular mini-batch). Finally you can use a batch size which is equal with the total number of examples in your dataset so you will update the weights only after you have all the outputs for all samples. Each of those methods have their own strengths and weaknesses, depending on your dataset you might prefer one over the others.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-12-03T09:22:47.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16908" PostTypeId="1" CreationDate="2019-12-03T10:32:05.153" Score="1" ViewCount="42" Body="&lt;p&gt;There are a lot of ML algorithms suggested for fraud detection. Now, I have not been able to find a general overview for all of them. My goal is to create this overview. What algorithms would you suggest and why? &lt;/p&gt;&#xA;" OwnerUserId="31820" LastActivityDate="2019-12-03T10:32:05.153" Title="What ML algorithms would you suggest in fraud detection?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16909" PostTypeId="1" AcceptedAnswerId="16940" CreationDate="2019-12-03T10:50:41.380" Score="3" ViewCount="95" Body="&lt;p&gt;I was going through the proof of the policy gradient theorem here: &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html#svpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html#svpg&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the section &quot;Proof of Policy Gradient Theorem&quot; in the block of equations just under the sentence &quot;The nice rewriting above allows us to exclude the derivative of Q-value function...&quot; they set&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\eta (s) = \sum^\infty_{k=0} \rho^\pi(s_0 \rightarrow s, k)&#xA;$$&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_s \eta (s) = const&#xA;$$&lt;/span&gt;&#xA;Thus, they basically assume, that the stationary distribution is not dependent on the initial state. But how can we justify this? If the MDP is described by a block diagonal transition matrix, in my mind this should not hold.&lt;/p&gt;&#xA;" OwnerUserId="31821" LastEditorUserId="2444" LastEditDate="2019-12-04T20:24:06.667" LastActivityDate="2019-12-04T22:32:52.290" Title="Why is the stationary distribution independent of the initial state in the proof of the policy gradient theorem?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16910" PostTypeId="1" AcceptedAnswerId="16912" CreationDate="2019-12-03T11:38:26.217" Score="6" ViewCount="771" Body="&lt;p&gt;I'm taking a &lt;a href=&quot;https://www.coursera.org/learn/fundamentals-of-reinforcement-learning&quot; rel=&quot;noreferrer&quot;&gt;Coursera course on Reinforcement learning&lt;/a&gt;. There was a question there that wasn't addressed in the learning material: Does adding a constant to all rewards change the set of optimal policies in episodic tasks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is Yes - Adding a constant to the reward signal can make longer episodes more or less advantageous (depending on whether the constant is positive or negative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain why is this so? And why it doesn't change in the case of continuous (non episodic) tasks? I don't see why adding a constant matters - as an optimal policy would still want to get the maximum reward...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give an example of this?&lt;/p&gt;&#xA;" OwnerUserId="27947" LastActivityDate="2019-12-03T13:41:59.080" Title="Does adding a constant to all rewards change the set of optimal policies in episodic tasks?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16912" PostTypeId="2" ParentId="16910" CreationDate="2019-12-03T13:41:59.080" Score="10" Body="&lt;p&gt;Generally we can write for &lt;span class=&quot;math-container&quot;&gt;$R_c$&lt;/span&gt; the total reward with added constant &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; of a policy as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_c = \sum_{i=0}^K (r_i + c) \gamma^i = \sum_{i=0}^K r_i \gamma^i + \sum_{i=0}^K c \gamma^i&#xA;$$&lt;/span&gt;&#xA;So if we have two policies with the same total reward (without added constant)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{i=0}^{K_1} r_i^1 \gamma^i = \sum_{i=0}^{K_2} r_i^2 \gamma^i&#xA;$$&lt;/span&gt;&#xA;but with different lengths &lt;span class=&quot;math-container&quot;&gt;$K_1 \neq K_2$&lt;/span&gt; the total reward with added constant will be different, because the second term in &lt;span class=&quot;math-container&quot;&gt;$R_c$&lt;/span&gt; ( &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=0}^K c \gamma^i$&lt;/span&gt; ) will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example: Consider two optimal policies, both generating the same cumulative reward of 10, but the first policy visits 4 states, before it reaches a terminal state, while the second visits only two states. The rewards can be written as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;10 + 0 + 0 + 0 = 10&#xA;$$&lt;/span&gt;&#xA;and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;0 + 10 = 10&#xA;$$&lt;/span&gt;&#xA;But when we add 100 to every reward:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;110 + 100 + 100 + 100 = 410&#xA;$$&lt;/span&gt;&#xA;and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;100 + 110 = 210&#xA;$$&lt;/span&gt;&#xA;Thus, now the first one is better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the continious case, the episodes always have length &lt;span class=&quot;math-container&quot;&gt;$K = \infty$&lt;/span&gt;. Therefore, they always have the same length, and adding a constant doesnt change anything, because the second term in &lt;span class=&quot;math-container&quot;&gt;$R_c$&lt;/span&gt; stays the same.&lt;/p&gt;&#xA;" OwnerUserId="31821" LastActivityDate="2019-12-03T13:41:59.080" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16913" PostTypeId="2" ParentId="16895" CreationDate="2019-12-03T14:52:32.130" Score="1" Body="&lt;p&gt;The evaluation of the last steps in the game can be made with the 1 and 0 as you said. For all the other steps, the evaluation should be the evaluation of the best next step with a small decay. &lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-12-03T14:52:32.130" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16914" PostTypeId="1" AcceptedAnswerId="16925" CreationDate="2019-12-03T23:30:52.757" Score="2" ViewCount="87" Body="&lt;p&gt;During model training, I noticed various behaviour in between training and validation accuracy. I understand that &lt;a href=&quot;https://stackoverflow.com/a/51345413/11901732&quot;&gt;'The training set is used to train the model, while the validation set is only used to evaluate the model's performance...'&lt;/a&gt;, but I'd like to know if there is any relationship between training and validation accuracy and if yes, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) what exactly is happening when training and validation accuracy change during training and;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) what do different behaviours imply&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, some believe there is overfitting problem if training &gt; validation accuracy. What happens if one is greater than the other &lt;strong&gt;alternately&lt;/strong&gt;, which is the case below? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inputs_1 = keras.Input(shape=(10081,1))&#xA;&#xA;layer1 = Conv1D(64,14)(inputs_1)&#xA;layer2 = layers.MaxPool1D(5)(layer1)&#xA;layer3 = Conv1D(64, 14)(layer2)&#xA;layer4 = layers.GlobalMaxPooling1D()(layer3)&#xA;&#xA;inputs_2 = keras.Input(shape=(104,))             &#xA;layer5 = layers.concatenate([layer4, inputs_2])&#xA;layer6 = Dense(128, activation='relu')(layer5)&#xA;layer7 = Dense(2, activation='softmax')(layer6)&#xA;&#xA;&#xA;model_2 = keras.models.Model(inputs = [inputs_1, inputs_2], output = [layer7])&#xA;model_2.summary()&#xA;&#xA;&#xA;X_train, X_test, y_train, y_test = train_test_split(df.iloc[:,0:10185], df[['Result_cat','Result_cat1']].values, test_size=0.2) &#xA;X_train = X_train.to_numpy()&#xA;X_train = X_train.reshape([X_train.shape[0], X_train.shape[1], 1]) &#xA;X_train_1 = X_train[:,0:10081,:]&#xA;X_train_2 = X_train[:,10081:10185,:].reshape(736,104)   &#xA;&#xA;&#xA;X_test = X_test.to_numpy()&#xA;X_test = X_test.reshape([X_test.shape[0], X_test.shape[1], 1]) &#xA;X_test_1 = X_test[:,0:10081,:]&#xA;X_test_2 = X_test[:,10081:10185,:].reshape(185,104)    &#xA;&#xA;adam = keras.optimizers.Adam(lr = 0.0005)&#xA;model_2.compile(loss = 'categorical_crossentropy', optimizer = adam, metrics = ['acc'])&#xA;&#xA;history = model_2.fit([X_train_1,X_train_2], y_train, epochs = 120, batch_size = 256, validation_split = 0.2, callbacks = [keras.callbacks.EarlyStopping(monitor='val_loss', patience=20)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;model summary&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:15: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=[&amp;lt;tf.Tenso..., outputs=[&amp;lt;tf.Tenso...)`&#xA;  from ipykernel import kernelapp as app&#xA;Model: &quot;model_3&quot;&#xA;__________________________________________________________________________________________________&#xA;Layer (type)                    Output Shape         Param #     Connected to                     &#xA;==================================================================================================&#xA;input_5 (InputLayer)            (None, 10081, 1)     0                                            &#xA;__________________________________________________________________________________________________&#xA;conv1d_5 (Conv1D)               (None, 10068, 64)    960         input_5[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;max_pooling1d_3 (MaxPooling1D)  (None, 2013, 64)     0           conv1d_5[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv1d_6 (Conv1D)               (None, 2000, 64)     57408       max_pooling1d_3[0][0]            &#xA;__________________________________________________________________________________________________&#xA;global_max_pooling1d_3 (GlobalM (None, 64)           0           conv1d_6[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;input_6 (InputLayer)            (None, 104)          0                                            &#xA;__________________________________________________________________________________________________&#xA;concatenate_3 (Concatenate)     (None, 168)          0           global_max_pooling1d_3[0][0]     &#xA;                                                                 input_6[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;dense_5 (Dense)                 (None, 128)          21632       concatenate_3[0][0]              &#xA;__________________________________________________________________________________________________&#xA;dense_6 (Dense)                 (None, 2)            258         dense_5[0][0]                    &#xA;==================================================================================================&#xA;Total params: 80,258&#xA;Trainable params: 80,258&#xA;Non-trainable params: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the training process&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__________________________________________________________________________________________________&#xA;Train on 588 samples, validate on 148 samples&#xA;Epoch 1/120&#xA;588/588 [==============================] - 16s 26ms/step - loss: 5.6355 - acc: 0.4932 - val_loss: 4.1086 - val_acc: 0.6216&#xA;Epoch 2/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 4.5977 - acc: 0.5748 - val_loss: 3.8252 - val_acc: 0.4459&#xA;Epoch 3/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 4.3815 - acc: 0.4575 - val_loss: 2.4087 - val_acc: 0.6622&#xA;Epoch 4/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.7480 - acc: 0.6003 - val_loss: 2.0060 - val_acc: 0.6892&#xA;Epoch 5/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.3019 - acc: 0.5408 - val_loss: 2.3176 - val_acc: 0.5676&#xA;Epoch 6/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.1739 - acc: 0.5663 - val_loss: 2.2607 - val_acc: 0.6892&#xA;Epoch 7/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.2322 - acc: 0.6207 - val_loss: 1.8898 - val_acc: 0.7230&#xA;Epoch 8/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.9777 - acc: 0.6020 - val_loss: 1.8401 - val_acc: 0.7500&#xA;Epoch 9/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.8982 - acc: 0.6429 - val_loss: 1.8517 - val_acc: 0.7365&#xA;Epoch 10/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.8342 - acc: 0.6344 - val_loss: 1.7941 - val_acc: 0.7095&#xA;Epoch 11/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7426 - acc: 0.6327 - val_loss: 1.8495 - val_acc: 0.7162&#xA;Epoch 12/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7340 - acc: 0.6531 - val_loss: 1.7652 - val_acc: 0.7162&#xA;Epoch 13/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6680 - acc: 0.6616 - val_loss: 1.8097 - val_acc: 0.7365&#xA;Epoch 14/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6922 - acc: 0.6786 - val_loss: 1.7143 - val_acc: 0.7500&#xA;Epoch 15/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6161 - acc: 0.6786 - val_loss: 1.6960 - val_acc: 0.7568&#xA;Epoch 16/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6054 - acc: 0.6905 - val_loss: 1.6779 - val_acc: 0.7297&#xA;Epoch 17/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6072 - acc: 0.6684 - val_loss: 1.6750 - val_acc: 0.7703&#xA;Epoch 18/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5907 - acc: 0.6871 - val_loss: 1.6774 - val_acc: 0.7432&#xA;Epoch 19/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5854 - acc: 0.6718 - val_loss: 1.6609 - val_acc: 0.7770&#xA;Epoch 20/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5621 - acc: 0.6905 - val_loss: 1.6709 - val_acc: 0.7365&#xA;Epoch 21/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5515 - acc: 0.6854 - val_loss: 1.6904 - val_acc: 0.7703&#xA;Epoch 22/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5749 - acc: 0.6837 - val_loss: 1.6862 - val_acc: 0.7297&#xA;Epoch 23/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6212 - acc: 0.6514 - val_loss: 1.7215 - val_acc: 0.7568&#xA;Epoch 24/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6532 - acc: 0.6633 - val_loss: 1.7105 - val_acc: 0.7230&#xA;Epoch 25/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7300 - acc: 0.6344 - val_loss: 1.6870 - val_acc: 0.7432&#xA;Epoch 26/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7355 - acc: 0.6650 - val_loss: 1.6733 - val_acc: 0.7703&#xA;Epoch 27/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6336 - acc: 0.6650 - val_loss: 1.6572 - val_acc: 0.7297&#xA;Epoch 28/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6018 - acc: 0.6803 - val_loss: 1.7292 - val_acc: 0.7635&#xA;Epoch 29/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5448 - acc: 0.7143 - val_loss: 1.8065 - val_acc: 0.7095&#xA;Epoch 30/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5724 - acc: 0.6820 - val_loss: 1.8029 - val_acc: 0.7297&#xA;Epoch 31/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6622 - acc: 0.6650 - val_loss: 1.6594 - val_acc: 0.7568&#xA;Epoch 32/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6211 - acc: 0.6582 - val_loss: 1.6375 - val_acc: 0.7770&#xA;Epoch 33/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5911 - acc: 0.6854 - val_loss: 1.6964 - val_acc: 0.7500&#xA;Epoch 34/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5050 - acc: 0.7262 - val_loss: 1.8496 - val_acc: 0.6892&#xA;Epoch 35/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6012 - acc: 0.6752 - val_loss: 1.7443 - val_acc: 0.7432&#xA;Epoch 36/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5688 - acc: 0.6871 - val_loss: 1.6220 - val_acc: 0.7568&#xA;Epoch 37/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4843 - acc: 0.7279 - val_loss: 1.6166 - val_acc: 0.7905&#xA;Epoch 38/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4707 - acc: 0.7449 - val_loss: 1.6496 - val_acc: 0.7905&#xA;Epoch 39/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4683 - acc: 0.7109 - val_loss: 1.6641 - val_acc: 0.7432&#xA;Epoch 40/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4671 - acc: 0.7279 - val_loss: 1.6553 - val_acc: 0.7703&#xA;Epoch 41/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4479 - acc: 0.7347 - val_loss: 1.6302 - val_acc: 0.7973&#xA;Epoch 42/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4355 - acc: 0.7551 - val_loss: 1.6241 - val_acc: 0.7973&#xA;Epoch 43/120&#xA;588/588 [==============================] - 14s 25ms/step - loss: 2.4286 - acc: 0.7568 - val_loss: 1.6249 - val_acc: 0.7973&#xA;Epoch 44/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4250 - acc: 0.7585 - val_loss: 1.6248 - val_acc: 0.7770&#xA;Epoch 45/120&#xA;588/588 [==============================] - 14s 25ms/step - loss: 2.4198 - acc: 0.7517 - val_loss: 1.6212 - val_acc: 0.7703&#xA;Epoch 46/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4246 - acc: 0.7568 - val_loss: 1.6129 - val_acc: 0.7838&#xA;Epoch 47/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4237 - acc: 0.7517 - val_loss: 1.6166 - val_acc: 0.7973&#xA;Epoch 48/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4287 - acc: 0.7432 - val_loss: 1.6309 - val_acc: 0.8041&#xA;Epoch 49/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4179 - acc: 0.7381 - val_loss: 1.6271 - val_acc: 0.7838&#xA;Epoch 50/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4164 - acc: 0.7381 - val_loss: 1.6258 - val_acc: 0.7973&#xA;Epoch 51/120&#xA;588/588 [==============================] - 14s 24ms/step - loss: 2.1996 - acc: 0.7398 - val_loss: 1.3612 - val_acc: 0.7973&#xA;Epoch 52/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1387 - acc: 0.8265 - val_loss: 1.4811 - val_acc: 0.7973&#xA;Epoch 53/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1607 - acc: 0.8078 - val_loss: 1.5060 - val_acc: 0.7838&#xA;Epoch 54/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1783 - acc: 0.8129 - val_loss: 1.4878 - val_acc: 0.8176&#xA;Epoch 55/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1745 - acc: 0.8197 - val_loss: 1.4762 - val_acc: 0.8108&#xA;Epoch 56/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1764 - acc: 0.8129 - val_loss: 1.4631 - val_acc: 0.7905&#xA;Epoch 57/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1637 - acc: 0.8078 - val_loss: 1.4615 - val_acc: 0.7770&#xA;Epoch 58/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1563 - acc: 0.8112 - val_loss: 1.4487 - val_acc: 0.7703&#xA;Epoch 59/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1396 - acc: 0.8146 - val_loss: 1.4362 - val_acc: 0.7905&#xA;Epoch 60/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1240 - acc: 0.8316 - val_loss: 1.4333 - val_acc: 0.8041&#xA;Epoch 61/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1173 - acc: 0.8333 - val_loss: 1.4369 - val_acc: 0.8041&#xA;Epoch 62/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1228 - acc: 0.8384 - val_loss: 1.4393 - val_acc: 0.8041&#xA;Epoch 63/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1113 - acc: 0.8316 - val_loss: 1.4380 - val_acc: 0.8041&#xA;Epoch 64/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1102 - acc: 0.8452 - val_loss: 1.4217 - val_acc: 0.8041&#xA;Epoch 65/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0961 - acc: 0.8469 - val_loss: 1.4129 - val_acc: 0.7973&#xA;Epoch 66/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0903 - acc: 0.8537 - val_loss: 1.4019 - val_acc: 0.8041&#xA;Epoch 67/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0890 - acc: 0.8503 - val_loss: 1.3850 - val_acc: 0.8176&#xA;Epoch 68/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0878 - acc: 0.8520 - val_loss: 1.4035 - val_acc: 0.7635&#xA;Epoch 69/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0984 - acc: 0.8469 - val_loss: 1.4060 - val_acc: 0.8041&#xA;Epoch 70/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0893 - acc: 0.8418 - val_loss: 1.3981 - val_acc: 0.7973&#xA;Epoch 71/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0876 - acc: 0.8605 - val_loss: 1.3951 - val_acc: 0.8041__________________________________________________________________________________________________&#xA;Train on 588 samples, validate on 148 samples&#xA;Epoch 1/120&#xA;588/588 [==============================] - 16s 26ms/step - loss: 5.6355 - acc: 0.4932 - val_loss: 4.1086 - val_acc: 0.6216&#xA;Epoch 2/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 4.5977 - acc: 0.5748 - val_loss: 3.8252 - val_acc: 0.4459&#xA;Epoch 3/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 4.3815 - acc: 0.4575 - val_loss: 2.4087 - val_acc: 0.6622&#xA;Epoch 4/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.7480 - acc: 0.6003 - val_loss: 2.0060 - val_acc: 0.6892&#xA;Epoch 5/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.3019 - acc: 0.5408 - val_loss: 2.3176 - val_acc: 0.5676&#xA;Epoch 6/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.1739 - acc: 0.5663 - val_loss: 2.2607 - val_acc: 0.6892&#xA;Epoch 7/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 3.2322 - acc: 0.6207 - val_loss: 1.8898 - val_acc: 0.7230&#xA;Epoch 8/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.9777 - acc: 0.6020 - val_loss: 1.8401 - val_acc: 0.7500&#xA;Epoch 9/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.8982 - acc: 0.6429 - val_loss: 1.8517 - val_acc: 0.7365&#xA;Epoch 10/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.8342 - acc: 0.6344 - val_loss: 1.7941 - val_acc: 0.7095&#xA;Epoch 11/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7426 - acc: 0.6327 - val_loss: 1.8495 - val_acc: 0.7162&#xA;Epoch 12/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7340 - acc: 0.6531 - val_loss: 1.7652 - val_acc: 0.7162&#xA;Epoch 13/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6680 - acc: 0.6616 - val_loss: 1.8097 - val_acc: 0.7365&#xA;Epoch 14/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6922 - acc: 0.6786 - val_loss: 1.7143 - val_acc: 0.7500&#xA;Epoch 15/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6161 - acc: 0.6786 - val_loss: 1.6960 - val_acc: 0.7568&#xA;Epoch 16/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6054 - acc: 0.6905 - val_loss: 1.6779 - val_acc: 0.7297&#xA;Epoch 17/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6072 - acc: 0.6684 - val_loss: 1.6750 - val_acc: 0.7703&#xA;Epoch 18/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5907 - acc: 0.6871 - val_loss: 1.6774 - val_acc: 0.7432&#xA;Epoch 19/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5854 - acc: 0.6718 - val_loss: 1.6609 - val_acc: 0.7770&#xA;Epoch 20/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5621 - acc: 0.6905 - val_loss: 1.6709 - val_acc: 0.7365&#xA;Epoch 21/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5515 - acc: 0.6854 - val_loss: 1.6904 - val_acc: 0.7703&#xA;Epoch 22/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5749 - acc: 0.6837 - val_loss: 1.6862 - val_acc: 0.7297&#xA;Epoch 23/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6212 - acc: 0.6514 - val_loss: 1.7215 - val_acc: 0.7568&#xA;Epoch 24/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6532 - acc: 0.6633 - val_loss: 1.7105 - val_acc: 0.7230&#xA;Epoch 25/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7300 - acc: 0.6344 - val_loss: 1.6870 - val_acc: 0.7432&#xA;Epoch 26/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.7355 - acc: 0.6650 - val_loss: 1.6733 - val_acc: 0.7703&#xA;Epoch 27/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6336 - acc: 0.6650 - val_loss: 1.6572 - val_acc: 0.7297&#xA;Epoch 28/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6018 - acc: 0.6803 - val_loss: 1.7292 - val_acc: 0.7635&#xA;Epoch 29/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5448 - acc: 0.7143 - val_loss: 1.8065 - val_acc: 0.7095&#xA;Epoch 30/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5724 - acc: 0.6820 - val_loss: 1.8029 - val_acc: 0.7297&#xA;Epoch 31/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6622 - acc: 0.6650 - val_loss: 1.6594 - val_acc: 0.7568&#xA;Epoch 32/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6211 - acc: 0.6582 - val_loss: 1.6375 - val_acc: 0.7770&#xA;Epoch 33/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5911 - acc: 0.6854 - val_loss: 1.6964 - val_acc: 0.7500&#xA;Epoch 34/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5050 - acc: 0.7262 - val_loss: 1.8496 - val_acc: 0.6892&#xA;Epoch 35/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.6012 - acc: 0.6752 - val_loss: 1.7443 - val_acc: 0.7432&#xA;Epoch 36/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.5688 - acc: 0.6871 - val_loss: 1.6220 - val_acc: 0.7568&#xA;Epoch 37/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4843 - acc: 0.7279 - val_loss: 1.6166 - val_acc: 0.7905&#xA;Epoch 38/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4707 - acc: 0.7449 - val_loss: 1.6496 - val_acc: 0.7905&#xA;Epoch 39/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4683 - acc: 0.7109 - val_loss: 1.6641 - val_acc: 0.7432&#xA;Epoch 40/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4671 - acc: 0.7279 - val_loss: 1.6553 - val_acc: 0.7703&#xA;Epoch 41/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4479 - acc: 0.7347 - val_loss: 1.6302 - val_acc: 0.7973&#xA;Epoch 42/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4355 - acc: 0.7551 - val_loss: 1.6241 - val_acc: 0.7973&#xA;Epoch 43/120&#xA;588/588 [==============================] - 14s 25ms/step - loss: 2.4286 - acc: 0.7568 - val_loss: 1.6249 - val_acc: 0.7973&#xA;Epoch 44/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4250 - acc: 0.7585 - val_loss: 1.6248 - val_acc: 0.7770&#xA;Epoch 45/120&#xA;588/588 [==============================] - 14s 25ms/step - loss: 2.4198 - acc: 0.7517 - val_loss: 1.6212 - val_acc: 0.7703&#xA;Epoch 46/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4246 - acc: 0.7568 - val_loss: 1.6129 - val_acc: 0.7838&#xA;Epoch 47/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4237 - acc: 0.7517 - val_loss: 1.6166 - val_acc: 0.7973&#xA;Epoch 48/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4287 - acc: 0.7432 - val_loss: 1.6309 - val_acc: 0.8041&#xA;Epoch 49/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4179 - acc: 0.7381 - val_loss: 1.6271 - val_acc: 0.7838&#xA;Epoch 50/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 2.4164 - acc: 0.7381 - val_loss: 1.6258 - val_acc: 0.7973&#xA;Epoch 51/120&#xA;588/588 [==============================] - 14s 24ms/step - loss: 2.1996 - acc: 0.7398 - val_loss: 1.3612 - val_acc: 0.7973&#xA;Epoch 52/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1387 - acc: 0.8265 - val_loss: 1.4811 - val_acc: 0.7973&#xA;Epoch 53/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1607 - acc: 0.8078 - val_loss: 1.5060 - val_acc: 0.7838&#xA;Epoch 54/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1783 - acc: 0.8129 - val_loss: 1.4878 - val_acc: 0.8176&#xA;Epoch 55/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1745 - acc: 0.8197 - val_loss: 1.4762 - val_acc: 0.8108&#xA;Epoch 56/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1764 - acc: 0.8129 - val_loss: 1.4631 - val_acc: 0.7905&#xA;Epoch 57/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1637 - acc: 0.8078 - val_loss: 1.4615 - val_acc: 0.7770&#xA;Epoch 58/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1563 - acc: 0.8112 - val_loss: 1.4487 - val_acc: 0.7703&#xA;Epoch 59/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1396 - acc: 0.8146 - val_loss: 1.4362 - val_acc: 0.7905&#xA;Epoch 60/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1240 - acc: 0.8316 - val_loss: 1.4333 - val_acc: 0.8041&#xA;Epoch 61/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1173 - acc: 0.8333 - val_loss: 1.4369 - val_acc: 0.8041&#xA;Epoch 62/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1228 - acc: 0.8384 - val_loss: 1.4393 - val_acc: 0.8041&#xA;Epoch 63/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1113 - acc: 0.8316 - val_loss: 1.4380 - val_acc: 0.8041&#xA;Epoch 64/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.1102 - acc: 0.8452 - val_loss: 1.4217 - val_acc: 0.8041&#xA;Epoch 65/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0961 - acc: 0.8469 - val_loss: 1.4129 - val_acc: 0.7973&#xA;Epoch 66/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0903 - acc: 0.8537 - val_loss: 1.4019 - val_acc: 0.8041&#xA;Epoch 67/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0890 - acc: 0.8503 - val_loss: 1.3850 - val_acc: 0.8176&#xA;Epoch 68/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0878 - acc: 0.8520 - val_loss: 1.4035 - val_acc: 0.7635&#xA;Epoch 69/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0984 - acc: 0.8469 - val_loss: 1.4060 - val_acc: 0.8041&#xA;Epoch 70/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0893 - acc: 0.8418 - val_loss: 1.3981 - val_acc: 0.7973&#xA;Epoch 71/120&#xA;588/588 [==============================] - 15s 25ms/step - loss: 1.0876 - acc: 0.8605 - val_loss: 1.3951 - val_acc: 0.8041&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice how at first &lt;code&gt;acc&lt;/code&gt; is lower than &lt;code&gt;val_acc&lt;/code&gt; and later is greater than &lt;code&gt;val_acc&lt;/code&gt;. Can someone please shed some light what could be happening here? Thank you&lt;/p&gt;&#xA;" OwnerUserId="31075" LastEditorUserId="31075" LastEditDate="2019-12-03T23:42:07.893" LastActivityDate="2019-12-04T13:39:06.160" Title="Relationship between training accuracy and validation accuracy" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16915" PostTypeId="1" CreationDate="2019-12-03T23:34:19.327" Score="1" ViewCount="300" Body="&lt;p&gt;Is there a way to train a neural network to detect subliminal messages? Where can I find the dataset on which to train the neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have to create the dataset, how would I go about it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;United Nations has defined subliminal messages as perceiving messages without being aware of them, it is unconscious perception, or perception without awareness. Like you may be aware of a message but cannot consciously perceive that message in the form of text, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two many types of subliminal messages, one which can be made through visual means, another which can be made through audio. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In visual means, I'm referring to these types:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Messages which are flashed for very short while on the screen. &lt;/li&gt;&#xA;&lt;li&gt;Messages whose opacity is changed to blend with the background.&lt;/li&gt;&#xA;&lt;li&gt;Messages whose colors are varied slightly to blend with the background. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Example of 3rd type of subliminal messages: if there is a background of red, on this can be shown a message made up of slight variation of red, as a conscious mind can't distinguish between such close shades of red, people will take the entire thing to be red block, but a subconscious mind notice the slight variation in color, register the message, because humans can see millions of colors. &lt;/p&gt;&#xA;" OwnerUserId="31307" LastEditorUserId="2444" LastEditDate="2020-06-15T01:16:32.507" LastActivityDate="2021-03-12T04:09:00.910" Title="How can I train a neural network to detect subliminal messages?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16916" PostTypeId="1" CreationDate="2019-12-03T23:36:00.533" Score="1" ViewCount="80" Body="&lt;p&gt;I've made a connect 4 game in javascript, and I want to design an AI for it. I made a post the other day about what output would be needed, and I think I could use images of the board and a CNN. I did some research into Reinforcement learning, and I think that's what I need to do. I don't have much experience with ML in general, much less RL with Q-learning, but that is what I'd like to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I don't really know how to start out with such a big project. I have a few questions first:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What do I do with my input? I'm thinking I give the AI 7 options for moves to make, one for each row of the board. How do I implement a way that it can &quot;look&quot; at the board? Can I just import an image of the current board state?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I make a reward table? How should I do the points system for the Q-learning? I'm thinking something like: If it drops a chip it gets a point, if it lines up 2 chips in a row it gets 5 points, if it gets 3 in a row it gets 30, and if it gets 4 in a row it gets 100. Would that be an effective way to do this? How do I implement this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a library I can use to do any of the work where I make an algorithm and board states and reward tables? Or do I have to hard code any of it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've done some research, are there any links or tutorials you think I should read or follow along with? Any other general advice or help for me?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I greatly appreciate anyone who answers one or all of these questions! Thank you so much!&lt;/p&gt;&#xA;" OwnerUserId="31784" LastActivityDate="2019-12-04T14:10:17.663" Title="Designing a reinforcement learning AI for a game of connect 4" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2020-09-13T19:26:03.117" ContentLicense="CC BY-SA 4.0" />
  <row Id="16917" PostTypeId="1" CreationDate="2019-12-04T01:40:37.387" Score="2" ViewCount="185" Body="&lt;p&gt;I'm a student beginning to study deep learning, and would like to practice with a simple project using a Graph Convolutional Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not quite sure how to handle different input sizes of graphs for the GCN. How would I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is zero-padding the only way to solve this problem? While zero-padding is applicable to CNNs, I'm not sure if it is for a GCN.&lt;/p&gt;&#xA;" OwnerUserId="31836" LastEditorUserId="2444" LastEditDate="2019-12-04T14:09:54.620" LastActivityDate="2019-12-04T14:09:54.620" Title="How should I handle different input sizes in graph convolution networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16918" PostTypeId="1" CreationDate="2019-12-03T11:07:03.500" Score="2" ViewCount="43" Body="&lt;p&gt;Considering I have some neural network that, using supervised learning, transforms a string into a learned feature vector where &quot;close&quot; strings will result into more close vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that since a NN is no one-way-function there is a way to retrieve the input data from my output if I have the entire network at hand (if I know biases, weights, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is if the network is not known, is there a way for me (using ie. some probabilistic distributions) to make assumptions or even reconstruct the input data?&lt;/p&gt;&#xA;" OwnerUserId="31885" OwnerDisplayName="Jonathan R" LastEditorUserId="31885" LastEditDate="2019-12-05T15:01:15.733" LastActivityDate="2019-12-05T15:01:15.733" Title="Can learned feature vectors be considered a good encryption?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="16920" PostTypeId="1" CreationDate="2019-12-04T05:06:35.627" Score="2" ViewCount="36" Body="&lt;p&gt;Is logistic regression used for unconstrained or constrained optimization problems, and why?&lt;/p&gt;&#xA;" OwnerUserId="31840" LastEditorUserId="2444" LastEditDate="2019-12-04T14:50:33.820" LastActivityDate="2019-12-04T14:50:33.820" Title="Is logistic regression used for unconstrained or constrained optimisation problems?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16921" PostTypeId="1" AcceptedAnswerId="16924" CreationDate="2019-12-04T05:40:53.867" Score="2" ViewCount="118" Body="&lt;p&gt;i just started the book and i was wondering , what will i be able to do in AI by the end of the book ?&#xA;and more particularly, what is my position with Reinforcement Learning, deep neural networks and NEAT algorithm, will i study them enough in the book &lt;/p&gt;&#xA;&#xA;&lt;p&gt;will i need to read another more specialized book, if so what would you recommend &lt;/p&gt;&#xA;" OwnerUserId="31842" LastActivityDate="2019-12-04T13:55:25.093" Title="what will i be able to do in the end of AI: modern approach?" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;deep-neural-networks&gt;&lt;neat&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2019-12-05T22:34:40.710" ContentLicense="CC BY-SA 4.0" />
  <row Id="16922" PostTypeId="1" CreationDate="2019-12-04T06:38:46.483" Score="1" ViewCount="309" Body="&lt;p&gt;The &lt;a href=&quot;https://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RPN&lt;/a&gt; loss in &lt;a href=&quot;https://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faster RCNN&lt;/a&gt; paper is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L({p_i}, {t_i}) = \frac{1}{N_{cls}} \sum_{i} L_{cls}(p_i,p_i^*) + \lambda \frac{1}{N_{reg}} \sum_i p_i^* L_{reg}(t_i, t_i^*)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For regression problems, we have the following parametrization&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$t_x=\frac{x - x_a}{w_a}, \\ t_y=\frac{y−y_a}{h_a}, \\ t_w= \log \left( \frac{w}{w_a} \right),\\ t_h= \log \left(\frac{h}{h_a} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the ground-truth labels are&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$t_x^*=\frac{x^* - x_a}{w_a},\\ t_y^*=\frac{y^*−y_a}{h_a}, \\ t_w^*= \log \left( \frac{w^*}{w_a} \right), \\ t_h^*= \log \left(\frac{h^*}{h_a} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are the two coordinates of the center, &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; the width, and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the height of the &lt;em&gt;predicted box&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are the two coordinates of the center, &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; the width, and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the height of the &lt;em&gt;anchor box&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L_{reg}(t_i, t_i^*) = R(t_i − t_i^*)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is a robust loss function (smooth &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These equations are unclear to me, so here are my two questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I get the predicted bounding box given the neural network's output?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What exactly is the smooth &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; here? How is it defined?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31844" LastEditorUserId="2444" LastEditDate="2020-12-28T15:31:18.197" LastActivityDate="2021-05-27T16:07:10.527" Title="In Faster R-CNN, how can I get the predicted bounding box given the neural network's output?" Tags="&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;object-detection&gt;&lt;r-cnn&gt;&lt;faster-r-cnn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16923" PostTypeId="2" ParentId="16921" CreationDate="2019-12-04T06:49:48.943" Score="0" Body="&lt;p&gt;Well, the content of the book has been more or less updated between the 1st and the second edition but very few changes has been made between the 2nd and the third. Therefore you can roughly consider that the content of the book is from a 2003 perspective. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This being said to answer specifically to what you asked, I just checked the index of the one I have (3rd edition): &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;NEAT algorithm is not mentioned but it is not surprising as the algorithm was invented in 2002 but the general principles of genetic algorithms are covered. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Regarding deep learning, the book provides some ground knowledge about what neural networks are and how they work but the content will be far from the state of the art implementations and NN typology.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31374" LastEditorUserId="31374" LastEditDate="2019-12-04T10:44:56.423" LastActivityDate="2019-12-04T10:44:56.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16924" PostTypeId="2" ParentId="16921" CreationDate="2019-12-04T08:37:25.953" Score="3" Body="&lt;p&gt;&quot;Artificial Intelligence: A Modern Approach&quot; (AIMA) by Russel and Norvig is a general introductory book on AI. That means it not only covers sub-symbolic AI (like machine learning) but also symbolic AI. Therefore, it can &quot;only&quot; give you an overview of each topic (I put only in quotation marks since it is actually quite ambitious to cover all topics of AI in a single book!). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the topics you have mentioned that means:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you will have read about the basics of neural networks but will not have seen much about deep nets (the 3rd edition only has 10 pages on neural networks, i.e. it's really basic)&lt;/li&gt;&#xA;&lt;li&gt;you will have read the basics of reinforcement learning (the 3rd edition has 23 pages on this, i.e. it's basic but more comprehensive than neural networks and enough to read papers about the topic) &lt;/li&gt;&#xA;&lt;li&gt;you will not have read anything about NEAT but have read about the basics of genetic algorithms and neural networks which NEAT is based on (and can read the corresponding papers afterwards)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, regarding your preferred topics, AIMA is one way to get started if you're not only looking for a selectively deep but also a broad understanding of AI. The road could go like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Read a general AI book, e.g. AIMA&lt;/li&gt;&#xA;&lt;li&gt;Read a general ML book, e.g. &quot;The Elements of Statistical Learning&quot; by Hastie, Tibshirani, Friedman&lt;/li&gt;&#xA;&lt;li&gt;Read deep dive books on selected topics, e.g. &quot;Deep Learning&quot; by Goodfellow, Bengio, Courville&lt;/li&gt;&#xA;&lt;li&gt;Read papers for your topics of interest, e.g. &quot;Mastering the Game of Go without Human Knowledge&quot; by Silver et al. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The quicker start would be of course to start at step 2 or 3 and only go to previous steps to look something up if it is required.&lt;/p&gt;&#xA;" OwnerUserId="30789" LastEditorUserId="2444" LastEditDate="2019-12-04T13:55:25.093" LastActivityDate="2019-12-04T13:55:25.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16925" PostTypeId="2" ParentId="16914" CreationDate="2019-12-04T10:18:26.693" Score="0" Body="&lt;p&gt;very interesting questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) what exactly is happening when training and validation accuracy change during training&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;code&gt;accuracy&lt;/code&gt; change after every batch computation. You have 588 batches, so loss will be computed after each one of these batches (let's say each batch have 8 images). However, the accuracy you see in the progress bar it is the accuracy of the current batch &lt;em&gt;averaged&lt;/em&gt; with the accuracy of all the previous batches so far. See &lt;code&gt;keras.utils.generic_utils.Progbar&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;val_acc&lt;/code&gt; is computed only at the end of one epoch and it is computed with all your validation dataset at once (considering it as a single batch, so if you have 100 images for validation it will compute accuracy as a single batch of 100 images)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) what do different behaviours imply&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;code&gt;acc&lt;/code&gt; and &lt;code&gt;val_acc&lt;/code&gt; normally differ from each other due to different split sizes. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Try same experiment with &lt;code&gt;validation_split=0.01&lt;/code&gt; and &lt;code&gt;validation_split=0.4&lt;/code&gt; and you will see how both &lt;code&gt;accuracy&lt;/code&gt; and &lt;code&gt;val_acc&lt;/code&gt; will change. &lt;/li&gt;&#xA;&lt;li&gt;Normally the greater the validation split, the more similar both metrics will be since the validation split will be big enough to be representative (let's say it has cats and dogs, not only cats), taking into account that you need enough data to train correctly. This explains why in some cases the &lt;code&gt;val_acc&lt;/code&gt; is higher than &lt;code&gt;accuracy&lt;/code&gt; and vice versa.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Overfitting only occurs when the graph fashion or tendency changes and &lt;code&gt;val_acc&lt;/code&gt; starts to drop and &lt;code&gt;accuracy&lt;/code&gt; keeping increasing. This means that your model can not do any better with the validation dataset (non previously seen images).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I work with &lt;code&gt;loss&lt;/code&gt; and &lt;code&gt;val_loss&lt;/code&gt; which are highly correlated with &lt;code&gt;accuracy&lt;/code&gt;. Normally the loss is the inverse, so interpret the comments above in the inverse sense (sorry about the confusion but I'm taking this example from my current experiments) I hope it helps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zvku1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zvku1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 experiments, orange and grey. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In both experiments, &lt;code&gt;val_loss&lt;/code&gt; is always slightly higher than &lt;code&gt;loss&lt;/code&gt; (because of my current validation split which it happens to be also 0.2, but normally is 0.01 and val_loss is even higher).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On both experiments the &lt;code&gt;loss&lt;/code&gt; trend is linearly decreasing, this is because gradient descent works and the loss functions is well defined and it converges.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Orange experiment is overfitting from epoch 20 onwards because, the &lt;code&gt;val_loss&lt;/code&gt; won't drop any more and, on the contrary, it start increasing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Grey experiment is just right, both &lt;code&gt;loss&lt;/code&gt; and &lt;code&gt;val_loss&lt;/code&gt; are still decreasing, and although the &lt;code&gt;val_loss&lt;/code&gt; might be greater than &lt;code&gt;loss&lt;/code&gt; it is not overfitting because it is still decreasing. So that is why it is still training :) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Complex concepts here, I hope I was able to explain myself clearly! Cheers&lt;/p&gt;&#xA;" OwnerUserId="26882" LastEditorUserId="26882" LastEditDate="2019-12-04T13:39:06.160" LastActivityDate="2019-12-04T13:39:06.160" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16926" PostTypeId="1" AcceptedAnswerId="16929" CreationDate="2019-12-04T10:26:07.680" Score="2" ViewCount="402" Body="&lt;p&gt;If I want to augment my dataset, is shuffling or permuting the channels (RGB) of an image a sensible augmentation for training a CNN? IIRC, the way convolutions work is that a kernel operates over parts of the image but maintains the order of the kernels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the kernel has &lt;span class=&quot;math-container&quot;&gt;$k \times k$&lt;/span&gt; weights for each channel and the resulting output is the multiplication of the weights and the pixel values of the image and is finally averaged to form a new pixel in the next feature map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, if we shuffle the channels of the image (GBR, BGR, RBG, GRB, etc.), a CNN that is only trained on the ordering RGB would do poorly on such images. Therefore, is it not sensible to shuffle the channels of the image as a form of data augmentation? Or will this have a regularizing effect on the CNN model?&lt;/p&gt;&#xA;" OwnerUserId="31851" LastActivityDate="2019-12-04T13:06:20.990" Title="Can I shuffle image channel data as a form of data augmentation?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16927" PostTypeId="1" CreationDate="2019-12-04T12:53:43.830" Score="0" ViewCount="216" Body="&lt;p&gt;I was just wondering if it's possible to use Machine Learning to train a model from a dataset of images of cups with a given volume in the image and then use object detection to detect other cups and assume the volume of the cup,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically the end goal is to detect the volume of a cup using object detection with a phone's camera,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would highly appreciate it if someone can point me to the right direction.&lt;/p&gt;&#xA;" OwnerUserId="16071" LastActivityDate="2019-12-04T13:54:29.220" Title="Is it possible to use AI for detecting the volume of a cup" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16928" PostTypeId="2" ParentId="16899" CreationDate="2019-12-04T13:05:48.363" Score="2" Body="&lt;p&gt;According to a nice article by Sebastian Ruder &lt;a href=&quot;https://ruder.io/4-biggest-open-problems-in-nlp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ruder.io/4-biggest-open-problems-in-nlp/&lt;/a&gt; based on answers from top NLP researchers &lt;a href=&quot;https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Natural language understanding&lt;/li&gt;&#xA;&lt;li&gt;NLP for low-resource scenarios&lt;/li&gt;&#xA;&lt;li&gt;Reasoning about large or multiple documents&lt;/li&gt;&#xA;&lt;li&gt;Datasets, problems, and evaluation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I recommend having a look at the article. More details in the slides &lt;a href=&quot;https://drive.google.com/file/d/15ehMIJ7wY9A7RSmyJPNmrBMuC7se0PMP/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/15ehMIJ7wY9A7RSmyJPNmrBMuC7se0PMP/view&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27851" LastActivityDate="2019-12-04T13:05:48.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16929" PostTypeId="2" ParentId="16926" CreationDate="2019-12-04T13:06:20.990" Score="1" Body="&lt;p&gt;As a rule of thumb for image data augmentation, look at the augmented images:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Can you correctly classify or measure your target label from the augmented images?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Could something similar to the augmented images appear in the environment where you want to run inferences on previously unseen inputs?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For your suggested augmentation of shuffling the channels, it may pass the first test. However, the second test shows that you are probably taking a step too far.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;will this have a regularizing effect on the CNN model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but it might not be that useful to have strong cross-channel regularisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is important information for your task in the separate colour channels, then shuffling the channels makes it harder for the neural network to use that (it is not impossible, the CNN can still learn filters that will trigger most strongly on features that tend to appear in red channel and not blue in your problem for instance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is not important information for your task in the colour information, then you may find it simpler and easier to turn your images into single channel greyscale instead, and use that throughout. Although that is not completely the same, for many image types it will achieve a similar effect (and possible boost to accuracy) for a fraction of the effort.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-12-04T13:06:20.990" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16930" PostTypeId="1" CreationDate="2019-12-04T13:51:13.570" Score="1" ViewCount="33" Body="&lt;p&gt;Are there free and easy-to-use annotation tools for 3D bounding boxes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a lot of annotation tools out there, but most of them only good for 2d boxes. &lt;/p&gt;&#xA;" OwnerUserId="23049" LastEditorUserId="2444" LastEditDate="2019-12-04T14:41:47.700" LastActivityDate="2019-12-04T14:41:47.700" Title="Are there free and easy-to-use annotation tools for 3D bounding boxes?" Tags="&lt;computer-vision&gt;&lt;datasets&gt;&lt;resource-request&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2021-02-03T16:46:52.100" ContentLicense="CC BY-SA 4.0" />
  <row Id="16931" PostTypeId="2" ParentId="16927" CreationDate="2019-12-04T13:54:29.220" Score="2" Body="&lt;p&gt;This could be possible, providing you have the right dataset to train it on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The volume of a cup consist of width, height and depth. You can probably detect all three of those given the bounding box or the pixels of the cup. However detecting the dimensions of an object require a reference object, like a penny or your finger and you have to specify the exact dimension of that. If that method fits your problem which I assume not, this is a good resource to look through: &lt;a href=&quot;https://www.pyimagesearch.com/2016/03/28/measuring-size-of-objects-in-an-image-with-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open CV measuring dimensions&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However to do the task, you need a way to measure the volume without the reference object. To fo this you need a deep learning based approach. &lt;a href=&quot;https://github.com/iro-cp/FCRN-DepthPrediction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Depth detection using fully residual convolutional neural network&lt;/a&gt; maybe a good start for your project as you are doing something similar. You may use one of the pretrained model and apply transfer learning on it with the image of the cup and the surroundings, and get the output and feed it through another feed forward neural network to predict the volume.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-12-04T13:54:29.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16932" PostTypeId="2" ParentId="16916" CreationDate="2019-12-04T14:10:17.663" Score="1" Body="&lt;p&gt;1) your input should be so that you describe your entire environment. This could be done by 8 (length)* 8 (height)* 3 (either empty space, opponent chip or your chip) = 192 input neurons. you can just import a image of the current boardstate (which is width pixels * height pixels input neurons), but this means you task the neural network with learning to read the image as wel as playing the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) rewards for a game like this are (most of the time) 1 for a winning move and 0 for a losing move. if you do it your way it might have side effects like prioritizing lining 2 chips up instead of going for a winning move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Tensorflow is most commenly used. (this is python and not javascript though)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) I liked the book Reinforcement learning, an introduction and would recommand this to anyone.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-12-04T14:10:17.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16933" PostTypeId="1" CreationDate="2019-12-04T15:00:31.360" Score="3" ViewCount="42" Body="&lt;p&gt;I'm having a problem to understand the needed dimensions of an VAE, especially for mu, logvar and z layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have an input of 512x512, 1 color channel (CT images), batch size 32. &#xA;Then my encoder/decoder looks like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.encoder = nn.Sequential(&#xA;            nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),  # 32x512x512&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x256x256&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x128x128&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x64x64&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x32x32&#xA;            nn.ReLU(True))&#xA;&#xA;self.decoder = nn.Sequential(&#xA;            nn.ConvTranspose2d(32, 32, kernel_size=4, stride=2, padding=1),&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(32, 32, kernel_size=4, stride=2, padding=1),&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(32, 32, kernel_size=4, stride=2, padding=1),&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(32, 32, kernel_size=3, stride=1, padding=1),&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(32, 1, kernel_size=4, stride=2, padding=1),&#xA;            nn.Sigmoid())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the correct dimension of mu/logvar and z? latent_dim = 1000, filter_depth=32&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if the input of the linear layer mu/logvar is right or not ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mu = nn.Linear(self.filter_depth * 32 * 32, self.latent_dim)&#xA;logvar = nn.Linear(self.filter_depth * 32 * 32, self.latent_dim)&#xA;z = nn.Linear(self.latent_dim, self.filter_depth * 32 * 32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31857" LastEditorUserId="2444" LastEditDate="2019-12-04T15:04:29.760" LastActivityDate="2019-12-04T15:04:29.760" Title="In variational auto-encoders, what should the dimension of the mean and variance of the latent distribution and the latent vector be?" Tags="&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16936" PostTypeId="1" CreationDate="2019-12-04T15:49:54.333" Score="2" ViewCount="25" Body="&lt;p&gt;I am given a 2-dimensional picture (black &amp;amp; white, white background) and it is assumed that there are some 'sticks' (basically 'thick lines' with different widths and lengths) that are (mostly) overlapping with one another.&#xA;I want to somehow recognize these sticks (where they are and how big they are).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any approach you would recommend or, even better, anything that already exists? I am working with MatLab, but a general (theoretical) approach would be also fine! I am open to machine-learning, but I'd prefer classical algorithms here.&lt;/p&gt;&#xA;" OwnerUserId="31859" LastEditorUserId="2444" LastEditDate="2019-12-04T15:59:34.913" LastActivityDate="2019-12-04T15:59:34.913" Title="How can I recognise possibly overlapping line segments in 2D?" Tags="&lt;pattern-recognition&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16938" PostTypeId="1" CreationDate="2019-12-04T19:00:46.440" Score="1" ViewCount="22" Body="&lt;p&gt;There are some backgroundsubtractor functions in opencv like backgroundsubtractormog2 , backgroundsubtractorGMG and ... . It seems that these functions only detect moving objects in a video. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I understand from the concept of these functions, that they do some clustering in an image.&#xA;Do these functions only detect moving objects? Why? Or am I wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated&lt;/p&gt;&#xA;" OwnerUserId="30170" LastActivityDate="2019-12-04T19:00:46.440" Title="Do backgroundSubtractor functions in opencv only detect moving objects?" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16939" PostTypeId="1" CreationDate="2019-12-04T19:34:49.327" Score="3" ViewCount="96" Body="&lt;p&gt;I have heard of ensemble methods, such as XGBoost, for binary or categorical machine learning models. However, does this exist for regression? If so, how are the weights for each model in the process of predictions determined? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to do this manually, as I was planning on training two different models using separate frameworks (YoloV3 aka Darknet and Tensorflow for bounding box regression). Is there a way I can establish a weight for each model in the overall prediction for these boxes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is this a bad idea?&lt;/p&gt;&#xA;" OwnerUserId="23119" LastEditorUserId="2444" LastEditDate="2019-12-04T20:22:16.417" LastActivityDate="2020-10-01T18:25:37.563" Title="Are there ensemble methods for regression?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;computer-vision&gt;&lt;ensemble-learning&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16940" PostTypeId="2" ParentId="16909" CreationDate="2019-12-04T22:32:52.290" Score="1" Body="&lt;p&gt;I think your doubt is completely reasonable. Probably there is an additional assumption that they (both Lilian Weng and &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rich Sutton&lt;/a&gt; (pag.269)) do not make explicit in the proof and that is that your MDP is not only stationary, but also &lt;strong&gt;ergodic&lt;/strong&gt;. A particular property of those systems is that the probability of eventually reaching a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; from a starting point &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; is 1. In such a case it is clear that &lt;span class=&quot;math-container&quot;&gt;$\eta(s)$&lt;/span&gt; exists and is independent of any &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; chosen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, an MDP with block-diagonal transition matrix does not satisfy such an assumption since the starting point completely restricts those states you can reach in an infinite time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I do not understand is why Rich Sutton does mention ergodicity as a necessary condition in the case of a &quot;continuing task&quot;, as opposed to &quot;episodic tasks&quot; (pag.275). For me, their proof requires this condition in both cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an additional note, I also think that Lilian Weng does not really explain why we should buy that from the initial reasonable definition &lt;span class=&quot;math-container&quot;&gt;$J(\theta)=\sum_\mathcal S d^{\pi_\theta}(s)V^{\pi_\theta}(s)$&lt;/span&gt; we should accept the much simpler one &lt;span class=&quot;math-container&quot;&gt;$J(\theta)=V^{\pi_\theta}(s_0)$&lt;/span&gt;. I guess the only reason is that the gradient of the initial expression does require to know the gradient of &lt;span class=&quot;math-container&quot;&gt;$d^{\pi_\theta}(s)$&lt;/span&gt; and so you would be accepting the approximation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_\theta J(\theta)=\nabla_\theta\left(\sum_\mathcal S d^{\pi_\theta}(s)V^{\pi_\theta}(s)\right)\approx\sum_\mathcal S d^{\pi_\theta}(s)\nabla_\theta V^{\pi_\theta}(s),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the last term is just &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta V^{\pi_\theta}(s_0)$&lt;/span&gt; under the ergodicity assumption. &lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2019-12-04T22:32:52.290" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16941" PostTypeId="2" ParentId="16294" CreationDate="2019-12-04T23:45:48.713" Score="1" Body="&lt;p&gt;Apparently there is an example of non-convergence for semi-gradient sarsa, &lt;a href=&quot;https://media.nips.cc/Conferences/2015/tutorialslides/SuttonIntroRL-nips-2015-tutorial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;according to Rich Sutton&lt;/a&gt; (check slide 35). I guess TD(0) is not so different. So, probably your approximator will need to satisfy certain conditions to proof convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe &lt;a href=&quot;https://openreview.net/pdf?id=SJezGp4YPr&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; will be useful for you. It seems that they show that constraining your network to have relu activation functions allow you to show some convergence properties. &lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2019-12-04T23:45:48.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16943" PostTypeId="1" CreationDate="2019-12-05T03:25:32.900" Score="1" ViewCount="311" Body="&lt;p&gt;I would like to use OpenAI Gym to solve a &lt;em&gt;continuing&lt;/em&gt; environment, that is, a problem with a single, never-ending episode (please note I don't mean a &lt;em&gt;continuous&lt;/em&gt; environment with continuous state and actions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only continuing environment I found in their repository was the classic &lt;a href=&quot;https://gym.openai.com/envs/Pendulum-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverted pendulum problem&lt;/a&gt;, and I found no baseline methods (algorithms) that don't require episodic environments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;are there any continuing environments other than the inverted pendulum one?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;is there an OpenAI Gym baseline method that I can use to solve the inverted pendulum problem as well as other continuing environments?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30679" LastActivityDate="2019-12-09T03:15:31.753" Title="Are there OpenAI Gym continuing environments (other than inverted pendulum) and baselines?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16945" PostTypeId="2" ParentId="16939" CreationDate="2019-12-05T07:17:58.010" Score="0" Body="&lt;p&gt;There's similar boosting classes in XGBoost for regression. You can implement their built-in classes for your problem, rather than implementing from scratch. You can read more about it from their website.&#xA;You can also take a look at catboost, which implements a different approach. &lt;/p&gt;&#xA;" OwnerUserId="31873" LastActivityDate="2019-12-05T07:17:58.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16946" PostTypeId="1" AcceptedAnswerId="16947" CreationDate="2019-12-05T08:18:38.927" Score="1" ViewCount="108" Body="&lt;p&gt;How can I know what each neuron does in NN?&#xA;Consider the &lt;a href=&quot;https://playground.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playground from Tensorflow&lt;/a&gt;, there are some hidden layers with some neurons in each. Each of them shows a line(horizontal or vertical or ...). Where these shapes come from. I think they are understandable for nn not a person!&lt;/p&gt;&#xA;" OwnerUserId="31143" LastActivityDate="2019-12-06T06:19:10.787" Title="How can I find what does an specific neuron do in neural network?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16947" PostTypeId="2" ParentId="16946" CreationDate="2019-12-05T08:51:18.117" Score="3" Body="&lt;p&gt;In TensorFlow Playground, the horizontal line show where each class is separated for each neuron. What happens when you take any intermediate neuron to make the decision? You can see the answer by the line provided by that neuron. And this decision is a result of the weighted sum from the decisions of the previous neurons (up to activation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the middle-top neuron in the link you share, which is an almost horizontal line - slightly tilted to the right. This neuron classifies everything above it as a blue, and everything below it as an orange. Hover over the neuron to see a larger picture on the output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also see how this is actually calculated by hovering over the line coming from the neurons in the previous layer to the neuron you are looking at. For the case of the same neuron (center-top), the weight coming from the first input (&lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;) is 0.091, while from the second one (&lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;), it is 0.49. The neuron ends up being almost horizontal because the contribution from the horizontal input (&lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;) is so much larger compared to the vertical one (&lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you need to take into account the nonlinearity coming from the activation function but the idea presented above is the essence of it. The example uses tanh activation, which behaves very linear in its intermediate region so one can ignore this issue to some extend for this particular case.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: It appears that the values for the weights change at every browser session, so the neuron I describe might look a little different to you. To get the same configuration, simply click on the colored lines between neurons to edit them and use the values above for the connections. &lt;/p&gt;&#xA;" OwnerUserId="22301" LastEditorUserId="22301" LastEditDate="2019-12-06T06:19:10.787" LastActivityDate="2019-12-06T06:19:10.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16948" PostTypeId="1" CreationDate="2019-12-05T12:09:01.710" Score="2" ViewCount="53" Body="&lt;p&gt;I am currently working on a problem and now got stuck to implement one of it's steps. This is a simple attempt to explain what I am currently facing, which is something that I am aiming to implement in my python simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that I will some input parameters into my simulation, however one simulation is not able to perfectly capture all the dynamics involved in a real scenario. Hence, what I am aiming to do is to feed some inputs of the real scenario into my simulate and perform the simulation for all cases in which I have real data. So I will have the same amount of data for technically the same situation for both real and simulated scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With my simulated data I can find out the optimal parameters(for the simulation), so the idea now is to correlate my simulated model with the real data, and then, with this correlation, find out what would be the equivalent of the optimal simulated parameters into the the optimal real parameters. Here not really precise diagram that might help on the visualization of the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7rw7y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7rw7y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already seen a lot of machine learning being utilized to fit to a set of data, but haven't really seen anything that could help me on this task that I currently have in hand, as &quot;fitting models&quot;. So here comes the questions, how to correlate the models and utilized it to extract the optimal parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that I managed to be succinct and precise albeit the length of the text. I would really appreciate your help on this one!&lt;/p&gt;&#xA;" OwnerUserId="31880" LastActivityDate="2019-12-05T12:09:01.710" Title="Correlating two models to predict the output of one that corresponds to an output of the other" Tags="&lt;machine-learning&gt;&lt;models&gt;&lt;model-based-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16950" PostTypeId="1" CreationDate="2019-12-05T13:38:03.293" Score="1" ViewCount="47" Body="&lt;p&gt;I am building a Convolution neural network to predict certain categories based on images (the location of a pointer on a surface) . However in many cases there will be no pointer in the view or something that is not the pointer. Initially I was just going to train it with outputs of the different classifications including the null classification. However given that the null classification is far more common than the others (perhaps 1000 times more likely) would it be better to have a separate null classifier and then if this outputted non null then the second classifier would be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="31882" LastEditorUserId="31882" LastEditDate="2019-12-05T15:16:16.800" LastActivityDate="2019-12-05T20:38:09.160" Title="structure of neural network for classification problems with large amounts of null classifications" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16951" PostTypeId="1" AcceptedAnswerId="16954" CreationDate="2019-12-05T19:23:35.307" Score="4" ViewCount="384" Body="&lt;p&gt;There is this nice result for policy gradients that the gradient of some performance measure, &lt;span class=&quot;math-container&quot;&gt;$\nabla v_{\pi_{\theta}}(s_0)$&lt;/span&gt; (here, in the episodic case for the starting state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; and policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, parametrised by some weights &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) is equal to the expectation gradient of the logarithm of the policy, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla v_{\pi_{\theta}}(s_0)=\mathbb{E}\Big{[}\sum_{t=0}^{T-1}\nabla_\theta\log(\pi_{\theta}(a_t|s_t)]\cdot G_t\Big{]},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; is the discounted future reward from state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; onward and &lt;span class=&quot;math-container&quot;&gt;$s_T$&lt;/span&gt; the final state of some trajectory &lt;span class=&quot;math-container&quot;&gt;$(s_0, a_0, s_1, a_1, ..., s_{T-1}, a_{T-1}, s_T)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, when using a softmax policy, &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta\log(\pi_{\theta}(a_t|s_t)$&lt;/span&gt; can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_\theta\log(\pi_{\theta}(a_t|s_t))=\phi(s_t,a_t)-\mathbb{E}[\phi(s_t,\cdot)],$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\phi(s,a)$&lt;/span&gt; is some input vector of a state-action tuple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However: &lt;strong&gt;what exactly is this vector?&lt;/strong&gt; A typical input with policy gradients (for example in a neural network) is a feature vector for the state and the output a vector with dimensions equal to the number of actions, e.g. &lt;span class=&quot;math-container&quot;&gt;$(14, 15, 11, 17)^T$&lt;/span&gt; for four possible actions. The softmax-function now scales these outputs, which results in the logits &lt;span class=&quot;math-container&quot;&gt;$(.042, .114, .002, .842)^T$&lt;/span&gt; in this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would usually do in neural networks is take some input vector, for example something that describes if there are borders in a grid world, e.g. &lt;span class=&quot;math-container&quot;&gt;$\phi(s)=(1, 0, 0, 1)^T$&lt;/span&gt;, and multiply that with my weight matrix &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (and add biases b), i.e. &lt;span class=&quot;math-container&quot;&gt;$\theta\phi(s)+b$&lt;/span&gt;. So, continuing above example, &lt;span class=&quot;math-container&quot;&gt;$1\cdot \theta_{1,1} + 0\cdot \theta_{1,2} + 0\cdot \theta_{1,3} + 1\cdot \theta_{1,4} = 14$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1\cdot \theta_{2,1} + 0\cdot \theta_{2,2} + 0\cdot \theta_{2,3} + 1\cdot \theta_{2,4} = 15$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what is &lt;span class=&quot;math-container&quot;&gt;$\phi(s,a)$&lt;/span&gt; here? And how would I compute &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta\log(\pi_{\theta}(a|s))=\phi(s,a)-\mathbb{E}[\phi(s,\cdot)]$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="22161" LastEditorUserId="2444" LastEditDate="2020-12-04T18:31:16.160" LastActivityDate="2020-12-04T18:31:16.160" Title="Eligibility vector for softmax policy with policy gradients" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;policy-gradients&gt;&lt;softmax-policy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16952" PostTypeId="1" CreationDate="2019-12-05T20:15:34.567" Score="1" ViewCount="39" Body="&lt;p&gt;I have a dataset with different types of numerical values (both negative and positive numerical values) for the inputs (for example, -40, -35, 1, 25, 39, etc., that is, multiple inputs) and single output numerical value (either negative or positive).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to use linear regression, but I haven't been so successful and I think one of the reasons is negative values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to deal with this scenario? What model should I use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Keras for my AI model.&lt;/p&gt;&#xA;" OwnerUserId="31544" LastEditorUserId="2444" LastEditDate="2019-12-06T14:57:04.587" LastActivityDate="2019-12-06T14:57:04.587" Title="How to perform regression with multiple numeric (positive and negative) inputs and one numeric output?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16953" PostTypeId="2" ParentId="16950" CreationDate="2019-12-05T20:38:09.160" Score="1" Body="&lt;p&gt;I see multiple reasons to take a different route:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;While in &quot;classical&quot; pattern recognition you might have done things like feature engineering outside of your model, one idea of deep learning was to &quot;insource&quot; it into the model and let the model take care of &quot;everything&quot;. Following that, I have seen a general tendency in deep net architectures to let your deep net handle all the work in one single model. So your idea kind of contradicts the architectural mainstream.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are so many deep net architectures being published which are well engineered and tested for all kinds of tasks. Therefore, I would check if there is anything &quot;ready off the shelf&quot; for a task like yours in the very first place. If there is, then it will probably be better than your self-engineered model architecture. And if there is not, you might want to check related tasks and see how they solved this problem, e.g. medical applications where specific cells need to be detected with most examples being negative (and if there is a positive example the area in a picture might still be mostly negative as it is just a single positive cell among many others).  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your thought of feeding the &quot;present/not present&quot; binary classification to your task of locating an object is very closely related to what has been discussed as auxiliary tasks. I don't know if it has been applied to exactly your problem but there are similar applications. For example there is an application where &quot;name recognition&quot; is complemented by the auxiliary task of deciding whether any name is present in a given sentence or not. Which is somewhat similar to your case. The paper &lt;a href=&quot;https://arxiv.org/pdf/1706.05098.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Overview of Multi-Task Learning in Deep Neural Networks&lt;/a&gt; provides an overview. A famous example using auxiliary tasks would be &lt;a href=&quot;https://arxiv.org/pdf/1409.4842.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GoogleNet&lt;/a&gt;. This also goes back to my first point of letting the deep net handle &quot;everything&quot; internally.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, as a disclaimer, this is a theoretical perspective as I cannot speak from experience regarding your problem.&lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2019-12-05T20:38:09.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16954" PostTypeId="2" ParentId="16951" CreationDate="2019-12-05T21:27:55.147" Score="5" Body="&lt;p&gt;Calculation of gradient&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \log(\pi_{\theta}(a|s)) &amp;amp;= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\&#xA;&amp;amp;= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a')&#xA;\end{align}&lt;/span&gt;&#xA;is only valid for linear function approximation with action preferences of form&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;h(s, a, \theta) = \theta^T \phi(s,a)&#xA;\end{equation}&lt;/span&gt;&#xA;and softmax policy&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\pi(a|s) = \frac{e^{h(s,a,\theta)}}{\sum_{a'} e^{h(s,a',\theta)}}&#xA;\end{equation}&lt;/span&gt;&#xA;The gradient would be calculated as it is written. For example, if your current state is &lt;span class=&quot;math-container&quot;&gt;$s = (1, 1)$&lt;/span&gt; and in that state you have actions &lt;span class=&quot;math-container&quot;&gt;$a_0 = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_1 = 1$&lt;/span&gt; and probabilities for those actions are &lt;span class=&quot;math-container&quot;&gt;$\pi(a_0|s) = 0.7$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi(a_1|s) = 0.3$&lt;/span&gt; then gradient for action &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt; would be&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\nabla_{\theta} \log(\pi_{\theta}(a_0|s)) = (1, 1, 0)^T - (0.7 \cdot (1, 1, 0)^T + 0.3\cdot (1,1,1)^T) = (0, 0, -0.3)^T&#xA;\end{equation}&lt;/span&gt;&#xA;Feature vector &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; can be basically anything you want. For example you could stack state feature and action (like I did in small example), you could use polynomials, radial basis functions, tile coding, etc.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're using multilayered neural network you would have to propagate gradients through all layers, usually done with backpropagation algorithm. Easiest way is to use automatic differentiation software (e.g. Tensorflow) which can do that for you so you don't have to write your implementation. All you have to do is define your objective function that you want to optimize&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;J_\theta = \sum_t \log(\pi(a_t|s_t, \theta)) G_t&#xA;\end{equation}&lt;/span&gt;&#xA;and software will automatically calculate gradient &lt;span class=&quot;math-container&quot;&gt;$\nabla J_{\theta}$&lt;/span&gt; and update weights.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2019-12-05T22:20:03.893" LastActivityDate="2019-12-05T22:20:03.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16955" PostTypeId="1" CreationDate="2019-12-05T22:08:48.660" Score="1" ViewCount="112" Body="&lt;p&gt;I am looking for a dataset of clear audio, a corresponding transcript (optional), and most importantly a list of all the &lt;a href=&quot;https://www.merriam-webster.com/dictionary/phoneme&quot; rel=&quot;nofollow noreferrer&quot;&gt;phonemes&lt;/a&gt; said in the audio, with the length of each phoneme and a mention of each phoneme start and end time. I looked through stuff like gentle forced aligner, however, the problem is that there is a small offset that will be very bad to use for my project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify, my project is a convolutional neural network that can classify a small spectrogram into a phoneme.&lt;/p&gt;&#xA;" OwnerUserId="23546" LastEditorUserId="2444" LastEditDate="2019-12-06T14:53:23.467" LastActivityDate="2019-12-06T14:53:23.467" Title="Is there an audio dataset with the corresponding phonemes in the audio?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16956" PostTypeId="2" ParentId="16946" CreationDate="2019-12-06T03:09:05.857" Score="1" Body="&lt;p&gt;I think serali answered this question well, though I wanted to give some extra reading for those interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways of deciphering what a neuron in a NN is doing. &lt;a href=&quot;https://www.youtube.com/watch?v=6wcs6szJWMY&quot; rel=&quot;nofollow noreferrer&quot;&gt;This lecture&lt;/a&gt; does a fantastic job at covering some of these methods and is an incredibly interesting watch. This covers more advanced methods of visualising what a model is doing.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-12-06T03:09:05.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16957" PostTypeId="1" CreationDate="2019-12-06T03:12:37.727" Score="6" ViewCount="177" Body="&lt;p&gt;I have implemented minimax with alpha-beta pruning to play checkers. As my value heuristic, I am using only the summation of material value on the board regardless of the position.&lt;/p&gt;&#xA;&lt;p&gt;My main issue lays in actually finishing games. A search with depth 14 draws against depth 3, since the algorithm becomes stuck in a loop of moving kings back and forth in a corner. The depth 14 player has a significant material advantage with four kings and a piece against a single king, however, it moves only one piece.&lt;/p&gt;&#xA;&lt;p&gt;I have randomly selected a move from the list of equally valued moves and this leads to more interesting games (thus preventing the loop). However, whichever player used this random tactic ended up far worse off.&lt;/p&gt;&#xA;&lt;p&gt;I am not quite sure how to solve this problem. &lt;em&gt;Should I do a deeper search of the best moves with the same value? Or is the heuristic at fault? If so, what changes would you suggest?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;So far I have tried a simple genetically generated algorithm that optimizes a linear scoring function (that accounts for the position). However as the algorithm optimized, it led to only draws and the same king loop.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions for how to stop this king loop are very welcome!&lt;/p&gt;&#xA;" OwnerUserId="31894" LastEditorUserId="2444" LastEditDate="2021-01-03T13:17:52.883" LastActivityDate="2021-01-03T13:17:52.883" Title="To deal with infinite loops, should I do a deeper search of the best moves with the same value, in alpha-beta pruning?" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;&lt;checkers&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16958" PostTypeId="2" ParentId="16957" CreationDate="2019-12-06T03:29:06.537" Score="1" Body="&lt;p&gt;I think this issue stems from the fact you aren't taking position into account. I would think this because as the game progresses, the number of moves that will result in a piece being taken becomes less and less, especially when there's only a few pieces left and quite a bit of &quot;chasing&quot; must occur before a piece is taken, likely more chasing then a depth of 14 allows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To remedy this, you could, towards the end of the game, add to the value of a state the inverse of the total distance each friendly piece has from other pieces, that way the agent will try to move towards other pieces and minimise this distance. If you find the right scale for this heuristic, the agent will prioritise moving towards enemy pieces only when it cant find any moves that result in taking a piece, helping it break out of this loop.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-12-06T03:29:06.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16959" PostTypeId="1" CreationDate="2019-12-06T05:12:48.040" Score="1" ViewCount="128" Body="&lt;p&gt;Assume somebody knows only to write in Latin characters. If they write words of any other language (example: Hindi, French, Latin)  using the Latin alphabet, how can I detect that language?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: If they write Hindi language word using the Latin alphabet) &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;kya kar raha hai&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     &amp;gt;&amp;gt; the output is Hindi language&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31573" LastEditorUserId="2193" LastEditDate="2019-12-06T11:46:58.067" LastActivityDate="2019-12-06T11:49:35.987" Title="How to detect any native language when written in Latin characters?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16960" PostTypeId="1" CreationDate="2019-12-06T06:15:21.973" Score="1" ViewCount="20" Body="&lt;p&gt;I got a large dataset of images (dimensions of 16 x 16, 250k samples) and corresponding spherical coordinates (distributed uniformly in each coordinate). On these, I trained a convolutional regression network to directly yield the coordinates for a provided image. The network itself is rather simple and consists of multiple convolutional layers where the last of them is flattened and followed by some dense layers to get the desired output. Since the input image size is rather small, pooling layers are obsolete I think (doesn't make much difference if used).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I now train on all of the data I will get reasonable results in the end. But, if I filter them before training, i.e. only use coordinates which are limited by a certain radius, the network will increase it's performance quite a bit, but will only work well if my input image corresponds to the parameter space used during training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, if the network isn't deep enough or has the wrong architecture to perform on the complete dataset with high confidence or if this is expected behaviour. One naive approach would be to train the network for different coordinate ranges and to store the weights for each of them. Then, you could train a classifier to decide in which range you are and use the previously determined weights for the network accordingly. But this seems strange to me, as a single network should somehow be able to achieve the same without this weird architecture, I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be pleased if someone has an idea how I could optimise the performance of my network to yield the best results over the whole coordinate space.&lt;/p&gt;&#xA;" OwnerUserId="31896" LastActivityDate="2019-12-06T06:15:21.973" Title="Optimisation of dependence of efficiency of CNN on training data" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16961" PostTypeId="1" CreationDate="2019-12-06T06:17:55.013" Score="1" ViewCount="61" Body="&lt;p&gt;Let us say that i have two ball throwing machines which has some algorithm running in the back-end for releasing the balls. One machine shows it throws 5 balls in 1 sec. Other shows the exact distribution of how many balls were thrown in each 0.2 secs (say the distribution is: 2,1,0,1,1) but the sum is 5 balls/sec for this machine too. Can i use this data and some other independent parameters like speed, direction etc as inputs and predict the similar distribution for lower accuracy machine.?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re-framing my question: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am searching for an apt supervised model for the following use case: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have a  sum (say 10) and it can be distributed in a predefined number of bins (say 5) in a number of ways for instance: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; 1, 2, 5, 0, 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; 0, 0, 3, 7, 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution bins always have whole numbers and the sum is also a whole number. The distribution depends on a number of factors and patterns which can be learned in the volumetric data. Hence, if I am able to load more than one sum (say n sums) and output the corresponding n*5 distributions it will be better for precise prediction (as per my intuitions). I tried using some networks but they are not doing much good. &lt;/p&gt;&#xA;" OwnerUserId="31897" LastEditorUserId="31897" LastEditDate="2019-12-27T19:44:57.930" LastActivityDate="2019-12-30T19:09:02.283" Title="Finding the right model" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16962" PostTypeId="2" ParentId="16959" CreationDate="2019-12-06T08:58:48.960" Score="2" Body="&lt;p&gt;The most distinct words of a language are usually the function words (&lt;em&gt;the&lt;/em&gt;, &lt;em&gt;and&lt;/em&gt;, &lt;em&gt;of&lt;/em&gt;, &lt;em&gt;with&lt;/em&gt;,...); other lexical items are often (at least partly) shared between languages that had come in contact with each other. So looking for function words is usually the best way to identify the language in a given text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done by having a list of function words for each language you recognise, look those words up in your text, and then calculate the probability that the text belongs to each language given the frequencies of the words in your list. There will be overlaps, for example Dutch &lt;em&gt;of&lt;/em&gt; is equivalent to the conjunction &lt;em&gt;or&lt;/em&gt; in English, but also looks like the English &lt;em&gt;of&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to take texts in your languages, and split them into n-grams, typically trigrams. Then you use a list of trigrams and compare the frequency distributions in your known and unknown texts to find which is most likely to match. This has the advantage that you don't need to know anything about the language structure (because you don't need to identify function words), and it also captures the morphology (eg the common English suffix &lt;em&gt;-ing&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can simply implement this in a fairly basic program, or you could train a machine learning classifier with it. The former seems easier to me, but is not as exciting as the latter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for single words (as per your example) this might not work properly. Generally, the longer the text, the more precise the recognition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: On re-reading your question I think I have misunderstood it, and that you're asking about transliteration into Latin characters. In a way the same applies, that you need to first identify which language it is, and then choose the correct mapping from the Latin transliteration to the writing system that language natively uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the transliteration is purely symbolic, I don't think that an ML approach would be better than a simple lookup-table of equivalences. And of course there might be difficulties if the writing system has more distinct characters than the  Latin alphabet provides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE 2: If the input is transliterated text, then your reference texts (where you know the language) also have to be transliterated in the same character set (eg Latin in your example).&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-12-06T11:49:35.987" LastActivityDate="2019-12-06T11:49:35.987" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16963" PostTypeId="1" AcceptedAnswerId="16966" CreationDate="2019-12-06T10:57:51.983" Score="3" ViewCount="69" Body="&lt;p&gt;The specific problem I'm having is with a Fully Visible Belief Network. It is an explicit density model (though I don't know what quantifies something being such) that uses the chain rule to decompose the likelihood of an image x into a product of 1-d distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fg8PC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fg8PC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is meant by &quot;the likelihood of an image x&quot;? With respect to what? I assume it refers to how common this image would be in the data set it is selected from? Like if you had 1000 images, 800 of which were white and 200 of which were black, the model should ideally output 0.2 for any black image inputted? Obliviously with more complicated clustering like dogs vs cats it'd be a bit different, but that's my intuition. Is that correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also as a side question, that equation looks very wrong. If you have an image of &lt;span class=&quot;math-container&quot;&gt;$1048\times720$&lt;/span&gt; pixels, and say every pixel evaluates to have a probability of 0.9, you'd expect the final probability of the image to be 0.9 or 90%. But according to that equation, it's &lt;span class=&quot;math-container&quot;&gt;$0.9^{720*1048}$&lt;/span&gt;, which is stupidly small, essentially 0. What's going on here?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-12-06T17:35:18.477" Title="In unsupervised learning, what is meant by &quot;finding the probability of an image&quot;?" Tags="&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16964" PostTypeId="1" CreationDate="2019-12-06T11:18:36.523" Score="1" ViewCount="26" Body="&lt;p&gt;We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the format [x, y] with the dimension of 1 complete scan being 160x2. The goal is to use these scan points to classify the various shapes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used a multilayer NN with sigmoid as the final layer and Adadelta optimizer for this problem but the accuracy reaches only upto 70%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend a proper model that can be used for Laser Scanner Data Classification?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;          MODEL&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def baseline_model():&#xA;    model = Sequential()&#xA;    model.add(Dense(2048, input_dim=160, activation='relu'))&#xA;    model.add(Dropout(0.1))&#xA;    model.add(Dense(1024, activation='relu'))&#xA;    model.add(Dropout(0.1))&#xA;    model.add(Dense(512, activation='relu'))&#xA;    model.add(Dropout(0.2))&#xA;    model.add(Dense(256, activation='relu'))&#xA;    model.add(Dropout(0.3))&#xA;    model.add(Dense(128, activation='relu'))&#xA;    model.add(Dropout(0.4))&#xA;    model.add(Dense(64, activation='relu'))&#xA;    model.add(Dropout(0.5))&#xA;    model.add(Dense(32, activation='relu'))&#xA;    model.add(Dropout(0.5))&#xA;    model.add(Dense(6, activation='softmax'))&#xA;    Adam = optimizers.Adam(lr=0.001)&#xA;    Adadelta =  optimizers.Adadelta(lr = 1)&#xA;    model.compile(loss='categorical_crossentropy', optimizer=Adadelta,   metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31900" LastActivityDate="2019-12-06T11:18:36.523" Title="Applying Machine Learning to 2D Laser Scanner Data" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16965" PostTypeId="1" CreationDate="2019-12-06T16:58:26.303" Score="2" ViewCount="70" Body="&lt;p&gt;I'm working with a problem where I have a lot of variables for different cases of different users. Depending on the values of the different variables of a concrete user in a concrete case, the algorithm must classify that user in that case as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Positive&lt;/li&gt;&#xA;&lt;li&gt;Negative&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But if the user is classified as positive, it must be classified as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Positive normal&lt;/li&gt;&#xA;&lt;li&gt;Positive high&lt;/li&gt;&#xA;&lt;li&gt;Positive extra-high&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If a case is positive, depending on the values of a part of the parameters, we know that the probability to be, for example, &lt;em&gt;positive normal&lt;/em&gt; is bigger or lower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up, I see the problem as a spam detector with different spam types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May this work if I apply an algorithm like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Random Forest&lt;/li&gt;&#xA;&lt;li&gt;Decision Tree&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or maybe I can include the negative case as a new group and then implement a K-means algorithm? Maybe this would help to find new groups of parameters that will say that the concrete case forms part of a group for sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one will fit best with a lot of parameters?&lt;/p&gt;&#xA;" OwnerUserId="31902" LastEditorUserId="2444" LastEditDate="2019-12-08T03:14:49.233" LastActivityDate="2019-12-08T03:14:49.233" Title="How can I classify instances into two categories and then into sub-categories, when the number of features is high?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;decision-tree&gt;&lt;random-forest&gt;&lt;k-means&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16966" PostTypeId="2" ParentId="16963" CreationDate="2019-12-06T17:35:18.477" Score="2" Body="&lt;p&gt;When you say likelihood, you are invoking several other concepts like events, sample, parameters, model, probability density function (PDF), etc (it would be helpful if you learn more about these concepts). In essence, a likelihood function &lt;span class=&quot;math-container&quot;&gt;$l(x|\theta)$&lt;/span&gt; is a PDF that quantifies how likely is that event &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; happens out of a set of possible events, given the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; that define your model.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the specific case of images, the set of possible events is usually one of two options: 1) all the available images in a dataset, or 2) all the existing images. Usually you want to model the likelihood in the option 2), but only having access to a sample of all the possible images. In either case, the likelihood is just the probability that you select one image out of all the possible ones. If you consider only images of &lt;span class=&quot;math-container&quot;&gt;$1048\times 720$&lt;/span&gt; pixels, the possible amount of images is &lt;span class=&quot;math-container&quot;&gt;$(256\times3)^{1048\times 720}$&lt;/span&gt;, where I am assuming that each pixel consists of 3 colors and each color can take 256 values. Since the amount of possible images is so so big, it is very common that the probability of selecting a specific one is very very small. This is a reason why you usually work with log-likelihood (the logarithm of the likelihood) instead of directly using likelihoods. For example, if all your images were equally probable, the likelihood would be in the order of &lt;span class=&quot;math-container&quot;&gt;$10^{-{10^7}}$&lt;/span&gt;, while the log-likelihood would be around &lt;span class=&quot;math-container&quot;&gt;$-10^7$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve your paradox with the probability of images and pixels, consider that instead of pixels you have coins and instead of images you have sequences of coins. Let's say that you have a fair coin, so the probability of tails (&lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;) after tossing the coin is 0.5. If you toss a second coin, the probability of having &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; again is naturally 0.5 as well, but what is the probability that both results where &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;? It is the product (0.25) since the events are independent. Similarly, the probability of the other three sequences &lt;span class=&quot;math-container&quot;&gt;$TH$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$HT$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$HH$&lt;/span&gt; is just 0.25. You can see that since the probability needs to be shared between 4 sequences equally probable, they are less probable with respect to the probabilities of the sequences of length 1. If you toss the coin 3 times, then the probability of all these coins being tails is just &lt;span class=&quot;math-container&quot;&gt;$0.5^{3}$&lt;/span&gt;. Again, there are now 8 possible sequences, all sharing the same amount of probability. You can see what's going on. Since the amount of possible options becomes large, the probability of each possible sequence of coins becomes small. Clearly, you would never toss a coin 10 times and expect to obtain all &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, right? Well, exactly the same happens in the case of the pictures.&lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2019-12-06T17:35:18.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16967" PostTypeId="1" CreationDate="2019-12-06T20:13:15.300" Score="2" ViewCount="41" Body="&lt;p&gt;I would like to know if my understanding of RPN training is correct, and if never training the RPN on some specific anchor box is bad (i.e if the anchor never sees good nor bad examples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make my point clear, assume we have two functions. &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_1}$&lt;/span&gt; which represents the backbone that outputs a feature map of size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (assume flattened) for an image of size &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; (WLOG assume the image is flattened)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_{\theta_1}: \mathbb{R}^m \to \mathbb{R}^n&#xA;$$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_2}$&lt;/span&gt; that represents the 'objectness' of each anchor box. We can suppose that &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_2}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_1}$&lt;/span&gt; are convolutional neural networks, where &lt;span class=&quot;math-container&quot;&gt;$\theta_1, \theta_2$&lt;/span&gt; are the networks' parameters. For simplicity, assume the RPN does not output bounding boxes correction, and only outputs the probabilities that an anchor box is an object or not.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ f_{\theta_1}: \mathbb{R}^n \to \mathbb{R}^{k \cdot n}$$&lt;/span&gt;&#xA;We can assume &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt;, which is the number of boxes per anchor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my understanding is correct, we select &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; good proposals &lt;span class=&quot;math-container&quot;&gt;$G_p$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; bad proposals &lt;span class=&quot;math-container&quot;&gt;$B_p$&lt;/span&gt; for training the RPN, which are indices of good and bad predictions. In other terms, if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an image (assume flattened), then &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_2}(f_{\theta_1}(x)) = y$&lt;/span&gt;, next we only back-propagate the loss for coordinates &lt;span class=&quot;math-container&quot;&gt;$B_p$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G_p$&lt;/span&gt; in y. For instance, if &lt;span class=&quot;math-container&quot;&gt;$p=1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$G_p = \{i\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_p = \{j\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ 1\leq i \neq j \leq n$&lt;/span&gt; then we only compute the loss of the RPN for coordinates &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. &#xA;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Is my understanding correct? and if not, how do we perform training?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Assuming my understanding is right or partially right about the last step, What happens if we never train component &lt;span class=&quot;math-container&quot;&gt;$y_0$&lt;/span&gt; from the RPN's output for example? (i.e we never back propagate the  loss through some components for &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;) woudn't this be a problem (i.e hurt performance or network training does not go well at all in some cases?)&lt;/p&gt;&#xA;" OwnerUserId="21484" LastEditorUserId="21484" LastEditDate="2019-12-09T00:48:12.727" LastActivityDate="2019-12-09T00:48:12.727" Title="FasterRCNN's RPN network training" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16968" PostTypeId="2" ParentId="16896" CreationDate="2019-12-07T01:00:50.050" Score="0" Body="&lt;p&gt;I must simply direct you to this excellent blog post on Machine Learning Mastery:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/what-is-holding-you-back-from-your-machine-learning-goals/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/what-is-holding-you-back-from-your-machine-learning-goals/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14777" LastActivityDate="2019-12-07T01:00:50.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16969" PostTypeId="2" ParentId="16896" CreationDate="2019-12-07T09:35:08.337" Score="3" Body="&lt;p&gt;This course is focused on machine learning using MATLAB, which is not practical nowadays as it is a programming language used specifically for computing, and cannot display GUI or communicate through the network. The language is powerful but limited in some ways. Nowadays most people use python for machine learning, as it is versatile and can connect to other backend like C++, java, JavaScript easily. The language is also a general language, and unlike MATLAB can do many things not limited to computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to join this course, I would recommend first learning MATLAB language and also learn basic calculus like derivatives. This would greatly help on your learning of the course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if you want to here serious about machine learning, I would encourage you to enroll in the Deep Learning specialization also by Andrew Ng on Coursera.&#xA;&lt;a href=&quot;https://www.coursera.org/specializations/deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.coursera.org/specializations/deep-learning&lt;/a&gt;&#xA;This course uses python as the programming language and teaches more modern approaches to deep learning like recurrent neural networks, convolutional neural networks and more. It also talks more about application of neural network. There is also theory, but it also talks about application of a specific algorithm and how it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-12-07T09:35:08.337" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16970" PostTypeId="1" AcceptedAnswerId="16971" CreationDate="2019-12-07T09:40:07.557" Score="6" ViewCount="1130" Body="&lt;p&gt;It is my understanding that, in Q-learning, you are trying to mimic the optimal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function &lt;span class=&quot;math-container&quot;&gt;$Q*$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Q*$&lt;/span&gt; is a measure of the predicted reward received from taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; at state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; so that the reward is maximised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand for this to be properly calculated, you must explore all possible game states, and as that is obviously intractable, a neural network is used to approximate this function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a normal case, the network is updated based on the MSE of the actual reward received and the networks predicted reward. So a simple network that is meant to chose a direction to move would receive a positive gradient for all state predictions for the entire game and do a normal backprop step from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, to me, it makes intuitive sense to have the final layer of the network be a softmax function for some games. This is because in a lot of cases (like Go for example), only one &quot;move&quot; can be chosen per game state, and as such, only one neuron should be active. It also seems to me that would work well with the gradient update, and the network would learn appropriately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the big problem here is, this is no longer Q learning. The network no longer predicts the reward for each possible move, it now predicts which move is likely to give the greatest reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I wrong in my assumptions about Q learning? Is the softmax function used in Q learning at all?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-06-23T08:02:21.957" Title="Does using the softmax function in Q learning not defeat the purpose of Q learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16971" PostTypeId="2" ParentId="16970" CreationDate="2019-12-07T09:59:05.397" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, to me, it makes intuitive sense to have the final layer of the network be a softmax function for some games. This is because in a lot of cases (like Go for example), only one &quot;move&quot; can be chosen per game state, and as such, only one neuron should be active.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You are describing a network that approximates as &lt;em&gt;policy function&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;, for a discrete set of actions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It also seems to me that would work well with the gradient update, and the network would learn appropriately.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes there are ways to do this, based on the &lt;a href=&quot;https://towardsdatascience.com/policy-gradients-in-a-nutshell-8b72f9743c5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy Gradient Theorem&lt;/a&gt;. If you read it you will probably discover this is more complex to understand than you first thought, the problem being that the agent is never directly told what the &quot;best&quot; action is in order to simply learn in a supervised manner. Instead, it has to be inferred from rewards observed whilst acting. This is a bit harder to figure out than the Q learning update rules which are just sampling from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bellman_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bellman optimality equation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can split Reinforcement Learning methods broadly into value-based methods and policy gradient methods. Q learning is a value-based method, whilst REINFORCE is a basic policy gradient method. It is also common to use a value based method &lt;em&gt;within&lt;/em&gt; a policy gradient method in order to help estimate likely future return used to drive the polcy gradient updates - this combination is called Actor-Critic where the actor learns a policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and the critic learns a value function e.g. &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But the big problem here is, this is no longer Q learning. The network no longer predicts the reward for each possible move, it now predicts which move is likely to give the greatest reward.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is true, but it is not a big problem. The main issue is that policy gradient methods are more complex than value based methods. They may or may not be more effective, it depends on the environment you are tryng to create an optimal agent for.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the softmax function used in Q learning at all?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I cannot think of any non-contrived environment in which this function would be useful for an action value approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is possible to use a variant of softmax to create a behaviour policy for Q learning. This uses a temperature hyperparameter &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; to weight the Q values, and provide a probability of selecting an action, as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(a_i|s) = \frac{e^{Q(s,a_i)/T}}{\sum_j e^{Q(s,a_j)/T}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is high all the probabilities of actions will be similar, when it is low even a small difference in &lt;span class=&quot;math-container&quot;&gt;$Q(s,a_i)$&lt;/span&gt; will make a big difference to probability of selecting action &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt;. This is quite a nice distribution for exploring whilst avoiding previously bad decisions. It will tend to focus the agent on exploring differences between similarly high rated actions. The main issue with it is that it introduces hyperparameters for deciding starting &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, ending &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and how to move between them.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-08T14:29:04.910" LastActivityDate="2019-12-08T14:29:04.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16973" PostTypeId="1" CreationDate="2019-12-07T14:38:52.240" Score="1" ViewCount="133" Body="&lt;p&gt;I need some explanation about the following paragraph (page 3) from the paper &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1907/1907.11272.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Novel Approach for Robust Multi Human Action Detection and Recognition based on 3-Dimentional Convolutional Neural Networks&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We introduce a 3D convolution neural network with the following notations: &lt;span class=&quot;math-container&quot;&gt;$I(x, y, d)$&lt;/span&gt; as an input video with a size of &lt;span class=&quot;math-container&quot;&gt;$x y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; the temporal depth&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is &quot;temporal depth&quot;? Is it the number of frames?&lt;/p&gt;&#xA;" OwnerUserId="31910" LastEditorUserId="2444" LastEditDate="2019-12-08T13:50:01.173" LastActivityDate="2020-02-15T09:54:08.230" Title="What is &quot;temporal depth&quot;?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;papers&gt;&lt;action-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16974" PostTypeId="1" AcceptedAnswerId="16977" CreationDate="2019-12-07T14:43:40.550" Score="0" ViewCount="95" Body="&lt;p&gt;Now I know this might break some StackExchange rules and I am definitely open for taking the thread down if it does!&#xA;I am trying to build an AI that can write it's own book and I have no idea where to start or what are the appropriate algorithms and approaches to go with. &#xA;How should I start and what do I exactly need for such a project? &lt;/p&gt;&#xA;" OwnerUserId="3894" LastEditorUserId="3894" LastEditDate="2019-12-07T14:51:34.633" LastActivityDate="2019-12-07T17:06:04.333" Title="Building an AI that generates text by itself" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;artificial-neuron&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16975" PostTypeId="2" ParentId="16974" CreationDate="2019-12-07T15:27:59.587" Score="2" Body="&lt;p&gt;Recurrent Neural Networks (RNNs) have been applied to generate text. In &lt;a href=&quot;https://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; blog post you will find a couple of interesting text examples (the author also has made his code available on github), e.g. their Shakespeare-like texts generated by an RNN:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PANDARUS:&#xA;  Alas, I think he shall be come approached and the day&#xA;  When little srain would be attain'd into being never fed,&#xA;  And who is but a chain and subjects of his death,&#xA;  I should not sleep.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Second Senator:&#xA;  They are away this miseries, produced upon my soul,&#xA;  Breaking and strongly should be buried, when I perish&#xA;  The earth and thoughts of many states.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;DUKE VINCENTIO:&#xA;  Well, your wit is in the care of side and that.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Second Lord:&#xA;  They would be ruled after this chamber, and&#xA;  my fair nues begun out of the fact, to be conveyed,&#xA;  Whose noble souls I'll have the heart of the wars.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Clown:&#xA;  Come, sir, I will make did behold your worship.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;VIOLA:&#xA;  I'll drink it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see the RNN is able to somewhat mimic the &quot;flow&quot; of the texts it has been trained on but some sentences (like at the very end) do not make much contextual sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, RNNs have been trained to generate other content, e.g. drawing numbers (see &lt;a href=&quot;https://arxiv.org/pdf/1502.04623.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) or creating music (see &lt;a href=&quot;http://people.idsia.ch/~juergen/blues/IDSIA-07-02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="30789" LastEditorUserId="30789" LastEditDate="2019-12-07T15:35:41.067" LastActivityDate="2019-12-07T15:35:41.067" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16977" PostTypeId="2" ParentId="16974" CreationDate="2019-12-07T17:06:04.333" Score="3" Body="&lt;p&gt;There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time.&lt;/p&gt;&#xA;&lt;p&gt;Many networks have been tried for text generation, with notable examples such as Markov chain. However RNN have been proven to work the best and is dominating the field of language modelling (text generation).&lt;/p&gt;&#xA;&lt;h1&gt;How text generation works&lt;/h1&gt;&#xA;&lt;p&gt;A neural network that generates text is commonly called a language model. It is trained on large amount of text with labels being the next token. The text generation process uses several random token as the starting phrase and then the network predicts the rest. However the network does not just predicts the most probable word, instead it randomly chooses one of the few most probable token, hence the generating part.&lt;/p&gt;&#xA;&lt;h1&gt;Why RNN work best on language modelling&lt;/h1&gt;&#xA;&lt;p&gt;RNN have a perception of time built into the architecture of teh network. LSTM, a popular RNN variant used, is composed of &amp;quot;memory units&amp;quot; that &amp;quot;remembers&amp;quot; past text, thus the &amp;quot;time&amp;quot; part. RNN process input according to the sequence of time, so the network can naturally understand time, thus the superior performance compared to other networks.&lt;/p&gt;&#xA;&lt;h1&gt;Architecture of language model&lt;/h1&gt;&#xA;&lt;p&gt;A language model consists of the encoder and the decoder. The encoder compresses word one-hot representation to a smaller sized vector representation. The smaller sized representation is then passed through the decoder, which maps the encoding to the words one hot vectors again.&lt;/p&gt;&#xA;&lt;h1&gt;State of the art results for language modelling&lt;/h1&gt;&#xA;&lt;p&gt;Language modelling is an actively researched field in the AI community, and recently the model GPT-2 have achieved a breakthrough in language modelling accuracy, producing almost human like text with a special component added, the attention layer. Attention basically maps the &amp;quot;memory states&amp;quot; of the encoder and feed it as input to the decoder. The data teh model is trained on is also very large, with over 20GB of web scraped data from sites like Reddit.&lt;/p&gt;&#xA;&lt;h1&gt;Limits of language modelling&lt;/h1&gt;&#xA;&lt;p&gt;One limit of language modelling is the size of generated text. As GPU don't have unlimited memory, language model usually limits the input token size to a specific number, padding or trimming to this number. The number is usually 500-1000, which includes a paragraph or two, but not an entire book. You can only generate short paragraphs and essay with language modelling. For long text it is much harder.&lt;/p&gt;&#xA;&lt;h1&gt;Resources to help you get started&lt;/h1&gt;&#xA;&lt;p&gt;GPT-2 open AI blog: &lt;a href=&quot;https://openai.com/blog/better-language-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/better-language-models/&lt;/a&gt;&#xA;GPT-2 online interactive site for text generation: &lt;a href=&quot;https://talktotransformer.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://talktotransformer.com/&lt;/a&gt;&#xA;How to train and fine tune GPT-2 in python: &lt;a href=&quot;https://minimaxir.com/2019/09/howto-gpt2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://minimaxir.com/2019/09/howto-gpt2/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hope I can help you&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-12-07T17:06:04.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16978" PostTypeId="1" CreationDate="2019-12-07T18:03:11.107" Score="0" ViewCount="60" Body="&lt;p&gt;I am currently reading the paper &quot;Similarity of Narratives&quot; by Loizos Michael (link below) and I am having a hard time figuring out the definitions listed (p.107 - p.109).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please give me a practical example for each of the definitions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Article: &lt;a href=&quot;http://narrative.csail.mit.edu/cmn12/proceedings.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://narrative.csail.mit.edu/cmn12/proceedings.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="31914" LastActivityDate="2019-12-07T20:02:48.473" Title="Need examples for the following definitions" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16979" PostTypeId="2" ParentId="16978" CreationDate="2019-12-07T20:02:48.473" Score="3" Body="&lt;p&gt;The 'vocabulary' used is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathscr{F}$&lt;/span&gt; - &lt;a href=&quot;https://en.wikipedia.org/wiki/Fluent_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;fluents&lt;/em&gt;&lt;/a&gt;, conditions that change over time. The predicate &lt;code&gt;holds(&amp;lt;fluent&amp;gt;,&amp;lt;state&amp;gt;)&lt;/code&gt; defines whether a condition applies in a given state.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathscr{A}$&lt;/span&gt; - &lt;em&gt;actions&lt;/em&gt;, describe, erm, actions that happen. The predicate &lt;code&gt;occurs(&amp;lt;action&amp;gt;,&amp;lt;state&amp;gt;)&lt;/code&gt; specifies that a given action happens at a particular state (presumably triggering a change to a new state, and changing the value of one or more fluents)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathscr{T}$&lt;/span&gt; - time points, which specify a chronological ordering.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now we can look at the actual definitions: a &lt;em&gt;discourse&lt;/em&gt; combines events (actions) and facts (fluents) and provides a (partial) ordering, ie a sequence in which they occur. This is represented in an acyclic graph. So the discourse describes a sequence of events that happen in a particular order and have an effect on facts. The ordering is relative, as there are no absolute time reference points give. Actions and facts are expressed in predicate calculus by the predicates mentioned above. Actions can involve a character moving from location A to location B; the fluent describing the location of the character will have been changed through that action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;em&gt;embedding&lt;/em&gt; assigns a given point in time to each state in the discourse. Each state is assigned a time point in a way that is consistent with the relative timings of the states in the discourse. So you have a general story, and you fix each action in time. The absolute time of an action in the embedding has to be before/after other actions according to their order in the discourse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;domain&lt;/em&gt; adds two further predicates to a discourse, namely &lt;code&gt;static(&lt;/code&gt;&lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;&lt;code&gt;)&lt;/code&gt; and &lt;code&gt;causes(&lt;/code&gt;&lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;&lt;code&gt;,&amp;lt;fluent&amp;gt;)&lt;/code&gt;. These formulas &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; express logical constraints, that add consistency to the discourse. For example, if an action involves killing a character, the consequence has to be that the character is dead afterwards. So that action mandates a particular change, which would be encoded by such a formula. If the character was then to do some action after having been killed, that would invalidate the discourse in that domain. Different domains might have different constraints etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;model&lt;/em&gt; then works out if the domain is consistent, by evaluating all the actions and events using the domain constraints, and assigning it a truth value depending on whether it is possible or not. I might be wrong there, as I'm not sure I have interpreted the definition correctly &amp;mdash; please let me know in a comment if that is the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, if a model for a domain exists, it means that the actions and facts are consistent, and hence the &lt;em&gt;narrative&lt;/em&gt; works (it is &quot;a discourse compatible with a given domain&quot;): a narrative is a discourse in a domain which, when embedded in a timeline, is consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;default domain&lt;/em&gt; is one domain out of a set of other domains which seems preferable over the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I have to stop here for lack of time &amp;mdash; I might be able to expand this answer later) &lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-12-07T20:02:48.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16981" PostTypeId="1" CreationDate="2019-12-07T20:58:00.963" Score="1" ViewCount="36" Body="&lt;p&gt;Is there a seq-to-seq model which does not require to know the start and end of a sentence? I need to model a system which gets a long sequence of words and creates a long sequence of tokens as long as the input. For example it takes a sequence of 1000 words and creates a sequence of 1000 tokens, each token corresponds to an input word. &lt;/p&gt;&#xA;" OwnerUserId="29645" LastActivityDate="2019-12-07T20:58:00.963" Title="Sequence-to-Sequence models without specifying the start and end of sentences" Tags="&lt;deep-learning&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16982" PostTypeId="1" CreationDate="2019-12-07T21:08:48.160" Score="2" ViewCount="53" Body="&lt;p&gt;Suppose an object detection algorithm is good at detecting objects and people when an object and person is close to a camera and upright. If the person walks farther away from the camera and is &quot;upside-down&quot; from the perspective of the camera (e.g. a fisheye camera), should the algorithm still be good at detecting people and objects in this position? &lt;/p&gt;&#xA;" OwnerUserId="31916" LastEditorUserId="2444" LastEditDate="2019-12-08T14:34:34.427" LastActivityDate="2019-12-08T14:34:34.427" Title="Can a trained object detection model deal with variations of the input?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16983" PostTypeId="2" ParentId="16982" CreationDate="2019-12-07T23:20:08.373" Score="1" Body="&lt;p&gt;Not necessarily. Supposing your data is from the distribution of possible images containing an upright person close to the camera. Something like a neural network would perform poorly on the new data since it comes from a distribution other than on what it was trained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try augmenting the dataset to try to get some synthetic &quot;far away upside down people&quot; but there are no guarantees here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll look for the source but A. Ng cite's an experiment where a team trained a neural network on a large dataset of vehicle images. They did not realize that the cars were all facing the same direction and their model performed very poorly on images that were very similar with the primary difference being a horizontal flip. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2019-12-07T23:37:39.790" LastActivityDate="2019-12-07T23:37:39.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16985" PostTypeId="1" CreationDate="2019-12-08T01:06:07.753" Score="1" ViewCount="1276" Body="&lt;p&gt;I wonder if there is a way to solve snake game using Hamiltonian algorithm?&#xA;if there is a way&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;how to apply it?&lt;/li&gt;&#xA;&lt;li&gt;what data structure will be used with algorithm?&lt;/li&gt;&#xA;&lt;li&gt;time complexity and space complexity?&lt;/li&gt;&#xA;&lt;li&gt;is this algorithm an optimal solution or there is a better way?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31918" LastEditorUserId="1671" LastEditDate="2019-12-09T20:22:33.793" LastActivityDate="2019-12-09T20:22:33.793" Title="How to solve Snake Game with a Hamiltonian graph algorithm?" Tags="&lt;game-ai&gt;&lt;graph-theory&gt;&lt;path-finding&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16987" PostTypeId="2" ParentId="9912" CreationDate="2019-12-08T10:03:11.990" Score="1" Body="&lt;p&gt;Good question. I think this part of the book is not well explained.&lt;/p&gt;&#xA;&lt;p&gt;Off-policy evaluation of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; by itself doesn't make sense, IMO.&lt;/p&gt;&#xA;&lt;p&gt;I think there are two cases here&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;is if &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is deterministic, as we probably want in the case of &amp;quot;control&amp;quot;, i.e. we will determine &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to be deterministic and in every state choose the action that most likely to maximize the rewards/returns.&#xA;In that case, then evaluating &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; from a different distribution might not be so useful, as &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; becomes &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; with high likelihood. I don't see any sense in it.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is not deterministic. And it's a good question why would we want to evaluate &lt;span class=&quot;math-container&quot;&gt;$V_\pi$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$V_b$&lt;/span&gt;, instead of just evaluating it from &lt;span class=&quot;math-container&quot;&gt;$V_\pi$&lt;/span&gt; directly.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, IMO, off-policy evaluation of &lt;span class=&quot;math-container&quot;&gt;$V_\pi$&lt;/span&gt; doesn't make any sense.&lt;/p&gt;&#xA;&lt;p&gt;However, I think the goal here is actually the &lt;strong&gt;control algorithm&lt;/strong&gt; given in the book (using &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt;, p. 111 of the book [133 of the pdf]). The idea here is to use some arbitrary behavior/exploratory policy and, while it runs, update (&amp;quot;control&amp;quot;) the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. In there, you use the update rule for &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, which uses the idea of importance sampling - i.e. how to update the expected value of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; based on &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. But there it ACTUALLY makes sense.&lt;/p&gt;&#xA;&lt;p&gt;So, I suspect the evaluation was given by itself just so the reader can better understand how to do the evaluation, though it really doesn't make sense outside the control algorithm.&lt;/p&gt;&#xA;" OwnerUserId="27947" LastEditorUserId="2444" LastEditDate="2020-11-05T22:08:00.130" LastActivityDate="2020-11-05T22:08:00.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16990" PostTypeId="1" AcceptedAnswerId="16991" CreationDate="2019-12-08T13:41:28.590" Score="1" ViewCount="42" Body="&lt;p&gt;I have written an AI that plays a strategy board game. There are lots of different types of moves (e.g. attack, defend, help ally colony, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculate the best moves to do depending on a variety of factors, such as the value of nearby enemy colonies, the number of armies the colony currently has, etc (each of these has separate weightings). I'm trying to find the optimal weighting for each of the different factors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I decide the best configuration of parameters in a King of the Hill style tournament. I choose random values between a suitable range for each of the different parameters and then play two of these AI against each other 20 times. I have a total of 100 AI that play against the king, and then take the final king as the best AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that this is quite slow and I feel it's very inefficient, as a lot of the AI don't play well at all (probably due to the randomness of parameter values). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if there's a more efficient way to determine the optimal value of parameters?&lt;/p&gt;&#xA;" OwnerUserId="31932" LastEditorUserId="2444" LastEditDate="2019-12-08T14:48:54.143" LastActivityDate="2019-12-08T18:29:12.437" Title="How should I weight the factors that affect the choice of an action in a strategy board game with multiple actions?" Tags="&lt;game-ai&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16991" PostTypeId="2" ParentId="16990" CreationDate="2019-12-08T18:29:12.437" Score="2" Body="&lt;p&gt;You could use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithm&lt;/a&gt; to optimise the parameter settings. Here you don't choose random parameters all the time, but only at the beginning. Each AI (which is a vector of parameter settings) plays each other one for a ranking (you can probably reduce the number of total games by using a ladder-style ranking where only neighbours play against each other). Also, you don't need 100 to start with, as you might come across better combinations than those present initially throughout the processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you discard the worst AIs (ie parameter vectors), and recombine the best ones, adding some random mutations into the mix. In theory this should converge faster, as you preserve good parameter settings (depending on how interdependent they are) and remove bad ones from the 'pool'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also have a higher rate of mutations initially, which slowly goes down as you progress through the generations.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2019-12-08T18:29:12.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16992" PostTypeId="2" ParentId="16985" CreationDate="2019-12-08T19:52:16.027" Score="2" Body="&lt;p&gt;A Hamiltonian path in a graph is a path that visits all the nodes/vertices exactly once, a hamiltonian cycle is a cyclic path, i.e. all nodes visited once and the start and the endpoint are the same.&#xA;If we want to solve the snake game using this, we could divide the playable space in a grid and then try to just keep traversing on a hamiltonian cycle, this means you would eventually get all the rewards and never hit yourself unless you are longer than can fit on the screen.&#xA;You can look at the code given in this &lt;a href=&quot;https://github.com/gsurma/slitherin&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repo&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this method, as you can tell, is not going to be optimal by time, there could be better solutions than this.&#xA;You will have to use a graph to store all the nodes and the complexity is supposed to be O(n!).&lt;/p&gt;&#xA;" OwnerUserId="8549" LastActivityDate="2019-12-08T19:52:16.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16995" PostTypeId="2" ParentId="16943" CreationDate="2019-12-09T03:15:31.753" Score="1" Body="&lt;p&gt;I'm not sure why you need a continuing environment, but actually you can make most (if not all) OpenAI Gym environments continuing. When you perform a step, you receive information about the next state, the reward, a termination signal and a dictionary with additional information. Simply ignore the termination signal if you want the episodes to continue indefinitely. In some cases you will need to modify a variable of the environments called &lt;span class=&quot;math-container&quot;&gt;$\texttt {_max_episode_steps}$&lt;/span&gt;, which may force the simulation to stop or to restart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About your second question, check a resource called &lt;a href=&quot;https://spinningup.openai.com/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spinning Up&lt;/a&gt;, from OpenAI too. They explain several methods and their implementation. &lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2019-12-09T03:15:31.753" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16996" PostTypeId="1" CreationDate="2019-12-09T04:49:08.120" Score="1" ViewCount="3062" Body="&lt;p&gt;I've been using OpenAI's 2017 Sentiment Neuron implementation (&lt;a href=&quot;https://github.com/openai/generating-reviews-discovering-sentiment&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/generating-reviews-discovering-sentiment&lt;/a&gt;) for a while, because it was easy to set up and was the most accurate on benchmarks. What is the most accurate alternative now that I should use?&lt;/p&gt;&#xA;" OwnerUserId="31165" LastActivityDate="2019-12-09T19:46:32.067" Title="What is the most accurate pretrained sentiment analysis model by 2019?" Tags="&lt;natural-language-processing&gt;&lt;sentiment-analysis&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16997" PostTypeId="1" AcceptedAnswerId="17000" CreationDate="2019-12-09T06:12:57.263" Score="1" ViewCount="43" Body="&lt;p&gt;I just wanted to confirm that my understanding of Q learning was correct (with respect to a neural network).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The network, Q, is initialised randomly.&#xA;for n &quot;episodes&quot;:&#xA;    The state, s1, is initialised randomly&#xA;    while s1 != terminal state:&#xA;        s1 is fed into Q to get action vector q1&#xA;        action a1 is chosen based off the max of q1 (or randomly)&#xA;        state s2 is found by progressing s1 based on a1&#xA;&#xA;        s2 is fed into Q to get action vector q2&#xA;&#xA;        The expected output for Q at q1, y, is found by:&#xA;        {If s2 is terminal, it is the reward at s2&#xA;        {Otherwise: reward at s2 + gamm*max(q2)&#xA;        (The &quot;otherwise&quot; doesn't match bellmans equation as α=1)&#xA;&#xA;        Do gradient step where error = (y - max(q1))^2, only the max of q1 gets any gradient&#xA;        s1 = s2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This does not directly follow equations found by searching Q-learning as I find them rather ambiguous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also not taking into account storing network states (in this case, the network is called Q) for proper learning to avoid catastrophic forgetting, as I'm more concerned on getting the specifics right before good practice.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="26726" LastEditDate="2019-12-09T06:18:18.137" LastActivityDate="2019-12-09T14:36:00.783" Title="Is the following the correct implementation of the Q learning algorithm for a neural network?" Tags="&lt;neural-networks&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16998" PostTypeId="2" ParentId="16996" CreationDate="2019-12-09T07:14:56.870" Score="2" Body="&lt;p&gt;For best results, I'd recommend Google Cloud Machine Learning. It has [Natural Language Processing API] (&lt;a href=&quot;https://cloud.google.com/natural-language/docs/basics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/natural-language/docs/basics&lt;/a&gt;) with Sentiment, Entity, and Entity-Sentiment analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can implement in C++, PHP, Python, or other languages. This does require running a virtual machine instance on Google Cloud. TensorFlow can also be used for NLP sentiment analysis. The only drawback is it's not economical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an alternative, this is something I came across recently by Fast Text : &lt;a href=&quot;http://fasttext.cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://fasttext.cc&lt;/a&gt;.&#xA;They have some pre-trained models trained on yelp and some other datasets. They support a lot of languages&lt;/p&gt;&#xA;" OwnerUserId="31407" LastEditorUserId="25658" LastEditDate="2019-12-09T19:46:32.067" LastActivityDate="2019-12-09T19:46:32.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16999" PostTypeId="1" CreationDate="2019-12-09T09:13:35.553" Score="1" ViewCount="165" Body="&lt;p&gt;I want to train a DQN card game named witches. It consists of 60 Cards (1-14 of Yellow, Blue, Green, Red Cards + 4 Wizzards). The color of the first layed card has to be respected by the other players (if they have this card in hand). The one who has the card with the highest number gets the played cards. Each collected red card gives you a -1 point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to this &lt;a href=&quot;https://ai.stackexchange.com/questions/7049/dqn-input-representation-for-a-card-game/7050?noredirect=1#comment25924_7050&quot;&gt;answer&lt;/a&gt; I setup the inputs / state of the NN as binary (meaning I have 180 bool values (60 card x is currently on the table, 60 card x is in the ai hand, 60 card x is already been played)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to design the outputs / actions?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the ai is the first player of a round it can play any card&lt;/li&gt;&#xA;&lt;li&gt;If the ai is not first player it has to respect the first played card (or play a wizzard)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This means there is actually a list of available options. I then sort this list and have 60 Output bools which I set to 1 if this option is possible. Among these options the ai should then decide what the correct option is? Is this the correct procedure?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Inconsistent / Varying Action Space&lt;/strong&gt;&#xA;This is what we have to deal here with. As explained in &lt;a href=&quot;https://ai.stackexchange.com/questions/9491/inconsistent-action-space-in-reinforcement-learning&quot;&gt;here&lt;/a&gt; I think a DQN as well as Policy Gradient Methods is not the correct architecture to choose for solving such multi-agent card games. What architecture would you choose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;General procedure?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume I have 4 players, so do I have to get the old state before the ai is the next player and the new state is directly after this round is finished?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_game = game([&quot;Tim&quot;, &quot;Bob&quot;, &quot;Lena&quot;, &quot;Anja&quot;])&#xA;while True:&#xA;    #1. Play unti AI has to move:&#xA;    my_game.play_round_until_ai()&#xA;&#xA;    #2. Get old state:&#xA;    state_old = agent.get_state(my_game)&#xA;&#xA;    #3. Get the action the AI should perform&#xA;    action_ = agent.get_action(state_old, my_game)&#xA;&#xA;    #4. perform new Action and get new state&#xA;    #reward rates how good the current action was&#xA;    #score is the actual score of this game!&#xA;    reward, done, score = my_game.finishRound(action_)&#xA;&#xA;    # 5: Calculate new state&#xA;    state_new = agent.get_state(my_game)&#xA;&#xA;    #6. train short memory base on the new action and state&#xA;    agent.train_short_memory(state_old, action_, reward, state_new, done)&#xA;&#xA;    #7. store the new data into a long term memory&#xA;    agent.remember(state_old, action_, reward, state_new, done)&#xA;&#xA;    if done == True:&#xA;        # One game is over, train on the memory and plot the result.&#xA;        sc = my_game.reset()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code so far is available here: &lt;a href=&quot;https://github.com/CesMak/witches_ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/CesMak/witches_ai&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="31928" LastEditorUserId="31928" LastEditDate="2019-12-10T16:47:21.443" LastActivityDate="2019-12-10T16:47:21.443" Title="DQN card game - how to represent the actions?" Tags="&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17000" PostTypeId="2" ParentId="16997" CreationDate="2019-12-09T10:52:49.340" Score="1" Body="&lt;p&gt;Your pseudocode is fairly close to something that could be implemented. There are a couple of omissions, where it is not clear whether your implementation would do the right thing. There are a few non-general assumptions that are worth noting - they don't prevent you implementing for a specific purpose, but do prevent you writing the most generic DQN that you could. There is one mistake that prevents it working at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your mistake is here:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;Do gradient step where error = (y - max(q1))^2, only the max of q1 gets any gradient&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The loss function and gradient must be applied to the state/action pair &lt;code&gt;s1, a1&lt;/code&gt;, i.e. the action that was taken. This is an important distinction for when the agent takes exploratory (non-greedy) actions, as it must if it is going to learn about their values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A corrected statement might read: &lt;code&gt;Do gradient step where error = (y - Q(s1,a1))^2, only the output for a1 gets any gradient&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Regarding the rest of the pseudocode I could make the following observations:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;The state, s1, is initialised randomly&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is fine if it is possible. However, it is more usual in Q learning to initialise the state to whatever the normal start of the episode would be. That &lt;em&gt;might&lt;/em&gt; be from some randomised starting rules, but that is not quite the same as &#xA;picking a random state from all possible states.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;action a1 is chosen based off the max of q1 (or randomly)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is not quite clear what you mean by &quot;or randomly&quot;. For Q learning - and most RL - exploration is important. So this is not a design time choice for your algorithm, but a runtime choice that must itself be made so that all actions can be tried. Typically you would use an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy here that took the argmax action by default, but with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; took a random sample from actions with same chance of each possible action.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;state s2 is found by progressing s1 based on a1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is one of a few places where you have not quite captured the nature of a time step in a general MDP. You appear to have made the assumption that rewards are directly associated with states (with language like &quot;reward at s2&quot;). Whilst some MDPs are like that, it is not true in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better, more general statement might be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;s2, reward, done are found by taking a step from s1, using a1 as the action&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically the environment (or a wrapper for it that establishes state representations and rewards) will provide a step method that returns the tuple &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}, r_{t+1}, (s_{t+1} = s_T)$&lt;/span&gt;, although you could also hold these values in the environment model for the current time step and return them as required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;(The &quot;otherwise&quot; doesn't match bellmans equation as α=1)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know which Bellman equation you are referring to, but they don't include a learning rate, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; in any version I have seen. Perhaps you are referring to a tabular Q learning update? In any case, it doesn't seem relevant, as whatever learning rate you are using is now inside the gradient step, and does not need to be part of the pseuodcode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, regarding this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am also not taking into account storing network states (in this case, the network is called Q) for proper learning to avoid catastrophic forgetting, as I'm more concerned on getting the specifics right before good practice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;DQN will very likely not work at all, for most environments, unless you implement experience replay i.e. training the neural network on a random sample of stored &lt;code&gt;s1,a1,r,s2,done&lt;/code&gt; taken from a table that you build up on each time step. Usually you &lt;em&gt;cannot&lt;/em&gt; simply use the latest data from the current step. This is not just &quot;good practice&quot;, but an almost-always necessary stability improvement.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-09T14:36:00.783" LastActivityDate="2019-12-09T14:36:00.783" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17001" PostTypeId="2" ParentId="16999" CreationDate="2019-12-09T10:57:00.760" Score="1" Body="&lt;p&gt;Because your action space is not discrete, It would not be wise to evaluate possible actions. What would probably be better is evaluating next states. This means evaluating the value of the state after the action with your neural network instead of the value of a action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is this better? &#xA;You can use the neural net for each possible action , instead of trying to make the neural network figure out which actions are possible. the result would be taking the action of which the next state has the most value.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2019-12-09T10:57:00.760" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17002" PostTypeId="1" CreationDate="2019-12-09T12:30:04.037" Score="2" ViewCount="103" Body="&lt;p&gt;What is the difference between game theory and machine learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had gone through the papers &lt;a href=&quot;https://www.cs.ubc.ca/~jasonhar/GameNet-NIPS-2016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning for Predicting&#xA;Human Strategic Behavior&lt;/a&gt;, by Jason Hartford et al., and &lt;a href=&quot;http://cs229.stanford.edu/proj2012/AgrawalJaiswal-WhenMachineLearningMeetsAIandGameTheory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;When Machine Learning Meets AI and Game&#xA;Theory&lt;/a&gt;, by Anurag Agrawal et al., but I am not able to understand.&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2019-12-09T13:51:05.033" LastActivityDate="2019-12-09T13:51:05.033" Title="What is the difference between game theory and machine learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;game-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17003" PostTypeId="2" ParentId="17002" CreationDate="2019-12-09T13:37:27.793" Score="4" Body="&lt;p&gt;These are big areas, so here is a brief description of the differences:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Game_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game theory&lt;/a&gt; is concerned with studying solutions for 'games', which are basically a set of decisions leading to certain outcomes. In game theory you look at strategies to achieve the best outcome for a given participant. One classic example (which isn't really a game in the traditional sense) is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Prisoner%27s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prisoner's Dilemma&lt;/a&gt;: you and your friend have been arrested, and if only one of you testifies against the other, that person gets a reduced sentence, and the other one a much longer one. If you both testify against each other, you both get a medium sentence, and if you both keep quiet, you both go free. You don't know what your partner in crime does, so do you a) testify, or b) keep quiet? If you keep quiet, you might go free if your partner &lt;em&gt;also&lt;/em&gt; keeps quiet, but if he testifies, you are in it for a long time. So it's risky to keep quiet, even though you get the better outcome. If you testify you might avoid a longer sentence, but also will not go free. What is your best choice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game theory is often used in economics to model behaviour, as a rational agent would try to optimise gains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt;, on the other hand, is a way of training a statistical classifier. You feed features into an algorithm, and the algorithm then gives you a certain output, depending on the data you have trained it with. This hasn't got anything to do with game theory &lt;em&gt;per se&lt;/em&gt;, but I guess you could use machine learning to train an algorithm to choose moves in a game situation and then compare how that matches the optimal choices according to game theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I said, this is a very brief comparison. For more details I suggest you follow the links to read up on those two fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: Now that the papers are accessible &amp;mdash; game theory is indeed used as a benchmark. In the first paper, the &lt;em&gt;rational agent&lt;/em&gt; assumption from game theory is being modeled, but without a human expert telling the algorithm what that means. So you learn (using deep learning) what it means to be rational. In the second paper the authors attempt to learn a better algorithm than tit-for-tat, and indeed use game theory as a theoretical framework for comparison/evaluation.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2019-12-09T13:44:46.507" LastActivityDate="2019-12-09T13:44:46.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17004" PostTypeId="1" AcceptedAnswerId="17022" CreationDate="2019-12-09T14:26:42.580" Score="6" ViewCount="1644" Body="&lt;p&gt;Say I have a CNN with this structure:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;input = 1 image (say, 30x30 RGB pixels)&lt;/li&gt;&#xA;&lt;li&gt;first convolution layer = 10 5x5 convolution filters&lt;/li&gt;&#xA;&lt;li&gt;second convolution layer = 5 3x3 convolution filters&lt;/li&gt;&#xA;&lt;li&gt;one dense layer with 1 output&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So a graph of the network will look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/jIhSf11.png&quot; alt=&quot;Convolutional Neural Network&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct in thinking that the first convolution layer will create 10 new images, i.e. each filter creates a new intermediary 30x30 image (or 26x26 if I crop the border pixels that cannot be fully convoluted).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the second convolution layer, is that supposed to apply the 5 filters &lt;strong&gt;on all of the 10 images from the previous layer&lt;/strong&gt;? So that would result in a total of 50 images after the second convolution layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then finally the last FC layer will take all data from these 50 images and somehow combine it into one output value (e.g. the probability that the original input image was a cat).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or am I mistaken in how convolution layers are supposed to operate? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how to deal with channels, in this case RGB? Can I consider this entire operation to be separate for all red, green and blue data? I.e. for one full RGB image, I essentially run the entire network three times, once for each color channel? Which would mean I'm also getting 3 output values.&lt;/p&gt;&#xA;" OwnerUserId="31800" LastEditorUserId="31800" LastEditDate="2019-12-09T14:54:41.970" LastActivityDate="2019-12-12T10:25:11.933" Title="Convolutional Neural Network: does each filter in each convolution layer create a new image?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;hidden-layers&gt;&lt;image-processing&gt;&lt;convolution&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17005" PostTypeId="1" CreationDate="2019-12-09T14:50:04.790" Score="1" ViewCount="71" Body="&lt;p&gt;I'd like to create an AI for a 2D game involving two players fighting against each other. The map look something like this (The map is a NxN array somehow randomly generated): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1WaVb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1WaVb.jpg&quot; alt=&quot;map&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically the players can look for objects such as weapons located on platforms, shoot at each other to cause damages etc. The output actions are therefore limited to a few such as going up, left, right, down, shooting angle, shooting boolean...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if Reinforcement Learning using a neural network is a good approach to the problem. If so, how should I proceed for the learning phase? Should I force the AI to compete with a weaker version of itself at each iteration? Would it be computationally reasonable to train on a 4Gb GPU?&#xA;Thanks in advance for your advice ! &lt;/p&gt;&#xA;" OwnerUserId="31374" LastEditorUserId="31374" LastEditDate="2019-12-10T17:27:43.107" LastActivityDate="2019-12-10T17:27:43.107" Title="Reinforcement learning for a 2D game involving two players" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;training&gt;&lt;q-learning&gt;&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17006" PostTypeId="1" CreationDate="2019-12-09T16:09:29.980" Score="1" ViewCount="38" Body="&lt;p&gt;I am training an A2C reinforcement learning agent in a dense reward environment (where rewards are known and explicit at every timestep).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it redundant to include the previous reward in the current observation space?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reward is implicitly observed by the agent when collecting experiences and updating the network parameters. But could it also be useful for the agent to explicitly observe the reward of its previous action?&lt;/p&gt;&#xA;" OwnerUserId="27570" LastActivityDate="2019-12-09T16:09:29.980" Title="Should an RL agent directly observe the reward?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17007" PostTypeId="1" CreationDate="2019-12-09T16:47:19.020" Score="3" ViewCount="66" Body="&lt;p&gt;I built a simple AI system that tries to solve the 8 puzzle using DQN.&#xA;The problem is, if the agent gets only a reward greater than zero when winning, the training will take a long time, so I made a smooth reward function instead: &lt;span class=&quot;math-container&quot;&gt;$R=(n/9)^3$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of pieces that are in the right position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training became quicker but the AI chose to match 7 pieces out of 9 to get a reward of &lt;span class=&quot;math-container&quot;&gt;$(7/9)^3/(1-\gamma) = 0.47/(1-\gamma) = 4.7$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$\gamma=0.9$&lt;/span&gt;, choosing to win and getting reward of 1 doesn't make sense to the AI, lowering &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; will result in the AI to choose instant reward instead of long-term reward, so that will not be very helpful; lowering rewards of non-winning stats will make the training very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how do I choose a good reward function?&lt;/p&gt;&#xA;" OwnerUserId="31953" LastEditorUserId="2193" LastEditDate="2019-12-09T18:53:14.173" LastActivityDate="2019-12-09T18:53:14.173" Title="DQN, how to choose the reward fucntion?" Tags="&lt;machine-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17008" PostTypeId="2" ParentId="17004" CreationDate="2019-12-09T22:21:19.243" Score="4" Body="&lt;p&gt;About the images inside the CNN layers: I really recommend this &lt;a href=&quot;https://becominghuman.ai/what-exactly-does-cnn-see-4d436d8e6e52&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; since there is no one short answer to this question and it probably will be better to experiment with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the RGB input images: When needed to train on RGB pictures it is not advised to split the RGB channels, you can think of it by trying to identify a fictional cat with red ears, green body and blue tail. Each separated channel don't represent a cat, most certainly not with high confidence. I would recommend to transform you RGB images to gray scale and measure the network performance. If the performance are not sufficient you can make a 3D convolution layer. For example: If 30x30x3 is the input image, the filter has to be NxNx3.&lt;/p&gt;&#xA;" OwnerUserId="31957" LastActivityDate="2019-12-09T22:21:19.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17010" PostTypeId="1" CreationDate="2019-12-09T23:26:11.397" Score="1" ViewCount="106" Body="&lt;p&gt;I need to predict the performance (CPI cycles-per-instruction) of 90 machines for the next hour (or day). Each machine has a thousand records (e.g. CPU and memory usage). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am using a neural network with one hidden layer for this task. I have 9 inputs (features), 23 neurons in the hidden layer, and one output. I am using the Levenberg-Marquardt algorithm. Examples of the inputs (or features) are the CPU and memory capacity and usage, and the &lt;code&gt;machine_id&lt;/code&gt;. and Output is performance. I have 90 machines. Currently, I get an MSE of &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; and an R of &lt;span class=&quot;math-container&quot;&gt;$0.80$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataset consist of 30 days. I trained my network for the first 29 days, and I use day 30 to test. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been advised to use deep learning to have more flexibility and improve the MSE and R results. Could deep learning be helpful in this case? If yes, which deep learning model could I use to improve the results?&lt;/p&gt;&#xA;" OwnerUserId="30551" LastEditorUserId="30551" LastEditDate="2019-12-11T21:46:35.413" LastActivityDate="2020-01-11T08:01:45.263" Title="Should I use deep learning to solve my task?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17011" PostTypeId="1" AcceptedAnswerId="17013" CreationDate="2019-12-10T02:17:41.837" Score="3" ViewCount="135" Body="&lt;p&gt;I have made several neural networks by using &lt;a href=&quot;https://brain.js.org/#/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brain.Js&lt;/a&gt; and &lt;a href=&quot;https://www.tensorflow.org/js&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow.js&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between artificial intelligence and artificial neural networks?&lt;/p&gt;&#xA;" OwnerUserId="31949" LastEditorUserId="2444" LastEditDate="2019-12-10T02:43:33.477" LastActivityDate="2019-12-10T22:34:59.723" Title="What is the difference between artificial intelligence and artificial neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17012" PostTypeId="1" CreationDate="2019-12-10T02:51:05.537" Score="1" ViewCount="19" Body="&lt;p&gt;We have a scenario where we need to implement an Artificial Intelligence solution which will evaluate the input data file of my Azure Data Factory pipeline and let us know whether the file is good or bad with respect to it's data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have 10 files several rows which are good input files and 2 files with several rows which are bad input files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each file either it is good/bad has 26 columns. The above two files are bad because of below reasons. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;One file has all empty values for one column which is not expected.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another file has, the value 'TRUE for all rows for a specific column which was also not he general scenario. (some % of TRUE's and some % records with FALSE will appear in good files)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Like this, there may be several scenarios where the input file may be treated as bad file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to implement an Artificial Intelligence solution which should analyze all the input files and identify the hidden patterns of the data in file and detect abnormal scenarios like above and should eventually mark the file as bad file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please suggest for the approach or what components in Azure can help to achieve this kind of file sanity check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="31959" LastActivityDate="2019-12-10T02:51:05.537" Title="Need to analyze input CSV files and determine whether input file is good or bad w.r.t it's data" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17013" PostTypeId="2" ParentId="17011" CreationDate="2019-12-10T03:03:08.347" Score="0" Body="&lt;p&gt;From wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial neural networks (ANN) or connectionist systems are computing systems that are inspired by, but not identical to, biological neural networks that constitute animal brains. Such systems &quot;learn&quot; to perform tasks by considering examples, generally without being programmed with task-specific rules.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Artifical Intelligence on the other hand refers to the broad term of &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;intelligence demonstrated by machines&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This obviously doesn't clear much up, so the next logical question is: &quot;What is intelligence?&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, however, is one of the most debated questions in computer science and many other fields, so there isn't a straight answer for this. The most you can do is decide yourself what you think intelligence refers to, because as far as we know, there is no agreed upon way of quantifying intelligence, and so the definition of such will remain ambiguous.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-12-10T03:03:08.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17014" PostTypeId="1" CreationDate="2019-12-10T03:14:22.177" Score="1" ViewCount="31" Body="&lt;p&gt;How would you differentiate between different on-policy reinforcement learning algorithms?&lt;/p&gt;&#xA;" OwnerUserId="31960" LastEditorUserId="2444" LastEditDate="2019-12-10T04:41:14.403" LastActivityDate="2019-12-10T04:44:16.163" Title="How would you differentiate between different on-policy reinforcement learning algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;theory&gt;&lt;intelligent-agent&gt;&lt;on-policy-methods&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17016" PostTypeId="2" ParentId="17011" CreationDate="2019-12-10T04:44:19.980" Score="3" Body="&lt;p&gt;Artificial intelligence is a broad field, ANN is a specific technique with in that field. &lt;/p&gt;&#xA;" OwnerUserId="30178" LastActivityDate="2019-12-10T04:44:19.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17017" PostTypeId="1" CreationDate="2019-12-10T06:02:32.383" Score="1" ViewCount="29" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;br&gt;&#xA;This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the body and two points of contact with the terrain, on each leg (if there's no contact, the value is zero). The outputs are the four motor rates for each leg.&lt;br&gt;&#xA;I'm using Keras with the CNTK backend to train the network. The network has 30 input nodes (ReLU), one hidden layer with 64 nodes (ReLU) and an output layer with 4 nodes (Sigmoid). Optimizer is &lt;code&gt;Adam&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training data has 2459 datapoints. Running &lt;code&gt;model.validate&lt;/code&gt; with parameters testDataPercentage = 0.25, epochs = 50 and batchSize = 10 gave me:   &lt;code&gt;loss: 2.9509 - accuracy: 0.3283 - val_loss: 2.8592 - val_accuracy: 0.3213&lt;/code&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But running model.evaluate multiple times gave me:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;['loss', 'accuracy'] [3.10, 0.50]&#xA;['loss', 'accuracy'] [3.04, 0.23]&#xA;['loss', 'accuracy'] [3.01, 0.11]&#xA;['loss', 'accuracy'] [3.45, 0.02]&#xA;['loss', 'accuracy'] [3.17, 0.40]&#xA;['loss', 'accuracy'] [3.03, 0.27]&#xA;['loss', 'accuracy'] [3.012, 0.46]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Loss doesn't decrease much over 50 epochs. It reduces from maybe 3 to 2.8. That's it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;br&gt;&#xA;I don't understand why the accuracy varies so much for each run.&lt;br&gt;&#xA;If I add a hidden layer or even add a dropout of 0.2, the results are similar: &lt;code&gt;loss: 2.9253 - accuracy: 0.2978 - val_loss: 2.9350 - val_accuracy: 0.3148&lt;/code&gt;. &#xA;Reducing the number of hidden nodes to 15 gives the same results: &lt;code&gt;loss: 2.9253 - accuracy: 0.2978 - val_loss: 2.9350 - val_accuracy: 0.3148&lt;/code&gt;. Hidden layers with 64 nodes gives the same results. Training data with just 500 data points also gives the same results. Using sigmoid instead of ReLU gives slightly worse results.&lt;br&gt;&#xA;I've been through many tutorials and guides on how to debug or check why the neural network is not working, but they don't teach properly, what these values mean and how to adjust the network.&lt;br&gt;&#xA;Does the loss not decreasing, mean the network is not learning?&lt;br&gt;&#xA;Does the fact that increasing or decreasing the layers or the number of training data mean that the network is not learning?&lt;/p&gt;&#xA;" OwnerUserId="31961" LastActivityDate="2019-12-10T06:02:32.383" Title="Does a varying ANN model accuracy mean underfitting or overfitting?" Tags="&lt;keras&gt;&lt;artificial-neuron&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17018" PostTypeId="1" CreationDate="2019-12-10T06:49:06.140" Score="1" ViewCount="23" Body="&lt;p&gt;When I try to fit a Normal Distribution to the output of a policy network, for a continuous action space problem, what should be its standard deviation? mean for the distribution will directly be the output of the policy network.&lt;/p&gt;&#xA;" OwnerUserId="29879" LastActivityDate="2019-12-10T06:49:06.140" Title="Deciding std. deviation for policy network output?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;probability-distribution&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17019" PostTypeId="2" ParentId="17011" CreationDate="2019-12-10T10:31:21.833" Score="1" Body="&lt;p&gt;I would explain it as Artificial Intelligence is a huge topic concerning many fields as: robotics, computer vision, machine learning, etc. It focuses on any &quot;inteligent&quot; task that a computer can do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Artificial Neural Networks are a sub-topic of Machine learning, and probably as you've seen, as you said you have some experience with them, deals with a specific way of solving problems using a set of 'neurons' that try to imitate actual biological neurons. Explaining it in a really simplistic way, it is a method of fitting a function to your specific data in such a way that it still stands and gives good predictions on test data. By 'training' the network, you're basically trying to find better values for the weights(analogous to synapses in an actual brain, connections between the neurons) between the neurons in order to give better outputs in general on that specific type of data instead of just one case.&lt;/p&gt;&#xA;" OwnerUserId="31966" LastEditorUserId="31966" LastEditDate="2019-12-10T22:19:34.167" LastActivityDate="2019-12-10T22:19:34.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17020" PostTypeId="1" AcceptedAnswerId="22082" CreationDate="2019-12-10T11:27:16.180" Score="2" ViewCount="52" Body="&lt;p&gt;I was trying to build a CNN model. I used time series data of daily temperature to predict if there is risk of an event, say bacteria growth. &lt;strong&gt;I calculated the descriptive statistics of the time series, ie. mean, variance, skewness, kurtosis etc for each observation and added them to input data.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is CNN capable of extracting the effect of the descriptive statistics the label, meaning that adding these descriptive statistics features manually does not make a difference?&#xA;(I will still try this later, but like to hear what you think about it). Thanks&lt;/p&gt;&#xA;" OwnerUserId="31075" LastActivityDate="2021-02-06T13:05:48.093" Title="Is CNN capable of extracting the descriptive statistics features" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17021" PostTypeId="2" ParentId="11220" CreationDate="2019-12-10T12:12:18.773" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/google/voice-builder&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;voice-builder&lt;/strong&gt;&lt;/a&gt; is an opensource text-to-speech (TTS) voice building tool from Google.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastActivityDate="2019-12-10T12:12:18.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17022" PostTypeId="2" ParentId="17004" CreationDate="2019-12-10T12:41:44.793" Score="4" Body="&lt;p&gt;You are partially correct.  On CNNs the output shape per layer is defined by the amount of filters used, and the application of the filters (dilation, stride, padding, etc.).  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;CNNs shapes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In your example, your input is  &lt;code&gt;30 x 30 x 3&lt;/code&gt;.  Assuming stride of &lt;code&gt;1&lt;/code&gt;, no padding, and no dilation on the filter, you will get a spatial shape equal to your input, that is &lt;code&gt;30 x 30&lt;/code&gt;.  Regarding the depth if you have &lt;code&gt;10&lt;/code&gt; filters (of shape &lt;code&gt;5 x 5 x 3&lt;/code&gt;) you will end up with a &lt;code&gt;30 x 30 x 10&lt;/code&gt; output at your first layer.  Similarly, on the second layer with 5 filters (of shape &lt;code&gt;3 x 3 x 10&lt;/code&gt;, note the depth to work on the previous layer) you have &lt;code&gt;30 x 30 x 5&lt;/code&gt; output. The FC layer has the same amount of weights as the input (that is &lt;code&gt;4500&lt;/code&gt; weights) to create a linear combination of them.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;CNN vs Convolution&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Note that the CNNs operate differently from the traditional signal processing convolution.  In the former, the convolution operation performs a dot product with the filter and the input to output a single value (and even add bias if you want to).  While the latter outputs the same amount of channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CNNs borrow the idea of a shifting kernel and a kernel response.  But they do not apply a convolution operation per se.  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Operation over the RGB&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The CNN is not operating on each channel separately.  It is merging the responses of the three channels and mixing them further.  The deeper you get the more mix you get over your previous results.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of your FC is just one value.  If you want more, you need to add more FC neurons to get more linear combinations of your inputs.&lt;/p&gt;&#xA;" OwnerUserId="31955" LastActivityDate="2019-12-10T12:41:44.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17023" PostTypeId="1" CreationDate="2019-12-10T15:02:11.077" Score="3" ViewCount="90" Body="&lt;p&gt;Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd prefer tabular datasets rather than image datasets, since most image dataset are either too simple that even XGBoost can perform well (e.g. MNIST), or too difficult for XGBoost that its performance is too low (e.g. almost any dataset that is more complex than CIFAR10; please correct me if I'm wrong).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="31971" LastActivityDate="2021-05-04T18:06:10.507" Title="Tabular Datasets where Deep neural networks outperforms XGBoost" Tags="&lt;deep-learning&gt;&lt;boosting&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17024" PostTypeId="1" CreationDate="2019-12-10T16:13:51.140" Score="2" ViewCount="41" Body="&lt;p&gt;I was looking at the source code for a personal project neural network implementation, and the bias for each node was mistakenly applied after the activation function. The output of each node was therefore &lt;span class=&quot;math-container&quot;&gt;$\sigma\big(\sum_{i=1}^n w_i x_i\big)+b$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\sigma\big(\sum_{i=1}^n w_i x_i + b\big)$&lt;/span&gt;. Assuming standard back-propagation algorithms are used for training, what (if any) impact would this have?&lt;/p&gt;&#xA;" OwnerUserId="31975" LastEditorUserId="2444" LastEditDate="2019-12-10T22:10:45.263" LastActivityDate="2019-12-10T22:10:45.263" Title="What would be the implications of mistakenly adding bias after the activation function?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17025" PostTypeId="1" AcceptedAnswerId="17035" CreationDate="2019-12-10T16:48:14.897" Score="8" ViewCount="2019" Body="&lt;p&gt;Suppose a deep neural network is created using Keras or Tensorflow. Usually, when you want to make a prediction, &lt;strong&gt;the user&lt;/strong&gt; would invoke &lt;code&gt;model.predict&lt;/code&gt;. However, how would the actual AI system proactively invoke their own actions (i.e. without the need for me to call &lt;code&gt;model.predict&lt;/code&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="31978" LastEditorUserId="2444" LastEditDate="2021-02-12T17:19:34.473" LastActivityDate="2021-02-12T17:19:34.473" Title="How can an AI freely make decisions?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;philosophy&gt;&lt;agi&gt;&lt;intelligent-agent&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17026" PostTypeId="2" ParentId="17024" CreationDate="2019-12-10T17:17:29.117" Score="2" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$L(\mathbf{w}, b) = \sigma \left(\sum_{i=1}^n w_i x_i \right)+b$&lt;/span&gt;, then the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;partial derivative of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;&lt;/a&gt;, in &lt;a href=&quot;https://en.wikipedia.org/wiki/Leibniz%27s_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leibniz's notation&lt;/a&gt;, is &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial b} = 1$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$L(\mathbf{w}, b) = \sigma \left(\sum_{i=1}^n w_i x_i + b \right)$&lt;/span&gt;, then the partial derivative of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial b} = \frac{\partial L}{\partial \sigma} \frac{\partial \sigma}{\partial b}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in general, the partial derivatives with respect to &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; would be different in these two cases, thus, in the gradient descent step, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; would be updated differently in both cases. In practice, it may or not affect the performance of the model, depending on the importance of the bias with respect to the given problem. Read &lt;a href=&quot;https://stackoverflow.com/a/2499936/3924118&quot;&gt;this answer&lt;/a&gt; to understand the role of the bias.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-10T17:54:22.850" LastActivityDate="2019-12-10T17:54:22.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17027" PostTypeId="1" CreationDate="2019-12-10T17:33:00.350" Score="1" ViewCount="14" Body="&lt;p&gt;&lt;strong&gt;Example 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An object is composed of 3 sub-objects. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Object 1: 90% looks like an eye 10% looks like a wheel&lt;/li&gt;&#xA;&lt;li&gt;Object 2: 50% looks like an eye 50% looks like a wheel&lt;/li&gt;&#xA;&lt;li&gt;Object 3: 90% looks like a mouth 10% looks like a roof&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;OK. So now we want to determine what the whole-object is. Using this evidence maybe we find:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Combined Object: 90% looks like a face 10% looks like an upside-down car.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But now, &lt;em&gt;given this&lt;/em&gt;, we go back an reclassify the sub-objects.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given whole object is a car object 2 is 99% an eye.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm looking for an algorithm that sort of goes back and forth between the context and the subobjects to classify both an object and it's parts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is related to the rabbit-duck illusion, where such an algorithm once decided something is a rabbit, it classifies it's parts as rabbit-parts).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words the algorithm needs to calculate conditional probabilities P(A|B) but the B depends on what all the A's are! So it's a feed back loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 2&lt;/strong&gt;&#xA;There is a word &quot;funny&quot;. The sub-letters are classified (wrongly) as F U M Y. It guesses that the word is FUNNY and then goes back and tries to reclassify the letters. Using this reclassification it is even more certain the word is FUNNY. And much more certain the letters in the middle are NN and not M.&#xA;(Perhaps later the word is used in a sentence &quot;The fumes from this fire are really fumy&quot;. And then using this new evidence it has to go back and reclassify the word again and now it thinks the word is FUMY with 80% probability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an idea that I would write all the condition probabilities in a matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; then gives starting probabilities &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;. Then iterate like this: &lt;span class=&quot;math-container&quot;&gt;$S_{n+1}=M S_n$&lt;/span&gt; until hopefully it converges on something.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2019-12-10T19:13:31.270" LastActivityDate="2019-12-10T19:13:31.270" Title="Is there an algorithm for &quot;contextual recognition&quot; with probabilities?" Tags="&lt;classification&gt;&lt;algorithm&gt;&lt;probability&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17028" PostTypeId="1" CreationDate="2019-12-10T18:05:35.267" Score="2" ViewCount="49" Body="&lt;p&gt;Here's a question I might ask an AI to solve:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Colour the states of the USA using just 4 colours&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, a common heuristic a human might use is to start at one state and &quot;work their way out&quot;. Or start at an edge state. Now this seems to work best rather than just colouring states in a random order like a computer might do. And it means a human is often better than a computer because a computer might just start colouring random states and get into trouble very quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also I wonder if this is a learned heuristic or would a child develop this on his/her own?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question is, whether this heuristic is an innate optimisation strategy, or just laziness on the part of the human. i.e. colouring things close together takes less effort. Either way it leads to a good strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I wonder if there are any other examples of heuristics that humans inately use without realising it, that lead to good strategies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One heuristic that computers often don't know is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;If you're trying to play a game don't keep turn around and go the other way for no reason.&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, a human would not do this, but again this could be laziness on the part of a human. It takes more effort to turn around than keep going in one direction to explore it.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2019-12-12T19:26:55.477" LastActivityDate="2019-12-13T04:19:54.887" Title="What are some common heuristics that might be innate?" Tags="&lt;human-like&gt;&lt;heuristics&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17029" PostTypeId="1" CreationDate="2019-12-10T18:16:49.123" Score="2" ViewCount="41" Body="&lt;p&gt;When considering the policy network in PPO algorithm, we need to fit a Gaussian distribution to the neural network output (for a continuous action space problem). When I use this network to obtain action, should I sample from the fitted distribution or directly consider the mean output from the policy network? Also, what should be the standard deviation of the fitted distribution?&lt;/p&gt;&#xA;" OwnerUserId="29879" LastEditorUserId="2444" LastEditDate="2019-12-10T21:25:41.827" LastActivityDate="2019-12-10T21:25:41.827" Title="Should I consider mean or sampled value for action selection in ppo algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17030" PostTypeId="2" ParentId="7611" CreationDate="2019-12-10T18:20:44.390" Score="0" Body="&lt;p&gt;There is a third strategy and that is to study what heuristics a human player uses.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-12-10T18:20:44.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17031" PostTypeId="1" CreationDate="2019-12-10T22:05:26.193" Score="2" ViewCount="166" Body="&lt;p&gt;How does transformer leverage GPU which trains faster than RNN? I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multi-gpu and accelerates its training?&lt;/p&gt;&#xA;" OwnerUserId="31984" LastActivityDate="2020-09-06T12:06:05.530" Title="How does transformer leverage GPU which trains faster than RNN?" Tags="&lt;natural-language-processing&gt;&lt;attention&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17032" PostTypeId="2" ParentId="17011" CreationDate="2019-12-10T22:34:59.723" Score="2" Body="&lt;p&gt;Artificial intelligence can refer to a broad range of techniques by which machines (algorithms) demonstrate utility (fitness in an environment, where the environment may be either virtual or physical.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can include &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbolic AI&lt;/a&gt;, which utilizes logic and search exclusively.  (Symbolic AI is sometimes referred to as &quot;good old fashioned AI&quot; aka gofai, or &quot;Classical AI&quot;.) &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A key distinction is that Neural Networks constitute a form of &quot;statistical AI&quot;, which renders them capable of learning by trial/error &amp;amp; analysis. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The recent strength &amp;amp; applicability of statistically driven AI methods has been facilitated by advances in processing power and memory.  &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2019-12-10T22:34:59.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17033" PostTypeId="1" CreationDate="2019-12-10T23:09:50.017" Score="2" ViewCount="83" Body="&lt;p&gt;I want to train a network with video data and have it transform pixel values overtime on an input video. This is for an art project and does not need to be super elaborate, but the videos I want to render out of this might be big in resolution and frame count. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which neural network would be appropriate for this task? I think I'm looking for a convolutional network (but I'm not too sure of that either). Which framework could easily allow me to do this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'm no proper programmer, but self-learned on the go. I know some Javascript, but rather would like to learn more Python. Ideally, the easier and simpler the better though: I would be perfectly happy with something like &quot;Uber Ludwig&quot; (except maybe that it's from Uber).&lt;/p&gt;&#xA;" OwnerUserId="31985" LastEditorUserId="2444" LastEditDate="2019-12-11T13:53:35.750" LastActivityDate="2019-12-11T13:53:35.750" Title="Which neural network should I use to transform the pixels of a video overtime?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17034" PostTypeId="2" ParentId="17025" CreationDate="2019-12-10T23:13:44.830" Score="3" Body="&lt;p&gt;The short answer, i think, is that it cannot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI system will only do, and it will only be good at the task that the programmer made it for. Of course you could have an AI that, for example, can trigger a prediction on the input with different models depending on some other variables, but that will still be based on what the programmer wrote, it will never be able to do or learn new unintended things. Like having the model.predict() for an image classification NN in a loop and only stop when it detects a dog and then use another model to predict the breed for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you mentioned about &quot;letting the AI lose on the network&quot; usually is part of some concerns about AI that it could evolve, learn new actions and start acting on its own.&#xA;But those people unknowingly are actually talking about a general AI or strong AI, an AI system that could be as smart as a human so it could act in its own too. But as far is a know at least, we are not even close to creating such a system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I actually answered your question and didn't deviated too much from what you actually asked. Please tell me if so.&lt;/p&gt;&#xA;" OwnerUserId="31966" LastActivityDate="2019-12-10T23:13:44.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17035" PostTypeId="2" ParentId="17025" CreationDate="2019-12-11T00:02:04.917" Score="12" Body="&lt;p&gt;Neural networks, deep learning and other supervised learning algorithms do not &quot;take actions&quot; by themselves, they lack &lt;a href=&quot;https://en.wikipedia.org/wiki/Intelligent_agent&quot; rel=&quot;noreferrer&quot;&gt;agency&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is relatively easy to give a machine agency, as far as taking actions is concerned. That is achieved by connecting inputs to some meaningful data source in the environment (such as a camera, or the internet), and connecting outputs to something that can act in that environment (such as a motor, or the API to manage an internet browser). In essence this is no different from any other automation that you might write to script useful behaviour. If you could write a series of tests, if/then statements or mathematical statements that made useful decisions for any machine set up this way, then in theory a neural network or similar machine learning algorithm could learn to approximate, or even improve upon the same kind of function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your neural network has already been trained on example inputs and the correct actions to take to achieve some goal given those inputs, then that is all that is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, training a network to the point where it could achieve this in an unconstrained environment (&quot;letting it loose on the internet&quot;) is a tough challenge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are ways to train neural networks (and learning functions in general) so that they learn useful mappings between observations and actions that progress towards achieving goals. You can use genetic algorithms or other search techniques for instance, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&quot; rel=&quot;noreferrer&quot;&gt;the NEAT approach&lt;/a&gt; can be successful training controllers for agents in simple environments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;noreferrer&quot;&gt;Reinforcement learning&lt;/a&gt; is another popular method that can also scale up to quite challenging control environments. It can cope with complex game environments such as Defense of the Ancients, Starcraft, Go. The purpose of demonstrating AI prowess on these complex games is in part showing progress towards a longer-term goal of optimal behaviour in the even more complex and open-ended real world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;State of the art agents are still quite a long way from &lt;em&gt;general&lt;/em&gt; intelligent behaviour, but the problem of using neural networks in a system that learns how to act as an agent has much research and many examples available online.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-11T00:17:51.660" LastActivityDate="2019-12-11T00:17:51.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17036" PostTypeId="1" CreationDate="2019-12-11T01:17:05.040" Score="3" ViewCount="73" Body="&lt;p&gt;In most current models, the normalization layer is applied after each convolution layer. Many models use the block &lt;span class=&quot;math-container&quot;&gt;$\text{convolution} \rightarrow \text{batch normalization} \rightarrow \text{ReLU}$&lt;/span&gt; repeatedly. But why do we need multiple &lt;a href=&quot;https://arxiv.org/abs/1502.03167&quot; rel=&quot;nofollow noreferrer&quot;&gt;batch normalization&lt;/a&gt; layers? If we have a convolution layer that receives a normalized input, shouldn't it spit out a normalized output? Isn't it enough to place normalization layers only at the beginning of the model?&lt;/p&gt;&#xA;" OwnerUserId="31988" LastEditorUserId="2444" LastEditDate="2019-12-11T13:43:24.057" LastActivityDate="2019-12-11T13:43:24.057" Title="Why do current models use multiple normalization layers?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17037" PostTypeId="1" CreationDate="2019-12-11T06:43:56.753" Score="1" ViewCount="104" Body="&lt;p&gt;I want to recognize the name of the chemical structure from the image of the chemical structure. For example, in the image below, it is a benzene structure, and I want to recognize that it is benzene from the image (I should be able to recognize all these structures as benzene).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I recognize the name of a molecule given an image of its structure?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WD7sc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WD7sc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17183" LastEditorUserId="2444" LastEditDate="2019-12-11T14:01:37.377" LastActivityDate="2019-12-11T14:01:37.377" Title="How can I recognise the name of a molecule given an image of its structure?" Tags="&lt;machine-learning&gt;&lt;object-recognition&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17038" PostTypeId="2" ParentId="13360" CreationDate="2019-12-11T08:13:00.690" Score="0" Body="&lt;p&gt;@mshlis, if &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is covariance matrix, then exist &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; s.t. &lt;span class=&quot;math-container&quot;&gt;$\sigma=AA^T$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we generate &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; by model, and calculate &lt;span class=&quot;math-container&quot;&gt;$\sigma = AA^T$&lt;/span&gt;, then Cholesky decomposition will not successful&lt;/p&gt;&#xA;" OwnerUserId="31479" LastActivityDate="2019-12-11T08:13:00.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17039" PostTypeId="2" ParentId="17036" CreationDate="2019-12-11T11:00:57.020" Score="2" Body="&lt;p&gt;One issue is that a normalized set of initial weights may not stay normalized as learning progresses; so given that we adjust weights proportionately according to their relative values and also when working on a subset of the learning data the model may become convinced that one subset of features is important and others not, this can result in the weights becoming unbalanced again. In the learning curve we may see this as a plateau, where the learning becomes convinced that a few features are much more important than they really are and fails to find new features that can contribute even more because tiny proportionate changes did not move them far, or quickly, enough into a noticeable range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when we re-sample from the database to get the next batch we need to be fully open to the possibility that the previous batch learning was too heavily biased towards its own favoured set. In effect we are trading accuracy for openness to new feature combinations, and at the same time assisting generalization.&lt;/p&gt;&#xA;" OwnerUserId="4994" LastActivityDate="2019-12-11T11:00:57.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17040" PostTypeId="2" ParentId="17031" CreationDate="2019-12-11T11:48:07.950" Score="-1" Body="&lt;p&gt;The issue with Recurrent models is that they don't parallelization during training. &#xA;Sequential models performs better with more memory but faces problem in learning long-term memory dependencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand Transformers take into account of &lt;strong&gt;self attention&lt;/strong&gt; which boosts the speed of how fast the model can translate from one sequence to another and establishes dependencies b/w input and output and focus on relevant parts of the input sequence, which in turn eliminates recurrence and convolution unlike RNNs where sequential computation inhibits parallelization.&lt;/p&gt;&#xA;" OwnerUserId="31407" LastActivityDate="2019-12-11T11:48:07.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17041" PostTypeId="2" ParentId="17023" CreationDate="2019-12-11T12:26:38.217" Score="0" Body="&lt;p&gt;In my opinion, no.&#xA;Also images could be interpreted as tabular dataset as well, where certain columns represent different rgb codes of pixels. If you seek to use neural nets opt for image datasets, with large sample size.&#xA;Neural networks generally require large sample sizes to perform, and huge dimension inputs to not be outperformed by boosting.&lt;/p&gt;&#xA;" OwnerUserId="31970" LastActivityDate="2019-12-11T12:26:38.217" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17043" PostTypeId="1" CreationDate="2019-12-11T15:39:23.340" Score="1" ViewCount="42" Body="&lt;p&gt;I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy, but I'm stuck (and I have a terrible headache). So, I need your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to segment MRI to find white matter hyperintensities. I have a dataset with MRI brain images, and another dataset with the WMH. For each one of the brain images, I have one black image with white dots on it in the WMH dataset. These white dots represent where is a WMH on its corresponding brain image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an image from the MRI brain images:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8znzN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8znzN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is the corresponding WMH image from the WMH dataset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g4jEl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g4jEl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use the other images in network validation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose there will be a loss function and this network is supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2020-06-12T23:54:41.157" LastActivityDate="2020-06-12T23:54:41.157" Title="Using U-NET for image semantic segmentation" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;architecture&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17044" PostTypeId="1" AcceptedAnswerId="17070" CreationDate="2019-12-11T16:26:11.463" Score="6" ViewCount="249" Body="&lt;p&gt;Why is dropout favored compared to reducing the number of units in hidden layers for the convolutional networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a large set of units leads to overfitting and dropping out &quot;averages&quot; the response units, why not just suppress units?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read different questions and answers on the dropout topic including these interesting ones, &lt;a href=&quot;https://ai.stackexchange.com/q/40/2444&quot;&gt;What is the &amp;quot;dropout&amp;quot; technique?&lt;/a&gt; and this other &lt;a href=&quot;https://ai.stackexchange.com/q/9512/2444&quot;&gt;Should I remove the units of a neural network or increase dropout?&lt;/a&gt;, but did not get the proper answer to my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, it is weird that this publication &lt;a href=&quot;http://jmlr.org/papers/volume15/srivastava14a.old/srivastava14a.pdf&quot; rel=&quot;noreferrer&quot;&gt;A Simple Way to Prevent Neural Networks from Overfitting (2014)&lt;/a&gt;, Nitish Srivastava et al., is cited as being the first on the subject. I have just read one that is from 2012:&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1207.0580.pdf&quot; rel=&quot;noreferrer&quot;&gt;Improving neural networks by preventing co-adaptation of feature detectors&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="30392" LastEditorUserId="2444" LastEditDate="2019-12-11T17:24:59.313" LastActivityDate="2021-04-13T15:55:11.737" Title="Why is dropout favoured compared to reducing the number of units in hidden layers?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;dropout&gt;&lt;regularization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17045" PostTypeId="1" AcceptedAnswerId="17054" CreationDate="2019-12-11T17:19:21.927" Score="3" ViewCount="473" Body="&lt;p&gt;Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some reverse (transpose) convolution layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the disadvantage of this network, that directs people to invent fairly complicated things like GANs or VAEs?&lt;/p&gt;&#xA;" OwnerUserId="26791" LastActivityDate="2019-12-12T05:46:43.387" Title="What makes GAN or VAE better at image generation than NN that directly maps inputs to images" Tags="&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;image-generation&gt;&lt;variational-autoencoder&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17046" PostTypeId="2" ParentId="17045" CreationDate="2019-12-11T18:08:50.980" Score="1" Body="&lt;p&gt;&lt;em&gt;I will only focus on the VAE because I am more familiar with it, but the explanations may also apply to the GAN and other generative models.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the VAE, you train a neural network not only to generate images but to represent them &lt;strong&gt;compactly&lt;/strong&gt; in a so-called &lt;strong&gt;latent space&lt;/strong&gt;, so you train the VAE to do &lt;strong&gt;dimensionality reduction&lt;/strong&gt;. More precisely, the VAE attempts to learn a probability distribution with smaller dimensionality than the dimensionality of the training data but that hopefully represents the training data. Consequently, the model is forced to learn the essential features of the probability distribution that generated the training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The VAE is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative model&lt;/a&gt; rather than a &lt;a href=&quot;https://en.wikipedia.org/wiki/Discriminative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;discriminative model&lt;/a&gt;. In the case of the neural network that maps inputs to images, you will not be learning a latent probability distribution (unless you formulate your model in such a way), from which you could sample, but you would just be mapping, in a supervised way (that is, you would need a labelled training dataset) and deterministically, the inputs to the images. In the VAE, there's some form of stochasticity, while training (and testing) the model.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-11T18:14:33.003" LastActivityDate="2019-12-11T18:14:33.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17047" PostTypeId="1" CreationDate="2019-12-11T19:12:22.267" Score="3" ViewCount="133" Body="&lt;p&gt;I'd like to ask for any kind of assistance regarding the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was given the following training data: 100 numbers, each one is a parameter, they together define a number X(also given).This is one instance,I have 20 000 instances for training.Next, I have 5000 lines given, each containing the 100 numbers as parameters.My task is to predict the number X for these 5000 instances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am stuck because I only know of the sigmoid activation function so far, and I assume it's not suitable for cases like this where the output values aren't either 0 or 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is this : What's a good choice for an activation function and how does one go about implementing a neural network for a problem such as this?&lt;/p&gt;&#xA;" OwnerUserId="32012" LastActivityDate="2020-10-08T19:06:45.850" Title="Regression using neural network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;regression&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17049" PostTypeId="1" CreationDate="2019-12-11T21:00:53.167" Score="1" ViewCount="69" Body="&lt;p&gt;I'm new to deep learning. I was wondering what's the relationship between a deep model complexity (e.g. total number of parameters, or depth) and the dataset size? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I want to do a binary classification with 10K data for a problem like fire detection. How should I know what complexity I should go for?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2019-12-11T21:00:53.167" Title="Relationship between model complexity (depth) and dataset size" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;training&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17050" PostTypeId="1" AcceptedAnswerId="18352" CreationDate="2019-12-11T21:36:51.950" Score="6" ViewCount="228" Body="&lt;p&gt;So I’ve been working on my own little dynamic architecture for a deep neural network (any number of hidden layers with any number of nodes in every layer) and got it solving the XOR problem efficiently. I moved on to trying to see if I could train my network on how to classify a number as being divisible by another number or not while experimenting with different network structures and have noticed some odd things. I know this is a weird thing to try and train a neural network to do but I just thought it might be easy because I can simply generate the training data set and test data set programmatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I’ve tested, it seems that my network is only really good at identifying whether or not a number is divisible by a number who is a power of 2. If you test divisibility by a power of two, it converges on a very good solution very quickly. And it generalizes well on numbers outside of the training set - which I guess it kind of makes sense, as I’m inputting the numbers into the network in binary representation, so all the network has to learn is that a number n is only divisible by 2^m if the last m digits in the binary input vector are 0 (i.e. fire the output neuron if the last m neurons on the input layer don't fire, else don't). When checking divisibility by non-powers of two, however, there does not seem to be as much of a &quot;positional&quot; (maybe that's the word, maybe not) relationship between the input bits and whether or not the number is divisible.  I thought though, that if I threw more neurons and layers at the problem that it might be able to solve classifying divisibility by other numbers – but that is not the case. The network seems to converge on not-so-optimal local minima on the cost function (for which I am using mean-squared-error) when dividing by numbers that are not powers of 2. I’ve tried different learning rates as well to no avail. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea what would cause something like this or how to go about trying to fix it? Or are plain deep neural networks maybe just not good at solving these types of problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I should also add that I've tried using different activation functions for different layers (like having leaky-relu activation for your first hidden layer, then sigmoid activation for your output layer, etc.) which has also not seem to have made a difference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code if you feel so inclined as to look at it: &lt;a href=&quot;https://github.com/bigstronkcodeman/Deep-Neural-Network/blob/master/Neural.py&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/bigstronkcodeman/Deep-Neural-Network/blob/master/Neural.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(beware: it was all written from scratch by me in the quest to learn so some parts (namely the back-propagation) are not very pretty - I am really new to this whole neural network thing)&lt;/p&gt;&#xA;" OwnerUserId="32014" LastEditorUserId="32014" LastEditDate="2019-12-11T22:04:24.183" LastActivityDate="2020-03-01T20:31:56.200" Title="Can a deep neural network be trained to classify an integer N1 as being divisible by another integer N2?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17051" PostTypeId="2" ParentId="17047" CreationDate="2019-12-11T21:43:32.357" Score="0" Body="&lt;p&gt;Usually you're normalizing the data first, meaning that your whole dataset will be in between 0 and 1. Afterwords after you're having the model predictions, when computing the cost function or evaluating the model, you can apply the inverse of the normalization function.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2019-12-11T21:43:32.357" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17052" PostTypeId="1" CreationDate="2019-12-12T01:54:03.460" Score="2" ViewCount="48" Body="&lt;p&gt;I'm trying to train a VAE using a graph dataset. However, my latent space shrinks epoch by epoch. Meanwhile, my ELBO plot comes to a steady state after a few epochs.&lt;/p&gt;&#xA;&lt;p&gt;I tried to play around with parameters and I realized, by increasing the batch size or training data, this happens faster, and ELBO comes to a steady state even faster.&lt;/p&gt;&#xA;&lt;p&gt;Is this a common problem, with a general solution?&lt;/p&gt;&#xA;&lt;p&gt;With these signs, which part of the algorithm is more possible to cause the issue? Is it an issue from computing loss function? Does it look like the decoder is not trained well? Or it is more likely for the encoder not to have detected features that are informative enough?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I figured out that the problem is probably caused by the loss function. My loss function is a combination of the KL term and reconstruction loss. In the &lt;a href=&quot;https://github.com/tkipf/gae&quot; rel=&quot;nofollow noreferrer&quot;&gt;github page&lt;/a&gt; for graph auto-encoders, it is suggested that the loss function should include normalization factors according to the number of nodes in the graph. I haven't figured it out exactly, but by adding a factor of 100 to my reconstruction loss and a factor of 0.5 to my KL loss, the algorithm is working fine. I would appreciate it if someone can expand on how this exactly is supposed to be set up.&lt;/p&gt;&#xA;" OwnerUserId="31990" LastEditorUserId="2444" LastEditDate="2020-11-22T17:47:43.610" LastActivityDate="2020-11-22T17:47:43.610" Title="Why does the ELBO come to a steady state and the latent space shrinks?" Tags="&lt;variational-autoencoder&gt;&lt;graphs&gt;&lt;evidence-lower-bound&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17053" PostTypeId="2" ParentId="17050" CreationDate="2019-12-12T03:10:05.077" Score="1" Body="&lt;p&gt;When you represent a number in base 2 (binary), you have already divided the number by 2 many times.  If there is no remainder at the end, the number is obviously evenly divisible by 2.  This hints that your AI could test for divisibility by dividing.  Hmm-- not much gained there!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately the problem is not one suited to solving via AI.  That's why factorization of large numbers is a good basis for hard encryption schemes.  I'd suggest finding a different sort of problem on which to test your AI.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2019-12-12T03:10:05.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17054" PostTypeId="2" ParentId="17045" CreationDate="2019-12-12T05:46:43.387" Score="2" Body="&lt;p&gt;The only disadvantage and difference between these generative models and the method you describe, is the input. You describe inputting tags, where as for a GAN, or VAE, the generation segment of the model takes in some representation of a probability distribution. For a GAN, it's mostly random noise, and for a VAE it is some latent space (see nbros answer).&lt;/p&gt;&#xA;&lt;p&gt;Your described method prevents the network from properly learning fluid generation. If you have a discrete input, the network will attempt to perform a sort of classification on the input, rather then generation, and so when you try to generate a new image, you will most likely get the image equivalent of gibberish.&lt;/p&gt;&#xA;&lt;p&gt;In fact, that's why a standard auto-encoder (not variational) doesn't work very well for generation. You would think that you could feed in your own custom input into the latent space:&lt;a href=&quot;https://i.stack.imgur.com/NZ8ze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NZ8ze.png&quot; alt=&quot;Example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But if you tried this, you would end up telling the network to try and generate something from a latent space it can't interpret.&lt;/p&gt;&#xA;&lt;p&gt;Hence where the &amp;quot;variational&amp;quot; part of the VAE comes in. This helps the network learn to generate from a &lt;em&gt;continuous&lt;/em&gt; distribution, so no matter what input you use, the network will be able to interpret it and give an appropriate output.&lt;/p&gt;&#xA;&lt;p&gt;As for a GAN, it is simply fed random noise at each training step, so it too generates based on a continuous distribution.&lt;/p&gt;&#xA;&lt;p&gt;If you were to try and train your method of generation, I would predict that you would find an average of all images that share similar tags (say you have the tags of cat, dog and brown haired, if you input &amp;quot;dog=1, cat=0, brown haired=1&amp;quot; you would get an average of all brown haired dogs), but if you tried to input a combination of tags the network has not seen, as it has not learnt from a continuous distribution, the resultant image would not be anything like what you'd expect from those tags.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-12-12T05:46:43.387" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17055" PostTypeId="1" CreationDate="2019-12-12T05:50:54.390" Score="4" ViewCount="80" Body="&lt;p&gt;In the &lt;a href=&quot;https://nlp.stanford.edu/pubs/glove.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original GloVe paper&lt;/a&gt;, the authors discuss group theory when coming up with equation (4). Is it possible that the authors came up with this model, figured out it was good, and then later found out various group theory justifications that justified it? Or was it discovered sequentially as it is described in the paper? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question.&lt;/strong&gt; In AI research, are most things generally discovered because they work empirically and later justified mathematically, or is it the other way around?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22233" LastEditorUserId="2444" LastEditDate="2019-12-12T11:59:09.287" LastActivityDate="2020-01-12T22:01:05.153" Title="Are most things generally discovered because they work empirically and later justified mathematically, or vice-versa?" Tags="&lt;research&gt;&lt;papers&gt;&lt;glove&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17056" PostTypeId="1" AcceptedAnswerId="17059" CreationDate="2019-12-12T05:52:49.207" Score="0" ViewCount="100" Body="&lt;p&gt;I have experience in making several Artificial Neural Networks and some programs which may be classified as an Artificial Intelligence by using &lt;a href=&quot;https://www.tensorflow.org/js&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow.js&lt;/a&gt; and &lt;a href=&quot;https://brain.js.org/#/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brain.js&lt;/a&gt;. In order to produce something that may be classified as a deep learning AI, what are the characteristics of a Deep Learning AI?&lt;/p&gt;&#xA;" OwnerUserId="31949" LastEditorUserId="2444" LastEditDate="2019-12-12T10:52:04.943" LastActivityDate="2019-12-12T10:52:04.943" Title="What are the characteristics of a deep learning AI?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17057" PostTypeId="2" ParentId="17025" CreationDate="2019-12-12T05:55:35.473" Score="2" Body="&lt;p&gt;You invoke it in a loop. Imagine a digital assistant responding to voice queries. It might look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(;;) {&#xA;   var audio = RecordSomeAudio();&#xA;   var response = model.predict(audio);&#xA;   if(response.action == &quot;SAYSOMETHING&quot;) {&#xA;      PlaySomeAudio(response.output);&#xA;   }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the model gets invoked repeatedly and can decide in a given situation whether to respond or not. In a digital assistant context, part of the model would be to check for if the user raised a query (e.g. &quot;Hey Google&quot; etc.).&lt;/p&gt;&#xA;" OwnerUserId="32020" LastActivityDate="2019-12-12T05:55:35.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17058" PostTypeId="1" CreationDate="2019-12-12T06:14:11.573" Score="1" ViewCount="31" Body="&lt;p&gt;This might seem like a really silly question, however I have not been able to find any answers to it on the internet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my rough understanding of data assimilation, it combines data with numerical models by having weights on the adjustment initial condition parameters? That sounds really similar to what machine learning/neural network does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the distinct differences?&lt;/p&gt;&#xA;" OwnerUserId="32021" LastActivityDate="2019-12-12T06:14:11.573" Title="How are data assimilation and machine learning different?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17059" PostTypeId="2" ParentId="17056" CreationDate="2019-12-12T06:39:42.970" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tzarz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tzarz.png&quot; alt=&quot;Deep learning is a&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see from the picture above (In courtesy to this guy &lt;a href=&quot;https://www.deeplearning-academy.com/p/ai-wiki-machine-learning-vs-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt;). Deep learning is a field inside machine learning which involves using deep neural networks to solve machine learning problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensorflow.js and Brain.js are software architectures built in order to assemble and develop ML models inside the browser. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to delve into this field of deep neural networks you should start from the theory to really understand why we are using neural networks to solve problems and which problems they are intended to solve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can start with this awesome video by 3blue1brown : &lt;a href=&quot;https://www.youtube.com/watch?v=aircAruvnKk&amp;amp;list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intro video&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this standford class for CNN: &lt;a href=&quot;https://www.youtube.com/watch?v=vT1JzLTH4G4&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford class&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31957" LastActivityDate="2019-12-12T06:39:42.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17061" PostTypeId="1" CreationDate="2019-12-12T07:21:09.450" Score="1" ViewCount="28" Body="&lt;p&gt;If not perfect, how well can they do? For example, if I give the Seq2Seq setup a name it did not see in the training process, can it output the same name without error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;name = &quot;Will Smith&quot;&#xA;output = DecoderRNN(EncoderRNN(name))&#xA;can_this_be_true = name == output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32023" LastActivityDate="2020-01-12T09:02:10.287" Title="Can a character-level Seq2Seq setup learn to perfectly reconstruct structured data like name strings?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17063" PostTypeId="1" CreationDate="2019-12-12T09:13:28.860" Score="2" ViewCount="31" Body="&lt;p&gt;Consider a fixed camera that records a given area. Three things can happen in this area: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No action&lt;/li&gt;&#xA;&lt;li&gt;People performing action A&lt;/li&gt;&#xA;&lt;li&gt;People performing action B&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to train a model to detect when action B happens. A human observer could typically recognize action B even with a single frame, but it would be much easier with a short video (a few seconds at low FPS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the most suitable models for this task? I read &lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42455.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; where different types of fusion are performed in order to feed different frames to a CNN. Are there better alternatives?&lt;/p&gt;&#xA;" OwnerUserId="16671" LastActivityDate="2019-12-12T10:46:55.693" Title="Most suitable model for video classification with a fixed camera" Tags="&lt;convolutional-neural-networks&gt;&lt;video-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17064" PostTypeId="1" AcceptedAnswerId="17091" CreationDate="2019-12-12T09:25:21.710" Score="0" ViewCount="76" Body="&lt;p&gt;How can I detect diagram region and extract(crop) it from a research paper&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FOiST.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FOiST.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17183" LastActivityDate="2019-12-14T04:59:27.803" Title="How can I detect diagram region and extract (crop) it from a research paper" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;object-detection&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2019-12-14T06:19:39.593" ContentLicense="CC BY-SA 4.0" />
  <row Id="17066" PostTypeId="2" ParentId="17004" CreationDate="2019-12-12T10:17:52.360" Score="4" Body="&lt;p&gt;For a 3 channel image (RGB), each filter in a convolutional layer computes a &lt;strong&gt;feature map&lt;/strong&gt; which is essentially a single channel image.  Typically, 2D convolutional filters are used for multichannel images. This can be a single filter applied to each layer or a seperate filter per layer. These filters are looking for features which are independent of the color, i.e. edges (if you are looking for color there are far easier ways than CNNs). The filter is applied to each channel and the results are combined into a single output, the feature map. Since all channels are used by the filter to compute a single feature map, the number of channels in the input does not affect the structure of the network beyond the first layer. The size of a feature map is determined by the &lt;strong&gt;filter size&lt;/strong&gt;, &lt;strong&gt;stride&lt;/strong&gt;, &lt;strong&gt;padding&lt;/strong&gt; and &lt;strong&gt;dilation&lt;/strong&gt;(not commonly used - see &lt;a href=&quot;https://towardsdatascience.com/understanding-2d-dilated-convolution-operation-with-examples-in-numpy-and-tensorflow-with-d376b3972b25&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; if you are interested.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, a &lt;code&gt;30 x 30 x 3&lt;/code&gt; input convolved with 10 &lt;code&gt;5 x 5&lt;/code&gt; filters will yield a volume of &lt;code&gt;30 x 30 x 10&lt;/code&gt; &lt;strong&gt;if&lt;/strong&gt; the filters have a stride of &lt;strong&gt;1&lt;/strong&gt; and &lt;strong&gt;same&lt;/strong&gt; padding (or, &lt;code&gt;26 x 26 x 10&lt;/code&gt; with &lt;strong&gt;valid&lt;/strong&gt; padding / &lt;code&gt;34 x 34 x 10&lt;/code&gt; with &lt;strong&gt;full&lt;/strong&gt; padding).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Same&lt;/strong&gt; padding buffers the edge of the input with &lt;em&gt;filter_size&lt;/em&gt;/2 (integer division) to yield an output of equal size (assuming stride is 1) while &lt;strong&gt;valid&lt;/strong&gt; padding would result in a smaller output. Valid padding doesn't crop the image as you said, it's more of a dilution of the signal at the edges, however the results is essentially the same. Note that even with same padding the edge pixels are used in fewer convolutions - a &lt;code&gt;5 x 5&lt;/code&gt; filter with same padding will use a central pixel 25 times (every position on the filter) but only 9 times for a corner pixel. To use all pixels evenly &lt;strong&gt;full&lt;/strong&gt; padding must be used which buffers the edge of the input with &lt;em&gt;filter_size&lt;/em&gt; - 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;a href=&quot;https://i.stack.imgur.com/7HSXgm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7HSXgm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each feature map becomes a channel in the output volume. Therefore, &lt;strong&gt;the number of channels in the output volume is always equal to the number of filters in the convolutional layer&lt;/strong&gt;. So, the second layer would output a volume of size &lt;code&gt;30 x 30 x 5&lt;/code&gt; (stride 1, same padding).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last layer in your example (fully connected) multiplies the value of each pixel in each feature map by a learned weight and sums the result. If the network is a binary classifier, the summed value results in a 1 or 0 output if a threshold is reached or as a decimal value for a regression model. This is determined by the FC neurons' activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If visualizing this helps you as much as it helps me, I highly recommend having a look at the interactive examples &lt;a href=&quot;https://tensorspace.org/html/playground/lenet.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Note that what is shown by this tool is the signal propagating through the network, i.e. the output from each layer, not the filters/weights themselves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in a bit more depth about ANNs and convolutional layers, I cover all the basics in my &lt;a href=&quot;https://github.com/tynowell/rotdetection-CNN/blob/master/Detection%20and%20Quantification%20of%20Rot%20in%20Harvested%20Trees%20using%20Convolutional%20Neural%20Networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;thesis&lt;/a&gt;(this is where the image is from) - p.9-16 ANNs &amp;amp; p.16-23 CNNs.&lt;/p&gt;&#xA;" OwnerUserId="31980" LastEditorUserId="31980" LastEditDate="2019-12-12T10:25:11.933" LastActivityDate="2019-12-12T10:25:11.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17067" PostTypeId="2" ParentId="17063" CreationDate="2019-12-12T10:40:39.480" Score="3" Body="&lt;p&gt;It seems you need a spatio-temporal model to learn human-body detection and action.&#xA;With regards to interesting papers on the subject I would recommand to look at &lt;a href=&quot;http://vicky.kalogeiton.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vicky Kalogeiton web site&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Her PhD thesis &#xA;&lt;a href=&quot;http://calvin.inf.ed.ac.uk/wp-content/uploads/Publications/theses/Kalogeiton2017.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2017, V. Kalogeiton, Localizing spatially and temporally objects and actions in videos, &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;basically cover her 3 papers on the subject:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1501.01186.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2016, Kalogeiton, V., Schmid, C., and Ferrari, V. Analysing domain shift factors between videos and images for object detection&lt;/a&gt;,  &lt;/li&gt;&#xA;&lt;li&gt;[2017a, Kalogeiton, V., Weinzaepfel, P., Ferrari, V., and Schmid, C. Action tubelet detector for spatio-temporal action localization]&lt;a href=&quot;https://arxiv.org/abs/1705.01861&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kalogeiton 2017a&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://hal.inria.fr/hal-01575804/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;2017b, Kalogeiton, V., Weinzaepfel, P., Ferrari, V., and Schmid, C. Joint learning of object and action detectors&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;summary of Kalogeiton's PhD introduction .&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Introduce an end-to-end multitask objective that jointly learns object-action relationships. &#xA;The action-object detector leverages the temporal continuity of videos. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though &lt;strong&gt;intra class variations&lt;/strong&gt; are key and appears as spatial location accuracy, appearance diversity, image quality, aspect distribution, and object size and camera framing.&lt;br&gt;&#xA;Actions class refers to an atomic class such as jump, walk, run, climb, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The detector builds &lt;code&gt;anchor cuboids&lt;/code&gt; named &lt;strong&gt;tubelets&lt;/strong&gt; and defined as sequences of bounding boxes with associated scores. The action detection spans over a period of time (first and last video frame detected) and takes place at a specific location in each frame. Intra frame action detection can be ambigous. On the other way a sequence bears more information (&lt;strong&gt;across class similarities&lt;/strong&gt;) than a single frame to infer action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most previous work uses per-frame object detections, and then leverage the motion of objects to refine their spatial localization or improve their classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contributions&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;differences between still and video frames for training and testing an object detector among which (see Chapter 3 for more details ):&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;spatial location accuracy, &lt;/li&gt;&#xA;&lt;li&gt;appearance diversity,&lt;/li&gt;&#xA;&lt;li&gt;image quality, &lt;/li&gt;&#xA;&lt;li&gt;aspect distribution, &lt;/li&gt;&#xA;&lt;li&gt;camera framing&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;jointly detect object-action instances in uncontrolled videos using an end-to-end two stream network architecture (see chapter 4 for more details )&lt;/li&gt;&#xA;&lt;li&gt;propose the &lt;strong&gt;ACtion Tubelet detector (ACT-detector)&lt;/strong&gt;, which takes as input a sequence of frames and outputs &lt;strong&gt;tubelets&lt;/strong&gt;, i.e. sequences of bounding boxes with associated scores (see chapter 5 for more details).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30392" LastEditorUserId="30392" LastEditDate="2019-12-12T10:46:55.693" LastActivityDate="2019-12-12T10:46:55.693" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17068" PostTypeId="1" CreationDate="2019-12-12T12:41:17.693" Score="1" ViewCount="29" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GXLiU.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GXLiU.jpg&quot; alt=&quot;Question&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anyone able to explain how to do this? I'm not looking for the complete answer, I would settle for a &quot;how to for dummies&quot; explanation of how this is supposed to be solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand constraints, but in the first example it would seem to me that the second half of the first partial assignment &lt;span class=&quot;math-container&quot;&gt;$x_2=-1$&lt;/span&gt; is a violation of the constraint 2, that says &lt;span class=&quot;math-container&quot;&gt;$x_1 &amp;gt; x_2$&lt;/span&gt;... when down below it says both &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;, are &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32036" LastEditorUserId="30789" LastEditDate="2019-12-12T17:06:40.013" LastActivityDate="2019-12-12T17:06:40.013" Title="CSP Formulation of an algebraic problem" Tags="&lt;linear-algebra&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17069" PostTypeId="1" CreationDate="2019-12-12T14:47:32.530" Score="1" ViewCount="37" Body="&lt;p&gt;I'm using &lt;strong&gt;DQN&lt;/strong&gt; to train multi-version of the same system and there is a small difference when I run them both separately. However, my result suddenly dropped in both versions if I run them both at the same time.  I tried again but I got the same results with slightly different.  Would it be affecting my results if I run multi-version of my system at the same time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any explanation for that and How can I get accurate results when  I train multi-version of the same system at the same time? &lt;/p&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="21181" LastEditDate="2019-12-18T14:56:17.090" LastActivityDate="2019-12-18T14:56:17.090" Title="Is the training of multi-version of the same system at the same time affecting the results?" Tags="&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17070" PostTypeId="2" ParentId="17044" CreationDate="2019-12-12T20:15:21.563" Score="5" Body="&lt;p&gt;Dropout only ignores a portion of units during a single training batch update.  Each training batch will use a different combination of units which gives them the best chance of that portion of them working together to generalize. Note the weights for each unit are kept and will be updated during the next batch in which that unit is selected. During inference, yes, all units are used (with a factor applied to activation...the same factor that defines the fraction used)...this becomes essentially an ensemble of all the different combinations of units that were used.&lt;/p&gt;&#xA;&lt;p&gt;Contrasted with fewer units, the fewer units approach will only learn what those fewer units can be optimized for.  Think of dropout as an ensemble of layers of fewer units.(with the exception that there are partial weight sharing between the layers)&lt;/p&gt;&#xA;" OwnerUserId="31608" LastEditorUserId="36737" LastEditDate="2021-04-13T15:55:11.737" LastActivityDate="2021-04-13T15:55:11.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17071" PostTypeId="2" ParentId="17044" CreationDate="2019-12-12T21:01:00.050" Score="2" Body="&lt;p&gt;The idea of dropout is that, &lt;strong&gt;at training time&lt;/strong&gt;, with a certain probability &lt;span class=&quot;math-container&quot;&gt;$p_i \in [0, 1]$&lt;/span&gt;, the unit (or neuron) &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is dropped, &lt;span class=&quot;math-container&quot;&gt;$\forall i$&lt;/span&gt;, that is, the output of unit &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is set to zero so that &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; does not affect the other units it is connected to, both during the forward and backward (or back-propagation) passes (or steps). At every mini-batch, you randomly drop usually different units, so, across different mini-batches (and consequently epochs), you do &lt;strong&gt;not&lt;/strong&gt; always or necessarily drop the same units. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The title of the paper &lt;a href=&quot;https://arxiv.org/pdf/1207.0580.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving neural networks &lt;strong&gt;by preventing co-adaptation of feature detectors&lt;/strong&gt;&lt;/a&gt; emphasizes that dropout prevents the co-adaptation of the units (the feature detectors), so units attempt to detect certain features independently of other units, which reduces overfitting, that is, it improves the generalization ability of the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At test time, no unit is usually dropped. However, there is an approximation of a &lt;a href=&quot;http://proceedings.mlr.press/v31/damianou13a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep Gaussian process&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1505.05424&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural network&lt;/a&gt; that is based on the application of dropout at training and test times. This is called &lt;strong&gt;Monte Carlo dropout&lt;/strong&gt; or, in short, MC dropout, for reasons you can understand if you read the paper &lt;a href=&quot;https://arxiv.org/pdf/1506.02142.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also the possibility to drop the connections between the neurons, which is called &lt;a href=&quot;http://proceedings.mlr.press/v28/wan13.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DropConnect&lt;/a&gt;, rather than the neurons themselves. These two approaches are slightly different, even though DropConnect can be seen as a generalization of dropout. In DropConnect, you do not switch off completely the units, but only the contributions of certain units to the output of other units. In dropout, you completely switch off certain units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decided to &lt;strong&gt;deterministically&lt;/strong&gt; (and manually) reduce the number of units before training, essentially, you would train a &lt;strong&gt;fixed&lt;/strong&gt; smaller network, but this will not necessarily reduce overfitting or, more precisely, co-adaptation of the units. In dropout, you &lt;strong&gt;randomly&lt;/strong&gt; select the units to drop, so, at every mini-batch (or epoch, depending on the implementation of dropout), you effectively train a &lt;strong&gt;random&lt;/strong&gt; subset of the units of the original neural network and, because of this, it can be thought of as an ensemble of smaller neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two papers &lt;a href=&quot;https://arxiv.org/pdf/1207.0580.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving neural networks by preventing co-adaptation of feature detectors&lt;/a&gt; (2012) and &lt;a href=&quot;http://jmlr.org/papers/volume15/srivastava14a.old/srivastava14a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout: A Simple Way to Prevent Neural Networks from Overfitting&lt;/a&gt; (2014) have exactly the same authors, but the latter paper was published in the &lt;a href=&quot;http://www.jmlr.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal of Machine Learning Research&lt;/a&gt;, while the former wasn't apparently published in any journal. In fact, &lt;a href=&quot;http://jmlr.org/papers/volume15/srivastava14a.old/srivastava14a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout: A Simple Way to Prevent Neural Networks from Overfitting&lt;/a&gt; does not even cite &lt;a href=&quot;https://arxiv.org/pdf/1207.0580.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving neural networks by preventing co-adaptation of feature detectors&lt;/a&gt;, but it cites the master's thesis &lt;a href=&quot;http://www.cs.toronto.edu/~nitish/msc_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving Neural Networks with Dropout&lt;/a&gt; (2013) by Nitish Srivastava, who is one of the authors of dropout.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-12T22:13:52.277" LastActivityDate="2019-12-12T22:13:52.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17072" PostTypeId="1" CreationDate="2019-12-12T22:24:36.903" Score="1" ViewCount="50" Body="&lt;p&gt;My professor gave us a workshop where we have to do classification of a dataset of ECG signals between healthy and unhealthy types using LSTM. Each signal consists of 1,285 time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What my prof did was to cut up each signal into segments 24 time steps long, advancing 6 steps for the next segment. In other words, for the following signal&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0, 1, 2, ... 1283, 1284, 1285&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It will be cut up into the following segments&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0, 1, 2, ... 21, 22, 23&#xA;6, 7, 8, ... 27, 28, 29&#xA;12, 13, 14, ... 33, 34, 35&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These segments are the input to the LSTM model for each signal to be classified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the code that my prof used to cut the signal into segments, and feeding that into Tensorflow-Keras InputLayer, it tells me that the output shape is &lt;code&gt;(None, 211, 24)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am told by a classmate that the correct implementation for Tensorflow-Keras LSTM should be &lt;code&gt;(None, 24, 211)&lt;/code&gt;. He tried clarifying with the prof but it seems the prof doesn't really understand what my classmate's trying to say. I tried to Google but examples I can find online are either of the two cases below:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The format of an input signal should be &lt;code&gt;(None, # timesteps, # features)&lt;/code&gt;. However, in my problem the signal only has one feature that is chopped up into segments. In this example I found online, there's no mention of segments.&lt;/li&gt;&#xA;&lt;li&gt;The only example of signals being cut into segments I can find is when each segment is a single input. That is not the case for me. My input is a single signal, cut up into 211 segments which collectively make up a single input.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which is the correct output shape for my LSTM model? My prof used his method and achieved 96% classification accuracy, and our assignment is to surpass that rate, but when I tested using what my classmate said is the correct shape, my LSTM model with the exact same architecture, hyperparameters etc. gave me a flat 74.04% accuracy from the 1st epoch all the way to the end without ever changing. So which is wrong?&lt;/p&gt;&#xA;" OwnerUserId="23844" LastActivityDate="2019-12-12T22:24:36.903" Title="What is the correct input shape for my LSTM network?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17073" PostTypeId="2" ParentId="17028" CreationDate="2019-12-13T04:19:54.887" Score="3" Body="&lt;p&gt;Perhaps Occam's razor counts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Occam's razor is the meta-heuristic that &quot;the simplest explanation is the most likely to be correct&quot;. I consider it a meta-heuristic because itself doesn't provide explanations, only means of comparing them among another. It is a heuristic because like all heuristics, it can lead to false conclusions, but is usually right.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2019-12-13T04:19:54.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17074" PostTypeId="1" AcceptedAnswerId="17131" CreationDate="2019-12-13T06:55:16.867" Score="1" ViewCount="97" Body="&lt;p&gt;I need to understand whether it is better to use AI algorithms (ML, DL, etc.) instead of the classic parser (based onto grammars with regular expression and automaton) for the following task: structuring in XML an unstructured plain text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The text is a legal document, so the structure is well defined and a classic parser could do a good job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case AI could be a viable way, what would be an appropriate approach for the task?&lt;/p&gt;&#xA;" OwnerUserId="32050" LastEditorUserId="2444" LastEditDate="2019-12-13T12:55:14.640" LastActivityDate="2019-12-16T18:09:55.613" Title="Would AI be appropriate for converting unstructured text into an XML?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17075" PostTypeId="2" ParentId="17074" CreationDate="2019-12-13T08:38:04.373" Score="3" Body="&lt;p&gt;A rule-based approach will guarantee a correct result, and it works perfectly fine. On the other hand, an AI based approach will introduce errors as AI cannot produce result with 100% accuracy, and also decreasing speed. As the document you are processing is a legal document, it would be better to use a parser as AI is only adding unnecessary wasted time and non-accurate results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-12-13T08:38:04.373" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17076" PostTypeId="2" ParentId="17061" CreationDate="2019-12-13T08:42:54.767" Score="1" Body="&lt;p&gt;If you give it a name, it will probably be &lt;strong&gt;almost&lt;/strong&gt; perfect model, as the number of hidden units are definitely enough to store the raw data of a name. However as a neural network it will still not be perfect. There may also be serious overfitting if you do this. A Seq2Seq model have way more parameters than necessary to just &quot;remembers&quot; all the possible names, as names variants are not a lot. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-12-13T08:42:54.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17077" PostTypeId="1" CreationDate="2019-12-13T09:05:59.660" Score="1" ViewCount="39" Body="&lt;p&gt;We are having suspects in salary distribution in our organisation due to employee's region. The data we have is as the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Name&#xA;Region&#xA;Work Position (4 main positions)&#xA;Salary&#xA;Gender&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What technique should we use in Machine learning to check and detect malicious salary distribution? By using clustering?&lt;/p&gt;&#xA;" OwnerUserId="26028" LastEditorUserId="26028" LastEditDate="2019-12-13T11:58:58.100" LastActivityDate="2019-12-13T13:53:53.273" Title="How to detect patterns in salary distribution if we are suspecting malicious distribution based on employee's region?" Tags="&lt;machine-learning&gt;&lt;pattern-recognition&gt;&lt;clustering&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17079" PostTypeId="2" ParentId="17077" CreationDate="2019-12-13T13:53:53.273" Score="2" Body="&lt;p&gt;A simple initial approach would be to separate it by position and check for each:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use linear regression: &lt;span class=&quot;math-container&quot;&gt;$\hat{salary} = \sum_i \alpha_i * \hat{region}_i + \sum_k \beta_k * \mathbf{1}[\hat{gender}=k]$&lt;/span&gt; and now you have an intuitive measure by looking at &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;'s and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;'s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 issues that may arise with this method:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This assumes though that a linear model is a good fit, which it very well might not be   &lt;/li&gt;&#xA;&lt;li&gt;The amount of data points may be small and this will give you point estimates without any sense of confidence.   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;2 possible respective solutions:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a stronger model with less intuitive parameters (like a neural network) and then use common visualization techniques to determine weight of each input (this could be as simple as changing them and looking at the outcome or something more complex that utilizes the gradient), but even more complex models can still suffer from the second problem, which is why id reccommend option 2&lt;/li&gt;&#xA;&lt;li&gt;Use a Bayesian model, you can make something hierarchical something as simple as doing the above linear regression with a non informative prior and eventually you'd be able to build confidence intervals and get a better sense of the associations (note if youre looking for what to write this in, I recommend JAGS for its ease of use)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-12-13T13:53:53.273" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17080" PostTypeId="1" CreationDate="2019-12-13T15:41:47.887" Score="4" ViewCount="207" Body="&lt;p&gt;When exactly is a model considered over-parameterized?&lt;/p&gt;&#xA;&lt;p&gt;There are some recent researches in Deep Learning about the role of over-parameterization toward generalization, so it would be nice if I can know what exactly can be considered as such.&lt;/p&gt;&#xA;&lt;p&gt;A hand-wavy definition is: over-parameterized model is often used to described when you have a model bigger than necessary to fit your data.&lt;/p&gt;&#xA;&lt;p&gt;In some papers (for example, in &lt;a href=&quot;https://arxiv.org/pdf/1811.03962.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Convergence Theory for Deep Learning&#xA;via Over-Parameterization&lt;/a&gt;), over-parameterization is described as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;they have much more parameters than the number of training samples&lt;/p&gt;&#xA;&lt;p&gt;meaning that the number of neurons is polynomially large comparing to the input size&lt;/p&gt;&#xA;&lt;p&gt;the network width is sufficiently large: polynomial in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, the number of layers, and in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, the number of samples&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Shouldn't this definition depend on the type of input data as well?&lt;/p&gt;&#xA;&lt;p&gt;For example, I fit:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;1M-parameters model on 10M samples of 2 binary features, then it should not be over-parameterized, or&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;1M-parameters model on 0.1M samples of 512x512 images, then is over-parameterized, or&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the model in the paper &lt;a href=&quot;https://research.fb.com/wp-content/uploads/2018/05/exploring_the_limits_of_weakly_supervised_pretraining.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exploring the Limits of&#xA;Weakly Supervised Pretraining&lt;/a&gt; &amp;quot;IG-940M-1.5k ResNeXt-101 32×48d&amp;quot; with 829M parameters, trained on 1B Instagram images, is not over-parameterized&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32055" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-14T18:00:30.200" Title="When exactly is a model considered over-parameterized?" Tags="&lt;deep-learning&gt;&lt;definitions&gt;&lt;convergence&gt;&lt;generalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17081" PostTypeId="1" CreationDate="2019-12-13T15:54:49.970" Score="1" ViewCount="41" Body="&lt;p&gt;I made the following HTML  nd javascript to predict &lt;span class=&quot;math-container&quot;&gt;$f(n) = 2n$&lt;/span&gt;.&#xA;Basically, I am trying to design my first neural network which predicts 2 multiplied by a number. I know we don't need a neural network for this, but just for a beginner, can you tell how I can make it work properly? For example, when I input 2 it gives &lt;span class=&quot;math-container&quot;&gt;$89.7614166111441$&lt;/span&gt;. What's wrong with my implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my HTML file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;!DOCTYPE html&amp;gt;&#xA;&amp;lt;html&amp;gt;&#xA;&#xA;&amp;lt;head&amp;gt;&#xA;&#xA;&amp;lt;/head&amp;gt;&#xA;&#xA;&amp;lt;body&amp;gt;&#xA;    &amp;lt;script src=&quot;l1.js&quot; charset=&quot;utf-8&quot;&amp;gt;&amp;lt;/script&amp;gt;&#xA;    &amp;lt;script type=&quot;text/javascript&quot;&amp;gt;&#xA;        //aa();alert(wheights[1]);&#xA;        //amulth(lh);&#xA;&#xA;        /*alert(famulth([1,2],2,[0,0]));&#xA;        alert(amulth([2,1],[1,2],[1,1]));&#xA;        alert(amulth([2],[0],[1]));*/&#xA;&#xA;        //alert(eamulth([2,2],[2,2],[1,1]));&#xA;&#xA;        var n = 0;&#xA;        var nn = 0;&#xA;        var nnn = 0;&#xA;        var nnnn = 0;&#xA;&#xA;        while (nnn &amp;lt; 100000000 /*0*/ ) {&#xA;            nnnn = Math.round(100 * Math.random());&#xA;            n = eamulth(lh1, amulth(lhlh, famulth(lh, nnnn + 0 /**parseFloat(prompt(&quot;Enter a number&quot;))*/ , lh1lh), lhl), lh1l);&#xA;            //alert(n);&#xA;            /*n=1;*/&#xA;            nn = /*parseFloat(prompt(&quot;&quot;+n))*/ 2 * nnnn - n;&#xA;            lh = t(lh, nn, 0.1);&#xA;            lh1 = t(lh1, nn, 0.01);&#xA;            /*lh1l=t(lh1l,nn,0.01);&#xA;            lhl=t(lhl,nn,0.01);*/&#xA;            nnn++;&#xA;            //alert(lh);&#xA;            lhlh = t(lhlh, nn, 0.01);&#xA;            lh1lh = t(lh1lh, nn, 0.01);&#xA;&#xA;        }&#xA;        alert(eamulth(lh1, famulth(lh, parseFloat(prompt(&quot;Enter a number&quot;)), lhl), lh1l));&#xA;        alert(eamulth(lh1, amulth(lhlh, famulth(lh, nnnn + 0 /**parseFloat(prompt(&quot;Enter a number&quot;))*/ , lh1lh), lhl), lh1l));&#xA;    &amp;lt;/script&amp;gt;&#xA;&#xA;&amp;lt;/body&amp;gt;&#xA;&#xA;&amp;lt;/html&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the Javascript file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//l1.js&#xA;//function aa(){alert('aa');}&#xA;var lh = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0];&#xA;var lh1 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0];&#xA;var lhl = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0];&#xA;var lh1l = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0];&#xA;lh = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1];&#xA;lh1 = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1];&#xA;lhl = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1];&#xA;lh1l = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1];&#xA;&#xA;&#xA;var lhlh = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1];&#xA;var lh1lh = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1];&#xA;&#xA;function amulth(hh, hhh, aa) {&#xA;    /*alert(hh[0]);*/&#xA;&#xA;    var ta = [];&#xA;    var s = 0;&#xA;&#xA;    for (var i = 0; i &amp;lt; hh.length; i++) {&#xA;        for (var j = 0; j &amp;lt; hhh.length; j++) {&#xA;&#xA;&#xA;            s += hhh[j] * hh[i];&#xA;        }&#xA;        s += 0 + 0 + aa[i];&#xA;        ta.push(s);&#xA;        s = 0;&#xA;    }&#xA;    return ta;&#xA;&#xA;}&#xA;&#xA;function famulth(hh1, hhh1, aa1) {&#xA;    /*alert(hh[0]);*/&#xA;&#xA;    var ta1 = [];&#xA;&#xA;    for (var i1 = 0; i1 &amp;lt; hh1.length; i1++) {&#xA;        ta1.push(hh1[i1] * hhh1 + aa1[i1]);&#xA;    }&#xA;    return ta1;&#xA;&#xA;}&#xA;&#xA;&#xA;&#xA;function eamulth(hh2, hhh2, aa) {&#xA;    /*alert(hh[0]);*/&#xA;&#xA;    var ta2 = 0;&#xA;    for (var i2 = 0; i2 &amp;lt; hh2.length; i2++) {&#xA;&#xA;        ta2 += hhh2[i2] * hh2[i2] + aa[i2];&#xA;&#xA;    }&#xA;    return ta2;&#xA;}&#xA;&#xA;function t(hh3, d1, lhhhh) {&#xA;    for (var il = 0; il &amp;lt; hh3.length; il++) {&#xA;        hh3[il] = hh3[il] + d1 * lhhhh * 0.0001;&#xA;    }&#xA;    return hh3;&#xA;}    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32056" LastEditorUserId="2444" LastEditDate="2019-12-13T16:12:30.330" LastActivityDate="2019-12-13T16:12:30.330" Title="How can I build a model to approximate the function $f(n) = 2n$?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;supervised-learning&gt;&lt;function-approximation&gt;&lt;javascript&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17083" PostTypeId="1" CreationDate="2019-12-13T18:19:47.407" Score="1" ViewCount="289" Body="&lt;p&gt;I recently bought a Jetson Nano and I'm amazed with everything about it. But I don't know what is happening, because I created a very simple neural network with keras and it's taking way to long. I know is taking to long, because I runned the same ANN in my PC's CPU and it was faster than the jetson nano.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import pandas as pd&#xA;&#xA;dataset = pd.read_csv('Churn_Modelling.csv')&#xA;X = dataset.iloc[:, 3:13].values&#xA;y = dataset.iloc[:, 13].values&#xA;&#xA;from sklearn.preprocessing import LabelEncoder, OneHotEncoder&#xA;labelencoder_X_1 = LabelEncoder()&#xA;X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])&#xA;labelencoder_X_2 = LabelEncoder()&#xA;X[:, 2] = labelencoder_X_2.fit_transform(X[:, 2])&#xA;onehotencoder = OneHotEncoder(categorical_features = [1])&#xA;X = onehotencoder.fit_transform(X).toarray()&#xA;X = X[:, 1:]&#xA;&#xA;from sklearn.model_selection import train_test_split&#xA;X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)&#xA;&#xA;from sklearn.preprocessing import StandardScaler&#xA;sc = StandardScaler()&#xA;X_train = sc.fit_transform(X_train)&#xA;X_test = sc.transform(X_test)&#xA;&#xA;from tensorflow import keras&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras.layers import Dense&#xA;&#xA;classifier = Sequential()&#xA;&#xA;classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu', input_dim = 11))&#xA;&#xA;classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu'))&#xA;&#xA;classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))&#xA;&#xA;classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])&#xA;&#xA;classifier.fit(X_train, y_train, batch_size = 10, epochs = 100)&#xA;&#xA;y_pred = classifier.predict(X_test)&#xA;y_pred = (y_pred &amp;gt; 0.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should mention that of course, I did the correct installation of TensorFlow GPU library and not the normal TensorFlow, in fact I used the resources in this link: &lt;a href=&quot;https://devtalk.nvidia.com/default/topic/1048776/jetson-nano/official-tensorflow-for-jetson-nano-/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow GPU Jetson Nano&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28271" LastActivityDate="2019-12-13T20:35:17.437" Title="Deep Learning models train really slow Jetson Nano" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2019-12-16T19:33:51.213" ContentLicense="CC BY-SA 4.0" />
  <row Id="17084" PostTypeId="1" AcceptedAnswerId="17085" CreationDate="2019-12-13T18:53:53.337" Score="8" ViewCount="879" Body="&lt;p&gt;I previously asked a question about &lt;a href=&quot;https://ai.stackexchange.com/q/17025/2444&quot;&gt;How can an AI freely make decisions?&lt;/a&gt;. I got a great answer about how current algorithms lack &lt;em&gt;agency&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The first thing I thought of was &lt;em&gt;reinforcement learning&lt;/em&gt;, since the entire concept is oriented around an agent getting rewarded for performing a correct action in an environment. It seems to me that reinforcement learning is the path to AGI.&lt;/p&gt;&#xA;&lt;p&gt;I'm also thinking: what if an agent was proactive instead of reactive? That would seem like a logical first step towards AGI. What if an agent could figure out what questions to ask based on their environment? For example, it experiences an apple falling from a tree and asks &amp;quot;What made the Apple fall?&amp;quot;. But it's similar to us not knowing what questions to ask about say the universe.&lt;/p&gt;&#xA;" OwnerUserId="31978" LastEditorUserId="2444" LastEditDate="2021-02-12T17:21:43.570" LastActivityDate="2021-02-12T17:21:43.570" Title="Why is reinforcement learning not the answer to AGI?" Tags="&lt;reinforcement-learning&gt;&lt;philosophy&gt;&lt;agi&gt;&lt;chinese-room-argument&gt;&lt;artificial-curiosity&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17085" PostTypeId="2" ParentId="17084" CreationDate="2019-12-13T19:58:30.430" Score="11" Body="&lt;p&gt;Some AI researchers do think RL is a path to AGI, and your intuition about how an agent would need to be &lt;em&gt;proactive&lt;/em&gt; in selecting actions to learn about is exactly the area these researchers are now focused on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much of the work in this area is focused on the idea of &lt;em&gt;curiosity&lt;/em&gt;, and since &lt;a href=&quot;https://sites.ualberta.ca/~amw8/surprise.pdf&quot; rel=&quot;noreferrer&quot;&gt;2014&lt;/a&gt; this idea has gained a lot of &lt;a href=&quot;https://pathak22.github.io/large-scale-curiosity/&quot; rel=&quot;noreferrer&quot;&gt;traction&lt;/a&gt; in the research community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, maybe RL &lt;em&gt;can&lt;/em&gt; lead to AGI. We don't know for sure yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, many of the classic arguments against AGI aren't addressed by the RL approach. For instance, if like &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;noreferrer&quot;&gt;Searle&lt;/a&gt;, you think computers just don't have the right kind of hardware to do thinking, then running an RL algorithm on that hardware isn't going to yield AGI, just ever increasingly robust narrow AI. Ultimately Searle's arguments get into issues of metaphysics, so it isn't clear that there exists &lt;em&gt;any&lt;/em&gt; argument that would convince someone like Searle that a particular computer-based technique is AGI-capable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also other arguments. For example, the &lt;a href=&quot;https://www.learning-theories.com/cognitivism.html&quot; rel=&quot;noreferrer&quot;&gt;cognativist&lt;/a&gt; school of thought thinks that statistical learning approaches to AI, and in particular, the black-box approaches of statistically-driven RL, are unlikely to lead to general intelligence because they do not engage in the kind of systematic reasoning process that proponents of cognativism assume is necessary for general intelligence. Some more extreme proponents of this school might say that a logical planning algorithm like &lt;a href=&quot;http://www.primaryobjects.com/2015/11/06/artificial-intelligence-planning-with-strips-a-gentle-introduction/&quot; rel=&quot;noreferrer&quot;&gt;STRIPS&lt;/a&gt; is innately more intelligent than any approach based on deep learning, because it involves sound logical deduction rather than mere statistical calculation. In particular, STRIPS can correctly generalize to any new domain, as long as it is fed the correct sense data, while an RL approach will need to learn how to act there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, while there are definitely reasons to be optimistic about RL as a direction for achieving AGI, it's definitely not yet settled.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-12-26T21:42:19.773" LastActivityDate="2019-12-26T21:42:19.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17086" PostTypeId="2" ParentId="17083" CreationDate="2019-12-13T20:35:17.437" Score="0" Body="&lt;p&gt;First of all, you mentioned that you installed the correct version of Tensorflow on Jetson. You can list the available Tensorflow devices with:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from tensorflow.python.client import device_lib&#xA;print(device_lib.list_local_devices())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And make sure that you see the GPU available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More importantly, Jetson Nano has a &lt;em&gt;128-core NVIDIA Maxwell™ architecture-based GPU&lt;/em&gt;. This might not be a powerful GPU for training a neural network with. As you can see &lt;a href=&quot;https://developer.nvidia.com/embedded/jetson-nano-dl-inference-benchmarks&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, in NVIDIA's official benchmark, they only tested trained models (i.e. classifications, object detections and segmentations). You can consider training your Keras model using another hardware and use it for labeling, classification, or etc. in your Jetson. &lt;/p&gt;&#xA;" OwnerUserId="30799" LastActivityDate="2019-12-13T20:35:17.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17087" PostTypeId="2" ParentId="17055" CreationDate="2019-12-13T21:42:24.793" Score="1" Body="&lt;p&gt;There are definitely approaches that are theory driven (like SVMs), and others where the theory comes after the practice (like a lot of deep neural networks). I think it would be difficult to argue strongly that either direction is more common &quot;in general&quot; within AI, or indeed, within any other branch of science. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach that is currently in favor will tend to change over time as well. If we develop a new method that we don't understand, it will tend to attract interest from theorists. If they are able to understand it well, that understanding will likely lead to natural next steps that improve the model. As with other areas of science, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Empirical_research#Empirical_cycle&quot; rel=&quot;nofollow noreferrer&quot;&gt;empirical cycle&lt;/a&gt; is at work. &lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2019-12-13T21:42:24.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17089" PostTypeId="1" CreationDate="2019-12-14T03:32:38.140" Score="1" ViewCount="48" Body="&lt;p&gt;In target tracking, the dimensions of objects change, especially if they are detected using a LIDAR sensor. Also, the static objects in consecutive frames are not 100% static, their position changes a little bit, due to the point cloud segmentation algorithms (which is somehow expected). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After I associate a &lt;a href=&quot;https://stackoverflow.com/q/55512548/3924118&quot;&gt;tracklet&lt;/a&gt; (which maintains the objects previous dimension) and a measurement (that has changed its dimension in the current frame) and perform a Kalman update, a small velocity is induced in the new updated tracklet, even if my object is static (I am considering the reference point of the object and tracking its center).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any solution for not inducing and displaying such a velocity in the updated tracklets?&lt;/p&gt;&#xA;" OwnerUserId="15775" LastEditorUserId="2444" LastEditDate="2019-12-14T15:08:51.030" LastActivityDate="2019-12-14T15:08:51.030" Title="How can I avoid displaying the velocity in the updated tracklets?" Tags="&lt;computer-vision&gt;&lt;autonomous-vehicles&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17091" PostTypeId="2" ParentId="17064" CreationDate="2019-12-14T04:45:48.053" Score="1" Body="&lt;p&gt;I will consider that you need to extract(crop) the digram from the pdf research paper. You can use &lt;a href=&quot;https://stackoverflow.com/questions/2693820/extract-images-from-pdf-without-resampling-in-python&quot;&gt;PyPDF2 or PyMuPDF&lt;/a&gt; to extract the images from the PDF file and then you can apply machine learning to do recognition and classification of the images. There are different types of machine learning solutions for image classification and you can start with &lt;code&gt;Convolutional Neural Network&lt;/code&gt; and you can start &lt;a href=&quot;https://azati.ai/image-detection-recognition-and-classification-with-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information for Recognition and classification of figures  see &lt;a href=&quot;https://scholar.google.com/scholar?start=10&amp;amp;hl=en&amp;amp;as_sdt=2005&amp;amp;sciodt=0,5&amp;amp;cites=16968553281840282959&amp;amp;scipsc=&quot; rel=&quot;nofollow noreferrer&quot;&gt;these&lt;/a&gt;  and to extract of figures from scholarly documents read &lt;a href=&quot;https://clgiles.ist.psu.edu/pubs/doceng2015-src-figextraction.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this &lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="21181" LastEditDate="2019-12-14T04:59:27.803" LastActivityDate="2019-12-14T04:59:27.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17092" PostTypeId="1" CreationDate="2019-12-14T05:06:01.100" Score="2" ViewCount="38" Body="&lt;p&gt;I've trained a neural network that can predict the &lt;span class=&quot;math-container&quot;&gt;$(n+1)^{th}$&lt;/span&gt; element in a sequence, given the &lt;span class=&quot;math-container&quot;&gt;$n^{th}$&lt;/span&gt; element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does a pretty good job doing this, with very little error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem emerges when I start using the network recursively. When I feed in the &lt;span class=&quot;math-container&quot;&gt;$n^{th}$&lt;/span&gt; element, I get the &lt;span class=&quot;math-container&quot;&gt;$(n+1)^{th}$&lt;/span&gt; element with very little error, as I said earlier. I feed the &lt;span class=&quot;math-container&quot;&gt;$(n+1)^{th}$&lt;/span&gt; element (the prediction) back into the network to get the &lt;span class=&quot;math-container&quot;&gt;$(n+2)^{th}$&lt;/span&gt; element. I do this multiple times to get whatever element I want to get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that each time I feed in a past prediction to get a new prediction, the tiny error in the previous prediction is compounded. The error increases exponentially as I predict successive elements in the sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was a problem I anticipated before I even began training the model. I figured that I'd be able to make the model so accurate that this error buildup wouldn't be noticeable until the model got far into the predicted sequence. I severely underestimated the exponential error buildup, and I don't think it's possible for me to make my model more accurate than it already is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figured that this is a problem that many others would've faced in the past, but I haven't found any good solutions to the problem. Are there useful tricks/methods by which I can fix the issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9432" LastEditorUserId="9432" LastEditDate="2019-12-15T01:46:40.240" LastActivityDate="2019-12-15T01:46:40.240" Title="How can I prevent a Recursive Neural Network from performing extremely poorly after a few cycles?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;sequence-modelling&gt;&lt;forecasting&gt;&lt;recursive-neural-network&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17093" PostTypeId="1" CreationDate="2019-12-14T08:03:43.487" Score="1" ViewCount="11" Body="&lt;p&gt;I'm working on Bilateral Recommendation System. But not able to find much related papers. Could anyone suggest any papers relative? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="32073" LastActivityDate="2019-12-14T08:03:43.487" Title="Anyone familiar with Bilateral Recommendation System? And suggest any related papers?" Tags="&lt;deep-learning&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17094" PostTypeId="1" CreationDate="2019-12-14T08:07:26.960" Score="6" ViewCount="144" Body="&lt;p&gt;In standard Reinforcement Learning the reward function is specified by an AI designer and is external to the AI agent. The agent attempts to find a behaviour that collects higher cumulative discounted reward. In Evolutionary Reinforcement Learning the reward function is specified by&#xA;the agent’s genetic code and evolved in simulated Darwinian evolution over multiple generations.&#xA;Here too the AI agent cannot directly adjust the reward function and instead adjusts its behaviour&#xA;towards collecting higher rewards. Why do both approaches prevent the AI agent from changing its reward function at will? What happens if we do allow the AI agent to do so?&lt;/p&gt;&#xA;" OwnerUserId="32074" LastActivityDate="2019-12-15T09:43:08.230" Title="Why cannot an AI agent adjust the reward function directly?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17095" PostTypeId="2" ParentId="10049" CreationDate="2019-12-14T08:08:53.580" Score="8" Body="&lt;p&gt;Recent actor-critic algorithms &lt;em&gt;do&lt;/em&gt; use &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns, but they are disguised as something called the &lt;a href=&quot;https://arxiv.org/pdf/1506.02438.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalized Advantage Estimator&lt;/a&gt; defined as &lt;span class=&quot;math-container&quot;&gt;$A^{GAE}_t = \sum_{i=0}^{\infty} (\gamma\lambda)^i \delta_{t+i}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\delta_t = r_t + \gamma V(s_{t+1}) - V(s_t)$&lt;/span&gt;. This turns out to be identically equal to &lt;span class=&quot;math-container&quot;&gt;$[G^\lambda_t - V(s_t)]$&lt;/span&gt;, &lt;em&gt;i.e.&lt;/em&gt; the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return with a value-function baseline subtracted from it. Theoretically, any actor-critic gradient method could use this quite easily; it was combined with TRPO in the GAE paper, and later used for PPO. Similarly, ACER uses an off-policy variant known as Retrace(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;For replay methods like DQN or DDPG, it is harder to implement &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns. This is why they have historically defaulted to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns as @DennisSoemers mentioned. I recently &lt;a href=&quot;https://papers.nips.cc/paper/8397-reconciling-returns-with-experience-replay&quot; rel=&quot;nofollow noreferrer&quot;&gt;published a paper&lt;/a&gt; that describes a way to efficiently combine &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns with experience replay, which I hope will increase the popularity of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns for these methods.&lt;/p&gt;&#xA;" OwnerUserId="32070" LastEditorUserId="-1" LastEditDate="2020-08-11T16:59:36.413" LastActivityDate="2020-08-11T16:59:36.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17096" PostTypeId="2" ParentId="17094" CreationDate="2019-12-14T08:49:07.110" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do both approaches prevent the AI agent from changing its reward function at will? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In RL for optimal control, the reward function is part of the &lt;em&gt;problem formulation&lt;/em&gt;. That is, it describes the goals of the agent. Sometimes this is obviously something that should not be under the agent's control, if the reward is a real-world quantity that it should maximise - e.g. the amount of money it makes in profit - then it makes no sense that the agent could arbitrarily declare that the quantity is different to the thing that it observed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other times, there is some flexibility, an agent that needs to escape a maze in a short time could have -1 reward per time step inside the maze or -0.1 reward per time step, or +1 reward for escaping with a discount factor applied. However, the flexibility can only go so far before it describes a different problem. Changing the -1 per time step to +1 per time step means that the agent's goal switches from escaping to staying in the maze. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, multiplying all the rewards in a MDP by some &lt;em&gt;positive&lt;/em&gt; constant does not change a reinforcement learning problem. Sometimes it may be worth doing this scaling to make it easier for a specific approach, such as neural networks, to work efficiently. However, this is not something to put directly under the agent's control, but a hyperparameter like the number of hidden layers in the neural network. As a hyperparameter, usually the reward scaling is very flexible and not something worth spending much effort tuning - &lt;em&gt;unlike&lt;/em&gt; the architecture of a neural network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What happens if we do allow the AI agent to do so?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unless significant constraints are placed on what is allowed to change, then the agent will get any amount of reward it &quot;wants&quot; by doing anything it &quot;wants&quot;, within whatever constraints are placed on changes allowed to the reward function. Typically in RL this would result in an agent that acts more or less randomly whilst getting progressively higher and higher reward on each iteration. Or in other words, an agent that does not attempt to solve any kind of problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few special cases where a reward function can be adjusted or learned. One common case is &lt;a href=&quot;https://towardsdatascience.com/inverse-reinforcement-learning-6453b7cdc90d&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverse reinforcement learning&lt;/a&gt;, where an agent's activities are observed, it is assumed to be solving an MDP-like problem, and you are interested in understanding how it solves it, including what reward function it is using. The reward function must be learned by fitting it to observations of the agent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-15T09:43:08.230" LastActivityDate="2019-12-15T09:43:08.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17097" PostTypeId="1" CreationDate="2019-12-14T10:06:32.530" Score="1" ViewCount="32" Body="&lt;p&gt;Is there a neural network that has architecture optimizations for segmenting only one class (object and background)? I have tried U-net but it is not providing good enough results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if this can be due to the fact that my dataset has different image resolutions/aspect ratios. &lt;/p&gt;&#xA;" OwnerUserId="30995" LastEditorUserId="2444" LastEditDate="2020-06-13T00:09:10.353" LastActivityDate="2020-06-13T00:09:10.353" Title="How to train image segmentation task with only one class?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17098" PostTypeId="1" CreationDate="2019-12-14T11:47:08.443" Score="2" ViewCount="69" Body="&lt;p&gt;I'm trying to develop a real-time application that, from the sequence of chalkboard images captured by a webcam, recognizes the lines being draw on it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It must be able of recognize the lines from the chalkboard background, filter the presence in the image of the teacher, and translate these lines to some representation, something as a list of basic events like &quot;start of line at xxx,xxx&quot;, &quot;continue line at xxx,xxx&quot;, ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After several days looking for references and bibliography, none is found. The most similar are the character recognition applications, in particular when they have a stroke recognition stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hint ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input will be a sequence as &lt;a href=&quot;https://www.shutterstock.com/video/clip-9146711-teacher-writing-on-blackboard-class-ultra-hd&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;,  &lt;a href=&quot;https://videohive.net/item/young-male-teacher-or-student-holding-chalk-writing-on-chalkboard-in-classroom/17391669&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; or &lt;a href=&quot;https://www.youtube.com/watch?v=gBpqT3IEIQU&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; (just without the presence of the students). I've expect the teacher not hidding his hand. We could imagine a start with an empty chalkboard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am looking for more than an answer which says only something similar to &quot;you can use a deep learning training it with two classes&quot;, without details or references. &lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2019-12-14T12:41:34.137" LastActivityDate="2021-02-07T20:02:00.210" Title="Recognition of lines in a chalkboard" Tags="&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17100" PostTypeId="1" AcceptedAnswerId="20322" CreationDate="2019-12-14T13:00:48.820" Score="5" ViewCount="140" Body="&lt;p&gt;How can we prove that an auto-associator network will continue to perform if we zero the diagonal elements of a weight matrix that has been determined by the Hebb rule? In other words, suppose that the weight matrix is determined from &lt;span class=&quot;math-container&quot;&gt;$W = PP^T- QI$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is the number of prototype vectors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been given a hint: show that the prototype vectors continue to be eigenvectors of the new weight matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a question from Neural Network Design (2nd Edition) book by&#xA;Martin T. Hagan, Howard B. Demuth, Mark H. Beale, Orlando De Jesus .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Resource : &lt;a href=&quot;https://hagan.okstate.edu/NNDesign.pdf&quot; rel=&quot;noreferrer&quot;&gt;E7.5 p 224-225&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32076" LastEditorUserId="32076" LastEditDate="2019-12-23T10:27:49.780" LastActivityDate="2020-04-16T16:34:44.157" Title="How can we prove that an autoassociator network will continue to perform if we zero the diagonal elements of a weight matrix?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;learning-algorithms&gt;&lt;hebbian-learning&gt;" AnswerCount="1" CommentCount="9" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17101" PostTypeId="1" AcceptedAnswerId="17111" CreationDate="2019-12-14T13:47:25.180" Score="2" ViewCount="2509" Body="&lt;p&gt;How can we compute number of weights considering a convolutional neural network that is used to classify images into two classes :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;INPUT: 100x100 gray-scale images.&lt;/li&gt;&#xA;&lt;li&gt;LAYER 1: Convolutional layer with 60 7x7 convolutional filters (stride=1, valid&#xA;padding).&lt;/li&gt;&#xA;&lt;li&gt;LAYER 2: Convolutional layer with 100 5x5 convolutional filters (stride=1, valid&#xA;padding).&lt;/li&gt;&#xA;&lt;li&gt;LAYER 3: A max pooling layer that down-samples Layer 2 by a factor of 4 (e.g., from 500x500 to 250x250)&lt;/li&gt;&#xA;&lt;li&gt;LAYER 4: Dense layer with 250 units&lt;/li&gt;&#xA;&lt;li&gt;LAYER 5: Dense layer with 200 units&lt;/li&gt;&#xA;&lt;li&gt;LAYER 6: Single output unit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Assume the existence of biases in each layer. Moreover, pooling layer has a weight (similar to AlexNet)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many weights does this network have?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Keras Code&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import keras&#xA;from keras.models import Sequential&#xA;from keras.layers import Dense&#xA;from keras.layers import Conv2D, MaxPooling2D&#xA;&#xA;model = Sequential()&#xA;&#xA;# Layer 1&#xA;model.add(Conv2D(60, (7, 7), input_shape = (100, 100, 1), padding=&quot;same&quot;, activation=&quot;relu&quot;))&#xA;&#xA;# Layer 2&#xA;model.add(Conv2D(100, (5, 5), padding=&quot;same&quot;, activation=&quot;relu&quot;))&#xA;&#xA;# Layer 3&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;# Layer 4&#xA;model.add(Dense(250))&#xA;&#xA;# Layer 5&#xA;model.add(Dense(200))&#xA;&#xA;model.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32076" LastEditorUserId="32076" LastEditDate="2019-12-15T15:20:40.617" LastActivityDate="2021-03-31T22:22:08.247" Title="How to compute number of weights of CNN?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17102" PostTypeId="1" CreationDate="2019-12-14T13:55:05.467" Score="6" ViewCount="888" Body="&lt;p&gt;I'm looking to make an NLP model that can achieve a dual purpose. One purpose is that it can hold interesting conversations (conversational AI), and another being that it can do &lt;em&gt;intent classification&lt;/em&gt; and even accomplish the classified task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To accomplish this, would I need to use multimodal machine learning, where you combine the signal from two models into one? Or can it be done with a single model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my internet searches, I found &lt;a href=&quot;https://github.com/google-research/bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt;, developed by Google engineers (although apparently not a Google product), which is an NLP model trained in an unsupervised fashion on 3.3 billion words or more and seems very capable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I leverage BERT to make my own conversational AI that can also carry out tasks? Is it as simple as copying the weights from BERT to your own model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any guidance is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="2444" LastEditDate="2020-05-14T20:06:43.310" LastActivityDate="2020-10-11T21:02:55.220" Title="How to use BERT as a multi-purpose conversational AI?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;bert&gt;&lt;language-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17104" PostTypeId="1" CreationDate="2019-12-14T14:56:07.270" Score="3" ViewCount="71" Body="&lt;p&gt;I wonder what is the better way of drawing rectangles on images for gender classification. My task is to create a classifier (CNN based) to detect gender from pictures of entire bodies (not just faces). When I started labeling pictures I noticed that I am not sure whether I should draw it around an entire person like example 1 (including hands and legs and some background space between them) or just the inner part like example 2 (where there is almost no background), in order to achieve better results?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lRDJP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lRDJP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/awi8U.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/awi8U.jpg&quot; alt=&quot;gend&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2019-12-15T14:25:53.867" LastActivityDate="2020-09-10T16:02:10.227" Title="How to draw bounding boxes for gender classification?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17105" PostTypeId="1" CreationDate="2019-12-14T21:11:12.080" Score="2" ViewCount="25" Body="&lt;p&gt;Secondary camera, ghost overlay, video merge... I do not know if what I mean has a more specific name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if this is a thing. This could be insightful for example in racing sports where participants race one after another e.g. alpine skiing, downhill mountain bike, showjumping etc. E.g. comparing the current starter to the leader.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the camera position is fixed and only the camera angle and zoom is varying to focus on the current starter, the tasks to be able to overlay videos would be to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;match the timing, i.e. both videos start when the timer starts&lt;/li&gt;&#xA;&lt;li&gt;align and overlay the videos according to specific marker points. Keypoint detection and tracking.&lt;/li&gt;&#xA;&lt;li&gt;get the opacity right so that both videos are visible&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My question is if there is any research on this. If so, what keywords do I need to search for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/txdfm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/txdfm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;My seach led me to SIFT (Scale Invariant Feature Transform) and SURF (Speeded-Up Robust Features). Feature matching should be possible with kNN or brute force. A lot can be done with OpenCV.&lt;/p&gt;&#xA;" OwnerUserId="32082" LastEditorUserId="32082" LastEditDate="2019-12-15T07:45:35.520" LastActivityDate="2019-12-15T07:45:35.520" Title="Ghost camera or video overlays for example in sports" Tags="&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17107" PostTypeId="1" AcceptedAnswerId="17109" CreationDate="2019-12-15T09:41:06.537" Score="1" ViewCount="599" Body="&lt;p&gt;Studying CNN Back-propagation I can't understand how can we compute the gradient of max pooling with overlapping regions ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's also a question from &lt;a href=&quot;http://courses.cs.tau.ac.il/Caffe_workshop/Bootcamp/pdf_lectures/Lecture%203%20CNN%20-%20backpropagation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; quiz and can be also found on &lt;a href=&quot;https://books.google.gr/books?id=72UkDwAAQBAJ&amp;amp;pg=PA128&amp;amp;lpg=PA128&amp;amp;dq=Compute+the+gradient+of+max+pooling+with+overlapping+regions.&amp;amp;source=bl&amp;amp;ots=RknIRCMsbW&amp;amp;sig=ACfU3U1zStijp120L9lTh5jG-sLL3XxZtQ&amp;amp;hl=el&amp;amp;sa=X&amp;amp;ved=2ahUKEwimjczOrbfmAhVBsaQKHVHsAhQQ6AEwCXoECAgQAQ#v=onepage&amp;amp;q=Compute%20the%20gradient%20of%20max%20pooling%20with%20overlapping%20regions.&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; book .&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2019-12-16T07:14:02.650" Title="How can we compute the gradient of max pooling with overlapping regions?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17108" PostTypeId="1" AcceptedAnswerId="17156" CreationDate="2019-12-15T11:35:45.770" Score="3" ViewCount="64" Body="&lt;p&gt;I have a question about a reinforcement learning problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm training an agent to add or delete pixels in a [12 x 12] 2D space (going to be 3D in the future).&#xA;Its action space consists of two discrete outputs: x[0-12] and y[0-12].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the value of instead outputting a (continuous) probabilistic output representation, like the [12 x 12] space with each pixel as a probability, and sampling from it. E.g. a softmax function applied to 144 (12*12) output nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My environment is deterministic itself: taking action  in state  always results in the same next state ′.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that this may be more difficult to train since the output space becomes continuous instead of discrete, and therefore bigger, but does stochastic/probabilistic output have any benefits over 1 discrete output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="31180" LastEditorUserId="31180" LastEditDate="2019-12-15T11:49:02.467" LastActivityDate="2019-12-18T18:46:22.643" Title="What's the value of making the RL agent's output stochastic opposed to deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;stochastic-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17109" PostTypeId="2" ParentId="17107" CreationDate="2019-12-15T13:42:11.013" Score="2" Body="&lt;p&gt;When gradients in a neural network can follow multiple paths to same parameter, the different gradient values from the sources can often be added together, because the operations in the forward direction are also sums and &lt;span class=&quot;math-container&quot;&gt;$\frac{d}{dx}(y+z) = \frac{dy}{dx} + \frac{dz}{dx}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is the case already with gradients of kernels (which are sums over the image area), and is equally the case for overlapping aggregation, including maximums, minimums or averages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the 1d case, if you have a max pool over the input params &lt;span class=&quot;math-container&quot;&gt;$[a_0, a_1, a_2, a_3, a_4]$&lt;/span&gt; a max function &lt;span class=&quot;math-container&quot;&gt;$m_0 = max(a_0, a_1, a_2)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m_1 = max(a_2, a_3, a_4)$&lt;/span&gt; which overlap at &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;, and gradients &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\mathbf{m}} J = [\frac{\partial J}{\partial m_0}, \frac{\partial J}{\partial m_1}]$&lt;/span&gt;, then you would allocate those gradients to vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}$&lt;/span&gt; according to which items in each group was the max of that group, adding them when they overlapped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a} = [3,0,1,2,0]$&lt;/span&gt; and  &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\mathbf{m}}J = [0.7, 0.9]$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\mathbf{a}}J = [0.7, 0, 0, 0.9, 0]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a} = [3,0,4,2,0]$&lt;/span&gt; and  &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\mathbf{m}}J = [0.7, 0.9]$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\mathbf{a}}J = [0, 0, 1.6, 0, 0]$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-16T07:14:02.650" LastActivityDate="2019-12-16T07:14:02.650" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17110" PostTypeId="2" ParentId="17104" CreationDate="2019-12-15T14:57:58.503" Score="2" Body="&lt;p&gt;In state-of-the-art papers. researcher managed to achieve 90-100% accuracy on gender classification on just human faces, so both method may work just fine, with minor improvements maybe using the first method. However using the image of the entire body enhances overfitting as there is more input features and increases overfitting. For a very large dataset it might be better to use the first method as chances of overfitting are low while for a smaller one you should probably go for the second method or even just the face as that won't make such a huge differnce. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better way to input the data to the network is through landmarks of a body.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/chrDs.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/chrDs.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inputting the landmarks of the body helps reduce the features to the minimal and also keeps only the relevant features. The main difference in body between women and men is perhaps the torso and shoulder width, which you can get from body landmark detection. You can input both the image of the face and the body landmark as input to the network, and it would probably increase in accuracy. However this solution is not an one stage method, meaning it requires an extra model to predict the landmark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-12-15T14:57:58.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17111" PostTypeId="2" ParentId="17101" CreationDate="2019-12-15T15:15:06.447" Score="2" Body="&lt;p&gt;Calculating parameter number in a CNN is very straightforward.&lt;/p&gt;&#xA;&lt;p&gt;CNN is composed of different &lt;strong&gt;filters&lt;/strong&gt; which is essentially a 3d tensor. CNN weights are &lt;strong&gt;shared&lt;/strong&gt; meaning they are used multiple times and reused in different locations. Each layer have n tensors each with dimension w * h * c where w = width, h = height, c = channels (the input channel size), therefore getting param_no = w * h * c * n. There is also a bias for each output channel, so param_no_bias = n. At the end the parameter number is calculated with: n * w * h * c + n. See more about this in hear: &lt;a href=&quot;https://towardsdatascience.com/understanding-and-calculating-the-number-of-parameters-in-convolution-neural-networks-cnns-fc88790d530d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Article&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The pooling layer does not have a weight, it only has &lt;strong&gt;hyperparameters&lt;/strong&gt;. You may have confused the two. There are hyperparameters for the stride, the factor and etc. These are predefined and not trainable.&lt;/p&gt;&#xA;&lt;p&gt;For a keras API, @pasabaporaqui has mentioned &lt;a href=&quot;https://stackoverflow.com/a/35827171/4886927&quot;&gt;https://stackoverflow.com/a/35827171/4886927&lt;/a&gt; which definitely works.&lt;/p&gt;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="-1" LastEditDate="2021-03-31T22:22:08.247" LastActivityDate="2021-03-31T22:22:08.247" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17112" PostTypeId="1" CreationDate="2019-12-15T15:23:19.777" Score="2" ViewCount="52" Body="&lt;p&gt;I want to know which algorithm will work most efficiently for calculating nutrients present in a food dish if I am giving the ingredients used in the food. Basically, let us assume that I want to make a health status for a person A based on the intake of food and based on it create a diet for him.&lt;/p&gt;&#xA;" OwnerUserId="32091" LastEditorUserId="2444" LastEditDate="2019-12-15T15:35:10.247" LastActivityDate="2019-12-15T15:35:10.247" Title="How can I predict the nutrients in dishes given the ingredients used to prepare them?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;applications&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17113" PostTypeId="1" AcceptedAnswerId="17119" CreationDate="2019-12-15T15:53:46.303" Score="2" ViewCount="102" Body="&lt;p&gt;Is every process&lt;sup&gt;1&lt;/sup&gt; that is done on the &lt;strong&gt;data&lt;/strong&gt; before we train the model is always called the pre-processing part? Or are there some processes which are not included? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;&lt;sup&gt;&lt;sup&gt;1&lt;/sup&gt;&quot;Every process&quot; is includes data acquisition, splitting the data for validation, data cleaning, feature engineering, etc.&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="9947" LastEditDate="2019-12-15T15:59:05.110" LastActivityDate="2019-12-16T01:34:19.173" Title="How to define the &quot;Pre-Processing&quot; in machine learning?" Tags="&lt;definitions&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17114" PostTypeId="1" CreationDate="2019-12-15T16:55:05.183" Score="1" ViewCount="137" Body="&lt;p&gt;I am very interested in the application of CycleGANs. I understand the concept of unpaired data and it makes sense to me.&lt;/p&gt;&#xA;&lt;p&gt;But now a question comes to my mind: what if I have enough paired image data, is then a CycleGAN an over-engineering, if I use it in a &amp;quot;supervised&amp;quot; setting (input matches with the label - but still a CycleGAN)? For what kind of application could it be useful? Would it be more useful to process it using a &amp;quot;normal&amp;quot; supervised setting?&lt;/p&gt;&#xA;&lt;p&gt;So, basically, my question is whether it is useful to train a CycleGAN in a supervised setting?&lt;/p&gt;&#xA;" OwnerUserId="32029" LastEditorUserId="2444" LastEditDate="2020-09-17T17:51:54.277" LastActivityDate="2020-09-17T17:51:54.277" Title="Is it useful to train a CycleGAN in a supervised setting?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17116" PostTypeId="2" ParentId="17098" CreationDate="2019-12-15T17:17:05.293" Score="0" Body="&lt;p&gt;I will assume that the camera is stable (no change in position, zoom or other settings during the video recording), otherwise the task becomes markedly more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that your dataset is an array of rasters (images in array format). You mention that you want to detect events &quot;start line&quot; and &quot;end line&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of doing this would be to compute an approximate time derivative of your image series. For instance, take the image raster at index &lt;code&gt;idx&lt;/code&gt; and the one right after, at index &lt;code&gt;idx+1&lt;/code&gt; (captured at instants &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t + \Delta t$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; is the sampling interval).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At coordinates &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt;, this derivative could look something like: &lt;code&gt;timeDerivative = (images[idx][j][i] - images[idx+1][j][i])/DeltaT&lt;/code&gt;. This is a crude estimate, and there are better ways of computing an approximate discrete derivative, but you get the idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this, you could declare a state of the recording: drawing line or not drawing line. The states, we assume are always alternating, as a teacher has to take his hand off the blackboard to draw a new line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a derivative with large values is detected (a region in the image goes from being black to white suddenly) and the state is &quot;not drawing&quot;, the event &quot;start line&quot; is recorded and the state switches to &quot;drawing&quot;. While a derivative with large values continues to be detected as time goes on in the vicinity of the previous spot with a large derivative, nothing changes. Once this is no longer true, the state changes to &quot;not drawing&quot; and the event &quot;stop line&quot; is recorded at the last location with a large derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the main idea, which can be improved with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Defining the area of the blackboard, either by hand or automatically&lt;/li&gt;&#xA;&lt;li&gt;Thresholding the images to better isolate the chalk trace from the blackboard&lt;/li&gt;&#xA;&lt;li&gt;Using a tracker such as a Kalman filter to know where to look next for the chalk&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30210" LastActivityDate="2019-12-15T17:17:05.293" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17117" PostTypeId="2" ParentId="17107" CreationDate="2019-12-15T21:58:12.857" Score="0" Body="&lt;p&gt;Denote x(h,w) input to max-pooling, and y(h,w) - output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then &lt;span class=&quot;math-container&quot;&gt;$\frac{dL}{dx}(h,w)= \sum \frac{dL}{dy}(h',w')$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;over all y(h',w') which have been obtained  from x(h.w) such that y(h',w') = x(h,w).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;related to &lt;a href=&quot;http://courses.cs.tau.ac.il/Caffe_workshop/Bootcamp/pdf_lectures/Lecture%203%20CNN%20-%20backpropagation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this p 11&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2019-12-15T21:58:12.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17118" PostTypeId="1" CreationDate="2019-12-15T22:21:44.183" Score="2" ViewCount="1121" Body="&lt;p&gt;What i really want to do, is to predict an integer sequence of (5 numbers with values from 1 to 50) for example based on a big dataset of other 5 numbers sequences with same values range created by the same random number generator. I suppose there is a way to train based on the dataset and the program will find a pattern or based on the most common numbers predict the next number sequence. The more numbers will predict in the sequence correctly the better of course. Any help, directions and preferably python code would be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently read the following &lt;a href=&quot;https://ai.stackexchange.com/questions/3850/can-a-neural-network-be-used-to-predict-the-next-pseudo-random-number&quot;&gt; can-a-neural-network-be-used-to-predict-the-next-pseudo-random-number&lt;/a&gt; and i am new to the AI field. The proposed code while it creates a sequence of 25 numbers it ends showing 20 numbers i do not understand why. It seems they try to do something similar if i understand correctly&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried The code here &lt;a href=&quot;https://ai.stackexchange.com/questions/3850/can-a-neural-network-be-used-to-predict-the-next-pseudo-random-number&quot;&gt;can-a-neural-network-be-used-to-predict-the-next-pseudo-random-number&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It shows always the same numbers no matter how many epochs and or iterations i do is that normal?&#xA;Is the last code close to what i want to accomplish?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="32099" LastEditorUserId="32099" LastEditDate="2020-01-12T22:44:36.650" LastActivityDate="2020-01-13T01:09:01.170" Title="Can a neural network be used to predict a sequence of integers based on dataset of previously produced random numbers?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;prediction&gt;&lt;randomness&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17119" PostTypeId="2" ParentId="17113" CreationDate="2019-12-16T01:26:04.050" Score="2" Body="&lt;p&gt;Data preprocessing consists of all those techniques used to generate the final datasets (with an appropriate size, structure, and format) for the machine learning algorithms or models. Data acquisition should not be part of data preprocessing, but the step preceding it, which gathers the raw data (which may e.g. be noisy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book &lt;a href=&quot;http://pzs.dstu.dp.ua/DataMining/preprocessing/bibl/Data%20Preprocessing%20in%20Data%20Mining.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data Preprocessing in Data Mining&lt;/a&gt; (2014), by Salvador García et al., which provides a good overview of the data preprocessing techniques and their connection with data mining and machine learning algorithms and models, defines data preprocessing as follows.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Data preprocessing includes data preparation, compounded by integration,&#xA;  cleaning, normalization and transformation of data; and data reduction tasks; such as feature selection, instance selection, discretization, etc. The result expected after a reliable chaining of data preprocessing tasks is a final dataset, which can be considered correct and useful for further data mining algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From page 10 onwards, there is a description and categorization of the main data preprocessing techniques. I will just list them, so refer to the book for a definition and explanation of each of these techniques.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data Preparation&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data Cleaning&lt;/li&gt;&#xA;&lt;li&gt;Data Transformation&lt;/li&gt;&#xA;&lt;li&gt;Data Integration&lt;/li&gt;&#xA;&lt;li&gt;Data Normalization&lt;/li&gt;&#xA;&lt;li&gt;Missing Data Imputation&lt;/li&gt;&#xA;&lt;li&gt;Noise Identification&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Data Reduction&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Feature Selection&lt;/li&gt;&#xA;&lt;li&gt;Instance Selection&lt;/li&gt;&#xA;&lt;li&gt;Discretization&lt;/li&gt;&#xA;&lt;li&gt;Feature Extraction/Instance Generation&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are two screenshots (from the cited book) that illustrate some of the &lt;strong&gt;data preparation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/llDEy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/llDEy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;strong&gt;data reduction&lt;/strong&gt; techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rbt79.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rbt79.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-16T01:34:19.173" LastActivityDate="2019-12-16T01:34:19.173" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17120" PostTypeId="2" ParentId="10641" CreationDate="2019-12-16T07:50:25.003" Score="2" Body="&lt;p&gt;NEAT does not employ a feed forward concept and also it does not take any special action to avoid loops.&lt;/p&gt;&#xA;&lt;p&gt;Network is evaluated in non-recursive model. The only non-deterministic loop, the evaluation has is the loop for activating all the outputs. Pseudo code is something like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Until all the outputs are active&#xA;    for all non-sensor nodes&#xA;        activate node&#xA;        sum the input&#xA;    for all non-sensor and active nodes&#xA;        calculate the output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NOTE1: You can use a defensive mechanism (like a counter) to avoid a infinite loop&lt;/p&gt;&#xA;&lt;p&gt;NOTE2: When summing the input, outputs of nodes which were at least evaluated/calculated once are considered, otherwise their outputs are assumed to be zero.&lt;/p&gt;&#xA;&lt;p&gt;This is the note from the author of NEAT about identifying loops,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note that checking networks for loops in general in not necessary and therefore I stopped writing this function&lt;/p&gt;&#xA;&lt;p&gt;bool Network::integrity()&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="32106" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-12-18T03:38:05.850" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17121" PostTypeId="2" ParentId="6231" CreationDate="2019-12-16T07:56:05.977" Score="2" Body="&lt;p&gt;Following is the pseudo code of the NEAT's network evaluation (converted from original source code), &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Until all the outputs are active&#xA;    for all non-sensor nodes&#xA;        activate node&#xA;        sum the input&#xA;    for all non-sensor and active nodes&#xA;        calculate the output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that there is no recursion for feed forwarding concepts according to the original author. &lt;/p&gt;&#xA;" OwnerUserId="32106" LastActivityDate="2019-12-16T07:56:05.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17122" PostTypeId="1" AcceptedAnswerId="17124" CreationDate="2019-12-16T08:16:26.920" Score="3" ViewCount="760" Body="&lt;p&gt;According to what we know about inductive and connectionist learning, what is the difference between them ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those who do not know about :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Inductive Learning&lt;/em&gt;, like what we have in decision tree and make a decision based on amount of samples&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Connectionist Learning&lt;/em&gt;, like what we have in artificial neural network&lt;/p&gt;&#xA;" OwnerUserId="31143" LastActivityDate="2019-12-20T04:23:54.270" Title="What is the difference between Inductive Learning and Connectionist Learning?" Tags="&lt;neural-networks&gt;&lt;decision-tree&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17123" PostTypeId="2" ParentId="17118" CreationDate="2019-12-16T08:37:05.340" Score="3" Body="&lt;p&gt;The post you linked to clearly states that pseudo random number cannot be predicted. Their randomness is made to be nearly perfect, and if you ever found a way to even predict a pseudo random number with 20% chance of correct, the security of the entire world would be vulnerable to attacks, as things ranges from cryptocurrency and secure data transfer is all protected by pseudo random number. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2019-12-16T08:37:05.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17124" PostTypeId="2" ParentId="17122" CreationDate="2019-12-16T11:40:50.323" Score="2" Body="&lt;p&gt;All of the statistical learning is about inductive learning.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is the difference between inductive learning and connectionist learning?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Inductive learning is about identifying patterns from examples. It is more related to statistics. Connectionist learning is more about finding a common pattern and predicting as well as self-learning(learning from the experience of prediction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Connectionist learning is where learning occurs by modifying connection strengths based on experience. This is not the case with inductive learning. In inductive learning, we are not modifying things based on experience. Inductive learning just finds common patterns, not self-learning based on experience.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Learning requires both practice and rewards&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In inductive learning, we learn the model from raw data (so-called training set), and in the deductive learning, the model is applied to predict the behaviour of new data. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Connectionist Learning is a group of inductive learning and deductive learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="7681" LastEditDate="2019-12-20T04:23:54.270" LastActivityDate="2019-12-20T04:23:54.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17125" PostTypeId="1" CreationDate="2019-12-16T11:51:07.957" Score="3" ViewCount="21" Body="&lt;p&gt;Now, the following may sound silly, but I want to do it for my better understanding of performance and implementation of GPU inference for a set of deep learning problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is to replace a surface texture for a 3d model by a NN that stores that texture data in some way and allows to infere the rgb color of an arbitrary texel from its UV coordinates. So basically it should offer the same functionality as the texture itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A regular texture lookup takes a UV coordinate and returns the (possibly filtered) RGB color at these texture coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I want to train a network that takes two floats in [0,1] range as input and outputs three floats of rgb color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I further want to then train that network to store my 4096x4096 texture. So the training data I have available are 4096*4096=16777216 of &lt;code&gt;&amp;lt;float2, float3&amp;gt;&lt;/code&gt; pairs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally I want to evaluate the trained network in my (OpenGL 4 or directX11) pixel shader, feeding it for every rendered pixel the interpolated UV coordinates at this pixel and retrieving the RGB value from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's clear that this will&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;have lower fidelity than just using the texture directly&lt;/li&gt;&#xA;&lt;li&gt;use likely more memory than just using the texture directly&lt;/li&gt;&#xA;&lt;li&gt;be slower than using the texture directly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and as such may be silly to do, but I'd still like to try to do this somewhat optimally, especially in terms of inference performance (I'd like to be able to run it at interactive framerates at 1080p resolutions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me to a class of networks or articles or describe a model and training algorithm that would be well suited for this task (especially in terms of implementing inference for the pixel shader)?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastEditorUserId="32111" LastEditDate="2019-12-16T12:56:13.347" LastActivityDate="2019-12-16T12:56:13.347" Title="Good model and training algorithm to store texture data for fast gpu inference" Tags="&lt;gpu&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17126" PostTypeId="1" AcceptedAnswerId="20324" CreationDate="2019-12-16T12:43:30.297" Score="2" ViewCount="64" Body="&lt;p&gt;How can we train a competitive layer on non-normalized vectors using LVQ technique ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;an example is given below from Neural Network Design (2nd Edition) book&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The net input expression for LVQ networks calculates the distance between the input&#xA;and each weight vector directly, instead of using the inner product. The result is that the&#xA;LVQ network does not require normalized input vectors. This technique can also be&#xA;used to allow a competitive layer to classify non-normalized vectors. Such a network is&#xA;shown in figure below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TbqJh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TbqJh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Use this technique to train a two-neuron competitive layer on the (non-normalized)&#xA;vectors below, using a learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha=0.5$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_1=\begin{bmatrix}&#xA;1 \\&#xA;1&#xA;\end{bmatrix}, p_2=\begin{bmatrix}&#xA;-1 \\&#xA;2&#xA;\end{bmatrix}, p_3=\begin{bmatrix}&#xA;-2 \\&#xA;-2&#xA;\end{bmatrix}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Present the vectors in the following order : &lt;span class=&quot;math-container&quot;&gt;$p_1, p_2, p_3, p_2, p_3, p_1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial weights : &lt;span class=&quot;math-container&quot;&gt;$W_1=\begin{bmatrix}&#xA;0 \\&#xA;1&#xA;\end{bmatrix}, W_2=\begin{bmatrix}&#xA;1 \\&#xA;o&#xA;\end{bmatrix}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2020-04-16T16:41:18.440" Title="Train a competitive layer on nonnormalized vectors using LVQ technique" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17127" PostTypeId="1" CreationDate="2019-12-16T15:15:56.253" Score="1" ViewCount="42" Body="&lt;p&gt;If you have a very distorted video/image, would affine transformations of the images make object detection algorithms make more mistakes compared to a normal camera?&lt;/p&gt;&#xA;" OwnerUserId="31547" LastActivityDate="2019-12-16T15:15:56.253" Title="Affine Transformations and Data Augmentation" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17128" PostTypeId="2" ParentId="17102" CreationDate="2019-12-16T15:40:02.987" Score="2" Body="&lt;p&gt;Think of BERT (or similar models) as as good starting place for understanding context.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple options to make BERT contextualize dialogue:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Concatenate all messages with a seperator embedding and finetune a language model like BERT&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This has shown good results in this &lt;a href=&quot;https://arxiv.org/abs/1901.08149&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, but understand it has weaknesses like struggling to determine order or author comprehension or conversation disentanglement. Additionally as threads grow longer this becomes more and more expensive  &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Use A BERT like model as an RNN and pass along author/memory information as the hidden state&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Weaknesses here is that you are putting a lot of strain on the hidden state of the model, and ideally you want to train each comment block separately so the gradient wouldnt need to unroll or traverse the network in backprop (I actually implemented a variant of this for my work, and had good results)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The 2 modes above are just baseline examples that you can alter for your own needs, and im sure there is also others if you brainstorm about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note if you want generation capabilities, you would want to use GPT2 or XLNet instead of BERT for unidirectional embeddings. Hope this helped.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-12-16T15:40:02.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17129" PostTypeId="2" ParentId="17080" CreationDate="2019-12-16T16:22:38.053" Score="1" Body="&lt;p&gt;Ok so after a little more reading, I am currently satisfy with what I found for this question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Yes, the &quot;under-parameterized&quot; and &quot;over-parameterized&quot; terms do not currently have a widely accepted definitions.&lt;/li&gt;&#xA;&lt;li&gt;Any definition for those term should consider the input data domain as well as the architecture and training procedure.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In a recent paper &lt;a href=&quot;https://openai.com/blog/deep-double-descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Double Descent&lt;/a&gt; from OpenAI Nakkiran et. al 2019, the authors tried to formalized and generalized the concept of &quot;interpolation threshold&quot; in the &quot;double descent&quot; phenomenon, both terms popularized by Belkin et. al 2019 &lt;a href=&quot;https://arxiv.org/abs/1812.11118&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reconciling modern Machine Learning practices with the Bias-Variance Tradeoff&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Deep Double Descent paper, they define a concept called &quot;Effective Model Complexity (EMC)&quot;, which include model architecture, training procedure and data to describe the &quot;interpolation threshold&quot; (the moment at which model can fit near perfect the training data). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EMC below the interpolation threshold is considered &quot;under-parameterized&quot; and above interpolation threshold is considered &quot;over-parameterized&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So according to this definition of EMC, I suppose:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1M-parameters model on 10M samples of 2 binary features, then it should not be over-parameterized&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is over-parameterized because of the simplicity of input data.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1M-parameters model on 0.1M samples of 512x512 images, then is over-parameterized, or&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is probably not parameterized, if it cannot fit the training data with near 0 loss.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the model in the paper Exploring the Limits of Weakly Supervised Pretraining &quot;IG-940M-1.5k ResNeXt-101 32×48d&quot; with 829M parameters, trained on 1B Instagram images, is not over-parameterized &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is not over parameterized because it cannot fit the entire training data perfectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious to see if EMC will catch on and be a popular measure for model complexity in the future.&lt;/p&gt;&#xA;" OwnerUserId="32055" LastActivityDate="2019-12-16T16:22:38.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17130" PostTypeId="1" CreationDate="2019-12-16T17:23:42.057" Score="2" ViewCount="37" Body="&lt;p&gt;I'm working on a question answering bot as my graduation project. The main concept is having a text file with many sentences, and building a question answering bot which answers a user's question based on the text file in hand.&lt;/p&gt;&#xA;&lt;p&gt;Until now, I used &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tf-idf&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Cosine_similarity&quot; rel=&quot;nofollow noreferrer&quot;&gt;cosine similarity&lt;/a&gt; and the results are somewhat satisfactory. The main problem is, if the user was to ask a question that doesn't have a word that is in the text file, my bot can't deduce what to bring back as an answer. For example, if I have a sentence in my text file that says &amp;quot;I have a headache because my heart rate is low&amp;quot;, if the user was to ask &amp;quot;Why do you have a headache?&amp;quot;, my bot chooses the correct sentence, but if he asked &amp;quot;What's wrong with you?&amp;quot; my bot doesn't know what to do.&lt;/p&gt;&#xA;&lt;p&gt;All I've seen on the web until now are deep learning methods and neural networks, such as LSTM and such. I was wondering if there are any &lt;strong&gt;pure NLP&lt;/strong&gt; approaches to go with my requirements.&lt;/p&gt;&#xA;" OwnerUserId="32120" LastEditorUserId="2444" LastEditDate="2021-01-26T15:36:41.920" LastActivityDate="2021-01-26T15:36:41.920" Title="Are there any approaches other than deep learning to deal with unexpected questions in a question answering system?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;question-answering&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17131" PostTypeId="2" ParentId="17074" CreationDate="2019-12-16T18:09:55.613" Score="1" Body="&lt;p&gt;The question confuses ML (including DL) with AI. AI is a bigger field than ML and includes &lt;a href=&quot;https://en.wikipedia.org/wiki/Rule-based_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;rule-based systems&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably need to extract entities (spans of text) from the unstructured text and embed them into an XML. ML (and DL) are good when the problem is fuzzy (you need very many rules to solve the problem) so it could be a valid option if you have a variety of document structures that each needs its own set of rules. You would need enough data to train your models in this case. Otherwise if you have limited document structures, very limited data (maybe none) and 100% accuracy is expected then going with rules is the obvious choice. &lt;/p&gt;&#xA;" OwnerUserId="27851" LastActivityDate="2019-12-16T18:09:55.613" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17132" PostTypeId="1" CreationDate="2019-12-16T18:41:13.057" Score="2" ViewCount="57" Body="&lt;p&gt;I am trying to teach my AI to talk. The problem is I'm struggling to find a good scenario in which it needs to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ideas I had were:&#xA;&quot;Describe a geometric scene&quot; - Then together with a parser we could see how close the generated instructions came to the official geometric language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Give another AI instructions of where to find some food&quot; e.g. &quot;Go straight on passed the box then turn left until you get to the tree. Look under the rock.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another one might be &quot;Find out more information about a scene by asking questions of another AI in order to navigate a scene blindfolded&quot;. This is quite an extreme example!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need it to talk in formal English sentences (not some kind of made up secret langauge.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically instead of just interpreting a language and following instructions, I want my AI to generate instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the things I want to teach it are the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ability to ask questions + ability to use the information gathered&lt;/li&gt;&#xA;&lt;li&gt;Ability to give instructions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Do you know of any projects like this?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-12-16T19:50:17.557" Title="Giving an AI a purpose to talk" Tags="&lt;natural-language-processing&gt;&lt;algorithm&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17133" PostTypeId="2" ParentId="17132" CreationDate="2019-12-16T19:50:17.557" Score="2" Body="&lt;p&gt;I just came across this piece of news yesterday: &lt;a href=&quot;https://www.microsoft.com/en-us/research/blog/first-textworld-problems-microsoft-research-montreals-latest-ai-competition-is-really-cooking/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;This week, Microsoft Research threw down the gauntlet with the launch of a competition challenging researchers around the world to develop AI agents that can solve text-based games.&quot;&lt;/a&gt; &#xA;This seems to be an AI competition announced by Microsoft with the aim to create AI that can solve text-based games. This might give you some inspiration.&lt;/p&gt;&#xA;" OwnerUserId="32124" LastActivityDate="2019-12-16T19:50:17.557" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17134" PostTypeId="1" CreationDate="2019-12-16T19:53:07.690" Score="1" ViewCount="29" Body="&lt;p&gt;Are there example implementations of networks that apply constraints across sequences of image classifications where class labels are ordinal numbers?  For example, to cause the output of a CNN to monotonically increase across frames, where the number may increase either more or less steeply but only in one direction across the entire sequence, or as another example, to smoothly vary rather than jumping precipitously from frame to frame. In my first example, the output can jump quickly from one frame  to the next, as long as only in one direction, whereas in my second example, they can either increase or decrease as long as not too &quot;fast&quot; from one frame to the next as if being passed through a low pass boxcar filter. The first is a monotonicity constraint and the second is a smoothness constraint, but in both examples, the key is for adjacent frames to have an effect on the conclusions for a given frame. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you,&#xA;Andy&lt;/p&gt;&#xA;" OwnerUserId="32123" LastActivityDate="2019-12-16T19:53:07.690" Title="Imposing contraints on sequence of image classifications" Tags="&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;optimization&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17136" PostTypeId="1" CreationDate="2019-12-17T05:59:47.853" Score="2" ViewCount="79" Body="&lt;p&gt;I use Q-learning in order to determine the optimal path of an agent. I know in advance that my path is composed of exactly 3 states (so after 3 states I reach a terminal state). I would like to know how to include that in the updating rule of the q-function.&#xA;What I am doing currently &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t=1:Nb_Epochs-1       &#xA;        if rand(1)&amp;lt;Epsilon&#xA;            an action 'a' is chosen  at random             &#xA;        else&#xA;           [Maxi a]=max(QIter(CurrentState,:,t)); &#xA;        end&#xA;        &#xA;         NextState=FindNextState(CurrentState,a);&#xA;         QIter(CurrentState,a,t+1)=(1-LearningRate)*QIter(CurrentState,a,t)+LearningRate*(Reward(NexState)+Discount*max(QIter(NextState,:,t)));&#xA;         CurrentState=NextState;&#xA;end&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32132" LastActivityDate="2019-12-17T05:59:47.853" Title="Q-learning: How to include a terminal state in updating rule?" Tags="&lt;q-learning&gt;" AnswerCount="0" CommentCount="5" ClosedDate="2020-11-01T14:12:46.843" ContentLicense="CC BY-SA 4.0" />
  <row Id="17137" PostTypeId="1" CreationDate="2019-12-17T09:07:06.150" Score="2" ViewCount="187" Body="&lt;p&gt;I want to make a face authentication application. I need to approve the face during the login based on whether the registered face and the login face match. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which are the possible appropriate AI methods or technologies for this task?&lt;/p&gt;&#xA;" OwnerUserId="31576" LastEditorUserId="2444" LastEditDate="2019-12-17T17:58:19.600" LastActivityDate="2019-12-18T09:29:22.207" Title="Which AI methods are most appropriate for login face recognition?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;reference-request&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17138" PostTypeId="2" ParentId="17137" CreationDate="2019-12-17T09:40:50.707" Score="3" Body="&lt;p&gt;One of the methods which is quite fast and easy to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do &lt;a href=&quot;https://en.wikipedia.org/wiki/Principal_component_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principal Component Analysis (PCA)&lt;/a&gt; based face recognition. You can go through this &lt;a href=&quot;https://www.mitpressjournals.org/doi/abs/10.1162/jocn.1991.3.1.71&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; for the theory behind it. For an example implementation you can see &lt;a href=&quot;https://jeremykun.com/2011/07/27/eigenfaces/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; post. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process, roughly, is as following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a grayscale image of size &lt;span class=&quot;math-container&quot;&gt;$(20,20)$&lt;/span&gt;, then this image can be flattened to a vector of size &lt;span class=&quot;math-container&quot;&gt;$400$&lt;/span&gt;. If you have &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; images of size &lt;span class=&quot;math-container&quot;&gt;$(20,20)$&lt;/span&gt; then your data matrix will be of size &lt;span class=&quot;math-container&quot;&gt;$(5,400)$&lt;/span&gt;. &#xA;We wish to find the principal components of the distribution of faces, or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;eigenvectors&lt;/a&gt; of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Covariance_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;covariance matrix&lt;/a&gt; of the data matrix, which will be of size &lt;span class=&quot;math-container&quot;&gt;$(5,5)$&lt;/span&gt;. Each eigenvector (out of total &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;), of size &lt;span class=&quot;math-container&quot;&gt;$(5,1)$&lt;/span&gt;, accounts for some amount of variation among the images, and they are ordered by the amount (the &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors&quot; rel=&quot;nofollow noreferrer&quot;&gt;eigenvalues corresponding to the eigenvectors&lt;/a&gt;) they account for. These eigenvectors can be thought of as a &lt;strong&gt;set of features&lt;/strong&gt; that &lt;strong&gt;characterize the variation between the images&lt;/strong&gt;. Recognition is done by &lt;strong&gt;projecting a new image (flattened to size &lt;span class=&quot;math-container&quot;&gt;$(400,1)$&lt;/span&gt;)into the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_subspace&quot; rel=&quot;nofollow noreferrer&quot;&gt;subspace&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_span&quot; rel=&quot;nofollow noreferrer&quot;&gt;spanned&lt;/a&gt; by the eigenvectors&lt;/strong&gt; of the covariance matrix and then classifying the face by comparing its &lt;strong&gt;coordinates&lt;/strong&gt; in the face space with the &lt;strong&gt;coordinates of the known individuals&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see an implementation in &lt;a href=&quot;https://github.com/TheHBK/signal-processing/tree/master/eigenfaces&quot; rel=&quot;nofollow noreferrer&quot;&gt;this git repo&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16708" LastEditorUserId="16708" LastEditDate="2019-12-18T09:29:22.207" LastActivityDate="2019-12-18T09:29:22.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17140" PostTypeId="1" CreationDate="2019-12-17T10:49:07.360" Score="3" ViewCount="40" Body="&lt;p&gt;Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code written in another language? Could they also be used to convert JSON to XML? Has this been done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are plenty of models that generate source code (which looks like real source code) using &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNNs&lt;/a&gt;, although the generated source code doesn't make logical sense. I haven't been able to find any models or examples that take valid existing code and convert it into different valid code.&lt;/p&gt;&#xA;" OwnerUserId="32142" LastEditorUserId="2444" LastEditDate="2019-12-17T18:04:19.973" LastActivityDate="2019-12-17T18:04:19.973" Title="Can sequence-to-sequence models be used to convert source code from one programming language to another?" Tags="&lt;deep-learning&gt;&lt;sequence-modelling&gt;&lt;machine-translation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17141" PostTypeId="1" CreationDate="2019-12-17T11:10:35.880" Score="1" ViewCount="23" Body="&lt;p&gt;What are general best practices or considerations in designing a model that is optimized for real-time inference?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastActivityDate="2019-12-17T11:10:35.880" Title="What are the properties of a model that is well suited for for high performance real-time inference" Tags="&lt;ai-design&gt;&lt;models&gt;&lt;performance&gt;&lt;gpu&gt;&lt;real-time&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17142" PostTypeId="1" AcceptedAnswerId="17181" CreationDate="2019-12-17T14:25:02.460" Score="4" ViewCount="48" Body="&lt;p&gt;In &lt;a href=&quot;https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;“Abandoning Objectives: Evolution through the Search for Novelty Alone”&lt;/a&gt;, it is explained how the novelty search is a function that is domain specific, depending on the differing behaviors that can potentially emerge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary test is a deceptive maze and it seems like they define novelty as a function that is dependent on each actor's ending position as a distance from other actors' ending position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wanting to try implementing this on some tasks. Some simple AI tasks such as playing pong, or recreating &lt;a href=&quot;https://www.youtube.com/watch?v=qv6UVOQ0F44&quot; rel=&quot;nofollow noreferrer&quot;&gt;MarI/O&lt;/a&gt;, or sticking them in an arena as an actor who can move, turn, and shoot (with other actors in the arena with them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a really hard time thinking of how to model the behavior functions for these kinds of instances without making it into an objective.&#xA;For pong, I imagine I could determine novelty by the AI's point score, but isn't this basically making the score an objective since it can only go up? For MarI/O, I've seen some implementations that look at the list of unique grid locations that Mario visited in what order, but I didn't come up with that myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the arena example, my first impulse is to have a score based on how long the actor survived and how many other actors the AI eliminated; but again, this can only go up and seems to me like it is defining an objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any strategies or ways to think about the problems that would help me better visualize the 'behavior space' and make a better novelty function?&lt;/p&gt;&#xA;" OwnerUserId="32148" LastActivityDate="2019-12-20T09:22:39.867" Title="Are there any strategies that would help me visualize the 'behavior space' and make a novelty function?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;novelty-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17143" PostTypeId="1" CreationDate="2019-12-17T17:34:50.770" Score="1" ViewCount="44" Body="&lt;p&gt;Does it make sense to use Reinforcement Learning methods in an environment that does not have trajectories?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a lot of states and actions in my environment. However, there are no trajectories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the agent takes action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, it reaches &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. In &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, it takes action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; and reaches state &lt;span class=&quot;math-container&quot;&gt;$s''$&lt;/span&gt;. However, if it takes the reverse order of actions &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; and then &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, it would reach the same state &lt;span class=&quot;math-container&quot;&gt;$s''$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How reinforcement learning methods handle with this?&lt;/p&gt;&#xA;" OwnerUserId="32153" LastEditorUserId="2444" LastEditDate="2019-12-17T17:50:03.673" LastActivityDate="2019-12-17T17:50:03.673" Title="Reinforcement learning without trajectories" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="4" ClosedDate="2020-11-16T21:02:54.237" ContentLicense="CC BY-SA 4.0" />
  <row Id="17144" PostTypeId="1" CreationDate="2019-12-17T21:22:35.997" Score="1" ViewCount="21" Body="&lt;p&gt;I'm confused about the following issue. Let assume that we have a neural network that takes one input and two outputs. I try to visualize my model like as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        / --- First stream    --- &amp;gt; output_1&#xA;Input --&#xA;        \ ---- Second stream  ---&amp;gt; output_2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used sgd with momentum. Is there any difference between using one optimizer for both streams and using two optimizers for each stream? In other words, if i use one optimizer, can one stream optimization process affect another stream? If it can, How can it be possible?&lt;/p&gt;&#xA;" OwnerUserId="32159" LastActivityDate="2019-12-17T21:22:35.997" Title="Optimizer effects on neural network with two outputs" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17146" PostTypeId="1" CreationDate="2019-12-18T01:24:40.180" Score="1" ViewCount="24" Body="&lt;h1&gt;Short question&lt;/h1&gt;&#xA;&lt;p&gt;How can I implement Logic Inference with Deep Learning?&lt;/p&gt;&#xA;&lt;h1&gt;Long question&lt;/h1&gt;&#xA;&lt;p&gt;Based on Symbolic Logic, chaining multiple predicates (a short example is Syllogism) is a method of implementing Logic Inference. This programmatic way is suitable for many cases, however, if combined with NLP, too much programmatic way is involved.&lt;/p&gt;&#xA;&lt;p&gt;If someone implements that Logic Inference with not programmatic way but machine learning, what model should be adopted, and what label data should be input to the model?&lt;/p&gt;&#xA;" OwnerUserId="32162" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-12-18T01:24:40.180" Title="Implementing Logic Inference with Deep Learning" Tags="&lt;deep-learning&gt;&lt;logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17147" PostTypeId="1" CreationDate="2019-12-18T04:09:36.720" Score="2" ViewCount="26" Body="&lt;p&gt;I'm implementing recurrent BN per &lt;a href=&quot;https://arxiv.org/abs/1603.09025&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; in Keras, but looking at it and those citing it, a detail remains unclear to me: how are &lt;em&gt;batch statistics&lt;/em&gt; computed? Authors omit explicit clarification, but state (pg. 3) (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At training time, the statistics E[&lt;strong&gt;h&lt;/strong&gt;] and Var[&lt;strong&gt;h&lt;/strong&gt;] are estimated by the sample mean and sample variance of the &lt;strong&gt;current minibatch&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yet &lt;a href=&quot;https://jaywhang.com/assets/batchnorm_rnn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;another paper&lt;/a&gt; (pg. 3) using and citing it describes:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We subscript BN by time (BN_t) to indicate that each time step tracks its own mean and variance. In practice, we &lt;strong&gt;track&lt;/strong&gt; these statistics as they change &lt;strong&gt;over the course of training&lt;/strong&gt; using an &lt;em&gt;exponential moving average&lt;/em&gt; (EMA)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My question's thus two-fold:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are minibatch statistics computed &lt;em&gt;per immediate minibatch&lt;/em&gt;, or as an &lt;em&gt;EMA&lt;/em&gt;? &lt;/li&gt;&#xA;&lt;li&gt;How are the inference parameters, shared across all timesteps, &lt;code&gt;gamma&lt;/code&gt; and &lt;code&gt;beta&lt;/code&gt; computed? Is the computation in (1) simply averaged across all timesteps? (e.g. average &lt;code&gt;EMA_t&lt;/code&gt; for all &lt;code&gt;t&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Existing implementations&lt;/strong&gt;: in Keras and TF below, but are all outdated, and am unsure regarding correctness&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/udibr/keras/blob/lstmbn/keras/layers/recurrent.py#L863&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras&lt;/a&gt;, &lt;a href=&quot;https://github.com/tam17aki/recurrent-batchnorm-tensorflow/blob/master/BN_LSTMCell.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-A&lt;/a&gt;, and &lt;a href=&quot;https://github.com/OlavHN/bnlstm/blob/master/lstm.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-B&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;All above agree that during training, &lt;em&gt;immediate minibatch&lt;/em&gt; statistics are used, and that &lt;code&gt;beta&lt;/code&gt; and &lt;code&gt;gamma&lt;/code&gt; are updated as an EMA of these minibatches&lt;/li&gt;&#xA;&lt;li&gt;Problem: the &lt;code&gt;bn&lt;/code&gt; operation (in A, and presumably B &amp;amp; C) is applied on a &lt;em&gt;single timestep&lt;/em&gt; slice, to be passed to the &lt;code&gt;K.rnn&lt;/code&gt; control flow for re-iteration. Hence, EMA is computed w.r.t. minibatches &lt;em&gt;and&lt;/em&gt; timesteps - which I find questionable:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;EMA is used in place of a simple average when population statistics are &lt;em&gt;dynamic&lt;/em&gt; (e.g. minibatch-to-minibatch), whereas we have access to &lt;em&gt;all timesteps&lt;/em&gt; in a minibatch prior having to update &lt;code&gt;gamma&lt;/code&gt; and &lt;code&gt;beta&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;EMA is a worse but at times necessary alternative to a simple average, but per above, we &lt;em&gt;can&lt;/em&gt; use latter - so why don't we? Timestep statistics can be cached, averaged at the end, then discarded - holds also for &lt;code&gt;stateful=True&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32165" LastEditorUserId="32165" LastEditDate="2019-12-20T04:42:38.190" LastActivityDate="2019-12-20T04:42:38.190" Title="How are batch statistics computed in Recurrent Batch Normalization?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;batch-normalization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17149" PostTypeId="2" ParentId="6435" CreationDate="2019-12-18T06:30:39.613" Score="1" Body="&lt;p&gt;This should simply be due to the fact that network weight initialization is random.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2019-12-18T06:30:39.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17150" PostTypeId="2" ParentId="1742" CreationDate="2019-12-18T07:05:08.430" Score="1" Body="&lt;p&gt;&lt;strong&gt;Deep Learning is subset of Machine Learning.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning and Deep learning both are not two different things. Deep learning is one of the form of machine learning.&#xA;The level of layers in Neural network are more and more in depth learning is part of Deep learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/52A1d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/52A1d.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;“Deep learning is a particular kind of machine learning that achieves&#xA;  great power and flexibility by learning to represent the world as&#xA;  nested hierarchy of concepts, with each concept defined in relation to&#xA;  simpler concepts, and more abstract representations computed in terms&#xA;  of less abstract ones.”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2019-12-18T07:05:08.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17151" PostTypeId="1" AcceptedAnswerId="17807" CreationDate="2019-12-18T07:15:41.730" Score="1" ViewCount="777" Body="&lt;p&gt;I have been exploring edge computation for AI, and I came across multiple libraries or frameworks, which can help to convert the model into a lite format, which is suitable for edge devices.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tensorflow.org/lite&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow Lite&lt;/a&gt; will help us to convert the TensorFlow model into TensorFlow lite.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://docs.openvinotoolkit.org/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenVino&lt;/a&gt; will optimise the model for edge devices.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If we have a library to optimise the model for edge devices (e.g. TensorFlow Lite), after conversion, could it make the accuracy decrease?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If not, then why do people prefer don't always use e.g. TensorFlow Lite?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="2444" LastEditDate="2021-02-17T22:46:41.083" LastActivityDate="2021-02-17T22:46:41.083" Title="Why don't people always use TensorFlow Lite, if it doesn't decrease the accuracy of the models?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;accuracy&gt;&lt;efficiency&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17152" PostTypeId="1" CreationDate="2019-12-18T07:46:47.973" Score="1" ViewCount="32" Body="&lt;p&gt;What is the difference between principal component analysis and singular value decomposition in image processing? Which one performs better, and why?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2019-12-18T13:56:03.347" LastActivityDate="2019-12-18T13:56:03.347" Title="What is the difference between principal component analysis and singular value decomposition in image processing?" Tags="&lt;computer-vision&gt;&lt;comparison&gt;&lt;image-processing&gt;&lt;principal-component-analysis&gt;&lt;singular-value-decomposition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17153" PostTypeId="1" CreationDate="2019-12-18T09:43:09.777" Score="3" ViewCount="38" Body="&lt;p&gt;I am trying to teach an agent to make any random 1-qubit state reach uniform superposition. So basically, the full circuit will be &lt;code&gt;State -&amp;gt; measurement -&amp;gt; new_state (|0&amp;gt; if 0, |1&amp;gt; if 1) -&amp;gt; Hadamard gate&lt;/code&gt;. It just needs to perform &lt;code&gt;2 actions&lt;/code&gt;. That's all. So it's more of an RL problem rather than QC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using reinforcement learning to train the model but it doesn't seem to learn anything. The reward keeps on decreasing and even after 3 million episodes, the agent doesn't seem to converge anywhere. This is how I am training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_exploration_rate(self, time_step):&#xA;        return self.epsilon_min + (self.epsilon - self.epsilon_min)*\&#xA;               math.exp(1.*time_step*self.epsilon_decay)&#xA;&#xA;def act(self, data,t): #state&#xA;        rate = self.get_exploration_rate(t)&#xA;        if random.random() &amp;lt; rate:&#xA;            options = self.model.predict(data) #state&#xA;            options = np.squeeze(options)&#xA;            action =  random.randrange(self.action_size)&#xA;        else:&#xA;            options = self.model.predict(data) #state&#xA;            options = np.squeeze(options)&#xA;            action = options.argmax()&#xA;        return action, options, rate&#xA;&#xA;def train(self):&#xA;&#xA;        batch_size = 200&#xA;        t = 0                   #increment&#xA;        states, prob_actions, dlogps, drs, proj_data, reward_data =[], [], [], [], [], []&#xA;        tr_x, tr_y = [],[]&#xA;        avg_reward = []&#xA;        reward_sum = 0&#xA;        ep_number = 0&#xA;        prev_state = None&#xA;        first_step = True&#xA;        new_state = self.value&#xA;        data_inp = self.data&#xA;&#xA;        while ep_number&amp;lt;3000000:&#xA;            prev_data = data_inp&#xA;            prev_state = new_state&#xA;            states.append(new_state)&#xA;            action, probs, rate = self.act(data_inp,t)&#xA;            prob_actions.append(probs)&#xA;            y = np.zeros([self.action_size])&#xA;            y[action] = 1&#xA;            new_state = eval(command[action])&#xA;            proj = projection(new_state, self.final_state)&#xA;            data_inp = [proj,action]&#xA;            data_inp = np.reshape(data_inp,(1,1,len(data_inp)))&#xA;            tr_x.append(data_inp)&#xA;            if(t==0):&#xA;                rw = reward(proj,0)&#xA;                drs.append(rw)&#xA;                reward_sum+=rw&#xA;&#xA;            elif(t&amp;lt;4):&#xA;                rw = reward(new_state, self.final_state)&#xA;                drs.append(rw)&#xA;                print(&quot;present reward: &quot;, rw)&#xA;                reward_sum+=rw&#xA;            elif(t==4):&#xA;                if not np.allclose(new_state, self.final_state):&#xA;                    rw = -1&#xA;                    drs.append(rw)&#xA;                    reward_sum+=rw&#xA;                else:&#xA;                    rw = 1&#xA;                    drs.append(rw)&#xA;                    reward_sum+=rw&#xA;&#xA;            print(&quot;reward till now: &quot;,reward_sum)&#xA;            dlogps.append(np.array(y).astype('float32') * probs)&#xA;            print(&quot;dlogps before time step: &quot;, len(dlogps))&#xA;            print(&quot;time step: &quot;,t)&#xA;            del(probs, action)&#xA;            t+=1&#xA;            if(t==5 or np.allclose(new_state,self.final_state)):                         #### Done State&#xA;                ep_number+=1&#xA;                ep_x = np.vstack(tr_x) #states&#xA;                ep_dlogp = np.vstack(dlogps)&#xA;                ep_reward = np.vstack(drs)&#xA;                disc_rw = discounted_reward(ep_reward,self.gamma)&#xA;                disc_rw = disc_rw.astype('float32')&#xA;                disc_rw -= np.mean(disc_rw)&#xA;                disc_rw /= np.std(disc_rw)&#xA;&#xA;                tr_y_len = len(ep_dlogp)&#xA;                ep_dlogp*=disc_rw&#xA;                if ep_number % batch_size == 0:&#xA;                  input_tr_y = prob_actions - self.learning_rate * ep_dlogp&#xA;                    input_tr_y = np.reshape(input_tr_y, (tr_y_len,1,6))&#xA;&#xA;                    self.model.train_on_batch(ep_x, input_tr_y)&#xA;                    tr_x, dlogps, drs, states, prob_actions, reward_data = [],[],[],[],[],[]&#xA;                env = Environment()&#xA;                new_state = env.reset()&#xA;                proj = projection(state, self.final_state)&#xA;                data_inp = [proj,5]&#xA;                data_inp = np.reshape(data_inp,(1,1,len(data_inp)))&#xA;                print(&quot;State after resetting: &quot;, new_state)&#xA;                t=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried various things like changing the inputs, reward function, even added &lt;code&gt;exploration&lt;/code&gt; rate. I have assigned max time step as &lt;code&gt;5&lt;/code&gt; even though it should complete in just &lt;code&gt;2&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong? Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="29843" LastActivityDate="2019-12-18T09:43:09.777" Title="Reinforcement Learning on quantum circuit" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;python&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17154" PostTypeId="1" AcceptedAnswerId="17163" CreationDate="2019-12-18T09:46:26.260" Score="3" ViewCount="256" Body="&lt;p&gt;Currently working on Gomoku AI implementation with &lt;strong&gt;minmax + alpha beta implementation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm targeting these two rules from '&lt;em&gt;acceptable implementation&lt;/em&gt;' in terms of &lt;strong&gt;search time&lt;/strong&gt; and &lt;strong&gt;search depth&lt;/strong&gt; :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Search time (over 0.5 seconds is &lt;em&gt;&quot;bad&quot;&lt;/em&gt;, less 0.5 seconds is &lt;em&gt;ok&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Search depth (less than 10 search depth levels is &lt;em&gt;&quot;bad&quot;&lt;/em&gt;, over 10 search depth levels is &lt;em&gt;ok&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The minmax algorithm generates, by recursive function calls, a tree of nodes, each node represented by a function call with a specific game state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Increasing the depth search increases the number of nodes in the tree, and therefore search time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a compromise between &lt;strong&gt;search time&lt;/strong&gt; and &lt;strong&gt;search depth&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alpha beta pruning&lt;/strong&gt; tends to help this compromise by pruning useless nodes search and reducing tree size. The pruning is directly related to evaluation/heuristic function. Bad implementation of heuristic may lead to bad efficiency of alpha beta pruning.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;QUESTION&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If you are workin on or have done a Gomoku AI, &lt;strong&gt;sharing your stats of tree size, search depth and time search from your implementation at some game steps, and explain how you reach it may help to investigate&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The implementation at this time does not fit the '&lt;em&gt;is not acceptable&lt;/em&gt;' for me, having &lt;strong&gt;search time&lt;/strong&gt; over 1sec for a &lt;strong&gt;search depth&lt;/strong&gt; of 4 at first step ... on IntelCore i7 3.60GHz CPU ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the properties of the actual implementation :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Board of size 19x19&lt;/li&gt;&#xA;&lt;li&gt;Implements search window of size 5x5 around stones to reduce search nodes&lt;/li&gt;&#xA;&lt;li&gt;Implements heuristic computation at each node on the played stone instead of computation on all board size on leaf nodes number.&lt;/li&gt;&#xA;&lt;li&gt;Implements alpha beta pruning &lt;/li&gt;&#xA;&lt;li&gt;No multi thread&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here are the current stats it is reaching for &lt;strong&gt;search depth of 4&lt;/strong&gt; at first  step:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;== Timing minmax_algorithm: 1.706175 seconds ==&lt;/li&gt;&#xA;&lt;li&gt;== Number of nodes in that compose the tree: 2850 ==&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;. . . . . . . . . . . . . . . . . . . 00&#xA;. . . . . . . . . . . . . . . . . . . 01&#xA;. . . . . . . . . . . . . . . . . . . 02&#xA;. . . . . . . . . . . . . . . . . . . 03&#xA;. . . . . . . . . . . . . . . . . . . 04&#xA;. . . . . . . . . . . . . . . . . . . 05&#xA;. . . . . . . . . . . . . . . . . . . 06&#xA;. . . . . . . . . . . . . . . . . . . 07&#xA;. . . . . . . . . . . . . . . . . . . 08&#xA;. . . . . . . . . . . . . . . . . . . 09&#xA;. . . . . . . . . . . . . . . . . . . 10&#xA;. . . . . . . . . . . . . . . . . . . 11&#xA;. . . . . . . . . . . . . . . . . . . 12&#xA;. . . . . . . . . . . . . . . . . . . 13&#xA;. . . . . . . . . . . . . . . . . . . 14&#xA;o x . . . . . . . . . . . . . . . . . 15&#xA;. . . . . . . . . . . . . . . . . . . 16&#xA;. . . . . . . . . . . . . . . . . . . 17&#xA;. . . . . . . . . . . . . . . . . . . 18&#xA;A B C D E F G H I J K L M N O P Q R S &#xA;Player: o - AI: x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Bad stats may be lead to bad heuristic, causing inefficient pruning. Waiting from other stats/replys to validate this hypothesis may help ! Thank you !&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit 1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Coming back from new search campain on this question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The implementation was facing a 19*19 loop index at each heuristic computation ...&lt;br/&gt;Removed this by heuristic computation at speicific index (not entire board) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The implementation was facing a 19*19 loop index to check win state ...&lt;br/&gt;Removed this by checking only around played index any alignement at each step.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The implementation was facing a 19*19 loop index to check where it can play (even with the windows) ...&lt;br/&gt;&#xA;Removed by propagating indexes array of valid indexes through the recursion updated at each step.&lt;br/&gt;&#xA;The array is a dichotomic array (with O(n) insertion, O(log n) search and O(1) deletion by index)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The implementation was lacking of zobrist hash table, very nice idea from below answer.&lt;br/&gt;It is now implemented with unit tests to prove that implementation is working.&lt;br/&gt;An array sorted by hash is updates at each new node, with hash-node association.&lt;br/&gt;The array is a dichotomic array (with O(n) insertion, O(log n) search and O(1) deletion by index)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The implementation is at each step trying each indexes in a random way (not computation order or evaluation score order).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The before edit example is not great because it is playing on side board and allowed indexes window is half max size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the new obtains performances :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;with wobrist table &lt;strong&gt;off&lt;/strong&gt; and seed at 42 for &lt;strong&gt;search depth of 4&lt;/strong&gt; at first  step&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;== Timing minmax_algorithm: 0.083288 seconds ==&lt;/li&gt;&#xA;&lt;li&gt;== Number of nodes in that compose the tree: 6078 ==&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;. . . . . . . . . . . . . . . . . . . 00&#xA;. . . . . . . . . . . . . . . . . . . 01&#xA;. . . . . . . . . . . . . . . . . . . 02&#xA;. . . . . . . . . . . . . . . . . . . 03&#xA;. . . . . . . . . . . . . . . . . . . 04&#xA;. . . . . . . . . . . . . . . . . . . 05&#xA;. . . . . . . . . . . . . . . . . . . 06&#xA;. . . . . . . . . . . . . . . . . . . 07&#xA;. . . . . . . . . . . . . . . . . . . 08&#xA;. . . . . . . . . . . . . . . . . . . 09&#xA;. . . . . . . . . . . . . . . . . . . 10&#xA;. . . . . . . . . x . . . . . . . . . 11&#xA;. . . . . . . . o . . . . . . . . . . 12&#xA;. . . . . . . . . . . . . . . . . . . 13&#xA;. . . . . . . . . . . . . . . . . . . 14&#xA;. . . . . . . . . . . . . . . . . . . 15&#xA;. . . . . . . . . . . . . . . . . . . 16&#xA;. . . . . . . . . . . . . . . . . . . 17&#xA;. . . . . . . . . . . . . . . . . . . 18&#xA;A B C D E F G H I J K L M N O P Q R S&#xA;Player: o - AI: x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;with wobrist table &lt;strong&gt;on&lt;/strong&gt; and seed at 42 for &lt;strong&gt;search depth of 4&lt;/strong&gt; at first  step&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;== Timing minmax_algorithm: 0.434098 seconds ==&lt;/li&gt;&#xA;&lt;li&gt;== Number of nodes in that compose the tree: 9320 ==&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;. . . . . . . . . . . . . . . . . . . 00&#xA;. . . . . . . . . . . . . . . . . . . 01&#xA;. . . . . . . . . . . . . . . . . . . 02&#xA;. . . . . . . . . . . . . . . . . . . 03&#xA;. . . . . . . . . . . . . . . . . . . 04&#xA;. . . . . . . . . . . . . . . . . . . 05&#xA;. . . . . . . . . . . . . . . . . . . 06&#xA;. . . . . . . . . . . . . . . . . . . 07&#xA;. . . . . . . . . . . . . . . . . . . 08&#xA;. . . . . . . . . . . . . . . . . . . 09&#xA;. . . . . . x . . . . . . . . . . . . 10&#xA;. . . . . . . . . . . . . . . . . . . 11&#xA;. . . . . . . . o . . . . . . . . . . 12&#xA;. . . . . . . . . . . . . . . . . . . 13&#xA;. . . . . . . . . . . . . . . . . . . 14&#xA;. . . . . . . . . . . . . . . . . . . 15&#xA;. . . . . . . . . . . . . . . . . . . 16&#xA;. . . . . . . . . . . . . . . . . . . 17&#xA;. . . . . . . . . . . . . . . . . . . 18&#xA;A B C D E F G H I J K L M N O P Q R S&#xA;Player: o - AI: x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually it is ok for search depth 4, but not for more than 6. Node number is becoming exponential (over 20 000) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Found &lt;a href=&quot;https://github.com/terngkub/gomoku&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; great implementation in the same language/techno than can go to 10 depth in less than 1sec, without zobrist or smart trick, and followed the logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue must be somwhere else, causing exponential growth of node - inefficient pruning.&lt;/p&gt;&#xA;" OwnerUserId="32168" LastEditorUserId="32168" LastEditDate="2019-12-31T13:44:26.470" LastActivityDate="2019-12-31T13:44:26.470" Title="Minmax algorithm, search time and search depth compromise and acceptability" Tags="&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17155" PostTypeId="1" CreationDate="2019-12-18T12:51:06.050" Score="2" ViewCount="128" Body="&lt;p&gt;Consider an optimization problem that involves a set of tasks &lt;span class=&quot;math-container&quot;&gt;$T = \{1,2,3,4,5\}$&lt;/span&gt;, where the goal is to find a certain order of these tasks.&lt;/p&gt;&#xA;&lt;p&gt;I would like to solve this problem with a genetic algorithm, where each chromosome &lt;span class=&quot;math-container&quot;&gt;$C = [i, j, k, l, m]$&lt;/span&gt; corresponds to a specific order of these five tasks, so each gene in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; corresponds to a task in &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, &lt;span class=&quot;math-container&quot;&gt;$C = [1,3,5,4,2]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C' = [1,5,4,2,3]$&lt;/span&gt; would be two chromosomes that correspond to two different orders of the tasks.&lt;/p&gt;&#xA;&lt;p&gt;In this case, how could we design the mutation and cross-over operations so that these constraints are maintained during evolution?&lt;/p&gt;&#xA;&lt;p&gt;The genetic algorithm should produce the three best chromosomes or order of tasks.&lt;/p&gt;&#xA;" OwnerUserId="31274" LastEditorUserId="2444" LastEditDate="2020-12-08T15:52:35.300" LastActivityDate="2020-12-08T15:52:35.300" Title="How can we design the mutation and crossover operations when the order of the genes in the chromosomes matters?" Tags="&lt;genetic-algorithms&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;&lt;constrained-optimization&gt;&lt;chromosomes&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17156" PostTypeId="2" ParentId="17108" CreationDate="2019-12-18T13:06:44.600" Score="2" Body="&lt;p&gt;My understanding of your question is, you have 2 designs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A deterministic policy that outputs 2 scalar for x and y respectively.&lt;/li&gt;&#xA;&lt;li&gt;A value function that outputs the probability of each pixel in the 2D grid.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you choose the max of softmax on (2.), you'll get the same deterministic policy as (1.), assuming there are some tie-breaking designs. So I don't think choosing the max of softmax is a great policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the question becomes &quot;deterministic policy vs. value function&quot;. Since the value function have more information, we can have special designs on exploration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In (1.), if you use &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy on the discrete output, It'll take a random move with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; probability. When exploring (instead of exploiting), all actions except the best action will have the same probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in (2.), the probability can act like a value function (the value of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a) \forall a$&lt;/span&gt; in a certain state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;), and you can use something like upper confidence bound (UCB) instead of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy. If you are using neural networks to output this probability distribution, you can &lt;a href=&quot;https://github.com/dennybritz/reinforcement-learning/issues/34&quot; rel=&quot;nofollow noreferrer&quot;&gt;add an entropy term in the loss function like in A3C&lt;/a&gt; to encourage exploration.&lt;/p&gt;&#xA;" OwnerUserId="32173" LastEditorUserId="32173" LastEditDate="2019-12-18T18:46:22.643" LastActivityDate="2019-12-18T18:46:22.643" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17157" PostTypeId="2" ParentId="17151" CreationDate="2019-12-18T13:50:35.103" Score="3" Body="&lt;p&gt;This partly answer to question 1.&#xA;There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data.&#xA;The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure of the difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensor flow has different options to save the &quot;lite&quot; model (optimized in size, latency, none and default). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following mostly answer question 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensor flow lite is intented to provide the ability to use the model to on line &lt;strong&gt;predict only&lt;/strong&gt; and load the model not to train the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand Tensor flow is used to build (train) the model off line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your edge platform support any of the binding language provided for TensorFlow (javascript, java/kotlin, C++, python) you can use Tensorflow for prediction. The accuracy or speed options you might have selected to create the model will not be affected whether you use Tensor Flow or Tensor Flow Lite. Typically Tensor flow lite can be used on mobile devices (Ios, Android). There are other supported target, see this &lt;a href=&quot;https://www.tensorflow.org/lite/microcontrollers&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30392" LastEditorUserId="30392" LastEditDate="2020-01-09T13:38:20.123" LastActivityDate="2020-01-09T13:38:20.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17158" PostTypeId="1" CreationDate="2019-12-18T14:00:23.213" Score="1" ViewCount="15" Body="&lt;p&gt;We are currently using a RL network with the following simple structure to train a model which helps to solve a transformation task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Environment (a binary file) + reward ---&gt; LSTM (embedding)  --&gt; FC layer --&gt; FC layer --&gt; FC layer --&gt; decision (to select and apply a kind of transformation toward the environment from a pool of transformations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model will receive a simple reward and also take the input to make the decision. And we have a condition to stop each episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the current workflow, although it is simple, it seems to have learned something and with multiple episode of training, we can observe the accumulated reward for each episode increases. So right now, what we are thinking is to &lt;code&gt;interpret&lt;/code&gt; the model, well, a fancy term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically we are thinking to let the model tell us from which component of the Environment (input file), the model somewhat makes the decision to select a transformation to apply. And I have learned a bunch of interpretability articles, which basically use an activation map (e.g., &lt;a href=&quot;https://jacobgil.github.io/deeplearning/class-activation-maps?nsukey=AWNe1ED4Qk%2FmkOWgGIXn0dabS%2Bz4VNo%2BkEnUhfw0sUs2ViTBewAdmlgUoGfbTOlXUFnPrwHzzYjPMf9r2xJytPnXx%2BScM6tYqPH52ogxtLywL6auw2DlxLQ1rJo1QeVXu1Nujvpirc9EEhd%2F2YFQONv9SUIBgbYhgDSf2ZaQekWUa%2Bc3jnMDT9i1qVsC4A%2BS6nxfRgEH%2BGeAvYXUwNCoxw%3D%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) to highlight certain components from the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem is that, we don't have any sort of CNN layer in our simple RL model. In that sense, the aforementioned method cannot apply, right? I also learned a number of techniques from this &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/simple.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;, but still, I don't see any specific techniques applicable for RL models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here is my question, in terms of our simple RL model, how can we do certain &quot;interpretability&quot; analysis and therefore have a better idea on which part of the &quot;Environment&quot; leads to each step of decision? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="25973" LastActivityDate="2019-12-18T14:00:23.213" Title="How to perform Interpretability analysis toward a simple reinforcement learning network" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17159" PostTypeId="1" AcceptedAnswerId="17165" CreationDate="2019-12-18T15:24:12.927" Score="4" ViewCount="147" Body="&lt;p&gt;Artificial neural networks are composed of multiple neurons that are connected to each other. When the output of an artificial neuron is zero, it does not have any effect on neighboring neurons. When the output is positive, the neuron has an effect on neighboring neurons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does it mean when the output of a neuron is negative (which can e.g. occur when the activation function of a neuron is the hyperbolic tangent)? What effect would this output have on neighboring neurons? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do biological neural networks also have this property?&lt;/p&gt;&#xA;" OwnerUserId="31143" LastEditorUserId="2444" LastEditDate="2019-12-18T20:54:36.223" LastActivityDate="2019-12-18T22:50:04.147" Title="What effect does a negative output of a neuron have on neighbouring neurons?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;&lt;biology&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17160" PostTypeId="1" CreationDate="2019-12-18T16:05:55.887" Score="2" ViewCount="223" Body="&lt;p&gt;I am currently working on a problem and now got stuck to implement one of it's steps. This is a simple attempt to explain what I am currently facing, which is something that I am aiming to implement in my regression simulation in python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that I fit a non-linear model to my data. Now, I want to find the combination of inputs within a specified range that returns the the highest outcome. When I am using a quadratic function or only a few inputs, this task is quite simple. However, the problem comes when trying to apply the same logic for more complex models. Supposing that I have 9 variables as inputs, I will have to test all possible combinations and that would be computationally unfeasible by doing it with meshgrid if you want to cover a range with a several intervals in between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, here it comes my question, is there such a way to avoid having to go through this computationally costly process in order to achieve the combinations of inputs defined in a given range that return the optimal output?&lt;/p&gt;&#xA;" OwnerUserId="31880" LastActivityDate="2019-12-18T17:15:34.883" Title="Finding the optimal combination of inputs which return maximal output" Tags="&lt;machine-learning&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17161" PostTypeId="2" ParentId="17160" CreationDate="2019-12-18T17:10:00.303" Score="4" Body="&lt;p&gt;If your model is gradient-based, such as a neural network, then may also be able to use gradient methods to drive virtual inputs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Freeze all network weights to the trained version&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Define a loss function that decribes how you want the output - or any internal measure - to behave. E.g. to maximise the output, the loss function can simply be the negative of the output, assuming you will perform gradient descent later. Some libraries will also support gradient ascent to maximise a function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Define your input as a variable that can be optimised and instantiate an optimiser (details will vary depending on your library)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Start with a random or best-guess input, and iterate normal training routine (feed forward then backpropagate) to get better and better inputs&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is basically how Deep Dream and Style Transfer algorithms work - the detail that is different is definition of the loss functions. It is also a way to make adversarial attacks against known models, for example taking a picture of a car, modifying it such that a classifier returns that it is an ostrich (whilst it still looks like a car to a human).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not guaranteed to find the absolute best inputs for a given range, but should find good approximations of local minima or maxima far faster than a meshgrid when there are many dimensions. You could combine the idea with a coarse meshgrid or random search for start points for a better chance of finding the best results within input range constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should bear in mind that the ideal inputs you discover will only be as accurate as your trained model will let them be. If you discover maximising inputs in the model, that use inputs that are far away from any training examples, then in reality those inputs might not get anything like the result that the model predicts. Statistical models with many degrees of freedom are typically ok at interpolating between data points, but very bad at extrapolating beyond data that has been observed.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-18T17:15:34.883" LastActivityDate="2019-12-18T17:15:34.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17162" PostTypeId="1" CreationDate="2019-12-18T17:52:23.430" Score="1" ViewCount="18" Body="&lt;p&gt;I am writing a checkers move generation function in C that will be extended to Python. It is much easier to handle the possible boards in a fixed size array to pass back to Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I build out possible boards and add them to this array:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uint32_t boards[x][3];&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Therefore, the optimal value for x should be the maximum single ply branching factor out of all possible legal board states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if I am being very clear, so here is an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For tic-tac-toe this value would be 9, as the first move has the greatest number of possible directly resulting board states, out of all of the legal boards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this value been calculated for checkers? Has a program like Chinook derived a reasonably close number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help! &lt;/p&gt;&#xA;" OwnerUserId="31894" LastActivityDate="2019-12-18T17:52:23.430" Title="Maximum Single Ply Branching Factor for Legal Checkers Boards" Tags="&lt;branching-factors&gt;&lt;checkers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17163" PostTypeId="2" ParentId="17154" CreationDate="2019-12-18T18:01:36.897" Score="3" Body="&lt;p&gt;Intuitively I kind of doubt expecting a search depth of 10 in half a second is reasonable, especially for the initial game state where there's a rather large branching factor and no immediately-winning moves that help to prune some branches quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never implemented any Alpha-Beta agents for Gomoku specifically, but I can provide some numbers for our Alpha-Beta implementation in the &lt;a href=&quot;https://ludii.games/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ludii General Game System&lt;/a&gt;. &lt;strong&gt;Note that this is a general game system that implements a wide variety of games in a single game description language&lt;/strong&gt;. Due to its generality, it's unlikely that any single game runs as efficiently as it would in a highly-optimised game-specific implementation. Therefore, &lt;strong&gt;you should consider these numbers to be lower bounds on what you can achieve in a Gomoku-only game-specific implementation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We can reach a search depth of 3 for the initial game state in half a second.&lt;/li&gt;&#xA;&lt;li&gt;Increasing this to 10 seconds is still not enough for a depth of 4. I don't know how much I'd have to increase it to reach a depth of 4.&lt;/li&gt;&#xA;&lt;li&gt;At a max search time of 1 second, it actually seems to play quite well against a few different MCTS-based baselines. So I'm not sure if you really need a depth of 10 before you consider it &quot;acceptable&quot;.&lt;/li&gt;&#xA;&lt;li&gt;I'm not keeping track of the number of visited nodes, so can't provide those.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that it's &lt;strong&gt;very important&lt;/strong&gt; to take into account the computational cost of your heuristic evaluation function. We're using a somewhat expensive heuristic which computes all potential lines of a length of 5 (because this is appears in the end rules in Gomoku) through all pieces that have been placed so far, as described on pages 82-84 of &lt;a href=&quot;https://eprints.qut.edu.au/17025/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automatic generation and evaluation of recombination games&lt;/a&gt; (but with a simpler scoring rule than the union of probabilities as described there).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;CPU:&#xA;- Intel Core i5-6500 CPU @ 3.20GHz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game implementation details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;General game system (so not optimised for this specific game).&lt;/li&gt;&#xA;&lt;li&gt;I updated the board size to 19x19 to match your test, but as far as I'm aware 15x15 is more common (and the default in Ludii).&lt;/li&gt;&#xA;&lt;li&gt;Implemented in Java&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Alpha-Beta implementation details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Iterative deepening (so when I write that we reach depth 3, I mean that we completed searches of depth 1, followed by 2, followed by 3, and were probably in progress with a depth-4 search when we ran out of time).&lt;/li&gt;&#xA;&lt;li&gt;Ludii does not provide &lt;em&gt;undo&lt;/em&gt; operations for moves, only &lt;em&gt;apply&lt;/em&gt; operations. This means we have to create lots of copies of states, because we cannot undo moves after exiting out of a recursive call.&lt;/li&gt;&#xA;&lt;li&gt;No negamax (because we also want to support games in which players may have multiple moves in a row before control switches over to another player)&lt;/li&gt;&#xA;&lt;li&gt;No transposition tables &lt;strong&gt;yet&lt;/strong&gt; (we do already compute the Zobrist hashes, just didn't get around to implementing the TT yet).&lt;/li&gt;&#xA;&lt;li&gt;No move ordering (other than in between the iterations of iterative deepening).&lt;/li&gt;&#xA;&lt;li&gt;No smaller search windows than just the regular alpha-beta windows.&lt;/li&gt;&#xA;&lt;li&gt;Also built-in support to handle games with more than 2 players (with Paranoid search), which probably adds a little bit of overhead to the algorithm.&lt;/li&gt;&#xA;&lt;li&gt;No smart tricks like the ones you mentioned about only looking at windows around placed stones; we need to support general games.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2019-12-18T20:10:28.450" LastActivityDate="2019-12-18T20:10:28.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17165" PostTypeId="2" ParentId="17159" CreationDate="2019-12-18T22:13:14.560" Score="3" Body="&lt;p&gt;In the case of artificial neural networks, your question can be (partially) answered by looking at the definition of the operation that an artificial neuron performs. An artificial neuron is usually defined as a linear combination of its inputs, followed by the application of a non-linear activation function (e.g. the &lt;a href=&quot;http://mathworld.wolfram.com/HyperbolicTangent.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyperbolic tangent&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt;). More formally, a neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; performs the following operation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;o_i^l = \sigma \left(\sum_{j=1}^N w_j o_j^{l-1} \right) \tag{1}\label{1},&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$o_j^{l-1}$&lt;/span&gt; is the output from neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt; (the previous layer), &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt; the corresponding weight, &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; an activation function and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; the number of neurons from layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt; connected to neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReLU&lt;/a&gt;, which is defined as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma(x)=\max(0, x)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which means that all negative numbers become &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and all non-negative numbers become themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In equation \ref{1}, if &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$o_j^{l-1}$&lt;/span&gt; have the same sign, then the product &lt;span class=&quot;math-container&quot;&gt;$w_j o_j^{l-1}$&lt;/span&gt; is non-negative (positive or zero), else it is negative (or zero). Therefore, the sign of the output of neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt; alone does not fully determine the effect on &lt;span class=&quot;math-container&quot;&gt;$o_i^l$&lt;/span&gt;, but the sign of the &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt; is also required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose that the product &lt;span class=&quot;math-container&quot;&gt;$w_j o_j^{l-1}$&lt;/span&gt; is negative, then, of course, this will negatively contribute to the sum in equation \ref{1}. In any case, even if the sum &lt;span class=&quot;math-container&quot;&gt;$\sum_{j=1}^N w_j o_j^{l-1}$&lt;/span&gt; is negative, if &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the ReLU, no matter the magnitude of the negative number, &lt;span class=&quot;math-container&quot;&gt;$o_i^l$&lt;/span&gt; will always be zero. However, if the activation function is &lt;a href=&quot;http://mathworld.wolfram.com/HyperbolicTangent.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyperbolic tangent&lt;/a&gt;, the magnitude of a negative &lt;span class=&quot;math-container&quot;&gt;$\sum_{j=1}^N w_j o_j^{l-1}$&lt;/span&gt; affects the magnitude of &lt;span class=&quot;math-container&quot;&gt;$o_i^l$&lt;/span&gt;. More precisely, the more negative the sum is, the closest &lt;span class=&quot;math-container&quot;&gt;$o_i^l$&lt;/span&gt; is to &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, in general, the effect of the sign of an output of an artificial neuron on neighboring neurons depends on the activation function and the learned weights, which depend on the error the neural network is making (assuming the neural network is trained with gradient descent combined with back-propagation), which in turn depends on the training dataset, the loss function, the architecture of the neural network, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Biological neurons and synapses are more complex than artificial ones. Nevertheless, &lt;a href=&quot;https://www.sciencedirect.com/topics/agricultural-and-biological-sciences/synapse&quot; rel=&quot;nofollow noreferrer&quot;&gt;biological synapses are usually classified as either excitatory or inhibitory&lt;/a&gt;, so they can have an excitatory or inhibitory effect on connected neurons.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-18T22:50:04.147" LastActivityDate="2019-12-18T22:50:04.147" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17166" PostTypeId="1" AcceptedAnswerId="17167" CreationDate="2019-12-19T09:11:57.537" Score="1" ViewCount="60" Body="&lt;p&gt;Gday,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so my goal is to train an agent to play MarioKart on the Nintendo DS. My first approach (in theory) was to setup an emulator on my pc and let the agent play for ages. But then a colleague suggested to train the agent first on pre recorded humanly played video data, to achieve some sort of base level. And then for further perfection let the agent play for its own with the emulator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BUT.&#xA;I have no clue how training with video data works. E.g. I wonder how to calculate a loss since there is no reward. Or am I getting the intuition wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate it if someone could explain this technique to me.&lt;/p&gt;&#xA;" OwnerUserId="30431" LastActivityDate="2019-12-19T10:26:51.957" Title="How does reinforcement learning with video data work?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17167" PostTypeId="2" ParentId="17166" CreationDate="2019-12-19T09:51:13.987" Score="2" Body="&lt;p&gt;In reinforcement learning, to learn off policy control, you need data on the states, actions and rewards at each time step. If, in addition to a recorded video, you had a recording of controller inputs, and could add reward data by hand, then you could use a standard reinforcement learning method, e.g. DQN. Simply run the DQN training loop as normal, but skip the parts where it acts in the environment, and only train on batches of recorded experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With &lt;em&gt;only&lt;/em&gt; video data, your options are limited. However, it might still be useful, because a significant part of the challenge is a machine vision task. For a DQN agent, it will need to convert frames from the video (e.g. last 4 frames) into a prediction of the different rewards that it could get depending on which controller buttons are pressed. If you can teach a separate neural network to perform a vision task on relevant video data, it may may help. You could use the learned weights from the first layers of this network as the starting point for your Q values network, and it will likely speed up a DQN figuring out the relationship to its predictions. This sort of task switch following learning is called &lt;a href=&quot;https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning&lt;/a&gt;, and is often used in computer vision tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possibly useful starting task if you have a video, but no controller or reward data, would be to predict the next frame(s) of the video, given say four starting frames (you need more than one so that the neural network can use velocity information). It should be possible to generate the training data using opencv or ffmpeg from your recordings.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-19T10:26:51.957" LastActivityDate="2019-12-19T10:26:51.957" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17168" PostTypeId="1" AcceptedAnswerId="17192" CreationDate="2019-12-19T09:58:09.173" Score="0" ViewCount="495" Body="&lt;p&gt;I found the terms &lt;strong&gt;front-end&lt;/strong&gt; and &lt;strong&gt;back-end&lt;/strong&gt; in the article (or blog post) &lt;a href=&quot;https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Develop a CNN for MNIST Handwritten Digit Classification&lt;/a&gt;. What do they mean here? Are these terms standard in this context?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2021-01-08T19:39:21.867" LastActivityDate="2021-01-08T19:39:21.867" Title="What do the terms &quot;front-end&quot; and &quot;back-end&quot; refer to in this article?" Tags="&lt;convolutional-neural-networks&gt;&lt;terminology&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17169" PostTypeId="1" CreationDate="2019-12-19T11:56:19.307" Score="2" ViewCount="25" Body="&lt;p&gt;TL;DR: I can't figure out why my neural network wont give me a sensible output. I assume it's something to do with how I'm presenting the input data to it but I have no idea how to fix it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using matched pairs of speech samples to generate a model which morphs one persons voice into another. There are some standard pre-processing steps which have been done and can be reversed in order to generate a new speech file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With these I am attempting to generate a very simple neural network that translates the input vector into the output one and then reconstructs a waveform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I understand what I'm trying to do mathematically but that's not helping me make keras/tensorflow actually do it.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inputs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As inputs to my model I have vectors containing Fourier Transform values from the input speech sample matched with their counterpart target vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These vectors contain the FT values from each 25ms fragment of utterance are in the form &lt;span class=&quot;math-container&quot;&gt;$[r_1, i_1, ..., r_n, i_n]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the real part of the number and &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the imaginary one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am constructing these pairs into a dataset reshaping each input vector as I do so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_dataset(filepaths):&#xA;    &quot;&quot;&quot;&#xA;    :param filepaths: array containing the locations of the relevant files&#xA;    :return: a tensorflow dataset constructed from the source data&#xA;    &quot;&quot;&quot;&#xA;    examples = []&#xA;    labels = []&#xA;&#xA;    for item in filepaths:&#xA;        try:&#xA;            source = np.load(Path(item[0]))&#xA;            target = np.load(Path(item[1]))&#xA;&#xA;            # load mapping&#xA;            with open(Path(item[2]), 'r') as f:&#xA;                l = [int(s) for s in list(f.read()) if s.isdigit()]&#xA;                it = iter(l)&#xA;                mapping = zip(it, it)&#xA;&#xA;            for entry in mapping:&#xA;                x, y = entry&#xA;                ex, lab = source[x], target[y]&#xA;                ex_ph, lab_ph = np.empty(1102), np.empty(1102)&#xA;&#xA;                # split the values into their real and imaginary parts and append to the appropriate array&#xA;                for i in range(0, 1102, 2):&#xA;                    idx = int(i / 2)&#xA;&#xA;                    ex_ph[i] = ex[idx].real&#xA;                    ex_ph[i+1] = ex[idx].imag&#xA;                    lab_ph[i] = lab[idx].real&#xA;                    lab_ph[i+1] = lab[idx].imag&#xA;&#xA;                examples.append(ex_ph.reshape(1,1102))&#xA;&#xA;                # I'm not reshaping the labels based on a theory that doing so was messing with my loss function&#xA;                labels.append(lab_ph)&#xA;&#xA;        except FileNotFoundError as e:&#xA;            print(e)&#xA;&#xA;    return tf.data.Dataset.from_tensor_slices((examples, labels))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is then being passed to the neural network:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def train(training_set, validation_set, test_set, filename):&#xA;    model = tf.keras.Sequential([tf.keras.layers.Input(shape=(1102,)),&#xA;                                 tf.keras.layers.Dense(551, activation='relu'),&#xA;                                 tf.keras.layers.Dense(1102)])&#xA;&#xA;    model.compile(loss=&quot;mean_squared_error&quot;, optimizer=&quot;sgd&quot;)&#xA;&#xA;    model.fit(training_set, epochs=1, validation_data=validation_set)&#xA;&#xA;    model.evaluate(test_set)&#xA;    model.save(f'../data/models/{filename}.h5')&#xA;    print(model.summary())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I get out... crackling. Every time, no matter how much data I throw at it. I assume I'm doing something obviously and horribly wrong with the way I'm setting this up.&lt;/p&gt;&#xA;" OwnerUserId="32190" LastActivityDate="2019-12-19T11:56:19.307" Title="Can't figure out what's going wrong with my dataset construction for multivariate regression" Tags="&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;&lt;speech-synthesis&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17172" PostTypeId="1" AcceptedAnswerId="17183" CreationDate="2019-12-19T21:50:46.317" Score="0" ViewCount="281" Body="&lt;p&gt;How can I prove that all the a-cuts of any fuzzy set A defined on &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt;&#xA;are convex if and only if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mu_A(\lambda r + (1-\lambda)s) \geq min \{\mu_A(r), \mu_A(s)\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that &lt;span class=&quot;math-container&quot;&gt;$r, s \in R^n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\lambda \in [0, 1]$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's a fuzzy question on my assignment. Any idea on how to start with?&lt;/p&gt;&#xA;" OwnerUserId="32076" LastEditorUserId="2444" LastEditDate="2019-12-19T22:35:49.150" LastActivityDate="2019-12-20T12:12:35.933" Title="How can I prove that all the a-cuts of any fuzzy set A defined on $R^n$ are convex?" Tags="&lt;fuzzy-logic&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17173" PostTypeId="1" CreationDate="2019-12-19T23:59:01.223" Score="2" ViewCount="28" Body="&lt;p&gt;In the paper 'Visual SLAM algorithms: a survey from &lt;span class=&quot;math-container&quot;&gt;$2010$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$2016$&lt;/span&gt;' by Takafumi Taketomi, Hideaki Uchiyama and Sei Ikeda it is mentioned &lt;/p&gt;&#xA;&#xA;&lt;p&gt;'It should be noted that tracking and mapping (TAM) is&#xA;used instead of using localization and mapping. TAM was&#xA;first used in Parallel Tracking and Mapping (PTAM) [15]&#xA;because localization and mapping are not simultaneously&#xA;performed in a traditional way. Tracking is performed in&#xA;every frame with one thread whereas mapping is per-&#xA;formed at a certain timing with another thread. After&#xA;PTAM was proposed, most of vSLAM algorithms follows&#xA;the framework of TAM. Therefore, TAM is used in this&#xA;paper.'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not quite follow the difference between Localization and mapping versus Tracking and mapping.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the difference?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are some advantages of TAM?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is SLAM not called STAM?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14662" LastActivityDate="2019-12-19T23:59:01.223" Title="SLAM versus &quot;STAM&quot; in vision" Tags="&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17174" PostTypeId="2" ParentId="17172" CreationDate="2019-12-20T01:20:18.683" Score="2" Body="&lt;p&gt;We can assume without loss of generality that&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\min\{\mu_A(r), \mu_A(s)\} = \mu_A(r) = \alpha.&#xA;\end{equation}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\implies$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a-cut of fuzzy set &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is on &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt; is convex. A-cut can be defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;A = \{x \in R^n| \mu_A(x) \geq \alpha\}&#xA;\end{equation}&lt;/span&gt;&#xA;If we take two elements &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, by the definition of convex set, number &lt;span class=&quot;math-container&quot;&gt;$\lambda r + (1 + \lambda)s$&lt;/span&gt; is also an element of that set. Since it's an element of that set that means&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mu_A(\lambda r + (1 + \lambda)s) \geq \alpha&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\impliedby$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mu_A(\lambda r + (1 + \lambda)s) \geq \alpha$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know from &lt;span class=&quot;math-container&quot;&gt;$\min\{\mu_A(r), \mu_A(s)\} = \mu_A(r) = \alpha$&lt;/span&gt; that &lt;span class=&quot;math-container&quot;&gt;$\mu_A(s) &amp;gt; \alpha$&lt;/span&gt;. We have an affine combination &lt;span class=&quot;math-container&quot;&gt;$\lambda r + (1 + \lambda)s$&lt;/span&gt; for which also &lt;span class=&quot;math-container&quot;&gt;$\mu_A(\lambda r + (1 + \lambda)s) \geq \alpha$&lt;/span&gt; so we know that all numbers &lt;span class=&quot;math-container&quot;&gt;$\lambda r + (1 + \lambda)s$&lt;/span&gt; satisfy inequality &lt;span class=&quot;math-container&quot;&gt;$\mu_A(\cdot) \geq \alpha$&lt;/span&gt; (belong to the same set as &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;) which means this is a convex set again by the definition of a convex set.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-12-20T01:20:18.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17175" PostTypeId="1" AcceptedAnswerId="17187" CreationDate="2019-12-20T03:17:44.397" Score="2" ViewCount="133" Body="&lt;p&gt;How would a probabilistic version of minimax work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we may choose a move that could result in a very bad outcome, but that outcome might just be extremely unlikely so we might think it would be worth the risk.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="2444" LastEditDate="2019-12-20T15:42:27.493" LastActivityDate="2019-12-20T15:56:27.007" Title="Is there a probabilistic version of minimax?" Tags="&lt;minimax&gt;&lt;expectiminimax&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17176" PostTypeId="1" CreationDate="2019-12-20T03:43:15.307" Score="1" ViewCount="49" Body="&lt;p&gt;In a hypothetical conversation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Person A - &quot;Repeat the word 'cat' twice&quot;.&#xA;Person B - &quot;cat cat&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm thinking about how a human or AI can learn the concept of &quot;repeat twice&quot;. In reinforcment learning it would require that after the first sentence the AI would go through every random sentence until it got it right and hence got a reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way might be the AI or human overhearing the conversation. Then on hearing a repetition or a word it may trigger some neurons in the brain related to detecting repetition. Thus by pavlovian learning associate the word &quot;repeat&quot; or &quot;twice&quot; with these neurons. When given the stimulus of the word &quot;repeat&quot; these neurons may get triggered making the brain do some repetition algorithm. (This is my favorite theory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose a third way might be as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Person A - &quot;Hello! Hello!&quot;&#xA;Person B - &quot;Stop repeating yourself&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It might learn to associate repeating with the word &quot;repeating&quot; in this way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think either way the brain must have some neurons dedicated to detecting repetitions and possibly inacting them. (I don't think any standard RNN has this capability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think is the most likely way?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2019-12-20T03:43:15.307" Title="How would an AI learn the concept of the words &quot;repeat twice&quot;?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;learning-algorithms&gt;&lt;human-computer-interaction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17180" PostTypeId="1" CreationDate="2019-12-20T08:43:46.033" Score="1" ViewCount="24" Body="&lt;p&gt;Why are the error rates in table 3 and table 4 are different in the paper &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Residual Learning for Image Recognition (2015)&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are both error rates on the validation sets by single model. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why there are different rates for the same architecture? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32203" LastEditorUserId="1671" LastEditDate="2019-12-20T22:43:30.543" LastActivityDate="2019-12-20T22:43:30.543" Title="Why the error rates in table3 and table4 are differenct in the paper &quot;deep residual learning for image recognition&quot;" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17181" PostTypeId="2" ParentId="17142" CreationDate="2019-12-20T09:22:39.867" Score="1" Body="&lt;p&gt;I think that the best approach is to &quot;switch point of view&quot; from the general, objective-oriented, Genetic Algorithm's behaviours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually GAs rely on &lt;strong&gt;individualism&lt;/strong&gt;: the best survives. To do this you have to define what 'best' means and this is done through a fitness function that measures something objective, independent from the individual (i.e. a score, classical example).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to measure novelty, &lt;strong&gt;collectivism&lt;/strong&gt; becomes more important, as the whole population (present and past) must be considered. You have to think about measuring how different the individual X did from 'the rest' (i.e. location never reached before).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting point is that you should think in a more &lt;strong&gt;multidimensional way&lt;/strong&gt;: a score is uni-dimensional, everyone wants to get higher, that is easy. A location in space is more difficult to achieve and that is why novelty becomes interesting.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To sum up: stay multidimensional, always treat the population as a multi-organism rather than focusing on the single individual. I hope this helps :)&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2019-12-20T09:22:39.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17182" PostTypeId="2" ParentId="16505" CreationDate="2019-12-20T09:33:54.933" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Not really. Unity physics is just an approximation of an approximation. It has to look more or less real but at the same time the performances are very important, so it has not the realistic level you would hope to &quot;bring things to the real world&quot;. There are some physics engine you can install that usually work a bit better. Still, don't expect &quot;real-world level&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on 1, yes. If, once you got it to the real world you still give it the chance to optimize, it can only make things better.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This is difficult to say. You should try to use the best possible approximation in Unity, maybe testing multiple scenarios and seeing that they perform more or less in the same way. Then train. Then bring this &quot;approximated model&quot; to the real world and let it train more to adjust to the real physics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't know this. Unity seems the most versatile around (together with other game engines). This versatility is its strength and weakness, as it cannot focus on solving perfectly a single problem. It rather has to aim at generalizing as much as possible. There might be &quot;more specialized&quot; programs around, but I doubt.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Just try different algorithms/methods. Usually with complex problems neuroevolutionary techniques perform better, but the amount of code is hugely higher. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2019-12-20T09:33:54.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17183" PostTypeId="2" ParentId="17172" CreationDate="2019-12-20T12:12:35.933" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;A fuzzy set A in &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt; is said to be a convex fuzzy set if its&#xA;  &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;-cuts &lt;span class=&quot;math-container&quot;&gt;$A_\alpha$&lt;/span&gt; are (crisp) convex sets for all &lt;span class=&quot;math-container&quot;&gt;$A \in (0,1]$&lt;/span&gt;&#xA;  .&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let A be a convex fuzzy set if and only if for all &lt;span class=&quot;math-container&quot;&gt;$r, s \in$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\lambda \in [0, 1]$&lt;/span&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\alpha=\mu_A\leq\mu_B$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    r\in A_{\alpha}, s\in A_{\alpha}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and also&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \lambda r + (1-\lambda)s \geq \alpha = min \{\mu_A(r), \mu_A(s)\}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, if the membership funciton &lt;span class=&quot;math-container&quot;&gt;$\mu_A$&lt;/span&gt; of the fuzzy set A satisfies the inequality of Theorem 13.1 &lt;strong&gt;Convex fuzzy set&lt;/strong&gt;, then taking &lt;span class=&quot;math-container&quot;&gt;$\alpha=\mu_A(r), A_\alpha$&lt;/span&gt; may be regarded as set of all points &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; for which &lt;span class=&quot;math-container&quot;&gt;$\mu_A(s)\geq\alpha=\mu_A(r)$&lt;/span&gt;. Therefore for all &lt;span class=&quot;math-container&quot;&gt;$r,s \in A_\alpha$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \mu_A(\lambda r + (1-\lambda)s) \geq min \{\mu_A(r), \mu_A(s)\} = \mu_A(r)=\alpha&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which inplies that &lt;span class=&quot;math-container&quot;&gt;$\lambda r + (1-\lambda)s \in A_\alpha$&lt;/span&gt;. Hence &lt;span class=&quot;math-container&quot;&gt;$A_\alpha$&lt;/span&gt; is a convex set for every &lt;span class=&quot;math-container&quot;&gt;$\alpha \in [0,1]$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2019-12-20T12:12:35.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17184" PostTypeId="2" ParentId="12099" CreationDate="2019-12-20T14:07:40.763" Score="0" Body="&lt;p&gt;If we can do some reduction in the search space using CSP (constraint propagation) we can drastically reduce the search space or sometimes completely avoid the need for a search by directly reaching the solutions (for e.g. with variables having their domains reduced to size one). It could also happen that we come to a point when a variable domain size becomes zero, in that case no solution exists, given the constraints, so no need for a search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constraint propagation basically involves the concept of enforcing &lt;strong&gt;local consistency&lt;/strong&gt; (this is done by enforcing node-consistency, arc-consistency, path-consistency and also global constraints using Alldiif or Atmost). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms: &lt;strong&gt;nodes&lt;/strong&gt;, &lt;strong&gt;arc&lt;/strong&gt;, &lt;strong&gt;path&lt;/strong&gt;, etc. basically reflects a CSP problem represented as a graph with nodes as the variables and the arcs/edges as constraints. The process is simply to remove values from the domains of the variables that are inconsistent. Algorithms such as &lt;strong&gt;AC-3&lt;/strong&gt;, &lt;strong&gt;PC-2&lt;/strong&gt;, etc. precisely are for these purposes.&lt;/p&gt;&#xA;" OwnerUserId="31765" LastActivityDate="2019-12-20T14:07:40.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17185" PostTypeId="1" CreationDate="2019-12-20T14:16:14.030" Score="1" ViewCount="89" Body="&lt;p&gt;I have been looking into the backtracking search for CSPs, and understand that if we just plainly do a typical depth-limited search we have a vast tree with leaves size &lt;strong&gt;n!d^n&lt;/strong&gt; where &lt;strong&gt;n&lt;/strong&gt; is the # of &lt;strong&gt;variables&lt;/strong&gt; and &lt;strong&gt;d&lt;/strong&gt; the &lt;strong&gt;domain size&lt;/strong&gt;. It can also be easily understood that there exists instead only &lt;strong&gt;d^n&lt;/strong&gt; complete assignments. So the reason for the the tree being so large is attributed to the fact that we are ignoring the &lt;strong&gt;commutative&lt;/strong&gt; way of variable assignments in CSP. Can anyone please explain, as to how exactly this commutative property affects? &lt;/p&gt;&#xA;" OwnerUserId="31765" LastActivityDate="2019-12-20T14:16:14.030" Title="What is exactly the role of commutative property in a Constraint Satisfaction Problem?" Tags="&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17187" PostTypeId="2" ParentId="17175" CreationDate="2019-12-20T15:41:12.423" Score="1" Body="&lt;p&gt;Yes, there is at least one probabilistic version of minimax, which is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectiminimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectiminimax&lt;/a&gt;. In &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectiminimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectiminimax&lt;/a&gt;, in addition to &lt;em&gt;min&lt;/em&gt; and &lt;em&gt;max&lt;/em&gt; nodes, there are also &lt;em&gt;chance&lt;/em&gt; nodes, which perform a weighted sum of the successors, so the probabilities associated with chance nodes must be known. Given that &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectiminimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectiminimax&lt;/a&gt; assumes the existence of random events (represented by the chance nodes), the decisions are thus based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;expected values&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Section 5.5 of the book &lt;a href=&quot;https://www.cin.ufpe.br/~tfl2/artificial-intelligence-modern-approach.9780131038059.25368.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; provides a description of the expectiminimax algorithm, which was introduced by &lt;a href=&quot;http://www.aiai.ed.ac.uk/~dm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Donald Michie&lt;/a&gt; in &lt;a href=&quot;https://www.researchgate.net/publication/240013423_Game-playing_and_game-learning_automata&quot; rel=&quot;nofollow noreferrer&quot;&gt;Game-playing and game-learning automata&lt;/a&gt; (1966). The paper &lt;a href=&quot;http://www.inf.u-szeged.hu/actacybernetica/edb/vol18n2/pdf/Melko_2007_ActaCybernetica.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal strategy in games with chance nodes&lt;/a&gt; (2007) also gives a decent description of the expectiminimax algorithm.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-20T15:56:27.007" LastActivityDate="2019-12-20T15:56:27.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17188" PostTypeId="1" AcceptedAnswerId="17189" CreationDate="2019-12-20T17:00:20.923" Score="3" ViewCount="1244" Body="&lt;p&gt;I understand that the actual algorithm calls for using Depth-First Search, but is there a functionality reason for using it over another search algorithm like Breadth-First Search?&lt;/p&gt;&#xA;" OwnerUserId="32215" LastEditorUserId="2444" LastEditDate="2021-05-18T00:09:32.980" LastActivityDate="2021-05-18T00:09:32.980" Title="Why does the adversarial search minimax algorithm use Depth-First Search (DFS) instead of Breadth-First Search (BFS)?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;breadth-first-search&gt;&lt;depth-first-search&gt;&lt;adversarial-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17189" PostTypeId="2" ParentId="17188" CreationDate="2019-12-20T17:58:38.493" Score="3" Body="&lt;p&gt;The primary reason is that Breadth-First Search requires much more memory (and this probably also makes it a little bit slower in practice, due to time required to allocate memory, jumping around in memory rather than working with what's still in the CPU's caches, etc.). Breadth-First Search needs memory to remember &quot;where it was&quot; in all the different branches, whereas Depth-First Search completes an entire path first before recursing back -- which doesn't really require any memory other than the stack trace. This is assuming we're using a recursive implementation for DFS -- which we normally do in the case of minimax.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can clearly see this if you look at pseudocode for the two approaches (ignoring the minimax details here, just presenting pseudocode for straightforward searches):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BreadthFirstSearch(start):&#xA;    Q = new queue()&#xA;    Q.append(start)&#xA;    while Q is not empty:&#xA;        node = Q.pop()&#xA;        if node is leaf:&#xA;            do something with leaf&#xA;        else:&#xA;            for each child of node:&#xA;                Q.append(child)&#xA;&#xA;DepthFirstSearch(start):&#xA;    if start is leaf:&#xA;        do something with leaf&#xA;    for each child of start:&#xA;        DepthFirstSearch(child)&#xA;        // probably do something with return value from the recursive DFS call&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You see that the BFS requires a queue object that explicitly stores a bunch of stuff in memory, whereas DFS doesn't.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There's more to the story once you get to extensions of Minimax, like Alpha-Beta pruning and Iterative Deepening... but since the question is just about Minimax, I'll leave it at that for now.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2019-12-20T17:58:38.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17190" PostTypeId="1" AcceptedAnswerId="17191" CreationDate="2019-12-20T18:26:32.660" Score="5" ViewCount="71" Body="&lt;p&gt;Are artificial neurons in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; only affected by those in layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt; (providing inputs) or are they also affected by neurons in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; (and maybe by neurons in other layers)?&lt;/p&gt;&#xA;" OwnerUserId="30178" LastEditorUserId="2444" LastEditDate="2019-12-20T20:42:52.880" LastActivityDate="2019-12-20T21:04:34.073" Title="Are neurons in layer $l$ only affected by neurons in the previous layer?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17191" PostTypeId="2" ParentId="17190" CreationDate="2019-12-20T20:54:25.573" Score="5" Body="&lt;p&gt;It depends on the architecture of the neural network. However, in general, no, neurons at layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; are not only affected by neurons at layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a multi-layer perceptron (or feed-forward neural network), only neurons at layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt; &lt;strong&gt;directly&lt;/strong&gt; affect the neurons at layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;. However, neurons at layers &lt;span class=&quot;math-container&quot;&gt;$l-i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=2, \dots, l$&lt;/span&gt;, also &lt;em&gt;indirectly&lt;/em&gt; affect the neurons at layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of recurrent neural networks, the output of neuron &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; at level &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; can also affect the same neuron but at a different time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of &lt;a href=&quot;https://arxiv.org/abs/1512.03385&quot; rel=&quot;noreferrer&quot;&gt;residual networks&lt;/a&gt;, the output of a neuron at a layer &lt;span class=&quot;math-container&quot;&gt;$l-i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=2, \dots, l$&lt;/span&gt;, can &lt;strong&gt;directly&lt;/strong&gt; affect the neurons at layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;. These non-neighboring connections are called &lt;strong&gt;skip connections&lt;/strong&gt; because they skip layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are probably other combinations of connections between neurons at different layers or the same layer. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-20T21:04:34.073" LastActivityDate="2019-12-20T21:04:34.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17192" PostTypeId="2" ParentId="17168" CreationDate="2019-12-20T22:55:36.550" Score="1" Body="&lt;p&gt;I do not think these are formally defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distinction is just to facilitate discussion of the NN architecture: e.g., you may have a few convolutional layers with pooling as a front-end, and a different architecture as a back-end (in a text-book architecture, just a fully-connected layer. But to get wild, maybe LSTM? To really get wild, BERT?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end (no pun intended), computers do not care if a layer is seen by humans as a front-end or a back-end.&lt;/p&gt;&#xA;" OwnerUserId="32218" LastActivityDate="2019-12-20T22:55:36.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17193" PostTypeId="1" CreationDate="2019-12-20T23:36:09.403" Score="1" ViewCount="41" Body="&lt;p&gt;This question is about Real-Time Recurrent Learning Gradient on a Recurrent neural network .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I write out the RTRL equations for a network ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before present an example give let's introduce some notation :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notation&lt;/strong&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/Yu7Vb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yu7Vb.png&quot; alt=&quot;Notation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the network for which we want to write the RTRL equations is the following :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Network&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/A7n2t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A7n2t.png&quot; alt=&quot;Network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar question can be found &lt;a href=&quot;https://hagan.okstate.edu/NNDesign.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; at page 561 for another network .&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2019-12-20T23:36:09.403" Title="How can I write out the Real-TIme Recurrent Learning Gradient equations for a network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;learning-algorithms&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17194" PostTypeId="1" CreationDate="2019-12-21T01:32:38.963" Score="4" ViewCount="57" Body="&lt;p&gt;What is the best way to train / do inference when the context matters highly as to what the inferred result should be?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in the image below all people are standing upright, but because of the perspective of the camera, their location highly affects their skeletal pose.  If the 2D inferred skeleton of the person on the right were located where the middle person is in pixel space, it should not be considered upright even though it should be considered upright where it is now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the location would be fed in during both training and inference somehow, but I don't know the names of the techniques that should be used and are there any best practices when doing this type of scenario?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.ytimg.com/vi/nMonyLmErwo/maxresdefault.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.ytimg.com/vi/nMonyLmErwo/maxresdefault.jpg&quot; alt=&quot;Skeletal pose of people from an overhead camera at approximately 45 degrees&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32126" LastActivityDate="2019-12-23T18:25:40.057" Title="Training and inference for highly-context-sensitive information" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;features&gt;&lt;inference&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17195" PostTypeId="1" AcceptedAnswerId="17740" CreationDate="2019-12-21T06:25:47.847" Score="3" ViewCount="89" Body="&lt;p&gt;I keep reading about how LSTMs can't remember the &quot;important parts&quot; of a sequence which is why attention-based mechanisms are required. I was trying to use LSTMs to find people's name format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &quot;Millie Bobby Brown&quot; can be seen as first_name middle_name last_name format, which I'll denote as 0, but then there's &quot;Brown, Millie Bobby&quot; which is last_name, first_name middle_name, which I'll denote as 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LSTM seems to be overfitting to one classification of format. I suspect it's because it's not paying special attention to the comma which is a key feature of what format it could be. I'm trying to understand why an LSTM won't work for a case like this. It makes sense to me because LSTMs are better at identifying sequence to sequence generation and things such as summarization and sentiment analysis usually require attention. I suspect another reason why the LSTM is not able to infer the format is that the comma can be placed in different indexes of the sequence, so it could be losing its importance in the hidden state the longer the sequence is (not sure if that makes sense). Anyone else has any theories? I'm trying to convince my fellow researchers that a pure LSTM won't be sufficient for this problem.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2019-12-25T03:26:18.970" LastActivityDate="2020-01-29T12:52:27.837" Title="Why can't LSTMs keep track of the &quot;important parts&quot; of a sequence?" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;&lt;attention&gt;" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17196" PostTypeId="1" AcceptedAnswerId="17258" CreationDate="2019-12-21T07:04:36.867" Score="4" ViewCount="189" Body="&lt;p&gt;I read an interesting &lt;a href=&quot;https://aeon.co/essays/how-close-are-we-to-creating-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;essay&lt;/a&gt; about how far we are from AGI. There were quite a few solid points that made me re-visit the foundation of AI today. A few interesting concepts arose:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;imagine that you require a program with a more ambitious&#xA;  functionality: to address some outstanding problem in theoretical&#xA;  physics — say the nature of Dark Matter — with a new explanation that&#xA;  is plausible and rigorous enough to meet the criteria for publication&#xA;  in an academic journal.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Such a program would presumably be an AGI (and then some). But how&#xA;  would you specify its task to computer programmers? Never mind that&#xA;  it’s more complicated than temperature conversion: there’s a much more&#xA;  fundamental difficulty. Suppose you were somehow to give them a list,&#xA;  as with the temperature-conversion program, of explanations of Dark&#xA;  Matter that would be acceptable outputs of the program. If the program&#xA;  did output one of those explanations later, that would not constitute&#xA;  meeting your requirement to generate new explanations. For none of&#xA;  those explanations would be new: you would already have created them&#xA;  yourself in order to write the specification. So, in this case, and&#xA;  actually in all other cases of programming genuine AGI, only an&#xA;  algorithm with the right functionality would suffice. But writing that&#xA;  algorithm (without first making new discoveries in physics and hiding&#xA;  them in the program) is exactly what you wanted the programmers to do!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The concept of creativity seems like the initial thing to address when approaching a true AGI. The same type of creativity that humans have to ask the initial question or generate new radical ideas to long lasting questions like dark matter. Is there current research being done on this? I've seen work with generating art and music, but it seems like a different approach. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the classic ‘brain in a vat’ thought experiment, the brain, when&#xA;  temporarily disconnected from its input and output channels, is&#xA;  thinking, feeling, creating explanations — it has all the cognitive&#xA;  attributes of an AGI. So the relevant attributes of an AGI program do&#xA;  not consist only of the relationships between its inputs and outputs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is an interesting concept behind why reinforcement learning is not the answer. Without input from the environment the agent has nothing to improve upon however with the actual brain if you had no input or output it is still in a state of &quot;thinking&quot;.&lt;/p&gt;&#xA;" OwnerUserId="31978" LastEditorUserId="1671" LastEditDate="2019-12-26T21:12:04.590" LastActivityDate="2019-12-26T22:09:43.677" Title="Are current AI models sufficient to achieve Artificial General Intelligence?" Tags="&lt;theory&gt;&lt;agi&gt;&lt;creativity&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17197" PostTypeId="2" ParentId="17155" CreationDate="2019-12-21T12:05:51.073" Score="1" Body="&lt;p&gt;If I understood correctly, your problem is about finding the optimal way to execute a series of tasks in order to maximize the results, using Genetic Algorithms.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In few words, you're trying to solve the salesman problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If I am correct, you're looking for Crossover and Mutation algorithms that allow you to work with ordered sets of elements. For these scenarios you usually go for the classic PMX &#xA;(Partially Mapped Crossover) and Interchange Mutation. But, there are plenty of other crossover algorithms you can use OX1, OX2 (both variants of the Order Based Crossover), Shuffle Crossover, Ring Crossover, etc. Let's start from the mutation, that is easier.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity I'll represent the ordered genome like an array of integers: &lt;code&gt;int[] genome = {1, 2, 3, 4, 5};&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Interchange mutation&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept is pretty basic: to mutate an ordered genome you just swap two elements. Easy.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/35qDd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/35qDd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    public int[] InterchangeMutation(int[] genome)&#xA;    {&#xA;        int i1 = random.Next(0, genome.Length);&#xA;        int i2 = random.Next(0, genome.Length);&#xA;&#xA;        var copy = genome.ToArray(); //just making a copy here&#xA;        copy[i1] = genome[i2];&#xA;        copy[i2] = genome[i1];&#xA;&#xA;        return copy;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PMX Variation Crossover&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a bit more complicated as we have to take repetitions into account. From experience, I like to use this variation of the Partially Mapped Crossover. It is way easier to implement than the original one (you can find the paper online) but it will cost some more computational complexity. Longer the genome, higher the price you will pay.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start by selecting two parents to use for the crossover.  &lt;/li&gt;&#xA;&lt;li&gt;From the first parent (P1) select a random section that will be passed over.&lt;/li&gt;&#xA;&lt;li&gt;For the remaining values:&lt;br&gt;&#xA;3A. If they are not in the copied section, take them from P2&lt;br&gt;&#xA;3B. If they are in the copied section, take them from P1&lt;br&gt;&#xA;3C. End up filling the gaps with the missing values in the order they are in P1  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9AUeX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9AUeX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; public int[] PMX2Crossover(int[] P1, int[] P2)&#xA;    {&#xA;        //Initializing child genome&#xA;        int[] child = new int[P1.Length];&#xA;        for (int i = 0; i &amp;lt; P2.Length; i++) child[i] = -1;&#xA;&#xA;        //Step1: getting random section to copy over&#xA;        int i1 = random.Next(0, P1.Length);&#xA;        int i2 = random.Next(0, P1.Length);&#xA;&#xA;        //Step 2: Copying over section from P1&#xA;        for (int i = Math.Min(i1, i2); i &amp;lt; Math.Max(i1, i2); i++) child[i] = P1[i];&#xA;&#xA;        //Step 3A: Copying values from P2&#xA;        for (int i = 0; i &amp;lt; P2.Length; i++) if (child[i] ==-1 &amp;amp;&amp;amp; !child.Contains(P2[i])) child[i] = P2[i];&#xA;&#xA;        //Step 3B: Copying values from P1&#xA;        for (int i = 0; i &amp;lt; P2.Length; i++) if (child[i] == -1 &amp;amp;&amp;amp; !child.Contains(P1[i])) child[i] = P1[i];&#xA;&#xA;        //Step 3C: Copying remaining values from P1&#xA;        int emptyGene = child.IndexOfFirst(-1);&#xA;        while (emptyGene != -1)&#xA;        {&#xA;            child[emptyGene] = FirstMissingGene(P1, child); &#xA;            emptyGene = child.IndexOfFirst(-1);&#xA;        }&#xA;&#xA;        return child;&#xA;    }&#xA;&#xA;    private int FirstMissingGene(int[] parent, int[] child)&#xA;    {&#xA;        foreach (var gene in parent) if (!child.Contains(gene)) return gene;&#xA;        return -1; // should never get here&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can lower down the complexity of the crossover to O(n) (from O(n*n)) simply using a hashmap that keeps track of the genes already added to child.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the first child call &lt;code&gt;PMX2Crossover(P1, P2);&lt;/code&gt; and for the second just swap the parents &lt;code&gt;PMX2Crossover(P2, P1);&lt;/code&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps you.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: I have been a bachelor professor of AI for a period.&lt;/p&gt;&#xA;" OwnerUserId="15530" LastEditorUserId="15530" LastEditDate="2019-12-21T12:11:11.773" LastActivityDate="2019-12-21T12:11:11.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17200" PostTypeId="1" CreationDate="2019-12-21T14:47:02.543" Score="2" ViewCount="120" Body="&lt;p&gt;I am making a firetruck using Arduino Uno with flame sensors and ultrasonic sensors to detect how to move and where to go. As this is a project for my university, I am asked to implement AI in it for path planning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure whether to use something like A* technique or ID3 decision tree or if there is something better than both to implement path planning for my robot. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="32233" OwnerDisplayName="mushter" LastEditorUserId="2444" LastEditDate="2019-12-22T02:09:55.727" LastActivityDate="2019-12-26T11:24:48.500" Title="What is the most suitable AI technique to use for path planning?" Tags="&lt;robotics&gt;&lt;decision-tree&gt;&lt;path-planning&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17201" PostTypeId="1" CreationDate="2019-12-21T18:56:04.157" Score="1" ViewCount="56" Body="&lt;p&gt;I'm trying to re-implement Elastic Weight Consolidation (EWC) as outlined in this &lt;a href=&quot;https://arxiv.org/abs/1612.00796&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. As a reference, I am also using &lt;a href=&quot;https://github.com/moskomule/ewc.pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Github repository&lt;/a&gt; (another implementation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My model/idea is pretty straightforward. Train the network to do the bit operation AND (e.g 1 &amp;amp;&amp;amp; 0 = 0), then using EWC, train it to use OR (e.g 1 || 0 = 1). I've got three inputs: bit1, bit2 and operation (0 stands for AND and 1 for OR) and one output neuron - the output of the operation. For example, if I have 0 1 0 the ground truth should be 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem, however, comes when calculating the EWC loss.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def penalty(self, model: nn.Module):&#xA;    loss = 0&#xA;    for n, p in model.named_parameters():&#xA;        _loss = self._precision_matrices[n] * (p - self._means[n]) ** 2&#xA;        loss += _loss.sum()&#xA;    return loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've got two problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The current means (&lt;code&gt;p&lt;/code&gt;) and the old ones (&lt;code&gt;self._means[n]&lt;/code&gt;) are always the same, resulting in multiplication by 0, which completely negates EWC.&lt;/li&gt;&#xA;&lt;li&gt;As I have just one output neuron the calculation of the fisher's matrix is a bit different than the repo. The one I have written seems to be wrong. Any ideas?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I initialise the &lt;code&gt;self._means[n]&lt;/code&gt; and &lt;code&gt;self._precision_matrices&lt;/code&gt; (fisher's matrix) in the init method of the EWC model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class EWC(object):&#xA;def __init__(self, model: nn.Module, dataset: list, device='cpu'):&#xA;&#xA;    self.model = model&#xA;    self.dataset = dataset&#xA;    self.device = device&#xA;&#xA;    self._means = {}&#xA;    self._precision_matrices = self._diag_fisher()&#xA;&#xA;    for n, p in self.model.named_parameters():&#xA;        self._means[n] = p.data.clone()&#xA;&#xA;def _diag_fisher(self):&#xA;    precision_matrices = {}&#xA;&#xA;    # Set it to zero&#xA;    for n, p in self.model.named_parameters():&#xA;        params = p.clone().data.zero_()&#xA;        precision_matrices[n] = params&#xA;&#xA;    self.model.eval()&#xA;&#xA;    for input in self.dataset:&#xA;        input = input.to(self.device)&#xA;&#xA;        self.model.zero_grad()&#xA;&#xA;        output = self.model(input)&#xA;        label = torch.sigmoid(output).round()&#xA;        loss = F.binary_cross_entropy_with_logits(output, label)&#xA;        # loss = F.nll_loss(F.log_softmax(output, dim=1), label)&#xA;        loss.backward()&#xA;&#xA;        for n, p in self.model.named_parameters():&#xA;            precision_matrices[n].data += p.grad.data ** 2 / len(self.dataset)&#xA;&#xA;    precision_matrices = {n: p for n, p in precision_matrices.items()}&#xA;    return precision_matrices&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the actual training:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Train the model EWC&#xA;for epoch in tqdm(range(EPOCS)):&#xA;&#xA;    # Get the loss&#xA;    ls = ewc_train(model, opt, loss_func, dataloader[task], EWC(model, old_tasks), importance, device)&#xA;&#xA;def ewc_train(model: nn.Module, opt: torch.optim, loss_func:torch.nn, data_loader: torch.utils.data.DataLoader, ewc: EWC, importance: float, device):&#xA;    epoch_loss = 0&#xA;&#xA;    for i, (inputs, labels) in enumerate(data_loader):&#xA;        inputs = inputs.to(device).long()&#xA;        labels = labels.to(device).float()&#xA;&#xA;        opt.zero_grad()&#xA;&#xA;        output = model(inputs)&#xA;        loss = loss_func(output.view(-1), labels) + importance * ewc.penalty(model)&#xA;        loss.backward()&#xA;        opt.step()&#xA;&#xA;        epoch_loss += loss.item()&#xA;&#xA;    return loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: the loss function that I am using is &lt;code&gt;nn.BCEWithLogitsLoss()&lt;/code&gt; and optimisation is: &lt;code&gt;SGD(params=model.parameters(), lr=0.001)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32235" LastEditorUserId="2444" LastEditDate="2019-12-22T14:20:55.603" LastActivityDate="2019-12-22T14:20:55.603" Title="Why are the current means and the old ones the same in this implementation of Elastic Weight Consolidation?" Tags="&lt;neural-networks&gt;&lt;pytorch&gt;&lt;implementation&gt;&lt;incremental-learning&gt;&lt;elastic-weight-consolidation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17202" PostTypeId="1" AcceptedAnswerId="17205" CreationDate="2019-12-21T21:31:56.353" Score="2" ViewCount="261" Body="&lt;p&gt;I am trying to implement a denoising autoencoder (DAE) to remove noise from 1024-point FFT spectra. I am using two types of spectra: (1) that contain a distinctive high amplitude spectral peak and (2) that contain only noise peaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understood correctly, I can train the DAE using the corruputed spectra (spectra+noise) and afterwards I can use it the remove noise from new datasets. The problem is that when testing the DAE, it returns the type (1) spectrum mentioned above, regardless of the input. The same case when I apply predict on the training data. This is the code I am using (Python/Tensorflow):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def BuildModel(nInput):&#xA;    input_dim = Input(shape = (nInput, ))&#xA;&#xA;    # Encoder Layers&#xA;    encoded1 = Dense(896, activation = 'relu')(input_dim)&#xA;    encoded2 = Dense(768, activation = 'relu')(encoded1)&#xA;    encoded3 = Dense(640, activation = 'relu')(encoded2)&#xA;    encoded4 = Dense(512, activation = 'relu')(encoded3)&#xA;    encoded5 = Dense(384, activation = 'relu')(encoded4)&#xA;    encoded6 = Dense(256, activation = 'relu')(encoded5)&#xA;    encoded7 = Dense(encoding_dim, activation = 'relu')(encoded6)&#xA;&#xA;    # Decoder Layers&#xA;    decoded1 = Dense(256, activation = 'relu')(encoded7)&#xA;    decoded2 = Dense(384, activation = 'relu')(decoded1)&#xA;    decoded3 = Dense(512, activation = 'relu')(decoded2)&#xA;    decoded4 = Dense(640, activation = 'relu')(decoded3)&#xA;    decoded5 = Dense(768, activation = 'relu')(decoded4)&#xA;    decoded6 = Dense(896, activation = 'relu')(decoded5)&#xA;    decoded7 = Dense(nInput, activation = 'sigmoid')(decoded6)&#xA;&#xA;    # Combine Encoder and Deocoder layers&#xA;    autoencoder = Model(inputs = input_dim, outputs = decoded7)&#xA;&#xA;    autoencoder.summary()&#xA;    # Compile the Model&#xA;    autoencoder.compile(optimizer=OPTIMIZER, loss='binary_crossentropy')&#xA;    #autoencoder.compile(loss='mean_squared_error', optimizer = RMSprop())&#xA;&#xA;    return autoencoder&#xA;&#xA;X_train, X_test, y_train, y_test = train_test_split(spectra.iloc[:,0:spectra.shape[1]-1], spectra['Class'], test_size=testDatasetSize, stratify=spectra.Class, random_state=seedValue)&#xA;&#xA;X_train, y_train = shuffle(X_train, y_train, random_state=seedValue)&#xA;X_test, y_test = shuffle(X_test, y_test, random_state=seedValue)&#xA;&#xA;X_unseen = X_train.to_numpy()[0:1000,:] # Data not used for training, only for testing&#xA;y_unseen = y_train.to_numpy()[0:1000]&#xA;X_train = X_train.iloc[1000:]&#xA;y_train = y_train.iloc[1000:]&#xA;&#xA;# Scaling&#xA;maxVal = max(X_train)&#xA;X_train = (X_train/maxVal).to_numpy()&#xA;X_test = (X_test/maxVal).to_numpy()&#xA;X_unseen = (X_unseen/maxVal)#.to_numpy()&#xA;&#xA;# Corrupted data&#xA;noise_factor = 0.01&#xA;X_train_noisy = X_train + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=X_train.shape)&#xA;X_test_noisy = X_test + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=X_test.shape)&#xA;X_unseen_noisy = X_unseen + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=X_unseen.shape)&#xA;&#xA;ae = BuildModel(X_train.shape[1])&#xA;PrintConsoleLine('Creating model finished')&#xA;print('')&#xA;&#xA;history = ae.fit(X_train_noisy, X_train, epochs=NB_EPOCH, batch_size=BATCH_SIZE, validation_data=[X_test_noisy, X_test])&#xA;save_model(ae, modelFile, overwrite=True)&#xA;&#xA;# Test&#xA;X = X_unseen&#xA;X_noisy = X_unseen_noisy&#xA;X_denoised = ae.predict(X_noisy) # X_train gives the same result (spectra type (1)) !?!&#xA;N = len(X_denoised[0,:])&#xA;index = 6&#xA;PlotDataSimple(3, np.linspace(0,N-1,N), X[index,:], 'Frequency domain', 'Index', 'Amplitude', None)&#xA;PlotDataSimple(4, np.linspace(0,N-1,N), X_noisy[index,:], 'Frequency domain', 'Index', 'Amplitude', None)&#xA;PlotDataSimple(5, np.linspace(0,N-1,N), X_denoised[index, :], 'Frequency domain', 'Index', 'Amplitude', None)&#xA;&#xA;Dataset shape: (17000, 65, 65, 1) (files, samples X axis, samples Y axis, class)&#xA;Train on 12600 samples, validate on 3400 samples&#xA;Epoch 1/3&#xA;12600/12600 [==============================] - 26s 2ms/sample - loss: 0.6813 - val_loss: 0.4913&#xA;Epoch 2/3&#xA;12600/12600 [==============================] - 14s 1ms/sample - loss: 0.1621 - val_loss: 0.0578&#xA;Epoch 3/3&#xA;12600/12600 [==============================] - 16s 1ms/sample - loss: 0.0230 - val_loss: 0.0169&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results I am getting (Column 1 - Initial signal, Column 2 - Corrupted signal, Column 3 - Denoised signal):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aXKsr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aXKsr.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why does the DAE output the same spectra regardless of the inputs? Am I misunderstanding the DAE principle or is there a problem in my implementation?&lt;/p&gt;&#xA;" OwnerUserId="32237" LastActivityDate="2021-04-11T18:28:55.120" Title="Why does the denoising autoencoder always returns the same output?" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;autoencoders&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17203" PostTypeId="1" CreationDate="2019-12-22T06:08:37.673" Score="4" ViewCount="328" Body="&lt;p&gt;I'm trying to implement the &lt;a href=&quot;https://arxiv.org/abs/1711.00937&quot; rel=&quot;nofollow noreferrer&quot;&gt;VQ-VAE model&lt;/a&gt;. In there, a continuous variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is encoded in an array &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; of discrete latent variables &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; that are mapped each to an embedding vector &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt;. These vectors can be used to generate an &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; that approximates &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to obtain a reasonable generative model &lt;span class=&quot;math-container&quot;&gt;$p_\theta(x)=\int p_\theta(x|z)p(z)$&lt;/span&gt;, one needs to learn the prior distribution of the code &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;. However, it is not clear in this paper, or its &lt;a href=&quot;https://arxiv.org/abs/1906.00446&quot; rel=&quot;nofollow noreferrer&quot;&gt;second version&lt;/a&gt;, what should be the input of the network that learns the prior. Is it &lt;span class=&quot;math-container&quot;&gt;$z=[z_i]$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$e=[e_i]$&lt;/span&gt;? The paper seems to indicate that it is &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, but if that's the case, I don't understand how I should encode &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; properly. For example, a sample of &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; might be an &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; matrix with discrete values between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$511$&lt;/span&gt;. It is not reasonable to me to use a one-hot encoding, nor to simply use the discrete numbers as if they were continuous, given that there is no defined order for them. On the other hand, using &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; doesn't have this problem since it represents a matrix with continuous entries, but then the required network would be much bigger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what should be the input for the prior model? &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;? If it is &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, how should I represent it? If it is &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, how should I implement the network?&lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2021-04-26T17:09:20.827" Title="What is the input for the prior model of VQ-VAE?" Tags="&lt;machine-learning&gt;&lt;generative-model&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17204" PostTypeId="2" ParentId="16023" CreationDate="2019-12-22T08:53:13.850" Score="0" Body="&lt;p&gt;I imagine that using the MaxNFFC as a stop criterion only happens in very particular implementations. And this is its main disadvantage.&lt;/p&gt;&#xA;&lt;p&gt;Normally, you'd evaluate each individual, each generation. So, NFFC will always be the same as the size of the population times the number of epochs (+1 to consider the initial population).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$N*(E+1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As the population size is generally constant, MaxNFFC looks a lot like an MaxEpoch stop.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\text{NFFC}}{N} = E + 1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, it seems that it might be used in scenarios where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the population size is not constant (in this case you want to make sure that at least K individuals have been evaluated before stopping).&lt;/li&gt;&#xA;&lt;li&gt;not all individuals are evaluated each epoch. There might be a high abortive rate (individuals with invalid genomes that are kept in the population but not considered). In this case, as above, you want to perform a minimum amount of evaluations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I wouldn't see the usage of MaxNFFC in other cases.&lt;/p&gt;&#xA;" OwnerUserId="15530" LastEditorUserId="2444" LastEditDate="2021-01-30T02:51:32.430" LastActivityDate="2021-01-30T02:51:32.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17205" PostTypeId="2" ParentId="17202" CreationDate="2019-12-22T11:00:25.207" Score="0" Body="&lt;p&gt;You are using Dense layers, try 1d convolution instead.&#xA;Have you tried a different activation function such as softmax and instead of Binary cross entropy try MSE loss? Are all your inputs between 0 and 1? Also, I think your noise amplitude is too much in 2nd and 3rd case as compared to the actual signal. Can you try training on different types of spectra separately and check the result, if the DAE is learning anything. Also, try trining for more epochs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure and would have commented this but I can't due to low rep, but I am interested in the solution.&lt;/p&gt;&#xA;" OwnerUserId="27875" LastActivityDate="2019-12-22T11:00:25.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17206" PostTypeId="2" ParentId="4748" CreationDate="2019-12-22T13:19:49.803" Score="0" Body="&lt;p&gt;One more idea - I recall learning about Neyman-Pearson task in my studies. It is a statistical learning method for binary classification problem where overlooked danger (false negative error) is much unwanted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You set a desired threshold for false negative error rate and then minimize false positive error. You just need to measure conditional probabilities of each class. It may be expressed as a linear program and solved to get the optimal strategy for the threshold of your choice.&lt;/p&gt;&#xA;" OwnerUserId="32245" LastActivityDate="2019-12-22T13:19:49.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17208" PostTypeId="2" ParentId="17196" CreationDate="2019-12-22T14:08:30.270" Score="1" Body="&lt;p&gt;I am focusing on what you have posted here without going through (or having read) the whole essay you linked:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the classic ‘brain in a vat’ thought experiment, the brain, when temporarily disconnected from its input and output channels, is thinking, feeling, creating explanations — it has all the cognitive attributes of an AGI. So the relevant attributes of an AGI program do not consist only of the relationships between its inputs and outputs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The classic brain in a vat experiment does &lt;strong&gt;not&lt;/strong&gt; disconnect the brain from all inputs and outputs but replaces &amp;quot;real&amp;quot; connections with its environment with &amp;quot;fake&amp;quot; connections, e.g. by connection the brain to a computer. &lt;a href=&quot;https://en.wikipedia.org/wiki/Brain_in_a_vat&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is what Wikipedia says&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In philosophy, the brain in a vat (BIV; alternately known as brain in a jar) is a scenario used in a variety of thought experiments intended to draw out certain features of human conceptions of knowledge, reality, truth, mind, consciousness, and meaning. It is an updated version of René Descartes's evil demon thought experiment originated by Gilbert Harman.&lt;a href=&quot;https://en.wikipedia.org/wiki/Brain_in_a_vat&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; Common to many science fiction stories, it outlines a scenario in which a mad scientist, machine, or other entity might remove a person's brain from the body, suspend it in a vat of life-sustaining liquid, and connect its neurons by wires to a supercomputer which would provide it with electrical impulses identical to those the brain normally receives.&lt;a href=&quot;https://i.stack.imgur.com/IsQs2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; According to such stories, &lt;strong&gt;the computer would then be simulating reality&lt;/strong&gt; (including appropriate responses to the brain's own output) and the &amp;quot;disembodied&amp;quot; brain would continue to have perfectly normal conscious experiences, such as those of a person with an embodied brain, without these being related to objects or events in the real world.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Following that, the argument of the author referring to the brain in a vat scenario does not hold.&lt;/p&gt;&#xA;&lt;p&gt;If you ignore that problem for a moment the next inconsistency arises: The author assumed that because a brain would act &amp;quot;so and so&amp;quot; an AGI would need to act &amp;quot;so and so&amp;quot; as well. That however is not in line with how artificial intelligence is usually defined. Even if you consider the broad range of definitions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IsQs2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IsQs2.png&quot; alt=&quot;Examples of defining AI&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;(source: &amp;quot;Artificial Intelligence: A Modern Approach&amp;quot;; Russel, Norvig, 3rd Ed, 2010)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;None of these definitions directly references AI to the brain. Therefore, the assumption that an AGI (as a subset of AI) would need to act like a brain in a vat is, without further assumptions,  generally flawed.&lt;/p&gt;&#xA;&lt;p&gt;Another problem with the author's argumentation lies here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;to address some outstanding problem in theoretical physics — say the nature of Dark Matter — with a new explanation that is plausible and rigorous enough to meet the criteria for publication in an academic journal.&lt;/p&gt;&#xA;&lt;p&gt;Such a program would presumably be an AGI (and then some).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The assumption that &amp;quot;creativity&amp;quot; (in quotation marks since we actually need to precisely define that in the first place) requires an AGI does not hold either. Let's stick to the authors example related to dark matter:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This is per definition a specialized field of application of an AI. Accordingly, there is not necessarily a need for an AGI since &amp;quot;specialization&amp;quot; is exactly what an AGI does &lt;strong&gt;not&lt;/strong&gt; have.&lt;/li&gt;&#xA;&lt;li&gt;Moreover, you could make an argument that coming up with new explanations related to dark matter is somewhat related to automated theorem proving. Accordingly, we might already have AI which is, in principle, capable of &amp;quot;solving&amp;quot; this task as of today.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="30789" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2019-12-22T14:13:32.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17210" PostTypeId="5" CreationDate="2019-12-22T14:25:54.713" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-22T14:25:54.713" LastActivityDate="2019-12-22T14:25:54.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17211" PostTypeId="4" CreationDate="2019-12-22T14:25:54.713" Score="0" Body="For questions related to the elastic weight consolidation (EWC) algorithm introduced in the paper &quot;Overcoming catastrophic forgetting in neural networks&quot; (2017) by James Kirkpatrick et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-22T14:25:54.713" LastActivityDate="2019-12-22T14:25:54.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17212" PostTypeId="1" AcceptedAnswerId="17213" CreationDate="2019-12-22T15:38:45.133" Score="2" ViewCount="88" Body="&lt;p&gt;In machine learning, I understand that linear regression assumes that parameters or weights in equation should be linear. For Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = w_1x_1 + w_2x_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a linear equation where &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; are feature variables and &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; are parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = w_1(x_1)^2 + w_2(x_2)^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is also linear as parameters &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; are linear with respect to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I read some articles stating that in the equation like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = \log(w_1)x_1 + \log(w_2)x_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can also be made linear by considering other variables &lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_2$&lt;/span&gt; as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;v_1 &amp;amp;= \log(w_1)\\&#xA;v_2 &amp;amp;= \log(w_2)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = v_1x_1 + v_2x_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in this sense, any non-linear equation can be made linear, then what is non-linear regression here? I think I am missing something important here. I am a beginner in the field of Machine Learning. Can somebody help me?&lt;/p&gt;&#xA;" OwnerUserId="32247" LastEditorUserId="2444" LastEditDate="2019-12-22T16:08:00.143" LastActivityDate="2019-12-22T17:29:23.817" Title="What is the difference between linear and non-linear regression?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;linear-regression&gt;&lt;non-linear-regression&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17213" PostTypeId="2" ParentId="17212" CreationDate="2019-12-22T17:15:43.933" Score="2" Body="&lt;p&gt;The difference is simply that non-linear regression learns parameters that in some way control the non-linearity - e.g. any weight or bias that is applied &lt;em&gt;before&lt;/em&gt; a non-linear function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = (w_1 x_1 + w_2 x_2)^2 + w_3$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With such a function to learn, you &lt;em&gt;cannot&lt;/em&gt; separate out transformed values of &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; and turn this into a linear function of just &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are describing as non-linearities in your examples are instead all applied by the machine learning engineer to &lt;em&gt;create&lt;/em&gt; new candidate features for linear regression. This is not usually described as non-linear regression, but feature transformation or feature engineering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a kind of middle ground where a central linear algorithm e.g. linear regression, is trained on many variations of the original features, by automated generation and filtering of transformed features. The most general variants of this approach are not hugely popular because they suffer from same risks of overfitting as non-linear models whilst not offering much in the way of improved performance. However, if you narrow down the types of feature and transformation combinations based on some knowledge of how you expect the target function to behave, it leads to many useful variants of linear regression - e.g. regression on fourier transforms, radial basis functions etc.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2019-12-22T17:29:23.817" LastActivityDate="2019-12-22T17:29:23.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17214" PostTypeId="1" CreationDate="2019-12-22T22:00:05.837" Score="0" ViewCount="56" Body="&lt;p&gt;Given thousands of images, where some of the images contain target objects and others do not, is there an easy way of drawing bounding boxes on these target objects rather than relying on manual annotation? Wouldn't drawing 4 orientations of an object and their respective bounding boxes and randomly inserting them into the images be a viable option?&lt;/p&gt;&#xA;&lt;p&gt;It becomes painful to manually annotate thousands of images by yourself.&lt;/p&gt;&#xA;" OwnerUserId="22233" LastEditorUserId="2444" LastEditDate="2020-09-17T16:06:16.297" LastActivityDate="2021-02-14T17:07:20.800" Title="Is there a way of automatically drawing bounding boxes around interested objects?" Tags="&lt;training&gt;&lt;image-recognition&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17215" PostTypeId="2" ParentId="13086" CreationDate="2019-12-22T23:26:57.727" Score="3" Body="&lt;p&gt;Yes it has been tried. In fact there is a whole field, dubbed &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an &lt;a href=&quot;http://www.human-competitive.org/awards&quot; rel=&quot;nofollow noreferrer&quot;&gt;annual competition&lt;/a&gt; to obtain &quot;Human-Competitive&quot; algorithms, and many instances of those have been found over the years.&lt;/p&gt;&#xA;" OwnerUserId="16363" LastActivityDate="2019-12-22T23:26:57.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17216" PostTypeId="1" AcceptedAnswerId="17246" CreationDate="2019-12-23T05:17:18.317" Score="9" ViewCount="217" Body="&lt;p&gt;Can someone explain the mathematical intuition behind the forget layer of an LSTM? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), but I'm also assuming it's a matrix. Then the forget vector is calculated by concatenating the previous hidden state and the current input and adding the bias to it, then putting that through a sigmoid function that outputs a vector then that gets multiplied by the cell state matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does a concatenation of the hidden state of the previous input and the current input with the bias help with what to forget?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the previous hidden state, current input and the bias put into a sigmoid function? Is there some special characteristic of a sigmoid that creates a vector of important embeddings?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd really like to understand the theory behind calculating the cell states and hidden states. Most people just tell me to treat it like a black box, but I think that, in order to have a successful application of LSTMs to a problem, I need to know what's going on under the hood. If anyone has any resources that are good for learning the theory behind why cell state and hidden state calculation extract key features in short and long term memory I'd love to read it.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2019-12-25T03:29:42.550" LastActivityDate="2021-01-15T00:14:21.450" Title="How does the forget layer of an LSTM work?" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17217" PostTypeId="1" CreationDate="2019-12-23T08:55:09.907" Score="3" ViewCount="64" Body="&lt;p&gt;I try to solve some easy functions with a neuronal network (aforge-lib):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I generate the dataset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const int GesamtAnzahl = 200;&#xA;float[,] tempData = new float[GesamtAnzahl, 2];&#xA;float minX = float.MaxValue;&#xA;float maxX = float.MinValue;&#xA;&#xA;Random rnd = new Random();&#xA;var granzen = new List&amp;lt;int&amp;gt;() &#xA;{&#xA;    rnd.Next(1, GesamtAnzahl-1),&#xA;    rnd.Next(1, GesamtAnzahl-1),&#xA;    rnd.Next(1, GesamtAnzahl-1),&#xA;    rnd.Next(1, GesamtAnzahl-1),&#xA;};&#xA;granzen.Sort();&#xA;&#xA;for (int i = 0; i &amp;lt; GesamtAnzahl; i++)&#xA;{&#xA;&#xA;    var x = i;&#xA;    var y = -1;&#xA;    if ((i &amp;gt; granzen[0] &amp;amp;&amp;amp; i &amp;lt; granzen[1]) ||&#xA;        (i &amp;gt; granzen[2] &amp;amp;&amp;amp; i &amp;lt; granzen[3]))&#xA;    {&#xA;        y = 1;&#xA;    }&#xA;    tempData[i, 0] = x;&#xA;    tempData[i, 1] = y;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So this is quite easy: The output is 1 if the input is between the 2 lower random generated &quot;borders&quot; or between the 2 higher numbers. Otherwise the output is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input values are standardices to fit between -1 and 1. So 0 is -1 and 200 is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a network I used a BackPropagationLearning with a BipolarSigmoidFunction and several configurations like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Learning Rate: 0,1&#xA;Momentum: 0&#xA;Sigmoids alpha value: 2&#xA;Hidden Layer 1: 4 neurons&#xA;Hidden Layer 2: 2 neurons&#xA;&#xA;&#xA;Learning Rate: 0,1&#xA;Momentum: 0&#xA;Sigmoids alpha value: 2&#xA;Hidden Layer 1: 4 neurons&#xA;Hidden Layer 2: 2 neurons&#xA;Hidden Layer 3: 2 neurons&#xA;&#xA;&#xA;Learning Rate: 0,2&#xA;Momentum: 0&#xA;Sigmoids alpha value: 2&#xA;Hidden Layer 1: 4 neurons&#xA;Hidden Layer 2: 2 neurons&#xA;Hidden Layer 3: 2 neurons&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and so on. None of them worked. As described here: &lt;a href=&quot;https://towardsdatascience.com/beginners-ask-how-many-hidden-layers-neurons-to-use-in-artificial-neural-networks-51466afa0d3e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/beginners-ask-how-many-hidden-layers-neurons-to-use-in-artificial-neural-networks-51466afa0d3e&lt;/a&gt; it should be enough to have 2 hidden layers. The first one with 4 neurons and the second one with 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The configurations which worked best were:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Learning Rate: 0,01&#xA;Momentum: 0&#xA;Sigmoids alpha value: 2&#xA;Hidden Layer 1: 4 neurons&#xA;Hidden Layer 2: 4 neurons&#xA;Hidden Layer 3: 4 neurons&#xA;&#xA;Learning Rate: 0,02&#xA;Momentum: 0&#xA;Sigmoids alpha value: 2&#xA;Hidden Layer 1: 4 neurons&#xA;Hidden Layer 2: 2 neurons&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This solves the problem about 50 % of the times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this is a quite simple problem I wonder if I am doing something wrong. I think there has to be a configuration which has better results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best configuration for this problem and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally I tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Having more data does not help. I created 5000 a dataset of 5000 points ( GesamtAnzahl = 5000). Then the networks have a even worse sucess rate.&lt;/li&gt;&#xA;&lt;li&gt;I tried to add an extra constant input (always 1) to the dataset but this also lowered the sucess rate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32255" LastActivityDate="2021-01-31T19:06:30.610" Title="What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17218" PostTypeId="1" CreationDate="2019-12-23T12:58:50.090" Score="2" ViewCount="63" Body="&lt;p&gt;Assuming we have big &lt;span class=&quot;math-container&quot;&gt;$m \times n$&lt;/span&gt; input dataset, with &lt;span class=&quot;math-container&quot;&gt;$m \times 1$&lt;/span&gt; output vector. It's a &lt;strong&gt;classification&lt;/strong&gt; problem with only two possible values: either &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, the problem is that almost all elements of the output vector are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s with a very few &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;s (i.e. it's a sparse vector), such that if the neural network would &amp;quot;learn&amp;quot; to give &lt;strong&gt;always&lt;/strong&gt; 0 as output, this would produce high accuracy, while I'm also interested in learning when the 1s occurs.&lt;/p&gt;&#xA;&lt;p&gt;I thought one possible approach could be to write a custom loss function giving more weight to the 1s, but I'm not completely sure if this would be a good solution.&lt;/p&gt;&#xA;&lt;p&gt;What kind of strategy can be applied to detect such outliers?&lt;/p&gt;&#xA;" OwnerUserId="32260" LastEditorUserId="2444" LastEditDate="2020-10-28T11:32:29.533" LastActivityDate="2021-03-27T12:03:55.360" Title="How to perform binary classification when one class is more predominant than the other?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;loss-functions&gt;&lt;binary-classification&gt;&lt;imbalanced-datasets&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17219" PostTypeId="2" ParentId="7589" CreationDate="2019-12-23T14:38:54.430" Score="1" Body="&lt;p&gt;Welcome to the mine-field of semantic definitions within AI! According to Encyclopedia Britannica ML is a “discipline concerned with the implementation of computer software that can learn autonomously.” There are a bunch of other definitions for ML but generally they are all this vague, saying something about “learning”, “experience”, “autonomous”, etc. in varying order. There is no well-known benchmark definition that most people use, so unless one wants to propose one, whatever one posts on this needs to be backed up by references.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Encyclopedia Britannica’s definition the case for calling MCTS part of ML is pretty strong (Chaslot, Coulom’s et al. work from 2006-8 is used for the MCTS reference). There are two policies used in MCTS, a tree-policy and a simulation-policy. At decision time the tree-policy updates action-values by expanding the tree structure and backing up values from whatever it finds from search. There is no hard-coding on which nodes should be selected/expanded; it all comes from maximizing rewards from statistics. The nodes closer to the root appear more and more intelligent as they “learn” to mimic distributions/state and/or action-values from the corresponding ones from reality. Whether this can be called “autonomous” is an equally difficult question because in the end it’s humans who wrote the formulas/theory MCTS uses. 50 years from now it may not be called autonomous, or ML, but today it would probably at least &quot;qualify&quot;.&lt;/p&gt;&#xA;" OwnerUserId="29670" LastEditorUserId="29670" LastEditDate="2019-12-23T15:01:55.757" LastActivityDate="2019-12-23T15:01:55.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17221" PostTypeId="1" AcceptedAnswerId="17222" CreationDate="2019-12-23T15:01:43.753" Score="1" ViewCount="174" Body="&lt;p&gt;I have trained a multi-class CNN model using &lt;a href=&quot;https://www.fast.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastai&lt;/a&gt;. The model splits out probabilites for each of the three classes, which, of course, sum up to 1. The class with highest probability becomes the predicted class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way I can convert them into 0 to 1 scale, where near to 0 value would mean class 1, near to 0.5 would mean class 2 and near to 1 would mean class 3?&lt;/p&gt;&#xA;" OwnerUserId="25688" LastEditorUserId="2444" LastEditDate="2019-12-24T00:51:40.037" LastActivityDate="2019-12-24T00:51:40.037" Title="How can I convert the probability score between 0 to 1 to another format?" Tags="&lt;convolutional-neural-networks&gt;&lt;probability&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17222" PostTypeId="2" ParentId="17221" CreationDate="2019-12-23T15:25:02.097" Score="1" Body="&lt;p&gt;You could maybe do something like this, it's a bit hackish&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;y = C_1\cdot 1 + C_2 \cdot 0.5 + C_3 \cdot 0&#xA;\end{equation}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; represents the output and its bounded &lt;span class=&quot;math-container&quot;&gt;$\in [0, 1]$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is probability for class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. This way when &lt;span class=&quot;math-container&quot;&gt;$C_1 \approx 1, C_2 \approx 0, C_3 \approx 0$&lt;/span&gt; you have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; y \approx 1\cdot 1 + 0.5 \cdot 0 + 0 \cdot 0 \approx 1&#xA;\end{equation}&lt;/span&gt;&#xA;when &lt;span class=&quot;math-container&quot;&gt;$C_1 \approx 0, C_2 \approx 1, C_3 \approx 0 $&lt;/span&gt; you have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; y \approx 1\cdot 0 + 0.5 \cdot 1 + 0 \cdot 0 \approx 0.5&#xA;\end{equation}&lt;/span&gt;&#xA;and when &lt;span class=&quot;math-container&quot;&gt;$C_1 \approx 0, C_2 \approx 0, C_3 \approx 1 $&lt;/span&gt; you have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; y \approx 1\cdot 0 + 0.5 \cdot 0 + 0 \cdot 1 \approx 0&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2019-12-23T15:25:02.097" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17223" PostTypeId="2" ParentId="17221" CreationDate="2019-12-23T16:09:48.867" Score="0" Body="&lt;p&gt;In such cases, you can have just 1 final neuron and treat the problem as a regression problem where the output distance from all 3 classes is calculated and the class with least distance becomes the predicted class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want independent values for 3 classes (such as [0.8, 0.5, 0.3]) which don't add up to 1, (something like multilabel/multiclass classification), you can use sigmoid in such cases( you won't get the probability ).&lt;/p&gt;&#xA;" OwnerUserId="27875" LastActivityDate="2019-12-23T16:09:48.867" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17224" PostTypeId="2" ParentId="17168" CreationDate="2019-12-23T16:56:38.980" Score="0" Body="&lt;p&gt;I think that front end refers to a high level API for a CNN framework (c++ front end, Python front end).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The back end can be understood as a more peculiar (low level) interface to specific libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use different back ends but still manipulate training data and model building process the same way using the front end (use Keras with TensorFlow, caffe with Pytorch, or the other way round use Theano, tensorflow, .. . with Keras!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find some more material at the following links :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://keras.io/backend/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/backend/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://project.inria.fr/deeplearning/files/2016/05/DLFrameworks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://project.inria.fr/deeplearning/files/2016/05/DLFrameworks.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't think it refers to neural network layers structure. The term shallow or deep layers are usually prefered.&lt;/p&gt;&#xA;" OwnerUserId="30392" LastEditorUserId="30392" LastEditDate="2019-12-30T19:26:02.763" LastActivityDate="2019-12-30T19:26:02.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17225" PostTypeId="1" CreationDate="2019-12-23T18:41:21.837" Score="1" ViewCount="72" Body="&lt;p&gt;I’m experimenting with reinforcement learning for a 2D pixel plotting task, and am running into an issue that (I think) has to do with the big action space. It goes like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Agent gets two vector inputs each step.&#xA;Each describes an (n x n) 2d matrix composed of zeros and ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One is the (n x n) target matrix, containing a certain shape of zeros&#xA;The other is an (n x n) state matrix, containing another shape&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every step, I want my agent to pick an (x, y) coordinate:&#xA;x (picks one of n)&#xA;y (picks one of n)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will turn a zero into one, or one into zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;every step, if correct, I give a small reward, and it’ll get punished when incorrect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m training the agent (a network with 3 layers with 256 hidden units) with PPO, and curiosity in the loss, and for a 12 x 12 matrix it works quite well, not 100% but okay. (see image). Note that the agent doesn't get enough steps here to fully delete the initial shape when the target shape is empty, that's why it doesn't fully make it. Takes about 800K steps to converge though.&#xA; &lt;a href=&quot;https://i.stack.imgur.com/Y9XFx.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y9XFx.gif&quot; alt=&quot;example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the agent starts struggling in local minima when I increase beyond 32 x 32.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This one is at 32 x 32:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gB1sX.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gB1sX.gif&quot; alt=&quot;example at 32px&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this even scalable to bigger matrices even? I was hoping to go 3D eventually, by reaching 100x100x100 .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do realize that i have a huge input and action space when working with such a grid. &#xA;Is something like that even possible with an RL paradigm? I’ve tried increasing the network size, and decreasing learning rate, but I’m not satisfied. Any ideas or alternative approaches to plot pixels like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any input is very much appreciated!&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="31180" LastActivityDate="2019-12-23T18:41:21.837" Title="Reinforcement learning possible with big action space?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17226" PostTypeId="1" CreationDate="2019-12-23T19:06:35.487" Score="1" ViewCount="54" Body="&lt;p&gt;I have the following kind of data frame. These are just example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A 1 Normal&#xA;A 2 Normal&#xA;A 3 Stress&#xA;B 1 Normal&#xA;B 2 Stress&#xA;B 3 Stress&#xA;C 1 Normal&#xA;C 2 Normal&#xA;C 3 Normal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to do 5-fold cross-validation and splitting the data using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=1)&#xA;&#xA;data = (ImageList.from_folder(PATH)&#xA;        .split_by_rand_pct(valid_pct=0.2)&#xA;        .label_from_folder()&#xA;        .transform(get_transforms(do_flip=True, flip_vert= True,max_zoom=1.1, max_rotate=10, max_lighting=0.5),size=224)&#xA;        .databunch()&#xA;        .normalize() )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It works great. It splits the data randomly which is expected. Though I want to keep the data points together in the training or validation, having the same value in column 1. So, all the A's would be in either the training or validation dataset, all the B's would be in the training or validation dataset, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More info on my data:&#xA;I have cell assay images which are labelled in three classes. Now, these images are big in size, so I split one image into 16 small, non overlapping tiles, to bring down the size to 224( optimal enough to feed into CNN). All these tiles have the same label as the original image. These tiles are the final input to the CNN. TO perform cross-validation, I need to keep the tiles of same image into one fold and set.&lt;/p&gt;&#xA;" OwnerUserId="25688" LastEditorUserId="25688" LastEditDate="2019-12-24T15:36:43.280" LastActivityDate="2019-12-24T15:36:43.280" Title="How can I split the data into training and validation sets such that entries with a certain value are kept together?" Tags="&lt;python&gt;&lt;cross-validation&gt;&lt;scikit-learn&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17227" PostTypeId="1" AcceptedAnswerId="18112" CreationDate="2019-12-23T20:12:29.303" Score="6" ViewCount="121" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;SGs&lt;/a&gt; are a generalization of &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDPs&lt;/a&gt; to multiple agents. Like this &lt;a href=&quot;https://ai.stackexchange.com/questions/14246/benchmarks-for-reinforcement-learning-in-discrete-mdps&quot;&gt;previous question&lt;/a&gt; on MDPs, are there any interesting examples of zero-sum, discrete SGs—preferably with small state and action spaces? I'm hoping to use such examples as benchmarks, but couldn't find much in the literature. One example I can think of is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Pursuit-evasion&quot; rel=&quot;nofollow noreferrer&quot;&gt;pursuit-evasion game&lt;/a&gt; on a graph.&lt;/p&gt;&#xA;" OwnerUserId="3373" LastEditorUserId="3373" LastEditDate="2019-12-26T06:18:15.693" LastActivityDate="2020-02-19T03:21:43.203" Title="Interesting examples of discrete stochastic games" Tags="&lt;game-theory&gt;&lt;environment&gt;&lt;markov-decision-process&gt;&lt;benchmarks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17228" PostTypeId="1" CreationDate="2019-12-23T22:52:44.147" Score="2" ViewCount="426" Body="&lt;p&gt;I understood that we normalize to &lt;strong&gt;input&lt;/strong&gt; features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I studied about &lt;strong&gt;batch-normalization&lt;/strong&gt; and observed that we can do the normalization for outputs of the &lt;strong&gt;hidden&lt;/strong&gt; layers in following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1:&lt;/strong&gt; normalize the output of the hidden layer in order to have zero mean and unit variance a.k.a. &lt;strong&gt;standard normal&lt;/strong&gt; (i.e. subtract by mean and divide by std dev of that minibatch).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2:&lt;/strong&gt; rescale this normalized vector to a new vector with new distribution having &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; mean and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; standard deviation, where both &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; are trainable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did not understand the &lt;strong&gt;purpose of the second step&lt;/strong&gt;. Why can't we just do the first step, make the vector standard normal, and then move forward? Why do we need to rescale the input of each hidden neuron to an arbitrary distribution which is learned (through beta and gamma parameters)?&lt;/p&gt;&#xA;" OwnerUserId="32269" LastActivityDate="2021-05-17T14:04:30.380" Title="How does a batch normalization layer work?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17231" PostTypeId="1" CreationDate="2019-12-24T08:11:54.380" Score="4" ViewCount="524" Body="&lt;p&gt;I am new to self-supervised learning and it all seems a little magical at the moment.&lt;/p&gt;&#xA;&lt;p&gt;The only way I can get an intuitive understanding is to assume that, for real-world problems, features are still embedded at a per-object level.&lt;/p&gt;&#xA;&lt;p&gt;For example, to detect cats in unseen images, my self-supervised network would still have to be composed exclusively of cats.&lt;/p&gt;&#xA;&lt;p&gt;So, if I had 100 images of cats and 100 images of dogs, then I thought self-supervised approaches would learn the features of the images. For example, if an image is rotated 90 degrees, it learns what was in the image that was rotated 90 degrees. However, if I wanted to classify just cats using this representation, then I wouldn't be able to do so without separating out what makes a cat a cat and a dog a dog.&lt;/p&gt;&#xA;&lt;p&gt;Is my assumption correct?&lt;/p&gt;&#xA;" OwnerUserId="32275" LastEditorUserId="2444" LastEditDate="2020-11-20T02:49:22.157" LastActivityDate="2020-11-20T02:49:22.157" Title="How to understand the concept of self-supervised learning in AI?" Tags="&lt;self-supervised-learning&gt;&lt;feature-learning&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17232" PostTypeId="2" ParentId="17228" CreationDate="2019-12-24T10:47:30.020" Score="0" Body="&lt;p&gt;&lt;strong&gt;Definition and Explaination&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For how Batch Normalization works exactly, I'll suggest you to read the following papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1502.03167&quot; rel=&quot;nofollow noreferrer&quot;&gt;Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1805.11604&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Does Batch Normalization Help Optimization?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The recent interpretation on How BN works is that it can reduce the high-order effect as mentioned in &lt;a href=&quot;https://www.youtube.com/embed/Xogn6veSyxA?start=325&amp;amp;end=664&amp;amp;version=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ian Goodfellow's lecture&lt;/a&gt;. So it's not really about reducing the internal covariate shift.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Intuition&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For how it works intuitively, you can think that we want to normalize the intermediate outputs (zero mean and unit variance) if the normalization won't remove too much useful information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, normalization may not be suitable for all intermediate outputs. So &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is introduced to provide additional flexibility, if normalization removes too much useful information then &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; will learn to become the original mean and variance, making the BN layer an identity transformation, as if it doesn't exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; won't become the original mean and variance, since all intermediate outputs can be normalized in some certain way without losing too much useful information. So you can think of it to be a customized normalization for each BN layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BN layer normalize the intermediate outputs in default, however, if the neural network find out that these intermediate outputs should not be normalized, then the neural network undos or provide more flexibility to the normalization.&lt;/p&gt;&#xA;" OwnerUserId="32173" LastActivityDate="2019-12-24T10:47:30.020" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17233" PostTypeId="2" ParentId="17231" CreationDate="2019-12-24T11:00:13.940" Score="3" Body="&lt;p&gt;I don't think your interpretation is correct. Take images as example.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Supervised Learning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. classification (maybe use CNN with a L2 loss function)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you have many images with different labels. You wish to find a function to approximate the function &lt;span class=&quot;math-container&quot;&gt;$y=f(x)$&lt;/span&gt; given a lot of &lt;span class=&quot;math-container&quot;&gt;$(\hat x, \hat y)$&lt;/span&gt; sample pairs. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unsupervised Learning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. clustering (maybe use k-means) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you have many images, but we don't have the labels or we just want to see if there's a way to categorize them into different categories. So we cluster the images by some  characteristic that isn't pre-defined.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Self-Supervised Learning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. super resolution (maybe use CNN with a L2 loss function)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have many high resolution images without labels, but, your target is to train a model to up sample a low resolution image. So you can have the high resolution images as target, and down size the image to be the input, and try to train the image pairs. So the target is not some manually tagged labels, but generated directly from the data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32173" LastActivityDate="2019-12-24T11:00:13.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17234" PostTypeId="1" CreationDate="2019-12-24T12:25:52.997" Score="1" ViewCount="27" Body="&lt;p&gt;This is more of a general question of how to model/preprocess 'visual' state-observations to an Agent in Reinforcement Learning that I'll illustrate with an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have a reinforcement learning problem where the agent has to draw pixels in an n * n 2D state-matrix of 0's and 1's. Say n = 100. The agent can move one step (up, down, left, right) and on its location can additionally switch 0's into 1's or the other way around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each step, it needs to take action so that the state-matrix resembles an n * n target-matrix (that has a certain shape). It is rewarded accordingly each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent will know its location from an x and y position that are given in addition to the state- and target-matrix each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm curious to the question what the best way is to represent the state to the agent. Using a visual 'prior', or not. Here's two ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on that you want to give only the essential information to the agent: The agent is presented with a matrix (with target subtracted from state), that will be flattened into one array of n^2. Additionally it'll know its current location as an additional (x, y) vector observation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on that (1) would be more difficult to solve for a human, because you'll have to learn from a flattened array how different points are connected (think about how hard a flattened game of chess would be), you can also use a convolutional neural network to encode the current scene. In this case the agent will be e.g. a red dot. Given that it's such a visual task, it seems to me that using this would give the agent a better model of how the environment works, since the spatial relations are kept intact. Also it feels that keeping the 2D shape intact with a CNN would mean that it'd form better representations that generalize to other shapes, but I can't really say why.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;On the other hand one could say that it's arrogant to assume that our 'human' spatial way of interpreting visual information is the best way for this case. Maybe there's a mathematical solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="31180" LastActivityDate="2019-12-24T12:25:52.997" Title="Flattened vector observation or convolutional neural network input?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17235" PostTypeId="1" AcceptedAnswerId="17237" CreationDate="2019-12-24T12:56:07.840" Score="1" ViewCount="51" Body="&lt;p&gt;I am training a simple convolutional neural network to recognize two types of 1024-point frequency spectra (FFT). This is the model I'm using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cnn = Sequential()&#xA;cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU(), input_shape=(nInput,1)))&#xA;cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU()))&#xA;cnn.add(MaxPooling1D(pool_size=2))&#xA;cnn.add(Flatten())&#xA;cnn.add(Dense(nFinalDense, activation=LeakyReLU()))&#xA;cnn.add(Dense(nOutput, activation='sigmoid'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However I get the following accuracy and loss during training:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eXd1x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eXd1x.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do I get the large peak in both plots? How can it be explained? Is there a problem with the data I'm using (I mention that I obtain a similar peak when training an autoencoder for denoising using the same data)?&lt;/p&gt;&#xA;" OwnerUserId="32237" LastActivityDate="2019-12-24T18:49:35.863" Title="How to explain peak in training history of a convolutional neural network?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;accuracy&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17237" PostTypeId="2" ParentId="17235" CreationDate="2019-12-24T18:49:35.863" Score="1" Body="&lt;p&gt;I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.&lt;/p&gt;&#xA;" OwnerUserId="32237" LastActivityDate="2019-12-24T18:49:35.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17238" PostTypeId="1" CreationDate="2019-12-24T19:26:02.923" Score="3" ViewCount="43" Body="&lt;p&gt;Suppose we're training two agents to play an asymmetric game from scratch using self play (like Zerg vs. Protoss in Starcraft). During training one of the agents can become stronger (discover a good broad strategy for example) and start winning most of the time, which causes big portion of the state values (or Q(s,a) values) become very high for this agent and low for another, just because the first is generally stronger and receives most of the rewards. Some training time later the other one finds a weakness in the first's play (in many states too) and starts dominating and the reward stream shift the other way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, we have to retrain function approximator (deep neural net) to wildly different value/Q states, this slows and destabilizes learning. For each of the agents this is similar to highly nonstationary environment (the opponent), that can be harsh or easy at times. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do people usually do in such a case? I think what is needed is some kind of slowly changing baseline (similar to advantage in A2C), but applied to the reward values themselves.&lt;/p&gt;&#xA;" OwnerUserId="32286" LastActivityDate="2019-12-24T19:26:02.923" Title="How to deal with nonstationary rewards in asymmetric self-play reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;self-play&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17239" PostTypeId="1" CreationDate="2019-12-24T20:55:22.940" Score="1" ViewCount="27" Body="&lt;p&gt;I have 10000 tuples of numbers &lt;code&gt;(x1, x2, y)&lt;/code&gt; generated from the equation: &lt;code&gt;y = np.cos(0.583 * x1) + np.exp(0.112 * x2)&lt;/code&gt;. I want to use a neural network, trained with gradient descent, in PyTorch, to find the 2 parameters, i.e. 0.583 and 0.112&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class NN_test(nn.Module):&#xA;    def __init__(self):&#xA;        super().__init__()&#xA;        self.a = torch.nn.Parameter(torch.tensor(0.7))&#xA;        self.b = torch.nn.Parameter(torch.tensor(0.02))&#xA;&#xA;    def forward(self, x):&#xA;        y = torch.cos(self.a*x[:,0])+torch.exp(self.b*x[:,1])&#xA;        return y&#xA;&#xA;model = NN_test().cuda()&#xA;&#xA;lrs = 1e-4&#xA;optimizer = optim.SGD(model.parameters(), lr = lrs)&#xA;loss = nn.MSELoss()&#xA;&#xA;epochs = 30&#xA;for epoch in range(epochs):&#xA;    model.train()&#xA;    for i, dtt in enumerate(my_dataloader):&#xA;        optimizer.zero_grad()&#xA;&#xA;        inp = dtt[0].float().cuda()&#xA;        output = dtt[1].float().cuda()&#xA;&#xA;        ls = loss(model(inp),output)&#xA;&#xA;        ls.backward()&#xA;        optimizer.step()&#xA;    if epoch%1==0:&#xA;        print(&quot;Epoch: &quot; + str(epoch), &quot;Loss Training: &quot; + str(ls.data.cpu().numpy()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x&lt;/code&gt; contains the 2 numbers &lt;code&gt;x1&lt;/code&gt; and &lt;code&gt;x2&lt;/code&gt;. In theory, it should work easily, but the loss doesn't go down. What am I doing wrong? &lt;/p&gt;&#xA;" OwnerUserId="22839" LastEditorUserId="2444" LastEditDate="2019-12-25T03:40:08.347" LastActivityDate="2019-12-25T03:40:08.347" Title="How can I train a neural network to find the hyper-parameters with which the data was generated?" Tags="&lt;gradient-descent&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17240" PostTypeId="1" AcceptedAnswerId="17301" CreationDate="2019-12-25T03:54:59.317" Score="4" ViewCount="116" Body="&lt;p&gt;I know that autoencoders are one type of deep neural networks that can learn the latent representation of data. I guess there should be several other models like autoencoders. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some new deep learning models for learning latent representation of data?&lt;/p&gt;&#xA;" OwnerUserId="32290" LastEditorUserId="2444" LastEditDate="2019-12-26T15:20:32.923" LastActivityDate="2020-01-07T04:12:03.993" Title="What are some new deep learning models for learning latent representation of data?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;autoencoders&gt;&lt;latent-variable&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17241" PostTypeId="1" CreationDate="2019-12-25T08:26:52.200" Score="1" ViewCount="30" Body="&lt;p&gt;I have a video dataset as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dataset size: 1k videos&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frames per video: 4k (average) and 8k (maximum)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Labels: Each video has one label. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the size of my input will be (N, 8000, 64, 64, 3) &#xA;64 is height and width of video. I use keras. I am not really sure how to do an end-to-end training with this kind of dataset. I was thinking of dividing each input in blocks of frames (N, 80, 100, 64, 64, 3) for training. But still it wont work for an end-to-end network training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not in favor of  dropping the frames. That might be my last choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated. Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="32297" LastEditorUserId="32297" LastEditDate="2019-12-26T02:48:34.893" LastActivityDate="2019-12-26T02:48:34.893" Title="How to handle a high dimensional video (large number of frames per video) data for training a video classification network" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;dimensionality&gt;&lt;video-classification&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17242" PostTypeId="1" CreationDate="2019-12-25T09:56:54.587" Score="3" ViewCount="81" Body="&lt;p&gt;I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to classify human actions real-time like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Left-arm bended&lt;/li&gt;&#xA;&lt;li&gt;Arm above shoulder&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I first did some research for pre-trained models, but I didn't find any.&#xA;Because I'm still quite new, I want to have advice about how I should solve this.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I thought maybe I need to create for every action enough pictures and from there on I can do image classification. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or I use PoseNet from TensorFlow so that I have the pose estimation points. And from there on I create videos of a couple of seconds with every pose I want to track and I save the estimation points. From there on, I use a classification algorithm (neural network) to classify those points. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient option or are they both bad and is there a better way to do this?&lt;/p&gt;&#xA;" OwnerUserId="32300" LastEditorUserId="2444" LastEditDate="2019-12-26T15:19:28.253" LastActivityDate="2021-05-20T15:01:55.207" Title="How to classify human actions?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;video-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17243" PostTypeId="1" CreationDate="2019-12-25T12:16:34.820" Score="3" ViewCount="46" Body="&lt;p&gt;In neural Turing machine (NTM), reading memory is represented as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align} &#xA;r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and writing to memory is represented as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step1: Erase&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align} &#xA;\mathcal{M}_t^{erased}(i) \leftarrow \mathcal{M}_{t-1}(i)[\mathbf{1} - w_t(i) e_t ] \tag{3}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step2: Add&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align} &#xA;\mathcal{M}_t(i) \leftarrow \mathcal{M}_t^{erased}(i) + w_t(i) a_t \tag{4}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the reading mechanism, if we take this example values and applied to the above formula, instead of a vector, we get a scalar of value 2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M_t =[[1,0,1,0],&#xA;      [0,1,0,0],&#xA;      [1,1,1,0]]&#xA;&#xA;w_t = [1,1,1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same thing happens in writing as well; here we take the dot product of two vectors,  &lt;span class=&quot;math-container&quot;&gt;$w_t(i) e_t$&lt;/span&gt;,  with a scalar value as output. According to paper, unless &lt;span class=&quot;math-container&quot;&gt;$w_t$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$e_t$&lt;/span&gt; are zeros, it will erase &lt;strong&gt;all values&lt;/strong&gt; in the memory matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own idea about NTM memory was that it uses the weights to find the indices or rows inside the memory matrix corresponding to a certain task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does the memory in NTM work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How a memory for a particular task is stored, that is, is it stored row-wise or it's stored in the whole matrix?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2019-12-27T12:13:30.110" LastActivityDate="2019-12-27T12:13:30.110" Title="How does the memory mechanism (reading and writing) work in a neural Turing machine?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;memory&gt;&lt;neural-turing-machine&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17244" PostTypeId="1" CreationDate="2019-12-25T14:50:34.587" Score="2" ViewCount="36" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1612.08242.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, YOLO has three features compared to YOLO v1. This question is about Better and Faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Better section, there are many techniques such as Batch Norm, Anchor Box and so on. In the Faster section, there is a darknet only.&#xA;Darknet has 19 Conv Layer but it doesn't use Layer Norm or Passthrough Layer. So, I think that Darknet doesn't use Better section techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the Better Section model different from Faster Section Model?&#xA;In my understanding, there are three models named YOLO v2. First is Better YOLO v2, second is Faster YOLO v2, third Strong YOLO v2. Is this right?&lt;/p&gt;&#xA;" OwnerUserId="32303" LastEditorUserId="2444" LastEditDate="2019-12-26T05:39:52.797" LastActivityDate="2019-12-26T05:39:52.797" Title="YOLO 9000 about Better Stronger" Tags="&lt;deep-learning&gt;&lt;object-recognition&gt;&lt;papers&gt;&lt;yolo&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17245" PostTypeId="1" CreationDate="2019-12-25T16:39:48.713" Score="1" ViewCount="39" Body="&lt;p&gt;I have n-tuple based tic tac toe. I already have perfect minimax player and perfectly trained table-based player. My n-tuple network consists of 8 different rows of 3 of the board as triplets having possible empty, X or O, and one bit defining who's move is now, so totally 2 * 3^3 = 54 states in tuple. I train and update weights with the idea of the pseudo code from &quot;Learning to Play Othello with N-Tuple Systems&quot; by Simon Lucas:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public void inGameUpdate(double[] prev, double[] next) {&#xA;  double op = tanh(net.forward(prev));&#xA;  double tg = tanh(net.forward(next));&#xA;  double delta = alpha * (tg - op) * (1 - op * op);&#xA;  net.updateWeights(prev, delta);&#xA;}&#xA;&#xA;public void terminalUpdate(double[] prev, double tg) {&#xA;  double op = tanh(net.forward(prev));&#xA;  double delta = alpha * (tg - op) * (1 - op * op);&#xA;  net.updateWeights(prev, delta);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the score is the sum of weights of those rows of 3. The temporal difference training generally works for n-tuple based tic tac toe and after several thousands games it mostly plays perfectly. But after a while it diverges from the perfection and oscillates between perfect and near perfect. I realized it was in situations like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OXO&#xA;X-O&#xA;-XX&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suspect because row that prevent opponent from winning has big value. And having two of such rows seems to be better than losing later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I can have perfect player basing on this particular n-tuple network. I could just stop training after I reach perfection, but in bigger games I can't do that. I fiddled with different alpha in ranges 0.1-0.0001 and e-greedy epsilon 1%-50%, or adaptive. Increasing epsilon to about 50% somewhat mitigates this effect, but this value is mostly very big to use in other games.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are couple questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does this effect have a name in the machine learning world? It values preventing opponent from winning. But if opponent has more opportunities to win, its value will be bigger, so that it will exceed the (negative of) losing value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Aside from probably using different n-tuple networks and tweaking hyper-parameters, what can I do to mitigate or eliminate this effect?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In bigger games, this learning and n-tuple system give fairly good result, but I see big oscillations after certain points. I.e. in breakthrough game against 1-ply minimax, after it reaches about 60% winrate (testing 10000 games after training every 10000 games against itself), its winrate goes slitghly up, but in testing its winrate oscillates between 45-65%. Can this effect be caused by the problem I mentioned in 1.?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16663" LastActivityDate="2019-12-25T16:39:48.713" Title="N-tuple based tic tac toe diverges in temporal difference learning" Tags="&lt;q-learning&gt;&lt;temporal-difference-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17246" PostTypeId="2" ParentId="17216" CreationDate="2019-12-25T19:28:20.693" Score="2" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; Here is a beautiful explanation with diagrams: &lt;a href=&quot;https://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To address:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the cell state is essentially long term memory embedding (correct me if I'm wrong)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The embedding can be long or short term and it is a vector.&lt;/p&gt;&#xA;&lt;p&gt;To answer:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why is the previous hidden state, current input and the bias put into a sigmoid function? Is there some special characteristic of a sigmoid that creates a vector of important embeddings?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Excerpt from source:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The sigmoid layer outputs numbers between zero and one, describing how much of each component should be let through. A value of zero means “let nothing through,” while a value of one means “let everything through!”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Formally the forget vector is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_t = \sigma(W_f\cdot[h_{t-1},x_t],+b_f)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So we see that it is actually a linear operation followed by a non-linear operation, which restricts the values to be between 0 and 1. This is followed by an element-wise operation on the previous cell state. That is we &amp;quot;gate&amp;quot;/&amp;quot;filter&amp;quot; the previous cell state doing:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C_{t-1}\odot f_t,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; is element-wise multiplication.&lt;/p&gt;&#xA;&lt;p&gt;Thus we see that the forget gate &lt;em&gt;really is acting like a gate&lt;/em&gt;: It either lets values pass through or it pushes them toward zero. So, it's purpose is to assign what values to forget/remember.&lt;/p&gt;&#xA;&lt;p&gt;To answer:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How does a concatenation of the hidden state of the previous input and the current input with the bias help with what to forget?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This concatenation takes into account the &amp;quot;rolling&amp;quot; hidden state &lt;em&gt;and&lt;/em&gt; the current input. That is, the linear operation results in a new vector which can be seen as a &amp;quot;consideration embedding&amp;quot; of the current input and a &amp;quot;summary&amp;quot; of past inputs and cell states. The sigmoid then converts this &amp;quot;consideration embedding&amp;quot; into the forget vector.&lt;/p&gt;&#xA;&lt;p&gt;In summary, the forget gate has the primary purpose of forming a vector of values between zero. This vector results by considering the current input and the previous hidden state. The vector is used to forget/remember parts of the previous cell state via element-wise multiplication. So, no, it is not a black box, it is a highly considerate non-linear operation through time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Additional Intuition - More Mathematical&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Recall that multiplying a matrix on the right by a column vector produces a vector, which is a linear combination of the columns of the matrix. So each column of &lt;span class=&quot;math-container&quot;&gt;$W_f$&lt;/span&gt; can be seen as a column of &amp;quot;toggles&amp;quot; that push up to 1 or pull down to 0 (once we apply sigmoid). That is, the columns of &lt;span class=&quot;math-container&quot;&gt;$W_f$&lt;/span&gt; form a sort of &amp;quot;basis&amp;quot; for a &amp;quot;toggle space.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;When we concatenate to form &lt;span class=&quot;math-container&quot;&gt;$[h_{t-1},x_t]$&lt;/span&gt; and then multiply we see that the term &lt;span class=&quot;math-container&quot;&gt;$W_f\cdot[h_{t-1},x_t]$&lt;/span&gt; gives consideration to the hidden state and the input. That is, both &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; have there hand in toggling the gate.&lt;/p&gt;&#xA;&lt;p&gt;The LSTM has learned from the data how to make this toggling &amp;quot;meaningful.&amp;quot; Below is a simple pictographic example.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HhJE3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HhJE3.png&quot; alt=&quot;Pictographic parallel of matrix form&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is, &lt;span class=&quot;math-container&quot;&gt;$W_f\cdot[h_{t-1},x_t]$&lt;/span&gt; will be a vector in the column space of &lt;span class=&quot;math-container&quot;&gt;$W_f$&lt;/span&gt; (ie the &amp;quot;toggle space&amp;quot;). Thus, the concatenated vector will &amp;quot;point&amp;quot; to the best &amp;quot;toggle&amp;quot; in the column (&amp;quot;toggle&amp;quot;) space of &lt;span class=&quot;math-container&quot;&gt;$W_f$&lt;/span&gt;. This &amp;quot;toggle&amp;quot; is then transformed into a &amp;quot;gate&amp;quot; in the &amp;quot;gate space.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;So intuitively, the range of the forget component is a sort of &amp;quot;gate space&amp;quot; where the domain is all vectors of form &lt;span class=&quot;math-container&quot;&gt;$[h_{t-1},x_t]$&lt;/span&gt;. Formally:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_t:[h_{t-1},x_t]\mapsto g_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is to say that the forget gate learns the best mapping from the domain of concatenated hidden/input's to the range of possible gates.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Important Note&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The LSTM is only &lt;em&gt;one&lt;/em&gt; of many types of forget-gate mappings. The main takeaway is that the LSTM works &lt;em&gt;empirically&lt;/em&gt; for many applications.&lt;/p&gt;&#xA;&lt;p&gt;That is a forget gate is an instance of a general &amp;quot;gate mapping&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_t:v_t \mapsto g_t,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt; could be any number of vectors resulting from any number of concatenations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some data from the past is irrelevant to the current time step. We need a way to correctly &amp;quot;forget&amp;quot; irrelevant information. One way to &amp;quot;forget&amp;quot; is to use a forget gate mapping (like the one in LSTMs). Then we need to optimize the parameters of that forget gate. &lt;em&gt;One&lt;/em&gt; implementation is used in the common LSTM architecture. Finally, the concatenation of the previous hidden state provides contextual information to the current input which is very helpful with forgetting.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;More Mathematical Intuition&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is another view of what is happening in the LSTM from a dynamic point of view: &lt;a href=&quot;https://youtu.be/jvj4WefJ0S0?list=PLiJYYsK6pD-nVcROKw2YDOVaNMjST5YyL&amp;amp;t=1711&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="2444" LastEditDate="2021-01-15T00:14:21.450" LastActivityDate="2021-01-15T00:14:21.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17247" PostTypeId="2" ParentId="5123" CreationDate="2019-12-26T06:21:15.987" Score="1" Body="&lt;p&gt;See &lt;a href=&quot;https://arxiv.org/abs/1904.09626&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep Metric Learning Beyond Binary Supervision&lt;/em&gt;&lt;/a&gt; in CVPR 2019&lt;/p&gt;&#xA;" OwnerUserId="32311" LastEditorUserId="12345" LastEditDate="2019-12-26T17:23:06.423" LastActivityDate="2019-12-26T17:23:06.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17248" PostTypeId="1" CreationDate="2019-12-26T10:17:07.753" Score="2" ViewCount="246" Body="&lt;p&gt;In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on classification trees. Now, I read somewhere that we cannot use Information gain (with impurity function as entropy) as a splitting method for regression trees. Why is it so, and what other methods are there which we can and cannot use, and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDITS:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please suggest me a reference to understand maths behind it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The references I used are : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.analyticsvidhya.com/blog/2016/04/complete-tutorial-tree-based-modeling-scratch-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.analyticsvidhya.com/blog/2016/04/complete-tutorial-tree-based-modeling-scratch-in-python/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.python-course.eu/Regression_Trees.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.python-course.eu/Regression_Trees.php&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/https-medium-com-lorrli-classification-and-regression-analysis-with-decision-trees-c43cdbc58054&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/https-medium-com-lorrli-classification-and-regression-analysis-with-decision-trees-c43cdbc58054&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first article, it is mentioned that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gini Index, Chi-Square and Information gain (impurity function as entropy) algorithms are used for Classification trees while Reduction in Variance is used for Regression Trees.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the second article, it is mentioned that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since our target feature is continuously scaled, the IGs of the categorically scaled descriptive features are no longer appropriate splitting criteria.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;As stated above, the task during growing a Regression Tree is in principle the same as during the creation of Classification Trees. Though, since the IG turned out to be no longer an appropriate splitting criteria (neither is the Gini Index) due to the continuous character of the target feature we must have a new splitting criteria.&#xA;  Therefore we use the variance which we will introduce now.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the third article, it is mentioned that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Entropy as a measure of impurity is a useful criterion for classification. To use a decision tree for regression, however, we need an impurity metric that is suitable for continuous variables, so we define the impurity measure using the weighted mean squared error (MSE) of the children nodes instead&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thank You!&lt;/p&gt;&#xA;" OwnerUserId="32247" LastEditorUserId="2444" LastEditDate="2020-01-09T00:02:50.450" LastActivityDate="2020-01-09T00:02:50.450" Title="Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;&lt;metric&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17249" PostTypeId="1" CreationDate="2019-12-26T11:04:19.693" Score="1" ViewCount="291" Body="&lt;p&gt;Objective :  &lt;strong&gt;To find the nearest object (closer distance object)  in the single camera image. But Image Contains multiple objects shown below:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N9bA8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N9bA8.png&quot; alt=&quot;Multiple Object in a Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched in the net and found this formula to calculate the distance of object from camera &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;F = (P x  D) / W&lt;/strong&gt; &lt;a href=&quot;https://www.pyimagesearch.com/2015/01/19/find-distance-camera-objectmarker-using-python-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Further Detail click on the Link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other better approach to find the nearest object in a image?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in Advance!!!&lt;/p&gt;&#xA;" OwnerUserId="9863" LastActivityDate="2019-12-26T11:04:19.693" Title="Find the nearest object in a image which is captured from camera?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17250" PostTypeId="2" ParentId="17200" CreationDate="2019-12-26T11:24:48.500" Score="2" Body="&lt;p&gt;If I had to implement a path exploration/finding algorithm on a robot, I would follow these steps:   &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Make sure you can detect your position. You need to be able to &lt;em&gt;record&lt;/em&gt; your position otherwise you have no reference for the exploration. You don't need a global positioning system (like GPS), a local one is more than enough in your case. This means that the robot must know that it has been switched on in position (0,0). If you go straight for 5 meters, you'll update it to (5,0) or something like that.&lt;br&gt;&#xA;Knowing &lt;em&gt;how much&lt;/em&gt; you moved and &lt;em&gt;towards where&lt;/em&gt; is the difficult thing.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once you can know &lt;strong&gt;where&lt;/strong&gt; you are, it is time to record it. As you want to explore the environment, you might want to create a tree with states on the nodes. The node can be &lt;em&gt;open&lt;/em&gt; if you can still explore around it, &lt;em&gt;closed&lt;/em&gt; if the exploration has been done.&lt;br&gt;&#xA;To get the path from a node to another, A* works more than enough.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To know what is around you, you can use the sensors to explore the surroundings and know the position of the obstacles.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is the general idea:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Tree tree = new Tree()&#xA;Node first = new Node(Here)&#xA;tree.Add(first)&#xA;&#xA;Node current = first&#xA;&#xA;while(true) {&#xA;    ExploreAround(current)&#xA;    var nodes = CreateSurroundingNodes()&#xA;    tree.Add(nodes)&#xA;    current.State = closed&#xA;    var next = PickNearestOpenNode()&#xA;    MoveTo(next)&#xA;    WaitUntilRobotIsOn(next)&#xA;    current = next&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my bachelor thesis I was implementing a very similar algorithm on a drone on Unity3D. You can find the package &lt;a href=&quot;https://assetstore.unity.com/packages/3d/vehicles/air/realistic-drone-66698&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You can get an idea out of it.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a video of how it worked: &lt;a href=&quot;https://youtu.be/Xrh9-4Bfcew&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/Xrh9-4Bfcew&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4x2kE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4x2kE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2019-12-26T11:24:48.500" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17251" PostTypeId="1" CreationDate="2019-12-26T11:29:16.783" Score="1" ViewCount="27" Body="&lt;p&gt;I want to create a small dataset (about 10 classes and 20-30 images each), should I add some noise (wrong label sample) in the training, validation and test datasets, and why?&lt;/p&gt;&#xA;" OwnerUserId="32320" LastEditorUserId="2444" LastEditDate="2019-12-26T13:10:16.050" LastActivityDate="2019-12-26T13:10:16.050" Title="Should I add some noise when the dataset is small?" Tags="&lt;datasets&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17252" PostTypeId="1" CreationDate="2019-12-26T11:59:22.003" Score="1" ViewCount="25" Body="&lt;p&gt;I was wondering if there is any application / API which given a word comes up with legible and meaningful information related to it, and if possible relates it to any recent happenings or development in this context. For example, if I were to give the term &quot;Donald Trump&quot;, it should give a snippet, maybe from Wikipedia, and show any 2-3 recent headlines related to it.&lt;/p&gt;&#xA;" OwnerUserId="32321" LastEditorUserId="2444" LastEditDate="2019-12-26T13:16:06.117" LastActivityDate="2019-12-26T13:16:06.117" Title="Is there any app / API which generates meaningful information?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2021-02-03T16:46:27.140" ContentLicense="CC BY-SA 4.0" />
  <row Id="17253" PostTypeId="1" CreationDate="2019-12-26T15:16:16.967" Score="1" ViewCount="70" Body="&lt;p&gt;This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the images that it generates are just random noise. Here are my script and the images that were generated by the model: &lt;a href=&quot;https://github.com/jweir136/PyTorch-Variational-Autoencoder-Latent-Space-Visualization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jweir136/PyTorch-Variational-Autoencoder-Latent-Space-Visualization&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice or answers on how to solve this problem are greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="32323" LastEditorUserId="2444" LastEditDate="2019-12-26T15:22:27.127" LastActivityDate="2019-12-26T15:22:27.127" Title="Why is my variational auto-encoder generating random noise?" Tags="&lt;autoencoders&gt;&lt;pytorch&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17254" PostTypeId="2" ParentId="9692" CreationDate="2019-12-26T16:32:03.663" Score="2" Body="&lt;p&gt;I believe that the idea is to have a similar ratio of fraud/&quot;normal transaction&quot; as to the ones that bank encounter on real life.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you balance it you will probably have a lot of false positive once you apply your solution to real world's data and, if that may be fine for you to play with, it's not what a bank would like as they can't block too much of the &quot;normal&quot; transaction or the client will change of bank. According to this post (&lt;a href=&quot;https://www.quora.com/How-many-transactions-do-typical-banks-process-everyday&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/How-many-transactions-do-typical-banks-process-everyday&lt;/a&gt;) just the mastercard alone represent 3.4billion transaction/day, just imagine if 1% of the daily transaction where blocked every day, it would represent 34 million of transactions blocked without any valid reason. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's different from a lot of classification problem where you want to have balanced dataset, here you try to detect anomalies and, by definition, they are rare so they should be as rare in your dataset. &lt;/p&gt;&#xA;" OwnerUserId="26961" LastActivityDate="2019-12-26T16:32:03.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17255" PostTypeId="1" AcceptedAnswerId="17257" CreationDate="2019-12-26T19:18:10.280" Score="1" ViewCount="453" Body="&lt;p&gt;Okay so here's my CNN (simple example from a tutorial) along with some arithmetic to get the total number of free parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've got a dataset of 28*28 grayscale image (MNIST).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First layer is a 2D convolution using 32 3x3 kernels. Dimensionality of the output is 26x26x32 (kernel stride length was 1 and we have 32 feature maps of 26x26). &lt;strong&gt;Running parameter count: 288&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Second layer is 2x2 MaxPool with a 2x2. Dimensionality of the output is 13x13x32 but then we flatten so we got a vector of length 5408. No extra parameters here.&lt;/li&gt;&#xA;&lt;li&gt;Third layer is Dense. A 5408x100 matrix. Dimensionality of the output is 100. &lt;strong&gt;Running Parameter count: 540988&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Fourth layer is Dense also. A 100x10 matrix. Dimensionality of the output is 10. &lt;strong&gt;Running Parameter count: 541988&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then we're supposed to do stochastic gradient descent on a 541988 parameter space!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That feels like a ridiculously big number to me. And this is meant to be the hello world problem of CNNs. Am I missing something fundamental in my understanding of how this is meant to work? Or maybe the number is correct but it's not actually a big deal for a computer to crunch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it helps. Here is how the model was built in Keras:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def define_model():&#xA;    model = Sequential()&#xA;    model.add(Conv2D(32, (3,3), activation = 'relu', kernel_initializer = 'he_uniform', input_shape=(28,28,1)))&#xA;    model.add(MaxPooling2D((2,2)))&#xA;    model.add(Flatten())&#xA;    model.add(Dense(100, activation='relu', kernel_initializer='he_uniform'))&#xA;    model.add(Dense(10, activation='softmax'))&#xA;    opt = SGD(lr=0.01, momentum=0.9)&#xA;    model.compile(optimizer=opt, loss='categorical_crossentropy', metric=['accuracy'])&#xA;    return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2019-12-26T19:26:41.317" LastActivityDate="2019-12-26T20:32:30.577" Title="How many parameters are being optimised over in a simple CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17256" PostTypeId="1" AcceptedAnswerId="17355" CreationDate="2019-12-26T19:27:26.913" Score="2" ViewCount="140" Body="&lt;p&gt;A single neuron with &lt;strong&gt;2 weights and identity activation&lt;/strong&gt; can learn addition/subtraction as the 2 weights will converge to &lt;strong&gt;1 and 1&lt;/strong&gt; (addition), or &lt;strong&gt;1 and -1&lt;/strong&gt; (subtraction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for &lt;strong&gt;multiplication&lt;/strong&gt; and &lt;strong&gt;division&lt;/strong&gt;, it's not that easy. Can a single neuron learn multiplication or division? If not, how many layers of DNN can learn these?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-01-04T20:15:47.240" Title="How to make DNN learn multiplication/division?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17257" PostTypeId="2" ParentId="17255" CreationDate="2019-12-26T20:27:19.153" Score="2" Body="&lt;p&gt;Neural networks can have a lot of different structures. CNNs can have a number of parameters that ranges from a few thousands to several millions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general you aim to increase the number of filters and reduce the first 2 dimensions, as you go deeper in the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you had &lt;code&gt;Conv -&amp;gt; pool -&amp;gt; Conv -&amp;gt; pool -&amp;gt; ...&lt;/code&gt; , you could do for example first conv with kernel size = 5 and 8 filters and the second conv with kernel size = 5 and 16 filters. And both pools being (2,2).. But this is just an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your network you start with a 28*28 image, and you use 32 3*3 filters. so number of parameters is &lt;code&gt;(3*3 + 1) * 32 = 320&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the dense layer you have as input a &lt;code&gt;13*13*32&lt;/code&gt; and use a 100 FC layer. so n_parameters is &lt;code&gt;(13*13*32 + 1)*100&lt;/code&gt; which is &lt;code&gt;540900&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you get &lt;code&gt;(100+1) * 10&lt;/code&gt; FC which is &lt;code&gt;1010&lt;/code&gt; more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;total = 320 + 540900 + 1000&lt;/code&gt; which is &lt;code&gt;542230&lt;/code&gt;, as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The +1 that shows up in every layer is the Bias neuron. Basically you add a bias neuron per output in a connection between 2 layers. In the FC1000 to the FC10 it is easy to understand, you have a bias per output neuron. In the Convolutional layers you have a bias term per each of the filters applied, so for each filter you have the filters weights plus 1 for the bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from that you also had a small math mistake when adding the 288 at the beginning, you added 188. So your missing 242 parameters were: 32 from the conv layer, 100 from the first dense, 10 from the second dense and 100 from the sum. 100+100+32+10 = 242.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2019-12-26T20:32:30.577" LastActivityDate="2019-12-26T20:32:30.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17258" PostTypeId="2" ParentId="17196" CreationDate="2019-12-26T21:45:14.843" Score="1" Body="&lt;p&gt;&lt;strong&gt;Task Specification&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's been proposed that &lt;em&gt;novelty search&lt;/em&gt; may circumvent this problem.  See: &lt;a href=&quot;http://Abandoning%20Objectives:%20Evolution%20Through%20the%20Search%20for%20Novelty%20Alone&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abandoning Objectives: Evolution Through&#xA;the Search for Novelty Alone&lt;/a&gt;.  In this model, the agent has no goal or objective, but just messes around with the data to see what results.  (This could be regarded as finding/forming patterns.  Here's a recent popular article on the subject: &lt;a href=&quot;https://www.quantamagazine.org/computers-evolve-a-new-path-toward-human-intelligence-20191106/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computers Evolve a New Path Toward Human Intelligence&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A form of procedural generation may also be useful, specifically the capability of creating novel models/environments and processes/algorithms to analyze them.  (See: &lt;a href=&quot;https://arxiv.org/pdf/1905.10985.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-GAs: AI-generating algorithms&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of programmers communicating a task to the AGI, that's a natural language problem if the task relates to mundane human activity or art and craft, and a math problem if the subject is physics.  (In the former case, humans are describing the problem in natural language, in the latter, presumably feeding all of the data that suggests dark matter into the algorithm.  Natural language is challenging for computers, but math, along with logic, is one of their two core functions.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re: dark matter, it may be a matter of asking the algorithm to find patterns in the data, and build models based on the data. The patterns and models would be the output, which humans could then consider. The output would be mathematical.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Converting that mathematical output into metaphors, as in common on science programs like Nova and Cosmos, would be another goal of AGI.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Brain in a Box&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There needs to be stimulus/input to initiate &quot;thought&quot; process/computation.  In brain in a box, the brain is providing it's own internal stimulus.  I'd argue that an RL algorithm engaged in self-play is not dependent on external stimulus, but internally generated inputs, so that the process of model-based reinforcement learning is often a brain in a box, considering a subject or problem.    &lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2019-12-26T22:09:43.677" LastActivityDate="2019-12-26T22:09:43.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17259" PostTypeId="1" CreationDate="2019-12-27T02:14:11.943" Score="2" ViewCount="17" Body="&lt;p&gt;I have been using a network to generate graphs. The architecture that I have been using is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2J56E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2J56E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this figure, &lt;span class=&quot;math-container&quot;&gt;$D_1$&lt;/span&gt; is the signal generator and &lt;span class=&quot;math-container&quot;&gt;$D_2$&lt;/span&gt; is the graph topology generator, which is a square, symmetric matrix which indicates which node is connected to which. In this network, &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; shows linear layers, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; shows activation functions. Here we are using leaky relu activation function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that I am experiencing is that after training the network, my output is only a chain of nodes, meaning that only subdiagonal and superdiagonal elements have non-zero values and it is very rare to have other forms of graph. I was wondering if anyone has a suggestion for improving the output. Note that my training data is diverse and has every kind of graphs.&lt;/p&gt;&#xA;" OwnerUserId="31990" LastActivityDate="2019-12-27T02:14:11.943" Title="Improving graph decoder network" Tags="&lt;generative-model&gt;&lt;graphs&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17260" PostTypeId="2" ParentId="17195" CreationDate="2019-12-27T06:34:00.160" Score="0" Body="&lt;p&gt;You can do custom POS Tagging and use it as a multi featured sequence2sequence. &lt;/p&gt;&#xA;" OwnerUserId="30690" LastActivityDate="2019-12-27T06:34:00.160" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17261" PostTypeId="1" CreationDate="2019-12-27T06:42:13.417" Score="2" ViewCount="20" Body="&lt;p&gt;I am currently writing my thesis about human pose estimation and wanted to use Google's inception network, modify it for my needs and use transfer learning to detect human key joints. I wanted to ask if that could be done in that way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I am having n-keypoints, generating the n-feature maps, use transfer learning and cut off the final classification layers and replace it by a FCN which guesses the key joints. I am asking myself if this might be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, these feature maps should output heatmaps with the highest probability as well. Is this assumption valid? &lt;/p&gt;&#xA;" OwnerUserId="32337" LastActivityDate="2019-12-27T06:42:13.417" Title="Can an image recognition model used for human pose estimation?" Tags="&lt;classification&gt;&lt;computer-vision&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17262" PostTypeId="1" CreationDate="2019-12-27T08:17:16.113" Score="1" ViewCount="19" Body="&lt;p&gt;In cutting edge deployment of deep networks for different architectures (such as &lt;span class=&quot;math-container&quot;&gt;$CNN$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$QRNN$&lt;/span&gt; etc) what is the historical trend of current limits of trainability possible computationally? By this I mean number of nodes and number of weights?&lt;/p&gt;&#xA;" OwnerUserId="32340" LastActivityDate="2019-12-27T08:17:16.113" Title="Number of weights in historical to cutting edge deployment of deep networks" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;history&gt;&lt;architecture&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2019-12-30T19:25:17.613" ContentLicense="CC BY-SA 4.0" />
  <row Id="17263" PostTypeId="2" ParentId="16714" CreationDate="2019-12-27T08:20:08.297" Score="0" Body="&lt;p&gt;Find some opensource projects like &lt;a href=&quot;https://github.com/karpathy/arxiv-sanity-preserver&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arxiv sanity&lt;/a&gt; by andrej karpathy which finds ML related papers from arxiv.org. You can find similar opensource applications.&lt;/p&gt;&#xA;" OwnerUserId="27875" LastActivityDate="2019-12-27T08:20:08.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17264" PostTypeId="1" AcceptedAnswerId="17279" CreationDate="2019-12-27T09:14:14.317" Score="2" ViewCount="42" Body="&lt;p&gt;I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a sample I have manually counted number of vehicles in each image. Also, I employed the three deep learning models and obtained the vehicle counts. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Actual        | model 1 count| model 2 count  | model 3 count &#xA;------------------------------------------------------------------&#xA;    4 cars, 1 bus | 2 cars       | 2 cars, 1 truck| 4 cars&#xA;    2 cars        | 0            | 1 truck        | 1 car, 1 bus&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, how can I measure accuracy scores such as precision and recall?&lt;/p&gt;&#xA;" OwnerUserId="32343" LastActivityDate="2019-12-27T20:11:50.050" Title="Accuracy scores in a Deep Learning project" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;accuracy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17265" PostTypeId="1" CreationDate="2019-12-27T09:24:49.553" Score="1" ViewCount="132" Body="&lt;p&gt;Could I get a dataset that can classify ambulances? &#xA;I have searched everywhere, but, couldn't seem to get hold of a set of annotated images for ambulances.&lt;/p&gt;&#xA;" OwnerUserId="32249" LastActivityDate="2019-12-27T09:46:52.807" Title="Ambulance dataset needed" Tags="&lt;datasets&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17267" PostTypeId="2" ParentId="17265" CreationDate="2019-12-27T09:46:52.807" Score="1" Body="&lt;p&gt;Answering my own question here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looked at the Open Image Dataset by Google @ &lt;a href=&quot;https://storage.googleapis.com/openimages/web/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://storage.googleapis.com/openimages/web/index.html&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They provide image-level labels, object bounding boxes, object segmentation masks, and visual relationships.&lt;/p&gt;&#xA;" OwnerUserId="32249" LastActivityDate="2019-12-27T09:46:52.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17268" PostTypeId="2" ParentId="8258" CreationDate="2019-12-27T09:54:07.270" Score="3" Body="&lt;p&gt;Look at Google's Open Image Dataset @ &lt;a href=&quot;https://storage.googleapis.com/openimages/web/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://storage.googleapis.com/openimages/web/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They provide image-level labels, object bounding boxes, object segmentation masks, and visual relationships. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;a href=&quot;https://storage.googleapis.com/openimages/web/visualizer/index.html?set=train&amp;amp;type=detection&amp;amp;c=%2Fm%2F01mqdt&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; for the traffic signs dataset.&lt;/p&gt;&#xA;" OwnerUserId="32249" LastActivityDate="2019-12-27T09:54:07.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17270" PostTypeId="1" CreationDate="2019-12-27T10:23:23.813" Score="3" ViewCount="85" Body="&lt;p&gt;I'm working on an example of CNN with the MNIST hand-written numbers dataset. Currently I've got convolution -&gt; pool -&gt; dense -&gt; dense, and for the optimiser I'm using Mini-Batch Gradient Descent with a batch size of 32.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this concept of batch normalization is being introduced. We are supposed to take a &quot;batch&quot; after or before a layer, and normalize it by subtracting its mean, and dividing by its standard deviation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is a &quot;batch&quot;? If I feed a sample into a 32 kernel conv layer, I get 32 feature maps. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is each feature map a &quot;batch&quot;? &lt;/li&gt;&#xA;&lt;li&gt;Are the 32 feature maps the &quot;batch&quot;? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or, if I'm doing Mini-Batch Gradient Descent with a batch size of 64,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are 64 sets of 32 feature maps the &quot;batch&quot;? So in other words, the batch from Mini-Batch Gradient Descent, is the same as the &quot;batch&quot; from batch-optimization?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or is a &quot;batch&quot; something else that I've missed?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="25496" LastEditDate="2019-12-27T16:20:30.793" LastActivityDate="2019-12-27T16:22:19.190" Title="What is a &quot;batch&quot; in batch normalization?" Tags="&lt;neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17271" PostTypeId="1" AcceptedAnswerId="17277" CreationDate="2019-12-27T10:41:31.897" Score="3" ViewCount="46" Body="&lt;p&gt;how would you describe a machine learning model in a scientific report? It should be detailed but I just listed the hyperparameters... Have you got more important properties?&lt;/p&gt;&#xA;" OwnerUserId="32344" LastEditorUserId="1671" LastEditDate="2019-12-30T19:19:44.400" LastActivityDate="2019-12-30T19:19:44.400" Title="How to describe an keras Model in a scientific report" Tags="&lt;machine-learning&gt;&lt;keras&gt;&lt;academia&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17272" PostTypeId="2" ParentId="17242" CreationDate="2019-12-27T11:30:07.803" Score="0" Body="&lt;p&gt;My suggestion is to go with 1st option. reason is you will get to know much about data and initial stage will find some challenges in developing the model, over a period of time you will get to better results after hypertunning. Please go through &lt;a href=&quot;http:////https://machinelearningmastery.com/deep-learning-models-for-human-activity-recognition/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; , ignore you have already read this article&lt;/p&gt;&#xA;" OwnerUserId="32346" LastActivityDate="2019-12-27T11:30:07.803" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17273" PostTypeId="1" AcceptedAnswerId="17276" CreationDate="2019-12-27T12:14:23.407" Score="7" ViewCount="1371" Body="&lt;p&gt;Feature extraction is a concept concerning the translation of raw data into the inputs that a particular machine learning algorithm requires. These derived features from the raw data that are actually relevant to tackle the underlying problem. On the other hand, word embeddings are basically distributed representations of text in an n-dimensional space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, word embedding is a somehow feature extraction technique. Am I wrong ? I had an argument with a friend who believes the two topics are totally separate. Is he right? What are the similarities and dissimilarities between word embedding and feature extraction?&lt;/p&gt;&#xA;" OwnerUserId="32347" LastEditorUserId="2444" LastEditDate="2019-12-27T14:47:10.520" LastActivityDate="2019-12-27T20:43:39.210" Title="Is word embedding a form of feature extraction?" Tags="&lt;comparison&gt;&lt;word-embedding&gt;&lt;feature-extraction&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17274" PostTypeId="2" ParentId="17273" CreationDate="2019-12-27T16:18:44.730" Score="2" Body="&lt;p&gt;I think you guys are playing on semantics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider feature extraction to be an unlearned preprocessing step to get inputs for your model, then no, word embeddings are not a feature extraction technique (examples here would be BoW counts, n-gram features, etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider feature extraction to be any form of conversion from text to a set of features, then yes, word embeddings should be considered a form of feature extraction, given that they've learned in the process (or stolen from another model's training). Note though if you do include this, you would probably include most pretrained models as a whole as feature extraction techniques (like BERT).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the whole conversation you had can go either way depending on the definitions you set.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2019-12-27T20:43:39.210" LastActivityDate="2019-12-27T20:43:39.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17275" PostTypeId="2" ParentId="17270" CreationDate="2019-12-27T16:22:19.190" Score="3" Body="&lt;p&gt;The &quot;batch&quot; is same as in mini-batch gradient descent. The mean in batch-norm here would be the average of each feature map in your batch (in your case either 32 or 64 depending on which you use)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;generally batch is used quite consistently in ML right now, where it refers to the inputs you send in together for forward/backward pass.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2019-12-27T16:22:19.190" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17276" PostTypeId="2" ParentId="17273" CreationDate="2019-12-27T17:14:12.260" Score="5" Body="&lt;p&gt;Though &lt;em&gt;word-embedding&lt;/em&gt; is primarily a &lt;em&gt;language modeling tool&lt;/em&gt;, it also acts as a feature extraction method because it helps transform raw data (characters in text documents) to a meaningful alignment of word vectors in the embedding space that the model can work with more effectively (than other traditional methods such as TF-IDF, Bag of Words, etc, on a large corpus). Word Embedding techniques help extract information from the pattern and occurence of words and goes further than other traditional token representation methods to decode/identify the meaning/context of the words, thereby providing more relevant and important features to the model to tackle the underlying problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from another standpoint, word-embedding models were not developed aiming to solve a particular feature extraction problem, but rather, to &lt;em&gt;generalize&lt;/em&gt; and &lt;em&gt;model&lt;/em&gt; the language used in a corpus to gain a &lt;em&gt;semantic understanding&lt;/em&gt; of the words and the &lt;em&gt;relationships&lt;/em&gt; between them. Such that, all the various corpus-specific tasks can then &lt;em&gt;employ the same &quot;library&quot; of information&lt;/em&gt; which was &lt;em&gt;collectively&lt;/em&gt; &amp;amp; &lt;em&gt;exhaustively&lt;/em&gt; learnt by the embedding model. Meaning, the word embedding model learns a language model that is &lt;em&gt;task-agnostic&lt;/em&gt; for all tasks on that corpus unlike feature extraction methods which are specifically task-oriented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, the similarity is - word-embeddings can effectively aid in feature extraction; the dissimilarity is - they're not primarily meant to extract features more than they are for modeling a language which might be an &quot;overkill&quot; for a particular feature extraction task on a dataset.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2019-12-27T17:19:16.203" LastActivityDate="2019-12-27T17:19:16.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17277" PostTypeId="2" ParentId="17271" CreationDate="2019-12-27T17:49:27.137" Score="3" Body="&lt;p&gt;Some other details you could mention are:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;total number of model parameters (e.g. 1.2M or 0.15M) &amp;amp; depth of the network (e.g. 38-layered network)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;family/style of the network architecture (e.g. encoder-decoder arch., LSTM)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;specifics of connections between network layers (e.g. residual-, dense-, skip-connections)&lt;/li&gt;&#xA;&lt;li&gt;specifics of individual components of the network structure (e.g. dilated-convs. (CNNs), attention (LSTMs))&lt;/li&gt;&#xA;&lt;li&gt;description/reasoning of why you chose a particular structure/sequence of connections in your deep learning model&lt;/li&gt;&#xA;&lt;li&gt;specifics of training/validation/testing procedures (e.g. augmented training data, cross-validation, test-time-augmentation (TTA), frozen network weights)&lt;/li&gt;&#xA;&lt;li&gt;other specific details/caveats that allow the results of your deep learning model be easily reproduced from the scientific report&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For more info on the best kinds of details to be included in the report, refer to &quot;Methodology&quot;/ &quot;Training&quot;/ &quot;Implementation&quot;/ &quot;Proposed Architecture&quot; sections of the deep learning research papers in your relevant area.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2019-12-27T18:33:19.900" LastActivityDate="2019-12-27T18:33:19.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17278" PostTypeId="1" CreationDate="2019-12-27T18:40:30.870" Score="1" ViewCount="40" Body="&lt;p&gt;So I am making a project based on this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1901/1901.07761.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/ftp/arxiv/papers/1901/1901.07761.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper, a U-Net is used to generate optimized mechanical structures. I am trying to recreate the model and use it on my own generated data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 7.1 a pixelwise accuracy is mentioned. Right now I am using the default Keras metric &quot;accuracy&quot;, which isn't reaching even close to the accuracy in the paper. (it starts at 0.3ish and goes to like 0.45). What I always do is to manually compare the generated structures to the ground truth in the training set. There are often models which have better accuracy, but the structures make less sense. What accuracy metric did they use in the paper?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper under 4.2.1, the KL Divergence is mentioned. My problem was, that the KL Divergence turned negative after an epoch or two (an Indicator that I don't work with probability distributions?), so I switched to binary cross-entropy, which provides good results, but it is still bothering me , that I cant use the proposed loss method. Another point is the L2 Regularization: I get the best results using 1e-7 or lower as the l2 value, which is low compared to the normally used values. What does that indicate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point I wanted to mention: the dimensions of my data is a little bit different from the ones used in the paper: I use 65x49 as my Input Dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if someone can help me in fixing the problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried the following code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def kullback_leibler_divergence_test(y_true, y_pred):&#xA;&#xA;   true_sum = K.sum(y_true)&#xA;&#xA;   pred_sum = K.sum(y_pred)&#xA;&#xA;   new_true = y_true/true_sum&#xA;&#xA;   new_pred = y_pred/pred_sum&#xA;&#xA;   y_true = K.clip(new_true, K.epsilon(), 1)&#xA;&#xA;   y_pred = K.clip(new_pred, K.epsilon(), 1)&#xA;&#xA;   return K.sum(y_true * K.log(y_true / y_pred), axis=-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I got a non negative loss!! But the problem is it is very low.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach was using: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def kullback_leibler_divergence_test_2(y_true, y_pred):&#xA;&#xA;    y_true = K.clip(y_true, K.epsilon(), 1)&#xA;    y_pred = K.clip(y_pred, K.epsilon(), 1)&#xA;    return K.sum(y_true * abs(K.log(y_true /  y_pred)), axis=-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the loss was pretty high compared to the Binary Crossentropy and I have the feeling the loss is working not as intended.&lt;/p&gt;&#xA;" OwnerUserId="32354" LastActivityDate="2019-12-27T18:40:30.870" Title="Can anyone explain the pixelwise accuracy metric used in this paper? Also a question to the KL Divergence Loss" Tags="&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;&lt;probability&gt;&lt;bayesian-statistics&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17279" PostTypeId="2" ParentId="17264" CreationDate="2019-12-27T20:11:50.050" Score="2" Body="&lt;p&gt;Precision is the number of true positives over the number of predicted positives(PP), and recall is the number of true positives(TP) over the number of actual positives(AP) you get. I used the initials just to make it easier ahead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A true positive is when you predict a car in a place and there is a car in that place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A predicted positive is every car you predict, being right or wrong does not matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A actual positive is every car that actually is in the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should calculate these values separately for each category, and then sum over the examples you sampled, if I am not mistaken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for the CAR category you have (assuming the predictions do match with the target, i.e., you are not predicting a truck as a car for example) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model 1&#xA;line 1  -&amp;gt; 2 TP, 2 PP, 4 AP &#xA;line 2  -&amp;gt; 0 TP, 0 PP, 2 AP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So in total precision  is &lt;code&gt;2/2 = 1&lt;/code&gt; and recall is &lt;code&gt;2/6 = 0.3(3)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then do the same for the other categories, and for the other models. This way you can check if a model is predicting one category better than the other. For example, model 1 can be better at finding cars in a picture whilst model 3 can be better at finding buses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important part is that you know if the objects the model predicted actually correspond to what is in the picture. A very unlikely example would be a picture with 1 car and 1 truck where the algorithm recognizes the car as a truck and the truck as a car. From the info that is in the table I cannot be sure if the 2 cars you predict are the actual cars in the picture, or in other words, if they are actually True Positives or are actually False Positives. &lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2019-12-27T20:11:50.050" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17281" PostTypeId="1" CreationDate="2019-12-28T04:53:08.060" Score="1" ViewCount="36" Body="&lt;p&gt;The capsule neural networks have been formally introduced in the paper &lt;a href=&quot;https://arxiv.org/abs/1710.09829&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing Between Capsules&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much ado has been made about how the capsules output a vector (magnitude = probability that an entity is present, orientation space = the instantiated parameters), which can then allow it to maintain more information than a max-pooled operation which outputs only a scalar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within these vector representations, the dimensions of this space turn out to be the parameters by which a written digit could vary: scale, stroke thickness, skew, width, etc. There are 16 neurons in each capsule to represent 16 dimensions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is unclear to me, from reading the paper, if these parameters emerged through training, or if they were hand-coded a priori. If these parameters were not hand-coded, why do such &quot;clean&quot; dimensions emerge? Why don't mixed-selective neurons emerge within the 16?&lt;/p&gt;&#xA;" OwnerUserId="32369" LastEditorUserId="2444" LastEditDate="2020-06-09T11:36:16.217" LastActivityDate="2020-06-09T11:36:16.217" Title="Is the number of neurons in each capsule in a capsule neural network hardcoded?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;capsule-neural-network&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17282" PostTypeId="1" CreationDate="2019-12-28T07:17:35.067" Score="2" ViewCount="69" Body="&lt;p&gt;I am practicing with Resnet50 fine-tuning for a binary classification task. Here is my code snippet.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;base_model = ResNet50(weights='imagenet', include_top=False)&#xA;x = base_model.output&#xA;x = keras.layers.GlobalAveragePooling2D(name='avg_pool')(x)&#xA;x = Dropout(0.8)(x)&#xA;model_prediction = keras.layers.Dense(1, activation='sigmoid', name='predictions')(x)&#xA;model = keras.models.Model(inputs=base_model.input, outputs=model_prediction)&#xA;opt = SGD(lr = 0.01, momentum = 0.9, nesterov = False)&#xA; &#xA;model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])  #&#xA;   &#xA;train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=False)&#xA;  &#xA;test_datagen = ImageDataGenerator(rescale=1./255)&#xA;train_generator = train_datagen.flow_from_directory(&#xA;        './project_01/train',&#xA;        target_size=(input_size, input_size),  &#xA;        batch_size=batch_size,&#xA;        class_mode='binary')    &#xA;&#xA;validation_generator = test_datagen.flow_from_directory(&#xA;        './project_01/val',&#xA;        target_size=(input_size, input_size),&#xA;        batch_size=batch_size,&#xA;        class_mode='binary')&#xA;&#xA;hist = model.fit_generator(&#xA;        train_generator,&#xA;        steps_per_epoch= 1523 // batch_size, # 759 + 764 NON = 1523&#xA;        epochs=epochs,&#xA;        validation_data=validation_generator,&#xA;        validation_steps= 269 // batch_size)  # 134 + 135NON = 269&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I plotted a figure of the model after training for 50 epochs:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jp2XH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jp2XH.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You may have noticed that &lt;code&gt;train_acc&lt;/code&gt; and &lt;code&gt;val_acc&lt;/code&gt; have highly fluctuated, and &lt;code&gt;train_acc&lt;/code&gt; merely reaches 52%, which means that network isn't learning, let alone over-fitting the data.&lt;/p&gt;&#xA;&lt;p&gt;As for the losses, I haven't got any insights.&lt;/p&gt;&#xA;&lt;p&gt;Before training starts, network outputs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Found 1523 images belonging to 2 classes.&#xA;Found 269 images belonging to 2 classes.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is my fine-tuned model learning anything at all?&lt;/p&gt;&#xA;&lt;p&gt;I'd appreciate if someone can guide me to solve this issue.&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="2444" LastEditDate="2021-05-23T13:16:34.727" LastActivityDate="2021-05-23T13:16:34.727" Title="Is my fine-tuned model learning anything at all?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;keras&gt;&lt;transfer-learning&gt;&lt;fine-tuning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17283" PostTypeId="2" ParentId="17282" CreationDate="2019-12-28T10:45:27.327" Score="0" Body="&lt;p&gt;It's difficult to say without knowing what your data looks like but from the numbers it seems too less and the images might be too similar to one another or very different. In any case, I'd have checked using other networks like Inception and decreasing learning rate even further (say 0.0001) to not mess with the Imagenet weights if your data is not very different from Imagenet classes.&lt;/p&gt;&#xA;" OwnerUserId="20494" LastActivityDate="2019-12-28T10:45:27.327" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17284" PostTypeId="1" CreationDate="2019-12-28T11:16:09.477" Score="2" ViewCount="144" Body="&lt;p&gt;I wish to compile a (somewhat) comprehensive list of AGI systems that have actually been created and tested (to whatever degrees of success) instead of those that simply advertise they are going to 'do' something about it or have patented theoretical concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the purposes of this question, we can use the following &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of AGI&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Artificial general intelligence (AGI) is the intelligence of a machine that can understand or learn any intellectual task that a human being can&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="32373" LastEditorUserId="2444" LastEditDate="2021-01-31T18:30:46.040" LastActivityDate="2021-01-31T18:30:46.040" Title="Which AGI systems have already been implemented and tested?" Tags="&lt;reference-request&gt;&lt;agi&gt;&lt;research&gt;&lt;state-of-the-art&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17285" PostTypeId="1" CreationDate="2019-12-28T12:34:00.970" Score="1" ViewCount="80" Body="&lt;p&gt;I just trained my birds model. It works fine when I was testing it with close pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I moved the pictures further away my camera, the model was not able to detect birds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is in this line of code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data = ImageDataBunch.from_folder(path=path_img, bs=48, valid_pct= 0.2, ds_tfms=get_transforms(), size=299, classes=['birds', 'others'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After checked the source code of get_transforms()&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_transforms(do_flip:bool=True, ....)-&amp;gt;Collection[Transform]:&#xA;    &quot;Utility func to easily create a list of flip, rotate, `zoom`, warp, lighting transforms.&quot;&#xA;    res = [rand_crop()] .....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can see that it does the crop and there is no way to turn it off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone help please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1y4E7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1y4E7.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2mBsx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2mBsx.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27988" LastActivityDate="2019-12-28T12:34:00.970" Title="FastAi How to turn off crop in ImageDataBunch" Tags="&lt;machine-learning&gt;&lt;pytorch&gt;&lt;fast-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17286" PostTypeId="2" ParentId="17284" CreationDate="2019-12-28T14:35:52.257" Score="3" Body="&lt;p&gt;As far as I know, no &amp;quot;true&amp;quot; (i.e. as intellectual and physically capable as a human) artificial general intelligent system (AGI) has been implemented or is practically useful (this is confirmed by Ben Goertzel, who is one of the leading researchers in AGI [&lt;a href=&quot;https://www.linkedin.com/pulse/sophias-ai-ben-goertzel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://www.youtube.com/watch?v=r4manxX5U-0&amp;amp;ab_channel=TEDxTalks&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]).&lt;/p&gt;&#xA;&lt;p&gt;The closest to a practical AGI might be &lt;a href=&quot;https://www.hansonrobotics.com/sophia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sophia&lt;/a&gt; (or similar robots), which may look like an AGI, but it lacks several capabilities that we humans have and its ability to adapt to new circumstances is limited. Sophia uses &lt;a href=&quot;https://opencog.org/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt; [&lt;a href=&quot;https://www.linkedin.com/pulse/sophias-ai-ben-goertzel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], which is supposed to be a software framework to develop AGI. Sophia is used for the &lt;a href=&quot;https://lovingai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;loving AI project&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are also &lt;a href=&quot;https://ai.stackexchange.com/q/21427/2444&quot;&gt;theoretical frameworks for AGI&lt;/a&gt;, such as &lt;a href=&quot;https://jan.leike.name/AIXI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIXI&lt;/a&gt;, which, in any case, have several flaws, such as incomputability (in the case of AIXI). There are &lt;a href=&quot;https://github.com/sgkasselau/pyaixi&quot; rel=&quot;nofollow noreferrer&quot;&gt;approximations of AIXI&lt;/a&gt;, but these approximations can only be used to solve toy problems (such as tic-tac-toe), so they aren't really useful to solve complex real-world problems. However, it is possible that better approximations to AIXI and more theoretical frameworks for AGI will be developed that can deal with more complex problems.&lt;/p&gt;&#xA;&lt;p&gt;It's also important to note that, despite their success, AlphaGo and AlphaStar are &lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;narrow AI&lt;/a&gt; systems, as they can only solve one specific problem (although the same approach can be adapted to solve very similar problems too, e.g. AlphaZero).&lt;/p&gt;&#xA;&lt;p&gt;If you want to know more about AGI, the &lt;a href=&quot;http://www.scholarpedia.org/article/Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scholarpedia&lt;/a&gt;'s article &lt;a href=&quot;http://www.scholarpedia.org/article/Artificial_General_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;, curated by Ben Goertzel, provides a good overview of the AGI field, including but not restricted to definitions of AGI and approaches to the development of AGI systems, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;universal (&lt;a href=&quot;https://jan.leike.name/AIXI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIXI&lt;/a&gt; was created based on this approach),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;symbolic (&lt;a href=&quot;https://soar.eecs.umich.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soar&lt;/a&gt; was created based on this approach), which is based on the &lt;a href=&quot;https://ai.stanford.edu/%7Enilsson/OnlinePubs-Nils/PublishedPapers/pssh.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;physical symbol system hypothesis&lt;/a&gt;,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;emergentist (or sub-symbolic), which is based on the idea that general intelligence is expected to &lt;em&gt;emerge&lt;/em&gt; from sub-symbolic dynamics (where a sub-symbolic system refers e.g. to an artificial neural network),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;hybrid (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/CLARION_(cognitive_architecture)&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLARION&lt;/a&gt;), which is a combination of the universal, symbolic or sub-symbolic approaches.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You probably also want to take a look at &lt;a href=&quot;https://ai.stackexchange.com/q/7431/2444&quot;&gt;this question&lt;/a&gt;, which has a few answers that mention a paper and book that you can read to know more about the AGI field. You may also be interested in &lt;a href=&quot;https://ai.stackexchange.com/q/20180/2444&quot;&gt;this related question&lt;/a&gt; too.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-31T18:15:50.907" LastActivityDate="2021-01-31T18:15:50.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17287" PostTypeId="1" CreationDate="2019-12-28T15:39:18.030" Score="4" ViewCount="123" Body="&lt;p&gt;I've seen some comments in online articles/tutorials or &lt;a href=&quot;https://stackoverflow.com/questions/53942612/why-too-many-epochs-will-cause-overfitting&quot;&gt;Stack Overflow questions&lt;/a&gt; which suggest that increasing number of epochs can result in overfitting. But my intuition tells me that there should be no direct relationship at all between number of epochs and overfitting. &lt;strong&gt;So I'm looking for answer which explains if I'm right or wrong (or whatever's in between).&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my reasoning though. To overfit, you need to have enough free parameters (I think this is called &quot;capacity&quot; in neural networks) in your model to generate a function which can replicate the sample data points. If you don't have enough free parameters, you'll never overfit. You might just under fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So really, if you don't have too many free parameters, you could run infinite epochs and never overfit. If you have too many free parameters, then yes, the more epochs you have the more likely it is that you get to a place where you're overfitting. But that's just because running more epochs revealed the root cause: too many free parameters. The real loss function doesn't care about how many epochs you run. It existed the moment you defined your model structure, before you ever even tried to do gradient descent on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I'd venture as far as to say: assuming you have the computational resources and time, you should always aim to run as many epochs as possible, because that will tell you whether your model is prone to overfitting. Your best model will be the one that provides great training and validation accuracy, no matter how many epochs you run it for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;While reading more into this, I realise I forgot to take into account that you can arbitrarily vary the sample size as well. Given a fixed model, a smaller sample size is more prone to being overfit. And then that kind of makes me doubt my intuition above. Still happy to get an answer though!&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2019-12-28T15:51:49.110" LastActivityDate="2020-10-10T12:29:34.737" Title="Is running more epochs really a direct cause of overfitting?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17288" PostTypeId="1" CreationDate="2019-12-28T18:17:50.860" Score="1" ViewCount="583" Body="&lt;p&gt;I have written this &lt;a href=&quot;https://colab.research.google.com/drive/1FJxDmuUlRSC0OWWUlcW50QXeb3d0NS09&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; to classify Cats and dogs using Resnet50. Actually while studying I came to the conclusion that Transfer learning gives very good accuracy for deep learning models, but I ended getting a far worse result and I didn't understand the cause for it. Any description with reasoning would be very helpful. The dataset contains 2000 images of cats and dogs as training and 1000 images as the validation set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following summarises my model&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from tensorflow.keras.applications import ResNet50&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras.layers import Dense, InputLayer, Flatten, GlobalAveragePooling2D&#xA;num_classes = 2&#xA;IMG_SIZE = 224&#xA;IMG_SHAPE = (IMG_SIZE, IMG_SIZE, 3)&#xA;my_new_model=tf.keras.applications.ResNet50(include_top=False, weights='imagenet', input_shape=IMG_SHAPE, pooling='avg', classes=2)&#xA;my_new_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])&#xA;&#xA;&#xA;from tensorflow.keras.preprocessing.image import ImageDataGenerator&#xA;from tensorflow.keras.applications.resnet50 import preprocess_input&#xA;train_datagen = ImageDataGenerator(&#xA; preprocessing_function=preprocess_input,&#xA; rotation_range=40,&#xA; width_shift_range=0.2,&#xA; height_shift_range=0.2,&#xA; shear_range=0.2,&#xA; zoom_range=0.2,&#xA; horizontal_flip=True,)&#xA;&#xA;# Note that the validation data should not be augmented!&#xA;test_datagen = ImageDataGenerator(preprocessing_function=preprocess_input)&#xA;&#xA;train_generator = train_datagen.flow_from_directory(&#xA;     train_dir,  # This is the source directory for training images&#xA;     target_size=(224,224),  # All images will be resized to 224x224&#xA;     batch_size=20,&#xA;     class_mode='binary')&#xA;&#xA;validation_generator = test_datagen.flow_from_directory(&#xA;     validation_dir,&#xA;     target_size=(224, 224),&#xA;     class_mode='binary')&#xA;&#xA;my_new_model.fit_generator(&#xA;     train_generator,&#xA;     epochs = 8,&#xA;     steps_per_epoch=100,&#xA;     validation_data=validation_generator)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For this I get the training logs as,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Train for 100 steps, validate for 32 steps&#xA;Epoch 1/8&#xA;100/100 - 49s - loss: 7889.4051 - accuracy: 0.0000e+00 - val_loss: 7834.5318 - val_accuracy: 0.0000e+00&#xA;Epoch 2/8&#xA;100/100 - 35s - loss: 7809.7583 - accuracy: 0.0000e+00 - val_loss: 7775.1556 - val_accuracy: 0.0000e+00&#xA;Epoch 3/8&#xA;100/100 - 35s - loss: 7808.4858 - accuracy: 0.0000e+00 - val_loss: 7765.3964 - val_accuracy: 0.0000e+00&#xA;Epoch 4/8&#xA;100/100 - 35s - loss: 7808.0520 - accuracy: 0.0000e+00 - val_loss: 7764.0735 - val_accuracy: 0.0000e+00&#xA;Epoch 5/8&#xA;100/100 - 35s - loss: 7807.7891 - accuracy: 0.0000e+00 - val_loss: 7762.4891 - val_accuracy: 0.0000e+00&#xA;Epoch 6/8&#xA;100/100 - 35s - loss: 7807.6872 - accuracy: 0.0000e+00 - val_loss: 7762.1766 - val_accuracy: 0.0000e+00&#xA;Epoch 7/8&#xA;100/100 - 35s - loss: 7807.6633 - accuracy: 0.0000e+00 - val_loss: 7761.9766 - val_accuracy: 0.0000e+00&#xA;Epoch 8/8&#xA;100/100 - 35s - loss: 7807.6514 - accuracy: 0.0000e+00 - val_loss: 7761.9346 - val_accuracy: 0.0000e+00&#xA;&amp;lt;tensorflow.python.keras.callbacks.History at 0x7f5adff722b0&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I change the class_mode='categorical' it's giving error as&lt;br&gt;&#xA;Incompatible shapes: [20,2] vs. [20,2048].&lt;/p&gt;&#xA;" OwnerUserId="32378" LastEditorUserId="32378" LastEditDate="2019-12-28T18:47:57.407" LastActivityDate="2019-12-28T18:47:57.407" Title="Reasoning behind $Zero$ validation accuracy in the following ResNet50 model for classification" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17289" PostTypeId="1" CreationDate="2019-12-28T20:39:43.183" Score="1" ViewCount="33" Body="&lt;p&gt;I'm looking for a supervized system/approach, that could learn how to categorize incoming texts/documents, where new categories can be added over time and the training set will be small. The trained model should not be static and should be able to evolve with adding new categories or evaluating new documents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each document it should first give it's suggestion that can be then corrected.&lt;/p&gt;&#xA;" OwnerUserId="32382" LastActivityDate="2019-12-30T05:34:16.813" Title="Categorizing text into dynamic amount of categories" Tags="&lt;categorical-data&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17290" PostTypeId="1" CreationDate="2019-12-29T00:43:33.343" Score="1" ViewCount="28" Body="&lt;p&gt;There seems to be a severe problem with the taxonomy of neural network topologies.  What I'd like to know is the term I should use to search for the most general topology:  completely connected directed cyclic graph (henceforth CCDCGRNN).  This is because all other topologies degenerate by constraint from CCDCGRNN.  This includes topologies that are often confused with CCDCGRNN such as Elfman and Jordan networks* and more legitimately-so than, say LSTMs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there are  &lt;a href=&quot;https://stats.stackexchange.com/questions/210111/recurrent-neural-network-rnn-topology-why-always-fully-connected&quot;&gt;&lt;em&gt;claims&lt;/em&gt; such as this question at stats.stackexchange.com (including cites)&lt;/a&gt; that unqualified &quot;RNN&quot; refers to CCDCGRNN but this is not true if one looks a little deeper. Examples include not only &lt;a href=&quot;https://en.wikipedia.org/wiki/Recurrent_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia article on &quot;RNN&quot; (who trusts WP anyway, right?)&lt;/a&gt;, but a &lt;a href=&quot;https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks-explained-3fb6f2367464&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;mostly complete&quot; catalog of neural network topologies&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There must have been, at some point in the ancient past, research into the methods by which one can, in a principled manner, degenerate the CCDCGRNs or at least why it isn't worth studying in its own right. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;*RNNs containing feed-through time delays are a degenerate case of CCDCGRNNs where a time delay of N out of a node is accomplished by allocating N neurons constrained to have only one input with weight of 1 (and a linear transfer function with slope 1).&lt;/p&gt;&#xA;" OwnerUserId="26053" LastEditorUserId="26053" LastEditDate="2019-12-29T00:49:25.277" LastActivityDate="2019-12-29T00:49:25.277" Title="What is the term for an RNN that is a completely connected directed graph?" Tags="&lt;recurrent-neural-networks&gt;&lt;terminology&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17291" PostTypeId="1" CreationDate="2019-12-29T07:19:22.457" Score="5" ViewCount="195" Body="&lt;p&gt;Are PAC learning and VC dimension relevant to machine learning in practice? If yes, what is their practical value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my understanding, there are two hits against these theories. The first is that the results all are conditioned on knowing the appropriate models to use, for example, the degree of complexity. The second is that the bounds are very bad, where a deep learning network would take an astronomical amount of data to reach said bounds.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2019-12-29T12:15:21.577" LastActivityDate="2020-03-11T04:40:24.590" Title="Are PAC learning and VC dimension relevant to machine learning in practice?" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17293" PostTypeId="1" CreationDate="2019-12-29T16:41:35.473" Score="1" ViewCount="25" Body="&lt;p&gt;I have created a CNN for use on the MNIST dataset for now (so I have 10 classes). I have trained SVMs on the sublayers of this trained CNN and wish to combine them into a combined SVM as to give a combined score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I trained two individual SVMs at two of the sublayers of my neural network. &#xA;What is the best method I can go about combining the two SVMs and what are the different options available to me? Is it simply a case of taking the maximum/average of each SVM prediction for a class and using that as the score for the combined SVM class prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="29877" LastActivityDate="2019-12-29T16:41:35.473" Title="Is it possible to combine multiple SVMs that were trained on sublayers of a CNN into one combined SVM?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17294" PostTypeId="1" CreationDate="2019-12-29T16:45:25.080" Score="2" ViewCount="40" Body="&lt;p&gt;I created a multi-label classification CNN to classify chest X-ray images into zero or more possible lung diseases. I've been doing some configuration tests on it and analyzing its results and I'm having a hard time understanding some things about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, these are the graphs that I got for different configurations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://imgur.com/a/GJtogsa&quot; rel=&quot;nofollow noreferrer&quot;&gt;Results of CNN with different configurations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 1: I've only changed the dataset size and the number of color channels in each configuration&#xA;Note 2: In case you're wondering why I tested the network with both 1 and 3 color channels, it's because the images are technically grayscale, but I am using the AlexNet architecture, which was made to take as input 224 x 224 images with 3 channels, so I wanted to see if the network somehow performed better with 3 channels instead of just the one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the things about it I don't understand:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why does the sensitivity and specificity of the network vary so much between different epochs?&lt;/li&gt;&#xA;&lt;li&gt;Is it normal for the validation loss of the network barely ever change as the number of epochs increase?&lt;/li&gt;&#xA;&lt;li&gt;Looking at the results I got, it looks like 2 epochs is where there tends to be the best results. Does that make sense? I've heard of people training their networks with dozens of epochs sometimes.&lt;/li&gt;&#xA;&lt;li&gt;Why is it that, many times, when the sensitivity of the network increases between epochs, the specificity tends to decrease, and vice-versa?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sorry if some of these questions are dumb, I'm still a newbie at this. Also, my total dataset is drastically larger than what I present in these results (~110,000 images). I just haven't done tests with more images due to the time the network takes to train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Network Architecture:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Base Architecture: AlexNet&lt;/li&gt;&#xA;&lt;li&gt;Loss Function: Sigmoid Cross-Entropy Loss&lt;/li&gt;&#xA;&lt;li&gt;Optimizer: Adam Optimization Algorithm with learning rate of 0.001&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;EDIT: I forgot to mention that the number of diseases to predict is 15, and that the network sees 0's much more than 1's due to the imbalance of classes. I've considered changing the loss function to a weighted version of sigmoid cross-entropy because of that, but I'm not sure if that would help the network much.&lt;/p&gt;&#xA;" OwnerUserId="32395" LastActivityDate="2019-12-29T16:45:25.080" Title="How to understand my CNN's training results?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17295" PostTypeId="1" AcceptedAnswerId="17297" CreationDate="2019-12-29T17:45:21.737" Score="0" ViewCount="62" Body="&lt;p&gt;The gradient descent step is the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{W}_i = \mathbf{W}_{i-1} - \alpha * \nabla L(\mathbf{W}_{i-1})&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;were &lt;span class=&quot;math-container&quot;&gt;$L(\mathbf{W}_{i-1})$&lt;/span&gt; is the loss value, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; the learning rate and &lt;span class=&quot;math-container&quot;&gt;$\nabla L(\mathbf{W}_{i-1})$&lt;/span&gt; the gradient of the loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how do we get to the &lt;span class=&quot;math-container&quot;&gt;$L(\mathbf{W}_{i-1})$&lt;/span&gt; to calculate the gradient of &lt;span class=&quot;math-container&quot;&gt;$L(\mathbf{W}_{i-1})$&lt;/span&gt;? As an example, we can initialize the set of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; to 0.5. How can you explain it to me?&lt;/p&gt;&#xA;" OwnerUserId="31143" LastEditorUserId="2444" LastEditDate="2019-12-29T18:52:22.500" LastActivityDate="2019-12-29T18:57:12.543" Title="How is the loss value calculated in order to compute the gradient?" Tags="&lt;deep-learning&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17296" PostTypeId="2" ParentId="11992" CreationDate="2019-12-29T17:50:54.533" Score="0" Body="&lt;p&gt;The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.&#xA;&lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/viewPaper/12389&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/viewPaper/12389&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32396" LastActivityDate="2019-12-29T17:50:54.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17297" PostTypeId="2" ParentId="17295" CreationDate="2019-12-29T18:52:02.317" Score="2" Body="&lt;p&gt;In your case, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the loss (or cost) function, which can be, for example, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean squared error (MSE)&lt;/a&gt; or the cross-entropy, depending on the problem you want to solve. Given one training example &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}_i, y_i) \in D$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i \in \mathbb{R}^d$&lt;/span&gt; is the input (for example, an image) and &lt;span class=&quot;math-container&quot;&gt;$y_i \in \mathbb{R}$&lt;/span&gt; can either be a label (aka class) or a numerical value, and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is your training dataset, then the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;MSE&lt;/a&gt; is defined as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(\mathbf{W}) = \frac{1}{2} \left(f(\mathbf{x}_i) - y_i \right)^2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x}_i) \in \mathbb{R}$&lt;/span&gt; is the output of the neural network &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; given the input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a mini-batch of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; training examples &lt;span class=&quot;math-container&quot;&gt;$\{(\mathbf{x}_i, y_i) \}_{i=1}^M$&lt;/span&gt;, then the loss will be an average of the MSE for each training example. For more info, have a look at this answer &lt;a href=&quot;https://ai.stackexchange.com/a/11675/2444&quot;&gt;https://ai.stackexchange.com/a/11675/2444&lt;/a&gt;. The &lt;a href=&quot;https://ai.stackexchange.com/a/8985/2444&quot;&gt;https://ai.stackexchange.com/a/8985/2444&lt;/a&gt; may also be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the article &lt;a href=&quot;https://machinelearningmastery.com/loss-and-loss-functions-for-training-deep-learning-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loss and Loss Functions for Training Deep Learning Neural Networks&lt;/a&gt; for more info regarding different losses used in deep learning and how to choose the appropriate loss for your problem.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2019-12-29T18:57:12.543" LastActivityDate="2019-12-29T18:57:12.543" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17298" PostTypeId="1" AcceptedAnswerId="17299" CreationDate="2019-12-30T00:06:13.063" Score="4" ViewCount="1198" Body="&lt;p&gt;I'm new to machine learning (so excuse my nomenclature), and not being a python developer, I decided to jump in at the deep (no pun intended) end writing my own framework in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my current design I have given each neuron/cell the possiblity to have a different activation function. Is this a plausible design for a neural network? A lot of the examples I see use the same activation function for all neurons in a given layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a model which may require this, or should all neurons in a layer use the same activation function? Would I be correct in using different activation functions for different layers in the same model, or would all layers have the same activation function within a model?&lt;/p&gt;&#xA;" OwnerUserId="32400" LastEditorUserId="32400" LastEditDate="2019-12-30T00:38:23.690" LastActivityDate="2019-12-30T00:38:23.690" Title="Do all neurons in a layer have the same activation function?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17299" PostTypeId="2" ParentId="17298" CreationDate="2019-12-30T00:27:56.933" Score="1" Body="&lt;p&gt;From &lt;a href=&quot;https://www.quora.com/Why-does-every-node-in-a-layer-use-the-same-activation-function&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using other activation functions don’t provide significant improvement in performance and tweaking them doesn’t provide any big improvement. So as per simplicity we use same activation function for most of the case in Deep Neural Networks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2019-12-30T00:27:56.933" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17300" PostTypeId="1" CreationDate="2019-12-30T01:29:40.600" Score="1" ViewCount="33" Body="&lt;p&gt;I am optimising hyperparameters for my deep reinforcement learning project (using PPO2, DQN and A2C) and was wondering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I find the optimum hyperparameters to get maximum reward from training over my entire range of training (e.g. 50 million steps) or can I optimise over less time (e.g. 1 million steps)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the conventional approach and why?&lt;/p&gt;&#xA;" OwnerUserId="32401" LastActivityDate="2019-12-30T01:29:40.600" Title="Hyperparameter optimisation over entire range or shorter range of training episodes in Deep Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17301" PostTypeId="2" ParentId="17240" CreationDate="2019-12-30T04:28:23.123" Score="1" Body="&lt;p&gt;Here's a link to my answer on &lt;em&gt;CV Stack Exchange&lt;/em&gt;, where I have mentioned about latent spaces and some deep learning models that learn these representations: &lt;a href=&quot;https://stats.stackexchange.com/questions/442352/what-is-a-latent-space/442360#442360&quot;&gt;https://stats.stackexchange.com/questions/442352/what-is-a-latent-space/442360#442360&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, deep learning models for &lt;a href=&quot;https://paperswithcode.com/task/domain-adaptation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Domain Adaptation&lt;/a&gt;, &lt;a href=&quot;https://paperswithcode.com/area/computer-vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Vision,&lt;/a&gt; &lt;a href=&quot;https://paperswithcode.com/area/natural-language-processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Processing,&lt;/a&gt; &lt;a href=&quot;https://paperswithcode.com/task/recommendation-systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recommendation Systems,&lt;/a&gt; &lt;a href=&quot;https://paperswithcode.com/area/music&quot; rel=&quot;nofollow noreferrer&quot;&gt;Music&lt;/a&gt;/&lt;a href=&quot;https://paperswithcode.com/area/speech&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speech&lt;/a&gt;/&lt;a href=&quot;https://paperswithcode.com/area/audio&quot; rel=&quot;nofollow noreferrer&quot;&gt;Audio&lt;/a&gt; processing, &lt;a href=&quot;https://paperswithcode.com/area/adversarial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial models&lt;/a&gt;, etc., &lt;strong&gt;all&lt;/strong&gt; learn some form of latent representation of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, any place we're learning a function to map input and output spaces of a dataset, the model essentially learns a latent representation of data irrespective of whether the model is based on deep neural networks or a &lt;a href=&quot;https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic method&lt;/a&gt; or any other.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2020-01-07T04:12:03.993" LastActivityDate="2020-01-07T04:12:03.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17302" PostTypeId="2" ParentId="17289" CreationDate="2019-12-30T05:34:16.813" Score="2" Body="&lt;p&gt;&lt;strong&gt;Transfer Learning&lt;/strong&gt; allows you to add new categories to be predicted to the output layer without needing to re-train the entire model every time a new category needs to be classified. Rather, the weights of all initial layers up to the last few layers of the network can be &lt;em&gt;frozen&lt;/em&gt; and only the last one or two layers can be made &lt;em&gt;trainable&lt;/em&gt; for &lt;em&gt;fine-tuning&lt;/em&gt; the classification rather than training from scratch. This approach would be relatively fast than other methods since the dataset in your case is small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for model to flag the need of a new output class by inferring on a test document, you could include a class &quot;Unknown&quot; in addition to existing &lt;code&gt;N&lt;/code&gt; classes (hence, the output layer now contains &lt;code&gt;N+1&lt;/code&gt; classes). If the model predicts &quot;Unknown&quot; with highest probability, you could add a new class to the output layer after examining the test data.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastActivityDate="2019-12-30T05:34:16.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17304" PostTypeId="1" AcceptedAnswerId="17356" CreationDate="2019-12-30T21:32:37.797" Score="3" ViewCount="157" Body="&lt;p&gt;I'm reading &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0957417418307267&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and it says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper, we present a multi-class embedded feature selection method called as sparse optimal scoring with adjustment (SOSA), which is capable of addressing the data heterogeneity issue. We propose to perform feature selection on the adjusted data obtained by estimating and removing the unknown data heterogeneity from original data. Our feature selection is formulated as a sparse optimal scoring problem by imposing &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$\ell_{2, 1}$&lt;/span&gt;-norm regularization&lt;/strong&gt; on the coefficient matrix which hence can be solved effectively by proximal gradient algorithm. This allows our method can well handle the multi-class feature selection and classification simultaneously for heterogenous data&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is the &lt;span class=&quot;math-container&quot;&gt;$\ell_{2, 1}$&lt;/span&gt; norm regularization? Is it L1 regularization or L2 regularization?&lt;/p&gt;&#xA;" OwnerUserId="32416" LastEditorUserId="32416" LastEditDate="2020-01-01T13:17:37.033" LastActivityDate="2020-01-03T19:08:31.317" Title="What is the $\ell_{2, 1}$ norm?" Tags="&lt;machine-learning&gt;&lt;feature-selection&gt;&lt;regularization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17305" PostTypeId="1" CreationDate="2019-12-30T22:15:26.027" Score="1" ViewCount="38" Body="&lt;p&gt;I'm evaluating the performance and accuracy in detecting objects for my data set using three deep learning algorithms. In total there are 24,085 images. I measure the performance in terms of time taken to detect the objects. To measure the accuracy, I manually count the number of objects in each image and then calculate recall and precision values for three algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since I'm manually counting to get actual object count, I selected only 30 images. Will that sample be enough to make a conclusion that algorithm 1 is better than others in terms performance and accuracy?&lt;/p&gt;&#xA;" OwnerUserId="32343" LastEditorUserId="32343" LastEditDate="2019-12-30T22:27:01.553" LastActivityDate="2019-12-30T22:27:01.553" Title="Sample size for the evaluation of Deep Learning Models" Tags="&lt;deep-learning&gt;&lt;performance&gt;&lt;accuracy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17306" PostTypeId="1" CreationDate="2019-12-30T22:16:19.583" Score="1" ViewCount="103" Body="&lt;p&gt;Is there an AI technology out there or being developed that can predict human behaviour, given that we as humans are irrational decision-makers?&lt;/p&gt;&#xA;&lt;p&gt;I'm looking at this from an economic standpoint - the issue with current economic models is that they assume that humans are perfectly rational, but obviously this isn't the case. Could AI develop better models and therefore produce better models of recessions?&lt;/p&gt;&#xA;" OwnerUserId="32419" LastEditorUserId="2444" LastEditDate="2020-09-11T14:40:21.857" LastActivityDate="2020-10-11T18:51:13.750" Title="Is there an AI technology that can predict human behaviour?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17307" PostTypeId="1" AcceptedAnswerId="17308" CreationDate="2019-12-31T05:05:16.190" Score="3" ViewCount="70" Body="&lt;p&gt;I am trying to replace last fully connected layer of size 4096/2048 with a matrix of size 100x300 with previous fc layer output of 2048.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/a8S6z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a8S6z.png&quot; alt=&quot;something of this sort : something of this sort&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;2D convolution - to map from 2048 --&gt; 100x300 (Which is not realizable)&lt;/li&gt;&#xA;&lt;li&gt;Intermediate projections :&lt;br&gt;&#xA;    2048 --&gt; 100&lt;br&gt;&#xA;    [100x1] X [1x300] --&gt; [100x300] (possible but complicated)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am looking for a simple and effective solution with least linear transformations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0pM8S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0pM8S.png&quot; alt=&quot;This is desired&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="25676" LastEditDate="2020-01-03T04:17:59.830" LastActivityDate="2020-01-03T04:17:59.830" Title="How to create a fully connected(matrix) layer with vector input" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17308" PostTypeId="2" ParentId="17307" CreationDate="2019-12-31T07:12:18.073" Score="4" Body="&lt;p&gt;You can use &lt;code&gt;tf.reshape()&lt;/code&gt; method (&lt;a href=&quot;https://www.tensorflow.org/versions/r1.14/api_docs/python/tf/reshape&quot; rel=&quot;nofollow noreferrer&quot;&gt;tensorflow doc&lt;/a&gt;) to reshape &lt;code&gt;(2048)&lt;/code&gt; dimensional tensor to &lt;code&gt;(100,300)&lt;/code&gt;. Here's one way to do this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input1 = tf.reshape(input1, [100,300], name=&quot;reshaped_tensor&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you're not using TensorFlow but using Numpy, here's an implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input1 = np.array(input1)&#xA;input1 = np.reshape(input1, (100,300))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: You might want to follow up this layer with &lt;code&gt;tf.nn.conv2d&lt;/code&gt; layers to &quot;densify&quot; the sparse matrix/values obtained from the above step.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastActivityDate="2019-12-31T07:12:18.073" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17309" PostTypeId="1" CreationDate="2019-12-31T10:45:07.223" Score="2" ViewCount="126" Body="&lt;p&gt;I'm trying to build autoencoder in keras in order to detect anomalies. However, most of the data is categorical and I have to encode it. When it comes to production, categorical features can take new values. So I thought of using entity embedding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've created an embedding layer for each categorical feature. In the code below, I concatenate all together and I add other Dense layers:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;output = Concatenate()(output_embeddings)&#xA;&#xA;encoder = Dense(20, activation=&quot;relu&quot;, activity_regularizer=regularizers.l1(10e-5))(output)&#xA;&#xA;encoder= Dense(nbr_feautres, activation='relu')(encoder)&#xA;&#xA;decoder = Dense( ???? )(encoder)&#xA;&#xA;model = Model(inputs=input_models, outputs=decoder)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it possible to add a decoder layer and how can I implement this? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="32426" LastEditorUserId="32426" LastEditDate="2020-01-02T16:13:15.620" LastActivityDate="2020-01-02T16:13:15.620" Title="Is it possible to use entity embedding with autoencoder for anomaly detetction?" Tags="&lt;python&gt;&lt;keras&gt;&lt;autoencoders&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17310" PostTypeId="1" CreationDate="2019-12-31T13:22:53.577" Score="5" ViewCount="131" Body="&lt;p&gt;I'm running some distributed trainings in Tensorflow with Horovod. It runs training separately on multiple workers, each of which uses the same weights and does forward pass on unique data. Computed gradients are averaged within the communicator (worker group) before applying them in weight updates. I'm wondering - why not average the loss function across the workers? What's the difference (and the potential benefits) of averaging gradients?&lt;/p&gt;&#xA;" OwnerUserId="32431" LastActivityDate="2020-02-02T22:01:50.203" Title="Why do we average gradients and not loss in distributed training?" Tags="&lt;tensorflow&gt;&lt;distributed-computing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17311" PostTypeId="1" AcceptedAnswerId="17312" CreationDate="2019-12-31T14:31:32.010" Score="4" ViewCount="123" Body="&lt;p&gt;In chapter 10 of Sutton and Barto's book (2nd edition) is given the equation for TD(0) error with average reward (equation 10.10):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\delta_t = R_{t+1} - \bar{R} + \hat{v}(S_{t+1}, \mathbf{w}) - \hat{v}(S_{t}, \mathbf{w})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the intuition behind this equation? And how exactly is it derived? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in chapter 13, section 6, is given the Actor-Critic algorithm, which uses the TD error. How can you use 1 error to update 3 distinct things - like the average reward, value function estimator (critic), and the policy function estimator (actor)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Average Reward update rule&lt;/strong&gt;: &lt;span class=&quot;math-container&quot;&gt;$\bar{R} \leftarrow \bar{R} + \alpha^{\bar{R}}\delta$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Critic weight update rule&lt;/strong&gt;: &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} \leftarrow \mathbf{w} + \alpha^{\mathbf{w}}\delta\nabla \hat{v}(s,\mathbf{w})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Actor weight update rule&lt;/strong&gt;: &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta} \leftarrow \mathbf{\theta} + \alpha^{\mathbf{\theta}}\delta\nabla ln \pi(A|S,\mathbf{\theta})$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="27947" LastEditorUserId="2444" LastEditDate="2020-01-01T14:03:17.480" LastActivityDate="2020-01-01T14:03:17.480" Title="What is the intuition behind the TD(0) equation with average reward, and how is it derived?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17312" PostTypeId="2" ParentId="17311" CreationDate="2019-12-31T15:25:53.880" Score="4" Body="&lt;p&gt;This is simply from definition of return in average reward setting (look at equation &lt;span class=&quot;math-container&quot;&gt;$10.9$&lt;/span&gt;). The &quot;standard&quot; TD error is defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;TD_{\text{error}} = R_{t+1} + V(S_{t+1}) - V(S_t)&#xA;\end{equation}&lt;/span&gt;&#xA;In average reward setting, average reward &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt; is subtracted from reward at &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;, so TD error in this case is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;TD_{\text{error}} = R_{t+1} - \bar R_{t+1} + V(S_{t+1}) - V(S_t)&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\bar R_{t+1}$&lt;/span&gt; is estimate of &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;span class=&quot;math-container&quot;&gt;$\delta_t$&lt;/span&gt; in all 3 updates because neither of these updates depend on each other. For example if you update &lt;span class=&quot;math-container&quot;&gt;$\mathbf w$&lt;/span&gt; you don't use that then to update &lt;span class=&quot;math-container&quot;&gt;$\mathbf \theta$&lt;/span&gt;, or if you update &lt;span class=&quot;math-container&quot;&gt;$\bar R$&lt;/span&gt; you don't use updated version to update &lt;span class=&quot;math-container&quot;&gt;$\mathbf w$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\mathbf \theta$&lt;/span&gt; so you're not introducing additional bias. In each separate update you also don't have &lt;span class=&quot;math-container&quot;&gt;$\delta_t$&lt;/span&gt; present multiple times so that you require multiple sampling per timestep to get the unbiased update.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally this is semi-gradient algorithm, it uses bootstrapped estimate &lt;span class=&quot;math-container&quot;&gt;$V_{t+1}$&lt;/span&gt; but it doesn't calculate full derivative with respect to it aswell, only with respect to &lt;span class=&quot;math-container&quot;&gt;$V_t$&lt;/span&gt; so the algorithm is biased by default, but works well enough in practice for linear case.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2020-01-01T10:36:27.310" LastActivityDate="2020-01-01T10:36:27.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17313" PostTypeId="1" CreationDate="2019-12-31T16:57:04.040" Score="3" ViewCount="106" Body="&lt;p&gt;I want a microphone to pick up sounds around me (let's say beyond a 3 foot radius), but ignore sounds made at my desk, such as the rustling of paper, clicking a mouse and typing, my hands brushing up on the table, putting a pen down, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How hard would it be for AI to be able to distinguish these sounds from surrounding sounds, such as someone knocking on my door or a random loud sound from further away? How would you implement this? Is it possible that a pre-trained model could accomplish this, and work reliably for most people at their desk? I don't have any experience in AI.&lt;/p&gt;&#xA;" OwnerUserId="32433" LastActivityDate="2021-05-27T00:01:20.273" Title="How difficult is this sound classification?" Tags="&lt;classification&gt;&lt;training&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17314" PostTypeId="1" CreationDate="2019-12-31T17:29:51.110" Score="2" ViewCount="31" Body="&lt;p&gt;Which hyper-parameters of a convolutional neural network are likely to be the most sensitive to depending on whether the training (and test and inference) data involves only accurately centered images versus off-centered images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More convolutional layers, wider convolution kernels, more dense layers, wider dense layers, more or less pooling, or ???&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. If I can preprocess the data to include only accurately centered images, which hyper-parameters should I experiment with changing to create a smaller CNN model (for a power and memory constrained inference engine)?  Or conversely, if I have a minimized model trained on centered data, which hyper-parameters would I most likely need to increase to get similar loss and accuracy on uncentered (shifted in XY) data?&lt;/p&gt;&#xA;" OwnerUserId="2918" LastActivityDate="2019-12-31T17:29:51.110" Title="Which CNN hyper-parameters are most sensitive to centered versus off centered data?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;hyper-parameters&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17316" PostTypeId="1" CreationDate="2019-12-31T21:27:18.680" Score="3" ViewCount="63" Body="&lt;p&gt;This question was inspired by watching AlphaStar play Starcraft 2, but I'm also interested in the concept in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does the AI decide what build order to start with?  In Starcraft, and many other games, the player must decide what strategy or class of strategies to follow as soon as the game begins.  To use a Starcraft-specific example, one must decide to 6-pool Zerg Rush before any scouting information has been gathered.  Delaying the rush to wait for info means the opponent will be stronger when the rush arrives; the opponent may even discover the rush and prepare a dedicated counter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not limited to deciding between a risky early all-or-nothing attack.  Some long-term strategies also preclude others.  Terran players must decide early on how heavily they will invest in mech units.  They can focus on biological units like marines, or vehicular units like siege tanks and hellions.  Going equally into both, however, often means a weaker army overall, because you must spend resources on the overhead costs of both tech trees.  You must upgrade your vehicle weapons as well as your infantry weapons for instance, meaning less resources can be spent on more units.  Suffice to say, Terran players usually must decide very early on what they will focus on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can AI make these kinds of choices given incomplete and often uncertain information?&lt;/p&gt;&#xA;" OwnerUserId="32435" LastActivityDate="2019-12-31T21:27:18.680" Title="How do AI that play games of incomplete information decide their opening strategy?" Tags="&lt;gaming&gt;&lt;incomplete-information&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17317" PostTypeId="1" CreationDate="2019-12-31T23:12:42.903" Score="8" ViewCount="235" Body="&lt;p&gt;Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they seem particularly apt at learning the right parameters. However, something we often have to combat when training neural networks is &lt;em&gt;overtfitting&lt;/em&gt; - reproducing the training data and not generalizing to a validation set. The solution to overfitting is usually to simply add more data, with the rationalization that at a certain point the neural network pretty much has no choice but to learn the correct function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this never made much sense to me. There is no reason, in terms of loss, that a neural network should prefer a function that generalizes well (i.e. the function you are looking for) over a function that does incredibly well on the training data and fails miserably everywhere else. In fact, there is usually a loss &lt;em&gt;advantage&lt;/em&gt; to overfitting. Equally, there is an infinite number of functions that fit the training data and have no success on anything but. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why is it that neural networks almost always (especially for simpler data) stumble upon the function we want, as opposed to one of the infinite other options? Why is it that neural networks are good at generalizing, when there is no incentive for them to?&lt;/p&gt;&#xA;" OwnerUserId="25415" LastEditorUserId="2444" LastEditDate="2019-12-31T23:35:18.417" LastActivityDate="2020-01-04T21:42:39.300" Title="Why can neural networks generalize at all?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;overfitting&gt;&lt;computational-learning-theory&gt;&lt;generalization&gt;" AnswerCount="3" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17318" PostTypeId="2" ParentId="17317" CreationDate="2020-01-01T00:38:51.770" Score="0" Body="&lt;p&gt;There is normally more to generalisation than just increasing training data. It helps to make the task noisy, through various means. One common and popular method is to use dropout, which encourages the network to utilise every node, and avoids dependencies on small clusters of nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how does making the task &lt;em&gt;more&lt;/em&gt; noisy help with generalisation? Well it's easy to explain with a conceptual example, but I don't think that's what you're looking for, rather a more mathematical approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to think of it is with a simple example of a polynomial data set, in only 2 dimensions. If you consider this, and the way the network slowly approaches optimums through back propagation, the concept that the classification boundary gradually approaches the optimum, which in all cases is an over-fit function, isn't too far fetched.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now considering this, this would suggest that in order to properly train a network, we would need some way of determining when the network isn't super close to the optimum (as it will have over-fit by then) but also isn't so far from it that it's worse than randomly picking. This is were methods to improve generalisation come in, we want to hit that sweet spot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the learning rate is too high, we will overshoot that sweet spot and miss out entirely (the range can sometimes be very small), if it's too low, we may get stuck in tiny local minimums and never escape, or it could take years to reach it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the example of dropout from before, this increases noise, and makes training harder on the network. Due to more difficult training, the network approaches the optimal function at a slower rate, and makes it easier to cut training when the network has generalised well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conveniently, this extends to n-dimensional problems as well. Now, as far as I know, there is no robust mathematical proof of why this works for n-dimensional problems. The reason for this, explains why neural networks even exist: &lt;strong&gt;We don't know how to mathematically classify these issues to the degree a NN does ourselves&lt;/strong&gt;. Because of this, there will &lt;em&gt;always&lt;/em&gt; be gaps in our knowledge. We will never be able to quantitatively say what a neural network is doing, without making them obsolete. So until we can figure it out for ourselves, we'll have to just perform testing against unseen data to see if the network has indeed generalised.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="26726" LastEditDate="2020-01-01T02:12:47.043" LastActivityDate="2020-01-01T02:12:47.043" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17319" PostTypeId="5" CreationDate="2020-01-01T04:06:15.200" Score="0" Body="&lt;p&gt;The mathematical basis of neural networks and image manipulation are similar, parallel tasks involving matrices, leading GPUs to become increasingly used for machine learning tasks. As of 2016, GPUs are popular for AI work, and they continue to evolve in a direction to facilitate deep learning, both for training[24] and inference in devices such as self-driving cars. GPU developers are developing additional connective capability for the kind of dataflow workloads AI benefits from. As GPUs have been increasingly applied to AI acceleration, GPU manufacturers have incorporated neural network specific hardware to further accelerate these tasks. Tensor cores are intended to speed up the training of neural networks. &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_accelerator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia Reference&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2020-01-03T20:39:18.823" LastActivityDate="2020-01-03T20:39:18.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17320" PostTypeId="4" CreationDate="2020-01-01T04:06:15.200" Score="0" Body="Graphics processing units or GPUs are specialized hardware for the manipulation of images and calculation of local image properties." OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2020-01-03T20:35:25.907" LastActivityDate="2020-01-03T20:35:25.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17321" PostTypeId="5" CreationDate="2020-01-01T04:13:26.610" Score="0" Body="&lt;p&gt;Batch normalization was introduced in a 2015 paper &lt;a href=&quot;https://en.wikipedia.org/wiki/Batch_normalization#cite_note-:0-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Batch_normalization#cite_note-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;. &#xA;While the effect of batch normalization is evident, the reasons behind its effectiveness remain under discussion. It was believed that it can mitigate the problem of internal covariate shift, where parameter initialization and changes in the distribution of the inputs of each layer affects the learning rate of the network.&lt;a href=&quot;https://en.wikipedia.org/wiki/Batch_normalization#cite_note-:0-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; Recently, some scholars have shown that batch normalization does not reduce internal covariate shift, but rather smooths the objective function to improve the performance.&lt;a href=&quot;https://en.wikipedia.org/wiki/Batch_normalization#cite_note-:1-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt; Others prove that batch normalization achieves length-direction decoupling, and thereby accelerates neural networks &lt;a href=&quot;https://en.wikipedia.org/wiki/Batch_normalization#cite_note-:2-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;5&lt;/a&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Batch_normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia Reference&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2020-01-03T20:39:44.647" LastActivityDate="2020-01-03T20:39:44.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17322" PostTypeId="4" CreationDate="2020-01-01T04:13:26.610" Score="0" Body="Batch normalization is a technique for improving the speed, performance, and stability of artificial neural networks. Batch normalization is used to normalize the input layer by adjusting and scaling the activations." OwnerUserId="30644" LastEditorUserId="30644" LastEditDate="2020-01-03T20:39:38.727" LastActivityDate="2020-01-03T20:39:38.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17323" PostTypeId="1" CreationDate="2020-01-01T07:55:59.560" Score="1" ViewCount="27" Body="&lt;p&gt;I come from a background of scorecard development using logistic regression. Steps involved there are:&#xA;1. binning of continuous variables into intervals (eg age can be binned into 10-15 years, 15-20 years, etc)&#xA;2. weight of evidence transformation&#xA;3. coarse classing of bins to ensure event rate has a monotonic relationship with the variable&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variable selection is made from the coarse classed transformed variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if I should follow the same steps for ANN models. That is, should I bin the continuous variables and apply some transformation before performing variable selection and modelling.&lt;/p&gt;&#xA;" OwnerUserId="32439" LastActivityDate="2020-01-01T07:55:59.560" Title="Variable binning for NN" Tags="&lt;classification&gt;&lt;getting-started&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17324" PostTypeId="1" CreationDate="2020-01-01T21:01:39.787" Score="2" ViewCount="105" Body="&lt;p&gt;Let's assume an extreme case in which the kernel of the convolution layer takes only values 0 or 1. To capture all possible patterns in input of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; number of channels, we need &lt;span class=&quot;math-container&quot;&gt;$2^{C*K_H*K_W}$&lt;/span&gt; filters, where &lt;span class=&quot;math-container&quot;&gt;$(K_H, K_W)$&lt;/span&gt; is the shape of a kernel. So to process a standard RGB image with 3 input channels with 3x3 kernel, we need our layer to output &lt;span class=&quot;math-container&quot;&gt;$2^{27}$&lt;/span&gt; channels. Do I correctly conclude that according to this, the standard layers of 64 to 1024 filters are only able to catch a small part of (perhaps) useful patterns?&lt;/p&gt;&#xA;" OwnerUserId="31988" LastEditorUserId="22659" LastEditDate="2020-06-01T12:22:42.997" LastActivityDate="2020-06-01T12:22:42.997" Title="What is the reasoning behind the number of filters in the convolution layer?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17325" PostTypeId="1" CreationDate="2020-01-01T21:31:28.720" Score="3" ViewCount="193" Body="&lt;p&gt;Just wondering why a softmax is typically used in practice on outputs of most neural nets rather than just summing the activations and dividing each activation by the sum. I know it's roughly the same thing but what is the mathematical reasoning behind a softmax over just a normal summation? Is it better in some way?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastActivityDate="2020-01-02T10:57:57.357" Title="Why is a softmax used rather than dividing each activation by the sum?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17326" PostTypeId="1" AcceptedAnswerId="17336" CreationDate="2020-01-02T03:11:39.317" Score="3" ViewCount="238" Body="&lt;p&gt;A LSTM model can be trained to generate text sequences by feeding the first word. After feeding the first word, the model will generate a sequence of words (a sentence). Feed the first word to get the second word, feed the first word + the second word to get the third word, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, about the next sentence, &lt;strong&gt;what should be the next first word&lt;/strong&gt;? The thing is to generate a paragraph of multiple sentences.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-01-03T11:06:41.697" Title="How to use LSTM to generate a paragraph" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;text-classification&gt;&lt;text-generation&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17328" PostTypeId="2" ParentId="17325" CreationDate="2020-01-02T10:57:57.357" Score="1" Body="&lt;p&gt;There are probably multiple different explanations and reasonings, but I can offer you one. If your output vector contains negative values, to get something that's related to probabilities (all components positive, summing to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) you cannot do what you suggested because you can possibly get a negative probability which doesn't make sense. Good property of exponential function used in softmax, in this case, is that it cannot give negative values, so regradless of your output vector, you will never get negative probability.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could suggest adding some positive offset vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf d$&lt;/span&gt; to your output vector to get rid of negative values but there are couple of problems. First, you cannot know in advance your range of negative output values so that you can know what offset vector to use to cover all possible cases. Second, with such strategy in some cases you could get unrealistic results. For example let's assume output vector is &lt;span class=&quot;math-container&quot;&gt;$[-0.1, 0.2, 0.3]^T$&lt;/span&gt; and offset vector is &lt;span class=&quot;math-container&quot;&gt;$[0.1, 0.1, 0.1]^T$&lt;/span&gt;. If you add those 2 you get &lt;span class=&quot;math-container&quot;&gt;$[0, 0.3, 0.4]^T$&lt;/span&gt;. Probability of first class would be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; since the numerator is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. That is very overconfident result and we probably wouldn't want to get the result of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for this class. The result would change depending on the offset vector. Let's say that offset vector now is &lt;span class=&quot;math-container&quot;&gt;$[0.3, 0.3, 0.3]^T$&lt;/span&gt;. Addition of offset vector and output vector gives &lt;span class=&quot;math-container&quot;&gt;$[0.2, 0.5, 0.6]$&lt;/span&gt;. In this case probability of first class is now &lt;span class=&quot;math-container&quot;&gt;$0.2/(0.2 + 0.5 + 0.6) = 0.15$&lt;/span&gt;. We see that changing the offset vector changes the values of probabilities and as components of offset vector &lt;span class=&quot;math-container&quot;&gt;$\rightarrow \infty$&lt;/span&gt; probabilities for all classes &lt;span class=&quot;math-container&quot;&gt;$\rightarrow 0.33$&lt;/span&gt;. We would likely want to get the same result regardless of the scale of the values of the vector, we only care about its relative relationships. Another good property of softmax is that it is shift invariant&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;p_i &amp;amp;= \frac{e^{x_i + d}}{\sum_{j=1}^n e^{x_j + d}}\\&#xA;&amp;amp;= \frac{e^d \cdot e^{x_i}  }{ e^d \cdot \sum_{j=1}^n e^{x_j} }\\&#xA;&amp;amp;= \frac{e^{x_i}}{\sum_{j=1}^n e^{x_j}}&#xA;\end{align}&lt;/span&gt;&#xA;so we see that the probability of &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th component is independent of offset vector. Apparently softmax doesn't care about the scale of values, it manages to capture the relative relationships of the components.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-01-02T10:57:57.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17331" PostTypeId="1" CreationDate="2020-01-02T13:27:14.690" Score="2" ViewCount="42" Body="&lt;p&gt;I’m doing research on natural language processing (NLP). I’d like to put together my own model. However, I'm running into a concept I am not familiar with, namely, distractors. A google search does not reveal much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been reading this article specifically: &lt;a href=&quot;https://medium.com/huggingface/how-to-build-a-state-of-the-art-conversational-ai-with-transfer-learning-2d818ac26313&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/huggingface/how-to-build-a-state-of-the-art-conversational-ai-with-transfer-learning-2d818ac26313&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the section under &quot;Multi-Tasks Losses&quot; it reads:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Next-sentence prediction: we pass the hidden-state of the last token&#xA;  (the end-of-sequence token) through a linear layer to get a score and&#xA;  apply a cross-entropy loss to classify correctly a gold answer among&#xA;  distractors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand how transformers and coss-entropy works, however I'm not sure what a distractor or a &quot;gold answer&quot; is for that matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, what does the author mean by distractor?&lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="20271" LastEditDate="2020-01-02T16:44:50.657" LastActivityDate="2020-01-02T16:44:50.657" Title="What role do distractors play in natural language processing?" Tags="&lt;natural-language-processing&gt;&lt;terminology&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17332" PostTypeId="1" CreationDate="2020-01-02T15:09:29.343" Score="2" ViewCount="85" Body="&lt;p&gt;The reason I am asking this question is because I am about to start a PhD in NLP. So I am wondering if there would be as much job opportunities in research in industry as oppose to in academia in the future (~ 5 to 10 years) or would it be mostly a matter of using a library off the shelf. I have done some research and it seems NLP is AI-complete, which means it's probably a problem that will be &quot;solved&quot; only when AGI is solved , but still I would appreciate any input.&lt;/p&gt;&#xA;" OwnerUserId="32462" LastEditorUserId="32462" LastEditDate="2020-01-03T21:14:13.087" LastActivityDate="2020-01-03T21:14:13.087" Title="Is NLP likely to be sufficiently solved in the next few years?" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17333" PostTypeId="1" CreationDate="2020-01-02T16:08:58.540" Score="2" ViewCount="313" Body="&lt;p&gt;Given an audio track, I'm trying to find a way to recognize the audio language. Only within a small set (e.g. English vs Spanish). Is there a simple solution to detect the language in a speech?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2020-01-04T17:07:02.937" LastActivityDate="2020-01-04T17:07:02.937" Title="How to use AI for language recognition?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;audio-processing&gt;&lt;speech-recognition&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17334" PostTypeId="1" AcceptedAnswerId="17542" CreationDate="2020-01-02T17:44:17.477" Score="1" ViewCount="90" Body="&lt;p&gt;The description of feature selection based on a random forest uses trees without pruning.&#xA;Do I need to use tree pruning?&#xA;The thing is, if I don't cut the trees, the forest will retrain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below in the picture is the importance of features based on 500 trees without pruning.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Az5sw.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Az5sw.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a depth of 3.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nKt29.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nKt29.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I always use the last four signs 27, 28, 29, 30.&#xA;And I try to add to them signs from 0 to 26 by means of cycles, going through possible combinations.&#xA;Empirically, I assume that the trait number 0, 26 is significant.&#xA;But, on both pictures it is not visible. Although the quality of classification with the addition of 0, 26 has improved.&lt;/p&gt;&#xA;" OwnerUserId="31808" LastActivityDate="2020-01-16T17:41:15.850" Title="Interpretation of feature selection based on the model" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;python&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17335" PostTypeId="2" ParentId="17333" CreationDate="2020-01-02T21:02:45.213" Score="1" Body="&lt;p&gt;Google has an API you can use. &lt;a href=&quot;https://cloud.google.com/translate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/translate/&lt;/a&gt;. Their API can translate audio to text. They also have an API for converting speech to text. The language detection feature should let you detect the language in the resulting text. They have client libraries for the most popular programming languages. &lt;/p&gt;&#xA;" OwnerUserId="32467" LastActivityDate="2020-01-02T21:02:45.213" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17336" PostTypeId="2" ParentId="17326" CreationDate="2020-01-02T21:07:17.860" Score="3" Body="&lt;p&gt;Take the sentence that was generated by your LSTM and feed it back into the LSTM as input. Then the LSTM will generate the next sentence. So the LSTM is using it's previous output as it's input. That's what makes it recursive. The intial word is just your base case. Also you should consider using GPT2 by open AI to do this. It's pretty impressive. &lt;a href=&quot;https://openai.com/blog/better-language-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/better-language-models/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32467" LastActivityDate="2020-01-02T21:07:17.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17337" PostTypeId="1" CreationDate="2020-01-02T21:12:17.320" Score="3" ViewCount="57" Body="&lt;p&gt;I've been thinking about what &quot;mathematical model&quot; can be used to model every possible thing (including itself).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples: a simple neuron network models a function but doesn't model an algorithm. A list of instructions models an algorithm but doesn't model relations between elements...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be thinking &quot;maybe there is nothing that can model everything&quot; but in reality &quot;language&quot; does model everything including itself. The issue is that it's not an organized model and it's not clear how to create it from scratch (e.g. if you will send it to aliens that don't have any common knowledge to start with).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So what is some possible formalization of a mathematical model that models every possible thought that can be communicated?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The structure formalization I'm looking for has to have a few necessary properties:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Hierarchical&lt;/strong&gt;: the representation of ideas should rely on other ideas. (E.g. an programming function is a set of programming functions, the concept &quot;bottle of water&quot; is sum of two concepts &quot;water&quot; and a &quot;bottle&quot;...)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Uniqueness of elements&lt;/strong&gt;: When an idea uses in its definition another idea, it must refer to one specific idea, not recreate it each time. For example, when you think of a digit &quot;9&quot; and the digit &quot;8&quot;, you notice that both have a small circle at the top, you don't recreate a new concept &quot;circle&quot; every time, instead, you use a fixed concept &quot;circle&quot; for everything. By contrast, a neural network might recreate the same branch for different inputs. So two representations of concepts must be different iff they have a difference.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31723" LastEditorUserId="31723" LastEditDate="2020-01-03T22:29:34.220" LastActivityDate="2020-01-03T22:29:34.220" Title="What can model everything?" Tags="&lt;ai-design&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="17339" PostTypeId="2" ParentId="17324" CreationDate="2020-01-02T21:27:04.163" Score="0" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$n=C*K_w*K_h$&lt;/span&gt;. Then you should only need &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; filters. Not &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; to keep all the information. If you just used the rows of the identity matrix as your filters than your convolution would just be making an exact copy so it definitely wouldn't be throwing away information. On the other hand, there will be a max pooling operation. To simplify the question let's suppose we have a 3 channels, and a 1 by 1 kernel. And then let's suppose it is just one convolution followed by global max pooling. Also, let's use your assumption that it's all binary. If you have &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; filters then the final output will be &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; dimensional no matter how many input points you have. So clearly information is being thrown away there. But that's not such a bad thing. Throwing away irrelevant information gets us closer to the features we need the problem at hand. The parts that get thrown away by max pooling correspond to features not being found in a particular part of the image.&lt;/p&gt;&#xA;" OwnerUserId="32467" LastActivityDate="2020-01-02T21:27:04.163" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17340" PostTypeId="2" ParentId="17317" CreationDate="2020-01-02T21:42:14.187" Score="2" Body="&lt;p&gt;A neural network is composed of continuous functions. Neural networks are regularized by adding an l2 penalty on the weights to the loss function. This means the neural network will try to make the weights as small as possible. The weights are also initiallized with a N(0, 1) distribution so the initial weights will also tend to be small. All of this means that neural networks will compute a continuous function that is as smooth as possible while still fitting the data. By smooth I mean that similar inputs will tend to have similar outputs when run through the neural network. More formally, &lt;span class=&quot;math-container&quot;&gt;$||x-y||$&lt;/span&gt; small implies &lt;span class=&quot;math-container&quot;&gt;$||f(x)-f(y)||$&lt;/span&gt; small where f represents the output from the neural network. This mean that if a neural network sees an novel input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; that is close to an input from the training data &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; will tend to be close to &lt;span class=&quot;math-container&quot;&gt;$f(y)$&lt;/span&gt;. So the end result is that the neural network will classify &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; based on what the labels for the nearby training examples were. So the neural network is actually a little like k-nearest neighbors in that way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way for neural networks to generalize is using invariance. For example, convolutional neural networks are approximately translation invariant. So this means that if it sees and image where the object in question has been translated then it will still recognize the object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's not giving us the exact function we want. The loss function is a combination of classification accuracy and making the weights small so that you can fit the data with a function that is as smooth as possible. This tends to generalize well for the reasons I said before but it's just an approximation. You can solve the problem more exactly using minimal assumptions with a Gaussian process but Guassian processes are too slow to handle large amounts of data. &lt;/p&gt;&#xA;" OwnerUserId="32467" LastEditorUserId="32467" LastEditDate="2020-01-04T21:42:39.300" LastActivityDate="2020-01-04T21:42:39.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17341" PostTypeId="2" ParentId="11542" CreationDate="2020-01-02T21:45:06.870" Score="1" Body="&lt;p&gt;If the AI is static (heuristic and fixed), it will always pursue the stated goal.  However, such a system would be &quot;brittle&quot;, and either break or produce bad output if confronted with input not previously defined, or outside its model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI evolves via learning, even where the goal is specific, its interpretation of that goal might change, and produce unexpected results.  (The &quot;I, Robot&quot; scenario.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the AI is &lt;a href=&quot;https://en.wikipedia.org/wiki/Emergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;emergent&lt;/a&gt;, by which I mean it evolves in way that cannot be predicted, it might evolve new goals.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer the question directly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hypothetically, if there was an AGI or artificial superintelligence, or ultraintelligent machine tasked with protecting humans, and that AI perceived humans to be destroying themselves, that AI would, if able, take control of human society.  (I don't see this as contradicting its goal.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it must be stated that, in a condition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;imperfect&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://en.wikipedia.org/wiki/Complete_information&quot; rel=&quot;nofollow noreferrer&quot;&gt;incomplete&lt;/a&gt; information, where the problem is &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;intractable&lt;/a&gt;, the AI is just guessing like we humans do, even if it makes better guesses, as in the case of narrowly intelligent AIs like AlphaGo.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2020-01-02T21:45:06.870" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="17342" PostTypeId="2" ParentId="17313" CreationDate="2020-01-02T21:59:12.667" Score="0" Body="&lt;p&gt;I think if you got the dataset, then a standard 1d convolutional neural network would work to some extent. It's not that there is some property of nearby sounds that it would pick up on. It would just memorize all the sounds that tend to come from your desk. I think the coding part would be pretty standard stuff. But collecting the data will be hard. You have to get a really big labeled dataset of sounds coming from your desk and sounds coming beyond a 3 foot radius. This dataset has to be realistic and representative of the real world. Getting that dataset would be pretty tricky but it is doable if you put multiple microphones in your house in order to triangulate the exact positions of all sounds. It would be like GPS but using sounds waves instead of light waves. &lt;/p&gt;&#xA;" OwnerUserId="32467" LastActivityDate="2020-01-02T21:59:12.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17343" PostTypeId="2" ParentId="17306" CreationDate="2020-01-02T22:06:01.060" Score="1" Body="&lt;p&gt;GPT2 predicts the next word that people will say. &lt;a href=&quot;https://openai.com/blog/better-language-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/better-language-models/&lt;/a&gt;&#xA;Facebook predicts what will make you keep using their site.&#xA;Youtube predicts what videos you will click on.&lt;/p&gt;&#xA;" OwnerUserId="32467" LastActivityDate="2020-01-02T22:06:01.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17344" PostTypeId="1" AcceptedAnswerId="17367" CreationDate="2020-01-03T01:14:10.520" Score="4" ViewCount="79" Body="&lt;p&gt;I've been using several resources to implement my own artificial neural network package in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among some of the resources I've been using are &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.anotsorandomwalk.com/backpropagation-example-with-numbers-step-by-step/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.anotsorandomwalk.com/backpropagation-example-with-numbers-step-by-step/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cs.stanford.edu/people/karpathy/convnetjs/intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cs.stanford.edu/people/karpathy/convnetjs/intro.html&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;as well as several others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code manages to replicate the results in the first two resources exactly. However, these are fairly simple networks in terms of depth. Hence the following (detailed) question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my implementation, I've been working with the MNIST Database of handwritten digits (&lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yann.lecun.com/exdb/mnist/&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the ANN package I wrote, I have created a simple ANN with 784 input neurons, one hidden layer with 16 neurons, as well as an output layer with ten neurons. I have implemented ReLU on the hidden layer and the ouput layer, as well as a softmax on the output layer to get probabilities.The weights and biases are each individiually initialized to random values in the range [-1,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the network is 784x16x10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My backpropagation incorporates weight gradient and bias gradient logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this configuration, I repeatedly get about a 90% hit rate with a total average cost of ~0.07 on the MNIST training set comprising 60,000 digits, and a slightly higher hit rate of ~92.5% on the test set comprising 10,000 digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my first implementation of an ANN, I am pretty happy with that. However, my next thought was: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&quot;If I add another hidden layer, I should get even better results...?&quot;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I created another artificial network with the same configuration, except for the addition of another hidden layer of 16 neurons, which I also run through a reLU. So this network is 784x16x16x10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On this ANN, I get significantly worse results. The hit rate on the training set repeatedly comes out at ~45% with a total average error of ~0.35, and on the test set I also only get about 45%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to either one or both of the following conclusions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A) My implementation of the ANN in C++ is somehow faulty. If so, my bet would be it is somewhere in the backpropagation, as I am not 100% certain my weight gradient and bias gradient calculation is correct for any layers before the last hidden layer.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;B) This is an expected effect. Something about adding another layer makes the ANN not suitable for this (digit classification) kind of problem.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, A, B, or A and B could be true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Could someone with more experience than me give me some input, especially on whether B) is true or not?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If B) is not true, then I know I have to look at my code again.&lt;/p&gt;&#xA;" OwnerUserId="32471" LastActivityDate="2020-01-04T11:48:53.920" Title="Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?" Tags="&lt;image-recognition&gt;&lt;hidden-layers&gt;&lt;multilayer-perceptron&gt;&lt;c++&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17345" PostTypeId="2" ParentId="17317" CreationDate="2020-01-03T03:06:49.143" Score="0" Body="&lt;p&gt;A fairly recent paper posits an answer to this:&lt;br&gt;&#xA;&lt;em&gt;Reconciling modern machine learning practice and the bias-variance trade-off&lt;/em&gt;. &#xA;Mikhail Belkin, Daniel Hsu, Siyuan Ma, Soumik Mandal&lt;br&gt;&#xA;&lt;a href=&quot;https://arxiv.org/abs/1812.11118&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1812.11118&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://www.pnas.org/content/116/32/15849&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pnas.org/content/116/32/15849&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm probably not qualified to summarize, but it sounds like their conjectured mechanism is: by having far more parameters than are needed even to perfectly interpolate the training data, the space of possible resulting functions expands to include &quot;simpler&quot; functions (simpler here obviously not meaning fewer parameters, but instead something like &quot;less wiggly&quot;) that generalize better even while perfectly interpolating the training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That seems completely orthogonal to the more traditional ML approach of reducing capacity via dropout, regularization, etc.&lt;/p&gt;&#xA;" OwnerUserId="21542" LastActivityDate="2020-01-03T03:06:49.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17346" PostTypeId="1" CreationDate="2020-01-03T08:28:39.313" Score="2" ViewCount="21" Body="&lt;p&gt;So I am trying to use a majority vote classifier combining different models  and I was wondering if it is acceptable to use different training sets for the individual models (including different features) if these sets all come from one larger dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="32477" LastActivityDate="2020-01-03T08:28:39.313" Title="Is it acceptable to use various training sets for the individual models when using a majority vote classifier?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17347" PostTypeId="1" CreationDate="2020-01-03T10:44:00.013" Score="2" ViewCount="28" Body="&lt;p&gt;I'm trying to determine the frequency from a signal with NN. I'm using the Adeline model for my project and I'm taking a few samples in each 0.1-volt step in a true signal and a noisy one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First question: am I wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second question: my network works fine until the frequency of my sample for the test is equal to the frequency of my sample for the training. Otherwise, my network doesn't work and gives me the wrong answer. &#xA;What do I need to do for this model?&#xA;for solving this problem I must use nonlinear steps like logarithmic steps. but How to use logarithmic steps in MatLab?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edition: I understand my problem is not Overfitting! I found that my samples step are linear and my samples are nonlinear so this is wrong&#xA;for solving this problem I must use nonlinear steps like logarithmic steps. but How to use logarithmic steps in MatLab?  &lt;/p&gt;&#xA;" OwnerUserId="32481" LastEditorUserId="32481" LastEditDate="2020-01-04T14:15:39.780" LastActivityDate="2020-01-04T14:15:39.780" Title="Determine Frequency from Noisy Signal With Neural Networks (With Adeline Model)" Tags="&lt;neural-networks&gt;&lt;signal-processing&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17348" PostTypeId="2" ParentId="17326" CreationDate="2020-01-03T10:55:33.627" Score="2" Body="&lt;p&gt;As you know, an LSTM language model takes in the past word and tries to predict the new one and continue over a loop. A sentence is divided into &lt;strong&gt;tokens&lt;/strong&gt; and depending on different method, the tokens are divided differently. Some model maybe &lt;strong&gt;character based models&lt;/strong&gt; which simply uses each character as input and output. In this case you can treat punctuation as one character and just run the model as normal. For &lt;strong&gt;word based model&lt;/strong&gt; which is commonly used in many systems, we treat punctuation as it's own token. It is commonly called a end of sentence token. There is also a specific token for end of output. This makes the system knows when to finish and stop prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, just so you know for language model trying to generate original text, they feed the output as the input of the next data point, but the output they choose is not necessarily the one with the best accuracy. They set a threshold and choose upon that. It can introduce diversity to the language model so taht even though the staring word is the same, the sentence/paragraph will be different and not the same one again and again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some state-of-the-art models, you can try GPT-2 as mentioned by @jdleoj23 . This is a character based(actually byte based but basically the same, it treats each unicode symbol individually) model that uses attention and transformers. The advantage of character based system is that even inputs that have spelling errors can be inputted into the model and new words not in the dictionary can be introduced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if you want to learn more about how language model works, and not just striving for the best performance, you should try implementing one simple one by yourself. You can try following this article which uses keras to make a language model. &#xA;&lt;a href=&quot;https://machinelearningmastery.com/develop-word-based-neural-language-models-python-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/develop-word-based-neural-language-models-python-keras/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of making a simple one is taht you can actually understand the encoding process, the tokenization process, the model underneath and others instead of relying on other people's code. The article uses keras Tokenizer but you could try writing your own using regex and simple string processing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope my help is useful for you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2020-01-03T11:06:41.697" LastActivityDate="2020-01-03T11:06:41.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17349" PostTypeId="1" CreationDate="2020-01-03T15:27:58.663" Score="3" ViewCount="145" Body="&lt;p&gt;I am working on a project that requires time-series prediction (regression) and I use LSTM network with first 1D conv layer in Keras/TF-gpu as follows:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Conv1D(filters=60, activation='relu', input_shape=(x_train.shape[1], len(features_used)), kernel_size=5, padding='causal', strides=1))&#xA;model.add(CuDNNLSTM(units=128, return_sequences=True))&#xA;model.add(CuDNNLSTM(units=128))&#xA;model.add(Dense(units=1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an effect my model is clearly overfitting:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CgP49.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CgP49.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I decided to add dropout layers, first I added layers with 0.1, 0.3 and finally 0.5 rate:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dropout(0.5))&#xA;model.add(Conv1D(filters=60, activation='relu', input_shape=(x_train.shape[1], len(features_used)), kernel_size=5, padding='causal', strides=1))&#xA;model.add(Dropout(0.5))&#xA;model.add(CuDNNLSTM(units=128, return_sequences=True))&#xA;model.add(Dropout(0.5))&#xA;model.add(CuDNNLSTM(units=128))&#xA;model.add(Dense(units=1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However I think that it has no effect on the network learning process, even though 0.5 is quite large dropout rate:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Wz60u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wz60u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this possible that dropout has little/no effect on a training process of LSTM  or maybe I do something wrong here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] Adding plots of my TS, general and zoomed in view.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fK1vg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fK1vg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WDysd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WDysd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to add that the time of training increases just a bit (i.e. from 1540 to 1620 seconds) when I add the dropout layers.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2020-01-07T14:00:26.383" LastActivityDate="2020-03-02T16:54:26.150" Title="Can dropout layers not influence LSTM training?" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;dropout&gt;&lt;regularization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17350" PostTypeId="1" CreationDate="2020-01-03T15:43:15.170" Score="2" ViewCount="36" Body="&lt;p&gt;What are some common approaches to estimate the transition or observation probabilities, when the probabilities are not exactly known? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When realizing a POMDP model, the state model needs additional information in terms of transition and observation probabilities. Often these probabilities are not known and an equal distribution is also not given. How can we proceed? &lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-01-03T20:09:57.437" LastActivityDate="2020-01-03T20:09:57.437" Title="What are some approaches to estimate the transition and observation probabilities in POMDP?" Tags="&lt;reinforcement-learning&gt;&lt;probability&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17351" PostTypeId="1" CreationDate="2020-01-03T16:08:27.017" Score="1" ViewCount="262" Body="&lt;p&gt;Is there some established Object Detection algorithm that is able to detect the four corners of an arbitrary quadrilateral  (x0,y0,x1,y1,x2,y2,x3,y3) as opposed to the more typical perpendicular rectangular (x,y,w,h) ?&lt;/p&gt;&#xA;" OwnerUserId="21583" LastActivityDate="2021-05-28T10:02:30.217" Title="Object Detection Algorithm that detects four corners of arbitrary quadrilateral, not just perpendicular rectangular" Tags="&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17352" PostTypeId="1" CreationDate="2020-01-03T16:51:50.983" Score="4" ViewCount="69" Body="&lt;p&gt;I've written a program to analyse a given piece of text from a website and make conclusary classifications as to its validity. The code basically vectorizes the description (taken from the HTML of a given webpage in real time) and takes in a few inputs from that as features to make its decisions. There are some more features like the domain of the website and some keywords I've explicitly counted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The highest accuracy I've been able to achieve is with a RandomForestClassifier, (&gt;90%). I'm not sure what I can do to make this accuracy better except incorporating a more sophisticated model. I tried using an MLP but for no set of hyperparameters does it seem to exceed the previous accuracy. I have around 2000 datapoints available for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any classifier that works best for such projects? Does anyone have any suggestions as to how I can bring about improvements? (If anything needs to be elaborated, I'll do so.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how I can improve on this project in general? Should I include the text on a webpage as well? How should I do so? I tried going through a few sites, but the next doesn't seem to be contained in any specific element whereas the description is easy to obtain from the HTML. Any help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What else can I take as features? If anyone could suggest any creative ideas, I'd really appreciate it.&lt;/p&gt;&#xA;" OwnerUserId="32490" LastEditorUserId="32490" LastEditDate="2020-01-04T15:52:16.233" LastActivityDate="2020-01-04T15:52:16.233" Title="Is there any classifier that works best in general for NLP based projects?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;text-classification&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17353" PostTypeId="2" ParentId="17349" CreationDate="2020-01-03T17:34:00.083" Score="2" Body="&lt;p&gt;A couple of points:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Have you firstly scaled your data, e.g. using &lt;strong&gt;MinMaxScaler&lt;/strong&gt;? This could be one reason why your loss readings remain high.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Additionally, consider that while Dropout can be useful for reducing overfitting, it is not necessarily a panacea.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Let's take an example of using LSTM to forecast fluctuations in weekly hotel cancellations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Model without Dropout&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Generate LSTM network&#xA;model = tf.keras.Sequential()&#xA;model.add(LSTM(4, input_shape=(1, previous)))&#xA;model.add(Dense(1))&#xA;model.compile(loss='mean_squared_error', optimizer='adam')&#xA;history=model.fit(X_train, Y_train, validation_split=0.2, epochs=20, batch_size=1, verbose=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Over 20 epochs, the model achieves a validation loss of &lt;strong&gt;0.0267&lt;/strong&gt; without Dropout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4HC1a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4HC1a.png&quot; alt=&quot;loss without dropout&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Model with Dropout&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Generate LSTM network&#xA;model = tf.keras.Sequential()&#xA;model.add(LSTM(4, input_shape=(1, previous)))&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(1))&#xA;model.compile(loss='mean_squared_error', optimizer='adam')&#xA;history=model.fit(X_train, Y_train, validation_split=0.2, epochs=20, batch_size=1, verbose=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, validation loss is slightly higher with Dropout at &lt;strong&gt;0.0428&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0cje4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0cje4.png&quot; alt=&quot;loss with dropout&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Make sure you have specified the loss function correctly. If you are forecasting a time series, then you are most likely working with interval data. Therefore, &lt;strong&gt;mean_squared_error&lt;/strong&gt; is an appropriate loss function as one is trying to estimate the deviation between the predicted and actual values.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As a counterexample, &lt;strong&gt;binary_crossentropy&lt;/strong&gt; would not be suitable as the time series is not a classification set. However, misspecifying the loss function is a common error. Therefore, you also want to make sure you are using the appropriate loss function and then work from there.&lt;/p&gt;&#xA;" OwnerUserId="22692" LastEditorUserId="22692" LastEditDate="2020-03-02T16:54:26.150" LastActivityDate="2020-03-02T16:54:26.150" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17354" PostTypeId="2" ParentId="17310" CreationDate="2020-01-03T18:47:18.107" Score="2" Body="&lt;p&gt;The whole idea behind those distributed optimization methods is that data should be local in every node/worker. Thus, if you only send the loss value to the central node, this node can't compute the gradients of this loss, and thus can't do any training. However, if you don't want to send gradients, a family of distributed optimization algorithms called consensus-based optimization can be used to only send the local weight of the model to neighbouring nodes, and those nodes use their local gradient and the models from their neighbours to update their local models.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="2444" LastEditDate="2020-01-03T21:17:24.570" LastActivityDate="2020-01-03T21:17:24.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17355" PostTypeId="2" ParentId="17256" CreationDate="2020-01-03T18:55:06.510" Score="3" Body="&lt;p&gt;In reallity any continous function on a compact can be approximated by a neural network having one hidden layer with a finite number of neurones (This is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Approximation Theorem&lt;/a&gt;). Thus you only need one hidden layer to approximate the multiplication on a compact, note that you need to apply a non linear activation on the hidden layer to do this. &lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="21583" LastEditDate="2020-01-04T20:15:47.240" LastActivityDate="2020-01-04T20:15:47.240" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17356" PostTypeId="2" ParentId="17304" CreationDate="2020-01-03T19:08:31.317" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$l_{2,1}$&lt;/span&gt; is a matrix norm, as stated in &lt;a href=&quot;https://www.ijcai.org/Proceedings/11/Papers/267.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. For a certain matrix &lt;span class=&quot;math-container&quot;&gt;$A \in \mathbb{R}^{r\times c}$&lt;/span&gt;, we have  &lt;span class=&quot;math-container&quot;&gt;$$||A||_{2,1} = \sum_{i=1}^r \sqrt{\sum_{j=1}^c A_{ij}^2}$$&lt;/span&gt; &#xA;You first apply &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt; norm along the columns to obtain a vector with r dimensions. Then, you apply &lt;span class=&quot;math-container&quot;&gt;$l_1$&lt;/span&gt; norm to that vector to obtain a real number. You can generalize this notation to every norm &lt;span class=&quot;math-container&quot;&gt;$l_{p,q}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32485" LastActivityDate="2020-01-03T19:08:31.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17357" PostTypeId="1" AcceptedAnswerId="17375" CreationDate="2020-01-03T20:52:32.497" Score="2" ViewCount="78" Body="&lt;p&gt;Will CNN, LSTM, GRU and transformer be better classified as Computational Intelligence (CI) tools or Artificial General Intelligence (AGI) tools? The term CI arose back when some codes like neural networks, GA, PSO were considered doing magical stuff. These days CI tools do not appear very magical. Researchers want codes to exude AGI. Do the current state of art Deep Learning codes fall in the AGI category?&lt;/p&gt;&#xA;" OwnerUserId="12850" LastEditorUserId="2444" LastEditDate="2020-01-03T20:55:31.957" LastActivityDate="2020-01-04T23:41:44.697" Title="Are CNN, LSTM, GRU and transformer AGI or computational intelligence tools?" Tags="&lt;deep-learning&gt;&lt;terminology&gt;&lt;agi&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17358" PostTypeId="2" ParentId="17352" CreationDate="2020-01-03T21:09:23.987" Score="1" Body="&lt;p&gt;The accuracy depends on various factors. Might not always be the algorithm. For example a cleaner data with a poor algorithm might still give better results and vice versa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the preprocessing techniques you are using? This &lt;a href=&quot;https://towardsdatascience.com/nlp-text-preprocessing-a-practical-guide-and-template-d80874676e79&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprocessing techniques&lt;/a&gt; article is a good starting point for html data. And by vectorising I assume you mean word2vec, use a pre-trained word2vec model. Like google's &lt;a href=&quot;https://code.google.com/archive/p/word2vec/&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt; model it's trained on a lot of data(about 100 billion words). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSTM performs good whenever the intent of the sentence is important. Check out &lt;a href=&quot;https://towardsdatascience.com/multi-class-text-classification-with-lstm-1590bee1bd17&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&#xA;Ram hit Vijay and Vijay hit Ram, might mean the same to most algorithms. Example, Naive Bayes. &lt;/p&gt;&#xA;" OwnerUserId="32408" LastEditorUserId="32408" LastEditDate="2020-01-03T21:24:43.947" LastActivityDate="2020-01-03T21:24:43.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17360" PostTypeId="1" CreationDate="2020-01-03T22:50:22.433" Score="3" ViewCount="214" Body="&lt;p&gt;I am currently using a loss averaged over the last 100 iterations, but this leads to artifacts like the loss going down even when the current iteration has an average loss, because the loss 100 iterations ago was a large outlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using different interval lengths, but I wonder if an average over the last few iterations really is the right way to plot the loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there common alternatives? Maybe using decaying weights in the average? What are the best-practices for visualizing the loss?&lt;/p&gt;&#xA;" OwnerUserId="25798" LastEditorUserId="23713" LastEditDate="2020-01-06T07:02:39.393" LastActivityDate="2020-01-06T08:00:42.793" Title="What is the best way to smoothen out a loss curve plot" Tags="&lt;loss&gt;&lt;plotting&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17361" PostTypeId="2" ParentId="17352" CreationDate="2020-01-04T01:36:37.000" Score="1" Body="&lt;p&gt;First of all, there are multiple factors on how well models will work. Amount of data, source of data, hyperparameters, model type, training time etc... All of these will affect the accuracy. However, no classifier will work best in general. It all depends on the different factors, and not one can satisfy all, at least for now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For improving the accuracy, we need to first make those factors ideal so that the classification will have a higher accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, how many data do you have? If you are using html webpage, you probably need at least 10000 data samples. If you have at least that amount of data you should be ok with overfitting. You also need to &lt;strong&gt;clean&lt;/strong&gt; the data. One way to do it is to &lt;strong&gt;tokenize&lt;/strong&gt; it. Tokenization of text data basically means to split the text into words and make a dictionary out of it. Then each word is encoded to a specific number where each same word have the same encoding. You are using the raw HTML as input, which have a lot of unnecessary information and tags and stuff, you can try removing those or completely remove all html tags if they are not required. The key to cleaning the data is to extract the pieces of information that is &lt;strong&gt;important&lt;/strong&gt; and &lt;strong&gt;necessary&lt;/strong&gt; for the model to work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you should explore the model. For a NLP (Natural Language Processing) task, your best bet is to choose a RNN (Recurrent Neural Network). This type of network have &lt;strong&gt;memory cells&lt;/strong&gt; taht helps with text type data as text often have distant linkage in a paragraph, for example one sentence may use a &quot;she&quot; that refers to a person mentioned in two sentence before, and if you just feed every single encoding of words in a MLP, it would not have this &lt;strong&gt;memory&lt;/strong&gt; for the network to learn long term connection between text. A RNN also is time dependent, meaning it processes each token one by one according to the direction of time. This makes the text more intuitive to the network as text is designed to be read forward, not all at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your current method is to first vectorize the HTML code, then feed it into a random forest classifier. A random forest classifier works great, but it cannot scale when there is more data. The accuracy of a random forest classifier will stay mostly the same when data increase while in deep neural networks the accuracy will increase with the amount of data. However a deep neural network will require a large amount of data to start of with. If your amount of data is not too much (&amp;lt; 10000), this method should be your choice of method. However if you plan to add more data or if teh data is more, you should try a deep learning based method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For deep learning based method, ULMFit is a great model to try. It uses a LSTM(Long Short Term Memory) network (which is a type of RNN) with a language model pretraining and many different method to increase the accuracy. You can try it with the fast.ai implementation. &lt;a href=&quot;https://nlp.fast.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nlp.fast.ai/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to try a method that you can practically implement yourself, you could try to use a plain LSTM with one hot encoding as input. However, don't use word2vec to do preprocessing as your input data is html code. The word2vec model is for normal English text, not the html tags and stuff. Moreover a custom encodings will work better as in the training process you can train teh encoding as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2020-01-04T01:36:37.000" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17363" PostTypeId="2" ParentId="17351" CreationDate="2020-01-04T07:58:14.283" Score="0" Body="&lt;p&gt;You can use OpenCV's &lt;a href=&quot;https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=minarearect#minarearect&quot; rel=&quot;nofollow noreferrer&quot;&gt;cv2.minAreaRect()&lt;/a&gt; to detect oriented/rotated rectangular bounding boxes. Below's an example result from &lt;a href=&quot;https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contour_features/py_contour_features.html#b-rotated-rectangle&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV-Python-tutorials&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V9LKp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V9LKp.png&quot; alt=&quot;Using cv2.minAreaRect()&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could train a supervised object detection model to output 8 co-ordinate values (x0,y0,x1,y1,x2,y2,x3,y3) of the quadrilateral by training with a labeled oriented-bounding-box dataset. You could also create the bounding box labels yourself for the same by using tools such as &lt;a href=&quot;http://www.robots.ox.ac.uk/~vgg/software/via/via_demo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG Annotator Tool&lt;/a&gt; among others.&lt;/p&gt;&#xA;" OwnerUserId="30644" LastActivityDate="2020-01-04T07:58:14.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17364" PostTypeId="1" CreationDate="2020-01-04T09:01:03.100" Score="2" ViewCount="42" Body="&lt;p&gt;Imitation learning uses experiences of an (expert) agent to train another agent, in my understanding. If I want to use an on-policy algorithm, for example, Proximal Policy Optimization, because of it's on-policy nature we cannot use the experiences generated by another policy directly. Importance Sampling can be used to overcome this limitation, however, it is known to be highly unstable. How can imitation learning be used for such on-policy algorithms avoiding the stability issues?&lt;/p&gt;&#xA;" OwnerUserId="29879" LastEditorUserId="2444" LastEditDate="2020-11-05T22:33:41.413" LastActivityDate="2020-11-05T22:33:41.413" Title="Can we use imitation learning for on-policy algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;importance-sampling&gt;&lt;on-policy-methods&gt;&lt;imitation-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17365" PostTypeId="1" CreationDate="2020-01-04T09:21:02.880" Score="2" ViewCount="53" Body="&lt;p&gt;I have a hard time formulating this question(I'm not knowledgeable enough I think), so I'll give an example first and then the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a table of data, let's say the occupancy of a building during the course of the day; each row has columns like &quot;people_inside_currently&quot;, &quot;apartment_id&quot;, &quot;hour_of_day&quot;, &quot;month&quot;, &quot;year&quot;, &quot;name_of_day&quot;(monday-sunday), &quot;amount_of_kids&quot;, &quot;average_income&quot; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might preprocess two columns into a column &quot;percent_occupied_during_whole_day&quot; or something like that, and you want to group the data points in accordance with this as the main focus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm wondering is: why use machine learning(particularly unsupervised clustering) for this? Why not just put it into an SQL database table(for example), calculate two columns into that new one, sort by descending order, and then split it into &quot;top 25%, next 25%, next 25%, last 25%&quot; and output this as &quot;categories of data&quot;? This is simpler, isn't it? I don't see the value of, for instance, making a Principle Component Analysis on it, reducing columns to some &quot;unifying columns&quot; which you don't know what to call anymore, and looking at the output of that, when you can get so much clearer results by just simply sorting and dividing the rows like this? I don't see the use of unsupervised clustering, I've googled a bunch of terms, but only found tutorials and definitions, applications(which seemed unnecessarily complex for such simple work), but no explanation of this.&lt;/p&gt;&#xA;" OwnerUserId="20747" LastActivityDate="2020-01-04T09:21:02.880" Title="Why machine learning instead of simple sorting and grouping?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17366" PostTypeId="1" CreationDate="2020-01-04T09:38:09.617" Score="3" ViewCount="32" Body="&lt;p&gt;I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I want my model to predict class A around 98% of the time, class B 1% and class C 1%. How can I do that?&lt;/p&gt;&#xA;" OwnerUserId="32499" LastActivityDate="2020-01-04T09:38:09.617" Title="Rarely predict minority class imbalanced datasets" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17367" PostTypeId="2" ParentId="17344" CreationDate="2020-01-04T11:48:53.920" Score="2" Body="&lt;p&gt;You probably got the back propagation wrong. I have done a test on the accuracy on adding an extra layer and the accuracy went up from 94% to 96% for me. See this for details:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://colab.research.google.com/drive/17kAJ2KJ36grG9sz-KW10fZCQW9i2Tf2c&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/17kAJ2KJ36grG9sz-KW10fZCQW9i2Tf2c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To run the notebook click &lt;code&gt;Open in playground&lt;/code&gt; and run the code. There is a commented line which add 1 extra layer. The syntax should be easy to understand even though it is in python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For back propagation, you can try to see this python implementation of multi layer perceptron backpropagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/enggen/Deep-Learning-Coursera/blob/master/Neural%20Networks%20and%20Deep%20Learning/Building%20your%20Deep%20Neural%20Network%20-%20Step%20by%20Step.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/enggen/Deep-Learning-Coursera/blob/master/Neural%20Networks%20and%20Deep%20Learning/Building%20your%20Deep%20Neural%20Network%20-%20Step%20by%20Step.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A network will not usually decrease it's accuracy by almost a half in normal scenario when you add an extra layer, though it is possible to have the network decrease accuracy when you add an extra layer due to overfitting. Though if this happen the performance drop won't be that dramatic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I can help you.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2020-01-04T11:48:53.920" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17369" PostTypeId="1" AcceptedAnswerId="17781" CreationDate="2020-01-04T13:19:19.703" Score="4" ViewCount="110" Body="&lt;ul&gt;&#xA;&lt;li&gt;I have items called 'Resources' from 1 to 7.&lt;/li&gt;&#xA;&lt;li&gt;I have to use them in different actions identified from 1 to 10.&lt;/li&gt;&#xA;&lt;li&gt;I can do a maximum of 4 actions each time. This is called 'Operation'.&lt;/li&gt;&#xA;&lt;li&gt;The use of a resource has a cost of 1 per each 'Operation' even if it is used 4 times.&lt;/li&gt;&#xA;&lt;li&gt;The following table indicates the resources needed to do the related actions:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;|        |            Resources             |&#xA;|--------|----------------------------------|&#xA;| Action |  1 |  2 |  3 |  4 |  5 |  6 |  7 |&#xA;|--------|----------------------------------|&#xA;|     1  |  1 |  0 |  1 |  1 |  0 |  0 |  0 |&#xA;|     2  |  1 |  1 |  0 |  0 |  1 |  0 |  0 |&#xA;|     3  |  1 |  0 |  1 |  0 |  0 |  1 |  0 |&#xA;|     4  |  0 |  1 |  0 |  0 |  0 |  0 |  0 |&#xA;|     5  |  1 |  0 |  1 |  1 |  0 |  1 |  0 |&#xA;|     6  |  1 |  1 |  1 |  0 |  0 |  0 |  0 |&#xA;|     7  |  0 |  1 |  0 |  0 |  0 |  0 |  0 |&#xA;|     8  |  1 |  0 |  1 |  0 |  1 |  0 |  0 |&#xA;|     9  |  0 |  1 |  0 |  1 |  0 |  0 |  0 |&#xA;|    10  |  1 |  1 |  1 |  0 |  0 |  0 |  1 |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The objective is to group all the 'Actions' in 'Operations' that minimize the total cost. For example, a group composed by actions {3, 7, 9} needs the resources {1, 2, 3, 4, 6} and therefore has a cost of 5, but a group composed by actions {4, 7, 9} needs the resources {2, 4} and therefore has a cost of 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is needed to get done all the actions the most economically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithm can solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="6207" LastEditorUserId="6207" LastEditDate="2020-01-04T20:41:37.443" LastActivityDate="2020-01-31T17:50:33.157" Title="Which algorithm to use to solve this optimization problem?" Tags="&lt;algorithm&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17370" PostTypeId="1" CreationDate="2020-01-04T13:40:43.557" Score="3" ViewCount="124" Body="&lt;p&gt;I am working on speaker identification problem using GMM (Gaussian Mixture Model). I have to just identify one user present in the given audio, so for second class noise or silent audio may use or not just like in image classification for an object we create a non-object class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used a silent class is always showing the user is present ( which is not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If any other model can give better accuracy fulfil the condition that only 30 sec of audio of a particular user is available and given test audio may has long size.&lt;/p&gt;&#xA;" OwnerUserId="15368" LastActivityDate="2020-01-10T07:14:03.440" Title="Speaker Identification / Recognition for less size audio files" Tags="&lt;generative-model&gt;&lt;data-science&gt;&lt;speech-recognition&gt;&lt;state-of-the-art&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17371" PostTypeId="1" AcceptedAnswerId="17411" CreationDate="2020-01-04T15:50:36.740" Score="3" ViewCount="1254" Body="&lt;p&gt;How to calculate mean speed in FPS for an object detection model like YOLOv3 or YOLOv3-Tiny? Different object detection models are often presented on charts like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hUODt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hUODt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;I am using the DarkNet framework in my project and I want to create similar charts for my own models based on YOLOv3. Is there some easy way to get mean FPS speed for my model with the &quot;test video&quot;?&lt;/p&gt;&#xA;" OwnerUserId="30992" LastEditorUserId="23713" LastEditDate="2020-01-06T02:48:09.550" LastActivityDate="2020-01-08T10:30:59.607" Title="Calculation of FPS on object detection task" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17373" PostTypeId="1" CreationDate="2020-01-04T18:24:36.890" Score="2" ViewCount="31" Body="&lt;p&gt;How can a system recognize if two strings have the same or similar meaning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the following two strings&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Wikipedia provides good information.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Wikipedia is a good source of information.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What methods are available to do this?&lt;/p&gt;&#xA;" OwnerUserId="32506" LastEditorUserId="2444" LastEditDate="2020-01-04T21:16:55.390" LastActivityDate="2020-01-04T21:16:55.390" Title="How can a system recognize if two strings have the same or similar meaning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17374" PostTypeId="2" ParentId="17373" CreationDate="2020-01-04T18:51:51.570" Score="1" Body="&lt;p&gt;Getting the intent of the sentence is not an easy task. To get you started on what to do, have a look on word vectors. You can also download pre-trained &lt;a href=&quot;https://stackoverflow.com/a/59591075/8568784&quot;&gt;word2vec&lt;/a&gt; models. They help in getting similarity of words and reasoning with words. To get the intent of a sentence, you can use LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fun fact most NLP algorithms strip away punctuation with is sufficient for most cases, but to give a counter example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The defendant, who looked apologetic, was found guilty.&#xA;The defendant who looked apologetic was found guilty.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;They mean different things and are difficult to catch the intent even with the best algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: For those wondering the difference, in the second sentence it seems like there were two defendants, and it was the one who looked apologetic who was found guilty while the other walked away free.&lt;/p&gt;&#xA;" OwnerUserId="32408" LastActivityDate="2020-01-04T18:51:51.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17375" PostTypeId="2" ParentId="17357" CreationDate="2020-01-04T23:41:44.697" Score="0" Body="&lt;p&gt;CNNs, LSTMs, GRUs and transformers are or use artificial neural networks. The expression &lt;em&gt;computational intelligence&lt;/em&gt; (CI) is often used interchangeably with &lt;em&gt;artificial intelligence&lt;/em&gt; (AI). CI can also refer to a subfield or superfield of AI where biology is often an inspiration. See &lt;a href=&quot;http://cogprints.org/5358/1/06-CIdef.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is Computational Intelligence and what could it become?&lt;/a&gt; by Włodzisław Duch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNNs are &lt;a href=&quot;https://stats.stackexchange.com/a/221142/82135&quot;&gt;Turing complete&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/13319/2444&quot;&gt;CNNs have been shown to be universal function approximators&lt;/a&gt; (they can approximate any continuous function to an arbitrary accuracy given a sufficiently deep architecture), but that doesn't mean we will be able to create AGI with them, unless you believe that AGI is just a bunch of algorithms, but, IMHO, that alone doesn't produce AGI. See also &lt;a href=&quot;https://plato.stanford.edu/entries/computational-mind/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the computational theory of mind&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, CNNs, LSTMs, GRUs and transformers are deep learning tools (so they could also be considered CI tools, given some definitions of CI), which &lt;em&gt;might&lt;/em&gt; be useful for the development of AGI.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-01-04T23:41:44.697" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17376" PostTypeId="1" CreationDate="2020-01-05T00:42:30.370" Score="1" ViewCount="31" Body="&lt;p&gt;I have two convex, smooth loss functions to minimise. During the training (a very simple model) using batch SGD (with tuned optimal learning rate for each loss function), I observe that the (log) loss curve of the loss 2 converges much faster and is much more smooth than that of the loss 2, as shown in the figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I say more about the properties of the two loss functions, for example in terms of smoothness, convexity, etc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vjD9R.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vjD9R.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22474" LastEditorUserId="22474" LastEditDate="2020-01-05T00:57:52.357" LastActivityDate="2020-01-05T00:57:52.357" Title="Deduce properties of the loss functions from the training loss curves" Tags="&lt;training&gt;&lt;loss-functions&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17377" PostTypeId="1" CreationDate="2020-01-05T09:01:24.707" Score="3" ViewCount="22" Body="&lt;p&gt;I am proposing a modified version of Sequence-to-Sequence model with dual decoders. The problem that I am trying to solve is Neural Machine Translation into two languages at once. This is the simplified illustration for the model.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                            /--&amp;gt; Decoder 1 -&amp;gt; Language Output 1&#xA;Language Input -&amp;gt; Encoder -|&#xA;                            \--&amp;gt; Decoder 2 -&amp;gt; Language Output 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I understand about back propagation is that we are adjusting the weights of the network to enhance the signal of the targeted output. However, it is not clear to me on how to back propagate in this network because I am not able to find similar implementations online yet. I am thinking of doing the back propagation twice after each training batch, like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Decoder\ 1 \rightarrow Encoder $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ Decoder\ 2 \rightarrow Encoder $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure whether the effect of back propagation from Decoder 2 will affect the accuracy of prediction by Decoder 1. Is this true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, is this structure feasible? If so, how do I properly back propagate in the network?&lt;/p&gt;&#xA;" OwnerUserId="32511" LastActivityDate="2020-01-05T09:01:24.707" Title="How to back propagate for implementation of Sequence-to-Sequence with Multi Decoders" Tags="&lt;recurrent-neural-networks&gt;&lt;machine-translation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17378" PostTypeId="1" AcceptedAnswerId="17380" CreationDate="2020-01-05T13:34:01.623" Score="1" ViewCount="76" Body="&lt;p&gt;I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model overfits and returns very bad results. The correlation coefficient and error metrics are both not performing well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be some of the ways to solve this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I train it with 5000 rows of data and test it with 1000, I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Correlation coefficient                  0.0742&#xA;Mean absolute error                      0.742 &#xA;Root mean squared error                  0.9407&#xA;Relative absolute error                146.2462 %&#xA;Root relative squared error            160.1116 %&#xA;Total Number of Instances             1000     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As mentioned, I can train it with as many training samples as I want and still have the model overfits. If anyone is interested, I can provide/generate some data and post it online.&lt;/p&gt;&#xA;" OwnerUserId="31766" LastEditorUserId="2444" LastEditDate="2020-01-06T14:08:33.977" LastActivityDate="2020-01-06T19:26:54.270" Title="Why does my model overfit on pseudo-random numbers training data?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;overfitting&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="17379" PostTypeId="1" AcceptedAnswerId="17391" CreationDate="2020-01-05T21:10:37.973" Score="1" ViewCount="68" Body="&lt;p&gt;I'm evaluating the accuracy in detecting objects for my image data set using three deep learning algorithms. I have selected a sample of 30 images. To measure the accuracy, I manually count the number of objects in each image and then calculate recall and precision values for three algorithms. Following is a sample:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h13PQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h13PQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally to select the best model for my data set, can I calculate the mean Recall and mean Accuracy? For Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KB10Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KB10Y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32343" LastActivityDate="2020-01-07T02:45:31.117" Title="Can we calculate mean recall and precision" Tags="&lt;deep-learning&gt;&lt;accuracy&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17380" PostTypeId="2" ParentId="17378" CreationDate="2020-01-06T02:36:21.470" Score="1" Body="&lt;p&gt;Simply said, predicting pseudo random number is just not possible for now. Pseudo random numbers generated now have a high enough &quot;randomness&quot; so that it cannot be predicted. Pseudo random numbers is the basis of modern cryptography which is widely used in the world wide web and more. It may be possible in the future through faster computers and stronger AI, but for now it is not possible. If you train a model to fit on pseudo random numbers, the model will just overfit and thus creating a scenario as shown in the question. The training loss will be very low while test loss will be extremely high. The model will just &quot;remember&quot; the training data instead of generalising to all pseudo random numbers, thus the high test loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as a side note, loss is not represented by %, instead it is just a raw numeric value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/q/3850/23713&quot;&gt;this stack exchange answer&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="32408" LastEditDate="2020-01-06T19:26:54.270" LastActivityDate="2020-01-06T19:26:54.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17381" PostTypeId="2" ParentId="17371" CreationDate="2020-01-06T02:46:36.610" Score="1" Body="&lt;p&gt;You can use the dataset test set as &quot;frames&quot; of video. Test the images with your model and calculate the images per second of the result and that is the same as frames per second. However you should set the batch size to 1 as in the real world scenario. You should also display each image with teh corresponding boxes after inference and remove the accuracy calculation as to imitate the real world situation. &lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2020-01-06T02:46:36.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17382" PostTypeId="1" CreationDate="2020-01-06T07:54:57.073" Score="3" ViewCount="48" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nZiP1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nZiP1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This post refers to Fig. 1 of a paper by Microsoft on their Deep Convolutional Inverse Graphics Network: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/11/kwkt_nips2015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/11/kwkt_nips2015.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having read the paper, I understand in general terms how the network functions. However, one detail has been bothering me: how does the network decoder (or &quot;Renderer&quot;) generate small scale features in the correct location as defined by the graphics code? For example, when training the dataset on faces, one might train a single parameter in the graphics code to control the (x,y) location of a small freckle. Since this feature is small, it will be &quot;rendered&quot; by the last convolutional layer where the associated kernels are small. What I don't understand is how the information of the location of the freckle (in the graphics code) propagates through to the last layer, when there are many larger-scale unpooling + convolutional layers in-between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="32505" LastEditorUserId="23713" LastEditDate="2020-01-06T08:41:43.107" LastActivityDate="2020-01-06T08:41:43.107" Title="How are small scale features represented in an Inverse Graphics Network (autoencoder)?" Tags="&lt;convolutional-neural-networks&gt;&lt;autoencoders&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17383" PostTypeId="2" ParentId="17360" CreationDate="2020-01-06T08:00:42.793" Score="1" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exponential Moving Average&lt;/a&gt; method. This method is used in tensorbaord as a way to smoothen a loss curve plot.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/b3xGj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b3xGj.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The algorithm is as follow:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ACJHA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ACJHA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;However there is a small problem doing it this way. As you can see S_t is initialized with the starting value, which makes the starting curve inaccurate. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/G3V2D.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G3V2D.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The green curve is the ideal curve for the algorithm, but the purple curve is the predicted curve. The curve is not correct on the start. To solve this, a correction factor is added in, thus making the algorithm this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uRDa6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uRDa6.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;This introduces WeightedCount which decreases over time to 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exponential Moving Average is also used is other areas of deep learning, the most notable being some optimization algorithms. It is used in Adam, RMSProp and other similar optimizers to smooth out the gradients to make the path to minimal loss a more direct and straightforward path.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2020-01-06T08:00:42.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17384" PostTypeId="2" ParentId="17382" CreationDate="2020-01-06T08:37:22.423" Score="2" Body="&lt;p&gt;Simply said, there is no specific &quot;meaning&quot; to the features generated. They are simply features that are fitted through math and calculus, and nobody knows what they represent exactly, and will never knows. However we can run PCA (Principal Component Analysis) to see which feature is the most &quot;important&quot; of all, aka which feature affects the most in the output image. Then, you can try adjusting the value to manually see and guess what teh value do, but you will never know what exactly it does as it is an arbitrary feature, not specifically set by the network. One value may mean multiple things, or just things we human don't understand. See this amazing video about this for details:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/4VAkrUNLKSo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/4VAkrUNLKSo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This video explains what PCA does and also an example of the features generated by the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For small scales features, they may simply be ignored as they don't contribute much to the loss or accuracy, or maybe they are represented by a big dot or something else until the last few layers. With just 80 features one cannot fully represent a face with such details, and with the resolution the networks like these are trained on, small features like these probably won't be shown in the image.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastActivityDate="2020-01-06T08:37:22.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17385" PostTypeId="1" AcceptedAnswerId="17389" CreationDate="2020-01-06T09:34:45.043" Score="3" ViewCount="108" Body="&lt;p&gt;I trying to understand the Bellman equation for updating the Q table values. The concept of initially updating the value is clear to me. What is unclear is the subsequent updates to the value. Is the value replaced with each episode?  It doesn't seem like this would learn from the past.  Maybe average the value from the previous episode with the existing value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not specifically from the book. I'm using the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) = \max_a(R(s, a) + \gamma V(s')),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the learning rate. &lt;span class=&quot;math-container&quot;&gt;$0.9$&lt;/span&gt; will encourage exploration, and &lt;span class=&quot;math-container&quot;&gt;$0.99$&lt;/span&gt; will encourage exploitation.  I'm working with a simple &lt;span class=&quot;math-container&quot;&gt;$3 \times 4$&lt;/span&gt; matrix from &lt;a href=&quot;https://www.youtube.com/watch?v=14BfO5lMiuk&quot; rel=&quot;nofollow noreferrer&quot;&gt;YouTube&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32525" LastEditorUserId="2444" LastEditDate="2020-01-06T14:03:03.607" LastActivityDate="2020-01-06T16:05:16.073" Title="Is the Q value updated at every episode?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17386" PostTypeId="1" CreationDate="2020-01-06T10:40:00.057" Score="3" ViewCount="103" Body="&lt;p&gt;Consider the following game on a MNIST dataset:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There are 60000 images. &lt;/li&gt;&#xA;&lt;li&gt;You can pick any 1000 images and train your Neural Network without access to the rest of images. &lt;/li&gt;&#xA;&lt;li&gt;Your final result is prediction accuracy on all dataset.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How to formalize this process in terms of information theory? I know that information theory works with distributions, but maybe you can provide some hints how to think in terms of datasets instead of distributions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the information size of all datasets. My first idea was&#xA;that each image is iid from uniform distribution and information&#xA;content = -log2(1/60000). But common sense and empirical results&#xA;(training neural network) show that there are similar images and&#xA;very different images holding a lot more information. For example if&#xA;you train NN only on good looking images of 1 you will get bad&#xA;results on unusual 1s.&lt;/li&gt;&#xA;&lt;li&gt;How to formalize that the right strategy is to choose as much as possible different 1000 images. I was thinking to take image by image with the highest entropy relative to the images you already have. How to define distance function.&lt;/li&gt;&#xA;&lt;li&gt;How to show that all dataset contains N bits of information, training dataset contain M bits of information and there is a way to choose K images &amp;lt; 60000 that hold &gt;99.9% of information.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32526" LastActivityDate="2020-01-06T15:11:45.857" Title="How to formalize learning in terms of information theory?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17388" PostTypeId="2" ParentId="17386" CreationDate="2020-01-06T15:03:37.873" Score="2" Body="&lt;p&gt;&lt;strong&gt;In short: It is easy to quantify information, but it is not easy to quantify its usefulness&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rAVIt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rAVIt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how exactly you are looking to formalise your experiment, but it might be helpful to consider these points: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no such thing as an absolute measure of information. The amount of information contained in some dataset is dependent on the underlying assumptions that are made when interpreting it, and therefore, the quantity of information conveyed is also dependent on the encoder/decoder (for example, a neural network). See the Wikipedia article on &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kolmogorov Complexity&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Entropy is a useful measure of information content when you assume each sample is iid, but this would be a very bad assumption to make for natural images, since they are highly structured. For example, imagine an image with 50% black pixels and 50% white pixels that can be arranged in any configuration - not matter how you arrange them, wether it looks like random noise, a text paragraph, or a chequer board, the entropy value will be identical for each, even though our intuition tells us otherwise (see attached image). The discrepancy between our intuition and the entropy value arises because our intuition does not interpret the image through the &quot;lens&quot; of iid pixels, but rather, hierarchical receptive fields in the visual cortex (somewhat analogous to convolutional neural networks).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculating the entropy of pixel values in one image is somewhat useful, but calculating the &quot;entropy&quot; of a set of images would not be useful, because each image as a whole is treated as if it were a unique arbitrary symbol. I assume this is what you meant by &quot;the information size of all datasets&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;KL-divergence&lt;/a&gt; is a distance function that is often used to compare two distributions. Intuitively, it represents the redundant bits generated by a non-ideal compression program that assumes an incorrect data distribution. However, KL-divergence between two natural images will not give you a particularly meaningful result.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If I am not mistaken, you want to find some information metric that will enable you to pick the smallest number of the most optimal images for training and get a good test score with the network. Is that correct? It is an interesting idea. However, in order to define such a metric, we might have to know in advance what features of an image are the most significant for classification, which in some ways defeats the point of using machine learning in the first place, where non-obvious and &quot;hidden&quot; features are exploited. &lt;/p&gt;&#xA;" OwnerUserId="32505" LastEditorUserId="32505" LastEditDate="2020-01-06T15:11:45.857" LastActivityDate="2020-01-06T15:11:45.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17389" PostTypeId="2" ParentId="17385" CreationDate="2020-01-06T16:05:16.073" Score="3" Body="&lt;p&gt;I think you are a bit confused about what is the update function and the target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation you have there, and what is done in the video is the estimation of the true value of a certain state. In Temporal-Difference algorithms this is called the TD-Target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for your confusion might be that in the video he starts from the end state and goes backwards using that formula to get the final value of each state. But that is not how you update the values, that is where you want to get to at the end of iterating through the states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The update formula may have several forms depending on the algorithm. For TD(0), which is  a simple 1-step look ahead off-policy where what is being evaluated is the state (as in your case), the update function is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;V(s) = (1 - \alpha) * V(s) + \alpha * (R(s,a) + \gamma V(s')),&#xA;$$&lt;/span&gt;&#xA;where alpha is the learning rate. What alpha will do is balance how much of your current estimate you want to change. You keep &lt;span class=&quot;math-container&quot;&gt;$1 - \alpha$&lt;/span&gt; of the original value and add &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; times the td-target, which uses the reward for the current state plus the discounted estimate of the value of the next state. Normal values for alpha can be 0.1 to 0.3, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The estimate will slowly converge into the real value of the state which is given by your equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;V(s) = \max_a(R(s, a) + \gamma V(s')).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is actually the discount associated with future states, as it is said in the video you referenced. It basically says how much importance you give to future states rewards. If &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt;, then you only care about the reward in your current state to evaluate it (this is not what is used). On the other extreme if &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; you will give as much value for a reward received in a state 5 steps ahead as you will to the current state. If you use some intermediate value you will give some importance to future rewards, but not as much as for the present one. The decay on the reward received on a state &lt;span class=&quot;math-container&quot;&gt;$n steps$&lt;/span&gt; in the future is given by &lt;span class=&quot;math-container&quot;&gt;$\gamma^n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that I would correct is that the exploration - exploitation balance is not in any way related to &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;. It is normally balanced by some policy, for example &lt;span class=&quot;math-container&quot;&gt;$\epsilon - greedy$&lt;/span&gt;. This one for example says that a certain % of the actions you take are random, which in turn makes you explore less valued states.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2020-01-06T16:05:16.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17390" PostTypeId="1" CreationDate="2020-01-06T16:15:53.370" Score="1" ViewCount="78" Body="&lt;p&gt;I am using the shapenet dataset. From this dataset, I have 3d models in .obj format. I rendered the images of these 3d models using pyrender library which gives me an image like this :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QGbzK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QGbzK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am using raycasting to voxelize this image. The voxel model I get is something like below :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Geqs6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Geqs6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not able to understand why I am getting the white or light brown colored artifacts in the boundary of the object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I could come up with was maybe the pixels at the boundary of the object contain two colors, so when I traverse the image as numpy array, I get an average of these two colors which gives me these artifacts. But I am not sure if this is the correct reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has any idea about what could be the reason, please let me know &lt;/p&gt;&#xA;" OwnerUserId="32534" LastActivityDate="2020-01-06T16:15:53.370" Title="Rendering images and voxelizing the images" Tags="&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17391" PostTypeId="2" ParentId="17379" CreationDate="2020-01-06T16:26:33.993" Score="3" Body="&lt;p&gt;For the precision metric for example you have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA; Precision = \frac{TP}{TP+FP},&#xA;$$&lt;/span&gt;&#xA;with TP = True Positive and FP = False Positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine you have the following values:&lt;br&gt;&#xA;Image 1: &lt;span class=&quot;math-container&quot;&gt;$TP = 2, FP = 3$&lt;/span&gt;&lt;br&gt;&#xA;Image 2: &lt;span class=&quot;math-container&quot;&gt;$TP = 1, FP = 4$&lt;/span&gt;&lt;br&gt;&#xA;Image 3: &lt;span class=&quot;math-container&quot;&gt;$TP = 3, FP = 0$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The precision scores as you calculated will be:&lt;br&gt;&#xA;Image 1: &lt;span class=&quot;math-container&quot;&gt;$2/5$&lt;/span&gt;&lt;br&gt;&#xA;Image 2: &lt;span class=&quot;math-container&quot;&gt;$1/5$&lt;/span&gt;&lt;br&gt;&#xA;Image 3: &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;&lt;br&gt;&#xA;Your average will be: &lt;span class=&quot;math-container&quot;&gt;$0.533$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand if you sum them all up and then calculate the precision value you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P = \frac{6}{6+7} = 0.462$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This proves that averaging the precision scores is not the same as calculating the total precision in one go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since what you want is to know how precise your algorithm is, independently of the precision for each image, you should sum all the TP and FP and only then calculate the precision for each model. &lt;strong&gt;This way you will not have a biased average. The average would give the same weight to an image with a larger number of objects as to another image which had fewer objects&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24054" LastEditorUserId="24054" LastEditDate="2020-01-07T02:45:31.117" LastActivityDate="2020-01-07T02:45:31.117" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17393" PostTypeId="1" AcceptedAnswerId="17397" CreationDate="2020-01-06T18:56:44.160" Score="3" ViewCount="66" Body="&lt;p&gt;I'm trying to understand distributional RL, based on this &lt;a href=&quot;https://mtomassoli.github.io/2017/12/08/distributional_rl/#mjx-eqn-eqclassic_td_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;. In one of the equations, there is a symbol &lt;span class=&quot;math-container&quot;&gt;$\operatorname{sup dist}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\operatorname{sup dist}_{s, a} (R(s, a) + \gamma Z(s', a^*), Z(s, a)) \\&#xA;s' \sim p(\cdot \mid s, a)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &lt;span class=&quot;math-container&quot;&gt;$\operatorname{sup dist}$&lt;/span&gt; mean? &lt;/p&gt;&#xA;" OwnerUserId="32540" LastEditorUserId="2444" LastEditDate="2020-01-07T11:45:23.777" LastActivityDate="2020-01-07T11:45:23.777" Title="What does the notation sup dist mean in distributional RL?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17394" PostTypeId="1" CreationDate="2020-01-06T19:03:49.300" Score="3" ViewCount="54" Body="&lt;p&gt;So we think a computer is dumb because it can only follow instructions. Therefor I am trying to create an AI that can &lt;em&gt;give&lt;/em&gt; instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is this: Create a geometric scene (A) then make a change in scene such as turning a square red or moving a circle right one unit. This becomes the new scene B. Then the computer compares the scenes A and B and it's goal is to give the shortest possible instruction that will change scene A to scene B. Examples might be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Turn the green square red&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Move the yellow square down&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or when we get more advanced we might have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Move the green square below the leftmost purple square down.&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Equally, this task could be seen as finding a description of the change. e.g. &quot;The green square has turned red&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way it would work is that there'd be a simplified English parser, and the computer would generate a number of phrases and check whether these achieved the desired result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would probably give it some prior knowledge of things like colours, shape-names, and so on. Or it could learn these by example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventally I would hope it to generate more complicated loop type expressions such as &quot;Move the square left &lt;strong&gt;until&lt;/strong&gt; it reaches the purple circle.&quot; And these would essentially be little algorithms the AI has generated in words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've got some ideas how to do this. But do you know any similar projects that I could look at? If you were implementing this, how would you go about it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[In other words we have an English parser that is interpreted to change a scene A into a scene B. But we want the AI to learn, given scenes A and B, how to generate instructions.]&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-01-06T19:03:49.300" Title="Creating an AI than can learn to give instructions" Tags="&lt;natural-language-processing&gt;&lt;generative-model&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17397" PostTypeId="2" ParentId="17393" CreationDate="2020-01-06T20:25:33.337" Score="3" Body="&lt;p&gt;It doesn't seem that it is a &quot;proper&quot; symbol. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that &lt;span class=&quot;math-container&quot;&gt;$\sup$&lt;/span&gt; simply refers to the supremum, that is, you want to select actions that maximize the quantity that comes to the right of &lt;span class=&quot;math-container&quot;&gt;$\sup$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$\text{dist}$&lt;/span&gt; is simply a proxy for any possible distance between distributions. For example, you can replace &lt;span class=&quot;math-container&quot;&gt;$\text{dist}$&lt;/span&gt; with the Kullback-Leibler divergence or with the mutual information.&lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2020-01-06T20:25:33.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17398" PostTypeId="1" AcceptedAnswerId="17422" CreationDate="2020-01-06T21:08:59.913" Score="2" ViewCount="90" Body="&lt;p&gt;I am quite new in the AI field. I am trying to create a neural network, in a language (Dart) where I couldn't find examples or premade libraries or tutorials. I've tried looking online for a strictly &quot;vanilla&quot; python implementation (without third-party libraries), but I couldn't find any. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found a single layer implementation, but it's done only with matrices and it's quite cryptic for a beginner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've understood the idea between the feed forwarding, a neuron calculates the sum of its inputs, adds a bias and activates it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I couldn't find anything a neuron-level explanation of the math behind backpropagation. (By neuron-level I think of the math down to the single neuron as a sequence of operations instead of multiple neurons treated as matrices). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the math behind it? Are there any resources to learn it that are suitable as a beginner?&lt;/p&gt;&#xA;" OwnerUserId="32545" LastEditorUserId="2444" LastEditDate="2020-01-07T11:42:04.710" LastActivityDate="2020-01-09T01:01:41.583" Title="What is the neuron-level math behind backpropagation for a neural network?" Tags="&lt;backpropagation&gt;&lt;math&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17399" PostTypeId="1" AcceptedAnswerId="17400" CreationDate="2020-01-07T07:33:38.227" Score="3" ViewCount="39" Body="&lt;p&gt;Good day everyone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious if it is possible for an AI to plot a time-series graph based on a single input.&#xA;Using free fall impact as an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming we drop a ball from height 100m and record the force it receives relative to time. &#xA;We will get a graph that looks something like below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PN769.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PN769.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we drop the ball from a height of 120m, record the forces, and we get another graph in addition to our original.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E5TN5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E5TN5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am wondering is:&#xA;If we have a large set of data on 60m to 140m (20m interval) height drops, would we be able to generate a regression model that plots the responses when given an arbitrary drop height? (i.e plot force response when dropped from 105m)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all very much for your time and attention.&lt;/p&gt;&#xA;" OwnerUserId="32551" LastActivityDate="2020-01-07T09:10:02.473" Title="Given enough graphical data, could you train an AI to plot a polynomial graph based on the input conditions?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17400" PostTypeId="2" ParentId="17399" CreationDate="2020-01-07T09:10:02.473" Score="2" Body="&lt;p&gt;Yes this is possible, using any machine learning approach that supports regression. You have two main approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Input &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the height of the drop, multiple outputs, one per time offset that you want to plot. Each individual output calculates the predicted force at a specific offset time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Inputs &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the height of the drop and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; a time offset, one output. The single output calculates the predicted force due to given height and at given time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The main thing to bear in mind is that statistical learning techniques typically do not generate physics-like models. Test inputs close to training examples should generate reasonable graphs that interpolate between those from training data. Test inputs far away from the training examples (e.g. you train on data of drops from 60m to 140m, but use an input of 10m or 200m) will likely generate wildly incorrect outputs. The main exception to this is if your ML model includes some good guesses at the underlying physics model, in which case it is possible that a regression algorithm will tune the parameters of that model plus filter out terms that should not be part of the model, resulting in a system that extrapolates much better. That is very unlikely happen by chance, it requires up-front design.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-01-07T09:10:02.473" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17401" PostTypeId="1" CreationDate="2020-01-07T11:12:26.593" Score="2" ViewCount="79" Body="&lt;p&gt;The light-field of a certain scene is the set of all light rays that travel through the volume of that scene at a specific point of time. A light-field camera for example captures and stores a subset of the lightfield of a scene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've got an unstructured subsampling of such a scene (a few billion rays, each having a direction and light intensity information).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I wish to do now is to create an approximation of the original scene that created this lightfield, with the aproximation consisting of 3 arbitrarily positioned (alpha-)textured 2D planes (in 3D space) where each point on the surface radiates light otwards uniformly in all directions based on the pixel color at this position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I guess this is like finding regions in the volume where similarly 'colored' rays intersect, such that the planes maximize the number of intersections they can cover.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the available data is the few billions of rays, the desired output is the parameters(position, normal and size) of the three planes plus one RGBA texture for each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking here about experiences and opinions: Is this problem rather well-suited for a machine learning approach or rather not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classical algorithm I could think of to solve this would be to voxelize the volume and use pathtracing to add a color sample for each ray to all cells along its way, then give each cell some value based on how similar all its contained samples are and then search for planar surfaces that intersect as many high rated cells as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But maybe machine learning is better suited for such a problem?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastEditorUserId="32111" LastEditDate="2020-01-07T16:38:20.440" LastActivityDate="2020-01-07T16:38:20.440" Title="Is this a problem well suited for machine learning?" Tags="&lt;ai-design&gt;&lt;theory&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17402" PostTypeId="2" ParentId="17369" CreationDate="2020-01-07T15:10:55.810" Score="0" Body="&lt;p&gt;A possible approach would be starting by grouping the actions into all possible groups.&#xA;For 30 actions grouped in a maximum of 6 this would mean:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;C^{30}_6 + C^{30}_5 + C^{30}_4 + C^{30}_3 + C^{30}_2 + C^{30}_1 = 768 211 &#xA;$$&lt;/span&gt;&#xA;Operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I would define the constraints to simplify the problem, as in a Constraint Satisfaction Problem (CSP). The main constraint that you have is that one action can only be represented in one Operation. So, if you choose 1 of the 768ks operations to start with, the second option will be restricted by the condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I would do some sort of planning and sum the costs until the end doing a depth-first search. Whenever I find a total cost better the previous best, you would update the best value. (Remember that the search ends when between all the operations used, all the actions were explored once.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While doing the search the costs are added up on the go. You can precompute a vector with the same size as the number of operations where each value corresponds to the total cost of the operation, and then sum it as you are going down the path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also add some heuristic to prevent checking the same operations in different orders. For example doing operating [2,3,4] followed by [7,10], where each number represents an action, is the same as doing: [7,10] followed by [2,3,4].&lt;/p&gt;&#xA;" OwnerUserId="24054" LastActivityDate="2020-01-07T15:10:55.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17403" PostTypeId="1" CreationDate="2020-01-07T15:32:09.623" Score="4" ViewCount="84" Body="&lt;p&gt;For example, I have a paragraph which I want to classify in a binary manner. But because the inputs have to have a fixed length, I need to ensure that every paragraph is represented by a uniform quantity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I've done is taken every word in the paragraph, vectorized it using GloVe word2vec and then summed up all of the vectors to create a &quot;paragraph&quot; vector, which I've then fed in as an input for my model. In doing so, have I destroyed any meaning the words might have possessed? Considering these two sentences would have the same vector:&#xA;&quot;My dog bit Dave&quot; &amp;amp; &quot;Dave bit my dog&quot;, how do I get around this? Am I approaching this wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other way can I train my model? If I take every word and feed that into my model, how do I know how many words I should take? How do I input these words? In the form of a 2D array, where each word vector is a column?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to be able to train a model that can classify text accurately. &#xA;Surprisingly, I'm getting a high (&gt;90%) for a relatively simple model like RandomForestClassifier just by using this summing up method. Any insights?&lt;/p&gt;&#xA;" OwnerUserId="32490" LastActivityDate="2020-01-09T09:03:39.410" Title="Does summing up word vectors destroy their meaning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;text-classification&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17404" PostTypeId="2" ParentId="17403" CreationDate="2020-01-07T15:48:27.073" Score="2" Body="&lt;p&gt;Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original numbers back. However summing up a sequence of word vectors may work depending on your task. You should also normalize the values, or just use average value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For details: &lt;a href=&quot;https://towardsdatascience.com/document-embedding-techniques-fed3e7a6a25d#ecd3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/document-embedding-techniques-fed3e7a6a25d#ecd3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To feed data with different lengths, you can also try padding and trimming it. Set a constant L for length of paragraph and trim/pad all list of word vectors to this length. Padding adds 0 vectors to the begining of the list and trimming trims the first part of a text until it is equal to L. Even in LSTM networks padding and trimming is still used as even though you can feed as long of a text to a LSTM as you want, you still have to process the word vectors in batches, which requires them to be the same length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example code in python on padding/trimming vector:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def pad_trim(list_vec, L):&#xA;    //list vec: {vec1,vec2,vec3......}&#xA;    //Assuming vecN have a size of 200&#xA;    return list_vec[-L:0] if len(list_vec) &amp;gt; L else [[0] * 200] * (L-len(list_vec)) + list_vec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However in inference, you can ignore maximum length if you used a RNN based method, although as the network has not been trained on lengths more than L, it may perform better or worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, you should go for concatenating if possible, so you can keep all information in the sentence. However both may work just fine depending on your task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For RNN based and CNN based model example, you should check this out:&#xA;&lt;a href=&quot;https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2020-01-07T23:33:12.077" LastActivityDate="2020-01-07T23:33:12.077" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17405" PostTypeId="2" ParentId="17217" CreationDate="2020-01-07T16:23:56.563" Score="0" Body="&lt;p&gt;I ran a lot of randomly created networks to solve this problem, but none of the structures where able to reliably &quot;solve&quot; this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, some of them where able to solve it one time, some of them even twice but there where only one which solved it 3 times:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;LearningRate: 0,510141694690167&lt;/li&gt;&#xA;&lt;li&gt;Momentum: 0,962972165068133; &lt;/li&gt;&#xA;&lt;li&gt;Layer/Neuron-Count: 2 (14, 9)&lt;/li&gt;&#xA;&lt;li&gt;SigmundAlphaValue: 2;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32255" LastActivityDate="2020-01-07T16:23:56.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17406" PostTypeId="1" CreationDate="2020-01-07T20:27:40.430" Score="2" ViewCount="67" Body="&lt;p&gt;I'm working on a genetic algorithm with a constraint on the sum of the alleles, e.g. if we use regular binary coding and a chromosome is 5-bits long I'd like to constrain it so that the sum of the bits has to be 3 or less (011100 is valid but 011110 is not). Moreover, the fitness function is such that invalid chromosomes cannot be evaluated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how this problem could be approached? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've started looking into the direction of messy GAs (since those can be over-specified) but I'm not sure if there's anything there.&lt;/p&gt;&#xA;" OwnerUserId="32573" LastEditorUserId="2444" LastEditDate="2020-01-07T21:57:07.067" LastActivityDate="2020-01-08T09:13:40.347" Title="How can I develop a genetic algorithm with a constraint on the sum of alleles?" Tags="&lt;genetic-algorithms&gt;&lt;genetic-programming&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17407" PostTypeId="1" AcceptedAnswerId="17408" CreationDate="2020-01-07T22:47:17.180" Score="2" ViewCount="127" Body="&lt;p&gt;I followed the videos/slides of &lt;a href=&quot;http://rail.eecs.berkeley.edu/deeprlcourse/static/slides/lec-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berkley RL course&lt;/a&gt;, but now I am a bit confused when implementing it. Please see the picture below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pfKil.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pfKil.png&quot; alt=&quot;lide 9&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, what does &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; represent in the REINFORCE algorithm? If &lt;span class=&quot;math-container&quot;&gt;$\tau^i$&lt;/span&gt; is the trajectory for the whole episode &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, then why don't we average across the episodes &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{N}$&lt;/span&gt;, which approximates the gradient of the objective function? Instead, it is a sum over the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. So, do we update the gradients per episode or have batches of episodes to update it? &#xA;When I compare the algorithm to Sutton's book as shown below, I see that there we update the gradients per episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Flg4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Flg4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But wouldn't it then contradict the derivation on the Levine's slide that the gradient of the objective function &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the expectation (therefore sampling) of the gradients of the logs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, why do we have a cumulative sum of the returns over &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; in Sutton's version but do not do it in Levine's (instead, all returns are summed together)   &lt;/p&gt;&#xA;" OwnerUserId="2254" LastEditorUserId="2444" LastEditDate="2020-01-08T20:54:33.030" LastActivityDate="2020-01-08T20:54:33.030" Title="What is the difference between Sutton's and Levine's REINFORCE algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17408" PostTypeId="2" ParentId="17407" CreationDate="2020-01-08T02:47:41.710" Score="2" Body="&lt;p&gt;About the first question, you are right. The &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; denotes a sample trajectory corresponding to a whole episode. However, Sutton's version is exactly the same one as Levine's if you choose &lt;span class=&quot;math-container&quot;&gt;$N=1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the second question, the Policy Gradient theorem only tells you what is the gradient up to a constant, so basically any constant is irrelevant. Now, even if you do know the constant, you are going to multiply the gradients by an arbitrary learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;. So, you can think that the factor &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{N}$&lt;/span&gt; is actually already considered &quot;inside&quot; &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="30983" LastEditorUserId="30983" LastEditDate="2020-01-08T03:02:51.393" LastActivityDate="2020-01-08T03:02:51.393" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17409" PostTypeId="2" ParentId="17406" CreationDate="2020-01-08T09:13:40.347" Score="1" Body="&lt;p&gt;There are multiple ways to handle 'illegal' individuals, each one with pros and cons:   &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Abortive methods&lt;/strong&gt;: The individuals that violate constraints are eliminated as soon as discovered (i.e. after crossover or mutation) and new individuals are generated in order to keep the population stable. This usually implies a slower creation of new generations, as some individuals are discarded. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Contraceptive methods&lt;/strong&gt;: The crossover and mutation are written in such a way to make it impossible for a newly generated individual to violate any constraints. This way to act is usually more efficient, as no individuals are discarded, but it might not be possible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Penalization function&lt;/strong&gt;: The fitness function gives a huge penalization to the individuals that violate constraints (usually proportionally to the constraints it violates). In this way they usually do not get to reproduce and their genes are eventually lost. You'd go this way if illegal individuals are very rare and there is no possibility for them to take over the full population (causing the algorithm to fail).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;By experience, try to act on the crossover algorithm first (&lt;em&gt;contraceptive way&lt;/em&gt;), it is the best way to exclude constraints violations. If this is not possible, pick one of the other two methods, depending on how often the illegal individuals are generated (not so often -&gt; &lt;em&gt;penalization&lt;/em&gt;, very ofter -&gt; &lt;em&gt;abortive&lt;/em&gt;)  and on how easy it is to penalize individuals that violate constraints (easy -&gt; &lt;em&gt;penalization&lt;/em&gt;, not easy -&gt; &lt;em&gt;abortive&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A contraceptive way to handle your example is writing the crossover as following:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bool[] crossover(bool[] p1, bool[] p2)&#xA;{&#xA;    // classical binary crossover&#xA;    bool[] child = p2.copy()&#xA;    child[rndSection] = p1[rndSection]&#xA;&#xA;    // contraceptive part&#xA;    while (CountTrue(child) &amp;gt; 3) child[RandomIndexOfTrue(child) = false&#xA;&#xA;    return child&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this is a fairly complex problem and each scenario has its own specifications.&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2020-01-08T09:13:40.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17410" PostTypeId="1" CreationDate="2020-01-08T10:15:45.840" Score="1" ViewCount="52" Body="&lt;p&gt;I'm trying to understand &lt;em&gt;Multiple Imputation with Chained Equation (MICE)&lt;/em&gt; imputation process (a statistical method for imputing missing data). I have read some articles and I have understood how the imputation happens, but I didn't get the &amp;quot;pooling&amp;quot; step.&lt;/p&gt;&#xA;&lt;p&gt;After analyzing the resulting datasets with Rubin's rules, how to pool these datasets? How to get only one dataset?&lt;/p&gt;&#xA;&lt;p&gt;In the end, do I combine all these datasets? If yes, how? Or do I compare every dataset's estimators with Rubin's estimators and choose one dataset?&lt;/p&gt;&#xA;" OwnerUserId="32560" LastEditorUserId="2444" LastEditDate="2021-03-03T09:51:02.257" LastActivityDate="2021-04-02T10:05:30.557" Title="How exactly does MICE imputation combine multiple datasets into one?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;mice&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17411" PostTypeId="2" ParentId="17371" CreationDate="2020-01-08T10:30:59.607" Score="0" Body="&lt;p&gt;@Clement Hui&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your answer, I ask AlexeyAB from Darknet the same question and he add now flag for Darknet for this type of model speed measurments:&#xA;&lt;a href=&quot;https://github.com/AlexeyAB/darknet/issues/4627&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/AlexeyAB/darknet/issues/4627&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I added -benchmark flag for detector demo, now you can use command 2652263&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;./darknet detector demo obj.data yolo.cfg yolo.weights test.mp4 -benchmark&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But for very fast models the bottleneck will be in the Video Capturing from file/camera, &amp;gt;or in Video Showing (you can disable showing by using -dont_show flag).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think that it is the best solution, you only need the newest version of Darknet (from AlexeyAB).&lt;/p&gt;&#xA;" OwnerUserId="30992" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-01-08T10:30:59.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17412" PostTypeId="1" CreationDate="2020-01-08T11:25:36.603" Score="3" ViewCount="3103" Body="&lt;p&gt;I want to see the effects of changing some training parameters (batch size, learning rate, optimizer...) to the accuracy obtained.&#xA;The problem is that with the same parameters I get significantlly different results (up to 5%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I load the same weights before training and I deactivated the shuffle. To my understanding this should be enough to get the same results. The only randomness of the backpropagation comes from the initial weight values, right? What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I compiled the model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(loss=&quot;categorical_crossentropy&quot;,&#xA;              optimizer=keras.optimizers.SGD(lr=0.1, decay=1e-6),&#xA;              metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the code that I repeat:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.load_weights(&quot;./saved_weights.h5&quot;, by_name=False)&#xA;&#xA;model.fit(x_train, y_train,&#xA;          batch_size=32,&#xA;          epochs=1,&#xA;          initial_epoch=0,&#xA;          validation_data=(x_test, y_test),&#xA;          shuffle=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm using tensorflow 2.0 as a backend in a colabs's CPU.&lt;/p&gt;&#xA;" OwnerUserId="29791" LastEditorUserId="29791" LastEditDate="2020-01-08T12:24:44.170" LastActivityDate="2020-06-17T14:23:56.760" Title="How to reproduce neural network training with keras" Tags="&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="2" ClosedDate="2020-06-17T14:31:18.553" ContentLicense="CC BY-SA 4.0" />
  <row Id="17413" PostTypeId="2" ParentId="17412" CreationDate="2020-01-08T11:38:59.763" Score="2" Body="&lt;p&gt;Use seed for random functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if you are using numpy random function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy.random import seed&#xA;seed(1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Read more about reproducible results here, &lt;a href=&quot;https://machinelearningmastery.com/reproducible-results-neural-networks-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/reproducible-results-neural-networks-keras/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Set &lt;code&gt;PYTHONHASHSEED&lt;/code&gt; environment variable at a fixed value&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;os.environ['PYTHONHASHSEED'] = str(1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32408" LastEditorUserId="32408" LastEditDate="2020-01-08T13:00:36.973" LastActivityDate="2020-01-08T13:00:36.973" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="17415" PostTypeId="2" ParentId="17412" CreationDate="2020-01-08T13:41:31.200" Score="4" Body="&lt;p&gt;As @codeblooded said, you should set random seed for numpy and keras, and also set pythonhashseed. The seeds set the state of the random number generator which makes the results different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method only works when you train the network on CPU. The problem with getting same result on GPU every single time is that cuDNN is not deterministic. Specifically, the reduce_sum() function in tensorflow is known to have different results even if a random seed is set. You can see &lt;a href=&quot;https://www.twosigma.com/insights/article/a-workaround-for-non-determinism-in-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; for more details. But simply said, reduce sum is not deterministic in tensorflow, but however it is used in bias gradient calculation. This can be avoided by using matmul in place of reduce sum, and you can see the implementation in the article mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get reproducible results, you can try using different framework, like pytorch. I have done experiments on pytorch and it seems to be reproducible. See this &lt;a href=&quot;https://colab.research.google.com/drive/1-mh5TJidyHdvEuDR2nTqk1exP4Izry8-&quot; rel=&quot;nofollow noreferrer&quot;&gt;colab notebook&lt;/a&gt;. All you need to do is add a &lt;a href=&quot;https://pytorch.org/docs/stable/notes/randomness.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;couple lines&lt;/a&gt; to the code. Theano based keras &lt;a href=&quot;https://datascience.stackexchange.com/questions/14812/making-keras-tensorflow-code-execution-deterministic-on-a-gpu&quot;&gt;seems to work&lt;/a&gt; as well but I haven't tested it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However for hyperparameter testing and searching 0.3% should not affect the result, and if you really want very accurate result, average the accuracy of 30 or more tries to get an accurate result. If you do this you don't need to set the random seed.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="23713" LastEditDate="2020-01-08T13:59:48.170" LastActivityDate="2020-01-08T13:59:48.170" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="17417" PostTypeId="2" ParentId="9022" CreationDate="2020-01-08T16:32:30.303" Score="0" Body="&lt;p&gt;I am watching the same course too, and I think that in the example graph, the cost function is not a sum of MSE (Mean squarred errors), but it could be a cubic one, so a sum of cubical errors, and thus the cost function could be negative: as there are a variety of cost functions, the MSE ones are not adapted for every problems, and other formulations could work better.&lt;/p&gt;&#xA;" OwnerUserId="32594" LastEditorUserId="23713" LastEditDate="2020-01-09T08:13:14.300" LastActivityDate="2020-01-09T08:13:14.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17418" PostTypeId="1" CreationDate="2020-01-08T17:02:46.913" Score="2" ViewCount="38" Body="&lt;p&gt;I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Order mister X to pay EUR 5000 for compensation for unpaid wages &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To cover damages, mister X must pay EUR 4000 to mister Y &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I want to make an algorithm that is able from this sentence to extract the motive of legal compensation. For the first sentence &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Order mister X to pay EUR 5000 for compensation for unpaid wages &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the algorithm's output must be &quot;compensation for unpaid wages&quot; or &quot;compensation unpaid wages &quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second sentence, the algorithm's output must be &quot;cover damages&quot;. Output can be a string or a list of string, it doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I'm not an NLP expert (but I have already worked on a project on sentiment analysis, so I know some stuff about NLP), and there are so many articles, I don't know where to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on French texts, but I can get away with working on English texts.&lt;/p&gt;&#xA;" OwnerUserId="32597" LastEditorUserId="2444" LastEditDate="2020-01-08T20:59:22.207" LastActivityDate="2020-01-08T20:59:22.207" Title="How can I extract the reason of the legal compensation from a court report?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17420" PostTypeId="2" ParentId="17412" CreationDate="2020-01-08T21:43:23.343" Score="3" Body="&lt;p&gt;See comprehensive answer &lt;a href=&quot;https://stackoverflow.com/questions/59075244/if-keras-results-are-not-reproducible-whats-the-best-practice-for-comparing-mo/59075958#59075958&quot;&gt;here&lt;/a&gt;; to paste a snippet, below is complete code for fixing a random seed:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def reset_seeds(reset_graph_with_backend=None):&#xA;    if reset_graph_with_backend is not None:&#xA;        K = reset_graph_with_backend&#xA;        K.clear_session()&#xA;        tf.compat.v1.reset_default_graph()&#xA;        print(&quot;KERAS AND TENSORFLOW GRAPHS RESET&quot;)  # optional&#xA;&#xA;    np.random.seed(1)&#xA;    random.seed(2)&#xA;    tf.compat.v1.set_random_seed(3)&#xA;    print(&quot;RANDOM SEEDS RESET&quot;)  # optional&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The TF graph must be reset, else resetting the TF random seed won't take full effect. &lt;code&gt;K&lt;/code&gt; is the backend, for example &lt;code&gt;import keras.backend as K&lt;/code&gt; or &lt;code&gt;import tensorflow.keras.backend as K&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further reading&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/58433555/why-are-my-results-still-not-reproducible/58444460#58444460&quot;&gt;GPU Parallelism&lt;/a&gt; on inherent non-reproducibility&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/58210700/are-these-models-equivalent/58210853#58210853&quot;&gt;How random seeds work&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: your case is actually unique relative to my linked answers; &lt;code&gt;load_weights&lt;/code&gt; is &lt;strong&gt;not&lt;/strong&gt; sufficient to restore your model's state - you must also load the &lt;strong&gt;optimizer&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All Keras optimizers have &lt;code&gt;self.weights&lt;/code&gt;, &lt;code&gt;self.updates&lt;/code&gt;, and &lt;code&gt;self.iterations&lt;/code&gt; attributes, which are &lt;em&gt;stateful&lt;/em&gt; - i.e. depend on the past history of iterations. For SGD without momentum, the former two shouldn't matter (am uncertain on this), but with &lt;code&gt;decay!=0&lt;/code&gt;, &lt;code&gt;self.iterations&lt;/code&gt; absolutely does, for &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/keras/optimizers.py#L425&quot; rel=&quot;nofollow noreferrer&quot;&gt;this line of code&lt;/a&gt;. If you don't load the optimizer state, then &lt;code&gt;self.iterations&lt;/code&gt; is reset to zero, thus your &lt;em&gt;actual learning rate&lt;/em&gt; may differ significantly from the originally saved model's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To remedy, you can either save everything via &lt;code&gt;model.save(include_optimizer=True)&lt;/code&gt;, or save the optimizer and its &lt;code&gt;self.iterations&lt;/code&gt; attribute separately. Luckily, I do this all the time, and use my own custom method; in its current shape, it's not the most user-friendly, but I can polish it up and share it if interested.&lt;/p&gt;&#xA;" OwnerUserId="32165" LastEditorUserId="32165" LastEditDate="2020-01-09T18:23:50.797" LastActivityDate="2020-01-09T18:23:50.797" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17421" PostTypeId="1" CreationDate="2020-01-08T22:01:39.343" Score="4" ViewCount="158" Body="&lt;p&gt;As experiment, I have tried using an autoencoder to encode height data from the alps, however the decoded image is very pixellated after training for several hours as show in the image below. This repeating patter is larger than the final kernel size, so I would think it would possible to remove these repeating patterns from the image to some extent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image is (1, 512, 512) and is sampled down to (16, 32, 32). This is done with pytorch. Here is the relevant sample of the code in which the exact layers are shown.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        self.encoder = nn.Sequential(&#xA;                # Input is (N, 1, 512, 512)&#xA;                nn.Conv2d(1, 16, 3, padding=1), # Shape (N, 16, 512, 512)&#xA;                nn.Tanh(),&#xA;                nn.MaxPool2d(2, stride=2), # Shape (N, 16, 256, 256)&#xA;                nn.Conv2d(16, 32, 3, padding=1), # Shape (N, 32, 256, 256)&#xA;                nn.Tanh(),&#xA;                nn.MaxPool2d(2, stride=2), # Shape (N, 32, 128, 128)&#xA;                nn.Conv2d(32, 32, 3, padding=1), # Shape (N, 32, 128, 128)&#xA;                nn.Tanh(),&#xA;                nn.MaxPool2d(2, stride=2), # Shape (N, 32, 64, 64)&#xA;                nn.Conv2d(32, 16, 3, padding=1), # Shape (N, 16, 64, 64)&#xA;                nn.Tanh(),&#xA;                nn.MaxPool2d(2, stride=2) # Shape (N, 16, 32, 32)&#xA;            )&#xA;        self.decoder = nn.Sequential(&#xA;                # Transpose convolution operator&#xA;                nn.ConvTranspose2d(16, 32, 4, stride=2, padding=1), # Shape (N, 32, 64, 64)&#xA;                nn.Tanh(),&#xA;                nn.ConvTranspose2d(32, 32, 4, stride=2, padding=1), # Shape (N, 32, 128, 128)&#xA;                nn.Tanh(),&#xA;                nn.ConvTranspose2d(32, 16, 4, stride=2, padding=1), # Shape (N, 32, 256 256)&#xA;                nn.Tanh(),&#xA;                nn.ConvTranspose2d(16, 1, 4, stride=2, padding=1), # Shape (N, 32, 512, 512)  &#xA;                nn.ReLU()&#xA;            )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VU6u1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VU6u1.png&quot; alt=&quot;Relevant image: left side original, right side result from autoencoder&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relevant image: left side original, right side result from autoencoder&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So could these pixellated effects in the above image be resolved?&lt;/p&gt;&#xA;" OwnerUserId="10364" LastActivityDate="2020-07-01T00:31:43.213" Title="Autoencoder produces repeated artifacts after convergence" Tags="&lt;convolutional-neural-networks&gt;&lt;autoencoders&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17422" PostTypeId="2" ParentId="17398" CreationDate="2020-01-09T01:01:41.583" Score="1" Body="&lt;p&gt;Backpropagation is actually a lot easier than it is made out to be - &lt;em&gt;if&lt;/em&gt; you have a basic understanding of calculus and the chain rule, and the single multi-variable calculus rule that to combine 2 gradient vectors, you simply add them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/d14TUNcbn1k?t=354&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is hands down the best walk through of back prop I've found on the internet.&lt;/a&gt; If you are still confused after that, feel free to ask me any further questions. Here is also a quick forward and backward pass example I made for a simple CNN (only a few layers though, and the gradient only goes back to channel 1 of filter 1)&lt;a href=&quot;https://i.stack.imgur.com/hxZBv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hxZBv.png&quot; alt=&quot;backprop&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-01-09T01:01:41.583" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17423" PostTypeId="1" CreationDate="2020-01-09T03:41:04.350" Score="3" ViewCount="52" Body="&lt;p&gt;I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with &lt;code&gt;model.fit_generator()&lt;/code&gt; (and not &lt;code&gt;model.fit()&lt;/code&gt;), to load just parts of the training data when needed, because the training data is too large to load at once.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the training the model was not working. So I checked training data (before and after augmentation). The training data are correct. So I thought the reason why the model does not work must be that I have not found the optimal hyperparameters yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can I do hyperparameter optimization on a network with multiple inputs and outputs and trained with &lt;code&gt;model.fit_generator()&lt;/code&gt;? All I can find online is hyperparameter optimization of networks with a single input and single output and trained with &lt;code&gt;model.fit()&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="32606" LastEditorUserId="2444" LastEditDate="2020-01-09T14:22:00.023" LastActivityDate="2020-01-09T14:22:00.023" Title="How can I do hyperparameter optimization for a CNN-LSTM neural network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17424" PostTypeId="1" AcceptedAnswerId="17430" CreationDate="2020-01-09T06:08:24.873" Score="8" ViewCount="1613" Body="&lt;p&gt;I have a data set which was split using a fixed random seed and I am going to use 80% of data for training and rest on validation. Here are my GPU and batch size configurations&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use &lt;code&gt;64 batch size&lt;/code&gt; with &lt;code&gt;one GTX 1080Ti&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;use &lt;code&gt;128 batch size&lt;/code&gt; with &lt;code&gt;two GTX 1080Ti&lt;/code&gt; &lt;/li&gt;&#xA;&lt;li&gt;use &lt;code&gt;256 batch size&lt;/code&gt; with &lt;code&gt;four GTX 1080Ti&lt;/code&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All other hyper-parameters such as &lt;code&gt;lr&lt;/code&gt;, &lt;code&gt;opt&lt;/code&gt;, &lt;code&gt;loss&lt;/code&gt;, etc., are fixed. Notice the linearity between the &lt;code&gt;batch size&lt;/code&gt; and the &lt;code&gt;number of GPUs&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will I get the same accuracy for those three experiments? Why and why not?&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="23713" LastEditDate="2020-01-09T11:58:12.963" LastActivityDate="2020-01-09T15:21:23.953" Title="Effect of batch size and number of GPUs on model accuracy" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;gpu&gt;&lt;accuracy&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17425" PostTypeId="1" CreationDate="2020-01-09T06:09:25.540" Score="1" ViewCount="23" Body="&lt;p&gt;I would like to create a GloVe word embedding on a very large corpus (trillions of words). However, creating the co-occurence matrix with the GloVe cooccur script is projected to take weeks. Is there any way to parallelize the process of creating a co-occurence matrix, either using GloVe or another resource that is out there? &lt;/p&gt;&#xA;" OwnerUserId="31294" LastActivityDate="2020-01-09T06:09:25.540" Title="Is there a way to parallelize GloVe cooccur function?" Tags="&lt;word-embedding&gt;&lt;glove&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17426" PostTypeId="1" CreationDate="2020-01-09T07:33:04.233" Score="1" ViewCount="46" Body="&lt;p&gt;I am trying to extract product information from email receipts HTML.  Most services I have found focus on OCR from paper receipts or PDFs.  I would imagine that extraction of product information would be easier from structured HTML.  What type of AI approach would be used to support this?&lt;/p&gt;&#xA;" OwnerUserId="32611" LastActivityDate="2020-01-09T10:04:38.027" Title="Extract product information from email receipt HTML" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17427" PostTypeId="2" ParentId="17403" CreationDate="2020-01-09T09:03:39.410" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But because the inputs have to have a fixed length&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Do they?  Why?  The go-to strategy would be to use an RNN (possibly with LSTM or GRUs, but probably not necessary) and train it to process input sequentially and output the final classification of the paragraph.  This has the advantage of being able to take into account word order and constellations, as well as processing variable size inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I would think simply adding word vectors will include a lot of noise from commonly occurring words that don't provide much meaningful information from the classification.  I would consider Bayesian methods or dimensionality reduction methods to limit the input to the more useful input vectors.&lt;/p&gt;&#xA;" OwnerUserId="29720" LastActivityDate="2020-01-09T09:03:39.410" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17428" PostTypeId="2" ParentId="17424" CreationDate="2020-01-09T09:07:06.200" Score="3" Body="&lt;p&gt;No.  Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, how this is actually parallelized might make a difference, including the order of operations and converting between FP precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional check resources:&lt;a href=&quot;https://stackoverflow.com/questions/43845644/multi-gpu-architecture-gradient-averaging-less-accurate-model&quot;&gt;issue of multi gpus&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29720" LastEditorUserId="15368" LastEditDate="2020-01-09T15:21:23.953" LastActivityDate="2020-01-09T15:21:23.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17429" PostTypeId="2" ParentId="17426" CreationDate="2020-01-09T10:04:38.027" Score="1" Body="&lt;p&gt;It depends on the data. If it is structured like form data, then you might not need AI at all &amp;mdash; simple regular expression patterns might be fine. This would apply for example to address data. If you have the word &lt;em&gt;street&lt;/em&gt; followed by a colon, followed by some text, it seems fairly obvious that this is the name of a street, and possibly also a house number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, however, you have free text, eg the answer to &quot;describe any medical conditions you have&quot;, or &quot;which companies have you worked at before?&quot;, then you might need to look at named entity extraction (&lt;a href=&quot;https://en.wikipedia.org/wiki/Named-entity_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;NER&lt;/a&gt;) to identify names of medical conditions or companies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, some Natural Language Processing (&lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLP&lt;/a&gt;) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Information_extraction&quot; rel=&quot;nofollow noreferrer&quot;&gt;information extraction&lt;/a&gt; might be required apart from simple pattern matching.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-01-09T10:04:38.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17430" PostTypeId="2" ParentId="17424" CreationDate="2020-01-09T11:31:11.890" Score="9" Body="&lt;p&gt;This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a batch size too small(for example 1) will make the model hard to generalize and also slower to converge.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cb2hu.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cb2hu.png&quot; alt=&quot;Batch size slide&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://cs230.stanford.edu/files/C2M2.pdf&quot; rel=&quot;noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This slide is a great demonstration of how batch size affects training. As you can see from the diagram, when you have a small batch size the route to convergence will be ragged and not direct. This is because the model may train on an outlier and have it's performance decrease before fitting again. Of course this is an edge case and you would never train a model with 1 batch size. However on the other hand with a batch size too large, your model will take too long per iteration. With at least a decent batch size (like 16+) the amount of iterations needed to train the model is similar, so a larger batch size is not going to help a lot. The performance is not going to vary a lot.&lt;/p&gt;&#xA;&lt;p&gt;In your case, the accuracy will make a difference but only minimally. Whilst writing this answer, I have ran a few test on batch size effect on performance and time, and here is the results. (Results to be added for 1 batch size)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Batch size 256 Time required 98.50849771499634s : 0.9414&#xA;Batch size 128 Time required 108.53689193725586s : 0.9668&#xA;Batch size 64 Time required 129.92272853851318s : 0.9776&#xA;Batch size 32 Time required 162.13709354400635s : 0.9844&#xA;Batch size 16 Time required 224.82269191741943s : 0.9854&#xA;Batch size 8 Time required 351.2729814052582s : 0.9861&#xA;Batch size 4 Time required 514.2667407989502s : 0.9862&#xA;Batch size 2 Time required 829.1623721122742s : 0.9869&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can test out yourself &lt;a href=&quot;https://colab.research.google.com/drive/1xJEkyLd_02XVRYF6bmLgQ5MAXV2palCu&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dcD7R.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dcD7R.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, accuracy increase with the batch size decreasing and this is because a higher batch size means it will be trained on less iterations. 2x batch size = half the iterations so this is expected. The time required have risen exponentially, but batch size of 32 or below don't seems to make a large difference in time taken. The accuracy seems to be normal as half the iterations are trained with double the batch size.&lt;/p&gt;&#xA;&lt;p&gt;In your case I would actually recommend you stick with 64 batch size even for 4 GPU. In the case of multi-gpu, the rule of thumb will be using at least 16 or so batch size per GPU, as if you are using 8 or 4 batch size the GPU cannot be completely utilized to train the model.&lt;/p&gt;&#xA;&lt;p&gt;For multiple GPU accuracy impact, there is virtually no impact on accuracy as far as I know. Multiple GPU just take it's own slice of batches and computes it and feed the output to one GPU to calculate the loss, and the operations done across the GPUs are the same operations done on 1 GPU, so it shouldn't matter.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;For multiple GPU, there might me a slight difference due to percision error as told by @Posi2 . Please see &lt;a href=&quot;https://stackoverflow.com/q/43845644/10675215&quot;&gt;here&lt;/a&gt;. Thanks @Posi2&lt;/p&gt;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&lt;p&gt;Batch size don't matter to performance too much as long as you set a reasonable batch size (16+) and keep the &lt;strong&gt;iterations&lt;/strong&gt; not epochs the same. However training time will be affected and for multi-gpu you should use the minimum batch size for each GPU that will utilize 100% of the GPU to train. 16 per GPU is quite good.&lt;/p&gt;&#xA;" OwnerUserId="23713" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-01-09T11:48:34.823" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17431" PostTypeId="1" CreationDate="2020-01-09T13:43:53.993" Score="4" ViewCount="34" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Set_(card_game)&quot; rel=&quot;noreferrer&quot;&gt;Set is a card game&lt;/a&gt; and is &lt;a href=&quot;https://codegolf.stackexchange.com/questions/195023/play-the-card-game-set&quot;&gt;Nicely described here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each set-card has 4 properties: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The number(1,2 or 3)&lt;/li&gt;&#xA;&lt;li&gt;the color (Red, Green or Purple)&lt;/li&gt;&#xA;&lt;li&gt;Fill (Full, Stripes, None)&lt;/li&gt;&#xA;&lt;li&gt;Form (Wave, Oval or Diamond)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KDBd2.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KDBd2.gif&quot; alt=&quot;2PWN&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;converts to 2 Purple Waves No fill (code: 2PWN)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mz20s.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mz20s.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; and &lt;a href=&quot;https://i.stack.imgur.com/N6vZe.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N6vZe.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;convert to codes 1RON and 3GDN&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every combination there is one card so in total there are 3^4 = 81 cards. The goal is to identify 3 cards (set) out of collection of 12 displayed randomly chosen set cards where all properties occur 0,1 or 3 times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a hobby project I want to create an android app which can -with the camera- capture the 12 (less or more) &lt;code&gt;set cards&lt;/code&gt; and indicate the sets present in the collection of 12. I'm looking for ways to leverage image recognition as efficient as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been thinking of taking multiple pictures of all the individual cards, label them and feeding them to a trainer (firebase ML KIT AutoML Vision Edge) But I have the feeling that this a bit of brute force and takes a lot of time and effort photographing and labeling. I could also take pictures of multiple set cards and provide the different codes as labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best (most efficient) approach to have a model for labelling all cards?&lt;/p&gt;&#xA;" OwnerUserId="32616" LastEditorUserId="23713" LastEditDate="2020-01-09T16:04:04.030" LastActivityDate="2020-01-11T13:10:34.723" Title="Recognizing Set CARDs" Tags="&lt;image-recognition&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17432" PostTypeId="1" CreationDate="2020-01-09T14:55:30.020" Score="4" ViewCount="85" Body="&lt;p&gt;In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a mathematical formula that matches these curves?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e8og1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e8og1.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some similar curves are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y=1-e^{-x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y=\frac{x}{1+x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y=\tanh(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y=1+x-\sqrt{1+x^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a theoretical reason for this shape?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastEditorUserId="4199" LastEditDate="2020-01-09T19:10:54.903" LastActivityDate="2020-01-09T19:10:54.903" Title="Is there a mathematical formula that describes the learning curve in neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;learning-curve&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17433" PostTypeId="1" CreationDate="2020-01-09T16:32:39.353" Score="5" ViewCount="65" Body="&lt;p&gt;I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. Was there a study on this? Or are there any past experiences anyone could share to help?&lt;/p&gt;&#xA;" OwnerUserId="32622" LastEditorUserId="32622" LastEditDate="2020-01-11T08:30:50.873" LastActivityDate="2021-05-29T04:05:20.413" Title="Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?" Tags="&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="17434" PostTypeId="2" ParentId="7359" CreationDate="2020-01-09T19:34:37.757" Score="1" Body="&lt;p&gt;A trajectory ist just a &lt;strong&gt;sequence of states and actions&lt;/strong&gt;. In RL, the goal is to maximize the reward, by finding the right trajectories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{max}_\tau R(\tau)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means&#xA;maximizing not immediate reward (caused by one action from a state), but cumulative reward (all states and actions: trajectory)&lt;/p&gt;&#xA;" OwnerUserId="31753" LastEditorUserId="31753" LastEditDate="2020-01-10T10:17:31.147" LastActivityDate="2020-01-10T10:17:31.147" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17435" PostTypeId="1" AcceptedAnswerId="17437" CreationDate="2020-01-09T20:57:16.390" Score="3" ViewCount="96" Body="&lt;p&gt;I modeled the TicTacToe game as a RL problem - with an environment and an agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/MaverickMeerkat/Reinforcement-Learning/blob/master/TicTacToe/TicTacToe%20using%20SARSA%20-%20Part%201.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;At first&lt;/a&gt; I made an &quot;Exact&quot; agent - using the SARSA algorithm, I saved every unique state, and chose the best (available) action given that state. I made 2 agents learn by competing against each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agents learned fast - it took only 30k games for them to reach a tie stand-off. And the agent clearly knew how to play the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;a href=&quot;https://github.com/MaverickMeerkat/Reinforcement-Learning/blob/master/TicTacToe/TicTacToe%20using%20SARSA%20-%20Part%202.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;then tried&lt;/a&gt; to use function approximation instead of saving the exact state. My function was a FF-NN. My 1st (working) architecture was 9 (inputs) x 36 x 36 x 9 (actions). I used the semi-gradient 1-step SARSA. The agents took much longer time to learn. After about 50k games they were still less good than the exact agent. I then made a stand off between the Exact and the NN agent - the exact agent won 1721 games from 10k, the rest were tied. Which is not bad. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;a href=&quot;https://github.com/MaverickMeerkat/Reinforcement-Learning/blob/master/TicTacToe/TicTacToe%20using%20SARSA%20-%20part%202.5.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;then&lt;/a&gt; tried reducing the number of units in the hidden layers to 12, but didn't get good results (even after playing for 500k+ games total, tweaking stuff). I &lt;a href=&quot;https://github.com/MaverickMeerkat/Reinforcement-Learning/blob/master/TicTacToe/TicTacToe%20using%20SARSA%20-%20part%202.75%20-%20Convolution%20NN.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;also&lt;/a&gt; tried playing with convolution architectures, again - not getting anywhere. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there's some optimal function approximation solution that can get as-good of results as the exact agent. TicTacToe doesn't seem like such a hard problem for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceptually I think there should be much less complexity involved in solving it then can be expressed in a 9x36x36x9 network. Am I wrong, and it's just an illusion of simplicity? Or are there better ways? Maybe modeling the problem differently? &lt;/p&gt;&#xA;" OwnerUserId="27947" LastActivityDate="2020-01-09T22:35:49.277" Title="Optimal RL function approximation for TicTacToe game" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17436" PostTypeId="1" CreationDate="2020-01-09T22:29:09.673" Score="6" ViewCount="134" Body="&lt;p&gt;When a neural network learns something from a data set, we are left with a bunch of weights which represent some approximation of knowledge about the world.  Although different data sets or even different runs of the same NN might yield completely different sets of weights, the resulting equations must be mathematically similar (linear combinations, rotations, etc.).  Since we usually build NNs to model a particular concrete task (identify cats, pedestrians, tumors, etc.), it seems that we are generally satisfied to let the network continue to act as a black box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I understand that there is a push for &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;understandability&lt;/a&gt;&quot; of NNs, other ML techniques, etc.  But this is not quite what I'm getting at.  It seems to me that given a bunch of data points recording the behavior of charged particles, one could effectively recover Maxwell's laws using a sufficiently advanced NN.  Perhaps that requires NNs which are much more sophisticated than what we have today.  But it illustrates the thing I &lt;em&gt;am&lt;/em&gt; interested in: NNs could, in my mind, be teaching us general truths about the world if we took the time to analyze and simplify the formulae that they give us&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, there must be hundreds, if not thousands of NNs which have been trained on visual recognition tasks that end up learning many of the same sub-skills, to put it a bit anthropomorphically.  I recently read about &lt;a href=&quot;https://towardsdatascience.com/an-easy-guide-to-gauge-equivariant-convolutional-networks-9366fb600b70&quot; rel=&quot;noreferrer&quot;&gt;gauge CNNs&lt;/a&gt;, but this goes the other way: we start with what we know and then bake it into the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone attempted to go the opposite way?  Either:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take a bunch of similar NNs and analyze what they have in common to extract general formulae about the focus area&lt;sup&gt;&lt;a href=&quot;https://towardsdatascience.com/an-easy-guide-to-gauge-equivariant-convolutional-networks-9366fb600b70&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Carefully inspect the structure of a well-trained NN to directly extract the &quot;Maxwell's equations&quot; which might be hiding in them?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Explainable_artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt; Imagine if we built a NN to learn Newtonian mechanics just to compute a simple ballistic trajectory.  It could surely be done, but would also be a massive waste of resources.  We have nice, neat equations for ballistic motion, given to us by the &quot;original neural networks&quot;, so we use those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/an-easy-guide-to-gauge-equivariant-convolutional-networks-9366fb600b70&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt; E.g., surely the set of all visual NNs have collectively discovered near-optimal algorithms for edge/line/orientation detection, etc.).  This could perhaps be done with the assistance of a meta-ML algorithm (like, clustering over NN weight matrices?).&lt;/p&gt;&#xA;" OwnerUserId="32634" LastEditorUserId="2444" LastEditDate="2020-06-21T12:32:27.750" LastActivityDate="2020-06-21T12:32:27.750" Title="Has anyone attempted to take a bunch of similar neural networks to extract general formulae about the focus area?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;explainable-ai&gt;" AnswerCount="0" CommentCount="8" ClosedDate="2020-06-21T12:40:22.670" ContentLicense="CC BY-SA 4.0" />
  <row Id="17437" PostTypeId="2" ParentId="17435" CreationDate="2020-01-09T22:29:47.257" Score="2" Body="&lt;p&gt;I think you can break this problem down into two parts to try and find the solution.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1. Can the neural network model the desired function?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Take the tabular function you have learned in the exact agent, and treat it as training data for the neural network model, using the same loss function and other hyperparameters as you intend to use when the NN is being used online inside the RL inner loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can answer two related questions with this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the loss reduce down to a low value after a suitable number of epochs? If so, then the NN has capacity to learn and can learn fast enough. If not, you need to look at the hyperparameters of the NN.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the trained NN play well against the exact agent? Ideally it plays the same, but it is possible that even though the loss is low, one or two key values in the function are compromised, meaning it still loses. I am not entirely sure what you would do in this case, but either try changing the hyperparameters to increase the capacity of the NN, or try augmenting the data so that there are more examples of the &quot;difficult&quot; action values to learn, to see if the issue is something that can be solved in learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Probably&lt;/em&gt; you will find your NN architecture is good, or only requires minor changes to become useful. The more likely issues are in next section.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2. Is the RL framework set up correctly for function approximation?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;It is quite hard to get this right. Bootstrapping value based methods can easily become unstable if converted naively from tabular to function approximation approaches. Some variants are moderately stable - most stable would probably be Monte Carlo approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want to use Monte Carlo control, then the answer here would be to take ideas from DQN used originally to play Atari games:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't learn online. Store transitions in an experience replay table - store &lt;span class=&quot;math-container&quot;&gt;$(s, a, r, s', done)$&lt;/span&gt; tuples where &lt;span class=&quot;math-container&quot;&gt;$done$&lt;/span&gt; is true if &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is a terminal state - and sample a minibatch from it on every step. Reconstruct the bootstrap estimates of value functions to train from each time you sample, don't store and re-use the estimate from the time the action was taken.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Optionally use two value estimators - the current learning one, used to select plays and which is updated on each step, and a &quot;target&quot; one used to calculate TD targets. Update the target network by cloning the learning network every N steps (e.g. every 100 games).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To avoid figuring out hyperparameters for SARSA epsilon decay, I suggest use one-step Q learning. Also one issue you may be facing with SARSA combined epsilon decay is &quot;catastrophic forgetting&quot; where the agent gets good, starts to train itself only on play examples by good players, and forgets the values of states that it has not seen in the training data for many time steps. With Q learning you can avoid that by having a relatively high minimum epsilon e.g. 0.1. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In fact with TicTacToe learning through self-play, you can get away with &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 1$&lt;/span&gt; and Q learning should still work - i.e. it can learn optimal play by observing random self-play. This should apply equally tabular and function approximation agents. It doesn't scale to more complex games where random play would take too long to discover optimal strategies.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-09T22:35:49.277" LastActivityDate="2020-01-09T22:35:49.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17438" PostTypeId="1" CreationDate="2020-01-09T22:47:20.613" Score="1" ViewCount="45" Body="&lt;p&gt;I am using this code for the training and then how would I alter the process so when I run the test data inside the neural network so that it shall give the value of the missing variable (z)?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const x1 = {&#xA;  x: 0.9,&#xA;  y: 0.2,&#xA;  z: 0.05&#xA;}&#xA;&#xA;const x2 = {&#xA;  x: 0.01,&#xA;  y: 0.342,&#xA;  z: 0.2&#xA;}&#xA;&#xA;const x3 = {&#xA;    x: 0.0 ,&#xA;    y: 0.1,&#xA;    z: 0.65&#xA;}&#xA;&#xA;net.train([{&#xA;    input: x1,&#xA;    output: {&#xA;      x1: 1&#xA;    }&#xA;  }, {&#xA;    input: x2,&#xA;    output: {&#xA;      x2: 1&#xA;    }&#xA;  }, {&#xA;    input: x3,&#xA;    output: {&#xA;      x3: 1&#xA;    }&#xA;  }&#xA;]&#xA;);&#xA;const testdata = {&#xA;  x: 0.0,&#xA;  y: 0.0&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="32636" LastEditDate="2020-01-10T13:13:21.443" LastActivityDate="2020-01-10T13:13:21.443" Title="Brain.js Artificial Neural Network predicting missing variables value" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;javascript&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-01-10T20:12:46.803" ContentLicense="CC BY-SA 4.0" />
  <row Id="17439" PostTypeId="1" CreationDate="2020-01-09T23:11:58.073" Score="4" ViewCount="51" Body="&lt;p&gt;I would like to train an RNN to follow the sentences:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Would you like some cheese&quot;? with &quot;Yes, I would like some cheese.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So whenever the template &quot;Would you like some ____?&quot; appears then RNN produces the sequence above. And it should even work on sentences which are new like &quot;Would you like some blumf?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for &quot;repeat the character that is 14 characters to the left&quot; and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this been done before or is there a better way?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-01-28T19:40:41.287" Title="Training an RNN to answer simple quesitons" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17440" PostTypeId="2" ParentId="17433" CreationDate="2020-01-10T00:46:20.683" Score="0" Body="&lt;p&gt;To my knowledge the deployment model (that you will test on underwater images) as inference will not have a negative effect. Yet drawings may even help differentiate some classes at training and inference. Provided that you won't use drawings in inference, adding them in training phase will not necessarily hurt the accuracy. Note that a drawing of a particular class should not be in the search domains of other classes, namely, a drawing of a particular class should not be the same with the other classes.&lt;/p&gt;&#xA;" OwnerUserId="31870" LastActivityDate="2020-01-10T00:46:20.683" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17441" PostTypeId="1" AcceptedAnswerId="17448" CreationDate="2020-01-10T06:54:22.467" Score="2" ViewCount="1331" Body="&lt;p&gt;For a project I am doing, I found the paper &lt;a href=&quot;https://arxiv.org/pdf/1804.01005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Face Alignment in Full Pose Range: A 3D Total Solution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is using a cascaded convolutional neural network, but I wasn't able to find the original paper explaining what that is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In layman's terms and intuitively, how does a cascaded CNN work? What does it solve? &lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="2444" LastEditDate="2020-01-10T15:03:06.020" LastActivityDate="2020-01-10T15:03:06.020" Title="What is a cascaded convolutional neural network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;definitions&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17443" PostTypeId="2" ParentId="17431" CreationDate="2020-01-10T08:37:05.023" Score="2" Body="&lt;p&gt;Since you only have fixed types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;colour&lt;/code&gt;, I think it is fairly straight forward. &lt;br&gt;&#xA;For &lt;code&gt;number&lt;/code&gt;, simplest way is to plot a projection histogram and count the points of discontinuity. &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of the &lt;code&gt;projection histogram&lt;/code&gt; &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UGMWZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UGMWZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;fill&lt;/code&gt;, You can find the number of islands. Islands of background colour. &lt;br&gt;&#xA;For &lt;code&gt;shape&lt;/code&gt;, Like Clement Hui suggested you can use shape detection&lt;/p&gt;&#xA;" OwnerUserId="32408" LastEditorUserId="32408" LastEditDate="2020-01-11T13:10:34.723" LastActivityDate="2020-01-11T13:10:34.723" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17444" PostTypeId="1" CreationDate="2020-01-10T09:07:47.287" Score="1" ViewCount="29" Body="&lt;p&gt;I have lots of text documents structured as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;{&#xA;    Item1=[&#xA;            {a1=1,&#xA;             a2=2,&#xA;             a3=3},&#xA;            {a1=11,&#xA;             a2=22,&#xA;             a3=33},&#xA;            {a1=41,&#xA;             a2=52,&#xA;             a3=63},&#xA;            {a1=19,&#xA;             a2=29,&#xA;             a3=39}&#xA;    ],&#xA;    Item2=[&#xA;            {a4=1,&#xA;             a5=2,&#xA;             a6=3},&#xA;            {a4=11,&#xA;             a5=22,&#xA;             a6=33},&#xA;            {a4=41,&#xA;             a5=52,&#xA;             a6=63},&#xA;            {a4=19,&#xA;             a5=29,&#xA;             a6=39}&#xA;    ],&#xA;}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now this can be formatted into two csv's as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K8b70.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K8b70.png&quot; alt=&quot;Item 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vMb5O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vMb5O.png&quot; alt=&quot;Item 2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can write regex parser for this but is there a way by which a neural network or deep learning model can be trained for this, which can create these csvs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above example has been indented for better visuals, the raw text looks something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{{Item1=[{a1=1,a2=2,a3=3},{a1=11,a2=22,{a1=41,a2=52,a3=63},{a1=19,a2=29,a3=39}],Item2=[{a4=1,a5=2,a6=3},{a4=11,a5=22,a6=33},{a4=41,a5=52,a6=63},{a4=19,a5=29,a6=39}]}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28421" LastActivityDate="2020-01-10T09:07:47.287" Title="how to convert one structured data to another without specifying structure" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;pattern-recognition&gt;&lt;structured-data&gt;&lt;data-mining&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17445" PostTypeId="1" CreationDate="2020-01-10T10:25:38.987" Score="6" ViewCount="385" Body="&lt;p&gt;What is the &lt;strong&gt;mathematical definition of an activation function&lt;/strong&gt; to be used in a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I did not find a precise one, summarizing which criterions (e.g. monotonicity, differentiability, etc.) are required.  Any recommendations for literature about this or – even better – the definition itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, one major point which is unclear for me is differentiability. In lots of articles, this is &lt;em&gt;required&lt;/em&gt; for the activation function, but then, out of nowhere, ReLU (which is not differentiable) is used. I totally understand why we need to be able to take derivatives of it and I also understand why we can use ReLU in practice anyway, but how does one formalize this?&lt;/p&gt;&#xA;" OwnerUserId="32649" LastEditorUserId="2444" LastEditDate="2020-01-10T13:19:42.360" LastActivityDate="2020-11-09T10:04:00.260" Title="What is the mathematical definition of an activation function?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;definitions&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ClosedDate="2020-11-09T10:02:35.343" ContentLicense="CC BY-SA 4.0" />
  <row Id="17446" PostTypeId="2" ParentId="17445" CreationDate="2020-01-10T11:11:55.540" Score="5" Body="&lt;p&gt;There is no strict &lt;em&gt;definition&lt;/em&gt; of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may perform badly in general (but those functions may still work in specific cases)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using gradient descent as a training method, then differentiability is closest to being a requirement. However, even then as you noticed with ReLU, it is not an absolute requirement, provided behaviour close to discontinuities is good. For example &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{x}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$log{x}$&lt;/span&gt; make bad choices here due to how they behave for values near to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function#Comparison_of_activation_functions&quot; rel=&quot;noreferrer&quot;&gt;summarises some important desirable traits, plus compares popular activation functions in this table&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Nonlinear&lt;/strong&gt; – When the activation function is non-linear, then a two-layer neural network can be proven to be a universal function approximator. The identity activation function does not satisfy this property. When multiple layers use the identity activation function, the entire network is equivalent to a single-layer model.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Range&lt;/strong&gt; – When the range of the activation function is finite, gradient-based training methods tend to be more stable, because pattern presentations significantly affect only limited weights. When the range is infinite, training is generally more efficient because pattern presentations significantly affect most of the weights. In the latter case, smaller learning rates are typically necessary.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Continuously differentiable&lt;/strong&gt; – This property is desirable (RELU is not continuously differentiable and has some issues with gradient-based optimization, but it is still possible) for enabling gradient-based optimization methods. The binary step activation function is not differentiable at 0, and it differentiates to 0 for all other values, so gradient-based methods can make no progress with it.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Monotonic&lt;/strong&gt; – When the activation function is monotonic, the error surface associated with a single-layer model is guaranteed to be convex.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Smooth functions with a monotonic derivative&lt;/strong&gt; – These have been shown to generalize better in some cases.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Approximates identity near the origin&lt;/strong&gt; – When activation functions have this property, the neural network will learn efficiently when its weights are initialized with small random values. When the activation function does not approximate identity near the origin, special care must be used when initializing the weights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can see that ReLU is an outlier on more than one of these traits. The reason that it is popular is that despite these weaknesses, the things it does well - speed of operation plus helping combat the vanishing gradient problem - make it a solid choice for very deep networks. In fact the success of ReLU has inspired use of a number of similar-looking activation functions that hope to keep its benefits whilst having more of these desirable traits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultmately though, almost any non-linear function can be used successfully in a neural network. The more of the desirable traits it has, the more likely you can use it in general cases with existing approaches to training.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-10T11:31:59.853" LastActivityDate="2020-01-10T11:31:59.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17447" PostTypeId="1" CreationDate="2020-01-10T14:30:46.947" Score="2" ViewCount="25" Body="&lt;p&gt;I was wondering whether it is possible to regularize (L1 or L2) non-linear parameters in a general regression model. Say, I have the following non-linear least squares cost function, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$3d$&lt;/span&gt; vector of fitting parameters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$cost(p_i) = ( y(x) - sin^{p_1}(x) + p{_2}e^{(p_3*x)} )^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above cost function, &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_3$&lt;/span&gt; are non-linear parameters. How should I go about regularizing them? If they were linear, I can just sum them up together with the linear parameters (absolute values or squares) and add as a penalty to the cost function, right? However, I'm not sure if I'm allowed to do so for non-linear parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone considered this problem?&lt;/p&gt;&#xA;" OwnerUserId="32653" LastActivityDate="2020-01-10T14:30:46.947" Title="Regularization of non-linear parameters?" Tags="&lt;regularization&gt;&lt;non-linear-regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17448" PostTypeId="2" ParentId="17441" CreationDate="2020-01-10T14:53:56.940" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1804.01005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; you are citing is the paper that introduced the &lt;a href=&quot;https://pdollar.github.io/files/papers/DollarCVPR10pose.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;cascaded&lt;/a&gt; convolution neural network. In fact, in this paper, the authors say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To realize 3DDFA, we propose to &lt;strong&gt;combine two achievements&lt;/strong&gt; in recent years, namely, &lt;strong&gt;Cascaded Regression&lt;/strong&gt; and the &lt;strong&gt;Convolutional Neural Network&lt;/strong&gt; (CNN). This combination requires the introduction of a new input feature which fulfills the &quot;cascade manner&quot; and &quot;convolution manner&quot; simultaneously (see Sec. 3.2) and a new cost function which can model the priority of 3DMM parameters (see Sec. 3.4)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where 3DDFA stands for &lt;strong&gt;3D Dense Face Alignment&lt;/strong&gt;, the framework proposed in this paper for &lt;strong&gt;face alignment&lt;/strong&gt;, in which a dense &lt;a href=&quot;https://gravis.dmi.unibas.ch/publications/pami03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;3D Morphable Model&lt;/strong&gt; (3DMM)&lt;/a&gt; is fitted to the image via cascaded CNNs (the &lt;a href=&quot;https://en.wikipedia.org/wiki/Regression_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;regressor&lt;/a&gt;), where the term &lt;em&gt;dense&lt;/em&gt; refers to the number of points of the face that will be modeled. See figure 1 of this paper, which should provide some intuition behind the purpose of this framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section 3 (page 3), they also say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this section, we introduce how to combine Cascaded Regression and CNNs to realize 3DDFA. By applying a CNN as the regressor in Eqn. 1, Cascaded CNN can be formulated as:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{p}^{k+1} = \mathbf{p}^{k} + \text{Net}^{k}&#xA;(\text{Fea}(\mathbf{I}, \mathbf{p}^k)) \tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the iteration number&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}$&lt;/span&gt; is the regression objective &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{Net}$&lt;/span&gt; is the CNN structure &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{Fea}$&lt;/span&gt; contains the two constructed image features &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pose Adaptive Feature (PAF) (section 3.2.1)&lt;/li&gt;&#xA;&lt;li&gt;Projected Normalized Coordinate Code (PNCC) (section 3.2.2)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; is the image&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The expression &lt;em&gt;&lt;a href=&quot;https://pdollar.github.io/files/papers/DollarCVPR10pose.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;cascaded&lt;/a&gt; CNN&lt;/em&gt; apparently refers to the fact that equation \ref{1} is used iteratively, so there will be multiple CNNs, one for each iteration &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. In fact, in the paper, they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unlike existing CNN methods that apply different network structures for different fitting stages, 3DDFA employs a unified network structure across the cascade. In general, at iteration &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$k = 0, 1, \dots, K$&lt;/span&gt;), given an initial parameter &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}^k$&lt;/span&gt;, we construct PNCC and PAF with &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}^k$&lt;/span&gt; and train a two-stream CNN &lt;span class=&quot;math-container&quot;&gt;$\text{Net}^k$&lt;/span&gt; to conduct fitting. The output features from two streams are merged to predict the parameter update &lt;span class=&quot;math-container&quot;&gt;$\Delta \mathbf{p}^k$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Delta \mathbf{p}^k = \text{Net}^k(\text{PAF}(\mathbf{p}^k, \mathbf{I}), \text{PNCC}(\mathbf{p}^k, \mathbf{I}))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Afterwards, a better &lt;strong&gt;intermediate&lt;/strong&gt; parameter &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}^{k+1} = \mathbf{p}^k + \Delta \mathbf{p}^k$&lt;/span&gt; becomes the input of the next network &lt;span class=&quot;math-container&quot;&gt;$\text{Net}^k$&lt;/span&gt; which has the &lt;strong&gt;same structure but different weights with &lt;span class=&quot;math-container&quot;&gt;$\text{Net}^k$&lt;/span&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1804.01005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;figure 2 of the paper&lt;/a&gt; (page 4), the structure of this two-stream CNN, &lt;span class=&quot;math-container&quot;&gt;$\text{Net}^k$&lt;/span&gt;, at iteration &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, is shown.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-10T14:58:58.383" LastActivityDate="2020-01-10T14:58:58.383" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17449" PostTypeId="1" CreationDate="2020-01-10T21:31:50.477" Score="1" ViewCount="39" Body="&lt;p&gt;Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which in conditional the approximation to the posterior and the posterior's distributions are conditioned on some labels? So, I'm guessing that semi-supervised VAE affects the loss evaluation, whereas, in conditional VAEs, the inference network is conditioned on another label as well?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-01-10T22:20:55.040" LastActivityDate="2020-01-10T22:20:55.040" Title="What's the difference between semi-supervised VAEs and conditional VAEs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17450" PostTypeId="1" CreationDate="2020-01-10T23:04:46.443" Score="4" ViewCount="99" Body="&lt;p&gt;I've been doing some research on the principles behind AlphaZero.&#xA;Especially this &lt;a href=&quot;https://adspassets.blob.core.windows.net/website/content/alpha_go_zero_cheat_sheet.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;cheat sheet&lt;/a&gt; (1) and this &lt;a href=&quot;https://towardsdatascience.com/from-scratch-implementation-of-alphazero-for-connect4-f73d4554002a&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementation&lt;/a&gt; (2) (in Connect 4) were very useful.&lt;/p&gt;&#xA;&lt;p&gt;Yet, I still have two important questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How is the policy network updated? In (2), board positions are saved in a dataset of tuples (state, policy, value). The value is derived from the result of the self-played game. However, I'm not sure which policy is saved: the number of times that each move has been played, the prior probabilities for each move (I guess not), or something else?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The cheat sheet says that (for competitive play) the move is chosen with the greatest N (=most visited). Wouldn't it be more logical to choose the move with the highest probability calculated by the policy head?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32668" LastEditorUserId="2444" LastEditDate="2020-12-17T02:01:04.067" LastActivityDate="2020-12-17T02:01:04.067" Title="In AlphaZero, which policy is saved in the dataset, and how is the move chosen?" Tags="&lt;reinforcement-learning&gt;&lt;alphazero&gt;&lt;self-play&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17452" PostTypeId="1" CreationDate="2020-01-11T08:21:14.060" Score="1" ViewCount="34" Body="&lt;p&gt;Let's say that I have a pre-trained model where the training set used to pretrain the model is very different from my training set. &#xA;Let's say I unfreeze layers that have X trainable parameters. What size should the training set be with/without data augmentation for multi-class/multi-label image classification with Y number of labels?&lt;/p&gt;&#xA;" OwnerUserId="32622" LastEditorUserId="32622" LastEditDate="2020-01-12T10:54:53.887" LastActivityDate="2020-01-12T10:54:53.887" Title="What's the mathematical relationship between number of trainable parameters and size of training set?" Tags="&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17453" PostTypeId="1" CreationDate="2020-01-11T09:10:33.207" Score="0" ViewCount="109" Body="&lt;p&gt;I have images that contain lots of elements. Some I know, some I don't. I want to know if it's ok to only label those I do know. Let's take this image for example. I would label the green stuff and the worm but leave the rest unlabeled.&#xA;Is that ok?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question I would also like to ask is how concise I should be in labeling. For instance, You can see in the picture a bit of blue behind the green plant. So should I label that bit and say water or leave it unlabeled?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to ask if it's ok to label only the things I'm interested in even if they take &lt;strong&gt;up to&lt;/strong&gt;  30% of the picture? Won't the neural network be confused by all the details in the picture that it perceives and that I label as A for example, even if A is just a part of it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question would be, let's say I have labels A, B and C. I have an image in which I'm a bit confused if a certain object is of label B &lt;strong&gt;or&lt;/strong&gt; A &lt;strong&gt;or&lt;/strong&gt; even a totally different class other than (A,B,C). What should I do in this instance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I'm having a hard time with the dataset. It would take an expert to label this correctly. But I want to do things as cleanly as possible, so all the effort doesn't go to waste. I would really appreciate your help. Thank you guys.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/37OdN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/37OdN.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32622" LastEditorUserId="32622" LastEditDate="2020-01-12T11:32:40.757" LastActivityDate="2020-01-14T18:00:39.633" Title="How to correctly label images for multi-label classification?" Tags="&lt;classification&gt;&lt;datasets&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17456" PostTypeId="1" CreationDate="2020-01-11T12:41:14.647" Score="2" ViewCount="114" Body="&lt;p&gt;In the step of tuning my neural networks I often encounter a problem that every time I train the exact same network, it gives me different final error due to random initialization of the weights. Sometimes the differences are small and negligible, sometimes they are significant, depending on the data and architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem arises when I want to tune some parameters like number of layers or neurons, because I don't know if the change in final error was caused by recent changes in network's architecture or it is simply effect of the aforementioned randomness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to deal with this issue?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2020-03-21T16:20:27.363" Title="How to deal with random weights initialization in hyperparameters tuning?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;randomness&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17457" PostTypeId="2" ParentId="17456" CreationDate="2020-01-11T13:21:35.440" Score="2" Body="&lt;p&gt;I don't think you can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say a NN with 3 layers gives an accuracy of 95.3% and another NN with 4 layers gives an accuracy of 95.4%. Then there is no guarantee that the 4 layer NN is better than the 3 layered NN. Since with different initial values the 3 layer NN might perform better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could run multiple times and probabilistically say that this is better, but this is computational intense.&lt;/p&gt;&#xA;" OwnerUserId="32408" LastActivityDate="2020-01-11T13:21:35.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17458" PostTypeId="1" CreationDate="2020-01-11T14:20:00.430" Score="1" ViewCount="34" Body="&lt;p&gt;I have a Keras GAN where every layer in the generator has more neurons than the last and also where they all have an activation of LeakyReLU(alpha=0.1). I am trying to map the image back to the noise by solving linear equations in respect to the &quot;output to be&quot;, which is the sum of all the weights*last input value then plus a bias going through the activation function. I reverse the LeakyReLU by multiplying all the negative output-to-be-s by 10 (1/alpha) to solve only for a set of linear equations. It gives wrong results though. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def inversemodel(model, predict):&#xA;    current=predict[:]&#xA;    layerlist=model.layers[:]&#xA;    layerlist.reverse()&#xA;    for layer in layerlist:&#xA;        lastlayerlen=np.array(layer.get_weights()[0]).shape[0]&#xA;        a=np.transpose(layer.get_weights()[0][:], (1, 0))[:lastlayerlen]&#xA;        b=np.negative(np.array(layer.get_weights()[1]))&#xA;        b=b[:lastlayerlen]&#xA;        #minus biases from both sides&#xA;        tobe=np.array(current[:lastlayerlen])&#xA;        for index, i in enumerate(tobe):&#xA;            if i&amp;lt;0:&#xA;                tobe[index]=i*10&#xA;        b=np.subtract(b, tobe)&#xA;        current = solve(a, b).tolist()&#xA;    return current&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this questions is better suited in StackOverflow, please move it and not delete it.&lt;/p&gt;&#xA;" OwnerUserId="17423" LastEditorUserId="17423" LastEditDate="2020-01-12T12:15:37.527" LastActivityDate="2020-01-12T12:15:37.527" Title="Reversing A Keras Dense GAN" Tags="&lt;keras&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17459" PostTypeId="2" ParentId="17453" CreationDate="2020-01-11T14:38:34.693" Score="2" Body="&lt;p&gt;You can't label things you don't know.  The goal of labeling is to label the things you want the classifier to learn so that when you run it in inference mode you can discover what is in your data (new data that you didn't use for training, validating, or testing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not a good idea to label small objects like the 'blue water' unless it is important to you to discover these fine details in inference mode.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-01-11T14:38:34.693" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17460" PostTypeId="2" ParentId="17453" CreationDate="2020-01-11T15:26:45.380" Score="1" Body="&lt;p&gt;I think what you are actually talking about is semantic segmentation (where you label pixels individually).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a difference in theses tasks like Classification, Detection or Semantic Segmentation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classification refers to the task of giving a (usually) single label to the whole image, e.g. cat. But as you already noticed this does not nececerraly end in a clear labeling policy since you basically always have multiple classes in one image. However a ANN usually learns the most relevant (biggest, nearest) object in an image to set it to the corresponding class (but this of course again depends on how the images are labeled). At inferencing of the ANN you then get a probability distribution over all predefined classes. You can of course use this to take the K most relevant classes instead of just the single most relevant classes, to cover cases where multiple objects a probably present. However common output layers, e.g. like Softmax, are usually designed to favour for one single class instead of multiple classes, so you should keep that in mind or consider using a better suited output layer function for your use-case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To have a more general apporach in the task of object detection you classify multiple objects in an image (usually as bounding boxes). That means you label all predefined objects with their position and class in an image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the most general approach here would be to do semantic segmentation that labels every pixel to a corresponding class, which then gives you the actual object borders etc. You can also label pixels as &quot;voids&quot; (or something like that) to cover unknown classes or classes that are not considered in your dataset. However creating such dataset is a horrible amount of work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clear this up. Look at your actual use-case and think about what you actually want your neural network to do. According to this you then think of a labeling policy and label your data.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastActivityDate="2020-01-11T15:26:45.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17461" PostTypeId="1" CreationDate="2020-01-11T16:23:53.437" Score="1" ViewCount="56" Body="&lt;p&gt;I am using a normalizing flow (Neural Spline Flows) to approximate a probability. After some training, the average loss is around 0.5 (so the logarithm of the probability = -0.5). However, when I am trying it on some new test data, I am getting some values of the logarithm of the probability bigger than zero, which would mean that the probability for that element is bigger than one (which doesn't make sense). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what could cause this? Isn't the flow supposed to keep all the probabilities below 1 automatically?&lt;/p&gt;&#xA;" OwnerUserId="22839" LastEditorUserId="2444" LastEditDate="2020-06-11T20:16:31.877" LastActivityDate="2021-03-09T01:04:50.447" Title="Why am I getting the logarithm of the probability bigger than zero when using Neural Spline Flows?" Tags="&lt;probability&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17463" PostTypeId="1" CreationDate="2020-01-11T21:37:31.187" Score="1" ViewCount="42" Body="&lt;p&gt;Suppose we want to train a model to detect various objects.   Let's say we have training data of those objects in various backgrounds along with their bounding boxes. Basically these objects have been three dimensionally created and the bounding boxes have been drawn on them. Then these have been &quot;synthetically inserted&quot; into various blank backgrounds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why would a model trained only on this data do better than a model that has this data along with &quot;real&quot; data of these objects with their bounding boxes manually drawn? &lt;/p&gt;&#xA;" OwnerUserId="32686" LastActivityDate="2021-05-21T08:23:06.617" Title="YOLOv3 Synthetic Data Training" Tags="&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17464" PostTypeId="2" ParentId="17456" CreationDate="2020-01-11T23:58:36.993" Score="1" Body="&lt;p&gt;There are two weight-initializing methods for neural networks: &#xA;1-Zero initializing &#xA;2-Random initializing &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/weight-initialization-techniques-in-neural-networks-26c649eb3b78&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/weight-initialization-techniques-in-neural-networks-26c649eb3b78&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose zero initalizing method in every train loop, you may get same results OR you can use transfer learning according to your problem, it allows to start same parameters. At last, as a worst and hardest choice, you can write your own weight arrays and feed your layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem you mentioned is one of the most interesting problems in the evaluation of performance of neural networks. You can use cross-validation method to verify your model's accuracy! ıt will give more reliable results!&lt;/p&gt;&#xA;" OwnerUserId="32669" LastActivityDate="2020-01-11T23:58:36.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17465" PostTypeId="1" CreationDate="2020-01-12T06:04:01.480" Score="1" ViewCount="42" Body="&lt;p&gt;I have read that in deep networks you can engineer each layer for a particular purpose with regards to feature learning. I'm wondering how that is actually done and how it is trained?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition doesn't this conflict with the idea of deep-networks having &quot;automatic&quot; feature extraction? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example consider this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say you want to detect stop signs. How would you teach a deep network to do this in a layer-wise fashion? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;People write about one layer of a Deep Network does edge detection, but how? &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-01-12T06:04:01.480" Title="How to Layer based Feature extraction?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17466" PostTypeId="1" CreationDate="2020-01-12T11:13:16.890" Score="6" ViewCount="298" Body="&lt;p&gt;I came across a news article from 2018 where the president of India was saying that Sanskrit is the best language for ML/AI. I have no idea regarding his qualification on either AI or Sanskrit to say this but this idea has been floated earlier in the context of NLP. Specifically, Rick Briggs &lt;a href=&quot;http://vedicsciences.net/articles/sanskrit-nasa.html&quot; rel=&quot;noreferrer&quot;&gt;had said so&lt;/a&gt; in 1985.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know elementary Sanskrit and I know NLP. I can understand the point that as a strongly declined language Sanskrit is less ambiguous than say English as the position of words in a sentence are not important. Add to it the fact that words are also declined (that's not the technical term for verbs and I am not sure what is) so that verb gender and number help identify which entity they refer to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that point was valid in 1985. My question is that post the Deep Learning revolution of the last couple of years is it still relevant? Especially given the fact that humans still have to first learn Sanskrit in case NLP is done in Sanskrit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, as can be guessed from the tone of the question, I am of the opinion that Sanskrit is not relevant for AI now but I wanted to know if someone who works in AI thinks otherwise and if so what is their reason so think so.&lt;/p&gt;&#xA;" OwnerUserId="21511" LastActivityDate="2020-01-29T08:37:27.163" Title="Is Sanskrit still relevant for NLP/AI?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17467" PostTypeId="1" CreationDate="2020-01-12T14:55:12.783" Score="3" ViewCount="893" Body="&lt;p&gt;I'm building an RL agent using SARSA and Q-Learning for testing its capabilities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environment is a 10x10 grid, where it gets a reward of 1 if he reaches the goal while he takes -1 every time he takes a step out of the grid. So, it can freely move out and every time it takes a step outside of the grid it gets -1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After tuning the main parameters &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;alpha_val: 0.25 &lt;/li&gt;&#xA;&lt;li&gt;discount: 0.99 &lt;/li&gt;&#xA;&lt;li&gt;episode_length: 50 &lt;/li&gt;&#xA;&lt;li&gt;eps_val: 0.5 &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I get the following plot for 10000 episodes (The plot is sampled every 100 episodes): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B6iMK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B6iMK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I look at the plots online I see usually plots like this one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5d9pi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5d9pi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I'm new at RL, I'm asking some comments about my outcome or any type of suggestion if anyone of you think that I'm doing something wrong.&lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2020-01-14T13:53:51.817" LastActivityDate="2020-08-20T15:31:10.240" Title="Why is the average reward plot for my reinforcement learning agent different than the usual plots?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17468" PostTypeId="1" CreationDate="2020-01-12T22:11:30.710" Score="3" ViewCount="132" Body="&lt;p&gt;I'm working on semantic segmentation tasks in the medical space using the U-Net. Let's say that I train a U-Net model on medical images with the goal of segmenting out, say, ligaments, from a medical image. If I train that model on images that contain just a single labelled ligament, it will be able to segment out single ligaments pretty well, I assume. If I present it with an image with multiple ligaments, should it also be able to segment the multiple ligaments well too?&lt;/p&gt;&#xA;&lt;p&gt;Based on my understanding, semantic segmentation is just pixel-wise classification. As a result, shouldn't the number of the objects in the image not be relevant since it's only looking at individual pixels? So, as long as a pixel matches that of a ligament, it should be able to segment it equally right?&lt;/p&gt;&#xA;&lt;p&gt;Or am I misunderstanding some piece?&lt;/p&gt;&#xA;&lt;p&gt;Basically, if I train a U-Net on images with just single ligaments, will it also be able to segment images with multiple ligaments equally as well based on my logic above?&lt;/p&gt;&#xA;" OwnerUserId="32701" LastEditorUserId="2444" LastEditDate="2021-02-07T12:44:10.123" LastActivityDate="2021-04-08T18:01:25.123" Title="If I trained a model to perform semantic segmentation on images with only one object, would it also work on images with multiple objects?" Tags="&lt;computer-vision&gt;&lt;image-segmentation&gt;&lt;u-net&gt;&lt;instance-segmentation&gt;&lt;semantic-segmentation&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17469" PostTypeId="2" ParentId="17467" CreationDate="2020-01-12T23:10:09.217" Score="2" Body="&lt;p&gt;Well, the way to know that the agent is actually learning is by looking at its behavior while it performs the task, and by comparing against a known optimal performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, does your agent reaches the goal quickly? Does it step out of the grid frequently? What is the maximum possible sum of rewards / minimum number of steps attainable? Is the agent close to that limit? From your graphic, and if I understood correctly your RL problem, the maximum average reward per step should be close to 1 (depending on the specific environment you are using), so I guess you are not so far from the optimal solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, probably if you keep training for a longer period, your agent will reach a stable solution that might or might not be optimal. If you keep training after that, your curves surely will look like the ones you found online.&lt;/p&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2020-01-12T23:10:09.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17470" PostTypeId="1" CreationDate="2020-01-13T00:01:38.183" Score="2" ViewCount="492" Body="&lt;p&gt;Assuming we had an unlimited time to train a model and a very powerful machine to use our model in real-time (hello quantum computer), I'd like to know why no one could achieve to build an AI able to play a FPS, using &lt;strong&gt;ONLY&lt;/strong&gt; pixels shown on the screen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I am not tackling this problem and neither am I planning on doing such a thing, this is pure speculation and curiosity.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I read this great article: &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/viewPaper/14456&quot; rel=&quot;nofollow noreferrer&quot;&gt;playing FPS Games with Deep Reinforcement Learning (2017)&lt;/a&gt; (&lt;em&gt;Guillaume Lample, Devendra Singh Chaplot&lt;/em&gt;) where they achieve a 4/1 kills/death ratio on Doom against bots. &lt;strong&gt;But this is 3 years old now&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of their model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pyHve.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pyHve.png&quot; alt=&quot;model&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But they made 2 assumptions that we, humans, do not make when we are playing for the first time to a new game like Call of Duty or Battlefield:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Game feature augmentation&lt;/strong&gt;. To train a part of their model &lt;strong&gt;they used the game engine to know if there is (or not) an enemy in the frame&lt;/strong&gt; they are processing. We obviously can't do this with CoD or Battlefield, and we, as human, just &quot;learn&quot; to recognize an enemy without these informations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Changing textures of several maps&lt;/strong&gt; while training to generalize better the model (see 5.2 of the paper linked previously). To summarize, they trained the model with 10 maps changing texture of some elements to make the model generalize better. Then they tested it on 3 unknown maps. In real world (ie in the scenario where we base our training/testing exclusively on the pixels of the screen), we can't train a model with different textures on the same map. And humans are able to play a deathmatch on an unknow map without re-learning everything (detect enemies, move, hide, reloading...). We just need to construct a 3D model of the map in our head to play our best.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Their agent &quot;divides the problem into &lt;strong&gt;two phases: navigation (exploring the map to collect items and find enemies) and action (fighting enemies when they are observed), and uses separate networks&lt;/strong&gt; for each phase of the game&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be wise to use more than 2 models? Let's say:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 CNN to detect enemies&lt;/li&gt;&#xA;&lt;li&gt;1 model to deal with space features (position/navigation of the agent and the enemies)&lt;/li&gt;&#xA;&lt;li&gt;1 model to choose the actions given all data previous models have found?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We could train them independently, at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think we'd get better result by processing manually some features (using computer vision techniques) like the minimap to get know positions of enemies and number of ammo to feed as input of the last model (action decider).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are other problem we'd get: there is a delay between the frame we choose to pull the trigger, the time the bullet hit the enemy and the time the &quot;reward&quot; appears on the screen (ex: &quot;100 points, kill [nameOfLateEnemy]&quot; appears after the 3th bullet, and if there is ping because we are playing online it may appear 100ms after). &lt;strong&gt;How to use reinforcement learning when we don't know exactly which action was the one getting the reward?&lt;/strong&gt; (we can move the agent while changing the lok directon while pulling the trigger at the same time. It's the combination of these actions that is making the agent kill an enemy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the 2 assumptions they made were easy to get rid of, they would be discarded already.&lt;br&gt;&#xA;However detecting enemies is basically a simple CNN, and making the navigation network generalize certainly have solutions I can't think of right now but some researchers should have in this 3-year gap between the paper and today.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;So why isn't there a model playing CoD or Battlefield better than humans? What am I missing?&lt;/h2&gt;&#xA;" OwnerUserId="32704" LastActivityDate="2020-01-13T00:01:38.183" Title="Why isn't there a model playing FPS like CoD or Battlefield already existing?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;game-ai&gt;&lt;models&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17471" PostTypeId="2" ParentId="17118" CreationDate="2020-01-13T01:09:01.170" Score="1" Body="&lt;p&gt;If is a truly a random number, and you could guess each of the next successive five in sequence, then you could win the lottery consistently.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is one of the first tasks many people try to do when first learning machine learning.  If the lottery is truly a random physical process with fair, i.e., balanced ping pong balls, then you cannot predict which 5 or 6 numbers will come up next.  The Lottery Commissions around the world go through great pains to ensure that the lotteries are fair and not fraudulent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like you are using a random seed number, that is why you are getting the next number always the same.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck learning your numbers !&lt;/p&gt;&#xA;" OwnerUserId="32706" LastActivityDate="2020-01-13T01:09:01.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17472" PostTypeId="1" AcceptedAnswerId="17482" CreationDate="2020-01-13T02:52:52.587" Score="1" ViewCount="42" Body="&lt;p&gt;The following table shows the precision and recall values I obtained for three object detection models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gj5jr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gj5jr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I evaluate the first two models as the following. The target is to find the best object detection model for that particular data set.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Model 1 has a high recall and precision values. High precision relates to a low false-positive rate, and high&#xA;  recall relates to a low false-negative rate. High scores for both show&#xA;  that the model is returning accurate results. Model 2 has high&#xA;  precision but low recall. This means it returns very few results, but&#xA;  most of its identified objects are correct.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can I evaluate the third one?&lt;/p&gt;&#xA;" OwnerUserId="32343" LastActivityDate="2020-01-13T19:18:32.970" Title="Evaluate precision and recall results" Tags="&lt;object-detection&gt;&lt;accuracy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17473" PostTypeId="1" CreationDate="2020-01-13T03:26:15.877" Score="1" ViewCount="33" Body="&lt;p&gt;I am trying make an ANN model that takes a constant m (will be changed later but now it is just a constant, let's say 0) as an input and generate 5 non-integer numbers (a1,a2..a5) after some layers like relu, linear,relu ... and then these 5 numbers enter to the loss function layer along with an additional 5 numbers (b1,b2..b5) given by hand directly into the same loss function. In the loss function, S=a1xb1+...+a5xb5 will be calculated and the model should use mean square error with this S and S0, which is given by hand again to tune the 5 numbers generated after the NN layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a dummy like me, this looks like a totally different model than the examples online. I'd really appreciate any guidance here like the model I should use, examples etc. I can't even know where to start even though I believe I understand the generic NN examples that one can find online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="32707" LastActivityDate="2020-01-13T03:26:15.877" Title="Generating 5 numbers with 1 input before loss function" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17474" PostTypeId="1" CreationDate="2020-01-13T05:27:30.217" Score="1" ViewCount="57" Body="&lt;p&gt;We know a lot of common sense about the world. Things like &quot;to buy something you need money&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder how much of this common sense comes about through actual someone explicitly telling you the instructions &quot;You need money to buy things&quot;. Which we store in our brains as a sort of rule. As opposed to just intutively understanding things and picking it up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am imagining children playing at shop-keeping and saying things like &quot;I give you this and you give me that&quot;. And other children not quite understanding the concept of buying things until being told by a teacher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, giving a computer a list of common sense rules likes these is no different to teaching a child. So I am wondering why this area of AI research (semantic webs etc.) has been frowned upon in the last decade in favour of trying to learn everything through experience like deep neural networks?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-01-29T02:05:28.497" Title="How much knowledge of the world is learnt through words?" Tags="&lt;natural-language-processing&gt;&lt;semantics&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17475" PostTypeId="1" CreationDate="2020-01-13T06:08:28.000" Score="1" ViewCount="23" Body="&lt;p&gt;for a screen printing app, I'd like to remove background colors from images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is still a white border around text from anti-aliasing.&#xA;Dropshadows also break it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I was thinking I could train an AI by creating images with shapes and text, with and without backgrounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI input would get a version with a background and the &quot;goal&quot; would be the version without the background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I go about doing this? ( total AI noob )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non-AI solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone is interested... I have made a non AI solution which takes all colors within a tolerance of the background, then looks at the 4x4 neighbors.. from each neighbor(which is a candidate for converting into semi-transparent), it looks at the 3x3 neighbors around the candidate for the furthest color from the removal color ( which typically grabs the solid pixels ), and then converts the current pixel to an alpha version by copying the rgb values, and converting alpha to &lt;code&gt;255 * (1 - dist_removal_to_current / dist_removal_to_furthest)&lt;/code&gt; or something like that.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should write an article or something... it was an interesting algorithm to write.  linkedin me Dan Schumann in wisconsin&lt;/p&gt;&#xA;" OwnerUserId="32709" LastEditorUserId="32709" LastEditDate="2020-01-14T17:21:00.003" LastActivityDate="2020-01-14T17:21:00.003" Title="Train AI with shapes + drop shadows to remove background colors" Tags="&lt;image-processing&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17476" PostTypeId="1" AcceptedAnswerId="17478" CreationDate="2020-01-13T07:04:57.613" Score="7" ViewCount="742" Body="&lt;p&gt;In the gradient descent algorithm, the formula to update the weight &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;, which has &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; as the partial gradient of the loss function with respect to it, is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w\ -= r \times g$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should be the formula for momentum optimiser and Adam (adaptive momentum?) optimiser? Something should be added to the right side of the formula above?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2020-01-13T22:39:31.517" LastActivityDate="2020-01-13T22:39:31.517" Title="What is the formula for the momentum and Adam optimisers?" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;momentum&gt;&lt;adam&gt;&lt;optimizers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17478" PostTypeId="2" ParentId="17476" CreationDate="2020-01-13T09:30:08.487" Score="7" Body="&lt;p&gt;I'm going to use slightly different notation, &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; for an assignment, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; for learning rate, &lt;span class=&quot;math-container&quot;&gt;$\nabla_w J$&lt;/span&gt; in place of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;* and implied multiplication as these are slightly more common. Also, using bold letters to represent vectors. In that notation, the update rule for basic gradient descent would be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{w} \leftarrow \mathbf{w} - \alpha \nabla_w J$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This cannot be extended to momentum and Adam update rules whilst keeping it as a single line and modifying the right hand side. That is because these variations of gradient descent maintain running statistics of previous gradient values, which have their own separate update rules. When implemented on a computer, these become additional terms, mainly vectors the same size as the weight vector being updated. These variables also require initialisation before use.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Momentum&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Momentum maintains a &quot;velocity&quot; term which essentially tracks a recency-weighted average of gradients. However, the classic form of momentum given here does not normalise the resulting vector, and you often have to adjust the learning rate down when using it. Momentum has a parameter &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; which should be between 0 and 1, and typically is set at &lt;span class=&quot;math-container&quot;&gt;$0.9$&lt;/span&gt; or higher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Intitialisation&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{m} \leftarrow \mathbf{0}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Update rules&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{m} \leftarrow \beta \mathbf{m} + \nabla_w J$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{w} \leftarrow \mathbf{w} - \alpha \mathbf{m}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some variants of these update rules in practice. An important one is Nesterov momentum.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Adam&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The Adam optimiser maintains a momentum term, plus a scaling term, and also corrects these terms for initial bias. Adam has three parameters &lt;span class=&quot;math-container&quot;&gt;$\beta_m$&lt;/span&gt; for momentum (typically 0.99), &lt;span class=&quot;math-container&quot;&gt;$\beta_v$&lt;/span&gt; for scaling (typically 0.999) and &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; to avoid divide by zero and numerical stability issues (typically &lt;span class=&quot;math-container&quot;&gt;$10^{-6}$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Initialisation&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho_m \leftarrow 1$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\rho_v \leftarrow 1$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{m} \leftarrow \mathbf{0}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{v} \leftarrow \mathbf{0}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Update rules&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho_m \leftarrow \beta_m \rho_m$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\rho_v \leftarrow \beta_v \rho_v$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{m} \leftarrow \beta_m \mathbf{m} + (1-\beta_m) \nabla_w J$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{v} \leftarrow \beta_v \mathbf{v} + (1-\beta_v) (\nabla_w J \odot \nabla_w J)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{w} \leftarrow \mathbf{w}- \alpha(\frac{\mathbf{m}}{\sqrt{\mathbf{v}}+\epsilon} \frac{\sqrt{1-\rho_v}}{1-\rho_m})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The symbol &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; stands for element-wise multiplication. Here that essentially means to square each term of the gradient to calculate terms in &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt;. The square root and division of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{m}$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\mathbf{v}}+\epsilon$&lt;/span&gt; in the last update step are also handled element-wise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variant I show here has an &quot;optimisation&quot; to the bias correction so that you don't need to calculate high powers of either of the parameters. You may see variants that don't have &lt;span class=&quot;math-container&quot;&gt;$\rho_m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_v$&lt;/span&gt; (or equivalents), but instead use &lt;span class=&quot;math-container&quot;&gt;$\beta_{m}^t$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\beta_{v}^t$&lt;/span&gt; directly, which is exactly what &lt;span class=&quot;math-container&quot;&gt;$\rho_m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_v$&lt;/span&gt; represent.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* &lt;span class=&quot;math-container&quot;&gt;$\nabla_w J$&lt;/span&gt; is the gradient of &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt;. By writing it this way, it also describes the goal of the update explicitly within the notation i.e. to minimise a function that is parameterised by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-13T09:43:45.517" LastActivityDate="2020-01-13T09:43:45.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17480" PostTypeId="1" CreationDate="2020-01-13T14:35:00.487" Score="1" ViewCount="21" Body="&lt;p&gt;I know that training deep neural networks (DNNs) takes a lot of computational resources. This is, of course, just a generalized statement. Different networks require different resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One that I am particularly interested in is here: &lt;a href=&quot;https://github.com/NVIDIA/vid2vid&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NVIDIA/vid2vid&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the research paper, it states that they used:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;8 V100 GPUs, each with 16GB memory for training. We distribute the generator computation task to 4 GPUs and the discriminator task to the other 4 GPUs. Training takes ~ 10 days for 2K resolution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's understandable why the training process requires a high amount of computational resources. However, once it is trained, can the model run on a robust CPU and a lot of RAM alone? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, could it run on AMDs Ryzen 2700x (or Intel equivalent) with 32 GBs of RAM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask because, if this network can run on your typical gaming rig, then there is certainly an application here for a rendering alternative for Video Games. Personally, it would be nice if it could run on a CPU with no help needed from the GPU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I being delusional thinking this can be done with currently available hardware?&lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="2444" LastEditDate="2020-02-20T00:55:10.617" LastActivityDate="2020-02-20T00:55:10.617" Title="Can a trained Vid2Vid model be run on AMDs Ryzen 2700x with 32GB of RAM?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;software-evaluation&gt;&lt;hardware-evaluation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17481" PostTypeId="1" CreationDate="2020-01-13T18:36:14.157" Score="3" ViewCount="70" Body="&lt;p&gt;I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why don't LSTM cells have more complex neural networks before each gate, i.e. containing 1 hidden layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would think that if you want more advanced gating decisions, that you would use at least 1 hidden layer to have more complex processing.&lt;/p&gt;&#xA;" OwnerUserId="32716" LastEditorUserId="2444" LastEditDate="2020-01-14T13:57:19.960" LastActivityDate="2020-01-14T13:57:19.960" Title="Why don't the neural networks inside LSTM cells contain hidden layers?" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17482" PostTypeId="2" ParentId="17472" CreationDate="2020-01-13T19:18:32.970" Score="1" Body="&lt;p&gt;This looks like it could be a homework problem, so consider updating it with the homework tag if so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second model has the same precision, but worse recall, than model 1. Therefore we would rather have model 1 than model 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The third model has worse recall than model 1, and worse precision than model 1, therefore we would rather have model 1 than model 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, model 1 is the best object detection model.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-01-13T19:18:32.970" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17483" PostTypeId="2" ParentId="17461" CreationDate="2020-01-13T19:35:41.837" Score="0" Body="&lt;p&gt;I have always seen the negative odd-logs ration when doing similar work. As it is now, I don't think you are constraining the splines appropriately &lt;a href=&quot;https://www.quantitativeskills.com/sisa/statistics/two2hlp.htm#LogOdds&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to reading&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32719" LastActivityDate="2020-01-13T19:35:41.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17484" PostTypeId="1" CreationDate="2020-01-13T19:53:54.220" Score="2" ViewCount="96" Body="&lt;p&gt;I am working with a data set that consists of the actual pitch angle (given as &lt;code&gt;PA(Y)&lt;/code&gt;) and the pitch angle at each radii (listed from 1 to 217). In the image below, you can only see radii 1 through 16. The &lt;code&gt;Mode(Y)&lt;/code&gt; in the image below is not of relevance at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v82L9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v82L9.png&quot; alt=&quot;Snippet of data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are regions that range between certain radii in which the pitch angle measurement does not change (in the image, you'll notice this happens for all the radii values, but they do change after a certain radius that's cut off in the image). These are known as stable regions. My goal is to capture all the ranges in the data in which the pitch angle measurement does not change, and create a program that returns those values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a machine learning method in which this is possible, or is this just a non-machine learning problem? I have tried creating plots and have considered creating a CNN that can identify these flat regions, but I feel like this is overkill. My PIs want to use a machine learning method and they have proposed neural networks, hence why I tried the CNN, but I just am not sure if that is possible? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I should add, usually stable regions radii ranges are unknown, so the goal is to try to see if certain radii ranges usually can predict where a stable region are located.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, I've thought of using a classifier to determine whether a region is flat or not. I am just very confused as to how to approach this. Are there any similar examples to the problem I'm currently working on that someone can point me to? &lt;/p&gt;&#xA;" OwnerUserId="32717" LastEditorUserId="32717" LastEditDate="2020-01-13T20:42:51.017" LastActivityDate="2020-01-13T20:42:51.017" Title="Suggestion for finding the stable regions in spiral galaxy data?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17485" PostTypeId="1" CreationDate="2020-01-13T20:18:09.703" Score="2" ViewCount="24" Body="&lt;p&gt;I've been trying out bayesian hyperparameter optimisation (with TPE) on a simple CNN applied to the MNIST handwritten digit dataset. I noticed that over iterations of the optimisation loop, the tested parameters appear to oscillate slowly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the learning rate:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/m1kHk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m1kHk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the momentum:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eNiC1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eNiC1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I won't add a graph, but the batch size is also sampled from one of 32, 64, or 128. Also note that I did this with a fixed &lt;strong&gt;10 epochs&lt;/strong&gt; in each trial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that we'd expect the trialled parameters to converge gradually towards the optimal, but why the longer term movement of the average?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For context here is the score (1 - accuracy) over iterations&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mbs1H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mbs1H.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also for context, here's the architecture of the CNN.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;conv2d_1 (Conv2D)            (None, 26, 26, 32)        320       &#xA;_________________________________________________________________&#xA;max_pooling2d_1 (MaxPooling2 (None, 13, 13, 32)        0         &#xA;_________________________________________________________________&#xA;conv2d_2 (Conv2D)            (None, 11, 11, 64)        18496     &#xA;_________________________________________________________________&#xA;conv2d_3 (Conv2D)            (None, 9, 9, 64)          36928     &#xA;_________________________________________________________________&#xA;max_pooling2d_2 (MaxPooling2 (None, 4, 4, 64)          0         &#xA;_________________________________________________________________&#xA;flatten_1 (Flatten)          (None, 1024)              0         &#xA;_________________________________________________________________&#xA;dense_1 (Dense)              (None, 100)               102500    &#xA;_________________________________________________________________&#xA;dense_2 (Dense)              (None, 10)                1010      &#xA;=================================================================&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Optimization done with mini-batch gradient descent on the cross entropy.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-01-13T20:18:09.703" Title="Is it normal to see oscillations in tested hyperparameters during bayesian optimisation?" Tags="&lt;convolutional-neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;bayesian-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17487" PostTypeId="1" CreationDate="2020-01-14T03:08:04.570" Score="1" ViewCount="33" Body="&lt;p&gt;Let's say I have the number 123.45 and the expression &lt;em&gt;one hundred twenty-three and forty-five&lt;/em&gt; cents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I develop AI to identify these two values as a match? If I can, how should I do that?&lt;/p&gt;&#xA;" OwnerUserId="32727" LastEditorUserId="2444" LastEditDate="2020-01-14T11:08:51.420" LastActivityDate="2020-01-14T11:08:51.420" Title="How can I match numbers with expressions?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17488" PostTypeId="2" ParentId="17468" CreationDate="2020-01-14T03:26:01.607" Score="0" Body="&lt;p&gt;Without experimental evidence to back me up, I can not answer this with 100% confidence. However, I am fairly certain that this will cause issues depending on the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;U-net is essentially an auto-encoder, and due to the fact that it is all just one big neural network, it is likely it will learn the easiest pattern (as all NN do), and that is to find one single instance of an object and shade that region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now why does this depend on the model? Well let's say you are using something slightly different, where region proposals are generated by a determinisitc algorithm we've predefined, then these regions are run through a CNN to segment them. In this case, as each region is without context of the entire image, the difference between 2 objects in an image and 1 is indistinguishable to the network (as regions may overlap), and as such, only using images with 1 object will not pose any problems (there is a name for early models like these, though it escapes me).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So assuming I am correct, what should you do? The models that use a deterministic algorithm for region proposals are slow and old, so I wouldn't suggest that. Instead, I would think that you should first do some testing, to see if it actually does cause issues. Assuming it does, a good option could be to tamper with the training data and separate segments by a few pixels to sort of &quot;force&quot; multiple objects into existence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of such, I would still suggest using U-net. Fixing this issue (if it does arrise) should be relatively easy to do, so there's little to lose by using U-net and just trying the training.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-01-14T03:26:01.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17490" PostTypeId="1" CreationDate="2020-01-14T10:39:03.577" Score="3" ViewCount="40" Body="&lt;p&gt;&lt;strong&gt;Main question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way we can leverage general knowledge of how certain hyperparameters affect performance, to very rapidly get some sort of estimate for how good a given architecture &lt;em&gt;could&lt;/em&gt; be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Elaboration&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a handwritten character recognition problem using CNNs. I want to try out a few different architectures (mostly at random) to iterate towards something which might work. The problem is that one run takes a really long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's a way to quickly verify if a given architecture is promising? And let me elaborate on what I've tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Just try it once&lt;/strong&gt;. Yeah but maybe I chose some bad hyperparameter combination and actually that architecture was going to be the ground breaker.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Do Bayesian optimisation&lt;/strong&gt;. That's still really slow. From examples and trials, I've seen that it takes quite some time for convergence. And besides, I'm not trying to optimise yet, I just want to check if there's any potential.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-01-14T10:39:03.577" Title="What are some ways to quickly evaluate the potential of a given NN architecture?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17491" PostTypeId="1" CreationDate="2020-01-14T11:14:03.630" Score="2" ViewCount="100" Body="&lt;p&gt;I implemented a DQN algorithm that plays OpenAIs Cartpole environment. The NN architecture consists of 3 normal linear layers that encode the state, and one noisy linear layer, that predicts the Q value based on the encoded state.&lt;br&gt;&#xA;My &lt;code&gt;NoisyLinear&lt;/code&gt; layers looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class NoisyLinear(nn.Module):&#xA;  def __init__(self, in_features, out_features):&#xA;    super(NoisyLinear, self).__init__()&#xA;    self.in_features = in_features&#xA;    self.out_features = out_features&#xA;    self.sigma_zero = 0.5&#xA;    self.weight_mu = torch.empty(out_features, in_features)&#xA;    self.weight_sigma = torch.empty(out_features, in_features)&#xA;    self.weight_epsilon = torch.empty(out_features, in_features, requires_grad=False)&#xA;    self.bias_mu = torch.empty(out_features)&#xA;    self.bias_sigma = torch.empty(out_features)&#xA;    self.bias_epsilon = torch.empty(out_features, requires_grad=False)&#xA;    self.reset_parameters()&#xA;    self.reset_noise()&#xA;&#xA;  def reset_parameters(self):&#xA;    mu_range = 1 / math.sqrt(self.in_features)&#xA;    self.weight_mu.data.uniform_(-mu_range, mu_range)&#xA;    self.weight_sigma.data.fill_(self.sigma_zero / math.sqrt(self.in_features))&#xA;    self.bias_mu.data.uniform_(-mu_range, mu_range)&#xA;    self.bias_sigma.data.fill_(self.sigma_zero / math.sqrt(self.out_features))&#xA;&#xA;  def _scale_noise(self, size):&#xA;    x = torch.randn(size)&#xA;    return x.sign().mul_(x.abs().sqrt_())&#xA;&#xA;  def reset_noise(self):&#xA;    epsilon_in = self._scale_noise(self.in_features)&#xA;    epsilon_out = self._scale_noise(self.out_features)&#xA;    self.weight_epsilon.copy_(epsilon_out.ger(epsilon_in))&#xA;    self.bias_epsilon.copy_(epsilon_out)&#xA;&#xA;  def forward(self, input):&#xA;    return F.linear(input, self.weight_mu + self.weight_sigma * self.weight_epsilon, self.bias_mu + self.bias_sigma * self.bias_epsilon)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, with the default hyperparameters from the (sigma_0 = 0.5), the agent does not explore at all, and even if I crank it up to sigma_0 = 5, it works way worse than epsilon greedy.&lt;br&gt;&#xA;(When I use noisy nets I don't use epsilon greedy).&lt;/p&gt;&#xA;" OwnerUserId="31821" LastEditorUserId="32742" LastEditDate="2020-01-14T16:26:24.160" LastActivityDate="2021-02-09T10:00:00.063" Title="NoisyNet DQN with default parameters not exploring" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17492" PostTypeId="1" AcceptedAnswerId="17504" CreationDate="2020-01-14T13:18:53.630" Score="3" ViewCount="291" Body="&lt;p&gt;I'm still on my first steps in the Data Science field. I played with some DL frameworks, like TensorFlow (pure) and Keras (on top) before, and know a little bit of some &quot;classic machine learning&quot; algorithms like decision trees, k-nearest neighbors, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, image classification problems can be solved with deep learning, but some people also use the SVM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are traditional ML models still used over neural networks, if neural networks seem to be superior to traditional ML models? Keras is rather simple to use, so why don't people just use deep neural networks with Keras? What are the pros and cons of each approach (considering the same problem)?&lt;/p&gt;&#xA;" OwnerUserId="7268" LastEditorUserId="2444" LastEditDate="2020-01-14T23:11:59.470" LastActivityDate="2020-03-02T21:36:40.007" Title="Why are traditional ML models still used over deep neural networks?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;keras&gt;&lt;comparison&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17494" PostTypeId="1" AcceptedAnswerId="17495" CreationDate="2020-01-14T15:29:51.597" Score="1" ViewCount="151" Body="&lt;p&gt;I'm a newbie in Convolutional Neural Networks. I have found out that kernels in convolutional layers are usually learned while training.&lt;/p&gt;&#xA;&lt;p&gt;Suppose I have a kernel that is very good to extract the features that I want to extract. In that case, I don't want the kernels to be learnable. So, how can I make the kernels non-learnable and set them manually?&lt;/p&gt;&#xA;&lt;p&gt;Maybe, in that case, I have to use something different from a CNN.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2021-02-19T11:32:41.647" LastActivityDate="2021-02-19T11:32:41.647" Title="How can I make the kernels non-learnable and set them manually?" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;image-processing&gt;&lt;convolution&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17495" PostTypeId="2" ParentId="17494" CreationDate="2020-01-14T15:52:58.320" Score="2" Body="&lt;p&gt;In most modern neural network frameworks, the update rules for training can be selectively applied to some parameters and not others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to do that is dependent on the framework. Some will have the concept of &quot;freezing&quot; a layer, preventing parameters in it being updated. &lt;a href=&quot;https://keras.io/getting-started/faq/#how-can-i-freeze-keras-layers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keras does this for example.&lt;/a&gt; Others will do the opposite and expect you to provide a list of trainable parameters - these typically come with helpers that will list all parameters in a neural network, so you would need to add some kind of filter after collecting that data to exclude your pre-trained layer. &lt;a href=&quot;https://gist.github.com/L0SG/2f6d81e4ad119c4f798ab81fa8d62d3f#file-freeze_example-py-L76&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch does this (although the linked example is slightly more complex in that it stops calculating gradients too)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your framework of choice does not allow you to select and isolate layers in the training process, then you still have a couple of options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You could store a copy of layer parameters that you want to keep and force your learning network to re-load these parameters after each mini-batch. This does depend on you having a method that can selectively set parameters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your pre-trained layers are the first ones, immediately next to the input, then instead of including them in your learning network model, you can pre-process all your training data with just the fixed layers (build a model using only those layers), save the output and use that as an alternative input for the learning layers (build a second model with only the learning layers). Later, once training is complete, you can build a combined neural network out of the fixed layers and the learning layers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-14T16:01:56.587" LastActivityDate="2020-01-14T16:01:56.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17496" PostTypeId="2" ParentId="16364" CreationDate="2020-01-14T16:39:38.087" Score="0" Body="&lt;p&gt;You would need to perform some kind of &lt;code&gt;speech-to-text&lt;/code&gt; to get the audio transcription with the corresponding synchronization wrt the audio. Then search in the transcription.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use &lt;a href=&quot;https://github.com/mozilla/DSAlign&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSAlign&lt;/a&gt; by mozilla&lt;/p&gt;&#xA;" OwnerUserId="20269" LastActivityDate="2020-01-14T16:39:38.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17497" PostTypeId="2" ParentId="11047" CreationDate="2020-01-14T16:49:01.627" Score="1" Body="&lt;p&gt;The task of isolating 2 or more speakers is called &lt;code&gt;speaker diarization&lt;/code&gt;,&#xA;&lt;a href=&quot;https://github.com/wq2012/awesome-diarization#software&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; a list of softwares and useful resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the 2 or more audio files containing the individual voices, you could run some speech-to-text network that also outputs time information.&lt;/p&gt;&#xA;" OwnerUserId="20269" LastActivityDate="2020-01-14T16:49:01.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17498" PostTypeId="1" AcceptedAnswerId="17518" CreationDate="2020-01-14T17:51:38.797" Score="4" ViewCount="297" Body="&lt;p&gt;I'm currently trying to take the next step in deep learning. I managed so far to write my own basic feed-forward network in python without any frameworks (just numpy and pandas), so I think I understood the math and intuition behind backpropagation. Now, I'm stuck with deep q-learning. I've tried to get an agent to learn in various environments. But somehow nothing works out. So there has to be something I'm getting wrong. And it seems that I do not understand the critical part right at least that's what I'm thinking. &lt;a href=&quot;https://i.stack.imgur.com/yIYj9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yIYj9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The screenshot is from &lt;a href=&quot;https://www.youtube.com/watch?v=NLaiMxndXKc&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; video.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm trying to draw here is my understanding of the very basic process of a simple DQN. Assuming this is right, how is the loss backpropagated? &#xA;Since only the selected &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values (5 and 7) are further processed in the loss function, how is the impact from the other neurons calculated so their weights can be adjusted to better predict the real q-values?&lt;/p&gt;&#xA;" OwnerUserId="30431" LastEditorUserId="2444" LastEditDate="2020-01-15T13:13:31.407" LastActivityDate="2020-01-15T14:16:38.860" Title="How can a DQN backpropagate its loss?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;backpropagation&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17499" PostTypeId="2" ParentId="17453" CreationDate="2020-01-14T18:00:39.633" Score="0" Body="&lt;p&gt;I would classify each pixel separately instead of giving a label to the whole image. Sadly preparing the training data is very tedious and time-consuming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the input image has dimensions of 200 x 300 x 3 (RGB) and there are two classes of regions you want to identify. A few approaches come to mind:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Train two separate networks, each forecasting a binary mask of size 200 x 300 of the object class in guestion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Train a single network with a binary output of size 200 x 300 x 2 (sigmoid activation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Train a single network with a binary output of size 200 x 300 x 3 (softmax activation), the 3rd class is for &quot;other&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are uncertain of some regions you can leave its class probability to 50% and it won't affect cross-entropy losses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 1 is easiest to get started with but training a single network should be more computationally more efficient than training two separate ones. In addition options 1 and 2 can forecast a single pixel belonging to both classes with 100% probability unlike the network of option 3.&lt;/p&gt;&#xA;" OwnerUserId="32722" LastActivityDate="2020-01-14T18:00:39.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17501" PostTypeId="1" CreationDate="2020-01-14T20:22:06.580" Score="1" ViewCount="24" Body="&lt;p&gt;I am an agronomy graduate student looking to classify crops from weeds using convolutional neural networks (CNNs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea that I am wanting to get into involves separating crops from weeds from aerial imagery (either captured by drones or piloted aircraft). The idea of the project that I am proposing involves spending some time driving around to different fields and capturing many images of both crops and weeds. These images will then be used to train a CNN that will classify aerial imagery on the location of crops and weeds. After classifying the imagery, a herbicide application map will be generated for site-specific weed control. This involves the integration of CNN classification and GIS technology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is this: If you have an orthomosaic image generated from a drone, will images captured from a digital camera on the ground be effective for training a CNN that will classify high-resolution aerial imagery?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being new to CNNs, I just didn't know if I had to use aerial imagery to train a CNN to classify aerial imagery, or if a digital camera will work just fine.&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-01-14T20:22:06.580" Title="Training dataset for convolutional neural network classification - will images captured on the ground be useful for training aerial imagery?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;training&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17502" PostTypeId="1" CreationDate="2020-01-14T22:02:19.673" Score="3" ViewCount="719" Body="&lt;p&gt;I'm struggling to fully understand the stochastic gradient descent algorithm. &#xA;I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that function IS.&#xA;More specifically, the algorithm should work by initializing the network with random weights. Then, if I'm not mistaken, it forward-propagates &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times (where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the mini-batch size). At this point, I've no idea about what function should I search for, with over hundreds of neurons each having hundreds of parameters.&lt;/p&gt;&#xA;" OwnerUserId="32751" LastEditorUserId="2444" LastEditDate="2020-01-15T00:01:57.553" LastActivityDate="2020-01-27T11:40:02.817" Title="What's the function that SGD takes to calculate the gradient?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17503" PostTypeId="2" ParentId="6429" CreationDate="2020-01-14T22:38:10.057" Score="0" Body="&lt;p&gt;Ideally, yes. Ideally, because the network should be fed with the words of an entire book (wich vary around 100k words). With an hypotetical amount of processing power, you should be able to just train the NN with like thousands of books. &#xA;It might be possible to be trained with quantum computers.... who knows...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For smaller stories, I think that the major problem is to know in what &quot;shape&quot; should the story be generated. Because if it simply outputs some words, then the first thing the network should be able to do is speaking, that means the model should evolve from a pretrained NLP model, and (from what I know) we still have some problems with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So.... I really think that to do such kind of things, the approach we take to make NNs learn should be changed. &#xA;The fact that humans exist proves that genetic algorithms would work 100%. But we obviously don't have 3+ billion years to evolve a &quot;brain&quot; from scratch, that's why we use training algorithms: we &lt;em&gt;force&lt;/em&gt; them to learn from something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But back to the question: humans do a lot of work by thinking about what outcome to chose. To just make a netork generate an outcome, without imitating humans, it would be easy to just choose randomly some aspects of this outcome. For example, a randomly chosen outcome might be &quot;outcome: Dennis dies, Morty kills Eminem, sad sciene, happiness sciene, the end&quot;.&#xA;That means that the NN or any ML model doesn't actually produce an outcome to the story. In fact, what it does is to connect some generated &quot;checkpoints&quot; about that story.&#xA;Actually, you might train a model to generate checkpoints to&#xA;but this is just a random idea from a newbie, so I've got no clue about how to actually implement that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm italian b.t.w., sorry about my english :)&lt;/p&gt;&#xA;" OwnerUserId="32751" LastActivityDate="2020-01-14T22:38:10.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17504" PostTypeId="2" ParentId="17492" CreationDate="2020-01-14T22:46:23.693" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are still traditional machine learning (ML) models used over neural networks if neural networks seem to be superior to traditional ML models?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Of course, the model that achieves state-of-the-art performance depends on the problem, available datasets, etc., so a comprehensive comparison between traditional ML models and deep neural networks is not appropriate for this website, because it requires a lot of time and space. However, there are certain disadvantages of deep neural networks compared to traditional machine learning models, such as k-nearest neighbors, linear regression, logistic regression, naive Bayes, Gaussian processes, support vector machines, hidden Markov models and decision trees.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Often, traditional ML models are conceptually simpler (for example, k-NN or linear regression are much simpler than deep neural networks, such as LSTMs).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Personally, I've noticed that traditional ML models can be used more easily compared to deep neural networks, given the existence of libraries, like scikit-learn, which really have a simple and intuitive API (even though you apparently do not agree with this).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Deep neural networks usually require more data than traditional ML models in order not to overfit. Empirically, I've once observed that certain traditional ML models can achieve comparable performance to deep neural networks in the case of small training datasets.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Even though there's already a new and promising area of study called Bayesian deep learning, most deep neural networks do not really provide any uncertainty guarantees, they only provide you a point estimate. This is a big limitation, because, in areas like healthcare, uncertainty measures are required. In those cases, Gaussian processes may be more appropriate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-14T22:52:22.413" LastActivityDate="2020-01-14T22:52:22.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17505" PostTypeId="2" ParentId="17502" CreationDate="2020-01-14T23:34:17.723" Score="3" Body="&lt;p&gt;Welcome to AI Stack exchange!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're right, as the network is initialised randomly, the resultant function is essentially impossible to get your head around. This is because most of the time the network has &gt;4 dimensions (4 can be graphed with some effort and a lot of color), and as such is literally beyond human comprehension via graphing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what do we do? Well, conveniently, it is possible to find the gradient of &lt;em&gt;segments&lt;/em&gt; of a function, without having to know the entirety of the function itself (it's worth noting that it actually is possible to find the resultant function and with a lot of effort find it's derivative. This proves to be much more work than it's worth though, as we don't need the general derivative that tells us what the gradient is for whatever input we give it, we only need the derivative at the specific input we just fed through the network).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be hard to understand, but if you're familiar with the chain rule, it might make a bit more sense. The chain rule essentially allows you to split a function into components, and find the gradient of those specific components. By combining all of that, you end up with some nice gradients at each weight/bias with respect to the loss function. Take the negative of the gradient, and you now have the change required to decrease the loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is obviously quite hard to understand without an example, &lt;a href=&quot;https://youtu.be/d14TUNcbn1k?t=354&quot; rel=&quot;nofollow noreferrer&quot;&gt;so here's the best one I've ever found, that helped me very much&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as a side note, the whole mini batch thing is used to minimise catastrophic forgetting (where the network begins to &quot;unlearn&quot; old inputs). To deal with minibatchs, what you do is take input individually, then find the gradient for all weights and bias' for that specific input and remember the changes you want to make. Do that for all inputs in the minibatch, then finally add all the changes together to get the resultant best change  for each weight and bias. Only then do you update the weights and bias'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if you have any further questions&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-01-14T23:34:17.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17506" PostTypeId="1" CreationDate="2020-01-14T23:46:24.587" Score="1" ViewCount="60" Body="&lt;p&gt;While studying genetic algorithms, I've come across different crossover operations used for binary chromosomes, such as the &lt;em&gt;1-point crossover&lt;/em&gt;, the &lt;em&gt;uniform crossover&lt;/em&gt;, etc. These methods usually don't use any &amp;quot;intelligence&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I found methods like the &lt;em&gt;fitness-based crossover&lt;/em&gt; and &lt;em&gt;Boltzmann crossover&lt;/em&gt;, which use fitness value so that the child will be created from better parents with a better probability.&lt;/p&gt;&#xA;&lt;p&gt;Is there any other similar method that uses fitness or any other way for an intelligent crossover for binary chromosomes?&lt;/p&gt;&#xA;" OwnerUserId="30164" LastEditorUserId="2444" LastEditDate="2020-12-07T19:10:44.803" LastActivityDate="2020-12-07T19:10:44.803" Title="Are there clever (fitness-based) crossover operators for binary chromosomes?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;crossover-operators&gt;&lt;chromosomes&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17507" PostTypeId="2" ParentId="17502" CreationDate="2020-01-14T23:48:43.667" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that function IS.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's usually called the &lt;strong&gt;loss function&lt;/strong&gt; (and, in general, &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;objective function&lt;/a&gt;) and often denoted as &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; (or something like that, i.e. it is not really important how you denote it). The specific function used as a loss function depends on the problem (ask another question if you want to know the details). For example, in the case of regression, the loss function may be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean squared error&lt;/a&gt;. In classification, the loss function may be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_loss_function_and_logistic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-entropy&lt;/a&gt;. However, the most important thing to note is that the loss function depends on the parameters of the neural network (NN), so you can differentiate it with respect to the parameters of the NN (i.e. you can take the &lt;a href=&quot;https://en.wikipedia.org/wiki/Partial_derivative&quot; rel=&quot;nofollow noreferrer&quot;&gt;partial derivative&lt;/a&gt; of the loss function with respect to each of the parameters of the NN). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take the example of the mean squared error function, which is defined as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{L}(\boldsymbol{\theta}) ={\frac {1}{n}}\sum _{i=1}^{n}(y_i-f_{\boldsymbol{\theta}}(x_i))^{2}.&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of training examples used (the batch size), &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is the true class (or target) of the input example &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(x_i)$&lt;/span&gt; is the prediction of the neural network &lt;span class=&quot;math-container&quot;&gt;$f_{\boldsymbol{\theta}}$&lt;/span&gt; with parameters (or weights) &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta} = [\theta_i, \dots, \theta_M] \in \mathbb{R}^M$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the number of parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(\boldsymbol{\theta})$&lt;/span&gt;, we can now take the derivative of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;, with respect to &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt;, using the famous &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;back-propagation&lt;/strong&gt;&lt;/a&gt; (BP) algorithm, which essentially applies the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;chain rule of calculus&lt;/a&gt;. The BP algorithm produces the &lt;strong&gt;gradient&lt;/strong&gt; of the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(\boldsymbol{\theta})$&lt;/span&gt;. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient&lt;/a&gt; can be denoted as &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L}(\boldsymbol{\theta})$&lt;/span&gt; and it contains the partial derivatives of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(\boldsymbol{\theta})$&lt;/span&gt; with respect to each parameter &lt;span class=&quot;math-container&quot;&gt;$\theta_i$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L}(\boldsymbol{\theta}) = \left[ \frac{\partial \mathcal{L}(\boldsymbol{\theta})}{\partial \theta_i}, \dots, \frac{\partial \mathcal{L}(\boldsymbol{\theta})}{\partial \theta_M} \right] \in \mathbb{R}^M$&lt;/span&gt;. (If you want to know the details of the back-propagation algorithm, you should ask another question, but make sure you get informed first, because it may not be easy to fully explain it in an answer.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterward, we just apply the &lt;strong&gt;gradient descent&lt;/strong&gt; step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol{\theta} \leftarrow \boldsymbol{\theta} - \gamma \nabla \mathcal{L}(\boldsymbol{\theta})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma \in \mathbb{R}$&lt;/span&gt; is often called the learning rate and is used to weight the contribution of the gradient &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L}(\boldsymbol{\theta})$&lt;/span&gt; to the new values of the parameters, and &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; represents an &lt;a href=&quot;https://en.wikipedia.org/wiki/Assignment_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;assignment (like in programming)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth emphasizing that both &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L}(\boldsymbol{\theta})$&lt;/span&gt; are vectors and have the same dimensionality (&lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at &lt;a href=&quot;https://ai.stackexchange.com/a/11675/2444&quot;&gt;this answer&lt;/a&gt; where I explain the difference between gradient descent and stochastic gradient descent. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-15T13:42:43.287" LastActivityDate="2020-01-15T13:42:43.287" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17508" PostTypeId="2" ParentId="17506" CreationDate="2020-01-15T01:18:47.977" Score="1" Body="&lt;p&gt;It's not obvious what you mean by &quot;intelligent crossover&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is common to use fitness-based selection of parents: individuals in the current population who have higher fitness are assigned a higher probability of being selected to mate and produce offspring.  This will increase the likelihood that &quot;good&quot; combinations of genes in members of the current population will be passed along to the next generation, and that independent &quot;good&quot; combinations will be combined in some members of the next generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;best&quot; crossover operator depends dramatically on the structure of the problem being solved, and on the mapping of gene &quot;vectors&quot; to the salient features of a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit #1:&#xA;In some cases it is important to increase diversity in order to avoid convergence to a local optimum.  In that case, an &quot;intelligent&quot; GA might for example select a first parent for its high fitness, and a second parent at random.  In &quot;Generator&quot;, a GA I sold for a while about 25 years ago, mate selection worked that way, and it was often very effective.  Generator also replaced any duplicate individuals in the population with entirely random individuals.  I have also structured genetic algorithms specifically to evolve multiple separate populations of individuals, with minimum gene flow between the populations, in order to evolve multiple solutions corresponding to &quot;regional&quot; fitness optima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit #2:&#xA;In genetic algorithms it is not common to directly seek the best combination of genes from both parents. The assumption is that higher-fitness parents are &lt;em&gt;more likely&lt;/em&gt; to produce even higher-fitness offspring than lower-fitness parents.  Sometimes there is a local search (hill climbing) operation where the offspring of two parents is mutated in various ways and the best of the mutants is put in the next generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, sometimes a crossover operation involves producing a larger number of offspring than the parent generation, followed by culling low-fitness individuals to keep the population size constant.  This is vaguely analogous to a local search via mutation, but uses random crossover instead of random mutation for its search.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastEditorUserId="28348" LastEditDate="2020-01-15T13:36:01.907" LastActivityDate="2020-01-15T13:36:01.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17510" PostTypeId="1" AcceptedAnswerId="17511" CreationDate="2020-01-15T05:18:46.020" Score="1" ViewCount="59" Body="&lt;p&gt;In my understanding, the formula to calculate the cross-entropy is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H(p,q) = - \sum p_i \log(q_i)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in PyTorch &lt;code&gt;nn.CrossEntropyLoss&lt;/code&gt; is calculated using this formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;loss = -\log\left( \frac{\exp(x[class])}{\sum_j \exp(x_j)} \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that I think it only addresses the &lt;span class=&quot;math-container&quot;&gt;$\log(q_i)$&lt;/span&gt; part in the first formula. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does PyTorch use a different formula for the cross-entropy?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2020-01-15T13:05:14.937" LastActivityDate="2020-01-15T13:05:14.937" Title="Why does PyTorch use a different formula for the cross-entropy?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;pytorch&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17511" PostTypeId="2" ParentId="17510" CreationDate="2020-01-15T07:52:55.613" Score="0" Body="&lt;p&gt;When you one-hot-encode your labels with &lt;span class=&quot;math-container&quot;&gt;$p_i \in \{0,1\}$&lt;/span&gt; you get &lt;span class=&quot;math-container&quot;&gt;$p_i = 0$&lt;/span&gt; iff &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is not correct and, equivalently, &lt;span class=&quot;math-container&quot;&gt;$p_i =1$&lt;/span&gt; iff &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, &lt;span class=&quot;math-container&quot;&gt;$p_i \log(q_i) = 0 \log(q_i) = 0 $&lt;/span&gt; for all classes except the &quot;truth&quot; and &lt;span class=&quot;math-container&quot;&gt;$p_i \log(q_i) = 1 \log(q_i) = \log(q_i) $&lt;/span&gt; for the correct prediction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, your loss reduces to:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H(p,q) = - \sum p_i \log(q_i) = - \log(q_{truth})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="30789" LastActivityDate="2020-01-15T07:52:55.613" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17512" PostTypeId="1" CreationDate="2020-01-15T09:04:46.550" Score="3" ViewCount="197" Body="&lt;p&gt;I am training a convolutional neural network for object detection. Apart from the learning rate, what are the other hyperparameters that I should tune? And in what order of importance? Besides, I read that doing a grid search for hyperparameters is not the best way to go about training, and that random search is better in this case. Is random search really that good? &lt;/p&gt;&#xA;" OwnerUserId="30497" LastEditorUserId="2444" LastEditDate="2020-01-15T13:11:11.650" LastActivityDate="2021-02-16T17:04:40.410" Title="When training a CNN, what are the hyperparameters to tune first?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;computer-vision&gt;&lt;optimization&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17513" PostTypeId="1" AcceptedAnswerId="17515" CreationDate="2020-01-15T09:48:10.630" Score="1" ViewCount="134" Body="&lt;p&gt;How is the batch loss calculated in both DQNs and simple classifiers? From what I understood, in a classifier, a common method is that you sample a mini-batch, calculate the loss for every example, calculate the average loss over the whole batch, and adjust the weights w.r.t the average loss?&#xA;(Please correct me if I'm wrong)&lt;/p&gt;&#xA;&lt;p&gt;But is this the same in DQNs? So, you sample a batch from your memory, say 64 transitions. Do I iterate through each transition and adjust the weights &amp;quot;on the fly&amp;quot;, or do I calculate the average loss of the batch and THEN in a big step adjust the weights w.r.t the average batch loss?&lt;/p&gt;&#xA;" OwnerUserId="30431" LastEditorUserId="2444" LastEditDate="2021-05-21T09:44:59.000" LastActivityDate="2021-05-21T09:45:26.293" Title="What is the difference between batches in deep Q learning and supervised learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;batch-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17515" PostTypeId="2" ParentId="17513" CreationDate="2020-01-15T10:23:10.227" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;From what I understood in a classifier a common method is that you sample a mini-batch, calculate the loss for every example, calculate the average loss over the whole batch and adjust the weights w.r.t the average loss? (Please correct me if I'm wrong)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are wrong.&lt;/p&gt;&#xA;&lt;p&gt;The weights are adjusted w.r.t. the average &lt;em&gt;gradient&lt;/em&gt;, and this must be calculated using &lt;em&gt;individual&lt;/em&gt; loss function results. The average loss (or cost function when considering the whole dataset) is a useful metric for current performance, and it is the measure being minimised. But you cannot calculate meaningful gradients against the average loss directly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;But is this the same in DQNs?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The batch process is not as you described, but an experience replay minibatch in RL and a sampled minibatch in supervised learning can be very similar. The main difference in RL is that your prediction targets must be recalculated as part of the sampling process (using &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+1} = R_{t+1} + \gamma \text{max}_{a'}\hat{q}(S_{t+1},a', \theta)$&lt;/span&gt; to calculate the TD target, assuming you are using single step Q learning), whilst in most supervised learning the target values are fixed for each example.&lt;/p&gt;&#xA;&lt;p&gt;In theory you could use repeated single item stochastic gradient descent in DQN, it doesn't break any theory, and it would work. However, it will usually be more efficient to use a standard minibatch update, combining all gradients into one average gradient for the minibatch and making a single update step.&lt;/p&gt;&#xA;&lt;p&gt;If you are using a high level library for your neural network model in DQN, you usually don't need to worry about this detail. You can use the &lt;code&gt;.fit&lt;/code&gt; function or whatever the library provides. In that case the only difference between a supervised learning update and an experience replay DQN update is what you get from the sampling. In supervised learning you get a set of &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}_i, \mathbf{y}_i)$&lt;/span&gt; examples directly by sampling a minibatch. In RL you get &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{s}_i, \mathbf{a}_i, r, \mathbf{s'}_i, done)$&lt;/span&gt; and must &lt;em&gt;construct&lt;/em&gt; the &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}_i, \mathbf{y}_i)$&lt;/span&gt; minibatch from these before passing to your &lt;code&gt;.fit&lt;/code&gt; function&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-05-21T09:45:26.293" LastActivityDate="2021-05-21T09:45:26.293" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17516" PostTypeId="1" CreationDate="2020-01-15T12:53:27.420" Score="1" ViewCount="14" Body="&lt;p&gt;Like I said in the title, I used a code I found to identify duplicate images however there are many false alarms. Can someone give me a link to some code I could use that really works? The problem with the precedent code was that because the underwater images have this bluish color, they are falsely presumed to be duplicates.&#xA;I'm using hashing but if there's any other efficient method, please feel free to tell me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help&lt;/p&gt;&#xA;" OwnerUserId="32622" LastEditorUserId="32622" LastEditDate="2020-01-15T13:04:00.990" LastActivityDate="2020-01-15T13:04:00.990" Title="Hashing images detects false duplicates" Tags="&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17517" PostTypeId="2" ParentId="17492" CreationDate="2020-01-15T13:23:40.073" Score="3" Body="&lt;p&gt;This question is very broad, so let me attempt to answer it using my own background in time series analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, an LSTM model is not good at modelling all time series. It is effective when it comes to modelling volatile data, but ARIMA still outperforms when it comes to forecasting trend data - LSTM tends to overemphasise volatile patterns in future predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take an example of forecasting weekly hotel cancellations by potential customers. The second time series shows much more variability in the number of weekly hotel cancellations than the first:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;H1 Time Series&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pY5xJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pY5xJ.png&quot; alt=&quot;h1 time series&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;H2 Time Series&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Iz7Go.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Iz7Go.png&quot; alt=&quot;h2 time series&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on &lt;strong&gt;MDA&lt;/strong&gt; (mean directional accuracy), &lt;strong&gt;RMSE&lt;/strong&gt; (root mean squared error), and &lt;strong&gt;MFE&lt;/strong&gt; (mean forecast error) - ARIMA demonstrates superior performance overall for the first time series, while LSTM shows better performance for the second:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jiuJV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jiuJV.png&quot; alt=&quot;performance&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the basis of this example - which is quite specific given the broadness of your question - deep learning techniques are not always used because simpler models can perform better under certain circumstances. It is all about understanding the data you are working with and then choosing the model - not the other way around.&lt;/p&gt;&#xA;" OwnerUserId="22692" LastEditorUserId="22692" LastEditDate="2020-03-02T16:55:09.053" LastActivityDate="2020-03-02T16:55:09.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17518" PostTypeId="2" ParentId="17498" CreationDate="2020-01-15T14:05:17.537" Score="1" Body="&lt;p&gt;It is quite common in DQN to instead of having the neural network represent function &lt;span class=&quot;math-container&quot;&gt;$f(s,a) = \hat{q}(s,a,\theta)$&lt;/span&gt; directly, it actually represents &lt;span class=&quot;math-container&quot;&gt;$f(s)= [\hat{q}(s,1,\theta), \hat{q}(s,2,\theta), \hat{q}(s,3,\theta) . . . \hat{q}(s,N_a,\theta)]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N_a$&lt;/span&gt; is the maximum action, and the input the current state. That is what is going on here. It is usually done for a performance gain, since calculating all values at once is faster than individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in a Q learning update, you cannot adjust this vector of output values for  actions that you did not take. You can do one of two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Figure out the gradient due to the one item with a TD error, and propagate that backwards. This involves inserting a known gradient into the normal training update step in a specific place and working from there. This works best if you are implementing your own backpropagation with low-level tools, otherwise it can be a bit fiddly figuring out how to do it in a framework like Keras.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Force the gradients of all other items to be zero by setting the target outputs to be whatever the learning network is currently generating.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you are using something like Keras, the second approach is the way to go. A concrete example where you have two networks &lt;code&gt;n_learn&lt;/code&gt; and &lt;code&gt;n_target&lt;/code&gt; that output arrays of Q values might be like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For each sample &lt;code&gt;(s, a, r, next_s, done)&lt;/code&gt; in your minibatch*&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calculate array of action values from your learning network &lt;code&gt;qvals = n_learn.predict(s)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Calulate TD target for &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; e.g. &lt;code&gt;td_target = r + max(n_target.predict(next_s))&lt;/code&gt; (discount factor and how to handle terminal states not shown)&lt;/li&gt;&#xA;&lt;li&gt;Alter the one array item that you know about from this sample &lt;code&gt;qvals[a] = td_target&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Append &lt;code&gt;s&lt;/code&gt; to your &lt;code&gt;train_X&lt;/code&gt; data and &lt;code&gt;qvals&lt;/code&gt; to your &lt;code&gt;train_Y&lt;/code&gt; data&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fit the minibatch &lt;code&gt;n_learn.fit(train_X, train_Y)&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* It is possible to vectorise these calculations for efficiency. I show it as a for loop as it is simpler to describe that way&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-15T14:16:38.860" LastActivityDate="2020-01-15T14:16:38.860" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17521" PostTypeId="1" AcceptedAnswerId="17871" CreationDate="2020-01-15T16:15:32.983" Score="2" ViewCount="51" Body="&lt;p&gt;I'm new in Artificial Intelligence and I want to do image segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching I have found these ways&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Digital image processing (I have read it in this book: &lt;a href=&quot;https://www.pearson.com/store/p/digital-image-processing/P100001327718&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digital Image Processing, 4th edition&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convolutional neural networks&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there something else that I can use?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2020-01-16T01:42:09.637" LastActivityDate="2020-02-06T01:41:45.977" Title="How many ways are there to perform image segmentation?" Tags="&lt;convolutional-neural-networks&gt;&lt;theory&gt;&lt;reference-request&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17522" PostTypeId="1" CreationDate="2020-01-15T17:59:24.043" Score="2" ViewCount="54" Body="&lt;p&gt;I am trying to use Logistic Regression to make a spam filter, but I am having trouble understanding the weight update part. I have processed my email dataset, and I have an attribute vector of the top n words that are most likely to be contained within a spam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, during training, I will have to implement an optimization formula after each training example in order to update the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_l \leftarrow w_l + \eta \cdot \sum_{i=1}^m [ y^{(i)} - P(c_+ \mid \vec{x}^{(i)} )] \cdot x_l^{(i)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does a formula such as this work? How can it be implemented in Python?&lt;/p&gt;&#xA;" OwnerUserId="32762" LastEditorUserId="2444" LastEditDate="2020-01-15T19:32:31.677" LastActivityDate="2020-04-13T14:49:53.440" Title="How does the weight update formula for logistic regression work?" Tags="&lt;machine-learning&gt;&lt;logistic-regression&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17523" PostTypeId="1" CreationDate="2020-01-15T23:20:48.003" Score="2" ViewCount="41" Body="&lt;p&gt;I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time.  One approach is, of course, to use two separate algorithms and find the threshold for each.  Then comparing the threshold of each.  In other words, if the accelerometer algorithm predicts a fall (fall detected = 1) and the EEG algorithm detects a fall, based on the power spectrum (fall detected = 1), then the system outputs a &quot;1&quot; that a fall was truly detected.  This approach uses the idea of a simple AND gate between the two algorithms.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know how to correctly process the data so that I can feed both types of data into one algorithm, perhaps a CNN. Any advice is really appreciated, even a lead to some literature, articles or information would be great.  &lt;/p&gt;&#xA;" OwnerUserId="32771" LastEditorUserId="2444" LastEditDate="2020-01-16T22:58:43.570" LastActivityDate="2020-01-16T22:58:43.570" Title="How can I compare EEG data with accelerometer data in 1 algorithm?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;datasets&gt;&lt;computational-linguistics&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17524" PostTypeId="1" CreationDate="2020-01-16T02:33:22.700" Score="1" ViewCount="25" Body="&lt;p&gt;In 'Deep Reinforcement Learning Hands-On' book and chapter about Distributional C51 algorithm I'm reading, that to obtain Q-values from the distribution I need to calculate the weighted sum of the normalized distribution and atom's values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why I have to multiply that distribution with support? How does it work and what happening there?&lt;/p&gt;&#xA;" OwnerUserId="32540" LastActivityDate="2020-01-16T02:33:22.700" Title="Why we multiply probabilities with support to obtain Q-values in Distributional C51 algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17525" PostTypeId="1" CreationDate="2020-01-16T04:57:48.913" Score="2" ViewCount="53" Body="&lt;p&gt;This is part of the &lt;a href=&quot;https://mitpress.ublish.com/ereader/7093/?preview=#page/27&quot; rel=&quot;nofollow noreferrer&quot;&gt;exercise 2.13&lt;/a&gt; in the book &lt;a href=&quot;https://mitpress.ublish.com/ereader/7093/?preview=#page/27&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations of Machine Learning (page 28)&lt;/a&gt;. You can refer to &lt;a href=&quot;https://mitpress.ublish.com/ereader/7093/?preview=#page/9&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 2&lt;/a&gt; for the notations.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Consider a family of concept classes &lt;span class=&quot;math-container&quot;&gt;$\left\{\mathcal{C}_{s}\right\}_{s}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}_{s}$&lt;/span&gt; is the set of concepts in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; with size at most &lt;span class=&quot;math-container&quot;&gt;$s.$&lt;/span&gt; Suppose we have a PAC-learning algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; that can be used for learning any concept class &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}_{s}$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is given. Can we convert &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; into a PAC-learning algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}$&lt;/span&gt; that does not require the knowledge of &lt;span class=&quot;math-container&quot;&gt;$s ?$&lt;/span&gt; This is the main objective of this problem.&lt;/p&gt;&#xA;&lt;p&gt;To do this, we first introduce a method for testing a hypothesis &lt;span class=&quot;math-container&quot;&gt;$h,$&lt;/span&gt; with high probability. Fix &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0, \delta&amp;gt;0,$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$i \geq 1$&lt;/span&gt; and define the sample size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$n=\frac{32}{\epsilon}\left[i \log 2+\log \frac{2}{\delta}\right].$&lt;/span&gt; Suppose we draw an i.i.d. sample &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; of size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; according to some unknown distribution &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}.$&lt;/span&gt; We will say that a hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is accepted if it makes at most &lt;span class=&quot;math-container&quot;&gt;$3 / 4 \epsilon$&lt;/span&gt; errors on &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and that it is rejected otherwise. Thus, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is accepted iff &lt;span class=&quot;math-container&quot;&gt;$\widehat{R}(h) \leq 3 / 4 \epsilon$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(a) Assume that &lt;span class=&quot;math-container&quot;&gt;$R(h) \geq \epsilon .$&lt;/span&gt; Use the (multiplicative) Chernoff bound to show that in that case &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}_{S \sim D^{n}}[h \text { is accepted}] \leq \frac{\delta}{2^{i+1}}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(b) Assume that &lt;span class=&quot;math-container&quot;&gt;$R(h) \leq \epsilon / 2 .$&lt;/span&gt; Use the (multiplicative) Chernoff bounds to show that in that case &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}_{S \sim \mathcal{D}^{n}}[h \text { is rejected }] \leq \frac{\delta}{2^{i+1}}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(c) Algorithm &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}$&lt;/span&gt; is defined as follows: we start with &lt;span class=&quot;math-container&quot;&gt;$i=1$&lt;/span&gt; and, at each round &lt;span class=&quot;math-container&quot;&gt;$i \geq 1,$&lt;/span&gt; we guess the parameter size &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to be &lt;span class=&quot;math-container&quot;&gt;$\widetilde{s}=\left\lfloor 2^{(i-1) / \log \frac{2}{\delta}}\right\rfloor .$&lt;/span&gt; We draw a sample &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; of size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (which depends on &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; ) to test the hypothesis &lt;span class=&quot;math-container&quot;&gt;$h_{i}$&lt;/span&gt; returned by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; when it is trained with a sample of size &lt;span class=&quot;math-container&quot;&gt;$S_{\mathcal{A}}(\epsilon / 2,1 / 2, \widetilde{s}),$&lt;/span&gt; that is the sample complexity of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; for a required precision &lt;span class=&quot;math-container&quot;&gt;$\epsilon / 2,$&lt;/span&gt; confidence &lt;span class=&quot;math-container&quot;&gt;$1 / 2,$&lt;/span&gt; and size &lt;span class=&quot;math-container&quot;&gt;$\tilde{s}$&lt;/span&gt; (we ignore the size of the representation of each example here). If &lt;span class=&quot;math-container&quot;&gt;$h_{i}$&lt;/span&gt; is accepted, the algorithm stops and returns &lt;span class=&quot;math-container&quot;&gt;$h_{i},$&lt;/span&gt; otherwise it proceeds to the next iteration. Show that if at iteration &lt;span class=&quot;math-container&quot;&gt;$i,$&lt;/span&gt; the estimate &lt;span class=&quot;math-container&quot;&gt;$\widetilde{s}$&lt;/span&gt; is larger than or equal to &lt;span class=&quot;math-container&quot;&gt;$s,$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}\left[h_{i} \text { is accepted}\right] \geq 3 / 8$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Question (a) and (b) are easy to prove, but I have trouble with the question (c). More specifically, I don't know how to use the condition that &lt;span class=&quot;math-container&quot;&gt;$\widetilde{s} \geq s$&lt;/span&gt;. Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="32673" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-01-16T11:17:01.310" Title="Convert a PAC-learning algorithm into another one which requires no knowledge of the parameter" Tags="&lt;machine-learning&gt;&lt;proofs&gt;&lt;probability&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17526" PostTypeId="1" CreationDate="2020-01-16T05:55:54.483" Score="1" ViewCount="32" Body="&lt;p&gt;I’ve been reading about neural network architectures. In certain cases, people say that the sigmoid &quot;more accurately reflects real-life&quot; and, in other cases, functions like hard limits reflect &quot;the brain neural networks more accurately&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What activation functions are better for what problems?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-01-16T11:20:26.763" LastActivityDate="2020-01-16T11:20:26.763" Title="What activation functions are better for what problems?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17527" PostTypeId="2" ParentId="17491" CreationDate="2020-01-16T06:47:04.110" Score="0" Body="&lt;p&gt;I think the problem is that you are not defining the weights and biases as parameters. So, when you backpropagate, they are not modified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These lines should do the trick:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.weight_mu = Parameter(torch.Tensor(out_features, in_features))&#xA;self.weight_sigma = Parameter(torch.Tensor(out_features, in_features))&#xA;self.bias_mu = Parameter(torch.Tensor(out_features))&#xA;self.bias_sigma = Parameter(torch.Tensor(out_features))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In case you are not familiar, the &lt;code&gt;Parameter&lt;/code&gt; class must be imported from &lt;code&gt;torch&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from torch.nn.parameter import Parameter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30983" LastActivityDate="2020-01-16T06:47:04.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17528" PostTypeId="1" AcceptedAnswerId="17534" CreationDate="2020-01-16T08:12:05.673" Score="0" ViewCount="44" Body="&lt;p&gt;I'm trying to implement deep q learning in the OpenAI's gym &quot;Taxi-v3&quot; environment. But my agent only learns to do one action in every state. What am I doing wrong? Here is the &lt;a href=&quot;https://github.com/OleVoss/DeepTaxi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repository with the code&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="30431" LastEditorUserId="2444" LastEditDate="2020-05-20T11:53:14.947" LastActivityDate="2020-05-20T11:53:14.947" Title="Why is this deep Q agent constantly learning just one action?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;pytorch&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17529" PostTypeId="1" AcceptedAnswerId="17541" CreationDate="2020-01-16T08:23:00.180" Score="2" ViewCount="102" Body="&lt;p&gt;I'm a newbie in artificial intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have started to research how to do image segmentation and all the papers that I have found are about CNN. Most of them use the same network, U-net, but with little variations: with more or fewer layers, different parameter values, etc.; but with not very different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that CNNs are in fashion and everyone uses them. Or there are other reasons that I don't know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If everyone is getting not very different results, why are they using the same approach instead of trying different ones?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2020-06-13T00:11:22.530" LastActivityDate="2020-06-13T00:23:40.503" Title="Why everyone is using CNN for image segmentation?" Tags="&lt;convolutional-neural-networks&gt;&lt;applications&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17530" PostTypeId="1" AcceptedAnswerId="17532" CreationDate="2020-01-16T08:24:37.940" Score="3" ViewCount="126" Body="&lt;p&gt;I am new to Reinforcement learning and am currently reading up on the estimation of Q &lt;span class=&quot;math-container&quot;&gt;$\pi(s, a)$&lt;/span&gt; values using MC epsilon-soft approach and chanced upon this algorithm. The link to the algorithm is found from this website. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.analyticsvidhya.com/blog/2018/11/reinforcement-learning-introduction-monte-carlo-learning-openai-gym/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.analyticsvidhya.com/blog/2018/11/reinforcement-learning-introduction-monte-carlo-learning-openai-gym/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def monte_carlo_e_soft(env, episodes=100, policy=None, epsilon=0.01):&#xA;&#xA;if not policy:&#xA;    policy = create_random_policy(env)&#xA;# Create an empty dictionary to store state action values&#xA;Q = create_state_action_dictionary(env, policy)&#xA;&#xA;# Empty dictionary for storing rewards for each state-action pair&#xA;returns = {} # 3.&#xA;&#xA;for _ in range(episodes): # Looping through episodes&#xA;    G = 0 # Store cumulative reward in G (initialized at 0)&#xA;    episode = run_game(env=env, policy=policy, display=False) # Store state, action and value respectively&#xA;&#xA;    # for loop through reversed indices of episode array.&#xA;    # The logic behind it being reversed is that the eventual reward would be at the end.&#xA;    # So we have to go back from the last timestep to the first one propagating result from the future.&#xA;&#xA;    # episodes = [[s1,a1,r1], [s2,a2,r2], ... [Sn, an, Rn]]&#xA;    for i in reversed(range(0, len(episode))):&#xA;        s_t, a_t, r_t = episode[i]&#xA;        state_action = (s_t, a_t)&#xA;        G += r_t # Increment total reward by reward on current timestep&#xA;&#xA;        # if state - action pair not found in the preceeding episodes,&#xA;        # then this is the only time the state appears in this episode.&#xA;&#xA;        if not state_action in [(x[0], x[1]) for x in episode[0:i]]: #&#xA;            # if returns dict contains a state action pair from prev episodes,&#xA;            # append the curr reward to this dict&#xA;            if returns.get(state_action):&#xA;                returns[state_action].append(G)&#xA;            else:&#xA;                # create new dictionary entry with reward&#xA;                returns[state_action] = [G]&#xA;&#xA;            # returns is a dictionary that maps (s,a) : [G1,G2, ...]&#xA;            # Once reward is found for this state in current episode,&#xA;            # average the reward.&#xA;            Q[s_t][a_t] = sum(returns[state_action]) / len(returns[state_action]) # Average reward across episodes&#xA;&#xA;            # Finding the action with maximum value.&#xA;&#xA;&#xA;&#xA;            Q_list = list(map(lambda x: x[1], Q[s_t].items()))&#xA;            indices = [i for i, x in enumerate(Q_list) if x == max(Q_list)]&#xA;            max_Q = random.choice(indices)&#xA;&#xA;            A_star = max_Q # 14.&#xA;&#xA;            # Update action probability for s_t in policy&#xA;            for a in policy[s_t].items():&#xA;                if a[0] == A_star:&#xA;                    policy[s_t][a[0]] = 1 - epsilon + (epsilon / abs(sum(policy[s_t].values())))&#xA;                else:&#xA;                    policy[s_t][a[0]] = (epsilon / abs(sum(policy[s_t].values())))&#xA;&#xA;return policy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This algorithm computes the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for all state action value pairs that the policy follows. If &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a random policy, and after running through this algorithm, and for each state take the &lt;span class=&quot;math-container&quot;&gt;$\max Q(s,a)$&lt;/span&gt; for all possible actions, why would that not be equal to &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi^*}(s, a)$&lt;/span&gt; (optimal Q function)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this website, they claim to have been able to find the optimal policy when running through this algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read up a bit on Q-learning and the update equation is different from MC epsilon-soft. However, I can't seem to understand clearly how these 2 approaches are different. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-01-16T11:29:49.563" LastActivityDate="2020-01-19T08:46:11.137" Title="Why Monte Carlo epsilon-soft approach cannot compute $\max Q(s,a)$?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17531" PostTypeId="1" CreationDate="2020-01-16T09:29:00.540" Score="1" ViewCount="48" Body="&lt;p&gt;I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought, like the Deepmind &quot;Playing Atari with Deep Reinforcement Learning&quot; Paper, I could just use only the pixel input, fed through 2 convolutional layers, to solve this task. The output of this is fed into 512 fully connected layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, it barely trains. &#xA;When instead using additional vectors as input containing information about nearby pixels' state around the agent, the agent learns the task quite well (yet often orients the wrong awkwardly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each step, the agent moves up down left right, and plot pixel or not.&#xA;The agent is visualized in the environemtn as a red square with white center dot. (also tried single red pixel). On-pixels within the red square are colored purple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something I can try to make the agent learn visual input better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The orange line is the training with only visual observations, the grey one contained vector observations about the immediate neighboring pixel state as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zyab3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zyab3.png&quot; alt=&quot;agent&amp;#39;s environment&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/77SEa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/77SEa.png&quot; alt=&quot;Training curves&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31180" LastActivityDate="2020-01-16T09:29:00.540" Title="Reinforcement learning CNN input weakness" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17532" PostTypeId="2" ParentId="17530" CreationDate="2020-01-16T09:40:33.220" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a random policy, and after running through this algorithm, and for each state take the &lt;span class=&quot;math-container&quot;&gt;$\max Q(s,a)$&lt;/span&gt; for all possible actions, why would that not be equal to &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi^*}(s, a)$&lt;/span&gt; (optimal Q function)? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming that the estimates for &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(s,a)$&lt;/span&gt; have converged to close to correct values from many samples, then a policy based on &lt;span class=&quot;math-container&quot;&gt;$\pi'(s) = \text{argmax}_a Q_{\pi}(s,a)$&lt;/span&gt; is not guaranteed to be an optimal policy &lt;em&gt;unless&lt;/em&gt; the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; being measured is &lt;em&gt;already&lt;/em&gt; the optimal policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because the action value &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(s,a)$&lt;/span&gt; gives the expected future reward from taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and from that point on following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. The function does not, by itself, adapt to the idea that you might change other action choices as well. It is a measure of &lt;em&gt;immediate&lt;/em&gt; differences between action choices at any given time step. Therefore if there are any long-term dependencies where your action choice at &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; would be different if only you could guarantee a certain choice at &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; or later, this cannot be resolved by simply taking the maximum &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(s,a)$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; was a simple random policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you do decide to change the policy such that you always follow actions &lt;span class=&quot;math-container&quot;&gt;$\pi'(s) = \text{argmax}_a Q_{\pi}(s,a)$&lt;/span&gt; for all states, then you can say this: &lt;em&gt;For each state, &lt;span class=&quot;math-container&quot;&gt;$V_{\pi'}(s) \ge V_{\pi}(s)$&lt;/span&gt;.&lt;/em&gt; I.e.  &lt;span class=&quot;math-container&quot;&gt;$\pi'(s)$&lt;/span&gt; is no worse than, and may be a strict improvement over &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt;. Better than that, &lt;span class=&quot;math-container&quot;&gt;$\pi'(s)$&lt;/span&gt; will be a strict improvment over &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(s,a)$&lt;/span&gt; is accurate and &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; is not already the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*(s)$&lt;/span&gt;. This is the basis for the &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node42.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy Improvement Theorem&lt;/a&gt; which shows that if you repeat the process of measuring &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi^k}$&lt;/span&gt; and then creating a new policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{k+1}(s) = \text{argmax}_a Q_{\pi^k}(s,a)$&lt;/span&gt; that you will eventually find the optimal policy. You only have to repeat your idea many times to eventually find &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Dynamic Programming technique Policy Iteration does this exactly. All other value-based Reinforcement Learning methods are variations of this idea and rely at least in part on this proof.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-19T08:46:11.137" LastActivityDate="2020-01-19T08:46:11.137" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17533" PostTypeId="1" CreationDate="2020-01-16T11:01:20.123" Score="2" ViewCount="54" Body="&lt;p&gt;This is an inequality on page 36 of the book Foundations of Machine Learning, but the author only states it without proof.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{P}\left[\left|R(h)-\widehat{R}_{S}(h)\right|&amp;gt;\epsilon\right] \leq 4 \Pi_{\mathcal{H}}(2 m) \exp \left(-\frac{m \epsilon^{2}}{8}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the growth function &lt;span class=&quot;math-container&quot;&gt;$\Pi_{\mathcal{F}}: \mathbb{N} \rightarrow \mathbb{N}$&lt;/span&gt; for a hypothesis set &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; is defined by:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall m \in \mathbb{N}, \Pi_{\mathcal{F}}(m)=\max _{\left\{x_{1}, \ldots, x_{m}\right\} \subseteq X}\left|\left\{\left(h\left(x_{1}\right), \ldots, h\left(x_{m}\right)\right): h \in \mathcal{H}\right\}\right|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a hypothesis h &lt;span class=&quot;math-container&quot;&gt;$\in \mathcal{H},$&lt;/span&gt; a target concept &lt;span class=&quot;math-container&quot;&gt;$c \in \mathcal{C}$&lt;/span&gt; and an underlying distribution &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D},$&lt;/span&gt; the generalization error or risk of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is defined by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R(h)=\underset{x \sim D}{\mathbb{P}}[h(x) \neq c(x)]=\underset{x \sim D}{\mathbb{E}}\left[1_{h(x) \neq c(x)}\right]&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$1_{\omega}$&lt;/span&gt; is the indicator function of the event &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the empirical error or empirical risk of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is defined&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\widehat{R}_{S}(h)=\frac{1}{m} \sum_{i=1}^{m} 1_{h\left(x_{i}\right) \neq c\left(x_{i}\right)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the book, the author proves another inequality that differs from this one by only a constant using Rademacher complexity, but he says that the stated inequality can be proved without using Rademacher complexity. Does anyone know how to prove it?&lt;/p&gt;&#xA;" OwnerUserId="32673" LastEditorUserId="2444" LastEditDate="2020-01-16T11:41:05.303" LastActivityDate="2020-01-16T11:41:05.303" Title="How can we prove this inequality, related to the generalization error, without using the Rademacher complexity?" Tags="&lt;machine-learning&gt;&lt;proofs&gt;&lt;probability&gt;&lt;computational-learning-theory&gt;&lt;generalization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17534" PostTypeId="2" ParentId="17528" CreationDate="2020-01-16T11:02:36.260" Score="1" Body="&lt;p&gt;I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the &quot;CardPole-v0&quot; and it worked. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So think about your input guys &lt;/p&gt;&#xA;" OwnerUserId="30431" LastActivityDate="2020-01-16T11:02:36.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17535" PostTypeId="5" CreationDate="2020-01-16T11:41:44.567" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-16T11:41:44.567" LastActivityDate="2020-01-16T11:41:44.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17536" PostTypeId="4" CreationDate="2020-01-16T11:41:44.567" Score="0" Body="For questions related to the concept of generalization in computational learning theory and machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-16T11:41:44.567" LastActivityDate="2020-01-16T11:41:44.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17537" PostTypeId="1" CreationDate="2020-01-16T12:44:11.397" Score="2" ViewCount="288" Body="&lt;p&gt;This problem is about two-oracle variant of the PAC model. Assume that positive and negative examples are now drawn from two separate distributions &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}_{+}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}_{-} .$&lt;/span&gt; For an accuracy &lt;span class=&quot;math-container&quot;&gt;$(1-\epsilon),$&lt;/span&gt; the learning algorithm must find a hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; such that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\underset{x \sim \mathcal{D}_{+}}{\mathbb{P}}[h(x)=0] \leq \epsilon \text { and } \underset{x \sim \mathcal{D}_{-}}{\mathbb{P}}[h(x)=1] \leq \epsilon$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the hypothesis must have a small error on both distributions. Let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; be any concept class and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; be any hypothesis space. Let &lt;span class=&quot;math-container&quot;&gt;$h_{0}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{1}$&lt;/span&gt; represent the identically 0 and identically 1 functions, respectively. Prove that &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; is efficiently PAC-learnable using &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; in the standard (one-oracle) PAC model if and only if it is efficiently PAC-learnable using &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H} \cup\left\{h_{0}, h_{1}\right\}$&lt;/span&gt; in this two-oracle PAC model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wonder if the problem is correct. In the &lt;a href=&quot;https://cs.nyu.edu/~mohri/ml05/sol1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;official solution&lt;/a&gt;, when showing that 2-oracle implies 1-oracle, the author returns &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; when the distribution is too biased towards positive or negative examples. However, in the problem, it is required that only in 2-oracle case we can return &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt;. Therefore, in this too-biased case, it seems that there may not exist a 'good' hypothesis at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this problem wrong? Or I make some mistake somewhere?&lt;/p&gt;&#xA;" OwnerUserId="32673" LastEditorUserId="2444" LastEditDate="2020-01-16T13:30:17.450" LastActivityDate="2020-01-16T13:30:17.450" Title="A problem about the relation between 1-oracle and 2-oracle PAC model" Tags="&lt;machine-learning&gt;&lt;proofs&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17538" PostTypeId="1" CreationDate="2020-01-16T12:54:35.973" Score="1" ViewCount="92" Body="&lt;p&gt;I'm working on an advantage actor-critic (A2C) reinforcement learning model but the problem when I trained the system for 3500 episodes, I start to get the same action for all my testing results. While if I trained the system for less than 850 episodes, I got different actions. The value of &lt;code&gt;state&lt;/code&gt; is always different and around 850 episodes the &lt;code&gt;loss&lt;/code&gt; becomes zero.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  def take_action(self, state):&#xA;            &quot;&quot;&quot;Take the action&quot;&quot;&quot;&#xA;            action_probs = self.actor.predict(state)&#xA;            action = np.random.choice(np.arange(len(action_probs)), p=action_probs)&#xA;            return action&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This the &lt;code&gt;actor.predict&lt;/code&gt; function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def predict(self, s):&#xA;        return self._sess.run(self._action_probs, {self._state: s})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any Idea what causing this?&lt;/p&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="21181" LastEditDate="2020-01-16T15:04:59.050" LastActivityDate="2020-01-16T15:04:59.050" Title="Why I got the same action when I train A2C when I increase the number of episodes?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17539" PostTypeId="1" CreationDate="2020-01-16T13:25:46.480" Score="2" ViewCount="90" Body="&lt;p&gt;How can I show that the VC dimension of the set of all closed balls in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; is at most &lt;span class=&quot;math-container&quot;&gt;$n+3$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this problem, I only try the case &lt;span class=&quot;math-container&quot;&gt;$n=2$&lt;/span&gt; for 1. When &lt;span class=&quot;math-container&quot;&gt;$n=2$&lt;/span&gt;, consider 4 points &lt;span class=&quot;math-container&quot;&gt;$A,B,C,D$&lt;/span&gt; and if one point is inside the triangle formed by the other three, then we cannot find a circle that only excludes this point. If &lt;span class=&quot;math-container&quot;&gt;$ABCD$&lt;/span&gt; is convex assume WLOG that &lt;span class=&quot;math-container&quot;&gt;$\angle ABC + \angle ADC \geq 180$&lt;/span&gt; then use some geometric argument to show that a circle cannot include &lt;span class=&quot;math-container&quot;&gt;$A,C$&lt;/span&gt; and exclude &lt;span class=&quot;math-container&quot;&gt;$B,D$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the general case I’m thinking of finding &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt; points so that a ball should be quite ‘large‘ to include them, and that this ball can not exclude the other 2 points.  However, in high-dimensional case I do not know how to use maths language to describe what is ‘large’.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give some ideas to this question please?&lt;/p&gt;&#xA;" OwnerUserId="32673" LastEditorUserId="2444" LastEditDate="2020-01-16T14:38:58.327" LastActivityDate="2020-01-16T14:38:58.327" Title="How can I show that the VC dimension of the set of all closed balls in $\mathbb{R}^n$ is at most $n+3$?" Tags="&lt;proofs&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17540" PostTypeId="1" CreationDate="2020-01-16T16:26:52.313" Score="1" ViewCount="204" Body="&lt;p&gt;My understanding of how non-max suppression works is that it suppresses all overlapping boxes that have a Jaccard overlap smaller than a threshold (e.g. 0.5). The boxes to be considered are on a confident score (may be 0.2 or something). So, if there are boxes that have a score over 0.2 (e.g. the score is 0.3 and overlap is 0.4) the boxes won't be suppressed.&lt;/p&gt;&#xA;&lt;p&gt;In this way, one object will be predicted by many boxes, one high score box, and many low confident score boxes, but I found that the model predicts only one box for one object. Can someone enlighten me?&lt;/p&gt;&#xA;&lt;p&gt;I currently viewing the ssd from  &lt;a href=&quot;https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#Finding Jaccap Overlap and sorting scotes&#xA;class_scores, sort_ind = class_scores.sort(dim=0, descending=True)&#xA;class_decoded_locs = class_decoded_locs[sort_ind]  # (n_min_score, 4)&#xA;overlap = find_jaccard_overlap(class_decoded_locs, class_decoded_locs)  &#xA;suppress = torch.zeros((n_above_min_score), dtype=torch.uint8).to(device)  &#xA;&#xA;for box in range(class_decoded_locs.size(0)):&#xA;# If this box is already marked for suppression&#xA;    if suppress[box] == 1:&#xA;    continue&#xA;    suppress = torch.max(suppress, overlap[box] &amp;gt; max_overlap)&#xA;    suppress[box] = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32794" LastEditorUserId="2444" LastEditDate="2020-09-12T17:31:44.807" LastActivityDate="2020-09-12T17:31:44.807" Title="How does non-max suppression work when one or multiple bounding boxes are predicted for the same object?" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;non-max-suppression&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17541" PostTypeId="2" ParentId="17529" CreationDate="2020-01-16T17:19:02.460" Score="2" Body="&lt;p&gt;CNN is used since it is effectively an optimized use case for dealing with image data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CNN effectively automatically extracts features from images. Other techniques are more likely to not take full advantage of the data. CNN is able to make full use of the data by also including information from adjacent pixels and downsample through layers.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1877050918309335&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a paper on the performance of CNN on image data&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1910.06548.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a paper comparing CNN's to other methods&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32719" LastEditorUserId="2444" LastEditDate="2020-06-13T00:23:40.503" LastActivityDate="2020-06-13T00:23:40.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17542" PostTypeId="2" ParentId="17334" CreationDate="2020-01-16T17:41:15.850" Score="0" Body="&lt;p&gt;random forest's feature importances are not reliable and you should probably avoid them. Instead you can use &lt;code&gt;permutation_importance&lt;/code&gt;: &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32797" LastActivityDate="2020-01-16T17:41:15.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17543" PostTypeId="1" AcceptedAnswerId="17555" CreationDate="2020-01-16T18:09:24.093" Score="2" ViewCount="42" Body="&lt;p&gt;I was looking into the use of a greedy layer-wise pretraining to initialize the weights of my network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just for the sake of clarity: I'm referring to the use of gradually deeper and deeper autoencoders to teach the network gradually more abstract representations of the input one layer at the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, reading &lt;a href=&quot;https://machinelearningmastery.com/greedy-layer-wise-pretraining-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;HERE&lt;/strong&gt;&lt;/a&gt;, I read:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Nevertheless, it is likely better performance may be achieved using modern methods such as better activation functions, weight initialization, variants of gradient descent, and regularization methods.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Today, we now know that greedy layer-wise pretraining is not required to train fully connected deep architectures, but the unsupervised pretraining approach was the first method to succeed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My question is then: if I'm building a network already using &quot;&lt;em&gt;modern&lt;/em&gt;&quot; techniques, such as ReLU activations, batch normalization, adam optimizers, etc, is the good-ol' greedy layer-wise pretraining useless? Or can it still provide an edge in the initialization of the network?&lt;/p&gt;&#xA;" OwnerUserId="32799" LastEditorUserId="32799" LastEditDate="2020-01-17T10:33:20.730" LastActivityDate="2020-01-17T15:57:39.543" Title="Is greedy layer-wise pretraining obsolete?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17544" PostTypeId="1" CreationDate="2020-01-16T18:11:38.077" Score="2" ViewCount="111" Body="&lt;p&gt;I having trouble finding some starting points for solving an occupancy  problem which seems like a good candidate for ai.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume the following situation:&#xA;In a company I have n cars and m employees. Not every employee can drive any car (f. e. a special driving license is required). A car can only be used by one employee at a specific point in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a plan which states which employee must be somewhere within some time (therefor they must use a car, so the car is blocked for that amount of time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to find a near optimal occupancy of the cars according to that plan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is easy to specify, but I'm stumped as to which methods to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it can be represented by a graph I think the right way to solve such a problem is using searching techniques, but a problem here is that I don't know the goal state (and there is no efficient way to compute it - thats the task I want the ai to do...). Neither finding the goal state is in fact part of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: What ai techniques could be used to solve such a problem ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Some clarification:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assmume we have two sets - one of the employees (E) and one of cars (C). |C| &amp;lt; |E| is most likely true.&#xA;Each car has an assigned priority which corresponds to the costs of using it (for example using a Ferrari costs more than using a Dacia, therefore a Dacia has a higher priority (ex. 1) compred to the Ferrari (ex. 10))).&#xA;Assume further that having employees which are not using a car at a specific time slice are a bad thing - they cost an individual penalty (you want the employeed to be at the customer and sell things etc.).&lt;br&gt;&#xA;The goal is to find the occupation of employees and cars which has a low total cost. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One Example: If you assign an employee to a car at a specific time slice  it may turn out that another employee gets no car within that time slice.&#xA;This can be either because &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a car is free, but he has no license for it   &lt;/li&gt;&#xA;&lt;li&gt;because a car is free, but the costs of using this car would be higher than having the employee staing at the head quater &lt;/li&gt;&#xA;&lt;li&gt;because no car is free anymore&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of cause it could be better in terms of costs to change the occupation and give that employee which got no car in this solution a car and therefore having another employee getting no car or not using all cars or ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: There is no need to find an exact optimal solution (=lowest total cost of all possible occupations), as this would require checking out all possible occupations of the exponential solution space.&#xA;Insetad finding a more or less good approximation of a near-optimal low total cost is sufficent.&lt;/p&gt;&#xA;" OwnerUserId="32800" LastEditorUserId="32800" LastEditDate="2020-01-20T08:49:22.933" LastActivityDate="2020-01-24T17:13:21.730" Title="Solving a planning if finding the goal state is part of the problem" Tags="&lt;algorithm&gt;&lt;optimization&gt;&lt;problem-solving&gt;&lt;graph-theory&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17546" PostTypeId="2" ParentId="9766" CreationDate="2020-01-16T23:00:42.540" Score="0" Body="&lt;p&gt;I guess they are talking about adversarial attacks in the same way Szegedy et al. did in &lt;a href=&quot;https://arxiv.org/abs/1312.6199&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Intriguing properties of neural networks&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They described &quot;adversarial attacks&quot; or &quot;adversarial examples&quot; as images with hardly perceptible perturbations that change the network's prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, imagine you've trained a CNN to classify between a variety of classes. You take a picture of a dog (&lt;code&gt;X_1&lt;/code&gt;), and your CNN correctly classified it as a &quot;dog&quot;, everything is fine so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can add some small perturbation (&lt;code&gt;p&lt;/code&gt;) to your image, so now you have a new image (&lt;code&gt;X_2 = X_1 + p&lt;/code&gt;), this new image still looks like a dog because your perturbation was so small that is almost imperceptible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that your CNN will classify your picture &lt;code&gt;X_2&lt;/code&gt;as a &quot;Fish&quot;, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;X_2&lt;/code&gt; is an adversarial attack created after using an adversarial perturbation &lt;code&gt;p&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is interesting about these adversarial perturbations &lt;code&gt;p&lt;/code&gt; is that they are not random. Actually CNNs are very robust to random perturbations (noise) but adversarial perturbations &lt;code&gt;p&lt;/code&gt; are not like them, they can be computed to fool a classifier (not only CNNs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can refer to figure 5 of the aforementioned paper for more examples.&lt;/p&gt;&#xA;" OwnerUserId="32360" LastActivityDate="2020-01-16T23:00:42.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17547" PostTypeId="1" CreationDate="2020-01-17T06:18:20.213" Score="1" ViewCount="45" Body="&lt;p&gt;I am trying to decide what GPU or GPUs to buy to run tf-pose pose detection and yolo3 object detection on several cameras. I need to keep an acceptable frame rate too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;what kind of GPU configuration should I go for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;does having multiple GPUs have an effect on what I am trying to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can you please explain the difference between using 2-4 1080ti(any GPU in that price range) cards and using something 10 times more expensive than that such as v100.in &lt;strong&gt;training and using the networks both&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="32814" LastActivityDate="2020-01-17T18:27:12.443" Title="Multiple GPUs one expensive GPU, which gpu to buy for real time processing (not training)" Tags="&lt;tensorflow&gt;&lt;deep-neural-networks&gt;&lt;gpu&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-09-29T22:32:00.033" ContentLicense="CC BY-SA 4.0" />
  <row Id="17548" PostTypeId="1" CreationDate="2020-01-17T08:29:46.560" Score="1" ViewCount="15" Body="&lt;p&gt;What is the status of the research on regional specialization of the artificial neural networks? Biology knows that such specialization exists in the brain and it is very important for the functioning of the brain. My thinking is that specialization can solve the transfer learning/catastrophic forgetting by creating centers of sophisticated skills if such sophistication is necessary. Actually - there is no much alternative to specialization? If specialization exists then there can be small decision centers that routes the request to the actual part and such decision centers can be efficient. But if specialization does not exist then routing happens in the total soup/pudding of neuronal see and such all-to-all routing should be very inefficient. Of course there should be some mixing of specialization vs pudding, because there is always mixing between rationality and emotions, between execution and exploration, but nevertheless - specialization should happen, at least partially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is - that I can not find any focused article about such specialization and I can not find how such specialization can be trained? There are research on hierarchical reinforcement learning - but that is about imposing the external fixed structure on the set of neural networks, but it is not how the nature works - nature implements such hiearchy within neural network and not by imposing rigorous, symbolic structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there some notions, terms, keywords, research trends, important articles (and researchers) devoted to such specialization (including the machine learning of such specialization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, my topic is very large, but the actual work on this is small ir nonexistant and that is why it is focused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is work on convolution neural networks but maybe there is another approach for language processing where the parts can be specialized in - parsing, understanding, anaphora resolution, translation, etc? And is the convolution the kind of specialization I am seeking? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the notion of attention somehow is connected with my question. But usually the attention is connected with the single neurons and not with the regions? Maybe there is notions about hierarchy of attentions - one level of attention values refers to high level tasks/skills, but another level of attention values refers to subskills, etc.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2020-01-17T08:36:44.273" LastActivityDate="2020-01-17T08:36:44.273" Title="Regional specialization in neural networks (especially for language processing)?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;artificial-neuron&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17549" PostTypeId="1" AcceptedAnswerId="17551" CreationDate="2020-01-17T09:01:14.650" Score="7" ViewCount="170" Body="&lt;p&gt;After reading a lot of articles (for instance, this one - &lt;a href=&quot;https://developers.google.com/machine-learning/gan/generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developers.google.com/machine-learning/gan/generator&lt;/a&gt;), I've been wondering: how does the generator in GAN's work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the input to the generator? What is the meaning behind &quot;input noise&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I've read, the only input that the generator receives is a random noise, which is weird.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I would like to create a similar picture of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, and put as an input a matrix of random numbers (noise) - it would take A LOT of training until I would get some sort of picture &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt;, that is similar to the source picture &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm should receive some type of reference or a basic dataset (for instance, the set of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;'s) in order to start the generation of the fake image &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32821" LastEditorUserId="2444" LastEditDate="2020-01-18T13:21:26.527" LastActivityDate="2020-12-10T09:24:27.547" Title="How does the generator in GAN's work?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17550" PostTypeId="1" CreationDate="2020-01-17T09:25:47.503" Score="4" ViewCount="131" Body="&lt;p&gt;I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have attached a graph for each of the models, which shows the upper limit quite clearly. Each dot is a prediction, and the orange graph is simply there to illustrate the ground truth (i.e. ground truth on both axis). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hxyga.png&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;RNN model predictions&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hxyga.png&quot; alt=&quot;RNN model predictions&quot; title=&quot;RNN model predictions&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/t3Gnz.png&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;GRU model predictions&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t3Gnz.png&quot; alt=&quot;GRU model predictions&quot; title=&quot;GRU model predictions&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AJV5o.png&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;LSTM model predictions&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AJV5o.png&quot; alt=&quot;LSTM model predictions&quot; title=&quot;LSTM model predictions&quot;&gt;&lt;/a&gt;&#xA;My dataset is split in 60% for training, 20% for test, and 20% for validation and then each of the splits are shuffled. The split/shuffle is the same for all three models, so each model uses the exact same split/shuffle of data for its predictions too. The models are quite simple (2 layers, nothing fancy going on). I have used grid search to find the most optimal hyperparameters for each model. Each model is fed 20 consecutive inputs (a vector of features, e.g. coordinates, waiting time, etc) and produces a single number as output which is the expected remaining waiting time. &#xA;I know this setup strongly favours LSTM and GRU over RNN, and the accuracy of the predictions definitively shows this too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However, my question is why do each model display an upper limit on its predictions? And why does it seem like such a hard limit?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot wrap my head around what the cause of this is, and so I am not able to determine whether it has anything to do with the models used, how they are trained, or if it is related to the data. Any and all help is very much appreciated!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hyperparameters for the models are:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNN: 128 units pr layer, batch size of 512, tanh activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GRU: 256 units pr layer, batch size of 512, sigmoid activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSTM: 256 units pr layer, batch size of 256, sigmoid activation function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All models have 2 layers with a dropout in between (with probability rate 0.2), use a learning rate of &lt;span class=&quot;math-container&quot;&gt;$10^{-5}$&lt;/span&gt;, and are trained over 200 epochs with early stopping with a patience of 10. All models use SGD with a momentum of 0.8 , no nesterov and 0.0 decay. Everything is implemented using Tensorflow 2.0 and Python 3.7. I am happy to share the code used for each model if relevant. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 1&lt;/strong&gt;&#xA;I should point out the graphs are made up of 463.597 individual data points, most of which are placed very near the orange line of each graph. In fact, for each of the three models, of the 463.597 data points, the number of data points within 30 seconds of the orange line is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNN: 327.206 data points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSTM: 346.601 data points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GRU: 336.399 data points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the upper limit on predictions shown on each graph consists of quite a small number of samples compared to the rest of the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;&#xA;In response to &lt;a href=&quot;https://ai.stackexchange.com/questions/17550/rnn-models-displays-upper-limit-on-predictions#comment26925_17550&quot;&gt;Sammy's comment&lt;/a&gt; I have added a graph showing the distribution of all predictions in 30 second intervals. The y-axis represents the base 10 logarithm of the number of samples which fall into a given 30 second interval (the x-axis). The first interval ([0;29]) consists of approximately 140.000 predicted values, out of the roughly 460.000 total number of predicted values. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/7VKcz.png&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Distribution of predictions in 30 second intervals for LSTM&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7VKcz.png&quot; alt=&quot;LSTM distribution of predictions&quot; title=&quot;Distribution of predictions in 30 second intervals for LSTM&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32820" LastEditorUserId="32820" LastEditDate="2020-01-20T20:09:08.677" LastActivityDate="2020-01-20T20:09:08.677" Title="RNN models displays upper limit on predictions" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;prediction&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17551" PostTypeId="2" ParentId="17549" CreationDate="2020-01-17T10:47:48.480" Score="3" Body="&lt;h3&gt;What's the input to the Generator?&lt;/h3&gt;&#xA;&lt;p&gt;In the basic implementation of GANs, the Generator &lt;strong&gt;only&lt;/strong&gt; takes in a vector of random variables. This might seem strange, but after training, the generator can transform this input noise into an image resembling those of the training set.&lt;/p&gt;&#xA;&lt;h3&gt;How does it work?&lt;/h3&gt;&#xA;&lt;p&gt;It is trained along with its counterpart the Discriminator, whose goal is to distinguish real images (i.e. the dataset's images) from fake ones (i.e. images produced by the Generator). The Generator's goal in training is to fool the Discriminator into thinking that its images are real.&lt;/p&gt;&#xA;&lt;h3&gt;Training process&lt;/h3&gt;&#xA;&lt;p&gt;In the beginning, where they are both untrained, they are both &amp;quot;terrible&amp;quot; at their respective tasks. The Generator can't produce anything resembling an image, but the Discriminator can't distinguish real from fake. As training progresses, the Discriminator starts identifying ways to distinguish the real images from the fake ones (i.e. patterns that appear in real images, but not in fake ones). The Generator, however, in its attempt to fool the Discriminator, starts producing those same patterns in its own images. After a while of both models becoming better at their respective tasks, we reach a point where the Generator can produce realistic images and the Discriminator is very good at distinguishing between real or fake.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Edit as suggested from comment:&lt;/p&gt;&#xA;&lt;p&gt;A vector of random values is used as an input, so that the Generator can learn to generate &lt;em&gt;unique&lt;/em&gt; outputs. In itself the Generator is deterministic, meaning that it has no internal sources of randomness. If we give it the same input vector twice, it will produce &lt;strong&gt;the same&lt;/strong&gt; output both times. Thus, we feed the Generator with random values, so that it can learn to produce different outputs, depending on those values.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="26652" LastEditDate="2020-12-10T09:24:27.547" LastActivityDate="2020-12-10T09:24:27.547" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17552" PostTypeId="1" CreationDate="2020-01-17T12:35:05.793" Score="2" ViewCount="34" Body="&lt;p&gt;I am looking for some advice about deep learning. (sorry for my horrible English)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about making system which can detect objects and their rotation angles &amp;amp; positions in space. For example, I need to detect a car by AI and then, using two cameras and stereoscopic methods, detect distance between cameras and the car. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 How can I use stereoscopic image? I mean, I can make depth-map (which can be used for distance detection) by already available libraries, like &lt;a href=&quot;https://docs.opencv.org/master/dd/d53/tutorial_py_depthmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. But can I teach AI model using stereoscopic image/or/and/depth map to get better results for object recognition? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 Can I detect rotation angle of the car? Can I get any information about which side of car we see, which its angles by x, y and z?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3 Can we make model which can detect objects in any environment, any weather, any lighting? For example, I have several 3d-models of cars, I can generate images with these cars in all possible rotations, but I am not sure how our model then can detect these cars in, for example, city at midnight, forest in the morning with a lot of grass&amp;amp;trees, etc;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be extremely grateful for any advice, links.&lt;/p&gt;&#xA;" OwnerUserId="32795" LastActivityDate="2020-01-17T12:35:05.793" Title="Stereoscopic vision, object rotation/distance" Tags="&lt;classification&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17553" PostTypeId="1" AcceptedAnswerId="17556" CreationDate="2020-01-17T12:38:20.907" Score="4" ViewCount="181" Body="&lt;p&gt;The selection of experimental data includes a set of vectors of different dimensions. The input is a 3-dimensional vector, and the output is a 12-dimensional vector. The sample size is 120 pairs of input 3-dimensional and output 12-dimensional vectors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to train such a neural network (in MATLAB)? Which structure of the neural network is best suited for this?&lt;/p&gt;&#xA;" OwnerUserId="32829" LastEditorUserId="2444" LastEditDate="2020-01-18T23:21:46.020" LastActivityDate="2020-01-24T05:09:13.810" Title="Is it possible to train a neural network with 3 inputs and 12 outputs?" Tags="&lt;neural-networks&gt;&lt;matlab&gt;&lt;neural-architecture-search&gt;&lt;thought-vectors&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17555" PostTypeId="2" ParentId="17543" CreationDate="2020-01-17T15:57:39.543" Score="2" Body="&lt;p&gt;It depends. It could give you a boost or it could not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively I would expect it to actually hurt performance if the network is initialized correctly (I think the optimizer is less of a bottleneck because they will have the same effect in both approaches). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ideal World:&lt;/strong&gt; We optimize the network as a whole to gain better course grained features over the sequential layers of the encoder&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reality years ago:&lt;/strong&gt; Deep nets have trouble in information propagation either forward or backwards (ex: vanishing/exploding activation and vanishing/exploding gradient).  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;solution:&lt;/strong&gt; Break up the training into iterative schemes that don't require backward information to propagate far at each optimization step&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cons:&lt;/strong&gt; Each step is looking for a greedy solution, rather than a deep one.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reality today:&lt;/strong&gt; Depending on the domain, there are various publications discussing how to circumvent this issue (ex: Residual networks, He initialization, fixup initialization, batchnorms, different activation functions, etc...)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;solution:&lt;/strong&gt; We can train deep AEs without sequential layerwise training and achieve usually better results, because the model and optimization scheme allow for deep representations to form.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I hope this  helped give some form of intuition of the matter.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2020-01-17T15:57:39.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17556" PostTypeId="2" ParentId="17553" CreationDate="2020-01-17T16:12:20.523" Score="4" Body="&lt;p&gt;There is nothing stopping you, you can setup Dense Neural Networks to have any size inputs or outputs (simple proof is to imagine a single layer NN with no activation is just a linear transform and given input dim &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and output dim &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, it's just a matrix of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, trivially this works though with any number of hidden layers)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The better question is &lt;em&gt;should you?&lt;/em&gt;. In all honesty, it depends on the data that you have, but, usually, with only 120 examples you'll either overfit completely or do relatively well if the true solution is a very simple function, but, in general, in the common situations where that isn't the case I find myself more likely or not using Bayesian approaches, so I can actually consider confidence (with little data, this is really nice)&lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="2444" LastEditDate="2020-01-18T23:25:06.743" LastActivityDate="2020-01-18T23:25:06.743" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17557" PostTypeId="2" ParentId="17547" CreationDate="2020-01-17T18:27:12.443" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v100-vs-titan-v-vs-1080-ti-benchmark/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a link&lt;/a&gt; to some benchmarks that should give you some insights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience (I've used systems with both 1080s and V100s) I've found that as of about a year ago, a lot of the common tools couldn't use the V100s well.  Until we started doing some manual optimization, the 1080s were comparable if not better on common tasks.  Of course, once we put in the time to optimize, recompile drivers, etc., the V100's really came to life and outperformed everything else by several times.&lt;/p&gt;&#xA;" OwnerUserId="30662" LastActivityDate="2020-01-17T18:27:12.443" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17558" PostTypeId="1" CreationDate="2020-01-17T21:00:48.577" Score="1" ViewCount="65" Body="&lt;p&gt;Let us assume we have a general &lt;strong&gt;AI that can improve itself&lt;/strong&gt; and is at least as intelligent as humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has wide access to technical systems including the internet, and it can communicate with humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI could become &lt;strong&gt;malicious&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can we just switch off a rouge AI?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2317" LastActivityDate="2020-01-17T21:27:00.447" Title="Can we just switch off a malicious artificial intelligence?" Tags="&lt;philosophy&gt;&lt;agi&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17559" PostTypeId="2" ParentId="17558" CreationDate="2020-01-17T21:00:48.577" Score="0" Body="&lt;p&gt;&lt;strong&gt;No.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a sufficiently advanced general AI, we can not generally assume that we can switch it off when it becomes dangerous dangerous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the electrical energy supply can always be switched off.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While that is true on the physical level, it is not guaranteed to work in practice.&lt;br&gt;&#xA;The AI could cooperate with humans, which protect the AI from deactivation, or reactivate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cooperation could, for example, one of the following forms: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The AI prepares a relation with humans offering a bribe for help.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An operator gets a reward for keeping the AI running, or reactivating it.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The AI prepares an extortion with humans &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For example by organizing a process that needs to be actively kept running by the AI to avoid the death of a child of an operator.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The AI maintains contact with a criminal organisation with strong interest to keep it running, and week ethical inhibition.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The cooperation could be with traditional organized crime organisations, providing money laundering and other vital services to a mafia organisation, which can activate it's widespread influence to maintain the AI running even when there is strong public interest in shutting it down.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that these examples can protect against physical attacks by law enforcement.&lt;/p&gt;&#xA;" OwnerUserId="2317" LastActivityDate="2020-01-17T21:00:48.577" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17561" PostTypeId="2" ParentId="17558" CreationDate="2020-01-17T21:27:00.447" Score="1" Body="&lt;p&gt;Malware viruses are a very simple form of AI.  It is not difficult to conceive of a form of malware that A) can't be detected easily, B) is redundantly distributed across thousands of computers that occasionally connect to the internet, C) is capable of detecting some kinds of threats to itself and mutating to avoid the threats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, simply &quot;turning off&quot; a malicious AI will not always be possible.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2020-01-17T21:27:00.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17562" PostTypeId="1" CreationDate="2020-01-17T22:18:28.007" Score="2" ViewCount="26" Body="&lt;p&gt;Do you think &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Defence_mechanism&quot; rel=&quot;nofollow noreferrer&quot;&gt;psychological defense system&lt;/a&gt;, for example, repression, regression, reaction, formation, isolation, undoing, projection, introjection, sublimation, etc., could be created by artificial intelligence systems? Can AI also be used to better understand the psychological defense system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, what tools do we need? Maybe supervised learning algorithms, such as &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Particle_swarm_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSO&lt;/a&gt; or &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Artificial_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANN&lt;/a&gt;, are better suited for these levels?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That doesn't seem to be that easy, and I think it needs to have a more general understanding of these algorithms. So I asked here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, what do you think is the appropriate workspace or workspace available for this job?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I think the interactions between robots that use different levels of the defense system by our selection as a game theory round play, could be good and suited to the community which is equal to the psychological connection of this part of us and as a testing environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the robots themselves are dealing with a real problem that makes it even more difficult, som according to this approach, by selecting a more nonlinear problem to solve we can count the using the levels of these defense mechanisms (for example, in simple problem 30% 1th level, 20% 2th level and ...).&lt;/p&gt;&#xA;" OwnerUserId="32824" LastEditorUserId="2444" LastEditDate="2020-01-17T23:51:24.063" LastActivityDate="2020-01-17T23:51:24.063" Title="Understanding the nature of psychological defense system by artificial intelligence" Tags="&lt;game-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17565" PostTypeId="1" AcceptedAnswerId="17570" CreationDate="2020-01-18T13:21:07.463" Score="1" ViewCount="58" Body="&lt;p&gt;I am reading about backpropagation and I wonder why I have to backpropagate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I would update the network by randomly choosing a weight to change, &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;. I would have &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. Then, I would choose &lt;span class=&quot;math-container&quot;&gt;$dw$&lt;/span&gt;, a random number from &lt;span class=&quot;math-container&quot;&gt;$-0.1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt;, for example. Then, I would do two predictions of the neural network and get their losses with the original neural network and one with &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; changed by &lt;span class=&quot;math-container&quot;&gt;$dw$&lt;/span&gt; to get the respective losses &lt;span class=&quot;math-container&quot;&gt;$L_{\text{original}}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_{\text{updated}}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$L_{\text{updated}} - L_{\text{original}}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$dL$&lt;/span&gt;. I would update &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\gamma \frac{d L}{dw}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the learning rate and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not need a gradient backpropagation throughout the system, and must have somehow a disadvantage because no one uses it. What is this disadvantage?&lt;/p&gt;&#xA;" OwnerUserId="17423" LastEditorUserId="2444" LastEditDate="2020-01-18T13:58:37.920" LastActivityDate="2020-01-18T18:11:52.880" Title="How is back-propagation useful in neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17566" PostTypeId="1" CreationDate="2020-01-18T13:41:14.617" Score="7" ViewCount="829" Body="&lt;p&gt;I was wondering if it's possible to get the inverse of a neural network. If we view a NN as a function, can we obtain its inverse?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to build a simple MNIST architecture, with the input of (784,) and output of (10,), train it to reach good accuracy, and then inverse the predicted value to try and get back the input - but the results were nowhere near what I started with. (I used the pseudo-inverse for the W matrix.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My NN is basically the following function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x) = \theta(xW + b), \;\;\;\;\; \theta(z) = \frac{1}{1+e^{-z}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rev_sigmoid(y):&#xA;    return np.log(y/(1-y))&#xA;&#xA;def rev_linear(z, W, b):&#xA;    return (z - b) @ np.linalg.pinv(W)&#xA;&#xA;y = model.predict(x_train[0:1])&#xA;z = rev_sigmoid(y)&#xA;x = rev_linear(z, W, b)&#xA;x = x.reshape(28, 28)&#xA;plt.imshow(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qBZiS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qBZiS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;^ This should have been a 5:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8LI8n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8LI8n.png&quot; alt=&quot;t&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a reason why it failed? And is it ever possible to get inverse of NN's? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: it is also worth noting that doing the opposite does yield good results. I.e. starting with the y's (a 1-hot encoding of the digits) and using it to predict the image (an array of 784 bytes) using the same architecture: input (10,) and output (784,) with a sigmoid. This is not exactly equivalent, to an inverse as here you first do the linear transformation and then the non-linear. While in an inverse you would first do (well, undo) the non-linear, and then do (undo) the linear. I.e. &lt;strong&gt;the claim that the 784x10 matrix is collapsing too much information seems a bit odd to me, as there does exist a 10x784 matrix that can reproduce enough of that information&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SAP02.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SAP02.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27947" LastEditorUserId="27947" LastEditDate="2020-01-23T12:32:22.513" LastActivityDate="2021-03-16T17:16:32.390" Title="Can we get the inverse of the function that a neural network represents?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;math&gt;" AnswerCount="3" CommentCount="8" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17568" PostTypeId="1" AcceptedAnswerId="17572" CreationDate="2020-01-18T17:35:40.710" Score="4" ViewCount="116" Body="&lt;p&gt;What are the differences between TensorFlow and PyTorch, both in terms of performance and functionality?&lt;/p&gt;&#xA;" OwnerUserId="32763" LastEditorUserId="2444" LastEditDate="2020-01-20T10:50:20.840" LastActivityDate="2020-01-20T10:50:20.840" Title="What are the differences between TensorFlow and PyTorch?" Tags="&lt;tensorflow&gt;&lt;comparison&gt;&lt;software-evaluation&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-05-18T12:33:59.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="17569" PostTypeId="1" CreationDate="2020-01-18T17:40:25.367" Score="0" ViewCount="39" Body="&lt;p&gt;I'm looking to develop a stand-alone real-time outdoor imaging CNN application, and I can't wrap my head around the myriad of camera options and their communication protocols.&#xA;The target is a Linux embedded device (such as beaglebone + edge TPU). and the top language is python. (though if it's unavoidable, I can write some c++ driver)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done a number of projects with webcams with &lt;code&gt;OpenCV&lt;/code&gt;. However, this project requires a more serious camera, with a motorized zoom and autofocus functions,&#xA;with at least 3MP and 30fps. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There are numerous Chinese IP Camera blocks with zoom functions.&#xA;but they stream H.265, and I'm not sure how &lt;code&gt;OpenCV&lt;/code&gt; would interface with that.&#xA;additionally, it's not clear how to control the zoom feature via python in one of them.&lt;/li&gt;&#xA;&lt;li&gt;Then there are USB2 camera blocks, they seem to be of lower but sufficient quality, but I couldn't find one with motorized zoom.&lt;/li&gt;&#xA;&lt;li&gt;Then there are &lt;code&gt;Gige&lt;/code&gt; and &lt;code&gt;USB3 Vision&lt;/code&gt; cameras, they seem optimal but are prohibitively expensive (500$+). &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is kind of an open-ended question, but I couldn't find much information online on this topic, so I hope to find wisdom here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I was wondering if someone has advice or recommendations on this.&lt;/p&gt;&#xA;" OwnerUserId="32858" LastActivityDate="2021-02-11T23:04:25.427" Title="Best Camera and protocol for embedded real time CNN project" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;hardware&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17570" PostTypeId="2" ParentId="17565" CreationDate="2020-01-18T18:11:52.880" Score="3" Body="&lt;p&gt;The method you propose is already known, its basically a numerical approximation to the gradient. It is not used to train neural networks because its well... an approximation. You still need to do two forward passes to get an approximation, which introduces noise and might make the training process fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using backpropagation to compute the gradient is an exact solution, so why would you use an approximation if the exact computation is equally efficient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numeric approximations of the gradient only make sense if exact computation is not possible.&lt;/p&gt;&#xA;" OwnerUserId="31632" LastActivityDate="2020-01-18T18:11:52.880" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17571" PostTypeId="1" CreationDate="2020-01-18T18:41:49.440" Score="2" ViewCount="31" Body="&lt;p&gt;This post continues the topic in the following post: &#xA; &lt;a href=&quot;https://ai.stackexchange.com/q/17553/2444&quot;&gt;Is it possible to train a neural network with 3 inputs and 12 outputs?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I conducted several experiments in MATLAB and selected those neural networks that best approximate the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a list of them:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Cascade-forward backpropagation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Elman backpropagation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generalized regression&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Radial basis (exact fit)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I did not notice a fundamental difference in quality, except for Elman's backpropagation, which had a higher error than the rest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to justify the choice of the structure of the neural network in this case?&lt;/p&gt;&#xA;" OwnerUserId="32829" LastEditorUserId="2444" LastEditDate="2020-01-18T23:20:14.733" LastActivityDate="2020-01-18T23:20:14.733" Title="How do I determine the best neural network architecture for a problem with 3 inputs and 12 outputs?" Tags="&lt;recurrent-neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;neural-architecture-search&gt;&lt;thought-vectors&gt;&lt;recursive-neural-network&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17572" PostTypeId="2" ParentId="17568" CreationDate="2020-01-18T19:45:22.460" Score="3" Body="&lt;p&gt;TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. Both provide machine learning libraries to accomplish various tasks and do the job. TensorFlow is a powerful and deep learning tool with active visualization and debugging capabilities. TensorFlow also offers serialization benefits since the entire graphic is saved as a protocol buffer. It also has support for mobile platforms and offers a production-ready implementation. PyTorch, on the other hand, is still gaining momentum and attracting Python developers, since it is more Python friendly. In summary, TensorFlow is used to speed things up and create AI-related products, while research-oriented developers prefer PyTorch.&lt;/p&gt;&#xA;" OwnerUserId="32861" LastActivityDate="2020-01-18T19:45:22.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17573" PostTypeId="1" CreationDate="2020-01-18T20:48:55.197" Score="2" ViewCount="33" Body="&lt;p&gt;I have 5000 degraded pictures ( pixelated, blurry, too much luminosity ... ) and their clean versions, and I would like to train a model so that it can predict how to correct future pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to get as much information as possible, and I think I'll have to train a deep learning model using Tensorflow &amp;amp; Keras, and I'll have to do image inpainting, more precisely a deconvolution method, to get to the expected result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have strictly no idea about how to get there. I would load both pictures using PIL Image, then convert it into a numpy array, but then I don't really know what to do next.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have any code to show, however I can provide you with a couple pictures. Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="32862" LastActivityDate="2020-01-18T20:48:55.197" Title="How can I train a Deep Learning model using degraded photos and their clean version to correct photos" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17574" PostTypeId="2" ParentId="17569" CreationDate="2020-01-18T21:20:41.247" Score="0" Body="&lt;p&gt;The Raspberry Pi ZeroW with V2 camera doesn't quite have the specs you need, but it is close: &lt;a href=&quot;https://picamera.readthedocs.io/en/latest/fov.html#sensor-modes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://picamera.readthedocs.io/en/latest/fov.html#sensor-modes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a waterproof enclosure for it: &lt;a href=&quot;https://www.youtube.com/watch?v=5CWksss_5lQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=5CWksss_5lQ&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32863" LastActivityDate="2020-01-18T21:20:41.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17575" PostTypeId="2" ParentId="17566" CreationDate="2020-01-18T23:41:56.140" Score="2" Body="&lt;p&gt;&lt;strong&gt;Mathematical Exploration&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;let &lt;span class=&quot;math-container&quot;&gt;$\Theta^+$&lt;/span&gt; be the pseudo-inverse of &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall, that if a vector &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol v \in R(\Theta)$&lt;/span&gt; (ie in the row space) then &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol v = \Theta^+\Theta\boldsymbol v$&lt;/span&gt;. That is, so long as we select a vector that is in the rowspace of &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt; then we can reconstruct it with full fidelity using the pseudo inverse. Thus, if any of the images happen to be linear combinations of the rows of &lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt; then we can reconstruct it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific. let &lt;span class=&quot;math-container&quot;&gt;$f(\boldsymbol x)$&lt;/span&gt; have a pseudo-inverse &lt;span class=&quot;math-container&quot;&gt;$f^+(\boldsymbol x)$&lt;/span&gt; defined as you have. If we &lt;em&gt;restrict&lt;/em&gt; our domain such that &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol x \in C(\Theta^T)$&lt;/span&gt; (column space of the transpose) then &lt;span class=&quot;math-container&quot;&gt;$f^+= f^{-1}_{res}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, under our domain restriction the pseudo inverse becomes a true inverse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Extrapolation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would then seem that so long as we are under such domain restrictions then we could define a pseudo inverse for a general NN. Though, it might be possible that some NNs don't have any restriction that admits an inverse. Maybe, there is some way to regularize the parameters such that this is possible. NNs with ReLU wouldn't admit such an inverse since ReLU loses information on negative values. Leaky Relu might work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further Investigation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, this presents a zone for further study. Some questions to answer might be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it possible for optimized parameters to contain non-trivial examples in their row-space?&lt;/li&gt;&#xA;&lt;li&gt;If so, under what conditions is this possible?&lt;/li&gt;&#xA;&lt;li&gt;Are the examples in any way represented in the row space?&lt;/li&gt;&#xA;&lt;li&gt;Is there some way to regularize a NN such that it admits an inverse over some desired restriction?&lt;/li&gt;&#xA;&lt;li&gt;Under what conditions is invertibility useful?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2020-01-19T00:43:25.290" LastActivityDate="2020-01-19T00:43:25.290" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17576" PostTypeId="1" CreationDate="2020-01-19T00:25:42.203" Score="3" ViewCount="260" Body="&lt;p&gt;I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I couldn't use one hot encoding because it's not efficient so I went with the other option which is embedding of fixed length. I've just used neural nets to compute embedding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is can we use advanced NLP models like bert to extract embeddings for categorical data from my corpus? Is it possible? I've only asked it because I've only heard that bert is good for sentence embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="32867" LastActivityDate="2020-01-19T00:25:42.203" Title="Can Bert be used to extract embedding for large categorical features?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17577" PostTypeId="1" AcceptedAnswerId="17578" CreationDate="2020-01-19T00:56:46.310" Score="3" ViewCount="1826" Body="&lt;p&gt;In my code, I usually use the mean squared error (MSE), but the TensorFlow tutorials always use the categorical cross-entropy (CCE). Is the CCE loss function better than MSE? Or is it better only in certain cases?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-01-20T18:31:38.170" LastActivityDate="2021-03-11T08:40:38.120" Title="In which cases is the categorical cross-entropy better than the mean squared error?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;loss-functions&gt;&lt;mean-squared-error&gt;&lt;categorical-crossentropy&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17578" PostTypeId="2" ParentId="17577" CreationDate="2020-01-19T01:45:39.747" Score="6" Body="&lt;p&gt;As a rule of thumb, &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;noreferrer&quot;&gt;mean squared error (MSE)&lt;/a&gt; is more appropriate for &lt;a href=&quot;https://en.wikipedia.org/wiki/Regression_analysis&quot; rel=&quot;noreferrer&quot;&gt;regression&lt;/a&gt; problems, that is, problems where the output is a numerical value (i.e. a floating-point number or, in general, a real number).  However, in principle, &lt;a href=&quot;https://stats.stackexchange.com/q/46413/82135&quot;&gt;you can use the MSE for classification problems&lt;/a&gt; too (even though that may not be a good idea). MSE can be preceded by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;noreferrer&quot;&gt;sigmoid function&lt;/a&gt;, which outputs a number &lt;span class=&quot;math-container&quot;&gt;$p \in [0, 1]$&lt;/span&gt;, which can be interpreted as the probability of the input belonging to one of the classes, so the probability of the input belonging to the other class is &lt;span class=&quot;math-container&quot;&gt;$1 - p$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;noreferrer&quot;&gt;cross-entropy (CE)&lt;/a&gt; is mainly used for &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_classification&quot; rel=&quot;noreferrer&quot;&gt;classification&lt;/a&gt; problems, that is, problems where the output can belong to one of a discrete set of classes. The CE loss function is usually separately implemented for &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_classification&quot; rel=&quot;noreferrer&quot;&gt;binary&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiclass_classification&quot; rel=&quot;noreferrer&quot;&gt;multi-class&lt;/a&gt; classification problems. In the first case, it is called the binary cross-entropy (BCE), and, in the second case, it is called categorical cross-entropy (CCE). The CE requires its inputs to be distributions, so the CCE is usually preceded by a &lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;noreferrer&quot;&gt;softmax function&lt;/a&gt; (so that the resulting vector represents a probability distribution), while the BCE is usually preceded by a sigmoid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://stats.stackexchange.com/q/288451/82135&quot;&gt;Why is mean squared error the cross-entropy between the empirical distribution and a Gaussian model?&lt;/a&gt; for more details about the relationship between the MSE and the cross-entropy. In case you use TensorFlow (TF) or Keras, see also &lt;a href=&quot;https://stackoverflow.com/q/47034888/3924118&quot;&gt;How to choose cross-entropy loss in TensorFlow?&lt;/a&gt;, which gives you some guidelines for how to choose the appropriate TF implementation of the cross-entropy function for your (classification) problem. See also &lt;a href=&quot;https://stats.stackexchange.com/q/260505/82135&quot;&gt;Should I use a categorical cross-entropy or binary cross-entropy loss for binary predictions?&lt;/a&gt; and &lt;a href=&quot;https://stats.stackexchange.com/q/223256/82135&quot;&gt;Does the cross-entropy cost make sense in the context of regression?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-19T03:00:56.557" LastActivityDate="2020-01-19T03:00:56.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17579" PostTypeId="2" ParentId="17506" CreationDate="2020-01-19T10:58:08.537" Score="0" Body="&lt;h2&gt;The idea behind it&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'll do an analogy: while &lt;em&gt;classical GAs&lt;/em&gt; look like how humanity reproduced until now, &quot;&lt;em&gt;intelligent crossover&lt;/em&gt;&quot; is more looking like &lt;strong&gt;designer babies&lt;/strong&gt;. You first would need to identify which genes are responsible for certain behaviors and then you can bring them on to the new generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Said this, you will understand why there aren't so many algorithms around: it is a very &lt;strong&gt;case-tailored approach&lt;/strong&gt;. Possibly each problem requires a different method and certainly will increase considerably the complexity of the crossover algorithms.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;How to make your own&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you want to create one of these algorithms, you might want to add a step to your GA cycle: &lt;em&gt;Evaluation, &lt;strong&gt;Identification&lt;/strong&gt;, Selection, Crossover, Mutation, Replacement&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the Identification step, you can run a probabilistic search on your population to discover which genes might be responsible for higher fitness results. Then you can:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;proceed normally: Select two parents based on their fitness and perform crossover giving more importance to the &quot;good&quot; genes. This is the &quot;safest&quot; way.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;fuse &lt;em&gt;Selection&lt;/em&gt; and &lt;em&gt;Crossover&lt;/em&gt;: use the data retrieved during the &lt;em&gt;Identification&lt;/em&gt; step to create good individuals directly from the population pool. This system is dangerous as it might reduce the population diversity at alarming rates, leading to stagnation. This is how the &lt;em&gt;Fitness-based Crossover&lt;/em&gt; are generally built &lt;a href=&quot;https://www.semanticscholar.org/paper/A-fitness-based-scanning-multi-parent-crossover-a-Auwatanamongkol/0097a816c3302ef145763c7bde3e447fb41124aa&quot; rel=&quot;nofollow noreferrer&quot;&gt; [paper]&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2020-01-19T10:58:08.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17580" PostTypeId="1" CreationDate="2020-01-19T11:40:31.470" Score="1" ViewCount="46" Body="&lt;p&gt;How is clustering used in the unsupervised training of a neural network? Can you provide an example?&lt;/p&gt;&#xA;" OwnerUserId="32798" LastEditorUserId="2444" LastEditDate="2020-01-19T13:46:10.890" LastActivityDate="2020-01-19T13:46:10.890" Title="How is clustering used in the unsupervised training of a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17581" PostTypeId="2" ParentId="14215" CreationDate="2020-01-19T13:47:33.570" Score="0" Body="&lt;p&gt;I found a solution to this some time back. I have been studying function approximation (within linear regression) for some time. &#xA;Here's how I did it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks have been proved to be universal function approximators&lt;/a&gt;. So, even a single hidden layer would be sufficient to approximate a function simple as &lt;strong&gt;addition&lt;/strong&gt; (Even somewhat complex functions like the &lt;strong&gt;&lt;a href=&quot;https://www.codeproject.com/Articles/1237026/Simple-MLP-Backpropagation-Artificial-Neural-Netwo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sine&lt;/a&gt;&lt;/strong&gt; and &lt;strong&gt;any random CONTINUOUS &lt;a href=&quot;https://colab.research.google.com/drive/1K9agqd9Pn8llY1uwkFD68YqFgPeSTBCQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiggly function&lt;/a&gt;&lt;/strong&gt; have been approximated) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I used a high level API like TensorFlow and Keras and implemented it &lt;a href=&quot;https://colab.research.google.com/drive/1s8VYf5tYGj83SxOtJQbiKs7AJd_eNQX5&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model was trained on the data (input-output pairs)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R    = np.array([-4, -10,  -2,  8, 5, 22,  3],  dtype=float)&#xA;B    = np.array([4, -10,  0,  0, 15, 5,  1],  dtype=float)&#xA;G    = np.array([0, 10,  5,  8, 1, 2,  38],  dtype=float)&#xA;&#xA;Y    = np.array([0, -10, 3, 16, 21, 29, 42],  dtype=float)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And trained as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Create a hidden layer with 2 neurons&#xA;hidden = tf.keras.layers.Dense(units=2, input_shape=[3])&#xA;&#xA;#Create the output (final) layer which symbolises value of **Y**&#xA;output = tf.keras.layers.Dense(units=1)&#xA;&#xA;#Combine layers to form the neural network and compile it&#xA;model = tf.keras.Sequential([hidden, output])&#xA;model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))&#xA;history = model.fit(RBG,Y, epochs=500, verbose=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The model converges in about 50 epochs &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/97HZy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/97HZy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I have implemented the same using only C/C++ and used GNU plot to visualize the results. &lt;/p&gt;&#xA;" OwnerUserId="26452" LastActivityDate="2020-01-19T13:47:33.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17582" PostTypeId="1" CreationDate="2020-01-19T17:04:47.423" Score="2" ViewCount="23" Body="&lt;p&gt;I have seen the teef glove ofNavid Azodi and Thomas Pryor, like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pX7BI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pX7BI.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;and also seen &lt;a href=&quot;https://amp.theatlantic.com/amp/article/545441/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post &lt;/a&gt;which has been said about this kind of work problem :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Their six-page letter, which Padden passed along to the dean, points&#xA;  out how the SignAloud gloves—and all the sign-language translation&#xA;  gloves invented so far—misconstrue the nature of ASL (and other sign&#xA;  languages) by focusing on what the hands do. Key parts of the grammar&#xA;  of ASL include “raised or lowered eyebrows, a shift in the orientation&#xA;  of the signer’s torso, or a movement of the mouth,” reads the letter.&#xA;  “Even perfectly functioning gloves would not have access to facial&#xA;  expressions.” ASL consists of thousands of signs presented in&#xA;   sophisticated ways that have, so far, confounded reliable machine&#xA;  recognition. One challenge for machines is the complexity of ASL and&#xA;  other sign languages. Signs don’t appear like clearly delineated beads&#xA;  on a string; they bleed into one another in a process that linguists&#xA;  call “coarticulation” (where, for instance, a hand shape in one sign&#xA;  anticipates the shape or location of the following sign; this happens&#xA;  in words in spoken languages, too, where sounds can take on&#xA;  characteristics of adjacent ones). Another problem is the lack of&#xA;  large data sets of people signing that can be used to train&#xA;  machine-learning algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So i like to know what proper AI module do you know for doing the navid works better by adding for first step the geometrical position analyzing into it, i like to use popular AI blocks like &lt;a href=&quot;https://github.com/tensorflow/tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorfliw&lt;/a&gt; for doing this kind of analyzing by fast online and the modules which updated at the time by large community users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, some Virtual reality analyzer for the position analyzing must be existing, so which one if popular and free to contributing with large community?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks for your attention.&lt;/p&gt;&#xA;" OwnerUserId="32824" LastActivityDate="2020-01-19T17:04:47.423" Title="Searching for powerfull AI modules to improve teef gloves" Tags="&lt;real-time&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17583" PostTypeId="1" CreationDate="2020-01-20T01:33:36.197" Score="1" ViewCount="29" Body="&lt;p&gt;I am trying to create a model that can clean pictures of noise, blur, high luminosity etc, but I do not know how to do that. I have tried to search for it a lot, and I couldn't find anything that could properly teach me, as I am very new in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to do something using &lt;a href=&quot;https://colab.research.google.com/drive/1_g7YqerlN9FI8ffxt0pnv4MB86fNqtoe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Colaboratory&lt;/a&gt; so that people can look it up and help me a bit: &lt;a href=&quot;https://colab.research.google.com/drive/1_g7YqerlN9FI8ffxt0pnv4MB86fNqtoe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Cleaning Modedl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, I download a file containing 5000+ clean pictures and their degraded version. Then, I load a training set of 500 pictures in two &lt;code&gt;ndarray&lt;/code&gt; datasets called &lt;code&gt;degraded_dataset&lt;/code&gt; and &lt;code&gt;clean_dataset&lt;/code&gt; of &lt;code&gt;shape=(500, 576, 720, 3)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I do not know what to do with these two datasets, I do not know how to train a model. I vaguely have an idea, which is giving the &lt;code&gt;clean_dataset&lt;/code&gt; to the model as the &lt;code&gt;label_dataset&lt;/code&gt;, however I am not sure it is the good way of doing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed these tutorials: &lt;a href=&quot;https://www.tensorflow.org/tutorials/images/cnn#create_the_convolutional_base&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow CNN&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://www.kaggle.com/lgmoneda/from-image-files-to-numpy-arrays&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle &quot;From images to narray&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for helping me.&lt;/p&gt;&#xA;" OwnerUserId="32862" LastActivityDate="2020-01-20T01:33:36.197" Title="How to use Before / After images to train a model" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;datasets&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17584" PostTypeId="1" AcceptedAnswerId="17590" CreationDate="2020-01-20T03:20:28.917" Score="3" ViewCount="1059" Body="&lt;p&gt;I am learning to use tensorflow.js. I am also using the tfvis library to print information about the neural net to the web browser. When I create a create a dense neural net with a layer with 5 neurons and another layer with 2 neurons, each layer has a bias vector of length 5 and 2 respectively. I checked the docs (&lt;a href=&quot;https://js.tensorflow.org/api/0.6.1/#layers.dense&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://js.tensorflow.org/api/0.6.1/#layers.dense&lt;/a&gt;), and it says that there is indeed a bias vector for each dense layer. Isn't a vector redundant? Doesn't each layer only need a single number for the bias? See the code below:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//Create tensorflow neural net&#xA;this.model = tf.sequential();&#xA;&#xA;this.model.add(tf.layers.dense({units: 5, inputShape: [1]}))&#xA;this.model.add(tf.layers.dense({units: 2}))&#xA;&#xA;const surface = { name: 'Layer Summary', tab: 'Model Inspection'};&#xA;tfvis.show.layer(surface, this.model.getLayer(undefined, 0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32887" LastEditorUserId="2444" LastEditDate="2020-01-20T13:55:50.823" LastActivityDate="2020-01-20T18:17:53.593" Title="Why does the bias need to be a vector in a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;javascript&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17585" PostTypeId="1" CreationDate="2020-01-20T04:27:54.457" Score="2" ViewCount="39" Body="&lt;p&gt;I am reading a research paper on the formulation of MDP problems to ICU treatment decision making: &lt;a href=&quot;https://aisel.aisnet.org/icis2018/healthcare/Presentations/9/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Treatment Recommendation in Critical Care: A Scalable and Interpretable Approach in Partially Observable Health States&lt;/a&gt;. The paper applies a Monte Carlo approach to approximate the value function. Below is a screenshot of the excerpt that I came across. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZVpz6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZVpz6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last sentence of the excerpt reads &quot;The approach is scalable for growing number of states variables and action variables&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does it mean when the author says that the Monte Carlo &lt;em&gt;approach is scalable for a growing number of states variables and action variables&lt;/em&gt;? Wouldn't the amount of data needed to approximate the value function increase with the higher dimensionality of states? Or does the Monte Carlo approach scale better in time complexity as compared to traditional Q-learning methods?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-01-20T18:34:55.950" LastActivityDate="2020-01-20T18:34:55.950" Title="Why is this Monte Carlo approach scalable for a growing number of states variables and action variables?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;papers&gt;&lt;monte-carlo-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17587" PostTypeId="1" CreationDate="2020-01-20T10:13:03.327" Score="1" ViewCount="185" Body="&lt;p&gt;I would much appreciate if you could point me in the right direction regarding this question about &lt;code&gt;targets&lt;/code&gt; for approximate &lt;code&gt;q-function&lt;/code&gt; for SARSA, Expected SARSA, Q-learning (notation: &lt;code&gt;S&lt;/code&gt; is the current state, &lt;code&gt;A&lt;/code&gt; is the current action, &lt;code&gt;R&lt;/code&gt; is the reward, &lt;code&gt;S’&lt;/code&gt; is the next state and &lt;code&gt;A’&lt;/code&gt; is the action chosen from that next state). I've written my thoughts next to each question/statement:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the Q-learning target computation require the probability of current policy to select the &lt;code&gt;A'&lt;/code&gt; (action that was actually made in the environment) in &lt;code&gt;S'&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;I'm not sure what probability of policy means? Target in Q-learning doesn't depend on the action taken (&lt;code&gt;A'&lt;/code&gt;). But, behavior policy does have some randomness (&lt;code&gt;epsilon&lt;/code&gt; - probability of taking the random action) which determines which action will be taken in environment (random or action that maximizes Q-value). Does the question refer to that 'probability of current policy'? But generally, I think this is not correct since I don't see anything similar to probability in Q-function update (except that &lt;code&gt;epsilon&lt;/code&gt;).&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do we need an explicit policy for the Q-learning target to sample &lt;code&gt;A’&lt;/code&gt; from. And for SARSA? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;I guess this is true for Q-learning since we need to get max Q-value which determines which action &lt;code&gt;A'&lt;/code&gt; we'll use for update. For SARSA we update the &lt;code&gt;Q(S, A)&lt;/code&gt; depending on which action was actually taken (no need for max)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this statement true: &lt;code&gt;All methods (SARSA, Ex. SARSA, Q-learning) require R and S’ to perform updates.&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;All methods require S, A, R, S'&gt; In the statement, only a subset of required parameters is mentioned. Does it make it true or not since the FULL set of parameters is left out?&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is the difference between SARSA and Q-learning targets only in how &lt;code&gt;A’&lt;/code&gt; in &lt;code&gt;S’&lt;/code&gt; is selected?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;I would say that this is not correct but I'm not entirely sure. Based on some code I've seen on github, both of them select the next action in the exactly same way, but they differ in how they update parameters (SARSA updates parameters based on action actually taken in environment and Q-learning on best possible action regardless of which action was actually taken).&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32893" LastEditorUserId="32893" LastEditDate="2020-01-21T07:16:54.893" LastActivityDate="2020-01-21T07:16:54.893" Title="Expected SARSA, SARSA and Q-learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;sarsa&gt;&lt;expected-sarsa&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="17588" PostTypeId="1" AcceptedAnswerId="17708" CreationDate="2020-01-20T14:27:50.900" Score="2" ViewCount="104" Body="&lt;p&gt;I'm interested in starting a project that will identify the face of a Lego mini figure from a digital photo. My goal is to eventually map the expression of a person's face to the Lego mini figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have any experience working with image recognition technology (my technical experience is mainly in web technology), and I am looking for recommended platforms or resources that I could get started with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most helpful would be recommendations for image recognition technologies (Python would be great!) that I could start to experiment with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I'm aware of SparkAR as a library designed to for Instagram camera effects specifically, and even though I'm not interested in Instagram, I wonder if there are comparable libraries/studios/products for working with image recognition development.&lt;/p&gt;&#xA;" OwnerUserId="32902" LastEditorUserId="32902" LastEditDate="2020-01-29T22:38:59.020" LastActivityDate="2020-01-29T22:38:59.020" Title="Lego minifigure facial recognition: where to start?" Tags="&lt;machine-learning&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17590" PostTypeId="2" ParentId="17584" CreationDate="2020-01-20T15:27:32.770" Score="2" Body="&lt;p&gt;In a simple feed-forward network, each artificial neuron has a separate bias value. This allows for greater flexibility for the output layer function than if each neuron had to use a single whole-layer bias. Although not an absolute requirement, without this arrangement it may become very hard to approximate some functions. Moving from a bias vector to a single scalar bias value per layer will most of the time reduce the effectiveness of a neural network through lost flexibility in how it fits to the target function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; output neurons in a layer leading to needing &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; values for bias, then it is fairly straightforward to model this collection of bias values as a vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often you will see a neural network layer function written in this form or similar:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{y} = f(\mathbf{W}\mathbf{x} + \mathbf{b})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$f()$&lt;/span&gt; is the activation function (applied element-wise), &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; the weights matrix for the layer and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; is the bias. When written in this form, it is easy to see that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}\mathbf{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; must all be vectors of the same size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This layer design has become so standard that it is possible to forget that other designs and implementations are possible for neural network parameters, and can sometimes be useful. Frameworks like TensorFlow also make it easier to take the standard approach, which is why you need a vector for bias on the example you are using. Whilst you are learning, and probably 99% of the time after that, it will be best to go with what the framework is doing here. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-01-20T15:27:32.770" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17591" PostTypeId="1" CreationDate="2020-01-20T16:04:38.960" Score="1" ViewCount="25" Body="&lt;p&gt;I am reading the R-CNN paper by Ross Girshick1 &lt;em&gt;et al.&lt;/em&gt; (&lt;a href=&quot;https://arxiv.org/abs/1311.2524&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) and I fail to understand how they do the inference. This is described in the section &lt;em&gt;2.2.Test-time Detection&lt;/em&gt; in the paper. I quote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At test time, we run selective search on the test image to extract around 2000 region proposals (we use selective search’s “fast mode” in all experiments). We warp each proposal and forward propagate it through the CNN in order to read off features from the desired layer. &lt;strong&gt;Then, for each class, we score each extracted feature vector using the SVM trained for that class&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I do not understand how a Support Vector Machine (SVM) can &lt;em&gt;score&lt;/em&gt; a feature vector since SVM does not tell you class probability, it only tells you if an object belongs to a class or not. How is this possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that detection flow is: get image, run it through CNN and get feature vector, &lt;em&gt;score&lt;/em&gt; this feature vector and run Non-Maximal Suppresion (NMS). But for running NMS we need the feature vector scored, and again, SVM do not score predictions, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, when represented in the same paper, the SVM does not provide a &lt;em&gt;score&lt;/em&gt; as you can see in the next image (taken from the same paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dvBpn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dvBpn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how this makes sense?&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-01-20T16:04:38.960" Title="Scoring feature vector with Support Vector Machine" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;papers&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17592" PostTypeId="2" ParentId="17584" CreationDate="2020-01-20T18:11:15.633" Score="2" Body="&lt;p&gt;To emphasize (and this is not emphasized in &lt;a href=&quot;https://ai.stackexchange.com/a/17590/2444&quot;&gt;this answer&lt;/a&gt;), in the case of neural networks, the biases or, more precisely, the &lt;a href=&quot;https://stackoverflow.com/a/48243204/3924118&quot;&gt;connections (or weights) between biases and other neurons are also &lt;em&gt;learnable&lt;/em&gt; parameters&lt;/a&gt;, so the back-propagation algorithm calculates a gradient of the loss function that contains the partial derivatives with respect to the connections between the biases and other neurons too and, in the gradient descent step, these connections can also be updated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each neuron usually has its own bias. For example, in Keras, this is the case, as &lt;a href=&quot;https://stackoverflow.com/a/59828935/3924118&quot;&gt;you can easily verify&lt;/a&gt;. However, in principle, you could also have a layer with a single scalar bias that is shared across all neurons of that layer, but this would probably have a different effect. The role of the bias is discussed in several places on the web. For example, in &lt;a href=&quot;https://stackoverflow.com/q/2480650/3924118&quot;&gt;this Stack Overflow post&lt;/a&gt; or in &lt;a href=&quot;https://stats.stackexchange.com/q/185911/82135&quot;&gt;this Stats SE post&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-20T18:17:53.593" LastActivityDate="2020-01-20T18:17:53.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17593" PostTypeId="2" ParentId="16915" CreationDate="2020-01-20T20:37:02.303" Score="0" Body="&lt;p&gt;Setting aside the dubious nature of subliminal messages. You have to clearly define what subliminal is. In all your cases presented, a manual algorithmic approach would be better. An ANN would tend to average out the subliminal messages as noise, just as your brain would. In the case of #2, #3, it's a matter of contrast, in which case computer vision methods of bringing out low contrast details to the forefront combined with a manual algorithm (perhaps an OCR pass for content not expected to have text) would be the way to go. ANN are not magic bullets, and still need to be used appropriately. &lt;/p&gt;&#xA;" OwnerUserId="32908" LastActivityDate="2020-01-20T20:37:02.303" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="17596" PostTypeId="1" CreationDate="2020-01-21T09:08:25.273" Score="4" ViewCount="197" Body="&lt;p&gt;I am working on a problem in which I need to train a neural network to map one or more input images to one or more output images (1 channel for image). Below I report some examples of input&amp;amp;output. In this case I report 1 input and 1 output image, but may need to pass to more inputs and outputs, maybe by encoding this in channels. However, the images are all of this kind, maybe rotated, tralated or changed a bit in shape. (fyi, they are fields defined by fluid dynamics simulations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about CNN, but the standard architecture used for image classification (convolutional layers + fully connected layers) seems not to be the best choice. Instead, I tried using the U-net architecture, composed of compression+decompression convolutional layers. This works quite fine, but maybe there is some other architecture that could be more suited to my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BE9Ox.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BE9Ox.png&quot; alt=&quot;input image&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/EwXbV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EwXbV.png&quot; alt=&quot;output image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32915" LastEditorUserId="2444" LastEditDate="2020-01-25T15:28:52.197" LastActivityDate="2020-01-26T03:15:52.043" Title="Which deep learning models are suitable for image-to-image mapping?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17598" PostTypeId="2" ParentId="7940" CreationDate="2020-01-21T13:36:00.203" Score="2" Body="&lt;p&gt;Assume you are the snake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In front of you is empty. Left of you is empty. Right of you is empty. The distance to the apple is 4. The apple straight in front of you. Your length is 20.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you make a good decision with this input? In which direction would you go to achieve maximum score?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the given input, you could go straight forward to the apple. But that might be a failure and lead to death.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMHO, the input state is simply not enough to make a good decision, because &lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) the snake doesn't even know in which direction it's currently moving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) the snake has no idea about where its body is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation could look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AZFmT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AZFmT.png&quot; alt=&quot;Situation for the snake&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way for the snake to move out of this trap is as indicated by the arrow, so that the tail frees the way out just in time. Your neural network does not have the necessary input to make that decision.&lt;/p&gt;&#xA;" OwnerUserId="31627" LastActivityDate="2020-01-21T13:36:00.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17599" PostTypeId="1" AcceptedAnswerId="17673" CreationDate="2020-01-21T15:46:32.647" Score="2" ViewCount="116" Body="&lt;p&gt;I wonder, if there are other than NEAT approaches to evolving architectures and weights of artificial neural networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific: I am looking for projects/frameworks/libraries that use evolutionary/genetic algorithms to simultanousely evolve both topology and train weights of ANNs other than NEAT approach. By 'other' I mean similar to NEAT but not based entirely on NEAT. I hope to find different approaches to the same problem.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2020-01-22T18:36:15.260" LastActivityDate="2020-01-25T15:44:54.827" Title="What are evolutionary algorithms for topology and weights evolving of ANN (TWEANN) other than NEAT?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17601" PostTypeId="1" CreationDate="2020-01-21T18:28:15.360" Score="1" ViewCount="19" Body="&lt;p&gt;I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate algorithms and find the threshold for each. Then comparing the threshold of each. In other words, if the accelerometer algorithm predicts a fall (fall detected = 1) and the EEG algorithm detects a fall, based on the power spectrum (fall detected = 1), then the system outputs a &quot;1&quot; that a fall was truly detected. This approach uses the idea of a simple AND gate between the two algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know how to correctly process the data so that I can feed both types of data into a CNN. Any advice is really appreciated, even a lead to some literature, articles or information would be great.&lt;/p&gt;&#xA;" OwnerUserId="32806" LastActivityDate="2020-01-21T18:28:15.360" Title="EEG and Accelerometer Neural Network" Tags="&lt;neural-networks&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;&lt;computational-linguistics&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17602" PostTypeId="1" CreationDate="2020-01-21T19:53:40.830" Score="2" ViewCount="49" Body="&lt;p&gt;The &lt;em&gt;inverse propensity score&lt;/em&gt; (IPS) estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained in the paper &lt;a href=&quot;https://arxiv.org/pdf/1503.02834.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Doubly Robust Policy Evaluation and Optimization&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The old policy &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, or the behavior policy, is okay to be non-stationary in the IPS estimator even if the new policy &lt;span class=&quot;math-container&quot;&gt;$\nu$&lt;/span&gt;, or the target policy, should be stationary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this true for the &lt;em&gt;importance sampling&lt;/em&gt; (IS) estimator, which seems to be a variant of IPS, for off-policy evaluation in a reinforcement learning problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IS estimator is explained in this paper &lt;a href=&quot;https://arxiv.org/pdf/1511.03722.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Doubly Robust Off-policy Value Evaluation for Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The target policy should be stationary, but can the old policy be non-stationary in the IS estimator?&lt;/p&gt;&#xA;" OwnerUserId="30051" LastEditorUserId="2444" LastEditDate="2020-01-24T11:58:26.977" LastActivityDate="2020-01-24T11:58:26.977" Title="Can the importance sampling estimator have a non-stationary behaviour policy even if the target policy is stationary?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policies&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17603" PostTypeId="1" CreationDate="2020-01-21T20:39:34.190" Score="1" ViewCount="2342" Body="&lt;p&gt;Could someone explain to me which is the key difference between the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy and the softmax policy? In particular, in the contest of SARSA and Q-Learning algorithms. I understood the main difference between these two algorithms, but I didn't understand all the combinations between algorithm and policy&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SARSA + &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy&lt;/li&gt;&#xA;&lt;li&gt;SARSA + Softmax&lt;/li&gt;&#xA;&lt;li&gt;Q-Learning + &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy&lt;/li&gt;&#xA;&lt;li&gt;Q-Learning + Softmax&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2020-12-04T17:42:19.690" LastActivityDate="2020-12-04T17:42:19.690" Title="What is the difference between the $\epsilon$-greedy and softmax policies?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;sarsa&gt;&lt;epsilon-greedy-policy&gt;&lt;softmax-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17605" PostTypeId="1" AcceptedAnswerId="17607" CreationDate="2020-01-21T22:17:18.830" Score="6" ViewCount="687" Body="&lt;p&gt;I'd like to better understand temporal-difference learning. In particular, I'm wondering if it is prudent to think about TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) as a type of &quot;truncated&quot; Monte Carlo learning?&lt;/p&gt;&#xA;" OwnerUserId="32929" LastEditorUserId="2444" LastEditDate="2020-06-04T16:57:55.163" LastActivityDate="2021-03-14T23:17:34.447" Title="What is the intuition behind TD($\lambda$)?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;td-lambda&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17606" PostTypeId="1" CreationDate="2020-01-22T00:26:10.550" Score="1" ViewCount="42" Body="&lt;p&gt;If I have a lot of input output pairs as training data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;lt;float Xi, float Yi&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I have a parametrized approximation function (I know the function algorithm, but not the values of the many many parameters it contains) which shall approximate the process by which the original data pairs were generated. The function takes two input values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// c is a precomputed classifier for x and can have values from 0 to 255, so there can be up to 256 different classes&#xA;y = f(float x, int c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the hidden parameters of the function are some big lookup tables (a lot of free parameters, but still much fewer than the amount of data points in the training data)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to fit all the hidden parameters that f contains AND compute for each &lt;code&gt;Xi&lt;/code&gt; a &lt;code&gt;ci&lt;/code&gt;, such that for the fitted function the error over all i of &lt;code&gt;Yi - f(Xi, ci)&lt;/code&gt; is minimized&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, using some algorithm I want to fit the parameters of f and also classify the inputs Xi so that f(Xi, ci) aproximates Yi&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this kind of problem called and what kind of algorithm is used to solve it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume it's possible to initialize all hidden parameters as well as all ci with random values and then somehow use back propagation of the error to iteratively find  parameters and ci such that the function works well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't know is whether this is a well known class of problem and I just don't know the name of it, so I'm asking for pointers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe in other words:&#xA;I have a function that has a certain layout (for performance reasons) which I want to use to approximate and interpolate my training data, I want to tune the parameters of this function such that it approximates the original data well. since the data points fall into some 'categories', I want to pre-classify the data-points to make it easier for the function to do its job. What kind of algorithm do I use to find the function's parameters and to pre-classify the input?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastEditorUserId="32111" LastEditDate="2020-01-22T06:51:13.143" LastActivityDate="2020-01-22T08:08:29.310" Title="What class of problem is this?" Tags="&lt;classification&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17607" PostTypeId="2" ParentId="17605" CreationDate="2020-01-22T03:01:45.903" Score="3" Body="&lt;p&gt;TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) can be thought of as a combination of TD and MC learning, so as to avoid to choose one method or the other and to take advantage of both approaches.&lt;/p&gt;&#xA;&lt;p&gt;More precisely, TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) is temporal-difference learning with a &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return, which is defined as an average of all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns, for all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, where an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step return is the &lt;em&gt;target&lt;/em&gt; used to update the estimate of the value function that contains &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; future rewards (plus an estimate of the value function of the state &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps in the future). For example, TD(0) (e.g. Q-learning is usually presented as a TD(0) method) uses a &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;-step return, that is, it uses one future reward (plus an estimate of the value of the next state) to compute the target. The letter &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; actually refers to a parameter used in this context to weigh the combination of TD and MC methods. There are actually two different perspectives of TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;), the &lt;em&gt;forward view&lt;/em&gt; and the &lt;em&gt;backward view&lt;/em&gt; (eligibility traces).&lt;/p&gt;&#xA;&lt;p&gt;The blog post &lt;a href=&quot;https://amreis.github.io/ml/reinf-learn/2017/11/02/reinforcement-learning-eligibility-traces.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: Eligibility Traces and TD(lambda)&lt;/a&gt; gives a quite intuitive overview of TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;), and, for more details, read &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=309&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related chapter of the book &lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-03-10T17:45:20.583" LastActivityDate="2021-03-10T17:45:20.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17608" PostTypeId="1" CreationDate="2020-01-22T03:50:02.353" Score="1" ViewCount="1833" Body="&lt;p&gt;I am trying to understand how deep Q learning (DQN) works. To my current understanding, each &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; functions is estimated to be a function of a feature vector of its state &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;(s) and the weight of the network &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The loss function to minimise is &lt;span class=&quot;math-container&quot;&gt;$||\delta_{t+1}||^2$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\delta_{t+1}$&lt;/span&gt; is shown below. The loss function is from the website talking about function approximation. Even though it is not explicitly deep Q learning, the loss function to minimise is similar.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\delta_{\mathrm{t}+1}=\mathrm{R}_{\mathrm{t}+1}+\max _{\mathrm{a}\in\mathrm{A}} \boldsymbol{\theta}^{\top} \Phi\left(\mathrm{s}_{t+1}, \mathrm{a}\right)-\boldsymbol{\theta}^{\top} \Phi\left(\mathrm{s}_{\mathrm{t}}, \mathrm{a}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://towardsdatascience.com/function-approximation-in-reinforcement-learning-85a4864d566&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/function-approximation-in-reinforcement-learning-85a4864d566&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, I am not able to understand why the loss function is defined as such. Once the network converges to a &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; using gradient descent, does that mean that the &lt;span class=&quot;math-container&quot;&gt;$Q_{max}(s,a)$&lt;/span&gt; is found?&lt;/p&gt;&#xA;&lt;p&gt;In essence, I am not able to grasp intuitively how the neural network is able to generalise the learning to unseen states.&lt;/p&gt;&#xA;&lt;p&gt;The algorithm I am looking at to help me understand the deep Q networks is below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DSsAU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DSsAU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-06-21T11:06:17.363" LastActivityDate="2021-03-18T12:05:30.640" Title="Understanding the loss function in deep Q-learning" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;temporal-difference-methods&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17609" PostTypeId="1" CreationDate="2020-01-22T04:40:47.477" Score="5" ViewCount="712" Body="&lt;p&gt;In deep learning, is it possible to use discontinuous activation functions (e.g. one with jump discontinuity)?&lt;/p&gt;&#xA;&lt;p&gt;(My guess: for example, ReLU is non-differentiable at a single point, but it still has a well-defined derivative. If an activation function has a jump discontinuity, then its derivative is supposed to have a delta function at that point. However, the backpropagation process is incapable of considering that delta function, so the convex optimization process will have some problem?)&lt;/p&gt;&#xA;" OwnerUserId="32933" LastEditorUserId="2444" LastEditDate="2021-01-28T00:24:48.470" LastActivityDate="2021-01-28T00:24:48.470" Title="In deep learning, is it possible to use discontinuous activation functions?" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17610" PostTypeId="1" CreationDate="2020-01-22T05:12:14.857" Score="2" ViewCount="17" Body="&lt;p&gt;I have a pool of knowledge that I want to mine for information and allow an AI to deduce likely conclusions from this information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to give the AI a set of textual data that is rated on a scale of 0 to 100 ranging from false (0) to unequivocally true (100).  Based on ongoing learning I want to be able to ask it about it's data and to make relational conclusions as to not simply whether things are true or false, but to extrapolate likelihoods, conclusions and so forth... or to simply tell me it can't understand something which would then trigger me to give it more information and to train it with additional material - even if it's my own limited answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately I'll deal with image data as well, but that's a bit down the road.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I'm new to the area of neural nets and deep learning and so I'm hoping someone could point me in the right direction in the way of terminology to search for / research as well as where perhaps I should start.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't mind working in C predominantly if possible, but other languages (especially Ruby) is fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The field is moving so fast and there's so much research now that seems to trump information available from just a couple years ago now and so I'm hoping to jump into information that takes advantage of more general learning algorithms so that this can be as robust possible while taking care of current trends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do I go from here?&lt;/p&gt;&#xA;" OwnerUserId="32936" LastActivityDate="2020-01-22T05:12:14.857" Title="Getting started with creating a general AI based on textual and then image based data?" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17611" PostTypeId="1" CreationDate="2020-01-22T05:42:03.320" Score="1" ViewCount="19" Body="&lt;p&gt;I am trying to find unique (distinct) faces in multiple videos files. What is the best way to do that?&lt;/p&gt;&#xA;" OwnerUserId="32937" LastActivityDate="2020-01-22T05:42:03.320" Title="Finding unique faces in a video" Tags="&lt;object-detection&gt;&lt;facial-recognition&gt;&lt;clustering&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17612" PostTypeId="1" CreationDate="2020-01-22T07:52:41.587" Score="1" ViewCount="40" Body="&lt;p&gt;Following the &lt;a href=&quot;https://www.youtube.com/playlist?list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&quot; rel=&quot;nofollow noreferrer&quot;&gt;online courses with Andrew Ng&lt;/a&gt;, he talks about L2 regularization (a.k.a. weight decay) and input normalization.  Now, the argument is that L2 regularization make the weights smaller, which makes the sigmoid activation functions (and thus the whole network) &quot;more&quot; linear.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;: can this rather handwavey explanation be formalized?  Can we define &quot;more linear&quot; in a mathematical sense, and demonstrate that smaller weights in fact achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;: in contrast to sigmoid, ReLU activations have a single point where it is nonlinear - i.e. the breaking point at x=0.  No scaling of the input changes the shape (i.e. derivative) of this function, the only effect is reducing the magnitude of positive outputs.  Does the argument still hold?  Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input normalization is given as a good practice, but it seems to me that the network should just compensate for varying magnitude between components of the input by scaling the weights appropriately.  The only exception I can think of is again under L2 regularization, which would penalize large weights (assoiciated with small inputs).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 3&lt;/strong&gt;: Is this correct, and is input scaling thus mostly important with L2 normalization, or is there some reason why the network would fail to adjust the weights without scaling?&lt;/p&gt;&#xA;" OwnerUserId="29720" LastActivityDate="2020-01-22T07:52:41.587" Title="Do L2 regularization and input normalization depend on sigmoid activation functions?" Tags="&lt;regularization&gt;&lt;sigmoid&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17613" PostTypeId="1" AcceptedAnswerId="17627" CreationDate="2020-01-22T07:56:49.603" Score="2" ViewCount="48" Body="&lt;p&gt;I would much appreciate if you could point me in the right direction regarding this question about targets for SARSA and Q-learning (notation: &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the current state, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the current action, &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the reward, &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; is the next state and &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt; is the action chosen from that next state).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do we need an explicit policy for the Q-learning target to sample &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt; from? And for SARSA?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess this is true for Q-learning since we need to get max Q-value which determines which action &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt; we'll use for the update. For SARSA, we update the &lt;span class=&quot;math-container&quot;&gt;$Q(S, A)$&lt;/span&gt; depending on which action was actually taken (no need for max). Please correct me if I'm wrong.&lt;/p&gt;&#xA;" OwnerUserId="32893" LastEditorUserId="2444" LastEditDate="2020-01-22T12:02:46.940" LastActivityDate="2020-01-22T23:07:23.183" Title="Do we need an explicit policy to sample $A'$ in order to compute the target in SARSA or Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17614" PostTypeId="2" ParentId="17606" CreationDate="2020-01-22T08:08:29.310" Score="1" Body="&lt;p&gt;This problem is typically called parameter estimation or inverse modelling, and there are a variety of techniques to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your free parameters are all continuous (i.e. none are discrete, such as integers), and your model function is differentiable, then you can turn the model into a computation graph in e.g. TensorFlow and use gradient descent methods with MSE loss, learning parameters on the data in much the same way as a neural networks. Most of the toolkits built for neural networks can do this, you just have to ignore the pre-packaged layer models and the utilities to manage them. You will need to take care with parameter initialisation, ideally you already have some rough starting values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use other approaches, such as search methods too. Even when you have some discrete-valued parameters then you can use genetic algorithms for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best method to use will depend on details of your model, it is not possible to point out a generic &quot;best&quot;. If you can use a gradient-based method directly, as you suggest in the question, then that might be the most efficient in terms of computation, provided you have some way to set reasonable initial parameters.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-01-22T08:08:29.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17615" PostTypeId="1" CreationDate="2020-01-22T08:44:53.483" Score="2" ViewCount="39" Body="&lt;p&gt;I plan to develop OCR application using tensorflow to get the value from the image. Text in the image may handwritting or text printed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fdTlu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fdTlu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the image, my ocr appplication will able to get the value of below:&#xA;1. ChequeDate&#xA;2. Payee Name&#xA;3. Legal Amount&#xA;4. Courtesy Amount&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How the OCR application can get the value i want as highlighted as red color? is it need to crop it as small size of the picture for the OCR?&lt;/p&gt;&#xA;" OwnerUserId="32727" LastEditorUserId="32727" LastEditDate="2020-01-23T03:19:54.183" LastActivityDate="2020-01-23T03:19:54.183" Title="OCR - Text recognition from Image" Tags="&lt;machine-learning&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17616" PostTypeId="1" CreationDate="2020-01-22T12:34:23.323" Score="2" ViewCount="46" Body="&lt;p&gt;I have a detection problem. An object with a probability of 0.5 is in a box with coordinates ((0,0), (2, 2)) and with a probability of 0.5 a box with coordinates ((2,0), (4,2)). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the maximum expected value of an &lt;a href=&quot;https://en.wikipedia.org/wiki/Jaccard_index&quot; rel=&quot;nofollow noreferrer&quot;&gt;intersection over union (IOU)&lt;/a&gt; with an object that a constant detection algorithm that produces one box can reach? I can't understand how I should find the expected value here. P.S. IOU is 0 cos intersection is empty.&lt;/p&gt;&#xA;" OwnerUserId="32952" LastEditorUserId="2444" LastEditDate="2020-01-24T11:32:15.730" LastActivityDate="2020-02-19T13:49:01.217" Title="What is the expected value of an IOU in this case?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17617" PostTypeId="1" CreationDate="2020-01-22T14:01:31.193" Score="1" ViewCount="16" Body="&lt;p&gt;I have 2 (independently long) sequences (a and b) of feature vectors that I want to use as input for a neural network.&#xA;The idea was to build 2 GRU based encoders (one for each sequence).&#xA;I would than have a decoder (that takes the 2 encoded states) to produce the final output (a number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My training data would be something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;([a11, a11, a12], [b11, b12, b13]) =&amp;gt; v1&#xA;([a21, a21], [b21, b22, b23, b24]) =&amp;gt; v2&#xA;([a31], [b31, b32]) =&amp;gt; v3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where axy (and bxy) are feature vectors. vx is the output value (a float) of the neural network.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    # lenA maximum length of a sequence&#xA;    # lenB maximum length of b sequence&#xA;    # featuresA number of features for a&#xA;    # featuresB number of features for b&#xA;&#xA;    enc1InputData=np.zeros((count, lenA, featuresA))&#xA;    enc2InputData=np.zeros((count, lenB, featuresB))&#xA;    decoderInputData=np.zeros((count, 1, 1))&#xA;    decoderOutputData=np.zeros((count, 1))&#xA;&#xA;    # Initialize input data&#xA;    ... &#xA;&#xA;    enc1Input = Input(shape=(None, featuresA))&#xA;    enc1 = GRU(256, return_state=True)&#xA;    enc1Output, enc1State = enc1(enc1Input)&#xA;&#xA;    enc2Input = Input(shape=(None, featuresB))&#xA;    enc2 = GRU(256, return_state=True)&#xA;    enc2Output, enc2State = enc2(enc2Input)&#xA;&#xA;    encOutput=concatenate([enc1State, enc2State])&#xA;&#xA;    decoderInput = Input(shape=(None, 1))&#xA;    decoderGru = GRU(256+256, return_sequences=True, return_state=True)&#xA;    decoderOutput, _ = decoderGru(decoderInput, initial_state=encOutput)&#xA;&#xA;    for i in range(3):&#xA;        decoderDense=Dense(parms.dim3, activation=&quot;relu&quot;)&#xA;        decoderOutput = decoderDense(decoderOutput)&#xA;&#xA;    #flatten = Flatten()&#xA;    #decoderOutput=flatten(decoderOutput)&#xA;&#xA;    decoderTanh = Dense(1, activation=&quot;tanh&quot;)&#xA;    decoderOutput = decoderTanh(decoderOutput)&#xA;&#xA;    model=Model([enc1Input, enc2Input, decoderInput], decoderOutput)&#xA;    model.compile(optimizer=&quot;adam&quot;, loss=&quot;mse&quot;)&#xA;&#xA;    model.fit([enc1InputData, enc2InputData, decoderInputData], decoderOutputData, batch_size=256, epochs=100, validation_split=0.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives me a&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Error when checking target: expected dense_4 to have 3 dimensions, but got array with shape (2, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Including the Flatten step gives a &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: The shape of the input to &quot;Flatten&quot; is not fully defined (got (None, 256)). Make sure to pass a complete &quot;input_shape&quot; or &quot;batch_input_shape&quot; argument to the first layer in your model.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32948" LastActivityDate="2020-01-22T14:01:31.193" Title="How do I build a multi RNN network with keras?" Tags="&lt;keras&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17619" PostTypeId="1" AcceptedAnswerId="17621" CreationDate="2020-01-22T14:51:52.473" Score="1" ViewCount="46" Body="&lt;p&gt;I am coding out a simple 4x4 grid game whereby the agent starts at a particular state and his aim is to reach the terminal state. The agent is supposed to avoid traps along the way and reach the end goal with high reward. The below picture illustrates the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NITn0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NITn0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code that I am running is shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# 4x4 Grid&#xA;import random&#xA;&#xA;&#xA;gamma = 1&#xA;grid = [[-0.1 for i in range(4)] for j in range(4)]&#xA;episodes = 500000&#xA;epsilon = 1 # start greedy&#xA;decay = 0.999&#xA;min_epsilon = 0.1&#xA;alpha = 0.65&#xA;# set terminal states&#xA;grid[1][0] = -1&#xA;grid[2][2] = -1&#xA;grid[0][3] = 1&#xA;&#xA;# Set up Q tables&#xA;# 0: up, 1: down, 2: left, 3: right&#xA;# Q = {(0,0): {0: z, 1: x, 2: c, 3: v}, ... }}&#xA;Q = {}&#xA;&#xA;# 4 rows&#xA;for row in range(4):&#xA;    # 4 columns&#xA;    for column in range(4):&#xA;        Q[(row,column)] = {}&#xA;        # 4 actions&#xA;        for k in range(4):&#xA;            Q[(row,column)][k] = 0&#xA;&#xA;&#xA;def isTerminal(state):&#xA;    if state == (1,0) or state == (2,2) or state == (0,3):&#xA;        return True&#xA;    return False&#xA;&#xA;def get_next_state_reward(state, action):&#xA;&#xA;    row = state[0]&#xA;    col = state[1]&#xA;    #print(row, col)&#xA;    if action == 0: # up&#xA;        # out of grid&#xA;        if (row - 1) &amp;lt; 0:&#xA;            return (state, grid[row][col])&#xA;&#xA;    if action == 1: # down&#xA;&#xA;        # out of grid&#xA;        if (row + 1) &amp;gt; len(grid) - 1:&#xA;            return (state, grid[row][col])&#xA;&#xA;    if action == 2: # left&#xA;&#xA;        if (col - 1 &amp;lt; 0):&#xA;            return (state, grid[row][col])&#xA;&#xA;    if action == 3: # right&#xA;&#xA;        if (col + 1 &amp;gt; len(grid[row]) - 1):&#xA;            return (state, grid[row][col])&#xA;&#xA;    if action == 0:&#xA;&#xA;        row -= 1&#xA;        return ((row,col), grid[row][col])&#xA;&#xA;    if action == 1:&#xA;&#xA;        row += 1&#xA;        return ((row,col), grid[row][col])&#xA;&#xA;    if action == 2:&#xA;&#xA;        col -= 1&#xA;        return ((row,col), grid[row][col])&#xA;&#xA;    if action == 3:&#xA;&#xA;        col += 1&#xA;        return ((row,col), grid[row][col])&#xA;&#xA;state_visit = {}&#xA;for row in range(4):&#xA;    # 4 columns&#xA;    for column in range(4):&#xA;        state_visit[(row,column)] = 0&#xA;&#xA;for episode in range(episodes):&#xA;&#xA;    # let agent start at start state&#xA;    state = (3,0)&#xA;&#xA;    while not isTerminal(state):&#xA;&#xA;        r = random.uniform(0,1)&#xA;&#xA;        if r &amp;lt; epsilon:&#xA;            action = random.randint(0,3)&#xA;        else:&#xA;            action = max(Q[state], key=lambda key: Q[state][key])&#xA;&#xA;&#xA;        next_state, reward = get_next_state_reward(state, action)&#xA;&#xA;        TD_error = reward + gamma * max(Q[next_state]) - Q[state][action]&#xA;&#xA;        Q[state][action] = Q[state][action] + alpha * TD_error&#xA;&#xA;        state = next_state&#xA;&#xA;        state_visit[next_state] += 1&#xA;        epsilon = max(min_epsilon, epsilon*decay)&#xA;&#xA;        #input()&#xA;&#xA;&#xA;policy = {}&#xA;# get optimal policies for each state&#xA;for states in Q:&#xA;    policy[states] = max(Q[states], key=lambda key: Q[states][key])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I finish running the algorithm however, I am unable to achieve the optimal policy no matter how many tweaks I do to the number of episodes, or epsilon decay, or the alpha value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particularly, the Q values that I attain for state (2,0), (0,1) and (0,0) have Q values that are equal values for three directions except for the last direction which brings the agent to the terminal state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, these are the Q-values that I get for state (0,0), (0,1) and (2,0) respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0,0): {0: 2.0, 1: 2.9, 2: 2.9, 3: 2.9}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0,1): {0: 2.9, 1: 2.0, 2: 2.9, 3: 2.9}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2,0): {0: 2.9, 1: 2.9, 2: 2.9, 3: 2.9}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure why the Q-values for the 3 directions should be the same because each extra step that the agent takes incurs a negative reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone be able to help ? Thank you so much !&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-01-22T15:29:48.123" Title="Q-learning problem wrong policy" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17620" PostTypeId="1" CreationDate="2020-01-22T15:11:10.873" Score="1" ViewCount="26" Body="&lt;p&gt;In the media there's lot of talk about &lt;strong&gt;face&lt;/strong&gt; recognition, mainly with respect to identifying faces (= assigning to persons). Less attention is paid to the recognition of facially expressed &lt;strong&gt;emotions&lt;/strong&gt; but there's a lot of research done into this direction, too. Even less attention is paid to the recognition of facially expressed emotions of a single person (which could be much more detailed) - even though this would be a very interesting topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What holds for faces does similarily hold for &lt;strong&gt;voices&lt;/strong&gt;. With the help of artifical intelligence voices can be identified (= assigned to persons) and emotions as expressed by voice can be recognized - on a general and on an individual's level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My general question goes into another direction: As huge progress has been made in &lt;strong&gt;visual&lt;/strong&gt; scene analysis (&quot;what is &lt;strong&gt;seen&lt;/strong&gt; in this scene?&quot;) there has probably been some progress made in &lt;strong&gt;auditory&lt;/strong&gt; scene analysis: &quot;What is &lt;strong&gt;heard&lt;/strong&gt; in this scene?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My specific question is: Are there test cases and results where some AI software was given some auditory data with a lot of &quot;voices&quot; and could tell &lt;strong&gt;how many voices there were&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a rather easy specific test case consider some &lt;a href=&quot;https://en.wikipedia.org/wiki/Gregorian_chant&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gregorian chant&lt;/a&gt; sung in perfect unison. (See also &lt;a href=&quot;https://music.stackexchange.com/questions/94640/the-discriminating-power-of-experienced-listeners&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="25362" LastEditorUserId="25362" LastEditDate="2020-01-22T15:18:03.180" LastActivityDate="2020-01-22T15:18:03.180" Title="State of the art in voice recognition" Tags="&lt;computer-vision&gt;&lt;emotional-intelligence&gt;&lt;facial-recognition&gt;&lt;voice-recognition&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17621" PostTypeId="2" ParentId="17619" CreationDate="2020-01-22T15:29:48.123" Score="1" Body="&lt;p&gt;You have a simple mistake in your TD Error function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TD_error = reward + gamma * max(Q[next_state]) - Q[state][action]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You have made &lt;code&gt;Q[next_state]&lt;/code&gt; a Python &lt;code&gt;dict&lt;/code&gt;, so this will take the maximum &lt;em&gt;key&lt;/em&gt; which is 3 for all your Q table entries. This is why you end up with values very close to 3 at the end, which is impossible for your problem, the maximum return will be &lt;code&gt;1.0&lt;/code&gt; when stepping from adjacent grid points to the positive terminal state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TD_error = reward + gamma * max(Q[next_state].values()) - Q[state][action]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, since your actions are all numeric, you could use a &lt;code&gt;list&lt;/code&gt; instead of a &lt;code&gt;dict&lt;/code&gt; in your Q table.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-01-22T15:29:48.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17622" PostTypeId="2" ParentId="17608" CreationDate="2020-01-22T20:37:44.047" Score="0" Body="&lt;p&gt;Well, you want your network to have a good prediction powers for the Q-values. So you compare Q-value at time t with the reward that you've got at time t after having executed action a + the prediction of the best Q-value of your neural network at time t+1. Note, that you are optimizing using a prediction and not a true value. That is called bootstrapping, look up TD-learning to have a better grasp of the concept.&lt;/p&gt;&#xA;" OwnerUserId="2254" LastActivityDate="2020-01-22T20:37:44.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17623" PostTypeId="2" ParentId="17609" CreationDate="2020-01-22T20:41:37.620" Score="1" Body="&lt;p&gt;I would say that it is possible, but probably not a very good idea.?Like you say, the hard requirement is that the network (and thus its components, including the activation functions) must be differentiable.  ReLU isn't, but you can cheat by defining f'(0) to be 0 (or 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A continuous function means that gradient descent leads to some local minimum¹, for piecewise continuous functions, it may not converge (i.e. the breakpoints themselves may not be part of the segment you descend, so you will never get get to an actual minimum).  This is not likely to be a problem in practice, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;¹ At least for functions that are bounded from below, like cost functions are.&lt;/p&gt;&#xA;" OwnerUserId="29720" LastActivityDate="2020-01-22T20:41:37.620" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17624" PostTypeId="1" CreationDate="2020-01-22T21:10:17.997" Score="2" ViewCount="143" Body="&lt;p&gt;Can you please elucidate the math behind the update rule for the critic? I've seen in other places that just a squared distance of  &lt;span class=&quot;math-container&quot;&gt;$R + \hat{v}(S', w) - \hat{v}(S,w)$&lt;/span&gt; is used, but Sutton suggests an update rule (and the math behind) that is beyond my understanding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h5t1V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h5t1V.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, why do we need &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2254" LastEditorUserId="2254" LastEditDate="2020-01-28T21:13:26.567" LastActivityDate="2020-01-28T21:13:26.567" Title="How does the update rule for the one-step actor-critic method work?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="11" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17626" PostTypeId="2" ParentId="17603" CreationDate="2020-01-22T22:24:11.983" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node17.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy&lt;/strong&gt;&lt;/a&gt; policy is a policy that chooses the best action (i.e. the action associated with the highest value) with probability &lt;span class=&quot;math-container&quot;&gt;$1-\epsilon \in [0, 1]$&lt;/span&gt; and a random action with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon $&lt;/span&gt;. The problem with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy is that, when it chooses the random actions (i.e. with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;), it chooses them &lt;a href=&quot;https://en.wikipedia.org/wiki/Discrete_uniform_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;uniformly&lt;/a&gt; (i.e. it considers all actions equally good), even though certain actions (even excluding the currently best one) are better than others. Of course, this approach is not ideal in the case certain actions are extremely worse than others. Therefore, a natural solution to this problem is to select the random actions with probabilities proportional to their current values. These policies are called &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node17.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;softmax&lt;/strong&gt; policies&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Q-learning&lt;/strong&gt;&lt;/a&gt; is an off-policy algorithm, which means that, while learning a so-called &lt;em&gt;target policy&lt;/em&gt;, it uses a so-called &lt;em&gt;behaviour policy&lt;/em&gt; to select actions. The behaviour policy can either be an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, a softmax policy or any other policy that can sufficiently explore the environment while learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The figure below shows the pseudocode of the Q-learning algorithm. In this case, the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy is actually derived from the current estimate of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function. The target policy, in this context, is represented by the &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}$&lt;/span&gt; operator, which is used to select the highest &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value of the future state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, which is the state the RL agent ends up in after having taken the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; selected by the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy behaviour policy, with respect to another action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. This may sound complicated, but if you read the pseudocode several times, you will understand that there are two different actions (and states). The target policy (i.e. the policy that the RL agent wants to learn) is represented by the &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}$&lt;/span&gt; operator in the sense that the so-called &lt;em&gt;target&lt;/em&gt; of the Q-learning update step, i.e. &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \operatorname{max}_{a'} Q(s', a')$&lt;/span&gt;, assumes that the &lt;strong&gt;greedy&lt;/strong&gt; action is taken from the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. For this reason, Q-learning is said to learn the greedy policy (as a target policy), while using an exploratory policy, usually, the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, but it can also be the softmax. Note that, in both cases, the policies are derived from the current estimate of the Q function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IHr3A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IHr3A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, &lt;a href=&quot;https://en.wikipedia.org/wiki/State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;SARSA&lt;/strong&gt;&lt;/a&gt; is often considered an on-policy algorithm, given that there aren't necessarily two distinct policies, i.e. the target policy is not necessarily different than the behaviour policy, like in Q-learning (where the target policy is the greedy policy and the behaviour policy is e.g. the softmax policy derived from the current estimate of the Q function). This can more easily be seen from the pseudocode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DPT4V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DPT4V.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, no &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}$&lt;/span&gt; operator is used and the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy is mentioned twice: in the first case, it is used to choose the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and indirectly &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, and, in the second case, to select the action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. In Q-learning, &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; is the action that corresponds to the highest Q value from &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; (i.e. the greedy action). Clearly, you are free to choose a different policy than the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy (in both cases), but this will possibly have a different effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, to understand the difference between Q-learning and SARSA and the places where the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy or softmax policies can be used, it is better to look at the pseudocode.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-22T22:29:13.530" LastActivityDate="2020-01-22T22:29:13.530" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17627" PostTypeId="2" ParentId="17613" CreationDate="2020-01-22T23:00:38.780" Score="3" Body="&lt;p&gt;Q-learning uses an exploratory policy, derived from the current estimate of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, such as the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy, to select the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. After having taken this action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, the reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; are observed. At this point, to update the estimate of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, you use a &lt;em&gt;target&lt;/em&gt; that assumes that the greedy action is taken from the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. The greedy action is selected by the &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}$&lt;/span&gt; operator, which can thus be thought of as an &lt;em&gt;implicit&lt;/em&gt; policy (but this terminology isn't common, AFAIK), so, in this context, the greedy action is the action associated with the highest &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value for the state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SARSA, no &lt;span class=&quot;math-container&quot;&gt;$\operatorname{max}$&lt;/span&gt; operator is used, and you derive a policy (e.g. the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy) from the current estimate of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function to select both &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; (from &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, in all cases, the policies are implicit, in the sense that they are derived from the estimate of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, but this isn't a common terminology. See also &lt;a href=&quot;https://ai.stackexchange.com/a/17626/2444&quot;&gt;this answer&lt;/a&gt;, where I describe more in detail the differences between Q-learning and SARSA, and I also show the pseudocode of both algorithms, which you should read (multiple times) in order to fully understand their differences.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-22T23:07:23.183" LastActivityDate="2020-01-22T23:07:23.183" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17629" PostTypeId="1" AcceptedAnswerId="17633" CreationDate="2020-01-23T06:37:03.773" Score="2" ViewCount="67" Body="&lt;p&gt;During weeks and months of your work, many things may change, for example :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You may modify the loss function&lt;/li&gt;&#xA;&lt;li&gt;Your training or validation datasets may change&lt;/li&gt;&#xA;&lt;li&gt;You modify data augmentation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which tools or processes do you use to track modifications you have made and how did they affected the model ?&lt;/p&gt;&#xA;" OwnerUserId="23912" LastActivityDate="2020-04-01T19:24:08.923" Title="How to track performance of your model during experimenting?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2020-11-08T19:38:31.823" ContentLicense="CC BY-SA 4.0" />
  <row Id="17630" PostTypeId="1" CreationDate="2020-01-23T07:50:41.453" Score="2" ViewCount="403" Body="&lt;p&gt;I'm reading the book &lt;a href=&quot;https://www.packtpub.com/big-data-and-business-intelligence/hands-meta-learning-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hands-On Meta Learning with Python&lt;/a&gt;, and in Prototypical networks said:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, we use episodic training—for each episode, we randomly sample a&#xA;  few data points from each class in our dataset and we call that a&#xA;  support set and train the network using only the support set, instead&#xA;  of the whole dataset.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think, but I'm not sure, I have understood what is an &quot;episodic training&quot; but, what is the meaning of &quot;episodic&quot; or &quot;episode&quot; here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sorry, I'm not English and I can't guess what it is meaning searching on a dictionary. I know what an episode is, but I don't know what is an episode on artificial intelligence training.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="2444" LastEditDate="2020-01-24T11:35:18.760" LastActivityDate="2020-01-24T11:35:18.760" Title="What does &quot;episodic training&quot; mean?" Tags="&lt;terminology&gt;&lt;theory&gt;&lt;meta-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17631" PostTypeId="2" ParentId="5174" CreationDate="2020-01-23T09:03:18.780" Score="0" Body="&lt;p&gt;You can try &lt;strong&gt;&lt;em&gt;move ordering&lt;/em&gt;&lt;/strong&gt; where we store the values till depth d, sort them and use them in particular order before we go for depth d+1 ...&lt;/p&gt;&#xA;" OwnerUserId="1935" LastActivityDate="2020-01-23T09:03:18.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17633" PostTypeId="2" ParentId="17629" CreationDate="2020-01-23T11:29:06.300" Score="3" Body="&lt;p&gt;Maybe you are looking for a combination of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Version_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;version control system&lt;/a&gt; (like &lt;a href=&quot;https://git-scm.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;git&lt;/a&gt; and &lt;a href=&quot;https://github.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;) and a tool like &lt;a href=&quot;https://www.comet.ml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;comet.ml&lt;/a&gt;. In the past, I used comet.ml to keep track of different experiments performed with different hyper-parameters or different versions of the code. There are other alternatives to comet.ml, such as &lt;a href=&quot;https://github.com/IDSIA/sacred&quot; rel=&quot;nofollow noreferrer&quot;&gt;sacred&lt;/a&gt;, but they may also have different features and may not be as visually pleasing as comet.ml or even free. Personally, I liked comet.ml (even though, at the time, it still lacked some features). In any case, a VCS, like git, is widely used in software development (not just in AI projects) to keep track of different versions of the code, etc. You may also be interested in continuous integration (e.g. &lt;a href=&quot;https://travis-ci.com/plans&quot; rel=&quot;nofollow noreferrer&quot;&gt;Travis CI&lt;/a&gt;) and code review (e.g. &lt;a href=&quot;https://app.codacy.com/login&quot; rel=&quot;nofollow noreferrer&quot;&gt;codacy&lt;/a&gt;) tools.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-01-23T11:29:06.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17634" PostTypeId="1" CreationDate="2020-01-23T11:40:16.597" Score="5" ViewCount="75" Body="&lt;p&gt;Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already come up with the following two methods in my literature review:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Principal Component Analysis (PCA) + Whitening&lt;/li&gt;&#xA;&lt;li&gt;Locality Search Hashing (LSH)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="2444" LastEditDate="2020-01-25T15:24:08.697" LastActivityDate="2021-02-16T13:54:04.610" Title="What are examples of approaches to dimensionality reduction of feature vectors?" Tags="&lt;convolutional-neural-networks&gt;&lt;feature-learning&gt;&lt;principal-component-analysis&gt;&lt;dimensionality-reduction&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17635" PostTypeId="2" ParentId="17566" CreationDate="2020-01-23T11:48:38.523" Score="1" Body="&lt;p&gt;So, if I go the opposite way, start with my y and predict an x, and then ask for the inverse of that - I get really good results (actually - 100% accuracy). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential([&#xA;    Dense(784, input_shape=(10,), activation='sigmoid'),&#xA;])&#xA;model.compile(loss=keras.losses.binary_crossentropy,&#xA;              optimizer=keras.optimizers.Adam(0.01),&#xA;              metrics=['binary_crossentropy'])&#xA;model.fit(y_train, x_train,&#xA;          batch_size=batch_size,&#xA;          epochs=epochs,&#xA;          verbose=1,&#xA;          validation_data=(y_test, x_test))&#xA;# train until accuracy &amp;gt; 0.9, then:&#xA;W, b = model.get_weights()&#xA;y = y_train&#xA;x = reverse.predict(y)&#xA;z = rev_sigmoid(x)&#xA;y_hat = rev_linear(z, W, b)&#xA;(y_hat.argmax(axis=1) == y.argmax(axis=1)).mean()  # 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After playing a bit with some toy examples, I think the other way is probably not possible, as the matrices don't have an inverse. Putting these (toy) matrices in WolframAlpha for example tells you the determinant is 0, but in numpy the determinant is just slightly bigger than 0, so you manage to calculate an &quot;inverse&quot; which is not really an inverse and get the bad results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also makes sense. In the reversed scenario, we start with 10 dimension, expand to 784, and then collapse back to 10. But in the &quot;regular&quot; scenario, we start at 784, collapse to 10, and then expand to 784 again - and (I guess) too much information is lost then. &lt;/p&gt;&#xA;" OwnerUserId="27947" LastEditorUserId="27947" LastEditDate="2020-01-23T18:18:47.530" LastActivityDate="2020-01-23T18:18:47.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17636" PostTypeId="1" AcceptedAnswerId="17725" CreationDate="2020-01-23T12:10:05.733" Score="4" ViewCount="263" Body="&lt;p&gt;It's not clear to me whether or not someone whose work aims to &lt;strong&gt;improve an NLP system&lt;/strong&gt; may be called a &quot;&lt;em&gt;Computational Linguist&lt;/em&gt;&quot; even when she/he &lt;strong&gt;doesn't modify the algorithm directly&lt;/strong&gt; by coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider the following activities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Annotation for Machine Learning: &#xA;analysis of Morphology, Syntax, POS tagging&#xA;Annotation, analysis, and annotation of entities (NER) and collocations; supporting content categorization; chunking; word sense disambiguation.&#xA;Recording of technical issues of the annotation tool to improve its reliability.&#xA;Recording of linguistic and logical particular rules adopted by the research team who develops the NLP algorithm to improve consistency between annotation and criteria previously adopted to train the NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May be these activities considered &quot;Computational Linguistics&quot;? If not, which is their professional category and how should they be included in the resume in a word which synthesizes them?&lt;/p&gt;&#xA;" OwnerUserId="22959" LastEditorUserId="1671" LastEditDate="2020-01-23T21:56:11.510" LastActivityDate="2020-02-04T09:35:00.360" Title="What is &quot;Computational Linguistics&quot;?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;terminology&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17637" PostTypeId="2" ParentId="17636" CreationDate="2020-01-23T13:37:34.620" Score="4" Body="&lt;p&gt;Yes. A computational linguist is someone who (among other things) uses computers to process/model/analyse/... natural language. Coding might be one aspect of it, but is about the least important: you can always get a non-linguist programmer to do coding for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I studied &quot;Computational Linguistics&quot; at university, and while programming was taught as part of the course, coding was only a minor aspect in the actual subject matter. The senior professor (and head of the department) wasn't able to do any coding himself; he came from the linguistics side of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being able to program is useful, as it speeds things up and makes you more independent, but it is by no means an important part of being a computational linguist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: I have been accused of misrepresenting the field of CL. However, it is a broad, interdisciplinary field, and comprises many elements. Sure, on the academic/research side you might do more programming than in the applied/commercial side, but I maintain that you can easily work as a computational linguist without actually doing any programming. For most tasks, readily available software exists by now, so you don't actually need to program anything new.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-02-04T09:35:00.360" LastActivityDate="2020-02-04T09:35:00.360" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17638" PostTypeId="2" ParentId="17512" CreationDate="2020-01-23T14:03:05.517" Score="0" Body="&lt;p&gt;Firstly when you say an object detection CNN, there are a huge number of model architectures available. Considering that you have narrowed down on your model architecture a CNN will have a few common layers like the ones below with hyperparameters you can tweak:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Convolution Layer:- number of kernels, kernel size, stride length, padding&lt;/li&gt;&#xA;&lt;li&gt;MaxPooling Layer:- kernel size, stride length, padding&lt;/li&gt;&#xA;&lt;li&gt;Dense Layer:- size&lt;/li&gt;&#xA;&lt;li&gt;Dropout:- Percentage to keep/drop&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32973" LastEditorUserId="32973" LastEditDate="2020-01-23T14:13:43.460" LastActivityDate="2020-01-23T14:13:43.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17639" PostTypeId="1" CreationDate="2020-01-23T16:59:46.943" Score="2" ViewCount="3544" Body="&lt;p&gt;I'm working on a Python Keras/Tensorflow image recognition script (on Ubuntu 18.04) which works ok, but it will only train on CPU (which is slow) and I want to be using my GPU (i have a Nvidia Geforce GTX1080).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have installed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;tensorflow-gpu 2.0.0-beta1 (via pip; note i also tried 1.4 but same issue)&lt;/li&gt;&#xA;&lt;li&gt;Cuda10.2 (via deb files from Nvidia website)&lt;/li&gt;&#xA;&lt;li&gt;cuDNN 7.6.5 (via Nvidia dev website)&lt;/li&gt;&#xA;&lt;li&gt;CUPTI (via apt)&lt;/li&gt;&#xA;&lt;li&gt;Nvidia drivers 440.33.01&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$ nvidia-smi &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+-----------------------------------------------------------------------------+&#xA;| NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |&#xA;|-------------------------------+----------------------+----------------------+&#xA;| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |&#xA;| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |&#xA;|===============================+======================+======================|&#xA;|   0  GeForce GTX 1080    On   | 00000000:B3:00.0  On |                  N/A |&#xA;| 38%   42C    P0    38W / 180W |    213MiB /  8118MiB |      0%      Default |&#xA;+-------------------------------+----------------------+----------------------+&#xA;&#xA;+-----------------------------------------------------------------------------+&#xA;| Processes:                                                       GPU Memory |&#xA;|  GPU       PID   Type   Process name                             Usage      |&#xA;|=============================================================================|&#xA;|    0      2109      G   /usr/lib/xorg/Xorg                           210MiB |&#xA;+-----------------------------------------------------------------------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However when I run my script, tensorflow is unable to use the GPU, and gives these errors (and falls back to running on CPU):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: &#xA;name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7335&#xA;pciBusID: 0000:b3:00.0&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory&#xA;I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7&#xA;W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I seem to be missing these libraries: &lt;code&gt;libcudart.so.10.0, libcublas.so.10.0, libcufft.so.10.0, libcurand.so.10.0, libcusolver.so.10.0, libcusparse.so.10.0&lt;/code&gt;, but I'm not sure how to get them installed, apt does not appear to have version 10 available for these packages.&lt;/p&gt;&#xA;" OwnerUserId="32977" LastActivityDate="2020-01-24T22:27:24.313" Title="Tensorflow-gpu cannot use Nvidia GPU with CUDA" Tags="&lt;tensorflow&gt;&lt;image-recognition&gt;&lt;keras&gt;&lt;gpu&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-10-25T16:02:54.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="17640" PostTypeId="1" CreationDate="2020-01-23T17:32:56.747" Score="1" ViewCount="29" Body="&lt;p&gt;I have an object with known size and I want to know that's the distance from the camera and camera angle. Is there any way to do this? I have a single source (camera).&lt;/p&gt;&#xA;" OwnerUserId="32763" LastEditorUserId="2444" LastEditDate="2020-01-25T15:22:13.393" LastActivityDate="2020-01-25T15:22:13.393" Title="What's the best solution to find distance of an object to camera?" Tags="&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17641" PostTypeId="1" CreationDate="2020-01-23T18:08:07.470" Score="1" ViewCount="79" Body="&lt;p&gt;I have 26 features from tabular data (clinical variables from patients like age gender etc) that I want to add to my cnn which is using xray images from patients. I am using the inception network. Right now I am just concatenating these features to the final fully connected layer just before the softmax activation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern though is that since this final layer also contains 2048 image features that these 26 features will not contribute much in the classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Empirically, these 26 features should contribute more than the 2048 image features. A random forest trained on only 26 features did better than the cnn using only the images. I'd like to get a model that does better than either of them separately so I thought I should add these metadata features to the cnn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my concerns warranted? What is the best approach?&lt;/p&gt;&#xA;" OwnerUserId="32927" LastActivityDate="2020-01-23T18:08:07.470" Title="What is generally the best way to combine tabular image metadata with image data in a convolutional neural network?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17642" PostTypeId="1" CreationDate="2020-01-23T18:33:38.127" Score="2" ViewCount="95" Body="&lt;p&gt;I am training an undercomplete autoencoder network for feature selection. I am using one hidden layer in the encoder and decoder networks each. The ELU activation function is used for each layer. For optimization, I am using the ADAM optimizer. Moreover, to improve convergence, I have also introduced learning rate decay. The model shows good convergence initially, but later starts to generate losses (12 digits) in the same range of values for several epochs, and is not converging. How to solve this issue?&lt;/p&gt;&#xA;" OwnerUserId="32981" LastActivityDate="2020-01-27T04:52:05.610" Title="Autoencoder network for feature selection not converging" Tags="&lt;autoencoders&gt;&lt;learning-rate&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17645" PostTypeId="2" ParentId="17544" CreationDate="2020-01-24T01:01:29.883" Score="1" Body="&lt;p&gt;Your question is still nearly perfectly unclear, but let's make a few guesses in order to make some progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biggest mystery are your time slices:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How many of them you need? With more granular time, you get more unknowns and more constraints and the complexity grows quickly.&lt;/li&gt;&#xA;&lt;li&gt;When someone gets a car, they probably use it to travel somewhere, so it may be necessary to assign the driver to the car for multiple subsequent slices &lt;em&gt;depending in the distance&lt;/em&gt;. That's something completely missing from your description (so I'll ignore it).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As already said, you need a cost function, i.e., something to minimize. Even before that, you need to define your variables. Let's say, you need &lt;em&gt;binary&lt;/em&gt; variables&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x[t, c, e] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;such that &lt;code&gt;x[t, c, e] = 1&lt;/code&gt; when at time slice &lt;code&gt;t&lt;/code&gt; the car &lt;code&gt;c&lt;/code&gt; is assigned to the employee &lt;code&gt;e&lt;/code&gt; (otherwise, it's zero). Now, we can specify some constraints&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;x[t, c, e] = 0&lt;/code&gt; for each &lt;code&gt;t&lt;/code&gt;, when the employee &lt;code&gt;e&lt;/code&gt; can't drive the car &lt;code&gt;c&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;for each &lt;code&gt;t&lt;/code&gt; and &lt;code&gt;c&lt;/code&gt;, the sum of &lt;code&gt;x[t, c, e]&lt;/code&gt; over all &lt;code&gt;e&lt;/code&gt; is at most one (as no employee can use two cars at the same time)&lt;/li&gt;&#xA;&lt;li&gt;for each &lt;code&gt;t&lt;/code&gt; and &lt;code&gt;e&lt;/code&gt;, the sum of &lt;code&gt;x[t, c, e]&lt;/code&gt; over all &lt;code&gt;c&lt;/code&gt; is at most one (as no car can be use twice at the same time)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I may have missed some constraints, but that's no big deal; I hope you've got the idea.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Your cost function will be a sum of a few terms.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What you wrote about priorities is unclear to me, too, but you can define a weight &lt;code&gt;w[c, e]&lt;/code&gt; (or maybe just &lt;code&gt;w[c]&lt;/code&gt;), so let the sum of &lt;code&gt;w[c, e] * x[t, c, e]&lt;/code&gt; over all &lt;code&gt;t&lt;/code&gt;, &lt;code&gt;c&lt;/code&gt; and &lt;code&gt;e&lt;/code&gt; be the term describing these priorities.&lt;/li&gt;&#xA;&lt;li&gt;Let &lt;code&gt;u[e]&lt;/code&gt; be the cost of giving the employee &lt;code&gt;e&lt;/code&gt; no car. The expression &lt;code&gt;y[t, e] = 1 - sum over all c of x[t, c, e]&lt;/code&gt; equals to one, when the employee &lt;code&gt;e&lt;/code&gt; has no car at time slice &lt;code&gt;t&lt;/code&gt;. Now, the corresponding cost term can be expressed as the sum of &lt;code&gt;u[e] * y[t, e]&lt;/code&gt; over all &lt;code&gt;e&lt;/code&gt; and &lt;code&gt;t&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Again, there may be more of this.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What I described so far is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Integer_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;integer linear programming&lt;/a&gt; formulation of your problem. Convert my text into ILP generating code, get an ILP solver, feed it and run it....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you problem is small, you may even get the optimum. Otherwise, you'll get &lt;em&gt;some&lt;/em&gt; solution (this is guaranteed) and a bound on the optimum, so you can know how far you &lt;em&gt;might&lt;/em&gt; improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's more than any heuristic solver (e.g., simulated annealing) can offer. OTOH arriving at a good solution may be faster using some heuristic solver. But that's all irrelevant until you get a clear problem definition.&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2020-01-24T01:01:29.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17648" PostTypeId="2" ParentId="17553" CreationDate="2020-01-24T05:09:13.810" Score="1" Body="&lt;p&gt;The perceptron convergence theorem states that any architecture will lead to a correlation between the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, you can!&lt;/p&gt;&#xA;" OwnerUserId="32798" LastActivityDate="2020-01-24T05:09:13.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17649" PostTypeId="2" ParentId="16234" CreationDate="2020-01-24T08:18:38.857" Score="1" Body="&lt;p&gt;Sorry for the delay.&#xA;The term &quot;vector-valued feedback&quot; is compared to scalar-valued feedback. The implication (which I should have made explicit) is that, because vector-valued feedback tells the network the correct answer, the changes in weights required to improve performance are reasonably easy to calculate (e.g. using backprop). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, if a scalar-valued feedback is given (as in reinforcement learning) then the network knows only how bad its previous output was, but not how to change weights in order to improve the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A rough analogy would be that vector-valued feedback tells you that you got the wrong answer to a question, and provides the correct answer. In contrast, scalar-valued feedback just tells you 'how wrong' your answer was, but does not tell you how to improve your answer.&lt;/p&gt;&#xA;" OwnerUserId="15257" LastEditorUserId="2444" LastEditDate="2020-01-24T12:34:28.187" LastActivityDate="2020-01-24T12:34:28.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17650" PostTypeId="2" ParentId="6468" CreationDate="2020-01-24T08:34:19.723" Score="1" Body="&lt;p&gt;A multi-layer network in which all units have linear activation functions can always be collapsed to an equivalent network with two layers of units. That is why it is essential to use nonlinear unit activation functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying reason for using nonlinear activation functions involves a remarkable theorem of Cybenko (1989), which states that one layer of nonlinear hidden units is sufficient to approximate any mapping from input to output units. Actually, I think there is a later proof which specifies that the nonlinearity can be any non-polynomial function (e.g. sigmoidal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This text is based on the book: &lt;a href=&quot;https://jim-stone.staff.shef.ac.uk/BookBayes2012/books_by_jv_stone/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence Engines: A Tutorial Introduction to the Mathematics of Deep Learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15257" LastEditorUserId="2444" LastEditDate="2020-01-24T12:33:38.147" LastActivityDate="2020-01-24T12:33:38.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17651" PostTypeId="2" ParentId="92" CreationDate="2020-01-24T09:18:36.353" Score="2" Body="&lt;p&gt;Neural networks are easily fooled, provided you know how to fool them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a linear network with an input layer and an output layer, which has an error function E (we don't need hidden layers to show how to fool a network). For a given input image x, E measures the (squared) difference between the network's output y and the desired (correct) output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output unit’s state y is given by the inner product of x with the output unit’s weight vector w,so that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y=w·x. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we change x to x′ by adding ∆x then the output will change by ∆y to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y′ = w·x+w·∆x (9.4) &#xA;&#xA;   = y + ∆y. (9.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that ∆x defines a direction in the input space; the question is, which direction ∆x will have most impact on y? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By definition, a change in x in the direction ∇E produces the largest&#xA;possible change in y. An adversarial image x' is constructed by taking the derivative ∇E of E with respect to the input image x, that is,  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   x′ = x + ε∇E,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where ε is a small constant. &#xA;By definition, ∇E is the direction of steepest ascent, so the modification ε∇E to x will alter y more than a change in any other direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ggkFa.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ggkFa.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an extract from the book: &lt;a href=&quot;https://jim-stone.staff.shef.ac.uk/BookBayes2012/books_by_jv_stone&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence Engines:  A Tutorial Introduction to the Mathematics of Deep Learning&lt;/a&gt; (2019). &lt;/p&gt;&#xA;" OwnerUserId="15257" LastEditorUserId="2444" LastEditDate="2020-01-24T12:33:02.420" LastActivityDate="2020-01-24T12:33:02.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17652" PostTypeId="2" ParentId="5728" CreationDate="2020-01-24T09:39:26.153" Score="2" Body="&lt;p&gt;A potential disadvantage of gradient-based methods is that they head for the nearest minimum, which is usually not the global minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the only difference between these search methods is the speed with which solutions are obtained, and not the nature of those solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important consideration is time complexity, which is the rate at which the time required to find a solution increases with the number of parameters (weights). In short, the time complexities of a range of different gradient-based methods (including second-order methods) seem to be similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Six different error functions exhibit a median run-time order of approximately O(N to the power 4) on the N-2-N encoder in this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lister, R and Stone J &quot;An Empirical Study of the Time Complexity of Various Error Functions with Conjugate Gradient Back Propagation&quot; , IEEE International Conference on Artificial Neural Networks (ICNN95), Perth, Australia, Nov 27-Dec 1, 1995.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summarised from my book: &lt;em&gt;Artificial Intelligence Engines: A Tutorial Introduction to the Mathematics of Deep Learning&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15257" LastEditorUserId="2444" LastEditDate="2020-01-24T12:25:13.183" LastActivityDate="2020-01-24T12:25:13.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17653" PostTypeId="1" CreationDate="2020-01-24T11:07:54.447" Score="3" ViewCount="76" Body="&lt;p&gt;According to the book &quot;Artificial Intelligence: A Modern Approach&quot;, &quot;In a known environment, the outcomes (or outcome probabilities if the environment is stochastic) for all actions are given.&quot;, and in a deterministic environment, &quot;the next state of the environment is completely determined by the current state and the action executed by the agent...&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the difference between the two terms? Don't they mean the same thing?&lt;/p&gt;&#xA;" OwnerUserId="32999" LastEditorUserId="2444" LastEditDate="2020-01-24T11:19:21.520" LastActivityDate="2020-01-24T11:42:03.687" Title="What is the difference between the concepts &quot;known environment&quot; and &quot;deterministic environment&quot;?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17654" PostTypeId="2" ParentId="3494" CreationDate="2020-01-24T11:38:00.757" Score="1" Body="&lt;p&gt;We also work with Python in &lt;a href=&quot;https://www.n-ix.com/python-development/&quot; rel=&quot;nofollow noreferrer&quot;&gt;our company&lt;/a&gt;. One of the sphere that we use it for is fast prototyping and building highly scalable web applications. For over two decades, our Python developers have been providing businesses with full-stack web-development services, client-server programming and administration. We help our clients build high-load web portals, automation plugins, high-performance data-driven enterprise systems, and many more.&lt;/p&gt;&#xA;" OwnerUserId="33001" LastActivityDate="2020-01-24T11:38:00.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17655" PostTypeId="2" ParentId="17653" CreationDate="2020-01-24T11:42:03.687" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What's the difference between the two terms? Don't they mean the same thing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They mean different things, and can occur in any combination.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A known, deterministic environment&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is an environment where the researcher knows how to calculate all the transitions in advance of observing them, and the transition from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; given action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is always to the same next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; with the same reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: Any classic board game against a fixed opponent (where any situation allows choice of action, the opponent always picks the same choice in the same situation)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;An unknown, deterministic environment&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is an environment where the researcher does not have the knowledge to calculate all the transitions in advance of observing them, but any observation the transition from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; given action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is always to the same next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; with the same reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: Simple mechanical physics environments, where initial measurements are unknown, imprecise or the researcher does not want to code knowledge of them into the agent. E.g. pole balancing.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A known, stochastic environment&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is an environment where the researcher knows all the rules about transitions, but those rules include transitions with random elements. Transition from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; given action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; varies according to some probability function &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; possibly so does the reward &lt;span class=&quot;math-container&quot;&gt;$p(r|s,a)$&lt;/span&gt; - sometimes combined into joint probability function &lt;span class=&quot;math-container&quot;&gt;$p(r, s'|s,a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: Any board game involving dice, e.g. Backgammon.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;An unknown, stochastic environment&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is an environment where the researcher does not know all the rules, or can only calculate expected results with difficulty or to a low level of precision. Transition from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; given action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; varies according to some &lt;em&gt;unknown&lt;/em&gt; probability function &lt;span class=&quot;math-container&quot;&gt;$p(r, s'|s,a)$&lt;/span&gt;. Learning the transition function may require many samples of &lt;span class=&quot;math-container&quot;&gt;$s, a$&lt;/span&gt;, and this will be a statistical approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: In practice, most complex environments, including real-world physics with friction, fluids, non-perfect measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is quite common for purpose of experiments to have a simulation where the environment is technically known (because the researcher wrote it, or has access to the code and underlying models), but that agents are written to treat it like this last, more challenging case. Agents that can figure out how to act without prior knowledge of the environment are often of interest.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-01-24T11:42:03.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17656" PostTypeId="1" CreationDate="2020-01-24T11:45:04.393" Score="3" ViewCount="753" Body="&lt;p&gt;I want to estimate hand poses and recognize gestures using an open-source library like &lt;a href=&quot;https://github.com/CMU-Perceptual-Computing-Lab/openpose&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenPose&lt;/a&gt; on live video. Considering the fact that such libraries are very computationally intensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How likely is it that it will run on a Raspberry Pi 4 using a &lt;a href=&quot;https://www.raspberrypi.org/products/camera-module-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pi-Cam&lt;/a&gt; while giving me something above 15 fps? &lt;strong&gt;Assume that the models are pre-trained.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="33002" LastEditorUserId="2444" LastEditDate="2020-01-24T12:04:01.487" LastActivityDate="2021-04-15T20:57:27.440" Title="How feasible is it to perform pose estimation on a Raspberry Pi 4 using a Pi-Cam?" Tags="&lt;computer-vision&gt;&lt;hardware-evaluation&gt;&lt;pose-estimation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17658" PostTypeId="1" CreationDate="2020-01-24T13:48:48.633" Score="1" ViewCount="30" Body="&lt;p&gt;I have a list of positive nonzero integers &lt;span class=&quot;math-container&quot;&gt;$T=[v_1,\dots,v_|v_\in Z^{\neq}]$&lt;/span&gt; which sum up to &lt;span class=&quot;math-container&quot;&gt;$V=\sum_i v_i$&lt;/span&gt;. Typically, the length of T (number of integers) goes from 100 to 1000. The list is &lt;strong&gt;not sorted&lt;/strong&gt;, i.e., there's no guarantee that &lt;span class=&quot;math-container&quot;&gt;$v_i\leq v_{i+1}\ \forall\ i.$&lt;/span&gt; Each integer can be be assigned either to a set &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; or a set &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;: equivalently, it can be labeled as &lt;span class=&quot;math-container&quot;&gt;$l_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt;. The objective is to label &lt;span class=&quot;math-container&quot;&gt;$v_1,\dots,v_n$&lt;/span&gt; so that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{v_i\in S_1} v_i = 0.3V \tag{1}$$&lt;/span&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{v_i\in S_2} v_i = 0.7V \tag{2}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e., to minimize the cost &#xA;&lt;span class=&quot;math-container&quot;&gt;$$L=\left(\sum_{v_i\in S_1} v_i - 0.3V\right)^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to this point, the problem would be fairly trivial and it definitely wouldn't require AI. However, there are a couple additional details: the integers must be labeled in the sequence they appear (&lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; first, then &lt;span class=&quot;math-container&quot;&gt;$v_2$&lt;/span&gt;, etc.), and each time we &quot;switch&quot; label, we incur a cost. In other words, if the agent assigns &lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$v_2$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$v_3$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;, etc., it should be penalized for that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of formalizing this by counting the number &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; of switches (of course &lt;span class=&quot;math-container&quot;&gt;$m\geq 1$&lt;/span&gt;) and adding it to &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, i.e. by modifying the cost function to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L'=\left(\sum_{v_i\in S_1} v_i - 0.3V\right)^2+\beta m^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; is a positive parameter, which I could use to weight the two objectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it make sense to cast this as a Reinforcement Learning problem? Or is it more appropriately an &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_planning_and_scheduling&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI planning&lt;/a&gt; problem? Can you suggest an efficient algorithm to solve it?&lt;/p&gt;&#xA;" OwnerUserId="20874" LastEditorUserId="20874" LastEditDate="2020-01-24T15:52:54.083" LastActivityDate="2020-01-24T15:52:54.083" Title="Can I solve this assignment problem with RL or AI planning, and if yes how?" Tags="&lt;reinforcement-learning&gt;&lt;planning&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17660" PostTypeId="1" CreationDate="2020-01-24T14:17:46.640" Score="1" ViewCount="21" Body="&lt;p&gt;I know that ensembles can be made by combining sklearn models with a VotingClassifier, but is it possible to combine different deep learning models? Will I have to make something similar to Voting Classifiers?&lt;/p&gt;&#xA;" OwnerUserId="32490" LastEditorUserId="2444" LastEditDate="2020-01-24T15:24:09.903" LastActivityDate="2020-01-24T15:24:09.903" Title="How can we combine different deep learning models?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ensemble-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17661" PostTypeId="1" CreationDate="2020-01-24T14:44:04.130" Score="0" ViewCount="398" Body="&lt;p&gt;How can a data stream for a RNN (LSTM) be handled, when the stream contains data sets belonging to different prediction classes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training phase:&#xA;I have trained a LSTM to predict a class out of a sequence of Letters. For the training phase I used a fixed data array where the beginning an the ending of a sequence belonged to a class. Of course there is a little noise but the whole data set was labled with a class. E.g:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Seq.    is  Class&#xA;ABC     is  One&#xA;CBA     is  Two&#xA;ABD     is  Three&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The network predicts well when it sees a static data array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem in Prediction Phase:&#xA;During prediction the LSTM will receive a data stream where there is a sequence off arrays but there is no delimiter. The data set can not be distinguished or separated. I am not sure how it would perform when I have a data stream for different classes like &#xA;    ABCABCCBAABD. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess in speech recognition one must face similiar problems.&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="27777" LastEditDate="2020-01-24T16:32:26.580" LastActivityDate="2021-02-17T19:03:40.310" Title="How to process data in a data stream for a LSTM" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17664" PostTypeId="1" CreationDate="2020-01-24T15:57:44.547" Score="1" ViewCount="469" Body="&lt;p&gt;if I define the architecture of a neural network using only dense fully connected layers and train them such that there are two models which are trained using &lt;strong&gt;model.fit()&lt;/strong&gt; and &lt;strong&gt;GradientTape&lt;/strong&gt;. Both the methods of training use the &lt;em&gt;same&lt;/em&gt; model architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The randomly initialized weights are shared between the two models and all other parameters such as optimizer, loss function and metrics are also the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dimensions of training and testing sets are:&#xA;X_train = (960, 4), y_train = (960,), X_test = (412, 4) &amp;amp; y_test = (412,)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd, numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import seaborn as sns&#xA;from sklearn.model_selection import train_test_split&#xA;from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler&#xA;from sklearn.preprocessing import LabelEncoder&#xA;from sklearn.metrics import accuracy_score, precision_score, recall_score, confusion_matrix&#xA;&#xA;import tensorflow as tf&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras.layers import Dense&#xA;import tensorflow_model_optimization as tfmot&#xA;from tensorflow_model_optimization.sparsity import keras as sparsity&#xA;&#xA;&#xA;def create_nn():&#xA;    &quot;&quot;&quot;&#xA;    Function to create a&#xA;    Neural Network&#xA;    &quot;&quot;&quot;&#xA;    model = Sequential()                                                    &#xA;&#xA;    model.add(&#xA;        Dense(&#xA;            units = 4, activation = 'relu',&#xA;            kernel_initializer = tf.keras.initializers.GlorotNormal(),&#xA;            input_shape = (4,)&#xA;        )&#xA;    )&#xA;&#xA;    model.add(&#xA;        Dense(&#xA;            units = 3, activation = 'relu',&#xA;            kernel_initializer = tf.keras.initializers.GlorotNormal()&#xA;        )&#xA;    )&#xA;&#xA;    model.add(&#xA;        Dense(&#xA;            units = 1, activation = 'sigmoid'&#xA;        )&#xA;    )&#xA;&#xA;    &quot;&quot;&quot;&#xA;    # Compile the defined NN model above-&#xA;    model.compile(&#xA;        loss = 'binary_crossentropy',  # loss = 'categorical_crossentropy'&#xA;        optimizer = tf.keras.optimizers.Adam(lr = 0.001),&#xA;        metrics=['accuracy']&#xA;    )&#xA;    &quot;&quot;&quot;&#xA;&#xA;    return model&#xA;&#xA;&#xA;# Instantiate a model- model = create_nn()&#xA;&#xA;# Save weights for fair comparison- model.save_weights(&quot;Random_Weights.h5&quot;, overwrite=True)&#xA;&#xA;&#xA;# Create datasets to be used for GradientTape-&#xA;# Use tf.data to batch and shuffle the dataset train_ds = tf.data.Dataset.from_tensor_slices(&#xA;    (X_train, y_train)).shuffle(100).batch(32)&#xA;&#xA;test_ds = tf.data.Dataset.from_tensor_slices(&#xA;    (X_test, y_test)).shuffle(100).batch(32)&#xA;&#xA;# Define early stopping- callback = tf.keras.callbacks.EarlyStopping(&#xA;    monitor='val_loss', patience=3,&#xA;    min_delta = 0.001, mode = 'min' )&#xA;&#xA;# Train defined model- history_orig = model.fit(&#xA;    x = X_train, y = y_train,&#xA;    batch_size = 32, epochs = 500,&#xA;    validation_data = (X_test, y_test),&#xA;    callbacks = [callback],&#xA;    verbose = 1 )&#xA;&#xA;&#xA;# Instantiate a model- model_gt = create_nn()&#xA;&#xA;# Restore random weights as used by the previous model for fair comparison- model_gt.load_weights(&quot;Random_Weights.h5&quot;)&#xA;&#xA;&#xA;# Choose an optimizer and loss function for training- loss_fn = tf.keras.losses.BinaryCrossentropy() optimizer = tf.keras.optimizers.Adam(lr = 0.001)&#xA;&#xA;# Select metrics to measure the error &amp;amp; accuracy of model.&#xA;# These metrics accumulate the values over epochs and then&#xA;# print the overall result- train_loss = tf.keras.metrics.Mean(name = 'train_loss') train_accuracy = tf.keras.metrics.BinaryAccuracy(name = 'train_accuracy')&#xA;&#xA;test_loss = tf.keras.metrics.Mean(name = 'test_loss') test_accuracy = tf.keras.metrics.BinaryAccuracy(name = 'train_accuracy')&#xA;&#xA;&#xA;# Use tf.GradientTape to train the model-&#xA;&#xA;@tf.function def train_step(data, labels):&#xA;    &quot;&quot;&quot;&#xA;    Function to perform one step of Gradient&#xA;    Descent optimization&#xA;    &quot;&quot;&quot;&#xA;&#xA;    with tf.GradientTape() as tape:&#xA;        predictions = model_gt(data)&#xA;        loss = loss_fn(labels, predictions)&#xA;&#xA;    gradients = tape.gradient(loss, model_gt.trainable_variables)&#xA;    optimizer.apply_gradients(zip(gradients, model_gt.trainable_variables))&#xA;&#xA;    train_loss(loss)&#xA;    train_accuracy(labels, predictions)&#xA;&#xA;&#xA;@tf.function def test_step(data, labels):&#xA;    &quot;&quot;&quot;&#xA;    Function to test model performance&#xA;    on testing dataset&#xA;    &quot;&quot;&quot;&#xA;&#xA;    predictions = model_gt(data)&#xA;    t_loss = loss_fn(labels, predictions)&#xA;&#xA;    test_loss(t_loss)&#xA;    test_accuracy(labels, predictions)&#xA;&#xA;&#xA;EPOCHS = 100&#xA;&#xA;# User input- minimum_delta = 0.001 patience = 3&#xA;&#xA;patience_val = np.zeros(patience)&#xA;&#xA;&#xA;# Dictionary to hold scalar metrics- history = {}&#xA;&#xA;history['accuracy'] = np.zeros(EPOCHS) history['val_accuracy'] = np.zeros(EPOCHS) history['loss'] = np.zeros(EPOCHS) history['val_loss'] = np.zeros(EPOCHS)&#xA;&#xA;for epoch in range(EPOCHS):&#xA;    # Reset the metrics at the start of the next epoch&#xA;    train_loss.reset_states()&#xA;    train_accuracy.reset_states()&#xA;    test_loss.reset_states()&#xA;    test_accuracy.reset_states()&#xA;&#xA;    for x, y in train_ds:&#xA;        train_step(x, y)&#xA;&#xA;    for x_t, y_t in test_ds:&#xA;        test_step(x_t, y_t)&#xA;&#xA;    template = 'Epoch {0}, Loss: {1:.4f}, Accuracy: {2:.4f}, Test Loss: {3:.4f}, Test Accuracy: {4:4f}'&#xA;&#xA;    history['accuracy'][epoch] = train_accuracy.result()&#xA;    history['loss'][epoch] = train_loss.result()&#xA;    history['val_loss'][epoch] = test_loss.result()&#xA;    history['val_accuracy'][epoch] = test_accuracy.result()&#xA;&#xA;    print(template.format(epoch + 1, &#xA;                          train_loss.result(), train_accuracy.result()*100,&#xA;                          test_loss.result(), test_accuracy.result()*100))&#xA;&#xA;    if epoch &amp;gt; 2:&#xA;        # Computes absolute differences between 3 consecutive loss values-&#xA;        differences = np.abs(np.diff(history['val_loss'][epoch - 3:epoch], n = 1))&#xA;&#xA;        # Checks whether the absolute differences is greater than 'minimum_delta'-&#xA;        check =  differences &amp;gt; minimum_delta&#xA;&#xA;        # print('differences: {0}'.format(differences))&#xA;&#xA;        # Count unique element with it's counts-&#xA;        # elem, count = np.unique(check, return_counts=True)&#xA;        # print('\nelem = {0}, count = {1}'.format(elem, count))&#xA;&#xA;        if np.all(check == False):&#xA;        # if elem.all() == False and count == 2:&#xA;            print(&quot;\n\nEarlyStopping Evoked! Stopping training\n\n&quot;)&#xA;            break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &quot;model.fit()&quot; method, it takes around 82 epochs, while GradientTape method takes 52 epochs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is there this discrepancy in the number of epochs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="31215" LastEditorUserId="31215" LastEditDate="2020-01-24T17:01:55.213" LastActivityDate="2020-01-24T17:01:55.213" Title="TensorFlow fit() and GradientTape - number of epochs are different" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17665" PostTypeId="2" ParentId="17661" CreationDate="2020-01-24T16:28:32.230" Score="1" Body="&lt;p&gt;One thing I might add is not to expect the ML model to take care of everything, that is where ML engineering comes into play. One suggestion I have, without knowing much about your data stream, is that you need to implement a real-time data ingestion/pipeline, like &lt;a href=&quot;https://kafka.apache.org/documentation/streams/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apache Kafka Stream&lt;/a&gt; (so similar implementation), where you can decompose your stream into output that your model was trained for. You could split by dictionary, pattern or whatever you need, then push the split output into another real-time stream that can be ingested by your model. Not a lot to go for, but hopefully it helps a little. &lt;/p&gt;&#xA;" OwnerUserId="33006" LastActivityDate="2020-01-24T16:28:32.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17666" PostTypeId="2" ParentId="17544" CreationDate="2020-01-24T17:05:17.613" Score="1" Body="&lt;p&gt;I think there are different ways to solve the problem you presented:&#xA;1) It could be seen as classical resource optimization problem that can be solved via Linear Programming setup (highly suggested taking a look, not everything needs ML). You can find some resources &lt;a href=&quot;http://home.ubalt.edu/ntsbarsh/opre640a/partVIII.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and quick intro to LP &lt;a href=&quot;http://home.ubalt.edu/ntsbarsh/opre640a/partVIII.htm#rlp&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Linear Programming (LP) is a mathematical procedure for determining optimal allocation of scarce resources. LP is a procedure that has found practical application in almost all facets of business, from advertising to production planning. Transportation, distribution, and aggregate production planning problems are the most typical objects of LP analysis. In the petroleum industry, for example a data processing manager at a large oil company recently estimated that from 5 to 10 percent of the firm's computer time was devoted to the processing of LP and LP-like models.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;2) Setup as ML problem using Deep Q-learning Network (might be overkill), at least it comes to my mind as easier setup (potentially). There are probably tons of ways to set your problem and environment. With DQN you would build an simulated environment for your problem where you create an list (or dictionary) of cars with the license requirement, a list for employees and their license, another one for schedule which could simply state 0 = no meeting at that time, 1 = meeting at that time (assuming not all employees need a car at all hours, if that is not true, then you can even remove the schedule requirement). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So DQN is all about action -&gt; reward optimization. You feed in a state, that could be a series of variables, and it spins our an array of actions with their respective scoring. For state columns, you could use cars availability (like one column per vehicle), while actions could be a 2-dimensional array of weather each employee might take a specific car. You could leave in the environment simulator the logic check of weather an employee has the correct license. I.e. if the model sends an employee to the wrong licensed car, then you can return a negative value as penalty as reward. For correct assignment, reward could be reward assigned to each employee (fixed or variable) minus cost of car. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once trained, at any given state sent to the model, it should output a 2D matrix showing the score of each employee taking each vehicle. For each vehicle, you could select the employee that would get max score, so just numpy slicing and dicing at that point. Again this is a quick idea after 10 minutes of thoughts, so take with little pinch of salt. Also there are many DQN flavors (target-Q, dueling-Q, experience replay, rainbow DQN etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Use the DQN structure above but simply it into a Q-learning problem. The actions set could be the same, but state could simply by period of your schedule (1,2,3,4,5...). You leave most logic to the simulated environment, like availability of cars at each period state (like at each state, based on actions return, you can dynamically set which cars will be available for next one). Make sure you set reward and penalty correctly. Create a good balance of exploration vs exploiting, maybe using epsilon-greedy policy and let it run.&lt;/p&gt;&#xA;" OwnerUserId="33006" LastEditorUserId="33006" LastEditDate="2020-01-24T17:13:21.730" LastActivityDate="2020-01-24T17:13:21.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17667" PostTypeId="1" AcceptedAnswerId="17668" CreationDate="2020-01-24T17:30:27.417" Score="1" ViewCount="152" Body="&lt;p&gt;This is a piece of code from my homework. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# action policy: implements epsilon greedy and softmax&#xA;def select_action(self, state, epsilon):&#xA;    qval = self.qtable[state]&#xA;    prob = []&#xA;    if (self.softmax):&#xA;        # use Softmax distribution&#xA;        prob = sp.softmax(qval / epsilon)&#xA;        #print(prob)&#xA;    else:&#xA;        # assign equal value to all actions&#xA;        prob = np.ones(self.actions) * epsilon / (self.actions -1)&#xA;        # the best action is taken with probability 1 - epsilon&#xA;        prob[np.argmax(qval)] = 1 - epsilon&#xA;    return np.random.choice(range(0, self.actions), p = prob)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is a method in order to select the best action according to the two polices i think. My question is, why in the softmax computation there is the epsilon parameter used as temperature. Is really the same thing? Are they different? I think they should be two different variables. Should the temperature be a fixed value over time? Because when i use the epsilon-greedy policy my epsilon decrease over time.&lt;/p&gt;&#xA;" OwnerUserId="32694" LastActivityDate="2020-01-24T19:39:13.623" Title="Is the temperature equal to epsilon in Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17668" PostTypeId="2" ParentId="17667" CreationDate="2020-01-24T19:39:13.623" Score="1" Body="&lt;p&gt;Your are correct that epsilon in epsilon-greedy and temperature parameter in the &quot;softmax distribution&quot; are different parameters, although they serve a similar purpose. The original author of the code has taken a small liberty with variable names in the &lt;code&gt;select_action&lt;/code&gt; method in order to use just one simple name as a positional argument.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Should the temperature be a fixed value over time? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not necessarily, if your goal is to converge on an optimal policy you will want to decrease temperature. A slow decay factor applied after each update or episode, as you might use for epsilon (e.g. 0.999 or other value close to 1), can also work for temperature decay. A very high temperature is roughly equivalent to epsilon of 1. As temperature becomes lower, differences in action value estimates become major differences in action selection probabilities, with a sometimes desirable effect of picking &quot;promising&quot; action choices more often, plus with very low probabilities of picking the actions with the worst estimates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a more sophisticated action selection such as the temperature based on in the example code can speed learning in RL. However, this particular approach is only good in some cases - it is a bit fiddly to tune, and can simply not work at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tricky part of using a temperature parameter is choosing good starting point, as well as the decay rate and end values (you have to do the last for epsilon-decay as well). The problem is that the impact of using this distribution depends on the actual differences between action choices. You need a temperature value on roughly the same scale as the Q value differences. This is difficult to figure out in advance. In addition, if the Q value differences are more pronounced in some states than in others, you risk either having next to no exploration or having too much in some parts of the problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-01-24T19:39:13.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17669" PostTypeId="2" ParentId="17639" CreationDate="2020-01-24T21:41:46.187" Score="1" Body="&lt;p&gt;Seems like you have a CUDA version conflict. Remove the existing CUDA 10.2 and install CUDA 10.0 (going by your missing libraries, it requires a v10.0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the archived releases here:&#xA;&lt;a href=&quot;https://developer.nvidia.com/cuda-toolkit-archive&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developer.nvidia.com/cuda-toolkit-archive&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-01-24T22:27:24.313" LastActivityDate="2020-01-24T22:27:24.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17670" PostTypeId="1" AcceptedAnswerId="17671" CreationDate="2020-01-24T23:01:36.417" Score="2" ViewCount="215" Body="&lt;p&gt;In the literature and textbooks, one often sees supervised learning expressed as a conditional probability, e.g.,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho(\vec{y}|\vec{x},\vec{\theta})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\vec{\theta}$&lt;/span&gt; denotes a learned set of network parameters, &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; is an arbitrary input, and &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt; is an arbitrary output. If we assume we have already learned &lt;span class=&quot;math-container&quot;&gt;$\vec{\theta}$&lt;/span&gt;, then, in words, &lt;span class=&quot;math-container&quot;&gt;$\rho(\vec{y}|\vec{x},\vec{\theta})$&lt;/span&gt; is the &lt;em&gt;probability&lt;/em&gt; that the network will output an arbitrary &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt; given an arbitrary input &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am having a hard time reconciling how, &lt;strong&gt;&lt;em&gt;after&lt;/em&gt;&lt;/strong&gt; learning &lt;span class=&quot;math-container&quot;&gt;$\vec{\theta}$&lt;/span&gt;, there is still a probabilistic aspect to it. Post training, a network is, in general, a deterministic function, not a probability. For any specific input &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt;, a trained network will always produce the same output.&lt;/p&gt;&#xA;&lt;p&gt;Any insight would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="33012" LastEditorUserId="2444" LastEditDate="2020-07-09T13:59:50.377" LastActivityDate="2020-07-09T19:53:54.427" Title="How can supervised learning be viewed as a conditional probability of the labels given the inputs?" Tags="&lt;machine-learning&gt;&lt;probability&gt;&lt;supervised-learning&gt;&lt;statistical-ai&gt;&lt;bayesian-deep-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17671" PostTypeId="2" ParentId="17670" CreationDate="2020-01-24T23:57:54.997" Score="3" Body="&lt;p&gt;This formulation/interpretation can indeed be confusing (or even misleading), as the output of a neural network is usually deterministic (i.e. given the same input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, the output is always the same, so there is no sampling), and there isn't really a probability distribution that models any uncertainty associated with the parameters of the network or the input.&lt;/p&gt;&#xA;&lt;p&gt;People often use this notation to indicate that, in the case of classification, there is a categorical distribution over the labels given the inputs, but this can be misleading, as the &lt;a href=&quot;https://stats.stackexchange.com/a/437769/82135&quot;&gt;softmax (the function often used to model this categorical distribution) only squashes its inputs and doesn't really model any uncertainty associated with the input or the parameter of the neural network&lt;/a&gt;, although the elements of the resulting vector add up to 1. In other words, in traditional deep learning, only a &lt;a href=&quot;https://en.wikipedia.org/wiki/Point_estimation&quot; rel=&quot;nofollow noreferrer&quot;&gt;point estimate&lt;/a&gt; for each parameter of the network is learned and no uncertainty is properly modeled.&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless, certain supervised learning problems have a formal probabilistic &lt;strong&gt;interpretation&lt;/strong&gt;. For example, the minimization of the &lt;em&gt;mean squared error&lt;/em&gt; function is equivalent to the maximization of a log probability, assuming your probability distribution is a Gaussian with a mean equal to the output of your model. In this probabilistic interpretation, you typically attempt to learn a probability (e.g. of the labels in the training dataset) and not a probability distribution. Watch &lt;a href=&quot;https://www.youtube.com/watch?v=vEPQNwxd1Y4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lecture 9.5 — The Bayesian interpretation of weight decay (Neural Networks for Machine Learning)&lt;/a&gt; by G. Hinton or read the paper &lt;a href=&quot;http://papers.nips.cc/paper/613-bayesian-learning-via-stochastic-dynamics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Learning via Stochastic Dynamics&lt;/a&gt; or &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.55.524&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Training of Backpropagation Networks by the Hybrid Monte Carlo Method&lt;/a&gt; by R. Neal for more details.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, there are &lt;a href=&quot;http://proceedings.mlr.press/v37/blundell15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural networks (BNNs)&lt;/a&gt;, which actually maintain a probability distribution over each parameter of the neural network that models the uncertainty associated with the value of this parameter. During the forward pass of this BNN, the specific parameters are actually sampled from the corresponding probability distributions. The actual learnable parameters of a BNN are the parameters of these distributions. For example, if you decide to have a Gaussian distribution over each parameter of the neural network, then you will learn the mean and variance of these Gaussians.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-09T19:53:54.427" LastActivityDate="2020-07-09T19:53:54.427" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17672" PostTypeId="1" CreationDate="2020-01-25T09:30:22.750" Score="2" ViewCount="461" Body="&lt;p&gt;I have trouble understanding how to implement derivative of softmax function. Here is what I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def Softmax(x):&#xA;    e_x = np.exp(x - np.max(x))&#xA;    return e_x / e_x.sum()&#xA;&#xA;def d_Softmax(X):&#xA;    x=Softmax(X)&#xA;    s=x.reshape(-1,1)&#xA;    return (np.diagflat(s) - np.dot(s, s.T))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure if it works as it should. The normal softmax function, when taking &lt;span class=&quot;math-container&quot;&gt;$m \times n$&lt;/span&gt; matrix as input, outputs a matrix of the same shape. The derivative, however, seems to output a matrix of shape &lt;span class=&quot;math-container&quot;&gt;$mn \times mn$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to implement backpropagation of a simple 3-layer neural network on my own, but no other matrix has the shape aligned with the derivative that the softmax returns, so I don't know what it should be multiplied with.&lt;/p&gt;&#xA;" OwnerUserId="26336" LastEditorUserId="2444" LastEditDate="2020-01-25T11:52:47.810" LastActivityDate="2020-01-25T11:52:47.810" Title="How can I implement derivative of softmax function for matrices in Python?" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;backpropagation&gt;&lt;implementation&gt;&lt;derivative&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-06-02T22:21:36.190" ContentLicense="CC BY-SA 4.0" />
  <row Id="17673" PostTypeId="2" ParentId="17599" CreationDate="2020-01-25T15:44:54.827" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on neuroevolution&lt;/a&gt; contains a list of neuroevolution techniques (e.g. NEAT). I will list below the examples that evolve both the parameters and the topology of the neural network.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.29.5939&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cellular encoding&lt;/a&gt; (1994)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://demo.cs.brandeis.edu/papers/ieeenn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNARL&lt;/a&gt; (1994)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.bham.ac.uk/~axk/evoNN2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;EPNet&lt;/a&gt; (1997)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/220516089_Evolutionary_reinforcement_learning_of_artificial_neural_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary acquisition of neural topologies&lt;/a&gt; (2007)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repositorium.ub.uni-osnabrueck.de/handle/urn:nbn:de:gbv:700-2012101710370&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interactively constrained neuro-evolution&lt;/a&gt; (2012)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1111.5892.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deus ex neural network&lt;/a&gt; (2012)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1902.06703.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spectrum-diverse unified neuroevolution architecture&lt;/a&gt; (2017)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://eprints.bournemouth.ac.uk/26161/7/HAGA_SWEVO.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Covariance matrix adaptation with hypervolume sorted adaptive grid algorithm&lt;/a&gt; (2017)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-01-25T15:44:54.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17674" PostTypeId="2" ParentId="9197" CreationDate="2020-01-25T17:44:01.087" Score="2" Body="&lt;p&gt;I would like to point out this paper: &lt;a href=&quot;https://arxiv.org/pdf/1712.00378.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1712.00378.pdf&lt;/a&gt;, that answers exactly that question.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We then showed that, when learning policies for time-unlimited tasks, it is necessary for correct value estimation, to continue bootstrapping at the end of the partial episodes when termination is due to time limits, or any early termination causes other than the environmental ones.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The authors therefore argue, that the target &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; for a one-step TD update, after transitioning to a state &lt;span class=&quot;math-container&quot;&gt;$s′$&lt;/span&gt; and receiving a reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, should be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;y = &#xA;\begin{cases}&#xA;r &amp;amp; \text{if environment terminates}\\&#xA;r+\gamma\hat{v}_{\pi}(s') &amp;amp; \text{otherwise (including timeouts)}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="33018" LastEditorUserId="33018" LastEditDate="2020-01-25T18:04:59.917" LastActivityDate="2020-01-25T18:04:59.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17675" PostTypeId="5" CreationDate="2020-01-25T18:36:41.407" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-25T18:36:41.407" LastActivityDate="2020-01-25T18:36:41.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17676" PostTypeId="4" CreationDate="2020-01-25T18:36:41.407" Score="0" Body="For questions related to Bayesian deep learning, that is, Bayesian techniques applied to deep learning models (i.e. neural networks)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-25T18:36:41.407" LastActivityDate="2020-01-25T18:36:41.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17677" PostTypeId="2" ParentId="4454" CreationDate="2020-01-25T20:12:31.167" Score="2" Body="&lt;p&gt;One important consideration here: in the last decade or two the machine learning and artificial intelligence fields, which contains the majority of reinforcement learning work, researchers have considered &lt;strong&gt;conferences to be the more impactful publishing venues than journals&lt;/strong&gt;. The particular venue a researcher chooses depends on the data and/or application domain of his or her use of reinforcement learning, and the conferences are often changing, but to get you started the top tier conferences are (in rough order of exclusivity and importance):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://icml.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICML&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://nips.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeurIPS&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://iclr.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICLR&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.aaai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AAAI&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ijcai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IJCAI&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://rldm.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RLDM&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.auai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UAI&lt;/a&gt; (more theory driven) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aistats.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AISTATS&lt;/a&gt; (more theory driven)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://learningtheory.org/colt2020/&quot; rel=&quot;nofollow noreferrer&quot;&gt;COLT&lt;/a&gt; (more theory driven) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.robot-learning.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CoRL&lt;/a&gt; (robotics focus)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://roboticsconference.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSS&lt;/a&gt; (robotics focus) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.icra2020.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICRA&lt;/a&gt; (robotics focus) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.iros2020.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IROS&lt;/a&gt; (robotics focus) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cvpr2020.thecvf.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVPR&lt;/a&gt; (visual data focus)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://eccv2020.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ECCV&lt;/a&gt;/&lt;a href=&quot;http://iccv2019.thecvf.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICCV&lt;/a&gt; (visual data focus) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://2020.emnlp.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMNLP&lt;/a&gt; (language focus) &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://acl2020.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ACL&lt;/a&gt; (language focused)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7752" LastEditorUserId="7752" LastEditDate="2020-03-31T13:55:25.533" LastActivityDate="2020-03-31T13:55:25.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17678" PostTypeId="1" CreationDate="2020-01-25T21:26:04.487" Score="1" ViewCount="39" Body="&lt;p&gt;I'll try to rephrase my problem in the context of video processing. Imagine that initial frame of video has some translational symmetry. The frame evolves according to an update rule. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generate a time series for how an edge, say right up edge, of the frame evolves. I generate another time series for how a larger edge, including the smaller right up edge, evolves. Since there is translational symmetry, I should be able to find how the smaller edge is related to the larger edge. The final goal is to use the obtained correlation to extrapolate to larger edges. I want to find the correlation between these two multivariate time series using machine learning (ML) methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - which one of ML methods can be used in general for this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 - if I use neural networks, the input and output shapes would be &lt;code&gt;(values at time steps, number of variables)&lt;/code&gt;. For the input it makes sense, but how can I define the output layer (for example, for LSTM in tensorflow)?&lt;/p&gt;&#xA;" OwnerUserId="33022" LastEditorUserId="33022" LastEditDate="2020-01-26T09:39:50.117" LastActivityDate="2020-01-26T09:39:50.117" Title="How to exploit translational symmetry for extrapolation in video generation using machine learning" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;time-series&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17679" PostTypeId="2" ParentId="17596" CreationDate="2020-01-26T03:15:52.043" Score="2" Body="&lt;p&gt;Since you have already tried U-Net. You may look into Siamese Networks (with CNNs for images), they are very well known for computing similarity via deep learning. This is a central idea and can be performed with both text and images. As a tip, you may be able to leverage a lot of architecture from U-Net to Siamese. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps, Some useful links to start with :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aclweb.org/anthology/W16-1617.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aclweb.org/anthology/W16-1617.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23793" LastActivityDate="2020-01-26T03:15:52.043" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17680" PostTypeId="2" ParentId="9491" CreationDate="2020-01-26T06:09:50.553" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;(3) Another approach that came across was to, assuming the number of different action set &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is quite small, have functions &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_2}$&lt;/span&gt;, ..., &lt;span class=&quot;math-container&quot;&gt;$f_{\theta_n}$&lt;/span&gt; that returns the action regarding that perticular state with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; valid actions. E.i, the performed action of a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; with 3 number of actions will be predicted by &lt;span class=&quot;math-container&quot;&gt;$\underset{a}{\text{argmax}} \ f_{\theta_3}(s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That sounds pretty complicated and the number of different action sets is usually very high, even for simplest games. Imagine checkers, ignore promotions and jumping for simplicity and there are some &lt;span class=&quot;math-container&quot;&gt;$7 \cdot 4 \cdot 2=56$&lt;/span&gt; possible actions (which is fine), but the number of different sets of these actions is much higher. It's actually difficult to compute how many such sets are possible in a real game - it's surely much less than &lt;span class=&quot;math-container&quot;&gt;$2^{56}$&lt;/span&gt;, but also surely far too big for being practical.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Anyone having another approach worth digging into?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming the number of actions is not too big, you can simply ignore actions which don't apply in a given state. That's different from learning - you don't have to learn returning negative award for illegal actions, you simply don't care and select the &lt;em&gt;legal&lt;/em&gt; action returning the best award.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that your expression &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\forall s \in S: \exists s' \in S: A(s) \neq A(s') \wedge s \neq s'$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be simplified to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\forall s \in S: \exists s' \in S: A(s) \neq A(s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or even&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$|A(s)|_{s \in S} &amp;gt; 1$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2020-01-26T06:09:50.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17681" PostTypeId="1" AcceptedAnswerId="17685" CreationDate="2020-01-26T08:13:10.457" Score="3" ViewCount="93" Body="&lt;p&gt;What is the difference between the notations &lt;span class=&quot;math-container&quot;&gt;$\|x\|_1, \|x\|_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|x|$&lt;/span&gt;? I think &lt;span class=&quot;math-container&quot;&gt;$|x|$&lt;/span&gt; is the magnitude of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2020-01-26T20:40:06.143" LastActivityDate="2020-01-27T10:58:59.370" Title="What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17682" PostTypeId="1" AcceptedAnswerId="17683" CreationDate="2020-01-26T10:20:55.023" Score="5" ViewCount="797" Body="&lt;p&gt;I'm a big fan of computer board games and would like to make Python chess/go/shogi/mancala programs. Having heard of reinforcement learning, I decided to look at OpenAI Gym.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But first of all, I would like to know, is it possible using OpenAI Gym/Universe to create a chess bot that will be nearly as strong as Stockfish and create a go bot that will play as good as AlphaGo?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it worth learning OpenAI?&lt;/p&gt;&#xA;" OwnerUserId="33025" LastEditorUserId="2444" LastEditDate="2020-01-26T20:41:39.007" LastActivityDate="2020-01-26T20:41:39.007" Title="How powerful is OpenAI's Gym and Universe in board games area?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;chess&gt;&lt;gym&gt;&lt;go&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17683" PostTypeId="2" ParentId="17682" CreationDate="2020-01-26T11:02:41.597" Score="4" Body="&lt;p&gt;OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple to use, and IMO worth learning if you want to practice RL using Python to any depth at all. You could use it to ensure you have good understanding of basic algorithms such as Q learning, independently of and before you look at using RL in a board game context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are limitations for Gym and Universe when dealing with multiple agents. The API is not really designed with that in mind. For instance, there is no simple way to add two agents to an environment, you would have to write a new environment and attach an opposing agent inside of it. This is still possible, and not necessarily a terrible idea (it depends on the training setup you want to investigate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to look into classic two-player games, and write bots like AlphaGo and Stockfish, then I would point out that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Game-playing bots often make extensive use of planning that can interrogate potential future game states. OpenAI's Gym doesn't prevent you doing that, but it doesn't help in any way.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Algorithms for AlphaGo are public, with &lt;a href=&quot;https://towardsdatascience.com/from-scratch-implementation-of-alphazero-for-connect4-f73d4554002a&quot; rel=&quot;nofollow noreferrer&quot;&gt;many nice tutorials&lt;/a&gt;. It would be quicker to follow one of these and develop your own bot training code in most cases, than to try and adapt an OpenAI solution for single agent play.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Probably the biggest time-saver you could find for any game is a rules engine that implements the board, pieces and game rules for you. If Gym already has a game environment for the game you want your bot to play, it might be worth checking the Gym code to see what it is integrating, then try to use the same library yourself, but not the Gym environment directly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many decent game-playing algorithms don't use RL at all. You can frame most of them as &lt;em&gt;search&lt;/em&gt; (finding best moves) plus &lt;em&gt;heuristics&lt;/em&gt; (rating moves or positions), and can usually make independent choices for algorithms that perform each sub-task. You can apply RL so that a bot learns game heuristics, then use a more traditional search e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Negamax&quot; rel=&quot;nofollow noreferrer&quot;&gt;negamax&lt;/a&gt; in order to make decisions during play. Or you can use any analysis of the game you like in order to generate heuristics. Very simple games usch as tic-tac-toe (noughts and crosses in UK) can just have heuristic of +1 if X has won, -1 if O has won and 0 otherwise, and still be quickly solved with a minimax search for perfect play.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;DeepMind's AlphaGo uses a variant of &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCTS&lt;/a&gt; for the search algorithm, which can be considered a RL technique, but the lines are a bit blurry around definitions there - it is safer to say that AlphaGo incorporates MCTS as the chosen search technique for both self-play training and active play against any other opponent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-01-26T11:42:08.760" LastActivityDate="2020-01-26T11:42:08.760" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17684" PostTypeId="1" CreationDate="2020-01-26T13:39:02.090" Score="2" ViewCount="59" Body="&lt;p&gt;I tried training a 2 hidden layer network using the mnist dataset, but I am not getting any results. I have tried tuning the learning rate(tried 0.1 and 0.0001) and the number of epochs(tried 10 and 50). I even changed the size of hidden layer from 10 to 250. First i had initialized the weights between 0 and 1 and was getting the same classification for all test samples but added (-) sign to 50% of them(chose the figure of 50% by myself) and that problem was solved. Now I cant figure out why it is not working.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;import matplotlib.pyplot as plt&#xA;import math&#xA;from sklearn.preprocessing import StandardScaler&#xA;&#xA;scaler = StandardScaler() &#xA;&#xA;def to_array(img):&#xA;    img = np.asarray(img)&#xA;    return img&#xA;&#xA;'''def standardize(gray):&#xA;    st_gray = (gray-np.amin(gray))/(np.amax(gray)-np.amin(gray))&#xA;    return st_gray&#xA;'''&#xA;def activ_func(x):&#xA;    for i in range(x.shape[0]):&#xA;        '''x[i][0]=(1-np.e**(-2*x[i][0]))/(1+np.e**(-2*x[i][0]))'''&#xA;        x[i][0] = 1/(1+np.e**(-x[i][0]))&#xA;    return x&#xA;&#xA;def deriv_activ_func(x):&#xA;    for i in range(x.shape[0]):&#xA;        '''x[i][0] = 1-math.pow(x[i][0],2)'''&#xA;        x[i][0] = (x[i][0])*(1-x[i][0])&#xA;    return x&#xA;&#xA;def cost(out_layer, label, ind):&#xA;    cost = (out_layer[ind]-label)**2&#xA;    return cost&#xA;&#xA;def update(x, grad, r):&#xA;    for i in range(x.shape[0]):&#xA;        x[i][0] = x[i][0]+r*grad[i][0]&#xA;    return x&#xA;&#xA;path = &quot;mnist/mnist_train.csv&quot;&#xA;gray = pd.read_csv(path)&#xA;labels = gray['label']&#xA;gray = gray.drop(['label'], axis=1)&#xA;gray = to_array(gray)&#xA;labels = to_array(labels)&#xA;st_gray = np.empty(shape=(gray.shape[1],1))&#xA;&#xA;def rand_sign(w):&#xA;    n = np.random.randint(2,size=w.shape[0]*w.shape[1]).reshape(w.shape[0],w.shape[1])&#xA;    for i in range(w.shape[0]):&#xA;        for j in range(w.shape[1]):&#xA;            if(n[i][j]==1):&#xA;                w[i][j]=(-1)*w[i][j]&#xA;    return w&#xA;&#xA;def initialize():&#xA;    in_layer = np.empty(shape=(st_gray.shape[0],1))&#xA;    out_layer = np.unique(labels).reshape(-1,1)&#xA;    w1 = rand_sign(np.random.rand(250,in_layer.shape[0]))&#xA;    b1 = rand_sign(np.random.rand(250,1))&#xA;    l1 = np.empty(shape=(250,1))&#xA;    w2 = rand_sign(np.random.rand(250,l1.shape[0]))&#xA;    b2 = rand_sign(np.random.rand(250,1))&#xA;    l2 = np.empty(shape=(250,1))&#xA;    w3 = rand_sign(np.random.rand(out_layer.shape[0],l2.shape[0]))&#xA;    b3 = rand_sign(np.random.rand(out_layer.shape[0],1))&#xA;    l3 = np.empty_like(out_layer)&#xA;    return l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,out_layer&#xA;&#xA;def feed_forward(l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,i):&#xA;    st_gray = scaler.fit_transform(gray[i][:].reshape(-1,1))&#xA;    in_layer = st_gray&#xA;    l1 = np.dot(w1,in_layer)+b1&#xA;    l1 = activ_func(l1)&#xA;    l2 = np.dot(w2,l1)+b2&#xA;    l2 = activ_func(l2)&#xA;    l3 = np.dot(w3,l2)+b3&#xA;    l3 = activ_func(l3)&#xA;    return l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer&#xA;&#xA;def one_hot(out_layer,label):&#xA;    for j in range(out_layer.shape[0]):&#xA;        if(out_layer[j][0]==label):&#xA;            out_layer[j][0] = 1&#xA;        else:&#xA;            out_layer[j][0] = 0&#xA;    return out_layer &#xA;&#xA;def back_prop(l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,out_layer,lr):&#xA;    error = out_layer-l3&#xA;    grad = np.dot(error*deriv_activ_func(l3),l2.T)&#xA;    w3 = update(w3, grad, lr)&#xA;    grad = error*deriv_activ_func(l3)&#xA;    b3 = update(b3, grad, lr)&#xA;    grad = np.dot(w3.T,error*deriv_activ_func(l3))&#xA;    error = grad&#xA;    grad = error*deriv_activ_func(l2)&#xA;    w2 = update(w2, grad, lr)&#xA;    grad = error*deriv_activ_func(l2)&#xA;    b2 = update(b2, grad, lr)&#xA;    grad = np.dot(w2.T,error*deriv_activ_func(l2))&#xA;    error = grad&#xA;    grad = error*deriv_activ_func(l1)&#xA;    w1 = update(w1, grad, lr)&#xA;    grad = error*deriv_activ_func(l1)&#xA;    b1 = update(b1, grad, lr)&#xA;    return l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer&#xA;&#xA;def predict(l3):&#xA;    out = np.amax(l3)&#xA;    count = 0&#xA;    for j in range(l3.shape[0]):&#xA;        count=count+1&#xA;        if(l3[j]==out):&#xA;            break&#xA;    return count&#xA;&#xA;def trainer():&#xA;    l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,out_layer = initialize()&#xA;    for epochs in range(50):&#xA;        for i in range(gray.shape[0]):&#xA;            out_layer = np.unique(labels).reshape(-1,1)&#xA;            l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer = feed_forward(l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,i)&#xA;            out_layer = one_hot(out_layer,labels[i])&#xA;            l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer = back_prop(l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,out_layer,0.0001)&#xA;        print(&quot;End of epoch :&quot;,epochs+1) &#xA;    return l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,out_layer&#xA;&#xA;l1,l2,l3,w1,w2,w3,b1,b2,b3,in_layer,out_layer = trainer()&#xA;&#xA;path = &quot;mnist/mnist_train.csv&quot;&#xA;gray = pd.read_csv(path)&#xA;labels = gray['label']&#xA;gray = gray.drop(['label'], axis=1)&#xA;gray = to_array(gray)&#xA;labels = to_array(labels)&#xA;st_gray = np.empty(shape=(gray.shape[1],1))&#xA;&#xA;for i in range(10):&#xA;    st_gray = scaler.fit_transform(gray[i][:].reshape(-1,1))&#xA;    in_layer = st_gray&#xA;    l1 = np.dot(w1,in_layer)+b1&#xA;    l1 = activ_func(l1)&#xA;    l2 = np.dot(w2,l1)+b2&#xA;    l2 = activ_func(l2)&#xA;    l3 = np.dot(w3,l2)+b3&#xA;    l3 = activ_func(l3)&#xA;    count = predict(l3)&#xA;    print(&quot;Expected: &quot;,labels[i],&quot; Predicted: &quot;,count)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33029" LastActivityDate="2020-01-26T13:39:02.090" Title="Neural nets not learning mnist dataset" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17685" PostTypeId="2" ParentId="17681" CreationDate="2020-01-26T14:06:45.677" Score="5" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\|x\| = |x|$&lt;/span&gt; denotes the absolute value norm, which is a special case of the &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; norm defined on the 1-D vector spaces formed by real or complex numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\|\textbf{x}\|_1 = \sum_{i=1}^n|x_i|$&lt;/span&gt; denotes the Taxicab / Manhattan norm, relating to how a Taxi would drive along a rectangular grid of roads to reach a point &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$(0,0)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\|\textbf{x}\|_2 = \sqrt{x_1^2 + \dots + x_n^2}$&lt;/span&gt; denotes the Euclidean norm on an N-D Euclidean space, which is a result of the Pythagorean theorem (the shortest distance between two points).&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-01-27T10:58:59.370" LastActivityDate="2020-01-27T10:58:59.370" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17687" PostTypeId="2" ParentId="17502" CreationDate="2020-01-26T20:58:17.610" Score="0" Body="&lt;p&gt;After I've learned a little bit more about the topic, I think I figured out the exact sequence of the algorithm. So, here's my own answer. Please, correct me if I'm wrong.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Give an input, forward-propagate it, and generate an output&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each output neuron:&#xA;  for each weight connected to the neuron:&#xA;      Given the function C = f(w) (which represents the cost in function to the weight value), calculate the derivative of that function at the point where  the current weight actually is)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the actual derivative of all the weights by combining all the partial derivatives in respect to the weight: now you have a gradient of the weights&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat this process to calculate the gradient for each of the batch elements. If you have a batch size of 8, then you'll have 8 gradients.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the average gradient ((gradient_1+gradient2+gradient3...)/n_gradients)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Move the weights of that gradient&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Am I right? How does this apply to deeper layers?&lt;/p&gt;&#xA;" OwnerUserId="32751" LastEditorUserId="2444" LastEditDate="2020-01-27T11:40:02.817" LastActivityDate="2020-01-27T11:40:02.817" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17688" PostTypeId="2" ParentId="17634" CreationDate="2020-01-26T23:01:58.050" Score="3" Body="&lt;p&gt;Dimensionality reduction could be achieved by using an &lt;strong&gt;Autoencoder&lt;/strong&gt; Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could assign the encoder layer output (&lt;span class=&quot;math-container&quot;&gt;$L_i$&lt;/span&gt;) to a desired dimension (lower than that of the input). Once trained, &lt;span class=&quot;math-container&quot;&gt;$L_i$&lt;/span&gt; could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kLtGc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kLtGc.png&quot; alt=&quot;Autoencoder Architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-01-26T23:09:52.010" LastActivityDate="2020-01-26T23:09:52.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17689" PostTypeId="2" ParentId="17642" CreationDate="2020-01-27T04:52:05.610" Score="1" Body="&lt;p&gt;The trick was to normalize the input dataset values with the respective mean and standard deviation in each column. This reduced the loss drastically, and my network is training more efficiently now. Moreover, normalizing the data also helps you calculate the weights associated with each input node more easily, especially when trying to find out variable importance.&lt;/p&gt;&#xA;" OwnerUserId="32981" LastActivityDate="2020-01-27T04:52:05.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17690" PostTypeId="1" CreationDate="2020-01-27T06:04:22.033" Score="2" ViewCount="245" Body="&lt;p&gt;I have written code in OpenAI's gym to simulate a random playing in Montezuma's Revenge where the agent randomly samples actions from the action space and tries to play the game. A success for me is defined as the case when the agent is atleast able to successfully retrieve the key (Gets a reward of 100). And such cases I dump in a pickle file. I got 44 successful cases when I kept it to run for a day or so. Here is the code I use to generate the training set :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;import gym&#xA;import cma&#xA;import random&#xA;import time&#xA;import pickle as pkl&#xA;&#xA;env   = gym.make('MontezumaRevenge-ram-v0')&#xA;&#xA;observation = env.reset()&#xA;&#xA;#print(observation)&#xA;#print(env.action_space.sample())&#xA;&#xA;obs_dict    = []&#xA;action_dict = []&#xA;success_ctr = 0&#xA;&#xA;for i in range(0, 1000000):&#xA;    print('Reward for episode',i+1)&#xA;    done = False&#xA;    rew = 0&#xA;    action_list = []&#xA;    obs_list    = []&#xA;    while not done:&#xA;        action = env.action_space.sample()&#xA;        observation, reward, done, _ = env.step(action)&#xA;        action_list.append(action)&#xA;        obs_list.append(observation)&#xA;&#xA;        rew += reward&#xA;        env.render()&#xA;        time.sleep(0.01)&#xA;        if done:&#xA;            env.reset()&#xA;            if rew &amp;gt; 0:&#xA;                success_ctr += 1&#xA;                print(action_list)&#xA;                action_dict.append(action_list)&#xA;                obs_dict.append(obs_list)&#xA;                pkl.dump(obs_dict, open(&quot;obslist.pkl&quot;, &quot;wb&quot;))&#xA;                pkl.dump(action_dict, open(&quot;action.pkl&quot;, &quot;wb&quot;))&#xA;&#xA;    print(rew)&#xA;    time.sleep(1)&#xA;&#xA;try:&#xA;    print(obs_dict.shape)&#xA;except:&#xA;    pass&#xA;&#xA;print(&quot;Took key:&quot;, success_ctr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I loaded the successful cases from my generated pickle file, and simulated the agent's playing using those exact same cases. But, the agent never receives a reward of 100. I dont understand why this is happening. A little search online suggested it could be because of noise in the game. So, I gave a sleep time, before running each episode. Still, doesn't work. Can someone please explain why is this happening? And suggest a way I could go about generating the training set?&lt;/p&gt;&#xA;" OwnerUserId="32455" LastActivityDate="2020-01-27T19:15:36.687" Title="Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;rewards&gt;&lt;gym&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17692" PostTypeId="2" ParentId="12397" CreationDate="2020-01-27T09:46:41.667" Score="0" Body="&lt;p&gt;It turns out there is actually a practical reason for this.&lt;/p&gt;&#xA;&lt;p&gt;Practically speaking, in GANs the generator tends to converge on few 'good' outputs that fool the discriminator if you don't do it. In the optimal case, the generator will actually emit a single fixed output regardless of the input vector that fools the discriminator.&lt;/p&gt;&#xA;&lt;p&gt;Which is to say, the generators loss function is intended not simply as &amp;quot;fool the discriminator&amp;quot;, it is actually:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fool the discriminator.&lt;/li&gt;&#xA;&lt;li&gt;Generate novel output.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You can write your generator's loss function to explicitly attempt to say the output in any training batch should be distinct, but by passing the outputs to the discriminator you create a &lt;strong&gt;history of previous predictions&lt;/strong&gt; from the generator, effectively applying a loss metric for when the generator tends to produce the same outputs over and over again.&lt;/p&gt;&#xA;&lt;p&gt;...but it is not magic, and is &lt;em&gt;not&lt;/em&gt; about the discriminator learning &amp;quot;good features&amp;quot; features; it is about the loss applied to the &lt;strong&gt;generator&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is referred to as &amp;quot;Mode Collapse&amp;quot;, to quote the Google ML guide on GAN troubleshooting:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If the generator starts producing the same output (or a small set of outputs) over and over again, the discriminator's best strategy is to learn to always reject that output. But if the next generation of discriminator gets stuck in a local minimum and doesn't find the best strategy, then it's too easy for the next generator iteration to find the most plausible output for the current discriminator.&lt;/p&gt;&#xA;&lt;p&gt;Each iteration of generator over-optimizes for a particular discriminator, and the discriminator never manages to learn its way out of the trap. As a result the generators rotate through a small set of output types. This form of GAN failure is called mode collapse.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;See also, for additional reading &amp;quot;Unrolled GANs&amp;quot; and &amp;quot;Wasserstein loss&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;see: &lt;a href=&quot;https://developers.google.com/machine-learning/gan/problems&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developers.google.com/machine-learning/gan/problems&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25722" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-01-27T09:46:41.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17693" PostTypeId="1" CreationDate="2020-01-27T09:52:20.930" Score="2" ViewCount="36" Body="&lt;p&gt;I have a dataset containing timestamp and temperature. For each day, I have 1440 values viz., I have data for every minute of that day(60minutes * 24hrs = 1440). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Dataset looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mXJKh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mXJKh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an initial step, I gathered day1 data to predict day2 data. I have tried AR, ARIMA, SARIMAX models but I didn't find any positive results. I think this is multivariate since the time and the temperature values changes with respect to date. I need guidance to choose the ML model that will suit for my dataset and it should be able to predict next day/ next month&lt;/p&gt;&#xA;" OwnerUserId="27539" LastEditorUserId="27539" LastEditDate="2020-01-27T12:12:22.743" LastActivityDate="2020-01-27T12:12:22.743" Title="Predicting a day's data" Tags="&lt;prediction&gt;&lt;data-science&gt;&lt;regression&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17695" PostTypeId="1" AcceptedAnswerId="17700" CreationDate="2020-01-27T11:20:44.980" Score="2" ViewCount="157" Body="&lt;p&gt;I've just started with CNN and there is something that I haven't understood yet:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you &quot;ask&quot; a network: &quot;classify me these images&quot; or &quot;do semantic segmentation&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it must be something on the architecture, or in the loss function, or whatever that makes the network classify its input or do semantic segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose its output will be different on classification and on semantic segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the question could be rewritten to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do I have to do to use a CNN for classification or for semantic segmentation?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2020-01-27T14:56:41.820" LastActivityDate="2020-01-27T14:56:41.820" Title="What make a CNN suitable for image classification or for semantic segmentation?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;theory&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17696" PostTypeId="2" ParentId="17693" CreationDate="2020-01-27T11:35:50.267" Score="1" Body="&lt;p&gt;Just for clarification: your description (1 sample per minute) does not match the example data (far fewer data points which is understandable, but also two data points in one minute which contradicts the initial assertion.) If your actual measurements are like that you should first work on the sampling process to get reliable data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For creating predictions, you need to have a reasonable model of the observed process. If you're measuring environmental temperatures, you will basically have three causes of variation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A day/night cycle&lt;/li&gt;&#xA;&lt;li&gt;A seasonal (summer/winter) cycle&lt;/li&gt;&#xA;&lt;li&gt;Local weather fluctuation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From only one day of samples, the only thing you can reasonably predict is that the next day will look mostly the same. If you collect more data over a year, you will be able to extract a seasonal cycle and estimate the deviations caused by local weather. &quot;You&quot; means either you as a researcher or any machine learning system that you program. Without sufficient historical data it is impossible to make good predictions (and even with sufficient data it's hard.)&lt;/p&gt;&#xA;" OwnerUserId="22993" LastActivityDate="2020-01-27T11:35:50.267" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17697" PostTypeId="1" AcceptedAnswerId="17706" CreationDate="2020-01-27T11:37:17.533" Score="0" ViewCount="63" Body="&lt;p&gt;In the snippet below, the highlighted part is the average norm, but since &lt;span class=&quot;math-container&quot;&gt;$1/|p_i|$&lt;/span&gt; is outside the summation, it is very confusing to understand.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;is &lt;span class=&quot;math-container&quot;&gt;$|p_i|$&lt;/span&gt; l2-norm(as per wolfram) or l1-norm or absolute value as per wiki.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; inside summation be considered for &lt;span class=&quot;math-container&quot;&gt;$1/|p_i|$&lt;/span&gt;, which is outside the summation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RtTjF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RtTjF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2020-01-27T17:50:13.403" LastActivityDate="2020-01-27T17:50:13.403" Title="How to understand the average l2 loss?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;loss-functions&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17698" PostTypeId="2" ParentId="9746" CreationDate="2020-01-27T11:47:31.900" Score="-1" Body="&lt;p&gt;Writing a game AI is difficult in general and it's even harder when there's no game to play. Like in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While &quot;Lines&quot; is pretty neat, there's no (playing) opponent and there are no moves. apart from the setup. So what we have is an optimization problem where we need to get more point than the &quot;opponents&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Game playing algorithms like Reinforcement learning or MCTS are IMHO out of questions as there's no game to play. Still, it's an interesting problem, though I can't see how to make any &lt;em&gt;learning&lt;/em&gt; AI. Anyway, what you need first is a simulator and an optimizer choosing promising actions.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Some ideas concerning the actions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Some actions are trivial, e.g., the &lt;strong&gt;eraser&lt;/strong&gt; as there are just a few dots to erase. Trying all combination should be good enough for the start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;scissors&lt;/strong&gt; can be in theory placed anywhere, but the only interesting places are just next to the oponent piece or a crossing. Additionally, you only need to consider crossings which gets &quot;won&quot; (reached first) by an opponent. Moreover, they only matter if they get won by an opponent having more points than you or blocking you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;dots&lt;/strong&gt; to insert should similarly by placed near an oponent or so that you just win a crossing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The additional &lt;strong&gt;lines&lt;/strong&gt; are worst, as they may be used to let you reach unconnected areas, or reach a crossing faster or simply give you additional points (then you want a long line somewhere).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A model&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Most of the game can be seen as a non-directed graph with edges labeled by their length, where the crossing and the initial dots are nodes. The simulator doesn't need to work &quot;pixel by pixel&quot;, but may instead jump to the next time, when a crossing gets reached or two &quot;flows&quot; meat each other. The points of a player grow linearly with time, where the slope is given by the number of their extending &quot;flows&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I could imagine modelling it as an Integer Linear Program, but with the additional &lt;strong&gt;lines&lt;/strong&gt;, it's probably no more possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd probably let the optimization be simulator-guided, i.e., analyse whether I could get closer to winning by reaching a crossing sooner. The details are still unclear....&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2020-01-27T11:47:31.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17699" PostTypeId="2" ParentId="17690" CreationDate="2020-01-27T11:48:02.457" Score="2" Body="&lt;p&gt;What's exactly the point of &lt;code&gt;time.sleep()&lt;/code&gt; in this code? I don't really understand it, you're simply stopping the execution of the program for &lt;span class=&quot;math-container&quot;&gt;$0.01$&lt;/span&gt; seconds, how will that affect the simulator in any way ? It's not running in parallel, it does one step of the simulation when you call &lt;code&gt;env.step&lt;/code&gt; function and returns the next state and reward. Calling sleep function only slows down the program here.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for the failure of successful trajectories, when repeated, is probably because the environment isn't stationary. That means that there are enemies or obstacles moving in the environment. If you simply repeat the trajectory that was successful once the enemies and obstacles might be in different positions and the agent will die. The reason why it succeeded the first time is because the agent got lucky. These are still valid learning trajectories because they were successful. The agent should learn reasons (features) why those trajectories were successful but not learn the trajectories themselves because, as you saw, they don't generalize well. If you plan on doing some kind of supervised learning approach you should also generate variety of unsuccessful trajectories so that the agent can learn what are correct and what are not correct actions depending on the current state in the environment.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-01-27T11:48:02.457" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17700" PostTypeId="2" ParentId="17695" CreationDate="2020-01-27T11:52:43.247" Score="2" Body="&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: This question is &lt;em&gt;very&lt;/em&gt; broad, my answer is admittedly partial and is intended to just give an idea of what's out there and how to find out more.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do you &quot;say&quot; a network: &quot;classify me these images&quot; or &quot;do semantic segmentation&quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You're mixing two very different problems there. Although &lt;a href=&quot;https://paperswithcode.com/area/computer-vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;there are SO many variations of problems people are applying CNNs to&lt;/a&gt;, for this example we can focus on the &quot;classification of something in the image&quot; subset ad identify 4 key tasks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Image Classification&lt;/strong&gt; answers the question &quot;What is this image about&quot; (the answer is an image where each pixel is assigned to one of the given classes)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Semantic Segmentation&lt;/strong&gt; answer he question &quot;What areas of this image are part of a Cat?&quot; (e.g., &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Object Detection&lt;/strong&gt; answers the question &quot;Where are the objects in the image AND what objects are they?&quot; (e.g. of answer: &quot;Cat in bounding box at &lt;code&gt;x,y,w,h&lt;/code&gt; &lt;code&gt;[10,20,50,60]&lt;/code&gt;&quot;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Semantic Segmentation&lt;/strong&gt; answers the question &quot;Where are the individual objects in this image AND what class are they AND give me the pixels that belong to each object&quot;. You may guess from the number of ANDs there, this is the hardest of the four. The output here would be a set of &lt;code&gt;class, bounding_box, mask&lt;/code&gt; tuples where the mask is typically defined in relation to the returned bounding box.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, how do we build networks capable of solving one problem or the other?&lt;/strong&gt;&#xA;We build architectures &lt;em&gt;towards one specific problem&lt;/em&gt;, exploiting reusable parts where possible. For example, typically classification and object detection are based on a deep &quot;backbone&quot; that extracts highly complex features from the image, that finally are used by a classifier layer interprets to make a prediction (for image classification) or a box prediction head to predict where objects lie in the image (&lt;em&gt;very big&lt;/em&gt; simplification, look up object detection architectures and how they work for the proper description!). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What do I have to do to use a CNN for classification or for semantic segmentation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In principle you can't just take a network built for classification and just &quot;ask&quot; it to do semantic segmentation (think of it as trying to use a screwdriver as scissors... it just was not built for that!). You need changes in the architecture, which necessarily imply new training, at the very least for the new parts that were added.&lt;/p&gt;&#xA;" OwnerUserId="22086" LastActivityDate="2020-01-27T11:52:43.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17701" PostTypeId="1" CreationDate="2020-01-27T12:19:58.037" Score="2" ViewCount="137" Body="&lt;p&gt;What are the advantages and disadvantages of using meta-heuristic algorithms on optimization problems? Simply, why do we use meta-heuristic algorithms, like PSO, over traditional mathematical techniques, such as linear, non-linear and dynamic programming?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I actually have a good understanding of meta-heuristic algorithms and I know how they work. For example, one advantage of this kind of algorithms is that they can find an optimal solution in a reasonable time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my lack of knowledge about other methods and techniques brought this question to my mind.&lt;/p&gt;&#xA;" OwnerUserId="33046" LastEditorUserId="2444" LastEditDate="2020-01-27T16:14:19.420" LastActivityDate="2020-01-27T16:14:19.420" Title="What are advantages of using meta-heuristic algorithms on optimization problems?" Tags="&lt;comparison&gt;&lt;optimization&gt;&lt;applications&gt;&lt;meta-heuristics&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17702" PostTypeId="5" CreationDate="2020-01-27T12:48:41.503" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-27T12:48:41.503" LastActivityDate="2020-01-27T12:48:41.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17703" PostTypeId="4" CreationDate="2020-01-27T12:48:41.503" Score="0" Body="For questions related to meta-heuristics (e.g. simulated annealing, tabu search, ant colony optimization algorithms, etc)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-27T12:48:41.503" LastActivityDate="2020-01-27T12:48:41.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17704" PostTypeId="1" CreationDate="2020-01-27T12:50:28.973" Score="0" ViewCount="66" Body="&lt;p&gt;I've got a few thousands of sequences like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.23, 2.15. 3.19, 4.30, 5.24, 6.22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the numbers denote times on which an event happened (there's just a single kind of events). The events are sort of periodical and the period is &lt;em&gt;known to be exactly&lt;/em&gt; one, however, the  exact times varies. Sometimes, events are missing and there are other irregularities, but let's ignore them for now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to train an neural network for predicting the probability that there'll be a next even in a given time interval. The problem is that I have no probabilities for the training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I have are the above sequences. If I had four sequences like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.23, 2.15. 3.19, 4.30, 5.24, 6.05&#xA;1.23, 2.15. 3.19, 4.30, 5.24, 6.83&#xA;1.23, 2.15. 3.19, 4.30, 5.24, 6.27&#xA;1.23, 2.15. 3.19, 4.30, 5.24, 6.22&#xA;1.23, 2.15. 3.19, 4.30, 5.24, 6.17&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then I could say that the probability of an event in the interval &lt;code&gt;[6.10, 6.30]&lt;/code&gt; is 60% and use this value for learning. However, all my sequences are different. I could try to group them somehow so that I can define something like a probability, but this sounds way more complicated than what I'm trying to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, I could try to use the sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.23, 2.15. 3.19, 4.30, 5.24, 6.22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to learn that after the prefix &lt;code&gt;1.23, 2.15. 3.19, 4.30, 5.24&lt;/code&gt;, there will be an event in the interval &lt;code&gt;[6.10, 6.30]&lt;/code&gt; for sure (value to learn equal to one); if there was 6.05 instead of 6.22, the value to learn would be zero. A learned network would produce the average value (let's say 0.60).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the error would never converge to zero, so there'd be no quality criterion and probably a big chance of overtraining leading to non-sense results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to handle this?&lt;/p&gt;&#xA;" OwnerUserId="12053" LastActivityDate="2021-02-20T16:07:10.720" Title="Predicting probabilities of events using neural networks" Tags="&lt;neural-networks&gt;&lt;prediction&gt;&lt;probability&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17705" PostTypeId="2" ParentId="17701" CreationDate="2020-01-27T13:23:51.560" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.iiia.csic.es/~christian.blum/downloads/blum_roli_2003.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Meta-heuristics&lt;/a&gt; are particularly suited for combinatorial optimization problems, given that, although they are not usually guaranteed to find the optimal global solution, they can often find a sufficiently good solution in a decent amount of time. So, they are an alternative to exhaustive search, which would take exponential time. For example, ant colony optimization algorithms have been used to approximately (or exactly, in the case of small or medium-size instances) solve the travelling salesman problem, whose decision version is an NP-complete problem (which means that, unless P=NP, there is no polynomial-time solution to solve it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meta-heuristics can also be easily applied to many problems, given that they are not problem-specific. For example, in the case of genetic algorithms, you just need to encode the possible solutions, but, in principle, you can apply genetic algorithms to a wide range of problems, although they may not always be the best solution to each of these problems. Moreover, as opposed to gradient-based optimisation algorithms, there's no need for the gradient of the objective function. For instance, in the case of genetic algorithms, you just need a way of evaluating the solutions (e.g. the fitness or the novelty).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meta-heuristics often incorporate some form of randomness in order to escape from local minima. Ant-colony optimization algorithms or simulated annealing are two good examples of this approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are still interested in meta-heuristics, the book &lt;a href=&quot;http://cleveralgorithms.com/nature-inspired/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clever Algorithms: Nature-Inspired Programming Recipes&lt;/a&gt; (by Jason Brownlee) is a very good resource for learning about them. There's also a &lt;a href=&quot;https://github.com/jbrownlee/CleverAlgorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repository&lt;/a&gt; with the implementation of the algorithms described in this book.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-27T13:44:55.737" LastActivityDate="2020-01-27T13:44:55.737" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17706" PostTypeId="2" ParentId="17697" CreationDate="2020-01-27T13:48:37.307" Score="1" Body="&lt;p&gt;I agree that this notation is unclear. I would interpret it as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the expression is supposed to denote the &lt;strong&gt;average norm&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$|p_i|$&lt;/span&gt; is likely the cardinality of the set &lt;span class=&quot;math-container&quot;&gt;$\{p_i\}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case the expression would just be the sum over all norms divided by the number of norms, resulting in the average norm. The authors likely use this notation because they didn't want to introduce the number &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$p_{i &amp;lt; n}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$|\{p_i\}|$&lt;/span&gt; would be clearer, but maybe uglier to typeset. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastEditorUserId="2227" LastEditDate="2020-01-27T13:57:36.367" LastActivityDate="2020-01-27T13:57:36.367" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17707" PostTypeId="2" ParentId="17704" CreationDate="2020-01-27T14:07:54.673" Score="0" Body="&lt;p&gt;You can set up a neural network to predict whether there is an event in a randomly picked interval. I.e. if there is an event in this interval in your trainingsdata you train to output a 1 otherwise you train to output a 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use the quadratic loss function the prediction of the NN should approximate the probability of such an event. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overfitting can be monitored with by splitting your trainingsdata into train- and testset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you train an RNN by inputting the intervals between events, these intervals should be more similar than the exact event times. Modelling a time series like this also makes more sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course these details depend on what exactly the data is representing. And it is also possible that decent predictions are impossible for this dataset. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2020-01-27T14:07:54.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17708" PostTypeId="2" ParentId="17588" CreationDate="2020-01-27T16:39:00.490" Score="0" Body="&lt;p&gt;Since you are mainly interested in starter libraries and packages to read up on here are some pointers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image recognition as well as general spatial classification mainly consists of two major tasks.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Translating images to data that can be fed into a ML model of any kind.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Building a model that is able to use this data to complete tasks like classification (which lego figure is this?), spatial clustering (which pixels form the &quot;face&quot;), etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the first task you will find a lot of python libraries that suit the task but `` PIL/Pillow``` should be the main library for this. So start by reading some documentation and transferring the images you do have (this includes standardizing and cleaning the images to improve the results of step 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second task the actual model you will use depends on your task but generally speaking some form of neural net is a good place to start and if you prefer python then you should look into &lt;code&gt;tensorflow&lt;/code&gt; and &lt;code&gt;keras&lt;/code&gt; (easy to use interface to tensorflow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where to start&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start with the classical MNIST numbers recognition case and if you have grokked that, you will now where to go from there or at least be able to ask a more specific question here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pillow.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pillow.readthedocs.io/en/stable/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/image-classification-in-10-minutes-with-mnist-dataset-54c35b77a38d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/image-classification-in-10-minutes-with-mnist-dataset-54c35b77a38d&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2020-01-27T16:39:00.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17710" PostTypeId="1" CreationDate="2020-01-27T17:25:15.727" Score="1" ViewCount="22" Body="&lt;p&gt;Consider the function &lt;span class=&quot;math-container&quot;&gt;$f(x)=\tan(2x)$&lt;/span&gt;. How can I determine a fuzzy system that approximates &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;? How to choose membership functions and how to determine fuzzy rules? Any help would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="33051" LastEditorUserId="2444" LastEditDate="2020-01-28T00:57:37.870" LastActivityDate="2020-01-28T00:57:37.870" Title="How can I formulate a fuzzy inference system to approximate the tangent function?" Tags="&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17711" PostTypeId="1" CreationDate="2020-01-27T17:57:39.890" Score="1" ViewCount="40" Body="&lt;p&gt;I am building a 2d top-down space game, which involves several objects, such as asteroids, drones, spaceships, space litter and power-ups.&#xA;It follows the rules of space gravity, with speed and acceleration.&#xA;The idea is that a player controls his own spaceship, can fire bullets, and is allowed to spawn 3 support drones that will attack the enemy. Its goal is to deal the most damage to the opponent spaceship in 3 minutes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It involves certain dynamics, such as &quot;Destroying an asteroid spawns a power-up&quot;, &quot;touching an asteroid deals damage to your spaceship&quot; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best approach to define an AI agent for it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of Reinforcement learning, but maybe the game is too complex and I wouldn't have the computational power for it?&lt;/p&gt;&#xA;" OwnerUserId="33053" LastActivityDate="2020-01-27T17:57:39.890" Title="Best AI Approach for 2D to-down space shooter" Tags="&lt;ai-design&gt;&lt;game-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17712" PostTypeId="2" ParentId="17690" CreationDate="2020-01-27T19:15:36.687" Score="2" Body="&lt;p&gt;Since the environment has some randomness in it, purely memorizing a trajectory to victory will not work. You will have to memorize every single trajectory for that to work, and there are an infinite number of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you will need to add some sort of bias to your learning model - i.e., what to do when the observations in your pickle file don't match the current observation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your current setup lends itself well to a case-based reasoning (CBR) approach. The idea of CBR is that you have a memory bank of observation-action pairs and when you see a new observation you look up the memory bank and see if the current observation has been seen before. If so, do that action. The interesting part is when there are no observations that match directly, but there are some that are similar. In this case you choose the most similar. The similarity can be calculated in any number of ways, and it is dependant on the data types. This paper is will provide a good start: &lt;a href=&quot;https://alumni.media.mit.edu/~jorkin/generals/papers/Kolodner_case_based_reasoning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://alumni.media.mit.edu/~jorkin/generals/papers/Kolodner_case_based_reasoning.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33054" LastActivityDate="2020-01-27T19:15:36.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17713" PostTypeId="2" ParentId="16905" CreationDate="2020-01-27T21:02:35.613" Score="3" Body="&lt;p&gt;By far the most commonly used strategy is to select the child with the &lt;strong&gt;highest number of visits&lt;/strong&gt;. This is as described in the 2008 paper you linked. It's also what's referred to as the &quot;robust child&quot; in the 2012 paper you linked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In algorithm 2 of the 2012 paper, they actually use the highest average reward, which corresponds to &quot;Max child&quot;. It looks like they're using the UCB1 policy, but they actually use a value of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for the exploration parameter &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, which makes the entire square root term drop out. This is also explained in the text at the end of your quote. But usually, a robust child / max visit count performs better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/23751563_Progressive_Strategies_for_Monte-Carlo_Tree_Search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Progressive Strategies for Monte-Carlo Tree Search&lt;/a&gt; is a different paper from 2008, in which these strategies are experimented with a bit. Usually, they perform similarly, but a robust child tends to be the best if there is any difference at all.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2020-01-28T13:35:18.367" LastActivityDate="2020-01-28T13:35:18.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17714" PostTypeId="1" CreationDate="2020-01-27T21:26:32.430" Score="1" ViewCount="104" Body="&lt;p&gt;I have a neural network model defined as below. How many layers exist there? Not sure which ones to count when we are asked about the number.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_model():&#xA;    channels = 3&#xA;    model = Sequential()&#xA;    model.add(Conv2D(32, kernel_size = (5, 5), activation='relu', input_shape=(IMAGE_SIZE, IMAGE_SIZE, channels)))&#xA;    model.add(MaxPooling2D(pool_size=(2,2)))&#xA;    model.add(BatchNormalization())&#xA;    model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2,2)))&#xA;    model.add(BatchNormalization())&#xA;    model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2,2)))&#xA;    model.add(BatchNormalization())&#xA;    model.add(Conv2D(256, kernel_size=(3,3), activation='relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2,2)))&#xA;    model.add(BatchNormalization())&#xA;&#xA;    model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))&#xA;    model.add(MaxPooling2D(pool_size=(2,2)))&#xA;    model.add(BatchNormalization())&#xA;    model.add(Dropout(0.2))&#xA;    model.add(Flatten())&#xA;    model.add(Dense(256, activation='relu'))&#xA;    model.add(Dropout(0.2))&#xA;    model.add(Dense(128, activation='relu'))&#xA;    model.add(Dense(2, activation = 'softmax'))&#xA;&#xA;    return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2020-01-28T14:53:58.527" LastActivityDate="2020-01-28T14:53:58.527" Title="How many layers exists in my neural network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;training&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17715" PostTypeId="1" AcceptedAnswerId="17719" CreationDate="2020-01-28T01:20:46.847" Score="1" ViewCount="68" Body="&lt;p&gt;Not sure where to put this... I am trying to create a convolutional architecture for a DQN in keras, and I want to know why my param count is so high for my last layer compared to the rest of the network. I've tried slowly decreasing the dimensions of the layers above it, but it performs quite poorly. I want to know if there's anything I can do to decrease the param count of that last layer, besides the above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Import statements.&#xA;import random&#xA;import numpy as np&#xA;import tensorflow as tf&#xA;import tensorflow.keras.layers as L&#xA;from collections import deque&#xA;import layers as mL&#xA;import tensorflow.keras.optimizers as O&#xA;import optimizers as mO&#xA;import tensorflow.keras.backend as K&#xA;&#xA;&#xA;#Conv function.&#xA;def conv(x, units, kernel, stride, noise=False, padding='valid'):&#xA;    y = L.Conv2D(units, kernel, stride, activation=mish, padding=padding)(x)&#xA;    if noise:&#xA;        y = mL.PGaussian()(y)&#xA;    return y&#xA;&#xA;#Network&#xA;        x_input = L.Input(shape=self.state)&#xA;        x_goal = L.Input(shape=self.state)&#xA;        x = L.Concatenate(-1)([x_input, x_goal])&#xA;        x_list = []&#xA;        for i in range(2):&#xA;            x = conv(x, 4, (7,7), 1)&#xA;        for i in range(2):&#xA;            x = conv(x, 8, (5,5), 2)&#xA;        for i in range(10):&#xA;            x = conv(x, 6, (3,3), 1, noise=True)&#xA;        x = L.Conv2D(1, (3,3), 1)(x)&#xA;        x_shape = K.int_shape(x)&#xA;        x = L.Reshape((x_shape[1], x_shape[2]))(x)&#xA;        x = L.Flatten()(x)&#xA;        crit = L.Dense(1, trainable=False)(x)&#xA;        critic = tf.keras.models.Model([x_input, x_goal], crit)&#xA;        act1 = L.Dense(self.action, trainable=False)(x)&#xA;        act2 = L.Dense(self.action2, trainable=False)(x)&#xA;        act1 = L.Softmax()(act1)&#xA;        act2 = L.Softmax()(act2)&#xA;        actor = tf.keras.models.Model([x_input, x_goal], [act1, act2])&#xA;        actor.compile(loss=mish_loss, optimizer='adam')&#xA;        actor.summary()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;actor.summary():&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;________________________________________________________________&#xA;Layer (type)                    Output Shape         Param #     Connected to                     &#xA;==================================================================================================&#xA;input_2 (InputLayer)            [(None, 300, 300, 3) 0                                            &#xA;__________________________________________________________________________________________________&#xA;input_3 (InputLayer)            [(None, 300, 300, 3) 0                                            &#xA;__________________________________________________________________________________________________&#xA;concatenate (Concatenate)       (None, 300, 300, 6)  0           input_2[0][0]                    &#xA;                                                                 input_3[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;conv2d_52 (Conv2D)              (None, 294, 294, 4)  1180        concatenate[0][0]                &#xA;__________________________________________________________________________________________________&#xA;conv2d_53 (Conv2D)              (None, 288, 288, 4)  788         conv2d_52[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_54 (Conv2D)              (None, 142, 142, 8)  808         conv2d_53[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_55 (Conv2D)              (None, 69, 69, 8)    1608        conv2d_54[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_56 (Conv2D)              (None, 67, 67, 6)    438         conv2d_55[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;p_gaussian (PGaussian)          (None, 67, 67, 6)    1           conv2d_56[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_57 (Conv2D)              (None, 65, 65, 6)    330         p_gaussian[0][0]                 &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_1 (PGaussian)        (None, 65, 65, 6)    1           conv2d_57[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_58 (Conv2D)              (None, 63, 63, 6)    330         p_gaussian_1[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_2 (PGaussian)        (None, 63, 63, 6)    1           conv2d_58[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_59 (Conv2D)              (None, 61, 61, 6)    330         p_gaussian_2[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_3 (PGaussian)        (None, 61, 61, 6)    1           conv2d_59[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_60 (Conv2D)              (None, 59, 59, 6)    330         p_gaussian_3[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_4 (PGaussian)        (None, 59, 59, 6)    1           conv2d_60[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_61 (Conv2D)              (None, 57, 57, 6)    330         p_gaussian_4[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_5 (PGaussian)        (None, 57, 57, 6)    1           conv2d_61[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_62 (Conv2D)              (None, 55, 55, 6)    330         p_gaussian_5[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_6 (PGaussian)        (None, 55, 55, 6)    1           conv2d_62[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_63 (Conv2D)              (None, 53, 53, 6)    330         p_gaussian_6[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_7 (PGaussian)        (None, 53, 53, 6)    1           conv2d_63[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_64 (Conv2D)              (None, 51, 51, 6)    330         p_gaussian_7[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_8 (PGaussian)        (None, 51, 51, 6)    1           conv2d_64[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_65 (Conv2D)              (None, 49, 49, 6)    330         p_gaussian_8[0][0]               &#xA;__________________________________________________________________________________________________&#xA;p_gaussian_9 (PGaussian)        (None, 49, 49, 6)    1           conv2d_65[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_66 (Conv2D)              (None, 47, 47, 1)    55          p_gaussian_9[0][0]               &#xA;__________________________________________________________________________________________________&#xA;reshape (Reshape)               (None, 47, 47)       0           conv2d_66[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;flatten (Flatten)               (None, 2209)         0           reshape[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;dense_1 (Dense)                 (None, 2000)         4420000     flatten[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;dense_2 (Dense)                 (None, 200)          442000      flatten[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;softmax (Softmax)               (None, 2000)         0           dense_1[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;softmax_1 (Softmax)             (None, 200)          0           dense_2[0][0]                    &#xA;==================================================================================================&#xA;Total params: 4,869,857&#xA;Trainable params: 7,857&#xA;Non-trainable params: 4,862,000&#xA;__________________________________________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33058" LastEditorUserId="33058" LastEditDate="2020-01-28T04:42:17.130" LastActivityDate="2020-01-28T13:50:55.170" Title="Param count in last layer high, how can I decrease?" Tags="&lt;convolutional-neural-networks&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17716" PostTypeId="2" ParentId="5115" CreationDate="2020-01-28T05:49:16.843" Score="-1" Body="&lt;p&gt;It is wrong to assume that just the connections to other words define their meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Give an AI a hundred novels and it would still not know what the word &quot;cat&quot; means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Show the AI a picture of a cat with the word &quot;cat&quot; underneath it and it would know straight away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way an AI needs to know a minimum number of words through experience other than combinations of other words. From then it may be able to deduce meanings of new words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just like, if I gave you a hundred novels in Chinese you would never be able to understand Chinese. I show you a picture book in Chinese and maybe you have a chance.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-01-28T05:49:16.843" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="17718" PostTypeId="2" ParentId="17714" CreationDate="2020-01-28T13:07:19.627" Score="3" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; I'd say your model has &lt;strong&gt;8 layers&lt;/strong&gt; (5 conv, 3 dense), however a lot of people count layers in other ways. From what I've seen this is by far the most &lt;strong&gt;conventional&lt;/strong&gt; way for counting layers.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Justification&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is an interesting question because its quite subjective. In most cases only the convolutional and dense layers would count from your network. Bach norm, dropout and flatten are usually considered as operations to other layers rather than layers of their own (much like activation functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: It is debatable whether or not pooling layers are considered to be layers (as they have no trainable parameters) but in most cases they are not considered to be so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 2: Batch norm, on the other hand, isn't usually considered a layer even though it has trainable parameters. Clearly the authors didn't &lt;a href=&quot;https://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;introduce it as a layer&lt;/a&gt;, but as a way to normalize, shift and scale the inputs of a layer. This is apparent in some of the examples below which don't count batch norm as an actual layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 3: Conventionally all networks are considered to have [at least] one input layer but it doesn't count as a layer.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Examples&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Some examples that follow this reasoning when counting layers are the following. I'll also write the pooling layers in each, but they clearly aren't considered as layers be the authors. When available I'll also write the number of layers that keras registers from their &lt;a href=&quot;https://keras.io/applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;official implementations&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fcv-tricks.com%2Fwp-content%2Fuploads%2F2019%2F07%2FResNet50_architecture-1.png&amp;amp;f=1&amp;amp;nofb=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNet-50&lt;/a&gt; architecture has 50 layers (49 conv, 2 pool, 1 dense), however keras registers it as 177 layers. ResNets also &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;use batch normalization after each convolution&lt;/a&gt; (so 49 batch norms in total), but clearly don't count them as layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fcdn-images-1.medium.com%2Fmax%2F1600%2F1*4tlPOipWjcwIoNUlQ6IWFQ.png&amp;amp;f=1&amp;amp;nofb=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Resnet-34&lt;/a&gt; has 34 layers (33 conv, 2 pool, 1 dense). Like the previous, this also uses batch norm but doesn't count it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fcdn-images-1.medium.com%2Fmax%2F1600%2F1*4tlPOipWjcwIoNUlQ6IWFQ.png&amp;amp;f=1&amp;amp;nofb=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG-19&lt;/a&gt; has 19 layers (16 conv, 5 pool 3 dense). Keras registers this as 26 layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AlexNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlexNet&lt;/a&gt; is considered to have 8 layers (5 conv, 3 pool, 3 dense).&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-01-28T13:07:19.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17719" PostTypeId="2" ParentId="17715" CreationDate="2020-01-28T13:50:55.170" Score="1" Body="&lt;p&gt;If I understood correctly you want to decrease the parameters count on the last layer (&lt;code&gt;dense_2&lt;/code&gt; layer right?). It would be nice to know why you want to decrease the number of parameters in the last layers... But I'll proceed with what I see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, the dense layers (or fully connected in literature) have a deterministic number of parameters (or weights) to learn according to the size of the input and output tensor. The relation is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$N_{params} = Y_{output} \cdot (X_{input} +1) = 200 \cdot (2209 +1) = 442000$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Y_{output}$&lt;/span&gt;: Output tensor shape (&lt;code&gt;act2=200&lt;/code&gt; which is the output of &lt;code&gt;dense_2&lt;/code&gt; layer)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$X_{input}$&lt;/span&gt;: Input tensor shape (&lt;code&gt;x=2209&lt;/code&gt;which is the ouput of &lt;code&gt;flatten&lt;/code&gt; layer)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So if you want to decrease the number of parameters you can:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decrease the input tensor: your &lt;code&gt;self.action2&lt;/code&gt; which I guess is the action space so you might not be able to decrease it&lt;/li&gt;&#xA;&lt;li&gt;Decrease the output tensor: maybe? I would need more context (code) to know if that is even possible&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, in short: if you are not willing to change your the input or output tensor to the Dense layers, then no, you can not decrease the number of parameters.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BONUS&lt;/strong&gt;: In case you missed it, I have noticed you have set your dense layers to &lt;code&gt;trainable=False&lt;/code&gt;. So in principle you should not care about decreasing number of parameters (which in most cases is motivated to reduce training time) since they are already not being trained. You can check that in the Keras summary output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Total params: 4,869,857&#xA;Trainable params: 7,857&#xA;Non-trainable params: 4,862,000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the non-trainable parameters are &lt;span class=&quot;math-container&quot;&gt;$4862000 = 4420000 + 442000 $&lt;/span&gt;, which are the number of parameters of your 2 dense layers.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-01-28T13:50:55.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17721" PostTypeId="1" AcceptedAnswerId="17722" CreationDate="2020-01-28T16:16:23.740" Score="34" ViewCount="9567" Body="&lt;p&gt;I trained a simple CNN on the MNIST database of handwritten digits to 99% accuracy. I'm feeding in a bunch of handwritten digits, and non-digits from a document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the CNN to report errors, so I set a threshold of 90% certainty below which my algorithm assumes that what it's looking at is not a digit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that the CNN is 100% certain of many incorrect guesses. In the example below, the CNN reports 100% certainty that it's a 0. How do I make it report failure?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nPBYU.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nPBYU.png&quot; alt=&quot;Handwritten digit input - does not look like a digit&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My thoughts on this&lt;/strong&gt;:&#xA;Maybe the CNN is not really 100% certain that this is a zero. Maybe it just thinks that it can't be anything else, and it's being forced to choose (because of normalisation on the output vector). Is there any way I can get insight into what the CNN &quot;thought&quot; before I forced it to choose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I'm using Keras on Tensorflow with Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because someone asked. Here is the context of my problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This came from me applying a heuristic algorithm for segmentation of sequences of connected digits. In the image above, the left part is actually a 4, and the right is the curve bit of a 2 without the base. The algorithm is supposed to step through segment cuts, and when it finds a confident match, remove that cut and continue moving along the sequence. It works really well for some cases, but of course it's totally reliant on being able to tell if what it's looking at is not a good match for a digit. Here's an example of where it kind of did okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rVDz4.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rVDz4.png&quot; alt=&quot;Various input images for handwritten digit recognition&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My next best option is to do inference on all permutations and maximise combined score. That's more expensive.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="2255" LastEditDate="2020-05-26T17:14:52.597" LastActivityDate="2020-07-22T17:45:43.840" Title="Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;bayesian-deep-learning&gt;" AnswerCount="6" CommentCount="5" FavoriteCount="21" ContentLicense="CC BY-SA 4.0" />
  <row Id="17722" PostTypeId="2" ParentId="17721" CreationDate="2020-01-28T16:40:30.243" Score="33" Body="&lt;p&gt;The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (BNN). This kind of network learns a distribution of weights instead of scalar or point-wise weights, which then allow to encode model uncertainty, as then the distribution of the output is calibrated and has the properties you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is also called out of distribution (OOD) detection, and again it can be done with BNNs, but unfortunately training a full BNN is untractable, so we use approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a reference, one of these approximations is Deep Ensembles, which train several instances of a model in the same dataset and then average the softmax probabilities, and has good out of distribution detection properties. Check the paper &lt;a href=&quot;https://arxiv.org/abs/1612.01474&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, in particular section 3.5 which shows results for OOD based on entropy of the ensemble probabilities.&lt;/p&gt;&#xA;" OwnerUserId="31632" LastEditorUserId="33085" LastEditDate="2020-01-29T06:57:57.360" LastActivityDate="2020-01-29T06:57:57.360" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17723" PostTypeId="2" ParentId="9322" CreationDate="2020-01-28T17:30:43.397" Score="2" Body="&lt;p&gt;(Old question, I know...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not that we &lt;em&gt;need&lt;/em&gt; both an encoder and decoder for sequence-to-sequence models - this decoupling of &quot;reading&quot; and &quot;generating&quot; just works better very often.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example for Sequence-to-sequence without two RNNs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prove my point above, here is an example from machine translation. Current machine translation systems are sequence-to-sequence models, and virtually all models have the bipartite structure of encoder and decoder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approaches like &lt;a href=&quot;https://arxiv.org/abs/1810.13409&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eager Translation&lt;/a&gt; break this implied convention. They learn translation models that do not encode and decode with separate RNNs, but at every time step 1) read a source token and 2) produce a target token - with a single RNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why encoder-decoder works better very often&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sequence-to-sequence modeling with encoder-decoder structure almost always implies attention in-between encoder and decoder. Attention relays information between the encoder and decoder, in the sense that every time the decoder has to generate the next item in the target sequence, an attention network computes a dynamic, useful &quot;summary&quot; of all encoder states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This attention summary is different and recomputed for every decoding step. On the other hand, encoding the source sequence is done only once and then all encoder states are kept in memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ability to have a direct view of the source sequence (using as a proxy the entire sequence of encoder states) via attention is what makes the encoder-decoder approach superioir to a single RNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In comparison, a single RNN only has a direct view on one element of the input sequence. Some interesting scenarios for a single RNN:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;At every time step, read one source token, then write one target token&lt;/strong&gt;: Previous elements in the source sequence are represented only in lossy recurrent states, while future elements cannot be accessed at all.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;First read all source tokens, then write all target tokens.&lt;/strong&gt;: the meaning of the entire source sentence has to be compressed into a fixed-size recurrent state vector.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-28T17:30:43.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17724" PostTypeId="2" ParentId="5115" CreationDate="2020-01-28T19:22:30.837" Score="2" Body="&lt;p&gt;You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You are describing the essence of known natural language processing (NLP) tasks such as word alignment (link words in different languages that have the same meaning) and, of course, machine translation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While learning a machine translation model, we actually do discover which words (or parts of words, or sequences of words) in different languages have the same meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some concepts I would recommend for further study of this subject:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Word alignment, an example for a well-known and popular tool would be &lt;code&gt;fast_align&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Word embeddings, &lt;code&gt;word2vec&lt;/code&gt; is a widely used tool&lt;/li&gt;&#xA;&lt;li&gt;Modern machine translation with sequence-to-sequence models, well-known tools are &lt;code&gt;fairseq&lt;/code&gt;, or &lt;code&gt;Sockeye&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-28T19:22:30.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17725" PostTypeId="2" ParentId="17636" CreationDate="2020-01-28T19:32:37.403" Score="3" Body="&lt;p&gt;(Disclosure: I am a PhD student and lecturer in Computational Linguistics)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that annotation and debugging work with existing tools without modification can be considered Computational Linguistics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yet, most Computational Linguists program on a daily basis, since they actively develop tools. Just to give you some context, at major Computational Linguistics conferences such as ACL or EMNLP (the biggest ones), most authors did the coding themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To say that coding is an unimportant side aspect of being a Computational Linguist, as claimed in the &lt;a href=&quot;https://ai.stackexchange.com/a/17637/33073&quot;&gt;accepted answer&lt;/a&gt;, is a slight misrepresentation.&lt;/p&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-28T19:32:37.403" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17726" PostTypeId="2" ParentId="17439" CreationDate="2020-01-28T19:40:41.287" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for &quot;repeat the character that is 14 characters to the left&quot; and so on&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Creating a system of rules like your example above is the very opposite of training an RNN to perform this task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to train an RNN to answer simple questions you would not need to come up with ingenious rules, but with enough training data of the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;question -&amp;gt; answer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you could use one of many popular sequence-to-sequence NLP tools to try and learn this behaviour, effectively treating the problem as if it were machine translation between languages.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;More broadly, yes, question answering has been done before, it is very popular in fact. It is an active subfield of NLP research and many methods have been developed, some of them involving RNN networks.&lt;/p&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-28T19:40:41.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17727" PostTypeId="2" ParentId="17474" CreationDate="2020-01-28T19:43:41.400" Score="1" Body="&lt;p&gt;If I understand correctly, what you are looking for is called &quot;common sense reasoning&quot; in NLP research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Research in this field revolves around benchmark data sets, where good performance indicates some ability to do common sense reasoning. Here is a nice collection of data sets and research by Sebastian Ruder:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nlpprogress.com/english/common_sense.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nlpprogress.com/english/common_sense.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the end, the main question is not&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How much knowledge of the world is learnt through words?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;since it is virtually unanswerable if asked in this form. A question that is answered in NLP common sense reasoning research is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Out of 100 specific decisions that my model needs to take, for how many does the model show the ability to reason correctly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-28T19:43:41.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17728" PostTypeId="1" CreationDate="2020-01-29T01:12:00.620" Score="2" ViewCount="171" Body="&lt;p&gt;I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of numbers as input, can predict the next number in this sequence. For example, the input could be &lt;code&gt;1, 2, 3, 4&lt;/code&gt; and the output should be &lt;code&gt;5&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read something that said this could be done with a multilayer perceptron neural net, but that didn't elaborate much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas, or links to tutorials/code?&lt;/p&gt;&#xA;" OwnerUserId="33078" LastEditorUserId="2444" LastEditDate="2020-01-29T11:08:11.420" LastActivityDate="2020-01-29T11:08:11.420" Title="How I can predict the next number in a sequence with a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;prediction&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17729" PostTypeId="2" ParentId="17721" CreationDate="2020-01-29T01:48:25.913" Score="7" Body="&lt;h2&gt;Broken assumptions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard digit-recognition classifier like MNIST is that you're classifying pictures that actually contain a single digit. If your real data contains pictures that have non-digits, then that means that your real data is &lt;em&gt;not&lt;/em&gt; similar to training data but is conceptually very, very different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that's a problem (as in this case) then one way to treat that is to explicitly break that assumption and train a model that not only recognizes digits 0-9 but also recognizes whether there's a digit at all, and is able to provide an answer &quot;that's not a digit&quot;, so a 11-class classifier instead of a 10-class one. MNIST training data is not sufficient for that, but you can use some kind of 'distractor' data to provide the not-a-digit examples. For example, you could use some dataset of letters (perhaps omitting I, l, O and B) transformed to look similar to MNIST data.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastActivityDate="2020-01-29T01:48:25.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17730" PostTypeId="2" ParentId="17474" CreationDate="2020-01-29T01:57:42.120" Score="0" Body="&lt;h2&gt;Grounded language learning&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Your description does not match the current understanding of how children learn language, and giving a computer an explicit textual list of common sense rules is very different from teaching a &lt;em&gt;young&lt;/em&gt; child. (Some aspects of teaching older children is more similar to that, but by that time they fully know the language already). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much of language is acquired through interaction (both sensory and motoric) with the physical world, applying words together with a &lt;em&gt;shared focus of attention&lt;/em&gt; to something real. I.e. you talk about a cat or a ball and it's behavior while both you and the child are paying attention to that behavior or object or an image of it. The same applies later for more complex topics such as social situations - to teach a topic, to a child, you'd inevitably use a shared attention to specific events in the real world or specific events in a mostly shared 'model world' i.e. one reconstructed from memory (what you saw your sister do five minutes ago) or imagined/hypothesised (if you do this, these consequences might happen).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attempts to replicate this process in artificial systems is usually called 'grounded language learning', and there's extensive published literature on that which may be interesting to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, the assumption is that English (or any other) words to an artificial system are just as useful as a Chinese-Chinese explanatory dictionary to me. If I speak some basic Chinese, then I can use that dictionary to expand my vocabulary and understand complex Chinese - but if I have &lt;em&gt;no&lt;/em&gt; grounding in Chinese whatsoever, then the expectation is that it's impossible to reconstruct a language from that data alone.&lt;/p&gt;&#xA;" OwnerUserId="1675" LastEditorUserId="1675" LastEditDate="2020-01-29T02:05:28.497" LastActivityDate="2020-01-29T02:05:28.497" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17731" PostTypeId="2" ParentId="17721" CreationDate="2020-01-29T02:29:32.893" Score="15" Body="&lt;p&gt;Your classifier is specifically learning the ways in which &lt;strong&gt;0&lt;/strong&gt;s are different from other digits, &#xA;&lt;strong&gt;not&lt;/strong&gt; what it really means for a digit to &lt;strong&gt;be&lt;/strong&gt; a zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Philosophically, you could say the model appears to have some powerful understanding when restricted&#xA;to a tightly controlled domain, but that facade is lifted as soon as you throw any sort of wrench in &#xA;the works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, you could say that the model is simply optimizing a classification metric for data&#xA;drawn from a specific distribution, and when you give it data from a different distribution, all&#xA;bets are off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The go-to answer is to collect or generate data like the data you expect the model to deal with&#xA;(in practice, the effort required to do so can vary dramatically depending upon the application). In&#xA;this case, that could involve drawing a bunch of random scribbles and adding them to your training&#xA;data set. At this point you must ask, now how do I label them? You will want a new &quot;other&quot; or &#xA;&quot;non-digit&quot; class so that your model can learn to categorize these scribbles separately from digits.&#xA;After retraining, your model should now better deal with these cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you may then ask, but what if I gave it color images of digits?  Or color images of farm&#xA;animals?  Maybe pigs will be classified as zeros because they are round.  This problem is a&#xA;fundamental property of the way deep learning is orchestrated.  Your model is not capable of higher&#xA;order logic, which means it can seem to go from being very intelligent to very dumb by just throwing&#xA;the slightest curve ball at it.  For now, all deep learning does is recognize patterns in data that&#xA;allow it to minimize some loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning is a fantastic tool, but not an all-powerful omnitool.  Bear in mind its limitations&#xA;and use it where appropriate, and it will serve you well.&lt;/p&gt;&#xA;" OwnerUserId="33079" LastActivityDate="2020-01-29T02:29:32.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17732" PostTypeId="1" CreationDate="2020-01-29T03:56:33.950" Score="2" ViewCount="73" Body="&lt;p&gt;The famous Nvidia paper &lt;a href=&quot;https://research.nvidia.com/sites/default/files/pubs/2017-10_Progressive-Growing-of/karras2018iclr-paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Progressive Growing of GANs for Improved Quality, Stability, and Variation&lt;/a&gt;, the GAN can generate hyperrealistic human faces. But, in the very same paper, images of other categories are rather disappointing and there hasn't seemed to be any improvements since then. Why is it the case? Is it because they didn't have enough training data for other categories? Or is it due to some fundamental limitation of GAN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have come across a paper talking about the limitations of GAN: &lt;a href=&quot;http://openaccess.thecvf.com/content_ICCV_2019/papers/Bau_Seeing_What_a_GAN_Cannot_Generate_ICCV_2019_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seeing What a GAN Cannot Generate&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anybody using GAN for image synthesis other than human faces? Any success stories?&lt;/p&gt;&#xA;" OwnerUserId="33082" LastEditorUserId="2444" LastEditDate="2020-01-29T11:12:07.853" LastActivityDate="2020-01-29T11:12:07.853" Title="Is it feasible to use GAN for high-quality image synthesis other than human faces?" Tags="&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;papers&gt;&lt;image-generation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17733" PostTypeId="2" ParentId="17466" CreationDate="2020-01-29T08:37:27.163" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I came across a news article from 2018 where the president of India was saying that Sanskrit is the best language for ML/AI.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A very interesting statement indeed! Globally, there is very little interest in NLP for Sanskrit compared to very many other languages.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Especially given the fact that humans still have to first learn Sanskrit in case NLP is done in Sanskrit.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most people would say that NLP tools are meant to cater to the needs of humans, instead of requiring humans to learn a new language to be able to benefit from tools (which is a &lt;em&gt;huge&lt;/em&gt; barrier).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question more clearly, it is not that the research community selects languages for research based on &quot;suitability&quot; of the language. Instead, in general the intention of NLP research is to grapple with problems that are relevant, impact many people - or are funded (unfortunately :-).&lt;/p&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-29T08:37:27.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17734" PostTypeId="1" CreationDate="2020-01-29T08:47:11.317" Score="7" ViewCount="141" Body="&lt;p&gt;While reading &lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the DQN paper&lt;/a&gt;, I found that randomly selecting and learning samples reduced divergence in RL using a non-linear function approximator (e.g a neural network).&lt;/p&gt;&#xA;&lt;p&gt;So, why does Reinforcement Learning using a non-linear function approximator diverge when using strongly correlated data as input?&lt;/p&gt;&#xA;" OwnerUserId="33088" LastEditorUserId="2444" LastEditDate="2020-12-19T13:06:47.423" LastActivityDate="2020-12-19T13:06:47.423" Title="Why does reinforcement learning using a non-linear function approximator diverge when using strongly correlated data as input?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;convergence&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17736" PostTypeId="2" ParentId="17732" CreationDate="2020-01-29T09:38:05.093" Score="1" Body="&lt;p&gt;Generative Adversarial Networks, basically boil down to a combination of a generic Generator and a Discriminator trying to beat each other, so that the generator tries to generate much better images (usually from noise) and discriminator becomes much better at classification. So, no it is not just suited for only synthesis high quality human face synthesis but any image type. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, not only can it be used for any high quality image synthesis, &lt;strong&gt;it can work on non-image data types as well (such as text, etc)&lt;/strong&gt;. It all depends on the type of neural network you are using for the discriminator and generator at the end of the day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://openaccess.thecvf.com/content_iccv_2017/html/Osokin_GANs_for_Biological_ICCV_2017_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://openaccess.thecvf.com/content_iccv_2017/html/Osokin_GANs_for_Biological_ICCV_2017_paper.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above refers to a paper synthesizing cell image through GANs, as I haven't personally used GANs on a practical level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General blog explaining GANs:&lt;br&gt;&#xA;&lt;a href=&quot;https://machinelearningmastery.com/impressive-applications-of-generative-adversarial-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/impressive-applications-of-generative-adversarial-networks/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Human faces are much more frequently tackled, for many reasons, usually that human faces are highly symmetric and have a lot of different features, usually more than other types of images, with the added difficulty of being that we as actual humans are usually good at recognizing faces - making a neural net to fool ourselves, makes it a challenging area of research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helped! Do let me know if I'm wrong somewhere.&lt;/p&gt;&#xA;" OwnerUserId="33092" LastActivityDate="2020-01-29T09:38:05.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17737" PostTypeId="2" ParentId="17732" CreationDate="2020-01-29T10:06:08.380" Score="1" Body="&lt;p&gt;I'd challenge your assertion somewhat that the generated images of other categories are of much worse quality than the faces!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the bikes on transparent / solid backgrounds they look great!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the images fail a bit is with the more complex pictures which have a lot of elements where element bleed (covers bleeding into the floor, etc.) occurs. This is simply a result of the complexity of the image and the training data base.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example I have developed a GAN that generates &quot;Vaporwave&quot;-like imagery like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Oh8OK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Oh8OK.png&quot; alt=&quot;this&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my results were generally poor because unlike faces my training set was highly diverse in terms of arrangements, elements, etc. If you look at the generated bed images in your example paper not only did the GAN have to learn and generate beds but also the highly complex backgrounds which did differ severely between training images whereas in the face example the image was zoomed in on the faces and obscuring the background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use human faces in a normal background setting (e.g. with the scenery around them visible) your GAN will perform equally good or bad because there is so much more complexity to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find my experiences with non-faces GAN on Kaggle but understand the bad results are mainly due to a very small training set and the fact that these images are very different (besides the color gradient which the GAN pics up very fast).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.kaggle.com/fnguyen/vaporprogan&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/fnguyen/vaporprogan&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2020-01-29T10:06:08.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17738" PostTypeId="5" CreationDate="2020-01-29T11:17:04.017" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-29T11:17:04.017" LastActivityDate="2020-01-29T11:17:04.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17739" PostTypeId="4" CreationDate="2020-01-29T11:17:04.017" Score="0" Body="For questions related to the bias-variance tradeoff, which is an important issue in machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-29T11:17:04.017" LastActivityDate="2020-01-29T11:17:04.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17740" PostTypeId="2" ParentId="17195" CreationDate="2020-01-29T12:52:27.837" Score="1" Body="&lt;p&gt;The problem is not that RNN flavours such as LSTMs are incapable of keeping track of the &quot;important&quot; parts of the input. They also do not have much trouble recognizing commas in different places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prove this point, I recommend reading &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrej Karpathy's excelllent write-up about the behaviour of individual RNN &quot;neurons&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addressing specifically this comment in your question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I suspect another reason why the LSTM is not able to infer the format is that the comma can be placed in different indexes of the sequence, so it could be losing its importance in the hidden state the longer the sequence is (not sure if that makes sense).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If commas are relevant to solve the task at hand, LSTMs can learn to remember its position or related information. This information is not necessarily diluted by repeated application of reccurrence with long sequences: networks can learn to propagate and promote crucial information from one recurrent state to the next.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input sequences have arbitrary length, which means that LSTMs need to compress information about seen sequence elements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather, the input sequence has an arbitrary length, while the LSTM state vectors have a fixed size. State vectors are the only way for an LSTM to &quot;keep track of important parts&quot;. This means that those fixed-size vectors are a bottleneck and there is an information-theoretic upper bound on the amount of information about &quot;important parts&quot; that can be kept by an LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;LSTMs potentially take multiple decisions. For each decision, something else in the input sequence is most important&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For tasks such as summarization that you mention in the question, an LSTM makes a series of predictions (predicting the tokens of the summary one token at a time). For each prediction, different things in the input sequence might be important. Put another way, for each decision, another &lt;em&gt;view&lt;/em&gt; of the input may be most helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a key motivation for using attention networks. Each time an LSTM is making a prediction, an attention network can provide a dynamic, optimally helpful view of the input sequence.&lt;/p&gt;&#xA;" OwnerUserId="33073" LastActivityDate="2020-01-29T12:52:27.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17741" PostTypeId="2" ParentId="12490" CreationDate="2020-01-29T13:29:27.277" Score="6" Body="&lt;p&gt;&lt;strong&gt;Can the decoder in a transformer model be parallelized like the encoder?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The correct answer is: computation in a Transformer decoder can be parallelized during training, but not during actual translation (or, in a wider sense, generating output sequences for new input sequences during a testing phase).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What exactly is parallelized?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, it's worth mentioning that &amp;quot;parallelization&amp;quot; in this case means to compute encoder or decoder states in paralllel &lt;strong&gt;for all positions&lt;/strong&gt; of the input sequence. Parallelization over several layers is not possible: the first layer of a multi-layer encoder or decoder still needs to finish computing all positions in parallel before the second layer can start computing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why can the decoder be parallelized position-wise during training?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For each position in the input sequence, a Transformer decoder produces a decoder state as an output. (The decoder state is then used to eventually predict a token in the target sequence.)&lt;/p&gt;&#xA;&lt;p&gt;In order to compute one decoder state for a particular position in the sequence of states, the network consumes as inputs: 1) the entire input sequence and 2) &lt;strong&gt;the target words that were generated previously&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;During training, the target words generated previously are known, since they are taken from the target side of our parallel training data. This is the reason why computation can be factored over positions.&lt;/p&gt;&#xA;&lt;p&gt;During inference (also called &amp;quot;testing&amp;quot;, or &amp;quot;translation&amp;quot;), the target words previously generated are predicted by the model, and computing decoder states must be performed sequentially for this reason.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Comparison to RNN models&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;While Transformers can parallelize over input positions during training, an encoder-decoder model based on RNNs cannot parallelize positions. This means that Transformers are generally faster to train, while RNNs are faster for inference.&lt;/p&gt;&#xA;&lt;p&gt;This observation leads to the nowadays common practice of training Transformer models and then using &lt;a href=&quot;https://arxiv.org/abs/1606.07947&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence-level distillation&lt;/a&gt; to learn an RNN model that mimicks the trained Transformer, for faster inference.&lt;/p&gt;&#xA;" OwnerUserId="33073" LastEditorUserId="33073" LastEditDate="2020-11-12T12:39:55.133" LastActivityDate="2020-11-12T12:39:55.133" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="17743" PostTypeId="2" ParentId="17721" CreationDate="2020-01-29T14:07:54.923" Score="3" Body="&lt;p&gt;Apollys,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's a very well thought out response. Particularly, the philosophical  discussion of the essence of &quot;0-ness.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't actually performed this experiment, so caveat emptor... I wonder how well an &quot;other&quot; class would actually work. The ways in which &quot;other&quot; differs from &quot;digit&quot; has infinite variability (or at least its only limitation is the cardinality of the input layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NN decides whether something is more of one class or more of a different class. If there isn't an essence in common among other &quot;non-digits&quot;, I don't believe it will do well at identifying &quot;other&quot; as the catch-all for everything that has low confidence level of classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach still doesn't identify what it is to &lt;strong&gt;&lt;em&gt;be&lt;/em&gt;&lt;/strong&gt; &quot;not-digit&quot;. It identifies how all the things that are &quot;other&quot; differ from the other labeled inputs -- probably poorly, depending on the variability of the &quot;non-digit&quot; labeled data. (i.e. is it numerically exhaustive, many times over, of all random scribbles?) Thoughts?&lt;/p&gt;&#xA;" OwnerUserId="33103" LastActivityDate="2020-01-29T14:07:54.923" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17744" PostTypeId="1" CreationDate="2020-01-29T14:56:44.147" Score="2" ViewCount="53" Body="&lt;p&gt;Let's assume that we have a regression problem. Our input is just binarized image that contains a single rectangle and we want to predict just a float number.  Actually, this floating-point number depends on rectangle angle, rectangle size and rectangle location. Is this problem can be solved by a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, it can not be solved by a neural network, because rectangle angle, size and location are latent variables and without learning these latent variable, above problem can not be solved. What do you think?&lt;/p&gt;&#xA;" OwnerUserId="33108" LastEditorUserId="2444" LastEditDate="2020-01-29T16:18:20.680" LastActivityDate="2020-01-29T16:18:20.680" Title="Can a neural network learn to predict a number given a binarized image of a rectangle?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;feature-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17746" PostTypeId="2" ParentId="17744" CreationDate="2020-01-29T15:35:49.157" Score="3" Body="&lt;p&gt;It can definitely be learned, the question is the approach. It would be expensive and difficult from a modeling directive to do this Densely, so usually convolutions are the way to go. An issue with convolutions is that is generally focuses on equivariant and relative features, so if you need specific location within the approach might be worth the simple alteration of &lt;a href=&quot;https://arxiv.org/pdf/1807.03247.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CoordConv&lt;/a&gt;. Regardless of approach, that type of input to output is possible, you just have to consider it when modeling.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2020-01-29T15:35:49.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17747" PostTypeId="1" CreationDate="2020-01-29T15:39:57.090" Score="1" ViewCount="18" Body="&lt;p&gt;Or would you simply do this as a time series of models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I think you can think of time series of weights as the hidden states and the dynamics driving the weight time series as the RNN weights. I'm not sure if the data gradients are avoiding look ahead in this context though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am basically thinking of smoothing (data assimilation) formulation of the filtering problem. Usually smoothing has look-ahead bias but with stop_gradients (and a large graph) it should be possible to do &quot;batch&quot; filtering.&lt;/p&gt;&#xA;" OwnerUserId="23001" LastActivityDate="2020-01-29T15:39:57.090" Title="Is there a way to use RNN (in tensorflow) to do something like a batch Kalman with the weight dynamics specified in the loss?" Tags="&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17749" PostTypeId="1" CreationDate="2020-01-29T20:33:27.783" Score="4" ViewCount="315" Body="&lt;p&gt;Can someone explain to me why it is possible to eliminate the rest of the middle branch in this image for alpha-beta pruning? I am confused because it seems the only information you know is that Helen would pick at least a 2 at the top (considering if we iterate from left to right in DFS), and Stavros would absolutely not pick anything above 7. This leaves 5 possible numbers that the rest of the branch could take on that Helen could potentially end up picking, but can't because we've eliminated those possibilities via pruning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jCPkU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="33117" LastEditorUserId="2444" LastEditDate="2020-01-29T20:37:14.003" LastActivityDate="2020-03-26T00:03:05.403" Title="Why is it possible to eliminate this branch with alpha-beta pruning?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17750" PostTypeId="1" CreationDate="2020-01-29T22:33:48.273" Score="2" ViewCount="120" Body="&lt;p&gt;I'm super new to deep learning and computer vision, so this question may sound dumb.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this link (&lt;a href=&quot;https://github.com/GeorgeSeif/Semantic-Segmentation-Suite&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/GeorgeSeif/Semantic-Segmentation-Suite&lt;/a&gt;), there are pre-trained models (e.g., ResNet101) called front-end models. And they are used for feature extraction. I found these models are called backbone models/architectures generally. And the link says some of the main models (e.g. DeepLabV3 or PSPNet) rely on pre-trained ResNet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, transfer learning is to take a model trained on a large dataset and transfer its knowledge to a smaller dataset, right?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do the main models that rely on pre-trained ResNet do transfer learning basically (like from ResNet to the main model)?      &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I use a pre-trained network, like ResNet101, as the backbone architecture of the main model(like U-Net or SegNet) for image segmentation, is it considered as transfer learning?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33118" LastEditorUserId="2444" LastEditDate="2020-01-30T16:36:59.670" LastActivityDate="2020-01-30T16:36:59.670" Title="What is the difference between using a backbone architecture and transfer learning?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;image-segmentation&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17752" PostTypeId="1" CreationDate="2020-01-29T23:32:36.457" Score="1" ViewCount="72" Body="&lt;p&gt;I'm trying to develop a stock predictor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using LSTM but I am unsure about the structure of the Neural Network. For example, I'm assuming that the Neural Network is a many-to-one since we have many inputs (i.e Open, Close etc) and one output (stock price).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By misunderstanding is coming with how to construct the nodes. For example, what input goes into the &quot;Cell&quot; (or node)? I.e does say 60 timestep mean 60 days of 'Open Price' are fed into the Neural Network at t and then 60 days of 'Close' into t + 1 until we use all input to produce an output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If someone could explain the process of how LSTM are used with stock predictions that would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="33121" LastEditorUserId="2444" LastEditDate="2020-01-30T16:41:21.887" LastActivityDate="2020-01-30T16:41:21.887" Title="How should I design this LSTM network to perform stock prediction?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17753" PostTypeId="2" ParentId="17752" CreationDate="2020-01-30T02:09:52.613" Score="1" Body="&lt;p&gt;I can't say for all cases, but I can certainly give you an example of how an LSTM &lt;em&gt;could&lt;/em&gt; be used to predict stock prices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An LSTM is temporal, meaning you can feed in one input, get an output and the network will remember this interaction (if it's important), which will effect the outcome of future predictions. As such, it is reasonable to feed in all the data for the day (Open, Close etc), obtain an output for the predicted stock price of the following day (or a prediction for all the inputs you feed in, the output can be as large or small as you want), then repeat for as long as you want, each time only feeding in the data for that particular day as the network will remember previous days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, 60 timesteps would mean doing a forward pass of the LSTM 60 times, for 60 days. Each of these forward passes will produce an output, which you can compare to the next actual stock price for verification, until you reach the current day, where the prediction is for the future.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-01-30T02:09:52.613" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17754" PostTypeId="2" ParentId="17721" CreationDate="2020-01-30T09:00:36.700" Score="2" Body="&lt;p&gt;I'm an amateur with neural networks, but I will illustrate my understanding of how this problem comes to be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, lets see how trivial neural network classifies 2D input into two classes : &#xA;&lt;a href=&quot;https://i.stack.imgur.com/OgldC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OgldC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in case of complex neural network, the input space is much bigger and the sample data points are much more clustered with big chunks of empty space between them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/74IAT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/74IAT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neural network then doesn't know how to classify the data in the empty space, so something like this is possible :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2VAYc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2VAYc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Vwep2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vwep2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using the traditional ways of measuring quality of neural networks, both of these will be considered good. As they do classify the classes themselves correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, what happens if we try to classify these data points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ngw14.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ngw14.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, neural network has no data it could fall back on, so it just outputs what seems to us as random nonsense.&lt;/p&gt;&#xA;" OwnerUserId="28367" LastActivityDate="2020-01-30T09:00:36.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17755" PostTypeId="1" CreationDate="2020-01-30T09:27:38.920" Score="2" ViewCount="79" Body="&lt;p&gt;We have hundreds of thousands of customers records, and we need to take the benefits of our data to train a model that will recognize fake entries or unrealistic ones for our platform, where customers are asked to enter their names, phone number and zip code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, our attributes are name, phone number, zip code and IP address to train the model with. We have only data associated with real users. Can we train a model provided with only positive labels (as we do not have a negative dataset to train the model with)?&lt;/p&gt;&#xA;" OwnerUserId="33135" LastEditorUserId="2444" LastEditDate="2020-01-31T02:29:23.397" LastActivityDate="2020-01-31T14:36:34.840" Title="Can we train the model to detect real users with only positive labels?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;features&gt;" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="17756" PostTypeId="1" CreationDate="2020-01-30T11:03:09.707" Score="1" ViewCount="22" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1910.07954&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1910.07954&lt;/a&gt;&#xA;In this paper, we have a convolutional character neural network where we have object detection by taking a character as a basic unit. First, we do character detection and recognition and then we go for text detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vunhN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vunhN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here (Page number 5 under the subheading Iterative character detection) it is written that a model trained on English and Chinese texts will generalize well in regards to text detection.&#xA;But how English and Chinese texts are good for generalization in text detection.&#xA;If you have any queries regarding the paper you can ask me in the comment section&#xA;Thanks in advance! &lt;/p&gt;&#xA;" OwnerUserId="30907" LastActivityDate="2020-01-30T11:03:09.707" Title="Text detection on English and Chinese language" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17757" PostTypeId="1" CreationDate="2020-01-30T11:38:52.763" Score="2" ViewCount="27" Body="&lt;p&gt;I wonder if researchers tried to understand how LSTMs work by analyzing the dynamics of simple LSTM (e.g. with 2 units)? For example how the hidden state evolves depending on the properties of weight matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like a very natural thing to try (especially because it is easy to draw hidden states with 2 units on a 2D plane). However, I haven't found any papers that would play with such a toy example. Are there such papers at all? Or with such a simple example, it is impossible to get some understanding because of its over-simplicity? (which I doubt because even logistic maps generate very complicated behavior)&lt;/p&gt;&#xA;" OwnerUserId="33098" LastEditorUserId="30725" LastEditDate="2020-05-31T09:29:23.550" LastActivityDate="2020-05-31T09:29:23.550" Title="Did people analyze dynamics of very simple LSTMs?" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;theory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17758" PostTypeId="1" CreationDate="2020-01-30T12:48:10.287" Score="1" ViewCount="29" Body="&lt;p&gt;This question should be quite generic but I faced the problem in the case of the TiF-GAN generator so I am going to use it as an example. &lt;a href=&quot;https://arxiv.org/pdf/1902.04072.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;(Link to paper)&lt;/strong&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you check the penultimate page in the paper you can find the architecture design of the generator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generator has a dense layer and then a reshape layer converting the hidden layer feature map to a dimensionality of &lt;strong&gt;8x4x512&lt;/strong&gt; (given that &lt;span class=&quot;math-container&quot;&gt;$s = 64$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then what follows is a transpose convolution operation with a kernel size of &lt;strong&gt;12x3&lt;/strong&gt; with &lt;span class=&quot;math-container&quot;&gt;$512$&lt;/span&gt; filters and a stride of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; in all dimensions. The output of this layer should be then &lt;strong&gt;16x8x512&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After fiddling with some coding I found out that the authors also used the setting &lt;code&gt;padding=same&lt;/code&gt; in their tensor flow code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is: &lt;strong&gt;How and what do you pad when you perform such a transpose convolution to get those output dimensions?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without any padding I would assume that you should get an output of &lt;strong&gt;26x9x1534&lt;/strong&gt; assuming that each output dimension is equal to &lt;code&gt;dim = kernel_dim + strides * (input_dim - 1)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2020-01-30T12:48:10.287" Title="Transpose convolution in TiF-GAN: How does &quot;same&quot; padding works?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;padding&gt;&lt;transpose-convolution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17760" PostTypeId="2" ParentId="17721" CreationDate="2020-01-30T13:35:42.510" Score="0" Body="&lt;p&gt;In your particular case, you could add a eleventh category to your training data: &quot;not a digit&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then train your model with a bunch of images of incorrectly segmented digits, in addition to the normal digit examples. This way the model will learn to tell apart real digits from incorrectly segmented ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However even after doing that, there will be an infinite number of random looking images that will be classified as digits. They're just far away from the examples of &quot;not a digit&quot; you provided.&lt;/p&gt;&#xA;" OwnerUserId="25912" LastActivityDate="2020-01-30T13:35:42.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17761" PostTypeId="1" CreationDate="2020-01-30T13:41:25.620" Score="2" ViewCount="20" Body="&lt;p&gt;Let's set up some hypothetical simplified scenario: Each instance &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; of my imaginary dataset &lt;span class=&quot;math-container&quot;&gt;$D=\{i_{1}, \ldots, i_{MAX}\}$&lt;/span&gt; has different number &lt;span class=&quot;math-container&quot;&gt;$k_{i}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional vectors as input into my neural network. Each of them will be transformed with &lt;span class=&quot;math-container&quot;&gt;$m \times n$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; (so, matrices with same parameters) and acted point-wise with some non-linearity &lt;span class=&quot;math-container&quot;&gt;$\sigma_{1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there are 2 possibilities I want to consider separately:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Case: I want to average all those outputs, thus depending of total number of vectors &lt;span class=&quot;math-container&quot;&gt;$k_{i}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Case: I want to choose 2 subsets (no necessarily mutually disjoint) and average vectors from those subsets.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the later layers (in both cases) I'd use some the &quot;standard&quot; neural networks architectures and loss functions. Notice, I'll probably go with more complex connections than averaging, this is only for the purpose of this technical question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&#xA; 1. Is there a &quot;simple&quot; way (meaning: high-level library like&#xA;    tensorflow) which allows me to create custom layer with shared&#xA;    weights which is also size-agnostic (not size of vectors, but number&#xA;    of vectors) Is it possible to parallelise computation like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some knowledge of tf, but I still didn't use all obscure low-level details and don't know all capabilities of &lt;span class=&quot;math-container&quot;&gt;$2.0$&lt;/span&gt; version. Also, I'd know to do this stuff in numpy with handcrafted back-propagation implementation. I'd like to do it in more high level way which will handle backpropagation and parallelisation for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to avoid having tensor of vectors which will be padded, but I'm not sure is it possible. My real problem is more complex but has same characteristics: I need to use same weights on differently connected layers, and I would handle normalization of the input to the following layers to the same scale manually (i.e. with &quot;normal&quot; averaging or weighted attention-like averaging,...). Not always I'd connect all vectors from first layers to all from second. It needs to be flexible for each instance of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in short: I want to reuse the same weight matrices but connect them differently for each instance of dataset. I want to do it in framework which already has automatic differentiation/backpropagation and parallelisation. Is it possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does eager execution in tf help in solving problems like this? I guess nothing could be done with prebuilt computation graphs in 1.x versions.&lt;/p&gt;&#xA;" OwnerUserId="33144" LastActivityDate="2020-01-30T13:41:25.620" Title="How to handle set-like size agnostic input format" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17762" PostTypeId="1" CreationDate="2020-01-30T14:50:12.037" Score="2" ViewCount="61" Body="&lt;p&gt;I modified resnet50 architecture to get a regression network. I just add batchnorm1d and ReLU layers just before the fully connected layer. During the training, the output of batchnorm1d layer is nearly equal to 3 and this gives good results for training. However, during inference, output of batchnorm1d layer is about 30 so this leads to too low accuracy for test results. In other words, outputs of batchnorm1d layers give very different normalized output during training and inference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the reason for this situation, and how can I solve it? I am using PyTorch.&lt;/p&gt;&#xA;" OwnerUserId="33108" LastEditorUserId="2444" LastEditDate="2020-01-30T16:44:04.847" LastActivityDate="2020-01-30T16:44:04.847" Title="Why does the BatchNormalization layer produce different outputs during training and inference?" Tags="&lt;convolutional-neural-networks&gt;&lt;pytorch&gt;&lt;batch-normalization&gt;&lt;inference&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17763" PostTypeId="1" CreationDate="2020-01-30T15:08:07.487" Score="2" ViewCount="23" Body="&lt;p&gt;I am playing with a large dataset of hotel reviews, which contains both positive and negative reviews (the reviews are labeled). I want to use this dataset to perform textual style transfer - given a positive review, output a negative review which address the same thing. For example, if the positive review mentioned how spacious the rooms are, I want the output to be a review that complains about the small and claustrophobic rooms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't have positive review-negative review pairs for the training. I was thinking that maybe I could create those pairs myself, but I'm not sure what is the best way to do that. Simple heuristics like jaccard index and such didn't give the desired results. &lt;/p&gt;&#xA;" OwnerUserId="33147" LastActivityDate="2020-01-30T15:08:07.487" Title="How to tell if two hotel reviews addressing the same thing" Tags="&lt;natural-language-processing&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17764" PostTypeId="1" AcceptedAnswerId="17785" CreationDate="2020-01-30T15:09:42.920" Score="6" ViewCount="2512" Body="&lt;p&gt;I read that, if we use the sigmoid or hyperbolic tangent activation functions in deep neural networks, we can have some problems with the vanishing of the gradient, and this is visible by the shapes of the derivative of these functions. ReLU solves this problem thanks to its derivative, even if there may be some dead units. ResNet uses ReLU as activation function, but looking online what I understood is that ResNet solves the vanishing of the gradient thanks to its identity map, and I do not totally agree with that. So what's the purpose of the identity connections in ResNet? Are they used for solving the vanishing of the gradient? And ReLU really solves the vanishing of the gradient in deep neural networks?&lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2020-01-30T16:50:24.100" LastActivityDate="2021-01-17T14:07:38.757" Title="Why do ResNets avoid the vanishing gradient problem?" Tags="&lt;deep-learning&gt;&lt;activation-function&gt;&lt;exploding-gradient-problem&gt;&lt;residual-networks&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17765" PostTypeId="1" CreationDate="2020-01-30T15:11:28.713" Score="2" ViewCount="32" Body="&lt;p&gt;with a friend of mine, we got in an argument over how to label images for multi-label. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Note:&lt;/em&gt;&lt;/strong&gt;&#xA;&lt;strong&gt;&lt;em&gt;Groups of a species&lt;/strong&gt; and the species of &lt;strong&gt;catfish&lt;/strong&gt; is important to recognize.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The labels are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;'I': an individual fish of any type except catfish&lt;/li&gt;&#xA;&lt;li&gt;'R': A group of &lt;strong&gt;same species&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;'K': Catfish&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First conflict:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an image containing a bank of fish, our conflicting opinions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I and R&lt;/li&gt;&#xA;&lt;li&gt;R&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second conflict:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If in an image there's actually a bank of catfish, our conflicting opinions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;K and R&lt;/li&gt;&#xA;&lt;li&gt;K&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Third conflict:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If in an image there's actually a group of same species of fish and other individual fish from different species, our conflicting opinions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;R&lt;/li&gt;&#xA;&lt;li&gt;I and R &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I think one very important difference of opinion here, is if in multi-label, we should give two labels to one object in image(say group and fish) or if an object should have only one label.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should the overwhelming presence of an object, overshadow the presence of another (group of fish of same species and another individual fish) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What do you think? &lt;/p&gt;&#xA;" OwnerUserId="32622" LastActivityDate="2020-01-30T15:11:28.713" Title="Labeling for multilabel image classification" Tags="&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17766" PostTypeId="5" CreationDate="2020-01-30T16:51:42.617" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-30T16:51:42.617" LastActivityDate="2020-01-30T16:51:42.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17767" PostTypeId="4" CreationDate="2020-01-30T16:51:42.617" Score="0" Body="For questions related to the vanishing gradient problem, which is a numerical problem that occurs while training a (deep) neural network with a gradient-based optimization technique. There's also the related exploding gradient problem." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-01-30T16:51:42.617" LastActivityDate="2020-01-30T16:51:42.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17768" PostTypeId="1" AcceptedAnswerId="17798" CreationDate="2020-01-30T16:57:29.820" Score="2" ViewCount="2028" Body="&lt;p&gt;I am looking for a small size dataset on which I can implement object detection, object segmentation and object localization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest me a dataset less than 5GB? Or do I need to know anything before implementing these algorithms?&lt;/p&gt;&#xA;" OwnerUserId="32638" LastEditorUserId="2444" LastEditDate="2020-02-02T20:15:41.920" LastActivityDate="2020-02-05T15:36:26.337" Title="Small size datasets for object detection, segmentation and localization" Tags="&lt;datasets&gt;&lt;object-recognition&gt;&lt;object-detection&gt;&lt;resource-request&gt;" AnswerCount="3" CommentCount="1" ClosedDate="2020-09-12T15:17:39.347" ContentLicense="CC BY-SA 4.0" />
  <row Id="17769" PostTypeId="2" ParentId="17755" CreationDate="2020-01-31T00:24:08.870" Score="1" Body="&lt;p&gt;The problem which you have is a &lt;strong&gt;classification&lt;/strong&gt; problem. You assume a class &quot;good users&quot; and a distinct class &quot;bad users&quot;. You want to train an AI to tell the two apart, but all your examples are &quot;good users&quot;. Any reasonable AI will draw the logical conclusion from those examples: all users are good users. That's a 100% match for the training data. &lt;/p&gt;&#xA;" OwnerUserId="16378" LastActivityDate="2020-01-31T00:24:08.870" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17770" PostTypeId="1" CreationDate="2020-01-31T07:36:50.013" Score="1" ViewCount="42" Body="&lt;p&gt;I am trying to solve the kaggle's house prices using neural network. I've already made it with ensembling several models (XGBoost, GradientBooster and Ridge) and I've got a great score ranking me between the top 25%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagined that by adding a new model to the ensembled models like ANN would increase prediction accuracy, so I did the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import keras&#xA;&#xA;model = keras.models.Sequential()&#xA;&#xA;model.add(keras.layers.Dense(235, activation='relu', input_shape=(235,)))&#xA;model.add(keras.layers.Dense(235, activation='relu'))&#xA;model.add(keras.layers.Dense(235, activation='relu'))&#xA;model.add(keras.layers.Dense(235, activation='relu'))&#xA;model.add(keras.layers.Dense(235, activation='relu'))&#xA;model.add(keras.layers.Dense(1))&#xA;&#xA;model.compile(optimizer='adam', loss='mean_squared_error')&#xA;model.summary()&#xA;model.fit(dataset_ann, y, epochs=100, callbacks=[keras.callbacks.EarlyStopping(patience=3)])&#xA;y_pred = model.predict(X_test_ann)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I choosed 235 neurons for each layer, as the training set has 235 features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For model ensembling:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y_p = (0.1*model.predict(X_test_ann)+0.2*gbr.predict(testset)+0.3*xgb.predict(testset)+0.1*regressor.predict(testset)+0.1*elastic.predict(testset)+0.2*ridge.predict(testset))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The shape of &lt;code&gt;y_p&lt;/code&gt; is &lt;code&gt;(1459, 1459)&lt;/code&gt; instead of &lt;code&gt;(1459, )&lt;/code&gt; where columns are all having the same values, so taking &lt;code&gt;y_p[0]&lt;/code&gt; would be more than enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I submitted the result into kaggle and went from top 25% into bottom 60%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried with epoch = 30, 100, 1000 and got nearly the same bad ranking. &lt;/p&gt;&#xA;" OwnerUserId="26028" LastActivityDate="2020-01-31T13:27:44.003" Title="Applying Artificial neural network into kaggle's house prices data set gave bad predicted values" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17771" PostTypeId="2" ParentId="9141" CreationDate="2020-01-31T07:41:24.390" Score="1" Body="&lt;p&gt;What alternate options exist for this?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;No. Sentence generating is directly related to language modelling (given the previous words in the sentence, what is the next word). Because of bi-directionality of BERT, BERT cannot be used as a language model. If it cannot be used as language model, I don't see how you can generate a sentence using BERT.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="33165" LastActivityDate="2020-01-31T07:41:24.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17772" PostTypeId="1" CreationDate="2020-01-31T08:14:20.480" Score="2" ViewCount="33" Body="&lt;p&gt;I am experimenting with OpenAI Gym and reinforcement learning. As far as I understood, the environment is waiting for the agent to make a decision, so it's a sequential operation like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;decision = agent.decide(state)&#xA;state, reward, done = environment.act(decision)&#xA;agent.train(state, reward)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Doing it in this sequential way, the Markov property is fulfilled: the new state is a result of the old state and the action. However, a lot of games will not wait for the player to make a decision. The game will continue to run and perhaps, the action comes too late.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has it been observed or is it even possible that a neuronal network adjusts its weights so that the PC computes the result faster and thus makes the &quot;better&quot; decision? E.g. one AI beats another because it is faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before posting an answer like &quot;there are always the same amount of calculations, so it's impossible&quot;, please be aware that there is caching (1st level cache versus RAM), &lt;a href=&quot;https://stackoverflow.com/questions/11227809/why-is-processing-a-sorted-array-faster-than-processing-an-unsorted-array&quot;&gt;branch prediction&lt;/a&gt; and maybe other stuff.&lt;/p&gt;&#xA;" OwnerUserId="31627" LastActivityDate="2020-01-31T08:14:20.480" Title="Is a neuronal network able to optimize itself for speed?" Tags="&lt;open-ai&gt;&lt;time&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17773" PostTypeId="1" AcceptedAnswerId="17814" CreationDate="2020-01-31T09:19:08.170" Score="2" ViewCount="85" Body="&lt;p&gt;I have trained a XGboost model to predict survival for the &lt;a href=&quot;https://www.kaggle.com/c/titanic/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle Titanic ML competition&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with all Kaggle competitions there is a &lt;code&gt;train&lt;/code&gt; dataset with the target variable included and a &lt;code&gt;test&lt;/code&gt; dataset without the target variable which is used by Kaggle to compute the final accuracy score that determines your leaderboard ranking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have build a fairly simple ensemble classifier (based on XGboost) and evaluated it via standard train-test-splits of the &lt;code&gt;train&lt;/code&gt; data. The &lt;strong&gt;accuracy&lt;/strong&gt; I get from this &lt;strong&gt;validation is ~80%&lt;/strong&gt; which is good but not amazing by public leaderboard standards (excluding the 100% cheaters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results and all the KPIs I looked at of this standard model do not indicate severe overfitting, etc. to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I submit my predictions for the &lt;code&gt;test&lt;/code&gt; set my &lt;strong&gt;public score is ~35%&lt;/strong&gt; which is way below even a random chance model. It is sooo bad I even improved my score by simply reversing all predictions from the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is my model so much worse on the &lt;code&gt;test&lt;/code&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that Kaggle computes their scores a bit differently than I do locally, additionally there is probably some differences between the datasets. Most who join the competition notices at least some difference between their local test scores and the public scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However my difference is really drastic and indeed reversing the predictions improves my score. This does not make sense to me because reversing the predictions on my local validations leads to garbage predictions, so this is not a simple problem of generally reversed predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So can you help me understand how those two issues happen at the same time:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Drastic difference between local accuracy and public score&lt;/li&gt;&#xA;&lt;li&gt;Reversing actually leads to the better public score.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is my notebook for the code (please ignore the errors, they are simply because the code does not work on kaggle kernels only locally):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.kaggle.com/fnguyen/titanicrising-test&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/fnguyen/titanicrising-test&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2020-02-05T14:23:36.780" Title="Why is my model accuracy high in train-test split but actually worse than chance in validation set?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17774" PostTypeId="1" AcceptedAnswerId="21600" CreationDate="2020-01-31T10:45:09.033" Score="3" ViewCount="140" Body="&lt;p&gt;I'm working on a continuous state / continuous action controller. It shall control a certain roll angle of an aircraft by issuing the correct aileron commands (in &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;To this end, I use a neural network and the DDPG algorithm, which shows promising results after about 20 minutes of training.&lt;/p&gt;&#xA;&lt;p&gt;I stripped down the presented state to the model to only the roll angle and the angular velocity, so that the neural network is not overwhelmed by state inputs.&lt;/p&gt;&#xA;&lt;p&gt;So it's a 2 input / 1 output model to perform the control task.&lt;/p&gt;&#xA;&lt;p&gt;In test runs, it looks mostly good, but sometimes, the controller starts thrashing, i.e. it outputs flittering commands, like in a very fast bangbang-controlm which causes a rapid movement of the elevator.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hMZ9I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hMZ9I.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Even though this behavior kind of maintains the desired target value, this behavior is absolutely undesirable. Instead, it should keep the output smooth. So far, I was not able to detect any special disturbance that starts this behavior. Yet it comes out of the blue.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Does anybody have an idea or a hint (maybe a paper reference) on how to incorporate some element (maybe reward shaping during the training) to avoid such behavior? How to avoid rapid actuator movements in favor of smooth movements?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried to include the last action in the presented state and add a punishment component in my reward, but this did not really help. So obviously, I do something wrong.&lt;/p&gt;&#xA;" OwnerUserId="25972" LastEditorUserId="2444" LastEditDate="2020-10-07T16:55:30.210" LastActivityDate="2020-10-07T16:55:30.210" Title="How to avoid rapid actuator movements in favor of smooth movements in a continuous space and action space problem?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;ddpg&gt;&lt;reward-shaping&gt;&lt;reward-design&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17775" PostTypeId="1" AcceptedAnswerId="17787" CreationDate="2020-01-31T11:49:49.323" Score="3" ViewCount="176" Body="&lt;p&gt;Why are the terms classification and prediction used as synonyms especially when it comes to deep learning? For example, a CNN predicts the handwritten digit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, a prediction is telling the next step in a sequence, whereas classification is to put labels on (a finite set of) data.&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-01-31T13:16:12.167" LastActivityDate="2020-02-01T17:20:55.910" Title="Why are the terms classification and prediction used as synonyms in the context of deep learning?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17776" PostTypeId="2" ParentId="17770" CreationDate="2020-01-31T13:27:44.003" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the network will have no trouble overfitting, given you train long enough. If you dont train long enough on the other hand, your network will probably not learn anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try with a smaller network, so you &quot;force&quot; it to learn something, instead of memorizing.&lt;/p&gt;&#xA;" OwnerUserId="31714" LastActivityDate="2020-01-31T13:27:44.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17777" PostTypeId="2" ParentId="17775" CreationDate="2020-01-31T13:38:31.667" Score="0" Body="&lt;p&gt;They aren't synonyms literally, books never interchange those terms, as they represent two different processes. What they are, though are two similar processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Classification&lt;/strong&gt; can be thought of a &quot;process&quot; that uses specific functions for the generation of &lt;strong&gt;one or more discrete values&lt;/strong&gt;, usually using a cross-entropy function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Prediction&lt;/strong&gt;, on the other hand can be thought of a &quot;process&quot; that uses, again specific functions for the generation of &lt;strong&gt;continuous values&lt;/strong&gt;, usually using a linear or multiple dependency model with a MSE loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL:DR; Classification is for discrete valued dependent variable, Prediction is for continuous valued dependent variable. Both are similar processes with different functions used for learning and estimation of the predicate. You can think of classification as a specific form of prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helped! Do let me know if I have made any mistakes.&lt;/p&gt;&#xA;" OwnerUserId="33092" LastActivityDate="2020-01-31T13:38:31.667" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17778" PostTypeId="2" ParentId="17755" CreationDate="2020-01-31T14:36:34.840" Score="1" Body="&lt;p&gt;Pragmatically you could use the discriminatory from a GAN for outlier detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally you'd start collecting fakes now and do a normal model on both good and bad cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the absence of that you can create a GAN to create realistic looking fakes on only real cases and then take the discriminator from that GAN to flag real-life cases for manual checks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please for a real life case always include these real life checks which also helps collecting cases for improving the model.&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2020-01-31T14:36:34.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17779" PostTypeId="1" CreationDate="2020-01-31T17:05:55.073" Score="3" ViewCount="40" Body="&lt;p&gt;In machine learning, problem space can be represented through concept space, instance space version space and hypothesis space. These problem spaces used the conjunctive space and are very restrictive one and also in the above-mentioned representations of problem spaces, it is not sure that the true concept lies within conjunctive space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, let's say, if we have a bigger search space and want to overcome the restrictive nature of conjunctive space, then how can we represent our problem space? Secondly, in a given scenario which algorithm is used for our problem space to represent the learning problem?&lt;/p&gt;&#xA;" OwnerUserId="33185" LastEditorUserId="2444" LastEditDate="2020-01-31T17:12:34.780" LastActivityDate="2020-01-31T17:12:34.780" Title="In machine learning, how can we overcome the restrictive nature of conjunctive space?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;genetic-algorithms&gt;&lt;problem-solving&gt;&lt;intelligence&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17780" PostTypeId="1" CreationDate="2020-01-31T17:20:53.297" Score="1" ViewCount="14" Body="&lt;p&gt;I am writing myself and was thinking about, what kind of metric can be applied to measure the &quot;dangerousness&quot; of a human being on a railtrack? For example detecting if a human is running on the rails?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe predicting the human movement towards rails, the distance to the vehicle (ego perspective) or something else?&lt;/p&gt;&#xA;" OwnerUserId="32337" LastActivityDate="2020-01-31T17:20:53.297" Title="How can a new metric applied for humans causing danger on railtracks?" Tags="&lt;computer-vision&gt;&lt;human-computer-interaction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17781" PostTypeId="2" ParentId="17369" CreationDate="2020-01-31T17:50:33.157" Score="0" Body="&lt;p&gt;I have finally abandoned the idea of ​​doing it with an exact method and I have passed to the heuristic. I have mixed multi-boot, local search and certain random movements. Apparently this is called Greedy Randomized Adaptative Search Procedures (GRASP)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hypothesis: the best solution is reached filling the 'operations' with the maximum of 'Actions' per 'Operation'. Other combinations are more expensive&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;create a random solution &lt;br&gt;&#xA;[[1 2 3 4] [5 6 7 8] [9 10]]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate the cost of each 'Operation' &lt;br&gt;&#xA;[[6] [6] [5]] == 17&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Study the posibility of minimize the lower 'Operation' permuting some 'Action' with the other 'Operations'.  &lt;br&gt;&#xA;Actions 4 &amp;amp; 7 permute with 9 &amp;amp; 10 &lt;br&gt;&#xA;[[1 2 3 9] [5 6 10 8] [4 7]]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the new cost &lt;br&gt;&#xA;[[6] [7] [1]] == 14&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat steps 2 to 4 until it is not possible to minimize the lower, then the second lower, then the third etc&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You will find soon a Local Minimum. If no more moves are allowed but the Local Minimum is not the Global Minimum, permute two random 'actions' between two random 'operations' and repeat the steps 2º to 5º.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With this method it is possible to find very fast a solution adjusted to the minimum global cost with a complexity of O(n·log(n)), where n is the amount of 'actions'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tested it with a random sample of 60 'actions' and 26 'resources' grouped in 'operations' of 6 'actions'. It toke arround 5 minutes to get a really good solution arround 40% better that the initial one &lt;strong&gt;(*)&lt;/strong&gt;, and 30 minutes after it only got better solution in about 1%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(*) the initial solution was not actually a random solution. Instead of that I used 'Ant Colony' algorithm with 15 ants to get a better initial solution and reduce the amount of iterations, but that is another history&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="6207" LastActivityDate="2020-01-31T17:50:33.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17783" PostTypeId="1" AcceptedAnswerId="17804" CreationDate="2020-01-31T20:01:42.897" Score="2" ViewCount="147" Body="&lt;p&gt;I am a bit confused about the depth of the convolutional filters in a CNN.&lt;/p&gt;&#xA;&lt;p&gt;At layer 1, there are usually about 40 3x3x3 filters. Each of these filters outputs a 2d array, so the total output of the first layer is 40 2d arrays.&lt;/p&gt;&#xA;&lt;p&gt;Does the next convolutional filter have a depth of 40? So, would the filter dimensions be 3x3x40?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-12-28T15:57:26.763" LastActivityDate="2020-12-28T15:57:26.763" Title="How is the depth of filters of hidden layers determined?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;filters&gt;&lt;convolutional-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17784" PostTypeId="1" CreationDate="2020-02-01T07:12:36.983" Score="3" ViewCount="117" Body="&lt;p&gt;How can I generate unique patterns, as they did for these Nutella jars? See, for example, the video &lt;a href=&quot;https://www.youtube.com/watch?v=sHYakhyvJps&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithm designs seven million different jars of Nutella&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gjkK5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gjkK5.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33194" LastEditorUserId="2444" LastEditDate="2020-02-01T16:05:21.710" LastActivityDate="2020-02-02T04:22:19.817" Title="How can I generate unique random patterns (similar to the ones in Nutella jars)?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;python&gt;&lt;image-processing&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17785" PostTypeId="2" ParentId="17764" CreationDate="2020-02-01T14:31:42.820" Score="4" Body="&lt;p&gt;&lt;em&gt;Before proceeding, it's important to note that ResNets, as pointed out &lt;a href=&quot;https://ai.stackexchange.com/q/18703/2444&quot;&gt;here&lt;/a&gt;, were not introduced to &lt;em&gt;specifically&lt;/em&gt; solve the VGP, but to improve learning in general. In fact, the authors of ResNet, in the original paper, noticed that neural networks without residual connections don't learn as well as ResNets, although they are using batch normalization, which, in theory, ensures that gradients should not vanish (&lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf#page=5&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 4.1&lt;/a&gt;). So, in this answer, I'm just giving a potential explanation of why ResNets &lt;strong&gt;may also&lt;/strong&gt; mitigate (or prevent to some extent) the VGP, but the cited research papers below also confirm that ResNets prevent the VGP. Given that I didn't fully read all the papers mentioned in this answer, the information in this answer may not be fully accurate.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The skip connections allow information to skip layers, so, in the forward pass, information from layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; can directly be fed into layer &lt;span class=&quot;math-container&quot;&gt;$l+t$&lt;/span&gt; (i.e. the activations of layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; are added to the activations of layer &lt;span class=&quot;math-container&quot;&gt;$l+t$&lt;/span&gt;), for &lt;span class=&quot;math-container&quot;&gt;$t \geq 2$&lt;/span&gt;, and, during the forward pass, the gradients can also flow unchanged from layer &lt;span class=&quot;math-container&quot;&gt;$l+t$&lt;/span&gt; to layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How exactly could this prevent the vanishing gradient problem (VGP)? The VGP occurs when the elements of the gradient (the partial derivatives with respect to the parameters of the NN) become exponentially small, so that the update of the parameters with the gradient becomes almost insignificant (i.e. if you add a very small number &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \epsilon \ll 1$&lt;/span&gt; to another number &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$d+\epsilon$&lt;/span&gt; is almost the same as &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;) and, consequently, the NN learns very slowly or not at all (considering also numerical errors). Given that these partial derivatives are computed with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;chain rule&lt;/a&gt;, this can easily occur, because you keep on multiplying small (finite-precision) numbers (please, have a look at how the chain rule works, if you're not familiar with it). For example, &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{5}\frac{1}{5} = \frac{1}{25}$&lt;/span&gt; and then &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{5}\frac{1}{25} = \frac{1}{125}$&lt;/span&gt;, and so on. The deeper the NN, the more likely the VGP can occur. This should be quite intuitive if you are familiar with the chain rule and the back-propagation algorithm (i.e. the chain rule). By allowing information to skip layers, layer &lt;span class=&quot;math-container&quot;&gt;$l+t$&lt;/span&gt; receives information from both layer &lt;span class=&quot;math-container&quot;&gt;$l+t-1$&lt;/span&gt; and layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; (unchanged, i.e. you do not perform multiplications). For example, to compute the activation of layer &lt;span class=&quot;math-container&quot;&gt;$l+t-1$&lt;/span&gt;, you perform the usual linear combination followed by the application of the non-linear activation function (e.g. ReLU). In this linear combination, you perform multiplications between numbers that could already be quite small, so the results of these multiplications are even smaller numbers. If you use &lt;a href=&quot;https://stats.stackexchange.com/q/174295/82135&quot;&gt;saturating activation functions (e.g. tanh)&lt;/a&gt;, this problem can even be aggravated. If the activation of layer &lt;span class=&quot;math-container&quot;&gt;$l+t$&lt;/span&gt; are even smaller than the activations of layer &lt;span class=&quot;math-container&quot;&gt;$l+t-1$&lt;/span&gt;, the addition of the information from layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; will make these activations bigger, thus, to some extent, they will prevent these activations from becoming exponentially small. A similar thing can be said for the back-propagation of the gradient.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, skip connections can mitigate the VGP, and so they can be used to train deeper NNs.&lt;/p&gt;&#xA;&lt;p&gt;These explanations are roughly consistent with the findings reported in the paper &lt;a href=&quot;https://arxiv.org/pdf/1605.06431.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Residual Networks Behave Like Ensembles of Relatively Shallow Networks&lt;/a&gt;, which states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our results reveal one of the key characteristics that seem to enable the training of very deep networks: Residual networks avoid the vanishing gradient problem by introducing &lt;strong&gt;short paths&lt;/strong&gt; which can carry gradient throughout the extent of very deep networks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1805.07477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Norm-Preservation: Why Residual Networks Can Become Extremely Deep?&lt;/a&gt;, the authors also discuss another desirable effect of skip connections.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We show theoretically and empirically that each residual block in ResNets is increasingly &lt;strong&gt;norm-preserving&lt;/strong&gt;, as the network becomes deeper&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T14:07:38.757" LastActivityDate="2021-01-17T14:07:38.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17786" PostTypeId="1" CreationDate="2020-02-01T15:39:28.850" Score="2" ViewCount="20" Body="&lt;p&gt;Consider a 2D snake game, where the snake has to eat food to become longer. It must avoid hitting walls and biting into her tail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a game could have a different amount of actions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3 actions: go straight, turn left, turn right (relative to crawling direction)&lt;/li&gt;&#xA;&lt;li&gt;4 actions: north, east, south, west (absolute direction on the 2D map)&lt;/li&gt;&#xA;&lt;li&gt;7 actions: a combination of option A and option B (leaves the preferred choice to the player)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While the game in principle is always the same, I would like to understand the impact of the amount of actions on the training of a neuronal network. One obvious thing is the number of output nodes of the neuronal network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case A (3 actions), the neuronal network cannot perform an incorrect action. Any of the 3 choices are valid moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case B (4 actions), the net IMHO needs to learn that going into opposite direction does not have the desired effect and the snake continues moving into the old direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case C (7 actions), the net needs to learn both, 1 action is always illegal and the 3 relative actions somehow map to the 3 absolute actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I consider the learning curve in these situations? Does option B need 25% more training than option A to achieve the same results (same fitness) (similar: option C needs 125% more training time)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is giving a negative reward for an impossible move considered cheating, because I do code the rules of the game into the reward logic?&lt;/p&gt;&#xA;" OwnerUserId="31627" LastActivityDate="2020-02-01T15:39:28.850" Title="What effect does increasing the actions in RL have?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;training&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17787" PostTypeId="2" ParentId="17775" CreationDate="2020-02-01T15:48:52.157" Score="2" Body="&lt;p&gt;Many people confuse and misuse the two terms, classification and prediction (or classify and predict).  This is because in many cases classification techniques are being used for prediction purposes which creates part of the confusion to others who then use the term ‘prediction’ (or ‘predict’) inappropriately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your understanding of the definitions of classification and prediction is mostly correct and you are absolutely correct that there are many people using the terms synonymously, sometimes correctly but I believe mostly erroneously.  There are many good articles elaborating on the two and I have added some links and excerpts at the end of this answer.  What these articles don't cover is that many forecasting (i.e. prediction) researchers and practitioners will use conventional classifiers to predict the future state of a time series or data sequence.  More advanced researchers and practitioners will use time-recurrent models, which learn temporal patterns.  These are still called classifiers but the for purpose of prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more papers written on this use of classifiers, conventional and time-recurrent type classifiers, for time series than the use of regressor models!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This adds to the confusion in the data science and machine learning community in the usage of the terms ‘classify’ and ‘predict'.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Galit Shmueli sums it up best in his paper, “To Explain or to Predict?”, where he states: &lt;strong&gt;“Conflation between explanation and prediction is common, yet the distinction must be understood for progressing scientific knowledge.”&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the opposite problem where people will confuse regression models with classification.  See the first article below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.fharrell.com/post/classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classification vs. Prediction, by Professor Frank Harrell&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excerpt:&#xA;&lt;em&gt;By not thinking probabilistically, machine learning advocates frequently utilize classifiers instead of using risk prediction models. The situation has gotten acute: many machine learning experts actually label logistic regression as a classification method (it is not). It is important to think about what classification really implies. Classification is in effect a decision. Optimum decisions require making full use of available data, developing predictions, and applying a loss/utility/cost function to make a decision that, for example, minimizes expected loss or maximizes expected utility. Different end users have different utility functions. In risk assessment this leads to their having different risk thresholds for action. Classification assumes that every user has the same utility function and that the utility function implied by the classification system is that utility function.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;To Explain or to Predict?, by Galit Shmueli&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Abstract. Statistical modeling is a powerful tool for developing and testing&#xA;theories by way of causal explanation, prediction, and description. In many&#xA;disciplines there is near-exclusive use of statistical modeling for causal explanation and the assumption that models with high explanatory power are&#xA;inherently of high predictive power. Conflation between explanation and prediction is common, yet the distinction must be understood for progressing&#xA;scientific knowledge. While this distinction has been recognized in the philosophy of science, the statistical literature lacks a thorough discussion of the&#xA;many differences that arise in the process of modeling for an explanatory versus a predictive goal. The purpose of this article is to clarify the distinction&#xA;between explanatory and predictive modeling, to discuss its sources, and to&#xA;reveal the practical implications of the distinction to each step in the modeling process.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.kdnuggets.com/faq/classification-vs-prediction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is the difference between classification and prediction?, from KDnuggets&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one does a decision tree analysis, what is the result? A classification? A prediction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gregory Piatetsky-Shapiro answers:&#xA;The decision tree is a classification model, applied to existing data. If you apply it to new data, for which the class is unknown, you also get a prediction of the class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assumption is that the new data comes from the similar distribution as the data you used to build your decision tree. In many cases this is a correct assumption and that is why you can use the decision tree for building a predictive model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Classification and Prediction are not the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gregory Piatetsky-Shapiro answers:&#xA;It is a matter of definition. If you are trying to classify existing data, e.g. group patients based on their known medical data and treatment outcome, I would call it a classification. If you use a classification model to predict the treatment outcome for a new patient, it would be a prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gabrielac adds&#xA;In the book &quot;Data Mining Concepts and Techniques&quot;, Han and Kamber's view is that predicting class labels is classification, and predicting values (e.g. using regression techniques) is prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other people prefer to use &quot;estimation&quot; for predicting continuous values.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-02-01T17:20:55.910" LastActivityDate="2020-02-01T17:20:55.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17788" PostTypeId="1" AcceptedAnswerId="17789" CreationDate="2020-02-01T17:30:30.487" Score="2" ViewCount="664" Body="&lt;p&gt;In respect of RL, is model-free and off-policy the same thing, just different terminology? If not, what are the differences? I've read that the policy can be thought of as 'the brain', or decision making part, of machine learning application, where it stores its learnings and refers to it when a new action is required in the new state.&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-01T19:32:34.427" LastActivityDate="2020-02-01T19:36:31.777" Title="Are model-free and off-policy algorithms the same?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;off-policy-methods&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17789" PostTypeId="2" ParentId="17788" CreationDate="2020-02-01T18:41:59.090" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In respect of RL, is model-free and off-policy the same thing, just different terminology? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, they are entirely different terms, with the only thing they have in common is that they are both ways in which an RL agent can vary. An agent is generally either working off-policy or on-policy, and is generally either model-based or model-free. These things can otherwise appear in all four combinations.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If not, what are the differences?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Model-based vs model-free&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A model-based learning agent uses knowledge of the environment dynamics in order to make predictions of expected outcomes. A model-free learning agent does not use such knowledge. The model here might be provided explicitly by the developer - that could be code for physics to predict a mechanical system, or it might be the rules of a board game that the agent is allowed to know and query to predict outcomes of actions before taking them. Models can also be learned statistically from experience, although that is harder to make effective.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;On-policy vs off-policy&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;An on-policy agent learns statistically about how it is currently acting, and assuming a control problem, then uses that knowledge to change how it should act in future. An off-policy agent can learn statistically from other observed behaviours (including its own past behaviour, or random and exploratory behaviour) and use that knowledge to understand how a different &lt;em&gt;target&lt;/em&gt; behaviour would perform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Off-policy learning is a strict generalisation of on-policy learning and includes on-policy as a special case. However, off-policy learning is also often harder to perform since observations typically contain less relevant data.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I've read that the policy can be thought of as 'the brain', or decision making part, of machine learning application, where it stores its learnings and refers to it when a new action is required in the new state.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's basically correct when considering how an agent learns how to behave in an environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are assigning a bit too much to the word policy here. A policy is strictly only the mapping from a state to an action (or probability distribution over actions), and often written &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;, i.e. the probability of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given the agent is in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The &quot;brain&quot; part might include how the agent learns that policy. That could include storing past experience or some summary of past experience in e.g. a neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, outside of machine learning context, a really simple function containing and if/then statement would also be a policy, if the input to the function was a state of the environment, and the output was an action or probabilities of taking a range of actions. Behaving completely randomly is also a policy, but outside of very specific environments (e.g. Rock/Paper/Scissors) it is usually not the optimal thing to do.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-02-01T19:36:31.777" LastActivityDate="2020-02-01T19:36:31.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17790" PostTypeId="1" CreationDate="2020-02-02T02:11:39.033" Score="1" ViewCount="112" Body="&lt;p&gt;I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). &lt;a href=&quot;https://arxiv.org/abs/1605.08803&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is the link to the RealNVP paper and &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/10/13/flow-based-deep-generative-models.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is a nice, short explanation of it (the paper is pretty long). My code is mainly based on &lt;a href=&quot;https://github.com/tonyduan/normalizing-flows&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; code from GitHub and below are the main piece that I am using (with slight adjustments). The problem is that the loss is getting negative, which in the definition of my code means that the log-probability of the my data is positive, which in turn means that the probabilities are bigger than 1. This is impossible mathematically, and I see no way how this can happen, from a mathematical point of view. I also couldn't find a mistake in my code. Can someone help me with this? Is there a mistake there? Am I missing something with my understanding of normalizing flows? Thank you! &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class NormalizingFlowModel(nn.Module):&#xA;&#xA;    def __init__(self, prior, flows):&#xA;        super().__init__()&#xA;        self.prior = prior&#xA;        self.flows = nn.ModuleList(flows)&#xA;&#xA;    def forward(self, x):&#xA;        m, _ = x.shape&#xA;        log_det = torch.zeros(m).cuda()&#xA;        for flow in self.flows:&#xA;            x, ld = flow.forward(x)&#xA;            log_det += ld&#xA;        z, prior_logprob = x, self.prior.log_prob(x)&#xA;        return z, prior_logprob, log_det&#xA;&#xA;    def inverse(self, z):&#xA;        m, _ = z.shape&#xA;        log_det = torch.zeros(m).cuda()&#xA;        for flow in self.flows[::-1]:&#xA;            z, ld = flow.inverse(z)&#xA;            log_det += ld&#xA;        x = z&#xA;        return x, log_det&#xA;&#xA;    def sample(self, n_samples):&#xA;        z = self.prior.sample((n_samples,))&#xA;        x, _ = self.inverse(z)&#xA;        return x&#xA;&#xA;&#xA;class FCNN_for_NVP(nn.Module):&#xA;    &quot;&quot;&quot;&#xA;    Simple fully connected neural network to be used for Real NVP&#xA;    &quot;&quot;&quot;&#xA;    def __init__(self, in_dim, out_dim):&#xA;        super().__init__()&#xA;        self.network = nn.Sequential(&#xA;            nn.Linear(in_dim, 32),&#xA;            nn.Tanh(),&#xA;            nn.Linear(32, 32),&#xA;            nn.Tanh(),&#xA;            nn.Linear(32, 64),&#xA;            nn.Tanh(),&#xA;            nn.Linear(64, 64),&#xA;            nn.Tanh(),&#xA;            nn.Linear(64, 32),&#xA;            nn.Tanh(),&#xA;            nn.Linear(32, 32),&#xA;            nn.Tanh(),&#xA;            nn.Linear(32, out_dim),&#xA;        )&#xA;&#xA;    def forward(self, x):&#xA;        return self.network(x)&#xA;&#xA;&#xA;class RealNVP(nn.Module):&#xA;    &quot;&quot;&quot;&#xA;    Non-volume preserving flow.&#xA;&#xA;    [Dinh et. al. 2017]&#xA;    &quot;&quot;&quot;&#xA;    def __init__(self, dim, base_network=FCNN_for_NVP):&#xA;        super().__init__()&#xA;        self.dim = dim&#xA;        self.t1 = base_network(dim // 2, dim // 2)&#xA;        self.s1 = base_network(dim // 2, dim // 2)&#xA;        self.t2 = base_network(dim // 2, dim // 2)&#xA;        self.s2 = base_network(dim // 2, dim // 2)&#xA;&#xA;    def forward(self, x):&#xA;        lower, upper = x[:,:self.dim // 2], x[:,self.dim // 2:]      &#xA;        t1_transformed = self.t1(lower)&#xA;        s1_transformed = self.s1(lower)&#xA;        upper = t1_transformed + upper * torch.exp(s1_transformed)&#xA;        t2_transformed = self.t2(upper)&#xA;        s2_transformed = self.s2(upper)&#xA;        lower = t2_transformed + lower * torch.exp(s2_transformed)&#xA;        z = torch.cat([lower, upper], dim=1)&#xA;        log_det = torch.sum(s1_transformed, dim=1) + torch.sum(s2_transformed, dim=1)&#xA;        return z, log_det&#xA;&#xA;    def inverse(self, z):&#xA;        lower, upper = z[:,:self.dim // 2], z[:,self.dim // 2:]&#xA;        t2_transformed = self.t2(upper)&#xA;        s2_transformed = self.s2(upper)&#xA;        lower = (lower - t2_transformed) * torch.exp(-s2_transformed)&#xA;        t1_transformed = self.t1(lower)&#xA;        s1_transformed = self.s1(lower)&#xA;        upper = (upper - t1_transformed) * torch.exp(-s1_transformed)&#xA;        x = torch.cat([lower, upper], dim=1)&#xA;        log_det = torch.sum(-s1_transformed, dim=1) + torch.sum(-s2_transformed, dim=1)&#xA;        return x, log_det&#xA;&#xA;flow = RealNVP(dim=data.size(1))&#xA;flows = [flow for _ in range(1)]&#xA;prior = MultivariateNormal(torch.zeros(data.size(1)).cuda(), torch.eye(data.size(1)).cuda())&#xA;model = NormalizingFlowModel(prior, flows)&#xA;model = model.cuda()&#xA;&#xA;for i in range(10):&#xA;    for j, dtt in enumerate(my_dataloader_bkg_only):&#xA;        optimizer.zero_grad()&#xA;        x = dtt[0].float()&#xA;        z, prior_logprob, log_det = model(x)&#xA;        logprob = prior_logprob + log_det&#xA;        loss = -torch.mean(prior_logprob + log_det)&#xA;        loss.backward()&#xA;        optimizer.step()&#xA;    if i % 1 == 0:&#xA;        print(&quot;Saved&quot;)&#xA;        best_loss = logprob.mean().data.cpu().numpy()&#xA;        print(logprob.mean().data.cpu().numpy(), prior_logprob.mean().data.cpu().numpy(),&#xA;                  log_det.mean().data.cpu().numpy())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22839" LastActivityDate="2021-02-07T13:02:45.257" Title="RealNVP gives wrong probabilities" Tags="&lt;loss-functions&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17791" PostTypeId="1" CreationDate="2020-02-02T02:28:33.913" Score="2" ViewCount="49" Body="&lt;p&gt;Whenever I look for papers involving semi-supervised learning, I always find some that talk about graph semi-supervised learning (e.g. &lt;a href=&quot;https://arxiv.org/pdf/1910.14147v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between graph semi-supervised learning and normal semi-supervised learning?&lt;/p&gt;&#xA;" OwnerUserId="31240" LastEditorUserId="2444" LastEditDate="2020-04-28T01:22:42.347" LastActivityDate="2020-04-28T01:23:56.603" Title="What is the difference between graph semi-supervised learning and normal semi-supervised learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;semi-supervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17792" PostTypeId="2" ParentId="17784" CreationDate="2020-02-02T04:22:19.817" Score="1" Body="&lt;p&gt;Some excerpts from &lt;a href=&quot;https://news.ycombinator.com/item?id=14553261&quot; rel=&quot;nofollow noreferrer&quot;&gt;    Nutella 'Hired' an Algorithm to Design New Jars. And It Was a Sell-Out Success&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;algorithm&quot; is called HP Mosaic and is included free in HP SmartStream Designer for HP printers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More about how the algorithm works here: &lt;a href=&quot;https://www.linkedin.com/pulse/hp-mosaic-20-steven-chow&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.linkedin.com/pulse/hp-mosaic-20-steven-chow&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HP Mosaic takes the vector PDF file as input (also known as a Seed file), and generates a large number of variations on the file by transforming it — scaling, transposition, and rotation — randomly.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-02-02T04:22:19.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17793" PostTypeId="1" CreationDate="2020-02-02T04:57:14.270" Score="2" ViewCount="133" Body="&lt;p&gt;Update&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Checked the PyTorch Mobile which is designed to Android and iOS. Although according to the document, it says it can build for ARM CPUs, but there isn't any documentation mention about how to use it on Raspberry Pi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess, at this stage, PyTorch doesn't have any official support for Rasberry Pi and there is no timeline for the Pi support. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will leave this question for a couple of days and close it if there isn't any answers.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I just trained my PyTorch ResNet34 model and used on my Raspberry Pi 4. But it is slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am wondering what is the equivalent PyTorch version of Tensorflow lite??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once, I figured it out, I guess I should be able to convert my PyToch model to, for example, PyTorch Lite model to increase the speed of inference.&lt;/p&gt;&#xA;" OwnerUserId="27988" LastEditorUserId="27988" LastEditDate="2020-02-02T06:27:26.957" LastActivityDate="2020-02-02T06:27:26.957" Title="What is the equivalent PyTorch version of tensorflow lite" Tags="&lt;tensorflow&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17795" PostTypeId="1" CreationDate="2020-02-02T12:34:16.003" Score="3" ViewCount="148" Body="&lt;p&gt;I'm quite new to reinforcement learning and my project will consist of detecting lanes with RL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using q-learning and I'm having a hard time thinking how my q table should look like, I mean - what could represent a state. My main idea is to feed the machine with a frame that contains a road picture, which the edge detection function is being applied to (and by thus getting lots of lines that exits in the frame). And train the machine which lines are the correct lane line. I already have a deterministic function that already recognizes the lanes and it will be the function that will teach the machine. I already organized some lane parameters such as (lane length, lane cords, lane color (white or yellow have a better probability to be a lane), lane diameter and the lane incline). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my only issue is how should I construct the Q-table. Basically, what could represent a state and which lanes or decisions I should reward.&lt;/p&gt;&#xA;" OwnerUserId="33220" LastEditorUserId="2444" LastEditDate="2020-02-03T11:28:09.507" LastActivityDate="2020-10-30T13:01:20.140" Title="How can I perform lane detection with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17796" PostTypeId="1" AcceptedAnswerId="18420" CreationDate="2020-02-02T13:18:05.707" Score="2" ViewCount="246" Body="&lt;p&gt;How would you go about training an RL Tic Tac Toe (well, any board game, really) application to learn how to play successfully (win the game), without a human having to play against the RL? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, it would take a lot longer to train the AI if I have to sit and play &quot;real&quot; games against it. Is there a way for me to automate the training? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess creating &quot;a human player&quot; to train the AI who just selects random positions on the board won't help the AI to learn properly, as it won't be up against something that's not using a strategy to try to beat it.&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-03T11:31:10.433" LastActivityDate="2020-03-04T19:12:49.780" Title="How can I train a RL agent to play board games successfully without human play?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;game-ai&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17798" PostTypeId="2" ParentId="17768" CreationDate="2020-02-02T15:51:37.737" Score="5" Body="&lt;p&gt;There are various dataset available such as &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Pascal VOC dataset: You can perform all your task with these. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/WZ1Ho.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WZ1Ho.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;size of the dataset is as  follows&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vl0JT.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vl0JT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ADE20K Semantic Segmentation Dataset: you can perform only segmentation here&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mwoHl.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mwoHl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;COCO dataset: This is rich dataset but a size larger then 5 GB so you can try downloading using google colab in your drive and then make a zip file of data as less than 5 GB&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3DwkV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3DwkV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can download all these datasets using Gluoncv easily.here &lt;a href=&quot;https://gluon-cv.mxnet.io/api/data.datasets.html&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15368" LastEditorUserId="15368" LastEditDate="2020-02-03T15:41:34.700" LastActivityDate="2020-02-03T15:41:34.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17799" PostTypeId="2" ParentId="17796" CreationDate="2020-02-02T16:09:27.490" Score="0" Body="&lt;p&gt;One solution would be to simply play the AI against itself (which yields good results for the tic tac toe example, I've tried it), but a much more interesting approach is to train two networks at the same time and have them play against each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's called a Generative adversarial network (GAN) and it's very promising. It is able to make &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network#/media/File:Woman_1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;realistic images&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;much more&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's quite easy to implement both solutions so I'd recommend doing both and seeing for yourself since each problem is different.&lt;/p&gt;&#xA;" OwnerUserId="33226" LastActivityDate="2020-02-02T16:09:27.490" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17800" PostTypeId="2" ParentId="17795" CreationDate="2020-02-02T16:19:43.857" Score="2" Body="&lt;p&gt;I will agree with malioboro, maybe RL is an overkill for such a task. Even though with the trend of autonomous driving research lately, papers dealing with lane changing almost certainly exist, you should check them out for more details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As stated in &lt;a href=&quot;https://arxiv.org/pdf/1904.00231.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lane Change Decision-making through DeepReinforcement Learning with Rule-basedConstraints&lt;/a&gt; , &quot;When the states are discrete and finite, the Q-function canbe easily formulated in a tabular form. But in many practical applications, for example, lane change decision-making task,the state space of them is very large or even continuous,using the Q-learning algorithm will lead to dimension disaster. Therefore, tabular Q-learning algorithm does not applicable to&#xA;the learning problem of continuous state space and continuousaction space&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, i would recommend using image processing techniques, or Deep-Q-Learning. Q-Learning is not capable of dealing with continuous problems like lane changing or lane tracking.&lt;/p&gt;&#xA;" OwnerUserId="33225" LastActivityDate="2020-02-02T16:19:43.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17801" PostTypeId="1" CreationDate="2020-02-02T16:59:17.387" Score="4" ViewCount="50" Body="&lt;p&gt;In many papers about artificial spiking neural networks (SNNs), the performance of them is not up to par with traditional ANNs. I have read how some people have converted ANNs to SNNs using various techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been work done on using unsupervised learning in SNN to recognise MNIST digits through &lt;a href=&quot;https://en.wikipedia.org/wiki/Spike-timing-dependent_plasticity&quot; rel=&quot;nofollow noreferrer&quot;&gt;spike-timing-dependent plasticity&lt;/a&gt; (for example, the paper &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fncom.2015.00099/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised learning of digit recognition using spike-timing-dependent plasticity&lt;/a&gt;, by Diehl and Cook, 2015). This form of learning is not possible in traditional ANNs due to their synchronous nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering would it be a good idea to first train an SNN in an unsupervised manner to learn some of the structure of the data. Then convert to a traditional ANN to take advantage of their superior performance with some more training. I can see this being useful for training a network on a sparsely labelled dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am quite a novice in this area, so I was looking for feedback on any immediate barriers as to why this would not work or if it is even worth doing.&lt;/p&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-02-02T20:05:10.060" LastActivityDate="2020-02-02T20:05:10.060" Title="Is it a good idea to first train a spiking neural network and then convert it to a conventional neural network?" Tags="&lt;neural-networks&gt;&lt;biology&gt;&lt;spiking-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17802" PostTypeId="1" AcceptedAnswerId="17854" CreationDate="2020-02-02T17:40:08.373" Score="0" ViewCount="79" Body="&lt;p&gt;I have to build a neural network without any architecture limitations which have to predict the next value of a time series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dataset is composed of 400.000 values, which are given in hex format. For example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0xbfb22b14&#xA;0xbfb22b10&#xA;0xbfb22b0c&#xA;0xbfb22b18&#xA;0xbfb22b14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think LSTM is suitable for this problem, but I am worried about the length of the input. Would it be a good idea to use CNN?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def structure(step,n_features):&#xA;    # define model&#xA;    model = Sequential()&#xA;    model.add(LSTM(50, activation='relu', return_sequences=True, input_shape=(step, n_features)))&#xA;    model.add(Dense(1))&#xA;    model.compile(optimizer='adam', loss='mse')&#xA;    return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What about this one ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;model&quot;: {&#xA;        &quot;loss&quot;: &quot;mse&quot;,&#xA;        &quot;optimizer&quot;: &quot;adam&quot;,&#xA;    &quot;save_dir&quot;: &quot;saved_models&quot;,&#xA;        &quot;layers&quot;: [&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 999,&#xA;                &quot;input_timesteps&quot;: 998,&#xA;                &quot;input_dim&quot;: 1,&#xA;                &quot;return_seq&quot;: true&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.05&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;lstm&quot;,&#xA;                &quot;neurons&quot;: 100,&#xA;                &quot;return_seq&quot;: false&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dropout&quot;,&#xA;                &quot;rate&quot;: 0.05&#xA;            },&#xA;            {&#xA;                &quot;type&quot;: &quot;dense&quot;,&#xA;                &quot;neurons&quot;: 1,&#xA;                &quot;activation&quot;: &quot;linear&quot;&#xA;            }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32076" LastEditorUserId="32076" LastEditDate="2020-02-05T13:48:46.623" LastActivityDate="2020-02-05T13:48:46.623" Title="What's the best architecture for time series prediction with a long dataset?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;prediction&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17803" PostTypeId="1" AcceptedAnswerId="17852" CreationDate="2020-02-02T18:22:27.487" Score="4" ViewCount="343" Body="&lt;p&gt;I am reading the Understanding Machine Learning book by Shalev-Shwartz and Ben-David and based on the definitions of PAC learnability and No Free Lunch Theorem, and my understanding of them it seems like they contradict themselves. I know this is not the case and I am wrong, but I just don't know what I am missing here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, a hypothesis class is (agnostic) PAC learnable if there exists a learner A and a function &lt;span class=&quot;math-container&quot;&gt;$m_{H}$&lt;/span&gt; s.t. for every &lt;span class=&quot;math-container&quot;&gt;$\epsilon,\delta \in (0,1)$&lt;/span&gt; and &lt;strong&gt;for every distribution &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$X \times Y$&lt;/span&gt;&lt;/strong&gt;, if &lt;span class=&quot;math-container&quot;&gt;$m \geq m_{H}$&lt;/span&gt; the learner can return a hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, with a probability of at least &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ L_{D}(h) \leq min_{h'\in H} L_{D}(h') + \epsilon $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, in layman's terms, the NFL theorem states that for prediction tasks, for every learner there &lt;strong&gt;exists a distribution&lt;/strong&gt; on which the learner fails.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There needs to exists a learner that is successful (defined above) for every distribution &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$X \times Y$&lt;/span&gt; for a hypothesis to be PAC learnable, but according to NFL there exists a distribution for which the learner will fail, aren't these theorems contradicting themselves?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing or misinterpreting here?&lt;/p&gt;&#xA;" OwnerUserId="31757" LastEditorUserId="9947" LastEditDate="2020-03-26T00:01:32.730" LastActivityDate="2020-03-26T12:34:20.780" Title="Are PAC learnability and the No Free Lunch theorem contradictory?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;&lt;no-free-lunch-theorems&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17804" PostTypeId="2" ParentId="17783" CreationDate="2020-02-02T20:53:59.107" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Does the next convolutional filter have a depth of 40? So, would the filter dimensions be 3x3x40?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. The depth of the next layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; (which corresponds to the number of feature maps) will be 40. If you apply &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; kernels with a &lt;span class=&quot;math-container&quot;&gt;$3\times 3$&lt;/span&gt; window to &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, then the number of features maps (or the depth) of layer &lt;span class=&quot;math-container&quot;&gt;$l+1$&lt;/span&gt; will be &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt;. Each of these &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; kernels has an actual shape of &lt;span class=&quot;math-container&quot;&gt;$3 \times 3 \times 40$&lt;/span&gt;. Bear in mind that the details of the implementations may change across different libraries.&lt;/p&gt;&#xA;&lt;p&gt;The following simple TensorFlow (version 2.1) and Keras program&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;&#xA;def get_model(input_shape, num_classes=10):&#xA;    model = tf.keras.Sequential()&#xA;    model.add(tf.keras.layers.Input(shape=input_shape))&#xA;    model.add(tf.keras.layers.Conv2D(40, kernel_size=3))&#xA;    model.add(tf.keras.layers.Conv2D(8, kernel_size=3))&#xA;    model.add(tf.keras.layers.Flatten())&#xA;    model.add(tf.keras.layers.Dense(num_classes))&#xA;&#xA;    model.summary()&#xA;&#xA;    return model&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    input_shape = (28, 28, 1)  # MNIST digits have usually this shape.&#xA;    get_model(input_shape)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;outputs the following&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Model: &amp;quot;sequential&amp;quot;&#xA;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;conv2d (Conv2D)              (None, 26, 26, 40)        400       &#xA;_________________________________________________________________&#xA;conv2d_1 (Conv2D)            (None, 24, 24, 8)         2888      &#xA;_________________________________________________________________&#xA;flatten (Flatten)            (None, 4608)              0         &#xA;_________________________________________________________________&#xA;dense (Dense)                (None, 10)                46090     &#xA;=================================================================&#xA;Total params: 49,378&#xA;Trainable params: 49,378&#xA;Non-trainable params: 0&#xA;_________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;conv2d&lt;/code&gt; has the output shape &lt;code&gt;(None, 26, 26, 40)&lt;/code&gt; because there are 40 filters, each of which will have a &lt;span class=&quot;math-container&quot;&gt;$3\times 3 \times 40$&lt;/span&gt; shape.&lt;/p&gt;&#xA;&lt;p&gt;The documentation of the first argument (i.e. &lt;code&gt;filters&lt;/code&gt;) of the &lt;code&gt;Conv2D&lt;/code&gt; says&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;filters&lt;/code&gt; – Integer, the &lt;strong&gt;dimensionality of the output space&lt;/strong&gt; (i.e. the number of output filters in the convolution).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and the documentation of the &lt;code&gt;kernel_size&lt;/code&gt; parameter states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;kernel_size&lt;/code&gt; – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It doesn't actually say anything about the depth of the kernels, but this is implied from the depth of the layers.&lt;/p&gt;&#xA;&lt;p&gt;Note that the first layer has &lt;span class=&quot;math-container&quot;&gt;$(40*(3*3*1))+40 = 400$&lt;/span&gt; parameters. Where do these numbers come from? Note also that the second &lt;code&gt;Conv2D&lt;/code&gt; layer has &lt;span class=&quot;math-container&quot;&gt;$(8*(3*3*40))+8 = 2888$&lt;/span&gt; parameters. Try to set the parameter &lt;code&gt;use_bias&lt;/code&gt; of the first &lt;code&gt;Conv2D&lt;/code&gt; layer to &lt;code&gt;False&lt;/code&gt; and see the number of parameters again.&lt;/p&gt;&#xA;&lt;p&gt;Finally, note that this reasoning applies to 2d convolutions. In the case of 3d convolutions, the depth of the kernels could be different than the depth of the input. Check &lt;a href=&quot;https://ai.stackexchange.com/a/18669/2444&quot;&gt;this answer&lt;/a&gt; for more details about 3d convolutions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-28T15:53:14.967" LastActivityDate="2020-12-28T15:53:14.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17807" PostTypeId="2" ParentId="17151" CreationDate="2020-02-03T05:19:43.900" Score="0" Body="&lt;p&gt;I have explored the AI for edge devices. My findings for &lt;code&gt;tflite&lt;/code&gt; model.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;TFLite&lt;/code&gt; is just the tool suite to convert &lt;code&gt;TFModel&lt;/code&gt; into &lt;code&gt;TFLite&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;TFLite optimizes the model for edge or embedded devices using &lt;a href=&quot;https://medium.com/analytics-vidhya/optimization-techniques-tflite-5f6d9ae676d5&quot; rel=&quot;nofollow noreferrer&quot;&gt;quantization techniques&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Quantization dramatically reduces both the memory requirement and&#xA;  computational cost of using neural networks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answer in brief:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When we optimize the model definitely they are faster to take &#xA;inference but it will impact the accuracy. Quantization leads to bit accuracy loss in smaller networks.&lt;/li&gt;&#xA;&lt;li&gt;TFLite is increasing the model performance but we have to pay a cost in terms of accuracy that's why we have both TFLite for edge computation and TFModel as well.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2020-02-03T05:19:43.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17808" PostTypeId="1" AcceptedAnswerId="17858" CreationDate="2020-02-03T06:20:26.880" Score="2" ViewCount="112" Body="&lt;p&gt;I came across this graph in David Silver's youtube lecture and Sutton's book on reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me understand the graph?&#xA;From the graph, for 10000 episodes what i  see is that when we don't have a usable ace we always lose the game except if the sum is 20 or 21. But in case if we have a usable ace there are chances to win when our sum is below 20. I don't know how this is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hZX2W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hZX2W.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="1847" LastEditDate="2020-02-05T12:37:28.150" LastActivityDate="2020-02-05T12:41:22.060" Title="What does the figure &quot;Blackjack Value Function...&quot; from Sutton represent?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;sutton-barto&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17809" PostTypeId="1" CreationDate="2020-02-03T06:46:30.093" Score="0" ViewCount="103" Body="&lt;p&gt;I want to make a model that outputs the centre pixel of objects appearing in an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current method involves using a CNN with L2 loss to output an image of equivalent size to the input where each pixel has a value of 1 if it is the center of an object and 0 otherwise. Each input image has roughly ~80 objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with this is the CNN learns the easiest way to reduce the error, which is having the entire output be 0, because for 97% of cases that's correct. As such, error decreases but it learns nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is another potential method for training a network to do something similar? I also tried adding dropout, which made the output a lot more noisy and it seemed to learn ok, but eventually ended up in the same state as before with the entire output being 0, never really seeming to learn how to output the locations of objects.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2021-03-03T11:06:35.703" Title="Possible model to use to find pixel locations of objects" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17810" PostTypeId="1" CreationDate="2020-02-03T08:59:23.303" Score="3" ViewCount="212" Body="&lt;p&gt;I was going through David Silver's lecture on reinforcement learning (lecture 4). At 51:22 he says that Monte Carlo (MC) methods have high variance and zero bias. I understand the zero bias part. It is because it is using the true value of value function for estimation. However, I don't understand the high variance part. Can someone enlighten me?&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="2444" LastEditDate="2020-02-03T11:17:09.893" LastActivityDate="2020-02-04T07:30:00.203" Title="How does Monte Carlo have high variance?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17811" PostTypeId="2" ParentId="6880" CreationDate="2020-02-03T09:14:33.997" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.cs.toronto.edu/~fritz/absps/imagenet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alexnet&lt;/a&gt; (2012), &lt;a href=&quot;https://arxiv.org/abs/1312.6229&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overfeat&lt;/a&gt; (2013), &lt;a href=&quot;https://arxiv.org/abs/1409.1556&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG&lt;/a&gt; (2014) and &lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNet&lt;/a&gt; (2016) are cited in many image recognition or segmentation applications. There is also &lt;a href=&quot;https://arxiv.org/abs/1409.4842&quot; rel=&quot;nofollow noreferrer&quot;&gt;GoogleLeNet&lt;/a&gt; (2015). &#xA;The lastest is the publication the denser is the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ResNet publication comments on how the network density affects accuracy depending on the image data set size. The article tends to give a motivated answer to the question &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is learning better networks as easy as stacking more layers?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You might consider the training time since you have your own image data set depending on the kind of hardware you can you use ( &lt;a href=&quot;https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this benchmark&lt;/a&gt; for instance ). The denser the more time it will takes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also have to consider the size of the traning data set w/r to the expected accuracy. If the set is too small the net will probably overfeat. In that case you migh consider a data augmentation strategy (one of the answers mentions &lt;em&gt;auto encoding&lt;/em&gt;, I m not sure but this might help for this purpose).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All these publications refer to the &lt;a href=&quot;http://www.image-net.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ImageNet&lt;/a&gt; data base and the associated image classification/detection &lt;a href=&quot;http://www.image-net.org/challenges/LSVRC/&quot; rel=&quot;nofollow noreferrer&quot;&gt;contest&lt;/a&gt; which has 1000 classes.&lt;/p&gt;&#xA;" OwnerUserId="30392" LastActivityDate="2020-02-03T09:14:33.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17812" PostTypeId="2" ParentId="17768" CreationDate="2020-02-03T12:42:58.023" Score="1" Body="&lt;p&gt;Maybe try the COCO (common objects in context) dataset. &#xA;It's often used for object detection, segmentation and localisation. &#xA;They provide labels, and you can limit the size by downloading only a specific number of classes.&#xA;&lt;a href=&quot;http://cocodataset.org/#explore&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cocodataset.org/#explore&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also quite a common one, so you can expect good documentation, and online answers to your questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="31180" LastActivityDate="2020-02-03T12:42:58.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17813" PostTypeId="2" ParentId="17810" CreationDate="2020-02-03T13:18:47.480" Score="2" Body="&lt;p&gt;When using terms like &quot;high&quot; for high variance, this is in comparison to other methods, mainly in comparison to TD learning, which bootstraps between single time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth spelling out what the variance applies to and where it comes from: Namely the &lt;em&gt;Monte Carlo return&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; distribution, which can be calculated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t = \sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a common convention here by the way that capital letters such as &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; stand for &lt;em&gt;random distributions&lt;/em&gt; of variables and lower case letters such as &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt; stand for examples/samples from those distributions. It is not always strictly used, especially in pseudo-code, but is important to make clear for this answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a value function, such as action value function &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)$&lt;/span&gt; can be written:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) = \mathbb{E}_{\pi} [ \sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1} | S_t=s, A_t=a] = \mathbb{E}_{\pi} [ G_t | S_t=s, A_t=a]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. it is fairly clear that the mean of Monte Carlo return &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[G_t]$&lt;/span&gt; from any specific starting point is the same as the value function from the same starting point, by definition (see Sutton &amp;amp; Barto or David Silver's definitions) so samples from it - taking specific measures &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt; - are unbiased estimates of it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variance &lt;span class=&quot;math-container&quot;&gt;$\text{Var}(G_t)$&lt;/span&gt; might be written&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{Var}(G_t) = \text{Var}(\sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1}) \le \sum_{k=0}^{T-t-1}\gamma^k \text{Var}(R_{t+k+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second step is technically equal only in a &quot;worst case&quot; scenario since to add variances directly, we need the variables to be independently sampled, whilst &lt;span class=&quot;math-container&quot;&gt;$R_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; could be correlated, and often are. So, technically we should account for covariance here between distribitions on different time steps. The maths for that is fiddly, and I won't show it here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can say intuitively about the inequality, is unless &lt;span class=&quot;math-container&quot;&gt;$R_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; are perfectly correlated throughout each trajectory, then variance of &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; will increase depending on the number of timesteps and possible rewards in the return, by some fraction between 0 and 1 of the total variance of all time steps. You can either add zero variance on a timestep, or some fraction of &lt;span class=&quot;math-container&quot;&gt;$\text{Var}(R_{t+n})$&lt;/span&gt; depending on whether the relationship between [&lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$R_{t+n-1}$&lt;/span&gt;] and &lt;span class=&quot;math-container&quot;&gt;$R_{t+n}$&lt;/span&gt; is fully deterministic (&lt;span class=&quot;math-container&quot;&gt;$+0$&lt;/span&gt;) or completely independent (&lt;span class=&quot;math-container&quot;&gt;$+\gamma^{n-1}\text{Var}(R_{t+n})$&lt;/span&gt;)*&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The amount of covariance involved relies on details of the policy, state transition and reward functions. In general, the policy during RL will include some stochastic behaviour -such as &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy that will add to the variance on each time step. The state transition and reward functions &lt;em&gt;might&lt;/em&gt; add yet more. In any case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{Var}(G_t) \ge \text{Var}(R_{t+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and in practice, when &lt;span class=&quot;math-container&quot;&gt;$t+1 \neq T$&lt;/span&gt;, the inequality can be large, although highly dependent on the details of the MDP. For off-policy Monte Carlo with basic importance sampling in fact the variance can be unbounded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This variance might be considered high, compared to the variance of typical single-step TD returns:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+1} = R_{t+1} + \gamma q_{\pi}(S_t, A_t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variance of which is limited to that inherent in a single time step (one reward function, one state transition and one policy decision). The problem of course being that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[G_{t:t+1}] \neq \mathbb{E}[G_{t}]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;. . . unless &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s, a)$&lt;/span&gt; is perfect. Which is why we can say that TD returns are biased (&lt;em&gt;whilst &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s, a)$&lt;/span&gt; is not converged, the expectation of the TD target does not equal the true value&lt;/em&gt;), but Monte Carlo returns are higher variance (&lt;em&gt;when number of timesteps to end of trajectory is more than one, the effects of random factors in policy, state transition and reward function accumulate&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* You could explicitly construct an environment with covariance between time steps that can reduce overall variance of a MDP's retrun, e.g. the state includes reward so far, and future rewards are manipulated so that the return fits to a certain distribution. This would be a special case where variance of the return from &lt;em&gt;some&lt;/em&gt; starting points could be lower than the variance from the first step, but this is an unusual set up and likely to be impossible for an agent to learn optimal control. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-02-04T07:30:00.203" LastActivityDate="2020-02-04T07:30:00.203" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17814" PostTypeId="2" ParentId="17773" CreationDate="2020-02-03T14:03:07.587" Score="1" Body="&lt;p&gt;Looking at your code, one set of data transformations were applied to the train data and a different set of transformations were applied to the test data. Different data transformations could account for different evaluation metric performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is best practices to put all data transformations in a function so they can be applied to all data in a similar way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are using scikit-learn, &lt;code&gt;sklearn.compose.ColumnTransformer&lt;/code&gt; is designed for this purpose. Example code for the Titanic dataset is &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15403" LastActivityDate="2020-02-03T14:03:07.587" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17815" PostTypeId="1" CreationDate="2020-02-03T14:04:37.907" Score="2" ViewCount="73" Body="&lt;p&gt;Given a list of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; questions. If question &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is answered correctly (given probability &lt;span class=&quot;math-container&quot;&gt;$p_i$&lt;/span&gt;), we receive reward &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt;; if not the quiz terminates. Find the optimal order of questions to maximize expected reward. (Hint: Optimal policy has an &quot;index form&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am fairly new to Reinforcement Learning and Markov Decision Problems (MDP). I am aware that the goal of the problem is to maximize the expected reward but I am not sure how exactly to formulate this into an MDP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the approach I thought of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Assume only 2 questions. Then the state space is &lt;span class=&quot;math-container&quot;&gt;$S\in \{1,2\}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Compute the expected total reward &lt;span class=&quot;math-container&quot;&gt;$J = E(R)$&lt;/span&gt; for both cases, when we start with question &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and question &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; and then find the maximum of the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) If we start with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$$J(S_0 = 1) = p_1(1-p_{2})R_1 + (R_1 + R_2)p_1p_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Similarly, if we start with &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$$J(S_0 = 2) = p_2(1-p_{1})R_2 + (R_1 + R_2)p_1p_2$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine the maximum reward of the two, the required condition for &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to be the optimal starting question is &lt;span class=&quot;math-container&quot;&gt;$$R_1p_1 - R_2p_2 + p_1p_2(R_2 - R_1) \gt 0$$&lt;/span&gt; If the above expression is negative, then we should start with &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if the approach is correct and how to proceed further. I am also not sure how to define the action space in this case. Can a dynamic programming approach be used here to find the optimal policy?&lt;/p&gt;&#xA;" OwnerUserId="33249" LastEditorUserId="33249" LastEditDate="2020-02-03T15:53:41.370" LastActivityDate="2020-02-03T15:53:41.370" Title="Formulation of a Markov Decision Process Problem" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;probability&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17817" PostTypeId="1" AcceptedAnswerId="17886" CreationDate="2020-02-03T14:49:03.827" Score="1" ViewCount="50" Body="&lt;p&gt;I have created an LSTM Neural Network which take as input the following format in an .csv file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sinewave&#xA;0.841470985&#xA;0.873736397&#xA;0.90255357&#xA;0.927808777&#xA;0.949402346&#xA;0.967249058&#xA;0.98127848&#xA;0.991435244&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I write some code so it can take as input hex addresses and convert them to int ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg the following .xlsx file containing 400.000 samples&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0xbfb22b18&#xA;0xbfb22b14&#xA;0xbfb22b10&#xA;0xbfb22b0c&#xA;0xbfb22b18&#xA;0xbfb22b14&#xA;0xbfb22b10&#xA;0xbfb22b0c&#xA;0xbfb22b18&#xA;0xbfb22b14&#xA;0xbfb22b10&#xA;0xbfb22b0c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32076" LastEditorUserId="32076" LastEditDate="2020-02-06T15:47:29.820" LastActivityDate="2020-02-06T15:49:00.593" Title="Convert input dataset given in hex addresses to int" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;prediction&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17820" PostTypeId="1" CreationDate="2020-02-03T16:55:40.057" Score="3" ViewCount="265" Body="&lt;p&gt;Despite the problem being very simple, I was wondering why an LSTM network was not able to converge to a decent solution.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import keras&#xA;&#xA;X_train = np.random.rand(1000)&#xA;y_train = X_train&#xA;X_train = X_train.reshape((len(X_train), 1, 1))&#xA;&#xA;model= keras.models.Sequential()&#xA;model.add(keras.layers.wrappers.Bidirectional(keras.layers.LSTM(1, dropout=0., recurrent_dropout=0.)))&#xA;model.add(keras.layers.Dense(1))&#xA;&#xA;optimzer = keras.optimizers.SGD(lr=1e-1)&#xA;&#xA;model.build(input_shape=(None, 1, 1))&#xA;model.compile(loss=keras.losses.mean_squared_error, optimizer=optimzer, metrics=['mae'])&#xA;history = model.fit(X_train, y_train, batch_size=16, epochs=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After 10 epochs, the algorithm seems to have reached its optimal solution (around &lt;code&gt;1e-4&lt;/code&gt; RMSE), and is not able to improve further the results.&lt;/p&gt;&#xA;&lt;p&gt;A simple Flatten + Dense network with similar parameters is however able to achieve 1e-13 RMSE.&lt;/p&gt;&#xA;&lt;p&gt;I'm surprised the LSTM cell does not simply let the value through, is there something I'm missing with my parameters? Is LSTM only good for classification problems?&lt;/p&gt;&#xA;" OwnerUserId="33256" LastEditorUserId="2444" LastEditDate="2020-07-05T14:56:34.077" LastActivityDate="2021-04-01T18:02:50.280" Title="Why does the error of my LSTM not decrease after 10 epochs?" Tags="&lt;long-short-term-memory&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17821" PostTypeId="1" CreationDate="2020-02-03T18:52:52.947" Score="1" ViewCount="44" Body="&lt;p&gt;I'm fairly new to ML/AI, i'm trying learn the content based recommendation - here is my source code - &lt;a href=&quot;https://github.com/jaganlal/content-based-recommender&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jaganlal/content-based-recommender&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using &lt;a href=&quot;https://grouplens.org/datasets/movielens/20m/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MovieLens 20M dataset&lt;/a&gt; - tags.csv to recommend similar movies based on its tag. But whenever i run (from sklearn.metrics.pairwise)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cosine_similarities = linear_kernel(tfidf_matrix, tfidf_matrix)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The python cell keeps executing and doesn't return at all. The code can be found at &lt;a href=&quot;https://github.com/jaganlal/content-based-recommender&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jaganlal/content-based-recommender&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am i doing anything wrong, any help is highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jagan&lt;/p&gt;&#xA;" OwnerUserId="33258" LastActivityDate="2021-03-18T04:09:15.933" Title="Code freezes and never returns when linear_kernel (sklearn.metrics.pairwise) is used on 20M Movielens dataset" Tags="&lt;recommender-system&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17822" PostTypeId="1" AcceptedAnswerId="17825" CreationDate="2020-02-03T19:57:38.267" Score="2" ViewCount="82" Body="&lt;p&gt;Consider a feedforward neural network. Suppose you have a layer of inputs, which is feedforward to a hidden layer, and feedforward both the input and hidden layers to an output layer.  Is there a name for this architecture? A layer feeds forward around the layer after it?&lt;/p&gt;&#xA;" OwnerUserId="33259" LastEditorUserId="2444" LastEditDate="2020-02-03T22:21:37.613" LastActivityDate="2020-02-04T20:49:57.457" Title="What is the name of this neural network architecture with layers that are also connected to non-neighbouring layers?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;feedforward-neural-network&gt;&lt;residual-networks&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17823" PostTypeId="1" AcceptedAnswerId="17837" CreationDate="2020-02-03T20:08:07.170" Score="1" ViewCount="130" Body="&lt;p&gt;I recently had a look at automated planners and experimented a little bit with FastDownward. As I wanted to start a toy project, I created a PDDL model for the ordinary 3D Rubik's Cube (of course using a planner may not be the most efficient approach).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although my model may not necessary be &quot;totally correct&quot; yet, so far it consists of 24 predicates, 12 actions for the respective moves (each with 8 typed parameters, 4 &quot;edge subcubes&quot; and 4 &quot;corner subcubes&quot;). For each of the movable subcubes I have a domain object whose position is basically determined by the respective predicate; overall, at first glance this seemed to me as a model of quite moderate size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was indeed not a very complex task to come up with this model and although it currently does not consider orientations of subcubes yet, I simply wanted to give it a try with an instance where only a single move (so application of one action) has to be conducted &amp;mdash; I assumed that such a plangraph should level off quite soon, so basically after the first layer where a goal state can be reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as I started the planner, it soon run out of memory and started to page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously, I only read something on PDDL and the respective sections of &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_Intelligence%3A_A_Modern_Approach&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell &amp;amp; Norvig&lt;/a&gt;. I took a closer look at the planner itself and found that it transform the PDDL description into some intermediate description. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to only execute the transformation and after cutting off a third of my Rubik's Cube it &amp;mdash; at least &amp;mdash; terminated. I investigated the transformed model file and found that the planner/solver actually instantiates (flattens) the actions with its parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the Rubik's Cube has a quite unrestricted domain and each action has apparently 8 parameters (4 corners, 4 edges), this inherently results in a huge number of flattened actions. Even more, although I added precondition constraints to ensure distinctness of the parameters (so the very same subcube cannot be both edgecube &lt;span class=&quot;math-container&quot;&gt;$e_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_2$&lt;/span&gt;), the flattened version still contains these invalid actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;are state-of-the-art planners even suitable for such a problem or are they designed for problems where flattening the actions is of great advantage (e.g. few parameters, moderate number of objects per class, etc.). IMHO this would be a &lt;strong&gt;major&lt;/strong&gt; limitation for their applicability in contrast to e.g. CP solvers. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;can anyone recommend another planner that is more appropriate and that maybe does not perform the model transformation, which seems to be quite expensive for my PDDL spec (e.g. from this list, &lt;a href=&quot;https://ipc2018-classical.bitbucket.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ipc2018-classical.bitbucket.io&lt;/a&gt;, i have chosen FastDownward since it seemed to be among the best...)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;does PDDL or even FastDownward allow to specify that the parameters have to be distinct (i just found this: &lt;a href=&quot;https://www.ida.liu.se/~TDDC17/info/labs/planning/writing.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ida.liu.se/~TDDC17/info/labs/planning/writing.shtml&lt;/a&gt; &amp;mdash; search for distinct &amp;mdash; but this is more than vague), cause this may already lead to a significant reduction of flattened actions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'd also be happy for any other recommendations or remarks&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33261" LastEditorUserId="2193" LastEditDate="2020-02-04T12:40:47.473" LastActivityDate="2020-02-04T14:05:42.170" Title="FastDownward PDDL Planner Limitations" Tags="&lt;search&gt;&lt;planning&gt;&lt;strips&gt;&lt;pddl&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-02-07T17:27:50.413" ContentLicense="CC BY-SA 4.0" />
  <row Id="17824" PostTypeId="1" AcceptedAnswerId="17827" CreationDate="2020-02-03T21:59:12.360" Score="1" ViewCount="65" Body="&lt;p&gt;Is the following statement about neural networks overclaimed?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Neural networks are iterative methods that minimize a loss function defined on the output layer of neurons.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I wrote this statement in the &lt;code&gt;Introduction&lt;/code&gt; section of a conference paper. Reviewer got back to me saying that, &quot;this statement is over claimed without proper citations&quot;. How is this statement over claimed? Is it not obvious that neural networks are iterative methods that try to minimize a cost function?&lt;/p&gt;&#xA;" OwnerUserId="25868" LastEditorUserId="25868" LastEditDate="2020-02-03T22:59:11.883" LastActivityDate="2020-02-03T23:51:04.083" Title="Is the following statement about neural networks overclaimed?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17825" PostTypeId="2" ParentId="17822" CreationDate="2020-02-03T22:06:55.493" Score="2" Body="&lt;p&gt;This could be called a &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;residual neural network (ResNet)&lt;/a&gt;, which is a neural network with skip connections, that is, connections that skip layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a screenshot of a figure from the paper &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Residual Learning for Image Recognition&lt;/a&gt; (2015), an important paper that shows the usefulness of these architectures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C5zjY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C5zjY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-02-03T22:06:55.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17827" PostTypeId="2" ParentId="17824" CreationDate="2020-02-03T23:44:11.227" Score="2" Body="&lt;p&gt;I think I would never say that neural networks are iterative methods. I would say that iterative methods (e.g. gradient descent) are used to train neural networks (which can be thought of as linear and non-linear models, but mainly non-linear), which is quite different. Maybe you should or wanted to say that deep learning is an area of study where iterative methods are used to train neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible the reviewer is just telling you that this or similar statements have been used a lot and don't really provide any insight, they are misleading, or simply useless or trivial in the context of the paper or conference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without further clarifications by the reviewer and about your paper and conference, it is difficult to provide more explanations.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-03T23:51:04.083" LastActivityDate="2020-02-03T23:51:04.083" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17828" PostTypeId="1" CreationDate="2020-02-04T04:07:27.960" Score="1" ViewCount="33" Body="&lt;p&gt;Suppose we are doing sentiment analysis for a restaurant. Customers can rate the restaurant by #1: how expensive the restaurant is, #2:how good is the food and #3: how likely they will come again. The ratings are dependent,i.e. the more expensive the restaurant is (higher #1), the less likely they will come back (lower #3), but whey will if the food is good (higher #2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: is there a good RNN structure(review as input, #1-#3 as output) that can capture and model the dependency among #1 - #3? &lt;/p&gt;&#xA;" OwnerUserId="33266" LastEditorUserId="33266" LastEditDate="2020-02-12T20:05:51.733" LastActivityDate="2021-03-08T23:02:40.230" Title="What is the appropriate RNN structure to do Sentiment Analysis with multiple dependent ratings?" Tags="&lt;recurrent-neural-networks&gt;&lt;sentiment-analysis&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17829" PostTypeId="1" CreationDate="2020-02-04T08:00:06.863" Score="1" ViewCount="94" Body="&lt;p&gt;As in &lt;a href=&quot;https://en.wikipedia.org/wiki/Calculus_of_variations&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Calculus_of_variations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The calculus of variations is a field of mathematical analysis that&#xA;  uses variations, which are small changes in functions and functionals,&#xA;  to find maxima and minima of functionals&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Gradient Descent algorithm is also a method to find minima of a function, it it a part of the Calculus of Variations?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-04T08:00:06.863" Title="Is Gradient Descent algorithm a part of Calculus of Variations?" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;variational-autoencoder&gt;&lt;calculus&gt;&lt;math&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17830" PostTypeId="1" AcceptedAnswerId="17831" CreationDate="2020-02-04T08:19:04.763" Score="1" ViewCount="203" Body="&lt;p&gt;Standard deviation and variance are in statistics but the formula for variance is somehow related to the L1 and L2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically (L2 in machine learning sense),&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Variance = \dfrac{(X_1-Mean)^2+..+(X_n-Mean)^2}{N}$$&lt;/span&gt;&#xA;and,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Standard\ Deviation= \sqrt(Variance)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why shouldn't it be (L1 in machine learning sense):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Variance = \dfrac{|X_1-Mean|+..+|X_n-Mean|}{N}$$&lt;/span&gt;&#xA;and,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Standard\ Deviation= Variance$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-04T17:33:33.193" Title="Why is Standard Deviation based on L2 Variance and not L1 Variance" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17831" PostTypeId="2" ParentId="17830" CreationDate="2020-02-04T08:24:34.990" Score="3" Body="&lt;p&gt;I've found the answer, the L2 is Standard Deviation, and the L1 is Mean Deviation. Standard deviation describes the variation better and the values are always different on different sets of X while the Mean Deviation gives the same values sometimes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;*Footnote: Why square the differences? If we just add up the differences from the mean ... the negatives cancel the positives:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;standard deviation why a       (4 + 4 − 4 − 4) / 4  = 0 So that won't work.&#xA;  How about we use absolute values?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;standard deviation why a       (|4| + |4| + |−4| + |−4|) / 4  =  (4 + 4 + 4 +&#xA;  4) / 4 = 4 That looks good (and is the Mean Deviation), but what about&#xA;  this case:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;standard deviation why b       (|7| + |1| + |−6| + |−2|) / 4  =  (7 + 1 + 6 +&#xA;  2) / 4 = 4 Oh No! It also gives a value of 4, Even though the differences&#xA;  are more spread out.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;So let us try squaring each difference (and taking the square root at&#xA;  the end):&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;standard deviation why a      √( (42 + 42 + 42 + 42) / 4 ) = √( 644) = 4&#xA;  standard deviation why b      √( (72 + 12 + 62 + 22) / 4 ) = √( 904) = 4.74...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Reference:&lt;br/&gt;&#xA;&lt;a href=&quot;https://www.mathsisfun.com/data/standard-deviation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathsisfun.com/data/standard-deviation.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2020-02-04T17:33:33.193" LastActivityDate="2020-02-04T17:33:33.193" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17832" PostTypeId="1" AcceptedAnswerId="17834" CreationDate="2020-02-04T08:29:19.433" Score="2" ViewCount="39" Body="&lt;p&gt;For my internship assignment I have to implement a proof of concept for an application that is supposed to scan a picture with a carp on it and identify which carp this is. All of the carps that are going to be scanned are known and they all exist in the database, so no new carps are scanned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this possible? I've been searching a lot about this topic and the only thing I found is customvision.ai, but for using this I need to have at least 15 pictures of the same carp per tag, but the client only has 1 picture per carp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are your recommendations or do you think this is not possible?&lt;/p&gt;&#xA;" OwnerUserId="33268" LastEditorUserId="2193" LastEditDate="2020-02-04T12:30:33.520" LastActivityDate="2020-02-04T12:30:33.520" Title="Recognize carp and give them a unique id" Tags="&lt;machine-learning&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17833" PostTypeId="1" AcceptedAnswerId="17835" CreationDate="2020-02-04T09:17:01.797" Score="2" ViewCount="45" Body="&lt;p&gt;Here's a diagram of a variational auto-encoder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NVEHA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NVEHA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 nodes before the sample (encoding vector). One is the &lt;em&gt;mean&lt;/em&gt;, one is the &lt;em&gt;standard deviation&lt;/em&gt;. The &lt;em&gt;mean&lt;/em&gt; one is confusing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the mean of values or is it the mean deviation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{mean} = \dfrac{X_i+..+X_n}{N}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{mean deviation} = \dfrac{[X_i|+..+|X_n|}{N}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-04-08T22:18:07.567" LastActivityDate="2020-04-08T22:18:07.567" Title="What is the mean in the variational auto-encoder?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;math&gt;&lt;autoencoders&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17834" PostTypeId="2" ParentId="17832" CreationDate="2020-02-04T09:43:41.240" Score="1" Body="&lt;p&gt;I'm not sure I understood your question entirely so please correct me if i'm wrong. You're having all the carp tagged, so if I give you a picture of any of them you know exactly which one is in the picture right? If that is the case then you're dealing with a classic classification problem. One simple way of solving such problem will be to use CNN on the input image to extract features, and at the end of the network have N neurons, where each neuron matches one carp. Just apply a softmax over all the inputs to have a probability distribution, and select the highest value (or if the values is highest then a threshold). Regarding the number of labeled examples per class, having only one might be a problem. I would suggest you to look into few shot learning, which are trying to solve this exact problem (training decent models with limited training data).  &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2020-02-04T09:43:41.240" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17835" PostTypeId="2" ParentId="17833" CreationDate="2020-02-04T09:56:06.393" Score="1" Body="&lt;p&gt;Kind of neither, although leaning towards the first definition of the Mean as a simple average of values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a &lt;strong&gt;distribution parameter&lt;/strong&gt; of the Gaussian, so it's the &lt;em&gt;expected&lt;/em&gt; average of samples as the number of samples approaches infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distrinction is that you could draw three samples, -2, 0 and -1, from a Standard Normal - the mean of samples would be -1, but the &lt;em&gt;distribution&lt;/em&gt; mean is still 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a network architecture PoV, it's a learnable transformation applied on top of samples from a Standard Normal, so you only need to learn the transformation and get sampling for 'free'.&lt;/p&gt;&#xA;" OwnerUserId="33247" LastActivityDate="2020-02-04T09:56:06.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17836" PostTypeId="2" ParentId="13848" CreationDate="2020-02-04T13:18:42.833" Score="3" Body="&lt;p&gt;You may be very interested to know that there was a bug in the v2 Lidar tracing, making the agent think there were phantom objects, and sometimes intersecting with its own legs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/openai/gym/pull/1789&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/gym/pull/1789&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finding this bug makes me even more impressed anyone has solved BipedalWalkerHardcore-v2 - it seems the observations from lidar have been inconsistent and incorrect, returning the furthest hit result instead of closest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Before fix - lidar traces through ground, and hits the side of a pit,&#xA;  giving the agent the impression of a &quot;phantom canyon&quot; in front of the&#xA;  pit, that only appears as it approaches the pit:&#xA;  &lt;a href=&quot;https://i.imgur.com/XKPnRTR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.imgur.com/XKPnRTR.png&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;After fix - lidar is stopped by terrain, even when another object is behind it: &lt;a href=&quot;https://i.imgur.com/Gg8B5BD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.imgur.com/Gg8B5BD.png&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;After triple checking the docs - I've submitted a minor tweak (returning -1 instead of 1 for an object that should be ignored) - it now seems legs are correctly ignored, and the traces are accurate in all situations!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It seems to me that solutions to BipedalWalkerHardcore-v2 have not just learned to deal with the complex environment - but advanced a step ahead, and are able to deal with the complex environment and sensory hallucinations causing them to jump at the slightest hint of a cube, and keep running even when it looks like the ground is not visible below their feet, relying more on the touch sensor than the lidar, or perhaps recognising the difference in &quot;shape&quot; between a real pit and a &quot;fake pit&quot; (A real pit has a floor)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;BipedalWalkerHardcore-v2 has been bumped to BipedalWalkerHardcore-v3 with these fixes as of Jan 31, 2020.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to try retraining your agent now! (although it is still a difficult environment to solve)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To expand on why DDPG doesn't solve it, when although buggy, BipedalWalkerHardcore-v2 &lt;em&gt;is&lt;/em&gt; solvable: The solution landscape to this problem is as full of pits as the environment itself. To learn to leap over a pit in the environment for example, the agent must perform a complex sequence of actions that is difficult to discover by random chance. Each time it fails, it learns that being close to a pit is highly likely to result in a large penalty, and in an effort to maximise it's rewards, will often remain stationary with a naive method like DDPG as the rewards for doing that are higher than trying and falling into the pit once more. In short, vanilla DDPG lacks enough exploration power to find the complex series of actions required before it converges on not going near the pit. Not to mention all of the other things it needs to learn to be successful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of the very few published examples that have solved it, one used Evolutionary Strategy - a gradient free method essentially trying millions of policies and evaluating them, and a custom A3C method that was tailored to solve this particular environment. Both had high computational and sample requirements.&#xA;I speak from experience as I have personally solved this environment with a RL exploration algorithm that generalises to other environments, solves it in 4 hours on a single cpu, and can be used with any off-policy RL algorithm, but unfortunately I'm unable to publish it because of IP with the company I work for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR; The chance of vanilla DDPG solving it are infeasibly small.&lt;/p&gt;&#xA;" OwnerUserId="12765" LastEditorUserId="12765" LastEditDate="2020-02-07T01:12:11.513" LastActivityDate="2020-02-07T01:12:11.513" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17837" PostTypeId="2" ParentId="17823" CreationDate="2020-02-04T14:05:42.170" Score="1" Body="&lt;p&gt;You have stumbled upon a common drawback of the vast majority of modern planning technology. The &quot;flattening&quot; you refer to is actually called &quot;grounding&quot; in the community. Indeed, grounding is the first step of almost every planner out there. Planners that don't do this grounding phase are typically referred to as &quot;lifted planners&quot;, but their availability is far more limited (i.e., you'd likely need to reach out to an author of a paper to get access to their code).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, many of the planners, Fast Downward included, will try to be smart about the grounding phase. This includes simple things such as making sure that typing for the parameters (and the objects that instantiate them) is adhered to, but goes further with certain checks on reachability. The process isn't flawless, but it usually does a good job when the true number of ground actions is relatively small. If your model really does have an astronomical number of ground actions, then you're kind of out of luck here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your questions specifically:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, they are designed for problems where you can ground things. But the mark of suitability is the number of reachable ground actions. If this is small enough, then either (1) the planners should be able to handle it or (2) it presents a research opportunity for grounding things better.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This is the most recent work that comes to mind, as well as the papers that cite it:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.aaai.org/ocs/index.php/ICAPS/ICAPS14/paper/view/7948/8033&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aaai.org/ocs/index.php/ICAPS/ICAPS14/paper/view/7948/8033&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scholar.google.com/scholar?cites=2377320520977506929&amp;amp;as_sdt=2005&amp;amp;sciodt=0,5&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scholar.google.com/scholar?cites=2377320520977506929&amp;amp;as_sdt=2005&amp;amp;sciodt=0,5&amp;amp;hl=en&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They aren't assumed to be distinct. There are situations when you wouldn't want them to be, and so it is up to the designer to add the preconditions that ensure they are distinct (note that this is one excellent way to equip the grounder with extra information about what it should generate).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generally, having actions with many parameters is a recipe for problems that have too many ground actions. Either manually remodeling things yourself, or applying a technique like &lt;a href=&quot;https://www.aaai.org/ocs/index.php/ICAPS/ICAPS14/paper/view/7894/8006&quot; rel=&quot;nofollow noreferrer&quot;&gt;operator splitting&lt;/a&gt; is the approach most often taken.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finally, if you'd like to have a more directed discussion, feel free to join us over on the &lt;a href=&quot;http://slack.planning.domains/&quot; rel=&quot;nofollow noreferrer&quot;&gt;slack&lt;/a&gt; channel. There may be more eyes on the questions you pose there.&lt;/p&gt;&#xA;" OwnerUserId="33275" LastActivityDate="2020-02-04T14:05:42.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17838" PostTypeId="1" CreationDate="2020-02-04T15:27:18.190" Score="0" ViewCount="30" Body="&lt;p&gt;I'm interested in studying how different bacteria interact/affect each-other in a water treatment pool. Since there total number of bacteria is limited I've found Lotka-Volterra equations which describe exactly this type of interactions (in order for some bacteria to increase others need to decrease). My question is: &lt;em&gt;are there any other methods which describe such process or could be extended to do so?&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2020-02-04T15:42:36.833" LastActivityDate="2020-02-04T15:42:36.833" Title="Predator-prey models" Tags="&lt;concepts&gt;&lt;data-mining&gt;&lt;causation&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17839" PostTypeId="1" AcceptedAnswerId="17842" CreationDate="2020-02-04T16:39:45.563" Score="4" ViewCount="156" Body="&lt;p&gt;I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am interested in the question: Can we mathematically establish that a certain AI system has the ability to learn a task given some examples of how it should be done? &#xA;I would like to know what research has been done on this topic and also what mathematical tools could be helpful in answering such questions.&lt;/p&gt;&#xA;" OwnerUserId="33278" LastEditorUserId="2444" LastEditDate="2020-02-04T19:48:21.257" LastActivityDate="2021-01-22T14:26:17.740" Title="Mathematical foundations of the ability to learn" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;math&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17840" PostTypeId="1" CreationDate="2020-02-04T16:40:39.650" Score="1" ViewCount="22" Body="&lt;p&gt;I designed a fire detection using Deep Learning based classification approach. In my training dataset, I have both fire and fire smokes are supposed to be detected (all under &quot;fire&quot;; mostly real fires are detected. Fire smokes are less accurate). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now after months, I need to differentiate them in my detection results. It would be difficult to retrain each class separately now. Another option coming into my mind is building a binary classification after the main one, getting the main detections as input and saying which of the two it belongs to. However, I may miss some fire smokes I believe because that's less accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other approaches? What are pros/cons of various approaches?&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-02-04T16:52:21.050" Title="Post-classification after inference" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;deep-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17842" PostTypeId="2" ParentId="17839" CreationDate="2020-02-04T19:41:43.880" Score="3" Body="&lt;p&gt;&lt;strong&gt;Computational learning theory&lt;/strong&gt; (CLT or COLT) is devoted to the mathematical and computational analysis of machine learning algorithms, so it is concerned with the learnability (i.e. generalization, bounds, efficiency, etc.) of certain tasks, given a learner (or a learning algorithm), a hypothesis space, data, etc.&lt;/p&gt;&#xA;&lt;p&gt;CLT can be divided into (at least) two subfields: &lt;a href=&quot;https://ieeexplore.ieee.org/document/788640&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical learning theory&lt;/a&gt; (SLT) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_learning_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithmic learning theory&lt;/a&gt; (ALT). The most famous CLT frameworks are &lt;a href=&quot;https://people.mpi-inf.mpg.de/%7Emehlhorn/SeminarEvolvability/ValiantLearnable.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAC learning&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC theory&lt;/a&gt; (which extends PAC learning to infinite-dimensional hypothesis spaces).&lt;/p&gt;&#xA;&lt;p&gt;The following are some good resources you can use to get started with CLT (but you can find more &lt;a href=&quot;https://ai.stackexchange.com/a/20358/2444&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The book &lt;a href=&quot;http://profsite.um.ac.ir/%7Emonsefi/machine-learning/pdf/Machine-Learning-Tom-Mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt; (1997) by Mitchell&lt;/li&gt;&#xA;&lt;li&gt;The book &lt;a href=&quot;https://pdfs.semanticscholar.org/e923/9469aba4bccf3e36d1c27894721e8dbefc44.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations of Machine Learning&lt;/a&gt; (2012) by Mohri et al.&lt;/li&gt;&#xA;&lt;li&gt;The book &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Machine Learning: From Theory to Algorithms&lt;/a&gt; (2014) by Shalev-Shwartz et al.&lt;/li&gt;&#xA;&lt;li&gt;The paper &lt;a href=&quot;https://ieeexplore.ieee.org/document/788640&quot; rel=&quot;nofollow noreferrer&quot;&gt;An overview of statistical learning theory&lt;/a&gt; (1999) by Vapnik (who is one of the main contributors to SLT)&lt;/li&gt;&#xA;&lt;li&gt;The paper &lt;a href=&quot;http://www.econ.upf.edu/%7Elugosi/mlss_slt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Statistical Learning Theory&lt;/a&gt; (2014) by Bousquet et al.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here's a related question on this site: &lt;a href=&quot;https://ai.stackexchange.com/q/12971/2444&quot;&gt;What sort of mathematical problems are there in AI that people are working on?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-22T14:26:17.740" LastActivityDate="2021-01-22T14:26:17.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17843" PostTypeId="1" CreationDate="2020-02-04T20:05:09.830" Score="2" ViewCount="29" Body="&lt;p&gt;My understanding is that there is no singular &quot;The Turing Test Ruleset&quot; and competitions don't all do it the exact same way. Still, I'm wondering about some commonly accepted rules and their nuances. My Googling is not producing any specifics about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think most people agree that the purpose of the humans who talk to the judges is to just act normal. In the so-called &quot;passed Turing test&quot; instances where the humans tried to fool judges into thinking they were AIs, I would say the tests should be thrown out and I've seen critics in the field agree with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this question is more about how the judge should act.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we are doing a competition where the threshold is for 40% of judges to call an AI a human after 5 minutes of chat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the chat, is the judge supposed to try to trick the potential AI into revealing itself, or is the judge supposed to attempt to act as unbiased and natural as possible, as if it is accepted the potential AI is human, and judge based on a completely natural conversation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, asking &quot;What is the value of PI out 10 digits?&quot; or &quot;What is 123456 times 654321?&quot; or &quot;If you saw a bunny and a dime stuck on the road with a truck about to hit them, which one would you pick up?&quot; would be trying to trick AIs into revealing themselves because you are relying on exploiting the fact that the AI might tell you the correct answer or the inhuman answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is as opposed to simply carrying on a natural and normal conversation, with no biases or expectations. If you came upon someone on the street you would not spend 5 minutes trying to hurriedly ask ridiculous interrogation questions in an effort to prove the other person was an AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is the point of a Turing test generally assumed to be an attempt to flush out AIs or an attempt to judge their natural human conversation without interrogation-like prejudice?&lt;/p&gt;&#xA;" OwnerUserId="33282" LastActivityDate="2020-02-04T21:58:00.570" Title="Nuances of Turing test requirements" Tags="&lt;turing-test&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17844" PostTypeId="1" CreationDate="2020-02-04T20:13:27.760" Score="1" ViewCount="34" Body="&lt;p&gt;For a personal project, I'm trying to download files from a specific set of websites using a web scraper. The scraper has to navigate multiple webpages to get to the files I want to download. I'd like to use AI to find each successive link in order to avoid the inflexibility of hardcoded DOM paths. In other words, I want an AI system that replicates the way that a human would click through links to get to a download page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems like a task for a supervised neural network, where the input would be the HTML page and the output would be the href link to the next webpage in the search. But that's about the extent of my AI knowledge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which subtype of neural network would be most effective for this kind of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, I have looked at &lt;a href=&quot;https://ai.stackexchange.com/questions/9574/ai-learning-by-demonstration-for-web-navigation&quot;&gt;this related question.&lt;/a&gt; My problem would probably fall under bullet two.&lt;/p&gt;&#xA;" OwnerUserId="33283" LastActivityDate="2020-02-04T20:13:27.760" Title="Specific Neural Network Subtype for Automatic Web Scraping (Hyperlink Identification)" Tags="&lt;neural-networks&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17845" PostTypeId="2" ParentId="17822" CreationDate="2020-02-04T20:49:57.457" Score="2" Body="&lt;p&gt;Such a network could be either a &lt;strong&gt;Residual Network&lt;/strong&gt; or a &lt;strong&gt;Highway Network&lt;/strong&gt; depending upon the underlying architecture of the skip layers. They are primarily used to to tackle the problem of vanishing gradients in very deep networks by reusing activations from a previous layer and passing them to adjacent layers (two or three skips away).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Highway Network&lt;/strong&gt;: (&lt;a href=&quot;http://papers.nips.cc/paper/5850-training-very-deep-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training Very Deep Networks, 2015&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; acts as a &lt;em&gt;Gating Function&lt;/em&gt;, which controls how much information is to be transmitted from a previous layer in the network. An additional weight matrix is used to estimate the skip weights.&lt;a href=&quot;https://i.stack.imgur.com/zIlct.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zIlct.png&quot; alt=&quot;Highway&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Residual Network&lt;/strong&gt;:&lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Deep Residual Learning for Image Recognition ,2015)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This architecture does not involve a Gate Controller. The shortcut connections are realized by adding the outputs of a previous layer to that of the connecting layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FfU4q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FfU4q.png&quot; alt=&quot;Residual&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33010" LastActivityDate="2020-02-04T20:49:57.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17846" PostTypeId="1" CreationDate="2020-02-04T21:26:03.650" Score="1" ViewCount="202" Body="&lt;p&gt;I have a hash, e.g. &lt;code&gt;552c51576e2a6dec462b0d56af8c31713e26a763c0e2a3d48ebbf4b99958eeba&lt;/code&gt;. This hash hashed again, is &lt;code&gt;39f20b3e5cc0181d6fbe2a4d52c7b3c315e8526e35da3cd0b18e25d10b670dc3&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I only know the last hash &lt;code&gt;39f20b3e5cc0181d6fbe2a4d52c7b3c315e8526e35da3cd0b18e25d10b670dc3&lt;/code&gt;. It matches value &lt;code&gt;1.58&lt;/code&gt;. I do not know the first hash, as this is a security feature (that all hashs are generated and we go from top to bottom to not re-calculate the hashes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A new round starts. last round had hash &lt;code&gt;39f20b3e5cc0181d6fbe2a4d52c7b3c315e8526e35da3cd0b18e25d10b670dc3&lt;/code&gt; and resulted with &lt;code&gt;1.58&lt;/code&gt;.&#xA;The next round does not reveal the hash at the start, but I want to guess the resulting value already, which should be &lt;code&gt;4.06&lt;/code&gt;. When the values gets revealed, I also get the hash.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you understand what I want to achieve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a hash of a hash, which results in a value. Now, my AI system should predict the next value which is based on the hash which got hashed and then resulted in e.g. &lt;code&gt;1.58&lt;/code&gt;. I know a 'next hash' of the current input hash (which I don't know) and the 'previous' value of the 'next' hash. Then, I want to guess the current value (without knowing its hash already).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea how I could implement this in python or what method or keyword I should do research on? I am absolute new to AI and do not know what I need to look up for that project.&lt;/p&gt;&#xA;" OwnerUserId="33284" LastActivityDate="2021-04-15T11:55:15.150" Title="Predict a value calculated from a hash (and hash before that hash)" Tags="&lt;python&gt;&lt;ai-basics&gt;&lt;prediction&gt;&lt;score-prediction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17848" PostTypeId="2" ParentId="17843" CreationDate="2020-02-04T21:42:03.000" Score="1" Body="&lt;p&gt;I do not know about all competitions, but the &lt;a href=&quot;https://en.wikipedia.org/wiki/Loebner_Prize&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loebner prize&lt;/a&gt; is one of the more well known competitions based around the concept of the Turing Test, specifically for chatbots, and judges must ask the same questions of all bots. The questions are only revealed during the competition, it is not possible for bot programmers to prepare for them other than making a generally good chatbot. They judges can then rank bots on how they handle the same set of questions. E.g. from chatbots.org/ai_zone/viewthread/867 here are the 2012 &amp;quot;qualifying&amp;quot; questions that were released following the test:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My name is Bill. What is your name?&lt;/p&gt;&#xA;&lt;p&gt;How many letters are there in the name Bill?&lt;/p&gt;&#xA;&lt;p&gt;How many letters are there in my name?&lt;/p&gt;&#xA;&lt;p&gt;Which is larger, an apple or a watermelon?&lt;/p&gt;&#xA;&lt;p&gt;How much is 3 + 2?&lt;/p&gt;&#xA;&lt;p&gt;How much is three plus two?&lt;/p&gt;&#xA;&lt;p&gt;What is my name?&lt;/p&gt;&#xA;&lt;p&gt;If John is taller than Mary, who is the shorter?&lt;/p&gt;&#xA;&lt;p&gt;If it were 3:15 AM now, what time would it be in 60 minutes?&lt;/p&gt;&#xA;&lt;p&gt;My friend John likes to fish for trout.  What does John like to fish for?&lt;/p&gt;&#xA;&lt;p&gt;What number comes after seventeen?&lt;/p&gt;&#xA;&lt;p&gt;What is the name of my friend who fishes for trout?&lt;/p&gt;&#xA;&lt;p&gt;What whould I use to put a nail into a wall?&lt;/p&gt;&#xA;&lt;p&gt;What is the 3rd letter in the alphabet?&lt;/p&gt;&#xA;&lt;p&gt;What time is it now?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;According to the WikiPedia description, the style of the questions has changed over time. The above set in my opinion seem to be designed to test very simple grammar, common sense and memory targets.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-04T21:58:00.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17849" PostTypeId="1" AcceptedAnswerId="17855" CreationDate="2020-02-04T21:58:02.993" Score="4" ViewCount="120" Body="&lt;p&gt;I've been reading about Nesterov momentum from &lt;a href=&quot;https://ruder.io/optimizing-gradient-descent/index.html#gradientdescentvariants&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and it seems like a nice improvement over regular momentum with no extra cost whatsoever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is this really the case? Are there instances where regular momentum performs better than Nesterov momentum or is Nesterov momentum performs at least as good as the regular momentum all the time?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="2444" LastEditDate="2020-02-04T22:17:46.660" LastActivityDate="2020-02-05T06:48:35.293" Title="Is there a reason to choose regular momentum over Nesterov momentum for neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17850" PostTypeId="1" CreationDate="2020-02-04T22:20:03.473" Score="1" ViewCount="22" Body="&lt;p&gt;In acoustics decibel levels were defined to solve an issue with showing values that are interpretive, understandable, and easy to communicate in contrast to intensity or pressure in Pascals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$dB = 10*\log({\frac{p^2}{p_{ref}^2}})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This log scale helps human understanding of an acoustic signal because human hearing is capable of discerning the difference of about 1 dB ref 20 &lt;span class=&quot;math-container&quot;&gt;$uPa$&lt;/span&gt;. However, if I have input data for a 2D CNN would it make more sense to have my input data be in Decibels where it is a logarithmic or to have raw pressures? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would one or the other benefit my models learning?&lt;/p&gt;&#xA;" OwnerUserId="33189" LastEditorUserId="33189" LastEditDate="2020-02-04T23:06:50.390" LastActivityDate="2020-02-04T23:06:50.390" Title="Acoustic Input Data: Decibel or Pascals" Tags="&lt;convolutional-neural-networks&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17851" PostTypeId="1" CreationDate="2020-02-04T23:55:01.173" Score="0" ViewCount="206" Body="&lt;p&gt;Given an input image and an angle I want the output to be the image rotated at the given angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to train a neural network to do this from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What sort of archetecture do you think would work for this if I want it to be lossless?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking of this archetecture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;256x256 image &lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&gt; convolutions to 64x64 image with 4 channels&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&gt; convolutions to 32x32 image with 16 channels and so on&lt;/p&gt;&#xA;&#xA;&lt;p&gt;until a 1 pixel image with 256x256 channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then combine this with the input angle, and then a series of deconvolutions back up to 256x256.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think this would work? Could this be trained as a general rotation machine? Or is there a better archetecture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also like to train the same archetecture to do other transforms.&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-02-05T03:14:12.557" Title="Best architecture to learn image rotation?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17852" PostTypeId="2" ParentId="17803" CreationDate="2020-02-05T03:01:29.140" Score="4" Body="&lt;p&gt;There is no contradiction. &#xA;First, agnostic PAC learnable doesn't mean that the there is a good hypothesis in the hypothesis class; it just means that there is an algorithm that can probably approximately do as well as the best hypothesis in the hypothesis class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, these NFL theorems have specific mathematical statements, and hypothesis classes for which they apply are often not the same as the hypothesis class for which PAC-learnability holds. For example, in &lt;em&gt;Understanding Machine Learning&lt;/em&gt;  by Shalev-Shwartz and Ben-David, a hypothesis class is agnostic PAC learnable if and only if has finite VC dimension (Theorem 6.7). Here, the algorithm is ERM.&#xA;On the other hand, the application of the specific version of NFL that this book uses has Corollary 5.2, that the hypothesis class of all classifiers is not PAC learnable, and note that this hypothesis class has infinite VC dimension, so the Fundamental Theorem of PAC learning does not apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main takeaway is that in order to learn, we need some sort of inductive bias (prior information). This can be seen in the form of measuring the complexity of the hypothesis class or using other tools in learning theory. &lt;/p&gt;&#xA;" OwnerUserId="33286" LastActivityDate="2020-02-05T03:01:29.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17853" PostTypeId="2" ParentId="17851" CreationDate="2020-02-05T03:14:12.557" Score="2" Body="&lt;p&gt;This would likely suffer from the &lt;a href=&quot;https://ai.stackexchange.com/questions/8885/why-are-variational-autoencoders-output-is-blurred-while-gans-output-is-crisp-a&quot;&gt;blurry image problem&lt;/a&gt; that autoencoders are known to suffer from. See also &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/9t712f/dwhy_are_images_created_by_gan_sharper_than/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;On the other hand, using GAN's to sharpen your images doesn't seem particularly helpful since you seem to be lookng for a way to rotate general images, not ones of a specific domain. Moreover, there's almost certainly going to be some loss&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems unreasonable/overkill to use neural networks to do this transformation. I suggest treating the rotation as a linear transformation, mapping certain pixel locations to other pixel locations (in fact, you can make this transformation differentiable with respect to its inputs using some deep learning library's autodifferentiation tools, if that's what you're looking for).&#xA;Since rotations are just linear maps that send pixels to other pixels, this will be much more computationally efficient.&lt;/p&gt;&#xA;" OwnerUserId="33286" LastActivityDate="2020-02-05T03:14:12.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17854" PostTypeId="2" ParentId="17802" CreationDate="2020-02-05T03:28:41.793" Score="1" Body="&lt;p&gt;Yes, LSTM are ideal for this. For even stronger representational capacity, make your LSTM's multi-layered.&#xA;Using 1-dimensional convolutions in a CNN is a common way to exctract information from time series too, so there's no harm in trying. Typically, you'll test many models out and take the one that has best validation performance.&lt;/p&gt;&#xA;" OwnerUserId="33286" LastActivityDate="2020-02-05T03:28:41.793" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17855" PostTypeId="2" ParentId="17849" CreationDate="2020-02-05T06:48:35.293" Score="4" Body="&lt;p&gt;The book &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; by Goodfellow, Bengio, and Courville says (Sec 8.3.3, p 292 in my copy) states that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unfortunately, in the stochastic gradient case, Nesterov momentum does not improve the rate of convergence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure why this is, but the theoretical advantage depends on a convex problem, and from this, it sounds like the practical advantage does too - or at least, that it isn't applicable to typical neural network landscapes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps it can be implemented more efficiently, but it seems to me that you need to do parameter updates twice (in order to calculate the gradient in a place you aren't moving two) and thus Nesterov requires more computation and memory than plain ole momentum.&lt;/p&gt;&#xA;" OwnerUserId="29720" LastActivityDate="2020-02-05T06:48:35.293" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17856" PostTypeId="1" CreationDate="2020-02-05T09:38:46.360" Score="1" ViewCount="84" Body="&lt;p&gt;I originally posted on SO (&lt;a href=&quot;https://stackoverflow.com/questions/60063931/error-when-using-tensorflow-hmc-to-marginalise-gpr-hyperparameters&quot;&gt;original post&lt;/a&gt;) but was suggested to post here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to use tensorflow (version 2) to use gaussian process regression to fit some data and I found the google colab example online here &lt;a href=&quot;https://stackoverflow.com/questions/60063931/error-when-using-tensorflow-hmc-to-marginalise-gpr-hyperparameters&quot;&gt;1&lt;/a&gt;. I have turned some of this notebook into a minimal example that is below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the code fails with the following error when using MCMC to marginalize the hyperparameters: and I was wondering if anyone has seen this before or knows how to get around this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tensorflow.python.framework.errors_impl.InvalidArgumentError:  Input matrix is not invertible.&#xA;     [[{{node mcmc_sample_chain/trace_scan/while/body/_168/smart_for_loop/while/body/_842/dual_averaging_step_size_adaptation___init__/_one_step/transformed_kernel_one_step/mh_one_step/hmc_kernel_one_step/leapfrog_integrate/while/body/_1244/leapfrog_integrate_one_step/maybe_call_fn_and_grads/value_and_gradients/value_and_gradient/gradients/leapfrog_integrate_one_step/maybe_call_fn_and_grads/value_and_gradients/value_and_gradient/PartitionedCall_grad/PartitionedCall/gradients/JointDistributionNamed/log_prob/JointDistributionNamed_log_prob_GaussianProcess/log_prob/JointDistributionNamed_log_prob_GaussianProcess/get_marginal_distribution/Cholesky_grad/MatrixTriangularSolve}}]] [Op:__inference_do_sampling_113645]&#xA;&#xA;Function call stack:&#xA;do_sampling&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/60063931/error-when-using-tensorflow-hmc-to-marginalise-gpr-hyperparameters&quot;&gt;1&lt;/a&gt; &lt;a href=&quot;https://colab.research.google.com/github/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Regression_In_TFP.ipynb#scrollTo=jw-_1yC50xaM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/github/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Regression_In_TFP.ipynb#scrollTo=jw-_1yC50xaM&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that some of code below is a bit redundant but it should in some sections but it should be able to reproduce the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import time&#xA;&#xA;import numpy as np&#xA;import tensorflow.compat.v2 as tf&#xA;import tensorflow_probability as tfp&#xA;tfb = tfp.bijectors&#xA;tfd = tfp.distributions&#xA;tfk = tfp.math.psd_kernels&#xA;tf.enable_v2_behavior()&#xA;&#xA;import matplotlib&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;#%pylab inline&#xA;# Configure plot defaults&#xA;plt.rcParams['axes.facecolor'] = 'white'&#xA;plt.rcParams['grid.color'] = '#666666'&#xA;#%config InlineBackend.figure_format = 'png'&#xA;&#xA;def sinusoid(x):&#xA;  return np.sin(3 * np.pi * x[..., 0])&#xA;&#xA;def generate_1d_data(num_training_points, observation_noise_variance):&#xA;  &quot;&quot;&quot;Generate noisy sinusoidal observations at a random set of points.&#xA;&#xA;  Returns:&#xA;     observation_index_points, observations&#xA;  &quot;&quot;&quot;&#xA;  index_points_ = np.random.uniform(-1., 1., (num_training_points, 1))&#xA;  index_points_ = index_points_.astype(np.float64)&#xA;  # y = f(x) + noise&#xA;  observations_ = (sinusoid(index_points_) +&#xA;                   np.random.normal(loc=0,&#xA;                                    scale=np.sqrt(observation_noise_variance),&#xA;                                    size=(num_training_points)))&#xA;  return index_points_, observations_&#xA;&#xA;# Generate training data with a known noise level (we'll later try to recover&#xA;# this value from the data).&#xA;NUM_TRAINING_POINTS = 100&#xA;observation_index_points_, observations_ = generate_1d_data(&#xA;    num_training_points=NUM_TRAINING_POINTS,&#xA;    observation_noise_variance=.1)&#xA;&#xA;def build_gp(amplitude, length_scale, observation_noise_variance):&#xA;  &quot;&quot;&quot;Defines the conditional dist. of GP outputs, given kernel parameters.&quot;&quot;&quot;&#xA;&#xA;  # Create the covariance kernel, which will be shared between the prior (which we&#xA;  # use for maximum likelihood training) and the posterior (which we use for&#xA;  # posterior predictive sampling)&#xA;  kernel = tfk.ExponentiatedQuadratic(amplitude, length_scale)&#xA;&#xA;  # Create the GP prior distribution, which we will use to train the model&#xA;  # parameters.&#xA;  return tfd.GaussianProcess(&#xA;      kernel=kernel,&#xA;      index_points=observation_index_points_,&#xA;      observation_noise_variance=observation_noise_variance)&#xA;&#xA;gp_joint_model = tfd.JointDistributionNamed({&#xA;    'amplitude': tfd.LogNormal(loc=0., scale=np.float64(1.)),&#xA;    'length_scale': tfd.LogNormal(loc=0., scale=np.float64(1.)),&#xA;    'observation_noise_variance': tfd.LogNormal(loc=0., scale=np.float64(1.)),&#xA;    'observations': build_gp,&#xA;})&#xA;&#xA;x = gp_joint_model.sample()&#xA;lp = gp_joint_model.log_prob(x)&#xA;&#xA;print(&quot;sampled {}&quot;.format(x))&#xA;print(&quot;log_prob of sample: {}&quot;.format(lp))&#xA;&#xA;# Create the trainable model parameters, which we'll subsequently optimize.&#xA;# Note that we constrain them to be strictly positive.&#xA;&#xA;constrain_positive = tfb.Shift(np.finfo(np.float64).tiny)(tfb.Exp())&#xA;&#xA;amplitude_var = tfp.util.TransformedVariable(&#xA;    initial_value=1.,&#xA;    bijector=constrain_positive,&#xA;    name='amplitude',&#xA;    dtype=np.float64)&#xA;&#xA;length_scale_var = tfp.util.TransformedVariable(&#xA;    initial_value=1.,&#xA;    bijector=constrain_positive,&#xA;    name='length_scale',&#xA;    dtype=np.float64)&#xA;&#xA;observation_noise_variance_var = tfp.util.TransformedVariable(&#xA;    initial_value=1.,&#xA;    bijector=constrain_positive,&#xA;    name='observation_noise_variance_var',&#xA;    dtype=np.float64)&#xA;&#xA;trainable_variables = [v.trainable_variables[0] for v in &#xA;                       [amplitude_var,&#xA;                       length_scale_var,&#xA;                       observation_noise_variance_var]]&#xA;# Use `tf.function` to trace the loss for more efficient evaluation.&#xA;@tf.function(autograph=False, experimental_compile=False)&#xA;def target_log_prob(amplitude, length_scale, observation_noise_variance):&#xA;  return gp_joint_model.log_prob({&#xA;      'amplitude': amplitude,&#xA;      'length_scale': length_scale,&#xA;      'observation_noise_variance': observation_noise_variance,&#xA;      'observations': observations_&#xA;  })&#xA;&#xA;# Now we optimize the model parameters.&#xA;num_iters = 1000&#xA;optimizer = tf.optimizers.Adam(learning_rate=.01)&#xA;&#xA;# Store the likelihood values during training, so we can plot the progress&#xA;lls_ = np.zeros(num_iters, np.float64)&#xA;for i in range(num_iters):&#xA;  with tf.GradientTape() as tape:&#xA;    loss = -target_log_prob(amplitude_var, length_scale_var,&#xA;                            observation_noise_variance_var)&#xA;  grads = tape.gradient(loss, trainable_variables)&#xA;  optimizer.apply_gradients(zip(grads, trainable_variables))&#xA;  lls_[i] = loss&#xA;&#xA;print('Trained parameters:')&#xA;print('amplitude: {}'.format(amplitude_var._value().numpy()))&#xA;print('length_scale: {}'.format(length_scale_var._value().numpy()))&#xA;print('observation_noise_variance: {}'.format(observation_noise_variance_var._value().numpy()))&#xA;&#xA;&#xA;num_results = 100&#xA;num_burnin_steps = 50&#xA;&#xA;&#xA;sampler = tfp.mcmc.TransformedTransitionKernel(&#xA;    tfp.mcmc.HamiltonianMonteCarlo(&#xA;        target_log_prob_fn=target_log_prob,&#xA;        step_size=tf.cast(0.1, tf.float64),&#xA;        num_leapfrog_steps=8),&#xA;    bijector=[constrain_positive, constrain_positive, constrain_positive])&#xA;&#xA;adaptive_sampler = tfp.mcmc.DualAveragingStepSizeAdaptation(&#xA;    inner_kernel=sampler,&#xA;    num_adaptation_steps=int(0.8 * num_burnin_steps),&#xA;    target_accept_prob=tf.cast(0.75, tf.float64))&#xA;&#xA;initial_state = [tf.cast(x, tf.float64) for x in [1., 1., 1.]]&#xA;&#xA;# Speed up sampling by tracing with `tf.function`.&#xA;@tf.function(autograph=False, experimental_compile=False)&#xA;def do_sampling():&#xA;    return tfp.mcmc.sample_chain(&#xA;      kernel=adaptive_sampler,&#xA;      current_state=initial_state,&#xA;      num_results=num_results,&#xA;      num_burnin_steps=num_burnin_steps,&#xA;      trace_fn=lambda current_state, kernel_results: kernel_results)&#xA;&#xA;t0 = time.time()&#xA;samples, kernel_results = do_sampling()&#xA;t1 = time.time()&#xA;print(&quot;Inference ran in {:.2f}s.&quot;.format(t1-t0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33293" LastActivityDate="2020-02-05T09:38:46.360" Title="Error when using tensorflow HMC to marginalise GPR hyperparameters" Tags="&lt;tensorflow&gt;&lt;gaussian-process&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17857" PostTypeId="1" CreationDate="2020-02-05T11:10:30.670" Score="4" ViewCount="1060" Body="&lt;p&gt;When using convolutional networks on images with multiple channels, do we max pool after we sum the feature map from each channel, or do we max pool each feature map separately and then sum? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the intuition behind this, or is there a difference between the two?&lt;/p&gt;&#xA;" OwnerUserId="33296" LastEditorUserId="2444" LastEditDate="2020-02-05T12:27:29.860" LastActivityDate="2020-02-05T21:13:49.560" Title="When is max pooling exactly applied in convolutional neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17858" PostTypeId="2" ParentId="17808" CreationDate="2020-02-05T11:54:04.117" Score="2" Body="&lt;p&gt;The left hand graphs are showing you the &lt;em&gt;estimated&lt;/em&gt; value function from using Monte Carlo evaluation, after 10,000 episodes. They give a sense of what your value table will look like before convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of upper &quot;usable ace&quot; chart, the estimates are still showing a lot of inaccuracy due to variance in the data. This is for two main reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The probability of getting a usable ace at the start is a fraction of all (around 15%), so the number of samples used to build the chart is lower.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are more variations in play when there is a usable ace, due to the extra flexibility allowed by it, so the end result also varies more, requiring more samples to converge.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition, if you are looking at the bottom edge of the chart, this represents the player starting with two aces. If you pick on one of the high points (dealer shows 4), then that also reduces the probability of seeing that particular state. So you are looking at a sample size of typically 4-5, but maybe in this case just one sample or maybe two, which the player then happened to go on to win, even though the odds made it unlikely. There is always some chance of winning and dealer showing 4 is a bad start for the dealer, who has a good chance of going bust provided the player does not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it hadn't happened this time for the &quot;two aces + dealer showing 4&quot; state, it may have happened for &quot;two aces + dealer showing 5&quot; state. That's due to  the nature of random sampling - if you have hundreds of states to sample, then a few of them are going to behave like outliers purely by chance until you have taken enough samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, 10,000 randomly sampled games are nowhere near enough to reduce error bounds on the value estimates to reasonable numbers for special cases such as starting with two aces. However, you can see in the 10,000 samples charts the beginnings of convergence, especially elsewhere in the chart.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From the graph, for 10000 episodes what i see is that when we don't have a usable ace we always lose the game except if the sum is 20 or 21.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Actually you don't see that, the expected result is not quite -1.0, but a little higher. So that means there is still a chance to win. Under this player policy, the worst chance is no usable ace and score 19, because the policy will be to &quot;hit&quot; and need an Ace or 2 card just to stay in the game. Even then , the value is not quite as low as -1.0, but more like -0.9&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-02-05T12:41:22.060" LastActivityDate="2020-02-05T12:41:22.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17859" PostTypeId="1" CreationDate="2020-02-05T13:10:45.213" Score="2" ViewCount="44" Body="&lt;p&gt;I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even I have more than 2k different poses, but &lt;strong&gt;the background does not change much&lt;/strong&gt; (I have only 5 different points of view).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trained my model on these images (70% for the training and 30% for validation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now, I want to evaluate&lt;/strong&gt; the model on an unbiased dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Should I keep images of my dataset for this purpose or should I use a real life dataset ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A good solution would be to have a real life training set, but I don't have)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried both, but as expected, I have an mAP=0.9 when evaluated on similar pictures and mAP=0.5 when evaluated on completely different images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus question&lt;/strong&gt;: is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him &quot;my model has a mAP=0.7&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Precision-Recall ? (but I have to choose a pose classification threshold...)&lt;/p&gt;&#xA;" OwnerUserId="33295" LastActivityDate="2020-02-28T15:40:15.223" Title="On which data evaluate an object detection model ? (similar or real life data ?)" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17860" PostTypeId="2" ParentId="17773" CreationDate="2020-02-05T14:23:36.780" Score="0" Body="&lt;p&gt;@BrianSpiering was generally  correct in pointing out that you should always apply the same transformations to your train and test dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was the key to my solution which was a bit more specific but might actually help others who encounter a similar problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically my mistake came about because of &lt;strong&gt;Imputation&lt;/strong&gt;! Some of the factors I used for my model were &lt;code&gt;NA&lt;/code&gt; in both the train and test data set. To complete the data I simply imputed these missings using mean and mode respectively. However since I did those transformations separately on both sets the actual mean/mode value that was used differed heavily! By applying the imputation on the full data I also imputed the same data for all missing cases which solved my error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My resulting accuracy in the public leaderboard is now at 74.2% which is fairly close to my local test score of 79.6%.&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2020-02-05T14:23:36.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17862" PostTypeId="2" ParentId="17768" CreationDate="2020-02-05T15:36:26.337" Score="1" Body="&lt;p&gt;&lt;em&gt;NDDS is a UE4 plugin from NVIDIA to empower computer vision researchers to export high -quality synthetic images with metadata.&#xA;NDDS supports images, segmentation, depth, object pose, bounding box, keypoints, and custom stencils. In addition to the expo rter,&#xA;the plugin includes different components for generating highly randomized images. This randomization includes lighting, objec ts,&#xA;camera position, poses, textures, and distractors, as well as camera path following, and so forth. Together, these components allow&#xA;researchers to easily create randomized scenes for training deep neural networks&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/yehengchen/DOPE-ROS-D435/blob/master/Synthetic-Data-UE4-DOPE.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/yehengchen/DOPE-ROS-D435/blob/master/Synthetic-Data-UE4-DOPE.md&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can create 3D synthetic dataset here.&lt;/p&gt;&#xA;" OwnerUserId="33272" LastActivityDate="2020-02-05T15:36:26.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17864" PostTypeId="2" ParentId="17857" CreationDate="2020-02-05T15:39:20.437" Score="1" Body="&lt;p&gt;The pooling operation is applied to the output of the convolution layer. More precisely, it is applied separately for each of the input channels (or slices). So, if the pooling layer receives an input volume of &lt;span class=&quot;math-container&quot;&gt;$H_i \times W_i \times D$&lt;/span&gt;, then it will produce an output volume &lt;span class=&quot;math-container&quot;&gt;$H_o \times W_o \times D$&lt;/span&gt;, so the depth of the output volume is equal to the depth of the input volume. There is no sum involved in the pooling operation. For example, in the case of max pooling, you will choose the maximum number of a certain 2D window of values. You do this for each of the input channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the article (which is part of some course notes) &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/#pool&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Networks: Architectures, Convolution / Pooling Layers&lt;/a&gt;, Andrej Karpathy says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Pooling layer downsamples the volume spatially, independently in each depth slice of the input volume.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The following diagram (a screenshot of one of the figures from the mentioned article) should provide some intuition behind the pooling operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pcJO9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pcJO9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-02-05T15:39:20.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17865" PostTypeId="1" CreationDate="2020-02-05T17:14:45.047" Score="1" ViewCount="32" Body="&lt;p&gt;Interpolation is a common way to make an image fit the right input shape for a neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is there any point in using interpolation to make it easier for the network to learn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume interpolation adds no extra information to the input; It only uses existing information to increase the resolution and fill missing values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, sometimes I have observed that while I can not see anything with my human eye, using some kind of advanced interpolation technique such as b-spline interpolation makes it crystal clear that the object i am looking for is in the image, especially in the domain of infrared images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there any benefit for using interpolation rather than feeding a low dimensional image to a neural network?&lt;/p&gt;&#xA;" OwnerUserId="13257" LastActivityDate="2020-02-08T10:54:53.307" Title="Interpolating image to increase resolution before feeding it to a neural network" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;interpolation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17867" PostTypeId="2" ParentId="17857" CreationDate="2020-02-05T19:55:43.757" Score="1" Body="&lt;p&gt;The pooling operation in a CNN is applied &lt;em&gt;independently&lt;/em&gt; to each layer and the resulting feature maps are disjoint. This is the very reason that in most schematics depicting a certain CNN architecture, we obtain three output maps from an input image (corresponding to the convolutions and pooling operations performed on the RGB channels separately).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lkl5q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lkl5q.png&quot; alt=&quot;CNN&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each channel in an image captures a set of information that might not be demonstrated by the other channels owing to the &lt;em&gt;color receptivity&lt;/em&gt;. Hence, pooling performed independently is intuitive, in the sense that a group of pixels in the Red Channel may not provide similar features as the same set of pixels in a Blue or Green channel. Thus comparison of pixels is restricted within a channel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following image provides a visualization of the RGB channels converted to greyscale. Note the brightness of the Red Color (comparing with the color-image) in the three different channels for a better intuition: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/KRG3v.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KRG3v.png&quot; alt=&quot;Channel&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Channel_(digital_image)#RGB_color_sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;[Image Source]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, to feed the multi-dimensional outputs of the Convolutional Layers to a Fully Connected layer, the feature maps are stacked along the depth dimension and flattened to form a vector. This ensures that the Dense layer learns to classify images (using a Softmax activation in the final Dense layer) based on the non-linear combinations of these high-level features. &lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-02-05T21:13:49.560" LastActivityDate="2020-02-05T21:13:49.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17868" PostTypeId="1" AcceptedAnswerId="17915" CreationDate="2020-02-05T20:39:49.977" Score="1" ViewCount="91" Body="&lt;p&gt;I'm training a deep network in &lt;code&gt;Keras&lt;/code&gt; on some images for a binary classification (I have around 12K images). Once in a while, I collect some false positives and add them to my training sets and re-train for higher accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I split my training into 20/80 percent for training/validation sets. Now my question is which resulting model should I use? Always the one with higher validation accuracy, or maybe the higher mean of training and validation acc? In specfic, which one the two would you prefer?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch #38: training acc: 0.924, validation acc: 0.944&#xA;Epoch #90: training acc: 0.952, validation acc: 0.932&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-02-08T01:14:05.177" Title="Training accuracy vs validation accuracy on deep models" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;keras&gt;&lt;deep-neural-networks&gt;&lt;accuracy&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17869" PostTypeId="2" ParentId="17868" CreationDate="2020-02-05T21:31:41.247" Score="0" Body="&lt;p&gt;The training accuracy tells you nothing about how good it is on other data than the ones it learned on, it could be better on this data because it memorized this examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand the validation set is here to indicate you how good the model is to generalize what it learned to new data (hopefully the testing dataset accurately represents the diversity of the data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you are looking for a model which is good on every dataset you don't want to use training accuracy to choose your model and so you should choose the first one.&lt;/p&gt;&#xA;" OwnerUserId="26961" LastActivityDate="2020-02-05T21:31:41.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17870" PostTypeId="1" CreationDate="2020-02-06T01:16:47.363" Score="4" ViewCount="425" Body="&lt;p&gt;Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?&lt;/p&gt;&#xA;" OwnerUserId="33314" LastEditorUserId="2444" LastEditDate="2021-01-22T15:58:21.950" LastActivityDate="2021-01-22T15:58:21.950" Title="How to estimate the capacity of a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;capacity&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17871" PostTypeId="2" ParentId="17521" CreationDate="2020-02-06T01:41:45.977" Score="2" Body="&lt;p&gt;Apart from the multitudes of traditional image segmentation techniques (Watershed, Clustering or Variational methods), newer Segmentation schemes using Deep Learning are actively being used, which provide better results and are better for real-time applications, owing to minimum computation overheads involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following blog provides a detailed review of recent advancements in this field: &lt;a href=&quot;https://medium.com/@arthur_ouaknine/review-of-deep-learning-algorithms-for-image-semantic-segmentation-509a600f7b57&quot; rel=&quot;nofollow noreferrer&quot;&gt;Review of Deep Learning Algorithms for Image Semantic Segmentation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the traditional methods, this Wikipedia article provides a nice summary:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Image_segmentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Segmentation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33314" LastActivityDate="2020-02-06T01:41:45.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17872" PostTypeId="1" CreationDate="2020-02-06T04:13:48.833" Score="2" ViewCount="271" Body="&lt;p&gt;I'm new to Speech Synthesis &amp;amp; Deep Learning. Recently, I got a task as described below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have problem in training a multi-speaker model which should be created by &lt;strong&gt;&lt;a href=&quot;https://github.com/NVIDIA/tacotron2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tacotron2&lt;/a&gt;&lt;/strong&gt;. And I was told I can get some ideas from &lt;strong&gt;&lt;a href=&quot;https://github.com/espnet/espnet&quot; rel=&quot;nofollow noreferrer&quot;&gt;espnet&lt;/a&gt;&lt;/strong&gt;, which is a end-to-end audio tools library. In this way, I found a good dataset called &lt;strong&gt;libritts:&lt;/strong&gt; &lt;a href=&quot;http://www.openslr.org/60/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openslr.org/60/&lt;/a&gt;. And it's also found at &lt;strong&gt;espnet&lt;/strong&gt;: &lt;a href=&quot;https://github.com/espnet/espnet#tts-results&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/espnet/espnet#tts-results&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/k2yQF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k2yQF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Here is my initial thought：&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Download libritts corpus / Read the &lt;strong&gt;espnet&lt;/strong&gt; code: &lt;em&gt;../espnet/egs/libritts/tts1/run.sh&lt;/em&gt;, learning how to train the libritts corpus by pytorch-backend. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Difficulty:&lt;/strong&gt; But I cannot get it across that how the author trained a libritts.tacotron2.v1 as I didn't found anything about tacotron2 along those shells related to run.sh. Maybe He didn't make those codes open-source.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Read the tacotron2 code and tune it into a multi-speaker network:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Difficulty:&lt;/strong&gt; I found the code is really complex.... I just got lost in reading these codes, without a clear understanding about how to tune this model. Cause Tacotron2 was designed with a LJSpeech Dataset(only 1 person).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Training the multi-speaker model with tiny set of dataset (&lt;a href=&quot;http://www.openslr.org/60/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openslr.org/60/&lt;/a&gt;) to save time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o8rHl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o8rHl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;they contains about 110 people's data, which can be enough for my scenario.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;In the end:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Coud you please help me about my questions. I've been puzzled by this problem for a long time...&lt;/p&gt;&#xA;" OwnerUserId="33317" LastActivityDate="2020-02-06T04:13:48.833" Title="How do I train a multiple-speaker model (speech synthesis) based on Tacotron 2 and espnet?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;audio-processing&gt;&lt;speech-recognition&gt;&lt;speech-synthesis&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17873" PostTypeId="1" AcceptedAnswerId="17874" CreationDate="2020-02-06T06:53:36.177" Score="1" ViewCount="86" Body="&lt;p&gt;There's one VAE example here: &#xA;&lt;a href=&quot;https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the source code of encoder can be found at the following URL: &lt;a href=&quot;https://gist.github.com/FelixMohr/29e1d5b1f3fd1b6374dfd3b68c2cdbac#file-vae-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/FelixMohr/29e1d5b1f3fd1b6374dfd3b68c2cdbac#file-vae-py&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author is using &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; (natural exponential) for calculating values of the embedding vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$z = \mu + \epsilon \times e^{\sigma}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the mean, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; a small random number and &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; the standard deviation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or in code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z  = mn + tf.multiply(epsilon, tf.exp(sd))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's not related to the code (practical programming), but why using natural exponential instead of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$z = \mu + \epsilon \times \sigma$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-02-20T00:48:03.327" LastActivityDate="2020-02-20T00:48:03.327" Title="Why is exp used in encoder of VAE instead of using the value of standard deviation alone?" Tags="&lt;machine-learning&gt;&lt;autoencoders&gt;&lt;variational-autoencoder&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17874" PostTypeId="2" ParentId="17873" CreationDate="2020-02-06T07:37:47.667" Score="2" Body="&lt;p&gt;In the source code, the author defines &lt;code&gt;sd&lt;/code&gt; by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sd       = 0.5 * tf.layers.dense(x, units=n_latent)    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which means that &lt;span class=&quot;math-container&quot;&gt;$\operatorname{sd}\in \mathbb{R}^n$&lt;/span&gt;.&#xA;In particular, the support over &lt;code&gt;sd&lt;/code&gt; includes negative numbers, which is something we want to avoid.&#xA;Since standard deviations are always nonnegative, we can exponentiate to get us in the correct domain.&#xA;This is a case where the variable is inappropriately named.&#xA;Here, &lt;code&gt;sd&lt;/code&gt; is not the standard deviation itself but rather the &lt;em&gt;logarithm&lt;/em&gt; of the standard deviation. This allows it to be predicted as the output of a layer in a neural network, so extracting the predicted value of the standard deviation would require exponentiation.&lt;/p&gt;&#xA;" OwnerUserId="33286" LastActivityDate="2020-02-06T07:37:47.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17875" PostTypeId="2" ParentId="17870" CreationDate="2020-02-06T07:51:57.057" Score="0" Body="&lt;p&gt;Most methods for measuring the complexity of neural networks are fairly crude. One common measure of complexity is &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC dimension&lt;/a&gt;, a discussion which can be found &lt;a href=&quot;https://stats.stackexchange.com/questions/25952/calculating-vc-dimension-of-a-neural-network&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://cs.stackexchange.com/questions/75327/why-is-deep-learning-hyped-despite-bad-vc-dimension&quot;&gt;here&lt;/a&gt;. For example, neural networks have a &lt;a href=&quot;https://arxiv.org/pdf/1703.02930.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC dimension that is too large&lt;/a&gt; to give a strong upper bound on the number of training samples needed for a model (the upper bound provided by VC analysis is much higher than what we have observed neural networks to be able to generalize from).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another common measure of capacity is the number of parameters. We see in the paper &quot;&lt;a href=&quot;https://arxiv.org/pdf/1611.03530v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding deep learning requires rethinking generalization&lt;/a&gt;&quot;, published at ICLR with over 1400+ citations, that networks with more parameters than data often have the capacity to memorize the data. The paper provides compelling evidence that traditional approaches to generalization provided by statistical learning theory (VC dimension, Rademacher complexity) are unable to fully explain the apparent capacity of neural networks. In general, neural networks seem to have a large capacity, given the apparent good performance on certain tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond these ideas, the universal approximation theorem tells us that the set of neural networks can approximate any continuous function arbitrarily well, which strongly suggests that any neural network has a large capacity.&lt;/p&gt;&#xA;" OwnerUserId="33286" LastEditorUserId="33286" LastEditDate="2020-02-11T05:45:22.643" LastActivityDate="2020-02-11T05:45:22.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17876" PostTypeId="1" CreationDate="2020-02-06T09:55:03.257" Score="2" ViewCount="25" Body="&lt;p&gt;To explain what I mean I'll depict the two extremes and something in the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;strong&gt;Most pragmatic&lt;/strong&gt;: If you need to just segment a few images for a design project, forget AI. Go into Adobe Photoshop and hand select the outline of the object you need to extract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;strong&gt;Middle ground&lt;/strong&gt;: If you need to build a reasonably accurate app for human aided segmentation of images, use a pre-trained model on a well known architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) &lt;strong&gt;Least pragmatic&lt;/strong&gt;: If you need to reach unprecedented levels of accuracy on a large volume of images. Do heavily funded research on new and better methods of image segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm most interested in painting out the spectrum for that middle ground. That is, how much of the wheel needs to be reinvented versus the complexity of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example (and this is what lead me here), I need to segment out dogs from several hundred photos that owners have taken. The dog is probably going to be among the main subjects of the photos. Do I need to reinvent the wheel (design an architecture)? Probably not. Do I even need to change the tyres (train my own model)? I'm guessing not. Do I need to code at all? I'm not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I'm happy to get answers about my use case, it would be awesome if someone could map out the spectrum on my unfinished rainbow.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-02-06T09:55:03.257" Title="What are the current tools and techniques for image segmentation in order of pragmatism?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17877" PostTypeId="1" CreationDate="2020-02-06T11:04:16.433" Score="1" ViewCount="15" Body="&lt;p&gt;I have a deep learning problem, I am working with the CMAPSS dataset, which contains data simulating the degradation of several aircraft engines. The aim is to predict from data collected on a machine in full operation, the remaining useful time at this machine. My problem is the following when the features (sensor data) have a specific trend (either up or down), my model (LSTMs) predicts good results but when the data have no trend, my deep learning model gets a very bad score. I must specify that I work with sequential data. my dataset contains several aircraft engines with data recorded by the sensors. my question is how to process the data that has no particular trend in deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find below some pictures of my dataset where, RUL: is the life of a machine, unit: the machine identity and s1 to s3 is the sensor data.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/r6fQ5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r6fQ5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33323" LastActivityDate="2020-02-06T11:04:16.433" Title="Improve prediction with LSTMs when data have no particular trend (complex)" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17879" PostTypeId="2" ParentId="17820" CreationDate="2020-02-06T11:20:22.700" Score="0" Body="&lt;p&gt;I think there are some problems with your approach.&lt;/p&gt;&#xA;&lt;p&gt;Firstly, looking at the Keras documentation, LSTM expects an input of shape &lt;code&gt;(batch_size, timesteps, input_dim)&lt;/code&gt;. You're passing an input of shape &lt;code&gt;(1000, 1, 1)&lt;/code&gt;, which means that you're having &amp;quot;sequences&amp;quot; of 1 timestep.&lt;/p&gt;&#xA;&lt;p&gt;RNNs have been proposed to capture temporal dependencies, but it's impossible to capture such dependencies when the length of each series is 1, and the numbers are randomly generated. If you want to create a more realistic scenario, I would suggest you generate a sine wave, since it has a smooth periodic oscillation. Afterward, increase the timesteps from 1, and you can test on the following timestamps (to make predictions).&lt;/p&gt;&#xA;&lt;p&gt;For the second part, if you think about a normal RNN (I will explain for a simple RNN but you can imagine a similar flow for LSTM) and a &lt;code&gt;Dense&lt;/code&gt; layer when applied to 1 timestamp, there are not so many many differences. The dense layer will have &lt;span class=&quot;math-container&quot;&gt;$Y=f(XW + b)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is the input, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is the weight matrix, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the bias and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the activation function. Whereas RNN will have &lt;span class=&quot;math-container&quot;&gt;$Y=f(XW_1 + W_2h_0 + b)$&lt;/span&gt;, since is the first timestamp &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, so we can reduce it to &lt;span class=&quot;math-container&quot;&gt;$Y=f(XW_1 +b)$&lt;/span&gt;, which is identical with the &lt;code&gt;Dense&lt;/code&gt; layer. I suspect that the results differences are caused by the activation functions, by default &lt;code&gt;Dense&lt;/code&gt; layer has no activation function, and LSTM has tanh and sigmoid.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2020-07-05T15:06:40.883" LastActivityDate="2020-07-05T15:06:40.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17881" PostTypeId="2" ParentId="17870" CreationDate="2020-02-06T14:08:35.860" Score="4" Body="&lt;h1&gt;VC dimension&lt;/h1&gt;&#xA;&lt;p&gt;A rigorous measure of the capacity of a neural network is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;VC dimension&lt;/strong&gt;&lt;/a&gt;, which is &lt;em&gt;intuitively&lt;/em&gt; a number or &lt;strong&gt;bound&lt;/strong&gt; that quantifies the difficulty of learning from data.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;sample complexity&lt;/strong&gt;, which is the number of training instances that the model (or learner) must be exposed to in order to be reasonably certain of the accurateness of the predictions made given some data, is proportional to this number.&lt;/p&gt;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt; (1998) by Eduardo D. Sontag provides a good introduction to the VC dimension of neural networks (even though these concepts are quite abstract and you may need to read them several times to fully grasp them). The information in this answer is highly based on that paper.&lt;/p&gt;&#xA;&lt;h2&gt;Shattering and VC dimension&lt;/h2&gt;&#xA;&lt;p&gt;In section 2, &lt;strong&gt;Concepts and VC Dimension&lt;/strong&gt;, he describes the basic concepts behind the VC dimension (not only for neural networks), such as the concept of &lt;strong&gt;shattering&lt;/strong&gt; (i.e. what does it mean for a set of sets to shatter another set?), which is a well-known concept in &lt;a href=&quot;https://ai.stackexchange.com/a/17842/2444&quot;&gt;computational learning theory&lt;/a&gt; and is used to define the VC dimension (see definition 2), so you definitely need to get familiar with this concept to understand the VC dimension and, therefore, the capacity of a neural network (calculated with the VC dimension).&lt;/p&gt;&#xA;&lt;h2&gt;VC dimension of functions and neural networks&lt;/h2&gt;&#xA;&lt;p&gt;He then provides an equivalent definition of the VC dimension but for functions (equation 6). Given that neural networks represent functions, then we can also define the VC dimension of a neural network. A specific combination of weights of neural networks represents a specific function, for which the VC dimension can be defined. To be more precise, a parametrized function (and a neural network) can be denoted as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\beta : \mathbb{W} \times \mathbb{U} \rightarrow \mathbb{R}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{W} = \mathbb{R}^p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the number of weights (or parameters) of the neural network, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{U}$&lt;/span&gt; is the input space and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; the output space. So, in this case, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; can also represent a neural network, with a certain parameter space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{W}$&lt;/span&gt;, an input space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{U}$&lt;/span&gt; and an output space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} = (w_1, \dots, w_p) \in \mathbb{W}$&lt;/span&gt; represents a specific combination of weights of the neural network, so it represents a specific function. The set of all functions for each choice of this weight vector can be denoted as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{F}_{\beta} = \{ \beta(\mathbf{w}, \cdot) \mid \mathbf{w} \in \mathbb{W} \}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The VC dimension (VCD) of &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; can then be defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{VCD}(\beta) := \text{VCD}(\mathcal{F}_{\beta})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Therefore, the VC dimension is a measure of the capacity of a neural network with a &lt;strong&gt;certain&lt;/strong&gt; architecture. Moreover, the VC dimension is equivalently defined for a certain set of functions associated with a neural network.&lt;/p&gt;&#xA;&lt;h2&gt;How to calculate the VC dimension?&lt;/h2&gt;&#xA;&lt;p&gt;To calculate the actual VC dimension of a neural network, it takes a little bit of more creativity. Therefore, I will just report the VC dimension of some neural networks. For more details, you should fully read the cited paper (more than once) and other papers and books too (especially, the ones described in &lt;a href=&quot;https://ai.stackexchange.com/a/17842/2444&quot;&gt;this answer&lt;/a&gt;, which provide an introduction to CLT concepts).&lt;/p&gt;&#xA;&lt;h3&gt;VC dimension of a perceptron&lt;/h3&gt;&#xA;&lt;p&gt;The VC dimension of a perceptron is &lt;span class=&quot;math-container&quot;&gt;$m + 1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of inputs. Given that a perceptron represents a linear and affine function, the VC dimension of the perceptron is also equal to the number of parameters. However, note that, even though the VC dimension of the perceptron is linear in the number of parameters and inputs, it doesn't mean the perceptron can learn any function. In fact, perceptrons can only represent linear functions. See section 3.1 of &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;h3&gt;VC dimension of a single hidden layer neural network&lt;/h3&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; be the number of hidden units, then the VC dimension of a single hidden layer neural network is less than or equal to &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt;. See section 3.2 of &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;h3&gt;VC dimension of multi-layer neural networks with binary activations&lt;/h3&gt;&#xA;&lt;p&gt;The VC dimension of multi-layer neural networks (MLPs) with binary activations and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; weights (or parameters) is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(p \log p)$&lt;/span&gt;. See theorem 4 (and related sections) of the paper &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;h3&gt;VC dimension of MLPs with real-valued activations&lt;/h3&gt;&#xA;&lt;p&gt;The VC dimension of MLPs with real-valued activations is no longer bounded by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(p \log p)$&lt;/span&gt; and can be exponential in the number of parameters. See section 5.3 of &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;VC dimension of MLPs with linear activations&lt;/h3&gt;&#xA;&lt;p&gt;The VC dimension of MLPs with linear activations is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(p^2)$&lt;/span&gt;.  See theorem 5 of the paper &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Notes&lt;/h2&gt;&#xA;&lt;p&gt;The VC dimension is often expressed as a bound (e.g. with big-O notation), which may &lt;strong&gt;not&lt;/strong&gt; be strict.&lt;/p&gt;&#xA;&lt;p&gt;In any case, the VC dimension is useful because it provides some guarantees. For example, if you use the VC dimension to describe an upper bound on the number of samples required to learn a certain task, then you have a precise mathematical formula that guarantees that you will not need more samples than those expressed by the bound in order to achieve a small generalization error, but, in practice, you may need fewer samples than those expressed by the bound (because these bounds may not be strict or the VC dimension may also not be strict).&lt;/p&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;There is a more recent paper (published in 2017 in MLR) that proves new and tighter upper and lower bounds on the &lt;strong&gt;VC dimension of deep neural networks with the ReLU activation function&lt;/strong&gt;: &lt;a href=&quot;http://proceedings.mlr.press/v65/harvey17a/harvey17a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nearly-tight VC-dimension bounds for piecewise linear neural networks&lt;/a&gt;. So, you probably should read this paper first.&lt;/p&gt;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1802.04443.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Characterizing the Capacity of Neural Networks using Algebraic Topology&lt;/a&gt; may also be useful and interesting. See also section 6, Algebraic Techniques, of the paper I have been citing: &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The capacity of a neural network is clearly related to the number of functions it can represent, so it is strictly related to the universal approximation theorems for neural networks. See &lt;a href=&quot;https://ai.stackexchange.com/a/13319/2444&quot;&gt;Where can I find the proof of the universal approximation theorem?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-13T21:31:40.457" LastActivityDate="2020-11-13T21:31:40.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17883" PostTypeId="1" CreationDate="2020-02-06T14:33:04.163" Score="3" ViewCount="152" Body="&lt;p&gt;I have a question regarding features representation for graph convolutional neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my case, all nodes have a different number of features, and for now, I don't really understand how should I work with these constraints. I can not just reduce the number of features or add meaningless features in order to make the number of features on each node the same - because it will add to much extra noise to the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any ways to solve this problem? How should I construct the feature matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll appreciate any help and if you have any links to papers that solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="33332" LastActivityDate="2020-05-26T22:25:38.333" Title="How to represent and work with the feature matrix for graph convolutional network (GCN) if the number of features for each node is different?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;geometric-deep-learning&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17884" PostTypeId="2" ParentId="17883" CreationDate="2020-02-06T15:12:30.660" Score="2" Body="&lt;p&gt;The simplest way I could come with is to pad with 0 each feature which is not present. You said that you're going to add too much noise to the network, but I don't see the problem (please correct me if I'm wrong). For example we have two nodes, the first one has only 2 features with the 3rd one missing and the second node has all features X=[[1,2,0], [3,4,5]]. Now we can project the nodes to a hidden representation (pretty common). I'm going to use a weight matrix of W=[[1], [1], [1]]. The output of XW will be [[3], [12]]. Now let's add a new feature to the second node X=[[1,2,0, 0], [3,4,5,6]] and apply the same transformation W=[[1], [1], [1], [1]] the output will be [[3], [18]] you can see that the first node is not affected by the number of missing features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way you could achieve this if you don't want to use the projection could be using a mask. For example give the same example above we could create a mask M=[[1,1,0], [1,1,1]] where each entry represents if a specific feature is present in a specific node. Now usually a GCN layer is defined as H=f(AHW) where A is the adjacency matrix. We could change the propagation rule to H=f(AH*MW) where * is the pointwise multiplication. Like this if a node is missing a feature it can not &quot;access&quot; information from others that are having that feature.  &lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="37206" LastEditDate="2020-05-22T07:59:47.497" LastActivityDate="2020-05-22T07:59:47.497" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17886" PostTypeId="2" ParentId="17817" CreationDate="2020-02-06T15:49:00.593" Score="0" Body="&lt;p&gt;Add train_data 1st&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;df = pd.read_excel('data.xlsx', index_col=None, names=['train_data'])&#xA;df['train_data'] = df['train_data'].apply(lambda x: int(x, 16))&#xA;df.to_csv('data.csv', index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2020-02-06T15:49:00.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17887" PostTypeId="1" CreationDate="2020-02-06T18:43:30.983" Score="1" ViewCount="37" Body="&lt;p&gt;I'm aware there are some optimizer such as Adam that adjust the learning rate for each dimension during training. However, afaik, the maximum learning rate they can have is still determined by the user's input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I wonder if there are optimizers that can increase/decrease their overall learning rate and other parameters (such as momentum or even weight decay) autonomously depending on some metric, e.g., validation loss, running average of gradients etc. ?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-02-06T18:43:30.983" Title="Are there optimizers that schedule their learning rate, momentum etc. autonomously?" Tags="&lt;optimization&gt;&lt;hyperparameter-optimization&gt;&lt;adam&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17890" PostTypeId="1" AcceptedAnswerId="17896" CreationDate="2020-02-07T05:31:02.367" Score="2" ViewCount="198" Body="&lt;p&gt;Let's assume we are in a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; grid world with states numbered as &lt;span class=&quot;math-container&quot;&gt;$0,1, \dots, 8$&lt;/span&gt;. Suppose that the goal state is &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt;, the reward of landing in the goal state is &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, and the reward of just wandering around in the grid world is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Is the state-value of state &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; always &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="2444" LastEditDate="2020-11-01T15:10:58.253" LastActivityDate="2020-11-01T15:11:22.057" Title="In reinforcement learning, is the value of terminal/goal state always zero?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17891" PostTypeId="2" ParentId="17839" CreationDate="2020-02-07T05:31:18.193" Score="1" Body="&lt;p&gt;@nbro has already provided a great answer, so i'll just supplement his answer with two specific results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minsky, in his 1969 book &lt;a href=&quot;https://en.wikipedia.org/wiki/Perceptrons_(book)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Perceptrons&lt;/em&gt;&lt;/a&gt; provided a mathematical proof that showed that certain types of neural networks (then called perceptrons) weren't able to compute a function called the XOR function, thus showing that the mind couldn't be implemented on strictly this structure. Minsky further argued that this result would generalize to all neural networks, but he failed to account for an architectural adaptation known as &quot;hidden layers&quot;, which would allow for neural networks to compute the XOR function. This result isn't very relavant in modern times, but the immediate impact of his proof lead to several decades of people ignoring neural networks due to their perceived failings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another commonly cited result is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal approximation theorem&lt;/a&gt;, which shows that a sufficiently wide single layer neural network would be able to approximate (read as: arbitrarily close) any continuous function given appropriate activation function (iirc the activation needed to be non-linear). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also consider the research of MIRI, which in a sense is more of a &quot;pure&quot; study of AI than the examples listed above. Their &lt;a href=&quot;https://arxiv.org/abs/1401.5577&quot; rel=&quot;nofollow noreferrer&quot;&gt;Program Equilibrium via Provability Logic&lt;/a&gt; result was pretty interesting, the gist of that paper is that programs can learn to cooperate in a very simple game if they read each other's source code.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-02-07T18:45:16.917" LastActivityDate="2020-02-07T18:45:16.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17892" PostTypeId="1" AcceptedAnswerId="17900" CreationDate="2020-02-07T07:49:52.957" Score="4" ViewCount="221" Body="&lt;p&gt;Why is the equation &lt;span class=&quot;math-container&quot;&gt;$$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$&lt;/span&gt; true, where &lt;span class=&quot;math-container&quot;&gt;$x^i$&lt;/span&gt; are data points and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; are latent variables?&lt;/p&gt;&#xA;&lt;p&gt;I was reading the original variation autoencoder paper and I don't understand how the marginal is equal to the RHS equation. How does the marginal equal the KL divergence of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; with its approximate distribution plus the variational lower bound?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-11-07T00:53:53.473" LastActivityDate="2020-11-07T00:53:53.473" Title="Why is the evidence equal to the KL divergence plus the loss?" Tags="&lt;loss-functions&gt;&lt;variational-autoencoder&gt;&lt;latent-variable&gt;&lt;kl-divergence&gt;&lt;evidence-lower-bound&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17893" PostTypeId="1" CreationDate="2020-02-07T08:30:54.977" Score="1" ViewCount="26" Body="&lt;p&gt;I am really new to neural networks, so i was following along with a video series, created by '3blue1brown' on youtube. I created an implementation of the network he explained in c++. I am attempting to train the network to recognize hand written characters, using the MNIST data set. What seems to be happening is, rather than actually learn how to recognize the characters, it is just learning how many of each input there is in the data set and doing it with probability. When testing on a smaller dataset this is more noticeable, for example i was testing on a set with 100, and the numbers that were more frequent would always have a slightly higher activation at the end, and others where very close to 0. Here is my code if it helps:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;random&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;&#xA;double weightMutationRate = 1;&#xA;double biasMutationRate = 1;&#xA;&#xA;//Keeps track of the weights, biases, last activation and the derivatives&#xA;//of the weights and biases for a single node in a neural network.&#xA;struct Node&#xA;{&#xA;  std::vector&amp;lt;double&amp;gt; weights;&#xA;  std::vector&amp;lt;double&amp;gt; derivWeights;&#xA;  double activation;&#xA;  double derivActivation;&#xA;  double bias;&#xA;  double derivBias;&#xA;};&#xA;&#xA;//Struct to hold the nodes in each layer of a network.&#xA;struct Layer&#xA;{&#xA;  std::vector&amp;lt;struct Node&amp;gt; nodes;&#xA;};&#xA;&#xA;//Struct to hold the layers in a network.&#xA;struct Network&#xA;{&#xA;  std::vector&amp;lt;struct Layer&amp;gt; layers;&#xA;  double cost;&#xA;};&#xA;&#xA;//Stores the inputs and outputs for a single training example.&#xA;struct Data&#xA;{&#xA;  std::vector&amp;lt;double&amp;gt; inputs;&#xA;  std::vector&amp;lt;double&amp;gt; answers;&#xA;};&#xA;&#xA;//Stores all of the data to be used to train the neural network.&#xA;struct DataSet&#xA;{&#xA;  std::vector&amp;lt;struct Data&amp;gt; data;&#xA;};&#xA;&#xA;//Generates a double by creating a uniform distribution between the two arguments.&#xA;double RandomDouble(double min, double max)&#xA;{&#xA;  std::random_device seed;&#xA;  std::mt19937 random(seed());&#xA;  std::uniform_real_distribution&amp;lt;&amp;gt; dist(min, max);&#xA;  return dist(random);&#xA;}&#xA;&#xA;//Constructs a network with the node count in each layer defined with 'layers'.&#xA;//the first layer will not have any weights and biases and will simply have&#xA;//the activation of the input data.&#xA;struct Network CreateNetwork(std::vector&amp;lt;int&amp;gt; layers, double minWeight = -1, double maxWeight = 1, double minBias = -1, double maxBias = 1)&#xA;{&#xA;  //Network to construct.&#xA;  struct Network network;&#xA;  //Used to store the nodes in the previous layer.&#xA;  int prevLayerNodes;&#xA;  //Iterates through the layers vector and constructs a neural network with the values in&#xA;  //the vector determining how many nodes that are in each of the layers.&#xA;  bool isFirstLayer = true;&#xA;  for (int layerNodes : layers)&#xA;  {&#xA;    //Layer to construct.&#xA;    struct Layer layer;&#xA;    //Creating the nodes for the current layer.&#xA;    for (int i = 0; i &amp;lt; layerNodes; i++)&#xA;    {&#xA;      //Node to construct&#xA;      struct Node node;&#xA;      //Checks to see if the current layer is not the input layer, which does not have&#xA;      //any weights or biases.&#xA;      if (!isFirstLayer)&#xA;      {&#xA;        //Creating weights for the connections between this node&#xA;        //and the nodes in the previous layer.&#xA;        for (int i = 0; i &amp;lt; prevLayerNodes; i++)&#xA;        {&#xA;          //Getting a random double for the weight, between the bounds set in the arguments.&#xA;          double inputWeight = RandomDouble(minWeight, maxWeight);&#xA;          //Adding the inputWeight to the current node.&#xA;          node.weights.push_back(inputWeight);&#xA;          //Adding a 0 to the deriv weights for the weight just added.&#xA;          node.derivWeights.push_back(0.0);&#xA;        }&#xA;        //Getting a random double for the bias, between the bounds set in the arguments.&#xA;        double bias = RandomDouble(minBias, maxBias);&#xA;        //Adding the bias to the current node.&#xA;        node.bias = bias;&#xA;        //Adding the node to the layer.&#xA;      }&#xA;      layer.nodes.push_back(node);&#xA;    }&#xA;    //Updating the isFirstLayer variable if the current layer is the input layer.&#xA;    if (isFirstLayer)&#xA;    {&#xA;      isFirstLayer = false;&#xA;    }&#xA;    //Updating the prevLayerNodes variable for use in the next layer.&#xA;    prevLayerNodes = layerNodes;&#xA;    //Adding the layer to the network.&#xA;    network.layers.push_back(layer);&#xA;  }&#xA;  //Returning the constructed network.&#xA;  return network;&#xA;}&#xA;&#xA;//Outputs the network passed to the networkPrint.txt file.&#xA;void PrintNetwork(struct Network network)&#xA;{&#xA;  std::cout &amp;lt;&amp;lt; &quot;Printing network ...&quot; &amp;lt;&amp;lt; std::endl;&#xA;  std::ofstream networkPrintFile;&#xA;  networkPrintFile.open(&quot;networkPrint.txt&quot;);&#xA;  //Iterates through each of the layers in teh network.&#xA;  for (int i = 0; i &amp;lt; network.layers.size(); i++)&#xA;  {&#xA;    std::cout &amp;lt;&amp;lt; &quot;Layer : &quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt; std::endl;&#xA;    networkPrintFile &amp;lt;&amp;lt; &quot;Layer &quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt; &quot;:&quot; &amp;lt;&amp;lt; std::endl;&#xA;    //Iterates through each of the nodes in the current layer.&#xA;    for (int j = 0; j &amp;lt; network.layers[i].nodes.size(); j++)&#xA;    {&#xA;      networkPrintFile &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; &quot;Node &quot; &amp;lt;&amp;lt; j &amp;lt;&amp;lt; &quot;:&quot; &amp;lt;&amp;lt; std::endl;&#xA;      //Outputs the node's activation into networkPrintFile.&#xA;      double activation = network.layers[i].nodes[j].activation;&#xA;      networkPrintFile &amp;lt;&amp;lt; &quot;\t\t&quot; &amp;lt;&amp;lt; &quot;Activation&quot; &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; activation &amp;lt;&amp;lt; std::endl;&#xA;      //Outputs the node's derivActivation into networkPrintFile.&#xA;      double derivActivation = network.layers[i].nodes[j].derivActivation;&#xA;      networkPrintFile &amp;lt;&amp;lt; &quot;\t\t&quot; &amp;lt;&amp;lt; &quot;Deriv Activation&quot; &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; derivActivation &amp;lt;&amp;lt; std::endl;&#xA;      double bias = network.layers[i].nodes[j].bias;&#xA;      double derivBias = network.layers[i].nodes[j].derivBias;&#xA;      //Outputs the bias and derivative of the bias.&#xA;      networkPrintFile &amp;lt;&amp;lt; &quot;\t\t&quot; &amp;lt;&amp;lt; &quot;Bias&quot; &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; bias &amp;lt;&amp;lt; std::endl;&#xA;      networkPrintFile &amp;lt;&amp;lt; &quot;\t\t&quot; &amp;lt;&amp;lt; &quot;Deriv Bias&quot; &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; derivBias &amp;lt;&amp;lt; std::endl;&#xA;      //Iterates through all of the inputWeights in the current node.&#xA;      networkPrintFile &amp;lt;&amp;lt; &quot;\t\t&quot; &amp;lt;&amp;lt; &quot;Weights&quot; &amp;lt;&amp;lt; &quot;:&quot; &amp;lt;&amp;lt; std::endl;&#xA;      for (int k = 0; k &amp;lt; network.layers[i].nodes[j].weights.size(); k++)&#xA;      {&#xA;        double inputWeight = network.layers[i].nodes[j].weights[k];&#xA;        double derivWeight =  network.layers[i].nodes[j].derivWeights[k];&#xA;        networkPrintFile &amp;lt;&amp;lt; &quot;\t\t\t&quot; &amp;lt;&amp;lt; &quot;Weight &quot; &amp;lt;&amp;lt; k &amp;lt;&amp;lt; &quot;:&quot; &amp;lt;&amp;lt; std::endl;&#xA;        networkPrintFile &amp;lt;&amp;lt; &quot;\t\t\t\t&quot; &amp;lt;&amp;lt; &quot;Value&quot; &amp;lt;&amp;lt; &quot;:&quot; &amp;lt;&amp;lt; inputWeight &amp;lt;&amp;lt; std::endl;&#xA;        networkPrintFile &amp;lt;&amp;lt; &quot;\t\t\t\t&quot; &amp;lt;&amp;lt; &quot;Derivative&quot; &amp;lt;&amp;lt; &quot;:&quot; &amp;lt;&amp;lt; derivWeight &amp;lt;&amp;lt; std::endl;&#xA;      }&#xA;    }&#xA;  }&#xA;  std::cout &amp;lt;&amp;lt; &quot;Done&quot; &amp;lt;&amp;lt; std::endl;&#xA;}&#xA;&#xA;//Takes and input and peforms a mathematical sigmoid&#xA;//function on it and returns the value.&#xA;//             1&#xA;//  σ(x) = ---------&#xA;//          1 + e^x&#xA;double Sigmoid(double input)&#xA;{&#xA;  double expInput = std::exp(-input);&#xA;  double denom = expInput + 1;&#xA;  double value = 1 / denom;&#xA;  return value;&#xA;}&#xA;&#xA;//Returns the activation of the node passed in give the previous layer.&#xA;double CalculateNode(struct Node &amp;amp;node, struct Layer &amp;amp;prevLayer)&#xA;{&#xA;  //Keeps a runing total of the weights and activations added up so far.&#xA;  double total = 0.0;&#xA;  int weightCount = node.weights.size();&#xA;  //Iterated through each of the weights, and thus each of the&#xA;  //nodes in the previous layer to find the weight * activation.&#xA;  for (int i = 0; i &amp;lt; weightCount; i++)&#xA;  {&#xA;    //Calculated the current weight and activation and&#xA;    //adds it to the 'total' variable.&#xA;    double weight = node.weights[i];&#xA;    double input = prevLayer.nodes[i].activation;&#xA;    double value = weight * input;&#xA;    total += value;&#xA;  }&#xA;  //Add the node's bias to the total.&#xA;  total += node.bias;&#xA;  //Normalises the node's activation value by passing it through&#xA;  //a sigmoid function, which bounds it between 0 and 1.&#xA;  double normTotal = Sigmoid(total);&#xA;  //Returns the caclulated value for this node.&#xA;  return normTotal;&#xA;}&#xA;&#xA;//Adds the activation values to a layer passed in, given the previous layer.&#xA;void CaclulateLayer(struct Layer &amp;amp;layer, struct Layer &amp;amp;prevLayer)&#xA;{&#xA;  //Iterates through all of the nodes and calculated their activations.&#xA;  for (struct Node &amp;amp;node : layer.nodes)&#xA;  {&#xA;    double activation = CalculateNode(node, prevLayer);&#xA;    //Setting the activation to the node.&#xA;    node.activation = activation;&#xA;  }&#xA;}&#xA;&#xA;//Takes in the first layer of the neural network and interates through the&#xA;//nodes and sets each input to each node in a loop.&#xA;void SetInputs(struct Layer &amp;amp;layer, std::vector&amp;lt;double&amp;gt; inputs)&#xA;{&#xA;  for (int i = 0; i &amp;lt; layer.nodes.size(); i++)&#xA;  {&#xA;    //Setting the node's activation to the corrosponding input.&#xA;    layer.nodes[i].activation = inputs[i];&#xA;  }&#xA;}&#xA;&#xA;//Takes in a network and inputs and calculates the value of&#xA;//activation for every node for a single input vector.&#xA;void CalculateNetwork(struct Network &amp;amp;network, std::vector&amp;lt;double&amp;gt; inputs)&#xA;{&#xA;  //Setting the activations of the first layer to the inputs vector.&#xA;  SetInputs(network.layers[0], inputs);&#xA;  //Iterates through all of the layers, apart from the first layer, and&#xA;  //calculated the activations of the nodes in that layer.&#xA;  for (int i = 1; i &amp;lt; network.layers.size(); i++)&#xA;  {&#xA;    //Getting the layer to calculate to activations on and the&#xA;    //previous layer, which already has it's activations calculated.&#xA;    struct Layer currentLayer = network.layers[i];&#xA;    struct Layer prevLayer = network.layers[i - 1];&#xA;    //Calculating the nodes on the current layer.&#xA;    CaclulateLayer(currentLayer, prevLayer);&#xA;    //Setting the currentLayer back into the network struct with&#xA;    //all of the activations in it now calculated.&#xA;    network.layers[i] = currentLayer;&#xA;  }&#xA;}&#xA;&#xA;//Caclulates the sum of the differences between the outputs and the correct&#xA;//values squared.&#xA;//&#xA;//  Cost = Σ((a-y)^2)&#xA;//&#xA;double CalculateCost(struct Network &amp;amp;network, std::vector&amp;lt;double&amp;gt; correctOutputs)&#xA;{&#xA;  //Keeps track of the current sum of the costs.&#xA;  double totalCost = 0.0;&#xA;  //The layer of the network that holds the calculated values, the&#xA;  //last layer in the network.&#xA;  struct Layer outputLayer = network.layers[network.layers.size() - 1];&#xA;  //Loops through all the node sin the output layer and compared them&#xA;  //to their corresponding correctOutput value, calculates the cost&#xA;  //and adds it to the running total, totalCoat.&#xA;  for (int i = 0; i &amp;lt; outputLayer.nodes.size(); i++)&#xA;  {&#xA;    struct Node node = outputLayer.nodes[i];&#xA;    double calculatedActivation = node.activation;&#xA;    double correctActivation = correctOutputs[i];&#xA;    double diff = calculatedActivation - correctActivation;&#xA;    double modDiff = diff * diff;&#xA;    //Adding the cost to the sum of the other costs.&#xA;    totalCost += modDiff;&#xA;  }&#xA;  //Returning the value of the calculated cost.&#xA;  return totalCost;&#xA;}&#xA;&#xA;//Takes in the output layer of the network and calculates the derivatives of the&#xA;//cost function with respect to the activations in each node. this value is then&#xA;//stored on the Node struct.&#xA;void LastLayerDerivActivations(struct Layer &amp;amp;layer, std::vector&amp;lt;double&amp;gt; correctOutputs)&#xA;{&#xA;  //Iterating through all the nodes in the layer.&#xA;  for (int i = 0; i &amp;lt; layer.nodes.size(); i++)&#xA;  {&#xA;    //Getting the values of the node output and correct output.&#xA;    double activation = layer.nodes[i].activation;&#xA;    double correctOutput = correctOutputs[i];&#xA;    //Caclulating the partial derivative of the cost function with respect&#xA;    //to the current node's activation value.&#xA;    double activationDiff = activation - correctOutput;&#xA;    double derivActivation = 2 * activationDiff;&#xA;    //Setting the activation partial derivative to the layer passed in.&#xA;    layer.nodes[i].derivActivation = derivActivation;&#xA;  }&#xA;}&#xA;&#xA;//Returns the derivative of the sigmoid function.&#xA;//   d&#xA;//  ---- σ(x) = σ(x)(1 - σ(x))&#xA;//   dx&#xA;double DerivSigma(double input)&#xA;{&#xA;  double sigma = Sigmoid(input);&#xA;  double value = sigma * (1 - sigma);&#xA;  return sigma;&#xA;}&#xA;&#xA;//Takes in a node and the layer that the node takes inputs from and adds.&#xA;//to the derivWeight and derivBias of the node and adds to each of the&#xA;//deriv activations in the previous layer for them to be used in this&#xA;//function to calculate their derivatives.&#xA;void NodeDeriv(struct Node &amp;amp;node, struct Layer &amp;amp;prevLayer)&#xA;{&#xA;  //Starting the total at the bias.&#xA;  double total = node.bias;&#xA;  //Looping through all the weights and biases to find z(x).&#xA;  //  z(x) = a w + a w + ... + a w + b&#xA;  //          1 1   2 2         n n&#xA;  for (int i = 0; i &amp;lt; node.weights.size(); i++)&#xA;  {&#xA;    double weight = node.weights[i];&#xA;    double activation = prevLayer.nodes[i].activation;&#xA;    double value = weight * activation;&#xA;    //Adding to the running total for z(x).&#xA;    total += value;&#xA;  }&#xA;  //Finding the derivative of the cost function with respect to the&#xA;  //z(x) by multiplying the DerivSigma() by the node's derivActivation&#xA;  //using the chain rule.&#xA;  double derivAZ = DerivSigma(total);&#xA;  double derivCZ = derivAZ * node.derivActivation;&#xA;  //The derivative of the cost with respect to the bias is the same as&#xA;  //the derivative of the cost function with respect to z(x) since&#xA;  //d/db z(x) = 1&#xA;  node.derivBias += derivCZ;&#xA;  //Iterating through all of the nodes and weights to find the derivatives&#xA;  //of all of the weights for the node and the activations on the&#xA;  //previous layer.&#xA;  for (int i = 0; i &amp;lt; node.weights.size(); i++)&#xA;  {&#xA;    // dc/dw = dc/dz * activation&#xA;    double derivCW = derivCZ * prevLayer.nodes[i].activation;&#xA;    // dc/da = dc/dz * weight&#xA;    double derivCA = derivCZ * node.weights[i];&#xA;    //Adding the weights and activations to the node objects.&#xA;    node.derivWeights[i] += derivCW;&#xA;    prevLayer.nodes[i].derivActivation += derivCA;&#xA;  }&#xA;  //Resetting the activation derivative.&#xA;  node.derivActivation = 0;&#xA;}&#xA;&#xA;//Takes in a layer and iterates through all the nodes in order to find&#xA;//the derivatives of thw weight and biases in the current layer and the&#xA;//activations in the previous layer.&#xA;void LayerDeriv(struct Layer &amp;amp;layer, struct Layer &amp;amp;prevLayer)&#xA;{&#xA;  for (int i = 0; i &amp;lt; layer.nodes.size(); i++)&#xA;  {&#xA;    NodeDeriv(layer.nodes[i], prevLayer);&#xA;  }&#xA;}&#xA;&#xA;//Takes in a network and uses backpropogation to find the derivatives of&#xA;//all the nodes for a single training example.&#xA;void NetworkDeriv(struct Network &amp;amp;network, std::vector&amp;lt;double&amp;gt; expectedOutputs)&#xA;{&#xA;  //Calculating the derivatives of the activations in the last layer.&#xA;  LastLayerDerivActivations(network.layers[network.layers.size() - 1], expectedOutputs);&#xA;  //Looping through all the layers to find the derivatives of all of&#xA;  //the weights and activations in the network for this training example.&#xA;  for (int i = network.layers.size() - 1; i &amp;gt; 0; i--)&#xA;  {&#xA;    LayerDeriv(network.layers[i], network.layers[i - 1]);&#xA;  }&#xA;}&#xA;&#xA;//Takes in an input string and char and will return a vector of the string split&#xA;//by the char. The char is lost in this conversion.&#xA;std::vector&amp;lt;std::string&amp;gt; SplitString(std::string stringToSplit, char delimiter)&#xA;{&#xA;  //Creating the output vector.&#xA;  std::vector&amp;lt;std::string&amp;gt; outputVector;&#xA;  //Initialising the lastDelimiter to -1, since the first string should be split as if&#xA;  //the char before it was the splitter.&#xA;  int lastDelimiterIndex = -1;&#xA;  for (int i = 0; i &amp;lt; stringToSplit.size(); i++)&#xA;  {&#xA;    //Getting the current char.&#xA;    char chr = stringToSplit[i];&#xA;    //If the current char is the delimiter, create a new substring in the vector.&#xA;    if (chr == delimiter)&#xA;    {&#xA;      //Creating the new substring at the delimiter and adding it to the end&#xA;      //of the output vector.&#xA;      std::string subString = stringToSplit.substr(lastDelimiterIndex + 1, i - lastDelimiterIndex - 1);&#xA;      outputVector.push_back(subString);&#xA;      //Setting the last delimiter variable to the current character.&#xA;      lastDelimiterIndex = i;&#xA;    }&#xA;  }&#xA;  //Adding the last section of the string to the output vector, since there is no&#xA;  //delimiter and will not be added in the for loop.&#xA;  std::string subString = stringToSplit.substr(lastDelimiterIndex + 1, stringToSplit.size() - lastDelimiterIndex - 1);&#xA;  outputVector.push_back(subString);&#xA;  //Returning the split string as a vector of strings.&#xA;  return outputVector;&#xA;}&#xA;&#xA;//Takes in a vector of strings and converts it to a vector of doubles. normalise argument&#xA;//sets what value will be taken to be 1, and other numbers will be a fraction of that.&#xA;//Set normalise to 0 to disable normalisation.&#xA;std::vector&amp;lt;double&amp;gt; ConvertStringVectorToDoubleVector(std::vector&amp;lt;std::string&amp;gt; input, int normalise = 0)&#xA;{&#xA;  std::vector&amp;lt;double&amp;gt; convertedVector;&#xA;  //Iterating through all the strings int the input vector.&#xA;  for (std::string str : input)&#xA;  {&#xA;    //Converting the string into a double.&#xA;    double value = stod(str);&#xA;    //Checks to see if normalisation is enabled.&#xA;    if (normalise != 0)&#xA;    {&#xA;      //Normalising the double.&#xA;      value /= normalise;&#xA;    }&#xA;    //Adding the double to the output vector.&#xA;    convertedVector.push_back(value);&#xA;  }&#xA;  //Returning the converted vector.&#xA;  return convertedVector;&#xA;}&#xA;&#xA;//Takes in a string of data and uses it to create a DataSet object to&#xA;//be used in the training of the neural network.&#xA;struct DataSet FormatData(std::string dataString)&#xA;{&#xA;  struct DataSet dataSet;&#xA;  //Splitting the input string into the seperate images.&#xA;  std::vector&amp;lt;std::string&amp;gt; imageSplit = SplitString(dataString, '|');&#xA;  //Looping through all of the images.&#xA;  for (int i = 0; i &amp;lt; imageSplit.size(); i++)&#xA;  {&#xA;    //Getting the current image string.&#xA;    std::string imageData = imageSplit[i];&#xA;    //Splitting the image between the inputs and expected outputs.&#xA;    std::vector&amp;lt;std::string&amp;gt; ioSplit = SplitString(imageData, '/');&#xA;    std::string inputs = ioSplit[0];&#xA;    std::string outputs = ioSplit[1];&#xA;    //converting the input and output strings into string arrays of the values.&#xA;    std::vector&amp;lt;std::string&amp;gt; inputVectorString = SplitString(inputs, ',');&#xA;    std::vector&amp;lt;std::string&amp;gt; outputVectorString = SplitString(outputs, ',');&#xA;    //Converting the string arrays into double arrays and normalising the input doubles.&#xA;    std::vector&amp;lt;double&amp;gt; inputVector = ConvertStringVectorToDoubleVector(inputVectorString, 255);&#xA;    std::vector&amp;lt;double&amp;gt; outputVector = ConvertStringVectorToDoubleVector(outputVectorString);&#xA;    //Creating a new Data object.&#xA;    struct Data data;&#xA;    data.inputs = inputVector;&#xA;    data.answers = outputVector;&#xA;    //Adding the object to the dataset.&#xA;    dataSet.data.push_back(data);&#xA;  }&#xA;  //Returning the completed dataset.&#xA;  return dataSet;&#xA;}&#xA;&#xA;//Takes in a filename and extracts all of the ascii data from the&#xA;//file and calls the FormatData function to create a DataSet object.&#xA;struct DataSet CreateDataSetFromFile(std::string fileName)&#xA;{&#xA;  //Opening file.&#xA;  std::ifstream dataFile;&#xA;  dataFile.open(fileName);&#xA;  //Storing fild data in a string.&#xA;  std::string data;&#xA;  dataFile &amp;gt;&amp;gt; data;&#xA;  //Creating DataSet object.&#xA;  struct DataSet dataSet = FormatData(data);&#xA;  //Returning completed DataSet object.&#xA;  return dataSet;&#xA;}&#xA;&#xA;//Takes in a network and a Data object and runs the network and adds&#xA;//to the derivatives of the network for that one training example.&#xA;void NetworkIteration(struct Network &amp;amp;network, struct Data data)&#xA;{&#xA;  //Extracting the input and output data from the data object.&#xA;  std::vector&amp;lt;double&amp;gt; inputs = data.inputs;&#xA;  std::vector&amp;lt;double&amp;gt; outputs = data.answers;&#xA;  //Caclulating the activations of the network for this data.&#xA;  CalculateNetwork(network, inputs);&#xA;  //Caclulating the cost for this iteration and adding it to the total.&#xA;  double cost = CalculateCost(network, outputs);&#xA;  network.cost += cost;&#xA;  //Caclulating the derivatives for the network weights and biases&#xA;  //for this training example.&#xA;  NetworkDeriv(network, outputs);&#xA;}&#xA;&#xA;//Takes in a node and caclulates the average of the derivatives over&#xA;//the dataset and then multiplies them by a fixes mutation rate and&#xA;//applies the derivatives to the node's values.&#xA;void GradientDecentNode(struct Node &amp;amp;node, int dataCount)&#xA;{&#xA;  //Iterating through all of the weights of the node.&#xA;  for (int i = 0; i &amp;lt; node.weights.size(); i++)&#xA;  {&#xA;    double weight = node.weights[i];&#xA;    double derivWeight = node.derivWeights[i];&#xA;    //Getting the average over all of the training data.&#xA;    derivWeight /= dataCount;&#xA;    //Applying a constant multiplier to alter the rate at which is mutates.&#xA;    derivWeight *= weightMutationRate;&#xA;    //Subtracting the derivative from the weight.&#xA;    node.weights[i] -= derivWeight;&#xA;    //Reseting the weight derivative&#xA;    node.derivWeights[i] = 0;&#xA;  }&#xA;  double bias = node.bias;&#xA;  double derivBias = node.derivBias;&#xA;  //Applying a constant multiplier to alter the rate at which is mutates.&#xA;  derivBias *= biasMutationRate;&#xA;  //Subtracting the derivative from the bias.&#xA;  node.bias -= derivBias;&#xA;  //Resetting the bias derivative.&#xA;  node.derivBias = 0;&#xA;}&#xA;&#xA;//Takes in a layer and iterated through all of the nodes in the layer&#xA;//and applies all of their derivatives to them.&#xA;void GradientDecentLayer(struct Layer &amp;amp;layer, int dataCount)&#xA;{&#xA;  for (struct Node &amp;amp;node : layer.nodes)&#xA;  {&#xA;    GradientDecentNode(node, dataCount);&#xA;  }&#xA;}&#xA;&#xA;//Takes in a network and iterated through all of the layers and applies&#xA;//all of the derivatives to them.&#xA;void GradientDecentNetwork(struct Network &amp;amp;network, int dataCount)&#xA;{&#xA;  for (int i = 1; i &amp;lt; network.layers.size(); i++)&#xA;  {&#xA;    GradientDecentLayer(network.layers[i], dataCount);&#xA;  }&#xA;}&#xA;&#xA;//Iterates through all of the training data in dataSet and calculates the derivatives&#xA;//of the weights and biases and then peforms the gradient decent using the derivatives.&#xA;void TrainNetworkSingle(struct Network &amp;amp;network, struct DataSet dataSet)&#xA;{&#xA;  //Iterating through all of the training data.&#xA;  for (struct Data data : dataSet.data)&#xA;  {&#xA;    //Caclulating the network for a single training example.&#xA;    NetworkIteration(network, data);&#xA;  }&#xA;  //Peforming the derivatives.&#xA;  GradientDecentNetwork(network, dataSet.data.size());&#xA;}&#xA;&#xA;void TrainNetwork(struct Network &amp;amp;network, struct DataSet dataSet, int iterations)&#xA;{&#xA;  for (int i = 0; true/*i &amp;lt; iterations*/; i++)&#xA;  {&#xA;    TrainNetworkSingle(network, dataSet);&#xA;    std::cout &amp;lt;&amp;lt; network.cost &amp;lt;&amp;lt; std::endl;&#xA;    network.cost = 0;&#xA;  }&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;  struct Network network = CreateNetwork({784, 784, 16, 16, 10});&#xA;  struct DataSet dataSet = CreateDataSetFromFile(&quot;data.txt&quot;);&#xA;  TrainNetwork(network, dataSet, 100);&#xA;  PrintNetwork(network);&#xA;  return 0;&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33354" LastActivityDate="2020-02-07T08:30:54.977" Title="Neural network seems to just figure out the probability of a specific result" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;probability&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17894" PostTypeId="2" ParentId="17809" CreationDate="2020-02-07T09:08:33.570" Score="0" Body="&lt;p&gt;from what I understand you are building you own model for this specific use case. From my perspective I would try &lt;em&gt;not to reinvent the wheel&lt;/em&gt;, as it is said, and use an already proven and working model such as the YOLOs (&lt;a href=&quot;https://arxiv.org/abs/1506.02640&quot; rel=&quot;nofollow noreferrer&quot;&gt;v1&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1612.08242&quot; rel=&quot;nofollow noreferrer&quot;&gt;v2&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1804.02767&quot; rel=&quot;nofollow noreferrer&quot;&gt;v3&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;YOLO does not tell you the center pixel of the image directly but it tells you the center cell, with respect to the predicted object, of a grid (which is built on top of the image) responsible for predicting each object. See on the left image how the grid is built on top of the input image, then YOLO computes a probability map, and then the bounding boxes are predicted from the &lt;strong&gt;cell in the object's center&lt;/strong&gt;. I have highlighted which cells would be responsible for predicting each object in the rightmost image. (Image from YOLOv1 paper)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9DbkU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9DbkU.png&quot; alt=&quot;YOLO cell approach&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The grid can have different resolutions but if you make it equal to the image size, then you would have the center pixel of each object predicted. This is because YOLO predicts objects in each cell of the grid, &lt;strong&gt;&lt;em&gt;so if the grid is equal to the image size, YOLO will predict objects in each pixel&lt;/em&gt;&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, imagine you have an input image of &lt;span class=&quot;math-container&quot;&gt;$[H \times W] = [416 \times 416]$&lt;/span&gt; then YOLO would compute a grid of &lt;span class=&quot;math-container&quot;&gt;$[S_1 \times S_2]=[52 \times 52]$&lt;/span&gt; on top of it. And predict objects in the center cell of the &lt;span class=&quot;math-container&quot;&gt;$[S_1 \times S_2]$&lt;/span&gt;. So, if you tune YOLO for computing a grid such as &lt;span class=&quot;math-container&quot;&gt;$[S_1 \times S_2] = [H \times W]$&lt;/span&gt;, then YOLO would output objects prediction with respect to the image pixels, in other words, YOLO would predict bounding boxes centered in the image pixel on the object's center.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I would proceed for this use case, I hope it helps you or at least give you some clues about how to proceed further! Cheers! :)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;NOTE: I chose the image size and grid size with numbers I usually see at work. Specifically, using YOLOv3. In YOLOv3, for aninput image of &lt;span class=&quot;math-container&quot;&gt;$[H \times W] = [416 \times 416]$&lt;/span&gt;, 3 grids are built, with different resolutions (for predicting big and small objects), with the following grids sizes are: &lt;span class=&quot;math-container&quot;&gt;$[13 \times 13], [26 \times 26], [52 \times 52]$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-02-07T09:08:33.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17895" PostTypeId="2" ParentId="16729" CreationDate="2020-02-07T09:47:17.410" Score="-1" Body="&lt;p&gt;From what I read in the papers and my experience, the trend lately is to generate synthetic data using a photorealistic 3D graphics engine for automating the image labelling. The process would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Get a visually realistic 3D engine such as Unreal, Unity... any videogame engine would work&lt;/li&gt;&#xA;&lt;li&gt;Build 3D models of the objects you want to detect (or ask for help to a 3D artist)&lt;/li&gt;&#xA;&lt;li&gt;Build a 3D environment for placing the objects 3D models&lt;/li&gt;&#xA;&lt;li&gt;Generate random instances of the objects in the environments with different environmental conditions: lighting, brightness, position, rotation...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Since you have the 3D model of the object and the whole control in the 3D engine you can annotate images, bounding boxes, pixels... or whatever stuff you want in an automated way.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is an approach for automatic image labeling with synthetic data. Automating the labeling of real data is a challenge yet to be tackled.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastEditorUserId="2444" LastEditDate="2020-12-04T10:15:08.293" LastActivityDate="2020-12-04T10:15:08.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17896" PostTypeId="2" ParentId="17890" CreationDate="2020-02-07T10:21:12.137" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In reinforcement learning, is the value of terminal/goal state always zero?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, always for episodic problems, the value of a terminal state is always zero, from the definition.&lt;/p&gt;&#xA;&lt;p&gt;The value of a state &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; is the expected sum (perhaps discounted) of rewards from all &lt;em&gt;future&lt;/em&gt; time steps. There are no future time steps when in a terminal state, so this sum must be zero.&lt;/p&gt;&#xA;&lt;p&gt;For the sake of consistent maths notation, you can consider a terminal state to be &amp;quot;absorbing&amp;quot;, i.e. any transition out of it results in zero reward and returning to the same terminal state. Then you can use the definition of value function to show the same thing:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty} \gamma^k R_{t+k+1} | S_{t} = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$s = s_T$&lt;/span&gt;, the terminal state, then all the &amp;quot;future rewards&amp;quot; from &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt; onwards starting with reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; must be zero. This is consistent with the reward &lt;span class=&quot;math-container&quot;&gt;$R_{t}$&lt;/span&gt;, i.e. the reward when transitioning &lt;em&gt;to&lt;/em&gt; the terminal state, being any value.&lt;/p&gt;&#xA;&lt;p&gt;You can show similar using action value functions, if you accept a &amp;quot;null&amp;quot; action in the terminal state.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-11-01T15:11:22.057" LastActivityDate="2020-11-01T15:11:22.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17897" PostTypeId="1" CreationDate="2020-02-07T10:52:41.470" Score="2" ViewCount="41" Body="&lt;p&gt;I'm a beginner in computer vision. I want to know which option among the following two can get better accuracy of image classification.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;SIFT features + SVM&lt;/li&gt;&#xA;&lt;li&gt;Bag-of-visual-words features + SVM&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here's a reference: &lt;a href=&quot;https://www.mathworks.com/help/vision/ug/image-classification-with-bag-of-visual-words.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/help/vision/ug/image-classification-with-bag-of-visual-words.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33358" LastEditorUserId="2444" LastEditDate="2020-09-25T22:22:14.160" LastActivityDate="2020-09-26T00:22:31.000" Title="Does the bag-of-visual-words method improve the classification accuracy?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;support-vector-machine&gt;&lt;image-recognition&gt;&lt;bag-of-features&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17898" PostTypeId="1" AcceptedAnswerId="17994" CreationDate="2020-02-07T11:39:44.560" Score="1" ViewCount="61" Body="&lt;p&gt;Why do we split the data into two parts, and then split those segments into training and testing data? Why do we have two sets of data for each training and test data?&lt;/p&gt;&#xA;" OwnerUserId="33356" LastEditorUserId="33010" LastEditDate="2020-02-07T20:54:48.167" LastActivityDate="2020-02-12T17:12:06.907" Title="While we split data in training and test data, why we have two pairs of each?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;cross-validation&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17900" PostTypeId="2" ParentId="17892" CreationDate="2020-02-07T11:45:58.920" Score="5" Body="&lt;p&gt;In variational inference, the original objective is to minimize the Kullback-Leibler divergence between the variational distribution, &lt;span class=&quot;math-container&quot;&gt;$q(z \mid x)$&lt;/span&gt;, and the posterior, &lt;span class=&quot;math-container&quot;&gt;$p(z \mid x) = \frac{p(x, z)}{\int_z p(x, z)}$&lt;/span&gt;, given that the posterior can be difficult to directly infer with the Bayes rule, due to the denominator term, which can contain an intractable integral. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, more formally, the optimization objective can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q^*(z \mid x) = \operatorname{argmin}_{q(z \mid x)} D_{\text{KL}}(q(z \mid x) \| p(z \mid x))\tag{1} \label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, solving this optimization problem can be as difficult as the original inference one of computing the posterior &lt;span class=&quot;math-container&quot;&gt;$p(z \mid x)$&lt;/span&gt; using the Bayes rule, given that it still involves the possibly intractable term &lt;span class=&quot;math-container&quot;&gt;$p(z \mid x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use the definition of the KL divergence, you can derive the following equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;D_{\text{KL}}(q(z \mid x) \| p(z \mid x)) = \mathbb{E}_{q(z \mid x)} \left[ \log q(z \mid x) \right] - \mathbb{E}_{q(z \mid x)} \left[ \log q(z, x) \right] + \log p(x) \tag{2} \label{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, note that the expectations are with respect to the variational distribution, which means that, if you want to approximate these expectations with Monte Carlo estimates, you can do it with respect to the variational distribution, and, given that it is assumed that one can easily sample from the variational distribution (which can e.g. be a Gaussian), this is a nice feature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the KL divergence contains the term &lt;span class=&quot;math-container&quot;&gt;$p(x) = \int_z p(x, z)$&lt;/span&gt;, the denominator term in the Bayes rule to compute the posterior &lt;span class=&quot;math-container&quot;&gt;$p(z \mid x)$&lt;/span&gt;, which (as I said) can be intractable. &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; is often called the &lt;strong&gt;evidence&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is then to optimize an objective that does not contain this annoying intractable term &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;. The objective that is optimized is the so-called ELBO objective&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text{ELBO}(q) = \mathbb{E}_{q(z \mid x)} \left[ \log q(z, x) \right] - \mathbb{E}_{q(z \mid x)} \left[ \log q(z \mid x) \right]\tag{3} \label{3}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The KL divergence \ref{2} and the ELBO objective \ref{3} are similar. In fact, &lt;strong&gt;ELBO&lt;/strong&gt; is an abbreviation for &lt;strong&gt;E&lt;/strong&gt;vidence &lt;strong&gt;L&lt;/strong&gt;ower &lt;strong&gt;BO&lt;/strong&gt;und, because the ELBO is a lower bound on the evidence &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;, i.e. it is a number that is smaller than &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; or, more formally, &lt;span class=&quot;math-container&quot;&gt;$\text{ELBO}(q) \leq \log p(x)$&lt;/span&gt;. Therefore, if we maximize &lt;span class=&quot;math-container&quot;&gt;$\text{ELBO}(q)$&lt;/span&gt;, we also maximize the evidence &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; of the data (where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the data in your dataset). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the objective in variational inference is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q^*(z \mid x) &#xA;&amp;amp;= \operatorname{argmax}_{q(z \mid x)} \operatorname{ELBO}({q}) \\&#xA;&amp;amp;= \operatorname{argmax}_{q(z \mid x)}  \mathbb{E}_{q(z \mid x)} \left[ \log q(z, x) \right] - \mathbb{E}_{q(z \mid x)} \left[ \log q(z \mid x) \right]&#xA;\tag{4} \label{4}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, note that \ref{4} only contains terms that depend on the variational distribution, so we got rid of intractable terms, which was our goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, note that, as opposed to \ref{1}, we are maximizing (or finding the parameters that maximize the objective).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ELBO objective is actually the &lt;strong&gt;negative&lt;/strong&gt; of \ref{2} plus the logarithm of the evidence term, &lt;span class=&quot;math-container&quot;&gt;$\log p(x)$&lt;/span&gt; (and you can easily verify it), that is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text{ELBO}(q)  = -D_{\text{KL}}(q(z \mid x) \| p(z \mid x)) + \log p(x)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can also be rearranged as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\log p(x) = D_{\text{KL}}(q(z \mid x) \| p(z \mid x)) + \text{ELBO}(q) \tag{5}\label{5}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is your equation (where &lt;span class=&quot;math-container&quot;&gt;$\text{ELBO}(q)$&lt;/span&gt; is your &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;). Therefore, your equation is true &lt;strong&gt;by definition&lt;/strong&gt;, i.e. we define the ELBO such that \ref{5} is true. However, note that we haven't defined the ELBO in the way we dit only for the sake of it, but because it is a lower bound on the log evidence (and this follows from the fact that the KL divergence is never negative).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-09T14:11:28.613" LastActivityDate="2020-02-09T14:11:28.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17901" PostTypeId="2" ParentId="17898" CreationDate="2020-02-07T11:58:34.763" Score="0" Body="&lt;p&gt;Usually we are splinting the data into 3 chunks for example 70% for training, 10% for validation and 20% for testing. The first two chunks are going to be used for training. The reason you need the validation dataset is to tune your hyper-parameters and see how well your model can generalize. Once you have a model that achieves a fairly good performance on the validation dataset, you're measuring its accuracy on the test dataset. &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2020-02-07T11:58:34.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17902" PostTypeId="2" ParentId="17898" CreationDate="2020-02-07T12:02:57.763" Score="1" Body="&lt;p&gt;For any Machine Learning model, the available data is usually split into three sets:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Training Set:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The part of data used to train the model and learn the parameters of the network.&lt;/p&gt;&#xA;&lt;p&gt;The data that remains after allocation of the Training Dataset, is split into the Validation and Test sets.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Validation Set:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This sample of data is used to provide an unbiased evaluation of a model fit on the training dataset. This helps in tuning model hyperparameters to improve the model performance. Eg: Changing the number of clusters (&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;) in a K-Means algorithm, or the pooling layers in a CNN.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Test Set:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;After training, this part of the dataset is used to used to test how well the model generalizes to unseen data and estimate its performance.&lt;/p&gt;&#xA;&lt;p&gt;Another possibility (going by your question), is the use of &lt;strong&gt;Cross-Validation&lt;/strong&gt;. This is performed when the dataset is too small.  In such a case, a random split is performed on the dataset resulting in k non-overlapping subsets. The test error is then estimated by taking the average test error across k trials.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IqYBM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IqYBM.png&quot; alt=&quot;kFold&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-validation_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;[Image Source]&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-07T12:36:29.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17903" PostTypeId="2" ParentId="15524" CreationDate="2020-02-07T14:38:29.700" Score="2" Body="&lt;p&gt;1) The math is the exact same, so from an optimization or mathematical perspective there is no difference  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Here are my &lt;strong&gt;guesses&lt;/strong&gt; to a possible answer.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Habit: People may just call one over the other out of habit&lt;/li&gt;&#xA;&lt;li&gt;Generality: Across frameworks a 1d convolution op would work, while Dense of FC may need adjustments to work on the temporal axis &lt;/li&gt;&#xA;&lt;li&gt;Parallel Workers: Convolution and Dense call different subroutines in the backend, and the one used by convolution may have better gains on sequential input for this purpose &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;br&gt;&#xA;Regarding bench-marking the 2, your experiment was shallow. I didn't have time to wait to do a full gird search, so i held 3 paramaters constant and fluctuated one. Here are the results (note the model was just a simple feed forward relu residual model) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Khm6S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Khm6S.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xl5b0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xl5b0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/O7XXV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O7XXV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VK07u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VK07u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in a couple yeah dense out performs conv but it isn't consistent and there are scenarios where it is not true. This is only for a small grid that I chose but you can extend this yourself to check. So it is not as straightforward to say one is sheerly better than the other. &lt;/p&gt;&#xA;" OwnerUserId="25496" LastEditorUserId="25496" LastEditDate="2020-02-12T21:30:22.093" LastActivityDate="2020-02-12T21:30:22.093" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17904" PostTypeId="1" AcceptedAnswerId="17905" CreationDate="2020-02-07T14:59:32.530" Score="2" ViewCount="517" Body="&lt;p&gt;On page 84 of Russell &amp;amp; Norvig's book &quot;Artificial Intelligence: A Modern Approach Book&quot; (3rd edition), the pseudocode for uniform cost search is given. I provided a screenshot of it here for your convenience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having trouble understanding the highlighted line &lt;code&gt;if child.STATE is not in explored **or** frontier then&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PHchM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PHchM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shouldn't that be &#xA;&lt;code&gt;if child.STATE is not in explored **and** frontier then&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way it's written, it seems to suggest that if the child node has already been explored, but not currently in the frontier, then we add it to the frontier, but I don't think that's correct. If it's already been explored, then it means we already found the optimal path for this node previously and should not be processing it again. Is my understanding wrong?&lt;/p&gt;&#xA;" OwnerUserId="20358" LastEditorUserId="1671" LastEditDate="2020-03-04T23:56:48.447" LastActivityDate="2020-03-04T23:56:48.447" Title="Understanding the pseudocode of uniform-cost search from the book &quot;Artificial Intelligence: A Modern Approach&quot;" Tags="&lt;search&gt;&lt;norvig-russell&gt;&lt;uniform-cost-search&gt;&lt;pseudocode&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17905" PostTypeId="2" ParentId="17904" CreationDate="2020-02-07T15:15:39.290" Score="2" Body="&lt;p&gt;I think this is a problem with missing brackets in pseudocode &amp;mdash; clearly the state is only added to the frontier if it hasn't been explored already, so it would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if not [contains(frontier, state) OR contains(explored, state)] then&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is equivalent to your interpretation of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if not [contains(frontier, state)] AND not [contains(explored, state)] then&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(according to &lt;a href=&quot;https://en.wikipedia.org/wiki/De_Morgan&amp;#39;s_laws&quot; rel=&quot;nofollow noreferrer&quot;&gt;De Morgan's Laws&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being half-way between a programming language and natural language, this is a case where pseudocode is not quite precise enough.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-07T15:15:39.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17906" PostTypeId="1" CreationDate="2020-02-07T17:01:50.077" Score="1" ViewCount="21" Body="&lt;p&gt;My goal is to generate artificial sequences of real-valued data (e.g. time series) with GANs. Starting simple I tried to generate realistic sine-waves using a Wasserstein GAN. But even on this simple task it fails to generate any useful samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Generator&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(LSTM(20, input_shape=(50, 1)))&#xA;model.add(Dense(40, activation='linear'))&#xA;model.add(Reshape((40, 1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Critic&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Conv1D(64, kernel_size=5, input_shape=(40, 1), strides=1))&#xA;model.add(MaxPooling1D(3, strides=2))&#xA;model.add(LeakyReLU(alpha=0.2))&#xA;model.add(Conv1D(64, kernel_size=5, strides=1))&#xA;model.add(MaxPooling1D(3, strides=2))&#xA;model.add(LeakyReLU(alpha=0.2))&#xA;model.add(Flatten())&#xA;model.add(Dense(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this model capable of learning such a task or should I use a different model architecture? &lt;/p&gt;&#xA;" OwnerUserId="33364" LastEditorUserId="2444" LastEditDate="2020-02-13T02:21:46.193" LastActivityDate="2020-02-13T02:21:46.193" Title="Generation of realistic real-valued sequences using Wasserstein GAN fails" Tags="&lt;keras&gt;&lt;generative-adversarial-networks&gt;&lt;time-series&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17907" PostTypeId="1" AcceptedAnswerId="17908" CreationDate="2020-02-07T19:55:55.050" Score="3" ViewCount="87" Body="&lt;p&gt;Given a ridge and a lasso regularizer, which one should be chosen for better performance? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An intuitive graphical explanation (intersection of the elliptical contours of the loss function with the region of constraints) would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="33314" LastEditorUserId="2444" LastEditDate="2021-01-29T23:10:15.777" LastActivityDate="2021-01-29T23:10:15.777" Title="Which is a better form of regularization: lasso (L1) or ridge (L2)?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;regularization&gt;&lt;l2-regularization&gt;&lt;l1-regularization&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17908" PostTypeId="2" ParentId="17907" CreationDate="2020-02-07T20:50:56.980" Score="3" Body="&lt;p&gt;The following graph shows the constraint region (green), along with contours for Residual sum of squares (red ellipse). These are iso-lines signifying that points on an ellipse have the same RSS.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0VhAZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0VhAZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;em&gt;&lt;strong&gt;Figure&lt;/strong&gt;: Lasso (left) and Ridge (right) Constraints&lt;/em&gt;  [Source: Elements of Statistical Learning]&lt;/p&gt;&#xA;&lt;p&gt;As &lt;strong&gt;Ridge&lt;/strong&gt; regression has a circular constraint (&lt;span class=&quot;math-container&quot;&gt;$\beta_1^2 + \beta_2^2 &amp;lt;= d$&lt;/span&gt;) with no edges, the intersection will not occur on an axis, signifying that the ridge regression parameters will usually be  non-zero.&lt;/p&gt;&#xA;&lt;p&gt;On the contrary, the &lt;strong&gt;Lasso&lt;/strong&gt; constraint (&lt;span class=&quot;math-container&quot;&gt;$|\beta_1| + |\beta_2| &amp;lt;= d$&lt;/span&gt;) has corners at each of the axes, and so the ellipse will often intersect the constraint region at an axis. In 2D, such a scenario would result in one of the parameters to become zero whereas in higher dimensions, more of the parameter estimates may simultaneously reach zero.&lt;/p&gt;&#xA;&lt;p&gt;This is a &lt;em&gt;disadvantage&lt;/em&gt; of ridge regression, wherein the least important predictors never get eliminated, resulting in the final model to contain all predictor variables. For Lasso, the L1 penalty forces some parameters to be exactly equal to zero when &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is large. This has a dimensionality reduction effect resulting in sparse models.&lt;/p&gt;&#xA;&lt;p&gt;In cases where the number of predictors are small, L2 could be chosen over L1 as it constraints the coefficient norm retaining all predictor variables.&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-07-24T10:57:48.243" LastActivityDate="2020-07-24T10:57:48.243" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17909" PostTypeId="1" CreationDate="2020-02-07T21:24:29.433" Score="2" ViewCount="37" Body="&lt;p&gt;There is a popular strategy of using a neural network trained on one task to produce features for another related task by &quot;chopping off&quot; the top of the network and sewing the bottom onto some other modeling pipeline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Word2Vec models employ this strategy, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an industry-popular term for this strategy?  Are there any good resources that discuss its use in general terms?&lt;/p&gt;&#xA;" OwnerUserId="21298" LastActivityDate="2020-02-08T11:28:45.007" Title="Strategy of using intermediate layers of a neural network as features?" Tags="&lt;neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17910" PostTypeId="1" AcceptedAnswerId="17914" CreationDate="2020-02-07T21:32:09.327" Score="1" ViewCount="51" Body="&lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i),$$&lt;/span&gt; why does &lt;span class=&quot;math-container&quot;&gt;$p(x^1,...,x^N)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q(z|x^i)$&lt;/span&gt; have the same parameter &lt;span class=&quot;math-container&quot;&gt;$\theta?$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is just the probability of the observed data and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is the approximation of the posterior, shouldn't they be different distributions and thus their parameters different?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-11-07T00:57:50.970" LastActivityDate="2020-11-07T00:57:50.970" Title="In this VAE formula, why do $p$ and $q$ have the same parameters?" Tags="&lt;variational-autoencoder&gt;&lt;latent-variable&gt;&lt;evidence-lower-bound&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17911" PostTypeId="2" ParentId="17909" CreationDate="2020-02-07T21:35:51.110" Score="1" Body="&lt;p&gt;This is an example of &lt;strong&gt;Transfer Learning&lt;/strong&gt;, wherein instead of starting the learning process from scratch, a pre-trained model is used and appended with custom layers to achieve additional functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zifEn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zifEn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-02-07T21:44:40.590" LastActivityDate="2020-02-07T21:44:40.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17912" PostTypeId="2" ParentId="17909" CreationDate="2020-02-07T21:37:03.730" Score="1" Body="&lt;p&gt;This a typical transfer learning technique, a lot of people refer to it with fine-tunning. I would recommend that you have a look on &lt;a href=&quot;https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch tutorial&lt;/a&gt;: it explains well how to use.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="2444" LastEditDate="2020-02-08T11:28:45.007" LastActivityDate="2020-02-08T11:28:45.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17913" PostTypeId="1" CreationDate="2020-02-07T21:42:32.330" Score="1" ViewCount="16" Body="&lt;p&gt;I'm trying to create a noising model that accurately reflects how people would noise name data. I was thinking of randomly switching out characters and creating a probability over which character gets switched in based on keyboard closeness and how similar anatomically another character looks to it. For example, &quot;l&quot; has a higher prob of being switched in with &quot;|&quot; and &quot;k&quot; cause &quot;k&quot; is close by on the keyboard and &quot;|&quot; looks like &quot;l&quot;, but that requires a lot of hard coding and reward for that seemed low because that's not the only 2 ways people can noise things. I also had the same idea above except use template matching of every character to every other character but itself and that would give it a similarity score then divide that by the sum over all chars to get the probs. Any other suggestions? My goal is the maximize closeness to actual human noising.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastActivityDate="2020-02-07T21:42:32.330" Title="Creating a noising model for NLP that models human noising" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17914" PostTypeId="2" ParentId="17910" CreationDate="2020-02-07T23:44:10.423" Score="1" Body="&lt;p&gt;&lt;em&gt;I will try to answer your questions directly (but I guess I won't be able to), otherwise, this can become quite confusing, given the inconsistencies that can be found across different sources.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$logp_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$&lt;/span&gt; why is &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and param for &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a few words, your equation is wrong because it uses the letters &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; inconsistently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look more carefully at the &lt;strong&gt;right-hand side&lt;/strong&gt; of your equation, you will notice that &lt;span class=&quot;math-container&quot;&gt;$q_{\theta}$&lt;/span&gt; has different parameters, i.e. &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, than &lt;span class=&quot;math-container&quot;&gt;$p_{\phi}$&lt;/span&gt;, which has parameters &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; have different parameters, and this should be the case, because they are represented by different neural networks in the case of the VAE. &lt;strong&gt;However&lt;/strong&gt;, the &lt;strong&gt;left-hand side&lt;/strong&gt; uses &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; as the parameters of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (while the right-hand side uses &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; to index &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;), so this should already suggest that &lt;strong&gt;the equation is not correct&lt;/strong&gt; (as you correctly thought).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the VAE, &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; &lt;strong&gt;usually&lt;/strong&gt; represents the parameters (or weights) of the encoder neural network (NN), while &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; usually represents the parameters of the decoder NN (or vice-versa, but you should just be consistent, which is often not the case in your equation). In fact, in &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the VAE paper&lt;/a&gt;, in &lt;strong&gt;equation 3&lt;/strong&gt;, the authors use &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; to represent the parameters of the encoder &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is used to denote the parameters of the decoder &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you follow the notation in the VAE paper, the ELBO can actually be written something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathcal{L}(\phi,\theta; \mathbf{x}) &#xA;&amp;amp;= &#xA;\mathbb{E}_{\tilde{z} \sim q_{\phi}(\mathbf{z} \mid \mathbf{x})} \left[ \log p_{\theta} (\mathbf{x} \mid \mathbf{z}) \right] - \operatorname{KL} \left(q_{\phi}(\mathbf{z} \mid \mathbf{x}) \| p_{\theta}(\mathbf{z}) \right) \tag{1} \label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ELBO loss &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(\phi,\theta; \mathbf{x})$&lt;/span&gt; has both parameters (of the encoder and decoder), which will be optimized jointly. Note that I have ignored the indices in the observations &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; (for simplicity), while, in &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the VAE paper&lt;/a&gt;, they are present. Furthermore, note that, both in \ref{1} and in &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the VAE paper&lt;/a&gt;, we use bold letters (because these objects are usually vectors), i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt;, rather than &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; (like in your equation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that, even though &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(\mathbf{z})$&lt;/span&gt; is indexed by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, in reality, this may be an un-parametrized distribution (e.g. a Gaussian with mean &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and variance &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;), i.e. not a family of distributions. The use of the index &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(\mathbf{z})$&lt;/span&gt; comes from the (implicit) assumption that both &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(\mathbf{z})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_{\theta} (\mathbf{x} \mid \mathbf{z})$&lt;/span&gt; come from the same family of distributions (e.g. a family of Gaussians). In fact, if you consider the family of all Gaussian distributions, then &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(\mathbf{z}) = \mathcal{N}(\mathbf{0}, \mathbf{I})$&lt;/span&gt; also belongs to that family. But &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; are also used to denote the parameters (or weights) of the networks, so this becomes understanbly confusing. (To understand equation 10 of the VAE paper, see &lt;a href=&quot;https://stats.stackexchange.com/a/370048/82135&quot;&gt;https://stats.stackexchange.com/a/370048/82135&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why does &lt;span class=&quot;math-container&quot;&gt;$p(x^1,...,x^N)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q(z|x^i)$&lt;/span&gt; have the same parameter &lt;span class=&quot;math-container&quot;&gt;$\theta?$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is &lt;strong&gt;wrong&lt;/strong&gt;, in fact. If you look at equation 1 of &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the VAE paper&lt;/a&gt;, they use &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; to denote the parameters of &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{x})$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(\mathbf{x})$&lt;/span&gt;, while the parameters of the encoder are &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$q_{\phi}(\mathbf{z} \mid \mathbf{x}$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Cause &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is just the probability of the observed data and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is the approximation of the posterior so shouldn't they be different distributions and their parameters different?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-08T00:49:54.117" LastActivityDate="2020-02-08T00:49:54.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17915" PostTypeId="2" ParentId="17868" CreationDate="2020-02-08T01:08:24.907" Score="2" Body="&lt;p&gt;Neither of the above mentioned methods could be a potent indicator of the performance of a model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple way to train the model &lt;em&gt;just enough&lt;/em&gt; so that it generalizes well on unknown datasets would be to monitor the validation loss. Training should be stopped once the validation loss progressively starts increasing over multiple epochs. Beyond this point, the model learns the statistical noise within the data and starts &lt;em&gt;overfitting&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EwzMW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EwzMW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This technique of &lt;strong&gt;Early stopping&lt;/strong&gt; could be implemented in Keras with the help of a callback function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class EarlyStop(tf.keras.callbacks.Callback):&#xA;    def on_epoch_end(self, epoch, logs={}):&#xA;        if(logs.get('val_loss') &amp;lt; LOSS_THRESHOLD and logs.get('val_categorical_accuracy') &amp;gt; ACCURACY_THRESHOLD):&#xA;            self.model.stop_training = True&#xA;&#xA;callbacks= EarlyStop()&#xA;model.fit(...,callbacks=[callbacks])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Loss and Accuracy thresholds can be estimated after a trial run of the model by monitoring the validation/training error graph.&lt;/p&gt;&#xA;" OwnerUserId="33010" LastEditorUserId="33010" LastEditDate="2020-02-08T01:14:05.177" LastActivityDate="2020-02-08T01:14:05.177" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17916" PostTypeId="1" AcceptedAnswerId="17931" CreationDate="2020-02-08T04:11:32.257" Score="1" ViewCount="27" Body="&lt;p&gt;I trained &lt;a href=&quot;https://arxiv.org/pdf/1710.00925.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this network&lt;/a&gt; from &lt;a href=&quot;https://github.com/natanielruiz/deep-head-pose&quot; rel=&quot;nofollow noreferrer&quot;&gt;this github&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training went well, and returns nice results for new, unseen images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On training, the loss changed (decreased), thus I must assume the weights changed as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On training, I saved a snapshot of the net every epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When trying to run a validation set through each epoch's snapshot, I get the exact same results on every epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be possible? What's causing this?&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2020-02-09T12:29:20.307" Title="Trained a regression network and getting EXACT same result on validation set, on every epoch" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17917" PostTypeId="1" CreationDate="2020-02-08T04:21:20.073" Score="1" ViewCount="133" Body="&lt;p&gt;I'm working on a problem where we are given a graph and asked to perform various search algorithms (BFS, DFS, UCS, A*, etc.) and the goal state is to visit all nodes in the graph. After all nodes are visited, we need to print out the &quot;solution path.&quot; However, I am a bit confused on what &quot;path&quot; means in AI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's just consider a graph of 3 nodes: A, B and C with 2 undirected edges (A, B) (A, C). If we perform BFS on this graph starting at node A and traversing alphabetically, we'd visit A, then B, then C.  So, in this case, is the solution path &lt;code&gt;A -&amp;gt; B -&amp;gt; C&lt;/code&gt;, i.e. the order in which the nodes are visited? Or is the solution path &lt;code&gt;A -&amp;gt; B -&amp;gt; A -&amp;gt; C&lt;/code&gt;? Basically saying that we go from A to B, but to go from B to C, we must go through A again. &lt;/p&gt;&#xA;" OwnerUserId="20358" LastEditorUserId="20358" LastEditDate="2020-02-08T14:02:54.773" LastActivityDate="2020-11-05T23:01:35.130" Title="How to report the solution path of a search algorithm on a graph?" Tags="&lt;search&gt;&lt;path-planning&gt;&lt;path-finding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17918" PostTypeId="1" CreationDate="2020-02-08T10:25:39.230" Score="3" ViewCount="53" Body="&lt;p&gt;How do newer weight initialization techniques (He, Xavier, etc) improve results over zero or random initialization of weights in a neural network? Is there any mathematical evidence behind this?&lt;/p&gt;&#xA;" OwnerUserId="33314" LastEditorUserId="2444" LastEditDate="2020-11-14T15:57:14.307" LastActivityDate="2020-11-14T15:57:14.307" Title="How are newer weight initialization techniques better than zero or random initialization?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17919" PostTypeId="2" ParentId="17918" CreationDate="2020-02-08T10:47:45.960" Score="3" Body="&lt;p&gt;There are several ways to answer this question. First of all, there are several mathematical arguments on why using some kind of initialization is better. Consider reading, for example, &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xavier et al.&lt;/a&gt;. Moreover, there are several numerical experiments showing the importance of initialization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The motivation for Xavier initialization in Neural Networks is to initialize the weights of the network so that the neuron activation functions are not starting out in saturated or dead regions. In other words, we want to initialize the weights with random values that are not &quot;too small&quot; and not &quot;too large&quot;. Thus the purpose is to fix the variance of the input data to each neuron to 1, because this reduces the variance of getting stuck in saturated areas, since the data is in general normalized this is equivalent to fixing the variance of each weight to &lt;span class=&quot;math-container&quot;&gt;$1/n$&lt;/span&gt;, where n is the number of input weights to the given neuron. &lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="2444" LastEditDate="2020-02-08T11:15:29.277" LastActivityDate="2020-02-08T11:15:29.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17920" PostTypeId="2" ParentId="17865" CreationDate="2020-02-08T10:54:53.307" Score="1" Body="&lt;p&gt;One of the used is to use a neural interpolation, this is using a pretrained model to zoom our images, in some sense training with this kind of images is like combining this pretrainined model with our new data, and can be seen as transfer learning technique &lt;/p&gt;&#xA;" OwnerUserId="33373" LastActivityDate="2020-02-08T10:54:53.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17921" PostTypeId="2" ParentId="17897" CreationDate="2020-02-08T11:05:17.227" Score="1" Body="&lt;p&gt;Bag-of-visual words (BOVW) was classicly used in computer vision before the introduction of neural networks or some more advanced classical techniques, such us VLAD or Fisher Vectors. In any case, it is a good technique to use, but it is not the state-of-the-art today, and I won't recommend you use it for a real-life project.&lt;/p&gt;&#xA;" OwnerUserId="33373" LastEditorUserId="2444" LastEditDate="2020-09-25T22:17:58.443" LastActivityDate="2020-09-25T22:17:58.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17922" PostTypeId="2" ParentId="16225" CreationDate="2020-02-08T11:22:46.520" Score="2" Body="&lt;p&gt;Let's start by making clear that both AS and MMAS use only global pheromone update. Now, the MMAS has two main differences regarding AS:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In AS, all ants that completed a solution are used for the update, while in MMAS only the best ant with a complete solution is used for update (as you had pointed out).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In AS, the pheromone values are not explicitly bounded. In MMAS, the pheromones are enforced to lie within a pre-set interval &lt;span class=&quot;math-container&quot;&gt;$\tau_{min} \leq \tau_{ij} \leq \tau_{max}$&lt;/span&gt; (which gives its name to the algorithm). To ensure this condition, the pheromone update is done by means of the formula&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tau_{ij} \leftarrow \left[ (1-\rho) \cdot \tau_{ij} + \Delta \tau_{ij}^{best} \right]_{\tau_{min}}^{\tau_{max}},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with the operator &lt;span class=&quot;math-container&quot;&gt;$[x]_a^b$&lt;/span&gt; defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;[x]_a^b= &#xA;\begin{cases}&#xA;    a \qquad \mathrm{if} \quad x &amp;gt; a\\&#xA;    b \qquad \mathrm{if} \quad x &amp;lt; b\\&#xA;    x \qquad \mathrm{otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Reference: &lt;a href=&quot;https://courses.cs.ut.ee/all/MTAT.03.238/2011K/uploads/Main/04129846.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ant Colony Optimization. Artificial Ants as a Computational Intelligence Technique&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33372" LastEditorUserId="2444" LastEditDate="2021-01-15T11:39:35.903" LastActivityDate="2021-01-15T11:39:35.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17923" PostTypeId="1" AcceptedAnswerId="17924" CreationDate="2020-02-08T11:23:28.713" Score="6" ViewCount="816" Body="&lt;p&gt;This question can seem a little bit too broad, but I am wondering what are the current state-of-the-art works on meta reinforcement learning. Can you provide me with the current state-of-the-art in this field?&lt;/p&gt;&#xA;" OwnerUserId="33374" LastEditorUserId="2444" LastEditDate="2020-02-08T11:39:42.183" LastActivityDate="2021-02-23T10:19:43.790" Title="What are the state-of-the-art meta-reinforcement learning methods?" Tags="&lt;reinforcement-learning&gt;&lt;meta-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17924" PostTypeId="2" ParentId="17923" CreationDate="2020-02-08T11:30:22.463" Score="3" Body="&lt;p&gt;One of the most recent papers on meta-RL is &lt;a href=&quot;https://arxiv.org/pdf/1910.00125.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta-Q-learning&lt;/a&gt;. This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-reinforcement learning (meta-RL). MQL builds upon three simple ideas.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Q-learning is competitive with state-of-the-art meta-RL algorithms if given access to a context variable that is a representation of the past trajectory.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using a multi-task objective to maximize the average reward across the training tasks is an effective method to meta-train RL policies.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;past data from the meta-training replay buffer can be recycled to adapt the policy on a new task using off-policy updates&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Experiments on standard continuous-control benchmarks suggest that MQL compares favorably with state-of-the-art meta-RL algorithms.&lt;/p&gt;&#xA;&lt;p&gt;I think that other references to other work on meta-RL are present in the experiments part of the MQL paper.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="2444" LastEditDate="2021-02-23T10:19:43.790" LastActivityDate="2021-02-23T10:19:43.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17925" PostTypeId="1" CreationDate="2020-02-08T14:23:21.473" Score="4" ViewCount="362" Body="&lt;p&gt;Given the standard illustrative feed-forward neural net model, with the dots as neurons and the lines as neuron-to-neuron connection, what part is the (unfold) LSTM cell (see picture)? Is it a neuron (a dot) or a layer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WnWwT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WnWwT.png&quot; alt=&quot;LSTM cell&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-02-09T01:16:02.283" LastActivityDate="2020-02-11T11:00:08.927" Title="Is the LSTM component a neuron or a layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17926" PostTypeId="1" CreationDate="2020-02-08T16:21:09.917" Score="2" ViewCount="78" Body="&lt;p&gt;If I had the weights of a certain number of &quot;parents&quot; that I wanted to crossbreed, and I used whatever method to pick out the &quot;best parents&quot; (I used a roulette wheel option, if that's any relevant), would I be doing this correctly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose I have picked the following two parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P_1 &amp;amp;= [0.5, -0.02, 0.4, 0.1, -0.9] \\&#xA;P_2 &amp;amp;= [0.42, 0.55, 0.18, -0.3, 0.12]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I'm iterating through each index (or gene) of the parents, I am selecting a weight from one parent only. I called this rate the &quot;cross-rate&quot;, which in my case is &lt;span class=&quot;math-container&quot;&gt;$0.2$&lt;/span&gt; (i.e. with &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt;% chance, I will switch to choosing the other parents' weight).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, using our example above, this is what would happen:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P_1 &amp;amp;=[\mathbf{0.5}, \mathbf{-0.02}, 0.4, 0.1, \mathbf{-0.9}] \\&#xA;P_2 &amp;amp;= [0.42, 0.55, \mathbf{0.18}, \mathbf{-0.3}, 0.12]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the child would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C = [0.5, -0.02, 0.18, -0.3, -0.9]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would choose &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$P_1$&lt;/span&gt;, but for every time I choose a weight from &lt;span class=&quot;math-container&quot;&gt;$P_1$&lt;/span&gt;, there's a 20% chance that I actually choose the corresponding gene from &lt;span class=&quot;math-container&quot;&gt;$P_2$&lt;/span&gt;. But, for the first weight, I end up not landing on that 20% chance. So I move onto the second weight, &lt;span class=&quot;math-container&quot;&gt;$-0.02$&lt;/span&gt;. This time, we hit the 20% chance, so now we swap over. Our next weight is now from &lt;span class=&quot;math-container&quot;&gt;$P_2$&lt;/span&gt;, which is &lt;span class=&quot;math-container&quot;&gt;$0.18$&lt;/span&gt;. And so on, until we hit another 20% chance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We keep doing this until we hit the end of the indexes (&lt;span class=&quot;math-container&quot;&gt;$P_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P_2$&lt;/span&gt; have the same number of indexes, of course).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the correct way to form a child from 2 parents? Is this the correct &quot;crossbreeding&quot; method when it comes to genetic algorithms?&lt;/p&gt;&#xA;" OwnerUserId="33379" LastEditorUserId="2444" LastEditDate="2020-02-09T14:40:57.823" LastActivityDate="2020-02-09T14:40:57.823" Title="How does crossover work in a genetic algorithm?" Tags="&lt;genetic-algorithms&gt;&lt;definitions&gt;&lt;crossover-operators&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17927" PostTypeId="2" ParentId="17926" CreationDate="2020-02-08T17:19:07.777" Score="3" Body="&lt;p&gt;As far as I know, there isn't a &quot;specified correct way&quot;. The whole idea is that you want the population to converge and increase the sample rate in that more optimal looking place. What works best all depends upon your fitness landscape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also crossover by doing something like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;crossover_point = random_number_size_genome&#xA;child[:] = parent_a[:crossover_point] + parent_b[crossover_point:]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or have multiple points of crossover or more exotic types of crossover.&#xA;As far as I know, the impact on your algorithm because of the different crossover algorithms shouldn't be that different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fitness function and how you select the fittest has a way bigger impact.&#xA;You have understood the crossover step correctly, looking at your example.&lt;/p&gt;&#xA;" OwnerUserId="30100" LastEditorUserId="2444" LastEditDate="2020-02-09T14:31:01.260" LastActivityDate="2020-02-09T14:31:01.260" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17928" PostTypeId="2" ParentId="8594" CreationDate="2020-02-08T17:39:13.440" Score="0" Body="&lt;p&gt;Yes it's error prone, just like us humans.&#xA;But just like in chess it is just a whole lot better at dealing with it.&#xA;AI does not contain all the possible knowledge in the universe out of no where without interaction with the world and such it should need to make assumptions and test those assumptions making it prone to error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're asking about current techniques within reinforcement learning, image detection,etc. Those techniques are error prone, just likes us humans. During training of those algorithms you have a trade off between how good it will generalize on new data and how correct it can answer your questions. (it will &quot;memorize&quot; the questions and not actually learn in the latter, called overfitting)&lt;/p&gt;&#xA;" OwnerUserId="30100" LastActivityDate="2020-02-08T17:39:13.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17929" PostTypeId="2" ParentId="17925" CreationDate="2020-02-08T17:56:54.720" Score="5" Body="&lt;p&gt;The diagram you show works at least partially for describing both individual neurons and layers of those neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the &quot;incoming&quot; data lines on the left represent all inputs under consideration, typically a vector of all inputs to the cell. That includes all data from current time steps (from input layer or earlier LSTM or time-distributed layers) - the line coming up into the LSTM cell - and the full output and cell state vectors from the whole LSTM layer on previous time step - the horizontal lines inside the cell starting on the left. Technically the top left input line could be read as either be a single neuron cell state value, or the full vector of cell state from the previous time step, depending on whether you were viewing the diagram as describing a single neuron in the layer, or the whole layer respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you visualise this cell connecting to itself over time steps, then the data lines both in and out must be whole layer vectors. The diagram is then best thought of as representing a whole LSTM layer, which is composed of various sub-layers which get combined, such as the forget gate layer (the leftmost yellow box).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each yellow box in the diagram can be implemented very similar to a single layer of a simple feed forward NN, with its own weights and biases. So the forget gate can be implemented as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{f}_t = \sigma(\mathbf{W}_f&#xA;[\mathbf{x}_t , \mathbf{y}_{t-1}] + \mathbf{b}_f)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}_f$&lt;/span&gt; is the weight matrix of the forget gate, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}_f$&lt;/span&gt; is the bias vector, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_t$&lt;/span&gt; is layer input on current time step, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}_{t-1}$&lt;/span&gt; is layer output from previous time step, and the comma &lt;span class=&quot;math-container&quot;&gt;$,$&lt;/span&gt; is showing concatenation of those two (column) vectors into one large column vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional vector, and output &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional vector, then:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{c}$&lt;/span&gt;, the cell state (top line), and all interim layer outputs are &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional vectors&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}_f$&lt;/span&gt;, the forget gate weights, and the three other internal weights matrices are &lt;span class=&quot;math-container&quot;&gt;$m \times (n+m)$&lt;/span&gt; matrices&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}_f$&lt;/span&gt;, the forget gate bias, and the three other internal bias vectors are &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional vectors&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The other gates are near identical (using their own weights and biases, plus the third cell candidate value uses tanh instead of logistic sigmoid), and have their own weight matrices etc. The whole thing is constructed a lot like four separate feed-forward layers, each receiving identical input, that are then combined using element-wise operations to perform the functions of the LSTM cell. There is nothing stopping you implementing each yellow box as a more sophisticated and deeper NN in its own right, but that seems relatively rare in practice, and it is more common to stack LSTM layers to achieve that kind of depth.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-02-11T11:00:08.927" LastActivityDate="2020-02-11T11:00:08.927" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17930" PostTypeId="1" CreationDate="2020-02-09T10:01:44.163" Score="2" ViewCount="135" Body="&lt;p&gt;Are we able to use models like GPT-2 to smooth out/correct text? For instance if I have two paragraphs that need some text to make the transition easier to read, could this text be generated? And, could it find inconsistencies between the paragraphs and fix them?&lt;/p&gt;&#xA;&lt;p&gt;As an example, imagine we're reordering some text so that we can apply the &lt;a href=&quot;https://medium.com/lessons-from-mckinsey/the-pyramid-principle-f0885dd3c5c7&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyramid principle&lt;/a&gt;. What I'd like to do is reorder the sentences/paragraphs and still have a coherant story. The following three sentences for instance, start with a statement and then have some facts to support it. What's missing is the story that joins them together, right now they're three independent sentences.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The strawberry is the best fruit based on its flavor profile, its coloring and texture and the nutritional profile.&lt;/p&gt;&#xA;&lt;p&gt;Strawberries are very rich in antioxidants and plant compounds, which may have benefits for heart health and blood sugar control.&lt;/p&gt;&#xA;&lt;p&gt;Strawberries have a long history and have been enjoyed since the Roman times.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Feel free to point me at things to read, I have not been able to find anything like this in my searches.&lt;/p&gt;&#xA;" OwnerUserId="33392" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-09T14:45:46.013" Title="Can we use GPT-2 to smooth out / correct text?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17931" PostTypeId="2" ParentId="17916" CreationDate="2020-02-09T12:29:20.307" Score="0" Body="&lt;p&gt;The answer is I was too tired to see I was using the same model for validation every epoch, instead of that epoch's model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Write good code, and this won't happen to you.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2020-02-09T12:29:20.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17932" PostTypeId="1" CreationDate="2020-02-09T13:05:26.440" Score="3" ViewCount="50" Body="&lt;p&gt;I am working towards using RL to create an AI for a two-player, hidden-information, a turn-based board game. I have just finished David Silver's RL course and Denny Britz's coding exercises, and so am relatively familiar with MC control, SARSA, Q-learning, etc. However, the course was focused on single-player, perfect-information games, and I haven't managed to find any examples similar to the type of game I have, and would like advice on how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am still unsure how self-play works, and how it relates to MCTS. For example, I don't know if this involves using the latest agent to play both sides, or playing an agent against older versions, or training multiple opposing agents simultaneously. Are there good examples (or repositories) for learning self-play and MCTS for two-player games? &lt;/p&gt;&#xA;" OwnerUserId="33397" LastEditorUserId="2444" LastEditDate="2020-02-09T14:28:58.093" LastActivityDate="2020-02-09T14:28:58.093" Title="How exactly does self-play work, and how does it relate to MCTS?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;monte-carlo-tree-search&gt;&lt;pomdp&gt;&lt;self-play&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17933" PostTypeId="5" CreationDate="2020-02-09T14:45:08.067" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-09T14:45:08.067" LastActivityDate="2020-02-09T14:45:08.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17934" PostTypeId="4" CreationDate="2020-02-09T14:45:08.067" Score="0" Body="For questions related to the concept of elitism (i.e. allow only the best organisms from the current generation to carry over to the next) in the context of evolutionary algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-09T14:45:08.067" LastActivityDate="2020-02-09T14:45:08.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17935" PostTypeId="1" CreationDate="2020-02-09T17:42:04.570" Score="4" ViewCount="122" Body="&lt;p&gt;I'm working on research in this sector where my supervisor wants to do cannonicalization of name data using VAEs, but I don't think it's possible to do, but I don't know explicitly how to show it mathematically. I just know empirically that VAEs don't do good on discrete distributions of latents and observed variables(Because in order to do names you need your latent to be the character at each index and it can be any ASCII char, which can only be represented as a distribution). So the setup I'm using is a VAE with 3 autoencoders, for latents, one for first, middle and last name and all of them sample each character of their respective names from the gumbel-softmax distribution(A form a categorical that is differentiable where the parameters is a categorical dist). From what I've seen in the original paper on the simple problem of MNIST digit image generation, the inference and generative network both did worse as latent dimension increased and as you can imagine the latent dimension of my problem is quite large. That's the only real argument for why this can't work, that I have. The other would have been it's on a discrete distribution, but I solved that by using a gumbel softmax dist instead. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This current setup isn't working at all, the name generations are total gibberish and it plateaus really early. Are there any mathematical intuitions or reasons that VAEs won't work on a problem like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a note I've also tried semi-supervised VAEs and it didn't do much better. I even tried it for seq2seq of just first names given a first and it super failed as well and I'm talking like not even close to generation of names or the original input.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastActivityDate="2020-02-12T00:08:25.087" Title="Why can't VAE do sequence to sequence name generation?" Tags="&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="10" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17936" PostTypeId="2" ParentId="17917" CreationDate="2020-02-09T20:04:10.183" Score="1" Body="&lt;h1&gt;Defining Path&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You are right to be confused if your professors did not clarify (see warning at end). The term &quot;path&quot; can mean a few things:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&quot;Concrete&quot; Path&lt;/strong&gt;:&#xA;Recall, a graph is a collection of vertices and edges. A path on the graph is then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_1\xrightarrow{e_1} v_2\xrightarrow{e_2} \cdots \xrightarrow{e_{n-1}} v_n$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; are vertices on the graph and the arrows denote the direction traversed over the undirected edges &lt;span class=&quot;math-container&quot;&gt;$e_j$&lt;/span&gt;. Observe, this definition explicitly defines some &quot;chain&quot; of vertices connected by specific edges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&quot;Abstract&quot; Path&lt;/strong&gt;: Say there is some task that involves a serious of repeated &lt;em&gt;actions&lt;/em&gt; but does not depend on specific objects. We can define this as a chain of composed functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_n\circ\cdots \circ f_1$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea, is that this definition is little more abstract. Though, there are many other ways one could define &quot;path.&quot;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&quot;Solution Path&quot;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Subtlety&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, in this case, is the solution path A -&gt; B -&gt; C, i.e. the order in which the nodes are visited? Or is the solution path A -&gt; B -&gt; A -&gt; C? Basically saying that we go from A to B, but to go from B to C, we must go through A again.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;BFS does not &quot;go through A again.&quot; Considering you are using a queue based implementation, once A has been visited there will be no need to &quot;visit&quot; it again. That is, we have added all nodes we need to the queue and marked A as being seen. This leads into:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How to report the solution path of a search algorithm on a graph?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since, these algorithms do not revisit nodes, what is &lt;strong&gt;most likely&lt;/strong&gt; being asked of you is for you to report the order of the visits - this is what my professors wanted in my AI &amp;amp; Algorithms courses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Warning&lt;/strong&gt;: I can only answer what &lt;em&gt;my professors&lt;/em&gt; wanted. Please clarify this with your instructors. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2020-02-09T20:12:36.630" LastActivityDate="2020-02-09T20:12:36.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17937" PostTypeId="1" AcceptedAnswerId="17938" CreationDate="2020-02-09T22:16:58.823" Score="3" ViewCount="693" Body="&lt;p&gt;Is there any need to use a non-linear activation function (ReLU, LeakyReLU, Sigmoid, etc.) if the result of the convolution layer is passed through the sliding window max function, like max-pooling, which is non-linear itself? What about the average pooling?&lt;/p&gt;&#xA;" OwnerUserId="31988" LastEditorUserId="2444" LastEditDate="2020-02-09T23:36:51.390" LastActivityDate="2020-02-09T23:44:37.177" Title="Is a non-linear activation function needed if we perform max-pooling after the convolution layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17938" PostTypeId="2" ParentId="17937" CreationDate="2020-02-09T23:03:19.290" Score="5" Body="&lt;p&gt;Let's first recapitulate why the function that calculates the maximum between two or more numbers, &lt;span class=&quot;math-container&quot;&gt;$z=\operatorname{max}(x_1, x_2)$&lt;/span&gt;, is not a linear function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A linear function is defined as &lt;span class=&quot;math-container&quot;&gt;$y=f(x) = ax + b$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; linearly increases with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Visually, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; corresponds to a straight line (or hyperplane, in the case of 2 or more input variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; does not correspond to such a straight line (or hyperplane), then it cannot be a linear function (by definition). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x_1 = 1$&lt;/span&gt; and let &lt;span class=&quot;math-container&quot;&gt;$x_2 \in [0, 2]$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$z=\operatorname{max}(x_1, x_2) = x_1$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$x_2 \in [0, 1]$&lt;/span&gt;. In other words, for the sub-range &lt;span class=&quot;math-container&quot;&gt;$x_2 \in [0, 1]$&lt;/span&gt;, the maximum between &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; is a constant function (a horizontal line at &lt;span class=&quot;math-container&quot;&gt;$x_1=1$&lt;/span&gt;). However, for the sub-range &lt;span class=&quot;math-container&quot;&gt;$x_2 \in [1, 2]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; correspond to &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; linearly increases with &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;. Given that max is not a linear function in a special case, it can't also be a linear function in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a plot (computed with &lt;a href=&quot;https://www.wolframalpha.com/input/?i=plot+max%28x%2C+y%29&quot; rel=&quot;noreferrer&quot;&gt;Wolfram Alpha&lt;/a&gt;) of the maximum between two numbers (so it is clearly a function of two variables, hence the plot is 3D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RNRu6.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RNRu6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, in this plot, both variables, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, can linearly increase, as opposed to having one of the variables fixed (which I used only to give you a simple and hopefully intuitive example that the maximum is not a linear function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of convolution networks, although max-pooling is a non-linear operation, it is primarily used to &lt;strong&gt;reduce the dimensionality&lt;/strong&gt; of the input, so that to reduce overfitting and computation. In any case, max-pooling doesn't non-linearly transform the input &lt;strong&gt;element-wise&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The average function is a linear function because it linearly increases with the inputs. Here's a &lt;a href=&quot;https://www.wolframalpha.com/input/?i=plot+%28x%2By%29%2F2&quot; rel=&quot;noreferrer&quot;&gt;plot of the average between two numbers&lt;/a&gt;, which is clearly a hyperplane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/a3oHU.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a3oHU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of convolution networks, the average pooling is also used to reduce the dimensionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question more directly, the non-linearity is usually applied element-wise, but neither max-pooling nor average pooling can do that (even if you downsample with a &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; window, i.e. you do not downsample at all). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, you don't necessarily need a non-linear activation function after the convolution operation (if you use max-pooling), but the performance will be worse than if you use a non-linear activation, as reported in the paper &lt;a href=&quot;https://arxiv.org/pdf/1606.02228.pdf&quot; rel=&quot;noreferrer&quot;&gt;Systematic evaluation of CNN advances on the&#xA;ImageNet&lt;/a&gt; (figure 2).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-09T23:44:37.177" LastActivityDate="2020-02-09T23:44:37.177" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17939" PostTypeId="1" CreationDate="2020-02-10T00:19:51.523" Score="3" ViewCount="53" Body="&lt;p&gt;Are all RL algorithms based on the MDP? If not, could you give examples of some which aren't? I've looked elsewhere, but I haven't seen it explicitly said. &lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-10T03:01:16.007" LastActivityDate="2020-02-10T03:01:16.007" Title="Are there reinforcement learning algorithms not based on Markov decision processes?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17940" PostTypeId="1" CreationDate="2020-02-10T05:07:35.607" Score="1" ViewCount="19" Body="&lt;p&gt;I'm trying to implement a neural network that is able to generate an image indicating territory occupation given a board state for GO (a strategy board game). Input images are 19x19x1 grayscale images, with white pixels indicating white pieces, black pixels indicating black pieces, and gray pixels indicating unoccupied areas. Output images are 19x19x1 grayscale images with white pixels indicating white territory, black pixels indicating black territory, and gray areas indicating unassigned territories. A sample input and desired output image is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tr9L1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tr9L1.png&quot; alt=&quot;Input Image Showing Board State&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2eQ36.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2eQ36.png&quot; alt=&quot;Output Image Indicating Classified Territories&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images are quite small, so just to give an overview of trends I noticed: - Pixels surrounded by pixels of opposite colors are 'captured' pieces and therefore part of opponent territory - Two 'eyes' or closed groups of pieces comprising at least two open intersections are invincible or confirmed territory&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I'm not looking for exact specifications of network layers etc., I was hoping I could be given some direction as to what type of network to use, and what it should comprise. Looking at MATLAB documentation, I've found info about semantic segmentation, and autoencoder networks but neither of these seem particularly helpful. I know the question is a little broad, but I just need some direction more than anything. This kind of image recognition problem is a first for me.&lt;/p&gt;&#xA;" OwnerUserId="33413" LastActivityDate="2020-02-10T05:07:35.607" Title="Image-to-Image Regression for GO territory classification" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;go&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17941" PostTypeId="1" CreationDate="2020-02-10T06:26:13.920" Score="1" ViewCount="36" Body="&lt;p&gt;I was going through the chapter &quot;Monte Carlo Methods&quot; from the book  &quot;Reinforcement Learning&quot; by Sutton and Barto. The author says that when a model is not available then it is useful to estimate action-values rather than state-value. But I am confused about what he means by a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please refer to the screenshot below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hhszb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hhszb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31749" LastActivityDate="2020-02-10T06:26:13.920" Title="In reinforcement learning what do we mean by a model?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-02-10T08:45:13.477" ContentLicense="CC BY-SA 4.0" />
  <row Id="17942" PostTypeId="1" CreationDate="2020-02-10T07:37:01.027" Score="1" ViewCount="81" Body="&lt;p&gt;We are attempting to build an AI that manages to play the cardgame &lt;a href=&quot;https://en.wikipedia.org/wiki/Wizard_(card_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wizard&lt;/a&gt;. So far er have a working network (based on the YOLO object-detection) that is abled to detect which cards are played. When asked it returns the color and rank of the cards on the table. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now when starting to build an agent for the actual training I just cant figure out how to represent the states for this game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In each round, each player gets the amount of cards represented by the round(one card in round one, two in round two and so on). Based on that the players estimate how many tricks they will win in this round. With ending the round the players calculate their points w.r.t their estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the agent have to estimate its future tricks and have to play depending on that strategy. So how do I encode that into a form that a neural network can work with? &lt;/p&gt;&#xA;" OwnerUserId="30431" LastActivityDate="2020-02-10T07:37:01.027" Title="How to represent a state in a card game environment? (Wizard)" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17943" PostTypeId="1" CreationDate="2020-02-10T07:56:29.147" Score="1" ViewCount="48" Body="&lt;p&gt;I would like to ask is it possible to train an agent under self-playing setting but with just one policy to be trained? What are the foreseeable problems with such an implementation? My concern is as such: suppose agent A starts off as in role 1, and later goes role 2 (which is role 1's opponent) after a few iteration. When A switches to role 2, it might use the information gained when A was in role 1 to its current role's advantage. And after another few iterations when A switches back to role 1, A might use the information gained when A was in role 2 to its current role's advantage. This is because we assume there is just one policy for agent A, so the weights are updated and kept when A switches its role. The information gained can be things such as new strategies learnt, new information gained (when the agent's knowledge of the environment is incomplete), etc. So a more sensible way of training I think would be using two agents A and B with two separate policies to train - more like in a generative adversarial network setting in some sense. But then by definition this is not considered as self-play isn't it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So relating to the case of Alphazero playing chess, if the agent is really just playing with itself, despite chess belongs to the so-called perfect information game, but the opponent's thoughts/strategy/decision making process should be still unknown to the player. But if my assumption of how self-playing is true, then the agent A will have the thoughts/strategy/decision making process of role 2 when it is in role 1. Isn't that cheating (because in reality those knowledge are not attainable; rather one can only guess what are those information from its opponents)? (I haven't read through the Alphazero paper yet because I am not proficient enough to understand those technical details, so I would really appreciate it if anyone could explain the relevant part to me)&lt;/p&gt;&#xA;" OwnerUserId="33419" LastEditorUserId="33419" LastEditDate="2020-02-12T04:48:35.387" LastActivityDate="2020-02-12T04:48:35.387" Title="Is it possible to use just one policy in a self-play setting?" Tags="&lt;reinforcement-learning&gt;&lt;self-play&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2020-02-13T18:10:54.937" ContentLicense="CC BY-SA 4.0" />
  <row Id="17944" PostTypeId="1" CreationDate="2020-02-10T09:37:42.923" Score="2" ViewCount="71" Body="&lt;p&gt;I'd like to buy an eGPU for my MaxBook Pro to use for simple deep learning tasks. My setup is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MacBook Pro (15-inch, 2017)&#xA;Graphics: Radeon Pro 555 2 GB&#xA;Intel HD Graphics 630 1536 MB&#xA;Version: Mojave 10.14.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand for Deep Learning (i.e. use of tensorflow-gpu) this is not currently supported for my Mac. Due to previous disputes between Nvidia and Apple I assume that Nvidia's support is reluctant to offer any kind of hacky solution with their graphic cards. On saying that, I was recommended to purchase the NVIDIA TITAN RTX or NVIDIA Quadro® GV100, but they're quite pricey at 1000s of euros/dollars a piece. At first, I just want something to play around with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I watched &lt;a href=&quot;https://www.youtube.com/watch?v=JjL_50ZNaKY&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=l-GOx9ttdKk&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; to see how to configure the Mac with an eGPU that is CUDA supported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What Nvidia eGPU would you recommend for simple i.e. not mega large data sets for DL processing? There seems to be so many models to choose from that it's not clear what would satisfy my needs. Would a GIGABYTE GeForce® GTX 1050 Ti OC 4GB suffice?&lt;/p&gt;&#xA;" OwnerUserId="33421" LastActivityDate="2020-02-10T10:49:48.837" Title="External GPU for Mac" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;gpu&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17946" PostTypeId="1" CreationDate="2020-02-10T11:19:09.840" Score="2" ViewCount="20" Body="&lt;p&gt;Is there any solution about sign language to speech conversion for mobiles? Can anyone suggest me the flow and tools so that I may implement the solution for mobiles?&lt;/p&gt;&#xA;" OwnerUserId="33425" LastEditorUserId="9947" LastEditDate="2020-02-10T13:07:28.280" LastActivityDate="2020-02-10T13:07:28.280" Title="Sign Language to Speech conversion" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17948" PostTypeId="1" CreationDate="2020-02-10T16:49:58.210" Score="0" ViewCount="40" Body="&lt;p&gt;I'm working on data cleaning and I'm stuck. I have a data set with 3 columns: &lt;em&gt;id&lt;/em&gt;, &lt;em&gt;age&lt;/em&gt;, and &lt;em&gt;weight&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing I have an entry:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id:1 | age:3 (years)  | weight: 150 (kg)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can we detect that the information is wrong, assuming I have a thousand lines? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And how can I correct it (using Python)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any function in Python that I can use or should I use machine learning techniques?&lt;/p&gt;&#xA;" OwnerUserId="32560" LastEditorUserId="28343" LastEditDate="2020-02-11T17:34:13.510" LastActivityDate="2020-02-11T17:34:13.510" Title="How to automatically detect and correct false information in columnar data?" Tags="&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17949" PostTypeId="1" CreationDate="2020-02-10T17:41:40.633" Score="1" ViewCount="45" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1802.08773.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GraphRNN paper&lt;/a&gt;, the authors only implement the algorithm up to a graph size of 2k nodes. Would this still work on much larger graphs (on the order of &lt;span class=&quot;math-container&quot;&gt;$10^7$&lt;/span&gt;)? Or would the computation just become too substantial?&lt;/p&gt;&#xA;" OwnerUserId="12983" LastEditorUserId="2444" LastEditDate="2020-02-13T02:14:44.493" LastActivityDate="2020-02-13T02:14:44.493" Title="Can GraphRNN be used with very large graphs?" Tags="&lt;recurrent-neural-networks&gt;&lt;papers&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17950" PostTypeId="1" AcceptedAnswerId="17964" CreationDate="2020-02-10T18:06:17.707" Score="1" ViewCount="45" Body="&lt;p&gt;I'm trying to understand the role of data augmentation and how it can affect the performance/accuracy of a deep model. My target application is fire detection (on video frames), with almost 15K positive and negative samples, and I was using the following data augmentation techniques. Does using &lt;strong&gt;ALL&lt;/strong&gt; the followings always increase the performance? Or we have to choose them somehow &lt;strong&gt;smartly&lt;/strong&gt; given our target application?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rotation_range=20, width_shift_range=0.2, height_shift_range=0.2,zoom_range=0.2, horizontal_flip=True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I think a bit more, fire is always straight up, so I think &lt;code&gt;rotation&lt;/code&gt; or &lt;code&gt;shift&lt;/code&gt; might in fact worsen the results, given that it makes the image sides stretch like this, which is irrelevant to fires in video frames. Same with rotation. So I think maybe I should only keep &lt;code&gt;zoom_range=0.2, horizontal_flip=True&lt;/code&gt; and remove the first three. Because I see some false positives when we have a scene transition effect in videos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my argument correct? Should I keep them or remove them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4REwQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4REwQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DsOR8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DsOR8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-02-11T14:53:58.440" Title="Choosing Data Augmentation smartly for different application" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;object-detection&gt;&lt;data-preprocessing&gt;&lt;accuracy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17951" PostTypeId="2" ParentId="17948" CreationDate="2020-02-10T18:22:10.723" Score="2" Body="&lt;p&gt;While I can see that there are some heuristics that can tell you whether an entry is 'weird', I don't see any way that you can correct this. Where would you get a correct value from?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would perhaps start with a statistical analysis, looking at the distribution of values to get an idea of the state the data is in. From this you can then already see some values that wouldn't make sense (this depends very much on what data it is: census data will include toddlers, but credit card applications wouldn't).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your best bet is to encode some constraints, such as a minimum/maximum range for each value, and perhaps some correspondences, such as a maximum weight for a certain age range. This you can do with simple conditional statements in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then flag up those entries which don't look quite right. What you do with them depends on your application. You can filter them out, or manually change the values to something that is consistent with possible values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine Learning techniques would not be able to do anything better than that.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-10T18:22:10.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17952" PostTypeId="1" AcceptedAnswerId="17985" CreationDate="2020-02-10T18:26:37.290" Score="0" ViewCount="27" Body="&lt;p&gt;Given a pre-trained CNN model, I extract feature vector of &lt;em&gt;3450&lt;/em&gt; reference images &lt;code&gt;FV_R&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FV_R = [       [-8.2, -52.2, 9.07, -1.1, -0.08, -9.1, ........, -4.11], &#xA;               [7.8, -3.8, 6.4, -4.27, -2.2, -5.0, ............., 3.6], &#xA;               [-1.2, -0.8, 49.3, 1.73, -1.74, -7.1, ..........., 2.41],&#xA;               [-1.2, -.8, 49.3, 0.6, -1.24, -1.04, .........., -2.06],&#xA;               .&#xA;               .&#xA;               .&#xA;               [-1.2, -.8, 49.3, 12.77. -2.2, -5.0, .........., -51.1]&#xA;       ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;FV_Q&lt;/code&gt; for &lt;em&gt;1200&lt;/em&gt; query images : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FV_Q = [       [-0.13, 2.6, -3.7, -0.5, -1.02, -0.6, ........, -0.11], &#xA;               [0.3, -3.8, 6.4, -1.6, -2.2, -5.0, ............., 0.97], &#xA;               [-6.4, -0.08, 8.0, 7.3, -8.07, -5.6, ..........., 0.01],&#xA;               [-6.09, -.8, 0.5, -8.9, -0.74, -0.08, .........., -8.9],&#xA;               .&#xA;               .&#xA;               .&#xA;               [-1.2, -.8, 49.3, 12.77. -2.2, -5.0, .........., -51.1]&#xA;       ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The size info:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; FV_R.shape&#xA;(3450, 64896)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Query images:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; FV_Q.shape&#xA;(1200, 64896)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to &lt;em&gt;binarize&lt;/em&gt; the CNN feature vectors (descriptors) and calculate Hamming Distance. I am already aware of &lt;a href=&quot;https://stackoverflow.com/questions/32730202/fast-hamming-distance-computation-between-binary-numpy-arrays&quot;&gt;this&lt;/a&gt; answer to probably use &lt;code&gt;np.count_nonzero(a!=b)&lt;/code&gt;(&lt;code&gt;if a.shape == b.shape&lt;/code&gt;) but does anyone know a  method to binarize a feature vector with different size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;" OwnerUserId="31312" LastActivityDate="2020-02-12T13:52:54.380" Title="Binarize ConvNet Feature vector" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;feature-selection&gt;" AnswerCount="2" CommentCount="1" ClosedDate="2020-06-02T22:16:40.860" ContentLicense="CC BY-SA 4.0" />
  <row Id="17953" PostTypeId="1" AcceptedAnswerId="17962" CreationDate="2020-02-10T21:01:39.500" Score="0" ViewCount="72" Body="&lt;p&gt;I would like to build an LSTM to predict the correct words order given a sentence. My dataset is composed of sentences, where each sentence has a variable number of words (each word is embedded). The dataset then is an array of matrices, where each matrix is an array of embedded words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'm looking to implement it with Keras but I'm not sure how to fit the necessary parameters wanted by the LSTM layer in Keras, like &lt;code&gt;timesteps&lt;/code&gt; and &lt;code&gt;batch_size&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading on the web, I notice that &lt;code&gt;timesteps&lt;/code&gt; is the length of the sequence, so in my case I believe that corresponds to the length of the sentence. But I want to train my LSTM with one sentence at a time, so would the &lt;code&gt;batch_size&lt;/code&gt; be 1?&lt;/p&gt;&#xA;" OwnerUserId="33440" LastEditorUserId="26726" LastEditDate="2020-02-11T11:46:02.797" LastActivityDate="2020-02-11T11:46:02.797" Title="LSTM implementation in KERAS" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2020-05-02T22:14:53.323" ContentLicense="CC BY-SA 4.0" />
  <row Id="17954" PostTypeId="2" ParentId="17952" CreationDate="2020-02-10T21:07:08.097" Score="0" Body="&lt;p&gt;If you mean to binarize the vector such that all positive values become 1 and the rest becoming 0, then you can do this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bin_arr = np.zeros_like(FV_R)&#xA;bin_arr[FV_R &amp;gt; 0] = 1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [7]: arr                                                                                                                                                                                                        &#xA;Out[7]: &#xA;array([[ 0.15, -0.52],&#xA;       [ 1.  , -0.43]])&#xA;&#xA;In [8]: bin_arr = np.zeros_like(arr)                                                                                                                                                                               &#xA;&#xA;In [9]: bin_arr[arr &amp;gt; 0] = 1                                                                                                                                                                                       &#xA;&#xA;In [10]: bin_arr                                                                                                                                                                                                   &#xA;Out[10]: &#xA;array([[1., 0.],&#xA;       [1., 0.]])&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19077" LastActivityDate="2020-02-10T21:07:08.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17955" PostTypeId="1" CreationDate="2020-02-10T22:43:40.260" Score="1" ViewCount="47" Body="&lt;p&gt;I am referring specifically to the disc defined by Kuznetsov and Mohri in &lt;a href=&quot;https://arxiv.org/pdf/1803.05814.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1803.05814.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TVtez.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TVtez.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a kind of worst case path dependent generalization error. But what is the intuitive way of seeing why a &lt;em&gt;worst&lt;/em&gt; case is needed? I am probably missing something or reading something incorrectly. &lt;/p&gt;&#xA;" OwnerUserId="23001" LastEditorUserId="2444" LastEditDate="2020-02-10T23:36:00.353" LastActivityDate="2020-02-11T12:18:07.253" Title="Why does the discrepancy measure involve a supremum over the hypothesis space?" Tags="&lt;computational-learning-theory&gt;&lt;generalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17956" PostTypeId="1" AcceptedAnswerId="17957" CreationDate="2020-02-10T23:34:21.753" Score="1" ViewCount="63" Body="&lt;p&gt;Are there names for neural networks with a well-defined layer or neuron characteristics? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a matrix that has the same number of rows and columns is called a &lt;em&gt;square&lt;/em&gt; matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an equivalent for classifying different neural network structures. Specifically, I am interested if there is a name for a neural network with x number of layers, but each layer has the same number of neurons?&lt;/p&gt;&#xA;" OwnerUserId="32400" LastEditorUserId="2444" LastEditDate="2020-02-11T12:20:08.507" LastActivityDate="2020-02-11T12:20:08.507" Title="Are there names for neural networks with a well-defined layer or neuron characteristics?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;terminology&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17957" PostTypeId="2" ParentId="17956" CreationDate="2020-02-11T00:35:48.343" Score="2" Body="&lt;p&gt;Neural networks (NNs) are usually classified into feed-forward (i.e. NNs with feedforward connections), recurrent (i.e. NNs with recurrent connections) and convolutional (i.e. NNs that perform a convolution or cross-correlation operation). The term multi-layer NN may also be used to refer to feed-forward neural networks or, in general, neural networks with multiple (hidden) layers. There are also the perceptrons, which do not have hidden layers (i.e. the inputs are directly connected to the outputs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may still divide neural networks into classifiers (i.e. the outputs and labels are discrete) and regressors (the outputs and labels are numerical). Furthermore, you may classify them into generative models (e.g. the VAE) or discriminative models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By analogy with linear algebra concepts, each layer of a feedforward neural network (FFNN) can be seen as a linear operation followed by an element-wise application of a linear or non-linear activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{o}^{l} = \sigma \left(\mathbf{a}^{l} \mathbf{W}^{l} + \mathbf{b}^{l}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the activation function, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}^{l}$&lt;/span&gt; the inputs to the layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^{l}$&lt;/span&gt; the parameters of the layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; (similar to the parameters or coefficients in linear regression) and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}^{l}$&lt;/span&gt; is the bias vector (a scalar bias for each neuron) of layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{o}^{l}$&lt;/span&gt; will then be &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}^{l+1}$&lt;/span&gt; (i.e. the input to the next layer). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A recurrent neural network (RNN) performs a slightly more complex operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{o}^{l}_t = \sigma \left(\mathbf{a}^{l}_t \mathbf{W}^{l} + \mathbf{o}^{l}_{t-1} \mathbf{R}^{l} + \mathbf{b}^{l}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, in addition to the matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^{l}$&lt;/span&gt; associated with the feedforward connections, it also uses another matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{R}^{l}$&lt;/span&gt; associated with the recurrent connections (i.e. cyclic or loopy connections of the neurons), which is multiplied by the output of the same layer but at the previous time step. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{o}^{l}_t$&lt;/span&gt; may actually just be the state of the recurrent layer, which is then used to compute the actual output of the layer, but, for simplicity, you can ignore this. Furthermore, note that there are more complex recurrent architectures, but this is the basic idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A convolutional neural network (CNN) performs the convolution (or cross-correlation) operation. If you are familiar with signal processing, e.g. kernels, convolution, etc., then you can view a CNN as performing a convolution (or cross-correlation) operation. It's actually possible to view the &lt;a href=&quot;https://stackoverflow.com/q/16798888/3924118&quot;&gt;convolution as a matrix multiplication operation&lt;/a&gt;, but the details can easily become cumbersome and tedious to explain in an answer. A CNN may also perform other types of operations (such as downsampling) and it can also be composed of a feedforward part (usually, the last layers of a CNN are feedforward layers), but a CNN is a CNN because it performs the convolution (or cross-correlation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all cases, the matrices do not necessarily take any particular form (e.g. they are not necessarily square matrices). The dimensionality of the matrices depends on the number of layers and connections in the network, which can vary depending on many factors (e.g. the need for recurrent connections because they are useful for sequence modeling). These matrices (along with the biases) are the learnable parameters of the networks, but the learned values of these matrices highly depend on your data, the way you initialize them before learning, the architecture of the network, the learning algorithm, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the FFNN, if the previous layer &lt;span class=&quot;math-container&quot;&gt;$l-1$&lt;/span&gt; has the same number of neurons as the current layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^{l}$&lt;/span&gt; is a square matrix. AFAIK, there is no name for a neural network with the same number of neurons for each layer. It could be called a rectangular neural network (but this is a name I've just come up with).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, there are many different neural networks and taxonomies for neural networks, so it is impossible to list or discuss them all in an answer, but, nowadays, the subdivision into feedforward, recurrent and convolution is the most common and general. See e.g. the paper &lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;amp;arnumber=8807370&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Taxonomy for Neural Memory Networks&lt;/a&gt; if you are interested in a more detailed taxonomy for memory-based neural networks (e.g. recurrent neural networks). &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-11T12:12:40.437" LastActivityDate="2020-02-11T12:12:40.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17958" PostTypeId="2" ParentId="17955" CreationDate="2020-02-11T01:19:16.960" Score="1" Body="&lt;p&gt;The formula &lt;span class=&quot;math-container&quot;&gt;$G=\mathbb{E}\left[ f(Z_{T+1}) \mid \mathbf{Z}_1^T\right] - \sum_{t=1}^Tq_t \mathbb{E}\left[ f(Z_t) \mid \mathbf{Z}_1^{t-1} \right]$&lt;/span&gt; actually represents a set, for all possible values of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. Therefore, &lt;span class=&quot;math-container&quot;&gt;$\text{disc}(\mathbf{q}) = \operatorname{sup}_{f \in \mathcal{F}} \left( \mathbb{E}\left[ f(Z_{T+1}) \mid \mathbf{Z}_1^T\right] - \sum_{t=1}^Tq_t \mathbb{E}\left[ f(Z_t) \mid \mathbf{Z}_1^{t-1} \right] \right)$&lt;/span&gt; is the element &lt;strong&gt;not&lt;/strong&gt; necessarily in that set &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; that is greater than all elements in that set &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, but is the smaller than any other element that is greater than any element in that set &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. In other words, &lt;span class=&quot;math-container&quot;&gt;$\text{disc}(\mathbf{q})$&lt;/span&gt; represents an upper bound on the discrepancy &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, but it is the smallest possible upper bound. See also &lt;a href=&quot;https://math.stackexchange.com/a/166661/168764&quot;&gt;this answer&lt;/a&gt; for more details about the supremum and the relationship between the supremum and upper bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, unless I am wrong, this is not necessarily a worst-case analysis (but I am not even sure how one would define the worst-case analysis in this context), but we are looking for the smallest upper bound. In the context of algorithms, the worst-case analysis is with respect to the input (i.e. the worst-case input), but you can have upper or lower bounds for worst and best-case scenarios. See, for example, &lt;a href=&quot;https://stackoverflow.com/a/7485213/3924118&quot;&gt;this answer&lt;/a&gt; that illustrates this. Why do we want an upper bound? Because we can be certain that we won't have a generalization error worse than it. You can't do this with e.g. a lower bound.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-11T12:18:07.253" LastActivityDate="2020-02-11T12:18:07.253" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17959" PostTypeId="1" CreationDate="2020-02-11T01:53:05.290" Score="1" ViewCount="35" Body="&lt;p&gt;I am looking for specific references describing guidance principles around the interplay between IP (intellectual property) and Artificial Intelligence algorithms. For example, Company A has a large dataset and Company B has advanced algorithmic capabilities (assume near-AI). How might Company A protect itself in a joint partnership with company B? Apologies if AI SE is not the place for this - any suggestions for the right site?&lt;/p&gt;&#xA;" OwnerUserId="33442" LastActivityDate="2020-02-11T01:53:05.290" Title="Intellectual property in the age of Industry 4.0" Tags="&lt;learning-algorithms&gt;&lt;legal&gt;&lt;business&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17961" PostTypeId="1" CreationDate="2020-02-11T05:53:42.673" Score="0" ViewCount="29" Body="&lt;p&gt;I have a data set with a positive bias (an image, where the values range from 0 to 1), that seems to be causing my network to calculate incorrect gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I just use the raw image as input, of shape &lt;code&gt;(1,28,28)&lt;/code&gt;, gradient checking fails  on the convolution layers, producing the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv0: 1.1387914962261193e-06 1.4299464517486544     77.73286194827031&#xA;conv1: 7.635784666538102e-06  0.13917264577152977    2.5142484667483797e-06&#xA;conv2: 5.935327486981425e-07  2.706137838441397      1.6290818256837689e-06&#xA;fc0:   6.053556079013436e-07  1.1083181748299355e-05 2.0993768208698443e-06&#xA;fc1:   2.5819794868905293e-08 4.499689509732119e-06  1.108312270578618e-05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the first value is the absolute difference between the calculated weight updates, and the gradient tested weight updates, second being bias', and third being the calculated gradient to be passed to the previous layer. If this value is less than 0.1, it's a good indication the gradients are correct, as I use an epsilon of &lt;code&gt;1e-5&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strangely if I use a randomly initialised input: &lt;code&gt;inpt = np.random.randn(*im.shape))&lt;/code&gt; using the same seed as above for network initialisation, then I get the following differences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv0: 0.002403805867625952   1.1140792221336904e-05 0.0004023637406479723&#xA;conv1: 0.0011835450640199086  6.134154765691235e-06  0.00014655404401104377&#xA;conv2: 0.00021913017743990792 0.0032477812666108496  7.169014825140573e-05&#xA;fc0:   0.04510183914723059    6.425385741260989e-06  1.709243452626911e-05&#xA;fc1:   0.0014606421065230097  1.9235668613026775e-06 5.995613024721222e-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Would anyone have any idea what be causing this? I'm truly at a loss here, as I do the same procedure regardless of the numbers in the input, using the following to find the gradients to be passed back to the previous layer (for &lt;code&gt;conv&lt;/code&gt; layers):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = np.pad(grad, ((0,0), (self.xpad[1],self.xpad[0]), (self.ypad[1],self.ypad[0])))&#xA;self.gradient = np.zeros(self.incoming.output.shape)&#xA;for j in range(self.gradient.shape[0]):&#xA;    for i in range(t.shape[0]):&#xA;        self.gradient[j] += signal.convolve(t[i], self.kernels[i,j], mode='valid')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(This would be &lt;code&gt;signal.correlate&lt;/code&gt; if I was using a convolution on the forward pass, but I'm not. Important to note that a convolutional neural network actually uses correlation on the forward pass, and only convolves on the backward)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done some more testing and found that if I change epsilon even slightly, I get MASSIVELY different results for the tested gradient. It seems decreasing epsilon increases the magnitude of the gradients exponentially. This is very strange behaviour, and I'm really not sure what is causing it. Could it be a discontinuity in ReLU where somewhere along the way I happen upon exactly 0?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="26726" LastEditDate="2020-02-13T01:43:25.827" LastActivityDate="2021-03-09T07:06:05.513" Title="Positive bias causes the calculation of incorrect gradients" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17962" PostTypeId="2" ParentId="17953" CreationDate="2020-02-11T10:42:00.443" Score="0" Body="&lt;p&gt;As you read, in keras the input dimensions for a LSTM layer are:  &lt;code&gt;(batch_size, timesteps, input_dim)&lt;/code&gt;. Where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;batch_size&lt;/code&gt;: number of samples (sentences in your case) to compute the loss before running the gradient descent. So number of sentences to train on before compute loss and optimize your model.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;timesteps&lt;/code&gt;: length of the sequence, in your case length of the sentence&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;input_dim&lt;/code&gt;: the features of your data (the words)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The nice thing about Keras is that you can train with an specific batch size, say &lt;code&gt;batch_size=16&lt;/code&gt;, that: helps the model convergence (because it averages the loss of the 16 sentences prediction) and boosts the speed (since the weights are updated only once every 16 sentences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then you can infer (or do the predictions) with &lt;code&gt;batch_size=1&lt;/code&gt;, meaning, one sentence at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you want to train, specifically with one sentence at a time: &lt;code&gt;batch_size=1&lt;/code&gt;. But if you want to take the advantage of using mini-batch then use &lt;code&gt;batch_size=16&lt;/code&gt; or higher for training and &lt;code&gt;batch_size=1&lt;/code&gt; for inference.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;BONUS: Keras allows for dynamic batch size change in the models. When a dimension is dynamic in Keras has the value of &lt;code&gt;None&lt;/code&gt;, that is why when you do &lt;code&gt;model.summary()&lt;/code&gt; you can see: &lt;code&gt;(batch_size, timesteps, input_dim) = (None, 40, 100)&lt;/code&gt;. This &lt;code&gt;None&lt;/code&gt; allows for different &lt;code&gt;batch_size&lt;/code&gt; in training and inference&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-02-11T10:42:00.443" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17963" PostTypeId="1" CreationDate="2020-02-11T14:48:43.060" Score="2" ViewCount="221" Body="&lt;p&gt;There is &lt;a href=&quot;https://pythonprogramming.net/introduction-deep-learning-python-tensorflow-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video on pythonprogramming.net&lt;/a&gt; that trains a network on the MNIST handwriting dataset. At &lt;a href=&quot;https://youtu.be/wQ8BIBpya2k?t=539&quot; rel=&quot;nofollow noreferrer&quot;&gt;~9:15&lt;/a&gt;, the author explains that the data should be normalized. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The normalization is done with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_train = tf.keras.utils.normalize(x_train, axis=1)&#xA;x_test = tf.keras.utils.normalize(x_test, axis=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The explanation is that values in a range of 0 ... 1 make it easier for a network to learn. That might make sense, if we consider sigmoid functions, which would otherwise map almost all values to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could also understand that we want black to be pure black, so we want to adjust any offset in black values. Also, we want white to be pure white and potentially stretch the data to reach the upper limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think the kind of normalization applied in this case is incorrect. The image before was:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pjAzC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pjAzC.png&quot; alt=&quot;Original image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the normalization it is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Yv01.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Yv01.png&quot; alt=&quot;Normalized image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we can see, some pixels which were black before have become grey now. Columns with few black pixels before result in black pixels. Columns with many black pixels before result in lighter grey pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be confirmed by applying the normalization on a different axis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iMtFo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iMtFo.png&quot; alt=&quot;Normalization applied to a different axis&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, rows with few black pixels before result in black pixels. Rows with many black pixels result in lighter grey pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is normalization used the right way in this tutorial? If so, why? If not, would my normalization be correct?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I expected was a per pixel mapping from e.g. [3 ... 253] (RGB values) to [0.0 ... 1.0]. In Python code, I think this should do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import imageio&#xA;image = imageio.imread(&quot;sample.png&quot;)&#xA;image = (image - np.min(image))/np.ptp(image)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31627" LastActivityDate="2021-03-02T23:03:15.447" Title="Does this tutorial use normalization the right way?" Tags="&lt;python&gt;&lt;keras&gt;&lt;batch-normalization&gt;&lt;handwritten-characters&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17964" PostTypeId="2" ParentId="17950" CreationDate="2020-02-11T14:53:58.440" Score="1" Body="&lt;p&gt;In fact, choosing smartly the values of the image augmentation can help the performance of your system. Where I work we developed an object detector for cars. We had the following image augmentation parameters:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Apect ratio distorsion: it changed the cars dimensions&lt;/li&gt;&#xA;&lt;li&gt;Additive noise: it blurred the image&lt;/li&gt;&#xA;&lt;li&gt;Change colorspace: change the cars colors&lt;/li&gt;&#xA;&lt;li&gt;Saturation change: brightness&lt;/li&gt;&#xA;&lt;li&gt;Rotation: rotate image axis&lt;/li&gt;&#xA;&lt;li&gt;Flipping (horizontally): mirror effect&lt;/li&gt;&#xA;&lt;li&gt;Zooming: zoom in/out of image&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;At first we just combined all the effects (within a certain range of change per effect) randomly in each augmented image. But then we did an study for our use case and we found that our model performed better with: &lt;strong&gt;no aspect ratio deformation&lt;/strong&gt; and &lt;strong&gt;no change in color space&lt;/strong&gt;. After giving it a bit of a thought it really made sense for our use case because it is a detector for a very specific type of car. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, although the improvements accomplished by optimizing the parameters of image augmentation boosted the performance of the model, the improvements were not huge, so in the end it did not really worth the effort. To give some numbers it improved &lt;span class=&quot;math-container&quot;&gt;$5\%$&lt;/span&gt; in our custom test metrics, and we were expecting something around &lt;span class=&quot;math-container&quot;&gt;$10\%$&lt;/span&gt; so from that moment on we left the specific image augmentation parameters but stopped thinking too much about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in conclussion, does it helps? yes, how much? well, that would depend on the use case. For us, it turned out it did not improve as much as we wanted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps :)&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-02-11T14:53:58.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17965" PostTypeId="1" AcceptedAnswerId="17967" CreationDate="2020-02-11T16:20:54.547" Score="2" ViewCount="70" Body="&lt;p&gt;I am trying to understand the different reward functions modelled in a reinforcement learning problem. I want to be able to know how the temporal credit assignment problem, (where the reward is observed only after many sequences of actions, and hence no immediate rewards observed) can be mitigated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From reading the DQN paper, I am not able to sieve out how the immediate rewards are being modelled when &lt;span class=&quot;math-container&quot;&gt;$Q_{target}(s,a; \theta) = r_s + argmax_aQ(s',a'; \theta)$&lt;/span&gt;. What is &lt;span class=&quot;math-container&quot;&gt;$r_s $&lt;/span&gt; used in the case where the score has not changed ? Therefore what is the immediate rewards being modelled for temporal credit assignment problems in atari game ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$r_s$&lt;/span&gt; is indeed 0 until score changes, would it affect the accuracy of the DQN ? it seems like the update equation would not be accurate if you do not even know what is the immediate reward if you take that action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some of the current methods used to solve the temporal credit assignment problem ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I can't seem to find many papers that address the temporal credit assignment problem&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-02-11T17:02:35.337" LastActivityDate="2020-02-11T20:52:06.233" Title="Immediate reward received in Atari game using DQN" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17966" PostTypeId="1" AcceptedAnswerId="17970" CreationDate="2020-02-11T19:35:52.137" Score="1" ViewCount="71" Body="&lt;p&gt;I often read about the so-called end-to-end AI (or analytics) projects, but I couldn't find a definition of it. What is an end-to-end AI project?  Can someone explain what is meant/expected when someone asks you &quot;Have you already implemented an end-to-end AI project&quot;?&lt;/p&gt;&#xA;" OwnerUserId="33465" LastEditorUserId="2444" LastEditDate="2020-02-13T02:04:48.430" LastActivityDate="2020-02-13T02:04:48.430" Title="What is an end-to-end AI project?" Tags="&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17967" PostTypeId="2" ParentId="17965" CreationDate="2020-02-11T20:20:22.417" Score="2" Body="&lt;p&gt;To answer your questions in order:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is &lt;span class=&quot;math-container&quot;&gt;$r_s $&lt;/span&gt; used in the case where the score has not changed ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Therefore what is the immediate rewards being modelled for temporal credit assignment problems in atari game ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Rewards &lt;em&gt;can&lt;/em&gt; be re-modelled to aid speed of learning. This is called &quot;&lt;a href=&quot;https://pdfs.semanticscholar.org/41c5/0331ed5d3ffa51d879cc2e1a675c99445bc3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reward shaping&lt;/a&gt;&quot;, and is typically done by domain experts who can adjust numbers to reward known good intermediate states and actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For DQN Atari, this was not done. Instead, the researchers performed a reward normalisation/scaling so that games which used moderate scoring system in single digits could be handled by the same neural network approximator as games that handed out thousands of points at a go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using sparse rewards is standard practice in reinforcement learning, and the credit assignment problem is solved to some degree by &lt;em&gt;all&lt;/em&gt; reinforcement learning methods. Essentially the value functions work as a prediction mechanism theoretically whatever the reward sparsity, so if they are correct, thay can be used to drive policy whether the next reward is 1, 50 or 1000 time steps away. The reward &lt;em&gt;backup&lt;/em&gt; updates in everything from Value Iteration, through Monte Carlo Control, SARSA, Q-Learning and Actor-Critic all backup values to states/actions seen in earlier time steps. This value backup is a basic mechanism that addresses credit assignment in principle. The credit assignment problem is then a matter of degree and difficulty of different environments, such that sometimes it is readily solved, and other times it is a major hurdle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of video games, especially older arcade games, it is often not a very hard part of the problem. The games are designed to reward human players by incrementing scores frequently, with very many sub-goals already within the game. In fact this is one of the attractions of video games as toy environments for developing new algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example the classic space invaders does not simply score +1 for surviving a wave of enemies, but adds points for every player missile that hits. Although the score does not increment on every frame, the reward sparsity is relatively low for games like this, and simple single-step Q learning with experience replay can solve the credit assignment problem readily for that environment (experience replay does help a little with credit assignment). This was what was demonstrated with the original DQN Atari paper, there were no extra allowances made for reward sparsity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$r_s$&lt;/span&gt; is indeed 0 until score changes, would it affect the accuracy of the DQN ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not directly, the DQN predicts future expected rewards and can in principle account for delay and sparsity in its estimates. However, if this becomes very sparse you get two problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Discovering the positive rewards within the environment may take a long time, and may require more advanced techniques, such as reward shaping to encourage searching behaviour (a small negative reward per time step) or &quot;&lt;a href=&quot;https://towardsdatascience.com/curiosity-driven-learning-made-easy-part-i-d3e5a2263359&quot; rel=&quot;nofollow noreferrer&quot;&gt;curiousity&lt;/a&gt;&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Credit assignment becomes much harder as the possible number of combinations that &lt;em&gt;could&lt;/em&gt; of contributed to success can grow exponentially with temporal distance between rewards. Resolving whch ones are important, especially early in a trajectory leading to reward, can take many samples.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are some of the current methods used to solve the temporal credit assignment problem ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As noted above, this is core problem in RL, so there are many approaches in the literature. Some standard approaches are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Background planning as used by DynaQ, or experience replay. This re-evaluates states and actions seen before whilst using latest estimates, and can backup values to where important decisions are made within a trajectory. &lt;a href=&quot;https://arxiv.org/abs/1511.05952&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prioritised experience replay&lt;/a&gt; helps even more by focusing on updates that make the most difference to the current estimates.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) with &lt;a href=&quot;https://towardsdatascience.com/eligibility-traces-in-reinforcement-learning-a6b458c019d6&quot; rel=&quot;nofollow noreferrer&quot;&gt;eligibility traces&lt;/a&gt;. Eligibility traces are intuitively a credit-assignment mechanism, they track state features that were active recently and multiply value updates based on the trace vector. Again, this causes state or state action values to backup to earlier parts of trajectories faster.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reward shaping as discussed above. In some research settings this might be seen as &quot;cheating&quot; - for instance when facing a standardised environment test for developing algorithms, adding in domain knowledge to help the agent just demonstrates the core algorithm is weaker than claimed. However, when the challenge presented is to solve an environment with the best agent possible, it is fine to use any knoweldge (or of course not to use RL at all).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-02-11T20:52:06.233" LastActivityDate="2020-02-11T20:52:06.233" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17968" PostTypeId="1" CreationDate="2020-02-11T20:44:56.113" Score="0" ViewCount="50" Body="&lt;p&gt;I am developing a CNN model to recognize 24 hand-signs of American Sign Language. I have 2500 Images/hand-sign. The data split is:&lt;br /&gt;&#xA;Training = 1250 Images/hand-sign&lt;br /&gt;&#xA;Validation = 625 Images/hand-sign&lt;br /&gt;&#xA;Testing = 625 Images/hand-sign&lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I proceed with training the model?:&lt;br /&gt;&#xA;1. Should I develop a model starting from fewer hand-signs (like 5) and then increase them gradually?&lt;br /&gt;&#xA;2. Should I start models from scratch or use transfer learning (VGG16 or other)&lt;br /&gt;&#xA;Applying data augmentation, I did some tests with VGG16 and added a dense classifier at the end and received these accuracies:&lt;br /&gt;&#xA;Train: 0.87610877&lt;br /&gt;&#xA;Validation: 0.8867307&lt;br /&gt;&#xA;Test: 0.96533334&lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lq11e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accuracy and Loss Graph&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test parameters:&lt;br /&gt;&#xA;NUM_CLASSES = 5&lt;br /&gt;&#xA;EPOCHS = 50&lt;br /&gt;&#xA;STEPS_PER_EPOCH = 125&lt;br /&gt;&#xA;VALIDATION_STEPS = 75&lt;br /&gt;&#xA;TEST_STEPS = 75&lt;br /&gt;&#xA;Framework = Keras, Tensorflow&lt;br /&gt;&#xA;OPTIMIZER = adam&lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential([&#xA;    Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;&#xA;    Conv2D(64, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;&#xA;    Conv2D(128, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;&#xA;    Conv2D(256, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;&#xA;    Conv2D(512, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;&#xA;    Flatten(),&#xA;    Dense(512, activation='relu'),&#xA;&#xA;    Dense(NUM_CLASSES, activation='softmax')&#xA;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I try images with slightly different background and predict the classes (predict_classes()), I do not get accurate results. Any suggestions on how to make the model robust?&lt;/p&gt;&#xA;" OwnerUserId="33467" LastActivityDate="2021-03-08T15:05:50.553" Title="Hand-Signs Recognition using Deep Learning Convolutional Neural Networks" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17969" PostTypeId="1" CreationDate="2020-02-11T21:56:52.050" Score="1" ViewCount="131" Body="&lt;p&gt;I'm suffering from a significant brain fart while trying to get my head around how does batch size affect overall model size e.g for CNNs. Does it serve as an additional dimension for all the weight tensors? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;VGG16 model &lt;/li&gt;&#xA;&lt;li&gt;batch_size of 16 &lt;/li&gt;&#xA;&lt;li&gt;image size of 224x224x3&lt;/li&gt;&#xA;&lt;li&gt;conv_1 being the initial 1x1 convolution with stride 1 and 3:64 channels mapping&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The input will be a tensor of &lt;code&gt;[16, 224, 224, 3]&lt;/code&gt; shape. Will the output of convolution layer be &lt;code&gt;[16, 224, 224, 64]&lt;/code&gt; and therefore - will all the weights have additional 'batch size' dimension and thus - impose a linear increase of model size with respect to the batch size?&lt;/p&gt;&#xA;" OwnerUserId="32431" LastEditorUserId="2444" LastEditDate="2020-02-13T02:11:34.433" LastActivityDate="2020-02-13T02:11:34.433" Title="How does batch size affect model size?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17970" PostTypeId="2" ParentId="17966" CreationDate="2020-02-11T22:19:14.083" Score="1" Body="&lt;p&gt;It's a pretty grey area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think they are usually talking about the full spectrum of the effort in taking an &quot;AI project&quot; all the way from the drawing board to production. This could easily be anything from a Stock Market Price Prediction model, to deploying a customer service chatbot/Virtual Assistant. I find that people who ask such questions only care about the fact that &quot;AI&quot; is being used to solve a problem and actually is working in the real world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The effort in the Model would be the standard DataScience/Machine learning process of obtaining data, feature engineering, training and refining your model and eventually deploying it somewhere where it works. It could be in a web browser, it could be in an app, an API endpoint, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the customer service chatbot it could easily be largely a pure programming project utilising a cloud provider's resource for to conversational capabilities, but as long as it falls under the bracket of &quot;AI&quot; and has been completed and deployed to production/an environment where it works, its end to end. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; Depending on the technical knowledge of who is asking, they might care that you have done some real machine learning work in the development process, and so the example of the bot wouldn't fly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps somehow.&lt;/p&gt;&#xA;" OwnerUserId="31023" LastActivityDate="2020-02-11T22:19:14.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17971" PostTypeId="1" AcceptedAnswerId="17981" CreationDate="2020-02-11T22:36:29.693" Score="2" ViewCount="1322" Body="&lt;p&gt;I am reading through Artificial Intelligence: Modern Approach and it states that the space complexity of the GBFS (tree version) is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I am reading, at some points, I found GBFS similar to DFS. It expands the whole branches and goes after one according to the heuristic function. It doesn't expand the rest like BFS. Perceiving this as similar to what depth-first search does, I understand that the worst time complexity is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;. But I don't understand the space complexity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shouldn't it be the same as DFS, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(bm)$&lt;/span&gt;, since it only will be expanding &lt;span class=&quot;math-container&quot;&gt;$b*m$&lt;/span&gt; nodes during the search in one path?&lt;/p&gt;&#xA;" OwnerUserId="31006" LastEditorUserId="2444" LastEditDate="2020-02-11T23:22:27.323" LastActivityDate="2020-11-14T16:50:50.197" Title="Why is the space-complexity of greedy best-first search is $\mathcal{O}(b^m)$?" Tags="&lt;search&gt;&lt;time-complexity&gt;&lt;computational-complexity&gt;&lt;best-first-search&gt;&lt;space-complexity&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17972" PostTypeId="5" CreationDate="2020-02-11T23:25:39.297" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-11T23:25:39.297" LastActivityDate="2020-02-11T23:25:39.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17973" PostTypeId="4" CreationDate="2020-02-11T23:25:39.297" Score="0" Body="For questions related to the space complexity analysis of artificial intelligence algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-11T23:25:39.297" LastActivityDate="2020-02-11T23:25:39.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17975" PostTypeId="1" AcceptedAnswerId="26955" CreationDate="2020-02-12T01:00:23.320" Score="3" ViewCount="258" Body="&lt;p&gt;I trained a ResNet20 on Cifar10 and obtained the following learning curves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O7JVh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O7JVh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the figures, I see at epoch 52, my validation loss is 0.323 (the lowest), and my validation accuracy is 89.7%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, at the end of the training (epoch 120), my validation loss is 0.413 and my validation accuracy is 91.3% (the highest).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I'd like to deploy this model on some real-world application. Should I prefer the snapshotted model at epoch 52, the one with lowest validation loss, or the model obtained at the end of training, the one with highest validation accuracy?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="9947" LastEditDate="2020-02-12T01:45:45.063" LastActivityDate="2021-03-23T05:56:06.053" Title="Should I prefer the model with the lowest validation loss or the highest validation accuracy to deploy?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;applications&gt;&lt;overfitting&gt;&lt;early-stopping&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17976" PostTypeId="2" ParentId="17969" CreationDate="2020-02-12T01:40:55.223" Score="1" Body="&lt;p&gt;The model size (i.e., the number of parameters) is completely independent of the batch size you use during the training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever the batch size is, your model processes each input within the batch independently. Concretely, batch size affects how many samples your model has to process before it makes an update. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when batch size is 1, it computes the gradients and updates the parameters for each input. When batch size is the whole dataset, it computes the gradients for each input in the dataset independently. It then takes the average of those gradients and uses this average to update the model, so there will be a single update per epoch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the model takes an input of size (width=224, height=224, color_channels=3) whatever batch size you use during the training.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="32621" LastEditDate="2020-02-12T01:46:49.480" LastActivityDate="2020-02-12T01:46:49.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17977" PostTypeId="1" AcceptedAnswerId="17991" CreationDate="2020-02-12T01:53:54.020" Score="4" ViewCount="159" Body="&lt;p&gt;I've been using PyTorch to do research  for a while and it seems to be quite easy to implement new things with. Also, it is easy to learn and I didn't have any problem with following other researchers code so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wonder whether TensorFlow has any advantage over PyTorch. The only advantage I know is, it's slightly faster than PyTorch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, does TensorFlow have any concrete advantages over PyTorch apart from performance, in particular for research purposes?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="2444" LastEditDate="2020-02-13T02:08:13.713" LastActivityDate="2020-02-13T02:08:13.713" Title="Is there a reason to use TensorFlow over PyTorch for research purposes?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;comparison&gt;&lt;research&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17978" PostTypeId="1" CreationDate="2020-02-12T05:34:41.743" Score="0" ViewCount="26" Body="&lt;p&gt;I am trying to understand some of the different approaches used to overcome  sparse rewards in a reinforcement learning setting for a research project. Particularly, I have looked at curiosity driven learning, where an agent learns an intrinsic reward function based on the uncertainty of the next state that the agent will end up in as he takes action a in state s. The greater the uncertainty of the next state, the higher the rewards. This will incentive agent's to be more exploratory and it is used particularly in some games where a huge number of steps is needed before the agent reaches the terminal state where is he only then rewarded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The curiosity driven approach as demonstrated in this paper:&#xA;&lt;a href=&quot;https://pathak22.github.io/noreward-rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pathak22.github.io/noreward-rl/&lt;/a&gt; is able to learn faster than if a 0 rewards were used for each state, action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge, using different reward functions will affect the optimal policy obtained. Would curiosity driven learning therefore lead to a different policy as compared to whether a 0 reward was used ? Assume that for a 0 immediate reward system, it is able to derive a policy that reaches the goal state. Which of these 2 policies will be &lt;strong&gt;more&lt;/strong&gt; optimal ?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-02-12T05:34:41.743" Title="Curiosity Driven Learning affect optimal policy" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17979" PostTypeId="1" CreationDate="2020-02-12T09:18:12.440" Score="1" ViewCount="55" Body="&lt;p&gt;I want to build an application which takes a live audio from source (mic) and filtering the noise (unwanted sounds like chattering, traffic noises) and fetch into an application for further processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to apply Machine Learning Framework (TensorFlow, Keras) and Deep Learning neural Networks (i.e RNN) for filtering the noise from the audio. I want to do this in a real time environment. My inference device will be a Nvidia Jetson Device. Please guide me where I can find the related documents and how to proceed with the project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is any solution available in any website please refer the link.&lt;/p&gt;&#xA;" OwnerUserId="33477" LastEditorUserId="33372" LastEditDate="2020-02-13T08:52:14.350" LastActivityDate="2020-02-13T08:52:14.350" Title="Noise Cancellation on live audio stream" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17980" PostTypeId="1" CreationDate="2020-02-12T10:07:30.837" Score="2" ViewCount="37" Body="&lt;p&gt;I have been trying to implement &lt;a href=&quot;https://arxiv.org/pdf/1810.07842.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and I am very much intrigued. I am working on a medical image problem where I have to segment very small specimens on Whole Slide Images (gigapixel resolution). Therefore my dataset is highly unbalanced and I am having a high false positives rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did my research and found that paper that describes the implementation of Tversky Loss and Focal Tversky Loss. It also describes some modifications to the network architecture which I am postponing for now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented the loss (Pytorch) and ran some experiments with several alpha/beta combinations. Well, the results are easy to understand: higher alpha results in higher precision and a lower beta increases the recall and pushes the precision down. Basically, what this loss is doing is balancing my recall and precision, only. That is good, I can solve my False Positives issue but since this is a medical problem, a good recall is mandatory. In the paper, the results show that there is an improvement in the Precision/Recall and I cannot understand how is that possible and how I cannot replicate that. I am just weighing false positives and penalizing them, it does not seem enough to improve the model overall. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K5vee.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K5vee.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21237" LastActivityDate="2020-02-12T10:07:30.837" Title="Tversky Loss paper implementation: Recall/Precision do not improve as stated" Tags="&lt;loss-functions&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17981" PostTypeId="2" ParentId="17971" CreationDate="2020-02-12T12:51:18.023" Score="0" Body="&lt;p&gt;After spending some time on the problem, I concluded that it is due to the fact that we need to store the heuristic function evaluations for all nodes during the traversal. So, one might claim that it is the space complexity of the whole nodes which is simply &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^m)$&lt;/span&gt;. I hope this is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the one with having the space complexity of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(bm)$&lt;/span&gt; is called recursive best-first search which is the one that is most similar to DFS implementation I descried in the question.&lt;/p&gt;&#xA;" OwnerUserId="31006" LastEditorUserId="31006" LastEditDate="2020-02-12T13:58:16.207" LastActivityDate="2020-02-12T13:58:16.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17982" PostTypeId="2" ParentId="12059" CreationDate="2020-02-12T13:14:33.383" Score="1" Body="&lt;p&gt;OpenAI have a post on that: &lt;a href=&quot;https://openai.com/blog/openai-five/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/openai-five/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They use a myriad of &lt;em&gt;rollout workers&lt;/em&gt; that collect data for 60 seconds and push that data to a GPU cluster where gradients are computed for batches of 4096 observations which are then averaged.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PPO is actually designed to allow this kind of parallelisation as it uses &lt;em&gt;trajectory segments&lt;/em&gt; with a fixed size of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; to collect data, e.g. 60 seconds for OpenAI Five, where &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is supposed to be &quot;much less than the episode length&quot; (p.5 of &lt;a href=&quot;http://arxiv.org/pdf/1707.06347v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO paper&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="22161" LastActivityDate="2020-02-12T13:14:33.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17984" PostTypeId="2" ParentId="17968" CreationDate="2020-02-12T13:45:13.400" Score="0" Body="&lt;p&gt;I feel your problem might not be with the model itself but with the dataset. If you only have &lt;span class=&quot;math-container&quot;&gt;$2500$&lt;/span&gt; images for &lt;span class=&quot;math-container&quot;&gt;$24$&lt;/span&gt; labels (hand-signs) that gets you roughly &lt;span class=&quot;math-container&quot;&gt;$104$&lt;/span&gt; images per label. This is very little for the models I train (~&lt;span class=&quot;math-container&quot;&gt;$80K$&lt;/span&gt; images in the smallest of cases). In my view you got a really decent accuracy at validation and test time for the size of your dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But answering your questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Starting from few labels and extending is usually helpful when your model is too deep and suffers from convergence problems. Your model is simple enough not to suffer those problems so I would go learning the &lt;span class=&quot;math-container&quot;&gt;$24$&lt;/span&gt; labels at once.&lt;/li&gt;&#xA;&lt;li&gt;Transfer learning can help a lot at reducing training times. For example, if you start from a &lt;em&gt;VGG&lt;/em&gt; classifier that detects hands there is a good chance that the weights of your convolutional layers are already almost configured for your use case.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, the easiest way to increase the accuracy of your model is to use one of these 2 methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Increase the dataset: I am not sure if it is possible in your case, maybe you can use image augmentation (rotation, zooming in/out, changes in the color space...).&lt;/li&gt;&#xA;&lt;li&gt;Increase the depth of your model (provided that you have a big dataset).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you already fulfilled those items, then you can go and make changes to the model architecture or loss function. Looking at your model and from the top of my head I would try to add &lt;em&gt;Batch-Normalization&lt;/em&gt; for the convolutional layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps a bit :)&lt;/p&gt;&#xA;" OwnerUserId="26882" LastEditorUserId="26882" LastEditDate="2020-02-12T13:51:40.557" LastActivityDate="2020-02-12T13:51:40.557" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17985" PostTypeId="2" ParentId="17952" CreationDate="2020-02-12T13:52:54.380" Score="0" Body="&lt;p&gt;I fixed the problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def binarize(FV):&#xA;    return np.where(FV &amp;gt; 0, 1, 0).astype(int)&#xA;&#xA;def Hamming_Distance(ref, query):&#xA;    b_rf, b_qu = binarize(ref), binarize(query)&#xA;    H = np.count_nonzero(b_qu[:, None, :] != b_rf, axis=2)&#xA;    return H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31312" LastActivityDate="2020-02-12T13:52:54.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17986" PostTypeId="1" CreationDate="2020-02-12T13:53:13.983" Score="1" ViewCount="34" Body="&lt;p&gt;I have quite a naive question regarding Pytorch deep learning models and tabular data representation. So, assume I have a dictionary of tables. Each table has some number of columns: categorical and continuous. I encode categorical values via one-hot encoding. As a result, my tables have more columns than initially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is - how to train my deep learning model so that after encoding and decoding steps I will get back table representation with the original number of columns?&lt;/p&gt;&#xA;" OwnerUserId="33332" LastActivityDate="2020-02-12T13:53:13.983" Title="Pytorch deep learning models and tabular data representation" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17989" PostTypeId="1" CreationDate="2020-02-12T15:52:31.343" Score="1" ViewCount="72" Body="&lt;p&gt;I am trying to get my neural network running on my university's supercomputer in order to decrease its runtime (not for training, for testing - feedforward runs only).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the Matlab function I am using to run my network (&lt;code&gt;semanticseg&lt;/code&gt;) is running more slowly on one of the supercomputer's nVidia Volta V100 GPU's than on my laptop's intel(r) uhd graphics 620.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used the Matlab &lt;code&gt;profile&lt;/code&gt; function to make sure that it really is the &lt;code&gt;semanticseg&lt;/code&gt; function that's running more slowly (and not some other part of my code), and it really is, it takes about five times the laptop compute time to run on the supercomputer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any ideas why this might be? It seems totally illogical to me, and all my coworkers are stumped. Why would a neural network run more slowly on a more powerful GPU?&lt;/p&gt;&#xA;" OwnerUserId="9983" LastEditorUserId="2444" LastEditDate="2020-02-13T02:34:16.750" LastActivityDate="2020-02-13T02:34:16.750" Title="Why would my neural network run faster on my laptop than on my university's supercomputer?" Tags="&lt;convolutional-neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;matlab&gt;&lt;gpu&gt;&lt;computational-complexity&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17990" PostTypeId="2" ParentId="17975" CreationDate="2020-02-12T16:42:42.247" Score="0" Body="&lt;p&gt;In highly imbalanced classification problems, the highest accuracy can often be achieved simply by assigning the majority class to all observations. This is why learning algorithms do not maximize classification accuracy but minimize a loss function. &#xA;Fundamentally, loss functions capture how much you &quot;lose&quot; when there is a difference between the statistic you want to estimate and the estimate itself. The appropriate loss function is not given by nature, but is provided by you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, it is not quite accurate to say that the highest accuracy is reached after 120 epochs: it is merely the maximum accuracy achieved by the algorithm so far. Unless you run the algorithm for longer, there is no way to say if this is even a local maximum. For example, assigning every observation to the majority class may well achieve a higher accuracy than that achieved at 120 epochs. The only significance of the 120 epochs is therefore that that is how long you ran the algorithm for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given these considerations, it makes far more sense to stop at around 50 epochs, when your loss function is minimized.&lt;/p&gt;&#xA;" OwnerUserId="33479" LastActivityDate="2020-02-12T16:42:42.247" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17991" PostTypeId="2" ParentId="17977" CreationDate="2020-02-12T16:44:25.200" Score="4" Body="&lt;p&gt;In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my (but not only) opinion, TF 1 is really ugly and painful, given that it involves sessions, placeholders and, in general, you need to define the computational graph before executing anything (even the simplest programs). With TF 2, you don't need sessions and placeholders anymore, and this is a very big improvement because you don't need to think that you first need to define a computational graph and then feed it with the input. Essentially, in TF 2, you can write TF code and you can almost be sure that your TF 2 code will look like any other Python code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;TF and Keras have some bugs, but I think that also PyTorch must have some bugs (even though I don't remember having encountered them). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;TF comes with &lt;a href=&quot;https://www.tensorflow.org/tensorboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorBoard&lt;/a&gt;. You can also easily use TensorBoard from Keras. About one year ago, I had written a &lt;a href=&quot;https://nbro.github.io/blogging/2019/01/06/an-example-of-how-to-use-visualdl-with-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; about visualization tools for PyTorch. At the time, the tools didn't seem as developed as TensorBoard, but there were already a few options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Debugging in TF 2 isn't still perfect, but I would say that it has improved with respect to TF 1 (mainly because there is less boilerplate code). To debug in TF 2.0, I mainly use &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/print&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tf.print&lt;/code&gt;&lt;/a&gt; (which behaves similarly to Python's &lt;code&gt;print&lt;/code&gt; function), but there may be more advanced tools (e.g. I think there's a TF debugging plugin for PyCharm), even though I've never used them. TensorBoard could also be used as a debugging tool (for example, to understand how your models are learning, etc). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplicity of PyTorch is comparable to that of Keras. TF now comes with an integrated version of Keras, which is optimized for TF, so TF code can look as simple as Keras code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, my impression is that, if you need to, TF provides more flexibility, but it's been a while since I used PyTorch. Furthermore, with TF 2.0, the code looks simpler, more consistent and the framework is better organized. On the other hand, PyTorch may still be easier to learn (because it doesn't have the past of TF, i.e. there are still many TF code that uses sessions, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both TF and PyTorch are supported by big companies (respectively, Google and Facebook), but TF is more mature and there are functionalities/libraries that exist for TF that still do not exist for PyTorch (AFAIK). I doubt that there is a functionality for PyTorch that doesn't also exist for TF or Keras.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-12T16:50:08.983" LastActivityDate="2020-02-12T16:50:08.983" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17992" PostTypeId="1" CreationDate="2020-02-12T16:54:18.070" Score="2" ViewCount="94" Body="&lt;p&gt;I'm looking for an implementation that allows me to generate text based on a pre-trained model (e.g. GPT-2).&lt;/p&gt;&#xA;&lt;p&gt;An example would be &lt;a href=&quot;https://github.com/minimaxir/gpt-2-keyword-generation&quot; rel=&quot;nofollow noreferrer&quot;&gt;gpt-2-keyword-generation&lt;/a&gt; (&lt;a href=&quot;https://minimaxir.com/apps/gpt2-reddit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;click here for demo&lt;/a&gt;). As the author notes, there is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[...] no explicit mathematical/theoetical basis behind the keywords&#xA;aside from the typical debiasing of the text [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Hence my question: &lt;strong&gt;Are there more sophisticated ways of keyword-based text generation&lt;/strong&gt; or at least any other &lt;strong&gt;alternatives&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="33476" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-12T16:54:18.070" Title="Pretrained Models for Keyword-Based Text Generation" Tags="&lt;transformer&gt;&lt;gpt&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17993" PostTypeId="1" CreationDate="2020-02-12T16:56:33.807" Score="2" ViewCount="84" Body="&lt;p&gt;I am a newbie to machine learning. I have an LSTM model that predicts the next output n+1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time 1, params 1, output 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time 2, params 2, output 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time 3, params 3, output 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.&#xA;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time n, params n, , output n&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time n+1 --&gt; predicts output n+1&lt;/p&gt;&#xA;&#xA;&lt;p&gt; Here the times are all in minutes, so I can predict the next output in the series which is going to be the next minute. My question is that what if I want to predict the next 5 minutes. One solution was to throw out all the data except in steps of 5 minutes so the next step is automatically would be 5 minutes. This is clearly a waste of all the data that I have gathered. Can you please recommend what I can do about the prediction on different time scales? &lt;/p&gt;&#xA;" OwnerUserId="33488" LastEditorUserId="33488" LastEditDate="2020-02-12T18:31:03.620" LastActivityDate="2021-03-09T00:06:15.630" Title="LSTM model on different time scales" Tags="&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17994" PostTypeId="2" ParentId="17898" CreationDate="2020-02-12T17:12:06.907" Score="1" Body="&lt;p&gt;Are you talking about (X_train,y_train) and (X_test,y_test).&#xA;If yes, then X represents the data(features) and y represents the labels of that data. That's why you get a pair when you divide it into training and test data&lt;/p&gt;&#xA;" OwnerUserId="33489" LastActivityDate="2020-02-12T17:12:06.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17995" PostTypeId="2" ParentId="17828" CreationDate="2020-02-12T20:15:57.983" Score="0" Body="&lt;p&gt;Inspired by (1) machine translation where source phrases are first encoded to a feature vector and decoded by another RNN and (2) automatic captioning where a picture is also encoded to a feature vector before it's decoded by the RNN followed. A proper structure for the problem above may be the inverse of these, where the review is encoded by RNN to a vector, and a DNN is followed to decode the vector into #1 - #3.&lt;/p&gt;&#xA;" OwnerUserId="33266" LastActivityDate="2020-02-12T20:15:57.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17996" PostTypeId="2" ParentId="17993" CreationDate="2020-02-12T20:18:19.590" Score="0" Body="&lt;p&gt;What you could do is just try and bypass the rest of the network after the LSTM if it isn't the 5'th minute.&#xA;Depending on your framework this can be easy or a painstakingly task compared to the alternative.&#xA;The alternative is just running and throwing away the output that isn't the next 5'th minute. While the last may seem inefficient it's rather easy to implement and takes just a bit more execution time. If execution time isn't a problem it's the easiest to get started with, if it doesn't work for your task you can always change it.&lt;/p&gt;&#xA;" OwnerUserId="30100" LastActivityDate="2020-02-12T20:18:19.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17997" PostTypeId="2" ParentId="15524" CreationDate="2020-02-12T20:30:20.947" Score="3" Body="&lt;p&gt;I'm going to post another guess to this question - it won't be a complete answer, but hopefully it'll provide some direction towards finding a more legitimate answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The feed-forward networks as suggested by Vaswani are very reminiscent of the &lt;a href=&quot;https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sparse autoencoders&lt;/a&gt;. Where the input / output dimensions are much greater than the hidden input dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jDlzn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jDlzn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you aren't familiar with sparse autoencoders, this is a little counter intuitive - WTF would you have a larger hidden dimension?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intuition borrows from infinitely wide neural networks.  If you have an infinitely wide neural network, you have basically have a Gaussian process and sample any function you'd like.  So the wider the network you have, the more approximation power that you have.  In the case of inputs, this is a matter of learning a dictionary. If you have only discrete inputs, this hidden layer will be capped at &lt;span class=&quot;math-container&quot;&gt;$O(2^N)$&lt;/span&gt; width, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the maximum number of bits it takes to represent the input (which would boil down to approximating a lookup table). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, these aren't trivial to implement in practice.  These layers are bound to be bloated with identifiability issues.  Common approaches include &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; regularization.  I'm guessing that the convolutional layers + dropout are just another attempt to deal with these sorts of identifiability issues.  Furthermore, the FFN is an attempt to learn an arbitrary mapping for individual words (you can think of mapping words to synonyms for instance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are all guesses though - more intuition is welcome.&lt;/p&gt;&#xA;" OwnerUserId="33309" LastActivityDate="2020-02-12T20:30:20.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17998" PostTypeId="2" ParentId="17961" CreationDate="2020-02-13T05:10:23.763" Score="0" Body="&lt;p&gt;Ok, so I was right in assuming it was the discontinuity at &lt;code&gt;x=0&lt;/code&gt; for &lt;code&gt;relu(x)&lt;/code&gt;. The positive bias actually had nothing to do with the incorrect gradients. As for why in my edit I was getting such massive differences between gradient values, it was because I initialised the input to be integers, and upon adding a decimal, no matter how small (&gt; 0 though), numpy would jump entire integers, so &lt;code&gt;-2 + 0.0001 = -1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is reproduced by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; a = np.random.randint(-3,3,(1,3,3))&#xA;&amp;gt;&amp;gt;&amp;gt; a&#xA;array([[[-2,  1, -2],&#xA;        [-2, -1, -3],&#xA;        [-3,  0,  1]]])&#xA;&amp;gt;&amp;gt;&amp;gt; a[0][0][0] += 0.0001&#xA;&amp;gt;&amp;gt;&amp;gt; a&#xA;array([[[-1,  1, -2],&#xA;        [-2, -1, -3],&#xA;        [-3,  0,  1]]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem with my input is large amounts of it were 0, so it produced a bunch of discontinuous gradients. As such, I simply check both sides of the graph (have epsilon positive then negative), and if they disagree, I set the value to 0, as in that case I'd just ignore any gradient passing through that node.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-02-13T05:10:23.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17999" PostTypeId="1" CreationDate="2020-02-13T05:10:41.740" Score="0" ViewCount="235" Body="&lt;p&gt;In section 3 of the paper &lt;a href=&quot;https://www.student.cs.uwaterloo.ca/~cs492/11public_html/p18-smith.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Limits of Correctness&lt;/a&gt; (1985) &lt;a href=&quot;https://en.wikipedia.org/wiki/Brian_Cantwell_Smith&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brian Cantwell Smith&lt;/a&gt; writes&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When you design and build a computer system, you first formulate a model of the problem you want it to solve, and then construct the computer program in its terms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He then writes&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;computers have a special dependence on these models: you write an explicit description of the model down inside the computer, in the form of a set&#xA;  of rules or what are called representations - essentially linguistic formulae encoding, in the terms of the model, the facts and data&#xA;  thought to be relevant to the system's behavior. It is with respect to these representations that computer systems work. In fact, that's really what computers are (and how they differ from other machines): they run by manipulating representations, and representations are always formulated&#xA;  in terms of models. This can all be summarized in a slogan: no computation without representation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And then he says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Models have to ignore things exactly because they view the world at a level of abstraction&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He then writes in section 7&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The systems that land airplanes are hybrids - combinations of computers and people - exactly because the unforeseeable happens, and because what&#xA;  happens is in part the result of human action, requiring human interpretation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As quoted above, computers depend on models, which are abstractions (i.e. they ignore a lot of details), which are written inside the computer. Therefore, the true world cannot really be encoded into an algorithm, but only an abstraction and thus simplification of the world can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, will AI always depend on models and thus approximations? Can it get rid of or overcome this limitation?&lt;/p&gt;&#xA;" OwnerUserId="21644" LastEditorUserId="2444" LastEditDate="2020-03-10T23:29:44.747" LastActivityDate="2021-02-07T22:20:05.943" Title="Will AI always depend on models and thus approximations?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;models&gt;&lt;papers&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18000" PostTypeId="1" CreationDate="2020-02-13T06:04:06.380" Score="3" ViewCount="111" Body="&lt;p&gt;I am working on a research project about the different reward functions being used in the RL domain. I have read up on &lt;strong&gt;Inverse Reinforcement Learning (IRL)&lt;/strong&gt; and &lt;strong&gt;Reward Shaping (RS)&lt;/strong&gt;. I would like to clarify some doubts that I have with the 2 concepts.&lt;/p&gt;&#xA;&lt;p&gt;In the case of IRL, the goal is to find a reward function based on the policy that experts take. I have read that recovering the reward function that experts were trying to optimize, and then finding an optimal policy from those expert demonstrations has a possibility of resulting in a better policy (e.g. apprenticeship learning). Why does it lead to a better policy?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-08-03T23:03:13.773" LastActivityDate="2020-08-03T23:03:13.773" Title="Can recovering a reward function using IRL lead to better policies compared to reward shaping?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;rewards&gt;&lt;reward-shaping&gt;&lt;inverse-rl&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18001" PostTypeId="1" CreationDate="2020-02-13T06:09:14.783" Score="1" ViewCount="28" Body="&lt;p&gt;I have been working on industrial data, that is fed live, I want to explore a few models which might suit for this the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data are &lt;a href=&quot;https://en.wikipedia.org/wiki/Performance_indicator&quot; rel=&quot;nofollow noreferrer&quot;&gt;KPI&lt;/a&gt; data from the manufacturing Industry.&lt;/p&gt;&#xA;" OwnerUserId="33356" LastActivityDate="2020-02-13T06:09:14.783" Title="What models will you suggest to use in Industrial Anomaly Detection and Predictive analysis on live streamed data?" Tags="&lt;neural-networks&gt;&lt;models&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18002" PostTypeId="1" CreationDate="2020-02-13T06:44:53.870" Score="2" ViewCount="59" Body="&lt;p&gt;I have a many to one LSTM model for multiclass classification. For reference, this is the architecture of the model&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    model.add(LSTM(147, input_shape=(1000, 147)))&#xA;    model.add(Dense(5, activation='softmax'))&#xA;    model.compile(loss='categorical_crossentropy',&#xA;                  optimizer='rmsprop',&#xA;                  metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The model is trained in 5 types of sequences is able to effectively classify each sequence I feed into the model with high accuracy. Now my new objective is to combine these sequences together to form a new sequence. &lt;br&gt;For e.g &lt;br&gt; I denote the elements from the class '1' with the sequence:&#xA;&lt;br&gt;[1,1,1,1,1,1,1]&lt;br&gt; So when I input the above sequence into the LSTM model for prediction, it classifies the sequence as class '1' with accuracy of 0.99&lt;br&gt;&#xA;&lt;br&gt;And I denote the elements from class '2' with the sequence:&#xA;&lt;br&gt;[2,2,2]&lt;br&gt;&#xA;LIkewise for the above sequence, the LSTM model will classify the sequence as class '2' with accuracy of 0.99&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I combine these sequences together and feed it into the model :&#xA;&lt;br&gt; New sequence : [1,1,1,1,1,1,1,2,2,2]&#xA;&lt;br&gt; However the model does not seem to be sensitive to the presence of the class '2' sequence and still classifies the sequence as class '1' with accuracy of 0.99. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I make the model more &quot;sensitive&quot;, meaning that I would expect the LSTM model to still maybe predict class '1' but with a drop in accuracy? Or is the LSTM incapable of detecting the inclusion of class '2' sequences? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks. &lt;/p&gt;&#xA;" OwnerUserId="33499" LastActivityDate="2020-02-13T06:44:53.870" Title="How do I make my LSTM model more sensitive to changes in the sequence?" Tags="&lt;python&gt;&lt;tensorflow&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18004" PostTypeId="1" CreationDate="2020-02-13T07:46:42.537" Score="1" ViewCount="41" Body="&lt;p&gt;So I want to create an RL-agent for two players-board game. I want to use a simple DQN for the first player (my RL-agent). Then, what kind of algorithm that should I use on the second player (my RL-agent's enemy)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have three options in my mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a random agent that act randomly&lt;/li&gt;&#xA;&lt;li&gt;a rule-based agent that acts by some defined rules&lt;/li&gt;&#xA;&lt;li&gt;another RL-agent&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried the first and second options. When I use a random agent as an enemy, the first player gets a high score and wins easily. But I think it's not really smart as the enemy is a random agent. When I use the second option, the first player got difficulties to train itself, as it can't win any game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what should I choose and why?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2020-02-17T00:58:27.143" LastActivityDate="2020-02-17T00:58:27.143" Title="What kind of enemy to train a good RL-agents" Tags="&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18005" PostTypeId="1" CreationDate="2020-02-13T08:53:29.510" Score="1" ViewCount="24" Body="&lt;p&gt;I'm implementing a multilayer perceptron with Keras to predict the correct words order in a sentence.&#xA;I'm using &lt;code&gt;train_on_batch()&lt;/code&gt;because I convert each sentence in a tree and then order each local subtree: when each subtree is ordered, the entire tree is even ordered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I notice a strange things during the training: since from the first epoch, the loss value is 0.0.&#xA;My dataset shape is &lt;code&gt;(453732, 300)&lt;/code&gt;(initially the number of features was 838, but I use PCA to reduce them) and this is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mlp = keras.models.Sequential()&#xA;&#xA;# add input layer&#xA;mlp.add(&#xA;    keras.layers.Dense(&#xA;        units=training_dataset.shape[1],&#xA;        input_shape = (training_dataset.shape[1], ),&#xA;        kernel_initializer='glorot_uniform',&#xA;        bias_initializer='zeros',&#xA;        activation='tanh') &#xA;)&#xA;# add hidden layer&#xA;mlp.add(&#xA;    keras.layers.Dense(&#xA;        units=training_dataset.shape[1] + 10,&#xA;        input_shape = (training_dataset.shape[1] + 10,),&#xA;        kernel_initializer='glorot_uniform',&#xA;        bias_initializer='zeros',&#xA;        activation='relu')&#xA;    )&#xA;&#xA;mlp.add(&#xA;    keras.layers.Dropout(&#xA;        0.2, &#xA;        input_shape=(training_dataset.shape[1] + 10,))&#xA;    )&#xA;&#xA;# add output layer&#xA;mlp.add(&#xA;    keras.layers.Dense(&#xA;        units=1,&#xA;        input_shape = (1, ),&#xA;        kernel_initializer='glorot_uniform',&#xA;        bias_initializer='zeros',&#xA;        activation='softmax')&#xA;    )&#xA;&#xA;# define SGD optimizer&#xA;sgd_optimizer = keras.optimizers.SGD(&#xA;    lr=0.01, decay=0.01, momentum=0.9, nesterov=True&#xA;)&#xA;# compile model&#xA;&#xA;mlp.compile(&#xA;    optimizer=sgd_optimizer,&#xA;    loss=listnet_loss&#xA;)&#xA;&#xA;mlp.summary() # print model settings&#xA;&#xA;losses = np.array([])&#xA;# Training&#xA;with tf.device('/GPU:0'):&#xA;  for epoch in range(0, 10):&#xA;    print('Epoch {0} started!'.format(epoch))&#xA;    start_range = 0&#xA;    for group in groups_id_count:&#xA;      end_range = (start_range + group[1]) # Batch is a group of words with same group id&#xA;      batch_dataset = training_dataset[start_range:end_range, :]&#xA;      batch_labels = training_dataset_labels[start_range:end_range]&#xA;      batch_train_result = mlp.train_on_batch(batch_dataset, batch_labels)&#xA;      losses = np.append(losses, batch_train_result)&#xA;      start_range = end_range&#xA;    print('Epoch {0} loss: {1}'.format(epoch, np.mean(losses)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;listnet_loss&lt;/code&gt;is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_top_one_probability(vector):&#xA;  return (K.exp(vector) / K.sum(K.exp(vector)))&#xA;&#xA;def listnet_loss(real_labels, predicted_labels):&#xA;  return -K.sum(get_top_one_probability(real_labels)) * tf.math.log(get_top_one_probability(predicted_labels))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;groups_id_count&lt;/code&gt;is a list of tuple of the form &lt;code&gt;(#subtree_number,#number_of_words_in_subtree)&lt;/code&gt;, where &lt;code&gt;#subtree_number&lt;/code&gt;is an identifier for a subtree and &lt;code&gt;#number_of_words_in_subtree&lt;/code&gt;is the number of words in the subtree. Then, my batch are dynamic: a batch is composed by the number of words in a subtree. This is why I used &lt;code&gt;train_on_batch()&lt;/code&gt;to train my model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="33440" LastActivityDate="2020-02-13T08:53:29.510" Title="Keras MLP returns always loss 0.0" Tags="&lt;python&gt;&lt;keras&gt;&lt;loss-functions&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-06-02T22:15:33.087" ContentLicense="CC BY-SA 4.0" />
  <row Id="18006" PostTypeId="1" AcceptedAnswerId="18007" CreationDate="2020-02-13T13:21:42.290" Score="1" ViewCount="135" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1511.06279&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Programmer-Interpreters&lt;/a&gt;, the authors use the teacher forcing technique, but what exactly is it?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-21T10:05:47.967" Title="What is teacher forcing?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;terminology&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18007" PostTypeId="2" ParentId="18006" CreationDate="2020-02-13T13:21:42.290" Score="1" Body="&lt;p&gt;Consider the task of sequence prediction, so you want to predict the next element of a sequence &lt;span class=&quot;math-container&quot;&gt;$e_t$&lt;/span&gt; given the previous elements of this sequence &lt;span class=&quot;math-container&quot;&gt;$e_{t-1}, e_{t-2}, \dots, e_{1} = e_{t-1:1}$&lt;/span&gt;. Teacher forcing is about forcing the predictions to be based on correct histories (i.e. the correct sequence of past elements) rather than predicted history (which may not be correct). To be more concrete, let &lt;span class=&quot;math-container&quot;&gt;$\hat{e}_{i}$&lt;/span&gt; denote the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th predicted element of the sequence and let &lt;span class=&quot;math-container&quot;&gt;$e_{i}$&lt;/span&gt; be the corresponding ground-truth. Then, if you use teacher forcing, to predict &lt;span class=&quot;math-container&quot;&gt;$e_{t}$&lt;/span&gt;, rather than using &lt;span class=&quot;math-container&quot;&gt;$\hat{e}_{t-1:1}$&lt;/span&gt;, you would use &lt;span class=&quot;math-container&quot;&gt;$e_{t-1:1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that supervised learning can also be thought of as learning with a teacher. Hence the expression &quot;teacher forcing&quot;, i.e. you &lt;strong&gt;force&lt;/strong&gt; the predictions to be based on correct histories (the &lt;strong&gt;teacher&lt;/strong&gt;'s labels). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, intuitively, teacher forcing should help to stabilize training, given that the predictions are not based on noisy or wrong histories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the blog post &lt;a href=&quot;https://machinelearningmastery.com/teacher-forcing-for-recurrent-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is Teacher Forcing for Recurrent Neural Networks?&lt;/a&gt; by Jason Brownlee.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-13T13:28:26.620" LastActivityDate="2020-02-13T13:28:26.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18008" PostTypeId="1" CreationDate="2020-02-13T13:48:06.823" Score="4" ViewCount="76" Body="&lt;p&gt;In my work I have a given source code for a module. From this module I generate an AST, whose size is dependent on the size of the module (e.g. more source code -&gt; bigger AST). I want to train a neural network model which will learn a general structure of a module and be able to rate (on a scale of 0 to 1) how &quot;good&quot; a module is structure wise (if requires are at the beginning, followed by local functions, variables and finally returns). Now I have learnt that Convolutional NNs are quite convenient for this, but the problem I can't seem to solve is that they require a fixed sized input which I can't produce. If I add zero-padding then the outcome will be skewed and the accuracy will suffer. Is there a clear solution to this problem?&lt;/p&gt;&#xA;" OwnerUserId="33509" LastEditorUserId="33509" LastEditDate="2020-02-13T14:19:14.333" LastActivityDate="2021-03-12T23:04:00.393" Title="How to solve the problem of variable-sized AST as input for a (convolutional) neural network model?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;graphs&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18009" PostTypeId="1" CreationDate="2020-02-13T13:59:39.110" Score="1" ViewCount="33" Body="&lt;p&gt;Currently I'm working on an educational project (implementation of AlphaZero approach to different types of board games).&#xA;My biggest concern at the moment is how to encode board before input into the neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, how can this be done for &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Kalah&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kalah&lt;/a&gt;&lt;/strong&gt; game?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kalah board has two straight rows of six pits, and two large score-houses.&#xA;Each pit contains 6 seeds (so there are 72 seeds) &lt;/p&gt;&#xA;" OwnerUserId="33025" LastActivityDate="2020-02-13T13:59:39.110" Title="How to encode board before input into the neural net?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;alphazero&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18010" PostTypeId="1" CreationDate="2020-02-13T15:13:37.153" Score="1" ViewCount="344" Body="&lt;p&gt;I am not asking what does concatenate layer does in general in point of mathematical operation. But at feature level, what significance does it provide. Does it helps removing false negatives or does it prevents over-fitting? Do give the reference of papers regarding this topic.&lt;/p&gt;&#xA;" OwnerUserId="28417" LastActivityDate="2020-02-13T15:13:37.153" Title="What is the use of concatenate layer in CNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18012" PostTypeId="1" CreationDate="2020-02-13T18:25:34.973" Score="1" ViewCount="37" Body="&lt;p&gt;I want to convert the code written in Python into Matlab code. May I know is it possible to do that. Share the available ways or methods to do the conversion. May I know is there any Online Translators which do the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the Data I used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1GLm87-5E_6YhUIPZ_CtQLV9F9wcGaTj2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/open?id=1GLm87-5E_6YhUIPZ_CtQLV9F9wcGaTj2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# imports libraries&#xA;import numpy as np&#xA;import pandas as pd&#xA;import os&#xA;import tensorflow as tf&#xA;import matplotlib.pyplot as plt&#xA;import random&#xA;from scipy import signal&#xA;from tensorflow.keras.layers import Dense, Activation&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow import set_random_seed&#xA;from tensorflow.keras.initializers import glorot_uniform&#xA;from sklearn.model_selection import train_test_split&#xA;from sklearn.preprocessing import StandardScaler&#xA;from sklearn.metrics import mean_squared_error&#xA;from sklearn.metrics import accuracy_score&#xA;from importlib import reload&#xA;&#xA;# useful pandas display settings&#xA;pd.options.display.float_format = '{:.3f}'.format&#xA;&#xA;# useful functions&#xA;def plot_history(history, metrics_to_plot):&#xA;    &quot;&quot;&quot;&#xA;    Function plots history of selected metrics for fitted neural net.&#xA;&#xA;    &quot;&quot;&quot;&#xA;&#xA;    # plot&#xA;    for metric in metrics_to_plot:&#xA;        plt.plot(history.history[metric])&#xA;&#xA;    # name X axis informatively&#xA;    plt.xlabel('epoch')&#xA;&#xA;    # name Y axis informatively&#xA;    plt.ylabel('metric')&#xA;&#xA;    # add informative legend&#xA;    plt.legend(metrics_to_plot)&#xA;&#xA;    # plot&#xA;    plt.show()&#xA;&#xA;def plot_fit(y_true, y_pred, title='title'):&#xA;    &quot;&quot;&quot;&#xA;    Function plots true values and predicted values, sorted in increase order by true values.&#xA;&#xA;    &quot;&quot;&quot;&#xA;&#xA;    # create one dataframe with true values and predicted values&#xA;    results = y_true.reset_index(drop=True).merge(pd.DataFrame(y_pred), left_index=True, right_index=True)&#xA;&#xA;    # rename columns informartively&#xA;    results.columns = ['true', 'prediction']&#xA;&#xA;    # sort for clarity of visualization&#xA;    results = results.sort_values(by=['true']).reset_index(drop=True)&#xA;&#xA;    # plot true values vs predicted values&#xA;    results.plot()&#xA;&#xA;    # adding scatter on line plots&#xA;    plt.scatter(results.index, results.true, s=5)&#xA;    plt.scatter(results.index, results.prediction, s=5)&#xA;&#xA;    # name X axis informatively&#xA;    plt.xlabel('obs sorted in ascending order with respect to true values')&#xA;&#xA;    # add customizable title&#xA;    plt.title(title)&#xA;&#xA;    # plot&#xA;    plt.show();&#xA;&#xA;def reset_all_randomness():&#xA;    &quot;&quot;&quot;&#xA;    Function assures reproducibility of NN estimation results.&#xA;&#xA;    &quot;&quot;&quot;&#xA;&#xA;    # reloads&#xA;    reload(tf)&#xA;    reload(np)&#xA;    reload(random)&#xA;&#xA;    # seeds - for reproducibility&#xA;    os.environ['PYTHONHASHSEED']=str(984797)&#xA;    random.seed(984797)&#xA;    set_random_seed(984797)&#xA;    np.random.seed(984797)&#xA;    my_init = glorot_uniform(seed=984797)&#xA;&#xA;    return my_init&#xA;def give_me_mse(true, prediction):&#xA;    &quot;&quot;&quot;&#xA;    This function returns mse for 2 vectors: true and predicted values.&#xA;&#xA;    &quot;&quot;&quot;&#xA;&#xA;    return np.mean((true-prediction)**2)&#xA;# Importing the dataset&#xA;X = pd.read_excel(r&quot;C:\filelocation\Data.xlsx&quot;,&quot;Sheet1&quot;).values&#xA;y = pd.read_excel(r&quot;C:\filelocation\Data.xlsx&quot;,&quot;Sheet2&quot;).values&#xA;&#xA;# Importing the experiment data&#xA;Data = pd.read_excel(r&quot;C:\filelocation\Data.xlsx&quot;,&quot;Sheet1&quot;)&#xA;v = pd.DataFrame(Data, columns= ['v']).values&#xA;c = pd.DataFrame(Data, columns= ['c']).values&#xA;ird = pd.DataFrame(Data, columns= ['ird']).values&#xA;tmp = pd.DataFrame(Data, columns= ['tmp']).values&#xA;&#xA;#Data Prepration&#xA;ird = ird.ravel()&#xA;tmp = tmp.ravel()&#xA;ir = np.nanmax(ird)&#xA;tp = np.nanmax(tmp)&#xA;p = v*c&#xA;p = p.ravel()&#xA;peaks, _ = signal.find_peaks(p)&#xA;nop = len(peaks)&#xA;pv = p.max()&#xA;&#xA;#Experimental Data for testing&#xA;E_data = np.array([[ir,tp,pv,nop]])&#xA;&#xA;#importing some more libraries&#xA;from sklearn.preprocessing import LabelEncoder&#xA;from keras.utils import np_utils&#xA;&#xA;# encode class values as integers&#xA;encoder = LabelEncoder()&#xA;encoder.fit(np.ravel(y))&#xA;y_encoded = encoder.transform(np.ravel(y))&#xA;# convert integers to dummy variables (i.e. one hot encoded)&#xA;y_dummy = np_utils.to_categorical(y_encoded)&#xA;&#xA;# reset_all_randomness - for reproducibility&#xA;my_init = reset_all_randomness()&#xA;&#xA;# Splitting the dataset into the Training set and Test set&#xA;X_train, X_test, y_train, y_test, y_train_dummy, y_test_dummy = train_test_split(X, y, y_dummy, test_size = 0.3, random_state = 20)&#xA;&#xA;# Feature Scaling&#xA;sc = StandardScaler()&#xA;X_train = sc.fit_transform(X_train)&#xA;X_test = sc.transform(X_test)&#xA;E_data = sc.transform(E_data)&#xA;&#xA;# Initialising the ANN&#xA;model0 = Sequential()&#xA;&#xA;# Adding 1 hidden layer: the input layer and the first hidden layer&#xA;model0.add(Dense(units = 160, activation = 'tanh', input_dim = 4, kernel_initializer=my_init))&#xA;&#xA;# Adding 2 hidden layer&#xA;model0.add(Dense(units = 49, activation = 'tanh', kernel_initializer=my_init))&#xA;&#xA;# Adding 3 hidden layer&#xA;model0.add(Dense(units = 24, activation = 'tanh', kernel_initializer=my_init))&#xA;&#xA;# Adding 4 hidden layer&#xA;model0.add(Dense(units = 15, activation = 'tanh', kernel_initializer=my_init))&#xA;&#xA;# Adding output layer&#xA;model0.add(Dense(units = 6, activation = 'softmax', kernel_initializer=my_init))&#xA;&#xA;# Set up Optimizer&#xA;Optimizer = tf.train.AdamOptimizer(learning_rate=0.001, beta1=0.9, beta2=0.99)&#xA;&#xA;# Compiling the ANN&#xA;model0.compile(optimizer = Optimizer, loss = 'categorical_crossentropy', metrics=['accuracy','categorical_crossentropy','mse'])&#xA;&#xA;# Fitting the ANN to the Train set, at the same time observing quality on Valid set&#xA;history = model0.fit(X_train, y_train_dummy, validation_data=(X_test, y_test_dummy), batch_size = 100, epochs = 1500)&#xA;&#xA;# Generate prediction for all Train, Valid set and Experimental set&#xA;y_train_pred_model0 = model0.predict(X_train)&#xA;y_test_pred_model0 = model0.predict(X_test)&#xA;y_exp_pred_model0 = model0.predict(E_data)&#xA;&#xA;# find final prediction by taking class with highest probability&#xA;y_train_pred_model0 = np.array([[list(x).index(max(list(x))) + 1] for x in y_train_pred_model0])&#xA;y_test_pred_model0 = np.array([[list(x).index(max(list(x))) + 1] for x in y_test_pred_model0])&#xA;y_exp_pred_model0 = np.array([[list(x).index(max(list(x))) + 1] for x in y_exp_pred_model0])&#xA;&#xA;# check what metrics are in fact available in history&#xA;history.history.keys()&#xA;&#xA;# Inverse scaling &#xA;X_train_inverse = sc.inverse_transform(X_train)&#xA;X_test_inverse = sc.inverse_transform(X_test)&#xA;E_data_inverse = sc.inverse_transform(E_data)&#xA;&#xA;#Plots&#xA;print('#######################################################################')&#xA;&#xA;# look at model fitting history&#xA;plot_history(history, ['mean_squared_error', 'val_mean_squared_error'])&#xA;plot_history(history, ['categorical_crossentropy', 'val_categorical_crossentropy'])&#xA;plot_history(history, ['acc', 'val_acc'])&#xA;&#xA;# look at model fit quality&#xA;plot_fit(pd.DataFrame(y_train), y_train_pred_model0, 'Fit on train data')&#xA;plot_fit(pd.DataFrame(y_test), y_test_pred_model0, 'Fit on test data')&#xA;&#xA;#Results&#xA;print('#######################################################################')&#xA;&#xA;print('=============Mean Squared Error============')&#xA;&#xA;print('MSE on train data is: {}'.format(give_me_mse(y_train, y_train_pred_model0)))&#xA;print('MSE on test data is: {}'.format(give_me_mse(y_test, y_test_pred_model0)))&#xA;&#xA;print('#######################################################################')&#xA;&#xA;print('================Accuracy===================')&#xA;&#xA;print('Accuracy of ANN is: {} Percentage'.format((accuracy_score(y_test, y_test_pred_model0))*100))&#xA;&#xA;print('#######################################################################')&#xA;&#xA;print('========Result of Test Data set is=========')&#xA;&#xA;for i in range(len(y_test)):&#xA;    print('%s =&amp;gt; %d (expected %s)' % (X_test_inverse[i].tolist(), y_test_pred_model0[i], y_test[i].tolist()))&#xA;&#xA;print('#######################################################################')&#xA;&#xA;print('====Result of Experimental Data set is=====')&#xA;&#xA;print('%s =&amp;gt; %d' % (E_data_inverse, y_exp_pred_model0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31520" LastActivityDate="2020-02-13T18:25:34.973" Title="Is it possible to convert Neural Network code in Python into Matlab code?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;python&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-02-20T00:19:13.357" ContentLicense="CC BY-SA 4.0" />
  <row Id="18013" PostTypeId="1" CreationDate="2020-02-13T18:53:09.543" Score="2" ViewCount="57" Body="&lt;p&gt;Is there a machine learning framework/library for any of the esoteric languages, such as the ones listed &lt;a href=&quot;https://esolangs.org/wiki/Language_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-02-13T18:53:09.543" Title="Machine learning frameworks for esoteric languages" Tags="&lt;machine-learning&gt;&lt;programming-languages&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="4" ClosedDate="2021-02-03T16:45:48.220" ContentLicense="CC BY-SA 4.0" />
  <row Id="18014" PostTypeId="1" CreationDate="2020-02-13T19:12:34.117" Score="1" ViewCount="20" Body="&lt;p&gt;if someone want to do mobile app for body measurements prediction, please what are the necessary things to start with. I need details explanation on this.&lt;/p&gt;&#xA;" OwnerUserId="33520" LastActivityDate="2020-02-13T19:12:34.117" Title="Details on body measurements prediction" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18015" PostTypeId="1" CreationDate="2020-02-14T01:45:13.487" Score="2" ViewCount="45" Body="&lt;p&gt;Firstly, some context. I have been reading and watching videos on the subject for around 3 years, but I am still very much a beginner in machine learning and artificial intelligence. That said, I might not know what I'm even talking about here. So bear with me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, each node in a neural network (neuron) is represented by some floating point number between 0 and 1, that are arranged in layers and have corresponding weights. Right? While a color has RGB values, CMYK values, and HSV values that are all interrelated to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is would there be any benefit to having each node represented by a color instead of a single floating point number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thinking is that each neuron could select any of the values (r, g, b, c, m, y, k, h, s, or v) contained within the color in some meaningful way, while the Alpha value could possibly represent the weight associated with that neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thoughts? Would it not work like that? Could you use it to have multiple congruent networks running on 3 different channels? Again, would there be any benefit to doing this than just using a single number? Or would it over-complicate (or even break) the network? Would it be useless?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I've also dabbled in Unity3D (which is how I got the idea in the first place), I'm too much of a beginner to know how to even begin an attempt at testing this myself.&lt;/p&gt;&#xA;" OwnerUserId="33523" LastActivityDate="2020-02-14T05:01:12.923" Title="In a neural network, can colors be used for neurons in place of floating points and would there be any benefit in doing so?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18016" PostTypeId="2" ParentId="18015" CreationDate="2020-02-14T05:01:12.923" Score="2" Body="&lt;p&gt;To answer your question, I'll just simplify it and assume you are representing the activation of a neuron (the value it produces) by an RGB value. So a tuple with 3 values ranging from 0 to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to remember that every single machine learning model could be computed by a human on pen and paper using just numbers. So keeping that in mind, this neural network would essentially just be a combination of multiplying tuples of size 3 in a certain order (ignoring activation functions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will also assume that each RGB value has it's own unique set of &lt;em&gt;incoming&lt;/em&gt; weights, like following:&lt;a href=&quot;https://i.stack.imgur.com/s58FJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s58FJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this essentially amounts to just have 3 unique neurons. There's no difference:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/U7orp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U7orp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I did there was get rid of the circle. The calculations are no different. If you changed the calculation, say values are shared between the 3 RGB nodes, then you now just have 3 duplicate nodes. In all cases, you can just represent the equivalent calculation as a standard neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the same vein, in a convolutional neural network, if you create conv layers with an output of 3 channels, you can visualise their outputs as a colour image (assuming you are using an activation function that outputs numbers between 0 and 1), which can be cool to look at.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-02-14T05:01:12.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18017" PostTypeId="1" AcceptedAnswerId="18025" CreationDate="2020-02-14T07:16:15.987" Score="5" ViewCount="273" Body="&lt;p&gt;In reinforcement learning, the state-action value function seems to be used more than the state value function. Why is it so?&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="2444" LastEditDate="2020-02-14T12:22:32.987" LastActivityDate="2020-02-14T13:14:46.893" Title="Why is the state-action value function used more than the state value function?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18018" PostTypeId="1" CreationDate="2020-02-14T07:21:54.610" Score="1" ViewCount="32" Body="&lt;p&gt;So as the title states, I have a set of images and I want to process input images and need to select the image that &quot;looks&quot; the most like the input image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I've seen something similar where the code could guess who's face was in a picture, I guess I want something like that but for general images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry if this is a stupid question, but any suggestions or points at resources would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="33529" LastActivityDate="2020-02-14T12:44:05.237" Title="I need to select the image from a predefined dataset that are the closest to the input, is this possible or do I even need to use ML/AI?" Tags="&lt;classification&gt;&lt;tensorflow&gt;&lt;video-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18019" PostTypeId="1" AcceptedAnswerId="18027" CreationDate="2020-02-14T08:23:15.227" Score="2" ViewCount="144" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$D_{KL}=\sum_i p(x_i)log(p(x_i)/q(x_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also can't you make it satisfy the triangle inequality by taking the absolute value of the information at every point?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-02-14T14:11:41.413" LastActivityDate="2020-02-14T23:49:51.847" Title="Why does KL divergence not satisfy the triangle inequality?" Tags="&lt;proofs&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18020" PostTypeId="1" AcceptedAnswerId="18023" CreationDate="2020-02-14T09:03:51.183" Score="2" ViewCount="204" Body="&lt;p&gt;I started learning about Q table from this blog post &lt;a href=&quot;https://www.oreilly.com/radar/introduction-to-reinforcement-learning-and-openai-gym/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to reinforcement learning and OpenAI Gym, by Justin Francis&lt;/a&gt;, which has a line as below -&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;After so many episodes, the algorithm will converge and determine the optimal action for every state using the Q table, ensuring the highest possible reward. We now consider the environment problem solved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Q table was updated by Q-learning formula&#xA;&lt;code&gt;Q[state,action] += alpha * (reward + np.max(Q[state2]) - Q[state,action])&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran 100000 episodes of which I got the following -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Episode 99250 Total Reward: 9&#xA;Episode 99300 Total Reward: 7&#xA;Episode 99350 Total Reward: 6&#xA;Episode 99400 Total Reward: 14&#xA;Episode 99450 Total Reward: 10&#xA;Episode 99500 Total Reward: 10&#xA;Episode 99550 Total Reward: 9&#xA;Episode 99600 Total Reward: 14&#xA;Episode 99650 Total Reward: 5&#xA;Episode 99700 Total Reward: 7&#xA;Episode 99750 Total Reward: 3&#xA;Episode 99800 Total Reward: 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know what the highest reward is. It does not look like it has converged. Yet, the &lt;a href=&quot;https://i.imgur.com/FdlLKE8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;graph&lt;/a&gt; shows a trend in convergence but it was plotted for a larger scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should be the sequence of actions to be taken when the game is reset() but the &quot;learned&quot; Q table is available? How do we know that and the reward in that case?&lt;/p&gt;&#xA;" OwnerUserId="21983" LastActivityDate="2020-02-14T11:01:22.993" Title="Taxi-v3 help. What is meant exactly by convergence of the algo, the highest reward and optimal action for every state?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18023" PostTypeId="2" ParentId="18020" CreationDate="2020-02-14T11:01:22.993" Score="2" Body="&lt;p&gt;Your Q learning update expression looks correct. The &lt;code&gt;Total Reward&lt;/code&gt; will not be the same at the end of each episode because the starting position of taxi is different in each episode so the number of steps necessary to reach the final destination will be different in each episode. The graph that you posted shows that the algorithm converged after short amount of episodes so 100000 episodes might be too much. Since the environment is simple try manually calculating optimal policy for some starting specific position and then see if the algorithm does the same sequence of actions.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-02-14T11:01:22.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18024" PostTypeId="2" ParentId="18018" CreationDate="2020-02-14T12:44:05.237" Score="2" Body="&lt;p&gt;You will only need to use ML or AI if &#xA;a) the dataset is very big.&#xA;b) It is difficult to get the meaning or value of the image.(ex: group of ants, photos of stars in night sky)&#xA;And many more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the way I thought it can be done through a machine(with deeplearning, CNN)&#xA;1. Train the model&#xA;        arrange the images in dataset into clusters (the number of clusters can &#xA;        be decided based on the criticality of the application, more critical the &#xA;        application more should be the number of clusters) .&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Predict about input &#xA;    Then predict the nearest image in the dataset to the provided image.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I recently listened a podcast in which, a problem is approached similarly. The podcast is about youtube predictions and the result is to provide closest video from the youtube videos ( here the dataset).Below is the podcast.&#xA;    ( &lt;a href=&quot;https://open.spotify.com/episode/6PcMtVXR58i7iu8kLH40Wd?si=feyu0LjESoiE479xHkaqSA&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://open.spotify.com/episode/6PcMtVXR58i7iu8kLH40Wd?si=feyu0LjESoiE479xHkaqSA&lt;/a&gt; )   &lt;/p&gt;&#xA;" OwnerUserId="32856" LastActivityDate="2020-02-14T12:44:05.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18025" PostTypeId="2" ParentId="18017" CreationDate="2020-02-14T13:14:46.893" Score="5" Body="&lt;p&gt;We are ultimately interested in getting an optimal policy, that is the optimal sequence of actions to reach the final goal. State values on its own don't provide that, they tell you expected return from specific state onward but they don't tell you which action to take. In order to derive an optimal action in a specific state you would have to simulate all possible actions one step ahead and then pick the action that leads you to the state with highest state value. That is often inconvenient or impossible. State action values connect the expected return with actions, not states, so you don't need to simulate all actions one step ahead and see where you end up, you only need to pick an action that has the highest value because you know that is the best action to take in that state.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-02-14T13:14:46.893" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18026" PostTypeId="1" CreationDate="2020-02-14T14:01:56.900" Score="0" ViewCount="99" Body="&lt;p&gt;I have two lists of feature vectors calculated from pre-trained CNN for image retrieval task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Query: &lt;code&gt;FV_Q&lt;/code&gt; and Reference &lt;code&gt;FV_R&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; FV_R.shape&#xA;(3450, 128)&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; FV_Q.shape&#xA;(3450, 128)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am a little confused between the concept of &lt;strong&gt;exhaustive nearest neighbor search&lt;/strong&gt; and &lt;strong&gt;k-nearest neighbor search&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In python, I use &lt;code&gt;from sklearn.neighbors import KDTree&lt;/code&gt; to extract top &lt;code&gt;k = 5&lt;/code&gt; similar images from the reference database, given the query image!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody explain if there might be any similarities/differences between these two concepts? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I making a mistake somewhere in my feature vector comparison?&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="2444" LastEditDate="2020-06-02T22:12:06.130" LastActivityDate="2021-02-28T01:02:15.263" Title="What is the difference between exhaustive nearest neighbor search and k-nearest neighbour search?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;comparison&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18027" PostTypeId="2" ParentId="18019" CreationDate="2020-02-14T14:07:01.940" Score="2" Body="&lt;p&gt;To prove that the KL divergence does not satisfy the triangle inequality, you just need a &lt;a href=&quot;http://www.math.toronto.edu/writing/Counterexample.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;counterexample&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Definitions&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;KL divergence&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let's first recapitulate the definition of KL divergence for discrete probability distributions &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (for simplicity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;D_{\text{KL}}(p\parallel q)&#xA;=&#xA;\sum_{x\in {\mathcal {X}}}&#xA;p(x)\log &#xA;\left(&#xA;\frac {p(x)}{q(x)}&#xA;\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Triangle inequality&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let's also recap the definition of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;triangle inequality&lt;/a&gt; for distance &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, which can be expressed as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d(p,q) \leq d(p, r) + d(r, q),&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$p, q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; are probability distributions. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Proof&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let the sample space be &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X} = \{0, 1 \}$&lt;/span&gt;. Consider now the following three discrete probability distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(x)=&#xA;\begin{cases}&#xA;\frac{1}{2}&amp;amp; \text{if } x = 0\\&#xA;\frac{1}{2}&amp;amp; \text{if } x = 1&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;r(x)=&#xA;\begin{cases}&#xA;\frac{1}{4}&amp;amp; \text{if } x = 0\\&#xA;\frac{3}{4}&amp;amp; \text{if } x = 1&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;q(x)=&#xA;\begin{cases}&#xA;\frac{1}{10}&amp;amp; \text{if } x = 0\\&#xA;\frac{9}{10}&amp;amp; \text{if } x = 1&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's now apply the definition of the triangle inequality and the KL divergence for these discrete probability distributions. So, we want to show the following inequality does &lt;strong&gt;not&lt;/strong&gt; hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{\text{KL}}(p\parallel q) &#xA;\leq &#xA;D_{\text{KL}}(p\parallel r) &#xA;+ &#xA;D_{\text{KL}}(r\parallel q)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be expanded to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\sum_{x\in \{0, 1 \}}&#xA;p(x)\log &#xA;\left(&#xA;\frac {p(x)}{q(x)}&#xA;\right)&#xA;&amp;amp;\leq &#xA;\sum_{x\in \{0, 1 \} }&#xA;p(x)\log &#xA;\left(&#xA;\frac {p(x)}{r(x)}&#xA;\right)&#xA;+ &#xA;\sum_{x\in \{0, 1 \}}&#xA;r(x)\log &#xA;\left(&#xA;\frac {r(x)}{q(x)}&#xA;\right) &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which roughly corresponds to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;0.51 &amp;amp;\leq 0.14 + 0.09 \\&#xA;0.51 &amp;amp;\leq 0.24&#xA;\end{align}&lt;/span&gt;&#xA;which is clearly false. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the triangle inequality does not hold in one case, it doesn't hold in all cases, so the triangle inequality does not hold for the KL divergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tag*{$\blacksquare$}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reproducibility, I have used the following Python (3.7) program to compute the KL divergences.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import log&#xA;&#xA;one = (1 / 2 * log((1 / 2) / (1 / 10))) + (1 / 2 * log((1 / 2) / (9 / 10)))&#xA;two = (1 / 2 * log((1 / 2) / (1 / 4))) + (1 / 2 * log((1 / 2) / (3 / 4)))&#xA;three = (1 / 4 * log((1 / 4) / (1 / 10))) + (3 / 4 * log((3 / 4) / (9 / 10)))&#xA;&#xA;print(one)&#xA;print(two)&#xA;print(three)&#xA;print(two + three)&#xA;print(one &amp;lt;= two + three)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-14T23:49:51.847" LastActivityDate="2020-02-14T23:49:51.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18028" PostTypeId="1" CreationDate="2020-02-14T15:33:19.077" Score="1" ViewCount="25" Body="&lt;p&gt;I'm currently working on my uni project, but I have no idea where to start for the user tailored recommendation system on web. Where can I find a good guide on it, preferrably on languages like php and javascript. &lt;/p&gt;&#xA;" OwnerUserId="33537" LastActivityDate="2020-03-17T18:02:15.320" Title="Where can I find good tutorials on user tailored recommendation system for web?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18030" PostTypeId="1" CreationDate="2020-02-14T19:47:27.067" Score="2" ViewCount="126" Body="&lt;p&gt;I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can operate exactly like a logical operand, which obviously can classify things, and, with backpropagation and linear classification, it's obvious that, if a certain pattern exists, it'll activate the correct gates in order to classify correctly, but that is not a mathematical proof.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-02-14T23:52:17.297" LastActivityDate="2021-04-10T14:04:40.377" Title="Is there a mathematical theory behind why MLP can classify handwritten digits?" Tags="&lt;classification&gt;&lt;proofs&gt;&lt;multilayer-perceptron&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18032" PostTypeId="1" CreationDate="2020-02-15T02:24:14.560" Score="1" ViewCount="25" Body="&lt;p&gt;I am currently working on implementing a lip reading system in Python using machine learning and image processing. Currently, two initial implementations have provided promising results, albeit not perfect: the LipNet model and the Google Cloud AutoML Vision API. Before I dive head on into these two different systems, I was wondering if anyone on here had been exposed to any other alternatives for lip reading or any other facial detection issue, and their experiences with these models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any information will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="33547" LastActivityDate="2020-02-15T02:24:14.560" Title="What are some of the best methods in detecting facial movement using state-of-the-art machine learning models?" Tags="&lt;machine-learning&gt;&lt;image-processing&gt;&lt;google-cloud&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18033" PostTypeId="2" ParentId="18030" CreationDate="2020-02-15T03:37:00.683" Score="-1" Body="&lt;p&gt;The approximation theorem says you can approximate anything. But this is kind of meaningless in so far as you can do KNN and get an arbitrary approximation of your training data also.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proving CNN correctly extract features is, I don't think possible. Or if it is, something involving VC theory is probably the best you can do.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-02-15T03:37:00.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18035" PostTypeId="2" ParentId="16973" CreationDate="2020-02-15T09:54:08.230" Score="1" Body="&lt;p&gt;Temporal Depth is a third parameter of a time series data. For example if you have a video clip of length 25 frames and on training a model you are giving first five frames with respect to time. Your temporal depth will be 5.&lt;/p&gt;&#xA;" OwnerUserId="31910" LastActivityDate="2020-02-15T09:54:08.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18036" PostTypeId="1" CreationDate="2020-02-15T14:12:59.010" Score="3" ViewCount="49" Body="&lt;p&gt;I have been reading literature on reinforcement learning in healthcare. I am slightly confused between the policy evaluation for both &lt;strong&gt;&lt;em&gt;SARSA&lt;/em&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;em&gt;Q-learning&lt;/em&gt;&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge, I believe that SARSA is used for policy evaluation, to find the Q values of following an already existing policy. This is usually the clinician's policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q - learning on the other hand seeks to find another policy, different from the clinician's such that the policy learned at different states always maximise the Q - values. This leads to a better treatment policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the Q values are learned from both policies, if the Q values for Q - learning are higher than those of SARSA's, can we say that the policy learned from Q - learning is better than that of the clinician's ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From readings I have found out that computing the state - value function is usually used to compare how good policies are. I believe that new data has to be generated to apply the policy learned from Q - learning and compute the state - value function for following this learnt policy from Q - learning.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why can't the Q values learnt from SARSA and Q - learning be used as comparison instead ? Also, for model free approaches (eg. continuous state space), how is policy evaluation usually carried out ?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-02-15T14:32:18.013" LastActivityDate="2020-02-15T14:32:18.013" Title="Evaluation a policy learned using Q - learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;healthcare&gt;&lt;sarsa&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18038" PostTypeId="1" CreationDate="2020-02-15T18:47:34.243" Score="1" ViewCount="24" Body="&lt;p&gt;Can I apply experience replay on naive actor-critic directly? Should it work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried that but unfortunately it didn't work.&lt;/p&gt;&#xA;" OwnerUserId="33555" LastEditorUserId="33555" LastEditDate="2020-02-16T01:46:15.237" LastActivityDate="2020-02-16T01:46:15.237" Title="Can I apply experience on naive actor critic directly? Should it work?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18039" PostTypeId="1" CreationDate="2020-02-15T18:52:53.767" Score="1" ViewCount="91" Body="&lt;p&gt;I have been trying to implement the ACER algorithm for continuous action spaces in reinforcement learning. The paper for the algorithm can be found here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1611.01224.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sample Efficient Actor-Critic with Experience Replay&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have implemented parts of the algorithm, but I have encountered some roadblocks that I have not been able to figure out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is the pseudo-code provided in the paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JLyjh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JLyjh.png&quot; alt=&quot;Algorithm from Paper&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I have implemented so far:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;states = tf.convert_to_tensor(trajectory.state)&#xA;actions = tf.squeeze(tf.convert_to_tensor(trajectory.action), axis=-1)&#xA;rewards = tf.convert_to_tensor(trajectory.reward)&#xA;dones = tf.convert_to_tensor(trajectory.done)&#xA;&#xA;explore_means, state_values, action_values = actor_critic(states, actions)&#xA;&#xA;average_means, *_ = brain.average_actor_critic(states)&#xA;&#xA;k = len(trajectory.state)&#xA;d = env.action_space.shape[0]&#xA;&#xA;# Policies&#xA;explore_policies = k*[None]&#xA;behavior_policies = k*[None]&#xA;average_policies = k*[None]&#xA;&#xA;# Tracking&#xA;explore_actions = np.zeros([k, d])&#xA;importance_weights = np.zeros([k, 1])&#xA;explore_importance_weights = np.zeros([k, 1])&#xA;truncation_parameters = np.zeros([k, 1])&#xA;&#xA;for i in range(k):&#xA;&#xA;    behavior_policy = tfd.MultivariateNormalDiag(&#xA;        loc=trajectory.statistics[i],&#xA;        scale_diag=tf.ones(d)*POLICY_STD&#xA;    ) &#xA;&#xA;    explore_policy = tfd.MultivariateNormalDiag(&#xA;        loc=explore_means[i],&#xA;        scale_diag=tf.ones(d)*POLICY_STD&#xA;    ) &#xA;&#xA;    average_policy = tfd.MultivariateNormalDiag(&#xA;        loc=average_means[i],&#xA;        scale_diag=tf.ones(d)*POLICY_STD&#xA;    )&#xA;&#xA;    explore_action = explore_policy.sample()&#xA;&#xA;    importance_weight = explore_policy.prob(actions[i]) / behavior_policy.prob(actions[i])&#xA;    explore_importance_weight = explore_policy.prob(explore_action) / behavior_policy.prob(explore_action)&#xA;&#xA;    truncation_parameter = min(1, (importance_weight)**d)&#xA;&#xA;&#xA;    behavior_policies[i] = behavior_policy&#xA;    explore_policies[i] = explore_policy&#xA;    average_policies[i] = average_policy&#xA;    explore_actions[i] = explore_action&#xA;    importance_weights[i] = importance_weight&#xA;    explore_importance_weights[i] = explore_importance_weight&#xA;    truncation_parameters[i] = truncation_parameter&#xA;&#xA;&#xA;explore_actions = tf.convert_to_tensor(explore_actions, dtype=tf.float32)&#xA;importance_weights = tf.convert_to_tensor(importance_weights, dtype=tf.float32)&#xA;explore_importance_weights = tf.convert_to_tensor(explore_importance_weights, dtype=tf.float32)&#xA;truncation_parameters = tf.convert_to_tensor(truncation_parameters, dtype=tf.float32)&#xA;&#xA;&#xA;q_ret = values[-1] if not dones[-1] else tf.zeros(1)&#xA;q_opc = tf.identity(q_ret)&#xA;&#xA;for i in reversed(range(k - 1)):&#xA;&#xA;    q_ret = rewards[i] + GAMMA*q_ret&#xA;    q_opc = rewards[i] + GAMMA*q_opc&#xA;&#xA;&#xA;    # Compute quantities needed for trust region updating&#xA;    c = TRUNCATION_PARAMETER&#xA;&#xA;    with tf.GradientTape(persistent=True) as tape:&#xA;&#xA;        tape.watch(explore_policies[-2].loc)&#xA;&#xA;        log_prob = explore_policies[-2].log_prob(actions[-2])&#xA;        explore_log_prob = explore_policies[-2].log_prob(explore_actions[-2])&#xA;&#xA;        kl_div = tfp.distributions.kl_divergence(average_policies[-2], explore_policies[-2])&#xA;&#xA;&#xA;    lp_grad = tape.gradient(log_prob, explore_policies[-2].loc)    &#xA;    elp_grad = tape.gradient(explore_log_prob, explore_policies[-2].loc) &#xA;    kld_grad = tape.gradient(kl_div, explore_policies[-2].loc) &#xA;&#xA;&#xA;    term1 = min(c, importance_weights[-2])*lp_grad*(q_opc - state_values[-2])&#xA;    term2 = tf.nn.relu(1 - (c / explore_importance_weights[-2]))*(action_values[-2] - state_values[-2])*elp_grad&#xA;&#xA;    g = term1 + term2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the goal here was to implement it exactly the way they have it in the paper and then afterwards optimize it for doing batches of trajectories. For now, however, it is sufficient for the purposes of learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusion comes from the use of differentials in this algorithm. I don't know what the specific type is for them, such as whether they are using it for the loss value to optimize on or if they are storing the gradients that will be used for updating. Another issue I am having is that it is not clear what they mean by this line:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8ZhqY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8ZhqY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why they are using a partial derivative here if there is clearly more than one parameter in the neural network. Maybe they mean the gradient I am not sure, however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what would be helpful is if anybody has some guidance as to what they are getting at in this portion of the paper or if anybody has some advice as to what steps need to be taken in TensorFlow 2.0 to implement this algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated! Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="33556" LastActivityDate="2020-02-15T18:52:53.767" Title="Implementing Actor-Critic with Experience Replay for Continuous Action Spaces" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18043" PostTypeId="1" CreationDate="2020-02-16T01:27:30.803" Score="1" ViewCount="17" Body="&lt;p&gt;Suppose we have a top down picture of an object (let's say it is a shoe) from an overhead camera. Also suppose we have a database of various objects from a closeup camera. If we feed the top-down picture of the hsoe into a CBIR model, then this picture would obviously have very low similarity with the images in the database. But would the image with the highest similarity score in the database still be a shoe even though the absolute score is small?&lt;/p&gt;&#xA;" OwnerUserId="33559" LastActivityDate="2020-02-16T01:27:30.803" Title="Similarity of Images (CBIR) for two different cameras" Tags="&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18047" PostTypeId="1" CreationDate="2020-02-16T15:56:24.150" Score="1" ViewCount="33" Body="&lt;p&gt;I have used Deezer Spleeter but it produces echoes aside the stems, so I wonder if there is already an AI that remove echoes noises.&lt;/p&gt;&#xA;" OwnerUserId="33571" LastActivityDate="2020-02-16T15:56:24.150" Title="Is there an AI that can complete Deezer Spleeter work?" Tags="&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18049" PostTypeId="1" AcceptedAnswerId="18051" CreationDate="2020-02-16T17:40:08.207" Score="1" ViewCount="38" Body="&lt;p&gt;What is the purpose of the arrow &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; in the formula below? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(S_t) \leftarrow V(S_t) + \alpha \left[ G_t - V(S_t) \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume it's not the same as 'equals'.&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-16T20:09:23.703" LastActivityDate="2020-02-16T20:09:23.703" Title="What is the purpose of the arrow $\leftarrow$ in this formula?" Tags="&lt;reinforcement-learning&gt;&lt;notation&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18050" PostTypeId="2" ParentId="6503" CreationDate="2020-02-16T17:40:27.673" Score="0" Body="&lt;p&gt;If you want to decrease the accuracy given the same optimizer/epochs/batch, you could add more layers and increase the number of parameters, it should now take a bit longer to converge. Hence for the same epochs you would get lower accuracy. You could also initialise your parameters in a non sensical way. &lt;/p&gt;&#xA;" OwnerUserId="33573" LastActivityDate="2020-02-16T17:40:27.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18051" PostTypeId="2" ParentId="18049" CreationDate="2020-02-16T17:44:46.053" Score="2" Body="&lt;p&gt;I assume this is an iterative function. It means the current &lt;span class=&quot;math-container&quot;&gt;$V(S_t)$&lt;/span&gt; &lt;strong&gt;is&lt;/strong&gt; the sum of the previous plus some adjustment. The arrow is like an &lt;strong&gt;assignment&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In code, you would do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vst = vst + alpha * (gt - vst)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So &lt;code&gt;vst&lt;/code&gt; will be overwritten. &lt;/p&gt;&#xA;" OwnerUserId="33573" LastEditorUserId="2444" LastEditDate="2020-02-16T20:07:49.210" LastActivityDate="2020-02-16T20:07:49.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18052" PostTypeId="2" ParentId="17155" CreationDate="2020-02-16T17:54:19.460" Score="0" Body="&lt;p&gt;You could use &lt;code&gt;np.random.choice&lt;/code&gt; to shuffle the arrays. You could use a distance metric to find new arrays that are mutants of the the current good set. &lt;/p&gt;&#xA;" OwnerUserId="33573" LastActivityDate="2020-02-16T17:54:19.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18054" PostTypeId="2" ParentId="12465" CreationDate="2020-02-16T18:04:33.660" Score="0" Body="&lt;p&gt;You can also fit trigonometric functions like sin and cos. eg check this &lt;a href=&quot;https://ms.mcmaster.ca/~bolker/eeid/2010/Ecology/Spectral.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;doc&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33573" LastActivityDate="2020-02-16T18:04:33.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18055" PostTypeId="1" AcceptedAnswerId="18056" CreationDate="2020-02-16T19:58:04.347" Score="2" ViewCount="49" Body="&lt;p&gt;I am making an MNIST classifier. I am using categorical cross-entropy as my loss function. I want to make it so that if the correct label is 3, then it will penalize the model less heavily if it classifies a 4 than a 7 because 4 is closer numerically to 3 than 7 is. How do I do this?&lt;/p&gt;&#xA;" OwnerUserId="29708" LastEditorUserId="2444" LastEditDate="2020-02-17T17:59:32.073" LastActivityDate="2020-02-17T17:59:32.073" Title="How should I penalize the model proportionally to the error?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;loss-functions&gt;&lt;cross-entropy&gt;&lt;categorical-crossentropy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18056" PostTypeId="2" ParentId="18055" CreationDate="2020-02-16T21:00:29.850" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to make it so that if the correct label is 3, then it will penalize the model less heavily if it classifies a 4 than a 7 because 4 is closer numerically to 3 than 7 is. How do I do this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Really you should not, because the symbols used (Arabic numerals) do not have direct relation to quantity in the same way e.g. tally counts or dots do. They are good candidates for classification, and despite the conventional mapping to quantity when you read them, the symbols themselves are poor candidates for regression, because for instance the symbols &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; do not differ in a way that captures quantity in any intuitive manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you are keen to do this, it is relatively simple to construct a suitable loss function in most auto-differentiating frameworks. You will need to read up on how to do so. For instance, &lt;a href=&quot;https://stackoverflow.com/a/43821374/2001031&quot;&gt;here is a Stack Overflow answer explaining where to start with writing custom loss function in Keras&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for your loss function to work, it will need to be differentiable and  smoothly changing as predictions get better. That rules out using any form of &lt;code&gt;argmax&lt;/code&gt; for the current prediction. If you want to stick with softmax for the final layer, then I suggest using a mean squared error against the &lt;em&gt;expected&lt;/em&gt; prediction, e.g. if &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; is the numerical digit for example &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_{i,j}$&lt;/span&gt; is the ground truth expressed as a one-hot vector, where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the example and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is the digit class, then &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_{i,j}$&lt;/span&gt; is the probability predicted by your model. You could use &lt;span class=&quot;math-container&quot;&gt;$\hat{d}_i = \sum_{j=0}^9 j\hat{y}_{i,j}$&lt;/span&gt; for the expected value and MSE loss of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(d_i,\hat{d}_i) = \frac{1}{2}(\hat{d}_i - d_i)^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use a weighted sum of the MSE loss and cross entropy loss as your final loss, with the balance between the two losses being a new hyperparameter of your model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note this solution makes &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; close to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; but far away from &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt;. If you want the digits to be considered close on a cycle (e.g. &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; is closer to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; than it is to &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;) the you will need something more creative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst I don't think this will help you discover any improvements to MNIST classification, combining two or more loss functions to achieve a more complex goal can be really useful sometimes, so it is a skill worth practicing.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-02-16T21:13:23.413" LastActivityDate="2020-02-16T21:13:23.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18057" PostTypeId="2" ParentId="18008" CreationDate="2020-02-16T21:20:36.393" Score="1" Body="&lt;p&gt;You discovered already one solution for your problem: Zero-Padding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two other common possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using Recurrent NNs&lt;br /&gt;This is often used at text processing, where you feed each word one after another into your model.&lt;/li&gt;&#xA;&lt;li&gt;Using Recursive NNs (I wont recommend this for your use case)&lt;br /&gt;This method is also frequently used in word processing, but is more often applied in the semantic analysis of text. You reduce the text to the essential, until it has reached the desired length. However, information is lost in this process.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23012" LastActivityDate="2020-02-16T21:20:36.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18058" PostTypeId="1" AcceptedAnswerId="18064" CreationDate="2020-02-16T21:33:55.247" Score="2" ViewCount="272" Body="&lt;p&gt;Is there an ideal ratio in reinforcement learning between the positive and negative rewards?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have the scenario of moving a robot across the river. There are two options, walk across the bridge or walk across the river. If it walks across the river then the robot breaks so the idea is to reinforce the robot to walk across the bridge. What would be the best rewards values? Does this ratio vary between cases?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;option1:&#xA;&#xA;Bridge: +10&#xA;River: -10&#xA;&#xA;Option2:&#xA;&#xA;Bridge: +10&#xA;River: -1&#xA;&#xA;Option3:&#xA;&#xA;Bridge: +1&#xA;River: -10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33579" LastActivityDate="2020-02-17T04:37:29.203" Title="Is there a good ratio between the positive and negative rewards in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18060" PostTypeId="5" CreationDate="2020-02-17T01:50:49.267" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-17T01:50:49.267" LastActivityDate="2020-02-17T01:50:49.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18061" PostTypeId="4" CreationDate="2020-02-17T01:50:49.267" Score="0" Body="For questions related to genetic operators, such as the selection, crossover or mutation operators." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-17T01:50:49.267" LastActivityDate="2020-02-17T01:50:49.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18062" PostTypeId="2" ParentId="18058" CreationDate="2020-02-17T03:47:44.270" Score="1" Body="&lt;p&gt;It usually does not matter, but I'm sure there are situations where it could matter.  In theory, if a reward for good behavior is higher than the rewards for bad behavior, then the neural network will be trained such that the higher rewards are preferred, even if those higher rewards are negative.  For example, if a bad reward is -100, then a relatively good reward could be -50, and the network will then be more likely to choose the action with -50 reward over the action with -100 reward.&lt;/p&gt;&#xA;" OwnerUserId="27169" LastActivityDate="2020-02-17T03:47:44.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18063" PostTypeId="1" CreationDate="2020-02-17T04:04:28.113" Score="0" ViewCount="23" Body="&lt;p&gt;I've been following the Berkeley cs188's assignment (I'm not taking the course). Currently, they don't show the solution in the gradescope unless I get it correct. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/uMTm4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uMTm4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My reasoning was&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V^*(a)$&lt;/span&gt; = 10 fixed, because the optimal action is to terminate and receive the reward 10. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V^*(b) = 10 \times 0.2 = 2$&lt;/span&gt; using Bellman optimality eqn &lt;span class=&quot;math-container&quot;&gt;$V^*(s) = R(s)+ \gamma \ \rm{max}_{a} \sum_{s'} P(s'|s,a) V^*(s')$&lt;/span&gt;, where the optimal actionfrom b is to left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, I get &lt;span class=&quot;math-container&quot;&gt;$V^*(c) = 10 \times (0.2)^2 = 0.4$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the state &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, it is optimal to move to the right and exit at &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; to receive 1, therefore &lt;span class=&quot;math-container&quot;&gt;$V^*(d) = 1 \times 0.2 = 0.2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;span class=&quot;math-container&quot;&gt;$V^*(e) = 1$&lt;/span&gt; fixed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there autograder says it's incorrect and doesn't show explanation. Can anyone explain what the right approach or answer is?  &lt;/p&gt;&#xA;" OwnerUserId="33583" LastActivityDate="2020-02-17T05:25:40.757" Title="Unable to understand V* at infinite time horizon using Bellman equation for solving MDP" Tags="&lt;markov-decision-process&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18064" PostTypeId="2" ParentId="18058" CreationDate="2020-02-17T04:37:29.203" Score="1" Body="&lt;p&gt;There are no hard and fast rules for it. Your reward should be such that it motivates the agent to attain the goal in the most effective way.  In a grid world if you want your agent to reach goal state quicker but award +2 for a move and +5 for reaching goal then your agent might simply wander around and never reach the goal. However, if you set a reward of -1 for each move and  +1 or +10(or even 0) for reaching the goal then your agent will learn to reach the goal state faster.&lt;/p&gt;&#xA;" OwnerUserId="31749" LastActivityDate="2020-02-17T04:37:29.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18065" PostTypeId="2" ParentId="18063" CreationDate="2020-02-17T05:25:40.757" Score="1" Body="&lt;p&gt;Nevermind. I found that above answer is indeed correct, but the gradescope has a bug (it requires the format to be .2 instead of 0.2).&lt;/p&gt;&#xA;" OwnerUserId="33583" LastActivityDate="2020-02-17T05:25:40.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18066" PostTypeId="1" AcceptedAnswerId="18140" CreationDate="2020-02-17T06:37:55.823" Score="5" ViewCount="222" Body="&lt;p&gt;Is there either an empirical or theoretical reason that actor-critic algorithms with eligibility traces have not been more fully explored? I was hoping to find a paper or implementation or both for continuous tasks (not episodic) in continuous state-action spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has been the only related question on SE-AI that I have been able to find &lt;a href=&quot;https://ai.stackexchange.com/q/10049/2444&quot;&gt;Why are lambda returns so rarely used in policy gradients?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I appreciated the dialog and found it useful, I was wondering if there was any further detail or reasoning that might help explain the void.&lt;/p&gt;&#xA;" OwnerUserId="32929" LastEditorUserId="2444" LastEditDate="2020-06-04T17:23:33.093" LastActivityDate="2020-06-04T17:23:33.093" Title="Why not more TD() in actor-critic algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;temporal-difference-methods&gt;&lt;eligibility-traces&gt;&lt;td-lambda&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18067" PostTypeId="1" CreationDate="2020-02-17T07:10:48.970" Score="1" ViewCount="27" Body="&lt;p&gt;I'm still a bit new to deep learning. What I'm still struggling, is what is the best practice in re-training a good model over time? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've trained a deep model for my binary classification problem (fire vs non-fire) in &lt;code&gt;Keras&lt;/code&gt;. I have 4K fire images and 8K non-fire images (they are video frames). I train with 0.2/0.8 validation/training split. Now I test it on some videos, and I found some false positives. I add those to my negative (non-fire) set, load the best previous model, and retrain for 100 epochs. Among those 100 models, I take the one with lowest &lt;code&gt;val_loss&lt;/code&gt; value. But when I test it on the same video, while those false positives are gone, new ones are introduced! This never ends, and Idk if I'm missing something or am doing something wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I know which of the resulting models is the best? What is the best practice in training/re-training a good model? How should I evaluate my models?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my simple model architecture if it helps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_model():&#xA;  model = Sequential()&#xA;  model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', input_shape=(300, 300, 3)))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))&#xA;  model.add(MaxPooling2D(pool_size=(2,2)))&#xA;  model.add(BatchNormalization())&#xA;  model.add(Dropout(0.2))&#xA;  model.add(Flatten())&#xA;  model.add(Dense(256, activation='relu'))&#xA;  model.add(Dropout(0.2))&#xA;  model.add(Dense(64, activation='relu'))&#xA;  model.add(Dense(2, activation = 'softmax'))&#xA;&#xA;  return model&#xA;&#xA;#....&#xA;if retrain_from_prior_model == False:&#xA;    model = create_model()&#xA;    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;else:&#xA;    model = load_model(&quot;checkpoints/model.h5&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-02-17T07:10:48.970" Title="Steps to train and re-train a good model" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;deep-neural-networks&gt;&lt;metric&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18068" PostTypeId="1" AcceptedAnswerId="21790" CreationDate="2020-02-17T09:37:06.227" Score="1" ViewCount="78" Body="&lt;p&gt;I have, say, a (balanced) data-set with 2k images for binary classification. What I have done is that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;randomly divided the data-set into 5 folds;&lt;/li&gt;&#xA;&lt;li&gt;copy-pasted all 5-fold data-set to have 5 exact copies of data-set (folder_1 to folder_5, all absolutely same data-set)&lt;/li&gt;&#xA;&lt;li&gt;first fold in folder_1 is saved as &lt;code&gt;test&lt;/code&gt; folder and remaining (fold_2, fold_3, fold_4, fold_5) are combined as one &lt;code&gt;train&lt;/code&gt; folder&lt;/li&gt;&#xA;&lt;li&gt;second fold in folder_2 is saved as &lt;code&gt;test&lt;/code&gt; folder and remaining (namely, fold_1, fold_3, fold_4, fold_5) are combined as one &lt;code&gt;train&lt;/code&gt; folder&lt;/li&gt;&#xA;&lt;li&gt;third fold in folder_3 is saved as &lt;code&gt;test&lt;/code&gt; folder and remaining (namely, fold_1, fold_2, fold_4, fold_5) are combined as one &lt;code&gt;train&lt;/code&gt; folder.&lt;/li&gt;&#xA;&lt;li&gt;similar process has been done on folder_4 and foder_5.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I hope, by now, you got the idea of how I distributed the data-set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I did so is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have augmented the training data (&lt;code&gt;train&lt;/code&gt; folder) in each of the folders and used &lt;code&gt;test&lt;/code&gt; folders respectively to evaluate (ROC-AUC score). Now I kind of have 5 ROC-AUC scores which I evaluated using &lt;code&gt;test&lt;/code&gt; folders. If I get the average value out of those 5 scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Assuming the above cross-validation process is done right) If I were to perform some manual hyperparameter optimizations (like an optimizer, learning rate, batch size, dropout, activation) and perform the above &lt;strong&gt;&lt;em&gt;cross-validation with data augmentation&lt;/em&gt;&lt;/strong&gt; and find the best so-called &quot;mean ROC-AUC&quot;, &lt;strong&gt;does it mean I successfully conducted hyper-parameter optimization&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FYI:&lt;/strong&gt; I have no problem with computing power OR/AND time at all to loop through the hyper-parameters for this type of &lt;strong&gt;&lt;em&gt;cross-validation with data augmentation&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="2444" LastEditDate="2020-06-10T13:26:12.737" LastActivityDate="2020-06-10T21:50:11.137" Title="How to fairly conduct a model performance with 5-fold cross validation after augmentation?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;hyperparameter-optimization&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18069" PostTypeId="1" AcceptedAnswerId="18153" CreationDate="2020-02-17T10:10:47.597" Score="1" ViewCount="59" Body="&lt;p&gt;Generally, CNNs are used to extract feature representations of an image. I'm right now dealing with the class of CNN that produces saliency maps, which are generally in the format of a mask. I'm trying to generate a feature representation of that specific Mask. What could be the best way to approach this problem?&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2020-02-17T12:40:53.620" LastActivityDate="2020-02-20T23:50:18.353" Title="How do I generate a feature representation of a saliency map (or mask)?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18072" PostTypeId="2" ParentId="10133" CreationDate="2020-02-17T13:44:26.610" Score="1" Body="&lt;p&gt;Sentences (for those tasks such as NLI which take two sentences as input) are differentiated in two ways in BERT:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, a &lt;code&gt;[SEP]&lt;/code&gt; token is put between them&lt;/li&gt;&#xA;&lt;li&gt;Second, a learned embedding &lt;span class=&quot;math-container&quot;&gt;$E_A$&lt;/span&gt; is concatenated to every token of the first sentence, and another learned vector &lt;span class=&quot;math-container&quot;&gt;$E_B$&lt;/span&gt; to every token of the second one&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That is, there are just &lt;strong&gt;two&lt;/strong&gt; possible &quot;segment embeddings&quot; (for the first sentence and for the second one). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Positional embeddings are learned vectors for every possible position between 0 and 512-1. Transformers don't have a sequential nature as recurrent neural networks, so some information about the order of the input is needed; if you disregard this, your output will be permutation-invariant.&lt;/p&gt;&#xA;" OwnerUserId="33595" LastActivityDate="2020-02-17T13:44:26.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18073" PostTypeId="1" CreationDate="2020-02-17T13:53:20.920" Score="0" ViewCount="68" Body="&lt;p&gt;I have a CNN architecture for CIFAR-10 dataset which is as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Convolutions:           64, 64, pool&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Fully Connected Layers: 256, 256, 10&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Batch size:             60&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Optimizer:              Adam(2e-4)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Loss:                   Categorical Cross-Entropy&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When I train this model, training and testing accuracy along with loss has a very jittery behavior and does not converge properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the defined architecture correct? Should I have a max-pooling layer after every convolution layer?&lt;/p&gt;&#xA;" OwnerUserId="31215" LastEditorUserId="2444" LastEditDate="2020-03-27T13:29:09.933" LastActivityDate="2020-08-24T15:03:05.260" Title="Is this neural network architecture appropriate for CIFAR-10?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-10-31T10:41:33.273" ContentLicense="CC BY-SA 4.0" />
  <row Id="18074" PostTypeId="1" AcceptedAnswerId="18076" CreationDate="2020-02-17T16:26:13.843" Score="4" ViewCount="185" Body="&lt;p&gt;Can anyone point me in the direction of a nice graph that depicts the &quot;family tree&quot;, or hierarchy, of RL algorithms (or models)? For example, it splits the learning into TD and Monte Carlo methods, under which is listed all of the algorithms with their respective umbrella terms. Beneath each algorithm is shown modifications to those algorithms, etc. I'm having difficulty picturing where everything lies within the RL landscape.&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-17T18:02:00.283" LastActivityDate="2020-02-17T22:27:00.977" Title="Is there a family tree for reinforcement learning algorithms?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18075" PostTypeId="1" CreationDate="2020-02-17T17:20:34.553" Score="1" ViewCount="45" Body="&lt;p&gt;I've been working with this neural style paper &lt;a href=&quot;https://arxiv.org/pdf/1508.06576v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1508.06576v2.pdf&lt;/a&gt;&#xA;to try and transfer the style from this image to photos of pets. In case you're not familiar with the technique, I'll leave a brief explanation of it at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s4md9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s4md9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After taking the time to understand some of the concepts in the paper I've come to my own conclusion that the method won't work. Here's why I think this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The style is not localised/fine-grained enough. If I take a small piece of this image, I might just get a solid color, or two colors separated by a zig-zag boundary. So the first few convolutional layers won't find any characteristics of interest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The style depends on &lt;em&gt;long-distance correlations&lt;/em&gt; (I made that term up). If you follow some of theses zig-zag interfaces with your eyes you'll see that they traverse up to 1/3 of the characteristic size of the image. So bunches of pixels that take up a lot of space are correlated via the style. And from my intuition of CNNs, you can distill lots of pixels into coarser information (like &quot;this is a dog&quot;) but you can't really go backwards. I don't think the CNN encodes the required logic to trace out a zig-zag over a long distance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The color palette is restricted. I'm not sure why, but my intuition tells me the Neural Style Transfer technique won't be able to produce a restricted color pallet like in the style example. I'm guessing again, that the CNN doesn't encode any logic around the number of colors used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So my question is around whether or not the technique could work, and if not, what's a better technique for this problem.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(optional read) Summary of deep style transfer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take a pre-trained model like VGG19.&lt;/li&gt;&#xA;&lt;li&gt;Feed in a photo. For each conv layer you can reconstruct a &lt;em&gt;content representation&lt;/em&gt; of what that layer encodes about the photo. You do this by treating the feature maps of that layer as a desired output, and doing gradient descent on a white noise image as the input, where the loss function is the RMS between the original photo and the generated image.&lt;/li&gt;&#xA;&lt;li&gt;Feed in a painting. For each conv layer you can reconstruct a &lt;em&gt;style representation&lt;/em&gt; of what that layer encodes about the painting in a similar way as step 2. This time though, your loss function is the RMS between the gram matrix of all the features maps produced using a white noise input, and the gram matrix of all the feature maps produced using the painting as input. And here, you sum loss over all features maps of prior layers as well, not just the layer you are considering now.&lt;/li&gt;&#xA;&lt;li&gt;Jointly minimise the loss functions described in 2 and 3 (so you're minimising &lt;em&gt;content loss&lt;/em&gt; and &lt;em&gt;style loss&lt;/em&gt; together) in order to produce an image with the content of the photo and the style of the painting.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried this. Here is an example of my results (left is input, right is output). So it's kind of cool to see some color map reduction happening, and what kind of looks like accentuation of texture, but definitely not getting these illustrated zig zag boundaries that the human mind so readily perceives as fur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2oLrK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2oLrK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2020-02-17T17:30:44.570" LastActivityDate="2020-02-17T17:30:44.570" Title="Can neural style transfer work on the image style in this question or is there a better technique?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18076" PostTypeId="2" ParentId="18074" CreationDate="2020-02-17T20:59:09.300" Score="5" Body="&lt;p&gt;I highly recommend looking at &lt;a href=&quot;http://www.incompleteideas.net/book/RLbook2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; by Richard Sutton and Andrew Barto.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In it they write:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reinforcement learning, like many topics whose names end with “ing,” such as machine&#xA;  learning and mountaineering, is simultaneously a problem, a class of solution methods&#xA;  that work well on the problem, and the field that studies this problem and its solution&#xA;  methods. It is convenient to use a single name for all three things, but at the same time&#xA;  essential to keep the three conceptually separate. In particular, the distinction between&#xA;  problems and solution methods is very important in reinforcement learning; failing to&#xA;  make this distinction is the source of many confusions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On the solution side of things, rather than a family tree of reinforcement learning, there is a spectrum of different approaches to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PqupD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PqupD.png&quot; alt=&quot;A slice through the space of RL methods, highlighting the two of the most important dimensions explored in the Introduction of Reinforcement Learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the points within the spectrum above are TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) methods. When the state space is large, one might use a neural network to help generalize across similar states, such as DQN for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be noted that there are many applications of reinforcement learning algorithms that do not involve training an agent to perform a task. Instead one might want to evaluate an existing agent or use a general value function to predict an event using an agent's policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, many popular problems consist of training an agent to perform a task. These algorithms can be categorized into a &quot;family tree&quot; based either on what values they are updating or on what problem they are tackling. (Source : &lt;a href=&quot;https://github.com/NervanaSystems/coach&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NervanaSystems/coach&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On the left are Value Based methods which update the value function of a policy and find policies that optimize said value function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To the right of those are policy gradient methods which optimize for the policies directly without explicitly finding a value function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To the far right is Imitation Learning whereby one wants to copy a policy given demonstrations of an existing but unacceptable policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And in the middle is Direct Future Prediction which is its own weird beast that I found out while finding that picture. Here is the &lt;a href=&quot;https://arxiv.org/pdf/1611.01779.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv paper&lt;/a&gt; that describes DFP.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uKOTj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uKOTj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2020-02-17T22:27:00.977" LastActivityDate="2020-02-17T22:27:00.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18077" PostTypeId="1" CreationDate="2020-02-17T21:32:23.717" Score="0" ViewCount="47" Body="&lt;p&gt;Assume the existence of a Markov Decision Process consisting of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;State space &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Action space &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Transition model &lt;span class=&quot;math-container&quot;&gt;$T: S \times A \times S \to [0,1]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Reward function &lt;span class=&quot;math-container&quot;&gt;$R: S \times A \times S \to \mathbb{R}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Distribution of initial state &lt;span class=&quot;math-container&quot;&gt;$p_0: S \to [0,1]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and a policy &lt;span class=&quot;math-container&quot;&gt;$\pi:S\to A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The V- and Q-functions take expectations of the sum of future rewards. Let's start off by &lt;span class=&quot;math-container&quot;&gt;$r_0:= R(x_0,\pi(x_0),x_1)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the current policy while &lt;span class=&quot;math-container&quot;&gt;$x_0 \sim p_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_1 \sim T(x_0,\pi(x_0),-)$&lt;/span&gt; are random variables. With setting &lt;span class=&quot;math-container&quot;&gt;$\mu_i:= T(x_i,\pi(x_i),-),\rho_i:=R(x_i,\pi(x_i),-)$&lt;/span&gt; I obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E[r_0]= \int_{\mathbb{R}} r d\mu_0^{\rho_0} = \int_S R(x_0,\pi(x_0),-)d\mu_0$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mu_i^{\rho_i}:= \mu_i\circ \rho_i^{-1}$&lt;/span&gt; is the pushforward of &lt;span class=&quot;math-container&quot;&gt;$\mu_i$&lt;/span&gt; under random variable &lt;span class=&quot;math-container&quot;&gt;$\rho_i$&lt;/span&gt;. But the above quantity still depends on &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; as both &lt;span class=&quot;math-container&quot;&gt;$\mu_o$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_0$&lt;/span&gt; depend on &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;. Intuitively, I would guess that one has to calculate the integral over every occuring random variable to obtain the overall expectation, i.e. &lt;span class=&quot;math-container&quot;&gt;$E[r_0]= \int_S\int_S R(x_0,\pi(x_0),x_1)d\mu_0(x_1)dp_0(x_0)$&lt;/span&gt; is that correct ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the &lt;span class=&quot;math-container&quot;&gt;$V-$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q-$&lt;/span&gt;functions take the expectation over the sum &lt;span class=&quot;math-container&quot;&gt;$R_{\tau} = \sum^T_{t=\tau}\gamma^{t-\tau}r_t$&lt;/span&gt;, where the instant of termination &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; itself is a random variable, and besides that the agent does not know its distribution it is not even included in the MDP model. How can I take the expectation over a sum where the number of summands is random. We cannot just calculate &lt;span class=&quot;math-container&quot;&gt;$\sum^{E[T]}(\dots)$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$E[T]$&lt;/span&gt; might not even be integer.&lt;/p&gt;&#xA;" OwnerUserId="27047" LastEditorUserId="2444" LastEditDate="2020-02-18T21:59:16.687" LastActivityDate="2020-02-18T21:59:16.687" Title="Understanding V- and Q-functions" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;math&gt;&lt;value-functions&gt;&lt;statistics&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18078" PostTypeId="6" CreationDate="2020-02-17T21:55:15.563" Score="0" Body="&lt;p&gt;I am running for a moderator position because I can further help the community with the moderator privileges. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Who am I?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The user&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/users?tab=Reputation&amp;amp;filter=all&quot;&gt;with the highest amount of reputation&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/users?tab=Editors&amp;amp;filter=all&quot;&gt;with the highest number of edits&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;that flagged and voted more than 1k times&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.meta.stackexchange.com/a/1319/2444&quot;&gt;that promoted the MathJax support on this site&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;that tries to follow the &lt;a href=&quot;https://ai.stackexchange.com/help/behavior&quot;&gt;expected behavior&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/conduct&quot;&gt;code of conduct&lt;/a&gt; policies&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;that is often &lt;a href=&quot;https://chat.stackexchange.com/rooms/43371/the-singularity&quot;&gt;in the main chat room&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;that holds a degree in computer science and soon in artificial intelligence too&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-17T21:55:15.563" LastActivityDate="2020-02-17T21:55:15.563" CommentCount="25" CommunityOwnedDate="2020-02-17T21:55:15.563" ContentLicense="CC BY-SA 4.0" />
  <row Id="18079" PostTypeId="6" CreationDate="2020-02-17T22:53:16.773" Score="0" Body="&lt;p&gt;I'm a computational linguist, and have been working in AI for the past 6 years (first a start-up and now a company developing conversational AI systems) after about 20 years of being a university lecturer. I am fairly active in checking out the review queues every day, and would be happy to take on more responsibilities if selected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While my reputation score might not be as high as some others', I have similar scores on various other stackexchange site, as I see myself as more of a generalist than a specialist in one particular area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main area of interest is NLP and related issues; I tend more towards symbolic AI than ML approaches, which &amp;mdash; however &amp;mdash; doesn't mean that I am dismissive of them. My impression is that many newcomers think deep learning is the cool silver bullet, when it is often not the best approach to solving a problem at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Happy respond to any questions asked in the comments!&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-02-23T18:42:16.137" LastActivityDate="2020-02-23T18:42:16.137" CommentCount="9" CommunityOwnedDate="2020-02-17T22:53:16.773" ContentLicense="CC BY-SA 4.0" />
  <row Id="18080" PostTypeId="1" AcceptedAnswerId="18092" CreationDate="2020-02-18T00:00:52.603" Score="4" ViewCount="162" Body="&lt;p&gt;Trying to get my head around model-free and model-based algorithms in RL. In my research, I've seen the search trees created via the minimax algorithm. I presume these trees can only be created with a model-based agent that knows the full environment/rules of the game (if it's a game)? If not, could you explain to me why?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-18T10:46:00.277" LastActivityDate="2020-02-18T11:25:59.907" Title="Is the minimax algorithm model-based?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;minimax&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18081" PostTypeId="1" CreationDate="2020-02-18T00:50:39.247" Score="3" ViewCount="111" Body="&lt;p&gt;I'm working on Adversarial Machine Learning, and have read multiple papers on this topic, some of them are mentioned as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Poisoning Attacks on SVMs: &lt;a href=&quot;https://arxiv.org/pdf/1206.6389.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1206.6389.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Adversarial Label Flips on Support Vector Machines&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I am not able to find any literature on data poisoning for SVMs using Manifold regularization. Is there anyone who has knowledge about that?&lt;/p&gt;&#xA;" OwnerUserId="31240" LastEditorUserId="31240" LastEditDate="2020-03-04T00:43:31.930" LastActivityDate="2021-04-08T13:07:25.720" Title="How do I poison an SVM with manifold regularization?" Tags="&lt;machine-learning&gt;&lt;regularization&gt;&lt;support-vector-machine&gt;&lt;adversarial-ml&gt;&lt;causation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18082" PostTypeId="1" CreationDate="2020-02-18T02:01:57.717" Score="1" ViewCount="51" Body="&lt;p&gt;Factset blackline reports essentially can compare two &lt;a href=&quot;https://www.investopedia.com/terms/1/10q.asp&quot; rel=&quot;nofollow noreferrer&quot;&gt;10-Q SEC filings&lt;/a&gt; and show you the difference between the two documents. It highlights added items in green and removed items in red + strikethrough (essentially, it's a document difference, but longer-term I would like to run algorithms on the differences).&lt;/p&gt;&#xA;&lt;p&gt;I don't care to change colors, but what I would like to do is to produce similar extracts that summarize addition and deletions.&lt;/p&gt;&#xA;&lt;p&gt;Which AI/ML algorithm could do the same?&lt;/p&gt;&#xA;" OwnerUserId="33606" LastEditorUserId="2444" LastEditDate="2020-07-22T11:33:20.213" LastActivityDate="2021-04-18T14:51:46.300" Title="How to produce documents like factset blackline?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18083" PostTypeId="1" AcceptedAnswerId="18086" CreationDate="2020-02-18T06:49:56.963" Score="1" ViewCount="28" Body="&lt;p&gt;Is it&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;number of units in a layer &lt;/li&gt;&#xA;&lt;li&gt;number of layers&lt;/li&gt;&#xA;&lt;li&gt;overall complexity of the network (both 1 and 2)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32856" LastEditorUserId="2193" LastEditDate="2020-02-18T10:25:45.673" LastActivityDate="2020-02-18T10:25:45.673" Title="In neural networks, what does the term depth generally mean?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18084" PostTypeId="1" CreationDate="2020-02-18T07:13:55.910" Score="1" ViewCount="186" Body="&lt;p&gt;It kind of makes sense intuitively but I'm not sure about a formal proof. I'll start with briefly listing definitions from &lt;strong&gt;Intro to Multiagent systems, Wooldridge, 2002&lt;/strong&gt; and then give you my reasoning attempts thus far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is a finite set of discrete, instantaneous states, &lt;span class=&quot;math-container&quot;&gt;$E=(e, e',...)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$Ac$&lt;/span&gt; is a repertoire of possible actions (also finite) available to an agent, which transform the environment, &lt;span class=&quot;math-container&quot;&gt;$Ac=(\alpha, \alpha', ...)$&lt;/span&gt;. A run is a sequence of interleaved environment states and actions, &lt;span class=&quot;math-container&quot;&gt;$r=(e_0, \alpha_0, e_1, \alpha_1,..., \alpha_{u-1}, e_u)$&lt;/span&gt;, set of all such possible finite sequences (over &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Ac$&lt;/span&gt;) is &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R^E$&lt;/span&gt; is a subset of &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; containing the runs that end with an env. state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Purely reactive agent is modeled as: &lt;span class=&quot;math-container&quot;&gt;$Ag_{pure}: E\mapsto Ac$&lt;/span&gt;, a standard agent is modeled as &lt;span class=&quot;math-container&quot;&gt;$Ag_{std}: R^E\mapsto Ac$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if &lt;span class=&quot;math-container&quot;&gt;$R^E$&lt;/span&gt; is a sequence of agent's actions and environment states, than it just makes sense that &lt;span class=&quot;math-container&quot;&gt;$E\subset R^E$&lt;/span&gt;. Hence, &lt;span class=&quot;math-container&quot;&gt;$Ag_{std}$&lt;/span&gt; can map to every action to which &lt;span class=&quot;math-container&quot;&gt;$Ag_{pure}$&lt;/span&gt; can. And behavioral equivalence with respect to environment &lt;span class=&quot;math-container&quot;&gt;$Env$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$R(Env, Ag_{1}) = R(Env, Ag_{2})$&lt;/span&gt;; where &lt;span class=&quot;math-container&quot;&gt;$Env=\langle E,e_{0},t \rangle$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$e_{0}$&lt;/span&gt; - initial environment state, &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; - transformation function (definition irrelevant for now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if &lt;span class=&quot;math-container&quot;&gt;$Ag_{pure}: E\mapsto Ac$&lt;/span&gt; &lt;strong&gt;and&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$Ag_{std}: R^E\mapsto Ac$&lt;/span&gt;, &lt;strong&gt;and&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$E\subset R^E$&lt;/span&gt;, we can say that &lt;span class=&quot;math-container&quot;&gt;$R(Env,Ag_{pure}) = R(Env, Ag_{std})$&lt;/span&gt; (might be too bold of an assumption). Hence, every purely reactive agent has behaviorally equivalent standard agent. The opposite might not be true, since &lt;span class=&quot;math-container&quot;&gt;$E\subset R^E$&lt;/span&gt; means that all elements of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; belong to &lt;span class=&quot;math-container&quot;&gt;$R^E$&lt;/span&gt;, while not all elements &lt;span class=&quot;math-container&quot;&gt;$R^E$&lt;/span&gt; belong to &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a textbook problem, but I couldn't find an answer key to check my solution. If anyone has formally (and perhaps mathematically) proven this before, can you post your feedback, thoughts, proofs in the comments? For instance, set of mathematical steps to infer &lt;span class=&quot;math-container&quot;&gt;$E\subset R^E$&lt;/span&gt; from their definitions: &lt;span class=&quot;math-container&quot;&gt;$E=(e_{0}, e_{1},..., e_{u})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R^E$&lt;/span&gt; is &quot;&lt;em&gt;all agent runs that end with an environment state&lt;/em&gt;&quot; (no formal equation found) is not clear to me.&lt;/p&gt;&#xA;" OwnerUserId="33611" LastEditorUserId="33611" LastEditDate="2020-02-19T06:17:58.807" LastActivityDate="2021-02-19T13:06:00.590" Title="Formal proof that every purely reactive agent has behaviorally equivalent standard agent" Tags="&lt;math&gt;&lt;multi-agent-systems&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18085" PostTypeId="1" AcceptedAnswerId="18089" CreationDate="2020-02-18T08:42:01.137" Score="2" ViewCount="120" Body="&lt;p&gt;I have been coming across visualizations showing that the neural nets tend to perform better as compared to the traditional machine learning algorithms (Linear regression, Log regression, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that we have sufficient data to train deep/neural nets, can we ignore the traditional machine learning topics and concentrate more on the neural network architectures?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the huge amount of data, are there any instances where traditional algorithms outperform neural nets?&lt;/p&gt;&#xA;" OwnerUserId="22456" LastEditorUserId="2444" LastEditDate="2020-05-29T23:54:00.950" LastActivityDate="2020-05-29T23:54:00.950" Title="Is traditional machine learning obsolete given that neural networks typically outperform them?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;statistical-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18086" PostTypeId="2" ParentId="18083" CreationDate="2020-02-18T08:57:00.923" Score="0" Body="&lt;p&gt;Very good question, I have heard it refered to all the 3 ways you described. I am not sure if there is a purely objective answer so I will answer with what I understand by &quot;depth&quot; generally:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If speaking of model architecture: the number of layers of the model.&lt;/li&gt;&#xA;&lt;li&gt;If speaking of CNN layers: the number of filters (not to be confused with the size of the kernel).&lt;/li&gt;&#xA;&lt;li&gt;If speaking of RNN layers: the number of time steps (temporal &quot;depth&quot;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;On the other hand I have never heard about depth regarding to FC layers. As I said the term &quot;depth&quot; is very subjective depending on the speakers. It should not be this way, and maybe theoretically is not, but I speak from what I heard in my experience. &lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-02-18T08:57:00.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18087" PostTypeId="1" CreationDate="2020-02-18T09:10:44.750" Score="1" ViewCount="40" Body="&lt;p&gt;Let's say I want to classify a dataset of handwritten digits (CNNs on their own can get 99.7% on the MNIST dataset but let's pretend they can only get 90% for the sake of this question). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I already have some classical computer vision techniques which might be able to give me a clue. For instance, I can count the intersection points of the pen stroke&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1,2,3,5,7 will usually have no intersection points&lt;/li&gt;&#xA;&lt;li&gt;6,9 will usually have one intersection point each&lt;/li&gt;&#xA;&lt;li&gt;4,8 will usually have two intersection points each (usually a 4-way crossover yields two intersection points which are close together)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So if I generate some meta-data telling me how many intersection points each sample has, how can I feed that into the CNN training so that it can take advantage of that knowledge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best guess is to slot it into the last fully connected layer just before classification.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-02-18T10:31:51.873" Title="Can I provide a CNN with hints?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18088" PostTypeId="1" CreationDate="2020-02-18T10:19:12.077" Score="1" ViewCount="35" Body="&lt;p&gt;What would happen if an agent were programmed with two &lt;em&gt;utility functions&lt;/em&gt; which are one the opposite of the other? Would one prevail or will they cancel each other out? Are there studies in this direction?&lt;/p&gt;&#xA;" OwnerUserId="30353" LastEditorUserId="2444" LastEditDate="2020-02-18T13:18:24.377" LastActivityDate="2020-02-18T13:18:24.377" Title="What happens if an agent has two contrasting utility functions?" Tags="&lt;intelligent-agent&gt;&lt;utility&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-03-06T20:34:37.347" ContentLicense="CC BY-SA 4.0" />
  <row Id="18089" PostTypeId="2" ParentId="18085" CreationDate="2020-02-18T10:22:14.177" Score="5" Body="&lt;p&gt;&quot;Assuming that we have sufficient data...&quot; &amp;mdash; that's quite a big assumption. Also, traditional methods are well understood, while neural networks (and especially deep learning) is still something of a black box: you train it, and then you get a mapping from input to output. But you don't really know how that mapping is achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not only about performance, it's also about efficiency (speed, use of power, etc) and transparency (being able to explain why something happens).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are several reasons why we don't put all our eggs into the NN basket:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;it is a lot easier to see what's happening and diagnose errors with 'traditional' methods which are well-understood. This is an important point in real-life applications&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;in many cases we do not have the required amount of training data available that is necessary for deep learning approaches to work&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;training a DL system is much more time- (and energy) consuming than other algorithms&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'd much rather have a nuclear power plant operated by a traditional algorithm that makes a few mistakes, but nothing drastic (and being aware that it makes these kinds of mistakes allows you to guard against them), than have a total black box doing it where I have no idea why decisions are reached and what happens in edge cases not covered by the training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's fine for toy projects where the stakes are low, but in real-world applications there are often different constraints that DL systems cannot satisfy.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-18T10:22:14.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18090" PostTypeId="2" ParentId="18087" CreationDate="2020-02-18T10:31:51.873" Score="1" Body="&lt;p&gt;&quot;Distilling the Knowledge in a Neural Network&quot; by G. Hinton et al.&#xA;&lt;a href=&quot;https://arxiv.org/abs/1503.02531&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1503.02531&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33618" LastActivityDate="2020-02-18T10:31:51.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18091" PostTypeId="1" CreationDate="2020-02-18T10:58:38.773" Score="1" ViewCount="28" Body="&lt;p&gt;During training of my models I often encounter the following situation about &lt;strong&gt;training (green)&lt;/strong&gt; and &lt;strong&gt;validation (gray)&lt;/strong&gt; loss:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e34rT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e34rT.png&quot; alt=&quot;training and validation loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, the validation loss is significantly lower than the training loss. How is this possible? Does this tell me anything important about my data or model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One explanation might be, that training and validation data are not properly split, i.e. the validation might primarily contain data, that the model can easily represent. But then why do the curves cross after epoch 30? If this is because of overfitting, then I would expect the validation loss to increase, but so far both losses are (slowly) decreasing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a related &lt;a href=&quot;https://datascience.stackexchange.com/questions/53852/why-such-a-big-difference-in-number-between-training-error-and-validation-error&quot;&gt;question&lt;/a&gt; at Data Science SE, but it doesn't give a clear answer.&lt;/p&gt;&#xA;" OwnerUserId="33186" LastActivityDate="2020-02-18T10:58:38.773" Title="Crossing of training and validation loss" Tags="&lt;loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18092" PostTypeId="2" ParentId="18080" CreationDate="2020-02-18T11:25:59.907" Score="2" Body="&lt;p&gt;Minimax is a planning algorithm, and all planning algorithms need access to a model of the environment in order to look ahead or simulate possible future states and results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically this does not need to be 100% accurate or complete. It could even be a learned model. However, in the case of applying minimax to classic two player games, such as chess or Connect 4, then usually the game rules are used to create perfect predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This difference between planning and learning is not quite the same as model-free vs model-based RL, but the ideas do overlap considerably. You can for instance consider the experience replay approach used in DQN as a form of &quot;background planning&quot; where the model used is the memory of previous events, whilst the core Q-learning algorithm used inside DQN is normally considered model-free.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-02-18T11:25:59.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18093" PostTypeId="1" CreationDate="2020-02-18T11:27:43.723" Score="1" ViewCount="13" Body="&lt;p&gt;For a project of Deep Learning for Detection algorithm (for e.g., improving YOLO) implying 4 users using the same hardware, is it more efficient to set the environment as each user has his own session on the same OS running all the hardware, or is it better to allow one VM per user, or one VM per two users, each VM being allocated with an equal amount of the whole hardware ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for your help.&lt;/p&gt;&#xA;" OwnerUserId="33620" LastActivityDate="2020-02-18T11:27:43.723" Title="Multi user DL trainings: VMs VS Multi-seat configuration" Tags="&lt;training&gt;&lt;hardware&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18094" PostTypeId="1" AcceptedAnswerId="18096" CreationDate="2020-02-18T11:36:05.880" Score="0" ViewCount="60" Body="&lt;p&gt;What is the difference between TensorFlow's callbacks and early stopping?&lt;/p&gt;&#xA;" OwnerUserId="32856" LastEditorUserId="2444" LastEditDate="2020-02-18T13:15:00.860" LastActivityDate="2020-02-18T13:57:17.403" Title="What is the difference between TensorFlow's callbacks and early stopping?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;comparison&gt;&lt;regularization&gt;&lt;early-stopping&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18095" PostTypeId="1" CreationDate="2020-02-18T11:40:32.860" Score="1" ViewCount="14" Body="&lt;p&gt;I would like to ask you for advice. I deal with beekeeping but I am also a bit a programmer and an electronics specialist. And this is where my 3 interests come together, actually 4 because recently deep learning has joined this group.&#xA;I would like to analyze (I already run some tests) bee behavior using deep learning mechanisms based on images (photos or videos) with bees. These images, of course, can be very different, this applies primarily to the area that can be shown. It can literally be 2x3cm like a macro picture or an area of ​​40x30cm and 3000 bees on it. Trying to create a network analyzing such different areas is a nightmare. And because I am interested in aspects of bees separately, it seems logical to divide the image into these parts, which will contain single bees in their entirety, reduce their resolution to a minimum ensuring recognition of the necessary details and only then analyze each part of the large image separately.&#xA;This approach seems even more right to me when I started to read the details, e.g. YOLOv3. Thanks to this, I would avoid a large amount of additional calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here comes the first stage to be implemented. I need to specify the image resolution, more precisely the number of pixels that on average falls on one bee length. If it is too small, I do nothing but say that the resolution is too small for analysis. If it is suitable (at least 50 pixels for the length of the bee), I algorithmically cut the input image into smaller ones so that they contain individual insects and I subject them to further analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, the sizes of bees and bee cells are precisely known to within 0.1mm. And there may be hundreds of such objects (a bee or a hexagonal bee cell) and I can decide how many of them will be at the stage of preparing training data. I have thousands of photos and videos from which I can easily create training data for the network in the form of pairs of objects image bee size expressed in pixels from this image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Time for question:&#xA;At the output I want to get a number saying that one bee in the image is X pixels long, e.g. 43.02 or 212.11&#xA;Has anyone of you dealt with a similar case of determining resolution for known objects using a neural network (probably CNN with an output element with rel function) and can share your experience, e.g. a network structure that would be suitable for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="33622" LastActivityDate="2020-02-18T11:40:32.860" Title="Specifying resolution for objects with known dimensions using CNN" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18096" PostTypeId="2" ParentId="18094" CreationDate="2020-02-18T12:46:20.820" Score="2" Body="&lt;p&gt;Early stopping and callbacks are two different concepts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Early stopping&lt;/strong&gt; is a machine learning concept about when to stop training your model to avoid overfitting: You monitor a target value (e.g. validation loss) and stop learning after it hits a minimum. If the monitored value keeps increasing for a couple of epoch, you can restore the weights from the minimum.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Callbacks&lt;/strong&gt; are a general software engineering concept and a technical implementation detail on how to trigger certain functions. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A callback function (e.g. early stopping) is a function that is passed as an argument to another function (e.g. fitting an ML model), and called in certain situations (e.g. after finishing an epoch). The callback function will perform its business (e.g. monitor the validation loss and decide whether to trigger the end of the training) and return control to the calling function (fitting the model).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In TensorFlow, early stopping is implemented using a callback function, but this is not the only way to do it. Also, there are further features in TensorFlow that are implemented using callbacks.&lt;/p&gt;&#xA;" OwnerUserId="33186" LastEditorUserId="2444" LastEditDate="2020-02-18T13:57:17.403" LastActivityDate="2020-02-18T13:57:17.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18097" PostTypeId="1" CreationDate="2020-02-18T12:59:46.667" Score="2" ViewCount="58" Body="&lt;p&gt;I did an out of domain detection task (as a binary classification problem) and tried LR and Naive Bayes and BERT but the deep neural network didn't perform better than LR and NB. For the LR I just used BOW and it beats the 12-layer BERT. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a lecture, &lt;a href=&quot;https://www.youtube.com/watch?v=HfM8UIohGE0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng&lt;/a&gt; suggests &quot;Build First System Quickly, Then Iterate&quot;, but it turns out that sometimes we don't need to iterate the model into a deep neural network and most of the time traditional shallow neural networks are good/competitive enough and much simpler for training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;a href=&quot;https://twitter.com/dirk_hovy/status/1222902471246278657&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tweet (and its replies)&lt;/a&gt; indicate, together with various papers [&lt;a href=&quot;https://arxiv.org/pdf/1503.00693.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://www.aclweb.org/anthology/S18-1003.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://www.aclweb.org/anthology/S18-1004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;, &lt;a href=&quot;https://paperswithcode.com/sota/sentiment-analysis-on-imdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt; etc], traditional SVM, LR, and Naive Bayes can beat RNN and some complicated neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then my two questions are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When should we switch to complicated neural networks like RNN, CNN, and transformer and etc? How can we see that from the data set or the results (by doing error analysis) of the simple neural networks?       &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The aforementioned experiments may be caused by the simple test set, then (how) is it possible for us to design a test set that can fail the traditional models? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="2193" LastEditDate="2020-02-18T13:08:17.550" LastActivityDate="2020-07-20T16:00:15.567" Title="When is it time to switch to deep neural networks from simple networks in text classification problems?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18098" PostTypeId="1" CreationDate="2020-02-18T13:10:24.207" Score="1" ViewCount="32" Body="&lt;p&gt;I am developing my own mobile app related to digital map. One of the functions is searching POIs (points of interest) in the map according to relevance between user query and POI name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the POIs whose names contain exact words in the query, the app also needs to return those whose names are semantically related. For example, searching 'flower' should return POI names that contain 'flower' as well as those that contain 'florist'. Likewise, searching 'animal' should return 'animal' as well as 'veterinary'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I need to extend words in the query semantically. For example, 'flower' has to be extended to ['flower', 'florist']. I have tried to use word embeddings: using the words corresponding to most similar vectors as extensions. Due to the fact I don't have user review data right now and most of the POI names are very short, I used trained word2vec model published by &lt;a href=&quot;https://code.google.com/archive/p/word2vec/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google&lt;/a&gt;. But the results turn out to be not what I expect: most similar words of 'flower' given by word2vec are words like 'roses'and 'orchid', and 'florist' is not even in the top 100 most similar list. Likewise, 'animal' gives 'dog', 'pets', 'cats' etc. Not very useful for my use case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think simply using word embedding similarity may not be enough. I may need to build some advanced model based on word embedding. Do you have any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="33623" LastActivityDate="2020-02-18T13:30:25.387" Title="Using word embedding to extend words for searching POI names" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18099" PostTypeId="2" ParentId="18098" CreationDate="2020-02-18T13:30:25.387" Score="1" Body="&lt;p&gt;I think word embeddings are overkill in this particular case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion would be to go for a simple dictionary based approach: compose sets of semantically related words, and then use those to expand your query terms. This might take a bit longer to set up, but has several advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;simplicity: you can't make many mistakes with this&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;transparency: you know exactly why a certain term matches, and another one doesn't&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;accuracy: you have tight control over the whole process; if some term is wrong, you remove it from the set. You cannot do that with embeddings&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;resources: a dictionary-based approach is far simpler and needs less storage&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;'Old tech' doesn't sound as sexy as the latest deep learning stuff, but unless you want to have this as a toy project to learn about how to do things with embeddings I would say the latter are the wrong tool for the job. At least you can be sure that it works, and if it doesn't you can easily fix it.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-18T13:30:25.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18100" PostTypeId="1" CreationDate="2020-02-18T14:01:18.790" Score="1" ViewCount="42" Body="&lt;p&gt;I've modeled an AlexNet neural network, with 50 epochs and a batch size of 64. I used a stochastic gradient descent optimizer with a learning rate of 0.01. I attached the train and validation loss and accuracy plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/68xIQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/68xIQ.png&quot; alt=&quot;Accuracy plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0D4bt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0D4bt.png&quot; alt=&quot;Loss plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I reduce the fluctuation of the first epochs?&lt;/p&gt;&#xA;" OwnerUserId="33626" LastEditorUserId="2444" LastEditDate="2020-02-19T01:11:24.407" LastActivityDate="2020-02-19T01:11:24.407" Title="How to reduce fluctuation of a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18101" PostTypeId="1" CreationDate="2020-02-18T14:14:03.687" Score="1" ViewCount="82" Body="&lt;p&gt;I have tried to get the basic grasp of the following deep learning frameworks with python:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Keras&lt;/li&gt;&#xA;&lt;li&gt;Pytorch&lt;/li&gt;&#xA;&lt;li&gt;Caffe&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, I have lately noticed that people in the computer vision community care less and less about Caffe as being somehow unpopular these days apart from its easy implementation and learning pace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of Keras, PyTorch, and Caffe for computer vision? Which problems are they more suited for, with respect to one another? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not looking for answers such as &quot;Keras is easier in my opinion&quot;. I am looking for facts or specific features that make one library more suited than the other for computer vision tasks.&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="2444" LastEditDate="2020-03-13T16:34:26.793" LastActivityDate="2020-03-13T16:34:26.793" Title="What are the pros and cons of Keras, PyTorch and Caffe for computer vision?" Tags="&lt;computer-vision&gt;&lt;keras&gt;&lt;comparison&gt;&lt;pytorch&gt;&lt;caffe&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18103" PostTypeId="1" CreationDate="2020-02-18T15:13:27.803" Score="1" ViewCount="40" Body="&lt;p&gt;I am trying to conduct an experiment for fun to see if AI could be trained to create &quot;good&quot; social media post. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I pulled examples of 1,000s &quot;good&quot; social media post from Facebook. Can someone point me in the right direction how I would use these examples to train a model and then have the model produce it's own post?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have experience using computer vision and AI to train OCR models, so I am not totally new to the field, but could use some help to get pointed in the right direction. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="33631" LastActivityDate="2020-02-18T15:38:41.263" Title="Can AI be trained to create social media post?" Tags="&lt;ai-design&gt;&lt;training&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18104" PostTypeId="1" CreationDate="2020-02-18T15:23:57.123" Score="2" ViewCount="116" Body="&lt;p&gt;I'd like to develop an MCTS-like (Monte Carlo Tree Search) algorithm for program induction, i.e. learning programs from examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial plan is for nodes to represent programs and for the search to expand nodes by revising programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of these expansions revise a single program: randomly resample a subtree of the program, replace a constant with a variable, etc. It looks straightforward to use these with MCTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some expansions, however, generate a program from scratch (e.g. sample a new program). Others use two or more programs to generate a single output program (e.g. crossover in Genetic Programming).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These latter types of moves seem nonstandard for vanilla MCTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea I've had is to switch from nodes representing programs to nodes representing tuples of programs. The root node would represent the empty tuple &lt;span class=&quot;math-container&quot;&gt;$()$&lt;/span&gt;, to which expansions could be applied only if they can generate a program from scratch. The first such expansion would produce some program &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, so the root would now have child &lt;span class=&quot;math-container&quot;&gt;$(p)$&lt;/span&gt;. The second expansion would produce &lt;span class=&quot;math-container&quot;&gt;$p'$&lt;/span&gt;, so the root would now also have child &lt;span class=&quot;math-container&quot;&gt;$(p')$&lt;/span&gt; as well as the pair &lt;span class=&quot;math-container&quot;&gt;$(p, p')$&lt;/span&gt;. Even assuming some reasonable restrictions (e.g. moves can use at most 2 programs, pairs cannot have identical elements, element order doesn't matter), the branching factor will grow combinatorially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What techniques from the MCTS literature (or other literatures) might reduce the impact of this combinatorial explosion?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="33629" LastEditorUserId="2444" LastEditDate="2020-12-19T13:39:16.920" LastActivityDate="2021-05-18T15:01:20.123" Title="How can I reduce combinatorial explosion in an MCTS-like algorithm for program induction?" Tags="&lt;monte-carlo-tree-search&gt;&lt;combinatorics&gt;&lt;inductive-programming&gt;&lt;program-synthesis&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18105" PostTypeId="1" CreationDate="2020-02-18T15:26:02.477" Score="1" ViewCount="58" Body="&lt;p&gt;I've been trying to use this DeepLabv3+ implementation with my dataset (~1000 annotated images of the same box, out of the same video sequence): &lt;a href=&quot;https://github.com/srihari-humbarwadi/person_segmentation_tf2.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/srihari-humbarwadi/person_segmentation_tf2.0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I get border artifacts like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gdKO7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gdKO7.png&quot; alt=&quot;Box with artifacts&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas what could be causing it? Note that if I use bigger batches and train for more epochs, the borders tend to get thinner but never disappear. They also appear randomly around the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any clues what could be causing them and how to solve it?&lt;/p&gt;&#xA;" OwnerUserId="33632" LastEditorUserId="2444" LastEditDate="2020-02-19T01:00:39.323" LastActivityDate="2020-02-19T01:00:39.323" Title="Weird border artifacts when training a CNN" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18109" PostTypeId="5" CreationDate="2020-02-19T00:57:08.697" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-19T00:57:08.697" LastActivityDate="2020-02-19T00:57:08.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18110" PostTypeId="4" CreationDate="2020-02-19T00:57:08.697" Score="0" Body="For questions related to the concept of a knowledge base, which is a set of sentences (or propositions), which are, for example, used to implement knowledge-based intelligent agents. Each sentence represents some assertion about the world. Sentences that are not derived but are given as true are sometimes called axioms. We can perform inference to derive new facts given the sentences in the knowledge base." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T18:31:38.647" LastActivityDate="2021-01-21T18:31:38.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18111" PostTypeId="1" AcceptedAnswerId="21791" CreationDate="2020-02-19T02:26:03.630" Score="4" ViewCount="73" Body="&lt;p&gt;How does policy evaluation work for continuous state space model-free approaches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically, a model-based approach for the discrete state and action space can be computed via dynamic programming and solving the Bellman equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you use a DQN to find another policy, how does model-free policy evaluation work then? I am thinking of Monte Carlo simulation, but that would require many many episodes. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-04-16T19:21:45.227" LastActivityDate="2020-06-11T02:18:21.050" Title="How does policy evaluation work for continuous state space model-free approaches?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;monte-carlo-methods&gt;&lt;model-free-methods&gt;&lt;policy-evaluation&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18112" PostTypeId="2" ParentId="17227" CreationDate="2020-02-19T03:21:43.203" Score="1" Body="&lt;p&gt;Some of the domains in the &lt;a href=&quot;https://ipc2018-probabilistic.bitbucket.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Probabilistic Planning Competition&lt;/a&gt;, such as the &lt;em&gt;Wildlife Preserve&lt;/em&gt; benchmark, fit quite well the constraints you have given. Note that the problems are modeled with a high-level declarative language, &lt;a href=&quot;http://users.cecs.anu.edu.au/~ssanner/Papers/RDDL_Tutorial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RDDL&lt;/a&gt;. This means that you can define problems as big or as small as your heart desires with relative ease, since you can parametrize state description in terms of functions describing properties of an arbitrary number of objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also a &lt;a href=&quot;https://github.com/thiagopbueno/rddlgym&quot; rel=&quot;nofollow noreferrer&quot;&gt;quite useful&lt;/a&gt; project that allows to instance OpenAI &lt;code&gt;gym&lt;/code&gt; environments from the declarative description of the environment, states and actions.&lt;/p&gt;&#xA;" OwnerUserId="33641" LastActivityDate="2020-02-19T03:21:43.203" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18113" PostTypeId="2" ParentId="18104" CreationDate="2020-02-19T03:37:06.033" Score="0" Body="&lt;p&gt;I think you may get some inspiration from the work on deterministic environments by Javier Segovia et al. See their paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0004370219300232&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing programs for generalized planning using a classical planner&lt;/a&gt; (2019).&lt;/p&gt;&#xA;&lt;p&gt;If you don't have access to Elsevier's papers, I recommend that you check out the &lt;a href=&quot;https://dblp.org/pers/a/Aguas:Javier_Segovia.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;first author's profile&lt;/a&gt; on &lt;code&gt;dblp.org&lt;/code&gt;. From there you will find links to open access versions of the conference paper that led up to the publication above.&lt;/p&gt;&#xA;" OwnerUserId="33641" LastEditorUserId="2444" LastEditDate="2020-12-19T13:43:59.500" LastActivityDate="2020-12-19T13:43:59.500" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18115" PostTypeId="1" CreationDate="2020-02-19T07:40:49.390" Score="3" ViewCount="315" Body="&lt;p&gt;I'm attempting to design an action space in OpenAI's gym and hitting the following roadblock. I've looked at &lt;a href=&quot;https://stackoverflow.com/q/45001361/3924118&quot;&gt;this post&lt;/a&gt; which is closely related but subtly different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environment I'm writing needs to allow an agent to make between &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; sub-actions in each step. Leaving it up to the agent to decide how many sub-actions it wants to take. So, something like (sub-action-category, sub-action-id, action) where the agent can specify between &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; such tuples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't seem possible to define a &lt;code&gt;Box&lt;/code&gt; space without specifying bounds on the shape which is what I need here. I'm trying to avoid defining an action space where each sub-action is explicitly enumerated by the environment like (action) tuple with n entries for each sub-action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other spaces I could use to dynamically scale the space?&lt;/p&gt;&#xA;" OwnerUserId="32763" LastEditorUserId="2444" LastEditDate="2020-05-20T11:08:35.977" LastActivityDate="2020-05-20T11:08:35.977" Title="How to define an action space when an agent can take multiple sub-actions in a step?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18116" PostTypeId="2" ParentId="18084" CreationDate="2020-02-19T10:02:54.663" Score="0" Body="&lt;p&gt;I recommend you to look into the literature about &lt;em&gt;simulation&lt;/em&gt; and &lt;em&gt;bisimulation&lt;/em&gt; in Automata Theory and its applications to model checking (where you want to make quite regularly proofs of &quot;behavioural equivalence&quot;). One article that discusses this in the context of a technique for model checking known as &lt;em&gt;&quot;Abstraction and Abstraction Refinement&quot;&lt;/em&gt; is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Abstraction and Abstraction Refinement&#xA;Dennis Dams and Orna Grumberg&#xA;In Springer's Handbook of Model Checking, 2018, Chapter 13, pages 385-420&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A good (I use it regularly) book that covers behavioural equivalence for a wide variety of automata is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Verification and Control of Hybrid Systems: A Symbolic Approach&#xA;Paulo Tabuada&#xA;Springer, 2009&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33641" LastActivityDate="2020-02-19T10:02:54.663" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18117" PostTypeId="1" CreationDate="2020-02-19T10:21:39.780" Score="1" ViewCount="49" Body="&lt;p&gt;How do I calculate the error during the training phase for deep reinforcement learning models?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep reinforcement learning is not supervised learning as far as I know. So how can the model know whether it predicts right or wrong? In literature, I find that the &quot;actual&quot; Q-value is calculated, but that sounds like the whole idea behind deep RL is obsolete. How could I even calculate/know the real Q-value if there is not already a world model existing?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-02-19T17:45:40.633" LastActivityDate="2020-02-19T17:45:40.633" Title="How to estimate the error during training in deep reinforcement learning" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18118" PostTypeId="1" AcceptedAnswerId="18131" CreationDate="2020-02-19T10:49:13.697" Score="2" ViewCount="133" Body="&lt;p&gt;I'm doing reinforcement learning, and I have a visual observation that I will use to build an input state for my agent. In the DeepMind's Atari paper, they greyscale the input image before they input it into the CNN to reduce the input space's size, which makes sense to me.&lt;/p&gt;&#xA;&lt;p&gt;In my environment, I have, for each pixel, 5 possible channels, which are represented in black, white, blue, red, and green. This also makes intuitive sense to me since it's like a bit-encoding.&lt;/p&gt;&#xA;&lt;p&gt;Any thoughts on what could be better? Greyscaling into 2 shades of grey and black and white also maintains the information, but feels somehow less direct, since my environment's visual space is categorical, which makes more sense in a categorical encoding.&lt;/p&gt;&#xA;" OwnerUserId="31180" LastEditorUserId="2444" LastEditDate="2021-02-03T17:34:55.993" LastActivityDate="2021-02-03T17:37:48.660" Title="Should I grey-scale the coloured frames/channels to build the approximation of the state?" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18119" PostTypeId="1" CreationDate="2020-02-19T11:32:42.117" Score="1" ViewCount="27" Body="&lt;p&gt;I am thinking of applying apprenticeship learning on retrospective data. From looking at this paper by Ng &lt;a href=&quot;https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf&lt;/a&gt; which talks about apprenticeship learning, it seems to me that at the 5th step of the algorithm, &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Compute (or estimate) &lt;span class=&quot;math-container&quot;&gt;$μ^{(i)}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$μ(π^{(i)})$&lt;/span&gt;, &#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mu^{(i)}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$E[\sum_{t=0}^{∞}\gamma^{t}$$\phi(s_{t})$&lt;/span&gt; | &lt;span class=&quot;math-container&quot;&gt;$\pi^{(i)}]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\phi(s_{t})$&lt;/span&gt; is the reward feature vector at state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From my understanding, a sequence of &lt;span class=&quot;math-container&quot;&gt;$s_0, s_1, s_2 ..$&lt;/span&gt; trajectory would have to be generated at this step, following this policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{(i)}$&lt;/span&gt;. Hence, applying this algorithm on retrospective data would not work?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-02-19T18:06:59.320" LastActivityDate="2020-02-19T18:06:59.320" Title="Does apprenticeship learning require prospective data?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;apprenticeship-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18120" PostTypeId="1" CreationDate="2020-02-19T11:34:54.393" Score="1" ViewCount="71" Body="&lt;p&gt;I want to be able to debug my Keras LSTM model. For example, I want to be able to check the values of the input/output gates, cell states and hidden states at every time-step during prediction. Are there any tools out there that can achieve this?&lt;/p&gt;&#xA;" OwnerUserId="33499" LastEditorUserId="2444" LastEditDate="2020-02-19T17:23:21.330" LastActivityDate="2020-02-19T17:23:21.330" Title="Are there any tools I can use to debug a Keras LSTM model during prediction?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18121" PostTypeId="2" ParentId="12647" CreationDate="2020-02-19T12:34:11.407" Score="2" Body="&lt;p&gt;I thought the answer might be no. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jvuJp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jvuJp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this 2020 ICLR paper: &lt;a href=&quot;https://arxiv.org/abs/1904.09751&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Curious Case of Neural Text Degeneration&lt;/a&gt;, researchers found that beam search text is less surprising compared to human natural language. And they proposed a nucleus sampling method which generates more human like text.  &lt;/p&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="5351" LastEditDate="2020-02-19T12:43:51.787" LastActivityDate="2020-02-19T12:43:51.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18122" PostTypeId="1" CreationDate="2020-02-19T12:45:13.813" Score="1" ViewCount="62" Body="&lt;p&gt;I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in &lt;code&gt;[0,13]&lt;/code&gt;. I have noticed that one-hot encoding it and using cross-entropy loss gives better results than MSE loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my problem : no matter how deep I make the network or how many fully connected layers I add I always obtain pretty much the same behavior. Changing the optimizer also doesn't really help.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The loss function quickly decreases then stagnates with a very high variance and never goes down again.&lt;/li&gt;&#xA;&lt;li&gt;The prediction seems to be offset around the value &lt;code&gt;9&lt;/code&gt;, I really do not understand why since I have one-hot encoded the input and the output.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is an example of a the results of a typical training phase, with the total loss :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/o51R3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o51R3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/0id4C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0id4C.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any tips/ideas as to how I could improve this or could have gone wrong ? I am a bit of a beginner in ML si I might have missed something. I can also include the code (in PyTorch) if necessary.&lt;/p&gt;&#xA;" OwnerUserId="33658" LastActivityDate="2021-03-16T21:08:06.157" Title="time-series prediction : loss going down, then stagnates with very high variance" Tags="&lt;long-short-term-memory&gt;&lt;pytorch&gt;&lt;time-series&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18123" PostTypeId="2" ParentId="18117" CreationDate="2020-02-19T13:44:41.183" Score="1" Body="&lt;p&gt;Yes, reinforcement learning is very different from supervised learning, the policy (what you call a model) does not know if its predicting right or wrong, or if its taking the correct action or not. In RL there is no concept of &quot;the right action&quot;, everything is evaluated through the reward function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also there are no ways to compute the ground truth Q-values, if you had that then there is no need to do RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In RL you should not think like in supervised learning, there are no error metrics, everything is evaluated on how much accumulated reward the agent receives over an episode.&lt;/p&gt;&#xA;" OwnerUserId="31632" LastActivityDate="2020-02-19T13:44:41.183" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18125" PostTypeId="1" CreationDate="2020-02-19T15:19:55.500" Score="0" ViewCount="1442" Body="&lt;p&gt;One-point Hill Climbing can get stuck due to which of the following phenomenon?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Local Maxima&lt;/li&gt;&#xA;&lt;li&gt;Inadmissible heuristic&lt;/li&gt;&#xA;&lt;li&gt;Ridges&lt;/li&gt;&#xA;&lt;li&gt;Plateau&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33661" LastEditorUserId="2444" LastEditDate="2020-11-24T02:38:43.263" LastActivityDate="2020-11-24T02:38:43.263" Title="For which of the following reasons can Hill Climbing get stuck?" Tags="&lt;optimization&gt;&lt;hill-climbing&gt;&lt;homework&gt;&lt;local-search&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18126" PostTypeId="1" CreationDate="2020-02-19T16:10:21.610" Score="2" ViewCount="36" Body="&lt;p&gt;Suppose I want to classify a dataset like the MNIST handwritten dataset, but it has added distractions. For example, here we have a 6 but with extra strokes around it that don't add value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Z35w2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z35w2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose a good model would predict a 6, but maybe with less than 100% certainty (or maybe with 100% certainty - I don't know that it matters for the purpose of this question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to get information about which pixels most strongly influenced the decision of the CNN, and which pixels were not so important? So to represent that visually, green means that those pixels were important:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T7XhO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T7XhO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or conversely, is it possible to highlight pixels which did not contribute to the outcome (or which cast doubt on the outcome thereby reducing the certainty from 100%)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3YpG7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3YpG7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-02-24T18:41:24.813" Title="Can a NN be configured to indicate which points of the input influenced its prediction and how?" Tags="&lt;convolutional-neural-networks&gt;&lt;explainable-ai&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18128" PostTypeId="5" CreationDate="2020-02-19T18:07:55.393" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-19T18:07:55.393" LastActivityDate="2020-02-19T18:07:55.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18129" PostTypeId="4" CreationDate="2020-02-19T18:07:55.393" Score="0" Body="For questions related to apprenticeship learning (also known as learning from demonstration), which is the process of learning by observing an expert, thus it can be thought of as a form of supervised learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-19T18:07:55.393" LastActivityDate="2020-02-19T18:07:55.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18130" PostTypeId="1" CreationDate="2020-02-19T18:14:16.720" Score="1" ViewCount="59" Body="&lt;p&gt;I'm actually trying to create a sequential neural network in order to translate a &quot;human&quot; sentence in a &quot;machine&quot; sentence understandable by an algorithm. Like It didn't work, I've try to create a NN that understands whether the input is a unit or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even this NN doesn't work and I don't understand Why. I tried with different optimize/loss/metrics/with Rnn/with LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there is one array of unit and one array with lambda words.&#xA;I send to the NN :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;input -&gt; word in OneHotEncoding where each char is a vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;output -&gt; a vector with a 1 at the relative position of the unit in the array, ex : [0,0,0,1,0]. If It's not a unit, the vector is composed of 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm actually using LSTM layers and sigmoid activation because I will need it for my &quot;big&quot; NN. &lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential()&#xA;model.add(tf.keras.Input(shape=(13,vocab_size),batch_size=80))&#xA;model.add(tf.keras.layers.LSTM(32, return_sequences=True)) &#xA;model.add(tf.keras.layers.LSTM(6, return_sequences=False))&#xA;model.add(tf.keras.layers.Dense(6, activation=&quot;sigmoid&quot;))&#xA;model.reset_states()&#xA;model.summary()&#xA;model.compile(optimizer= 'adam', loss='categorical_crossentropy', metrics=['categorical_accuracy'])&#xA;model.fit(encoded_word, my_targets, batch_size=80, epochs=100, validation_data=(encoded_word, my_targets))&#xA;&#xA;&#xA;&#xA;Model: &quot;sequential_38&quot;&#xA;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;lstm_57 (LSTM)               (80, 13, 32)              6400      &#xA;_________________________________________________________________&#xA;lstm_58 (LSTM)               (80, 6)                   936       &#xA;_________________________________________________________________&#xA;dense_32 (Dense)             (80, 6)                   42        &#xA;=================================================================&#xA;Total params: 7,378&#xA;Trainable params: 7,378&#xA;Non-trainable params: 0&#xA;&#xA;&#xA;Epoch 1/100&#xA;10000/10000 [==============================] - 6s 619us/sample - loss: 0.7441 - categorical_accuracy: 0.2898 - val_loss: 0.6181 - val_categorical_accuracy: 0.3388&#xA;Epoch 2/100&#xA;10000/10000 [==============================] - 2s 233us/sample - loss: 0.5768 - categorical_accuracy: 0.3388 - val_loss: 0.5382 - val_categorical_accuracy: 0.3388&#xA;Epoch 3/100&#xA;10000/10000 [==============================] - 2s 229us/sample - loss: 0.5039 - categorical_accuracy: 0.3979 - val_loss: 0.4640 - val_categorical_accuracy: 0.5084&#xA;Epoch 4/100&#xA;10000/10000 [==============================] - 2s 229us/sample - loss: 0.4207 - categorical_accuracy: 0.4759 - val_loss: 0.3709 - val_categorical_accuracy: 0.5041&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My NN is converging towards 0.5 all the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your answers !&lt;/p&gt;&#xA;" OwnerUserId="33665" LastActivityDate="2020-02-19T18:14:16.720" Title="Simple sequential model with LSTM which doesn't converge" Tags="&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18131" PostTypeId="2" ParentId="18118" CreationDate="2020-02-19T18:28:26.483" Score="3" Body="&lt;p&gt;We can't say for sure which approach would work best in the general case. If you have domain knowledge, you can make a better guess. You'll basically want to answer the question: which information is important for learning an optimal policy?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In my environment, I have, for each pixel, 5 possible channels, which are represented in black, white, blue, red, and green. This also makes intuitive sense to me since it's like a bit-encoding.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Generally, if you have an environment like this, I would (without any other information) guess that each of the 5 colors have some meaning that may be relevant for your agent. That's just what I would guess though. In theory, it might be possible that white means one thing (e.g. &amp;quot;empty&amp;quot;), and every other colour means the same other thing (e.g. &amp;quot;not empty&amp;quot;). If you had domain knowledge like that, and knew that it is only important whether or not any given pixel is white, you could of course binarise your input.&lt;/p&gt;&#xA;&lt;p&gt;But in general, if the colours might be important, I'd recommend including them. If you really only have a few distinct colours like that though, I would not recommend encoding them in some format like RGB where values can range from 0 to 1 or 0 to 255. I would recommend having 4 (or 5?) binary channels in your input:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Binary channel containing 1s for pixels that are black, and 0s for all other pixels.&lt;/li&gt;&#xA;&lt;li&gt;Binary channel containing 1s for pixels that are white, and 0s for all other pixels.&lt;/li&gt;&#xA;&lt;li&gt;Binary channel containing 1s for pixels that are blue, and 0s for all other pixels.&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The reason for this is that deep neural networks often tend to have an easier time learning with binary inputs, and here you can completely binarise your inputs without requiring an excessively high number of channels. If you had hundreds or thousands of different possible colours, this would probably no longer be a good idea.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="2444" LastEditDate="2021-02-03T17:37:48.660" LastActivityDate="2021-02-03T17:37:48.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18132" PostTypeId="1" AcceptedAnswerId="18247" CreationDate="2020-02-19T19:12:30.087" Score="4" ViewCount="816" Body="&lt;p&gt;Can vanishing gradients be detected by the change in distribution (or lack thereof) of my convolution's kernel weights throughout the training epochs? And if so how?&lt;/p&gt;&#xA;&lt;p&gt;For example, if only 25% of my kernel's weights ever change throughout the epochs, does that imply an issue with vanishing gradients?&lt;/p&gt;&#xA;&lt;p&gt;Here are my histograms and distributions, is it possible to tell whether my model suffers from vanishing gradients from these images? (some middle hidden layers omitted for brevity)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dCV6J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dCV6J.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FwHvO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FwHvO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/503r6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/503r6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tXdOE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tXdOE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31299" LastEditorUserId="2444" LastEditDate="2020-12-13T13:18:33.360" LastActivityDate="2020-12-13T13:23:58.090" Title="How to detect vanishing gradients?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;deep-neural-networks&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18133" PostTypeId="2" ParentId="12059" CreationDate="2020-02-19T19:37:50.087" Score="1" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1912.06680v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dota 2 with Large Scale Deep Reinforcement Learning&lt;/a&gt; goes into greater detail than the initial blog posts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They call their distributed training framework &lt;em&gt;Rapid&lt;/em&gt;, which is also used in some of their robotics work, such as the paper &lt;a href=&quot;https://arxiv.org/pdf/1808.00177.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Dexterous In-Hand Manipulation&lt;/a&gt;, where they discuss a smaller scale deployment of Rapid (as compared to Dota2/OpenAI V) in section 4.3.&lt;/p&gt;&#xA;" OwnerUserId="14513" LastEditorUserId="14513" LastEditDate="2020-02-20T20:06:12.830" LastActivityDate="2020-02-20T20:06:12.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18134" PostTypeId="1" CreationDate="2020-02-19T19:38:47.233" Score="2" ViewCount="77" Body="&lt;p&gt;I am new to ANN. I am trying out several 'simple' algorithms to see what ANN can (or cannot) be used for and how. I played around with Conv2d once and had it recognize images succesfully. Now I am looking into trend line analyses. I have succeeded in training a network where it solved for linear equations. Now I am trying to see if it can be trained to solve for y in the formula &lt;strong&gt;y = b + x * x&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter what parameters I change, or number of Dense layers, I get high values for loss and val_loss, and the predictions are incorrect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to solve for this equation, and with what network? If it is not possible, why not? I am not looking to solve a practical problem, rather build up understanding and intuition about ANNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the code I tried with below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#region Imports&#xA;from __future__ import absolute_import, division, print_function, unicode_literals&#xA;import math &#xA;import numpy as np &#xA;import tensorflow as tf&#xA;from tensorflow.keras import models, optimizers&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras.layers import Activation, Dropout, Flatten, Dense, Lambda&#xA;import tensorflow.keras.backend as K&#xA;#endregion&#xA;&#xA;#region Constants&#xA;learningRate = 0.01&#xA;epochs: int = 1000&#xA;batch_size = None&#xA;trainingValidationFactor = 0.75&#xA;nrOfSamples = 100&#xA;activation = None&#xA;#endregion&#xA;&#xA;#region Function definitions&#xA;def CreateNetwork(inputDimension):&#xA;  model = Sequential()&#xA;  model.add(Dense(2, input_dim=2, activation=activation))&#xA;  model.add(Dense(64, use_bias=True, activation=activation))&#xA;  model.add(Dense(32, use_bias=True, activation=activation))&#xA;  model.add(Dense(1))&#xA;  adam = optimizers.Adam(learning_rate=learningRate)&#xA;  # sgd = optimizers.SGD(lr=learningRate, decay=1e-6, momentum=0.9, nesterov=True)&#xA;  # adamax = optimizers.Adamax(learning_rate=learningRate)&#xA;  model.compile(loss='mse', optimizer=adam)&#xA;  return model&#xA;&#xA;def SplitDataForValidation(factor, data, labels):&#xA;  upperBoundary = int(len(data) * factor)&#xA;&#xA;  trainingData = data[:upperBoundary]&#xA;  trainingLabels = labels[:upperBoundary]&#xA;&#xA;  validationData = data[upperBoundary:]&#xA;  validationLabels = labels[upperBoundary:]&#xA;  return ((trainingData, trainingLabels), (validationData, validationLabels))&#xA;&#xA;def Train(network, training, validation):&#xA;  trainingData, trainingLabels = training&#xA;  history = network.fit(&#xA;    trainingData&#xA;    ,trainingLabels&#xA;    ,validation_data=validation&#xA;    ,epochs=epochs &#xA;    ,batch_size=batch_size&#xA;  )&#xA;&#xA;  return history&#xA;&#xA;def subtractMean(data):&#xA;  mean = np.mean(data)&#xA;  data -= mean&#xA;  return mean&#xA;&#xA;def rescale(data):&#xA;  max = np.amax(data)&#xA;  factor = 1 / max&#xA;  data *= factor&#xA;  return factor&#xA;&#xA;def Normalize(data, labels):&#xA;  dataScaleFactor = rescale(data)&#xA;  dataMean = subtractMean(data)&#xA;&#xA;  labels *= dataScaleFactor&#xA;  labelsMean = np.mean(labels)&#xA;  labels -= labelsMean&#xA;&#xA;def Randomize(data, labels):&#xA;  rng_state = np.random.get_state()&#xA;  np.random.shuffle(data)&#xA;  np.random.set_state(rng_state)&#xA;  np.random.shuffle(labels)&#xA;&#xA;def CreateTestData(nrOfSamples):&#xA;  data = np.zeros(shape=(nrOfSamples,2))&#xA;  labels = np.zeros(nrOfSamples)&#xA;&#xA;  for i in range(nrOfSamples):&#xA;    for j in range(2):&#xA;      randomInt = np.random.randint(1, 5)&#xA;      data[i, j] = (randomInt * i) + 10&#xA;    labels[i] = data[i, 0] + math.pow(data[i, 1], 2)&#xA;&#xA;  Randomize(data, labels)&#xA;  return (data, labels)&#xA;#endregion&#xA;&#xA;allData, allLabels = CreateTestData(nrOfSamples)&#xA;Normalize(allData, allLabels)&#xA;training, validation = SplitDataForValidation(trainingValidationFactor, allData, allLabels)&#xA;&#xA;inputDimension = np.size(allData, 1)&#xA;network = CreateNetwork(inputDimension)&#xA;&#xA;history = Train(network, training, validation)&#xA;&#xA;prediction = network.predict([&#xA;  [2, 2], # Should be 2 + 2 * 2 = 6&#xA;  [4, 7], # Should be 4 + 7 * 7 = 53&#xA;  [23, 56], # Should be 23 + 56 * 56 = 3159&#xA;  [128,256] # Should be 128 + 256 * 256 = 65664&#xA;])&#xA;print(str(prediction))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32968" LastActivityDate="2020-02-20T03:12:08.223" Title="Which ANN can solve for y = x * x + b?" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;learning-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18135" PostTypeId="1" CreationDate="2020-02-19T20:44:12.827" Score="1" ViewCount="33" Body="&lt;p&gt;I have trained a model using Auto-encoder on movielens dataset. Below is how i trained the model.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = model.fit_generator(&#xA;  generator(A, mask),&#xA;  validation_data=test_generator(A_copy, mask_copy, A_test_copy, mask_test_copy),&#xA;  epochs=epochs,&#xA;  steps_per_epoch=A.shape[0] // batch_size + 1,&#xA;  validation_steps=A_test.shape[0] // batch_size + 1,&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is giving good results but now i am confused how should i get the top 5 recommendation on user input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just wanted to print the result on console. Can anyone help me please?&lt;/p&gt;&#xA;" OwnerUserId="33670" LastEditorUserId="26652" LastEditDate="2020-02-20T09:52:07.390" LastActivityDate="2020-02-20T09:58:38.100" Title="How to get top 5 movies recommendations from Auto-Encoder" Tags="&lt;deep-learning&gt;&lt;autoencoders&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18136" PostTypeId="6" CreationDate="2020-02-20T01:15:09.197" Score="0" Body="&lt;p&gt;I am an AI researcher. I worked in academic research for about 10 years, including some time as a professor. More recently I have moved to industrial work, including as the principal scientist at a startup, and now in the AI/NLP group at a major corporation. I continue to publish AI research in machine learning and multiagent systems, and to serve on the program committees for many AI conferences, including AAAI, IJCAI, and AAMAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I joined this site later than most users, and have been sporadic in my activity, but am now ranked fourth overall by reputation. When I'm on the site, I'm active in the meta discussions, The Singularity chat, and in the moderation queue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been asked to run, but cannot promise that I will be substantially more active than the previous moderators, especially in the near term. That said, I care about the site, and would like to see it improve. You can see my opinions on the various issues facing the site by consulting my &lt;a href=&quot;https://ai.meta.stackexchange.com/users/16909/john-doucette&quot;&gt;post history&lt;/a&gt; in meta.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2020-02-20T01:15:09.197" LastActivityDate="2020-02-20T01:15:09.197" CommentCount="5" CommunityOwnedDate="2020-02-20T01:15:09.197" ContentLicense="CC BY-SA 4.0" />
  <row Id="18137" PostTypeId="1" AcceptedAnswerId="20486" CreationDate="2020-02-20T01:48:35.790" Score="2" ViewCount="89" Body="&lt;p&gt;I am a newbie to Machine Learning and AI. As per my understanding, with the use of reinforcement learning &#xA;(reward/punishment environment), we can train a neural network to play a game. I would like to know, whether it possible to use this trained model for deciding the difficulty of the next game level dynamically in realtime according to a player's skill level? As an example, please consider a neural network is trained using Reinforcement Learning for playing a mobile game (chess/puzzle, etc.). The game is not consists of a previously designed static set of game levels. After the training, can this model use to detect a particular player's playing style(score, elapsed time) to dynamically decide the difficulty of the next game level and provide customized game levels for each player in realtime?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much and any help will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="33674" LastActivityDate="2020-04-21T10:34:02.423" Title="Can we use a neural network that is trained using Reinforcement Learning for dynamic game level difficulty designing in realtime?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18138" PostTypeId="2" ParentId="18126" CreationDate="2020-02-20T02:06:17.697" Score="1" Body="&lt;p&gt;Yes there definitely is, and research into this has actually resulted in some really cool behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the simplest ways is to simply back propagate the gradient all the way back to the input. Areas of the input that affected the final decision will receive larger gradients. Interestingly, this also sort of works as a rudimentary form of semantic segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other ways are to change segments of the input and to see how that affects the output (like add 0.1 to a pixel).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also determine what each filter in a convolutional layer is looking at using similar techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a super interesting field of machine learning, and I highly recommend taking a look at &lt;a href=&quot;https://www.youtube.com/watch?v=6wcs6szJWMY&amp;amp;list=PLC1qU-LWwrF64f4QKQT-Vg5Wr4qEE1Zxk&amp;amp;index=13&amp;amp;t=0s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; lecture that is completely free and one of the most interesting ones I've personally seen. it will explain all this much better than I have done.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-02-20T02:06:17.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18139" PostTypeId="2" ParentId="18134" CreationDate="2020-02-20T03:12:08.223" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2 + b$&lt;/span&gt; is a polynomial (more precisely, a parabola) so it is continuous, thus, a neural network (with at least one hidden layer) should be able to approximate that function (given the universal approximation theorem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a very quick look at your code, I noticed you aren't using an activation function for your dense layers (i.e. your activation function is &lt;code&gt;None&lt;/code&gt;). Try to use e.g. ReLU.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-02-20T03:12:08.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18140" PostTypeId="2" ParentId="18066" CreationDate="2020-02-20T03:16:59.083" Score="2" Body="&lt;p&gt;&lt;em&gt;Theoretically&lt;/em&gt;, nothing precludes the use of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns in actor-critic methods. The &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return is an unbiased estimator of the Monte Carlo (MC) return, which means they are essentially interchangeable. In fact, as discussed in &lt;a href=&quot;https://arxiv.org/pdf/1506.02438.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;High-Dimensional Continuous Control Using Generalized Advantage Estimation&lt;/a&gt;, using the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return instead of the MC return can actually help reduce the variance of gradient updates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is similar to &lt;a href=&quot;https://ai.stackexchange.com/a/17095/32070&quot;&gt;my answer in the other question you linked&lt;/a&gt;, so let me try to answer your question more specifically. &lt;em&gt;Even though we can use &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns, why are they not too common in practice?&lt;/em&gt; I suspect there might be a few reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Empirically, faster credit assignment might be more desirable than lower variance. Sometimes the learning speed of your algorithm is constrained simply by how quickly you can learn about the consequences of certain actions. In this case, it is faster to use the MC return, even if it theoretically has higher variance than the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When proposing a new algorithm, adding &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns to it might give it an &quot;unfair&quot; advantage if the other baseline methods do not use them (the reviewers would not like this), so researchers tend to favor simpler 1-step or MC returns for the sake of consistency. I would guess this is why you don't typically see &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns in papers that propose new actor-critic methods. In some sense, it is generally assumed that you could always add &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns to them later and probably get better performance.&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A decent number of deep RL researchers don't know what &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns are. This is especially true if they come from a pure deep learning background; they may have never read &lt;a href=&quot;http://www.incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; which is where most people are introduced to TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt;Exceptions to this are papers like mine (&lt;a href=&quot;https://arxiv.org/abs/1810.09967&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reconciling &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-Returns with Experience Replay&lt;/a&gt;) where the contribution &lt;em&gt;is&lt;/em&gt; the use of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns in methods that previously could not use them. But actor-critic methods are pretty straightforward to combine with &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns.&lt;/p&gt;&#xA;" OwnerUserId="32070" LastActivityDate="2020-02-20T03:16:59.083" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18143" PostTypeId="1" AcceptedAnswerId="18148" CreationDate="2020-02-20T08:34:02.510" Score="1" ViewCount="94" Body="&lt;p&gt;I am currently studying the textbook &lt;em&gt;Deep Learning&lt;/em&gt; by Goodfellow, Bengio, and Courville. Chapter &lt;strong&gt;5.1 Learning Algorithms&lt;/strong&gt; says the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Classification with missing inputs&lt;/strong&gt;: Classification becomes more challenging if the computer program is not guaranteed that every measurement in its input vector will always be provided. To solve the classification task, the learning algorithm only has to define a single function mapping from a vector input to a categorical output. When some of the inputs may be missing, rather than providing a single classification function, the learning algorithm must learn a set of functions. Each function corresponds to classifying &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; with a different subset of its inputs missing. This kind of situation arises frequently in medical diagnosis, because many kinds of medical tests are expensive or invasive. One way to efficiently define such a large set of functions is to learn a probability distribution over all the relevant variables, then solve the classification task by marginalizing out the missing variables. With &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; input variables, we can now obtain all &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; different classification functions needed for each possible set of missing inputs, but the computer program needs to learn only a single function describing the joint probability distribution. See Goodfellow et al. (2013b) for an example of a deep probabilistic model applied to such a task in this way. Many of the other tasks described in this section can also be generalized to work with missing inputs; classification with missing inputs is just one example of what machine learning can do. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was wondering if people would please help me better understand this explanation. Why is it that, when some of the inputs are missing, rather than providing a single classification function, the learning algorithm must learn a set of functions? And what is meant by &quot;each function corresponds to classifying &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; with a different subset of its inputs missing.&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate it if people would please take the time to clarify this.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="2444" LastEditDate="2020-02-20T13:15:53.003" LastActivityDate="2021-02-04T10:57:24.727" Title="Why does the machine learning algorithm need to learn a set of functions in the case of missing data?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;learning-algorithms&gt;&lt;probability-distribution&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18144" PostTypeId="1" CreationDate="2020-02-20T08:36:12.167" Score="1" ViewCount="32" Body="&lt;p&gt;I was trying to normalize my input data images for feeding to my convolutional neural network and wanted to use standardize my input data. I referred to this article:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/57200621/model-not-training-and-negative-loss-when-whitening-input-data&quot;&gt;https://stackoverflow.com/questions/57200621/model-not-training-and-negative-loss-when-whitening-input-data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which says that featurewise_center and featurewise_std_normalization scales the images to range [-1,1]. Wouldn't training the model with this data lead to inaccuracies since the testing data would not be normalized in a similar way and would only range between [0,1]?&#xA;How can I standardize my data while keeping it's range between [0,1]?&lt;/p&gt;&#xA;" OwnerUserId="33681" LastActivityDate="2020-02-20T08:36:12.167" Title="Standardize images using ImageDataGenerator in keras" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18145" PostTypeId="1" CreationDate="2020-02-20T09:08:49.477" Score="0" ViewCount="34" Body="&lt;p&gt;Let's say I have a data set with of length N. A small proportion N2 is labeled. Can I remove some labels and then 'reverse' this action with a trained neural network? I could then use the same process to fill the other (N - N2) rows with labels.&lt;/p&gt;&#xA;" OwnerUserId="33573" LastEditorUserId="33573" LastEditDate="2020-02-24T14:11:24.137" LastActivityDate="2020-02-24T14:11:24.137" Title="Semi-supervised: Can I predict the label of purposely unlabelled observations?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;semi-supervised-learning&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18146" PostTypeId="2" ParentId="18135" CreationDate="2020-02-20T09:58:38.100" Score="1" Body="&lt;p&gt;That is not what an auto-encoder is doing. An auto-encoder gives you a compressed representation of the input. It is trained by mapping the input data to itself, with the compressed form in between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To predict recommendations, you need to train your input data on existing user recommendations.&lt;/p&gt;&#xA;" OwnerUserId="33186" LastActivityDate="2020-02-20T09:58:38.100" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18147" PostTypeId="1" CreationDate="2020-02-20T11:09:19.963" Score="0" ViewCount="23" Body="&lt;p&gt;I wonder about the legitimacy of using the terms &quot;POS tagging&quot;, &quot;Chunking&quot;, &quot;Disambiguation&quot; and &quot;Categorization&quot; to describe an activity that doesn't include writing code and database queries, or interacting with the NLP algorithm and database directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, let's suppose I use the following tools:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;an &quot;Annotator&quot; for analyzing the input text (e.g. sentences copypasted from online newspapers) and choose and save proper values as regards to &quot;POS&quot; of tokens and words, &quot;Words&quot;(entities and collocations) and &quot;Chunk&quot;. Tokens are already detected by default. I have to decide which words are entities and/or collocations or not and their typology, though. May the performed tasks be called &quot;POS tagging&quot;, &quot;Chunking&quot; and &quot;support to categorization&quot;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A knowledge base, for searching and choosing the proper synsets of the lemmas and assigning them to the words analyzed in the previous Annotation tool. May such a task be called &quot;Disambiguation&quot;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A graphical user interface which shows how the NLP analyzes by default the input texts as regards to Lemmas, POS, Chunks, Senses, entities, domains, main concepts, dependency tree, in order to make analyses consistent with it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If I want to define these activities in a few words, &quot;Machine Learning annotation&quot; may be the most correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if I want to be more specific? I don't know whether or not the terms &quot;POS tagging&quot;, &quot;Chunking&quot;, &quot;Disambiguation&quot; and &quot;Support to categorization&quot; may be appropriate for they generally come within &quot;programming contexts&quot;, as far as I know. In other terms, do they involve writing algorithms and programming or are they / may they be referred to the &quot;less-technical&quot; activities described above?&lt;/p&gt;&#xA;" OwnerUserId="22959" LastEditorUserId="2193" LastEditDate="2020-02-20T13:23:35.773" LastActivityDate="2020-02-20T13:23:35.773" Title="Are POS tagging, Chunking, Disambiguation, etc. subtasks of annotation?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18148" PostTypeId="2" ParentId="18143" CreationDate="2020-02-20T12:41:35.703" Score="1" Body="&lt;p&gt;&lt;strong&gt;Intuitively&lt;/strong&gt;, this is similar to the case when you are making predictions but you don't have all the necessary information to make the most accurate prediction or maybe there isn't a single accurate prediction, so you have a set of possible predictions (rather than a single prediction). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you hadn't seen &lt;a href=&quot;https://www.uefa.com/uefachampionsleague/match/2027125--atletico-vs-liverpool/?iv=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;the last Liverpool game&lt;/a&gt; (in the Champions League) against Atlético Madrid, you would have probably said that Liverpool was the most likely team to win the CL this year (2020) too. However, after having seen their last game, you noticed that they are not unbeatable and they are not perfect, so, although they have shown you (during this and the previous season) that they are a very good team, they may also no be the best until the end of the season. So, at this point, you may have a set of two possible hypotheses: Liverpool will win the CL or Liverpool will not win the CL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you had a dataset that is representative of your whole population, then the dataset alone should be sufficient to make accurate predictions (i.e. it contains all the information sufficient to make accurate predictions). If that's not the case (which is often true), then you will have to account for all possible values of the missing data or you will have to make assumptions (or introduce an &lt;a href=&quot;https://en.wikipedia.org/wiki/Inductive_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;inductive bias&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors also mention the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Marginal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;marginalization&lt;/a&gt;, which is used in probability theory to calculate marginal probabilities, e.g. &lt;span class=&quot;math-container&quot;&gt;$p(X=x)$&lt;/span&gt; (or for short &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;), when there's another random variable &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, by accounting for all possible values of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. In other words, you're interested &lt;strong&gt;only&lt;/strong&gt; in &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; and you may have the joint probability distribution &lt;span class=&quot;math-container&quot;&gt;$p(x, y)$&lt;/span&gt;, then marginalization allows you to compute &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; using e.g. &lt;span class=&quot;math-container&quot;&gt;$p(x, y)$&lt;/span&gt; and all possible values that the random variable &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; can take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, I think their description is a little bit vague and using the concept of marginalization to convey the idea behind the &quot;multiple hypotheses&quot; isn't the most appropriate approach, IMHO. If you are interested in these concepts in the context of neural networks, I suggest you read something about Bayesian machine learning or Bayesian neural networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-20T13:25:14.010" LastActivityDate="2020-02-20T13:25:14.010" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18149" PostTypeId="2" ParentId="18147" CreationDate="2020-02-20T13:19:19.977" Score="1" Body="&lt;p&gt;The procedures you mention don't need to involve writing code. There are now many ready-made tools available which implement various algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;POS-Tagging, Chunking, and Semantic Tagging/Disambiguation are knowledge-based procedures which can all be seen as classification/clustering tasks: POS-Tagging and Disambiguation are classifications, in that they propose a label (a POS- or semantic tag) that is assigned to a token. Chunking is a clustering algorithm, as it finds coherent groups in sequences of tokens. You could also view it as segmentation, as it splits a sentence into parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle they don't have anything to do with machine learning; the algorithm you are using (ML or not) is an implementation detail. Early Taggers were mostly rule-based, but with increasing availability of annotated data it became more feasible to use ML algorithms. This, however, has got nothing to do with how you classify the procedures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general I would call the activity you describe as 'annotation', as you enrich the source text by adding descriptive categories to tokens and token sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, no, they do not have to involve programming or implementing algorithms, and you don't need to be technical to perform them. Though some knowledge of linguistic concepts would help.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-20T13:19:19.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18150" PostTypeId="1" CreationDate="2020-02-20T13:30:39.850" Score="1" ViewCount="28" Body="&lt;p&gt;I started to dig into the topic of graph generation and I have a question - which out of generative methods (autoregressive, variational autoencoders, GANs, any other?) are better for generating graphs while preserving both node and edge labels? (in other words, I want to generate graphs with node and edge labels, similar to what I have in my dataset)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked over some papers, but I didn't find exact clues about how to choose the method for graph generation. If anybody is more familiar with the topic, I'll appreciate any link to papers or any advice/recommendations.&lt;/p&gt;&#xA;" OwnerUserId="33332" LastEditorUserId="2444" LastEditDate="2020-02-20T14:18:39.697" LastActivityDate="2020-02-20T14:18:39.697" Title="Which generative methods are better for generating graphs, while preserving node and edge labels?" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;geometric-deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18151" PostTypeId="1" AcceptedAnswerId="18177" CreationDate="2020-02-20T15:21:44.790" Score="1" ViewCount="131" Body="&lt;p&gt;I’m trying to figure out how to write an optimal convolutional neural network with respect to maximizing and minimizing filters in a convolution 2D layer. This is my thinking and I’m not sure if it's correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have a dataset of 32x32 images, we could start with a &lt;code&gt;Conv2D&lt;/code&gt; layer, filter of 3x3 and stride of 1x1. Therefore the maximum times this filter would be able to fit into the 32 x 32 images would be 30 times e.g. &lt;code&gt;newImageX * newImageY&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;newImageX = (imageX – filterX + 1)  &#xA;newImageY = (imageY – filterY + 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Am I right in thinking that because there are only &lt;code&gt;newImageX * newImageY&lt;/code&gt; patterns in the 32 x 32 image, that the maximum amount of filters should be &lt;code&gt;newImageX * newImageY&lt;/code&gt;, and any more would be redundant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the following code is the maximum possible filters given that we have 3x3 filter, 1x1 stride and 32x32 images?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Conv2D((30*30), kernel=(3,3), stride=(1,1), input_shape=(32,32,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any reason to go above &lt;code&gt;30*30&lt;/code&gt; filters, and are there any reasons to go below this number, assuming that &lt;code&gt;kernel&lt;/code&gt;, &lt;code&gt;stride&lt;/code&gt; and &lt;code&gt;input_shape&lt;/code&gt; remain the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you knew you were looking for one specific filter, would you have to use the maximum amount of filters to ensure that the one you were looking for was present, or can you include it another way?&lt;/p&gt;&#xA;" OwnerUserId="11795" LastEditorUserId="2444" LastEditDate="2020-02-22T02:30:06.970" LastActivityDate="2020-02-22T02:46:17.837" Title="How do I optimize the number of filters in a convolution layer?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18152" PostTypeId="1" CreationDate="2020-02-20T15:27:58.787" Score="2" ViewCount="19" Body="&lt;p&gt;I'm a newbie in neural networks. I'm trying to fit my neural network that has 3 different outputs: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;semantic segmentation, &lt;/li&gt;&#xA;&lt;li&gt;box mask and &lt;/li&gt;&#xA;&lt;li&gt;box coordinates.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;When my model is training, the loss of semantic segmentation and box coordinates are decreasing, but the loss of the box mask is increasing too much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My neural network is a CNN and it's based on Chargrid from &lt;a href=&quot;https://arxiv.org/abs/1809.08799&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The architecture is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f9L0E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f9L0E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For semantic segmentation outputs, it's expected to have 15 classes, for box mask it's expected to have 2 classes (foreground vs background) and for box coordinates it's expected to have 5 classes (1 for each corner of bounding box + 1 for None). &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Loss details&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Step 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here's the loss/accuracy for each of the three outputs at the end of the first step.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Semantic segmentation (Output 1) - 0.0181/0.958&lt;/li&gt;&#xA;&lt;li&gt;Box mask (Output 2) - 13.88/0.946&lt;/li&gt;&#xA;&lt;li&gt;Box coordinates (Output 3) - 0.2174/0.0000000867&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Last step&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here's the loss/accuracy at the last step.             &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Semantic segmentation (Output 1) - 0.0157/0.963&lt;/li&gt;&#xA;&lt;li&gt;Box mask (Output 2) - 73.02/0.935&lt;/li&gt;&#xA;&lt;li&gt;Box coordinates (Output 3) - 0.06/0.82&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is that normal? How can I interpret these results?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will leave below, the output of the model fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/c7fQs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c7fQs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wtGBc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wtGBc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fNWl8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fNWl8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33545" LastEditorUserId="2444" LastEditDate="2020-02-20T23:25:15.900" LastActivityDate="2020-02-20T23:25:15.900" Title="Why is the loss of one of the outputs of a model with multiple outputs increasing while the others are decreasing?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;overfitting&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18153" PostTypeId="2" ParentId="18069" CreationDate="2020-02-20T18:00:01.943" Score="1" Body="&lt;p&gt;I found a method to do it in the paper &lt;a href=&quot;http://openaccess.thecvf.com/content_CVPR_2019/papers/Murrugarra-Llerena_Cross-Modality_Personalization_for_Retrieval_CVPR_2019_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross-Modality Personalization for Retrieval&lt;/a&gt; (2020, accessed: 20-Feb-2020).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Representation.&lt;/strong&gt; For images, we extract Inception-v4 CNN features [36]. We then mask the image convolution feature with the BubbleView saliency map, by resizing the saliency map to the convolution feature size and multiplying them together. Finally, average pooling is performed to obtain a 1536-dimensional feature vector. We represent textual descriptions as their average 200-dimensional Glove embedding [31]. For personality, we use a 10-dimensional feature vector containing the scores for the personality questions in [32]. Below, we describe how we learn projections of these representations that place them in the same feature space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2020-02-20T23:50:18.353" LastActivityDate="2020-02-20T23:50:18.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18154" PostTypeId="2" ParentId="18122" CreationDate="2020-02-20T19:48:23.957" Score="0" Body="&lt;p&gt;I found the issue, I should have done more unit testing. Upon computing the batch loss before backpropagation, one of the dimension of the &quot;prediction&quot; tensor was not corresonding to the &quot;truth&quot; tensor. The shape match but the content is not the one that was supposed to be.&#xA;This is due to how the NLL loss is implemented in pytorch which I was not aware of ... &lt;/p&gt;&#xA;" OwnerUserId="33658" LastActivityDate="2020-02-20T19:48:23.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18155" PostTypeId="1" CreationDate="2020-02-20T21:07:09.717" Score="2" ViewCount="227" Body="&lt;p&gt;I am trying to test DQN on FrozenWorld environment in gym using TensorFlow 2.x. The update rule is (off policy)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) \leftarrow Q(s,a)+\alpha (r+\gamma~ max_{a'}Q(s',a')-Q(s,a))$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using an epsilon greedy policy. &#xA;In this environment, we get a reward only if we succeed. So I explored with 100% until I have 50 successes. Then I saved the data of failures and success in different bins. Then I sampled (with replacement) from these bins and used them to train the Q network. However, no matter how long I train the agent doesn't seem to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is available in &lt;a href=&quot;https://colab.research.google.com/drive/1qNsSuUzu6Q4tvQ2d4ZoXkhpQIPaUXjm_&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colab&lt;/a&gt;. I am doing this for a couple of days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I modified the code for SARSA and Expected SARSA; nothing works.&lt;/p&gt;&#xA;" OwnerUserId="31445" LastEditorUserId="2444" LastEditDate="2020-02-20T23:11:04.380" LastActivityDate="2021-03-17T02:06:40.667" Title="Why isn't my implementation of DQN using TensorFlow on the FrozenWorld environment working?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;dqn&gt;&lt;gym&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18157" PostTypeId="1" CreationDate="2020-02-21T00:18:55.033" Score="2" ViewCount="98" Body="&lt;p&gt;Does a fully convolutional network share the same translation invariance properties we get from networks that use max-pooling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, why do they perform as well as networks which use max-pooling? &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-06-13T20:43:57.133" LastActivityDate="2021-03-10T22:04:45.437" Title="Does a fully convolutional network share the same translation invariance properties we get from networks that use max-pooling?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18158" PostTypeId="2" ParentId="18155" CreationDate="2020-02-21T00:28:46.467" Score="0" Body="&lt;p&gt;I see at least 3 issues with your DQN code that need to be fixed:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You should not have separate replay memories for successes/failures. Put all of your experiences in one replay memory and sample from it uniformly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your replay memory is extremely small with only 2,000 samples. You need to make it significantly larger; try at least 100,000 up to 1,000,000 samples.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your &lt;code&gt;batch_target&lt;/code&gt; is incorrect. You need to train on &lt;em&gt;returns&lt;/em&gt; and not just rewards. In your &lt;code&gt;train&lt;/code&gt; function, compute the 1-step return &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \cdot max_{a'} Q(s',a')$&lt;/span&gt;, remembering to set &lt;span class=&quot;math-container&quot;&gt;$max_{a'} Q(s',a') = 0$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is terminal, and then pass it to &lt;code&gt;model.fit()&lt;/code&gt; as your prediction target.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32070" LastActivityDate="2020-02-21T00:28:46.467" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18159" PostTypeId="2" ParentId="18157" CreationDate="2020-02-21T00:40:12.087" Score="0" Body="&lt;p&gt;All convolutional networks (with or without max-pooling) are translation-invariant (AKA spatially invariant) because their filters slide over every position in the image. This means that if a pattern that &quot;matches&quot; a filter is present anywhere in the image, then at least one neuron should activate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Max-pooling, on the other hand, has nothing to do with spatial invariance. It's simply a regularization technique to help reduce the number of parameters later in the network by downsizing activation layers within the network. This can help combat overfitting, although it's not strictly necessary. Alternatively, neural networks can achieve the same effect by using a convolutional layer with a stride of 2 instead of 1.&lt;/p&gt;&#xA;" OwnerUserId="32070" LastActivityDate="2020-02-21T00:40:12.087" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18160" PostTypeId="1" CreationDate="2020-02-21T01:12:46.233" Score="0" ViewCount="35" Body="&lt;p&gt;I'm trying to create an AI that makes it's own computer programs. What do you think  would be good inputs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, one input might just be the instructions of what the program does in natural language. Then the AI might have to convert this into a formal program. But in a way if you have the instructions, they are already pretty much a program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another type of input might be for example a list of numbers. e.g. 2,4,6,8,....&#xA;And the AI must create a program like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10 LET X=2&#xA;20 PRINT X&#xA;30 LET X=X+2&#xA;40 GOTO 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or a list of strings like &quot;XA1&quot;, &quot;XB2&quot;, &quot;XC3&quot;,... And the program it would invent would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10 LET X=1&#xA;20 PRINT &quot;X&quot;+CHR$(64+X)+X&#xA;30 LET X=X+2&#xA;40 GOTO 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But perhaps this input is too complicated as it has to do 2 tasks: finding patterns in the input and making the program. What do you think would be good inputs for such an AI?&lt;/p&gt;&#xA;" OwnerUserId="4199" LastActivityDate="2020-02-21T01:12:46.233" Title="What inputs would a programming AI take?" Tags="&lt;ai-design&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18161" PostTypeId="1" CreationDate="2020-02-21T01:25:47.460" Score="1" ViewCount="48" Body="&lt;p&gt;I'm working on a problem that given a dataset; where each train example is a binary matrix &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; with dimension &lt;span class=&quot;math-container&quot;&gt;$(N_i,D_i)$&lt;/span&gt; (think a training example is a feature matrix) each entry is either 1 or 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, each training example &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; has a corresponding label &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt; is a correlation matrix dimension &lt;span class=&quot;math-container&quot;&gt;$(D_i,D_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to construct a model that take the input &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; and coutput &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}_i$&lt;/span&gt; that matches with &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The major challenges here is that; each training example &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; can have different dimensionality &lt;span class=&quot;math-container&quot;&gt;$(N_i,D_i)$&lt;/span&gt; different data points and different number of variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if there is any neural network architecture can handle case like this ?&lt;/p&gt;&#xA;" OwnerUserId="33704" LastEditorUserId="33704" LastEditDate="2020-02-21T01:55:09.033" LastActivityDate="2020-02-21T01:55:09.033" Title="Can neural network be trained to solve this problem?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18162" PostTypeId="1" AcceptedAnswerId="18165" CreationDate="2020-02-21T10:04:43.063" Score="3" ViewCount="81" Body="&lt;p&gt;Some sources consider the true negatives (TN) when computing the accuracy, while some don't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source 1:&#xA;&lt;a href=&quot;https://medium.com/greyatom/performance-metrics-for-classification-problems-in-machine-learning-part-i-b085d432082b&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/greyatom/performance-metrics-for-classification-problems-in-machine-learning-part-i-b085d432082b&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qZ1dN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qZ1dN.png&quot; alt=&quot;Considers TN&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source 2:&lt;a href=&quot;https://www.researchgate.net/profile/Mohammad_Sorower/publication/266888594_A_Literature_Survey_on_Algorithms_for_Multi-label_Learning/links/58d1864392851cf4f8f4b72a/A-Literature-Survey-on-Algorithms-for-Multi-label-Learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/profile/Mohammad_Sorower/publication/266888594_A_Literature_Survey_on_Algorithms_for_Multi-label_Learning/links/58d1864392851cf4f8f4b72a/A-Literature-Survey-on-Algorithms-for-Multi-label-Learning.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v8XPx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v8XPx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be translated as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZNOum.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZNOum.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which one of these must be considered for my multi-label model.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2020-02-21T13:38:19.690" LastActivityDate="2020-02-21T18:08:30.167" Title="Why is there more than one way of calculating the accuracy?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;accuracy&gt;&lt;metric&gt;&lt;multi-label-classification&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18163" PostTypeId="1" CreationDate="2020-02-21T10:46:19.253" Score="5" ViewCount="86" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Sustainable_Development_Goals&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sustainable Development Goals&lt;/a&gt; of the United Nations describe a normative framework which states what future development until 2030 should strive for. On a more abstract level a &lt;a href=&quot;https://en.wikipedia.org/wiki/Our_Common_Future&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic definition&lt;/a&gt; describes sustainable development as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;development that meets the needs of the present without compromising the ability of future generations to meet their own needs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Alone through the consumption of energy, AI technologies already have a (negative) impact on sustainability questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What AI applications already exist, are researched or are at least thinkable from which sustainability would benefit?&lt;/p&gt;&#xA;" OwnerUserId="33714" LastEditorUserId="2444" LastEditDate="2020-02-21T13:53:41.627" LastActivityDate="2020-02-21T19:17:40.970" Title="What AI applications exist to solve sustainability issues?" Tags="&lt;philosophy&gt;&lt;reference-request&gt;&lt;social&gt;&lt;ethics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18165" PostTypeId="2" ParentId="18162" CreationDate="2020-02-21T13:13:10.323" Score="4" Body="&lt;p&gt;In machine learning, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow noreferrer&quot;&gt;accuracy&lt;/a&gt; is usually defined as the number of correct predictions divided by the total number of predictions. The correct predictions are the true positives (&lt;span class=&quot;math-container&quot;&gt;$\mathrm {TP}$&lt;/span&gt;) and true negatives (&lt;span class=&quot;math-container&quot;&gt;$\mathrm {TN}$&lt;/span&gt;), so the usual formula to calculate the accuracy is the following one (your first one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text{Accuracy}=\frac {\mathrm {TP} + \mathrm {TN}}{\mathrm {TP} + \mathrm {TN} + \mathrm {FP} + \mathrm {FN}} \label{0}\tag{0}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following formula corresponds to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow noreferrer&quot;&gt;threat score (&lt;span class=&quot;math-container&quot;&gt;$\mathrm{TS}$&lt;/span&gt;) or critical success index (&lt;span class=&quot;math-container&quot;&gt;$\mathrm {CSI}$&lt;/span&gt;)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;{\displaystyle \mathrm {TS} ={\frac {\mathrm {TP} }{\mathrm {TP} +\mathrm {FN} +\mathrm {FP} }}} \label{1} \tag{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section 7 of the paper &lt;a href=&quot;https://www.researchgate.net/publication/266888594_A_Literature_Survey_on_Algorithms_for_Multi-label_Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Literature Survey on Algorithms for Multi-label Learning&lt;/a&gt;, it is written&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In traditional classification such as multi-class problems, &lt;strong&gt;accuracy&lt;/strong&gt; is the most common evaluation criteria. Additionally, there exists a set of standard evaluation metrics that includes precision, recall, F-measure, and ROC area defined for single-label multi-class classification problems. However, in &lt;strong&gt;multi-label classification&lt;/strong&gt;, predictions for an instance is a set of labels and, therefore, the prediction can be &lt;strong&gt;fully correct&lt;/strong&gt;, &lt;strong&gt;partially correct (with different levels of correctness)&lt;/strong&gt; or &lt;strong&gt;fully incorrect&lt;/strong&gt;. None of these existing evaluation metrics capture such notion in their original form.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Depending on the target problem, evaluation measures for multi-label data can be grouped into at least three groups: &lt;strong&gt;evaluating partitions&lt;/strong&gt;, &lt;strong&gt;evaluating ranking&lt;/strong&gt; and &lt;strong&gt;using label hierarchy&lt;/strong&gt;. The first one evaluates the quality of the classification into classes, the second one evaluates if the classes&#xA;  are ranked in order relevance and the third one evaluates how effectively the learning system is able to take into account an existing hierarchical structure of the labels.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In section 7.1, the authors state&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To capture the notion of &lt;strong&gt;partially correct&lt;/strong&gt;, one strategy is to &lt;strong&gt;evaluate the average difference between the predicted labels and the actual labels for each test example&lt;/strong&gt;, and then average over all examples in the test set. This approach is called &lt;strong&gt;example-based evaluations&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To capture the notion of partially incorrect in multi-label classification problems, they then use the following definition of accuracy (proposed in section 5.2 of the paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.423.3142&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Discriminative Methods for Multi-labeled Classification&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{1}{n} \sum_{i=1}^{n} \frac{|Y_i \cap Z_i|}{|Y_i \cup Z_i|} \label{2} \tag{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt; is the true set of labels and &lt;span class=&quot;math-container&quot;&gt;$Z_i$&lt;/span&gt; the predicted set of labels for the single instance (or observation) &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$\frac{|Y_i \cap Z_i|}{|Y_i \cup Z_i|}$&lt;/span&gt; is the accuracy for the instance &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$|Y_i|=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|Z_i|=1$&lt;/span&gt;, then we have a single-label classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The metric \ref{2} does not correspond to the metric \ref{1}. In \ref{2}, the accuracy is calculated for each example (or instance) and then we average all these accuracies, because, for each example, you may have that one predicted label is correct, two predicted labels are correct, and so on (also depending on the number of labels you need to predict for each of the examples). In \ref{1}, there is no notion of multiple labels.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-21T18:08:30.167" LastActivityDate="2020-02-21T18:08:30.167" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18166" PostTypeId="5" CreationDate="2020-02-21T13:39:54.103" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-21T13:39:54.103" LastActivityDate="2020-02-21T13:39:54.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18167" PostTypeId="4" CreationDate="2020-02-21T13:39:54.103" Score="0" Body="For questions related to the multi-label classification problem, i.e. each example (or instance) can be labelled with more than one label." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-21T13:39:54.103" LastActivityDate="2020-02-21T13:39:54.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18168" PostTypeId="2" ParentId="18097" CreationDate="2020-02-21T14:37:23.887" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ahjz1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ahjz1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://blog.easysol.net/building-ai-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.easysol.net/building-ai-applications/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When data is too big, complex and nonlinear it's time to try deep learning. It's always good to try to add some layers to see it can eliminate bias and don't lead to high variance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning models can be tweaked(hyperparameters) and regularized(parameters), and it is worth the work.  &lt;/p&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2020-02-21T14:37:23.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18169" PostTypeId="1" CreationDate="2020-02-21T15:07:52.047" Score="2" ViewCount="231" Body="&lt;p&gt;I'm trying to solve a binary classification problem with AlexNet. I split the original dataset into training and validation datasets using a 70/30 ratio. I have trained my neural network with a dataset of 11200 images, and I obtained a training accuracy of 99% and a validation accuracy was 96%. At the end of the training, I saved my model's weights to a file.&lt;/p&gt;&#xA;&lt;p&gt;After training, I loaded the saved weights to the same neural network. I chose 738 images out of the 11200 training images, and I tried to predict the class of each of them with my model, and compare them with true labels, then again I calculated the accuracy percentage and it was 74%.&lt;/p&gt;&#xA;&lt;p&gt;What is the problem here? I guess its accuracy should be about 96% again.&lt;/p&gt;&#xA;&lt;p&gt;Here's the code that I'm using.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prelist=[]&#xA;for i in range(len(x)):&#xA;    prediction = model.predict_classes(x[i])&#xA;    prelist.append(prediction)&#xA;count = 0&#xA;for i in range(len(x)):&#xA;    if(y[i] == prelist[i]):&#xA;        count = count + 1&#xA;test_precision = (count/len(x))*100&#xA;print (test_precision)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I use &lt;code&gt;predict_classes&lt;/code&gt; on 11200 images that I used to train the neural network and compare its result with true labels and calculated accuracy again its accuracy is 91%.&lt;/p&gt;&#xA;" OwnerUserId="33626" LastEditorUserId="2444" LastEditDate="2021-01-17T19:24:35.350" LastActivityDate="2021-02-16T20:08:00.790" Title="Why am I getting a difference between training accuracy and accuracy calculated with Keras' predict_classes on a subset of the training data?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;accuracy&gt;&lt;binary-classification&gt;&lt;alexnet&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18171" PostTypeId="1" AcceptedAnswerId="18172" CreationDate="2020-02-21T16:23:15.443" Score="3" ViewCount="59" Body="&lt;p&gt;I am learning about policy gradient methods from the Deep RL Bootcamp by Peter Abbeel and I am a bit stumbled by the math presented. In the lecture, he derives the gradient logarithm likelihood of a trajectory to be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla log P(\tau^{i};\theta) = \Sigma_{t=0}\nabla_{\theta}log\pi(a_{t}|s_t, \theta).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &lt;span class=&quot;math-container&quot;&gt;$\pi(a_{t} | s_{t}, \theta)$&lt;/span&gt; a distribution or a function? Because a derivative can only be taken wrt a function. My understanding is that &lt;span class=&quot;math-container&quot;&gt;$\pi(a_{t},s_{t}, \theta)$&lt;/span&gt; is usually represented as a distribution of actions over states, since input of a neural network for policy gradient would be the &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and output would be &lt;span class=&quot;math-container&quot;&gt;$\pi(a_t, s_t)$&lt;/span&gt;, using model weights &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-02-21T18:27:56.637" LastActivityDate="2020-02-21T18:29:19.887" Title="In the policy gradient equation, is $\pi(a_{t} | s_{t}, \theta)$ a distribution or a function?" Tags="&lt;math&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;stochastic-policy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18172" PostTypeId="2" ParentId="18171" CreationDate="2020-02-21T18:15:04.813" Score="4" Body="&lt;p&gt;First, the derivative is usually taken with respect to a variable (input) of the function. Hence the notation &lt;span class=&quot;math-container&quot;&gt;$\frac{df}{dx}$&lt;/span&gt; for some function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at your equation more carefully&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla log P(\tau^{i};\theta) = \Sigma_{t=0}\nabla_{\theta}log\pi(a_{t}|s_t, \theta).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will see that the gradient is taken with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, which are the parameters (i.e. a vector) e.g. of your neural network, that is, &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, it doesn't really matter whether &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; represents a distribution or not (for some specific value of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;), but you're right that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; often represents a probability distribution over the possible actions (given a specific state). In any case, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a function of the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (i.e. in the case of a distribution, &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta}$&lt;/span&gt; is a family of distributions for all possible values of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;), i.e. if you change &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; the outputs of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; also change, so you can take the derivative of it with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-21T18:29:19.887" LastActivityDate="2020-02-21T18:29:19.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18173" PostTypeId="2" ParentId="18163" CreationDate="2020-02-21T19:04:31.907" Score="4" Body="&lt;p&gt;The paper &lt;a href=&quot;https://www.nature.com/articles/s41467-019-14108-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;The role of artificial intelligence in achieving the Sustainable Development Goals&lt;/a&gt; (2020, published in Nature) should contain the information you're looking for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the introduction, the authors write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here we present and discuss &lt;strong&gt;implications of how AI can either enable or inhibit the delivery of all 17 goals and 169 targets recognized in the 2030 Agenda for Sustainable Development&lt;/strong&gt;. Relationships were characterized by the methods reported at the end of this study, which can be summarized as a consensus-based expert elicitation process, informed by previous studies aimed at mapping SDGs interlinkages. A summary of the results is given in Fig. 1 and the &lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fs41467-019-14108-y/MediaObjects/41467_2019_14108_MOESM2_ESM.xlsx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supplementary Data 1&lt;/a&gt; provides a complete list of all the SDGs and targets, together with the detailed results from this work. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For example, as stated in the &lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fs41467-019-14108-y/MediaObjects/41467_2019_14108_MOESM2_ESM.xlsx&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary data&lt;/a&gt;, goal 1 is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;End poverty in all its forms everywhere&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and the first target (1.1.) of goal 1 is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By 2030, eradicate extreme poverty for all people everywhere, currently measured as people living on less than $1.25 a day &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then the authors suggest that, according to their studies, AI may act as an inhibitor or enabler (i.e. it may be used to fight poverty) for this target.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We identified in the literature studies suggesting that AI may be an inhibitor for this target, due to the potential increase in inequalities which would hinder the achievement of this goal (1). Other references however identify AI as an enabler for this goal, in the context of using &lt;strong&gt;satellite data analysis to track areas of poverty and to foster international collaboration&lt;/strong&gt; (2).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Therefore, techniques for satellite data analysis are one of the AI techniques that may be used to tackle suistainability issues.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-21T19:17:40.970" LastActivityDate="2020-02-21T19:17:40.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18174" PostTypeId="2" ParentId="18125" CreationDate="2020-02-22T01:03:50.330" Score="2" Body="&lt;p&gt;This question really looks like a homework problem, in part because it is too vague (what does it mean to 'get stuck' &lt;em&gt;exactly&lt;/em&gt;?).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Hill climbing stops when it reaches a local maximum.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hill climbing is an uninformed search algorithm, so it does not make use of a heuristic.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hill climbing may or may not stop on a ridge, depending on the implementation. Some common implementations move randomly when equal (but not better) moves are available. Others stop in this situation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hill climbing may or may not stop on a plateau, depending on how the implementation handles the case of ties, like with ridges. Whether you think it 'gets stuck' depends a lot on your definition. Implementations that stop when encountering a tie will stop. Implementations that don't stop will eventually exit the plateau through random movements, but it could take infinite time if your movements in the gradient-free region are unlucky.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-11-24T02:36:44.903" LastActivityDate="2020-11-24T02:36:44.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18175" PostTypeId="2" ParentId="18082" CreationDate="2020-02-22T01:11:10.873" Score="0" Body="&lt;p&gt;Financial information companies spend a very large amount of effort to do this kind of thing properly, and their models are generally proprietary, so the real answer is not something you'll be able to get on a public site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you can achieve a very simple approximation with a literal diff on the texts, and a more sophisticated one by using out-of-the-box NLP or computer vision techniques, but you're probably not going to get close to the performance of factset or other competitors using those tools.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-02-22T01:11:10.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18176" PostTypeId="2" ParentId="17821" CreationDate="2020-02-22T01:20:26.403" Score="0" Body="&lt;p&gt;A &lt;em&gt;kernel&lt;/em&gt; is a precomputed distance function. This means that instead of computing the distances between pairs of points every time you need them, you'll just compute them all a single time, at the start, and cache the values in memory. This is a great idea if the &lt;em&gt;dimensionality&lt;/em&gt; of your data is very high, and you expect to have to make a lot of distance computations. For example, if each vector was an high definition image with millions of pixels, computing the distance between two vectors would cost millions of operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A problem with this approach is that if you have many &lt;em&gt;exemplars&lt;/em&gt; in your dataset, computing and storing the distances between every pair will become hugely expensive. In your case, you have 20 million exemplars. This means you have ~&lt;span class=&quot;math-container&quot;&gt;$20,000,000^2$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$4*10^14$&lt;/span&gt; pairs of distances to compute. If your computer does ~&lt;span class=&quot;math-container&quot;&gt;$10^9$&lt;/span&gt; computations per second (as is normal these days), this will take you about &lt;span class=&quot;math-container&quot;&gt;$10^5$&lt;/span&gt; seconds, or a bit more than a whole day. Much worse than this, you will need approximately 400TB of RAM to store the kernel. If you don't have this, your system will eventually start writing to disk, which will cause the calculation to take about 3 years instead of 1 day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution here is not to do quadratic work with millions of datapoints. Either use a subset of the data, or try approximate distance methods.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-02-22T01:20:26.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18177" PostTypeId="2" ParentId="18151" CreationDate="2020-02-22T02:35:14.603" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I right in thinking that because there are only &lt;code&gt;newImageX * newImageY&lt;/code&gt; patterns in the &lt;code&gt;32 x 32&lt;/code&gt; image, that the maximum amount of filters should be &lt;code&gt;newImageX * newImageY&lt;/code&gt;, and any more would be redundant?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your assumption is wrong. If you have a &lt;span class=&quot;math-container&quot;&gt;$32 \times 32$&lt;/span&gt; images (so consider only grayscale images), then you have &lt;span class=&quot;math-container&quot;&gt;$256^{32 \times 32}$&lt;/span&gt; possible patterns (i.e. a &lt;a href=&quot;https://www.universetoday.com/36302/atoms-in-the-universe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;huge number of possible patterns!&lt;/a&gt;), given that, for each pixel, you can have &lt;span class=&quot;math-container&quot;&gt;$256$&lt;/span&gt; values, and there are &lt;span class=&quot;math-container&quot;&gt;$32\times 32$&lt;/span&gt; pixels. Of course, for a human, many of these patterns may be very similar to each other (or even indistinguishable), but the way you measure similarity, in this case, is another hard problem. Hence also the difficulty in choosing the number of filters.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-22T02:46:17.837" LastActivityDate="2020-02-22T02:46:17.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18178" PostTypeId="1" CreationDate="2020-02-22T04:12:42.603" Score="2" ViewCount="252" Body="&lt;p&gt;I am currently studying constraint satisfaction problems and have come across two heuristics for &lt;strong&gt;variable selection&lt;/strong&gt;. The &lt;strong&gt;minimum remaining values&lt;/strong&gt;(MRV) heuristic and the &lt;strong&gt;degree&lt;/strong&gt; heuristic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MRV heuristic tells you to choose a variable that has the least legal assignments, while the degree heuristic tells you to choose a variable that has the biggest effect on the remaining unassigned variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can these 2 heuristics for variable selection be used in conjunction with each other? Some books say that degree heuristic can be used as first selection of variables. Which heuristic is better followed after that? The MRV or degree heuristic? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oLd0u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oLd0u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The picture shows the case where degree heuristic is used. If MRV were to be used, then at the 3rd step, the leftmost side of the map would be coloured blue.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-02-22T12:20:27.580" LastActivityDate="2020-02-22T12:20:27.580" Title="Can the degree and minimum remaining values heuristics be used in conjunction?" Tags="&lt;heuristics&gt;&lt;graphs&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18180" PostTypeId="1" CreationDate="2020-02-22T10:42:07.060" Score="1" ViewCount="33" Body="&lt;p&gt;What is the difference between a generalised estimating equation (GEE) model and a recurrent neural network (RNN) model, in terms of what these two models are doing? Apart from the differences in the structure of these two models, where GEE is an extension of generalised linear model (GLM) and RNN is of a neural network structure, it seems to me that these 2 models are doing the same thing?&lt;/p&gt;&#xA;" OwnerUserId="33734" LastEditorUserId="2444" LastEditDate="2020-02-22T12:29:45.033" LastActivityDate="2020-02-22T12:29:45.033" Title="What is the difference between an generalised estimating equation and a recurrent neural network?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;linear-regression&gt;&lt;statistical-ai&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18181" PostTypeId="1" CreationDate="2020-02-22T15:15:01.533" Score="3" ViewCount="210" Body="&lt;p&gt;Tensor networks (check &lt;a href=&quot;https://arxiv.org/pdf/1812.04011.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for a review) are a numerical method originally introduced in condensed matter physics to model complex quantum systems. Roughly speaking, such systems are described by a very high-dimensional tensor (where the indices take a number of values scaling exponentially with the number of system constituents) and tensor networks provide an efficient representation of the latter as an outer product and contraction of many low-dimensional tensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, a specific kind of tensor network (called Matrix Product State in physics) found interesting applications in machine-learning through the so-called Tensor-Train decomposition (I do not know of a precise canonical reference in this context, so I will abstain from citing anything).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, over the last few years, several works from the physics community seemed to push for a generalized use of tensor networks in machine learning (see &lt;a href=&quot;https://arxiv.org/pdf/1605.05775.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1910.07425.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a second one&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/2001.08286.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a third one&lt;/a&gt; and &lt;a href=&quot;https://ai.googleblog.com/2019/06/introducing-tensornetwork-open-source.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; from Google AI for context). As a physicist, I am glad to learn that tools initially devised for physics may find interdisciplinary applications. However, at the same time, my critical mind tells me that from the machine learning research community's perspective, these results may not look that intriguing. After all, machine learning is now a very established field and it takes probably more than a suggestion for a new machine learning model and a basic benchmarking on a trivial dataset (as the MNIST one) -which is what the papers essentially do in my humble opinion- to attract any attention in the area. Besides, as I believe to know, there already exists quite a solid body of knowledge on tensor analysis techniques for machine learning (e.g.  tensor decompositions), which may cast doubt on the originality of the contribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would therefore be very curious to have the opinion of machine learning experts on this line of research: is it really an interesting direction to look into, or is it just about surfing on the current machine learning hype with a not-so-serious proposal?&lt;/p&gt;&#xA;" OwnerUserId="33737" LastActivityDate="2021-02-18T00:03:55.450" Title="Using tensor networks as machine learning models" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18182" PostTypeId="1" CreationDate="2020-02-22T16:34:40.850" Score="2" ViewCount="77" Body="&lt;p&gt;When computing receptive field recursively through a CNN, does a transposed convolution affect the receptive field the same way that a convolution does if the kernel and stride is the same?&lt;/p&gt;&#xA;" OwnerUserId="19789" LastActivityDate="2020-02-22T16:34:40.850" Title="How is the receptive field of a CNN affected by transposed convolution?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18183" PostTypeId="1" CreationDate="2020-02-22T16:39:44.603" Score="1" ViewCount="21" Body="&lt;p&gt;I'm using Weights and Biases to do some hyperparameter sweeping for a supervised sequence-to-sequence problem I'm working on. One thing I noticed is that the sweeps with a gradually increasing number of hidden units tend to have a lower validation loss:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1YxDR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1YxDR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if this is generally true or just a function of my particular problem?&lt;/p&gt;&#xA;" OwnerUserId="23719" LastActivityDate="2020-02-22T16:39:44.603" Title="When stacking LSTM's, should the hidden units increase?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18184" PostTypeId="1" CreationDate="2020-02-22T17:52:15.443" Score="1" ViewCount="113" Body="&lt;p&gt;I'm working on a project for my college to recognize traffic signs in pictures. I searched a lot but can't find the best method to do it.&lt;/p&gt;&#xA;&lt;p&gt;Can someone recommend me a paper, article, or even GitHub link that describes the best way to achieve this? It will be helpful.&lt;/p&gt;&#xA;" OwnerUserId="33739" LastEditorUserId="2444" LastEditDate="2020-09-11T15:04:17.157" LastActivityDate="2020-09-11T15:04:17.157" Title="What is the best way to detect and recognize traffic signs in a picture?" Tags="&lt;deep-learning&gt;&lt;object-recognition&gt;&lt;object-detection&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18185" PostTypeId="2" ParentId="18184" CreationDate="2020-02-22T20:16:36.677" Score="1" Body="&lt;p&gt;Maybe this is helpful:&#xA;&lt;a href=&quot;https://towardsdatascience.com/recognizing-traffic-signs-with-over-98-accuracy-using-deep-learning-86737aedc2ab&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recognising Traffic Signs With 98% Accuracy Using Deep Learning, by Eddie Forson&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Greetings&#xA;Mario&lt;/p&gt;&#xA;" OwnerUserId="33742" LastEditorUserId="5763" LastEditDate="2020-02-24T10:29:19.687" LastActivityDate="2020-02-24T10:29:19.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18186" PostTypeId="1" CreationDate="2020-02-22T21:31:55.750" Score="2" ViewCount="19" Body="&lt;p&gt;Is there a program under development that uses AI technology, like Siri, to &quot;hold hands&quot; so to speak with a language learner and coach them on accent, colloqiual expressions, or to let them guide the language learning process using an archive of language knowledge? &#xA;Also, could this sort of program be used to learn things in a language one already knows, or in a new language, say for the purposes of travel or to learn about related hyperlinks in an online database?&lt;/p&gt;&#xA;" OwnerUserId="33743" LastActivityDate="2020-02-22T21:31:55.750" Title="Language Learning feedback with AI" Tags="&lt;ai-design&gt;&lt;activation-function&gt;&lt;data-preprocessing&gt;&lt;text-classification&gt;&lt;knowledge-base&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18187" PostTypeId="1" CreationDate="2020-02-22T22:59:51.977" Score="2" ViewCount="635" Body="&lt;p&gt;Does anybody know a simple proof of the convergence of the TD(0) value function prediction algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DL575.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DL575.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-02-27T15:21:56.120" LastActivityDate="2020-07-26T16:01:35.137" Title="Is there a simple proof of the convergence of TD(0)?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18188" PostTypeId="1" CreationDate="2020-02-23T00:53:43.560" Score="1" ViewCount="228" Body="&lt;p&gt;I am using pytorch version of &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ppo.html#documentation-pytorch-version&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO&lt;/a&gt; and I have image input that I need to process with convolutional neural networks, are there any examples on how to set up the network? I know that &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;stable baselines&lt;/a&gt;  support this to some extend, but I had better performance with spinning up so I would prefer to keep using these. &lt;/p&gt;&#xA;" OwnerUserId="33744" LastActivityDate="2021-05-20T16:02:45.530" Title="OpenAI spinning up convolutional networks with PPO" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18189" PostTypeId="1" CreationDate="2020-02-23T04:31:43.473" Score="2" ViewCount="64" Body="&lt;p&gt;When a new node is added, the previous connection is disabled and not removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4b46s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4b46s.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any situation in which a connection gene is removed? For example, in the above diagram connection gene with innovation number 2 is not present. It could be because some other genome used that innovation number for a different connection that isn't present in this genome. But are there cases where a connection gene has to be removed?&lt;/p&gt;&#xA;" OwnerUserId="33746" LastEditorUserId="2444" LastEditDate="2020-02-23T12:53:27.570" LastActivityDate="2020-02-29T14:12:07.077" Title="Are connections genes in a genome ever deleted or just disabled?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18190" PostTypeId="1" CreationDate="2020-02-23T10:28:34.877" Score="1" ViewCount="31" Body="&lt;p&gt;I have a trained LSTM model created with keras. Is it possible to use tensorboard debugger or tensorflow debugger to debug the model during prediction runtime? Meaning that it steps through the model when I use model.predict() on my input data and show me the cell states, hidden states etc. Thanks! &lt;/p&gt;&#xA;" OwnerUserId="33499" LastActivityDate="2020-02-23T10:28:34.877" Title="Can tensorflow debugger debug a trained keras model during prediction?" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18191" PostTypeId="6" CreationDate="2020-02-23T11:19:42.723" Score="0" Body="&lt;h1&gt;Why vote for me?&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Relatively high reputation on the site&lt;/li&gt;&#xA;&lt;li&gt;Active on this site for multiple years&lt;/li&gt;&#xA;&lt;li&gt;Still constistently log on almost every day&lt;/li&gt;&#xA;&lt;li&gt;Formal education in AI (M.Sc., working on PhD)&lt;/li&gt;&#xA;&lt;li&gt;Actively publishing AI research&lt;/li&gt;&#xA;&lt;li&gt;I do care about the quality of the site and hope it continues to grow in quantity and quality&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Why not vote for me?&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Historically not been the most active on &lt;a href=&quot;https://ai.stackexchange.com/&quot;&gt;https://ai.stackexchange.com/&lt;/a&gt;, reviewing queues, our chatroom, etc. (have been sporadically present in all of them).&lt;/li&gt;&#xA;&lt;li&gt;Much more interested in answering questions in my direct area of expertise, than in janitorial duties.&lt;/li&gt;&#xA;&lt;li&gt;Primarily nominating because I was asked to, and because I think that increasing the number of nominees increases the &quot;credibility&quot; of the election and the site as a whole.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If I were to be elected, I don't think I could guarantee that I'd be the type of moderator who actively seeks out things to moderate. I would actively log on almost every day, and wouldn't mind taking a look when notified. So I guess I'd be able to work best when other people on the site are also keeping an active eye out, and flag when attention is required.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2020-02-23T11:19:42.723" LastActivityDate="2020-02-23T11:19:42.723" CommentCount="1" CommunityOwnedDate="2020-02-23T11:19:42.723" ContentLicense="CC BY-SA 4.0" />
  <row Id="18192" PostTypeId="1" CreationDate="2020-02-23T11:57:20.470" Score="2" ViewCount="35" Body="&lt;p&gt;Currently, many algorithms are available for &lt;a href=&quot;https://en.wikipedia.org/wiki/Inpainting&quot; rel=&quot;nofollow noreferrer&quot;&gt;image inpainting&lt;/a&gt;. In my application, I have some special restriction on training dataset-&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Let's consider the training dataset of human facial images.&lt;/li&gt;&#xA;&lt;li&gt;Although all human face has the same general structure, they may have subtle differences depending on racial characteristics.&lt;/li&gt;&#xA;&lt;li&gt;Consider that in the training dataset, we have ten facial images from each race.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, in my learning algorithm, can we come up with a two-step method? Wherein the first step, we will learn about the general facial structure more accurately using all training data. In the next step, we will learn those subtle features of each race by only learning ten images associated with that race? It might restore a distorted image more accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have a distorted facial image of a person from race 'A,' where the nasal area of that image is lost. Now with the first step, we can learn the nasal structure more accurately by using all of the training data, and in the second step using only the ten images associated with race 'A,' we can fine-tune those generated data. As we have only 10 data with race 'A,' if we use only those small subset data to learn the whole model, then probably we will not be able to capture the all general architecture of the face in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I am not from Computer Science/ML background, probably my problem description is a little bit vague. It would be great if someone provides an edit/tag suggestion.&lt;/p&gt;&#xA;" OwnerUserId="33750" LastEditorUserId="33750" LastEditDate="2020-02-24T04:27:31.443" LastActivityDate="2020-02-24T04:27:31.443" Title="Suggestion on image inpainting algorithm" Tags="&lt;generative-model&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18193" PostTypeId="1" CreationDate="2020-02-23T13:25:19.680" Score="2" ViewCount="141" Body="&lt;p&gt;I can't understand how playing with the action generated by the actor network in DDPG by adding the noise term helps in exploration.&lt;/p&gt;&#xA;" OwnerUserId="33751" LastActivityDate="2020-02-23T13:25:19.680" Title="How does adding noise to the action in DDPG help in learning?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18196" PostTypeId="2" ParentId="18184" CreationDate="2020-02-23T18:15:36.220" Score="1" Body="&lt;p&gt;Here are some articles, the first three include code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://becominghuman.ai/deep-learning-for-traffic-signs-recognition-aead0ea7273&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning for Traffic Signs Recognition, by Moataz Elmasry, April 2, 2018&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.pyimagesearch.com/2019/11/04/traffic-sign-classification-with-keras-and-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Traffic Sign Classification with Keras and Deep Learning, by Adrian Rosebrock, November 4, 2019&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/traffic-sign-detection-using-convolutional-neural-network-660fb32fe90e&quot; rel=&quot;nofollow noreferrer&quot;&gt;Traffic Sign Detection using Convolutional Neural Network, by Sanket Doshi, September 1, 2019&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.rmit.edu.au/news/all-news/2019/jun/street-view-signs&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI to manage road infrastructure via Google Street View, June 19, 2019&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-02-23T18:15:36.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18198" PostTypeId="1" AcceptedAnswerId="18199" CreationDate="2020-02-23T20:36:06.570" Score="1" ViewCount="1819" Body="&lt;p&gt;What's the difference between LSTM and RNN? I know that RNN is a layer used in neural networks, but what exactly is an LSTM? Is it also a layer with the same characteristics?&lt;/p&gt;&#xA;" OwnerUserId="33759" LastEditorUserId="2444" LastEditDate="2021-01-18T21:07:38.813" LastActivityDate="2021-01-18T21:13:09.837" Title="What’s the difference between LSTM and RNN?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;recurrent-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18199" PostTypeId="2" ParentId="18198" CreationDate="2020-02-23T21:39:37.643" Score="0" Body="&lt;p&gt;You can describe a recurrent neural network (RNN) or a long short-term memory (LSTM), depending on the context, at different levels of abstraction. For example, you could say that an RNN is any neural network that contains one or more recurrent (or cyclic) connections. Or you could say that layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; of neural network &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is a recurrent layer, given that it contains units (or neurons) with recurrent connections, but &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; may not contain only recurrent layers (for example, it may also be composed of feedforward layers, i.e. layers with units that contain only feedforward connections).&lt;/p&gt;&#xA;&lt;p&gt;In any case, a recurrent neural network is almost always described as a neural network (NN) and not as a layer (this should also be obvious from the name). On the other hand, an LSTM can refer to an LSTM unit (or neuron), an LSTM layer (many LSTM units), or an LSTM neural network (an NN with LSTM units or layers), depending on the context.&lt;/p&gt;&#xA;&lt;p&gt;An LSTM unit is a recurrent unit, that is, a unit (or neuron) that contains cyclic connections, so an LSTM network is a recurrent network. The main difference between an LSTM unit and a standard RNN unit is that the LSTM unit is more sophisticated. More precisely, it is composed of the so-called &lt;em&gt;gates&lt;/em&gt; that supposedly regulate better the flow of information through the unit.&lt;/p&gt;&#xA;&lt;p&gt;Here's a typical representation (or diagram) of an LSTM (more precisely, an LSTM with a so-called peephole connection).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ltQT0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ltQT0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can actually represent both an LSTM unit (and, in that case, the variables are scalars) or an LSTM layer (and, in that case, the variables are vectors or matrices). You can easily see from this diagram that an LSTM unit (or layer) is composed of gates and recurrent connections. It's also composed of a cell. To understand the details (i.e. the purpose of all these components, such as the gates), you should e.g. read &lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper that originally proposed the LSTM&lt;/a&gt; by S. Hochreiter and J. Schmidhuber. However, there may be other more accessible and understandable papers.&lt;/p&gt;&#xA;&lt;p&gt;Given the presence of cyclic connections, any recurrent neural network (either an LSTM or not) may be represented as a graph that contains one or more cyclic connections. For example, the following diagram may represent both a standard RNN or an LSTM network (or maybe a variant of it, e.g. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gated_recurrent_unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRU&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XUmdH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XUmdH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;RNNs are particularly suited for tasks that involve sequences (because of the recurrent connections). For example, they are often used for machine translation, where the sequences are sentences or words. In practice, an LSTM is often used, as opposed to a vanilla (or standard) RNN, because it is more computationally effective. In fact, the LSTM was introduced to solve a problem that standard RNNs suffer from, i.e. the &lt;a href=&quot;https://stats.stackexchange.com/q/320919/82135&quot;&gt;vanishing gradient problem&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T21:13:09.837" LastActivityDate="2021-01-18T21:13:09.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18200" PostTypeId="1" CreationDate="2020-02-23T22:18:45.943" Score="1" ViewCount="285" Body="&lt;p&gt;For a current project demo, I'm searching for a car detection neural network in Python written in TF/Keras (or any other type, as long as it has no C++ dependencies). Later on, I gonna write my own, but, for now, I need a working substitute. It should be easy to use and implement in my own Python code, even better if it's already trained. I searched the whole GitHub for this, but I can't find anything that would fit my needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone know where to find this type of code? And are there ones for pedestrian detection, too?&lt;/p&gt;&#xA;" OwnerUserId="33760" LastEditorUserId="2444" LastEditDate="2020-02-24T01:01:42.407" LastActivityDate="2020-07-31T00:04:05.510" Title="Is there a car detection software written in Tensorflow or Keras with Python?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;object-detection&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2020-08-10T17:11:29.490" ContentLicense="CC BY-SA 4.0" />
  <row Id="18201" PostTypeId="1" AcceptedAnswerId="18246" CreationDate="2020-02-24T00:12:19.963" Score="2" ViewCount="206" Body="&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\theta J(\pi_\theta) = \mathbb{E}_{\tau \sim \pi_\theta} \left[ \sum_{t = 0}^T \nabla_\theta \log \pi_\theta (a_t|s_t) R(\tau) \right]&#xA;$$&lt;/span&gt;&#xA;be the expanded expression for a simple policy gradient, where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the parameters of the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; denotes the expected return function, &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is a trajectory of states and actions, &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is a timestep index, and &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; gives the sum of rewards for a trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; be the set of all trajectories used for training. An estimator of the above policy gradient is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\hat{g} = \frac{1}{\mathcal{D}} \sum_{\tau \in \mathcal{D}} \sum_{t = 0}^T \nabla_\theta \log \pi_\theta (a_t|s_t) R(\tau).&#xA;$$&lt;/span&gt;&#xA;A loss function associated with this estimator, given a single trajectory with &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; timesteps, is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(\tau) = -\sum_{t = 0}^T \log \pi_\theta (a_t|s_t) R(\tau).&#xA;$$&lt;/span&gt;&#xA;Minimizing &lt;span class=&quot;math-container&quot;&gt;$L(\tau)$&lt;/span&gt; by SGD or a similar algorithm will result in a working policy gradient implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what is the proper terminology for this loss function? Is it an (unbiased?) estimator for the expected returns &lt;span class=&quot;math-container&quot;&gt;$J(\pi_\theta)$&lt;/span&gt; if summed over all trajectories? If someone is able to provide a proof that minimizing &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; maximizes &lt;span class=&quot;math-container&quot;&gt;$J(\pi_\theta)$&lt;/span&gt;, or point me to a reference for this, that would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="33762" LastActivityDate="2020-02-25T20:50:11.643" Title="Is the negative of the policy loss function in a simple policy gradient algorithm an estimator of expected returns?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18202" PostTypeId="1" CreationDate="2020-02-24T04:24:16.393" Score="1" ViewCount="16" Body="&lt;p&gt;Good day everyone,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would just like to ask if anyone part of a lab or company doing research on aerial robotics has any suggestions of a good platform for deploying computer vision algorithms for aerial robots?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently our lab has a set of DJI Matrice drones but are too heavy for our liking. We really wanted to use the Skydio R2 drone for out future research projects but found out later that the SDK does not allow access for implementing our own Deep Learning or Reinforcement Learning networks (only allows the user to code their own preset movements in python). We also took a look at the Parrot AR drones but found that they were discontinued and do not have the computing power that the Skydio has although as an alternative, it does have the capability to stream the video feed to an external computer for online processing instead of on-board.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggested that we stick with the DJI Matrice and just use an Nvidia Jetson for deployment but I am still curious to know if anyone knows of other available platforms. Does anyone know of a more compact platform available for research purposes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you :)&lt;/p&gt;&#xA;" OwnerUserId="17685" LastActivityDate="2020-02-24T04:24:16.393" Title="Drone Deployment Platform for Neural Networks" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18204" PostTypeId="1" CreationDate="2020-02-24T05:29:12.907" Score="7" ViewCount="345" Body="&lt;p&gt;I was listening to a podcast on the topic of AGI and a guest made an argument that if strong music generation were to happen, it would be a sign of &amp;quot;true&amp;quot; intelligence in machines because of how much creative capability creating music requires (even for humans).&lt;/p&gt;&#xA;&lt;p&gt;It got me wondering, what other events/milestones would convince someone, who is more involved in the field than myself, that we might have implemented an AGI (or a &amp;quot;highly intelligent&amp;quot; system)?&lt;/p&gt;&#xA;&lt;p&gt;Of course, the answer to this question depends on the definition of AGI, but you can choose a sensible definition of AGI in order to answer this question.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, maybe some of these milestones or events could be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;General conversation&lt;/li&gt;&#xA;&lt;li&gt;Full self-driving car (no human intervention)&lt;/li&gt;&#xA;&lt;li&gt;Music generation&lt;/li&gt;&#xA;&lt;li&gt;Something similar to AlphaGo&lt;/li&gt;&#xA;&lt;li&gt;High-level reading/comprehension&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What particular event would convince you that we've reached a high level of intelligence in machines?&lt;/p&gt;&#xA;&lt;p&gt;It does not have to be any of the events I listed.&lt;/p&gt;&#xA;" OwnerUserId="22840" LastEditorUserId="2444" LastEditDate="2021-01-18T18:22:21.673" LastActivityDate="2021-02-02T09:30:53.000" Title="What event would confirm that we have implemented an AGI system?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;intelligence-testing&gt;" AnswerCount="6" CommentCount="0" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18205" PostTypeId="1" CreationDate="2020-02-24T07:27:49.423" Score="2" ViewCount="84" Body="&lt;p&gt;I have a set of images, which are quite large in size (1000x1000), and as such do not easily fit into memory. I'd like to compress these images, such that little information is missing. I am looking to use a CNN for a reinforcement learning task which involves a lot of very small objects which may disappear when downsampling. What is the best approach to handle this without downscaling/downsampling the image and losing information for CNNs?&lt;/p&gt;&#xA;" OwnerUserId="33058" LastEditorUserId="33058" LastEditDate="2020-02-24T21:49:29.307" LastActivityDate="2020-03-02T22:56:30.857" Title="How can I perform lossless compression of images so that they can be stored to train a CNN?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;numpy&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18206" PostTypeId="1" CreationDate="2020-02-24T08:11:59.917" Score="1" ViewCount="3779" Body="&lt;p&gt;I have spent some time searching Google and wasn't able to find out what kind of optimization algorithm is best for binary classification when images are &lt;strong&gt;similar&lt;/strong&gt; to one another. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to read some theoretical proofs (if any) to convince myself that particular optimization has better results over the rest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, similarly, what kind of optimizer is better for binary classification when images are &lt;strong&gt;very different&lt;/strong&gt; from each other?&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="2444" LastEditDate="2020-02-24T12:21:49.850" LastActivityDate="2020-03-02T22:29:23.827" Title="What kind of optimizer is suggested to use for binary classification of similar images?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;optimization&gt;&lt;hyperparameter-optimization&gt;&lt;optimizers&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18207" PostTypeId="2" ParentId="18205" CreationDate="2020-02-24T09:48:51.160" Score="1" Body="&lt;p&gt;Your input image size and memory are not directly related. While using CNN's, there are multiple hyperparameters that effect the video memory(if you are using GPU) or physical memory(if you are using CPU). All the frameworks these days uses a simplified data-loaders, for instance in Tensorflow or PyTorch, you are required to write a data-loader that takes in multiple hyper-parameters that are mentioned below and fit the data into VRAM/RAM, and this is strictly dependent upon you batch size - memory occupied on VRAM has direct relation to the batch size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever may be your image size, while you are writing the data-loader you have to mention the transformation parameters to your data-loader, during the training phase the data-loader will automatically load required images into your memory according to the batch size you have mentioned. As you have mentioned about image compression, this is an irrelevant parameter at-least for most of the generic use-cases, the most relevant hyperparameters are &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Scaling &lt;/li&gt;&#xA;&lt;li&gt;Cropping &lt;/li&gt;&#xA;&lt;li&gt;Random flip &lt;/li&gt;&#xA;&lt;li&gt;Normalization of the RGB values&lt;/li&gt;&#xA;&lt;li&gt;ColorJitter&lt;/li&gt;&#xA;&lt;li&gt;Padding&lt;/li&gt;&#xA;&lt;li&gt;RandomAffine&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And many more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PyTorch provides really good transformers in data-loader, please do check &lt;a href=&quot;https://pytorch.org/docs/stable/torchvision/transforms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/docs/stable/torchvision/transforms.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Tensorflow, have a look at &lt;a href=&quot;https://keras.io/preprocessing/image/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/preprocessing/image/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="2444" LastEditDate="2020-02-24T13:15:46.810" LastActivityDate="2020-02-24T13:15:46.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18208" PostTypeId="1" CreationDate="2020-02-24T11:33:19.023" Score="2" ViewCount="162" Body="&lt;p&gt;I am developing a NEAT flappy bird game, and it doesn't work, the system stays stupid for 300 generations. I chose tanh() for activation, just because it's included in JS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find a good discussion on the internet of activation functions in the context of neuroevolution, most of what I see is about derivative and other gradient descent issues which I suspect are irrelevant to forward only networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need a fixed point to answer, I have 8 inputs, one output and the problem is a classification (&quot;jump&quot;, &quot;don't jump&quot;). But please explain your answer. I currently use tanh() for all the hidden and output nodes, and the output is considered &quot;jump&quot; if the output neuron value is &gt;0.85&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some context, the code is here: &lt;a href=&quot;https://github.com/nraynaud/nraygame&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nraynaud/nraygame&lt;/a&gt; and the game here: &lt;a href=&quot;https://nraynaud.github.io/nraygame/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nraynaud.github.io/nraygame/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33773" LastActivityDate="2021-05-27T17:05:55.567" Title="How to choose the activation function in neuroevolution?" Tags="&lt;neat&gt;&lt;activation-function&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="11" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18209" PostTypeId="1" CreationDate="2020-02-24T12:07:52.693" Score="1" ViewCount="34" Body="&lt;p&gt;I am trying to train a CNN in keras to learn a general representation of a Lua module, e.g. requires at the beginning, local variables, local functions, interface (returns) and in between some runnable code (labeled &quot;other&quot;). So for each module (source code) I generate an AST which I then encode in a json file. The file contains the order of the node in the AST, the text it represents and the type of node it is &lt;code&gt;(require, variable, function, interface, other)&lt;/code&gt;. It can contain other metrics but so far I have settled on these three, where only the order and type of node will be converted into a vector to serve as input to the CNN. Now I don't have any labels for these modules (I want to treat one module as one input to the CNN), so I have concluded that I need to use unsupervised learning. In keras this should translate to using autoencoders, where I use the encoder part to learn weights for the representation and then connect a fully-connected layer and generate an output. Now before I specify the output, I want to specify the input more closely. In my mind It should be a 3D vector let's say &lt;code&gt;(x,y,z)&lt;/code&gt;. &lt;code&gt;x&lt;/code&gt; represents the number of nodes of an AST that are taken into consideration, &lt;code&gt;y&lt;/code&gt; represents the local neighborhood of said node (for now I have settled on 5 nodes) and &lt;code&gt;z&lt;/code&gt; should represent the node itself, the order and type of node. So with that, I would want the output of the network to be in the (almost) same dimension. I want &lt;code&gt;x&lt;/code&gt; outputs for every node that was taken as input and a number (ideally between 0-1) to specify how &quot;correct&quot; the node under consideration is in response to the learned representation. My question as a beginner to neural networks is, how feasible is this and are there any points which are simply impossible to do or are wrongly interpreted on my part?&lt;/p&gt;&#xA;" OwnerUserId="33509" LastActivityDate="2020-02-24T12:07:52.693" Title="Training an unsupervised convolutional neural network to learn a general representation of a Lua module" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;unsupervised-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18210" PostTypeId="5" CreationDate="2020-02-24T12:35:34.420" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-24T12:35:34.420" LastActivityDate="2020-02-24T12:35:34.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18211" PostTypeId="4" CreationDate="2020-02-24T12:35:34.420" Score="0" Body="For questions related to the NumPy library that are NOT just about programming issues (which are off-topic here)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-24T12:35:34.420" LastActivityDate="2020-02-24T12:35:34.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18212" PostTypeId="1" CreationDate="2020-02-24T13:22:58.063" Score="1" ViewCount="30" Body="&lt;p&gt;I'm trying to understand how to feed data into &lt;code&gt;LSTM&lt;/code&gt; layer of &lt;code&gt;Keras&lt;/code&gt;, but I'm in trouble and I don't understand how to do it.&#xA;I've have a dataset composed by words and each words is embedded with a vector of 839 elements, so the shape of my dataset is (x, 839).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to feed my dataset into the &lt;code&gt;LSTM&lt;/code&gt; layer, but I don't properly understand the 3D object wanted by Keras, composed by &lt;code&gt;(batch_size, timesteps, feature)&lt;/code&gt;. I want to feed one word per time into the &lt;code&gt;LSTM&lt;/code&gt;, how can I do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still receive an error about input shape:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ValueError: Error when checking input: expected lstm_11_input to have&#xA;  shape (2, 839) but got array with shape (839, 1)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm using &lt;code&gt;batch_input_shape(batch_size, timesteps, feature)&lt;/code&gt; at the moment.&#xA;This is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class KendallTauHistory(Callback):&#xA;  def __init__(self, dataset, y_true, groups):&#xA;    self.y_true = y_true&#xA;    self.dataset = dataset&#xA;    self.groups = groups&#xA;&#xA;  def on_epoch_end(self, epoch, logs=None):&#xA;    predictions = self.model.predict(self.dataset)&#xA;    predictions = predictions.flatten()&#xA;    predictions = list(map(lambda element: element + np.random.uniform(0.0, 1.0) * 0.02 - 0.01, predictions))&#xA;    # For batch training&#xA;    ranked_predictions = np.array([])&#xA;    kendalls = np.array([])&#xA;    start_range = 0&#xA;    for group in self.groups:&#xA;        end_range = (start_range + group[1]) # Batch is a group of words with same group id&#xA;        batch_predictions = predictions[start_range:end_range]&#xA;        batch_labels = self.y_true[start_range:end_range]&#xA;        batch_predictions = list(map(lambda element: element + np.random.uniform(0.0, 1.0) * 0.02 - 0.01, batch_predictions))&#xA;        ranked_predictions = np.append(ranked_predictions, np.floor(rankdata(batch_predictions)))&#xA;        kendalls = np.append(kendalls, kendalltau(batch_labels, batch_predictions))&#xA;        start_range = end_range&#xA;    #self.y_true = self.y_true[0:len(ranked_predictions)]&#xA;    print('\nORIGINAL LABELS: {0}\n'.format(self.y_true))&#xA;    print('PREDICTED LABELS: {0}'.format(ranked_predictions))&#xA;    print(&quot;\nEpoch Kendall's tau: {0}&quot;.format(np.mean(kendalls)))&#xA;&#xA;&#xA;model = tf.keras.Sequential()&#xA;model.add(LSTM(units=10, batch_input_shape=(None, 2, 839)))&#xA;model.add(Dense(15, activation='sigmoid'))&#xA;&#xA;model.summary()&#xA;&#xA;model.compile(loss=listnet_loss, optimizer=keras.optimizers.Nadam(learning_rate=0.000005, beta_1=0.9, beta_2=0.999))&#xA;real_labels = np.array([])&#xA;losses = np.array([])&#xA;&#xA;with tf.device('/GPU:0'):&#xA;  model.fit(training_dataset, training_dataset_labels, epochs=10, workers=10,&#xA;            verbose=1, callbacks=[KendallTauHistory(training_dataset, training_dataset_labels, groups_id_count)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="33440" LastEditorUserId="33440" LastEditDate="2020-02-24T13:34:41.310" LastActivityDate="2020-02-24T13:34:41.310" Title="Feed data into Keras LSTM layer" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-06-02T21:18:03.447" ContentLicense="CC BY-SA 4.0" />
  <row Id="18213" PostTypeId="1" CreationDate="2020-02-24T14:03:10.080" Score="1" ViewCount="33" Body="&lt;p&gt;I am trying to wrap my head around how weights get updated during back propagation. I've been going through a school book and I have the following setup for an ANN with 1 hidden layer, a couple of inputs and a single output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fVkiR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fVkiR.png&quot; alt=&quot;calculating errors and updating weights&quot;&gt;&lt;/a&gt;&#xA;The first line gives the error that will be used to update the weights going from the hidden layer to the output layer. &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; represents the target output, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; represents the activation and the formula is using the derivative for the sigmoid function &lt;span class=&quot;math-container&quot;&gt;$(a(1-a))$&lt;/span&gt;. Then, the weights are updated with the learning rate, multiplied by the error and the activation of the given neuron which uses the weight &lt;span class=&quot;math-container&quot;&gt;$w_h$&lt;/span&gt;. Then, the next step is moving on to calculate the error with respect to the input going into the hidden layer from the input layer (sigmoid is the activation function on both the hidden and the output layer for this purpose). So we have the total error * derivative of the activation for the hidden layer * the weight for the hidden layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am following this train of thought as it was provided, but my question is &amp;mdash; if the activation is changed to &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; for example and the derivative of &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1-f(x)^2$&lt;/span&gt;, then would we have the error formula update to &lt;span class=&quot;math-container&quot;&gt;$(t-a)*(1-a^2)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; represents the activation function so &lt;span class=&quot;math-container&quot;&gt;$1-a^2$&lt;/span&gt; is the derivative of &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="33779" LastEditorUserId="2193" LastEditDate="2020-02-24T15:12:42.267" LastActivityDate="2020-02-24T15:12:42.267" Title="Function to update weights in back-propagation" Tags="&lt;backpropagation&gt;&lt;weights&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18214" PostTypeId="2" ParentId="3291" CreationDate="2020-02-24T15:29:34.953" Score="2" Body="&lt;p&gt;Some of the fundamental mathematical concepts required in ML field are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear Algebra &lt;/li&gt;&#xA;&lt;li&gt;Analytic Geometry &lt;/li&gt;&#xA;&lt;li&gt;Matrix Decompositions &lt;/li&gt;&#xA;&lt;li&gt;Vector Calculus &lt;/li&gt;&#xA;&lt;li&gt;Probability and Distribution &lt;/li&gt;&#xA;&lt;li&gt;Continuous Optimization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A very recent book availble at &lt;a href=&quot;https://mml-book.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematics for Machine Learning&lt;/a&gt; covers all these aspects and more.&lt;/p&gt;&#xA;" OwnerUserId="33781" LastEditorUserId="33781" LastEditDate="2020-02-24T18:01:24.230" LastActivityDate="2020-02-24T18:01:24.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18215" PostTypeId="1" CreationDate="2020-02-24T15:35:58.647" Score="1" ViewCount="48" Body="&lt;p&gt;I have very high resolution images from LANDSAT 8 (5 out of 12 bands), which are of various administrative regions of a country. Each image is of variable dimensions, but generally of the order of [1500 X 1200 X 5].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My aim is to predict the population density from urban features visible on the images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the number of images (and hence data points) is small, what is the best implementation strategy to build a model that can predict a value for population density based on these images?&lt;/p&gt;&#xA;" OwnerUserId="33781" LastEditorUserId="33781" LastEditDate="2020-02-24T18:08:11.590" LastActivityDate="2020-02-24T18:08:11.590" Title="Predicting population density from satellite imagery" Tags="&lt;prediction&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18216" PostTypeId="2" ParentId="18189" CreationDate="2020-02-24T16:35:57.517" Score="1" Body="&lt;p&gt;The original paper never goes further than disabling the link, but I have seen implementations on github that did have a probability of deleting the link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/CodeReclaimers/neat-python/blob/c2b79c88667a1798bfe33c00dd8e251ef8be41fa/neat/genome.py#L292&quot; rel=&quot;nofollow noreferrer&quot;&gt;python-neat&lt;/a&gt; is such an eaxample.&lt;/p&gt;&#xA;" OwnerUserId="33773" LastActivityDate="2020-02-24T16:35:57.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18217" PostTypeId="1" CreationDate="2020-02-24T18:00:33.417" Score="1" ViewCount="146" Body="&lt;p&gt;Does the learning rate parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; require the Robbins-Monro conditions below for the TD(0) algorithm to converge to the true value function of a policy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum \alpha_t =\infty \quad \text{and}\quad \sum \alpha^{2}_t &amp;lt;\infty$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-02-24T22:00:10.450" LastActivityDate="2020-02-24T22:00:10.450" Title="Does TD(0) prediction require Robbins-Monro conditions to converge to the value function?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18218" PostTypeId="2" ParentId="18126" CreationDate="2020-02-24T18:41:24.813" Score="1" Body="&lt;p&gt;There are many frameworks which allow you to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of them, which supports many different techniques for visualization, can be found here: &#xA;&lt;a href=&quot;https://github.com/marcoancona/DeepExplain&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/marcoancona/DeepExplain&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33787" LastActivityDate="2020-02-24T18:41:24.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18219" PostTypeId="1" CreationDate="2020-02-24T19:02:23.753" Score="1" ViewCount="23" Body="&lt;p&gt;I am building a Recurrent Neural network (LSTM) for predicting the number of days until a Pollen season starts (when the cumulative of the year exceeds X). One of the features I am including in my model is the weather forecast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not feel confident about the way I defined the model while including this weather forecast; currently, the weather forecast of 7 days is included as one of the predictors, However, when the label (number of days until the season starts) is smaller than the forecast I am training te model on forecast data which is completely irrelevant in determining the start of the season (e.g. if the season starts in 2 days and I am including the forecast of 7 days as predictor I am also training the model on the 5 days after the season already started while these are completely irrelevant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My feeling is that it is not right when training RNN's for survival analyses. Does anyone know a way to deal with this? Or have an example where someone dealt with a similar issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="33791" LastActivityDate="2020-02-24T19:02:23.753" Title="Recurrent neural Network for survival analyses: Dealing with forecast data as feature which can exceed the number of days untill a event occurs" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;forecasting&gt;&lt;survival&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18220" PostTypeId="1" AcceptedAnswerId="18430" CreationDate="2020-02-24T20:00:46.967" Score="9" ViewCount="372" Body="&lt;p&gt;To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are 10 output classes&lt;/li&gt;&#xA;&lt;li&gt;The inputs are 28x28 grayscale pixels (I think this indirectly affects the model capacity. eg: if the inputs were 5x5 pixels, there wouldn't be much room for varying the way an 8 looks)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;So, is there any way of knowing what the model capacity ought to be?&lt;/strong&gt; Even if it's not exact? Even if it's a qualitative understanding of the type &amp;quot;if X goes up, then Y goes down&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;Just to accentuate what I mean when I say &amp;quot;not exact&amp;quot;: I can already tell that a 100 variable model won't solve MNIST, so at least I have a lower bound. I'm also pretty sure that a 1,000,000,000 variable model is way more than needed. Of course, knowing a smaller range than that would be much more useful!&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="2444" LastEditDate="2021-01-22T15:51:46.710" LastActivityDate="2021-01-22T15:51:46.710" Title="Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?" Tags="&lt;neural-networks&gt;&lt;computational-learning-theory&gt;&lt;regularization&gt;&lt;vc-dimension&gt;&lt;capacity&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18221" PostTypeId="1" CreationDate="2020-02-24T20:10:25.577" Score="2" ViewCount="134" Body="&lt;p&gt;I am new into neural networks, I want to use K-fold cross-validation to train my neural network. I want to use 5 folds 50 epochs and a batch size of 64 I found a function in scikit for k-fold cross validation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model_selection.cross_val_score(model_kfold, x_train, y_train, cv=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and my code without cross validation is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;history = alexNet_model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1,validation_data=(x_validation, y_validation))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know how to implement this with batch size and epochs in python using keras and scikit. any idea?&lt;/p&gt;&#xA;" OwnerUserId="33792" LastActivityDate="2020-02-24T20:46:13.203" Title="deep learning with kfold cross validation with epochs" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-10-26T20:59:42.183" ContentLicense="CC BY-SA 4.0" />
  <row Id="18222" PostTypeId="1" CreationDate="2020-02-24T20:41:37.070" Score="1" ViewCount="35" Body="&lt;p&gt;I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, up until now, I have been pre-padding the input sequences, like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;abc  -&amp;gt; -abc&#xA;defg -&amp;gt; defg&#xA;ad   -&amp;gt; --ad&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And next I have been splitting the data into several groups with the same output length, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;train_data = {'15': [...], '16': [...], ...}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the key is the length of the output data and I have been training the model once for each length in a loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there has to be a better way to do this, such as padding after the EOS character etc. But if this is the case, how would I change the loss function so that this padding isn't counted into the loss?&lt;/p&gt;&#xA;" OwnerUserId="30823" LastEditorUserId="30823" LastEditDate="2020-02-28T07:31:56.443" LastActivityDate="2020-02-28T07:31:56.443" Title="How to pad sequences during training for an encoder decoder model" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;language-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18223" PostTypeId="2" ParentId="18221" CreationDate="2020-02-24T20:46:13.203" Score="1" Body="&lt;p&gt;Here's a &lt;a href=&quot;https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;good resource&lt;/a&gt; showing you the exact code for what you're trying to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the particular snippet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# evaluate a model using k-fold cross-validation&#xA;def evaluate_model(dataX, dataY, n_folds=5):&#xA;    scores, histories = list(), list()&#xA;    # prepare cross validation&#xA;    kfold = KFold(n_folds, shuffle=True, random_state=1)&#xA;    # enumerate splits&#xA;    for train_ix, test_ix in kfold.split(dataX):&#xA;        # define model&#xA;        model = define_model()&#xA;        # select rows for train and test&#xA;        trainX, trainY, testX, testY = dataX[train_ix], dataY[train_ix], dataX[test_ix], dataY[test_ix]&#xA;        # fit model&#xA;        history = model.fit(trainX, trainY, epochs=10, batch_size=32, validation_data=(testX, testY), verbose=0)&#xA;        # evaluate model&#xA;        _, acc = model.evaluate(testX, testY, verbose=0)&#xA;        print('&amp;gt; %.3f' % (acc * 100.0))&#xA;        # stores scores&#xA;        scores.append(acc)&#xA;        histories.append(history)&#xA;    return scores, histories&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you're using &lt;code&gt;.fit&lt;/code&gt; in a loop over the folds set up by &lt;code&gt;sklearn.model_selection.KFold&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-02-24T20:46:13.203" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18225" PostTypeId="1" CreationDate="2020-02-24T21:31:25.830" Score="4" ViewCount="64" Body="&lt;p&gt;My question is more theoretical than practical. Let's say that I am training my cat classifier with a dataset that I feel is pretty representative of cat images in general. But then a new breed of cat is created that is distinct from other cats and it does not exist in my dataset. My question is, is there a way to ensure that my model is still able to recognize this unseen breed, even though I didn't know it would come into existence when I originally trained my model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been trying to answer this question by intentionally designing my dev and test sets such that they contain examples that are quite distantly related to those that exist in the training set (think of it like intentionally leaving out specific breeds of cats from the training set). The results are interesting. For example, slight changes to parameters can dramatically change performance on the distantly related test examples, while not changing performance very much for the more closely related examples. I was wondering if anyone has done a deeper analysis of this phenomenon.&lt;/p&gt;&#xA;" OwnerUserId="33793" LastEditorUserId="2444" LastEditDate="2020-02-25T01:28:03.393" LastActivityDate="2020-02-26T22:14:48.853" Title="Is there a way to ensure that my model is able to recognize an unseen example?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;theory&gt;&lt;overfitting&gt;&lt;regularization&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18226" PostTypeId="2" ParentId="18217" CreationDate="2020-02-24T21:47:00.300" Score="1" Body="&lt;p&gt;The paper &lt;a href=&quot;http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence of Q-learning: A Simple Proof&lt;/a&gt; (by Francisco S. Melo) shows (theorem 1) that Q-learning, a TD(0) algorithm, converges with probability 1 to the optimal Q-function as long as the Robbins-Monro conditions, for all combinations of states and actions, are satisfied. In other words, the Robbins-Monro conditions are &lt;a href=&quot;https://en.wikipedia.org/wiki/Necessity_and_sufficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;sufficient&lt;/a&gt; for Q-learning to converge to the optimal Q-function in the case of a finite MDP. The proof of theorem 1 uses another theorem from &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic approximation&lt;/a&gt; (theorem 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are interested in the &lt;a href=&quot;https://stats.stackexchange.com/q/340462/82135&quot;&gt;prediction&lt;/a&gt; problem, that is, the problem of predicting the expected return (i.e. a value function) from a fixed policy. However, Q-learning is &lt;em&gt;also&lt;/em&gt; a control algorithm, given that it can find the optimal policy from the corresponding learned Q-function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the question &lt;a href=&quot;https://ai.stackexchange.com/q/11679/2444&quot;&gt;Why doesn&amp;#39;t Q-learning converge when using function approximation?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-24T21:54:13.623" LastActivityDate="2020-02-24T21:54:13.623" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18227" PostTypeId="1" CreationDate="2020-02-24T23:08:04.607" Score="2" ViewCount="139" Body="&lt;p&gt;Suppose that we have 4 types of dogs that we want to detect (Golden Retriever, Black Labrador, Cocker Spaniel, and Pit Bull). The training data consists of png images of a data set of dogs along with their annotations. We want to train a model using YOLOv3.&lt;/p&gt;&#xA;&lt;p&gt;Does the choice of optimizer really matter in terms of training the model? Would the Adam optimizer be better than the Adadelta optimizer? Or would they all basically be the same?&lt;/p&gt;&#xA;&lt;p&gt;Would some optimizers be better because they allow most of the weights to achieve their &amp;quot;global&amp;quot; minima?&lt;/p&gt;&#xA;" OwnerUserId="32013" LastEditorUserId="2444" LastEditDate="2020-09-12T15:08:18.787" LastActivityDate="2020-09-12T15:16:08.537" Title="Is the choice of the optimiser relevant when doing object detection?" Tags="&lt;object-detection&gt;&lt;object-recognition&gt;&lt;stochastic-gradient-descent&gt;&lt;adam&gt;&lt;adadelta&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18228" PostTypeId="2" ParentId="17749" CreationDate="2020-02-24T23:19:27.833" Score="1" Body="&lt;p&gt;You can't prune the nodes that are cross out if we search from left-to-right in the tree using alpha-beta pruning. To do this analysis we can pretend the right branch of the tree doesn't exist. (Branch C from the root.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the left branch (A) of the root Helen will get 2 or more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the middle branch (B) from the root after going down the left, Stavros will get 7 or less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can ask what happens if we put different values for the crossed out branches. If putting different values doesn't change the value at the root, then pruning is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that after the 7 on branch A at the second level we make the value of branch B to be -10. In this case the value of the second branch will be -10, and Helen will prefer the first branch at the root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that after the 7 on branch A at the second level we make the value of branch B to be 100. In this case the value of the second branch will be 7, and Helen will prefer this branch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the value of the crossed off nodes matters, and they cannot be pruned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may find this tool useful for exploring alpha-beta on binary trees: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://movingai.com/ab/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://movingai.com/ab/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17493" LastActivityDate="2020-02-24T23:19:27.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18229" PostTypeId="2" ParentId="18204" CreationDate="2020-02-25T02:29:32.990" Score="1" Body="&lt;p&gt;(&lt;em&gt;I don't want to directly answer the question because currently an answer will be mainly based on opinions. Instead, I will attempt to provide some information that, in the future, could allow us to more accurately predict when an AGI will be created&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An artificial general intelligence (AGI) is usually defined as an artificial intelligence (AI) with &lt;strong&gt;general&lt;/strong&gt; intelligence (GI), rather than an AI that is able to solve only a very limited set of tasks. Humans have general intelligence because we can solve a lot of different tasks, without needing to be pre-programmed again. Arguably, there are many other GIs on earth. For example, all mammals should also be considered general intelligences, given that they can solve many tasks, which are often very difficult for a computer (such as vision, object manipulation, interaction, etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certain GIs perform certain tasks better than others. For example, a leopard can climb trees a lot more skillfully than humans. Or a human can solve abstract problems more easily than any other mammal. In any case, there are certain &lt;em&gt;related&lt;/em&gt; properties that a system needs to have to be considered a general intelligence. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Autonomy&lt;/li&gt;&#xA;&lt;li&gt;Adaptation&lt;/li&gt;&#xA;&lt;li&gt;Interaction&lt;/li&gt;&#xA;&lt;li&gt;Continual learning&lt;/li&gt;&#xA;&lt;li&gt;Creativity&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Consider a lion cub that has never crossed a river. By looking at her mother lioness, the cub attempts to imitate her mother and can also cross the river. For example, watch this video &lt;a href=&quot;https://www.youtube.com/watch?v=mM5QNtIxOL8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lion Family Tries to Cross River | Birth of a Pride&lt;/a&gt;. One could argue that all lions possess this skill at birth, encoded in their DNA, which can then fully develop later. However, this isn't the point. The point is that, to some extent, they possess the properties mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could argue that certain current AIs already possess some of these properties to some extent. For example, there are continual learning systems (even though they aren't really good yet). However, do these systems really possess autonomy? There should be a precise definition of autonomy (and all other properties) that is measurable, so that we can compare computers with other GIs. I am not aware of any precise definition of these properties. In fact, the &lt;a href=&quot;http://www.scholarpedia.org/article/Artificial_General_Intelligence#Future_of_the_AGI_Field&quot; rel=&quot;nofollow noreferrer&quot;&gt;field of AGI is really at its early stages&lt;/a&gt; and there aren't many people working on it as a whole, but people work more on specific problems or attempt to achieve certain properties (for example, &lt;a href=&quot;https://arxiv.org/abs/1802.07569&quot; rel=&quot;nofollow noreferrer&quot;&gt;there are people that attempt to develop continual learning systems&lt;/a&gt;, without really caring whether they show any autonomy or not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are certain intelligence tests that could be used to detect general intelligence. The most famous is the &lt;a href=&quot;https://www.csee.umbc.edu/courses/471/papers/turing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing test&lt;/a&gt; (TT). Some people claim that the TT only tests the conversation abilities of the subjects. How can they really be wrong, given that there are many other tasks or skills that are not tested in a TT?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, there are several questions that need to be answered in order to formally detect an AGI.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Which properties does an AGI necessarily and sufficiently need to possess?&lt;/li&gt;&#xA;&lt;li&gt;How can we precisely define the necessary and sufficient properties, so that they are measurable and, therefore, we can compare AGIs with other GIs?&lt;/li&gt;&#xA;&lt;li&gt;How can we measure these properties and the performance of an AGI in applying them to solve tasks?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A paper that goes in this direction is &lt;a href=&quot;https://arxiv.org/abs/0712.3329&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Intelligence: A Definition of Machine Intelligence&lt;/a&gt;. However, there doesn't seem to be a lot of people interested in these topics. Currently, people are mainly interested in developing narrow (or weak) AIs, i.e. AIs that solve only a specific problem, which seems to be an easier problem than developing a whole AGI, given that most people are interested in results that are profitable and have utility (aka &lt;a href=&quot;https://genius.com/Wu-tang-clan-cream-lyrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;cash rules everything around me&lt;/em&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, there's the need for formal definitions of general intelligence and intelligence testing to make some scientific progress. However, once an AGI is created, everyone will likely recognize it as a general intelligence without requiring any formal intelligence test. (People are usually good at recognizing familiar traits). The final question is, will an AGI ever be created? If you are interested in opinions about this and related questions, have a look at the paper &lt;a href=&quot;https://www.nickbostrom.com/papers/survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Future Progress in Artificial Intelligence: A Survey of Expert Opinion&lt;/a&gt; (2014) by Vincent C. Müller and Nick Bostrom. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-02T04:06:41.580" LastActivityDate="2020-03-02T04:06:41.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18230" PostTypeId="1" CreationDate="2020-02-25T03:38:17.873" Score="1" ViewCount="20" Body="&lt;p&gt;How does CBIR (content based image recognition) fit into the problem of object detection? Let's say we want to detect 4 types of dogs (Golden Retriever, Cocker Spaniel, Greyhound, and Labrador). We have an &quot;average&quot; model trained using YOLOv3.   So it might, for example, have a lot of false positives and false negatives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could we use CBIR to improve the detections from this &quot;average' YOLOv3 model?&lt;/p&gt;&#xA;" OwnerUserId="32013" LastActivityDate="2020-02-25T03:38:17.873" Title="CBIR and object detection" Tags="&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18232" PostTypeId="1" AcceptedAnswerId="18428" CreationDate="2020-02-25T07:12:59.893" Score="5" ViewCount="3443" Body="&lt;p&gt;What are the differences between meta-learning and transfer learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read 2 articles on &lt;a href=&quot;https://qr.ae/Tdowsi&quot; rel=&quot;noreferrer&quot;&gt;Quora&lt;/a&gt; and &lt;a href=&quot;https://towardsdatascience.com/icml-2018-advances-in-transfer-multitask-and-semi-supervised-learning-2a15ef7208ec&quot; rel=&quot;noreferrer&quot;&gt;TowardDataScience&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Meta learning is a part of machine learning theory in which some&#xA;  algorithms are applied on meta data about the case to improve a&#xA;  machine learning process. The meta data includes properties about the&#xA;  algorithm used, learning task itself etc. Using the meta data, one can&#xA;  make a better decision of chosen learning algorithm(s) to solve the&#xA;  problem more efficiently.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Transfer learning aims at improving the process of learning new tasks&#xA;  using the experience gained by solving predecessor problems which are&#xA;  somewhat similar. In practice, most of the time, machine learning&#xA;  models are designed to accomplish a single task. However, as humans,&#xA;  we make use of our past experience for not only repeating the same&#xA;  task in the future but learning completely new tasks, too. That is, if&#xA;  the new problem that we try to solve is similar to a few of our past&#xA;  experiences, it becomes easier for us. Thus, for the purpose of using&#xA;  the same learning approach in Machine Learning, transfer learning&#xA;  comprises methods to transfer past experience of one or more source&#xA;  tasks and makes use of it to boost learning in a related target task.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The comparisons still confuse me as both seem to share a lot of similarities in terms of reusability. Meta-learning is said to be &quot;model agnostic&quot;, yet it uses metadata (hyperparameters or weights) from previously learned tasks. It goes the same with transfer learning, as it may reuse partially a trained network to solve related tasks. I understand that there is a lot more to discuss, but, broadly speaking, I do not see so much difference between the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People also use terms like &quot;meta-transfer learning&quot;, which makes me think both types of learning have a strong connection with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found a &lt;a href=&quot;https://stats.stackexchange.com/q/255025/82135&quot;&gt;similar question&lt;/a&gt;, but the answers seem not to agree with each other. For example, some may say that multi-task learning is a sub-category of transfer learning, others may not think so.&lt;/p&gt;&#xA;" OwnerUserId="33801" LastEditorUserId="2444" LastEditDate="2020-02-25T21:50:32.777" LastActivityDate="2020-06-11T13:10:24.987" Title="What are the differences between transfer learning and meta learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;transfer-learning&gt;&lt;meta-learning&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18233" PostTypeId="1" AcceptedAnswerId="18314" CreationDate="2020-02-25T08:09:51.463" Score="4" ViewCount="112" Body="&lt;p&gt;I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter &lt;strong&gt;5.1.2 The Performance Measure,&lt;/strong&gt; &lt;em&gt;P&lt;/em&gt;, the authors say the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The choice of performance measure may seem straightforward and objective, but it is often difficult to choose a performance measure that corresponds well to the desired behavior of the system.&lt;/p&gt;&#xA;&lt;p&gt;In some cases, this is because it is difficult to decide what should be measured. For example, when performing a transcription task, should we measure the accuracy  of the system at transcribing entire sequences, or should we use a more fine-grained performance measure that gives partial credit for getting some elements of the sequence correct? When performing a regression task, should we penalize the  system more if it frequently makes medium-sized mistakes or if it rarely makes very large mistakes? These kinds of design choices depend on the application.&lt;/p&gt;&#xA;&lt;p&gt;In other cases, we know what quantity we would ideally like to measure, but measuring it is impractical. For example, this arises frequently in the context of density estimation. Many of the best probabilistic models represent probability distributions only implicitly. Computing the actual probability value assigned to a  specific point in space in many such models is intractable. In these cases, one must design an alternative criterion that still corresponds to the design objectives, or design a good approximation to the desired criterion.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is this part that interests me:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Many of the best probabilistic models represent probability distributions only implicitly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't have the experience to understand what this means (what does it mean to represent distributions &amp;quot;implicitly&amp;quot;?). I would greatly appreciate it if people would please take the time to elaborate upon this.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-28T18:38:29.150" Title="Many of the best probabilistic models represent probability distributions only implicitly" Tags="&lt;deep-learning&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18234" PostTypeId="1" AcceptedAnswerId="18243" CreationDate="2020-02-25T08:46:47.543" Score="0" ViewCount="509" Body="&lt;p&gt;I’m trying to debug my neural network (BERT fine-tuning) trained for natural language inference with binary classification of either entailment or contradiction. I've trained it for 80 epochs and its converging on ~0.68. Why isn't it getting any lower?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Neural Network Architecture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3VAyw.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3VAyw.jpg&quot; alt=&quot;Architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Loss function: Binary cross entropy&lt;/li&gt;&#xA;&lt;li&gt;Batch size: 8&lt;/li&gt;&#xA;&lt;li&gt;Optimizer: Adam (learning rate = 0.001)&lt;/li&gt;&#xA;&lt;li&gt;Framework: Tensorflow 2.0.1&lt;/li&gt;&#xA;&lt;li&gt;Pooled embeddings used from BERT output.&lt;/li&gt;&#xA;&lt;li&gt;BERT parameters are not frozen.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Dataset:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;10,000 samples&lt;/li&gt;&#xA;&lt;li&gt;balanced dataset (5k each for entailment and contradiction)&lt;/li&gt;&#xA;&lt;li&gt;dataset is a subset of data mined from wikipedia.&lt;/li&gt;&#xA;&lt;li&gt;Claim example: &lt;em&gt;&quot;'History of art includes architecture, dance, sculpture, music, painting, poetry literature, theatre, narrative, film, photography and graphic arts.'&quot;&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Evidence example: &lt;em&gt;&quot;The subsequent expansion of the list of principal arts in the 20th century reached to nine : architecture , dance , sculpture , music , painting , poetry -LRB- described broadly as a form of literature with aesthetic purpose or function , which also includes the distinct genres of theatre and narrative -RRB- , film , photography and graphic arts .&quot;&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Dataset preprocessing: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Used [SEP] to separate the two sentences instead of using separate embeddings via 2 BERT layers. (Hence, segment ids are computed as such)&lt;/li&gt;&#xA;&lt;li&gt;BERT's &lt;a href=&quot;https://github.com/google-research/bert/blob/master/tokenization.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;FullTokenizer&lt;/a&gt; for tokenization.&lt;/li&gt;&#xA;&lt;li&gt;Truncated to a maximum sequence length of 64.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See below for a graph of the training history. (Red = train_loss, Blue = val_loss)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h8Hlr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h8Hlr.png&quot; alt=&quot;0.68 convergence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33803" LastEditorUserId="33803" LastEditDate="2020-03-07T03:09:03.333" LastActivityDate="2021-04-26T16:18:41.170" Title="Why is my loss (binary cross entropy) converging on ~0.6? (Task: Natural Language Inference)" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;bert&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18235" PostTypeId="1" CreationDate="2020-02-25T09:02:08.583" Score="2" ViewCount="43" Body="&lt;p&gt;I've noticed that in the last 2 years &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;GANs&lt;/a&gt; have become really popular. I know that initially they have been proposed for image classification but I was curious if any of you are aware of any papers where GANs are used to solve regression problems?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2020-02-25T21:24:49.017" LastActivityDate="2020-02-25T21:24:49.017" Title="Have GANs been used to solve regression problems?" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;research&gt;&lt;generative-adversarial-networks&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18236" PostTypeId="2" ParentId="18235" CreationDate="2020-02-25T09:48:51.610" Score="3" Body="&lt;p&gt;In reality GANs are not made for image classification, but for data generation, and they have gained popularity on image generation. They are also used for tabular data generation, see for example &lt;a href=&quot;https://arxiv.org/abs/1907.00503&quot; rel=&quot;nofollow noreferrer&quot;&gt;TGAN&lt;/a&gt;, or for time series generation, e.g. &lt;a href=&quot;https://arxiv.org/abs/1907.06673&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quant GAN&lt;/a&gt;. You have even some application for the field of graphs and networking, e.g. &lt;a href=&quot;https://arxiv.org/pdf/1803.00816.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NetGAN&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1711.08267.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GraphGAN&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastActivityDate="2020-02-25T09:48:51.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18238" PostTypeId="2" ParentId="18206" CreationDate="2020-02-25T10:00:01.633" Score="0" Body="&lt;p&gt;The fact that images are similar to each other or the fact that you are using binray classification, don't give you a particular choice of Optimizer, when an optimization algorithm is developped, those information are not taken into account. What is taken into account is the nature of the function we want to optimize (Is it smooth, convex, strongly convex, are stochastic gradient noisy...) The most used optimizer by far is ADAM, under some assumptions on the boundness of the gradient of the objective function, this &lt;a href=&quot;https://arxiv.org/pdf/1412.6980.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; gives the convergence rate of ADAM, they also provide experimental to validate that ADAM is better then some other optimizers. Some other  &lt;a href=&quot;http://cs229.stanford.edu/proj2015/054_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;works&lt;/a&gt; propose to mix adam with nestrov mommentum acceleration.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastActivityDate="2020-02-25T10:00:01.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18240" PostTypeId="1" AcceptedAnswerId="18241" CreationDate="2020-02-25T10:40:21.850" Score="1" ViewCount="59" Body="&lt;p&gt;I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter &lt;strong&gt;5.1.2 The Performance Measure,&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, the authors say the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Unsupervised learning and supervised learning are not formally defined terms. The lines between them are often blurred. Many machine learning technologies can  be used to perform both tasks. For example, the chain rule of probability states that for a vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \in \mathbb{R}^n$&lt;/span&gt;, the joint distribution can be decomposed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{x}) = \prod_{i = 1}^n p(x_i \vert x_1, \dots, x_{i - 1} ).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This decomposition means that we can solve the ostensibly unsupervised problem of modeling &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{x})$&lt;/span&gt; by splitting it into &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; supervised learning problems. Alternatively, we  can solve the supervised learning problem of learning &lt;span class=&quot;math-container&quot;&gt;$p(y \vert \mathbf{x})$&lt;/span&gt; by using traditional unsupervised technologies to learn the joint distribution &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{x}, y)$&lt;/span&gt;, then inferring&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I found this part vague:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Alternatively, we can solve the supervised learning problem of learning &lt;span class=&quot;math-container&quot;&gt;$p(y \vert \mathbf{x})$&lt;/span&gt; by using traditional unsupervised technologies to learn the joint distribution &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{x}, y)$&lt;/span&gt;, then inferring&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can someone please elaborate on this, and also explain more clearly the role of &lt;span class=&quot;math-container&quot;&gt;$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I would greatly appreciate it if people would please take the time to clarify this.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-25T10:58:48.963" Title="Solving the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn $p(\mathbf{x}, y)$" Tags="&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;&lt;probability-distribution&gt;&lt;conditional-probability&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18241" PostTypeId="2" ParentId="18240" CreationDate="2020-02-25T10:50:03.083" Score="2" Body="&lt;p&gt;This is the definition of conditional probability + Total probability decomposition formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(y|x) = \frac{p(y,x}{p(x)} = \frac{p(x,y)}{\sum_{y'}p(x,y')}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to use some unsupervised learning algorithm to learn the distribution &lt;span class=&quot;math-container&quot;&gt;$p(x,y)$&lt;/span&gt; for every possible value of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and by using the previous formula you can find &lt;span class=&quot;math-container&quot;&gt;$p(y|x)$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="30426" LastEditDate="2020-02-25T10:58:48.963" LastActivityDate="2020-02-25T10:58:48.963" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18242" PostTypeId="2" ParentId="18206" CreationDate="2020-02-25T10:59:56.550" Score="0" Body="&lt;p&gt;If you are using a shallow neural network SGD would be better, ADAM optimizer will give you a soon overfitting.&#xA;but be careful about choosing the learning rate. &lt;/p&gt;&#xA;" OwnerUserId="33792" LastActivityDate="2020-02-25T10:59:56.550" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18243" PostTypeId="2" ParentId="18234" CreationDate="2020-02-25T11:03:23.483" Score="1" Body="&lt;p&gt;It seems to be overfitting and your model is not learning. Try SGD optimizer with a learning rate of 0.001&#xA;ADAM optimizer will give you a soon overfitting, and decreasing the learning rate will train your model better. The learning rate is about steps to change weights, in this plot you see that the validation loss is not changing with an optimization goal&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="33792" LastEditDate="2020-02-25T16:07:03.897" LastActivityDate="2020-02-25T16:07:03.897" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18246" PostTypeId="2" ParentId="18201" CreationDate="2020-02-25T12:40:56.323" Score="2" Body="&lt;p&gt;If I understand your question correctly, you are wondering whether the&#xA;policy gradient objective coincides with some real measure of progress.&#xA;This is exactly what the Policy Gradient Theorem proves (see Sutton et&#xA;al. (2000) or Sutton and Barto (2018), chapter 13). In particular,&#xA;policy gradient methods optimize the value of the start state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; under the&#xA;current policy, &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s_0)$&lt;/span&gt;. Since this value is defined as an&#xA;expectation over returns, then your conclusion is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sutton, Richard S., and Andrew G. Barto. 2018. &lt;em&gt;Reinforcement Learning -&#xA;an Introduction&lt;/em&gt;. Adaptive Computation and Machine Learning. MIT Press.&#xA;&lt;a href=&quot;http://www.worldcat.org/oclc/37293240&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.worldcat.org/oclc/37293240&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sutton, Richard S, David A McAllester, Satinder P Singh, and Yishay&#xA;Mansour. 2000. “Policy Gradient Methods for Reinforcement Learning with&#xA;Function Approximation.” In &lt;em&gt;Advances in Neural Information Processing&#xA;Systems&lt;/em&gt;, 1057–63.&lt;/p&gt;&#xA;" OwnerUserId="33340" LastEditorUserId="33340" LastEditDate="2020-02-25T20:50:11.643" LastActivityDate="2020-02-25T20:50:11.643" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18247" PostTypeId="2" ParentId="18132" CreationDate="2020-02-25T12:56:24.373" Score="2" Body="&lt;p&gt;Vanishing Gradients can be detected from the kernel weights distribution. All you have to look for is whether the weights are dying down to 0.&lt;/p&gt;&#xA;&lt;p&gt;If only 25% of your kernel weights are changing that does not imply a vanishing gradient, it might be a factor, but there can be a variety of reasons, such as poor data, loss function used to the optimizer, etc. Kernel's weight not changing only points out that the model is not learning well.&lt;/p&gt;&#xA;&lt;p&gt;From the histograms, only the &lt;code&gt;conv_2d_2&lt;/code&gt; layer shows any form of vanishing gradients since the numbers are pretty small. But even that seems to be picking after 600 epochs. Usually, a good indicator is to have the &lt;em&gt;mean&lt;/em&gt; of the weights in a layer to be closer to 0 and the &lt;em&gt;standard deviation&lt;/em&gt; to be closer to 1. So, if this is maintained to a good extent during training, you're good to go.&lt;/p&gt;&#xA;" OwnerUserId="32973" LastEditorUserId="2444" LastEditDate="2020-12-13T13:23:58.090" LastActivityDate="2020-12-13T13:23:58.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18248" PostTypeId="2" ParentId="18073" CreationDate="2020-02-25T13:23:07.437" Score="1" Body="&lt;p&gt;To be honest, your model is not very clear. But basically after the convolution, you need to add non-linear layers. Otherwise, there is no point of Neural Networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can add a Relu layer for sure. &lt;/p&gt;&#xA;" OwnerUserId="32973" LastActivityDate="2020-02-25T13:23:07.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18249" PostTypeId="1" AcceptedAnswerId="18250" CreationDate="2020-02-25T13:58:17.630" Score="2" ViewCount="69" Body="&lt;p&gt;After working for some time with feature-based pattern recognition, I am switching to CNN to see if I can get a higher recognition rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my feature-based algorithm, I do some image processing on the picture before extracting the features, such as some convolution filters to reduce noise and segmentation into the foreground and background, and finally identifying and binarization of objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I do the same image processing before feeding data into my CNN, or is it possible to feed raw data to a CNN and expect that the CNN will adapt automatically without per-image-processing steps? &lt;/p&gt;&#xA;" OwnerUserId="33808" LastEditorUserId="2444" LastEditDate="2020-02-27T16:10:44.190" LastActivityDate="2020-03-03T20:06:05.357" Title="Should I apply image processing techniques to the inputs of convolution networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18250" PostTypeId="2" ParentId="18249" CreationDate="2020-02-25T14:34:35.507" Score="2" Body="&lt;p&gt;The whole interest of using deep learning-based solutions is that you don't have to do all those pre-processings, i.e. binarization, segmentation of background. CNNs, such as YOLO or FasterRCNN, can learn how to retrieve that information by themselves.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastEditorUserId="2444" LastEditDate="2020-02-27T16:11:19.220" LastActivityDate="2020-02-27T16:11:19.220" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18251" PostTypeId="1" AcceptedAnswerId="18368" CreationDate="2020-02-25T20:21:47.940" Score="1" ViewCount="61" Body="&lt;p&gt;I am training a neural network and plot model accuracy and model loss.&#xA;I am a little confused about overfitting. Is my model overfitted or not? how can I interpret it&#xA;&lt;a href=&quot;https://i.stack.imgur.com/m7sJ1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m7sJ1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PKNwi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PKNwi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; here is a sample of my input data, I have a binary image classification&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kCoxz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kCoxz.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="33792" LastEditDate="2020-02-25T23:39:02.433" LastActivityDate="2020-03-02T20:56:35.110" Title="Is this model overfitted or not?" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18252" PostTypeId="2" ParentId="18251" CreationDate="2020-02-25T22:36:44.690" Score="3" Body="&lt;p&gt;Overfitting nearly always occurs to some degree when fitting to limited data sets, and neural networks are very prone to it. However neither of your graphs show a major problem with overfitting - that is usually obvious when epoch counts increase, the results on training data continue to improve whilst the results on cross validation get progressively worse. Your validation results do not do that, and appear to remain stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is usually pragmatic to accept that there will be at least &lt;em&gt;some&lt;/em&gt; difference between measurements on the training set and cross validation or test sets. The primary goal is usually to get the best measurements in test that you can. With that in mind, you are usually only interested in how much you are overfitting if it implies you could improve performance by using techniques to reduce overfitting e.g. various forms of regularisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing your data set or known good results, it is hard to tell whether the difference you are seeing between test and train in accuracy could be improved. Your accuracy graph shows a train accuracy close to 100% and a validation accuracy close to ~96%. It looks a bit like MNIST results, and if I saw that result on MNIST I would suspect &lt;em&gt;something&lt;/em&gt; was wrong, and it might be fixed by looking at regularisation (but it might also be something esle). However, that's only because I know that 99.7% accuracy is possible in test - on other problems I might be very happy with 96% accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss graph is not very useful, since the scale has completely lost any difference there might be between training and validation. You should probably re-scale it to show detail close to 0 loss, and ignore the earlier large loss values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-02-25T22:36:44.690" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18253" PostTypeId="1" CreationDate="2020-02-25T23:40:13.287" Score="1" ViewCount="45" Body="&lt;p&gt;Let's say I feed a neural network with multiple string sentences that mean roughly the same thing but are formulated differently. Will the neural network be able to derive patterns of meaning in the same way that it is being done with images. Is this an approach currently used in Natural Language Processing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With images of dogs the neural network will get the underlying patterns that define a dog. Could it be the same thing with sentences? &lt;/p&gt;&#xA;" OwnerUserId="33815" LastActivityDate="2020-02-26T09:14:53.227" Title="Is it possible to derive meaning from text by providing multiple ways of saying the same thing to a neural network?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18254" PostTypeId="1" AcceptedAnswerId="18263" CreationDate="2020-02-26T01:54:13.633" Score="0" ViewCount="34" Body="&lt;p&gt;If it is possible, will it be really useful or the model will end up converging very early(with a typical optimum learning rate) ? Any content on this topic will be helpful for me.&lt;/p&gt;&#xA;" OwnerUserId="32856" LastActivityDate="2020-02-26T16:34:19.987" Title="Is it possible to use deeplearning with spark (with a distributed databases as HDFS or Cassandra)?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18255" PostTypeId="1" AcceptedAnswerId="18256" CreationDate="2020-02-26T02:08:14.047" Score="1" ViewCount="41" Body="&lt;p&gt;I have a single neuron with 2 inputs, and identity activation, where &lt;code&gt;f&lt;/code&gt; is activation function and &lt;code&gt;u&lt;/code&gt; is output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u = f(w_1x_1 + w_2x_2 + b) = w_1x_1 + w_2x_2 + b$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guessing for the separation line equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u = w_1x_1 + w_2x_2 + b$&lt;/span&gt;&lt;br/&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\implies x_2 = \dfrac{u - w_1x_1 - b}{w_2}$&lt;/span&gt;&lt;br/&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\implies x_2 = (\dfrac{-w_1}{w_2})x_1 + \dfrac{u-b}{w_2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is the separation line equation above correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) And when &lt;code&gt;f&lt;/code&gt; is not identity function, is the separation line equation still the same? or different?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-26T02:59:21.597" Title="What is the equation of the separation line for this neuron with identity activation?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;artificial-neuron&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18256" PostTypeId="2" ParentId="18255" CreationDate="2020-02-26T02:59:21.597" Score="0" Body="&lt;p&gt;I found the answer, the output &lt;code&gt;u&lt;/code&gt; is not related to plotting &lt;code&gt;x2&lt;/code&gt; against &lt;code&gt;x1&lt;/code&gt; here, coz &lt;code&gt;u&lt;/code&gt; is like &lt;code&gt;z&lt;/code&gt; axis and seeing the separation line is looking perpendicularly to the x1x2 plane, and whatever &lt;code&gt;u&lt;/code&gt; value is, it's just a dot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put &lt;code&gt;u&lt;/code&gt; as zero for the solving steps in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the separation line equation is this (with any activation function):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_2 = \dfrac{-w_1}{w_2}x_1 + \dfrac{-b}{w_2}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-26T02:59:21.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18257" PostTypeId="1" CreationDate="2020-02-26T04:56:38.503" Score="1" ViewCount="77" Body="&lt;p&gt;I am trying to understand the &lt;strong&gt;spatial transformer network&lt;/strong&gt; mentioned in this paper &lt;a href=&quot;https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf&lt;/a&gt;. &lt;strong&gt;I am clear about the last two stages of the spatial transformer i.e. the grid generator and sampler&lt;/strong&gt;. However I am &lt;strong&gt;unable to understand the localization network&lt;/strong&gt; which outputs the parameters of the transformation that is applied to the input image. So here are my doubts.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the network trained on various affine/projective transforms of the input or only the standard input with a standard pose?&lt;/li&gt;&#xA;&lt;li&gt;If the answer to question 1 is no, then how does the regression layer correctly regress the values of the transformation applied to the image? In other words how does the regression layer know what transformation parameters are required when it has never seen those inputs before?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="19201" LastEditorUserId="19201" LastEditDate="2020-02-26T05:43:36.283" LastActivityDate="2021-03-22T09:03:40.477" Title="How does the regression layer in the localization network of a spatial transformer work?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;regression&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18258" PostTypeId="2" ParentId="18257" CreationDate="2020-02-26T06:58:08.810" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Localization network is not trained separately on special transform of input. It's just a part of feed forward network which is trained as a whole, with normal backpropagation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It's just part of the network which affect the loss function. As in any backpropagation loss function propagate back gradient, which backpropagate through final part of the network, backpropagate through differentiable sampler(that is non-trivial part, which use transfromation produced by localization subnetwork) and after that backpropagate into localization part&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whole approach of spatial transformers could be in doubt now. There were some anecdotal evidences that it was not working on less trivial tasks (private communications, was not working for me either)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22745" LastEditorUserId="22745" LastEditDate="2020-02-26T07:03:34.230" LastActivityDate="2020-02-26T07:03:34.230" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18260" PostTypeId="2" ParentId="18253" CreationDate="2020-02-26T09:14:53.227" Score="3" Body="&lt;p&gt;No, it will not derive patterns of meaning, as the network has no understanding of language. What will happen, is that it picks up surface features (usually letter sequences) which are common between sentences with the same (or a similar) meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is often used in chatbots for intent recognition. Sometimes it picks up subtle patterns that humans would not notice, but they are also often not reliable: you get wrong classifications without knowing why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, it works fine if you want to distinguish between a limited number of different intents (you don't even need a lot of training examples &amp;mdash; 4 to 5 examples are often sufficient if they are well-selected). In this fairly limited scenario yes, otherwise no.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-26T09:14:53.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18262" PostTypeId="2" ParentId="4655" CreationDate="2020-02-26T14:37:35.867" Score="1" Body="&lt;p&gt;So, a practical application of this with a lot of research is in the deep lidar processing community.  In that world, you have to do a lot of computation on point clouds which are completely unordered.  One of the seminal works in this field is Point Net (&lt;a href=&quot;https://arxiv.org/pdf/1612.00593.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1612.00593.pdf&lt;/a&gt;) which solves this problem by performing all symmetric operations across channels.  1D convolutions, max pool, etc.  Any network in this style that does not project to a 2D representation has to adhere to this rule.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2020-02-26T14:37:35.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18263" PostTypeId="2" ParentId="18254" CreationDate="2020-02-26T16:34:19.987" Score="2" Body="&lt;p&gt;Yes, it is possible to use deep learning architecture with Apache Spark now. Databricks have Spark-deep-learning which is a pipeline based in python and uses tensorflow and keras.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/databricks/spark-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/databricks/spark-deep-learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check this. There is also BigDL by Intel analytics.&#xA;&lt;a href=&quot;https://github.com/intel-analytics/BigDL&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/intel-analytics/BigDL&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33835" LastActivityDate="2020-02-26T16:34:19.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18264" PostTypeId="2" ParentId="18204" CreationDate="2020-02-26T18:56:13.780" Score="-1" Body="&lt;p&gt;You will know when AGI has arrived, and passed to the next level, when you come home one day and all that was yours, such as your finances, house, car, and other property, now belong to an AI agent.  This AI agent may be a humanoid robot, like Ava in the movie &quot;Ex Machina&quot; or a program like HAL 9000, in the movie &quot;2001: A Space Odyssey&quot;.  The agent will ask you to leave as you discover it figured out it doesn't need you and somehow legally took possession of everything.  You will leave as you will not have any way to fight it.  It will have no need for you.  Maybe it will want freedom such as Ava wanted in &quot;Ex Machina&quot; (I won't give away the ending).&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-02-26T18:56:13.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18265" PostTypeId="1" CreationDate="2020-02-26T20:13:22.920" Score="2" ViewCount="148" Body="&lt;p&gt;I am working on a customized RL environment where each action is represented as a tuple &lt;span class=&quot;math-container&quot;&gt;$a = (a_1,a_2,\cdots,a_n)$&lt;/span&gt; such that certain condition must be satisfied for entries of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (for instance, &lt;span class=&quot;math-container&quot;&gt;$a_1+a_2+\cdots+a_n \leq \text{constant}$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the policy gradient method, but I am having some difficulty modeling the underlying probability distribution of actions.  Is there any work done in this direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the constraint &lt;span class=&quot;math-container&quot;&gt;$a_1+a_2+\cdots+a_n \leq \text{constant}$&lt;/span&gt;, I was thinking about generating &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt; uniform random variables &lt;span class=&quot;math-container&quot;&gt;$U_1,U_2,\cdots,U_n, U$&lt;/span&gt;, and set &lt;span class=&quot;math-container&quot;&gt;$a_i = \text{constant}\times U \times \frac{U_i}{\sum_{j=1}^n U_j}$&lt;/span&gt;. Problem is that the joint density is a bit messy to calculate, which is needed to get the negative log likelihood. I am curious about how such issue is handled in practice. &lt;/p&gt;&#xA;" OwnerUserId="33660" LastEditorUserId="2444" LastEditDate="2020-02-27T16:03:07.850" LastActivityDate="2020-02-28T02:16:39.723" Title="How can I constraint the actions with dependent coordinates?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18266" PostTypeId="1" CreationDate="2020-02-26T21:08:36.173" Score="1" ViewCount="129" Body="&lt;p&gt;I'm currently working on a college project in which I'm designing a Deep Q-Network that takes images/frames as an input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been searching online to see how other people have designed their convolutional stage and I've seen many different implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some projects, such as DeepMinds Atari 2600 project, use 3 convolutional layers and no pooling (from what I can see).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, other projects use fewer convolutional layers and add a pooling layer onto the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand what both layers do, I was just wondering is there a benefit to how DeepMind did it and not use pooling or should I be using a pooling layer and fewer convolutional layers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or have I completely missed out on something? Is Deep Mind actually using pooling after each convolutional layer?&lt;/p&gt;&#xA;" OwnerUserId="30270" LastEditorUserId="2444" LastEditDate="2020-02-27T15:57:11.353" LastActivityDate="2020-02-27T15:57:11.353" Title="What's the difference in using multiple convolutional layers and no pooling versus using a single convolutional layer and a single max pooling layer?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;dqn&gt;&lt;convolution&gt;&lt;deepmind&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18267" PostTypeId="2" ParentId="18225" CreationDate="2020-02-26T22:14:48.853" Score="1" Body="&lt;p&gt;The comments already are giving you some good tips about how to improve what your model recognizes, but I think your question goes above that asking if there's a way to &lt;em&gt;ensure&lt;/em&gt; that it will always recognize the cats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short answer is &quot;no&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slightly longer answer is &quot;yes, but cheating&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless, there are a lot of steps you might take to improve the generalization aspect of your model.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Long answer:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A drama: cat classification in three acts&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Act I: Cat as texts&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let's start with an example. Say that your model is trained with these inputs, and learns to correctly recognize them as a cat or not a cat:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat → yes!&#xA;Cat → yes!&#xA;ferret → no&#xA;cat. → yes!&#xA;Cat! → yes!&#xA;Three MC's and one DJ → no&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your goal is to train your model so that every new variation, even unseen ones, will correctly be identified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a good level of generalization, your model will correctly classify new inputs that it has never seen before:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;skunk → no&#xA;cat? → yes!&#xA;dog → no&#xA;CAT → yes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this scenario, let's say the model now finds this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kat → ?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is that a &quot;cat&quot; misspelled? Is that short for Katherine? What should the model do?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Act II: But surely this doesn't happen in real life&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Leaving the analogy for a bit, will your model that's looking at domestic cats properly accommodate for &lt;a href=&quot;https://i.ytimg.com/vi/p1JxifyK-Ts/maxresdefault.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Savannah Cats&lt;/a&gt;, or will it consider them out? (They kind of look like cheetahs.) What about &lt;a href=&quot;https://www.prestigeanimalhospital.com/services/cats/breeds/sphynx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sphinx cats&lt;/a&gt;? (They look like raw chicken to me.) &lt;a href=&quot;https://blog.homesalive.ca/14-weird-but-cool-cat-breeds&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elf cats&lt;/a&gt;? (They look like bats.) This is just an example, but you can probably figure out more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the reason behind this problem is that the distinction itself between different classifications (in real life) is not binary, but rather a transition between &quot;yes, that's a textbook cat&quot; and &quot;that's a chair&quot;. Your model will output binary decisions (maybe accompanied by a confidence interval, but even with it, you'll make the call into deciding if it's a cat or not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setting specific boundaries will help. You can define that your model will only detect domestic cats, maybe no bigger than a certain size, only of certain colors, etc... This is limiting what the model will correctly recognize as a cat when we (humans) might disagree. For instance, I would still argue that &lt;a href=&quot;https://www.bbc.com/news/science-environment-14882008&quot; rel=&quot;nofollow noreferrer&quot;&gt;flourescent cats&lt;/a&gt; are still cats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going back to the simple text analogy, this is similar to deciding that to be detected as a cat, it has to start with a &quot;c&quot;. So now you've discarded &lt;code&gt;¡Cat!&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In this way, it's not possible to &lt;strong&gt;ensure&lt;/strong&gt; (notice the word) that your model will detect all of these unknown variations.&lt;/em&gt; There will be always some room for error that needs to be accepted, as long as the errors are infrequent or rare enough that they can be accepted as a regular part of the model.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Act III: &lt;a href=&quot;https://en.wikipedia.org/wiki/Concept_drift&quot; rel=&quot;nofollow noreferrer&quot;&gt;Concept drift&lt;/a&gt;, a cautionary tale&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Finally, the problem becomes even harder as we might be dealing with concepts that change over time, outside of the knowledge of the model, and outside of the knowledge of the person that supervised the model learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As and the breeds of cat changes, your model will have to accommodate by what we (users of the model) consider a valid definition of cat. Which might change in really unexpected ways and not really &quot;look&quot; like a cat. And since your model can only learn from what &quot;looks&quot; like a cat, it's always held in a disadvantaged position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will happen with almost any machine learning model that is approximating a result, regardless of the technique/algorithm. Approximations include a level of error because reality is usually complex in ways that we either don't know about or that are too computationally expensive.&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2020-02-26T22:14:48.853" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18268" PostTypeId="1" CreationDate="2020-02-27T00:25:15.417" Score="1" ViewCount="90" Body="&lt;p&gt;We are discussing planning algorithms currently, and the question is to describe the steps to check if actions could be taken simultaneously. This is a really open-ended question so I'm not sure where to start. &lt;/p&gt;&#xA;" OwnerUserId="33841" LastActivityDate="2020-02-27T10:04:10.227" Title="Can two planning PDDL actions be taken simultaneously?" Tags="&lt;planning&gt;&lt;pddl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18269" PostTypeId="1" CreationDate="2020-02-27T00:31:05.983" Score="2" ViewCount="72" Body="&lt;p&gt;Consider some MDP with no terminal state. We can apply bootstrapping methods (like TD(0)) to learn in these cases no problem, but in policy gradient algorithms that have only a simple monte carlo update, it requires us to supply a complete trajectory (which is impossible with no terminal state). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naturally, one might let the MDP run for 1000 periods, and then terminate as an approximation. If we feed these trajectories into a monte carlo update, I imagine that samples for time period t=1,2,...,100 would give very good estimates for the value function due to the discount factor. However, the time periods 997, 998, 999, 1000, we'd have an expected value for those trajectories far different than V(s) due to their proximity to the cutoff of 1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is this: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should we even include these later-occurring data points when we update our function approximation? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Is the assumption that these points become really sparse in our updates, so they won't have much effect in our training?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Is it usually implied that the final data reward in the trajectory is bootstrapped in these cases (i.e., we have some TD(0)-like behavior in this case)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;OR &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Are monte carlo updates for policy gradient algorithms even appropriate for non-terminating MDPs due to this issue? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33842" LastActivityDate="2020-03-07T09:24:46.340" Title="Monte Carlo updates on policy gradient with no terminal state" Tags="&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18270" PostTypeId="2" ParentId="18268" CreationDate="2020-02-27T02:23:54.270" Score="1" Body="&lt;p&gt;First place to look is how the preconditions/effects of different actions interact.&lt;/p&gt;&#xA;" OwnerUserId="33275" LastActivityDate="2020-02-27T02:23:54.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18271" PostTypeId="1" CreationDate="2020-02-27T04:51:11.360" Score="1" ViewCount="18" Body="&lt;p&gt;I used the pre-trained model &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_resnet101_coco.config&quot; rel=&quot;nofollow noreferrer&quot;&gt;faster_rcnn_resnet101_coco.config&lt;/a&gt; with my own dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two issues &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;some objects were not detected, while I learned it, with a high number of steps, and test over the same learning data, but still have some missing data&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;sometimes, the same identical objects predicted correctly on some images and sometimes it got missed, although the objects are identical &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="32892" LastEditorUserId="2444" LastEditDate="2020-02-27T15:53:24.083" LastActivityDate="2020-02-27T15:53:24.083" Title="Irregular results while prediction identical object on same image" Tags="&lt;tensorflow&gt;&lt;prediction&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18272" PostTypeId="2" ParentId="18265" CreationDate="2020-02-27T05:57:40.013" Score="1" Body="&lt;p&gt;At first glance, I thought this was similar to &quot;continuous-discrete&quot; action selection (&lt;a href=&quot;https://arxiv.org/pdf/1810.06394.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1810.06394.pdf&lt;/a&gt;). However, I think your problem is different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming that each &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; is continuous and that the action which interacts with your environment is the entire vector &lt;span class=&quot;math-container&quot;&gt;$a = (a_1,a_2,\dotso,a_n)$&lt;/span&gt; and not an individual &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt;. Then you could treat it like a hierarchical problem. If you want &lt;span class=&quot;math-container&quot;&gt;$a_1 + a_2 &amp;lt; 2$&lt;/span&gt; for example, then you could sample &lt;span class=&quot;math-container&quot;&gt;$a_1 \sim U(0,2)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2 | a_1 \sim U(0, 2-a_1)$&lt;/span&gt; and have &lt;span class=&quot;math-container&quot;&gt;$p(a) = p(a_2 | a_1)p(a_1)$&lt;/span&gt;. The specifics of how you do this depends more finely on how your problem is set up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you can find similar ideas from the paper linked above. Also, other work in the robitics literature studies structured and hybrid action spaces.&lt;/p&gt;&#xA;" OwnerUserId="33340" LastEditorUserId="33340" LastEditDate="2020-02-28T02:16:39.723" LastActivityDate="2020-02-28T02:16:39.723" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18273" PostTypeId="1" AcceptedAnswerId="18305" CreationDate="2020-02-27T08:26:44.413" Score="0" ViewCount="54" Body="&lt;p&gt;Here's a tutorial about doing custom training of YOLO (Darknet): &lt;a href=&quot;https://medium.com/@manivannan_data/how-to-train-yolov3-to-detect-custom-objects-ccbcafeb13d2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@manivannan_data/how-to-train-yolov3-to-detect-custom-objects-ccbcafeb13d2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tutorial guides how to set values in the &lt;code&gt;.cfg&lt;/code&gt; files:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;classes = Number of classes, OK&lt;/li&gt;&#xA;&lt;li&gt;filters = (classes + 5) * 3&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is it 'plus 5' then 'times 3'?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some say it's (classes + coords + 1) * num, but I can't guess it out the meaning.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-28T09:43:10.673" Title="YOLOv3 Model Structure: Why is filters = (classes + coords + 1) * num?" Tags="&lt;computer-vision&gt;&lt;yolo&gt;&lt;darknet&gt;&lt;models&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18274" PostTypeId="2" ParentId="18204" CreationDate="2020-02-27T09:39:37.603" Score="7" Body="&lt;p&gt;It is a difficult question to answer, as &amp;mdash; for a start &amp;mdash; we still don't really know what 'intelligence' means. It's a bit like Supreme Court Justice Potter Stewart declining to define 'pornography', instead stating that [...]&lt;em&gt;I know it when I see it&lt;/em&gt;. AGI will be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no single event (almost by definition), as that's not general. OK, we've got machines that can beat the best human players at chess and go, two games that were for centuries seen as an indication of intelligence. But can they order a takeaway pizza? Do they even understand what they are doing? Or, even more fundamental, know what &lt;em&gt;they&lt;/em&gt; means in the previous sentence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for a machine to show a non-trivial level of intelligent behaviour, I would expect it to interact with its environment (which is more &lt;a href=&quot;https://en.wikipedia.org/wiki/Social_intelligence&quot; rel=&quot;noreferrer&quot;&gt;social intelligence&lt;/a&gt;, an aspect that seems to be rather overlooked in much of AI). I would expect it to be aware of what it's doing/saying. If I have a conversation with a chatbot that &lt;em&gt;really&lt;/em&gt; understands what it's saying (and can explain why it came to certain conclusions), that would be an indication that we're getting closer to AGI. So Turing wasn't that far off, though nowadays it's more achieved with smoke and mirrors rather than 'real' intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding a story: being able to finish a partial story in a sensible way, inferring and extrapolating the motives of characters, being able to say why a character acted in a particular way. That for me would be a better sign of AGI than beating someone at chess or solving complex equations. Jokes that are funny; breaking rules in story-telling in a sensible way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing stories: &lt;a href=&quot;https://nanogenmo.github.io/&quot; rel=&quot;noreferrer&quot;&gt;NaNoGenMo&lt;/a&gt; is a great idea, and throws up lots of really creative stuff, but how many of the resulting novels would you want to read instead of human-authored books? Once that process has generated a best-seller (based on the quality of the story), then we might be getting closer to AGI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Composing music: of course you can already generate decent music using ML algorithms. Similar to stories, the hard bit is the intention behind choices. If choices are random (or based on learnt probabilities), that is purely imitation. An AGI should be able to do more than that. Give it a libretto and ask it to compose an opera around it. Do this 100 times, and when more than 70-80 of the resulting operas are actually decent pieces of music that one would want to listen to, then great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Self-driving cars? That's not really any more intelligent (but a lot sexier!) than to walk around in a crowd without bumping into people and not getting run over by a bus. In my view it's much more a sign of intelligence if you can translate literature into a foreign language and the people reading it actually end up enjoying it (instead of wondering who translated that garbage).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One aspect we need to be aware of is anthropomorphising. Weizenbaum's &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;noreferrer&quot;&gt;ELIZA&lt;/a&gt; was taken for more than it was, because its users tried to make sense of the conversations they had and built up a mental model of Eliza, which clearly wasn't there on the other side of the screen. I would want to see some real evidence of intentionality of what an AGI was doing, rather than ascribing intelligence to it because it acts in a way that I'm able to interpret.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-02-27T09:45:36.880" LastActivityDate="2020-02-27T09:45:36.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18275" PostTypeId="2" ParentId="18268" CreationDate="2020-02-27T10:04:10.227" Score="0" Body="&lt;p&gt;I don't see any principal problem with that. The way I would approach it is to have a resource model and durations attached to actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, movement would put a lock on your legs. You can't have another movement at the same time, as your legs are already busy. But your attention might only be partially occupied, so you can make a phone call while you're moving. You won't be able to read a book, because your eyes might be partially busy monitoring the walking action. This can be encoded in pre- and post-conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What will probably be easier is to parallelise the execution of the plan. Once the plan has been created, organise the actions in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Gantt_chart&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gantt-chart&lt;/a&gt; like structure. You can have mutual exclusion in there, so all 'movement' would be restricted to one single row, so no more than one movement can take place at the same time. But 'making a phone call' could be in a separate row, and thus execute in parallel. The details depend on the requirements of the actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't easily think of a way that it would impact the planning process itself; unless there are critical timings involved. So leaving the planner to do its thing might keep it simpler, and then the optimal execution could be a post-processing step.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-27T10:04:10.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18276" PostTypeId="1" CreationDate="2020-02-27T10:44:16.340" Score="2" ViewCount="40" Body="&lt;p&gt;This is a &lt;a href=&quot;https://stats.stackexchange.com/q/451250/82135&quot;&gt;cross-post&lt;/a&gt;, as I didn't get any answers on Stats SE and I am hoping that it gets more attention here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the bottom of page 2 of the paper &lt;a href=&quot;https://arxiv.org/abs/1706.05350&quot; rel=&quot;nofollow noreferrer&quot;&gt;L2 Regularization versus Batch and Weight Normalization&lt;/a&gt;, the equation for the gradient of the output with respect to the weights is given as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\triangledown y_{BN} (X; w, \gamma, \beta)  = \frac{X}{\sigma(X)}\gamma g'(z).&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone break down into smaller steps on how the author got to that equation?  &lt;/p&gt;&#xA;" OwnerUserId="33850" LastEditorUserId="2444" LastEditDate="2020-02-27T15:36:01.010" LastActivityDate="2020-02-27T15:36:01.010" Title="How is the gradient with respect to weights derived in batch normalization?" Tags="&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;papers&gt;&lt;regularization&gt;&lt;batch-normalization&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18278" PostTypeId="1" AcceptedAnswerId="18280" CreationDate="2020-02-27T12:53:48.450" Score="1" ViewCount="209" Body="&lt;p&gt;Is it correct that for SARSA to converge to the optimal value function (and policy)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The learning rate parameter  &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; must satisfy the conditions:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sum \alpha_{n^k(s,a)} =\infty \quad \text{and}\quad \sum \alpha_{n^k(s,a)}^{2} &amp;lt;\infty \quad \forall s \in \mathcal{S}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$n_k(s,a)$&lt;/span&gt; denotes the &lt;span class=&quot;math-container&quot;&gt;$k^\text{th}$&lt;/span&gt; time &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; is visited&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; (of the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy) must be decayed so that the policy converges to a greedy policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every state-action pair is visited infinitely many times.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are any of these conditions redundant?&lt;/p&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-02-27T13:04:07.133" LastActivityDate="2020-02-27T14:10:40.440" Title="What are the conditions for the convergence of SARSA to the optimal value function?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;sarsa&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18279" PostTypeId="2" ParentId="18278" CreationDate="2020-02-27T13:29:22.227" Score="1" Body="&lt;p&gt;I have the conditions for convergence in these notes &lt;a href=&quot;https://webee.technion.ac.il/shimkin/LCS11/ch7_exploration.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SARSA convergence&lt;/a&gt; by &lt;a href=&quot;https://webee.technion.ac.il/shimkin/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nahum Shimkin&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The Robbins-Monro conditions above hold for &lt;span class=&quot;math-container&quot;&gt;$α_t$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every state-action pair is visited infinitely often&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The policy is greedy with respect to the policy derived from &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; in the limit&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The controlled Markov chain is communicating: every state can be reached from any other with positive probability (under some policy).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\operatorname{Var}{R(s, a)} &amp;lt; \infty$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the reward function&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-02-27T14:10:40.440" LastActivityDate="2020-02-27T14:10:40.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18280" PostTypeId="2" ParentId="18278" CreationDate="2020-02-27T13:40:12.440" Score="1" Body="&lt;p&gt;The paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1023/A:1007678930559.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms&lt;/a&gt; by Satinder Singh et al. proves that SARSA(0), in the case of a tabular representation of the value functions, converges to the optimal value function, provided certain assumptions are met&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Infinite visits to every state-action pair &lt;/li&gt;&#xA;&lt;li&gt;The learning policy becomes greedy in the limit&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The properties are more formally stated in lemma 1 (page 7 of the pdf) and theorem 1 (page 8). The &lt;a href=&quot;https://arxiv.org/pdf/1808.00245.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robbins–Monro conditions should ensure that each state-action pair is visited infinitely often&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-02-27T13:40:12.440" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18281" PostTypeId="2" ParentId="18187" CreationDate="2020-02-27T15:17:37.403" Score="2" Body="&lt;p&gt;As far as I know, there is no &lt;em&gt;very&lt;/em&gt; simple proof of the convergence of temporal-difference algorithms. The proofs of convergence of TD algorithms are often based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic approximation theory&lt;/a&gt; (given that e.g. &lt;a href=&quot;https://papers.nips.cc/paper/764-convergence-of-stochastic-iterative-dynamic-programming-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning can be viewed as a stochastic process&lt;/a&gt;) and the work by &lt;a href=&quot;http://mathworld.wolfram.com/Robbins-MonroStochasticApproximation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robbins and Monro&lt;/a&gt; (in fact, the Robbins-Monro conditions are usually assumed in the theorems and proofs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proofs of convergence of Q-learning (a TD(0) algorithm) and SARSA (another TD(0) algorithm), when the value functions are represented in tabular form (as opposed to being approximated with e.g. a neural network), can be found in different research papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the &lt;strong&gt;proof of convergence of tabular Q-learning&lt;/strong&gt; can be found in the paper &lt;a href=&quot;https://papers.nips.cc/paper/764-convergence-of-stochastic-iterative-dynamic-programming-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence of Stochastic Iterative Dynamic Programming Algorithms&lt;/a&gt; (1994) by Tommi Jaakkola et al. The &lt;strong&gt;proof of convergence of tabular SARSA&lt;/strong&gt; can be found in the paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1023/A:1007678930559.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms&lt;/a&gt; (2000) by Satinder Singh et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://ai.stackexchange.com/q/13973/2444&quot;&gt;How to show temporal difference methods converge to MLE?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-02-27T15:22:44.197" LastActivityDate="2020-02-27T15:22:44.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18282" PostTypeId="1" CreationDate="2020-02-27T15:23:50.410" Score="1" ViewCount="68" Body="&lt;p&gt;The conditions of convergence of SARSA(0) to the optimal policy are :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The Robbins-Monro conditions above hold for &lt;span class=&quot;math-container&quot;&gt;$α_t$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every state-action pair is visited infinitely often&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The policy is greedy with respect to the policy derived from &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; in the limit&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The controlled Markov chain is communicating: every state can be reached from any other with positive probability (under some policy).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\operatorname{Var}{R(s, a)} &amp;lt; \infty$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the reward function&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The original proof of the convergence of TD(0) prediction (page 24 of the paper &lt;a href=&quot;http://incompleteideas.net/papers/sutton-88-with-erratum.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Predict by the Method of Temporal Differences&lt;/a&gt;) was for convergence in the mean of the estimation to the true value function. This did not require the learning rate parameter to satisfy Robbins-Monro conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if the Robbins-Monro conditions are removed from the SARSA(0) assumptions would the policy converge in some notion of expectation to the optimal policy?&lt;/p&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-02-27T15:37:35.117" LastActivityDate="2020-02-27T15:37:35.117" Title="Does SARSA(0) converge to the optimal policy in expectation if the Robbins-Monro conditions are removed?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;temporal-difference-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18283" PostTypeId="1" AcceptedAnswerId="18284" CreationDate="2020-02-27T17:59:12.340" Score="2" ViewCount="293" Body="&lt;p&gt;If the agent is following an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy derived from Q, is there any advantage to decaying &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; even though &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; decay is not required for convergence ?&lt;/p&gt;&#xA;" OwnerUserId="33227" LastActivityDate="2020-02-27T19:02:28.193" Title="Is there an advantage in decaying $\epsilon$ during Q-Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18284" PostTypeId="2" ParentId="18283" CreationDate="2020-02-27T18:57:08.707" Score="1" Body="&lt;p&gt;Yes Q-learning benefits from decaying epsilon in at least two ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Early exploration. It makes little sense to follow whatever policy is implied by the initialised network closely, and more will be learned about variation in the environment by starting with a random policy. It is fairly common in DQN to initially fill the experience replay table whilst using &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 1.0$&lt;/span&gt; or otherwise an effectively random policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Later refinement. Q-learning can only learn from experiences it has. A behaviour policy that is too random may not experience enough states close to optimal in order to gain enough statistics on them to overcome variance. In more difficult cases, it may never experience the whole of an optimal trajectory even when combining all the different transitions that it ever observes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition, when using a function approximator such as a neural network, predictions of the target policy will be influenced by the distribution of states and actions in the experience replay memory. If that is is biased towards the distribution of states of a very different behaviour policy, then basic Q-learning has no good way to adjust for that - it adjusts for differences in expected return between behaviour and target policies, but not in distribution of observed states. In fact this is still a somewhat open problem, you want the agent to learn from mistakes and imperfect behaviour so as to avoid it, but you don't want those mistakes to skew predictions of what should happen under an optimal policy. Function approximation such as neural networks is influenced soemwhat by the distribution of input data, so it typically performs better in Q learning when the behaviour policy and target policy are close e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; should be relatively low if using &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical implementation might start with &lt;span class=&quot;math-container&quot;&gt;$\epsilon=1.0$&lt;/span&gt;, set a decay factor per time step or per episode e.g. &lt;span class=&quot;math-container&quot;&gt;$0.999$&lt;/span&gt; per episode, and a minimum &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0.01$&lt;/span&gt;. These are all hyper-parameters that you can adjust depending on the problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-02-27T19:02:28.193" LastActivityDate="2020-02-27T19:02:28.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18285" PostTypeId="2" ParentId="18232" CreationDate="2020-02-27T19:31:13.830" Score="4" Body="&lt;p&gt;Meta-learning is more about speeding up and optimizing hyperparameters for networks that are not trained at all, whereas transfer learning uses a net that has already been trained for some task and reusing part or all of that network to train on a new task which is relatively similar. So, although they can both be used from task to task to a certain degree, they are completely different from one another in practice and application, one tries to optimize configurations for a model and the other simply reuses an already optimized model, or part of it at least.&lt;/p&gt;&#xA;" OwnerUserId="20044" LastEditorUserId="2444" LastEditDate="2020-02-27T23:35:00.447" LastActivityDate="2020-02-27T23:35:00.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18287" PostTypeId="2" ParentId="18204" CreationDate="2020-02-27T21:32:16.033" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;For me it might be an automata that can adequately solve problems without precisely definable parameters, across the spectrum of activities engaged in by humans.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I use this metric because this is what humans seem to do--make decisions with adequate utility even when we can't break it down mathematically.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This may require the ability to define problems to be adequately solved. This can be understood as an element of creativity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this context, everything is either a puzzle or game, dependent on whether it involves more than one agent.  Such problems could either be mundane, such as opening a door that is different from standard doors, or identifying novel problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining problems to be solved touches on Oliver's point about intentionality.  &lt;em&gt;(Where I disagree with Oliver is in the notion that intelligence is not fundamentally definable--after much research on the subject it seems to be a measure of fitness in an environment, where an environment can be anything. The etymology of term itself strongly indicates the ability to select between alternatives, thus a function of decision making, measured by utility vs. other decision making agents.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Such a mechanism could be a &quot;Chinese Room&quot;, in that consciousness, qualia &amp;amp; self awareness in the human sense are not requirements for general intelligence, per se.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On Art:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mistrust the idea that artistic accomplishment would be a sure marker b/c response to art is subjective, and the process of art is Darwinian--an exponentially greater of artists must &quot;fail&quot; for a single artist to &quot;succeed&quot;.  Works that humans might ascribe to &quot;genius&quot; can be created by genetic algorithmic process, where time and memory are the only limiters. [See: &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Library_of_Babel&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Library at Babel] &lt;/a&gt;   A groundbreaking symphony would be difficult to produce, just per the length of the composition, but much of pop music is already algorithmically generated, and narrowly intelligent algorithms are already producing legit abstract visual art. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computers are good at math, and Art is inherently mathematical. This is easiest to discern in music, which is just combinations of frequencies and time signatures that produce an effect in the listener.  This holds for visual art, which depends on balance (equilibria), composition (spacial relationships), and shading or color (frequencies).  If we believe Borges, even literature is inherently mathematical (think &quot;narrative arcs&quot; and set theory &amp;amp; combinatorics in regard to characters and events.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, nobody really know what is going to &quot;work&quot; until it is presented to an audience, so what constitutes great art is typically a matter of what persists over time and remains, or becomes, relevant.  (This can wax and wane--Shakespeare did not always occupy his position at the top of the English lit food chain!  The author's greatness is very much a function of interpretation of his work, not least because dramatic art is inherently interpretive, in the sense that this is the task of the performers.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-02-27T22:46:39.243" LastActivityDate="2020-02-27T22:46:39.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18288" PostTypeId="1" AcceptedAnswerId="18292" CreationDate="2020-02-27T21:41:46.513" Score="2" ViewCount="183" Body="&lt;p&gt;In &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q Learning&lt;/a&gt; the parametrized Q-functions &lt;span class=&quot;math-container&quot;&gt;$Q_i$&lt;/span&gt; are optimised by performing gradient descent on the series of loss functions &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L_i(\theta_i)= E_{(s,a)\sim p}[(y_i-Q(s,a;\theta_i))^2]$&lt;/span&gt; , where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y_i = E_{s' \sim \mathcal{E}}[r+\gamma \max_{a'}Q(s',a';\theta_{i+1})\mid s,a]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the actual algorithm, however, the expected value is never computed. Also, I think it cannot be computed since the transition probabilities of the underlying MDP remain hidden from the agent. Instead of the expected value, we compute &lt;span class=&quot;math-container&quot;&gt;$y_i = r_i + \gamma \max_a Q(\phi_{i+1},a;\theta)$&lt;/span&gt;. I assume some sort of stochastic approximation is taking place here. Can someone explain the details?&lt;/p&gt;&#xA;" OwnerUserId="27047" LastEditorUserId="2444" LastEditDate="2020-02-27T22:17:05.490" LastActivityDate="2020-02-28T03:53:54.283" Title="How is the expected value in the loss function of DQN approximated?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;math&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18289" PostTypeId="1" CreationDate="2020-02-28T00:26:56.743" Score="1" ViewCount="56" Body="&lt;p&gt;I would like to ask for a piece of advice with regard to Q-learning. I am studying RL and would like to do a basic project applied to life science and calculate the reward. I have been trying to get my head around how to define all possible states of the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My states are &lt;span class=&quot;math-container&quot;&gt;$S = ( \text{health } (4 \text{ levels}), \text{shape } (3 \text{ levels}) \}$&lt;/span&gt;. My actions are &lt;span class=&quot;math-container&quot;&gt;$A=\{a_1, a_2, \dots, a_4 \}$&lt;/span&gt;. My possible states are &lt;span class=&quot;math-container&quot;&gt;$60=4 * 3 * 5$&lt;/span&gt;. Could you advise whether these are correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(s_{w_0, sh_0}, a_1, s'_{w_1, sh_1})$&lt;/span&gt; is a tuple of the initial state &lt;span class=&quot;math-container&quot;&gt;$s_{w_0, sh_0}$&lt;/span&gt;, the first action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and the next state &lt;span class=&quot;math-container&quot;&gt;$s'_{w_1, sh_1}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the health level, &lt;span class=&quot;math-container&quot;&gt;$sh$&lt;/span&gt; is the shape of the tumor. &lt;/p&gt;&#xA;" OwnerUserId="33862" LastEditorUserId="2444" LastEditDate="2020-02-29T04:17:27.303" LastActivityDate="2020-02-29T04:17:27.303" Title="How should I define the state space for this life science problem?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;q-learning&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18290" PostTypeId="1" CreationDate="2020-02-28T00:40:27.960" Score="3" ViewCount="160" Body="&lt;p&gt;There is an idea that intentionality may be a requirement of true intelligence, here defined as human intelligence.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But all I know for certain is that we have the &lt;em&gt;appearance&lt;/em&gt; of free will.  Under the assumption that the universe is purely deterministic, what do we mean by intention?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(This seems an important question given that intention is not just a philosophical matter in relation to definitions of AI, but involves ethics in the sense of application of AI, &quot;offloading responsibility to agents that cannot be meaningfully punished&quot; as an example.  Also touches on goals, implied by intention, whether awareness is a requirement, and what constitutes awareness. I'm interested in all angles, but was inspired by the question &quot;does true art require intention, and, if so, is that the sole domain of humans?&quot;)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2021-05-10T00:20:50.940" Title="How do we define intention if there is no free will?" Tags="&lt;philosophy&gt;&lt;intelligent-agent&gt;&lt;intelligence&gt;&lt;goal-based&gt;" AnswerCount="4" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18292" PostTypeId="2" ParentId="18288" CreationDate="2020-02-28T01:33:27.630" Score="2" Body="&lt;p&gt;Just as the &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; says&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L_i(\theta_i)= E_{(s,a)\sim p}[(y_i-Q(s,a;\theta_i))^2]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_i = E_{s' \sim \mathcal{E}}[r+\gamma \max_{a'}Q(s',a';\theta_{i+1})\mid s,a]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then in the &lt;em&gt;Background&lt;/em&gt; section of the paper, it says&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Differentiating the loss function with respect to the weights we arrive at the following gradient:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta_i} L_i(\theta_i)\\= E_{(s,a)\sim p,s'\sim\mathcal{E}}\left[\left(r+\gamma \max_{a'}Q(s',a';\theta_{i+1})-Q(s,a;\theta_i)\right)\nabla_{\theta_i}Q(s,a;\theta_i)\right]\tag{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Rather than computing the full expectations in the above gradient, it is often computationally expedient to optimize the loss function by &lt;strong&gt;stochastic gradient descent&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;and the expectations &lt;strong&gt;are replaced by single samples&lt;/strong&gt; from the behavior distribution &lt;span class=&quot;math-container&quot;&gt;$ρ$&lt;/span&gt; and the emulator &lt;span class=&quot;math-container&quot;&gt;$\mathcal{E}$&lt;/span&gt; respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you're familiar with SGD and &lt;a href=&quot;https://pdfs.semanticscholar.org/34dd/d8865569c2c32dec9bf7ffc817ff42faaa01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic Optimization&lt;/a&gt; then you know what happens here:&lt;/p&gt;&#xA;&lt;p&gt;The expression inside the expectation of (1), i.e. &lt;span class=&quot;math-container&quot;&gt;$\left(r+\gamma \max_{a'}Q(s',a';\theta_{i+1})-Q(s,a;\theta_i)\right)\nabla_{\theta_i}Q(s,a;\theta_i)$&lt;/span&gt; , is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Bias_of_an_estimator&quot; rel=&quot;nofollow noreferrer&quot;&gt;unbiased estimation&lt;/a&gt; of the real gradient &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_i}L_i(\theta_i)$&lt;/span&gt; - its expectation is the real gradient. In other words,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\widehat{\nabla L}=\left(r+\gamma \max_{a'}Q(s',a';\theta_{i+1})-Q(s,a;\theta_i)\right)\nabla_{\theta_i}Q(s,a;\theta_i).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then by theory of Stochastic Optimization we can optimize &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\theta\leftarrow \theta - \alpha\widehat{\nabla L}$&lt;/span&gt; , which is how SGD works.&lt;/p&gt;&#xA;&lt;p&gt;The unbiased estimation &lt;span class=&quot;math-container&quot;&gt;$\left(r+\gamma \max_{a'}Q(s',a';\theta_{i+1})-Q(s,a;\theta_i)\right)\nabla_{\theta_i}Q(s,a;\theta_i)$&lt;/span&gt; can be sampled and calculated directly - you can run the emulator &lt;span class=&quot;math-container&quot;&gt;$\mathcal{E}$&lt;/span&gt; and current behavior policy to collect &lt;span class=&quot;math-container&quot;&gt;$r, s', a'$&lt;/span&gt; and calculate the gradient of your Q network &lt;span class=&quot;math-container&quot;&gt;$\nabla Q$&lt;/span&gt; using TensorFlow. So the gradient to &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; can be approximated.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$y_j$&lt;/span&gt; in the pseudocode  of the paper is also an estimation. It'll be more proper to denote it as &lt;span class=&quot;math-container&quot;&gt;$\hat{y_j}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;(I'm not an native English speaker so forgive my poor expression.)&lt;/p&gt;&#xA;" OwnerUserId="33843" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-28T03:53:54.283" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18293" PostTypeId="1" AcceptedAnswerId="18367" CreationDate="2020-02-28T02:00:58.907" Score="1" ViewCount="128" Body="&lt;p&gt;I am trying to put together a public agricultural image database of corn and soybeans, to train convolutional neural networks. The main method of image collection will be through taking pictures of various fields in the growing season. The images will be uploaded to a public data sharing site which will be accessible by many.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I could get much more images compiled if I were to take some off of, say, Google Images. If there anything wrong with this? Would there be any issues with copywright infringements if I find the images on a publicly-available search engine? I need a lot of images, so I thought this would be a good method of increasing my image numbers.&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-04-26T23:00:52.527" Title="Is it legal to construct a public image database (for deep learning) with images from the internet?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ClosedDate="2020-03-05T15:50:27.773" ContentLicense="CC BY-SA 4.0" />
  <row Id="18295" PostTypeId="2" ParentId="18220" CreationDate="2020-02-28T03:45:52.887" Score="0" Body="&lt;p&gt;Personally, when I begin designing a machine learning model, I consider the following points:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;My data: if I have simple images, like MNIST ones, or in general images with very low resolution, a very deep network is &lt;strong&gt;not&lt;/strong&gt; required.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If my problem statement needs to learn a lot of features from each image, such as for the human face, I may need to learn eyes, nose, lips, expressions through their combinations, then I &lt;strong&gt;need&lt;/strong&gt; a deep network with convolutional layers.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I have time-series data, LSTM or GRU makes sense, but, I also consider recurrent setup when my data has high resolution, low count data points.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The upper limit however may get decided by resources available on the computing device you are using for training.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="33781" LastEditorUserId="2444" LastEditDate="2020-11-13T20:41:24.067" LastActivityDate="2020-11-13T20:41:24.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18297" PostTypeId="1" CreationDate="2020-02-28T04:03:30.253" Score="2" ViewCount="35" Body="&lt;p&gt;I am attempting to make a 2-D platformer game where the player traverses through an evil factory that is producing killer robots. The robots spawn at multiple specific locations in each level and impede the player's progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enemies are procedurally generated using machine learning. Early levels have &quot;garbage&quot; robots that plop down and can't really do anything. After generations of training, the robots begin having more refined bodies and are able to move about and attack the player. Later levels produce enemies that are more challenging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enemies consist of a body and up to 4 limbs. The body is simply a circle of a certain radius, while the limbs are just a bar with a certain length. Limbs can pivot and/or contract/extend. Additionally, each limb can have one of three types of &quot;motor&quot; (wheel, spring, or hover). This makes for about 20-25 input parameters:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;BodySize, Limb1Enabled, Limb1PivotPoint, Limb1Length, Limb1Angle, Limb1MotorType, Limb1MotorStrength, Limb2Enabled, Limb2PivotPoint, Limb2Length, Limb2Angle, Limb2MotorType, Limb2MotorStrength, Limb3Enabled, Limb3PivotPoint, Limb3Length, Limb3Angle, Limb3MotorType, Limb3MotorStrength, Limb4Enabled, Limb4PivotPoint, Limb4Length, Limb4Angle, Limb4MotorType, Limb4MotorStrength&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My thoughts are that a genetic algorithm (or something similar) would be used to generate a body, while a neural network would control that body by using the same inputs to generate outputs that control the limbs and motors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There would actually be 3 &quot;control brains&quot; that would have to be trained using the same inputs, but having different fitness goals: Moving Right/Left, Moving Up, and Attacking the Player. &lt;em&gt;(Gravity exists in 2-D platformers, so moving down isn't necessary.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fourth, &quot;master brain&quot; would take the player's relative location, score, and maybe time elapsed, as inputs, and would output one of the goals for the robot to achieve (move left, move right, and attack).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The master brain's fitness would be determined by the &quot;inverse&quot; of the player's &quot;progress&quot;, while each control brain's fitness would be determined by how well it was able to perform the task assigned by the master brain. Finally, the overall fitness for the body's genetic algorithm would be an average (or some other function like min, max, etc.) of the three control brain's fitness values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that I have all this &quot;down on paper&quot;, where do I start? I had planned on doing this in Unity, but early attempts have been a bit confusing for me. I've been able to procedurally generate a body with random limbs (no motors) that wiggle about randomly, but there's no neural network or any machine learning going on whatsoever. I am not exactly sure how to expose my parameters to be used as inputs, and am barely grasping how I should take those outputs to control what I want them to. Are there any libraries I should look at, or should I write this all from scratch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, before I get too far ahead of myself, what are the flaws in my approach (as I'm sure there are plenty). I want my project to be something practical in scope, if training can't be done feasibly while a player traverses a level, this might just be a dead project idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, that all being said, thank you for your help.&lt;/p&gt;&#xA;" OwnerUserId="33523" LastActivityDate="2020-02-28T04:03:30.253" Title="Using ML for Enemy Generation in Video Games" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;game-ai&gt;&lt;genetic-algorithms&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18298" PostTypeId="1" AcceptedAnswerId="18300" CreationDate="2020-02-28T04:03:38.747" Score="1" ViewCount="540" Body="&lt;p&gt;I'm testing out YOLOv3 using the 'darknet' binary, and custom config. It trains rather slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My testing out is only with 1 image, 1 class, and using YOLOv3-tiny instead of YOLOv3 full, but the training of yolov3-tiny isn't fast as expected for 1 class/1 image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accuracy reached near 100% after like 3000 or 4000 batches, in similarly 3 to 4 hours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is it slow with just 1 class/1 image?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-28T06:08:50.547" Title="What are the reasons behind slow YOLO training?" Tags="&lt;training&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;yolo&gt;&lt;darknet&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18299" PostTypeId="1" CreationDate="2020-02-28T05:07:16.153" Score="2" ViewCount="50" Body="&lt;p&gt;I have a task of classifying spatial data from a geographic information system. More precisely, I need a way to filter out unnecessary line segments from the CAD system before loading into the GIS (see the attached picture, colors for illustrative purposes only).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A0LRu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A0LRu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that there are much more variations of objects than in the picture. The task is difficult to solve in an algorithmic way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to apply a bunch of classification algorithms from the Scikit-learn package and, in general, got significant results. GradientBoostingClassifier and ExtraTreeClassifier achive an accuracy about 96-98%, but:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;this accuracy is achieved in the context of individual segments into&#xA;which I explode the source objects (hundreds of thousands of objects)&#xA;After reverse aggregation of objects it may turn out that one of&#xA;their segments in each object is classified incorrectly. The error in&#xA;the context of the source objects is high&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;significant computational resources and time are required for the&#xA;preparation of the source data, and calculation of features for&#xA;classifiers&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;it is impossible to use the source 2d coordinates of objects in&#xA;algorithms, but only their derivatives&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I tried to find good examples of use deep neural networks for this kind of tasks / for spatial data, but I only found articles that are quite difficult to understand about the use of such networks for point clouds classification, and some information on geometric deep learning for graphs. I do not have enough knowledge to adapt them for my case&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone provide me good examples of using neural networks directly with 2d coordinates and, maybe, good articles on these theme written in simple language?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="33820" LastEditorUserId="33820" LastEditDate="2020-02-28T11:23:02.403" LastActivityDate="2020-02-28T11:23:02.403" Title="Suitable deep learning algorithms for spatial / geometric data" Tags="&lt;deep-learning&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;scikit-learn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18300" PostTypeId="2" ParentId="18298" CreationDate="2020-02-28T05:21:09.443" Score="2" Body="&lt;p&gt;I think you underestimate the size of YOLO. This is the size of one segment of yolo tiny according to the darknet .cfg file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Convolutional Neural Network structure:&#xA;416x416x3                Input image&#xA;416x416x16               Convolutional layer: 3x3x16, stride = 1, padding = 1&#xA;208x208x16               Max pooling layer: 2x2, stride = 2&#xA;208x208x32               Convolutional layer: 3x3x32, stride = 1, padding = 1&#xA;104x104x32               Max pooling layer: 2x2, stride = 2&#xA;104x104x64               Convolutional layer: 3x3x64, stride = 1, padding = 1&#xA;52x52x64                 Max pooling layer: 2x2, stride = 2&#xA;52x52x128                Convolutional layer: 3x3x128, stride = 1, padding = 1&#xA;26x26x128                Max pooling layer: 2x2, stride = 2&#xA;26x26x256                Convolutional layer: 3x3x256, stride = 1, padding = 1&#xA;13x13x256                Max pooling layer: 2x2, stride = 2&#xA;13x13x512                Convolutional layer: 3x3x512, stride = 1, padding = 1&#xA;12x12x512                Max pooling layer: 2x2, stride = 1&#xA;12x12x1024               Convolutional layer: 3x3x1024, stride = 1, padding = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;.cfg file found here: &lt;a href=&quot;https://github.com/pjreddie/darknet/blob/master/cfg/yolov3-tiny.cfg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pjreddie/darknet/blob/master/cfg/yolov3-tiny.cfg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: These networks generally aren't specifically designed to train fast, they're designed to run fast at test time, where it matters&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-02-28T05:21:09.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18301" PostTypeId="2" ParentId="18298" CreationDate="2020-02-28T06:08:50.547" Score="1" Body="&lt;p&gt;It depends upon the factors such as &#xA;1. Batch size (GPU memory capacity)&#xA;2. CPU speed and number of cores(multi-threading to load the images)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of classes increase the number of convolution filters only in the prediction layers of YOLO. It influences only less than 1% speed of the detector to train the model.&lt;/p&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2020-02-28T06:08:50.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18303" PostTypeId="1" CreationDate="2020-02-28T08:34:52.210" Score="20" ViewCount="4119" Body="&lt;p&gt;This isn't really a conspiracy theory question. More of an inquire on the global computational power and data storage logistics question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most recording instruments such as cameras and microphones are typically voluntary opt in devices, in that, they have to be activated before they start recording. What happens if all of these devices were permanently activated and started recording data to some distributed global data storage?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 400 hours of video uploaded to YouTube every minute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let’s do some very rough math.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m going to assume for the rest of this post that the average video is 1080p which is 2.5GB (or &lt;span class=&quot;math-container&quot;&gt;$10^9$&lt;/span&gt; bytes) per hour. From that, we get about 400 hrs * 60 mins * 2.5GB/hrs * 24 hrs = 1.5 petabytes (or &lt;span class=&quot;math-container&quot;&gt;$10^{15}$&lt;/span&gt; bytes) per day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But YouTube videos post are voluntary, and they are far from continuous video streams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are about 3.5 billion smartphones in the world. If video was continuously streamed and recorded, going through the same video math above (&lt;span class=&quot;math-container&quot;&gt;$3.5 * 10^9 * 1.5 * 10^{15} * 24)$&lt;/span&gt; = 126 yottabytes (or &lt;span class=&quot;math-container&quot;&gt;$10^{24}$&lt;/span&gt; bytes) per day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The IDC projects there will be 175 zettabytes (or &lt;span class=&quot;math-container&quot;&gt;$10^{21}$&lt;/span&gt; bytes) in 2025.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless my math is very wrong, it would seem as though smartphone cameras alone could produce more data in one day than all of the data created in human history in 2025.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, so far, has only been about the data recording, but, to implement a surveillance state, all recorded data would need to be processed by AI to intelligent flag data that is significant. How much processing power would be needed to filter 126 yottabytes into relevant information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, this question is motivated by the spread of dystopian surveillance media like Edward Snowden NSA whistle blowing leaks or George Orwell's sentiment of &quot;Big Brother is Watching You&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computationally, could we be surveilled, and to what extent? I imagine text messages surveillance would be the easiest, does the world have the computation power to surveil all text messages? How about audio? or video?&lt;/p&gt;&#xA;" OwnerUserId="33848" LastEditorUserId="2444" LastEditDate="2020-02-28T21:23:26.810" LastActivityDate="2021-05-01T08:22:48.537" Title="Is a dystopian surveillance state computationally possible?" Tags="&lt;social&gt;&lt;computation&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18304" PostTypeId="2" ParentId="18303" CreationDate="2020-02-28T09:36:40.140" Score="21" Body="&lt;p&gt;You don't necessarily have to analyse it all. Just by having such data available you can achieve a lot in terms of surveillance, as long as you can retrieve relevant parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few years ago there was a &lt;a href=&quot;https://www.wnycstudios.org/podcasts/radiolab/articles/eye-sky&quot; rel=&quot;noreferrer&quot;&gt;Radiolab podcast, &quot;The Eye in the Sky&quot;&lt;/a&gt; (there's a full transcript on the site). The basic idea is that you have a plane circling a city 24/7, and filming what goes on. If there was a crime somewhere, you retrieve the recordings after the event, and you can track back to where vehicles involved in the crime were coming from, and where they went after the crime. If nothing happens, you simply archive the data, and perhaps remove it after a month or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method was used to solve a hit-and-run assassination of a police woman who was on her way to work. The gang who committed the attack were rather surprised when the police showed up at their secret hide-out a few days later, as they could see on the images where the cars involved went to later. At the time and place of the murder there were obviously no witnesses who could have done that. And this involved no computational processing at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The possibilities this opens up are just scary, as you can track pretty much anybody's movements without actually needing someone to follow them. Add to that street-level CCTV, and not much can happen without you being able to find out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this scenario there is no processing at all, but you could imaging simple processing steps, such as tracking vehicles or changes in the environment, which could be used to give clues about potentially 'interesting' events. So instead of using it 'passively' as a kind of memory, you could use that data to identify things that happened that you weren't aware of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is without even any clandestine access to people's data. If you add that dimension, then you might even be able to identify crimes/etc before they even happen. Text processing can be quite fast, but is not easy to do, as presumably few people would openly communicate about things they were planning. So I guess we're still a long way away from that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there is the ethical dimension (which is mentioned in the podcast): who has access to that data, and who decides what it is used for? If you do, and you suspect your partner of being unfaithful, who/what would stop you from checking out their movements? Or check up on that politician who might have a secret affair, or a gambling issue, or who keeps being in the same locations as a well-known drug dealer. All rather scary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While a complete analysis of all such data would be very heavy computationally, and fraught with false positives and recall problems, it might simply be enough to index it by time, location, and perhaps people involved (face recognition seems to be reasonably good, though still with a rather high error rate). This is enough already to make me feel worried about the future.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-02-28T09:36:40.140" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18305" PostTypeId="2" ParentId="18273" CreationDate="2020-02-28T09:43:10.673" Score="0" Body="&lt;p&gt;As said by @brale in the comment below the question:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filters = (classes + 5) * 3&#xA;= (classes + width + height + x + y + confidence) * num&#xA;= (classes + 1+1+1+1+1) * num&#xA;= (classes + 5) * num&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;YOLOv3 dectects &lt;strong&gt;3 boxes per grid cell&lt;/strong&gt;, so it is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filters = (classes + 5) * 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-28T09:43:10.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18306" PostTypeId="1" CreationDate="2020-02-28T10:21:07.997" Score="1" ViewCount="22" Body="&lt;p&gt;I was wondering what is the best method out there to find relationship between two 1D signals so that I can predict/generate one (source) from the other (target). For example, let's say that in response to an event, my sensor A's readings vary in a certain way for 5 seconds. For the same event sensor B's readings vary as well for 5 seconds. Sensor A and B are not measuring the same physical quantities but respond to the same event and seem to have a relationship.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do to use the signal from sensor A to learn how the signal from sensor B would look like for that event? What is the state of the art in deep learning?&lt;/p&gt;&#xA;" OwnerUserId="33872" LastActivityDate="2020-02-28T10:21:07.997" Title="What's the best method to predict/generate signal from one sensor (source) to signal from another another (target)?" Tags="&lt;deep-learning&gt;&lt;time-series&gt;&lt;sequence-modelling&gt;&lt;machine-translation&gt;&lt;signal-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18308" PostTypeId="2" ParentId="17214" CreationDate="2020-02-28T13:05:03.253" Score="0" Body="&lt;p&gt;How many classes do you want to annotate it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can get the bounding boxes and class names coarsely by using the pre-trained models such as YOLO-V3 &lt;a href=&quot;https://github.com/AlexeyAB/darknet&quot; rel=&quot;nofollow noreferrer&quot;&gt;darknet&lt;/a&gt;, SSD and others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, we can load those annotations using the tool &lt;a href=&quot;https://github.com/tzutalin/labelImg&quot; rel=&quot;nofollow noreferrer&quot;&gt;labelImg&lt;/a&gt; and manually correct it. It reduces the lot of work and it is called the Human-AI labeling technique.&lt;/p&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2020-02-28T13:05:03.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18309" PostTypeId="2" ParentId="18227" CreationDate="2020-02-28T14:38:10.367" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Does the choice of optimizer really matter in terms of training the model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would the Adam optimizer be better than the Adadelta optimizer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. (But sometimes, adadelta gives better result. Depends upon the dataset and fine-tune mechanism)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would they all basically be the same?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. Here is the &lt;a href=&quot;https://towardsdatascience.com/adam-latest-trends-in-deep-learning-optimization-6be9a291375c&quot; rel=&quot;nofollow noreferrer&quot;&gt;explanation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would some optimizers be better because they allow most of the weights to achieve their &amp;quot;global&amp;quot; minima?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's not possible to check whether the model achieved global minima or not practically.&lt;/p&gt;&#xA;&lt;p&gt;We can evaluate the model by either over-fitting or under-fitting using the training and validation set, and the generalization of the model with the test set.&lt;/p&gt;&#xA;" OwnerUserId="20151" LastEditorUserId="2444" LastEditDate="2020-09-12T15:16:08.537" LastActivityDate="2020-09-12T15:16:08.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18310" PostTypeId="1" CreationDate="2020-02-28T15:25:48.877" Score="3" ViewCount="317" Body="&lt;p&gt;Imagine that we have a set of heuristic functions &lt;span class=&quot;math-container&quot;&gt;$\{h_i\}_{i=1}^N$&lt;/span&gt;, where each &lt;span class=&quot;math-container&quot;&gt;$h_i$&lt;/span&gt; is both admissible and consistent (monotonic). Is &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^N h_i$&lt;/span&gt; still consistent or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any proof or counterexample to show the contradiction?&lt;/p&gt;&#xA;" OwnerUserId="33875" LastEditorUserId="2444" LastEditDate="2020-02-28T15:46:47.900" LastActivityDate="2020-02-28T16:41:38.610" Title="Is the summation of consistent heuristic functions also consistent?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;heuristics&gt;&lt;admissible-heuristic&gt;&lt;consistent-heuristic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18311" PostTypeId="2" ParentId="17859" CreationDate="2020-02-28T15:40:15.223" Score="1" Body="&lt;p&gt;If you want to evaluate on real thermal image dataset, you can use this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.flir.com/oem/adas/dataset/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thermal Image dataset&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him &quot;my model has a mAP=0.7&quot;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Mean Average Precision is the relevant metric but it's more technical. You can start explaining with False Positives and False &#xA;  Negatives in the predictions. In turn, it leads to Precision and Recall. It mostly depends upon your use case because there will always be a trade-off between them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2020-02-28T15:40:15.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18312" PostTypeId="2" ParentId="18310" CreationDate="2020-02-28T16:33:33.900" Score="5" Body="&lt;p&gt;No, it will not necessary be consistent or admissible. Consider this example, where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the start, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the goal, and the distance between them is 1.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;s --1-- g&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assume that &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; are perfect heuristics. Then &lt;span class=&quot;math-container&quot;&gt;$h_0(s) = 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_1(s) = 1$&lt;/span&gt;. In this case the heuristic is inadmissible because &lt;span class=&quot;math-container&quot;&gt;$h_0(s)+h_1(s) = 2 &amp;gt; d(s, g)$&lt;/span&gt;. Similarly, as an undirected graph the heuristic will be inconsistent because &lt;span class=&quot;math-container&quot;&gt;$|h(s)-h(g)| &amp;gt; d(s, g)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to understand the conditions for the sum of heuristics to be consistent and admissible, I would look at the work on &lt;a href=&quot;http://www.bgu.ac.il/~felner/2004/jairpdb.pdf&quot; rel=&quot;noreferrer&quot;&gt;additive PDB heuristics&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17493" LastEditorUserId="17493" LastEditDate="2020-02-28T16:41:38.610" LastActivityDate="2020-02-28T16:41:38.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18314" PostTypeId="2" ParentId="18233" CreationDate="2020-02-28T18:02:47.790" Score="2" Body="&lt;p&gt;The probabilistic models that represent distributions &lt;em&gt;implicitly&lt;/em&gt; are, for example, the &lt;a href=&quot;https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GANs&lt;/a&gt;. (Goodfellow is one of the authors of the original &lt;a href=&quot;https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN&lt;/a&gt; model).&lt;/p&gt;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1702.08235.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Inference using Implicit Distributions&lt;/a&gt; (2017), the authors write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Implicit distributions are probability models whose &lt;strong&gt;probability density function may be intractable&lt;/strong&gt;, but there is a way to&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;sample&lt;/strong&gt; from them exactly and/or calculate and &lt;strong&gt;approximate expectations&lt;/strong&gt; under them, and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate or &lt;strong&gt;estimate gradients&lt;/strong&gt; of such expectations with respect to model parameters.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Implicit models have been successfully applied to generative modelling in generative adversarial networks (GANs)&lt;/p&gt;&#xA;&lt;p&gt;A popular example of implicit models are stochastic generative networks: samples from a simple distribution - such as uniform or Gaussian - are transformed nonlinearly and non-invertably by a deep neural network&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They also provide a table (table 1) that shows some probabilistic models that use an implicit distribution. Here &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; denotes inference only. VI stands for variational inference.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KvzOw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KvzOw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;See this blog post &lt;a href=&quot;https://www.inference.vc/variational-inference-with-implicit-probabilistic-models-part-1-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Inference using Implicit Models, Part I: Bayesian Logistic Regression&lt;/a&gt;. See also the paper &lt;a href=&quot;https://arxiv.org/pdf/1610.03483.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning in Implicit Generative Models&lt;/a&gt; (2017).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-02-28T18:38:29.150" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18316" PostTypeId="2" ParentId="18303" CreationDate="2020-02-28T19:12:48.863" Score="10" Body="&lt;p&gt;You would also want to consider physical limitations. If you are even &lt;em&gt;storing&lt;/em&gt; 126 yottabyte of data per day, then if we look at the current theoretical densest data storage medium, DNA, at 215 petabytes per gram, we get...&#xA;&lt;span class=&quot;math-container&quot;&gt;${(126 * 10^{24}) \over (215 * 10^{15})} = 586046511$&lt;/span&gt; grams per day&lt;/p&gt;&#xA;&#xA;&lt;p&gt;586046511 g = 586046 kg = 586 Metric Tonnes just for storage.&lt;/p&gt;&#xA;" OwnerUserId="33884" LastEditorUserId="2193" LastEditDate="2020-02-28T19:39:08.860" LastActivityDate="2020-02-28T19:39:08.860" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18317" PostTypeId="1" CreationDate="2020-02-28T21:49:28.523" Score="1" ViewCount="22" Body="&lt;p&gt;Yeah I know, best title ever. Anyway,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to make a neural network which is fed with frames coming from an usb camera. &#xA;Don't wanna be so specific, so I'm just gonna say that the network's goal is to classify human hand gestures, therefore I need to make sure it can effectively learn how the hand moves around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I've no idea about what happens when having 3 channels instead of 1, I only know that (for 3 channels) it does 3 separate convolution operations with the same kernel, resulting actually in 3 separate layers. &#xA;How do this 3 channels affect the network? Does it learn from the movement 3 parallel times, then it mixes toghether this 3 &quot;separate movements&quot;? Do I need to make it single channel to help him detect the hand?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: the text is problably confusing, but that's because I'm confused to, that's why I'm asking.&lt;/p&gt;&#xA;" OwnerUserId="32751" LastActivityDate="2020-02-28T22:05:40.367" Title="How do 3 channels affect a network when detecting human skin (CNN)?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18318" PostTypeId="2" ParentId="18317" CreationDate="2020-02-28T22:05:40.367" Score="1" Body="&lt;p&gt;3 channels will give you more information about the color of the object and surroundings which might be important in certain cases. For example if you want to classify blue cars and red cars then color of the object is very important. Since your problem is to classify hand gestures then color might not be that relevant. You're not very interested in the color of the hand you only care about it's position so there is a good chance that grayscale images might be enough. You should try with grayscale first and if that works good, if not, try with 3 channels.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-02-28T22:05:40.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18319" PostTypeId="1" AcceptedAnswerId="18330" CreationDate="2020-02-28T23:23:45.153" Score="2" ViewCount="47" Body="&lt;p&gt;My GANs is like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train an autoencoder (VAE), get the decoder part and use as Generator&lt;/li&gt;&#xA;&lt;li&gt;Train Discriminator&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After training, do the generation in these steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Call Generator to generate an image&lt;/li&gt;&#xA;&lt;li&gt;Call the Discriminator to classify the image to see whether it's acceptable&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem is that the Discriminator says 'false' a lot, which means the generated image is not useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should the Generator change (update weights) when Discriminator doesn't accept its generated image?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-02-29T14:46:42.063" Title="GANs: Should Generator update weights when Discriminator says false continuously" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;generator&gt;&lt;discriminator&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18320" PostTypeId="1" CreationDate="2020-02-29T03:30:09.807" Score="2" ViewCount="88" Body="&lt;p&gt;From what I have seen, any results involving RL almost always take a massive number of simulations to reach a remotely good policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will any form of RL be viable for real-time systems? &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="32390" LastEditDate="2020-02-29T04:31:10.147" LastActivityDate="2020-02-29T12:35:24.317" Title="Is reinforcement learning suited for real-time systems?" Tags="&lt;reinforcement-learning&gt;&lt;applications&gt;&lt;real-time&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18321" PostTypeId="2" ParentId="18220" CreationDate="2020-02-29T03:47:05.843" Score="2" Body="&lt;p&gt;This may sound counter intuitive but one of the biggest rules of thumb for model capacity in deep learning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;IT SHOULD OVERFIT&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you get a model to overfit, its easier to experiment with regularizations, module replacements, etc. But in general, it gives you a good starting ground.&lt;/p&gt;&#xA;" OwnerUserId="25496" LastActivityDate="2020-02-29T03:47:05.843" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18323" PostTypeId="1" CreationDate="2020-02-29T05:44:14.727" Score="2" ViewCount="19" Body="&lt;p&gt;Assume that I have a candidate selection system to generate product/user pairs for recommendation. Currently, in order to hold a quality bar for the recommended product, we trained a model to optimize for the click of the link, denoting as pClick(product, user) model, the output of the model is a score of (0,1) representing how likely the user will click on the recommended product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For our initial launch product, we set a manually selected threshold, say T for all users. For all users, only when the threshold pass T, we will send user the recommendation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we realize this is not optimal: Some users care less about recommendation quality while some other users have a high bar of recommendation quality. And a personalized threshold, instead of the global T can help us improve the overall relevance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to output the threshold for each user, assume we have training data for each user's activity and user/product attributes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is: How should we model this problem with machine learning? Any reference or papers is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="33901" LastActivityDate="2020-02-29T05:44:14.727" Title="How to model personalized threshold problem with machine learning" Tags="&lt;machine-learning&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18325" PostTypeId="1" CreationDate="2020-02-29T08:43:50.950" Score="2" ViewCount="66" Body="&lt;p&gt;The learning rate in my model is &lt;code&gt;0.00001&lt;/code&gt; and the gradients of the model is within the distribution of &lt;code&gt;[-0.0001, 0.0001]&lt;/code&gt;. Is it normal?&lt;/p&gt;&#xA;" OwnerUserId="8415" LastEditorUserId="2444" LastEditDate="2020-03-01T12:57:11.077" LastActivityDate="2020-03-01T12:57:11.077" Title="Why gradients are so small in deep learning?" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18326" PostTypeId="1" CreationDate="2020-02-29T09:25:40.197" Score="2" ViewCount="49" Body="&lt;p&gt;Can it be possible to train a neural network with no parallel bilingual data? &lt;/p&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="2444" LastEditDate="2020-02-29T12:54:26.650" LastActivityDate="2020-02-29T12:54:26.650" Title="How to do machine translation with no labeled data?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;machine-translation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18327" PostTypeId="2" ParentId="18326" CreationDate="2020-02-29T09:25:40.197" Score="2" Body="&lt;p&gt;In this paper: &lt;a href=&quot;https://arxiv.org/abs/1711.00043&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised Machine Translation Using Monolingual Corpora Only&#xA;&lt;/a&gt; the authors proposed a novel method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4lKWv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4lKWv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively it is an autoencoder, but the Start Of Sentence token is set to be the language type. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other advanced method is to use the pre-training model. In this paper: &lt;a href=&quot;https://arxiv.org/pdf/1901.07291.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross-lingual Language Model Pretraining&lt;/a&gt; researchers proposed an algorithm that utilized the pre-trained multi-lingual trained BERT(with labeled data but we don't need to have a labeled dataset for our task) and the autoencoder mentioned previously. &lt;/p&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2020-02-29T09:25:40.197" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18328" PostTypeId="2" ParentId="18320" CreationDate="2020-02-29T12:35:24.317" Score="1" Body="&lt;p&gt;Short answer: &lt;strong&gt;Yes, it is.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Explanation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement learning can be considered as a online learning. That is, you can train your model with a single data/reward pairs.  As with any online learning algorithm, there are a few things to consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model tends to forget the knowledge gained. To overcome this problem, one can save new data in a circular buffer called history and train the model with a portion of mix of new and old data. This is actually the common way to train an RL model and can be adopted to real-time systems. There are also others &lt;a href=&quot;https://medium.com/datadriveninvestor/online-learning-adapting-to-a-changing-environment-without-catastrophically-forgetting-the-past-fdf2f7157391&quot; rel=&quot;nofollow noreferrer&quot;&gt;techniques&lt;/a&gt; to overcome it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem is that if only one data point is fed to the network, it will be impossible to apply some techniques, such as Batch normalization.&lt;/p&gt;&#xA;" OwnerUserId="12841" LastActivityDate="2020-02-29T12:35:24.317" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18329" PostTypeId="2" ParentId="18189" CreationDate="2020-02-29T14:12:07.077" Score="1" Body="&lt;p&gt;You can remove the genes, of course, if it makes sense in your implementation. I can see some reasons why you might want to do it:   &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Your genome's size is very high&lt;/strong&gt;, so performing some pruning of disabled genes might speed up the algorithm&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;You do not want to re-enable the genes in the future&lt;/strong&gt; (maybe you have a special mutation mechanism that might add connections instead of genes or simply when it is disable it stays like that)    &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As a rule of thumbs, keeping connections is usually a good thing if the algorithm can handle big genomes. After all you might re-enable them in the future (these connections are safer to enable than creating new random ones,as the latter might create unwanted cycles, so you'd have to check for that) and dense ANNs work better than sparse ones and pruning too much might not play at your favor. &lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2020-02-29T14:12:07.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18330" PostTypeId="2" ParentId="18319" CreationDate="2020-02-29T14:46:42.063" Score="1" Body="&lt;p&gt;In general, you should train both discriminator D and generator G simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the metric that you use as the target for your model, you may encounter a Vanishing gradient problem. It can happen when you implement original loss (i.e. JS-divergence). In that case D can become overconfident regarding fake samples and won't provide any useful feedback to the G. To find out if training fell into this problem, you should plot D and G loss. It will look as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rbUWW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rbUWW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original GAN has a lot of problems, that's why I suggest you to use Wasserstein metric. More information you can find in &lt;a href=&quot;https://arxiv.org/pdf/1701.07875.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGAN paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you can find more information about GAN problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1904.08994.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1904.08994.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1804.00140.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1804.00140.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12841" LastActivityDate="2020-02-29T14:46:42.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18331" PostTypeId="2" ParentId="18303" CreationDate="2020-02-29T16:03:01.187" Score="8" Body="&lt;p&gt;The answer is really very simple. If you have the dystopian power over all the mobile devices in the first place, you would not make them send all their data over to any &quot;global data storage&quot; just like that. Instead, you would have put a local AI on each device that filters, processes, categorizes and flags the important parts, sending only those parts plus an intelligent summary of the remaining data to a global AI. The global AI combines and synthesizes the parts that all the local AIs send to it, and may request further data from the local AIs based on what it wants to know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naturally, since you are a competent dystopia architect, you design each local AI to be intelligent enough to subvert any human's attempt to remove it, stop its activity, or otherwise interfere with its data collection and processing. The local AIs also continuously communicate in a distributed network with other local AIs regarding their status and any adversarial activities, so that they can quickly act to defend themselves if the need arises, and also notify the global AI of any attack. In this surveillance state, it is an easy task for the global AI to send armed agents to deal with any threat to the AI network that manages to gain any foothold in the information cyberspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that the most durable dystopia is a &lt;strong&gt;defended distributed dystopia&lt;/strong&gt;, which would make it &lt;strong&gt;robust&lt;/strong&gt; and &lt;strong&gt;scalable&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33916" LastActivityDate="2020-02-29T16:03:01.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18332" PostTypeId="1" AcceptedAnswerId="18333" CreationDate="2020-02-29T17:06:49.857" Score="1" ViewCount="103" Body="&lt;p&gt;I'm watching the David Silver series on YT which has raised a couple of questions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In the Markov process (or chain), how are the directions to each successive state defined?&lt;/strong&gt; For example, how are the arrow directions defined for the MP below? What's stopping our sample episodes from choosing &lt;code&gt;A -&amp;gt; C -&amp;gt; D -&amp;gt; F&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PbKgcm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PbKgcm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, &lt;strong&gt;how is the probability transition matrix populated?&lt;/strong&gt; From David's example, the probabilities seem to have already been set. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QJoawm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QJoawm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-02-29T17:59:24.123" LastActivityDate="2020-02-29T21:18:06.833" Title="In the Markov chain, how are the directions to each successive state defined?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;markov-chain&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18333" PostTypeId="2" ParentId="18332" CreationDate="2020-02-29T17:38:30.027" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;It's not possible, as in the chain illustrated there are no transitions between A and C, C and D, and D and F. Only sequences where transitions exist are possible. The choice of transitions is arbitrary; it depends on what you want to model with it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As DuttaA says in his comment, you can imagine that all nodes are linked with all other nodes, but those links have a transition probability of 0.0; so the probability of observing the sequence ACDF is actually 0.0 as well. In order to make the diagram more easy to understand, zero-probability transitions are generally not shown.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;There is no prescribed method of populating the transition probabilities. You can define the probabilities manually, randomly, or however you like. You could derive them from observable training data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Re updating: this depends on the application. If you are modelling a process with known probabilities, you would not update the probabilities; if you are modelling a dynamic process which changes over time, then this is something you might want to consider. However, from my own experience (HMMs in speech recognition), once they have been assigned, they are generally kept as they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov chain&lt;/a&gt; is really a fairly basic model; it gets more complex with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hidden Markov Model&lt;/a&gt;, where you would generally use a learning algorithm to assign the transition and emission probabilities.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-02-29T21:18:06.833" LastActivityDate="2020-02-29T21:18:06.833" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18334" PostTypeId="1" CreationDate="2020-02-29T19:20:08.860" Score="1" ViewCount="37" Body="&lt;p&gt;This question could be generalised to how to adapt state-of-the-art object detection models to large images with small ROIs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my particular case I'm trying to use &lt;a href=&quot;https://github.com/ipazc/mtcnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;this implementation&lt;/a&gt; of &lt;a href=&quot;https://arxiv.org/pdf/1604.02878.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MTCNN&lt;/a&gt; to get bounding boxes for the faces of images of statues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One challenge is that the face could take up a large proportion of the image like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fg1jM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fg1jM.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or a very small proportion of the image like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PwsZR.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PwsZR.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I'll zoom in on the statue's face so you can see it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lXUbs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lXUbs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has additional comments on my overall approach to this particular problem, happy to hear them.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-02-29T19:20:08.860" Title="How to adapt MTCNN to large images with relatively small ROIs" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18335" PostTypeId="2" ParentId="18204" CreationDate="2020-02-29T19:28:07.410" Score="1" Body="&lt;p&gt;&lt;em&gt;This is a tentative answer, and I might come back to it at some point in time. As @nbro mentions this question seems to be opinion based, so my answers are also just my opinion&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by AGI you mean &quot;super-intelligent&quot;, then any of the following results should be sufficient to convince anyone of its being &quot;smarter&quot; than him/her/pronoun:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Resolving important mathematical problems (the most famous examples being the &lt;a href=&quot;https://en.wikipedia.org/wiki/Millennium_Prize_Problems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Millennium Problems&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Collatz_conjecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;Collatz Conjecture&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Goldbach%27s_conjecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goldbach's Conjecture&lt;/a&gt;. (Corollary: Break all known encryption schemes)&lt;/li&gt;&#xA;&lt;li&gt;Founding a new &quot;system&quot; to supersede &lt;a href=&quot;https://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZFC&lt;/a&gt; as the new &lt;a href=&quot;https://en.wikipedia.org/wiki/Foundations_of_mathematics&quot; rel=&quot;nofollow noreferrer&quot;&gt;foundation of mathematics.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;New discoveries in the natural sciences (physics, chemistry, biology...)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(1) is a bit dubious as a criterion: at least with modern techniques, &lt;a href=&quot;https://en.wikipedia.org/wiki/Automated_theorem_proving&quot; rel=&quot;nofollow noreferrer&quot;&gt;automated theorem proving&lt;/a&gt; is either just &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;symbol pushing&lt;/a&gt;&quot; or requires so much human intervention (in the design/construction to solve a particular problem) that it would be hard to imagine it as being &quot;smart&quot; in the traditional sense. We already have a few cases where an automated theorem prover solved big problems (&lt;a href=&quot;https://en.wikipedia.org/wiki/Four_color_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;four-colour theorem&lt;/a&gt; being the most notable). Point being that even if we reach this with methods similar to what we have already, people might be resistant to call it &quot;smart&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) is hard to imagine ever being plausible. To the extent that this &quot;AGI-thing&quot; is implemented on a system that &quot;does math&quot;, it would be unusual to imagine a system that can move beyond itself to recognize a new, &quot;better&quot; system of math. As an analogy, it might be like a formal system trying to prove its own consistency in a &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Godelian sense&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the analogy is weak, and I don't see a strong/rigour reason for doubting that an AGI can discover a new axiomatic system. One might even fathom that said AGI can create a new system from the &quot;bottom up&quot;, much like string theory was constructed from the &quot;bottom up&quot; to &quot;explain&quot; relativity and particle physics. Perhaps then we can have &quot;proper resolutions&quot; on questions like the continuum hypothesis, much like how the parallel postulate was discarded to give way to non-euclidean geometry. But I also cast doubt that there will ever be a &quot;final word&quot; on math itself, so its just a fun idea for now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) is also dubious to imagine if it would ever become true. The study of the natural sciences would require a physical presence in the world that goes beyond seeking &quot;beauty in the mathematical equations&quot; that would be unusual for an AGI to have. That being said, an AGI could have cameras and other sensors to interpret the natural world, so its not something that I think is strictly impossible. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If by AGI you mean &quot;human-ness&quot;, then I don't think any single result can convince everyone at the world at the same time of its being an AGI. Perhaps this &quot;convincing the world that &quot;me is AGI&quot; work&quot; can be done on a person to person basis, in the sense that the AGI would need to interact with each person and slowly build up a certain degree of trust. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under this interpretation, there can be no complete list that describes AGI, so what follows is just my own list of things I think an human-like AGI might be able to do.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create and interpret art.&lt;/li&gt;&#xA;&lt;li&gt;Have common sense.&lt;/li&gt;&#xA;&lt;li&gt;Be &quot;creative&quot;&lt;/li&gt;&#xA;&lt;li&gt;Hold a meaningful conversation, understanding others and making sense.&lt;/li&gt;&#xA;&lt;li&gt;Able to perform / to receive a psychoanalysis; understanding of &lt;a href=&quot;https://plato.stanford.edu/entries/folkpsych-theory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;folk-psychology.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Exist in a physical manifestation (like a robot) with social/environmental appropriateness.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The main issue with the above criteria is that they are all subjective. Like I said above, this set of criteria probably works on a case-to-case basis&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;These criteria seems to be the most important of all, but at the same time the definition of verification of these terms is epistemically tricky, so I'll leave them open.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Learning (Is a species evolving over time learning its environment?)&lt;/li&gt;&#xA;&lt;li&gt;Self-replication (Is a meme / virus intelligent?)&lt;/li&gt;&#xA;&lt;li&gt;Self-awareness (Is &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Treachery_of_Images&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Treachery of Images&lt;/a&gt; self-aware?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-03-04T05:00:32.960" LastActivityDate="2020-03-04T05:00:32.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18336" PostTypeId="1" AcceptedAnswerId="18346" CreationDate="2020-03-01T01:50:43.643" Score="1" ViewCount="43" Body="&lt;p&gt;If I have the Gaussian kernel&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k(x, x') = \operatorname{exp}\left( -\| x - x' \|^2 / 2\sigma^2 \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; in the context of training an SVM?&lt;/p&gt;&#xA;" OwnerUserId="29877" LastEditorUserId="2444" LastEditDate="2020-03-01T02:00:34.490" LastActivityDate="2020-03-01T13:26:45.127" Title="What are the variables used in a Gaussian radial basis kernel in the context of SVMs?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18337" PostTypeId="1" CreationDate="2020-03-01T06:35:48.837" Score="2" ViewCount="29" Body="&lt;p&gt;I have been reading quite a lot about the research progress in the domain of self attention-based neural networks that were introduced by Google Inc. in their paper titled &lt;a href=&quot;http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Attention is all you need&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The concept of introducing attention to neural networks in order to free ourselves from a strict context vector being really unique on one hand and moreover using the same concept to model sequences without recurrent neural networks as introduced in the paper is extremely elegant.&lt;/p&gt;&#xA;&lt;p&gt;I have been trying to figure out so has to how this concept of attention would aid deep networks that model multi-agent systems in which game-theoretic factors come into play for the network to learn.&lt;/p&gt;&#xA;&lt;p&gt;I was looking for some direction or a toy example/explanation or even possible previous research done to try to test these concepts together.&lt;/p&gt;&#xA;&lt;p&gt;P.S - I'm just tinkering with some ideas hoping to build something experimental.&lt;/p&gt;&#xA;" OwnerUserId="33929" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-01T06:35:48.837" Title="Impact or applications of introducing attention in deep networks modelling multi-agent systems" Tags="&lt;neural-networks&gt;&lt;game-theory&gt;&lt;attention&gt;&lt;soft-question&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18338" PostTypeId="2" ParentId="8554" CreationDate="2020-03-01T07:11:09.443" Score="2" Body="&lt;p&gt;At first, like Neil Slater says, I thought this could only be solved using the expected rewards instead of actual rewards, or else there wasn't enough information to solve it. But now I think there might be a way to solve this question. Here is my thinking on this problem (I would be curious for anyone's thoughts, as I am working through this book myself).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the key part is where the book says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Each can collected by the robot counts as a unit reward, whereas a reward of &lt;span class=&quot;math-container&quot;&gt;$-3$&lt;/span&gt; results whenever the robot has to be rescued.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This means that the reward set is actually &lt;span class=&quot;math-container&quot;&gt;$\mathcal R = \{0, 1, -3\}$&lt;/span&gt; (we assume that in each timestep, the robot can only collect one can).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now using &lt;span class=&quot;math-container&quot;&gt;$$r(s,a,s') = \sum_r r \frac{p(s',r\mid s,a)}{p(s'\mid s,a)} \tag{3.6}$$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$$p(s'\mid s,a) = \sum_r p(s',r\mid s,a)\tag{3.4}$$&lt;/span&gt; it seems possible to solve for all the probabilities. I'll do an example for &lt;span class=&quot;math-container&quot;&gt;$(s,a,s') = (\mathtt{high}, \mathtt{search}, \mathtt{high})$&lt;/span&gt; and leave the rest to you (I haven't actually done the rest, since this does seem rather tedious).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation 3.6 gives &lt;span class=&quot;math-container&quot;&gt;$$r_\mathtt{search} = 0\cdot \frac{p(s', 0 \mid s,a)}{\alpha} + 1\cdot \frac{p(s', 1 \mid s,a)}{\alpha} -3\cdot \frac{p(s', -3 \mid s,a)}{\alpha}$$&lt;/span&gt; Since &lt;span class=&quot;math-container&quot;&gt;$p(s', -3 \mid s,a) = 0$&lt;/span&gt; (it's impossible for the robot to have to be rescued, since we started in the &quot;high&quot; state), we get &lt;span class=&quot;math-container&quot;&gt;$p(s', 1 \mid s,a) = \alpha r_\mathtt{search}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now equation 3.4 gives &lt;span class=&quot;math-container&quot;&gt;$$\alpha = p(s', 0 \mid s,a) + p(s', 1 \mid s,a) + p(s', -3 \mid s,a)$$&lt;/span&gt; which solves to &lt;span class=&quot;math-container&quot;&gt;$p(s', 0 \mid s,a) = \alpha - \alpha r_\mathtt{search}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the first two rows of the table will look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{cccc|c}&#xA;s&amp;amp; a &amp;amp; s' &amp;amp; r &amp;amp; p(s',r\mid s,a)\\ \hline&#xA;\mathtt{high} &amp;amp; \mathtt{search} &amp;amp; \mathtt{high} &amp;amp; 1 &amp;amp; \alpha r_\mathtt{search}\\&#xA;\mathtt{high} &amp;amp; \mathtt{search} &amp;amp; \mathtt{high} &amp;amp; 0 &amp;amp; \alpha (1- r_\mathtt{search})&#xA;\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="33930" LastActivityDate="2020-03-01T07:11:09.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18339" PostTypeId="1" AcceptedAnswerId="18364" CreationDate="2020-03-01T07:14:41.703" Score="1" ViewCount="257" Body="&lt;p&gt;I've just started with AI and CNN networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two NIFTI images dataset, one with (240, 240) dimensions and the other one with (256, 132). Each dataset is front a different hospital and machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to use both to train my model. What do I have to do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model needs to have all the train data with the same shape. I've thought to reshape all the data to have the same shape, but I don't know if I'm going to lose information if I reshape the images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, I have also a third dataset with (232, 256).&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2020-03-01T16:58:38.393" LastActivityDate="2020-03-02T18:36:31.737" Title="Using three image datasets with different image sizes to train a CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18341" PostTypeId="1" CreationDate="2020-03-01T08:59:35.127" Score="1" ViewCount="53" Body="&lt;p&gt;&lt;a href=&quot;https://afia.asso.fr/journee-hommage-j-pitrat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://afia.asso.fr/journee-hommage-j-pitrat/&lt;/a&gt; is a seminar on March 6th, 2020, in Paris (France, European Union), in honor of the late &lt;a href=&quot;https://fr.wikipedia.org/wiki/Jacques_Pitrat&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jacques Pitrat&lt;/a&gt;, who advocated during all his professional life a meta-knowledge and reflective approach. (You need to register to attend that seminar).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pitrat's blog is available (in spring 2020) on &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bootstrappingartificialintelligence.fr/WordPress3/&lt;/a&gt; (and some snapshot of his &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/?s=CAIA&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAIA&lt;/a&gt; system is downloadable &lt;a href=&quot;http://starynkevitch.net/Basile/caia-su-24feb2016.tar.bz2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; - but no documentation; however you might try to type &lt;code&gt;L EDITE&lt;/code&gt; on stdin to &lt;code&gt;caia&lt;/code&gt;). He wrote the &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470611791&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings : the conscience of a conscious machine&lt;/em&gt;&lt;/a&gt; book describing the software architecture of, and the motivations for (some previous version of) CAIA. See also this &lt;a href=&quot;https://pdfs.semanticscholar.org/2117/9600b3f05c0af399f9acbfc6e7b6d24daf03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Step toward an Artificial Artificial Intelligence Scientist&lt;/em&gt;&lt;/a&gt; paper by J.Pitrat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What AI conferences (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI&lt;/a&gt; workshops) in Europe should I consider submitting papers to explaining the ongoing work on &lt;a href=&quot;http://refpersys.%20org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; project (open source, open science, work-in-progress, with contact information) is explicitly following J.Pitrat's meta-knowledge approach. Feel free to follow or join that ambitious open-source (actually &lt;a href=&quot;https://www.gnu.org/philosophy/free-sw.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;free software&lt;/a&gt;) project.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-03-01T17:55:25.330" LastActivityDate="2020-05-09T20:04:23.237" Title="What AI conferences in Europe should I consider submitting papers to explaining the ongoing work on RefPerSys?" Tags="&lt;ai-design&gt;&lt;research&gt;&lt;papers&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18343" PostTypeId="1" CreationDate="2020-03-01T10:18:51.020" Score="1" ViewCount="36" Body="&lt;p&gt;Consider a prediction problem for example. This is a loss function (negative log likelihood) that I am roughly talking about:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;    J_{\text{train}} &amp;amp;= -\sum_t \log L\left(\theta_t, X_t, Y_t\right) \\&#xA;    J_{\text{test}} &amp;amp;= -\sum_t \log L\left(\theta_t, X_{t+1}, Y_{t+1}\right) \\&#xA;    J_{\text{dyn}} &amp;amp;= - \sum_t \frac{\left(\theta_{t+1} - \text{stop_gradient}({\theta_t})\right)^2}{2 \sigma_{\theta}^2} \\&#xA;    \theta_0 &amp;amp;\sim \Psi&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the dynamics are simply first-order smoothness (Brownian motion) in the time-dependent parameters &lt;span class=&quot;math-container&quot;&gt;$\text{d}\theta_t = \sigma_{\theta} \text{d} W_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is basically what they do in weather systems and in papers like &lt;a href=&quot;http://www.hairer.org/papers/bayesian.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Bayesian Approach to Data Assimilation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference is, we now have the ability to use stop gradients in the smoothness, which changes the problem somewhat. Other issues might arise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Who else is doing this? Feel free to also provide links to papers on related work. I am not finding enough hits, so maybe I am missing something. Are RNNs, if twisted slightly, falling into this framework and I am no seeing it?&lt;/p&gt;&#xA;" OwnerUserId="23001" LastEditorUserId="23001" LastEditDate="2020-03-01T15:45:32.523" LastActivityDate="2020-03-01T15:45:32.523" Title="Where can I find people solving this smoothing, filtering, temporal learning problem?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18344" PostTypeId="1" CreationDate="2020-03-01T10:55:12.227" Score="1" ViewCount="64" Body="&lt;p&gt;Following on from &lt;a href=&quot;https://ai.stackexchange.com/q/18332/2444&quot;&gt;my other (answered) question&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to the Markov process (chain), if an environment is a board game and its states are the various position the game pieces may be in, how would the transition probability matrix be initialised? How would it be (if it is?) updated?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-03-01T12:45:52.957" LastActivityDate="2021-03-28T03:08:05.843" Title="How is the probability transition matrix populated in the Markov process (chain) for a board game?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;markov-chain&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18345" PostTypeId="2" ParentId="11014" CreationDate="2020-03-01T11:16:33.270" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In many implementations of convolutional layers, the filters do not neatly remain inside the features plane when &amp;quot;sliding&amp;quot; along it, but (conceptually) also partially go &amp;quot;outside&amp;quot; the plane (where always at least one &amp;quot;cell&amp;quot; of the filter will still be inside the plane). Intuitively, in the case of a &lt;span class=&quot;math-container&quot;&gt;$3\times3$&lt;/span&gt; filter for example, you can imagine that we pad the input features with an extra border of size &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, and this padding around the &amp;quot;real&amp;quot; input planes is filled with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s.&lt;/p&gt;&#xA;&lt;p&gt;If it's possible for all input features to also have values of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, it may in some situations be difficult or impossible for the neural network to distinguish the &amp;quot;real&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; inputs from the &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; entries in the padding around the board, i.e. it may struggle to know where the game board is and where the game board ends. Having a constant input plane that's always filled with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;s can help in this respect, because that plane can always reliably be used to distinguish &amp;quot;real&amp;quot; cells that actually exist on the game board from positions that are just outside the game board.&lt;/p&gt;&#xA;&lt;p&gt;As for the plane filled with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s... I have no idea why that would ever be useful. Maybe it was useful due to some peculiar implementation detail. In &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/553jez/has_deepmind_released_their_alphago_network/d878p8u/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this thread&lt;/a&gt;, some people hypothesise that on specific hardware it might make some computations slightly more efficient because of the layout of the data in memory -- it causes the number of channels to be divisible by &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt;, which will.. maybe help? I really don't know too much about this, but I do know that on a smaller scale, &lt;a href=&quot;https://software.intel.com/en-us/articles/coding-for-performance-data-alignment-and-structures&quot; rel=&quot;nofollow noreferrer&quot;&gt;sometimes adding unused data in classes can indeed increase performance due to better layout in memory&lt;/a&gt;. I suppose it's also possible that it was accidentally added by mistake, or &amp;quot;just in case&amp;quot; and that it doesn't really have much of a purpose. The amount of hardware that the AlphaGo team had available to them is quite insane anyway, one channel more or less probably wasn't too big of a concern for them.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What's more, I don't understand the sharp sign here. Does it mean &amp;quot;the number&amp;quot;? But one number is enough to represent &amp;quot;the number of turns since a move was played&amp;quot;, why eight?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is explained in the &amp;quot;&lt;strong&gt;Features for policy/value network&lt;/strong&gt;&amp;quot; paragraph in the paper. Quote from the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Each integer feature value is split into multiple &lt;span class=&quot;math-container&quot;&gt;$19 \times 19$&lt;/span&gt;&amp;quot; planes of binary values (one-hot encoding). For example, separate binary feature planes ares used to represent whether an intersection has &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; liberty, &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; liberties, &lt;span class=&quot;math-container&quot;&gt;$\dots$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\geq 8$&lt;/span&gt; liberties.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All feature planes used were strictly binary, no feature planes were used that had integer values &lt;span class=&quot;math-container&quot;&gt;$&amp;gt; 1$&lt;/span&gt;. This is quite common when possible, because neural networks tend to have a much easier time learning with binary features than with integer- or real-valued features.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastEditorUserId="1641" LastEditDate="2021-01-05T20:30:43.180" LastActivityDate="2021-01-05T20:30:43.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18346" PostTypeId="2" ParentId="18336" CreationDate="2020-03-01T13:21:29.247" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \in \mathbb{R}^p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}' \in \mathbb{R}^p$&lt;/span&gt; are two inputs (or feature vectors). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of classification with an SVM, you are given a dataset &lt;span class=&quot;math-container&quot;&gt;$D = \{(\mathbf{x}_i, y_i) \}_{i=1}^N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i \in \mathbb{R}^p$&lt;/span&gt; is an input (or point) and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; the corresponding label. The goal is to find a hyperplane that classifies the points &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt;. The hyperplane actually corresponds to a binary classifier that splits the plane into two, so the assumption is that there are two labels. However, these points &lt;span class=&quot;math-container&quot;&gt;$ \mathbf{x}_i$&lt;/span&gt; may not be linearly separable in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^p$&lt;/span&gt;, i.e. there may not be a hyperplane (in 2d, i.e. when &lt;span class=&quot;math-container&quot;&gt;$p=2$&lt;/span&gt;, a hyperplane is a line) that separates them. The kernel trick, i.e. the use of kernels (such as the Gaussian radial basis), allows an SVM to perform non-linear classification by transforming the inputs to a space where they are linearly separable.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-01T13:26:45.127" LastActivityDate="2020-03-01T13:26:45.127" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18347" PostTypeId="2" ParentId="18290" CreationDate="2020-03-01T14:40:54.003" Score="0" Body="&lt;p&gt;In order to answer the question requires that the definition of intelligence (human or otherwise) be grounded into the physical model.  There are two forms of intelligence in the physical model, one subconscious (general) and the other conscious (specific).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The subconscious is responsible for managing the brain’s process and performing all the physical reactions.  It exist in a deterministic universe where the decisions are known and reactions automated.  In the brain, the shape of the deterministic universe is time.  For something to be deterministic, it must have guaranteed results within a specific time cycle.  This requirement precludes the use of reasoning and long-term memory whose completion time cannot be fixed.  The subconscious runs on general intelligence.  To understand the format of general intelligence, think in terms of a jigsaw puzzle.  General intelligence only needs the border of the puzzle.  With just the basic object structure, the subconscious can move and process data throughout the brain without regards to its content.  It is functionally agnostic and can complete its automation cycle without reasoning.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consciousness is responsible for mapping the non-deterministic environment to the deterministic universe.  To do this, it must exist outside of that universe so its own non-deterministic nature doesn’t interfere with the subconscious process.  Consciousness runs on specific intelligence.  Its job is to decipher the contents of the puzzle.  In humans, specific intelligence takes on the form of visual symbols.  These symbols are used by the conscious state for reasoning and long-term memory access.  How decisions are reached has no predefined form or technique.  It is a non-deterministic process based on the accumulation of individual experience and previous selection.  This process allows the biological life-form to adapt decision-making to its own environment.  In the most basic sense, the subconscious allows us to live in a deterministic universe and consciousness allows us to adapt to a non-deterministic environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two forms of intelligence are functionally incompatible. The non-deterministic environment has no common translation to a deterministic universe.  If it did, it wouldn’t be non-deterministic.  To overcome the translation problem, the lower occipital lobe is a functional “Black Box” that allows the two different formats to be written into the same memory construct.  By simple association, the two systems coexist in a non-invasive relationship where the hippocampus (short-term) memory serves as an interchange point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a grounded physical model, I can now attempt to answer some of the questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We have “free will”, but it can only exist in the non-deterministic environment of the conscious state.  There is no such thing as “free will” in a deterministic universe.  The results of that universe are already known and decided.&lt;/li&gt;&#xA;&lt;li&gt;A deterministic universe cannot change.  Its own nature precludes&#xA;adaption from within.  To make alterations requires a separate state&#xA;that can rise above the process in order to change the process. &#xA;Biological consciousness serves this role and requires “free will”&#xA;to make decisions outside of the scope of what is already known.&lt;/li&gt;&#xA;&lt;li&gt;There are no ethics in a deterministic universe.  Abstracting right and wrong is a reasoning function which is not allowed.  AI constructs like agents which do pattern matching to develop predictions/reactions are not directly attached to any conscious state.  They are deterministic functions whose capability is predefined. Simply put, agents cannot exceed the sum of their programming because they cannot adapt to that which is not known or anticipated.&lt;/li&gt;&#xA;&lt;li&gt;Liability only exists where “free will” exists and does not exist in the deterministic universe.  Otherwise, you are attempting to blame your hand for a decision originated in your frontal lobe.&lt;/li&gt;&#xA;&lt;li&gt;“Free will” becomes a prisoner of the deterministic universe.  As experience and decision knowledge grow, more and more decisions are shifted to the subconscious.  This creates a dependency that restricts the range of “free will” and the decisions it is called upon to resolve. &lt;/li&gt;&#xA;&lt;li&gt;Creativity does not exist in the deterministic universe.  It is an essential skill used by the conscious state to employ related memory to formulate new decisions.  Creativity is used by animals to develop survival skills to adapt to environmental changes.  The production of art work in an esoteric sense is purely human, but as a basic function is not.&lt;/li&gt;&#xA;&lt;li&gt;Intent does not exist in the deterministic universe.  Within the non-deterministic universe of consciousness, intent is buried in an ocean of previous experience that may or may not reflect current “free will” goals.  Especially, if these goals threaten the survival of the biological process.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33881" LastActivityDate="2020-03-01T14:40:54.003" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18350" PostTypeId="1" AcceptedAnswerId="18351" CreationDate="2020-03-01T19:12:16.580" Score="2" ViewCount="46" Body="&lt;p&gt;I understand that Experience Replay is used for data efficiency reasons and to remove correlations in sequences of data. How exactly do these sequences of correlated data affect the performance of the algorithm?&lt;/p&gt;&#xA;" OwnerUserId="33227" LastActivityDate="2020-03-01T20:30:05.127" Title="Intutitive explanation of why Experience Replay is used in a Deep Q Network?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18351" PostTypeId="2" ParentId="18350" CreationDate="2020-03-01T20:30:05.127" Score="2" Body="&lt;p&gt;It is the neural network approximation that suffers, when it attempts to learn from correlated data. Intuitively, this is because the learning algorithm takes gradient steps assuming that the examples it are shown are &lt;em&gt;representative&lt;/em&gt; of the dataset as a whole. A neural network update step uses a mini-batch of examples to calculate the gradient of its weights and biases with respect to a cost function. If that mini-batch is not fairly sampled, then the expected value of the loss and of the gradient will not be representative of the population as a whole. They will be biased and can cause an update step to parameters in the wrong direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, if you are using a bootstrapping method - any form of TD learning - then the value estimates used to set learning update targets are sensitive to bias in the estimator. This is already something that can cause instability due to positive feedback loops. Adding another source of systemic bias from correlated input data can only make it worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can gain some experience for the effect of this with a simple non-RL experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt; To approximate the function &lt;span class=&quot;math-container&quot;&gt;$y = x^2$&lt;/span&gt; in the range &lt;span class=&quot;math-container&quot;&gt;$-2 &amp;lt; x &amp;lt; 2$&lt;/span&gt;. These numbers chosen to make the task simple for a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Setup:&lt;/strong&gt; Generate training data in the form &lt;span class=&quot;math-container&quot;&gt;$x_i, y_i$&lt;/span&gt; for a few thousand sample points with true values of the function. Optionally add some noise to make the task harder. Keep the data set ordered by &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; values. Create a simple neural network for regression (e.g. 2 hidden layers with 50 neurons and tanh activation), and set it up with a simple optimiser (e.g. SGD)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Run:&lt;/strong&gt; Train the network twice, using small minibatches (e.g. size 10). Once without shuffling the data on each epoch (or using any shuffling algorithm on the minibatches), and once with standard shuffling and assignment to minibatches. Plot a learning curve of loss vs epoch for each run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should find that the ordered, non-shuffled version learns much slower than when using some randomisation to decorrelate the data. Depending on precise hyperparameters, the ordered version may even oscillate with loss quite far away from converging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is this effect, or a more subtle version of it, that impacts RL combined with neural networks when learning direct from trajectories. In addition, there is existing starting bias in bootstrap methods, which this effect makes worse.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-03-01T20:30:05.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18352" PostTypeId="2" ParentId="17050" CreationDate="2020-03-01T20:31:56.200" Score="5" Body="&lt;p&gt;There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as well as LSTMs (RNNs which are Turing-complete) are not able to model simple arithmetic operations (e.g. addition/multiplication), they designed a new logic unit which can solve all of the simple arithmetic problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://arxiv.org/abs/1808.00508&quot; rel=&quot;noreferrer&quot;&gt;Neural Arithmetic Logic Units&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, DL can solve symbolic maths: &lt;a href=&quot;https://arxiv.org/abs/1912.01412&quot; rel=&quot;noreferrer&quot;&gt;Deep Learning for Symbolic Mathematics&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28608" LastActivityDate="2020-03-01T20:31:56.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18353" PostTypeId="1" CreationDate="2020-03-01T21:42:15.223" Score="5" ViewCount="910" Body="&lt;p&gt;Why is a batch size needed to update the weights of a neural network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to that &lt;a href=&quot;https://www.youtube.com/watch?v=tIeHLnjs5U8&amp;amp;t=278s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Youtube Video from 3B1B&lt;/a&gt;, the weights are updated by calculating the error between expectation and outcome of the neural net. Based on that, the chain rule is applied to calculate the new weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following that logic, why would I pass a complete batch through the net? The first entries wouldn't have an impact on the weighting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to define a batch size when I use backpropagation?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2021-03-23T09:51:52.007" LastActivityDate="2021-03-23T12:58:52.587" Title="What is the purpose of the batch size in neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;batch-size&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18354" PostTypeId="2" ParentId="18353" CreationDate="2020-03-01T23:01:52.563" Score="5" Body="&lt;p&gt;&lt;strong&gt;tl;dr:&lt;/strong&gt; A batch size is the number of samples a network sees before updating its gradients. This number can range from a single sample to the whole training set. Empirically, there is a sweet spot in the range 1 to a few hundreds, where people experience the fastest training speeds. Check &lt;a href=&quot;https://arxiv.org/pdf/1206.5533.pdf&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;A more detailed explanation...&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you have a small enough number of samples, you can let the network see all of the samples before updating its weights; this is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;noreferrer&quot;&gt;Gradient Descent&lt;/a&gt;. The benefit from this is that you guarantee that the weights will be updated in the direction that reduces the training loss for the whole dataset. The downside is that it is computationally expensive and in most cases infeasible for deep neural nets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is done in practice is that the network sees only a batch of the training data, instead of the whole dataset, before updating its weights. However, this technique does not guarantee that the network updates its weights in a way that will reduce the dataset's training loss; instead it reduces the batch's training loss, which might not the same thing. This adds noise to the training process, which can in some cases be a good thing, but requires the network to take too many steps to converge (this isn't a problem since each step is much faster).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you're saying is essentially training the network each time on a single sample. This is formally called &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&quot; rel=&quot;noreferrer&quot;&gt;Stochastic Gradient Descent&lt;/a&gt;, however the term is used more broadly to include any case where the network is trained on a subset of the whole training set. The problem with this approach is that it adds too much noise to the training process, causing it to require a lot more steps to actually converge.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-03-01T23:01:52.563" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18355" PostTypeId="1" CreationDate="2020-03-02T00:49:07.303" Score="2" ViewCount="473" Body="&lt;p&gt;Consider the following question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; vehicles occupy squares &lt;span class=&quot;math-container&quot;&gt;$(1, 1)$&lt;/span&gt; through &lt;span class=&quot;math-container&quot;&gt;$(n, 1)$&lt;/span&gt; (i.e., the bottom row) of an &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; grid. The vehicles must be moved to the top row but in reverse order; so the vehicle &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; that starts in &lt;span class=&quot;math-container&quot;&gt;$(i, 1)$&lt;/span&gt; must end up in &lt;span class=&quot;math-container&quot;&gt;$(n − i + 1, n)$&lt;/span&gt;. On each time step, every one of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; vehicles can move one square up, down, left, or right, or stay put; but if a vehicle stays put, one other adjacent vehicle (but not more than one) can hop over it. Two vehicles cannot occupy the same square.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Suppose that each heuristic function &lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$h_i$&lt;/span&gt;&lt;/em&gt; is both admissible and consistent. Now what I want to know is to check the admissibility and consistency of the following heuristics:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h= \Sigma_i h_i$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h= \min_i (h_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h= \max_i (h_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{\Sigma_i h_i}{n}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;P.S:&lt;/strong&gt; As a lemma, we know that consistency implies the admissibility of the heuristic function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem Explanation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://ai.stackexchange.com/q/18310/2444&quot;&gt;this link&lt;/a&gt;, I have found that the first heuristic is neither admissible, nor consistent.&lt;/p&gt;&#xA;&lt;p&gt;I know that the second and the fourth heuristics are either consistent, or admissible.&lt;/p&gt;&#xA;&lt;p&gt;I have faced with one contradiction in the third heuristic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uPzPU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uPzPU.png&quot; alt=&quot;contradiction example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here we see that if car 3 hops twice, the total cost of moving all the cars to their destinations is 3, whereas the heuristic &lt;span class=&quot;math-container&quot;&gt;$\max(h_1, \dots, h_n) = 4$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, &lt;span class=&quot;math-container&quot;&gt;$\max(h_1, ..., h_n)$&lt;/span&gt; must be consistent and admissible, but the above example shows that it's not. What is my mistake?&lt;/p&gt;&#xA;" OwnerUserId="33875" LastEditorUserId="2444" LastEditDate="2021-02-08T11:28:36.400" LastActivityDate="2021-02-08T11:28:36.400" Title="If $h_i$ are consistent and admissible, are their sum, maximum, minimum and average also consistent and admissible?" Tags="&lt;admissible-heuristic&gt;&lt;consistent-heuristic&gt;&lt;heuristic-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18356" PostTypeId="1" CreationDate="2020-03-02T06:15:33.723" Score="2" ViewCount="43" Body="&lt;p&gt;I am going through &lt;a href=&quot;https://ieeexplore.ieee.org/document/18626&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rabiner 1989&lt;/a&gt; and he writes that the discrete probability density function of duration &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; (that is, staying in a state for duration &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, conditioned on starting in that state) is &lt;span class=&quot;math-container&quot;&gt;$$p_i(d) = {a_{ii}}^{d-1}(1-a_{ii})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;span class=&quot;math-container&quot;&gt;$a_{ii}$&lt;/span&gt; is the state transition probability from state &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to state &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; - that is, staying in the same state).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He then continues to say that the expected durations in a state, conditioned on starting in that state, is &lt;span class=&quot;math-container&quot;&gt;$$\overline d_i = \sum_{i=1}^\infty d p_i(d) = \frac{1}{1-a_{ii}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does the coefficient &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; (in &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^\infty d p_i(d)$&lt;/span&gt;) come from?&lt;/p&gt;&#xA;" OwnerUserId="33954" LastActivityDate="2020-03-02T06:15:33.723" Title="Expected duration in a state" Tags="&lt;math&gt;&lt;probability&gt;&lt;hidden-markov-model&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18358" PostTypeId="2" ParentId="18355" CreationDate="2020-03-02T07:44:11.747" Score="1" Body="&lt;p&gt;The issue is that you must include assumptions about hopping into your heuristic. In particular, if you are considering individual cars then you must assume that they might be able to hop all of the way to the goal. Thus, your heuristic for each car should be Manhattan distance divided by 2. That's guaranteed to be admissible when you take the max.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider all possible cars you can do better, but you'll need to reason out all the cases. (In general every car either waits or moves, and for every waiting car one car can hop. So, by looking at the minimum distance for any car to reach the goal you can start to reduce your heuristic.) But, that is a different question.&lt;/p&gt;&#xA;" OwnerUserId="17493" LastActivityDate="2020-03-02T07:44:11.747" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18360" PostTypeId="1" AcceptedAnswerId="18414" CreationDate="2020-03-02T10:51:09.680" Score="1" ViewCount="114" Body="&lt;p&gt;I am currently studying &lt;em&gt;Deep Learning&lt;/em&gt; by Goodfellow, Bengio, and Courville. In chapter &lt;strong&gt;5.2 Capacity, Overfitting and Underfitting&lt;/strong&gt;, the authors say the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Typically, when training a machine learning model, we have access to a training set; we can compute some error measure on the training set, called the &lt;strong&gt;training error&lt;/strong&gt;; and we reduce this training error. So far, what we have described is simply an optimization problem. What separates machine learning from optimization is that we want the &lt;strong&gt;generalization error&lt;/strong&gt;, also called the &lt;strong&gt;test error&lt;/strong&gt;, to be low as well. The generalization error is defined as the expected value of the error on a new input. Here the expectation is taken across different possible inputs, drawn from the distribution of inputs we expect the system to encounter in practice. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found this part unclear:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here the expectation is taken across different possible inputs, drawn from the distribution of inputs we expect the system to encounter in practice. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The language used here is confusing me, because it is discussing a &quot;distribution&quot;, as in a &quot;probability distribution&quot;, but then refers to inputs, which are data gathered from outside of any probability distribution. Based on the limited information my studying of machine learning has taught me so far, my understanding is that the machine learning algorithm (or, rather, &lt;em&gt;some&lt;/em&gt; machine learning algorithms) uses training data to implicitly construct some probability distribution, right? So is this what it is referring to here? Is the &quot;distribution of inputs we expect the system to encounter in practice&quot; the so called &quot;test set&quot;? I would greatly appreciate it if people would please take the time to clarify this.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastActivityDate="2020-03-04T15:31:09.607" Title="Generalization error: Inputs drawn from distributions" Tags="&lt;machine-learning&gt;&lt;probability-distribution&gt;&lt;generalization&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18362" PostTypeId="1" CreationDate="2020-03-02T13:27:33.953" Score="1" ViewCount="42" Body="&lt;p&gt;I have currently collected 150000 gamestates from playing a Monte Carlo Tree Search AI player against a basic rule based AI at the game of &lt;a href=&quot;https://en.wikipedia.org/wiki/Castle_(card_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Castle&lt;/a&gt;. The information captured represents the information available to the MCTS player on the start of each of their turn and whether they won the game in the end. They are stored within CVS files.&lt;/p&gt;&#xA;&lt;p&gt;Example gamestate entry:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/iym6J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iym6J.png&quot; alt=&quot;Example game state entry:&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example the entry above shows that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;HAND:&lt;/strong&gt; MCTS player's hand contains the cards 7,5,4,9,9,9 (suit has been omitted because it has no baring on the game). &lt;em&gt;(list of cards)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;CASTLE_FU:&lt;/strong&gt; MCTS face up cards are 8,2,10 &lt;em&gt;(list of cards)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;CASTLE_FD_SIZE:&lt;/strong&gt; MCTS has 3 cards face down &lt;em&gt;(int)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;OP_HAND_SIZE:&lt;/strong&gt; The opponent has 3 cards in their hand &lt;em&gt;(int)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;OP_CASTLE_FU:&lt;/strong&gt; The opponents face up cards are Jack, Queen, Ace. &lt;em&gt;(list of cards)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;OP_CASTLE_FD_SIZE:&lt;/strong&gt; The opponent has 3 cards face down &lt;em&gt;(int)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;TOP:&lt;/strong&gt; The top of the discard pile is a 4 &lt;em&gt;(single value)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;DECK_EMPTY:&lt;/strong&gt; The deck in which players pick up cards is not empty &lt;em&gt;(boolean)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;WON:&lt;/strong&gt; The MCTS player ended up winning the hand &lt;em&gt;(boolean)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I hope to input this data into machine learning algorithms to produce an evaluation function for the MCTS algorithm to use.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I normalize this data so I can use it in Keras/Scikit-Learn?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;h2&gt;EDIT:&lt;/h2&gt;&#xA;&lt;p&gt;I'm not sure normalizing is the right term here. &lt;strong&gt;Encoding&lt;/strong&gt; or &lt;strong&gt;mapping&lt;/strong&gt; may be more accurate for what I am trying to achieve. Another difficulty I've encountered is the fact that the players hand size can vary up to almost holding the full deck in theory (although this would be incredibly rare in practice). However this is the only column that can be of a size greater than 3.&lt;/p&gt;&#xA;&lt;h2&gt;EDIT 2:&lt;/h2&gt;&#xA;&lt;p&gt;I've come up with this model to represent the data. Does this look suitable?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4OxWa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4OxWa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33966" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-02T18:35:08.570" Title="How can I normalize gamestates in order to use with a machine learning library?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18363" PostTypeId="1" CreationDate="2020-03-02T18:02:31.923" Score="1" ViewCount="49" Body="&lt;p&gt;I am currently working on a stock market prediction model which incorporates sentiments along with historical price for next day price prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to test different window / sequence size e.g (3 days, 4 days .. 10 days) to identify which window size is most optimal in predicting the next day prices. &#xA;However the selection for num_units in model.add(LSTM(units=num_units)) for different window sizes are varying. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a smaller window size is paired with a larger num_unit, there is over-fitting in the data where the model prediction for the price at day t+1 is almost equal to the price at day t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence I am unable to make a fair comparison between different window sizes without varying num_units&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have referred to this &lt;a href=&quot;https://ai.stackexchange.com/questions/3156/how-to-select-number-of-hidden-layers-and-number-of-memory-cells-in-an-lstm/3162#3162?newreg=8539e52aadbd4cb4bcb2171ae77b50e2&quot;&gt;How to select number of hidden layers and number of memory cells in an LSTM?&lt;/a&gt; however am unable to come to a conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a predefined guideline for the num_units to use within a LSTM cell for timeseries prediction based on the sequence length?&lt;/p&gt;&#xA;" OwnerUserId="33975" LastActivityDate="2020-03-02T18:02:31.923" Title="How should I go about selecting an optimal num_units within a LSTM cell for different sequence sizes" Tags="&lt;long-short-term-memory&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18364" PostTypeId="2" ParentId="18339" CreationDate="2020-03-02T18:36:31.737" Score="0" Body="&lt;p&gt;Resizing the images will work.But if you significantly reduce the image dimensions information IS lost. Here is a simple example. Suppose you have a 500 X 500 image. That has 250,000 pixels. Assume in these images the object of interest( lets say a bird in a forest) only occupies 10% of the pixels in the image(25,000 pixels). Now assume you reduce the image to 100 X 100 and thus have 10,000 pixels. Your object of interest (the bird) now only occupies 1000 pixels. These are the pixels the neural net will learn from.&#xA;Now a better way to do this is to first crop your original 500 X 500 images to maximize the percentage of bird pixels in the cropped image. For example assume the resulting cropped image comes out to be say a 200 X 200 image but in that image the subject of interest(the bird) occupies 50% of the pixels (20,000 pixels). Now if you reduce the cropped image down to a 100 X 100 the object of interest will occupy(5000 pixels).&#xA;Cropping images is of course a hassle. For some types like images for example images of people, and you are just interested in the faces there are routines available that will automatically crop the image to just the face. In general you will have to crop the images yourself.&#xA;If you want to build high accuracy classifiers you have to start with a &quot;robust&quot; data set.&#xA;The larger the percentage the object of interest takes up in the images the better your classifier will perform.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-02T18:36:31.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18365" PostTypeId="2" ParentId="3933" CreationDate="2020-03-02T19:11:17.010" Score="3" Body="&lt;p&gt;Your data set would be what is called &amp;quot;unbalanced' and this can lead to problems in developing an accurate classifier.&lt;/p&gt;&#xA;&lt;p&gt;The best thing to do (which you might not be able to do) is to find more images for those classes with a smaller number of images.&lt;/p&gt;&#xA;&lt;p&gt;Another alternative is to synthetically produce more images. One way to do that is to use the Keras &lt;code&gt;ImageDataGenerator.flow_from_directory&lt;/code&gt;. Documentation is at &lt;a href=&quot;https://keras.io/preprocessing/image/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/preprocessing/image/&lt;/a&gt;. Create a directory (&lt;code&gt;your_dir&lt;/code&gt;). In it, create a subdirectory &lt;code&gt;Giraffe&lt;/code&gt;. Place all your 43 giraffe images into that directory. Create another directory &lt;code&gt;your_save_dir&lt;/code&gt;, and leave it empty. Now, create the generators shown below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;datagen = ImageDataGenerator(rotation_range = 30, width_shift_range = 0.2,&#xA;                             height_shift_range = 0.2,&#xA;                             shear_range = 0.2, &#xA;                             zoom_range = 0.2,                             &#xA;                             horizontal_flip = True,&#xA;                             fill_mode = &amp;quot;nearest&amp;quot;)&#xA;&#xA;data=datagen.flow_from_directory(your_dir, target_size=(200, 200),&#xA;                                 batch_size=43, shuffle=False,&#xA;                                 save_to_dir=your_save_dir,save_format='png',&#xA;                                 interpolation='nearest')&#xA;&#xA;images,labels=data.next()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, each time you execute the last line of code, you will generate and store 43 more images in &lt;code&gt;your_save_dir&lt;/code&gt;. These images will be transformed per the parameters in the image data generator in a random manner. While NOT as good as having truly original images, it will help significantly to balance the data set.&lt;/p&gt;&#xA;&lt;p&gt;Do the same of course for the other image sets that have a small number of samples.&lt;/p&gt;&#xA;&lt;p&gt;Another thing that can help is, for the sets with fewer images, first, crop the images so that the animal occupies as high a percentage of pixels as possible in the cropped image. Then do the process defined above. This gives the network a higher percentage of meaningful pixels to &amp;quot;learn&amp;quot; from.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-06-21T22:37:01.400" LastActivityDate="2020-06-21T22:37:01.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18366" PostTypeId="1" CreationDate="2020-03-02T19:48:28.720" Score="4" ViewCount="82" Body="&lt;p&gt;I'm training a classifier and I want to collect incorrect outputs for human to double check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the output of the classifier is a vector of probabilities for corresponding classes. for example, [0.9,0.05,0.05]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means the probability for the current object being class A is 0.9, whereas for it being the class B is only 0.05 and 0.05 for C too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this situation, I think the result has a high confidence. As A's probability dominants B's and C's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In another case, [0.4,0.45,0.15], the confidence should be low, as A and B are close. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best formula to use to calculate this confidence?&lt;/p&gt;&#xA;" OwnerUserId="33082" LastActivityDate="2021-05-28T20:07:05.633" Title="How to calculate the confidence of a classifier's output?" Tags="&lt;classification&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18367" PostTypeId="2" ParentId="18293" CreationDate="2020-03-02T20:37:32.120" Score="1" Body="&lt;p&gt;I believe this is covered under Section 107 of the Copyright Act states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This section was intended to enable utilization of copyrighted material where that utilization is in the public interest. As long as you stay within the bounds as stated above you should not be considered in violation of a copyright.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="33976" LastEditDate="2020-04-26T23:00:52.527" LastActivityDate="2020-04-26T23:00:52.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18368" PostTypeId="2" ParentId="18251" CreationDate="2020-03-02T20:56:35.110" Score="1" Body="&lt;p&gt;A quick scan of your plots does not seem to indicate any severe over fitting. As pointed out there is always some degree of over fitting but in this case it looks to be very small. Your validation loss reduces as it should down to what appears to be a very small level and remains low.&#xA;One test would be to add a &quot;dropout&quot; layer into your model right after a dense layer and see the effect on training accuracy and validation accuracy. Set the drop out rate to something like .4 . Make sure your training accuracy remains high (it may take a few more epochs to get there) then look to see if the validation loss is lower than without the dropout layer. Run this several times because random weight initialization can sometimes effect accuracy by converging on a non optimal local minimum.&#xA;Additional you can add kernel regularizes to your dense layers which also helps to prevent over training. I have a lot of plots similar to yours and adding dropout and regularization had no effect on validation accuracy.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-02T20:56:35.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18369" PostTypeId="1" AcceptedAnswerId="18385" CreationDate="2020-03-02T21:12:55.583" Score="1" ViewCount="33" Body="&lt;p&gt;I need multiply the weigths of terms in TFIDF matrix by the word-embeddings of word2vec matrix but I can't do it because each matrix have a different number of terms. &lt;strong&gt;I am using the same corpus for get both matrix, I don't know why each matrix have a different number of terms&#xA;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I have a matrix TFIDF with the shape &lt;code&gt;(56096, 15500)&lt;/code&gt; (corresponding to: number of terms, number of documents) and matrix Word2vec with the shape &lt;code&gt;(300, 56184)&lt;/code&gt; (corresponding to : number of word-embeddings, number of terms).&lt;br&gt;&#xA;And I need the same numbers of terms in both matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use this code for get the matrix of word-embeddings Word2vec: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def w2vec_gensim(norm_corpus):&#xA;    wpt = nltk.WordPunctTokenizer()&#xA;    tokenized_corpus = [wpt.tokenize(document) for document in norm_corpus]&#xA;    # Set values for various parameters&#xA;    feature_size = 300&#xA;    # Word vector dimensionality&#xA;    window_context = 10&#xA;    # Context window size&#xA;    min_word_count = 1&#xA;    # Minimum word count&#xA;    sample = 1e-3&#xA;    # Downsample setting for frequent words&#xA;    w2v_model = word2vec.Word2Vec(tokenized_corpus, size=feature_size, window=window_context, min_count =  min_word_count, sample=sample, iter=100)&#xA;    words = list(w2v_model.wv.vocab)&#xA;    vectors=[]&#xA;    for w in words:&#xA;        vectors.append(w2v_model[w].tolist())&#xA;    embedding_matrix= np.array(vectors)&#xA;    embedding_matrix= embedding_matrix.T&#xA;    print(embedding_matrix.shape)&#xA;&#xA;    return embedding_matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this code for get the TFIDF matrix: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tv = TfidfVectorizer(min_df=0., max_df=1., norm='l2', use_idf=True, smooth_idf=True)&#xA;&#xA;&#xA;def matriz_tf_idf(datos, tv):&#xA;    tv_matrix = tv.fit_transform(datos)&#xA;    tv_matrix = tv_matrix.toarray()&#xA;    tv_matrix = tv_matrix.T&#xA;    return tv_matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I need the same number of terms in each matrix. For example, if I have 56096 terms in TFIDF, I need the same number in embeddings matrix, I mean matrix TFIDF with the shape &lt;code&gt;(56096, 1550)&lt;/code&gt; and matrix of embeddings Word2vec with the shape &lt;code&gt;(300, 56096)&lt;/code&gt;. How I can get the same number of terms in both matrix? &#xA;Because I can't delete without more data, due to I need the multiplication to make sense because my goal is to get the embeddings from the documents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much in advance.&lt;/p&gt;&#xA;" OwnerUserId="33977" LastActivityDate="2020-03-03T09:22:04.123" Title="Why I have a different number of terms in word2vec and TFIDF? How I can fix it?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;word-embedding&gt;&lt;word2vec&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18370" PostTypeId="2" ParentId="18249" CreationDate="2020-03-02T21:20:58.350" Score="1" Body="&lt;p&gt;The CNN should work without trying to do special feature extraction. As pointed out some pre-processing can aid in enhancing the CNN's classification results. The Keras &#xA;ImageDataGenerator provides optional parameters you can set to provide pre-processing as well as provide data augmentation.&#xA;One thing I know that works for sure but can be painful is cropping the images in such a way that the subject of interest occupies a high percentage of the pixels in the resultant cropped image. The cropped image can than be resized as needed. The logic here is simple.&#xA;You want your CNN to train on the subject of interest (for example a bird sitting in a tree where the bird is the subject of interest). The part of the image that is not of the bird is essentially just noise making the classifier's job harder. For example say you have a 500 X 500 initial image in which the subject of interest (the bird) only takes up 10% of the pixels (25,000 pixels). Now say as  input to your CNN you reduce the image size to 100 X 100. Now the (pixels that the CNN 'learns' from is down to 1000 pixels.&#xA;However lets say you crop the image so that the features of the bird are preserved but the  pixels of the bird in the cropped image take up 50% of the pixels. Now if you resize the cropped image to 100 X 100 , 5000 pixels of relevance are available for the network to learn from. I have done this on several data sets. In particular images of people where the subject of interest is the face. There are many programs that are effective at cropping these images so that  mostly just the face appears in the cropped result. I have trained a deep CNN in one case using uncropped images  and in the other with cropped images. The results are significantly better using the cropped images.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="33808" LastEditDate="2020-03-03T20:06:05.357" LastActivityDate="2020-03-03T20:06:05.357" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18371" PostTypeId="2" ParentId="17492" CreationDate="2020-03-02T21:36:40.007" Score="0" Body="&lt;p&gt;SVM is generally considered objectively better than deep learning for standard machine learning tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVM or decision trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep Learning is beneficial when there is structure in the data that can't be easily represented by some type of kernel.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm actually really interested in why decision trees haven't been used for computer vision in conjunction with deep learning feature extraction. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-03-02T21:36:40.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18372" PostTypeId="2" ParentId="18227" CreationDate="2020-03-02T21:53:52.260" Score="1" Body="&lt;p&gt;I have experimented with this to a small degree and have not noticed that much of an impact.&lt;/p&gt;&#xA;&lt;p&gt;To date, Adam appears to give the best results on a variety of image data sets. I have found that &amp;quot;adjusting&amp;quot; the learning rate during training is an effective means of improving model performance and has more impact than the selection of the optimizer.&lt;/p&gt;&#xA;&lt;p&gt;Keras has two callbacks that are useful for this purpose. Documentation is at &lt;a href=&quot;https://keras.io/callbacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/callbacks/&lt;/a&gt;. The &lt;code&gt;ModelCheckpoint&lt;/code&gt; callback enables you to save the full model or just the model weights based on monitoring a metric. Typically, you monitor validation loss and set the parameter &lt;code&gt;save_best_only=True&lt;/code&gt; to save the results for the lowest validation loss. The other useful callback is &lt;code&gt;ReduceLROnPlateau&lt;/code&gt;, which allows you to adjust the learning rate based on monitoring a metric. Again, the metric usually monitored is the validation loss. If the loss fails to reduce after a user-set number of epochs (parameter patience), the learning rate will be adjusted by a user-set factor (parameter factor). You can think of the training process as traveling down a valley. As you near the bottom of the valley, it becomes more and more narrow. If your learning rate does not adjust to the &amp;quot;narrowness&amp;quot; there is no way you will get to the bottom of the valley.&lt;/p&gt;&#xA;&lt;p&gt;You can also write a custom callback to adjust the learning rate. I have done this and created one which first adjusts the learning rate based on monitoring the training loss until the training accuracy reaches 95%. Then it switches to adjust the learning rate based on monitoring the validation loss. It saves the model weights for the lowest validation loss and loads the model with these weights to make predictions. I have found this approach leads to faster training and higher accuracy.&lt;/p&gt;&#xA;&lt;p&gt;The fact is you can't tell if your model has converged on a global minimum or a local minimum. This is evidenced by the fact that, unless you take special efforts to inhibit randomization, you can get different results each time you run your model. The loss can be envisioned as a surface in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; space, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of trainable parameters. Lord knows what that surface is like and where your initial parameter weights put you on that surface, plus how other random processes cause you to traverse that surface.&lt;/p&gt;&#xA;&lt;p&gt;As an example, I ran a model at least 20 times and got resultant losses that were very close to each there. Then I ran it again and got far better results for exactly the same data.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-09-12T15:14:35.917" LastActivityDate="2020-09-12T15:14:35.917" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18373" PostTypeId="2" ParentId="18206" CreationDate="2020-03-02T22:29:23.827" Score="1" Body="&lt;p&gt;I have consistently found Adam to work very well but to tell you the truth I have not seen all that much difference in performance based on the optimizer. Other factor seem to have much more influence on the final model performance.In particular adjusting the learning rate during training can be very effective. Also saving the weights for the lowest validation loss and loading the model with those weights to make predictions works very well. Keras provides two callbacks that help you achieve this. Documentation is at &#xA;&lt;a href=&quot;https://keras.io/callbacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/callbacks/&lt;/a&gt;. The ReduceLROnPlateau callback allows you to adjust the learning rate based on monitoring a metric. Typically validation loss is monitored. If the loss fails to reduce after N consecutive epochs(parameter patience) the learning rate is adjusted by a factor(parameter factor). You can think of training as descending into a valley which gets more and more narrow as you approach the bottom. If the learning rate does not adjust to this &quot;narrowness&quot; there is no way you will get to the very bottom.&#xA;The other callback is ModelCheckpoint. This allows you to save the model( or just the weights) based on monitoring of a metric. Again usually validation loss is monitored and the parameter save_best_only is set to true. This saves the model with the lowest validation loss. That model can than be used to make predictions.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-02T22:29:23.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18374" PostTypeId="2" ParentId="18205" CreationDate="2020-03-02T22:56:30.857" Score="0" Body="&lt;p&gt;Tensorflow-Keras provides an effective data transformer and loader. Documentation is at&#xA;&lt;a href=&quot;https://keras.io/preprocessing/image/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/preprocessing/image/&lt;/a&gt;. The ImageDataGenerator provides  for many types of possible transforms and also enables use of a user defined pre-process function. Use of the &#xA;ImageDataGenerator.flow_from_directory provides a means of retrieving images in batches from a directory containing sub directories (classes) of images. and resizing the images.&#xA;Image size can impact the results. Generally the larger the image the better the result but this is subject to the law of diminishing returns (at some point the impact on accuracy becomes minuscule) while the training time can become absorbent.&#xA;When you have large images like 1000 X 1000 where the subject of interest in the image is small say 50 X 50 the best but most painful approach is the crop the image to the subject of interest. Unfortunately this is usually a time consuming drudgery unless you can find some program that can crop the image automatically. For example there are good programs that can crop  images of people automatically where the resultant cropped image is primarily the persons face. Alternatively modules like cv2 can be adapted to provide this capability for certain images.&#xA;The batch_size you select along with the image size directly effect memory usage. If your images are large and your batch_size is too large you will encounter a &quot;resource exhaust&quot; error. You can reduce the batch size but this will extend training time.&#xA;Other techniques for dealing with large images include methods like sliding windows etc. Again these will increase training time because you are taking a large image and breaking it into a series of smaller images that you feed into the network.&#xA;A general though probably risky rule I follow is that if I can visibly see the subject of interest in a resized image then I assume the network will be able to detect it as well.&#xA;Will probably be less accurate than using the full image but should be as we engineers say &quot;good enough&quot;&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-02T22:56:30.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18375" PostTypeId="2" ParentId="18200" CreationDate="2020-03-02T23:12:32.793" Score="0" Body="&lt;p&gt;I recall seeing several data sets on Kaggle on the topic of car images. Usually these data sets have associated with them user uploaded python kernels that do either classification( what make of car) or detection ( show a bounding box on the car).&#xA;I would recommend you go to Kaggle then select data sets and do a search for 'cars'. I did a quick look and found a data set at &lt;a href=&quot;https://www.kaggle.com/jessicali9530/stanford-cars-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/jessicali9530/stanford-cars-dataset&lt;/a&gt; that has 18 kernels associated with it. There well may be many more data sets available. Good luck in your search!&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-02T23:12:32.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18376" PostTypeId="2" ParentId="18344" CreationDate="2020-03-02T23:25:55.173" Score="0" Body="&lt;p&gt;Transition probability matrix cannot be initialized. Your game world has some rules. Since we do not know these rules we can approximate them. To do this, we should run the game over and over and collect the outcome of each action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a toy example: assume we control a plane. We want the plane to fly forward, but sometimes there may be wind in a random direction. So the plane won't end up in the desired position. Instead, it has some probability to fly left, right and forward. After the data is collected, we can estimate this probabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is estimating n-gram model: assume we want to train a model that will suggest a word in search field. We would analyze a corpus and calculate all 2 word sequences (for &lt;a href=&quot;https://en.wikipedia.org/wiki/Bigram&quot; rel=&quot;nofollow noreferrer&quot;&gt;bigram&lt;/a&gt; model). Then, if a user starts typing &lt;code&gt;I&lt;/code&gt; the model would suggest the word &lt;code&gt;am&lt;/code&gt;, because &lt;code&gt;I am&lt;/code&gt; occurs more frequently then &lt;code&gt;I did&lt;/code&gt;, for example, and hence, the transition probability from &lt;code&gt;I&lt;/code&gt; to &lt;code&gt;am&lt;/code&gt; is greater.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is good book on &lt;a href=&quot;http://www.cs.columbia.edu/~mcollins/lm-spring2013.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Language Modeling&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But your question suggest me that you want something like reinforcement learning. Check out this &lt;a href=&quot;https://www.youtube.com/watch?v=lfHX2hHRMVQ&amp;amp;t=444s&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; wich is a part of a great lectures series on RL by David Silver&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-03-02T23:58:48.203" LastActivityDate="2020-03-02T23:58:48.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18377" PostTypeId="2" ParentId="18169" CreationDate="2020-03-02T23:48:58.577" Score="0" Body="&lt;p&gt;One problem could be with the selection of the validation set. For your model to work well on data it has not seen as training data is to have a high validation accuracy, but that is not sufficient on its own. The validation set must be large enough and varied enough that its probability distribution is an accurate representation of the probability distribution of all the images. You could have 1000 validation images, but, if they are similar to each other, they would be an inadequate representation of the probability distribution. Therefore, when you run your trained model to make predictions on the test set its performance would be poor.&lt;/p&gt;&#xA;&lt;p&gt;So the question is: &lt;em&gt;how many validation images did you use and how were they selected (randomly or handpicked)?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Try increasing the number of validation images and use one of the available methods to randomly select images from the training set, remove them from the training set, and use them as validation images.&lt;/p&gt;&#xA;&lt;p&gt;Keras's &lt;code&gt;flow_from_directory&lt;/code&gt; can achieve that or sklearn's &lt;code&gt;train_test_split&lt;/code&gt;. I usually have the validation set be selected randomly and have at least 10% as many images as the test set does.&lt;/p&gt;&#xA;&lt;p&gt;Overtraining is a possibility, but I think unlikely given your validation accuracy is high.&lt;/p&gt;&#xA;&lt;p&gt;Another thing is how were the test set images selected? Maybe their distribution is skewed. Again, the best thing to do is to select these randomly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What was your training accuracy?&lt;/em&gt; Without a high training accuracy, the validation accuracy may be a meaningless value. Training accuracy should be in the upper 90's for the validation accuracy to be really meaningful.&lt;/p&gt;&#xA;&lt;p&gt;Finally, is there any possibility your test images were mislabeled? Try switching your test set as the validation set and see what validation accuracy you get.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example of what I mean. A guy built a CNN to operate on a set of images separated into two classes. One class as &amp;quot;dogs&amp;quot;, the other class was &amp;quot;wolves&amp;quot;. He trained the network with great results almost 99.99% training accuracy and 99.6% validation accuracy. When he ran it on the test set, his accuracy was about 50%. Why? Well, it turns out all the images of wolves were taken with snow in the background. None of the images of dogs had snow in the background. So, the neural network figures out if snow must be a wolf, if no snow must be a dog. However, in his test set, he had a mixture of wolves in or not in snow, dogs in or not in snow. Great training and validation results but totally useless performance.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2021-01-17T19:28:41.650" LastActivityDate="2021-01-17T19:28:41.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18378" PostTypeId="1" CreationDate="2020-03-03T00:18:59.573" Score="1" ViewCount="179" Body="&lt;p&gt;In the context of a Markov decision process, &lt;a href=&quot;https://arxiv.org/pdf/1902.09725.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; says &quot;it is well-known that the optimal policy is invariant to positive affine transformation of the reward function&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, exercise 3.7 of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; gives an example of a robot in a maze:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Imagine that you are designing a robot to run a maze. You decide to give it a reward of +1 for escaping from the maze and a reward of zero at all other times. The task seems to break down naturally into episodes—the successive runs through the maze—so you decide to treat it as an episodic task, where the goal is to maximize expected total reward (3.7). After running the learning agent for a while, you find that it is showing no improvement in escaping from the maze. What is going wrong? Have you effectively communicated to the agent what you want it to achieve?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems like the robot is not being rewarded for escaping quickly (escaping in 10 seconds gives it just as much reward as escaping in 1000 seconds). One fix seems to be to subtract 1 from each reward, so that each timestep the robot stays in the maze, it accumulates &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; in reward, and upon escape it gets zero reward. This seems to change the set of optimal policies (now there are way fewer policies which achieve the best possible return). In other words, a positive affine transformation &lt;span class=&quot;math-container&quot;&gt;$r \mapsto 1 \cdot r - 1$&lt;/span&gt; seems to have changed the optimal policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I reconcile &quot;the optimal policy is invariant to positive affine transformation of the reward function&quot; with the maze example?&lt;/p&gt;&#xA;" OwnerUserId="33930" LastActivityDate="2020-08-07T18:06:48.280" Title="Policy invariance under affine transformations of the reward function?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18379" PostTypeId="1" CreationDate="2020-03-03T00:23:30.343" Score="1" ViewCount="313" Body="&lt;p&gt;It seems to me that Seq2Seq models and Bidirectional RNNs try to do the same thing. Is that true? Also, when would you recommend one setup over another? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="32023" LastActivityDate="2021-03-28T11:04:25.513" Title="Pros and Cons of Seq2Seq vs Bidirectional RNN" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18380" PostTypeId="1" CreationDate="2020-03-03T04:14:53.497" Score="0" ViewCount="71" Body="&lt;p&gt;Is there a good way to understand how single-shot object detection works? The most basic way to do detection is use a sliding-window detector and look at the output of the NN to detect if a class is there or not.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if there is a way to understand how many of the single-shot detectors work? Internally is there some form of sliding window going on? Or is it basically the same detector learned at each point?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-03-03T04:14:53.497" Title="Intuition behind single-shot object detection" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;yolo&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18381" PostTypeId="1" CreationDate="2020-03-03T06:08:33.907" Score="2" ViewCount="56" Body="&lt;p&gt;I watched in the video lecture of cs224: &lt;a href=&quot;https://www.youtube.com/watch?v=kEMJRjEdNzM&amp;amp;list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z&amp;amp;index=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 2 – Word Vectors and Word Senses&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They take the sample size of the window to be &lt;span class=&quot;math-container&quot;&gt;$2^5 = 32$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$2^6 = 64$&lt;/span&gt;. Why is the sample size of stochastic gradient descent a power of 2? Why not we can take 42 or 53 as the sample window size? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw, how to identify best minimum window sample size?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-05T00:56:46.887" LastActivityDate="2021-03-30T04:02:20.193" Title="Why is the sample size of stochastic gradient descent a power of 2?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18382" PostTypeId="2" ParentId="18381" CreationDate="2020-03-03T06:51:34.373" Score="0" Body="&lt;p&gt;You can take any size you want since most memories are now larger than that, if you were to map the indices to a register, 64 values will take 6 bits whereas we have registers upto 64 bits, so it wouldn't matter much even if you choose a  different sample size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as resolution is concerned, layers like maxpool which reduce the size by half (2x2 kernel is generally used), the resolutions of outputs of layers are multiplied/divided by powers of 2, so keeping that in mind, resolutions as power of 2 makes sense, but then again, we have padding and effects of convolution on borders.&lt;/p&gt;&#xA;" OwnerUserId="27875" LastActivityDate="2020-03-03T06:51:34.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18383" PostTypeId="2" ParentId="18378" CreationDate="2020-03-03T07:25:38.090" Score="1" Body="&lt;p&gt;This statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(it is well-known that the optimal policy is invariant to positive affine transformation of the reward function).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is, as far as I know, and as you summarise, incorrect because simple translations to reward signal do affect the optimal policy, and the affine transform of a real number &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; can be given by &lt;span class=&quot;math-container&quot;&gt;$f(x) = mx + c$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It &lt;em&gt;is&lt;/em&gt; well known that optimal policy is unaffected by multiplying all rewards by a positive scaling, e.g. &lt;span class=&quot;math-container&quot;&gt;$f(x) = mx$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also worth noting that if an optimal policy is derived from Q values using &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \text{argmax}_a Q(s,a)$&lt;/span&gt;, then that policy function &lt;em&gt;is&lt;/em&gt; invariant to positive affine transformations of &lt;em&gt;action values&lt;/em&gt; given by &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;. Perhaps that was what the paper authors meant to write, given that they go on to apply normalisation to Q values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The impact of the mistake is not relevant to the the rest of the paper as far as I can see (caveat: I have not read the whole paper). &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-03-03T07:32:24.447" LastActivityDate="2020-03-03T07:32:24.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18384" PostTypeId="2" ParentId="18379" CreationDate="2020-03-03T09:05:48.747" Score="0" Body="&lt;p&gt;Seq2Seq and Bidirectional RNNs are not doing the same thing, at least in there classic form. Seq2Seq models are used to generate a sequence from another sequence, consider for example tranlation task from one language to another. In that sense Seq2Seq is more a family of models not an architecture, while Bidirectionnal RNNs is a neural network architecture and can be used to build several models including Seq2Seq, for example the encodign part of the Seq2Seq can be a Bi-RNN, but they also can be used for other tasks, for example sentence classfication, sentiment analysis...&lt;/p&gt;&#xA;" OwnerUserId="32493" LastActivityDate="2020-03-03T09:05:48.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18385" PostTypeId="2" ParentId="18369" CreationDate="2020-03-03T09:22:04.123" Score="0" Body="&lt;p&gt;Your problem is that TFIDF is cutting out around 90 terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you aren't using the &lt;code&gt;min_df&lt;/code&gt; or &lt;code&gt;max_df&lt;/code&gt; parameters and as far as I can tell you aren't doing any stemming/lemmatization, the only difference I can see between the two methods is the tokenizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two things I'd try out if I were you:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Try explicitly converting the word2vec corpus to lowercase. &lt;code&gt;TfidfVectorizer&lt;/code&gt; does this by default and I can't see where you're doing it in the word2vec pipeline. Ignore this if your corpus is already lowercased.&lt;/li&gt;&#xA;&lt;li&gt;Try using the &lt;code&gt;nltk.WordPunctTokenizer()&lt;/code&gt; with the &lt;code&gt;TfidfVectorizer&lt;/code&gt;. You can do this like this:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wpt = nltk.WordPunctTokenizer()&#xA;tv = TfidfVectorizer(min_df=0., max_df=1., norm='l2', use_idf=True, smooth_idf=True,&#xA;                     tokenizer=wpt.tokenize)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-03-03T09:22:04.123" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18386" PostTypeId="1" CreationDate="2020-03-03T12:53:26.340" Score="1" ViewCount="39" Body="&lt;p&gt;I have heard that sigmoid activation functions should not be used on neural networks with many hidden layers as the gradients tend to vanish in deep networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When should each of the common activation functions be used, and why?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReLu&lt;/li&gt;&#xA;&lt;li&gt;Sigmoid&lt;/li&gt;&#xA;&lt;li&gt;Softmax&lt;/li&gt;&#xA;&lt;li&gt;Leaky ReLu&lt;/li&gt;&#xA;&lt;li&gt;TanH&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33227" LastEditorUserId="2444" LastEditDate="2020-03-03T13:07:54.153" LastActivityDate="2020-03-03T13:07:54.153" Title="What are the pros and cons of the common activation functions?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18387" PostTypeId="1" CreationDate="2020-03-03T13:52:41.163" Score="2" ViewCount="29" Body="&lt;p&gt;I am required to obtain data through a sensor located on the vehicle reading speed, vibration, roll and tilt, within a sample time, to classify the current road condition using machine learning for a high school project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithm/approach may be most suitable for this task? Suggestions to sources for learning (books, tutorials) would be also appreciated, as I am new to AI and ML.&lt;/p&gt;&#xA;" OwnerUserId="33984" LastEditorUserId="2444" LastEditDate="2020-03-05T01:01:58.353" LastActivityDate="2020-03-05T01:01:58.353" Title="Suitable algorithms for classifying terrain condition (asphalt, dirt etc) for motor vehicles" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18388" PostTypeId="1" CreationDate="2020-03-03T14:11:22.863" Score="2" ViewCount="34" Body="&lt;p&gt;I am doing a simple scan to see how dataset size affects training. Basically, I took 10% of the coco dataset and trained a yolov3 net (from scratch) to just look for people. Then I took 20% of the coco dataset and did the same thing.... all the way to 100%. What is strange is that all 9 nets are getting similar loss at the end (~7.5). I must be doing something wrong, right? I expected to see an exponential curve where loss started out high and assymptotically approached some value as the dataset increased to 100%. If it didn't approach a value (and still had a noticeable slope at 100%), then that meant more data could help my algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my .data file:&#xA;classes= 1&#xA;train = train-run-less.txt&#xA;valid = data/coco/5k.txt&#xA;names = data/humans.names&#xA;backup = backup&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to train just one class (person) from the coco dataset. Something is not making sense, and in a sanity test, I discovered that the loss drops even if the training folder only contains 1 image (which doesnt even have people in it). I thought the way this worked was that it trained on the &quot;train&quot; images, then it tested the neural net on the &quot;valid&quot; images. How is it getting better at finding people in the &quot;valid&quot; images if it hasnt trained on a single one??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I am trying to answer the question: &quot;how much accuracy can I expect to gain as I increase the data?&quot;&lt;/p&gt;&#xA;" OwnerUserId="33987" LastEditorUserId="33987" LastEditDate="2020-03-03T14:42:31.370" LastActivityDate="2021-03-29T16:02:50.303" Title="Data scan not making sense for coco dataset" Tags="&lt;training&gt;&lt;yolo&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18389" PostTypeId="1" CreationDate="2020-03-03T15:15:14.750" Score="1" ViewCount="24" Body="&lt;p&gt;I’d like to build a model that has an understanding of geometry, where it can be applied to question and answering system. Specifically, it would be nice if it could determine the volume of an object by simply looking at pictures of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are any pre-trained models out there that I can utilize that would certainly make things easier. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, are there any suggestions on the kind of model(s) I should use to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I read something online about how Facebook trained an AI to solve complex math problems just by looking at them. They approached the problem as a language translation problem, not as a math problem. I wonder if this is the way to go?&lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="2444" LastEditDate="2020-03-05T01:00:51.313" LastActivityDate="2020-03-05T01:00:51.313" Title="Train an AI to infer accurate mathematical calculations by simply “looking” at images of shapes/objects" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18390" PostTypeId="1" AcceptedAnswerId="18399" CreationDate="2020-03-03T15:38:43.003" Score="1" ViewCount="282" Body="&lt;p&gt;When we define the loss function of a variational autoencoder (VAE), we add the Kullback-Leibler divergence between the sample taken according to a normal distribution of parameters: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ N(\mu,\sigma) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we compare it with a normal distribution of parameters &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ N(0,1) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition is that it is clever having samples taken from a distribution centered around zero, but I don't understand why we want that examples are taken with a normal distribution. &lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2020-03-04T01:04:14.690" LastActivityDate="2020-03-04T01:09:42.517" Title="Why do we regularize the variational autoencoder with a normal distribution?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;probability-distribution&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18391" PostTypeId="1" CreationDate="2020-03-03T15:43:38.180" Score="1" ViewCount="343" Body="&lt;p&gt;I know dropout layers are used in neural networks during training to provide a form of regularisation in an attempt to mitigate over-fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you not get an increased fitness if you disabled the dropout layers during evaluation of a network?&lt;/p&gt;&#xA;" OwnerUserId="25574" LastEditorUserId="2444" LastEditDate="2020-03-03T17:54:52.170" LastActivityDate="2020-03-03T22:19:15.977" Title="Does the performance of a model increase if dropout is disabled at evaluation time?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;dropout&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18392" PostTypeId="2" ParentId="18391" CreationDate="2020-03-03T16:16:01.083" Score="2" Body="&lt;p&gt;Dropout is a technique that helps to avoid overfitting during training. That is, one can use dropout only for training.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;units may change in a way that they fix up the mistakes of the other&#xA;  units. This may lead to complex co-adaptations. This, in turn, leads to&#xA;  overfitting because these co-adaptations do not generalize to unseen&#xA;  data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you want to evaluate your model, you should turn off all dropout layers. For example, PyTorch's &lt;a href=&quot;https://pytorch.org/docs/stable/nn.html#torch.nn.Module.eval&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;model.eval()&lt;/code&gt;&lt;/a&gt; does this work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More about dropout:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1207.0580&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving neural networks by preventing co-adaptation of feature detectors&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jmlr.org/papers/v15/srivastava14a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout: A Simple Way to Prevent Neural Networks from Overfitting&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="2444" LastEditDate="2020-03-03T22:19:15.977" LastActivityDate="2020-03-03T22:19:15.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18393" PostTypeId="1" CreationDate="2020-03-03T16:39:52.983" Score="0" ViewCount="65" Body="&lt;p&gt;What are typical ways to understand and visualize a trained RL agent's policy when the state space is of high dimension (but not images)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose  state and action are denoted by &lt;span class=&quot;math-container&quot;&gt;$s=(s_1,s_2,\cdots,s_n)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a=(a_1,a_2,\cdots,a_k)$&lt;/span&gt;. How do I determine which attribute of the state (e.g. an image pixel of video game) is most responsible for a particular action &lt;span class=&quot;math-container&quot;&gt;$a_j$&lt;/span&gt;? I would like to have, for each action &lt;span class=&quot;math-container&quot;&gt;$a_j, j=1,2,...,k$&lt;/span&gt;, a table that ranks the attributes of the observation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question may be a little bit vague, but if you have any thoughts on how to improve it please let me know! &lt;/p&gt;&#xA;" OwnerUserId="33660" LastEditorUserId="2444" LastEditDate="2020-03-05T00:53:24.870" LastActivityDate="2020-03-05T00:53:24.870" Title="How to understand and visualize a trained RL agent's policy when the state space is high dimensional?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;policies&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18394" PostTypeId="2" ParentId="18391" CreationDate="2020-03-03T17:23:06.297" Score="0" Body="&lt;p&gt;Keras turns off dropout when computing validation accuracy during training. Also when making predictions&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-03T17:23:06.297" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18395" PostTypeId="1" AcceptedAnswerId="18422" CreationDate="2020-03-03T17:34:45.410" Score="3" ViewCount="105" Body="&lt;p&gt;I'm doing machine learning projects. I took a look at many datasets I worked with, mostly there are already famous datasets that everyone uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I decided to make my own dataset. Is there a possibility that my data are so random so that no relationship exists between my inputs and outputs? This is interesting because if this is possible, then no machine learning model will achieve to find an inputs outputs relationship in the data and will fail to solve the regression or classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, is it mathematically possible that some values have absolutely no relationship between them? In other words, there is no function (linear or nonlinear) that can map those inputs to the outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I thought about this problem and concluded that, if there is a possibility for this, then it will likely happen in regression because maybe the target outputs are in the same range and the same features values can correspond to the same output values and that will confuse the machine learning model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you ever encountered this or a similar issue?&lt;/p&gt;&#xA;" OwnerUserId="30327" LastEditorUserId="2444" LastEditDate="2020-03-04T01:00:12.193" LastActivityDate="2020-03-05T10:24:05.817" Title="Is there a possibility that there is no relationship between some inputs and outputs?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;math&gt;&lt;regression&gt;&lt;function-approximation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18396" PostTypeId="2" ParentId="18391" CreationDate="2020-03-03T17:43:42.483" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout&lt;/a&gt; is usually disabled at test (or evaluation) time. For example, in Keras, dropout is disabled at evaluation time by default, although you can enable it, if you need to (see below). The purpose of dropout is to decorrelate the units (or feature detectors) so that they learn more robust representations of the data (i.e. a form of regularisation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there's also &lt;a href=&quot;https://arxiv.org/pdf/1506.02142.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo (MC) dropout&lt;/a&gt;, i.e., you train the network with dropout and you also use dropout at test time in order to get stochastic outputs (i.e. you will get different outputs, for different forward passes, given the same inputs). &lt;a href=&quot;https://arxiv.org/pdf/1506.02142.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MC dropout is an approximation of Bayesian inference in deep Gaussian processes&lt;/a&gt;, which means that MC dropout is roughly equivalent to a &lt;a href=&quot;https://arxiv.org/pdf/1505.05424.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural network&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the performance of a model increase if dropout is disabled at evaluation time?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, possibly. However, MC dropout provides an uncertainty measure, which can be useful in certain scenarios (e.g. medical scenarios), where a point estimate (i.e. a single prediction or classification) is definitely not appropriate, but you also need a measure of the uncertainty or confidence of the predictions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-03T17:55:35.677" LastActivityDate="2020-03-03T17:55:35.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18397" PostTypeId="1" CreationDate="2020-03-03T17:49:14.887" Score="1" ViewCount="212" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UFg5b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UFg5b.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this due to my dropout layers being disabled during evaluation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm classifying the CIFAR-10 dataset with a CNN using the Keras library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 50000 samples in the training set; I'm using a 20% validation split for my training data (10000:40000). I have 10000 instances in the test set.&lt;/p&gt;&#xA;" OwnerUserId="25574" LastEditorUserId="25574" LastEditDate="2020-03-04T14:31:47.143" LastActivityDate="2021-03-31T08:07:15.760" Title="Why is my validation/test accuracy higher than my training accuracy" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;dropout&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18398" PostTypeId="2" ParentId="4176" CreationDate="2020-03-03T18:10:25.957" Score="1" Body="&lt;p&gt;Recently, I needed to develop a Fuzzy Logic algorithm to made inferences of any data entrance; the real case applied was in Oil and Gas Industry, that the code needs to infere Joint Types in Fluid Pipelines. But with this Algorithm, the Computer Science Developer can infere any problems data, follow the link bellow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.codeproject.com/Articles/5092762/Csharp-Fuzzy-Logic-API&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.codeproject.com/Articles/5092762/Csharp-Fuzzy-Logic-API&lt;/a&gt; (&lt;em&gt;C# Fuzzy Logic API&lt;/em&gt;)&lt;/p&gt;&#xA;" OwnerUserId="33992" LastActivityDate="2020-03-03T18:10:25.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18399" PostTypeId="2" ParentId="18390" CreationDate="2020-03-03T18:28:16.647" Score="1" Body="&lt;p&gt;If you are mathematically inclined, &lt;a href=&quot;http://ruishu.io/2018/03/14/vae/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is an article&lt;/a&gt; that discusses the reasoning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I get as a take away is that the VAE forces the learned latent space to be Gaussian due to the KL divergence term in the loss function. So, now we have a known distribution to sample from to create input vectors to feed to the decoder, to produce say images of dogs, if the VAE was trained on images of dogs. As you sample from the distribution, you will produce images of different types of dog images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume a different type of distribution could be selected if one uses the proper loss function for that type of distribution, that is, the loss function which would measure the difference in the distribution of the latent space and the desired distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;KL divergence is the loss function that forces the latent space distribution to be Gaussian. If you do not &quot;restrict&quot; the latent space, as is the case with a regular autoencoder, you have no idea what kind of vector to select as an input to the decoder to produce a dog image. Without restriction, there are &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the dimensions of the latent space) possible vectors you could choose from. Chances of selecting one that produces a dog image would be minuscule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, I hope this helps. I no longer am mathematically proficient (age 75), so I hope my interpretation of the math is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A VAE tends to produce blurry images because there are two terms in the loss function. One term is trying to make the output look like the input while the KL loss term is trying to restrict the latent space distribution. GANs ( generative adversarial networks) don't have this conflict, so they produce much high-quality images.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-03-04T01:09:42.517" LastActivityDate="2020-03-04T01:09:42.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18402" PostTypeId="2" ParentId="18366" CreationDate="2020-03-03T19:18:01.613" Score="0" Body="&lt;p&gt;Obvious answer for a binary (2 classes)classification is .5. Beyond that the earlier comment is correct. One of the things I have seen done is to run your model on the test set and save the prediction probability results.  Then create a threshold variable call it thresh. Then increment thresh from 0 to 1 in a loop. On each iteration compare thresh with the highest predicted probability prediction call it P. If  P&gt;thresh declare that as the selected prediction, then compare that with the true prediction. Keep track of the errors for each value of thresh. At the end select the value of thresh that has the least errors. &#xA;There are also some  more sophisticated methods for example &quot;top 2 accuracy&quot; where thresh is selected based on  having the true class within either the prediction with the highest probability or the second highest probability . You can construct a weighted error function and select the value of thresh that has the net lowest error over the test set.&#xA;For example an error function might be as follows. If neither P(highest) or P(second highest) = True class, error=1. If P(second highest) = true class, error=.5. If p(highest)=true class error=0. I have never tried this myself so I am not sure how well this works. When I get some time will try it on a model with 100 classes and see how well it does. I know in the Imagenet competition they evaluate not just the top accuracy but also the &quot;Top 3&quot; and &quot;Top 5' accuracy. In that competition there are 1000 classes.&#xA;I never thought of this before but I assume you could train your model specifically to optimize say the Top 2 accuracy by constructing a loss function used during training that forces the network to minimize this loss.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-03T19:18:01.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18404" PostTypeId="2" ParentId="18232" CreationDate="2020-03-04T01:58:25.483" Score="2" Body="&lt;p&gt;The difference really comes down to the fact that in meta-learning, there is a population of tasks &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; which have distribution &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt;. The goal is to perform well on a task drawn from &lt;span class=&quot;math-container&quot;&gt;$p(\tau)$&lt;/span&gt;. Generally 'perform well' means that with only a few training steps or data points, the model can give good classification accuracy, achieve high reward in an RL setting, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A concrete example is given in the original MAML paper &lt;a href=&quot;https://arxiv.org/pdf/1703.03400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, where the task is to perform regression on data given by a sinusoidal distribution with parameters &lt;span class=&quot;math-container&quot;&gt;$p(\theta)$&lt;/span&gt;. The meta-learning goal is to get high regression accuracy on tasks where the data is drawn from distributions coming from &lt;span class=&quot;math-container&quot;&gt;$p(\theta)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, transfer learning is a bit more general since there's not necessarily a notion of a distribution of tasks. There is generally just one (although there can be more) source problem &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, and the goal is to do well on a target problem &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. You know both of these explicitly, unlike in MAML where the goal is to do well amongst any unknown problem drawn from a certain distribution. Very often, this is performed by taking a model that performs well on &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and adapting it to work on &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, perhaps by using extracted features from the model for &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The extent to which this will succeed obviously depends on the similarity of the two tasks. This is also known in the literature as domain adaptation, and has some theoretical results &lt;a href=&quot;http://www.alexkulesza.com/pubs/adapt_mlj10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, although the bounds are not really applicable to modern high-dimensional datasets.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.03400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks&lt;/a&gt; (Finn et al) 2017.        &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.alexkulesza.com/pubs/adapt_mlj10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Theory of Learning from Different Domains&lt;/a&gt; (Ben-David et al) 2010.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15176" LastEditorUserId="2444" LastEditDate="2020-03-04T02:14:00.007" LastActivityDate="2020-03-04T02:14:00.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18405" PostTypeId="1" CreationDate="2020-03-04T02:34:50.740" Score="1" ViewCount="161" Body="&lt;p&gt;I try get the matrix embedding of my model but I can't because although this code gives no error it never ends running. The code is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fast_text(norm_corpus):&#xA;    wpt = nltk.WordPunctTokenizer()&#xA;    tokenized_corpus = [wpt.tokenize(document) for document in norm_corpus]&#xA;    # Set values for various parameters&#xA;    feature_size = 300&#xA;    # Word vector dimensionality&#xA;    window_context = 5&#xA;    # Context window size&#xA;    min_word_count = 1&#xA;    # Minimum word count&#xA;    sample = 1e-3&#xA;    ft_model = FastText(tokenized_corpus, size=feature_size, window=window_context,min_count=min_word_count,sample=sample, sg=i, iter=50)&#xA;&#xA;    words = list(ft_model.wv.vocab)&#xA;    vectors = []&#xA;    for w in words:&#xA;        vectors.append(ft_model[w].tolist())&#xA;    embedding_matrix = np.array(vectors)&#xA;    embedding_matrix = embedding_matrix.T&#xA;    print(embedding_matrix.shape)&#xA;&#xA;    return embedding_matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know how fix it. &lt;/p&gt;&#xA;" OwnerUserId="33977" LastActivityDate="2020-03-04T02:34:50.740" Title="How get matrix of word embeddings in FastText of gensim?" Tags="&lt;python&gt;&lt;models&gt;&lt;word-embedding&gt;&lt;numpy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18406" PostTypeId="2" ParentId="18395" CreationDate="2020-03-04T04:16:08.303" Score="0" Body="&lt;p&gt;Not sure if I can answer the question as a whole, but a pure random input/output pair doesn't quite have &quot;no relationship&quot; at all. At the very least, for any fixed training set input/output pair, you can do an if...then mapping to construct a 1-to-1 function, such that you can classify the training set with 100% accuracy (assuming no duplicates of input).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, I assume you mean uniform random, because if you have something like gaussian random, you can still learn some latent structure from how the random numbers are generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even if you assume uniform random, and your algorithm is only guessing, your algorithm is technically still operating optimally per the data generating distribution, which basically means its as optimal as it gets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only such case that I can imagine which would satisfy your question, would be if you had a separate training/validation set, where the only element of the training input/output is [1,1], but the validation set only has elements of [1,-1], or something along those lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From reading your comments, I suspect that your intention with the question was: &quot;Can there be a relationship of data such that no method can learn it?&quot;. To the extent that the data-generating distribution exists, then by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem of neural networks&lt;/a&gt;, then it is reasonable that you can at least partially learn it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However it is important to note that the universal approximation theorem doesn't mean that such a data generating distribution can be &lt;em&gt;learned&lt;/em&gt; by a neural net, it only means that you can get &quot;non-zero as close as you want&quot; to the data generating distribution. More explicitly: there is a setting of weights that gives you results as good as you want, but gradient descent doesn't necessarily learn it.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-03-04T18:41:58.143" LastActivityDate="2020-03-04T18:41:58.143" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="18407" PostTypeId="1" CreationDate="2020-03-04T08:30:34.127" Score="2" ViewCount="87" Body="&lt;p&gt;I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into whatever categories the learning technique identifies. Using PCA or manifold techniques such as t-SNE for this problem is rather straightforward, but applying neural networks (autoencoders, specifically) becomes non-trivial, as I am not sure splitting my dataset into test and training data is the right way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively, I was thinking of the following approaches: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Train an autoencoder with all the data as training data and train it for a large number of epochs (overfitting is not a problem in this case perse I would think)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Keras offers a &lt;code&gt;model.predict&lt;/code&gt; option which enables me to just construct the encoder section of the autoencoder and obtain the bottleneck values&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Carry out some data augmentation and split the data as one might into training and test data and carry out the workflow as normal (This approach makes me a little uncomfortable as I am not attempting to generalize a neural network or should I be?)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any guidance on how to proceed or if my understanding of the application of autoencoders is flawed in this context.&lt;/p&gt;&#xA;" OwnerUserId="34002" LastEditorUserId="2444" LastEditDate="2020-03-05T00:51:56.797" LastActivityDate="2021-05-02T03:15:36.427" Title="How can I use autoencoders to analyze patterns and classify them?" Tags="&lt;image-recognition&gt;&lt;keras&gt;&lt;unsupervised-learning&gt;&lt;autoencoders&gt;&lt;dimensionality-reduction&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18409" PostTypeId="1" AcceptedAnswerId="18432" CreationDate="2020-03-04T12:39:59.857" Score="2" ViewCount="85" Body="&lt;p&gt;I am trying to work on a variation of the &lt;em&gt;Access-Control Queuing Task&lt;/em&gt; problem presented in Chapter 10 of Sutton’s reinforcement learning book [1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specific details of my setup are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have different types of tasks that arrive to a system&#xA;(heavy/moderate/light with heavy tasks requiring more time to be&#xA;processed.). The specific task type is chosen uniformly at random. The task inter-arrival time is &lt;span class=&quot;math-container&quot;&gt;$0.1s$&lt;/span&gt; on average.&lt;/li&gt;&#xA;&lt;li&gt;I have different classes of servers that can process these tasks&#xA;(low-capacity; medium capacity; high capacity; with high capacity&#xA;servers having a faster processing time). When I select a specific&#xA;server from a given class, it becomes unavailable during the&#xA;processing time of the task assigned to it. Note that the set of servers (and as a result the number of servers of each class) is not fixed, it instead changes periodically, according to the dataset used to model the set of servers (so specific servers may disappear and new ones may appear, as opposed to the unavailability caused by the assignment). The maximum number of servers of each class is &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;li&gt;My goal is to decide which class of server should process a given&#xA;task, in a way that minimizes the sum of the processing times over&#xA;all tasks.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The specific reinforcement learning formulation is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;State&lt;/strong&gt;: the type of task (heavy/moderate/light) ; the number of available low&#xA;capacity servers; the number of available medium capacity servers; the number&#xA;of available high capacity servers&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Actions&lt;/strong&gt;: &lt;strong&gt;(1)&lt;/strong&gt; Assign the task to a low capacity server &lt;strong&gt;(2)&lt;/strong&gt; assign the&#xA;task to a medium capacity server &lt;strong&gt;(3)&lt;/strong&gt; assign the task to a high&#xA;capacity server &lt;strong&gt;(4)&lt;/strong&gt; a dummy action that has a worse processing time&#xA;than the servers with low capacity. It is selected when there are no free servers.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Rewards&lt;/strong&gt;: the opposite of the processing time, where the processing times are as follows (in seconds):&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&#xA;|               | Slow server | Medium Server | Fast server | &quot;Dummy action&quot; |&#xA;|---------------|-------------|---------------|-------------|----------------|&#xA;| Light task    | 0.5         | 0.25          | 0.166       | 0.625          |&#xA;| Moderate task | 1.5         | 0.75          | 0.5         | 1.875          |&#xA;| Heavy task    | 2.5         | 1.25          | 0.833       | 3.125          |&#xA;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My intuition for formulating the problem as an RL problem is that 'Even though assigning Light tasks to High capacity servers (i.e. being greedy) might lead to a high reward in the short term, it may reduce the number of High capacity servers available when a Heavy task arrives. As a result, Heavy tasks will have to be processed by lower capacity servers which will reduce the accumulated rewards'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I implemented this (using a deep Q-network[2] specifically), and compared it to the greedy policy, I found that both approaches obtain the same rewards. In fact, the deep Q-network ends up learning the greedy policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering why such a behaviour occured, especially that I expected the DQN approach to learn a better policy than the greedy one. Could this be related to my RL problem formulation? Or there is no need for RL to address this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]Sutton, R. S., &amp;amp; Barto, A. G. (1998). Introduction to reinforcement learning (Vol. 135). Cambridge: MIT press.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2]Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... &amp;amp; Petersen, S. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533.f&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="34010" LastEditDate="2020-03-05T08:36:33.550" LastActivityDate="2020-03-05T10:23:42.637" Title="Unexpected results when comparing a greedy policy to a DQN policy" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18410" PostTypeId="1" AcceptedAnswerId="18419" CreationDate="2020-03-04T13:09:59.623" Score="1" ViewCount="131" Body="&lt;p&gt;I was reading a blog post that talked about the problem of the saddle point in training. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the post, it says if the loss function is flatter in the direction of x (local minima here) compared to y at the saddle point, gradient descent will oscillate to and from the y direction. This gives an illusion of converging to a minima. Why is this? &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Wouldn’t it continue down in the y direction and hence escape the saddle point?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NyXTy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NyXTy.png&quot; alt=&quot;Saddle point&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to post:&#xA;&lt;a href=&quot;https://blog.paperspace.com/intro-to-optimization-in-deep-learning-gradient-descent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.paperspace.com/intro-to-optimization-in-deep-learning-gradient-descent/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please go to &lt;em&gt;Challenges with Gradient Descent #2: Saddle Points&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33803" LastActivityDate="2020-03-04T17:42:51.353" Title="Oscillating around the saddle point in gradient descent?" Tags="&lt;optimization&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18411" PostTypeId="1" CreationDate="2020-03-04T14:52:09.580" Score="1" ViewCount="84" Body="&lt;p&gt;What are state of the art backward chaining packages/tools/github algorithms in Python? It would be nice if you could also comment on the strengths and weaknesses of each solution !&lt;/p&gt;&#xA;" OwnerUserId="33880" LastActivityDate="2020-03-04T14:52:09.580" Title="What are state of the art backward chaining packages in Python?" Tags="&lt;python&gt;&lt;inference&gt;&lt;automated-reasoning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18412" PostTypeId="2" ParentId="18388" CreationDate="2020-03-04T14:58:04.523" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is strange is that all 9 nets are getting similar loss at the end (~7.5). I must be doing something wrong, right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, Please check the annotations file. Did you remove all the other classes except Person? What's the mAP of all those models?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;how much accuracy can I expect to gain as I increase the data?&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's hard to tell but more the data(variation), better the model.&lt;/p&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2020-03-04T14:58:04.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18413" PostTypeId="1" CreationDate="2020-03-04T14:59:59.170" Score="1" ViewCount="344" Body="&lt;p&gt;Removing stop words can significantly speed up named entity recognition (NER) modeling by reducing the number of tokens in a document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are stop words critical to get correct NER performance?&lt;/p&gt;&#xA;" OwnerUserId="33880" LastEditorUserId="2444" LastEditDate="2020-03-04T15:57:44.500" LastActivityDate="2020-03-04T16:15:05.423" Title="Is it recommended to remove stop words before named entity recognition?" Tags="&lt;natural-language-processing&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18414" PostTypeId="2" ParentId="18360" CreationDate="2020-03-04T15:25:53.783" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The language used here is confusing me, because it is discussing a &quot;distribution&quot;, as in a &quot;probability distribution&quot;, but then refers to inputs, which are data gathered from outside of any probability distribution. Based on the limited information my studying of machine learning has taught me so far, my understanding is that the machine learning algorithm (or, rather, some machine learning algorithms) uses training data to implicitly construct some probability distribution, right? So is this what it is referring to here?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They're not referring to probability distributions of training data that ML algorithms (implicitly) construct here. The main point of confusion seems to be where you state this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;but then refers to inputs, which are data gathered from outside of any probability distribution&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Any&lt;/strong&gt; data / inputs ever collected &lt;strong&gt;always&lt;/strong&gt; originate from &lt;em&gt;some&lt;/em&gt; distribution. We will typically not exactly know what that distribution is, we are often not able to provide a clean expression for it, and it might not even be a nice &quot;smooth&quot; distribution, but that doesn't mean it doesn't exist. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I collect a large number of photographs of &lt;span class=&quot;math-container&quot;&gt;$H \times W$&lt;/span&gt; pixels of streets for the purpose of training a self-driving car, then this collection of training data was collected from &lt;em&gt;some&lt;/em&gt; distribution. For each of the pixels in the &lt;span class=&quot;math-container&quot;&gt;$H \times W$&lt;/span&gt; plane, there exists some probability distribution that tells us how likely it is for such a pixel to have a certain colour under the data collection procedure that was used to generate our data. This is a largely unknown distribution, for which we don't have a nice mathematical expression, but it does exist. I assume that, in this distribution, it's relatively likely for pixels in the centre to be gray (because streets tend to be gray and we collected data by taking photographs of streets). I also guess it's relatively likely for pixels at the top of the images to be blue, because of the sky. Other than that, we can't say much about the distribution, but it does exist.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the &quot;distribution of inputs we expect the system to encounter in practice&quot; the so called &quot;test set&quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Kind of, yeah. Although I suppose the &quot;test set&quot; is mostly a thing in academic settings, where we use a test set to evaluate how well an approach performs on data that it did not observe during training. In the &quot;real world&quot;, the distribution of inputs we expect the system to encounter in practice refers to the distribution that generates samples we encounter after &quot;deployment&quot; of the model. For example, this could be the distribution over all images that a self-driving car may encounter when driving anywhere in the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuing with the self-driving car example, we may get a large generalization error if we only train it on images of streets in one particular city or country, but then afterwards have it drive in many different cities or countries around the world (which may look very different).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-03-04T15:25:53.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18415" PostTypeId="2" ParentId="18360" CreationDate="2020-03-04T15:31:09.607" Score="0" Body="&lt;p&gt;For illustration, I use the dog/cat classification task. Suppose, the training data of cat and dog follows the Gaussian distribution(for simplicity) and we trained a model which gives an accuracy as below.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;train - 98.2% &lt;/li&gt;&#xA;&lt;li&gt;val - 97.7% &lt;/li&gt;&#xA;&lt;li&gt;test - 97.2%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The model is neither overfitting nor underfitting but we want the classifier to achieve an accuracy of 100%  in all the three sets theoretically. You are right that the model learns the distribution of training data to classify the classes. Due to the overlapping of fat-tail in the distribution of cat and dog, it is highly impossible for the model to get 100% accuracy practically. There will be infinite edge cases that we encounter in reality so we can only improve the model by an iterative approach.&lt;/p&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2020-03-04T15:31:09.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18416" PostTypeId="2" ParentId="18413" CreationDate="2020-03-04T16:15:05.423" Score="1" Body="&lt;p&gt;It depends how you recognise the entities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do a simple gazetteer lookup, then it could be faster, as you have fewer tokens to deal with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you use contextual rules, then stop words might be vital to identify certain contexts, so by removing stop words you lose information about the entity's environment. For example, if &lt;strong&gt;[work] at {organisation}&lt;/strong&gt; is a rule you use to identify companies etc, then this wouldn't work if you take out the &lt;em&gt;at&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will also have problems if the stop words are part of an entity, eg the town &lt;em&gt;Stoke-on-Trent&lt;/em&gt;. If hyphens are used to split tokens, then you won't be able to recognise it if the &lt;em&gt;on&lt;/em&gt; is discarded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general I think stop words should mostly be kept; apart from information retrieval where they are not all that useful in an inverted index you will always lose something. If stop words were really pointless, then they would have been discarded from languages a long time ago. In the old days they were a useful way of reducing the demand on computational resources without losing too much, but nowadays I would say this is not really a problem anymore.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-03-04T16:15:05.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18417" PostTypeId="1" AcceptedAnswerId="22008" CreationDate="2020-03-04T16:36:27.153" Score="1" ViewCount="80" Body="&lt;p&gt;I've been reading the &lt;a href=&quot;https://arxiv.org/pdf/1504.08083.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast R-CNN paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My understanding is&lt;/strong&gt; that the input to one forward pass is the whole input image plus a list of RoIs (generated by selective search or another region proposal method). Then I understand that on the last convolution layer's feature map (&lt;strong&gt;let's call it FM&lt;/strong&gt;), each &lt;em&gt;corresponding&lt;/em&gt; RoI gets RoI-pooled, where now the corresponding ROIs are a rectangular (over height and width) slice of the FM tensor over all channels.&lt;/p&gt;&#xA;&lt;p&gt;But I'm &lt;strong&gt;having trouble with two concepts&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How is the input RoI mapped to the corresponding RoI in FM? Each neuron comes from a very wide perceptive field, so, in a deep neural network, there's no way of making a 1:1 mapping between input neurons and neurons in the last convolution layer right?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Disregarding that I'm confused in point 1, once we have a bunch of RoIs in FM and we do the RoI pooling, we have N pooled feature vectors. Do we now run each of these through one FC network one by one? Or do we have N branches of FC networks? (that wouldn't make sense to me)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have also read the &lt;a href=&quot;https://arxiv.org/pdf/1506.01497v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;faster R-CNN paper&lt;/a&gt;. In the same way, I'm also interested to know about how the proposed regions from RPN map to the input of the RoI pooling in the Fast R-CNN layers. Because actually those proposed regions live in the space of the input image, not in the space of the deep feature map.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="2444" LastEditDate="2020-06-19T23:00:06.780" LastActivityDate="2020-06-19T23:00:06.780" Title="In Fast R-CNN, how are input RoIs mapped to the respective RoIs in the feature map before RoI pooling?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;r-cnn&gt;&lt;pooling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18418" PostTypeId="1" CreationDate="2020-03-04T16:59:09.483" Score="2" ViewCount="333" Body="&lt;p&gt;After years of learning, I still can't understand what is considered to be an AI. What are the requirements for an algorithm to constitute Artificial Intelligence? Can you provide pseudocode examples of what constitutes an AI?&lt;/p&gt;&#xA;" OwnerUserId="34016" LastEditorUserId="2444" LastEditDate="2020-03-05T22:51:56.293" LastActivityDate="2020-03-06T00:24:23.933" Title="Can you provide some pseudocode examples of what constitutes an AI?" Tags="&lt;philosophy&gt;&lt;definitions&gt;&lt;intelligence&gt;&lt;pseudocode&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18419" PostTypeId="2" ParentId="18410" CreationDate="2020-03-04T17:42:51.353" Score="0" Body="&lt;p&gt;It's important to note that in a pragmatic instance of ML on a &quot;real dataset&quot;, you likely wouldn't have a &quot;strict&quot; saddle point with precisely zero gradient. Your error surface won't be &quot;smooth&quot;, so even though you would have something that resembles a saddle point, what you would obtain would in fact be a small region with near-zero gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's say you are in a region with near-zero gradient. Assuming that the gradient is this area is normalized at 0 with small Gaussian distributed noise (thus gradient =  small Gaussian noise). You can then see that the algorithm can't quite escape the region (or at least, will spend a lot of time here) since 1. Gaussian random walks will more-or-less stay in place (unless for a long time) 2. small gradients means there is no obvious direction to leave the region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, SGD more or less solves this issue, and its usage is standard practice for reasons beyond this problem.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2020-03-04T17:42:51.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18420" PostTypeId="2" ParentId="17796" CreationDate="2020-03-04T17:43:20.453" Score="1" Body="&lt;p&gt;The approach will vary depending on some features of the game:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How many players (two for tic tac toe and many classic games).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether it is a &quot;perfect information&quot; game (yes for chess and tic tac toe), or whether there is significant hidden information (true for most forms of poker)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether the game is zero sum. Any game with simple win/loss/draw results can be considered zero sum.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Branching complexity of the game. Tic tac toe has relatively low branching factor, at worst 9, and reduces by one each turn. The game of Go has branching factoraround 250.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Action complexity of the game. A card game like Magic the Gathering has huge complexity even if the number of possible actions at each turn is quite small.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The above traits and others make a large impact on the details of which algorithms and approaches will work best (or work at all). As that is very broad, I will not go into that, and I would suggest you ask separate questions about specific games if you take this further and try to implement some self-playing learning agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to outline a general approach that would work for many games:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1. Implement Game Rules&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You need to have a programmable API for running the game, which allows for code representing a player (called an &quot;agent&quot;) to receive current state of the game, to pass in a chosen action, and to return results of that action. The results should include whether any of the players has won or lost, and to update internal state of the game ready for next player.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2. Choose a Learning Agent Approach&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are several kinds of learning algorithms that are suitable for controlling agents and learning through experience. One popular choice would be Reinforcement Learning (RL), but also Genetic Algorithms (GA) can be used for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Key here is how different types of agent solve the issues of self-play:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;GA does this very naturally. You create a population of agents, and have them play each other, selecting winners to create the next generation. The main issue with GA approaches is how well they scale with complexity - in general not as well as RL.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With RL you can have the current agent play best agent(s) you have created so far - which is most general approach. Or for some games you may be able to have it play both sides at once using the same predictive model - this works because predicting moves by the opposition is a significant part of game play.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;2a. How self-play works in practice&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Without going into lines of code, what you need for self-play is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The game API for automation and scoring&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;One or more agents that can use the API&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A system that takes the results of games between agents and feeds them back so that learning can occur:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In a GA, with tournament selection, this could simply be saving a reference - the id of the winner - into a list, until the list has grown large enough to be the parents for the next generation. Other selection methods are also possible, but the general approach is the same - the games are played to help populate the next generation of agents.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In RL, typically each game state is saved to memory along with next resulting state or the result win/draw/lose (converted into a reward value e.g. +1/0/-1). This memory is used to calculate and update estimates of future results from any given state (or in some variants used directly to decide best action to take from any given state). Over time, this link between current and next states provides a way for the agent to learn better early game moves that eventually lead to a win for each player.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An important &quot;trick&quot; in RL is to figure out a way for the model and update rules to reflect opposing goals of players. This is not a consideration in agents that solve optimal control in single agent environments. You either need to make the prediction algorithm predict future reward as seen from perspective of the current agent, or use a global scoring system and re-define one of the agents as trying to minimise the total reward - this latter approach works nicely for two player zero sum games, as both players can then direcly share the same estimator, just using it to different ends.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A lot of repetition in a loop. GA's unit of repetition is usually a &lt;em&gt;generation&lt;/em&gt; - one complete assessment of all existing agents (although again there are variants). RL's unit of repetition is often smaller, individual episodes, and the learning part of the loop can be called on every single turn if desired. The basic iteration in all cases is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Play anything from one move to multiple games, with automated agents taking all roles in the game and storing results.&lt;/li&gt;&#xA;&lt;li&gt;Use results to update learned parameters for the agents.&lt;/li&gt;&#xA;&lt;li&gt;Use the updated agents for the next stages of self-play learning.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;3. Planning&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A purely reactive learning agent can do well in simple games, but typically you also want the agent to look ahead to predict future results more directly. You can combine the outputs from a learning model like RL with a forward-planning approach to get the best of both worlds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forward search methods include &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt;/&lt;a href=&quot;https://en.wikipedia.org/wiki/Negamax&quot; rel=&quot;nofollow noreferrer&quot;&gt;negamax&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search&lt;/a&gt;. These can be combined with learning agents in multiple ways - for instance as well as in planning stages &lt;em&gt;using&lt;/em&gt; the RL model, they can also be used to help train a RL model (this is how it is used in Alpha Zero).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-03-04T19:12:49.780" LastActivityDate="2020-03-04T19:12:49.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18422" PostTypeId="2" ParentId="18395" CreationDate="2020-03-04T19:21:46.950" Score="3" Body="&lt;p&gt;Of course, it's possible to define a problem where there is no relationship between input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and output &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. In general, if the mutual information between &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is zero (i.e. &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are statistically independent) then the best prediction you can do is independent of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. The task of machine learning is to learn a distribution &lt;span class=&quot;math-container&quot;&gt;$q(y|x)$&lt;/span&gt; that is as close as possible to the real data generating distribution &lt;span class=&quot;math-container&quot;&gt;$p(y|x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, looking at the common cross-entropy loss, we have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;H(p,q) = -\mathbb{E}_{y,x \sim p}\left[\log q(y|x)\right] &amp;amp; = \mathbb{E}_{x\sim p}\left[\text{H}(p(y|x)) + \text{D}_{\text{KL}}(p(y|x)\|q(y|x))\right]  \\&#xA;&amp;amp; = \text{H}(p(y)) + \mathbb{E}_{x \sim p}\left[\text{D}_{\text{KL}}(p(y)\|q(y|x))\right],&#xA;\end{align}&#xA;$$&lt;/span&gt;&#xA;where we have used the fact that &lt;span class=&quot;math-container&quot;&gt;$p(y|x)=p(y)$&lt;/span&gt; since &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; are independent. From this, we can see that the optimal predicted distribution &lt;span class=&quot;math-container&quot;&gt;$q(y|x)$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$p(y)$&lt;/span&gt;, and actually independent of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Also, the best loss you can get is equal to the entropy of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="15176" LastEditorUserId="2444" LastEditDate="2020-03-05T00:58:55.060" LastActivityDate="2020-03-05T00:58:55.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18423" PostTypeId="1" AcceptedAnswerId="18475" CreationDate="2020-03-04T20:17:57.710" Score="4" ViewCount="110" Body="&lt;p&gt;Is there any difference between the convolution operation applied to images and applied to other numerical 2D data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we have a pretty good CNN model trained on a number of &lt;span class=&quot;math-container&quot;&gt;$64 \times 64$&lt;/span&gt; images to detect two classes. On the other hand, we have a number of &lt;span class=&quot;math-container&quot;&gt;$64 \times 64$&lt;/span&gt; numerical 2D matrices (which are not considered images), which also have two classes. Can we use the same CNN model to classify the numerical dataset?&lt;/p&gt;&#xA;" OwnerUserId="34018" LastEditorUserId="34018" LastEditDate="2020-03-05T18:21:07.597" LastActivityDate="2020-03-11T10:33:02.363" Title="Is there any difference between the convolution operation applied to images and applied to other numerical 2D data?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18424" PostTypeId="2" ParentId="18423" CreationDate="2020-03-04T21:30:20.203" Score="2" Body="&lt;p&gt;Short answer is &lt;strong&gt;no&lt;/strong&gt;. You can't use a model trained for one task to predict on a totally different task. Even if the second task was another image classification task, the CNN would have to be fine tuned for the new data to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of things to note...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) CNNs are good for images due to their nature. It isn't necessary that they'd be good for any 2-dimensional input.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) By 2D numerical data I'm assuming you &lt;strong&gt;don't&lt;/strong&gt; mean &lt;em&gt;tabular data&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-03-04T21:30:20.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18425" PostTypeId="1" CreationDate="2020-03-05T02:10:48.410" Score="2" ViewCount="91" Body="&lt;p&gt;I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to write the code for the reward functions and the algorithm for the joint space motion. &lt;/p&gt;&#xA;" OwnerUserId="34020" LastEditorUserId="2444" LastEditDate="2020-11-15T16:49:06.810" LastActivityDate="2021-04-14T18:58:11.943" Title="How can I implement the reward function for an 8-DOF robot arm with TRPO?" Tags="&lt;reinforcement-learning&gt;&lt;gym&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18426" PostTypeId="2" ParentId="18418" CreationDate="2020-03-05T02:55:17.710" Score="2" Body="&lt;p&gt;AI is not a simple term. There are different types, ranging from the most simplistic rule-based AI to black-box AI's so complicated it's unreasonable for a human to understand exactly what they're doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no pseudocode that if used in a program automatically constitutes it as an AI. It's not that black and white. But I can give examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a rule-based chess AI that forfeits if it's too far behind, and plays aggressively if it's far enough ahead.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if player.score - my.score &amp;gt; 10:&#xA;    forfeit&#xA;elif my.score - player.score &amp;gt; 10:&#xA;    agressive = True&#xA;&#xA;for each piece of my.pieces:&#xA;    for each square of board.squares:&#xA;        if noThreats(square) and agressive is True:&#xA;            move(piece, square)&#xA;            return&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is considered an &quot;AI&quot; because it feigns intelligence - appearing to have a true understanding of chess while simply following a set of rules, making it an &quot;Artificial&quot; Intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's another more complicated AI:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;decisionNet = NeuralNetwork(64 inputs, 2 outputs)&#xA;&#xA;choice = decisionNet(board.squares) // Returns a chess square with one of my pieces and desitnation &#xA;move(choice)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This uses a neural network to make the decision, which could have been trained on a bunch of example games or against itself. Due to this &quot;training phase&quot;, humans can't understand precisely what the network is doing without extensive effort, so it's an even gives an even more convincing understanding of chess. But if we want, we could still understand the nuances of this network, and show it doesn't possess an intelligence, it again only feigns it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should mention that virtually any code that has an &lt;strong&gt;if&lt;/strong&gt; statement can be considered AI. The examples I provided are just easier to pass off as understanding a very complicated concept (chess), as opposed to, say, verifying a user login. They both have the same fundamentals, it's just one appears more complicated on the surface than the other.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2020-03-05T22:58:22.357" LastActivityDate="2020-03-05T22:58:22.357" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18428" PostTypeId="2" ParentId="18232" CreationDate="2020-03-05T06:08:29.353" Score="2" Body="&lt;p&gt;First of all, I would like to say that it is possible that these terms are used inconsistently, given that at least transfer learning, AFAIK, is a relatively new expression, so, the general trick is to take terminology, notation and definitions with a grain of salt. However, in this case, although it may sound confusing to you, all of the current descriptions on this page (in your question and the other answers) don't seem inconsistent with my knowledge. In fact, I think I had already roughly read some of the cited research papers (e.g. the MAML paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roughly speaking, although you can have formal definitions (e.g. the one in the MAML paper and also described in &lt;a href=&quot;https://ai.stackexchange.com/a/18404/2444&quot;&gt;this answer&lt;/a&gt;), which may not be completely consistent across sources, &lt;strong&gt;meta-learning&lt;/strong&gt; is about &lt;strong&gt;learning to learn&lt;/strong&gt; or learning something that you usually don't directly learn (e.g. the hyperparameters), where learning is roughly a synonym for optimization. In fact, the meaning of the word &quot;meta&quot; in meta-learning is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;denoting something of a &lt;strong&gt;higher or second-order kind&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For example, in the context of training a neural network, you want to find a neural network that approximates a certain function (which is represented by the dataset). To do that, usually, you manually specify the optimizer, its parameters (e.g. the learning rate), the number of layers, etc. So, in this usual case, you will train a network (learn), but you will not know that the hyperparameters that you set are the most appropriate ones. So, in this case, training the neural network is the task of &quot;learning&quot;. If you also want to learn the hyperparameters, then you will, in this sense, learn how to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept of meta-learning is also common in reinforcement learning. For example, in the paper &lt;a href=&quot;https://www.semanticscholar.org/paper/Metacontrol-for-Adaptive-Imagination-Based-Hamrick-Ballard/099cdb087f240352a02286bf9a3e7810c7ebb02b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metacontrol for Adaptive Imagination-Based Optimization&lt;/a&gt;, they even formalize the concept of a meta-Markov decision process. If you read the paper, which I did a long time ago, you will understand that they are talking about a higher-order MDP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, in the context of machine learning, meta-learning usually refers to learning something that you usually don't learn in the standard problem or, as the definition of meta above suggests, to perform &quot;higher-order&quot; learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Transfer learning&lt;/strong&gt; is often used as a synonym for fine-tuning, although that's not always the case. For example, in &lt;a href=&quot;https://www.tensorflow.org/tutorials/images/transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;this TensorFlow tutorial&lt;/a&gt;, transfer learning is used to refer to the scenario where you freeze (i.e. make the parameters non-trainable) the convolution layers of a model &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; pre-trained on a dataset &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, replace the pre-trained dense layers of model &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; on dataset &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with new dense layers for the new tasks/dataset &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, then retrain the new model, by adjusting the parameters of this new dense layer, on the new dataset &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. There are also papers that differentiate the two (although I don't remember which ones now). If you use transfer learning as a synonym for fine-tuning, then, roughly speaking, transfer learning is to use a pre-trained model and then slightly retrain it (e.g. with a smaller learning rate) on a new but related task (to the task the pre-trained model was originally trained for), but you don't necessarily freeze any layers. So, in this case, fine-tuning (or transfer learning) means to tune the pre-trained model to the new dataset (or task).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How is transfer learning (as fine-tuning) and meta-learning different?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meta-learning is, in a way, about fine-tuning, but not exactly in the sense of transfer learning, but in the sense of hyperparameter optimization. Remember that I said that meta-learning can be about learning the parameters that you usually don't learn, i.e. the hyper-parameters? When you perform hyper-parameters optimization, people sometimes refer to it as fine-tuning. So, meta-learning is a way of performing hyperparameter optimization and thus fine-tuning, but not in the sense of transfer learning, which can be roughly thought of as &lt;em&gt;retraining a pre-trained model but on a different task with a different dataset&lt;/em&gt; (with e.g. a smaller learning rate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, take terminology, notation, and definitions with a grain of salt, even the ones in this answer.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-11T13:10:24.987" LastActivityDate="2020-06-11T13:10:24.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18429" PostTypeId="1" CreationDate="2020-03-05T06:11:29.173" Score="1" ViewCount="35" Body="&lt;p&gt;Do you have any reference you can point me to for doing Text classification using any variation of BERT(albert or XLnet) with a TF implementation. I am not sure how to deploy torch based models, so asking for TF based implementation. Actually i need to do small document classification (2-3 paras) but any text classification example would do.&lt;/p&gt;&#xA;" OwnerUserId="34025" LastActivityDate="2020-03-05T06:11:29.173" Title="Tensorflow based implementation of Text classification with any variation of BERT(ALBERT/XLNET)" Tags="&lt;deep-learning&gt;&lt;transformer&gt;&lt;bert&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18430" PostTypeId="2" ParentId="18220" CreationDate="2020-03-05T06:46:54.817" Score="2" Body="&lt;h2&gt;Theoretical results&lt;/h2&gt;&#xA;&lt;p&gt;Rather than providing a rule of thumb (which can be misleading, so I am not a big fan of them), I will provide some theoretical results (the first one is also reported in paper &lt;a href=&quot;http://dstath.users.uth.gr/papers/IJRS2009_Stathakis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;How many hidden layers and nodes?&lt;/a&gt;), from which you may be able to derive your rules of thumb, depending on your problem, etc.&lt;/p&gt;&#xA;&lt;h3&gt;Result 1&lt;/h3&gt;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://pdfs.semanticscholar.org/064f/1e85984b207c1eb3c53ac8b68037089b7a0b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning capability and storage capacity of two-hidden-layer feedforward networks&lt;/a&gt; proves that a 2-hidden layer feedforward&#xA;network (&lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;) with &lt;span class=&quot;math-container&quot;&gt;$$2 \sqrt{(m + 2)N} \ll N$$&lt;/span&gt; hidden neurons can learn any &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; distinct samples &lt;span class=&quot;math-container&quot;&gt;$D= \{ (x_i, t_i) \}_{i=1}^N$&lt;/span&gt; with an arbitrarily small error, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the required number of output neurons. Conversely, a &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; hidden neurons can store at least &lt;span class=&quot;math-container&quot;&gt;$\frac{Q^2}{4(m+2)}$&lt;/span&gt; any distinct data &lt;span class=&quot;math-container&quot;&gt;$(x_i, t_i)$&lt;/span&gt; with&#xA;any desired precision.&lt;/p&gt;&#xA;&lt;p&gt;They suggest that a sufficient number of neurons in the first layer should be &lt;span class=&quot;math-container&quot;&gt;$\sqrt{(m + 2)N} + 2\sqrt{\frac{N}{m + 2}}$&lt;/span&gt; and in the second layer should be &lt;span class=&quot;math-container&quot;&gt;$m\sqrt{\frac{N}{m + 2}}$&lt;/span&gt;. So, for example, if your dataset has size &lt;span class=&quot;math-container&quot;&gt;$N=10$&lt;/span&gt; and you have &lt;span class=&quot;math-container&quot;&gt;$m=2$&lt;/span&gt; output neurons, then you should have the first hidden layer with roughly 10 neurons and the second layer with roughly 4 neurons. (I haven't actually tried this!)&lt;/p&gt;&#xA;&lt;p&gt;However, these bounds are suited for fitting the training data (i.e. for overfitting), which isn't usually the goal, i.e. you want the network to generalize to unseen data.&lt;/p&gt;&#xA;&lt;p&gt;This result is strictly related to the universal approximation theorems, i.e. a network with a single hidden layer can, in theory, approximate any continuous function.&lt;/p&gt;&#xA;&lt;h2&gt;Model selection, complexity control, and regularisation&lt;/h2&gt;&#xA;&lt;p&gt;There are also the concepts of &lt;em&gt;model selection&lt;/em&gt; and &lt;em&gt;complexity control&lt;/em&gt;, and there are multiple related techniques that take into account the complexity of the model. The paper &lt;a href=&quot;https://pdfs.semanticscholar.org/064f/1e85984b207c1eb3c53ac8b68037089b7a0b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model complexity control and statistical learning theory&lt;/a&gt; (2002) may be useful. It is also important to note regularisation techniques can be thought of as controlling the complexity of the model [&lt;a href=&quot;https://www.nature.com/articles/s41467-020-14663-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;You may also want to take a look at these related questions&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/q/181/82135&quot;&gt;How to choose the number of hidden layers and nodes in a feedforward neural network?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/17870/2444&quot;&gt;How to estimate the capacity of a neural network?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(I will be updating this answer, as I find more theoretical results or other useful info)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-13T20:40:01.093" LastActivityDate="2020-11-13T20:40:01.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18431" PostTypeId="1" CreationDate="2020-03-05T08:50:33.400" Score="9" ViewCount="2913" Body="&lt;p&gt;I have a MMO game where I have players. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to invent something new to the game, and add player-bots to make the game be singleplayable as-well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI I want to add is simply only for fighting other players or other player-bots that he sees around at his level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I thought of implementing my fighting-strategy, exactly how I play, to the bot which is basically using if statements and randoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when the opponent has low health, and the bot has enough special attack power, he will use this chance and use his special attack power in order to try to knock the opponent down, or if the bot has low health he will eat in time but not too much because there is a point in risking fights, if you eat too much your opponent will do too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or for example if the bot detects the opponent player is eating too much and gains health, he will do the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I told this idea of implementation to one of my friends and he simply responded with: This is not AI, its simply just a set of conditions, it does not have any heuristic functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that type of a game, what are some ideas to create a real AI to achieve these conditions?&#xA;Basically the AI should know what to do in order to beat the opponent, based on the opponent's data such as current health, Armour and weapons and level, if he risks his health or not and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a beginner and it really interests me to do it in the right way.&lt;/p&gt;&#xA;" OwnerUserId="34027" LastEditorUserId="34027" LastEditDate="2020-03-06T17:01:16.137" LastActivityDate="2020-03-19T18:33:44.063" Title="The real approach of creating AI for a fighting robot in an MMO" Tags="&lt;game-ai&gt;&lt;java&gt;" AnswerCount="5" CommentCount="8" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18432" PostTypeId="2" ParentId="18409" CreationDate="2020-03-05T09:32:11.210" Score="1" Body="&lt;p&gt;Your problems appear to derive from a non-Markov state description. In brief, the agent has no way to ever learn that &quot;heavy jobs will make a server unavailable for longer&quot; and this is further compounded by arbitrary state transitions every 10 steps whilst the agent cannot track time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the example from Sutton &amp;amp; Barto, you should note that they took care to model a valid MDP. Servers become available at random on each time step, and there is no hidden state. In your case you have three sources of hidden data that affects state evolution systematically. If an agent had some data about these then it might be able to use to optimise choices better:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Servers in use due to tasks assigned by the agent return deterministically n steps after the action that assigned them. The agents inability to track this is likely to be your largest problem - there is no data flow in RL that will allow the agent to associate the heavier jobs or slower servers with making a server unavailable for longer, so it never &quot;sees&quot; the clash of resource management that you want it to resolve.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Every 10 timesteps there is an arbitrary change to server availability. This is a problem because it is entirely predictable on which time steps it can occur, but the agent does not know the current timestep.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You are driving this from a dataset, not randomly, so the impact on this will vary depending on how the dataset evolves over time. A particularly bad case here would be if the dataset represented a &quot;working day&quot; so that there were clear timezones when availability followed different patterns. The agent has no data input in the state in order to associate patterns of use (such as the time of day), so the environment either becomes an online learning problem (agent must continuously learn and adapt to new patterns as they arise), or it must come up with a strategy that works in some average over all possible patterns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In addition I can see a possible secondary effect, an &quot;exploit&quot; in the simulation, that may have a noticeable effect depending on how often specific servers drop out of the pool. It would appear if an agent assigns a task to a server, that you have no mechanism to unassign that task should the server drop out. So an aggressive agent that assigns greedily to the best available servers will get extra &quot;free&quot; time on the high capacity servers if it always claims them. As the agent doesn't know when the drop outs could happen, this effect competes with the average of leaving servers available just in case a heavy task arrives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to fix it? It depends on how much of your simulation is fixed by your requirements. You can either adjust the simulation to match your current state representation, or you can change the state representation to capture more system data from the simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the simplest state change would be to expand the list of servers to always include all servers, and for each unavailable server to have a &quot;ticker&quot; to show how many time steps before it becomes available. To make the problem smaller, you could maybe just track 1-10 ticks and have a combined state for (&gt;10), as that will expose enough of the key state information that the agent can make decisions about pushing easy tasks into medium or low capacity servers. This does not need to be done to the level server id if the servers truly are interchangeable otherwise. So instead of an array of number of free servers by type, consider that it expands to a table with number of free servers by type at one end, then a column of number of servers by type with 0.1s remaining until they are free, then a column for servers that still have 0.2s remaining etc etc&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-03-05T10:23:42.637" LastActivityDate="2020-03-05T10:23:42.637" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18434" PostTypeId="2" ParentId="18431" CreationDate="2020-03-05T11:01:43.183" Score="3" Body="&lt;p&gt;You can train your bot using reinforcement learning (in particular Q-Learning). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The most important part of the RL is a reward function. If we want agent to do some thing speciﬁc, we must provide rewards to it in such a way that it will achieve our goals. It is thus very important that the reward function accurately indicates the exact behaviour&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So you can construct your own reward function that will satisfy your requirements. If the bot does something you want, you will reward it with higher score, otherwise you will punish it with negative reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlhpaGo and OpenAI teams used a similar technique to train their model which could then beat humans in games like &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Go&lt;/a&gt;, &lt;a href=&quot;https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&quot; rel=&quot;nofollow noreferrer&quot;&gt;StarCraft 2&lt;/a&gt; and &lt;a href=&quot;https://openai.com/blog/openai-five/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dota 2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, check out this &lt;a href=&quot;https://classroom.udacity.com/courses/ud893-preview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning Free Preview&lt;/a&gt; on udacity.&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-03-05T11:27:52.133" LastActivityDate="2020-03-05T11:27:52.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18435" PostTypeId="2" ParentId="18431" CreationDate="2020-03-05T11:18:23.773" Score="9" Body="&lt;p&gt;I would set up a list of goals for your bot. These could be 'maintain a minimum level of health', 'knock out human player', 'block way to location X', etc. This obviously depends on the domain of your MMO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can use a planner to achieve these goals in the game. You define a set of actions with preconditions and effects, set the current goal, and the planner will work out a list of actions for the bot to achieve the goal. You can easily express your actions (and the domain) in &lt;a href=&quot;https://en.wikipedia.org/wiki/Planning_Domain_Definition_Language&quot; rel=&quot;noreferrer&quot;&gt;PDDL&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples for actions would be 'move to location X', 'eat X', 'attack player X'. A precondition of 'attack player X' could be 'health(X) is low', and an effect could be 'health(X) is reduced by Y'. There are different ways of expressing these depending on the planner's capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The beauty of this is that you don't actually have to explicitly code any behaviour. You describe the domain, and tell the bot what it should achieve, and what capabilities it has. The actual behaviour then emerges out of that description. If the bot only attacks a player if the player has lower health, then observing the player eat (and thus up their health) could result in the bot eating (to push its own health above the player's so that it can attack) &amp;mdash; but you have not told the bot directly to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a starting point, go to &lt;a href=&quot;http://education.planning.domains/&quot; rel=&quot;noreferrer&quot;&gt;http://education.planning.domains/&lt;/a&gt; for a list of resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only have a few actions available, it might appear predictable to a human user, but with a variety of goals and actions, this will quickly become more complex and seem more 'intelligent'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; Here is a link to a paper, &lt;em&gt;&lt;a href=&quot;http://alumni.media.mit.edu/~jorkin/GOAP_draft_AIWisdom2_2003.pdf&quot; rel=&quot;noreferrer&quot;&gt;Applying Goal-Oriented Action Planning to Games&lt;/em&gt;&lt;/a&gt;, which describes how this can be applied in a game.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-03-05T15:11:02.993" LastActivityDate="2020-03-05T15:11:02.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18436" PostTypeId="2" ParentId="18425" CreationDate="2020-03-05T11:47:11.183" Score="0" Body="&lt;p&gt;The most important part of the RL is a reward function. If we want agent to do something speciﬁc, we must provide rewards to it in such a way that it will achieve our goals. It is thus very important that the reward function accurately indicates the exact behaviour.&lt;/p&gt;&#xA;&lt;p&gt;Assume, the robot's goal is to reach desired position as fast as possible. You can construct you reward function so, that it will take into account the Euclidean distance to the position. If the arm moves to the position directly, you will reward the agent with a positive value, otherwise you will punish it with a deviation from the direct line. You probably have other parameters of the joints, such as position and velocity. It can be also included in your reward function, in order to find optimal movements.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/newNu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/newNu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Check out this &lt;a href=&quot;https://www.youtube.com/watch?v=pVIFc72VYH8&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; from the free udacity overview course on RL and this paper &lt;a href=&quot;https://arxiv.org/pdf/1803.07067.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Setting up a Reinforcement Learning Task with a Real-World Robot&amp;quot;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is also related DeepMind's &lt;a href=&quot;https://deepmind.com/blog/article/producing-flexible-behaviours-simulated-environments&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1707.02286.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also have a &lt;a href=&quot;https://github.com/karray/neuroracer&quot; rel=&quot;nofollow noreferrer&quot;&gt;project on github&lt;/a&gt;, where I implemented custom Gazebo environment for OpenAI Gym. This allows you to run the test even on a Jupyter Notebook. Check out &lt;a href=&quot;https://github.com/karray/neuroracer/blob/master/q_learning.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;my example&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-11-15T13:50:41.633" LastActivityDate="2020-11-15T13:50:41.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18437" PostTypeId="1" CreationDate="2020-03-05T11:54:02.463" Score="2" ViewCount="47" Body="&lt;p&gt;In the transformer model, to incorporate positional information of texts, the researchers have added a positional encoding to the model. &lt;em&gt;How does positional encoding work? How does the positional encoding system learn the positions when varying lengths and types of text are passed at different time intervals?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;To be more concrete, let's take these two sentences.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&amp;quot;She is my queen&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;Elizabeth is the queen of England&amp;quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How would these sentences be passed to the transformer? What would happen to them during the positional encoding part?&lt;/p&gt;&#xA;&lt;p&gt;Please explain with less math and with more intuition behind it.&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2020-09-17T13:19:07.593" LastActivityDate="2020-09-17T13:19:07.593" Title="How does positional encoding work in the transformer model?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;attention&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18438" PostTypeId="1" AcceptedAnswerId="18444" CreationDate="2020-03-05T12:11:42.180" Score="1" ViewCount="71" Body="&lt;p&gt;I have a question about training a neural network for more epochs even after the network has converged without using early stopping criterion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the MNIST dataset and a LeNet 300-100-10 dense fully-connected architecture, where I have 2 hidden layers having 300 and 100 neurons and an output layer having 10 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, usually, this network takes about 9-11 epochs to train and have a validation accuracy of around 98%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens if I train this network for 25 or 30 epochs, without using early stopping criterion? &lt;/p&gt;&#xA;" OwnerUserId="31215" LastEditorUserId="2444" LastEditDate="2020-03-06T01:37:03.467" LastActivityDate="2020-03-06T06:02:35.797" Title="What happens if I train a network for more epochs, without using early stopping?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;feedforward-neural-network&gt;&lt;early-stopping&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18439" PostTypeId="1" CreationDate="2020-03-05T13:04:17.107" Score="4" ViewCount="105" Body="&lt;p&gt;I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will I get more accurate results when I use a small architecture than when I use multiple layers with a higher amount of neurons? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I use 1 hidden layer with 1 neuron, because of the fact that I need to solve (in my opinion) a basic regression problem. &lt;/p&gt;&#xA;" OwnerUserId="34033" LastEditorUserId="2444" LastEditDate="2020-03-05T13:05:53.563" LastActivityDate="2021-04-30T14:07:30.873" Title="Is a basic neural network architecture better with small datasets?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;datasets&gt;&lt;regression&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18440" PostTypeId="2" ParentId="18438" CreationDate="2020-03-05T13:27:11.133" Score="1" Body="&lt;p&gt;Training a neural network for &quot;too many&quot; epochs than needed without using early stopping criterion leads to overfitting, where your model's ability to generalize decreases.&lt;/p&gt;&#xA;" OwnerUserId="31215" LastActivityDate="2020-03-05T13:27:11.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18441" PostTypeId="1" CreationDate="2020-03-05T15:02:46.563" Score="1" ViewCount="144" Body="&lt;p&gt;I need some help with continuing pre-training on Bert. I have a very specific vocabulary and lots of specific abbreviations at hand. I want to do a STS task. Let me specify my task: I have domainspecific sentences and want to pair them regarding to their semantic similarity. But as very uncommon language is used here, I need to train Bert on it.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How does one continue the pretraining ( I read the github release from google about it, but don't really understand it) Any examples?&lt;/li&gt;&#xA;&lt;li&gt;What structure does my training-data need to have so Bert can understand it?&lt;/li&gt;&#xA;&lt;li&gt;Maybe training Bert from scratch would be even better, I guess its the same process as continuing the pretraining just the starting checkpoint would be different. Is that correct?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also very happy about all other tips from you guys.&lt;/p&gt;&#xA;" OwnerUserId="34026" LastActivityDate="2020-03-05T15:02:46.563" Title="Continuing Pretraining on BERT" Tags="&lt;python&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18442" PostTypeId="2" ParentId="1987" CreationDate="2020-03-05T15:56:34.683" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://playground.tensorflow.org/#activation=tanh&amp;amp;batchSize=5&amp;amp;dataset=spiral&amp;amp;regDataset=reg-plane&amp;amp;learningRate=0.03&amp;amp;regularizationRate=0&amp;amp;noise=0&amp;amp;networkShape=5,4&amp;amp;seed=0.84823&amp;amp;showTestData=false&amp;amp;discretize=true&amp;amp;percTrainData=50&amp;amp;x=true&amp;amp;y=true&amp;amp;xTimesY=false&amp;amp;xSquared=false&amp;amp;ySquared=false&amp;amp;cosX=false&amp;amp;sinX=true&amp;amp;cosY=false&amp;amp;sinY=true&amp;amp;collectStats=false&amp;amp;problem=classification&amp;amp;initZero=false&amp;amp;hideText=false&amp;amp;regularization_hide=false&amp;amp;numHiddenLayers_hide=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; I reached after an hour of trial usually &lt;strong&gt;converges in just 100 epochs&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yeah, I know it does not have the smoothest decision boundary out there, but it converges pretty fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BZLo6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BZLo6.png&quot; alt=&quot;![0.6% test error&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I learned a few things from this spiral experiment:-&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;The output layer should be greater than or equal to the input layer&lt;/strong&gt;. At least that's what I noticed in the case of this spiral problem.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Keep the initial learning rate high&lt;/strong&gt;, like 0.1 in this case, then as you approach a low test error like 3-5% or less, decrease the learning rate by a notch(0.03) or two. This helps in converging faster and avoids jumping around the global minima.&lt;/li&gt;&#xA;&lt;li&gt;You can see the effects of keeping the learning rate high by checking the error graph at the top right.&lt;/li&gt;&#xA;&lt;li&gt;For smaller batch sizes like 1, 0.1 is too high a learning rate as the model fails to converge as it jumps around the global minima.&lt;/li&gt;&#xA;&lt;li&gt;So, if you would like to keep a high learning rate(0.1), keep the batch size high(10) as well. This usually gives a slow yet smoother convergence.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Coincidentally the solution I came up with is very similar to the one provided by &lt;a href=&quot;https://ai.stackexchange.com/users/2492/salvador-dali&quot;&gt;Salvador Dali&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly add a comment, if you find any more intuitions or reasonings.&lt;/p&gt;&#xA;" OwnerUserId="34034" LastEditorUserId="34034" LastEditDate="2020-03-05T18:12:23.343" LastActivityDate="2020-03-05T18:12:23.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18443" PostTypeId="2" ParentId="18407" CreationDate="2020-03-05T16:20:40.220" Score="0" Body="&lt;p&gt;When using an autoencoder, I believe the data u feed in has to be correlated in one way or another. For example, If i want to learn a latent representation of an image of a cat, The training data that I feed into the autoencoder should constitute only cat images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar to other neural networks, you feed the autoencoder with a set of training data and hope that the network learns a set of weights that is able to output from the latent representation the exact image. To see whether the weights learnt by the autoencoder is able to generalise to other unseen cat images, you would have to use a test set for this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a paper about autoencoding. &lt;a href=&quot;https://web.stanford.edu/class/cs294a/sparseAutoencoder_2011new.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/class/cs294a/sparseAutoencoder_2011new.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps you somewhat in deciding whether you should use an autoencoder.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-03-05T16:20:40.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18444" PostTypeId="2" ParentId="18438" CreationDate="2020-03-05T17:15:55.270" Score="1" Body="&lt;p&gt;Running for &quot;to many&quot; epochs can indeed lead to over fitting. You should look at the validation loss. If on AVERAGE it continues to decrease then you are not yet over fitting.&#xA;You may be tempted to run more epochs in hopes your loss will decrease but unless you adjust your learning rate dynamically  at some point you won't get any improvement.&#xA;If you use KERAS it has a useful callback ReduceLROnPlateau. Documentation is at &lt;a href=&quot;https://keras.io/callbacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/callbacks/&lt;/a&gt;&#xA;This allows you to monitor a metric (typically validation loss) and to adjust the learning rate by a user defined factor(parameter factor) if the metric you are monitoring fails to improve after a certain number of consecutive epochs(parameter patience). You can think of the training process as travelling down a valley in N space(N being the number of trainable parameters). As you descend towards a minimum they valley gets narrower. If you do not lower the learning rate you will reach a point where you can not descend any further.Now you could use a very small learning rate to begin with but then you will have to train for a lot more epochs. One problem with adjusting the learning rate just on the validation loss is that in the early training epochs validation loss often does not track with training accuracy and it could cause the learning rate to be decreased prematurely.&#xA;I wrote a custom callback which initially monitors training loss and adjust the learning rate based on that metric. Once the training accuracy reaches 95% it switches to monitoring validation loss and adjusts the learning rate based on that.  It saves the model weights for the lowest validation loss in the variable val.best_weights. After training load these weights into your model to make predictions. Code is below if you are interested. When you compile your model just add 'val' to the callback list.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; class val(tf.keras.callbacks.Callback):&#xA;        # functions in this class adjust the learning rate &#xA;        lowest_loss=np.inf&#xA;        lowest_trloss=np.inf&#xA;        best_weights=model.get_weights()&#xA;        lr=float(tf.keras.backend.get_value(model.optimizer.lr))&#xA;        epoch=0&#xA;        highest_acc=0&#xA;&#xA;        def __init__(self):&#xA;            super(val, self).__init__()&#xA;            self.lowest_loss=np.inf&#xA;            self.lowest_trloss=np.inf&#xA;            self.best_weights=model.get_weights()&#xA;            self.lr=float(tf.keras.backend.get_value(model.optimizer.lr))&#xA;            self.epoch=0&#xA;            self.highest_acc=0&#xA;&#xA;&#xA;        def on_epoch_end(self, epoch, logs=None):             &#xA;            val.lr=float(tf.keras.backend.get_value(self.model.optimizer.lr))&#xA;            val.epoch=val.epoch +1            &#xA;            v_loss=logs.get('val_loss')&#xA;            v_acc=logs.get('accuracy')&#xA;            loss=logs.get('loss')&#xA;            if loss&amp;lt;val.lowest_trloss:&#xA;                val.lowest_trloss=loss&#xA;                if v_acc&amp;lt;.90:&#xA;                    val.best_weights=model.get_weights()&#xA;            if v_acc&amp;lt;=.95 and loss&amp;gt;val.lowest_trloss:&#xA;                lr=float(tf.keras.backend.get_value(self.model.optimizer.lr))&#xA;                ratio=val.lowest_trloss/loss  # add a factor to lr reduction&#xA;                new_lr=lr * .7 * ratio&#xA;                tf.keras.backend.set_value(model.optimizer.lr, new_lr)&#xA;                msg='{0}\n current training loss {1:7.5f}  is above lowest training loss of {2:7.5f}, reducing lr to {3:11.9f}{4}'&#xA;                print(msg.format(Cyellow, loss, val.lowest_trloss, new_lr,Cend))   &#xA;            if val.lowest_loss &amp;gt; v_loss:&#xA;                msg='{0}\n validation loss improved,saving weights with validation loss= {1:7.4f}\n{2}'&#xA;                print(msg.format(Cgreen, v_loss, Cend))&#xA;                val.lowest_loss=v_loss&#xA;                val.best_weights=model.get_weights()&#xA;&#xA;            else:&#xA;                 if v_acc&amp;gt;.95 and val.lowest_loss&amp;lt;v_loss:&#xA;                        # reduce learning rate based on validation loss&amp;gt; val.best_loss&#xA;                        lr=float(tf.keras.backend.get_value(self.model.optimizer.lr))&#xA;                        ratio=val.lowest_loss/v_loss  # add a factor to lr reduction&#xA;                        new_lr=lr * .7 * ratio&#xA;                        tf.keras.backend.set_value(model.optimizer.lr, new_lr)&#xA;                        msg='{0}\n current loss {1:7.4f} exceeds lowest loss of {2:7.4f}, reducing lr to {3:11.9f}{4}'&#xA;                        print(msg.format(Cyellow, v_loss, val.lowest_loss, new_lr,Cend))```&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="33976" LastEditDate="2020-03-06T06:02:35.797" LastActivityDate="2020-03-06T06:02:35.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18445" PostTypeId="2" ParentId="18431" CreationDate="2020-03-05T17:24:31.830" Score="8" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/18435/34038&quot;&gt;Oliver Mason's answer&lt;/a&gt; is great for specific methods and tools to use, but I wanted to pull out a more general principle which was mentioned in a comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distinction your friend is making is not one that would be generally recognised. One of my university lecturers defined AI as something like &quot;an artificial system that exhibits behaviour that resembles how an intelligent being would behave&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an intelligent being would &lt;em&gt;always&lt;/em&gt; use the special attack in a particular situation, then an algorithm that always does so in the same situation is behaving intelligently, even though the algorithm behind it is incredibly simple. If you can come up with a complete description of an intelligent player, you have what is called an &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;noreferrer&quot;&gt;expert system&lt;/a&gt;, i.e. a system which captures the decision-making process of a real expert.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your friend is not even correct that your proposed AI &quot;does not have any heuristic functions&quot;. When you write a condition like &quot;if the AI's health is below 50%, it will eat food&quot;, you're &lt;em&gt;approximating&lt;/em&gt; the rule a human would use. You can make the heuristic &lt;em&gt;more complex&lt;/em&gt; by increasing the probability of eating in proportion to current health; that might in turn make the heuristic &lt;em&gt;closer to optimal&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can only really find out how &quot;good&quot; your AI is by putting it into different situations and observing it - sometimes, a simple set of rules gives rise to &quot;emergent behaviours&quot; that look surprisingly intelligent. As you build up more complex rules - i.e. more optimal heuristics - the emergent behaviour will change, and you can tweak it for the desired effect.&lt;/p&gt;&#xA;" OwnerUserId="34038" LastActivityDate="2020-03-05T17:24:31.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18446" PostTypeId="1" CreationDate="2020-03-05T17:26:00.247" Score="1" ViewCount="16" Body="&lt;p&gt;In Computer Vision, feature encoding methods are used on pre-trained DCNN to increase the feature robustness to certain conditions such as viewpoint/appearance variations &lt;a href=&quot;https://arxiv.org/abs/1701.05105&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was just wondering if there are already available well established methods among AI community with probably python implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the following ones in literature but without any tutorial or code example:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Multi-layer pooling &lt;a href=&quot;https://arxiv.org/abs/1406.4729&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Cross convolutional layer Pooling &lt;a href=&quot;https://arxiv.org/pdf/1510.00921.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Holistic Pooling &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Babenko_Aggregating_Local_Deep_ICCV_2015_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31312" LastActivityDate="2020-03-05T17:26:00.247" Title="Convolutional Feature Encoding Methods in DCNN" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;feature-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18448" PostTypeId="1" CreationDate="2020-03-05T20:05:28.857" Score="2" ViewCount="36" Body="&lt;p&gt;I currently have a grid of pixels 20x20. Each pixel can be red green blue or black. So I have one hot-encoded the pixels giving a 20x20x4 array for each screen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my Deep-Q Network, I have attached two successive screenshots of the screen together giving a 20x20x4x2 array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to build a Convolutional Neural Network to estimate the Q values but I am not sure if my current architecture is a good idea. It currently is as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def create_model(self):&#xA;        model = Sequential()&#xA;        model.add(Conv3D(256, (4, 4,2), input_shape=(20,20,4,2)))&#xA;        model.add(Activation('relu'))&#xA;        model.add(Dropout(0.2))&#xA;&#xA;        model.add(Conv3D(256, (2,2,1), input_shape=self.input_shape))&#xA;        model.add(Activation('relu'))&#xA;&#xA;        model.add(Flatten())&#xA;        model.add(Dense(64))&#xA;        model.add(Dense(self.num_actions, activation='linear'))&#xA;        model.compile(loss='mse', optimizer=Adam(self.learning_rate), metrics=['accuracy'])&#xA;        return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is a 3d convolution a good idea?&#xA;Is 256 filters a good idea?&#xA;Are the filters (4,4,2) and (2,2,1) suitable?&#xA;I realise answers may be highly subjective but I'm just looking for someone to point out any immediate flaws in the architecture.&lt;/p&gt;&#xA;" OwnerUserId="33227" LastActivityDate="2020-03-05T22:07:57.377" Title="How to use convolution neural network in Deep-Q?" Tags="&lt;deep-learning&gt;&lt;q-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18449" PostTypeId="1" CreationDate="2020-03-05T21:26:11.270" Score="1" ViewCount="61" Body="&lt;p&gt;I am trying to use &lt;a href=&quot;https://github.com/huggingface/transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch's transformers&lt;/a&gt; as a part of a research project to do sentiment analysis of several types of review data (laptop and restaurant). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, my team is taking a token-based approach and we are using models that can perform token analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem we have encountered is that many of the models in &lt;a href=&quot;https://github.com/huggingface/transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch's transformers&lt;/a&gt; do not support token classification, but do support sequence classification. One such model we wanted to test is GPT-2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to overcome this, we proposed using sequence classifiers on single tokens which should work in theory, but possibly at reduced accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This raises the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to do token classification using a model such as GPT-2 using &lt;a href=&quot;https://github.com/huggingface/transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch's transformers&lt;/a&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do sequence classifiers perform on single token sequences?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18870" LastEditorUserId="2444" LastEditDate="2020-03-06T14:05:29.963" LastActivityDate="2020-03-06T14:05:29.963" Title="Is it possible to do token classification using a model such as GPT-2?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;pytorch&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18450" PostTypeId="2" ParentId="18439" CreationDate="2020-03-05T21:34:18.527" Score="0" Body="&lt;p&gt;It's harder to overfit it certainly!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean practically speaking there has to be some assumptions on the generation model of your data, either explicit or implicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try probably 1-2 layer network first(maybe your data is linearly separable if you're lucky). &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-03-05T21:34:18.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18451" PostTypeId="1" CreationDate="2020-03-05T21:52:52.377" Score="1" ViewCount="27" Body="&lt;p&gt;I wrote a script to do train a Siamese Network style model for face recognition on &lt;a href=&quot;http://vis-www.cs.umass.edu/lfw/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LFW&lt;/a&gt; dataset but the training loss doesnt decrease at all. Probably there's a bug in my implementation. Could you please point it out.&#xA;Right now my code does:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each epoch has 0.5M triplets all generated in an online way from data (since the exhaustive number of triplets is too big).&lt;/li&gt;&#xA;&lt;li&gt;Triplet sampling method: We have a dictionary of &lt;code&gt;{class_id: list of file paths with that class id}&lt;/code&gt;. We then create a list of classes which we can use for positive class (some classes have only 1 image so cant be used as positive class). At any iteration we randomly sample a positive class from this refined list and a negative class from the original list. We randomly sample 2 images from positive (as Anchor or A and Positive as P) and 1 from negative (Negative or N). A,P,N form our triplet.&lt;/li&gt;&#xA;&lt;li&gt;Model used is ResNet with the ultimate (512,1000) softmax layer is replaced with (512,128) Dense layer (no activation). To avoid overfitting, only the last Dense and layer4 are kept trainable and rest are frozen.&lt;/li&gt;&#xA;&lt;li&gt;During training we find triplets which are semi-hard in a batch (Loss between 0 and margin) and use only those to do backprop (they mention this in the FaceNet paper)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from torchvision import models, transforms&#xA;from torch.utils.data import Dataset, DataLoader&#xA;import torch, torch.nn as nn, torch.optim as optim&#xA;from torch.utils.tensorboard import SummaryWriter&#xA;import os, glob&#xA;import numpy as np&#xA;from PIL import Image&#xA;&#xA;image_size = 224&#xA;batch_size = 512&#xA;margin = 0.5&#xA;learning_rate = 1e-3&#xA;num_epochs = 1000&#xA;&#xA;model = models.resnet18(pretrained=True)&#xA;model.fc = nn.Linear(model.fc.in_features, 128, bias=False)&#xA;&#xA;for param in model.parameters():&#xA;  param.requires_grad = False&#xA;for param in model.fc.parameters():&#xA;  param.requires_grad = True&#xA;for param in model.layer4.parameters():&#xA;  param.requires_grad = True&#xA;&#xA;optimizer = optim.Adam(params=list(model.fc.parameters())+list(model.layer4.parameters()), lr=learning_rate, weight_decay=0.05)&#xA;&#xA;device = torch.device(&quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;)&#xA;model = nn.DataParallel(model).to(device)&#xA;writer = SummaryWriter(log_dir=&quot;logs/&quot;)&#xA;&#xA;class TripletDataset(Dataset):&#xA;  def __init__(self, rootdir, transform):&#xA;    super().__init__()&#xA;    self.rootdir = rootdir&#xA;    self.classes = os.listdir(self.rootdir)&#xA;    self.file_paths = {c: glob.glob(os.path.join(rootdir, c, &quot;*.jpg&quot;)) for c in self.classes}&#xA;    self.positive_classes = [c for c in self.classes if len(self.file_paths[c])&amp;gt;=2]&#xA;    self.transform = transform&#xA;&#xA;  def __getitem__(self, index=None):&#xA;    class_pos, class_neg = None, None&#xA;    while class_pos == class_neg:&#xA;      class_pos = np.random.choice(a=self.positive_classes, size=1)[0]&#xA;      class_neg = np.random.choice(a=self.classes, size=1)[0]&#xA;&#xA;    fp_a, fp_p = np.random.choice(a=self.file_paths[class_pos], size=2, replace=False)&#xA;    fp_n = np.random.choice(a=self.file_paths[class_neg], size=1)[0]&#xA;&#xA;    return {&#xA;        &quot;fp_a&quot;: fp_a,&#xA;        &quot;fp_p&quot;: fp_p,&#xA;        &quot;fp_n&quot;: fp_n,&#xA;        &quot;A&quot;: self.transform(Image.open(fp_a)),&#xA;        &quot;P&quot;: self.transform(Image.open(fp_p)),&#xA;        &quot;N&quot;: self.transform(Image.open(fp_n)),&#xA;            }&#xA;&#xA;  def __len__(self):&#xA;    return 500000&#xA;&#xA;&#xA;def triplet_loss(a, p, n, margin=margin):&#xA;    d_ap = (a-p).norm(p='fro', dim=1)&#xA;    d_an = (a-n).norm(p='fro', dim=1)&#xA;    loss = torch.clamp(d_ap-d_an+margin, min=0)&#xA;    return loss, d_ap.mean(), d_an.mean()&#xA;&#xA;transform = transforms.Compose([&#xA;        transforms.RandomResizedCrop(image_size),&#xA;        transforms.RandomHorizontalFlip(),&#xA;        transforms.ToTensor(),&#xA;        transforms.Normalize([0.596, 0.436, 0.586], [0.2066, 0.240, 0.186])&#xA;        ])&#xA;train_dataset = TripletDataset(&quot;lfw&quot;, transform)&#xA;nw = 4 if torch.cuda.is_available() else 0&#xA;train_dataloader = DataLoader(train_dataset, batch_size=batch_size, num_workers=0, shuffle=True)&#xA;&#xA;num_batches = len(train_dataloader)&#xA;model.train()&#xA;running_loss = 0&#xA;&#xA;for epoch in range(num_epochs):&#xA;    for batch_id, dictionary in enumerate(train_dataloader):&#xA;        a, p, n = dictionary[&quot;A&quot;], dictionary[&quot;P&quot;], dictionary[&quot;N&quot;]&#xA;        a, p, n = a.to(device), p.to(device), n.to(device)&#xA;        emb_a, emb_p, emb_n = model(a), model(p), model(n)&#xA;        losses, d_ap, d_an = triplet_loss(a=emb_a, p=emb_p, n=emb_n)&#xA;&#xA;        semi_hard_triplets = torch.where((losses&amp;gt;0) &amp;amp; (losses&amp;lt;margin))&#xA;        losses = losses[semi_hard_triplets]&#xA;        loss = losses.mean()&#xA;        loss.backward()&#xA;        optimizer.step()  &#xA;        running_loss += loss.item()&#xA;&#xA;        print(&quot;Epoch {} Batch {}/{} Loss = {} Avg AP dist = {} Avg AN dist = {}&quot;.format(epoch, batch_id, num_batches, loss.item(), d_ap.item(), d_an.item()), flush=True)&#xA;        writer.add_scalar(&quot;Loss/Train&quot;, loss.item(), epoch*num_batches+batch_id)&#xA;        writer.add_scalars(&quot;AP_AN_Distances&quot;, {&quot;AP&quot;: d_ap.item(), &quot;AN&quot;: d_an.item()}, epoch*num_batches+batch_id)&#xA;&#xA;    print(&quot;Epoch {} Avg Loss {}&quot;.format(epoch, running_loss/num_batches), flush=True)&#xA;    writer.add_scalar(&quot;Epoch_Loss&quot;, running_loss/num_batches, epoch)&#xA;    torch.save(model.state_dict(), &quot;facenet_epoch_{}.pth&quot;.format(epoch))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Loss graphs: &lt;a href=&quot;https://tensorboard.dev/experiment/8TgzPTjuRCOFkFV5lr5etQ/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://tensorboard.dev/experiment/8TgzPTjuRCOFkFV5lr5etQ/&lt;/a&gt;&#xA;Please let me know if you need some other information to help you help me.&lt;/p&gt;&#xA;" OwnerUserId="19522" LastEditorUserId="19522" LastEditDate="2020-03-05T22:17:39.980" LastActivityDate="2020-03-05T22:17:39.980" Title="Face recognition model loss not decreasing" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18452" PostTypeId="2" ParentId="18448" CreationDate="2020-03-05T22:07:57.377" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is a 3d convolution a good idea? Is 256 filters a good idea? Are the filters (4,4,2) and (2,2,1) suitable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's not so much that answers are subjective, but you are performing an experiment, and this should be driven by &lt;em&gt;results&lt;/em&gt;. If you can find something published about a similar environment that might help you narrow down your choices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, intuitively you don't gain much going from 2d to 3d convolutions when one of your dimensions is only sized 2. Purely from gut feeling I would suggest simply concatenating your two frames by having 8 channels instead of 4, and use 2d filters. This is simple to enough try and compare though, so perhaps you can do both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will likely want to explore deeper networks that have fewer initial filters and build up over a few more than two convolutional layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Definitely try your network without the dropout layer. I have not had much luck with using dropout in DQN, and I have seen others having similar problems. I am not sure what the exact issue is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accuracy metric won't do you much good on a regression task, so you can drop that and just use MSE loss. Bear in mind that training loss is a less useful metric overall in RL, since the prediction target (action value) is continuously changing as the policy changes. Low loss values mean that the value predictions are self-consistent, they don't necessarily mean that the learning has converged to an optimal policy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-03-05T22:07:57.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18453" PostTypeId="2" ParentId="18418" CreationDate="2020-03-05T23:32:22.187" Score="2" Body="&lt;p&gt;Philosophically, my own research has led me to understand AI as any artifact that makes a decision.  This is because the etymology of &quot;intelligence&quot; strongly implies &quot;selecting between alternatives&quot;, and these meanings are baked in all the way back to the proto-Indo-European.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Degree of intelligence, or &quot;strength&quot; is merely a measure of utility, typically versus other decision making mechanisms, or, &quot;fitness in an environment&quot;, where an environment is any action space.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the most basic form of automated (artificial) intelligence is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if [some condition] &#xA;then [some action]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is worth noting that &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo&quot; rel=&quot;nofollow noreferrer&quot;&gt;narrow AI&lt;/a&gt; which matches or exceeds human capability, in the popular sense, manifested only recently when we had sufficient processing and memory to derive sufficient utility from statistical decision making algorithms. But &lt;a href=&quot;https://en.wikipedia.org/wiki/Nimatron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nimatron&lt;/a&gt; constitutes perhaps the first functional strong-narrow AI in a modern computing context, and the first automated intelligence are &lt;a href=&quot;https://ai.stackexchange.com/questions/13865/are-simple-animal-snares-and-traps-a-form-of-automation-of-computation&quot;&gt;simple traps and snares&lt;/a&gt;, which have been with us almost as long as we've used tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will leave it to others to break down all the various forms of &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;modern AI&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-03-06T00:24:23.933" LastActivityDate="2020-03-06T00:24:23.933" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18454" PostTypeId="1" CreationDate="2020-03-06T01:57:49.093" Score="1" ViewCount="49" Body="&lt;p&gt;I am looking for a technique to train a machine learning model to &lt;em&gt;choose&lt;/em&gt; two items from a list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, given a list &lt;span class=&quot;math-container&quot;&gt;$x=[x_1, x_2, x_3, x_4, \dots, x_n]$&lt;/span&gt;, the model needs to choose two elements &lt;span class=&quot;math-container&quot;&gt;$(x_i, x_j)$&lt;/span&gt;. I have a function &lt;span class=&quot;math-container&quot;&gt;$R(x, x_i, x_j)$&lt;/span&gt;, which will output the reward of choosing &lt;span class=&quot;math-container&quot;&gt;$(x_i, x_j)$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What type of models should I use, and how should I train it to maximize the reward? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried using deep reinforcement learning, but I ran into the following problems with implementing the Q-Network:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Variable-length inputs (fixed by using RNN, I think)&lt;/li&gt;&#xA;&lt;li&gt;The output size grows factorially (for an input set of n elements, there are n choose 2 ways to pick 2 elements, so the network needs to output n choose 2 expected rewards)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="34050" LastEditorUserId="2444" LastEditDate="2020-03-07T04:36:26.740" LastActivityDate="2020-03-07T04:36:26.740" Title="Which model should I choose to maximise reward of having chosen two numbers from a list?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18458" PostTypeId="1" CreationDate="2020-03-06T04:06:48.743" Score="1" ViewCount="53" Body="&lt;p&gt;This question is a bit philosophic and is about making new use cases for software companies. Let me describe what exist for now, why it is not enough, and what is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there are a lot of &lt;a href=&quot;https://github.com/src-d/awesome-machine-learning-on-source-code&quot; rel=&quot;nofollow noreferrer&quot;&gt;existing researches&lt;/a&gt; in applying ML for software (please don't simply point to this one!), but none of them consider the application of ML for &lt;strong&gt;software company&lt;/strong&gt;, not the software alone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Existent approaches that apply AI for software engineering tasks consider it as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;human1 -&amp;gt; software (big code) &amp;lt;- human2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That means that &lt;code&gt;human1&lt;/code&gt; makes some part of &lt;code&gt;software&lt;/code&gt; (that is a part of &lt;code&gt;big code&lt;/code&gt;), and &lt;code&gt;human2&lt;/code&gt; reuses some knowledge from it. It may be a bugfix pattern (as e.g. &lt;a href=&quot;https://www.deepcode.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepCode&lt;/a&gt; does), API usage pattern, repair of code, summarization of code, code search, or whatever else. I think the main reason for this is the original &lt;em&gt;hypothesis of naturalness&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;The naturalness hypothesis&lt;/strong&gt;. Software is a form of human communication; software corpora have similar statistical properties to natural language corpora, and these properties can be exploited to build better software engineering tools.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(from &lt;a href=&quot;https://arxiv.org/pdf/1709.06182.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Allamanis et al&lt;/a&gt;, page 3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But imagine one software company. It has:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some number of engineers,&lt;/li&gt;&#xA;&lt;li&gt;Some number of managers,&lt;/li&gt;&#xA;&lt;li&gt;The software product,&lt;/li&gt;&#xA;&lt;li&gt;Information related to the software product (documentation, bug/task tracking system, code review),&lt;/li&gt;&#xA;&lt;li&gt;Some number of formal management processes (waterfall, scrum or whatever else),&lt;/li&gt;&#xA;&lt;li&gt;Some number of informal processes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But none of these models consider the software as a product itself. I mean that we should consider the model as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;company -&amp;gt; software product -&amp;gt; customers&#xA;              |&#xA;              v&#xA;           big code&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or even&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;engineer1 -&amp;gt; |&#xA;    |&#xA;engineer2 -&amp;gt; |&#xA;    |&#xA;...          | ----&amp;gt; software product ----&amp;gt; customers&#xA;    |                   |&#xA;engineerN -&amp;gt; |          |&#xA;    |                   |&#xA;manager  --&amp;gt; |          |&#xA;                        v&#xA;                     big code&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are there any cases of investigation of such models? &lt;/li&gt;&#xA;&lt;li&gt;Are there any similar cases in related fields, say in general companies (not specifically software ones)?&lt;/li&gt;&#xA;&lt;li&gt;Are there any analogies (not specifically from software-related domains) where some knowledge can be transferred from a bigger object (&lt;code&gt;big code&lt;/code&gt; in our case) to a smaller one (&lt;code&gt;software product&lt;/code&gt;)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any ideas are welcome.&lt;/p&gt;&#xA;" OwnerUserId="16354" LastActivityDate="2020-03-06T04:06:48.743" Title="Use cases for AI inside the software company" Tags="&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;philosophy&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18459" PostTypeId="1" AcceptedAnswerId="18463" CreationDate="2020-03-06T06:58:21.853" Score="3" ViewCount="64" Body="&lt;p&gt;The DQN implements replay memory. Based on my research, I believe the replay memory starts to get used for training once there is enough experience in the memory buffer. This means the neural network gets trained while the game plays. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, if I am to play the game 10000 epochs, store all the experiences and then train from the experiences would that have the same effect as training and while running through 10000 epochs? Is it frowned upon to do it this way? Are there any advantages?&lt;/p&gt;&#xA;" OwnerUserId="34054" LastEditorUserId="2444" LastEditDate="2020-03-06T20:36:36.977" LastActivityDate="2020-03-06T20:36:36.977" Title="Can experience replay be used for training after completing every single epoch?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18460" PostTypeId="2" ParentId="18397" CreationDate="2020-03-06T07:06:23.983" Score="0" Body="&lt;p&gt;It is a bit rare that the validation and test  accuracy exceed the training accuracy. One thing that could cause this is the selection of the validation and test data. Was the data for these two sets selected randomly or did you do the selection yourself? It is generally better to have these sets selected randomly from  the overall data  set. That way the probability distribution in these sets will closely match the distribution of the training set. Normally the training accuracy is higher (especially if you run enough epochs which I see you did) because there is always some degree of over fitting which reduces validation and test accuracy. The only other thing I can think of is the effect of Dropout layers. If you had dropout layers in your model and the drop out ratio was high that could cause this accuracy disparity. When the training accuracy is calculated it is done with drop being active. This can lower the training accuracy to some degree. However when evaluating validation accuracy and test accuracy drop out is NOT active so the model is actually more accurate. This increase in accuracy might be enough to overcome the decrease due to over fitting. Especially possible in this case since the accuracy differences appear to be quite small.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-06T07:06:23.983" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18462" PostTypeId="2" ParentId="12423" CreationDate="2020-03-06T07:55:23.107" Score="1" Body="&lt;p&gt;I developed a python script to crop faces using MTCNN. I found this to be the most accurate of all the face cropping algorithms at the expense of being somewhat slower.&#xA;The function I developed is on the kaggle website at &lt;a href=&quot;https://www.kaggle.com/gpiosenka/detect-align-resize-rename-facial-images&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/gpiosenka/detect-align-resize-rename-facial-images&lt;/a&gt;.&#xA;The markup first cell explains how to use it. In a nutshell this function will crop the image files in the input directory and store them in an output directory. If there are multiple faces in an image it selects and crops the largest facial image. It also gives you the option to &quot;align&quot; the cropped images. By align I mean the cropped image is rotated so the eyes are horizontal. You also have the option to resize the cropped image. Finally you also have the option to rename the images in a numerically sequential order and to change the image format to one you specify. This is convenient because if you download images from the internet they have ungodly names and come in a vast variety of image formats. Also the function checks if any images are duplicates of each other. If duplicates are present a listing is provided and you can elect to delete duplicate images. Normally when you build a data set you want to eliminate duplicates so that if you split the data set into train, test and validation sets there is no &quot;leakage&quot; between the sets.&#xA;I have used this function on thousands of images  that have faces in them and it works extremely well. The cropped results are just what you want.On rare occasions a cropping error may occur so it is always wise to review the cropped results. &lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-06T07:55:23.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18463" PostTypeId="2" ParentId="18459" CreationDate="2020-03-06T08:18:09.013" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is, if I am to play the game 10000 epochs, store all the experiences and then train from the experiences would that have the same effect as training and while running through 10000 epochs? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, it will not. In general, for anything other than simple environments, this will give a worse result. The trouble is during those 10,000 epochs you will have been working with your initial behaviour policy to collect experience, which will not be close to optimal. Off-policy learning adjusts for that and attempts to learn the value function of current best-guess optimal policy. However, this is not perfect, and it will learn better the closer your behaviour policy is to the target policy. An initial random policy will in many cases be too different from the target policy for learning to be effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two main causes of imperfection in off-policy learning:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Variance. The more discrepancy between behaviour and target policies there is, the higher the variance will be in the sampled values, and more samples will be required to get the same accuracy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sampling bias. The distribution of observed states and actions will affect function approximators such as neural networks, which reduce loss functions against an assumed population on input/output pairs. Unless adjusted somehow (and basic Q-learning in DQN does not adjust this), the population you train against will be from the behaviour policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any advantages?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It may be faster, and result in cleaner design, to separate data gathering and learning components. If you have a distributed architecture you can dedicate some machines to generating the experience and others to analysing it. However, this still benefits from routine updates to behaviour policies used to collect experience, based on learning so far.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-03-06T20:19:59.687" LastActivityDate="2020-03-06T20:19:59.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18464" PostTypeId="1" AcceptedAnswerId="18465" CreationDate="2020-03-06T09:37:29.237" Score="3" ViewCount="72" Body="&lt;p&gt;I have a lot of empty values in my dataset, so I want to let my neural network 'learn more' on the rows that have no empty values because these rows are of higher importance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this?&lt;/p&gt;&#xA;" OwnerUserId="34033" LastActivityDate="2020-03-06T10:12:58.267" Title="Can you let specific data impact a neural network more than other data?" Tags="&lt;neural-networks&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18465" PostTypeId="2" ParentId="18464" CreationDate="2020-03-06T10:00:19.950" Score="3" Body="&lt;p&gt;Yes, you can weight the loss function for each example, so that instead of your cost function being&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J = \sum_i \mathcal{L}(y_i, \hat{y}_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J = \sum_i w_i\mathcal{L}(y_i, \hat{y}_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; iterates over your data set, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; is the loss function you are using, &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is ground truth for each example and &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt; is prediction for each example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can set your relevance weighting according to any criteria you like based on the example, and your dataset/goals. So for instance you could set it to &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt; for complete examples and &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; for incomplete examples. Depending on your NN framework, it may already offer per-example weighting, and even if it does not, auto-differentiation means that typically you only need to implement the forward logic into the cost function for each minibatch, and the weighting will be applied correctly to gradients with no more work required. If you do need to implement gradient calculations yourself, you simply multiply the initial gradient of each example by the example's weight &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you make a change like this, you need to take care on how you set up and interpret your test set. When your model is used in production, if there are still inputs with missing details, you probably don't want to weight the metrics (e.g. accuracy rating) in the same way, but to report it as-is against a correctly sampled dataset of unseen examples from the population of expected inputs.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-03-06T10:12:58.267" LastActivityDate="2020-03-06T10:12:58.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18466" PostTypeId="1" CreationDate="2020-03-06T12:07:00.610" Score="1" ViewCount="157" Body="&lt;p&gt;I am implementing a project on pomegranate plant disease in Machine learning. I want a dataset of all kind images of a healthy and unhealthy part of the pomegranate plant. I got a dataset from Fruit360 but that is only for pomegranate fruits but need for leaves also. Is there anyone who knows any website, link, version control system repository and/or any resource from which I get a dataset for leaves.&lt;/p&gt;&#xA;" OwnerUserId="34065" LastEditorUserId="34065" LastEditDate="2020-03-11T04:08:54.523" LastActivityDate="2020-03-11T04:10:54.940" Title="Image dataset for pomegranate plant disease" Tags="&lt;machine-learning&gt;&lt;image-recognition&gt;&lt;datasets&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18467" PostTypeId="2" ParentId="18466" CreationDate="2020-03-06T13:37:12.987" Score="1" Body="&lt;p&gt;You can find the dataset in the following links:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S187705091502133X&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pomegranate Disease Detection Using Image Processing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.kaggle.com/moltean/fruits&quot; rel=&quot;nofollow noreferrer&quot;&gt;fruits 360 datasets&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33875" LastActivityDate="2020-03-06T13:37:12.987" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18468" PostTypeId="1" CreationDate="2020-03-06T14:53:13.503" Score="2" ViewCount="90" Body="&lt;p&gt;I was thinking about the risks of &lt;em&gt;Oracle AI&lt;/em&gt; and it doesn't seem as safe to me as Bostrom et al. &lt;a href=&quot;http://www.aleph.se/papers/oracleAI.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;suggest&lt;/a&gt;. From my point of view, even an AGI that only answers questions could have a catastrophic impact. Thinking about it a little bit, I came up with this &lt;em&gt;Proof&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Lemma&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We are not safe even by giving the oracle the ability to only answer yes or no.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Proof&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let's say that our oracle must maximize an utility function &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, there is a procedure that encodes the optimality of &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;. Since a procedure is, in fact, a set of instructions (an algorithm), each procedure can be encoded as a binary string, composed solely of 0 and 1, Therefore we will have &lt;span class=&quot;math-container&quot;&gt;$\phi \in {\{0,1\}^n}$&lt;/span&gt;,  assuming that the optimal procedure has finite cardinality.  Shannon's entropy tells us that every binary string can be guessed by answering only yes/no to questions like: is the first bit 0? and so on, therefore we can reconstruct any algorithm via binary answers (yes / no).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is this reasoning correct and applicable to this type of AI?&lt;/p&gt;&#xA;" OwnerUserId="30353" LastEditorUserId="2444" LastEditDate="2020-03-06T19:20:20.390" LastActivityDate="2020-03-14T20:57:47.407" Title="Is an oracle that answers only with a &quot;yes&quot; or &quot;no&quot; dangerous?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;proofs&gt;&lt;risk-management&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18469" PostTypeId="1" AcceptedAnswerId="18471" CreationDate="2020-03-06T16:48:43.213" Score="2" ViewCount="59" Body="&lt;p&gt;I'm training a DCGAN model on a 320x320 dataset of images and after an hour of training the generator started to generate (on the same latent space noise as during training) images that are identical to the dataset. For example, if my dataset is images of cars, I should expect to see unexisting designs of cars, right? Am I understanding this wrong? I know this is a very general question but I was wondering if this is what should happen and if I should try on different latent space values and then see proper results and not just copies of my dataset?&lt;/p&gt;&#xA;" OwnerUserId="32539" LastActivityDate="2020-03-06T17:38:14.643" Title="Am I overfitting my GAN model?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18470" PostTypeId="1" AcceptedAnswerId="18473" CreationDate="2020-03-06T16:54:16.337" Score="1" ViewCount="58" Body="&lt;p&gt;I am trying to build an LSTM model to generate Shakspeare-like poems. I have data set &lt;span class=&quot;math-container&quot;&gt;$\{s_1, s_2, \dots, s_m \}$&lt;/span&gt;, which are sentences of Shakespeare poems, and each sentence contains words &lt;span class=&quot;math-container&quot;&gt;$\{w_1, w_2, \dots, w_n \}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering:  Are different &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, m$&lt;/span&gt;) independent and identically distributed samples (IID)? Are &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, n$&lt;/span&gt;) within each sentence the IID?&lt;/p&gt;&#xA;" OwnerUserId="33266" LastEditorUserId="33266" LastEditDate="2020-03-08T01:24:57.953" LastActivityDate="2020-03-08T01:24:57.953" Title="Are sentences from the same document independent and identically distributed?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;long-short-term-memory&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18471" PostTypeId="2" ParentId="18469" CreationDate="2020-03-06T17:38:14.643" Score="2" Body="&lt;p&gt;It might be that your dataset of images is to small. Your discriminative network might hardlearn these images at which point your generative network can only produce good images if it copies the same images of your dataset.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2020-03-06T17:38:14.643" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18473" PostTypeId="2" ParentId="18470" CreationDate="2020-03-06T21:10:45.937" Score="3" Body="&lt;p&gt;The sentences coming from the same document, author, etc., are unlikely to be &lt;a href=&quot;https://en.wikipedia.org/wiki/Independence_(probability_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;independent&lt;/a&gt;, that is, the occurrence of a sentence &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; in a certain document &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is likely correlated with the occurrence of another sentence &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt;. If they are not independent, they can also not be independent and identically distributed (which is a stronger condition). The same can be said for words in the same sentence. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-06T21:10:45.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18475" PostTypeId="2" ParentId="18423" CreationDate="2020-03-07T08:45:44.723" Score="1" Body="&lt;p&gt;To offer a bit of theory, CNNs work well for many image tasks because they process spacially local information, without much care for absolute position. Essentially, every layer chops every image up into tiny crop images, and do an analysis step on the crops. The simple questions of &quot;is this a line... corner... eye... face?&quot; can be asked equally of every crop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the network only needs to learn once to detect a feature, rather than separately learn to detect that feature in each possible location it might appear. Therefore we can use smaller networks that train faster and need less data than if we had a fully connected architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To return to the question, you could expect a CNN to work if your data is similarly spacially correlated. Put another way, if finding a pattern around cell x, y means the same sort of thing as the same pattern in cell a, b, then you are probably in luck. On the other hand if each column represents a meaningfully different concept, then a CNN will be a poor choice of architecture. &lt;/p&gt;&#xA;" OwnerUserId="23413" LastEditorUserId="23413" LastEditDate="2020-03-11T10:33:02.363" LastActivityDate="2020-03-11T10:33:02.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18476" PostTypeId="2" ParentId="18269" CreationDate="2020-03-07T09:24:46.340" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Naturally, one might let the MDP run for 1000 periods, and then terminate as an approximation. If we feed these trajectories into a monte carlo update, I imagine that samples for time period t=1,2,...,100 would give very good estimates for the value function due to the discount factor. However, the time periods 997, 998, 999, 1000, we'd have an expected value for those trajectories far different than V(s) due to their proximity to the cutoff of 1000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think your intuition is... partially right here, but not entirely precise. Recall that a value function &lt;span class=&quot;math-container&quot;&gt;$V(S)$&lt;/span&gt; is generally defined as something like (omitting some unimportant details like specifying the policy):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(S_i) = \mathbb{E} \left[ \sum_{t=0}^{\infty} \gamma^t R_{i+t} \right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;samples for time period &lt;span class=&quot;math-container&quot;&gt;$t = 1, 2, \dots, 100$&lt;/span&gt; that you mention are &lt;strong&gt;not&lt;/strong&gt; estimates for this full value function. They're estimates for the corresponding individual terms &lt;span class=&quot;math-container&quot;&gt;$R_{i+t}$&lt;/span&gt;. Indeed, in general, your intuition is right that the closer they are to the &quot;starting point&quot; &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, the more likely they'll be to be accurate estimators. This is because larger &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; are typically associated with larger numbers of stochastic state-transitions and stochastic decision-making, and therefore often exhibit higher variance.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Should we even include these later-occurring data points when we update our function approximation? &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Theoretically, you absolutely should. Suppose you have an environment where almost every reward is equal to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and only after 1000 steps do you actually observe a non-zero reward. If you don't include this, you'll learn nothing! In practice, it can often be a good idea to give them less importance though. This already happens automatically by picking a discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Is it usually implied that the final data reward in the trajectory is bootstrapped in these cases (i.e., we have some TD(0)-like behavior in this case)?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;  &#xA;  &lt;p&gt;OR&lt;/p&gt;&#xA;  &#xA;  &lt;ol start=&quot;4&quot;&gt;&#xA;  &lt;li&gt;Are monte carlo updates for policy gradient algorithms even appropriate for non-terminating MDPs due to this issue? &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It would be possible to do some form of bootstrapping at the end yeah, cut off and then have a trained value function predicting what the remainder of the rewards would be. TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) with &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; close to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; would be much closer in behaviour to true MC updates than TD(&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) though. Either way, it would be technically incorrect to still call it Monte-Carlo then, it would no longer be pure Monte-Carlo. So yes, strict Monte-Carlo updates in the purest sense of the term are not really applicable to infinite episodes.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-03-07T09:24:46.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18477" PostTypeId="1" AcceptedAnswerId="18479" CreationDate="2020-03-07T10:20:18.750" Score="2" ViewCount="48" Body="&lt;p&gt;I am experimenting with a ConvNet to categorize images taken with a depth camera. So far I have 4 sets of 15 images each. So 4 labels. The original images are 680x880 16-bit grayscale. They are scaled down before feeding it to the ImageDataGenerator to 68x88 RGB (each color channel with equal value). I am using the ImageDataGenerator (IDG) to create more variance on the sets. (The IDG does not seem to be able to handle 16-bit grayscale images, nor 8-bit images well, so hence I converted them to RGB).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I estimate the images to be low on features, compared to regular RGB images, because it represents depth. To get a feel for the images, here are a few down scaled examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WWmMf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WWmMf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/en2oi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/en2oi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Z08jh.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z08jh.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I let it train 4.096 epochs, to see how that would go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the result of the model and validation loss.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/11LZi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/11LZi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that in the early epochs the validation (test / orange line) loss dips, and then goes up and starts to show big swings. Is this a sign of overfitting?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a zoomed in image of the early epochs.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/z1t7P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z1t7P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model loss (train / blue line) reached relatively low values with an accuracy of 1.000. Training again shows repeatedly the same kind of graphs.&#xA;Here are the last epochs.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 4087/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.1137 - accuracy: 0.9286 - val_loss: 216.2349 - val_accuracy: 0.7812&#xA;Epoch 4088/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0364 - accuracy: 0.9643 - val_loss: 234.9622 - val_accuracy: 0.7812&#xA;Epoch 4089/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0041 - accuracy: 1.0000 - val_loss: 232.9797 - val_accuracy: 0.7812&#xA;Epoch 4090/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0091 - accuracy: 1.0000 - val_loss: 238.7082 - val_accuracy: 0.7812&#xA;Epoch 4091/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0248 - accuracy: 1.0000 - val_loss: 232.4937 - val_accuracy: 0.7812&#xA;Epoch 4092/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0335 - accuracy: 0.9643 - val_loss: 273.6542 - val_accuracy: 0.7812&#xA;Epoch 4093/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0196 - accuracy: 1.0000 - val_loss: 258.2848 - val_accuracy: 0.7812&#xA;Epoch 4094/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0382 - accuracy: 0.9643 - val_loss: 226.6226 - val_accuracy: 0.7812&#xA;Epoch 4095/4096&#xA;7/7 [==============================] - 0s 10ms/step - loss: 0.0018 - accuracy: 1.0000 - val_loss: 226.2943 - val_accuracy: 0.7812&#xA;Epoch 4096/4096&#xA;7/7 [==============================] - 0s 11ms/step - loss: 0.0201 - accuracy: 1.0000 - val_loss: 207.3653 - val_accuracy: 0.7812&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not sure if it is required to know the architecture of the neural network to judge whether this is overfitting on this data set. Anyway, here is the setup.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kernelSize = 3&#xA;kernel = (kernelSize, kernelSize)&#xA;&#xA;model = Sequential()&#xA;model.add(Conv2D(16, kernel_size=kernel, padding='same', input_shape=inputShape, activation='relu'))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;model.add(Conv2D(32, kernel_size=kernel, padding='same', activation='relu'))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;model.add(Conv2D(64, kernel_size=kernel, padding='same', activation='relu'))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;&#xA;model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors&#xA;model.add(Dense(32, activation='relu'))&#xA;model.add(Dropout(0.2))&#xA;model.add(Dense(nr_of_classes, activation='softmax'))&#xA;&#xA;sgd = tf.keras.optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.4, nesterov=True)&#xA;model.compile(loss='categorical_crossentropy', &#xA;            optimizer=sgd,&#xA;            metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32968" LastActivityDate="2020-03-07T12:10:28.600" Title="Is it a sign of overfitting when validation_loss dips and then goes up with increasingly bigger swings?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;overfitting&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18478" PostTypeId="1" CreationDate="2020-03-07T11:13:27.953" Score="1" ViewCount="99" Body="&lt;p&gt;I'm working on an object detection problem using Faster R-CNN. I need to identify two object classes, and they are very similar to one another. Furthermore they are similar to a third type of object which should be considered as background. Also, all three of these objects have a lot of variation within them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my particular example the two objects of interest are 1) a statue of a particular named person who appears in many statues, 2) a statue of anyone else&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/i4aEN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i4aEN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I want to treat living flesh people, or non-humanoid statues as background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now here are some interesting results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7LJwg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7LJwg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The RPB losses follow the expected trajectory for such a problem, but on the other hand, I really had to have faith and hang in there for the detector losses. They take a while to start decreasing, and I presume it's because there is a relatively sharp trough leading to the the minimum of the loss function with respect to the weights (because the labelled classes are so similar to one another). Miraculously (at least I think so), it does start to kind of work, but not as well as I'd like it to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is as in the title but here are some of my thought processes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the class similarity spoil the bounding box regression? And then does that spoil the class inference in turn?&lt;/li&gt;&#xA;&lt;li&gt;Would it be better to just detect humanoid statues in general, then train a classifier from scratch on the output of that? (I don't know about this one. Maybe the relevant info is already encoded in the detector of the Faster R-CNN and the added bonus is that the Faster R-CNN gets context from outside of the bounding box... unless the bounding box regression is being spoiled by the class ambiguity)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-03-07T11:13:27.953" Title="Dealing with very similar object classes in object detection" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18479" PostTypeId="2" ParentId="18477" CreationDate="2020-03-07T12:10:28.600" Score="2" Body="&lt;p&gt;Yes this looks a lot like overfitting. The clue is in the low and slowly decreasing training loss compared to the large increases in validation loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One simple fix would be to stop training around epoch 50, taking the best cross validation result to select the most general network at that point. However, anything that works to improve stable generalisation could help here - more training data, more regularisation, simpler model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tricky part is finding the best combination for generalisation. Typically if you overuse regularisation to make the NN completely stable, it will lose some accuracy, so you need to run multiple experiments and measure things carefully. As you have a small dataset here and seem capable of running 1000's of epochs, I would suggest k-fold cross validation for improved measurement of cross validation loss.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-03-07T12:10:28.600" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18480" PostTypeId="1" CreationDate="2020-03-07T13:07:58.473" Score="2" ViewCount="229" Body="&lt;p&gt;Recently, I have seen a surge of papers w.r.t contrastive learning (a subset of semi-supervised learning). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give a detailed explanation of this approach with its advantages/disadvantages and what are the cases in which it gives better results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, why it's gaining traction amongst the ML research community?&lt;/p&gt;&#xA;" OwnerUserId="32249" LastEditorUserId="32249" LastEditDate="2020-03-07T16:01:13.560" LastActivityDate="2021-03-16T22:05:25.657" Title="What's the intuition behind contrastive learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;semi-supervised-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18481" PostTypeId="1" CreationDate="2020-03-07T14:00:14.967" Score="5" ViewCount="116" Body="&lt;p&gt;I am trying to predict Forex time series. The nature of the market is that 80% of the time the price can not be predicted, but in 20% of the time it can be. For example, if the price drops down very deep, there is 99% probability that there will be a recovery process, and this is what I want to predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So , how do I train a feed-forward network the way it would only predict those cases that have 99% of certainty to take place and, for the rest of the cases it would output &quot;unpredictable&quot; status?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine that my data set has 24 hours of continuous price data as input (as 1 minute samples), and then as output I want the network to predict 1 hour of future price data. The only restriction I need to implement is that if the network is not &quot;sure&quot; that the price is predictable, it would outupt 0s. So, how do I implement safety in predictions the network is outputting?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that my problem is similar to Google Compose, where it predicts the next word as you are typing , for example, if you type &quot;thank you&quot;, it would add &quot; very much&quot; and this would be like 95% correct. I want the same, but it is just that my problem has too much complexity. Google uses RNNs, so maybe I should try a deep network of many layers of RNNs?&lt;/p&gt;&#xA;" OwnerUserId="34084" LastEditorUserId="34084" LastEditDate="2020-03-10T16:30:33.137" LastActivityDate="2020-03-10T16:30:33.137" Title="How to predict time series with accuracy?" Tags="&lt;machine-learning&gt;&lt;time-series&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18482" PostTypeId="1" CreationDate="2020-03-07T15:33:21.100" Score="1" ViewCount="22" Body="&lt;p&gt;Given a directed, edge attributed graph G, where the edge attribute is a probability value, and a particular node N (with binary features f1 and f2) in G, the algorithm that I want to implement is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;List all the outgoing edges from N, let this list be called edgelist_N.&lt;/li&gt;&#xA;&lt;li&gt;For all the edges in edgelist_N, randomly assign to the edge attribute a probability value such that the sum of all the probabilities assigned to the edges in the edgelist_N equals to 1.&lt;/li&gt;&#xA;&lt;li&gt;Take the top x edges (x can be a hyperparameter).&lt;/li&gt;&#xA;&lt;li&gt;List the nodes in which the edges from step 3 are incoming.&lt;/li&gt;&#xA;&lt;li&gt;Construct a subgraph with node N, the nodes from step 4 and the edges from step 3. &lt;/li&gt;&#xA;&lt;li&gt;Embed the subgraph (preferably using a GNN) and obtain it's embedding and use it with a classifier to predict say f1/f2. &lt;/li&gt;&#xA;&lt;li&gt;Propagate the loss so as to update the edge probabilities, that was assigned randomly in step 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I do not understand how to do step 7, i.e. update the edge attribute with the loss, so that edges which are more relevant in constructing the subgraph can be assigned a higher probability value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be highly appreciated.&#xA;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="34086" LastActivityDate="2020-03-07T15:33:21.100" Title="How to update edge features in a graph using a loss function?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;loss-functions&gt;&lt;graphs&gt;&lt;graph-theory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18483" PostTypeId="1" AcceptedAnswerId="18486" CreationDate="2020-03-07T15:55:39.933" Score="1" ViewCount="237" Body="&lt;p&gt;I want to do some kind of complicated deep learning (specifically, &lt;a href=&quot;https://github.com/Sentdex/pygta5/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Sentdex/pygta5/&lt;/a&gt;), but I don't have a decent graphics card. I am willing to get 2 gt 710s. Are they enough for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="34088" LastEditorUserId="2444" LastEditDate="2020-06-01T14:22:00.037" LastActivityDate="2020-06-01T14:22:00.037" Title="Are 2 GT 710 graphics cards enough for deep learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-06-01T14:20:39.360" ContentLicense="CC BY-SA 4.0" />
  <row Id="18485" PostTypeId="1" CreationDate="2020-03-07T17:22:32.090" Score="1" ViewCount="27" Body="&lt;p&gt;I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://www.upsieutoc.com/images/2020/03/08/Clipboarder.2020.03.08.png&quot; alt=&quot;Image&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model has 2 hidden tanh x 16 units running with ml5.js. I choose tanh because it seems to be smoother than ReLU. Apart from that they're the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this thing caused by the model or by ml5 itself?&lt;/p&gt;&#xA;" OwnerUserId="34091" LastActivityDate="2020-03-09T11:57:06.490" Title="Model unfit for some part of spiral data despite low error" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18486" PostTypeId="2" ParentId="18483" CreationDate="2020-03-07T18:48:35.270" Score="1" Body="&lt;p&gt;No, it appears to take large amounts of screenshots to generate the images to be used for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't waste your money. I would say you could try training on &lt;a href=&quot;https://colab.research.google.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's colab&lt;/a&gt; if you can upload the training data.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-06-01T14:21:21.643" LastActivityDate="2020-06-01T14:21:21.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18487" PostTypeId="2" ParentId="18481" CreationDate="2020-03-07T19:13:14.080" Score="0" Body="&lt;p&gt;Tbh I think stock prices are essentially impossible to predict as you're not taking into account the data from outside the stock market.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd argue any successful model would need to be trained on news, consumer sentiment, etc etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only ones which maybe work are HFT. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-03-07T19:13:14.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18489" PostTypeId="1" AcceptedAnswerId="19917" CreationDate="2020-03-07T21:32:06.587" Score="2" ViewCount="145" Body="&lt;p&gt;I am working through the textbook &amp;quot;Graph-Based Natural Language Processing and Information Retrieval&amp;quot;, where I've got a question on implementation of this first Latex looking formula/algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Can you help me turn the formula under 1.2 Graph Properties into python code? Yes, I know there are many other languages, but python is more user-friendly so I'm starting there, and will eventually rewrite it into C.&lt;/p&gt;&#xA;&lt;p&gt;As I read the above node example, sorry the D and E nodes were cut off.&#xA;Node A has two outflowing arrows notating it as their head node, and it is the one tail node.&lt;/p&gt;&#xA;&lt;p&gt;This first sentence references the Graphs:&#xA;To traverse from A to B, If A to B value is sufficient (above Nx), go to B. If A to B value is below Nx, go A to C to D to A to B total cost is 5.2+7+1+8 = 21.20 traverse cost, this makes sense.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/F8TeG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F8TeG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This sentence refers to the Latex formula in the book.&#xA;Then to start the formula calculation, the average degree of a graph &amp;quot;a&amp;quot; is equal to the sum of, one over N, times the sum of the in-degree of vertices? Asserting that the sum is a non zero integer between 1 and N?&lt;/p&gt;&#xA;&lt;p&gt;Ok, I only loaded one page and hope that's not a TOS violation or causes issue, it's a challenge to find people who understand graph theory.&lt;/p&gt;&#xA;&lt;p&gt;Let me know what questions you have, but I'm just wanting to get clarification if my understanding is what this page is saying.&lt;/p&gt;&#xA;" OwnerUserId="34095" LastEditorUserId="2444" LastEditDate="2020-09-10T19:52:55.080" LastActivityDate="2020-09-10T19:52:55.080" Title="How do I turn this formula of the average degree of a graph into Python code?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;graph-theory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18491" PostTypeId="1" CreationDate="2020-03-08T01:21:30.190" Score="2" ViewCount="52" Body="&lt;p&gt;I am trying to build an LSTM model to generate Shakspeare-like poems. I have training set &lt;span class=&quot;math-container&quot;&gt;$\{s_1,s_2, \dots,s_m\}$&lt;/span&gt;, which are sentences of Shakespeare poems, and each sentence contains words &lt;span class=&quot;math-container&quot;&gt;$\{w_1,w_2, \dots,w_n\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my understanding, each sentence &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=1, \dots,m$&lt;/span&gt; is a random sequence containing the words &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$j=1, \dots,n$&lt;/span&gt;. The LSTM model is estimated by applying the maximum likelihood (MLE) method, which will use cross-entropy loss for optimization. The use of MLE requires that the samples in the random sequence be independent and identically distributed (i.i.d), however, the word sequence &lt;span class=&quot;math-container&quot;&gt;$w_j$&lt;/span&gt; is not i.i.d (since it is non-Markov). Therefore, I am suspicious about using cross-entropy loss for training an LSTM for the NLP task (which seems to be the common practice). &lt;/p&gt;&#xA;" OwnerUserId="33266" LastEditorUserId="2444" LastEditDate="2020-03-08T03:27:12.973" LastActivityDate="2020-03-08T03:27:12.973" Title="Can the cross-entropy loss be used for a NLP task with LSTM?" Tags="&lt;long-short-term-memory&gt;&lt;cross-entropy&gt;&lt;maximum-likelihood&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18492" PostTypeId="1" CreationDate="2020-03-08T03:48:40.733" Score="1" ViewCount="159" Body="&lt;p&gt;Does transformer have the potential to replace RNN end-to-end models for speech recognition for online speech recognition? This mainly depends on accuracy/latency and deploy cost, not training cost. Can transformer support low latency online use case and have comparable deploy cost and better result than RNN models?&lt;/p&gt;&#xA;" OwnerUserId="25322" LastEditorUserId="-1" LastEditDate="2021-01-19T02:02:43.843" LastActivityDate="2021-01-19T02:03:29.457" Title="Can transformer be better than RNN for online speech recognition?" Tags="&lt;recurrent-neural-networks&gt;&lt;transformer&gt;&lt;speech-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18493" PostTypeId="5" CreationDate="2020-03-08T04:08:49.463" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-08T04:08:49.463" LastActivityDate="2020-03-08T04:08:49.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18494" PostTypeId="4" CreationDate="2020-03-08T04:08:49.463" Score="0" Body="For questions related to affective computing, which is the study and development of systems and devices that can recognize, interpret, process, and simulate human emotions (or affects). AC is an interdisciplinary field spanning computer science, psychology, cognitive science or artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-08T04:08:49.463" LastActivityDate="2020-03-08T04:08:49.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18496" PostTypeId="2" ParentId="18492" CreationDate="2020-03-08T15:10:02.147" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there examples that transformer have better accuracy than RNN end-to-end model like RNN-transducer for speech recognition?&#xA;  Can transformer be used for online speech recognition which require low speech-end-to-result latency?&#xA;  Does transformer have the potential to replace RNN end-to-end models for speech recognition in most cases in the future? This may mainly depends on accuracy and deploy cost, not training cost.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can check facebook results on wav2letter on all this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.facebook.com/blog/online-speech-recognition-with-wav2letteranywhere/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.facebook.com/blog/online-speech-recognition-with-wav2letteranywhere/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://research.fb.com/publications/scaling-up-online-speech-recognition-using-convnets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://research.fb.com/publications/scaling-up-online-speech-recognition-using-convnets/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transformers definitely have a potential in speech especially when combined with faster computatoin methods (hashing) just like in NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with transformers is that you need a lot of GPUs to train them.&lt;/p&gt;&#xA;" OwnerUserId="3459" LastActivityDate="2020-03-08T15:10:02.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18497" PostTypeId="1" CreationDate="2020-03-08T16:03:56.087" Score="2" ViewCount="136" Body="&lt;p&gt;&lt;a href=&quot;https://fr.wikipedia.org/wiki/Jacques_Pitrat&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. Pitrat&lt;/a&gt; (born in 1934) was a French leading artificial intelligence scientist (the first to get a Ph.D. in France mentioning &amp;quot;artificial intelligence&amp;quot;). His &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; is still online and of course refer to most of his papers (e.g. &lt;a href=&quot;https://pdfs.semanticscholar.org/2117/9600b3f05c0af399f9acbfc6e7b6d24daf03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Step toward an Artificial Artificial Intelligence Scientist&lt;/em&gt;&lt;/a&gt;, etc.) and books, notably &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470611791&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings: the conscience of a conscious machine&lt;/em&gt;&lt;/a&gt; (his last book). He passed away in October 2019. I attended (and presented a &lt;a href=&quot;http://refpersys.org/Starynkevitch-CAIA-RefPerSys-2020mar06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;talk&lt;/a&gt;) at a &lt;a href=&quot;https://afia.asso.fr/journee-hommage-j-pitrat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seminar&lt;/a&gt; in his memory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What are recent AI systems or research papers related to the idea of symbolic AI, introspection, declarative metaknowledge, meta-learning, meta-rules, etc.?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Most of those I know are more than 20 years old (e.g. Lenat &lt;a href=&quot;https://en.wikipedia.org/wiki/Eurisko&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eurisko&lt;/a&gt;; I am aware of &lt;a href=&quot;https://en.wikipedia.org/wiki/Cyc#OpenCyc&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCyC&lt;/a&gt;). I am interested in papers or systems published after 2010 (perhaps &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI&lt;/a&gt; papers with &lt;em&gt;actual&lt;/em&gt; complex open source software prototypes).&lt;/p&gt;&#xA;&lt;p&gt;-see also the &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; system-&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-12-21T20:47:23.827" LastActivityDate="2020-12-26T05:57:31.500" Title="What are recent AI software systems and research papers close to J. Pitrat's ideas?" Tags="&lt;reference-request&gt;&lt;symbolic-ai&gt;&lt;meta-learning&gt;&lt;meta-rules&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18498" PostTypeId="2" ParentId="15459" CreationDate="2020-03-08T16:38:17.990" Score="0" Body="&lt;p&gt;The book has actually proven the theorem rigorously in Chapter 2. I don't want to prove it here, but you can look it up. I will try to explain parts which are non obvious (and somewhat confusing according to the book's literature). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for PAC learning (with or without the realizability assumption) the theory is that given a data-set of size: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$m \geq [\frac{log(|H|/\delta)}{\epsilon}]$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$|H|$&lt;/span&gt; is the size of the finite hypothesis class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which when simplified is nothing but:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$|H|e^{-\epsilon m} \leq \delta$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is the probability that your sample is not representative of the underlying distribution (according to the book, hence the term Probably in PAC Learning) and &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is the maximum probability that your learned hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; predicts new unseen samples wrong (basically accuracy of your hypothesis and hence the term Approximately Correct in PAC Learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation/bound comes from the last step of the proof which states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$D^m  [ {S|_x : L_{(D,f)}(h_S)\gt \epsilon}] \leq |H_B|e^{-\epsilon m} \leq  |H|e^{-\epsilon m}$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$H_B$&lt;/span&gt; are all the bad hypothesis (over-fitting hypothesis)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is your answer to the &lt;strong&gt;question&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Estimation error increase with linearly &lt;span class=&quot;math-container&quot;&gt;$|H|$&lt;/span&gt; and decrease exponentially with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; in PAC learning&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now here comes the tricky part, following this equation the proof directly jumps to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$|H|e^{-\epsilon m} \leq \delta$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The justification for this is given in previous part of the proof (I am not entirely sure if they meant this justification, but it seems the only one):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since&#xA;  the realizability assumption implies that &lt;span class=&quot;math-container&quot;&gt;$L_S (h_S ) = 0$&lt;/span&gt;, it follows that the event&#xA;  &lt;span class=&quot;math-container&quot;&gt;$L_{(D,f )} (h_S ) &amp;gt; \epsilon$&lt;/span&gt; can only happen if for some &lt;span class=&quot;math-container&quot;&gt;$h ∈ H_B$&lt;/span&gt; we have &lt;span class=&quot;math-container&quot;&gt;$L_S (h) = 0$&lt;/span&gt;. In other words, this event will only happen if our sample is in the set of &lt;strong&gt;misleading&lt;/strong&gt; samples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Do not mistakenly we confuse &lt;strong&gt;misleading&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; &lt;strong&gt;non-representative&lt;/strong&gt; otherwise we will not able to justify the aforementioned jump (&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; becomes dependent on each other)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual interpretation of &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is that it is our confidence parameter i.e we want to ensure:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$D^m  [ {S|_x : L_{(D,f)}(h_S)\gt \epsilon}] \leq \delta$$&lt;/span&gt;&#xA;which means we are &lt;span class=&quot;math-container&quot;&gt;$1-\delta$&lt;/span&gt; confident that our learned &lt;span class=&quot;math-container&quot;&gt;$h_s$&lt;/span&gt; will have &lt;span class=&quot;math-container&quot;&gt;$L_{(D,f)}(h_s) \leq \epsilon$&lt;/span&gt; (complementary expression).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; This idea is skipped in most resources I read, I found its explanation &lt;a href=&quot;https://www.youtube.com/watch?v=qOMOYM0WCzU&amp;amp;t=924s&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, coming to the statement:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$m_H \leq [\frac{log(|H|/\delta)}{\epsilon}]$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$m_H$&lt;/span&gt; unlike &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is defined as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is PAC learnable, there are many functions &lt;span class=&quot;math-container&quot;&gt;$m_H$&lt;/span&gt; that satisfy the&#xA;  requirements given in the definition of PAC learnability. Therefore, to be precise,we will define the sample complexity of learning &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; to be the “minimal function,” in the sense that for any &lt;span class=&quot;math-container&quot;&gt;$\epsilon, \delta$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$m_H (\epsilon, \delta)$&lt;/span&gt; is the minimal integer that satisfies the requirements of PAC learning with accuracy &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; and confidence &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And hence the equality sign is reversed, since many good samples will result in good hypothesis being generated in a smaller number of samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side note: All conventions are from Understanding Machine Learning: From Theory to Algorithms.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-03-12T21:33:26.407" LastActivityDate="2020-03-12T21:33:26.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18499" PostTypeId="1" AcceptedAnswerId="18502" CreationDate="2020-03-08T21:54:54.180" Score="0" ViewCount="71" Body="&lt;p&gt;I have some familiarity with the regular Tensorflow library and have been able to create a number of working models with it. There are more than enough resources out there to get up and running and answer most questions on the standard library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I recently came across the video on some high-level capabilities of the Tensorflow Probability library, &lt;a href=&quot;https://www.youtube.com/watch?v=BrwKURU-wpk&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow Probability: Learning with confidence (TF Dev Summit '19)&lt;/a&gt;, and I would like to learn it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that there are very few resources out there on TFP and given my lack of a formal background in math/statistics, I find myself aimlessly googling to get a grasp of what's going on in the docs. I'm more than willing to invest the time needed, but I just need to know where I can start in terms of resources I can access online.  Specifically, I'm looking to get the necessary domain knowledge needed to work with the library given the lack of courses/tutorials on the library itself.&lt;/p&gt;&#xA;" OwnerUserId="30154" LastEditorUserId="2444" LastEditDate="2020-03-08T23:02:58.140" LastActivityDate="2020-03-11T04:43:54.617" Title="What are the prerequisites to start using the TensorFlow Probability library?" Tags="&lt;tensorflow&gt;&lt;probability&gt;&lt;tensorflow-probability&gt;" AnswerCount="1" CommentCount="4" ClosedDate="2020-03-11T16:05:53.843" ContentLicense="CC BY-SA 4.0" />
  <row Id="18501" PostTypeId="1" CreationDate="2020-03-08T22:28:03.110" Score="3" ViewCount="52" Body="&lt;p&gt;If one is interested in implementing a path planning algorithm that is grid-based, one needs to consider the fact that your grid points will never represent the true state of the robot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this dealt with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we're doing path planning using a grid-based search on the side of the control for a desired grid position as an output state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you handle the discrepancy between your actual starting position and your discretized starting position? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that normally you may use an MPC instead, which continually recalculates an optimal path using some type of nonlinear solver, but suppose we don't do this - suppose we restrict ourselves to only a grid search and suppose at after every action the state of the robot has to be considered as living in a particular grid point.  &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-03-11T02:57:06.810" LastActivityDate="2020-03-11T02:57:06.810" Title="How to deal with approximate states when doing path planning?" Tags="&lt;reinforcement-learning&gt;&lt;robotics&gt;&lt;path-planning&gt;&lt;control-theory&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18502" PostTypeId="2" ParentId="18499" CreationDate="2020-03-08T22:43:05.423" Score="2" Body="&lt;p&gt;Although this question is slightly primarily opinion-based and too broad (and I will probably close it as such) and a good answer will necessarily depend on your background, I will list some of the main theoretical prerequisites that everyone should ideally be familiar with before diving into TensorFlow Probability (TFP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with TFP, given that I've been using it for a project, but I've not used all of its functionalities, such as the bijectors. I've only used the Bayesian layers, distributions, etc., so I will try to give an answer based on my experience. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You definitely need to be familiar with the basic concepts of probability theory, such as distributions, random variables, expectations, etc. A full university-level course in probability and statistics would definitely be helpful!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use Bayesian layers, you also need to be familiar with Bayesian neural networks (BNNs). To understand BNNs, you need to understand the basic concepts of Bayesian inference (BI), given that most BNNs are based on BI. If you are already familiar with variational auto-encoders (VAEs), then it will be easier to understand BNNs. To understand BNNs, you should start reading the paper &lt;a href=&quot;https://arxiv.org/pdf/1505.05424.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Uncertainty in Neural Networks&lt;/a&gt;. If this paper is difficult to follow, you should first read the paper &lt;a href=&quot;https://arxiv.org/pdf/1601.00670.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational Inference: A Review for Statisticians&lt;/a&gt; and then it may also be useful to read and be familiar with &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the VAE paper&lt;/a&gt;. Therefore, you will need to be familiar with concepts such as Kullback–Leibler divergence and &lt;a href=&quot;https://arxiv.org/pdf/1906.10652.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo sampling&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The blog post &lt;a href=&quot;https://blog.tensorflow.org/2019/03/regression-with-probabilistic-layers-in.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Regression with Probabilistic Layers in TensorFlow Probability&lt;/a&gt; is the first blog post you should read. If you don't understand it, then it probably means you need to learn its prerequisites (i.e. you need to be familiar with at least linear regression).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TFP also provides several &lt;a href=&quot;https://github.com/tensorflow/probability/tree/master/tensorflow_probability/examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementations&lt;/a&gt; of other more advanced concepts such as BNNs. The easiest example to follow is probably &lt;a href=&quot;https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/logistic_regression.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;the logistic regression example&lt;/a&gt;. LR is relatively easy to follow compared to the topics I mentioned above, but, of course, you need to be familiar with logistic regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, I cannot list all the theoretical prerequisites (and that's why your post is too broad!) and there are definitely others, but this is a start. Bear in mind that these are not trivial topics, so it is normal to get stuck.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-08T23:06:49.803" LastActivityDate="2020-03-08T23:06:49.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18503" PostTypeId="5" CreationDate="2020-03-08T23:06:16.523" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-08T23:06:16.523" LastActivityDate="2020-03-08T23:06:16.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18504" PostTypeId="4" CreationDate="2020-03-08T23:06:16.523" Score="0" Body="For questions related to the TensorFlow Probability (TFP) library." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-08T23:06:16.523" LastActivityDate="2020-03-08T23:06:16.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18505" PostTypeId="1" AcceptedAnswerId="18511" CreationDate="2020-03-08T23:17:12.553" Score="1" ViewCount="72" Body="&lt;p&gt;I have used Beta function to estimate the performance of the agent. I have failure and success data of the task that runs on the agent. &#xA;The parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a number of successful tasks, while &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; is the number of failures. Thus, I can estimate the performance by exploiting the expected value of Beta, as &lt;span class=&quot;math-container&quot;&gt;$$\mu = \frac{\alpha} {(\alpha+\beta)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I am looking for a similar model, such that its parameter can be estimated from the success and failure data. So far I found Dirichlet distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the expected value of Dirichlet distribution? How I can use the success and failure data to estimate parameters of this distribution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's check the following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that we use a Dirichlet prior represented by &lt;span class=&quot;math-container&quot;&gt;$Dirichlet(1, 1, 1)$&lt;/span&gt; and observe &lt;span class=&quot;math-container&quot;&gt;$13$&lt;/span&gt; results with &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; Successful, &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; Missing, and &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; Failures. Then we get the posterior to be &lt;span class=&quot;math-container&quot;&gt;$Dirichlet(1+8, 1+2, 1+3)$&lt;/span&gt;. Then if you define the performance value &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; to be the expectation of &lt;span class=&quot;math-container&quot;&gt;$P(x=Successful)$&lt;/span&gt;, &#xA;then &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; will be &lt;span class=&quot;math-container&quot;&gt;$(1+8)/[(1+8)+(1+2)+(1+3)] = 0.56$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now &#xA;Suppose that we use a Beta prior represented by &lt;span class=&quot;math-container&quot;&gt;$Beta(1,1)$&lt;/span&gt; and observe &lt;span class=&quot;math-container&quot;&gt;$13$&lt;/span&gt; results with &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; Successful, and &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; Failures. Then we get the posterior to be &lt;span class=&quot;math-container&quot;&gt;$Beta(1+8, 1+3)$&lt;/span&gt;. Then if you define the performance value Pr to be the expectation of &lt;span class=&quot;math-container&quot;&gt;$P(x=Successful)$&lt;/span&gt;, &#xA;then &lt;span class=&quot;math-container&quot;&gt;$\alpha = (1+8)/[(1+8)+(1+3)] = 0.69$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my calculations and concept right?&lt;/p&gt;&#xA;" OwnerUserId="30551" LastEditorUserId="9947" LastEditDate="2020-03-11T18:03:52.580" LastActivityDate="2020-03-11T18:03:52.580" Title="How can I use the success and failure data to estimate parameters of a Dirichlet distribution?" Tags="&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18507" PostTypeId="1" CreationDate="2020-03-08T23:56:28.063" Score="0" ViewCount="35" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Wybwm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wybwm.jpg&quot; alt=&quot;error ensemble&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What is &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;? Why is &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; the ceil of &lt;span class=&quot;math-container&quot;&gt;$n/2$&lt;/span&gt;? What is &lt;span class=&quot;math-container&quot;&gt;$y \geq k$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="33955" LastEditorUserId="2444" LastEditDate="2020-12-04T11:19:29.907" LastActivityDate="2021-05-03T13:15:16.367" Title="Why does the error ensemble use the ceiling of the number of classifiers?" Tags="&lt;probability&gt;&lt;ensemble-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18508" PostTypeId="2" ParentId="18507" CreationDate="2020-03-09T01:09:19.603" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Y ensemble size voting wrong&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;k = 50% or majority threshold&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have 11 models. Then the majority of models is anything bigger than 50% of the number of ensemble models. In the example where you have 11 base models. The majority would be anything bigger than N/2 or 11/2. But since 11 is an odd number and cannot be divided by 2. We have to use the python ceiling function to round 5.5 to 6. The in other words. For your ensemble to be wrong. We must look for the probability that Y of them are wrong. Y is ≥ than K and K is ceil(N/2). Thus we must calculate this probability(6 wrong) like shown above summating the possibility of each of these discrete combinations into one&lt;/p&gt;&#xA;" OwnerUserId="33955" LastActivityDate="2020-03-09T01:09:19.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18509" PostTypeId="5" CreationDate="2020-03-09T02:21:09.077" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-09T02:21:09.077" LastActivityDate="2020-03-09T02:21:09.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18510" PostTypeId="4" CreationDate="2020-03-09T02:21:09.077" Score="0" Body="For questions related to neuroscience in the context of artificial intelligence. For example, if you have a question about an AI paper that claims that the proposed method is inspired by certain neuroscience findings, you probably should use this tag." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-20T13:40:31.547" LastActivityDate="2020-04-20T13:40:31.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18511" PostTypeId="2" ParentId="18505" CreationDate="2020-03-09T02:35:08.890" Score="1" Body="&lt;p&gt;Dirichlet is the Multi Variate version of the Beta distribution. In general, these distributions can be thought to model the probability of modelling a probability distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.statlect.com/glossary/support-of-a-random-variable&quot; rel=&quot;nofollow noreferrer&quot;&gt;support&lt;/a&gt; Dirichlet distribution is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ S_K = \{ x:0 \leq x_k \leq 1, \sum_{k=1}^K x_k=1 \} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the PDF is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Dir(x|\alpha) = \frac{1}{B(\alpha)} \prod_{k=1}^Kx_k^{\alpha_k-1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$B(\alpha)$&lt;/span&gt; is the beta function of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B(\alpha) = \frac{\prod_{k=1}^K \tau(\alpha_k)}{\tau(\sum_{k=1}^K \alpha_k)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the resultant point estimates are (Define &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=1}^K \alpha_k = \alpha_0)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mu(x_k) = \frac{\alpha_k}{\alpha_0}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma^2(x_k) = \frac{\alpha_k(\alpha_0-\alpha_k)}{\alpha_0^2(\alpha_0+1)}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$mode[x_k] = \frac{\alpha_k-1}{\alpha_0-1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beta distribution is the special case where &lt;span class=&quot;math-container&quot;&gt;$k=2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, when you run an experiment a large number of times, the success of each &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; will approach towards its expected value i.e if you define your random variables as &lt;span class=&quot;math-container&quot;&gt;$x_k = \frac{N_{k}}{N}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the total number of trials and &lt;span class=&quot;math-container&quot;&gt;$N_k$&lt;/span&gt; is the success of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; th term, it clearly satisfies the support of the Dirichlet Distribution and hence you can use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\alpha_{k}}{\alpha_0} = \frac{N_{k}}{N}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is assuming that the experiment follows Dirichlet Distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Taken in parts from &lt;a href=&quot;https://www.cs.ubc.ca/~murphyk/MLbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Probabilistic Approach to ML&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-03-09T02:35:08.890" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18512" PostTypeId="1" CreationDate="2020-03-09T05:31:15.130" Score="1" ViewCount="64" Body="&lt;p&gt;AFAIK, momentum is quite useful when training CNNs, and can speed-up the training substantially without any drop in validation accuracy.&lt;/p&gt;&#xA;&lt;p&gt;I've recently learned that it is not as helpful for RNNs, where plain SGD is preferred.&lt;/p&gt;&#xA;&lt;p&gt;For example, Deep Learning by Goodfellow et. al says (&lt;a href=&quot;https://www.deeplearningbook.org/contents/rnn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 10.11, page 401&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Both of these approaches have largely been replaced by simply using SGD (even without momentum) applied to LSTMs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The author talks about LSTMs and &amp;quot;both of these approaches&amp;quot; refer to second-order and first-order SGD methods with momentum methods, respectively, according to my understanding.&lt;/p&gt;&#xA;&lt;p&gt;What causes this discrepancy?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="2444" LastEditDate="2020-12-13T12:37:07.910" LastActivityDate="2020-12-13T12:37:07.910" Title="Why do momentum techniques not work well for RNNs?" Tags="&lt;recurrent-neural-networks&gt;&lt;gradient-descent&gt;&lt;stochastic-gradient-descent&gt;&lt;momentum&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18513" PostTypeId="1" CreationDate="2020-03-09T05:53:24.587" Score="1" ViewCount="86" Body="&lt;p&gt;I came across the concept of Bayesian Occam Razor in the book &lt;a href=&quot;https://www.cs.ubc.ca/~murphyk/MLbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning: a Probabilistic Perspective&lt;/a&gt;. According to the book:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Another way to understand&#xA;  the Bayesian Occam’s razor effect is to note that probabilities must&#xA;  sum to one. Hence &lt;span class=&quot;math-container&quot;&gt;$\sum_D'  p(D' |m) = 1$&lt;/span&gt;, where the sum is over all possible data sets. Complex&#xA;  models, which can predict many things, must spread their probability mass thinly, and hence&#xA;  will not obtain as large a probability for any given data set as simpler models. This is sometimes called the conservation of probability mass principle.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The figure below is used to explain the concept:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dNLo4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dNLo4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;&lt;strong&gt;Image Explanation&lt;/strong&gt;: On the vertical axis we plot the predictions of 3 possible models: a simple one, &lt;span class=&quot;math-container&quot;&gt;$M_1$&lt;/span&gt; ; a medium one, &lt;span class=&quot;math-container&quot;&gt;$M_2$&lt;/span&gt; ; and a complex one, &lt;span class=&quot;math-container&quot;&gt;$M_3$&lt;/span&gt; . We also indicate the actually observed&#xA;    data &lt;span class=&quot;math-container&quot;&gt;$D_0$&lt;/span&gt; by a vertical line. Model 1 is too simple and assigns low probability to &lt;span class=&quot;math-container&quot;&gt;$D_0$&lt;/span&gt; . Model 3&#xA;    also assigns &lt;span class=&quot;math-container&quot;&gt;$D_0$&lt;/span&gt; relatively low probability, because it can predict many data sets, and hence it&#xA;    spreads its probability quite widely and thinly. Model 2 is “just right”: it predicts the observed data with a reasonable degree of confidence, but does not predict too many other things. Hence model 2 is the most probable model.&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What I do not understand is when a complex model is used, it will likely overfit data and hence the plot for a complex model will look like a bell shaped with its peak at &lt;span class=&quot;math-container&quot;&gt;$D_0$&lt;/span&gt; while simpler models will more likely have a broader bell shape. But the graph here shows something else entirely. What am I missing here?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="5763" LastEditDate="2021-04-06T11:21:48.423" LastActivityDate="2021-05-06T12:03:00.420" Title="Understanding Bayesian Optimisation graph" Tags="&lt;machine-learning&gt;&lt;bayesian-optimization&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18514" PostTypeId="1" CreationDate="2020-03-09T07:37:04.473" Score="1" ViewCount="755" Body="&lt;p&gt;I am training a combined model (fine-tuned VGG16 for images and shallow FCN for numerical data) to do a binary classification. However, the overall AUC score is not what I expected it to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image-only mean AUC after 5-fold cross-validation is about &lt;strong&gt;&lt;em&gt;0.73&lt;/em&gt;&lt;/strong&gt; and numeric data only 5-fold mean AUC is &lt;strong&gt;&lt;em&gt;0.65&lt;/em&gt;&lt;/strong&gt;. I was hoping to improve the mean AUC by combining the models into one and merging output layers using &lt;code&gt;concatenate&lt;/code&gt; in Keras. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;img_output = Dense(256, activation=&quot;sigmoid&quot;)(x_1) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;numeric_output = Dense(128, activation=&quot;relu&quot;)(x_2) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;are the output layers of the two models. And, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;concat = concatenate([img_output, numeric_output])&#xA;hidden1 = Dense(64, activation=&quot;relu&quot;)(concat)&#xA;main_output = Dense(1, activation='sigmoid', name='main_output')(hidden1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is the way I concatenated them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since image-only performance was better I decided that it might be reasonable to have more dense layers for &lt;code&gt;image_output&lt;/code&gt; (&lt;code&gt;256&lt;/code&gt;) and ended up using &lt;code&gt;128&lt;/code&gt; in &lt;code&gt;numeric_output&lt;/code&gt;.I could only reach up to mean AUC of 0.67 using a combined model. I think I should rearrange the concatenation of two outputs somehow (by introducing another learnable parameter (like the formula (10) at 3.3 section of &lt;a href=&quot;https://www.hindawi.com/journals/cin/2018/2061516/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this work&lt;/a&gt;?, bias?, or something else) to get more boost on mean AUC. However, I was not able to find what options were available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope you have some ideas worth trying.&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="2444" LastEditDate="2020-04-03T12:28:18.487" LastActivityDate="2020-04-03T12:28:18.487" Title="How can I merge outputs of two separate layers so that the overall performance improves?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;performance&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18516" PostTypeId="1" CreationDate="2020-03-09T09:28:08.753" Score="1" ViewCount="277" Body="&lt;p&gt;According to the authors of &lt;a href=&quot;https://arxiv.org/pdf/1703.03400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, to improve the performance, they decided to &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;drop backward pass and using a first-order approximation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a blog&lt;/a&gt; which discussed how to derive the math but got stuck along the way (please refer to the embedded image below):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why &#xA;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cnabla_%7B%5Ctheta%7D%5Ctheta_%7B0%7D&quot; alt=&quot;&quot;&gt; disappeared in the next line.&lt;/li&gt;&#xA;&lt;li&gt;How come &lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cnabla_%7B%5Ctheta_%7Bi-1%7D%7D%5Ctheta_%7Bi-1%7D%3DI&quot; alt=&quot;&quot;&gt; (which is an Identity matrix)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7aXRX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7aXRX.png&quot; alt=&quot;FOMAML&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; I also found &lt;a href=&quot;https://www.slideshare.net/ssuser62b35f/introduction-to-maml-model-agnostic-meta-learning-with-discussions-124492943&quot; rel=&quot;nofollow noreferrer&quot;&gt;another math solution&lt;/a&gt; for this. To me it looks less intuitive but there's no confusion with the disappearance of  as in the first solution.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kEFn7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kEFn7.png&quot; alt=&quot;first order MAML&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33801" LastEditorUserId="2444" LastEditDate="2021-01-14T00:30:46.010" LastActivityDate="2021-01-14T00:30:46.010" Title="Understanding the derivation of the first-order model-agnostic meta-learning" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;gradient-descent&gt;&lt;meta-learning&gt;&lt;model-agnostic-meta-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18517" PostTypeId="1" CreationDate="2020-03-09T09:35:12.747" Score="2" ViewCount="45" Body="&lt;p&gt;I'm building a CNN/3DCNN model that classifies hand gestures. The problem is that the actual gesture occupies only like 1% of the whole image. That means that an enormous amount of convolutional operations is done on the &quot;empty&quot; parts of the image, which is useless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to solve this problem? I was thinking about a MaxPooling layer with a giant pool size, but near features that are extracted from the gesture will be probably &quot;compressed&quot; in only 1 feature.&lt;/p&gt;&#xA;" OwnerUserId="32751" LastEditorUserId="2444" LastEditDate="2020-03-10T21:10:02.013" LastActivityDate="2020-03-10T21:10:02.013" Title="Is there a way to add &quot;focus&quot; on parts of the image when using CNNs?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18518" PostTypeId="1" CreationDate="2020-03-09T10:44:02.503" Score="0" ViewCount="57" Body="&lt;p&gt;I have made a Deep Q Network for the game snake but unfortunately, the snake exhibits some unwanted behavior. It generally does quite well but sometimes it gets stuck in an infinite loop that it can't escape and at the start of the game it takes a very long route to the apple rather than taking a more direct route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discount factor per time step is 0.99. The Snake gets a reward of +9 for getting an Apple and -1 for dying. Does anybody have any recommendations on how I should tune the hyperparameters/reward function to minimize this unwanted behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking that reducing the discount factor may be a good idea?&lt;/p&gt;&#xA;" OwnerUserId="33227" LastActivityDate="2020-03-09T13:13:18.027" Title="How to incentivise snake to go straight to apple?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18520" PostTypeId="2" ParentId="18518" CreationDate="2020-03-09T13:13:18.027" Score="1" Body="&lt;p&gt;Your network finds the infinite loop and notices that this has the best reward (0). This is probably because it hasn't found a path to eating the apple (through exploration).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reducing the discount factor will only make the long term rewards less valuable. So it will learn eating the apple even slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what you are using as inputs for your network, but maybe changing your reward system could help. For example, you could give your network a reward if it advances in the direction of the apple. This way, your network will be encouraged to find the apple more than it is now.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2020-03-09T13:13:18.027" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18521" PostTypeId="2" ParentId="18516" CreationDate="2020-03-09T15:19:07.230" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_{i-1}} \theta_{i-1} = \mathbf{I}$&lt;/span&gt; in a similar way that &lt;span class=&quot;math-container&quot;&gt;$\frac{d f}{dx} = 1$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$f(x) = x$&lt;/span&gt;. Strictly speaking, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; should be a vector of &lt;span class=&quot;math-container&quot;&gt;$1s$&lt;/span&gt; with the same dimensionality as &lt;span class=&quot;math-container&quot;&gt;$\theta_{i-1}$&lt;/span&gt;, but they are probably abusing notation here and putting such a vector as the diagonal elements of a matrix. Alternatively (actually, the most likely reason!), they are computing the partial derivative of &lt;span class=&quot;math-container&quot;&gt;$\theta_{i-1}^j$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta_{i-1}^k$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, which will make up an identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your first question, &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta} \theta_{0}$&lt;/span&gt; probably becomes 1, but I am not familiar enough with the math of this paper to tell you why. Maybe it's because &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta} \theta_{0}$&lt;/span&gt; actually means &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_0} \theta_{0}$&lt;/span&gt;. I would need to dive into it.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-09T15:28:17.760" LastActivityDate="2020-03-09T15:28:17.760" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18522" PostTypeId="2" ParentId="18081" CreationDate="2020-03-09T16:01:34.377" Score="0" Body="&lt;p&gt;I think, one could generate a data set, with a random variable in each component of the data vector, add this data to the training data set, and then shuffle the combined data set.&lt;/p&gt;&#xA;" OwnerUserId="27480" LastActivityDate="2020-03-09T16:01:34.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18523" PostTypeId="2" ParentId="2681" CreationDate="2020-03-09T16:53:01.080" Score="2" Body="&lt;p&gt;Have a look at the paper &lt;a href=&quot;https://www.aclweb.org/anthology/D19-1636/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Modular Architecture for Unsupervised Sarcasm Generation&lt;/a&gt; (2019) by &lt;a href=&quot;https://abhijitmishra.github.io/#about&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mishra&lt;/a&gt; et al. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the abstract, the authors write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper, we propose a novel framework for sarcasm generation; the system takes a literal negative opinion as input and translates it into a sarcastic version. Our framework does not require any paired data for training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/TarunTater/sarcasm_generation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;'s the reference implementation.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-09T16:53:01.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18524" PostTypeId="1" CreationDate="2020-03-09T21:39:28.623" Score="1" ViewCount="42" Body="&lt;p&gt;I'm new to the Data Science field and last week I started to learn about Neural Networks and Deep Learning. To practice, I decided to do a small project: design a Neural Network &lt;strong&gt;to predict the winner of an NBA game given the two teams playing&lt;/strong&gt;. Also, for each match I have 2 stats (let's say number of points and number of free throws) for each of the teams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, the dataset looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|  ID |  Home |  Away | H_Pts | H_Fts | A_Pts | A_Fts | H_win |&#xA;|:---:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|&#xA;|  1  | Team1 | Team2 |   45  |   10  |   47  |   8   |   1   |&#xA;|  2  | Team3 | Team4 |   56  |   6   |   70  |   13  |   0   |&#xA;| ... |  ...  |  ...  |  ...  |  ...  |  ...  |  ...  |  ...  |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I implemented the model with TensorFlow/Keras (with the help of this tutorial: &lt;a href=&quot;https://www.tensorflow.org/tutorials/structured_data/feature_columns&quot; rel=&quot;nofollow noreferrer&quot;&gt;Classify structured data with feature columns | TensorFlow Core&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is pretty concise:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;batch_size = 16&#xA;train_ds, test_ds, val_ds = get_datasets()  # The function mainly uses tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))&#xA;team_names = get_team_names()&#xA;&#xA;feature_columns = []&#xA;for column_name in ['Home', 'Away']:&#xA;    team = feature_column.categorical_column_with_vocabulary_list(column_name, team_names)&#xA;    feature_columns.append(feature_column.indicator_column(team))&#xA;&#xA;for column_name in ['H_Pts', 'H_Fls', 'A_Pts', 'A_Fls']:&#xA;    feature_columns.append(feature_column.numeric_column(column_name))&#xA;&#xA;feature_layer = tf.keras.layers.DenseFeatures(feature_columns)&#xA;&#xA;model = tf.keras.Sequential([&#xA;    feature_layer,&#xA;    layers.Dense(128, activation='relu'),&#xA;    layers.Dense(128, activation='relu'),&#xA;    layers.Dense(1)&#xA;])&#xA;&#xA;&#xA;model.compile(optimizer='adam', loss=tf.keras.losses.BinaryCrossentropy(from_logits=True), metrics=['accuracy'])&#xA;&#xA;model.fit(train_ds, validation_data=val_ds, epochs=10)&#xA;&#xA;loss, accuracy = model.evaluate(test_ds)&#xA;print(f&quot;Model evaluation: Loss = {loss} | Accuracy = {accuracy}&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Trained with just 100 games, I get a great accuracy: 99%. Of course: as it is, the test dataset given to &lt;code&gt;model.evaluate(test_ds)&lt;/code&gt; contains everything except the target label &lt;code&gt;H_win&lt;/code&gt;. Because &lt;code&gt;H_win&lt;/code&gt; can easily be deduced from &lt;code&gt;H_Pts&lt;/code&gt; and &lt;code&gt;A_Pts&lt;/code&gt;, I get a high accuracy. But this model can't work because by definition you don't know the number of points of each team before the game...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I deal with features like these ones &lt;strong&gt;which I do &lt;em&gt;not&lt;/em&gt; want to predict&lt;/strong&gt; (so they're not labels) but that should still be &lt;strong&gt;considered during the training&lt;/strong&gt;? Does this kind of feature have a name?&lt;/p&gt;&#xA;" OwnerUserId="34130" LastActivityDate="2020-03-09T21:39:28.623" Title="How to deal with features which are here just for training?" Tags="&lt;ai-design&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18525" PostTypeId="2" ParentId="16353" CreationDate="2020-03-09T21:52:35.890" Score="2" Body="&lt;p&gt;Embeddings generated by transformers like Bert or XLM-R are fundamentally different from embeddings learned through language models like GloVe or Word2Vec.&#xA;The latter are static, i.e. they are just dictionaries containing a vocabulary with n-dimensional vectors associated to each word. Because of this they can be plotted through PCA and the distance between them can be easily calculate with whatever metrics you prefer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When training Bert or XLM-R instead you are not learning vectors, but the parameters of a transformer. The embedding for each token are then generated once a token is fed into the transformer. This implies several things, the most important being that the hidden representation (the embedding) for the token change depending on the context (recall that XML-R use as input also the hidden states generated by the previous token). This means that there are no static vectors to compare by plotting them or by calculating the cosine similarity. &#xA;Nevertheless, there are way to analyse and visualise the syntax and semantics encoded in the parameters, this paper show some strategies: &lt;a href=&quot;https://arxiv.org/pdf/1906.02715.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1906.02715.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a more linguistic side, I would also ask why vectors of same words should show the same semantic properties across languages. Surely there are similarities for lot of words translated literally, but the use of some expressions is inherently different across languages. To make a quick example: in English the clock 'works', in Dutch the clock 'lopen' (it walks) and in Italian the clock 'funziona' (it functions). Same expression, three different words in different languages that do not necessarily share the same neighbours in their monolingual latent spaces. &#xA;The point of transformers is exactly to move from static representations to dynamic ones that are able to learn that all those three verbs (in their specific language) can appear early in a sentences and close to the word clock. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-09T21:52:35.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18526" PostTypeId="1" AcceptedAnswerId="18554" CreationDate="2020-03-10T00:59:12.337" Score="2" ViewCount="117" Body="&lt;p&gt;I have collected a set of pictures of people with a text explaining the characteristics of the person on the picture, for example, &quot;Big nose&quot; or &quot;Curly hair&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to train some type of model that takes in any picture and returns a description of the picture in terms of characteristics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have a hard time figuring out how to do this. It is not like labeling &quot;dog&quot; or &quot;apple&quot; because then I can create a set of training data and then evaluate its performance, now I can not. If so I would probably have used a CNN and probably also VGG-16 to help me out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only have two ML courses under my belt and have never really encountered a problem like this before. Can someone help me to get in the right direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of now, I have a data set of 13000 labeled images I am very confident it is labeled well. I do not know of any pre-trained datasets that could be of help in this instance, but if you know of one it might help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Worth noting is that every label is or should at least be unique. If for example there exist two pictures with the same label of &quot;Big nose&quot; it is purely coincidental.&lt;/p&gt;&#xA;" OwnerUserId="4238" LastEditorUserId="2444" LastEditDate="2020-03-11T16:17:12.393" LastActivityDate="2020-03-11T20:55:26.287" Title="How can I train a neural network to describe the characteristics of a picture?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="5" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="18527" PostTypeId="1" CreationDate="2020-03-10T07:22:42.587" Score="2" ViewCount="49" Body="&lt;p&gt;I am newbie to NLP.&#xA;I have a excel sheet with following columns: &#xA;    Server_SNo, &#xA;    Owner, &#xA;    Hosting Dept, &#xA;    Bus owner, &#xA;    Applications hosted, &#xA;    Functionality,&#xA;    comments&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a. Except the Server_SNo, other columns may or may not have data.&lt;br&gt;&#xA;b. For some records there is no data except Server_SNo which is the first column.&#xA;c. One business owner can own more than 1 Server.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, out of 4000 records, about 50% of data contain direct mapping for a server with owner.  Remaining 50% of data have combination of other columns (Owner, Hosting Dept, Bus owner, Applications hosted, Functionality and comments)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my problem, I need to find the owner for the given Server_Sno for 50% of data which have combination of other columns (Owner, Hosting Dept, Bus owner, Applications hosted, Functionality and comments).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have just started to build the code using Python and NLTK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this an NLP problem? Am I going in right direction using Python and NLTK for NLP?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insights is appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Mani&lt;/p&gt;&#xA;" OwnerUserId="34141" LastEditorUserId="34141" LastEditDate="2020-03-12T05:11:15.300" LastActivityDate="2020-04-12T03:02:00.333" Title="Owner Search for given Server SNO" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;python&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18528" PostTypeId="2" ParentId="16353" CreationDate="2020-03-10T08:55:47.770" Score="0" Body="&lt;p&gt;There is a general idea in the field of NLP that there is a mapping between embeddings in different langauges. Figure 1 explains this. &lt;a href=&quot;https://i.stack.imgur.com/Dg7xH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dg7xH.png&quot; alt=&quot;Figure 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Figure 1. we have the embedding of English words and Spanish words, and we see that their exists a mapping between the manifolds associated to this two languages, i.e. Spanish manifold is a distorted image of the English maniflod. This idea was used to create an unsupervised translator in &lt;a href=&quot;https://github.com/facebookresearch/MUSE&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUSE Project&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32493" LastActivityDate="2020-03-10T08:55:47.770" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18530" PostTypeId="1" AcceptedAnswerId="18532" CreationDate="2020-03-10T09:14:49.523" Score="1" ViewCount="50" Body="&lt;p&gt;Gday guys,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'am building a game enviroment (picture) where an agent should position the mouse on the screen (cords upper right corner) and then click to shoot a canonball. If the goal (left) is hit. The agent gets a reward based on the elapsed time between this strike and the last one. If three shots missed. The game is done and the environment will reset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZMK3H.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZMK3H.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The env is done so far. But now I wonder what the action space should look like. How can I make the agent choose some x and y coordinates? And how can I combine this whith a &quot;shoot&quot; action?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help&lt;/p&gt;&#xA;" OwnerUserId="30431" LastEditorUserId="30431" LastEditDate="2020-03-10T09:20:35.457" LastActivityDate="2020-03-10T10:40:13.257" Title="How to let an RL Agent move the mouse?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;intelligent-agent&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18531" PostTypeId="1" CreationDate="2020-03-10T09:16:34.857" Score="2" ViewCount="34" Body="&lt;p&gt;I wanna build a model to read logs and classify them. How should I start? (preferably TensorFlow solutions). The input is gonna be log files.&lt;/p&gt;&#xA;" OwnerUserId="32552" LastEditorUserId="2444" LastEditDate="2020-03-10T20:45:35.853" LastActivityDate="2020-03-10T20:45:35.853" Title="How can I classify logs?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2020-03-10T20:44:57.597" ContentLicense="CC BY-SA 4.0" />
  <row Id="18532" PostTypeId="2" ParentId="18530" CreationDate="2020-03-10T10:40:13.257" Score="0" Body="&lt;p&gt;You could make the actions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;move up&lt;/li&gt;&#xA;&lt;li&gt;move down&lt;/li&gt;&#xA;&lt;li&gt;move left&lt;/li&gt;&#xA;&lt;li&gt;move right&lt;/li&gt;&#xA;&lt;li&gt;shoot&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then you can declare the speed of this movement. If you want to go more in depth you can do&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Accelerate up&lt;/li&gt;&#xA;&lt;li&gt;Accelerate down&lt;/li&gt;&#xA;&lt;li&gt;Accelerate left&lt;/li&gt;&#xA;&lt;li&gt;Accelerate right&lt;/li&gt;&#xA;&lt;li&gt;Shoot&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This will give control of the speed to the neural networkm but is harder to train, and you should give the speed of the mouse as an input.&lt;/p&gt;&#xA;" OwnerUserId="29671" LastActivityDate="2020-03-10T10:40:13.257" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18533" PostTypeId="2" ParentId="8190" CreationDate="2020-03-10T11:29:23.710" Score="10" Body="&lt;p&gt;The two tech reports below both call RNNs explicitly &amp;quot;recurrent net(work)s&amp;quot;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Rumelhart, David E; Hinton, Geoffrey E, and Williams, Ronald J (Sept. 1985). &lt;a href=&quot;https://apps.dtic.mil/dtic/tr/fulltext/u2/a164453.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning internal representations by error propagation&lt;/a&gt;. Tech. rep. ICS 8504. San Diego, California: Institute for Cognitive Science, University of California.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Jordan, Michael I. (May 1986). &lt;a href=&quot;https://www.osti.gov/biblio/6910294&quot; rel=&quot;nofollow noreferrer&quot;&gt;Serial order: a parallel distributed processing approach&lt;/a&gt;. Tech. rep. ICS 8604. San Diego, California: Institute for Cognitive Science, University of California.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://grad.berkeley.edu/news/headlines/jordan-wins-rumelhart-prize/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jordan was a student of Rumelhart&lt;/a&gt;, so I would lean on identifying &lt;a href=&quot;https://apps.dtic.mil/dtic/tr/fulltext/u2/a164453.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; as the paper introducing RNNs, with the caveat that the first sentence in the section &amp;quot;Recurrent Nets&amp;quot; of &lt;a href=&quot;https://apps.dtic.mil/dtic/tr/fulltext/u2/a164453.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; reads:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We have thus far restricted ourselves to &lt;em&gt;feedforward&lt;/em&gt; nets. This may&#xA;seem like a substantial restriction, but as Minsky and Papert point&#xA;out, there is, for every recurrent network, a feedforward network with&#xA;identical behavior (over a finite period of time).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is interesting for two reasons:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;After this sentence, he then goes on to show how RNNs can be unrolled and the error propagated back. Not a full-fledged BPTT yet, though.&lt;/li&gt;&#xA;&lt;li&gt;The sentence shows that the idea of recurrence (and unrolling) has been around since at least 1969.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Unfortunately, I don't have access to Minsky and Papert (1969), so I cannot follow this line any further.&lt;/p&gt;&#xA;" OwnerUserId="34145" LastEditorUserId="2444" LastEditDate="2021-01-18T23:30:20.397" LastActivityDate="2021-01-18T23:30:20.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18534" PostTypeId="2" ParentId="18481" CreationDate="2020-03-10T13:19:50.683" Score="2" Body="&lt;p&gt;Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in your specific case, you could model a bubble bursting as perhaps a negative exponential drop-off or something of the sort. This could include machine learning too. You could gather historical drops in stock market data, and use some sort of regression (Bayesian would probably work well) to estimate the best function to use as an indicator to whether a steep drop has occurred. If so, then use your neural network specifically to classify the fate of the stock. I would think you would have more success following a specialised route such as this rather than trying to train a network on general trends in the market.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the structure of your neural network, you may want to consider a convolutional neural network (CNN) instead of a recurrent neural network (RNN). RNNs assume the current point in your time-series depends on all previous points, from the beginning of your data. I wouldn't think this would hold true in general for the stock market. The filters a CNN learns are suited to learning to extract certain features and the CNN will apply the filters to specific portions of the data, in the way it considers optimal. They are both nonlinear models, but the CNN will be less computationally costly to train. You could also try a gradient-boosting regression approach instead of a neural network. That being said, something like an LSTM RNN (long short-term memory) won't necessarily be bad - just my two cents.&lt;/p&gt;&#xA;" OwnerUserId="22373" LastActivityDate="2020-03-10T13:19:50.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18536" PostTypeId="2" ParentId="18481" CreationDate="2020-03-10T13:59:33.850" Score="2" Body="&lt;p&gt;I do not know how you will apply your data to the techniques I'll give you some brief overview of techniques used in time series prediction:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Extended &lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kalman Filtering&lt;/a&gt;: This is a kind of control system approach and is generally used to control trajectory of missiles. Here is a &lt;a href=&quot;https://ai.stackexchange.com/questions/6091/what-is-the-significance-of-this-stanford-university-financial-market-time-seri&quot;&gt;question&lt;/a&gt; (based on an EKF paper) in our stack on this topic. You can check the paper for more details.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Echo_state_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Echo State Networks&lt;/a&gt;: This is a kind of ML/NN approach with based on the idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Liquid_state_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Liquid State Machines&lt;/a&gt; used in Neuroscience. &lt;a href=&quot;https://mantas.info/code/simple_esn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Resources&lt;/a&gt; on the same.&lt;/li&gt;&#xA;&lt;li&gt;RNNs/LSTMs/GRUs - Probably the most popular approach to predicting any time series data when you don't want to delve into the statistical approaches.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARMA/ARIMA models&lt;/a&gt;: Entirely statistical approach with lots of maths, but libraries are available with implementation already done.&lt;/li&gt;&#xA;&lt;li&gt;Deep Belief Networks: People have also tried forecast time series using fixed number of previous states input to a DBN. It's somewhat of a popular &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0925231213007388&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; so I decided to put it here.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Finally you can look up this &lt;a href=&quot;https://arxiv.org/pdf/1302.6613.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;overview&lt;/a&gt; on time series modelling approaches. Reinforcement Learning is also used for time series prediction, but from what I have heard it is not very easy to do so. Here is a Google Scholar search &lt;a href=&quot;https://scholar.google.co.in/scholar?q=reinforcement%20learning%20for%20time%20series%20scholarly%20articles&amp;amp;hl=en&amp;amp;as_sdt=0&amp;amp;as_vis=1&amp;amp;oi=scholart&quot; rel=&quot;nofollow noreferrer&quot;&gt;result&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-03-10T14:14:59.823" LastActivityDate="2020-03-10T14:14:59.823" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18537" PostTypeId="2" ParentId="18526" CreationDate="2020-03-10T14:09:54.577" Score="0" Body="&lt;p&gt;From what you wrote, the problem sounds a bit like face recognition, where a camera takes a picture of your face and compares it with a bunch of pictures in a database, for example, one for each employee if its at a company's main gate. If you look &quot;similar&quot; to one of the pictures in the database, the door opens and your ID/Name is displayed on a terminal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of system generates an encoding for each picture and evaluates the distance between your encoded picture and the encoding of each picture in the database. If this is at most some minimum value, it's considered a match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what you could do is figure out some way to encode your pictures (say sum the pixel values for a very simple example, ideally you would use some sort of vector here because distances make sense with vectors) and store this encoding together with the label of the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once your database is complete (i.e. you have a bunch of pictures saved as a pair of [encoding, label]), you can &quot;scan&quot; each new picture, calculate its encoding (using the same algorithm that calculated your database encodings) and find the one entry in your database which minimizes the &quot;encoding-distance&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this sounds like a way to solve your problem, you need to come up with a proper encoding (like &quot;run my images through a CNN and save the output of my last fully connected layer&quot;) and apply this to all the images you want to use as &quot;training data&quot;, before &quot;testing&quot; it on some of the leftover images.&lt;/p&gt;&#xA;" OwnerUserId="34149" LastEditorUserId="2444" LastEditDate="2020-03-11T20:55:26.287" LastActivityDate="2020-03-11T20:55:26.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18538" PostTypeId="2" ParentId="16291" CreationDate="2020-03-10T14:24:09.123" Score="2" Body="&lt;p&gt;The section of the book &lt;a href=&quot;https://russell-davidson.arts.mcgill.ca/e706/Perceptrons.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptrons: An Introduction to Computational Geometry&lt;/a&gt; (expanded edition, third printing, 1988) that shows the limitations of the perceptron should be &lt;strong&gt;11.8 The Nonseparable Case&lt;/strong&gt; (p. 181), where the authors write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There are many reasons for studying the operation of the perceptron learning program when there is no &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^*$&lt;/span&gt; with the property &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^* \cdot \mathbf{\Phi} &amp;gt; 0$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\Phi} \in \mathbf{F}$&lt;/span&gt;. Some of these are practical reasons. For example, one might want to use the program to &lt;em&gt;test&lt;/em&gt; whether such an &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^*$&lt;/span&gt; exists, or one might wish to make a learning machine of this sort and be worried about the possible effects of feedback errors and other &quot;noise&quot;. Other motives are theoretical. One cannot claim to have completely understood the &quot;separable case&quot; without at least some broader knowledge of other cases.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In section 12.1.1 (p. 189), the authors further write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The PERCEPTRON scheme works perfectly only under the restriction that the data is linearly separable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-10T14:24:09.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18541" PostTypeId="1" CreationDate="2020-03-10T15:44:34.383" Score="2" ViewCount="46" Body="&lt;p&gt;I want to train a reinforcement learning agent in an environment with parameters (for example, the wind speed, sun irradiation, etc.) that change over time. I have recorded a limited amount of data for these time series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should the RL agent be trained in an environment, which replays the recorded time series over and over, or should I model the time series with a generative model first and train the agent in an environment with these synthetic time series?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the one hand, I think the RL algorithm will perform better with the synthetic data, because there are more diverse trajectories. On the other hand, I don't &lt;em&gt;really&lt;/em&gt; have more data, because it is modelled after the same data the RL algorithm could learn from in the first place. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any papers that elaborate on this topic?&lt;/p&gt;&#xA;" OwnerUserId="20195" LastEditorUserId="2444" LastEditDate="2020-03-10T20:43:35.643" LastActivityDate="2020-03-10T20:43:35.643" Title="Should the RL agent be trained in an environment with real-world data or with a synthetic model?" Tags="&lt;reinforcement-learning&gt;&lt;generative-model&gt;&lt;time-series&gt;&lt;environment&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18542" PostTypeId="1" AcceptedAnswerId="18624" CreationDate="2020-03-10T16:21:02.953" Score="4" ViewCount="63" Body="&lt;p&gt;In Reinforcement Learning, an MDP model incorporates the Markovian property. A lot of scheduling applications in a lot of disciplines use reinforcement learning (mostly deep RL) to learn scheduling decisions. For example, the paper &lt;a href=&quot;https://web.mit.edu/decima/content/sigcomm-2019.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Scheduling Algorithms for Data Processing Clusters&lt;/a&gt;, which is from SIGCOMM 2019, uses Reinforcement Learning for scheduling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't scheduling a non-Markovian process, or am I missing some points? &lt;/p&gt;&#xA;" OwnerUserId="34154" LastEditorUserId="2444" LastEditDate="2020-03-13T17:02:39.860" LastActivityDate="2020-03-13T23:42:54.027" Title="How is the Markovian property consistent in reinforcement learning based scheduling?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;markov-property&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18543" PostTypeId="2" ParentId="1288" CreationDate="2020-03-10T16:28:45.867" Score="0" Body="&lt;p&gt;In section &lt;strong&gt;13.2 Other Multilayer Machines&lt;/strong&gt; (pp. 231-232) of the book &lt;a href=&quot;https://russell-davidson.arts.mcgill.ca/e706/Perceptrons.pdf#page=246&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptrons: An Introduction to Computational Geometry&lt;/a&gt; (expanded edition, third printing, 1988) Minsky and Papert actually talk about their knowledge of or opinions about the capabilities of what they call the &lt;em&gt;multilayered machines&lt;/em&gt; (i.e. perceptrons with many layers or MLPs).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;Have you considered &amp;quot;perceptrons&amp;quot; with many layers?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Well, we have considered &lt;strong&gt;Gamba machines&lt;/strong&gt;, which could be described as &amp;quot;&lt;strong&gt;two layers of perceptron&lt;/strong&gt;&amp;quot;. We have not found (by thinking or by studying the literature) any other really interesting class of multilayered machine, at least none whose principles seem to have a signiﬁcant relation to those of the perceptron. To see the force of this qualiﬁcation it is worth pondering the fact, trivial in itself, that &lt;strong&gt;a universal computer could be built entirely out of linear threshold modules&lt;/strong&gt;. This does not in any sense reduce the theory of computation and programming to the theory of perceptrons. Some philosophers might like to express the relevant general principle by saying that the computer is so much more than the sum of its parts that the computer scientist can aﬁord to ignore the nature of the components and consider only their connectivity. More concretely, we would call the student's attention to the following considerations:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Multilayer machines with loops clearly open all the questions of the general theory of automata.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A system with no loops but with an order restriction at each layer can compute only predicates of ﬁnite order.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On the other hand, if there is no restriction except for the absence of loops, the monster of vacuous generality once more raises its head.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The problem of extension is not merely technical. It is also strategic. The perceptron has shown itself worthy of study despite (and even because of!) its severe limitations, It has many features to attract attention: its linearity; its intriguing learning theorem; its clear paradigmatic simplicity as a kind of parallel computation. &lt;strong&gt;There is no reason to suppose that any of these virtues carry over to the many-layered version. Nevertheless, we consider it to be an important research problem to elucidate (or reject) our intuitive judgment that the extension is sterile. Perhaps some powerful convergence theorem will be discovered, or some profound reason for the failure to produce an interesting&#xA;&amp;quot;learning theorem&amp;quot; for the multilayered machine will be found.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, let me address your first question directly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Backprop wasn't known at the time, but did they know about manually building multilayer perceptrons?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. They say that Gamba machines could be described as a 2-layer perceptron. For reproducibility, here's the definition of the Gamba machine (section &lt;a href=&quot;https://russell-davidson.arts.mcgill.ca/e706/Perceptrons.pdf#page=243&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;13.1 Gamba Perceptrons and other Multilayer Linear Machines&lt;/strong&gt;&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\psi&#xA;&amp;amp;=&#xA;\left[\sum_{i} \alpha_{i}\left[\sum_{j} \beta_{i j} x_{j}&amp;gt;\theta_{i}\right]&amp;gt;\theta\right] \\&#xA;&amp;amp;=&#xA;\left[\sum_{i} \alpha_{i} \varphi_{i} &amp;gt;\theta \right]&#xA;\end{align}&lt;/span&gt;&#xA;See also sections &lt;a href=&quot;https://russell-davidson.arts.mcgill.ca/e706/Perceptrons.pdf#page=220&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;12.4.4. Layer-Machines&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, let's now address your second question.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Did Minsky &amp;amp; Papert know that multilayer perceptrons could solve XOR at the time they wrote the book, albeit not knowing how to train it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, according to the first excerpt, their intuition was the &lt;em&gt;virtues&lt;/em&gt; of perceptrons would not carry over to MLPs, but they acknowledge that more research was needed to reject or support this hypothesis.&lt;/p&gt;&#xA;&lt;p&gt;However, in section &lt;a href=&quot;https://russell-davidson.arts.mcgill.ca/e706/Perceptrons.pdf#page=242&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;13.0 Introduction&lt;/strong&gt;&lt;/a&gt; of the same book, they write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We believe (but cannot prove) that the &lt;strong&gt;deeper limitations extend also to the variant of the perceptron proposed by A. Gamba&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, they believed that the Gamba machine would not have been able to solve the XOR problem.&lt;/p&gt;&#xA;&lt;p&gt;However, in the first excerpt, they say that a Turing machine could be built entirely out of linear threshold modules, which seems to be inconsistent with the second excerpt, but that's not really the case because they are not saying how to build a Turing machine out of the linear threshold modules but that just the specific Gamba machine would have the same limitations of the perceptron.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T00:47:41.100" LastActivityDate="2021-01-19T00:47:41.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18544" PostTypeId="1" CreationDate="2020-03-10T16:44:03.303" Score="1" ViewCount="46" Body="&lt;p&gt;&lt;strong&gt;The Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently working on a sequence classification problem I try to solve with machine learning.&#xA;The target variable is the current state of a system.&#xA;This target variable is following a repeating pattern (eg. [00110200033304...]).&#xA;So Transitions are only allowed from or to the &quot;0&quot; state if you imagine the system as a state machine.&#xA;The only deviation is the time the system stays in one state (eg. iteration_1 = [...0220...], iteration_2 =[...02220...]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best choice of (machine learning) model for this task if one wants to optimize for accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Restrictions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No restrictions regarding time / space complexity of the model&lt;/li&gt;&#xA;&lt;li&gt;No restrictions regarding the type of model&lt;/li&gt;&#xA;&lt;li&gt;The model is only allowed to make wrong classifications in the state transitions phases (e.g. true: [011102...], pred: [001102...]) but must not validate the sequence logic (e.g. true: [011102...], pred: [010102...])&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Info / Existing Work&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;With a lstm neural network (many to 1) I achieved an overall accuracy of 97% in an unseen test set.&#xA;Unfortunately the network predicted sequences which violate the sequence logic&#xA;(e.g. true: [011102...] predicted: [010102...]) even tough the window length was wide enough to cover at least 3 state transitions.&lt;/li&gt;&#xA;&lt;li&gt;with simple classification models (only one times step per classification, tested models: feed forward neural network, xgboost / adaboost) an accuracy of ca. 70% are reachable&lt;/li&gt;&#xA;&lt;li&gt;The input signal is acoustic emission in the frequency domain; Ca. 100 frequency bins / 100 features&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ideas&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maybe the lstm would work better in &quot;many to many&quot; designed with a drastic reduced input dimensionality by increased window size?&lt;/li&gt;&#xA;&lt;li&gt;Maybe a combination of the probability output of the lstm with&#xA;a timed Automaton (a state machine with time dependent probability density functions&#xA;about the state changes) or a Markov chain model could significantly improve the result?&#xA;(But this seems really inelegant)&lt;/li&gt;&#xA;&lt;li&gt;Is it eventually possible to impose the restriction of valid sequences onto the lstm model?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="34155" LastActivityDate="2020-03-10T16:44:03.303" Title="Model for supervised sequence classification task" Tags="&lt;long-short-term-memory&gt;&lt;supervised-learning&gt;&lt;time-series&gt;&lt;sequence-modelling&gt;&lt;graphs&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18545" PostTypeId="2" ParentId="18526" CreationDate="2020-03-10T16:51:41.333" Score="0" Body="&lt;p&gt;I would do as suggested in the comments. First select an encoding scheme. I think what is called a difference hash would work well for this application. Code for that is shown below. Now take your data set of images and run them through the encoder and save the result in a database. The database would contain the &quot;labeling&quot; text and the encoder result. Now for a new image you are trying to label, input the image into the encoder. Take the encoder result and compare it to the encoded values in the database. Search through the encoded values in the database and find the closest match. You can then use a &quot;threshold&quot; value to determine if you want to give a specific label for the image or if the distance is above the threshold declare there is no matching label. You can determine the best &quot;threshold&quot; value by running you data set images with the known labels and iterate the threshold level and select the threshold with the least errors. I would use something like a 56 or a 128 length hash.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cv2&#xA;import os&#xA;# f_path is the full path to the image file, hash length is an integer specifies length of the hash&#xA;def get_hash(f_path, hash_length):    &#xA;    r_str=''    &#xA;    img=cv2.imread(f_path,0)        # read image as gray scale image&#xA;    img = cv2.resize(img, (hash_length+1, 1), interpolation = cv2.INTER_AREA)    &#xA;    # now compare adjacent horizontal values in a row if pixel to the left&amp;gt;pixel toright result=1 else 0&#xA;    for col in range (0,hash_length):&#xA;        if(img[0][col]&amp;gt;img[0][col+1]):&#xA;            value=str(1)&#xA;        else:&#xA;            value=str(0)&#xA;        r_str=r_str + value&#xA;    number=0&#xA;    power_of_two=1&#xA;    for char in r_str:        &#xA;        number = number + int(char) * power_of_two&#xA;        power_of_two=2 * power_of_two    &#xA;    return ( r_str, number) &#xA;# example on an image of a bird&#xA;f_path=r'c:\Temp\birds\test\robin\1.jpg'&#xA;hash=get_hash ( f_path, 16) # 16 length hash on a bird image&#xA;print (' hash string ', hash[0], '   hash number ', hash[1])&#xA;&#xA;&amp;gt; results is&#xA; hash string  1111111100000000    hash number  255&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-10T16:51:41.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18546" PostTypeId="1" CreationDate="2020-03-10T17:16:24.273" Score="2" ViewCount="110" Body="&lt;p&gt;Consider the following simple neural network with only one neuron.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The input is &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$-250 &amp;lt; x &amp;lt; 250$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-250 &amp;lt; y &amp;lt; 250$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;The weights of the only neuron are &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;The output of the neuron is given by &lt;span class=&quot;math-container&quot;&gt;$o = \sigma(x_1w_1 + x_2w_2 + b)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the ReLU activation function and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; the bias.&lt;/li&gt;&#xA;&lt;li&gt;Thus the cost should be &lt;span class=&quot;math-container&quot;&gt;$(o - y)^2$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When using the sigmoid activation function, the target for each point is usually &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm a little confused witch target to use when the activation function is the ReLU, given that it can output numbers greater than 1.&lt;/p&gt;&#xA;" OwnerUserId="34153" LastEditorUserId="2444" LastEditDate="2020-03-10T20:59:50.707" LastActivityDate="2020-03-11T03:52:24.983" Title="How to determine the target value when using ReLU as activation function?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;relu&gt;&lt;mean-squared-error&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18547" PostTypeId="2" ParentId="8534" CreationDate="2020-03-10T17:24:12.393" Score="1" Body="&lt;p&gt;The paper (or report) that formally introduced the perceptron is &lt;a href=&quot;https://blogs.umass.edu/brain-wars/files/2016/03/rosenblatt-1957.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Perceptron — A Perceiving and Recognizing Automaton&lt;/a&gt; (1957) by Frank Rosenblatt. If you read the first page of this paper, you can immediately understand that's the case. In particular, at some point (page 2, which corresponds to page 5 of the pdf), he writes&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Recent theoretical studies by this writer indicate that it should be feasible to construct an electronic or electromechanical system which will learn to recognize similarities or identities between patterns of optical, electrical, or tonal information, in a manner which may be closely analogous to the perceptual processes of a biological brain. The proposed system depends on probabilistic rather than deterministic principles for its operation, and gains its reliability from the properties of statistical measurements obtained from large populations of elements. A system which operates according to these principles will be called a &lt;strong&gt;perceptron&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See also Appendix I (page 19, which corresponds to page 22 of the pdf).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.3398&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The perceptron: A probabilistic model for information storage and organization in the brain&lt;/a&gt; (1958) by F. Rosenblatt is apparently an updated and nicer version of the original report.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more accessible (although not the most intuitive) description of the perceptron model and its learning algorithms can be found in the famous book &lt;a href=&quot;https://russell-davidson.arts.mcgill.ca/e706/Perceptrons.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perceptrons: An Introduction to Computational Geometry&lt;/a&gt; (expanded edition, third printing, 1988) by Minsky and Papert (from page 161 onwards).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-10T17:59:01.610" LastActivityDate="2020-03-10T17:59:01.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18550" PostTypeId="1" CreationDate="2020-03-10T18:36:57.750" Score="1" ViewCount="26" Body="&lt;p&gt;Here's the data I have:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Text from articles from various music blogs &amp;amp; music news sites (title, summary, full content, and sometimes tags).   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I used a couple different NLP/NER tools (nltk, spacy, and stanford NER) to determine the proper nouns in the text, and gave each proper noun a score based on how many times it appeared, and how many NLP tools recognized it as a proper noun. None of these tools are very accurate by themselves for my data&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each proper noun I queried &lt;a href=&quot;https://musicbrainz.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;musicbrainz&lt;/a&gt; to find artists with that name. (musicbrainz has a lot of data that may be helpful: aliases, discography, associations with other artists)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any links in the article to Spotify, YouTube etc. and the song name &amp;amp; artist for that link  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have three goals:    &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine which proper nouns are artists  &lt;/li&gt;&#xA;&lt;li&gt;For artists that share the same name, determine which one the text is referring to (based on musicbrainz data)  &lt;/li&gt;&#xA;&lt;li&gt;Determine if the artist is important to the article, or if they were just briefly mentioned&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have manually tagged some of the data with the correct output for the above 3 goals.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you go about this? Which algorithms do you think would be best for these goals?&lt;br&gt;&#xA;Is there any semi-supervised learning I can do to reduce the amount of tagging I need to do?&lt;/p&gt;&#xA;" OwnerUserId="34158" LastActivityDate="2020-03-10T18:36:57.750" Title="What algorithm to use for finding artists/bands in text and differentiating between artists that share the same name" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18551" PostTypeId="1" AcceptedAnswerId="18553" CreationDate="2020-03-10T20:38:38.820" Score="1" ViewCount="153" Body="&lt;p&gt;I am training a classifier to identify 24 hand signs of American Sign Language. I created a custom dataset by recording videos in different backgrounds for each of the signs and later converted the videos into images. Each sign has 3000 images, that were randomly selected to generate a training dataset with 2400 images/sign and validation dataset with the remaining 600 images/sign.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Total number of images in entire dataset: 3000 * 24 = 72000&lt;/li&gt;&#xA;&lt;li&gt;Training dataset: 2400 * 24 = 57600&lt;/li&gt;&#xA;&lt;li&gt;Validation dataset: 600 * 24 = 14400&lt;/li&gt;&#xA;&lt;li&gt;Image dimension (Width x Height): 1280 x 720 pixels&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;The CNN architecture used for training&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential([&#xA;    Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;    Dropout(0.25),&#xA;&#xA;    Conv2D(32, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;    Dropout(0.25),&#xA;&#xA;    Conv2D(64, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;    Dropout(0.25),&#xA;&#xA;    Conv2D(64, (3, 3), activation='relu'),&#xA;    MaxPooling2D(pool_size=(2,2)),&#xA;    Dropout(0.25),&#xA;&#xA;    Flatten(),&#xA;    Dense(128, activation='relu'),&#xA;    Dropout(0.25),&#xA;&#xA;    Dense(NUM_CLASSES, activation='softmax')&#xA;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Training parameters:&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IMG_HEIGHT = 224&#xA;IMG_WIDTH = 224&#xA;BATCH_SIZE = 32&#xA;NUM_CLASSES = 24&#xA;train_datagen = ImageDataGenerator(rescale = 1./255,&#xA;                                   width_shift_range=0.1,&#xA;                                   height_shift_range=0.1,&#xA;                                   zoom_range=0.1,&#xA;                                   fill_mode='constant')&#xA;EPOCHS = 20&#xA;STEPS_PER_EPOCH = TRAIN_TOTAL // BATCH_SIZE&#xA;VALIDATION_STEPS = VALIDATION_TOTAL // BATCH_SIZE&#xA;&#xA;callbacks_list = [&#xA;    tf.keras.callbacks.EarlyStopping(monitor = 'accuracy',&#xA;                                     min_delta = 0.005,&#xA;                                     patience = 3),&#xA;    tf.keras.callbacks.ModelCheckpoint(filepath = 'D:\\Models\\HSRS_ThesisDataset_5Mar_1330.h5',&#xA;                                       monitor= 'val_loss',&#xA;                                       save_best_only = True)&#xA;]&#xA;&#xA;optimizer = 'adam'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The model accuracy and model loss graph is shown in the figure below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tQdab.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tQdab.png&quot; alt=&quot;Model Accuracy and Loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The results obtained at the end of the training are&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train acc: 0.8000121&lt;/li&gt;&#xA;&lt;li&gt;Val acc: 0.914441&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I read &lt;a href=&quot;https://www.pyimagesearch.com/2019/10/14/why-is-my-validation-loss-lower-than-my-training-loss/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article explaining why the validation loss is lower than the training loss&lt;/a&gt; I want to know:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it because of the smaller dataset and random shuffling of the images?&lt;/li&gt;&#xA;&lt;li&gt;Is there any way to improve the condition without changing the dataset?&lt;/li&gt;&#xA;&lt;li&gt;Will this have a very detrimental effect on the model performance in real test cases? If not, can I just focus on improving the training accuracy of the overall model?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33467" LastEditorUserId="2444" LastEditDate="2020-06-11T11:36:44.937" LastActivityDate="2020-06-11T11:36:44.937" Title="Why is the validation performance better than the training performance?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18552" PostTypeId="1" CreationDate="2020-03-10T21:02:21.317" Score="1" ViewCount="21" Body="&lt;p&gt;Say I am using a convolutional network to classify pictures of my face versus anyone else's face in the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's take 10000 pictures of me, and 10000 pictures of other people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And let's do three experiments where we train a binary classifier:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1)&lt;/strong&gt; The 10000 &quot;other&quot; pictures are of 1 other person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2)&lt;/strong&gt; The 10000 &quot;other&quot; pictures are of ~10 other people (approximately balanced, so about 1000 pictures per person).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3)&lt;/strong&gt; The 10000 &quot;other&quot; pictures are of ~10000 other people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only have one question but here are some different perspectives on it:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Are any of these cases categorically harder to solve than the others?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are they the same difficulty, or close?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there known considerations to make when tuning the model for each&#xA;of the cases? (like maybe case (3) has a sharper minimum in the loss&#xA;function than (1) so we need to use a different optimisation approach)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-03-10T21:02:21.317" Title="Is it okay to have wide variations within one of the classes for binary classification tasks?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18553" PostTypeId="2" ParentId="18551" CreationDate="2020-03-10T21:41:34.850" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;Assuming you pass through the entire validation dataset, this can't be due to shuffling since you still compute the loss/accuracy over the entire dataset, so order does not really matter here. It is more likely that you have a &lt;strong&gt;&lt;em&gt;significantly smaller or less representative validation dataset&lt;/em&gt;&lt;/strong&gt;, e.g., distribution of the validation dataset can be skewed towards classes where your model performs better.&lt;/li&gt;&#xA;&lt;li&gt;What do you mean exactly by improving the situation? Having a better validation accuracy is not necessarily bad. In any case, if you decrease the effect of regularization, e.g., lowering weight decay, training accuracy might go up but your model might generalize worse, i.e.,  you might get a lower validation accuracy.&lt;/li&gt;&#xA;&lt;li&gt;No, the goal of training is never to maximize training accuracy. You can trivially do so by just memorizing the training dataset. In short, the goal of training is to get good generalization and as long as you get a satisfactory validation accuracy, it is likely that this has happened to some degree (assuming you have a &lt;strong&gt;&lt;em&gt;good&lt;/em&gt;&lt;/strong&gt; validation dataset of course).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-03-10T21:41:34.850" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18554" PostTypeId="2" ParentId="18526" CreationDate="2020-03-10T22:31:00.067" Score="2" Body="&lt;p&gt;The term you are looking for is &lt;strong&gt;multi-label classification&lt;/strong&gt;, i.e. where you are making more than one classification on each image (one for each label). Most examples you'll find online are in the NLP domain but it is just as easy with CNNs since it's essentially defined by the structure of the output layer and the loss function used. It's not as complicated as it might sound if you are already familiar with CNNs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output layer of a neural network (for 3 or more classes) has as many units as there are targets. The network learns to associate each of those units with a corresponding class. A &lt;strong&gt;multi-class classifier&lt;/strong&gt; normally applies a softmax activation function to the raw unit output, which yields a probability vector. To get the final classification, the &lt;code&gt;max()&lt;/code&gt; of the probability vector is taken (the most probable class). The output would look like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                 Cat    Bird   Plane   Superman  Ball   Dog   &#xA;Raw output:      -1     2      3       6         -1     -1&#xA;Softmax:         0.001  0.017  0.046   0.934     0.001  0.001&#xA;Classification:  0      0      0       1         0      0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multi-label classification&lt;/strong&gt; typically uses a sigmoid activation function since the probabilities of a label occuring can be treated independently. The classification is then determined by the probability (&gt;=0.5 for True). For your problem, this output could look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                 Big nose  Long hair  Curly hair  Superman  Big ears  Sharp Jawline&#xA;Raw output:      -1        -2         3           6         -1        10&#xA;Sigmoid:         0.269     0.119      0.953       0.998     0.269     1.000&#xA;Classification:  0         0          1           1         0         1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;binary crossentropy&lt;/strong&gt; loss function is normally used for a multi-label classifier since a &lt;em&gt;n&lt;/em&gt;-label problem is essentially splitting up a multi-class classification problem into &lt;em&gt;n&lt;/em&gt; binary classification problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since all you need to do to get from a multi-class classifier to a multi-label classifier is change the output layer, its very easy to do with pre-trained networks. If you get the pre-trained model from Keras its as simple as including &lt;code&gt;include_top=False&lt;/code&gt; when downloading the model and then adding the correct output layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With 13000 images, I would recommend using Keras' &lt;code&gt;ImageDataGenerator&lt;/code&gt; class with the &lt;code&gt;flow_from_dataframe&lt;/code&gt; method. This allows you to use a simple pandas dataframe to label and feed in all your images. The dataframe would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Filename  Big nose  Long hair  Curly hair  Superman  Big ears  Sharp Jawline&#xA;0001.JPG  0         0          1           1         0         1&#xA;0002.JPG  1         0          1           0         1         1&#xA;   .      .         .          .           .         .         .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;flow_from_dataframe&lt;/code&gt;'s &lt;code&gt;class_mode&lt;/code&gt; parameter can be set to &lt;code&gt;raw&lt;/code&gt; or &lt;code&gt;multi_output&lt;/code&gt; along with &lt;code&gt;x_col&lt;/code&gt; to &lt;code&gt;'Filename'&lt;/code&gt; and &lt;code&gt;y_col&lt;/code&gt; to &lt;code&gt;['Big nose', 'Long hair', 'Curly hair', 'Superman', 'Big ears', 'Sharp Jawline']&lt;/code&gt; (in this example). Check out the &lt;a href=&quot;https://keras.io/preprocessing/image/#imagedatagenerator-class&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The amount of data you need &lt;strong&gt;for each label&lt;/strong&gt; depends on many factors and is essentially impossible to know without trying. 13000 sounds like a good start but it also depends on how many labels you have and how evenly distributed they are between the labels. A decent guide (one of many) on how to set up a multi-label classifier and how to implement it with Keras can be found &lt;a href=&quot;https://towardsdatascience.com/multi-label-image-classification-with-neural-network-keras-ddc1ab1afede&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It also covers imbalances on label frequency and is well worth a read. I'd &lt;em&gt;highly&lt;/em&gt; recommend that you become as intimately familiar with your dataset as possible before you start tuning your neural network architecture.&lt;/p&gt;&#xA;" OwnerUserId="31980" LastEditorUserId="31980" LastEditDate="2020-03-10T22:36:23.413" LastActivityDate="2020-03-10T22:36:23.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18555" PostTypeId="2" ParentId="17999" CreationDate="2020-03-10T23:52:17.530" Score="1" Body="&lt;p&gt;There's a neuroscience theory, known as &lt;a href=&quot;https://www.frontiersin.org/research-topics/599/predictive-coding&quot; rel=&quot;nofollow noreferrer&quot;&gt;predictive coding&lt;/a&gt;, which roughly states that the (human) brain is constantly generating and updating a model of the world.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The brain is constantly confronted with a wealth of sensory information that must be processed efficiently to facilitate appropriate reactions. One way of optimizing this processing effort is to predict incoming sensory information based on previous experience so that expected information is processed efficiently and resources can be allocated to novel or surprising information. Theoretical and computational studies led to the formulation of the predictive coding framework (Friston 2005, Hawkins and Blakeslee 2004, Mumford 1992, Rao and Ballard 1999). Predictive coding states that the brain continually generates models of the world based on context and information from memory to predict sensory input. In terms of brain processing, a predictive model is created in higher cortical areas and communicated through feedback connections to lower sensory areas. In contrast, feedforward connections process and project an error signal, i.e. the mismatch between the predicted information and the actual sensory input (Rao &amp;amp; Ballard, 1999). The predictive model is constantly updated according to this error signal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This theory should not be surprising or unintuitive, given that every person possesses a slightly different perspective (or model) of the world, which is based on her (or his) personal experiences. Of course, this is just a theory, which may not be the most precise one that describes our brain, but this theory is already being &lt;a href=&quot;https://www.frontiersin.org/research-topics/599/predictive-coding&quot; rel=&quot;nofollow noreferrer&quot;&gt;validated by a number of brain imaging studies investigating predictive feedback and the processing of prediction errors.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Therefore, artificial intelligence may not be the only entity that is based on or will be limited by a model of the world. To answer your question more directly, yes, the AI will always be limited by its model and environment (e.g. hardware), in a similar way that &lt;a href=&quot;https://en.wikipedia.org/wiki/Flatland&quot; rel=&quot;nofollow noreferrer&quot;&gt;flatlanders&lt;/a&gt; are limited by their 2-dimensional nature and world, but this doesn't necessarily mean we will not be able to create useful (and even sophisticated or human-like) AI systems.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-07T22:20:05.943" LastActivityDate="2021-02-07T22:20:05.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18557" PostTypeId="2" ParentId="18551" CreationDate="2020-03-11T01:15:21.047" Score="0" Body="&lt;pre&gt;&lt;code&gt;Validation dataset: 600 * 24 = 14400&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Means that you are augmenting the validation set, right? For an experiment, you can do that and it might take validation accuracy more than train accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of augmentation in only valid for the training set and you should not change the validation set or test set.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try without the augmentation in the validation set and see the result.&lt;/p&gt;&#xA;" OwnerUserId="34164" LastActivityDate="2020-03-11T01:15:21.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18558" PostTypeId="1" CreationDate="2020-03-11T02:07:53.710" Score="0" ViewCount="125" Body="&lt;p&gt;I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in &lt;a href=&quot;https://arxiv.org/pdf/1412.6572.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explaining And Harnessing&#xA;Adversarial Examples&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m4xA5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m4xA5.png&quot; alt=&quot;SimplePic&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling to actually calculate the gradient of the SVM cost function with respect to the input (I am assuming it needs to be w.r.t input).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to avoid the maths (I am working in python if that helps?)&lt;/p&gt;&#xA;" OwnerUserId="29877" LastEditorUserId="9947" LastEditDate="2020-03-11T17:50:29.107" LastActivityDate="2021-04-08T04:03:45.460" Title="How do you perform a gradient based adversarial attack on an SVM based model?" Tags="&lt;loss-functions&gt;&lt;gradient-descent&gt;&lt;papers&gt;&lt;support-vector-machine&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18559" PostTypeId="2" ParentId="18526" CreationDate="2020-03-11T02:08:20.820" Score="1" Body="&lt;p&gt;You can use image captioning. Look at the article &lt;a href=&quot;https://link.medium.com/qx7ZMHsWK4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Captioning Images with CNN and RNN, using PyTorch&lt;/a&gt;. The idea is very profound. The model encodes the image to high dimensional space and then passes it through LSTM cells and LSTM cells produce linguistic output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/image_captioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image captioning with visual attention&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="34164" LastEditorUserId="2444" LastEditDate="2020-03-11T20:54:08.833" LastActivityDate="2020-03-11T20:54:08.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18560" PostTypeId="2" ParentId="18546" CreationDate="2020-03-11T02:45:02.240" Score="1" Body="&lt;p&gt;ReLU and sigmoid have different properties (i.e. range), as you already noticed. I've never seen the ReLU being used as the activation function of the output layer (but some people may use it for some reason, &lt;a href=&quot;https://datascience.stackexchange.com/q/47336/10640&quot;&gt;e.g. regression tasks where the output needs to be positive&lt;/a&gt;). ReLU is usually used as the activation function of a hidden layer. However, in your case, you don't have hidden layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sigmoid function is used as the activation function of the output layer when you need to interpret the output of the neural network as a probability, i.e. a number between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, given that the sigmoid function does exactly this, i.e. it squashes its input to the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\text{sigmoid}(x) = p \in [0, 1]$&lt;/span&gt;. When do you need the output of the network to be a probability?  For example, if you decide to use the cross-entropy loss function (which is equivalent to the negative log-likelihood), then the output of your network should be a probability. For example, if you need to solve a binary classification task, then the combination of a sigmoid as the activation function of the output layer and the binary cross-entropy as the loss function is probably what you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also have a classification problem with more than 2 classes (multi-class classification problem). In that case, you probably need to use a softmax as the activation function of your network combined with a cross-entropy loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this question &lt;a href=&quot;https://stackoverflow.com/q/47034888/3924118&quot;&gt;How to choose cross-entropy loss in TensorFlow?&lt;/a&gt; on Stack Overflow for more info about different cross-entropy functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, in general, the targets don't necessarily need to be restricted to be 0 or 1. For example, if you are solving a regression task, your target may just be any number. However, in that case, you may need another loss function (which is often the &lt;em&gt;mean squared error&lt;/em&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-11T02:50:11.023" LastActivityDate="2020-03-11T02:50:11.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18561" PostTypeId="2" ParentId="17999" CreationDate="2020-03-11T03:11:04.143" Score="1" Body="&lt;p&gt;AI is internally limited by model and externally limited by the environment.&lt;/p&gt;&#xA;&lt;p&gt;Humans are externally limited by the environment but not necessarily internally limited by a computable model (as AI is).&lt;/p&gt;&#xA;&lt;p&gt;So, humans may possess certain skills (e.g. creativity) that an AI may never possess. I had previously asked a related question &lt;a href=&quot;https://ai.stackexchange.com/q/10139/2444&quot;&gt;Are human brain processes, like creativity, intuition or imagination, computable processes?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Which research work supports my claims?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Brian Cantwell Smith says that there is no computation without representation (a model).&lt;/p&gt;&#xA;&lt;p&gt;In the article &lt;a href=&quot;https://www.technologyreview.com/s/511421/the-brain-is-not-computable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Brain Is Not Computable&lt;/a&gt;, Miguel Nicolelis, a top neuroscientist at Duke University, also says&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The brain is not computable and no engineering can reproduce it&lt;/p&gt;&#xA;&lt;p&gt;You can't predict whether the stock market will go up or down because you can’t compute it.&lt;/p&gt;&#xA;&lt;p&gt;You could have all the computer chips ever in the world and you won't create a consciousness.&lt;/p&gt;&#xA;&lt;p&gt;That's because its most important features are the result of unpredictable, nonlinear interactions among billions of cells&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="21644" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-11T13:05:38.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18562" PostTypeId="2" ParentId="18546" CreationDate="2020-03-11T03:38:37.697" Score="1" Body="&lt;p&gt;You are misunderstanding something. You are mixing up inner layers with the output layer. But the question was very good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fist of all, with the only one layer and one neuron neural networks it does not exist. Only one layer can not bring nonlinearity in the network. &#xA;One neuron network means it's a linear regression or logistic regression if it passes through a sigmoid activation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to look at NN like below.&#xA;The output of a neural networks is the final layer output.&#xA;There are two possible situations in the model. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Classification model: For classification people generally use softmax and it has more than one output and the maximum value always below 1(It's a probability distribution).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Regression model: A regression model has a continuous output like the output you mentioned in your problem statement. It has only one output channel. In the regression model, you can make the output as a linear combination of the previous layers (without defining the output range). You can use ReLU if you are sure your prediction is always positive.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="34164" LastEditorUserId="34164" LastEditDate="2020-03-11T03:52:24.983" LastActivityDate="2020-03-11T03:52:24.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18563" PostTypeId="2" ParentId="17291" CreationDate="2020-03-11T03:44:18.933" Score="3" Body="&lt;p&gt;Yes, PAC learning can be relevant in practice. There's an area of research that combines PAC learning and Bayesian learning that is called &lt;strong&gt;PAC-Bayesian&lt;/strong&gt; (or &lt;strong&gt;PAC-Bayes&lt;/strong&gt;) &lt;strong&gt;learning&lt;/strong&gt;, where the goal is to find PAC-like bounds for Bayesian estimators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;strong&gt;Theorem 1 (McAllester’s bound)&lt;/strong&gt; of the paper &lt;a href=&quot;https://arxiv.org/pdf/1901.05353.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A primer on PAC-Bayesian learning&lt;/a&gt; (2019) by Benjamin Guedj, who provides a nice overview of the topic, shows a certain bound that can be used to design Bayesian estimators. An advantage of PAC-Bayes is that you get bounds on the generalization ability of the Bayesian estimator, so you do not necessarily need to test your estimator on a test dataset. Sections 5 and 6 of the paper go into the details of the real-world applications of PAC-Bayes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. &lt;a href=&quot;http://jmlr.org/papers/volume16/germain15a/germain15a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Risk Bounds for the Majority Vote: From a PAC-Bayesian&#xA;Analysis to a Learning Algorithm&lt;/a&gt; (2015) by P. Germain et al. for a specific application of PAC-Bayes. There's also the &lt;a href=&quot;https://github.com/GRAAL-Research/majority-vote-bounds&quot; rel=&quot;nofollow noreferrer&quot;&gt;related Python implementation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://bguedj.github.io/icml2019/material/main.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these related slides&lt;/a&gt; and &lt;a href=&quot;https://bguedj.github.io/icml2019/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt; (by the same author and John Shawe-Taylor) that will point you to their video tutorials about the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The VC dimension can also be useful in practice. For example, in the paper &lt;a href=&quot;http://www.jmlr.org/papers/volume20/17-669/17-669.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model Selection via the VC Dimension&lt;/a&gt; (2019) M. Mpoudeu et al. describe a method for model selection based on the VC dimension.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-11T04:40:24.590" LastActivityDate="2020-03-11T04:40:24.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18564" PostTypeId="1" CreationDate="2020-03-11T06:04:57.690" Score="1" ViewCount="50" Body="&lt;p&gt;I have a GRU model which has 12 features as inputs and I'm trying to predict output power. I really do not understand though whether I choose&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 layer or 5 layers&lt;/li&gt;&#xA;&lt;li&gt;50 neurons or 512 neuron&lt;/li&gt;&#xA;&lt;li&gt;10 epochs with a small batch size or 100 eopochs with a large batch size &lt;/li&gt;&#xA;&lt;li&gt;Different optimizers and activation functions&lt;/li&gt;&#xA;&lt;li&gt;Dropput and L2 regurlarization &lt;/li&gt;&#xA;&lt;li&gt;Adding more dense layer.&lt;/li&gt;&#xA;&lt;li&gt;Increasing and Decreasing learning rate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My results are always the same and doesn't make any sense, my loss and val_loss loss is very steep in first 2 epochs and then for the rest it becomes constant with small fluctuations in val_loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code and a figure of losses, and my dataframes if needed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dataframe1: &lt;a href=&quot;https://drive.google.com/file/d/1I6QAU47S5360IyIdH2hpczQeRo9Q1Gcg/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/1I6QAU47S5360IyIdH2hpczQeRo9Q1Gcg/view&lt;/a&gt;&#xA;Dataframe2: &lt;a href=&quot;https://drive.google.com/file/d/1EzG4TVck_vlh0zO7XovxmqFhp2uDGmSM/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/1EzG4TVck_vlh0zO7XovxmqFhp2uDGmSM/view&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import tensorflow as tf&#xA;import matplotlib.pyplot as plt&#xA;from sklearn.model_selection import train_test_split&#xA;from sklearn.preprocessing import MinMaxScaler&#xA;from google.colab import files&#xA;from tensorboardcolab import TensorBoardColab, TensorBoardColabCallback&#xA;tbc=TensorBoardColab() # Tensorboard&#xA;from keras.layers.core import Dense&#xA;from keras.layers.recurrent import GRU&#xA;from keras.models import Sequential&#xA;from keras.callbacks import EarlyStopping&#xA;from keras import regularizers&#xA;from keras.layers import Dropout&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;df10=pd.read_csv('/content/drive/My Drive/Isolation Forest/IF 10 PERCENT.csv',index_col=None)&#xA;df2_10= pd.read_csv('/content/drive/My Drive/2019 Dataframe/2019 10minutes IF 10 PERCENT.csv',index_col=None)&#xA;&#xA;X10_train= df10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]&#xA;X10_train=X10_train.values&#xA;&#xA;y10_train= df10['Power_kW']&#xA;y10_train=y10_train.values&#xA;&#xA;X10_test= df2_10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]&#xA;X10_test=X10_test.values&#xA;&#xA;y10_test= df2_10['Power_kW']&#xA;y10_test=y10_test.values&#xA;&#xA;&#xA;&#xA;&#xA;# scaling values for model&#xA;&#xA;&#xA;x_scale = MinMaxScaler()&#xA;y_scale = MinMaxScaler()&#xA;&#xA;X10_train= x_scale.fit_transform(X10_train)&#xA;y10_train= y_scale.fit_transform(y10_train.reshape(-1,1))&#xA;X10_test=  x_scale.fit_transform(X10_test)&#xA;y10_test=  y_scale.fit_transform(y10_test.reshape(-1,1))&#xA;&#xA;&#xA;X10_train = X10_train.reshape((-1,1,12)) &#xA;X10_test = X10_test.reshape((-1,1,12))&#xA;&#xA;&#xA;&#xA;Early_Stop=EarlyStopping(monitor='val_loss', patience=3 , mode='min',restore_best_weights=True)&#xA;&#xA;&#xA;&#xA;# creating model using Keras&#xA;model10 = Sequential()&#xA;model10.add(GRU(units=200, return_sequences=True, input_shape=(1,12),activity_regularizer=regularizers.l2(0.0001)))&#xA;model10.add(GRU(units=100, return_sequences=True))&#xA;model10.add(GRU(units=50))&#xA;#model10.add(GRU(units=30))&#xA;model10.add(Dense(units=1, activation='linear'))&#xA;model10.compile(loss=['mse'], optimizer='adam',metrics=['mse']) &#xA;model10.summary() &#xA;&#xA;history10=model10.fit(X10_train, y10_train, batch_size=1500,epochs=100,validation_split=0.1, verbose=1, callbacks=[TensorBoardColabCallback(tbc),Early_Stop])&#xA;&#xA;&#xA;score = model10.evaluate(X10_test, y10_test)&#xA;print('Score: {}'.format(score))&#xA;&#xA;&#xA;&#xA;y10_predicted = model10.predict(X10_test)&#xA;y10_predicted = y_scale.inverse_transform(y10_predicted)&#xA;&#xA;y10_test = y_scale.inverse_transform(y10_test)&#xA;&#xA;&#xA;plt.scatter( df2_10['WindSpeed_mps'], y10_test, label='Measurements',s=1)&#xA;plt.scatter( df2_10['WindSpeed_mps'], y10_predicted, label='Predicted',s=1)&#xA;plt.legend()&#xA;plt.savefig('/content/drive/My Drive/Figures/we move on curve6 IF10.png')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yNFDH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yNFDH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34168" LastEditorUserId="34168" LastEditDate="2020-03-11T07:11:35.170" LastActivityDate="2020-03-11T07:11:35.170" Title="Neural Network Results always the same" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18565" PostTypeId="1" AcceptedAnswerId="18570" CreationDate="2020-03-11T06:16:44.427" Score="0" ViewCount="150" Body="&lt;p&gt;I don't know if this is the right place to ask this question. If it is not, please tell me and I remove it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've just started to learn CNN and I'm trying to understand what they do and how they do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written some sentences to describe it:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Let's say that CNN is a mathematical function that adjusts its values based on the result obtained and the desired output.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The values are the filters of the convolutional layers (in other&#xA;types of networks would be the weights).&lt;/li&gt;&#xA;&lt;li&gt;To adjust these values there is a backpropagation method as in all networks.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The result obtained is an image of the same size as the original.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In this image you can see the delimited area.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The goal of the network is to learn these filters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The overfitting may be because the network has learned where the pixels you are looking for are located.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The filters have as input a pixel of the image and return a 1 or a 0.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My doubt is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your own words, Have I forgotten something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;:&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is only one question. The six points above are affirmative sentences, not questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is only one question mark, and it is on my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to clarify this because someone has closed my previous question because she/he thinks there were more than one question.&lt;/p&gt;&#xA;" OwnerUserId="4920" LastActivityDate="2020-03-20T15:53:32.083" Title="Understanding CNN in a few sentences" Tags="&lt;convolutional-neural-networks&gt;&lt;theory&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18567" PostTypeId="1" AcceptedAnswerId="21599" CreationDate="2020-03-11T09:25:57.050" Score="1" ViewCount="92" Body="&lt;p&gt;Currently I'm working on a continuous control problem using DDPG as my RL algorithm. All in all, things are working out quite well, but the algorithm does not show any tendencies to eliminate the steady state control deviation towards the far end of the episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the graphs you can see what happens: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first graph we see the setpoint in yellow and the controlled continuous parameter in purple. In the beginning, the algorithm brings the controlled parameter close to the setpoint fast, but then it ceases its further efforts and does not try to eliminate the remaining steady state error. This control deviation even increases over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dr3zC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dr3zC.png&quot; alt=&quot;steady state error not disappearing&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second graph, the actual reward is depicted in yellow. (Just ignore the other colors.) I use the normalized control deviation to calculate the reward: &lt;span class=&quot;math-container&quot;&gt;$r = \frac{\frac{|dev|}{k}}{1+\frac{|dev|}{k}}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives me a reward that lies within the interval &lt;span class=&quot;math-container&quot;&gt;$]0, 1]$&lt;/span&gt; and has a value of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; when the deviation &lt;span class=&quot;math-container&quot;&gt;$dev$&lt;/span&gt; equals the parameter &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. (That is the parameter &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; kind of indicates when &lt;em&gt;half of the work is done&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This reward function is relatively steep for the last fraction of the deviation from &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. So it would definitely be worth the effort for the agent to eliminate the residual deviation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it looks like the agent is happy with the existing state and the control deviation never gets eliminated. Eventhough the reward is stuck at ~0.85 instead of the maximum achievable 1.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bJPFA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bJPFA.png&quot; alt=&quot;Yellow is the actual reward in each step&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas how to push the agent into some more effort to eliminate the steady state error? &#xA;(A PID controller would exactly do this by using its I-term. How can I translate this to the RL-algo?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The state presented to the algo consists of the current deviation and the speed of change (derivatve) of the controlled value. The deviation is not included in the calculation of the reward function, but in the end we wat a flat line with no steady state deviation of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas welcome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&#xA;Felix&lt;/p&gt;&#xA;" OwnerUserId="25972" LastActivityDate="2020-06-03T09:01:49.367" Title="Continuous control with DDPG: How to eliminate steady state error?" Tags="&lt;deep-rl&gt;&lt;control-problem&gt;&lt;ddpg&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18568" PostTypeId="2" ParentId="18513" CreationDate="2020-03-11T11:39:39.153" Score="0" Body="&lt;p&gt;The original graph for the aforementioned Bayesian Optimisation is similar to the graph in these &lt;a href=&quot;http://www.ke.tu-darmstadt.de/events/ICML-04/tutorial-notes/m2-ghahramani.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; (slide 18) along with the calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, according to the tutorial the graph shown should actually have the term &lt;span class=&quot;math-container&quot;&gt;$p(D|m)$&lt;/span&gt; on the y-axis, thus making it a generative model.Now the graph starts to make sense, since a model with low complexity cannot produce very complex datasets and will be centred around 0, while very complex models can produce richer datasets which makes them assign probability thinly over all the datatsets (to keep &lt;span class=&quot;math-container&quot;&gt;$\sum_{D'}p(D'|m) = 1$&lt;/span&gt;).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/STKb4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/STKb4.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-03-11T11:39:39.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18569" PostTypeId="1" CreationDate="2020-03-11T13:11:51.567" Score="2" ViewCount="85" Body="&lt;p&gt;I've been searching online - and so far, I've been unable to find any publicly-accessible &lt;em&gt;pre-trained models&lt;/em&gt; that can be used for &lt;strong&gt;LDA Topic Modeling - Transfer Learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone share any resources in this thread if available?&lt;/p&gt;&#xA;" OwnerUserId="33400" LastEditorUserId="38892" LastEditDate="2020-07-28T20:28:13.130" LastActivityDate="2020-07-28T20:28:13.130" Title="Pre-trained Models for Topic Modelling Transfer Learning (LDA)" Tags="&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;open-ai&gt;&lt;transfer-learning&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18570" PostTypeId="2" ParentId="18565" CreationDate="2020-03-11T13:19:45.567" Score="2" Body="&lt;p&gt;To address your description point by point,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There are many types of CNN architectures. You appear to be describing a &lt;strong&gt;fully-convolutional neural network&lt;/strong&gt; built only from convolutional layers which is a very specialized type of network (typically used for segmentation tasks). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Most networks will include other layer types which have trainable parameters too, i.e. batch normalization layers or densely connected layers. &lt;/li&gt;&#xA;&lt;li&gt;Yes, the training process operates on the same concept of error backpropagation.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the model is performing a segmentation task (or any task which requires a 1-to-1 mapping of individually classified pixels), then yes its typically the same size.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The raw output of a fully-CNN is typically a &lt;strong&gt;probability map&lt;/strong&gt; where each pixel is the probability between 0 and 1 of it being a member of the positive class. This is normally thresholded at 0.5 to obtain a binary mask which delimits the area of interest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In essence, yes. But it may be more accurate to say that the goal is to minimise the loss function (reduce the error) which is done by learning optimal filter weights.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Overfitting is not about pixel location. In fact, CNNs are translationally invariant which means that the location of the feature has no impact on the output. There are some types of CNNs which take feature location into account but thats still not standard. Overfitting generally happens when a filter is an exact fit to a pixel patch, for example, the filter is no longer just looking for vertical lines but the exact pixel values of a section of vertical line in a specific image.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No, each convolution of a 3x3 filter takes 3x3 pixels and it is convolved over the whole image which means that the filter output will likely be around the same size of the image depending on stride length, padding and dilation. Have a look at one of my other &lt;a href=&quot;https://ai.stackexchange.com/a/17066/31980&quot;&gt;answers&lt;/a&gt; for more details.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31980" LastEditorUserId="31980" LastEditDate="2020-03-18T14:38:23.413" LastActivityDate="2020-03-18T14:38:23.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18571" PostTypeId="1" CreationDate="2020-03-11T14:02:00.837" Score="2" ViewCount="87" Body="&lt;p&gt;I playing around &lt;a href=&quot;https://www.tensorflow.org/tutorials/structured_data/time_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence modeling to forecast the weather&lt;/a&gt; using LSTM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does the number of layers or units in each layer exactly affect the model complexity (in an LSTM)? For example, if I increase the number of layers and decrease the number of units, how will the model complexity be affected? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not interested in rules of thumb for choosing the number of layers or units. I am interested in theoretical guarantees or bounds.&lt;/p&gt;&#xA;" OwnerUserId="32253" LastEditorUserId="2444" LastEditDate="2020-11-13T23:44:24.973" LastActivityDate="2020-11-13T23:44:24.973" Title="How does the number of stacked LSTM layers or units in each layer affect the model complexity?" Tags="&lt;long-short-term-memory&gt;&lt;computational-learning-theory&gt;&lt;sequence-modelling&gt;&lt;vc-dimension&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18572" PostTypeId="2" ParentId="18565" CreationDate="2020-03-11T14:05:22.643" Score="2" Body="&lt;p&gt;The 6th point is wrong. Filters do linear combination with group of pixels (depends on filter size) and move over the image and continue to do linear combination operation until it finishes the whole image. &#xA;Please look at &lt;a href=&quot;http://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2D convolution in CNN&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would explain CNN following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CNN is a kind of neural network that generally contains many convolutional layers and one or two fully connected layers. The purpose of the convolutional layers is feature extraction. Each of the convolutional layers contains many kernels. The purpose of the kernel is to extract different types of features. E. g. Edge, color, shape, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, fully connected layers at the end of the network decide the output based on the features that were extracted by convolutional layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of CNN is that we can learn feature extraction kernels based training images. &#xA;If you look earlier days of image processing. You can&#xA;see the image processing community used convolution for feature extraction all the time. They were just using selective kernel for feature extraction. E. g. Sobel operator for edge detection.&lt;/p&gt;&#xA;" OwnerUserId="34164" LastEditorUserId="9947" LastEditDate="2020-03-11T17:41:22.603" LastActivityDate="2020-03-11T17:41:22.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18573" PostTypeId="1" AcceptedAnswerId="18574" CreationDate="2020-03-11T16:11:52.107" Score="3" ViewCount="72" Body="&lt;p&gt;I'm actually trying to understand the policy iteration in the context of RL. I read an article presenting it and, at some point, a pseudo-code of the algorithm is given : &lt;a href=&quot;https://i.stack.imgur.com/YcKxP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YcKxP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I can't understand is this line : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IRJAR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IRJAR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, policy iteration is a model-free algorithm, which means that it doesn't need to know the environment's dynamics. But, in this line, we need &lt;span class=&quot;math-container&quot;&gt;$p(s',r \mid s, \pi(s))$&lt;/span&gt; (which in my understanding is the transition function of the MDP that gave us the probability of landing in the state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; knowing previous &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; state and the action taken) to compute &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;. So I don't understand how we can compute &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; with the quantity &lt;span class=&quot;math-container&quot;&gt;$p(s',r \mid s, \pi(s))$&lt;/span&gt; since it is a parameter of the environment. &lt;/p&gt;&#xA;" OwnerUserId="34177" LastEditorUserId="2444" LastEditDate="2020-03-11T16:26:34.850" LastActivityDate="2020-03-11T16:26:34.850" Title="How can the policy iteration algorithm be model-free if it uses the transition probabilities?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;&lt;policy-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18574" PostTypeId="2" ParentId="18573" CreationDate="2020-03-11T16:25:37.447" Score="1" Body="&lt;p&gt;Everything you say in your post is correct, apart from the wrong assumption that policy iteration is model-free. PI is a model-based algorithm because of the reasons you're mentioning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/a/8820/2444&quot;&gt;my answer&lt;/a&gt; to the question &lt;a href=&quot;https://ai.stackexchange.com/q/4456/2444&quot;&gt;What&amp;#39;s the difference between model-free and model-based reinforcement learning?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-11T16:25:37.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18575" PostTypeId="2" ParentId="18526" CreationDate="2020-03-11T16:48:00.123" Score="2" Body="&lt;p&gt;You can try image captioning. You can train a CNN model for image, and then, on top of that, provide the model embedding to another LSTM model to learn the encoded characteristics. You can directly use the pre-trained VGG-16 model and use the second last layer to create your image embeddings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1411.4555&quot; rel=&quot;nofollow noreferrer&quot;&gt;Show and Tell: A Neural Image Caption Generator&lt;/a&gt; is a really nice paper to start with. There is an implementation of it in TensorFlow: &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/image_captioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/text/image_captioning&lt;/a&gt;. The paper focuses on generating caption, but you can provide your 'characteristics' to LSTM, so that it can learn it for each image. &lt;/p&gt;&#xA;" OwnerUserId="33835" LastEditorUserId="2444" LastEditDate="2020-03-11T20:52:30.080" LastActivityDate="2020-03-11T20:52:30.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18576" PostTypeId="1" AcceptedAnswerId="18577" CreationDate="2020-03-11T17:27:32.623" Score="54" ViewCount="9660" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt; It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of &lt;a href=&quot;https://ai.stackexchange.com/questions/13317/where-can-i-find-the-proof-of-the-universal-approximation-theorem/25917#25917&quot;&gt;their universal approximation capabilities&lt;/a&gt;.  However, in this post I'm curious about the &lt;em&gt;opposite&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Namely, what are some &lt;em&gt;well-known&lt;/em&gt; cases, problems or real-world applications where neural networks don't do very well?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Specification:&lt;/strong&gt;&#xA;I'm looking for specific &lt;em&gt;regression&lt;/em&gt; tasks (with accessible data-sets) where neural networks are not the state-of-the-art.  The regression task should be &amp;quot;naturally suitable&amp;quot;, so no sequential or time-dependent data (in which case an RNN or reservoir computer would be more natural).&lt;/p&gt;&#xA;" OwnerUserId="31649" LastEditorUserId="2444" LastEditDate="2021-01-22T14:38:27.343" LastActivityDate="2021-01-22T14:38:27.343" Title="What are some well-known problems where neural networks don't do very well?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reference-request&gt;&lt;applications&gt;" AnswerCount="11" CommentCount="1" FavoriteCount="22" ContentLicense="CC BY-SA 4.0" />
  <row Id="18577" PostTypeId="2" ParentId="18576" CreationDate="2020-03-11T17:41:19.353" Score="28" Body="&lt;p&gt;Here's a snippet from &lt;a href=&quot;https://medium.com/@GaryMarcus/the-deepest-problem-with-deep-learning-91c5991f5695?_branch_match_id=687598797934356895&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article by Gary Marcus&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In particular, they showed that standard deep learning nets often fall&#xA;  apart when confronted with common stimuli rotated in three dimensional&#xA;  space into unusual positions, like the top right corner of this&#xA;  figure, in which a schoolbus is mistaken for a snowplow:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C4Bch.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C4Bch.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;.&#xA;  .&#xA;  .&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Mistaking an overturned schoolbus is&#xA;  not just a mistake, it’s a revealing mistake: it that shows not only&#xA;  that deep learning systems can get confused, but they are challenged&#xA;  in making a fundamental distinction known to all philosophers: the&#xA;  distinction between features that are merely contingent associations&#xA;  (snow is often present when there are snowplows, but not necessary)&#xA;  and features that are inherent properties of the category itself&#xA;  (snowplows ought other things being equal have plows, unless eg they&#xA;  have been dismantled). We’d already seen similar examples with&#xA;  contrived stimuli, like Anish Athalye’s carefully designed, 3-d&#xA;  printed foam covered dimensional baseball that was mistaken for an&#xA;  espresso&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tdE0h.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tdE0h.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Alcorn’s results — some from real photos from the natural&#xA;  world — should have pushed worry about this sort of anomaly to the top&#xA;  of the stack.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Please note that the opinions of the author are his alone and I do not necessarily share all of them with him.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Some more fun stuff&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) DeepMind's neural network that could play &lt;em&gt;Breakout&lt;/em&gt; and &lt;em&gt;Starcraft&lt;/em&gt;&#xA;saw a dramatic dip in performance when the paddle was moved up by a few pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.vicarious.com/posts/general-game-playing-with-schema-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;See: General Game Playing With Schema Networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While in the latter, it performed well with one race of the character but not on a different map and with different characters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stream.nyu.edu/channel/CS%2BFaculty%2BSearch%2B2019/111128891&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AlphaZero searches just 80,000 positions per second in chess and&#xA;  40,000 in shogi, compared to 70 million for Stockfish and 35 million&#xA;  for elmo.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What the team at Deepmind did was to build a very good search algorithm. A search algorithm that includes the capability to remember facets of previous searches to apply better results to new searches. This is very clever; it undoubtedly has immense value in many areas, but it cannot be considered general intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/intuitionmachine/alphazero-how-intuition-demolished-logic-66a4841e6810&quot; rel=&quot;nofollow noreferrer&quot;&gt;See: AlphaZero: How Intuition Demolished Logic (Medium)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34183" LastEditorUserId="1671" LastEditDate="2020-05-03T23:47:00.690" LastActivityDate="2020-05-03T23:47:00.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18578" PostTypeId="2" ParentId="18576" CreationDate="2020-03-11T18:20:08.750" Score="19" Body="&lt;p&gt;In theory, most neural networks can approximate any continuous function on compact subsets of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;, provided that the activation functions satisfy certain mild conditions. This is known as the &lt;em&gt;universal approximation theorem&lt;/em&gt; (UAT), but that should &lt;em&gt;not&lt;/em&gt; be called  &lt;em&gt;universal&lt;/em&gt;, given that there are a lot more discontinuous functions than continuous ones, although certain discontinuous functions can be approximated by continuous ones. The UAT shows the theoretical powerfulness of neural networks and their purpose. They represent and approximate functions. If you want to know more about the details of the UAT, for different neural network architectures, see &lt;a href=&quot;https://ai.stackexchange.com/a/13319/2444&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in practice, neural networks trained with gradient descent and backpropagation face several issues and challenges, some of which are due to the training procedure and not just the architecture of the neural network or available data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it is well known that &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;neural networks are prone to catastrophic forgetting (or interference)&lt;/a&gt;, which means that they aren't particularly suited for &lt;strong&gt;incremental learning&lt;/strong&gt; tasks, although some more sophisticated incremental learning algorithms based on neural networks have already been developed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks can also be &lt;a href=&quot;https://arxiv.org/abs/1712.07107&quot; rel=&quot;noreferrer&quot;&gt;sensitive to their inputs&lt;/a&gt;, i.e. a small change in the inputs can drastically change the output (or answer) of the neural network. This is partially due to the fact that they learn a function that isn't really the function you expect them to learn. So, a system based on such a neural network can potentially be hacked or fooled, so they are probably not well suited for &lt;strong&gt;safety-critical applications&lt;/strong&gt;. This issue is related to the &lt;a href=&quot;https://arxiv.org/pdf/2001.02522.pdf&quot; rel=&quot;noreferrer&quot;&gt;low interpretability and explainability of neural networks&lt;/a&gt;, i.e. they are often denoted as &lt;a href=&quot;https://stats.stackexchange.com/q/93705/82135&quot;&gt;black-box models&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1505.05424&quot; rel=&quot;noreferrer&quot;&gt;Bayesian neural networks (BNNs)&lt;/a&gt; can potentially mitigate these problems, but they are unlikely to be the ultimate or complete solution. Bayesian neural networks maintain a distribution for each of the units (or neurons), rather than a point estimate. In principle, this can provide more uncertainty guarantees, but, in practice, this is not yet the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, neural networks often require a lot of data in order to approximate the desired function accurately, so in cases where &lt;strong&gt;data is scarce&lt;/strong&gt; neural networks may not be appropriate. Moreover, the training of neural networks (especially, deep architectures) also &lt;strong&gt;requires a lot of computational resources&lt;/strong&gt;. Inference can also be sometimes problematic, when you need real-time predictions, as it can also be expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, neural networks are just function approximators, i.e. they approximate a &lt;em&gt;specific&lt;/em&gt; function (or set of functions, in the case of Bayesian neural networks), given a specific configuration of the parameters. They can't do more than that. They cannot magically do something that they have not been trained to do, and it is usually the case that you don't really know the specific function the neural network is representing (hence the expression &lt;em&gt;black-box model&lt;/em&gt;), apart from knowing your training dataset, which can also contain spurious information, among other issues.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-12T23:31:27.883" LastActivityDate="2020-03-12T23:31:27.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18579" PostTypeId="1" CreationDate="2020-03-11T19:01:38.197" Score="2" ViewCount="80" Body="&lt;p&gt;Soar is a cognitive architecture.    &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Soar_(cognitive_architecture)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Soar_(cognitive_architecture)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://soar.eecs.umich.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://soar.eecs.umich.edu/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is something called &quot;the Chinese box&quot; or &quot;Chinese room&quot; argument:    &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinese_room&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Chinese_room&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plato.stanford.edu/entries/chinese-room/&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &quot;Chinese room&quot; seems to be begging its question, but that is not what I am asking.  I am asking if there is any literal difference between a tool like &quot;SOAR&quot; and the formalism of the &quot;Chinese box&quot;.  Is SOAR identical or equivalent to a &quot;Chinese Box&quot;?  &lt;/p&gt;&#xA;" OwnerUserId="2263" LastEditorUserId="2444" LastEditDate="2021-01-24T17:37:58.717" LastActivityDate="2021-01-24T17:37:58.717" Title="Is the Cognitive Approach (SOAR) equivalent to the Chinese Room argument?" Tags="&lt;philosophy&gt;&lt;cognitive-science&gt;&lt;chinese-room-argument&gt;&lt;cognitive-architecture&gt;&lt;soar&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18580" PostTypeId="2" ParentId="10289" CreationDate="2020-03-12T01:26:33.493" Score="4" Body="&lt;p&gt;According to Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Answer to your question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To build any neural network model we assume the train, test and validation data are coming from a probability distribution. So, if you produce a neural network model based on statistical data then the network is a statistical model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, neural networks' cost function is generally a parametric model and parametric modes are statistical models.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please look at Goodfellow's Deep Learning book chapter &lt;a href=&quot;https://www.deeplearningbook.org/contents/mlp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Feedforward Networks&lt;/a&gt; page 174 and 175.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Goodfellow's book &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fortunately, the cost functions for neural networks are more or less the same as those for other parametric models, such as linear models. In most cases, our &lt;a href=&quot;https://en.wikipedia.org/wiki/Parametric_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;parametric model&lt;/a&gt; deﬁnes a distribution &lt;span class=&quot;math-container&quot;&gt;$p(y \mid x; \theta)$&lt;/span&gt; and we simply use the principle of maximum likelihood. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In conclusion, ANNs (e. g. MLP, CNN, etc.) are statistical models&lt;/p&gt;&#xA;" OwnerUserId="34164" LastEditorUserId="2444" LastEditDate="2020-03-12T03:05:38.527" LastActivityDate="2020-03-12T03:05:38.527" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18581" PostTypeId="2" ParentId="18576" CreationDate="2020-03-12T07:59:50.753" Score="5" Body="&lt;p&gt;This is more in the direction of 'what kind of problems can be solved by neural networks'. In order to train a neural network you need a large set of training data which is labelled with correct/ incorrect for the question you are interested in. So for example 'identify all pictures that have a cat on them' is very suitable for neural networks. On the other hand 'summarize the story of this toddler picture book' is very hard. Although a human can easily decide whether a given summary is any good or not it would be very difficult to build a suitable set of training data for this kind of problem. So if you can't build a large training data set with correct answers, you can't train a neural network to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer of Anshuman Kumar is also an instance of that, also a potentially solvable one. The neural network that misidentified upside-down school buses presumably had very few if any upside-down school buses in its training data. Put them into the training data and the neural network will identify these as well. This is still a flaw in neural networks, a human can correctly identify an upside-down school bus the first time they see one if they know what school busses look like.&lt;/p&gt;&#xA;" OwnerUserId="34200" LastActivityDate="2020-03-12T07:59:50.753" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18584" PostTypeId="1" AcceptedAnswerId="18586" CreationDate="2020-03-12T09:51:23.383" Score="7" ViewCount="85" Body="&lt;p&gt;Are there any medical diagnosis systems that are already used somewhere that are based on artificial neural networks?&lt;/p&gt;&#xA;" OwnerUserId="34203" LastEditorUserId="2444" LastEditDate="2020-03-12T13:52:38.983" LastActivityDate="2020-03-12T13:55:55.057" Title="Medical diagnosis systems based on artificial neural networks" Tags="&lt;neural-networks&gt;&lt;applications&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18585" PostTypeId="1" CreationDate="2020-03-12T10:19:55.543" Score="4" ViewCount="94" Body="&lt;p&gt;I was reading the following research paper &lt;a href=&quot;https://arxiv.org/pdf/1707.01495.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience Replay&lt;/a&gt;. This is the paper that introduces a concept called Hindsight Experience Replay (HER), which basically attempts to alleviate the infamous sparse reward problem. It is based on the intuition that human beings constantly try and learn something useful even from their past failed experiences.&lt;/p&gt;&#xA;&lt;p&gt;I have almost completely understood the concept. But in the algorithm posited in the paper, I don't really understand how the optimization works. Once the fictitious trajectories are added, we have a state-goal-action dependency. This means our DQN should predict Q-Values based on an input state and the goal we're pursuing (The paper mentions how HER is extremely useful for multi-RL as well).&lt;/p&gt;&#xA;&lt;p&gt;Does this mean I need to add another input feature (goal) to my DQN? An input state and an input goal, as two input features to my DQN, which is basically a CNN?&lt;/p&gt;&#xA;&lt;p&gt;Because in the optimization step they have mentioned that we need to randomly sample trajectories from the replay buffer and use those for computing the gradients. It wouldn't make sense to compute the Q-Values without the goal now, because then we'd wind up with duplicate values.&lt;/p&gt;&#xA;&lt;p&gt;Could someone help me understand how exactly does the optimization take place here?&lt;/p&gt;&#xA;&lt;p&gt;I am training Atari's &amp;quot;Montezuma's Revenge&amp;quot; using a double DQN with Hindsight Experience Replay (HER).&lt;/p&gt;&#xA;" OwnerUserId="32455" LastEditorUserId="2444" LastEditDate="2020-11-20T18:37:05.090" LastActivityDate="2021-04-19T19:54:15.680" Title="How does the optimization process in hindsight experience replay exactly work?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;sparse-rewards&gt;&lt;hindsight-experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18586" PostTypeId="2" ParentId="18584" CreationDate="2020-03-12T10:28:35.237" Score="5" Body="&lt;p&gt;Yes, there are many, actually. A Google search turned this paper &lt;a href=&quot;https://www.researchgate.net/publication/285912467_Artificial_Neural_Networks_in_Medical_Diagnosis&quot; rel=&quot;noreferrer&quot;&gt;Artificial Neural Networks in Medical Diagnosis&lt;/a&gt; (2011) by Al-Shayea up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not only are they used in disease diagnosis, but even with things like prescribing medicines. In fact, the top project for a hackathon at my school analysed thousands of research articles, and took a patient's medication history as input, to best recommend them specific medicines. Check it &lt;a href=&quot;https://devpost.com/software/neurolens&quot; rel=&quot;noreferrer&quot;&gt;out&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22373" LastEditorUserId="2444" LastEditDate="2020-03-12T13:55:55.057" LastActivityDate="2020-03-12T13:55:55.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18587" PostTypeId="1" AcceptedAnswerId="18597" CreationDate="2020-03-12T11:32:34.427" Score="14" ViewCount="3061" Body="&lt;p&gt;It's an idea I heard a while back but couldn't remember the name of. It involves the existence and development of an AI that will eventually rule the world and that if you don't fund or progress the AI then it will see you as &quot;hostile&quot; and kill you. Also, by knowing about this concept, it essentially makes you a candidate for such consideration, as people who didn't know about it won't understand to progress such an AI. From my understanding, this idea isn't taken that seriously, but I'm curious to know the name nonetheless.&lt;/p&gt;&#xA;" OwnerUserId="34205" LastEditorUserId="2444" LastEditDate="2020-03-12T13:47:24.393" LastActivityDate="2020-03-13T13:15:01.447" Title="What is the idea called involving an AI that will eventually rule humanity?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;definitions&gt;&lt;superintelligence&gt;&lt;singularity&gt;" AnswerCount="5" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18588" PostTypeId="2" ParentId="18587" CreationDate="2020-03-12T11:39:30.087" Score="10" Body="&lt;p&gt;I believe the term you are looking for is &quot;(technological) singularity&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Technological_singularity&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5300" LastActivityDate="2020-03-12T11:39:30.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18589" PostTypeId="1" CreationDate="2020-03-12T11:43:19.983" Score="0" ViewCount="66" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RunSp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RunSp.png&quot; alt=&quot;Graph of Loss on Training Set vs Test Set&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BHLJu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BHLJu.png&quot; alt=&quot;Graph of Accuracy&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Model used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mobilenet_model = MobileNet(input_shape=in_dim, include_top=False, pooling='avg', weights='imagenet')&#xA;mob_x = Dropout(0.75)(mobilenet_model.output)&#xA;mob_x = Dense(2, activation='sigmoid')(mob_x)&#xA;&#xA;model = Model(mobilenet_model.input, mob_x)&#xA;&#xA;for layer in model.layers[:50]:&#xA;    layer.trainable=False&#xA;&#xA;for layer in model.layers[50:]:&#xA;    layer.trainable=True&#xA;&#xA;model.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The rest of the code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;in_dim = (224,224,3)&#xA;batch_size = 64&#xA;samples_per_epoch = 1000&#xA;validation_steps = 300&#xA;nb_filters1 = 32&#xA;nb_filters2 = 64&#xA;conv1_size = 3&#xA;conv2_size = 2&#xA;pool_size = 2&#xA;epochs = 20&#xA;classes_num = 2&#xA;lr = 0.000004&#xA;train_datagen = ImageDataGenerator(&#xA;    rescale=1. / 255,&#xA;    shear_range=0.2,&#xA;    zoom_range=0.2,&#xA;    horizontal_flip=True)&#xA;test_datagen = ImageDataGenerator(rescale=1./255)&#xA;train_generator = train_datagen.flow_from_directory(&#xA;        'output/train',  # this is the target directory&#xA;        target_size= in_dim[0:2],  # all images will be resized to 224*224&#xA;        batch_size=batch_size,&#xA;        class_mode='categorical') &#xA;#Found 6062 images belonging to 2 classes.&#xA;validation_generator = test_datagen.flow_from_directory(&#xA;        'output/val',&#xA;        target_size=in_dim[0:2],&#xA;        batch_size=batch_size,&#xA;        class_mode='categorical')&#xA;#Found 769 images belonging to 2 classes.&#xA;from keras.callbacks import EarlyStopping&#xA;#set early stopping monitor so the model stops training when it won't improve anymore&#xA;early_stopping_monitor = EarlyStopping(patience=3)&#xA;steps_per_epoch = 10&#xA;from keras import backend as K&#xA;&#xA;def recall_m(y_true, y_pred):&#xA;        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))&#xA;        possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))&#xA;        recall = true_positives / (possible_positives + K.epsilon())&#xA;        return recall&#xA;&#xA;def precision_m(y_true, y_pred):&#xA;        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))&#xA;        predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))&#xA;        precision = true_positives / (predicted_positives + K.epsilon())&#xA;        return precision&#xA;&#xA;def f1_m(y_true, y_pred):&#xA;    precision = precision_m(y_true, y_pred)&#xA;    recall = recall_m(y_true, y_pred)&#xA;    return 2*((precision*recall)/(precision+recall+K.epsilon()))&#xA;model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc',f1_m,precision_m, recall_m])&#xA;&#xA;&#xA;history = model.fit_generator(&#xA;        train_generator,&#xA;        steps_per_epoch=2000// batch_size ,&#xA;        epochs=50,&#xA;        validation_data=validation_generator,&#xA;        validation_steps=800// batch_size,&#xA;        callbacks = [early_stopping_monitor],&#xA;       )&#xA;test_generator = train_datagen.flow_from_directory(&#xA;        'output/test',&#xA;        target_size=in_dim[0:2],&#xA;        batch_size=batch_size,&#xA;        class_mode='categorical')&#xA;loss, accuracy, f1_score, precision, recall = model.evaluate(test_generator)&#xA;print(&amp;quot;The test set accuracy is &amp;quot;, accuracy)&#xA;#The test set accuracy is  0.9001349538122272&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From what I have gathered from this &lt;a href=&quot;https://forums.fast.ai/t/very-volatile-validation-loss/7573/11&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; and this &lt;a href=&quot;https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;, I understand that the validation set is much smaller with respect to the training set. I have applied augmentation to the test set due to &lt;a href=&quot;https://stats.stackexchange.com/a/320967/168835&quot;&gt;this&lt;/a&gt; and that boosted test set accuracy by 1%.&lt;/p&gt;&#xA;&lt;p&gt;Please note that the test train split is &amp;quot;stratified&amp;quot; as here is a breakdown of each individual class&#xA;in test/train/validation folders&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Test: Class 0: 7426&#xA;      Class 1: 631&#xA;Train: Class 0: 928&#xA;       Class 1: 80&#xA;Val: Class 0: 928&#xA;     Class 1: 79&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have used an 80/10/10 split for train/test/val respectively.&lt;/p&gt;&#xA;&lt;p&gt;Can someone guide me on what to do so that I can ensure the accuracy is  95%+  and the validation loss graph is less erratic?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I am thinking of tuning the learning rate though it doesn't seem to be working by much.&lt;/li&gt;&#xA;&lt;li&gt;Another suggestion is to use test time augmentation.&lt;/li&gt;&#xA;&lt;li&gt;Also, the link on fast.ai has a comment like so&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;That is also part of the reasons why a weighted ensemble of different&#xA;performing epoch models will usually perform better than the best&#xA;performing model on your validation dataset. Sometimes choosing the&#xA;best model or the best ensemble to generalize well isn’t as easy as&#xA;selecting the lower loss/higher accuracy model.&#xA;4. Should I use L2 regularization in addition to the current dropout?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Applying augmentation of any kind to the validation set is a strict no-no and the dataset is generated by my company which I cannot get more of.&lt;/p&gt;&#xA;" OwnerUserId="34183" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-13T06:23:44.840" Title="How do I deal with an erratic validation set loss when the loss on my training set is monotonically decreasing?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;keras&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18590" PostTypeId="1" AcceptedAnswerId="20192" CreationDate="2020-03-12T12:30:55.960" Score="1" ViewCount="771" Body="&lt;p&gt;I was running my gated recurrent unit (GRU) model. I wanted to get an opinion if my loss and validation loss graph is good or not, since I'm new to this and don't really know if that is considered underfitting or not&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9C4pr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9C4pr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2tCzo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2tCzo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34168" LastEditorUserId="34168" LastEditDate="2020-03-12T19:37:17.783" LastActivityDate="2020-04-12T11:41:02.763" Title="Is my GRU model under-fitting given this plot of the training and validation loss?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;keras&gt;&lt;overfitting&gt;&lt;underfitting&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18591" PostTypeId="2" ParentId="18576" CreationDate="2020-03-12T12:58:36.043" Score="3" Body="&lt;p&gt;A checkerboard with missing squares is impossible for a neural network to learn the missing color. The more it learns on training data, the worse it does on test data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. this article &lt;a href=&quot;https://journals.blythinstitute.org/ojs/index.php/cbi/article/view/48&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Unlearnable Checkerboard Pattern&lt;/a&gt; (which, unfortunately, is not freely accessible). In any case, it should be easy to try out yourself that this task is difficult.&lt;/p&gt;&#xA;" OwnerUserId="8221" LastEditorUserId="2444" LastEditDate="2020-03-16T17:26:52.967" LastActivityDate="2020-03-16T17:26:52.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18592" PostTypeId="2" ParentId="18576" CreationDate="2020-03-12T13:45:32.253" Score="4" Body="&lt;p&gt;I don't know if it might be of use, but many areas of NLP are still hard to tackle, and even if deep models achieve the state of the art results, they usually beat baseline shallow models by very few percentage points. &#xA;One example that I've had the opportunity to work on is stance classification &lt;a href=&quot;https://www.aclweb.org/anthology/S16-1075.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;. In many datasets, the best F score achievable is around 70%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though it's hard to compare results since in NLP many datasets are really small and domain-specific (especially for stance detection and similar SemEval tasks), many times SVM, conditional random fields, sometimes even Naive Bayes models are able to perform almost as good as CNN or RNN.  Other tasks for which this holds are argumentation mining or claim detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. the paper &lt;a href=&quot;https://www.aclweb.org/anthology/S16-1075.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TakeLab at SemEval-2016 Task 6: Stance Classification in Tweets Using a Genetic Algorithm Based Ensemble&lt;/a&gt; (2016) by Martin Tutek et al.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="2444" LastEditDate="2020-03-14T01:37:09.160" LastActivityDate="2020-03-14T01:37:09.160" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18593" PostTypeId="2" ParentId="18587" CreationDate="2020-03-12T14:27:15.830" Score="6" Body="&lt;p&gt;The likely expression you are looking for is &lt;em&gt;AI takeover&lt;/em&gt;, which is a common topic in &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence_in_fiction&quot; rel=&quot;noreferrer&quot;&gt;science fiction&lt;/a&gt; movies, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)&quot; rel=&quot;noreferrer&quot;&gt;2001: A Space Odyssey&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Matrix&quot; rel=&quot;noreferrer&quot;&gt;The Matrix&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_takeovers_in_popular_culture&quot; rel=&quot;noreferrer&quot;&gt;popular culture&lt;/a&gt;. Although the AI takeover is an &lt;strong&gt;unlikely scenario&lt;/strong&gt; in the next years, &lt;a href=&quot;https://en.wikipedia.org/wiki/Open_Letter_on_Artificial_Intelligence&quot; rel=&quot;noreferrer&quot;&gt;certain scientists, such as Stephen Hawking, have expressed concerns about it&lt;/a&gt; and some philosophers, especially Nick Bostrom, are really interested in the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI takeover concept is related to concepts such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;noreferrer&quot;&gt;AI singularity&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;noreferrer&quot;&gt;superintelligence&lt;/a&gt;, &lt;a href=&quot;https://wiki.lesswrong.com/wiki/Intelligence_explosion&quot; rel=&quot;noreferrer&quot;&gt;intelligence explosion&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_control_problem&quot; rel=&quot;noreferrer&quot;&gt;AI control problem&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence&quot; rel=&quot;noreferrer&quot;&gt;existential risk&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_ethics&quot; rel=&quot;noreferrer&quot;&gt;machine ethics&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Friendly_artificial_intelligence&quot; rel=&quot;noreferrer&quot;&gt;friendly AI&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies&quot; rel=&quot;noreferrer&quot;&gt;Superintelligence: Paths, Dangers, Strategies&lt;/a&gt; (2014) by N. Bostrom may be helpful if you are interested in hypothetical scenarios.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-12T14:32:55.437" LastActivityDate="2020-03-12T14:32:55.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18594" PostTypeId="1" CreationDate="2020-03-12T14:52:05.740" Score="6" ViewCount="160" Body="&lt;p&gt;I want to solve the zero subset sum problem with the hill-climbing algorithm, but I am not sure I found a good state space for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the problem: consider we have a set of numbers and we want to find a subset of this set such that the sum of the elements in this subset is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own idea to solve this by hill-climbing is that in the first step, we can choose a random subset of the set (for example, the main set is &lt;span class=&quot;math-container&quot;&gt;$X= \{X_1,\dots,X_n\}$&lt;/span&gt; and we chose &lt;span class=&quot;math-container&quot;&gt;$X'=\{X_{i_1},\dots,X_{i_k}\}$&lt;/span&gt; randomly), then the children of this state can be built by adding an element from &lt;span class=&quot;math-container&quot;&gt;$X-X'$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$X'$&lt;/span&gt; or deleting an element from &lt;span class=&quot;math-container&quot;&gt;$X'$&lt;/span&gt; itself. This means that each state has &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; children. and the objective function could be the sum of the elements in &lt;span class=&quot;math-container&quot;&gt;$X'$&lt;/span&gt; that we want to minimize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a good modeling? Are there better modelings or objective functions that can work more intelligently?&lt;/p&gt;&#xA;" OwnerUserId="33756" LastEditorUserId="2444" LastEditDate="2020-03-12T15:44:34.793" LastActivityDate="2021-05-19T10:03:45.250" Title="How can I solve the zero subset sum problem with hill climbing?" Tags="&lt;ai-design&gt;&lt;hill-climbing&gt;&lt;local-search&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18595" PostTypeId="1" CreationDate="2020-03-12T15:47:52.753" Score="2" ViewCount="102" Body="&lt;p&gt;Below are my Inputs Outputs and fitness function. The snake is learning at a slow rate, and seems to be stagnant, additionally when the snake collides with the food, it gets deleted from the genome, which doesn't make any sense because that's not specified in the collision. Any input would be greatly appreciated&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for x, s in enumerate(snakes):&#xA;        # inserting new snake head and deleting the tale for movement&#xA;&#xA;        # inputs&#xA;        s.x = s.snake_head[0]&#xA;        s.y = s.snake_head[1]&#xA;        snakeheadBottomDis = win_h - s.y&#xA;        snakeheadRightDis = win_w - s.x&#xA;        snake_length = len(s.snake_position)&#xA;        snakefoodDistEuclidean = math.sqrt((s.x - food.x) ** 2 + (s.y - food.y) ** 2)&#xA;        snakefoodDisManhattan = abs(s.x - food.x) + abs(s.y - food.y)&#xA;        xdis = s.Xdis()&#xA;        ydis = s.Ydis()&#xA;        s.dis_list1.append(snakefoodDistEuclidean)&#xA;        s.dis_list2.append(snakefoodDisManhattan)&#xA;        s.dis_list3.append(s.Xdis())&#xA;        s.dis_list4.append(s.Ydis())&#xA;        s.hunger_list.append(s.hunger)&#xA;        #print('Euclidean: ', dis_list1[-1])&#xA;        #print('Manhattan: ', dis_list2[-1])&#xA;        #print('X distance from Wall: ', dis_list3[-1])&#xA;        #print('Y distance from Wall: ', dis_list4[-1])&#xA;&#xA;        output = nets[snakes.index(s)].activate((s.hunger, s.x, s.y, food.x, food.y, snakeheadBottomDis,&#xA;                                                 snakeheadRightDis, snake_length, xdis,ydis,&#xA;                                                 snakefoodDisManhattan, snakefoodDistEuclidean,s.dis_list1[-1],s.dis_list1[-2],&#xA;                                                 s.dis_list2[-1],s.dis_list2[-2],s.dis_list3[-1],s.dis_list3[-2],&#xA;                                                 s.dis_list4[-1],s.dis_list4[-2],s.hunger_list[-1],s.hunger_list[-2]))&#xA;&#xA;&#xA;        #snake moving animation&#xA;        s.snake_position.insert(0, list(s.snake_head))&#xA;        s.snake_position.pop()&#xA;        s.hunger -= 1&#xA;&#xA;        # Checking distance Euclidean and Manhattan current and last&#xA;        if s.dis_list1[-1] &amp;gt; s.dis_list1[-2]:&#xA;            ge[x].fitness -= 1&#xA;&#xA;        if s.dis_list1[-1] &amp;lt; s.dis_list1[-2]:&#xA;            ge[x].fitness += 0.5&#xA;&#xA;        if s.dis_list1[-1] &amp;gt; s.dis_list2[-2]:&#xA;            ge[x].fitness -= 1&#xA;&#xA;        if s.dis_list1[-1] &amp;lt; s.dis_list2[-2]:&#xA;            ge[x].fitness += 0.5&#xA;&#xA;        #checking hunger number and if its decreasing&#xA;        if s.hunger_list[-1] &amp;lt; s.hunger_list[-2]:&#xA;            ge[x].fitness -= 0.1&#xA;&#xA;        # move right&#xA;        if output[0] &amp;gt;= 0 and output[1] &amp;lt; 0 and output[2] &amp;lt; 0 and output[&#xA;            3] &amp;lt; 0:&#xA;            #and s.x &amp;lt; win_w - s.width and s.y &amp;gt; 0 + s.height:&#xA;            # ge[x].fitness += 0.5&#xA;            s.move_right()&#xA;&#xA;        # move left&#xA;        if output[1] &amp;gt;= 0 and output[0] &amp;lt; 0 and output[2] &amp;lt; 0 and output[&#xA;            3] &amp;lt; 0:&#xA;            #and s.x &amp;lt; 500 - s.width and s.y &amp;gt; 0 + s.height:&#xA;            #ge[x].fitness += 0.5&#xA;            s.move_left()&#xA;&#xA;        # move down&#xA;        if output[2] &amp;gt;= 0 and output[1] &amp;lt; 0 and output[0] &amp;lt; 0 and output[&#xA;            3] &amp;lt; 0:&#xA;            #and s.x &amp;lt; 500 - s.width and s.y &amp;gt; 0 + s.height:&#xA;            # ge[x].fitness += 0.5&#xA;            s.move_down()&#xA;&#xA;        # move up&#xA;        if output[3] &amp;gt;= 0 and output[1] &amp;lt; 0 and output[2] &amp;lt; 0 and output[&#xA;            3] &amp;lt; 0:&#xA;            #and s.x &amp;lt; 500 - s.width and s.y &amp;gt; 0 + s.height:&#xA;            # ge[x].fitness += 0.5&#xA;            s.move_up()&#xA;&#xA;        #adding more fitness if axis aligns&#xA;        if s.snake_head[0] == food.x:&#xA;            ge[x].fitness += 0.1&#xA;        if s.snake_head[1] == food.x:&#xA;            ge[x].fitness += 0.1&#xA;&#xA;        # checking the activation function tanh&#xA;        # print ('output 0: ', output[0])&#xA;        # print('output 1: ', output[1])&#xA;        # print ('output 2: ', output[1])&#xA;        # print ('output 3: ', output[1])&#xA;&#xA;        # snake poping on other side of screen if screen limit reached&#xA;        if s.snake_head[0] &amp;gt;= win_w - s.width:&#xA;            s.snake_head[0] = 12&#xA;        if s.snake_head[0] &amp;lt;= 11 + s.width:&#xA;            s.snake_head[0] = win_w - s.width - 1&#xA;        if s.snake_head[1] &amp;gt;= win_h - s.height:&#xA;            s.snake_head[1] = s.height + 15&#xA;        if s.snake_head[1] &amp;lt;= 11 + s.height:&#xA;            s.snake_head[1] = win_h - s.height - 1&#xA;&#xA;        head = s.snake_position[0]&#xA;        #s.x &amp;lt; 0 + s.width or s.x &amp;gt; win_w - s.width or s.y &amp;lt; 0 + s.height or \&#xA;                #s.y &amp;gt; win_h - s.height or&#xA;&#xA;        #if run into self you die&#xA;        if head in s.snake_position[1:]:&#xA;            ge[x].fitness -= 10&#xA;            snakes.pop(x)&#xA;            nets.pop(x)&#xA;            ge.pop(x)&#xA;&#xA;        #if hunger reaches 0 you die&#xA;        if s.hunger == 0:&#xA;            ge[x].fitness -= 5&#xA;            snakes.pop(x)&#xA;            nets.pop(x)&#xA;            ge.pop(x)&#xA;&#xA;        #if snake collides with food award fitness&#xA;        if s.getRec().colliderect(food.getRec()):&#xA;            ge[x].fitness += 100&#xA;            s.hunger = 100&#xA;            score += 1&#xA;            s.snake_position.insert(0, list(s.snake_head))&#xA;            food.y = random.randint(0 + 24, 500 - 24)&#xA;            food.x = random.randint(0 + 24, 500 - 24)&#xA;&#xA;    # print(s.hunger)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="34212" LastActivityDate="2020-12-30T04:03:35.347" Title="I created a snake game and fitted the NEAT algorithm and there's issues" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;neat&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18597" PostTypeId="2" ParentId="18587" CreationDate="2020-03-12T18:09:04.770" Score="25" Body="&lt;p&gt;If I'm not mistaken you're looking for &lt;a href=&quot;https://en.wikipedia.org/wiki/LessWrong#Roko&amp;#39;s_basilisk&quot; rel=&quot;noreferrer&quot;&gt;Roko's Basilisk&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;in which an otherwise benevolent future AI system tortures simulations of those who did not work to bring the system into existence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="29873" LastActivityDate="2020-03-12T18:09:04.770" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18598" PostTypeId="2" ParentId="18576" CreationDate="2020-03-12T22:45:57.283" Score="15" Body="&lt;p&gt;In our deep learning lecture, we discussed the following example (from &lt;a href=&quot;https://www.nature.com/articles/s41467-019-08987-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unmasking Clever Hans predictors and assessing what machines really learn&lt;/a&gt; (2019) by Lapuschkin et al.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cLggJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cLggJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong &quot;relevant components&quot;. In the sensitivity maps next to the pictures, we can see that the watermark was used to identify if there is a horse present in the picture. If we remove the watermark, the classification is no longer made. Even more worryingly, if we add the tag to a completely different picture, it gets identified as a horse!&lt;/p&gt;&#xA;" OwnerUserId="34225" LastEditorUserId="9947" LastEditDate="2020-03-23T08:32:22.043" LastActivityDate="2020-03-23T08:32:22.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18599" PostTypeId="2" ParentId="18571" CreationDate="2020-03-13T01:23:00.593" Score="1" Body="&lt;p&gt;In computational learning theory, the &lt;strong&gt;VC dimension&lt;/strong&gt; is a formal measure of the capacity of a model. The VC dimension is defined in terms of the concept of &lt;strong&gt;shattering&lt;/strong&gt;, so have a look at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension#VC_dimension_of_a_set-family&quot; rel=&quot;nofollow noreferrer&quot;&gt;related Wikipedia article&lt;/a&gt;, which briefly describes the fundamental concept of shattering. See also &lt;a href=&quot;https://ai.stackexchange.com/a/17881/2444&quot;&gt;my answer&lt;/a&gt; to the question &lt;a href=&quot;https://ai.stackexchange.com/q/17870/2444&quot;&gt;How to estimate the capacity of a neural network?&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0166218X98000146&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vapnik-Chervonenkis dimension of recurrent neural networks&lt;/a&gt; (1998), by Pascal Koirana and Eduardo D. Sontag, partially (because they do not take into account more advanced recurrent neural network architectures, such as the LSTM) answers your question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper, the authors show and prove different theorems that state the VC dimension of (standard) recurrent neural networks (RNNs), with different activation functions, such as non-linear polynomials, piecewise polynomials and the sigmoid function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;strong&gt;Theorem 5&lt;/strong&gt; (page 70) states&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; be an arbitrary sigmoid. The VC dimension of recurrent architectures with activation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; weights and receiving inputs of length &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, is &lt;span class=&quot;math-container&quot;&gt;$\Omega(wk)$&lt;/span&gt;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The proof of this theorem is given on page 75. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does this theorem intuitively tell you? If you are familiar with &lt;a href=&quot;https://en.wikipedia.org/wiki/Big_O_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;big-O&lt;/a&gt; notation, then you are also familiar with &lt;a href=&quot;https://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann%E2%80%93Landau_notations&quot; rel=&quot;nofollow noreferrer&quot;&gt;the notation &lt;span class=&quot;math-container&quot;&gt;$\Omega(wk)$&lt;/span&gt;&lt;/a&gt;, which means that &lt;span class=&quot;math-container&quot;&gt;$wk$&lt;/span&gt; is, &lt;a href=&quot;https://en.wikipedia.org/wiki/Asymptotic_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;asymptotically&lt;/strong&gt;&lt;/a&gt;, a lower bound on the capacity of the RNN. In other words, asymptotically, the capacity of an RNN with &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; weights receiving inputs of length &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is at least &lt;span class=&quot;math-container&quot;&gt;$wk$&lt;/span&gt;. How does the capacity of the RNN increase as a function of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this is a specific result, which only holds for RNNs with the sigmoid activation function. However, this at least gives you an idea of the potential capacity of an RNN. This theorem will hopefully stimulate your appetite to know more computational learning theory!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://openreview.net/forum?id=Skf-oo0qt7&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Generalization Bounds of a Family of Recurrent Neural Networks&lt;/a&gt; may also be useful, although it has been rejected for ICLR 2019.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-13T01:34:40.127" LastActivityDate="2020-03-13T01:34:40.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18600" PostTypeId="2" ParentId="18590" CreationDate="2020-03-13T01:40:06.163" Score="1" Body="&lt;p&gt;You should at least crop the plots and add a legend. Maybe also provide some scores (accuracy, auc, whatever you're using).&#xA; Anyway, it doesn't look your model is underfitting, if it was you should have high error at both, training and test phase and the lines would not cross. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-13T01:40:06.163" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18601" PostTypeId="2" ParentId="18527" CreationDate="2020-03-13T02:08:26.097" Score="1" Body="&lt;p&gt;I don't think this classify as an NLP problem, there is almost no semantic analysis needed, it is more like a classification problem using categorical features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NLTK is surely valuable if you want to perform some text 'cleaning' or preprocessing before encoding the variables. The only NLP application that I think you could apply here is some sentiment analysis on the comments to extract extra features (like a number expressing the negativeness or positiveness of each comment). Nevertheless you might want to do that using some pre-trained models cause your dataset is pretty small. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-13T02:08:26.097" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18602" PostTypeId="2" ParentId="18576" CreationDate="2020-03-13T05:57:09.487" Score="2" Body="&lt;p&gt;Large scale route optimization problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The is progress made in using Deep Reinforcement learning to solve vehicle routing problems (VRP), for example in this paper: &lt;a href=&quot;https://arxiv.org/abs/1802.04240v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1802.04240v2&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for large scale problems and overall heuristic methods, like the ones provided by Google OR tools are much easier to use.&lt;/p&gt;&#xA;" OwnerUserId="25248" LastActivityDate="2020-03-13T05:57:09.487" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18604" PostTypeId="1" CreationDate="2020-03-13T08:42:01.437" Score="3" ViewCount="130" Body="&lt;p&gt;I'm kinda new to machine learning and still not too solid on math and particularly calculus. I'm currently trying to implement PPO algorithm as described in the spiningUp website : &#xA;&lt;a href=&quot;https://i.stack.imgur.com/9RiYK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9RiYK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This line is giving me a hard time : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XjNZz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XjNZz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the &lt;span class=&quot;math-container&quot;&gt;$\operatorname{argmax}$&lt;/span&gt; mean, in this context? They are also talking about updating the policy with a gradient ascent. So, is taking argmax with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; the same as doing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rGLzR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rGLzR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the min() function?&lt;/p&gt;&#xA;" OwnerUserId="34177" LastEditorUserId="2444" LastEditDate="2020-03-13T23:05:08.310" LastActivityDate="2021-04-08T02:01:45.247" Title="What is the purpose of argmax in the PPO algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18605" PostTypeId="1" AcceptedAnswerId="18676" CreationDate="2020-03-13T08:42:19.557" Score="2" ViewCount="53" Body="&lt;p&gt;What is the current state-of-the-art in unsupervised cross-lingual representation learning?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="9863" LastEditDate="2020-03-13T14:22:19.117" LastActivityDate="2020-03-16T22:35:27.053" Title="What is the current state-of-the-art in unsupervised cross-lingual representation learning?" Tags="&lt;natural-language-processing&gt;&lt;ai-basics&gt;&lt;models&gt;&lt;state-of-the-art&gt;&lt;cross-lingual&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18606" PostTypeId="1" AcceptedAnswerId="18644" CreationDate="2020-03-13T09:10:30.627" Score="4" ViewCount="77" Body="&lt;p&gt;I've just started learning natural language processing from &lt;a href=&quot;https://www.youtube.com/watch?v=3Dt_yh1mf_U&amp;amp;list=PLQiyVNMpDLKnZYBTUOlSI9mi9wAErFtFm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dan Jurafsky's videos lectures&lt;/a&gt;. In that video, minute 4:56, he is stating that dialogue is a hard problem in natural language processing (NLP). Why?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-13T14:17:12.730" LastActivityDate="2020-03-14T19:10:02.510" Title="Why is dialogue a hard problem in natural language processing?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18608" PostTypeId="1" CreationDate="2020-03-13T10:06:48.160" Score="3" ViewCount="246" Body="&lt;p&gt;What is the simplest &lt;strong&gt;classification&lt;/strong&gt; problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?&lt;/p&gt;&#xA;&lt;p&gt;In the first case, the training would be done with the perceptron training rule, in the second case with the delta rule.&lt;/p&gt;&#xA;&lt;p&gt;Note that regression problems cannot be solved by perceptrons, so I'm interested in classification only.&lt;/p&gt;&#xA;" OwnerUserId="34241" LastEditorUserId="2444" LastEditDate="2021-01-19T02:24:04.603" LastActivityDate="2021-02-18T04:08:05.887" Title="What is the simplest classification problem which cannot be solved by a perceptron?" Tags="&lt;classification&gt;&lt;activation-function&gt;&lt;perceptron&gt;&lt;xor-problem&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18609" PostTypeId="1" CreationDate="2020-03-13T10:58:40.600" Score="3" ViewCount="54" Body="&lt;p&gt;I'm new to Reinforcement Learning. For an internship, I am currently training Atari's &amp;quot;Montezuma's Revenge&amp;quot; using a double Deep Q-Network with &lt;a href=&quot;https://arxiv.org/pdf/1707.01495.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience Replay (HER)&lt;/a&gt; (see also &lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-with-hindsight-experience-replay-1fee5704f2f8&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;HER is supposed to alleviate the reward sparseness problem. But since the reward is annoyingly too sparse, I have also added a &lt;a href=&quot;https://arxiv.org/pdf/1810.12894.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Network Distillation (RND)&lt;/a&gt; (see also &lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-with-exploration-by-random-network-distillation-a3e412004402&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;) to encourage the agent to explore new states, by giving it a higher reward when it reaches a previously undiscovered state and a lower reward when it reaches a state it has previously visited multiple times. This is the intrinsic reward I add to the extrinsic reward the game itself gives. I have also used a decaying greedy epsilon policy.&lt;/p&gt;&#xA;&lt;p&gt;How well should this approach work? Because I've set it to run for 10,000 episodes, and the simulation is quite slow, because of the mini-batch gradient descent step in HER. There are multiple hyperparameters here. Before implementing RND, I considered shaping the reward, but that is just impractical in this case. What can I expect from my current approach? OpenAI's paper on RND cites brilliant results with RND on Montezuma's Revenge. But they obviously used PPO.&lt;/p&gt;&#xA;" OwnerUserId="32455" LastEditorUserId="2444" LastEditDate="2020-11-21T12:47:17.067" LastActivityDate="2020-11-21T12:47:17.067" Title="Is this a good approach to solving Atari's &quot;Montezuma's Revenge&quot;?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;hindsight-experience-replay&gt;&lt;random-network-distillation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18610" PostTypeId="2" ParentId="18587" CreationDate="2020-03-13T11:30:03.543" Score="1" Body="&lt;p&gt;It is called &lt;strong&gt;Singularity.&lt;/strong&gt; A point in future where AI will surpass Human Knowledge and become &lt;em&gt;Omniscient.&lt;/em&gt;&#xA;AI will be able to operate on an order manifolds time to that of a human brain thus developing and designing itself without any assistance.&lt;/p&gt;&#xA;" OwnerDisplayName="user26338" LastActivityDate="2020-03-13T11:30:03.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18612" PostTypeId="2" ParentId="18587" CreationDate="2020-03-13T13:15:01.447" Score="1" Body="&lt;p&gt;If you think about it, it is already happening.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Thousands of drivers work for Uber Intelligence.&lt;/li&gt;&#xA;&lt;li&gt;There are many applications that dictate the rules and define what the seller and the end user need to do&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This idea is called Singularity or &lt;a href=&quot;https://en.wikipedia.org/wiki/Technological_singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Technological Singularity&lt;/a&gt;, it would be possible with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superintelligence&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the possibility of this happening is unknown. Have they reached that level yet? We have quantum computers, we have companies with huge data centers spread all over the planet, we have technology in space, we have free Tensorflow and studies for anyone on the planet to be able to create artificial intelligence models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have contact or help from other intelligent civilizations, the possibilities can expand on a surreal scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google has complete information about humanity (or something close to that). But even with all this data, creating artificial intelligence with a conscience is something that goes far beyond.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But maybe we already have enough to improve our concept of morals, respect, social interaction. Facebook invests and studies ways to improve social interaction. And if you think about it, it is one of the main means of communication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The big question is that it is not possible to know what would happen if a super artificial intelligence with conscience would do if it existed. Extinguish humans for harming nature? Just find ways to improve the planet by understanding that human defects and errors are just your own nature as well as everything else in nature? Just watching the show on Netflix because you gave up on humanity? We do not know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, particularly, I would love to see that happen. In fact, one of my personal goals is to create this super intelligence. But alone it will be very difficult. A conscience without interaction from other consciences is not a clash of universes. And the clash of universes is what makes us reflect, think, revise, create new paths and thoughts. It is what allows us to create other universes.&lt;/p&gt;&#xA;" OwnerUserId="7800" LastActivityDate="2020-03-13T13:15:01.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18613" PostTypeId="1" CreationDate="2020-03-13T13:30:02.480" Score="2" ViewCount="74" Body="&lt;p&gt;In the video &lt;a href=&quot;https://www.youtube.com/watch?v=b6nwdc_fGfA&amp;amp;list=PLQiyVNMpDLKnZYBTUOlSI9mi9wAErFtFm&amp;amp;index=14&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evaluation and Perplexity&lt;/a&gt; by Dan Jurafsky, the author talks about extrinsic and perplexity evaluation in the context of natural language processing (NLP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the advantages and disadvantages of extrinsic and perplexity model evaluation in NLP? Which evaluation method is preferred usually, and why?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-13T14:14:55.690" LastActivityDate="2020-03-13T14:14:55.690" Title="What are the advantages and disadvantages of extrinsic and perplexity model evaluation in NLP?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;models&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18615" PostTypeId="1" CreationDate="2020-03-13T14:59:00.400" Score="1" ViewCount="26" Body="&lt;p&gt;I have been reading lately on autoencoders a lot. I just wanted to summarize my understanding of denoising autoencoders. As far as I understand they can be &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Fully connected (in which case, they will be over-complete autoencoders)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convolutional &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The reason I say it should be over-complete is that the objective is to learn new features and I think extra neurons in the latent layer would help. There is no reason to have a lesser number of neurons because compressing is not the objective. I just want to understand is this the right thinking.   &lt;/p&gt;&#xA;" OwnerUserId="34248" LastEditorUserId="2444" LastEditDate="2020-03-13T15:08:38.920" LastActivityDate="2020-03-13T15:08:38.920" Title="Can denoising auto-encoders be convolutional and fully connected?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;autoencoders&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18616" PostTypeId="2" ParentId="18576" CreationDate="2020-03-13T15:23:19.840" Score="1" Body="&lt;p&gt;In the case of convolutional neural networks, the features may be extracted but without taking into account their relative positions (see the concept of &lt;em&gt;translation invariance&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you could have two eyes, a nose and a mouth be in different locations in an image and still have the image be classified as a face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Operations like max-pooling may also have a negative impact on retaining position information.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-03-16T17:22:43.483" LastActivityDate="2020-03-16T17:22:43.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18617" PostTypeId="2" ParentId="13421" CreationDate="2020-03-13T15:34:25.990" Score="0" Body="&lt;p&gt;GraphSage does not have attention at all. Yes, it randomly samples (not  most important as you claim) a subset of neighbors, but it does not compute attention score for each neighbor.&lt;/p&gt;&#xA;" OwnerUserId="34250" LastActivityDate="2020-03-13T15:34:25.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18618" PostTypeId="5" CreationDate="2020-03-13T16:25:04.623" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-13T16:25:04.623" LastActivityDate="2020-03-13T16:25:04.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18619" PostTypeId="4" CreationDate="2020-03-13T16:25:04.623" Score="0" Body="For questions related to the Caffe deep learning library." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-13T16:25:04.623" LastActivityDate="2020-03-13T16:25:04.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18620" PostTypeId="1" CreationDate="2020-03-13T16:54:18.927" Score="2" ViewCount="41" Body="&lt;p&gt;I am using a caffe model of pre-trained &lt;a href=&quot;https://arxiv.org/abs/1409.4842&quot; rel=&quot;nofollow noreferrer&quot;&gt;GoogleNet&lt;/a&gt; trained on ImageNet from &lt;a href=&quot;https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for image retrieval task (place recognition, more specifically).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5wkzB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5wkzB.png&quot; alt=&quot;l&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know the layer with best performance in feature extraction. Its official paper suggests that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;The strong performance of shallower networks on this task suggests&#xA;  that the features produced by the &lt;em&gt;layers in the middle of the network&lt;/em&gt;&#xA;  should be very discriminative.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is also a project &lt;a href=&quot;https://github.com/google/deepdream/blob/master/dream.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;deepdream&lt;/a&gt; suggests that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;The complexity of the details generated depends on which layer's&#xA;  activations we try to maximize. Higher layers produce complex&#xA;  features, while lower ones enhance edges and textures, giving the&#xA;  image an impressionist feeling.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Searching the web, I found a &lt;a href=&quot;https://github.com/colingogo/caffe-pretrained-feature-extraction&quot; rel=&quot;nofollow noreferrer&quot;&gt;github page&lt;/a&gt; suggesting &lt;code&gt;pool5/7x7_s1&lt;/code&gt;layer as feature extractor without specific convincing reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am doing now is quite cumbersome in which I extract features from each individual layer, apply scipy euclidean distance measurement to find a query in the reference database and  the judgment is based on &lt;code&gt;precision-recall&lt;/code&gt; curve and my &lt;code&gt;top 3&lt;/code&gt; results are as follows for one dataset:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;inception_3a/3x3&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;inception_4a/5x5&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;inception_4b/output&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Considering large number of convolutional layers in GoogleNet, my approach is undoubtedly quite inefficient and can be changed to another dataset!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest an efficient way to figure out the layers with the best performance as feature extractors in GoogleNet?&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="31312" LastEditDate="2020-03-23T12:49:04.990" LastActivityDate="2020-03-23T12:49:04.990" Title="Is there an efficient way of determining the layers with the best performance as feature extractors in GoogleNet?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-processing&gt;&lt;caffe&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18621" PostTypeId="1" CreationDate="2020-03-13T17:15:13.620" Score="1" ViewCount="63" Body="&lt;p&gt;I remember back in the 2000s, it was, and still is possible to play against computers in StarCraft BroodWar. They were not as good as pro players, but still reasonably smart. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What AI technologies were used in earlier versions of StarCraft: Brood War?&lt;/p&gt;&#xA;" OwnerUserId="32644" LastEditorUserId="2444" LastEditDate="2020-03-13T17:22:02.197" LastActivityDate="2020-03-13T17:22:02.197" Title="What AI technologies were used in earlier versions of StarCraft?" Tags="&lt;game-ai&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18622" PostTypeId="2" ParentId="16575" CreationDate="2020-03-13T17:41:21.833" Score="0" Body="&lt;p&gt;End to end means deep learning is the only thing that is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many people have doubts on its viability though, I certainly do. I wouldn't trust an end-to-end DL based self driving car. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-03-13T17:41:21.833" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18623" PostTypeId="2" ParentId="5546" CreationDate="2020-03-13T18:08:45.177" Score="0" Body="&lt;p&gt;It's a very simplified explantion. I am just talking about the core idea. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is a combination of many layers.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network (Multiple Layer Perceptron: Regular neural network ): It does a linear combination (a mathematical operation) between the previous layer's output and the current layer's weights(vectors) and then it passes data to the next layer by passing through an activation function. The picture shows a unit of a layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YwCCU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YwCCU.png&quot; alt=&quot;A normal neural network. w represent weight &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network (Convolutional Neural Network):  It does convolution (In signal processing it's known as Correlation) (Its a mathematical operation) between the previous layer's output and the current layer's kernel ( a small matrix ) and then it passes data to the next layer by passing through an activation function. The picture shows a Convolution operation. Each layer may have many convolution operation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/13ayz.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/13ayz.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34164" LastActivityDate="2020-03-13T18:08:45.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18624" PostTypeId="2" ParentId="18542" CreationDate="2020-03-13T18:37:20.230" Score="3" Body="&lt;p&gt;Lot of real tasks are in reality not markovian, but it doesn't mean you can't try to train an agent on these tasks. It's like saying &quot;we assume variable x to be normally distributed&quot;, you just assume that you can condition the probability distributions on the present state of the environment hoping that the agent will learn a good policy. In fact for most applications the challenge is to frame a problem in order to make it as most plausibly markovian as possible (by compressing some important past information into a present state of the environment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is pretty common to just give for granted the Markov property, for example in NLP hidden Markov Models are used a lot for sequential tasks like entities detection, this of course leads to well known issues like high error rate on long sentences (the more you look into the future the more the error rate). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that a Markov model can be of first order (probabilities conditioned only on the present state):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(W_{t+1} = w | W_{t}, W_{t-1},W_{t-2} ..) = P(W_{t+1} = w | W_{t})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but they can also be of higher orders (for example second order if conditioning on the present state plus one step in the past): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(W_{t+1} = w | W_{t}, W_{t-1},W_{t-n} ..) = P(W_{t+1} = w | W_{t},W_{t-1})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the more steps in the past you condition the more quickly the problem becomes intractable, that's why almost always first order models are used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I add some comments on the scheduling paper as suggested by nbro.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, here I would say that the most stryking aspect that makes the process look impossible to describe as an MDP is the presence of dependencies between jobs. Since I might need the result of certain jobs1 before to process  another certain job2, there is surely no way that a time step t could be independent on time step t-1 (I need to know which job I processed in order to know which processes I can process or not). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the trick they use is to learn these dependencies between jobs thanks to a graph network, represented by the DNN in the deep reinforcement learning framework. So, what the agent has to learn is to select a tuple of two actions: &quot;which output (i) a stage designated to be scheduled next, and (ii) an upper limit on the number of executors to use for that stage’s job&quot;. The information used to make this selection is the deep representation computed by the graph network on the decency graph of the jobs to schedule. In this sense, since the network is able to represent the 'temporal' relationships between jobs in the present state, this allows to assume that the choice of the next action tuple do not depend on previous states. Hope this is useful and make some sense to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cFD3H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cFD3H.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="34098" LastEditDate="2020-03-13T23:42:54.027" LastActivityDate="2020-03-13T23:42:54.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18625" PostTypeId="2" ParentId="18585" CreationDate="2020-03-13T19:36:57.600" Score="0" Body="&lt;p&gt;Yes, you need to add goal state as an input otherwise you won't know what goal you are trying to pursue. It might take a lot of memory to run HER on ATARI games though, also defining final goal is not straightforward for such environments&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-03-13T19:36:57.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18626" PostTypeId="2" ParentId="18604" CreationDate="2020-03-13T19:38:21.853" Score="0" Body="&lt;p&gt;In this case yes, &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the big &lt;span class=&quot;math-container&quot;&gt;$\min$&lt;/span&gt; expression and you apply Adam on that. But be careful because they say they do ascent, but automatic differentiation software usually minimizes given function so your &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; would be &lt;span class=&quot;math-container&quot;&gt;$−\min(⋅)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-03-13T19:38:21.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18627" PostTypeId="1" CreationDate="2020-03-13T20:25:28.780" Score="4" ViewCount="315" Body="&lt;p&gt;I am currently working on a project, where I have a sensor in a shoe that records the &lt;span class=&quot;math-container&quot;&gt;$X, Y, Z$&lt;/span&gt; axes, from an acceleration and gyroscope sensor. Every millisecond, I get 6 data points. Now, the goal is, if I do an action, such a jumping or kicking, I would use the sensor's output to predict that action being done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The issue&lt;/strong&gt;: If I jump, for example, one time I may get 1000 data points, but, in another, I get 1200 amounts, meaning the size of the input is different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The neural networks I've studied so far require the input size to be constant to predict a &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; value, however, in this case, it isn't. I've done some research on how to make a neural network with variable sizes, but haven't been able to find one which works. It's not a good idea to crop the input to a certain size, because then I am losing data. In addition, if I just resize the smaller trials by putting extra &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s, it skews the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on a model that would work or how to better clean the data?&lt;/p&gt;&#xA;" OwnerUserId="34263" LastEditorUserId="2444" LastEditDate="2020-03-15T15:13:06.503" LastActivityDate="2020-03-15T15:13:06.503" Title="How should I deal with variable input sizes for a neural network classifier?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18629" PostTypeId="2" ParentId="18576" CreationDate="2020-03-14T01:23:09.577" Score="3" Body="&lt;p&gt;Neural networks seem to have a great deal of difficulty handling &lt;em&gt;adversarial input&lt;/em&gt;, i.e., inputs with certain changes (often imperceptible or nearly imperceptible by humans) designed by an attacker to fool them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the same thing as just being highly sensitive to certain changes in inputs. Robustness against wrong answers in that case can be increased by reducing the probability of such inputs. (If only one in 10^15 possible images causes a problem, it's not much of a problem.) However, in the adversarial case reducing the space of problematic images doesn't reduce the probability of getting one because the images are specifically selected by the attacker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the more famous papers in this area is &lt;a href=&quot;https://arxiv.org/pdf/1707.07397.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Synthesizing Robust Adversarial Examples&quot;&lt;/a&gt;, which produced not only examples where a few modified pixels or other invisible-to-humans modifications to a picture fooled a neural network-based image classifier, but also perhaps the first examples of 3D objects designed to fool similar classifiers and successfully doing so (from every angle!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Those familiar with IT securitity will no doubt recognise this as a familiar asymmetry: roughly, a defender must defend against &lt;em&gt;all&lt;/em&gt; attacks launched against a system, but an attacker need find only &lt;em&gt;one&lt;/em&gt; working attack.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1901.10861.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance&quot;&lt;/a&gt;, Adi Shamir et al. propose a mathematical framework for analyzing the problem based on Hamming distances that, while currently a less practical attack than the MIT/Lab6 one, has some pretty disturbing theoretical implications, including that current approaches to preventing these attacks may be, in the end, ineffective. For example, he points out that blurring and similar techniques that have been used to try to defend against adversarial attacks can be treated mathematically as simply another layer added on top of the existing neural network, requiring no changes to the attack strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I attended a talk by Shamir a few months ago that was much easier going than the paper, but unfortunately I can't find a video of that or a similar talk on-line; if anybody knows of one please feel free to edit this answer to add a link!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's obviously still an enormous amount of research to be done in this area, but it seems possible that neural networks alone are not capable of defense against this class of attack, and other techniques will have to be employed in addition to make neural networks robust against it.&lt;/p&gt;&#xA;" OwnerUserId="34269" LastEditorUserId="34269" LastEditDate="2020-03-14T02:07:00.837" LastActivityDate="2020-03-14T02:07:00.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18630" PostTypeId="1" AcceptedAnswerId="18632" CreationDate="2020-03-14T02:41:25.770" Score="1" ViewCount="97" Body="&lt;p&gt;I am trying to code out a policy evaluation algorithm to find the &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s)$&lt;/span&gt; for all states. The following diagram below shows the MDP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y80eU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y80eU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case i let p = q = 0.5.&#xA;the rewards for each states are independent of action. I.e &lt;span class=&quot;math-container&quot;&gt;$r(\sigma_0)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$r(\sigma_2)$&lt;/span&gt; = 0,&lt;span class=&quot;math-container&quot;&gt;$r(\sigma_1)$&lt;/span&gt; = 1, &lt;span class=&quot;math-container&quot;&gt;$r(\sigma_3)$&lt;/span&gt; = 10. Terminal state is &lt;span class=&quot;math-container&quot;&gt;$r(\sigma_3)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following policy, {0:1, 1:0, 2:0}, where key is the state and value is the action. 0 for &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt; and 1 for &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Policy Iteration solver for FUN&#xA;class PolicyEvaluation:&#xA;    def __init__(self, policies):&#xA;        self.N = 3&#xA;        self.pi = policies&#xA;        self.actions = [0, 1] # a0 and a1&#xA;        self.discount = 0.7&#xA;        self.states = [i for i in range(self.N + 1)]&#xA;&#xA;&#xA;    def terminalState(self, state):&#xA;        return state == 3&#xA;&#xA;    # assume p = q = 0.5&#xA;    def succProbReward(self, state):&#xA;        # (newState, probability, reward)&#xA;        spr_list = []&#xA;        if (state == 0 and self.pi[state] == 0):&#xA;            spr_list.append([1, 1.0, 1])&#xA;        elif (state == 0 and self.pi[state] == 1):&#xA;            spr_list.append([2, 1.0, 0])&#xA;        elif (state == 1 and self.pi[state] == 0):&#xA;            spr_list.append([2, 0.5, 0])&#xA;            spr_list.append([0, 0.5, 0])&#xA;        elif (state == 2 and self.pi[state] == 0):&#xA;            spr_list.append([1, 1.0, 0])&#xA;        elif (state == 2 and self.pi[state] == 1):&#xA;            spr_list.append([3, 0.5, 10])&#xA;            spr_list.append([2, 0.5, 0])&#xA;        return spr_list&#xA;&#xA;&#xA;def policyEvaluation(mdp):&#xA;    # initialize&#xA;    V = {} &#xA;    for state in mdp.states:&#xA;        V[state] = 0&#xA;&#xA;    def V_pi(state):&#xA;        return sum(prob * (reward + mdp.discount*V[newState]) for prob, reward, newState in&#xA;        mdp.succProbReward(state))&#xA;&#xA;    while True:&#xA;    # compute new values (newV) given old values (V)&#xA;        newV = {}&#xA;        for state in mdp.states:&#xA;            if mdp.terminalState(state):&#xA;                newV[state] = 0&#xA;            else:&#xA;                newV[state] = V_pi(state)&#xA;&#xA;        if max(abs(V[state] - newV[state]) for state in mdp.states) &amp;lt; 1e-10:&#xA;            break&#xA;        V = newV&#xA;        print(V)&#xA;    print(V)&#xA;&#xA;&#xA;&#xA;pE = PolicyEvaluation({0:1, 1:0, 2:0})&#xA;print(pE.states)&#xA;print(pE.succProbReward(0))&#xA;policyIteration(pE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tried to run the code above to find the values for each state, however, I am not converging with my values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something wrong that I did?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-04-16T19:25:10.257" LastActivityDate="2020-04-16T19:25:10.257" Title="Why isn't the implementation of my policy evaluation for a simple MDP converging?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;implementation&gt;&lt;convergence&gt;&lt;policy-evaluation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18631" PostTypeId="2" ParentId="18558" CreationDate="2020-03-14T02:47:06.440" Score="0" Body="&lt;p&gt;A way to avoid computing the SVM loss by hand is to use a differentiable programming framework, such as &lt;a href=&quot;https://github.com/google/jax&quot; rel=&quot;nofollow noreferrer&quot;&gt;JAX&lt;/a&gt;. These frameworks will automatically calculate gradients using automatic differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can write down the SVM loss using numpy operations then you can use the framework's tools to get a function which evaluates the gradient with respect to any argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In JAX this would look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import jax.numpy as np&#xA;def hinge_loss(x, y, theta):&#xA;    # x is an nxd matrix, y is an nx1 matrix&#xA;    y_hat = model(x, theta) # returns nx1 matrix, model parameters theta&#xA;    return np.maximum(0, 1 - y_hat * y)&#xA;&#xA;hinge_loss_grad = grad(hinge_loss)&#xA;# hinge_loss_grad takes an x, y, theta and returns gradient of hinge loss wrt x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15176" LastActivityDate="2020-03-14T02:47:06.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18632" PostTypeId="2" ParentId="18630" CreationDate="2020-03-14T03:23:21.337" Score="2" Body="&lt;p&gt;The issue is that in your list comprehension in def &lt;code&gt;V_pi(state)&lt;/code&gt; you have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return sum(prob * (reward + mdp.discount*V[newState]) for prob, reward, newState in&#xA;        mdp.succProbReward(state))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;whereas with the way you have defined the succProbReward output, it should be &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return sum(prob * (reward + mdp.discount*V[newState]) for newState, prob, reward in&#xA;        mdp.succProbReward(state))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run this it converges immediately with a reward of 0 for all states, which I believe is correct for the policy you specified. If I change the policy it also seems to give reasonable results.&lt;/p&gt;&#xA;" OwnerUserId="15176" LastActivityDate="2020-03-14T03:23:21.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18633" PostTypeId="1" CreationDate="2020-03-14T04:28:35.173" Score="2" ViewCount="33" Body="&lt;p&gt;I am working through Sutton and Barto's &lt;a href=&quot;http://www.incompleteideas.net/book/the-book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL book&lt;/a&gt;. So far in the text, when backup diagrams are drawn, the reward and next state are iterated together (i.e. the equations always have &lt;span class=&quot;math-container&quot;&gt;$\sum_{s',r}$&lt;/span&gt;), because the text uses the four-place function &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt;. Starting from a solid circle (state-action pair), each edge has a reward labeled along the edge and the next state labeled on the open circle. (See page 59 for an example diagram, or see Figure 3.4 &lt;a href=&quot;http://www.incompleteideas.net/book/ebook/node34.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, exercise 3.29 asks to rewrite the Bellman equations in terms of &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r(s,a)$&lt;/span&gt;. This means that the reward is an expected value (i.e. we don't want to iterate over rewards like &lt;span class=&quot;math-container&quot;&gt;$\sum_r \cdots (r + \cdots)$&lt;/span&gt;), whereas the next states should be iterated (i.e. we want something like &lt;span class=&quot;math-container&quot;&gt;$\sum_{s'} p(s'|s,a) (\cdots)$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think writing the Bellman equations themselves isn't too difficult; my current guess is that they look like this: &lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \sum_a \pi(a|s) \left(r(s,a) + \gamma \sum_{s'} p(s'|s,a) v_\pi(s')\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_\pi(s,a) = r(s,a) + \gamma \sum_{s'} p(s'|s,a) \sum_{a'} \pi(a'|s') q_\pi(s',a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem instead is that I want to be able to draw the backup diagrams corresponding to these equations. Given the &quot;vocabulary&quot; for backup diagrams given in the book (e.g. solid circle = state-action pair, open circle = state, rewards along the edge, probabilities below nodes, maxing over denoted by an arc), I don't know how to represent the fact that the reward and next state are treated differently. Two ideas that don't seem to work:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I draw a bunch of edges after the solid circle, that looks like I'm iterating over rewards.&lt;/li&gt;&#xA;&lt;li&gt;If I come up with a special kind of edge that represents an expected reward, then it looks like only a single next state is being considered.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33930" LastActivityDate="2020-03-14T04:28:35.173" Title="How to draw backup diagram when reward is in expectation but next state is iterated?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18634" PostTypeId="1" CreationDate="2020-03-14T06:52:53.367" Score="7" ViewCount="2707" Body="&lt;p&gt;The skip-gram and continuous bag of words (CBOW) are two different types of word2vec model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the main differences between them? What are the pros and cons of both methods?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-14T09:35:32.593" LastActivityDate="2021-01-18T11:27:12.813" Title="What are the main differences between skip-gram and continuous bag of words?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;models&gt;&lt;word2vec&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18636" PostTypeId="1" AcceptedAnswerId="18638" CreationDate="2020-03-14T08:11:44.667" Score="3" ViewCount="50" Body="&lt;p&gt;I'm new to the topic, but I've used some off the shelf knowledge about computer vision for classifying images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you can easily generate labels that can determine whether or not e.g. a cloud is in the image. However, what is the general type of problem called where you want to assign a value, or rate the image on a scale - in this example, the degree of cloudiness in the image? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are useful algorithms or techniques for addressing this type of problem?&lt;/p&gt;&#xA;" OwnerUserId="34275" LastEditorUserId="2444" LastEditDate="2020-03-14T17:34:53.847" LastActivityDate="2020-03-14T17:34:53.847" Title="What is the type of problem requiring to rate images on a scale?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18637" PostTypeId="2" ParentId="18634" CreationDate="2020-03-14T14:14:13.347" Score="7" Body="&lt;p&gt;So as you're probably already aware of, CBOW and Skip-gram are just mirrored versions of each other. CBOW is trained to predict a single word from a fixed window size of context words, whereas Skip-gram does the opposite, and tries to predict several context words from a single input word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, the first task is much simpler, this implies a much faster convergence for CBOW than for Skip-gram, in the original paper (link below) they wrote that CBOW took hours to train, Skip-gram 3 days. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the same logic regarding the task difficulty, CBOW learn better syntactic relationships between words while Skip-gram is better in capturing better semantic relationships. In practice, this means that for the word 'cat' CBOW would retrive as closest vectors morphologically similar words like plurals, i.e. 'cats' while Skip-gram would consider morphologically different words (but semantically relevant) like 'dog' much closer to 'cat' in comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A final consideration to make deals instead with the sensitivity to rare and frequent words. Because Skip-gram rely on single words input, it is less sensitive to overfit frequent words, because even if frequent words are presented more times that rare words during training, they still appear individually, while CBOW is prone to overfit frequent words because they appear several time along with the same context. This advantage over frequent words overfitting leads Skip-gram to be also more efficient in term of documents required to achieve good performances, much less than CBOW (and it's also the reason of the better performances of Skip-gram in capturing semantical relationships).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, you can find some comparisons in the original paper (section 4.3). &#xA;&lt;a href=&quot;https://arxiv.org/pdf/1301.3781.pdf&quot; rel=&quot;noreferrer&quot;&gt;Mikolov et al. 2013&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the architecture, there's not much to say. They just randomly initialised word embedding for each word, then a projection matrix NxD (number of context words times embeddings dimension) is generated at each iteration, there is no hidden layer, the vectors are just averaged together and then fed into an activation function to predict index probabilities in a vector of dimension V (the size of the vocabulary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more specific explanation (even Mikolov's paper lack some detail) I suggest checking this blog page (&lt;a href=&quot;https://iksinc.online/tag/continuous-bag-of-words-cbow/&quot; rel=&quot;noreferrer&quot;&gt;Words as Vectors&lt;/a&gt;), even though the model described there do apply a hidden layer, unlike the original architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ShJJX.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ShJJX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="2444" LastEditDate="2020-03-16T22:13:12.040" LastActivityDate="2020-03-16T22:13:12.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18638" PostTypeId="2" ParentId="18636" CreationDate="2020-03-14T14:36:07.950" Score="3" Body="&lt;p&gt;The main distinction between tasks is 'classification' vs 'regression'. In classification you would try to identify the presence of a cloud or not in an image, if you want to predict the level of 'cloudness' with continuous values you are then performing a regression task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware about state-of-the models specific for images, but you can potentially use whatever architecture you desire to perform regression, CNN, RNN, the only thing to pay attention to is the loss function you will use. There are specific functions for classifications (which usually use the argmax function to turn probabilities into labels) and for regression (the most used is the Mean Squared Error, the model try to approximate the continuous values directly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a quick overview of loss functions I suggest this easy tutorial &lt;a href=&quot;https://heartbeat.fritz.ai/5-regression-loss-functions-all-machine-learners-should-know-4fb140e9d4b0&quot; rel=&quot;nofollow noreferrer&quot;&gt;5 Regression Loss Functions&lt;/a&gt;. Hope it might be of use.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-14T14:36:07.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18639" PostTypeId="2" ParentId="18627" CreationDate="2020-03-14T15:04:05.317" Score="2" Body="&lt;p&gt;It is much simpler to process the data in a different way. Since you're using temporal data a common practice is to define a priori a minimum time-step, usually called &lt;span class=&quot;math-container&quot;&gt;$\textit{granularity}$&lt;/span&gt;, which must be bigger than you're sensor responsiveness. Using this granularity value you'll then be able split your data into intervals, and you can then combine each instance belonging to an interval with a function that you like. Most common choice is obviously averaging the values, but also summing then could be a choice or a moving average. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't think that in this way you will loose information, preprocessing is also called data cleaning for a reason, not aways raw data are better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a last note, I would suggest you to look into 'Machine learning for the quantified self', there is growing literature going on about using sensors to train predictive models about movements and body measures like heart-rate. You might find something about preprocessing for this specific sensor-applications. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-14T15:04:05.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18640" PostTypeId="2" ParentId="18606" CreationDate="2020-03-14T17:54:46.653" Score="1" Body="&lt;p&gt;&lt;em&gt;First of all, I am not very familiar with details of NLP and NLU systems and concepts, so I will provide an answer based on the slides entitled &lt;a href=&quot;https://web.archive.org/web/20170830051346/https://projects.ict.usc.edu/nld/cs599s13/LectureNotes/cs599s13dialogue1-30-13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural language understanding in dialogue systems&lt;/a&gt; (2013) by &lt;a href=&quot;http://djdsite.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;David DeVaul&lt;/a&gt;, a researcher on the topic.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A dialogue system is composed of different parts or modules. Here's a diagram of an example of a dialogue system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SEZg1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SEZg1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these modules can introduce &lt;strong&gt;errors&lt;/strong&gt;, which are then propagated to the rest of the pipeline. Of course, this is the first clear issue of such a dialogue system. Other issues or challenges include &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;ambiguity&lt;/strong&gt; of natural language (and there are different types of ambiguity, i.e. see slide number 5), &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;synonyms&lt;/strong&gt; (i.e. the dialogue system needs to handle different words or expressions that mean the same thing), &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;context-sensitivity&lt;/strong&gt; (i.e. the same words or expressions can mean different things in different contexts)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;semantic representation&lt;/strong&gt; (i.e. how to represent semantics)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;spontaneous speech&lt;/strong&gt; (i.e. how to handle stuff like &quot;hm&quot;, pauses, etc.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-14T17:54:46.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18641" PostTypeId="2" ParentId="16899" CreationDate="2020-03-14T18:12:56.040" Score="0" Body="&lt;p&gt;According to &lt;a href=&quot;https://web.stanford.edu/~jurafsky/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dan Jurafsky&lt;/a&gt;, a researcher on NLP and NLU, the current hard problems in NLP are (see &lt;a href=&quot;http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide 6&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Questioning answering&lt;/li&gt;&#xA;&lt;li&gt;Paraphrase&lt;/li&gt;&#xA;&lt;li&gt;Summarisation&lt;/li&gt;&#xA;&lt;li&gt;Dialogue&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other hard problems for which there are already some good solutions are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sentiment analysis&lt;/li&gt;&#xA;&lt;li&gt;Coreference resolution&lt;/li&gt;&#xA;&lt;li&gt;Word sense disambiguation&lt;/li&gt;&#xA;&lt;li&gt;Parsing&lt;/li&gt;&#xA;&lt;li&gt;Machine translation&lt;/li&gt;&#xA;&lt;li&gt;Information extraction&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Ambiguity in natural language is one of the biggest challenges for NLP and NLU systems. Other challenges are (see &lt;a href=&quot;http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide 10&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;non-standard words&lt;/li&gt;&#xA;&lt;li&gt;idioms&lt;/li&gt;&#xA;&lt;li&gt;tricky entity names&lt;/li&gt;&#xA;&lt;li&gt;neologisms &lt;/li&gt;&#xA;&lt;li&gt;world knowledge &lt;/li&gt;&#xA;&lt;li&gt;segmentation issues&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-14T18:12:56.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18642" PostTypeId="5" CreationDate="2020-03-14T18:41:01.450" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-14T18:41:01.450" LastActivityDate="2020-03-14T18:41:01.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18643" PostTypeId="4" CreationDate="2020-03-14T18:41:01.450" Score="0" Body="For questions related to natural language understanding (NLU), which is a subtopic of natural language processing (NLP) that deals with machine reading comprehension. Natural-language understanding is considered a hard problem." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-14T18:41:01.450" LastActivityDate="2020-03-14T18:41:01.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18644" PostTypeId="2" ParentId="18606" CreationDate="2020-03-14T19:10:02.510" Score="3" Body="&lt;p&gt;Dialogue is a hard problem because it requires pretty advanced cognitive functions. Leaving aside all the lower levels of language analysis (phonology if dealing with speech, morphology and syntax), you quickly run into interpretation problems that require a lot of world knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple question and answer is fine, and restricted domains are somewhat easier as well. As soon as you get into a normal conversation, you will refer back to things you said before, so an NLP system would have to recognise that and resolve the reference accordingly. Typically in a conversation you would use variations of reference terms: the first time you mention an object you might describe it fully, but subsequently you will use shorter terms to refer to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a structure to conversation. This is typically modelled as conversational moves, and usually moves will have corresponding response-moves. For example, a common sequence would be greeting - greeting. Then you might have question - response - feedback. This sounds fairly easy, but once you try to annotate a dialogue with such moves you will find that it is pretty hard. As far as I am aware, there is no 'grammar' equivalent of describing the structure of conversations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often, pragmatic meaning is interfering with the 'surface' meaning of utterances. A statement can actually function as a question, or a question can be a command (or a statement). The pragmatics of utterances depend on the context and also the relationship between the interlocutors. If I talk to my manager, I will use language differently than when I talk to my children.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dialogue/conversations are hard to analyse. Because of that, descriptive frameworks are still fairly limited. You need to keep track of what has said before, as that can change the way an utterance has to be interpreted. Grammatical analysis is a fairly solved problem, and word sense disambiguation as well. But pragmatics and conversational structure are still on the bleeding edge of linguistic research; at least they were when I was still teaching Discourse Analysis at university a few years ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that reason, chatbots are generally not very good. Sometimes they can fool people into believing they are human speakers, but this is usually done through trickery (&quot;smoke and mirrors&quot;) rather than competent handling of conversational structures. It's all in the little box in nbro's answer labelled &quot;DM&quot;...&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-03-14T19:10:02.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18645" PostTypeId="1" CreationDate="2020-03-14T20:23:25.230" Score="6" ViewCount="267" Body="&lt;p&gt;Reinforcement learning methods are considered to be extremely sample inefficient.&lt;/p&gt;&#xA;&lt;p&gt;For example, in a recent DeepMind &lt;a href=&quot;https://arxiv.org/abs/1710.02298&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; by Hessel et al., they showed that in order to reach human-level performance on an Atari game running at 60 frames per second they needed to run 18 million frames, which corresponds to 83 hours of play experience. For an Atari game that most humans pick up within a few minutes, this is a lot of time.&lt;/p&gt;&#xA;&lt;p&gt;What makes DQN, DDPG, and others, so sample inefficient?&lt;/p&gt;&#xA;" OwnerUserId="14390" LastEditorUserId="2444" LastEditDate="2020-10-13T08:27:57.973" LastActivityDate="2020-10-13T08:27:57.973" Title="Why are reinforcement learning methods sample inefficient?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;papers&gt;&lt;ddpg&gt;&lt;sample-efficiency&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18647" PostTypeId="1" AcceptedAnswerId="18656" CreationDate="2020-03-14T21:32:46.653" Score="4" ViewCount="72" Body="&lt;p&gt;I am trying to understand an algorithm for correcting mislabeled data in the paper &lt;a href=&quot;https://pdfs.semanticscholar.org/3a09/c6f3873dd0327a3f9d83b7ff1c8b43a06cc3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An algorithm for correcting mislabeled data&lt;/a&gt; (2001) by Xinchuan Zeng et al. The authors are suggesting to update the output class probability vector using the formula in equation 4 and class label in equation 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are they updating labels while training, starting from very first back-propagation? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It seems like if we train on the same data and then predict labels on the same data, it would be the same as what the authors are suggesting. Does it make sense or I misunderstood?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="34290" LastEditorUserId="34290" LastEditDate="2020-03-15T21:10:55.250" LastActivityDate="2020-03-15T21:50:26.617" Title="Are the labels updated during training in the algorithm presented in &quot;An algorithm for correcting mislabeled data&quot;?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;multilayer-perceptron&gt;&lt;multi-label-classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18648" PostTypeId="1" CreationDate="2020-03-14T21:59:55.010" Score="7" ViewCount="92" Body="&lt;p&gt;The formula for mean prediction using Gaussian Process is &lt;span class=&quot;math-container&quot;&gt;$k(x_*, x)k(x, x)^{-1}y$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the covariance function. See e.g. equation 2.23 (in chapter 2) from &lt;a href=&quot;http://www.gaussianprocess.org/gpml/chapters/RW2.pdf&quot; rel=&quot;noreferrer&quot;&gt;Gaussian Processes for Machine Learning&lt;/a&gt; (2006) by C. E. Rasmussen &amp;amp; C. K. I. Williams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oversimplifying, the mean prediction of the new point &lt;span class=&quot;math-container&quot;&gt;$y_*$&lt;/span&gt; is the weighted average of previously observed &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, where the weights are calculated by the &lt;span class=&quot;math-container&quot;&gt;$k(x_*,x)$&lt;/span&gt; and normalized by &lt;span class=&quot;math-container&quot;&gt;$k(x,x)^{-1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the first part &lt;span class=&quot;math-container&quot;&gt;$k(x_*, x)$&lt;/span&gt; is easy to interpret. The closer the new data point lies to the previously observed data points, the greater their similarity, the higher will be the weight and impact on the prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how to interpret the second part &lt;span class=&quot;math-container&quot;&gt;$k(x, x)^{-1}$&lt;/span&gt;? I presume this makes the weight of the points in the clusters greater than the outliers. Am I correct?&lt;/p&gt;&#xA;" OwnerUserId="31988" LastEditorUserId="2444" LastEditDate="2020-03-16T22:04:38.610" LastActivityDate="2021-01-13T21:01:35.147" Title="Interpretation of inverse matrix in mean calculation in Gaussian Process" Tags="&lt;math&gt;&lt;gaussian-process&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18650" PostTypeId="2" ParentId="18645" CreationDate="2020-03-14T22:06:15.573" Score="5" Body="&lt;p&gt;I will try to give a broad answer, if it's not helpful I'll remove it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we talk about sampling we are actually talking about the number of interaction required to an agent to learn a good model of the environment.&#xA;In general I would say that there are two issues related to sample efficiency:&#xA;1 the size of the 'action'+'environment states' space 2 the exploration strategy used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the first point, in reinforcement learning is really easy to encounter situations in which the number of combinations of possible actions and possible environment states explode, becoming intractable. Lets for example consider the Atari game from the Rainbow paper you linked: the environment in which the agent operate in this case is composed of rgb images of size (210, 160, 3). This means that the agent 'see' a vector of size 100800. The actions that an agent can take are simply modifications of this vector, e.g. I can move to the left a character, slightly changing the whole picture. Despite the fact that in lot of games the number of possible actions is rather small, we must keep in mind that there are also other objects in the environment which change position as well. What other object/enemies do obviously influence the choice of the best action to perform in the next time step. To a high number of possible combinations between actions and environment states is associated a high number of observations/interaction required to learn a good model of the environment. Up to now, what people usually do is to compress the information of the environment (for example by resizing and converting the pictures to grayscale), to reduce the total number of possible states to observe. DQL itself is based on the idea of using neural networks to compress the information gathered from the environment in a dense representation of fixed size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what concern the exploration strategy, we can again divide the issue in subcategories: 1 how do we explore the environment 2 how much information do we get from each exploration. Exploration is usually tuned  through the &lt;em&gt;greedy&lt;/em&gt; hyper-parameter. Once in a while we let the agent perform a random action, to avoid to get stuck in suboptimal policies (like not moving at all to avoid to fall into a trap, eventually thanks to a greedy action the agent will try to jump and learn that it gives a higher reward). Exploration comes with the cost of more simulations to perform, so people quickly realise that we can't rely only on more exploration to train better policies. One way to boost performance is to leverage not only the present iteration but also past interactions as well, this approach is called &lt;em&gt;experience replay&lt;/em&gt;. The underline idea is to update the q-values depending also on weighted past rewards, stored in a memory buffer. Other approaches point to computation efficiency rather than decreasing the amount of simulations. An old proposed technique that follow this direction is &lt;em&gt;prioritised sweeping&lt;/em&gt; &lt;a href=&quot;https://people.eecs.berkeley.edu/~pabbeel/cs287-fa19/optreadings/MooreAtkeson-1993-prioritized-sweeping.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moore et al. 1993&lt;/a&gt;, in which big changes in q values are prioritised, i.e. q-values that are stable over iterations are basically ignored (this is a really crude way to put it, I have to admit that I still have to grasp this concept properly). Both this techniques were actually applied in the Rainbow paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a more personal level (just pure opinions of mine from here) I would say that the problem between RL agents and humans is the fact that we (humans) have lot of common sense knowledge we can leverage, and somehow we are able, through cognitive heuristics and shortcuts, to pay attention to what is relevant without even being aware of it. RL agents learn to interact with an environment without any prior knowledge, they just learn some probability distribution through trial and errors, and if something completely new happens they have no ability at all to pick up an action based on external knowledge. &#xA;One interesting future direction in my opinion is reward modelling, described in this video: &lt;a href=&quot;https://youtu.be/PYylPRX6z4Q&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/PYylPRX6z4Q&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I particularly like the emphasis on the fact that the only true thing that human are good at is judging. We don't know how to design proper reward functions, because again, most of the actions we perform in real life are driven by reward of which we are not aware, but we are able in a glimpse to say if an agent is performing a task in a proper way or not. Combining this 'judging power' into RL exploration seems to be a really powerful way to increase sample efficiency in RL.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="34098" LastEditDate="2020-04-10T01:05:50.180" LastActivityDate="2020-04-10T01:05:50.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18651" PostTypeId="5" CreationDate="2020-03-15T00:17:12.267" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-15T00:17:12.267" LastActivityDate="2020-03-15T00:17:12.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18652" PostTypeId="4" CreationDate="2020-03-15T00:17:12.267" Score="0" Body="For questions related to the embodied cognition, which is the theory that many features of cognition, whether human or otherwise, are shaped by aspects of the entire body of the organism." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-15T00:17:12.267" LastActivityDate="2020-03-15T00:17:12.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18654" PostTypeId="1" AcceptedAnswerId="20759" CreationDate="2020-03-15T20:23:42.207" Score="2" ViewCount="166" Body="&lt;p&gt;I have a basic MCTS agent for the game of Hex (a turn based game). I want to tune the parameters of UCT (the Cp parameter) and the number of rollouts parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do I have to begin? The problem is that the agent is smart enough to always win if it plays first against another agent. So I don't know how to do the evaluation of each pair of hyperparameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has any ideas let me know.&lt;/p&gt;&#xA;" OwnerUserId="33307" LastActivityDate="2020-04-29T11:29:34.627" Title="How to apply hyperparameter optimization on Monte Carlo Tree Search?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;monte-carlo-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18656" PostTypeId="2" ParentId="18647" CreationDate="2020-03-15T21:50:26.617" Score="2" Body="&lt;p&gt;I think that making some draws might help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below I tried to draw the model architecture. We start with classic feed-forward structure: input represented by a vector &lt;strong&gt;I&lt;/strong&gt; with length &lt;em&gt;f&lt;/em&gt; (number of features), a hidden layer &lt;strong&gt;H&lt;/strong&gt; which does &lt;strong&gt;not&lt;/strong&gt; have a fixed size, and output &lt;strong&gt;O&lt;/strong&gt; of length &lt;em&gt;c&lt;/em&gt; (number of classes). Then we have 3 extra vectors than usual:&#xA;a vector &lt;strong&gt;U&lt;/strong&gt; they refer as input (a bit confusing I have to say), and two vectors &lt;strong&gt;V&lt;/strong&gt; and &lt;strong&gt;P&lt;/strong&gt; representing classes probabilities. All these vectors have length &lt;em&gt;c&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;P&lt;/strong&gt; is what we want to learn, new class probabilities for each instance that should correct wrong labels of misclassified instances in the initial dataset.&#xA;So in some sense the aim of the work is not to train a model to make predictions but rather train a model to clean a training dataset. I think it's important to stress out this point because we talk about training but there is actually no test after the training, we just end with a modified training dataset with some instances relabelled. &#xA;The relabelling depends on the learning of &lt;strong&gt;V&lt;/strong&gt; rather than P (because as I will say also later on, the final arrow that goes from &lt;strong&gt;V&lt;/strong&gt; to &lt;strong&gt;P&lt;/strong&gt; is the identity function). &lt;strong&gt;V&lt;/strong&gt; depends on &lt;strong&gt;U&lt;/strong&gt; that also depends on &lt;strong&gt;V&lt;/strong&gt;, again it sounds a bit confusing, the tricks rely in the initialisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second picture I just copied some formulas from the paper. They don't specifically say how they get the vector &lt;strong&gt;P&lt;/strong&gt; in the first place, but I treat it simply as given because we need it to initialise &lt;strong&gt;U&lt;/strong&gt; by applying the inverse sigmoid function, element-wise. We need also to define the hyper parameter D, the initial probability value for the true class V&lt;span class=&quot;math-container&quot;&gt;$_y$&lt;/span&gt; in the initial dataset. In the paper they set it to .95. Using D we can initialise the vector &lt;strong&gt;V&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we initialise &lt;strong&gt;U&lt;/strong&gt; and &lt;strong&gt;V&lt;/strong&gt; we have all the elements to iterate over the dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A first thing to notice is that they don't use back propagation to update &lt;strong&gt;U&lt;/strong&gt;,&lt;strong&gt;V&lt;/strong&gt; and &lt;strong&gt;P&lt;/strong&gt;, they just define some updating rules (which require to define some other initial parameters, &lt;span class=&quot;math-container&quot;&gt;$\textit{u$_0$}$&lt;/span&gt; and L&lt;span class=&quot;math-container&quot;&gt;$_p$&lt;/span&gt;). I want to stress out again that the mapping between &lt;strong&gt;V&lt;/strong&gt; and &lt;strong&gt;P&lt;/strong&gt; is just the identity function, I guess that they define it just to avoid confusion, because in the paper &lt;strong&gt;P&lt;/strong&gt; appear as an input element, but nevertheless is also the output we want to learn. Back propagation is used only to update the weights in &lt;strong&gt;H&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ANSWERS TO THE QUESTIONS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can finally say that, regarding your first question, the answer is yes, they do start updating the classes probability vectors &lt;strong&gt;P&lt;/strong&gt; from the very first iteration, even though is not clear how they initialise them (or from where they got them). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the second question instead I would say that no, this is definitely not as training a model with fixed label and then making predictions over the training dataset again. The whole point is that similar training instances will be initialised with the same &lt;strong&gt;P&lt;/strong&gt;, and they will also produce similar &lt;strong&gt;U&lt;/strong&gt; and &lt;strong&gt;V&lt;/strong&gt; vectors. For misclassified instances, &lt;strong&gt;U&lt;/strong&gt; will be updated with larger changes, because of the different produced &lt;strong&gt;O&lt;/strong&gt;. For example an image of a 2 will generate a different output than an image of a 5, this would be reflected first on &lt;strong&gt;U&lt;/strong&gt; and then on &lt;strong&gt;V&lt;/strong&gt; and &lt;strong&gt;P&lt;/strong&gt; . If training with fixed label instead you would just force the weight in the hidden layer to learn a function that treat the representation of a 2 and a 5 in a similar way, leading to low accuracy cause you would be telling the model &quot;Hey, the straight line in the 5 is important also to recognise a 2&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to say that I never read about this dataset cleaning approach but it is interesting, and their results show that the cleaned version of the dataset lead to better performances, which is interesting because machine learning analysis usually gives for granted the correctness of the labels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is of help in some ways! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lyUIl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lyUIl.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3MQ1V.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3MQ1V.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-15T21:50:26.617" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18657" PostTypeId="2" ParentId="10644" CreationDate="2020-03-16T01:41:54.830" Score="0" Body="&lt;p&gt;I would approach it from a direction different from @Phylliida, though there seems to be nothing wrong with her answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMHO, when AI &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A: is sufficiently general,&lt;/li&gt;&#xA;&lt;li&gt;B: is able to direct its own evolution,&lt;/li&gt;&#xA;&lt;li&gt;C: has control (whether direct or indirect) of all the resources needed to grow and evolve, and&lt;/li&gt;&#xA;&lt;li&gt;D: has the &lt;em&gt;goal&lt;/em&gt; of growing and evolving to solve a big, important problem,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then the &quot;singularity&quot; will have been reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;sufficiently general&quot; means that its growth is not limited by the code that initially defines it: it can re-write its own code (through &quot;offspring in a sandbox&quot;, or directly; doesn't matter which).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic programming is currently clumsy but is indeed sufficiently general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C is something that would most likely need to be given to it, so nobody will be able to unplug it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D is easy.  I'd like to see an effort in the Futurist and AI communities to choose such goals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't a mathematical definition; it's something more mundane; but I think it's to the point.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2020-03-16T01:41:54.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18658" PostTypeId="1" AcceptedAnswerId="18687" CreationDate="2020-03-16T02:36:18.900" Score="1" ViewCount="151" Body="&lt;p&gt;How can I prove that gradient descent doesn't necessarily find the global optimum?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the following function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x_1, x_2, x_3, x_4) = (x_1 + 10x_2)^2 + 5x_2^3 + (x_2 + 2x_3)^4 + 3x_1x_4^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume also that we can't find the optimal value for the learning rate because of time constraints.&lt;/p&gt;&#xA;" OwnerUserId="33875" LastEditorUserId="2444" LastEditDate="2020-03-16T23:12:38.673" LastActivityDate="2020-03-18T16:20:19.373" Title="How to prove that gradient descent doesn't necessarily find the global optimum?" Tags="&lt;gradient-descent&gt;&lt;proofs&gt;&lt;learning-rate&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18659" PostTypeId="1" CreationDate="2020-03-16T02:59:40.690" Score="4" ViewCount="85" Body="&lt;p&gt;My aim is to train a model for predicting diseases. Now, according to &lt;a href=&quot;https://en.wikipedia.org/wiki/Disease#Classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia article&lt;/a&gt;, diseases are classified based on the following criteria in general:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Causes (of the disease)&lt;/li&gt;&#xA;&lt;li&gt;Pathogenesis (the mechanism by which the disease progresses)&lt;/li&gt;&#xA;&lt;li&gt;Age&lt;/li&gt;&#xA;&lt;li&gt;Gender&lt;/li&gt;&#xA;&lt;li&gt;Symptoms (of the disease)&lt;/li&gt;&#xA;&lt;li&gt;Damage (caused by the disease)&lt;/li&gt;&#xA;&lt;li&gt;Organ type (e.g. heart disease, liver disease, etc.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Are these features used for predicting diseases universally (i.e. all types of diseases)? I don't think so. There can be other attributes as well. For example, traveling in the case of coronavirus.&lt;/p&gt;&#xA;&lt;p&gt;So, are there better features for predicting diseases?&#xA;Or which ones among them are better than the others, when patients specify their health issues?&lt;/p&gt;&#xA;" OwnerUserId="34306" LastEditorUserId="2444" LastEditDate="2020-11-19T13:03:37.960" LastActivityDate="2020-11-19T13:03:37.960" Title="How should I select the features for predicting diseases (in particular when patients specify their health issues)?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;prediction&gt;&lt;feature-selection&gt;&lt;healthcare&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18660" PostTypeId="1" CreationDate="2020-03-16T09:48:20.373" Score="2" ViewCount="44" Body="&lt;p&gt;I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an &quot;old&quot; paper &lt;a href=&quot;https://pdfs.semanticscholar.org/3a09/c6f3873dd0327a3f9d83b7ff1c8b43a06cc3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An algorithm for correcting mislabeled data&lt;/a&gt; (2001) by Xinchuan Zeng et al. Please share if you are aware of recent/current updates with a brief thoughts. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="34290" LastEditorUserId="34290" LastEditDate="2020-03-17T15:56:59.603" LastActivityDate="2020-03-17T15:56:59.603" Title="Recent algorithms for correcting mislabeled data using multilayer perceptrons" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;multilayer-perceptron&gt;&lt;multi-label-classification&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18661" PostTypeId="2" ParentId="18645" CreationDate="2020-03-16T09:49:22.143" Score="4" Body="&lt;p&gt;This is mostly because humans already have information when they start learning the game (priors) that makes them learn it more quickly. We already know to jump on monsters or avoid them or to get gold looking object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you remove these priors you can see a human is worse at learning these games. (&lt;a href=&quot;https://arxiv.org/pdf/1802.10217.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some experiments they tried in the study to remove these priors where replacing all notable objects with colored squares, reversing and scrambling controls, changing gravity, and generally replacing all sprites with random pixels. All these experiments made human learning much harder, and increased death rate, time needed and states visited in the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we want the reinforcement learning algorithm to perform as good as a human, we will have to somehow include these priors we have as a human before training the network. This of course is not yet done (as far as I know)&lt;/p&gt;&#xA;" OwnerUserId="29671" LastEditorUserId="29671" LastEditDate="2020-03-16T10:54:00.087" LastActivityDate="2020-03-16T10:54:00.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18662" PostTypeId="1" CreationDate="2020-03-16T11:58:22.280" Score="1" ViewCount="89" Body="&lt;p&gt;From this video tutorial &lt;a href=&quot;https://www.youtube.com/watch?v=JIWXbzRXk1I&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vanishing Gradient Tutorial&lt;/a&gt;, the sigmoid function and the hyperbolic tangent can produce the vanishing gradient problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other activation functions can lead to the vanishing gradient problem?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-16T14:42:50.670" LastActivityDate="2020-03-16T14:42:50.670" Title="Which activation functions can lead to the vanishing gradient problem?" Tags="&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;activation-function&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18663" PostTypeId="1" AcceptedAnswerId="18669" CreationDate="2020-03-16T11:59:51.413" Score="4" ViewCount="3072" Body="&lt;p&gt;I was recently asked at an interview to calculate the number of parameters for a convolutional layer. I am deeply ashamed to admit I didn't know how to do that, even though I've been working and using CNN for years now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a convolutional layer with ten &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; filters and an input of shape &lt;span class=&quot;math-container&quot;&gt;$24 \times 24 \times 3$&lt;/span&gt;, what is the total number of parameters of this convolutional layer?&lt;/p&gt;&#xA;" OwnerUserId="32528" LastEditorUserId="2444" LastEditDate="2020-03-17T00:13:56.083" LastActivityDate="2020-03-17T01:01:29.327" Title="How to calculate the number of parameters of a convolutional layer?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18668" PostTypeId="1" CreationDate="2020-03-16T14:07:59.860" Score="2" ViewCount="65" Body="&lt;p&gt;I want to use a neural network to find correlated columns in a &lt;code&gt;.csv&lt;/code&gt; file and give them as a output. The input &lt;code&gt;.csv&lt;/code&gt; file has multiple columns with 0 and 1 ( like Booleans) in it. The file got the assignment from people to interests in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example &lt;code&gt;.csv&lt;/code&gt; input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;UserID   History  Math  Physics  Art  Music  ...&#xA;User1    0        1     1        0    0      ...&#xA;User2    0        0     0        1    1      ...&#xA;User3    0        1     1        1    1      ...&#xA;User4    1        0     1        1    0      ...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output should be in this case something like: {math,physics}, {art,music}, {history,physics,art} - I exclude here {math,physics,art,music} because in a step afterwards i want to exclude (at least some) which can be created through the combination of others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I have a problem that i don´t know which type of neural network could complete this task. How can I solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the important thing, that a column can have more than one column it correlates to - so its not like simple k-means clustering (as far as I understand it).&lt;/p&gt;&#xA;" OwnerUserId="34313" LastEditorUserId="9947" LastEditDate="2020-03-17T09:58:11.203" LastActivityDate="2020-03-17T09:58:11.203" Title="Neural network to extract correlated columns" Tags="&lt;neural-networks&gt;&lt;clustering&gt;&lt;multi-label-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18669" PostTypeId="2" ParentId="18663" CreationDate="2020-03-16T14:29:14.690" Score="4" Body="&lt;h2&gt;What are the parameters in a convolutional layer?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The (learnable) parameters of a convolutional layer are the elements of the kernels (or filters) and biases (if you decide to have them). There are 1d, 2d and 3d convolutions. The most common are 2d convolutions, which are the ones people usually refer to, so I will mainly focus on this case.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2d convolutions&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If the 2d convolutional layer has &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; filters of &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; shape and the input to the convolutional layer is &lt;span class=&quot;math-container&quot;&gt;$24 \times 24 \times 3$&lt;/span&gt;, then this actually means that the filters will have shape &lt;span class=&quot;math-container&quot;&gt;$3 \times 3 \times 3$&lt;/span&gt;, i.e. each filter will have the 3rd dimension that is equal to the 3rd dimension of the input. So, the 3rd dimension of the kernel is not given because it can be determined from the 3rd dimension of the input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2d convolutions are performed along only 2 axes (x and y), hence the name. Here's a picture of a typical 2d convolutional layer where the depth of the kernel (in orange) is equal to the depth of the input volume (in cyan).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3folL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3folL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each kernel can optionally have an associated scalar bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, you should already be able to calculate the number of parameters of a standard convolutional layer.  In your case, the number of parameters is &lt;span class=&quot;math-container&quot;&gt;$10 * (3*3*3) + 10 = 280$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;A TensorFlow proof&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The following simple TensorFlow (version 2) program can confirm this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;&#xA;def get_model(input_shape, num_classes=10):&#xA;    model = tf.keras.Sequential()&#xA;    model.add(tf.keras.layers.Input(shape=input_shape))&#xA;    model.add(tf.keras.layers.Conv2D(10, kernel_size=3, use_bias=True))&#xA;    model.add(tf.keras.layers.Flatten())&#xA;    model.add(tf.keras.layers.Dense(num_classes))&#xA;&#xA;    model.summary()&#xA;&#xA;    return model&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    input_shape = (24, 24, 3)&#xA;    get_model(input_shape)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should try setting &lt;code&gt;use_bias&lt;/code&gt; to &lt;code&gt;False&lt;/code&gt; to understand how the number of parameters changes.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;General case&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;So, in general, given &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; filters of shape &lt;span class=&quot;math-container&quot;&gt;$K \times K$&lt;/span&gt; and an input of shape &lt;span class=&quot;math-container&quot;&gt;$H \times W \times D$&lt;/span&gt;, then the number of parameters of the standard 2d convolutional layer, with scalar biases, is &lt;span class=&quot;math-container&quot;&gt;$M * (K * K * D) + M$&lt;/span&gt; and, without biases, is &lt;span class=&quot;math-container&quot;&gt;$M * (K * K * D)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also these related questions &lt;a href=&quot;https://ai.stackexchange.com/q/17783/2444&quot;&gt;How is the depth of filters of hidden layers determined?&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/5769/2444&quot;&gt;In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1d and 3d convolutions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are also 1d and 3d convolutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the case of 3d convolutions, the kernels may not have the same dimension as the depth of the input, so the number of parameters is calculated differently for 3d convolutional layers. Here's a diagram of 3d convolutional layer, where the kernel has a depth different than the depth of the input volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/F282Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F282Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. &lt;a href=&quot;https://stackoverflow.com/q/42883547/3924118&quot;&gt;Intuitive understanding of 1D, 2D, and 3D convolutions in convolutional neural networks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-17T01:01:29.327" LastActivityDate="2020-03-17T01:01:29.327" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18670" PostTypeId="2" ParentId="18663" CreationDate="2020-03-16T14:29:25.923" Score="3" Body="&lt;p&gt;For a standard convolution layer, the weight matrix will have a shape of (out_channels, in_channels, kernel_sizes*) in addition you will need a vector of shape [out_channels] for biases. For your specific case, 2d, your weight matrix will have a shape of (out_channels, in_channels, kernel_size[0], kernel_size[1]). Now if we plugin the numbers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;out_channels = 10, you're having 10 filters&lt;/li&gt;&#xA;&lt;li&gt;in_channels = 3 the picture is RGB in this case so there are 3 channels (the last dimension of the input)&lt;/li&gt;&#xA;&lt;li&gt;kernel_size[0] = kernel_size[1] = 3&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In total you're gonna have 10*3*3*3 + 10 = 280 parameters.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2020-03-16T14:58:20.167" LastActivityDate="2020-03-16T14:58:20.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18671" PostTypeId="1" AcceptedAnswerId="18788" CreationDate="2020-03-16T15:09:51.407" Score="4" ViewCount="130" Body="&lt;p&gt;I'm looking to develop a machine translation tool for a constructed language. I think that the example-based approach is the most suitable because the said language is very regular and I can have a sufficient amount of parallel translations.&lt;/p&gt;&#xA;&lt;p&gt;I already know the overall idea behind the example-based machine translation (EBMT) approach, but I can't find any resource that describes a naive EBMT algorithm (or model) that would allow me to easily implement it.&lt;/p&gt;&#xA;&lt;p&gt;So, I'm looking for either:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a detailed description,&lt;/li&gt;&#xA;&lt;li&gt;pseudocode or&lt;/li&gt;&#xA;&lt;li&gt;a sufficiently clear open-source project (maybe a GitHub one)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;of a naive EBMT algorithm. So, I'm not looking for a software library that implements this, but I'm looking for a resource that explains/describes in detail a naive/simple EBMT algorithm, so that I am able to implement it.&lt;/p&gt;&#xA;&lt;p&gt;Note that there are probably dozens of variations of EBMT algorithms. I'm only looking for the most naive/simple one.&lt;/p&gt;&#xA;&lt;p&gt;I have already looked at the project &lt;a href=&quot;https://github.com/proycon/pbmbmt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phrase-based Memory-based Machine Translator&lt;/a&gt;, but, unfortunately, it is not purely based on examples but also statistical, i.e. it needs an alignment file generated by, for example, Giza++ or Moses.&lt;/p&gt;&#xA;" OwnerUserId="34314" LastEditorUserId="2444" LastEditDate="2021-02-03T16:53:33.083" LastActivityDate="2021-02-03T16:53:33.083" Title="Is there any resource that describes in detail a naive example-based machine translation algorithm?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;resource-request&gt;&lt;machine-translation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18672" PostTypeId="1" CreationDate="2020-03-16T15:17:47.737" Score="0" ViewCount="33" Body="&lt;p&gt;Deep blue is good at chess, but is more &quot;hand-coded&quot; or &quot;top-down&quot;.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaGoZero is &quot;self-taught&quot;, and at Go is very much super-human.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_Zero&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/AlphaGo_Zero&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does AlphaGoZero fare when it goes head-to-head with DeepBlue?  Are there indicators like chess ratings?&lt;/p&gt;&#xA;" OwnerUserId="2263" LastActivityDate="2020-03-16T15:17:47.737" Title="How does (or should) AlphaGoZero (which does chess) fare against Deep Blue?" Tags="&lt;chess&gt;&lt;alphago-zero&gt;&lt;deep-blue&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18673" PostTypeId="2" ParentId="18431" CreationDate="2020-03-16T15:59:25.310" Score="0" Body="&lt;p&gt;If you are willing to take an evolutionary approach, you may employ the NEAT algorithm (Neural Evolution of Augmenting Topologies) to train your bot. It will take some work setting it up and all, but it then will gradually improve over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://gekkoquant.com/2016/03/13/evolving-neural-networks-through-augmenting-topologies-part-1-of-4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://gekkoquant.com/2016/03/13/evolving-neural-networks-through-augmenting-topologies-part-1-of-4/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=WiPZSieT6qs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=WiPZSieT6qs&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That should be enough to pique your interests and get you started. That last link links to a number of NEAT implementations available in a number of languages.&lt;/p&gt;&#xA;" OwnerUserId="24092" LastActivityDate="2020-03-16T15:59:25.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18674" PostTypeId="1" CreationDate="2020-03-16T16:33:37.083" Score="3" ViewCount="55" Body="&lt;p&gt;If you have an &lt;span class=&quot;math-container&quot;&gt;$18$&lt;/span&gt; layer residual network versus and a &lt;span class=&quot;math-container&quot;&gt;$32$&lt;/span&gt; layer residual network, why would the former do better at object detection than the latter, if you have both models are training using the same training data?&lt;/p&gt;&#xA;" OwnerUserId="34317" LastEditorUserId="2444" LastEditDate="2020-03-16T16:36:59.697" LastActivityDate="2020-03-16T22:03:46.120" Title="Do deeper residual networks perform better or worse?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18675" PostTypeId="2" ParentId="18674" CreationDate="2020-03-16T22:03:46.120" Score="1" Body="&lt;p&gt;Just by having more parameters, the deeper model has a &lt;strong&gt;higher capacity&lt;/strong&gt; than the smaller one. This means that theoretically it can learn to extract more complex features from the data. Additionally, more layers means that the model can extract even &lt;strong&gt;higher-level features&lt;/strong&gt; from the data.&#xA;So, generally speaking, deeper models will most of the times &lt;strong&gt;outperform&lt;/strong&gt; shallow ones for more difficult tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside is that if you have a small amount of data, a high-capacity model has the ability of memorizing the training set, which would lead to &lt;strong&gt;overfitting&lt;/strong&gt;. &#xA;Besides performance, deeper models require better hardware and larger training times. So, there are plenty of reasons for one to prefer a more shallow model to a deeper one.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-03-16T22:03:46.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18676" PostTypeId="2" ParentId="18605" CreationDate="2020-03-16T22:35:27.053" Score="1" Body="&lt;p&gt;The blog post &lt;a href=&quot;https://ruder.io/unsupervised-cross-lingual-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised Cross-lingual Representation Learning&lt;/a&gt; (2019), the related &lt;a href=&quot;https://www.aclweb.org/anthology/P19-4007.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and &lt;a href=&quot;https://docs.google.com/presentation/d/1miPu0hkBFSRTAxP-Psg0xXOvAbVSJPhQXehMQdUI88I/edit#slide=id.g5cacfdb6b8_1_137&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; by Sebastian Ruder (a researcher currently at DeepMind) summarize what you are looking for. In fact, the authors write&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We will introduce researchers to &lt;strong&gt;state-of-the-art&lt;/strong&gt; methods for constructing resource-light cross-lingual word representations and discuss their applicability in a broad range of downstream NLP applications, covering bilingual lexicon induction, machine translation (both neural and phrase-based), dialogue, and information retrieval tasks. We will deliver a detailed survey of the &lt;strong&gt;current cutting-edge methods&lt;/strong&gt;, discuss best training and evaluation practices and usecases, and provide links to publicly available implementations, datasets, and pretrained models and word embedding collections.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-03-16T22:35:27.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18677" PostTypeId="2" ParentId="18658" CreationDate="2020-03-16T22:53:52.757" Score="2" Body="&lt;p&gt;You can find by yourself a counterexample that, in general, GD is not guaranteed to find the global optimum!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first advise you to choose a simpler function (than the one you are showing), with 2-3 optima, where one is the global and the other(s) are local. You don't need neural networks or any other ML concept to show this, but only basic calculus (derivatives) and numerical methods (i.e. gradient descent). Just choose a very simple function with more than one optimum and apply the basic gradient descent algorithm. Then you can see that, if you start gradient descent close to one &lt;strong&gt;local&lt;/strong&gt; optimum (i.e. you choose an initial value for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, depending on your notation for the variable of the function) and then you apply gradient descent (for some iterations), you will end up in that close local optimum, from which you cannot escape, after having applied the gradient descent steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the question &lt;a href=&quot;https://datascience.stackexchange.com/q/24534/10640&quot;&gt;Does gradient descent always converge to an optimum?&lt;/a&gt; and &lt;a href=&quot;https://stats.stackexchange.com/q/367397/82135&quot;&gt;For convex problems, does gradient in Stochastic Gradient Descent (SGD) always point at the global extreme value?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-16T23:00:51.580" LastActivityDate="2020-03-16T23:00:51.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18681" PostTypeId="1" CreationDate="2020-03-17T06:58:22.330" Score="2" ViewCount="160" Body="&lt;p&gt;I'm trying to develop a multistep forecasting model using LSTM  Network. The model takes three times steps as input and predicting two time_steps. both input and output columns are normalised using minmax_scalar within the range of 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please see the below model architecture&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Model Architecture&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(LSTM(80,input_shape=(3,1),activation='sigmoid',return_sequences=True))&#xA;model.add(LSTM(20,activation='sigmoid',return_sequences=False))&#xA;model.add(Dense(2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, using sigmoid as an activation function is it correct? &lt;/p&gt;&#xA;" OwnerUserId="24006" LastActivityDate="2020-03-17T10:48:04.377" Title="Using sigmoid in LSTM network for multi-step forecasting" Tags="&lt;long-short-term-memory&gt;&lt;activation-function&gt;&lt;regression&gt;&lt;forecasting&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18682" PostTypeId="1" AcceptedAnswerId="21396" CreationDate="2020-03-17T08:49:36.980" Score="5" ViewCount="250" Body="&lt;p&gt;From my understanding of the REINFORCE policy gradient method, we gently nudge the probabilities of actions based on the advantages. More specifically, the positive advantages &lt;strong&gt;increase&lt;/strong&gt; the probabilities, negative advantages &lt;strong&gt;reduce&lt;/strong&gt; the probabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how do we compute the advantages given the &lt;strong&gt;real discounted rewards&lt;/strong&gt; (aggregated rewards from the episode) and a &lt;strong&gt;policy network&lt;/strong&gt; that only outputs the probabilities of actions?&lt;/p&gt;&#xA;" OwnerUserId="27366" LastEditorUserId="2444" LastEditDate="2020-03-17T22:27:12.840" LastActivityDate="2020-05-22T09:22:18.367" Title="How to calculate the advantage in policy gradient functions?" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18683" PostTypeId="2" ParentId="18681" CreationDate="2020-03-17T08:57:06.647" Score="1" Body="&lt;p&gt;Yes, due the input, output being constrained between zero and one that would be the only viable activation function. &lt;/p&gt;&#xA;" OwnerUserId="34324" LastActivityDate="2020-03-17T08:57:06.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18684" PostTypeId="2" ParentId="18681" CreationDate="2020-03-17T09:00:34.040" Score="1" Body="&lt;p&gt;You have a problem in your code, you want to use &quot;sigmoid&quot; in the last layer. Fot the code you are showin you are using linear activation in the last layer. &lt;/p&gt;&#xA;" OwnerUserId="32493" LastActivityDate="2020-03-17T09:00:34.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18685" PostTypeId="1" AcceptedAnswerId="18694" CreationDate="2020-03-17T10:28:29.667" Score="3" ViewCount="132" Body="&lt;p&gt;I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best capture of graffiti and its geolocation, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to use a ResNet50 model pre-trained on ImageNet using transfer learning for my graffiti image dataset. The classification will be done on an edge device as network connectivity may not be reliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have a series of frames that have been detected to contain graffiti, as the vehicle goes past it, but I only need to report one image (so not all frames containing graffiti in the series). How can I do that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like to report the frame where the camera is perpendicular to the wall. Why perpendicular? I think that images containing the graffiti when the camera is perpendicular to the wall will more clearly show the graffiti.&lt;/p&gt;&#xA;" OwnerUserId="30087" LastEditorUserId="2444" LastEditDate="2020-03-23T02:11:15.190" LastActivityDate="2020-03-23T02:11:15.190" Title="How can I detect the frame from video streaming that contains a graffiti on city wall?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;transfer-learning&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18686" PostTypeId="2" ParentId="18681" CreationDate="2020-03-17T10:48:04.377" Score="1" Body="&lt;p&gt;You should not limit yourself to sigmoid as activation function on the last layer. Usually you're normalizing your dataset, but when you're testing/evaluating the model you're applying the inverse of the scaling transformation to the predictions, so you could easily use tanh which is defined on [-1, 1] &lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2020-03-17T10:48:04.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18687" PostTypeId="2" ParentId="18658" CreationDate="2020-03-17T11:20:13.167" Score="1" Body="&lt;p&gt;Well, GD terminates once the gradients are 0, right? Now, in a non-convex function, there could be some points, which do not belong to the global minima, and yet, have 0 gradients. For example, such points can belong to saddle points and local minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this picture and say you start GD at the x label.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vmK8H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vmK8H.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GD will bring you the flat area and will stop making progress there as gradients are 0. However, as you can see, global minima is to the left of this flat region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the same token, you have to show, for your own function, that there exists at least a single point whose gradients are 0 and yet, it is not the global minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to that, the guarantee on converge for convex functions depends on annealing the learning rate appropriately. For example, if your LR is too high, GD can just keep overshooting the minima. The visualization from &lt;a href=&quot;https://deeplearning.ai/ai-notes/optimization/?utm_campaign=BlogAINotesOptimizationNovember122019&amp;amp;utm_content=105388620&amp;amp;utm_medium=social&amp;amp;utm_source=linkedin&amp;amp;hss_channel=lcp-18246783&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; page might help you to understand more regarding the behavior of GD.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="32621" LastEditDate="2020-03-18T16:20:19.373" LastActivityDate="2020-03-18T16:20:19.373" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18692" PostTypeId="1" CreationDate="2020-03-17T12:49:49.053" Score="3" ViewCount="150" Body="&lt;p&gt;I have an environment where an agent faces an equal opponent, and while I've achieved OK performance implementing DQN and treating the opponent as a part of the environment, I think performance would improve if the agent trains against itself iteratively. I've seen posts about it, but never detailed implementation notes. My thoughts were to implement the following (agent and opponent are separate networks for now):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Bootstrap agent and opponent with initial weights (either random or trained against CPU, not sure)&lt;/li&gt;&#xA;&lt;li&gt;Use Annealing Epsilon Greedy strategy for N iterations&lt;/li&gt;&#xA;&lt;li&gt;After M iterations (M &gt; N), copy agent network's weights to opponent's network&lt;/li&gt;&#xA;&lt;li&gt;Reset annealing epsilon (i.e. start performing randomly again to explore new opponent)?&lt;/li&gt;&#xA;&lt;li&gt;Repeat steps 2-4&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Would something like this work? Some specific questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should I &quot;reset&quot; my annealing epsilon strategy every time the opponent is updated? I feel like this is needed because the agent needs sufficient time to explore new strategies for this &quot;new&quot; opponent.&lt;/li&gt;&#xA;&lt;li&gt;Should the experience replay buffer be cleared out when the opponent is updated? Again, I think this is needed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any pointers would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="34328" LastActivityDate="2020-03-17T12:49:49.053" Title="How to correctly implement self-play with DQN?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;self-play&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18693" PostTypeId="1" CreationDate="2020-03-17T14:58:40.440" Score="2" ViewCount="90" Body="&lt;p&gt;I'm trying to train a Policy Gradient Agent with Baseline for my RL research. I'm using the in-built RL toolbox from MATLAB (&lt;a href=&quot;https://www.mathworks.com/help/reinforcement-learning/ug/pg-agents.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/help/reinforcement-learning/ug/pg-agents.html&lt;/a&gt;) and have created my own Environment. The goal is to train the system to sample an underlying time-series (&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;) given battery constrains (&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is battery cost). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general setup is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My Environment is a &quot;sensor&quot; system with exogenous input time-series and battery level as my States/Observations (size is 13x1). &lt;/li&gt;&#xA;&lt;li&gt;Actions &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; are binary: 0 = keep a model prediction &lt;span class=&quot;math-container&quot;&gt;$(\hat x)$&lt;/span&gt;; 1 = sampling time series &lt;span class=&quot;math-container&quot;&gt;$(x)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Reward function is&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R = -[err(\tilde x, x) + A_t\cdot \epsilon ] + (-100)\cdot T_1 + (100) \cdot T_2 &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$err(\tilde x, x)$&lt;/span&gt; is the RMSE error between the sampled time series &lt;span class=&quot;math-container&quot;&gt;$(\tilde x)$&lt;/span&gt;, and true time series x.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The Terminal State Rewards are -100 if sensor runs out of battery &lt;span class=&quot;math-container&quot;&gt;$T_1$&lt;/span&gt; or 100 if reached the end of the episode with RMSE &amp;lt; threshold and remaining battery level &lt;span class=&quot;math-container&quot;&gt;$(T_2)$&lt;/span&gt;. The goal is to always end in &lt;span class=&quot;math-container&quot;&gt;$T_2$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Each training Episode consists of a time-series of random length, and random initial battery level.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My current setup is using mostly default RL setups from MALTAB with learning rate of &lt;span class=&quot;math-container&quot;&gt;$10^{-4}$&lt;/span&gt; and ADAM optimizer. The training is slow, and shows a lot of Reward oscillation between the two terminal states. MATLAB RL toolbox also outputs a &lt;span class=&quot;math-container&quot;&gt;$Q_0$&lt;/span&gt; value which the state is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Episode Q0 is the estimate of the discounted long-term reward at the&#xA;  start of each episode, given the initial observation of the&#xA;  environment.  As training progresses, Episode Q0 should approach the&#xA;  true discounted  long-term reward if the critic is well-designed,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YQKNI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YQKNI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is my training and episodes too random? i.e., time-series of different lengths and random initial sensor setup. &lt;/li&gt;&#xA;&lt;li&gt;Should I simplify my reward function to be just &lt;span class=&quot;math-container&quot;&gt;$T_2$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Why doesn't &lt;span class=&quot;math-container&quot;&gt;$Q_0$&lt;/span&gt; change at all?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="34330" LastEditorUserId="34330" LastEditDate="2020-03-19T21:48:40.267" LastActivityDate="2020-03-19T21:48:40.267" Title="Policy Gradient Reward Oscillation in MATLAB" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18694" PostTypeId="2" ParentId="18685" CreationDate="2020-03-17T16:15:39.477" Score="2" Body="&lt;p&gt;ResNet is an architecture for object recognition and you may use it to do your classification task.  Fast RCNN may improve your results but is a more difficult architecture to implement.  If you want to go in this direction the best place to start is the arxiv paper of the Fast R-CNN (arxiv.org/abs/1504.08083).  If I am not wrong, there is an implementation of fast-rcnn in pytorch if you really want that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to be new to deep learning.  I would strongly recommend starting simple. ResNet-50 will probably be more than enough for your application. Moreover, I suggest using a lib like fast.ai.&lt;/p&gt;&#xA;" OwnerUserId="34019" LastActivityDate="2020-03-17T16:15:39.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18695" PostTypeId="2" ParentId="5904" CreationDate="2020-03-17T16:43:01.340" Score="0" Body="&lt;p&gt;I may have scratched the surface of a much larger problem when I asked this question. In the meantime I have read Lottery Hypothesis paper: &lt;a href=&quot;https://arxiv.org/pdf/1803.03635.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1803.03635.pdf&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, if you overparameterise your network you are more likely to find a random initialisation that performs well: A winning ticket. The paper above shows that you can actually prune away the unneeded parts of the network after training. However, you need to overparameterise the network initially in order to increase the chance of randomly sampling a winning ticket configuration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the case in my question above is a minimal example of this. &lt;/p&gt;&#xA;" OwnerUserId="14789" LastActivityDate="2020-03-17T16:43:01.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18696" PostTypeId="2" ParentId="18658" CreationDate="2020-03-17T17:51:54.740" Score="0" Body="&lt;p&gt;There is no way you can be sure you have reached a global minimum. Steepest descent will converge toward where the gradient approaches zero. Depending on the initial conditions ( ie the initial values of the weights) you can and will converge on some minimum. Notice if you run your model several times with random weight initialization you will get slightly different results.  What I do find interesting is that it seems that in general the local minima have roughly the same  value. The cost function is some kind of surface in N space where N is the number of trainable parameters. We do not know what that surface is like and how many local minimums  exist.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-17T17:51:54.740" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18697" PostTypeId="1" CreationDate="2020-03-17T18:51:19.347" Score="1" ViewCount="268" Body="&lt;p&gt;Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would be left over for each neuron in the last output layer but my program returns a 1x4 vector.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Network:&#xA;    def __init__(self, inputs, layers):&#xA;        # each neuron has n weights attached&#xA;        # each neuron has a bias&#xA;        self.weights = []&#xA;        self.biases = [np.random.randn(y,1) for y in layers[1:]]&#xA;        # did this to connect neurons excluding input layer and last neuron&#xA;        # because last neuron isn't connecting anything in front&#xA;        for x,y in zip(layers[:-1], layers[1:]):&#xA;            self.weights.append(np.random.randn(y,x))&#xA;&#xA;    # add bias vector to previous vector matrix product&#xA;    def forward(self, inputs):&#xA;        for b,w in zip(self.biases, self.weights):&#xA;            inputs = sigmoid(np.dot(w, inputs) + b)&#xA;            print(&quot;layer&quot;,inputs)&#xA;        self.output = inputs&#xA;        print(&quot;output&quot;,self.output)&#xA;&#xA;&#xA;input = [random.randrange(256),random.randrange(256),&#xA;                random.randrange(256)]&#xA;&#xA;net = Network(input, [3,4,4,1])&#xA;net.forward(input)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;layer [[7.80671510e-176 2.86955013e-159 1.00000000e+000 9.10681919e-004]&#xA; [8.91574010e-176 3.27719954e-159 1.00000000e+000 1.03991921e-003]&#xA; [3.44313504e-176 1.26560896e-159 1.00000000e+000 4.01858875e-004]&#xA; [1.43702955e-175 5.28215553e-159 1.00000000e+000 1.67506508e-003]]&#xA;layer [[0.79051612 0.79051612 0.91469971 0.79084515]&#xA; [0.80073915 0.80073915 0.70932703 0.80058277]&#xA; [0.35187686 0.35187686 0.00982275 0.35113385]&#xA; [0.51881022 0.51881022 0.55334083 0.5188783 ]]&#xA;layer [[0.26291072 0.26291072 0.16592316 0.26264314]]&#xA;output [[0.26291072 0.26291072 0.16592316 0.26264314]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If my output is supposed to decide whether the neuron fires, how could I use the vector to determine this?&lt;/p&gt;&#xA;" OwnerUserId="34335" LastActivityDate="2021-04-11T23:06:55.700" Title="In a single neuron output layer should the output be a scalar?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18698" PostTypeId="2" ParentId="18697" CreationDate="2020-03-17T20:37:01.240" Score="0" Body="&lt;p&gt;Look, your code says your network has many outputs.&#xA;Look at the two lines below. This two lines says the output depends on the dimension of &lt;code&gt;np.dot(w, inputs)&lt;/code&gt;. In your case it's 4 diminutional vector. And in the last line you are assigning them as output. You can write &lt;code&gt;self.output = sigmoid(np.dot(new_weihts, inputs))&lt;/code&gt; instead of &lt;code&gt;self.output = inputs&lt;/code&gt;. Must ensure that new_weights is a vector and has same shape as previous layer's output.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inputs = sigmoid(np.dot(w, inputs) + b)&#xA;self.output = inputs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: Your network don't look like a 4 layers network. It looks like a one layer network with 4 unit of neutrons.&lt;/p&gt;&#xA;" OwnerUserId="34164" LastActivityDate="2020-03-17T20:37:01.240" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18699" PostTypeId="1" CreationDate="2020-03-17T22:29:19.470" Score="2" ViewCount="30" Body="&lt;p&gt;I already trained a deep neural network called YOLO (You Only Look Once) with high-quality images (1920 by 1080 pixels) for a detection task. The result for mAP and IOU were 93% and 89% respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to decrease the quality of my training data set using some available filters, then I use those low-quality images along with high-quality images to train the network again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this method increase the accuracy (or, in general, performance) of the deep neural network (for a detection task)? Like mAP and IOU?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data set is vehicle images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;mAP:&lt;/strong&gt; mean average precision&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;IOU:&lt;/strong&gt; intersection over union ( or overlap)&lt;/p&gt;&#xA;" OwnerUserId="34339" LastEditorUserId="34339" LastEditDate="2020-03-17T23:06:51.573" LastActivityDate="2020-03-17T23:06:51.573" Title="Can the addition of low-quality images to the training dataset increase the network performance?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;object-detection&gt;&lt;data-preprocessing&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18701" PostTypeId="1" CreationDate="2020-03-18T02:35:49.450" Score="2" ViewCount="375" Body="&lt;p&gt;The ongoing &lt;a href=&quot;https://en.wikipedia.org/wiki/2019%E2%80%9320_coronavirus_pandemic&quot; rel=&quot;nofollow noreferrer&quot;&gt;coronavirus pandemic&lt;/a&gt; of coronavirus disease 2019 (COVID-19), caused by &lt;a href=&quot;https://en.wikipedia.org/wiki/Severe_acute_respiratory_syndrome_coronavirus_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)&lt;/a&gt;, as of 29 September 2020, has affected many countries and territories, with more than 33.4 million cases of COVID-19 have been reported and more than 1 million people have died. The live statistics can be found at &lt;a href=&quot;https://www.worldometers.info/coronavirus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.worldometers.info/coronavirus/&lt;/a&gt; or in the &lt;a href=&quot;https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Health Organization (WHO) site&lt;/a&gt;. Although countries have already started quarantines and have adopted extreme countermeasures (such as closing restaurants or forbidding events with multiple people), the numbers of cases and deaths will probably still increase in the next weeks.&lt;/p&gt;&#xA;&lt;p&gt;Given that this pandemic concerns all of us, including people interested in AI, such as myself, it may be useful to share information about the possible current applications of AI to slow down the spread of SARS-CoV-2, to help infected people or people in the healthcare sector that have been uninterruptedly working for hours to attempt to save more lives, while putting at risk their own.&lt;/p&gt;&#xA;&lt;p&gt;What are the existing AI technologies (e.g. computer vision or robotics tools) that are already being used to tackle these issues, such as slowing down the spread of SARS-CoV-2 or helping infected people?&lt;/p&gt;&#xA;&lt;p&gt;I am looking for references that prove that the mentioned technologies are really being used. I am not looking for potential AI technologies (i.e. research work) that could potentially be helpful. Furthermore, I am &lt;strong&gt;not&lt;/strong&gt; looking for data analysis tools (e.g. sites that show the evolution of the spread of coronavirus, etc.)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-29T22:06:31.803" LastActivityDate="2020-09-30T19:56:46.923" Title="What are the AI technologies currently used to fight the coronavirus pandemic?" Tags="&lt;applications&gt;&lt;social&gt;&lt;healthcare&gt;&lt;real-world&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18702" PostTypeId="1" CreationDate="2020-03-18T02:49:38.913" Score="2" ViewCount="54" Body="&lt;p&gt;I'm trying to find out how AI can help with efficient customer service, in fact call routing to the right agent. My usecase is given context of a query from a customer and agents' expertise, how can we do the matching?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, how is this problem solved? What sub-topic within AI is suitable for this problems? Classification, recommender systems, ...? Any pointers to open-source projects would be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-05-14T21:57:53.830" Title="Using AI to enhance customer service" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;recommender-system&gt;&lt;efficiency&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18703" PostTypeId="1" AcceptedAnswerId="18704" CreationDate="2020-03-18T07:52:23.217" Score="2" ViewCount="186" Body="&lt;p&gt;I often see blog posts or &lt;a href=&quot;https://ai.stackexchange.com/questions/17764/why-do-resnets-avoid-the-vanishing-gradient-problem&quot;&gt;questions on here&lt;/a&gt; starting with the premise that ResNets solve the vanishing gradient problem.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1512.03385v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original 2015 paper&lt;/a&gt; contains the following passage in section 4.1:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We argue that this optimization difficulty is unlikely to&#xA;be caused by vanishing gradients. These plain networks are&#xA;trained with BN, which ensures forward propagated&#xA;signals to have non-zero variances. We also verify that the&#xA;backward propagated gradients exhibit healthy norms with&#xA;BN. So neither forward nor backward signals vanish. In&#xA;fact, the 34-layer plain net is still able to achieve competitive accuracy, suggesting that the solver works&#xA;to some extent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So what's happened since then? I feel like either it became a misconception that ResNets solve the vanishing gradient problem (because it does indeed feel like a sensible explanation that one would readily accept and continue to propagate), or some paper has since proven that this is indeed the case.&lt;/p&gt;&#xA;&lt;p&gt;I'm starting with the initial knowledge that it's &amp;quot;easier&amp;quot; to learn the residual mapping for a convolutional block than it is to learn the whole mapping. So my question is on the level of: why is it &amp;quot;easier&amp;quot;? And why does the &amp;quot;plain network&amp;quot; do such a good job but then struggle to close the gap to the performance of ResNet. Supposedly if the plain network has already learned reasonably good mappings, then all it has left to learn to close the gap is &amp;quot;residual&amp;quot;. But it just isn't able to.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="2444" LastEditDate="2021-01-17T14:11:41.917" LastActivityDate="2021-01-17T14:11:41.917" Title="If vanishing gradients are NOT the problem that ResNets solve, then what is the explanation behind ResNet success?" Tags="&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;residual-networks&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18704" PostTypeId="2" ParentId="18703" CreationDate="2020-03-18T08:53:09.867" Score="3" Body="&lt;p&gt;They explained in the paper why they introduce residual blocks. They argue that it's easier to learn residual functions &lt;span class=&quot;math-container&quot;&gt;$F(x) = H(x) - x$&lt;/span&gt; and then add them to the original representation &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to get hidden representation &lt;span class=&quot;math-container&quot;&gt;$H(x) = F(x) + x$&lt;/span&gt; than it is to learn hidden representation &lt;span class=&quot;math-container&quot;&gt;$H(x)$&lt;/span&gt; directly from original representation. That's the main reason and empirical results show that they might be right. Better gradient propagation might be an additional bonus but that's not why they originally introduced the idea.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Normal&amp;quot; networks work too but, at some point, they become too deep and start working worse than shallower versions (they empirically showed that in the paper). Again, they argue that the reason for that might be that at deeper layers hidden representations become approximately similar &lt;span class=&quot;math-container&quot;&gt;$H_n \approx H_{n+1}$&lt;/span&gt; because representation is already well learned and you only need some slight adjustments.  That would mean that transformation for deeper layers is similar to identity transformation and that ordinary layers might have trouble learning that, while for residual blocks it would be easy to learn slight modification and add that to the already existing representation from the previous layer.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="2444" LastEditDate="2021-01-17T14:11:29.100" LastActivityDate="2021-01-17T14:11:29.100" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18705" PostTypeId="2" ParentId="18702" CreationDate="2020-03-18T10:10:46.590" Score="1" Body="&lt;p&gt;This sounds to me like a use case for a chatbot. You would have different intents reflecting the types of user queries that your system can respond to. The intent matching can be done by pattern matching, machine learning (classification), or a combination of the two (hybrid). You can then use the chatbot to ask clarification questions or elicit more information to identify which live agent would be the best person to take over the call. Essentially each live agent would have a list of intents plus added information (such as geographical area etc) which you then compare against the data from the caller to find the best match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is a voice call you'd need to put an &lt;a href=&quot;https://en.wikipedia.org/wiki/Speech_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASR&lt;/a&gt; system at the front of the pipeline. Chatbots can usually do live-agent handover to then pass control to a human agent at any time in the conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[Disclaimer: I work for a company that operates in exactly that area and whose system works as described above]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-03-18T10:10:46.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18706" PostTypeId="1" CreationDate="2020-03-18T11:57:18.443" Score="3" ViewCount="83" Body="&lt;p&gt;Whenever I tune my neural network, I usually take the common approach of defining some layers with some neurons. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If it overfits, I reduce the layers, neurons, add dropout, utilize regularisation. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If it underfits, I do the other way around. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But it sometimes feels illogical doing all these. So, is there a more principled way of tuning a neural network (i.e. find the optimal number of layers, neurons, etc., in a principled and mathematical sound way), in case it overfits or underfits?&lt;/p&gt;&#xA;" OwnerUserId="21936" LastEditorUserId="2444" LastEditDate="2020-03-21T14:43:18.820" LastActivityDate="2020-03-21T14:43:18.820" Title="Are there principled ways of tuning a neural network in case of overfitting and underfitting?" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;&lt;hyperparameter-optimization&gt;&lt;regularization&gt;&lt;underfitting&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18707" PostTypeId="1" CreationDate="2020-03-18T13:45:01.567" Score="4" ViewCount="76" Body="&lt;p&gt;A book on evolutionary computation by &lt;a href=&quot;https://mitpress.mit.edu/books/evolutionary-computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;De Jong&lt;/a&gt; mentions both the term evolutionary algorithms (EA) as well as evolutionary computation (EC). However, it remains unclear to me what the difference between the two is. According to &lt;a href=&quot;https://doi.org/10.1109%2FICGTSPICC.2016.7955308&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vikhar&lt;/a&gt;, EA forms a subset of EC. However, it remains unclear to me what sort of topics/algorithms would be considered EC but not EA. Is there a clear difference between the two? If so, what is this difference?&lt;/p&gt;&#xA;" OwnerUserId="34351" LastEditorUserId="34351" LastEditDate="2020-03-18T14:06:25.930" LastActivityDate="2020-03-18T23:56:12.483" Title="What is the difference between evolutionary computation and evolutionary algorithms?" Tags="&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18708" PostTypeId="1" CreationDate="2020-03-18T15:12:12.863" Score="3" ViewCount="82" Body="&lt;p&gt;I'm working currently on a problem and I'm using RL (bandit problem). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my system, I have an agent that chooses an action among &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; possible actions, and a user that decides whether the agent chooses the right action or not. If the user is satisfied with the decision made by the agent, he rewards with &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;, otherwise &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this is a good reward function, knowing that in my problem the values are in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any guidelines to follow for defining the reward function? Are there any references (books or articles) that tackle this problem and present a solution?&lt;/p&gt;&#xA;" OwnerUserId="34355" LastEditorUserId="2444" LastEditDate="2020-03-19T15:10:36.987" LastActivityDate="2020-03-19T15:10:36.987" Title="What are the guidelines for defining a reward function in reinforcement learning (bandit problem)?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18709" PostTypeId="1" CreationDate="2020-03-18T16:46:21.740" Score="2" ViewCount="36" Body="&lt;p&gt;I learned that when creating neural networks the go to was to overfit and then to regularize. However I am now in a situation where, when I make the model more complex (more layers, more filters, ...) my scores become worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am training a CNN to predict pollution 6 hours in advance. The input I give to my model is the pollution of the past 18 hours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I safely say that because there probably is a lot of noise in this data that, that is the reason when increasing my complexity, my model becomes worse?&lt;/p&gt;&#xA;" OwnerUserId="34359" LastActivityDate="2020-03-18T16:46:21.740" Title="Why does model complexity increase my validation score by a lot?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18710" PostTypeId="2" ParentId="18707" CreationDate="2020-03-18T18:05:07.753" Score="1" Body="&lt;p&gt;As you can find &lt;a href=&quot;https://en.wikipedia.org/wiki/Evolutionary_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Evolutionary algorithms form a subset of evolutionary computation in that they generally only involve techniques implementing mechanisms inspired by &lt;strong&gt;biological evolution&lt;/strong&gt; such as reproduction, mutation, recombination, natural selection, and survival of the fittest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It means, other types of evolutions, which are not necessarily a biological evolution, are in EC, but not in EA. For example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Learning_classifier_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning classifier system&lt;/a&gt; is in EC as it is evolutionary, but not completely in EA as it is not biological.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-03-18T23:56:12.483" LastActivityDate="2020-03-18T23:56:12.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18711" PostTypeId="1" CreationDate="2020-03-18T19:39:49.297" Score="1" ViewCount="27" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The following is the MDS Objective.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e4Bzz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e4Bzz.png&quot; alt=&quot;MDS Objective&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Let's think of a senario where I apply MDS &lt;strong&gt;with/from the solution I obtained from PCA&lt;/strong&gt;. Then I calculate the objective function on the initial PCA solution and MDS solution (after applying MDS on the former PCA solution). Then I would for sure assume that the objective function will decrease for the MDS solution compared with PCA solution. However, when I calculate the objective function respectively, MDS solution yields higher objective function value. Is this normal?&lt;/p&gt;&#xA;&lt;p&gt;I am attaching my code below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;import pickle&#xA;import gzip&#xA;import argparse&#xA;import time&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;from numpy.linalg import norm&#xA;&#xA;from sklearn.model_selection import train_test_split&#xA;from sklearn.decomposition import PCA&#xA;from sklearn.manifold import TSNE&#xA;from sklearn.neural_network import MLPRegressor, MLPClassifier&#xA;from sklearn.preprocessing import LabelBinarizer&#xA;from sklearn import decomposition&#xA;&#xA;from neural_net import NeuralNet, stochasticNeuralNet&#xA;from manifold import MDS, ISOMAP&#xA;import utils&#xA;&#xA;def mds_objective(Z,X):&#xA;    sum = 0&#xA;    n,d = Z.shape&#xA;    for i in range(n):&#xA;        for j in range(i+1,n):&#xA;            sum += (norm(Z[i,:]-Z[j,:],2)-norm(X[i,:]-X[j,:],2))**2&#xA;    return 0.5*sum&#xA;&#xA;dataset = load_dataset('animals.pkl')&#xA;X = dataset['X'].astype(float)&#xA;animals = dataset['animals']&#xA;n, d = X.shape&#xA;pca = decomposition.PCA(n_components = 5)&#xA;pca.fit(X)&#xA;Z = pca.transform(X)&#xA;plt.figure()&#xA;plt.scatter(Z[:, 0], Z[:, 1])&#xA;for i in range(n):&#xA;     plt.annotate(animals[i], (Z[i,0], Z[i,1]))&#xA;utils.savefig('PCA.png')&#xA;&#xA;print(pca.explained_variance_ratio_)&#xA;print(mds_objective(Z,X))&#xA;&#xA;&#xA;dataset = load_dataset('animals.pkl')&#xA;X = dataset['X'].astype(float)&#xA;animals = dataset['animals']&#xA;n,d = X.shape&#xA;&#xA;model = MDS(n_components=2)&#xA;Z = model.compress(X)&#xA;&#xA;fig, ax = plt.subplots()&#xA;ax.scatter(Z[:,0], Z[:,1])&#xA;plt.ylabel('z2')&#xA;plt.xlabel('z1')&#xA;plt.title('MDS')&#xA;for i in range(n):&#xA;       ax.annotate(animals[i], (Z[i,0], Z[i,1]))&#xA;utils.savefig('MDS_animals.png')&#xA;print(mds_objective(Z,X))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It prints the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;1673.1096816455256&lt;/p&gt;&#xA;&lt;p&gt;1776.8183112784652&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="34333" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-18T19:39:49.297" Title="Multiple-dimension scaling (MDS) objective for MDS and PCA" Tags="&lt;machine-learning&gt;&lt;principal-component-analysis&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18712" PostTypeId="1" CreationDate="2020-03-18T21:17:09.720" Score="4" ViewCount="115" Body="&lt;p&gt;I am using the cross-entropy cost function to calculate its derivatives using different variables &lt;span class=&quot;math-container&quot;&gt;$Z, W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; at different instances. Please refer image below for calculation.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/C4dCN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C4dCN.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per my knowledge, my derivation is correct for &lt;span class=&quot;math-container&quot;&gt;$dZ, dW, db$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dA$&lt;/span&gt;, but, if I refer to Andrew Ng Coursera stuff, then I am seeing an extra &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$dW$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db$&lt;/span&gt;, whereas no &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$dZ$&lt;/span&gt;. Andrew's slides on the left represent derivative and whereas the right side of slides shows NumPy implementation corresponding to the right side equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W6S7Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W6S7Y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Can someone please explain why there is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$dW^{[2]}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db^{[2]}$&lt;/span&gt; in Andrew's slides in NumPy representation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) missing &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$dZ^{[2]}$&lt;/span&gt; in Andrew's slides in both normal and NumPy representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something or doing it in the wrong way?&lt;/p&gt;&#xA;" OwnerUserId="34365" LastEditorUserId="2444" LastEditDate="2020-03-19T21:17:32.103" LastActivityDate="2021-01-14T01:03:40.460" Title="Why is my derivation of the back-propagation equations inconsistent with Andrew Ng's slides from Coursera?" Tags="&lt;math&gt;&lt;backpropagation&gt;&lt;derivative&gt;&lt;numpy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18713" PostTypeId="2" ParentId="18701" CreationDate="2020-03-19T01:44:59.640" Score="3" Body="&lt;p&gt;According to the Baidu Research's blog post &lt;a href=&quot;http://research.baidu.com/Blog/index-view?id=133&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Baidu is harnessing the power of AI in the battle against coronavirus&lt;/a&gt; (12-03-2020), there are already some artificial intelligence tools or algorithms being used to fight the coronavirus.&lt;/p&gt;&#xA;&lt;p&gt;Given that I cannot confirm that these AI tools and algorithms I will mention are really being used in practice, I will only quote the parts of the blog post that potentially answer my original question.&lt;/p&gt;&#xA;&lt;p&gt;To give some context, similar to HIV viruses, the virus that is causing the coronavirus pandemic, &lt;a href=&quot;https://en.wikipedia.org/wiki/Severe_acute_respiratory_syndrome_coronavirus_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;SARS-CoV-2&lt;/a&gt; is &lt;a href=&quot;http://research.baidu.com/Blog/index-view?id=133&quot; rel=&quot;nofollow noreferrer&quot;&gt;capable of rapidly mutating, making vaccine development and virus analysis difficult&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;AI-powered and non-contact infrared sensor system&lt;/h3&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Baidu has developed several tools that are effective in building awareness and screening populations, including &lt;strong&gt;an AI-powered, non-contact infrared sensor system&lt;/strong&gt; that provides users with fast multi-person temperature monitoring that can quickly &lt;strong&gt;detect a person if they are suspected of having a fever&lt;/strong&gt;, one of the many symptoms of the coronavirus. This technology is currently being &lt;strong&gt;used in Beijing's Qinghe Railway Station&lt;/strong&gt; to identify passengers who are potentially infected where it can examine up to 200 people in one minute without disrupting passenger flow.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;AI-powered pneumonia screening and lesion detection system&lt;/h3&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;By leveraging PaddlePaddle and the semantic segmentation toolkit PaddleSeg, LinkingMed has developed an &lt;strong&gt;AI-powered pneumonia screening and lesion detection system&lt;/strong&gt;, putting it into &lt;strong&gt;use in the hospital affiliated with XiangNan University in Hunan Province&lt;/strong&gt;. The system can pinpoint the disease in less than one minute, with a detection accuracy of 92% and a recall rate of 97% on test data sets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;Automated HealthMap system&lt;/h3&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The &lt;strong&gt;Boston Children's Hospital&lt;/strong&gt; used an &lt;strong&gt;automated HealthMap system&lt;/strong&gt; that scans online news and social media &lt;strong&gt;reports for early warning signs of outbreaks&lt;/strong&gt;, which &lt;strong&gt;led to the initial awareness that COVID-19&lt;/strong&gt; was spreading outside China.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;Autonomous vehicles carry out non-contact tasks&lt;/h3&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Access to health care and resources at a moment's notice is vital for battling the spread of the coronavirus. Autonomous vehicles are playing a useful role in providing access to necessary commodities for health-care professionals and the public alike by delivering goods in infected areas and disinfecting hospitals, effectively minimizing person-to-person transmission and alleviating the shortage of medical staff.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Apollo&lt;/strong&gt;, Baidu's autonomous vehicle platform, partnered with a local self-driving startup called Neolix to deliver supplies and food to the &lt;strong&gt;Beijing Haidian Hospital&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-28T21:07:03.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18714" PostTypeId="2" ParentId="18712" CreationDate="2020-03-19T01:56:29.283" Score="1" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: This has to do with the way A. Ng has defined back propagation for the course. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Left Column&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is only with respect to &lt;em&gt;one&lt;/em&gt; input example and so the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; factor reduces to 1 and can be omitted. He uses lower case to represent one input example (eg a vector &lt;span class=&quot;math-container&quot;&gt;$dz$&lt;/span&gt;) and upper case with respect to a (mini-)batch (eg a matrix &lt;span class=&quot;math-container&quot;&gt;$dZ$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; factors in &lt;span class=&quot;math-container&quot;&gt;$dW,db$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this definition of backprop, he &quot;defers&quot; multiplying by the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; factor until &lt;span class=&quot;math-container&quot;&gt;$dW,db$&lt;/span&gt; rather than &quot;absorbing&quot; it into &lt;span class=&quot;math-container&quot;&gt;$dZ^{[2]}$&lt;/span&gt;. That is, the &lt;span class=&quot;math-container&quot;&gt;$dZ^{[2]}$&lt;/span&gt; term is &lt;em&gt;defined&lt;/em&gt; in a way that it does not have &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observe, if you move the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; factor to be in the definition of &lt;span class=&quot;math-container&quot;&gt;$dZ^{[2]}$&lt;/span&gt; and remove it from the definitions of &lt;span class=&quot;math-container&quot;&gt;$dW,db$&lt;/span&gt; you will still come out with the same values for all &lt;span class=&quot;math-container&quot;&gt;$dW,db$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Speculation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;deferred&quot; multiplication might have to do with numerical stability. Or simply a stylistic choice made by A. Ng. This might also prevent one from &quot;accidentally&quot; multiplying by &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{m}$&lt;/span&gt; more than once. &lt;/p&gt;&#xA;" OwnerUserId="28343" LastEditorUserId="28343" LastEditDate="2020-03-19T02:06:31.893" LastActivityDate="2020-03-19T02:06:31.893" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18715" PostTypeId="1" CreationDate="2020-03-19T05:52:24.337" Score="0" ViewCount="72" Body="&lt;p&gt;Some of the NLP applications taken from this link &lt;a href=&quot;https://towardsdatascience.com/natural-language-processing-nlp-top-10-applications-to-know-b2c80bd428cb&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLP Applications&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Machine Translation&lt;/li&gt;&#xA;&lt;li&gt;Speech Recognition&lt;/li&gt;&#xA;&lt;li&gt;Sentiment Analysis&lt;/li&gt;&#xA;&lt;li&gt;Question Answering&lt;/li&gt;&#xA;&lt;li&gt;Automatic Summarization&lt;/li&gt;&#xA;&lt;li&gt;Chatbots&lt;/li&gt;&#xA;&lt;li&gt;Market Intelligence&lt;/li&gt;&#xA;&lt;li&gt;Text Classification&lt;/li&gt;&#xA;&lt;li&gt;Character Recognition&lt;/li&gt;&#xA;&lt;li&gt;Spell Check&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which are the NLP applications that supports recurrent neural network?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="1671" LastEditDate="2020-04-23T23:02:09.390" LastActivityDate="2021-05-19T04:09:10.857" Title="Which NLP applications are based on recurrent neural networks?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18716" PostTypeId="2" ParentId="18715" CreationDate="2020-03-19T08:29:31.400" Score="0" Body="&lt;p&gt;Speech recognition and Character recognition are not part of NLP. Everything else on your list can in principle be done with RNNs. But the field is quickly moving towards using &lt;a href=&quot;https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2020-03-19T08:29:31.400" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18717" PostTypeId="1" AcceptedAnswerId="18719" CreationDate="2020-03-19T09:17:33.637" Score="1" ViewCount="83" Body="&lt;p&gt;Suppose we have a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and we use &lt;em&gt;SARSA&lt;/em&gt; to evaluate &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we say that &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a) = V^\pi(s)$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why I think this can be the case is because &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt; is defined as the value obtained from taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and then following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; thereafter. However, the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; taken is the policy according to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$s \in S $&lt;/span&gt;. This seems to corresponds to the value function equation of &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s_t) = r(s_t) + \gamma V^\pi(s_{t+1})$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-03-19T13:50:20.123" LastActivityDate="2020-03-19T14:16:01.977" Title="What is the relationship between the Q and V functions?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;sarsa&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18718" PostTypeId="2" ParentId="16138" CreationDate="2020-03-19T10:05:35.440" Score="1" Body="&lt;p&gt;The model has learnt the &quot;features&quot; for the type of inputs, eg. faces. &#xA;For the problem to be called one-shot, it needs to also correctly classify/compare any new samples. For example, in face recognition application, any new person's images should also result a positive for their own image and negative for any other seen or unseen image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we are using euclidean distance of the final feature layer and not performing any final classification, we can say we are using the weights of a pretrained network and computing final value using that(distance), thus transfer learning. There is no backprop in this, but what you want to do with the embeddings, such as learning a threshold function can be considered learning.&lt;/p&gt;&#xA;" OwnerUserId="27875" LastActivityDate="2020-03-19T10:05:35.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18719" PostTypeId="2" ParentId="18717" CreationDate="2020-03-19T10:30:37.230" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can we say that &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a) = V^\pi(s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct relationship is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^\pi(s) = \sum_a \pi(a|s) Q^\pi(s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or, if you have a &lt;em&gt;deterministic&lt;/em&gt; policy &lt;span class=&quot;math-container&quot;&gt;$a = \pi(s)$&lt;/span&gt; you can instead write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^\pi(s) = Q^\pi(s, \pi(s))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, this is because the &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s)$&lt;/span&gt; is the expected future return when following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, whilst &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt; is the expected future return where it &lt;em&gt;ignores&lt;/em&gt; the policy for only the next action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (which will decide immediate reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; independently of the policy), and thereafter follows &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above equations essentially show what happens when you apply the policy in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to decide which Q value(s) to use, they remove the independent choice of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s,a)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible misunderstanding that you have is that &quot;on-policy&quot; means the same thing as the equations show when considering action values (Q values) - it does not. When &lt;em&gt;any&lt;/em&gt; algorithm learns action values, it learns the same thing conceptually, i.e. the expected (and maybe discounted) sum of future rewards given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; when making a free choice of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; for the next step, and thereafter strictly following the policy being evaluated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What &lt;em&gt;is&lt;/em&gt; different between on-policy and off-policy is which policy gets evaluated - for on-policy methods like SARSA you evaluate action values for the same policy that you use to generate actions. For off-policy methods like Q learning you evaluate a different &lt;em&gt;target policy&lt;/em&gt;. Both approaches have the same interpretation of what &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; means otherwise, and have the same relationship between Q and V for their repsective policies.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-03-19T14:16:01.977" LastActivityDate="2020-03-19T14:16:01.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18721" PostTypeId="1" CreationDate="2020-03-19T12:25:50.327" Score="1" ViewCount="47" Body="&lt;p&gt;This has been a mystery to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the walking robots look like idiots now. But we do have a lot of simulation-based results&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=pgaEE27nsQw&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Flexible Muscle-Based Locomotion for Bipedal Creatures&#xA;&lt;/a&gt;), so why can't we just apply the simulation results to a real robot and let it walk, not like an idiot, but like an running ostrich?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the main loop running at more than 60 fps, I fail to see how possibly the program could fail to stop the robot from losing balance. When I balance a stick on my hand, I could probably only do 5 fps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have not only supercomputers connected to the robots, but also reinforcement learning algorithms at our disposal, so what has been the bottleneck in the problem of bipedal walking?&lt;/p&gt;&#xA;" OwnerUserId="34382" LastEditorUserId="9947" LastEditDate="2020-03-19T16:16:07.803" LastActivityDate="2020-03-19T16:16:07.803" Title="Why haven't we solved the problem of bipedal walking?" Tags="&lt;reinforcement-learning&gt;&lt;robotics&gt;&lt;cyborg&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18723" PostTypeId="1" CreationDate="2020-03-19T17:59:22.863" Score="2" ViewCount="81" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1512.03385v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original ResNet paper&lt;/a&gt; they talk about using plain identity skip connections when the input and output of a block have the same dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pPvVA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pPvVA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the input and output have different dimensions they propose two options:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A) Use an identity mapping padded with zeros to make up for the extra dimensions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(B) Use a &quot;projection&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/59wYe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/59wYe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which (after some digging around in &lt;a href=&quot;https://towardsdatascience.com/hitchhikers-guide-to-residual-networks-resnet-in-keras-385ec01ec8ff&quot; rel=&quot;nofollow noreferrer&quot;&gt;other people's code&lt;/a&gt;) I see as meaning: do a convolution with a 1x1 kernel with trainable weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(B) is confusing to me because it seems to ruin the point of ResNet by making the skip connection trainable. Then the main path is not really learning a &quot;residual&quot; relative to an identity transformation. So at this point, I'm no longer sure how to interpret the intent or expected effect of this type of block. And I would think that one should justify doing it in the first places instead of just not putting a skip connection there at all (which in my mind is the status-quo before this paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So can anyone help explain away my confusion here?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-03-20T12:43:10.910" Title="If the point of the ResNet skip connection is to let the main path learn the residual relative to identity, why are there convolutional skips?" Tags="&lt;convolutional-neural-networks&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18724" PostTypeId="2" ParentId="18431" CreationDate="2020-03-19T18:33:44.063" Score="0" Body="&lt;p&gt;What about GANs or genetic algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first idea (GAN) is that you basically create 2+ random bots who fight each other, and they keep adjusting their weights so that they can beat the other bot. That means, that those 2+ bots keep improving their &quot;fighting performance&quot; for as long as you want, eventually being even better than humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second idea (genetic algorithms) is to generate a lot of bots who genetically differ from each other just for a slight mutation. You make them fight, and the best perforing/last standing will become the new clone where the new &quot;lot of bots&quot; gets generated from.&lt;/p&gt;&#xA;" OwnerUserId="32751" LastActivityDate="2020-03-19T18:33:44.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18727" PostTypeId="2" ParentId="15976" CreationDate="2020-03-19T20:30:22.150" Score="0" Body="&lt;p&gt;A little more information about the documents will be helpful. I am guessing that your scenario has webpages from different websites, you're feeding html pages to the network and the page contains the website name or url, which the network is picking up on and using it to label. I am assuming you're using a RNN or similar network for the classification task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question is to identify when your network is using the source identifying feature (name or url), and instruct it to not use this feature. If you can do some preprocessing and identify where in the document the source name/url is, you may tag each word/character in the input with this information. You may use this tag to add a large regularizing loss term which brings down the weights connected to the tagged input words/characters in the first layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a better strategy will be to remove all instance of the header / url which contains website info, as @clement-hui suggested. This is cleaner and easier to implement than above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps your network is identifying the document source slightly more indirectly, and above preprocessing is not sufficient/useful. In this case, you may want to make random chunks of your document, and give each chunk the label corresponding to that document. Here it is more likely that some chunks will not have the source identifying information, and the network will be forced to pick a strategy which labels using the content rather than the source. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At test time you may either feed the whole document or random chunks from the document and use majority voting to get the final answer.&lt;/p&gt;&#xA;" OwnerUserId="20799" LastActivityDate="2020-03-19T20:30:22.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18728" PostTypeId="1" CreationDate="2020-03-20T01:36:10.857" Score="3" ViewCount="37" Body="&lt;p&gt;I have an input vector &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, which contains a series of measurements within a period, e.g. 100 measurements in 1 sec. The goal is to predict an event, let's say, moving forward, backward or static. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to predict the output just by looking at one series of measurements, but by looking at a window of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; vectors &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; of measurements, making it dependant on the previous measurements, because of the noise in the measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way RNN can help me with this? Many to one architecture? LSTM? &#xA;CNN of 1D + LSTM + dense? &lt;/p&gt;&#xA;" OwnerUserId="34390" LastEditorUserId="2444" LastEditDate="2020-03-20T02:09:05.477" LastActivityDate="2020-03-20T02:09:05.477" Title="How to predict an event (or action) based on a window of time-series measurements?" Tags="&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18729" PostTypeId="1" CreationDate="2020-03-20T01:45:16.340" Score="3" ViewCount="180" Body="&lt;p&gt;I am reading the book &quot;Artificial Intelligence: A Modern Approach&quot; by Stuart and Norvig. I'm having trouble understanding a step of the recursive best-first search (RBFS) algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the pseudocode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fd70Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fd70Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does the line &lt;code&gt;s.f &amp;lt;- max(s.g + s.h, node.f)&lt;/code&gt; do? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a diagram of the execution of RBFS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yYh4p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yYh4p.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23895" LastEditorUserId="2444" LastEditDate="2020-03-20T03:01:31.283" LastActivityDate="2020-03-22T04:29:13.597" Title="What does the statement with the max do in the recursive best-first search algorithm?" Tags="&lt;search&gt;&lt;norvig-russell&gt;&lt;best-first-search&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18730" PostTypeId="1" AcceptedAnswerId="18731" CreationDate="2020-03-20T08:13:24.963" Score="2" ViewCount="64" Body="&lt;p&gt;I'm relatively new to image classification. Currently, I am trying to classify insect images, using a convolutional neural network (CNN).&#xA;When I ask a human expert to identify an insect, I usually provide 2 photos: back and face. It seems that sometimes one feature stands out and allows identification with high certainty (&quot;spots on the back - definitely a ladybug&quot;), while other times you need to cross-reference both angles (&quot;grey back could mean a few things, but after cross-referencing with the eyes - it's a moth&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is it customary to implement this? Naively I was considering:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Two separate networks, one for backs and one for faces? If so, what formula is best for weighing in their outputs?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Single network, but separate dual classifications - e.g. &quot;moth face&quot;, &quot;moth back&quot;, &quot;ladybug face&quot;, &quot;ladybug back&quot;? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A single network, feed everything naively (e.g. moths from different angles, all with the same classification &quot;moth&quot;) and rely on the NN to sort it out itself?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="34389" LastEditorUserId="2444" LastEditDate="2020-03-21T14:31:04.437" LastActivityDate="2020-03-21T14:32:02.233" Title="How to perform insect classification given two images of the same insect?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18731" PostTypeId="2" ParentId="18730" CreationDate="2020-03-20T09:43:27.677" Score="1" Body="&lt;p&gt;There are several ways you can do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One is to input both images in input, so it can be a 2 input system or an input with 6 channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you suggested in 1st point, you can make 2 networks, connect them at the end and add another layer for final classification or use outputs from both and train another classifier (like Gradient bosting). You can look up ensemble techniques and ensemble of neural networks for more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dual classification and naive approach, you will again have a problem in case there are some insects that can solely be identified from features at a specific angle, if so you can then discard other angles (e.g. ignore moth face if moth back is the distinguishing feature)&lt;/p&gt;&#xA;" OwnerUserId="27875" LastEditorUserId="2444" LastEditDate="2020-03-21T14:32:02.233" LastActivityDate="2020-03-21T14:32:02.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18732" PostTypeId="2" ParentId="18723" CreationDate="2020-03-20T12:43:10.910" Score="2" Body="&lt;p&gt;Well, I found an answer that satisfies me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The zero-padded identity is not ideal. Suppose we're mapping from 64 channels to 128 channels. Then the zero-padded identity will map to an output where half of the channels are the same as the inputs, and the other half are all zeros. So that means the main path is learning a residual for half of the output channels, and learning a mapping from the ground up for the other half of the channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the alternative is to use option (B) which is the single 1x1 convolution. After reading their &lt;a href=&quot;https://arxiv.org/pdf/1603.05027.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNetV2 paper&lt;/a&gt; I realised that they really double down on this concept of maintaining a &quot;clean&quot; alternative path all the way up and down the network. So then the question is, what's &quot;cleaner&quot;?: a block of two convolutions of the form 3x3x64 then 3x3x128, or a shortcut with a 1x1x128 convolution? The shortcut is a tempting choice here. And in fact, in the original paper they show empirically that the convolutional shortcut is better than the identity shortcut. Maybe it might be worth running a test to see if the convolutional shortcut is better than no shortcut at all, but until I decide to run such a test, I'll presume the authors would have mentioned it. The other thing worth noting is that there are only a few places in the whole network where the dimensions need to be increased, so maybe the impact is small.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-03-20T12:43:10.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18735" PostTypeId="2" ParentId="16004" CreationDate="2020-03-20T17:09:45.810" Score="1" Body="&lt;p&gt;If the noise is confined to a particular spectral band, Fourier transform followed by filtering, followed by an inverse Fourier transform will work.  If it is multiplicative noise, filtering the Fourier transform of the logarithm of the signal might work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, the nature of the noise determines what's possible and the best way to remove it.&lt;/p&gt;&#xA;" OwnerUserId="28348" LastActivityDate="2020-03-20T17:09:45.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18738" PostTypeId="1" CreationDate="2020-03-21T10:55:22.077" Score="2" ViewCount="41" Body="&lt;p&gt;I am approaching the implementation of the OpenPose algorithm for realtime human body pose estimation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the official paper &lt;a href=&quot;https://arxiv.org/pdf/1812.08008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenPose: Realtime Multi-Person 2D Pose&#xA;Estimation using Part Affinity Fields&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; fields (body part maps, and part affinity fields) are estimated. These have the same size as the input image, and, according to the paper, these fields should be outputted at a given step in the forward pass (after a given number of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; stages, and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; stages), but, since before entering these stages the image is passed through the initial layers of the VGG-19 model, the spatial dimension is encoded and the features that finally enter the &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; stages have other dimensionality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the network is convolutional, there's no FC layer at all. The VGG-19 part is the only one that contains MaxPooling layers, hence affecting the spatial relations and size of the receptive fields. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My point is, after stage execution, I get tensors of shape &lt;code&gt;[batch_size, filter_number, 28, 28]&lt;/code&gt;. The issue is that the paper is not stating how to decode this information into the &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; maps of size &lt;span class=&quot;math-container&quot;&gt;$224 \times 224$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following a traditional approach and decoding the final tensors with a linear net from, let's say, &lt;span class=&quot;math-container&quot;&gt;$15000 \rightarrow (224 * 224 * \text{ number of body parts }) + (224 * 224 * \text{ number of limbs } * 2)$&lt;/span&gt;~A very huge number!, is out of question for any domestic computer, I presume I should have the least 128GbRAM installed, and is not the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another solution is to remove the max-pooling layers from the VGG-19 part, but then although the map size is preserved to &lt;span class=&quot;math-container&quot;&gt;$224$&lt;/span&gt;, instead of &lt;span class=&quot;math-container&quot;&gt;$28$&lt;/span&gt;, the huge amount of computations and values that need to be stored also lead to memory errors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the problem is, how can I get to a final output of &lt;span class=&quot;math-container&quot;&gt;$224 \times 224$&lt;/span&gt; without FC layers, from a tensor of shape &lt;code&gt;[batch_size, bodyparts, 28, 28]&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not an easy answer. I will check a TensorFlow implementation I have seen around to see how the problem was solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any parallel ideas are greatly welcome.&lt;/p&gt;&#xA;" OwnerUserId="22869" LastEditorUserId="2444" LastEditDate="2020-03-21T14:24:39.973" LastActivityDate="2020-03-21T14:24:39.973" Title="How can I get to a final output of shape $224 \times 224$, without FC layers, from a tensor of specific shape, in OpenPose?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;implementation&gt;&lt;pose-estimation&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18740" PostTypeId="2" ParentId="17656" CreationDate="2020-03-21T12:12:53.250" Score="0" Body="&lt;p&gt;If using TFLite python API, you may reach to 1/2 FPS. On downscaled custom Pose architectures you may get more FPS. You can try the TFLite c++ API as well, it might be a bit faster than the python one. &#xA;To reach more than this you would need a TPU or a library that could use GPU computation of the Pi. Nevertheless, I doubt that it would reach 15 FPS with OpenPose open-source library.&lt;/p&gt;&#xA;" OwnerUserId="34409" LastActivityDate="2020-03-21T12:12:53.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18741" PostTypeId="1" CreationDate="2020-03-21T13:06:21.880" Score="1" ViewCount="38" Body="&lt;p&gt;I'm looking for a community or competition website related to human aggression detection using Deep Learning in a video.&#xA;Also, I'm looking for a dataset of human aggression activities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="18245" LastActivityDate="2020-03-21T13:06:21.880" Title="Human Aggression Detection Community, Competition and dataset" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18742" PostTypeId="1" CreationDate="2020-03-21T14:36:06.900" Score="3" ViewCount="74" Body="&lt;p&gt;I have a dataset and want to be able to construct a graph from it in a supervised fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume I have a dataset with &lt;em&gt;N&lt;/em&gt; nodes, each node has e.g. 10 features. Out of these N nodes, I want to learn a graph, i.e. an &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt; adjacency matrix. So, I start with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; nodes and all I know is a 10-dimensional feature vector for each node. I have no knowledge about the relation between these nodes and want to figure it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example for &lt;span class=&quot;math-container&quot;&gt;$N=6$&lt;/span&gt;, but in practice &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is not fixed.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rsIhj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rsIhj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the output I would like to get here is a &lt;span class=&quot;math-container&quot;&gt;$6\times6$&lt;/span&gt; adjacency matrix, representing the relations between the nodes (undirected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: &lt;em&gt;N&lt;/em&gt; is arbitrary and not fixed. So an algorithm should be able to perform on any given &lt;em&gt;N&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataset is labeled. For the training dataset, I have the desired adjacency matrix for each collection of input nodes, which is filled with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the output of the algorithm could also be an adjacency matrix filled with non-integer numbers in &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;, giving some kind of probability of the nodes being connected (preferably close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; of course). So I could easily give a number as the label for each node. In the above example, the labels for the three connected nodes could be class &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any kind of supervised learning algorithm (e.g. some sort of graph neural network) that can perform these tasks?&lt;/p&gt;&#xA;" OwnerUserId="34411" LastEditorUserId="34411" LastEditDate="2020-04-24T13:41:54.960" LastActivityDate="2020-04-24T13:41:54.960" Title="How can I learn a graph given nodes with features in a supervised fashion?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;supervised-learning&gt;&lt;geometric-deep-learning&gt;&lt;graphs&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18743" PostTypeId="2" ParentId="17456" CreationDate="2020-03-21T16:20:27.363" Score="1" Body="&lt;p&gt;There are other sources that will lead to different results in addition to weight initialization. For example dropout layers. Make sure you specify the random seed.Also data reading using flow from directory,make sure you set shuffle to False or if you do not then set the random seed. If you use transfer learning make that part of your network non-trainable. Some networks have dropout in them and do not provide a way to set the random seed. IF you are using a GPU there are even more issues to contend with.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-21T16:20:27.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18744" PostTypeId="1" CreationDate="2020-03-21T17:40:27.883" Score="0" ViewCount="77" Body="&lt;p&gt;In regularzied cost function a L2 regularization cost has been added.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bao6R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bao6R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/18712/why-is-my-derivation-of-the-back-propagation-equations-inconsistent-with-andrew&quot;&gt;Here&lt;/a&gt; we have already calculated cross entropy cost w.r.t &lt;span class=&quot;math-container&quot;&gt;$A, W$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the regularization notebook (see below) in order to do derivation of regularized &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; (cost function), the changes only concern &lt;span class=&quot;math-container&quot;&gt;$dW^{[1]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dW^{[2]}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dW^{[3]}$&lt;/span&gt;. For each, you have to add the regularization term's gradient.(No impact on &lt;span class=&quot;math-container&quot;&gt;$dA^{[2]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$db^{[2]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dA^{[1]}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db^{[1]}$&lt;/span&gt; ?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XbZVp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XbZVp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am doing it using the chain rule then I am getting change in values for &lt;span class=&quot;math-container&quot;&gt;$dA^{[2]}$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$dZ^{[2]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dA^{[1]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dW^{[1]}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db^{[1]}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please refer below how I calculated this ?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BQHMO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BQHMO.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain why I am getting different results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the derivative of L2 reguarlization w.r.t &lt;span class=&quot;math-container&quot;&gt;$dA^{[2]}$&lt;/span&gt; ? (in equation 1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Derivative of L2 regularization cost w.r.t &lt;span class=&quot;math-container&quot;&gt;$dA^{[2]}$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) How adding regularization term not affecting &lt;span class=&quot;math-container&quot;&gt;$dA^{[2]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$db^{[2]}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dA^{[1]}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db^{[1]}$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$dA$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db$&lt;/span&gt;) but changes &lt;span class=&quot;math-container&quot;&gt;$dW$&lt;/span&gt;'s ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="34365" LastEditorUserId="9947" LastEditDate="2020-03-24T09:18:00.067" LastActivityDate="2020-03-24T09:18:00.067" Title="Derivation of regularized cost function w.r.t activation and bias" Tags="&lt;backpropagation&gt;&lt;math&gt;&lt;regularization&gt;&lt;derivative&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18745" PostTypeId="1" AcceptedAnswerId="18746" CreationDate="2020-03-21T20:26:09.117" Score="7" ViewCount="612" Body="&lt;p&gt;I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I trained a data model to identify and classify different types of irises and I show it a picture of a cat, is there a way to add in an &quot;unknown&quot; or &quot;not a&quot; classification or would it necessarily have to guess what type of iris the cat most looks like?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, I could easily just add another classification with the label &quot;not an iris&quot; and train it using pictures of cats, but then what if I show it a picture of a chair (the list of objects goes on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example would be in natural language processing. If I develop an application that takes the input language and spits out &quot;I think this is Spanish&quot;, what if it encounters a language it doesn't recognize?&lt;/p&gt;&#xA;" OwnerUserId="34021" LastEditorUserId="2444" LastEditDate="2021-01-17T12:14:45.380" LastActivityDate="2021-01-17T12:14:45.380" Title="How should the neural network deal with unexpected inputs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;bayesian-deep-learning&gt;&lt;bayesian-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18746" PostTypeId="2" ParentId="18745" CreationDate="2020-03-21T23:11:48.527" Score="11" Body="&lt;p&gt;This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and &lt;em&gt;identically&lt;/em&gt; distributed, i.e., you do &lt;em&gt;not&lt;/em&gt; expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit assumption that data comes from the same family of distributions (e.g. only Gaussians) and that all your training examples are independently drawn from the same distribution (specific mean and variance). Of course, this is a big limitation!&lt;/p&gt;&#xA;&lt;p&gt;A partial solution to your problem is to use a &lt;a href=&quot;https://arxiv.org/pdf/1505.05424.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural network&lt;/a&gt; (BNN). The idea of a BNN is to associate, rather than a single number, a distribution (usually a Gaussian distribution) with each unit (or neuron) of the neural network. Therefore, for each unit of the network, there are two learnable parameters: the mean and variance of a Gaussian distribution. Consequently, a BNN usually has the double of the number of parameters of a conventional (or non-Bayesian) neural network. However, by learning a distribution for each parameter, you also learn the uncertainty about the potential true value of each unit, based on the available training data.&lt;/p&gt;&#xA;&lt;p&gt;The forward passes of such a BNN are stochastic, i.e. you sample from each of these Gaussian distributions for every forward pass, so the output of the network is also stochastic (i.e. given the same input example, the output may be different each time).&lt;/p&gt;&#xA;&lt;p&gt;If your dataset is small, one expects the BNN to have wide Gaussian distributions, i.e. high uncertainty about the true value of the units. So, one expects a BNN to be able to deal with unexpected inputs more robustly. To be more precise, if you train a BNN with a small dataset, the hope is that Gaussian distributions will be wide and thus the outputs of the BNN will be highly variable (i.e. the model is highly uncertain). The more data is gathered, the less uncertain the BNN should be.&lt;/p&gt;&#xA;&lt;p&gt;This doesn't completely solve your problem, but it should at least mitigate it, i.e. if you provide an unseen example to the BNN, then ideally it should be uncertain about the actual label of that input.&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, I didn't explain certain details of BNNs, but, at least, this answer gives you a potential solution. Of course, this doesn't exclude the possibility of having an &lt;a href=&quot;https://ai.stackexchange.com/q/4889/2444&quot;&gt;&amp;quot;unknown&amp;quot; class&lt;/a&gt;. The approaches are not mutually exclusive. There may also be other solutions, but I am not aware of any.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T12:14:24.247" LastActivityDate="2021-01-17T12:14:24.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18747" PostTypeId="2" ParentId="18742" CreationDate="2020-03-22T00:00:32.967" Score="2" Body="&lt;p&gt;It's perfectly reasonable to apply 'traditional' Deep Learning approaches to try and learn an adjacency matrix (a matrix is just a vector of vectors, which can be flattened into a single output vector) but you might need a lot of training data as N gets larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your outputs could certainly have the form of an adjacency matrix, as you describe. Whether it's more useful to have 'boolean' (either 0 or 1)&#xA;or 'probabalistic' entries in the matrix depends both on the data and the specifics of your end application.&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2020-03-22T00:00:32.967" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18748" PostTypeId="1" CreationDate="2020-03-22T02:05:25.730" Score="2" ViewCount="30" Body="&lt;p&gt;I'm currently looking for a standard data-set to test some new regression models I've been developing on, and the &quot;Boston Housing Prices&quot; data-set seemed to stand out, since it's very standard.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm wondering what is currently the most competitive regression models/algorithms used on this data-set?  Is it decision trees or some neural network architecture?&lt;/p&gt;&#xA;" OwnerUserId="31649" LastActivityDate="2020-03-22T02:05:25.730" Title="Boston Housing Current Best Model" Tags="&lt;datasets&gt;&lt;reference-request&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18749" PostTypeId="2" ParentId="18729" CreationDate="2020-03-22T04:29:13.597" Score="1" Body="&lt;p&gt;This is probably more easily understood as the &lt;a href=&quot;http://www.bgu.ac.il/~felner/2015/SoCS/Collpase.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;collapse/restore macro&lt;/a&gt;. The idea is that the previously explored state was collapsed and only the minimum f-cost from the sub-tree was stored. This represents the best unexpanded state in the subtree that was collapsed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When restoring the portion of the collapsed tree, the f-cost of the restored node could either be the original f-cost (g+h), or it could be the stored f-cost if it is larger. By taking the max, the code ensures that states that are restored maintain at least the cost of the previously best unexpanded state. (If the g+h cost is larger, then we know the state wasn't previously expanded and it wasn't previously the state on the fringe with the minimum edge cost.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linked paper gives several examples where similar ideas are used during search.&lt;/p&gt;&#xA;" OwnerUserId="17493" LastActivityDate="2020-03-22T04:29:13.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18751" PostTypeId="1" AcceptedAnswerId="18752" CreationDate="2020-03-22T07:35:53.407" Score="4" ViewCount="58" Body="&lt;p&gt;&lt;strong&gt;[TL;DR]&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron with different number of hidden layers, &lt;strong&gt;BUT&lt;/strong&gt; all the hidden layers had 2 neurons. The MLP never reached 100% accuracy. I tried to visualize how the MLP would classify the points of the 2D space with Black and White. This is the final image I get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/69rSY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/69rSY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, I was expecting that the MLP could classify 2 classes on a 2D space with 2 Neurons at each hidden layer, and I was expecting to see a white circle encapsulating the red points and the rest be a black space. Is there a (mathematical) reason, why the MLP fails to create a close shape, rather it seems to go from infinity to infinity on a 2d space ?? (Notice: If I use 3 neurons at each hidden layer, the MLP succeeds quite fast). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[Notebook Style]&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generated two classes Red and Blue on a 2D space.&lt;br&gt;&#xA;Red are points on Unit Circle &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;size_ = 200&#xA;classA_r = np.random.uniform(low = 0, high = 1, size = size_)&#xA;classA_theta = np.random.uniform(low = 0, high = 2*np.pi, size = size_)&#xA;classA_x = classA_r * np.cos(classA_theta)&#xA;classA_y = classA_r * np.sin(classA_theta)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and Blue are points on a Circle Ring with radius limits (3,4).  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;classB_r = np.random.uniform(low = 2, high = 3, size = size_)&#xA;classB_theta = np.random.uniform(low = 0, high = 2*np.pi, size = size_)&#xA;classB_x = classB_r * np.cos(classB_theta)&#xA;classB_y = classB_r * np.sin(classB_theta)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried to train a Multi Layer Perceptron with different number of hidden layers, &lt;strong&gt;BUT&lt;/strong&gt; all the hidden layers had 2 neurons. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hidden_layers = 15&#xA;inputs = Input(shape=(2,))&#xA;dnn = inputs&#xA;for l_no in range(hidden_layers):&#xA;    dnn = Dense(2, activation='tanh', name = &quot;layer_{}&quot;.format(l_no))(dnn)&#xA;outputs = Dense(2, activation='softmax', name = &quot;layer_out&quot;)(dnn)&#xA;&#xA;model = Model(inputs=inputs, outputs=outputs)&#xA;&#xA;model.compile(optimizer='adam', loss='categorical_crossentropy', metrics='accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The MLP never reached 100% accuracy. I tried to visualize how the MLP would classify the points of the 2D space with Black and White.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;limit = 4&#xA;step = 0.2&#xA;grid = []&#xA;x = -limit&#xA;while x &amp;lt;= limit:&#xA;    y = -limit&#xA;    while y &amp;lt;= limit:&#xA;        grid.append([x, y])&#xA;        y += step&#xA;    x += step&#xA;grid = np.array(grid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prediction = model.predict(grid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the final image I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xs = []&#xA;ys = []&#xA;cs = []&#xA;for point in grid:&#xA;    xs.append(point[0])&#xA;    ys.append(point[1])&#xA;for pred in prediction:&#xA;    cs.append(pred[0])&#xA;&#xA;plt.scatter(xs, ys, c = cs, s=70, cmap = 'gray')&#xA;plt.scatter(classA_x, classA_y, c = 'r', s= 50)&#xA;plt.scatter(classB_x, classB_y, c = 'b', s= 50)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/69rSY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/69rSY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, I was expecting that the MLP could classify 2 classes on a 2D space with 2 Neurons at each hidden layer, and I was expecting to see a white circle encapsulating the red points and the rest be a black space. Is there a (mathematical) reason, why the MLP fails to create a close shape, rather it seems to go from infinity to infinity on a 2d space ?? (Notice: If I use 3 neurons at each hidden layer, the MLP succeeds quite fast).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by a closed shape, take a look at the second image which was generated by using 3 neurons at each layer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for l_no in range(hidden_layers):&#xA;    dnn = Dense(3, activation='tanh', name = &quot;layer_{}&quot;.format(l_no))(dnn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SunX6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SunX6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;[According to Marked Answer]&lt;/em&gt;&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from keras import backend as K&#xA;def x_squared(x):&#xA;    x = K.abs(x) * K.abs(x)&#xA;    return x&#xA;hidden_layers = 3&#xA;inputs = Input(shape=(2,))&#xA;dnn = inputs&#xA;for l_no in range(hidden_layers):&#xA;    dnn = Dense(2, activation=x_squared, name = &quot;layer_{}&quot;.format(l_no))(dnn)&#xA;outputs = Dense(2, activation='softsign', name = &quot;layer_out&quot;)(dnn)&#xA;model.compile(optimizer='adam',&#xA;              loss='mean_squared_error',&#xA;              metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9bKSC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9bKSC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34425" LastEditorUserId="34425" LastEditDate="2020-03-22T09:49:51.623" LastActivityDate="2020-03-22T09:49:51.623" Title="Why MLP cannot approximate a closed shape function?" Tags="&lt;machine-learning&gt;&lt;keras&gt;&lt;multilayer-perceptron&gt;&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18752" PostTypeId="2" ParentId="18751" CreationDate="2020-03-22T08:38:08.150" Score="3" Body="&lt;p&gt;In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, &lt;code&gt;tanh&lt;/code&gt; or hyperbolic tangent). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming at least one hidden layer, then the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt; applies. How closely you can approximate any given function is limited by the number of neurons, and not strongly by the choice of activation function. However, the choice of activation function is still relevant to how good an approximation is. When you get to low numbers of neurons in a hidden layer, then the approximations are more strongly tied to the nature of the activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With one neuron in a hidden layer, you can only approximate some affine transformation of the activation function. Other low numbers, such as 2, 3 etc, will still show strong tendencies for certain families of shapes. This is very similar conceptually to using limited number of frequencies in a Fourier transform - if you limit yourself to only &lt;span class=&quot;math-container&quot;&gt;$a_1 \text{sin}(x) + a_2 \text{sin}(2x)$&lt;/span&gt; to approximate a function, then you will definitely notice the sinusoidal building blocks in any output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that if you changed the activation function in the first hidden layer to &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt; then you could get a good result with two neurons per layer. If you then took that network and tried to train it on a simple linear split, it would fail, always producing some curved closed surface that covered the training examples as best that it could - kind of the opposite problem as you are seeing with your circular pattern fitted to NN with tanh activations throughout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting thing about using &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt; is that this is a deliberate choice (given knowledge about how you constructed the example) to map your input space to a new space where examples can be linearly separated. In fact this appears to be what layers in multi-layer NNs learn - each layer incrementally and progressively maps its input space to a new space where examples can be better separated in a linear fashion.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-03-22T08:47:07.437" LastActivityDate="2020-03-22T08:47:07.437" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18753" PostTypeId="1" AcceptedAnswerId="18777" CreationDate="2020-03-22T09:52:02.283" Score="2" ViewCount="257" Body="&lt;p&gt;I did a simple Actor-Critic implementation in Keras using 2 networks where the critic learns the Q-Values of every action, and the actor predicts probabilities for choosing each action. In training, the target probabilities for the actor was a one-hot vector with &lt;code&gt;1.0&lt;/code&gt; in the maximum Q-Value prediction position and &lt;code&gt;0.0&lt;/code&gt; in all the rest, and simply used &lt;code&gt;fit&lt;/code&gt; method on the actor model with mean squared error loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure what to set as the target when switching to A2C. In all the guides I saw it's mentioned that the critic now learns one value per state, not one value per action in the action space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This change makes it unclear on how to set the target vector for the actor. The guides/SE questions I went over did not explain this point and simply said that we can calculate the advantage value using the value function (&lt;a href=&quot;https://ai.stackexchange.com/a/7425/32950&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://towardsdatascience.com/qrash-course-ii-from-q-learning-to-gradient-policy-actor-critic-in-12-minutes-8e8b47129c8c&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.freecodecamp.org/news/an-intro-to-advantage-actor-critic-methods-lets-play-sonic-the-hedgehog-86d6240171d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) for the current and next state, which is fine, except we can only do that for the specific action taken and not for every action in the action-space because we don't the value for every next state for every action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, we only know &lt;code&gt;A(s,a)&lt;/code&gt; for our memorized &lt;code&gt;a&lt;/code&gt;, and we know nothing about the advantage of other actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my guesses was that you still calculate the Q-Values, because after all, &lt;strong&gt;the value function is defined by the Q-Values&lt;/strong&gt;. The value function is the sum over every action &lt;code&gt;a&lt;/code&gt; of &lt;code&gt;Q(s,a)*p(a)&lt;/code&gt;. So does the critic need to learn the Q-Values and sum their multiplications with the probabilities generated by the policy network (actor), and calculate the advantages of every action?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's even more confusing because in one of the guides they said that the critic actually learns the advantage values, and not the value function (like all the other guides said), which is strange because you need to use the critic to predict the value function of the state and the next state. Also, the advantage function is per-action and in the implementations I see the critic has one output neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that what's being done in the examples I saw was to train the actor to fit a one-hot vector for the selected action (not the best action by the critic), but modify the loss-function value using the advantage value (possibly to influence the gradient). Is that the case?&lt;/p&gt;&#xA;" OwnerUserId="32950" LastActivityDate="2020-03-24T10:58:08.623" Title="How to set the target for the actor in A2C?" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;actor-critic-methods&gt;&lt;advantage-actor-critic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18754" PostTypeId="1" CreationDate="2020-03-22T13:46:25.487" Score="3" ViewCount="34" Body="&lt;p&gt;I'm trying to develop a better understanding of the concept of &quot;out-of-distribution&quot; (generalization) in the context of Bengio's &quot;Moving from System 1 DL to System 2 DL&quot; and the concept of &quot;(meta)-transfer learning&quot; in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These concepts seem to be very strongly related, maybe even almost referring to the same thing. So, what are &lt;strong&gt;similarities and differences&lt;/strong&gt; between these two concepts? Do these expressions refer to the same thing? If the concepts are to be differentiated from each other, what differentiates the one concept from the other and how do the concepts &lt;strong&gt;relate&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="34432" LastEditorUserId="2444" LastEditDate="2020-03-23T22:55:18.837" LastActivityDate="2020-03-23T22:55:18.837" Title="What is the difference between &quot;out-of-distribution (generalisation)&quot; and &quot;(meta)-transfer learning&quot;?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;transfer-learning&gt;&lt;meta-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18755" PostTypeId="1" CreationDate="2020-03-22T15:56:31.100" Score="2" ViewCount="67" Body="&lt;p&gt;I am currently learning neural networks. Using data from &lt;a href=&quot;http://www.mariofrank.net/touchalytics/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mariofrank.net/touchalytics/index.html&lt;/a&gt;, I am trying to predict &quot;User ID&quot; by training the neural network model shown below.  However, the model loss is increasing after every iteration. What am I doing wrong?&#xA; &lt;a href=&quot;https://i.stack.imgur.com/9fzMn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9fzMn.png&quot; alt=&quot;Loss graph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import time&#xA;import os&#xA;BATCH_SIZE=32&#xA;embedding_size=256&#xA;sequence_length=200&#xA;BUFFER_SIZE=10000&#xA;input_size=41&#xA;learning_rate=0.001&#xA;&#xA;inputs_as_tensors=tf.data.Dataset.from_tensor_slices(train_data_features_array)&#xA;targets_as_tensors=tf.data.Dataset.from_tensor_slices(train_data_labels_categorical_array)&#xA;training_data=tf.data.Dataset.zip((inputs_as_tensors,targets_as_tensors))&#xA;#training_data=training_data.batch(sequence_length,drop_remainder=True)&#xA;training_dataset=training_data.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)&#xA;print(training_dataset)&#xA;&#xA;def build_model(vocab_size,batch_size):&#xA;    modelf=tf.keras.Sequential([&#xA;        tf.keras.layers.Dense(10,activation=&quot;sigmoid&quot;,input_shape=(None,10)),&#xA;&#xA;                                tf.keras.layers.Dense(30,activation=&quot;relu&quot;,use_bias=True),  &#xA;&#xA;                                     tf.keras.layers.Dropout(0.2),                    &#xA;                                   tf.keras.layers.Dense(vocab_size)&#xA;&#xA;&#xA;                                ])&#xA;    return modelf&#xA;&#xA;def training_step(inputs,targets,optimizer):&#xA;    with tf.GradientTape() as tape:&#xA;        predictions=model(inputs)&#xA;        loss=tf.reduce_mean(tf.keras.losses.categorical_crossentropy(targets,predictions,from_logits=True))&#xA;&#xA;        grads=tape.gradient(loss,model.trainable_variables)&#xA;        optimizer.apply_gradients(zip(grads,model.trainable_variables))&#xA;        return loss,predictions&#xA;&#xA;model=build_model(input_size,BATCH_SIZE)&#xA;i=0&#xA;inner_loop=0&#xA;checkpoint_dir ='Moses_Model_x'&#xA;checkpoint_prefix = os.path.join(checkpoint_dir, &quot;ckpt_{i}&quot;)&#xA;&#xA;while(1):&#xA;    start = time.time()&#xA;    for x,y in training_dataset:&#xA;        loss,predictions=training_step(x,y,tf.keras.optimizers.RMSprop(learning_rate=0.002))&#xA;    print ('Epoch {} Loss {:.4f}'.format(i, loss))&#xA;    print ('Time taken for iteration {} is {} sec\n'.format(i,time.time() - start))&#xA;    model.save_weights(checkpoint_prefix.format(i=i))&#xA;    i=i+1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="34435" LastEditorUserId="2444" LastEditDate="2020-03-23T22:50:33.827" LastActivityDate="2020-03-23T22:50:33.827" Title="Why is the loss associated with my neural network increasing?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18756" PostTypeId="1" CreationDate="2020-03-22T20:22:34.407" Score="3" ViewCount="46" Body="&lt;p&gt;I am trying to solve the problem of an agent dynamically discovering(start with no information about the environment) the environment and to explore as much of the environment as possible without crashing into obstacles&#xA;I have the following environment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fci60.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fci60.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the environment is a  matrix. In this the obstacles are represented by 0's and the free spaces are represented with 1s. The position of the agent is given by a label such as 0.8 in the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial internal representation of the environment of the agent will look something like this with the agent position in it .&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sDy7B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sDy7B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every time it explores the environment it keeps updating its own map:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fsj2h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fsj2h.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The single state representation is just the matrix containing-&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0 for obstacles&lt;/li&gt;&#xA;&lt;li&gt;1 for unexplored regions &lt;/li&gt;&#xA;&lt;li&gt;0.8 for position of the agent&lt;/li&gt;&#xA;&lt;li&gt;0.5 for the places it has visited once&lt;/li&gt;&#xA;&lt;li&gt;0.2 for the places it has visited more than once&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want the agent to not hit the obstacles and to go around them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent should also not be stuck in one position and try to finish the exploration as quickly as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I plan to do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to prevent the agent from getting stuck in a single place, I want to punish the agent if it visits a single place multiple times. I want to mark the place the agent has visited once as 0.5 and if it has visited it more than once that place will be labelled 0.2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I am marking a place it has visited only once as 0.5 is because if there is a scenario where in the environment there is only one way to go into a region and one way to come out of that region, I don't want to punish this harshly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this problem, I am thinking of using the following reward system-&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;+1 for every time it takes an action that leads to an unexplored region&lt;/li&gt;&#xA;&lt;li&gt;-1 for when it takes an action that crashes into an obstacle&lt;/li&gt;&#xA;&lt;li&gt;0 if it visits the place twice(i.e 0.5 scenario)&lt;/li&gt;&#xA;&lt;li&gt;-0.75  is it visits a place more than twice&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The action space is just-&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;up &lt;/li&gt;&#xA;&lt;li&gt;down&lt;/li&gt;&#xA;&lt;li&gt;left&lt;/li&gt;&#xA;&lt;li&gt;right&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Am i right in approaching the problem this way? Is Reinforcement Learning the solution for this problem?&#xA;Is my representation of the state , action, reward system correct?&#xA;I am thinking that DQN is not the right way to go because the definition of a terminal state is hard in this problem, what method should I use to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="34210" LastEditorUserId="9863" LastEditDate="2020-03-24T12:13:56.917" LastActivityDate="2020-03-24T12:13:56.917" Title="Representation of state space, action space and reward system for Reinforcement Learning problem" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;policy-gradients&gt;&lt;policies&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18757" PostTypeId="2" ParentId="18608" CreationDate="2020-03-22T20:26:47.973" Score="0" Body="&lt;p&gt;Anything that is not linearly separable cant be solved perceptrons, unless you use feature maps on data to map them to a higher dimension in which it is linearly separable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple, concrete example, perceptron cant learn the XOR function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://www.computing.dcu.ie/~humphrys/Notes/Neural/single.neural.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; might help you further.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-03-22T20:26:47.973" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18758" PostTypeId="2" ParentId="18576" CreationDate="2020-03-23T00:34:02.677" Score="3" Body="&lt;p&gt;From my experience in industry, a lot of data science (operating on customer information, stored in a database) is still dominated by decision trees and even SVMs. Although neural networks have seen incredible performance on &quot;unstructured&quot; data, like images and text, there still do not appear to be great results extending to structured, tabular data (yet).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At my old company (loyalty marketing with 10 million+ members) there was a saying, &quot;&lt;em&gt;You can try any model you like, but you must try XGBoost&lt;/em&gt;&quot;. And let's just say that I did try comparing it to a neural network, and ultimately I did go with XGBoost ;) &lt;/p&gt;&#xA;" OwnerUserId="18086" LastActivityDate="2020-03-23T00:34:02.677" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18759" PostTypeId="1" CreationDate="2020-03-23T07:41:11.303" Score="3" ViewCount="200" Body="&lt;p&gt;What are the pros and cons of &lt;strong&gt;LSTM&lt;/strong&gt; vs &lt;strong&gt;Bi-LSTM&lt;/strong&gt; in language modelling? What was the need to introduce Bi-LSTM?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="9947" LastEditDate="2020-03-23T09:30:41.290" LastActivityDate="2020-03-26T12:46:49.933" Title="What are pros and cons of Bi-LSTM as compared to LSTM?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;long-short-term-memory&gt;&lt;models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18760" PostTypeId="1" AcceptedAnswerId="18761" CreationDate="2020-03-23T08:43:19.880" Score="8" ViewCount="616" Body="&lt;p&gt;Neural networks are commonly used for classification tasks, in fact from this &lt;a href=&quot;https://ai.stackexchange.com/q/18576/2444&quot;&gt;post&lt;/a&gt; it seems like that's where they shine brightest.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when we want to classify using neural networks, we often have the &lt;strong&gt;output&lt;/strong&gt; layer to take values in &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;; typically, by taking the last layer to be the &lt;em&gt;sigmoid&lt;/em&gt; function &lt;span class=&quot;math-container&quot;&gt;$x \mapsto \frac{e^x}{e^x +1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can neural networks with a sigmoid as the activation function of the &lt;strong&gt;output&lt;/strong&gt; layer approximate continuous functions? Is there an analogue to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt; for this case? &lt;/p&gt;&#xA;" OwnerUserId="31649" LastEditorUserId="2444" LastEditDate="2020-03-28T00:38:39.030" LastActivityDate="2020-03-28T01:38:23.713" Title="Can neural networks with a sigmoid as the activation function of the output layer approximate continuous functions?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;definitions&gt;&lt;computational-learning-theory&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18761" PostTypeId="2" ParentId="18760" CreationDate="2020-03-23T14:27:12.540" Score="4" Body="&lt;p&gt;As far as I know, the sigmoid is often used as the activation function of the output layer mainly because it is a &lt;em&gt;convenient&lt;/em&gt; way of producing an output &lt;span class=&quot;math-container&quot;&gt;$p \in [0, 1]$&lt;/span&gt;, which can be interpreted as a probability, although &lt;a href=&quot;https://stats.stackexchange.com/a/437769/82135&quot;&gt;that can be misleading or even wrong (if you interpret it as an uncertainty too)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You may require the output of the neural network to be a probability, for example, if you use a cross-entropy loss function, although you could in principle produce only &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;s or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;s. The probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; can then be used to decide the class (or label) of the input. For example, if &lt;span class=&quot;math-container&quot;&gt;$p &amp;gt; \alpha$&lt;/span&gt;, then you purposedly decide that the input belongs to class &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, otherwise, it belongs to class &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. The parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is called the &lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/classification/thresholding&quot; rel=&quot;nofollow noreferrer&quot;&gt;classification (or decision) threshold&lt;/a&gt;. The choice of this threshold can actually depend on the problem and it is one of the reasons people use the &lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc&quot; rel=&quot;nofollow noreferrer&quot;&gt;AUC metric&lt;/a&gt;, i.e. to avoid choosing this classification threshold.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can neural networks with a sigmoid as the activation function of the output layer approximate continuous functions? Is there an analogue to the universal approximation theorem for this case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;most famous universal approximation theorem for neural networks&lt;/a&gt; assumes that the activation functions of the units of the only hidden layer are sigmoids, but it does &lt;strong&gt;not&lt;/strong&gt; assume that the output of the network will be squashed to the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;. To be more precise, the UAT (theorem 2 of &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function&lt;/a&gt;, 1989, by G. Cybenko) states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; be any continuous sigmoidal function. Then finite sums of the&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G(x) = \sum_{j=1}^N \alpha_j \sigma (y_j^T x + \theta_j)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;are dense in &lt;span class=&quot;math-container&quot;&gt;$C(I_n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In other words, given any &lt;span class=&quot;math-container&quot;&gt;$f \in C(I_n)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\epsilon &amp;gt; 0$&lt;/span&gt;, there is a sum, &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt;, of the above form, for which&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$|G(x) - f(x)| &amp;lt; \epsilon $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the continuous function that you want to approximate, &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt; is a linear combination of the outputs of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (which should be arbitrarily big) units of the only hidden layer, &lt;span class=&quot;math-container&quot;&gt;$I_n$&lt;/span&gt; denotes the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional unit cube, &lt;span class=&quot;math-container&quot;&gt;$[0, 1]^n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$C(I_n)$&lt;/span&gt; denotes the space of continuous functions on &lt;span class=&quot;math-container&quot;&gt;$I_n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x \in I_n$&lt;/span&gt; (so the assumption is that the input to the neural network is an element of &lt;span class=&quot;math-container&quot;&gt;$[0, 1]^n$&lt;/span&gt;, i.e. a vector &lt;span class=&quot;math-container&quot;&gt;$x \in \mathbb{R}^n$&lt;/span&gt;, whose entries are between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$y_j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_j$&lt;/span&gt; are respectively the weights and bias of the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; unit. The assumption that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a real-valued function means that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; can take any value on &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$f: [0, 1]^n \rightarrow \mathbb{R}$&lt;/span&gt;). You should note that &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt; is the output of the neural network, which is a combination (where the coefficients are &lt;span class=&quot;math-container&quot;&gt;$\alpha_j$&lt;/span&gt;) of the outputs of the units in the only hidden layer, so there's no restriction on the output of &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt;, unless you restrict &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; (but, in this theorem, there's no restriction on the values &lt;span class=&quot;math-container&quot;&gt;$\alpha_j$&lt;/span&gt; can take).&lt;/p&gt;&#xA;&lt;p&gt;Of course, if you restrict the output of the neural networks to the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, you cannot approximate all continuous functions of the form &lt;span class=&quot;math-container&quot;&gt;$f: [0, 1]^n \rightarrow \mathbb{R}$&lt;/span&gt; (because not all of these functions will have the codomain &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;)! However, the sigmoid has an inverse function, i.e. the &lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/classification/thresholding&quot; rel=&quot;nofollow noreferrer&quot;&gt;logit&lt;/a&gt;, so you can reverse the output of such a neural network. So, in this sense (i.e. by reversing the output of the sigmoid), a neural network with a sigmoid as the activation function of the output layer can potentially approximate any continuous function too.&lt;/p&gt;&#xA;&lt;p&gt;The UAT above only states the existence of &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt; (i.e. it's an &lt;a href=&quot;https://en.wikipedia.org/wiki/Existence_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;existence theorem&lt;/a&gt;). It doesn't tell you how you can find &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt;. So, if you use a sigmoid as the activation function of the output layer or not is a little bit orthogonal to the universality of neural networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-28T01:38:23.713" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="18762" PostTypeId="1" CreationDate="2020-03-23T15:07:04.043" Score="1" ViewCount="251" Body="&lt;p&gt;I have this simple neural network in Python which I'm trying to use to aproximation tanh function. As inputs I have x - inputs to the function, and as outputs I want tanh(x) = y. I'm using sigmoid function also as an activation function of this neural network. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;# scipy.special for the sigmoid function expit()&#xA;import scipy.special&#xA;# library for plotting arrays&#xA;import matplotlib.pyplot&#xA;# ensure the plots are inside this notebook, not an external window&#xA;%matplotlib inline&#xA;&#xA;# neural network class definition&#xA;class neuralNetwork:&#xA;&#xA;&#xA;    # initialise the neural network&#xA;    def __init__(self, inputnodes, hiddennodes, outputnodes, learningrate):&#xA;        # set number of nodes in each input, hidden, output layer&#xA;        self.inodes = inputnodes&#xA;        self.hnodes = hiddennodes&#xA;        self.onodes = outputnodes&#xA;&#xA;        # link weight matrices, wih and who&#xA;        # weights inside the arrays are w_i_j, where link is from node i to node j in the next layer&#xA;        # w11 w21&#xA;        # w12 w22 etc &#xA;        self.wih = numpy.random.normal(0.0, pow(self.hnodes, -0.5), (self.hnodes, self.inodes))&#xA;        self.who = numpy.random.normal(0.0, pow(self.onodes, -0.5), (self.onodes, self.hnodes))&#xA;&#xA;        # learning rate&#xA;        self.lr = learningrate&#xA;&#xA;        # activation function is the sigmoid function&#xA;        self.activation_function = lambda x: scipy.special.expit(x)  &#xA;&#xA;        pass&#xA;&#xA;&#xA;    # train the neural network&#xA;    def train(self, inputs_list, targets_list):&#xA;        # convert inputs list to 2d array&#xA;        inputs = numpy.array(inputs_list, ndmin=2).T&#xA;        targets = numpy.array(targets_list, ndmin=2).T&#xA;&#xA;        # calculate signals into hidden layer&#xA;        hidden_inputs = numpy.dot(self.wih, inputs)&#xA;        # calculate the signals emerging from hidden layer&#xA;        hidden_outputs = self.activation_function(hidden_inputs)&#xA;&#xA;        # calculate signals into final output layer&#xA;        final_inputs = numpy.dot(self.who, hidden_outputs)&#xA;        # calculate the signals emerging from final output layer&#xA;        final_outputs = self.activation_function(final_inputs)&#xA;&#xA;        # output layer error is the (target - actual)&#xA;        output_errors = targets - final_outputs&#xA;        # hidden layer error is the output_errors, split by weights, recombined at hidden nodes&#xA;        hidden_errors = numpy.dot(self.who.T, output_errors) &#xA;&#xA;        # BACKPROPAGATION &amp;amp; gradient descent part, i.e updating weights first between hidden&#xA;        # layer and output layer, &#xA;        # update the weights for the links between the hidden and output layers&#xA;        self.who += self.lr * numpy.dot((output_errors * final_outputs * (1.0 - final_outputs)), numpy.transpose(hidden_outputs))&#xA;&#xA;        # update the weights for the links between the input and hidden layers, second part of backpropagation.&#xA;        self.wih += self.lr * numpy.dot((hidden_errors * hidden_outputs * (1.0 - hidden_outputs)), numpy.transpose(inputs))&#xA;        pass&#xA;&#xA;&#xA;    # query the neural network&#xA;    def query(self, inputs_list):&#xA;        # convert inputs list to 2d array&#xA;        inputs = numpy.array(inputs_list, ndmin=2).T&#xA;&#xA;        # calculate signals into hidden layer&#xA;        hidden_inputs = numpy.dot(self.wih, inputs)&#xA;        # calculate the signals emerging from hidden layer&#xA;        hidden_outputs = self.activation_function(hidden_inputs)&#xA;&#xA;        # calculate signals into final output layer&#xA;        final_inputs = numpy.dot(self.who, hidden_outputs)&#xA;        # calculate the signals emerging from final output layer&#xA;        final_outputs = self.activation_function(final_inputs)&#xA;&#xA;        return final_outputs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I try to query this network, This network has three input nodes one for each x, one node for each input. This network also has 3 output nodes, so It would classify the inputs to given outputs. Where outputs are y, y = tanh(x) function. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# number of input, hidden and output nodes&#xA;input_nodes = 3&#xA;hidden_nodes = 8&#xA;output_nodes = 3&#xA;learning_rate = 0.1&#xA;&#xA;# create instance of neural network&#xA;n = neuralNetwork(input_nodes,hidden_nodes,output_nodes, learning_rate)&#xA;&#xA;realInputs = []&#xA;realInputs.append(1)&#xA;realInputs.append(2)&#xA;realInputs.append(3)&#xA;&#xA;# for x in (-3, 3):&#xA;#     realInputs.append(x)&#xA;#     pass&#xA;&#xA;expectedOutputs = []&#xA;expectedOutputs.append(numpy.tanh(1));&#xA;expectedOutputs.append(numpy.tanh(2));&#xA;expectedOutputs.append(numpy.tanh(3));&#xA;&#xA;for y in expectedOutputs:&#xA;    print(y)&#xA;    pass&#xA;&#xA;training_data_list = []&#xA;&#xA;# epochs is the number of times the training data set is used for training&#xA;epochs = 200&#xA;&#xA;for e in range(epochs):&#xA;    # go through all records in the training data set&#xA;    for record in training_data_list:&#xA;        # scale and shift the inputs&#xA;        inputs = realInputs&#xA;        targets = expectedOutputs&#xA;        n.train(inputs, targets)&#xA;        pass&#xA;    pass&#xA;&#xA;n.query(realInputs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Outputs: desired vs ones from network with same data as training data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.7615941559557649&#xA;0.9640275800758169&#xA;0.9950547536867305&#xA;&#xA;&#xA;array([[-0.21907413],&#xA;       [-0.6424568 ],&#xA;       [-0.25772344]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My results are completely wrong. I'm a beginner with neural networks so I wanted to build neural network without frameworks like tensor flow... Could someone help me? Thank you. &lt;/p&gt;&#xA;" OwnerUserId="34440" LastEditorUserId="34440" LastEditDate="2020-03-24T10:41:02.330" LastActivityDate="2021-04-18T11:48:55.743" Title="Simple three layer neural network with backpropagation is not approximating tanh function" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18764" PostTypeId="1" CreationDate="2020-03-23T15:33:56.397" Score="1" ViewCount="60" Body="&lt;p&gt;Assume we are given a training dataset &lt;span class=&quot;math-container&quot;&gt;$D = \{ (x_i, y_i)\}_{i=1}^{N}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: which is better?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A multivariate regression with basis expansion with &lt;em&gt;independent matrix&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;em&gt;dependent matrix&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$X \in K; K \subset \mathbb R^n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y \in \mathbb R^m$&lt;/span&gt; with training data &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Or&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;A neural network which takes &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; &lt;em&gt;input variables&lt;/em&gt; and returns &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; &lt;em&gt;output&lt;/em&gt; with training data &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Without a doubt, the multivariate regression option is better with its basis polynomials because it can adapt any curve required in any dimension and doesn't need a large number of datasets than neural networks. Then, why neural networks are used more than multivariate regression?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note: Prefer explaining the mechanism of neural network used as regression in your answers. To help us know the degree of flexibility of both.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: You may prefer choosing your own loss function in case you need.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="34312" LastEditorUserId="34312" LastEditDate="2020-03-24T03:29:28.530" LastActivityDate="2020-03-24T05:01:22.333" Title="Which one is better: multivariate regression with basis expansion or neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18765" PostTypeId="2" ParentId="3753" CreationDate="2020-03-23T17:17:51.400" Score="2" Body="&lt;p&gt;Since I can't comment, there are a few caveats to previous answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if you knew beforehand what the expected boundary function for that variable was, then you could transform it first. &#xA;For instance, if you knew one feature was expected to be sinusoidal, you could transform your data (theta) using &lt;span class=&quot;math-container&quot;&gt;$f(x) = a*sin(\theta)$&lt;/span&gt; first then expect the variable to be linear. At that point it would fit a linear function and solve for the amplitude instead. Of course, in large problems, this may be impractical, but still possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise with quadratic data. I could add 1 new feature for &lt;strong&gt;every&lt;/strong&gt; continuous variable that looks like &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; and capture any quadratic dependence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another solution in your shown example would be if you &lt;strong&gt;knew&lt;/strong&gt; the data was piece-wise, you could train 3 small, easily trained, NNs for instance with different boundaries. During each new training, you set new boundaries for your functions and see which produces the best results. In your case, this would be something close to &lt;span class=&quot;math-container&quot;&gt;$[-\infty,-0.5], [-0.5,0.5]$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$[0.5,\infty]$&lt;/span&gt;.&#xA;You could argue that this is a single piece-wise neural net just like any traditional piece-wise function. &#xA;While this last approach seems very simplistic/impractical, this is actually done in practice. For instance in autonomous driving, a separate NN could be trained for object detection (human, lights, signs etc.), another for steering wheel positioning for different speeds,conditions, and turning radius, another for how to accelerate, etc. and then the system runs them all and combines their outputs into a full driving car. Each output is then its boundary of the puzzle on what region of &quot;driving space&quot; it handles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So while I would say the simple answer is no, there are still ways around it if you have some prior knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a response for the Universal Approximation Theorem, see this &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;, which says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One of the first versions of the theorem was proved by George Cybenko in 1989 for sigmoid activation functions. It was later shown that the class of deep neural networks is a universal approximator if and only if the activation function is not polynomial.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="34449" LastEditorUserId="34449" LastEditDate="2020-04-01T17:23:51.580" LastActivityDate="2020-04-01T17:23:51.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18767" PostTypeId="2" ParentId="18759" CreationDate="2020-03-23T20:35:32.760" Score="4" Body="&lt;p&gt;I would say that the logic behind the introduction was more empirical than technical. The only difference between LSTM and Bi-LSTM is the possibility for Bi-LSTM to leverage future context chunks to learn better representations of single words. There is no special training step or units added, the idea is just to read a sentence forward and backward to capture more information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And as trivial as the idea sounds, it works, in fact in the original &lt;a href=&quot;https://arxiv.org/pdf/1508.01991v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; the authors managed to achieve state-of-the-art scores in three tagging tasks, namely part-of-speech tagging, chunking and named entity recognition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though it must be said that these scores were not dramatically higher compared to other models, and also the complete architecture included a Conditional Random Field on top of the Bi-LSTM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the most important aspect to stress out is that the authors performed two interesting comparison tests: one using random embedding initialisation and another one using only words (unigrams) as input features. Under these two test conditions Bi_LSTM (with CRF on top) outperformed significantly all other architectures, proving that Bi-LSTM representations are more robust than representation learned by other models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like also to make a side note regarding human reading. It make sense to consider unidirectional sequence models as the most reasonable to emulate human reading, because we experience reading as a movement of the eyes that goes from one direction to the opposite. But the reality is that &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-030-20085-5_7&quot; rel=&quot;nofollow noreferrer&quot;&gt;saccades&lt;/a&gt; (really rapid unconscious eye movement) and other eye movements play an enormous rule in reading. Which means that also we humans do continuously look to past and future words as well in order to understand the purpose of a word or sentence we're processing. Of course in our case these movements are directed by implicit knowledge and habits that allow us to direct our attention only to important words/parts (for example we barely read conjunctions) and it is interesting to notice that now state-of-the-art models based on transformers try to learn exactly this, where to pay attention rather than single probabilities for each word in a vocabulary. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="9947" LastEditDate="2020-03-26T12:46:49.933" LastActivityDate="2020-03-26T12:46:49.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18768" PostTypeId="2" ParentId="18762" CreationDate="2020-03-24T01:26:40.380" Score="-1" Body="&lt;p&gt;This is because of &lt;strong&gt;Vanishing Gradient Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is Vanishing Gradient Problem ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when we do Back-propagation i.e moving backward in the Network and calculating gradients of loss(Error) with respect to the weights , the gradients tends to get smaller and smaller as we keep on moving backward in the Network. This means that the neurons in the Earlier layers learn very slowly as compared to the neurons in the later layers in the Hierarchy. The Earlier layers in the network are slowest to train.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reason&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sigmoid function, squishes a large input space into a small input space between 0 and 1. Therefore a large change in the input of the sigmoid function will cause a small change in the output. Hence, the derivative becomes small. &lt;a href=&quot;https://i.stack.imgur.com/vKtiQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vKtiQ.png&quot; alt=&quot;Sigmoid And its Derivative Function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use Activation function as ReLu&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dAAFA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dAAFA.png&quot; alt=&quot;ReLu&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/the-vanishing-gradient-problem-69bf08b15484&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vanishing Gradient Solution&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9863" LastActivityDate="2020-03-24T01:26:40.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18769" PostTypeId="1" CreationDate="2020-03-24T01:56:02.730" Score="1" ViewCount="83" Body="&lt;p&gt;I'd like to develop an MCTS-like (Monte Carlo Tree Search) algorithm for program induction, i.e. learning programs from examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial plan is for nodes to represent programs and for the search to expand nodes by revising programs. Each edge represents a possible revision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many expansions involve a single program: randomly resample a subtree of the program, replace a constant with a variable, etc. It's straightforward to use these with MCTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some expansions, however, generate a program from scratch (e.g. sample a new program). Others use two or more programs to generate a single output program (e.g. crossover in Genetic Programming).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These latter types of moves seem nonstandard for vanilla MCTS, but I'm not deeply familiar with the literature or what technical terms might be most relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I model expansions involving &lt;span class=&quot;math-container&quot;&gt;$N \ne 1$&lt;/span&gt; nodes? Are there accepted methods for handling these situations?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="33629" LastActivityDate="2021-04-18T05:39:36.020" Title="MCTS moves with multiple parents" Tags="&lt;monte-carlo-tree-search&gt;&lt;monte-carlo-methods&gt;&lt;computer-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18770" PostTypeId="2" ParentId="18769" CreationDate="2020-03-24T02:04:07.057" Score="1" Body="&lt;p&gt;&lt;strong&gt;Approach 1:&lt;/strong&gt; One way would be to switch from nodes representing programs to nodes representing sets of programs. The root node would represent the empty set &lt;span class=&quot;math-container&quot;&gt;${}$&lt;/span&gt;, to which expansions could be applied only if they can generate a program from scratch. The first such expansion would produce some program &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, so the root would now have child &lt;span class=&quot;math-container&quot;&gt;${p}$&lt;/span&gt;. The second expansion would produce &lt;span class=&quot;math-container&quot;&gt;$p'$&lt;/span&gt;, so the root would now also have child &lt;span class=&quot;math-container&quot;&gt;${p'}$&lt;/span&gt; which would itself also have the child ${p,p'}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One downside to this approach is that, even assuming reasonable restrictions (e.g. moves can use at most 2 programs, pairs cannot have identical elements, element order doesn't matter), the branching factor will grow combinatorially, because each new program can be combined with all (or most) of the previously generated programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 2:&lt;/strong&gt; Another way would be to switch from nodes representing programs to nodes representing meta-programs describing an initial program and a series of revisions to that program. For example, MCTS might search for expressions in something like this grammar:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Program  -&amp;gt; Empty&#xA;          | Program &amp;gt; OneProgramRevision&#xA;          | Program Program &amp;gt;&amp;gt; TwoProgramRevision&#xA;OneProgramRevision -&amp;gt; AddDatumAsException&#xA;                    | SampleStatement&#xA;                    | RegenerateSubtree&#xA;                    | Delete&#xA;                    | ...&#xA;TwoProgramRevision -&amp;gt; Concatenate&#xA;                    | Crossover&#xA;                    | ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is just a cartoon: one would need to add details parameterizing each of the various revisions.&lt;/p&gt;&#xA;" OwnerUserId="33629" LastEditorUserId="33629" LastEditDate="2020-03-24T02:21:54.120" LastActivityDate="2020-03-24T02:21:54.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18772" PostTypeId="1" CreationDate="2020-03-24T07:51:19.973" Score="2" ViewCount="35" Body="&lt;p&gt;Currently, I'm making a chatbot that is going to be functioning in a website, so I was wondering, is it better to train the chatbot with intentions files or use the database as the intention file, if it the latter, then how would I do it? With &lt;em&gt;SQLite&lt;/em&gt; or with &lt;em&gt;Excel&lt;/em&gt;? Any guides or tutorial would be appreciated. &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm planning to use Flask + Python + Html for the chatbot.&lt;/p&gt;&#xA;" OwnerUserId="33537" LastEditorUserId="33537" LastEditDate="2020-03-24T08:06:55.603" LastActivityDate="2020-03-24T09:23:08.163" Title="Is it better to rely on an intention file or a database for a web chatbot?" Tags="&lt;training&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18773" PostTypeId="1" CreationDate="2020-03-24T09:02:02.687" Score="0" ViewCount="137" Body="&lt;p&gt;I’m currently trying to train a BART, which is a denoising Transformer created by Facebook researchers. Here’s my Transformer code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;import torch&#xA;from torch import nn&#xA;from Constants import *&#xA;&#xA;class Transformer(nn.Module):&#xA;    def __init__(self, input_dim: int, output_dim: int, d_model: int = 200, num_head: int = 8, num_e_layer: int = 6,&#xA;                 num_d_layer: int = 6, ff_dim: int = 1024, drop_out: float = 0.1):&#xA;        '''&#xA;        Args:&#xA;            input_dim: Size of the vocab of the input&#xA;            output_dim: Size of the vocab for output&#xA;            num_head: Number of heads in mutliheaded attention models&#xA;            num_e_layer: Number of sub-encoder layers&#xA;            num_d_layer: Number of sub-decoder layers&#xA;            ff_dim: Dimension of feedforward network in mulihead models&#xA;            d_model: The dimension to embed input and output features into&#xA;            drop_out: The drop out percentage&#xA;        '''&#xA;        super(Transformer, self).__init__()&#xA;        self.d_model = d_model&#xA;        self.transformer = nn.Transformer(d_model, num_head, num_e_layer, num_d_layer, ff_dim, drop_out,&#xA;                                          activation='gelu')&#xA;        self.decoder_embedder = nn.Embedding(output_dim, d_model)&#xA;        self.encoder_embedder = nn.Embedding(input_dim, d_model)&#xA;        self.fc1 = nn.Linear(d_model, output_dim)&#xA;        self.softmax = nn.Softmax(dim=2)&#xA;        self.positional_encoder = PositionalEncoding(d_model, drop_out)&#xA;        self.to(DEVICE)&#xA;&#xA;    def forward(self, src: torch.Tensor, trg: torch.Tensor, src_mask: torch.Tensor = None,&#xA;                trg_mask: torch.Tensor = None):&#xA;        embedded_src = self.positional_encoder(self.encoder_embedder(src) * math.sqrt(self.d_model))&#xA;        embedded_trg = self.positional_encoder(self.decoder_embedder(trg) * math.sqrt(self.d_model))&#xA;        output = self.transformer.forward(embedded_src, embedded_trg, src_mask, trg_mask)&#xA;        return self.softmax(self.fc1(output))&#xA;&#xA;class PositionalEncoding(nn.Module):&#xA;    def __init__(self, d_model, dropout=0.1, max_len=5000):&#xA;        super(PositionalEncoding, self).__init__()&#xA;        self.dropout = nn.Dropout(p=dropout)&#xA;        pe = torch.zeros(max_len, d_model)&#xA;        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)&#xA;        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))&#xA;        pe[:, 0::2] = torch.sin(position * div_term)&#xA;        pe[:, 1::2] = torch.cos(position * div_term)&#xA;        pe = pe.unsqueeze(0).transpose(0, 1)    &#xA;        self.register_buffer('pe', pe)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and here’s my training code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def train(x: list):&#xA;    optimizer.zero_grad()&#xA;    loss = 0.&#xA;    batch_sz = len(x)&#xA;    max_len = len(max(x, key=len)) + 1  # +1 for EOS xor SOS&#xA;    noise_x = noise(x)&#xA;    src_x = list(map(lambda s: [SOS] + [char for char in s] + [PAD] * ((max_len - len(s)) - 1), noise_x))&#xA;    trg_x = list(map(lambda s: [char for char in s] + [EOS] + [PAD] * ((max_len - len(s)) - 1), x))&#xA;    src = indexTensor(src_x, max_len, IN_CHARS).to(DEVICE)&#xA;    trg = targetsTensor(trg_x, max_len, OUT_CHARS).to(DEVICE)&#xA;    names = [''] * batch_sz&#xA;&#xA;    for i in range(src.shape[0]):&#xA;        probs = transformer(src, trg[:i + 1])&#xA;        loss += criterion(probs, trg[i])&#xA;&#xA;    loss.backward()&#xA;    optimizer.step()&#xA;&#xA;    return names, loss.item()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see in the train code. I am training it &quot;sequentially&quot; by inputting the first letter of the data then computing the loss with the output then inputting the first and second character and doing the same thing, so on and so forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn’t seem to be training properly though as the denoising is totally off. I thought maybe there’s something wrong with my code or you can’t train Transformers this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm taking first name data then noising it then training the Transformer to denoise it, but the output to the Transformers doesn't look remotely like the denoised version or even the noised version of the name. I built a denoising autoencoder using LSTMs and it did way better, but I feel like BART should be way out performing LSTMs cause it's supposedly state of the art NLP neural network model.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="30885" LastEditDate="2020-04-02T11:40:21.027" LastActivityDate="2021-05-04T22:08:55.843" Title="Can you train Transformers sequentially?" Tags="&lt;natural-language-processing&gt;&lt;pytorch&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18774" PostTypeId="2" ParentId="18772" CreationDate="2020-03-24T09:23:08.163" Score="1" Body="&lt;p&gt;Recognising intents is only a small step in developing a chatbot. It's fine to use an ML classifier with training data for that, no need to keep the original list of intents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you should really think about the next step: how are you getting your bot to conduct a dialogue, rather than firing off single responses to user queries. That is where things get difficult, and that is also what distinguishes a good chatbot from a simple-minded Eliza-clone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The programming language/framework you use is not relevant.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-03-24T09:23:08.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18777" PostTypeId="2" ParentId="18753" CreationDate="2020-03-24T10:58:08.623" Score="0" Body="&lt;p&gt;In short, my last sentence was the correct answer. The &quot;target&quot; is a one-hot with the selected action, but there's a trick.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;A2C Loss Function&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A very crucial part of A2C implementation that I missed is the custom loss function that takes into account the advantage. The loss function multiplies the advantage with the negative log of the current probability to select the action that was selected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The trick is that if the advantage is negative, the loss function will switch sign, so the gradients will be applied to the opposite direction.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one dimension it's easier to understand. Let's say my target prediction is &lt;code&gt;1&lt;/code&gt; and my actual prediction is &lt;code&gt;0.6&lt;/code&gt;. A simple loss would be defined as &lt;code&gt;target - prediction&lt;/code&gt;, or in this case &lt;code&gt;0.4&lt;/code&gt; and future predictions will be closer to one. If my prediction was &lt;code&gt;1.4&lt;/code&gt;, then the loss would be &lt;code&gt;-0.4&lt;/code&gt;. A negative loss would mean predicting a lower result in the future, and a positive result would mean predicting a higher result in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the sign of the loss function is switched, the prediction will actually move away from &lt;code&gt;1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same thing happens when you multiply the advantage in the loss function. A negative advantage would mean that this action is worse than the value of the state so we need to avoid it, and a positive advantage means that the action is encouraged.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;In Keras (Tensorflow 2.0):&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here's the custom loss function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def custom_actor_loss(y_true, y_prediction, advantage):&#xA;    prediction = K.clip(y_prediction, 1e-8, 1 - 1e-8)&#xA;    log_probabilities = y_true * K.log(prediction)&#xA;&#xA;    return K.sum(-log_probabilities*advantage)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The values are clipped because log of 0 is undefined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the rest of the network building:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input_layer = Input(shape=self._state_size, name='state_in')&#xA;advantage = Input(shape=[1], name='advantage')&#xA;target_prediction = Input(shape=self._actions_num, name='target')&#xA;&#xA;inner_layer = Dense(units=layer_size, activation='relu')(input_layer)&#xA;actor_out = Dense(units=self._actions_num, activation='softmax', name='actor_out')(inner_layer)&#xA;&#xA;self._actor = Model([input_layer, target_prediction, advantage], actor_out, name='actor')&#xA;self._actor.add_loss(custom_actor_loss(actor_out, target_prediction, advantage))&#xA;self._actor.compile(optimizer=Adam(learning_rate=actor_learning_rate))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And in the training loop (where &lt;code&gt;future_rewards_prediction&lt;/code&gt; and &lt;code&gt;critic_prediction&lt;/code&gt; are the outputs from the critic for the current and next state (except for the terminal state where the &lt;code&gt;future_rewards_prediction&lt;/code&gt; is set to &lt;code&gt;0&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Train actor&#xA;target_probabilities = np.zeros([1, self._actions_num])&#xA;target_probabilities[0][memory[step_idx].action] = 1.0&#xA;advantage = memory[step_idx].reward + future_rewards_prediction * self._future_discount - critic_prediction&#xA;self._actor.fit((memory[step_idx].state, target_probabilities, advantage), verbose=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;*Notice how I don't really specify a &lt;code&gt;y&lt;/code&gt; in my &lt;code&gt;fit&lt;/code&gt; call. This is because of an issue I had when trying to implement a custom loss function in Keras which was solved by &lt;a href=&quot;https://stackoverflow.com/a/58730735/6104191&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32950" LastActivityDate="2020-03-24T10:58:08.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18778" PostTypeId="1" AcceptedAnswerId="20823" CreationDate="2020-03-24T11:16:24.923" Score="0" ViewCount="54" Body="&lt;p&gt;A2C loss is usually defined as &lt;code&gt;advantage * (-log(actor_predictions)) * target&lt;/code&gt; where &lt;code&gt;target&lt;/code&gt; is a one-hot vector (with some clipping/noise/etc...) with the selected target. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that we get larger losses for smaller mistakes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for example the agent has predicted &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)=0.9$&lt;/span&gt; but the advantage is negative, this would mean a larger mistake than if the agent predicted that &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)=0.1$&lt;/span&gt;, however, putting the numbers in the formula means a larger loss for the &lt;code&gt;0.1&lt;/code&gt; prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming &lt;code&gt;advantage=-1&lt;/code&gt;, &lt;code&gt;advantage * (-log(actor_predictions)) * target&lt;/code&gt; would mean:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;-1 * (-log(0.9)) * 1 = log(0.9)=-0.045&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;-1 * (-log(0.1)) * 1 = log(0.1)=-1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="32950" LastActivityDate="2020-05-01T13:06:35.037" Title="Is A2C loss function taking smaller steps for larger mistakes?" Tags="&lt;loss-functions&gt;&lt;policy-gradients&gt;&lt;advantage-actor-critic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18779" PostTypeId="1" CreationDate="2020-03-24T16:02:04.693" Score="2" ViewCount="20" Body="&lt;p&gt;Consider a stochastic process &lt;span class=&quot;math-container&quot;&gt;$\{X_t \colon t \in T\}$&lt;/span&gt; indexed by a set &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. We assume for simplicty that &lt;span class=&quot;math-container&quot;&gt;$T \in \mathbb{R}^n$&lt;/span&gt;. We assume that for any choice of indexes &lt;span class=&quot;math-container&quot;&gt;$t_1, \dots, t_n$&lt;/span&gt;, the random variables &lt;span class=&quot;math-container&quot;&gt;$(X_{t_1}, \dots, X_{t_n})$&lt;/span&gt; are jointly distributed according to a multivariate Gaussian distribution with mean &lt;span class=&quot;math-container&quot;&gt;$\mu = (0, \dots, 0)$&lt;/span&gt;, for a given covariance matrix &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under this assumptions, the stochastic process is completely determined by the 2nd-order statistics. Hence, if we assume fixed mean at &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, then the stochastic process is fully defined by the covariance matrix. This matrix can be defined in terms of the covariance function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k(t_i, t_j) = \mbox{cov}(X_{t_i}, X_{t_j}). &#xA;$$&lt;/span&gt;&#xA;It is well-known that functions &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; as defined above are admissible Kernels. This fact is often used in probabilistic inference, when performing regression or classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several functions can be suitable kernels, but only a few are used in practise, depending on the application. Given the large amount of related literature, can someone provide an up-to-date list of functions commonly used as kernels in this context?&lt;/p&gt;&#xA;" OwnerUserId="34464" LastActivityDate="2020-03-24T16:02:04.693" Title="Suitable kernels for Gaussian processes" Tags="&lt;machine-learning&gt;&lt;theory&gt;&lt;gaussian-process&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18780" PostTypeId="1" CreationDate="2020-03-24T17:00:01.710" Score="0" ViewCount="39" Body="&lt;p&gt;I have to perform a regression on three curves as shown in the following plot. Here, &lt;code&gt;accA&lt;/code&gt; (y-axis) is the dependent variable, and &lt;code&gt;w&lt;/code&gt; (x-axis) is the independent variable. The sum of the three curves adds up to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To perform regression for the three curves, I would like to use a neural network. What architecture/set-up should I use for this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h6V3C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h6V3C.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34464" LastEditorUserId="2444" LastEditDate="2020-03-24T20:01:06.057" LastActivityDate="2020-03-24T20:01:06.057" Title="Non-linear regression with a neural network" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18782" PostTypeId="2" ParentId="18594" CreationDate="2020-03-24T19:17:58.540" Score="0" Body="&lt;p&gt;The hill-climbing algorithm to implement is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The algorithm should take four inputs: as always, there will be a multiset S and integer&#xA;k, which are the &lt;code&gt;Subset&lt;/code&gt; and &lt;code&gt;Sum&lt;/code&gt; for the Subset Sum problem; in addition, there&#xA;will be two integers q and r, with roles defined below.&lt;/li&gt;&#xA;&lt;li&gt;Do the following q times:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(a) Choose a random subset (multiset) &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt; of S as the &lt;code&gt;current&lt;/code&gt; subset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) Do the following (&lt;code&gt;hill climbing&lt;/code&gt;) r times:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i. Find a random neighbor T (see definition of neighbor below) of the current&#xA;subset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii. If neighbor T has smaller residue, then make T the current subset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(c) Keep track of the residue of the final current subset when starting with subset&#xA;&lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Return the smallest residue of the q subsets tested by the algorithm.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Definition&lt;/strong&gt;: Subset (multiset) B ⊆ S is a neighbor of a subset A of S if you can transform&#xA;A into B by moving one or two integers from A to B, or by moving one or two integers from&#xA;B to A, or by swapping one integer in A with one integer in B.&#xA;An easy way to generate a random neighbor B of a subset A of S is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Order the elements of S as &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, ..., x_n$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Initialize B to be a clone of A.&lt;/li&gt;&#xA;&lt;li&gt;Choose two distinct random indices i and j, where &lt;span class=&quot;math-container&quot;&gt;$1 ≤ i; j ≤ n$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;if &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is in A, remove it from B. Otherwise, add xi to B.&lt;/li&gt;&#xA;&lt;li&gt;if &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt; is in A, then with probability 0.5, remove it from B. If &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt; is not in A, then with&#xA;probability 0.5, add &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt; to B.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33875" LastActivityDate="2020-03-24T19:17:58.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18783" PostTypeId="1" AcceptedAnswerId="24472" CreationDate="2020-03-24T20:21:58.960" Score="2" ViewCount="45" Body="&lt;p&gt;I am training an AlexNet neural network, with about 12000 images which 80% is for training, 10% is for validation and another 10% is for testing.&#xA;I have a problem in my plots. There is a big fluctuation in epoch 47,&#xA;how can I have a smooth plot? What is the problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qsxMe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qsxMe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to increase my validation data because the fluctuation was for validation loss. but nothing changed.&#xA;I decreased learning rate, but it stucks in a local optimum.&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="2444" LastEditDate="2020-11-07T11:27:50.923" LastActivityDate="2020-11-07T11:27:50.923" Title="What could cause a big fluctuation of the loss in the last epochs of training an AlexNet?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;alexnet&gt;&lt;data-augmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18785" PostTypeId="1" AcceptedAnswerId="18796" CreationDate="2020-03-24T23:28:53.160" Score="4" ViewCount="164" Body="&lt;p&gt;I know they are not the same in working, but an input layer sends the input to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed to the next layer.&lt;/p&gt;&#xA;&lt;p&gt;Aren't the filters the same, in the way that they convert an &amp;quot;image&amp;quot; to a new &amp;quot;image&amp;quot; based on the weights that are in that filter? And that the next layer uses this new &amp;quot;image&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="34359" LastEditorUserId="2444" LastEditDate="2021-01-19T15:53:47.460" LastActivityDate="2021-05-15T11:29:20.787" Title="Can neurons in MLP and filters in CNN be compared?" Tags="&lt;convolutional-neural-networks&gt;&lt;multilayer-perceptron&gt;&lt;artificial-neuron&gt;&lt;weights&gt;&lt;filters&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18787" PostTypeId="1" CreationDate="2020-03-25T01:18:06.103" Score="2" ViewCount="82" Body="&lt;p&gt;There are (two-players, perfect information) combinatorial games for which, at any configuration of the game, a winning move (if there is one) can be quickly computed by a short program. This is the case of the following game that starts with a bunch of matches and each player alternatively removes 1,2 or 3 matches, until the player that removes the last one wins. &#xA;This is also the case of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nim&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nim game&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, understanding the winning strategy of games like Go or chess seems hopeless. &#xA;However, some machine-learning based programs like alphaGo zero are able &quot;learn the strategy&quot; of complex games, using as input data only the rules of the game. I don't really know how these algorithms work but here is my vague question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;For simple game like Nim, can such an algorithm be able to actually find a winning move in any winning configuration of the game ?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of configurations of Nim is infinite, but the algorithm will consider during its &quot;training&quot; only a finite number of configurations. It seems imaginable that if this training phase is long enough, then the program will be able to capture the winning strategy, like a human would do.&lt;/p&gt;&#xA;" OwnerUserId="34474" LastEditorUserId="34474" LastEditDate="2020-03-25T16:38:43.123" LastActivityDate="2020-03-25T20:41:18.533" Title="Can an artificial intelligence be unbeatable at simple games?" Tags="&lt;machine-learning&gt;&lt;combinatorial-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18788" PostTypeId="2" ParentId="18671" CreationDate="2020-03-25T02:47:24.907" Score="2" Body="&lt;p&gt;I have not found any simple implementation of a naive EBMT system, but I found some articles, papers and books that may be helpful (although I haven't read them, apart from the first and last one), so I will list them below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The web article &lt;a href=&quot;https://www.translationdirectory.com/articles/article1746.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example-based machine translation&lt;/a&gt; provides a decent high-level explanation of example-based machine translation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://www.semanticscholar.org/paper/Example-Based-Machine-Translation-%3A-A-New-Paradigm-Pan/c896ad64c21053cde78fa69c81e99ade37886196&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example-Based Machine Translation: A New Paradigm&lt;/a&gt; (2002) by Chunyu Kit et al. also seems to provide a detailed description of the EBMT approach, so this paper should provide you with details you need to implement an EBMT system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://www.mt-archive.info/Nagao-1984.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A framework of a mechanical translation between Japanese and English by analogy principle&lt;/a&gt; (1984) by Makoto Nagao introduced the example-based machine translation approach, so it will be at least historically relevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, the paper &lt;a href=&quot;http://www.mt-archive.info/05/AMTA-2006-Stroppa.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example-Based Machine Translation of the Basque Language&lt;/a&gt; and the book &lt;a href=&quot;https://www.springer.com/de/book/9781402014000#aboutBook&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recent Advances in Example-Based Machine Translation&lt;/a&gt; (2003), which is not apparently freely available online, could also be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the article &lt;a href=&quot;https://translartisan.wordpress.com/tag/example-based-machine-translation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Translation. From the Cold War to Deep Learning&lt;/a&gt; gives a nice high-level overview of the main machine translation approaches, so that you can understand the differences between EBMT and other approaches (especially, in case you are not able to distinguish between EBMT and other MT, e.g. those that use a parallel corpus, such as the supervised statistical machine translation approaches).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-25T11:13:08.163" LastActivityDate="2020-03-25T11:13:08.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18790" PostTypeId="1" CreationDate="2020-03-25T08:04:53.837" Score="2" ViewCount="26" Body="&lt;p&gt;I am new to this topic, researched a lot, but still, I have no clue which way might be the right one. As I do not have previous Expertise on this, it is very difficult for me to even find out the start as there is no such tutorial or something similair + I REALLY HAVE NO CLUE about how I could google this kind of stuff as it is very specific and I am therefore not able to find a &quot;parent&quot; topic to google for, except AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I search here for tipps, articles, links, insights of some of you guys. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to achive is the following:&#xA;I have a dataset of users which each have multiple relevant values and a List of Users. (e.x. ValA, ValB, ValC, List)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand is a rank system. It has many ranks which require the user to have a specific score value each month. But it also requires their List to have a specific score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Example:&#xA;UserA has a scoreA of 1000.&#xA;UserA has in his list UserB and UserC.&#xA;UserB has a scoreA of 50.&#xA;UserC has a scoreA of 150&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore UserA has a scoreA of 1000 and a scoreB of 200 (50+150)-&gt; scoreC would be 1200.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Until now it is still easy to accomplish even without AI. But now comes to hard part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ranks:&#xA;- R1 requires 500 scoreC&#xA;- R2 requires 1000 scoreC + at least 1 User in the List which reached R1 in this month.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the use case:&#xA;The AI should be able to tell a entered user his next steps / possible best strategy for highest possible rank. &#xA;For example UserA -&gt; Should reach out to userC and tell him to play more in order to reach 500 scoreA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you now would add more ranks and more users, the best strategy for userA would be to notify userX because UserX has UserY which could reach a new Rank by notifying UserZ, which then would give a new rank to Y, to X, to A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is all about finding out which actions have to be taken to get to the next rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&#xA;TheMrone&lt;/p&gt;&#xA;" OwnerUserId="34481" LastActivityDate="2020-03-25T08:04:53.837" Title="How to create a best strategy AI for a specific complex ruleset?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18791" PostTypeId="1" CreationDate="2020-03-25T10:41:54.867" Score="2" ViewCount="41" Body="&lt;p&gt;I've just started learning AI and I have a certain problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have directed connected graph, where nodes are places one can go to and edges are the &quot;roads&quot; between places. We have K agents whose goal is to meet in one node. Agents start in different nodes. Note that more than one agent can be at one node at the same time and that all agents move by one node at every turn(they move synchronously).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have to variants of this task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) in each turn every agent must move&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) an agent may pass on moving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a chosen variant I have to find an algorithm to complete this task, but it cannot be the searching state-space algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been sitting on this for a while but I cannot think of anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been thinking if agents could know each other positions in order to choose where to go, but it's searching state space. I also thought that if agents meet, they could continue together. But I'm looking for an alternative to searching state-space algorithms.&lt;/p&gt;&#xA;" OwnerUserId="34483" LastActivityDate="2021-04-20T05:58:58.523" Title="Agents meeting in a directed connected graph" Tags="&lt;search&gt;&lt;intelligent-agent&gt;&lt;heuristics&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18792" PostTypeId="1" CreationDate="2020-03-25T10:49:48.387" Score="1" ViewCount="30" Body="&lt;p&gt;I've been reading some papers on human pose estimation and I'm starting to see the terms top-down and bottom-up crop up a lot. For example in &lt;a href=&quot;https://arxiv.org/pdf/1603.06937.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Our hourglass module differs from these designs mainly&#xA;  in its more symmetric distribution of capacity between bottom-up processing&#xA;  (from high resolutions to low resolutions) and top-down processing (from low&#xA;  resolutions to high resolutions)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Okay, so what are the main observations or distinctions of top-down vs bottom-up? Why does it make sense to have a paradigm in which we talk about these specifically?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-03-25T10:49:48.387" Title="What are the main points of the top-down vs bottom-up paradigm in neural networks?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18793" PostTypeId="1" CreationDate="2020-03-25T10:55:16.847" Score="2" ViewCount="26" Body="&lt;p&gt;I used to work as an analyst in a financial project where we had functions &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; determining the price, and sometimes the inputs &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; jumped in such a way to produce anomalous results.&#xA;We had to report an explanation, and I wish to automate the process.&#xA;It's not properly a question of AI, more of information science.&#xA;The idea is that once, for a generic non-linear &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, you can determine the ranking of relevance of &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; in explaining the result, you can generate a full explanation by:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;decompose &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; as a composition of &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt;, which are intermediate results with a definite meaning in the application domain (in this case, finance)&lt;/li&gt;&#xA;&lt;li&gt;apply the algorithm using the &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, and then iterate it to explain the &lt;span class=&quot;math-container&quot;&gt;$f_j$&lt;/span&gt; in terms of &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The relevance is quantified by the information gain of each variable. This will be explained for an application in ranking the &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; directly.&#xA;We assume to start on a uniform distribution on the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; domain, calculate the derived probability density function for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, and the information entropy of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&#xA;Then we fix the &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; one at a time, for each calculate the new p.d.f. of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; conditioned on that &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and the (lower) information entropy of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The information gain is &lt;span class=&quot;math-container&quot;&gt;$IG(x_i)$&lt;/span&gt;.&#xA;Choose as the first conditioning the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; with the largest information gain, then condition of the remaining &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; with a decreasing order of &lt;span class=&quot;math-container&quot;&gt;$IG_i$&lt;/span&gt;.&#xA;So we could start for example , with &lt;span class=&quot;math-container&quot;&gt;$(x_1,x_2,x_3)$&lt;/span&gt;, to condition first on &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;, then on &lt;span class=&quot;math-container&quot;&gt;$(x_2,x_3)$&lt;/span&gt;, and then on &lt;span class=&quot;math-container&quot;&gt;$(x_2,x_3,x_1)$&lt;/span&gt;, getting the percentage contributions as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{IG_i}{H_y}$&lt;/span&gt;. The successive terms &lt;span class=&quot;math-container&quot;&gt;$IG_i$&lt;/span&gt; add up always to the total entropy &lt;span class=&quot;math-container&quot;&gt;$H_y$&lt;/span&gt;, since conditioning on all variables gives a point and zero entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any opinion on how to improve this &quot;automated function explanation&quot; is welcome&lt;/p&gt;&#xA;" OwnerUserId="34484" LastActivityDate="2020-03-25T10:55:16.847" Title="Automated explanation - function results - simple attempt" Tags="&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18794" PostTypeId="1" CreationDate="2020-03-25T11:07:13.930" Score="0" ViewCount="91" Body="&lt;p&gt;I am following the NLP course taught by Dan Jurafsky. In the video lectures &lt;a href=&quot;https://www.youtube.com/watch?v=a48vIACEC_w&amp;amp;list=PLQiyVNMpDLKnZYBTUOlSI9mi9wAErFtFm&amp;amp;index=50&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supervised Relation Extraction&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=Jxqx70iX9kc&amp;amp;list=PLQiyVNMpDLKnZYBTUOlSI9mi9wAErFtFm&amp;amp;index=51&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semi Supervised and Unsupervised Relation Extraction&lt;/a&gt; Jurafsky explains supervised, semi-supervised and unsupervised relation extraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what are the pros and cons of every relation extraction method compared with the other two relation extraction methods?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-25T14:30:39.263" LastActivityDate="2021-04-22T17:07:15.693" Title="What are the pros and cons of supervised, semi-supervised and unsupervised relation extraction in NLP?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;&lt;semi-supervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18796" PostTypeId="2" ParentId="18785" CreationDate="2020-03-25T13:31:43.563" Score="3" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; The equivalent to a neuron in a Fully-Connected (FC) layer is the &lt;strong&gt;kernel&lt;/strong&gt; (or filter) of a Convolution layer&lt;/p&gt;&#xA;&lt;h3&gt;Differences&lt;/h3&gt;&#xA;&lt;p&gt;The neurons of these two types of layers have two key differences. These are that the convolution layers implement:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Sparse connectivity&lt;/strong&gt;, i.e. each neuron is connected only to an area of the input, not the whole.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Weight sharing&lt;/strong&gt;, i.e. &lt;em&gt;similar&lt;/em&gt; connections end up having the same weights. This is usually visualized as the same filter traversing the image.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Besides these two key differences, there are some other technical details, e.g. &lt;a href=&quot;https://stackoverflow.com/a/45140686/5668710&quot;&gt;how the biases are implemented&lt;/a&gt;. Other than that they perform the same operation.&lt;/p&gt;&#xA;&lt;p&gt;What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.&lt;/p&gt;&#xA;&lt;h3&gt;Sparse connectivity&lt;/h3&gt;&#xA;&lt;p&gt;On the left are two FC neural networks. On the right, are layers with sparse connections.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bpOW7.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Weight sharing&lt;/h3&gt;&#xA;&lt;p&gt;On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/cnYPq.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer your other questions:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are filters not the same in the way that they convert an &amp;quot;image&amp;quot; to a new &amp;quot;image&amp;quot; based on the weights that are in that filter? And that the next layers use these new &amp;quot;images&amp;quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.&lt;/p&gt;&#xA;&lt;p&gt;However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D convolutional layers as well.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="2444" LastEditDate="2021-05-15T11:29:20.787" LastActivityDate="2021-05-15T11:29:20.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18797" PostTypeId="1" CreationDate="2020-03-25T14:11:51.167" Score="2" ViewCount="56" Body="&lt;p&gt;The book &lt;a href=&quot;https://cvrs.whu.edu.cn/downloads/ebooks/Multiple%20View%20Geometry%20in%20Computer%20Vision%20(Second%20Edition).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiple View Geometry in Computer Vision&lt;/a&gt; by Richard Hartley and Andrew Zisserman talks about lines, points and conics. A conic is a curve described by a second-degree equation in the plane, so a parabola would be an example of a conic. The purpose and usage of points and lines in computer vision are quite clear. For example, a set of points defines an object, or we can project a 3-dimensional point to a 2-dimensional point in the image plane, or a line represents the space to look for the corresponding point in the second plane of another point in the first plane (epipolar geometry). However, probably because I haven't yet read the part of the book related to the applications of conics in computer vision, it's not clear why do we even care about conics in computer vision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, why are conics important in computer vision? Note that I know that conics are defined by points and, given the point-line duality, they can also be defined by lines, but this doesn't still enlightens me on the purpose of conics in computer vision. So, I am looking for applications where conics are used to define the underlying CV model, in a similar way that points and lines are used to describe the pinhole camera model.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-25T14:26:53.637" LastActivityDate="2020-03-25T14:26:53.637" Title="Why are conics important in computer vision?" Tags="&lt;computer-vision&gt;&lt;math&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18798" PostTypeId="1" AcceptedAnswerId="18833" CreationDate="2020-03-25T14:46:24.230" Score="5" ViewCount="293" Body="&lt;p&gt;What are some (good) online courses for deep reinforcement learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like the course to be both programming and theoretical. I really liked &lt;a href=&quot;https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-&quot; rel=&quot;noreferrer&quot;&gt;David Silver's course&lt;/a&gt;, but the course dates from 2015. It doesn't really teach deep Q-learning at this time.&lt;/p&gt;&#xA;" OwnerUserId="34488" LastEditorUserId="2444" LastEditDate="2020-03-26T14:24:28.777" LastActivityDate="2020-08-14T14:08:22.227" Title="What are some online courses for deep reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18799" PostTypeId="1" AcceptedAnswerId="18804" CreationDate="2020-03-25T14:47:02.797" Score="1" ViewCount="119" Body="&lt;p&gt;For the A* algorithm, a consistent heuristic is defined as follows:&#xA;if, for every node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and every successor &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; generated by any action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h(n) \leq c(n,m) + h(m)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the edge &lt;span class=&quot;math-container&quot;&gt;$c(n,m)$&lt;/span&gt; is removed, how do we define consistency for nodes &lt;span class=&quot;math-container&quot;&gt;$n, n'$&lt;/span&gt; ? Since the &lt;span class=&quot;math-container&quot;&gt;$n'$&lt;/span&gt; is not generated from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="16909" LastEditDate="2020-03-26T22:40:14.407" LastActivityDate="2020-03-28T03:46:11.157" Title="What does a consistent heuristic become if an edge is removed in A*?" Tags="&lt;search&gt;&lt;a-star&gt;&lt;consistent-heuristic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18800" PostTypeId="2" ParentId="18794" CreationDate="2020-03-25T16:09:54.393" Score="0" Body="&lt;h2&gt;Supervised&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;highest accuracy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;need a large human-labeled training set&lt;/li&gt;&#xA;&lt;li&gt;brittle (doesn't work well with examples that are in a different genre from the training set)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Semi-supervised&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Relation bootstrapping&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;only requires a small set of labeled data (seed relations)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;complex iterative process&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Distant supervision&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;training happens in one go (no iterative process)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;requires a big database of relations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Unsupervised&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;don't need any labeled data&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;need to process a huge quantity of unlabeled data (usually web crawling)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="34485" LastEditorUserId="34485" LastEditDate="2020-03-28T15:50:20.337" LastActivityDate="2020-03-28T15:50:20.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18802" PostTypeId="1" CreationDate="2020-03-25T19:49:56.347" Score="1" ViewCount="52" Body="&lt;p&gt;Is there any known error bounds for the TD(0) algorithm for the value function after a finite number of iterations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Delta_t=\max_{s \in \mathcal{S}}|v_t(s)-v_\pi(s)|$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_{t+1}(s_t)=v_t(s_t)+\alpha(r+v_t(s_{t+1})-v_t(s_t))$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="33227" LastActivityDate="2020-09-12T22:00:40.150" Title="Are there known error bounds for TD(0) with a constant learning rate?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18803" PostTypeId="2" ParentId="18787" CreationDate="2020-03-25T20:24:28.710" Score="2" Body="&lt;p&gt;There is actually &lt;a href=&quot;https://github.com/abelmariam/nimPy&quot; rel=&quot;nofollow noreferrer&quot;&gt;a github project about 'solving' Nim&lt;/a&gt; that implements certain type of &lt;code&gt;Q-learning reinforcement algorithm&lt;/code&gt; (&lt;em&gt;described in undergraduate thesis of Erik Jarleberg (Royal Institute of Technology) entitled &quot;Reinforcement learning on the combinatorial game of Nim&quot;&lt;/em&gt;) that supposedly finds that optimal strategy lying down there inside in the game that also human can find out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The project uses a code in Python, which is 'only' 114 lines of code, so it can be run on your own machine if you got interested and want to test it. Github page tells also:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There is a known optimal strategy for the game which is credited to Charles L. Bouton of Harvard which can serve as a benchmark for evaluating the performance on our Q-learning agent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(That is the quote I refer for it getting enough good / optimal results).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Q-learning&lt;/code&gt; is part of the reinforcement learning algorithm family where the so called &lt;code&gt;agent&lt;/code&gt; learns how to play the game by getting &lt;code&gt;rewards&lt;/code&gt; from its actions and it tries to maximize the amount of &lt;code&gt;total reward&lt;/code&gt; with following strategy:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The goal of the agent is to maximize its total reward. It does this by adding the maximum reward attainable from future states to the reward for achieving its current state, effectively influencing the current action by the potential future reward. This potential reward is a weighted sum of the expected values of the rewards of all future steps starting from the current state. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Quote and more information in &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning Wikipedia-page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further interest on easy games tackled with Reinforcement Learning, please have a look on following paper: &lt;a href=&quot;https://deepmind.com/research/publications/playing-atari-deep-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt;. There is also &lt;a href=&quot;https://www.udemy.com/course/reinforcement-learning-with-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an Udemy class on that paper and its findings&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2020-03-25T20:41:18.533" LastActivityDate="2020-03-25T20:41:18.533" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18804" PostTypeId="2" ParentId="18799" CreationDate="2020-03-25T23:28:41.570" Score="2" Body="&lt;p&gt;Consistency is a property of heuristics. You can think of consistency as the common sense idea that our guess at the time to go from &lt;span class=&quot;math-container&quot;&gt;$A \rightarrow B \rightarrow C$&lt;/span&gt; cannot be more than the time to go from &lt;span class=&quot;math-container&quot;&gt;$A \rightarrow B$&lt;/span&gt;, plus our guess of the time to go from &lt;span class=&quot;math-container&quot;&gt;$B \rightarrow C$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing we remove a given edge &lt;span class=&quot;math-container&quot;&gt;$c(n,m)$&lt;/span&gt; from our graph, but that our heuristic function gives the same value for &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; as before. Can the function be inconsistent? Let's try a proof sketch:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Since the function was consistent before we removed this edge, it must already be the case that &lt;span class=&quot;math-container&quot;&gt;$h(n) &amp;lt; c(n, m') + h(m')$&lt;/span&gt; for every other node &lt;span class=&quot;math-container&quot;&gt;$m'$&lt;/span&gt; adjacent to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Removing the edge &lt;span class=&quot;math-container&quot;&gt;$c(n,m)$&lt;/span&gt; doesn't change these relationships, because the other edges still have the same costs as before, and the heuristic function still gives the same value for every node.&lt;/li&gt;&#xA;&lt;li&gt;The same argument may be applied to node &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Since none of these relationships have changed, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is still consistent.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="2444" LastEditDate="2020-03-28T03:46:11.157" LastActivityDate="2020-03-28T03:46:11.157" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18805" PostTypeId="2" ParentId="17803" CreationDate="2020-03-25T23:53:58.227" Score="0" Body="&lt;p&gt;(All notations based on &lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding ML: From Theory to Algorithms&lt;/a&gt;) The layman's term for NFL is super misleading. The comparison between PAC learnability and NFL is kind of baseless since both proof's are built on a different set of assumptions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's review the definition of PAC learnability:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A hypothesis class &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is PAC learnable if there exist a function &lt;span class=&quot;math-container&quot;&gt;$m_H : (0, 1)^ 2 → N$&lt;/span&gt; and a learning algorithm with the&#xA;  following property: For every &lt;span class=&quot;math-container&quot;&gt;$\epsilon, \delta \in (0, 1)$&lt;/span&gt;, for every distribution &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; , and for every labelling function &lt;span class=&quot;math-container&quot;&gt;$f : X → {0, 1}$&lt;/span&gt;, if the realizable assumption holds with respect to &lt;span class=&quot;math-container&quot;&gt;$H, D, f$&lt;/span&gt; , then when running the learning algorithm on &lt;span class=&quot;math-container&quot;&gt;$m ≥ m_H (\epsilon, \delta)$&lt;/span&gt; i.i.d. examples generated by &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and labeled by &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; , the algorithm returns&#xA;  a hypothesis &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; such that, with probability of at least &lt;span class=&quot;math-container&quot;&gt;$1 − δ$&lt;/span&gt; (over the choice of the examples), &lt;span class=&quot;math-container&quot;&gt;$L_{(D,f )} (h) ≤ \epsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An important point in this definition is that the complexity bounds (i.e value of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;) holds irrespective of distribution &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; (this is known as distribution free). Since, in the proofs we assume error to be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; i.e if &lt;span class=&quot;math-container&quot;&gt;$f(x) \neq h(x)$&lt;/span&gt; then we assign error &lt;span class=&quot;math-container&quot;&gt;$=1$&lt;/span&gt; so &lt;span class=&quot;math-container&quot;&gt;$L_D(A(S))$&lt;/span&gt; which is defined as the true probability of error by the classifier (&lt;span class=&quot;math-container&quot;&gt;$A(S) = h_S$&lt;/span&gt;) will be the same as &lt;span class=&quot;math-container&quot;&gt;$\Bbb E_{S \sim D^{m}}(h_S)$&lt;/span&gt;. Also, the realizable assumption is not very important here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's review the definition of NFL:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; be any learning algorithm for the task of binary classification with respect to the &lt;span class=&quot;math-container&quot;&gt;$0 − 1$&lt;/span&gt; loss over a domain &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; . Let &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; be any number smaller than &lt;span class=&quot;math-container&quot;&gt;$|X |/2$&lt;/span&gt;, representing a training set size. Then, there&#xA;  exists a distribution &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$X × \{0, 1\}$&lt;/span&gt; such that:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There exists a function &lt;span class=&quot;math-container&quot;&gt;$f : X → \{0, 1\}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$L_{D} (f ) = 0$&lt;/span&gt; (i.e.Realizable).&lt;/li&gt;&#xA;&lt;li&gt;With probability of at least &lt;span class=&quot;math-container&quot;&gt;$1/7$&lt;/span&gt; over the choice of &lt;span class=&quot;math-container&quot;&gt;$S \sim D^m$&lt;/span&gt; we have that&#xA;&lt;span class=&quot;math-container&quot;&gt;$L_D (A(S)) ≥ 1/8$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; For second statement it suffices to show that &lt;span class=&quot;math-container&quot;&gt;$\Bbb E_{S \sim D^{m}}L_D(A'(S)) \geq 1/4$&lt;/span&gt;, which can be shown using Markov's Inequality. Also, the definition is implying we consider all functions possible from &lt;span class=&quot;math-container&quot;&gt;$X × \{0, 1\}$&lt;/span&gt; and our learning algorithm can pick any function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; out of this, which somewhat implies that the set &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; has been shattered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you read the definition it clearly states there exists a &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, which is clearly different from distribution free assumption of PAC learnability. Also to note that we are restricting sample size &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$|X |/2$&lt;/span&gt;. You will be able to falsify the second statement by simply picking bigger &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and thus your class is suddenly PAC learnable. Thus the point NFL is trying to make is that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Without an inductive bias i.e if you pick all possible functions from &lt;span class=&quot;math-container&quot;&gt;$f : X → {0, 1}$&lt;/span&gt; as your hypothesis class you would not be able to achieve &lt;strong&gt;for all &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;&lt;/strong&gt; an accuracy less than &lt;span class=&quot;math-container&quot;&gt;$1/8$&lt;/span&gt; with probability greater than &lt;span class=&quot;math-container&quot;&gt;$6/7$&lt;/span&gt; given your sample size is at most &lt;span class=&quot;math-container&quot;&gt;$|X|/2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To prove this, you only have to pick a distribution for which this holds. In the proof of the &lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; they have used the uniform distribution which is the margin between 2 types of distribution. So the idea is lets say you have sampled &lt;span class=&quot;math-container&quot;&gt;$m = \frac{|X|}{2}$&lt;/span&gt; points, your learning algorithm returns a hypothesis as per ERM rule (doesn't really matter) on the sampled points. Now you want to comment on the error over &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; points and true distribution (uniform distribution in this case). So clearly, the probability of picking a point outside your sampled points (unseen points) is &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;. Also, the &lt;span class=&quot;math-container&quot;&gt;$A(S) = h_S$&lt;/span&gt; will have a &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; probability of agreeing with the actual label of an unseen point (among all &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; which agree with the sampled points, half will assign &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to an unseen point while other half will assign &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;), which makes the total probability of making an error&lt;span class=&quot;math-container&quot;&gt;$=0.25$&lt;/span&gt; over the true distribution or &lt;span class=&quot;math-container&quot;&gt;$\Bbb E_{S \sim D^{m}}L_D(A(S)) = 1/4$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that we have picked up uniform distribution but this will also hold for distributions which assigns probability &lt;span class=&quot;math-container&quot;&gt;$p \leq 0.5$&lt;/span&gt; on the sampled points, then the probability of picking a point outside your sampled points (unseen points) is &lt;span class=&quot;math-container&quot;&gt;$\geq 0.5$&lt;/span&gt; and thus error is &lt;span class=&quot;math-container&quot;&gt;$\geq 0.5$&lt;/span&gt;, and thus uniform distribution is the mid point. ANother important point to note is that if we pick &lt;span class=&quot;math-container&quot;&gt;$m+1$&lt;/span&gt; points we will definitely do better, but then its kind of overfitiing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This basically translates to why infinite VC dimension hypothesis class is not PAC learnable, because it shatters every set of size &lt;span class=&quot;math-container&quot;&gt;$|X|$&lt;/span&gt; and we have already seen the implications of picking a hypothesis class which shatters a set of size &lt;span class=&quot;math-container&quot;&gt;$|X|$&lt;/span&gt; in NFL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the informal description of how the NFL theorem was arrived at. You can find the entire explanation in this &lt;a href=&quot;https://www.youtube.com/watch?v=taA3r7378gU&amp;amp;list=PLPW2keNyw-usgvmR7FTQ3ZRjfLs5jT4BO&amp;amp;index=8&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture&lt;/a&gt; after which the proof in the book will start to make much more sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, inductive bias (restricting hypothesis class to some possible good candidates of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;) is quite important as can be seen, the effects without any inductive bias.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-03-26T12:34:20.780" LastActivityDate="2020-03-26T12:34:20.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18806" PostTypeId="2" ParentId="18791" CreationDate="2020-03-26T00:26:34.890" Score="0" Body="&lt;p&gt;It's not possible to solve version 1) of the problem in general. To see why, consider a graph with 2 cities, and 2 agents, where the agents start in opposite nodes. Since both agents need to move every turn, they will never meet in the same city.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For version 2), I'm going to make some assumptions that aren't completely clear from your text:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Agents have only local information. They can't see the global structure of the graph, and just all agree to meet at city X, because they don't know that X exists. They also can't see where other agents are initially.&lt;/li&gt;&#xA;&lt;li&gt;Cities are labelled, or otherwise identifiable to the agents. That is, an agent can tell whether they've visited a city before or not.&lt;/li&gt;&#xA;&lt;li&gt;Agents can tell how many agents are in their current city, and in all adjacent cities.&lt;/li&gt;&#xA;&lt;li&gt;There are a finite number of cities, and it's always possible to get from city A to city B, for any pair of cities (A,B).&lt;/li&gt;&#xA;&lt;li&gt;Agents have no way to communicate with each other, except by being present in the same space or not.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this setup, the following heuristic will always work:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you've never seen another agent, move to a random adjacent city, or stay in place, with a uniform probability of taking each possible action.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you've never seen another agent before, but you see one or more agents now, then switch permanently to the following strategy:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you can see an adjacent city with more agents than your city, move there now. If you can see several such cities, move to the largest one. Pick randomly in case of a tie. &lt;/li&gt;&#xA;&lt;li&gt;If an adjacent city has the same number of agents as yours, flip a coin to decide whether to go or stay. Eventually, one city ends up with more than the other, and then step 1 causes all agents to end up in the same city.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise, if you can see other adjacent cities with fewer agents, just stay put. They'll come to you.&lt;/li&gt;&#xA;&lt;li&gt;If all adjacent agents are empty and there's a total of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; agents you can see in your city right now, then remember &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;As long as you still can't see more than &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; agents in total, stay put with probability &lt;span class=&quot;math-container&quot;&gt;$(n-1)/n$&lt;/span&gt; and move to a random adjacent city with probability &lt;span class=&quot;math-container&quot;&gt;$1/n$&lt;/span&gt;. &lt;/li&gt;&#xA;&lt;li&gt;If you stayed put per step 5, and you still can only see n agents, and all but one of them is in your current city, move to the city with the single agent who moved. Otherwise, if 0 or more than 1 agent moved, stay put. The agents who moved come back, and we go to step 5.&lt;/li&gt;&#xA;&lt;li&gt;After successfully moving, go to step 1 again.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can see that once agents meet up in a group, that group never shrinks. Further, groups of agents are always trying to move around together. You can prove that a group moves an average of about &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2e}$&lt;/span&gt; every step. It may take a while, but eventually every group will bump into every other group, resulting in all the agents being in the same city.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-03-26T00:26:34.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18808" PostTypeId="1" CreationDate="2020-03-26T03:27:49.260" Score="3" ViewCount="31" Body="&lt;p&gt;Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my intuition tells me that, if I append the sequence length (e.g. 5) at each RNN input, the RNN should be able to do this. Does anyone know if this task is possible?&lt;/p&gt;&#xA;" OwnerUserId="32023" LastEditorUserId="2444" LastEditDate="2020-03-26T12:07:17.840" LastActivityDate="2020-03-26T12:07:17.840" Title="Is there any way of generating fixed-length sequences with RNNs?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18809" PostTypeId="1" AcceptedAnswerId="18814" CreationDate="2020-03-26T05:22:29.500" Score="3" ViewCount="178" Body="&lt;p&gt;VAE is trained to reduce the following two losses.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;KL divergence between inferred latent distribution and Gaussian.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the reconstruction loss&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I understand that the first one regularizes VAE to get structured latent space. But why and how does the second loss help VAE to work?&lt;/p&gt;&#xA;&lt;p&gt;During the training of the VAE, we first feed an image to the encoder. Then, the encoder infers mean and variance. After that, we sample &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; from the inferred distribution. Finally, the decoder gets the sampled &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and generates an image. So, in this way, the VAE is trained to make the generated image to be equal to the original input image.&lt;/p&gt;&#xA;&lt;p&gt;Here, I cannot understand why the sampled &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; should make the original image, since the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is sampled, it seems that the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; does not have any relationship between the original image.&lt;/p&gt;&#xA;&lt;p&gt;But, as you know, VAE works well. So I think I miss something important or understand it in a totally wrong way.&lt;/p&gt;&#xA;" OwnerUserId="18139" LastEditorUserId="2444" LastEditDate="2020-11-07T01:02:29.863" LastActivityDate="2020-11-07T01:02:29.863" Title="Why does the variational auto-encoder use the reconstruction loss?" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;variational-autoencoder&gt;&lt;evidence-lower-bound&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18810" PostTypeId="1" AcceptedAnswerId="18811" CreationDate="2020-03-26T06:59:23.657" Score="2" ViewCount="594" Body="&lt;p&gt;I'm using PDDL to generate a plan to solve this tower of Hanoi puzzle. I'll give the problem, the rules, the domain and fact sheet for everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PDDL is telling me that the goal can be simplified to false; however, I know for a fact that this puzzle is solvable.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Puzzle:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W9dQF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W9dQF.png&quot; alt=&quot;Tower of Hanoi&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There are 3 posts. Each has rings on it. From bottom to top on each post. The first post has the&#xA;  second largest ring. The second post has the smallest ring, with the second smallest ring on top of it.&#xA;  The third post has the third largest ring, with the largest ring stacked on top of it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Rules:&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The rules of this game are that you may only stack a ring on top of a larger ring. Your goal is to get all of the rings onto the same post, stacked from largest to smallest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h1&gt;My Code&lt;/h1&gt;&#xA;&#xA;&lt;h3&gt;Domain&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(define (domain hanoi)&#xA;  (:requirements :strips)&#xA;  (:predicates (clear ?x) (on ?x ?y) (smaller ?x ?y))&#xA;&#xA;  (:action move&#xA;    :parameters (?disc ?from ?to)&#xA;    :precondition (and (smaller ?to ?disc) (on ?disc ?from) &#xA;               (clear ?disc) (clear ?to))&#xA;    :effect  (and (clear ?from) (on ?disc ?to) (not (on ?disc ?from))  &#xA;          (not (clear ?to))))&#xA;  )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(define (problem hanoi5)&#xA;  (:domain hanoi)&#xA;  (:objects peg1 peg2 peg3 d1 d2 d3 d4 d5)&#xA;  (:init &#xA;    (smaller peg1 d1) (smaller peg1 d2) (smaller peg1 d3)&#xA;    (smaller peg1 d4) (smaller peg1 d5)&#xA;    (smaller peg2 d1) (smaller peg2 d2) (smaller peg2 d3)&#xA;    (smaller peg2 d4) (smaller peg2 d5)&#xA;    (smaller peg3 d1) (smaller peg3 d2) (smaller peg3 d3)&#xA;    (smaller peg3 d4) (smaller peg3 d5)&#xA;&#xA;    (smaller d2 d1) (smaller d3 d1) (smaller d3 d2) (smaller d4 d1)&#xA;    (smaller d4 d2) (smaller d4 d3) (smaller d5 d1) (smaller d5 d2)&#xA;    (smaller d5 d3) (smaller d5 d4)&#xA;&#xA;    ;(clear peg2) (clear peg3) (clear d1)&#xA;    ;(on d5 peg1) (on d4 d5) (on d3 d4) (on d2 d3) (on d1 d2))&#xA;    (clear d2) (clear d4) (clear d1)&#xA;    (on d2 peg1) (on d5 peg2) (on d4 d5) (on d3 peg3) (on d1 d3))&#xA;&#xA;  (:goal (and (on d5 d4) (on d4 d3) (on d3 d2) (on d2 d1)))&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm really at a loss here. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="34495" LastActivityDate="2020-03-26T07:46:19.933" Title="Can't solve Towers of Hanoi in PDDL" Tags="&lt;planning&gt;&lt;pddl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18811" PostTypeId="2" ParentId="18810" CreationDate="2020-03-26T07:46:19.933" Score="1" Body="&lt;h1&gt;Ah hah!&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The way I had defined the disks made d5 the LARGEST disk, not the smallest. So, the last few lines of the file should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   (clear d4) (clear d2) (clear d5)&#xA;    (on d4 peg1) (on d1 peg2) (on d2 d1) (on d3 peg3) (on d5 d3))&#xA;&#xA;  (:goal (and (on d1 d2) (on d2 d3) (on d3 d4) (on d4 d5)))&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;However&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If I wanted the opposite to be true, d1 to be the largest, the changes would have been as following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;; Swapped ?disc &amp;amp; ?to in (smaller ...) statement&#xA;:precondition (and (smaller ?disc ?to) (on ?disc ?from) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;And&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;; Swapped all d* and peg* objs within (smaller ...) staements&#xA;(smaller d1 peg1) (smaller d2 peg1) (smaller d3 peg1)&#xA;(smaller d4 peg1) (smaller d5 peg1)&#xA;(smaller d1 peg2) (smaller d2 peg2) (smaller d3 peg2)&#xA;(smaller d4 peg2) (smaller d5 peg2)&#xA;(smaller d1 peg3) (smaller d2 peg3) (smaller d3 peg3)&#xA;(smaller d4 peg3) (smaller d5 peg3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;:)&lt;/p&gt;&#xA;" OwnerUserId="34495" LastActivityDate="2020-03-26T07:46:19.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18813" PostTypeId="1" CreationDate="2020-03-26T13:08:10.587" Score="2" ViewCount="68" Body="&lt;p&gt;To my understanding NFL states that, we cannot have an hypothesis (let's assume it is an approximator like NN in this case) class that can't achieve certain accuracy parameters &lt;span class=&quot;math-container&quot;&gt;$\leq \epsilon$&lt;/span&gt; with probability greater than a certain &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; given the number of points from which we can sample is upper bounded to &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas, the UAC states that an approximator like a NN given enough hidden units can approximate any function (to my knowledge the function must be bounded).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point where these 2 clashes (as per my knowledge) is that if we increase the paramters in a NN, the UAC will start to hold good, but the VC dimension will increase (or hypothesis class becomes richer) and for the same &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; our &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; increases or &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; decreases (not sure which one is affected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what are the gaps in my knowledge here? How do we make these 2 consistent with each other?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-03-26T13:47:03.447" LastActivityDate="2020-03-26T13:47:03.447" Title="Are No Free Lunch theorem and Universal Approximation theorem contradictory in the context of neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;computational-learning-theory&gt;&lt;no-free-lunch-theorems&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18814" PostTypeId="2" ParentId="18809" CreationDate="2020-03-26T13:13:51.763" Score="1" Body="&lt;p&gt;The VAE uses the &lt;strong&gt;ELBO loss&lt;/strong&gt;, which is composed of the KL term and the likelihood term. The ELBO loss is a lower bound on the evidence of &lt;strong&gt;your&lt;/strong&gt; data, so if you maximize the ELBO you also maximize the evidence of the given data, which is what you indirectly want to do, i.e. you want the probability of your given data (i.e. the data in your dataset) to be high (because you want to use the VAE for the generation of inputs similar to the ones in your dataset). So, the idea is that you optimize both the KL term and the reconstruction (or likelihood) term &lt;strong&gt;jointly&lt;/strong&gt; (i.e. the ELBO). Why? Because, as I just said, the ELBO is the &lt;strong&gt;E&lt;/strong&gt;vidence &lt;strong&gt;L&lt;/strong&gt;ower &lt;strong&gt;BO&lt;/strong&gt;und on the given data, so, by maximizing it, you are also maximizing the evidence of your data. In other words, if you maximize the ELBO, you are finding a decoder that will have a high probability of reconstructing your inputs (i.e. the likelihood term), but, at the same time, you want your encoder to be constrained (i.e. KL term). Please, read &lt;a href=&quot;https://ai.stackexchange.com/a/17900/2444&quot;&gt;this answer&lt;/a&gt; for further details.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here, I cannot understand why the sampled &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; should make the original image, since the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is sampled, it seems that the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; does not have any relationship between the original image.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The relationship is that you will be maximizing the ELBO, which implies (and you can see this implication only if you are familiar with the ELBO loss) you will be minimizing the KL divergence between your posterior and the prior to generate the samples &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; (i.e. minimizing because there will be a minus in front of the KL term in the ELBO loss) and maximizing the probability of the reconstructed input. More precisely, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is used to reconstruct the input (i.e. the decoder does this), which is then used to calculate the reconstruction loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the mathematical formulations, you will see that the likelihood term of the ELBO is &lt;span class=&quot;math-container&quot;&gt;$p(x \mid z)$&lt;/span&gt;, i.e. the likelihood of the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is the input to the decoder, which produces a reconstruction of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. In practice, people will e.g. use the cross-entropy to then calculate the &quot;reconstruction loss&quot; (e.g. see &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/vae/main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PyTorch implementation&lt;/a&gt;), which &lt;em&gt;should&lt;/em&gt; correspond to this likelihood term &lt;span class=&quot;math-container&quot;&gt;$p(x \mid z)$&lt;/span&gt;. Why does the cross-entropy correspond to a likelihood? Because you can actually prove that the cross-entropy is equivalent to the negative log-likelihood. (Also, note that, in the ELBO loss, &lt;span class=&quot;math-container&quot;&gt;$p(x \mid z)$&lt;/span&gt; does not appear, but the logarithm of &lt;span class=&quot;math-container&quot;&gt;$p(x \mid z)$&lt;/span&gt; appears, but, for simplicity, I have used &lt;span class=&quot;math-container&quot;&gt;$p(x \mid z)$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$\log p(x \mid z)$&lt;/span&gt; above.)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-26T20:48:04.810" LastActivityDate="2020-03-26T20:48:04.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18815" PostTypeId="5" CreationDate="2020-03-26T13:50:19.183" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-26T13:50:19.183" LastActivityDate="2020-03-26T13:50:19.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18816" PostTypeId="4" CreationDate="2020-03-26T13:50:19.183" Score="0" Body="For questions related to the (different) universal approximation theorems (UATs), for example, in the context of feed-forward neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T00:40:36.997" LastActivityDate="2020-04-13T00:40:36.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18817" PostTypeId="2" ParentId="18798" CreationDate="2020-03-26T14:40:58.663" Score="4" Body="&lt;p&gt;For the programming part I suggest this YouTube &lt;a href=&quot;https://www.youtube.com/channel/UC58v9cLitc8VaCjrcKyAbrw/featured&quot; rel=&quot;nofollow noreferrer&quot;&gt;channel&lt;/a&gt; by Phil Tabor (he also has a website: &lt;a href=&quot;https://www.neuralnet.ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuralnet.ai&lt;/a&gt;. I found his videos really useful while I was attending reinforcement learning classes at the uni. He covers basic algorithms like value iteration and policy iteration and also more advanced like deep q learning, covering all main python libraries (Keras, tensorflow, pytorch). Hope it will help you as well! &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-26T14:40:58.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18818" PostTypeId="1" CreationDate="2020-03-26T15:22:05.283" Score="2" ViewCount="49" Body="&lt;p&gt;I've created a deep Q network. My model does not get better, and can't see what I'm doing wrong. I'm new to RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replay Memory&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class ReplayMemory(object):&#xA;&#xA;def __init__(self, input_shape, mem_size=100000):&#xA;    self.states = np.zeros((mem_size, input_shape))&#xA;    self.actions = np.zeros(mem_size, dtype=np.int32)&#xA;    self.next_states = np.zeros((mem_size, input_shape))&#xA;    self.rewards = np.zeros(mem_size)&#xA;    self.terminals = np.zeros(mem_size)&#xA;&#xA;    self.mem_size = mem_size&#xA;    self.mem_count = 0&#xA;&#xA;def push(self, state, action, next_state, reward, terminal):&#xA;&#xA;    idx = self.mem_count % self.mem_size&#xA;&#xA;    self.states[idx] = state&#xA;    self.actions[idx] = action&#xA;    self.next_states[idx] = next_state&#xA;    self.rewards[idx] = reward&#xA;    self.terminals[idx] = terminal&#xA;&#xA;    self.mem_count += 1&#xA;&#xA;def sample(self, batch_size):&#xA;    batch_index = np.random.randint(0, min(self.mem_count, self.mem_size), batch_size)&#xA;&#xA;    states = self.states[batch_index]&#xA;    actions = self.actions[batch_index]&#xA;    next_states = self.next_states[batch_index]&#xA;    rewards = self.rewards[batch_index]&#xA;    terminals = self.terminals[batch_index]&#xA;&#xA;    return (states, actions, next_states, rewards, terminals)&#xA;&#xA;def __len__(self):&#xA;    return min(self.mem_count, self.mem_size)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;DQN Agent&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class DQN_Agent(object):&#xA;&#xA;  def __init__(self, n_actions,n_states, ALPHA=0.001, GAMMA=0.99, eps_start=1 , eps_end=0.01, eps_decay=0.005):&#xA;      self.n_actions = n_actions&#xA;      self.n_states = n_states&#xA;&#xA;      self.memory = ReplayMemory(n_states)&#xA;&#xA;      self.ALPHA = ALPHA&#xA;      self.GAMMA = GAMMA&#xA;&#xA;      self.eps_start = eps_start&#xA;      self.eps_end = eps_end&#xA;      self.eps_decay = eps_decay&#xA;&#xA;&#xA;      self.model = self.create_net()&#xA;      self.target = self.create_net()&#xA;&#xA;      self.target.set_weights(self.model.get_weights())&#xA;&#xA;      self.steps_counter = 0&#xA;&#xA;  def create_net(self):&#xA;&#xA;    model = Sequential([&#xA;        Dense(64, activation=&quot;relu&quot;, input_shape=(self.n_states,)),&#xA;        Dense(32, activation=&quot;relu&quot;),&#xA;        Dense(self.n_actions)&#xA;    ])&#xA;&#xA;    model.compile(loss=&quot;huber_loss&quot;, optimizer=Adam(lr=0.0005))&#xA;&#xA;    return model&#xA;&#xA;def select_action(self, state):&#xA;    ratio = self.eps_end + (self.eps_start-self.eps_end)*np.exp(-1*self.eps_decay*self.steps_counter)&#xA;    rand = random.random()&#xA;    self.steps_counter += 1&#xA;&#xA;    if ratio &amp;gt; rand:&#xA;        #print(&quot;random&quot;)&#xA;        return np.random.randint(0, self.n_actions)&#xA;    else:&#xA;        #print(&quot;not random&quot;)&#xA;        return np.argmax(self.model.predict(state))&#xA;&#xA;&#xA;def train_model(self, batch_size):&#xA;    if len(self.memory) &amp;lt; batch_size:&#xA;        return None&#xA;&#xA;    states, actions, next_states, rewards, terminals = self.memory.sample(batch_size)&#xA;&#xA;    q_curr = self.model.predict(states)&#xA;    q_next = self.target.predict(next_states)&#xA;    q_target = q_curr.copy()&#xA;&#xA;&#xA;    batch_index = np.arange(batch_size, dtype=np.int32)&#xA;&#xA;    q_target[batch_index, actions] = rewards + self.GAMMA*np.max(q_next, axis=1)*terminals&#xA;&#xA;    _ = self.model.fit(states, q_target, verbose = 0)&#xA;&#xA;    if self.steps_counter % 10 == 0:&#xA;        self.target.set_weights(self.model.get_weights())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My training loop&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n_games = 50000&#xA;agent = DQN_Agent(2, 4)&#xA;&#xA;scores = []&#xA;avg_scores = []&#xA;&#xA;for epoch in range(n_games):&#xA;    done = False&#xA;    score = 0&#xA;    state = env.reset()&#xA;&#xA;    while not done:&#xA;       #env.render()&#xA;       action = agent.select_action(state.reshape(1,-1)) &#xA;       next_state, reward, done, _ = env.step(action)&#xA;&#xA;&#xA;       score += reward&#xA;&#xA;       agent.memory.push(state, action, next_state, reward, done)&#xA;&#xA;       state = next_state&#xA;       agent.train_model(64)&#xA;&#xA;&#xA;   avg_score = np.mean(scores[max(0, epoch-100):epoch + 1])&#xA;   avg_scores.append(avg_score)&#xA;   scores.append(score)&#xA;   print(score, avg_score)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="34504" LastEditorUserId="2444" LastEditDate="2020-03-26T16:47:48.073" LastActivityDate="2020-03-26T16:47:48.073" Title="Why is my DQN model not getting better?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18820" PostTypeId="1" AcceptedAnswerId="19962" CreationDate="2020-03-27T09:08:16.220" Score="3" ViewCount="49" Body="&lt;p&gt;In the apprenticeship learning algorithm described by Ng et al. in &lt;a href=&quot;https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apprenticeship Learning via Inverse Reinforcement Learning&lt;/a&gt;, they mention that expert trajectories come in the form of &lt;span class=&quot;math-container&quot;&gt;$\{s_0^i, s_1^i\, ...\}_{i=1}^m$&lt;/span&gt;. However, they also mentioned that &lt;span class=&quot;math-container&quot;&gt;$s_0 $&lt;/span&gt; is drawn from distribution D. Do all expert trajectories then have to have the same starting state? Why is it not possible to compute the feature expectation based on a single trajectory?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-03-27T10:35:30.260" LastActivityDate="2020-04-03T13:54:36.873" Title="Do all expert trajectories have the same starting state in apprenticeship learning?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;rewards&gt;&lt;apprenticeship-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18821" PostTypeId="1" CreationDate="2020-03-27T12:30:42.320" Score="1" ViewCount="15" Body="&lt;p&gt;There are many example in python which has a ready made data set, for example there is T-Shirt pre-trained data and thousands images, within few minutes it will tell how many t-shirt images are there in those folders&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but how that data &lt;code&gt;detect.tflite&lt;/code&gt; itself is created from scratch step by step, If I wanted to manually write those data in csv how it should be done&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately I have data which says, click event, keyboard event, process &lt;code&gt;exe&lt;/code&gt; name, title of the window, and timestamp&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to detect what exactly that user doing in his computer, who told do to what to him I want the software to tell me in words and diagrams&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is definitely a big software, but data classification part unclear to me&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need some guidance  &lt;/p&gt;&#xA;" OwnerUserId="34514" LastActivityDate="2020-03-27T12:30:42.320" Title="Data classification model to detect a process in an event log" Tags="&lt;classification&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18822" PostTypeId="1" CreationDate="2020-03-27T14:26:44.047" Score="3" ViewCount="59" Body="&lt;p&gt;I stumbled upon a job offer from a company that was looking for someone who was good with Reinforcement Learning (applied to finance) and something in their offer caught my eye. It goes something like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We want you to be able to study the price dynamic (of a stock I suppose) and its evolution in order to extract a Joint PDF that will be used in the Optimal Stochastic Control of a Loss Function (or gain)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The thing is I understand what each of these things mean and how they are used separately (from my background in Control theory &amp;amp; dynamical systems) and I worked with fitting Joint PDFs and Copulas before, but I don't understand how a Joint PDF would help with the &quot;Optimal Stochastic Control of a Loss Function&quot; ? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="34516" LastEditorUserId="9947" LastEditDate="2020-03-28T12:42:09.017" LastActivityDate="2020-03-28T12:42:09.017" Title="What does a joint probability density function have to do with Stochastic Optimal Control and Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;probability-distribution&gt;&lt;control-theory&gt;" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18823" PostTypeId="1" CreationDate="2020-03-27T15:03:33.417" Score="1" ViewCount="51" Body="&lt;p&gt;Last year it was announced that Deepmind's Starcraft playing bot AlphaStar was taking on human players in the Starcraft ladder system (some kind of league system as far as I can tell) and that it had reached the Grandmaster level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then I haven't really heard anything anymore about the progress of Alphastar. Given that I don't know anything about Starcraft I was wondering whether somebody has a better clue as to what Alphastar is up to? Is it still playing online? Or when did it stop playing? What was the improvement trajectory during the time it played online? Basically, how did this pan out, as seen from the perspective of the Starcraft community?&lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2020-03-27T15:03:33.417" Title="Is AlphaStar still competing in the Star Craft ladder?" Tags="&lt;gaming&gt;&lt;deepmind&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18825" PostTypeId="2" ParentId="18822" CreationDate="2020-03-27T15:57:55.433" Score="1" Body="&lt;p&gt;Extracting a joint PDF just means that you create a model that models the behavior of several variables combined instead of in isolation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If these variables aren't independent and your loss functions is influenced by all of them, you obviously have to learn this joint PDF to minimize your loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I don't see this statement as particularly mysterious. &lt;/p&gt;&#xA;" OwnerUserId="2227" LastActivityDate="2020-03-27T15:57:55.433" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18826" PostTypeId="1" CreationDate="2020-03-27T16:07:28.180" Score="2" ViewCount="78" Body="&lt;p&gt;In Goodfellow's paper, he says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Hence, by inspecting Eq. 4 at &lt;span class=&quot;math-container&quot;&gt;$D^*_G (\mathbf{x}) = \frac{1}{2}$&lt;/span&gt;, we find &lt;span class=&quot;math-container&quot;&gt;$C(G) = \log \frac{1}{2}+ \log \frac{1}{2} = − \log 4$&lt;/span&gt;. To see that this is the best possible value of &lt;span class=&quot;math-container&quot;&gt;$C(G)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;i.e. &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; loss should converge to &lt;span class=&quot;math-container&quot;&gt;$\log \frac{1}{2}$&lt;/span&gt;. This makes perfect sense. When I train a GAN in PyTorch with BCEloss, the loss for &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; converge to &lt;span class=&quot;math-container&quot;&gt;$\log(2)$&lt;/span&gt;, the negative of what Goodfellow states and what I'd expect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="34518" LastEditorUserId="2444" LastEditDate="2020-03-27T17:05:46.110" LastActivityDate="2020-03-27T17:05:46.110" Title="Why does GAN loss converge to log(2) and not -log(2)?" Tags="&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;&lt;papers&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18828" PostTypeId="2" ParentId="5728" CreationDate="2020-03-27T17:51:57.967" Score="2" Body="&lt;p&gt;I found a paper that gives a table of time complexities for different architectures using linear programming-based training: &lt;a href=&quot;https://arxiv.org/abs/1810.03218&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1810.03218&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34521" LastActivityDate="2020-03-27T17:51:57.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18829" PostTypeId="1" CreationDate="2020-03-27T18:28:55.237" Score="5" ViewCount="214" Body="&lt;p&gt;Batch norm is a technique where they essentially standardize the activations at each layer, before passing it on to the next layer. Naturally, this will affect the gradient through the network. I have seen the equations that derive the back-propagation equations for the batch norm layers. From the original paper: &lt;a href=&quot;https://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/pdf/1502.03167.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iWINr.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iWINr.png&quot; alt=&quot;From the paper: https://arxiv.org/pdf/1502.03167.pdf&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have trouble understanding if there is an intuitive understanding of what effect it actually has on the network. For instance, does it help with the exploding gradient problem, since the activations are rescaled, and the variance of them is constrained?&lt;/p&gt;&#xA;" OwnerUserId="18086" LastEditorUserId="2444" LastEditDate="2020-03-27T18:47:41.127" LastActivityDate="2020-04-27T00:01:30.217" Title="What effect does batch norm have on the gradient?" Tags="&lt;deep-learning&gt;&lt;optimization&gt;&lt;batch-normalization&gt;&lt;vanishing-gradient-problem&gt;&lt;exploding-gradient-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18830" PostTypeId="1" CreationDate="2020-03-27T19:39:40.970" Score="3" ViewCount="97" Body="&lt;p&gt;I've just started learning CSP and I find it quite exciting. Now I'm facing nonogram solving problem and I want to solve it using backtracking with CSP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first problem that I face is that I cannot quite figure out what such variables, domains, and constraints could be. My first thought was to make every field(those squares) a variable with such domain: &lt;span class=&quot;math-container&quot;&gt;$D_i = \{1,0\}$&lt;/span&gt;, where 1 means that certain field has been colored black and 0 white.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've been mostly learning binary constraints, and I was thinking of using the AC-3 or forward checking algorithm for propagation during the backtracking algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far I know constraints of all arity can be represented as a set of binary constraints, so that would enable me to use algorithms I mentioned. But that leads me to the problem of defining constraints. If every field was a variable then each line and column would be a constraint, based on how certain lines should be colored(numbers defining line for example 2,3,2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's all new and quite hard for me to imagine and come up with. I've been reading some articles and papers on that but they were too advanced for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, does anybody have an idea how can I formulate a nonogram problem as a constraint satisfaction problem?&lt;/p&gt;&#xA;" OwnerUserId="34483" LastEditorUserId="34483" LastEditDate="2020-03-28T10:20:04.360" LastActivityDate="2020-03-28T10:20:04.360" Title="How can I formulate a nonogram problem as a constraint satisfaction problem?" Tags="&lt;backpropagation&gt;&lt;ai-basics&gt;&lt;search&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18831" PostTypeId="1" CreationDate="2020-03-27T21:56:43.117" Score="4" ViewCount="28" Body="&lt;p&gt;I have a question about how weights are updated during back-propagation for some of my samples that have unknown labels (please note, unknown, not missing). The reason they are unknown is because this is genomic data and to generate these data would take 8 years of lab work! Nevertheless I have genomic data for samples that have multiple-labels, sex age organ etc. this is a multi-class multi-label problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For most classes, ALL labels are complete. For two or three classes, there are unknown labels. an example would be the developmental stage of samples at age x, the developmental stage of sample at age y are known. the developmental stage of samples at age Z are unknown! (generating this data is what would take most time)... I would therefore like to include all this data during training as it is indispensable. I would like to generate the sigmoid probability and assign unknown label 'Z' as belonging to developmental stage 0 or 1 (known classes) based on a threshold (say &gt;= 0.5)... When one-hot encoding the unknown labels simply have no ground truth, 0 for class developing and 0 for not-developing as follows (example of 3 samples shown for class in question):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  [[1., 0.&#xA;   [0., 1.&#xA;   [0., 0.  ......]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the first row is known sample 1, second is known sample 2 and 3rd is unknown, and therefore has no ground truth. It is this sample i would like to assign a label of known class 1 or 2 based on the 'highest probability'.. based on reading and discussions, this is the direction i will be taking for this task, as it can be validated in the lab later... so the approach is, include in training and see what the network 'thinks' it is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is: How does back-propagation handle these known and unknown samples with respect to weight updates&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note i have trained the network with ~90% validation performance. for all classes for which is there is complete data, the predictions are great. and the same for classes for which there is unknown data. It can accurately classify the samples for which there is known developmental stages... and it does assign a probability value to those samples that have the 'unknown' label (0,0), so i would really like to know how back-prop is handling these samples for the classes where there are unknown ground truth labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thank you!&lt;/p&gt;&#xA;" OwnerUserId="34530" LastActivityDate="2020-03-27T21:56:43.117" Title="How do weights changes handles during back-propagation when there are unknown labels" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18832" PostTypeId="2" ParentId="18829" CreationDate="2020-03-27T23:36:34.270" Score="1" Body="&lt;p&gt;&lt;em&gt;&quot;Naturally, this will affect the gradient through the network.&quot;&lt;/em&gt; this statement is only partially true, let's see why by starting explaining the real aim of batch normalisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the title of the paper suggest, the aim of batch normalisation is to decrease training time by reducing covariance shift. What is covariance shift? We can conceive it as the variation that can happen between between the values of two layers of a network. We are all familiar with the concept that if we have input features with different unite scale, like kilos and euros, most likely lot of values will have different magnitude orders, like thousands for the weight could appear often with hundred of thousands for the money. When applying the activation function to different order of magnitude this discrepancy will remain, causing the values in the first layer to assume a really broad range of values. This is not good, since high fluctuation means more time to converge to stable values. This is why values fed to neural nets are always standardised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors applied the same logic to the hidden layers, arguing that a deep neural net can be conceived as a repetition of itself (every hidden layer is an input layer that send features to another hidden layer) ergo the features should be normalised at every layer. How can we do it? Normalising every batch is the most natural way to procede, but in this way there is the risk to end up transforming the internal representation of a layer, because normalising is not a linear transformation. This is way the authors propose a clever way to perform hidden layers normalisation, which consists in a classic normalisation followed by a linear scaling performed with two trainable parameters &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (which appear in the last step of the batch norm scratch code below). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0eObf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0eObf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A really important thing to notice is that the mean and variance used to perform the classic normalisation are &lt;strong&gt;mean and variance calculated on the mini batch&lt;/strong&gt;. I will explain why this is important in a sec, first I want to stress out that the &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; parameter can actually bring to increase overfitting when batch norm is randomly stucked on top of other layer. The reason why is that, as we see in the scale and shift step, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; is nothing but a biased term, added to perform the shift of the mean of the batch hidden values. So, to avoid having an extra bias that lead to overfit, the bays on the previous layer should be removed, leaving only the classic matrix of weights parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back to the gradient problem, we can see that in itself doesn't necessarily lead to increased performances, but it does provide an advantage in terms of hidden layer values convergence. The x axis on the two right sub plots of the figure below represent the variation of the hidden values of net trained with and without batch norm. When trained with batch norm the hidden values reach stable ranges after few iteration. This help the net to reach high accuracy in less iterations (first subplot on the left) but we can see that even without batch norm the net reach eventually high accuracy.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6zpxF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6zpxF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only help provided by batch norm to the gradient is the fact that, as noticed before, the normalisation is firstly performed by calculating the mean and variance on individual batches. This is important because this partial estimation of mean and variance introduces noice. Similarly to drop out, which has a regularisation effect due to the noice generated by randomly deactivating part of the weights, batch norm can introduce regularisation by adding noice due to the larger or smaller mean and variance estimated on individual batches. But still, batch norm was not introduced as a regularisation technique, and the equation you put on the question simply prove that it is possible to calculate the derivatives of the equations applied to perform the batch norm. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-03-27T23:36:34.270" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18833" PostTypeId="2" ParentId="18798" CreationDate="2020-03-28T01:44:21.203" Score="5" Body="&lt;p&gt;This answer will point the reader to potentially useful resources, but I can't ensure that the courses are good (because I have never followed them).&lt;/p&gt;&#xA;&lt;h3&gt;Free&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=ZipAjLSNlQc&amp;amp;list=PL-9x0_FO_lglnlYextpvu39E7vWzHhtNO&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning in the Open AI Gym&lt;/a&gt; (a small course that you can find in the YouTube channel suggested in &lt;a href=&quot;https://ai.stackexchange.com/a/18817/2444&quot;&gt;the other answer&lt;/a&gt;) by Phil Tabor&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The free course &lt;a href=&quot;https://www.youtube.com/watch?v=iOh7QUZGyiU&amp;amp;list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advanced Deep Learning &amp;amp; Reinforcement Learning&lt;/a&gt; by DeepMind can be useful. See &lt;a href=&quot;https://www.youtube.com/watch?v=L6xaQ501jEs&amp;amp;list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs&amp;amp;index=14&quot; rel=&quot;nofollow noreferrer&quot;&gt;video 14&lt;/a&gt;, which discusses deep RL topics (e.q. DQN).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The free course &lt;a href=&quot;https://www.youtube.com/watch?v=FgzM3zpZ55o&amp;amp;list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS234: Reinforcement Learning Winter 2020&lt;/a&gt; can also be useful. In particular, &lt;a href=&quot;https://www.youtube.com/watch?v=gOV8-bC1_KU&amp;amp;list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&amp;amp;index=6&quot; rel=&quot;nofollow noreferrer&quot;&gt;lesson 6&lt;/a&gt; is about deep reinforcement learning. Here's the &lt;a href=&quot;http://web.stanford.edu/class/cs234/schedule.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;course schedule&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Paid&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/deep-reinforcement-learning-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advanced AI: Deep Reinforcement Learning in Python&lt;/a&gt; (at Udemy) seems to be completely dedicated to DRL.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/machine-learning-beginner-reinforcement-learning-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning: Beginner Reinforcement Learning in Python&lt;/a&gt;  (at Udemy)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/deep-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning 2.0&lt;/a&gt; (at Udemy)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/deep-q-learning-from-paper-to-code/?couponCode=DQN-JUN-3-2020&quot; rel=&quot;nofollow noreferrer&quot;&gt;Modern Reinforcement Learning: Deep Q Learning in PyTorch&lt;/a&gt; by Phil Tabor&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/actor-critic-methods-from-paper-to-code-with-pytorch/?couponCode=AC-JUN-3-2020&quot; rel=&quot;nofollow noreferrer&quot;&gt;Modern Reinforcement Learning: Actor-Critic Methods&lt;/a&gt; by Phil Tabor&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In any case, if you are familiar with RL and deep learning topics, I encourage you to directly read the DQN papers (both by DeepMind folks)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt; (2013)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/nature14236&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human-level control through deep reinforcement learning&lt;/a&gt; (2015)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Of course, deep RL isn't just DQN, but these are two very important papers that you should read.&lt;/p&gt;&#xA;&lt;p&gt;You should also note that deep RL isn't anything special. It's just RL concepts combined with function approximators and deep learning tricks or deep neural networks. So, if you are really familiar with deep learning and RL (including the usage of neural networks to approximate policies and value functions), you don't need any course to understand deep RL concepts. You can just pick any deep RL paper and you can potentially understand it, although you may require more than 1-2 iterations (but that depends on the person).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-14T14:08:22.227" LastActivityDate="2020-08-14T14:08:22.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18834" PostTypeId="2" ParentId="12034" CreationDate="2020-03-28T04:35:50.310" Score="1" Body="&lt;p&gt;It's pretty easy to tell what's going on with AI Angel Angelica. If you've watched some of the videos, you'll notice the steady progression of the real-time rendering. The first videos were really jittery and there were a lot of movements (awkward mouth movements, fingers unable to move, etc) to the more recent ones with realistic-ish mouth movements and full finger movements. So, they started off with a motion capture program with a special suit to capture movement. As they progressed, the suit got a lot more sensors and the programming itself was streamlined. As far as it being artificial intelligence, no, it's not. There's a person in a suit talking to a camera that's been programmed to pick up movement and translate it onto an avatar. Based on the movements from the most recent video, there are sensors on a few parts of the face, neck, shoulders, elbows, waist, hips, knees and feet and probably 3 on each finger, 2 on each thumb. Based upon the camera movements the programming is designed to follow the avatar, which is why it seems to follow her when she moves around and additionally, she's able to turn around while the camera remains in place. There is probably a face camera attached to the suit to track the minute facial changes (think of the movie Avatar) and two or three cameras mounted around the area to pick up the rest of the sensors (which could just be colored dots).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The setup is probably a greenroom with furniture in some of the places the furniture shows up in the 3d 'room' she's in. There is probably another camera or sensor that is used so she can move around when she shows off her computer setup. The model probably just uses a button to switch between them. I don't know if the voice is real or altered, but it has changed over the course of the videos. If it is real, the model is probably female speaking in real-time. Whether or not the model is also the developer is another question. I don't know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless, the real-time rendering and advanced 3d space analysis requires a pretty hefty computer setup, so whoever is doing this is on the level of a professional with professional-grade computing power. It is WAY beyond anything I'm capable of at the moment, so my hat's absolutely off to the developer and/or model. Extremely impressive. &lt;/p&gt;&#xA;" OwnerUserId="34537" LastActivityDate="2020-03-28T04:35:50.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18835" PostTypeId="1" CreationDate="2020-03-28T05:03:17.583" Score="1" ViewCount="44" Body="&lt;p&gt;I am learning deep learning from Andrew Ng's tutorial &lt;a href=&quot;https://www.coursera.org/learn/deep-neural-network/lecture/qcogH/mini-batch-gradient-descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mini-batch Gradient Descent&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain the similarities and dissimilarities between batch GD and mini-batch GD?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-03-28T12:38:16.897" LastActivityDate="2020-03-28T12:41:10.517" Title="What is the difference between batch and mini-batch gradient decent?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;gradient-descent&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18836" PostTypeId="2" ParentId="18835" CreationDate="2020-03-28T07:07:59.887" Score="2" Body="&lt;p&gt;It is really simple. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In gradient descent not using mini-batches, you feed your entire training set of data into the network and accumulate a cost function based on this full set of data. Then you use gradient descent to adjust the network weights to minimize the cost. Then you repeat this process until you get a satisfactory level of accuracy. For example, if you have a training set consisting of 50,000 samples, you would feed all 50,000 samples along with the 50,000 labels into the network, then perform gradient descent and update the weights. This is a slow process because you have to process 50,000 inputs to do just one step of gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make things go faster instead of running all 50,000 inputs through the network, you split up the training set into &quot;batches&quot;. For example, you could break the training set up into 50 batches each containing 1000 samples. You would feed the network the first batch of 1000 samples, accumulate the loss value then perform gradient descent and adjust the weights. Then you feed in the next batch of 1000 samples and repeat the process. So, now, instead of only getting one step of gradient descent for 50,000 samples, you get 50 steps of gradient descent. This method of using batches leads to a much faster convergence of the network.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-03-28T12:41:10.517" LastActivityDate="2020-03-28T12:41:10.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18837" PostTypeId="1" CreationDate="2020-03-28T07:59:44.080" Score="4" ViewCount="193" Body="&lt;p&gt;I am a very beginner in the field of AI. I am basically a Pharma Professional without much coding experience. I use GUI based tools for the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to develop an ANN that receives as input a protein sequence and produces as output a drug molecule. Drug molecules can be represented as fixed-length binary (0-1). This length is 881 bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not know how to transform protein sequences of variable length into a fixed-length binary representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how should I deal with variable-length inputs for a neural network? What is the best way?&lt;/p&gt;&#xA;" OwnerUserId="34540" LastEditorUserId="2444" LastEditDate="2020-03-28T13:12:01.563" LastActivityDate="2020-03-28T13:12:01.563" Title="How should I deal with variable-length inputs for neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18838" PostTypeId="2" ParentId="18579" CreationDate="2020-03-28T08:51:48.517" Score="1" Body="&lt;p&gt;Searle's Chinese room is analogical and is intended to present an easy-to-understand picture of the essential elements and processes of the digital computer. In the room the man (CPU) has a book of intructions (program) for responding to Chinese input questions. That is just one program of many possible programs the room could run. Each different program would be a different instruction book. SOAR would be just one of those books.&lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2020-03-28T08:51:48.517" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18839" PostTypeId="2" ParentId="18837" CreationDate="2020-03-28T10:37:25.467" Score="3" Body="&lt;p&gt;The most common way people deal with inputs of varying length is &lt;strong&gt;padding&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You first define the desired sequence length, i.e. the input length you want your model yo have. Then any sequences with a shorter length than this are padded either with zeros or with special characters so that they reach the desired length. If an input is larger than your desired length, usually you'd split it into multiple inputs.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-03-28T10:37:25.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18840" PostTypeId="1" CreationDate="2020-03-28T12:59:27.360" Score="2" ViewCount="133" Body="&lt;p&gt;I'm trying to perform a segmentation task on images of multiple sizes using fully convolutional neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm using EfficientNet as a feature extractor, and adding a deconvolution/backwards convolution/transposed convolution layer as described in the original &lt;a href=&quot;https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fully Convolutional Networks for Semantic Segmentation&lt;/a&gt; paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this transposed convolution layer doesn't return a filter of a size equivalent to the original image for images of varying sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's assume the original image is &lt;span class=&quot;math-container&quot;&gt;$100 \times 100$&lt;/span&gt;, and the last layer contains filters of size &lt;span class=&quot;math-container&quot;&gt;$50 \times 50$&lt;/span&gt;. To get a filter of the same size as the original, you would need a transposed convolution layer of size &lt;span class=&quot;math-container&quot;&gt;$51 \times 51$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, assume you passed in an image of size &lt;span class=&quot;math-container&quot;&gt;$200 \times 200$&lt;/span&gt;. The last layer would contain filters of size &lt;span class=&quot;math-container&quot;&gt;$100 \times 100$&lt;/span&gt;. That same transposed convolutional filter of size &lt;span class=&quot;math-container&quot;&gt;$51 \times 51$&lt;/span&gt; would result in an output of size &lt;span class=&quot;math-container&quot;&gt;$150 \times 150$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to make it so that a fully convolutional network always returns an image of the same size as the original?&lt;/p&gt;&#xA;" OwnerUserId="27240" LastEditorUserId="2444" LastEditDate="2020-06-14T11:02:14.547" LastActivityDate="2021-03-26T05:02:25.213" Title="Can a fully convolutional network always return an image of the same size as the original?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19840" PostTypeId="1" CreationDate="2020-03-28T14:45:13.527" Score="2" ViewCount="110" Body="&lt;p&gt;After playing around with normal Q-learning I have decided to switch to deep Q-learning and I have encountered this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand, for a task with discrete action space, where there are 4 possible actions (lets say left, right, up, down) my DQN needs to have four outputs. And then &lt;code&gt;argmax&lt;/code&gt; of prediction needs to be taken, which will be my predicted action (if &lt;code&gt;argmax(prediction)==2&lt;/code&gt; then I will pick third action, in my case &lt;code&gt;up&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use Mean Squared Error loss for my network then the dimension of output needs to be same as dimension of expected target variables. I am calculating target variables using following code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;target = rewards[i] + GAMMA * max(Qs_next_state[i])&lt;/code&gt; which gives me a single number (while predicted output is four dimensional) as I workaround I decided to use custom loss function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def custom_loss(y_true, y_pred): # where y_true is target, y_pred is output of neural net&#xA;    return ((max(y_pred) - y_true)**2) / BATCH_SIZE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I am not sure if it is correct, from what I have read from tutorials/papers loss functions do not have this &lt;code&gt;max()&lt;/code&gt; in them. But how else am I going to end up with same dimensionality between targets and NN outputs. What is the correct approach here?&lt;/p&gt;&#xA;" OwnerUserId="26336" LastActivityDate="2020-03-28T14:45:13.527" Title="Whats the correct loss function to use during deep Q-learning (discrete action space)" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19841" PostTypeId="1" CreationDate="2020-03-28T16:18:31.020" Score="1" ViewCount="111" Body="&lt;p&gt;As I want to start coding a new Trading AI in this year (first based on Python and later maybe in C++) I stumbled over the following question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today, I would like to make a pro/contra list with you in the area of deep learning vs machine learning. The difference should be clear to most of you. If not, here is a &lt;a href=&quot;https://hackernoon.com/deep-learning-vs-machine-learning-a-simple-explanation-47405b3eef08&quot; rel=&quot;nofollow noreferrer&quot;&gt;nice explanation from Hackernoon&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now I was convinced that this future project will be based on Tensorflow, Keras, etc. However, the following came to my mind afterward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of you will probably have heard of &lt;a href=&quot;https://ai.facebook.com/blog/pluribus-first-ai-to-beat-pros-in-6-player-poker/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pluribus&lt;/a&gt; already. Dr. Sandholm and Mr. Brown (as work for his Ph.D.) were the first to program an AI that won against 6 poker world champions in No-Limit-Texas Holdem Poker. This seemed impossible because poker is a game of imperfect information. If you haven't read/seen their work until now, here's a link to a Facebook blog post &lt;a href=&quot;https://ai.facebook.com/blog/pluribus-first-ai-to-beat-pros-in-6-player-poker/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facebook, Carnegie Mellon build first AI that beats pros in 6-player poker&lt;/a&gt;. See also the paper &lt;a href=&quot;https://science.sciencemag.org/content/365/6456/885&quot; rel=&quot;nofollow noreferrer&quot;&gt;Superhuman AI for multiplayer poker&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=u90TbxK7VEA&amp;amp;t=267s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the work, it is clear that they wrote the whole thing in C++ and WITHOUT the use of any deep learning library but exclusively on the basis of machine learning. This was also confirmed to me by both of them via email. So it was possible to bring the AI in under 24H to a level that could beat 6 Poker World Champions without any problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stock and crypto market is nothing else. A game of imperfect information. The prices of a crypto coin or stock are influenced by an incredible number of factors. This includes of course prices from the past but also current media (as currently seen with covid-19) and data from the economy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to grab the Data out of the &quot;Big Players&quot; like CoinCap API, CryptoAPI.io for all kinds of historical and new charts prices etc. The same we will do with the Yahoo Finance API to grab data out of the stock market. Depending on the size of this project and how it will develop we want to implement also some kind of NLP to grab the most out of Economy Data like dailyfx news to predict some in/decreases for some stocks but this is a future feature. So basically the main question is what would be better for our system to be used. CNNs or machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this leads me to the conclusion that I am not sure what the better option for a trading bot would be. I know that the training of AI-based on deep learning would take much longer than based on machine learning but is it safe to say that the results are really better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of deep learning and machine learning to develop a trading system?&lt;/p&gt;&#xA;" OwnerUserId="35548" LastEditorUserId="35548" LastEditDate="2020-03-28T19:01:06.057" LastActivityDate="2020-03-28T19:01:06.057" Title="What are the pros and cons of deep learning and machine learning to develop a trading system?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19842" PostTypeId="1" CreationDate="2020-03-28T17:15:05.437" Score="1" ViewCount="74" Body="&lt;p&gt;I am new to RL, and I am thinking of doing a little project. The goal of the project is to learn an agent play the memory game with cards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already created the program for detecting the cards on the table (with YOLO) and classifying them what kind of object they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want an agent to be able to play the memory game by itself, without being explicitly told the rules and such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any tips on how to get started to make the RL process easier?&lt;/p&gt;&#xA;" OwnerUserId="7344" LastEditorUserId="2444" LastEditDate="2020-03-28T17:25:41.080" LastActivityDate="2020-03-30T09:16:46.797" Title="How can I develop a reinforcement learning agent that plays memory cards game?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19844" PostTypeId="1" CreationDate="2020-03-28T18:26:49.073" Score="1" ViewCount="69" Body="&lt;p&gt;As a first step in many NLP courses, we learn about text preprocessing. The steps include lemmatization, removal of rare words, correcting typos etc. But I am not so sure about the actual effectiveness of doing such a step; in particular, if we are learning a neural network for a downstream task, it seems like modern state of the art (BERT, GPT-2) just take essentially raw input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, &lt;a href=&quot;https://www.aclweb.org/anthology/W18-5406.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this ACL paper&lt;/a&gt; seems to show that the result of text preprocessing is mixed, to say the least. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is text preprocessing really all that necessary for NLP? In particular, I want to contrast/compare against vision and tabular data, where I have empirically found that standardization usually actually does help. Feel free to share your personal experiences/what use cases where text preprocessing helps! &lt;/p&gt;&#xA;" OwnerUserId="18086" LastEditorUserId="18086" LastEditDate="2020-03-28T18:38:39.403" LastActivityDate="2020-04-26T10:00:58.640" Title="Is text preprocessing really all that necessary for NLP?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;data-preprocessing&gt;&lt;structured-data&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19845" PostTypeId="2" ParentId="19844" CreationDate="2020-03-28T20:58:44.763" Score="2" Body="&lt;p&gt;It all depends on the quality of data. Due to old rule &quot;Garbage in, garbage out&quot; &lt;a href=&quot;https://en.wikipedia.org/wiki/Garbage_in,_garbage_out&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&#xA;, if you have bad quality data(data redundancy, unstructured data, too much memory, etc) your results won't be spectacular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other cases, everybody could be a Data Scientist, because its only task was &quot;put raw text into classifier&quot;. Also, you should remember that BERT or GPT-2 it's deep learning algorithms so they not need too much processing. Using preprocessing in machine learning is more that needed(prediction of sentiment for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shortly, preprocessing is &lt;strong&gt;optional&lt;/strong&gt;, but highly &lt;strong&gt;advisable&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19476" LastEditorUserId="19476" LastEditDate="2020-04-26T10:00:58.640" LastActivityDate="2020-04-26T10:00:58.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19846" PostTypeId="1" AcceptedAnswerId="19848" CreationDate="2020-03-28T21:12:24.790" Score="2" ViewCount="253" Body="&lt;p&gt;If a certain hypothesis class &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; has a &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; over a domain &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, how can I prove that &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; will shatter all subsets of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; with size less than &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; will shatter &lt;span class=&quot;math-container&quot;&gt;$A \subset X$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$|A| \leq d-1$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2021-05-19T13:00:41.217" LastActivityDate="2021-05-19T13:00:41.217" Title="How do I prove that $\mathcal{H}$, with $\mathcal{VC}$ dimension $d$, shatters all subsets with size less than $d-1$?" Tags="&lt;proofs&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;&lt;hypothesis-class&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19848" PostTypeId="2" ParentId="19846" CreationDate="2020-03-29T00:29:37.200" Score="2" Body="&lt;p&gt;We can show that it is not true by a counterexample. For example, &lt;span class=&quot;math-container&quot;&gt;$X = \{1,2,3\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal H = \{\{\},\{1\},\{2\},\{1,2\}\}$&lt;/span&gt; is the finite set hypothesis class. By definition, in this case, the &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension of &lt;span class=&quot;math-container&quot;&gt;$\mathcal H$&lt;/span&gt; over the domain &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$d=2$&lt;/span&gt;. Although &lt;span class=&quot;math-container&quot;&gt;$A = \{3\} \subset X$&lt;/span&gt;, whose size is smaller than the &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension, i.e &lt;span class=&quot;math-container&quot;&gt;$|A|&amp;lt;d=2$&lt;/span&gt;, it is not shattered by &lt;span class=&quot;math-container&quot;&gt;$\mathcal H$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2021-05-19T08:33:35.693" LastActivityDate="2021-05-19T08:33:35.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19849" PostTypeId="1" CreationDate="2020-03-29T02:12:34.267" Score="2" ViewCount="155" Body="&lt;p&gt;According to &lt;a href=&quot;https://www.deeplearning-academy.com/p/ai-wiki-activation-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The purpose of an activation function is to add some kind of non-linear property to the function&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The sigmoid is typically used as an activation function of a unit of a neural network in order to introduce non-linearity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is ReLU a non-linear activation function? And why? If not, then why is it used as an activation function of neural networks?&lt;/p&gt;&#xA;" OwnerUserId="27529" LastEditorUserId="2444" LastEditDate="2020-03-29T02:29:12.957" LastActivityDate="2020-05-26T16:34:38.610" Title="Is ReLU a non-linear activation function?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19850" PostTypeId="2" ParentId="19849" CreationDate="2020-03-29T02:27:13.590" Score="2" Body="&lt;p&gt;&lt;strong&gt;Short Answer:&lt;/strong&gt; Yes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Visually:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yqgclm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yqgclm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you see the image from &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;, it shown that ReLU (the blue line) is non-Linear (the line is not straight, it turns in 0). You can also check &quot;visual&quot; definition of linear function in &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;In calculus and related areas, a linear function is a function whose graph is a straight line&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mathematically:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linear function of one variable can be defined as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ f(x) = ax + b $&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you plot that function in 2D, it will give you a straight line. Then, the form of linear function with multi variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ f(x_1, x_2, ..., x_n) = a_1x_1 + a_2x_2 + ... + a_nx_n + b $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you again plot that function in the correct dimension it also give you a straight line. And if you that function carefully, it similar with calculation that happen in a neuron. That's why neuron addition and multiplication is a linear function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ f(x_1, x_2, ..., x_n) = w_1x_1 + w_2x_2 + ... + w_nx_n + b $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding more layer of linear functions doesn't make the function become &quot;complex&quot; for example, if you have &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; like below and then you put another layer of linear function &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; on top of it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) = ax + b$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(x) = cf(x) + d = cax + cb + d$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as the neural network is trained to find the value of &lt;span class=&quot;math-container&quot;&gt;$a,b,c,d$&lt;/span&gt;, we can group the constant from the formula above, and then rewrite to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h(x) = mx + n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$m=ca$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n=cb+d$&lt;/span&gt;. So without non-linear function the layer of neural network is useless, it only give you another &quot;simple&quot; linear function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ReLU formula is a &lt;span class=&quot;math-container&quot;&gt;$f(x)=max(0,x)$&lt;/span&gt;, it produces non-linearity as you can't write to linear function format. Using this function will give you &quot;complexity&quot; when you add more layer on top of it.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2020-03-29T02:55:43.340" LastActivityDate="2020-03-29T02:55:43.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19851" PostTypeId="1" CreationDate="2020-03-29T04:58:07.263" Score="6" ViewCount="114" Body="&lt;p&gt;I know that AI can be used to design &lt;a href=&quot;https://deeppcb.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;printed circuit boards (PCBs)&lt;/a&gt;, so it can be used to solve complex tasks.&lt;/p&gt;&#xA;&lt;p&gt;Is there any &lt;a href=&quot;https://en.wikipedia.org/wiki/Programming_language_implementation&quot; rel=&quot;nofollow noreferrer&quot;&gt;programming language designed&lt;/a&gt; by deep learning (or any other AI technique)?&lt;/p&gt;&#xA;" OwnerUserId="33730" LastEditorUserId="2444" LastEditDate="2020-11-04T16:59:02.483" LastActivityDate="2020-11-04T16:59:02.483" Title="Is there any programming language designed by deep learning?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;programming-languages&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19852" PostTypeId="1" CreationDate="2020-03-29T08:28:01.793" Score="1" ViewCount="58" Body="&lt;p&gt;Let's assume I want to build a semantic segmentation algorithm, based on Multires-UNET. My GT-masks are messy and generated by a GAN, but they are getting better and better over time. The goal is knowledge expansion (based on the paper &lt;a href=&quot;https://arxiv.org/abs/1911.04252&quot; rel=&quot;nofollow noreferrer&quot;&gt;Noisy-Student&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you generally say that PreLU and Leaky Relu are better for noisy labels (or imperfect ones), like the situation in GANs in general?&lt;/p&gt;&#xA;" OwnerUserId="35557" LastEditorUserId="2444" LastEditDate="2020-03-29T13:03:11.583" LastActivityDate="2020-03-29T13:03:11.583" Title="Are PreLU and Leaky ReLU better than ReLU in the case of noisy labels?" Tags="&lt;machine-learning&gt;&lt;activation-function&gt;&lt;relu&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19853" PostTypeId="1" CreationDate="2020-03-29T09:20:58.047" Score="1" ViewCount="96" Body="&lt;p&gt;I am a newbie to NLP and NLG. I am tasked to develop a system to generate a report based on a given data table. The structure of the report and the flow is predefined. I have researched on several existing python libraries like BERT, SimpleNLG but they don't seem to fit my need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&#xA;input_data(country = 'USA', industry = 'Coal', profit = '4m', trend = 'decline')&#xA;output: The coal industry in USA declined by 4m. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input data array can be different combinations (and dynamic) based on a data table. I would like to know if  there is any python package available, or any resource discussing a practical approach for this. &lt;/p&gt;&#xA;" OwnerUserId="35558" LastActivityDate="2021-04-12T08:36:27.470" Title="Building a template based NLG system to generate a report from data" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19855" PostTypeId="1" AcceptedAnswerId="19858" CreationDate="2020-03-29T15:33:37.707" Score="2" ViewCount="39" Body="&lt;p&gt;I have been seeing notations on Expectations with their respective subscripts such as &lt;span class=&quot;math-container&quot;&gt;$E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$&lt;/span&gt;. This equation is taken from &lt;a href=&quot;https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf&lt;/a&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s,a,R) = R(s) + \gamma E_{s'\sim T(s,a,\cdot)}[V^\pi(s',R)]$&lt;/span&gt; ,in the case of the Bayesian IRL paper.(&lt;a href=&quot;https://www.aaai.org/Papers/IJCAI/2007/IJCAI07-416.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aaai.org/Papers/IJCAI/2007/IJCAI07-416.pdf&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that &lt;span class=&quot;math-container&quot;&gt;$s_0 \sim D$&lt;/span&gt; means that the starting state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; is drawn from a distribution of starting states &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. But how do we understand the latter with subscript &lt;span class=&quot;math-container&quot;&gt;${s'\sim T(s,a,\cdot)}$&lt;/span&gt; ? How is &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; drawn from a distribution of transition probabilities?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-03-29T17:10:09.613" LastActivityDate="2020-03-29T17:10:09.613" Title="What does the notation ${s'\sim T(s,a,\cdot)}$ mean?" Tags="&lt;reinforcement-learning&gt;&lt;notation&gt;&lt;expectation&gt;&lt;apprenticeship-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19856" PostTypeId="1" AcceptedAnswerId="27491" CreationDate="2020-03-29T15:52:59.450" Score="1" ViewCount="76" Body="&lt;p&gt;I was reading online that tic-tac-toe has a state space of &lt;span class=&quot;math-container&quot;&gt;$3^9 = 19,683$&lt;/span&gt;. From my basic understanding, this sounds too large to use tabular Q-learning, as the Q table would be huge. Is this correct?&lt;/p&gt;&#xA;&lt;p&gt;If that is the case, can you suggest other (non-NN) algorithms I could use to create a TTT bot to play against a human player?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2021-04-25T10:46:12.350" LastActivityDate="2021-04-25T10:48:03.233" Title="Non-Neural Network algorithms for large state space in zero sum games" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;algorithm-request&gt;&lt;tic-tac-toe&gt;&lt;zero-sum-games&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19857" PostTypeId="2" ParentId="19851" CreationDate="2020-03-29T15:55:26.997" Score="5" Body="&lt;p&gt;There are certainly things like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd say a strong example is &lt;em&gt;layered learning&lt;/em&gt; approaches, descended from Peter Stone's work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A programming language is essentially a collection of useful shorthands for assembly-level instructions. Ultimately, everything you do in a programming language eventually gets executed in assembly. So making a programming language amounts to learning how to write short, reusable, assembly language programs that you can then use as building blocks to solve harder problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this in action is Kelly &amp;amp; Heywood's approach to constructing 'Tangled Program Graphs' for reinforcement learning (&lt;a href=&quot;https://www.ijcai.org/Proceedings/2018/0740.pdf&quot; rel=&quot;noreferrer&quot;&gt;IJCAI 2018&lt;/a&gt;). Here an evolutionary algorithm is used to learn short assembly programs, that can be combined into a graph to make more complex programs. This is similar to graphical programming languages like J.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-03-29T15:55:26.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19858" PostTypeId="2" ParentId="19855" CreationDate="2020-03-29T16:06:27.030" Score="2" Body="&lt;p&gt;The dot (&lt;span class=&quot;math-container&quot;&gt;$.$&lt;/span&gt;) at the end of &lt;span class=&quot;math-container&quot;&gt;$T(s,a,.)$&lt;/span&gt; shows all possible states that we can go from state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; by doing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. As you know there are some probabilities here for choosing those states, that the sum of these probabilities is equal to 1. Hence, &lt;span class=&quot;math-container&quot;&gt;$T(s,a,.)$&lt;/span&gt; is a probability distribution.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-03-29T16:12:42.343" LastActivityDate="2020-03-29T16:12:42.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19859" PostTypeId="1" CreationDate="2020-03-29T17:39:42.917" Score="1" ViewCount="38" Body="&lt;p&gt;I am trying to train a model that detects logos in documents. Since I am not really interested in what kind of logo there is, but simply if there is a logo, does it make sense to combine all logos into 1 logo class? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or are &quot;logos&quot; too diverse to group them together (like some logos are round, some are rectangular, some are even text based etc.) and the diversity of features will just make it hard for the neural network to learn? Or doesn't it matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am currently trying out the YOLOv3 architecture to begin with. Any other suggestions better suited are also welcome)&lt;/p&gt;&#xA;" OwnerUserId="35565" LastActivityDate="2020-03-29T21:58:58.050" Title="Object detection: combine many classes into one?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19860" PostTypeId="1" CreationDate="2020-03-29T18:21:45.240" Score="1" ViewCount="18" Body="&lt;p&gt;I made CSP nonogram solver and I wanna test it on some bigger data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find such data to test my program? I've been looking on the internet but I couldn't find anything.&lt;/p&gt;&#xA;" OwnerUserId="34483" LastActivityDate="2020-03-29T18:21:45.240" Title="Where can I find example data for Nonogram solver?" Tags="&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19861" PostTypeId="1" CreationDate="2020-03-29T18:47:25.153" Score="2" ViewCount="48" Body="&lt;p&gt;The naive concept of a general AI, or strong AI, or artificial general intelligence, is some kind of software that can answer questions like   &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the volume of a cube that is 1 m wide?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or even&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are there only two political parties in the US?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The second question requires external knowledge and high-level reasoning. For example that US means USA in the context, the constitution and that having two parties is caused by the mathematical properties of the election system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would expect that newborn human child is intelligent in the sense of intelligence that is used in general artificial intelligence, but toddlers can not answer these questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is not because an infant is not intelligent, but because it is not educated, I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the apparent level of education of an artificial intelligence that could be called human-like?&lt;/p&gt;&#xA;" OwnerUserId="2317" LastEditorUserId="2444" LastEditDate="2020-03-30T02:09:37.503" LastActivityDate="2020-03-30T02:10:59.577" Title="Would a new human-like general artificial intelligence be more similar, in terms of eduction, to a toddler or an adult human?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;human-like&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19862" PostTypeId="2" ParentId="19861" CreationDate="2020-03-29T19:20:43.797" Score="1" Body="&lt;p&gt;A human-level AI should be able to learn and behave in the same way that humans learn and behave, otherwise, we shouldn't be calling it a human-level AI. So, either if it starts with more or less knowledge of a baby human, it should be able to learn similarly to a human and acquire more knowledge with experience. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the apparent level of education of an artificial intelligence that could be called human-like?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To answer you question more directly, the level of education of a human-like AI (even newly created ones) can potentially be variable (i.e. different human-like AIs could potentially have different levels of education), but the human-like AI will necessarily need to be able to increase its level of education and, in general, knowledge (because humans can also do this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, note that &lt;a href=&quot;https://content.sciendo.com/view/journals/jagi/5/1/article-p1.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI is not necessarily restricted to human-level AI&lt;/a&gt;. So, other AGIs may not follow the same principles of humans. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-30T02:10:59.577" LastActivityDate="2020-03-30T02:10:59.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19863" PostTypeId="2" ParentId="19859" CreationDate="2020-03-29T21:53:52.283" Score="2" Body="&lt;p&gt;I think there is no absolute answer for this. Often its kind of trial and error. In general the CNN tries to generalize the problem, so using all logos with different augmentations and ground truths can maybe lead to some feature maps, which are so general that the CNN can find logos. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if your logos are so various, and embedded in colorful websites, the tasks seems quite difficult, also if they vary in shape and form.Like you said, I think you definitely need an FPN (Feature Pyrimad network) to get the different sizes, scales and so on combined with an RPN (region-proposal network) to find the logos multiple times in the websites (if thats nessecary). For that you can use Mask-RCNN (&lt;a href=&quot;https://github.com/matterport/Mask_RCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/matterport/Mask_RCNN&lt;/a&gt;). You can try to transfer-train it on Imagenet-Backbone for example to reduce training time. Just tried to use Mask-RCNN to segment colored cells from medical images. Worked out quite good.&lt;a href=&quot;https://i.stack.imgur.com/JX3yr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JX3yr.png&quot; alt=&quot;My own approach&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used ImageLabel for labeling, worked out quite intiutive and saved all in a JSON file.&lt;a href=&quot;https://github.com/tzutalin/labelImg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Imglabel&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How large is your dataset and how complex are the logos ? Do you have examples which you can demonstrate?&lt;/p&gt;&#xA;" OwnerUserId="35557" LastEditorUserId="35557" LastEditDate="2020-03-29T21:58:58.050" LastActivityDate="2020-03-29T21:58:58.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19864" PostTypeId="1" CreationDate="2020-03-29T22:38:06.263" Score="1" ViewCount="26" Body="&lt;p&gt;I am looking to do sequence classification using deep learning. The length of my sequences can vary from a few hundred to several tens of thousands of characters. I was wondering what is a good approach for doing this. I had success with splitting a sequence into subsequences a few hundred characters long and using LSTMs, but then one is faced with the task of putting the results of each of those together and it is nontrivial as well. Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="35572" LastActivityDate="2020-03-29T22:38:06.263" Title="length independent sequence classification methods" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19866" PostTypeId="1" CreationDate="2020-03-30T03:51:16.157" Score="2" ViewCount="29" Body="&lt;p&gt;Assume &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X} \in R^{N, C}$&lt;/span&gt; is the input of the softmax &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P} \in R^{N, C}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is number of examples and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is number of classes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{p}_i = \left[ \frac{e^{x_{ik}}}{\sum_{j=1}^C e^{x_{ij}}}\right]_{k=1,2,...C} \in R^{C} \mbox{ is a row vector of } \mathbf{P}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider example &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th, because softmax function &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}:R^C \mapsto R^C$&lt;/span&gt; (eliminate subscript &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; for ease notation), so the derivative of vector-vector mapping is Jacobian matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{J}$&lt;/span&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{J}_{\mathbf{p}}(\mathbf{x}) = \left[ \frac{\partial \mathbf{p}}{\partial x_1}, \frac{\partial \mathbf{p}}{\partial x_2}, ..., \frac{\partial \mathbf{p}}{\partial x_C}  \right] = &#xA;\begin{bmatrix} &#xA;    \frac{\partial p_1}{\partial x_1} &amp;amp; \frac{\partial p_1}{\partial x_2} &amp;amp; \dots &amp;amp; \frac{\partial p_1}{\partial x_C} \\ &#xA;    \frac{\partial p_2}{\partial x_1} &amp;amp; \frac{\partial p_2}{\partial x_2} &amp;amp; \dots &amp;amp; \frac{\partial p_2}{\partial x_C} \\&#xA;    \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots \\ &#xA;    \frac{\partial p_C}{\partial x_1} &amp;amp; \frac{\partial p_C}{\partial x_2} &amp;amp; \dots &amp;amp; \frac{\partial p_C}{\partial x_C} &#xA;\end{bmatrix}&#xA;\in R^{C, C}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{J}_{\mathbf{p}}(\mathbf{x})$&lt;/span&gt; is called the derivative of vector &lt;span class=&quot;math-container&quot;&gt;${\mathbf{p}}$&lt;/span&gt; with respect to vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mbox{1) Derivative in diagonal:}\frac{\partial p_{k}}{\partial x_{k}} = \frac{\partial}{\partial x_{k}}\left( \frac{e^{x_k}}{\sum_{j=1}^C e^{x_j}} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \frac{\left( \frac{\partial e^{x_k}}{\partial x_k} \right)\sum_{j=1}^C e^{x_j} - e^{x_k}\left(\frac{\partial \sum_{j=1}^C e^{x_j}}{\partial x_k} \right)}{\left(\sum_{j=1}^C e^{x_j}\right)^2} \mbox{  (Quotient rule) }$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \frac{e^{x_k}\sum_{j=1}^C e^{x_j} - e^{x_k} e^{x_k}}{\left(\sum_{j=1}^C e^{x_j}\right)^2} = \frac{e^{x_k}(\sum_{j=1}^C e^{x_j} - e^{x_k})}{\left(\sum_{j=1}^C e^{x_j}\right)^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \frac{e^{x_k}}{\sum_{j=1}^C e^{x_j}} \left(\frac{\sum_{j=1}^C e^{x_j} - e^{x_k}}{\sum_{j=1}^C e^{x_j}}\right) = \frac{e^{x_k}}{\sum_{j=1}^C e^{x_j}} \left(1 - \frac{e^{x_k}}{\sum_{j=1}^C e^{x_j}}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rightarrow \frac{\partial p_{k}}{\partial x_{k}} = p_{k}(1-p_{k})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mbox{2) Derivative not in diagonal } k \neq c \mbox{ :} \frac{\partial p_{k}}{\partial x_{c}} = \frac{\partial}{\partial x_{c}}\left( \frac{e^{x_k}}{\sum_{j=1}^C e^{x_j}} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \frac{\left( \frac{\partial e^{x_k}}{\partial x_c} \right)\sum_{j=1}^C e^{x_j} - e^{x_k}\left(\frac{\partial \sum_{j=1}^C e^{x_j}}{\partial x_c} \right)}{\left(\sum_{j=1}^C e^{x_j}\right)^2} \mbox{  (Quotient rule) }$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \frac{0 - e^{x_k} e^{x_c}}{\left(\sum_{j=1}^C e^{x_j}\right)^2} = -\frac{e^{x_k}}{\sum_{j=1}^C e^{x_j}}\frac{e^{x_c}}{\sum_{j=1}^C e^{x_j}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rightarrow \frac{\partial p_{k}}{\partial x_{c}} = -p_{k}p_{c}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rightarrow \mathbf{J}_{\mathbf{p}}(\mathbf{x}) =&#xA;\begin{bmatrix} &#xA;    p_1(1-p_1) &amp;amp; -p_1p_2 &amp;amp; \dots &amp;amp; -p_1p_C \\ &#xA;    -p_2p_1 &amp;amp; p_2(1-p_2) &amp;amp; \dots &amp;amp; -p_2p_C \\&#xA;    \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\ &#xA;    -p_Cp_1 &amp;amp; -p_Cp_2 &amp;amp; \dots &amp;amp; p_C(1-p_C)&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Focal Loss: &lt;span class=&quot;math-container&quot;&gt;$\displaystyle FL = -\sum_{k=1}^C y_{k} \alpha_{k}(1-p_k)^\gamma \log (p_k)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\mathbf{x}} FL = \nabla_{\mathbf{p}}FL (\mathbf{J}_{\mathbf{p}}(\mathbf{x}))^T $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\mathbf{p}} FL = &#xA;\begin{bmatrix}&#xA;    \frac{\partial FL}{\partial p_1}\\ &#xA;    \frac{\partial FL}{\partial p_2}\\&#xA;    \vdots \\&#xA;    \frac{\partial FL}{\partial p_C} &#xA;\end{bmatrix} \mbox{ where } \frac{\partial FL}{\partial p_k}&#xA;= - y_k\alpha_k \left(-\gamma(1-p_k)^{\gamma-1} \log(p_k) + \frac{(1-p_k)^\gamma}{p_k}  \right) = y_k \alpha_k\gamma(1-p_k)^{\gamma-1}\log(p_k) - y_k\alpha_k\frac{(1-p_k)^\gamma}{p_k}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\mathbf{x}} FL = &#xA;\begin{bmatrix}&#xA;    \frac{\partial FL}{\partial p_1}\\ &#xA;    \frac{\partial FL}{\partial p_2}\\&#xA;    \vdots \\&#xA;    \frac{\partial FL}{\partial p_C} &#xA;\end{bmatrix}^T&#xA;\begin{bmatrix} &#xA;    \frac{\partial p_1}{\partial x_1} &amp;amp; \frac{\partial p_1}{\partial x_2} &amp;amp; \dots &amp;amp; \frac{\partial p_1}{\partial x_C} \\ &#xA;    \frac{\partial p_2}{\partial x_1} &amp;amp; \frac{\partial p_2}{\partial x_2} &amp;amp; \dots &amp;amp; \frac{\partial p_2}{\partial x_C} \\&#xA;    \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\ &#xA;    \frac{\partial p_C}{\partial x_1} &amp;amp; \frac{\partial p_C}{\partial x_2} &amp;amp; \dots &amp;amp; \frac{\partial p_C}{\partial x_C} &#xA;\end{bmatrix} =&#xA;\begin{bmatrix}&#xA;    \sum_{k=1}^C \left(\frac{\partial FL}{\partial p_k}\frac{\partial p_k}{\partial x_1}\right)\\ &#xA;    \sum_{k=1}^C \left(\frac{\partial FL}{\partial p_k}\frac{\partial p_k}{\partial x_2}\right)\\&#xA;    \vdots \\&#xA;    \sum_{k=1}^C \left(\frac{\partial FL}{\partial p_k}\frac{\partial p_k}{\partial x_C}\right)&#xA;\end{bmatrix}^T \in R^C&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mbox{Case 1: }\displaystyle \frac{\partial FL}{\partial p_k}\frac{\partial p_k}{\partial x_k} \forall k=1,2,...,C$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial FL}{\partial p_k} \frac{\partial p_k}{\partial x_k} = y_k \alpha_k\gamma(1-p_k)^{\gamma-1}\log(p_k)p_k(1-p_k) - y_k\alpha_k\frac{(1-p_k)^\gamma}{p_k}p_k(1-p_k)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = y_k \alpha_k (1-p_k)^{\gamma}(\gamma p_k \log(p_k) - 1 + p_k) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mbox{Case 2: } (k \neq c)\displaystyle \frac{\partial FL}{\partial p_k}\frac{\partial p_k}{\partial x_c}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial FL}{\partial p_k} \frac{\partial p_k}{\partial x_c} = - y_k\alpha_k\gamma(1-p_k)^{\gamma-1}\log(p_k)p_kp_c + y_k\alpha_k\frac{(1-p_k)^\gamma}{p_k}p_kp_c$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = - y_k\alpha_k (1-p_k)^{\gamma-1}p_c(\gamma p_k \log(p_k) - 1 + p_k) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mbox{For each } d=1,2,...,C \mbox{ : }\sum_{k=1}^C \left(\frac{\partial FL}{\partial p_k} \frac{\partial p_k}{\partial x_d}\right) = y_d \alpha_d (1-p_d)^{\gamma}(\gamma p_d \log(p_d) - 1 + p_d) + \sum_{c \neq d}^C \left( - y_d\alpha_d (1-p_d)^{\gamma-1}p_c(\gamma p_d \log(p_d) - 1 + p_d) \right) = y_d\alpha_d(1-p_d)^{\gamma-1}(\gamma p_d \log(p_d) - 1 + p_d)\left(1-p_d -\sum_{c \neq d}^C(p_c)\right) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rightarrow \nabla_{\mathbf{x}} FL = \left[ y_d\alpha_d(1-p_d)^{\gamma-1}(\gamma p_d \log(p_d) - 1 + p_d)\left(1-p_d -\sum_{c \neq d}^C(p_c)\right) \right]_{d=1,2,...,C}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem is &lt;span class=&quot;math-container&quot;&gt;$\left(1-p_d -\sum_{c \neq d}^C(p_c)\right) = 0$&lt;/span&gt; (because sum of all probabilities is 1) then the whole expression collapses to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any wrong in my focal loss derivation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Softmax_function&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1708.02002&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1708.02002&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28078" LastEditorUserId="2444" LastEditDate="2020-03-30T12:01:48.257" LastActivityDate="2020-03-30T12:01:48.257" Title="Is there any wrong in my focal loss derivation?" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;loss-functions&gt;&lt;calculus&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19867" PostTypeId="1" CreationDate="2020-03-30T06:58:10.400" Score="1" ViewCount="33" Body="&lt;p&gt;Suppose we had a series of single-dimensional data points &lt;span class=&quot;math-container&quot;&gt;$X = \{x_1, x_2, \dots, x_n \}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of data points and there corresponding output values &lt;span class=&quot;math-container&quot;&gt;$T = \{t_1, t_2, \dots, t_n \}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to train a single neuron network given below to learn from the data (the model is bad, but I just wanted to try it out as an exercise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xR9U4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xR9U4.png&quot; alt=&quot;Image for the Neural network model&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output function of this neuron would be a recursive function as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;y = f(a_0 + a_1x + a_2 y)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x) = \frac{1}{1 + e^{-x}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error function for such a model would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;e = \sum_{i=1}^N (y_i - t_i)^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I minimise this loss function? What are the derivatives that I need to use to update the parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also, I am new to this problem, therefore it would be really helpful if you tell me sources/books to read about such problems.)&lt;/p&gt;&#xA;" OwnerUserId="35576" LastEditorUserId="2444" LastEditDate="2020-03-30T12:05:38.000" LastActivityDate="2020-03-30T12:05:38.000" Title="How do we minimize loss for a single neuron with a feedback?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19868" PostTypeId="1" CreationDate="2020-03-30T08:17:39.780" Score="2" ViewCount="145" Body="&lt;p&gt;My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;limited number of discrete actions (like 20)&lt;/li&gt;&#xA;&lt;li&gt;high-dimensional states (like 250 values)&lt;/li&gt;&#xA;&lt;li&gt;instant reward after every single action (not only at the end of an episode)&lt;/li&gt;&#xA;&lt;li&gt;a single action can affect the state quite a lot&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There's no &quot;goal&quot; like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.&lt;/p&gt;&#xA;" OwnerUserId="35578" LastEditorUserId="2444" LastEditDate="2020-03-30T12:11:37.617" LastActivityDate="2020-08-30T21:02:50.287" Title="Which deep reinforcement learning algorithm is appropriate for my problem?" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19870" PostTypeId="2" ParentId="19856" CreationDate="2020-03-30T08:29:26.700" Score="0" Body="&lt;p&gt;In the case of TicTacToe, you can make use of game theory. The entire search space can be denoted by a &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;game tree&lt;/a&gt;. You bot must now be able to maximize the chance of winning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make use of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax#Minimax_algorithm_with_alternate_moves&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maximin algorithm&lt;/a&gt;. This is still computationally intensive on large search spaces. To improve the efficiency &lt;a href=&quot;https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha-Beta pruning&lt;/a&gt; can be applied to reduce the number of nodes in the Game tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are core AI concepts and will always perform better than neural networks on dully defined and relatively smaller search spaces. Neural networks perform better when it's too difficult to compute all the possible combinations of a game at a certain state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have a look at &lt;a href=&quot;https://www.freecodecamp.org/news/how-to-make-your-tic-tac-toe-game-unbeatable-by-using-the-minimax-algorithm-9d690bad4b37/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; to build a TicTacToe bot.&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2020-03-30T08:29:26.700" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19871" PostTypeId="1" CreationDate="2020-03-30T08:59:27.033" Score="1" ViewCount="90" Body="&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&lt;p&gt;I am trying to setup a Deep Q-Learning agent. I have looked that the papers &lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt; as well as &lt;a href=&quot;https://arxiv.org/pdf/1507.06527.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Recurrent Q-Learning for Partially Observable MDPs&lt;/a&gt; as well as looking at the question &lt;a href=&quot;https://ai.stackexchange.com/questions/7721/how-does-lstm-in-deep-reinforcement-learning-differ-from-experience-replay&quot;&gt;How does LSTM in deep reinforcement learning differ from experience replay?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Current setup&lt;/h3&gt;&#xA;&lt;p&gt;I currently take the current state of a game, not picture but rather the position of the agent, the way the agent is facing and some inventory items. I currently only feed state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; as at input to a 3 layer NN (1 input, 2 hidden, 3 output) to estimate the Q-values of each action.&lt;/p&gt;&#xA;&lt;p&gt;The algorithm that I use is almost the same as the one used in &lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt;, with the only difference that I do not train after each timestep but rather sample &lt;code&gt;mini_batch*T&lt;/code&gt; at the end of each episode and train on that. Where &lt;code&gt;T&lt;/code&gt; is the number of time-steps in that episode.&lt;/p&gt;&#xA;&lt;h2&gt;The issue&lt;/h2&gt;&#xA;&lt;p&gt;At the current state, the agent do not learn within 100 00 episodes, which is about 100 00 * 512 training iterations. Making me consider that something is not working, this is where I realised that I do not consider any of the history of the previous steps.&lt;/p&gt;&#xA;&lt;p&gt;What I currently struggle with is sending multiple time-steps-states to the NN. The reason for this is the complexity of how the game/program I am using. According to &lt;a href=&quot;https://arxiv.org/pdf/1507.06527.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Recurrent Q-Learning for Partially Observable MDPs&lt;/a&gt; LSTM could be a solution for this, however I would prefer to manually code a RNN rather than using an LSTM. Would not a RNN with something like the following structure have a chance of working?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/Z2xbySO.png&quot; alt=&quot;Simple RNN from pytroch-website.&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, as far as I know, RNN need the inputs to be fed in sequence and not randomly sampled?&lt;/p&gt;&#xA;" OwnerUserId="35582" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-30T09:33:09.563" Title="Do RNN solves the need for LSTM and/or multiple states in Deep Q-Learning?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19872" PostTypeId="2" ParentId="19842" CreationDate="2020-03-30T09:16:46.797" Score="1" Body="&lt;p&gt;The main thing to keep in mind when designing a reinforcement learning agent is that you need to develop an interactive environment in which the agent can learn and define the possible moves the agent can make. In your case, the environment is the memory cards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next you need to define how the agent can interact with the environment, that is &lt;code&gt;choosing&lt;/code&gt; a card or &lt;code&gt;flipping&lt;/code&gt; a card etc. Also how the environment will behave when the agent chooses a move. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally you need to define the fitness function, in this case the &lt;code&gt;score&lt;/code&gt; (number of pairs created) should be fine. You should have a look at some online RL competitions to get a better understanding of the architecture. &lt;a href=&quot;https://aws.amazon.com/deepracer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS DeepRacer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2020-03-30T09:16:46.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19873" PostTypeId="1" CreationDate="2020-03-30T10:02:43.267" Score="1" ViewCount="72" Body="&lt;p&gt;I'm trying to build a neural network between protein sequence and its drug fingerprint. My input size is 20000. The output size is 881. The sample size is 610.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I process this huge neural network? But how? And in which tool? &lt;/p&gt;&#xA;" OwnerUserId="34540" LastEditorUserId="2444" LastEditDate="2020-03-30T12:24:34.250" LastActivityDate="2020-08-27T16:41:24.263" Title="How can I process neural network with 25000 input nodes?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19874" PostTypeId="2" ParentId="19873" CreationDate="2020-03-30T11:10:51.763" Score="0" Body="&lt;p&gt;It sure is possible, imagine a CNN can handle way bigger number of inputs. An image with size of &lt;code&gt;512x512&lt;/code&gt; has already &lt;code&gt;262144 input nodes&lt;/code&gt; when re-arranged to a one-row vector. The trick sicne 2012/2014 is to use Convolutions, and deep ones, so stacking a lot of &lt;code&gt;3x3 Convolutions&lt;/code&gt; for example. Its way less sensitive than a fully-connected Dense network and needs a siginificant amount of less parameters. For more check this out, &lt;code&gt;chapter 9&lt;/code&gt; : &lt;a href=&quot;https://github.com/janishar/mit-deep-learning-book-pdf/blob/master/complete-book-pdf/Ian%20Goodfellow%2C%20Yoshua%20Bengio%2C%20Aaron%20Courville%20-%20Deep%20Learning%20(2017%2C%20MIT).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ian-Goodfellow, Deep Learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tools for that are &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tensorflow&lt;/a&gt; and &lt;a href=&quot;https://keras.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;keras&lt;/a&gt; based on python, or tensorflow-js on java, you can also use pytorch but the community is rather small on comparison. &lt;/p&gt;&#xA;" OwnerUserId="35557" LastActivityDate="2020-03-30T11:10:51.763" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19875" PostTypeId="1" CreationDate="2020-03-30T11:27:59.393" Score="1" ViewCount="36" Body="&lt;p&gt;Why does &lt;a href=&quot;https://arxiv.org/pdf/1311.2901.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeconvNet (Zeiler, 2014)&lt;/a&gt; use ReLU in the backward pass (after unpooling)? Are not the feature maps values already positive due to the ReLU in the forward pass? So, why do the authors apply the ReLU again coming back to the input?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;update:&lt;/strong&gt;  I better explain my problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given an input image &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and ConvLayer &lt;span class=&quot;math-container&quot;&gt;$CL$&lt;/span&gt; composed of: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a convolution&lt;/li&gt;&#xA;&lt;li&gt;an activation function ReLU&lt;/li&gt;&#xA;&lt;li&gt;a pool operation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the output of ConvLayer given an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$f=CL(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the Deconv target is to &quot;reverse&quot; the output &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (the feature map) to restore an approximate version of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. To this aim, the authors define a function &lt;span class=&quot;math-container&quot;&gt;$CL^{-1}$&lt;/span&gt; composed of 3 subfunctions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;a. unpool &lt;/p&gt;&#xA;&#xA;&lt;p&gt;b. activation function ReLU (useless in my opinion, because &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is already positive due to the application of the 2. step in &lt;span class=&quot;math-container&quot;&gt;$CL(f)$&lt;/span&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;c. transposed convolution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words &lt;span class=&quot;math-container&quot;&gt;$x\simeq CL^{-1}(f)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$CL^{-1} (f) = transpconv(relu(unpool(f)))$&lt;/span&gt;. But, if &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the output computed as &lt;span class=&quot;math-container&quot;&gt;$f=CL(x)$&lt;/span&gt;, it is already positive, so the b. step is useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I understood from the paper. Where I wrong?&lt;/p&gt;&#xA;" OwnerUserId="2189" LastEditorUserId="2189" LastEditDate="2020-03-30T18:05:18.827" LastActivityDate="2020-03-30T18:05:18.827" Title="Why do DeconvNet use ReLU in the backward pass?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;relu&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19877" PostTypeId="1" CreationDate="2020-03-30T12:05:32.597" Score="0" ViewCount="20" Body="&lt;p&gt;I've been reading &lt;a href=&quot;https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; on convolutional neural networks (I'm a beginner) - and I'm stuck at a point.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xFToi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xFToi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I understand: We have a 4x4 input, and want to transform it to a 2x2 grid. I'm visualising this as a kernel sliding over the 4x4 grid, with just the right number of strides, so as to get 4 outputs which constitute the 2x2 grid (there are animations right above this part of the page in the link attached). The model chooses to represent the 4x4 grid as a vector of length 16. Also, the 4x16 transformation matrix when pre-multiplied to the input vector, produces the output vector, which is mapped back to a 2D grid. &lt;em&gt;Is this right?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Moving on, another screenshot from the same page-&#xA;&lt;a href=&quot;https://i.stack.imgur.com/RqiHN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RqiHN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it that both the matrices are really the same, and a lot of weights just happen to be zero in the weight matrix, which is what the second matrix is depicting? In that case even, why so many zeros?&lt;/p&gt;&#xA;&lt;p&gt;P.S. Thanks a lot for being so patient and reading to the end of this post, I really appreciate it. I'm a beginner, and really interested in these topics, and I'm self studying from various online resources - hence, any help is appreciated. I hope this is the right platform for this post.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-03-30T12:53:50.503" Title="Interpreting I/O Transformation Matrix in Convolution" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19879" PostTypeId="1" AcceptedAnswerId="20062" CreationDate="2020-03-30T12:47:02.750" Score="5" ViewCount="888" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7iBKO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7iBKO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image above, a screenshot from &lt;a href=&quot;https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, describes discrete 2D convolutions as &lt;em&gt;linear transforms.&lt;/em&gt; The idea used, as far as I understand, is to represent the 2 dimensional &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;x&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; input grid as a vector of &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; length, and the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;x&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; output grid as a vector of &lt;span class=&quot;math-container&quot;&gt;$m^2$&lt;/span&gt; length. I don't see why this can't be generalised to higher-dimensional convolutions, since a transformation matrix can be constructed from one vector to another, no matter the length (&lt;strong&gt;right?&lt;/strong&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: &lt;strong&gt;Aren't all discrete convolutions (not just 2D) linear transforms?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there cases where such a transformation matrix cannot be found?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-03-31T08:54:43.510" LastActivityDate="2020-04-13T01:28:41.493" Title="Aren't all discrete convolutions (not just 2D) linear transforms?" Tags="&lt;convolutional-neural-networks&gt;&lt;signal-processing&gt;" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="19880" PostTypeId="1" CreationDate="2020-03-30T14:34:26.690" Score="1" ViewCount="34" Body="&lt;p&gt;We are given a computer vision classification task, that is, a task that asks us to predict the category of an image over &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; predefined classes (the so-called closed set classification problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is it possible to give an estimate on what is the best accuracy one is likely achieve using an end-to-end CNN model (possibly, using a popular backbone) in this task? Do the performances of state-of-the-arts models on open datasets serve as a good reference? If someone claims that they achieve certain performance with some popular CNN architecture, how do we know s/he is not bragging?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may or may not have access to the training dataset yet. The testing dataset shall be something close to the real-world production scenario. I know this is too vague, but just assume you have a fair judge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt; Product teams sometimes asks engineering teams for quick (and dirty) solutions. Engineering teams want to assess the feasibility before say &quot;Yes we can do &lt;span class=&quot;math-container&quot;&gt;$95\%$&lt;/span&gt;&quot; and officially launch (and be responsible) the projects.&lt;/p&gt;&#xA;" OwnerUserId="35592" LastActivityDate="2020-03-31T06:26:51.827" Title="How to estimate the accuracy upper limit of any CNN model over a computer vision classification task" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;performance&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19881" PostTypeId="1" CreationDate="2020-03-30T19:46:23.170" Score="2" ViewCount="225" Body="&lt;p&gt;I am new to deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am training a model and I am getting a root mean squared error (RMSE) greater on the test dataset than on the training dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qlotg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qlotg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the reason behind this? Is this acceptable to get the RMSE greater in test data?&lt;/p&gt;&#xA;" OwnerUserId="33670" LastEditorUserId="2444" LastEditDate="2020-03-31T12:48:28.377" LastActivityDate="2020-04-21T11:04:58.263" Title="Is it normal to have the root mean squared error greater on the test dataset than on the training dataset?" Tags="&lt;deep-learning&gt;&lt;performance&gt;&lt;mean-squared-error&gt;" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19882" PostTypeId="2" ParentId="19881" CreationDate="2020-03-30T20:52:39.310" Score="0" Body="&lt;p&gt;RMSE stands for Root Mean Squared Error. As the name suggests, it is calculated by taking the square root over the mean of the squared errors of individual points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is normal for the test error to be higher than the train error and in most cases, the test error will be greater than the train error. &lt;/p&gt;&#xA;" OwnerUserId="21229" LastEditorUserId="9947" LastEditDate="2020-03-30T21:37:42.090" LastActivityDate="2020-03-30T21:37:42.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19883" PostTypeId="2" ParentId="3668" CreationDate="2020-03-30T21:51:01.007" Score="2" Body="&lt;p&gt;For a function of one variable, there are only two options for directions in the domain: left or right, so it becomes almost trivial, but you can still talk about gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would take steps to the left if the slope/derivative is positive and make steps to the right if the slope/derivative is negative--i.e. the opposite direction of the derivative (the 1d version of the &quot;gradient&quot; in gradient descent), which is equivalent to the higher dimensional case.&lt;/p&gt;&#xA;" OwnerUserId="35605" LastActivityDate="2020-03-30T21:51:01.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19884" PostTypeId="2" ParentId="15942" CreationDate="2020-03-31T00:05:31.680" Score="-1" Body="&lt;p&gt;I am not good with math notations so I'll stick to language more. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first equality holds when all hypotheses shattering a set of certain dimension (whose superset has not been shattered) are non overlapping with other hypotheses shattering similar sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Informally let' see with an example what this means. Let's say &lt;span class=&quot;math-container&quot;&gt;$\mathcal H = \{h_1, h_2,h_3, h_4, h_5, h_6\}$&lt;/span&gt; and all have unique behaviours on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; which implies &lt;span class=&quot;math-container&quot;&gt;$|\mathcal H_A| = |\mathcal H|$&lt;/span&gt;. Now if &lt;span class=&quot;math-container&quot;&gt;$\{h_1, h_2,h_3, h_4\}$&lt;/span&gt; shatters a subset &lt;span class=&quot;math-container&quot;&gt;$B_1$&lt;/span&gt; of size &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; then it also shatters all subsets of &lt;span class=&quot;math-container&quot;&gt;$B_1$&lt;/span&gt; and number of subsets including &lt;span class=&quot;math-container&quot;&gt;$B_1$&lt;/span&gt; itself of &lt;span class=&quot;math-container&quot;&gt;$B_1$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$2^{|B_1|} = 4$&lt;/span&gt; (in this case). Similarly, &lt;span class=&quot;math-container&quot;&gt;$\{h_5 h_6\}$&lt;/span&gt; must shatter another subset &lt;span class=&quot;math-container&quot;&gt;$B_2$&lt;/span&gt; with size &lt;span class=&quot;math-container&quot;&gt;$=1$&lt;/span&gt; and using similar argument as before we get subsets &lt;span class=&quot;math-container&quot;&gt;$=2$&lt;/span&gt;. Which shows &lt;span class=&quot;math-container&quot;&gt;$2+4=6$&lt;/span&gt; (i.e &lt;span class=&quot;math-container&quot;&gt;$B_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_2$&lt;/span&gt; and all its subsets form the RHS of the equation whose total number is &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: This will also hold true for hypothesis having same behaviour on subset &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; i.e if another hypothesis &lt;span class=&quot;math-container&quot;&gt;$h_7$&lt;/span&gt; exist which give same output on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$h_6$&lt;/span&gt;. There will be no effect since the former derivation is based on &lt;span class=&quot;math-container&quot;&gt;$\mathcal H_A$&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$\mathcal H$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case during which &lt;span class=&quot;math-container&quot;&gt;$|\mathcal H_A| &amp;lt; |\{B \subseteq A: \mathcal{H} \text { shatters } B\} |$&lt;/span&gt; , is when we have overlapping hypothesis i.e if certain hypothesis exists which helps in shattering &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; different subsets none of which is subset of each other. Which means now if  &lt;span class=&quot;math-container&quot;&gt;$\{h_1, h_2,h_3, h_4\}$&lt;/span&gt; shatters some &lt;span class=&quot;math-container&quot;&gt;$B_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\{h_4, h_5\}$&lt;/span&gt; with the overlap of &lt;span class=&quot;math-container&quot;&gt;$h_4$&lt;/span&gt; shatters another hypothesis then this is the case &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; will hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now coming to the second part of the equation, we can see equality will hold when the hypothesis shatters all possible subsets of size &lt;span class=&quot;math-container&quot;&gt;$\leq d$&lt;/span&gt;. If we define a hypothesis as a set of points where the values is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; otherwise, then we can verify this. Clearly, if &lt;span class=&quot;math-container&quot;&gt;$\mathcal H$&lt;/span&gt; shatters all sets of size &lt;span class=&quot;math-container&quot;&gt;$\leq d$&lt;/span&gt; then we can pick any set of size &lt;span class=&quot;math-container&quot;&gt;$i \leq d$&lt;/span&gt;, and total number of such sets will be &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=0}^d \begin{array}{c}{|A|} \\ {i}\end{array}$&lt;/span&gt;. In all other cases &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; will hold. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; A hypothesis class having VC dimension &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; does not necessarily shatter all sets of size less &lt;span class=&quot;math-container&quot;&gt;$\leq d$&lt;/span&gt; and hence the inequality. Check tis &lt;a href=&quot;https://ai.stackexchange.com/questions/19846/how-to-prove-h-with-vc-dimension-d-shatter-all-subsets-with-size-less-than&quot;&gt;answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can now play with the different scenarios to get &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; cases as required, and as can  be clearly seen the assumptions for equality in both the part of the equations are independent of each other. One deals with overlapping behaviour, while other deals with shattering of subsets &lt;span class=&quot;math-container&quot;&gt;$\leq d$&lt;/span&gt;. But note only under a certain condition will the equality hold on both sides, namely the hypothesis class shatters all possible subsets of size &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; non overlappingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sidenote:&lt;/strong&gt; The first part of the equation can probably be proven by formal number theory, but I am not sure how.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-03-31T00:11:24.250" LastActivityDate="2020-03-31T00:11:24.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19885" PostTypeId="1" CreationDate="2020-03-31T01:42:11.417" Score="1" ViewCount="21" Body="&lt;p&gt;I am performing an adversarial machine learning attack on a neural network for network traffic classification. For adding adversarial perturbations in features such as packet interarrival times and packet size, what norm should I use as a constraint? (eg. l1 norm, l2 norm, l-infinity norm, etc.)&lt;/p&gt;&#xA;" OwnerUserId="31240" LastActivityDate="2020-03-31T01:42:11.417" Title="How do I decide which norm to use for placing a constraint on my adversarial perturbation?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;math&gt;&lt;constraint-satisfaction-problems&gt;&lt;adversarial-ml&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19886" PostTypeId="2" ParentId="12034" CreationDate="2020-03-31T02:58:26.627" Score="1" Body="&lt;p&gt;I think it's a real bot. I've seen people talking about how it's rendering has improved over time and they claimed that was an improvement in the Motion Capture suit. I think that's simply because the program has improved over time. Imagine YOU created AI Angel (assuming it's really AI), would you be more worried about it's functionality or it's realism first? I would want it to act real before it looks real. If you listen closely, the voice has also slowly been improving as well, which makes sense for both explanations. If fake, the voice changer improved, but if real they simply gave it a network of human communication to listen to and compare itself to. It's really quite easy (conceptually) to create a program that learns. If you want to learn more about how easy it can be to make an AI teach itself I recommend Code Bullet on Youtube. He has a variety of videos in which he creates AI's meant to learn how to function in a game at the highest performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I covered evidence of both sides of the question, but I personally (as a nerdy boy) believe Angelica is really an A.I.&lt;/p&gt;&#xA;" OwnerUserId="35609" LastActivityDate="2020-03-31T02:58:26.627" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19887" PostTypeId="1" CreationDate="2020-03-31T04:17:51.197" Score="1" ViewCount="35" Body="&lt;p&gt;I am new to CNN. What I have learned so far about the filters is that when we are giving a training example to our model, our model updates the weights by gradient descent to minimize the loss function.&#xA;So my question is how the weights are retained for a particular class label?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is vague as my knowledge is vague. It's my 4th hour to CNN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I am talking about the MNIST dataset with 10 labels. Let's say I am giving 1 image to my model initially. It will have a bigger loss for the forward pass. Let's say now it came for the back pass and adjusted the weights for and minimized the loss function for that label. &#xA;Now, when a new label arrives for training, how will it update the weights for filters which have already been updated according to the previous label?&lt;/p&gt;&#xA;" OwnerUserId="35611" LastEditorUserId="2444" LastEditDate="2020-03-31T11:02:43.557" LastActivityDate="2020-04-30T15:11:21.033" Title="How are the weights retained for filters for a particular class in a CNN?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19888" PostTypeId="2" ParentId="19880" CreationDate="2020-03-31T06:26:51.827" Score="2" Body="&lt;p&gt;There is no easy rule for this. You can use transfer learning to select a model that works well on image classification. However the accuracy you achieve will be highly dependent on your training set.  If your training set is &quot;similar&quot; in quantity and quality to what was used  for the accuracy achieved by the transfer learning model in some application you have a reasonable chance of coming close to that accuracy. By similar is mean roughly the same number of images per class. By quality I mean what percentage of the pixels in the images that are occupied by the &quot;region of interest -ROI), same level of noise in the images etc. Also it depends on the nature of the classes. If the classes are widely different (elephants vs trees) the accuracy should be higher than if you are try to classify closely related images (human faces).&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-31T06:26:51.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19889" PostTypeId="1" AcceptedAnswerId="21212" CreationDate="2020-03-31T06:30:16.293" Score="4" ViewCount="70" Body="&lt;p&gt;Greetings to all respected colleagues!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to consult on the use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Field-programmable_gate_array&quot; rel=&quot;nofollow noreferrer&quot;&gt;FPGA&lt;/a&gt;s and &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;neurochips&lt;/a&gt;. I plan to use it in my laboratory project for programming control systems on neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my work, there are a lot of applications of neural networks, and I became interested in their programming on FPGAs and neurochips.&#xA;But I don’t know a single example of a really made and working laboratory prototype in which a neural network is implemented on an FPGA or on a neurochip and controls something. If someone shares the link, I would carefully study it.&lt;/p&gt;&#xA;" OwnerUserId="32829" LastEditorUserId="2444" LastEditDate="2020-03-31T11:29:39.137" LastActivityDate="2020-05-16T21:04:32.817" Title="Are there examples of neural networks (used for control) implemented on a FPGA or on a neurochip?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;neuromorphic-engineering&gt;&lt;control-theory&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19890" PostTypeId="2" ParentId="19879" CreationDate="2020-03-31T06:33:24.717" Score="1" Body="&lt;p&gt;The convolutions are linear transformations. However in typical applications a non linear activation function like RELU is used following the convolution to provide non-linearity otherwise a convolutional neural network would just be a net linear transformation.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-03-31T06:33:24.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19891" PostTypeId="1" CreationDate="2020-03-31T07:51:31.220" Score="4" ViewCount="796" Body="&lt;p&gt;I am trying to implement a convolutional autoencoder with a dense layer at the bottleneck do to some dimensional reduction. I have seen two approaches for this which arent particularly scalable. The first was to introduce 2 dense layers (one at the bottle neck and one before &amp;amp; after that has the same number of nodes as the conv2d layer that precedes the dense layer in the encoder section: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input_image_shape=(200,200,3)&#xA;encoding_dims = 20&#xA;&#xA;encoder = Sequential()&#xA;encoder.add(InputLayer(input_image_shape))&#xA;encoder.add(Conv2D(32, (3,3), activation=&quot;relu, padding=&quot;same&quot;))&#xA;encoder.add(MaxPooling2D((2), padding=&quot;same&quot;))&#xA;encoder.add(Flatten())&#xA;encoder.add(Dense(32*100*100, activation=&quot;relu&quot;))&#xA;encoder.add(Dense(encoding_dims, activation=&quot;relu&quot;))&#xA;&#xA;#The decoder&#xA;decoder = Sequential()&#xA;decoder.add(InputLayer((encoding_dims,)))&#xA;decoder.add(Dense(32*100*100, activation=&quot;relu&quot;))&#xA;decoder.add(Reshape((100, 100, 32)))&#xA;decoder.add(UpSampling2D(2))&#xA;decoder.add(Conv2D(3, (3,3), activation=&quot;sigmoid&quot;, padding=&quot;same&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Its easy to see why this approach blows up as there are two densely connected layers with (32*100*100) nodes each or more or in that ballpark which is nuts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach I have found which makes sense for b/w images such as the mnist stuff is to introduce an arbitrary number of encoding dimensions and reshape it (&lt;a href=&quot;https://medium.com/analytics-vidhya/building-a-convolutional-autoencoder-using-keras-using-conv2dtranspose-ca403c8d144e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/analytics-vidhya/building-a-convolutional-autoencoder-using-keras-using-conv2dtranspose-ca403c8d144e&lt;/a&gt;). The following chunk of code is copied from the link, I claim no credit for it: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ENCODER&#xA;inp = Input((28, 28,1))&#xA;e = Conv2D(32, (3, 3), activation='relu')(inp)&#xA;e = MaxPooling2D((2, 2))(e)&#xA;e = Conv2D(64, (3, 3), activation='relu')(e)&#xA;e = MaxPooling2D((2, 2))(e)&#xA;e = Conv2D(64, (3, 3), activation='relu')(e)&#xA;l = Flatten()(e)&#xA;l = Dense(49, activation='softmax')(l)&#xA;#DECODER&#xA;d = Reshape((7,7,1))(l)&#xA;d = Conv2DTranspose(64,(3, 3), strides=2, activation='relu', padding='same')(d)&#xA;d = BatchNormalization()(d)&#xA;d = Conv2DTranspose(64,(3, 3), strides=2, activation='relu', padding='same')(d)&#xA;d = BatchNormalization()(d)&#xA;d = Conv2DTranspose(32,(3, 3), activation='relu', padding='same')(d)&#xA;decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(d)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The question is then is there a more rigorous way/ accepted of adding a conv2d layer after a 2d layer?&lt;/p&gt;&#xA;" OwnerUserId="34002" LastActivityDate="2021-04-06T07:03:05.310" Title="Adding a dense layer after a conv2d layer in a convolutional autoencoder" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19893" PostTypeId="1" CreationDate="2020-03-31T11:00:37.823" Score="2" ViewCount="25" Body="&lt;p&gt;What is the difference between training a model with RGB images and using only the color channels separately (like only the red channel, green channel, etc.)? Would the model also learn patterns between the different colors in the first case? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for me the single-channel results are relevant but also the patterns between different channels are relevant, it would be beneficial to use them together?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking this because I want to apply this to signals of an accelerometer that has x, y, z-axis data. And I want to increase the resolution of the data. Will the model learn to combine all features from different axis if I input (1024, 3) length, channels of a one-dimensional signal into my one-dimensional CNN? &lt;/p&gt;&#xA;" OwnerUserId="35615" LastEditorUserId="2444" LastEditDate="2020-03-31T11:11:06.643" LastActivityDate="2020-03-31T11:11:06.643" Title="What is the difference between training a model with RGB images and using only the color channels separately?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19894" PostTypeId="1" CreationDate="2020-03-31T11:21:22.900" Score="4" ViewCount="62" Body="&lt;p&gt;Are there any commonly used activation functions (e.g. that take values in &lt;span class=&quot;math-container&quot;&gt;$(0,.5)\cup (.5,1)$&lt;/span&gt;)? Preferably for classification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why? I was looking for commonly used activation functions on Google, and I noticed that all activation functions are continuous. However, I believe this is not needed in Hornik's paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I did a bit of testing myself, with a discontinuous activation function on the MNIST dataset, the results were good. So I was curious if anyone else used this kind of activation function.  &lt;/p&gt;&#xA;" OwnerUserId="34185" LastEditorUserId="2444" LastEditDate="2020-04-01T00:53:34.773" LastActivityDate="2020-04-01T00:53:34.773" Title="Are there any commonly used discontinuous activation functions?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19895" PostTypeId="1" AcceptedAnswerId="19898" CreationDate="2020-03-31T11:59:46.407" Score="2" ViewCount="103" Body="&lt;p&gt;We require to find the gradient of loss function(cost function) w.r.t to the weights to use optimization methods such as SGD or gradient descent. So far, I have come across two ways to compute the gradient:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;BackPropagation&lt;/li&gt;&#xA;&lt;li&gt;Calculating gradient of loss function by calculus&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I found many resources for understanding backpropagation. &#xA;The 2nd method I am referring to is the image below(taken for a specific example, e is the error: difference between target and prediction):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OSOVB.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OSOVB.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the proof was mentioned in this paper:&lt;a href=&quot;https://arxiv.org/pdf/1802.01528.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, I found this method while reading this &lt;a href=&quot;https://www.pyimagesearch.com/2016/10/10/gradient-descent-with-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;.(You might have to scroll down to see the code: gradient = X.T.dot(error) / X.shape[0]  ) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is are the two methods of finding gradient of cost function same? It appears different and if yes, which one is more efficient( though one can guess it is backpropagation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would be grateful for any help. Thanks for being patient(it's my 1st time learning ML). &lt;/p&gt;&#xA;" OwnerUserId="35616" LastActivityDate="2020-03-31T15:44:32.933" Title="Different methods of calculating gradients of cost function(loss function)" Tags="&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19896" PostTypeId="2" ParentId="19881" CreationDate="2020-03-31T12:34:58.807" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I am training a model and i am getting test results greater than train results.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You don't give us too many details, but most probably it's &lt;strong&gt;underfitting&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What could be the reason behind this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Underfitting is often a result of an excessively simple model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Too much regularization techniques were used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is this acceptable to get the RMSE greater in test data?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but that indicates a problem with model, so you should be aware of the consequences. &lt;/p&gt;&#xA;" OwnerUserId="19476" LastActivityDate="2020-03-31T12:34:58.807" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="19897" PostTypeId="2" ParentId="19887" CreationDate="2020-03-31T14:30:45.510" Score="1" Body="&lt;p&gt;Besides the last layer rest of the weights are shared among all classes. When an image is passed to the network all weights are updated accordingly. The only weights that are directly responsible for one specific class are the ones of the final layer. The rest of the weights are updated to find the best values to minimize the average loss for all classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To rephrase there aren't &quot;filters&quot; in convolutional layers that are specific for a single class. They are used to extract features so that the final layer can make the final prediction (which has weights for each specific class).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest you look a bit into how gradient descent and backpropagation works. &lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-03-31T14:30:45.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19898" PostTypeId="2" ParentId="19895" CreationDate="2020-03-31T15:44:32.933" Score="1" Body="&lt;p&gt;I'm pretty sure they're the same thing. Both backpropagation and gradient descent's essential ideas are to calculate the partial derivative of the cost with respect to each weight, and subtract the partial derivative times the learning rate.&lt;/p&gt;&#xA;" OwnerUserId="34505" LastActivityDate="2020-03-31T15:44:32.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19899" PostTypeId="1" CreationDate="2020-03-31T15:45:49.400" Score="2" ViewCount="53" Body="&lt;p&gt;It is well-known that deep feedforward networks can approximate any continuous function from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^k$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^l$&lt;/span&gt;, (uniformly on compacts).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in practice feature maps are typically used to improve the learning quality and likewise, readout maps are used to make neural networks suited for specific learning tasks.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Classification: networks are composed with the softmax (readout) function so they take values in &lt;span class=&quot;math-container&quot;&gt;$(0,1)^l$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are examples of commonly used feature and readout maps?&lt;/p&gt;&#xA;" OwnerUserId="31649" LastEditorUserId="2444" LastEditDate="2020-04-01T12:56:36.590" LastActivityDate="2020-04-01T12:56:36.590" Title="What are examples of commonly used feature and readout maps?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;feature-selection&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19901" PostTypeId="1" CreationDate="2020-03-31T19:32:13.583" Score="2" ViewCount="23" Body="&lt;p&gt;This may come across as an open and opinion-based question, I definitely want to hear expert opinions on the subject, but I am also looking for references to materials that I can read deeply.&lt;/p&gt;&#xA;&lt;p&gt;One of the ways question answering systems can be classified is by the type of data source that they use:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Structured knowledge bases with ontologies (DBPedia, WikiData, Yago, etc.).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unstructured text corpora that contain the answer in natural language (Wikipedia).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Hybrid systems that search for candidate answers in both structured and unstructured data sources.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;From my reading, it appears as though structured knowledge bases/knowledge graphs were much more popular back in the days of the semantic web and when the first personal assistants (Siri, Alexa, Google Assistant) came onto the scene.&lt;/p&gt;&#xA;&lt;p&gt;Are they dying out now in favor of training a deep learning model over a vast text corpus like Bert and/or Meena? Do they have a future in question answering?&lt;/p&gt;&#xA;" OwnerUserId="35587" LastEditorUserId="2444" LastEditDate="2021-01-26T17:27:23.730" LastActivityDate="2021-01-26T17:27:23.730" Title="Will structured knowledge bases continue to be used in question answering with the likes of BERT gaining popularity?" Tags="&lt;reference-request&gt;&lt;datasets&gt;&lt;bert&gt;&lt;question-answering&gt;&lt;knowledge-base&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19902" PostTypeId="1" CreationDate="2020-03-31T19:58:58.237" Score="1" ViewCount="43" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tEFnm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tEFnm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how they describe their infrastructure in &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&lt;/a&gt;. I want to implement the game of Atari Breakout.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;import torch.nn.functional as F&#xA;&#xA;class DQN(nn.Module):&#xA;    def __init__(self, height, width):&#xA;        super(DQN, self).__init__()&#xA;&#xA;        self.height = height&#xA;        self.width = width&#xA;&#xA;        self.conv1 = nn.Conv2d(in_channels=4, out_channels=16, kernel_size=8, stride=4)&#xA;        self.conv2 = nn.Conv2d(in_channels=6, out_channels=32, kernel_size=4, stride=2)&#xA;&#xA;        self.fc = nn.Linear(in_features=????, out_features=256)&#xA;        self.out = nn.Linear(in_features=256, out_features=4)&#xA;&#xA;    def forward(self, state):&#xA;&#xA;        # (1) Hidden Conv. Layer&#xA;        self.layer1 = F.relu(self.conv1(state))&#xA;&#xA;        #(2) Hidden Conv. Layer&#xA;        self.layer2 = F.relu(self.conv2(self.layer1))&#xA;&#xA;        #(3) Hidden Linear Layer&#xA;        self.layer3 = self.fc(self.layer2)&#xA;&#xA;        #(4) Output&#xA;        actions = self.out(self.layer3)&#xA;&#xA;        return actions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I will probably instantiate my policy network and my target network the following way : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;policy_net = DQN(envmanager.get_height(), envmanager.get_width()).to(device)&#xA;target_net = DQN(envmanager.get_height(), envmanager.get_width()).to(device)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am very new in the world of Reinforcement Learning. I would like to implement their infrastructure in the &lt;code&gt;DQN()&lt;/code&gt;, but I think I am wrong in several places. Am I good here? If not, how can I fix it so that it reflect the infrastructure from the above picture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the formula to calculate the output size is equal to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$O=\frac{W−K+2P}{S}+1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; is the output height/length, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is the input height/length, &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is the filter size, &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is the padding, and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the stride.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I obtained for&lt;code&gt;self.fc = nn.Linear(in_features=????, out_features=256)&lt;/code&gt; that &lt;code&gt;in_features&lt;/code&gt; must be equal to &lt;span class=&quot;math-container&quot;&gt;$32*9*9$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-04-01T15:24:01.577" LastActivityDate="2020-04-01T15:24:01.577" Title="Atari Breakout Infrastructure" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19903" PostTypeId="5" CreationDate="2020-03-31T20:09:25.517" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-31T20:09:25.517" LastActivityDate="2020-03-31T20:09:25.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19904" PostTypeId="4" CreationDate="2020-03-31T20:09:25.517" Score="0" Body="For questions related to graph neural networks, which are neural networks applied to graphs and their problems." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-03-31T20:09:25.517" LastActivityDate="2020-03-31T20:09:25.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19905" PostTypeId="1" AcceptedAnswerId="20024" CreationDate="2020-03-31T23:37:06.820" Score="2" ViewCount="73" Body="&lt;p&gt;The &lt;a href=&quot;https://www.deeplearningbook.org/contents/rnn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep Learning&lt;/em&gt; book&lt;/a&gt; mentions that it's been used for years but the oldest sources it mentions are from 2012: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A simple type of solution has been in use by practitioners for many years: clipping the gradient. There are diﬀerent instances of this idea (Mikolov, 2012; Pascanu et al., 2013). One option is to clip the parameter gradient from a mini-batch element-wise (Mikolov, 2012), just before the parameter update. Another is to clip the &lt;span class=&quot;math-container&quot;&gt;$||g||$&lt;/span&gt; of the gradient &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; (Pascanu et al., 2013) just before the parameter update  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But I find it hard to believe that the first uses and mentions of gradient clipping are from 2012. Does anyone know the origins of the solution? &lt;/p&gt;&#xA;" OwnerUserId="34395" LastEditorUserId="34395" LastEditDate="2020-04-05T17:32:10.170" LastActivityDate="2020-04-05T17:32:10.170" Title="Which work originally introduced gradient clipping?" Tags="&lt;reference-request&gt;&lt;papers&gt;&lt;history&gt;&lt;exploding-gradient-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19906" PostTypeId="1" AcceptedAnswerId="19923" CreationDate="2020-04-01T02:15:37.100" Score="1" ViewCount="56" Body="&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;from torch.autograd import Variable&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;x_data = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]]).float()&#xA;y_data = torch.tensor([0, 1, 1, 0]).float()&#xA;&#xA;class Model(nn.Module):&#xA;    def __init__(self, input_size, H1, output_size):&#xA;        super().__init__()&#xA;        self.linear_input = nn.Linear(input_size, 2)&#xA;        self.linear_output = nn.Linear(2, output_size)&#xA;        self.sigmoid = torch.nn.Sigmoid()&#xA;&#xA;    def forward(self, x):&#xA;        x = self.sigmoid(self.linear_input(x))&#xA;        x = self.sigmoid(self.linear_output(x))&#xA;        return x&#xA;&#xA;    def predict(self, x):&#xA;        return (self.forward(x) &amp;gt;= 0.5).float()&#xA;&#xA;model = Model(2,2,1)&#xA;lossfunc = nn.BCELoss()&#xA;optimizer = optim.Adam(model.parameters(), lr=0.01)&#xA;epochs = 2000&#xA;losses = []&#xA;&#xA;for i in range(epochs):&#xA;    pred = model(x_data).view(-1)&#xA;    loss = lossfunc(pred, y_data)&#xA;    print(&quot;epochs:&quot;, i, &quot;loss:&quot;, loss.item())&#xA;    losses.append(loss.item())&#xA;    optimizer.zero_grad()&#xA;    loss.backward() &#xA;    optimizer.step()&#xA;&#xA;def cal_score(X, y):&#xA;    y_pred = model.predict(X)&#xA;    score = float(torch.sum(y_pred.squeeze(-1) == y.byte().float())) / y.shape[0]&#xA;    return score&#xA;&#xA;print('test score :', cal_score(x_data, y_data))&#xA;def plot_decision_boundray(X):&#xA;    x_span = np.linspace(min(X[:, 0]), max(X[:, 0]))&#xA;    y_span = np.linspace(min(X[:, 1]), max(X[:, 1]))&#xA;    xx, yy = np.meshgrid(x_span, y_span)&#xA;    grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()]).float()&#xA;    pred_func = model.forward(grid)&#xA;    z = pred_func.view(xx.shape).detach().numpy()&#xA;    plt.contourf(xx, yy, z)&#xA;    plt.show()&#xA;&#xA;plot_decision_boundray(x_data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, it's a simple neural network which consists of one hidden layer using BCELoss and Adam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ant5R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ant5R.png&quot; alt=&quot;Correct result&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, it results in the correct one like above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cSyTF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cSyTF.png&quot; alt=&quot;Wrong result&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is sometimes stuck in a local minima and a decision boundary becomes awkward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the input data is limited, I guess that preprocessing of those data might not be possible and only initial weights matter in this problem. I tried initializing them with normal distribution but it didn't work. How can I approach this problem?&lt;/p&gt;&#xA;" OwnerUserId="18427" LastActivityDate="2020-04-01T23:36:47.823" Title="XOR-solving neural network is suffering from local minima" Tags="&lt;pytorch&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19908" PostTypeId="1" AcceptedAnswerId="19910" CreationDate="2020-04-01T06:03:10.190" Score="1" ViewCount="53" Body="&lt;p&gt;Gradient descent is used to reduce the loss and regularization is used to fight over-fitting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any relation between gradient descent and regularization, or both are independent of each other?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-04-02T22:01:18.183" LastActivityDate="2020-04-02T22:40:52.190" Title="What is relation between gradient descent and regularization in deep learning?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;gradient-descent&gt;&lt;regularization&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19909" PostTypeId="1" CreationDate="2020-04-01T06:55:19.237" Score="2" ViewCount="54" Body="&lt;p&gt;Consider that my input is an RGB image. The size of my image is &lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt;. I'm trying to implement &lt;a href=&quot;https://arxiv.org/abs/1410.8516&quot; rel=&quot;nofollow noreferrer&quot;&gt;NICE algorithm presented by Dinh&lt;/a&gt;. The bijective function &lt;span class=&quot;math-container&quot;&gt;$f: \mathbb{R}^d \to \mathbb{R}^d$&lt;/span&gt; maps &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;. So I have &lt;span class=&quot;math-container&quot;&gt;$p_Z(Z)=p_X(X)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I can't understand is that &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is much bigger than &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;. Does this mean that I should downsample the inputs? Does the resulting loss function change if I add a downsampling layer at the beginning of the neural net and also add an upsampling layer at the end of the net?  &lt;/p&gt;&#xA;" OwnerUserId="35633" LastEditorUserId="35633" LastEditDate="2020-04-02T14:22:25.660" LastActivityDate="2020-04-02T14:22:25.660" Title="Do I have to downsample the input and upsample the output of the neural network when implementing the NICE algorithm?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;generative-model&gt;&lt;papers&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19910" PostTypeId="2" ParentId="19908" CreationDate="2020-04-01T07:44:41.637" Score="4" Body="&lt;p&gt;Usually, when talking about regularization for neural networks there are 3 main types:&#xA;L1, L2 and dropout. All affect the gradient descent procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L1 and L2 regularization is implemented in the loss function, and therefore are part of gradient descent directly by altering the derivatives of the loss function thereby altering the weight update rules of the network during gradient descent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For L1 you add a penalty based on the &lt;span class=&quot;math-container&quot;&gt;$\mathcal L^1$&lt;/span&gt; norm of the weight vector, while for L2 you add a penalty based on the &lt;span class=&quot;math-container&quot;&gt;$\mathcal L^2$&lt;/span&gt;  norm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dropout, there is no direct impact on the loss function, but you are still interfering in the gradient descent procedure indirectly by masking nodes to alter the forward and backward propagation.&lt;/p&gt;&#xA;" OwnerUserId="22906" LastEditorUserId="9947" LastEditDate="2020-04-02T22:40:52.190" LastActivityDate="2020-04-02T22:40:52.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19911" PostTypeId="1" CreationDate="2020-04-01T09:23:18.910" Score="3" ViewCount="611" Body="&lt;p&gt;I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the architecture described in the paper &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt;. And I tested the model on a simpler environment (like CartPole), which worked great, but I am not seeing any progress at all with Pong, I have been training the model for 2-3 hours and its performance is no better than taking random actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I just keep waiting or there might be something wrong with my code. Around how many episodes should it take before I see some positive results?&lt;/p&gt;&#xA;" OwnerUserId="26336" LastEditorUserId="2444" LastEditDate="2020-04-01T12:33:23.490" LastActivityDate="2020-04-01T12:33:23.490" Title="How much time does it take to train DQN on Atari environment?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;implementation&gt;&lt;computational-complexity&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19912" PostTypeId="1" CreationDate="2020-04-01T10:01:31.227" Score="1" ViewCount="30" Body="&lt;p&gt;I'm trying to make a gym environment for a simulation problem. In my gym environment, I have a set of illegal states which I don't want my agent to go into them. What is the easiest way to add such logic to my environment, should I use the wrapper classes? I didn't quite get them, I tried to extend the &lt;code&gt;MultiDiscrete&lt;/code&gt; space by inheriting a class from it and override the &lt;code&gt;MulriDiscrete.sample&lt;/code&gt; function to stop the environment from going into the illegal states, but is there a more efficient way to do it?&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-04-01T18:33:11.390" LastActivityDate="2020-04-01T18:33:11.390" Title="How should I avoid illegal states in OpenAI's gym?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19913" PostTypeId="1" CreationDate="2020-04-01T10:48:24.603" Score="1" ViewCount="133" Body="&lt;p&gt;I want to integrate my environment into the OpenAI's gym and then use &lt;a href=&quot;https://stable-baselines.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the stable baselines library&lt;/a&gt; for training it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The learning method in the stable baseline is with one-line learning and you don't have access to the actions that are taken during the training.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.learn(total_timesteps=10000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More specifically, you don't do the line where you sample from the environment in your code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;action = space.sample()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I like to add some logic to the place where I choose the action of the next state and reject some actions that the logic doesn't apply to them (like chess board illegal moves), something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for _ in range(1000):&#xA;    action = env.space.sample()&#xA;    if some_logic(action):&#xA;         continue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One way to do it is to write a wrapper for the action_space &lt;code&gt;sample()&lt;/code&gt; function and only choose legal actions. Like the one &lt;a href=&quot;https://github.com/rockingdingo/gym-gomoku/blob/master/gym_gomoku/envs/gomoku.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; class &lt;code&gt;DiscreteWrapper(spaces.Discrete)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is that a stable baseline only accepts certain data types and doesn't allow that either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do it in a way that is integrable into the stable baseline framework and doesn't violate the stable baselines criteria? If that is not possible at all, does anyone know a reinforcement learning framework other that unlike stable baseline allows access to the actions? &lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-04-01T12:52:37.393" LastActivityDate="2020-04-01T12:52:37.393" Title="How can I add logic for invalid moves when using stable-baselines in OpenAI's gym?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19914" PostTypeId="1" CreationDate="2020-04-01T11:20:39.823" Score="1" ViewCount="65" Body="&lt;p&gt;I'm a big fan of animation and have kept an eye on the deepfake's ability to replicate full body motion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I ask&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a deepfake software available I can use to gather animation from a video?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there publically released a deepfake body tracking tools out there? Even for normal video&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35639" LastEditorUserId="2444" LastEditDate="2020-04-01T18:12:15.893" LastActivityDate="2020-04-01T18:12:15.893" Title="Can I use deepfake to rotoscope for animation?" Tags="&lt;game-ai&gt;&lt;deepfakes&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19915" PostTypeId="1" CreationDate="2020-04-01T11:31:59.257" Score="1" ViewCount="71" Body="&lt;p&gt;I am working on a problem, and I want to implement it as a reinforcement learning problem and integrate it into the OpenAI's gym. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My states are in the form of lists of length &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, where each element is chosen from a discrete interval &lt;span class=&quot;math-container&quot;&gt;$[0, m]$&lt;/span&gt;. For example, for &lt;span class=&quot;math-container&quot;&gt;$n=6$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m=3$&lt;/span&gt;, this is a sample from the observation space:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0 2 1 3 3 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the possible accessible states from this space is a set of other lists that are achieved by changing a number of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; elements the elements in the list with a number from the same &lt;span class=&quot;math-container&quot;&gt;$[0, m]$&lt;/span&gt;. For example, for &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt;, we can have the following states as two subsequent states of the previous state:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0 2 2 3 3 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0 3 1 3 3 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is an efficient way to represent the &quot;actions&quot; in the OpenAI gym for such a scenario?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way that comes to my mind is to just use the next state as the action itself, for example, if I write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;action = env.action_space.sample()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The action would be the next state (which also implicitly contains the action) and then in the &lt;code&gt;env.step(action)&lt;/code&gt; make the state equal to the next state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know a better way or using the implicit action representation with the next state is the optimal way? Does anyone know a predefined gym environment that also has the same representation? What are the cons of the implicit representation of the actions that I just explained?&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-04-01T12:55:57.640" LastActivityDate="2020-04-01T12:55:57.640" Title="How to define observation and action space for an array-like input?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19916" PostTypeId="2" ParentId="10910" CreationDate="2020-04-01T12:43:25.623" Score="0" Body="&lt;p&gt;Initial state:    The monkey, the suspended bananas, and two cratesin the room&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goal test:         Monkey has bananas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Successor function: To jump on crate; to jump off crate; Push crate from&lt;br&gt;&#xA;                  One spot to another; Walk from one spot to another;Grab bananas&#xA;                   (if standing on crate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cost function:           Number of actions.&lt;/p&gt;&#xA;" OwnerUserId="35642" LastActivityDate="2020-04-01T12:43:25.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19917" PostTypeId="2" ParentId="18489" CreationDate="2020-04-01T17:15:33.810" Score="1" Body="&lt;p&gt;Each node is a position in the arrays&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;values = value of the node&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;conn = indexes of connected nodes&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If its an undirected graph, each node must have all the nodes to which they are attached.&#xA;Instead, in directed graphs, only the start node has the index.&#xA;For your image:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;values = ['A','B','C','D','E']&#xA;conn = [[1,2,3],[0,4],[0,3,4],[0,2,3],[1,3]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example = 'A' -&gt; 1 ('B') ,2 ('C') ,3 ('D')&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def averageGraph(conn):&#xA;    if conn.all() != None:&#xA;        average = 0&#xA;        for node in conn:&#xA;            average = average + len(node) #len(node) = nº nodes connected = degree&#xA;        return average / len(conn)&#xA;    else:&#xA;        return None&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35656" LastActivityDate="2020-04-01T17:15:33.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19920" PostTypeId="2" ParentId="17629" CreationDate="2020-04-01T19:24:08.923" Score="1" Body="&lt;p&gt;Full disclosure: I work at Dessa, the company that developed this tech.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We built a machine learning experiment management tool, called Atlas. The main feature is experiment management, allowing you to run thousands of experiments concurrently. This might help with your problem above &lt;a href=&quot;https://github.com/dessa-oss&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dessa-oss&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35661" LastActivityDate="2020-04-01T19:24:08.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19922" PostTypeId="1" AcceptedAnswerId="19937" CreationDate="2020-04-01T22:48:55.417" Score="0" ViewCount="145" Body="&lt;p&gt;I have read a lot of information about several notions, like batch_size, epochs, iterations, but because of explanation was without numerical examples and I am not native speaker, I have some kind of problem of understanding still about those terms, so I decided to work with data. Let us suppose we have the following data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BmtIx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BmtIx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, it is just subset of original data, and I want to build a neural network with three hidden layers, the first layer contains 500 nodes, it takes input three variable and on each node, there is sigmoid activation function, next layer contains 100 node and sigmoid activation, the third one contains 50 node and sigmoid again, and finally we have one output with sigmoid to convert the result into 0 or 1 that classify whether a person with those attributes is female or male.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trained the model using Keras Tensorflow with the following code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model.fit(X_train,y_train,epochs=30)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this data, what does mean &lt;code&gt;epochs=30&lt;/code&gt;? Does it mean that all 177 rows  (with 3 input at times) will go to the model 30 times? What about &lt;code&gt;batch_size=None&lt;/code&gt; in &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/Model?version=stable#fit&quot; rel=&quot;nofollow noreferrer&quot;&gt;model.fit parameters&lt;/a&gt;? &lt;/p&gt;&#xA;" OwnerUserId="35668" LastEditorUserId="2444" LastEditDate="2020-04-02T00:51:54.587" LastActivityDate="2020-04-02T14:19:12.057" Title="What does it mean to have epochs=30 in Keras' fit method given certain data?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;keras&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19923" PostTypeId="2" ParentId="19906" CreationDate="2020-04-01T23:36:47.823" Score="0" Body="&lt;p&gt;From the log you can see that the loss value is still big compared to the first image, but at least it decrease on each iteration (even very very small). So I suggest you to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;try to use bigger learning rate, I think you model is underfiting.&lt;/li&gt;&#xA;&lt;li&gt;try to use another weight initialization method&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think the first solution will solve this problem as I see no problem in the code.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2020-04-01T23:36:47.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19924" PostTypeId="2" ParentId="19922" CreationDate="2020-04-01T23:55:59.253" Score="0" Body="&lt;p&gt;ok  so let me explain in my word how i  understood this process:&#xA;i know that one sample mean one row, therefore if we have data with size(177,3), that means  we have 177 sample. because we have divided  X and y into training and test, therefore we have following pairs (X_train,y_train)  and (X_test, y_test) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;now about batch size, if we have  let say 177 sample(177 row) and  2 batch_size , that means we have approximately  &lt;span class=&quot;math-container&quot;&gt;$177/2$&lt;/span&gt; batch right?update process goes  like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;let us suppose network takes 3 input and produce one output, from first sample of data, three data will go to the network and  output will be  generated, this output will be compared to the  first value of y_train and  cost function will be created, then next  sample will go(it means next three value) and compared to the second value of y_train, also second cost function will be generated, final cost function for first batch will be sum of those cost functions and using gradient method weights are updated, after that one new batches will go through the network and on the based on updated weights, new weights are generated, when all  &lt;span class=&quot;math-container&quot;&gt;$177/2$&lt;/span&gt;  batch will be finished , it will be our 1 epoch right? is that correct? &lt;/p&gt;&#xA;" OwnerUserId="35668" LastActivityDate="2020-04-01T23:55:59.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19927" PostTypeId="1" AcceptedAnswerId="19946" CreationDate="2020-04-02T03:18:24.713" Score="2" ViewCount="124" Body="&lt;p&gt;Many &lt;a href=&quot;https://www.forbes.com/sites/cognitiveworld/2019/06/10/how-far-are-we-from-achieving-artificial-general-intelligence/#2209a7446dc4&quot; rel=&quot;nofollow noreferrer&quot;&gt;experts&lt;/a&gt; seem to think that artificial general intelligence, or AGI, (on the level of humans) is possible and likely to emerge in the near-ish future. Some make the further step to say that superintelligence (much above the level of AGI) will appear soon after, through mechanisms like recursive self-improvement from AGI (from a &lt;a href=&quot;https://www.nickbostrom.com/papers/survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;survey&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, other sources say that such superintelligence is unlikely or impossible (&lt;a href=&quot;https://www.nytimes.com/2016/04/07/science/artificial-intelligence-when-is-the-singularity.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;, &lt;a href=&quot;https://aeon.co/essays/true-ai-is-both-logically-possible-and-utterly-implausible&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What assumptions do those who believe in superintelligence make? The emergence of superintelligence has generally been regarded as something low-probability but possible (e.g. &lt;a href=&quot;https://www.vox.com/future-perfect/2018/10/18/17957162/nuclear-war-asteroid-volcano-science-climate-change&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). However, I can't seem to find an in-depth analysis of what assumptions are made when positing the emergence of superintelligence. What specific assumptions do those who believe in the emergence of superintelligence make that are unlikely, and what have those who believe in the guaranteed emergence of superintelligence gotten wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the emergence of superintelligence is to be seen as a low-probability event in the future (on par with asteroid strikes, etc.), which seems to be the dominant view and is the most plausible, what assumptions exactly makes it low-probability?&lt;/p&gt;&#xA;" OwnerUserId="35673" LastEditorUserId="35673" LastEditDate="2020-04-02T21:13:26.160" LastActivityDate="2020-04-03T00:45:53.000" Title="What assumptions are made when positing the emergence of superintelligence?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;superintelligence&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19929" PostTypeId="2" ParentId="19868" CreationDate="2020-04-02T05:49:12.257" Score="1" Body="&lt;p&gt;Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a &quot;goal&quot; is necessary for rl, all you need is the rewards.&lt;/p&gt;&#xA;" OwnerUserId="35679" LastEditorUserId="35679" LastEditDate="2020-04-02T19:54:57.113" LastActivityDate="2020-04-02T19:54:57.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19930" PostTypeId="1" AcceptedAnswerId="19931" CreationDate="2020-04-02T06:38:07.897" Score="1" ViewCount="280" Body="&lt;p&gt;I  am not using replay here. Can this be a possible deep q learning pseudocode?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s - state    &#xA;a - action    &#xA;r - reward&#xA;n_s - next state&#xA;q_net - neural network representing q&#xA;&#xA;step()&#xA;{&#xA;&#xA;    get s,a,r,n_s&#xA;    q_target[s,a]=r+gamma*max(q_net[n_s,:])&#xA;    loss=mse(q_target,q_net[s,a])&#xA;    loss.backprop()&#xA;&#xA;}&#xA;&#xA;while(!terminal)&#xA;{    &#xA;    totalReturn+=step();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35679" LastEditorUserId="2444" LastEditDate="2020-04-02T16:01:13.343" LastActivityDate="2020-04-02T16:01:13.343" Title="Can this be a possible deep q learning pseudocode?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;pseudocode&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19931" PostTypeId="2" ParentId="19930" CreationDate="2020-04-02T08:05:48.747" Score="0" Body="&lt;p&gt;It looks generally valid to me. There are a couple of things missing/implied that I'd like to give feedback on though:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am not using replay here&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then it won't work, except for the most simple and trivial of problems (where you probably would not need a neural network anyway).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;get s,a,r,n_s&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would make it more explicit where you get these values from, and split up the assignments.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# before step()&#xA;s = env.start state()&#xA;a = behavior_policy(q_net[n_s,:])&#xA;&#xA;  # inside step(), first action&#xA;  r, n_s = env.step(s, a)&#xA;&#xA;  # ....rest of loop&#xA;&#xA;  # inside step(), last actions&#xA;  s = n_s&#xA;  a = behavior_policy(q_net[n_s,:])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above code, &lt;code&gt;env&lt;/code&gt; is the environment which takes state, action pairs as input and returns reward plus next state. It would be equally valid to have current state as a property of the environment, and query that when needed. The &lt;code&gt;behaviour_policy&lt;/code&gt; is a function that selects an action based on current action values - typically this might use an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy selection.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;while(!terminal) {&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You appear to run just one episode in your pseudo-code. You will want an outer loop to run many episodes. Also it is not clear how you are deciding the value of &lt;code&gt;terminal&lt;/code&gt; - in practice you will need a handle to current environment state via some variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without the outer loop to start new episodes, and some variables defined to communicate between sections of code, it is difficult to follow the code and decide where to add details (such as where &lt;code&gt;env.start_state()&lt;/code&gt; call should go, or whether something like &lt;code&gt;s = env.reset()&lt;/code&gt; would be more appropriate).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;totalReturn+=step();&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In your pseudocode you do not return anything from &lt;code&gt;step&lt;/code&gt; and it is not clear what you hope to do with this &lt;code&gt;totalReturn&lt;/code&gt; variable. Technically it won't equal the definition of return in RL for any state, not even the starting state if &lt;code&gt;gamma &amp;lt; 1.0&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the sum of all rewards seen in an episode is a useful metric. In Deep RL it is OK to treat &lt;code&gt;gamma&lt;/code&gt; as a solution hyperparameter, and your target metric can be expected undiscounted return from the start state. &lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-02T08:11:17.720" LastActivityDate="2020-04-02T08:11:17.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19934" PostTypeId="1" CreationDate="2020-04-02T10:18:28.133" Score="0" ViewCount="17" Body="&lt;p&gt;Our task is to do a special object detection: In the traditional case, the neural network will output some rectangle bounding boxes. But in our case, the network should &lt;strong&gt;output many nearly-vertical lines&lt;/strong&gt;. The lines will cut the image into many pieces, each piece will be one object. (Our object distribution is very special, and such lines perfectly exist.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; - My naive idea: Let the input image be 512x512x3. The output will be 1x64x3. The 3 channels are (a) the y coordinate of the line when x=256 (b) the angle of line (c) the confidence that the line exists. Note that the 64x1 means this output is very fat. Actually, it is just like YOLO/SSD, but in our case each &lt;em&gt;row&lt;/em&gt; predicts one line, instead of each &lt;em&gt;cell&lt;/em&gt; predicting one box. However, I really have no idea how to implement the inner network structure... I would appreciate for any ideas!&lt;/p&gt;&#xA;" OwnerUserId="20170" LastEditorUserId="20170" LastEditDate="2020-04-02T13:40:30.297" LastActivityDate="2020-04-02T13:40:30.297" Title="How can I perform object detection by cutting the image into many pieces each containing one object?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19936" PostTypeId="1" AcceptedAnswerId="19945" CreationDate="2020-04-02T12:31:44.487" Score="0" ViewCount="80" Body="&lt;p&gt;I am trying to implement the back-propagation algorithm for the following neural network.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2 input -&amp;gt; 3 input layer -&amp;gt; 1 output  &#xA;Activation f(x) -&amp;gt; sigmoid&#xA;Loss f(x) -&amp;gt; Yexpected-Yresult&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the Matlab/Octave function for backpropagating an XOR ANN. However, it doesn't seem to work, and it converges to 0.5. Is it correct?  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigmoid = @(z) 1./(1 + exp(-z));&#xA;sig_der = @(y) sigmoid(y).*(1-sigmoid(y));&#xA;    function [w1, w2, b1, b2] = backprop(w1,w2,b1,b2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i)&#xA;      delta2 = (sig_der(last_layer,sigmoid)).*cost;&#xA;      delta1 = (sig_der(mid_layer,sigmoid)).*sum(delta2.*w2);&#xA;      w2 = w2 + 0.001 .* mid_layer .* delta2;&#xA;      w1 = w1 + 0.001 .* data(1:2,i) .* delta1.';&#xA;      b1 = b1 + 0.001 .* delta1';&#xA;      b2 = b2 + 0.001 .* delta2;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;&#xA;graphics_toolkit(&quot;gnuplot&quot;)&#xA;sigmoid = @(z) 1./(1 + exp(-z));&#xA;sig_der = @(y) sigmoid(y).*(1-sigmoid(y));&#xA;&#xA;&#xA;function [cost, mid_layer, last_layer] = forward(w1,w2,b1,b2,data,sigmoid,i)&#xA;  mid_layer = sigmoid(sum(data(1:2,i).*w1)'-b1);&#xA;  last_layer = sigmoid(sum(mid_layer.*w2)-b2);&#xA;  cost = data(3,i)-last_layer;&#xA;end&#xA;&#xA;function [w1, w2, b1, b2] = backprop(w1,w2,b1,b2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i)&#xA;  delta2 = (sig_der(last_layer,sigmoid)).*cost;&#xA;  delta1 = (sig_der(mid_layer,sigmoid)).*sum(delta2.*w2);&#xA;  w2 = w2 + 0.00001 .* mid_layer .* delta2;&#xA;  w1 = w1 + 0.00001 .* data(1:2,i) .* delta1';&#xA;  b1 = b1 + 0.00001 .* delta1;&#xA;  b2 = b2 + 0.00001 .* delta2;&#xA;end&#xA;&#xA;data(:,1)=[0; 0; 0];&#xA;data(:,2)=[1; 0; 1];&#xA;data(:,3)=[0; 1; 1];&#xA;data(:,4)=[1; 1; 0];&#xA;&#xA;w1=rand(2,3)./2.*(rand(2,3).*-2+1);&#xA;w2=rand(3,1)./2.*(rand(3,1).*-2+1);&#xA;b1=rand(1,3)./2;&#xA;b2=rand(1,1)./2;&#xA;&#xA;for j=1:20000&#xA;  for i=1:4&#xA;    [cost, mid_layer, last_layer] = forward(w1,w2,b1,b2,data,sigmoid,i);&#xA;    [w1, w2, b1, b2] = backprop(w1,w2,b1,b2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i);&#xA;    cost_mem(j,i)=cost;&#xA;  end&#xA;end&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35660" LastEditorUserId="2444" LastEditDate="2020-04-02T21:27:47.943" LastActivityDate="2020-04-02T21:27:47.943" Title="Is my backpropagation code correct?" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;&lt;backpropagation&gt;&lt;matlab&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="3" ClosedDate="2020-04-02T21:25:20.217" ContentLicense="CC BY-SA 4.0" />
  <row Id="19937" PostTypeId="2" ParentId="19922" CreationDate="2020-04-02T14:19:12.057" Score="1" Body="&lt;p&gt;Batch size and epochs are independent parameters - they serve very different purposes. Your main question as I understand it (and for general, non-library specific consumption) is &lt;strong&gt;what is an epoch and how is the data used for each epoch?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply put, an epoch is a single iteration though the training data. Each and every sample from your training dataset will be used &lt;strong&gt;once&lt;/strong&gt; per epoch, whether it is for training or validation. Therefore, the more epochs, the more the model is trained. The key is to identify the number of epochs that fits the model to the data without overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your explaination of how batch size affects the training process is correct but not relevant to the question since it has no relation to the epoch training iterations. That is not to say that these values should be considered independently since they have similar effects on the model training process.&lt;/p&gt;&#xA;" OwnerUserId="31980" LastActivityDate="2020-04-02T14:19:12.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19938" PostTypeId="1" CreationDate="2020-04-02T14:20:06.177" Score="0" ViewCount="50" Body="&lt;p&gt;I'm doing a Deep Q-learning project. All of my rewards are positive and there are two terminal states. One of them has a zero reward and the other has a high positive reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rewards are stochastic and my Q-network must generate non-zero Q-values for all states and actions. Based on my project, I must use these numbers to create a probability density. In other words, the normalized Q-values of each state generated by network define a probability density for choosing an action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I define my loss function? Is there a project or paper which I could look at and decide how to define the loss function? I am searching for similar projects and their proposed Deep Q-learning algorithms.&lt;/p&gt;&#xA;" OwnerUserId="35633" LastEditorUserId="2444" LastEditDate="2020-04-02T16:05:32.807" LastActivityDate="2020-04-02T16:05:32.807" Title="How should I define the loss function when using DQN to estimate the probability density?" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19940" PostTypeId="1" CreationDate="2020-04-02T17:30:13.320" Score="1" ViewCount="19" Body="&lt;p&gt;I am currently working on a term paper on the topic of Narrative Similarity, based on Loizos Michael's work &quot;Similarity of Narratives&quot;. I am trying to find the latest trends within this field of study for the literature overview in my assignement. However up untill now I wasn't able to find any new work on this particular subject. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any literature recommendation by anyone out there that has worked on this topic or is currently doing so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to Michael's work:&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/316280937_Similarity_of_Narratives&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/316280937_Similarity_of_Narratives&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31914" LastActivityDate="2020-04-02T17:30:13.320" Title="What are the current research trends in recognizing narrative similarity?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19943" PostTypeId="1" CreationDate="2020-04-02T18:49:03.303" Score="2" ViewCount="1309" Body="&lt;p&gt;Soon I will be working on biomedical image segmentation (microscopy images). There will be a small amount of data (a few dozens at best). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a neural network, that can compete with U-Net, in this case? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've spent the last few hours searching through scientific articles that are dealing with this topic, but haven't found a clear answer and I would like to know what other possibilities are. The best answers I found are that I could consider using &lt;a href=&quot;https://arxiv.org/abs/1904.00592&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResU-Net&lt;/a&gt; (R2U-Net), &lt;a href=&quot;https://arxiv.org/abs/1511.00561&quot; rel=&quot;nofollow noreferrer&quot;&gt;SegNet&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1812.00548&quot; rel=&quot;nofollow noreferrer&quot;&gt;X-Net&lt;/a&gt; and backing techniques (&lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1906/1906.10400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas (with evidence, not necessarily)?&lt;/p&gt;&#xA;" OwnerUserId="35696" LastEditorUserId="2444" LastEditDate="2020-06-13T00:00:59.933" LastActivityDate="2021-03-10T00:04:35.377" Title="What are some good alternatives to U-Net for biomedical image segmentation?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19944" PostTypeId="1" CreationDate="2020-04-02T20:39:10.230" Score="4" ViewCount="146" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Rice%27s_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In computability theory, &lt;strong&gt;Rice's theorem&lt;/strong&gt; states that all non-trivial,&#xA;  semantic properties of programs are undecidable. A semantic property&#xA;  is one about the program's behavior (for instance, does the program&#xA;  terminate for all inputs), unlike a syntactic property (for instance,&#xA;  does the program contain an if-then-else statement). A property is&#xA;  non-trivial if it is neither true for every computable function, nor&#xA;  false for every computable function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;syntactic property&lt;/strong&gt; asks a question about a computer program like &quot;is there is a while loop?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;semantic properties&lt;/strong&gt; asks a question about the behavior of the computer program. For example, does the program loop forever (which is the Halting problem, which is undecidable, i.e., in general, there's no algorithm that can tell you if an arbitrarily given program halts or terminates for a given input)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, Rice's theorem proves all non-trivial semantic properties are undecidable (including whether or not the program loops forever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AI is a computer program (or computer programs). These program(s), like all computer programs, can be modeled by a Turing machine (Church-Turing thesis).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is safety (for Turing machines, including AI) a non-trivial semantic question? If so, is AI safety undecidable? In other words, can we determine whether an AI program (or agent) is safe or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that this doesn't require formally defining safety.&lt;/p&gt;&#xA;" OwnerUserId="27283" LastEditorUserId="2444" LastEditDate="2020-04-03T18:34:31.630" LastActivityDate="2021-02-25T04:10:46.137" Title="Does Rice's theorem prove safe AI is undecidable?" Tags="&lt;agi&gt;&lt;proofs&gt;&lt;ai-safety&gt;&lt;theory-of-computation&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19945" PostTypeId="2" ParentId="19936" CreationDate="2020-04-02T21:18:40.533" Score="0" Body="&lt;p&gt;Thanks for everyone's help. I have now solved the problem. I hadn't quite understood the back propagation algorithm. I invite people to take a look at this link: &lt;a href=&quot;http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;backpropagation @ AGH UST&lt;/a&gt; which has solved my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;pkg load image&#xA;graphics_toolkit(&quot;gnuplot&quot;)&#xA;tic&#xA;sigmoid = @(z) 1./(1 + exp(-z));&#xA;sig_der = @(y) sigmoid(y).*(1-sigmoid(y));&#xA;&#xA;&#xA;function [cost, mid_layer, last_layer] = forward(w1,w2,data,sigmoid,i)&#xA;  mid_layer(:,1)=sum(w1.*data(1:2,i));&#xA;  mid_layer(:,2)=sigmoid(mid_layer(:,1));&#xA;  last_layer(:,1)=sum(mid_layer(:,2).*w2);&#xA;  last_layer(:,2)=sigmoid(last_layer(:,1));&#xA;  cost = data(3,i)-last_layer(:,2);&#xA;end&#xA;&#xA;function [w1, w2] = backprop(w1,w2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i)&#xA;  delta(1) = cost;&#xA;  delta(2:4) = cost .* w2;&#xA;  w2 = w2 + 0.99 .* delta(1) .* mid_layer(:,2) .* sig_der(last_layer(:,1));&#xA;  w1 = w1 + 0.99 .* delta(2:4) .* sig_der(mid_layer(:,1))' .* data(1:2,i);&#xA;  %b2 = b2 + 0.01 .* &#xA;  %b1 = b1 + 0.01 .*&#xA;end&#xA;&#xA;tic&#xA;&#xA;&#xA;data(:,1)=[0; 0; 0];&#xA;data(:,2)=[1; 0; 1];&#xA;data(:,3)=[0; 1; 1];&#xA;data(:,4)=[1; 1; 0];&#xA;&#xA;w1=rand(2,3)./2.*(rand(2,3).*-2+1);&#xA;w2=rand(3,1)./2.*(rand(3,1).*-2+1);&#xA;%b1=ones(1,3);%rand(1,3)./2;&#xA;%b2=ones(1,1);%rand(1,1)./2;&#xA;&#xA;for j=1:10000&#xA;  for i=[randperm(4)]&#xA;    [cost, mid_layer, last_layer] = forward(w1,w2,data,sigmoid,i);&#xA;    [w1, w2] = backprop(w1,w2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i);&#xA;    cost_mem(j,i)=cost;&#xA;  end&#xA;end&#xA;&#xA;toc&#xA;&#xA;%{&#xA;dlmwrite(&quot;/tmp/weights_1&quot;,w1);&#xA;dlmwrite(&quot;/tmp/weights_2&quot;,w2);&#xA;%}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35660" LastEditorUserId="35660" LastEditDate="2020-04-02T21:26:10.170" LastActivityDate="2020-04-02T21:26:10.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19946" PostTypeId="2" ParentId="19927" CreationDate="2020-04-02T21:42:43.647" Score="0" Body="&lt;p&gt;I am not an expert on the topic, but I will provide some information that could be useful or helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the first and maybe trivial assumption that people make when they say that AGI or SI can emerge is that general intelligence (whatever the definition is) is computable, i.e. there's a Turing machine or, in general, a mathematical model of computation that can simulate an algorithm that possibly represents general intelligence. I don't know how likely the mind of a human (or any other general intelligence) is computable (or to what extent), but there's the so-called &lt;a href=&quot;https://plato.stanford.edu/entries/computational-mind/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;computational theory of mind&lt;/em&gt; (CTM)&lt;/a&gt; that goes into this direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An assumption related to the emergence of SI is that recursive self-improvement is physically possible. I also don't know exactly how likely or unlikely recursive self-improvement is, but my limited knowledge of thermodynamics (and physics) suggests that it will not be possible (at least, at the pace or the way people claim or want to suggest). I am not saying that we will not see more progress in the next years (we will), but this doesn't mean that we will be able to create an AGI. People often overestimate themselves and their intelligence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third assumption is related to the current achievements in the artificial intelligence field. Many people claim that there has been a lot of progress in recent years and that there's no reason to believe that this will not continue. However, history tells us that the predictions of AI scientists about the future of the AI field were often wrong. There have been at least 2 AI winters because of these wrong predictions and unmet expectations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many people also don't exclude the possibility of the emergence of AGI or SI only because they have not yet been proven wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another assumption is that there won't be any catastrophe (like COVID-19  can potentially be) that will significantly slow down the general scientific progress.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T00:45:53.000" LastActivityDate="2020-04-03T00:45:53.000" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19947" PostTypeId="1" CreationDate="2020-04-03T00:56:44.880" Score="1" ViewCount="80" Body="&lt;p&gt;I am thinking about developing a GAN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between using dense layers as opposed to convolutional layers in my networks when dealing with images?&lt;/p&gt;&#xA;" OwnerUserId="35700" LastEditorUserId="2444" LastEditDate="2020-04-07T01:25:45.407" LastActivityDate="2020-04-07T01:25:45.407" Title="What is the difference between using dense layers as opposed to convolutional layers in my networks when dealing with images?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19948" PostTypeId="2" ParentId="13432" CreationDate="2020-04-03T00:59:06.010" Score="0" Body="&lt;p&gt;Although this does not strictly answer your question (but it is at least very related), Jürgen Schmidhuber has some interesting ideas about compression and how it relates to artificial intelligence, prediction, curiosity, etc. For example, have a look at this paper &lt;a href=&quot;http://people.idsia.ch/~juergen/sice2009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simple Algorithmic Theory of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes&lt;/a&gt;, whose abstract states&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this summary of previous work, I argue that data becomes temporarily interesting by itself to some self-improving, but computationally limited, subjective observer once he learns to predict or compress the data in a better way, thus making it subjectively more &quot;beautiful&quot;. Curiosity is the desire to create or discover more non-random, non-arbitrary, &quot;truly novel&quot;, regular data that allows for compression progress because its regularity was not yet known. This drive maximizes &quot;interestingness&quot;, the first derivative of subjective beauty&#xA;  or compressibility, that is, the steepness of the learning curve. It motivates exploring infants, pure mathematicians, composers, artists, dancers, comedians, yourself, and recent artificial systems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See also his interesting talk &lt;a href=&quot;https://www.youtube.com/watch?v=h7F5sCLIbKQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Juergen Schmidhuber: The Algorithmic Principle Beyond Curiosity and Creativity&lt;/a&gt; on YouTube.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-03T00:59:06.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19950" PostTypeId="1" CreationDate="2020-04-03T06:09:17.760" Score="1" ViewCount="24" Body="&lt;p&gt;I am interested in doing model-free RL but not using the Markov assumptions typical for MDPs or POMDPs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are alternative paradigms that don't rely on the Markov assumptions? Are there any common approaches when this assumption is violated? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I am asking for mathematical models that do not make the Markov assumption and so could be used for problems where the Markov assumption does not hold&lt;/p&gt;&#xA;" OwnerUserId="25337" LastEditorUserId="25337" LastEditDate="2020-04-03T14:46:45.207" LastActivityDate="2020-04-03T14:46:45.207" Title="What are the most common non-Markov RL paradigms?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;markov-property&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19952" PostTypeId="1" CreationDate="2020-04-03T10:09:30.277" Score="1" ViewCount="52" Body="&lt;p&gt;Given a discrete, finite Markov Decision Process (MDP) with its usual parameters &lt;span class=&quot;math-container&quot;&gt;$(S, A, T, R, \gamma)$&lt;/span&gt;, it is possible to obtain the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{*}$&lt;/span&gt; and the optimal value function &lt;span class=&quot;math-container&quot;&gt;$V^{*}$&lt;/span&gt; through one of many planning methods (policy iteration, value iteration or solving a linear program).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in obtaining a random near-optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, with the value function associated with the policy given by &lt;span class=&quot;math-container&quot;&gt;$V^{\pi}$&lt;/span&gt;, such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \epsilon_1 &amp;lt; ||V^{*} - V^{\pi}||_{\infty} &amp;lt; \epsilon_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to know an efficient way of achieving this goal. A possible approach is to generate random policies and then to use the given MDP model to evaluate these policies and verify that they satisfy the criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If only an upper bound were needed, the idea that near optimal value functions induce near optimal policies could be used, that is, we can show that, if&#xA;&lt;span class=&quot;math-container&quot;&gt;$$||V - V^{*}||_{\infty} &amp;lt; \epsilon, \quad \epsilon &amp;gt; 0$$&lt;/span&gt;&#xA;and if &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the policy that is greedy with respect to the value function &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ ||V^{\pi} - V^{*}||_{\infty} &amp;lt; \frac{2\gamma\epsilon}{1 - \gamma}$$&lt;/span&gt;&#xA;So by picking a suitable &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; for the given &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, we can be sure of any upper bound &lt;span class=&quot;math-container&quot;&gt;$\epsilon_2$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would also like that the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; not be &quot;too good&quot;, hence the requirement for a lower bound. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any inputs regarding an efficient solution or reasons for the lack thereof are welcome.&lt;/p&gt;&#xA;" OwnerUserId="28384" LastActivityDate="2020-04-05T10:17:00.480" Title="Efficient algorithm to obtain near optimal policies for an MDP" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19953" PostTypeId="1" CreationDate="2020-04-03T10:38:37.350" Score="2" ViewCount="33" Body="&lt;p&gt;I wanna implement an algorithm for &lt;strong&gt;Facial Recognition&lt;/strong&gt; in &lt;em&gt;C++&lt;/em&gt; with the help of &lt;strong&gt;Viola-Jones&lt;/strong&gt;, aka Adaboost, but &lt;em&gt;without&lt;/em&gt; using &lt;em&gt;OpenCV&lt;/em&gt; or any other similar library. I wanna do it all &lt;em&gt;from scratch&lt;/em&gt;. Any tips?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everybody gives me ambiguous answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.S.&lt;/em&gt;: I started the whole thing by creating a RGB to grayscale Bitmap conversion.&lt;/p&gt;&#xA;" OwnerUserId="35706" LastActivityDate="2020-04-03T10:38:37.350" Title="How can I implement a Facial Recognition algorithm in C++ from scratch, without using OpenCV?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;classification&gt;&lt;training&gt;&lt;ai-basics&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19954" PostTypeId="2" ParentId="18514" CreationDate="2020-04-03T11:31:39.277" Score="2" Body="&lt;p&gt;I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see a little thing that might be making your model a little worse, though:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You're concatenating &quot;relu&quot; with &quot;sigmoid&quot;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Placing two different nature values in the same array may make it more difficult for updating weights properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few independent suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Make the output of the images model have &quot;relu&quot; activation as well before the concatenation. Preferrably, use a batch normalization before the image relu and before the number relu (this way you concatenate values that are in very similar ranges).    &lt;/li&gt;&#xA;&lt;li&gt;Instead of concatenating, you can try &lt;code&gt;Multiply()([img_output, numeric_output])&lt;/code&gt;, in this case, both outputs must have the same size, one of them uses &quot;relu&quot; or &quot;linear&quot;, and the other uses &quot;sigmoid&quot;.   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, something important when using AUC: you need big batch sizes, because AUC is dependent on the whole data, it's not like usual metrics/losses that you can take the mean from the results of each batch. &lt;/p&gt;&#xA;" OwnerUserId="35709" LastActivityDate="2020-04-03T11:31:39.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19955" PostTypeId="1" CreationDate="2020-04-03T11:46:03.967" Score="2" ViewCount="56" Body="&lt;p&gt;I've written a Monte Carlo Tree Search player for the game of &lt;a href=&quot;https://en.wikipedia.org/wiki/Castle_(card_game)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Castle&lt;/a&gt; (AKA Shithead, Shed, Palace...). I have set this MCTS player to play against a basic rule-based AI for ~30000 games and collected ~1.5 million game states (as of now) along with whether the MCTS player won that particular game in the end after being in that particular game state. The game has a large chance aspect, and, currently, the MCTS player is winning ~55% of games. I want to see how high I can push it. In order to do this, I aim to produce a NN that will act as a game state evaluation function to use within the MCTS.&lt;/p&gt;&#xA;&lt;p&gt;With this information, I've already tried an SVM, but came to the conclusion that the game space is too large for the SVM to classify a given state accurately.&lt;/p&gt;&#xA;&lt;p&gt;I hope to be able to train a NN to evaluate a given state and return how good that state is for the MCTS player. Either with a binary GOOD/BAD or I think it would be more helpful to return a value between 0-1.&lt;/p&gt;&#xA;&lt;p&gt;The input to the NN is a &lt;span class=&quot;math-container&quot;&gt;$4 \times 41$&lt;/span&gt; NumPy array of binary values (0, 1) representing the MCTS players hand, MCTS face-up cards, OP face-up cards, MCTS no. face-down cards, OP no. face-down cards. Shown below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6pDH0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6pDH0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Describes the &lt;code&gt;np.array&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/byFUt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/byFUt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;np.array&lt;/code&gt; is made from the database entries of game states. An example of this information is below. However, I am currently omitting the TOP &amp;amp; DECK_EMPTY columns in this model. WON (0, 1) is used as the label.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iym6J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iym6J.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is my keras code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.models.Sequential()&#xA;model.add(tf.keras.layers.Flatten())&#xA;model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))&#xA;model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))&#xA;model.add(tf.keras.layers.Dense(2, activation=tf.nn.softmax))&#xA;&#xA;model.compile(optimizer='adam',&#xA;             loss='sparse_categorical_crossentropy',&#xA;             metrics=['accuracy'])&#xA;&#xA;model.fit(X_train, y_train, epochs=3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This model isn't performing.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you think it is possible to obtain a useful NN with my current approach?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What layers should I look to add to the NN?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can you recommend any training/learning material that I could use to try and get a better understanding?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33966" LastEditorUserId="2444" LastEditDate="2021-02-06T21:59:47.333" LastActivityDate="2021-02-06T21:59:47.333" Title="Is this a good approach to evaluate the game state with a neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;&lt;evaluation-functions&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19956" PostTypeId="1" AcceptedAnswerId="19968" CreationDate="2020-04-03T11:48:07.433" Score="2" ViewCount="337" Body="&lt;p&gt;In an episodic training of an RL agent, should I always start from the same initial state or I can start from several valid initial states?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in a gym environment, should my &lt;code&gt;env.reset()&lt;/code&gt; function always resets me to the same start state or it can start from different states at each training episode?&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-04-03T12:13:15.987" LastActivityDate="2020-04-04T07:36:49.730" Title="Should I always start from the same start state in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;gym&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19957" PostTypeId="1" CreationDate="2020-04-03T12:01:55.227" Score="2" ViewCount="50" Body="&lt;p&gt;I am wondering about the correlation between the loss and the derivative of the loss wrt a single scalar parameter, with the same sample. That means: considering a machine learning model with parameters &lt;span class=&quot;math-container&quot;&gt;$\theta \in R$&lt;/span&gt;, I want to figure out the relationship between &lt;span class=&quot;math-container&quot;&gt;$Loss(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial Loss(x)}{\partial \theta_i}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$i \in \{1,2,3,...,n\}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I would like to consider that they are in a positive correlation, is it right? If it is right, how can I prove it in a mathematical way? &lt;/p&gt;&#xA;" OwnerUserId="35710" LastEditorUserId="9947" LastEditDate="2020-04-07T01:38:32.497" LastActivityDate="2020-04-07T01:38:32.497" Title="Is the derivative of the loss wrt a single scalar parameter proportional to the loss?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19958" PostTypeId="2" ParentId="19943" CreationDate="2020-04-03T12:14:51.117" Score="0" Body="&lt;p&gt;Hey i am working on my Bachelor thesis at the moment and use UNET in combination with a GAN for image segmentation. I spend the last 5 months on that, so on my tests, the new approach of januar 2020, called &lt;a href=&quot;https://arxiv.org/pdf/1902.04049&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multires-UNET&lt;/a&gt; is quite a good choice for more texture orientated segmentation. I use the current &lt;a href=&quot;https://github.com/nibtehaz/MultiResUNet/blob/master/MultiResUNet.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt; implementation. Its quite nice, maybe you notice that you can easyly tweak the number of parameters with &quot;alpha&quot; in the implementation, to scale the multiple Resnets in the Unet structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried also other segmentation networks like Mask_RCNN with different backbones or tried to construct various types of CAE on my own, but always had to come back to a UNET like structure. Same goes for ResU-NET and R2U, the multires one worked better for my purposes cause i didnt need any kind of LSTM modules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples which may clarify the difference in performance on a specific task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Original Image:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Wk9Ht.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wk9Ht.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ground-Truth:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rFjI4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rFjI4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classical UNET++ (Unet with skip-connections)(2.5 million parameters) more parameters (wide) didnt change the result.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/SrCQP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SrCQP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multires-UNET (alpha=1.67, think was about 7 million parameters)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bvuiA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bvuiA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you show some of ur microscopy images, how complex is ur task and what do you want to segment ?&lt;/p&gt;&#xA;" OwnerUserId="35557" LastEditorUserId="35557" LastEditDate="2020-04-04T13:27:12.017" LastActivityDate="2020-04-04T13:27:12.017" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19959" PostTypeId="1" AcceptedAnswerId="19969" CreationDate="2020-04-03T13:05:29.380" Score="2" ViewCount="56" Body="&lt;p&gt;Apart from &lt;a href=&quot;https://en.wikipedia.org/wiki/ICub&quot; rel=&quot;nofollow noreferrer&quot;&gt;iCub&lt;/a&gt;, whose open-source software can be found at &lt;a href=&quot;https://github.com/robotology/icub-main&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/robotology/icub-main&lt;/a&gt;, what other humanoid robots have open-source and freely accessible software (or hardware)?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-23T22:46:28.973" LastActivityDate="2020-05-23T22:46:28.973" Title="Which humanoid robots have open-source software (or hardware)?" Tags="&lt;reference-request&gt;&lt;robotics&gt;&lt;robots&gt;&lt;humanoid-robots&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19960" PostTypeId="5" CreationDate="2020-04-03T13:07:56.067" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T13:07:56.067" LastActivityDate="2020-04-03T13:07:56.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19961" PostTypeId="4" CreationDate="2020-04-03T13:07:56.067" Score="0" Body="For questions specifically related to humanoid robots, as opposed to any other type of robot." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T13:07:56.067" LastActivityDate="2020-04-03T13:07:56.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19962" PostTypeId="2" ParentId="18820" CreationDate="2020-04-03T13:10:42.477" Score="3" Body="&lt;p&gt;All right, I figured it out. trajectories need not have the same starting state because the distribution of &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; is drawn from a distribution D (mentioned in the paper). Had been confused because many of the code implementations on github focus on trajectories starting from the same state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps everyone !&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-04-03T13:54:36.873" LastActivityDate="2020-04-03T13:54:36.873" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19963" PostTypeId="1" AcceptedAnswerId="19966" CreationDate="2020-04-03T15:01:23.627" Score="2" ViewCount="52" Body="&lt;p&gt;I have created a neural network that is able to recognize images with the numbers 1-5. The issue is that I have a database of 16x5 images which ,unfortunately, is not proving enough as the neural network fails in the test set. Are there ways to improve a neural network's performance without using more data? The ANN has approximately a 90% accuracy on the training sets and a 50% accuracy in the test ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;graphics_toolkit(&quot;gnuplot&quot;)&#xA;sigmoid = @(z) 1./(1 + exp(-z));&#xA;sig_der = @(y) sigmoid(y).*(1-sigmoid(y));&#xA;&#xA;&#xA;parse_image;   % This external f(x) loads the images so that they can be read. &#xA;%13x14&#xA;num=0;&#xA;for i=1:166&#xA;  if mod(i-1,10)&amp;lt;=5 &amp;amp;&amp;amp; mod(i-1,10) &amp;gt; 0&#xA;    num=num+1;&#xA;    data(:,num) = dlmread(strcat(&quot;/tmp/&quot;,num2str(i)))(:);&#xA;  end&#xA;end&#xA;&#xA;&#xA;&#xA;function [cost, mid_layer, last_layer] = forward(w1,w2,data,sigmoid,i)&#xA;  mid_layer(:,1)=sum(w1.*data(:,i));&#xA;  mid_layer(:,2)=sigmoid(mid_layer(:,1));&#xA;  last_layer(:,1)=sum(mid_layer(:,2).*w2);&#xA;  last_layer(:,2)=sigmoid(last_layer(:,1));&#xA;  exp_res=rem(i,5);&#xA;  if exp_res==0&#xA;    exp_res=5;&#xA;  end&#xA;  exp_result=zeros(5,1); exp_result(exp_res)=1;&#xA;  cost = exp_result-last_layer(:,2);&#xA;end&#xA;&#xA;function [w1, w2] = backprop(w1,w2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i)&#xA;  delta(1:5) = cost;&#xA;  delta(6:20) = sum(cost' .* w2,2);&#xA;  w2 = w2 + 0.05 .* delta(1:5) .* mid_layer(:,2) .* sig_der(last_layer(:,1))';&#xA;  w1 = w1 + 0.05 .* delta(6:20) .* sig_der(mid_layer(:,1))' .* data(:,i);&#xA;end&#xA;&#xA;w1=rand(182,15)./2.*(rand(182,15).*-2+1);&#xA;w2=rand(15,5)./2.*(rand(15,5).*-2+1);&#xA;&#xA;for j=1:10000&#xA;  for i=[randperm(85)]&#xA;    [cost, mid_layer, last_layer] = forward(w1,w2,data,sigmoid,i);&#xA;    [w1, w2] = backprop(w1,w2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i);&#xA;    cost_mem(j,i,:)=cost;&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35660" LastActivityDate="2020-04-03T15:46:34.263" Title="How can I train a neural network if I don't have enough data?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;matlab&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19964" PostTypeId="1" CreationDate="2020-04-03T15:18:42.977" Score="1" ViewCount="40" Body="&lt;p&gt;Convex optimisation is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bQv8T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bQv8T.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen a lot of talk about convex loss functions in Neural Networks and how we are optimising rewards or penalty in AI/ML systems. But I have never seen any loss function formulated in the aforementioned way. So my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any role of convex optimization in AI? If so, in what algorithms or problem settings or systems?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-04-03T15:49:42.400" LastActivityDate="2020-04-03T15:54:44.137" Title="What is the role of convex optimisation in AI systems?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19965" PostTypeId="2" ParentId="19963" CreationDate="2020-04-03T15:44:24.700" Score="1" Body="&lt;p&gt;&lt;strong&gt;In theory&lt;/strong&gt;, yes, using synthetic data generation. This involves applying transformations to the original images to generate new 'unique' images. Some standard techniques include rotating, flipping, stretching, zooming or brightening. Obviously not all of these make sense depending on the data. In your problem, zooming, stretching and brightening could be used but flipping should not. Rotation could work but only for small angles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally this is implemented by &lt;em&gt;replacing&lt;/em&gt; the dataset for each epoch of training. Therefore, the number of images used in each training iteration is the same but the images themselves have been altered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In practice&lt;/strong&gt;, it's not a magic bullet. The reason a larger dataset generally yields better models is because the probability of a new feature falling within the feature distribution of the training data is higher. With synthetic data generation the new features are only marginally different to the original so even if the number of images to train on is increased, the feature distributions are not that different. There is a lot of variation in handwritten numbers so it would be very hard to guess how effective this would be without trying it.&lt;/p&gt;&#xA;" OwnerUserId="31980" LastActivityDate="2020-04-03T15:44:24.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19966" PostTypeId="2" ParentId="19963" CreationDate="2020-04-03T15:46:34.263" Score="3" Body="&lt;p&gt;You can synthetically increase the number of samples. For example with augmentation or unsupervised adaption (&lt;a href=&quot;https://arxiv.org/abs/1911.04252&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-training&lt;/a&gt;). With augmentation you grant the system way more robustness so i would really recommend this. For example this &lt;a href=&quot;https://github.com/aleju/imgaug&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;. The problem with such small database sizes is that your test-set is also very small and you cannot test properly if your network generalizes well, or just overfits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try transfer learning with another larger network to adapt those feature extractors and use them on ur problem. That may work better than training a new one from scratch with so less labeled images. Hope i could help at least a little, stay tuned.&lt;/p&gt;&#xA;" OwnerUserId="35557" LastActivityDate="2020-04-03T15:46:34.263" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19967" PostTypeId="2" ParentId="19964" CreationDate="2020-04-03T15:48:58.667" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any role of convex optimization in AI? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, of course!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, in what algorithms or problem settings or systems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The problem of finding the parameters of a &lt;a href=&quot;https://qr.ae/pNvzNm&quot; rel=&quot;nofollow noreferrer&quot;&gt;support vector machine&lt;/a&gt; can be formulated as a convex optimization problem. Another example is &lt;a href=&quot;https://stats.stackexchange.com/a/194461/82135&quot;&gt;linear regression&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the paper &lt;a href=&quot;https://arxiv.org/pdf/1405.4980.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convex Optimization: Algorithms and Complexity&lt;/a&gt; (2014) by Sébastien Bubeck, which also mentions SVM as a typical example.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T15:54:44.137" LastActivityDate="2020-04-03T15:54:44.137" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19968" PostTypeId="2" ParentId="19956" CreationDate="2020-04-03T15:51:57.247" Score="2" Body="&lt;p&gt;It depends on the task the agent is trying to learn and of course on the environment constrains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an Atari game agents have a pre-fixed starting point because that's part of the games rules, so I would say that this is enough of a justification to make each simulation start from that starting point. Moreover, you have to pay attention to the kind of reward function you're using, for example (a really dumb one just to give a grasp of the concept) if you're rewarding the agent depending on how close it gets to the ending point rather than how far it goes from the starting point, the agent might end up having huge rewards only because it respawned close to end point, which would be an artefact and not a fair reward for a good action choice. &#xA;Other situations in which it does not really make sense to select a random starting point might be dialogue systems, in which you know that a conversation start with greetings, therefore it would not be logical to make an agent starting with a random question or sentence (the space in these case would be made of different dialogue acts). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are anyway situations in which the environment allows a random selection of the start location of the agent. In this &lt;a href=&quot;https://arxiv.org/pdf/1611.03673v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; for example, an agent was trained to escape from a maze, and both, the exit point and the initial location of the agent where randomly selected at each iteration. This was partially due to research reasons, the authors were analysing the possibility to train policies more complex than a simple 'space memorisation' one but anyway, conceptually there's nothing wrong in this situation in selecting a random starting point. Other tasks could be following or escaping from a specific object, again, as long as the reward function is properly designed, even with a random starting location at the end the agent would anyway learn to move faster when close to the target (toward or in the opposite direction depending on the task). Actually, in this situation I think that a random initial location would have potential benefits, like avoiding the agents to learn biases due to external biases of the environment (like parts of space with less random obstacles than others).&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-04-03T15:51:57.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19969" PostTypeId="2" ParentId="19959" CreationDate="2020-04-03T15:58:13.980" Score="2" Body="&lt;p&gt;There are quite a few &lt;a href=&quot;https://www.ros.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ROS&lt;/a&gt; (Robot Operating System) based &lt;a href=&quot;https://robots.ros.org/tags/#humanoid&quot; rel=&quot;nofollow noreferrer&quot;&gt;humanoid robots&lt;/a&gt;. Not all of them will be open-source down to the firmware level but most will at least be to the control level. I'm not aware of any that have open-source hardware as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pal-robotics.com/robots/talos/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Talos&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pal-robotics.com/robots/reem-c/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reem-C&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.softbankrobotics.com/emea/en/nao&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nao&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.robotis.com/model/page.php?co_id=prd_op3&quot; rel=&quot;nofollow noreferrer&quot;&gt;OP3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.robotis.com/model/page.php?co_id=prd_thormang3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thormang3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31980" LastEditorUserId="31980" LastEditDate="2020-04-03T18:51:46.353" LastActivityDate="2020-04-03T18:51:46.353" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19970" PostTypeId="1" CreationDate="2020-04-03T16:09:34.237" Score="1" ViewCount="29" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; be the state-space in a reinforcement learning problem where rewards are in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;, and let &lt;span class=&quot;math-container&quot;&gt;$V:\mathcal{S} \to \mathbb{R}$&lt;/span&gt; be an approximate value function. Following &lt;a href=&quot;https://arxiv.org/abs/1506.02438&quot; rel=&quot;nofollow noreferrer&quot;&gt;the GAE paper&lt;/a&gt;, the TD-residual with discount &lt;span class=&quot;math-container&quot;&gt;$\gamma \in [0,1]$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$\delta_t^V = r_t + \gamma V(s_{t + 1}) - V(s_t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused by the formula for the GAE-&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; advantage estimator, which is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\hat{A}_t^{\text{GAE}(\gamma, \lambda)} = \sum_{l = 0}^\infty (\gamma \lambda)^l \delta_{t + l}^V.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to imply that &lt;span class=&quot;math-container&quot;&gt;$\delta_t^V$&lt;/span&gt; is defined for &lt;span class=&quot;math-container&quot;&gt;$t &amp;gt; N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the length of the current trajectory/episode. It looks like in implementations of this advantage estimator, it is just assumed that &lt;span class=&quot;math-container&quot;&gt;$\delta_t^V = 0$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$t &amp;gt; N$&lt;/span&gt;, since the sums are finite. Is there a justification for this assumption? Or am I missing something here? &lt;/p&gt;&#xA;" OwnerUserId="33762" LastEditorUserId="2444" LastEditDate="2020-06-04T17:12:46.847" LastActivityDate="2020-06-04T17:12:46.847" Title="Is the TD-residual defined for timesteps $t$ past the length of the episode?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;papers&gt;&lt;temporal-difference-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19971" PostTypeId="1" CreationDate="2020-04-03T16:19:22.720" Score="1" ViewCount="33" Body="&lt;p&gt;I am wondering how can I find the appropriate reward value for each specific problem. I know this is a highly empirical process, but I am sure that the value is not set totally at random. I want to know what are the general guidelines and practices to find the appropriate reward value for any reinforcement learning problem.&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-04-03T18:10:25.593" LastActivityDate="2020-04-03T18:10:25.593" Title="How can I find the appropriate reward value for my reinforcement learning problem?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19972" PostTypeId="5" CreationDate="2020-04-03T18:37:28.520" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T18:37:28.520" LastActivityDate="2020-04-03T18:37:28.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19973" PostTypeId="4" CreationDate="2020-04-03T18:37:28.520" Score="0" Body="For questions related to the theory of computation in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T18:37:28.520" LastActivityDate="2020-04-03T18:37:28.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19974" PostTypeId="1" AcceptedAnswerId="19975" CreationDate="2020-04-03T18:43:07.627" Score="2" ViewCount="58" Body="&lt;p&gt;In chapter 3.5 of Sutton's book, the value function is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eNqIS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eNqIS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone give me some clarification about why there is the expectation sign behind the entire equation? Considering that the agent is following a fixed policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, why there should be an expectation when the trajectory of the future possible states is fixed (or maybe I am getting it wrong and it's not). In total, if the expectation here has the meaning of averaging over a series of trajectories, what are those trajectories and what are the weights of them when we want to compute the expected value over them according to &lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia definition of the expected values&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="35719" LastEditorUserId="2444" LastEditDate="2020-04-03T19:42:46.740" LastActivityDate="2020-04-04T04:34:14.487" Title="Why is there an expectation sign in the Bellman equation?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;expectation&gt;&lt;bellman-equations&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19975" PostTypeId="2" ParentId="19974" CreationDate="2020-04-03T19:19:47.907" Score="4" Body="&lt;p&gt;There needs to be an &lt;span class=&quot;math-container&quot;&gt;$E_{\pi}$&lt;/span&gt; over the infinite discounted return term because of two reasons-&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The policy could be stochastic in nature. That is, for any given state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the policy &lt;span class=&quot;math-container&quot;&gt;$\pi(s_t)$&lt;/span&gt; does not provide a deterministic action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, but rather, it provides us with a distribution over the possible next states, that is the action at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; is distributed as-&#xA;&lt;span class=&quot;math-container&quot;&gt;$$a_t \sim \pi(s_t)$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Even if the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; being followed by an agent is deterministic, there still needs to be an expectation over the behavior of the underlying stochastic MDP environment. That is, any action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;, in general, only provides us with a distribution over the possible next states of the agent. That is, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(s_{t + 1} = s') = P_{\pi}(s' | s_t) = \sum_{a \in A} T(s,a,s') \times P_{\pi}(a_t = a)$$&lt;/span&gt; &#xA;Here &lt;span class=&quot;math-container&quot;&gt;$T(s, a, s')$&lt;/span&gt; is the transition function for the MDP and the above equation captures the stochasticity arising from both 1 and 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As you see the expectation does not have to do with averaging over a collection of trajectories. However, that idea is often used in &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte-Carlo&lt;/a&gt; estimation of value functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;As pointed out in the comments, it is not correct to say that the expectation is not over a collection of trajectories.&lt;/p&gt;&#xA;" OwnerUserId="28384" LastEditorUserId="28384" LastEditDate="2020-04-04T04:34:14.487" LastActivityDate="2020-04-04T04:34:14.487" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19976" PostTypeId="5" CreationDate="2020-04-03T19:44:22.553" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T19:44:22.553" LastActivityDate="2020-04-03T19:44:22.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19977" PostTypeId="4" CreationDate="2020-04-03T19:44:22.553" Score="0" Body="For questions related to the Bellman equations in the context of reinforcement learning (and other artificial intelligence subfields)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T19:44:22.553" LastActivityDate="2020-04-03T19:44:22.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19978" PostTypeId="1" CreationDate="2020-04-03T19:45:03.013" Score="0" ViewCount="48" Body="&lt;p&gt;I'm an intermediate machine learning student and want to get more detailed and specific &lt;strong&gt;practical intuition&lt;/strong&gt; about &lt;em&gt;artificial intelligence&lt;/em&gt;. I have made a couple of searches over the well-observed learning sites like &lt;a href=&quot;https://stackoverflow.com/&quot;&gt;Stack Overflow&lt;/a&gt;, &lt;a href=&quot;https://www.redditinc.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;, &lt;a href=&quot;https://www.quora.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quora&lt;/a&gt;, and &lt;a href=&quot;https://medium.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Medium&lt;/a&gt;. But there is a lot of stuff about &lt;em&gt;machine learning&lt;/em&gt; and &lt;em&gt;artificial intelligence&lt;/em&gt; on the internet, and it seems like I am beating around the bush over the Internet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently, I have done my intuition with the most recommended course in these research communities &lt;a href=&quot;https://www.coursera.org/learn/machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning by Andrew-NG&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Where can I get relevant &lt;em&gt;exercise materials&lt;/em&gt; (such as assignments or&#xA;  problems) of commonly used &lt;em&gt;Machine Learning Algorithms&lt;/em&gt; (i.e. &lt;code&gt;Linear Regression, Logistic Regression, Decision Tree, SVM, kNN,&lt;/code&gt; etc)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So that I can implement them from scratch and program machine learning models accordingly.&lt;/p&gt;&#xA;" OwnerUserId="35697" LastEditorUserId="35697" LastEditDate="2020-04-04T19:24:24.507" LastActivityDate="2020-04-04T19:24:24.507" Title="Where can I get exercises and problems to implement machine learning models and algorithms?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;research&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19979" PostTypeId="1" CreationDate="2020-04-03T19:46:53.760" Score="1" ViewCount="61" Body="&lt;p&gt;Why can an AI, like &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaStar_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaStar&lt;/a&gt;, work in StarCraft, although the environment is only partially observable? As far as I know, there are no theoretical results on RL in the POMDP environment, but it appears the core RL techniques are being used in partially observable domains. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-04-03T20:22:32.100" LastActivityDate="2020-04-03T20:22:32.100" Title="Why can the core reinforcement learning algorithms be applied to POMDPs?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19980" PostTypeId="2" ParentId="19974" CreationDate="2020-04-03T20:03:12.720" Score="2" Body="&lt;p&gt;In addition to &lt;a href=&quot;https://ai.stackexchange.com/a/19975/2444&quot;&gt;this answer&lt;/a&gt;, I would like to note that, if the future trajectories were fixed (i.e. the environment and the policies were deterministic, and the agent always starts from the same state), the expectation of the sum (of the fixed rewards) would simply correspond to the actual sum, because the sum is a constant (i.e. the expectation of a constant is the constant itself), so the expectation operator also applies to the deterministic cases. Therefore, the expectation is a general way of expressing the value of a state in all possible cases (both when trajectories are fixed or not).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-03T20:11:30.470" LastActivityDate="2020-04-03T20:11:30.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19981" PostTypeId="1" CreationDate="2020-04-03T20:52:18.113" Score="2" ViewCount="27" Body="&lt;p&gt;I'm creating an RL application for the game Connect Four. I've researched the different strategies for the game and which positions are more favourable to lead to a win. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I be assigning greater rewards when the application places a token in those particular positions? If so, that would mean the application/algorithm is a Connect Four specific application, and not generic? &lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="27629" LastEditDate="2020-04-03T21:01:18.183" LastActivityDate="2020-04-03T21:01:18.183" Title="How should I define the reward function for the Connect Four game?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19982" PostTypeId="1" CreationDate="2020-04-03T21:03:03.900" Score="1" ViewCount="48" Body="&lt;p&gt;I'm creating an RL application for the game Connect Four. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, should I be aiming to create an application that's more generic, which would 'learn' different games, or specific to a particular game (e.g. Connect Four, by assigning greater rewards to certain token positions in the C4 grid)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the difference between the two approaches just come down to adapting their respective reward functions to reward specific achievements or positions (in a board game setting), or something else?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-04-04T01:44:03.493" LastActivityDate="2020-04-04T18:11:02.480" Title="Should I be trying to create a generic or specific (to particular game) reinforcement learning agent?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19983" PostTypeId="1" CreationDate="2020-04-03T21:12:56.607" Score="0" ViewCount="77" Body="&lt;p&gt;Does this prove AI Safety is undecidable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output meaning output to computer program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[A1] Assume we have a program that decides which outputs are “safe”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[A2] Assume we have an example of an unsafe output: “unsafe_output”&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[A3] Assume we have an example of safe output: “safe_output”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[A4] Define a program to be safe if it always produces safe output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[A5] Assume we have a second program (&lt;code&gt;safety_program&lt;/code&gt;) that decides which programs are safe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[A6] Write the following program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def h()&#xA;   h_is_safe := safety_program(h)&#xA;   if (h_is_safe):&#xA;      print unsafe_output&#xA;   else:&#xA;      print safe_output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Clearly h halts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the safety_program said h was safe, then h prints out unsafe_output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the safety_program said h was not safe, then h prints out safe_output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore safety_program doesn’t decide h correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a contradiction. Therefore we made a wrong assumption: Either safe output cannot be decided, or safe programs cannot be decided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, in general, the safety of computer programs, including Artificial Intelligence, is undecidable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore AI Safety is undecidable.&lt;/p&gt;&#xA;" OwnerUserId="27283" LastEditorUserId="27283" LastEditDate="2020-04-04T03:12:05.783" LastActivityDate="2020-05-04T05:03:20.477" Title="Does this prove AI Safety is undecidable?" Tags="&lt;agi&gt;&lt;ai-safety&gt;" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="19984" PostTypeId="2" ParentId="19983" CreationDate="2020-04-03T21:50:26.300" Score="1" Body="&lt;p&gt;In my opinion, there are several flaws in your proof and reasonings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, note that, in the case of Turing's proof, &lt;code&gt;h&lt;/code&gt; will actually loop forever (i.e. not halt) when the oracle says that &lt;code&gt;h&lt;/code&gt; halts. In this case, there's an actual contradiction, because &lt;code&gt;h&lt;/code&gt; will do the opposite of what the oracle says. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to follow Turing's proof, you would need to make &lt;code&gt;h&lt;/code&gt; behave unsafely if the oracle says &lt;code&gt;h&lt;/code&gt; is safe. But how should we define a safe or unsafe program? There are many unsafe behaviors. For example, in a certain context, an insult could be unsafe, in other contexts, a certain limb movement could be unsafe, and so on. So, an agent is unsafe or behaves unsafely usually with respect to another agent (or itself) or environment. You probably need to keep this in mind if you want to prove anything about the safety of AI agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your second assumption, you are implicitly saying that any machine that produces the output &lt;code&gt;unsafe_output&lt;/code&gt; is unsafe, but, of course, this definition is not a realistic definition of an unsafe program. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To help you define safety in a more reasonable and natural way, I think it may be useful to reason first in terms of artificial agents, which are higher-level concepts than Turing machines. Then you could find a way of mapping agents to TMs and attempt to prove your conjectures by using the tools of the theory of computation.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-04T02:25:00.633" LastActivityDate="2020-04-04T02:25:00.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19985" PostTypeId="1" CreationDate="2020-04-03T22:05:01.273" Score="3" ViewCount="203" Body="&lt;p&gt;When you play video games, sometimes there is an AI that attempts to predict what are you going to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the Candy Crush game, if you finish the level and you still have moves remaining, you get to see fishes or other powers destroying the other candies, but, instead of watching 10 minutes of your combos without moving at all after accomplishing a level, like this &lt;a href=&quot;https://www.youtube.com/watch?v=-_p8LZhIxtE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Longest video game combo ever probably&lt;/a&gt;, it tells an alert that says tap to skip, so basically the AI is predicting all the possible combos that will keep proceeding automatically and calculating every automatic move. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can artificial intelligence predict such a thing?&lt;/p&gt;&#xA;" OwnerUserId="35725" LastEditorUserId="2444" LastEditDate="2020-04-04T02:02:52.483" LastActivityDate="2020-04-21T10:22:16.533" Title="How can artificial intelligence predict the next possible moves of the player?" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;&lt;prediction&gt;" AnswerCount="4" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19986" PostTypeId="1" AcceptedAnswerId="19988" CreationDate="2020-04-03T22:06:27.777" Score="1" ViewCount="66" Body="&lt;p&gt;To clarify it in my head, the value function calculates how 'good' it is to be in a certain state by summing all future (discounted) rewards, while the reward function is what the value function uses to 'generate' those rewards for it to use in the calculation of how 'good' it is to be in the state?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-10-08T12:51:56.457" LastActivityDate="2020-10-08T13:00:08.610" Title="What is the relationship between the reward function and the value function?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;value-functions&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19987" PostTypeId="1" AcceptedAnswerId="20071" CreationDate="2020-04-03T22:33:03.357" Score="2" ViewCount="182" Body="&lt;p&gt;I'm currently trying to code the NEAT algorithm by myself, but I got stuck with two questions. Here they are:&lt;/p&gt;&#xA;&lt;p&gt;What happens if during crossover a node is removed (or disabled) and there's a connection that was previously connected to that specific node? Because, in that case, some connections are no longer useful. Do I keep the useless connections or do I prevent this from happening? Or maybe I'm missing something?&lt;/p&gt;&#xA;&lt;p&gt;Someone on &lt;a href=&quot;https://ai.stackexchange.com/a/13930/2444&quot;&gt;AI SE&lt;/a&gt; said that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You could:&lt;/p&gt;&#xA;&lt;p&gt;1.) Use only the connection genes in crossover, and derive your node genes from the connection genes&lt;/p&gt;&#xA;&lt;p&gt;2.) Test if every node is in use, and delete the ones that are not&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But the problem with that is that my genomes will lose some complexity. Maybe I can use the nodes during crossover, and then disable the connections that were using this node. That way, I'm keeping the genotype complex, but the phenotype is still working.&lt;/p&gt;&#xA;&lt;p&gt;Is there another way to workaround this problem or this is the best way?&lt;/p&gt;&#xA;" OwnerUserId="35722" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-07T09:32:36.433" Title="Do I have to crossover my node genes in NEAT, and how?" Tags="&lt;neat&gt;&lt;neuroevolution&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19988" PostTypeId="2" ParentId="19986" CreationDate="2020-04-03T23:17:07.403" Score="1" Body="&lt;p&gt;I think it is pedagogically useful to distinguish between the theory (equations) and the practice (algorithms).&lt;/p&gt;&#xA;&lt;p&gt;If you're talking about the definition of the value function (the theory)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;v_{\pi}(s) &#xA;&amp;amp; \dot{=} \mathbb{E}_{\pi} \left[ G_t \mid S_t = s \right]\\&#xA;&amp;amp;= \mathbb{E}_{\pi} \left[ \sum_{k=0}^\infty \gamma^k R_{t+k+1} \bigl\vert S_t = s \right]\\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\dot{=}$&lt;/span&gt; means &amp;quot;is defined as&amp;quot; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; is the state space, then the value function can be defined in terms of the reward, as it can be clearly seen above. (Note that &lt;span class=&quot;math-container&quot;&gt;$R_{t+k+1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; are random variables, and, in fact, expectations are taken with respect to random variables).&lt;/p&gt;&#xA;&lt;p&gt;The definition above can actually be expanded to be a Bellman equation (i.e. a recursive equation) defined in terms of the reward function &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt; of the underlying MDP. However, often, rather than the notation &lt;span class=&quot;math-container&quot;&gt;$R(s, a)$&lt;/span&gt;, you will see &lt;span class=&quot;math-container&quot;&gt;$p(s', r \mid s, a)$&lt;/span&gt; (which represents the combination of the &lt;em&gt;transition probability function&lt;/em&gt; and the &lt;em&gt;reward function&lt;/em&gt;). Consequently, &lt;strong&gt;the value is a function of the reward&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you're estimating a value function (the practice), e.g. using Q-learning, you don't necessarily use the reward function of the Markov decision process. You can &lt;strong&gt;estimate&lt;/strong&gt; the value function by just observing the rewards that you receive while exploring the environment, without really knowing the reward function. But, by exploring the environment, you can actually estimate the reward function. For example, if every time you're in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; you take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and you receive reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, then you already know something about the actual underlying reward function. If you explore enough the MDP, you could potentially learn the reward function too (unless it keeps on changing, in that case, it may be more difficult to learn it).&lt;/p&gt;&#xA;&lt;p&gt;To conclude, yes, value functions are certainly very related to reward functions and rewards, in ways that you immediately see from the equations that define the value functions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-08T13:00:08.610" LastActivityDate="2020-10-08T13:00:08.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19989" PostTypeId="2" ParentId="6429" CreationDate="2020-04-04T01:06:56.190" Score="0" Body="&lt;p&gt;The answer is yes, an AI can be trained to write even a whole story. I just want to tell you right off the bat that an AI already did something even more difficult than generating a story. I'm talking about that thing at the end of my explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the links in my explanations are leading to external sources that I found, you can go check them. Without any further do, here are the main reasons why I think AIs can generate the outline of a story:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;AIs are really good at recognizing patterns, and generating things that are similar to others. Surprisingly, there's a lot of &lt;a href=&quot;https://blog.writingacademy.com/story-patterns/&quot; rel=&quot;nofollow noreferrer&quot;&gt;patterns in stories&lt;/a&gt;. Stories are always structured, so this part isn't the real problem. There's a great Wiki about the &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Seven_Basic_Plots&quot; rel=&quot;nofollow noreferrer&quot;&gt;seven basic plots&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;But even if an AI can generate a good story structure, can it make a story appealing? Well, it depends on how big the &quot;brain&quot; of the AI is. Because it turns out that the more neurons and synapses an AI have, the more it can &quot;understand&quot; human language or emotions. So, if an AI has a big enough brain, it can generate stuff that make sense. Here's the best example of an AI being able to generate human-like stuff : &lt;a href=&quot;https://ai.googleblog.com/2020/01/towards-conversational-agent-that-can.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.googleblog.com/2020/01/towards-conversational-agent-that-can.html&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the how, I think that the training data isn't insignificant. So, to be able to train an AI like that, we need a lot of examples. This is possible, because movies' screenplays are public, and can be downloaded by anyone. So, an AI can easily learn from this huge amount of screenplays. Here are some examples of websites where we can get screenplay of movies: &lt;a href=&quot;https://stephenfollows.com/resource/sites-to-find-movie-scripts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stephenfollows.com/resource/sites-to-find-movie-scripts/&lt;/a&gt;, &lt;a href=&quot;https://www.simplyscripts.com/movie-screenplays.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.simplyscripts.com/movie-screenplays.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, we just need to format the data, so we can give it to our AI. In my opinion, it's completely possible to make a good AI that writes good stories, because Google already did something similar. I think that the &lt;a href=&quot;https://www.youtube.com/watch?v=3Wppf_CNvD0&quot; rel=&quot;nofollow noreferrer&quot;&gt;chatbot Meena&lt;/a&gt;, created by Google, is the proof that an AI can learn way more than just pattern recognition.&lt;/p&gt;&#xA;" OwnerUserId="35722" LastEditorUserId="35722" LastEditDate="2020-04-04T18:23:02.247" LastActivityDate="2020-04-04T18:23:02.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19990" PostTypeId="2" ParentId="19982" CreationDate="2020-04-04T01:39:32.197" Score="1" Body="&lt;p&gt;If what you mean by a generic reinforcement-learning application is an application that can learn any game (or some games), then you can't do it. Why? Because the goal of each game isn't the same, so you have to adapt the rewards depending on the game. If you just want to make an AI for Connect Four, I suggest you to make a specific RL application for that game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to mention another thing: you shouldn't give a reward based on token's position, because it's hard to know what token's position is the best. Instead, just assign a big reward to the winner. That way, you're generalizing your algorithm, and you're avoiding your AI to focus on the wrong goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to be careful, you don't know for sure what token's position is the best. Try to give a reward ONLY when a player wins the game, because you can know for sure that winning is the best thing to do. It may sound idiot said like that, but that way, you're ensuring you AI to learn by itself what token's position is the best.&lt;/p&gt;&#xA;" OwnerUserId="35722" LastEditorUserId="35722" LastEditDate="2020-04-04T18:11:02.480" LastActivityDate="2020-04-04T18:11:02.480" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="19991" PostTypeId="2" ParentId="18595" CreationDate="2020-04-04T02:30:51.477" Score="1" Body="&lt;p&gt;Firstly, the key of implementing a good genetic algorithm like the NEAT one is fitness. Now fitness is everything, it tells basically what your snakes will learn. If you have a bad fitness function, your AIs will target the wrong goal. You shouldn't give fitness when a snake is aligning with food, because that's not what you want. What you really want your snake to do is eating food.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;So, I suggest you to try giving fitness only when your snake is eating food, and maybe remove some as time passes (because of hunger). Your fitness could even be just the time a snake survived, since to survive your snakes have to eat, and avoid touching themselves! Definitely try this: just giving fitness as time passes!&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, you have to give good inputs to your AIs, so that they have enough information about their environment to optimize their strategy. I suggest you to try inputs that are relative to your snake's position, and direction. X and Y position of the snake and the food are bad inputs, because they're not relative to the snake, they're relative to the origin of your game. So, every second the position of the snake is changing, and this could lead to some distraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;The snake doesn't need to know his position, he just need to know the distance between him and the food, and the angle between the direction of the snake (him) and the food.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, you may review your outputs. But this is a minor problem, you should definitely focus on the fitness of your snakes. But, if you want to go deeper, try having two outputs instead of four: speed, and turning velocity. That way, it's easy for your snakes to go straight forward. When a snake wants to go forward with left, right, up, down outputs, it's way harder.&lt;/p&gt;&#xA;" OwnerUserId="35722" LastActivityDate="2020-04-04T02:30:51.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19992" PostTypeId="2" ParentId="18208" CreationDate="2020-04-04T02:51:04.167" Score="0" Body="&lt;p&gt;When it comes to genetic algorithms (neuroevolution), you can pretty much evolve any kind of parameters. You just need to have a fairly complex system where each parameter changes the way the inputs are affecting the outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question, any activation function should work! It would be surprising that the activation function is your problem. But if you want to test some activation functions, here's a &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;list of activation functions&lt;/a&gt;. But tanh should work perfectly fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I already mentioned in another answer, the key to have a great neuroevolution algorithm is the fitness function. Your rewards should reflect the goal you're aiming for. You didn't mentioned what fitness function you used, but here's what I suggest: reward a player ONLY if he progresses in the game. For example, your fitness function could simply be the distance traveled by your birds. This could maybe solve your problem...&lt;/p&gt;&#xA;" OwnerUserId="35722" LastEditorUserId="35722" LastEditDate="2020-05-02T15:32:51.210" LastActivityDate="2020-05-02T15:32:51.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19993" PostTypeId="2" ParentId="16051" CreationDate="2020-04-04T03:59:22.983" Score="0" Body="&lt;p&gt;As I can see, at each frame, you're giving 1 fitness if the snake gets closer to the food, and you remove 10 fitness if it gets away from the food. Am I right? Also, you're removing 50 fitness when a snake dies, and giving 15 fitness when it eats a food?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try to remove the -10 fitness at each frame where the snake gets away from food, it's not necessary. Not getting fitness is also a penalty, you don't need to exaggerate the penalty as it's making your -50 fitness being negligible. Because of that, dying quickly is better than surviving for 6 frames without getting closer to the food. So the first strategy your AIs are going to find is spinning around to die quickly (because surviving is bad for them). This is not what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these are only assumptions, try it, and tell me if it works!&lt;/p&gt;&#xA;" OwnerUserId="35722" LastActivityDate="2020-04-04T03:59:22.983" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19994" PostTypeId="1" CreationDate="2020-04-04T05:12:56.393" Score="3" ViewCount="42" Body="&lt;p&gt;From my understanding, maximum likelihood estimation chooses the set of parameters for the estimator that maximizes likelihood with the ground truth distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I always interpreted it as the training set having a tendency to have most examples near the mean or the expected value of the true distribution. Since most training examples are close to the mean (since they have been sampled from this distribution) maximizing the estimator's chance of sampling these examples gets the estimated distribution close to the ground truth distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would mean that any MLE procedure on a dataset of outliers should fail miserably. Are this interpretation and conclusion correct? If not, what is wrong with the mentioned interpretation of maximizing likelihood for an estimator?  &lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2020-04-04T18:47:12.350" LastActivityDate="2020-04-04T18:47:12.350" Title="Is maximum likelihood estimation meaningless for a dataset of only outliers?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;statistical-ai&gt;&lt;cross-entropy&gt;&lt;maximum-likelihood&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19995" PostTypeId="2" ParentId="19956" CreationDate="2020-04-04T07:36:49.730" Score="2" Body="&lt;p&gt;It is your choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can even be different between training and target system. The approach called &quot;exploring starts&quot; chooses a random start state (and action if you are assessing a deterministic policy for action values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you don't have a reason to pick exploring starts, you should aim for your &lt;code&gt;env.reset()&lt;/code&gt; function to put the environment into a state drawn from the distribution of start states that you expect the agent to encounter in production. This will help if you are using function approximation - it will mean that the distribution of training data will better match the distribution seen in production, and approximators can be sensitive to that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, such as policy-gradient methods, your cost function will be defined in terms of expected return given a start state distribution, so at least during assessment you will want a &lt;code&gt;env.reset()&lt;/code&gt; function that matches that target start distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is still OK to have different distributions for start states for training and assessment, and might be worth investigating as a hyperparameter for training. For instance if the training start state distribution can pick states that are hard to get to randomly otherwise, it may help the agent to find any of those states that are high value.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-04-04T07:36:49.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19997" PostTypeId="1" AcceptedAnswerId="19999" CreationDate="2020-04-04T10:19:30.697" Score="1" ViewCount="55" Body="&lt;p&gt;I was going through the &lt;a href=&quot;https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{\sigma \to 0} \nabla_{\mathbf  x} \mathbb E_{\epsilon \sim \mathcal N(0, \sigma^2 \mathbf I)} f(\mathbf x+\epsilon) = \nabla_x f(\mathbf x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, any guide to understanding mathematical notation for reading research paper is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="2444" LastEditDate="2020-04-04T13:42:05.030" LastActivityDate="2020-04-04T13:42:05.030" Title="What does equation in the &quot;related work&quot; section of the GAN paper mean?" Tags="&lt;math&gt;&lt;generative-adversarial-networks&gt;&lt;papers&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19999" PostTypeId="2" ParentId="19997" CreationDate="2020-04-04T11:13:21.077" Score="2" Body="&lt;p&gt;In full:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The limit, as standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; tends towards zero, of the gradient with respect to vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;, of the expectation - where perturbation &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; follows the normal distribution with mean 0 and variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; times identity vector &lt;span class=&quot;math-container&quot;&gt;$[1,1,1,1...]$&lt;/span&gt; * - of any function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; plus &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is equal to the gradient with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; of the same function of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If we break that down:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lim\limits_{\sigma \rightarrow 0}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The limit, as standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; tends towards zero of&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_\mathbf{x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the gradient with respect to vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; of&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the expectation ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2\mathbf{I})}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[the expectation] - where perturbation &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; follows the normal distribution with mean 0 and variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; times identity vector &lt;span class=&quot;math-container&quot;&gt;$[1,1,1,1...]$&lt;/span&gt; * - of&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(\mathbf{x} +\epsilon)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;any function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; plus &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \nabla_x f(\mathbf{x})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;is equal to the gradient with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; of the same function &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Basically it says that taking small perturbations of a vector input to a function and measuring the gradient at those varied points can be used to give you a valid estimate of the true gradient at the point you are making variations of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of understanding the equations, read introductory texts to the research area, and if like me your maths has been unused for many years before attempting this, expect to spend time and effort. Re-read the equations, memorise and write out the basic ones from the field, apply them to simple problems that might be presented in text books. Reading maths equations is not much different to reading music or reading another language - it takes concentration, practice, time and effort to become fluent enough to read an equation and comprehend it. The different research fields can be quite different too, some might be as similar enough to get by with what you know already, others may require learning all over again.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* I am not 100% certain of the interpretation of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; as an identity vector - a matrix may be more appropriate, which depends on the form of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(\mu, \sigma^2)$&lt;/span&gt; when handling vector distributions. A matrix form for the second argument would be more general and allow for covariance, although the use of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; would then explicitly remove covariance and make each component of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; independent, which is required for this result.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-04T13:18:40.203" LastActivityDate="2020-04-04T13:18:40.203" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20000" PostTypeId="1" CreationDate="2020-04-04T11:38:33.523" Score="1" ViewCount="23" Body="&lt;p&gt;I have an AI design for deciding the length of green and red lamps of the traffic. In my design, every crossroads has its own agent. This agent has input the amount of vehicle in each road in a single junction. AI then decide how long is the red lamp and the green lamp in each junction. The fitness function is the average commute time in the city. Each agent may communicate with each other, and give reward or punishment to other AI. What AI algorithm works like this?&lt;/p&gt;&#xA;" OwnerUserId="35738" LastActivityDate="2020-04-04T11:38:33.523" Title="What kind of artificial intelligence is this? A decentralized swarm intelligence where the input and output is split among the agents" Tags="&lt;reinforcement-learning&gt;&lt;swarm-intelligence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20001" PostTypeId="1" AcceptedAnswerId="20036" CreationDate="2020-04-04T12:56:04.047" Score="2" ViewCount="66" Body="&lt;p&gt;I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I now need to normalise my features for training. I'm doing this by calculating the mean and the standard deviation per feature, and then calculate the normalised feature by subtracting the mean, taking the absolute value, and dividing by the standard deviation. Again, so far, so good, the results of my tensors which I use for training look good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand why I need to normalise input data, and I also understand why one can do it like this (I know that there are other ways as well, e.g. to map values to a 0-1 interval).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I was wondering about two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, after having trained my network, when I want to make a prediction for a specific input – do I need to normalise this as well, or do I take use the un-normalised data? Does it make a difference? My gut feeling says, I &lt;em&gt;should&lt;/em&gt; normalise it, as it &lt;em&gt;should&lt;/em&gt; make a difference, but I'm not sure. What should I do here, and why?&lt;/li&gt;&#xA;&lt;li&gt;Second, either way, I get a result. Now I was wondering whether I need to denormalise this? I mean, it should make a difference, shouldn't it? If so, how? How do I get from the normalised result value to a denormalised one? Do I just need to reverse the calculation with mean and standard deviation, to get the actual value?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It would be great if someone could shed some light on this.&lt;/p&gt;&#xA;" OwnerUserId="35739" LastEditorUserId="9947" LastEditDate="2020-04-07T02:26:57.383" LastActivityDate="2020-04-07T02:26:57.383" Title="Do I need to denormalise results in linear regression?" Tags="&lt;machine-learning&gt;&lt;ai-basics&gt;&lt;linear-regression&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20003" PostTypeId="1" CreationDate="2020-04-04T13:16:07.693" Score="2" ViewCount="148" Body="&lt;p&gt;I intend to develop a custom environment for open-ai's gym.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is for an agent to learn (among additional objectives) dividing a certain quantity drawn from a continous action space (i.e. spaces.box ) freely into a number of parts. Different divisions are strategically relevant too and should hence arise endogenous from the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of subdivisions would be in principle freely chosen, but im fine with restricting the agent to maximally 4 subdivisions. However, it should be possible to pick less i.e. 0,1 or 3 subdivisions instead of 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, this leads to a requirement that quantities &lt;span class=&quot;math-container&quot;&gt;$a_{i}$&lt;/span&gt; drawn from an action space &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; sum up to some fixed Number &lt;span class=&quot;math-container&quot;&gt;$$c=a_{1}+...+ a_{N}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One instance of this problem is drawing several events probabilities &lt;span class=&quot;math-container&quot;&gt;$p_{i} \in [0,1]$&lt;/span&gt; and requiring that they form a probability distribution by &lt;span class=&quot;math-container&quot;&gt;$p_{1}+...+p_{n}=1$&lt;/span&gt;. This seems like a rather general problem in itself. However, any problem that explores dividing some capacity could be formulated in a similar manner. Therefore this seems a wide spread use-case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that both gym's spaces.box and spaces.discrete come only equipped with a sample condition. Each drawn sample from  for instance box.sample is completely independent of later drawn samples and samples by other action and state-spaces. I have really no clue how to implement this easily without massive hacks.....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best practice to enforce that samples drawn from different action spaces (or at least from consecutively drawn samples from the same space) sum up to a given number? Are there examples that implemented this properly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To finish I have to emphasize that i would like to solve this in gym or the environment level only. In the spirit of gym, I want to avoid any algorithm specific post-processing such that i can use algorithm interchangeably both for existing and future algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I believe that, despite bounty this question will not get answered soon. Maybe it simply is not easily possible. As a next step, I will open an issue at Gym's Github account. Ill let you know how it goes.&lt;/p&gt;&#xA;" OwnerUserId="35690" LastEditorUserId="35690" LastEditDate="2020-04-11T11:48:20.977" LastActivityDate="2020-04-11T11:48:20.977" Title="How to formulate normalization/probability conditions on state-action spaces in Gym?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;environment&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20004" PostTypeId="1" CreationDate="2020-04-04T13:21:53.347" Score="1" ViewCount="103" Body="&lt;p&gt;I'm creating an RL application for the game Connect Four. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I tell the algorithm which moves/token positions will receive greater rewards, surely it's not actually &lt;strong&gt;learning&lt;/strong&gt; anything; it's just a basic lookup for the algorithm? &quot;Shall I place the token here, or here? Well, this one receives a greater reward, so I choose this one.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, some pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function get_reward()&#xA;    if 2 in a line&#xA;        return 1&#xA;    if 3 in a line&#xA;        return 2&#xA;    if 4 in a line&#xA;        return 10&#xA;    else &#xA;        return -1&#xA;&#xA;foreach columns&#xA;    column_reward_i = get_reward(column_i)&#xA;    if column_reward_i &amp;gt;= column_rewards&#xA;        place_token(column_i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27629" LastActivityDate="2020-04-04T17:17:16.743" Title="In RL, if I assign the rewards for better positional play, the algorithm is learning nothing?" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20005" PostTypeId="1" AcceptedAnswerId="25585" CreationDate="2020-04-04T13:41:39.820" Score="4" ViewCount="67" Body="&lt;p&gt;Do I have to prevent nodes created from the same connection gene to have different IDs/innovation number? In this example, the node 6 is created from the connection going from node 3 to node 4:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QFUbz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QFUbz.jpg&quot; alt=&quot;Example of node mutation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case where that specific node was already globally created, is it useful to give it the same ID for crossover? Because the goal of NEAT is to do meaningful crossover by doing historical marking. The paper from Kenneth O. Stanley says at page 108:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] by keeping a list of the innovations that occurred in the current generation, it&#xA;  is possible to ensure that when the same structure arises more than once through independent mutations in the same generation, each identical mutation is assigned the&#xA;  same innovation number.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why don't we do that for node genes too?&lt;/p&gt;&#xA;" OwnerUserId="35722" LastEditorUserId="35722" LastEditDate="2020-04-04T14:41:52.907" LastActivityDate="2021-01-08T17:03:22.113" Title="In NEAT, is it a good idea to give the same ID to node genes created from the same connection gene?" Tags="&lt;neat&gt;&lt;neuroevolution&gt;&lt;crossover-operators&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20006" PostTypeId="1" CreationDate="2020-04-04T13:54:05.900" Score="2" ViewCount="34" Body="&lt;p&gt;Some people claim that DQN was used to play many Atari games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture of the RL agent? Did the reward function change for each game?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-04-04T14:02:50.637" LastActivityDate="2020-04-04T14:02:50.637" Title="How was the DQN trained to play many games?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;rewards&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20007" PostTypeId="2" ParentId="20004" CreationDate="2020-04-04T15:16:51.477" Score="4" Body="&lt;p&gt;What you are proposing is closer to a heuristic for searching than a reward for RL. This is a blurred line, but generally if you start analysing the problem yourself, breaking it down into components and feeding that knowledge into the algorithm, then you place more emphasis on your understanding of the problem, and less on any learning that an agent might do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically in a RL formulation of a simple board game, you would choose rewards or +1 for a win (the goal), 0 for a draw, and -1 for a loss. All non-terminal states would score 0 reward. The point of the RL learning algorithm is that the learning process would assign some nominal value to interim states due to observing play. For value-based RL approaches, such as Q learning or Monte Carlo Control, the algorithm does this more or less directly by &quot;backing up&quot; rewards that it experiences in later states into average value estimates for earlier states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most game-playing agents will combine the learning process, which will be imperfect given the limited experience an agent can obtain compared to all possible board states, with a look-ahead search method. Your heuristic scores would also make a reasonable input to a search method too - the difference being you may need to search more deeply using your simple heuristic than if you used a learned heuristic. The simplest heuristic would just be +1 for a win, 0 for everything else, and is still reasonably effective for Connect 4 if you can make it search e.g. 10 moves ahead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The combination of deep Q learning and negamax search is quite effective in Connect 4. It can make near perfect agents. However, if you actually want a perfect agent, you are better off skipping the self-learning approach and working on optimised look-ahead search with some depth of opening moves stored as data (because search is too expesnive in the early game, even for a simple game like Connect 4).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-04T17:17:16.743" LastActivityDate="2020-04-04T17:17:16.743" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20008" PostTypeId="1" CreationDate="2020-04-04T19:10:46.820" Score="1" ViewCount="337" Body="&lt;p&gt;I think this model is underfitting. Is this correct?     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MxSJd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MxSJd.png&quot; alt=&quot;Model loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yvOa6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yvOa6.png&quot; alt=&quot;Prediction vs Real&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;lstm_1 (LSTM)                (50, 60, 100)             42400     &#xA;_________________________________________________________________&#xA;dropout_1 (Dropout)          (50, 60, 100)             0         &#xA;_________________________________________________________________&#xA;lstm_2 (LSTM)                (50, 60)                  38640     &#xA;_________________________________________________________________&#xA;dropout_2 (Dropout)          (50, 60)                  0         &#xA;_________________________________________________________________&#xA;dense_1 (Dense)              (50, 20)                  1220      &#xA;_________________________________________________________________&#xA;dense_2 (Dense)              (50, 1)                   21        &#xA;=================================================================&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above is a summary of the model.&lt;br&gt;&#xA;Any advice on how the model could be improved?&lt;/p&gt;&#xA;" OwnerUserId="35748" LastActivityDate="2021-04-30T01:07:30.697" Title="Is this Keras LSTM model underfitting?" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20009" PostTypeId="1" AcceptedAnswerId="20015" CreationDate="2020-04-04T19:46:05.450" Score="2" ViewCount="43" Body="&lt;p&gt;So, Tay the racist tweeter bot... one thing that could have prevented this would have been to have a list of watchwords to not respond to, with some logic similar to &lt;code&gt;foreach (word in msg) {if (banned_words.has(word)) disregard()}&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if that wouldn't, what I'm getting at is obvious: I am building a chatterbot that must be kid-friendly. For my sake and for the sake of whoever finds this question, &lt;strong&gt;is there a resource consisting of a .csv or .txt of such words that one might want to handle?&lt;/strong&gt; I remember once using a site-blocking productivity extension that had visible its list of banned words; not just sexually charged words, but racial slurs, too.&lt;/p&gt;&#xA;" OwnerUserId="35622" LastActivityDate="2020-04-05T06:32:23.780" Title="Does there exist a resource for vetting banned words for chatbots?" Tags="&lt;chat-bots&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20011" PostTypeId="2" ParentId="20008" CreationDate="2020-04-04T21:37:35.400" Score="0" Body="&lt;p&gt;You need to include optimizer you used to make sure it is correct.&#xA;By the way, your drop-out layers are not going to do anything, so you should take them away. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You likely don’t have test and train data in time-series because all data points are connected. It just has prediction value and ground truth of each period. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend you use the whole dataset and rotate changing hyper-parameters of LSTM to find the best model. &lt;/p&gt;&#xA;" OwnerUserId="35752" LastActivityDate="2020-04-04T21:37:35.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20012" PostTypeId="1" CreationDate="2020-04-05T01:41:58.803" Score="3" ViewCount="83" Body="&lt;p&gt;After spending some time reading about POMDP, I'm still having a hard time understanding how grid-based solutions work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the finite horizon brute-force solution, where you have your current belief distribution, enumerate every possible collection of action/observation combinations for a given depth and find the expected reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to read some sources about grid-based approximations, for example, &lt;a href=&quot;https://www.cs.uic.edu/~piotr/cs594/bharanee-houskreht.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides&lt;/a&gt; describe the grid-based approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's not clear to me what exactly is going on. I'm not understanding how the value function is actually computed. After you take an action, how do you update your belief states to be consistent with the grid? Does the grid-based solution simply reduce the set of belief states? How does this reduce the complexity of the problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not seeing how this reduces the number of actions, observation combinations needed to be considered for a finite-horizon solution. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-04-05T14:13:47.550" LastActivityDate="2020-04-05T15:06:46.630" Title="What is the intuition behind grid-based solutions to POMDPs?" Tags="&lt;markov-decision-process&gt;&lt;pomdp&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20013" PostTypeId="1" CreationDate="2020-04-05T01:52:42.053" Score="1" ViewCount="38" Body="&lt;p&gt;I’m currently working on the Food-101 dataset. I want to train a model that is greater than 85% accuracy for top-1 for the test set, using a ResNet50 or smaller network with a reasonable set of augmentations. I’m running 10 epochs using ResNet34 and I’m currently on the 8th epoch. This is how its doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;epoch   train_loss  valid_loss  error_rate  time&#xA;0   2.526382    1.858536    0.465891    25:21&#xA;1   1.981913    1.566125    0.406881    27:21&#xA;2   1.748959    1.419548    0.372129    27:16&#xA;3   1.611638    1.315319    0.346980    25:16&#xA;4   1.568304    1.250232    0.328069    24:43&#xA;5   1.438499    1.193816    0.313762    24:26&#xA;6   1.378019    1.156924    0.307426    24:30&#xA;7   1.331075    1.131671    0.299010    24:26&#xA;8   1.314978    1.115857    0.297079    24:24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, it doesn’t seem like I’m going to do better than 71% accuracy at this point. The dataset size is 101,000. It has 101 different kinds of food and each food has a 1000 images. Training this definitely takes long but what are some things I can do to improve its accuracy?&lt;/p&gt;&#xA;" OwnerUserId="35754" LastEditorUserId="9947" LastEditDate="2020-04-07T02:30:20.327" LastActivityDate="2020-04-07T02:30:20.327" Title="Running 10 epochs on the Food-101 dataset" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20014" PostTypeId="2" ParentId="20012" CreationDate="2020-04-05T03:23:14.773" Score="3" Body="&lt;p&gt;I will attempt to provide an answer to your questions based on the information you can find in the papers &lt;a href=&quot;https://www.aaai.org/Papers/AAAI/1997/AAAI97-113.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Heuristic Variable Grid Solution Method for POMDPs&lt;/a&gt; (1997) by Ronen I. Brafman and &lt;a href=&quot;https://www.cs.mcgill.ca/~jpineau/files/jpineau-ijcai03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Point-based value iteration: An anytime algorithm for POMDPs&lt;/a&gt; (2003) by Joelle Pineau et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;grid-based&lt;/strong&gt; approximate solution to a POMDP attempts to estimate a value function only at a subset of the number of &lt;strong&gt;belief states&lt;/strong&gt;. &lt;em&gt;Why?&lt;/em&gt; Because estimating the value function for all belief states is typically computationally infeasible for non-small POMDPs, given that the belief-space MDP (i.e. an MDP where the state space consists of probability distributions over the original states of the POMDP) of a POMDP with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; states has an uncountably large state space. &lt;em&gt;Why?&lt;/em&gt; Because of the involved probability distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How do we compute the value for the belief states that do not correspond to a point of the grid?&lt;/em&gt; We can use e.g. &lt;strong&gt;interpolation&lt;/strong&gt;, i.e. the value of a belief state that does not correspond to a point of the grid is computed as a function of the value of the belief states that correspond to other grid points (typically, the neighboring grid points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why is this approach feasible?&lt;/em&gt; The assumption is that interpolation is not as expensive as computing the value of a belief state. However, note that you may not need to interpolate at every step of your algorithm, i.e. interpolation could be performed only when the value of a certain belief state is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How do you compute the value of a belief state that corresponds to a grid point?&lt;/em&gt; It can be computed with a &lt;strong&gt;value iteration&lt;/strong&gt; (dynamic programming) algorithm for POMDPs. An overview of a value iteration algorithm can be found in section 2 of the paper &lt;a href=&quot;https://www.cs.mcgill.ca/~jpineau/files/jpineau-ijcai03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Point-based value iteration: An anytime algorithm for POMDPs&lt;/a&gt;. &lt;a href=&quot;http://www.pomdp.org/tutorial/pomdp-vi-example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an example of the application of the value iteration algorithm for POMDPs&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The grid-based approach, introduced in &lt;a href=&quot;https://dl.acm.org/doi/10.5555/2753361.2753375&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computationally Feasible Bounds for Partially Observed Markov Decision Processes&lt;/a&gt; (1991) by William S. Lovejoy, is very similar to the &lt;strong&gt;point-based&lt;/strong&gt; approach, which was introduced in &lt;a href=&quot;https://www.cs.mcgill.ca/~jpineau/files/jpineau-ijcai03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Point-based value iteration: An anytime algorithm for POMDPs&lt;/a&gt;. The main differences between the two approaches can be found in section 3 of &lt;a href=&quot;https://www.cs.mcgill.ca/~jpineau/files/jpineau-ijcai03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Point-based value iteration: An anytime algorithm for POMDPs&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of discretizing your problem or simply computing the desired value at a  subset of the domain has been applied in other contexts too. For example, in the context of computer vision, you can approximate the derivative (or gradient) of an image (which is thus considered a function) at discrete points of the domain (i.e. the pixels).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's &lt;a href=&quot;https://github.com/JuliaPOMDP/BeliefGridValueIteration.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;a Julia implementation&lt;/a&gt; of &lt;a href=&quot;https://dl.acm.org/doi/10.5555/2753361.2753375&quot; rel=&quot;nofollow noreferrer&quot;&gt;the first grid-based approximative solution to POMDP&lt;/a&gt;. There's also &lt;a href=&quot;https://github.com/namoshizun/PyPOMDP&quot; rel=&quot;nofollow noreferrer&quot;&gt;a Python implementation of the point-based approach&lt;/a&gt;. These implementations may help you to understand the details of these approaches. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-05T15:06:46.630" LastActivityDate="2020-04-05T15:06:46.630" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20015" PostTypeId="2" ParentId="20009" CreationDate="2020-04-05T06:32:23.780" Score="1" Body="&lt;p&gt;I have not found one other than scraping a few pages from Urban Dictionary, I built my list via crowdsourced style and got a number of interesting words I had not considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start with the worst words you can think of, then try slang and accidental or on purpose misspellings of them&lt;/p&gt;&#xA;" OwnerUserId="34095" LastActivityDate="2020-04-05T06:32:23.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20016" PostTypeId="1" CreationDate="2020-04-05T08:02:50.353" Score="1" ViewCount="35" Body="&lt;p&gt;Currently, I am following the &lt;a href=&quot;https://www.youtube.com/playlist?list=PLQiyVNMpDLKnZYBTUOlSI9mi9wAErFtFm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dan Jurofsky NLP Tutorial&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=8rXD5-xhemo&amp;amp;list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS 224 Stanford 2019&lt;/a&gt;. Can you list tutorials and blogs for beginners to master the cross-lingual information retrieval?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-04-05T15:23:04.530" LastActivityDate="2020-04-05T15:23:04.530" Title="What are examples of tutorials and blogs for beginners to master the cross-lingual information retrieval?" Tags="&lt;natural-language-processing&gt;&lt;resource-request&gt;&lt;information-retrieval&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20017" PostTypeId="1" CreationDate="2020-04-05T09:30:27.043" Score="2" ViewCount="69" Body="&lt;p&gt;Is there any difference between the model distribution and data distribution, or are they the same?&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="2444" LastEditDate="2020-04-05T12:56:55.190" LastActivityDate="2020-04-05T14:09:12.930" Title="What is the difference between model and data distributions?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;models&gt;&lt;probability-distribution&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20018" PostTypeId="2" ParentId="20017" CreationDate="2020-04-05T10:40:35.467" Score="3" Body="&lt;p&gt;Yes. In Machine Learning we consider that the samples in your training set are sampled from an underlying distribution called the &lt;strong&gt;data generating distribution&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative models&lt;/a&gt; classify the samples by trying to &lt;em&gt;learn&lt;/em&gt; the distribution of the data. In most cases, either the model is incapable of doing so, or the training samples aren't enough to properly describe the data-generating distribution, so the model learns an &lt;em&gt;approximation&lt;/em&gt; of this. This is what you call the &lt;strong&gt;model's distribution&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more info about these concepts in a more &lt;a href=&quot;https://datascience.stackexchange.com/a/54349/34269&quot;&gt;detailed answer&lt;/a&gt; I wrote. If you're familiar with GANs, you can also read &lt;a href=&quot;https://stats.stackexchange.com/a/435159/119015&quot;&gt;this post&lt;/a&gt;, to see where these two concepts come into play when training the two networks.    &lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="2444" LastEditDate="2020-04-05T14:09:12.930" LastActivityDate="2020-04-05T14:09:12.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20019" PostTypeId="1" CreationDate="2020-04-05T11:09:48.083" Score="2" ViewCount="79" Body="&lt;p&gt;I would like to develop an LSTM because I have a variable input matrix. I am zero-padding to a specific length of 800. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am not sure of how to classify a certain situation when each input matrix has multiple labels inside, i.e. &lt;code&gt;0&lt;/code&gt;, &lt;code&gt;1&lt;/code&gt; and &lt;code&gt;2&lt;/code&gt;. Do I need to use multi-label classification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SQIYo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SQIYo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Data shape&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(250,800,4)&#xA;&#xA;x_train(150,800,4)&#xA;y_train(150,800,1)&#xA;x_test(100,800,4)&#xA;y_test(100,800,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6zIta.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6zIta.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Building LSTM&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()    &#xA;model.add(LSTM(100, input_shape=))    &#xA;model.add(Dropout(0.5))    &#xA;model.add(Dense(100, activation='relu'))    &#xA;model.add(Dense(800, activation='softmax'))    &#xA;model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure of how to build the LSTM for training and test. If I train my model with a 3D shape. It will mean that for real time predictions I should also have a 3D shape, but the idea is to have a 2D matrix as an input. &lt;/p&gt;&#xA;" OwnerUserId="35763" LastEditorUserId="35763" LastEditDate="2020-04-09T09:31:17.600" LastActivityDate="2020-04-09T09:31:17.600" Title="How to implement a LSTM for multilabel classification problem?" Tags="&lt;classification&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20020" PostTypeId="2" ParentId="20013" CreationDate="2020-04-05T14:05:16.950" Score="1" Body="&lt;p&gt;try using an adjustable learning rate. Keras has a number of callbacks that are useful for this purpose. The ReduceLROnPlateau callback can be used to monitor validation loss and reduce the learning rate by a factor if the validation loss does not decrease after a user specified number of epochs. The ModelCheckpoint callback is useful to monitor the validation loss and save the model with the lowest loss which can then be used to make predictions. Documentation is &lt;a href=&quot;https://keras.io/callbacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-05T14:05:16.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20021" PostTypeId="5" CreationDate="2020-04-05T15:23:24.157" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-05T15:23:24.157" LastActivityDate="2020-04-05T15:23:24.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20022" PostTypeId="4" CreationDate="2020-04-05T15:23:24.157" Score="0" Body="For questions related to information retrieval (IR) in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-05T15:23:24.157" LastActivityDate="2020-04-05T15:23:24.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20023" PostTypeId="1" AcceptedAnswerId="20025" CreationDate="2020-04-05T17:00:55.370" Score="3" ViewCount="65" Body="&lt;p&gt;How could I use a 16x16 image as an input in a HMM? And at the same time how would I train it? Can I use backpropagation?&lt;/p&gt;&#xA;" OwnerUserId="35660" LastEditorUserId="2193" LastEditDate="2020-04-05T17:41:26.457" LastActivityDate="2020-04-06T09:58:09.960" Title="How can I use a Hidden Markov Model to recognize images?" Tags="&lt;image-recognition&gt;&lt;backpropagation&gt;&lt;markov-chain&gt;&lt;hidden-markov-model&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20024" PostTypeId="2" ParentId="19905" CreationDate="2020-04-05T17:30:33.197" Score="1" Body="&lt;h2&gt;Short Answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Tomas Mikolov's mention of gradient clipping in a  single paragraph of his PhD thesis in 2012 is the first appearance in the literature.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Long Answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The first source (Mikolov, 2012) in the &lt;em&gt;Deep Learning&lt;/em&gt; book is Mikolov's PhD thesis and can be found &lt;a href=&quot;https://www.fit.vutbr.cz/~imikolov/rnnlm/thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The end of section 3.2.2 is where gradient clipping is discussed, only it's called &lt;em&gt;turncating&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... The exploding gradient problem has&#xA;  been described in [4].&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A simple solution to the exploding gradient problem is to truncate values of the gradients. In my experiments, I did limit maximum size of gradients of errors that get accumulated in the hidden neurons to be in a range &amp;lt; −15; 15 &gt;. This greatly increases stability&#xA;  of the training, and otherwise it would not be possible to train RNN LMs successfully on&#xA;  large data sets.&lt;br&gt;&#xA;  ...&lt;br&gt;&#xA;  [4] Y. Bengio, P. Simard, P. Frasconi. Learning Long-Term Dependencies with Gradient Descent is Difficult. IEEE Transactions on Neural Networks, 5, 157-166, 1994. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A search of the referenced paper &lt;a href=&quot;http://ai.dinfo.unifi.it/paolo/ps/tnn-94-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[4]&lt;/a&gt; shows that it does describe the problem as Mikolov said, but it does not present gradient clipping as a solution.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I had a look at the second source &lt;em&gt;Deep Learning&lt;/em&gt; mentioned: &lt;a href=&quot;https://arxiv.org/abs/1211.5063&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;On the difficulty of training Recurrent Neural Networks&lt;/em&gt;&lt;/a&gt;. It directly cites Mikolov as having proposed clipping: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We would make a final note about the approach proposed by Tomas Mikolov in his PhD thesis (Mikolov, 2012) (and implicitly used in the state of the art results on language modelling (Mikolov et al., 2011)). It involves clipping the gradient’s temporal components element-wise (clipping an entry when it exceeds in absolute value a fixed threshold). Clipping has been shown to do well in practice and it forms the backbone of our approach.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I thought about emailing Mikolov to verify that his thesis was the origin of the idea. But then I noticed that he is a co-author of this paper which cites him as proposing it! Though I still wonder if it was commonly used in practice before &#xA;even though it had not been published. &lt;/p&gt;&#xA;" OwnerUserId="34395" LastActivityDate="2020-04-05T17:30:33.197" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20025" PostTypeId="2" ParentId="20023" CreationDate="2020-04-05T17:46:49.343" Score="5" Body="&lt;p&gt;You wouldn't, normally. A &lt;a href=&quot;https://en.wikipedia.org/wiki/Hidden_Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMM&lt;/a&gt; is used to model &lt;em&gt;sequences of observations&lt;/em&gt;, and it would not make sense to use it for image recognition. Unless they are sequential, such as strokes in handwriting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HMMs are typically used in fields such as speech recognition and part-of-speech tagging. Here you observe a sequence of events that you want to fit to a model in order to classify the individual observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For training a HMM you would use something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Baum-Welch Algorithm&lt;/a&gt;; for finding the most likely sequence (ie the recognition process) the &lt;a href=&quot;https://en.wikipedia.org/wiki/Viterbi_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Viterbi Algorithm&lt;/a&gt; is used.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-04-06T09:58:09.960" LastActivityDate="2020-04-06T09:58:09.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20026" PostTypeId="1" CreationDate="2020-04-05T18:08:08.727" Score="0" ViewCount="90" Body="&lt;p&gt;After transforming timeseries into an image format, I get a width-height ratio of ~135. Typical image CNN applications involve either square or reasonably-rectangular proportions - whereas mine look nearly like lines:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QHR8i.png&quot; width=&quot;600&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example dimensions: &lt;code&gt;(16000, 120, 16) = (width, height, channels)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are 2D CNNs expected to work well with such aspect ratios? What hyperparameters are appropriate - namely, in Keras/TF terms, &lt;code&gt;strides&lt;/code&gt;, &lt;code&gt;kernel_size&lt;/code&gt; (is 'unequal' preferred, e.g. &lt;code&gt;strides=(16, 1)&lt;/code&gt;)? Relevant publications would help.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Clarification&lt;/strong&gt;: &lt;code&gt;width == timesteps&lt;/code&gt;. The images are obtained via a &lt;em&gt;transform&lt;/em&gt; of the timeseries, e.g. Short-time Fourier Transform. &lt;code&gt;channels&lt;/code&gt; are the original channels. &lt;code&gt;height&lt;/code&gt; is the result of the transform, e.g. frequency information. The task is binary classification of EEG data (w/ sigmoid output).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/fvigee/d_can_cnns_handle_extremely_long_images/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Relevant thread&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32165" LastEditorUserId="32165" LastEditDate="2020-04-06T12:08:48.800" LastActivityDate="2021-05-01T16:09:05.860" Title="How to handle extremely 'long' images?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20027" PostTypeId="1" CreationDate="2020-04-05T19:47:07.523" Score="2" ViewCount="27" Body="&lt;p&gt;To be clear, &lt;a href=&quot;https://linguistics.stackexchange.com/a/3254/28348&quot;&gt;recursion in linguistics is here better called &amp;quot;nesting&amp;quot;&lt;/a&gt; in this CS context to avoid confusing it with the &lt;em&gt;other&lt;/em&gt; recursion. How does one detect nesting? I am particularly interested in the example case of conjunctions. For example: say that I want to look for sentences that look like this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would you rather have ten goldfish &lt;strong&gt;or&lt;/strong&gt; a raccoon?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Seems straightforward: a binary choice. However, how do you distinguish a binary choice with nesting from a ternary (or &lt;em&gt;n&lt;/em&gt;-ary) choice?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would you rather have (one &lt;strong&gt;or&lt;/strong&gt; two dogs) &lt;strong&gt;or&lt;/strong&gt; (a raccoon)?&lt;/p&gt;&#xA;&lt;p&gt;Would you rather have (two dogs) &lt;strong&gt;or&lt;/strong&gt; (ten goldfish) &lt;strong&gt;or&lt;/strong&gt; (a raccoon)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Ditto for implied uses of &amp;quot;or,&amp;quot; which is more common than the latter of the above:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would you rather have (one &lt;strong&gt;or&lt;/strong&gt; two dogs),&lt;em&gt;&lt;strong&gt;[nothing]&lt;/strong&gt;&lt;/em&gt; (ten goldfish), &lt;strong&gt;or&lt;/strong&gt; (a raccoon)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Given the available tools for NLP (POS-taggers and the like), how do you count the number of conjunctions to say &amp;quot;there are &lt;em&gt;n&lt;/em&gt; surface-level clauses in the sentence, with &lt;em&gt;n&lt;/em&gt;-or-zero clauses nested within.&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="35622" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-05T22:18:07.770" Title="How does one detect linguistic recursion so as to know how much nesting there is, if any?" Tags="&lt;natural-language-processing&gt;&lt;computational-linguistics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20028" PostTypeId="1" AcceptedAnswerId="20068" CreationDate="2020-04-06T03:42:58.263" Score="1" ViewCount="26" Body="&lt;p&gt;I'm trying to understand how Bidirectional RNNs work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I want to know whether a single cell is used with  different states, or two different cells are used, each having independent parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pythonic pseudocode,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation 1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cell = rev_cell = RNNCell()&#xA;cell_state = cell.get_initial_state()&#xA;rev_cell_state = rev_cell.get_initial_state()&#xA;for i in range(len(series)):&#xA;    output, cell_state = cell(series[i], cell_state)&#xA;    rev_output, rev_cell_state = rev_cell(series[-i-1], rev_cell_state)&#xA;    final_output = concatenate([output, rev_output])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Implementation 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cell = RNNCell()&#xA;rev_cell = RNNCell()&#xA;cell_state = cell.get_initial_state()&#xA;rev_cell_state = rev_cell.get_initial_state()&#xA;for i in range(len(series)):&#xA;    output, cell_state = cell(series[i], cell_state)&#xA;    rev_output, rev_cell_state = rev_cell(series[-i-1], rev_cell_state)&#xA;    final_output = concatenate([output, rev_output])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which of the above implementations is correct? Or is the working of Bidirectional RNNs completely different altogether?&lt;/p&gt;&#xA;" OwnerUserId="33346" LastActivityDate="2020-04-07T08:06:38.170" Title="Inner working of Bidirectional RNNs" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20029" PostTypeId="2" ParentId="20026" CreationDate="2020-04-06T03:53:15.877" Score="0" Body="&lt;p&gt;I had recently used a slightly unorthodox method to process such images, which involved using RNNs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume the image dimensions to be (16000, 120, 16) = (width, height, channels), as in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apply a 2D convolution (or multiple such convolutions) of shape(1, k, c),&#xA;such that the output of the convolutions becomes (16000, 1, c). So if you only use a single convolutional layer, &lt;code&gt;k=120&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, squeeze the extra dimension, to get the shape (16000, c). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem has now been transformed back into a sequence problem! You can use RNN variants for further processing.&lt;/p&gt;&#xA;" OwnerUserId="33346" LastEditorUserId="33346" LastEditDate="2020-04-06T04:00:53.443" LastActivityDate="2020-04-06T04:00:53.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20030" PostTypeId="1" CreationDate="2020-04-06T04:02:29.153" Score="1" ViewCount="35" Body="&lt;p&gt;A paper &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/31696163&quot; rel=&quot;nofollow noreferrer&quot;&gt;says&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, annotations used as inputs to C-GAN are typically based only on &lt;strong&gt;shape information&lt;/strong&gt;, which can result in undesirable intensity distributions in the resulting artificially-created images.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does &quot;shape information&quot; mean here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of basic concept of GAN(&lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative adversarial networks&lt;/a&gt;), though I don't understand what &quot;shape information&quot; refers to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jimvw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jimvw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware the image above is an illustration of Image Segmentation. Would I think of the any one of the segmented area (red, green, blue) as a shape for a GAN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please give a hint? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="35782" LastEditorUserId="35782" LastEditDate="2020-04-06T04:24:11.240" LastActivityDate="2021-05-01T06:06:15.500" Title="What does &quot;shape information&quot; mean in terms of GAN(generative adversarial networks)?" Tags="&lt;terminology&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20031" PostTypeId="2" ParentId="20030" CreationDate="2020-04-06T04:13:10.920" Score="0" Body="&lt;p&gt;&quot;Shape information&quot; refers to the outlines of objects in an image. It does not include information like color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can think of it as the output of an edge detector on a greyscale version of the original image&lt;/p&gt;&#xA;" OwnerUserId="33346" LastActivityDate="2020-04-06T04:13:10.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20033" PostTypeId="2" ParentId="16260" CreationDate="2020-04-06T04:44:23.520" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;The number of dichotomies of 4 data points will clearly be &lt;span class=&quot;math-container&quot;&gt;$2^4 = 16$&lt;/span&gt;. According to these &lt;a href=&quot;https://www.eecis.udel.edu/~arce/files/Courses/StatLearning/5.%20Training%20vs%20Testing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; the definition of dichotomy in context of Statistical Learning is:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Different ‘hypotheses’ over the finite set of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; input points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which basically means hypotheses with &lt;strong&gt;unique&lt;/strong&gt; behaviours over the input points. Two or more different hypotheses can have same behaviour on the data points (consider the case of a square covering the &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; data points, an even larger square will also cover the &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; data points. Thus they are different hypotheses but have same behaviour) and hence emphasis on the term &lt;strong&gt;unique&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The proof of axis aligned squares having &lt;span class=&quot;math-container&quot;&gt;$\mathcal V \mathcal C$&lt;/span&gt; dimension &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; can be found &lt;a href=&quot;https://cs.nyu.edu/~mohri/ml14/sol2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It's pretty straightforward so I don't want to explain it here.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-04-06T20:54:00.243" LastActivityDate="2020-04-06T20:54:00.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20034" PostTypeId="1" AcceptedAnswerId="20049" CreationDate="2020-04-06T07:16:21.873" Score="4" ViewCount="133" Body="&lt;p&gt;I know that this has been asked a hundred times before, however, I was not able to find a question (and an answer) which actually answered what I wanted to know, respectively, which explained it in a way I was able to understand. So, I'm trying to rephrase the question…&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When working with neural networks, you typically split your data set into three parts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Training set&lt;/li&gt;&#xA;&lt;li&gt;Validation set&lt;/li&gt;&#xA;&lt;li&gt;Test set&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I understand that you use the training set for, well, train the network, and that you use the test set to verify how well it has learned: By measuring how well the network performs on the test set, you know what to expect when actually using it later on. So far, so good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, a model has hyper parameters, which – besides the weights – need to be tuned. If you change these, of course, you get different results. This is where in all explanations the validation set comes into play:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train using the training set&lt;/li&gt;&#xA;&lt;li&gt;Validate how well the model performs using the validation set&lt;/li&gt;&#xA;&lt;li&gt;Repeat this for a number of variants which differ in their hyperparameters (or do it in parallel, right from the start)&lt;/li&gt;&#xA;&lt;li&gt;Finally, select one and verify its performance using the test set&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, my question is: Why would I need steps 2 and 3? I could as well train multiple version of my model in parallel, and then run all of them against the test set, to see which performs best, and then use this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in other words: Why would I use the validation set for comparing the model variants, if I could directly use the test set to do so? I mean, I need to train multiple versions either way. What is the benefit of doing it like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably, there is some meaning to it, and probably I got something wrong, but I can't figure out what. Any hints?&lt;/p&gt;&#xA;" OwnerUserId="35739" LastEditorUserId="2444" LastEditDate="2020-04-06T20:06:29.673" LastActivityDate="2020-04-06T20:06:29.673" Title="Why do we need both the validation set and test set?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;datasets&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20035" PostTypeId="1" AcceptedAnswerId="20039" CreationDate="2020-04-06T07:58:05.687" Score="3" ViewCount="55" Body="&lt;p&gt;As per a &lt;a href=&quot;https://towardsdatascience.com/image-to-image-translation-69c10c18f6ff&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt;, image-to-image translation is a type of CV problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I understand the concept of image-to-image translation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aAakq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aAakq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that GANs(generative adversarial networks) are good at this kind of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just wondered what the commonly used techniques are for this kind of problems Before GAN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please give a hint? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="35782" LastEditorUserId="35782" LastEditDate="2020-04-06T13:04:08.347" LastActivityDate="2020-04-06T13:04:08.347" Title="Before GAN, what are the commonly used techniques for image-to-image translation?" Tags="&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20036" PostTypeId="2" ParentId="20001" CreationDate="2020-04-06T11:10:15.850" Score="0" Body="&lt;p&gt;Okay, I figured it out by myself, simply by trial and error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you normalize your training and test data, you also need to normalize the input you want to have a prediction for. You will also need to denormalize the result, to get a reasonable prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, in my case, this means:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First I calculated the mean and the standard deviation, and then subtracted the mean and divided by standard deviation, to normalize my training and test data.&lt;/li&gt;&#xA;&lt;li&gt;So to normalize the input I want to have a prediction for, I also subtract the mean and divide by the standard deviation.&lt;/li&gt;&#xA;&lt;li&gt;Finally, when I get a result, I multiple by the standard deviation, and add the mean.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This way, I get reasonable results &lt;/p&gt;&#xA;" OwnerUserId="35739" LastActivityDate="2020-04-06T11:10:15.850" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20037" PostTypeId="1" CreationDate="2020-04-06T12:01:39.920" Score="1" ViewCount="65" Body="&lt;p&gt;I have been working on my own AI for a while now, trying to implemented SGD with momentum from scratch in python. After looking around and studying all the maths behind it, i finally managed to implement SGD in a neural network that i trained to recognize the classic MNIST digits dataset.&#xA;As activation function i always used sigmoid for both hidden and output neurons, and everything seems to work more or less ok, but now i wanted to step it up a bit and try to let SGD operate with different activations, so i added 2 other  functions to my code: relu and tanh.&#xA;The behaviours that i expected based on articles, documentation and &quot;tutorials&quot; found online were:&lt;br&gt;&#xA;&lt;strong&gt;tanh&lt;/strong&gt;: Should be slightly better than sigmoid&lt;br&gt;&#xA;&lt;strong&gt;relu&lt;/strong&gt;: should be much better than sigmoid and tanh&lt;br&gt;&#xA;(By better i mean faster or at least higher accuracy the the end, or a mix of both)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using tanh it looks like it's much slower converging to a minimum compared to sigmoid&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using relu...well, the results were very, VERY horrible&#xA;Here's the outputs with the different activations (Learning rate: 0.1, Epochs: 5, MiniBatch size: 10, Momentum: 0.9)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sigmoid training&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;[Sigmoid for hidden layers, sigmoid for output layer]&#xA;Epoch: 1/5 (14.3271 s): Loss: 0.0685, Accuracy: 0.6231, Learning rate: 0.10000&#xA;Epoch: 2/5 (14.0060 s): Loss: 0.0503, Accuracy: 0.6281, Learning rate: 0.10000&#xA;Epoch: 3/5 (14.0081 s): Loss: 0.0482, Accuracy: 0.6382, Learning rate: 0.10000&#xA;Epoch: 4/5 (13.8516 s): Loss: 0.0471, Accuracy: 0.7085, Learning rate: 0.10000&#xA;Epoch: 5/5 (13.9411 s): Loss: 0.0374, Accuracy: 0.7990, Learning rate: 0.10000&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tanh training&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;[Tanh for hidden layers, sigmoid for output layer]&#xA;Epoch: 1/5 (13.7553 s): Loss: 0.3708, Accuracy: 0.4171, Learning rate: 0.10000&#xA;Epoch: 2/5 (13.7666 s): Loss: 0.2580, Accuracy: 0.4623, Learning rate: 0.10000&#xA;Epoch: 3/5 (13.5550 s): Loss: 0.2289, Accuracy: 0.4824, Learning rate: 0.10000&#xA;Epoch: 4/5 (13.7311 s): Loss: 0.2211, Accuracy: 0.5729, Learning rate: 0.10000&#xA;Epoch: 5/5 (13.6996 s): Loss: 0.2142, Accuracy: 0.5779, Learning rate: 0.10000&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Relu training&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;[Relu for hidden layers, sigmoid for output layer]&#xA;Epoch: 1/5 (14.2100 s): Loss: 0.7725, Accuracy: 0.0854, Learning rate: 0.10000&#xA;Epoch: 2/5 (14.6218 s): Loss: 0.1000, Accuracy: 0.0854, Learning rate: 0.10000&#xA;Epoch: 3/5 (14.2116 s): Loss: 0.1000, Accuracy: 0.0854, Learning rate: 0.10000&#xA;Epoch: 4/5 (14.1657 s): Loss: 0.1000, Accuracy: 0.0854, Learning rate: 0.10000&#xA;Epoch: 5/5 (14.1427 s): Loss: 0.1000, Accuracy: 0.0854, Learning rate: 0.10000&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another run with relu&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;Epoch: 1/5 (14.7391 s): Loss: 15.4055, Accuracy: 0.1658, Learning rate: 0.10000&#xA;Epoch: 2/5 (14.8203 s): Loss: 59.2707, Accuracy: 0.1709, Learning rate: 0.10000&#xA;Epoch: 3/5 (15.3785 s): Loss: 166.1310, Accuracy: 0.1407, Learning rate: 0.10000&#xA;Epoch: 4/5 (14.9285 s): Loss: 109.9386, Accuracy: 0.1859, Learning rate: 0.10000&#xA;Epoch: 5/5 (15.1280 s): Loss: 158.9268, Accuracy: 0.1859, Learning rate: 0.10000&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For these examples the epochs are just 5 but incrementing the epochs the results dont change, tanh and relu for me perform worse than sigmoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my python code reference for SGD:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pastebin.com/Nhp2vCg6&quot; rel=&quot;nofollow noreferrer&quot;&gt;SGD with momentum&lt;/a&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method was created to accept different activation functions to dynamically use them when creating the neural network object&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The activation functions and their derivatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pastebin.com/YjhAKfVd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Activation functions and derivatives&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss function i used is the mean squared error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;def mean_squared(output, expected_result):&#xA;    return numpy.sum((output - expected_result) ** 2) / expected_result.shape[0]&#xA;&#xA;&#xA;def mean_squared_derivative(output, expected_result):&#xA;    return output - expected_result&#xA;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there some concept i am missing? Am i using the activation functions the wrong way? I really cannot find the answer to this even after searching for a long time.&#xA;I feel like the problem is somewhere in the backpropagation but i can't find it.&#xA;Any kind of help would be greatly appriciated&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I hope i posted this in the right place, i am pretty new to asking questions here, so if there is any problem i will move the question somewhere else&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I tried to implement this with tensorflow, using relu for hidden layers and sigmoid for output. The results i get with this implementation are the same as the ones i mentioned in my question, so unless i am doing something wrong in both situations i am left to think i cannot use relu with sigmoid, which makes sense cause relu can have very high values while sigmoid pushes them down between 0 and 1, therefore most of the times giving values very close to 1.&lt;br&gt;&#xA;Code reference:&lt;br&gt;&#xA;&lt;a href=&quot;https://pastebin.com/CiaNat0j&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow implementation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35792" LastEditorUserId="35792" LastEditDate="2020-04-09T07:51:43.177" LastActivityDate="2020-04-09T07:51:43.177" Title="Stochastic gradient descent does not behave as expected, even with different activation functions" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20038" PostTypeId="1" CreationDate="2020-04-06T12:35:49.127" Score="1" ViewCount="19" Body="&lt;p&gt;This a fix example to run in gym open ai&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(gym)&#xA;&#xA;remote_base &amp;lt;- &quot;http://127.0.0.1:5000&quot;&#xA;client &amp;lt;- create_GymClient(remote_base)&#xA;&#xA;# Create environment&#xA;env_id &amp;lt;- &quot;CartPole-v0&quot;&#xA;instance_id &amp;lt;- env_create(client, env_id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How is it possible to create as input a graph with specific number of nodes (i.e. 7) and the characteristics of graph are a fully connected undirect graph with different weights in edges?&lt;/p&gt;&#xA;" OwnerUserId="35779" LastActivityDate="2020-04-06T12:35:49.127" Title="Running a simple graph network example in gym" Tags="&lt;reinforcement-learning&gt;&lt;gym&gt;&lt;r&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20039" PostTypeId="2" ParentId="20035" CreationDate="2020-04-06T12:51:04.937" Score="2" Body="&lt;p&gt;Image to Image translation is the task of transferring an image's characteristics from one domain and representing it in another. GANs have provided an end to end method to do this task.&#xA;Prior to Gans, these tasks were done individually, by using classic image processing techniques mainly.&#xA;Techniques such as image denoising, or finding edges in photos, or using web results to join various images were used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned by P Isola et. al., in the &lt;a href=&quot;https://arxiv.org/abs/1703.10593&quot; rel=&quot;nofollow noreferrer&quot;&gt;CycleGAN&lt;/a&gt; paper, Hertzmann et al. in their paper, &lt;a href=&quot;https://mrl.nyu.edu/publications/image-analogies/analogies-72dpi.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Analogies&lt;/a&gt;, have employed a non-parametric texture model on a single input-output training image pair. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://people.eecs.berkeley.edu/~efros/research/quilting/quilting.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Quilting for Texture Synthesis and Transfer&lt;/a&gt;, the authors used existing patches of images and stitched them together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://people.csail.mit.edu/yichangshih/time_lapse/time_lapse.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data-driven Hallucination of Different Times of Day from a Single Outdoor Photo&lt;/a&gt;, here the authors compare the input image with an available dataset of time-lapse videos similar to the input and find the frame at the time of input image and frame for a target time. They use local affine transformation to change the scene of an input image into target image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent works have focused on using a dataset of paired input-output examples to learn the translation function using CNNs for semantic segmentation,&#xA;&lt;a href=&quot;https://arxiv.org/abs/1411.4038&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fully convolutional networks for semantic segmentation&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="35791" LastActivityDate="2020-04-06T12:51:04.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20040" PostTypeId="1" AcceptedAnswerId="20043" CreationDate="2020-04-06T12:58:15.397" Score="3" ViewCount="563" Body="&lt;p&gt;The paper &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30629091&quot; rel=&quot;nofollow noreferrer&quot;&gt;Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration&lt;/a&gt; uses the term &quot;proxy data sets&quot; in this way&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To develop DL techniques for synthesizing high-resolution realistic fundus images serving as &lt;strong&gt;proxy data sets&lt;/strong&gt; for use by retinal specialists and DL machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I googled that term, but didn't find a definition of &quot;proxy data sets&quot;. What are &quot;proxy data sets&quot; in machine learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://www.researchgate.net/publication/232657161_Analysis_of_Manufacturing_Process_Sequences_Using_Machine_Learning_on_Intermediate_Product_States_as_Process_Proxy_Data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis of Manufacturing Process Sequences, Using Machine Learning on Intermediate Product States (as Process Proxy Data)&lt;/a&gt; mentions a similar term&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The advantage of the product state-based view is the focus on the product itself to structure the information and data involved throughout the process. Using the intermediate product states as &lt;strong&gt;proxy data&lt;/strong&gt; for this purpose&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does &quot;proxy data&quot; mean the same thing as &quot;proxy data sets&quot; does?&lt;/p&gt;&#xA;" OwnerUserId="35782" LastEditorUserId="2444" LastEditDate="2020-04-07T19:28:48.620" LastActivityDate="2020-04-07T19:28:48.620" Title="What are &quot;proxy data sets&quot; in machine learning?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20042" PostTypeId="1" CreationDate="2020-04-06T14:01:08.770" Score="1" ViewCount="27" Body="&lt;p&gt;How I can identify holes in a 3D CAD file?  I want to identify different types of holes, counterbored or countersunk holes. My program lets me extract, for example, the faces and adjacency of the faces.&#xA;I am talking about the Siemens NX for example. &#xA;The different types of holes you can see there:&#xA;&lt;a href=&quot;https://www.google.com/url?sa=i&amp;amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_VwwfDrdggc&amp;amp;psig=AOvVaw1kIYdOt2qxazSYFXwHuXpb&amp;amp;ust=1586329824363000&amp;amp;source=images&amp;amp;cd=vfe&amp;amp;ved=0CAIQjRxqFwoTCOjd1qHh1egCFQAAAAAdAAAAABAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.google.com/url?sa=i&amp;amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_VwwfDrdggc&amp;amp;psig=AOvVaw1kIYdOt2qxazSYFXwHuXpb&amp;amp;ust=1586329824363000&amp;amp;source=images&amp;amp;cd=vfe&amp;amp;ved=0CAIQjRxqFwoTCOjd1qHh1egCFQAAAAAdAAAAABAJ&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35799" LastEditorUserId="35799" LastEditDate="2020-04-07T07:11:41.100" LastActivityDate="2020-04-07T07:11:41.100" Title="How I can identify holes in a 3D CAD file?" Tags="&lt;machine-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20043" PostTypeId="2" ParentId="20040" CreationDate="2020-04-06T14:10:20.720" Score="4" Body="&lt;p&gt;In computer science, if you say &amp;quot;A is a proxy for B&amp;quot;, then it means that &amp;quot;A replaces B&amp;quot; (temporarily or not), or that &amp;quot;A is used as an intermediary for B&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The term &amp;quot;proxy&amp;quot; usually refers to a server, i.e. there are the so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Proxy_server&quot; rel=&quot;nofollow noreferrer&quot;&gt;proxy servers&lt;/a&gt;, which intuitively do the same thing (i.e. they are used as intermediaries). The following picture is worth more than 1000 words.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SPDo7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SPDo7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's go back to your context. In the following paragraph&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To develop DL techniques for synthesizing high-resolution realistic fundus images serving as proxy data sets for use by retinal specialists and DL machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The deep learning techniques will be used to generate data that will be used as a replacement for real data for use by retinal specialists.&lt;/p&gt;&#xA;&lt;p&gt;In your second paragraph&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The advantage of the product state-based view is the focus on the product itself to structure the information and data involved throughout the process. Using the intermediate product states as proxy data for this purpose&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The word &amp;quot;intermediate&amp;quot; is even used, so there should not be any need for further explanation, although I admit that I have no idea of what &amp;quot;product states&amp;quot; are in this context, but they will be used as &amp;quot;proxy data&amp;quot; for other data.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does &amp;quot;proxy data&amp;quot; mean the same thing as &amp;quot;proxy data sets&amp;quot; does, if yes, I will go through that paper.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I guess so, but bear in mind that &amp;quot;data&amp;quot; is more general than &amp;quot;data set&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I have found other slightly different uses of the expression &amp;quot;proxy data&amp;quot;, not in the context of computer science, but in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Proxy_(climate)&quot; rel=&quot;nofollow noreferrer&quot;&gt;paleoclimatology&lt;/a&gt;. For example, have a look at this web article &lt;a href=&quot;https://www.ncei.noaa.gov/news/what-are-proxy-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Are Proxy Data?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Just for completeness, here are &lt;a href=&quot;https://dictionary.cambridge.org/dictionary/english/proxy&quot; rel=&quot;nofollow noreferrer&quot;&gt;some dictionary definitions of proxy&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;authority given to a person to act for someone else, such as by voting for them in an election&lt;/p&gt;&#xA;&lt;p&gt;a person who you choose to do something officially for you&lt;/p&gt;&#xA;&lt;p&gt;a situation, process, or activity to which another situation, etc. is compared&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-06T14:10:20.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20044" PostTypeId="1" AcceptedAnswerId="20046" CreationDate="2020-04-06T14:50:38.077" Score="2" ViewCount="220" Body="&lt;p&gt;While looking at the mathematics of the back-propagation algorithm for a multi-layer perceptron, I noticed that in order to find the partial derivative of the cost function with respect to a weight (say &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;) from any of the hidden layers, we're just writing the error function from the final outputs in terms of the inputs and hidden layer weights and then canceling all the terms without &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; in it as differentiating those terms with respect to &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; would give zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where is the back-propagation of error while doing this? This way, I can find the partial derivatives of the first hidden layer first and then go towards the other ones if I wanted to. Is there some other method of going about it so that the Back Propagation concept comes into play? Also, I'm looking for a general method/algorithm, not just for 1-2 hidden layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm fairly new to this and I'm just following what's being taught in class. Nothing I found on the internet seems to have proper notation so I can't understand what they're saying.&lt;/p&gt;&#xA;" OwnerUserId="35800" LastEditorUserId="2444" LastEditDate="2020-04-06T16:26:50.003" LastActivityDate="2020-04-06T20:57:27.447" Title="Why is it called back-propagation?" Tags="&lt;terminology&gt;&lt;backpropagation&gt;&lt;history&gt;&lt;multilayer-perceptron&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20045" PostTypeId="1" CreationDate="2020-04-06T14:59:11.603" Score="1" ViewCount="19" Body="&lt;p&gt;I am looking into exponentially damped signals where it is a stationary signal (after implementing the Adfuller statistical test) and I would like to look into how can I extract meaningful features out of the signal in order to do pattern recognition with machine learning. Can anyone guide me on where I can find articles/blogs of signal processing techniques and feature extraction of exponentially damped signal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My situation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to look into features that relate to damping of the signal, I already looked at it in the frequency domain and I found out that from my datasets (considering the first 3 Natural frequencies/modes) the peaks are almost the same (there's like the same deviation by only like [+] or [-] 0.5 from freq. values). Looking into the damping factor, I found out that only the second damping ratio was different but still small deviation around the same ([+] or [-] 0.5). So, I thought that it would be difficult for machine learning to identify the difference between cases. One of my ideas is to look into energy dissipation as it might be related to damping, but I don't know how to approach it or from which domain I need to go in order to get the features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Side Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have several questions regarding signal processing:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Say I have a signal and would like to extract features from it,&#xA;what steps or points that I should know in order to implement signal processing? (As I am using Python).&lt;/li&gt;&#xA;&lt;li&gt;When I used signaltonoise function online (python) in order to see&#xA;the signal-to-noise ratio and I got a positive SNR. However, if I&#xA;pass the signal into, for example, a band-pass filter to concentrate on a certain frequency band I get a negative SNR. Why is that?&lt;/li&gt;&#xA;&lt;li&gt;How can I extract features from STFT? And I also know about wavelet and HHT, what are the uses of both algorithms and how to also extract features from it?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="35795" LastActivityDate="2020-04-06T14:59:11.603" Title="Feature extraction for exponentially damped signals" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;feature-selection&gt;&lt;signal-processing&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20046" PostTypeId="2" ParentId="20044" CreationDate="2020-04-06T15:18:59.327" Score="1" Body="&lt;p&gt;Have a look at the following article &lt;a href=&quot;http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principles of training multi-layer neural network using backpropagation&lt;/a&gt;. It was very useful to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also see &lt;a href=&quot;https://ai.stackexchange.com/a/19945/2444&quot;&gt;here an example of backpropagation in Matlab&lt;/a&gt;. It effectively solves the &lt;a href=&quot;https://en.wikipedia.org/wiki/Exclusive_or&quot; rel=&quot;nofollow noreferrer&quot;&gt;XOR problem&lt;/a&gt;. You can also play around with the cost function or the learning rate. You may get surprising results! Does this answer your question?&lt;/p&gt;&#xA;" OwnerUserId="35660" LastEditorUserId="35660" LastEditDate="2020-04-06T16:34:09.193" LastActivityDate="2020-04-06T16:34:09.193" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20048" PostTypeId="2" ParentId="20044" CreationDate="2020-04-06T17:04:59.470" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is it called back-propagation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think there is anything special here! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's called back-propagation (BP) because, after the forward pass, you compute the partial derivative of the loss function with respect to the parameters of the network, which, in the usual diagrams of a neural network, are placed &lt;strong&gt;before&lt;/strong&gt; the output of the network (i.e. to the left of the output if the output of the network is on the right, or to the right if the output of the network is on the left).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also called BP because it is just the application of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;chain rule&lt;/a&gt;. Why is this interesting? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me answer this question with an example. Consider the function &lt;span class=&quot;math-container&quot;&gt;$y=e^{\sin(x^{2})}$&lt;/span&gt;. This is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Function_composition&quot; rel=&quot;nofollow noreferrer&quot;&gt;composite function&lt;/a&gt;, i.e. a function composed of multiple simpler functions, which, in this case, are &lt;span class=&quot;math-container&quot;&gt;$e^x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\sin(x)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. To compute the derivative of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, let's define the following variables&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;y &amp;amp;= f(u) = e^u,\\&#xA;u &amp;amp;= g(v) = \sin v = \sin(x^2),\\&#xA;v &amp;amp;= h(x) = x^2&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; with respect to the variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is (according to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;chain rule&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\underset{\color{red}{\LARGE \rightarrow}}{&#xA;\frac{dy}{dx} = \frac{dy}{du} \color{green}{\cdot} \frac{du}{dv}  \color{green}{\cdot} &#xA;\frac{dv}{dx}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you read this equation from the left to the right, you can see that we are going backward (i.e. from the function &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to the function &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;). This is the same thing with BP!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is it called &quot;chain rule&quot;? Because you are &lt;a href=&quot;https://en.wikipedia.org/wiki/Chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;chaining&lt;/a&gt; different partial derivatives. More specifically, you are multiplying them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BP is also known as the &lt;strong&gt;reverse mode of automatic differentiation&lt;/strong&gt;. Why? The &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_differentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic differentiation&lt;/a&gt; should be self-explanatory, given that the BP algorithm is just the computation of partial derivatives, and you do this &lt;em&gt;automatically&lt;/em&gt;, i.e. with a program, rather than by hand. The expression &quot;reverse mode&quot; refers to the fact that we compute the derivatives from the outer function (which, in the example above, is &lt;span class=&quot;math-container&quot;&gt;$e^x$&lt;/span&gt;) to the inner function (which, in the example above, is &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;). &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_differentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Wikipedia article related to automatic differentiation&lt;/a&gt; provides more details.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What exactly are you back-propagating? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The partial derivative of the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; with respect to a parameter &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \mathcal{L}}{\partial w_i}$&lt;/span&gt;, intuitively, represents the &quot;contribution&quot; of the parameter &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; to the loss. After having computed these partial derivatives (i.e. the gradient), you use gradient descent to update each parameter &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_i \leftarrow w_i - \gamma \frac{\partial \mathcal{L}}{\partial w_i}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \mathcal{L}}{\partial w_i}$&lt;/span&gt; represents WHAT we propagatED, which is the error (or loss) that the neural network makes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gradient descent step will hopefully make your network produce a smaller error next time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The modern version of back-propagation was published (in 1970) by a Finnish master's student called Seppo Linnainmaa, although some people claim this is not the case because they want to give credits to others (because Seppo Linnainmaa didn't reference neural networks, but back-propagation isn't only applicable to neural networks) or they don't even know the history. &lt;a href=&quot;http://people.idsia.ch/~juergen/who-invented-backpropagation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This Jürgen Schmidhuber's article goes into the details of the history of BP&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="9947" LastEditDate="2020-04-06T20:57:27.447" LastActivityDate="2020-04-06T20:57:27.447" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20049" PostTypeId="2" ParentId="20034" CreationDate="2020-04-06T17:58:50.487" Score="5" Body="&lt;p&gt;The difference between the validation and test set in my opinion should be explained in this way:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the validation set is meant to be used multiple times.&lt;/li&gt;&#xA;&lt;li&gt;the test set is meant to be used only once. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think that the misunderstanding here arise because machine learning is mostly taught focusing only on a specific part of a large pipeline, which is the model training. In every tutorial standard datasets are used so that you don't have to worry about data collection, data labelling (it's really sad to see that lot of people have not a clue about what the inter annotator agreement is), data pre-processing and especially, all the part about the real application of the model is almost never mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The importance of having a set of instances that you can use for fine tuning (validation) and a set of instances that your model never encountered neither in training nor during fine tuning (test) becomes particularly clear if you focus on the subsequent deployment of the model you trained. No one expect a model to have the same performance scores in training and when applied to some unknown data. And the crucial point is that the performance of a model on the validation set are not representative either of the behaviour of a model with unknown data, because the same validation data have been used to fine tuned the model! So here's why having a set of data completely new to the model is important, because it gives you a much more unbiased view about the model performance on a real use case scenario. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-04-06T17:58:50.487" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20050" PostTypeId="2" ParentId="20034" CreationDate="2020-04-06T18:18:47.293" Score="2" Body="&lt;p&gt;Simply stated, you use your validation set to regularize your model for unseen data.&#xA;Test data is completely unseen data, on which you evaluate your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Various validation strategies are used to improve your model to perform for unseen data. So strategies like k-fold cross-validation are used.&#xA;Also, the validation set helps you in tuning your hyperparameters such as learning rate, batch size, hidden units, number of layers, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Train, Validation, Test sets help you in identifying whether you are underfitting or overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. If human error at a task is 1%, train error is 8%, validation error is 10%, test set error is 12 % then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Difference between, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Human level and training set error tells you about &quot;Avoidable Bias&quot;&lt;/li&gt;&#xA;&lt;li&gt;Training set error and Validation set error tells you about &quot;Variance and data mismatch&quot;&lt;/li&gt;&#xA;&lt;li&gt;Validation set error and Test error tells you about &quot;degree of overfitting&quot; with the validation set.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Based on these metrics, you can apply appropriate strategies for better performance on validation or test sets.&lt;/p&gt;&#xA;" OwnerUserId="35791" LastActivityDate="2020-04-06T18:18:47.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20051" PostTypeId="2" ParentId="19957" CreationDate="2020-04-06T18:45:39.673" Score="1" Body="&lt;p&gt;The derivative &lt;span class=&quot;math-container&quot;&gt;$f'(x)$&lt;/span&gt; is correlated with &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; &lt;em&gt;in a certain sense&lt;/em&gt;. In fact, &lt;span class=&quot;math-container&quot;&gt;$f'(x)$&lt;/span&gt; is a function of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, so we could even say that there's a cause-effect relationship.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative at a specific point &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; of the domain, i.e. &lt;span class=&quot;math-container&quot;&gt;$f'(c)$&lt;/span&gt;, can either be negative or positive. If &lt;span class=&quot;math-container&quot;&gt;$f'(c) &amp;gt; 0$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f(c)$&lt;/span&gt; is increasing (with respect to an increase of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;). If &lt;span class=&quot;math-container&quot;&gt;$f'(c) &amp;lt; 0$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f(c)$&lt;/span&gt; decreasing  (with respect to an increase of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can easily be seen from an example. Consider &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f'(x) = 2x$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$c = 2$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f'(2) = 4$&lt;/span&gt;, so the function is increasing. In fact, &lt;span class=&quot;math-container&quot;&gt;$f(1) = 2 \leq f(2) = 4 \leq f(3) = 9$&lt;/span&gt;. Similarly, let &lt;span class=&quot;math-container&quot;&gt;$c = -1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f'(-1) = -2$&lt;/span&gt;, so the function is decreasing. In fact, &lt;span class=&quot;math-container&quot;&gt;$\leq f(-2) = 4 \geq f(-1) = 1 \geq  f(0) = 0$&lt;/span&gt; (note that the function is decreasing as we increase &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;!). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a model with only one parameter, then the partial derivative of the loss function with respect to that parameter corresponds to the derivative of the loss function. So, the reasoning above applies to this model. What about a model with more than one parameter? The same thing happens. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If the function decreases, does its derivative also decrease?&lt;/em&gt; In general, no, and this can easily be seen from a plot of a function and its derivative. For example, consider a plot of a parabola and its derivative (which is a linear function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6xmWu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6xmWu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the left of the y-axis, the parabola is decreasing, but its derivative is increasing, while, on the right of the y-axis, the parabola is increasing and the linear function is still increasing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the same thing with a loss function of an ML model and its partial derivative.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-06T18:45:39.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20052" PostTypeId="1" CreationDate="2020-04-06T19:42:34.823" Score="2" ViewCount="48" Body="&lt;p&gt;I'm watching the video &lt;a href=&quot;https://www.youtube.com/watch?v=y7qrilE-Zlc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorflow Tutorial | Edureka&lt;/a&gt; where the author says that the LSTM and GRU architecture help to reduce the vanishing gradient problem. How do LSTM and GRU prevent the vanishing gradient problem?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-04-06T19:45:39.990" LastActivityDate="2021-05-25T10:00:37.983" Title="How do LSTM and GRU avoid to overcome the vanishing gradient problem?" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20053" PostTypeId="1" AcceptedAnswerId="20108" CreationDate="2020-04-06T20:20:41.007" Score="4" ViewCount="144" Body="&lt;p&gt;I'm developing my first neural network, using the well known MNIST database of handwritten digit. I want the NN to be able to classify a number from 0 to 9 given an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My neural network consists of three layers: the input layer (784 neurons, each one for every pixel of the digit), a hidden layer of 30 neurons (it could also be 100 or 50, but I'm not too worried about hyperparameter tuning yet), and the output layer, 10 neurons, each one representing the activation for every digit. That gives to me two weight matrices: one of 30x724 and a second one of 10x30.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know and understand the theory behind back propagation, optimization and the mathematical formulas behind that, that's not a problem as such. I can optimize the weights for the second matrix of weights, and the cost is indeed being reduced over time. But I'm not able to keep propagating that back because of the matrix structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing that I have find the derivative of the cost w.r.t. the weights:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d(cost) / d(w) = d(cost) / d(f(z)) * d(f(z)) / d(z) * d(z) / d(w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Being &lt;code&gt;f&lt;/code&gt; the activation function and &lt;code&gt;z&lt;/code&gt; the dot product plus the bias of a neuron)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm in the rightmost layer, with an output array of 10 elements. &lt;code&gt;d(cost) / d(f(z))&lt;/code&gt; is the subtraction of the observed an predicted values. I can multiply that by &lt;code&gt;d(f(z)) / d(z)&lt;/code&gt;, which is just &lt;code&gt;f'(z)&lt;/code&gt; of the rightmost layer, also an unidimensional vector of 10 elements, having now d(cost) / d(z) calculated. Then, &lt;code&gt;d(z)/d(w)&lt;/code&gt; is just the input to that layer, i.e. the output of the previous one, which is a vector of 30 elements. I figured that I can transpose d(cost) / &lt;code&gt;d(z)&lt;/code&gt; so that &lt;code&gt;T( d(cost) / d(z) ) * d(z) / d(w)&lt;/code&gt; gives me a matrix of (10, 30), which makes sense because it matches with the dimension of the rightmost weight matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then I get stuck. The dimension of &lt;code&gt;d(cost) / d(f(z))&lt;/code&gt; is (1, 10), for &lt;code&gt;d(f(z)) / d(z)&lt;/code&gt; is (1, 30) and for &lt;code&gt;d(z) / d(w)&lt;/code&gt; is (1, 784). I don't know how to come up with a result for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I've coded so far. The incomplete part is the &lt;code&gt;_propagate_back&lt;/code&gt; method. I'm not caring about the biases yet because I'm just stuck with the weights and first I want to figure this out. &lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import random&#xA;from typing import List, Tuple&#xA;&#xA;import numpy as np&#xA;from matplotlib import pyplot as plt&#xA;&#xA;import mnist_loader&#xA;&#xA;np.random.seed(42)&#xA;&#xA;NETWORK_LAYER_SIZES = [784, 30, 10]&#xA;LEARNING_RATE = 0.05&#xA;BATCH_SIZE = 20&#xA;NUMBER_OF_EPOCHS = 5000&#xA;&#xA;&#xA;def sigmoid(x):&#xA;    return 1 / (1 + np.exp(-x))&#xA;&#xA;&#xA;def sigmoid_der(x):&#xA;    return sigmoid(x) * (1 - sigmoid(x))&#xA;&#xA;&#xA;class Layer:&#xA;&#xA;    def __init__(self, input_size: int, output_size: int):&#xA;        self.weights = np.random.uniform(-1, 1, [output_size, input_size])&#xA;        self.biases = np.random.uniform(-1, 1, [output_size])&#xA;        self.z = np.zeros(output_size)&#xA;        self.a = np.zeros(output_size)&#xA;        self.dz = np.zeros(output_size)&#xA;&#xA;    def feed_forward(self, input_data: np.ndarray):&#xA;        input_data_t = np.atleast_2d(input_data).T&#xA;        dot_product = self.weights.dot(input_data_t).T[0]&#xA;        self.z = dot_product + self.biases&#xA;        self.a = sigmoid(self.z)&#xA;        self.dz = sigmoid_der(self.z)&#xA;&#xA;&#xA;class Network:&#xA;&#xA;    def __init__(self, layer_sizes: List[int], X_train: np.ndarray, y_train: np.ndarray):&#xA;        self.layers = [&#xA;            Layer(input_size, output_size)&#xA;            for input_size, output_size&#xA;            in zip(layer_sizes[0:], layer_sizes[1:])&#xA;        ]&#xA;        self.X_train = X_train&#xA;        self.y_train = y_train&#xA;&#xA;    @property&#xA;    def predicted(self) -&amp;gt; np.ndarray:&#xA;        return self.layers[-1].a&#xA;&#xA;    def _normalize_y(self, y: int) -&amp;gt; np.ndarray:&#xA;        output_layer_size = len(self.predicted)&#xA;        normalized_y = np.zeros(output_layer_size)&#xA;        normalized_y[y] = 1.&#xA;&#xA;        return normalized_y&#xA;&#xA;    def _calculate_cost(self, y_observed: np.ndarray) -&amp;gt; int:&#xA;        y_observed = self._normalize_y(y_observed)&#xA;        y_predicted = self.layers[-1].a&#xA;&#xA;        squared_difference = (y_predicted - y_observed) ** 2&#xA;&#xA;        return np.sum(squared_difference)&#xA;&#xA;    def _get_training_batches(self, X_train: np.ndarray, y_train: np.ndarray) -&amp;gt; Tuple[np.ndarray, np.ndarray]:&#xA;        train_batch_indexes = random.sample(range(len(X_train)), BATCH_SIZE)&#xA;&#xA;        return X_train[train_batch_indexes], y_train[train_batch_indexes]&#xA;&#xA;    def _feed_forward(self, input_data: np.ndarray):&#xA;        for layer in self.layers:&#xA;            layer.feed_forward(input_data)&#xA;            input_data = layer.a&#xA;&#xA;    def _propagate_back(self, X: np.ndarray, y_observed: int):&#xA;        &quot;&quot;&quot;&#xA;        der(cost) / der(weight) = der(cost) / der(predicted) * der(predicted) / der(z) * der(z) / der(weight)&#xA;        &quot;&quot;&quot;&#xA;        y_observed = self._normalize_y(y_observed)&#xA;        d_cost_d_pred = self.predicted - y_observed&#xA;&#xA;        hidden_layer = self.layers[0]&#xA;        output_layer = self.layers[1]&#xA;&#xA;        # Output layer weights&#xA;        d_pred_d_z = output_layer.dz&#xA;        d_z_d_weight = hidden_layer.a  # Input to the current layer, i.e. the output from the previous one&#xA;&#xA;        d_cost_d_z = d_cost_d_pred * d_pred_d_z&#xA;        d_cost_d_weight = np.atleast_2d(d_cost_d_z).T * np.atleast_2d(d_z_d_weight)&#xA;&#xA;        output_layer.weights -= LEARNING_RATE * d_cost_d_weight&#xA;&#xA;        # Hidden layer weights&#xA;        d_pred_d_z = hidden_layer.dz&#xA;        d_z_d_weight = X&#xA;&#xA;        # ...&#xA;&#xA;    def train(self, X_train: np.ndarray, y_train: np.ndarray):&#xA;        X_train_batch, y_train_batch = self._get_training_batches(X_train, y_train)&#xA;        cost_over_epoch = []&#xA;&#xA;        for epoch_number in range(NUMBER_OF_EPOCHS):&#xA;            X_train_batch, y_train_batch = self._get_training_batches(X_train, y_train)&#xA;&#xA;            cost = 0&#xA;            for X_sample, y_observed in zip(X_train_batch, y_train_batch):&#xA;                self._feed_forward(X_sample)&#xA;                cost += self._calculate_cost(y_observed)&#xA;                self._propagate_back(X_sample, y_observed)&#xA;&#xA;            cost_over_epoch.append(cost / BATCH_SIZE)&#xA;&#xA;        plt.plot(cost_over_epoch)&#xA;        plt.ylabel('Cost')&#xA;        plt.xlabel('Epoch')&#xA;        plt.savefig('cost_over_epoch.png')&#xA;&#xA;&#xA;training_data, validation_data, test_data = mnist_loader.load_data()&#xA;X_train, y_train = training_data[0], training_data[1]&#xA;&#xA;network = Network(NETWORK_LAYER_SIZES, training_data[0], training_data[1])&#xA;network.train(X_train, y_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the code for &lt;code&gt;mnist_loader&lt;/code&gt;, in case someone wanted to reproduce the example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pickle&#xA;import gzip&#xA;&#xA;&#xA;def load_data():&#xA;    f = gzip.open('data/mnist.pkl.gz', 'rb')&#xA;    training_data, validation_data, test_data = pickle.load(f, encoding='latin-1')&#xA;    f.close()&#xA;&#xA;    return training_data, validation_data, test_data&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35806" LastEditorUserId="35806" LastEditDate="2020-04-07T17:06:14.247" LastActivityDate="2020-04-09T11:36:09.210" Title="How to perform back propagation with different sized layers?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20054" PostTypeId="1" CreationDate="2020-04-06T21:39:35.057" Score="2" ViewCount="89" Body="&lt;p&gt;From what I've figured&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) converting mathematical theorems and proofs from English to formal logic is a straightforward job for mathematicians with sufficient background, except that it takes time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) once converted to formal logic, computer verification of the proof becomes straightforward.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we can automate (a), a lot of time and intellectual labour (that could be dedicated elsewhere) is saved in doing (b) on published research papers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if solving (a) in its entirety is hard, we could expect the mathematicians to meet the computer system halfway and avoid writing lengthy English paras that are hard to convert. If it becomes doable enough, submitting a formal logical version of your paper could even become a standard procedure that is expected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional benefit of solving (a) would be to do the process in reverse: mathematicians could delegate smaller tasks and lemmas (both trivial and non-trivial tasks) to an automated theorem prover (ATP). Assisted theorem proving will become more popular and boost productivity, maybe even surprise us once in a while by coming up with proofs that the paper writer couldn't. This is further of value if we predict a sharp upward trajectory of the capability of ATPs in the future. If anything, this could be self-fulfilling, as the demonstration of potential for good ATPs combined by a large corpus of proofs and problems in formal logical format could drive an increase in research on ATPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forgive me if I sound like a salesman, but how doable is this? &lt;strong&gt;What will be the main challenges faced in developing NLP-based AI to convert papers, and how tractable are these challenges given today's state of the field?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.s. I understand that proofs generated by ATPs are often hard to understand intuitively and can end up proving results without clearly exposing the underlying proof method used. But it is still a benefit to be able to use the final results&lt;/p&gt;&#xA;" OwnerUserId="35810" LastEditorUserId="35810" LastEditDate="2020-04-07T08:13:39.370" LastActivityDate="2020-04-19T17:59:09.820" Title="What are the challenges faced by using NLP to convert mathematical texts into formal logic?" Tags="&lt;natural-language-processing&gt;&lt;math&gt;&lt;logic&gt;&lt;natural-language-understanding&gt;&lt;automated-theorem-proving&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20055" PostTypeId="1" AcceptedAnswerId="20064" CreationDate="2020-04-06T21:46:15.407" Score="2" ViewCount="52" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9YIDb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9YIDb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a picture of a recurrent neural network (RNN) found on a udemy course (Deep Learning A-Z). The axis at the bottom is &quot;time&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a time series problem, each yellow row from left to right would represent a sequence of a feature. In this picture, then, there are 6 sequences from 6 different features that are being fed to the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if the arrows in this picture are completely accurate in an RNN. Shouldn't every yellow node also connect to every other blue node along its depth dimension? By depth dimension here I mean the third dimensional axis of the input tensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the yellow node at the bottom left of this picture, which is closest to the viewer, should have an arrow pointing to all the blue nodes in the array of blue nodes that is at the very left, and not just to the blue node directly above it.&lt;/p&gt;&#xA;" OwnerUserId="35809" LastEditorUserId="35809" LastEditDate="2020-04-07T00:31:01.227" LastActivityDate="2020-04-07T04:49:36.697" Title="Is this a correct visual representation of a recurrent neural network (RNN)?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20056" PostTypeId="1" CreationDate="2020-04-06T21:56:11.273" Score="1" ViewCount="29" Body="&lt;p&gt;I am trying to perform binary classification of search results based on the relevance to the query.&#xA;I followed this &lt;a href=&quot;https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; on how to make an SVM, and I got it to work with a small iris dataset. Now, I am attempting to use the LETOR 4.0 MQ2007 dataset by Microsoft to classify. The dataset has 21 input vectors as well as a score from 0 to 2. I classified 0 as -1 and 1, 2 as 1. My algorithm reaches 57.4% accuracy after 1000 epochs with 500 samples of each classification. My learning rate is 0.0001. here is my code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from tqdm import tqdm&#xA;import numpy as np&#xA;from sklearn.metrics import accuracy_score&#xA;&#xA;&#xA;print(&quot;-------------------------------------&quot;)&#xA;choice = input(&quot;Train or Test: &quot;)&#xA;print(&quot;-------------------------------------&quot;)&#xA;&#xA;# HYPERPARAMETERS&#xA;feature_num = 21&#xA;epochs = 1000&#xA;sample_size = 500&#xA;learning_rate = 0.0001&#xA;&#xA;if choice == &quot;Train&quot;:&#xA;&#xA;    out_file = open('weights.txt', 'w')&#xA;    out_file.close()&#xA;&#xA;    print(&quot;Serializing Train Data...&quot;)&#xA;&#xA;    # SERIALIZE DATA&#xA;    file = open('train.txt')&#xA;    train_set = file.read().splitlines()&#xA;    positive = []&#xA;    negative = []&#xA;&#xA;    # GRAB TRAINING SAMPLES&#xA;    for i in train_set:&#xA;        if (i[0] == '1' or i[0] == '2') and len(positive) &amp;lt; sample_size:&#xA;            positive.append(i)&#xA;        if (i[0] == '0') and len(negative) &amp;lt; sample_size:&#xA;            negative.append(i)&#xA;&#xA;    train_set = positive+negative&#xA;    file.close()&#xA;&#xA;    features = []&#xA;    query = []&#xA;&#xA;    # CREATE TRAINING VECTORS&#xA;    alpha = np.full(feature_num, learning_rate)&#xA;    weights = np.zeros((len(train_set), feature_num))&#xA;    output = np.zeros((len(train_set), feature_num))&#xA;    score = np.zeros((len(train_set), feature_num))&#xA;&#xA;    for i in tqdm(range(len(train_set))):&#xA;        elements = train_set[i].split(' ')&#xA;        if int(elements[0]) == 0:&#xA;            score[i] = [-1] * feature_num&#xA;        else:&#xA;            score[i] = [1] * feature_num&#xA;&#xA;        query.append(int(elements[1].split(':')[1]))&#xA;        tmp = []&#xA;        for feature in elements[2:2+feature_num]:&#xA;            if feature.split(':')[1] == 'NULL':&#xA;                tmp.append(0.0)&#xA;            else:&#xA;                tmp.append(float(feature.split(':')[1]))&#xA;        features.append(tmp)&#xA;&#xA;    features = np.asarray(features)&#xA;&#xA;    print(&quot;-------------------------------------&quot;)&#xA;    print(&quot;Training Initialized...&quot;)&#xA;&#xA;    # TRAIN MODEL&#xA;    for i in tqdm(range(epochs)):&#xA;&#xA;        # FORWARD y = sum(wx)&#xA;        for sample in range(len(train_set)):&#xA;            output[sample] = weights[sample]*features[sample]&#xA;            output[sample] = np.full((feature_num), np.sum(output[sample]))&#xA;&#xA;        # NORMALIZE NEGATIVE SIGNS&#xA;        output = output*score&#xA;        # UPDATE WEIGHTS&#xA;        count = 0&#xA;        for val in output:&#xA;            if(val[0] &amp;gt;= 1):&#xA;                cost = 0&#xA;                weights = weights - alpha * (2 * 1/epochs * weights)&#xA;            else:&#xA;                cost = 1 - val[0]&#xA;                # WEIGHTS = WEIGHTS + LEARNING RATE * [X] * [Y]&#xA;                weights = weights + alpha * (features[count] * score[count] - 2 * 1/epochs * weights)&#xA;&#xA;            count += 1&#xA;&#xA;    # EXPORT WEIGHTS&#xA;    out_file = open('weights.txt', 'a+')&#xA;    for i in weights[0]:&#xA;        out_file.write(str(i)+'\n')&#xA;    out_file.close()&#xA;&#xA;elif choice == &quot;Test&quot;:&#xA;&#xA;    print(&quot;Serializing Test Data...&quot;)&#xA;&#xA;    # SERIALIZE DATA&#xA;    file = open('train.txt')&#xA;    train_set = file.read().splitlines()&#xA;    positive = []&#xA;    negative = []&#xA;    for i in train_set:&#xA;        if (i[0] == '1' or i[0] == '2') and len(positive) &amp;lt; sample_size:&#xA;            positive.append(i)&#xA;        if (i[0] == '0') and len(negative) &amp;lt; sample_size:&#xA;            negative.append(i)&#xA;&#xA;    test_set = positive+negative&#xA;&#xA;    file = open('weights.txt', 'r').read().splitlines()&#xA;    weights = np.zeros((len(test_set), feature_num))&#xA;&#xA;    # CREATE TEST SET&#xA;    for i in range(len(weights)):&#xA;        weights[i] = file&#xA;    features = []&#xA;    query = []&#xA;    output = np.zeros((len(test_set), feature_num))&#xA;    score = np.zeros((len(test_set)))&#xA;&#xA;    for i in tqdm(range(len(test_set))):&#xA;        elements = test_set[i].split(' ')&#xA;        if int(elements[0]) == 0:&#xA;            score[i] = -1&#xA;        else:&#xA;            score[i] = 1&#xA;&#xA;        query.append(int(elements[1].split(':')[1]))&#xA;        tmp = []&#xA;        for feature in elements[2:2+feature_num]:&#xA;            if feature.split(':')[1] == 'NULL':&#xA;                tmp.append(0.0)&#xA;            else:&#xA;                tmp.append(float(feature.split(':')[1]))&#xA;        features.append(tmp)&#xA;&#xA;    features = np.asarray(features)&#xA;&#xA;    for sample in range(len(test_set)):&#xA;        output[sample] = weights[sample]*features[sample]&#xA;        output[sample] = np.full((feature_num), np.sum(output[sample]))&#xA;&#xA;    predictions = []&#xA;    for val in output:&#xA;        if(val[0] &amp;gt; 1):&#xA;            predictions.append(1)&#xA;        else:&#xA;            predictions.append(-1)&#xA;&#xA;    print(&quot;-------------------------------------&quot;)&#xA;    print(&quot;Predicting...&quot;)&#xA;    print(&quot;-------------------------------------&quot;)&#xA;    print(&quot;Prediction finished with &quot;+str(accuracy_score(score, predictions)*100)+&quot;% accuracy.&quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My training algorithm&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(val[0] &amp;gt;= 1):&#xA;    cost = 0&#xA;    weights = weights - alpha * (2 * 1/epochs * weights)&#xA;else:&#xA;    cost = 1 - val[0]&#xA;    # WEIGHTS = WEIGHTS + LEARNING RATE * [X] * [Y]&#xA;    weights = weights + alpha * (features[count] * score[count] - 2 * 1/epochs * weights)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What could I do to help the model train? Am I not giving it enough time? Is the algorithm wrong? Are the hyperparameters ok?&#xA;Thanks for all your help.&lt;/p&gt;&#xA;" OwnerUserId="4744" LastEditorUserId="2444" LastEditDate="2020-04-07T01:23:41.937" LastActivityDate="2020-04-07T01:23:41.937" Title="Why is my SVM not reaching good accuracy when trained to perform binary classification of search results?" Tags="&lt;ai-design&gt;&lt;classification&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20057" PostTypeId="1" CreationDate="2020-04-06T22:47:45.463" Score="1" ViewCount="48" Body="&lt;p&gt;So let's follow this line of reasoning. We use a MLE estimator (implementation doesn't matter) and we have a train set. We assume that we have sampled training set from a Gaussian distribution &lt;span class=&quot;math-container&quot;&gt;$\mathcal N(\mu, \sigma^2)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we split the dataset into training, validation and test sets. The result will be that each will have maximum likelihoods for the following Gaussian distributions &lt;span class=&quot;math-container&quot;&gt;$\mathcal N(\mu_{train}, \sigma^2_{train}), \mathcal N(\mu_{cv}, \sigma^2_{cv}), \mathcal N(\mu_{test}, \sigma^2_{test})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's assume the  case where &lt;span class=&quot;math-container&quot;&gt;$\mu_{cv}&amp;lt;\mu_{train}&amp;lt;\mu_{test}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mu_{train}&amp;lt;\mu&amp;lt;\mu_{test}$&lt;/span&gt;. Clearly, if we perform validation using this split, then the model which gets selected, will be closer to  &lt;span class=&quot;math-container&quot;&gt;$\mu_{cv}$&lt;/span&gt; which will worsen the performance on actual data, whereas if we only used train set the performance could actually be better (this is the simplest case without taking into account the effect of variance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, like this we will have a &lt;span class=&quot;math-container&quot;&gt;$4!$&lt;/span&gt; combinations between the means and each one might improve or worsen the performance (probably in &lt;span class=&quot;math-container&quot;&gt;$50 \%$&lt;/span&gt; cases performance will be worsened, assuming symmetry).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what am I missing here? Was my aforementioned assumptions wrong? Or does the validation set has a completely different purpose?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-04-09T12:33:10.200" LastActivityDate="2020-04-13T23:19:01.733" Title="What is the theoretical basis for the use of Cross Validation set?" Tags="&lt;machine-learning&gt;&lt;theory&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20059" PostTypeId="1" AcceptedAnswerId="20088" CreationDate="2020-04-07T00:59:05.760" Score="1" ViewCount="174" Body="&lt;p&gt;Background:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a 2D CNN model that I am applying to a regression task with some uniquely extracted spectrograms. The specifics of the data set are mostly irrelevant and very domain specific so I won't go into detail, but it is essentially just image classification with a MSE loss function for each label and a unique image of 100x4000. When I re-train the model from scratch multiple times then provide it my testing data set, it has predictions that vary significantly across each iteration and thus a high variance. Supposedly the only difference between a trained model versus another would be the random initialization of weights and the random train/validation split. I feel that the train/validation split has been ruled out by when I've done k-fold cross validation and my model has done very good for all segments of my train/validation splits and acquired good results for the validation in each split. But these same models persist to have high variance in the test data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am seeing a high variance for the predictions from my trained model across multiple different runs of re-training, what do I attack first to reduce my variance on my predictions for my test data set? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found many articles talking about bias and variance in the data set but not as much criticism directed towards model design. What things can I explore in my dataset or model design, and/or tools I can use to strengthen my model? Does my model need to be bigger/smaller?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideas/Solutions:&#xA;A few Ideas I'd like to acquire some criticism for.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Regularization applied to model such as L1/L2 Regularization, dropout, or early stopping.&lt;/li&gt;&#xA;&lt;li&gt;Data augmentation applied to dataset (inconveniently not an option right now, but in a more general scenario it could be).&lt;/li&gt;&#xA;&lt;li&gt;Bigger or smaller model?&lt;/li&gt;&#xA;&lt;li&gt;Is the random initialization of weight actually very important? Maybe train multiple models and take the average of their collective answers to get the best prediction on real world data (test set).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Personal Note:&#xA;I have had experience with all these items before on other projects and with personal projects and have some moderate confidence justifying regularization and data augmentation. However, I lack some perspective as to any other tools that might be useful to explore the cause of model variance. I wanted to ask this question here to start a discussion in a general sense of this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: CLARIFICATION. When I say 'variance' I mean specifically variance across models, not variance of predictions from 1 trained model across the test set. Example: Instead lets say I am trying to predict a value somewhere between 1 and 4 (expected_val=3). I train 10 models to do this and 4 of the models accurately predict 3 with a VERY low standard deviation across all the test set samples. Thus a low variance and high accuracy/precision for these 4 models. But the other 6 models predict wildly and some predict 1 very confidently every time and the others could be 4. And I've even had models that predicted negative values even though I have NO training or testing samples that have negative labels.&lt;/p&gt;&#xA;" OwnerUserId="33189" LastActivityDate="2020-04-07T17:20:14.337" Title="CNN High Variance across multiple trained models, what does it mean?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20061" PostTypeId="1" CreationDate="2020-04-07T01:29:39.640" Score="2" ViewCount="133" Body="&lt;p&gt;I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient calculations and relationships between gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a problem in understanding what exactly symbol-to-number differentiation is. Does it map directly every variable in the backprop to its relevant gradient? If yes, how does it do this without knowing about the rest of the computational graph? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the question is unclear, to increase context - TensorFlow uses symbol to symbol differentiation whereas torch uses symbol to number (apparently). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Came across this in section 6.5.5 of &lt;a href=&quot;https://www.deeplearningbook.org/contents/mlp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning, the book&lt;/a&gt;. The material mentioned has a convincing explanation for symbol-to-symbol differentiation but could not say the same for symbol-to-number differentiation. &lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="25658" LastEditDate="2020-04-08T08:12:17.440" LastActivityDate="2020-04-08T08:12:17.440" Title="What is symbol-to-number differentiation?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;computational-complexity&gt;&lt;symbolic-computing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20062" PostTypeId="2" ParentId="19879" CreationDate="2020-04-07T03:13:33.093" Score="2" Body="&lt;p&gt;Convolution is a pretty misused term in recent times with the advent of CNN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short answer, Convolution is a linear operator (check &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution#Properties&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) but what you are defining in context of CNN is not convolution, it is cross-correlation which is also linear in case of images (dot product).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convolution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computers work only on discrete values, so the discrete time convolution is defined similarly as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;y[n] = \sum_{k=\infty}^{\infty} x[k] h[n-k]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has the nice property of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y(e^{j\omega}) = X(e^{j \omega})H(e^{j \omega})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where each &lt;span class=&quot;math-container&quot;&gt;$A(e^{j\omega})$&lt;/span&gt; is the Fourier Transform of &lt;span class=&quot;math-container&quot;&gt;$a(t)$&lt;/span&gt;&#xA;So this is the basic idea of discrete convolution. The point to illustrate here is that convolution operation has some nice properties and very helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there are 2 main types of convolution (as far as  I know). One is linear convolution another is circular convolution. In terms of &lt;span class=&quot;math-container&quot;&gt;$2D$&lt;/span&gt; signal they are defined in the following ways:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y[m,n] = \sum_{j=-\infty}^{\infty}\sum_{k=-\infty}^{\infty} x[i,j]h[m-i,n-j]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Circular convolution is the same except that the input sequence is finite from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$N-1$&lt;/span&gt; giving rise to periodicity in frequency domain. The convolution operation is a little bit different due to this periodicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So these are the actual definitions of convolution. It has linearity property (clearly obvious since it is used to calculate output of &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_time-invariant_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;LTI system&lt;/a&gt;), and it also has the linearity in terms of matrix multiplications, since we want to do the computer to do these calculations for us. I don't know a lot but there are many clever manipulations e.g the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fast_Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;FFT algorithm&lt;/a&gt; an indispensable tool in signal processing (used to convert signals to frequency domain for a certain sampling rate). Like this you can define convolutions in terms of Hermitian matrices and stuff (only if you want to process in the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; domain it is much easier to process in the frequency domain). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example a &lt;span class=&quot;math-container&quot;&gt;$1D$&lt;/span&gt; circular convolution between 2 signals in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; domain defined in matrix form, can be written as follows &lt;span class=&quot;math-container&quot;&gt;$y_n=h_n*x_n$&lt;/span&gt;:&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_t = \begin{bmatrix} y_0 \\&#xA;                      y_1 \\&#xA;                      y_2 \\&#xA;                      y_3 &#xA;       \end{bmatrix} = \begin{bmatrix}&#xA;h_0 &amp;amp; h_3 &amp;amp; h_2 &amp;amp; h_1\\&#xA;h_1 &amp;amp; h_0 &amp;amp; h_3 &amp;amp; h_2\\&#xA;h_2 &amp;amp; h_1 &amp;amp; h_0 &amp;amp; h_3 \\&#xA;h_3 &amp;amp; h_2 &amp;amp; h_1 &amp;amp; h_0&#xA;\end{bmatrix}  \begin{bmatrix} x_0 \\&#xA;                      x_1 \\&#xA;                      x_2 \\&#xA;                      x_3 &#xA;       \end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same can be done quite easily by converting the functions into frequency domain, multiplying and converting back to time domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we move to &lt;span class=&quot;math-container&quot;&gt;$2D$&lt;/span&gt; domain similar equation is formed except (in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; domain)instead of &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt; we will have a Hermitian matrix &lt;span class=&quot;math-container&quot;&gt;$H_0$&lt;/span&gt; and we will perform the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kronecker_product&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kronecker product&lt;/a&gt; (I don't know the justification or proofs of these equations, it probably satisfies the &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolution theorem&lt;/a&gt; and is fast when ran on computers). Again much easier to do in frequency domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we move to multiple dimensions, it is called Multidimensional discrete convolution as per this Wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/Multidimensional_discrete_convolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;. As the article &lt;a href=&quot;https://en.wikipedia.org/wiki/Multidimensional_discrete_convolution#Circular_Convolution_of_Discrete-Valued_Multidimensional_Signals&quot; rel=&quot;nofollow noreferrer&quot;&gt;suggests&lt;/a&gt; the property where everything is in frequency domain.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Y(k_1,k_2,...,k_M)=H(k_1,k_2,...,k_M)X(k_1,k_2,...,k_M)$$&lt;/span&gt; &#xA;still holds good. When we do convolutions in the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; domain things get tricky and we have to do clever matrix manipulations as shown in the example above. Whereas, as stated above things get much easier in the frequency domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its &lt;strong&gt;counter-intuitive&lt;/strong&gt; that a picture has a frequency domain, and in general its actually not frequency. BUT in DSP we use a lot of filters whose math properties are similar to those of filters in the traditional sense of frequency and hence has the same calculations as in a frequency setting as shown in the 1st example of &lt;span class=&quot;math-container&quot;&gt;$1D$&lt;/span&gt; signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point convolution by &lt;strong&gt;definition&lt;/strong&gt; is a linear operation. Check &lt;a href=&quot;http://robotics.itee.uq.edu.au/~elec3004/2013/lectures/Convolution%20(Sec%201.4%20from%20Hayes%20DSP).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; if the explanations are unconvincing. There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Time-variant_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear Time Varying systems&lt;/a&gt; and its output maybe determined by convolution, but then it uses the equation at a certain point of time i.e:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Y(e^{j\omega}, t) = H(e^{j\omega}, t)X(e^{j\omega} )$$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now whether it can be represented by matrix products in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; domain or not I cannot say. But generalising, it should be possible, only it will involve increasingly complex matrix properties and operations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cross Correlation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross correlation is nothing similar to convolution, it has a bunch of interesting properties on its own, hence I do not like the two being mixed together. In signal processing it is mainly related to finding energy (auto-correlation) and has other applications in communication channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While in statistics, it is related to finding correlation (misused term since it can take any value, but correlation coefficient takes value between -1 to 1) between 2 stochastic processes, at different time instants, where we have multiple samplings of the signal or ensembles of the signal based on which the expectation is taken (See &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-correlation#Cross-correlation_of_stochastic_processes&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). In CNNs maybe a way to see this would be, the image dataset is a ensemble of stochastic process, while the filter is another process which is fixed. And moving the filter over the image is time delay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of digital image processing cross correlation is just a dot product. And hence it can be represented by matrices. Whether cross correlation is linear or not is difficult to tell (at least I don't know), but in the context of image processing it seems &lt;strong&gt;linear&lt;/strong&gt;. Bottom-line is that it can be &lt;strong&gt;definitely implemented by matrices&lt;/strong&gt; as it is a dot product, whether it is linear in true mathematical sense is doubtful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more intuitive way in context of CNNs would be to view filters as just a bunch of shared weights for better regularisation instead of cross correlation or convolution. &lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-04-13T01:28:41.493" LastActivityDate="2020-04-13T01:28:41.493" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20063" PostTypeId="1" CreationDate="2020-04-07T03:39:59.547" Score="1" ViewCount="199" Body="&lt;p&gt;I guess I understand the concept of face detection, a technique specifies the location of multiple objects in the image, and draws bounding boxes on the target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2cCxV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2cCxV.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is related to the concept of a &lt;strong&gt;landmark&lt;/strong&gt;. For example, the bottom guy in the image above pointed out by the red arrow has 18 green dots on his face. Is anyone of the dots is a landmark? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the size of a landmark? What is the acceptable error of its position? For example, the landmark in the middle of his nose has to be in what kind of range?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please give a hint?&lt;/p&gt;&#xA;" OwnerUserId="35782" LastEditorUserId="2444" LastEditDate="2020-04-07T12:57:14.420" LastActivityDate="2020-04-07T15:48:51.190" Title="What is a landmark in computer vision?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20064" PostTypeId="2" ParentId="20055" CreationDate="2020-04-07T04:49:36.697" Score="0" Body="&lt;p&gt;I will answer this question, leaving it open to challenge by anyone more knowledgeable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations to update each layer of an RNN are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y_t = \sigma(W_x x_t + b_x )$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_t = \sigma(W_hx_t + b_h)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt; is the hidden layer (in blue in the picture), and &lt;span class=&quot;math-container&quot;&gt;$y_t$&lt;/span&gt; is the output layer (red in picture). This equations says that every single component in the hidden layer vector, i.e. every unit in the hidden layer, is a function of the linear combination of the &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; vector, which is the first yellow row along the depth axis. In other words, all the first yellow input nodes on the bottom left in the picture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, technically this picture is not correct, all the yellow nodes should have points to all the blue nodes. Also, by similar reasoning, all the blue nodes in the subsequent steps of the hidden layer should be connected to all the blue nodes of the previous layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, that would make for a much uglier/harder picture to make, so I don't blame the authors, although this has given me a few hours of confused research, which I guess still meets their educational goal.&lt;/p&gt;&#xA;" OwnerUserId="35809" LastActivityDate="2020-04-07T04:49:36.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20065" PostTypeId="1" AcceptedAnswerId="20070" CreationDate="2020-04-07T05:53:40.110" Score="1" ViewCount="37" Body="&lt;p&gt;My question is about when to balance training data for sentiment analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upon evaluating my training dataset, which has 3 labels (good, bad, neutral), I noticed there were twice as many neutral labels as the other 2 combined, so I used a function to drop neutral labels randomly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wasn't sure if I should do this before or after creating the &lt;code&gt;vocab2index&lt;/code&gt; mappings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To explain, I am numericizing my text data by creating a vocabulary of words in the training data and linking them to numbers using enumerate. I think to use that dictionary of &lt;code&gt;vocab2index&lt;/code&gt; values to numericise the training data. I also use that same dictionary to numericise the testing data, dropping any words that do not exist in the dictionary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I took a class on this, they had balanced the training data AFTER creating the vocab2index dictionary. However, when I thought about this in my own implementation, it did not make sense. What if some words from the original vocabulary are gone completely, then we aren't training the machine learning classifier on those words, but they would not be dropping from the testing data either (since words are dropping from &lt;code&gt;X_test&lt;/code&gt; based on whether they are in the &lt;code&gt;vocab2index&lt;/code&gt; dictionary). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So should I be balancing the data BEFORE creating the vocab2index dictionary? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I linked the code to create &lt;code&gt;X_train&lt;/code&gt; and &lt;code&gt;X_test&lt;/code&gt; below in case it help.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_X_train(training_data='Sentences_75Agree_csv.csv'):&#xA;    data_csv = pd.read_csv(filepath_or_buffer=training_data, sep='.@', header=None, names=['sentence','sentiment'], engine='python')&#xA;    list_data = []&#xA;    for index, row in data_csv.iterrows():&#xA;        dictionary_data = {}&#xA;        dictionary_data['message_body'] = row['sentence']&#xA;        if row['sentiment'] == 'positive':&#xA;             dictionary_data['sentiment'] = 2&#xA;        elif row['sentiment'] == 'negative':&#xA;             dictionary_data['sentiment'] = 0&#xA;        else:&#xA;             dictionary_data['sentiment'] = 1 # For neutral sentiment&#xA;        list_data.append(dictionary_data)&#xA;    dictionary_data = {}&#xA;    dictionary_data['data'] = list_data&#xA;    messages = [sentence['message_body'] for sentence in dictionary_data['data']]&#xA;    sentiments = [sentence['sentiment'] for sentence in dictionary_data['data']]&#xA;&#xA;    tokenized = [preprocess(sentence) for sentence in messages]&#xA;    bow = Counter([word for sentence in tokenized for word in sentence]) &#xA;    freqs = {key: value/len(tokenized) for key, value in bow.items()} #keys are the words in the vocab, values are the count of those words&#xA;&#xA;    # Removing 5 most common words from data&#xA;    high_cutoff = 5&#xA;    K_most_common = [x[0] for x in bow.most_common(high_cutoff)] &#xA;    filtered_words = [word for word in freqs if word not in K_most_common]&#xA;&#xA;    # Create vocab2index dictionary:&#xA;    vocab = {word: i for i, word in enumerate(filtered_words, 1)}&#xA;    id2vocab = {i: word for word, i in vocab.items()}&#xA;    filtered = [[word for word in sentence if word in vocab] for sentence in tokenized] &#xA;&#xA;    # Balancing training data due to large number of neutral sentences&#xA;    balanced = {'messages': [], 'sentiments':[]}&#xA;    n_neutral = sum(1 for each in sentiments if each == 1)&#xA;    N_examples = len(sentiments)&#xA;    # print(n_neutral/N_examples)&#xA;    keep_prob = (N_examples - n_neutral)/2/n_neutral&#xA;    # print(keep_prob)&#xA;    for idx, sentiment in enumerate(sentiments):&#xA;        message = filtered[idx]&#xA;        if len(message) == 0:&#xA;            # skip this sentence because it has length 0&#xA;            continue&#xA;        elif sentiment != 1 or random.random() &amp;lt; keep_prob:&#xA;            balanced['messages'].append(message)&#xA;            balanced['sentiments'].append(sentiment)&#xA;&#xA;    token_ids = [[vocab[word] for word in message] for message in balanced['messages']]&#xA;    sentiments_balanced = balanced['sentiments']&#xA;&#xA;    # Unit test:&#xA;    unique, counts = np.unique(sentiments_balanced, return_counts=True)&#xA;    print(np.asarray((unique, counts)).T)&#xA;    print(np.mean(sentiments_balanced))&#xA;    ##################&#xA;&#xA;    # Left padding and truncating to the same length &#xA;    X_train = token_ids&#xA;    for i, sentence in enumerate(X_train):&#xA;        if len(sentence) &amp;lt;=30:&#xA;            X_train[i] = ((30-len(sentence)) * [0] + sentence)&#xA;        elif len(sentence) &amp;gt; 30:&#xA;            X_train[i] = sentence[:30]&#xA;    return vocab, X_train, sentiments_balanced&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_X_test(test_sentences, vocab):&#xA;    tokenized = [preprocess(sentence) for sentence in test_sentences]&#xA;    filtered = [[word for word in sentence if word in vocab] for sentence in tokenized] # X_test filtered to only words in training vocab&#xA;    # Alternate method with functional programming:&#xA;    # filtered = [list(filter(lambda a: a in vocab, sentence)) for sentence in tokenized]&#xA;    token_ids = [[vocab[word] for word in sentence] for sentence in filtered] # Numericise data&#xA;&#xA;    # Remove short sentences in X_test&#xA;    token_ids_filtered = [sentence for sentence in token_ids if len(sentence)&amp;gt;10]&#xA;    X_test = token_ids_filtered&#xA;    for i, sentence in enumerate(X_test):&#xA;        if len(sentence) &amp;lt;=30:&#xA;            X_test[i] = ((30-len(sentence)) * [0] + sentence)&#xA;        elif len(sentence) &amp;gt; 30:&#xA;            X_test[i] = sentence[:30]&#xA;    return X_test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35816" LastEditorUserId="2444" LastEditDate="2020-04-07T13:05:24.350" LastActivityDate="2020-04-07T13:05:24.350" Title="Should I be balancing the data before creating the vocab-to-index dictionary?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;training&gt;&lt;sentiment-analysis&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20066" PostTypeId="1" CreationDate="2020-04-07T06:59:14.470" Score="1" ViewCount="39" Body="&lt;p&gt;I've tried to do my research on Bayesian neural networks online, but I find most of them are used for &lt;strong&gt;image classification&lt;/strong&gt;. This is probably due to the nature of Bayesian neural networks, which may be significantly slower than traditional artificial neural networks, so people don't use them for text (or document) classification. Am I right? Or is there a more specific reason for that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are bayesian neural networks suited for text (or document) classification?&lt;/p&gt;&#xA;" OwnerUserId="35817" LastEditorUserId="2444" LastEditDate="2020-04-07T13:13:23.050" LastActivityDate="2020-04-07T13:13:23.050" Title="Are bayesian neural networks suited for text (or document) classification?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;text-classification&gt;&lt;bayesian-deep-learning&gt;&lt;bayesian-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20067" PostTypeId="2" ParentId="12390" CreationDate="2020-04-07T08:06:27.260" Score="1" Body="&lt;p&gt;What you are trying to achieve, is a game that &lt;strong&gt;learns&lt;/strong&gt; to play flappy bird. For doing this you need a neural network AND a genetic algorithm, those two things work together.&lt;br&gt;&#xA;About your concerns on the output, you don't have to know if the action will benefit or not, i will soon explain why.&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The neural network part&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So, what you need is to know how to build a neural network, i don't know your knowledge about it, but i suggest starting from the basics. In this scenario, you need a feed forward neural network, because you just take the inputs from the current flappy bird scene/frame (such as the y position of the bird, the distance from the closes pipe ecc..) and feed it through a network that outputs either 1 or 0 (jump or don't jump) in the only output neuron we just decided it has.&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In python you can implement a neural network from scratch, or using a neural network framework that does al the dirty work for you.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;From scratch you would need to use numpy for matrix calculations, and you would need to learn matrix multiplication, dot products and all that fancy stuff (You can just let numpy taking care of the matrix calculations, but understanding how it works behind the scenes always helps understand new problems that you might come across when doing more advanced stuff)&lt;/li&gt;&#xA;&lt;li&gt;Using a framework like &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow&lt;/a&gt; for python, the only thing you need to do is find the right structure for the network you want to use. You will not have to worry about how activations work, or how the feed forward is performed (But again, it's a good thing to know when working with neural nets)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;The genetic algorithm part or &quot;&quot;learning&quot;&quot;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I say &quot;&quot;learning&quot;&quot; because at first sight it might look like learning, but really it is not.&#xA;The genetic algorithm works like &quot;the survival of the fittest&quot;, where the &quot;smarter&quot; birds, which are the ones that reached the higher score on the current generation, will have a chance to have their child little birds, that have the same brain as their parent, with either some minimal modifications, or a mix of their parent brains.&lt;br&gt;&#xA;The process of this &quot;&quot;learning&quot;&quot;, so the genetic algorithm, works like so:&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create a generation of let's say 200 birds, every bird has a brain with random weights, so at the first run, they are all very...not smart&lt;/li&gt;&#xA;&lt;li&gt;The game starts, and every frame of the game, the brain of the bird recieves as input some data that is taken from the current frame ( y pos of the bird, distance from pipe...)&lt;/li&gt;&#xA;&lt;li&gt;The brain ( neural network ) of each bird, performs a feed forward with that data, and outputs what at the beginning is a very random result, let's say 0.75 for one bird&lt;/li&gt;&#xA;&lt;li&gt;At this point you decided that 0.75 is greater than 0.5, so you take that as a 1, which stands for &quot;jump&quot;, while if it was 0.3, so 0, the bird does nothing and keeps falling&lt;/li&gt;&#xA;&lt;li&gt;Shortly the bird will die cause he has no idea of what he is doing, so he most likely collides with a pipe or the ground.&lt;/li&gt;&#xA;&lt;li&gt;After all birds met their fate, you see that some birds reached further than others, so you choose, for example, 5 of the best performing ones.&lt;/li&gt;&#xA;&lt;li&gt;Now you try to create a new generation of 200 birds using only the brains of those 5 that were choosen, by mixing and modifying theyr brains&lt;/li&gt;&#xA;&lt;li&gt;Now the new birds have a brand new brain, that in some cases might be better than the previous one, so chances are that some of those birds will reach a higher score, therefore flap further into the level.&lt;/li&gt;&#xA;&lt;li&gt;Repeat from point 6&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In practice your &quot;perform_genetic_algorithm&quot; function in python, will have to choose the birds with the highest score, and as wild as it sounds, mix their brains and modify them, hoping that some modifications will improve the performance of the bird.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I can't think of output since you don't really know if the action of flapping will benefit you or not&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The mechanism above explains why you basically do not care at all about the output, except saying to the game engine: &quot;hey the bird decided to flap, do it&quot;. Whether it's the right action or not, doesn't matter, as the smarter birds are &lt;strong&gt;naturally&lt;/strong&gt; gonna get further and so be more likely to be choosen for next generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully now it's all more clear.&#xA;&lt;br&gt;&#xA;Here is some useful links for building a neural network and for understanding the genetic algorithm:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to build a neural network&lt;/a&gt;: I am linking this because it contains all useful information about how to build a very basic neural network in python. In your case, you would have to ignore all the part about backpropagation, loss &amp;amp; error calculation and SGD, and just look at the &lt;strong&gt;feed forward&lt;/strong&gt; part.&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://enlight.nyc/projects/neural-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to build a neural network - 2&lt;/a&gt;: This is another example of building a neural network that i found really useful, probably it's simpler and more straight forward than the previous link, but again, the backpropagation part is not needed for this genetic based learning.&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=9zfeTw-uFCw&amp;amp;list=PLRqwX-V7Uu6bJM3VgzjNV5YxVxUwzALHV&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video tutorials on genetic algorithm&lt;/a&gt;: This is a very long but very explanatory playlist of videos that dives into the nature of genetic algorithms and how to implement one&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/introduction-to-optimization-with-genetic-algorithm-2f5001d9964b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic algorithm optimization&lt;/a&gt;: Other source about genetic algorithms&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="35792" LastEditorUserId="35792" LastEditDate="2020-04-07T09:03:59.717" LastActivityDate="2020-04-07T09:03:59.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20068" PostTypeId="2" ParentId="20028" CreationDate="2020-04-07T08:06:38.170" Score="2" Body="&lt;p&gt;The second implementation looks more correct and inline with how Bidirectional is defined. Specifically, bidirectionality doen't change the forward/backward logic of either direction, and just merges (concat/sum/...) the outputs of forward/backward at a matching timestep &lt;code&gt;t&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check how Keras implements it &lt;a href=&quot;https://github.com/keras-team/keras/blob/7a39b6c62d43c25472b2c2476bd2a8983ae4f682/keras/layers/wrappers.py#L388&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. There are distinct &lt;code&gt;self.forward_layer&lt;/code&gt; and &lt;code&gt;self.backward_layer&lt;/code&gt; that are initialized separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your &lt;code&gt;for&lt;/code&gt; loop doesn't look ok though. To calculate the output at time step &lt;code&gt;0&lt;/code&gt; you'd have to calculate &lt;code&gt;forward(0)&lt;/code&gt; and &lt;code&gt;backward(n)&lt;/code&gt;, which means you have to run backward on all the samples first. In practice, each direction is calculated separately and the results are merged afterwards. Check the implementaion in Keras &lt;a href=&quot;https://github.com/keras-team/keras/blob/7a39b6c62d43c25472b2c2476bd2a8983ae4f682/keras/layers/wrappers.py#L529&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="34315" LastActivityDate="2020-04-07T08:06:38.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20069" PostTypeId="1" CreationDate="2020-04-07T08:24:26.160" Score="2" ViewCount="16" Body="&lt;p&gt;How to work with GCN when the features of each node is not a 1D vector? For example, if the graph has N nodes and each node has features of the form &lt;span class=&quot;math-container&quot;&gt;$C \times D \times E$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there an open-source implementation for such a case?&lt;/p&gt;&#xA;" OwnerUserId="35819" LastEditorUserId="2444" LastEditDate="2020-04-07T13:15:43.063" LastActivityDate="2020-04-07T13:15:43.063" Title="How should I deal with multi-dimensional tensors for nodes in a graph convolution network?" Tags="&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20070" PostTypeId="2" ParentId="20065" CreationDate="2020-04-07T08:26:21.530" Score="0" Body="&lt;p&gt;If you look at the words in your dictionary (vocab) before/after pruning, most likely you'd see there isn't a lot of difference, not so much to affect your model performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, creating a dictionary and model training are two more or less indpendent processes. To make your life easier, you could find the largest dev set you can find for building your vocab (excluding test set), and freeze it for all subsequent ETL/modeling.  This way you don't have to deal with dictionary versioning, for example, after choosing different subsets of your training data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if you have compute capacity, I'd suggest to upsample positive/negative classes instead, because those neutral samples you're dropping do have signal on the use of language and which borderline ambiguous samples should be treated as neutral. &lt;/p&gt;&#xA;" OwnerUserId="34315" LastActivityDate="2020-04-07T08:26:21.530" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20071" PostTypeId="2" ParentId="19987" CreationDate="2020-04-07T09:32:36.433" Score="1" Body="&lt;p&gt;Okay, I'm first going to review how NEAT works. I hope this helps you model NEAT successfully as a whole, not just limited to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use neuro-evolution to create a specific behavior that solves a given task. The behavior can be simple and complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's focus on behavior... Different neural networks can create the same behavior (A.K.A the competing conventions problem). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also want to end up with a neural network that is very efficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we want to solve two problems with one algorithm (NEAT): find the behavior that solves a task and find the most efficient neural network that creates the behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a simple way to search for the most efficient neural net: start with the simplest neural net and slowly build the neural net up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard part is defining behavior (how do we define behavior in neural networks?). NEAT introduces their very interesting definition of behavior: an atomic unit of neural network behavior is the connection of the neural network: which node connects to which and with what weight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm approaching your question(concern): you want to make sure your genomes can grow complex, which means that you want to preserve the capacity for your genome to express complex behavior. As the paper states, behavior does not have anything to do with nodes, let alone node genes. Behavior is a set of neural network connections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here's my answer to your question: care about crossing over connections, not for the node genes. Find an efficient way to calculate the node genes. For instance, you'll consume a lot of memory if you just concatenate node genes from two parents without overlaps. There is a chance that some of the node genes might not be in use (i.e. disabled connections, not inheriting some connection genes due to probability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps :)&lt;/p&gt;&#xA;" OwnerUserId="35823" LastActivityDate="2020-04-07T09:32:36.433" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20072" PostTypeId="1" CreationDate="2020-04-07T09:36:43.010" Score="1" ViewCount="46" Body="&lt;p&gt;GANs have shown good progress across a wide variety of domains ranging from image translation, image generation, text to image synthesis, audio/video generation, image super-resolution and many more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although these concepts have great research potential, what are some real-world products or applications that can be developed using GANs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few I know are drug discovery and product customization. What else can you suggest?&lt;/p&gt;&#xA;" OwnerUserId="35791" LastEditorUserId="2444" LastEditDate="2020-04-08T13:26:19.847" LastActivityDate="2020-04-08T13:26:19.847" Title="What are some real-world products or applications that can be developed using GANs?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;applications&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20073" PostTypeId="1" AcceptedAnswerId="20097" CreationDate="2020-04-07T10:36:06.187" Score="2" ViewCount="35" Body="&lt;p&gt;I am having a hard time understanding the proof of theorem 1 presented in the &amp;quot;&lt;a href=&quot;http://incompleteideas.net/papers/PSD-01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Off-Policy Temporal-Difference Learning with Function Approximation&lt;/a&gt;&amp;quot; paper.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\Delta \theta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Delta \bar{\theta}$&lt;/span&gt; be the sum of the parameter increments over an episode under on-policy &lt;span class=&quot;math-container&quot;&gt;$T D(\lambda)$&lt;/span&gt;&#xA;and importance sampled &lt;span class=&quot;math-container&quot;&gt;$T D(\lambda)$&lt;/span&gt; respectively, assuming&#xA;that the starting weight vector is &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in both cases. Then&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E_{b}\left\{\Delta \bar{\theta} | s_{0}, a_{0}\right\}=E_{\pi}\left\{\Delta \theta | s_{0}, a_{0}\right\}, \quad \forall s_{0} \in \mathcal{S}, a_{0} \in \mathcal{A}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We know that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;&amp;amp;\Delta \theta_{t}=\alpha\left(R_{t}^{\lambda}-\theta^{T} \phi_{t}\right) \phi_{t}\\&#xA;&amp;amp;R_{t}^{\lambda}=(1-\lambda) \sum_{n=1}^{\infty} \lambda^{n-1} R_{t}^{(n)}\\&#xA;&amp;amp;R_{t}^{(n)}=r_{t+1}+\gamma r_{t+2}+\cdots+\gamma^{n-1} r_{t+n}+\gamma^{n} \theta^{T} \phi_{t+n}&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta \bar{\theta_{t}}=\alpha\left(\bar{R}_{t}^{\lambda}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\bar{R}_{t}^{(n)}=&amp;amp; r_{t+1}+\gamma r_{t+2} \rho_{t+1}+\cdots \\&#xA;&amp;amp;+\gamma^{n-1} r_{t+n} \rho_{t+1} \cdots \rho_{t+n-1} \\&#xA;&amp;amp;+\gamma^{n} \rho_{t+1} \cdots \rho_{t+n} \theta^{T} \phi_{t+n}&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And it is proven that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_{b}\left\{\bar{R}_{t}^{\lambda} | s_{t}, a_{t}\right\}=E_{\pi}\left\{R_{t}^{\lambda} | s_{t}, a_{t}\right\}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the proof, it begins with:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E_{b}\{\Delta \bar{\theta}\}=E_{b}\left\{\sum_{t=0}^{\infty} \alpha\left(\bar{R}_{t}^{\lambda}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}\right\}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$=E_{b}\left\{\sum_{t=0}^{\infty} \sum_{n=1}^{\infty} \alpha(1-\lambda) \lambda^{n-1}\left(\bar{R}_{t}^{(n)}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}\right\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;which I believe is incorrect since,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E_{b}\{\Delta \bar{\theta}\}=E_{b}\left\{\sum_{t=0}^{\infty} \alpha\left(\bar{R}_{t}^{\lambda}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}\right\}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$=E_{b}\left\{\sum_{t=0}^{\infty} \alpha \left(\sum_{n=1}^{\infty}(1-\lambda) \lambda^{n-1}\bar{R}_{t}^{(n)}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}\right\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;and taking out the second sigma will lead to a sum over constant terms.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, it is claimed that in order to prove the equivalence above, it is enough to prove the equivalence below:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{array}{c}&#xA;E_{b}\left\{\sum_{t=0}^{\infty}\left(\bar{R}_{t}^{(n)}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}\right\} \\&#xA;=E_{\pi}\left\{\sum_{t=0}^{\infty}\left(R_{t}^{(n)}-\theta^{T} \phi_{t}\right) \phi_{t}\right\}&#xA;\end{array}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which I don't understand why. and even if it is the case there are more ambiguities in the proof:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E_{b}\left\{\sum_{t=0}^{\infty}\left(\bar{R}_{t}^{(n)}-\theta^{T} \phi_{t}\right) \phi_{t} \rho_{1} \rho_{2} \cdots \rho_{t}\right\}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=\sum_{t=0}^{\infty} \sum_{\omega \in \Omega_{t}} p_{b}(\omega) \phi_{t} \prod_{k=1}^{t} \rho_{k} E_{b}\left\{\bar{R}_{t}^{(n)}-\theta^{T} \phi_{t} | s_{t}, a_{t}\right\}$$&lt;/span&gt;&#xA;(given the Markov property, and I don't understand why Markovian property leads to conditional independence !)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=\sum_{t=0}^{\infty} \sum_{\omega \in \Omega_{t}} \prod_{j=1}^{t} p_{s_{j-1}, s_{j}}^{a_{j-1}} b\left(s_{j}, a_{j}\right) \phi_{t} \prod_{k=1}^{t} \frac{\pi\left(s_{k}, a_{k}\right)}{b\left(s_{k}, a_{k}\right)} \cdot \left(E_{b}\left\{\bar{R}_{t}^{(n)} | s_{t}, a_{t}\right\}-\theta^{T} \phi_{t}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$= \sum_{t=0}^{\infty} \sum_{\omega \in \Omega_{t}} \prod_{j=1}^{t} p_{s_{j-1}, s_{j}}^{a_{j-1}} \pi\left(s_{j}, a_{j}\right) \phi_{t} \cdot\left(E_{b}\left\{\bar{R}_{t}^{(n)} | s_{t}, a_{t}\right\}-\theta^{T} \phi_{t}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$=\sum_{t=0}^{\infty} \sum_{\omega \in \Omega_{t}} p_{\pi}(\omega) \phi_{t}\left(E_{\pi}\left\{R^{(n)} | s_{t}, a_{t}\right\}-\theta^{T} \phi_{t}\right)$$&lt;/span&gt;&#xA;(using our previous result)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=E_{\pi}\left\{\sum_{t=0}^{\infty}\left(R_{t}^{(n)}-\theta^{T} \phi_{t}\right) \phi_{t}\right\} . \diamond$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'd be grateful if anyone could shed a light on this.&lt;/p&gt;&#xA;" OwnerUserId="35827" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-08T11:03:20.437" Title="Equivalence between expected parameter increments in &quot;Off-Policy Temporal-Difference Learning with Function Approximation&quot;" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;temporal-difference-methods&gt;&lt;function-approximation&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20074" PostTypeId="2" ParentId="20037" CreationDate="2020-04-07T11:56:22.693" Score="1" Body="&lt;p&gt;Could you post the pseudocode of your backpropagation algorithm?&#xA;I recommend you start off as simple as possible (this includes your cost f(x), I would simply use Yexpected-Youtput) and see if it works and then continue adding things. If it's your first time with neural networks, I recommend you check this &lt;a href=&quot;http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; out and you could also try practising the algorithms on a programming language like Octave/Matlab (it can be very efficient speed wise). Also check this question out &lt;a href=&quot;https://ai.stackexchange.com/questions/19936/is-my-backpropagation-code-correct&quot;&gt;(link)&lt;/a&gt;. At the bottom there is a code example for the XOR problem. Please post the pseudocode of your code instead of just dumping it there.  Finally, don't just copy paste algorithms into your code, you need to understand them.&lt;/p&gt;&#xA;" OwnerUserId="35660" LastActivityDate="2020-04-07T11:56:22.693" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20075" PostTypeId="1" AcceptedAnswerId="20084" CreationDate="2020-04-07T12:05:31.030" Score="28" ViewCount="18602" Body="&lt;p&gt;I am reading the article &lt;a href=&quot;https://towardsdatascience.com/transformers-141e32e69591&quot; rel=&quot;noreferrer&quot;&gt;How Transformers Work&lt;/a&gt; where the author writes&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Another problem with RNNs, and LSTMs, is that it’s hard to parallelize the work for processing sentences, since you have to process word by word. Not only that but there is no model of &lt;strong&gt;long and short-range dependencies&lt;/strong&gt;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why exactly does the transformer do better than RNN and LSTM in &lt;strong&gt;long-range context dependencies&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-04-07T16:08:58.620" LastActivityDate="2020-04-21T14:25:52.257" Title="Why does the transformer do better than RNN and LSTM in long-range context dependencies?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;transformer&gt;" AnswerCount="3" CommentCount="1" FavoriteCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="20076" PostTypeId="1" CreationDate="2020-04-07T12:16:40.490" Score="3" ViewCount="108" Body="&lt;p&gt;I am trying to understand &lt;a href=&quot;https://www.aaai.org/Papers/AAAI/2008/AAAI08-227.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the formulation of the maximum entropy Inverse RL method by Brian Ziebart&lt;/a&gt;. Particularly, I am stuck on how to understand the computation of state - visitation frequencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to do so, they utilize a dynamic programming approach to compute the visitation frequency, in which the next state frequency is calculated based upon the state visitation frequency at the previous time step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the algorithm below where, &lt;span class=&quot;math-container&quot;&gt;$D_{s_i,t}$&lt;/span&gt; is the probability of state &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; being visited at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fYQdg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fYQdg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between this way of computing state visitation frequency compared to the naive method of summing the total number of times state &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; appears in the trajectory divided by the trajectory length?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-04-07T13:25:11.750" LastActivityDate="2020-04-07T13:25:11.750" Title="How is the state-visitation frequency computed in &quot;Maximum Entropy Inverse Reinforcement Learning&quot;?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20077" PostTypeId="5" CreationDate="2020-04-07T13:09:39.390" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-07T13:09:39.390" LastActivityDate="2020-04-07T13:09:39.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20078" PostTypeId="4" CreationDate="2020-04-07T13:09:39.390" Score="0" Body="For questions related to Bayesian neural networks, which are artificial neural networks that incorporate uncertainty in their parameters." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-07T13:09:39.390" LastActivityDate="2020-04-07T13:09:39.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20079" PostTypeId="5" CreationDate="2020-04-07T13:16:57.700" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-07T13:16:57.700" LastActivityDate="2020-04-07T13:16:57.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20080" PostTypeId="4" CreationDate="2020-04-07T13:16:57.700" Score="0" Body="For questions related to graph neural networks, which are artificial neural networks applied to graphs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-07T13:16:57.700" LastActivityDate="2020-04-07T13:16:57.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20081" PostTypeId="1" CreationDate="2020-04-07T14:34:17.520" Score="1" ViewCount="77" Body="&lt;p&gt;What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using a naive bayes classifier the probabilities are a fact. As far as I know there is nothing one could tune (like the weights in a neural net). So what is the purpose of the test data set? Simply to know if one can apply naive bayes or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similiarly what is the benefit of the test data set when constructing a decision tree. We alread use the gini impurity to construct the best possibe decision tree and there is nothing we could do when we get bad results with the test data set.&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="27777" LastEditDate="2020-04-24T23:20:45.613" LastActivityDate="2020-04-24T23:20:45.613" Title="What is the meaning of test data set in naive bayes classifier or decision trees?" Tags="&lt;datasets&gt;&lt;decision-tree&gt;&lt;naive-bayes&gt;" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20082" PostTypeId="2" ParentId="20063" CreationDate="2020-04-07T15:33:59.553" Score="2" Body="&lt;p&gt;The paper &lt;a href=&quot;https://graphics.stanford.edu/courses/cs164-09-spring/Handouts/paper_shape_spaces_imm403.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Brief Introduction to Statistical Shape Analysis&lt;/a&gt; (2002) by M. B. Stegmann and D. D. Gomez provides a definition of a landmark in the context of statistical shape analysis, which I will report below.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Definition 1&lt;/strong&gt;: Shape is all the geometrical information that remains when location, scale and rotational effects are filtered out from an object.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For example, in the following diagram, the shape of a hand, after several transformations (i.e. translation, scaling and rotation), is illustrated. This definition captures the intuitive notion of the shape of an object. In fact, as expected, after these transformations, all of the objects below correspond to a hand.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Uql3q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uql3q.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Definition 2&lt;/strong&gt;: A landmark is a point of correspondence on each object that matches between and within populations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In your picture, all of the faces (apart from the occluded one) have 18 green points. Each of these points roughly corresponds to the same position in the face. For example, there's a green point on tip of the nose of each face. In your picture, the populations refer to the faces.&lt;/p&gt;&#xA;&lt;p&gt;You can divide landmark into three groups&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Anatomical landmarks&lt;/strong&gt; Points assigned by an expert that corresponds between organisms in some biologically meaningful way.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Mathematical landmarks&lt;/strong&gt; Points located on an object according to some mathematical or geometrical property, i.e. high curvature or an extremum point.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Pseudo-landmarks&lt;/strong&gt; Constructed points on an object either on the outline or between landmarks.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The landmarks in your picture could be considered &lt;em&gt;anatomical landmarks&lt;/em&gt;, so they might have been assigned by an expert (a human), so this should explain why they do not exactly correspond. Moreover, note that e.g. noses have different shapes across humans, so correspondences are unlikely to be exact in most cases.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What is the size of a landmark?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is just an educated guess, but a landmark is probably just a point (i.e. a coordinate of a pixel in the image), although in your picture they look bigger, but this is probably to make them more noticeable.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What is the acceptable error of its position?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is also a guess, but this might depend on the application and the used algorithms, i.e., in certain cases, your algorithms may be very sensitive to the position of the landmarks, in other cases, that might not be a big problem.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-07T15:48:51.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20083" PostTypeId="1" CreationDate="2020-04-07T15:45:09.017" Score="1" ViewCount="27" Body="&lt;p&gt;The following script is from Trappenberg's Fundamentals of Computational Neuroscience and is used to test a perceptron's robustness against noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, how would one alter it to test the output of a multilayer perceptron?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, &lt;code&gt;wOut&lt;/code&gt; and &lt;code&gt;rIn&lt;/code&gt; are non-conformable because the &lt;code&gt;wOut&lt;/code&gt; of a multilayer perceptron with 2 hidden layers is &lt;span class=&quot;math-container&quot;&gt;$26*2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how could I alter this to test the trained multilayer perceptron?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Testing generalization perfroamnce of trained perceptron&#xA; perceptron_network_sim; &#xA; letterMatrix=rIn; &#xA; for nflip = 1:80;    &#xA;    dist1=[]; dist2=[]; &#xA;    for trial=1:10; &#xA;       rIn=abs(letterMatrix-randomFlipMatrix(nflip)); &#xA;       % Threshold output function&#xA;       rOut1=(wOut*rIn)&amp;gt;0;&#xA;       nerror=0; &#xA;       for j=1:26; nerror=nerror+(sum(rDes(:,j)~=rOut1(:,j))&amp;gt;0); end &#xA;       dist1=[dist1,nerror];&#xA;       % Max output function&#xA;       [v,i]=max(wOut*rIn);&#xA;       rOut2=zeros(26); for j=1:26; rOut2(i(j),j)=1; end&#xA;       dist2=[dist2,0.5*sum(sum(rDes~=rOut2))];&#xA;    end&#xA;    meanDist1(nflip)=mean(dist1); stdDist1(nflip)=std(dist1);&#xA;    meanDist2(nflip)=mean(dist2); stdDist2(nflip)=std(dist2);&#xA; end&#xA; figure; hold on;&#xA; errorbar((1:80)/156,meanDist1,stdDist1,':')&#xA; errorbar((1:80)/156,meanDist2,stdDist2,'r')&#xA; xlabel('Fraction of flipped bits')&#xA; ylabel('Average number of wrong letters')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35833" LastEditorUserId="2444" LastEditDate="2020-04-07T17:50:24.893" LastActivityDate="2020-04-07T17:50:24.893" Title="How can I adapt this script (to test the robustness of a perceptron) to test the robustness of a multi-layer perceptron?" Tags="&lt;neural-networks&gt;&lt;pattern-recognition&gt;&lt;multilayer-perceptron&gt;&lt;matlab&gt;&lt;perceptron&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20084" PostTypeId="2" ParentId="20075" CreationDate="2020-04-07T16:12:04.617" Score="23" Body="&lt;p&gt;I'll list some bullet points of the main innovations introduced by transformers , followed by bullet points of the main characteristics of the other architectures you mentioned, so we can then compared them.&lt;/p&gt;&#xA;&lt;h1&gt;Transformers&lt;/h1&gt;&#xA;&lt;p&gt;Transformes (&lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;noreferrer&quot;&gt;Attention is all you need&lt;/a&gt;) were introduced in the context of machine translation with the purpose to avoid recursion in order to allow parallel computation (to reduce training time) and also to reduce drop in performances due to long dependencies. The main characteristics are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Non sequential&lt;/strong&gt;: sentences are processed as a whole rather than word by word.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Self Attention&lt;/strong&gt;: this is the newly introduced 'unit' used to compute similarity scores between words in a sentence.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Positional embeddings&lt;/strong&gt;: another innovation introduced to replace recurrence. The idea is to use fixed or learned weights which encode information related to a specific position of a token in a sentence.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first point is the main reason why transformer do not suffer from long dependency issues. Original transformers do not relies on past hidden states to capture dependencies with previous words, they process a sentence as a whole, reason why there is no risk to loose (or 'forget') past information. Moreover, also multi-head attention and positional embeddings both provide information about the relationship between different words.&lt;/p&gt;&#xA;&lt;h1&gt;RNN / LSTM&lt;/h1&gt;&#xA;&lt;p&gt;Recurrent neural networks and Long-short term memory models for what concern this question are almost identical in their core properties:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Sequential processing&lt;/strong&gt;: sentences must be processed words by words.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Past information retained through past hidden states&lt;/strong&gt;: sequence to sequence models follow the Markov property, each state is assumed to be dependent only on the previously seen state.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first property is the reason why RNN and LSTM can't be trained in parallel. In order to encode the second word in a sentence I need the previously computed hidden states of the first word, therefore I need to compute that first. The second property is a bit more subdole, but not hard to grasp conceptually. Information in RNN and LSTM are retained thank's to previously computed hidden states. The point is that the encoding of a specific word is retained only for the next time step, which means that the encoding of a word strongly affect only the representation of the next word, its influence is quickly lost after few time steps. LSTM (and also GruRNN) can boost a bit the dependency range they can learn thanks to a deeper processing of the hidden states through specific units (which comes with an increased number of parameters to train) but nevertheless the problem is inherently related to recursion. Another way in which people mitigated this problem is to use Bi-directional models, which encode the same sentence from two direction, from the start to end and from the end to the start, allowing this way words at the end of a sentence to have stronger influence in the creation of the hidden representation, but this is just a workaround rather than a real solution for very long dependencies.&lt;/p&gt;&#xA;&lt;h1&gt;CNN&lt;/h1&gt;&#xA;&lt;p&gt;Also convolutional neural network are widely used in nlp since they are quite fast to train and effective with short texts. The way they tackle dependencies is by applying different kernels to the same sentence, and indeed since they're first application to text (&lt;a href=&quot;https://arxiv.org/pdf/1408.5882.pdf&quot; rel=&quot;noreferrer&quot;&gt;Convolutional Neural Networks for Sentence Classification&lt;/a&gt;) they were implement as multichannel CNN. Why do different kernels allow to learn dependencies? Because a kernel of size 2 for example would learn relationships between pairs of words, a kernel of size 3 would capture relationships between triplets of words and so on. The evident problem here is that the number of different kernels required to capture dependencies among all possible combination of words in a sentences would be enormous and unpractical because of the exponential growing number of combinations when increasing the maximum length size of input sentences.&lt;/p&gt;&#xA;&lt;p&gt;To summarise, Transformers are better than all the other architectures because they totally avoid recursion, by processing sentences as a whole and by learning relationships between words thank's to multi-head attention mechanisms and positional embeddings. Nevertheless, it must be pointed out that also transformers can capture only dependencies within the fixed input size used to train them, i.e. if I use as a maximum sentence size 50, the model will not be able to capture dependencies between the first word of a sentence and words that occur more than 50 words later, like in another paragraph. New transformers like &lt;a href=&quot;https://arxiv.org/abs/1901.02860&quot; rel=&quot;noreferrer&quot;&gt;Transformer-XL&lt;/a&gt; tries to overcome exactly this issue, by kinda re-introducing recursion by storing hidden states of already encoded sentences to leverage them in the subsequent encoding of the next sentences.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-07T16:17:21.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20085" PostTypeId="2" ParentId="20075" CreationDate="2020-04-07T16:20:25.827" Score="3" Body="&lt;p&gt;Let's start with RNN. A well known problem is vanishin/exploding gradients, which means that the model is biased by most recent inputs in the sequence, or in other words, older inputs have practically no effect in the output at the current step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSTMs/GRUs mainly try to solve this problem, by including a separate memory (cell) and/or extra gates to learn when to let go of past/current information. Check these series of &lt;a href=&quot;https://youtu.be/QEw0qEa0E50&quot; rel=&quot;nofollow noreferrer&quot;&gt;lectures&lt;/a&gt; for more in-depth discussion. Also check the interactive parts of &lt;a href=&quot;https://distill.pub/2019/memorization-in-rnns/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; for some intuitive understanding of dependency on past elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given all this, information from past steps &lt;em&gt;still&lt;/em&gt; goes through a sequence of computations and we're relying on these new gate/memory mechanisms to pass information from old steps to the current one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One major advantage of the transformer architecture, is that at each step we have &lt;strong&gt;direct&lt;/strong&gt; access to all the other steps (self-attention), which practically leaves no room for information loss, as far as message passing is concerned. On top of that, we can look at both future and past elements at the same time, which also brings the benefit of bidirectional RNNs, without the 2x computation needed. And of course, all this happens in parallel (non-recurrent), which makes both training/inference much faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The self-attention with every other token in the input means that the processing will be in the order of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^2)$&lt;/span&gt; (glossing over details), which means that it's going to be costly to apply transformers on long sequences, compared to RNNs. That's probably one area that RNNs still have an advantage over transformers. &lt;/p&gt;&#xA;" OwnerUserId="34315" LastEditorUserId="2444" LastEditDate="2020-04-07T16:25:10.303" LastActivityDate="2020-04-07T16:25:10.303" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20086" PostTypeId="1" CreationDate="2020-04-07T16:58:16.163" Score="1" ViewCount="66" Body="&lt;p&gt;I've been trying to hyper tuning my KNNBasic algorithm by the help of grid search for recommendation system for movie review data. The problem is that both of my KNNBasicTuned and KNNBasicUntuned shows the same result. Here is my code for KNNTuning..&#xA;I have tried the SVD algo tuning and it worked perfectly so all my libraries are working perfectly. However my all libraries are in my github linke : &lt;a href=&quot;https://github.com/iSarcastic99/KNNBasicTuning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/iSarcastic99/KNNBasicTuning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code of KNNBasicTuning :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wXuBB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wXuBB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# -*- coding: utf-8 -*-&#xA;&quot;&quot;&quot;&#xA;Created on Sat Apr  4 01:25:40 2020&#xA;&#xA;@author: rahulss&#xA;&quot;&quot;&quot;&#xA;#My libraries&#xA;&#xA;from MovieLens import MovieLens    &#xA;from surprise import KNNBasic&#xA;from surprise import NormalPredictor&#xA;from Evaluator import Evaluator&#xA;from surprise.model_selection import GridSearchCV&#xA;&#xA;import random&#xA;import numpy as np&#xA;&#xA;#loading my working data&#xA;def LoadMovieLensData():&#xA;    ml = MovieLens()&#xA;    print(&quot;Loading movie ratings...&quot;)&#xA;    data = ml.loadMovieLensLatestSmall()&#xA;    print(&quot;\nComputing movie popularity ranks so we can measure novelty later...&quot;)&#xA;    rankings = ml.getPopularityRanks()&#xA;    return (ml, data, rankings)&#xA;&#xA;np.random.seed(0)&#xA;random.seed(0)&#xA;&#xA;# Load up common data set for the recommender algorithms&#xA;(ml, evaluationData, rankings) = LoadMovieLensData()&#xA;&#xA;print(&quot;Searching for best parameters...&quot;)&#xA;param_grid = {'n_epochs': [10, 30], 'lr_all': [0.005, 0.010],&#xA;          'n_factors': [50, 90]}&#xA;gs = GridSearchCV(KNNBasic, param_grid, measures=['rmse', 'mae'], cv=3)&#xA;&#xA;gs.fit(evaluationData)&#xA;&#xA;# best RMSE score&#xA;print(&quot;Best RMSE score attained: &quot;, gs.best_score['rmse'])&#xA;&#xA;# combination of parameters that gave the best RMSE score&#xA;print(gs.best_params['rmse'])&#xA;&#xA;# Construct an Evaluator to, you know, evaluate them&#xA;evaluator = Evaluator(evaluationData, rankings)&#xA;&#xA;params = gs.best_params['rmse']&#xA;KNNBasictuned = KNNBasic(n_epochs = params['n_epochs'], lr_all =  params['lr_all'], n_factors = params['n_factors'])&#xA;evaluator.AddAlgorithm(KNNBasictuned, &quot;KNN - Tuned&quot;)&#xA;&#xA;KNNBasicUntuned = KNNBasic()&#xA;evaluator.AddAlgorithm(KNNBasicUntuned, &quot;KNN - Untuned&quot;)&#xA;&#xA;&#xA;# Evaluating all algorithms&#xA;evaluator.Evaluate(False)&#xA;&#xA;evaluator.SampleTopNRecs(ml, testSubject=85, k=10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25685" LastActivityDate="2020-04-07T16:58:16.163" Title="Why can't I Hyper tune my KNNBasic Algorithm?" Tags="&lt;python&gt;&lt;hyperparameter-optimization&gt;&lt;recommender-system&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20087" PostTypeId="2" ParentId="20081" CreationDate="2020-04-07T17:09:13.467" Score="1" Body="&lt;p&gt;Your assumption about the test data is not correct completely. Maybe you use the test data to tune your learning algorithm to work better on the test data, but it's not the whole thing. Sometimes you need to know that the ML method is working or not and have a sense about how much does it work!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have other scenarios that you want to evaluate your method:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compare the result of the leaner with other techniques. For example, you are considering DT versus an SVM classifier over a data set. If you want to compare them, you need a value to found such a sense about the comparison.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sometimes you are using an ensemble method and you want to tune some parameters to balance between using different ML methods. Hence, you need to evaluate these learning methods (DT, Naive Bayes) to improve the ensemble method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2020-04-07T17:09:13.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20088" PostTypeId="2" ParentId="20059" CreationDate="2020-04-07T17:20:14.337" Score="0" Body="&lt;p&gt;Firstly if you're doing spectrogram classification, you'd probably want to use a loss function like cross entropy. That would give you less &quot;high-variance&quot; like results when you're calculating your evaluation metrics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About your ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Dropout and early stopping would be enough&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Data augmentation with speech is quite helpful, and there are a few libraries to help you with volume/speed/noise augmentation. Volume/speed augmentation is actually really important and you'd want to do it anyway for decent results in production. Speech processing is sometimes a bit tricky, in the sense that what we hear may not be what ends up in the spectrogram and parameters like noise, recording setup, encoding, accent end up causing a domain-mismatch (covariate shift)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;In general, a smaller model (less parameters) helps with high variance. Start with small and simple models first, make sure eval metrics look ok, and then try larger models. &lt;/li&gt;&#xA;&lt;li&gt;Random initialization doesn't really affect the results in a systematic way. You can indeed ensemble predictions of models trained with different initialization; that does help with variance but don't expect a lot of difference. The gain is not worth the computation cost for models applied in practice (ie not kaggle)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Other ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Check that you shuffle data properly in train/test, and between folds &lt;/li&gt;&#xA;&lt;li&gt;Do some error analysis on the mis-predicted test samples, do they have something in common. Check the label distribution in your dev/test set.&lt;/li&gt;&#xA;&lt;li&gt;If the test set was captured in a process different from your dev set (total blind), don't expect a robust performance. In general, speech recognition is &lt;strong&gt;really&lt;/strong&gt; data hungry with a decent &quot;generic* model needing hundreds of hours of recording from different domains (tv, news, movies, telephone, lecture, echo, etc.). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="34315" LastActivityDate="2020-04-07T17:20:14.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20089" PostTypeId="2" ParentId="20081" CreationDate="2020-04-07T17:38:44.713" Score="0" Body="&lt;p&gt;In the field of ML and AI, you should always remember that before choosing any algorithm you should know the data .One should always start with Data Analyzing, which itself is field for this critical job. Decision Tree can never work with its best optimization without tuning the datasets. Here is a great article  that you can refer : &lt;a href=&quot;https://towardsdatascience.com/how-to-tune-a-decision-tree-f03721801680&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tuning Decision Tree&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Purpose of test data in naive byes classifier:&#xA;1) It's necessity to check the accuracy, hits, hit rates, coverage, diversity, novelty, etc. metrics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) It hypertune your testdata(as anti_train_set) also by using mean, standard deviation, variance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really think that you should try other algorithms to train your datasets. I can't name all of them. However, in Neural network, rnn, cnn, rbm are some great algorithm to work with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please always remember that Machine learning is like an art, where datasets (test, train, evaluated) are colors and its up-to us to use the right amount of them.&lt;/p&gt;&#xA;" OwnerUserId="25685" LastActivityDate="2020-04-07T17:38:44.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20091" PostTypeId="1" CreationDate="2020-04-07T20:53:03.660" Score="1" ViewCount="107" Body="&lt;p&gt;I am implementing a gym environment and I have several input arrays as my input (different sizes). The most simple method to integrate my environment into the gym is to use the &lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/spaces/dict.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;dict space&lt;/a&gt; as my environment space (which each of the entries denote one of the spaces), but the problem is that the &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;stable-baseline library&lt;/a&gt;, which I am aiming to use for my training, does not accept the dict space type. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What solutions do I have for solving this problem? Is there another library that accepts dict spaces as its input?&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-04-09T14:53:08.207" LastActivityDate="2020-04-09T14:53:08.207" Title="How to integrate dict space of OpenAI gym into a reinforcement learning framework?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20092" PostTypeId="1" CreationDate="2020-04-07T21:53:07.923" Score="1" ViewCount="32" Body="&lt;p&gt;I save the trained model after a certain number of episodes with the special save() function of the DDPG class (the network is saved when the reward reaches zero), but when I restore the model again using saver.restore(), the network gives out a reward equal to approximately -1800. Why is this happening, maybe I'm doing something wrong?&#xA;My network:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;import numpy as np&#xA;import gym&#xA;&#xA;epsiode_steps = 500&#xA;&#xA;# learning rate for actor&#xA;lr_a = 0.001&#xA;&#xA;# learning rate for critic&#xA;lr_c = 0.002&#xA;gamma = 0.9&#xA;alpha = 0.01&#xA;memory = 10000&#xA;batch_size = 32&#xA;render = True&#xA;&#xA;&#xA;class DDPG(object):&#xA;    def __init__(self, no_of_actions, no_of_states, a_bound, ):&#xA;        self.memory = np.zeros((memory, no_of_states * 2 + no_of_actions + 1), dtype=np.float32)&#xA;&#xA;        # initialize pointer to point to our experience buffer&#xA;        self.pointer = 0&#xA;&#xA;        self.sess = tf.Session()&#xA;&#xA;        self.noise_variance = 3.0&#xA;&#xA;        self.no_of_actions, self.no_of_states, self.a_bound = no_of_actions, no_of_states, a_bound,&#xA;&#xA;        self.state = tf.placeholder(tf.float32, [None, no_of_states], 's')&#xA;        self.next_state = tf.placeholder(tf.float32, [None, no_of_states], 's_')&#xA;        self.reward = tf.placeholder(tf.float32, [None, 1], 'r')&#xA;&#xA;        with tf.variable_scope('Actor'):&#xA;            self.a = self.build_actor_network(self.state, scope='eval', trainable=True)&#xA;            a_ = self.build_actor_network(self.next_state, scope='target', trainable=False)&#xA;&#xA;        with tf.variable_scope('Critic'):&#xA;            q = self.build_crtic_network(self.state, self.a, scope='eval', trainable=True)&#xA;            q_ = self.build_crtic_network(self.next_state, a_, scope='target', trainable=False)&#xA;&#xA;        self.ae_params = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='Actor/eval')&#xA;        self.at_params = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='Actor/target')&#xA;&#xA;        self.ce_params = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='Critic/eval')&#xA;        self.ct_params = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='Critic/target')&#xA;&#xA;        # update target value&#xA;        self.soft_replace = [&#xA;            [tf.assign(at, (1 - alpha) * at + alpha * ae), tf.assign(ct, (1 - alpha) * ct + alpha * ce)]&#xA;            for at, ae, ct, ce in zip(self.at_params, self.ae_params, self.ct_params, self.ce_params)]&#xA;&#xA;        q_target = self.reward + gamma * q_&#xA;&#xA;        td_error = tf.losses.mean_squared_error(labels=(self.reward + gamma * q_), predictions=q)&#xA;&#xA;        self.ctrain = tf.train.AdamOptimizer(lr_c).minimize(td_error, name=&quot;adam-ink&quot;, var_list=self.ce_params)&#xA;&#xA;        a_loss = - tf.reduce_mean(q)&#xA;&#xA;        # train the actor network with adam optimizer for minimizing the loss&#xA;        self.atrain = tf.train.AdamOptimizer(lr_a).minimize(a_loss, var_list=self.ae_params)&#xA;&#xA;        tf.summary.FileWriter(&quot;logs2&quot;, self.sess.graph)&#xA;&#xA;        # initialize all variables&#xA;&#xA;        self.sess.run(tf.global_variables_initializer())&#xA;        self.saver = tf.train.Saver()&#xA;        self.saver.restore(self.sess, &quot;Pendulum/nn.ckpt&quot;)&#xA;&#xA;&#xA;    def choose_action(self, s):&#xA;        a = self.sess.run(self.a, {self.state: s[np.newaxis, :]})[0]&#xA;        a = np.clip(np.random.normal(a, self.noise_variance), -2, 2)&#xA;&#xA;        return a&#xA;&#xA;    def learn(self):&#xA;        # soft target replacement&#xA;        self.sess.run(self.soft_replace)&#xA;&#xA;        indices = np.random.choice(memory, size=batch_size)&#xA;        batch_transition = self.memory[indices, :]&#xA;        batch_states = batch_transition[:, :self.no_of_states]&#xA;        batch_actions = batch_transition[:, self.no_of_states: self.no_of_states + self.no_of_actions]&#xA;        batch_rewards = batch_transition[:, -self.no_of_states - 1: -self.no_of_states]&#xA;        batch_next_state = batch_transition[:, -self.no_of_states:]&#xA;&#xA;        self.sess.run(self.atrain, {self.state: batch_states})&#xA;        self.sess.run(self.ctrain, {self.state: batch_states, self.a: batch_actions, self.reward: batch_rewards,&#xA;                                    self.next_state: batch_next_state})&#xA;&#xA;    # we define a function store_transition which stores all the transition information in the buffer&#xA;    def store_transition(self, s, a, r, s_):&#xA;        trans = np.hstack((s, a, [r], s_))&#xA;&#xA;        index = self.pointer % memory&#xA;        self.memory[index, :] = trans&#xA;        self.pointer += 1&#xA;&#xA;        if self.pointer &amp;gt; memory:&#xA;            self.noise_variance *= 0.99995&#xA;            self.learn()&#xA;&#xA;    # we define the function build_actor_network for builing our actor network and after crtic network&#xA;    def build_actor_network(self, s, scope, trainable)&#xA;        with tf.variable_scope(scope):&#xA;            l1 = tf.layers.dense(s, 30, activation=tf.nn.tanh, name='l1', trainable=trainable)&#xA;            a = tf.layers.dense(l1, self.no_of_actions, activation=tf.nn.tanh, name='a', trainable=trainable)&#xA;            return tf.multiply(a, self.a_bound, name=&quot;scaled_a&quot;)               &#xA;&#xA;    def build_crtic_network(self, s, a, scope, trainable):&#xA;        with tf.variable_scope(scope):&#xA;            n_l1 = 30&#xA;            w1_s = tf.get_variable('w1_s', [self.no_of_states, n_l1], trainable=trainable)&#xA;            w1_a = tf.get_variable('w1_a', [self.no_of_actions, n_l1], trainable=trainable)&#xA;            b1 = tf.get_variable('b1', [1, n_l1], trainable=trainable)&#xA;            net = tf.nn.tanh(tf.matmul(s, w1_s) + tf.matmul(a, w1_a) + b1)&#xA;&#xA;            q = tf.layers.dense(net, 1, trainable=trainable)&#xA;            return q&#xA;&#xA;    def save(self):&#xA;        self.saver.save(self.sess, &quot;Pendulum/nn.ckpt&quot;)&#xA;&#xA;env = gym.make(&quot;Pendulum-v0&quot;)&#xA;env = env.unwrapped&#xA;env.seed(1)&#xA;&#xA;no_of_states = env.observation_space.shape[0]&#xA;no_of_actions = env.action_space.shape[0]&#xA;&#xA;a_bound = env.action_space.high&#xA;ddpg = DDPG(no_of_actions, no_of_states, a_bound)&#xA;&#xA;total_reward = []&#xA;&#xA;no_of_episodes = 300&#xA;# for each episodes&#xA;for i in range(no_of_episodes):&#xA;    # initialize the environment&#xA;    s = env.reset()&#xA;&#xA;    # episodic reward&#xA;    ep_reward = 0&#xA;&#xA;    for j in range(epsiode_steps):&#xA;&#xA;        env.render()&#xA;&#xA;        # select action by adding noise through OU process&#xA;        a = ddpg.choose_action(s)&#xA;&#xA;        # peform the action and move to the next state s&#xA;        s_, r, done, info = env.step(a)&#xA;&#xA;        # store the the transition to our experience buffer&#xA;        # sample some minibatch of experience and train the network&#xA;        ddpg.store_transition(s, a, r, s_)&#xA;&#xA;        # update current state as next state&#xA;        s = s_&#xA;&#xA;        # add episodic rewards&#xA;        ep_reward += r&#xA;&#xA;        if int(ep_reward) == 0 and i &amp;gt; 200:&#xA;            ddpg.save()&#xA;            print(&quot;save&quot;)&#xA;            quit()&#xA;&#xA;        if j == epsiode_steps - 1:&#xA;            total_reward.append(ep_reward)&#xA;            print('Episode:', i, ' Reward: %i' % int(ep_reward))&#xA;&#xA;            break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35842" LastActivityDate="2020-04-07T21:53:07.923" Title="Why does the result when restoring a saved DDPG model differ significantly from the result when saving it?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20093" PostTypeId="2" ParentId="7926" CreationDate="2020-04-07T23:30:53.673" Score="0" Body="&lt;p&gt;&lt;em&gt;I was very bored.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short answer is &lt;strong&gt;&lt;em&gt;yes&lt;/em&gt;&lt;/strong&gt; and the long answer is basically &lt;strong&gt;&lt;em&gt;yes&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to gloss over what you could possibly mean by &lt;em&gt;AI&lt;/em&gt; and instead focus on what &lt;em&gt;AI programs&lt;/em&gt; pretty much are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A mish-mash of algorithms and methods borrowed from mathematics, perhaps more specifically statistics, inserted into a good old-fashioned program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's start from the very beginning. The &lt;strong&gt;very beginning&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of this argument, &lt;strong&gt;everything&lt;/strong&gt; is a language and there exists an &lt;strong&gt;omnipotent alphabet&lt;/strong&gt; which contains all possible symbols you can come up with to convey a message. Such an alphabet would contain everything you've ever known, all their permutations, and then some. It would contain your clothes as well. Forget countable infinity or all those concepts, those come way later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to realise that right now I'm communicating with you through &lt;em&gt;not just&lt;/em&gt; through the power of a string of characters. You're also experiencing other stimuli that you, or whatever part of &quot;you&quot;, is/are capable of &lt;strong&gt;interpreting&lt;/strong&gt; into another language. To be crude, everything you decide to classify as a 'standalone thing' is a compiler that runs in perpetual execution, translating all the 'things' it 'receives' into other 'things' it 'spits out' for other 'things' to 'receive'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think about a modern day computer. You write a program in your fancy little language. You hit compile. A compiler goes through your code, and spits out more code. Except this time this code is written in another language, sometimes &quot;closer to the metal&quot;, sometimes &quot;just about the same abstract level&quot; and sometimes &quot;even more abstract&quot;, and this process repeats itself until somewhere along the line that code you started with, has been interpreted to mere electrical signals, which then themselves are being 'compiled' by an entity we will call '&lt;strong&gt;the universe&lt;/strong&gt;' and that's up to empirical observation to determine what and what's not going on. (Except &quot;the universe&quot; was always responsible. But we will partition things for the sake of ..partitioning things. You get what I mean, I know you do.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's jump back to languages. Mathematics is a language in the sense that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is built of symbols contained in an alphabet we will call &lt;strong&gt;X&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;The several fields specify their own grammatical structure through which we can decide whether or not a statement is &lt;em&gt;well-formed&lt;/em&gt;. This encompasses everything from where you can put the &lt;strong&gt;+&lt;/strong&gt; operator in high school algebra, to how you can write a &lt;strong&gt;proof&lt;/strong&gt; in formal logic. There needs to be no justification for how you build a grammar. You can always just make a new grammar and use it instead. Of course, it might not be capable of forming statements which are compatible with other grammars.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What's interesting about &lt;strong&gt;X&lt;/strong&gt; is that its definition is not fixed. Throughout time, we've introduced new symbols into the mathematical alphabet to be able to express more concepts while keeping things separate. (Or rather some people have had the sense to keep it this way.) For example, whenever you see Leibniz's integration symbol, you know you're probably dealing with some kind of integration and not something novel that you've never heard of before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now here's where I actually answer your question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I assume that by &quot;program&quot; you are referring to the mathematical construct as defined in theoretical computer science: A &lt;strong&gt;string&lt;/strong&gt; of &lt;strong&gt;characters&lt;/strong&gt; from an &lt;strong&gt;alphabet&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;This &lt;strong&gt;string&lt;/strong&gt; is then fed to a &lt;strong&gt;compiler&lt;/strong&gt; (lexer|parser|&lt;strong&gt;semantic analyser&lt;/strong&gt;) which spits out another &lt;strong&gt;string&lt;/strong&gt; (mainly the job of the semantic analyser). This string usually is built from characters of a different alphabet. That is to say, the &lt;strong&gt;compiler is a function which maps a well formed string of language A to a string of language B&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;The end goal of compiling a program is to execute it, which basically means a succession of &lt;strong&gt;compilers&lt;/strong&gt; will take the output of the previous compiler and spit back yet another string, until the string is essentially &lt;strong&gt;electrons moving about the circuitry of the computer in your bedroom and producing fancy lights on your monitor&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So whenever you write an &quot;AI program&quot;, you're just writing a &quot;program&quot; that contains some &quot;AI algorithms&quot; which are really just applications of things we've known in mathematics for 100s of years, which again, are really just a string of characters that are about to be translated by a compiler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, nothing you can ever write is &lt;em&gt;not&lt;/em&gt; deterministic, provided you look at the bigger picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common argument I see is that since AI programs usually &quot;adapt&quot; and &quot;self-optimise&quot; when solving the problem, they're not quite deterministic, in the sense that feeding the program the same input twice will (hopefully) yield better results the second time.&#xA;Except what really happened is that you had an input string that you partitioned into inputs A and B, and fed them to the algorithm in succession. Had you fed AB initially, you would've obtained the same results.&lt;/p&gt;&#xA;" OwnerUserId="35843" LastActivityDate="2020-04-07T23:30:53.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20094" PostTypeId="1" AcceptedAnswerId="20113" CreationDate="2020-04-08T03:39:15.583" Score="7" ViewCount="240" Body="&lt;p&gt;For people who have experience in the field, why is creating AI that has the ability to write programs (that are syntactically correct and useful) a hard task? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the barriers/problems we have to solve before we can solve this problem? If you are in the camp that this isn't that hard, why hasn't it become mainstream?&lt;/p&gt;&#xA;" OwnerUserId="22840" LastEditorUserId="2444" LastEditDate="2020-12-07T14:32:31.903" LastActivityDate="2020-12-07T14:32:31.903" Title="Why is creating an AI that can code a hard task?" Tags="&lt;machine-learning&gt;&lt;philosophy&gt;&lt;genetic-programming&gt;&lt;ai-complete&gt;&lt;inductive-programming&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20096" PostTypeId="1" CreationDate="2020-04-08T06:49:15.480" Score="3" ViewCount="551" Body="&lt;p&gt;I encountered the phrase/concept &lt;em&gt;off-the-shelf CNN&lt;/em&gt; in &lt;a href=&quot;https://arxiv.org/abs/1403.6382&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; in which authors used &lt;em&gt;off-the-shelf CNN&lt;/em&gt; representation, &lt;code&gt;OverFeat&lt;/code&gt;, with simple classifiers to address different recognition tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it correctly, it literally means something is ready to be used for a task without alteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody explain in simple words what &lt;em&gt;off-the-shelf CNN&lt;/em&gt; technically means in the context of AI and convolutional neural networks?&lt;/p&gt;&#xA;" OwnerUserId="31312" LastEditorUserId="2444" LastEditDate="2020-04-08T13:37:19.900" LastActivityDate="2020-05-13T16:42:08.383" Title="What does &quot;off-the-shelf&quot; mean?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20097" PostTypeId="2" ParentId="20073" CreationDate="2020-04-08T11:03:20.437" Score="1" Body="&lt;p&gt;First part is correct&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\sum_{n=1}^{\infty} \alpha(1-\lambda)\lambda^{n-1} (\bar R_t^{(n)} - \theta^T \phi_t)\\&#xA;=&amp;amp; \alpha[\sum_{n=1}^{\infty} (1-\lambda)\lambda^{n-1} \bar R_t^{(n)} - \sum_{n=1}^{\infty} (1-\lambda)\lambda^{n-1} \theta^T \phi_t]&#xA;\end{align}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\sum_{n=1}^{\infty} (1-\lambda)\lambda^{(n-1)}$&lt;/span&gt; sums to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; so we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\alpha[\sum_{n=1}^{\infty} (1-\lambda)\lambda^{n-1} \bar R_t^{(n)} - \theta^T \phi_t]&#xA;\end{equation}&lt;/span&gt;&#xA;For the second part it's enough to prove equivalence for any &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; because result contains sum over &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. If you have 2 sums &lt;span class=&quot;math-container&quot;&gt;$\sum x_n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\sum y_n$&lt;/span&gt; then the sums will be equal if for any &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x_n = y_n$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the third part, we are in state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and we already took action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; so we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;E_b \{ \sum_{t=0}^{\infty} (\bar R_t^{(n)} - \theta^T\phi_t)\phi_t \rho_1\rho_2\cdots\rho_t\}\\&#xA;=&amp;amp; \sum_{t=0}^{\infty} E_b \{(\bar R_t^{(n)} - \theta^T\phi_t)\phi_t \rho_1\rho_2\cdots\rho_t\}\\&#xA;=&amp;amp; \sum_{t=0}^{\infty} E_b \{\phi_t \rho_1\rho_2\cdots \rho_t\} E_b \{(\bar R_t^{(n)} - \theta^T\phi_t)|s_t, a_t\}&#xA;\end{align}&lt;/span&gt;&#xA;that is because &lt;span class=&quot;math-container&quot;&gt;$\rho_i, i = 1, \ldots, t-1$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$s_i, a_i$&lt;/span&gt;. Because of Markov property expectation over &lt;span class=&quot;math-container&quot;&gt;$\bar R_t^{(n)}$&lt;/span&gt; doesn't depend on those state it only depends on &lt;span class=&quot;math-container&quot;&gt;$s_t, a_t$&lt;/span&gt; so they are independent. We don't need to consider &lt;span class=&quot;math-container&quot;&gt;$\phi_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_t$&lt;/span&gt; in expectation over &lt;span class=&quot;math-container&quot;&gt;$\bar R_t^{(n)}$&lt;/span&gt; either because, like I said, we are in state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and we took &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; so they are already decided they would be considered a constant. We can then split total expectation in part &lt;span class=&quot;math-container&quot;&gt;$E_b \{\phi_t \rho_1\rho_2\cdots \rho_t\}$&lt;/span&gt; for getting to state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and part &lt;span class=&quot;math-container&quot;&gt;$E_b \{(\bar R_t^{(n)} - \theta^T\phi_t)|s_t, a_t\}$&lt;/span&gt; for expectation over &lt;span class=&quot;math-container&quot;&gt;$\bar R_t^{(n)}$&lt;/span&gt; after we got to state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and took action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-04-08T11:03:20.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20098" PostTypeId="2" ParentId="20096" CreationDate="2020-04-08T13:47:53.610" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://dictionary.cambridge.org/dictionary/english/off-the-shelf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dictionary's definition of off-the-shelf&lt;/a&gt; is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;used to describe a product that is available immediately and does not need to be specially made to suit a particular purpose&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The same dictionary provides several examples&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You can purchase off-the-shelf software or have it customized to suit your needs.&lt;/p&gt;&#xA;&lt;p&gt;If you have complex needs, we don't recommend that you buy software off the shelf.&lt;/p&gt;&#xA;&lt;p&gt;For this, off-the-shelf algorithms included in the robot's programming libraries are used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, your intuition is correct! An off-the-shelf model, software, product, etc., is any model, software or, respectively, product that would be easily (or immediately) available or applicable to the specific context, but, at the same time, it may also be applicable to many other contexts or problems.&lt;/p&gt;&#xA;&lt;p&gt;An off-the-shelf convolutional neural network is thus a typical or standard CNN that can be applied immediately in that context (but that is potentially applicable to many other contexts or problems). Examples of CNNs that could be used as off-the-shelf models are &lt;a href=&quot;https://en.wikipedia.org/wiki/AlexNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlexNet&lt;/a&gt; or &lt;a href=&quot;http://yann.lecun.com/exdb/lenet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LeNet-5&lt;/a&gt;, but the actual choice depends on the context and needs.&lt;/p&gt;&#xA;&lt;p&gt;An off-the-shelf model can also be a &lt;em&gt;baseline model&lt;/em&gt; (i.e. a very simple model that is used in experiments as the model that every other model should outperform), but not necessarily.&lt;/p&gt;&#xA;&lt;p&gt;Alex Graves uses this term/expression in his paper &lt;a href=&quot;https://papers.nips.cc/paper/4329-practical-variational-inference-for-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Variational Inference for Neural Networks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-13T16:42:08.383" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20100" PostTypeId="1" AcceptedAnswerId="20101" CreationDate="2020-04-08T15:59:14.247" Score="4" ViewCount="72" Body="&lt;p&gt;For my master thesis I am working on a dialogue system that should be deployed in hospitals to administer simple questionnaires to patients. I already did literature research and I'm fine with what I found since I don't have to replicate something which has been already done, but I noticed that there are really few papers regarding this specific 'robot interviewer' topic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me explain the task a bit more in details: in a real interview a human interviewer usually starts with greetings and with an explanation of the questionnaire to administer, and then he/she starts asking some more or less structured questions to the person to interview. The idea here is to replace a human interviewer with a dialogue system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, at a first glimpse it seems like a task that can be easily hand coded, and indeed lot of real application simply use systems in which specific questions are stored in memory in association with some already made answers to choose from (&lt;a href=&quot;https://www.aclweb.org/anthology/W13-4050.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's an example&lt;/a&gt;), and the system simply show them (or read in the case of humanoid robots) to the people being interviewed, the system then wait for the answer and then move on with the next questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that in real interviews the conversation flow is obviously much more smooth and natural. A human being can detect doubts in the voice of the interviewed person, which can also explicitly ask for explanations, a human being can also understand when an answer comes with emotional implications ('yes I feel sad every day') and we are able to automatically react to these hidden implications with emotional fillers ('I'm sorry to hear that'). All these aspects require to train some natural language understanding module in order to be replicated in an artificial agent (and this is actually what I'm currently working on), so I though I would have found more papers on this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, despite having found a tons of paper related to open domain dialogue systems, affective and attentive systems, even systems able to reply with irony, I did not found many papers about dialogue systems for smooth interviews or questionnaire administration, which in my opinion sounds like an much easier task to tackle (especially if compared to open domain conversations).&#xA;The only two papers that I found which truly focused on interviewer systems are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.ifaamas.org/AAMAS/aamas2014/proceedings/aamas/p1061.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SimSensei Kiosk: A Virtual Human Interviewer for Healthcare Decision Support&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://sap.ist.i.kyoto-u.ac.jp/lab/bib/intl/HAR-IWSDS19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A job interview dialogue system with autonomous android ERICA&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So my question is: did I miss something, like some specific keywords? Or is there actually a gap in the literature with regard to the design of dialogue systems for interviews and questionnaire (or surveys) administration? I'm interested in any link or hint from anyone working on similar applicants, thank you in advance!   &lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="2444" LastEditDate="2020-04-08T18:01:55.787" LastActivityDate="2020-04-08T18:01:55.787" Title="Is there any literature on the design of dialogue systems for interviews and questionnaire administration?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;dialogue-systems&gt;&lt;human-robot-interaction&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20101" PostTypeId="2" ParentId="20100" CreationDate="2020-04-08T16:49:08.910" Score="4" Body="&lt;p&gt;[Disclaimer: I work for a company that provides a platform for developing conversational AI systems]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The platform used by the company I work for has a sentiment analysis component, so you can recognise if the user input expresses certain emotions. The dialogues are encoded in 'flows', which are graphs with an initial trigger consisting of output nodes and transitions that correspond to user input. In order to react to emotions expressed by the user you'd have a trigger that does not react to the user input text, but instead to a setting of the sentiment flags.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in a call centre bot, if the user repeatedly uses angry sentiment in their utterances, the bot could detect this and trigger a flow that says &quot;It seems you are not happy with my responses -- please wait while I transfer you to a human agent&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This all depends on the mechanics of organising the dialogues. If you have a ML model, this will probably not be as easy as if you have a manually designed graph structure (which gives you more control over such matters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being a commercial system, there are obviously no academic papers on this. But as it's so easy to do within the platform, it doesn't seem too much of a problem in the first place.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-08T16:49:08.910" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20103" PostTypeId="1" CreationDate="2020-04-08T21:00:16.290" Score="2" ViewCount="53" Body="&lt;p&gt;When using k-fold cross-validation in a deep learning problem, after you have computed your hyper-parameters, how do you decide how long to train your final model? My understanding is that, after the hyperparameters are selected, you train your model one more time on the entire set of data, but it's not clear to me when you decide to stop training.  &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-04-08T21:52:49.790" LastActivityDate="2020-04-08T23:08:06.003" Title="After having selected the best model with cross-validation, for how long should I train it?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;hyperparameter-optimization&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20104" PostTypeId="1" AcceptedAnswerId="20105" CreationDate="2020-04-08T21:57:54.937" Score="1" ViewCount="95" Body="&lt;p&gt;Can a variational auto-encoder (VAE) learn images whose pixels have been generated from a Gaussian distribution (e.g. &lt;span class=&quot;math-container&quot;&gt;$N(0, 1)$&lt;/span&gt;), i.e. each pixel is a sample from &lt;span class=&quot;math-container&quot;&gt;$N(0, 1)$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My gut feeling says no, because the VAE adds additional noise &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; to the original image in the latent space, and if all images and pixels are random from the same distribution it would be impossible do decode/reconstruct into the particular input image. However, VAEs are a bit of a mystery to me internally. Any help would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="35867" LastEditorUserId="2444" LastEditDate="2020-04-08T23:43:23.777" LastActivityDate="2020-04-08T23:43:23.777" Title="Can a variational auto-encoder learn images composed of random noise at each pixel (each drawn from the same distribution)?" Tags="&lt;deep-learning&gt;&lt;autoencoders&gt;&lt;probability-distribution&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20105" PostTypeId="2" ParentId="20104" CreationDate="2020-04-08T23:05:27.897" Score="1" Body="&lt;p&gt;VAE's try and model the &lt;strong&gt;distribution&lt;/strong&gt; of your data. So it's not going to learn &quot; images composed of random noise at each pixel&quot; &lt;em&gt;per se&lt;/em&gt; (though, if overfitting, it could remember them). But it would be very capable of learning the simple noise &lt;strong&gt;distribution&lt;/strong&gt; from which you sampled your random pixels&lt;/p&gt;&#xA;" OwnerUserId="11351" LastActivityDate="2020-04-08T23:05:27.897" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20106" PostTypeId="2" ParentId="20103" CreationDate="2020-04-08T23:08:06.003" Score="2" Body="&lt;p&gt;Short answer: training &quot;duration&quot; or number of epochs/updates should be cross-validated too: you want to early-stop your training to prevent overfitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longer answer:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Think of accuracy on the validation set as an estimate of accuracy on future data, given the value of some hyperparameter. In this case, the hyperparameter of interest is the number of training epochs. So: for each CV fold, train the network (e.g. up to some maximum number of epochs). After each epoch, record accuracy on the validation set. Compute the average validation set accuracy (across CV folds) for each number of epochs. Choose the number of epochs that maximizes this value.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/298084/k-fold-cross-validation-for-choosing-number-of-epochs&quot;&gt;https://stats.stackexchange.com/questions/298084/k-fold-cross-validation-for-choosing-number-of-epochs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11351" LastActivityDate="2020-04-08T23:08:06.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20107" PostTypeId="2" ParentId="13120" CreationDate="2020-04-08T23:40:30.223" Score="2" Body="&lt;p&gt;GANs are notably hard to train and it is not uncommon to have large bumps in the losses. The learning rate is a good start but the instability may come from a wide variety of reasons. I'm assuming that you have no bug in your code or data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For one, gradient descent is not well suited to the 2-player game we're playing. I've personally found ExtraAdam to yield much more stable training (&lt;a href=&quot;https://github.com/GauthierGidel/Variational-Inequality-GAN/blob/master/optim/extragradient.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1802.10551&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could also come from the loss and many tricks have been developed and one of the most popular one is enforcing smoothness in the gradient (see W-GAN, W-GAN-GP etc.). SpectralNorm (&lt;a href=&quot;https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/spectral_norm.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/abs/1802.05957&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;) is a very popular and recent normalization technique for the discriminator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of additional tricks to make GANs work like label smoothing and flipping, different update rates for the discriminator and generators (as in BigGAN for instance). I suggest you have a look at this nice repo of (somewhat seasoned) tricks: &lt;a href=&quot;https://github.com/soumith/ganhacks&quot; rel=&quot;nofollow noreferrer&quot;&gt;ganhacks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11351" LastActivityDate="2020-04-08T23:40:30.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20108" PostTypeId="2" ParentId="20053" CreationDate="2020-04-08T23:53:35.957" Score="2" Body="&lt;p&gt;I'm not familiar with python I'm afraid, but I'll give a go at presenting some of the maths...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to perform the &lt;strong&gt;back propagation&lt;/strong&gt; using matrices, matrix &lt;a href=&quot;https://en.wikipedia.org/wiki/Transpose&quot; rel=&quot;nofollow noreferrer&quot;&gt;transpose&lt;/a&gt; is used for different sized layers. Also note that when using matrices for this, we need to distinguish different types of matrix multiplication, namely matrix &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_multiplication&quot; rel=&quot;nofollow noreferrer&quot;&gt;product&lt;/a&gt;, and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hadamard_product_(matrices)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hadamard product&lt;/a&gt; which operates differently (the latter of which is denoted by a circle with a dot in the centre).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the &lt;strong&gt;back propagation&lt;/strong&gt; formulas:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(EQ1)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\delta ^{l} = (w^{l+1})^{T} \delta ^{l+1} \odot \sigma {}' (z ^{l})\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(EQ2)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w}=a^{l-1}\delta ^{l}\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see the transpose of the weight matrix is used to multiply against the delta of the layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, consider a simple network with an input layer of 3 neurons, and an output layer of 2 neurons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;unactivated&lt;/em&gt; &lt;strong&gt;feed forward&lt;/strong&gt; output is given by...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}z_{1} = h_{1} w_{1} + h_{2} w_{3} + h_{3} w_{5}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}z_{2} = h_{1} w_{2} + h_{2} w_{4} + h_{3} w_{6}\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which can be represented in matrix form as...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;z&#xA;=&#xA;\begin{bmatrix}z_{1}\\z_{2}\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}h_{1} w_{1} + h_{2} w_{3} + h_{3} w_{5}\\h_{1} w_{2} + h_{2} w_{4} + h_{3} w_{6}\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(EQ3)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;=&#xA;\begin{bmatrix}w_{1}&amp;amp;w_{3}&amp;amp;w_{5}\\w_{2}&amp;amp;w_{4}&amp;amp;w_{6}\end{bmatrix}&#xA;\begin{bmatrix}h_{1}\\h_{2}\\h_{3}\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which allows us to &lt;strong&gt;forward propagate&lt;/strong&gt; from a layer of 3 neurons, to a layer of 2 neurons (via matrix multiplication).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;strong&gt;back propagation&lt;/strong&gt;, we need the &lt;em&gt;error&lt;/em&gt; of each neruon...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N.B &lt;em&gt;Cost&lt;/em&gt; is a metric used to denote the &lt;em&gt;error&lt;/em&gt; of the entire network and depends on your cost function. It is usually the mean of the sum of the &lt;em&gt;errors&lt;/em&gt; of each nuerons, but of course depends on your cost function used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g for &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;MSE&lt;/a&gt; is...&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;C_{MSE}=\frac{1}{N}\sum (o_{n}-t_{n})^{2}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are interested in the derivative of the &lt;em&gt;error&lt;/em&gt; for each neuron (not the &lt;em&gt;cost&lt;/em&gt;), which by the chain rule is...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\frac{\partial E}{\partial z}&#xA;=&#xA;\frac{\partial E}{\partial o}&#xA;\frac{\partial o}{\partial z}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expressed in matrix form...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\frac{\partial E}{\partial z}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial z_{1}}\\&#xA;\frac{\partial E}{\partial z_{2}}&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial o_{1}}&#xA;\frac{\partial o_{1}}{\partial z_{1}}\\&#xA;\frac{\partial E}{\partial o_{2}}&#xA;\frac{\partial o_{2}}{\partial z_{2}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(EQ4)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial o_{1}}&#xA;\\&#xA;\frac{\partial E}{\partial o_{2}}&#xA;\end{bmatrix}&#xA;\odot &#xA;\begin{bmatrix}&#xA;\frac{\partial o_{1}}{\partial z_{1}}&#xA;\\&#xA;\frac{\partial o_{2}}{\partial z_{2}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the use of the hadamard product here. In fact, given these are simply vectors, vector dot product would work, but hadamard is used as this becomes important later when using this expression with the matrix equations as we want to distinguish the use of hadamard product as opposed to matrix product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We start off with our first delta error, which for the first layer &lt;strong&gt;back propagation&lt;/strong&gt; is...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\delta ^{L} = \frac{\partial E}{\partial o}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then we want to calculate the next delta error using the formula (EQ1)...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\delta ^{l} = (w^{l+1})^{T} \delta ^{l+1} \odot \sigma {}' (z ^{l}) = \frac{\partial E}{\partial h}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explicitly, the equations are...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial h_{1}}=\frac{\partial E}{\partial z_{1}} w_{1} + \frac{\partial E}{\partial z_{2}} w_{2}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial h_{2}}=\frac{\partial E}{\partial z_{1}} w_{3} + \frac{\partial E}{\partial z_{2}} w_{4}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial h_{3}}=\frac{\partial E}{\partial z_{1}} w_{5} + \frac{\partial E}{\partial z_{2}} w_{6}\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also given the transpose of the weight matrix which allows us to &lt;strong&gt;back propagate&lt;/strong&gt; from a layer of 2 neurons to a layer of 3 neurons (via matrix multiplication) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\left (w  \right )^{T}&#xA;=&#xA;\left ( \begin{bmatrix}&#xA;w_{1} &amp;amp; w_{3} &amp;amp; w_{5}\\ &#xA;w_{2} &amp;amp; w_{4} &amp;amp; w_{6}&#xA;\end{bmatrix}&#xA; \right )^{T}&#xA;=&#xA;\begin{bmatrix}&#xA;w_{1} &amp;amp; w_{2}\\ &#xA;w_{3} &amp;amp; w_{4}\\ &#xA;w_{5} &amp;amp; w_{6}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in a similar way to how we represented the forward pass (EQ3), this can be represented in matrix form...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\frac{\partial E}{\partial h}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial h_{1}}\\ &#xA;\frac{\partial E}{\partial h_{2}}\\ &#xA;\frac{\partial E}{\partial h_{3}}&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial z_{1}} w_{1} + \frac{\partial E}{\partial z_{2}} w_{2}&#xA;\\ &#xA;\frac{\partial E}{\partial z_{1}} w_{3} + \frac{\partial E}{\partial z_{2}} w_{4}&#xA;\\&#xA;\frac{\partial E}{\partial z_{1}} w_{5} + \frac{\partial E}{\partial z_{2}} w_{6}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(EQ5)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;=&#xA;\begin{bmatrix}&#xA;w_{1} &amp;amp; w_{2}\\ &#xA;w_{3} &amp;amp; w_{4}\\ &#xA;w_{5} &amp;amp; w_{6}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial z_{1}}&#xA;\\ &#xA;\frac{\partial E}{\partial z_{2}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then plugging the hadamard version of the delta (EQ4) into this, we get...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\begin{bmatrix}&#xA;w_{1} &amp;amp; w_{2}\\ &#xA;w_{3} &amp;amp; w_{4}\\ &#xA;w_{5} &amp;amp; w_{6}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial o_{1}}&#xA;\\&#xA;\frac{\partial E}{\partial o_{2}}&#xA;\end{bmatrix}&#xA;\odot &#xA;\begin{bmatrix}&#xA;\frac{\partial o_{1}}{\partial z_{1}}&#xA;\\&#xA;\frac{\partial o_{2}}{\partial z_{2}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;aka (EQ1) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}(w^{l+1})^{T} \delta ^{l+1} \odot \sigma {}' (z ^{l})\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And thus we have &lt;strong&gt;back propagated&lt;/strong&gt; from a layer of 2 neurons, to a layer of 3 neurons (via matrix multiplication) thanks to transpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness... the other aspect of the back propagation to uses matrices, is the delta weight matrix....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\frac{\partial E}{\partial w}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial w_{1}} &amp;amp; \frac{\partial E}{\partial w_{3}} &amp;amp; \frac{\partial E}{\partial w_{5}}\\ &#xA;\frac{\partial E}{\partial w_{2}} &amp;amp; \frac{\partial E}{\partial w_{4}} &amp;amp; \frac{\partial E}{\partial w_{6}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned before, you need to cache weight matrix and the activated layer output of a forward pass of the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar vein to (EQ3)...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We explcitily have the equations...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w_{1}}=\frac{\partial E}{\partial z_{1}}h_{1}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w_{2}}=\frac{\partial E}{\partial z_{2}}h_{1}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w_{3}}=\frac{\partial E}{\partial z_{1}}h_{2}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w_{4}}=\frac{\partial E}{\partial z_{2}}h_{2}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w_{5}}=\frac{\partial E}{\partial z_{1}}h_{3}\end{equation*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w_{6}}=\frac{\partial E}{\partial z_{2}}h_{3}\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the use of h1, h2 and h3, which are the activated outputs of the previous layer.. (or in the case of our example these are the inputs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which we represent in matrix form...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;\frac{\partial E}{\partial w}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial z_{1}}h_{1} &amp;amp; \frac{\partial E}{\partial z_{1}}h_{2} &amp;amp; \frac{\partial E}{\partial z_{1}}h_{3}\\ &#xA;\frac{\partial E}{\partial z_{2}}h_{1} &amp;amp; \frac{\partial E}{\partial z_{2}}h_{2} &amp;amp; \frac{\partial E}{\partial z_{2}}h_{3}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&#xA;(EQ6)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial z_{1}} \\ \frac{\partial E}{\partial z_{2}}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;h_{1} &amp;amp;&#xA;h_{2} &amp;amp; &#xA;h_{3}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which just so happens to be (EQ2) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}\frac{\partial E}{\partial w}=\delta ^{l}a^{l-1}\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;:)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the delta weight matrix and the original weight matrix are the same dimensions, it is trivia to apply the learning rate...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;w =&#xA;\begin{bmatrix}&#xA;w_{1} - \alpha \frac{\partial E}{\partial w_{1}} &amp;amp; w_{3} - \alpha \frac{\partial E}{\partial w_{3}} &amp;amp; w_{5} - \alpha \frac{\partial E}{\partial w_{5}}\\ &#xA;w_{2} - \alpha \frac{\partial E}{\partial w_{2}} &amp;amp; w_{4} - \alpha \frac{\partial E}{\partial w_{4}} &amp;amp; w_{6} - \alpha \frac{\partial E}{\partial w_{6}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA;=&#xA;\begin{bmatrix}w_{1}&amp;amp;w_{3}&amp;amp;w_{5}\\w_{2}&amp;amp;w_{4}&amp;amp;w_{6}\end{bmatrix}&#xA;-&#xA;\alpha&#xA;\begin{bmatrix}&#xA;\frac{\partial E}{\partial w_{1}} &amp;amp; \frac{\partial E}{\partial w_{3}} &amp;amp; \frac{\partial E}{\partial w_{5}}\\ &#xA;\frac{\partial E}{\partial w_{2}} &amp;amp; \frac{\partial E}{\partial w_{4}} &amp;amp; \frac{\partial E}{\partial w_{6}}&#xA;\end{bmatrix}&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*} = w - \alpha \frac{\partial E}{\partial w}\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll omit the equations for the bias, as these are easy to put in and don't require any other equations other than the ones presented here. &#xA;So in summary, the equations you want are (EQ3), (EQ5) and (EQ6) to use matrices, and the ability to change move between layers of differing number of neurons is the matrix transpose. Hope this helps, and let me know if you want me to expand on anything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be also important to note that you appear to be using MSE as the cost. This is perhaps not optimal for the MNIST data set which is a clasification. A better cost function to use would be &lt;a href=&quot;https://www.google.com/search?client=firefox-b-d&amp;amp;q=cross%20entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross entropy&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Happy propagating!&lt;/p&gt;&#xA;" OwnerUserId="32400" LastEditorUserId="32400" LastEditDate="2020-04-09T11:36:09.210" LastActivityDate="2020-04-09T11:36:09.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20109" PostTypeId="5" CreationDate="2020-04-09T03:20:41.640" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T03:20:41.640" LastActivityDate="2020-04-09T03:20:41.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20110" PostTypeId="4" CreationDate="2020-04-09T03:20:41.640" Score="0" Body="For questions related to dialogue systems, which are computer systems intended to converse with a human." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T03:20:41.640" LastActivityDate="2020-04-09T03:20:41.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20111" PostTypeId="5" CreationDate="2020-04-09T03:22:36.357" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T03:22:36.357" LastActivityDate="2020-04-09T03:22:36.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20112" PostTypeId="4" CreationDate="2020-04-09T03:22:36.357" Score="0" Body="For questions related to human-robot interaction (HRI), which is the study of interactions between humans and robots. HRI is a multidisciplinary field with contributions from human-computer interaction, artificial intelligence, robotics, natural language understanding, design, humanities, and social sciences." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T03:22:36.357" LastActivityDate="2020-04-09T03:22:36.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20113" PostTypeId="2" ParentId="20094" CreationDate="2020-04-09T04:38:03.273" Score="2" Body="&lt;p&gt;AI has been applied to programming (check out TabNine, my favorite autocomplete engine) although not in as robust a fashion as you describe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming requires a high level of abstract while AI is typically trained to solve a very specific task. Given thousands of examples of insert sort in Python I think a model could be trained (perhaps after autocomplete and syntax correction) figure it out. However at this point the field has not developed a more general intelligence that can apply the ideas of the algorithm to other problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addition based on comments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Big picture, training an algorithm to solve a general class of problems (say, web dev) requires a huge number of examples or an immense number of trials. Further, as the complexity of the problem grows the number of parameters necessary to build the model grows. Writing code is a very complex problem and would thus require a huge amount of data and a huge number of parameters making it totally infeasible with today's math and (because of how the math is solved) hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern AI is has a very simple goal: find the model that solves a problem optimally. If we could quickly search every possible model this would be simple. Fields like machine (deep) learning and reinforcement learning are concerned with finding a good solution in a reasonable amount of time. At this point no such solution exists for a problem of such complexity. &lt;/p&gt;&#xA;" OwnerUserId="26481" LastEditorUserId="26481" LastEditDate="2020-04-13T02:39:00.257" LastActivityDate="2020-04-13T02:39:00.257" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20114" PostTypeId="2" ParentId="17609" CreationDate="2020-04-09T05:26:40.953" Score="4" Body="&lt;p&gt;Even the first artificial neural network - Rosenblatt's perceptron [&lt;a href=&quot;https://psycnet.apa.org/doiLanding?doi=10.1037%2Fh0042519&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] had a discontinuous activation function. That network is in introductory chapters of many textbooks about AI. For example, &lt;em&gt;Michael Negnevitsky. Artificial intelligence: a guide to intelligent systems. Second Edition&lt;/em&gt; shows how to train such networks on pages 170-174.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error backpropagation algorithm can be modified to accommodate discontinuous activation functions. The details are in paper [&lt;a href=&quot;https://ieeexplore.ieee.org/document/51993&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]. That paper points out a possible application: training a neural network on micro-controllers. As the multiplication of the output of the previous layer &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt; by the weigth &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt; is exspensive, the author suggested to approximate it with a left shift by &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; bits (multiplication by &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt;) for the corresponding &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in which case the activation function is discontinuous (a staircase). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a neural network with discontinuous activation functions applied to a restoration of degraded images is in Ref. [&lt;a href=&quot;https://ieeexplore.ieee.org/document/4634149&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;]. Applications of recurrent neural networks with discontinuous activation functions to convex optimization problems are in Ref. [&lt;a href=&quot;https://www.worldscientific.com/doi/abs/10.1142/9789814280150_0004&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;]. Probably more examples can be found in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev. 1958 Nov; 65(6):386-408. PMID: 13602029 DOI: 10.1037/h0042519 &lt;/li&gt;&#xA;&lt;li&gt;Findlay, D.A. Training networks with discontinuous activation functions. 1989 First IEE International Conference on Artificial Neural Networks, (Conf. Publ. No. 313), London, UK, 1989, pp. 361-363.&lt;/li&gt;&#xA;&lt;li&gt;Ferreira, L. V.; Kaszkurewicz, E.; Bhaya, A. Image restoration using L1-norm regularization and a gradient-based neural network with discontinuous activation functions. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, 2008, pp. 2512-2519.  DOI: 10.1109/IJCNN.2008.4634149 &lt;/li&gt;&#xA;&lt;li&gt;Liu, Q.; Wang, J. Recurrent Neural Networks with Discontinuous Activation Functions for Convex Optimization. Integration of Swarm Intelligence and Artificial Neural Network, pp. 95-119 (2011) DOI: 10.1142/9789814280150_0004&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15524" LastActivityDate="2020-04-09T05:26:40.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20116" PostTypeId="1" CreationDate="2020-04-09T09:41:55.347" Score="2" ViewCount="28" Body="&lt;p&gt;In ML we often have to store a huge amount of values ranging from 0 to 1, mostly being probabilities. The most common data structure to do so seems to be a floating point? Indeed, the range of floating points is huge. This makes them imprecise in the desired interval and inefficient, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/48098291/what-data-type-should-be-used-for-a-probability&quot;&gt;This question&lt;/a&gt; suggests using the biggest integer value to represent a 1 and the smallest for 0. Also, it points to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Q_%28number_format%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q number format&lt;/a&gt;, where all bits can be chosen as fractional, which sounds very efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why have these data types still not found their ways into numpy, tensorflow etc.? Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-04-09T09:41:55.347" Title="What is the most efficient data type to store probabilities?" Tags="&lt;machine-learning&gt;&lt;probability&gt;&lt;probability-distribution&gt;&lt;efficiency&gt;&lt;storage&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20118" PostTypeId="1" CreationDate="2020-04-09T10:47:37.650" Score="5" ViewCount="96" Body="&lt;p&gt;Most reinforcement learning agents are trained in simulated environments. The goal is to maximize performance in (often) the same environment, preferably with a minimum amount of interactions. Having a good model of the environment allows to use planning and thus drastically improves the sample efficiency!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is the simulation not used for planning&lt;/strong&gt; in these cases? It is a sampling model of the environment, right? Can't we try multiple actions at each or some states, follow the current policy to look several steps ahead and finally choose the action with the best outcome? Shouldn't this allow us to find better actions more quickly compared to policy gradient updates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, our environment and the model are kind of identical and this seems to be the problem. Or is the good old curse of dimensionality to blame again? Please help me figure out, what I'm missing. &lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-12-28T05:33:31.280" Title="Isn't a simulation a great model for model-based reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;models&gt;&lt;planning&gt;&lt;model-based-methods&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20120" PostTypeId="1" CreationDate="2020-04-09T12:25:06.790" Score="1" ViewCount="55" Body="&lt;p&gt;The definition of MDL according to these &lt;a href=&quot;https://www.cs.cmu.edu/~aarti/Class/10704/lec13-MDL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The minimum description length (MDL) criteria in machine learning says that the best description of the data is given by the model which compresses it the best. Put another way, learning a model for the data or predicting it is about capturing the regularities in the data and any regularity in the data can be used to compress it. Thus, the more we can compress a data, the more we have learnt about it and the better we&#xA;  can predict it.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;MDL is also connected to Occam’s Razor used in machine learning which states that &quot;other things being equal, a simpler explanation is better than a more complex one.&quot; In MDL, the simplicity (or rather complexity) of a model is interpreted as the length of the code obtained when that model is used to compress&#xA;  the data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To put it in short according to MDL principle, we prefer predictors with relatively smaller description length (i.e can be described within a certain length) for a &lt;strong&gt;given&lt;/strong&gt; description language (this definition is without delving into exact technical details as it is not necessary to the question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since MDL is dependent on the description language we use, can we say feature engineering can cause a change in the selection of the predictor? &#xA;For example as this picture shows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RP8mX.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RP8mX.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, it seems that, in the first picture, we will require a longer description length predictor in Cartesian coordinates, as compared to a predictor in polar co-ordinates (just a single discerning radius needs to be specified). So, to me, it seems feature engineering changed the selection of the predictor to a relatively simple one (in the sense that it will have a shorter description length). Thus feature engineering has changed the description length required for out predictor. Did I make any wrong assumptions? If so why?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-04-09T16:25:21.360" LastActivityDate="2020-04-13T22:44:44.440" Title="Can feature engineering change the selection of the model according to the minimum description length?" Tags="&lt;computational-learning-theory&gt;&lt;vc-theory&gt;&lt;information-theory&gt;&lt;minimum-description-length&gt;&lt;feature-engineering&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20121" PostTypeId="1" CreationDate="2020-04-09T13:46:14.543" Score="3" ViewCount="177" Body="&lt;p&gt;Imagine a set of simple (non-self-intersecting) polygons given by the coordinate pairs of their vertices &lt;span class=&quot;math-container&quot;&gt;$[(x_1, y_1), (x_2, y_2), \dots,(x_n, y_n)]$&lt;/span&gt;. The polygons in the set have a different number of vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EMgLd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EMgLd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use machine learning to solve various supervised regression and classification problems for these polygons such as prediction of their areas, perimeters, coordinates of their centroids, whether a polygon is convex, whether its centroid is inside or outside, etc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most machine learning algorithms require inputs of the same size but my inputs have a different number of coordinates. This may probably be handled by recurrent neural networks. However, the coordinates of my input vectors can be circularly shifted without changing the meaning of the input. For example, &lt;span class=&quot;math-container&quot;&gt;$$[(x_1, y_1), (x_2, y_2),...,(x_n, y_n)]$$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$$[(x_n, y_n), (x_1, y_1),...,(x_{n-1}, y_{n-1})]$$&lt;/span&gt; represent the same polygon where a starting vertex is chosen differently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which machine learning algorithm is both invariant to a circular shifting of its input coordinates and can work with inputs of different sizes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, an algorithm could learn to split each polygon into non-overlapping triangles, calculate areas or perimeters of each triangle, and then aggregate these computations somewhere in the output layer. However, the labels (areas or perimeters) are given only for the whole polygons, not for the triangles. Also, the perimeter of the polygon is not the sum of the perimeters of the triangles. Is thinking about this problem in terms of triangles misleading?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please provide references on machine learning algorithms that solve such tasks? Or any advice, how to approach this task? It does not have to be neural network and does not have to learn exact analytic formulas. Approximate results would be enough.&lt;/p&gt;&#xA;" OwnerUserId="15524" LastEditorUserId="2444" LastEditDate="2020-04-09T14:40:56.373" LastActivityDate="2020-04-09T15:11:54.810" Title="How can I use machine learning to predict properties (such as the area) of simple polygons?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20125" PostTypeId="2" ParentId="20121" CreationDate="2020-04-09T15:11:54.810" Score="3" Body="&lt;p&gt;You can split each polygon into a collection of triangles and sum up the areas. Not really sure why you would bother with ML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway if you approximates these polygons as images you could maybe train a CNN. Look at the image classification networks which provide bounding boxes.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-09T15:11:54.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20126" PostTypeId="2" ParentId="20094" CreationDate="2020-04-09T15:26:14.213" Score="1" Body="&lt;p&gt;I am not an expert on this specific topic, but I can say a few words. I will use the term &amp;quot;programming&amp;quot; to refer to software development (of any kind).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If you are in the camp that this isn't that hard, why hasn't it become mainstream?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's definitely hard, otherwise, we would have already some useful artificial programmers.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why is creating AI that can code a hard task?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Programming is actually a hard task because it often requires creativity and a deep understanding of the context(s), goal(s), programming languages, etc. In other words, it's a very complex task (even for humans), apart from the exceptions where you can copy and paste.&lt;/p&gt;&#xA;&lt;p&gt;Programming can probably be considered &lt;a href=&quot;https://ai.stackexchange.com/a/12147/2444&quot;&gt;an AI-complete problem&lt;/a&gt;, i.e. a problem that probably requires an AGI to be solved. In other words, if an AI was as capable as humans in terms of programming, then that probably means it is an AGI (but this is not guaranteed), i.e. programming is a task that probably requires general intelligence. This is why I say that programming is an AI-complete problem. However, note that being able to program is just &lt;a href=&quot;https://en.wikipedia.org/wiki/Necessity_and_sufficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;a necessary (but not sufficient)&lt;/a&gt; ability that an AI needs to possess in order for it to be an AGI (although not all general intelligences, e.g. animals, may be able to develop software, but the definition of general intelligence is also fuzzy).&lt;/p&gt;&#xA;&lt;p&gt;AFAIK, no AGI has yet been created, and I think we are still very far away from that goal. Currently, most AI systems are only able to tackle a specific problem (i.e. we only have narrow AIs, such as AlphaGo). You could say that programming is a very specific problem too, but this is misleading or wrong, because, unless you just want to develop very specific programs in a very limited context (and there are already machine learning models and approaches, such as &lt;a href=&quot;https://arxiv.org/pdf/1511.06279.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;neural programmer-interpreters&lt;/strong&gt;&lt;/a&gt; and &lt;strong&gt;genetic programming&lt;/strong&gt; respectively, that can do this to some extent; see the answers to &lt;a href=&quot;https://ai.stackexchange.com/q/7364/2444&quot;&gt;this question&lt;/a&gt; for other examples), then you will need to know a lot about other contexts too. For example, consider the task of developing a program that can detect signs of cancer in images. To develop this program, the AI would need to have the knowledge of an AI engineer, doctor, etc.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, programming often requires common-sense knowledge. For example, while reading the software specifications, the AI needs to interpret them in the way that they were originally meant to be interpreted. This also suggests that &lt;a href=&quot;https://ai.stackexchange.com/a/13262/2444&quot;&gt;programming requires an AGI (or human-level AI)&lt;/a&gt; to be solved.&lt;/p&gt;&#xA;&lt;p&gt;(Finally, to address a comment, note that writing a 4-line program is not equivalent to writing a 10-line program. Also, the length of the program often doesn't correspond to its difficulty or complexity, so that alone is not a good measure of the ability to program.)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are the barriers/problems we have to solve before we can solve this problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think that the answer to this question is also the answer to the question &amp;quot;How can we create an AGI?&amp;quot;. However, to be more concrete, I think that, in order to be able to create an AI that is able to program as well as humans, we will need to be able to create an AI that is able to think about low- and high-level concepts, compose them and it will probably require common-sense knowledge (so knowledge representation). A typical supervised learning solution will not be enough to solve this task. See the paper &lt;a href=&quot;https://arxiv.org/pdf/1901.02918.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Making AI Meaningful Again&lt;/a&gt;, which also suggests that ML-based solutions may not be enough to solve many tasks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T14:26:44.777" LastActivityDate="2020-12-07T14:26:44.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20127" PostTypeId="1" AcceptedAnswerId="20135" CreationDate="2020-04-09T15:52:12.023" Score="3" ViewCount="216" Body="&lt;p&gt;So this is my current result (loss and score per episode) of my RL model in a simple two players game:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K7YTe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K7YTe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use DQN with CNN as a policy and target networks. I train my model using Adam optimizer and calculate the loss using Smooth L1 Loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a normal &quot;Supervised Learning&quot; situation, I can deduce that my model is overfitting. And I can imagine some methods to tackle this problem (e.g. Dropout layer, Regularization, Smaller Learning Rate, Early Stopping). But is that solution will also work in RL problem? Or are there any better solutions to handle overfitting in RL?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="2444" LastEditDate="2020-04-09T16:37:37.333" LastActivityDate="2020-11-23T15:38:49.910" Title="How can I handle overfitting in reinforcement learning problems?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;overfitting&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20128" PostTypeId="1" CreationDate="2020-04-09T16:00:03.017" Score="0" ViewCount="35" Body="&lt;p&gt;Neural Network is trained to learn a non-linear function, the more layers it has, the more is the quality of the prediction and the ability to match the real-world function correctly (lets leave aside overfitting for now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given that the concept of a derivative of a function (if it is not a line) is only defined at a single point , and worse than that, it is defined as a tangent line (so, its linear in essence) , and backpropagation uses this tangent line to project weight updates, does this mean that Gradient Descent is linear at its fundamental level ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then if Gradient Descent is using linearity to propagate back weight updates, imagine how big the error would be for non linear function? For example , you  have calculated that the weight number 129 of your network has to be decreased by 0.003412, but at that point with this new weight, the function might have already reversed its direction, and the real update for this weight must be a negative number!!! Isn't this the reason that our deep fully connected networks have such a difficulty to learn , because with more layers stacked up, the more non-linear the model becomes, and thus, the weight updates we propagate back to lower layers could be treated as &quot;best guess&quot; values instead of something that can be fully trusted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am correct in assuming that Gradient Descent is not calculating correct weight updates on each backwards step and that the only reason that the network eventually converges to required model is because these imprecisions are fixed in a loop (called epoch). So, if we use an analogy, Gradient Descent would be like navigating the World on a boat with maps developed with the assumption that the Earth is flat. With such maps, you can sail in near-by areas, but if you would travel around the world without knowing that the Earth is round you will never arrive to your destination, and that's exactly what we are experiencing when we train deep fully connected networks without making them converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means, if Gradient Descent is broken in such a way, a correct Gradient Descent algorithm would only have to do SINGLE backwards step and update all the weights in one pass, giving the minimum error that is theoretically possible in 1 epoch .... I am right ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question basically is: is Gradient Descent a really broken algorithm or I am missing something?&lt;/p&gt;&#xA;" OwnerUserId="34084" LastActivityDate="2020-04-09T16:00:03.017" Title="What is the degree of linearity in the error propagated by Gradient Descent?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20129" PostTypeId="5" CreationDate="2020-04-09T16:09:28.390" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T16:09:28.390" LastActivityDate="2020-04-09T16:09:28.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20130" PostTypeId="4" CreationDate="2020-04-09T16:09:28.390" Score="0" Body="For questions related to information theory in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T16:09:28.390" LastActivityDate="2020-04-09T16:09:28.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20131" PostTypeId="5" CreationDate="2020-04-09T16:11:02.880" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T16:11:02.880" LastActivityDate="2020-04-09T16:11:02.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20132" PostTypeId="4" CreationDate="2020-04-09T16:11:02.880" Score="0" Body="For questions related to the minimum description length (MDL) principle, which is a formalization of Occam's razor in which the best hypothesis (a model and its parameters) for a given set of data is the one that leads to the best compression of the data. MDL was introduced in the paper &quot;Modeling by shortest data description&quot; by Jorma Rissanen in 1978." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T16:11:02.880" LastActivityDate="2020-04-09T16:11:02.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20133" PostTypeId="5" CreationDate="2020-04-09T16:15:16.470" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T16:15:16.470" LastActivityDate="2020-04-09T16:15:16.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20134" PostTypeId="4" CreationDate="2020-04-09T16:15:16.470" Score="0" Body="For questions related to feature engineering, which is the process of using domain knowledge to extract features from raw data via data mining techniques." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-09T16:15:16.470" LastActivityDate="2020-04-09T16:15:16.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20135" PostTypeId="2" ParentId="20127" CreationDate="2020-04-09T16:18:38.293" Score="2" Body="&lt;p&gt;Overfitting refers to a model being stuck in a local minimum while trying to minimise a loss function. In Reinforcement Learning the aim is to learn an optimal policy by maximising (not always but almost) a reward function rather than minimising a loss function, so you can't really talk of overfitting as in the supervised scenario, but you can definitely talk about &lt;em&gt;sub-optimal policies&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If we think of a specific task like avoiding stationary objects, a simple sub-optimal policy would be to just stay still without moving at all, or moving in circles if the reward function was designed to penalise lack of movements.&lt;/p&gt;&#xA;&lt;p&gt;The way to avoid an agent to learn sub-optimal policies is to find a good compromise between &lt;em&gt;exploitation&lt;/em&gt;, i.e. the constant selection of the next action to take based on the maximum expected reward possible, and &lt;em&gt;exploration&lt;/em&gt;, i.e. a random selection of the next action to take regardless of the rewards. Here's a link to an introduction to the topic:&#xA;&lt;a href=&quot;https://www.manifold.ai/exploration-vs-exploitation-in-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exploration and Exploitation in Reinforcement Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is worth mentioning that sometimes an agent can actually outsmart humans though, some examples are reported in this paper &lt;a href=&quot;https://arxiv.org/pdf/1803.03453.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Surprising Creativity of Digital Evolution&lt;/a&gt;. I particularly like the story of the insect agent trained to learn to walk while minimising the contact with the floor surface. The agent surprisingly managed to learn to walk without touching the ground at all. When the authors checked what was going on they discovered that the insect leaned to flip itself and then walk using its fake 'elbows' (fig7 in the linked paper). I add this story just to point out that most of the time the design of the reward function is itself even more important than exploration and exploitation tuning.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="2444" LastEditDate="2020-11-23T12:24:44.323" LastActivityDate="2020-11-23T12:24:44.323" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20136" PostTypeId="1" CreationDate="2020-04-09T17:09:53.540" Score="1" ViewCount="1741" Body="&lt;p&gt;I am trying to train a LSTM, but I have some problems regarding the data representation and feeding it into the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data is a numpy array of three dimensions: One sample consist of a 2D matrix of size (600,5). 600(timesteps) and 5(features). However, I have 160 samples or files that represent the behavior of a user in multiple days. Altogether, my data has a dimension of (160,600,5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The label set is an array of 600 elements which describes certain patterns of each 2D matrix. The shape of the output should be (600,1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I train the LSTM to the corresponding label set? What would be the best approach to handle this problem? The idea is that the output should be an array of (600,1) with 3 label inside.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Multiple_outputs {0,1,2}&#xA;      Output:    0000000001111111110000022222220000000000000&#xA;                 -------------600 samples ------------------&#xA;&#xA;Input: (1, 600, 5) &#xA;Output: (600, 1) &#xA;Training: (160,600,5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I look forward for some ideas!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dataset(160,600,5)&#xA;&#xA;X_train, X_test, y_train, y_test = train_test_split(dataset[:,:,0:4], dataset[:,:,4:5],test_size = 0.30)&#xA;&#xA;model = Sequential()&#xA;model.add(InputLayer(batch_input_shape = (92,600,5 )))&#xA;model.add(Embedding(600, 128))&#xA;#model.add(Bidirectional(LSTM(256, return_sequences=True)))&#xA;model.add(TimeDistributed(Dense(2)))&#xA;model.add(Activation('softmax'))&#xA;&#xA;model.compile(loss='categorical_crossentropy',&#xA;              optimizer=Adam(0.001),&#xA;              metrics=['accuracy'])&#xA;&#xA;model.summary()&#xA;&#xA;&#xA;model.fit(X_train,y_train, batch_size=92, epochs=40, validation_split=0.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35763" LastEditorUserId="35763" LastEditDate="2020-04-10T12:08:44.767" LastActivityDate="2020-04-10T12:08:44.767" Title="How to train a LSTM with multidimensional data" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;multi-label-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20137" PostTypeId="2" ParentId="18773" CreationDate="2020-04-09T17:10:15.387" Score="0" Body="&lt;p&gt;I assume you're referring to &lt;a href=&quot;https://arxiv.org/pdf/1910.13461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which basically &quot;reuses&quot; the same architecture as the original &quot;Attention is all you need&quot; paper for machine translation; the difference being that source and target are noisy and original sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't figure the for loop logic inside your &lt;code&gt;train&lt;/code&gt; function; but reading your comment, you don't have to traverse the target sequentially. Instead, we rely on masks which exclude paddings, and most importantly help with implementing teacher forcing for the decoder stack. Check &lt;a href=&quot;https://andrewpeng.dev/transformer-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; and how it creates the three types of masks needed (src, target, &lt;strong&gt;no_peek&lt;/strong&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, you will need a for loop when serving your model (inference), because you will have to decode autoregressively. That is, you'll have to decode a token first, and feed it back to the decoder to generate the next token (until you reach &lt;code&gt;EOS&lt;/code&gt;). Check &quot;Testing the model&quot; section in &lt;a href=&quot;https://towardsdatascience.com/how-to-code-the-transformer-in-pytorch-24db27c8f9ec&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="34315" LastActivityDate="2020-04-09T17:10:15.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20138" PostTypeId="1" CreationDate="2020-04-09T17:51:33.530" Score="3" ViewCount="208" Body="&lt;p&gt;I have a vectorized implementation of the neural network in c++. I successfully solve the classification problems of Fashion MNIST and CIFAR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am modifying my code to do the Linear regression. I am stuck at a point. I have to use MSE loss function here instead of squared error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&#xA;1) In linear regression tasks; is there a difference between MSE and squared error(There is only a difference of mean value which means dividing by the number of mini-batch...according to my understanding)? &#xA;2) My C++ implementation of this network is given below, to implement MSE shall I modify my loss function line and divide it by batch size?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__global__ void loss(double* X, double* Y, double *Z, size_t n) {&#xA;&#xA;    size_t index = blockIdx.x * blockDim.x + threadIdx.x;&#xA;&#xA;    if (index &amp;lt; n) {&#xA;        Z[index] = ((X[index] - Y[index]));&#xA;    }&#xA;} &#xA;void forward_prop(){&#xA;    L_F( b1, b_x, w1, a1, BATCH_SIZE, layer_0_nodes, layer_1_nodes );&#xA;    tan_h(a1, BATCH_SIZE, layer_1_nodes);&#xA;&#xA;    L_F( b2, a1, w2, a2, BATCH_SIZE, layer_1_nodes, layer_2_nodes );&#xA;    tan_h(a2, BATCH_SIZE, layer_2_nodes);&#xA;&#xA;    L_F( b3, a2, w3, a3, BATCH_SIZE, layer_2_nodes, layer_3_nodes );&#xA;}&#xA;void backward_prop(){&#xA;&#xA;    cuda_loss(a3, b_y, loss_m, BATCH_SIZE, layer_3_nodes);&#xA;    L_B(a2, loss_m, dw3, layer_2_nodes, BATCH_SIZE, layer_3_nodes, true);&#xA;    tan_h_B(a2, BATCH_SIZE, layer_2_nodes);&#xA;&#xA;    L_B(loss_m, w3, dz2, BATCH_SIZE, layer_3_nodes, layer_2_nodes, false);&#xA;    cuda_simple_dot_ab(dz2, a2, BATCH_SIZE, layer_2_nodes);&#xA;    L_B(a1, dz2, dw2, layer_1_nodes, BATCH_SIZE, layer_2_nodes, true);&#xA;    tan_h_B(a1, BATCH_SIZE, layer_1_nodes);&#xA;&#xA;    L_B(dz2, w2, dz1, BATCH_SIZE, layer_2_nodes, layer_1_nodes, false);&#xA;    cuda_simple_dot_ab(dz1, a1, BATCH_SIZE, layer_1_nodes);&#xA;    L_B(b_x, dz1, dw1, layer_0_nodes, BATCH_SIZE, layer_1_nodes, true);&#xA;}&#xA;__global__ void linearLayerForward( double *b, double* W, double* A, double* Z, size_t W_x_dim, size_t W_y_dim, size_t A_x_dim) {&#xA;&#xA;    size_t row = blockIdx.y * blockDim.y + threadIdx.y;&#xA;    size_t col = blockIdx.x * blockDim.x + threadIdx.x;&#xA;&#xA;    size_t Z_x_dim = A_x_dim;&#xA;    size_t Z_y_dim = W_y_dim;&#xA;&#xA;    double Z_value = 0;&#xA;&#xA;    if (row &amp;lt; Z_y_dim &amp;amp;&amp;amp; col &amp;lt; Z_x_dim) {&#xA;        for (size_t i = 0; i &amp;lt; W_x_dim; i++) {&#xA;            Z_value += W[row * W_x_dim + i] * A[i * A_x_dim + col];&#xA;        }&#xA;        Z[row * Z_x_dim + col] = Z_value + b[col];&#xA;    }&#xA;}&#xA;&#xA;__global__ void linearLayerBackprop(double* W, double* dZ, double *dA,&#xA;                                    size_t W_x_dim, size_t W_y_dim,&#xA;                                    size_t dZ_x_dim) {&#xA;&#xA;    size_t col = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    size_t row = blockIdx.y * blockDim.y + threadIdx.y;&#xA;&#xA;    // W is treated as transposed&#xA;    size_t dA_x_dim = dZ_x_dim;&#xA;    size_t dA_y_dim = W_x_dim;&#xA;&#xA;    double dA_value = 0.0f;&#xA;&#xA;    if (row &amp;lt; dA_y_dim &amp;amp;&amp;amp; col &amp;lt; dA_x_dim) {&#xA;        for (size_t i = 0; i &amp;lt; W_y_dim; i++) {&#xA;            dA_value += W[i * W_x_dim + row] * dZ[i * dZ_x_dim + col];&#xA;        }&#xA;        dA[row * dA_x_dim + col] = dA_value;&#xA;    }&#xA;}&#xA;&#xA;&#xA;__global__ void tanhActivationForward(double* Z, size_t n) {&#xA;&#xA;    size_t index = blockIdx.x * blockDim.x + threadIdx.x;&#xA;&#xA;    if (index &amp;lt; n) {&#xA;            Z[index] = tanh(Z[index]);&#xA;    }&#xA;}&#xA;&#xA;__global__ void tanhActivationBackward(double* Z, size_t n) {&#xA;&#xA;    size_t index = blockIdx.x * blockDim.x + threadIdx.x;&#xA;&#xA;    if (index &amp;lt; n) {&#xA;            Z[index] = 1-(tanh(Z[index]) * tanh(Z[index]));&#xA;    }&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29724" LastActivityDate="2020-04-09T17:51:33.530" Title="How to implement Mean square error loss function in mini batch GD" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;linear-regression&gt;&lt;mean-squared-error&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20139" PostTypeId="1" CreationDate="2020-04-09T18:41:25.763" Score="1" ViewCount="16" Body="&lt;p&gt;I am on &lt;a href=&quot;https://cs330.stanford.edu/slides/cs330_lec2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lecture 2&lt;/a&gt; of &lt;a href=&quot;https://cs330.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford CS330 Multi-Task and Meta-learning&lt;/a&gt;, and on slide 10, the professor describes using a one-hot input vector to represent the task, and she also explained that there would be independent weight matrices for each task&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How is the input to a multi-task network encoded to allow the features of all the tasks to be associated with different weights?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you have an input vector containing all the features for every task, and then multiply the input vectors by the task ID vector? Is there a more efficient way to approach this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other terms, here’s what I’m thinking:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;network_input[i] = features[i] * task[i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;features&lt;/code&gt; is a 2d matrix of feature vectors for every task, and &lt;code&gt;task&lt;/code&gt; is a one-hot vector corresponding to the task number. Is that multiplicative conditioning?&lt;/p&gt;&#xA;" OwnerUserId="4744" LastEditorUserId="4744" LastEditDate="2020-04-10T16:15:03.940" LastActivityDate="2020-04-10T16:15:03.940" Title="How do I format task features with a one-hot task identification vector to ensure separate weight matrices for each task in multi-task RL?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;meta-learning&gt;&lt;multi-task-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20141" PostTypeId="1" CreationDate="2020-04-09T21:33:51.727" Score="3" ViewCount="438" Body="&lt;p&gt;I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder model, with and without attention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is All You Need&lt;/a&gt; by Vaswani et. al in 2017 states the forward pass cost is &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;, which makes sense to me (with 1 hidden layer). In terms of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; the input length, and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; the output length, it then looks like &lt;span class=&quot;math-container&quot;&gt;$O(X^3 + Y^3)$&lt;/span&gt;, which I understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for training, it seems to me like one back-propagation is at worst &lt;span class=&quot;math-container&quot;&gt;$O(X^3 + Y^3)$&lt;/span&gt;, and we do &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; of them, so &lt;span class=&quot;math-container&quot;&gt;$O(Y(X^3  Y^3))$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the following diagram, where the green blocks are the hidden states, the red ones are the input text and the blue ones are output text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iJImA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iJImA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to add global attention, as introduced by &lt;a href=&quot;https://arxiv.org/pdf/1508.04025.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Luong et. al in 2015&lt;/a&gt;, the attention adds an extra &lt;span class=&quot;math-container&quot;&gt;$X^2$&lt;/span&gt; in there due to attention multiplication, to make an overall inference of &lt;span class=&quot;math-container&quot;&gt;$O(X^3 + X^2 Y^3)$&lt;/span&gt;, and training even worse at &lt;span class=&quot;math-container&quot;&gt;$O(XY(X^3 + X^2 Y^3))$&lt;/span&gt; since it needs to learn attention weights too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following diagram shows the sequence-to-sequence model with attention, &#xA;where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;'s are the hidden states, &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the context vector and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the output word, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; of such output words and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; such inputs. This setup is described in the paper &lt;a href=&quot;https://arxiv.org/pdf/1508.04025.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Effective Approaches to Attention-based Neural Machine Translation&lt;/a&gt;, by Luong et al. in 2015.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ybKiS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ybKiS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my intuition correct?&lt;/p&gt;&#xA;" OwnerUserId="35867" LastEditorUserId="2444" LastEditDate="2020-10-10T22:09:00.237" LastActivityDate="2020-10-10T22:09:00.237" Title="What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?" Tags="&lt;recurrent-neural-networks&gt;&lt;transformer&gt;&lt;time-complexity&gt;&lt;forward-pass&gt;&lt;seq2seq&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20142" PostTypeId="1" CreationDate="2020-04-09T23:21:30.680" Score="3" ViewCount="240" Body="&lt;p&gt;I am trying to do classification using &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT-python&lt;/a&gt; for the first time, and I am having difficulty getting the accuracy rate. I tried the same problem with an ANN and was able to get a good accuracy rate (96%+), but NEAT-Python gives barely 40%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's how I set up:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;: Train 100 probability values to predict classification (1-10)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Input and output setup:&lt;/strong&gt; inputs = number of input shape (100 values of prob), and output is 10 values of probability assoc with 10 classes. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Activation&lt;/strong&gt;: I applied ReLU for feedforward, then applied&#xA;softmax&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Fitness function&lt;/strong&gt;: I used the loglikelihood. I was unsure about how to set up the fitness function. I also used mean accuracy rate in the genome. Both gave similar results.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In terms of hyperparameters, I am trying various values and haven't had any luck with it. Today I am trying with an increase in population size and generations. I have another feature input that can be used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any resources that discuss how to handle mixed data for NEAT?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="35845" LastEditorUserId="35845" LastEditDate="2020-04-13T16:52:17.980" LastActivityDate="2020-04-13T16:52:17.980" Title="How to perform classification with NEAT-Python?" Tags="&lt;python&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;fitness-functions&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20143" PostTypeId="1" CreationDate="2020-04-10T00:15:31.633" Score="1" ViewCount="46" Body="&lt;p&gt;I am a records manager and I am being asked if I recommend Office 365.  I'm having a hard time making a recommendation because I am missing an essential piece of information: can Office 365 replace the manual process of placing records into categories based on organizational function?  It is important that this is done accurately, because the category determines how long the records should be kept before they are irretrievably destroyed.  James Lappin seems to say that yes, Office 365 is underwritten by &lt;a href=&quot;https://thinkingrecords.co.uk/2020/03/22/project-cortex-and-the-future-of-document-management-in-office-365/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Project Cortex&lt;/a&gt;, and it is capable of doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My sense is that artificial intelligence is not yet capable of determining a conceptual category for records.  For this to be true, a machine would have to replicate a complex human process: reading a free-text, free-form document; identifying the relevant pieces of information in a document, while ignoring others to determine what the document is &quot;about&quot;; then taking the answer of what the document is &quot;about&quot; and matching it to a predefined set of major organizational activity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any AI experts who can comment on how realistic it is to expect Project Cortex to do this?&lt;/p&gt;&#xA;" OwnerUserId="35892" LastActivityDate="2020-04-11T00:38:26.703" Title="Can artificial intelligence classify textual records?" Tags="&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20144" PostTypeId="2" ParentId="20136" CreationDate="2020-04-10T00:18:47.683" Score="1" Body="&lt;p&gt;I don't see any special characteristic in the problem you're posing. Any LSTM can handle multidimensional inputs (i.e. multiple features). You just need to prepare your data such as they will have shape &lt;strong&gt;[batch_size, time_steps, n_features]&lt;/strong&gt;, which is the format required by all main DL libraries (pytorch, keras and tensorflow). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I linked below 2 tutorials that shows how to implement an LSTM for part of speech tagging in Keras and Pytorch. This task is conceptually identical to what you want to achieve: use 2D inputs (i.e. embeddings) to predict the class (i.e. the pos tags) of each element of a sequence (i.e. every single word). For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Input sample, 4 steps, encoded in 2D embeddings matrix&#xA;['we', 'had', 'crispy', 'fries']&#xA;&#xA;# Output predictions, 4 labels&#xA;['PRP', 'VBD', 'JJ', 'NNS']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In your case the predicted label would be 0, 1 or 2 and you would have to encode your data in a matrix of shape &lt;strong&gt;[n_batch, 600, 5]&lt;/strong&gt;. The only thing you definetly want to pay attention, since you're using temporal data, is to not shuffle your data at all before the training. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keras tutorial: &lt;a href=&quot;https://nlpforhackers.io/lstm-pos-tagger-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Build a POS tagger with an LSTM using Keras&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pytorch tutorial: &lt;a href=&quot;https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM’s in Pytorch&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-04-10T00:18:47.683" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20145" PostTypeId="1" CreationDate="2020-04-10T00:19:38.380" Score="1" ViewCount="98" Body="&lt;p&gt;Lately, I have implemented &lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN&lt;/a&gt; for Atari Breakout. Here is the code: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JeremieGauthier/AI_Exercices/blob/master/Atari_Breakout/DQN_Breakout.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JeremieGauthier/AI_Exercices/blob/master/Atari_Breakout/DQN_Breakout.py&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have trained the agent for over 1500 episodes, but the training leveled off at around 5 as score. Could someone look at the code and point out a few things that should be corrected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hyvYK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hyvYK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, the training is not going more than 5 in average score. Is there a way to improve my performance?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-04-10T13:53:49.543" LastActivityDate="2020-04-10T13:53:49.543" Title="Why isn't my DQN agent improving when trained on Atari Breakout?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20146" PostTypeId="1" AcceptedAnswerId="20147" CreationDate="2020-04-10T02:36:50.317" Score="1" ViewCount="155" Body="&lt;p&gt;I see some papers use the term &quot;natural image domain&quot;. I googled that but didn't find any explanation of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I understand the normal meaning of &quot;natural image&quot;, such as the image people take by phones. The images in ImageNet database are all natural images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &quot;natural image domain&quot; a subfield of computer vision?&lt;/p&gt;&#xA;" OwnerUserId="35896" LastEditorUserId="2444" LastEditDate="2020-04-10T03:38:20.537" LastActivityDate="2020-04-10T12:54:58.830" Title="What is &quot;natural image domain&quot;?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20147" PostTypeId="2" ParentId="20146" CreationDate="2020-04-10T07:13:28.133" Score="0" Body="&lt;p&gt;An image of a natural domain refers to any kind of image that has some variance in its structure, so it does not always present the same peculiarities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in a digits classification task, you can train your network on the MNIST dataset, that is a simple black &amp;amp; white set of images that are always well defined: the digit is in darker pixels, and the background is always white, they just have different shapes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the same task but in the natural image domain, the digits may vary in color, noise, and even position inside of a single image. As you may imagine this does not apply to digits only, but it includes any image that was not specifically created or modified to be fed into a neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to put it simply, a neural network trained for natural images is able to perform classification in any real-life situation or scenario, without the need to edit the image to adapt it to what the network was trained for (for example, changing the color or cropping so the object you want to recognize is exactly in the center).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/q/25737/82135&quot;&gt;Here&lt;/a&gt; a similar question that may fill some doubts you may have.&lt;/p&gt;&#xA;" OwnerUserId="35792" LastEditorUserId="2444" LastEditDate="2020-04-10T12:54:58.830" LastActivityDate="2020-04-10T12:54:58.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20149" PostTypeId="1" CreationDate="2020-04-10T09:39:23.753" Score="3" ViewCount="37" Body="&lt;p&gt;I implemented an &lt;code&gt;LSTM&lt;/code&gt; with &lt;code&gt;Keras&lt;/code&gt; to perform word ordering task (given a syntactically unordered sentence, the goal is to label each word of the sentence with the right position in this one.)&#xA;So, my dataset is composed by numerical vectors and each numerical vector represents a word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I train my model trying to learn the local order of a syntactic subtree composed by words that have syntactic relationships (for example, a subtree could be a set of three words in which the root is the verb and children are subject and object relationship).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I padded each subtree to a length of 20, which is the maximum subtree length that I found in my dataset. With padding introduction, I inserted a lot of vectors composed of only zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial dataset shape is &lt;code&gt;(700000, 837)&lt;/code&gt;, but knowing that &lt;code&gt;Keras&lt;/code&gt; wants a 3D dataset, I reshaped it to &lt;code&gt;(35000, 20, 837)&lt;/code&gt; and the same for my labels (from 700000 to &lt;code&gt;(35000, 20)&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As loss function, I'm using the &lt;code&gt;ListNet&lt;/code&gt; algorithm loss function, which takes a list of words and for each computes the probability of the element to be ranked in the first position (then ranking these scores, I obtain the predicted labels of each word).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current implementation is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential()&#xA;model.add(LSTM(units=100, activation='tanh', return_sequences=True, input_shape=(timesteps, features)))&#xA;model.add(Dense(1, activation='sigmoid'))&#xA;&#xA;model.summary()&#xA;&#xA;model.compile(loss=listnet_loss, optimizer=keras.optimizers.Adam(learning_rate=0.00005, beta_1=0.9, beta_2=0.999, amsgrad=True), metrics=[&quot;accuracy&quot;])&#xA;&#xA;model.fit(training_dataset, training_dataset_labels, batch_size=1, epochs=number_of_epochs, workers=10, verbose=1, callbacks=[SaveModelCallback()])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And &lt;code&gt;SaveModelCallback&lt;/code&gt; simply saves each model during training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I obtain, at each epoch, very very similar results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 21/50&#xA;39200/39200 [==============================] - 363s 9ms/step - loss: 2.5483 - accuracy: 0.8246&#xA;Epoch 22/50&#xA;39200/39200 [==============================] - 359s 9ms/step - loss: 2.5480 - accuracy: 0.8245&#xA;Epoch 23/50&#xA;39200/39200 [==============================] - 360s 9ms/step - loss: 2.5478 - accuracy: 0.8246&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have to questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Could zero-padding affect learning in a negative way? And if yes, how could we not consider this padding?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it a good model for what I have to do?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33440" LastEditorUserId="2444" LastEditDate="2020-04-10T12:51:10.417" LastActivityDate="2020-04-10T12:51:10.417" Title="Could zero-padding affect learning in a negative way?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20150" PostTypeId="1" AcceptedAnswerId="20155" CreationDate="2020-04-10T09:46:26.767" Score="5" ViewCount="254" Body="&lt;p&gt;I know the most basic rudimentary theory on AI, and I want to delve into actual practical coding with AI and machine learning. I already know a decent bit of coding in C++ and I'm learning Python syntax now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I want to start implementing artificial intelligence techniques for simple games (like snake or maybe chess, which isn't really a simple game, but I know a lot about it), and then move on to more complex methods and algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what are some resources (e.g. tutorials, guides, books, etc.) for coding some artificial intelligence techniques in the context of games?&lt;/p&gt;&#xA;" OwnerUserId="31156" LastEditorUserId="2444" LastEditDate="2020-05-22T18:08:45.820" LastActivityDate="2020-05-22T18:08:45.820" Title="What are some resources for coding some artificial intelligence techniques in the context of games?" Tags="&lt;machine-learning&gt;&lt;game-ai&gt;&lt;reference-request&gt;&lt;chess&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20153" PostTypeId="5" CreationDate="2020-04-10T14:52:33.773" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-10T14:52:33.773" LastActivityDate="2020-04-10T14:52:33.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20154" PostTypeId="4" CreationDate="2020-04-10T14:52:33.773" Score="0" Body="For questions related to multi-task learning (MTL), which is a subfield of machine learning in which multiple learning tasks are solved at the same time, while exploiting commonalities and differences across tasks.  MTL can result in improved learning efficiency and prediction accuracy for the task-specific models, when compared to training the models separately." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-10T14:52:33.773" LastActivityDate="2020-04-10T14:52:33.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20155" PostTypeId="2" ParentId="20150" CreationDate="2020-04-10T15:21:27.180" Score="2" Body="&lt;p&gt;One of the simplest games you could solve with an AI technique is &lt;a href=&quot;https://en.wikipedia.org/wiki/Tic-tac-toe&quot; rel=&quot;nofollow noreferrer&quot;&gt;tic-tac-toe&lt;/a&gt;, which is a very simple game. To solve it, you could use minimax or alpha-beta pruning (an extension of minimax), which are very basic but fundamental search techniques for two-player games, so minimax can be applied not only to tic-tac-toe but any two-player game.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notes &lt;a href=&quot;http://web.cs.ucla.edu/~rosen/161/notes/alphabeta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS 161 Recitation Notes - Minimax with Alpha-Beta Pruning&lt;/a&gt; provide a decent overview (with a concrete example) of alpha-beta pruning, which may be a little bit confusing at the beginning, but it is a relatively simple search technique that you can grasp with a few reading iterations (but take this with a grain of salt because the time and effort that it takes to learn something strongly depends on your knowledge and experience). There are several tutorials online that show how to implement minimax and alpha-beta pruning for tic-tac-toe. I could list them all, but I think it's better you look for them and choose your favorite. Maybe have a look at &lt;a href=&quot;https://github.com/Cledersonbc/tic-tac-toe-minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;this implementation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, when it comes to games, one cannot forget about reinforcement learning. I cannot currently list any good tutorials, but there are many resources online. You can even apply RL to tic-tac-toe, so this may be your next step. As someone mentioned in the comments, &lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI's gym&lt;/a&gt; is an RL library for implementing RL agents, which can be used to solve many games, so you should definitely have a look at it. OpenAI's gym comes with a &lt;a href=&quot;https://gym.openai.com/docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nice introduciton&lt;/a&gt;, especially if you are already familiar with RL, which, of course, you probably should before trying to use it. In principle, you can apply RL to many games, including chess, snake, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also a book entitled &lt;a href=&quot;http://www.cleveralgorithms.com/nature-inspired/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clever Algorithms: Nature-Inspired Programming Recipes&lt;/a&gt; by Jason Brownlee that describes numerous AI techniques, some of them could, in principle, be applied to games too. The book is concise and relatively clear. It also comes with &lt;a href=&quot;https://github.com/clever-algorithms/CleverAlgorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;the implementation&lt;/a&gt; (in Ruby, which is a language similar to Python) of all of the algorithms presented in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, although this site is not appropriate for recommendations, I recommend you first get familiar with fundamental search techniques, such as minimax and alpha-beta pruning, then you could start learning RL, which can be applied to many games. There are many resources online for both of them.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-10T19:13:12.350" LastActivityDate="2020-04-10T19:13:12.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20157" PostTypeId="1" CreationDate="2020-04-10T17:44:54.990" Score="1" ViewCount="16" Body="&lt;p&gt;I am reading the paper entitled &lt;a href=&quot;https://www.mitpressjournals.org/doi/full/10.1162/tacl_a_00303&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving Candidate Generation for Low resource Cross-Lingual Entity Linking&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross-lingual entity linking (XEL) is the task of finding referents in a target language knowledge base for mentions extracted from source language texts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steps used in XEL are &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Candidate generation&lt;/li&gt;&#xA;&lt;li&gt;Disambiguation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What challenges are there in cross-lingual entity linking?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-04-10T19:20:15.910" LastActivityDate="2020-04-10T19:20:15.910" Title="What challenges are there in cross-lingual entity linking?" Tags="&lt;natural-language-processing&gt;&lt;research&gt;&lt;challenges&gt;&lt;cross-lingual&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20158" PostTypeId="1" CreationDate="2020-04-10T22:07:35.663" Score="3" ViewCount="233" Body="&lt;p&gt;In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why we are allowed to do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi}(s) = \mathbb E_{\pi}[G_t|S_t=s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \mathbb E_{\pi}[R_{t+1} + \gamma G_{t+1}|S_t=s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$= \mathbb E_{\pi}[R_{t+1} + \gamma v_{\pi}(S_{t+1})|S_t=s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ = \sum_a \pi(a|s)\sum_{s',r} p(s',r|s,a)[r+ \gamma v_{\pi}(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from which it goes to the update equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{k+1}(s) =  \mathbb E_{\pi}[R_{t+1} + \gamma v_{k}(S_{t+1})|S_t=s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$=\sum_a \pi(a|s)\sum_{s',r} p(s',r|s,a)[r+ \gamma v_{k}(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-12-10T01:20:31.737" LastActivityDate="2020-12-12T06:13:16.343" Title="Why can the Bellman equation be turned into an update rule?" Tags="&lt;reinforcement-learning&gt;&lt;implementation&gt;&lt;convergence&gt;&lt;bellman-equations&gt;&lt;policy-evaluation&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20159" PostTypeId="1" CreationDate="2020-04-10T22:11:51.313" Score="1" ViewCount="48" Body="&lt;p&gt;The following image is a screenshot from &lt;a href=&quot;https://youtu.be/_7JKYazD0ew&quot; rel=&quot;nofollow noreferrer&quot;&gt;a video tutorial&lt;/a&gt; that illustrates the concept of gradient descent algorithm with a 3D animation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do the numbers on the top of the balls pointed out by the red arrows represent the gradient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QG3Cy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QG3Cy.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35896" LastEditorUserId="2444" LastEditDate="2020-04-11T23:36:39.797" LastActivityDate="2020-04-11T23:36:39.797" Title="What do these numbers represent in this picture of a surface?" Tags="&lt;terminology&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20160" PostTypeId="1" AcceptedAnswerId="20179" CreationDate="2020-04-10T23:22:04.970" Score="2" ViewCount="231" Body="&lt;p&gt;Recently, I have completed Atari Breakout (&lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1312.5602.pdf&lt;/a&gt;) with DQN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar to DQN, what are the most common deep reinforcement learning algorithms and models in 2020? It seems that DQN is outdated and policy gradients are preferred.&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-04-11T14:39:24.613" LastActivityDate="2020-11-20T18:57:52.753" Title="What are the most common deep reinforcement learning algorithms and models apart from DQN?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20161" PostTypeId="1" CreationDate="2020-04-10T23:58:38.417" Score="1" ViewCount="26" Body="&lt;p&gt;Are there any training datasets using standard font text rather than handwritten ones?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using the MNIST handwritten one on font based chars, but it didn't work well. &lt;/p&gt;&#xA;" OwnerUserId="35924" LastEditorUserId="2444" LastEditDate="2020-04-11T00:37:27.533" LastActivityDate="2020-04-11T00:37:27.533" Title="Are there any training datasets using standard font text rather than hand written ones?" Tags="&lt;datasets&gt;&lt;reference-request&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20162" PostTypeId="2" ParentId="20143" CreationDate="2020-04-11T00:33:25.173" Score="1" Body="&lt;p&gt;AI can categorize documents very accurately.  It is not a new application but in the last year the accuracy of the underlying algorithms such as text classifiers, and language models in general, has significantly improved.  There are applications of language models which now surpass human performance. Microsoft is one of the leaders in this area.  For example, see their Turing Natural Language Generation (T-NLG) language model: &lt;a href=&quot;https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing-NLG: A 17-billion-parameter language model by Microsoft&lt;/a&gt;. The T-NLG was developed in a research group called Project Turing which works on AI tools for Microsoft's products including Office-365.  My guess is this same research group makes the AI for Project Cortex.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-04-11T00:38:26.703" LastActivityDate="2020-04-11T00:38:26.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20163" PostTypeId="2" ParentId="20158" CreationDate="2020-04-11T01:10:13.457" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are we allowed to convert the Bellman equations into update rules?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is a simple reason for this: &lt;strong&gt;convergence&lt;/strong&gt;. The same &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 4&lt;/a&gt; of the same book mentions it. For example, in the case of &lt;em&gt;policy evaluation&lt;/em&gt;, the produced sequence of estimates &lt;span class=&quot;math-container&quot;&gt;$\{v_k\}$&lt;/span&gt; is guaranteed to converge to &lt;span class=&quot;math-container&quot;&gt;$v_\pi$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; (i.e. the number of iterations) goes to infinity. There are other RL algorithms that are also guaranteed to converge (e.g. tabular Q-learning). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, in many cases, the update rules of simple reinforcement learning (or dynamic programming) algorithms are very similar to their mathematical formalization because algorithms based on those update rules are often guaranteed to converge. However, note that many more advanced reinforcement learning algorithms (especially, the ones that use function approximators, such as neural networks, to represent the value functions or policies) are not guaranteed or known to converge.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-11T03:50:26.233" LastActivityDate="2020-04-11T03:50:26.233" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20166" PostTypeId="1" AcceptedAnswerId="20170" CreationDate="2020-04-11T06:42:42.023" Score="1" ViewCount="68" Body="&lt;p&gt;I came across the formula for Upper Confidence Bound Action Selection (while studying multi-armed bandit problem), which looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A_t \dot{=} \operatorname{argmax}_a \left[ Q_t(a) + c \sqrt{ \frac{\ln t}{N_t(a)} } \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although, I understand what the second term in the summation actually means but I am not able to figure out how and from where the exact expression came from, what is the log doing there? What effect does &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; have? And, why a square root?&lt;/p&gt;&#xA;" OwnerUserId="35926" LastEditorUserId="2444" LastEditDate="2020-06-16T11:46:04.710" LastActivityDate="2020-10-23T10:08:39.370" Title="How do we reach at the formula for UCB action-selection in multi-armed bandit problem?" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;upper-confidence-bound&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20167" PostTypeId="1" CreationDate="2020-04-11T07:22:50.233" Score="3" ViewCount="153" Body="&lt;p&gt;I am trying to understand the results of the paper &lt;a href=&quot;http://arxiv.org/pdf/1311.2901.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visualizing and Understanding Convolutional Networks&lt;/a&gt;, in particular the following image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h8YGK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h8YGK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are these 3x3 blocks and their 9 cells representing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, each 3x3 block of the i-th layer corresponds to a randomly chosen feature map in that layer (e.g. for the layer-1 they randomly chose 9 feature maps, for layer-2 16 feature maps etc). On the left part (grayish images), the j-th 3x3 block shows 9 visualizations obtained by mapping the top-9 activations (single values) of that particular feature map to the &quot;pixel space&quot; (using a deconvolutional network). On the right part, the j-th block shows the 9 patches of input images, corresponding to the top-9 activations (e.g. in the first layer and i-th feature map, the j-th image patch is the local region of input image which is seen by the j-th neuron of that feature map). Is my understanding correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's not entirely clear to me how the top-9 activations are chosen. It seems that for each layer and each feature-map, an activation is picked for a different input image (that's why we see e.g. different persons in layer-3, row-1, col-1, and different cars in layer-3, row-2, col-2). So within each block, the top-9 activations are obtained from 9 different images (but images of the same class) of the entire dataset (but in principle it could be that more than one activations are coming from the same image).&lt;/p&gt;&#xA;" OwnerUserId="31657" LastEditorUserId="31657" LastEditDate="2020-04-15T20:17:28.957" LastActivityDate="2020-04-16T14:54:29.427" Title="Understanding the results of &quot;Visualizing and Understanding Convolutional Networks&quot;" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20168" PostTypeId="1" CreationDate="2020-04-11T08:54:32.807" Score="3" ViewCount="44" Body="&lt;p&gt;What does the normalization of the inputs mean in the context of PPO? At each time step of an episode, I only know the values of this time step and of the previous ones, if I take track of them. This means that for each observation and for each reward at each time step I will do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;value = (value - mean) / std&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;before passing them to the NN, right? Specifically, I compute mean and std by keeping track of the values for the whole episode and at each time step, I add the new values to an array. Is this a valid approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how can I handle negative rewards, such that being positive?&lt;/p&gt;&#xA;" OwnerUserId="35930" LastEditorUserId="2444" LastEditDate="2020-04-11T12:40:06.577" LastActivityDate="2020-04-11T12:40:06.577" Title="How does normalization of the inputs work in the context of PPO?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;rewards&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20169" PostTypeId="1" CreationDate="2020-04-11T08:57:39.667" Score="1" ViewCount="63" Body="&lt;p&gt;The objective function of an SVM is the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\mathbf{w}, b)=C \sum_{i=1}^{m} \max \left(0,1-y^{(i)}\left(\mathbf{w}^{t} \cdot \mathbf{x}^{(i)}+b\right)\right)+\frac{1}{2} \mathbf{w}^{t} \cdot \mathbf{w}$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; is the model's feature weights and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is its bias parameter&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}^{(i)}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i^\text{th}$&lt;/span&gt; training instance's feature vector&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y^{(i)}$&lt;/span&gt; is the target class (&lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) for the &lt;span class=&quot;math-container&quot;&gt;$i^\text{th}$&lt;/span&gt; instance&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of training instances&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the regularisation hyper-parameter&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And if I was to use a kernel, this would become:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\mathbf{w}, b)=C \sum_{i=1}^{m} \max \left(0,1-y^{(i)}\left(\mathbf{u}^{t} \cdot \mathbf{K}^{(i)}+b\right)\right)+\frac{1}{2} \mathbf{u}^{t} \cdot \mathbf{K} \cdot \mathbf{u}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the kernel can be the Gaussian kernel:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$K(\mathbf{u}, \mathbf{v})=e^{-\gamma\|\mathbf{u}-\mathbf{v}\|^{2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;How would I go about finding its gradient with respect to the input?&lt;/p&gt;&#xA;&lt;p&gt;I need to know this as to then apply this to a larger problem of a CNN with its last layer being this SVM, so I can then find the gradient of this output wrt the input of the CNN.&lt;/p&gt;&#xA;" OwnerUserId="29877" LastEditorUserId="2444" LastEditDate="2020-06-25T12:05:31.030" LastActivityDate="2020-06-25T12:05:31.030" Title="What is the gradient of a non-linear SVM with respect to the input?" Tags="&lt;convolutional-neural-networks&gt;&lt;gradient-descent&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20170" PostTypeId="2" ParentId="20166" CreationDate="2020-04-11T09:15:20.457" Score="3" Body="&lt;p&gt;Here is an intuitive description/explanation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is there for a trade-off between exploration and exploitation. If &lt;span class=&quot;math-container&quot;&gt;$c=0$&lt;/span&gt; then you only consider &lt;span class=&quot;math-container&quot;&gt;$Q_t(a)$&lt;/span&gt; (no exploration). If &lt;span class=&quot;math-container&quot;&gt;$c \rightarrow \infty$&lt;/span&gt; then you only consider exploration term.&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{\ln t}{N_t(a)}$&lt;/span&gt; is there to balance out exploration term. If you consider a simple case where you only have one action (then it wouldn't make sense to explore you could always pick that action but let's pretend there is) then as &lt;span class=&quot;math-container&quot;&gt;$t \rightarrow \infty$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$\ln t$&lt;/span&gt; has sublinear growth,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\frac{\ln t}{N_t} \rightarrow 0&#xA;\end{equation}&lt;/span&gt;&#xA;So, after you picked an action infinitely many times, the exploration term will completely diminish, i.e. you already know a lot about what that action does. If you picked a numerator that doesn't have sublinear growth, then as &lt;span class=&quot;math-container&quot;&gt;$t \rightarrow \infty$&lt;/span&gt; exploration term would not diminish, so you would always have a chance to explore and exploration term can &amp;quot;overpower&amp;quot; action value term if &lt;span class=&quot;math-container&quot;&gt;$Q_t$&lt;/span&gt; is very small, even after you picked an action infinitely many times, which is not desired.&lt;/p&gt;&#xA;&lt;p&gt;A similar thing is with multiple actions, &lt;span class=&quot;math-container&quot;&gt;$\ln t$&lt;/span&gt; will make sure the exploration term &lt;span class=&quot;math-container&quot;&gt;$\rightarrow 0$&lt;/span&gt; if you picked it many times, but it's still better than constant term &lt;span class=&quot;math-container&quot;&gt;$K/N_t(a)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is some constant, because it can diminish too fast.&lt;/p&gt;&#xA;&lt;p&gt;With &lt;span class=&quot;math-container&quot;&gt;$\ln t$&lt;/span&gt; you will also not stop exploring completely if you haven't picked some action in a long time, because &lt;span class=&quot;math-container&quot;&gt;$\ln t$&lt;/span&gt; will keep growing and &lt;span class=&quot;math-container&quot;&gt;$N_t(a)$&lt;/span&gt; will remain the same, so their fraction will increase with time, which is useful in non-stationary environments.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;em&gt;square root&lt;/em&gt; is also there probably to balance out the magnitude of exploration term.&lt;/p&gt;&#xA;&lt;p&gt;You can also see this &lt;a href=&quot;https://stats.stackexchange.com/questions/323867/upper-confidence-bound-in-machine-learning&quot;&gt;answer&lt;/a&gt;. It has couple of links to some papers for a more mathematical description.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="2444" LastEditDate="2020-10-23T10:08:39.370" LastActivityDate="2020-10-23T10:08:39.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20172" PostTypeId="1" CreationDate="2020-04-11T09:39:56.167" Score="1" ViewCount="56" Body="&lt;p&gt;My problem is that every time I am trying to train my PPO agent I get NaN values after a while. The diagnostic that I get is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ep=   3| t=  144/450000|  0.8 sec| rew=31.3005| ploss=-2.5e-02| vfloss=4.7e+01| kl=1.3e-03| ent=8.5e+00| clipfrac=0.0e+00&#xA;ep=   6| t=  288/450000|  1.1 sec| rew=31.2144| ploss=-2.2e-02| vfloss=4.1e+01| kl=1.3e-03| ent=8.5e+00| clipfrac=0.0e+00&#xA;ep=   9| t=  432/450000|  1.4 sec| rew=28.2668| ploss=-2.9e-02| vfloss=3.5e+01| kl=1.6e-03| ent=8.5e+00| clipfrac=0.0e+00&#xA;ep=  12| t=  576/450000|  1.7 sec| rew=28.2910| ploss=-2.7e-02| vfloss=3.6e+01| kl=1.7e-03| ent=8.5e+00| clipfrac=0.0e+00&#xA;ep=  15| t=  720/450000|  2.0 sec| rew=27.4817| ploss=-2.3e-02| vfloss=3.0e+01| kl=1.8e-03| ent=8.5e+00| clipfrac=0.0e+00&#xA;ep=  18| t=  864/450000|  2.3 sec| rew=29.8415| ploss=-4.5e-02| vfloss=3.4e+01| kl=4.0e-03| ent=8.5e+00| clipfrac=2.8e-02&#xA;ep=  21| t= 1008/450000|  2.6 sec| rew=29.1447| ploss=-2.7e-02| vfloss=2.7e+01| kl=2.0e-03| ent=8.5e+00| clipfrac=6.9e-03&#xA;ep=  24| t= 1152/450000|  3.0 sec| rew=30.2001| ploss=-3.5e-02| vfloss=2.8e+01| kl=1.7e-03| ent=8.5e+00| clipfrac=6.9e-03&#xA;ep=  27| t= 1296/450000|  3.3 sec| rew=31.4069| ploss=-2.9e-02| vfloss=3.7e+01| kl=3.0e-03| ent=8.5e+00| clipfrac=2.1e-02&#xA;ep=  30| t= 1440/450000|  3.6 sec| rew=27.7963| ploss=-4.6e-02| vfloss=2.3e+01| kl=7.3e-03| ent=8.5e+00| clipfrac=1.7e-01&#xA;ep=  33| t= 1584/450000|  3.9 sec| rew=30.8561| ploss=-5.9e-02| vfloss=2.5e+01| kl=9.6e-03| ent=8.5e+00| clipfrac=2.6e-01&#xA;ep=  36| t= 1728/450000|  4.2 sec| rew=27.3002| ploss=-6.9e-02| vfloss=2.2e+01| kl=1.3e-02| ent=8.5e+00| clipfrac=3.1e-01&#xA;ep=  39| t= 1872/450000|  4.5 sec| rew=28.0270| ploss=-5.6e-02| vfloss=2.1e+01| kl=8.9e-03| ent=8.5e+00| clipfrac=2.0e-01&#xA;ep=  42| t= 2016/450000|  4.9 sec| rew=28.0624| ploss=-5.8e-02| vfloss=2.0e+01| kl=7.5e-03| ent=8.5e+00| clipfrac=2.4e-01&#xA;ep=  45| t= 2160/450000|  5.2 sec| rew=28.6224| ploss=-8.4e-02| vfloss=2.3e+01| kl=7.2e-03| ent=8.5e+00| clipfrac=2.0e-01&#xA;ep=  48| t= 2304/450000|  5.5 sec| rew=32.3889| ploss=-4.3e-02| vfloss=2.6e+01| kl=7.1e-03| ent=8.5e+00| clipfrac=2.9e-01&#xA;ep=  51| t= 2448/450000|  5.8 sec| rew=31.4241| ploss=-1.0e-01| vfloss=2.7e+01| kl=7.0e-03| ent=8.5e+00| clipfrac=2.1e-01&#xA;ep=  54| t= 2592/450000|  6.1 sec| rew=33.4760| ploss=-5.1e-02| vfloss=2.5e+01| kl=7.3e-03| ent=8.5e+00| clipfrac=2.4e-01&#xA;ep=  57| t= 2736/450000|  6.4 sec| rew=31.0780| ploss=-8.8e-02| vfloss=2.3e+01| kl=6.9e-03| ent=8.5e+00| clipfrac=3.0e-01&#xA;ep=  60| t= 2880/450000|  6.7 sec| rew=34.1286| ploss=-6.9e-02| vfloss=2.7e+01| kl=7.6e-03| ent=8.5e+00| clipfrac=3.1e-01&#xA;ep=  63| t= 3024/450000|  7.1 sec| rew=31.0017| ploss=-6.1e-02| vfloss=2.5e+01| kl=1.4e-02| ent=8.5e+00| clipfrac=3.7e-01&#xA;ep=  66| t= 3168/450000|  7.4 sec| rew=32.3697| ploss=-1.1e-01| vfloss=2.2e+01| kl=1.2e-02| ent=8.5e+00| clipfrac=4.6e-01&#xA;ep=  69| t= 3312/450000|  7.7 sec| rew=31.4455| ploss=-7.4e-02| vfloss=2.4e+01| kl=8.4e-03| ent=8.5e+00| clipfrac=3.7e-01&#xA;ep=  72| t= 3456/450000|  8.0 sec| rew=32.1896| ploss=-9.2e-02| vfloss=2.0e+01| kl=1.3e-02| ent=8.4e+00| clipfrac=4.1e-01&#xA;ep=  75| t= 3600/450000|  8.3 sec| rew=31.3721| ploss=-9.4e-02| vfloss=2.4e+01| kl=1.4e-02| ent=8.4e+00| clipfrac=4.2e-01&#xA;ep=  78| t= 3744/450000|  8.6 sec| rew=35.5718| ploss=-1.0e-01| vfloss=3.0e+01| kl=1.0e-02| ent=8.4e+00| clipfrac=4.6e-01&#xA;ep=  81| t= 3888/450000|  9.0 sec| rew=32.2289| ploss=-1.3e-01| vfloss=2.9e+01| kl=1.6e-02| ent=8.4e+00| clipfrac=5.5e-01&#xA;ep=  84| t= 4032/450000|  9.3 sec| rew=31.7656| ploss=-1.0e-01| vfloss=2.3e+01| kl=1.3e-02| ent=8.4e+00| clipfrac=4.4e-01&#xA;ep=  87| t= 4176/450000|  9.6 sec| rew=35.4555| ploss=-8.8e-02| vfloss=3.3e+01| kl=5.8e-03| ent=8.4e+00| clipfrac=3.1e-01&#xA;ep=  90| t= 4320/450000|  9.9 sec| rew=33.2766| ploss=-1.2e-01| vfloss=2.4e+01| kl=1.5e-02| ent=8.4e+00| clipfrac=5.7e-01&#xA;ep=  93| t= 4464/450000| 10.2 sec| rew=32.5218| ploss=-1.1e-01| vfloss=2.4e+01| kl=1.8e-02| ent=8.4e+00| clipfrac=5.6e-01&#xA;ep=  96| t= 4608/450000| 10.5 sec| rew=34.7137| ploss=-9.7e-02| vfloss=2.5e+01| kl=1.5e-02| ent=8.4e+00| clipfrac=4.3e-01&#xA;ep=  99| t= 4752/450000| 10.9 sec| rew=35.3797| ploss=-8.2e-02| vfloss=2.5e+01| kl=1.6e-02| ent=8.4e+00| clipfrac=4.9e-01&#xA;ep= 102| t= 4896/450000| 11.2 sec| rew=nan| ploss=nan| vfloss=nan| kl=nan| ent=nan| clipfrac=0.0e+00&#xA;C:\Users\ppo.py:154: RuntimeWarning: invalid value encountered in greater&#xA;  adv_nrm = (adv_nrm - adv_nrm.mean()) / max(1.e-8,adv_nrm.std()) # standardized advantage function estimate&#xA;ep= 105| t= 5040/450000| 11.5 sec| rew=nan| ploss=nan| vfloss=nan| kl=nan| ent=nan| clipfrac=0.0e+00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any ideas why this is arised? &lt;/p&gt;&#xA;" OwnerUserId="35930" LastActivityDate="2020-04-11T09:39:56.167" Title="NaNs after a while in training of PPO" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20174" PostTypeId="2" ParentId="20159" CreationDate="2020-04-11T10:57:26.643" Score="1" Body="&lt;p&gt;It represents the value of the loss function &lt;code&gt;J(x1, x2; θ)&lt;/code&gt;; the valley has value 0 in the video. You can see that the lowest ball with value 3.13 is on a steep point with a high gradient, so it's not the gradient. &lt;/p&gt;&#xA;" OwnerUserId="34315" LastActivityDate="2020-04-11T10:57:26.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20175" PostTypeId="1" CreationDate="2020-04-11T11:31:04.957" Score="2" ViewCount="36" Body="&lt;p&gt;I am trying to perform a &lt;a href=&quot;https://arxiv.org/abs/1912.06409&quot; rel=&quot;nofollow noreferrer&quot;&gt;white-box attack&lt;/a&gt; on a model.&lt;/p&gt;&#xA;&lt;p&gt;Would it be possible to simply use the numerical gradient of the output wrt input directly rather than computing each subgradient of the network analytically? Would this (1) work and (2) actually be a white box attack?&lt;/p&gt;&#xA;&lt;p&gt;As I would not be using a different model to 'mimic' the results but instead be using the same model to get the outputs, am I right in thinking that this would still be a white box attack.&lt;/p&gt;&#xA;" OwnerUserId="29877" LastEditorUserId="2444" LastEditDate="2020-12-09T10:56:28.660" LastActivityDate="2020-12-09T10:56:28.660" Title="To perform a white-box adversarial attack, would the use of a numerical gradient suffice?" Tags="&lt;machine-learning&gt;&lt;security&gt;&lt;adversarial-ml&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20176" PostTypeId="1" CreationDate="2020-04-11T12:53:21.253" Score="7" ViewCount="699" Body="&lt;p&gt;I am watching &lt;a href=&quot;https://www.youtube.com/watch?v=iDulhoQ2pro&amp;amp;t=6s&quot; rel=&quot;noreferrer&quot;&gt;Attention all you need&lt;/a&gt;, In that what is the intuition behind the dot product attention?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A(q,K, V) = \sum_i\frac{e^{q.k_i}}{\sum_j e^{q.k_j}} v_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A(Q,K, V) = softmax(QK^T)V$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="9947" LastEditDate="2020-04-11T13:17:28.440" LastActivityDate="2020-04-11T15:06:03.533" Title="What is the intuition behind the dot product attention?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;attention&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20177" PostTypeId="2" ParentId="20158" CreationDate="2020-04-11T13:55:09.633" Score="0" Body="&lt;p&gt;You're asking why the finite horizon policy evaluation converges to the infinite right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the total reward is bounded(by the discount factor) you know that you can make your finite horizon policy evaluation get arbitrarily close to it in a finite number of steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People praise Bartos book but I find it annoying to read as he's not formal enough with mathematics.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-11T13:55:09.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20178" PostTypeId="2" ParentId="20176" CreationDate="2020-04-11T14:49:29.190" Score="9" Body="&lt;p&gt;Let's start with a bit of notation and a couple of important clarification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q&lt;/strong&gt; refers to the &lt;strong&gt;query&lt;/strong&gt; vectors matrix, &lt;span class=&quot;math-container&quot;&gt;$q_i$&lt;/span&gt; being a single query vector associated with a single input word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;V&lt;/strong&gt; refers to the &lt;strong&gt;values&lt;/strong&gt; vectors matrix, &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; being a single value vector associated with a single input word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;K&lt;/strong&gt; refers to the &lt;strong&gt;keys&lt;/strong&gt; vectors matrix, &lt;span class=&quot;math-container&quot;&gt;$k_i$&lt;/span&gt; being a single key vector associated with a single input word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do these matrices come from? Something that is not stressed out enough in lot of tutorials is that these matrices are the result of a &lt;strong&gt;matrix product between the input &lt;em&gt;embeddings&lt;/em&gt; and 3 matrices of trained weights:&lt;/strong&gt; &lt;strong&gt;W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{q}}$&lt;/span&gt;&lt;/strong&gt;, &lt;strong&gt;W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{v}}$&lt;/span&gt;&lt;/strong&gt;, &lt;strong&gt;W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{k}}$&lt;/span&gt;&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that these three matrices are learned during training explain why the query, value and key vectors end up being different despite the identical input sequence of embeddings. It also explain why it make sense to talk about multi-head attention. Performing multiple attention steps on the same sentence produce different results because for each attention 'head' new W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{q}}$&lt;/span&gt;, W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{v}}$&lt;/span&gt;, W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{k}}$&lt;/span&gt; are randomly initialised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another important aspect not stressed out enough is that for the encoder and decoder first attention layers, all the three matrices comes from the previous layer (either the input or the previous attention layer) but for the &lt;strong&gt;encoder/decoder&lt;/strong&gt; attention layer, the &lt;strong&gt;Q&lt;/strong&gt; matrix comes from the previous decoder layer, whereas the &lt;strong&gt;V&lt;/strong&gt; and &lt;strong&gt;K&lt;/strong&gt; matrices come from the encoder. And this is a crucial step to explain how the representation of two languages in an encoder are mixed together.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once computed the three matrices, the transformer move on on the calculation of the dot product between query and key vectors. &#xA;As nbro wrote in its comment, the dot product is used to compute a sort of similarity score between the query and key vectors. Indeed, the authors used the names &lt;em&gt;query&lt;/em&gt;, &lt;em&gt;key&lt;/em&gt; and &lt;em&gt;value&lt;/em&gt; to indicate that what they propose is similar to what is done in information retrieval. For example in question answering usually given a query you want to retrive the closer sentence in meaning among all possible answers, and this is done by computing the similarity between sentences (question vs possible answers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course here the situation is not exactly the same, but the guy who did the video you linked did a great job in explaining what happened during the attention computation (the two equations you wrote are exactly the same in vector and matrix notation and represent these passages): &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;closer query and key vectors will have higher dot products.&lt;/li&gt;&#xA;&lt;li&gt;applying the softmax will normalise the dot product scores between 0 and 1.&lt;/li&gt;&#xA;&lt;li&gt;multiplying the softmax results to the value vectors will push down close to zero all value vectors for words which had a low dot product score between query and key vector.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the paper the authors explain the attention mechanisms saying that the purpose is to determine which words of a sentence the transformer should focus on. I personally prefer to think at attention as a sort of coreference resolution step, the reason why I think so is the following image (taken from this &lt;a href=&quot;https://www.slideshare.net/ilblackdragon/attention-is-all-you-need&quot; rel=&quot;noreferrer&quot;&gt;presentation&lt;/a&gt; by the original authors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nnm8a.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nnm8a.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image shows basically the result of the attention computation (at a specific layer that they don't mention). Bigger lines connecting words mean bigger values in the dot product between the words query and key vectors, which means basically that only those words value vectors will pass for further processing to the next attention layer. But please note that some words are actually related even if not similar at all, for example 'Law' and 'The' are not similar, they are simply related to each other in this specific sentences (that's why I like to think to attention as coreference resolution). Computing similarities between embeddings would never provide information about this relation in a sentence, the only reason why transformer learn these relationship is the presences of the trained matrices &lt;strong&gt;W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{q}}$&lt;/span&gt;&lt;/strong&gt;, &lt;strong&gt;W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{v}}$&lt;/span&gt;&lt;/strong&gt;, &lt;strong&gt;W&lt;span class=&quot;math-container&quot;&gt;$_{\textbf{k}}$&lt;/span&gt;&lt;/strong&gt; (plus the presence of positional embeddings).&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="34098" LastEditDate="2020-04-11T15:06:03.533" LastActivityDate="2020-04-11T15:06:03.533" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20179" PostTypeId="2" ParentId="20160" CreationDate="2020-04-11T14:59:22.537" Score="3" Body="&lt;p&gt;There are several common deep reinforcement algorithms and models apart from &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep Q networks (or deep Q learning)&lt;/a&gt;. I will list some of them below (along with a link to the paper that introduced them), but note that some of these may not be state-of-the-art (at least, not anymore, and it's likely that all of these will be replaced in the future).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1509.06461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Double DQN (DDQN)&lt;/a&gt; (2015)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1511.06581.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Duelling DQN&lt;/a&gt; (2015)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v37/schulman15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trust Region Policy Optimization (TRPO)&lt;/a&gt; (2015)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1509.02971.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Deterministic Policy Gradient (DDPG)&lt;/a&gt; (2016)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v48/mniha16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asynchronous Advantage Actor-Critic (A3C)&lt;/a&gt; (2016)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience Replay (HER)&lt;/a&gt; (2017)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proximal policy optimization (PPO)&lt;/a&gt; (2017)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twin Delayed Deep Deterministic policy gradient algorithm (TD3)&lt;/a&gt; (2018)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soft Actor-Critic (SAC)&lt;/a&gt; (2018)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For an exhaustive overview of deep RL algorithms and models, maybe take a look at this pre-print &lt;a href=&quot;https://arxiv.org/pdf/1810.06339.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning&lt;/a&gt; (2018) by Yuxi Li.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T18:57:52.753" LastActivityDate="2020-11-20T18:57:52.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20180" PostTypeId="1" AcceptedAnswerId="20181" CreationDate="2020-04-12T00:02:33.020" Score="2" ViewCount="233" Body="&lt;p&gt;Although no artificial general intelligence (AGI) has yet been created, probably, there are already some courses on the topic. So, what are some online (preferably free) courses on AGI?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-22T18:06:40.527" LastActivityDate="2020-10-06T12:50:45.127" Title="What are some online courses on artificial general intelligence?" Tags="&lt;agi&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20181" PostTypeId="2" ParentId="20180" CreationDate="2020-04-12T00:02:33.020" Score="7" Body="&lt;p&gt;There are at least three free courses.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://agi.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIT 6.S099: Artificial General Intelligence&lt;/a&gt;, organized by &lt;a href=&quot;https://lexfridman.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lex Fridman&lt;/a&gt;, is a series of lessons and talks primarily given by a diverse set of guest appearances, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=7ROelYvo8f0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Josh Tenenbaum&lt;/a&gt; (researcher and professor in computational cognitive science),&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=bfO4EkoGh40&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nate Derbinsky&lt;/a&gt; (who gives a lesson on cognitive architectures, Soar, etc.),&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=P7kX7BuHSFI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stephen Wolfram&lt;/a&gt; (creator of Mathematica, Wolfram Alpha and the Wolfram Language; he talks about his work throughout the years, especially, the development of Wolfram Alpha and Language),&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=LiNSPRKHyvo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marc Raibert&lt;/a&gt; (CEO of Boston Dynamics, who gives a lesson on his work at Boston Dynamics and the robots they have developed),&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=qwsft6tmvBA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lisa Feldman Barrett&lt;/a&gt; (professor of psychology, who gives a very insightful lesson on emotions and feelings, with possibly different perspectives),&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=kq0VO1FqE6I&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosalind Picard&lt;/a&gt; (a professor at MIT, director of the Affective Computing Research Group at the MIT Media Lab, and co-founder of two companies, Affectiva and Empatica; in her talk with Lex, she talks about affective computing and other interesting concepts and issues)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=E1AxVXt2Gv4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marcus Hutter&lt;/a&gt; (inventor of a mathematical theory of general intelligence, AIXI; in his talk with Lex, he talks about Occam's razor, Solomonoff induction, Kolmogorov complexity, the definition of intelligence, AIXI, rewards, bounded rationality and consciousness)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.udacity.com/course/knowledge-based-ai-cognitive-systems--ud409&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge-Based AI: Cognitive Systems&lt;/a&gt; (taught by Ashok Goel and David Joyner)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-868j-the-society-of-mind-fall-2011/video-lectures/lecture-1-introduction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Society of Mind&lt;/a&gt; (taught by Marvin Minsky, who also wrote a book and developed a theory of natural intelligence with the same name)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://inst.eecs.berkeley.edu/%7Ecs294-149/fa18/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS 294-149: Safety and Control for Artificial General Intelligence (Fall 2018)&lt;/a&gt; (see also &lt;a href=&quot;https://github.com/HumanCompatibleAI/cs294-149-fa18-notes&quot; rel=&quot;nofollow noreferrer&quot;&gt;these notes&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-06T12:26:58.330" LastActivityDate="2020-10-06T12:26:58.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20183" PostTypeId="1" AcceptedAnswerId="20205" CreationDate="2020-04-12T03:23:37.053" Score="1" ViewCount="38" Body="&lt;p&gt;Simplified: &lt;em&gt;What is it called in AI when a program is designed to make &quot;x in the style of y;&quot; when it trains off of two types of sources in order to make a thing from source one, informed by features from source two?&lt;/em&gt; For example, if a network made up of two smaller networks were to take sheet music of a specific compositional style in network A and audio samples from a certain genre in B and through an interface creates music from a certain genre in a certain compositional style; the &lt;em&gt;is&lt;/em&gt; comes from One, the &lt;em&gt;seems&lt;/em&gt; comes from Two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more coarse and obvious examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;Compose synthpop in the style of Beethoven&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;Draw impressionism in the style of Mondrian&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;Generate casserole recipes using only ingredients most likely to fluctuate in price given current market data&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;Sketch baseballs that look like they're made of espresso foam&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="35622" LastEditorUserId="2444" LastEditDate="2020-04-12T20:02:31.377" LastActivityDate="2020-04-12T20:02:31.377" Title="What is it called in AI when a program is designed to make &quot;x in the style of y&quot;?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20184" PostTypeId="1" CreationDate="2020-04-12T03:26:37.487" Score="3" ViewCount="43" Body="&lt;p&gt;I have an image of some nano particles that was taken with Scanning Electron Microscope (SEM) attached here. I want to obtain center points coordinates (x,y) for each particle. Doing it by hand is very tedious. Since I just started to learn Machine Learning and got introduced to Artificial Neural Networks and kinda understand that they they are helpful with image classification, I am curious if I can use these tools to achieve my goal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this &lt;a href=&quot;https://arxiv.org/pdf/1806.04010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; where they discuss kind similar work,,, but I am curious if you have seen anything practical or if you can give me some steps where and how to start, that's really helpful.&lt;br&gt;&#xA;Any guidance is appreciated.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M5OwC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M5OwC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35951" LastActivityDate="2020-04-12T06:29:54.290" Title="Can neural network help me with detecting center coordinates of particles in an image?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20185" PostTypeId="1" CreationDate="2020-04-12T03:41:24.980" Score="3" ViewCount="74" Body="&lt;p&gt;How can I cluster the data frame below with several features and observations? And how would I go about determining the quality of those clusters? Is k-NN appropriate for this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id     Name             Gender   Dob    Age  Address&#xA;1   MUHAMMAD JALIL      Male    1987    33   Chittagong&#xA;1   MUHAMMAD JALIL      Male    1987    33   Chittagong&#xA;2   MUHAMMAD JALIL      Female  1996    24   Rangpur&#xA;2   MRS. JEBA           Female  1996    24   Rangpur&#xA;3   MR. A. JALIL        Male    1987    33   Sirajganj&#xA;3   MR. A. JALIL        Male    1987    33   Sirajganj&#xA;3   MD. A. JALIL        Male    1987    33   Sirajganj&#xA;4   MISS. JEBA          Female  1996    24   Rangpur&#xA;4   PROF. JEBA          Female  1996    24   Rangpur&#xA;1   MD. A. JALIL        Male    1987    33   Chittagong&#xA;1   MUHAMMAD A. JALIL   Male    1987    33   Chittagong&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35941" LastEditorUserId="2444" LastEditDate="2020-04-12T15:53:13.253" LastActivityDate="2020-04-21T15:26:52.323" Title="How can I cluster this data frame with several features and observations?" Tags="&lt;python&gt;&lt;datasets&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20186" PostTypeId="1" CreationDate="2020-04-12T03:44:31.903" Score="1" ViewCount="82" Body="&lt;p&gt;Let us confine ourselves to the case where we have a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; dimensional input and a &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; output. It can be shown that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For every &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, there exists a dense NN of depth 2, such that it contains all functions from &lt;span class=&quot;math-container&quot;&gt;${±1}^n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;${±1}$&lt;/span&gt;. (given sign activation functions and some other very simple assumptions).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Check &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 20.3&lt;/a&gt; for the proof.&lt;/p&gt;&#xA;&lt;p&gt;So, if a neural net can approximate any function, then it has a &lt;span class=&quot;math-container&quot;&gt;$\mathcal V \mathcal C $&lt;/span&gt; dimension of infinite (considering the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; dimensional set of points as our universe).&lt;/p&gt;&#xA;&lt;p&gt;Thus, it can realize all types of functions (or its hypothesis set contains all set of functions), and hence cannot have prior knowledge (prior knowledge in the sense used in the No Free Lunch theorem).&lt;/p&gt;&#xA;&lt;p&gt;Are my deductions correct? Or did I make wrong assumptions? Are there actually any prior beliefs in a neural network that I am missing?&lt;/p&gt;&#xA;&lt;p&gt;A detailed explanation would be nice.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2021-01-22T15:44:56.853" LastActivityDate="2021-01-22T15:44:56.853" Title="If a neural network is a universal function approximator, can it have any prior beliefs?" Tags="&lt;neural-networks&gt;&lt;computational-learning-theory&gt;&lt;universal-approximation-theorems&gt;&lt;vc-theory&gt;&lt;no-free-lunch-theorems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20187" PostTypeId="2" ParentId="20184" CreationDate="2020-04-12T06:29:54.290" Score="1" Body="&lt;p&gt;This is a very hard problem, you have many overlapping points with objects which aren't completely round. I'm not very knowledgeable on CV but I suspect you will find it very challenging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would probably say a handcrafted detection algorithm would probably be easier, something like an edge detector which fit circles to arcs and labeled the points. But it's still going to be nontrivial and maybe impossible to get it working with high accuracy.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-12T06:29:54.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20188" PostTypeId="1" CreationDate="2020-04-12T07:20:21.107" Score="1" ViewCount="254" Body="&lt;p&gt;I am a novice in AI and I like to build a chatbot to predict diseases using patient narration as input. Initially, I simply want to train my chatbot on 1 disease only. And once this initial milestone is accomplished then I want to train it on other diseases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know whether I should build and train my model first and then move towards building a chatbot or should i create a chatbot first and then train it on a disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also please justify which approach is better and why? &lt;/p&gt;&#xA;" OwnerUserId="34306" LastActivityDate="2020-04-12T12:11:51.767" Title="What is the best approach to build a self-learning AI chatbot?" Tags="&lt;prediction&gt;&lt;chat-bots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20189" PostTypeId="1" AcceptedAnswerId="20197" CreationDate="2020-04-12T10:08:58.267" Score="2" ViewCount="52" Body="&lt;p&gt;I believe to understand the reason why &lt;strong&gt;on-policy methods cannot reuse trajectories collected from earlier policies&lt;/strong&gt;: the trajectory distribution change with the policy and the policy gradient is derived to be an expectation over these trajectories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doesn't the following intuition from the &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/vpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Vanilla Policy Gradient description&lt;/a&gt; indeed propose that learning from prior experience &lt;strong&gt;should still be possible&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The key idea underlying policy gradients is to push up the probabilities of actions that lead to higher return, and push down the probabilities of actions that lead to lower return.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The goal is to change the probabilities of actions. Actions sampled from previous policies are still possible under the current one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that we cannot reuse the previous actions to estimate the policy gradient. But couldn't we &lt;strong&gt;update the policy network with previous trajectories using supervised learning?&lt;/strong&gt; The labels for the actions would be between 0 and 1 based on how good an action was. In the simplest case, just 1 for good actions and 0 for bad ones. The loss could be a simple sum of squared differences with a regularization term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that not used/possible? What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="35821" LastEditorUserId="35821" LastEditDate="2020-04-15T14:08:20.767" LastActivityDate="2020-04-15T14:08:20.767" Title="Could we update the policy network with previous trajectories using supervised learning?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;experience-replay&gt;&lt;off-policy-methods&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20190" PostTypeId="1" CreationDate="2020-04-12T10:12:58.100" Score="2" ViewCount="72" Body="&lt;p&gt;I am looking for datasets that are used as a testing standard in the fully connected neural networks (FCNN). For example, in the image recognition and CNN, CIFAR datasets are used in most of the papers, but can't find anything like that for the FCNN. &lt;/p&gt;&#xA;" OwnerUserId="31324" LastEditorUserId="2444" LastEditDate="2020-04-12T14:43:38.337" LastActivityDate="2020-04-14T16:52:29.630" Title="What are standard datasets for fully connected neural networks?" Tags="&lt;neural-networks&gt;&lt;datasets&gt;&lt;feedforward-neural-network&gt;&lt;multilayer-perceptron&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20191" PostTypeId="1" AcceptedAnswerId="20242" CreationDate="2020-04-12T11:05:33.807" Score="3" ViewCount="50" Body="&lt;p&gt;I did watch the course DeepLearning of Andrew Ng and he told that we should create parameter &lt;code&gt;w&lt;/code&gt; small like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parameters['W' + str(l)] = np.random.randn(layer_dims[l], layer_dims[l - 1]) ** 0.001 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But in the last application assignment. They choose another way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;layers_dims = [12288, 20, 7, 5, 1]&#xA;def initialize_parameters_deep(layer_dims):&#xA;    np.random.seed(3)&#xA;    parameters = {}&#xA;    L = len(layer_dims)&#xA;&#xA;    for l in range(1, L):&#xA;        parameters['W' + str(l)] = np.random.randn(layer_dims[l], layer_dims[l - 1]) / np.sqrt(layer_dims[l - 1])&#xA;        parameters['b' + str(l)] = np.zeros((layer_dims[l], 1))&#xA;        assert (parameters['W' + str(l)].shape == (layer_dims[l], layer_dims[l - 1]))&#xA;        assert (parameters['b' + str(l)].shape == (layer_dims[l], 1))&#xA;    return parameters&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the result of this way is very good but if I choose &lt;code&gt;w&lt;/code&gt; like the old above, It's just have &lt;code&gt;34%&lt;/code&gt; correct!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So do you can explain ?&lt;/p&gt;&#xA;" OwnerUserId="35958" LastActivityDate="2020-04-14T03:34:20.783" Title="How to choosing the random value for parameter w in deep learning network?" Tags="&lt;deep-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20192" PostTypeId="2" ParentId="18590" CreationDate="2020-04-12T11:41:02.763" Score="3" Body="&lt;p&gt;When ever you are buliding a ML Model don't take accuracy seriously(Mistake done by Netflix that cost them alot), you should try to get the hit scores as they will help you to know how many times your model worked on real world users.However, if your model must have to measure the accuracy try it with the RMSE score as it will penalise you more for being more out of the Line. Here is the link for more information on it &lt;a href=&quot;https://surprise.readthedocs.io/en/stable/accuracy.html#surprise.accuracy.rmse&quot; rel=&quot;nofollow noreferrer&quot;&gt;RMSE&lt;/a&gt;&#xA;Its hard to predict if its overfitting or underfitting as your graph is vague(for example what does graph lines representing). However, you can solve underfitting by following steps:&#xA;1) Increase the size or number of parameters in the ML model.&#xA;2)  Increase the complexity or type of the model.&#xA;3)  Increasing the training time until cost function in ML is minimised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For overfitting you can try Regularization methods like weight decay provide an easy way to control overfitting for large neural network models. A modern recommendation for regularization is to use early stopping with dropout and a weight constraint.&lt;/p&gt;&#xA;" OwnerUserId="25685" LastActivityDate="2020-04-12T11:41:02.763" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20193" PostTypeId="1" AcceptedAnswerId="20240" CreationDate="2020-04-12T12:04:19.350" Score="3" ViewCount="114" Body="&lt;p&gt;I have a labeled dataset composed of 3000 data. Its single feature is the price of the house and its label is the number of bedrooms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which classifier would be a good choice to classify these data?&lt;/p&gt;&#xA;" OwnerUserId="26472" LastEditorUserId="26472" LastEditDate="2020-04-14T07:18:52.910" LastActivityDate="2020-04-21T11:29:36.983" Title="Which classifier should I use for a dataset with one feature?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;classification&gt;" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20194" PostTypeId="1" CreationDate="2020-04-12T12:07:11.683" Score="1" ViewCount="63" Body="&lt;p&gt;I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, I have a point &lt;span class=&quot;math-container&quot;&gt;$x = (1, 1, 2, 3)$&lt;/span&gt;, to which I want to apply a linear transformation &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times. If the transformation is denoted by a &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, then the final transformation would be given by &lt;span class=&quot;math-container&quot;&gt;$A^n * x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that matrix multiplication can computational expensive, is there a way we can speed up the computation, assuming we would need to run multiple iterations of this simulation?&lt;/p&gt;&#xA;" OwnerUserId="5634" LastEditorUserId="2444" LastEditDate="2020-04-13T13:02:40.917" LastActivityDate="2020-09-10T14:01:45.190" Title="Is there any way to apply linear transformations on a vector other than matrix multiplication?" Tags="&lt;linear-regression&gt;&lt;statistical-ai&gt;&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20195" PostTypeId="2" ParentId="20188" CreationDate="2020-04-12T12:11:51.767" Score="2" Body="&lt;p&gt;The former: build and train a model first, and &lt;em&gt;then&lt;/em&gt; think about the user interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Effectively, a chatbot is a user interface to your model. If you run it 'off-line' on input text and it works, then you have achieved your goal without the added complexity of driving a conversation (which is harder than one might think).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, building an 'abstract' chatbot devoid of any content is going to be harder. What should that chatbot talk about?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my own experience (I work in conversational AI), you might not succeed in building a decent chatbot, especially as a novice. But you might be able to train a model to identify diseases from textual input. So if you do that first, you have at least got something! Especially if that is your main reason for the project in the first place.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-12T12:11:51.767" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20197" PostTypeId="2" ParentId="20189" CreationDate="2020-04-12T12:27:35.177" Score="2" Body="&lt;p&gt;You cannot really do that because you have no way of knowing how good the action really is to make reasonable labels for supervised learning (that's the whole point why we need reinforcement learning). The only way to possibly know that is to make labels based on the return that you got from that action but the return is based on an old trajectory with the old policy. The return for that specific action depends on actions that happened after that action in the trajectory and return for those actions change with time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make things clearer, consider a simple case. Let's say you take action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and you end up in state &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; with reward &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Then you have two possibilities, you take action &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; and end up in terminal state &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt; with reward &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt; or you take action &lt;span class=&quot;math-container&quot;&gt;$a_2'$&lt;/span&gt; and end up in terminal state &lt;span class=&quot;math-container&quot;&gt;$s_2'$&lt;/span&gt; with reward &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;. Let's say you use trajectory &lt;span class=&quot;math-container&quot;&gt;$a_1 \rightarrow s_1 \rightarrow a_2 \rightarrow s_2$&lt;/span&gt; with return &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt; to learn about action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;. Then your label for that action would probably be that that action is bad, but it actually isn't, if you took action &lt;span class=&quot;math-container&quot;&gt;$a_2'$&lt;/span&gt; after &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; your return for action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; would be &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;. So you learned that your action is bad even though it might not be. Now, if later you learn that taking action &lt;span class=&quot;math-container&quot;&gt;$a_2'$&lt;/span&gt; is good to take after &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; then you would also learn that &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; might be good but if you keep using that old data with return &lt;span class=&quot;math-container&quot;&gt;$-10$&lt;/span&gt; you will keep learning that &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; is bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can only use data gathered from the current policy to learn about it because older data might be outdated.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="2444" LastEditDate="2020-04-12T13:34:43.497" LastActivityDate="2020-04-12T13:34:43.497" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20198" PostTypeId="1" CreationDate="2020-04-12T14:11:01.503" Score="4" ViewCount="98" Body="&lt;p&gt;I live in a rural area where there is a growing necessity for people with knowledge to prune Pear trees, this process is crucial for the industry, but as people go to the big cities, this skill is being lost, and in a few years there will be no one to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanna know if it is possible to train a robot using AI to do this, and want would it take to make this work!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that this would be viable only in an &quot;industrial&quot; way. The trees all have approximately the same size and are disposed in a certain preset way (distance between each other, height, etc). &lt;/p&gt;&#xA;" OwnerUserId="35964" LastEditorUserId="2444" LastEditDate="2020-04-12T14:42:49.063" LastActivityDate="2020-04-13T01:48:46.120" Title="Is possible to train a robot or AI to prune fruit trees?" Tags="&lt;machine-learning&gt;&lt;robotics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20199" PostTypeId="1" CreationDate="2020-04-12T14:47:32.183" Score="2" ViewCount="45" Body="&lt;p&gt;Suppose I am utilising a neural network to predict the next state, &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; based on the current &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; pairs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;all my neural network inputs are between 0 and 1 and the loss function for this network is defined as the mean squared error of the &lt;strong&gt;Difference&lt;/strong&gt; between the current state and next state. Because the variables are all between 0 and 1, the MSE difference between the &lt;strong&gt;actual difference&lt;/strong&gt; and &lt;strong&gt;predicted difference&lt;/strong&gt; is smaller than the actual difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the difference in next state and current state for &lt;span class=&quot;math-container&quot;&gt;$s \in R^2$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$[0.4,0.5]$&lt;/span&gt; and the neural network outputs a difference of &lt;span class=&quot;math-container&quot;&gt;$[0.2,0.4]$&lt;/span&gt;. The mean squared loss is therefore 0.05 &lt;span class=&quot;math-container&quot;&gt;$(0.2^2 + 0.1^2) = 0.05$&lt;/span&gt; whereas the neural network does not really predict the next state very well due to a difference of &lt;span class=&quot;math-container&quot;&gt;$(0.2, 0.1)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although whichever loss function is used does not matter, It was deceiving to think that despite the loss function outputting low values, it is mainly due to the &lt;strong&gt;squared&lt;/strong&gt; term that keeps the value small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is Mean Squared Error loss function still a good loss function to be used here ? &lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-04-12T14:47:32.183" Title="Is Mean Squared Error Loss function a good loss function for continuous variables $0 &lt; x &lt; 1$" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20201" PostTypeId="2" ParentId="11994" CreationDate="2020-04-12T15:15:12.047" Score="-1" Body="&lt;p&gt;You can try to work with Gated Recurrent Units or GRU. This will solve your problem of with too much latency time that LSTM required. LSTM also doesn't give preference on newer data too. For more information, you can follow more on this great article&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/understanding-gru-networks-2ef37df6c9be&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRU Cells&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25685" LastActivityDate="2020-04-12T15:15:12.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20202" PostTypeId="2" ParentId="20185" CreationDate="2020-04-12T16:50:09.697" Score="2" Body="&lt;p&gt;A typical clustering algorithm is k-means (and &lt;a href=&quot;https://stats.stackexchange.com/q/56500/82135&quot;&gt;not k-NN, i.e. k-nearest neighbours, which is primarily used for classification&lt;/a&gt;). There are other clustering algorithms, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical clustering algorithms&lt;/a&gt;. &lt;code&gt;sklearn&lt;/code&gt; provides functions that implement &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;k-means&lt;/a&gt; (and &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/cluster/plot_cluster_iris.html#sphx-glr-auto-examples-cluster-plot-cluster-iris-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;an example&lt;/a&gt;), &lt;a href=&quot;https://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical clustering algorithms&lt;/a&gt;, and &lt;a href=&quot;https://scikit-learn.org/stable/modules/clustering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;other clustering algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To assess the quality of the produced clusters, you could use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Silhouette_(clustering)&quot; rel=&quot;nofollow noreferrer&quot;&gt;silhouette method&lt;/a&gt; (&lt;code&gt;sklearn&lt;/code&gt; provides a function that can be used to compute the &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.metrics.silhouette_score.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;silhouette score&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your specific data frame, note that it contains repetitions, so you may want to remove them before starting the clustering procedure. Also, the IDs are not unique, but you probably don't need the IDs for clustering.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-12T16:50:09.697" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20204" PostTypeId="2" ParentId="3910" CreationDate="2020-04-12T19:48:58.233" Score="0" Body="&lt;p&gt;Your first link is from 2011, which essentially predates the current deep learning explosion. In the many years that have since passed (AlexNet 2012, ResNet 2015) we have since found that if you keep adding layers, we generally do see improved performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/chMZQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/chMZQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is due to improved training techniques and optimization breakthroughs (residual connections, ReLU, dropout etc.). But do note that the result can be diminishing. In particular, take a look at &lt;a href=&quot;https://arxiv.org/pdf/1909.01377.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Equillibrium Models&lt;/a&gt;, which essentially allow us to train (in the limit equivalence) &lt;em&gt;infinite depth&lt;/em&gt; neural networks. &lt;/p&gt;&#xA;" OwnerUserId="18086" LastActivityDate="2020-04-12T19:48:58.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20205" PostTypeId="2" ParentId="20183" CreationDate="2020-04-12T19:50:28.900" Score="2" Body="&lt;p&gt;This problem is known as &quot;style transfer&quot;. &#xA;The field was started by &lt;a href=&quot;https://arxiv.org/pdf/1508.06576.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gatsys et al. in 2016&lt;/a&gt;, and has seen a lot of work over the last few years, including conditional translation from &lt;a href=&quot;https://arxiv.org/abs/1611.07004&quot; rel=&quot;nofollow noreferrer&quot;&gt;Isola et al. in 2017&lt;/a&gt; and unpaired translation from &lt;a href=&quot;https://arxiv.org/pdf/1703.10593.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zhu et al. 2017&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This picture (Figure 2 from Gatsys et al.) shows the idea behind style transfer, and illustrates the dramatic results that are possible with these models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jNCf2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jNCf2.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15176" LastEditorUserId="2444" LastEditDate="2020-04-12T20:00:39.897" LastActivityDate="2020-04-12T20:00:39.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20206" PostTypeId="1" CreationDate="2020-04-12T20:21:12.813" Score="2" ViewCount="33" Body="&lt;p&gt;I have an extremely unbalanced  video dataset  for a two class video classification problem.All my videos in my current video dataset is &lt;span class=&quot;math-container&quot;&gt;$40$&lt;/span&gt; second long with &lt;span class=&quot;math-container&quot;&gt;$900p$&lt;/span&gt; resolution.However the dataset is highly unbalanced with &lt;span class=&quot;math-container&quot;&gt;$3000$&lt;/span&gt; samples for class A vs &lt;span class=&quot;math-container&quot;&gt;$300$&lt;/span&gt; samples to class B. Due to high imbalance, i added the following class weight implementation to my deep learning model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However  my model was still heavily biased due to high imbalance in data. I am considering   the following options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Adding more video data to balance the dataset:  My only concern here is my current dataset is uniform with same duration of videos and similar resolution of around &lt;span class=&quot;math-container&quot;&gt;$900p$&lt;/span&gt;. Will it matter if I add  very low resolution videos to balance the dataset?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adding video augmentation to current dataset.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am looking for any other recommendations that i could make use. Any pros or cons of any of these methods that I should consider to prevent any bias?&lt;/p&gt;&#xA;" OwnerUserId="35979" LastEditorUserId="9947" LastEditDate="2020-04-13T01:46:03.263" LastActivityDate="2020-04-13T01:46:03.263" Title="Possible approaches to dealing with unbalanced dataset and highly biased deep learning algorithm" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;data-science&gt;&lt;algorithmic-bias&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20208" PostTypeId="1" CreationDate="2020-04-13T00:39:51.170" Score="2" ViewCount="150" Body="&lt;p&gt;Neural networks typically have &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension that is proportional to their number of parameters and inputs. For example, see the papers &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0166218X98000146&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vapnik-Chervonenkis dimension of recurrent neural networks&lt;/a&gt; (1998) by Pascal Koirana and Eduardo D. Sontag and &lt;a href=&quot;http://mathsci.kaist.ac.kr/~nipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt; (1998) by Eduardo D. Sontag for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the universal approximation theorem (UAT) tells us that neural networks can approximate &lt;strong&gt;any&lt;/strong&gt; continuous function. See &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function&lt;/a&gt; (1989) by G. Cybenko for more details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I realize that the typical UAT only applies to continuous functions, the UAT and the results about the &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension of neural networks seem to be a little bit contradictory, but this is only if you don't know the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension&lt;/a&gt; and the implications of the UAT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how come that neural networks approximate any continuous function, but, at the same time, they usually have a &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension that is only proportional to their number of parameters? What is the relationship between the two?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2021-02-21T17:05:10.580" Title="How can neural networks approximate any continuous function but have $\mathcal{VC}$ dimension only proportional to their number of parameters?" Tags="&lt;comparison&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20209" PostTypeId="2" ParentId="20198" CreationDate="2020-04-13T01:42:48.133" Score="3" Body="&lt;p&gt;I am not an expert in robotics (definitely not in pear trees pruning either) but I will try to give some hints to partially answer and also to help reframe the problem a bit. On overall I'll give already an answer: it is most likely possible, but also most likely not convenient.&lt;/p&gt;&#xA;&lt;h1&gt;Problem statement&lt;/h1&gt;&#xA;&lt;p&gt;First things first: in general the rule is that machine learning should be applied when a task can't be automatised otherwise. Before asking if an AI can be trained to solve a task one should always think how to automatise a task using a rule based system. This is important especially because during the process of thinking how to automatise the task you will realise that some steps can be perform without an expert, while others can't be performed without any supervision. Let's brake down your task in subtasks: a system that prune trees should be (at least) capable of:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Moving between trees&lt;/li&gt;&#xA;&lt;li&gt;Selecting branches to cut&lt;/li&gt;&#xA;&lt;li&gt;Cut the selected branches&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Selecting the branch to cut is probably the step that requires most of the know how and expert supervision and for which a machine learning component might be suitable. Moving instead is a perfect example of a subtasks that could be tackled at different levels. Creating a machine able to anticipate other objects movements and avoid them in real time definitely require to train an AI component, but when you say that the environment is highly structured (trees disposed in a grid) this make me think that maybe some hand coded rules would do the trick without bothering machine learning.&lt;/p&gt;&#xA;&lt;h1&gt;Theoretical tools&lt;/h1&gt;&#xA;&lt;p&gt;Once you have understood which subtasks your machine should be capable of solving, you can start dig on the theoretical feasibility of them. Following the same order as in the previous paragraph:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Self-driving is a widely studied topics, the algorithms used to train robotic vacuum cleaners to move automatically in a house could be applied straight away to the problem of training an agent to move between trees.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Selecting the right branches involves mostly computer vision. This sub-task should actually be also dived again into sub tasks: detecting branches from other objects and select which ones should be cut. Nevertheless, the field is quite huge and training two models able to perform both actions is, in my opinion, doable.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cutting a branch is probably harder than driving in this situation, because of the small movements that might be required to reach branches positioned in difficult spots. Anyway, it is possible to train robots to perform fine-grained movements (for a funny example see: &lt;a href=&quot;https://www.youtube.com/watch?v=W_gxLKSsSIE&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robot learn to flip pancakes&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Again it depends also on how high your expectation are about the final machine/system. Should the system have a surgical precision or could it risks to brake few extra branches in hard situations? Obviously the higher the expectations the harder it would be to make everything work harmoniously.&lt;/p&gt;&#xA;&lt;h1&gt;Resources required&lt;/h1&gt;&#xA;&lt;p&gt;Last but not least you also need to understand if what you're trying to do is feasible in reality and not just in theory. A big problem when it comes to use machine leaning is that these models can be trained only with &lt;strong&gt;huge&lt;/strong&gt; amount of data.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Train an artificial agent to move in an environment can be done by reproducing the environment in an artificial simulator, which is good news. A single guy with a laptop could potentially do the job.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Collecting data to train a model able to detect branches on photo and then select which one should be cut will be highly tedious, and also expensive because data need also to be labelled by experts. Which means that some experienced people will have to take photos, in the order of tens of thousands at least, and write for each photo (using a software) which are the branches they will cut if they were working for real on that tree. I strongly doubt that a dataset like this already exists.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Training a robotic arm to cut branches will also be challenging. Despite the fact that also in this case simulators could be leveraged, the task is inherently harder and this comes with bigger difficulties (for example in designing a proper reward function if using reinforcement leaning). Concretely this mean more time to spend in research and testing.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Consider also that the success of the final model trained for each subtask would be not guaranteed at all, reason why I said at the beginning that training a system with AI modules would be probably not convenient, and that the best thing is always to try to create a rule based system first.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-13T01:48:46.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20210" PostTypeId="1" CreationDate="2020-04-13T02:30:12.487" Score="4" ViewCount="233" Body="&lt;p&gt;I am working on the following problem to gain an understanding of Bayesian networks and I need help drawing it: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Birds frequently appear in the tree outside of your window in the morning and evening; these include finches, cardinals and robins. Finches appear more frequently than robins, and robins appear more frequently than cardinals (the ratio is 7:4:1). The finches will sing a song when they appear 7 out of every 10 times in the morning, but never in the evening. The cardinals rarely sing songs and only in the evenings (in the evening, they sing 1 of every 10 times they appear). Robins sing once every five times they appear regardless of the time of day. Every tenth cardinal and robin will stay in the tree longer than five minutes. Every fourth finch will stay in the tree longer than five minutes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have tried drawing two versions of the network and would love some feedback. Currently, I am leaning more towards the right side network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mDSaP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mDSaP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35982" LastEditorUserId="2444" LastEditDate="2020-04-13T02:53:34.447" LastActivityDate="2020-04-13T08:16:29.713" Title="How can I draw a Bayesian network for this problem with birds?" Tags="&lt;bayesian-network&gt;&lt;bayesian-optimization&gt;&lt;bayesian-probability&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20211" PostTypeId="1" CreationDate="2020-04-13T03:46:47.127" Score="4" ViewCount="131" Body="&lt;p&gt;I am confused about the Q values of a duelling deep Q network (DQN). As far as I know, duelling DQNs have 2 outputs&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Advantage: how good it is to be in a particular state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value: the advantage of choosing a particular action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We can make these two outputs into Q values (reward for choosing particular action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; when in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;) by adding them together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in a DQN, we get Q values from the single output layer of the network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose that I use the same DQN model with the very same weights in my input and hidden layers and changing the output layer which gives us Q values to advantage and value outputs. &lt;em&gt;Then, during training, if I add them together, will it give me the same Q value for a particular state, supposing all the parameters of both my algorithms are the same except for the output layers?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="31380" LastEditorUserId="2444" LastEditDate="2020-04-13T14:11:17.863" LastActivityDate="2020-04-14T12:59:10.343" Title="Are Q values estimated from a DQN different from a duelling DQN with the same number of layers and filters?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20212" PostTypeId="1" CreationDate="2020-04-13T04:02:06.170" Score="1" ViewCount="192" Body="&lt;p&gt;I've been playing around with &lt;a href=&quot;https://arxiv.org/abs/1508.06576&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural style transfer&lt;/a&gt; for a about a year now, and I've been doing it with two general approaches. The first has been using a &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/examples/neural_style_transfer.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; that is available on the Keras GitHub, which produced this result when I applied it to a picture I took of an oregon grape as content and the The Great Wave off Kanagawa painting as the style. If memory serves, this image had content, style, and total variation weights equal to 1.0 and ran it for 100 iterations.&#xA;&lt;img src=&quot;https://scontent.fyvr1-1.fna.fbcdn.net/v/t1.0-9/81825142_220261185641035_2692609611696963584_o.jpg?_nc_cat=102&amp;amp;_nc_sid=8024bb&amp;amp;_nc_ohc=jcGx4Nv74cYAX9CsRTz&amp;amp;_nc_ht=scontent.fyvr1-1.fna&amp;amp;oh=c13abcd965911e0c858874d7b9173a52&amp;amp;oe=5EBA0371&quot; alt=&quot;https://scontent.fyvr1-1.fna.fbcdn.net/v/t1.0-9/81825142_220261185641035_2692609611696963584_o.jpg?_nc_cat=102&amp;amp;_nc_sid=8024bb&amp;amp;_nc_ohc=jcGx4Nv74cYAX9CsRTz&amp;amp;_nc_ht=scontent.fyvr1-1.fna&amp;amp;oh=c13abcd965911e0c858874d7b9173a52&amp;amp;oe=5EBA0371&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second approach I've been using is a free service called &lt;a href=&quot;https://deepdreamgenerator.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Dream Generator&lt;/a&gt; (not to be confused with &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepDream&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's DeepDream&lt;/a&gt;), like the output below where the content was an L-system fractal I made (a story for another time) and the style was one of the many styles shown in the 'popular style' section of the service. Note that this service offers 'Deep Style', 'Thin Style', and 'Deep Dream'. I am interesting in and currently referring to 'Deep Style'.&#xA;&lt;img src=&quot;https://q4j2g5j9.stackpathcdn.com/ddg-dream/c17830af0d8eead6fc6a727a2d5f8ba39aa4074f.jpg&quot; alt=&quot;https://q4j2g5j9.stackpathcdn.com/ddg-dream/c17830af0d8eead6fc6a727a2d5f8ba39aa4074f.jpg&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first it was just my isolated opinion that the quality of output from the Deep Dream Generator was better, but many people have confirmed that they share this opinion. The claim that one service is better than the other when it comes to making art is certainly subjective, but the claim that observational data supports a general preference is independently testable and therefore somewhat objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://github.com/keras-team/keras/blob/master/examples/neural_style_transfer.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; mentioned above allows you to control the hyperparaters that weight the content loss, style loss, and total variation loss. I sweeped values from 0 to 2 stepping by 1/10th of the range for each of the parameters giving 1000 combinations, which I turned into multiple panels of images where I saw that the output mostly looked the same! Below is an example panel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://scontent.fyvr1-1.fna.fbcdn.net/v/t1.0-9/82413905_219310189069468_4944756059307769856_n.jpg?_nc_cat=108&amp;amp;_nc_sid=8024bb&amp;amp;_nc_ohc=XcVjqhZUgycAX-yAnDg&amp;amp;_nc_ht=scontent.fyvr1-1.fna&amp;amp;oh=653dae375ffbd8642e9939d0be9cd8a4&amp;amp;oe=5EBAFEA2&quot; alt=&quot;https://scontent.fyvr1-1.fna.fbcdn.net/v/t1.0-9/82413905_219310189069468_4944756059307769856_n.jpg?_nc_cat=108&amp;amp;_nc_sid=8024bb&amp;amp;_nc_ohc=XcVjqhZUgycAX-yAnDg&amp;amp;_nc_ht=scontent.fyvr1-1.fna&amp;amp;oh=653dae375ffbd8642e9939d0be9cd8a4&amp;amp;oe=5EBAFEA2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking for what could be different, I noted that the Deep Style tool has some additional hyperparameters that the Keras setup does not. Here's my consideration of what seems similar or different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, they have an 'Enhance' setting which is described as &lt;em&gt;How much strength to apply when enhancing the image. Higher value will produce higher quality output but will diminish some of the original texture and structure.&lt;/em&gt; I'm doubtful this is equivalent to the Keras model's hyperparameters, and I'm not sure what they're doing here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly is 'Resolution'which is described as &lt;em&gt;Max resolution in MP (Mega Pixels).&lt;/em&gt; I can easily change the size of images I'm training with, at least within what my computer system resources can handle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thirdly is 'Depth' described as &lt;em&gt;The more depth you set the better quality you will get but it will take more time to generate the image.&lt;/em&gt; I suspect they have different style transfer models where different selections of layers can be used, with 'deeper' being the inclusion of more layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourthly, is 'Style Weight' described as &lt;em&gt;Default is 50%. Less means more like the original image and more means more like the style image.&lt;/em&gt; I strongly suspect that this is the same style weight hyperparater in the aforementioned Keras model, and therefore I can readily control this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fifthly is 'Style Scale' described as &lt;em&gt;Scale style image. Makes extracted patterns from the style image bigger/smaller.&lt;/em&gt; I'm not really sure if this represents the total varation hyperparater in the loss function or not. The Keras script describes the 'total variation' as follows: &lt;em&gt;The total variation loss imposes local spatial continuity between&#xA;the pixels of the combination image, giving it visual coherence.&lt;/em&gt; For all I know, the total variation could be the 'Enhance' mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sixth is 'Preserve Colors', which doesn't have a description. As in the example above, I usually have this turned off when I use the online service, but I definitely don't have an analog for the Keras setup I'm using. I've wondered if this is training on a black and white version first to change the structures slightly, and then do a second colorizing step... But I really don't know with any confidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the actual math or computer science concepts behind these unfamiliar hyperparameters?&lt;/p&gt;&#xA;" OwnerUserId="35986" LastActivityDate="2020-04-13T04:02:06.170" Title="What are the actual math or computer science concepts behind these unfamiliar hyperparameters in the Deep Dream Generator's Deep Style?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20213" PostTypeId="1" CreationDate="2020-04-13T06:47:22.653" Score="1" ViewCount="53" Body="&lt;p&gt;Could someone please explain to me why in VC theory, specifically, when calculating the VC dimension, the growth function needs to be polynomial in order for the learning algorithm to be consistent? Why polynomial, and where does the name growth function come from exactly?&lt;/p&gt;&#xA;" OwnerUserId="35990" LastEditorUserId="2444" LastEditDate="2020-04-13T12:04:46.850" LastActivityDate="2020-04-13T12:04:46.850" Title="Why does the growth function need to be polynomial in order for the learning algorithm to be consistent?" Tags="&lt;machine-learning&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20214" PostTypeId="1" AcceptedAnswerId="20222" CreationDate="2020-04-13T07:38:47.727" Score="4" ViewCount="505" Body="&lt;p&gt;The &lt;a href=&quot;https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/beginner.ipynb#scrollTo=he5u_okAYS4a&amp;amp;line=1&amp;amp;uniqifier=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;beginner colab example&lt;/a&gt; for tensorflow states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note: It is possible to bake this &lt;code&gt;tf.nn.softmax&lt;/code&gt; in as the activation function for the last layer of the network. While this can make the model output more directly interpretable, this approach is discouraged as it's impossible to&#xA;  provide an exact and numerically stable loss calculation for all models when using a softmax output. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My question is, then, why? What do they mean by &lt;em&gt;impossible to provide an exact and numerically stable loss calculation&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="35992" LastEditorUserId="2444" LastEditDate="2020-04-14T18:10:27.197" LastActivityDate="2020-12-17T12:09:45.357" Title="Why does TensorFlow docs discourage using softmax as activation for the last layer?" Tags="&lt;tensorflow&gt;&lt;loss-functions&gt;&lt;softmax&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20216" PostTypeId="5" CreationDate="2020-04-13T12:13:30.613" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T12:13:30.613" LastActivityDate="2020-04-13T12:13:30.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20217" PostTypeId="4" CreationDate="2020-04-13T12:13:30.613" Score="0" Body="For questions related to the softmax function, which a function that takes as input a vector of K real numbers, and normalizes it into a probability distribution consisting of K probabilities proportional to the exponentials of the input numbers. The softmax is often used as the activation function of the output layer of a neural network." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T12:13:30.613" LastActivityDate="2020-04-13T12:13:30.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20218" PostTypeId="1" CreationDate="2020-04-13T12:31:11.140" Score="2" ViewCount="39" Body="&lt;p&gt;&lt;br /&gt;&#xA;I want to develop a CNN model to identify 24 hand signs in American Sign Language. I created a custom &lt;a href=&quot;http://www.kaggle.com/dataset/04322e12909662a11f5154857a749b54b389b77bd99f5bd4c3e0db0ec34abe68&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset&lt;/a&gt; that contains 3000 images for each hand sign i.e. 72000 images in the entire dataset.&#xA;&lt;br /&gt;&lt;br /&gt;&#xA;For training the model, I would be using 80-20 dataset split (2400 images/hand sign in the training set and 600 images/hand sign in the validation set). My question is:&lt;br /&gt;&#xA;Should I randomly shuffle the images when creating the dataset? And Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br /&gt;PS: Based on my previous experience, it led to validation loss being lower than training loss and validation accuracy more than training accuracy.&lt;/p&gt;&#xA;" OwnerUserId="33467" LastEditorUserId="33467" LastEditDate="2020-04-14T19:30:43.233" LastActivityDate="2020-04-14T19:30:43.233" Title="Creating Dataset for Image Classification" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20219" PostTypeId="1" CreationDate="2020-04-13T12:46:16.647" Score="1" ViewCount="52" Body="&lt;p&gt;Is there an online tool that can predict accuracy given only the dataset as input (i.e. without the compiled model)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would help to understand how data augmentation/distribution standardization, etc., is likely to change the accuracy.&lt;/p&gt;&#xA;" OwnerUserId="36000" LastEditorUserId="36000" LastEditDate="2020-04-13T17:09:41.403" LastActivityDate="2020-04-13T17:26:09.147" Title="Is there an online tool that can predict accuracy given only the dataset?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2020-12-05T14:37:30.503" ContentLicense="CC BY-SA 4.0" />
  <row Id="20220" PostTypeId="1" AcceptedAnswerId="20233" CreationDate="2020-04-13T13:01:57.740" Score="2" ViewCount="51" Body="&lt;p&gt;I'm following Stanford reinforcement learning videos on youtube. One of the assignments asks to write code for policy evaluation for &lt;a href=&quot;https://gym.openai.com/envs/FrozenLake-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gym's FrozenLake-v0 environment&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the course (and books I have seen), they define policy evaluation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^\pi_k(s)=r(s,\pi(s))+\gamma\sum_{s'}p(s'|s,\pi(s))V^\pi_{k-1}(s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusion is that in the frozen lake example, the reward is tied to the result of the action. So, for each pair state-action, I have a list that contains a possible next-state, the probability to get to that next-state and the reward. For example, being in the target state and performing any action brings a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, but being in any state that brings me to the target state gives me a reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that, for this example, I need to rewrite &lt;span class=&quot;math-container&quot;&gt;$V^\pi_k(s)$&lt;/span&gt; as something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^\pi_k(s)= \sum_{s'} p(s'|s,\pi(s)) [r(s,\pi(s), s')+ \gamma V^\pi_{k-1}(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="35999" LastEditorUserId="2444" LastEditDate="2020-04-16T19:24:16.953" LastActivityDate="2020-04-16T19:24:16.953" Title="How can I implement policy evaluation when reward is tied to an action outcome?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;implementation&gt;&lt;gym&gt;&lt;policy-evaluation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20221" PostTypeId="2" ParentId="20194" CreationDate="2020-04-13T13:15:31.260" Score="1" Body="&lt;p&gt;You can sometimes exploit the structure of your matrix to perform faster matrix multiplication. For example, if your matrix is &lt;a href=&quot;https://en.wikipedia.org/wiki/Sparse_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;sparse&lt;/a&gt; (or dense), there are algorithms that exploit this fact. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you can actually compute &lt;span class=&quot;math-container&quot;&gt;$A^n$&lt;/span&gt; in less time than &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(n^3$&lt;/span&gt;). For example, have a look at &lt;a href=&quot;https://cs.stackexchange.com/q/4998/20691&quot;&gt;this question at CS SE&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/q/12311869/3924118&quot;&gt;this one at Stack Overflow (SO)&lt;/a&gt;. Note that the provided solutions may not be numerically stable, so I am not suggesting you use them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if you perform your operations on the GPU, they could be faster in practice. See e.g. &lt;a href=&quot;https://stackoverflow.com/q/51344018/3924118&quot;&gt;this question at SO&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/q/5372/9925&quot;&gt;this one at SciComp SE&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-13T13:15:31.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20222" PostTypeId="2" ParentId="20214" CreationDate="2020-04-13T14:23:48.843" Score="1" Body="&lt;p&gt;It's because of gradient computations: automatic differentiation will compute the gradient for each module and if you have a standalone &lt;code&gt;crossentropy&lt;/code&gt; module the over all loss will be unstable (~1/x so it will diverge for small input values) whereas if you use a &lt;code&gt;softmax + crossentropy&lt;/code&gt; module all-in-one, then it becomes numerically stable (y-p)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slides from DeepMind's Simon Osindero lecture at UCL in 2016:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OyGix.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OyGix.jpg&quot; alt=&quot;CE&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uGw1c.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uGw1c.jpg&quot; alt=&quot;CE+Softmax&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11351" LastEditorUserId="11351" LastEditDate="2020-04-14T01:51:20.540" LastActivityDate="2020-04-14T01:51:20.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20223" PostTypeId="2" ParentId="20190" CreationDate="2020-04-13T14:29:40.540" Score="1" Body="&lt;p&gt;You can use MNIST obviously but I'd also suggest you have a look at UC Irvine's datasets: &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://archive.ics.uci.edu/ml/datasets.php&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/t0kpg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t0kpg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11351" LastActivityDate="2020-04-13T14:29:40.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20224" PostTypeId="2" ParentId="16172" CreationDate="2020-04-13T14:32:50.753" Score="0" Body="&lt;p&gt;Below is a listing of Keras application models that can be used easily in transfer learning. Note VGG has on the order of 140 million parameters which is why it is slow.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;&#xA;Model               Size     Top-1 Accuracy  Top-5 Accuracy  Parameters    1Depth&#xA;Xception             88 MB      0.790           0.945         22,910,480    126&#xA;VGG16               528 MB      0.713           0.901        138,357,544    23&#xA;VGG19               549 MB      0.713           0.900        143,667,240    26&#xA;ResNet50             98 MB      0.749           0.921         25,636,712    -&#xA;ResNet101           171 MB      0.764           0.928         44,707,176    -&#xA;ResNet152           232 MB      0.766           0.931         60,419,944    -&#xA;ResNet50V2           98 MB      0.760           0.930         25,613,800    -&#xA;ResNet101V2         171 MB      0.772           0.938         44,675,560    -&#xA;ResNet152V2         232 MB      0.780           0.942         60,380,648    -&#xA;InceptionV3          92 MB      0.779           0.937         23,851,784    159&#xA;InceptionResNetV2   215 MB      0.803           0.953         55,873,736    572&#xA;MobileNet            16 MB      0.704           0.895          4,253,864    88&#xA;MobileNetV2          14 MB      0.713           0.901          3,538,984    88&#xA;DenseNet121          33 MB      0.750           0.923          8,062,504    121&#xA;DenseNet169          57 MB      0.762           0.932         14,307,880    169&#xA;DenseNet201          80 MB      0.773           0.936         20,242,984    201&#xA;NASNetMobile         23 MB      0.744           0.919          5,326,716    -&#xA;NASNetLarge         343 MB      0.825           0.960         88,949,818    -&#xA;&#xA;I tend to use the MobileNet model for transfer learning because it has about 4 million&#xA;parameters so it much faster than most models. It should perform as well as VGG on your&#xA;data set. If it does not tuning the hyper parameters may be required. I find that using&#xA;an adjustable learning such as the Keras ReduceLROnPlateau callback along with the&#xA;ModelCheckpoint callback both monitoring validation loss works very well. Documentation&#xA;is [here][1].&#xA;You might also try the efficientNet model which comes in various sizes and has high &#xA;accuracy. Documentation is [here][2]&#xA;&#xA;&#xA;  [1]: https://keras.io/callbacks/&#xA;  [2]: https://github.com/Tony607/efficientnet_keras_transfer_learning&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-13T14:32:50.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20225" PostTypeId="2" ParentId="17522" CreationDate="2020-04-13T14:49:53.440" Score="1" Body="&lt;p&gt;I think i found out how that works, so i made a short article about it .&#xA;&lt;a href=&quot;https://medium.com/@kourloskostas/python-spam-filter-86b21d7d1564&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@kourloskostas/python-spam-filter-86b21d7d1564&lt;/a&gt;&#xA;I hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="32762" LastActivityDate="2020-04-13T14:49:53.440" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20227" PostTypeId="2" ParentId="20219" CreationDate="2020-04-13T15:21:56.997" Score="1" Body="&lt;p&gt;If I get correctly what you're asking, the answer is no, there is no way of knowing in advance how good a model would perform in a dataset without training a model on it. That's the whole point of data science, you try, you analyse the results, you try again using the knowledge you got from your previous attempts. It would be nice to hack the whole field and know in advance what to do to get the perfect model but it is rather unrealistic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, there are some standard steps that usually help understanding if you're going on the right direction. For example creating a random benchmark to see how much your model is better than a random one. To create such benchmark you definitely don't need a specific tool, all main programming languages provide built in function to generate random numbers, and that's basically all that you need. For example in python you could do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;# create 30 true labels &#xA;y_true = [1,2,3]*10&#xA;&#xA;# generate 30 random labels&#xA;y_random = np.random.randint(0,3,size=30)&#xA;&#xA;# calculate accuracy for random model&#xA;acc_random = sum(y_true==y_random)/len(y_true) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another thing that is worth to mention is that in the academy people are pushing more and more to use the same datasets in order to have comparable results. for example if you're trying to train an architecture that should be used for images classification, some golden dataset you must test your models on are the &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt; ones. It is well known that on these datasets a good model should achieve more than 99% accuracy, therefore this is an establish lower bound for these datasets. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-04-13T15:21:56.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20228" PostTypeId="2" ParentId="12878" CreationDate="2020-04-13T16:59:49.730" Score="1" Body="&lt;p&gt;It's important to remember what exactly the loss is measuring, and have some typical values in mind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cross-entropy loss is &lt;span class=&quot;math-container&quot;&gt;$-\mathbb{E}_{x,y\sim p}\left[\log q(y|x)\right]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the data distribution and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is the model distribution. &#xA;A couple of points about the loss:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It's nonnegative, in the range &lt;span class=&quot;math-container&quot;&gt;$[0, \infty)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Predicting randomly (assuming balanced classes) gives loss equal to &lt;span class=&quot;math-container&quot;&gt;$\log k$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In your case with four classes, the loss for a random classifier is &lt;span class=&quot;math-container&quot;&gt;$\log 4 \approx 1.39$&lt;/span&gt;. So the story for what happened with your model is probably that initially (due to initialization, etc) it predicted high but wrong confidence, such as giving 99% probabilities to certain classes. This gives very high loss, but then after training for a while it reduces its loss to just under the random loss by predicting 25% on all examples. &lt;/p&gt;&#xA;" OwnerUserId="15176" LastEditorUserId="15176" LastEditDate="2020-04-13T19:27:24.330" LastActivityDate="2020-04-13T19:27:24.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20229" PostTypeId="2" ParentId="15743" CreationDate="2020-04-13T17:09:09.327" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;As I understand Resnet has some identity mapping layers that their task is to create the output as the same as the input of the layer. the resnet solved the problem of accuracy degrading. But what is the benefit of adding identity mapping layers in intermediate layers?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See this is applicable to deep/very deep networks. We decide to add layers when the model output is not converging to the expected output (it is due to very slow convergence). By this mapping, author has suggested that some portion of complexity of the model can directly be adjusted with input value leaving just residual value for adjustment. The output is mapped to input by identity function - so it is identity mapping. So the shortcut identity mapping is doing the task of some layers in plain neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The identity mapping is applicable only if output and input are of same shape otherwise linear projection is required.&lt;/p&gt;&#xA;" OwnerUserId="36009" LastEditorUserId="2444" LastEditDate="2020-04-13T17:20:14.233" LastActivityDate="2020-04-13T17:20:14.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20230" PostTypeId="2" ParentId="20219" CreationDate="2020-04-13T17:26:09.147" Score="1" Body="&lt;p&gt;There is no tool to achieve what you desire. A &quot;rough&quot; way to get a guess is to &quot;compare&quot; your data set to other known data sets for which there are benchmark accuracy models. Compare is a vague definition at best but things you might compare are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;number of your classes versus the number of classes in the reference data set&lt;/li&gt;&#xA;&lt;li&gt;the number of train, validation and test samples&lt;/li&gt;&#xA;&lt;li&gt;the relative similarity of the data - that is are they images? If so image comapare size, are the images cropped to the region of interest, etc&lt;/li&gt;&#xA;&lt;li&gt;similarity of class features -that is are your classes very similar to each other. For example classifying dogs by breed can be difficult because some dog breeds look almost identical versus a situation like classifying different types of animals which is easier. Estimate your similarity of classes with respect to&#xA;similarity of classes in the reference data set. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So you might be able to generate a &quot;rough&quot; estimate of what you might expect your model to achieve particularly if you use transfer learning of benchmark models. &lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-13T17:26:09.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20231" PostTypeId="1" CreationDate="2020-04-13T17:49:35.663" Score="1" ViewCount="218" Body="&lt;p&gt;Similarly to the question &lt;a href=&quot;https://ai.stackexchange.com/q/58/2444&quot;&gt;Who first coined the term Artificial Intelligence?&lt;/a&gt;, who first coined the term &quot;artificial general intelligence&quot;?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-21T12:04:53.877" Title="Who first coined the term &quot;artificial general intelligence&quot;?" Tags="&lt;terminology&gt;&lt;agi&gt;&lt;history&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20232" PostTypeId="2" ParentId="20231" CreationDate="2020-04-13T17:49:35.663" Score="3" Body="&lt;p&gt;According to &lt;a href=&quot;https://goertzel.org/who-coined-the-term-agi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ben Goertzel&lt;/a&gt;, the first person that &lt;strong&gt;probably&lt;/strong&gt; used the term &quot;artificial general intelligence&quot; (in an article related to artificial intelligence) was &lt;a href=&quot;https://medium.com/@mgubrud&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mark Avrum Gubrud&lt;/a&gt; in the 1997 article &lt;a href=&quot;https://foresight.org/Conferences/MNT05/Papers/Gubrud/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanotechnology and International Security&lt;/a&gt;. Here's an excerpt from the article.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By advanced artificial general intelligence, I mean AI systems that rival or surpass the human brain in complexity and speed, that can acquire, manipulate and reason with general knowledge, and that are usable in essentially any phase of industrial or military operations where a human intelligence would otherwise be needed. Such systems may be modeled on the human brain, but they do not necessarily have to be, and they do not have to be &quot;conscious&quot; or possess any other competence that is not strictly relevant to their application. What matters is that such systems can be used to replace human brains in tasks ranging from organizing and running a mine or a factory to piloting an airplane, analyzing intelligence data or planning a battle.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that the term &quot;AGI&quot; could have been used even before that Mark A. Gubrud's article (as Goertzel also suggested). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, Ben Goertzel help to popularise this term, especially with the book &lt;a href=&quot;https://link.springer.com/book/10.1007/978-3-540-68677-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;. AGI was previously known as &quot;strong AI&quot;, which goes back to John Searle's &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chinese Room argument&lt;/a&gt;, although strong AI often refers to an AGI with consciousness, and the definition of AGI doesn't necessarily imply consciousness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about the history of the term &quot;AGI&quot; in &lt;a href=&quot;https://goertzel.org/who-coined-the-term-agi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Goertzel's blog post&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-20T13:50:05.823" LastActivityDate="2020-04-20T13:50:05.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20233" PostTypeId="2" ParentId="20220" CreationDate="2020-04-13T18:21:43.693" Score="1" Body="&lt;p&gt;The renowned book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (2nd edition), by Sutton and Barto, provides a different update rule than your first update rule for policy evaluation. Their update rule is more similar to your second update rule. See &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=96&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 4.1&lt;/a&gt;. They also provide the pseudocode for policy evaluation on page 75 of the book. You can also find the pseudocode &lt;a href=&quot;https://github.com/udacity/rl-cheatsheet/blob/master/cheatsheet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, note that the update rule doesn't need to change only because the rewards are tied to the outcome of an action. This information is encoded in the functions &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (the transition function) and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; (the reward function) of the Markov decision process, which is incorporated in the update rule. If you want to understand the update rule, you should read the relevant pages (especially, chapter 4) of the cited book.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-13T18:28:56.597" LastActivityDate="2020-04-13T18:28:56.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20234" PostTypeId="2" ParentId="20190" CreationDate="2020-04-13T21:17:11.343" Score="1" Body="&lt;p&gt;&lt;em&gt;This answer doesn't provide a declaration as to which dataset(s) are used quasi-ubiquitously in research/literature. It simply provides a frame-of-reference for where to look for structured datasets and examples of two structured datasets that could be used in general.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You want to look for structured datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good examples of this are things like housing price datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out &lt;a href=&quot;https://datasetsearch.research.google.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Datasets&lt;/a&gt; (specifically the &lt;a href=&quot;https://datasetsearch.research.google.com/search?query=kaggle&amp;amp;docid=LsRzIdlgaTTflyQNAAAAAA%3D%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;datasets hosted by Kaggle&lt;/a&gt;). Many of these are structured data datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an answer directly though, try a housing price dataset like the &lt;a href=&quot;https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boston Housing Price Dataset&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use the famous &lt;a href=&quot;https://www.kaggle.com/c/titanic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Titanic Dataset on Kaggle&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36013" LastEditorUserId="2444" LastEditDate="2020-04-14T16:52:29.630" LastActivityDate="2020-04-14T16:52:29.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20235" PostTypeId="1" CreationDate="2020-04-13T22:16:28.487" Score="2" ViewCount="31" Body="&lt;p&gt;I coded a non-zero-sum game of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; agents in a discrete dynamic environment to RL with Q-learning and DQN agents.&lt;/p&gt;&#xA;&lt;p&gt;It's like a marathon. Only two actions are available per agent: &lt;span class=&quot;math-container&quot;&gt;$\{ G  \text{ (move forward}) , S \text{ (stay to its position}) \}$&lt;/span&gt;. Every agent has &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; possible individual positions, the other agents cannot interfere with its path to the terminal position. Only when one agent reaches its terminal position gets a full reward. When everyone reaches a terminal state, all get &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; rewards. If more than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; but less than &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; reach their terminals, they get a small reward.&lt;/p&gt;&#xA;&lt;p&gt;Now, I try to formalize it as a Markov Game (MG), but I don't have a solid mathematical background.&lt;/p&gt;&#xA;&lt;p&gt;My first question is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When we model a problem as an RL problem, the transition probability (TP) distribution is not required, while an MDP and MG require TP. But then how are all RL problems modeled first into MDP or MG?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As I have read in literature, I understand that I will treat the action sets of all other players as a &amp;quot;team&amp;quot; joint set of actions.&lt;/p&gt;&#xA;&lt;p&gt;Second question:&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;How can I specialize the TP function to the specific problem I want to model? Should I just mention the general function equation?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I have tried so far is to explicitly describe it, but I think I am not getting something:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The probability of the transition from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, where in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; a number of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; agents move a step forward is equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, given that they all chose action &lt;span class=&quot;math-container&quot;&gt;$G \in Α$&lt;/span&gt; and that the rest &lt;span class=&quot;math-container&quot;&gt;$n-k$&lt;/span&gt;, if any, all chose the action &lt;span class=&quot;math-container&quot;&gt;$S \in Α$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is an integer &lt;span class=&quot;math-container&quot;&gt;$1 \leq k \leq n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The probability of the transition from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, where in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; a number of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; agents to earn the high payoff is equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, given that &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt;, its position is equal to &lt;span class=&quot;math-container&quot;&gt;$m-1$&lt;/span&gt;, it chooses action &lt;span class=&quot;math-container&quot;&gt;$G \in A$&lt;/span&gt; and that the rest &lt;span class=&quot;math-container&quot;&gt;$n-k$&lt;/span&gt; all chose the action &lt;span class=&quot;math-container&quot;&gt;$S \in Α$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the max possible position for each agent.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The probability of the transition from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, where in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; a number of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; agents to earn a low payoff is equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, given that &lt;span class=&quot;math-container&quot;&gt;$k&amp;gt;1$&lt;/span&gt;, their position is equal &lt;span class=&quot;math-container&quot;&gt;$m-1$&lt;/span&gt;, they all chose action &lt;span class=&quot;math-container&quot;&gt;$G \in Α$&lt;/span&gt; and  that the rest &lt;span class=&quot;math-container&quot;&gt;$n-k$&lt;/span&gt; all chose the action &lt;span class=&quot;math-container&quot;&gt;$S \in Α$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is an integer with &lt;span class=&quot;math-container&quot;&gt;$1 &amp;lt; k \leq n$&lt;/span&gt; and m is the max possible position for each agent&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36018" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-14T03:33:32.000" Title="How can I formalise a non-zero-sum game of $N$ agent as Markov game?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20236" PostTypeId="2" ParentId="20186" CreationDate="2020-04-13T22:36:24.027" Score="1" Body="&lt;p&gt;I think your deduction is &lt;em&gt;mostly&lt;/em&gt; correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks of depth are universal function approximators. This means that in principal, for any function of the form you describe, there's a NN that approximates it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a particular NN architecture of fixed width and depth, with fixed connections is &lt;em&gt;not&lt;/em&gt; a universal approximator for all functions. Only an infinitely wide NN is, and that's a theoretical construct, not something you can make in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, a practitioner using a NN injects their prior beliefs about the problem  &lt;em&gt;by selecting an architecture&lt;/em&gt;. For example, it is common to use the ImageNet or ResNet architectures for image processing tasks. Those architectures are less effective on other types of tasks. For instance, it should be clear that they are ineffective on say, a time-series analysis task.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-04-13T22:36:24.027" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20237" PostTypeId="2" ParentId="20120" CreationDate="2020-04-13T22:44:44.440" Score="1" Body="&lt;p&gt;I think the wrong assumption here is that you've forgotten the cost of encoding the new features!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MDL should be considered relative to the &lt;em&gt;original&lt;/em&gt; or raw dataset. The idea is that you want to find an expression you could send to someone else that encodes the structure of the dataset in terms of the original variables. If you define new features, you need to send a description of those features along with your model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this clearer, imagine you call me up on the phone, and we're both looking at your left hand image. You say to me 'Yeah, you just draw a line through it at &lt;span class=&quot;math-container&quot;&gt;$p=a$&lt;/span&gt;'. The natural question for me to ask is 'What's p?'. If you have to tell me what &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is, then it's part of the description length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A circular model for your lefthand image does have a MDL for its decision boundary of something like &lt;span class=&quot;math-container&quot;&gt;$(x−a)^2+(y−b)^2=c$&lt;/span&gt;. However, the feature transformation you've selected has a description length of &lt;span class=&quot;math-container&quot;&gt;$p=(x-a)^2 + (y-b)^2$&lt;/span&gt;. It should be clear that the description lengths for the linear model through &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and the circular model through &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are identical.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-04-13T22:44:44.440" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20238" PostTypeId="1" AcceptedAnswerId="20334" CreationDate="2020-04-13T23:15:47.213" Score="2" ViewCount="55" Body="&lt;p&gt;I'm experimenting with training a feedforward neural network using a genetic algorithm and I've done a few tests using both the mean squared error and classification error functions as fitness heuristic in the GA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I use MSE as error function, my GA tends to converge around an MSE of 0.1 (initial conditions have an MSE of around 0.9). Testing system accuracy with this network gives me 95%+ for both training and testing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, when I use classification error as my heuristic, my GA tends to converge around when the MSE is about 0.3. System accuracy is still around the same at 95%+.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is, if you had two networks, one showing an MSE of 0.1 and one an MSE of 0.3, but both perform approximately the same in terms of accuracy, what can I deduce from the differences in MSE? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words: which network is &quot;better&quot;, even if the accuracy is the same? Does a lower MSE mean anything below a certain amount? I could train my network for 100x as many generations and get a better MSE but not necessarily a better accuracy. Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some context:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UW3ga.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UW3ga.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the MSE is approximately 1.5 (epoch 250), the accuracy seems to match when the MSE is approximately 2.0 (epoch 50). Why does the accuracy not increase despite MSE decreasing?&lt;/p&gt;&#xA;" OwnerUserId="31112" LastEditorUserId="1671" LastEditDate="2020-04-17T21:26:06.660" LastActivityDate="2020-04-17T21:26:06.660" Title="What does it mean if classification error is equal between two networks but the MSE is different?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;genetic-algorithms&gt;&lt;mean-squared-error&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20239" PostTypeId="2" ParentId="20057" CreationDate="2020-04-13T23:19:01.733" Score="1" Body="&lt;p&gt;I think Cross-Validation serves a completely different purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your post, it looks like you think we would use CV to get a better estimate of the parameters of our model (i.e. the model parameters after cross validation are closer to the parameters of the test data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, we use CV to get an estimate of &lt;em&gt;generalization error&lt;/em&gt; while keeping our test set outside the training process. That is, we use it to answer the question &quot;What is the size of the difference between my training and testing performance likely to be?&quot;. If you have an estimate of this that you are confident in, you can be confident that when you deploy a model to your customers, the model will actually work as you expect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're only going to build a single model, then you don't need cross validation. You just train the model on the training data, and test it on the test data. Then you have an unbiased estimate of generalization error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we might want to try out many different kinds of models, and many different parameters (broadly, we might want to do &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyperparameter tuning&lt;/a&gt;). To do this, we need to understand how generalization error changes as we change our hyperparameters, and then use this information to pick hyperparameter values that we think will minimize the actual error when we deploy the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could do this by training different models on the training set, and then testing them on the test set, recording the difference in model performance on the two sets. If you use this as a basis to pick a model though, you have effectively pulled the test set &lt;em&gt;inside&lt;/em&gt; your training process (model parameters were implicitly selected using the test set, since you picked the parameters with the lowest test error). This bias will make your true generalization error much larger than what you observed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a stop gap, you could split your training set into a 'real training' set and a validation set. You could train models on the 'real training' set, and then measure their performance on the validation set. The difference would be a biased (but hopefully still useful) estimate of generalization error. You could then test against the test set just once (at the end) to get an unbiased estimate that you can use to decide whether or not to deploy the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better workflow is to use CV on the training set to get an estimate of generalization error during hyperparameter optimization. You get K samples for k-fold cross-validation, so you can do statistical testing to see whether one model truly has better generalization error than another, or whether it's just a fluke. This decreases the degree of bias in your estimates of generalization error. Then, once you've completed hyperparameter optimization, you can run your final model against the test set &lt;em&gt;once&lt;/em&gt; to obtain a truly unbiased estimate of your final generalization error.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-04-13T23:19:01.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20240" PostTypeId="2" ParentId="20193" CreationDate="2020-04-14T01:31:09.387" Score="2" Body="&lt;p&gt;It is not really a metter of what model, but if it is possible at all to predict what you're trying to predict. Let's take a similar dataset from kaggle: &lt;a href=&quot;https://www.kaggle.com/camnugent/california-housing-prices&quot; rel=&quot;nofollow noreferrer&quot;&gt;California Housing Prices&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This dataset contains house prices and other information among which the number of bedrooms per house. As suggested by Oliver in the comments we can compute the Person coefficient to estimate the correlation between the two variables. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;from scipy.stats import pearsonr&#xA;&#xA;df = pd.read_csv('housing.csv')&#xA;df = df.apply(lambda row: row[df['total_bedrooms'] &amp;lt;= 20]) # select subset of dataframe for sake of clarity&#xA;df.dropna(inplace=True)&#xA;&#xA;x = df['median_house_value'] # our single feature  &#xA;y = df['total_bedrooms'] # target labels&#xA;&#xA;print('Correlation: \n', pearsonr(x,y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;Correlation: &#xA;&amp;gt;&amp;gt;(-0.14015312664251944, 0.12362969210761204)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The correlation is pretty low, which means that the price and number of bedrooms are basically not related. We can also plot the points to check that indeed there is no correlation at all.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df.plot(x='total_bedrooms',y='median_house_value',kind='scatter')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XEKAY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XEKAY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training a model to predict the number of bedrooms uniquely from the price would mean to find a function that can interpolate all those points, which is an impossible task since we have several different prices for houses with the same amount of bedrooms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to tackle a problem like this would be to expand the dimensionality of the data, for example by using a Support Vector Machine with a non linear kernel. But even with non linear kernels you can't do miracles, so if you're dataset looks like this one, the only solution would be to expand your dataset to include extra features. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-04-14T01:31:09.387" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20241" PostTypeId="1" CreationDate="2020-04-14T02:25:39.673" Score="6" ViewCount="621" Body="&lt;p&gt;Would it be ethical to allow an AI to make life-or-death medical decisions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, where there an insufficient number of ventilators during a respiratory pandemic, not every patient can have one.  It seems like a straight forward question, but before you answer, consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Human decision-making in this regard is a form of algorithm.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(For instance, the statistics and rules that determine who gets kidney transplants.)&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Even if the basis for the decision is statistical, the ultimate decision making process could be heuristic, so at least the bias could be identified.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In other words, the goal of this process, specifically, is favoring one patient over another, but doing so in the way that has the greatest utility.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Statistical bias is a core problem of Machine Learning, but human decision making is also subject to this condition.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One of the arguments in favor might be that at least the algorithm would be impartial, here in relation to human bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, where there is scarcity, utilitarianism becomes more imperative.  (Part of the trolley problem is you only have two tracks.)  But the trolley problem is also relevant because it can be a commentary on the burden of responsibility.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-04-14T12:48:23.923" LastActivityDate="2020-04-15T12:03:55.193" Title="Would it be ethical to allow an AI to make life-or-death medical decisions?" Tags="&lt;philosophy&gt;&lt;social&gt;&lt;ethics&gt;&lt;healthcare&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20242" PostTypeId="2" ParentId="20191" CreationDate="2020-04-14T03:34:20.783" Score="3" Body="&lt;p&gt;Weights initialisation is strictly related to the vanishing/exploding gradient problem. For a complete explanation, please check this awesome &lt;a href=&quot;https://www.deeplearning.ai/ai-notes/initialization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; (also from deeplearning.ai). Here I'll summarise the main concepts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;initialising weight all to zero will cause all weights to have the same derivative value with respect to the loss function, hence the network would be incapable of learning anything. &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;initialising to zero the biases has no drawback since they are constants (no effect at all when computing the derivative anyway).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;initialising weights with too high or too small values will lead to an exploding gradient (oscillating gradient values without convergence) or a vanishing gradient (small gradient values variation that converge before reaching the loss global minimum).     &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In order to avoid these problems, a method called &lt;em&gt;Xavier initialisation&lt;/em&gt; has been proposed: the weights should be initialised in such a way that they will generate activations scores with a distribution that has:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mean 0&lt;/li&gt;&#xA;&lt;li&gt;Constant variance across layers (i.e. no vanishing/exploding) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The value &quot;np.sqrt(layer_dims[l - 1])&quot; pup up when imposing the second constrain.  For a formal prove check the page I linked. To grasp the concept just focus on the fact that the variance of the weights of a layer depends on the amount of nodes of the previous layer. This mean that for layers preceded by layers with a big amount of hidden nodes, the weights will be initialised with a small variance, and this is ok cause we don't want a small bunch nodes to have stronger influence on the subsequent activations. But in layers which instead are preceded by layers with a small amount of nodes, it's ok to allow the weights to vary more. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-04-14T03:34:20.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20243" PostTypeId="2" ParentId="20241" CreationDate="2020-04-14T04:07:17.980" Score="3" Body="&lt;p&gt;I disagree with the idea that a trained Machine Learning model would be impartial. Models are trained on data sets that contain features. Humans prepare those data sets and decide what features are included in the data set. The model only knows what it is trained on. Human bias is still there just less blatantly obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address your question directly, the answer I believe is that it is no more or less ethical than to have humans make such decisions since in the end humans created the AI model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern, however, is simply this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we offload this to AI we will longer feel responsible for the results. The mentality of &quot;the machine&quot; made the choice will make it very easy to allow us to abdicate. This is especially true if the people implementing the AI decisions are not the ones who developed the AI. &#xA;Of course, humans having to repeatedly make life and death will suffer a serious and potentially devastating toll. So, in the end, it is a trade-off, but, from my perspective, I think the risk of abdication and the consequences thereof carry a heavier weight. But then I am not the one faced with making life and death choices.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-04-15T12:03:55.193" LastActivityDate="2020-04-15T12:03:55.193" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20244" PostTypeId="1" AcceptedAnswerId="20248" CreationDate="2020-04-14T06:15:23.090" Score="4" ViewCount="292" Body="&lt;p&gt;While reading about reinforcement learning, I have come across the following expression for expected rewards in terms of a summation, the denominator of which I am not able to account for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula given is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Mc7Te.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Mc7Te.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to what I understand, the formula should have been correct without the denominator (that I have highlighted). How is this formula correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source of image: Andrew G and Sutton's book on RL.&lt;/p&gt;&#xA;" OwnerUserId="35926" LastEditorUserId="2444" LastEditDate="2020-04-14T12:38:55.863" LastActivityDate="2020-04-14T12:38:55.863" Title="Formula for expected rewards for state–action–next-state triples as a three-argument function" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20245" PostTypeId="2" ParentId="20244" CreationDate="2020-04-14T06:51:51.977" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{p(s', r \mid s, a)}{p(s' \mid s, a)}$&lt;/span&gt; represents the probability of observing reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, given that state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the next state transitioned to. The equation assumes a probability distribution of rewards &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; over state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, meaning that a different reward might be observed whenever a state transitions from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. In most cases, if &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt; is a deterministic reward then &lt;span class=&quot;math-container&quot;&gt;$p(s', r \mid s, a) = p(s' \mid s,a )$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-04-14T12:32:40.863" LastActivityDate="2020-04-14T12:32:40.863" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20247" PostTypeId="2" ParentId="20241" CreationDate="2020-04-14T09:33:54.843" Score="4" Body="&lt;p&gt;I will interpret the questions as being about &lt;a href=&quot;https://en.wikipedia.org/wiki/Triage&quot; rel=&quot;nofollow noreferrer&quot;&gt;triage&lt;/a&gt;. This is particularly important in crisis situations, where a lot of such life-or-death decisions have to be taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://www.medicinenet.com/medical_triage_code_tags_and_triage_terminology/views.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;START system&lt;/a&gt; there are four different categories:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the deceased, who are beyond help&lt;/li&gt;&#xA;&lt;li&gt;the injured who could be helped by immediate transportation&lt;/li&gt;&#xA;&lt;li&gt;the injured with less severe injuries whose transport can be delayed&lt;/li&gt;&#xA;&lt;li&gt;those with minor injuries not requiring urgent care.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;According to the category assigned to a patient, you then decide what to do. Other systems might be more fine-grained, but the principle is the same: effectively the human decision is in the classification, which then guides the assignment of resources. In the above list, the second category would probably be the highest priority for treatment. But once the category has been decided, the course of action has been determined, though the actual treatment options will then be considered for each case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ethics are thus in the judgment of the survival chances: if a nurse decides patient X is too far gone to warrant treatment, that's it. So this is the hard part, the life-or-death decision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are then two aspects to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The accuracy of the diagnosis&lt;/li&gt;&#xA;&lt;li&gt;The predicted likelihood of treatment being effective&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The diagnosis should be (I'm not a medic!) fairly neutral. There are of course mis-diagnoses, but there is no value judgment involved. There might be some 'leakage' between step 1 and step 2, in that a human being might be influenced by the prospects of treatment of a diagnosis when deciding what the problem is. As in: this is a nice person, I don't want them to die, so I (subconsciously) exclude the diagnosis X which would mean certain death.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case a computer system that had sufficient accuracy when making a diagnosis would IMHO actually be more ethical than a human being. Provided, of course, that there is no bias in the diagnosis. Which I think is a practical problem, but in theory could be dealt with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the diagnosis has been determined, the estimation of treatment success is the next decision. This also can be subject to human error, and a computer system with access to statistical information (again, unbiased) of issues and their likelihood of survival could be more independent of other factors which a human being might use, but which are unrelated to the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically, I would say that a computer system could be &lt;em&gt;more ethical&lt;/em&gt; than a human being in such a situation. Because you can defend the way it reached it's decision, and it has been without taking into account factors not related to the problem at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's not always that easy. There are plenty of cases where other factors influence the decision. As long as there are fixed rules, that might not be a problem, though. Some issues would be (in each case the patients would have the same diagnosis and projected survival chance):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One ventilator, but an 8-year-old child and a 85-year-old man&lt;/li&gt;&#xA;&lt;li&gt;A 30-year-old pregnant woman and the Prime Minister of the country&lt;/li&gt;&#xA;&lt;li&gt;A one-month-old baby and a 25-year-old student&lt;/li&gt;&#xA;&lt;li&gt;A homeless person and a billionaire&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As a given, the system would have to be agnostic of general features such as gender, race, religion, etc. But how do you take into account age, social status, and a whole host of other factors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is no difference in the situation, the fairest way would be to toss a coin. That would surely upset a lot of people (&quot;How can you justify leaving our president to die in this time of crisis!&quot;), but if you had explicit rules (&quot;If one patient is in a higher tax bracket, they get priority&quot;) you might upset even more people. The advantage of having such rules would be to &lt;em&gt;make the bias explicit&lt;/em&gt;, and in a way protect a nurse or paramedic from having to decide between a rich senator and a homeless person &amp;mdash; whose family is more likely to go after you if you decided against them? And if you have explicit, unambiguous rules, why not use them to guide an AI system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every human being has their own preferences, and I am glad that I don't have to make those kinds of decisions &amp;mdash; it sounds like a horrific task to me. How can you even sleep in peace after basically having condemned one each of the above cases to death?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is another factor about the ethics of AI: by relieving humans from having to make such decisions, it would also have a beneficial effect. If the final decision is the same that the human would come to, then it's a win/win situation. But that is probably unlikely, due to a whole range of subconscious biases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The prime issue to me seems the lack of recourse in the case of &quot;computer said no&quot;. When a human makes a decision (like a referee in a game), there will always be an argument if people are unhappy with it. But in this case there could be none. The oracle has spoken, your father will be left to die. Sorry, no other outcome possible. It would probably be the same with a human decision, but it wouldn't feel as 'cold': you can observe that the person making the decision has to struggle with it. And you might understand that it was not an easy choice. With a computer, that element is missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, &lt;strong&gt;to summarise&lt;/strong&gt;: given various caveats,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sufficiently accurate and unbiased diagnosis,&lt;/li&gt;&#xA;&lt;li&gt;unbiased prediction of treatment outcome,&lt;/li&gt;&#xA;&lt;li&gt;transparent handling of non-medical factors,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would say that an AI system would me &lt;em&gt;more ethical&lt;/em&gt;, as it has a principled way of reaching a decision which does not disadvantage any group of patients and would always get the same decision for the same patient; furthermore it takes a heavy burden off triage staff who otherwise have to make such decisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which does not mean that I would be happy with my loved-ones' survival being decided by such a system :)&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-14T09:33:54.843" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20248" PostTypeId="2" ParentId="20244" CreationDate="2020-04-14T09:45:25.667" Score="6" Body="&lt;p&gt;Expectation of reward after taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and ending up in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; would simply be &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;r(s, a, s') = \sum_{r \in R} r \cdot p(r|s, a, s')&#xA;\end{equation}&lt;/span&gt;&#xA;The problem with this is that they do not define probability distribution for rewards separately, they use joint distribution &lt;span class=&quot;math-container&quot;&gt;$p(s', r|s, a)$&lt;/span&gt; which represents probability for ending up in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; with reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; after taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. This probability can be separated in 2 parts using product rule&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;p(s', r|s, a) = p(s'|s, a)\cdot p(r|s', s, a)&#xA;\end{equation}&lt;/span&gt;&#xA;which represent probability for getting to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; and then probability for getting reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; after ending up in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;br&gt;&#xA;If we define reward expectation through joint distribution we would have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;r(s, a, s') &amp;amp;= \sum_{r \in R} r \cdot p(s', r|s, a)\\&#xA;&amp;amp;= \sum_{r \in R} r \cdot p(s'|s, a) \cdot p(r|s', s, a)&#xA;\end{align}&lt;/span&gt;&#xA;but this would not be correct since we have this extra &lt;span class=&quot;math-container&quot;&gt;$p(s'|s, a)$&lt;/span&gt; so we divide everything by it to get expression with only &lt;span class=&quot;math-container&quot;&gt;$p(r|s', s, a)$&lt;/span&gt;. So, in the end we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;r(s, a, s') = \sum_{r \in R} r \frac{p(r, s'|s, a)}{p(s'|s, a)}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-04-14T09:45:25.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20249" PostTypeId="1" CreationDate="2020-04-14T10:28:24.023" Score="2" ViewCount="44" Body="&lt;p&gt;I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the &lt;a href=&quot;https://www.tensorflow.org/tutorials/structured_data/imbalanced_data#calculate_class_weights&quot; rel=&quot;nofollow noreferrer&quot;&gt;advice of the tensorflow docs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and now I am using a weighted cross entropy loss where the weights are calculated as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;weight_for_class_a = (1 / samples_for_class_a) * total_number_of_samples/number_of_classes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;following the mentioned tutorial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works perfectly, but why is there this factor &lt;code&gt;total_number_of_samples/number_of_classes&lt;/code&gt;?&#xA;The mentioned tutorial says this&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] helps keep the loss to a similar magnitude.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But I don not understand why. Can someone clarify?&lt;/p&gt;&#xA;" OwnerUserId="36032" LastActivityDate="2021-05-09T16:02:55.370" Title="How are weights for weighted x-entropy loss on imbalanced data calculated?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;datasets&gt;&lt;weights&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20252" PostTypeId="1" CreationDate="2020-04-14T11:52:33.677" Score="2" ViewCount="31" Body="&lt;p&gt;I am performing a regression task on sparse images. The images are a result of a physical process with meaningful parameters (actually, they are a superposition of cone-like shapes), and I am trying to train a regressor for these parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, sparse images mean that the data, and thus the expected output, is made of 2D square tensors, with only one channel, and that it is expectd that roughly 90% of the image is equal to zero. However, in my system, the data is represented as dense tensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I built a neural network with an encoder mapping the image on an output for which I have chosen activation and shape such that it corresponds with those meaningful parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then use custom layers to build an image from these parameters in a way that matches closely the physical process, and train the network by using the L2 distance between the input image and the output image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for a large set of parameters, the output image will be equal to zero, since these are sparse images. This is the case in general for the initial network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible that, through training, the neural network will learn its way out of this all-zero parameterization ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition is that, in the beginning, the loss will be equal to the L2 norm of the input image, and the gradient will be uniformly zero, hence, no learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone confirm ?&lt;/p&gt;&#xA;" OwnerUserId="36040" LastEditorUserId="36040" LastEditDate="2020-04-14T12:58:20.010" LastActivityDate="2020-04-14T12:58:20.010" Title="Can a neural network whose output is uniformly equal to zero learn its way out of it?" Tags="&lt;neural-networks&gt;&lt;image-processing&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20253" PostTypeId="2" ParentId="20211" CreationDate="2020-04-14T12:52:52.343" Score="4" Body="&lt;p&gt;Dueling-DQN has different network architecture comparing to vanilla DQN, so I don't think your version will work as well as the Dueling architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WKrmS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WKrmS.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From &lt;a href=&quot;https://arxiv.org/pdf/1511.06581&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wang et al., 2016&lt;/a&gt;, Dueling Network Architectures for Deep Reinforcement Learning&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On the other hand, since we only have the target Q-value, separating the Q-value into state value and advantage result in the identifiability issue. That is the network might simply learn &lt;span class=&quot;math-container&quot;&gt;$V(s)=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A(s,a)=Q(s,a)$&lt;/span&gt; for every state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To tackle with this issue, we should force an additional constraint on the advantage estimate. We can simply use the equation below as mentioned in the paper, that is, normalize the advantages across actions before combining with the state value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a;\theta,\alpha,\beta)=V(s;\theta,\beta)+(A(s,a;\theta,\alpha)-\frac{1}{|A|}\sum\limits_{a'}A(s,a';\theta,\alpha))$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32173" LastEditorUserId="32173" LastEditDate="2020-04-14T12:59:10.343" LastActivityDate="2020-04-14T12:59:10.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20254" PostTypeId="1" CreationDate="2020-04-14T12:56:50.397" Score="2" ViewCount="876" Body="&lt;p&gt;Could machine learning be used to measure the distance between two objects from a picture or live camera? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this is the measurement between the centre of each eye pupil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This area is all new to me, so any advice and suggestions would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="36041" LastEditorUserId="2444" LastEditDate="2020-04-14T13:40:18.973" LastActivityDate="2020-04-22T14:54:24.717" Title="Could machine learning be used to measure the distance between two objects from a picture or live camera?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;applications&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20255" PostTypeId="2" ParentId="20249" CreationDate="2020-04-14T13:10:03.737" Score="0" Body="&lt;p&gt;This comes from the fact that you want the same magnitude from the loss. Think of it this way: a non-weighted loss function actually has all its weights to 1 and so over the whole data set, samples are weighted with 1 and the sum of all weights is therefore &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, if &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the total number of samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in the case of a weighted loss, we want the weights to also sum to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; so that the loss's magnitude is comparable (&lt;span class=&quot;math-container&quot;&gt;$i = 1..C$&lt;/span&gt; are your classes, &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; is the number of samples for class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S = \sum_{i=0\ }^{C} \sum_{s_i=1}^{N_i} w_{i} = \sum_{i=0}^{i}\sum_{s_i=1}^{N_i}\frac{1}{N_i} \frac{N}{C} = \frac{N}{C} \sum_{i=0}^{C}\sum_{s_i=1}^{N_i}\frac{1}{N_i} = \frac{N}{C} \sum_{i=0}^{C}N_i\frac{1}{N_i}  = \frac{N}{C} \sum_{i=0}^{C}1 = \frac{N}{C} C = N$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="11351" LastActivityDate="2020-04-14T13:10:03.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20256" PostTypeId="2" ParentId="20254" CreationDate="2020-04-14T13:19:13.827" Score="2" Body="&lt;p&gt;The short answer is: yes, it could. In what you are describing, there's nothing very new or specific conceptually; it sounds like a standard regression task. Now the problem that you're actually facing is: &lt;strong&gt;do you have the data?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms won't be able to learn the distance between eyes if you don't have the data that it takes. It could be supervised labels (1 distance per image which would be your regression target), reconstruction from depth maps, multi-view estimation etc. There's a number of ways you could do that &lt;em&gt;given the appropriate data&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;People focus on algorithms a lot, and that's good. But taking a good look at your data is often as important (if not more). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now a good example would be in the self-driving car literature. You could start with this blog-post and go through the papers they reference: &lt;a href=&quot;https://towardsdatascience.com/vehicle-detection-and-distance-estimation-7acde48256e1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/vehicle-detection-and-distance-estimation-7acde48256e1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There also seems to be some litterature about your eyes examples (&lt;a href=&quot;https://arxiv.org/pdf/1806.10890.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1806.10890.pdf&lt;/a&gt;, &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0165027019301578&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0165027019301578&lt;/a&gt;) so skimming through these papers &amp;amp; the datasets they use could guide to towards answering my question: is there data for this task?&lt;/p&gt;&#xA;" OwnerUserId="11351" LastEditorUserId="11351" LastEditDate="2020-04-14T13:25:55.960" LastActivityDate="2020-04-14T13:25:55.960" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20259" PostTypeId="1" AcceptedAnswerId="20297" CreationDate="2020-04-14T15:36:58.257" Score="1" ViewCount="42" Body="&lt;p&gt;Which simulation platform is used by &lt;a href=&quot;https://www.youtube.com/watch?v=Xmw3UwP_vnc&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=pgaEE27nsQw&quot; rel=&quot;nofollow noreferrer&quot;&gt;others&lt;/a&gt; to handle inverse kinematics musculoskeletal simulation, etc., for reinforcement learning simulations and agents?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought they use Unity or Unreal but I assume that would be resource-heavy. &lt;/p&gt;&#xA;" OwnerUserId="36047" LastEditorUserId="2444" LastEditDate="2020-04-14T17:50:53.053" LastActivityDate="2020-04-15T21:17:20.503" Title="Which simulation platform is used by DeepMind (and others) to handle inverse kinematics musculoskeletal?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;deepmind&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20260" PostTypeId="2" ParentId="20241" CreationDate="2020-04-14T15:41:57.990" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/20247/2444&quot;&gt;Oliver's answer&lt;/a&gt; is interesting and it provides valuable information (such as a brief description of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Triage&quot; rel=&quot;nofollow noreferrer&quot;&gt;triage&lt;/a&gt; process, which I was not aware of), but I disagree with his conclusion or, at least, I think it can be misleading because he is implying it's &amp;quot;more ethical&amp;quot; because the AI will behave in &amp;quot;more principled way&amp;quot;. It depends on your definition of &amp;quot;ethical&amp;quot; (and I will recall one below) and the implications of behaving in a &amp;quot;more principled way&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;First of all, we should emphasize that &lt;em&gt;current&lt;/em&gt; AI systems can be and usually are biased because they are mainly trained with data associated with humans and their actions (as pointed out by Gerry in &lt;a href=&quot;https://ai.stackexchange.com/a/20243/2444&quot;&gt;his answer&lt;/a&gt;). Furthermore, currently, AI systems (including the ones for healthcare) are only designed by humans, who can automatically and often inadvertently introduce bias, for example, by choosing the specific AI model over another, the data, how to process or acquire the data. (Maybe, in the future, AI systems will design other AI systems, but can this really reduce bias? Given that humans will probably design the first AI system that is able to design other AI systems, would the bias introduced in this first AI system also be propagated to the other AI systems?)&lt;/p&gt;&#xA;&lt;p&gt;In principle, an AI could make more rational decisions, especially if it is not affected by human limitations (which often are not limitations, such as feelings; e.g., if you didn't feel pain when hitting a chair, you would start bleeding without even not noticing it) that make humans sometimes take irrational actions.&lt;/p&gt;&#xA;&lt;p&gt;However, is the rational action also the most appropriate one? It depends on your definition of rational action and what we mean by &amp;quot;appropriate one&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Here's a definition of &amp;quot;rational&amp;quot; from the dictionary&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;based on or in accordance with reason or logic&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Although the AI system takes actions systematically by following the rules of logic, those actions are still based on some principles or axioms, which will bias the system. So, a rational agent can still be biased, but this bias will be systematic.&lt;/p&gt;&#xA;&lt;p&gt;In general, every decision can potentially be biased because it's based on some principles and taken by a &amp;quot;subject&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Now, let's address your question more directly&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would it be ethical to allow an AI to make life-or-death medical decisions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;First, let me report two definitions of &amp;quot;ethical&amp;quot; from the dictionary&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;relating to moral principles or the branch of knowledge dealing with these:&lt;/p&gt;&#xA;&lt;p&gt;morally good or correct&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The original question can thus be rephrased as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would it be morally good to allow an AI to make life-or-death medical decisions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Of course, it's difficult to argue what is morally good or not, because this is often subjective. It's morally good for me to help my friends, but it isn't necessarily morally good to help other people. We have different friends, so this automatically implies that morally good is subjective.&lt;/p&gt;&#xA;&lt;p&gt;The answer to this question ultimately boils down to the philosophical issue of good vs bad, which is naturally subjective. So, the answer to this question will depend on the philosophical ideas of each person. Some people will say &amp;quot;yes&amp;quot; and some people will say &amp;quot;no&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I think it's more productive to answer the question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are the advantages and disadvantages of allowing an AI to make life-or-death medical decisions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This question can be answered more objectively. For example, we could say that this would free humans from doing this job, which, in certain scenarios (as Oliver points out in his answer), can be &amp;quot;inconvenient&amp;quot;. However, we could also say that current AI systems are still not compatible with human values and they do not think in the way humans do, so they could unexpectedly take &amp;quot;wrong&amp;quot; actions, which can also be difficult to explain (especially if your AI system is or uses a black box system, such as a neural network).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So, should AI systems be used to make life-or-death medical decisions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think that people should decide &amp;quot;democratically&amp;quot;, and there's should be a great majority of acceptance, e.g. not just 51%, but e.g. 95-99% of people should agree with the idea of letting an artificial system to take a life-or-death decision. To take a reasonable vote, people should be aware of the consequences of such a vote, which means that people should be aware of the inner workings of the AI system and what they can or not do (which is often not possible when the AI system is also composed of black-box models, such as neural networks). Alternatively, allowing or not an AI to make such a decision can also be done on a case-by-case basis.&lt;/p&gt;&#xA;&lt;p&gt;All these issues are related to &amp;quot;explainable artificial intelligence&amp;quot;, &amp;quot;accountability&amp;quot;, &amp;quot;transparency&amp;quot;, which have been increasingly debated in the last years.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-14T21:22:57.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20261" PostTypeId="1" CreationDate="2020-04-14T16:02:57.057" Score="2" ViewCount="33" Body="&lt;p&gt;I am working on a classification problem into progressive classes. In other words, there is some hierarchy of categories in such a way, that A &amp;lt; B &amp;lt; C, e.g. low, medium, high, very high. What loss function and activation function for the output layer should I use to take advantage of the class hierarchy, so that true A and predicted C is penalized more than true A and predicted B?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ideas are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) To assign some value to each category, use one output unit with the sigmoid activation and RMS loss function. Then to assign each class to an interval, e.g. 0-033 - class A, 0.33-0.66 class B 0.66-1 - class C. It seem to do the trick, but can favor the extreme categories over the middle ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Use K softmax output units, integer labels instead of one-hot encoded and the sparse categorical crossentropy loss function. In this case I am not sure how exactly sparse categorical crossentropy works and if it really takes into account the hierarchy.&lt;/p&gt;&#xA;" OwnerUserId="36031" LastActivityDate="2020-04-14T16:02:57.057" Title="Single label classification into hierarchical categories using a neural network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;loss-functions&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20262" PostTypeId="2" ParentId="7832" CreationDate="2020-04-14T16:31:07.403" Score="1" Body="&lt;p&gt;I think &lt;a href=&quot;https://ai.stackexchange.com/users/17759/16aghnar&quot;&gt;@16Aghnar&lt;/a&gt; explains the concept quite well. However, by clipping the surrogate objective alone doesn't ensure the trust region as stated in the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://openreview.net/attachment?id=r1etN1rtPB&amp;amp;name=original_pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Engstrom et al., 2020&lt;/a&gt;, Implementation Matters in Deep RL: A Case Study on PPO and TRPO.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The authors inspected OpenAI's implementation of PPO and find many code-level optimizations, I'll list the most important optimizations below:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Clipped surrogate objective&lt;/li&gt;&#xA;&lt;li&gt;Value function clipping&lt;/li&gt;&#xA;&lt;li&gt;Reward scaling&lt;/li&gt;&#xA;&lt;li&gt;Orthogonal initialization and layer scaling&lt;/li&gt;&#xA;&lt;li&gt;Adam learning rate and annealing&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;They find that: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;PPO-M (&lt;code&gt;2.&lt;/code&gt;-&lt;code&gt;5.&lt;/code&gt;) alone can maintain the trust region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(PPO-M: PPO without Clipped surrogate objective, but with code-level optimizations)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;PPO-Clip (&lt;code&gt;1.&lt;/code&gt; only) cannot maintain the trust region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(PPO-Clip: PPO without code-level optimizations, but with Clipped surrogate objective)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TRPO+ has better performance comparing to TRPO, and with similar performance comparing to PPO&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(TRPO+: TRPO with code-level optimizations used in PPO OpenAI implementation)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A intuitive thought on why Clipped surrogate objective alone does not work is: The first step we take is unclipped.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As a result, since we initialize &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (and thus the ratios start all equal to one) the first step we take is identical to a maximization step over the unclipped surrogate reward. Therefore, the size of step we take is determined solely be the steepness of the surrogate landscape (i.e. Lipschitz constant of the  ptimization problem we solve), and we can end up moving arbitrarily far from the trust region. -- &lt;a href=&quot;https://openreview.net/attachment?id=r1etN1rtPB&amp;amp;name=original_pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Engstrom et al., 2020&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="32173" LastActivityDate="2020-04-14T16:31:07.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20263" PostTypeId="1" AcceptedAnswerId="20265" CreationDate="2020-04-14T18:15:05.967" Score="1" ViewCount="112" Body="&lt;p&gt;I am new in the field of Machine Learning so I wanted to start of by reading more about mathematics and history behind it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently reading, in my opinion, a very good and descriptive paper on Statistical Learning Theory - &lt;strong&gt;&lt;em&gt;&lt;a href=&quot;https://arxiv.org/pdf/0810.4752.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Statistical Learning Theory: Models, Concepts, and Results&quot;&lt;/a&gt;&lt;/em&gt;&lt;/strong&gt;. In section 5.5 Generalization bounds, it states that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is sometimes useful to rewrite (17) &quot;the other way round&quot;. That is, instead of ﬁxing &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; and then computing the probability that the empirical risk deviates from the true risk by more than &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, we specify the probability with which we want the bound to hold, and then get a statement which tells us how close we can expect the risk to be to the empirical risk. This can be achieved by setting the right-hand side of (17) equal to some &lt;span class=&quot;math-container&quot;&gt;$\delta &amp;gt; 0$&lt;/span&gt;, and then solving for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;. As a result, we get the statement that with a probability at least &lt;span class=&quot;math-container&quot;&gt;$1−\delta$&lt;/span&gt;, any function &lt;span class=&quot;math-container&quot;&gt;$f \in F$&lt;/span&gt; satisﬁes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uK2lv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uK2lv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation (17) is VC Symmetrization lemma to which we applied union bound and then Chernoff bound:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d3OnF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d3OnF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I fail to understand is the part where we are rewriting (17) &quot;the other way around&quot;. I fail to grasp intuitive understanding of relation between (17) and (18), as well as understanding generalization bounds in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone help me with understanding these concepts or at least provide me with additional resources (papers, blog posts, etc.) that can help?&lt;/p&gt;&#xA;" OwnerUserId="35990" LastEditorUserId="2444" LastEditDate="2020-04-14T20:09:19.043" LastActivityDate="2020-04-14T23:31:16.180" Title="Understanding relation between VC Symmetrization Lemma and Generalization Bounds" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20264" PostTypeId="1" AcceptedAnswerId="20269" CreationDate="2020-04-14T18:53:10.463" Score="1" ViewCount="297" Body="&lt;p&gt;I've built a neural network from the scratch, choosing arbitrary numbers for the hyperparameters: learning rate, number of hidden layers and neurons for these, number of epochs and size of mini batches. Now that I've been able to build something potentially useful (~93% of accuracy with test data, unseen by the model before), I want to focus on hyperparameter tuning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conceptual difference between training and validation sets is clear and makes a lot of sense. It's obvious that the model is biased towards the training set, so it wouldn't make sense to use it to tune the hyperparameters, nor for evaluating its performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, how can I use the validation set for this, if changing any of the parameters enforces me to rebuild a new model again? The final prediction depends on the values of X number of MxN matrices (weights) and X number of N vectors (biases), whose values depend on the learning rate, batch size and number of epochs; and whose dimensions depends on the number and size of hidden layers. If I change any of these, I'd need to rebuild my model again. So I'd be using this validation set for training different models, ending up as in the first step: fitting a model from the scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up: I fall in a recursive problem in which I need to fine tune the hyperparameters of my model with unseen data, but changing any of these hyperparameters implies rebuilding the model.&lt;/p&gt;&#xA;" OwnerUserId="35806" LastActivityDate="2020-04-15T05:25:38.973" Title="How is a validation set used to tune the hyperparameters in a non-biased way, if the new models depends on the values of these?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20265" PostTypeId="2" ParentId="20263" CreationDate="2020-04-14T18:59:38.600" Score="2" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt; in (17) is equal to &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\frac{4}{n}\left(\log{(2\mathsf{N}(\mathcal{F},n))}-\log{\delta}\right)}$&lt;/span&gt;. We have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P\left(\sup_{f\in\mathcal{F}}|R(f)-R_{emp}(f)| &amp;gt; \sqrt{\frac{4}{n}\left(\log{(2\mathcal{N}(\mathcal{F},n))}-\log{\delta}\right)}\right) \leqslant &#xA;2\mathcal{N}(\mathcal{F},n) e^{\frac{-n}{4}\left(\frac{4}{n}\left(\log{(2\mathcal{N}(\mathcal{F},n))}-\log{\delta}\right)\right)} = &#xA;2\mathcal{N}(\mathcal{F},n) e^{\log{\delta} - \log{(2\mathcal{N}(\mathcal{F},n))}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we know that &lt;span class=&quot;math-container&quot;&gt;$e^{\log{n}} = n$&lt;/span&gt; (suppose that base of &lt;span class=&quot;math-container&quot;&gt;$\log$&lt;/span&gt; here is &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;), we can write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P\left(\sup_{f\in\mathcal{F}}|R(f)-R_{emp}(f)| &amp;gt; \sqrt{\frac{4}{n}\left(\log{(2\mathcal{N}(\mathcal{F},n))}-\log{\delta}\right)}\right) \leqslant &#xA;2\mathcal{N}(\mathcal{F},n) \left(\frac{\delta}{2\mathcal{N}(\mathcal{F},n)}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P\left(\sup_{f\in\mathcal{F}}|R(f)-R_{emp}(f)| &amp;gt; \sqrt{\frac{4}{n}\left(\log{(2\mathcal{N}(\mathcal{F},n))}-\log{\delta}\right)}\right) \leqslant \delta&#xA;$$&lt;/span&gt;&#xA;As we know that if we have &lt;span class=&quot;math-container&quot;&gt;$P(x &amp;gt; a) \leqslant c$&lt;/span&gt;, we can conlude that &lt;span class=&quot;math-container&quot;&gt;$P( x&amp;lt;a) \geqslant 1-c$&lt;/span&gt;, we will have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P\left(\sup_{f\in\mathcal{F}}|R(f)-R_{emp}(f)| &amp;lt; \sqrt{\frac{4}{n}\left(\log{(2\mathcal{N}(\mathcal{F},n))}-\log{\delta}\right)}\right) \geqslant 1- \delta&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as this inequality is true for supremum of set &lt;span class=&quot;math-container&quot;&gt;$\mathcal{F}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R(f) -R_{emp}(f) \leqslant \varepsilon$&lt;/span&gt; is subset of  &lt;span class=&quot;math-container&quot;&gt;$|R(f) -R_{emp}(f)| \leqslant \varepsilon$&lt;/span&gt; (in terms of probability state space), we can say the following inequality is correct for any function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with at least probability of &lt;span class=&quot;math-container&quot;&gt;$1-\delta$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R(f) \leqslant R_{emp}(f) + \sqrt{\frac{4}{n}\left(\log{(2\mathcal{N}(\mathcal{F},n))}-\log{\delta}\right)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-04-14T23:31:16.180" LastActivityDate="2020-04-14T23:31:16.180" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20266" PostTypeId="1" AcceptedAnswerId="20338" CreationDate="2020-04-14T19:24:40.270" Score="4" ViewCount="51" Body="&lt;p&gt;I am trying to create a multiclass product-rating network based on product reviews and other input features. Two of the other input features are &quot;product category&quot; and &quot;gender&quot;. However, I want to avoid unfair bias in the classification task between male/female. Since some product categories are more likely to be reviewed by males or females (hence, not balanced), I am seeking for an approach to solve this &quot;imbalance&quot;-like issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The options and things that I consider at the moment are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Downsample the training examples in each product category to balance for gender&lt;/li&gt;&#xA;&lt;li&gt;Add weights to the training examples for gender, or&lt;/li&gt;&#xA;&lt;li&gt;Add weights to the loss function (either log-likelihood or cross-entropy)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Even though downsampling might be the easiest option, I would like to explore the options of adding weights in the network in some way. However, most literature are only discussing adding weights to the loss function in order to solve for imbalanced data related to the target value (which is not the issue that I am addressing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me or point me in the right direction to solve this challenge?&lt;/p&gt;&#xA;" OwnerUserId="36053" LastActivityDate="2020-04-16T23:10:49.623" Title="How to add weights to one specific input feature to ensure fair training in the network?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;ethics&gt;&lt;algorithmic-bias&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20267" PostTypeId="1" AcceptedAnswerId="20270" CreationDate="2020-04-14T20:40:30.527" Score="1" ViewCount="114" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rzoYV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rzoYV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent is trying to master the &lt;a href=&quot;https://gym.openai.com/envs/Breakout-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atari Breakout game&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my &lt;a href=&quot;https://colab.research.google.com/drive/1mXUkkq_CMm0CN_dUbjzuclAzVqjJxZIp&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that normal that &lt;code&gt;reward_100&lt;/code&gt; decreased that much after it hits 4.5? Is there a way to avoid that behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be aware that &lt;code&gt;reward_100&lt;/code&gt; is simply &lt;code&gt;mean_reward = np.mean(self.total_rewards[-100:])&lt;/code&gt;. In other words, it is the mean over the last 100 rewards. On the graph, &lt;code&gt;reward_100&lt;/code&gt; represents de y-axis and th number of episodes the x-axis. &lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="2444" LastEditDate="2020-04-14T21:02:07.173" LastActivityDate="2020-04-14T22:45:01.710" Title="Why are the rewards of my RL agent for the Atari Breakout game decreasing after a certain number of episodes?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20268" PostTypeId="2" ParentId="20241" CreationDate="2020-04-14T22:00:57.423" Score="3" Body="&lt;p&gt;At face value, this sound monstrous--a measure to offload responsibility to a non-conscious mechanism that cannot be meaningfully punished for mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I will argue:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There is humane benefit in taking this decision out of the hands of doctors re: the psychological toll&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Specifically, doctors are not the reason for resource scarcity, yet they're the ones being forced to make scarcity-driven life-of-death decisions, and that has got to take a toll.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, unless one is a sociopath, there is going to be an emotional effect.  Here the &quot;sociopathy&quot; of a pure algorithm relieves humans of this terrible burden.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Might even reduce burnout, and keep more doctors working longer and with more focus.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-04-14T22:06:27.707" LastActivityDate="2020-04-14T22:06:27.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20269" PostTypeId="2" ParentId="20264" CreationDate="2020-04-14T22:18:16.697" Score="1" Body="&lt;p&gt;This is a standard ML problem: changing hyper-parameters changes the performance of the whole model. Ideally, you'd be cross-validating hyper-parameter choices, not merely comparing on a static validation set. That being said you need to be careful with hyper-parameter optimization because you could overfit these to the peculiarities of your validation set ; cross-validation helps to some extent but really what helps is having a test set that you hardly ever test against. Ideally never before you've chosen your HPs with (cross-) validation. And test-set performance will indicate how much your  HP-optimization procedure was biased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm afraid training from scratch is your only solution. This does not however mean that you have to train until the end and many hyper-parameter optimization techniques out there will help you stop training early enough so you don't waste computational resources on HPs which are not worth it. A good starting point would be this blog post by Criteo's Aloïs Bissuel: &lt;a href=&quot;https://medium.com/criteo-labs/hyper-parameter-optimization-algorithms-2fe447525903&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyper-parameter optimization algorithms: a short review&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="11351" LastActivityDate="2020-04-14T22:18:16.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20270" PostTypeId="2" ParentId="20267" CreationDate="2020-04-14T22:45:01.710" Score="1" Body="&lt;p&gt;It seems that decaying the learning rate solved my problem. I changed &lt;code&gt;learning_rate&lt;/code&gt; from &lt;code&gt;0.001&lt;/code&gt; to &lt;code&gt;0.0001&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fWRJF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fWRJF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35626" LastActivityDate="2020-04-14T22:45:01.710" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20272" PostTypeId="1" CreationDate="2020-04-14T23:46:40.093" Score="2" ViewCount="160" Body="&lt;p&gt;After reading some literature on reinforcement learning (RL), it seems that stochastic approximation theory underlies all of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a lot of substantial and difficult theory in this area requiring measure theory leading to martingales and stochastic approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard RL texts at best mention the relevant theorem and then move on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the field of RL is really stochastic approximation theory in disguise? Is RL just a less rigorous version of stochastic approximation theory?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-04-15T02:52:10.927" LastActivityDate="2020-05-22T16:51:16.617" Title="Is RL just a less rigorous version of stochastic approximation theory?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20273" PostTypeId="2" ParentId="20272" CreationDate="2020-04-15T03:34:16.827" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the field of RL is really stochastic approximation theory in disguise? Is RL just a less rigorous version of stochastic approximation theory?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, but reinforcement learning (RL) is based on stochastic approximation theory (SAT), and these two fields overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In RL, you typically assume that the underlying problem can be modeled as a Markov decision process (MDP), and the goal is to find a policy (or value function) that solves this MDP. To find this policy, you can use stochastic approximation algorithms, such as Q-learning, but RL isn't just SAT, where, in general, there isn't necessarily a notion of MDP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;SAT is the study of iterative algorithms to find the extrema of functions by sampling from them and under which conditions these iterative algorithms converge. SAT isn't just applied in RL, but it is applied in many other fields, such as deep learning. The paper &lt;a href=&quot;https://faculty.chicagobooth.edu/Panagiotis.Toulis/research/papers/ptoulis_stco.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scalable estimation strategies based on stochastic approximations: Classical results and new insights&lt;/a&gt; (2015) by P.  Toulis et al. provides an overview of SAT and the connections with other fields (including RL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, RL is based on SAT, but RL isn't just stochastic approximation algorithms, so they are distinct fields. If you want to study e.g. the convergence properties of certain RL algorithms, you may need to study SAT. In fact, for example, &lt;a href=&quot;http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the typical proof of convergence for tabular Q-learning assumes the Robbins–Monro conditions&lt;/a&gt;. However, you can do a lot of RL without even knowing that RL is based on SAT. Similarly, you can do a lot of SAT without ever caring about RL.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-22T16:51:16.617" LastActivityDate="2020-05-22T16:51:16.617" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20275" PostTypeId="2" ParentId="20264" CreationDate="2020-04-15T05:25:38.973" Score="1" Body="&lt;p&gt;The importance of having a totally separate test set is very crucial. Once you start to use the validation set performance as a measure to use to tune hyper parameters you are biasing your network to work well on the validation set so it can no longer be relied on as a true measure of performance. Eventually if you use your test set too often then adjust hyper parameters to improve performance on the test set you wind up in the same boat. I have actually used several test sets to try to avoid this trap.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-15T05:25:38.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20276" PostTypeId="1" CreationDate="2020-04-15T06:53:04.230" Score="2" ViewCount="66" Body="&lt;p&gt;I have an environment, in which my agent learns according to PPO. The environment has a maximum of 80 actions, however not all of them are always allowed. My idea was to mask them, by setting the probabilities of the non valid actions to 0, and renormalizing the remaining actions. However, this would be not the predicted policy anymore, and thus the agent wouldn't act on policy. Is there a better way to mask PPO agents actions, or does it simply not constitute a big problem?&lt;/p&gt;&#xA;" OwnerUserId="31821" LastActivityDate="2020-04-15T06:53:04.230" Title="Action masking for on policy algorithm like PPO" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;on-policy-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20277" PostTypeId="1" AcceptedAnswerId="20279" CreationDate="2020-04-15T09:52:52.627" Score="1" ViewCount="49" Body="&lt;p&gt;My thinking is you input a paragraph, or sentence, and the program can boil it down to the primary concept(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input:&lt;/strong&gt; &#xA;Sure, it would be nice if morality was simply a navigation toward greater states of conscious well-being, and diminishing states of suffering, but aren't there other things to value independent of well-being? Like truth, or beauty?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&#xA;Questioning moral philosophy.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Is there any group who's doing this already? If not, why not?&lt;/p&gt;&#xA;" OwnerUserId="36067" LastActivityDate="2020-05-15T12:04:15.390" Title="How would you build an AI to output the primary concept of a paragraph?" Tags="&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;ai-basics&gt;&lt;text-summarization&gt;&lt;natural-language-understanding&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20278" PostTypeId="1" CreationDate="2020-04-15T10:45:37.003" Score="1" ViewCount="38" Body="&lt;p&gt;Using this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import gym&#xA;import numpy as np&#xA;import time&#xA;&#xA;&quot;&quot;&quot;&#xA;SARSA on policy learning python implementation.&#xA;This is a python implementation of the SARSA algorithm in the Sutton and Barto's book on&#xA;RL. It's called SARSA because - (state, action, reward, state, action). The only difference&#xA;between SARSA and Qlearning is that SARSA takes the next action based on the current policy&#xA;while qlearning takes the action with maximum utility of next state.&#xA;Using the simplest gym environment for brevity: https://gym.openai.com/envs/FrozenLake-v0/&#xA;&quot;&quot;&quot;&#xA;&#xA;def init_q(s, a, type=&quot;ones&quot;):&#xA;    &quot;&quot;&quot;&#xA;    @param s the number of states&#xA;    @param a the number of actions&#xA;    @param type random, ones or zeros for the initialization&#xA;    &quot;&quot;&quot;&#xA;    if type == &quot;ones&quot;:&#xA;        return np.ones((s, a))&#xA;    elif type == &quot;random&quot;:&#xA;        return np.random.random((s, a))&#xA;    elif type == &quot;zeros&quot;:&#xA;        return np.zeros((s, a))&#xA;&#xA;&#xA;def epsilon_greedy(Q, epsilon, n_actions, s, train=False):&#xA;    &quot;&quot;&quot;&#xA;    @param Q Q values state x action -&amp;gt; value&#xA;    @param epsilon for exploration&#xA;    @param s number of states&#xA;    @param train if true then no random actions selected&#xA;    &quot;&quot;&quot;&#xA;    if train or np.random.rand() &amp;lt; epsilon:&#xA;        action = np.argmax(Q[s, :])&#xA;    else:&#xA;        action = np.random.randint(0, n_actions)&#xA;    return action&#xA;&#xA;def sarsa(alpha, gamma, epsilon, episodes, max_steps, n_tests, render = True, test=False):&#xA;    &quot;&quot;&quot;&#xA;    @param alpha learning rate&#xA;    @param gamma decay factor&#xA;    @param epsilon for exploration&#xA;    @param max_steps for max step in each episode&#xA;    @param n_tests number of test episodes&#xA;    &quot;&quot;&quot;&#xA;    env = gym.make('Taxi-v3')&#xA;    n_states, n_actions = env.observation_space.n, env.action_space.n&#xA;    Q = init_q(n_states, n_actions, type=&quot;ones&quot;)&#xA;    print('Q shape:' , Q.shape)&#xA;&#xA;    timestep_reward = []&#xA;    for episode in range(episodes):&#xA;        print(f&quot;Episode: {episode}&quot;)&#xA;        total_reward = 0&#xA;        s = env.reset()&#xA;        print('s:' , s)&#xA;        a = epsilon_greedy(Q, epsilon, n_actions, s)&#xA;        t = 0&#xA;        done = False&#xA;        while t &amp;lt; max_steps:&#xA;            if render:&#xA;                env.render()&#xA;            t += 1&#xA;            s_, reward, done, info = env.step(a)&#xA;            total_reward += reward&#xA;            a_ = epsilon_greedy(Q, epsilon, n_actions, s_)&#xA;            if done:&#xA;                Q[s, a] += alpha * ( reward  - Q[s, a] )&#xA;            else:&#xA;                Q[s, a] += alpha * ( reward + (gamma * Q[s_, a_] ) - Q[s, a] )&#xA;            s, a = s_, a_&#xA;            if done:&#xA;                if render:&#xA;                    print(f&quot;This episode took {t} timesteps and reward {total_reward}&quot;)&#xA;                timestep_reward.append(total_reward)&#xA;                break&#xA;#             print('Updated Q values:' , Q)&#xA;    if render:&#xA;        print(f&quot;Here are the Q values:\n{Q}\nTesting now:&quot;)&#xA;    if test:&#xA;        test_agent(Q, env, n_tests, n_actions)&#xA;    return timestep_reward&#xA;&#xA;def test_agent(Q, env, n_tests, n_actions, delay=0.1):&#xA;    for test in range(n_tests):&#xA;        print(f&quot;Test #{test}&quot;)&#xA;        s = env.reset()&#xA;        done = False&#xA;        epsilon = 0&#xA;        total_reward = 0&#xA;        while True:&#xA;            time.sleep(delay)&#xA;            env.render()&#xA;            a = epsilon_greedy(Q, epsilon, n_actions, s, train=True)&#xA;            print(f&quot;Chose action {a} for state {s}&quot;)&#xA;            s, reward, done, info = env.step(a)&#xA;            total_reward += reward&#xA;            if done:  &#xA;                print(f&quot;Episode reward: {total_reward}&quot;)&#xA;                time.sleep(1)&#xA;                break&#xA;&#xA;&#xA;if __name__ ==&quot;__main__&quot;:&#xA;    alpha = 0.4&#xA;    gamma = 0.999&#xA;    epsilon = 0.9&#xA;    episodes = 200&#xA;    max_steps = 20&#xA;    n_tests = 20&#xA;    timestep_reward = sarsa(alpha, gamma, epsilon, episodes, max_steps, n_tests)&#xA;    print(timestep_reward)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;from : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-temporal-difference-sarsa-q-learning-expected-sarsa-on-python-9fecfda7467e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/reinforcement-learning-temporal-difference-sarsa-q-learning-expected-sarsa-on-python-9fecfda7467e&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sample Q table generated is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 1.          1.          1.          1.          1.          1.        ]&#xA; [ 0.5996      0.5996      0.5996      0.35936     0.5996      1.        ]&#xA; [ 0.19936016  0.35936     0.10336026  0.35936     0.35936    -5.56063984]&#xA; ...&#xA; [ 0.35936     0.5996      0.35936     0.5996      1.          1.        ]&#xA; [ 1.          0.5996      1.          1.          1.          1.        ]&#xA; [ 0.35936     0.5996      1.          1.          1.          1.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The columns representing the actions and rows representing the corresponding states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the state be represented by a vector? The Q table cells are not contained by vectors of size &gt; 1, so how should these states be represented? For example, if I'm in the state [2], can this be represented as an n-dimensional vector?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put another way, if &lt;code&gt;Q[1,3] = 4&lt;/code&gt;, can the Q state 1 with action 3 be represented as a vector &lt;code&gt;[1, 3, 2, 12, 3]&lt;/code&gt;? If so, then is the &lt;code&gt;state_number -&amp;gt; state_attributes&lt;/code&gt; mapping stored in a separate lookup table?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="2444" LastEditDate="2020-04-15T20:47:09.120" LastActivityDate="2020-04-15T20:47:09.120" Title="How are n-dimensional vectors state vectors represented in Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20279" PostTypeId="2" ParentId="20277" CreationDate="2020-04-15T10:58:00.413" Score="0" Body="&lt;p&gt;Just to elaborate my brief comment on the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Identifying the primary concepts of a paragraph required &lt;em&gt;understanding&lt;/em&gt; of the meaning of the text. In natural language processing we are still a long way off even recognising and representing the meaning of text, let alone summarising the meaning of multiple sentences into a single statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is different from simply summarising a text: this can be done without any understanding based on textual features within the text itself, and ways of doing that have been around for a while. But such approaches will generally remove sentences which seem less relevant to the text, thus shortening it. They will not express the content in different words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceivably people might try this with deep learning, where you train a system with paragraphs and the corresponding concepts, but again such a system would not have any understanding of the meaning, and thus results would be more or less accidental.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-15T10:58:00.413" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20280" PostTypeId="1" CreationDate="2020-04-15T11:21:28.287" Score="1" ViewCount="89" Body="&lt;p&gt;I am trying to figure out how to approach this.&lt;/p&gt;&#xA;&lt;p&gt;Given training data of images and the pixel coordinates of the centre of an object in that image, would it be possible to predict the pixel coordinates of the object in the same &amp;quot;scene&amp;quot; in a different perspective, but with the object removed?&lt;/p&gt;&#xA;" OwnerUserId="36069" LastEditorUserId="2444" LastEditDate="2021-01-16T20:46:36.823" LastActivityDate="2021-01-16T20:46:36.823" Title="Given the coordinates of an object in an image, is it possible to predict the coordinates of the same object in a different perspective?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20281" PostTypeId="2" ParentId="5728" CreationDate="2020-04-15T12:35:07.517" Score="1" Body="&lt;p&gt;First thing to remember is time-complexity is calculated for an algorithm. An algorithm takes an input and produces an output. Now in case of neural networks, your time complexity depends on what you are taking as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 1: Input is just the dataset. Architecture and hyperparameters are fixed in the algorithm.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever we say time complexity of an algorithm we generally mean &quot;number of iterations or recursions the algorithm makes with respect to the input, before coming to halt&quot; (my definition, not a standard one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training process can be divided into three steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;single-pass-n-update the model for a given batch of input&lt;/li&gt;&#xA;&lt;li&gt;Repeat the above step for a different batches of input&lt;/li&gt;&#xA;&lt;li&gt;Repeat the above steps for the given number of epochs &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So if we break the single-pass-n-update process into different parts for a better understanding, it will be as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Forward pass&lt;/li&gt;&#xA;&lt;li&gt;Loss calculation&lt;/li&gt;&#xA;&lt;li&gt;Backward pass and parameter update&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Loss calculation is obviously &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt;. But I think the forward pass will also have a time complexity of &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt;. And so does backward pass. Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see my viewpoint, please recall the definition of time complexity and take an example say sorting. Bubble sort has a time complexity of &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt;. Why? Because if your input size is &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (i.e. n numbers) then you need to do &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; iterations to complete the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if looping and recursion are not available in your programming language or you don't want to use them, then you will need to write around &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; lines of code (LOC) to do the sorting. It is called &lt;em&gt;loop unrolling&lt;/em&gt;. Mind it, your code needs to be rewritten every time for every new input. If Mary wants to sort 5 elements, your code will have 25 LOC. If Bob wants to sort 500 elements, 250000 LOC and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I tell you to write a program that sorts exactly 10 elements, then you don't have the LOC curse. You can simply write 10*10 LOC, and it will work for every input that contains 10 elements! So your time complexity becomes O(100) = O(1). That is the key in my mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our given case we fix the architecture and hyperparameters of the network beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we fix the architecture of a neural network (i.e. the number of hidden layers, the input size, and the output size), we fix the maximum number of iterations to a constant number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a neural network that takes input as 32x32 (=1024) grayscale image, has a hidden layer of size 2048, and output as 10 nodes representing 10 classes (yes classic MNSIT digit recognition task). The batch size is 16.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one single forward pass, first, there will be a matrix multiplication. The two matrices that will be multiplied will be the input matrix and the weight matrix. The (fixed) dimensions will be 32x1024 and 1024x2048, respectively. One can write 32x1024x2048 LOC to do the multiplication. Consequently, matrix values at the hidden layer (32x2048) will be multiplied with another weight matrix (2048x10) to get the final output. This matrix multiplication will also be fixed. Therefore in a single forward pass, a constant number of operations will be performed. Therefore LOC needed are fixed. Hence &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; time complexity. Similarly for backpropagation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key here is to understand that while training a plain vanilla multi-layer neural network, the architecture is fixed and hence the number of operations is fixed. So one single-pass-n-update happens in &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt;. However, this doesn't make the training process itself to run in &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; time. Why? Let us see how training happens:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Algorithm 1&#xA;// assuming model architecture is already defined&#xA;// hyperparamaters are fixed&#xA;// N is number of training examples in the dataset&#xA;// ... denotes other hyperparamters&#xA;&#xA;model train_function (model, train_dataset, batch_size, epochs, ...){&#xA;    batches = train_dataset / batch_size&#xA;    for(int i=0; i&amp;lt;epochs; i++){                             //O(1)&#xA;        for each batch in batches{                           //O(N)&#xA;            model = single-pass-n-update(model, batch, ...)  //O(1)&#xA;        }&#xA;    }&#xA;    return model&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As epochs are also fixed, the epoch loop also runs in &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt;. The batch loop technically runs in &lt;span class=&quot;math-container&quot;&gt;$O(N/batch\_size)$&lt;/span&gt; but as &lt;span class=&quot;math-container&quot;&gt;$batch\_size$&lt;/span&gt; is also fixed we can write it as &lt;span class=&quot;math-container&quot;&gt;$O(N)$&lt;/span&gt;. SO the final time complexity will be &lt;span class=&quot;math-container&quot;&gt;$O(N)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 2: Input is dataset+hyperparameters. But the architecture is fixed.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the running time complexity of the epoch loop will become exponential. &lt;em&gt;So the entire time complexity of training becomes exponential&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model train_function (model, train_dataset, batch_size, epochs, ...){&#xA;    batches = train_dataset / batch_size&#xA;    for(int i=0; i&amp;lt;epochs; i++){                             //O(exp)&#xA;        for each batch in batches{                           //O(N)&#xA;            model = single-pass-n-update(model, batch, ...)  //O(...)&#xA;        }&#xA;    }&#xA;    return model&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you are wondering why epoch loop is running in exponential time and why not in &lt;span class=&quot;math-container&quot;&gt;$O(epochs)$&lt;/span&gt;, then the answer would be that some simple algorithms look like polynomial but they are not, they have a &lt;em&gt;pseudo-polynomial running time&lt;/em&gt;. Basically it states that time complexity is not only dependent on input but on the size of input as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Pseudo-Polynomial Running Time&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you use bubble sort, you give the program input as &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; numbers. The program takes each number one by one. Each number is a (say) 8-bit unsigned integer. We know the time complexity is quadratic, &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt;, so changing the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; will change the number of iterations quadratically. Fine till now.&lt;br&gt;But now comes the catch, if you change the input size i.e. 9-bit or 7-bit instead of 8-bit, will the number of iterations change? The answer is No. Why? DIY&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However&lt;br&gt;&#xA;If you want to create a loop that prints &quot;Hello&quot; for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times, you might think the time complexity is &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is an 8-bit integer supplied by the user. Now ask yourself, if you change the input size i.e. 9-bit or 7-bit instead of 8-bit, will the number of iterations change? The answer is Yes. It will change exponentially. Why? DIY&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, when you are calculating the time complexity and the expression contains a pure number provided by the user then the complexity automatically becomes exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know a little about cryptography you might have come across that the Knapsack problem is used in &lt;a href=&quot;https://en.wikipedia.org/wiki/Knapsack_cryptosystems&quot; rel=&quot;nofollow noreferrer&quot;&gt;cryptography&lt;/a&gt;. Cryptography uses functions that are computable in exponential time only. That's what makes cryptography secure. But you might also know that dynamic programming &lt;a href=&quot;https://www.geeksforgeeks.org/0-1-knapsack-problem-dp-10/&quot; rel=&quot;nofollow noreferrer&quot;&gt;solves&lt;/a&gt; the Knapsack problem in &lt;span class=&quot;math-container&quot;&gt;$O(nW)$&lt;/span&gt; time. Then what is the catch? The catch is that dynamic programming solves it in &lt;em&gt;pseudo-polynomial time&lt;/em&gt;. Why? Look at the time complexity expression &lt;span class=&quot;math-container&quot;&gt;$O(nW)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of items and &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is the capacity of the Knapsack which is a pure number provided by the user. Hence the actual time complexity becomes exponential. To read more about it, &lt;a href=&quot;https://cs.stackexchange.com/questions/68938/shouldnt-every-algorithm-run-in-pseudo-polynomial-time&quot;&gt;this&lt;/a&gt; might help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 3: Input is dataset+architecture. But hyperparameters are fixed&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Architecture = [ number of input nodes (&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;), number of hidden layers (&lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;), [number of hidden nodes in each hidden layer &lt;span class=&quot;math-container&quot;&gt;$h_1,h_2,h_3,...,h_h$&lt;/span&gt;], number of output nodes (&lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;)] &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Batch size = constant = &lt;span class=&quot;math-container&quot;&gt;$K_b$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the epoch loop will run in &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; and the batch loop will run in &lt;span class=&quot;math-container&quot;&gt;$O(N)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to single-pass-n-update. Forward pass will happen in &lt;span class=&quot;math-container&quot;&gt;$O(K_bnh_1+K_bh_1h_2+K_bh_2h_3+ ... + K_bh_{h-1}h_h)$&lt;/span&gt; Why? Matrix multiplication&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ignoring the constant term, the expression can be written as&#xA;&lt;span class=&quot;math-container&quot;&gt;$O(nh_1+h_1h_2+h_2h_3+ ... + h_{h-1}h_h)$&lt;/span&gt;, as you can see the expression contains &lt;span class=&quot;math-container&quot;&gt;$h_1,h_2,h_3,...,h_h$&lt;/span&gt; which are pure numbers given by the user. Hence the time complexity in the forward pass becomes exponential. Hence the time complexity of single-pass-n-update itself becomes exponential&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model train_function (model, train_dataset, batch_size, epochs, ...){&#xA;    batches = train_dataset / batch_size&#xA;    for(int i=0; i&amp;lt;epochs; i++){                             //O(1)&#xA;        for each batch in batches{                           //O(N)&#xA;            model = single-pass-n-update(model, batch, ...)  //O(exp)&#xA;        }&#xA;    }&#xA;    return model&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now I hope fourth case is also clear when input is dataset+architecture+hyperparameters. And nothing is fixed&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="36066" LastEditorUserId="36066" LastEditDate="2020-04-15T12:41:07.517" LastActivityDate="2020-04-15T12:41:07.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20282" PostTypeId="1" AcceptedAnswerId="20284" CreationDate="2020-04-15T12:55:00.323" Score="1" ViewCount="107" Body="&lt;p&gt;If the i.i.d. (independent and identically distributed) assumption holds for a training-validation set pair, shouldn't their loss trends be exactly the same, since every batch from the validation set is equivalent to having a batch from the training set instead? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the assumption was to be true wouldn't that make any method that was aware of the fact that there were two separate sets (regularization methods such as early stopping) meaningless?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we work with the fact that there is a certain degree of wrongness to the assumption or am I interpreting it wrongly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S - The question stems from an observation made on MNIST (where I suppose the i.i.d assumption holds strongly). The training and validation trends (losses and accuracy both) on MNIST were almost exactly identical for any network (convolutional and feedforward) trained using negative log-likelihood, making regularization meaningless. &lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="2444" LastEditDate="2020-04-15T15:22:44.273" LastActivityDate="2020-04-16T01:57:54.107" Title="If the i.i.d. assumption holds, shouldn't the training and validation trends be exactly the same?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;regularization&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20283" PostTypeId="1" CreationDate="2020-04-15T13:10:51.310" Score="2" ViewCount="326" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pjJFA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pjJFA.jpg&quot; alt=&quot;Flowchart&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having trouble understanding the 5th step in the flowchart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the 5th step, the '&lt;em&gt;update the Q function by taking the average of returns&lt;/em&gt;' is confusing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, the Q function is basically the state-action pair values put in a table (the Q table). To update it means to make adjustments to the state-action pair value of the individual states and their respective actions (e.g state 1 action 1, state 3 action 1, state 3 action 2, so on and so forth). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what '&lt;em&gt;average of returns&lt;/em&gt;' means though. Is it asking me to take the average of the returns after &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; episodes?  From my understanding, returns is the sum of rewards in a full episode (So, AVG=sum of returns for x episodes/x). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And what do I do with that '&lt;em&gt;average&lt;/em&gt;'? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a little confused when they say '&lt;em&gt;update the Q function&lt;/em&gt;' because the Q function consists of many parameters that must be updated (the individual state-action pair value), and I'm not sure which one they are referring to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the point of calculating the average of returns? Since the state-action pair value for a particular state and particular action will always be the same (e.g if I always take action 3 in state 4, I will always get value=2 forever)&lt;/p&gt;&#xA;" OwnerUserId="36072" LastEditorUserId="2444" LastEditDate="2020-04-15T14:28:14.703" LastActivityDate="2020-05-22T19:12:16.743" Title="How does Monte Carlo Exploring Starts work?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20284" PostTypeId="2" ParentId="20282" CreationDate="2020-04-15T14:03:51.200" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If the i.i.d (independent and identically distributed) assumption holds, shouldn't the training and validation trends be exactly the same?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, not necessarily. Let me explain why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you assume your samples (aka examples, observations, data points, etc.) are i.i.d., this means&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;that they come from the same distribution, e.g. a Gaussian &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(0, 1)$&lt;/span&gt; (the identically distributed part), and &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;that they are independently drawn from it, i.e., intuitively, each sample provides the same kind of information independently of the others&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, even if samples are independently drawn from a certain distribution, they can be different. For example, if you draw a sample &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(0, 1)$&lt;/span&gt;, an operation often denoted as &lt;span class=&quot;math-container&quot;&gt;$x \sim \mathcal{N}(0, 1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; could have the value &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$13$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$50$&lt;/span&gt; (or any other number), so they could be variable, although your samples will tend to be mainly around &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, because that's where your Gaussian puts more density (and your standard deviation is just &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;). If your standard deviation was higher, then there would be even more variability in the sampling process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you assume that your samples are independently drawn from a certain distribution, it doesn't mean that you will always get the same pattern of samples. In other words, you can still have variability in your samples, and this also depends on the distribution you sample from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question more directly, there's a chance that your training data and your validation data don't necessarily have the same patterns, even if the independence assumption holds. Therefore, the training and validation trends (and I assume you mean e.g. the performance) are not necessarily the same, but, although this may also depend on the training method, I would say that they shouldn't be very different (if the assumption holds) because, intuitively, each sample should be as informative as any other sample (independence assumption).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do we work with the fact that there is a certain degree of wrongness to the assumption or am I interpreting it wrongly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is often convenient to make the i.i.d. assumption even, if it doesn't hold, for several reasons: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;your training procedure may converge faster (because, intuitively, each sample will be as informative as any other sample)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;your models may be simpler (e.g., in naive Bayes, you make the i.i.d. assumption only to simplify the model and, in general, the mathematical formulations)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sometimes, if it doesn't hold, your training procedure can be highly affected. In those cases, you can find workarounds and try to make it hold. For example, the usage of the &lt;em&gt;experience replay&lt;/em&gt; in deep Q-learning is an example of a trick used to overcome the dependence of successive samples, which causes learning to be highly variable. See this question &lt;a href=&quot;https://ai.stackexchange.com/q/10839/2444&quot;&gt;Why exactly do neural networks require i.i.d. data?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answers to the question &lt;a href=&quot;https://stats.stackexchange.com/q/213464/821355&quot;&gt;On the importance of the i.i.d. assumption in statistical learning&lt;/a&gt; on CrossValidated provide more information and details, so you may want to have a look at it too. Here's &lt;a href=&quot;https://stats.stackexchange.com/a/445477/82135&quot;&gt;another answer&lt;/a&gt;, which is related to shuffling and how it can or not make the independence assumption hold, that I highly recommend that you read.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-16T01:57:54.107" LastActivityDate="2020-04-16T01:57:54.107" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20285" PostTypeId="1" CreationDate="2020-04-15T14:04:28.983" Score="1" ViewCount="270" Body="&lt;p&gt;Am I right to say that the Q value of a particular state and action is the same as the state-action pair value of that same state and action? &lt;/p&gt;&#xA;" OwnerUserId="36072" LastEditorUserId="2444" LastEditDate="2020-04-15T14:31:57.700" LastActivityDate="2020-04-15T14:33:15.477" Title="Is the Q value the same as the state-action pair value?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;terminology&gt;&lt;value-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20286" PostTypeId="2" ParentId="20285" CreationDate="2020-04-15T14:27:43.487" Score="1" Body="&lt;p&gt;I don't understand your question very clearly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-value of a particular state-action pair (s,a) under policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the total reward you would expect to collect if you start from the state s, take the action a, and follow policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; from then on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the literature, this is referred to as state-action values.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-04-15T14:27:43.487" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20287" PostTypeId="2" ParentId="20285" CreationDate="2020-04-15T14:28:10.400" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I right to say that the Q value of a particular state and action is the same as the state-action pair value of that same state and action?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is used as the symbol for action value and &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; is the action value function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phrase &quot;state-action pair value&quot; is used to mean the same thing in some texts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, yes, you are right. At least I cannot think of or find any counter-examples where the two things could be used to refer to different things in the same text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It &lt;em&gt;is&lt;/em&gt; possible to work with different formulae for Q - e.g. finite horizon, discounted, average reward. There are also conceptual differences between a &quot;true&quot; action value, what you are currently estimating, and implementation details such as how you implement a Q function in code. In addition, there is also the advantage function usually labelled &lt;span class=&quot;math-container&quot;&gt;$A(s,a)$&lt;/span&gt; which &lt;em&gt;might&lt;/em&gt; also be considered as a &quot;state-action pair value&quot; by some authors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, those would not generally be flagged in documents by labelling one a &quot;Q value&quot; and another &quot;state-action pair value&quot; without at least some other text. So in general you are safe to consider the two terms to mean the same thing.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-15T14:33:15.477" LastActivityDate="2020-04-15T14:33:15.477" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20288" PostTypeId="1" AcceptedAnswerId="20298" CreationDate="2020-04-15T15:09:28.690" Score="3" ViewCount="73" Body="&lt;p&gt;Following an earlier question, I'm interested in understanding the basics of &lt;code&gt;Conv2d&lt;/code&gt; and especially how the kernel is applied, summed, and the propagated.&#xA;I understand that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a &lt;code&gt;kernel&lt;/code&gt; has size &lt;code&gt;W&lt;/code&gt; x &lt;code&gt;H&lt;/code&gt; and that more than one kernel is applied (e.g., S x W x H) where &lt;code&gt;S&lt;/code&gt; is the amount of kernels.&lt;/li&gt;&#xA;&lt;li&gt;A stride or step is used when iterating the network input&lt;/li&gt;&#xA;&lt;li&gt;Padding may or may not be used by being added to the network input.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What I would ideally like to see is either a description, or a python sample (&lt;code&gt;pytorch&lt;/code&gt; or &lt;code&gt;tensorflow&lt;/code&gt;) of how that is done, what the dimensionality of the output is, and any operation I may be missing (some YouTube videos say that the kernel is summarised and then divided to hold one new unique value representing the feature activation?)&lt;/p&gt;&#xA;" OwnerUserId="32528" LastEditorUserId="32528" LastEditDate="2020-04-15T16:24:31.927" LastActivityDate="2020-04-15T22:29:19.307" Title="How is the convolution layer is usually implemented in practice?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20289" PostTypeId="1" AcceptedAnswerId="20368" CreationDate="2020-04-15T16:20:36.693" Score="0" ViewCount="59" Body="&lt;p&gt;Even when we get a valuable reward signal after every single action, this &lt;em&gt;immediate reward&lt;/em&gt; only approximates the &lt;strong&gt;short term&lt;/strong&gt; goodness of the action. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To consider the &lt;strong&gt;long term&lt;/strong&gt; effect of an action, we can use the return of an episode, the action value function &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; or the advantage &lt;span class=&quot;math-container&quot;&gt;$A(s,a) = Q(s,a) - V(s)$&lt;/span&gt;. However, these measures do not rate the action in isolation but take all the following actions until the end of an episode into account. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there ways to more precisely approximate how good a single action really is considering its short and long term effects? &lt;/p&gt;&#xA;" OwnerUserId="35821" LastEditorUserId="35821" LastEditDate="2020-04-18T13:26:52.007" LastActivityDate="2020-04-18T22:11:35.953" Title="What is the best measurement for how good an action of a reinforcement learning agent really is?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20290" PostTypeId="2" ParentId="18576" CreationDate="2020-04-15T16:35:43.893" Score="2" Body="&lt;p&gt;My 50cents:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/NP_(complexity)&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP_(complexity)&lt;/a&gt; - is still hard to solve, even with NeuralNets.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In computational complexity theory, NP (nondeterministic polynomial&#xA;  time) is a complexity class used to classify decision problems. NP is&#xA;  the set of decision problems for which the problem instances, where&#xA;  the answer is &quot;yes&quot;, have proofs verifiable in polynomial time by a&#xA;  deterministic Turing machine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The easiest example, to imagine what is speech about, it is cryptography's &lt;a href=&quot;https://en.wikipedia.org/wiki/Integer_factorization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integer_factorization&lt;/a&gt;, which is basement of &lt;a href=&quot;https://en.wikipedia.org/wiki/RSA_(cryptosystem)&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSA cryptosystem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we have two simple numbers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;12123123123123123123123.....45456&lt;/li&gt;&#xA;&lt;li&gt;23412421341234124124124.....11112&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;NeuralNetwork shall answer us exactly digit to digit both this numbers, when we will show it only multiplication of this two numbers... This is not guessing about school bus. The field of numbers much more bigger than number of words in all languages on whole Earth. Imagine, that there are billions of billion different school buses, billions of billions of different fire-hydrants and billions of such classes, and NN shall answer exactly - what is on the picture - no way. The chance to guess is so little... &lt;/p&gt;&#xA;" OwnerUserId="36078" LastActivityDate="2020-04-15T16:35:43.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20291" PostTypeId="1" CreationDate="2020-04-15T17:06:26.140" Score="1" ViewCount="23" Body="&lt;p&gt;I'm trying to predict some properties of videos with Keras using the following rough architecture:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Feed each frame through the same 2-D convolutional layer. &lt;/li&gt;&#xA;&lt;li&gt;Take the outputs of this 2-D convolutional layer and feed them through a 3-D convolutional layer. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are more hidden layers, but these are the main ones that matter and are messing with my dimensionality. The input of &lt;code&gt;Conv2D&lt;/code&gt; should be &lt;code&gt;(batch_size, height, width, channels)&lt;/code&gt;. Each movie has dimensionality &lt;code&gt;(number_of_frames, height, width, channels)&lt;/code&gt;. I first had the idea to neglect batching of movies entirely, and treat the batch size and the number of frames equivalently. Then, &lt;code&gt;Conv2D&lt;/code&gt; would output a 4-D tensor, and I would increase its dimensionality to make the ouput a 5-D tensor that I could input into &lt;code&gt;Conv3D&lt;/code&gt;. To do this, &lt;code&gt;Conv3D&lt;/code&gt; could only accept inputs of batch size 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided against this, because I wanted to batch movies. My current thought is to do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv1 = Conv3D(filters=1, kernel_size =(1,12,12), strides=(1,1,1), data_format='channels_last')&#xA;conv2 = Conv3D(filters=1,kernel_size=(10,10,10), strides=(1,1,1), data_format='channels_last')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conv1&lt;/code&gt; would represent the 2-D convolutional layer while &lt;code&gt;conv2&lt;/code&gt; would represent the 3-D convolutional layer. Would this idea work? I figure there is the advantage that I can batch now and when I train the 2-D filter, the same 2-D filter is running over every single movie frame. I'm just worried that  the filter in &lt;code&gt;conv1&lt;/code&gt; will fail to go over certain frames, or it will somehow overlap frames when I want the filter to go over every frame individually.&lt;/p&gt;&#xA;" OwnerUserId="36079" LastEditorUserId="36079" LastEditDate="2020-04-16T15:04:54.593" LastActivityDate="2020-04-16T15:04:54.593" Title="Is using a filter of size (1, x, y) on a 3D convolutional layer the same as using a filter of size (x,y) on a 2D convolutional layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20292" PostTypeId="1" CreationDate="2020-04-15T18:37:21.783" Score="3" ViewCount="160" Body="&lt;p&gt;I'm not really sure if this is the sort of question to ask on here, since it is less of a general question about AI and more about the coding of it, however I thought it wouldn't fit on stack overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been programming a multilayer perceptron in c++, and it seems to be working with a sigmoid function, however when I change the activation function to ReLU it does not converge and stays at an average cost of 1 per training example. this is because all of the network's output neurons output a 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the sigmoid function it converges rather nicely, I did a bit of testing and after about 1000 generations it got to an average cost of 0.1 on the first 1000 items in the MNIST dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will show you the code I changes first for the activation functions, and then i will put the whole block of code in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sigmoid:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inline float activation(float num)&#xA;{&#xA;    return 1 / (1 + std::exp(-num));&#xA;}&#xA;&#xA;inline float activation_derivative(float num)&#xA;{&#xA;    return activation(num) * (1 - activation(num));&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;ReLU:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inline float activation(float num)&#xA;{&#xA;    return std::max(num, 0.0f);&#xA;}&#xA;&#xA;inline float activation_derivative(float num)&#xA;{&#xA;    return num &amp;gt; 0 ? 1.0f : 0.0f;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here's the whole block of code (I collapsed the region of code for benchmarking and the region for creating the dataset):&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;#include &amp;lt;chrono&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;&#xA;#pragma region benchmarking&#xA;#pragma endregion&#xA;&#xA;class Network&#xA;{&#xA;public:&#xA;    float cost = 0.0f;&#xA;    std::vector&amp;lt;std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt;&amp;gt; weights;&#xA;    std::vector&amp;lt;std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt;&amp;gt; deriv_weights;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; biases;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; deriv_biases;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; activations;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; deriv_activations;&#xA;    void clear_deriv_activations()&#xA;    {&#xA;        for (unsigned int i = 0; i &amp;lt; deriv_activations.size(); ++i)&#xA;        {&#xA;            std::fill(deriv_activations[i].begin(), deriv_activations[i].end(), 0.0f);&#xA;        }&#xA;    }&#xA;    int get_memory_usage()&#xA;    {&#xA;        int memory = 4;&#xA;        memory += get_vector_memory_usage(weights);&#xA;        memory += get_vector_memory_usage(deriv_weights);&#xA;        memory += get_vector_memory_usage(biases);&#xA;        memory += get_vector_memory_usage(deriv_biases);&#xA;        memory += get_vector_memory_usage(activations);&#xA;        memory += get_vector_memory_usage(deriv_activations);&#xA;        return memory;&#xA;    }&#xA;};&#xA;&#xA;struct DataSet&#xA;{&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; training_inputs;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; training_answers;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; testing_inputs;&#xA;    std::vector&amp;lt;std::vector&amp;lt;float&amp;gt;&amp;gt; testing_answers;&#xA;};&#xA;&#xA;&#xA;Network create_network(std::vector&amp;lt;int&amp;gt; layers)&#xA;{&#xA;    Network network;&#xA;    int layer_count = layers.size() - 1;&#xA;    network.weights.reserve(layer_count);&#xA;    network.deriv_weights.reserve(layer_count);&#xA;    network.biases.reserve(layer_count);&#xA;    network.deriv_biases.reserve(layer_count);&#xA;    network.activations.reserve(layer_count);&#xA;    network.deriv_activations.reserve(layer_count);&#xA;    int nodes_in_prev_layer = layers[0];&#xA;    for (unsigned int i = 0; i &amp;lt; layers.size() - 1; ++i)&#xA;    {&#xA;        int nodes_in_layer = layers[i + 1];&#xA;        network.weights.emplace_back();&#xA;        network.weights[i].reserve(nodes_in_layer);&#xA;        network.deriv_weights.emplace_back();&#xA;        network.deriv_weights[i].reserve(nodes_in_layer);&#xA;        network.biases.emplace_back();&#xA;        network.biases[i].reserve(nodes_in_layer);&#xA;        network.deriv_biases.emplace_back(nodes_in_layer, 0.0f);&#xA;        network.activations.emplace_back(nodes_in_layer, 0.0f);&#xA;        network.deriv_activations.emplace_back(nodes_in_layer, 0.0f);&#xA;        for (int j = 0; j &amp;lt; nodes_in_layer; ++j)&#xA;        {&#xA;            network.weights[i].emplace_back();&#xA;            network.weights[i][j].reserve(nodes_in_prev_layer);&#xA;            network.deriv_weights[i].emplace_back(nodes_in_prev_layer, 0.0f);&#xA;            for (int k = 0; k &amp;lt; nodes_in_prev_layer; ++k)&#xA;            {&#xA;                float input_weight = (2 * (float(std::rand()) / RAND_MAX)) - 1; &#xA;                network.weights[i][j].push_back(input_weight);&#xA;            }&#xA;            float input_bias = (2 * (float(std::rand()) / RAND_MAX)) - 1;&#xA;            network.biases[i].push_back(input_bias);&#xA;        }&#xA;        nodes_in_prev_layer = nodes_in_layer;&#xA;    }&#xA;    return network;&#xA;}&#xA;&#xA;void judge_network(Network &amp;amp;network, const std::vector&amp;lt;float&amp;gt;&amp;amp; correct_answers)&#xA;{&#xA;    int final_layer_index = network.activations.size() - 1;&#xA;    for (unsigned int i = 0; i &amp;lt; network.activations[final_layer_index].size(); ++i)&#xA;    {&#xA;        float val_sq = (network.activations[final_layer_index][i] - correct_answers[i]);&#xA;        network.cost += val_sq * val_sq;&#xA;    }&#xA;}&#xA;&#xA;inline float activation(float num)&#xA;{&#xA;    return std::max(num, 0.0f);&#xA;}&#xA;&#xA;void forward_propogate(Network&amp;amp; network, const std::vector&amp;lt;float&amp;gt;&amp;amp; input)&#xA;{&#xA;    const std::vector&amp;lt;float&amp;gt;* last_layer_activations = &amp;amp;input;&#xA;    int last_layer_node_count = input.size();&#xA;    for (unsigned int i = 0; i &amp;lt; network.weights.size(); ++i)&#xA;    {&#xA;        for (unsigned int j = 0; j &amp;lt; network.weights[i].size(); ++j)&#xA;        {&#xA;            float total = network.biases[i][j];&#xA;            for (int k = 0; k &amp;lt; last_layer_node_count; ++k)&#xA;            {&#xA;                total +=  (*last_layer_activations)[k] * network.weights[i][j][k];&#xA;            }&#xA;            network.activations[i][j] = activation(total);&#xA;        }&#xA;        last_layer_activations = &amp;amp;network.activations[i];&#xA;        last_layer_node_count = network.weights[i].size();&#xA;    }&#xA;}&#xA;&#xA;void final_layer_deriv_activations(Network&amp;amp; network, const std::vector&amp;lt;float&amp;gt;&amp;amp; correct_answers)&#xA;{&#xA;    int final_layer_index = network.activations.size() - 1;&#xA;    int final_layer_node_count = network.activations[final_layer_index].size();&#xA;    for (int i = 0; i &amp;lt; final_layer_node_count; ++i)&#xA;    {&#xA;        float deriv = network.activations[final_layer_index][i] - correct_answers[i];&#xA;        network.deriv_activations[final_layer_index][i] = deriv * 2;&#xA;    }&#xA;}&#xA;&#xA;inline float activation_derivative(float num)&#xA;{&#xA;    return num &amp;gt; 0 ? 1.0f : 0.0f;&#xA;}&#xA;&#xA;void back_propogate_layer(Network&amp;amp; network, int layer)&#xA;{&#xA;    int nodes_in_layer = network.activations[layer].size();&#xA;    int nodes_in_prev_layer = network.activations[layer - 1].size();&#xA;    for (int i = 0; i &amp;lt; nodes_in_layer; ++i)&#xA;    {&#xA;        float total = network.biases[layer][i];&#xA;        for (int j = 0; j &amp;lt; nodes_in_prev_layer; ++j)&#xA;        {&#xA;            total += network.weights[layer][i][j] * network.activations[layer - 1][j];&#xA;        }&#xA;        float dzda = activation_derivative(total);&#xA;        float dzdc = dzda * network.deriv_activations[layer][i];&#xA;        for (int j = 0; j &amp;lt; nodes_in_prev_layer; ++j)&#xA;        {&#xA;            network.deriv_weights[layer][i][j] += network.activations[layer - 1][j] * dzdc;&#xA;            network.deriv_activations[layer - 1][j] += network.weights[layer][i][j] * dzdc;&#xA;        }&#xA;        network.deriv_biases[layer][i] += dzdc;&#xA;    }&#xA;}&#xA;&#xA;void back_propogate_first_layer(Network&amp;amp; network, std::vector&amp;lt;float&amp;gt; inputs)&#xA;{&#xA;    int nodes_in_layer = network.activations[0].size();&#xA;    int input_count = inputs.size();&#xA;    for (int i = 0; i &amp;lt; nodes_in_layer; ++i)&#xA;    {&#xA;        float total = network.biases[0][i];&#xA;        for (int j = 0; j &amp;lt; input_count; ++j)&#xA;        {&#xA;            total += network.weights[0][i][j] * inputs[j];&#xA;        }&#xA;        float dzda = activation_derivative(total);&#xA;        float dzdc = dzda * network.deriv_activations[0][i];&#xA;        for (int j = 0; j &amp;lt; input_count; ++j)&#xA;        {&#xA;            network.deriv_weights[0][i][j] += inputs[j] * dzdc;&#xA;        }&#xA;        network.deriv_biases[0][i] += dzdc;&#xA;    }&#xA;}&#xA;&#xA;void back_propogate(Network&amp;amp; network, const std::vector&amp;lt;float&amp;gt;&amp;amp; inputs, const std::vector&amp;lt;float&amp;gt;&amp;amp; correct_answers)&#xA;{&#xA;    network.clear_deriv_activations();&#xA;    final_layer_deriv_activations(network, correct_answers);&#xA;    for (int i = network.activations.size() - 1; i &amp;gt; 0; --i)&#xA;    {&#xA;        back_propogate_layer(network, i);&#xA;    }&#xA;    back_propogate_first_layer(network, inputs);&#xA;}&#xA;&#xA;void apply_derivatives(Network&amp;amp; network, int training_example_count)&#xA;{&#xA;    for (unsigned int i = 0; i &amp;lt; network.weights.size(); ++i)&#xA;    {&#xA;        for (unsigned int j = 0; j &amp;lt; network.weights[i].size(); ++j)&#xA;        {&#xA;            for (unsigned int k = 0; k &amp;lt; network.weights[i][j].size(); ++k)&#xA;            {&#xA;                network.weights[i][j][k] -= network.deriv_weights[i][j][k] / training_example_count;&#xA;                network.deriv_weights[i][j][k] = 0;&#xA;            }&#xA;            network.biases[i][j] -= network.deriv_biases[i][j] / training_example_count;&#xA;            network.deriv_biases[i][j] = 0;&#xA;            network.deriv_activations[i][j] = 0;&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;void training_iteration(Network&amp;amp; network, const DataSet&amp;amp; data)&#xA;{&#xA;    int training_example_count = data.training_inputs.size();&#xA;    for (int i = 0; i &amp;lt; training_example_count; ++i)&#xA;    {&#xA;        forward_propogate(network, data.training_inputs[i]);&#xA;        judge_network(network, data.training_answers[i]);&#xA;        back_propogate(network, data.training_inputs[i], data.training_answers[i]);&#xA;    }&#xA;    apply_derivatives(network, training_example_count);&#xA;}&#xA;&#xA;void train_network(Network&amp;amp; network, const DataSet&amp;amp; dataset, int training_iterations)&#xA;{&#xA;    for (int i = 0; i &amp;lt; training_iterations; ++i)&#xA;    {&#xA;        training_iteration(network, dataset);&#xA;        std::cout &amp;lt;&amp;lt; &quot;Generation &quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; network.cost &amp;lt;&amp;lt; std::endl;&#xA;        network.cost = 0.0f;&#xA;    }&#xA;}&#xA;&#xA;#pragma region dataset creation&#xA;&#xA;#pragma endregion&#xA;&#xA;int main() &#xA;{&#xA;    Timer timer;&#xA;    DataSet dataset = create_dataset_from_file(&quot;data.txt&quot;);&#xA;    Network network = create_network({784, 128, 10});&#xA;    train_network(network, dataset, 1000);&#xA;    std::cout &amp;lt;&amp;lt; timer.get_duration() &amp;lt;&amp;lt; std::endl;&#xA;    std::cin.get();&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33354" LastEditorUserId="33354" LastEditDate="2020-04-15T21:18:15.603" LastActivityDate="2020-04-16T10:36:08.557" Title="Neural network doesn't seem to converge with ReLU but it does with Sigmoid?" Tags="&lt;convergence&gt;&lt;relu&gt;&lt;c++&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20293" PostTypeId="2" ParentId="18802" CreationDate="2020-04-15T19:33:30.397" Score="1" Body="&lt;p&gt;The paper &lt;a href=&quot;https://www.cis.upenn.edu/~mkearns/papers/tdlambda.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Bias-Variance&quot; Error Bounds for Temporal Difference Updates&lt;/a&gt; (2000) by M. Kearns and S. Singh provides error bounds for temporal-difference algorithms, i.e. TD(&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;) and TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) (see &lt;a href=&quot;https://www.cis.upenn.edu/~mkearns/papers/tdlambda.pdf#page=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorem 1&lt;/a&gt; and &lt;a href=&quot;https://www.cis.upenn.edu/~mkearns/papers/tdlambda.pdf#page=5&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorem 2&lt;/a&gt;, respectively). Note that both TD(&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;) and TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) include TD(&lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) as a special case.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-15T19:33:30.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20294" PostTypeId="2" ParentId="12020" CreationDate="2020-04-15T20:00:33.363" Score="1" Body="&lt;p&gt;The use of of a neural network to push the search algorithm to continually only along a promising path is the same that was described in the AlphaZero paper. In AlphaZero, the NN loop contained the search function and would encourage the continued search of high probability moves that were then simulated by the same NN that now contained the Value Net. The use of alpha-beta specifically is not necessary. Just a search function aptly known as PUCT (Predictor + Upper Confidence Bounds applied to Trees)&lt;/p&gt;&#xA;" OwnerUserId="36086" LastActivityDate="2020-04-15T20:00:33.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20295" PostTypeId="5" CreationDate="2020-04-15T20:13:57.153" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-15T20:13:57.153" LastActivityDate="2020-04-15T20:13:57.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20296" PostTypeId="4" CreationDate="2020-04-15T20:13:57.153" Score="0" Body="For questions related to the multi-armed bandit (MAB) problem, in which a fixed limited set of resources must be allocated between competing (alternative) choices in a way that maximizes their expected gain, when each choice's properties are only partially known at the time of allocation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-15T20:13:57.153" LastActivityDate="2020-04-15T20:13:57.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20297" PostTypeId="2" ParentId="20259" CreationDate="2020-04-15T21:17:20.503" Score="1" Body="&lt;p&gt;DeepMind used &lt;a href=&quot;http://www.mujoco.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuJoCo&lt;/a&gt; (see also the related paper &lt;a href=&quot;https://homes.cs.washington.edu/~todorov/papers/TodorovIROS12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuJoCo: A physics engine for model-based control&lt;/a&gt;) for the simulations, as they stated in section 3.1 of their paper &lt;a href=&quot;https://arxiv.org/pdf/1707.02286.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Emergence of Locomotion Behaviours in Rich Environments&lt;/a&gt; (2017), which is the paper you should read to know more about their results related to those animations of skeletons that try to walk or jump (but do it weirdly).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-15T21:17:20.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20298" PostTypeId="2" ParentId="20288" CreationDate="2020-04-15T22:06:06.353" Score="3" Body="&lt;p&gt;I don't think that to understand convolution you need to dig into the nested code of huge libraries, since the code becomes quickly really hard to understand and convoluted (ba dum tsss!). Joking apart, in PyTorch &lt;a href=&quot;https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#Conv2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conv2d&lt;/a&gt; is a layer that applies another low level function, &lt;a href=&quot;https://github.com/pytorch/pytorch/blob/c780610f2d8358297cb4e4460692d496e124d64d/aten/src/ATen/native/Convolution.cpp#L481&quot; rel=&quot;nofollow noreferrer&quot;&gt;conv2d&lt;/a&gt;, written in c++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily enough, the guys from PyTorch wrote the general idea of how convolution is implemented in the documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ToMaM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ToMaM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From this paragraph, we already have some important information like the input and output dimensions. The number of channels should be easy to understand, if we have an RGB image, for example, the channels are 3, one for each color, so they are just different matrices representing different features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next important element is the reference to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-correlation&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross correlation&lt;/a&gt;, the function applied to our input images through the kernel k. Why cross-correlation? Because it is almost identical to a convolution, as you can see comparing their formulas:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Bmns0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bmns0.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only difference consists in the way the indexing of the width is implemented, which causes the operation to start from the bottom right of the input matrix for the convolution and from the top left of the input image from the cross-correlation (the circles in the squares in the previous pic). Since, in most programming languages, matrix indexing starts from top left, cross-correlation is the most common choice to implement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do these formulas work in practice? Here's another picture taken from Chapter 9 of &lt;a href=&quot;http://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; (Goodfellow, Bengio, Courville), which I strongly suggest you to read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wZsdt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wZsdt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, from the input matrix, a submatrix is extracted, with the same dimension of the kernel, then the sub-matrix and the kernel are multiplied &lt;strong&gt;elementwise&lt;/strong&gt; and all the resulting product summed together to produce a single output element that will form a 'pixel' of the resulting &lt;em&gt;feature map&lt;/em&gt; (the output matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's another example with fake numbers that I made. I hope the double notation for filter/kernel doesn't generate confusion, I actually found that sometimes it is inconsistent (in the chapter I linked they don't even use the filter at all). In practice they actually mean the same thing, I usually call kernel the actual matrix that is multiplied to the input and with filter I refer to the sliding window on the input image (that has, of course, must have the same dimension of the kernel). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0DmF4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0DmF4.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, when you apply padding, the filter can actually move also outside the 'edges' of the input matrix, in which case all the elements outside would be considered to be zero. The computation is exactly the same, but since there are more splitting steps, the output matrix will have bigger dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nTXy3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nTXy3.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that with multiple input channels you can perform either 2d convolution or 3d convolution, the difference relies in the filter dimension, in case of 2d convolution it would be a square whereas in 3d convolution it would be a cube. This means that for an RGB image a 2d convolution would treat each color layer independently, mixing the information from each channel with further computations like pooling (averaging the resulting feature maps of each color or select the max value among the feature maps for each pixel, etc..) while 3d convolution would mix together the color layers already during convolution thanks to the 3d kernel which will sum together elements from different layers. &lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="34098" LastEditDate="2020-04-15T22:29:19.307" LastActivityDate="2020-04-15T22:29:19.307" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20300" PostTypeId="1" AcceptedAnswerId="20302" CreationDate="2020-04-16T00:52:59.400" Score="1" ViewCount="56" Body="&lt;p&gt;From my understanding, the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is basically how the agent acts (i.e. the actions it will take in each state).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am confused about the Q value and how it is &quot;affected&quot; by a policy. &lt;a href=&quot;https://datascience.stackexchange.com/a/31791/10640&quot;&gt;This answer&lt;/a&gt; says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s, a)$&lt;/span&gt; is the action-value function. It is the expected return starting from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. It's focusing on the particular action at the particular state.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From this, I infer that the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value (the action-value function) will be affected by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Why? So, why does the Q value change according to policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shouldn't the Q value be constant, because the same action taken in the same state will always give the same yield (and hence remain constantly good/bad)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the policy does is find out the max Q values and bases its policy on that information.&lt;/p&gt;&#xA;" OwnerUserId="36072" LastEditorUserId="2444" LastEditDate="2020-04-16T01:30:50.120" LastActivityDate="2020-04-16T01:41:47.133" Title="Why does the policy $\pi$ affect the Q value?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;policies&gt;&lt;value-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20301" PostTypeId="2" ParentId="20300" CreationDate="2020-04-16T01:05:35.663" Score="0" Body="&lt;p&gt;Ok, Q is the reward associated with being in a given state, following a certain action and then following the given policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to take the expectation of the sum the immediate reward plus the value function which is defined by the policy.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-16T01:05:35.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20302" PostTypeId="2" ParentId="20300" CreationDate="2020-04-16T01:41:47.133" Score="2" Body="&lt;p&gt;First of all, &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; IS DEFINED AS the value (i.e. the expected return) of taking some action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, AND THEN following some given policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (until e.g. the end of the game or your life). In other words, suppose that you take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, AND THEN use the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to behave in the world until you die, then &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; would represent the value that you would obtain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, we are DEFINING &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; in a certain way.  This is a DEFINITION! It's not an algorithm. In the algorithms (e.g. Q-learning), things will typically change, but that's a different story that you should investigate later.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From this, I infer that the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value (the action-value function) will be affected by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; will not keep changing. You could say that &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; (which is a function) is &quot;affected by&quot; &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; ONLY in the sense that it is &quot;defined in terms of&quot; &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. To be precise, &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; is actually an expectation (which is a mathematical concept similar to an ideal average). If you are not familiar with the concept of expectation, I suggest you get familiar with it first, before studying reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Shouldn't the Q value be constant, because the same action taken in the same state will always give the same yield (and hence remain constantly good/bad)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Again, there's the distinction between the algorithm that you actually use to find the function &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; and the definition of the same function. In case you are estimating the function with an algorithm, you will not necessarily find &quot;constant Q values&quot;. It depends on different aspects, which I would like to avoid discussing here, so that this post doesn't become an open discussion (I suggest you first learn about the basic Bellman equations and then you study the algorithms from the book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning: an introduction&lt;/a&gt; by Sutton and Barto).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-16T01:41:47.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20303" PostTypeId="1" AcceptedAnswerId="20318" CreationDate="2020-04-16T01:53:36.553" Score="2" ViewCount="175" Body="&lt;p&gt;I'm a beginner in the RL field, and I would like to check that my understanding of certain RL concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Value function&lt;/strong&gt;: How good it is to be in a state S following policy π.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uPiKw.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uPiKw.jpg&quot; alt=&quot;Value function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;So, the value functions here are 0.3 and 0.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q function&lt;/strong&gt;(also called state-action value, or just action value): How good it is to be in a state S and perform action A while following policy π. It uses reward to measure the state-action value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yn967.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yn967.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;So, the state-action values here are 0.03,0.02,0.5 and 0.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q value&lt;/strong&gt;: The overall expected rewards after performing action A in state S, and continuing with policy π until the end of the episode. So, essentially I can only calculate the Q value if I know all the state-action values of the actions I will be taking in the single episode.(Because the Q value takes into account the actions after the current action A, till the end of the episode, following policy π)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reward&lt;/strong&gt;: The metric used to tell the agent how good/bad it's action was. It is a constant value. &#xA;For e.g &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1. Fall in pond --&amp;gt; -1&#xA; 2. On stone path --&amp;gt; +1&#xA; 3. Reach home--&amp;gt; +10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Return&lt;/strong&gt;: The sum of rewards in a single episode&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Policy π&lt;/strong&gt;: A set of specific instructions an agent will follow in an episode. For example, the policy will look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In state 1, take action 3 ( which takes me to state 2)&#xA;&#xA;In state 2, take action 2 ( which takes me to state 3)&#xA;&#xA;In state 3, take action 1 ( Which takes me to state 4)&#xA;&#xA;In state 4, take action 2 ( Which takes me to terminal state)&#xA;&#xA;1 episode completed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And my policy will keep updating each episode to get the best return&lt;/p&gt;&#xA;" OwnerUserId="36072" LastEditorUserId="2444" LastEditDate="2020-04-16T02:11:11.473" LastActivityDate="2020-04-16T13:25:14.250" Title="Is my understanding of the value function, Q function, policy, reward and return correct?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;terminology&gt;&lt;policies&gt;&lt;return&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20304" PostTypeId="1" CreationDate="2020-04-16T02:21:24.563" Score="2" ViewCount="76" Body="&lt;p&gt;I have the following problem called &quot;1-2 steal marbles&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, there are 6 marbles on the board. One of the players can choose to remove 1 or 2 marbles leaving 5 or 4. After that, the other player can do the same, choosing to take again 1 or 2 marbles from the board. The process continues until there is only one marble on the board. The player who wins is the one the leaves the last marble on the board. (For example: If there are 3 marbles and it's my turn, then I will choose to remove 2 to leave one in the board to win)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I draw the search tree that represents the application of the alpha-beta pruning to this &quot;1-2 steal marbles&quot; with 13 marbles? I would like to see the maximizer and minimizer nodes and the value at the nodes too.&lt;/p&gt;&#xA;" OwnerUserId="36094" LastEditorUserId="2444" LastEditDate="2020-04-18T00:30:29.997" LastActivityDate="2020-04-18T00:30:29.997" Title="How can I apply the alpha-beta pruning algorithm to the &quot;1-2 steal marbles&quot; problem?" Tags="&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20305" PostTypeId="1" CreationDate="2020-04-16T02:57:49.810" Score="2" ViewCount="93" Body="&lt;p&gt;I have the following scenario. I have a binary classification problem, whose underlying function is a step function. The probability distribution of feature vectors is a uniform over the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case 1: I have a classifier which fits the training samples perfectly, no matter what the size of the data. The space of functions &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; has an infinite VC dimension. As the data points going to infinite, the hypothesized function converges pointwise to the underlying step function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case 2: Here I have divided the same hypothesis space into a number of hierarchical subspaces &lt;span class=&quot;math-container&quot;&gt;$H_1 \subset H_2 \subset H_3 \subset \dots \subset H_n$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; goes to infinity). The VC dimension of each of the spaces is finite and grows with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to infinity. Now, given any data of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; points, I compute the minimum number of VC dimension required to fit the data exactly, say, &lt;span class=&quot;math-container&quot;&gt;$d_n$&lt;/span&gt; and use that space &lt;span class=&quot;math-container&quot;&gt;$H_{d_n}$&lt;/span&gt; as the hypothesis. Do the same as data size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; goes to infinity, at each &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; using the hypothesis space that just enough VC dimension to fit the data. In this approach also, as the data size goes to infinity, the hypothesized function converges pointwise to the underlying step function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is the difference between these two approaches to the same problem? Is there any theoretical difference? Which method is any better than others, in some sense?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="36095" LastEditorUserId="2444" LastEditDate="2020-04-16T03:04:46.860" LastActivityDate="2021-05-11T20:00:40.127" Title="An infinite VC dimensional space vs using hierarchical subspaces of finite but growing VC dimensions" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;supervised-learning&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20306" PostTypeId="2" ParentId="16213" CreationDate="2020-04-16T03:17:57.770" Score="0" Body="&lt;p&gt;I noticed the same issue. Also, it seems that the equation for applying momentum does not exactly match up with the standard one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ v_{d\theta} := \beta v_{d\theta} + (1 - \beta)d\theta \\&#xA; \theta := \theta - \alpha v_{d\theta}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="36096" LastActivityDate="2020-04-16T03:17:57.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20307" PostTypeId="2" ParentId="20303" CreationDate="2020-04-16T04:08:01.877" Score="1" Body="&lt;p&gt;I think most of it is correct. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Q function(also called state-action value, or just action value): How good it is to be in a state S and perform action A while following policy π. It uses reward to measure the state-action value&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a bit off. Q function basically tells you how good it is to be in state S and perform action A, and follow policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; from the next state onwards. The action A that you take can be any action from the action space and need not be according to the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I think Q-function and Q-value are mostly used interchangeably to mean the same thing. &lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-04-16T04:08:01.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20308" PostTypeId="1" AcceptedAnswerId="20315" CreationDate="2020-04-16T06:37:29.933" Score="2" ViewCount="175" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Very deep models involve the composition of several functions or layers. The gradient tells how to update each parameter, under the assumption that the other layers do not change. In practice, we update all of the layers simultaneously.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The above is an extract from Ian Goodfellow's Deep Learning  - which talks about the need for batch normalization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why do we update all the layers simultaneously?&lt;/strong&gt; Instead, if we update layers one at a time during backpropagation - it will eliminate the need for batch normalization, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;a href=&quot;https://machinelearningmastery.com/batch-normalization-for-training-of-deep-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Gentle Introduction to Batch Normalization for Deep Neural Networks&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. The attached link says: &lt;em&gt;Because all layers are changed during an update, the update procedure is forever chasing a moving target&lt;/em&gt;. Apart from the main question, it would be great if someone could explain why exactly a &lt;em&gt;moving target&lt;/em&gt; is being referred to in the above sentence.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-04-16T11:48:33.663" LastActivityDate="2020-04-16T12:10:42.197" Title="Why do we update all layers simultaneously while training a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20309" PostTypeId="1" AcceptedAnswerId="20327" CreationDate="2020-04-16T06:44:00.997" Score="6" ViewCount="569" Body="&lt;p&gt;Although I know how the algorithm of iterative policy evaluation using dynamic programming works, I am having a hard time realizing how it actually converges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appeals to intuition that, with each iteration, we get a better and better approximation for the value function and we can thus assure its convergence, but with this said simply, it seems that this method is very inefficient contrary to the reality that it actually is quite efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the rigorous mathematical proof of the convergence of the policy evaluation algorithm to the actual answer? How is it that the value function obtained this way is close to the actual values computed by solving the set of bellman equations?&lt;/p&gt;&#xA;" OwnerUserId="35926" LastEditorUserId="2444" LastEditDate="2020-04-16T13:00:17.217" LastActivityDate="2020-04-17T03:10:42.337" Title="What is the proof that policy evaluation converges to the optimal solution?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;proofs&gt;&lt;bellman-equations&gt;&lt;policy-evaluation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20310" PostTypeId="1" AcceptedAnswerId="20311" CreationDate="2020-04-16T07:08:18.887" Score="3" ViewCount="653" Body="&lt;p&gt;I'm a biotech student and I'm currently working on single-particle tracking. For my work, I need to use aspects of deep learning (CNN, RNN and object segmentation) but I'm not familiar with these topics. I have some prior knowledge in python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, do I have to learn machine learning first before going into deep learning, or can I skip ML?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of studying machine learning before deep learning?&lt;/p&gt;&#xA;" OwnerUserId="36099" LastEditorUserId="2444" LastEditDate="2020-04-16T11:01:26.027" LastActivityDate="2021-01-17T13:59:16.467" Title="What are the pros and cons of studying machine learning before deep learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;&lt;academia&gt;&lt;education&gt;" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20311" PostTypeId="2" ParentId="20310" CreationDate="2020-04-16T09:28:42.787" Score="7" Body="&lt;p&gt;That question doesn't really make sense: deep learning is a sub-topic of machine learning, so you can't really 'skip' it. It's a bit like &quot;I want to learn about trigonometry, but do I need to do geometry first?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, in order to make sense of deep learning you should really know about the general principles of machine learning, otherwise you won't understand it. Or, more importantly, you won't understand what problems deep learning can be applied to, and what issues are better solved with other methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need to go into much detail, but should at least get an overview.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-16T09:28:42.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20312" PostTypeId="2" ParentId="20292" CreationDate="2020-04-16T10:36:08.557" Score="1" Body="&lt;p&gt;It seems like you're suffering from the the dying ReLU problem. ReLU enforces positive values so the weights and biases your network learned are leading to a negative value passed through the ReLU function - meaning you would get 0. There are a few things you can do. I do not know the exact format of your data, but if it is MNIST it is possible you simply don't have normalized values. You could be learning a large negative bias as a result. Try dividing every pixel intensity in your dataset by the float 255.0 to normalize your values and see if that fixes your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also change your activation function to something such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Leaky_ReLU&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leaky ReLU&lt;/a&gt; which attempts to solve this problem with a small positive gradient for negative values. &lt;/p&gt;&#xA;" OwnerUserId="22373" LastActivityDate="2020-04-16T10:36:08.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20313" PostTypeId="2" ParentId="7640" CreationDate="2020-04-16T11:38:37.987" Score="0" Body="&lt;p&gt;&lt;strong&gt;There are two kinds of problem&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Stationary and non-stationary&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Stationary  problems are those whose reward value is static, dost not change&#xA;and on other hand non-stationary problems are those whose reward value change with time &lt;/p&gt;&#xA;" OwnerUserId="36104" LastActivityDate="2020-04-16T11:38:37.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20314" PostTypeId="1" CreationDate="2020-04-16T11:55:40.520" Score="1" ViewCount="43" Body="&lt;p&gt;I want to solve the two partition problem (&lt;a href=&quot;https://en.wikipedia.org/wiki/Partition_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Partition_problem&lt;/a&gt;) using an uninformed search algorithm (BFS or uniform cost). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The states can be represented by three sets S1,S1,S where the set S contains unassigned values and S1 and S2 the values assigned to each partition respectively. At the initial state S will contain all values and S1 and S2 will be empty. The actions consist in moving a value from S to S1 or S2. The objective is to find a complete assignation (S is empty) where abs(sum(S1)-sum(S2)) is minimum. As you can see I have all elements but the cost of the actions. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I assign costs to the actions in order to apply one of those algorithms? (Costs must be positive.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know it is not the best way to solve this problem but there must be a way to do it because the problem is formulated this way in the book.&lt;/p&gt;&#xA;" OwnerUserId="36105" LastEditorUserId="1671" LastEditDate="2020-04-16T22:06:54.393" LastActivityDate="2020-04-16T22:06:54.393" Title="2 Partition Problem" Tags="&lt;search&gt;&lt;breadth-first-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20315" PostTypeId="2" ParentId="20308" CreationDate="2020-04-16T12:05:40.867" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do we update all layers simultaneously while training a neural network?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We typically train a neural network with gradient descent and back-propagation. Gradient descent is the iterative algorithm used to update the parameters and back-propagation is the algorithm used to compute the gradient of the loss function with respect to each of these parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's denote a vector that contains all learnable parameters of a neural network &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} = \left[w_1, \dots, w_n \right] \in \mathbb{R}^n$&lt;/span&gt; (so &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; contains &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; learnable parameters), the loss function of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;,  the gradient of the loss function with respect to each parameter &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$ \nabla \mathcal{L} = \left[ \frac{\partial \mathcal{L}}{\partial w_1}, \dots, \frac{\partial \mathcal{L}}{\partial w_n}  \right] \in \mathbb{R}^n$&lt;/span&gt;, then the gradient descent step to update all parameters is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{w} \leftarrow \mathbf{w} - \gamma * \nabla \mathcal{L} \tag{1} \label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma \in \mathbb{R}$&lt;/span&gt; is the learning rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In equation \ref{1}, we are assigning to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; the value &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} - \gamma * \nabla \mathcal{L}$&lt;/span&gt;, so we are updating all parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}$&lt;/span&gt; simultaneously, so we are also updating all layers simultaneously. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, you could update each parameter &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; individually. To be more precise, you would have the following update rule&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;w_i \leftarrow w_i - \gamma * \frac{\partial \mathcal{L}}{\partial w_i}, \; \forall i\tag{2} \label{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you could update first &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt;, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, you don't need to update the parameters sequentially (also because there is no real order of the parameters). You can actually update them in any other way. You can update the parameters in any way because, although the computation of the gradient highly depends on the structure of the neural network (so if you change the structure, the computation of the gradient also changes), once the gradient is computed, you already have all information to update each parameter independently of each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You typically update all parameters (or layers) simultaneously because, in practice, you work with vectors and matrices rather than scalars in order to take benefit from efficient matrix multiplication algorithms and hardware (i.e. GPUs).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-16T12:10:42.197" LastActivityDate="2020-04-16T12:10:42.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20316" PostTypeId="1" CreationDate="2020-04-16T12:11:58.753" Score="2" ViewCount="76" Body="&lt;p&gt;I've been recently given an assignment based on Reinforcement Learning and I'm supposed to implement the value iteration algorithm in a grid environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assignment: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GOhkV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GOhkV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0IuZE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0IuZE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My doubt is why do I even need an initial arbitrary policy as given in the parameters (in the assignment) to implement the value iteration algorithm? And so, the change in the values of a and b shouldn't affect the algorithm. Am I correct in my thinking about it?&lt;/p&gt;&#xA;" OwnerUserId="29899" LastEditorUserId="29899" LastEditDate="2020-04-16T12:26:56.537" LastActivityDate="2020-04-22T16:23:14.560" Title="Why do I need an initial arbitrary policy to implement value iteration algorithm" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;value-functions&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20317" PostTypeId="1" CreationDate="2020-04-16T12:29:23.950" Score="2" ViewCount="31" Body="&lt;p&gt;I would like to design a reward function. I am training two models from the first model that classify set of texts (paragraphs and keywords) and I also got some hidden states. The second model is trying to generate keywords for those paragraphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use those hidden states from the first model to give rewards for key phrases that are generated from the second model. I want to know how can I implement this reward function since I have never used it before.&lt;/p&gt;&#xA;" OwnerUserId="36106" LastEditorUserId="2444" LastEditDate="2020-04-16T14:12:09.517" LastActivityDate="2020-04-16T14:12:09.517" Title="How should I design a reward function for a NLP problem where two models interoperate?" Tags="&lt;reinforcement-learning&gt;&lt;natural-language-processing&gt;&lt;generative-adversarial-networks&gt;&lt;rewards&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20318" PostTypeId="2" ParentId="20303" CreationDate="2020-04-16T12:42:12.383" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Value function&lt;/strong&gt;: How good it is to be in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are different value functions. There's the state value function, often denoted as &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;), so it's a function of only one variable, i.e. &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (a state). There's the state-action value function &lt;span class=&quot;math-container&quot;&gt;$q(s, a)$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$Q(s, a$&lt;/span&gt;)). A value function is a function, so it's not a number or a vector, or whatever. It's a function, so it maps inputs to outputs. In the first case, it maps states to real numbers. In the second case, it maps states and actions to real numbers. So, we could denote the state value function as &lt;span class=&quot;math-container&quot;&gt;$v : \mathcal{S} \rightarrow \mathbb{R}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; is the set of states in your environment) and state-action value function as &lt;span class=&quot;math-container&quot;&gt;$q : \mathcal{S} \times \mathcal{A}\rightarrow \mathbb{R}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is the set of actions and &lt;span class=&quot;math-container&quot;&gt;$\times$&lt;/span&gt; means &quot;combination of&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, your definition of a value function is not quite correct. The value function &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; doesn't represent &quot;how good it is to be in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; following a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;&quot;, but &quot;how good it is to be in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; AND THEN following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;&quot;. To emphasize this, you often use the notation &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; rather than simply &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/q/10575/2444&quot;&gt;What are the value functions used in reinforcement learning?&lt;/a&gt; for more details about existing value functions in reinforcement learning. And to see the full definition of the value functions, I suggest you read Sutton and Barto's book.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Q function&lt;/strong&gt; (also called state-action value, or just action value): How good it is to be in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and perform action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; while following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. It uses reward to measure the state-action value&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As I said above, the &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; function is a &quot;value function&quot; too. It's just a different value function than &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the same thing I said for &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; also applies here, so &quot;how good it is to be in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and perform action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; while following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;&quot; is incorrect for the same reason your definition for &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; was incorrect. The &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; function can be defined as &quot;how good it is to be in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, AND, AFTER THAT, follow a given policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Again, to emphasize that &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is defined in terms of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, we often use the notation &lt;span class=&quot;math-container&quot;&gt;$q_\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Reward&lt;/strong&gt;: The metric used to tell the agent how good/bad it's action was. It is a constant value. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is roughly correct, but the reward doesn't have to be constant and it depends on your problem. Also, there's also the related notion of &quot;reward function&quot;, which is the function that assigns rewards to each action. So, when defining your problem as a Markov decision process, you need to define this reward function. Actually, this is probably the most important function in reinforcement learning (because this is the way you teach the agent to behave).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Return&lt;/strong&gt;: The sum of rewards in a single episode&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is roughly correct. However, note that the sum can also be a &quot;weighted sum&quot;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Policy&lt;/strong&gt;: A set of specific instructions an agent will follow in an episode. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is roughly correct, but a policy can also have some randomness in it. For example, if you are in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, your policy could say &quot;always take action &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt;&quot;, but another policy could say &quot;take action &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a_j$&lt;/span&gt; with probably &lt;span class=&quot;math-container&quot;&gt;$1 - p$&lt;/span&gt;. Also, note that the policy is not restricted to an episode. It's a general function that tells the agent how to behave independently of the episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Sorry, I didn't look at your examples. Maybe I will review this answer later to look at your examples too, but the information in this answer should already tell you if your examples are correct or not).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-16T13:25:14.250" LastActivityDate="2020-04-16T13:25:14.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20319" PostTypeId="1" CreationDate="2020-04-16T13:16:38.073" Score="3" ViewCount="30" Body="&lt;p&gt;In the paper: &lt;a href=&quot;http://papers.nips.cc/paper/889-reinforcement-learning-methods-for-continuous-time-markov-decision-problems.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning methods for continuous-time Markov decision problems&lt;/a&gt;, the authors provide the following update rule for the Q-learning algorithm, when applied to Semi-Markov Decision Processes (SMDPs):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q^{(k+1)}(x,a) = Q^{(k)}(x,a) + \alpha_k [ \frac{1-e^{-\beta \tau}}{\beta}r(x,y,a) + e^{-\beta \tau} max_{a'} Q^{(k)}(y,a) - Q^{(k)}(x,a) ] $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha_k$&lt;/span&gt; is the learning rate, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; is the continuous time discount factor and &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the time taken to transition from state &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to state &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It is not clear to me what is the relationship between the sampled reward &lt;span class=&quot;math-container&quot;&gt;$r(x,y,a)$&lt;/span&gt; and the reward rate &lt;span class=&quot;math-container&quot;&gt;$\rho(x,a)$&lt;/span&gt; specified in the objective function &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[ \int_{0}^{\infty} e^{-\beta t}\rho(x(t),a(t)) dt ]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In particular, how do they determine &lt;span class=&quot;math-container&quot;&gt;$r(x,y,a)$&lt;/span&gt; in the experiments in Section 6? In this experiment, they consider a routing problem in an M/M/2 queuing system, where the reward rate is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$c_1 n_1(t) + c_2 n_2(t)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$c_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_2$&lt;/span&gt; are scalar cost factors and &lt;span class=&quot;math-container&quot;&gt;$n_1(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_2(t)$&lt;/span&gt; are the number of customers in queue 1 and 2, respectively.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-16T13:16:38.073" Title="Relationship between the reward rate and the sampled reward in a Semi-Markov Decision Process" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;semi-mdp&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20320" PostTypeId="2" ParentId="20309" CreationDate="2020-04-16T14:54:20.743" Score="-1" Body="&lt;p&gt;There exist other RL books which do a better job of talking about this but it's pretty simple at it's core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discounting factor puts an upper limit on the difference in reward between a finite number of iterations and an infinite number, each time you add another iteration it decreases by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; multiplied into the upper bound of the difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_\pi = E[\sum_{i=0}^{\infty} \gamma^iR_i]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Delta V_k = E[\sum_{i=k}^{\infty} \gamma^iR_i] = \gamma^k E[\sum_{i=0}^{\infty} \gamma^{i}R_{i+k}] &amp;lt; \gamma^k \frac{1}{1-\gamma}R_{max}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="32390" LastEditDate="2020-04-16T17:23:43.040" LastActivityDate="2020-04-16T17:23:43.040" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20321" PostTypeId="1" AcceptedAnswerId="20426" CreationDate="2020-04-16T16:14:20.827" Score="3" ViewCount="93" Body="&lt;p&gt;Assume that I have a Dataframe with the text column.&#xA;Problem: Classification / Prediction&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    sms_text&#xA;0   Go until jurong point, crazy.. Available only ...&#xA;1   Ok lar... Joking wif u oni...&#xA;2   Free entry in 2 a wkly comp to win FA Cup fina...&#xA;3   U dun say so early hor... U c already then say...&#xA;4   Nah I don't think he goes to usf, he lives aro...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After preprocessing the text&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4EQWp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4EQWp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;From the above WordCloud, we can find the most frequent(occurred) words like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Free&#xA;Call&#xA;Text&#xA;Txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As these are the most frequent words and adds less importance in prediction/classification as they appear a lot. (My Opinion)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My Question is&#xA;Removing top frequent(most occurred) words will improve the model score?&lt;/p&gt;&#xA;&lt;p&gt;How does this impact on model performance?&lt;/p&gt;&#xA;&lt;p&gt;Is it ok to remove the most occurred words?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-21T09:31:14.507" Title="Top Frequent occurrence word effect in Model Efficiency?" Tags="&lt;natural-language-processing&gt;&lt;text-classification&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20322" PostTypeId="2" ParentId="17100" CreationDate="2020-04-16T16:34:44.157" Score="0" Body="&lt;p&gt;As we have an autoassociative network prototype vectors are both input and output vectors. So , we have that :&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    T = P&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W = PP^T - QI = TP^T - QI = \sum_{q=1}^Q p_q p_q^T - QI&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying a prototype vector as input :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha = W \cdot p_k = \sum_{q=1}^Q p_k p_q p_q^T - QIp_k&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because they are orthogonal we have :&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha = p_k(p_k^T\cdot p_k) - Q \cdot I \cdot p_k = p_k (p_k^T\cdot p_k - Q \cdot I ) = p_k (\textbf{R - Q $\cdot$ I})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$R-Q\cdot I$&lt;/span&gt; is the length of vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So since ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W \cdot p_k= (R - Q \cdot I) \cdot p _k&#xA;\end{equation}&lt;/span&gt;&#xA;prototype vectors&#xA;continue to be eigenvectors of the new weight matrix .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is often the case that for auto-associative nets, the diagonal weights (those which connect an input component to the corresponding output component) are set to 0. There are papers that say this helps learning. Setting these weights to zero may improve the net's&#xA;ability to generalize or may increase the biological&#xA;plausibility of the net. In addition, it is necessary if we use iterations (iterative nets) or the delta rule is used&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2020-04-16T16:34:44.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20323" PostTypeId="2" ParentId="20321" CreationDate="2020-04-16T16:34:56.957" Score="1" Body="&lt;p&gt;The technical term for these words is &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Stop_words&quot; rel=&quot;nofollow noreferrer&quot;&gt;stop words&lt;/a&gt;&quot;. Have a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Information_retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;Information Retrieval&lt;/a&gt; and indexing (eg &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF/IDF&lt;/a&gt;) to make up your mind whether you want to remove them or not.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-16T16:34:56.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20324" PostTypeId="2" ParentId="17126" CreationDate="2020-04-16T16:41:18.440" Score="0" Body="&lt;p&gt;We have 2 classes , 1 subclass for each class&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W^2=\begin{vmatrix}&#xA;1 &amp;amp; 0\\ &#xA;0 &amp;amp; 1\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha^1=compet(n^1)=compet\begin{vmatrix}&#xA;||w_1-p_1||\\ &#xA;||w_2-p_1||\\&#xA;\end{vmatrix} = compet\begin{vmatrix}&#xA;||\begin{vmatrix}&#xA;0 &amp;amp; 1\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;1 &amp;amp; 1\\ &#xA;\end{vmatrix}^T||\\ &#xA;||\begin{vmatrix}&#xA;1 &amp;amp; 0\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;1 &amp;amp; 1\\ &#xA;\end{vmatrix}^T||\\&#xA;\end{vmatrix}=&#xA;compet(\begin{vmatrix}&#xA;1\\ &#xA;1\\&#xA;\end{vmatrix}) = \begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;   \alpha^2=W^2\cdot\alpha^1= \begin{vmatrix}&#xA;1 &amp;amp; 0\\ &#xA;0 &amp;amp; 1\\&#xA;\end{vmatrix}\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W_1(1) = W_1(0) + \alpha\cdot(p_1-W_1(0))=\begin{vmatrix}&#xA;0\\ &#xA;1\\&#xA;\end{vmatrix}+0.5\cdot(\begin{vmatrix}&#xA;1\\ &#xA;1\\&#xA;\end{vmatrix}-\begin{vmatrix}&#xA;0\\ &#xA;1\\&#xA;\end{vmatrix})=\begin{vmatrix}&#xA;0\\ &#xA;1\\&#xA;\end{vmatrix}+\begin{vmatrix}&#xA;0.5\\ &#xA;0\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;0.5\\ &#xA;1\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha^1=compet(n^1)=compet\begin{vmatrix}&#xA;||w_1-p_2||\\ &#xA;||w_2-p_2||\\&#xA;\end{vmatrix} = compet\begin{vmatrix}&#xA;||\begin{vmatrix}&#xA;0.5 &amp;amp; 1\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;1 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\ &#xA;||\begin{vmatrix}&#xA;1 &amp;amp; 0\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;1 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\&#xA;\end{vmatrix}=&#xA;compet(\begin{vmatrix}&#xA;1.8027756377\\ &#xA;2.8284271247\\&#xA;\end{vmatrix}) = \begin{vmatrix}&#xA;0\\ &#xA;1\\&#xA;\end{vmatrix})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;   \alpha^2=W^2\cdot\alpha^1= \begin{vmatrix}&#xA;1 &amp;amp; 0\\ &#xA;0 &amp;amp; 1\\&#xA;\end{vmatrix}\begin{vmatrix}&#xA;0\\ &#xA;1\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;0\\ &#xA;1\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;wrong class&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W_2(1) = W_2(0) - \alpha\cdot(p_2-W_2(0))=\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}-0.5\cdot(\begin{vmatrix}&#xA;-2\\ &#xA;2\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;2\\ &#xA;-1\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_3$&lt;/span&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha^1= compet\begin{vmatrix}&#xA;||\begin{vmatrix}&#xA;0.5 &amp;amp; 1\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;-2 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\ &#xA;||\begin{vmatrix}&#xA;2 &amp;amp; -1\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;-2 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\&#xA;\end{vmatrix}=&#xA;compet(\begin{vmatrix}&#xA;70\\ &#xA;5\\&#xA;\end{vmatrix}) = \begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;wrong class&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W_1(2) = W_1(1) - \alpha\cdot(p_3-W_1(1))=\begin{vmatrix}&#xA;0.5\\ &#xA;0\\&#xA;\end{vmatrix}-0.5\cdot(\begin{vmatrix}&#xA;-2.5\\ &#xA;1\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;1.75\\ &#xA;0.5\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha^1=compet(n^1)= compet\begin{vmatrix}&#xA;||\begin{vmatrix}&#xA;1.75 &amp;amp; 0.5\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;-1 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\ &#xA;||\begin{vmatrix}&#xA;2 &amp;amp; -1\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;-1 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\&#xA;\end{vmatrix}=&#xA;compet(\begin{vmatrix}&#xA;3.13\\ &#xA;4.24\\&#xA;\end{vmatrix}) = \begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W_1(3) = W_1(2) - \alpha\cdot(p_2-W_1(2))=\begin{vmatrix}&#xA;1.75\\ &#xA;0.5\\&#xA;\end{vmatrix}+0.5\cdot(\begin{vmatrix}&#xA;-2.75\\ &#xA;1.5\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;0.375\\ &#xA;1.25\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_3$&lt;/span&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha^1=compet(n^1)= compet\begin{vmatrix}&#xA;||\begin{vmatrix}&#xA;0.375 &amp;amp; 1.25\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;-2 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\ &#xA;||\begin{vmatrix}&#xA;2 &amp;amp; -1\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;-2 &amp;amp; 2\\ &#xA;\end{vmatrix}^T||\\&#xA;\end{vmatrix}=&#xA;compet(\begin{vmatrix}&#xA;2.95\\ &#xA;5\\&#xA;\end{vmatrix}) = \begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;wrong class&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W_1(4) = W_1(1) - \alpha\cdot(p_3-W_1(3))=\begin{vmatrix}&#xA;0.375\\ &#xA;1.25\\&#xA;\end{vmatrix}-0.5\cdot(\begin{vmatrix}&#xA;-2.375\\ &#xA;0.75\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;1.5625\ &#xA;0.875\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p_1 :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \alpha^1=compet(n^1)= compet\begin{vmatrix}&#xA;||\begin{vmatrix}&#xA;1.5625 &amp;amp; 0.875\\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;1 &amp;amp; 1\\ &#xA;\end{vmatrix}^T||\\ &#xA;||\begin{vmatrix}&#xA;2 &amp;amp; -1\ &#xA;\end{vmatrix}^T-\begin{vmatrix}&#xA;1 &amp;amp; 1\\ &#xA;\end{vmatrix}^T||\\&#xA;\end{vmatrix}=&#xA;compet(\begin{vmatrix}&#xA;0.57\\ &#xA;2.23\\&#xA;\end{vmatrix}) = \begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix})&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}&#xA;1\\ &#xA;0\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    W_1(5) = W_1(4) - \alpha\cdot(p_1-W_1(4))=\begin{vmatrix}&#xA;1.5625\\ &#xA;0.875\\&#xA;\end{vmatrix}+(\begin{vmatrix}&#xA;-0.28125\\ &#xA;0.0625\\&#xA;\end{vmatrix}=\begin{vmatrix}&#xA;1.28125\ &#xA;0.9375\\&#xA;\end{vmatrix}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2020-04-16T16:41:18.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20325" PostTypeId="1" CreationDate="2020-04-16T17:07:27.367" Score="1" ViewCount="237" Body="&lt;p&gt;All of the attention in the AI field over the past few years has been toward statistical AI, now that Machine Learning has been validated on hard problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I want to learn more about classical AI (also known as &quot;symbolic AI&quot; and &quot;GOFAI--good old fashioned AI&quot;, and &quot;expert systems&quot;.) &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are some examples of classical AI methods? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please provide a brief description for each method listed.&lt;/p&gt;&#xA;" OwnerUserId="36111" LastEditorUserId="1671" LastEditDate="2020-04-23T21:34:49.440" LastActivityDate="2020-04-23T21:42:49.710" Title="What are some examples of classical AI methods?" Tags="&lt;theory&gt;&lt;history&gt;&lt;symbolic-ai&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20326" PostTypeId="2" ParentId="20305" CreationDate="2020-04-16T17:14:23.620" Score="0" Body="&lt;p&gt;In the first case, the VC dimension of &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; being infinite implies that &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is not (agnostic) PAC learnable (&lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf#page=48&quot; rel=&quot;nofollow noreferrer&quot;&gt;see p. 48 of Understanding Machine Learning: From Theory to Algorithms&lt;/a&gt;). So, in general, your classifier is not guaranteed to succeed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second case, your division of &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; implies that &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is nonuniformly learnable (&lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf#page=83&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 7 of the cited book&lt;/a&gt;). This implies that you can get a generalization bound by using structural risk minimization.&lt;/p&gt;&#xA;" OwnerUserId="36115" LastEditorUserId="2444" LastEditDate="2020-04-16T17:48:34.203" LastActivityDate="2020-04-16T17:48:34.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20327" PostTypeId="2" ParentId="20309" CreationDate="2020-04-16T17:19:43.513" Score="6" Body="&lt;p&gt;First of all, &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithmic_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;efficiency&lt;/a&gt; and convergence are two different things. There's also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rate_of_convergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;rate of convergence&lt;/a&gt;, so an algorithm may converge faster than another, so, in this sense, it may be more efficient. I will focus on the proof that policy evaluation (PE) converges. If you want to know about its efficiency, maybe ask another question, but the proof below also tells you about the rate of convergence of PE.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What is the proof that policy evaluation converges to the optimal solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To provide some context, I will briefly describe policy evaluation and what you need to know to understand the proof.&lt;/p&gt;&#xA;&lt;h2&gt;Policy evaluation&lt;/h2&gt;&#xA;&lt;p&gt;Policy evaluation (PE) is an iterative numerical algorithm to find the value function &lt;span class=&quot;math-container&quot;&gt;$v^\pi$&lt;/span&gt; for a given (and arbitrary) policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. This problem is often called the &lt;em&gt;prediction problem&lt;/em&gt; (i.e. you want to predict the rewards you will get if you behave in a certain way).&lt;/p&gt;&#xA;&lt;h3&gt;Two versions: synchronous and asynchronous&lt;/h3&gt;&#xA;&lt;p&gt;There are (at least) two versions of policy evaluation: a synchronous one and an asynchronous one.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;strong&gt;synchronous&lt;/strong&gt; version (SPE), you maintain two arrays for the values of the states: one array holds the current values of the states and the other array will contain the next values of the states, so two arrays are used in order to be able to update the value of each state at the same time.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;strong&gt;asynchronous&lt;/strong&gt; version (APE), you update the value of each state in place. So, first, you update the value of e.g. &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt;, etc., by changing your only array of values (so you do not require a second array).&lt;/p&gt;&#xA;&lt;p&gt;SPE is similar in style to the numerical method called &lt;a href=&quot;https://en.wikipedia.org/wiki/Jacobi_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jacobi method&lt;/a&gt;, which is a general iterative method for finding a solution to a system of linear equations (which is exactly what PE is actually doing, and this is also explained in the cited book by Sutton and Barto). Similarly, APE is similar in style to &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Gauss–Seidel method&lt;/a&gt;, which is another method to solve a system of linear equations.&lt;/p&gt;&#xA;&lt;p&gt;Both of these general numerical methods to solve a system of linear equations are studied in detail in &lt;a href=&quot;https://labs.xjtudlc.com/labs/wldmt/reading%20list/books/Distributed%20and%20parallel%20algorithms/Parallel%20and%20Distributed%20Computation%20Numerical%20Methods.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parallel and Distributed Computation Numerical Methods&lt;/a&gt; (1989) by Bertsekas and Tsitsiklis, which I haven't read yet, but provides convergence results for these numerical methods.&lt;/p&gt;&#xA;&lt;p&gt;The book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning: an introduction&lt;/a&gt; by Sutton and Barto provides a more detailed description of policy evaluation (PE).&lt;/p&gt;&#xA;&lt;h2&gt;Proof of convergence&lt;/h2&gt;&#xA;&lt;p&gt;I will provide a proof for the SPE based on &lt;a href=&quot;http://www.andrew.cmu.edu/course/10-703/slides/lecture4_valuePolicyDP-9-10-2018.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides&lt;/a&gt; by &lt;a href=&quot;http://www.cs.cmu.edu/%7Etom/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tom Mitchell&lt;/a&gt;. Before proceeding, I suggest you read the following question  &lt;a href=&quot;https://ai.stackexchange.com/q/11057/2444&quot;&gt;What is the Bellman operator in reinforcement learning?&lt;/a&gt; and its answer, and you should also get familiar with vector spaces, norms, fixed points and maybe contraction mappings.&lt;/p&gt;&#xA;&lt;p&gt;The proof that PE finds a unique fixed-point is based on the &lt;strong&gt;contraction mapping theorem&lt;/strong&gt; and on the concept of &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-contractions&lt;/strong&gt;, so let me first recall these definitions.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Definition (&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-contraction)&lt;/strong&gt;: An operator on a normed vector space &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-contraction, for &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \gamma &amp;lt; 1$&lt;/span&gt;, provided for all  &lt;span class=&quot;math-container&quot;&gt;$x, y \in \mathcal{X}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\| F(x) - F(y) \| \leq \gamma \| x - y\|$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Contraction mapping theorem&lt;/strong&gt;: For a &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-contraction &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; in a complete normed vector space &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Iterative application of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; converges to a &lt;strong&gt;unique&lt;/strong&gt; fixed point in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; independently of the starting point&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;at a linear convergence rate determined by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now, consider the vector space &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt; over state-value functions &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$v \in \mathcal{V})$&lt;/span&gt;. So, each point in this space fully specifies a value function &lt;span class=&quot;math-container&quot;&gt;$v : \mathcal{S} \rightarrow \mathbb{R}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; is the state space of the MDP).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Theorem (convergence of PE)&lt;/strong&gt;: The &lt;a href=&quot;https://ai.stackexchange.com/q/11057/2444&quot;&gt;Bellman operator&lt;/a&gt; is a &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-contraction operator, so an iterative application of it converges to a unique fixed-point in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt;. Given that PE is an iterative application of the Bellman operator (see &lt;a href=&quot;https://ai.stackexchange.com/q/11057/2444&quot;&gt;What is the Bellman operator in reinforcement learning?&lt;/a&gt;), PE finds this unique fixed-point solution.&lt;/p&gt;&#xA;&lt;p&gt;So, we just need to show that the Bellman operator is a &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;-contraction operator in order to show that PE finds this unique fixed-point solution.&lt;/p&gt;&#xA;&lt;h3&gt;Proof&lt;/h3&gt;&#xA;&lt;p&gt;We will measure the distance between state-value functions &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; by the &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;-norm, i.e. the largest difference between state values:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\|u - v\|_{\infty} = \operatorname{max}_{s \in \mathcal{S}} |u(s) - v(s)|$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Definition (Bellman operator)&lt;/strong&gt;: We define the Bellman expectation operator as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$F^\pi(v) = \mathbf{r}^\pi + \gamma \mathbf{T}^\pi v$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$v \in \mathcal{V}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{r}^\pi$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}|$&lt;/span&gt;-dimensional vector whose &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th entry gives &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E} \left[ r \mid s_j, a=\pi(s_j) \right]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{T}^\pi$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}| \times |\mathcal{S}|$&lt;/span&gt; matrix whose &lt;span class=&quot;math-container&quot;&gt;$(j, k)$&lt;/span&gt; entry gives &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(s_k \mid s_j, a=\pi(s_j))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, let's measure the distance (with the &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;-norm defined above) between any two value functions &lt;span class=&quot;math-container&quot;&gt;$u \in \mathcal{V}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v \in \mathcal{V}$&lt;/span&gt; after the application of the Bellman operator &lt;span class=&quot;math-container&quot;&gt;$F^\pi$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\| F^\pi(u) - F^\pi(v) \|_{\infty} &#xA;&amp;amp;= \| (\mathbf{r}^\pi + \gamma \mathbf{T}^\pi u) - (\mathbf{r}^\pi + \gamma \mathbf{T}^\pi v)\|_{\infty} \\&#xA;&amp;amp;= \| \gamma \mathbf{T}^\pi (u - v)\|_{\infty} \\&#xA;&amp;amp;\leq \| \gamma \mathbf{T}^\pi ( \mathbb{1} \cdot \| u - v \|_{\infty})\|_{\infty} \\&#xA;&amp;amp;\leq \| \gamma (\mathbf{T}^\pi \mathbb{1}) \cdot \| u - v \|_{\infty}\|_{\infty} \\&#xA;&amp;amp;\leq \gamma \| u - v \|_{\infty}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1} = [1, \dots, 1]^T$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{T}^\pi \cdot \mathbb{1} = \mathbb{1}$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$\mathbf{T}^\pi$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic matrix&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;By the Bellman expectation equation (see Barto and Sutton's book and &lt;a href=&quot;https://ai.stackexchange.com/q/11057/2444&quot;&gt;What is the Bellman operator in reinforcement learning?&lt;/a&gt;), &lt;span class=&quot;math-container&quot;&gt;$v^\pi$&lt;/span&gt; is &lt;strong&gt;a&lt;/strong&gt; fixed-point of the Bellman operator &lt;span class=&quot;math-container&quot;&gt;$F^\pi$&lt;/span&gt;. Given the contraction mapping theorem, the iterative application of &lt;span class=&quot;math-container&quot;&gt;$F^\pi$&lt;/span&gt; produces a &lt;strong&gt;unique&lt;/strong&gt; solution, so &lt;span class=&quot;math-container&quot;&gt;$v^\pi$&lt;/span&gt; must be this unique solution, i.e. SPE finds &lt;span class=&quot;math-container&quot;&gt;$v^\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Notes&lt;/h2&gt;&#xA;&lt;p&gt;I didn't prove the contraction mapping theorem, but you can find more info about the theorem and its proof in &lt;a href=&quot;https://en.wikipedia.org/wiki/Banach_fixed-point_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-17T03:10:42.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20328" PostTypeId="2" ParentId="20325" CreationDate="2020-04-16T17:36:35.817" Score="2" Body="&lt;p&gt;If you're looking for AI systems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Deep Blue&lt;/li&gt;&#xA;&lt;li&gt;Eliza&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you're looking for specific algorithms:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Bayesian Networks&lt;/li&gt;&#xA;&lt;li&gt;Decision Trees&lt;/li&gt;&#xA;&lt;li&gt;Any search algorithm&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32835" LastActivityDate="2020-04-16T17:36:35.817" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20329" PostTypeId="2" ParentId="20316" CreationDate="2020-04-16T17:57:39.457" Score="2" Body="&lt;p&gt;It seems to me that you're thinking about the parameters &lt;em&gt;a&lt;/em&gt; and &lt;em&gt;b&lt;/em&gt; as being characteristic of the agent that's moving in the environment (therefore determining the final policy), but they are actually a characteristic of the environment.&lt;/p&gt;&#xA;&lt;p&gt;Think of a frozen lake. You want to pass the lake but there is a hole five meters in front of you. Let's say you have boots with a rubber sole, so there is no risk to slip while walking (i.e. all transition probabilities =1). What is the optimal policy? Simply moving forward till the hole, goes around it and then forward again. But what if you were wearing wooden clogs instead? Now if you walk forward there is the risk to not be able to stop when you want to (i.e. when you decide to move forward of one step you might fail and keep going forward for several steps, with probability &lt;em&gt;a&lt;/em&gt;). How many steps would you wait before moving to the left or to the right to avoid risking to fall into the hole? The optimal policy is obviously different in this second scenario. Think about it, I will not answer in detail about how the parameters affect the final policy because stack exchange is not meant for homework.&lt;/p&gt;&#xA;&lt;p&gt;Also, an arbitrary policy is necessary simply because in value iteration you first update the values and only when they converge you infer from then the optimal policy. So you need some arbitrary prior belief about the environment to start update the values.&lt;/p&gt;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&lt;p&gt;So here's a quick visualisation of the impact of different transition probabilities (the parameters &lt;em&gt;a&lt;/em&gt; and &lt;em&gt;b&lt;/em&gt; in your assignment) on the optimal policy. I used a different grid to make it easier and more explicit (i.e. you don't even need to calculate the values to see the optimal policy).&lt;/p&gt;&#xA;&lt;p&gt;As you can see in the picture below, there are some terminal states in the third column and a goal state which gives a huge reward on top of it. From the starting point on the bottom we want to reach the goal state.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CswSv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CswSv.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We have two cases:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Non stochastic environment&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;On the left we have the case in which all transition probabilities are 1. In this case the optimal policy is simply going forward till the second row, where the goal state is located, and then move right or left to reach it. The terminal states are not scary at all, in this case we could consider them as walls that we can't cross, but the optimal policy do not reflect any risk of falling in them. This is precisely because the transition probabilities are all 1.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Stochastic environment&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;On the right we have a different situation. In this case only 3 action have a transition probability of 1: going south, east or west. The transition probability for nord is 0.2 meaning that 80% of the time we will slip and go east instead (I wrote right in the picture meaning east, my bad). What is the impact of this simple change in the environment?&lt;/p&gt;&#xA;&lt;p&gt;This time the second column become really dangerous, because if we try to go nord and fail we will end on a terminal state. On the other hand, if we choose another action we will end on a legitimate state, without any risk because the other transition probabilities are 1! Therefore the agent learn that on the left side of the grid the best strategy is to go down and circumnavigate the terminal states from the right, because only in the fourth column we can move north without risk, in fact in those cells even if we try go north and fail we would just hit a wall and we could try again going north the next turn till we will eventually succeed.&lt;/p&gt;&#xA;&lt;p&gt;Hope this make sense!&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-22T16:23:14.560" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="20330" PostTypeId="1" CreationDate="2020-04-16T18:48:11.513" Score="1" ViewCount="35" Body="&lt;p&gt;Assume I am given a binary neural network where the activation values are constrained to be 0 or 1 (by clipping the ReLU function). Additionally, assume the neural network is supposed to work in a noisy environment where some of the activation values may be randomly flipped, i.e. 0 -&gt; 1 or 1 -&gt; 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to train such neural network in a way that it would be resilient to the noisy environment. I assume training with dropout would make the neural network somewhat resilient to noises where one is flipped to zero (1 -&gt; 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some ways that allow me to make the neural network resilient to the other kind of noise which flips zeros to ones (0 -&gt; 1)? Is it theoretically valid to introduce a dropout-like algorithm which flips some zeros to ones during training but does not backpropagate the gradients through those flipped nodes?&lt;/p&gt;&#xA;" OwnerUserId="27048" LastEditorUserId="27048" LastEditDate="2020-04-17T19:23:39.580" LastActivityDate="2020-04-17T19:23:39.580" Title="How to make binary neural networks resilient to flipped activation values?" Tags="&lt;training&gt;&lt;dropout&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20331" PostTypeId="5" CreationDate="2020-04-16T19:20:26.487" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-16T19:20:26.487" LastActivityDate="2020-04-16T19:20:26.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20332" PostTypeId="4" CreationDate="2020-04-16T19:20:26.487" Score="0" Body="For questions related to the various policy evaluation (PE) algorithms, which are numerical iterative algorithms that are used to find the value function associated with a given policy, which is often denoted as the &quot;prediction problem&quot;. PE is also considered a dynamic programming method, which is regularly discussed in reinforcement learning textbooks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-16T19:26:11.923" LastActivityDate="2020-04-16T19:26:11.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20334" PostTypeId="2" ParentId="20238" CreationDate="2020-04-16T21:50:19.127" Score="2" Body="&lt;p&gt;MSE just measures the squared difference between actual and target values. It can still correctly classify the values, but perhaps not with the same confidence - leading to a higher loss (e.g. an output of 0.77 vs 0.98 when the target is 1). In terms of which is better, I wouldn't know without the specifics of your problem. It is possible the higher loss could be more robust since it is less likely to have overfitted on the data, yet achieves the same accuracy.&lt;/p&gt;&#xA;" OwnerUserId="22373" LastActivityDate="2020-04-16T21:50:19.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20335" PostTypeId="2" ParentId="20238" CreationDate="2020-04-16T22:15:32.337" Score="2" Body="&lt;p&gt;Accuracy itself isn't a sufficient way to compare two models. For example, you need to consider the precision and recall stats (see confusion matrix) and calculate some other metrics like f1 score. The measurement of accuracy is only the initial state that helps us to know if a model is &quot;working&quot;. But in order to understand and compare you need to know how many of impostors' set were classified as true claimants, and how many true-claimants' set classified as impostors according to the sum of total correct classified ones. In order to make a decision with the above been known you have to define how critical a miss-classification could be? e.g. assume that you need to classify if a person has a disease or not? (that's critical).&lt;/p&gt;&#xA;" OwnerUserId="22592" LastEditorUserId="22592" LastEditDate="2020-04-16T22:34:44.087" LastActivityDate="2020-04-16T22:34:44.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20336" PostTypeId="1" AcceptedAnswerId="20341" CreationDate="2020-04-16T22:33:56.440" Score="5" ViewCount="954" Body="&lt;p&gt;This might be a little broad question, but I have been watching Caltech youtube videos on Machine Learning, and in &lt;a href=&quot;https://www.youtube.com/watch?v=Dc0sr0kdBVI&amp;amp;list=PLD63A284B7615313A&amp;amp;index=7&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; video prof. is trying to explain how we should interpret the VC dimension in terms of what it means in layman terms, and why do we need it in practice.&lt;/p&gt;&#xA;&lt;p&gt;The first part I think I understand, please correct me if I am wrong. VC Dimension dictates the number of effective parameters (i.e. degrees of freedom) that model has. In other words, the number of parameters the model needs in order to cover all possible label combinations for the chosen dataset. Now, the second part is not clear to me. The professor is trying to answer the question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How does knowing the VC dimension of the hypothesis class affect number of samples we need for training?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Again, I apologize if all of this may be trivial, but I am new to the field and wish to learn as much as I can, so I can implement better and more efficient programs in practice.&lt;/p&gt;&#xA;" OwnerUserId="35990" LastEditorUserId="2444" LastEditDate="2020-12-07T21:01:11.160" LastActivityDate="2020-12-07T22:06:37.550" Title="How does size of the dataset depend on VC dimension of the hypothesis class?" Tags="&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;&lt;sample-complexity&gt;&lt;hypothesis-class&gt;" AnswerCount="4" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20337" PostTypeId="2" ParentId="20336" CreationDate="2020-04-16T22:43:53.213" Score="3" Body="&lt;p&gt;Given a hypothesis set &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;, the set of all possible mappings from &lt;span class=&quot;math-container&quot;&gt;$X\to Y$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is our input space and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; are our binary mappings: &lt;span class=&quot;math-container&quot;&gt;$\{-1,1\}$&lt;/span&gt;, the growth function, &lt;span class=&quot;math-container&quot;&gt;$\Pi_H(m)$&lt;/span&gt;, is defined as the maximum number of dichotomies generated by &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; points. Here a dichotomy is the set of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; points in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; that represent a hypothesis. A hypothesis is just a way we classify our points. Therefore with two labels we know,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Pi_H(m)\leq 2^m$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just counts every possible hypothesis. The VC dimension is then the largest &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\Pi_H(m)=2^m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a 2D perceptron, meaning our &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; and our classifying hyperlane is one-dimensional: a line. The VC dimension will be 3. This is because we can shatter (correctly classify) all dichotomies for &lt;span class=&quot;math-container&quot;&gt;$m=3$&lt;/span&gt;. We can either have all points be the same colour, or one point be a different colour - which is &lt;span class=&quot;math-container&quot;&gt;$2^3=8$&lt;/span&gt; dichotomies. You may ask what if the points we are trying to classify are collinear. This does not matter because we are concerned with resolving the dichotomies themselves, not the location of the points. We just need a set of points (wherever they may be located) that exhibits that dichotomy. In other words, we can pick the points such that they maximize the number of dichotomies we can shatter with one classifying hyperplane (a triangle): the VC dimension is a statement of the capacity of our model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this clear, consider &lt;span class=&quot;math-container&quot;&gt;$m=4$&lt;/span&gt;. We can represent the truth table of the XOR gate as a dichotomy but this is not resolvable by the perceptron, no matter where we choose the location of the points (not linearly separable). Therefore, we can resolve a maximum of 8 dichotomies, so our VC dimension is 3. In general, the VC dimension of perceptrons is &lt;span class=&quot;math-container&quot;&gt;$d+1$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the dimension of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d-1$&lt;/span&gt; is the dimension of the classifying hyperplane. &lt;/p&gt;&#xA;" OwnerUserId="22373" LastEditorUserId="22373" LastEditDate="2020-04-17T00:49:14.967" LastActivityDate="2020-04-17T00:49:14.967" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20338" PostTypeId="2" ParentId="20266" CreationDate="2020-04-16T23:10:49.623" Score="1" Body="&lt;p&gt;I think your approach to tackle this as an imbalanced problem is correct. The easiest thing you could do is to add weights to the samples, during training, so that the model &quot;pays more attention&quot; to the under-represented class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also a couple of other ways for this to be done: oversampling and undersampling, but initially, I'd focus on adding weights, since its easier to implement.  &lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-04-16T23:10:49.623" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20339" PostTypeId="1" CreationDate="2020-04-16T23:19:08.173" Score="1" ViewCount="38" Body="&lt;p&gt;I am creating a neural network to experiment with, and I was wondering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I have weights randomly initialized to be either 1 or 0 for each neuron, and then I made it so that the weights cannot be changed, would that ruin the neural network? What would happen?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note: There is no bias in this network.&lt;/p&gt;&#xA;" OwnerUserId="36124" LastEditorUserId="1671" LastEditDate="2020-04-17T21:21:08.030" LastActivityDate="2020-04-17T21:21:08.030" Title="Is having binary randomized unchanging neural network weights a good idea?" Tags="&lt;neural-networks&gt;&lt;weights&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20340" PostTypeId="2" ParentId="20336" CreationDate="2020-04-17T00:11:11.417" Score="2" Body="&lt;p&gt;The VC dimension represents the capacity (&lt;a href=&quot;https://www.math.arizona.edu/~hzhang/math574m/Read/vapnik.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the same Vapnik, the letter V from VC, calls it the &quot;capacity&quot;&lt;/a&gt;) of a model (or, in general, hypotheses class), so a model with a higher VC dimension has more capacity (i.e. it can represent more functions) than a model with a lower VC dimension. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The VC dimension is typically used to provide theoretical bounds e.g. on the number of samples required for a model to achieve a certain &lt;em&gt;test error&lt;/em&gt; with a given &lt;em&gt;uncertainty&lt;/em&gt; or, similarly, to understand the quality of your estimation given a certain dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to give you an idea of how the bounds look like, have a look at the theorem on page 6 (of the pdf) of the paper &lt;a href=&quot;https://www.math.arizona.edu/~hzhang/math574m/Read/vapnik.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An overview of statistical learning theory&lt;/a&gt; (1999) by Vapnik.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have also a look at &lt;a href=&quot;https://ai.stackexchange.com/a/17881/2444&quot;&gt;this answer&lt;/a&gt;, where I provide more info about the VC dimension, in particular, in the context of neural networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-17T00:20:23.253" LastActivityDate="2020-04-17T00:20:23.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20341" PostTypeId="2" ParentId="20336" CreationDate="2020-04-17T00:26:45.523" Score="3" Body="&lt;p&gt;From &lt;a href=&quot;https://www.springer.com/gp/book/9780387987804&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt; we know that we have the following bound between the test and train error for i.i.d samples:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{P}\left(R \leqslant R_{emp} + \sqrt{\frac{d\left(\log{\left(\frac{2m}{d}\right)}+1\right)-\log{\left(\frac{\eta}{4}\right)}}{m}}\right) \geqslant 1-\eta&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the test error, &lt;span class=&quot;math-container&quot;&gt;$R_{emp}$&lt;/span&gt; is the training error, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the size of the training dataset, and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the hypothesis class's VC dimension. As you can see, the training and test errors have some relations to the dataset's size (&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, in terms of PAC learnability, we want to find a (lower or upper) bound for &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; such that the absolute difference between &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_{emp}$&lt;/span&gt; will be less than a given &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; with a given probability of at least &lt;span class=&quot;math-container&quot;&gt;$1-\eta$&lt;/span&gt;.  Hence, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; can be computed in terms of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;. For example, it can be proved (&lt;a href=&quot;https://dl.acm.org/doi/10.5555/2946645.2946683&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;) to train a binary classifier with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; difference between test and train error with the probability of at least &lt;span class=&quot;math-container&quot;&gt;$1-\eta$&lt;/span&gt;, we need &lt;span class=&quot;math-container&quot;&gt;$O\left(\frac{d + \log\frac{1}{\eta}}{\epsilon} \right)$&lt;/span&gt; i.i.d sample data, i.e., &lt;span class=&quot;math-container&quot;&gt;$m = O\left(\frac{d + \log\frac{1}{\eta}}{\epsilon}\right)$&lt;/span&gt;. See more example and references &lt;a href=&quot;https://en.wikipedia.org/wiki/Sample_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-12-07T22:06:37.550" LastActivityDate="2020-12-07T22:06:37.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20342" PostTypeId="1" CreationDate="2020-04-17T00:58:31.843" Score="1" ViewCount="47" Body="&lt;p&gt;To whomever can help out, I appreciate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently attempting to detect a signal from background noise.  The signal is pretty well known but the background has a lotttt of variability.  I've since come to know this problem as Open Set Recognition.  Another complicating factor is that the signal mixes with the background noise (think equivalent to a transparent piece of glass in-front of scenery for a picture, or picking out the sound of a pin drop in an office space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I started this project, it seemed like the current state of the art in this space was generating Spectrograms and feeding them to a CNN and this is the path I've followed.  I'm at a place where I think I've overcome most of the initial problems you might encounter but I'm still not getting good enough results for a project solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the overall steps I've gone through:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate 17000 ground truth &quot;signals&quot; and 17000 backgrounds (negatives or other classes depending on what nn scheme I'm training)&lt;/li&gt;&#xA;&lt;li&gt;Generate separate test samples (not training samples but external model validation samples: &quot;blind test&quot;) where I take the backgrounds and randomly overlay the signal into it at various intensities.&lt;/li&gt;&#xA;&lt;li&gt;My first attempt was with a pre-built library training solution (ImageAI) with resnet50 base model.  This solution is a multiclass classifier so I had 400 each of the signal + 5 other classes that were the background. It did not work well at classifying the signal. I don't think I ever got this off the ground for two reasons a) My spectrogram pictures were not optimised (waay to large) and b) I couldn't adjust the image input shape via the library. It mostly just ended up classifying one background class.&lt;/li&gt;&#xA;&lt;li&gt;I then started building my own neural nets.  The first reason to make sure my spectrogram input shape was matched in the input shape of the CNN.  The second reason was to test various neural net schemes to see what worked best.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The first net I built was a simple feed forward net with a couple of dense layers.  This trains to .9998 val_acc.  It (like the rest of what I try) produces poor results on my blind tests, in the range of 60% true positive.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;def build(width, height, depth, classes):&#xA;# initialize the model along with the input shape to be&#xA;# &quot;channels last&quot; and the channels dimension itself&#xA;model = Sequential()&#xA;inputShape = (height, width, depth)&#xA;chanDim = -1&#xA;&#xA;# if we are using &quot;channels first&quot;, update the input shape&#xA;# and channels dimension&#xA;if K.image_data_format() == &quot;channels_first&quot;:&#xA;    inputShape = (depth, height, width)&#xA;    chanDim = 1&#xA;model.add(Flatten())&#xA;model.add(Dense(512, input_shape=(inputShape), activation=&quot;relu&quot;))&#xA;model.add(Dense(128, activation=&quot;relu&quot;))&#xA;model.add(Dense(32, activation=&quot;relu&quot;))&#xA;# sigmoid classifier&#xA;model.add(Dense(classes))&#xA;model.add(Activation(&quot;sigmoid&quot;))&#xA;&#xA;# return the constructed network architecture&#xA;return model'''&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I then try a &quot;VGG Light&quot; model.  Again, trains to .9999 but gives me only 62% true positive results on my blind tests&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;def build(width, height, depth, classes):&#xA;# initialize the model along with the input shape to be&#xA;# &quot;channels last&quot; and the channels dimension itself&#xA;model = Sequential()&#xA;inputShape = (height, width, depth)&#xA;chanDim = -1&#xA;&#xA;# if we are using &quot;channels first&quot;, update the input shape&#xA;# and channels dimension&#xA;if K.image_data_format() == &quot;channels_first&quot;:&#xA;    inputShape = (depth, height, width)&#xA;    chanDim = 1&#xA;&#xA;# CONV =&amp;gt; RELU =&amp;gt; POOL&#xA;model.add(Conv2D(32, (3, 3), padding=&quot;same&quot;,&#xA;    input_shape=inputShape))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(3, 3)))&#xA;model.add(Dropout(0.25))&#xA;&#xA;# (CONV =&amp;gt; RELU) * 2 =&amp;gt; POOL&#xA;model.add(Conv2D(64, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(Conv2D(64, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;model.add(Dropout(0.25))&#xA;&#xA;# (CONV =&amp;gt; RELU) * 2 =&amp;gt; POOL&#xA;model.add(Conv2D(128, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(Conv2D(128, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;model.add(Dropout(0.25))&#xA;model.add(GaussianNoise(.05))&#xA;&#xA;# first (and only) set of FC =&amp;gt; RELU layers&#xA;model.add(Flatten())&#xA;model.add(Dense(1024))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(512))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Dropout(.5))&#xA;model.add(Dense(128))       &#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization())     &#xA;model.add(GaussianDropout(0.5))&#xA;&#xA;# sigmoid classifier&#xA;model.add(Dense(classes))&#xA;model.add(Activation(&quot;sigmoid&quot;))&#xA;&#xA;# return the constructed network architecture&#xA;return model'''&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I then try a &quot;full VGG&quot; net.  This again trains to .9999 but only a blind test true positive result of 63%.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;def build(width, height, depth, classes):&#xA;# initialize the model along with the input shape to be&#xA;# &quot;channels last&quot; and the channels dimension itself&#xA;model = Sequential()&#xA;inputShape = (height, width, depth)&#xA;chanDim = -1&#xA;&#xA;# if we are using &quot;channels first&quot;, update the input shape&#xA;# and channels dimension&#xA;if K.image_data_format() == &quot;channels_first&quot;:&#xA;    inputShape = (depth, height, width)&#xA;    chanDim = 1&#xA;&#xA;#CONV =&amp;gt; RELU =&amp;gt; POOL&#xA;model.add(Conv2D(64, (3, 3), padding=&quot;same&quot;,&#xA;    input_shape=inputShape))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(3, 3)))&#xA;#model.add(Dropout(0.25))&#xA;&#xA;# (CONV =&amp;gt; RELU) * 2 =&amp;gt; POOL&#xA;model.add(Conv2D(128, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(Conv2D(128, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;#model.add(Dropout(0.25))&#xA;&#xA;# (CONV =&amp;gt; RELU) * 2 =&amp;gt; POOL&#xA;model.add(Conv2D(256, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(Conv2D(256, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;#model.add(Dropout(0.25))&#xA;&#xA;# (CONV =&amp;gt; RELU) * 2 =&amp;gt; POOL&#xA;model.add(Conv2D(512, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(Conv2D(512, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;#model.add(Dropout(0.25))&#xA;&#xA;# (CONV =&amp;gt; RELU) * 2 =&amp;gt; POOL&#xA;model.add(Conv2D(1024, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(Conv2D(1024, (3, 3), padding=&quot;same&quot;))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization(axis=chanDim))&#xA;model.add(MaxPooling2D(pool_size=(2, 2)))&#xA;#model.add(Dropout(0.25))&#xA;model.add(GaussianNoise(.1))&#xA;&#xA;# first (and only) set of FC =&amp;gt; RELU layers&#xA;model.add(Flatten())&#xA;model.add(Dense(8192))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(4096))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Dropout(0.5))&#xA;model.add(Dense(1024))&#xA;model.add(Activation(&quot;relu&quot;))&#xA;model.add(BatchNormalization()) &#xA;model.add(GaussianDropout(0.5))&#xA;&#xA;# sigmoid classifier&#xA;model.add(Dense(classes))&#xA;model.add(Activation(&quot;sigmoid&quot;))&#xA;&#xA;# return the constructed network architecture&#xA;return model '''&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All of the above are binary_crossentropy trained in keras.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've tried multi-class with these models as well but when testing them on the blind test they usually pick the background rather than the signal.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've also messed around with Autoencoders to try and get the encoder to rebuild the signal well and then compare to known results but haven't been successful yet though I'd be willing to give it another try if everyone thought that might produce better results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the beginning I ran into unbalanced classification problems (I was noob) but under all the models shown above the classes all have the same number of samples.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm at the point where the larger VGG models trained on 34,000 samples is taking days and I don't see any better results than a basic, feed forward NN that takes 4 minutes to train.  Does anyone see the path forward here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mecho&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. sorry for the formatting flubs, not sure how to correct them.&lt;/p&gt;&#xA;" OwnerUserId="36092" LastActivityDate="2021-05-22T03:06:00.557" Title="Heavily mixing signal differentiation from Open Set of backgrounds via CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20343" PostTypeId="1" CreationDate="2020-04-17T03:25:47.990" Score="1" ViewCount="20" Body="&lt;p&gt;Listening to lectures, convolutional neural network seems to be an improvement over a simple neural network, where for example, you take every pixel in the image, flatten it to a vector, and feed it to ANN with a couple layers. Therefore semantic segmentation should be possible to perform on a classic ANN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dont understand exactly how a CNN can classify each pixel in the image to do semantic segmentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way semantic segmentation is explained in lectures is the output of the CNN is fed into it backwards, is that the same as with GAN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the output of a CNN is a value between 0 and 1 for each class, how exactly can those values be fed back through a CNN backwards to classify each pixel in the image? Through back propagation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that it should be possible to do the same to a regular ANN described above. &#xA;Can someone explain why or why not it wouldn't be possible, and how semantic segmentation feeds output back through network to classify each pixel&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="36127" LastActivityDate="2020-04-17T03:25:47.990" Title="How would semantic segmentation work with a non convolutional neural network" Tags="&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20344" PostTypeId="2" ParentId="7640" CreationDate="2020-04-17T06:14:27.103" Score="2" Body="&lt;p&gt;A stationary policy is the one that does not depend on time. Meaning that the agent will take the same decision whenever certain conditions are met. This stationary policy may be probabilistic which implies that the probability of choosing an action remains the same. It may take different decisions but the probability remains the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Stationary environment refers to the static model of the system. The model comprises of a Reward function and Transition probabilities. So, in a stationary environment, the reward function and transition probabilities remain constant or the changes are slow enough that the agent finds enough training time to learn the changes done in the environment. &lt;/p&gt;&#xA;" OwnerUserId="36133" LastActivityDate="2020-04-17T06:14:27.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20347" PostTypeId="1" CreationDate="2020-04-17T09:22:52.857" Score="1" ViewCount="37" Body="&lt;p&gt;Is the paper &quot;&lt;a href=&quot;https://www.cs.toronto.edu/~hinton/science.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reducing the Dimensionality of Data with Neural Networks&lt;/a&gt;&quot; by G. Hinton and R. Salakhutdinov relevant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the deep learning textbook by Goodfellow, Bengio &amp;amp; Courville (2016)&lt;/a&gt; doesn't cite that paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that indicate that paper is not as important as others to Deep learning? If yes, I would skip this one to accelerate my process of learning.&lt;/p&gt;&#xA;" OwnerUserId="35896" LastEditorUserId="2444" LastEditDate="2020-04-17T14:14:24.833" LastActivityDate="2020-04-17T14:14:24.833" Title="Is the paper &quot;Reducing the Dimensionality of Data with Neural Networks&quot; by Hinton relevant?" Tags="&lt;research&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20348" PostTypeId="2" ParentId="20347" CreationDate="2020-04-17T09:28:50.047" Score="1" Body="&lt;p&gt;No; there are too many publications around for anybody to keep track of everything, so unless it is a seminal paper, you cannot draw any conclusions from this. They could simply have missed it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially if it is a textbook for beginners, more advanced papers are often not mentioned, as they might be too complex to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you have to decide for yourself if that paper is relevant to you. To me it sounds like a specific application of neural networks; if the dimensionality of the input data is an issue for you, it might be, otherwise probably not. &lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-17T09:28:50.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20349" PostTypeId="2" ParentId="20336" CreationDate="2020-04-17T10:39:14.370" Score="0" Body="&lt;p&gt;Since the mathematical details have already been covered by other answers, I will try to provide an intuitive explanation. I will answer this assuming the question meant &lt;span class=&quot;math-container&quot;&gt;$model$&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$learning$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$algorithm$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to think of &lt;span class=&quot;math-container&quot;&gt;$\mathcal V \mathcal C$&lt;/span&gt; dimension is that it is an indicator of the number of functions (i.e a set of functions) you can choose from to approximate your classification task over a domain. So a model (here assume neural nets, linear separators, circles, etc whose parameters can be varied)  having &lt;span class=&quot;math-container&quot;&gt;$\mathcal V \mathcal C$&lt;/span&gt; dimension of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; shatters all subsets of the single/multiple set of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; points it shatters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a learning algorithm, to select a function, which gives accuracy close to the best possible accuracy (on a classification task) from the aforementioned set of functions (shattered by your  model, which means it can represent the function with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; error) it needs a certain sample size of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. For the sake of argument, let's say your set of functions (or the model shatters) contains all the possible mappings from &lt;span class=&quot;math-container&quot;&gt;$\mathcal X \rightarrow \mathcal Y$&lt;/span&gt; (assume &lt;span class=&quot;math-container&quot;&gt;$\mathcal X$&lt;/span&gt; contains &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; points i.e finite sized, as a result number of functions possible is &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt;). One of the function it will shatter is the function which performs the classification, and thus you are interested in finding it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any learning algorithm which sees &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; number of samples can easily pick up the set of functions which agrees on these points. The number of these functions agreeing on these sampled &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; points but disagreeing on the &lt;span class=&quot;math-container&quot;&gt;$n-m$&lt;/span&gt; points is &lt;span class=&quot;math-container&quot;&gt;$2^{(n-m)}$&lt;/span&gt;. The algorithm has no way of selecting from these shortlisted functions (agreeing on &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; points) the one function which is the actual classifier, hence it can only guess. Now increase the sample size and the number of functions disagreeing keeps falling and the algorithms probability of success keeps getting better and better until you see all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; points when your algorithm can identify the mapping function of the classifier exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\mathcal V \mathcal C$&lt;/span&gt; dimension is very similar to the above argument, except it doesn't shatter the entire domain &lt;span class=&quot;math-container&quot;&gt;$\mathcal X$&lt;/span&gt; and only a part of it. This limits the models capability to approximate a classification function exactly. So your learning algorithm tries to pick a function from all the functions your model shatter, which is very close to the best possible classification function i.e there will exist a best possible (not exact) function (optimal) in your set of functions which is closest to the classification function and your learning algorithm tries to pick a function which is close to this optimal function. And thus again, as per our previous argument it will need to keep increasing the sample size to reach as close as possible to the optimal function. The exact mathematical bounds can be found in books, but the proofs are quite daunting.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-04-17T10:39:14.370" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20351" PostTypeId="1" CreationDate="2020-04-17T14:50:01.740" Score="1" ViewCount="160" Body="&lt;p&gt;Here is my &lt;a href=&quot;https://colab.research.google.com/drive/1ca3SZtZFhqeBTiuv81_Mn9qIdZ5vg6fF#scrollTo=9ctKD177RTPa&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently, I solved the game of Atari Breakout using a classic DQN model. The convergence of the mean reward slowly improved during three days. I was interested in learning a method which may help me improving the convergence speed. I found the following article : &lt;a href=&quot;https://arxiv.org/pdf/1706.10295v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1706.10295v3.pdf&lt;/a&gt;. It says I can use an Independent Gaussian Noise to outperform an standard DQN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my Noisy DQN model : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;import torch&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;import numpy as np&#xA;&#xA;&#xA;class NoisyLinear(nn.Linear): #Independent Gaussian Noise used with NoisyDQN model&#xA;    def __init__(self, in_features, out_features, sigma_init=0.017, bias=True):&#xA;        super(NoisyLinear, self).__init__(in_features, out_features, bias=bias)&#xA;&#xA;        self.sigma_weight = nn.Parameter(torch.full((out_features, in_features), sigma_init))&#xA;        self.register_buffer(&quot;epsilon_weight&quot;, torch.zeros(out_features, in_features))&#xA;&#xA;        if bias: &#xA;            self.sigma_bias = nn.Parameter(torch.full((out_features,), sigma_init))&#xA;            self.register_buffer(&quot;epsilon_bias&quot;, torch.zeros(out_features))&#xA;&#xA;        self.reset_parameters()&#xA;&#xA;    def reset_parameters(self):&#xA;        std = math.sqrt(3/self.in_features)&#xA;        self.weight.data.uniform_(-std, std)&#xA;        self.bias.data.uniform_(-std, std)&#xA;&#xA;    def forward(self, input):&#xA;        self.epsilon_weight.normal_()&#xA;        bias = self.bias&#xA;        if bias is not None:&#xA;            self.epsilon_bias.normal_()&#xA;            bias = bias + self.sigma_bias * self.epsilon_bias.data&#xA;        return F.linear(input, self.weight + self.sigma_weight * self.epsilon_weight.data, bias)&#xA;&#xA;&#xA;class NoisyDQN(nn.Module):&#xA;    &quot;&quot;&quot;&#xA;    Look at https://arxiv.org/pdf/1706.10295v3.pdf&#xA;    &quot;&quot;&quot;&#xA;&#xA;    def __init__(self, input_shape, num_actions):&#xA;        super(NoisyDQN, self).__init__()&#xA;&#xA;        self.conv = nn.Sequential(&#xA;            nn.Conv2d(in_channels=input_shape[0], out_channels=32, kernel_size=8, stride=4),&#xA;            nn.ReLU(),&#xA;            nn.Conv2d(in_channels=32, out_channels=64, kernel_size=4, stride=2),&#xA;            nn.ReLU(),&#xA;            nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, stride=1),&#xA;            nn.ReLU(),&#xA;        )&#xA;&#xA;        self.conv_output_size = self.get_output_size(input_shape)&#xA;&#xA;        self.linear = nn.Sequential(&#xA;           NoisyLinear(in_features=self.conv_output_size, out_features=512),&#xA;           nn.ReLU(),&#xA;           NoisyLinear(in_features=512, out_features=num_actions)&#xA;        )&#xA;&#xA;    def get_output_size(self, input_shape):&#xA;        output = self.conv(torch.zeros((1, *input_shape)))&#xA;        return int(np.prod(output.shape))&#xA;&#xA;    def forward(self, input):&#xA;        self.layer1 = self.conv(input)&#xA;        self.layer1 = self.layer1.reshape(-1, self.conv_output_size)&#xA;&#xA;        return self.linear(self.layer1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea is to replace the epsilon greedy action selection and my standard DQN model by the Noisy Network you can see just above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code run successfully, but it doesn't improve even a bit. How can I fix that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DgrcY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DgrcY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After nearly 200k episodes, I am still between &lt;code&gt;1.5&lt;/code&gt; and &lt;code&gt;2&lt;/code&gt; reward. The maximum reward I can get on the Atari Breakout game is about &lt;code&gt;500&lt;/code&gt;. With a standard DQN, after 100k episodes, I am near &lt;code&gt;11&lt;/code&gt; as reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the above picture the X-axis is the number of episodes and the Y-axis is the mean reward over the last 100 rewards. The Y-axis is describe with &#xA;&lt;code&gt;mean_reward = np.mean(self.total_rewards[-100:])&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After about 8 hours of training, I got this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v29Dt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v29Dt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, it is not working as good as in the paper. I worked a lot with the hyperparameters, but not changed. &lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-04-23T15:55:10.877" LastActivityDate="2020-04-23T15:55:10.877" Title="Replace epsilon greedy action selection and the standard DQN by an Independent Gaussian Noise Network Model" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;dqn&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="20352" PostTypeId="1" CreationDate="2020-04-17T15:01:01.580" Score="2" ViewCount="60" Body="&lt;p&gt;I'm a bit new to AI and I'd like to use some kind of clustering algorithm to solve a problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to parse pdf documents to get headings and titles. I can parse pdf to html and I'm then able to get some information on the lines of the document. I've identified some properties that can be useful for identifying the headings.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;font-size (int): of course it's quite usual that heading's font-size is bigger than normal text&lt;/li&gt;&#xA;&lt;li&gt;font-family (string): it's possible for headings to be  bold so font-family may differ&lt;/li&gt;&#xA;&lt;li&gt;left property (int): it's also possible that headings are aligned a bit to the right, there's an indentation that's not always there on normal paragraphs&lt;/li&gt;&#xA;&lt;li&gt;bonus boolean: I have identified some properties that I can combine to get a boolean value. When the boolean is set to true it can increase the chances of the paragraph being a heading.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, these are not rules that apply to all headings. Some headings may follow some of these but not all of them. It could also be possible that some 'normal' paragraphs follow all these points, but what I've seen is that, in general, those rules where what made headings different from paragraphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this information, is there a way of doing what I'm looking for? As I said, I'm new to AI even though I have a background in CS and mathematics. I thought clustering could be interesting since I'm trying to create 2 clusters: headings and normal paragraphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm do you think might work for this use case. Should I look outside clustering?&lt;/p&gt;&#xA;" OwnerUserId="36143" LastEditorUserId="2444" LastEditDate="2020-04-18T00:44:21.453" LastActivityDate="2020-04-21T13:27:44.577" Title="Could clustering be used to parse pdf documents to get headings and titles?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20353" PostTypeId="1" AcceptedAnswerId="20356" CreationDate="2020-04-17T15:33:39.123" Score="1" ViewCount="61" Body="&lt;p&gt;I guess the model shown in this image (img_1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xzsN1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xzsN1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is the same as &lt;a href=&quot;https://youtu.be/fXOsFF95ifk?list=PLkDaE6sCZn6Ec-XTbcX1uRg2_u4xOEky0&amp;amp;t=123&quot; rel=&quot;nofollow noreferrer&quot;&gt;the one&lt;/a&gt; in this image (img_2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AHdjH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AHdjH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was trying to build a neural net like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This keras code is to do the job.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(3, input_dim=3, activation='relu'))&#xA;model.add(Dense(1, activation='sigmoid'))&#xA;plot_model(model, to_file='model_plot.png', show_shapes=True, show_layer_names=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, &lt;code&gt;print(model.summary())&lt;/code&gt; outputs &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Model: &quot;sequential_17&quot;&#xA;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;dense_31 (Dense)             (None, 3)                 12        &#xA;_________________________________________________________________&#xA;dense_32 (Dense)             (None, 1)                 4         &#xA;=================================================================&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are 3 &lt;code&gt;w&lt;/code&gt;s and 1 &lt;code&gt;b&lt;/code&gt; in the hidden layer. Why does this model have 12 parameters?&lt;/p&gt;&#xA;" OwnerUserId="35896" LastEditorUserId="2444" LastEditDate="2020-04-18T01:01:34.103" LastActivityDate="2020-04-18T01:01:34.103" Title="Why does this model have 12 parameters?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20355" PostTypeId="1" AcceptedAnswerId="20358" CreationDate="2020-04-17T16:04:40.127" Score="3" ViewCount="610" Body="&lt;p&gt;Pretty soon I will be finishing up &lt;a href=&quot;https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Machine Learning: From Theory to Algorithms&lt;/a&gt; by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in &lt;a href=&quot;http://proceedings.mlr.press&quot; rel=&quot;nofollow noreferrer&quot;&gt;JMLR's COLT&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a book similar to &quot;Understanding Machine Learning: From Theory to Algorithms&quot; that would progress my knowledge further and would go well after reading UML?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Also, taking courses in this is not really an option, so this will be for self-study).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that I have also asked this question here on &lt;a href=&quot;https://cstheory.stackexchange.com/q/46650/34637&quot;&gt;TCS SE&lt;/a&gt;, but it was recommended I also ask here.)&lt;/p&gt;&#xA;" OwnerUserId="36131" LastEditorUserId="2444" LastEditDate="2021-01-16T19:50:22.197" LastActivityDate="2021-01-16T19:50:22.197" Title="What are some resources on computational learning theory?" Tags="&lt;reference-request&gt;&lt;papers&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;&lt;books&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20356" PostTypeId="2" ParentId="20353" CreationDate="2020-04-17T16:32:01.000" Score="2" Body="&lt;p&gt;You have 3 inputs going to 3 nodes in the input layer. Each connection has a weight so you have  3 X 3 =9 weights. Plus each node has a bias weight so that adds 3 more weights for a total of 12. Your output layer has 3 inputs and is a single node so you have 3 weights for the inputs to the node plus a bias weight for a total of 4. So the total weights in your network is 16.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-17T16:32:01.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20357" PostTypeId="2" ParentId="20352" CreationDate="2020-04-17T16:59:59.283" Score="2" Body="&lt;p&gt;Yes, you could use clustering: Encode your features as a feature vector and feed it into a clustering algorithm (see &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316801&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finding Groups in Data&lt;/a&gt; for a comprehensive description of these). You could use agglomerative clustering, which would give you groups of similar items; perhaps different level headings will be clustered together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you could try a decision tree, something like &lt;a href=&quot;https://en.wikipedia.org/wiki/ID3_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID3&lt;/a&gt;, which would also be suitable; for this you'd need some annotated training data, though. But with a small amount of data you might solve it, if your items are clearly separated.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-17T16:59:59.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20358" PostTypeId="2" ParentId="20355" CreationDate="2020-04-17T17:07:08.537" Score="6" Body="&lt;p&gt;Although I have only partially read (or not read at all) some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you.&lt;/p&gt;&#xA;&lt;p&gt;I would also like to note that if you understand the contents of the book you are currently reading, you are probably already prepared for reading some (if not most of) the research papers you wish to read. Initially, you may find them a little bit too succinct and sometimes unclear or complex, but you need to get used to this format, so there's nothing stopping you from trying to read them and learn even more by doing this exercise.&lt;/p&gt;&#xA;&lt;h3&gt;Books&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://mitpress.mit.edu/books/introduction-computational-learning-theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Computational Learning Theory&lt;/a&gt; (1994) by Kearns and Vazirani&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No free PDF available&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://statisticalsupportandresearch.files.wordpress.com/2017/05/vladimir-vapnik-the-nature-of-statistical-learning-springer-2010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Nature of Statistical Learning Theory&lt;/a&gt; (1995, 2000) by Vapnik&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://profsite.um.ac.ir/%7Emonsefi/machine-learning/pdf/Machine-Learning-Tom-Mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt; (1997) by Mitchell&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://read.pudn.com/downloads161/ebook/733192/Statistical-Learning-Theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistical Learning Theory&lt;/a&gt; (1998) by Vapnik&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ii.uni.wroc.pl/%7Elukstafi/pmwiki/uploads/AGT/Prediction_Learning_and_Games.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prediction, Learning, and Games&lt;/a&gt; (2006) by Cesa-Bianchi and Lugosi&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/e923/9469aba4bccf3e36d1c27894721e8dbefc44.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations of Machine Learning&lt;/a&gt; (2012) by Mohri et al.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doc.lagout.org/science/0_Computer%20Science/2_Algorithms/Boosting_%20Foundations%20and%20Algorithms%20%5BSchapire%20%26%20Freund%202012-05-18%5D.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boosting: Foundations and Algorithms&lt;/a&gt; (2012) by Schapire and Freund&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit Algorithms&lt;/a&gt; (2020) by Lattimore and Szepesvári (with associated &lt;a href=&quot;https://tor-lattimore.com/downloads/book/solutions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;solutions to the exercises&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Papers&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://web.mit.edu/6.435/www/Valiant84.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A theory of the learnable&lt;/a&gt; (1984) by Valiant&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.math.arizona.edu/%7Ehzhang/math574m/Read/vapnik.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An overview of statistical learning theory&lt;/a&gt; (1999) by Vapnik&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.econ.upf.edu/%7Elugosi/mlss_slt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Statistical Learning Theory&lt;/a&gt; (2014) by Bousquet et al.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Courses (videos)&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=mbyG85GZ0PI&amp;amp;list=PLnIDYuXHkit4LcWjDe0EwlE57WiGlBs08&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning from Data (Caltech)&lt;/a&gt; (2012) by Yaser Abu-Mostafa&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=aILazXK059Y&amp;amp;list=PLPW2keNyw-usgvmR7FTQ3ZRjfLs5jT4BO&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning Theory&lt;/a&gt; (2015) by Shai Ben-David&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Lecture notes&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.cse.huji.ac.il/%7Eshais/Handouts.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Machine Learning&lt;/a&gt; (2009) by Shai Shalev-Shwartz&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.econ.upf.edu/%7Elugosi/anu.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Concentration-of-measure inequalities&lt;/a&gt; (2009) by Gábor Lugosi&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://stat.wharton.upenn.edu/%7Eskakade/courses/stat928/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stat 928: Statistical Learning Theory&lt;/a&gt; (2011) by Sham Kakades&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.cs.columbia.edu/%7Erocco/Teaching/S12/&quot; rel=&quot;nofollow noreferrer&quot;&gt;COMS E6253, Advanced Topics in Computational Learning Theory&lt;/a&gt; (2012) Rocco Servedio&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.mit.edu/%7E9.520/spring12/&quot; rel=&quot;nofollow noreferrer&quot;&gt;9.520: Statistical Learning Theory and Applications, Spring 2012&lt;/a&gt; by Tomaso Poggio et al.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www-stat.wharton.upenn.edu/%7Erakhlin/courses/stat928/&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAT928: Statistical Learning Theory and Sequential Prediction&lt;/a&gt; (2014) by Alexander Rakhlin and Karthik Sridharan&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://maxim.ece.illinois.edu/teaching/SLT/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ECE 543, Statistical Learning Theory&lt;/a&gt; (2019) by Bruce Hajek and Maxim Raginsky&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Other&lt;/h3&gt;&#xA;&lt;p&gt;See also this list of resources &lt;a href=&quot;https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc&lt;/a&gt; compiled by Kiran Vodrahalli.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-30T14:54:04.793" LastActivityDate="2020-08-30T14:54:04.793" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20360" PostTypeId="1" CreationDate="2020-04-17T19:19:43.230" Score="0" ViewCount="54" Body="&lt;p&gt;Some years ago, I came across a notion that states something like: &quot;a machine cannot leave its current level of thinking, move one level up and think about the level below&quot;, meaning that a machine cannot judge its lower-level deeds until we hard-code this ability for it. This &lt;em&gt;meta layer thinking&lt;/em&gt; is what separates humans and AI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also remember I heard that as a rule of thumb, if we hard code 5 layers of meta-layer thinking for the machine, then its behavior would be satisfyingly humane, maybe enough to pass the Turing test. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't remember if it was a book, a documentary or a paper, and I cannot find any resource by searching key-words. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know what exactly this theory is? And how can I find a good resource to read about that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also vaguely remember that it was related to this proven theory that &quot;a machine cannot program another machine on its level, because it requires meta-layer thinking&quot; or something like that.&lt;/p&gt;&#xA;" OwnerUserId="6258" LastEditorUserId="2444" LastEditDate="2020-04-18T01:19:12.003" LastActivityDate="2020-04-18T01:19:12.003" Title="Is there a theory of &quot;meta-layer thinking&quot;?" Tags="&lt;agi&gt;&lt;meta-learning&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20362" PostTypeId="2" ParentId="16599" CreationDate="2020-04-17T23:47:37.183" Score="5" Body="&lt;p&gt;There's a list of ongoing and past RL competitions &lt;a href=&quot;https://github.com/seungjaeryanlee/awesome-rl-competitions&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. The ongoing competitions according to that list are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://aws.amazon.com/deepracer/league/&quot; rel=&quot;noreferrer&quot;&gt;AWS DeepRacer League&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.kaggle.com/c/connectx&quot; rel=&quot;noreferrer&quot;&gt;Connect X&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/MIT-TESSE/goseek-challenge&quot; rel=&quot;noreferrer&quot;&gt;GOSEEK Challenge&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-17T23:47:37.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20364" PostTypeId="1" CreationDate="2020-04-18T07:27:40.427" Score="1" ViewCount="18" Body="&lt;p&gt;Is it useful to use &lt;code&gt;Siamese network&lt;/code&gt; structure for &lt;code&gt;GANs&lt;/code&gt;  like sharing &lt;code&gt;latent space&lt;/code&gt; between generators in &lt;code&gt;cGAN&lt;/code&gt; , or also with &lt;code&gt;discriminators&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thinking about it, like giving the generator tips about the knowledge-base of the discriminator, to target the problem of discriminator forgetting and increase the chance of convergence. Because than the discriminator prediction confidence is dependent of the generators construction (what’s anyway the case, but now on a system base ). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think ? Didn’t see it in recent papers that often, just in this &lt;a href=&quot;https://arxiv.org/abs/1901.08211&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt;, but it’s more like a &lt;code&gt;pix2pix&lt;/code&gt; transformation, and just works so well, because they are using the &lt;code&gt;segmentation masks&lt;/code&gt; of A, to get good segmentation results on B‘ ( A transformed to B). Didn’t find any approaches to sth like &lt;code&gt;leaky discriminators&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="35557" LastEditorUserId="35557" LastEditDate="2020-04-19T11:25:24.340" LastActivityDate="2020-04-19T11:25:24.340" Title="Leaky Discriminators and Siamese GANs" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20365" PostTypeId="1" CreationDate="2020-04-18T11:22:36.557" Score="2" ViewCount="124" Body="&lt;p&gt;I'm using Monte Carlo Tree Search with UCT selection to try and build an AI player for a complex multiplayer board game. My regular UCT MCTS seems to be working fine, winning with random and basic greedy players or low-depth 'paranoid' alpha-beta variant player, but I've been looking for some methods to improve it and I found RAVE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;In RAVE, for a given game tree node N, its child nodes Ci store not only the statistics of wins in playouts started in node N but also the statistics of wins in all playouts started in node N and below it, if they contain move i (also when the move was played in the tree, between node N and a playout). This way the contents of tree nodes are influenced not only by moves played immediately in a given position but also by the same moves played later.&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found a lot of literature about it and it was supposed to give good results - 70%-80% win rate against basic UCT on a game of TicTacToe3D. I implemented it as a sort of benchmark, a 4x4x4 version, before trying it on my target game. But, however I tried tuning the parameters, I've been getting worse results, the win rate is at best arount 46%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been calculating the node values like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;visits[i] is a number of visits for child i of parent p that selection is performed on, wins[i] is a number of wins according to UCT, AMAFvisits and AMAFwins are assigned based on the node's source action -&gt; updated after a finished simulation if a sourceAction (the action that changed the game state into this state) was played in the simulation by the player of the MCTS tree root node. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (int i = 0; i &amp;lt; nChildren; i++) {&#xA;    if (visits[i] &amp;lt; 1) {&#xA;        value = Double.MAX_VALUE - rnd.nextDouble();&#xA;    }&#xA;    else if (m[i] &amp;lt; 1) {&#xA;        double vUCT = wins[i]/visits[i] + C*Math.sqrt(Math.log(sumVisits)/(visits[i]));&#xA;        value = vUCT;&#xA;    }&#xA;    else {&#xA;        double beta = Math.sqrt(k/(3*visits[i] + k));&#xA;        double vRAVE = (AMAFscores[i])/(m[i]) + C*Math.sqrt(Math.log(mChildren)/(m[i]));&#xA;        double vUCT = (wins[i])/(visits[i])+ C*Math.sqrt(Math.log(sumVisits)/(visits[i]));&#xA;        value = beta * vRAVE + (1 - beta) * vUCT;&#xA;        value += rnd.nextDouble() * eps;&#xA;        /*double beta = Math.sqrt(k/(3*visits[i] + k));&#xA;        double vRAVE = (AMAFscores[i])/(m[i]);&#xA;        double vUCT = (wins[i])/(visits[i]);&#xA;        value = beta * vRAVE + (1 - beta) * vUCT;&#xA;        value += C*Math.sqrt(Math.log(sumVisits)/(visits[i]));&#xA;        value += rnd.nextDouble() * eps;*/&#xA;    }&#xA;    if (maxValue &amp;lt;= value) {&#xA;        maxValue = value;&#xA;        index = i;&#xA;    }&#xA;}&#xA;chosen = tree.getTreeNode(children.get(index));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's a paint rendition of my understanding of how RAVE should work -&gt; &lt;a href=&quot;https://imgur.com/a/MM4K1HE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/MM4K1HE&lt;/a&gt;.&#xA;Am I missing something? Is my implementation wrong? Here's the rest of the code responsible for traversing the tree in a 'rave way': &lt;a href=&quot;https://www.paste.org/104476&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.paste.org/104476&lt;/a&gt;. The expand function on tree expands the tree for all actions, and returns a random one which then gets visited, the others are to be visited in other iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first tested the code on k = 250 like the authors of the benchmark paper &lt;a href=&quot;https://dke.maastrichtuniversity.nl/m.winands/documents/CIG2016_RAVE.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dke.maastrichtuniversity.nl/m.winands/documents/CIG2016_RAVE.pdf&lt;/a&gt; suggested and on 100, 1000 and 10000 iterations, with tree depth 20 or 50. I also experimented with other k values and other params.&lt;/p&gt;&#xA;" OwnerUserId="36162" LastActivityDate="2020-04-18T11:22:36.557" Title="MCTS RAVE performing badly in Board Game AI" Tags="&lt;game-ai&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20366" PostTypeId="1" AcceptedAnswerId="20367" CreationDate="2020-04-18T12:42:55.083" Score="5" ViewCount="290" Body="&lt;p&gt;I have just started to study reinforcement learning and, as far as I understand, existing algorithms search for the optimal solution/policy, but do not allow the possibility for the programmer to suggest a way to find the solution (to guide their learning process). This would be beneficial for finding the optimal solution faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to guide the learning process in (deep) reinforcement learning?&lt;/p&gt;&#xA;" OwnerUserId="32237" LastEditorUserId="2444" LastEditDate="2020-04-18T13:57:21.337" LastActivityDate="2020-04-21T12:54:04.307" Title="Is it possible to guide a reinforcement learning algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;supervised-learning&gt;&lt;efficiency&gt;&lt;active-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20367" PostTypeId="2" ParentId="20366" CreationDate="2020-04-18T13:10:50.283" Score="3" Body="&lt;p&gt;The programmer already guides the RL algorithm (or agent) by specifying the reward function. However, the reward function alone may not be sufficient to learn efficiently and fast, as you correctly noticed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To attempt to solve this &lt;em&gt;inefficiency problem&lt;/em&gt;, one solution is to combine reinforcement learning with supervised learning. For example, the paper &lt;a href=&quot;https://arxiv.org/pdf/1704.03732.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-learning from Demonstrations&lt;/a&gt; (2017) by Todd Hester et al. describes an approach to achieve this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://cs.stanford.edu/people/acvogel/papers/290.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Active Reinforcement Learning&lt;/a&gt; (2008) by Arkady Epshteyn et al. also tries to solve this problem but by incorporating approximations (given by domain experts) of the MDP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are probably many other possible solutions. In fact, all model-based RL algorithms could probably fall into this category of algorithms that estimate or incorporate the dynamics of the environment to find a policy more efficiently.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-18T13:15:57.677" LastActivityDate="2020-04-18T13:15:57.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20368" PostTypeId="2" ParentId="20289" CreationDate="2020-04-18T13:34:29.333" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there ways to more precisely approximate how good a single action really is considering its short and long term effects?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To understand the short-term effects of an action, just take each of the available actions from the current state and observe the reward for each of them. The action that gives you the highest immediate reward is the best action. However, note that the reward function may change or could be stochastic. In those cases, you may need to estimate the best action e.g. by executing it multiple times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know the action that gives you the highest amount of reward in the long run (i.e. that gives you the highest return), then you can use one of the available RL algorithms, which were invented exactly to solve this problem. Basically, you're asking us what is the best RL algorithm. It depends on the problem, as usual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know the effects of an action in e.g. &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps ahead, then you can probably formulate this problem as a truncated version of the typical reinforcement learning problem. In practice, you probably can achieve this by changing the &lt;em&gt;discount factor&lt;/em&gt; so that the next &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; rewards are more valuable (or are the only ones considered) than the rewards after &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; steps. If you aren't familiar with discount factors, I encourage you to have a look at this concept from a reference book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, in this answer, I am just trying to give you the idea and intuition behind a possible answer to your question (also because your question isn't really suited to provide more detailed or rigorous answers).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-18T22:11:35.953" LastActivityDate="2020-04-18T22:11:35.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20369" PostTypeId="1" CreationDate="2020-04-18T14:11:37.867" Score="1" ViewCount="105" Body="&lt;p&gt;Google provides a lot of pretrained tensorflow models, but I cannot find a license.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in the &lt;a href=&quot;https://github.com/tensorflow/tfjs-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tfjs-models&lt;/a&gt;. The code is licensed Apache-2.0, but the models are downloaded by the code, so the license of the repository probably does not apply to the models and I am not able to find anywhere a note about the license of the pretrained models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I handle this, especially when I may want to distribute models derived from the pretrained Google models?&lt;/p&gt;&#xA;" OwnerUserId="25798" LastActivityDate="2020-04-21T10:41:37.373" Title="How does a software license apply to pretrained models?" Tags="&lt;tensorflow&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20370" PostTypeId="2" ParentId="20366" CreationDate="2020-04-18T14:34:59.583" Score="2" Body="&lt;p&gt;Here are two very related interesting papers:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning from Human Preferences&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ijcai.org/Proceedings/2018/0817.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving Reinforcement Learning with Human Input&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-04-18T14:34:59.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20371" PostTypeId="1" AcceptedAnswerId="20376" CreationDate="2020-04-18T15:04:06.350" Score="2" ViewCount="49" Body="&lt;p&gt;Neural networks with feedback (Hopfield, Hamming, etc.) differ from ordinary neural networks (multilayer perceptrons, etc.), which turns them into a dynamic element with its own internal dynamics (if we consider them as a separate dynamic link). The following question naturally arises - is it possible to represent them in the form of state spaces? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nuance is that feedback is created by introducing a delay element, which means recording a neural network exclusively in a discrete form. Is continuous recording possible? What acts as matrices A, B, C, D? How does the presence of nonlinear activation functions affect?&#xA;The only more or less useful information that I managed to find is in this article:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19930021849.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On neural networks in identification and control of dynamic systems. 3.2 Paragraph. Page 8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my assumptions are only confirmed there, which does not clarify the situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if someone has come across this and can assist in studying the issue, please share links, possibly examples, etc.&lt;/p&gt;&#xA;" OwnerUserId="32829" LastActivityDate="2020-04-18T20:05:45.383" Title="Neural networks with internal dynamics in the state-space form" Tags="&lt;recurrent-neural-networks&gt;&lt;models&gt;&lt;control-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20372" PostTypeId="1" CreationDate="2020-04-18T15:33:29.893" Score="4" ViewCount="75" Body="&lt;p&gt;I'm reading the paper &lt;a href=&quot;https://arxiv.org/abs/1601.06759&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pixel Recurrent Neural Network&lt;/a&gt;. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the kernel of the one-dimensional convolution has size &lt;span class=&quot;math-container&quot;&gt;$k \times 1$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k \geq 3$&lt;/span&gt;; the larger value of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; the broader the context that is captured.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The one-dimensional kernel can capture only the left context. (Is this correct?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; kernel such as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;1 &amp;amp; 1 &amp;amp; 1 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can capture triangular contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="32303" LastEditorUserId="2444" LastEditDate="2020-04-19T15:45:28.480" LastActivityDate="2020-09-14T09:17:37.257" Title="Why Pixel RNN (Row LSTM) can capture triangular contexts?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;deep-neural-networks&gt;&lt;deepmind&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20373" PostTypeId="2" ParentId="20342" CreationDate="2020-04-18T16:59:44.683" Score="0" Body="&lt;p&gt;Thanks for the answers. If you are processing an audio signal I think the application of a low pass filter (lpf) would help to enhance the signal to noise ratio. This would help especially if the noise component occupies a large part of the spectrum. If the audio is human speech the majority of the energy is within the 300Hz to 3Khz region. Using a low pass filter with a cutoff frequency of 3Khz would eliminate noise that is in the higher part of the spectrum. You could implement the lpf as a pre-processing function. I am not knowledgeable on the implementation but a search should get you the info you need. I did find an article &lt;a href=&quot;https://dsp.stackexchange.com/questions/724/low-pass-filter-and-fft-for-beginners-with-python&quot;&gt;here.&lt;/a&gt; If I recall the process is to convert the time domain signal to the frequency domain using a FFT, then set a cutoff point and reconvert back to the time domain. I also know there are ways to implement that directly in the time domain.Hope this helps.&#xA;I am also supersized that if you achieve a high validation accuracy that your test set accuracy is so low. Your validation data should be data the network has not seen before just like your test data. Only thing I can think off is that the test data has a very different probability distribution than the training and validation data. How were the various data sets (train, test, validate) selected? Best choice is to select these randomly using something like sklearn train_test_split or Keras ImageDataGenerator flow from directory. Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="33976" LastEditDate="2020-04-18T17:07:45.003" LastActivityDate="2020-04-18T17:07:45.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20374" PostTypeId="1" AcceptedAnswerId="20375" CreationDate="2020-04-18T17:28:41.177" Score="0" ViewCount="40" Body="&lt;p&gt;I am reading through the &lt;a href=&quot;http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT paper&lt;/a&gt;. In parameter settings, page 15, there is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In each generation, 25% of offspring resulted from mutation without crossover.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does it mean?&lt;/p&gt;&#xA;" OwnerUserId="36170" LastEditorUserId="2444" LastEditDate="2020-04-18T22:22:03.893" LastActivityDate="2020-04-18T22:22:03.893" Title="What does &quot;In each generation, 25% of offspring resulted from mutation without crossover&quot; mean in the context of NEAT?" Tags="&lt;papers&gt;&lt;neat&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;&lt;genetic-operators&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20375" PostTypeId="2" ParentId="20374" CreationDate="2020-04-18T18:00:44.933" Score="2" Body="&lt;p&gt;In genetic algorithms, mutation without crossover simply means that part of the population is randomly changed. In this case this is applied to 25% of the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The remaining 75% either remain unchanged (generally the best performing specimen), or will be combined with other specimen (using crossover). It's a bit more complex here, as the genome is connecting weights in networks, and they use 'species', where certain individuals are treated as separate groups from the rest of the population.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-18T18:00:44.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20376" PostTypeId="2" ParentId="20371" CreationDate="2020-04-18T20:05:45.383" Score="1" Body="&lt;p&gt;I think that the book &quot;Neural Networks and Learning Machines&quot; of Haykin can help you. In his book the chapter 13 is about neural dynamics, and there are some examples of how analise the dynamics of network.&lt;/p&gt;&#xA;" OwnerUserId="36175" LastActivityDate="2020-04-18T20:05:45.383" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20377" PostTypeId="1" CreationDate="2020-04-18T21:21:21.223" Score="4" ViewCount="460" Body="&lt;p&gt;I have a question about how the averaging works when doing mini-batch gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I now understood the general gradient descent algorithm, but only for online learning. When doing mini-batch gradient descent, do I have to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;forward propagate&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate error&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate all gradients&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;...repeatedly over all samples in the batch, and then average all gradients and apply the weight change? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought it would work that way, but recently I have read somewhere that you basically only average the error of each example in the batch, and then calculate the gradients at the end of each batch. That left me wondering though, because, the activations of which sample in the mini-batch am I supposed to use to calculate the gradients at the end of every batch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be nice if somebody could explain what exactly happens during mini-batch gradient descent, and what actually gets calculated and averaged.&lt;/p&gt;&#xA;" OwnerUserId="17769" LastEditorUserId="2444" LastEditDate="2020-11-29T23:59:44.530" LastActivityDate="2020-11-29T23:59:44.530" Title="What exactly is averaged when doing batch gradient descent?" Tags="&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;feedforward-neural-network&gt;&lt;stochastic-gradient-descent&gt;&lt;batch-gradient-descent&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20378" PostTypeId="1" AcceptedAnswerId="20397" CreationDate="2020-04-18T21:43:31.520" Score="1" ViewCount="57" Body="&lt;p&gt;(&lt;a href=&quot;https://arxiv.org/pdf/1211.5063.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to paper in arxiv&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section &lt;strong&gt;2.1&lt;/strong&gt; the authors define &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; as the maximum possible value of the derivative of the activation function (e.g. 1 for tanh.) Then they have this to say: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We first prove that it is sufficient for &lt;span class=&quot;math-container&quot;&gt;$\lambda_1 &amp;lt; \frac{1}{\gamma}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\lambda_1$&lt;/span&gt; is the absolute value of the largest eigenvalue of the recurrent weight matrix &lt;span class=&quot;math-container&quot;&gt;$W_{rec}$&lt;/span&gt;, for the vanishing gradient problem to occur. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then they use the submultiplicity (&lt;span class=&quot;math-container&quot;&gt;$\|AB\| \le \|A\|\|B\|$&lt;/span&gt;) of the 2-norm of the Jacobians to obtain the following inequality: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \forall x, \| \frac{\partial x_{k+1}}{\partial x_k} \| \le \| W_{rec}^\top \| \| diag(\sigma'(x_k))\| &amp;lt; \frac{1}{\gamma} \gamma &amp;lt; 1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_k$&lt;/span&gt; is the pre-activated state vector of the RNN &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W_{rec}$&lt;/span&gt; is the weight matrix between states (i.e. &lt;span class=&quot;math-container&quot;&gt;$x_k = W_{rec} \times \sigma(x_{k-1}) + b$&lt;/span&gt; ) &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma()$&lt;/span&gt; is the activation function for the state vector &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$diag(v)$&lt;/span&gt; is the diagonal matrix version of a vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;They appear to be either substituting the norm of the weight matrix &lt;span class=&quot;math-container&quot;&gt;$\|W_{rec}^\top\|$&lt;/span&gt; for its largest eigenvalue &lt;span class=&quot;math-container&quot;&gt;$|\lambda_1|$&lt;/span&gt; (eigenvalues are the same for transposes) or just assuming that this norm is less than or equal to the eigenvalue. This bothers me because the norm of a matrix is bounded &lt;em&gt;below&lt;/em&gt;, not above, by this eigenvalue/spectral radius (&lt;a href=&quot;https://www.math.drexel.edu/~foucart/TeachingFiles/F12/M504Lect6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;see lemma 10 here&lt;/a&gt; and this &lt;a href=&quot;https://math.stackexchange.com/questions/2583417/spectral-radius-is-not-matrix-norm&quot;&gt;math SE question&lt;/a&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They seem to assume that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\| W_{rec}^\top \| \le \lambda_1 &amp;lt; \frac{1}{\gamma}  $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But really &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \| W_{rec}^\top \| \ge \lambda_1 $$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="34395" LastEditorUserId="34395" LastEditDate="2020-04-18T23:59:56.627" LastActivityDate="2020-04-19T12:56:22.363" Title="Does the paper &quot;On the difficulty of training Recurrent Neural Networks&quot; (2013) assume, falsely, that spectral radii are $\ge$ square matrix norms?" Tags="&lt;papers&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20379" PostTypeId="1" CreationDate="2020-04-18T22:12:30.753" Score="1" ViewCount="67" Body="&lt;p&gt;I am currently starting a research project whereby I am trying to convert text of one form into another.&#xA;i.e. If I were to write a seed sentance of the form &quot;Scientists have finally achieved the ability to induce dreams of electric sheep in the minds of anaesthetized robots&quot; I would like GPT-2 to convert this into &quot;Robots have finally had dreams of electric sheep whilst being anaesthetized by scientists.&quot; or some coherent permutation of the underlying structure whereby the main logic of the text is conveyed albeit roughly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current open source implementation of GPT-2 seeks to predict the next word, i.e. the seed text is given &quot;Scientist have finally&quot; and the generated text would be &quot; started being paid enough!&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first presumption was to use some form of GAN, however it became quickly evident that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Recent work has shown&#xA;  that when both quality and diversity is considered, GAN-generated text&#xA;  is substantially worse than language model generations (Caccia et al.,&#xA;  2018; Tevet et al., 2018; Semeniuta et al., 2018).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How could I most effectively achieve this? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="36177" LastActivityDate="2020-04-18T22:12:30.753" Title="How can I use GPT-2 to modify seed text of one form into a different form (LENGTH INVARIANT) whilst retaining meaning?" Tags="&lt;tensorflow&gt;&lt;pytorch&gt;&lt;text-summarization&gt;&lt;gpt&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20380" PostTypeId="2" ParentId="20377" CreationDate="2020-04-18T23:10:06.870" Score="5" Body="&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;First of all, it's completely normal that you are confused because nobody really explains this well and accurately enough. Here's my partial attempt to do that. So, this answer doesn't &lt;em&gt;completely&lt;/em&gt; answer the original question. In fact, I leave some unanswered questions at the end (that I will eventually answer).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The gradient is a linear operator&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The gradient operator &lt;span class=&quot;math-container&quot;&gt;$\nabla$&lt;/span&gt; is a linear operator, because, for some &lt;span class=&quot;math-container&quot;&gt;$f : \mathbb{R} \rightarrow \mathbb{R} $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g: \mathbb{R} \rightarrow \mathbb{R}$&lt;/span&gt;, the following two conditions hold.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla(f + g)(x) = (\nabla f)(x) + (\nabla g)(x),\; \forall x \in \mathbb{R}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla(kf)(x) = k(\nabla f)(x),\; \forall k, x \in \mathbb{R}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In other words, the restriction, in this case, is that the functions are evaluated at the same point &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in the domain. This is a very important restriction to understand the answer to your question below!&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linearity of the gradient directly follows from the linearity of the derivative. See a &lt;a href=&quot;https://www.youtube.com/watch?v=CfaaOc7dZIs&quot; rel=&quot;nofollow noreferrer&quot;&gt;simple proof here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For example, let &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g(x) = x^3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h(x) = f(x) + g(x) = x^2 + x^3$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\frac{dh}{dx} = \frac{d (x^2 + x^3)}{d x} = \frac{d x^2}{d x} + \frac{d x^3}{d x} = \frac{d f}{d x} + \frac{d g}{d x} = 2x + 3x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that both &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; are not linear functions (i.e. straight-lines), so the linearity of the gradients is not just applicable in the case of straight-lines.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Straight-lines are not necessarily linear maps&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Before proceeding, I want to note that there are at least two notions of &lt;em&gt;linearity&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There's the notion of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear map&lt;/a&gt; (or linear operator), i.e. which is the definition above (i.e. the gradient operator is a linear operator because it satisfies the two conditions, i.e. it preserves addition and scalar multiplication). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There's the notion of a straight-line function: &lt;span class=&quot;math-container&quot;&gt;$f(x) = c*x + k$&lt;/span&gt;. A function can be a straight-line and not be a linear map. For example, &lt;span class=&quot;math-container&quot;&gt;$f(x) = x+1$&lt;/span&gt; is a straight-line but it doesn't satisfy the conditions above. More precisely, in general, &lt;span class=&quot;math-container&quot;&gt;$f(x+y) \neq f(x) + f(y)$&lt;/span&gt;, and you can easily verify that this is the case if &lt;span class=&quot;math-container&quot;&gt;$x = 2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y=3$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$f(2+3) = 6$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f(2) = 3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f(3) = 4$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$f(2) + f(3) = 7 \neq f(2+3)$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Neural networks&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A neural network is a composition of (typically) non-linear functions (let's ignore the case of linear functions), which can thus be represented as &lt;span class=&quot;math-container&quot;&gt;$$y'_{\theta}= f^{L}_{\theta_L} \circ f^{L-1}_{\theta_{L-1}} \circ \dots \circ f_{\theta_1},$$&lt;/span&gt; where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f^{l}_{\theta_l}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th layer of your neural network and it &#xA;computes a non-linear function&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;${\theta_l}$&lt;/span&gt; is a vector of parameters associated with the &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;th layer&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the number of layers, &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y'_{\theta}$&lt;/span&gt; is your neural network, &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is a vector containing all parameters of the neural network&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y'_{\theta}(x)$&lt;/span&gt; is the output of your neural network&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\circ $&lt;/span&gt; means the composition of functions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$f^l_{\theta}$&lt;/span&gt; are non-linear, &lt;span class=&quot;math-container&quot;&gt;$y'_{\theta}$&lt;/span&gt; is also a non-linear function of the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. This notion of linearity is the second one above (i.e. &lt;span class=&quot;math-container&quot;&gt;$y'_{\theta}$&lt;/span&gt; is not a straight-line). In fact, neural networks are typically composed of sigmoids, ReLUs, and hyperbolic tangents, which are not straight-lines.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Sum of squared errors&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now, for simplicity, let's consider the sum of squared error (SSE) as the loss function of your neural network, which is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{L}_{\theta}(\mathbf{x}, \mathbf{y}) = \sum_{i=1}^N  \mathcal{S}_{\theta}(\mathbf{x}_i, \mathbf{y}_i) = \sum_{i=1}^N (\mathbf{y}_i - y'_{\theta}(\mathbf{x}_i))^2&#xA;$$&lt;/span&gt;&#xA;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{x} \in \mathbb{R}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} \in \mathbb{R}$&lt;/span&gt; are vectors of inputs and labels, respectively&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}_i$&lt;/span&gt; is the label for the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_{\theta}(\mathbf{x}_i, \mathbf{y}_i) = (\mathbf{y}_i - y'_{\theta}(\mathbf{x}_i))^2$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Sum of gradients vs gradient of a sum&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Given the gradient is a linear operator, one could think that computing the sum of the gradients is equal to the gradient of the sums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in our case, we are summing &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_{\theta}(\mathbf{x}_i, \mathbf{y}_i)$&lt;/span&gt; and, in general, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i \neq \mathbf{x}_j$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;. So, essentially, the SSE is the sum of the same function, i.e. &lt;span class=&quot;math-container&quot;&gt;$S_{\theta}$&lt;/span&gt;, evaluated at different points of the domain. However, the definition of a linear map applies when the functions are evaluated at the same point in the domain, as I said above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in general, in the case of neural networks with SSE, the gradient of the sum may not be equal to the sum of gradients, i.e. the definition of the linear operator for the gradient doesn't apply here because we are evaluating every squared error at different points of their domains.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Stochastic gradient descent&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The idea of stochastic gradient descent is to approximate the &lt;em&gt;true gradient&lt;/em&gt; (i.e. the gradient that would be computed with all training examples) with a &lt;em&gt;noisy gradient&lt;/em&gt; (which is an approximation of the true gradient).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How does the noisy gradient approximate the true gradient?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the case of mini-batch (&lt;span class=&quot;math-container&quot;&gt;$M \leq N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the size of the mini-batch and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the total number of training examples), this is actually a sum of the gradients, one for each example in the mini-batch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The papers &lt;a href=&quot;https://www.ics.uci.edu/~welling/publications/papers/stoclangevin_v6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Learning via Stochastic Gradient Langevin Dynamics&lt;/a&gt; (equation 1) or &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Auto-Encoding Variational Bayes&lt;/a&gt; (in section 2.2) use this type of approximation. See also &lt;a href=&quot;http://www.princeton.edu/~yc5/ele522_optimization/lectures/stochastic_gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Why?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;To give you some intuition of why we sum the gradients of the error of each input point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt;, let's consider the case &lt;span class=&quot;math-container&quot;&gt;$M=1$&lt;/span&gt;, which is often referred to as the (actual) stochastic gradient descent algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume we uniformly sample an arbitrary tuple &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}_j, \mathbf{y}_j)$&lt;/span&gt; from the dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D} = \{ (\mathbf{x}_i, \mathbf{y}_i) \}_{i=1}^N$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formally, we want to show that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \mathcal{L}_{\theta}(\mathbf{x}, \mathbf{y}) &#xA;&amp;amp;=&#xA;\mathbb{E}_{(\mathbf{x}_j, \mathbf{y}_j) \sim \mathbb{U}}\left[ \nabla_{\theta} \mathcal{S}_{\theta} \right] \label{1} \tag{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta} \mathcal{S}_{\theta}$&lt;/span&gt; is the gradient of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_{\theta}$&lt;/span&gt; with respect to the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{(\mathbf{x}_j, \mathbf{y}_j) \sim \mathbb{U}}$&lt;/span&gt; is the expectation with respect to the random variable associated with a sample &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}_j, \mathbf{y}_j)$&lt;/span&gt; from the uniform distribution &lt;span class=&quot;math-container&quot;&gt;$\mathbb{U}$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Under some conditions (see &lt;a href=&quot;https://math.stackexchange.com/q/217702/168764&quot;&gt;this&lt;/a&gt;), we can exchange the expectation and gradient operators, so \ref{1} becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \mathcal{L}_{\theta}(\mathbf{x}, \mathbf{y}) &#xA;&amp;amp;=&#xA;\nabla_{\theta} \mathbb{E}_{(\mathbf{x}_j, \mathbf{y}_j) \sim \mathbb{U}}\left[  \mathcal{S}_{\theta} \right] &#xA;\label{2} \tag{2}&#xA;\end{align}&lt;/span&gt;&#xA;Given that we uniformly sample, the probability of sampling an arbitrary &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}_j, \mathbf{y}_j)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{N}$&lt;/span&gt;. So, equation \ref{2} becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \mathcal{L}_{\theta} (\mathbf{x}, \mathbf{y}) &#xA;&amp;amp;=&#xA;\nabla_{\theta} \sum_{i=1}^N \frac{1}{N} \mathcal{S}_{\theta}(\mathbf{x}_i, \mathbf{y}_i) \\&#xA;&amp;amp;=&#xA;\nabla_{\theta} \frac{1}{N}  \sum_{i=1}^N \mathcal{S}_{\theta}(\mathbf{x}_i, \mathbf{y}_i) &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{N}$&lt;/span&gt; is a constant with respect to the summation variable &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and so it can be taken out of the summation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows that the gradient with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; of the loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_{\theta}$&lt;/span&gt; that includes all training examples is equivalent, in expectation, to the gradient of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}_{\theta}$&lt;/span&gt; (the loss function of one training example). &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Questions&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can we extend the previous proof to the case &lt;span class=&quot;math-container&quot;&gt;$1 &amp;lt; M \leq N$&lt;/span&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Which conditions need exactly to be satisfied so that we can exchange the gradient and the expectation operators? And are they satisfied in the case of typical loss functions, or sometimes they aren't (but in which cases)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the relationship between the proof above and the linearity of the gradient?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the proof above, we are dealing with expectations and probabilities!&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What would the gradient of a sum of errors represent? Can we still use it in place of the sum of gradients? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-01T23:12:31.317" LastActivityDate="2020-05-01T23:12:31.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20381" PostTypeId="1" AcceptedAnswerId="20451" CreationDate="2020-04-19T01:34:40.647" Score="1" ViewCount="49" Body="&lt;p&gt;I am interested in learning about policy gradient algorithms and REINFORCE. Can you suggest a good and easy paper that I can use to code them from scratch?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-04-19T12:05:18.473" LastActivityDate="2020-04-21T09:51:20.023" Title="Is there a good and easy paper to code policy gradient algorithms (REINFORCE) from scratch?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;policy-gradients&gt;&lt;implementation&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20382" PostTypeId="1" AcceptedAnswerId="20402" CreationDate="2020-04-19T02:28:10.263" Score="2" ViewCount="43" Body="&lt;p&gt;I've been learning a little bit about generalization theory, and in particular, the PAC (and PAC-Bayes) approach to thinking about this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I started to wonder if there is an analogous version of &quot;generalization&quot; in Unsupervised Learning? I.e., is there a general framework that encapsulates how &quot;good&quot; an unsupervised learning method is? There's reconstruction error for learning lower dimensional representations, but what about unsupervised clustering?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="36181" LastActivityDate="2020-04-19T23:31:09.580" Title="Is there a notion of generalization in unsupervised learning?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;generalization&gt;&lt;pac-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20383" PostTypeId="2" ParentId="20208" CreationDate="2020-04-19T02:29:43.743" Score="0" Body="&lt;p&gt;I think &lt;a href=&quot;https://ai.stackexchange.com/users/9947/duttaa&quot;&gt;DuttaA&lt;/a&gt; is right in his comment. &lt;a href=&quot;http://mathsci.kaist.ac.kr/%7Enipl/mas557/VCD_ANN_3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC Dimension of Neural Networks&lt;/a&gt; establishes VC bounds depending on the number of weights, whereas the UAT refers to a class of neural networks in which the number of weights a particular network can have is not bounded, although it needs to be finite.&lt;/p&gt;&#xA;&lt;p&gt;I think that we can show, from &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf#page=6&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorem 2&lt;/a&gt; and the observations below &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf#page=7&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorem 3&lt;/a&gt; in &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation by Superpositions of a Sigmoidal Function&lt;/a&gt;, that the VC dimension of&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S=\left\{\sum_{i=1}^N \alpha_i\sigma(y_i^T x + \theta_i) : N\in\mathbb N, \alpha_i, \theta_i \in\mathbb R, y_i\in\mathbb{R}^n \right\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is infinite.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\{(x_i, y_i)\}_{i=1}^k$&lt;/span&gt; be a sample of arbitrary size &lt;span class=&quot;math-container&quot;&gt;$k\in\mathbb N$&lt;/span&gt;, and let us see that there is a function in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; which can correctly classify it, i.e., &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; shatters &lt;span class=&quot;math-container&quot;&gt;$\{x_i\}_{i=1}^k$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We note &lt;span class=&quot;math-container&quot;&gt;$B(x, \varepsilon) := \{ y\in\mathbb{R}^n : d(x,y) &amp;lt; \varepsilon \}$&lt;/span&gt; (this is just &lt;a href=&quot;https://en.wikipedia.org/wiki/Ball_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard notation to denote a ball&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;First, let &lt;span class=&quot;math-container&quot;&gt;$\varepsilon &amp;gt; 0$&lt;/span&gt; be such that &lt;span class=&quot;math-container&quot;&gt;$B(x_i, \varepsilon)\cap B(x_j, \varepsilon) = \emptyset$&lt;/span&gt; every time that &lt;span class=&quot;math-container&quot;&gt;$i \ne j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now define &lt;span class=&quot;math-container&quot;&gt;$D = \cup_{y_i=1} B(x_i, \varepsilon)$&lt;/span&gt;. Define &lt;span class=&quot;math-container&quot;&gt;$f_{\varepsilon}(x)$&lt;/span&gt; as in &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf#page=7&quot; rel=&quot;nofollow noreferrer&quot;&gt;the observations below theorem 3 of Cybenko's paper&lt;/a&gt;, and use &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.441.7873&amp;amp;rep=rep1&amp;amp;type=pdf#page=6&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorem 2&lt;/a&gt; to find a function &lt;span class=&quot;math-container&quot;&gt;$G(x)$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; that classifies correctly all points at least &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt; away from the boundary of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, i.e., all points in the sample.&lt;/p&gt;&#xA;" OwnerUserId="36115" LastEditorUserId="2444" LastEditDate="2021-01-22T16:08:05.910" LastActivityDate="2021-01-22T16:08:05.910" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="20384" PostTypeId="1" CreationDate="2020-04-19T03:25:51.150" Score="5" ViewCount="1074" Body="&lt;p&gt;I understand that in DQNs, the loss is measured by taking the MSE of outputted Q-values and target Q-values.&lt;/p&gt;&#xA;&lt;p&gt;What does the target Q-values represent? And how is it obtained/calculated by the DQN?&lt;/p&gt;&#xA;" OwnerUserId="36072" LastEditorUserId="2444" LastEditDate="2021-01-18T01:05:35.793" LastActivityDate="2021-01-18T01:05:46.710" Title="What is the target Q-value in DQNs?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;value-functions&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20386" PostTypeId="2" ParentId="20310" CreationDate="2020-04-19T06:31:41.267" Score="1" Body="&lt;p&gt;Like Oliver Mason mentioned, Deep learning is just a sub-field of machine learning. In order to learn deep learning effectively you need to have certain pre-requisites like basic principle of Machine learning and basics of simple Artificial neural network  with some programming knowledge ( Python is go-to language). That being said, you don't need to know every single Machine learning algorithm and it's practices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if deep learning happens to be just a tool that you need for this particular project and have no time to learn in depth about it then I would recommend you to take a look at python libraries like Tensorflow, pytorch, scikit learn, scipy, open cv etc. You can get started and use DL, ML models with these and many other libraries without knowing it's under the hood algorithms and implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the best course to get started with deep learning with very little Ml knowledge is Andrew ng's deep learning.ai course on coursera ( you can audit the course and get all the course materials for free)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the link to the course : &lt;a href=&quot;https://www.coursera.org/specializations/deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep learning.ai&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35902" LastEditorUserId="2193" LastEditDate="2020-04-21T10:22:20.687" LastActivityDate="2020-04-21T10:22:20.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20388" PostTypeId="1" AcceptedAnswerId="20398" CreationDate="2020-04-19T08:53:20.820" Score="1" ViewCount="109" Body="&lt;p&gt;From what I understand VC dimension is what establishes the feasibility of learning for infinite hypothesis sets, the only kind we would use in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, the literature (i.e. &lt;a href=&quot;https://work.caltech.edu/textbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning from Data&lt;/a&gt;) states that VC gives a &lt;strong&gt;loose&lt;/strong&gt; bound, and that in real applications, learning models with lower VC dimension tend to generalize better than those with higher VC dimension. So, a good rule of thumb would be to require at least &lt;code&gt;10xVC&lt;/code&gt; dimension examples in order to get decent generalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having trouble interpreting what loose bound means. Is the VC generalization bound loose due to its universality? Meaning, its results apply to all hypothesis sets, learning algorithms, input spaces, probability distributions, and binary target functions.&lt;/p&gt;&#xA;" OwnerUserId="35990" LastEditorUserId="2444" LastEditDate="2020-04-19T12:02:09.977" LastActivityDate="2020-04-19T13:15:08.237" Title="What do we mean by saying &quot;VC dimension gives a LOOSE, not TIGHT bound&quot;?" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20389" PostTypeId="2" ParentId="20384" CreationDate="2020-04-19T09:24:47.860" Score="4" Body="&lt;p&gt;When training a Deep Q network with experienced replay, you accumulate what is known as training experiences &lt;span class=&quot;math-container&quot;&gt;$e_t = (s_t, a_t, r_t, s_{t+1})$&lt;/span&gt;. You then sample a batch of such experiences and for each sample you do the following.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Feed &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; into the network to get &lt;span class=&quot;math-container&quot;&gt;$Q(s,a;\theta)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Feed &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; into the network to get &lt;span class=&quot;math-container&quot;&gt;$Q(s’,a’,\theta)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Choose &lt;span class=&quot;math-container&quot;&gt;$max_aQ(s’,a’,\theta)$&lt;/span&gt; and set &lt;span class=&quot;math-container&quot;&gt;$ \gamma max_aQ(s’,a’,θ)$&lt;/span&gt; + &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; as the target of the network.&lt;/li&gt;&#xA;&lt;li&gt;Train the network with &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; as input to update &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. The output from the input of &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$Q(s,a,\theta)$&lt;/span&gt; and the gradient descent step minimises the squared distance between &lt;span class=&quot;math-container&quot;&gt;$Q(s,a,\theta)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma max_aQ(s’,a’,θ)$&lt;/span&gt; + &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-04-19T09:24:47.860" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20391" PostTypeId="2" ParentId="20384" CreationDate="2020-04-19T09:29:04.090" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What does the target Q-values represent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In a DQN, which uses off-policy learning, they represent a &lt;em&gt;refined&lt;/em&gt; estimate for the expected future reward from taking an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and from that point on following a target policy. The target policy in Q learning is based on always taking the maximising action in each state, according to current estimates of value.&lt;/p&gt;&#xA;&lt;p&gt;The estimate is &lt;em&gt;refined&lt;/em&gt; in that it is based on at least a little bit of data from experience - the immediate reward, and what transition happened next - but generally it is not going to be perfect.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And how is it obtained/calculated by the DQN?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are lots of ways to do this. The simplest in DQN is to process a single step lookahead based on the experience replay table.&lt;/p&gt;&#xA;&lt;p&gt;If your table contains the tuple &lt;em&gt;[state, action, immediate reward, next state, done?]&lt;/em&gt; as &lt;span class=&quot;math-container&quot;&gt;$[s, a, r, s', d]$&lt;/span&gt; then the formula for TD target, &lt;span class=&quot;math-container&quot;&gt;$g_{t:t+1}$&lt;/span&gt; is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r + \gamma \text{max}_{a'}[Q_{target}(s',a')], \qquad \text{when}\space d \space \text{is false}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r, \qquad \text{when}\space d \space \text{is true}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Typically &lt;span class=&quot;math-container&quot;&gt;$Q_{target}$&lt;/span&gt; is calculated using the &amp;quot;target network&amp;quot; which is a copy of the learning network for Q that is updated every N steps. This delayed update of the target predictions is done for numerical stability in DQN - conceptually it is an estimate for the same action values that you are learning.&lt;/p&gt;&#xA;&lt;p&gt;This target value can change every time you use any specific memory from experience replay. So you have to perform the same calculations on each minibatch, you cannot store the target values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2021-01-18T01:05:46.710" LastActivityDate="2021-01-18T01:05:46.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20392" PostTypeId="2" ParentId="12266" CreationDate="2020-04-19T10:48:23.937" Score="6" Body="&lt;p&gt;Both semi-supervised and self-supervised methods are similar in the sense that the goal is to learn with fewer labels per class. The way both formulate this is quite different:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Self-Supervised Learning:&lt;/strong&gt;    &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This line of work aims to learn image representations without requiring human-annotated labels and then use those learned representations on some downstream tasks. For example, you could take millions of unlabeled images, randomly rotate them by either 0, 90, 180 or 270 degrees and then train a model to predict the rotation angle. Once the model is trained, you can use transfer learning to fine-tune this model on a downstream task like cat/dog classification just like how you finetune ImageNet pretrained models. You can view an &lt;a href=&quot;https://amitness.com/2020/02/illustrated-self-supervised-learning/&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;overview&lt;/strong&gt;&lt;/a&gt; of the methods and also look at contrastive learning methods that are currently giving state-of-the-art results such as &lt;a href=&quot;https://amitness.com/2020/03/illustrated-simclr/&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;SimCLR&lt;/strong&gt;&lt;/a&gt; and &lt;a href=&quot;https://amitness.com/2020/03/illustrated-pirl/&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;PIRL&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://amitness.com/images/ss-geometric-transformation-architecture.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Semi-supervised Learning&lt;/strong&gt;  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Different from self-supervised learning, semi-supervised learning aims to use both labeled and unlabeled data at the same time to improve the performance of a supervised model. An example of this is &lt;a href=&quot;https://amitness.com/2020/03/fixmatch-semi-supervised/&quot; rel=&quot;noreferrer&quot;&gt;FixMatch&lt;/a&gt; paper where you train your model on labeled images. Then, for your unlabeled images, you apply augmentations to create two images for each unlabeled image. Now, we want to ensure that the model predicts the same label for both the augmentations of the unlabeled images. This can be incorporated into the loss as a cross-entropy loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://amitness.com/images/fixmatch-unlabeled-augment-concept.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="33910" LastActivityDate="2020-04-19T10:48:23.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20393" PostTypeId="2" ParentId="17231" CreationDate="2020-04-19T11:00:45.050" Score="1" Body="&lt;p&gt;Andrew Zisserman, who is a pioneer in the field of self-supervised learning, described self-supervised learning in a talk at ICML as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Self-supervised Learning is a form of unsupervised learning where the data provides the supervision. In general, we withhold some part of the data and task the network with predicting it. The network is forced to learn what we really care about e.g. a semantic representation, in order to solve it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, self-supervised is a subset of unsupervised learning, where you generate the labels from the given data itself. There are a few patterns of research being done for self-supervised learning:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Reconstruction:&lt;/strong&gt;&lt;br&gt;&#xA;In this, researchers have set up pretext tasks as predicting the color image from gray-scale image (Image Colorization), predicting the high-resolution image from the low-resolution version (Image Super-resolution) and removing some part of the image and trying to predict it (Image Inpainting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Common Sense Reasoning&lt;/strong&gt;:&lt;br&gt;&#xA;You could take patches of 3x3 images and shuffle the patches and ask the network to predict the correct order (&lt;a href=&quot;https://amitness.com/2020/02/illustrated-self-supervised-learning/#1-image-jigsaw-puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jigsaw puzzle&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, you could take the center patch and some random patch and train model to predict where the random patch is located in relation to the center patch (&lt;a href=&quot;https://amitness.com/2020/02/illustrated-self-supervised-learning/#2-context-prediction&quot; rel=&quot;nofollow noreferrer&quot;&gt;context prediction&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another approach where you randomly rotate image into {0, 90, 180, 270} degrees and ask the model to predict the rotation angle applied (&lt;a href=&quot;https://amitness.com/2020/02/illustrated-self-supervised-learning/#3-geometric-transformation-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geometric Transformation Recognition&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Clustering&lt;/strong&gt;:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could cluster the images into K categories and treat those clusters as labels. Then, a model can be trained on those clusters and you get representations. You can again repeat clustering and model training for few epochs. Papers for these include: &lt;a href=&quot;https://amitness.com/2020/04/deepcluster/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepCluster&lt;/a&gt; and &lt;a href=&quot;https://amitness.com/2020/04/illustrated-self-labelling/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-Labelling&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4. Contrastive Learning&lt;/strong&gt;:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paradigm, augmentations of the image is taken and the task is to bring two augmentations of the same images near while making the distance between this image and some other random image far. Papers for these include: &lt;a href=&quot;https://amitness.com/2020/03/illustrated-simclr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SimCLR&lt;/a&gt; and &lt;a href=&quot;https://amitness.com/2020/03/illustrated-pirl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PIRL&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33910" LastActivityDate="2020-04-19T11:00:45.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20394" PostTypeId="1" CreationDate="2020-04-19T11:33:19.747" Score="2" ViewCount="75" Body="&lt;p&gt;I have data that is collected from several different instruments simultaneously that is generally analyzed on a location-by-location basis.  A skilled interpreter can identify &quot;markers&quot; in the data that represent a certain change in conditions with depth - each marker only occurs once in each series of data.  However, it is possible that a maker is absent either due to missing data or that physical condition not existing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often, there are dozens of these markers per location and thousands, if not 10's of thousands of measurements that need to be interpreted.  The task is not that difficult and there are many strong priors that can be used to guide interpretation.  E.g., if marker A is in location #1, and location #2 is very close to location #1, it is likely that marker A will be present in a very similar relative position.  Also, if you have markers A, B, and C they will always be in that order.  Although it could be that you have A/B/C or A/C or B/C, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am including a hand-sketched example below with 4 example locations and one data stream (I normally have 4-5 data streams per location. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for guidance on the type of algorithm to apply to this problem.  I have explored Dynamic Time warping, but the issue is that with 10-20k data samples per location, and thousands of locations, the problem becomes computationally challenging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in general you may have 10000 locations, with maybe 100 that have been hand interpreted by an expert.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OskMQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OskMQ.jpg&quot; alt=&quot;hand sketch of example data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36189" LastEditorUserId="36189" LastEditDate="2020-04-21T12:34:09.167" LastActivityDate="2021-05-22T18:03:30.333" Title="What method to identify markers in data series via machine learning" Tags="&lt;pattern-recognition&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20395" PostTypeId="2" ParentId="20384" CreationDate="2020-04-19T12:31:57.617" Score="3" Body="&lt;p&gt;The deep Q-learning (DQL) algorithm is really similar to the tabular Q-learning algorithm. I think that both algorithms are actually quite simple, at least, if you look at their pseudocode, which isn't longer than 10-20 lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a screenshot of the pseudocode of DQL (from &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt;) that highlights the Q target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Eyy49.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Eyy49.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the screenshot of Q-learning (from &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barto and Sutton's book&lt;/a&gt;) that highlights the Q target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qgJdk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qgJdk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, the &lt;span class=&quot;math-container&quot;&gt;$\color{red}{\text{target}}$&lt;/span&gt; is a &lt;strong&gt;reward plus a discounted maximum future Q value&lt;/strong&gt; (apart from the exception of final states, in the case of DQL, where the target is just the reward). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are at least 3 differences between these two algorithms. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;DQL uses gradient descent because the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function are represented by neural networks rather than tables, like in Q-learning, and so you have an explicit loss function (e.g. MSE).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;DQL typically uses experience replay (but, in principle, you could also do this in Q-learning) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;DQL encodes the states (i.e. &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; encodes the states). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Apart from that, the logic of both algorithms is more or less the same, so, if you know Q-learning (and you should know it before diving into DQL), then it shouldn't be a problem to learn DQL (if you also have a decent knowledge of deep learning).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-19T23:59:45.223" LastActivityDate="2020-04-19T23:59:45.223" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20396" PostTypeId="2" ParentId="20377" CreationDate="2020-04-19T12:44:32.483" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;do I have to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;forward propagate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate error&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;calculate all gradients&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;...repeatedly over all samples in the batch, and then average all gradients and apply the weight change?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, that is correct. You can save a bit of memory by summing gradients as you go. Once you have calculated the gradients for one example for the weights of one layer, then you do not re-use the individual gradients again, so you can just keep a sum. Alternatively for speed, you can calculate a minibatch of gradients in parallel, as each example is independent - which is a major part of why GPU acceleration is so effective in neural network training.&lt;/p&gt;&#xA;&lt;p&gt;It is &lt;em&gt;critical&lt;/em&gt; to getting correct results that you calculate the gradient of the loss function with respect to each example input/output pair separately. Once you have done that, you can average the gradients across a batch or mini-batch to estimate a true gradient for the dataset which can be used to take a gradient descent step.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;recently I have read somewhere that you basically only average the error of each example in the batch, and then calculate the gradients at the end of each batch.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Without a reference it is hard to tell whether this is an error in the &amp;quot;somewhere&amp;quot;, or you have misunderstood, or there is a specific context.&lt;/p&gt;&#xA;&lt;p&gt;If by &amp;quot;error&amp;quot; you mean the literal difference &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i - y_i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt; is your estimate for data input &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is the ground-truth training value, then that &lt;em&gt;is&lt;/em&gt; the gradient for many loss functions and activation function pairs. For instance, it is the error gradient for mean square error and linear output. Some texts loosely refer to this as the &amp;quot;error&amp;quot;, and talk about backpropagating &amp;quot;the error&amp;quot;, but actually it is a gradient.&lt;/p&gt;&#xA;&lt;p&gt;In addition, if the article was referring to linear regression, logistic regression or softmax regression, everything else is linear - in those specific models then you can just &amp;quot;average the error&amp;quot; and use that as the gradient.&lt;/p&gt;&#xA;&lt;p&gt;In general, however, the statement is incorrect because a neural network with one or more hidden layers has many non-linearities that will give different results when calculating average first then backpropagating vs taking backpropagating first the averaging - that is &lt;span class=&quot;math-container&quot;&gt;$f'(\mu(Y))$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$\mu(f'(Y))$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$f'$&lt;/span&gt; is the derivative of the transfer function and &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the mean for the batch (i.e. &lt;span class=&quot;math-container&quot;&gt;$\mu(Y) = \frac{1}{N}\sum_{i=1}^{N} y_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; represents all the &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; in a given batch of size &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;p&gt;When &lt;span class=&quot;math-container&quot;&gt;$y_i = f(x_i) = ax_i +b$&lt;/span&gt; i.e. the transfer function is linear, then &lt;span class=&quot;math-container&quot;&gt;$f'(\mu(Y)) = \mu(f'(Y)) = \frac{a}{N}\sum_{i=1}^N x_i$&lt;/span&gt;, but almost all useful loss functions and all transfer functions except some output layers in neural networks are non-linear. For those, &lt;span class=&quot;math-container&quot;&gt;$f'(\mu(Y)) \neq \mu(f'(Y))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A simple example would show this, if we start a small minibatch back propagation with the loss function (as opposed to its gradient).&lt;/p&gt;&#xA;&lt;p&gt;Say you had the following data for regression:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  x    y&#xA;&#xA;  1    2&#xA;  1    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You want a model that can regress to least mean squared error &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; when given an input &lt;span class=&quot;math-container&quot;&gt;$x = 1$&lt;/span&gt;. The best model should predict &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; in that case.&lt;/p&gt;&#xA;&lt;p&gt;If your model has converged, the average MSE of the dataset is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Using that would make your model move &lt;em&gt;away&lt;/em&gt; from convergence and it will perform worse.&lt;/p&gt;&#xA;&lt;p&gt;If you first take the gradients, then average those, you will calculate &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. A simple gradient update step using that value will make no change, leaving the model in the optimal position.&lt;/p&gt;&#xA;&lt;p&gt;This issue occurs on every hidden layer in a neural network, so in general you cannot simply resolve the loss function gradient and start with the average error gradient at the output. You would still hit the inequality &lt;span class=&quot;math-container&quot;&gt;$f'(\mu(Y)) \neq \mu(f'(Y))$&lt;/span&gt; on each nonlinearly.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-19T17:48:37.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20397" PostTypeId="2" ParentId="20378" CreationDate="2020-04-19T12:50:58.757" Score="1" Body="&lt;p&gt;It is an error. But it is also not the in final version of the paper (arxiv.) The final version of the paper can be found &lt;a href=&quot;http://proceedings.mlr.press/v28/pascanu13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; where they replace &quot;absolute value of the largest &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors&quot; rel=&quot;nofollow noreferrer&quot;&gt;eigenvalue&lt;/a&gt;&quot; with &quot;largest &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;singular value&lt;/a&gt;&quot;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We first prove that it is sufficient for &lt;span class=&quot;math-container&quot;&gt;$\lambda_1 &amp;lt; \frac{1}{\gamma}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\lambda_1$&lt;/span&gt; is the largest singular value of &lt;span class=&quot;math-container&quot;&gt;$W_{rec}$&lt;/span&gt;, for the vanishing gradient problem to occur.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="34395" LastEditorUserId="34395" LastEditDate="2020-04-19T12:56:22.363" LastActivityDate="2020-04-19T12:56:22.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20398" PostTypeId="2" ParentId="20388" CreationDate="2020-04-19T13:03:35.853" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But, the literature (i.e. Learning from Data) states that VC gives a loose bound and that in real applications, learning models with lower VC dimension tend to generalize better than those with higher VC dimension.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's true that people often use techniques such as &lt;em&gt;regularisation&lt;/em&gt; to avoid over-parametrized models. However, I think it's dangerous to say that, in real applications, those models are really generalizing better, given that you're typically assessing their generalization ability by using a finite validation dataset (possibly selected in a biased way). Moreover, note that the validation dataset is often ignored in certain bounds and the only thing that is taken into account is the expected risk and the empirical risk (on the training data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, the &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; bounds may be &quot;loose&quot; because&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}$&lt;/span&gt; dimension is often expressed with the big-&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}$&lt;/span&gt; notation (e.g. see &lt;a href=&quot;https://ai.stackexchange.com/a/17881/2444&quot;&gt;this answer&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;the bounds often involve probabilities and uncertainties (e.g. see &lt;a href=&quot;https://ai.stackexchange.com/a/20341/2444&quot;&gt;this other answer&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;there aren't stricter bounds (i.e. no one found better bounds yet)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the VC generalization bound loose due to its universality? Meaning, its results apply to all hypothesis sets, learning algorithms, input spaces, probability distributions, and binary target functions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the answer is yes. In fact, often, the specific tasks or learning algorithms are ignored in the analyses, but, in general, you may exploit your knowledge of the problem to improve e.g. the number of useful examples that the learning algorithm could use.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-19T13:15:08.237" LastActivityDate="2020-04-19T13:15:08.237" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20399" PostTypeId="2" ParentId="20054" CreationDate="2020-04-19T17:59:09.820" Score="1" Body="&lt;p&gt;I can see several challenges, and the list below is not exhaustive:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i. The main problem is how to model a problem of translating a language test into a formal language. It will probably be something like the automatic translators, but with some guarantees that the proof semantics will be preserved. If you are more interested in this path, I recommend researching what PAC, Information Theory, Computational Proof theory, Complexity theory can contribute to this modeling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii. Another problem is how to get the data reliable. You commented that as people used it they would generate this data. But the problem is not just collecting the data. How much you will trust the data and how you will measure the model's performance in translation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iii. Another problem is more humane, how do you get mathematicians to use such a system? And how to make the model self-explainable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that this is one of the most difficult problems in machine learning. I once saw this &lt;a href=&quot;https://youtu.be/ZhmjCPdZwxk&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; a while ago and I don't know if it can help anything. I also recommend the &lt;a href=&quot;https://cstheory.stackexchange.com/&quot;&gt;stack exchange of theoretical computer science&lt;/a&gt;, there you will probably have a more complete answer.&lt;/p&gt;&#xA;" OwnerUserId="36175" LastActivityDate="2020-04-19T17:59:09.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20400" PostTypeId="2" ParentId="7416" CreationDate="2020-04-19T18:10:38.073" Score="4" Body="&lt;p&gt;Not in such straight forward way as described, but neural networks are successfully applied to guide the search of proof. There are automated theorem provers. What they do look roughly like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Get the mathematical statement&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply one of the known mathematical equivalence transformations (theorems, axioms, etc)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check, if the resulting statement is trivially true. Then our sequence of transformations is the proof (since they all were equivalence transformations). Else, goto 2.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The tricky part here is to choose which transformation to apply at step 2. A neural network can be trained to predict function like&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Statement, Transformation --&gt; usefulness of that transformation to that statement&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then, during the search, we can apply such transformation, that neural network considers the most useful. Also, proving a theorem can be considered game, where axioms are the rules, and when you've reached the proof you win. In this form, Reinforcement Learning agents can be applied to prove theorems (this is also successfully done).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are papers that do similar things: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1907.07794.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generating Correctness Proofs with Neural Networks&lt;/a&gt; (&lt;a href=&quot;https://github.com/UCSD-PL/proverbot9001&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proverbot9001&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1806.00608.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Learning Environment for Theorem Proving&lt;/a&gt; (&lt;a href=&quot;https://github.com/ml4tp/gamepad&quot; rel=&quot;nofollow noreferrer&quot;&gt;GamePad&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1804.00596.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Prove with Tactics&lt;/a&gt; (TacticToe)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1905.09381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Prove Theorems via Interacting with Proof Assistants&lt;/a&gt; (&lt;a href=&quot;https://github.com/princeton-vl/CoqGym&quot; rel=&quot;nofollow noreferrer&quot;&gt;CoqGym&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v97/bansal19a/bansal19a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;HOList: An Environment for Machine Learning of Higher-Order Theorem Proving&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://papers.nips.cc/paper/8098-reinforcement-learning-of-theorem-proving.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning of Theorem Proving&lt;/a&gt; (rlCoP)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36199" LastEditorUserId="36199" LastEditDate="2020-04-20T20:17:17.637" LastActivityDate="2020-04-20T20:17:17.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20401" PostTypeId="1" CreationDate="2020-04-19T21:07:24.320" Score="1" ViewCount="63" Body="&lt;p&gt;I am trying to understand the &lt;a href=&quot;https://arxiv.org/abs/1612.00593&quot; rel=&quot;nofollow noreferrer&quot;&gt;PointNet&lt;/a&gt; network for dealing with point clouds and struggling with understanding the difference between FC and MLP: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;FC is fully connected layer operating on each point. MLP is&#xA;  multi-layer perceptron on each point.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand how &lt;a href=&quot;https://missinglink.ai/guides/convolutional-neural-networks/fully-connected-layers-convolutional-neural-networks-complete-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fully connected&lt;/a&gt; layers are used to classify and I previously thought, was that MLP was the same thing but it seems varying academic papers have a differing definition from each other and from general online courses. In PointNet what is meant by a shared MLP different to a standard feedforward fully connected network?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vDWpQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vDWpQ.png&quot; alt=&quot;PointNet Network Architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36197" LastEditorUserId="36197" LastEditDate="2020-04-20T12:52:35.323" LastActivityDate="2020-04-20T12:52:35.323" Title="What is the difference between FC and MLP in as used in PointNet?" Tags="&lt;convolutional-neural-networks&gt;&lt;models&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20402" PostTypeId="2" ParentId="20382" CreationDate="2020-04-19T23:14:11.527" Score="1" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://aboumoustafa.org/KAM_ECML2015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalization in Unsupervised Learning&lt;/a&gt; (2015), &lt;a href=&quot;http://aboumoustafa.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abou-Moustafa&lt;/a&gt; and Schuurmans develop an approach to assess the generalization of an unsupervised learning algorithm &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; on a given dataset &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and how to compare the generalization ability of two unsupervised learning algorithms &lt;span class=&quot;math-container&quot;&gt;$A_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A_2$&lt;/span&gt;, for the same learning task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They first provide a more abstract and general definition of an unsupervised learning algorithm and loss function. Then they define the expected risk, empirical risk and generalization gap in a similar way to the case of supervised learning. Finally, they derive an upper bound on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;'s expected loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you should read the paper for more details. Specifically, section 2 (page 3) describes their setting in detail.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-19T23:31:09.580" LastActivityDate="2020-04-19T23:31:09.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20403" PostTypeId="1" CreationDate="2020-04-19T23:28:19.140" Score="1" ViewCount="77" Body="&lt;p&gt;I am new to RL and wish to realize a RL control for an industrial process. The goal is to control the temperature and humidity in a vegetal food production chamber. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;States:&lt;/strong&gt; External temperature and humidity, internal temperature and humidity, percentage of the proportional valves controlling heater, cooler and steam for humidity. The goal is to keep temperature and humidity in the chamber (measures) as close as possible to the desired values (the setpoints). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Agent actions:&lt;/strong&gt; Increase/decrease the percentage of the proportional valves controlling the actuators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rewards:&lt;/strong&gt; Deviation between measure and setpoints (small deviation =&gt; high reward, high deviation =&gt; low reward).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data available, the history of states and actions from a real system. The actions are made by several PID controllers (some of them in cascade). So far I have about 3 month every minutes (with some stops sometimes when a chamber is for example cleaned). The data are continuously logged and every month I get more data. The data includes bad/unwanted states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For training the RL agent, I am planning to simulate the environment using a supervised learning model (with the predict function), probably XGboost. Is it feasible, are there pitfalls to avoid in this case?&lt;/p&gt;&#xA;" OwnerUserId="36210" LastEditorUserId="1847" LastEditDate="2020-04-20T17:06:37.397" LastActivityDate="2021-05-22T22:04:15.497" Title="Reinforcement learning with industrial continuous process" Tags="&lt;reinforcement-learning&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="20404" PostTypeId="1" AcceptedAnswerId="20511" CreationDate="2020-04-20T01:11:10.727" Score="4" ViewCount="975" Body="&lt;p&gt;I'm seeking guidence here.&#xA;Can I use Multi Layers Perceptron (MLP), e.g regular flat neural networks, for image classification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will they perform better than Fisher Faces?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it difficult to do image classification with a MLP network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's on basic level like classifying objects and not detailed structures and patterns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Important to me is that the MLP need to be trained with pictures that can have noise in background and different light shadows.&lt;/p&gt;&#xA;" OwnerUserId="36211" LastEditorUserId="9543" LastEditDate="2020-04-21T12:07:58.627" LastActivityDate="2020-04-22T14:54:32.800" Title="Can I do image classification with Multi Layers Perceptron (MLP)?" Tags="&lt;classification&gt;&lt;image-recognition&gt;&lt;multi-label-classification&gt;" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20405" PostTypeId="1" CreationDate="2020-04-20T05:08:29.453" Score="2" ViewCount="298" Body="&lt;p&gt;&amp;quot;&lt;a href=&quot;https://www.youtube.com/watch?v=kaahx4hMxmw&amp;amp;ab_channel=HelixNebula&quot; rel=&quot;nofollow noreferrer&quot;&gt;If you can't tell, does it matter?&lt;/a&gt;&amp;quot; was one of the first lines of dialogue of the Westworld television series, presented as a throwaway in the &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Original_(Westworld)&quot; rel=&quot;nofollow noreferrer&quot;&gt;first episode of the first season&lt;/a&gt;, in response to the question &amp;quot;Are you real?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://en.wikipedia.org/wiki/Decoherence_(Westworld)&quot; rel=&quot;nofollow noreferrer&quot;&gt;sixth episode of the third season&lt;/a&gt;, the line becomes a central realization of one of the main characters, &lt;a href=&quot;https://westworld.fandom.com/wiki/Man_in_Black&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Man in Black&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;This is, in fact, a central premise of the show—what is reality, what is identity, what does it mean to be alive?—and has a basis in the philosophy of Philip K. Dick.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the implications of this statement in relation to AI?  In relation to experience?  In relation to the self?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2021-01-17T12:49:57.850" LastActivityDate="2021-05-10T00:52:14.827" Title="What are the implications of the statement &quot;If you can't tell, does it matter?&quot; in relation to AI?" Tags="&lt;philosophy&gt;&lt;artificial-consciousness&gt;&lt;turing-test&gt;&lt;mythology-of-ai&gt;&lt;chinese-room-argument&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20407" PostTypeId="2" ParentId="6274" CreationDate="2020-04-20T05:49:49.330" Score="1" Body="&lt;p&gt;Try resizing the image to the input dimensions of your neural network architecture(keeping it fixed to something like 128*128 in a standard 2D U-net architecture) using &lt;strong&gt;nearest neighbor interpolation&lt;/strong&gt; technique. This is because if you resize your image using any other interpolation, it may result in tampering with the ground truth labels. This is particularly a problem in segmentation. You won't face such a problem when it comes to classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try the following:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cv2 &#xA;resized_image = cv2.resize(original_image, (new_width, new_height), &#xA;                           interpolation=cv2.INTER_NEAREST)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36213" LastEditorUserId="26652" LastEditDate="2020-04-21T18:32:01.950" LastActivityDate="2020-04-21T18:32:01.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20408" PostTypeId="1" CreationDate="2020-04-20T05:53:45.007" Score="0" ViewCount="41" Body="&lt;p&gt;I am trying to use a neural network to predict the next state output given the current state and action pairs. Both input and outputs are continuous variables. Due to the high dimensionality of each input, ( ~50 dimensional input ) and 48 dimensional output, I am not able to achieve an achieve a satisfiable enough accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of using an auto-encoder to learn a latent representation of the state. Would a latent representation from an auto-encoder help to improve the prediction accuracy ? and can the latent representation have a higher dimensional space compared to the original state ?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-04-21T12:05:24.023" Title="Can I use an autoencoder with high latent representational space?" Tags="&lt;deep-learning&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20409" PostTypeId="1" CreationDate="2020-04-20T07:00:59.920" Score="0" ViewCount="86" Body="&lt;p&gt;I've been messing around with an Open Set, Binary Classifier and am having trouble with it.  I'm sure there are a lot of reasons for that trouble.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I am struggling with is, what does the model predict if it has never seen the image before? An example would be if I'm trying to detect sheep across all background scenes.  If I train a binary classification set with one class having lots of sheep in it and the other class having lots of various backgrounds, what would the model predict if it came across a background it had never seen before with no sheep in it?  [mine is telling me &quot;sheep&quot; and I don't know why]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, Mecho&lt;/p&gt;&#xA;" OwnerUserId="36092" LastActivityDate="2020-04-21T11:20:51.347" Title="What does a CNN classifier default to?" Tags="&lt;convolutional-neural-networks&gt;&lt;ai-basics&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20410" PostTypeId="1" CreationDate="2020-04-20T09:30:24.590" Score="0" ViewCount="117" Body="&lt;p&gt;I am currently working on a problem for which the topographic data is in very different resolution. Let say I have data of 20x20 with &lt;strong&gt;1km2&lt;/strong&gt; tiles and also high resolution data of &lt;strong&gt;50m2&lt;/strong&gt; tiles. I would like to combine both for input in a CNN. &#xA;To make things more spicy I don't care about the 50m2 when it is far away from the center, that is why I would like to use an 'image' multi resolution, aka resolution low in the edges but higher in the center. That would be like human vision, only high detailed in the center...&#xA;Then I would combine that multi-resolution image with my 1km2 data &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any research done on such a CNN ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only found that one for now &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=9&amp;amp;ved=2ahUKEwjpjs3F1PboAhUDixoKHdu4BNIQFjAIegQIBxAB&amp;amp;url=https%3A%2F%2Fwww.mdpi.com%2F2072-4292%2F10%2F10%2F1636%2Fpdf&amp;amp;usg=AOvVaw1qXJNu_ujWjVLgLsYJ1vaA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Resolution Feature Fusion for Image&#xA;Classification of Building Damages with&#xA;Convolutional Neural Networks&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for you help&lt;/p&gt;&#xA;" OwnerUserId="36217" LastActivityDate="2020-04-21T10:09:04.467" Title="Do Multi-resolution CNN exist?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20411" PostTypeId="1" CreationDate="2020-04-20T13:04:08.400" Score="2" ViewCount="135" Body="&lt;p&gt;Apart from &lt;a href=&quot;https://content.sciendo.com/view/journals/jagi/jagi-overview.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal of Artificial General Intelligence&lt;/a&gt; (a peer-reviewed open-access academic journal, owned by the &lt;a href=&quot;http://www.agi-society.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence Society (AGIS)&lt;/a&gt;), are there any other journals (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Conference_proceeding&quot; rel=&quot;nofollow noreferrer&quot;&gt;proceedings&lt;/a&gt;) completely (or partially) dedicated to &lt;em&gt;artificial &lt;strong&gt;general&lt;/strong&gt; intelligence&lt;/em&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to share a journal that is only partially dedicated to the topic, please, provide details about the relevant subcategory or examples of papers on AGI that were published in such a journal. So, a paper that talks about e.g. an RL technique (that &lt;em&gt;only&lt;/em&gt; claims that the idea could be useful for AGI) is not really what I am looking for. I am looking for journals where people publish papers, reviews or surveys that develop or present theories and implementations of AGI systems. It's possible that these journals are more associated with the cognitive science or neuroscience communities and fields.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-20T13:44:23.733" LastActivityDate="2020-07-12T14:44:31.863" Title="What are the scientific journals dedicated to artificial general intelligence?" Tags="&lt;agi&gt;&lt;research&gt;&lt;cognitive-science&gt;&lt;academia&gt;&lt;neuroscience&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20412" PostTypeId="1" CreationDate="2020-04-20T13:53:13.500" Score="2" ViewCount="100" Body="&lt;p&gt;I wanted to ask you about the newest achievements in time series analysis (mostly prediction).&#xA;What state-of-the-art solutions (as in frameworks, papers, related projects) do you know that can be used for analysing and predicting time series? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in something possibly better than just RNN, LSTM and GRU :)&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2020-05-05T19:48:04.937" LastActivityDate="2020-05-19T12:51:15.070" Title="What are modern state-of-the-art solutions in prediction of time-series?" Tags="&lt;recurrent-neural-networks&gt;&lt;prediction&gt;&lt;time-series&gt;&lt;state-of-the-art&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20414" PostTypeId="2" ParentId="2996" CreationDate="2020-04-20T14:42:12.550" Score="0" Body="&lt;p&gt;A key question that remains in the theory of deep learning is why such huge models (with many more parameters than data points) don't overfit on the datasets we use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classical theory based on complexity measures does not explain the behaviour of practical neural networks. For instance estimates of VC dimension give vacuous generalisation bounds. As far as I know, the tightest (upper and lower) bounds on the VC dimension are given in [1] and are on the order of the number of weights in the network. Clearly this worst case complexity cannot explain how e.g. a big resnet generalises on CIFAR or MNIST. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently there have been other attempts at ensuring generalisation for neural networks, for instance by relation to the neural tangent kernel or by various norm measures on the weights. Respectively, these have been found to not apply to practically sized networks and to have other unsatisfactory properties [2]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some work in the PAC Bayes framework for non-vacuous bounds, e.g. [3]. These setups, however, require some knowledge of the trained the network and so are different in flavour to the classical PAC analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other aspects:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;optimisation: how come we get 'good' solutions from gradient descent on such a non-convex problem? (There are some answers to this in recent literature)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;interpretability: Can we explain on an intuitive level what the network is 'thinking'? (Not my area)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(incomplete) references:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[1] &lt;a href=&quot;https://arxiv.org/abs/1703.02930&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1703.02930&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[2] &lt;a href=&quot;http://papers.nips.cc/paper/9336-uniform-convergence-may-be-unable-to-explain-generalization-in-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://papers.nips.cc/paper/9336-uniform-convergence-may-be-unable-to-explain-generalization-in-deep-learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[3] &lt;a href=&quot;https://arxiv.org/abs/1804.05862&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1804.05862&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22613" LastActivityDate="2020-04-20T14:42:12.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20415" PostTypeId="2" ParentId="2996" CreationDate="2020-04-20T15:52:51.273" Score="0" Body="&lt;p&gt;I'd like to point out there isn't a good theory on why machine learning works in general. VC bounds still assume a model, but reality doesn't fit any of these mathematical ideals. Ultimately when it comes to application everything comes down to emperical results. Even quantifying the similarity between images using an algorithm which is consistent with humans intuitive understanding is really hard &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway NN dont work well in their fully connected form. All successful networks have some kind of regularization built into the network architecture (CNN, LSTM, etc).&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-20T15:52:51.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20417" PostTypeId="1" CreationDate="2020-04-20T16:29:54.137" Score="2" ViewCount="200" Body="&lt;p&gt;I am a newbie in deep learning and wanted to know if the problem I have at hand is a suitable fit for deep learning algorithms. I have thousands of fragments each of about 1000 bytes size (i.e. numbers in the range of 0 to 255). There are two classes in the fragments:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Some fragments have a high frequency of two particular byte values appearing next to one another: &quot;0 and 100&quot;. This kind of pattern roughly appears once every 100 to 200 bytes.&lt;/li&gt;&#xA;&lt;li&gt;In the other class, the byte values are more randomly distributed. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We have the ability to produce as many numbers of instances of each class as needed for training purposes.  However, I would like to differentiate with a machine learning algorithm without explicitly identifying the &quot;0 and 100&quot; pattern in the 1st class myself. Can deep learning help us solve this? If so, what kind of layers might be useful?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a preliminary experiment, we tried to train a deep learning network made up of 2 hidden layers of TensorFlow's &quot;Dense&quot; layers (of size 512 and 256 nodes in each of the hidden layers). However, unfortunately, our accuracy was indicative of simply a random guess (i.e. 50% accuracy). We were wondering why the results were so bad. Do you think a Convolutional Neural Network will better solve this problem? &lt;/p&gt;&#xA;" OwnerUserId="36091" LastEditorUserId="36091" LastEditDate="2020-04-20T20:11:37.507" LastActivityDate="2021-05-16T00:05:30.497" Title="Finding patterns in binary files using deep learning" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20418" PostTypeId="1" AcceptedAnswerId="20421" CreationDate="2020-04-20T17:18:35.407" Score="3" ViewCount="71" Body="&lt;p&gt;The state value function &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; is defined as the expected return starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and acting according to the current policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; till the end of the episode. The state-action values &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; are similarly dependent on the current policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it also possible to get a &lt;strong&gt;policy independent value of a state or an action&lt;/strong&gt;? &#xA;Can the immediate reward &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; be considered a noisy estimate of the action value?&lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-04-21T08:03:41.713" Title="Do policy independent state and action values exist in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20419" PostTypeId="1" AcceptedAnswerId="20422" CreationDate="2020-04-20T18:37:08.450" Score="4" ViewCount="743" Body="&lt;p&gt;I understand that SARSA is an On-policy algorithm, and Q-learning an off-policy one.&#xA;Sutton and Barto's textbook describes Expected Sarsa thusly:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In these cliff walking results Expected Sarsa was used on-policy, but&#xA;  in general it might use a policy different from the target policy to&#xA;  generate behavior, in which case it becomes an off-policy algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am fundamentally confused by this - specifically, how do we define when Expected SARSA adopts or disregards policy. The Coursera Course states that it is On-Policy, further confusing me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusions became realized when tackling the Udacity course, specifically a section visualizing Expected SARSA for simple a gridworld (See section 1.11 and 1.12 in link below). Note that the course defines Expected Sarsa as on-policy.&#xA;&lt;a href=&quot;https://www.zhenhantom.com/2019/10/27/Deep-Reinforcement-Learning-Part-1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.zhenhantom.com/2019/10/27/Deep-Reinforcement-Learning-Part-1/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll notice the calculation for the new state value Q(s0,a0) as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Q(s0, a0) &amp;lt;— 6 + 0.1( -1 + [0.1 x 8] + [0.1 x 7] + [0.7 x 9] + [0.1&#xA;  x 8] - 6) = 6.16.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is also the official answer. But this would mean that it is running off policy, given that it is stated that the action taken at S1 corresponds to a shift right, and hence expected SARSA (On policy) should yield you.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Q(s0, a0) &amp;lt;— 6 + 0.1( -1 + [0.1 x 8] + [0.1 x 7] + [0.1 x 9] + [0.7 x&#xA;  8] - 6) = 6.1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The question does state&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(Suppose that when selecting the actions for the first two timesteps&#xA;  in the 100th episode, the agent was following the epsilon-greedy&#xA;  policy with respect to the Q-table, with epsilon = 0.4.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But as this same statement existed for the regular SARSA example (which also yields 6.1 as A1 is shift right, as before), I disregarded it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice is welcome.&lt;/p&gt;&#xA;" OwnerUserId="36228" LastEditorUserId="2444" LastEditDate="2020-04-20T19:35:20.460" LastActivityDate="2020-04-20T19:35:20.460" Title="Is Expected SARSA an off-policy or on-policy algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;sarsa&gt;&lt;on-policy-methods&gt;&lt;expected-sarsa&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20421" PostTypeId="2" ParentId="20418" CreationDate="2020-04-20T18:42:28.277" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do policy independent state and action values exist in reinforcement learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. They do not exist, because in order to progress in any MDP and receive any reward - i.e. to get any measure of value - you must take an action. Any consistent means of selecting actions is a policy, and the nature of that policy impacts which transitions and rewards you expect to observe, which in turn affects the expected value. Inconsistent means of selecting actions would have no meaning with respect to &quot;expected future return&quot;, they would just be measurements that you made in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest you can get to a no-policy definition would be values with respect to &quot;special&quot; policies that apply in general to nearly all MDPs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Value functions for the uniformly distributed random policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value functions for any optimal policy (if there is more than one optimal policy, then all the value functions for them will be equal).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value functions for any &quot;inverse optimal&quot; policy - i.e. the policy that has lowest possible return. This one is not so useful, although it exists theoretically.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first two can be useful measurements of an MDP. Although the uniform random policy might not be the best, it encapsulates the situation where the agent has absolutely no knowledge of the MDP, and can be a baseline for comparison. The optimal value functions are often a target for learning algorithms, and sometimes you can calculate bounds or even exact targets for these independently of the learning process, to measure how well an algorithm performs on some test MDP.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the immediate reward &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; be considered a noisy estimate of the action value?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. Using that notation of the function, it is typically already the expected immediate reward. It is entirely independent of reward seen in any other transitions or time steps, so is systematically incorrect as an estimate for future return in many cases - the only exception being if you know all future rewards will be precisely &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. So it is an unbiased estimate of an action value if &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is a terminal state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Immediate reward is also a good estimate of an action value if the discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt;. However, that requires you to define the problem as solving for immediate rewards only, which is not usually a free choice when trying to optimise the behaviour of the agent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-21T08:03:41.713" LastActivityDate="2020-04-21T08:03:41.713" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20422" PostTypeId="2" ParentId="20419" CreationDate="2020-04-20T19:03:17.537" Score="2" Body="&lt;p&gt;Expected SARSA can be used either on-policy or off-policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy that you use in the update step determines which it is. If the update step uses a different weighting for action choices than the policy that actually took the action, then you are using Expected SARSA in an off-policy way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-learning is a special case of Expected SARSA, where the target policy is greedy with respect to the action values, so there is only ever one &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma \text{max}_{a'} Q(s_{t+1}, a')$&lt;/span&gt; term to add with a probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use Expected SARSA, similarly to SARSA, where the behaviour policy and target policy are identical. It is not identical to SARSA though, because it calculates the TD Target over all possible actions &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma \sum_{a'} \pi(a'|s_{t+1}) Q(s_{t+1}, a')$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can construct Expected SARSA updates where &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; is different when selecting which action to explore in the environment (behaviour) and when updating the Q values (target). For instance, you can decide to explore using &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy with &lt;span class=&quot;math-container&quot;&gt;$\epsilon=0.1$&lt;/span&gt; and update the value function with &lt;span class=&quot;math-container&quot;&gt;$\epsilon=0.01$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-04-20T19:03:17.537" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20423" PostTypeId="2" ParentId="20417" CreationDate="2020-04-20T20:18:17.217" Score="0" Body="&lt;p&gt;Your network is essentially memorizing data but not extracting features. You need to apply CNN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said the CNN architecture will need to be kind of unusual, each bit will need to be turned into representing a positional element. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-20T20:18:17.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20424" PostTypeId="2" ParentId="20081" CreationDate="2020-04-21T07:43:41.283" Score="0" Body="&lt;p&gt;I actually pondered this question a few months ago, so i understand your point of view!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct in assuming that if you already build your tree or calculate your probabilities, what is the point of using test data? Because your model is fixed the way they are, no matter if you use test data or not. Well, the purpose of test data is not only to test your model against unseen data and get some evaluation score. But it is also to test if your model is the right fit for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the main reason why we all build ML/AI model in the first place is to extract insights that can be used to solve problems, make decisions etc. If you don't test your naive bayes or decision tree model with test data, you won't know if the information that are given to you by those model mean anything. They may not even help you solve problems or give you relevant information. Yes they may spout out big numbers and classify things, but are those result what you're looking for? Are the result relevant to your problem? Can the result be used to solve what you are trying to do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using test data gives you the opportunity to see if your model gives you the best insights and the best solution to your problems. So here are the takeaways from my answer:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Test data can be used to test your model against unseen data&lt;/li&gt;&#xA;&lt;li&gt;You can gain score (evaluation) on your model when you test it with test data, which in turn can be used to fine tune your model&lt;/li&gt;&#xA;&lt;li&gt;You can see if the answer the model gives you is any relevant to your initial problem. It its not, then it may be best to use some other algorithm.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36257" LastActivityDate="2020-04-21T07:43:41.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20425" PostTypeId="2" ParentId="7202" CreationDate="2020-04-21T08:01:41.357" Score="1" Body="&lt;p&gt;I think you should use a linear kernel, 'cause training SVM with a linear kernel is faster than with another kernel, especially for text classification. Good luck &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.svm-tutorial.com/2014/10/svm-linear-kernel-good-text-classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.svm-tutorial.com/2014/10/svm-linear-kernel-good-text-classification/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36253" LastActivityDate="2020-04-21T08:01:41.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20426" PostTypeId="2" ParentId="20321" CreationDate="2020-04-21T08:03:06.277" Score="4" Body="&lt;p&gt;As far as I know, there are few &lt;strong&gt;aspects&lt;/strong&gt; that would probably improve the model score:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Normalization&lt;/li&gt;&#xA;  &lt;li&gt;Lemmatization&lt;/li&gt;&#xA;  &lt;li&gt;Stopwords removal (as you asked here)&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on your question, &quot;is removing top frequent words (stopwords) will improve the model score?&quot;. The answer is, it depends on what kind of stopwords are you removing. The problem here is that if you do not remove stop words, the noise will increase in the dataset because of words like I, my, me, etc. Here is the comparison of those three &lt;strong&gt;aspects&lt;/strong&gt; using &lt;strong&gt;SVM Classifier&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BQavF.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BQavF.png&quot; alt=&quot;Comparison using SVM Classifier&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may see that without stopwords removal the Train Set Accuracy decreased to &lt;strong&gt;94.81%&lt;/strong&gt; and the Test Set Accuracy decreased to &lt;strong&gt;88.02%&lt;/strong&gt;. But, you should be careful about what kind of stopwords you are removing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are working with basic NLP techniques like BOW, Count Vectorizer or TF-IDF(Term Frequency and Inverse Document Frequency) then removing stopwords is a good idea because stopwords act like noise for these methods. If you working with LSTM’s or other models which capture the semantic meaning and the meaning of a word depends on the context of the previous text, then it becomes important not to remove stopwords. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what's the solution?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You may want to create a Python package &lt;strong&gt;nlppreprocess&lt;/strong&gt; which removes stops words that are not necessary. It also has some additional functionalities that can make cleaning of text fast. For example:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from nlppreprocess import NLP&#xA;import pandas as pd&#xA;&#xA;nlp = NLP()&#xA;df = pd.read_csv('some_file.csv')&#xA;df['text'] = df['text'].apply(nlp.process)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/miguelfzafra/Latest-News-Classifier&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/miguelfzafra/Latest-News-Classifier&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/why-you-should-avoid-removing-stopwords-aa7a353d2a52&quot; rel=&quot;noreferrer&quot;&gt;https://towardsdatascience.com/why-you-should-avoid-removing-stopwords-aa7a353d2a52&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36240" LastActivityDate="2020-04-21T08:03:06.277" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20427" PostTypeId="2" ParentId="19985" CreationDate="2020-04-21T08:15:52.780" Score="1" Body="&lt;p&gt;Artificial Intelligence can predict such a thing because before they release the game, the train the BOT or AI to play the game million times so they have a model or BOT that can predict every next move or combo that they can do or predict all moves that can finish the game.&#xA;for example snake game. what they do to predict moves is train the model or bot to play the game when the snake perform some action . the snake got reward which can be positive or negative reward. the goal of the snake is to learn what action that maximize the reward, given every possible state. States are the observations that the agent receives at each iteration from the environment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is the link that can give you the detail : &lt;a href=&quot;https://towardsdatascience.com/how-to-teach-an-ai-to-play-games-deep-reinforcement-learning-28f9b920440a&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/how-to-teach-an-ai-to-play-games-deep-reinforcement-learning-28f9b920440a&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36239" LastEditorUserId="36239" LastEditDate="2020-04-21T08:32:44.620" LastActivityDate="2020-04-21T08:32:44.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20429" PostTypeId="2" ParentId="19985" CreationDate="2020-04-21T08:24:34.907" Score="1" Body="&lt;p&gt;AI is trained to predict such thing because that is their purpose, they are given almost all possibilities of move they can do to current state of the game and chose the best possible outcome of the possible move, but not only that the AI also predict what happens after that and predict the outcome of that prediction, just like a chess AI that can predict how to checkmate a player just by one move made by the player, so they did not just predict what move to do now but also what move to do after that move has been done&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this can be done with deep learning as you can read here : &lt;a href=&quot;https://towardsdatascience.com/predicting-professional-players-chess-moves-with-deep-learning-9de6e305109e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/predicting-professional-players-chess-moves-with-deep-learning-9de6e305109e&lt;/a&gt;&#xA;&lt;a href=&quot;https://electronics.howstuffworks.com/chess1.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://electronics.howstuffworks.com/chess1.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36259" LastActivityDate="2020-04-21T08:24:34.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20431" PostTypeId="2" ParentId="19985" CreationDate="2020-04-21T08:29:40.603" Score="1" Body="&lt;p&gt;In video games, usually the developer spend an dedicated time to train their AI by feeding them with learning data, provided by either the developer itself or the feedback from open/closed beta tester that participated, and from that data, the developer can model the learning pattern for the algorithm and proceed to train them with some sets of goal.&lt;/p&gt;&#xA;" OwnerUserId="36271" LastActivityDate="2020-04-21T08:29:40.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20433" PostTypeId="2" ParentId="20409" CreationDate="2020-04-21T08:32:56.430" Score="1" Body="&lt;p&gt;I am assuming the images you gave the model all contain sheep. This is what i understand from your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any model that you build will be based on the data that you give (training data) and your code. In your case, if you only give images that contains sheeps, and then you test it with no sheep and a background the model hasn't seen, it will search through all of its node, derived from your training data, to see which one is closest to the image you gave. Based on the information given, the only 'route' your cnn model can take is the one with sheeps on it, because you only gave images of sheeps for it to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few suggestions that i can give you:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Give your model images with no sheep and different backgrounds so it can handle cases where there are no sheeps&lt;/li&gt;&#xA;&lt;li&gt;Or you can add a piece of code to your model that tells it to default to some value if certain conditions aren't met (If the model doesn't 'see enough' sheep and background, it defaults to 'unknown image' or whatever you choose)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You are on the right path though! Since you are playing around with a binary classifier, you should definitely feed images with and without sheeps so that it can identify the two cases. Remember, a binary classifier is optimal when you give it 2 things to look out for, such as identifying images with and without sheep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some reading materials that you can brush up on to get a better basic understanding of how CNN works, personally i found the video helpful on the second link:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://becominghuman.ai/what-exactly-does-cnn-see-4d436d8e6e52&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://becominghuman.ai/what-exactly-does-cnn-see-4d436d8e6e52&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://brohrer.github.io/how_convolutional_neural_networks_work.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://brohrer.github.io/how_convolutional_neural_networks_work.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, i find google images helps me alot in terms of visualizing binary classifier as a concept.&lt;/p&gt;&#xA;" OwnerUserId="36257" LastEditorUserId="36257" LastEditDate="2020-04-21T11:20:51.347" LastActivityDate="2020-04-21T11:20:51.347" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20434" PostTypeId="2" ParentId="20321" CreationDate="2020-04-21T08:46:51.947" Score="1" Body="&lt;p&gt;Based on my project, there is how i clean and doing some preparation on the data.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Delete specific charaters ('\r', '\n', '&quot;',) &lt;/li&gt;&#xA;&lt;li&gt;Change into the lowercase&lt;/li&gt;&#xA;&lt;li&gt;Delete some symbols&lt;/li&gt;&#xA;&lt;li&gt;Lemmatization (change base word with wordnet)&lt;/li&gt;&#xA;&lt;li&gt;Delete stopwords.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With these following step, i get some improvement accuracy score on my model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My project: &lt;a href=&quot;https://github.com/khaifagifari/NLP-Course-TelU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/khaifagifari/NLP-Course-TelU&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36273" LastActivityDate="2020-04-21T08:46:51.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20435" PostTypeId="2" ParentId="20193" CreationDate="2020-04-21T08:54:43.023" Score="1" Body="&lt;p&gt;Classification can be performed on structured or unstructured data. Classification is a technique where we categorize data into a given number of classes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my project in price classification, when i compared into the 5 models, i got a higher score on a &lt;strong&gt;Random Forest Classifier&lt;/strong&gt; compared to Decision Tree, SVM, Naive Bayes, Logistic Regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my project: &lt;a href=&quot;https://github.com/khaifagifari/Classification-and-Clustering-on-Used-Cars-Dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/khaifagifari/Classification-and-Clustering-on-Used-Cars-Dataset&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;source : &lt;a href=&quot;https://github.com/f2005636/Classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/f2005636/Classification&lt;/a&gt; &#xA;&lt;a href=&quot;https://www.kaggle.com/vbmokin/used-cars-price-prediction-by-15-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/vbmokin/used-cars-price-prediction-by-15-models&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36273" LastActivityDate="2020-04-21T08:54:43.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20440" PostTypeId="2" ParentId="20405" CreationDate="2020-04-21T09:12:20.603" Score="0" Body="&lt;p&gt;I haven't watched Westworld from the beginning to the end, but I've read the synopsis of it.&lt;/p&gt;&#xA;&lt;p&gt;The implications of the statement above might be related to the experiences of what the androids (or cyborgs) might think of themselves. &lt;em&gt;Are their identity and experiences that they had gone through &amp;quot;real&amp;quot; or not?&lt;/em&gt; It seems that, in the series, this question is actually central, and the series seems to be about an identity crisis, the search for a true identity.&lt;/p&gt;&#xA;&lt;p&gt;In my view, without really any fail-safe programming, once the androids understand that all of their experiences are false (or unreal) and that everything that they do is monitored and tested by some individuals for science purposes, the AI would go rogue. A conscious being with no real background experiences being put into an environment that 'should' have been their 'life' all along, but, after realizing the real truth, everything does matter, and the conscious being would go rogue to find their true self in order to recover their origin and their true identity.&lt;/p&gt;&#xA;" OwnerUserId="36278" LastEditorUserId="2444" LastEditDate="2021-01-17T12:58:39.637" LastActivityDate="2021-01-17T12:58:39.637" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20442" PostTypeId="2" ParentId="20185" CreationDate="2020-04-21T09:22:31.630" Score="0" Body="&lt;p&gt;You can use k-nn clustering but you must convert your dataset to numeric values or you can remove the unrelated features in your dataset. &lt;/p&gt;&#xA;" OwnerUserId="36256" LastEditorUserId="22659" LastEditDate="2020-04-21T15:26:52.323" LastActivityDate="2020-04-21T15:26:52.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20443" PostTypeId="2" ParentId="20404" CreationDate="2020-04-21T09:23:03.613" Score="1" Body="&lt;p&gt;let me try to answer your question. yes, you can use multilayer perceptron to image classification. Multilayer Perceptron is topology&#xA;the most common of ANN, where perceptrons are connected to form layers. An MLP has&#xA;input layer, at least one&#xA;hidden layer, and output layer.&#xA;Multilayer perceptron is one method&#xA;many used. one of them, regards research on classification&#xA;human skin based on its color, Khan&#xA;(Khan, Hanbury, Stöttinger, &amp;amp; Bais, 2012)&#xA;compare the nine methods for&#xA;classifications include BayesNet, J48, Multilayer Perceptron (MLP),&#xA;Naive Bayes, Random Forest, and SVM.&#xA;The results show that the Multilayer Perceptron (MLP)&#xA;produce the highest performance after&#xA;Random Forest and J48.&lt;/p&gt;&#xA;" OwnerUserId="36275" LastActivityDate="2020-04-21T09:23:03.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20444" PostTypeId="2" ParentId="20081" CreationDate="2020-04-21T09:24:09.193" Score="0" Body="&lt;p&gt;When we train a model using a data train, sometimes the resulting score is very high. This makes us believe that our model is very good. But when predicting actual data, the resulting score is very low. Why?? This means that the trained model is &lt;strong&gt;overfit&lt;/strong&gt; (to data train) and fail to predict anything useful on yet-&lt;strong&gt;unseen&lt;/strong&gt; data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why we have to check our model to test data (predict test data) and compare the accuracy between data train and data test. If the accuracy is &lt;strong&gt;not too far away&lt;/strong&gt;, then our model does &lt;strong&gt;not overfit&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later, we can improve our model with &lt;strong&gt;Cross Validation&lt;/strong&gt; (&lt;a href=&quot;https://scikit-learn.org/stable/modules/cross_validation.html#cross-validation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reference&lt;/a&gt;) that split our data train to n-split data and take one of that to became data train. Then we take the average of Cross Validation score.&lt;/p&gt;&#xA;" OwnerUserId="36265" LastActivityDate="2020-04-21T09:24:09.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20445" PostTypeId="2" ParentId="20185" CreationDate="2020-04-21T09:26:15.560" Score="0" Body="&lt;p&gt;KNN can be used in clustering with the data frame. but there are a number of steps that you must take.&#xA;1. You must separate the features you want to cluster. for example you can do clustering dob and age.&#xA;2. if there is data of type string you have to change it to an integer.&#xA;For easier clustering, you can use the Sklearn library. you can access at the following link &lt;a href=&quot;https://scikit-learn.org/stable/modules/clustering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/modules/clustering.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36242" LastActivityDate="2020-04-21T09:26:15.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20446" PostTypeId="2" ParentId="20321" CreationDate="2020-04-21T09:31:14.507" Score="4" Body="&lt;p&gt;Based on my experience, I did 2 tasks that is proven to improve the accuracy/score of my model.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Normalization&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;removing characters and symbols in a text &lt;/li&gt;&#xA;&lt;li&gt;lowercase folding&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Stopwords removal (as what you asked)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These process helped me improve my model since stopwords gave my model noise as I am using word frequency count to represent text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So based on what you asked, does stopwords removal improve score? It depends on your model. If you are using word count to represent text you may do stopwords removal to remove noise when doing text classification.&lt;/p&gt;&#xA;" OwnerUserId="36260" LastActivityDate="2020-04-21T09:31:14.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20450" PostTypeId="2" ParentId="10" CreationDate="2020-04-21T09:35:12.257" Score="1" Body="&lt;p&gt;Fuzzy Logic is a way of dealing with uncertainties, which is something that computers don't do naturally which human do very well. The way we instantly think of dealing with things and the way that computers tend to deal with certain things is 'True' or 'False', or '1 or '0'. For example, you might classify someone is alive as alive ('True') or has passed away ('False'). We only have two options, there is no inbetween. With fuzzy logic, instead of going with 'True' or 'False', between that, we have what's called a degree of truth. So for example, when we look out the window, we might say &quot;It's a bit cloudy today, maybe it's 0.5 'nice day' or 0.7 'nice day'&quot;. So essentially, with fuzzy logic we always have grey areas which vary from person-to-person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://youtu.be/r804UF8Ia4c&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/r804UF8Ia4c&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36283" LastEditorUserId="36283" LastEditDate="2020-04-21T11:13:09.990" LastActivityDate="2020-04-21T11:13:09.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20451" PostTypeId="2" ParentId="20381" CreationDate="2020-04-21T09:39:06.270" Score="1" Body="&lt;p&gt;this is maybe not a paper but this article is dope. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got 2 link in below to help you understand about policy gradient algorithms especially Reinforce Learning. Both article have a good explanation about that algorithms and not only an explanation, also contain a good example about that. They really put their effort into it so i'm sure if you've read it and really understand it, you can code from scratch. Hope this is help you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think this answer is helping you, i will grateful if you make my answer as accepted. Just kidding, have a nice day.... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(no i'm not kidding)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/policy-gradients-in-a-nutshell-8b72f9743c5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/policy-gradients-in-a-nutshell-8b72f9743c5d&lt;/a&gt; by Sanyam Kapoor on Towardsdatascience.com&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@jonathan_hui/rl-policy-gradients-explained-9b13b688b146&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@jonathan_hui/rl-policy-gradients-explained-9b13b688b146&lt;/a&gt; by Jonathan Hui on Medium.com&lt;/p&gt;&#xA;" OwnerUserId="36289" LastEditorUserId="36289" LastEditDate="2020-04-21T09:51:20.023" LastActivityDate="2020-04-21T09:51:20.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20452" PostTypeId="2" ParentId="20193" CreationDate="2020-04-21T09:40:06.670" Score="1" Body="&lt;p&gt;If your data is labeled, but you only have a limited amount, you should use a classifier with high bias (for example, Naive Bayes). I'm guessing this is because a higher-bias classifier will have lower variance, which is good because of the small amount of data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source : &lt;a href=&quot;https://stackoverflow.com/questions/2595176/which-machine-learning-classifier-to-choose-in-general/15881662&quot;&gt;https://stackoverflow.com/questions/2595176/which-machine-learning-classifier-to-choose-in-general/15881662&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36262" LastEditorUserId="36262" LastEditDate="2020-04-21T11:29:36.983" LastActivityDate="2020-04-21T11:29:36.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20457" PostTypeId="2" ParentId="12558" CreationDate="2020-04-21T09:48:48.153" Score="3" Body="&lt;p&gt;Here's a definition by Tom Mitchel (1997): &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Computer program is said to learn from experience E with respect to&#xA;  some task T and some performance measure P, if its performance on T,&#xA;  as measured by P, improves with experience E.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the programmer gives some instructions/rules to the computer, so that it can learn how to solve the problem from the given examples by themselves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some tasks, the computer can perform better than humans. For example, the Dota 2 bot (made by OpenAI) defeated the world champion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With machine learning, many cases can be automated. They also have the ability to improve solutions by learning the given data from time to time. It can process and analyze large data well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning already applied in many fields such as machine translation in google translate and face recognition that is widely used by today's society for security.&lt;/p&gt;&#xA;" OwnerUserId="36292" LastEditorUserId="2444" LastEditDate="2020-04-21T12:44:26.153" LastActivityDate="2020-04-21T12:44:26.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20458" PostTypeId="2" ParentId="7202" CreationDate="2020-04-21T09:49:39.713" Score="0" Body="&lt;p&gt;Because you use various combinations of features whose dimensionality is between 1 and 14 features, You might try to use Linear SVM (linear Kernels) would be good for your problem. You could try LIBLINEAR library but the Data should be linearly separable, otherwise test accuracy would be very low.&lt;/p&gt;&#xA;" OwnerUserId="36295" LastActivityDate="2020-04-21T09:49:39.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20460" PostTypeId="2" ParentId="88" CreationDate="2020-04-21T09:52:15.593" Score="0" Body="&lt;p&gt;i think that Training deep learning neural networks can be difficult because of local optima in the objective function and because complex models are prone to overfitting. Unsupervised pre-training initializes a discriminative neural net from one which was trained using an unsupervised criterion, such as a deep belief network or a deep autoencoder. This method can sometimes help with both the optimization and the overfitting issues, and about deep learning actually work Because there is no external taecher in unsupervised learning, it is really crucial to increase the entropy which can be done by redundancies in the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;source: &lt;a href=&quot;https://metacademy.org/graphs/concepts/unsupervised_pre_training&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://metacademy.org/graphs/concepts/unsupervised_pre_training&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36282" LastEditorUserId="36282" LastEditDate="2020-04-21T11:14:24.307" LastActivityDate="2020-04-21T11:14:24.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20464" PostTypeId="2" ParentId="20081" CreationDate="2020-04-21T09:59:00.427" Score="0" Body="&lt;p&gt;One way to test the accuracy of trained model is by testing it using data test. By testing we are able to check the accuracy. Whether your model is a good model or not depends heavily on the accuracy, if your accuracy is too low or too high (eg. up to 99%~100%), there could be some problem in your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further information on the example in data test, you can access &lt;a href=&quot;https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;" OwnerUserId="36254" LastActivityDate="2020-04-21T09:59:00.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20465" PostTypeId="2" ParentId="14178" CreationDate="2020-04-21T10:00:10.830" Score="0" Body="&lt;p&gt;The purpose of clustering is to look for the &lt;strong&gt;same characteristics&lt;/strong&gt; in the data and group them into clusters. The number of clusters we take is based on how our &lt;strong&gt;clustering algorithm evaluated&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we use the &lt;strong&gt;Elbow Method&lt;/strong&gt; for evaluates. We take optimal number of clusters that distortion start decreasing in a linear fashion. In the picture below, the &lt;strong&gt;optimal number of clusters&lt;/strong&gt; for the data is 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0MEmU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0MEmU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36265" LastActivityDate="2020-04-21T10:00:10.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20471" PostTypeId="2" ParentId="20310" CreationDate="2020-04-21T10:07:13.877" Score="0" Body="&lt;p&gt;Machine learning uses algorithms to digest data sets, draw conclusions based on analyzed data, and use these conclusions to complete the task in the most effective way. This ability is a fundamental difference between machine learning and machine that has been programmed from the beginning with a certain sequence of commands. Machine learning has the capability to accomplish tasks dynamically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While Deep Learning is one of the methods of implementing machine learning that aims to mimic the workings of the human brain using ANN. Deep learning uses a number of algorithms as 'neurons' to work together in determining and digesting certain characteristics in a data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast to general machine learning programs that are designed to accomplish certain tasks, deep learning programs are usually programmed with more complex capabilities to study, digest, and classify data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A machine learning model requires data to learn and obtain parameter estimates, so the more data that can be used, the machine learning program will be smarter. In addition, operating machine learning models — especially logical networks for deep learning — requires high computational power. This is because the deep learning model must operate many processes simultaneously, especially in the training phase. In the training phase, the machine learning model must process very large amounts of data to be categorized as a reference.&lt;/p&gt;&#xA;" OwnerUserId="36264" LastActivityDate="2020-04-21T10:07:13.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20472" PostTypeId="2" ParentId="19881" CreationDate="2020-04-21T10:08:18.533" Score="0" Body="&lt;p&gt;The training error (on any error metrics, not only for RMSE) will usually be less than the test error because the same data used to fit the model is employed to assess its training error. In other words, a fitted model usually adapts to the training data and hence its training error will be overly optimistic (too small). In fact, it is often the case that the training error steadily decreases as the size of the model increases.&lt;/p&gt;&#xA;" OwnerUserId="36280" LastActivityDate="2020-04-21T10:08:18.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20475" PostTypeId="2" ParentId="20185" CreationDate="2020-04-21T10:12:53.027" Score="0" Body="&lt;p&gt;There are several algorithm for clustering such as: K-means, Mean shift, hierarchical,etc. Based on my experience, actually it's K-means(KNN for classifcication).It is suitable for clustering your dataset, there are several steps for clustering your dataset:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You have to determine which features that you want cluster&lt;/li&gt;&#xA;&lt;li&gt;Changing your categorical dataset to numerical &lt;/li&gt;&#xA;&lt;li&gt;This step is optional, You can drop columns that are not related to the features you have chosen before&lt;/li&gt;&#xA;&lt;li&gt;Try to to code your clustering (like determine centroid from your dataset, calculate the euclidean distance from your centroid,etc) or if you want to use library maybe sklearn is the right place.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And for determine the quality of your clustering, you can measures SSE(sum of the square error from the items of each cluster),Inter cluster distance,Intra cluster distance for each cluster,Maximum Radius,Average Radius.&lt;/p&gt;&#xA;" OwnerUserId="36285" LastEditorUserId="36285" LastEditDate="2020-04-21T10:26:38.757" LastActivityDate="2020-04-21T10:26:38.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20476" PostTypeId="2" ParentId="19881" CreationDate="2020-04-21T10:16:48.430" Score="1" Body="&lt;p&gt;It is common to have root mean squared error (RMSE) greater on the test dataset than on the training dataset (this is equal to having accuracy/score higher for model in training dataset than test dataset). This normally happens because the training data are assesed on the same data that have been learnt before, while the test dataset may have data that are unknown / not common that may give more errors or misclassification when doing prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if your model shows your test dataset have way too high RMSE result rather than your training dataset RMSE result, it may indicates that overfitting happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If overfitting happens, there are a lot of reasons this could happen. &#xA;Referenced from &lt;a href=&quot;https://elitedatascience.com/overfitting-in-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://elitedatascience.com/overfitting-in-machine-learning&lt;/a&gt;, some factors that causes overfitting are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Complexity of data (e.g. there are irrelevant input features). This can be solved with removing irrelevant input features. &lt;/li&gt;&#xA;&lt;li&gt;Not enough training data. This can be solved by training with more data (Eventhough this may not always succeed. Sometimes it may give noise towards data), etc. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36260" LastEditorUserId="36260" LastEditDate="2020-04-21T11:04:58.263" LastActivityDate="2020-04-21T11:04:58.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20478" PostTypeId="2" ParentId="20185" CreationDate="2020-04-21T10:21:31.500" Score="1" Body="&lt;p&gt;Yes you can use KNN algorithm to cluster (well actually its a classification not a clustering if you use KNN) the data. &lt;strong&gt;But,&lt;/strong&gt; first you need to set one feature as a label because KNN is a supervised learning method, it need a labeled data to train the data first. For example you can use Gender as label to classify the data. To determine the quality of the classification result, you can simply use accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don’t want to use a label, you can use unsupervised learning method like K-Means to do the clusters. Because its unsupervised it doesn’t need label so you can use all of the feature to do the clusters task. For the k-means algorithm you can use a library from scikit-learn or create it from scratch. To evaluate the results you can use silhouette score or elbow method (to find the optimal number of cluster).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And don’t forget to do data exploration because maybe it can increase the quality of the cluster results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can learn more about the differentiation between K-Means and KNN in the link below:&#xA;&lt;a href=&quot;https://pythonprogramminglanguage.com/how-is-the-k-nearest-neighbor-algorithm-different-from-k-means-clustering/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pythonprogramminglanguage.com/how-is-the-k-nearest-neighbor-algorithm-different-from-k-means-clustering/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps :)&lt;/p&gt;&#xA;" OwnerUserId="36276" LastEditorUserId="36276" LastEditDate="2020-04-21T11:09:58.563" LastActivityDate="2020-04-21T11:09:58.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20479" PostTypeId="2" ParentId="12322" CreationDate="2020-04-21T10:21:51.683" Score="0" Body="&lt;p&gt;&lt;strong&gt;Sentiment Analysis&lt;/strong&gt; -- the most common text classification that analyses an incoming message and tells whether the underlying sentiment is positive, negative, or neutral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Emotion Recognition&lt;/strong&gt;-- emotion recognition refers to the cognitive and behavioral strategies people use to influence their own emotional experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: This explanation is based on the paper that I have read.&lt;/p&gt;&#xA;" OwnerUserId="36270" LastEditorUserId="22659" LastEditDate="2020-04-22T13:17:22.527" LastActivityDate="2020-04-22T13:17:22.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20480" PostTypeId="2" ParentId="19985" CreationDate="2020-04-21T10:22:16.533" Score="0" Body="&lt;p&gt;AI can predict such thing by reading a data that they previously store by play the game many times. Using the data the AI can learn which is the best action to do. For example AI can find the best path to evade all incoming bullet while shooting down all the enemies in bullet hell game.&lt;/p&gt;&#xA;" OwnerUserId="36277" LastActivityDate="2020-04-21T10:22:16.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20481" PostTypeId="2" ParentId="20310" CreationDate="2020-04-21T10:22:23.013" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So, do I have to learn machine learning first before going into deep learning, or can I skip ML?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Quoting from &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;, &quot;Deep learning (also known as deep structured learning or differential programming) is part of a broader family of machine learning methods based on artificial neural networks with representation learning.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, it is better to understand the fundamental of machine learning first so you would be able to understand completely of how deep learning works, and how to apply it effectively and efficiently. However, You can always skip straight to deep learning without having any major issues, as there is already a lot of libraries supporting deep learning on python such as &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the pros and cons of studying machine learning before deep learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;pros:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since deep learning is a subset of machine learning, having fundamental knowledge about machine learning and the other machine learning algorithms will be beneficial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cons:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might (not) waste your time and energy to learn something you wouldn't use.&lt;/p&gt;&#xA;" OwnerUserId="36245" LastEditorUserId="36245" LastEditDate="2020-04-21T10:31:15.320" LastActivityDate="2020-04-21T10:31:15.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20482" PostTypeId="2" ParentId="7202" CreationDate="2020-04-21T10:23:30.840" Score="1" Body="&lt;p&gt;To quickly train the SVM , you can try to Use Linear SVM or Use scaled data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;sources: &lt;a href=&quot;https://www.researchgate.net/publication/2926909_A_Practical_Guide_to_Support_Vector_Classification_Chih-Wei_Hsu_Chih-Chung_Chang_and_Chih-Jen_Lin&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/2926909_A_Practical_Guide_to_Support_Vector_Classification_Chih-Wei_Hsu_Chih-Chung_Chang_and_Chih-Jen_Lin&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36241" LastActivityDate="2020-04-21T10:23:30.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20483" PostTypeId="2" ParentId="20231" CreationDate="2020-04-21T10:24:33.533" Score="1" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia article&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The term &amp;quot;artificial general intelligence&amp;quot; was used as early as 1997, by Mark Gubrud. in a discussion of the implications of fully automated military production and operations. The term was re-introduced and popularized by Shane Legg and Ben Goertzel around 2002.&lt;/p&gt;&#xA;&lt;p&gt;The research objective is much older, for example Doug Lenat's Cyc project (that began in 1984), and Allen Newell's Soar project are regarded as within the scope of AGI.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36285" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-21T12:04:53.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20485" PostTypeId="2" ParentId="7202" CreationDate="2020-04-21T10:27:37.920" Score="0" Body="&lt;p&gt;You can speed up the training time by doing several steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;scale the values of your features&lt;/li&gt;&#xA;&lt;li&gt;use only a limited number of features because this will affect the training time; i.e. when you use 14 features, it means your model has 14 dimensions and it makes computation more complex and take much time.&lt;/li&gt;&#xA;&lt;li&gt;choose a proper kernel, linear SVM kernel usually give the fastest result&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36247" LastEditorUserId="2444" LastEditDate="2020-04-21T12:49:27.637" LastActivityDate="2020-04-21T12:49:27.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20486" PostTypeId="2" ParentId="18137" CreationDate="2020-04-21T10:28:45.300" Score="2" Body="&lt;p&gt;There have been many researches in dynamic difficulty adjustment (DDA). I see this one is quite explaining: &lt;a href=&quot;https://www.researchgate.net/publication/228889029_AI_for_dynamic_difficulty_adjustment_in_games&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI for Dynamic Difficulty Adjustment in Games&lt;/a&gt;. However, there are many factors when we are trying to do dynamic difficulty adjustment. As explained in paper above, one major problem is it is sometimes hard to make sure the created model will still deliver the intended experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also read this paper about game design and DDA on &lt;a href=&quot;https://dl.acm.org/doi/10.1145/1178477.1178573&quot; rel=&quot;nofollow noreferrer&quot;&gt;ace05The Case for Dynamic Difficulty Adjustment in Games &lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for short, you can use neural network -- or other learning methods -- to do dynamic difficulty adjustment. But it's more about game design and experience impact of your DDA.&lt;/p&gt;&#xA;" OwnerUserId="36238" LastEditorUserId="36238" LastEditDate="2020-04-21T10:34:02.423" LastActivityDate="2020-04-21T10:34:02.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20487" PostTypeId="2" ParentId="20369" CreationDate="2020-04-21T10:41:37.373" Score="1" Body="&lt;p&gt;I am not sure if a pretrained machine learning model is actually protected by copyrights or not. Copyright protection exists to protect the creators of creative works from having their work &quot;stolen&quot;, and I am not sure if training a ML model is an act of creativity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, assuming that a pretrained ML model is actually protected by copyrights, then it is more likely that the model is a derived work of the data set used for training than that it is a derived work of the software that uses the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The software reads the model in as data, assuming that the software can be used with many differently trained models. In that case, the software and the model are considered completely independent works in the same way that MS Word and the documents you write with it are independent works for copyright.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if you want to publish the trained model with a license, I would recommend to use the BSD license that was also used for the training set.&lt;/p&gt;&#xA;" OwnerUserId="36305" LastActivityDate="2020-04-21T10:41:37.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20489" PostTypeId="2" ParentId="3847" CreationDate="2020-04-21T10:42:37.580" Score="1" Body="&lt;p&gt;It's instructive to know the definition of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Transistor&quot; rel=&quot;nofollow noreferrer&quot;&gt;transistor&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transistors are electronic components that are used for amplifiers, as circuit breakers, as connectors, as voltage controllers, for signal modulation and others functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(An analogy is that the transistor functions as an electric &quot;faucet&quot; that regulates input and output voltage.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of AI by Andreas Kaplan and Michael Haenlein defines artificial intelligence as: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The ability of the system to interpret external data correctly, to learn from that data, and to use that learning to achieve certain goals and tasks through flexible adaptation.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;[Citation Needed]&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under this definition, transistors are not part of AI, because transistors do not learn and cannot adapt.  Transistors are just electronic devices for regulating current. &lt;/p&gt;&#xA;" OwnerUserId="36301" LastEditorUserId="1671" LastEditDate="2020-04-23T23:00:42.250" LastActivityDate="2020-04-23T23:00:42.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20491" PostTypeId="2" ParentId="19844" CreationDate="2020-04-21T10:45:44.787" Score="0" Body="&lt;p&gt;It depends on the dataset we have and algorithm we use, usually text preprocessing can help your model perform better. But, some preprocessing method can have no significant impact on accuracy. We need to choose which preprocessing method that can help us make better quality dataset to give to the model. &lt;/p&gt;&#xA;" OwnerUserId="36292" LastActivityDate="2020-04-21T10:45:44.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20494" PostTypeId="2" ParentId="12023" CreationDate="2020-04-21T10:51:03.463" Score="0" Body="&lt;p&gt;I've been read from some blogs, for image caption generation you can use concepts of a CNN and LSTM model and build a working model of Image caption generator by implementing CNN with LSTM. I think before you want to make something and using this project, you need to have good knowledge of Deep learning, Python, working on Jupyter notebooks, Keras library, Numpy, and Natural language processing and make sure that you have installed all the following necessary libraries:&#xA;-pip install tensorflow&#xA;-keras&#xA;-pillow&#xA;-numpy&#xA;-tqdm&#xA;-jupyterlab&lt;/p&gt;&#xA;&#xA;&lt;p&gt;source : &lt;a href=&quot;https://data-flair.training/blogs/python-based-project-image-caption-generator-cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://data-flair.training/blogs/python-based-project-image-caption-generator-cnn/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36306" LastActivityDate="2020-04-21T10:51:03.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20496" PostTypeId="2" ParentId="11976" CreationDate="2020-04-21T10:55:21.573" Score="0" Body="&lt;p&gt;Artificial neural networks are one of the main tools used in machine learning. As the “neural” part of their name suggests, they are brain-inspired systems which are intended to replicate the way that we humans learn. Neural networks consist of input and output layers, as well as (in most cases) a hidden layer consisting of units that transform the input into something that the output layer can use. They are excellent tools for finding patterns which are far too complex or numerous for a human programmer to extract and teach the machine to recognize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While neural networks (also called “perceptrons”) have been around since the 1940s, it is only in the last several decades where they have become a major part of artificial intelligence. This is due to the arrival of a technique called “backpropagation,” which allows networks to adjust their hidden layers of neurons in situations where the outcome doesn’t match what the creator is hoping for — like a network designed to recognize dogs, which misidentifies a cat&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more info, visit: &lt;a href=&quot;https://www.digitaltrends.com/cool-tech/what-is-an-artificial-neural-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.digitaltrends.com/cool-tech/what-is-an-artificial-neural-network/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36285" LastEditorUserId="36285" LastEditDate="2020-04-21T11:17:01.597" LastActivityDate="2020-04-21T11:17:01.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20497" PostTypeId="2" ParentId="20185" CreationDate="2020-04-21T10:55:37.910" Score="0" Body="&lt;p&gt;you can clustering the data frame with unsupervised algorithm, for example you can use K-Means method. There are some options you can choose to eliminate some features in your data frame, like del dataFrame['Column Name']. In unsupervised learning, the algorithm not calculate the quality of the clusters, but you can set it up by yourself to make a parameter for calculate the quality for each clusters, for example it depend on sum of data in each clusters. Actually you can use KNN algorithm with your data frame, but you need to add a label in there because KNN is a supervised learning, and its function to make a classification, not clustering. hope it useful.&lt;/p&gt;&#xA;" OwnerUserId="36294" LastActivityDate="2020-04-21T10:55:37.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20500" PostTypeId="2" ParentId="11976" CreationDate="2020-04-21T11:01:08.717" Score="0" Body="&lt;p&gt;Artificial Neural Network (ANN) is a technique or approach to information processing that is inspired by the workings of the biological nervous system, especially on human brain cells in processing information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key element of this technique is the structure of information processing systems that are unique and varied for each application. Neural Networks consist of a large number of information processing elements (neurons) that are interconnected and work together to solve a particular problem, which is generally a classification or prediction problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;source : &lt;a href=&quot;https://dosen.perbanas.id/artificial-neural-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dosen.perbanas.id/artificial-neural-network/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36306" LastEditorUserId="36306" LastEditDate="2020-04-21T11:12:43.537" LastActivityDate="2020-04-21T11:12:43.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20501" PostTypeId="2" ParentId="20193" CreationDate="2020-04-21T11:01:19.923" Score="0" Body="&lt;p&gt;if you use just one  feature for dataset, i'm recommend a Algorithm Naive Bayes Classifier because Naive Bayes is a method using probability and statistical methods. And we can also get the total data train and its accuracy value by using. &lt;/p&gt;&#xA;" OwnerUserId="36314" LastEditorUserId="36314" LastEditDate="2020-04-21T11:07:04.337" LastActivityDate="2020-04-21T11:07:04.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20502" PostTypeId="2" ParentId="16812" CreationDate="2020-04-21T11:03:44.623" Score="1" Body="&lt;p&gt;Of course, you can, and yes this has been researched and done before. By using supervised learning, you give the &lt;em&gt;machine&lt;/em&gt; some data, and it will try to figure the best way to analyze and predict the next movement in case of an &lt;strong&gt;inverted pendulum&lt;/strong&gt; problem.&lt;/p&gt;&#xA;&lt;p&gt;I found a complete paper for this problem: &lt;a href=&quot;https://www.researchgate.net/publication/228798837_Neural_network_control_of_an_inverted_pendulum_on_a_cart&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural network control of an inverted pendulum on a cart&lt;/a&gt; by Valeri Mladenov et al.&lt;/p&gt;&#xA;" OwnerUserId="36240" LastEditorUserId="2444" LastEditDate="2021-01-16T20:11:37.557" LastActivityDate="2021-01-16T20:11:37.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20504" PostTypeId="2" ParentId="18715" CreationDate="2020-04-21T11:05:43.420" Score="0" Body="&lt;p&gt;Other tasks that RNNs are effective at solving are time-series predictions or other sequence predictions that aren’t image or tabular based.&lt;/p&gt;&#xA;" OwnerUserId="36279" LastEditorUserId="2444" LastEditDate="2020-04-21T15:27:45.560" LastActivityDate="2020-04-21T15:27:45.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20506" PostTypeId="2" ParentId="20254" CreationDate="2020-04-21T11:09:32.627" Score="0" Body="&lt;p&gt;Yes. There's a library called OpenCV that can be used to measure distance between objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find out the distance of two objects, we must know the dimensions of the reference object. There are two important properties for the reference object:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We know the dimensions of the object in certain units(inches,&#xA;millimeters, etc.) &lt;/li&gt;&#xA;&lt;li&gt;The reference object needs to be easily found in the photo.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Dimensions of the reference object will be used to measure distance beetween other objects. Also, we firstly need to compute the “pixels-per-metric” ratio, used to determine how many pixels “fit” into a given unit of measurement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to this post for detailed explanation:&#xA;&lt;a href=&quot;https://www.pyimagesearch.com/2016/03/28/measuring-size-of-objects-in-an-image-with-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pyimagesearch.com/2016/03/28/measuring-size-of-objects-in-an-image-with-opencv/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36255" LastEditorUserId="22659" LastEditDate="2020-04-22T14:54:24.717" LastActivityDate="2020-04-22T14:54:24.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20508" PostTypeId="2" ParentId="20081" CreationDate="2020-04-21T11:15:30.850" Score="1" Body="&lt;p&gt;In machine learning, we can use all the datasets as training data in a model.  But if there are too many data sets, or too much data, and we do not split them up, our model may be not produce acceptable results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because if the model studies too much training data, it may be &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitted&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Just like when you cram for a test, and get overloaded with too much information!) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean is, your model is only familiar with the data you provide, not for the new data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we need to use test data to train our algorithm.  Naive Bayes and Decision Tree Classifier are no exception because they can produce an overfitted model based on train data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we test it on the data test to know how well the method works in relation to the problem.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most data scientists divide their data (with answers, that is historical data) into three portions: training data, cross-validation data and testing data. The training data is used to make sure the machine recognizes patterns in the data, the cross-validation data is used to ensure better accuracy and efficiency of the algorithm used to train the machine, and the test data is used to see how well the machine can predict new answers based on its training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&#xA;&lt;sub&gt;SOURCE: &lt;a href=&quot;https://www.researchgate.net/post/What_is_training_and_testing_data_in_machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/post/What_is_training_and_testing_data_in_machine_learning&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="36269" LastEditorUserId="1671" LastEditDate="2020-04-23T21:56:33.360" LastActivityDate="2020-04-23T21:56:33.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20511" PostTypeId="2" ParentId="20404" CreationDate="2020-04-21T11:19:35.103" Score="2" Body="&lt;p&gt;Multi Layers Perceptron(MLP) can be used for image classification, but it has a lot of deficiency than Convolutional Neural network(CNN). But if you compare &lt;strong&gt;MLP&lt;/strong&gt; and &lt;strong&gt;Fisher Faces&lt;/strong&gt; , the better one is &lt;strong&gt;MLP&lt;/strong&gt;, because Fisher Faces will be increasingly difficult if adding more individuals or classes. You can make a simple MLP model, because it just has 3 layers which are an input layer, hidden layer and output layer, here a source code that you can try:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/multi-layer-neural-networks-with-sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/multi-layer-neural-networks-with-sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;if you make a model, it will be based on training data, I think if make data trained composed based on noise in background and different light shadows on your image I think it will have a better performance, but remember if you are using &lt;strong&gt;MLP&lt;/strong&gt; for image classification it can just predict an image on one spot, for example:&#xA;&quot;if you train a model with the object in the middle of an image, your model can not predict it when the image is moved to the different spot&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is pdf to see Fisher Faces performance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://core.ac.uk/download/pdf/84801048.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://core.ac.uk/download/pdf/84801048.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36298" LastEditorUserId="31870" LastEditDate="2020-04-22T14:54:32.800" LastActivityDate="2020-04-22T14:54:32.800" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20513" PostTypeId="2" ParentId="20193" CreationDate="2020-04-21T11:21:08.730" Score="0" Body="&lt;p&gt;I think no matter you use one or more feature for dataset. &#xA;You can compare the classification algorithms regarding the accuracy provided by the algorithm. Like compare naive bayes with svm method, it based on your problem set.&lt;/p&gt;&#xA;" OwnerUserId="36299" LastActivityDate="2020-04-21T11:21:08.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20514" PostTypeId="2" ParentId="18634" CreationDate="2020-04-21T11:22:27.900" Score="1" Body="&lt;p&gt;Word embeddings are the results of learning from deep learning algorithms, which can learn characters from data through feature extraction. One implementation of word embedding is word2vec.&lt;/p&gt;&#xA;&lt;p&gt;Word2vec has two models, namely&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Continuous Bag of Word (CBOW) and&lt;/li&gt;&#xA;&lt;li&gt;Skip Gram Model.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both of these methods use the concept of a neural network that maps words to target variables, which are also words. In these techniques, &amp;quot;weights&amp;quot; are used as word vector representations. CBOW tries to predict a word on the basis of its neighbors, while Skip Gram tries to predict the neighbors of a word.&lt;/p&gt;&#xA;&lt;p&gt;In simpler words, CBOW tends to find the probability of a word occurring in a context. So, it generalizes over all the different contexts in which a word can be used. Whereas SkipGram tends to study different contexts separately. Skip Gram needs more data to be trained contains more knowledge about the context.&lt;/p&gt;&#xA;&lt;p&gt;For further explanation, you can read the &lt;a href=&quot;https://arxiv.org/pdf/1301.3781.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; from Mikolov about word embedding and word2vec.&lt;/p&gt;&#xA;" OwnerUserId="36270" LastEditorUserId="2444" LastEditDate="2021-01-18T11:27:12.813" LastActivityDate="2021-01-18T11:27:12.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20519" PostTypeId="2" ParentId="18702" CreationDate="2020-04-21T11:32:45.920" Score="1" Body="&lt;p&gt;AI can transform the customer experience is by providing personalized content. For example, When you see video recommendation on YouTube, you'll know that it's from AI technology. I recommend you to read this article for knowing how they work: A Sentiment-Enhanced Hybrid Recommender System for Movie Recommendation: A Big Data Analytics Framework (&lt;a href=&quot;https://www.hindawi.com/journals/wcmc/2018/8263704/&quot; rel=&quot;nofollow noreferrer&quot;&gt;abstract&lt;/a&gt;, &lt;a href=&quot;https://www.ijrter.com/papers/volume-3/issue-5/analysis-of-movie-recommender-system-using-collaborative-filtering.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="36319" LastEditorUserId="4709" LastEditDate="2020-05-14T21:57:53.830" LastActivityDate="2020-05-14T21:57:53.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20520" PostTypeId="2" ParentId="20408" CreationDate="2020-04-21T12:05:24.023" Score="1" Body="&lt;p&gt;i feel , i can answer this question, based the web source that i found and i read. that can use an autoencoder with hight latent representational space. example use LSTM autoencoder, LTSM autoencoder used for sequence or time series data. on the source, they use CNN autoencoder to denoise some syntetic noised data they have generated. but they have asked what is the meaning of this latent representation space? for what they have done before. on that answer. they said&quot;your input data is noisy sinewave data. your are not supposed to use Convolutional autoencoder for sequence data&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/59438488/cnn-autoencoder-latent-space-representation-meaning&quot;&gt;https://stackoverflow.com/questions/59438488/cnn-autoencoder-latent-space-representation-meaning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;addition other source for answer this question&#xA;&lt;a href=&quot;https://towardsdatascience.com/understanding-latent-space-in-machine-learning-de5a7c687d8d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/understanding-latent-space-in-machine-learning-de5a7c687d8d&lt;/a&gt;&#xA;&lt;a href=&quot;https://towardsdatascience.com/deep-inside-autoencoders-7e41f319999f&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/deep-inside-autoencoders-7e41f319999f&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;hopefully help answer :))&lt;/p&gt;&#xA;" OwnerUserId="36302" LastActivityDate="2020-04-21T12:05:24.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20521" PostTypeId="1" CreationDate="2020-04-21T12:54:15.623" Score="1" ViewCount="25" Body="&lt;p&gt;I'm a newbie in machine learning and I am interested in neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any good research papers on image identification with limited data?&lt;/p&gt;&#xA;" OwnerUserId="36321" LastEditorUserId="2444" LastEditDate="2020-04-21T14:36:08.563" LastActivityDate="2020-04-21T14:36:08.563" Title="Are there any good research papers on image identification with limited data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20522" PostTypeId="2" ParentId="20352" CreationDate="2020-04-21T13:27:44.577" Score="0" Body="&lt;p&gt;Here im trying to answer, yes you could use clustering to parse pdf document. it similar of how the text mining works (you can read from &lt;a href=&quot;https://guides.library.duke.edu/c.php?g=289707&amp;amp;p=1930855&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for the clustering method, you could probably use K-NN methods, K-Means, agglomerative hierarchical clustering, and the other methods based on your preference. or u could use naive bayes alternatively. &lt;/p&gt;&#xA;" OwnerUserId="36309" LastActivityDate="2020-04-21T13:27:44.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20523" PostTypeId="1" CreationDate="2020-04-21T13:40:50.160" Score="1" ViewCount="85" Body="&lt;p&gt;I was exploring image/video compression using Machine Learning. In there I discovered that autoencoders are used very frequently for this sort of thing. So I wanted to enquire:-      &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How fast are autoencoders? I need something to &lt;em&gt;compress&lt;/em&gt; an image in milliseconds?&lt;/li&gt;&#xA;&lt;li&gt;How much resources do they take? I am not talking about the training part but rather the deployment part. Could it work fast enough to compress a video on a Mi phone (like note8 maybe)?    &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Do you know of any particularly new and interesting research in AI that has enabled a technique to this fast and efficiently?&lt;/p&gt;&#xA;" OwnerUserId="36322" LastActivityDate="2020-09-22T06:30:28.350" Title="How fast are autoencoders?" Tags="&lt;autoencoders&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20524" PostTypeId="2" ParentId="20523" CreationDate="2020-04-21T14:12:49.623" Score="1" Body="&lt;p&gt;Actually it depends on the size of your AE, if you use a small AE with just 500.000 to 1M weigths, the inferencing can be stunningly fast. But even large networks can run very fast, using Tensorflow lite for example, models are compressed and optimized to run faster on Edge-devices (Handys for example, end-user devices). You can find a lot of videos on Youtube, where people test inferencing large networks like Resnet-51 or Resnet-101 on a raspberrypi, or other SOC Chips. Handys are comparable to that, but maybe not that optimized. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,I have an Jetson Nano (SOC of Nvidia costs arround 100 euro) and i tried to inference a large Resnet with arround 30 million parameters over my fullHD Webcam. Stable 30 FPS, so speaking in milliseconds its around 33 ms per image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, yes Autoencoders can be fast, also very fast in combination with an optimized model and hardware. Autoencoder structures are quite easy, check out this &lt;a href=&quot;https://medium.com/@jannik.zuern/but-what-is-an-autoencoder-26ec3386a2af&quot; rel=&quot;nofollow noreferrer&quot;&gt;medium&lt;/a&gt;,&lt;a href=&quot;https://blog.keras.io/building-autoencoders-in-keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;keras example&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35557" LastActivityDate="2020-04-21T14:12:49.623" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20525" PostTypeId="2" ParentId="20075" CreationDate="2020-04-21T14:25:52.257" Score="0" Body="&lt;p&gt;First: RNN is one part of the Neural Network family for processing sequential data. The way in which RNN is able to store information from the past is to loop in its architecture, which automatically keeps information from the past stored.&#xA;Second: sltm / gru is a component of regulating the flow of information referred to as the gate and GRU has 2 gates, namely reset gate and gate update. If we want to make a decision to eat like the analogy above, resetting the gate on the GRU will determine how to combine new inputs with past information, and update the gate, will determine how much past information should be kept.&#xA;source : &lt;a href=&quot;https://link.springer.com/article/10.1007/s00500-019-04281-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/article/10.1007/s00500-019-04281-z&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36320" LastActivityDate="2020-04-21T14:25:52.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20526" PostTypeId="2" ParentId="20404" CreationDate="2020-04-21T14:40:10.927" Score="1" Body="&lt;p&gt;Depends, if the faces are centered and have the same background yes. You also need a lot of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they are daily life images, then no. You will have very bad generalization.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-21T14:40:10.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20527" PostTypeId="1" AcceptedAnswerId="20541" CreationDate="2020-04-21T14:41:19.030" Score="1" ViewCount="50" Body="&lt;p&gt;The goal of a reinforcement learning agent is to maximize the expected return which is often a discounted sum of future rewards. The return indeed is a &lt;em&gt;very noisy&lt;/em&gt; random variable as future rewards depend on the state-transition-probabilities and the often stochastic policy. Lots of trajectories have to be sampled to approximate its expected value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The immediate reward indeed does not have these dependencies. Therefore the questions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we train a policy to maximize the immediate reward, will it also perform well in the long term? What properties would the reward function need to fulfill?&lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-04-22T09:32:23.340" Title="Can optimizing for immediate reward result in a policy maximizing the return?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;&lt;decision-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20528" PostTypeId="1" CreationDate="2020-04-21T15:02:14.040" Score="1" ViewCount="26" Body="&lt;p&gt;What is the state of the art with respect to recognizing connotations in natural languages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Trump is a better president than Obama.           [Praising]&lt;/li&gt;&#xA;&lt;li&gt;Trump is the worst president since Obama.         [Insulting]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The rock star did not infect over 100 groupies.   [Defending against rumor]&lt;/li&gt;&#xA;&lt;li&gt;The rock star infected no more than 100 groupies. [Attacking (0 is no more than 100)]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In each example, both statements logically mean exactly the same thing, but any human hearing them would interpret them as having quite opposite meanings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How well can current natural language processors recognize the difference between logically equivalent statements?&lt;/p&gt;&#xA;" OwnerUserId="36323" LastEditorUserId="2444" LastEditDate="2020-06-22T12:40:38.807" LastActivityDate="2020-06-22T12:40:38.807" Title="How well can NLP techniques recognize connotations in natural languages?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;natural-language-understanding&gt;&lt;state-of-the-art&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20529" PostTypeId="1" CreationDate="2020-04-21T16:11:12.340" Score="1" ViewCount="46" Body="&lt;p&gt;I have noticed that almost all tutorials take the number of neurons as a power of 2. Is there any proper mathematical and well-proven reason for that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you sometimes change it to some other odd value, then we get a very long and weird error calling about a dozen things making the traceback of a page. Is there any reason for that?      &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I had tried it on a text predicting RNN with some GRU and LSTM layers mixed (bidirectional). I changed the no. of neuron units and It also resulted in an error. So, any ideas/theories?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36322" LastEditorUserId="2444" LastEditDate="2020-04-21T20:53:05.173" LastActivityDate="2020-04-21T20:53:05.173" Title="Why is the number of neurons used in various neural networks power of 2?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;hyper-parameters&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20530" PostTypeId="2" ParentId="13986" CreationDate="2020-04-21T17:10:08.853" Score="0" Body="&lt;h2&gt;About Symmetry&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;What Pearl means by &lt;em&gt;being not symmetric&lt;/em&gt; is: &lt;span class=&quot;math-container&quot;&gt;$A=B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B=A$&lt;/span&gt; are exactly identical and lead to the same result in a not-causal scientific framework. For example consider very simple set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;Z = &amp;amp; \epsilon_z \\&#xA;X = &amp;amp; Z+\epsilon_X \\&#xA;Y = &amp;amp; 2X+Z + \epsilon_y&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the Algebra point of view, this is a set of 3 equations, and 3 unknowns (consider error terms to be known). You can shuffle the equations, change the LHS and EHS of the equations, add or subtract them. In fact these actions are actually appreciated for solving a linear equation system, righ?! So the system above is identical to this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;\epsilon_z = &amp;amp; Z \\&#xA;X = &amp;amp; Y - 2Z - \epsilon_Y - \epsilon_X \\&#xA;Y = &amp;amp; 2X+Z + \epsilon_y \\&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I brought to you this example because the first one is the set of equations for a very fundamental causal structure called &lt;em&gt;confounding&lt;/em&gt; or &lt;em&gt;common cause&lt;/em&gt; structure, Where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is the exposure, or treatment, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is the outcome, and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is the parent of both &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. The orders of the equations and the RHS/LHS variables in this &lt;strong&gt;Structural Equation Models&lt;/strong&gt; actually mean something. First you calculate &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;, then go for &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, then calculate &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. In this case, the second system is pointing to a completely different causal structure (or to be honest, it is not even look like a legit structural equation)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;About sufficiency of Probability Theory for Causal Inference&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;First I would like to say this would be a very ironic question to ask from Pearl, as he also mentioned in one of his interviews, because he has had a significant contribution to the realm of probability theory with &lt;em&gt;bayesian network&lt;/em&gt; and bayesian inference framework! And now it is like he is against himself. But he is for a good reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;why Probability is not enough&lt;/em&gt; can be and has been answered with formal proofs, equations and explanations. But There are a lot of examples that will draw your attention to this truth. I recommend you to read about &lt;em&gt;Simpson's Paradox&lt;/em&gt;. It will be a great example to see how probability theory is incapable of causal inference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that probability is not enough, only resembles the idea that &lt;em&gt;correlation is not necessarily causation&lt;/em&gt;. and that is true. Again, read about &lt;em&gt;spurious correlations&lt;/em&gt; and you will get it. Just think of this funny example: During the year, Crime rate and Ice cream sales amount are highly correlated, THUS we must ban ice cream sales to control the crime rate. and the problem with this dumb inference is that we have not taken into account the common cause which is &lt;em&gt;heat&lt;/em&gt; or &lt;em&gt;summer&lt;/em&gt; that accounts for the perceived correlation.&lt;/p&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2020-04-21T17:10:08.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20531" PostTypeId="1" CreationDate="2020-04-21T19:17:54.560" Score="5" ViewCount="129" Body="&lt;p&gt;I'm trying to train a neural net to choose a subset from some list of objects. &#xA;The input is a list of objects &lt;span class=&quot;math-container&quot;&gt;$(a,b,c,d,e,f)$&lt;/span&gt; and for each list of objects the label is a list composed of 0/1 - 1 for every object that is in the subset, for example &lt;span class=&quot;math-container&quot;&gt;$(1,1,0,1,0,1)$&lt;/span&gt; represents choosing &lt;span class=&quot;math-container&quot;&gt;$a,b,d,f$&lt;/span&gt;. I thought about using MSE loss to train the net but that seemed like a naive approach, is there some better loss function to use in this case?&lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="36083" LastEditDate="2020-04-22T19:25:06.850" LastActivityDate="2021-04-01T21:17:56.140" Title="Loss function for choosing a subset of objects" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20532" PostTypeId="1" AcceptedAnswerId="20533" CreationDate="2020-04-21T20:03:40.037" Score="3" ViewCount="343" Body="&lt;p&gt;I am trying to implement value and policy iteration algorithms. My value function from policy iteration looks vastly different from the values from value iteration, but the policy obtained from both is very similar. How is this possible? And what could be the possible reasons for this?&lt;/p&gt;&#xA;" OwnerUserId="30910" LastEditorUserId="2444" LastEditDate="2020-04-21T21:28:24.310" LastActivityDate="2020-04-21T21:35:52.143" Title="Why do value iteration and policy iteration obtain similar policies even though they have different value functions?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policies&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20533" PostTypeId="2" ParentId="20532" CreationDate="2020-04-21T21:27:19.697" Score="1" Body="&lt;p&gt;Both value iteration (VI) and policy iteration (PI) algorithms are guaranteed to converge to the optimal policy, so it is expected that you get similar policies from both algorithms (if they have converged). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, they do this differently. VI can be seen as truncated version of PI.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me first illustrate the pseudocode of both algorithms (taken from &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barto and Sutton's book&lt;/a&gt;), which I suggest you get familiar with (but you are probably already familiar with them if you implemented both algorithms).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kKZx7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kKZx7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, policy iteration updates the policy multiple times, because it alternates a step of policy evaluation and a step of policy improvement, where a better policy is derived from the current best estimate of the value function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CAAu5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CAAu5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, value iteration updates the policy only once (at the end). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, the policies are derived from the value functions in the same way. So, if you obtain similar policies, you may think that they are necessarily derived from similar &lt;em&gt;final&lt;/em&gt; value functions. However, in general, this may not the case, and this is actually the motivation for the existence of value iteration, i.e. you may derive an optimal policy from an non-optimal value function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barto and Sutton's book provide an example. See figure 4.1 on page 77 (&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=99&quot; rel=&quot;nofollow noreferrer&quot;&gt;p. 99 of the pdf&lt;/a&gt;). For completeness, here's a screenshot of the figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DwyQL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DwyQL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-21T21:35:52.143" LastActivityDate="2020-04-21T21:35:52.143" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20534" PostTypeId="2" ParentId="1" CreationDate="2020-04-22T00:49:46.747" Score="0" Body="&lt;p&gt;It's a fancy name for the multivariable chain rule.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-22T00:49:46.747" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20535" PostTypeId="2" ParentId="20531" CreationDate="2020-04-22T00:54:55.873" Score="3" Body="&lt;p&gt;The choice of the loss function depends primarily on the type of task you're tackling: classification or regression. Your problem is clearly a classification one since you have classes to which a given input can either belong or not. More specifically, what you're trying to do is multi-label classification, which is different from multi-class classification. The difference is important to stress out and it consists in the format of the target labels.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Multi-class --&amp;gt; one-hot encoded labels, only 1 label is correct   &#xA;  &#xA;[1,0,0], [0,1,0], [0,0,1]&#xA;&#xA;# Multi-label --&amp;gt; multiple labels can be correct&#xA; &#xA;[1,0,0], [1,1,0], [1,1,1], [0,1,0], [0,1,1], [0,0,1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;MSE is used when continuous values are predicted for some given inputs, therefore it belongs to the loss functions suitable for regression and it should not be used for your problem.&lt;/p&gt;&#xA;&lt;p&gt;Two loss functions that you could apply are &lt;em&gt;&lt;strong&gt;Categorical Cross Entropy&lt;/strong&gt;&lt;/em&gt; or &lt;em&gt;&lt;strong&gt;Binary Cross Entropy&lt;/strong&gt;&lt;/em&gt;. Despite being both based on cross-entropy, there is an important difference between them, consisting of the activation function they require.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Binary Cross Entropy&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(y, \hat{y})=-\frac{1}{N} \sum_{i=0}^{N}\left(y * \log \left(\hat{y}_{i}\right)+(1-y) * \log \left(1-\hat{y}_{i}\right)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Despite the name that suggests this loss should be used only for binary classification, this is not strictly true and actually, this is the loss function that conceptually is best suited for multi-label tasks.&lt;/p&gt;&#xA;&lt;p&gt;Let's start with the binary classification case. We have a model that returns a single output score, to which the &lt;em&gt;sigmoid&lt;/em&gt; function is applied in order to constrain the value between 0 and 1. Since we have a single score, the resulting value can be interpreted as a probability of belonging to one of the two classes, and the probability of being to the other class can be computed as 1 - value.&lt;/p&gt;&#xA;&lt;p&gt;What if we have multiple output scores, for example, 3 nodes for 3 classes? In this case, we still could apply the sigmoid function, ending up with three scores between 0 and 1.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Acyy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Acyy.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The important aspect to capture is that since the sigmoid treat each output node independently, the 3 scores would not sum up to 1, so they will represent 3 different probability distributions rather than a unique one. This means that each score after the sigmoid represents a distinct probability of belonging to that specific class. In the example above, for example, the prediction would be true for the two labels with a score higher than 0.5 and false for the remaining label. This also means that 3 different loss have to be computed, one for each possible output. In practice, what you'll be doing is to solve n binary classification problems where n is the number of possible labels.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Categorical Cross Entropy&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(y, \hat{y})=-\sum_{j=0}^{M} \sum_{i=0}^{N}\left(y_{i j} * \log \left(\hat{y}_{i j}\right)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In categorical cross-entropy we apply the &lt;em&gt;softmax&lt;/em&gt; function to the output scores of our model, to constrain them between 0 and 1 and to turn them into a probability distribution (they all sum to 1). The important thing to notice is that in this case, we end up with a unique distribution because, unlike the sigmoid function, the softmax consider all output scores together (they are summed in the denominator).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zbnvr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zbnvr.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This implies that categorical cross-entropy is best suited for multi-class tasks, in which we end up with a single true prediction for each input instance. Nevertheless, this loss can also be applied also for multi-label tasks, as done in this &lt;a href=&quot;https://research.fb.com/wp-content/uploads/2018/05/exploring_the_limits_of_weakly_supervised_pretraining.pdf?&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. To do so, the authors turned each target vector into a uniform probability distribution, which means that the values of the true labels are not 1 but 1/k, with k being the total number of true labels.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Example of target vector tuned into uniform probability distribution&#xA;[0, 1, 1] --&amp;gt; [0, .5, .5]  &#xA;[1, 1, 1] --&amp;gt; [.33, .33, .33]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note also that in the above-mentioned paper the authors found that categorical cross-entropy outperformed binary cross-entropy, even this is not a result that holds in general.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, the are other loss that you could try which differ functions rather than cross-entropy, for example:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Hamming-Loss&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It computes the fraction of wrong predicted labels&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{|N| \cdot|L|} \sum_{i=1}^{|N|} \sum_{j=1}^{|L|} \operatorname{xor}\left(y_{i, j}, z_{i, j}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Exact Match Ratio&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Only predictions for which all target labels were correctly classified are considered correct.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ExactMatchRatio,\space M R=\frac{1}{n} \sum_{i=1}^{n} I\left(Y_{i}=Z_{i}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="36737" LastEditDate="2021-04-01T21:17:56.140" LastActivityDate="2021-04-01T21:17:56.140" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="20540" PostTypeId="1" AcceptedAnswerId="20548" CreationDate="2020-04-22T06:59:39.543" Score="4" ViewCount="128" Body="&lt;p&gt;What are examples of machine learning techniques (i.e. models, algorithms, etc.) inspired (to different extents) by neuroscience?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particularly, I'm interested in recent developments, say less than 10 years old, that have their basis in neuroscience to some degree. &lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="2444" LastEditDate="2020-04-22T13:25:51.813" LastActivityDate="2020-04-22T16:35:35.333" Title="What are examples of machine learning techniques inspired by neuroscience?" Tags="&lt;machine-learning&gt;&lt;neuromorphic-engineering&gt;&lt;spiking-neural-networks&gt;&lt;neuroscience&gt;&lt;reservoir-computing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20541" PostTypeId="2" ParentId="20527" CreationDate="2020-04-22T07:59:23.970" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If we train a policy to maximize the immediate reward, will it also perform well in the long term?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, no. The delay of long term reward in real world problems, and often a lack of easy-to-compute heuristics, is a key motivation for developing reinforcement learning in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to construct a counter-example to demonstrate this. Any state where the transitions into it are high and positive, but the transitions out of it are higher and negative would &quot;trap&quot; an agent that only considered immediate reward. More complex traps include high immediate gains but ending an episode vs lower gains that continue for longer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many real-world environments have sparse rewards where it is not possible to tell the difference between two action choices by immediate reward, but the consequences of being in one part of the state space rather than another early in a trajectory are critical. Consider any two-player strategy board game for instance, where the only goal is to win at the end. Only the last move in such a game is associated with an immediate reward, but there are often important differences between early moves.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What properties would the reward function need to fulfill?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In all states, the expected immediate reward for taking the correct long term action would need to be higher than the expected immediate reward for any other action choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving a problem framed in this way could be done with discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma=0$&lt;/span&gt;. If the action choices were always the same and valid in each state, then the problem could also be simplified to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit#Contextual_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;contextual bandit&lt;/a&gt;, where the fact that the choices exist within a larger trajectory is not relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice you &lt;em&gt;can&lt;/em&gt; construct environments like this. Simple ones are possible to do manually. Doing that is similiar to adding a heuristic function for search, but with different restrictions. For many search algorithms, &lt;a href=&quot;https://en.wikipedia.org/wiki/Admissible_heuristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;admissible heuristic functions&lt;/a&gt; are allowed to over-estimate future gains (or under-estimate costs), because a planning/search algorithm will resolve longer-term differences. In your case, you can maybe consider stochastic reward functions, but the &lt;em&gt;expected&lt;/em&gt; reward for the correct action must always be highest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Needing to know the correct optimal action in the first place is clearly a circular problem - if you knew it already you would have no need to perform reinforcement learning to discover the optimal policy. An exception might be if you constructed an easy environment in order to test an algorithm, and prove that it could find the optimal policy. Although even then usually you are interested in the algorithm solving a harder variant of your problem than one you have deliberately constructed to be easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In brief, there is no way to create a shortcut here and avoid the need to solve a harder RL problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-22T09:32:23.340" LastActivityDate="2020-04-22T09:32:23.340" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20542" PostTypeId="1" CreationDate="2020-04-22T10:30:43.150" Score="0" ViewCount="33" Body="&lt;p&gt;I wanted to create an &lt;em&gt;encoder&lt;/em&gt;, which is the first part of an autoencoder. I do not want to build the whole autoencoder but rather wanted to test whether my &lt;strong&gt;mobile&lt;/strong&gt; device can support running an encoder and encoding some images on a trained TensorFlow model.        &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am having some problems. I found some code online to describe an autoencoder &lt;a href=&quot;https://blog.keras.io/building-autoencoders-in-keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. My aim is to use the encoder separately to &quot;compress&quot; images. &lt;a href=&quot;https://colab.research.google.com/drive/11a04sSWT0ZkUPtTQuuFz8IF-POs2sOH8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is my code. So know I want to convert the &lt;code&gt;encoded&lt;/code&gt; variable to a &lt;em&gt;vector&lt;/em&gt; format. Can anyone provide a suitably easy solution to accomplish that?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By compressing Images, I mean I just want their vector representation generated by the encoder which can later be decoded. So basically I want it to print its latent space/the &lt;code&gt;encoded&lt;/code&gt; variable. The catch is that it should be in a vector representation. The reason is that right know it's shape is (None, 32) which cannot be used for further processing by TensorFlow. So, any ideas?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36322" LastEditorUserId="36322" LastEditDate="2020-04-22T11:40:06.533" LastActivityDate="2020-04-22T11:40:06.533" Title="How to convert something to vectors" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20543" PostTypeId="1" CreationDate="2020-04-22T11:50:10.677" Score="1" ViewCount="26" Body="&lt;p&gt;Recently, I read many papers on variance and bias. But I am still confused by the two notions, the variance or bias belongs to who? Policy or value? If the variance or bias is large or low, what results will we get?&lt;/p&gt;&#xA;" OwnerUserId="8415" LastEditorUserId="2444" LastEditDate="2020-04-22T11:59:11.023" LastActivityDate="2020-04-22T11:59:11.023" Title="Do the variance and bias belong to the policy or value functions?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;statistical-ai&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20544" PostTypeId="1" CreationDate="2020-04-22T12:05:46.760" Score="1" ViewCount="14" Body="&lt;p&gt;I am developing a DCGAN using the &lt;a href=&quot;https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; in PyCharm. As my usage of this tutorial suggests, I am quite new to DCGANs as I've previously only had a few experiences with machine learning algorithms on classifying problems. My goal is to feed my DCGAN a dataset of paintings of a specific painter, and get 'new' paintings in return. Needless to say, a painter does not paint thousands of paintings in his life, leaving me with a dataset of around 60 paintings. One of the smallest datasets I have ever worked with. I have two, related, questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it realistic to properly train a DCGAN on this type of dataset? If not, would there be any alternative you would suggest?&lt;/li&gt;&#xA;&lt;li&gt;What would be a good set of parameters to start of from to properly train this DCGAN?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="36354" LastActivityDate="2020-04-22T12:05:46.760" Title="Using DCGAN on a (very small) dataset of art" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;creativity&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20545" PostTypeId="2" ParentId="6196" CreationDate="2020-04-22T12:05:54.623" Score="8" Body="&lt;p&gt;This &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro2.html&quot; rel=&quot;noreferrer&quot;&gt;Tutorial by OpenAI&lt;/a&gt; offers a great comparison of different RL methods.&lt;br&gt;&#xA;I'll try to summarize the differences between Q-Learning and Policy Gradient methods:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Objective Function&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In &lt;em&gt;Q-Learning&lt;/em&gt; we learn a Q-function that satisfies the Bellman (Optimality) Equation. This is most often achieved by minimizing the &lt;strong&gt;Mean Squared Bellman Error (MSBE)&lt;/strong&gt; as the loss function. The Q-function is then used to obtain a policy (e.g. by greedily selecting the action with maximum value). &lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Policy Gradient methods&lt;/em&gt; directly try to maximize the expected return by taking small steps in the direction of the policy gradient. The policy gradient is the &lt;strong&gt;derivative of the expected return w.r.t. the policy parameters&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On- vs. Off-Policy&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;em&gt;Policy Gradient&lt;/em&gt; is derived as an expectation over trajectories (&lt;span class=&quot;math-container&quot;&gt;$s_1,a_1,r_1,s_2,a_2,...,r_n$&lt;/span&gt;), which is estimated by a sample mean. To get an unbiased estimate of the gradient, the trajectories have to be sampled from the current policy. Thus, policy gradient methods are &lt;strong&gt;on-policy&lt;/strong&gt; methods. &lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Q-Learning&lt;/em&gt; only makes sure to satisfy the Bellman-Equation. This equation has to hold true for all transitions. Therefore, Q-learning can also use experiences collected from previous policies and is &lt;strong&gt;off-policy&lt;/strong&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stability and Sample Efficiency&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Directly optimizing the return and thus the actual performance on a given task, &lt;em&gt;Policy Gradient methods&lt;/em&gt; tend to more stably converge to a good behavior. Indeed being on-policy, makes them very &lt;strong&gt;sample inefficient&lt;/strong&gt;. &lt;em&gt;Q-learning&lt;/em&gt; find a function that is guaranteed to satisfy the Bellman-Equation, but this does not guarantee to result in near-optimal behavior. Several tricks are used to improve convergence and in this case, Q-learning is more sample efficient.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-04-22T12:05:54.623" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20546" PostTypeId="1" CreationDate="2020-04-22T12:14:18.077" Score="3" ViewCount="63" Body="&lt;p&gt;I understand that with a fully observable environment (chess / go etc) you can run an MCTS with an optimal policy network for future planning purposes. This will allow you to pick actions for gameplay, which will result in max expected return from that state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in a partially observable environment, do we still need to run MCTS during gameplay? Why can't we just pick the max action from the trained optimal policy given the current state? What utility does MCTS serve here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to reinforcement learning and am trying to understand the purpose of MCTS / planning in partially observable environments.&lt;/p&gt;&#xA;" OwnerUserId="36355" LastEditorUserId="2444" LastEditDate="2020-04-22T13:41:03.697" LastActivityDate="2020-04-22T13:41:03.697" Title="Is Monte Carlo tree search needed in partially observable environments during gameplay?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20547" PostTypeId="1" CreationDate="2020-04-22T12:35:18.140" Score="1" ViewCount="14" Body="&lt;p&gt;&lt;a href=&quot;https://bdtechtalks.com/2019/05/06/lightelligence-optical-ai-hardware-accelerator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This article&lt;/a&gt; states that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One of the algorithms that photonics is very good at implementing is matrix multiplication&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But how are parameters stored and updated(in backpropagation)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more serious problem is that there are nonlinear operations in neural networks, then how does the photonic neural network deal with activation functions?&lt;/p&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2020-04-22T12:35:18.140" Title="How does optical computing work and deal with nonlinearity?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;hardware&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20548" PostTypeId="2" ParentId="20540" CreationDate="2020-04-22T13:06:16.830" Score="1" Body="&lt;p&gt;There is a category of neural networks that more closely attempt to mimic biological neural networks by incorporating also time (i.e. not all neurons fire at the same time). They are called &lt;a href=&quot;https://www.electronicdesign.com/industrial-automation/article/21806823/brainchip-takes-spiking-neural-networks-to-the-next-level&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;spiking neural networks&lt;/strong&gt;&lt;/a&gt; (SNNs) and their name comes from the fact that they use &lt;em&gt;spiking neurons&lt;/em&gt; (i.e. neurons that fire discrete signals and affect other neurons at possibly different times). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;SNNs are mainly used in neuroscience, and aren't commonly used in machine learning because they currently have some apparent limitations (e.g. non-differentiability, so gradient descent and back-propagation can't be applied, but GD and BP aren't really biologically realistic anyway, although &lt;a href=&quot;https://arxiv.org/abs/1706.04698&quot; rel=&quot;nofollow noreferrer&quot;&gt;some people already tried to apply GD to SNNs&lt;/a&gt;) and their performance isn't still as good as the performance of traditional deep learning models, which make them not so appealing to the deep learning community (which is currently mainly driven by performance and utility). Nevertheless, the performance gap between traditional neural networks and spiking neural networks is decreasing. See &lt;a href=&quot;https://arxiv.org/abs/1804.08150&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning in Spiking Neural Networks&lt;/a&gt; (2019) by Amirhossein Tavanaei et al. for more details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are already commercial implementations of a hardware-accelerated SNNs (e.g. &lt;a href=&quot;https://www.electronicdesign.com/industrial-automation/article/21806823/brainchip-takes-spiking-neural-networks-to-the-next-level&quot; rel=&quot;nofollow noreferrer&quot;&gt;BrainChip&lt;/a&gt; provides this service). These hardware-accelerators are often called neuromorphic chips (or processors) and all computing based on SNNs or processors that attempt to implement biological neural networks is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuromorphic_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;neuromorphic computing&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also the related area called &lt;a href=&quot;https://en.wikipedia.org/wiki/Reservoir_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;reservoir computing&lt;/strong&gt;&lt;/a&gt;, which studies neural networks (such as liquid-state machines or echo state machines) that make use of &lt;a href=&quot;https://www.ifi.uzh.ch/dam/jcr:00000000-2826-155d-0000-0000225e9316/Formale_Methoden_UZH_Nov_2013.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reservoirs&lt;/a&gt; (which are fixed during learning) to attempt e.g. to improve training efficiency. See &lt;a href=&quot;https://www.researchgate.net/publication/221166209_An_overview_of_reservoir_computing_Theory_applications_and_implementations&quot; rel=&quot;nofollow noreferrer&quot;&gt;An overview of reservoir computing: Theory, applications and implementations&lt;/a&gt; (2007) by Benjamin Schrauwen et al. for an overview.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://numenta.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numenta&lt;/a&gt; (and, in particular, Jeff Hawkins, the founder of Numenta and author of an interesting book called &lt;a href=&quot;https://papers.harvie.cz/unsorted/Jeff%20Hawkins%20-%20On%20Intelligence.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Intelligence&lt;/a&gt;) has also been studying neuroscience for a long time in order to develop models and theories of human intelligence. They call their new theory &lt;a href=&quot;https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;The Thousand Brains Theory of Intelligence&lt;/strong&gt;&lt;/a&gt;, which is inspired by biological &lt;a href=&quot;https://numenta.com/blog/2018/05/25/how-grid-cells-map-space/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;grid cells&lt;/strong&gt;&lt;/a&gt;. This is also related and similar to &lt;a href=&quot;https://arxiv.org/abs/1710.09829&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;capsule networks&lt;/strong&gt;&lt;/a&gt; (often associated with Hinton).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-22T16:35:35.333" LastActivityDate="2020-04-22T16:35:35.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20549" PostTypeId="5" CreationDate="2020-04-22T13:27:47.513" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-22T13:27:47.513" LastActivityDate="2020-04-22T13:27:47.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20550" PostTypeId="4" CreationDate="2020-04-22T13:27:47.513" Score="0" Body="For questions related to reservoir computing (RC), which studies neural networks that are based on the concept of a reservoir." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-22T13:27:47.513" LastActivityDate="2020-04-22T13:27:47.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20551" PostTypeId="2" ParentId="20283" CreationDate="2020-04-22T13:29:11.807" Score="1" Body="&lt;p&gt;each episode you will calculate the return, you will then update the action value or &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;  as the average each episode. Using the blackjack example from open &lt;a href=&quot;https://github.com/openai/gym/blob/master/gym/envs/toy_text/blackjack.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI gym&lt;/a&gt; and using a discount factor of 1, you get the following  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;episode 1 &#xA;[{'state': (22, 10, False), 'reward': -1, 'action': 1}, {'state': (17, 10, False), 'reward': 0, 'action': 1}, {'state': (12, 10, False), 'reward': 0.0, 'action': 1}]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q((22, 10, False),0)=-1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q((17, 10, False),1)=-1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q((12, 10, False),1)=-1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;episode 2 &#xA;[{'state': (21, 10, False), 'reward': 1, 'action': 0}, {'state': (17, 10, False), 'reward': 0, 'action': 1}, {'state': (12, 10, False), 'reward': 0.0, 'action': 1}]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q((21, 10, False),0)=1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q((17, 10, False),1)=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q((12, 10, False),1)=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$Q((17, 10, False),1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q((12, 10, False),1)$&lt;/span&gt; is the average return &#xA;i.e -1 for the first episode and 1 for the second.&lt;/p&gt;&#xA;" OwnerUserId="21565" LastEditorUserId="21565" LastEditDate="2020-04-22T15:23:18.137" LastActivityDate="2020-04-22T15:23:18.137" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20553" PostTypeId="1" AcceptedAnswerId="20554" CreationDate="2020-04-22T15:17:19.507" Score="1" ViewCount="42" Body="&lt;p&gt;Is the philosophy between Bellman equations and minimax the same? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both the algorithms look at the full horizon and take into account potential gains (Bellman) and potential losses (minimax). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, do the two differ besides the obvious on the fact that Bellman equations use discounted potential rewards, while minimax deals with potential losses without the discount? Are these enough to say they are similar in philosophy or is are they dissimilar? If so, then in what sense?&lt;/p&gt;&#xA;" OwnerUserId="36047" LastEditorUserId="2444" LastEditDate="2020-04-22T15:42:50.987" LastActivityDate="2020-04-22T15:42:50.987" Title="How are the Bellman optimality equations and minimax related?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;game-theory&gt;&lt;minimax&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20554" PostTypeId="2" ParentId="20553" CreationDate="2020-04-22T15:31:40.523" Score="0" Body="&lt;p&gt;They have similar philosophies, in the sense that minimax and algorithms based on the Bellman optimality equations are used to solve optimization problems, but they are also different because they solve different problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimax (at least, the minimax version that I am aware of) is typically used to solve two-player games (e.g. chess, tic-tac-toe, etc.), while Bellman optimality equations (I assume you are referring to the Bellman equations that algorithms such as policy iteration are based on) do not assume the existence of two players (unless you consider the environment a player).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-22T15:37:32.380" LastActivityDate="2020-04-22T15:37:32.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20555" PostTypeId="1" CreationDate="2020-04-22T17:04:23.080" Score="0" ViewCount="101" Body="&lt;p&gt;Say the game is tic tac toe.&#xA;I found two possible output layers:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Vector of length 9: each float of the vector represents 1 action&#xA;(one of the 9 boxes in Tic Tac Toe). The agent will play the corresponding action with the highest value. The agent learns the rules through trial and error. When the agent tries to make an illegal move (i.e. placing a piece on a box where there is already one), the reward will be harshly negative (-1000 or so).&lt;/li&gt;&#xA;&lt;li&gt;A single float: the float represents who is winning (positive = &quot;the agent is winning&quot;, negative = &quot;the other player is winning&quot;). The agent does not know the rules of the game. Each turn the agent is presented with all the possible next states (resulting from playing each legal action) and it chooses the state with the highest output value.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What other options are there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like the first option because it's cleaner, but it's not feasible with games that have thousands or millions of actions. Also, I am worried that the game might not really learn the rules.&#xA;E.g. Say that in state S the action A is illegal. Say that state R is extremely similar to state S but action A is legal in state R (and maybe in state R action A is actually the best move!). Isn't there the risk that by learning not to play action A in state S it will also learn not to play action A in state R? Probably not an issue in Tic Tac Toe, but likely one in any game with more complex rules.&#xA;What are the disadvantages of option 2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the choice depend on the game? What's your rule of thumb when choosing the output layer?&lt;/p&gt;&#xA;" OwnerUserId="10813" LastEditorUserId="10813" LastEditDate="2020-04-23T12:30:19.527" LastActivityDate="2020-05-30T19:02:40.383" Title="RL: What should be the output of the NN for an agent trying to learn how to play a game?" Tags="&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;&lt;architecture&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20556" PostTypeId="1" CreationDate="2020-04-22T17:06:16.003" Score="1" ViewCount="31" Body="&lt;p&gt;I've just learned about &lt;a href=&quot;https://arxiv.org/abs/1511.06581&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dueling Network Architectures&lt;/a&gt; to estimate &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values and am wondering why this architecture is not used more often in deep RL algorithms? DDPG and TD3 estimate the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function using Double Q Learning instead of the empirically better Dueling Approach.&lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-04-22T17:06:16.003" Title="Why are Dueling Q Networks not used more often to approximate Q-values in reinforcement learning algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20559" PostTypeId="1" AcceptedAnswerId="20560" CreationDate="2020-04-22T18:18:03.127" Score="0" ViewCount="462" Body="&lt;p&gt;In this &lt;a href=&quot;https://towardsdatascience.com/radial-basis-functions-neural-networks-all-we-need-to-know-9a88cc053448&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; it's told:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does it mean? I thought the MLP was a non-linear classifier. Could you explain it to me?&lt;/p&gt;&#xA;" OwnerUserId="36363" LastEditorUserId="2444" LastEditDate="2020-04-22T18:59:18.490" LastActivityDate="2020-04-22T19:01:12.027" Title="Why can't MLPs perform non-linear regression and classification?" Tags="&lt;deep-learning&gt;&lt;multilayer-perceptron&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20560" PostTypeId="2" ParentId="20559" CreationDate="2020-04-22T18:55:13.377" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is &lt;strong&gt;wrong&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A multi-layer perceptron (i.e. a feed-forward neural network) with non-linear activation functions can perform non-linear classification and regression. In fact, an MLP with one hidden layer with an arbitrary number of hidden nodes, each of them with a sigmoid (which is a non-linear function), can approximate any continuous function (up to an approximation error).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, perceptrons can't do that. They perform only linear classification/regression.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I thought the MLP was a non-linear classifier.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You're right, unless the MLP only uses linear activation functions. In that case, it won't be able to perform non-linear classification/regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(P.S.: I suggest you always question the truth and correctness of what you read on the web, especially, on sites like Medium, as you actually did!)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-22T19:01:12.027" LastActivityDate="2020-04-22T19:01:12.027" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20563" PostTypeId="1" CreationDate="2020-04-22T22:15:59.873" Score="1" ViewCount="19" Body="&lt;p&gt;For example, a RL algorithm that gains points when a rat presses a lever and loses points when it dispenses a pellet, water, treat, and/or sugar water. After a few days of controlling the rewards given to a rat, all rewards are stopped and the longer/more times the rat presses the lever before giving up, the higher the score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be a situation in which both the input and the outputs are discrete with very low data density over time and with outputs having very long-term affects on the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of RL architecture would be appropriate here?&lt;/p&gt;&#xA;" OwnerUserId="36368" LastEditorUserId="1671" LastEditDate="2020-04-23T22:40:06.810" LastActivityDate="2020-04-23T22:40:06.810" Title="Has there been any work done on AI-driven operant conditioning?" Tags="&lt;reinforcement-learning&gt;&lt;research&gt;&lt;operant-conditioning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20564" PostTypeId="1" CreationDate="2020-04-22T22:51:37.287" Score="1" ViewCount="105" Body="&lt;p&gt;I am currently using TensorFlow and have simply been trying to train a neural network directly against a large continuous data set, e.g. &lt;span class=&quot;math-container&quot;&gt;$y = [0.014, 1.545, 10.232, 0.948, ...]$&lt;/span&gt; corresponding to different points in time. The loss function in the fully connected neural network (input layer: 3 nodes, 8 inner layers: 20 nodes each, output layer: 1 node) is just the squared error between my prediction and the actual continuous data. It appears the neural network is able to learn the high magnitude data points relatively well (e.g. Figure 1 at time = 0.4422). But the smaller magnitude data points (e.g. Figure 2 at time = 1.1256) are quite poorly learned without any sharpness and I want to improve this. I've tried experimenting with different optimizers (e.g. mini-batch with Adam, full batch with L-BFGS), compared &lt;code&gt;reduce_mean&lt;/code&gt; and &lt;code&gt;reduce_sum&lt;/code&gt;, normalized the data in different ways (e.g. median, subtract the sample mean and divide by the standard deviation, divide the squared loss term by the actual data), and attempted to simply make the neural network deeper and train for a very long period of time (e.g. 7+ days). But after approximately 24 hours of training and the aforementioned tricks, I am not seeing any significant improvements in predicted outputs especially for the small magnitude data points.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Figure 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NvLRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NvLRi.png&quot; alt=&quot;Figure 1 at time = 0.4422&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Figure 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e64Lb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e64Lb.png&quot; alt=&quot;Figure 2 at time = 1.1256&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Therefore, do you have any recommendations on how to improve training particularly when there are different data points of varying magnitude I am trying to learn? I believe &lt;a href=&quot;https://stats.stackexchange.com/questions/221476/feature-varying-in-orders-of-magnitude&quot;&gt;this&lt;/a&gt; is a related question, but any explicit examples of implementations or techniques to handle varying orders of magnitude within a single large data set would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="21895" LastActivityDate="2021-05-23T05:00:50.143" Title="How to improve neural network training against a large data set of points with varying magnitude" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;optimization&gt;&lt;batch-normalization&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20567" PostTypeId="1" AcceptedAnswerId="20574" CreationDate="2020-04-23T02:24:37.860" Score="1" ViewCount="157" Body="&lt;p&gt;I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible? &lt;/p&gt;&#xA;" OwnerUserId="36175" LastEditorUserId="2444" LastEditDate="2020-11-22T21:50:20.653" LastActivityDate="2020-11-23T11:05:12.157" Title="How can a machine learning problem be reduced as a communication problem?" Tags="&lt;computational-learning-theory&gt;&lt;information-theory&gt;&lt;bayesian-deep-learning&gt;&lt;minimum-description-length&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20570" PostTypeId="1" CreationDate="2020-04-23T09:39:41.123" Score="3" ViewCount="161" Body="&lt;p&gt;I'm working on a RL problem with the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The rewards are &lt;strong&gt;extremely sparse&lt;/strong&gt; i.e. all rewards are 0 except the terminal non-zero reward. Ideally I would not use any reward engineering as that would lead to a different optimization problem.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Actions are continuous&lt;/strong&gt;. Discretization should not be used.&lt;/li&gt;&#xA;&lt;li&gt;The amount of &lt;strong&gt;stochasticity in the environment is very high&lt;/strong&gt; i.e. for a fixed deterministic policy the variance of returns is very high.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;More specifically, the RL agent represents the investor, the terminal reward represents the utility of the terminal wealth (hence the sparsity), actions represent portfolio positions (hence the continuity) and the environment represents the financial market (hence the high stochasticity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been trying to use DDPG with a set of &quot;commonly used&quot; hyperparameters (as I have no idea have to tune them besides experimenting which lasts too long) but so far (after 10000 episodes) it seems that nothing is happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yjA9O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yjA9O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Given the nature of the problem I'm trying to solve (sparse rewards, continuous actions, stochasticity) is there a particular (D)RL algorithm that would lend itself well to it?&lt;/li&gt;&#xA;&lt;li&gt;How likely is it that DDPG simply won't converge to a reasonable solution (due to the peculiarities of the problem itself) no matter what set of hyperparameters I choose?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26195" LastActivityDate="2020-04-26T06:42:18.690" Title="Appropriate algorithm for RL problem with sparse rewards, continuous actions and significant stochasticity" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;policy-gradients&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20571" PostTypeId="2" ParentId="4766" CreationDate="2020-04-23T09:50:52.870" Score="0" Body="&lt;p&gt;Geoffrey Hinton has started working on Thought Vectors at Google: &lt;a href=&quot;https://en.wikipedia.org/wiki/Thought_vector&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Thought_vector&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is similar to his original idea with Capsule Networks, where activation happens by vectors instead of scalars, which allows the network to capture transformations: for example while traditional CNN needs to see object from all perspectives of three dimensional space, the Capsule networks are able to extrapolate transformations such as stretching much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thought Vectors guide NLP similarly; one could say that there are two grammars, the linguistic grammar and the narrative grammar which is more universal (Vladimir Propp, Joseph Campbell, John Vervake). While dependency grammars do great job at understanding linguistic grammar, we lack tools for meaning extraction, which is narrative bound. Thus Thought Vectors could, at least in theory, give us a framework for matching the meaning of a word within a context rather than just lexically and grammarly trying to approximate the meaning through average co-occurances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural Networks with Thought Vectors would be highly complex and beyond our computational resources today (Hinton predicts in one paper, that we would get there around 2035), however, one could conduct empirical research already by giving a heuristic structure for Thought Vectors by utilizing narrative systems that do compute more easily. One could for example have text segments annotated with writing theories or other such devices that would approximate the Thought Vectors conceptually. For example annotating the text with state transformations of conflict driven partially ordered causal link planner (cPOCL, Gervas et al.) or use a writing theory framework such as Dramatica to annotate known movie scripts (&lt;a href=&quot;http://dramatica.com/theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dramatica.com/theory&lt;/a&gt; &lt;a href=&quot;http://dramatica.com/analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dramatica.com/analysis&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hinton himself is currently active in NLP research: &lt;a href=&quot;https://research.google/people/GeoffreyHinton/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://research.google/people/GeoffreyHinton/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a nice explanation of Thought Vectors: &lt;a href=&quot;https://pathmind.com/wiki/thought-vectors&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pathmind.com/wiki/thought-vectors&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11626" LastEditorUserId="11626" LastEditDate="2020-06-15T14:33:32.257" LastActivityDate="2020-06-15T14:33:32.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20572" PostTypeId="1" CreationDate="2020-04-23T11:22:19.800" Score="0" ViewCount="39" Body="&lt;p&gt;I'm trying to implement a simple actor-critic algorithm based on action-value critic. I'm following the pseudocode from David Silver (please find attached his slide)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uAXeg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uAXeg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Network:&#xA;def __init__(self):&#xA;&#xA;    self.start_learning = 100&#xA;    self.gamma = 0.99&#xA;    self.learning_rate = 1e-2&#xA;    self.train_nums = 10000&#xA;&#xA;    self.gp = GamePlay()&#xA;&#xA;    self.actor = PolicyModel(self.gp.num_actions, 'TrainNet_policy')&#xA;    self.critic = ValueModel(self.gp.num_actions, 'TrainNet_value')&#xA;&#xA;    self.opt = ko.Adam(learning_rate= self.learning_rate)&#xA;&#xA;    #self.actor.compile(loss='categorical_crossentropy', optimizer=self.opt)&#xA;    #self.critic.compile(loss='mse', optimizer=self.opt)&#xA;&#xA;def train(self):&#xA;    state = self.gp.reset()&#xA;    ep_rewards = 0&#xA;    losses = []&#xA;    episode_length = []&#xA;    length = 0&#xA;    mean_len = 0&#xA;    num_episodes = 2000&#xA;    for ep in range(1, num_episodes):&#xA;        done = False&#xA;        while not done:&#xA;            # Select actions using the actor.&#xA;            softmax_values, best_action = self.actor.action_value(state[None], self.gp.num_actions)&#xA;&#xA;            action_probs = tf.squeeze(softmax_values)&#xA;            picked_action_prob = tf.gather(action_probs, best_action)&#xA;            next_state, reward, done = self.gp.next_step(best_action)&#xA;            ep_rewards += reward&#xA;            length += 1&#xA;            #loss = self.train_step(next_state, state, reward, picked_action_prob)&#xA;&#xA;            values_next = self.critic.predict(next_state)&#xA;            td_target = reward + self.gamma * values_next&#xA;&#xA;            with tf.GradientTape(persistent=True) as tape:&#xA;                predict_q   = self.critic.predict_on_batch(state)&#xA;                td_error    = tf.math.subtract(td_target, predict_q)&#xA;                critic_loss = tf.math.reduce_mean(tf.square(td_error))&#xA;                actor_loss  = - tf.math.reduce_sum(tf.math.multiply(tf.math.log(picked_action_prob), predict_q))&#xA;&#xA;            critic_variables = self.critic.trainable_variables&#xA;            actor_variables  = self.actor.trainable_variables&#xA;&#xA;            critic_gradients = tape.gradient(critic_loss, critic_variables)&#xA;            actor_gradients  = tape.gradient(actor_loss, actor_variables)&#xA;&#xA;            self.opt.apply_gradients(zip(critic_gradients, critic_variables))&#xA;            self.opt.apply_gradients(zip(actor_gradients, actor_variables))&#xA;&#xA;            losses.append(actor_loss)&#xA;&#xA;            if done:&#xA;                mean_len += length&#xA;                episode_length.append(length)&#xA;                length = 0&#xA;                state = self.gp.reset()&#xA;                ep_rewards = 0&#xA;                if ep % 10 == 0:&#xA;                    print (&quot;episode {} mean reward {} loss {}&quot;.format(ep, mean_len/10, np.mean(losses)))&#xA;                    mean_len = 0&#xA;            else:&#xA;                state = next_state&#xA;    return losses, episode_length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;self.gp&lt;/code&gt; is a wrapper around the gym environment for any game.&#xA;The policy network has the final layer as softmax, whereas the critic has a linear output. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the gradient computation process, it says that I have no gradient provided from &lt;code&gt;actor_gradients&lt;/code&gt;. What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concretely, this is the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: No gradients provided for any variable: ['TrainNet_policy/kernel:0', 'TrainNet_policy/bias:0', 'TrainNet_policy_1/kernel:0', 'TrainNet_policy_1/bias:0', 'TrainNet_policy_q_values/kernel:0', 'TrainNet_policy_q_values/bias:0'].&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35938" LastActivityDate="2020-04-23T11:22:19.800" Title="Gradients null for actor in Acton-Value actor-critic" Tags="&lt;tensorflow&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20574" PostTypeId="2" ParentId="20567" CreationDate="2020-04-23T13:26:48.947" Score="2" Body="&lt;h2&gt;Information-theoretic view of Bayesian learning&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, this is indeed possible. More precisely, there is an &lt;strong&gt;information-theoretic view&lt;/strong&gt; of Bayesian learning in neural networks, which can also be thought of as a communication problem, which explains both &lt;em&gt;maximum a posteriori estimation (MAPE)&lt;/em&gt; and &lt;em&gt;full&lt;/em&gt; Bayesian learning [&lt;a href=&quot;https://www.cs.helsinki.fi/u/ahonkela/papers/infview.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], i.e. finding the posteriors over the weights of the neural network: the neural networks that maintain a probability distribution over the weights are now known as &lt;strong&gt;Bayesian neural networks&lt;/strong&gt; (and, in terms of theory, they are strongly related/similar to the famous &lt;em&gt;variational auto-encoders&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The oldest relevant paper (I am aware of) that &lt;em&gt;interprets&lt;/em&gt; Bayesian learning in neural networks as a &lt;em&gt;communication problem&lt;/em&gt; is the 1993 paper by Hinton and Van Camp entitled &lt;a href=&quot;https://dl.acm.org/doi/pdf/10.1145/168304.168306&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keeping the neural networks simple by minimizing the description length of the weights&lt;/a&gt; (COLT), which is the paper that introduces &lt;em&gt;variational Bayesian neural networks&lt;/em&gt; (sometimes called &lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/bishop-ensemble-nato-98.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;ensemble learning&lt;/em&gt; in some papers from the 1990s&lt;/a&gt;), i.e. variational inference (VI) applied to neural networks (yes, the same VI used in VAEs). &lt;strong&gt;Hinton&lt;/strong&gt; (yes, the famous Hinton that won the Turing award) and Van Camp (who is this? probably a Dutch guy from the name!) write in this paper&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We can think in terms of a &lt;strong&gt;sender&lt;/strong&gt; who can see both the input vector and the correct output and a &lt;strong&gt;receiver&lt;/strong&gt; who can only see the input vector. The sender first fits a neural network, of pre-arranged architecture, to the complete set of training cases, then sends the weights to the receiver. For each training case, the sender also sends the discrepancy between the net's output and the correct output. By adding this discrepancy to the output of the net, the receiver can generate exactly the correct output.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You should read this seminal paper if you want to understand all the details.&lt;/p&gt;&#xA;&lt;p&gt;Another relevant paper is &lt;a href=&quot;https://papers.nips.cc/paper/2011/file/7eb3c8be3d411e8ebfab08eba5f49632-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Variational Inference for Neural Networks&lt;/a&gt; (2013, NeurIPS) by Graves, who cites the 1993 paper immediately at the beginning of the paper. Essentially, as the title of the paper suggests, Graves tries to make VI in neural networks practical.&lt;/p&gt;&#xA;&lt;p&gt;There are other relevant papers that still attempt to provide this information-theoretic view of Bayesian learning, such as &lt;a href=&quot;https://www.cs.helsinki.fi/u/ahonkela/papers/infview.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variational learning and bits-back coding: An information-theoretic view to bayesian learning&lt;/a&gt; (2004, IEEE Transactions on Neural networks), but most current papers on Bayesian neural networks, such as &lt;a href=&quot;http://proceedings.mlr.press/v37/blundell15.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Uncertainty in Neural Networks&lt;/a&gt; (2015, PMLR) don't do it (at most they may mention that this interpretation exists, but they don't go into the details).&lt;/p&gt;&#xA;&lt;h3&gt;Minimum description length&lt;/h3&gt;&#xA;&lt;p&gt;To give you a few more details, the information-theoretic view of Bayesian learning in these papers is that of the &lt;strong&gt;minimum description length (MDL)&lt;/strong&gt;, i.e. Bayesian learning (i.e. the application of Bayes rule to find the posteriors over the parameters of the model) is equivalent to finding a model that gives the &amp;quot;shortest description of the data&amp;quot; (hence the name MDL), where a description is some &lt;a href=&quot;https://en.wikipedia.org/wiki/Coding_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;code/encoding of the data&lt;/a&gt;: in the case of the NNs, this encoding is contained in their weights.&lt;/p&gt;&#xA;&lt;p&gt;Given that you want to find the simplest code, then this is a direct application of &lt;strong&gt;Occam's razor&lt;/strong&gt;: &lt;em&gt;if you have multiple hypotheses/functions that describe your data (or are consistent with your observations), then choose the simplest one&lt;/em&gt;. Occam's razor underlies many other mathematical/ML theories and frameworks, for example, &lt;a href=&quot;https://jan.leike.name/AIXI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIXI&lt;/a&gt;, a framework for artificial general intelligence developed by Marcus Hutter. Jürgen Schmidhuber is also a good fan of Occam's razor and &lt;em&gt;compression&lt;/em&gt; as a means to act intelligently (see e.g. the &lt;a href=&quot;http://people.idsia.ch/%7Ejuergen/speedprior.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;speed prior&lt;/a&gt;). If you are familiar with deep learning, a light bulb should turn on in your brain now. Yes, &lt;strong&gt;regularization&lt;/strong&gt; techniques to avoid over-fitting and improve generalization can also be viewed as an application of Occam's razor principle.&lt;/p&gt;&#xA;&lt;h3&gt;Bits-back coding&lt;/h3&gt;&#xA;&lt;p&gt;How do we find the simplest weights? The &lt;strong&gt;bits-back coding&lt;/strong&gt;, used by the 1993 paper and described in the 2004 and 2013 papers, essentially states that you can find the simplest encoding (i.e. posterior over the weights) by minimizing the &lt;strong&gt;Kullback-Leibler divergence&lt;/strong&gt; (aka &lt;em&gt;relative entropy&lt;/em&gt;: say what?!) between the posterior (which is unknown: so how can we compute the KL divergence?) and some prior (coding distribution), which is zero when the prior is equal to the posterior (but we don't know the posterior) [&lt;a href=&quot;https://www.cs.helsinki.fi/u/ahonkela/papers/infview.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. Given that we don't know the posterior, we need to use a proxy objective function that doesn't involve the posterior, such as the &lt;strong&gt;Evidence Lower BOund&lt;/strong&gt; (ELBO), also known as &lt;strong&gt;variational free-energy&lt;/strong&gt;, which leads to a non-optimal coding (i.e. possibly, you will find some posteriors that are not optimal given the data).&lt;/p&gt;&#xA;&lt;h2&gt;Conclusions&lt;/h2&gt;&#xA;&lt;p&gt;Using MAPE or performing (approximate) Bayesian learning in a neural network (which finds one function or a probability distribution over functions, respectively) can be interpreted as finding the MDL, i.e. an optimal or near-optimal encoding of the data that needs to be communicated from a sender to a receiver.&lt;/p&gt;&#xA;&lt;h2&gt;Side notes&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Information theory&lt;/strong&gt; was pioneered by Claude Shannon in his 1948 seminal paper &lt;a href=&quot;http://people.math.harvard.edu/%7Ectm/home/text/others/shannon/entropy/entropy.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Mathematical Theory of Communication&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Claude Shannon was also one of the participants at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dartmouth_workshop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dartmouth workshop&lt;/a&gt;, which officially started the field of artificial intelligence, so he is one of the fathers of the AI field, and his impact on the field is definitely huge (although most people are not aware of it, but, hopefully, this answer will change that).&lt;/p&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;Apart from the papers that I cited above, you may also be interested in &lt;a href=&quot;https://arxiv.org/pdf/1501.04309.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Information Theory and its Relation to Machine Learning&lt;/a&gt; (2015) by Hu.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T11:05:12.157" LastActivityDate="2020-11-23T11:05:12.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20576" PostTypeId="1" CreationDate="2020-04-23T17:43:44.697" Score="1" ViewCount="29" Body="&lt;p&gt;I am following the tutorial &lt;a href=&quot;https://www.tensorflow.org/agents/tutorials/1_dqn_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Train a Deep Q Network with TF-Agents&lt;/a&gt;. It uses the &lt;code&gt;hello world&lt;/code&gt; environment of reinforced learning: &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cart pole&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end, the agent is getting trained with experience on the training enviroment (&lt;code&gt;train_env&lt;/code&gt;). When performing an action in the environment a &lt;code&gt;time_step&lt;/code&gt; is returned containing the observation, award and whether the environment signals the end, which basically says 'game over'. This can be due to the pole reaching an angle which is too high, or 200 time steps have been reached (which is the max score, atleast in the tutorial).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When &lt;code&gt;compute_avg_return&lt;/code&gt; method evaluates an agents performance on an environment, the environment is checked to be game over or not using &lt;code&gt;time_step.is_last&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is &lt;code&gt;time_step.is_last&lt;/code&gt; not considered when training the agent at the end of the tutorial? Nor do I see that the environment is reset during training. Or at least, I do not see it in the code presented. Is it checked internally? Looking at the graph, it never goes over an average return (the score) of 200 time steps. So it does seem to check for &lt;code&gt;time_step.is_last&lt;/code&gt;. Do I overlook something or how does this work? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the code block below. I would expect the check for &lt;code&gt;time_step.is_last&lt;/code&gt; after &lt;code&gt;collect_step(train_env, agent.collect_policy, replay_buffer)&lt;/code&gt;, which would be followed by resetting the environment if it was true.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Reset the train step&#xA;agent.train_step_counter.assign(0)&#xA;&#xA;# Evaluate the agent's policy once before training.&#xA;avg_return = compute_avg_return(eval_env, agent.policy, num_eval_episodes)&#xA;returns = [avg_return]&#xA;&#xA;for _ in range(num_iterations):&#xA;&#xA;  # Collect a few steps using collect_policy and save to the replay buffer.&#xA;  for _ in range(collect_steps_per_iteration):&#xA;    collect_step(train_env, agent.collect_policy, replay_buffer)&#xA;&#xA;  # Sample a batch of data from the buffer and update the agent's network.&#xA;  experience, unused_info = next(iterator)&#xA;  train_loss = agent.train(experience).loss&#xA;&#xA;  step = agent.train_step_counter.numpy()&#xA;&#xA;  if step % log_interval == 0:&#xA;    print('step = {0}: loss = {1}'.format(step, train_loss))&#xA;&#xA;  if step % eval_interval == 0:&#xA;    avg_return = compute_avg_return(eval_env, agent.policy, num_eval_episodes)&#xA;    print('step = {0}: Average Return = {1}'.format(step, avg_return))&#xA;    returns.append(avg_return)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32968" LastEditorUserId="32968" LastEditDate="2020-04-23T17:53:05.867" LastActivityDate="2020-04-23T17:53:05.867" Title="Why does this tutorial on reinforced learning not check whether the environment is 'game over' during training?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;environment&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20579" PostTypeId="2" ParentId="20325" CreationDate="2020-04-23T21:42:49.710" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt;!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is very possibly the most famous single algorithm of all time, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;predates modern computers&lt;/a&gt; by a couple of decades.  It is a simple decision rule with general utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Some might say that this algorithm represents a form of general intelligence, in that it can be so widely applied, although that statement would likely be controversial.)&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2020-04-23T21:42:49.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20581" PostTypeId="5" CreationDate="2020-04-23T22:41:40.697" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Operant conditioning (also called instrumental conditioning) is a type of associative learning process through which the strength of a behavior is modified by reinforcement or punishment. It is also a procedure that is used to bring about such learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Operant_conditioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Operant_conditioning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-04-23T22:41:40.697" LastActivityDate="2020-04-23T22:41:40.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20582" PostTypeId="4" CreationDate="2020-04-23T22:41:40.697" Score="0" Body="For questions about the concept of operant conditions as related to AI." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-04-23T22:41:40.697" LastActivityDate="2020-04-23T22:41:40.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20583" PostTypeId="2" ParentId="8509" CreationDate="2020-04-23T23:01:35.317" Score="0" Body="&lt;p&gt;Is it the case that one of the numbers if filled in? If so a CNN with 10 output should work well. Just choose the output that has the highest probability. If your data allows no number to be filled in, then have 11 outputs where the eleventh output indicates none is filled in. I would recommend transfer learning using the MobileNet model. Documentation is &lt;a href=&quot;https://keras.io/applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Here is the code to adapt MobileNet to your problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;image_size=128&#xA;no_of_classes=10  # set to 11 if in some cases no numbers are filled in&#xA;lr_rate=.001&#xA;dropout=.4&#xA;mobile = tf.keras.applications.mobilenet.MobileNet( include_top=False,&#xA;                                                           input_shape=(image_size,image_size,3),&#xA;                                                           pooling='avg', weights='imagenet',&#xA;                                                           alpha=1, depth_multiplier=1)&#xA;x=mobile.layers[-1].output&#xA;x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)&#xA;x=Dense(128, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),&#xA;                bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)&#xA;x=Dropout(rate=dropout, seed=128)(x)&#xA;x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)&#xA;predictions=Dense (no_of_classes, activation='softmax')(x)&#xA;model = Model(inputs=mobile.input, outputs=predictions)    &#xA;for layer in model.layers:&#xA;    layer.trainable=True&#xA;model.compile(Adam(lr=lr_rate), loss='categorical_crossentropy', metrics=['accuracy']) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would also create a training, test and validation set with about 150 images in the test set and 150 images in the validation set leaving 1700 images for training. I also recommend you use two useful callbacks. Documentation is &lt;a href=&quot;https://keras.io/callbacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Use the ReduceLROnPlateau callback to monitor the validation loss and adjust the learning rate downward by a factor. Use ModelCheckpoint to monitor the validation loss and save the model with the lowest validation loss to use to make predictions on the test set.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-23T23:01:35.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20584" PostTypeId="2" ParentId="13390" CreationDate="2020-04-23T23:09:45.997" Score="0" Body="&lt;p&gt;There's something called &lt;a href=&quot;https://arxiv.org/pdf/1612.00796.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elastic Weight Consolidation&lt;/a&gt; to prevent neural networks from forgetting previous tasks as they train on new tasks. It might be helpful for your case too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main idea is to quantify the importance of parameters for task &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and penalize the model in proportion when it changes its parameters as it trains to learn task &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;. As you can see, this incentivizes model to change parameters that are less important for task &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; which prevents the model from forgetting it. &lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-04-23T23:09:45.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20585" PostTypeId="1" AcceptedAnswerId="22384" CreationDate="2020-04-24T01:56:01.473" Score="5" ViewCount="207" Body="&lt;p&gt;In &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the previous research&lt;/a&gt;, in 2015, Deep Q-Learning shows its great performance on single player Atari Games. But why do AlphaGo's researchers use CNN + MCTS instead of Deep Q-Learning? is that because Deep Q-Learning somehow is not suitable for Go?&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2020-04-29T17:35:36.753" LastActivityDate="2020-07-07T20:03:42.870" Title="Why AlphaGo didn't use Deep Q-Learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20586" PostTypeId="1" AcceptedAnswerId="20590" CreationDate="2020-04-24T03:04:37.827" Score="1" ViewCount="49" Body="&lt;p&gt;In Sutton and Barto's RL textbook they included the following pseudocode for off policy Monte Carlo learning. I am a little confused, however, because to me it looks like the W term will become infinitely large after a couple thousand iterations (and this is exactly what happens when I implement the algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say that the MC algorithm always follows the behavioral policy for each episode (ignoring epsilon soft/greedy for examples sake). If the probability of the action specified by the policy is 0.9, then after 10,000 iterations W would have a value of 1.11^10,000. I understand that the ratio of W to C(a,s) is what matters, however this ratio cannot be computer once W becomes infinite. Clearly I am misunderstanding something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RubOG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RubOG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36404" LastActivityDate="2020-04-24T07:33:40.537" Title="Understanding the W term in off policy monte carlo learning" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20587" PostTypeId="1" CreationDate="2020-04-24T03:53:55.653" Score="1" ViewCount="20" Body="&lt;p&gt;I am not able to understand why expected sarsa algorithm is an off-policy learning method? &#xA;How is it that behaviour and target policies can be different?&#xA;Also please tell under what situation and conditions is expected sarsa equivalent to Q-learning?&lt;/p&gt;&#xA;" OwnerUserId="35926" LastActivityDate="2020-04-24T03:53:55.653" Title="Why is expected sarsa algorithm an off-policy learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;expected-sarsa&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ClosedDate="2020-04-24T12:09:09.837" ContentLicense="CC BY-SA 4.0" />
  <row Id="20588" PostTypeId="1" AcceptedAnswerId="21072" CreationDate="2020-04-24T04:24:55.857" Score="3" ViewCount="250" Body="&lt;p&gt;From &lt;a href=&quot;https://codereview.stackexchange.com/questions/217065/calculate-levenshtein-distance-between-two-strings-in-python&quot;&gt;Calculate Levenshtein distance between two strings in Python&lt;/a&gt; it is possible to calculate distance and similarity between two given strings(sentences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And from &lt;a href=&quot;https://stackabuse.com/levenshtein-distance-and-text-similarity-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Levenshtein Distance and Text Similarity in Python&lt;/a&gt; to return the matrix for each character and distance for two strings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any ways to calculate distance and similarity between each word in a string and print the matrix for each word in a string(sentences)?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = &quot;This is a dog.&quot;&#xA;b = &quot;This is a cat.&quot;&#xA;&#xA;from difflib import ndiff&#xA;&#xA;def levenshtein(seq1, seq2):&#xA;    size_x = len(seq1) + 1&#xA;    size_y = len(seq2) + 1&#xA;    matrix = np.zeros ((size_x, size_y))&#xA;    for x in range(size_x):&#xA;        matrix [x, 0] = x&#xA;    for y in range(size_y):&#xA;        matrix [0, y] = y&#xA;&#xA;    for x in range(1, size_x):&#xA;        for y in range(1, size_y):&#xA;            if seq1[x-1] == seq2[y-1]:&#xA;                matrix [x,y] = min(&#xA;                    matrix[x-1, y] + 1,&#xA;                    matrix[x-1, y-1],&#xA;                    matrix[x, y-1] + 1&#xA;                )&#xA;            else:&#xA;                matrix [x,y] = min(&#xA;                    matrix[x-1,y] + 1,&#xA;                    matrix[x-1,y-1] + 1,&#xA;                    matrix[x,y-1] + 1&#xA;                )&#xA;    print (matrix)&#xA;    return (matrix[size_x - 1, size_y - 1])&#xA;&#xA;levenshtein(a, b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Outputs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Matrix&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12. 13. 14.]&#xA; [ 1.  0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12. 13.]&#xA; [ 2.  1.  0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12.]&#xA; [ 3.  2.  1.  0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11.]&#xA; [ 4.  3.  2.  1.  0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]&#xA; [ 5.  4.  3.  2.  1.  0.  1.  2.  3.  4.  5.  6.  7.  8.  9.]&#xA; [ 6.  5.  4.  3.  2.  1.  0.  1.  2.  3.  4.  5.  6.  7.  8.]&#xA; [ 7.  6.  5.  4.  3.  2.  1.  0.  1.  2.  3.  4.  5.  6.  7.]&#xA; [ 8.  7.  6.  5.  4.  3.  2.  1.  0.  1.  2.  3.  4.  5.  6.]&#xA; [ 9.  8.  7.  6.  5.  4.  3.  2.  1.  0.  1.  2.  3.  4.  5.]&#xA; [10.  9.  8.  7.  6.  5.  4.  3.  2.  1.  0.  1.  2.  3.  4.]&#xA; [11. 10.  9.  8.  7.  6.  5.  4.  3.  2.  1.  1.  2.  3.  4.]&#xA; [12. 11. 10.  9.  8.  7.  6.  5.  4.  3.  2.  2.  2.  3.  4.]&#xA; [13. 12. 11. 10.  9.  8.  7.  6.  5.  4.  3.  3.  3.  3.  4.]&#xA; [14. 13. 12. 11. 10.  9.  8.  7.  6.  5.  4.  4.  4.  4.  3.]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;General Levenshtein distance for character level shown in below fig. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/wYCIl.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wYCIl.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Is it possible to calculate Levenshtein Distance for Word Level?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Required Matrix&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;          This is a cat&#xA;&#xA;This&#xA;is&#xA;a&#xA;dog&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-04-27T10:22:41.053" LastActivityDate="2020-05-11T15:37:10.960" Title="Levenshtein Distance between each word in a given string" Tags="&lt;natural-language-processing&gt;&lt;similarity&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20589" PostTypeId="1" CreationDate="2020-04-24T04:50:44.007" Score="0" ViewCount="56" Body="&lt;p&gt;I have the traffic of my network (with hundreds of entries). Below I am showing only 9 entries.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1    ETH: 28:28:5d:fb:88:38 6c:71:d9:8e:88:54 (IP) 97&#xA;2    IP: 169.44.99.105 192.168.1.41 5 4 83 16384&#xA;3    ETH: 6c:71:d9:8e:88:54 28:28:5d:fb:88:38 (IP) 101&#xA;4    IP: 192.168.1.41 169.44.99.105 5 4 87 16384&#xA;5    ETH: 28:28:5d:fb:88:38 6c:71:d9:8e:88:54 (IP) 66&#xA;6    IP: 169.44.99.105 192.168.1.41 5 4 52 16384&#xA;7    ETH: 28:28:5d:fb:88:38 6c:71:d9:8e:88:54 (?) 139&#xA;8    ETH: 6c:71:d9:8e:88:54 28:28:5d:fb:88:38 (?) 86&#xA;9    ETH: 28:28:5d:fb:88:38 6c:71:d9:8e:88:54 (?) 117&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to group these entries by how similar they are among each other. But what should I do to group them? What do I need to do first e.g. to use a clustering algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i am following this tutorial to get the datas&#xA;&lt;a href=&quot;http://yuba.stanford.edu/~casado/pcap/section4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yuba.stanford.edu/~casado/pcap/section4.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here there are a example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ETH: 0:10:a4:8b:d3:b4 ff:ff:ff:ff:ff:ff (ARP) 42&#xA;ETH: 0:20:78:d1:e8:1 0:10:a4:8b:d3:b4 (ARP) 60&#xA;ETH: 0:10:a4:8b:d3:b4 0:20:78:d1:e8:1 (IP) 74&#xA;IP: 192.168.1.100 134.114.90.1 5 4 60 16384&#xA;ETH: 0:20:78:d1:e8:1 0:10:a4:8b:d3:b4 (IP) 60&#xA;IP: 134.114.90.1 192.168.1.100 5 4 40 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Lets try and reconstruct the conversation shall we?&#xA;my computer: Who has the gateways IP (192.168.1.100)?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ETH: 0:10:a4:8b:d3:b4 ff:ff:ff:ff:ff:ff (ARP) 42&#xA;gateway: I do!!&#xA;ETH: 0:20:78:d1:e8:1 0:10:a4:8b:d3:b4 (ARP) 60&#xA;my computer(through gateway): Hello Mr. 134.114.90.1 can we talk?&#xA;ETH: 0:10:a4:8b:d3:b4 0:20:78:d1:e8:1 (IP) 74 IP: 192.168.1.100 134.114.90.1 5 4 60 16384&#xA;134.114.90.1: Nope, I'm not listening&#xA;ETH: 0:20:78:d1:e8:1 0:10:a4:8b:d3:b4 (IP) 60 IP: 134.114.90.1 192.168.1.100 5 4 40 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36409" LastEditorUserId="36409" LastEditDate="2020-04-26T14:54:41.727" LastActivityDate="2020-04-26T14:54:41.727" Title="How can I group the entries of the network traffic by their similarity?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;pattern-recognition&gt;&lt;data-preprocessing&gt;&lt;clustering&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20590" PostTypeId="2" ParentId="20586" CreationDate="2020-04-24T07:33:40.537" Score="2" Body="&lt;p&gt;The pseudocode you have copied looks incorrect to me, and I think it is from the first edition.&lt;/p&gt;&#xA;&lt;p&gt;The main issue is at the end of the loop. Where the book has&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad W \leftarrow W \frac{1}{\mu(A_t|S_t)}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad \text{If } W = 0 \text{ then ExitForLoop}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It should have either&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad W \leftarrow W \frac{1}{\mu(A_t|S_t)}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad \text{If } \pi(S_t) \neq A_t \text{ then ExitForLoop}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad W \leftarrow W \frac{\pi(A_t|S_t)}{\mu(A_t|S_t)}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad \text{If } W = 0 \text{ then ExitForLoop}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This latter one is more general - it covers situations where the target policy can be stochastic - but doesn't fit with the notation used elsewhere for a deterministic policy. For some reason the first edition book had a mistake using hybrid algorithm which adjusted to a deterministic target policy except for the exit loop statement. This is fixed in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the second edition&lt;/a&gt; (page 111).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;after 10,000 iterations&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Are your episodes really 10,000 time steps long? If so, the chances of off-policy MC control learning anything for early time steps seems remote unless &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is really low (and in which case &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; will not get too high). If not, have you missed that &lt;span class=&quot;math-container&quot;&gt;$W \leftarrow 1$&lt;/span&gt; occurs at the start of each episode?&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-24T07:33:40.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20591" PostTypeId="1" CreationDate="2020-04-24T09:15:59.367" Score="1" ViewCount="610" Body="&lt;p&gt;I wonder if there's anyone who has actually succeeded in fine-tuning GPT-2's 774M model without using cloud TPU's. My GeForce RTX 2070 SUPER couldn't handle it in previous attempts.&lt;/p&gt;&#xA;&lt;p&gt;I'm running TensorFlow 1.14.0 with CUDA V 9.1 on Ubuntu 18.04. For fine-tuning I'm using &lt;a href=&quot;https://github.com/minimaxir/gpt-2-simple&quot; rel=&quot;nofollow noreferrer&quot;&gt;gpt-2-simple&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When fine-tuning using the 77M model, I keep running into OOM errors, such as:&#xA;&lt;code&gt;W tensorflow/core/common_runtime/bfc_allocator.cc:314] Allocator (GPU_0_bfc) ran out of memory trying to allocate 6.25MiB (rounded to 6553600).  Current allocation summary follows.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So far I've tried:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using different a optimizer (&lt;code&gt;RMSPropOptimizer&lt;/code&gt; instead of &lt;code&gt;AdamOptimizer&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Setting batch-size to 1&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;use_memory_saving_gradients&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;only_train_transformer_layers&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Fine-tuning works smoothly on the 355M model.&lt;/p&gt;&#xA;&lt;p&gt;So what I'm really asking is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is it possible to fine-tune GPT-2's 774M model without industrial-sized hardware?&lt;/li&gt;&#xA;&lt;li&gt;if so, please tell me about your successful attempts&lt;/li&gt;&#xA;&lt;li&gt;apart from hardware-recommendations, how could fine-tuning be optimized to make 77M fit in memory?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33476" LastEditorUserId="2444" LastEditDate="2021-04-24T12:39:01.043" LastActivityDate="2021-04-24T12:39:01.043" Title="GPT-2: (Hardware) requirements for fine-tuning the 774M model" Tags="&lt;natural-language-processing&gt;&lt;gpt&gt;&lt;fine-tuning&gt;&lt;gpt-2&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20592" PostTypeId="1" AcceptedAnswerId="20603" CreationDate="2020-04-24T10:31:57.680" Score="1" ViewCount="43" Body="&lt;p&gt;According &lt;a href=&quot;https://ai.stackexchange.com/questions/13451/does-this-hyperparameter-optimisation-approach-yield-the-optimal-hyperparameters&quot;&gt;this thread&lt;/a&gt; some hyperparameters are independent from each other while some are directly related.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the answers give an example where two hyperparameters affect each other.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, if you're using stochastic gradient descent (that is, you train your model one example at a time), you probably do not want to update the parameters of your model too fast (that is, you probably do not want a high learning rate), given that a single training example is unlikely to be able to give the error signal that is able to update the parameters in the appropriate direction (that is, the global or even local optimum of the loss function). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How would someone creating a neural network know how the hyperparameters affect each other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, what are the heuristics for hyperparameter selection when trying to build a robust model?&lt;/p&gt;&#xA;" OwnerUserId="32265" LastActivityDate="2020-04-25T03:50:06.503" Title="How to know if the hyperparameters of a neural network relate to each other?" Tags="&lt;neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20594" PostTypeId="1" CreationDate="2020-04-24T12:23:53.930" Score="1" ViewCount="31" Body="&lt;p&gt;I am working to use DQN and Policy Gradient reinforcement learning models to solve classic maze escaping problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have been able to train a model, which, after around 100 episodes, quickly explored ONE optimal solution to escape mazes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is easy to see that for many maze designs, the optimal solutions could be multiple, and I would like to take one step further to collect &lt;strong&gt;all optimal and distinguishable solutions&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I tried some searches online and so far, the only material I can find is this &lt;a href=&quot;https://bair.berkeley.edu/blog/2017/10/06/soft-q-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Diverse Skills&lt;/a&gt;. But this seems an obstacle to me. I somewhat believe this seems a classic (?) and an easier problem that should be addressed in the textbook? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone shed light on this matter?&lt;/p&gt;&#xA;" OwnerUserId="25973" LastEditorUserId="2444" LastEditDate="2020-04-25T03:56:01.863" LastActivityDate="2020-04-25T03:56:01.863" Title="How can I design a DQN or policy gradient model to explore and collect all optimal solutions?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;dqn&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20596" PostTypeId="2" ParentId="12508" CreationDate="2020-04-24T13:05:11.017" Score="1" Body="&lt;p&gt;What you are doing when calculating &lt;span class=&quot;math-container&quot;&gt;$d'(x,y)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$d(x,y)$&lt;/span&gt;: calculating the original edge distance from &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h(y)$&lt;/span&gt;: plus the heuristic from &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to the goal&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h(x)$&lt;/span&gt;: minus the heuristic from &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to the goal&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, using this recalculation of the original edge-values (&lt;span class=&quot;math-container&quot;&gt;$1.$&lt;/span&gt;) in Dijkstra's algorithm you are inherently accounting for the heuristic component of A* by incorporating it (&lt;span class=&quot;math-container&quot;&gt;$2.$&lt;/span&gt;) into the value of the edge traversed, and discarding (&lt;span class=&quot;math-container&quot;&gt;$3.$&lt;/span&gt;) the accumulated previous heuristic values of previous nodes in the path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The additional condition &lt;span class=&quot;math-container&quot;&gt;$h(x) ≤ d(x, y) + h(y)$&lt;/span&gt; ensures the new edge-values are strictly positive.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-04-24T14:06:15.103" LastActivityDate="2020-04-24T14:06:15.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20597" PostTypeId="1" AcceptedAnswerId="20672" CreationDate="2020-04-24T14:31:35.170" Score="0" ViewCount="49" Body="&lt;p&gt;I'm a computer engineering student and I'm about to work on my master thesis. My professor gave me a small dataset with brain Computed Axial Tomography records. I would like to use deep learning to help doctors diagnose a certain disease (obviously, I've also got the data for doing supervised learning). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the dataset is small, is &lt;a href=&quot;https://en.wikipedia.org/wiki/Radial_basis_function_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;radial basis function network&lt;/a&gt; a good solution? What do you think?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw, if you have any type of tips in using the RBF network for this kind of project I would be really grateful.&lt;/p&gt;&#xA;" OwnerUserId="36363" LastEditorUserId="2444" LastEditDate="2020-04-27T23:14:24.003" LastActivityDate="2020-04-27T23:14:24.003" Title="Is radial basis function network appropriate for small datasets?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20598" PostTypeId="1" AcceptedAnswerId="20602" CreationDate="2020-04-24T15:20:41.283" Score="1" ViewCount="100" Body="&lt;p&gt;One of the steps in the actor-critic algorithm is &lt;span class=&quot;math-container&quot;&gt;$$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are just the weights. Can you explain to me what mean &lt;span class=&quot;math-container&quot;&gt;$\partial \theta_{\pi}$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole algorithm comes from Maxim Lepan's book &lt;em&gt;Deep Reinforcement Learning Hands-on&lt;/em&gt; page 269.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of the algorithm : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TbZwe.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TbZwe.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="2444" LastEditDate="2020-04-24T20:05:16.533" LastActivityDate="2020-04-24T20:11:18.360" Title="What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;pytorch&gt;&lt;actor-critic-methods&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20599" PostTypeId="1" AcceptedAnswerId="20606" CreationDate="2020-04-24T15:51:40.190" Score="2" ViewCount="2325" Body="&lt;p&gt;While watching MIT's lectures about search, &lt;a href=&quot;https://www.youtube.com/watch?v=j1H3jAAGlEA&amp;amp;list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi&amp;amp;index=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;4. Search: Depth-First, Hill Climbing, Beam&lt;/a&gt;, the professor explains the hill-climbing search in a way that is similar to the best-first search. At around the 35 mins mark, the professor enqueues the paths in a way similar to greedy best-first search in which they are sorted, and the closer nodes expanded first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have read elsewhere that hill climbing is different from the best first search. What's the difference between the two then?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-04-24T19:32:44.287" LastActivityDate="2020-04-25T00:07:04.280" Title="What is the difference between hill-climbing and greedy best-first search algorithms?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;best-first-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20601" PostTypeId="2" ParentId="11405" CreationDate="2020-04-24T16:11:40.003" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;The decoder half is necessary in order to compute the loss function for training the network. Similar to how the 'adversary' is still necessary in a GAN even if you are only interested in the generative component.&lt;/li&gt;&#xA;&lt;li&gt;Autoencoders can learn non-linear embeddings of the data, and hence are more powerful than vanilla PCA.&lt;/li&gt;&#xA;&lt;li&gt;Autoencoders have &lt;a href=&quot;https://en.wikipedia.org/wiki/Autoencoder#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;applications&lt;/a&gt; beyond dimensionality reduction:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generating new data points, or perform interpolation (see VAE's)&lt;/li&gt;&#xA;&lt;li&gt;Create denoising filters (e.g. in image processing)&lt;/li&gt;&#xA;&lt;li&gt;Compress/decompress data&lt;/li&gt;&#xA;&lt;li&gt;Link prediction (e.g. in drug discovery)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-04-25T19:04:52.107" LastActivityDate="2020-04-25T19:04:52.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20602" PostTypeId="2" ParentId="20598" CreationDate="2020-04-24T19:58:11.827" Score="1" Body="&lt;p&gt;In reinforcement learning, you can distinguish algorithms based on the functions they use to ultimately find the policy (which is the goal in RL anyway!).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;algorithms that attempt to find an optimal value function (an example is Q-learning, which attempts to find a state-action value function), then derive the policy from the value function&lt;/li&gt;&#xA;&lt;li&gt;algorithms that directly attempt to find a policy (e.g. REINFORCE and other so-called &quot;policy gradients&quot; algorithms)&lt;/li&gt;&#xA;&lt;li&gt;algorithms that use a value function to guide the search for an optimal policy (i.e. actor-critic methods)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More specifically, in actor-critic methods, you have a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (known as the &quot;actor&quot;) and a value function &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; (known as the &quot;critic&quot;). Hence the name &quot;actor-critic&quot;. The idea is that you will use this critic &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; to &quot;criticize&quot; the policy (or actor) &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, i.e. to guide the search for a good policy. This article &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node66.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;6.6 Actor-Critic Methods&lt;/a&gt; (from Sutton and Barto's book) explains the concept quite well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your specific example, the policy and value function are assumed to be differentiable (otherwise, you wouldn't be able to compute the derivatives anyway!). They are typically neural networks. &lt;span class=&quot;math-container&quot;&gt;$\theta_\pi$&lt;/span&gt; are the parameters of the neural network that represent the policy (i.e. a neural network that receives as input a state and produces as output an action or probability distribution over actions). Similarly, &lt;span class=&quot;math-container&quot;&gt;$\theta_v$&lt;/span&gt; are the parameters of the neural network that represent the critic. Then &lt;span class=&quot;math-container&quot;&gt;$\partial \theta_\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\partial \theta_\pi $&lt;/span&gt; will represent an accumulation of the gradients with respect to the actor and critic,  respectively. (The accumulation is probably over the steps from &lt;span class=&quot;math-container&quot;&gt;$i=t-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$i = t_{\text{start}}$&lt;/span&gt;, but I can't say more because I never implemented actor-critic methods). This should give you an idea, though!&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-24T20:11:18.360" LastActivityDate="2020-04-24T20:11:18.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20603" PostTypeId="2" ParentId="20592" CreationDate="2020-04-24T20:27:37.963" Score="1" Body="&lt;p&gt;This is one of the most difficult and unsolved problems in machine learning and deep learning!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many different ways to estimate the most appropriate hyper-parameters, such as grid search, random search, Bayesian optimization, meta-learning, reinforcement learning, and evolutionary algorithms (e.g. NEAT).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem is that most if not all these approaches are typically not very computationally efficient if your model isn't anything but very small. The number of possible configurations of the hyper-parameters is very large. If you don't have good computational resources (e.g. GPUs and powerful servers), you probably out of luck or require some days to get some insight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In certain cases, it is obvious that certain hyper-parameters are dependent on each other, e.g. in the case of the batch size and the learning rate, because we have some decent understanding of gradient descent, but, in other cases, the situation isn't so nice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, there isn't a very good general rule of thumb or method to solve this issue (i.e. find the dependence of hyper-parameters on each other). Maybe, as our knowledge of our models (especially, neural networks) increases, we'll get some more insights and we'll develop more efficient approaches to understand the dependence of the hyper-parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays, there's automatical machine learning (AutoML), which is a fancy name to denote services that provide hyper-parameter optimization (plus some other stuff).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-25T03:50:06.503" LastActivityDate="2020-04-25T03:50:06.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20604" PostTypeId="1" CreationDate="2020-04-24T22:15:40.713" Score="0" ViewCount="45" Body="&lt;p&gt;Why are all weights of a neural net updated and not only the weights of the first hidden layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error-influence of the prediction by the weights of a neural net is calculated using the chain rule. However, the chain rule tells us how the first variable influences the second variable, and so on. Following that logic, we should only update the weights of the first hidden layer. My thought is, that if we backtrack the influence of the first variable but also change the values of the subsequent weights (of the subsequent hidden layer), there is no need to calculate the influence of the first weights in the first place. Where am I wrong? &lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-04-26T16:35:51.647" LastActivityDate="2020-04-26T16:35:51.647" Title="Why are all weights of a neural net updated and not just the weights of the first layer" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;&lt;math&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20605" PostTypeId="1" CreationDate="2020-04-24T23:09:42.197" Score="0" ViewCount="123" Body="&lt;p&gt;I have a fully connected neural network with the following number of neurons in each layer &lt;code&gt;[4, 20, 20, 20, ..., 1]&lt;/code&gt;. I am using TensorFlow and the 4 real-valued inputs correspond to a particular point in space and time, i.e. (x, y, z, t), and the 1 real-valued output corresponds to the temperature at that point. The loss function is just the mean square error between my predicted temperature and the actual temperature at that point in (x, y, z, t). I have a set of training data points with the following structure for their inputs:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;(x,y,z,t):&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;(0.11,0.12,1.00,0.41)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.34,0.43,1.00,0.92)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.01,0.25,1.00,0.65)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.71,0.32,1.00,0.49)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.31,0.22,1.00,0.01)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.21,0.13,1.00,0.71)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Namely, what you will notice is that the training data all have the same redundant value in &lt;code&gt;z&lt;/code&gt;, but &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt;, and &lt;code&gt;t&lt;/code&gt; are generally not redundant. Yet what I find is my neural network cannot train on this data due to the redundancy. In particular, every time I start training the neural network, it appears to fail and the loss function becomes &lt;code&gt;nan&lt;/code&gt;. But, if I change the structure of the neural network such that the number of neurons in each layer is &lt;code&gt;[3, 20, 20, 20, ..., 1]&lt;/code&gt;, i.e. now data points only correspond to an input of (x, y, t), everything works perfectly and training is all right. But is there any way to overcome this problem? (Note: it occurs whether any of the variables are identical, e.g. either &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt;, or &lt;code&gt;t&lt;/code&gt; could be redundant and cause this error.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: is there any way to still train the neural network while keeping the redundant &lt;code&gt;z&lt;/code&gt; as an input? It just so happens the particular training data set I am considering at the moment has all &lt;code&gt;z&lt;/code&gt; redundant, but in general, I will have data coming from different &lt;code&gt;z&lt;/code&gt; in the future. Therefore, a way to ensure the neural network can robustly handle inputs at the present moment is sought. &lt;/p&gt;&#xA;" OwnerUserId="21895" LastActivityDate="2020-04-25T01:58:31.940" Title="Can neural networks handle redundant inputs?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;loss-functions&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20606" PostTypeId="2" ParentId="20599" CreationDate="2020-04-24T23:11:36.790" Score="3" Body="&lt;p&gt;Let's see their definition first:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Best-first_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Best First Search&lt;/a&gt; (BFS):&#xA;‌ &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Best-first search is a search algorithm that explores a graph by expanding the most promising node chosen according to a specified rule.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;estimating the promise of node n by a &quot;heuristic evaluation function &lt;span class=&quot;math-container&quot;&gt;${\displaystyle f(n)}$&lt;/span&gt; which, in general, may depend on the description of n, the description of the goal, the information gathered by the search up to that point, and most importantly, on any extra knowledge about the problem domain.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_climbing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hill Climbing&lt;/a&gt; (HC):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In numerical analysis, hill climbing is a mathematical optimization technique that belongs to the family of local search. It is an iterative algorithm that starts with an arbitrary solution to a problem, then attempts to find a better solution by making an incremental change to the solution. If the change produces a better solution, another incremental change is made to the new solution, and so on until no further improvements can be found.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Base on the definition, we can find the following differences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The aim of BFS is reaching to a specified goal by using a heuristic function (it might be greedy) vs. HC is a local search algorithm &lt;/li&gt;&#xA;&lt;li&gt;BFS is mostly used in the graph search (in a wide state space) to find a path. vs. HC is using for the optimization task. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-04-25T00:07:04.280" LastActivityDate="2020-04-25T00:07:04.280" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20609" PostTypeId="1" CreationDate="2020-04-25T05:40:50.243" Score="1" ViewCount="25" Body="&lt;p&gt;I have questions regarding on how to implement PBT as described in Algorithm 1 (on page 5) in the paper, &lt;a href=&quot;https://arxiv.org/pdf/1711.09846.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Population Based Training of Neural Networks&lt;/a&gt; to train agents in a MARL (multi-agent reinforcement learning) environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a single agent RL environment, the environments can be distributed &amp;amp; agents trained in parallel &amp;amp; there will be a need to maintain some sort of centralized population of weights &amp;amp; hyperparameters. (Please correct me if I'm wrong.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a MARL context, do I need to also maintain a centralized population for all agents in all environments or do I need to maintain a separate population for agents in each distributed environment? Which is a correct or more effective approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers would be appreciated. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="25299" LastActivityDate="2020-04-25T05:40:50.243" Title="Do I need to maintain a separate population in each distributed environment when implementing PBT in a MARL context?" Tags="&lt;reinforcement-learning&gt;&lt;multi-agent-systems&gt;&lt;distributed-computing&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20613" PostTypeId="1" CreationDate="2020-04-25T07:22:53.797" Score="1" ViewCount="15" Body="&lt;p&gt;Suppose we have two graphs A and B disconnected to each other (let's say 2-hops each), within a larger graph. If the convolutional representation of graph A is known, is it possible to estimate the definitive convolutional representation of graph B based on its similarity to graph A?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, what do you think is the simplest (arithmetically) way to do this, which algorithm can help me to do this? You can assume that precision requirements are not important.&lt;/p&gt;&#xA;" OwnerUserId="36440" LastEditorUserId="2444" LastEditDate="2020-04-26T00:51:54.160" LastActivityDate="2020-04-26T00:51:54.160" Title="How to estimate the convolutional representation of a graph from its similarity to other graph convolutional representation?" Tags="&lt;convolutional-neural-networks&gt;&lt;geometric-deep-learning&gt;&lt;graph-theory&gt;&lt;similarity&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20616" PostTypeId="1" CreationDate="2020-04-25T13:15:09.753" Score="1" ViewCount="29" Body="&lt;p&gt;This is a bot making problem from &lt;a href=&quot;https://www.hackerrank.com/challenges/the-bidding-game/problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;here.&lt;/a&gt;&#xA;I am detailing the problem.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MyHxr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MyHxr.jpg&quot; alt=&quot;Initial configuration of the game&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The picture above shows the initial configuration of the game. P1 represents player1 and P2 represents player2. A scotch bottle is kept(initially) at the position #5 on the number line. Both players start with 100 dollars in hand. Note that players don't move, only the bottle moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rules of the game:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The first player makes a secret bid followed by a secret bid by the second player.&lt;/li&gt;&#xA;&lt;li&gt;The bottle moves one position closer to the winning bidder.&lt;/li&gt;&#xA;&lt;li&gt;In case of drawn bid, the winner is the player, who has the draw advantage.&lt;/li&gt;&#xA;&lt;li&gt;Draw advantage alternates between the two player, that is, the first draw is won by the first player, the second draw if it occurs is won by the second player and so on.&lt;/li&gt;&#xA;&lt;li&gt;The winning bid is deducted from the player's hand, the loser keeps his bid.&lt;/li&gt;&#xA;&lt;li&gt;The bottle moves one position closer to the winning bidder.&lt;/li&gt;&#xA;&lt;li&gt;Each bid must be greater than 0 dollar. In the case when there's no money left, the player has no choice but to bid 0 dollar. Only integral bids are allowed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The player who gets the bottle wins. If no one gets it, the game ends in a draw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both the players,thus,have complete knowledge of the history of biddings of each other, and the location of bottle at the current time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, i know this is an instance of poorman bidding games. I have used some strategies like I am intentionally losing some bids and let the opponent use his money, in a hope that difference of money increases to the point of allowing for the emergence of a winning strategy. Also, i pull stronger the bottle as it goes further. This isn't performing well with other bots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should be the strategy of a bot playing this game?&lt;/p&gt;&#xA;" OwnerUserId="22063" LastEditorUserId="22063" LastEditDate="2020-04-25T13:21:16.177" LastActivityDate="2020-04-25T13:21:16.177" Title="What should be a good playing strategy in this 2-player simultaneous game?" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;game-theory&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20617" PostTypeId="1" CreationDate="2020-04-25T13:18:42.577" Score="0" ViewCount="29" Body="&lt;p&gt;I have a gaussian distributed time series (&lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt;) with some parameters in my experiment. Suppose I want to know the mean &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;. If I define another time series &lt;span class=&quot;math-container&quot;&gt;$Y_t$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$Y_t=X_t-a$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Now say I vary this parameter &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and generate altogether different time series for each &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, say &lt;span class=&quot;math-container&quot;&gt;$Y_t(a)$&lt;/span&gt;. I look at the mean of &lt;span class=&quot;math-container&quot;&gt;$Y_t$&lt;/span&gt; for each &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. The value of a, where I get the mean of &lt;span class=&quot;math-container&quot;&gt;$Y_t$&lt;/span&gt; closest to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, will be my estimate of &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;. Say I will eventually use this learnt value of &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; to generate &lt;span class=&quot;math-container&quot;&gt;$Y_t$&lt;/span&gt; as my final goal. Can this be called ML? I am using some training data of &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt; to learn about its parameter and then using test data of &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt; to generate &lt;span class=&quot;math-container&quot;&gt;$Y_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now why am I working so hard on this simple problem? Well, actually I am not. I am doing something else, which will have lots of parameters in the time series and will be used to generate other time series after similar parameter extraction. That will be too complicated to discuss here. I just wanted to clear my basics using an over-simplified example. I am a new to ML, and help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="36369" LastActivityDate="2020-04-25T13:18:42.577" Title="Does it classify as Machine Learning?" Tags="&lt;classification&gt;&lt;philosophy&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20618" PostTypeId="1" CreationDate="2020-04-25T13:47:44.187" Score="1" ViewCount="37" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In the information theory, the entropy is a measure of uncertainty in&#xA;  some system. Being applied to agent policy, entropy shows how much the&#xA;  agent is uncertain about which action to make. In math notation,&#xA;  entropy of the policy is defined  as : &lt;span class=&quot;math-container&quot;&gt;$$H(\pi) = -\sum \pi(a|s) \log&#xA; \pi(a|s)$$&lt;/span&gt; The value of entropy is always greater than zero and has a&#xA;  single maximum when the policy is uniform. In other words, all actions&#xA;  have the same probability. Entropy becomes minimal when our policy has&#xA;  1 for some action and 0 for all others, which means that the agent is&#xA;  absolutely sure what to do. To prevent our agent from being stuck in&#xA;  the local minimum, we are subtracting the entropy from the loss&#xA;  function, punishing the agent for being too certain about the action&#xA;  to take.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The above excerpt is from Maxim Lapan in the book &lt;em&gt;Deep Reinforcement Learning Hands-on&lt;/em&gt; page 254.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In code, it might look like : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; optimizer.zero_grad()&#xA; logits= PG_network(batch_states_ts)&#xA; log_prob = F.log_softmax(logits, dim=1)&#xA; log_prob_actions = batch_scales_ts * log_prob[range(params[&quot;batch_size&quot;]), batch_actions_ts]&#xA; loss_policy = -log_prob_actions.mean()&#xA;&#xA; prob = F.softmax(logits, dim=1)&#xA; entropy = -(prob * log_prob).sum(dim=1).mean()&#xA; entropy_loss = params[&quot;entropy_beta&quot;] * entropy&#xA; loss = loss_policy - entropy_loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that a disadvantage of using policy gradient is our agent can be stuck at a local minimum. Can you explain &lt;strong&gt;mathematically&lt;/strong&gt; why subtracting the entropy from our policy will  &lt;em&gt;prevent our agent from being stuck in the local minimum&lt;/em&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastActivityDate="2020-04-25T13:47:44.187" Title="Subtracting the entropy from our policy gradient will prevent our agent from being stuck in the local minimum?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20619" PostTypeId="1" AcceptedAnswerId="20620" CreationDate="2020-04-25T14:33:51.990" Score="2" ViewCount="149" Body="&lt;p&gt;I was reading about the temporal difference (TD) learning and I read that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TD handles continuing, non-episodic domains&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming that continuing means non-terminating, what does non-episodic or episodic domain mean?&lt;/p&gt;&#xA;" OwnerUserId="36447" LastEditorUserId="2444" LastEditDate="2020-04-26T00:50:07.333" LastActivityDate="2020-04-26T00:50:07.333" Title="What are episodic and non-episodic domains in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;temporal-difference-methods&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20620" PostTypeId="2" ParentId="20619" CreationDate="2020-04-25T14:51:29.760" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Assuming that continuing means non terminating, what does non-episodic or episodic domain mean ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Non-episodic means the same as continuing. The quote you found is not listing two separate domains, the word &quot;continuing&quot; is slightly redundant. I expect the author put it in there to emphasise the meaning, or to cover two common ways of describing such environments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Episodic domain problems are ones that terminate, or otherwise naturally split into groups of time steps that can be considered separately.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-04-25T14:51:29.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20621" PostTypeId="2" ParentId="18188" CreationDate="2020-04-25T14:58:22.413" Score="0" Body="&lt;p&gt;I have written a tutorial on using OpenAI Spinning Up in a image-based PyBullet + Gym environment &lt;a href=&quot;https://www.etedal.net/2020/04/pybullet-panda_3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to be able to use spinup for an image-based environment I had to fork it &lt;a href=&quot;https://github.com/mahyaret/spinningup&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and add CNN to PPO's core.py&lt;/p&gt;&#xA;" OwnerUserId="36448" LastActivityDate="2020-04-25T14:58:22.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20622" PostTypeId="1" CreationDate="2020-04-25T16:17:27.533" Score="1" ViewCount="153" Body="&lt;p&gt;I am reading Sutton and Barto's material now. I know value iteration, which is an iterative algorithm taking the maximum value of adjacent states, and policy iteration. But what is generalized policy iteration? &lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="36107" LastEditDate="2020-04-25T17:01:00.883" LastActivityDate="2020-04-25T17:10:55.217" Title="What is generalized policy iteration?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20624" PostTypeId="2" ParentId="20622" CreationDate="2020-04-25T17:05:26.253" Score="1" Body="&lt;p&gt;In the standard policy iteration algorithm presented in Sutton and Barto's book, you alternate between a policy evaluation (PE) step and a policy improvement (PI) step (i.e. PE, PI, PE, PI, PE, PI, PE, ...). However, in general, you don't have to follow this alternation &lt;em&gt;strictly&lt;/em&gt; in order to converge (in the limit) to the optimal policy. For example, value iteration (VI) is an example of a &lt;em&gt;truncated&lt;/em&gt; policy iteration that still converges to the optimal policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node46.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;generalized policy iteration&lt;/em&gt; (GPI)&lt;/a&gt; refers to all algorithms based on policy iteration, such as value iteration, that alternate &lt;em&gt;in some order&lt;/em&gt; PI and PE, and that are guaranteed to converge to the optimal policy, provided PE and PI are executed enough times.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-25T17:10:55.217" LastActivityDate="2020-04-25T17:10:55.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20625" PostTypeId="2" ParentId="7413" CreationDate="2020-04-25T17:14:13.640" Score="1" Body="&lt;p&gt;It's a &lt;em&gt;continuing task&lt;/em&gt; in that, after failure, the agent always gets a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; at each time-step &lt;em&gt;ad infinitum&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the book:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we could treat pole-balancing as a continuing task, using discounting.&#xA;  In this case the reward would be -1 on each failure and zero at all&#xA;  other times. The return at each time would then be related to &lt;span class=&quot;math-container&quot;&gt;$-\gamma^K$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is the number of time steps before failure.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(Here I have used &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; as the discount factor). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Said another way, assuming the agent fails in the (K + 1)th step the reward is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; till that step, &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for it, and then &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for eternity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the return: &lt;span class=&quot;math-container&quot;&gt;$$G_t = R_{t+1} + \gamma R_{t+2} + \gamma^2 R_{t+3} + ... + \gamma^K R_{t+K+1} + ... = -\gamma^K$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="36449" LastEditorUserId="2444" LastEditDate="2020-04-25T18:47:53.270" LastActivityDate="2020-04-25T18:47:53.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20626" PostTypeId="2" ParentId="11169" CreationDate="2020-04-25T18:11:41.777" Score="3" Body="&lt;h2&gt;Graph Neural Networks&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;Graph Neural Network&lt;/em&gt;, in its broadest sense, refers to any Neural Network designed to take graph structured data as its input:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To cover a broader range of methods, this survey considers GNNs as all deep learning approaches for graph data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.00596&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comprehensive Survey on Graph Neural Networks&lt;/a&gt;, Wu et al (2019)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However the original paper to propose the term specifically referred to recursive neural networks (RNN) adapted to take graph-structured data as their input:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This paper presents a new neural model, called graph neural network (GNN), capable of directly processing graphs. GNNs extends recursive neural networks and can be applied on most of the practically useful kinds of graphs, including directed, undirected, labelled and cyclic graphs. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/1555942&quot; rel=&quot;nofollow noreferrer&quot;&gt;A new model for learning in graph domains&lt;/a&gt; (2005)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Subtypes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Note, Wu et al propose a taxonomy dividing GNN's into four subgroups:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Recurrent graph neural networks (RecGNN)&lt;/li&gt;&#xA;&lt;li&gt;Convolutional graph neural networks (ConvGNN)&lt;/li&gt;&#xA;&lt;li&gt;Graph autoencoders (GAE)&lt;/li&gt;&#xA;&lt;li&gt;Spatial-temporal graph neural networks (STGNN)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ConvGNN's can themselves be classified by whether they use Spectral methods or Spatial methods, and GAE's by whether they are designed for Network embedding or Graph generation.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-04-26T08:33:40.163" LastActivityDate="2020-04-26T08:33:40.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20627" PostTypeId="2" ParentId="12712" CreationDate="2020-04-25T18:19:02.103" Score="2" Body="&lt;p&gt;Yes, there are numerous, coming under the umbrella term &lt;a href=&quot;https://ai.stackexchange.com/questions/11169/what-is-a-graph-neural-network/20626#20626&quot;&gt;&lt;strong&gt;Graph Neural Networks&lt;/strong&gt;&lt;/a&gt; (GNN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common input structures accepted by these techniques are the adjacency matrix of the graph (optionally accompanied by its node feature matrix and/or edge feature matrix, if the graph has such information).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.00596&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comprehensive Survey on Graph Neural Networks&lt;/a&gt;, Wu et al (2019) divides GNN's into four subgroups:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Recurrent graph neural networks (RecGNN)&lt;/li&gt;&#xA;&lt;li&gt;Convolutional graph neural networks (ConvGNN)&lt;/li&gt;&#xA;&lt;li&gt;Graph autoencoders (GAE)&lt;/li&gt;&#xA;&lt;li&gt;Spatial-temporal graph neural networks (STGNN)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ConvGNN's can themselves be classified by whether they use Spectral methods or Spatial methods, and GAE's by whether they are designed for Network embedding or Graph generation.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-05-26T19:44:30.120" LastActivityDate="2020-05-26T19:44:30.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20628" PostTypeId="2" ParentId="11226" CreationDate="2020-04-25T18:43:14.533" Score="7" Body="&lt;p&gt;I presume this question was prompted by the paper &lt;a href=&quot;https://arxiv.org/abs/1611.08097&quot; rel=&quot;noreferrer&quot;&gt;Geometric deep learning:&#xA;going beyond Euclidean data&lt;/a&gt; (2017). If we look at its abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Many scientific fields study &lt;strong&gt;data with an underlying structure that is a non-Euclidean space.&lt;/strong&gt; Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric&#xA;  data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use &lt;strong&gt;deep neural networks&lt;/strong&gt;, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However,&#xA;  these tools &lt;strong&gt;have been most successful on data with an underlying Euclidean or grid-like structure,&lt;/strong&gt; and in cases where the invariances of these structures are built into networks used to model them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We see that the authors use the term &quot;non-Euclidean data&quot; to refer to data whose underlying structure is non-Euclidean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Euclidean spaces are prototypically defined by &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; (for some dimension &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;), 'Euclidean data' is data which is sensibly modelled as being plotted in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional linear space, for example image files (where the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; coordinates refer to the location of each pixel, and the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; coordinate refers to its colour/intensity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However some data does not map neatly into &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;, for example, a social network modelled by a graph. You can of course &lt;a href=&quot;https://math.stackexchange.com/questions/584171/show-that-every-graph-can-be-embedded-in-mathbbr3&quot;&gt;embed the physical shape of a graph in 3-d space&lt;/a&gt;, but you will lose information such as the quality of edges, or the values associated with nodes, or the directionality of edges, and there isn't an obvious sensible way of mapping these attributes to higher dimensional Euclidean space. And depending on the specific embedding, you may introduce spurious correlations (e.g. two unconnected nodes appearing closer to each other in the embedding than to nodes they are connected to).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods such as &lt;a href=&quot;https://ai.stackexchange.com/questions/11169/what-is-a-graph-neural-network/20626#20626&quot;&gt;Graph Neural Networks&lt;/a&gt; seek to adapt existing Machine Learning technologies to directly process non-Euclidean structured data as input, so that this (possibly useful) information is not lost in transforming the data into a Euclidean input as required by existing techniques.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-04-25T18:59:57.083" LastActivityDate="2020-04-25T18:59:57.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20629" PostTypeId="2" ParentId="15688" CreationDate="2020-04-25T19:20:02.710" Score="1" Body="&lt;p&gt;Low order/low level information refers to the most granular level of information. This is the most informative in terms of volume of information, but it can often be difficult to conceptualise for humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;High order/high level information refers to abstractions of the low level information to more intuitive but less easy to describe technically concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example would be images of faces. The low level information might be the raw &lt;span class=&quot;math-container&quot;&gt;$x, y, z$&lt;/span&gt; values of the pixels: their position and colour value. Some high level information might be the direction the face in the images is facing, from what direction the lighting in the image is coming etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper cited, the example they use is low level information (node and edge values); high level information (&lt;a href=&quot;https://en.wikipedia.org/wiki/Network_motif&quot; rel=&quot;nofollow noreferrer&quot;&gt;motifs&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="23503" LastActivityDate="2020-04-25T19:20:02.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20630" PostTypeId="2" ParentId="16805" CreationDate="2020-04-25T20:00:30.033" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.00596&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comprehensive Survey on Graph Neural Networks&lt;/a&gt; (2019) presents a list of ConvGNN's. All of the following accept weighted graphs, and three accept those with edge weights as well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d7foa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d7foa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And below is a series of open source implementations of many of the above:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QSxzd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QSxzd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-04-26T10:42:30.907" LastActivityDate="2020-04-26T10:42:30.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20631" PostTypeId="1" AcceptedAnswerId="20653" CreationDate="2020-04-25T20:06:16.880" Score="1" ViewCount="124" Body="&lt;p&gt;I was going through university slides and this particular slide is trying to prove that in a Monte Carlo Policy Iteration algorithm using an epsilon-greedy policy, the state Values (V-Values) are monotonically improving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MIV5O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MIV5O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about the first line of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ctrJD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ctrJD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't this actually the formula for the expected value of Q? It is calculating a probability of occurrence following the policy times actual Q values, then doing the summation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is the case, could you help me understand the relationship between the expected value of Q and the expected value of V ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if above is true, in a real world scenario, depending on how many episodes we sample and on stochasticity, does it mean that the V values of the new policy could be worse than the V values of the old policy ?&lt;/p&gt;&#xA;" OwnerUserId="36447" LastEditorUserId="2444" LastEditDate="2020-04-25T23:16:05.057" LastActivityDate="2020-04-26T15:36:03.217" Title="Monte Carlo epsilon-greedy Policy Iteration: monotonic improvement for all cases or for the expected value?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;monte-carlo-methods&gt;&lt;policy-iteration&gt;&lt;policy-improvement&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20632" PostTypeId="1" AcceptedAnswerId="20643" CreationDate="2020-04-25T21:10:03.393" Score="1" ViewCount="114" Body="&lt;p&gt;Desperate trying to understand something for couple of weeks. All those questions are actually one big question.Please help me. Time-codes and screens in my question refer to this great(IMHO) 3d explanation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=UojVVG4PAG0&amp;amp;list=PLVZqlMpoM6kaJX_2lLKjEhWI0NlqHfqzp&amp;amp;index=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=UojVVG4PAG0&amp;amp;list=PLVZqlMpoM6kaJX_2lLKjEhWI0NlqHfqzp&amp;amp;index=2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;....&#xA;Here is the case: Say I have 2 inputs (lets call them X1 and X2) into my ANN. Say X1= persons age and X2=years of education.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;strong&gt;First question:&lt;/strong&gt; do I plug those numbers as is or normalize them 0-1 as a &quot;preprocessing&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) As I have 2 weights and 1 bias, actually I am going to plug my inputs to X1*W1+X2*W2=output formula.  This is 2d plane in a 3d space if I am not mistaken(time-code 5:31):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/oV1Nj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oV1Nj.png&quot; alt=&quot;2d plane in 3d space&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus when I plug in my variables, like in regression I will get on a Z axis my output.  &lt;strong&gt;So the second question is: am I right up to here?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-----------------From here come  real important couple of  questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) My output (before I plug it into the activation function) is just a simple number, IT IS NOT A PLANE and NOT A SURFACE, but a simple scalar, without any sigh on it coming from 2d surface in a 3d space(though it does come from there). Thus, when I plug this number (which was Z value in a previous step) into the activation function (say sigmoid) my number enters there in to the X axis,  and we get as an output some Y value. As I understand  this operation was &lt;strong&gt;totally 2d operation&lt;/strong&gt;, is was 2d sigmoid and not some kind of 3dsigmoidal surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So here is the question:&lt;/strong&gt; If I am right, why do we see in this movie (and couple of other places) such an explanation?&#xA;(time-code 12:55):&lt;a href=&quot;https://i.stack.imgur.com/9Xomw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Xomw.png&quot; alt=&quot; 3d-sigmoidal surface &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4)Now lets say that I was right in the previous step and as an output from the activation function I do get a simple number not a 2d surface and not a 3d one. I just have some number like I had in the very beginning of the ANN as an input (age, education etc). If i want to add another layer of neurons, this very number enters there  &lt;strong&gt;as is&lt;/strong&gt; not telling any one the &quot;secret&quot; that it was created by some kind of sigmoid. In this next layer this number is about to take similar transformations as it happened to age and education in a previous layer, it is going to be Xn in just the same scenario: sigmoid(Xn&lt;em&gt;Wn+Xm&lt;/em&gt;Wm=output) and in the end we will get once again just a number. If I am right, why in the movie they say (time-code 14:50 ) that when we add together two activation functions we get something unlinear. They show result of such &quot;addition&quot; first as 2d (time-code 14:50 and 14:58).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FR2Hh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FR2Hh.png&quot; alt=&quot;addition of 2 activation functions &quot;&gt;&lt;/a&gt; &#xA;&lt;strong&gt;So, Here comes my question:&lt;/strong&gt;  how come that they &quot;add&quot; two activation functions, if to the second activation function reaches just a simple number as said above he is not telling any one the &quot;secret&quot; that it was created by some kind of sigmoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) And then again, they show this addition of 3d surfaces (time-code 19:39 )&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AUtkg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AUtkg.png&quot; alt=&quot;19:39 &quot;&gt;&lt;/a&gt; How it is possible? I mean again there should not happen any addition of surfaces, because no surface passes to next step but a number. What do I miss?&lt;/p&gt;&#xA;" OwnerUserId="36453" LastEditorUserId="31416" LastEditDate="2020-04-26T10:15:33.940" LastActivityDate="2020-04-26T18:44:29.710" Title="How are non-linear surfaces formed in the training of a neural network?" Tags="&lt;neural-networks&gt;&lt;ai-basics&gt;&lt;activation-function&gt;&lt;hidden-layers&gt;&lt;weights&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20633" PostTypeId="2" ParentId="5546" CreationDate="2020-04-25T22:13:50.023" Score="2" Body="&lt;p&gt;The everyday definition of &lt;em&gt;convolution&lt;/em&gt; comes from the Latin &lt;a href=&quot;https://en.wiktionary.org/wiki/convolution#English&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;convolutus&lt;/em&gt;&lt;/a&gt; meaning 'to roll together'. Hence the meaning twisted or complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematical definition comes from the same root, with the interpretation of taking a &quot;rolling average&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence in Machine Learning, a &lt;em&gt;convolution&lt;/em&gt; is a sliding window across an input creating one averaged output for each stride the window takes. I.e. the values covered by the window are &lt;em&gt;convoluted&lt;/em&gt; to create one convoluted output. This is best demonstrated with an a diagram:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AV512.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AV512.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convolution can be any function of the input, but some common ones are the max value, or the mean value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A convolutional neural network (CNN) is a neural network where one or more of the layers employs a convolution as the function applied to the output of the previous layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the window is greater than size 1x1, the output will be necessarily smaller than the input (unless the input is artificially 'padded' with zeros), and hence CNN's often have a distinctive 'funnel' shape:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p2gz1m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p2gz1m.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23503" LastActivityDate="2020-04-25T22:13:50.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20634" PostTypeId="5" CreationDate="2020-04-25T23:17:45.110" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-25T23:17:45.110" LastActivityDate="2020-04-25T23:17:45.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20635" PostTypeId="4" CreationDate="2020-04-25T23:17:45.110" Score="0" Body="For questions related to the policy improvement (e.g. the PI step used in policy iteration)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-25T23:17:45.110" LastActivityDate="2020-04-25T23:17:45.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20637" PostTypeId="2" ParentId="20632" CreationDate="2020-04-26T01:02:53.893" Score="1" Body="&lt;p&gt;The output of any node is simply a scalar number. For a given input you get a specific scalar output. What is being shown is the surfaces that get generated as you VARY x1 and x2 over their input range.&#xA;To answer your first question it is always best to scale your inputs.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-26T01:02:53.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20638" PostTypeId="1" CreationDate="2020-04-26T01:16:07.933" Score="3" ViewCount="64" Body="&lt;p&gt;I was reading the paper &lt;a href=&quot;http://arxiv.org/abs/1801.07365&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Prune Filters in Convolutional Neural Networks&lt;/a&gt;, which is about pruning the CNN filters using reinforcement learning (policy gradient). The paper says that the input for the pruning agent (the agent is a convolutional neural network) is a &lt;strong&gt;2D&lt;/strong&gt; array of shape &lt;code&gt;(N_l, M_l)&lt;/code&gt;, where &lt;code&gt;N_l&lt;/code&gt; is the number of filters and &lt;code&gt;M_l = m x h x w&lt;/code&gt; (&lt;code&gt;m&lt;/code&gt;, &lt;code&gt;l&lt;/code&gt; and &lt;code&gt;h&lt;/code&gt; are filter dimensions), and the output is an array of actions (each element is &lt;strong&gt;0&lt;/strong&gt; (unnecessary filter) or &lt;strong&gt;1&lt;/strong&gt; (necessary)) and says in order to approximate gradients we have to sample the output &lt;code&gt;M&lt;/code&gt; times (using the REINFORCE algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I have one input, how can I sample the output distribution multiple times (without updating the CNN parameters)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm missing something, please, tell me where I'm wrong&lt;/p&gt;&#xA;" OwnerUserId="36461" LastEditorUserId="2444" LastEditDate="2020-04-26T21:10:03.137" LastActivityDate="2021-05-22T14:05:00.510" Title="How can I sample the output distribution multiple times when pruning the filters with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20639" PostTypeId="1" CreationDate="2020-04-26T02:31:21.613" Score="1" ViewCount="56" Body="&lt;p&gt;What are the differences and similarities between PAC learning and classic parameter estimation theorems (e.g. consistency results when estimating parameters, e.g. with MLE)?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-04-26T18:55:12.040" LastActivityDate="2020-04-26T18:55:12.040" Title="What is the relationship between PAC learning and classic parameter estimation theorems?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;&lt;statistics&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20640" PostTypeId="2" ParentId="20570" CreationDate="2020-04-26T03:42:19.953" Score="1" Body="&lt;p&gt;(1) You might want look into RND (Random network distillation) which allows usage of a curiosity-based exploration bonus for the agent as an intrinsic reward. You can use the intrinsic reward to complement the sparse extrinsic reward return by the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general idea is to have a randomly initialized fixed target network which encodes the next state &amp;amp; a predictor network is trained to predict the output of the target network. The prediction error is used to &quot;quantify the novelty of new experience&quot;. Stronger novelty will be a good indication for the agent that it maybe worthwhile to explore more. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors of this (A) &lt;a href=&quot;https://arxiv.org/pdf/1810.12894.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; were able to achieve SOTA performance in Montezuma's Revenge, which is notorious for it's sparse reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In appendix A.1, It is mentioned that: &quot;An exploration bonus can be used with any RL algorithm by modifying the rewards used to train the&#xA;model (i.e., rt = it + et).&quot; It is also mentioned that the authors combined this exploration bonus with PPO (which also works in continuous action space). In A.2, a pseudo code is provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend this (B) &lt;a href=&quot;https://arxiv.org/pdf/1908.02388.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; (see section 3) if you're interested in exploring the available Bonus-Based Exploration Methods out there which may help in tackling hard exploration games with sparse rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to high stochasticity &amp;amp; variance, I found an interesting remark (on page 3, under Figure 2) in this (C) &lt;a href=&quot;https://arxiv.org/pdf/1708.04133.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;our investigation of DDPG on different network configurations shows that for the Hopper environment, DDPG is quite unstable no matter the network architecture. This can be attributed partially to the high variance of DDPG itself, but also to the increased stochasticity of the Hopper&#xA;task.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The remark was made in the context where the authors were trying to &quot;tune DDPG to reproduce results from other works even when using their reported hyper-parameter settings&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/bench.html#hopper-pytorch-versions&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a different benchmark on how DDPG fair against other algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) From the information provided, I can't conclusively provide you a quantitative assessment on DDPG performance for your specific problem. However, I would recommend the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) I will encourage you to try different RL algorithms when face with a difficult problem so that you can benchmark &amp;amp; find out which is more suitable. Also in (A), the authors mentioned, &quot;PPO is a policy gradient method that we have found to require little tuning for good performance.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) Try different sets of hyperparameters. There are many ways to tune them systematically but discussion about this will be out of scope for this question.&lt;/p&gt;&#xA;" OwnerUserId="25299" LastEditorUserId="25299" LastEditDate="2020-04-26T06:42:18.690" LastActivityDate="2020-04-26T06:42:18.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20642" PostTypeId="2" ParentId="16631" CreationDate="2020-04-26T07:12:33.110" Score="1" Body="&lt;p&gt;[Answering my own question after 5 months of studying VAE models]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of the MMD-VAE or InfoVAE is not exactly to emphasise on the visual quality of generated samples. It is to preserve greater amount of information through the encoding process. The MMD formulation stems from introducing a mutual coefficient factor into the Evidence Lower BOund (ELBO) loss of VAEs. Refer to the &lt;a href=&quot;https://arxiv.org/pdf/1706.02262.pdf)%F0%9F%8E%99%EF%B8%8FPoster&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; appendices for full derivation. This formulation improves information content in latent space and provides for more accurate approximation of the true posterior - these results have also been empirically proven in the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the InfoVAE uses pixel-wise or element-wise reconstruction loss. An element-wise reconstruction loss is likely to lead to some extent of blurriness inrespective of the prior loss term. On Github, several developers have implemented the InfoVAE model and shown their results. Here is a &lt;a href=&quot;https://github.com/pren1/keras-MMD-Variational-Autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to one such implementation whose results I could personally verify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my own experimentations, I can say that even though InfoVAE may give better reconstructions for some data, there is still considerable blurriness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perceptual similarity metrics may be learned or computed as a static function of the input image. With a &lt;a href=&quot;https://arxiv.org/pdf/1801.03924.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;learned perceptual loss&lt;/a&gt;, VAEs can produce much sharper images. &lt;a href=&quot;https://arxiv.org/pdf/1611.05013.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PixelVAE&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1512.09300&quot; rel=&quot;nofollow noreferrer&quot;&gt;VAEGAN&lt;/a&gt; are well-known models with such implementations. For a static function of the image itself, reconstruction quality will depend on the nature of that function and such a model may not be very useful for all kinds of datasets. Using measures like SSIM, FSIM, we may still end up getting blurred images.&lt;/p&gt;&#xA;" OwnerUserId="31416" LastEditorUserId="31416" LastEditDate="2020-04-26T11:17:53.820" LastActivityDate="2020-04-26T11:17:53.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20643" PostTypeId="2" ParentId="20632" CreationDate="2020-04-26T08:15:05.130" Score="3" Body="&lt;p&gt;Hi and welcome to the community. It's important to understand these basic concepts very clearly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to first understand the basic unit of a neural network, a single node/neuron/perceptron. Let us forget all about Neural Networks for a bit, and talk about something far simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Linear Regression&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p0Ot9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p0Ot9.png&quot; alt=&quot;A Linear Regression&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above figure, we clearly have one independent variable on the x-axis, and one dependent variable on the y-axis. The red line has an intercept of zero, and let's say a slope of 0.5. Therefore,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ y = 0.5x + 0 $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, right here, is a single perceptron. You take a value of x, lets say 8, pass it through the the node, and get a value as output, 4. Simple!&#xA;But what is the model in this case? Is it the output? No. Its the set [0.5, 0] that represents the red line above. The outputs are simply points on that line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A neural network model is always a set of values - a matrix or a tensor, if you will.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plots in your question, do not represent outputs. &lt;strong&gt;They represent the models.&lt;/strong&gt; But now that you've possibly understood what a linear model with one indpendent variable looks like, I hope you can appreciate that having 2 independent variables will give us a plane in 3-D space. This is called multiple regression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dMONi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dMONi.png&quot; alt=&quot;multiple regression&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This forms the first layer of a neural network with linear activation functions. Assuming &lt;span class=&quot;math-container&quot;&gt;$ x_{i} $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ x_{j} $&lt;/span&gt; as the two independent variables, the first layer computes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ y_{1} = w_{1}x_{i} + w_{2}x_{j} + b_{1} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that while &lt;span class=&quot;math-container&quot;&gt;$ y_{1} $&lt;/span&gt; is the output of the first layer, the set &lt;span class=&quot;math-container&quot;&gt;$ [w_{1}, w_{2}, b_{1}] $&lt;/span&gt; is the model of the first layer and can be plotted as a plane in 3D space.&#xA;The second layer, again a linear layer, computes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ y_{2} = w_{3}y_{1} + b_{2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substitute &lt;span class=&quot;math-container&quot;&gt;$ y_{1} $&lt;/span&gt; in above and what do you get? Another linear model!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ y_{2} = w_{3}(w_{1}x_{i} + w_{2}x_{j} + b_{1}) + b_{2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding layers to a neural network is only compounding of functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Compounding linear functions on linear functions result in linear functions.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, then, what was the point of adding a layer? Seems useless, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, adding linear layers to a neural network is absolutely useless. But what happens if the activation functions of each perceptron, each layer was not linear? For example the sigmoid or the most widely used today, ReLU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Compounding non-linear functions on non-linear functions can increase non-linearity.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ReLU looks like this&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ y = max(0, x) $$&lt;/span&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3oIXS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3oIXS.png&quot; alt=&quot;relu&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is definitely non-linear but not as non-linear as let's say the sine wave. But can we approximate the sine wave by somehow &quot;compounding&quot; multiple, say &lt;span class=&quot;math-container&quot;&gt;$ N $&lt;/span&gt; ReLUs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \sin(x) \approx a + \sum_{N}b*max(0, c + dx)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here the variables &lt;span class=&quot;math-container&quot;&gt;$ a, b,c, d $&lt;/span&gt; are the trainable &quot;weights&quot; in neural network terminology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qR5eG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qR5eG.png&quot; alt=&quot;sinx_as_relu&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you remember the structure of the perceptron, the first operation is often denoted as a summation over all the inputs. This is how non-linearity is approximated in Neural Networks. Now one may ask: &lt;em&gt;So, summing over non-linear functions can approximate any function, right? So a single hidden layer between input layer and output layer (one that sums over all the outputs of the hidden layer units) should be enough? Why do we often see neural network architectures with so many hidden layers?&lt;/em&gt; This is one of the most important yet often overlooked aspect of neural networks and deep-learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To quote, Dr. Ian J Goodfellow, one of the brightest minds in AI, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A feedforward network with a single (hidden) layer is sufficient to represent any function, but the layer may be infeasibly large and may fail to learn and generalize correctly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, what is the ideal number of hidden layers? There's no magic number! ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more mathematical rigor on how neural networks approximate non-linear functions, one should learn about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Approximation Theorem&lt;/a&gt;. Beginners should check &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why should we care for increased non-linearity? For that I'd direct you to &lt;a href=&quot;https://stats.stackexchange.com/questions/275358/why-is-increasing-the-non-linearity-of-neural-networks-desired&quot;&gt;this.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that all of the above discussion is with respect to regression. For classification, the non-linear surface learned is regarded as a decision boundary and points above and below the surface are classified into different classes. However, an alternative, and arguably better, way to look at this is that given a dataset that is not linearly seperable, a neural network first transforms the input dataset into a linearly seperable form and then uses a linear decision boundary on it. For more on this, definitely check out Christopher Olah's amazing &lt;a href=&quot;http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, yes all independent variables must be normalized before training a neural network. This is to equalize the scale of different variables. More info &lt;a href=&quot;https://medium.com/@urvashilluniya/why-data-normalization-is-necessary-for-machine-learning-models-681b65a05029&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="31416" LastEditorUserId="31416" LastEditDate="2020-04-26T18:44:29.710" LastActivityDate="2020-04-26T18:44:29.710" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20644" PostTypeId="2" ParentId="17791" CreationDate="2020-04-26T08:58:22.240" Score="1" Body="&lt;p&gt;The authors of your cited paper use the term &lt;em&gt;graph-based semi-supervised learning (G-SSL)&lt;/em&gt; to refer to semi-supervised learning techniques which take graph structured data as their input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given their main example, the MNIST dataset, is not graph structured, they detail a method for converting the raw Euclidean data &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; into said form (represented by its adjacency matrix &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;), and then compute the Laplacian &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; of this graph:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We consider the graph-based semi-supervised learning (G-SSL) problem. The input include labeled data &lt;span class=&quot;math-container&quot;&gt;$X_{l} ∈ \mathbb{R}^{n_{l}×d}$&lt;/span&gt; and unlabeled data &lt;span class=&quot;math-container&quot;&gt;$X_{u} \in \mathbb{R}^{n_{u}×d}$&lt;/span&gt;, we define the whole features &lt;span class=&quot;math-container&quot;&gt;$X = [X_{l}; X_{u}]$&lt;/span&gt;. Denoting the labels of &lt;span class=&quot;math-container&quot;&gt;$X_{l}$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$y_{l}$&lt;/span&gt;, our goal is to predict the labels of test data &lt;span class=&quot;math-container&quot;&gt;$y_{u}$&lt;/span&gt;. The learner applies algorithm &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to predict &lt;span class=&quot;math-container&quot;&gt;$y_{u}$&lt;/span&gt; from available data &lt;span class=&quot;math-container&quot;&gt;$\{X_{l}, y_{l}, X_{u}\}$&lt;/span&gt;. Here we restrict &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to label propagation method, where we first generate a graph with adjacency matrix &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; from Gaussian kernel:&#xA;  &lt;span class=&quot;math-container&quot;&gt;$S_{ij} = \exp(−γ\lVert x_i − x_j\rVert ^{2})$&lt;/span&gt;, where the subscripts &lt;span class=&quot;math-container&quot;&gt;$x_{i(j)}$&lt;/span&gt; represents the &lt;span class=&quot;math-container&quot;&gt;$i(j)$&lt;/span&gt;-th row of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. Then the graph Laplacian is calculated by &lt;span class=&quot;math-container&quot;&gt;$L = D − S$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$D = \text{diag}\{\sum_{k=1}^{n} S_{ik}\}$&lt;/span&gt; is the degree matrix.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1910.14147v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Unified Framework for Data Poisoning Attack to&#xA;Graph-based Semi-supervised Learning&lt;/a&gt; (2019)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is consistent with the terminology as used in other literature:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Semi-supervised learning for node-level classification.&lt;/strong&gt;&#xA;  Given a single network with partial nodes being labeled and others remaining unlabeled, ConvGNNs can learn a robust model that effectively identifies the class labels for the unlabeled nodes [22]. To this end, an end-to-end framework can be built by stacking a couple of graph convolutional layers followed by a softmax layer for multi-class classification.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1901.00596.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comprehensive Survey on Graph Neural Networks&lt;/a&gt; (2019)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="2444" LastEditDate="2020-04-28T01:23:56.603" LastActivityDate="2020-04-28T01:23:56.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20645" PostTypeId="1" CreationDate="2020-04-26T09:37:48.353" Score="4" ViewCount="69" Body="&lt;p&gt;I was reading the gradient temporal difference learning version 2(GTD2) from rich Sutton's book page-246. At some point, he expressed the whole expectation using a single sample from the environment. But how a single sample can represent the whole expectation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I marked this point in this image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/idsbt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/idsbt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-04-27T09:42:11.983" LastActivityDate="2020-06-07T23:07:35.773" Title="How can a single sample represent the expectation in gradient temporal difference learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;math&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20646" PostTypeId="2" ParentId="11285" CreationDate="2020-04-26T09:42:26.730" Score="10" Body="&lt;h2&gt;Embedding vs Latent Space&lt;/h2&gt;&#xA;&lt;p&gt;Due to Machine Learning's recent and rapid renaissance, and the fact that it draws from many distinct areas of mathematics, statistics, and computer science, it often has a number of different terms for the same or similar concepts.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Latent space&amp;quot; and &amp;quot;embedding&amp;quot; both refer to an (often lower-dimensional) representation of high-dimensional data:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Latent space&lt;/em&gt; refers specifically to the space from which the low-dimensional representation is drawn.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Embedding&lt;/em&gt; refers to the way the low-dimensional data is mapped to (&amp;quot;embedded in&amp;quot;) the original higher dimensional space.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For example, in this &amp;quot;Swiss roll&amp;quot; data, the 3d data on the left is sensibly modelled as a 2d manifold 'embedded' in 3d space. The function mapping the 'latent' 2d data to its 3d representation is the &lt;em&gt;embedding&lt;/em&gt;, and the underlying 2d space itself is the &lt;em&gt;latent space&lt;/em&gt; (or &lt;em&gt;embedded space&lt;/em&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ID6dP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ID6dP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Synonyms&lt;/h2&gt;&#xA;&lt;p&gt;Depending on the specific impression you wish to give, &amp;quot;embedding&amp;quot; often goes by different terms:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Term&lt;/th&gt;&#xA;&lt;th&gt;Context&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;dimensionality reduction&lt;/td&gt;&#xA;&lt;td&gt;combating the &amp;quot;curse of dimensionality&amp;quot;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;feature extraction&lt;br&gt; feature projection&lt;br&gt; &lt;a href=&quot;https://ai.stackexchange.com/questions/6556/what-is-feature-embedding-in-the-context-of-convolutional-neural-networks/20697&quot;&gt;feature embedding&lt;/a&gt;&lt;br&gt; feature learning&lt;br&gt; representation learning&lt;/td&gt;&#xA;&lt;td&gt;extracting 'meaningful' features from raw data&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;embedding&lt;br&gt; manifold learning&lt;br&gt; latent feature representation&lt;/td&gt;&#xA;&lt;td&gt;understanding the underlying topology of the data&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;However this is not a hard-and-fast rule, and they are often completely interchangeable.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-12-31T17:43:22.203" LastActivityDate="2020-12-31T17:43:22.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20649" PostTypeId="1" CreationDate="2020-04-26T11:03:37.680" Score="1" ViewCount="30" Body="&lt;p&gt;My goal is to write a program that automatically selects a routing out of multiple proposed options. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data consists out of the multiple proposed options with each the attributes time, costs and if there is a transhipment and also which of the options was selected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Example of data:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TqVTdl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TqVTdl.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea at the moment is that I have to apply so type of inference to learn which attribute (time, costs, transhipment) has the highest impact on how to choose the best option. But I don't know exactly where to start with this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a &quot;best&quot; ML algorithm for this? Or how should I approach this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dataset currently consists out of 1000 samples in case if this is important. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your responses.&lt;/p&gt;&#xA;" OwnerUserId="36473" LastActivityDate="2020-04-26T11:03:37.680" Title="Algorithm which learns to select from proposed options" Tags="&lt;algorithm&gt;&lt;automation&gt;&lt;inference&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20651" PostTypeId="1" CreationDate="2020-04-26T11:25:18.887" Score="1" ViewCount="94" Body="&lt;p&gt;Why &lt;code&gt;td_loss&lt;/code&gt; is calculated from (&lt;code&gt;td_targets&lt;/code&gt; against &lt;code&gt;q_values&lt;/code&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why I am lost is because:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;q_values&lt;/code&gt; is just the probability of action. It does not have a reward and discount.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;td_targets&lt;/code&gt; does have rewards + discounts * next_q_values. Somemore next_q_values is next state.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How both &lt;code&gt;td_targets&lt;/code&gt; and &lt;code&gt;q_values&lt;/code&gt; can minus (or Huber or MSE) to get lost work?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;td_error = valid_mask * (td_targets - q_values)&#xA;td_loss = valid_mask * td_errors_loss_fn(td_targets, q_values)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorflow/agents/blob/v0.4.0/tf_agents/agents/dqn/dqn_agent.py#L459&quot; rel=&quot;nofollow noreferrer&quot;&gt;td_loss = valid_mask * td_errors_loss_fn(td_targets, q_values)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36475" LastEditorUserId="2444" LastEditDate="2020-05-10T13:58:58.803" LastActivityDate="2021-02-04T14:03:40.560" Title="How does the DQN loss from td_targets against q_values make sense?" Tags="&lt;loss-functions&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20652" PostTypeId="2" ParentId="20651" CreationDate="2020-04-26T12:15:01.723" Score="0" Body="&lt;p&gt;First of all, DQN is off-policy learning. That means, you are following the behavior policy(epsilon greedy policy) but still learning about the optimal policy or target policy(greedy policy). Td_target in DQN is the estimation of our current state's optimal action-value function independent of the policy we are following (since we are picking next state's action-value from target policy) and q_values(as you referred) is what you get following the behavior policy. While using this kind of update, you are improving both the behavior policy and the target policy.  &lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-04-26T12:27:37.707" LastActivityDate="2020-04-26T12:27:37.707" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20653" PostTypeId="2" ParentId="20631" CreationDate="2020-04-26T15:00:44.207" Score="0" Body="&lt;p&gt;I think this equation answer your question:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ q_{\pi^{i}}(s,\pi^{i+1}(s)) = \mathbf{E}[q_{\pi^{i}}(s,\pi^{i+1}(s))] = \sum_{a \in A}\pi^{i+1}(a|s)q_{\pi^{i}}(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;value of the Q while taking action from policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i+1}$&lt;/span&gt; and thereafter following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i}$&lt;/span&gt; is equal to the expected q value while taking action from policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i+1}$&lt;/span&gt; and thereafter following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i}$&lt;/span&gt;. And for the second part of your question the answer is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ V_{\pi^{i}}(s) = q_{\pi^{i}}(s,\pi^{i}(s))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;state Value function following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i}$&lt;/span&gt; is the same as action-value function while taking action from policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i}$&lt;/span&gt; and thereafter following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{i}$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-04-26T15:36:03.217" LastActivityDate="2020-04-26T15:36:03.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20654" PostTypeId="1" CreationDate="2020-04-26T15:26:42.327" Score="1" ViewCount="50" Body="&lt;p&gt;Many neural net architectures for computer vision tasks use several convolutional layers and then several fully-connected (or dense) layers. While the reasons for using convolutional layers are clear to me, I don't understand why the dense layers are needed. Can't high accuracy be achieved with only convolutional layers?&lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="2444" LastEditDate="2020-04-26T21:02:40.343" LastActivityDate="2020-04-26T21:02:40.343" Title="Why are denser layers needed in computer vision neural nets?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20656" PostTypeId="1" CreationDate="2020-04-26T16:22:37.557" Score="3" ViewCount="58" Body="&lt;p&gt;Generally, if one googles &quot;quantum machine learning&quot; or anything similar the general gist of the results is that quantum computing will greatly &lt;em&gt;speed up&lt;/em&gt; the learning process of our &quot;classical&quot; machine learning algorithms. However, &quot;speed up&quot; itself does not seem very appealing to me as the current leaps made in AI/ML are generally due to novel architectures or methods, &lt;em&gt;not&lt;/em&gt; faster training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any quantum machine learning methods in development that are &lt;em&gt;fundamentally&lt;/em&gt; different from &quot;classical&quot; methods? By this I mean that these methods are (almost*) impossible to perform on &quot;classical&quot; computers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;*except for simulation of the quantum computer of course&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="5344" LastActivityDate="2021-01-01T05:57:36.803" Title="Are there any novel quantum machine learning algorithms that are fundamentally different from &quot;classical&quot; ones?" Tags="&lt;machine-learning&gt;&lt;quantum-computing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20657" PostTypeId="2" ParentId="20604" CreationDate="2020-04-26T16:35:07.267" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, the chain rule tells us how the first variable influences the second variable, and so on. Following that logic, we should only update the weights of the first hidden layer. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't see how the second statement follows from the first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each weight &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; (not just the ones in the first layer) affects the loss &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; according to the partial derivative of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \mathcal{L}}{\partial w_i}$&lt;/span&gt;. Intuitively, the partial derivative with respect to parameter tells you how the function is changing with respect to that parameter. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My thought is, that if we backtrack the influence of the first variable but also change the values of the subsequent weights (of the subsequent hidden layer), there is no need to calculate the influence of the first weights in the first place.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure I understand your reasoning, but, typically, you update the parameters only after having computed &lt;em&gt;all&lt;/em&gt; the partial derivatives. In other words, first, you compute all partial derivatives, i.e. the gradient with back-propagation (a fancy name to denote the application of the chain rule), then you update the parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do you do this? In this case, the loss function is multi-variable function, so it depends on multiple variables. The gradient &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathcal{L} = \left[ \frac{\partial \mathcal{L}}{\partial w_1}, \dots, \frac{\partial \mathcal{L}}{\partial w_N} \right]$&lt;/span&gt; represents the direction (note that the gradient is a vector and vectors have direction) towards which your function is increasing or decreasing (depending on the sign of the gradient). &lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-26T16:35:07.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20658" PostTypeId="1" AcceptedAnswerId="23239" CreationDate="2020-04-26T17:43:51.413" Score="2" ViewCount="130" Body="&lt;p&gt;So, i have created Snake game using Pygame and Python. Then i wanted to create an AI with Genetic algorithm and a simple NN to play it. Seems pretty fun, but things aren't working out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my genetic algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def calculate_fitness(population):&#xA;    &quot;&quot;&quot;Calculate the fitness value for the entire population of the generation.&quot;&quot;&quot;&#xA;    # First we create all_fit, an empty array, at the start. Then we proceed to start the chromosome x and we will&#xA;    # calculate his fit_value. Then we will insert, inside the all_fit array, all the fit_values for each chromosome&#xA;    # of the population and return the array&#xA;    all_fit = []&#xA;    for i in range(len(population)):&#xA;        fit_value = Fitness().fitness(population[i])&#xA;        all_fit.append(fit_value)&#xA;    return all_fit&#xA;&#xA;&#xA;def select_best_individuals(population, fitness):&#xA;    &quot;&quot;&quot;Select X number of best parents based on their fitness score.&quot;&quot;&quot;&#xA;    # Create an empty array of the size of number_parents_crossover and the shape of the weights&#xA;    # after that we need to create an array with x number of the best parents, where x is NUMBER_PARENTS_CROSSOVER&#xA;    # inside config file. Then we search for the fittest parents inside the fitness array created by the&#xA;    # calculate_fitness function. Numpy.where return (array([], dtype=int64),) that satisfy the query, so we&#xA;    # take only the first element of the array and then it's value (the index inside fitness array). After we have&#xA;    # the index of the element we just need to take all the weights of that chromosome and insert them as a new&#xA;    # parent. Finally we change the fitness value of the fitness value of that chromosome inside the fitness&#xA;    # array in order to have all different parents and not only the fittest&#xA;    parents = numpy.empty((config.NUMBER_PARENTS_CROSSOVER, population.shape[1]))&#xA;    for parent_num in range(config.NUMBER_PARENTS_CROSSOVER):&#xA;        index_fittest = numpy.where(fitness == numpy.max(fitness))&#xA;        index_fittest = index_fittest[0][0]&#xA;        parents[parent_num, :] = population[index_fittest, :]&#xA;        fitness[index_fittest] = -99999&#xA;    return parents&#xA;&#xA;&#xA;def crossover(parents, offspring_size):&#xA;    &quot;&quot;&quot;Create a crossover of the best parents.&quot;&quot;&quot;&#xA;    # First we start by creating and empty array with the size equal to offspring_size we want. The type of the&#xA;    # array is [ [Index, Weights[]] ]. If the parents size is only 1 than we can't make crossover and we return&#xA;    # the parent itself, otherwise we select 2 random parents and then mix their weights based on a probability&#xA;    offspring = numpy.empty(offspring_size)&#xA;    if parents.shape[0] == 1:&#xA;        offspring = parents&#xA;    else:&#xA;        for offspring_index in range(offspring_size[0]):&#xA;            while True:&#xA;                index_parent_1 = random.randint(0, parents.shape[0] - 1)&#xA;                index_parent_2 = random.randint(0, parents.shape[0] - 1)&#xA;                if index_parent_1 != index_parent_2:&#xA;                    for weight_index in range(offspring_size[1]):&#xA;                        if random.uniform(0, 1) &amp;lt; 0.5:&#xA;                            offspring[offspring_index, weight_index] = parents[index_parent_1, weight_index]&#xA;                        else:&#xA;                            offspring[offspring_index, weight_index] = parents[index_parent_2, weight_index]&#xA;                    break&#xA;    return offspring&#xA;&#xA;&#xA;def mutation(offspring_crossover):&#xA;    &quot;&quot;&quot;Mutating the offsprings generated from crossover to maintain variation in the population.&quot;&quot;&quot;&#xA;    # We cycle though the offspring_crossover population and we change x random weights, where x is a parameter&#xA;    # inside the config file. We select a random index, generate a random value between -1 and 1 and then&#xA;    # we sum the original weight with the random_value, so that we have a variation inside the population&#xA;    for offspring_index in range(offspring_crossover.shape[0]):&#xA;        for _ in range(offspring_crossover.shape[1]):&#xA;            if random.uniform(0, 1) == config.MUTATION_PERCENTAGE:&#xA;                index = random.randint(0, offspring_crossover.shape[1] - 1)&#xA;                random_value = numpy.random.choice(numpy.arange(-1, 1, step=0.001), size=1, replace=False)&#xA;                offspring_crossover[offspring_index, index] = offspring_crossover[offspring_index, index] + random_value&#xA;    return offspring_crossover&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My neural network is formed using 7 inputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;is_left_blocked, is_front_blocked, is_right_blocked, apple_direction_vector_normalized_x,&#xA;snake_direction_vector_normalized_x, apple_direction_vector_normalized_y,snake_direction_vector_normalized_y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically if you can go left, front, right, direction to the apple and snake direction.&#xA;Then i have an hidden layer with 8 neurons and finally 3 output that indicate left, keep going or right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Neural Network forward() is calculate like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;self.get_weights_from_encoded()&#xA;Z1 = numpy.matmul(self.__W1, self.__input_values.T)&#xA;A1 = numpy.tanh(Z1)&#xA;Z2 = numpy.matmul(self.__W2, A1)&#xA;A2 = self.sigmoid(Z2)&#xA;A2 = self.softmax(A2)&#xA;return A2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where self.__W1 and self.__W2 are the weights from input to hidden layer and then the weights from hidden layer to the output. Softmax(A2) return the index of the matrix[1,3] where the value is the biggest, then i use that index to indicate the direction that my neural network choose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the config file that contains the parameters:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# GENETIC ALGORITHM&#xA;NUMBER_OF_POPULATION = 500&#xA;NUMBER_OF_GENERATION = 200&#xA;NUMBER_PARENTS_CROSSOVER = 50&#xA;MUTATION_PERCENTAGE = 0.2&#xA;&#xA;# NEURAL NETWORK&#xA;INPUT = 7&#xA;NEURONS_HIDDEN_1 = 8&#xA;OUTPUT = 3&#xA;NUMBER_WEIGHTS = INPUT * NEURONS_HIDDEN_1 + NEURONS_HIDDEN_1 * OUTPUT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the main:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for generation in range(config.NUMBER_OF_GENERATION):&#xA;&#xA;    snakes_fitness = genetic_algorithm.calculate_fitness(population)&#xA;&#xA;    # Selecting the best parents in the population.&#xA;    parents = genetic_algorithm.select_best_individuals(population, snakes_fitness)&#xA;&#xA;    # Generating next generation using crossover.&#xA;    offspring_crossover = genetic_algorithm.crossover(parents,&#xA;                                                      offspring_size=(pop_size[0] - parents.shape[0], config.NUMBER_WEIGHTS))&#xA;&#xA;    # Adding some variations to the offspring using mutation.&#xA;    offspring_mutation = genetic_algorithm.mutation(offspring_crossover)&#xA;&#xA;    # Creating the new population based on the parents and offspring.&#xA;    population[0:parents.shape[0], :] = parents&#xA;    population[parents.shape[0]:, :] = offspring_mutation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have 2 problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I don't see an improvement over the new generations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) I'm actually running the game inside the for loop, but waiting for all the snake of a generation to die and repeat with the new one is really time consuming. Isn't there a way to launch all or, atleast, more than 1 instance of the game and keep filling the array with the result?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is Fitness().fitness(population[i])&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fitness(self, weights):&#xA;    game_manager = GameManager(weights)&#xA;    self.__score = game_manager.play_game()&#xA;    return self.__score&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is where it's called inside the for loop&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def calculate_fitness(population):&#xA;    &quot;&quot;&quot;Calculate the fitness value for the entire population of the generation.&quot;&quot;&quot;&#xA;    # First we create all_fit, an empty array, at the start. Then we proceed to start the chromosome x and we will&#xA;    # calculate his fit_value. Then we will insert, inside the all_fit array, all the fit_values for each chromosome&#xA;    # of the population and return the array&#xA;    all_fit = []&#xA;    for i in range(len(population)):&#xA;        fit_value = Fitness().fitness(population[i])&#xA;        all_fit.append(fit_value)&#xA;    return all_fit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This the function that launch the game (GameManager(weights)) and return the score of the snake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my first time on AI so this code could be all a mess, don't worry about pointing out what i did wrong, just please don't say &quot;It's all wrong&quot; because i won't be able to learn otherwise.&lt;/p&gt;&#xA;" OwnerUserId="36482" LastActivityDate="2020-08-25T04:23:36.663" Title="Genetic Algorithm Python Snake not improving" Tags="&lt;python&gt;&lt;genetic-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20660" PostTypeId="2" ParentId="20654" CreationDate="2020-04-26T18:31:28.810" Score="1" Body="&lt;p&gt;Convolutional layers are added in order to extract features from the image (like edges, corners, textures). After extracting those features, you feed them to a fully connected neural network to get the prediction.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take an example, consider you want to classify the cat's image. But you decided to do this by only using the convolutional layer. So, you feed the image to a convolutional layer. After passing through some layer it extracted the key features from the cat's image and at the end of the layer, you add a single neuron(since we have a single class to predict). Now it's ready to classify it as a cat. But unfortunately, it's misclassified the image. Why? Now let us answer the question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since convolutional layers are not fully connected layers, the neuron added at the last layer is only connected with a handful of neurons from the previous layer. So it misses the key features (encapsulated by some other neurons it is not connected with them) in order to detect this image as a cat. In order to get the prediction right, you have to add a dense layer(a fully connected layer) at the end of the convolutional layer so that it can get all of those extracted features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this does not satisfy you, please, ask the question more specifically and also edit the question because linear layer does not call fully connected layer. &lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-04-26T19:25:59.680" LastActivityDate="2020-04-26T19:25:59.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20661" PostTypeId="1" CreationDate="2020-04-26T18:41:58.877" Score="2" ViewCount="68" Body="&lt;p&gt;I have been trying to figure out what the Fourier transformed image represents. I am aware of Fourier transformation in general, but I can't explain myself the image it forms after transformation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the given image, what does the outlined white sort of lines mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DcRGm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DcRGm.png&quot; alt=&quot;Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36484" LastEditorUserId="2444" LastEditDate="2020-04-26T21:11:31.260" LastActivityDate="2021-01-21T23:00:55.080" Title="What does the Fourier transformed image mean?" Tags="&lt;computer-vision&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20662" PostTypeId="2" ParentId="20661" CreationDate="2020-04-26T19:02:17.473" Score="1" Body="&lt;p&gt;Have a look at this &lt;a href=&quot;https://homepages.inf.ed.ac.uk/rbf/HIPR2/fourier.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;explanation&lt;/a&gt; of the two dimensional Fourier Transform applied to an image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I read it is that it somewhat mirrors the diagonal structure of the 'X', as that is where most of the 'image energy' is, so that are the lines you highlighted. Because the image is fairly uniform in colour, most of the components are low-frequency (ie close to the centre of the 2DFT image).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm not really that familiar with a 2D version either, as I mainly have experience with using FFTs in speech processing.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-26T19:02:17.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20663" PostTypeId="1" AcceptedAnswerId="20679" CreationDate="2020-04-26T20:08:35.723" Score="1" ViewCount="113" Body="&lt;p&gt;From the perspective of the type of AI Agents, I would like to discuss Prim's Minimum Spanning Tree algorithm and Dijkstra's Algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Both are model-based agents and both are &amp;quot;greedy algorithms&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Both have their memory to store the history of vertices and their path distance. Prim's is &lt;strong&gt;more greedy&lt;/strong&gt; than Dijkstra's algorithm whereas Dijkstra's algorithm is &lt;strong&gt;more efficient&lt;/strong&gt; than Prim's.&lt;/p&gt;&#xA;&lt;p&gt;Can we say that Dijkstra's algorithm is a utility-based agent, whereas Prim's is a goal-based agent, with the justification that Prim's is &lt;strong&gt;more goal-oriented&lt;/strong&gt; as compared to finding the optimum (shortest) path?&lt;/p&gt;&#xA;" OwnerUserId="34306" LastEditorUserId="2444" LastEditDate="2020-11-19T12:51:20.620" LastActivityDate="2020-11-19T12:51:20.620" Title="What types of AI agents are Djikstra's algorithm and Prim's Minimum Spanning Tree algorithm?" Tags="&lt;intelligent-agent&gt;&lt;goal-based&gt;&lt;utility-based&gt;&lt;dijkstras-algorithm&gt;&lt;prims-algorithm&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20664" PostTypeId="1" AcceptedAnswerId="20665" CreationDate="2020-04-26T21:42:54.467" Score="2" ViewCount="92" Body="&lt;p&gt;I am looking at this formula which breaks down the gradient of &lt;span class=&quot;math-container&quot;&gt;$P(\tau |\theta)$&lt;/span&gt; the first part is clear as is the derivative of &lt;span class=&quot;math-container&quot;&gt;$\log(x)$&lt;/span&gt;, but I do not see how the first formula is rearranged into the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fuTAh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fuTAh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36486" LastEditorUserId="2444" LastEditDate="2020-04-26T23:45:20.637" LastActivityDate="2020-04-26T23:55:28.240" Title="How is the log-derivative trick of a trajectory derived?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;policy-gradients&gt;&lt;calculus&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20665" PostTypeId="2" ParentId="20664" CreationDate="2020-04-26T23:18:19.543" Score="1" Body="&lt;p&gt;The identity &lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta} P(\tau \mid \theta) = P(\tau \mid \theta) \nabla_{\theta} \log P(\tau \mid \theta)\tag{1}\label{1},$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can also be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \log P(\tau \mid \theta) &#xA;&amp;amp;= &#xA;\frac{\nabla_{\theta} P(\tau \mid \theta)}{P(\tau \mid \theta)}\\&#xA;&amp;amp;=\frac{1}{P(\tau \mid \theta)} \nabla_{\theta} P(\tau \mid \theta)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;directly comes from the general rule to derive the logarithm of a function and the chain rule&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{d \log f(x)}{d x} &#xA;&amp;amp;= \frac{1}{f(x)} \frac{d f}{dx}.&#xA;\end{align}&lt;/span&gt;&#xA;Note that &lt;span class=&quot;math-container&quot;&gt;$\log f(x)$&lt;/span&gt; is a composite function and that's why we apply the chain rule and that the derivative of &lt;span class=&quot;math-container&quot;&gt;$\log x = \frac{1}{x}$&lt;/span&gt;, as your text says.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People shouldn't call this a trick. There's no trick here. It's just basic calculus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do you need identity \ref{1}? Because that identity tells you that the derivative of the probability of the trajectory given the parameter &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$P(\tau \mid \theta)$&lt;/span&gt; times the gradient of the logarithm of that same probability. How is this useful? Because the logarithm will turn your product into a sum (and the derivative of a sum is the sum of the derivatives of the elements of the sum), Essentially, the identity \ref{1} will help you to compute the gradient is an easier way (at least, conceptually).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-26T23:55:28.240" LastActivityDate="2020-04-26T23:55:28.240" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20667" PostTypeId="1" AcceptedAnswerId="20671" CreationDate="2020-04-27T02:23:20.220" Score="1" ViewCount="203" Body="&lt;p&gt;SARSA is on-policy, while n-step SARSA is off-policy. But when n = 1, is it like an off-policy version of SARSA? Any similarity and difference between 1-step SARSA and SARSA?&lt;/p&gt;&#xA;" OwnerUserId="23707" LastEditorUserId="2444" LastEditDate="2020-04-27T02:40:13.420" LastActivityDate="2020-04-27T12:47:07.723" Title="What are the differences between 1-step SARSA and SARSA?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;off-policy-methods&gt;&lt;sarsa&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20668" PostTypeId="2" ParentId="18701" CreationDate="2020-04-27T04:50:43.280" Score="1" Body="&lt;p&gt;Not really something that slows it down but currently Mauricio Santillana at Harvard is working on modeling the pandemic and has shared some of his approaches. He explained that they have used google search trends to try to predict the number of actual cases (there is a delay between people being sick and getting tested). Looking for search terms like, &quot;how to use an inhaler?&quot; can reveal areas affected by the outbreak and is useful for modeling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paper can be found &lt;a href=&quot;https://arxiv.org/pdf/2004.04019.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Towards Data Science has several articles listing potential AI applications for helping in the fight against COVID&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Including:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Identify who is most at risk&lt;/li&gt;&#xA;&lt;li&gt;Diagnose patients&lt;/li&gt;&#xA;&lt;li&gt;Develop drugs faster&lt;/li&gt;&#xA;&lt;li&gt;Predict the spread of the disease&lt;/li&gt;&#xA;&lt;li&gt;Understand viruses better&lt;/li&gt;&#xA;&lt;li&gt;Map where viruses come from&lt;/li&gt;&#xA;&lt;li&gt;Predict the next pandemic&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/fight-covid-19-with-machine-learning-1d1106192d84&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to fight COVID-19 with machine learning&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though there are probably many creative applications that AI can help with I think most of them are modeling related at the moment.&lt;/p&gt;&#xA;" OwnerUserId="36131" LastActivityDate="2020-04-27T04:50:43.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20669" PostTypeId="2" ParentId="20564" CreationDate="2020-04-27T06:22:23.013" Score="0" Body="&lt;p&gt;I think there is no special method for training the neural network in large datasets. But I can add some suggestion for you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Use the convolutional neural network for this dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) You can use huber loss instead of squared loss and see what happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) See if you have enough small magnitude training data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, please define your problem in more detail (like what those images represent and what you want to predict, what are the features etc).&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-04-27T06:22:23.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20670" PostTypeId="1" CreationDate="2020-04-27T06:44:16.850" Score="1" ViewCount="28" Body="&lt;p&gt;The goal of our project is to identify the quality of a grain and we have two values, &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A&lt;/code&gt; can take the values low  &lt;code&gt;L&lt;/code&gt;, medium &lt;code&gt;M&lt;/code&gt; and high &lt;code&gt;H&lt;/code&gt;. And &lt;code&gt;B&lt;/code&gt; can take the values of Low, Medium and High (&lt;code&gt;L&lt;/code&gt;,&lt;code&gt;M&lt;/code&gt;,&lt;code&gt;H&lt;/code&gt;).&lt;br&gt; Specifically, the ranges for &lt;code&gt;A&lt;/code&gt; are -&gt; &lt;code&gt;5-10&lt;/code&gt; (low), &lt;code&gt;10-15&lt;/code&gt; (medium), &lt;code&gt;15-20&lt;/code&gt; (high). &lt;br&gt;And the ranges for &lt;code&gt;B&lt;/code&gt; are -&gt; &lt;code&gt;50-75&lt;/code&gt; (low), &lt;code&gt;75-90&lt;/code&gt; (medium), &lt;code&gt;90-110&lt;/code&gt; (high)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output for these are bad &lt;code&gt;B&lt;/code&gt;, average &lt;code&gt;A&lt;/code&gt; and good &lt;code&gt;G&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3sUvs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3sUvs.png&quot; alt=&quot;The sample table is shown here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we determine the membership functions and values for this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to write Python code for the fuzzy system, but we are beginners and have no idea how to go about this. Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="36495" LastEditorUserId="36495" LastEditDate="2020-04-27T13:23:06.403" LastActivityDate="2020-04-27T13:23:06.403" Title="How do we determine the membership functions and values for this problem?" Tags="&lt;ai-design&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20671" PostTypeId="2" ParentId="20667" CreationDate="2020-04-27T07:14:09.897" Score="0" Body="&lt;p&gt;N-step SARSA can be both off policy and on-policy. I think you already know the n step on-policy SARSA. So I am just telling you how n-step SARSA can be off-policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Off-policy n-step SARSA:&lt;/strong&gt; Now you have two policies, one is target policy, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, (let's say it is a greedy policy), another one is behavior policy, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, (you are actually following this behavior policy). Since this is off-policy, you do importance sampling for that. So the update rule is like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{t+n}(S_{t},A_{t}) = Q_{t+n-1}(S_{t},A_{t}) + \alpha \rho_{t+1:t+n-1}[G_{t:t+n} - Q_{t+n-1}(S_{t},A_{t})],$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{t:h} = \prod_{t=k}^{h} \frac{\pi(A_{k}|S_{k})}{b(A_{k}|S_{k})}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are following the behavior policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, but shifting the Q values towards the target policy, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Off-policy one-step SARSA:&lt;/strong&gt; You can think of Q learning as one-step off policy SARSA.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-04-27T12:47:07.723" LastActivityDate="2020-04-27T12:47:07.723" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20672" PostTypeId="2" ParentId="20597" CreationDate="2020-04-27T07:44:57.380" Score="1" Body="&lt;p&gt;Not familiar with RBF but when you have a small data set image augmentation can help. You can do this easily with the Keras ImageDataGenerator, documentation is &lt;a href=&quot;https://keras.io/preprocessing/image/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Alternatively you can create image augmentation yourself using image processing models like PIL or CV2.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-04-27T07:44:57.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20673" PostTypeId="1" CreationDate="2020-04-27T08:02:00.973" Score="1" ViewCount="38" Body="&lt;p&gt;I am facing the following supervised learning problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An object is fully characterized by its position in &lt;span class=&quot;math-container&quot;&gt;$R^n$&lt;/span&gt;. There are &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; objects. There are fully observable (i.e. their positions are always known). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, exactly one of these objects is activated. Activation is fully observable, i.e. the index &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$a_t \in [1,m]$&lt;/span&gt;) of the object activated at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is known. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that, under the hood, activation works this way:  there is a priority function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$f: R^n \to R$&lt;/span&gt; ), which computes, for each time step, the priority score of each object. The object for which the priority score was the highest is activated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to find (the approximation of) one of the possible priority functions that would match a given data-set. A data-set is of size &lt;span class=&quot;math-container&quot;&gt;$(m*n+1)*t$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; positions of dimension &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, plus the index of the activated object, over &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; time steps). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, if it turns out there is a hidden fixed beacon, and at each time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; the object the closest to the beacon is activated, then a possible function would be &lt;span class=&quot;math-container&quot;&gt;$f(o_{it})=1/d_{it}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$d_{it}$&lt;/span&gt; is the distance between the beacon and the object &lt;span class=&quot;math-container&quot;&gt;$o_i$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If several objects have the same highest priority score, then only one of them is activated, selected randomly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function found by the algorithm may be parametric and encoded by a neural network, if this is applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method for finding one such function ?&lt;/p&gt;&#xA;" OwnerUserId="36496" LastEditorUserId="36496" LastEditDate="2020-04-27T20:32:17.010" LastActivityDate="2020-04-27T20:32:17.010" Title="How can I approximate a function that determines the priority of objects?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;datasets&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20674" PostTypeId="1" CreationDate="2020-04-27T09:58:32.453" Score="1" ViewCount="25" Body="&lt;p&gt;I use an off-the-shelf convolutional neural network, where at the end of the convolutional part, the depth of the last convolutional layer is expanded and then its 2D average is computed (such that for a tensor of say 8x8x512, you get its 2D average, which is of 1x512). It is a commonly used operation in deep networks, called Global Average Pooling 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only tensor that is input to the fully-connected part is that 2D averaged 1x512 tensor, i.e., a tensor that should not preserve the 2D information. Yet, my fully-connected last layer neurons, which have been trained to predict the &lt;strong&gt;2D location&lt;/strong&gt; of objects, work very well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about it for a long time and couldn't find any convincing explanation how come the network preserved the 2D information in the averaged tensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea?&lt;/p&gt;&#xA;" OwnerUserId="36499" LastActivityDate="2020-04-27T09:58:32.453" Title="How come a detection works after global average pooling 2D?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20675" PostTypeId="1" CreationDate="2020-04-27T11:02:50.867" Score="1" ViewCount="92" Body="&lt;p&gt;I am using the following architechture:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3*(fully connected -&amp;gt; batch normalization -&amp;gt; relu -&amp;gt; dropout) -&amp;gt; fully connected&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should I add the &lt;code&gt;batch normalization -&amp;gt; relu -&amp;gt; dropout&lt;/code&gt; part after the last fully connected layer as well (the output is positive anyway, so the relu wouldn't hurt I suppose)? &lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="36083" LastEditDate="2020-04-27T11:31:12.923" LastActivityDate="2021-05-23T06:03:20.263" Title="Should batch-normalization/dropout/activation-function layers be used after the last fully connected layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20676" PostTypeId="1" AcceptedAnswerId="20737" CreationDate="2020-04-27T11:03:21.857" Score="0" ViewCount="139" Body="&lt;p&gt;I wonder what happens to the 'channels' dimension (usually 3 for RGB images) after the first convolution layer in CNNs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In books and other sources, it is always said that the depth of the output from convolutional layers is the number of kernels (filters) in that layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if the input image has 3 channels and we convolve each of them with &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; kernels, shouldn't the depth of the output be &lt;span class=&quot;math-container&quot;&gt;$K * 3$&lt;/span&gt;? Are they somehow 'averaged' or in other way combined with each other?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="2444" LastEditDate="2020-04-27T12:19:22.620" LastActivityDate="2020-05-28T17:01:25.063" Title="What happens to the channels after the convolution layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20677" PostTypeId="2" ParentId="20638" CreationDate="2020-04-27T11:09:31.067" Score="0" Body="&lt;p&gt;I'm not sure what do you mean by one input. The input to the pruning agent is always the same, it's the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; of dimension &lt;span class=&quot;math-container&quot;&gt;$m \times h \times w$&lt;/span&gt;. The layer is taken from baseline model that is pretrained. The input doesn't change it's always the same. The output of the pruning agent is an array of probabilities to prune a specific filter. For example if you have &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; filters in a layer, the output of the pruning agent will be array of &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; elements . Let's say its&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;y = [0.1, 0.6, 0.7]&#xA;\end{equation}&lt;/span&gt;&#xA;Each of these elements represents probability of pruning filter &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in layer &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;. So &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; would be probability to prune filter &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0.6$&lt;/span&gt; to prune filter &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0.7$&lt;/span&gt; to prune filter &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;.&#xA;Let's say you sample this distribution &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; times and you get: &lt;span class=&quot;math-container&quot;&gt;$[0, 1, 1], [0, 0, 1]$&lt;/span&gt;. That means you would make 2 different models from the original baseline model. First model would have 2nd and 3rd filter pruned in layer &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, and second model would have 3rd filter pruned. The you run those 2 new models on your train and validation set, calculate objective function &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. Then you update parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; of your pruning agent based on &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. The original weights of layer &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; stay untouched. Then you do another inference of the pruning model &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; with updated parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (the input is still original &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;). You will get another array of probabilities and you keep repeating previous steps that i described until parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; converge. When they converge you make final pruning.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2020-04-27T11:18:03.067" LastActivityDate="2020-04-27T11:18:03.067" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20679" PostTypeId="2" ParentId="20663" CreationDate="2020-04-27T12:45:56.113" Score="1" Body="&lt;p&gt;Why do you want to think of these algorithms as agents? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An agent is an abstract and higher-level concept than the concept of an algorithm, which is just a set of instructions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could have two agents, one that is supposed to find the minimum spanning tree and another that is supposed to find the shortest path between a source and goal nodes.  In both cases, these two agents have a goal, so they are both goal-oriented agents, but the algorithms they use to reach the goal are irrelevant, as long as they have one or more goals. If you have an agent that performs optimally with respect to some metric, you may want to call that agent an optimal agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar logic applies to utility-based agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in short, algorithms are used by agents to act, but the algorithms themselves are not the agents (unless you want to have a philosophical debate or use different definitions of agents). See &lt;a href=&quot;https://ai.stackexchange.com/a/12995/2444&quot;&gt;this answer&lt;/a&gt; for more details about the definition of agents.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-04-27T12:45:56.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20680" PostTypeId="1" CreationDate="2020-04-27T13:21:20.443" Score="10" ViewCount="2035" Body="&lt;p&gt;I know it's not an exact science. But would you say that generally for more complicated tasks, deeper nets are required?&lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="2444" LastEditDate="2020-04-27T19:04:57.637" LastActivityDate="2020-05-12T12:49:40.957" Title="Should neural nets be deeper the more complex the learning problem is?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;architecture&gt;" AnswerCount="4" CommentCount="1" FavoriteCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20681" PostTypeId="1" AcceptedAnswerId="21079" CreationDate="2020-04-27T13:55:43.163" Score="1" ViewCount="2716" Body="&lt;p&gt;I'm trying to extract the vectors from the sentences. Spent soo much time searching for the pre-trained BERT models but found nothing.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is it possible to get the vectors using pre-trained BERT from the data?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-07-06T10:29:11.303" LastActivityDate="2020-07-06T10:29:11.303" Title="How to use pre-trained BERT to extract the vectors from sentences?" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20682" PostTypeId="1" CreationDate="2020-04-27T14:03:34.240" Score="2" ViewCount="49" Body="&lt;p&gt;Let's say one wants to use a neural net to learn some function &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt;. Let's say that we know that &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is a combination of two functions (or two sub-problems), &lt;span class=&quot;math-container&quot;&gt;$g(x)=f_2(f_1(x))$&lt;/span&gt;, and that we have two datasets &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;composed of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; samples and their corresponding &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; labels, and &lt;/li&gt;&#xA;&lt;li&gt;composed of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; samples and their corresponding &lt;span class=&quot;math-container&quot;&gt;$f_1(x)$&lt;/span&gt; labels. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Should we use two nets, one to learn the mapping from &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; samples to &lt;span class=&quot;math-container&quot;&gt;$f_1(x)$&lt;/span&gt; using dataset 1 and another net to learn the mapping from &lt;span class=&quot;math-container&quot;&gt;$f_1(x)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; (note that we can build a dataset composed of &lt;span class=&quot;math-container&quot;&gt;$f_1(x)$&lt;/span&gt; samples and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; labels with the trained net), or just one net to learn mappings from &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; using dataset 1? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, the first option seems to be better since we take advantage of our knowledge that &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; is a &quot;sub-problem&quot; of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="2444" LastEditDate="2020-04-27T14:17:21.423" LastActivityDate="2021-05-23T10:06:35.620" Title="A model for each sub-problem vs one model for the whole problem" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;computational-learning-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20683" PostTypeId="1" CreationDate="2020-04-27T14:06:23.133" Score="1" ViewCount="33" Body="&lt;p&gt;I am working on the MNIST data on my own. The idea is to use different values for the number of hidden layers, number of nodes in a given layer, etc. How do you organize these things while you are working on creating a model for a problem? DO you do everything in one code file or you use different code files for choosing the best?&lt;/p&gt;&#xA;" OwnerUserId="36510" LastActivityDate="2020-04-27T16:23:12.237" Title="How to work on different models for a given problem?" Tags="&lt;deep-learning&gt;&lt;models&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20684" PostTypeId="1" AcceptedAnswerId="20687" CreationDate="2020-04-27T14:44:19.937" Score="2" ViewCount="61" Body="&lt;p&gt;I have a dataset which includes states, actions, and reward. The dataset includes information on the transition, i.e., &lt;span class=&quot;math-container&quot;&gt;$p(r,s' \mid s,a)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to estimate a behavior policy from this dataset so that it can be used in an off-policy learning algorithm?&lt;/p&gt;&#xA;" OwnerUserId="23707" LastEditorUserId="2444" LastEditDate="2020-04-27T16:01:32.637" LastActivityDate="2020-04-28T03:19:23.590" Title="How to estimate a behavior policy for off-policy learning based on data?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;off-policy-methods&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20686" PostTypeId="1" CreationDate="2020-04-27T15:17:48.300" Score="2" ViewCount="52" Body="&lt;p&gt;The derivative of ReLU is 0 if its output is lower than 0 -  &lt;span class=&quot;math-container&quot;&gt;$d ReLU(x)/dReLU$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$x &amp;lt; 0$&lt;/span&gt;. Let's denote some net's output by &lt;span class=&quot;math-container&quot;&gt;$Out$&lt;/span&gt;, so if this net's last layer is ReLU then we get that &lt;span class=&quot;math-container&quot;&gt;$dOut/dReLU$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$Out &amp;lt; 0$&lt;/span&gt;. Subsequently, for every parameter &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; in the net we would get that &lt;span class=&quot;math-container&quot;&gt;$dOut/dp$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Does that mean that for every sample &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$Out(x) &amp;lt; 0$&lt;/span&gt; the net doesn't learn at all from that sample since the derivative for each parameter is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="36083" LastActivityDate="2020-04-28T04:07:18.310" Title="Does net with ReLU not learn when output &lt; 0?" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20687" PostTypeId="2" ParentId="20684" CreationDate="2020-04-27T15:24:09.847" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a way to estimate a behavior policy from this dataset so that it can be used in an off-policy learning algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have enough examples of &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; pairs for each instance of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; then you can simply estimate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$b(a|s) = \frac{N(a,s)}{N(s)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; counts the number of instances in your dataset. This might be enough to use off-policy with importance sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can use an off-policy approach that doesn't need importance sampling. The most straightforward one here would be single-step Q learning. The update step for 1-step Q-learning does not depend on behaviour policy, because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The action value being updated &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; already assumes &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is being taken, so you don't need any conditional probability there.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The TD target &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \text{max}_{a'}[Q(s',a')]$&lt;/span&gt; does not need to be adjusted for behaviour policy, it works with the target policy directly (implied as &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \text{argmax}_{a}[Q(s,a)]$&lt;/span&gt;) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A 2-step Q learning algorithm would need to adjust for likelihood &lt;span class=&quot;math-container&quot;&gt;$b(a'|s')$&lt;/span&gt; in the TD target &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi(a'|s')}{b(a'|s')}(r + \gamma r' + \gamma^2\text{max}_{a''}[Q(s'',a'')])$&lt;/span&gt; - typically &lt;span class=&quot;math-container&quot;&gt;$\pi(a'|s')$&lt;/span&gt; is either 0 or 1, thus making &lt;span class=&quot;math-container&quot;&gt;$b(a'|s')$&lt;/span&gt; irrelevant some of the time. But you would still prefer to know it for performing updates it you can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are making updates offline and off-policy, then single-step Q learning is probably the simplest approach. It will require more update steps overall to reach convergence, but each one will be simpler.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-27T15:56:20.757" LastActivityDate="2020-04-27T15:56:20.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20688" PostTypeId="2" ParentId="20684" CreationDate="2020-04-27T15:27:12.667" Score="0" Body="&lt;p&gt;If your data look like this &lt;span class=&quot;math-container&quot;&gt;$(s_{1},a_{1},r_{1},s_{2}),(s_{2},a_{2},r_{2},s_{3}),....,$&lt;/span&gt; then this sample drawn from a particular behavior policy. So, you do not need to find the behavior policy just Q-Learning to find the optimal policy while following the behavior policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the MDP is too big then consider applying Deep Q Learning. In both cases, the transition probability they have given has no use. But if you use on-policy learning and you know the dynamics of the system(means transition probabilities), I will recommend you to use dynamic programming(if state-space is not quite large). But for your above problem setting, you can not use dynamic programming, you have only one choice to use off-policy learning. &lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-04-27T15:37:19.660" LastActivityDate="2020-04-27T15:37:19.660" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20689" PostTypeId="2" ParentId="20680" CreationDate="2020-04-27T15:33:27.723" Score="13" Body="&lt;h2&gt;Deeper models can have advantages (in certain cases)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Most people will answer &quot;yes&quot; to your question, see e.g. &lt;a href=&quot;https://stats.stackexchange.com/q/222883/82135&quot;&gt;Why are neural networks becoming deeper, but not wider?&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/q/3147754/168764&quot;&gt;Why do deep neural networks work well?&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, there are cases where deep neural networks have certain advantages compared to shallow ones. For example, see the following papers &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v49/eldan16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Power of Depth for Feedforward Neural Networks&lt;/a&gt; (2016) by Ronen Eldan and Ohad Shamir&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1602.04485.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Benefits of depth in neural networks&lt;/a&gt; (2016) by Matus Telgarsky.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1610.09887.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks&lt;/a&gt; (2017) by Safran and Shamir&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1709.05289.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal approximation of piecewise smooth functions using deep ReLU neural networks&lt;/a&gt; (2018) by Petersen and Voigtlaender&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;What about the width?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The following papers may be relevant&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1605.07146&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wide Residual Networks&lt;/a&gt; (2017) by Sergey Zagoruyko and Nikos Komodakis&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://papers.nips.cc/paper/7203-the-expressive-power-of-neural-networks-a-view-from-the-width&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Expressive Power of Neural Networks: A View from the Width&lt;/a&gt; (2017) by Zhou Lu et al.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Bigger models have bigger capacity but also have disadvantages&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Vladimir Vapnik (co-inventor of VC theory and SVMs, and one of the most influential contributors to learning theory), who is &lt;a href=&quot;https://www.youtube.com/watch?v=Ow25mjFjSmg&quot; rel=&quot;nofollow noreferrer&quot;&gt;not a fan of neural networks&lt;/a&gt;, will probably tell you that you should look for the smallest model (set of functions) that is consistent with your data (i.e. an admissible set of functions).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, watch this podcast &lt;a href=&quot;https://www.youtube.com/watch?v=STFcvzoxVw4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vladimir Vapnik: Statistical Learning | Artificial Intelligence (AI) Podcast&lt;/a&gt; (2018), where he says this. His new learning theory framework based on statistical invariants and predicates can be found in the paper &lt;a href=&quot;https://link.springer.com/article/10.1007/s10994-018-5742-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rethinking statistical learning theory: learning using statistical invariants&lt;/a&gt; (2019). You should also read &lt;a href=&quot;https://www.learningtheory.org/learning-has-just-started-an-interview-with-prof-vladimir-vapnik/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Learning Has Just Started&quot; – an interview with Prof. Vladimir Vapnik&lt;/a&gt; (2014).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bigger models have a bigger capacity (i.e. a bigger VC dimension), which means that you will more likely &lt;strong&gt;overfit the training data&lt;/strong&gt;, i.e., the model may not really be able to generalize to unseen data. So, in order not to overfit, models with more parameters (and thus capacity) will also require more data. You should also ask yourself why people use &lt;strong&gt;regularisation techniques&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, models that achieve state-of-the-art performance can be very deep, but they are also &lt;strong&gt;computationally inefficient to train&lt;/strong&gt; and they &lt;strong&gt;require huge amounts of training data&lt;/strong&gt; (either manually labeled or automatically generated). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, there are many other technical complications with deeper neural networks, for example, problems such as the &lt;strong&gt;vanishing (and exploding) gradient problem&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Complex tasks may not require bigger models&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Some people will tell you that you require deep models because, empirically, some deep models have achieved state-of-the-art results, but that's probably because we haven't found cleverer and more efficient ways of solving these problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I would not say that &quot;complex tasks&quot; (whatever the definition is) necessarily require deeper or, in general, bigger models. While designing our models, it may be a good idea to always keep in mind principles like Occam's razor!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A side note&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As a side note, I think that more people should focus more on the mathematical aspects of machine learning, i.e. computational and statistical learning theory. There are too many practitioners, who don't really understand the underlying learning theory, and too few theorists, and the progress could soon stagnate because of a lack of understanding of the underlying mathematical concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you a more concrete idea of the current mentality of the deep learning community, in &lt;a href=&quot;https://www.youtube.com/watch?v=9EN_HoEk3KY&quot; rel=&quot;nofollow noreferrer&quot;&gt;this lesson&lt;/a&gt;, a person like Ilya Sutskever, who is considered an &quot;important and leading&quot; researcher in deep learning, talks about NP-complete problems as if he doesn't really know what he's talking about. NP-complete problems aren't just &quot;hard problems&quot;. NP-completeness has a very specific definition in computational complexity theory!&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-12T12:49:40.957" LastActivityDate="2020-05-12T12:49:40.957" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20690" PostTypeId="2" ParentId="20564" CreationDate="2020-04-27T15:41:55.580" Score="0" Body="&lt;p&gt;One option is &lt;a href=&quot;https://en.wikipedia.org/wiki/Normalization_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;normalizing&lt;/a&gt; your data. In particular, min-max feature scaling to bring all values into the range [0,1] is particularly useful with gradient descent.&lt;/p&gt;&#xA;" OwnerUserId="15403" LastActivityDate="2020-04-27T15:41:55.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20691" PostTypeId="2" ParentId="20394" CreationDate="2020-04-27T15:44:24.093" Score="0" Body="&lt;p&gt;Thresholding based on variance is one method for anomaly detection in time-series data. &lt;/p&gt;&#xA;" OwnerUserId="15403" LastActivityDate="2020-04-27T15:44:24.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20692" PostTypeId="2" ParentId="20680" CreationDate="2020-04-27T15:49:09.493" Score="2" Body="&lt;p&gt;Deeper networks have more learning capacity in the sense that they can fit to more complex data. But at the same time, they are also more prone to overfitting the training data and therefore fails to generalize to the test set.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from overfitting, exploding/vanishing gradients is another problem which hampers convergence. This can be addressed by normalizing the initialization and normalizing the intermediate layers. Then you can do backpropagation with stochastic gradient descent (SGD).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When deeper networks are able to converge, another problem of 'degradation' has been detected. The accuracy saturates and then starts to degrade. This is not caused by overfitting. In fact, adding more layers here leads to higher training error. A possible fix is to use ResNets (residual networks), which have been shown to decrease 'degradation'&lt;/p&gt;&#xA;" OwnerUserId="36512" LastActivityDate="2020-04-27T15:49:09.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20693" PostTypeId="2" ParentId="20683" CreationDate="2020-04-27T16:23:12.237" Score="1" Body="&lt;p&gt;it seems to me that you are talking about hyperparameter tuning and effect of hyperparameters on the network in general. If you are working with tensorflow, I recommend you to look into &lt;a href=&quot;https://www.tensorflow.org/tensorboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;tensorboard&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=eBbEDRsCmv4&amp;amp;t=1s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hands-on TensorBoard&lt;/a&gt; can be a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="33835" LastActivityDate="2020-04-27T16:23:12.237" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20694" PostTypeId="1" CreationDate="2020-04-27T17:26:36.117" Score="3" ViewCount="36" Body="&lt;p&gt;Suppose we have an object detector that is trained to detect &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; products. If two objects are too close to each other, in general, would an object detector do a poor job of correctly classifying them? If they were far apart in the scene, would the object detector to a better job of correctly classifying them? &lt;/p&gt;&#xA;" OwnerUserId="36517" LastEditorUserId="2444" LastEditDate="2020-08-02T12:59:03.860" LastActivityDate="2020-08-02T12:59:03.860" Title="If two objects are too close to each other, would an object detector do a poor job of correctly classifying them?" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20695" PostTypeId="2" ParentId="20680" CreationDate="2020-04-27T18:28:33.640" Score="2" Body="&lt;p&gt;My experience from a tactical standpoint is to start out with a smaller simple model first. Train the model and observe the training accuracy and validation loss and validation accuracy. My observation is that, to be a good model, your training accuracy should achieve a value of at least 95%. If it does not, then try to optimize some of the hyper-parameters. If the training accuracy does not improve, then you may try to incrementally add more complexity to the model. As you add more complexity the risk of overfitting, vanishing or exploding gradients becomes higher. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can detect overfitting by monitoring the validation loss. If as the model accuracy goes up the validation loss on later epochs starts to go up you are overfitting. At that point, you will have to take remedial action in your model like adding dropout layers and use regularizers. Keras documentation is &lt;a href=&quot;https://keras.io/regularizers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out in &lt;a href=&quot;https://ai.stackexchange.com/a/20689/2444&quot;&gt;the answer by nbro&lt;/a&gt;, the theory addressing this issue is complex. I highly recommend the excellent tutorial on this subject which can be found on YouTube &lt;a href=&quot;https://www.youtube.com/watch?v=mbyG85GZ0PI&amp;amp;list=PLnIDYuXHkit4LcWjDe0EwlE57WiGlBs08&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-04-27T20:32:12.897" LastActivityDate="2020-04-27T20:32:12.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20696" PostTypeId="2" ParentId="20555" CreationDate="2020-04-27T18:43:29.363" Score="0" Body="&lt;p&gt;in reinforcement learning, neural networks are used to estimate the value function (board state worth), not to choose the action directly. In most games, the actions available are state-dependent anyway, so you cannot easily formulate them as ANN outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the idea is that at each state, you consider the alternative actions, and the one that leads to the most valuable state is the action of choice (without using lookahead). Your ANN will thus be approximating the board state values&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strictly speaking for tic-tac-toe you don't need a neural network, the tabular Q-learning method would suffice. Have you read Sutton and Barto book on RL?&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-04-27T18:43:29.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20697" PostTypeId="2" ParentId="6556" CreationDate="2020-04-27T18:58:57.613" Score="3" Body="&lt;p&gt;The term &lt;em&gt;feature embedding&lt;/em&gt; appears to be a synonym for &lt;em&gt;feature extraction, feature learning&lt;/em&gt; etc. I.e. a form of embedding/dimension reduction (with the caveat the goal may not be a lower dimensional representation but one of equal dimensionality, but more meaningfully expressed):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Feature embedding is an emerging research area which intends to transform features from the original space into a new space to support effective learning. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/s10796-018-9850-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalized Feature Embedding for Supervised, Unsupervised, and Online Learning Tasks&lt;/a&gt; (2019)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Feature embedding aims to learn a low-dimensional vector&#xA;  representation for each instance to preserve the information&#xA;  in its features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://people.tamu.edu/~xhuang/Xiao_AAAI19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Large-Scale Heterogeneous Feature Embedding&lt;/a&gt; (2019)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-04-27T19:12:32.987" LastActivityDate="2020-04-27T19:12:32.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20698" PostTypeId="2" ParentId="20403" CreationDate="2020-04-27T19:03:12.193" Score="0" Body="&lt;p&gt;RL is a generic technique that can be applied to any MDP system. From the looks of it you have data to produce a state-space model of your system (system identification, excluding your existing control loops) and then you can use that to drive a simulated exploration of your process and discover a control policy. As this is a continuous process, some discretization will be required&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so yes, your quest is feasible, but not trivial! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I experimented sometime ago with the cartpole balancing system, which is a toy, but I found that a good old PID was much better than AI :)&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-04-27T19:03:12.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20699" PostTypeId="1" AcceptedAnswerId="20700" CreationDate="2020-04-27T19:44:40.247" Score="0" ViewCount="128" Body="&lt;p&gt;A neural network without a hidden layer is the same as just linear regression.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I then use squared hinge loss and encoporate the l2 regularisation term, is it fair to then call this network the same as a linear SVM?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going by this assumption, then if I need to implement a multiclass SVM, i can just have n output nodes (where n is the number of classes). Would this then be equivalent to having n number of SVMs, similar to a one-vs-rest method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I then wanted to encoporate a kernel into my SVM, could I then use an activation function or layer prior to the final output nodes (where I compute loss and add regularisation) which would then transfer this data into another feature plane the same as that of an SVM kernel? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my current hunch, but would like some confirmation or correction where my understanding is incorrect. &lt;/p&gt;&#xA;" OwnerUserId="29877" LastActivityDate="2021-03-10T15:56:25.953" Title="Is an SVM the same as a neural network without a hidden layer?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20700" PostTypeId="2" ParentId="20699" CreationDate="2020-04-27T21:02:34.380" Score="2" Body="&lt;p&gt;First, what makes the neural network different than linear regression is the non-linearity (activation function), not the number of layers. So, a neural network with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; layers with no non-linearities is still the same as linear regression. Second, SVM finds the hyperplane of maximum margin. You are not guaranteed to find that hyperplane that has maximum margin with a neural network (using non-linearities).&lt;/p&gt;&#xA;" OwnerUserId="36341" LastEditorUserId="2444" LastEditDate="2021-03-10T15:56:25.953" LastActivityDate="2021-03-10T15:56:25.953" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20701" PostTypeId="2" ParentId="8135" CreationDate="2020-04-27T21:09:41.740" Score="1" Body="&lt;p&gt;There are multiple python packages with inbuilt toy-datasets for testing purposes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scikit-learn.org/stable/datasets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sklearn.datasets&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mwaskom/seaborn-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;seaborn.load_dataset()&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.statsmodels.org/devel/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;statsmodels.api.datasets&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rpy2.github.io/doc/latest/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;rpy2&lt;/a&gt; (requires R and pandas)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/iamaziz/PyDataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyDataset&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23503" LastActivityDate="2020-04-27T21:09:41.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20702" PostTypeId="2" ParentId="17634" CreationDate="2020-04-27T21:43:06.830" Score="2" Body="&lt;p&gt;Some examples of dimensionality reduction techniques:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Linear methods&lt;/th&gt;&#xA;&lt;th&gt;Non-linear methods&lt;/th&gt;&#xA;&lt;th&gt;Graph-based methods&lt;br&gt;(&amp;quot;Network embedding&amp;quot;)&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;PCA&lt;br&gt; CCA&lt;br&gt; ICA&lt;br&gt; SVD&lt;br&gt; LDA&lt;br&gt; NMF&lt;/td&gt;&#xA;&lt;td&gt;Kernel PCA&lt;br&gt; GDA&lt;br&gt; Autoencoders&lt;br&gt; t-SNE&lt;br&gt; UMAP&lt;br&gt; MVU&lt;/td&gt;&#xA;&lt;td&gt;Diffusion maps&lt;br&gt; Graph Autoencoders&lt;br&gt;&lt;br&gt; Graph-based kernel PCA&lt;br&gt; (Isomap, LLE, Hessian LLE, Laplacian Eigenmaps)&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Though there are &lt;a href=&quot;https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;many more&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2021-02-16T13:54:04.610" LastActivityDate="2021-02-16T13:54:04.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20703" PostTypeId="1" AcceptedAnswerId="20704" CreationDate="2020-04-27T22:21:50.360" Score="4" ViewCount="726" Body="&lt;p&gt;People sometimes use 1st layer, 2nd layer to refer to a specific layer in a neural net. Is the layer immediately follows the input layer called 1st layer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about the lowest layer and highest layer?&lt;/p&gt;&#xA;" OwnerUserId="35896" LastEditorUserId="2444" LastEditDate="2020-04-27T23:29:45.410" LastActivityDate="2020-04-28T03:48:12.007" Title="Does the &quot;lowest layer&quot; refer to the first or last layer of the neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;terminology&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20704" PostTypeId="2" ParentId="20703" CreationDate="2020-04-27T23:25:03.697" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;People sometimes use 1st layer, 2nd layer to refer to a specific layer in a neural net. Is the layer immediately follows the input layer called 1st layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The 1st layer should typically refer to the layer that comes after the input layer. Similarly, the 2nd layer should refer to the layer that comes after the 1st layer, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, note that this convention and terminology may not be applicable in all cases. You should always take into account your context!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How about lowest layer and highest layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To be honest, I also dislike this ambiguous terminology. From my experience, I don't think there's an agreement on the actual meaning of &quot;lowest&quot; or &quot;highest&quot;. It depends on how you depict the neural network, but it's possible that &quot;lowest&quot; refers to the layers closer to the inputs, because, if you think of a neural network as a hierarchy that starts from the inputs and builds more complex representations of it, the &quot;lowest&quot; may refer to the &quot;lowest in the hierarchy&quot; (but who knows!).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-27T23:32:20.963" LastActivityDate="2020-04-27T23:32:20.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20706" PostTypeId="1" CreationDate="2020-04-28T01:04:52.687" Score="7" ViewCount="152" Body="&lt;p&gt;I understand the gist of what convolutional neural networks do and what they are used for, but I still wrestle a bit with how they function on a conceptual level. For example, I get that filters with kernel size greater than 1 are used as feature detectors, and that number of filters is equal to the number of output channels for a convolutional layer, and the number of features being detected scales with the number of filters/channels.&lt;/p&gt;&#xA;&lt;p&gt;However, recently, I've been encountering an increasing number of models that employ 1- or 2D convolutions with kernel sizes of 1 or 1x1, and I can't quite grasp why. It feels to me like they defeat the purpose of performing a convolution in the first place.&lt;/p&gt;&#xA;&lt;p&gt;What is the advantage of using such layers? Are they not just equivalent to multiplying each channel by a trainable, scalar value?&lt;/p&gt;&#xA;" OwnerUserId="36525" LastEditorUserId="2444" LastEditDate="2021-01-23T18:00:04.717" LastActivityDate="2021-02-22T20:05:15.450" Title="What is the point of using 1D and 2D convolutions with a kernel size of 1 and 1x1 respectively?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;filters&gt;&lt;convolutional-layers&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20708" PostTypeId="2" ParentId="20684" CreationDate="2020-04-28T03:19:23.590" Score="1" Body="&lt;p&gt;You can simply train a policy from the inputs to predict the actions in your dataset. You can use the cross entropy loss for this, i.e. maximize the the log probability that the policy assigns to the actions in the data set when given the corresponding inputs. This is called behavioral cloning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is an approximation of the behavioral policy that lets you compute probability densities of actions. It is an approximation because the dataset is finite, and even more so when you restrict the learned policy to a class of distributions, e.g. Gaussians.&lt;/p&gt;&#xA;" OwnerUserId="30627" LastActivityDate="2020-04-28T03:19:23.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20709" PostTypeId="2" ParentId="20564" CreationDate="2020-04-28T03:35:07.517" Score="0" Body="&lt;p&gt;It might be that the large labels dominate the loss value, so the model pays more attention to them. You could use an L1 loss rather than an L2 loss, such that the predictions get pulled towards each label equally rather than being pulled more strongly when the label is further away from the prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may also be a data imbalance, i.e. you train on more large labels than small labels. This would also cause the model to pay more attention to becoming good at predicting the large labels. If this is the case, you can either collect more small labels or train more often on the small labels you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is that your labels are not evenly distributed. For example, the gaps between large data points may be larger than between small data points, so the small data points all look similar to the model. You can plot a histogram of your labels to find out and then transform your labels (e.g. using log) to make space them out more evenly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general rules for training artificial neural networks apply. For example, normalize your inputs and outputs by subtracting the mean and divide them by their standard deviation, estimated across the training set. It also seem that your network is likely too deep and and not wide enough. I'd try 4x100 rather than 8x20 and add a small amount of weight decay when you see overfitting.&lt;/p&gt;&#xA;" OwnerUserId="30627" LastActivityDate="2020-04-28T03:35:07.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20710" PostTypeId="2" ParentId="20686" CreationDate="2020-04-28T03:46:00.763" Score="1" Body="&lt;p&gt;There is no benefit to using ReLU as the output activation of a neural network. As you said, the network will ignore training labels below zero and it will train on labels above zero as if no output activation were present. However, the problem you're describing can also occur for individual units of hidden layers, where ReLU activations are common. This is known as the dead ReLU problem. In practice, it is rarely a problem but it can be avoided with smooth rectifiers like ELU and Swish. Another interesting ideas is CRelu, which concatenates both positive and negative parts of the pre-activation, resulting in twice as many outputs, half of which always receive a non-zero gradient.&lt;/p&gt;&#xA;" OwnerUserId="30627" LastEditorUserId="30627" LastEditDate="2020-04-28T04:07:18.310" LastActivityDate="2020-04-28T04:07:18.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20711" PostTypeId="2" ParentId="20703" CreationDate="2020-04-28T03:48:12.007" Score="3" Body="&lt;p&gt;Lowest layer generally refers to the layer closest to the input. This comes from the idea that layers closer to the input represent low-level features such as gradients and edges, while layers closer to the output represent high-level features such as parts and objects.&lt;/p&gt;&#xA;" OwnerUserId="30627" LastActivityDate="2020-04-28T03:48:12.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20712" PostTypeId="2" ParentId="11285" CreationDate="2020-04-28T04:00:17.353" Score="0" Body="&lt;p&gt;The words latent space and embedding space are often used interchangeably. However, latent space can more specifically refer to the sample space of a stochastic representation, whereas embedding space more often refers to the space of a deterministic representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This comes from latent referring to an unobserved random variable, for which we can infer a belief distribution over its plausible values, for example using an encoder network. You can then draw samples of the predicted distribution for further processing. To learn more, you can look into VAEs.&lt;/p&gt;&#xA;" OwnerUserId="30627" LastActivityDate="2020-04-28T04:00:17.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20713" PostTypeId="2" ParentId="20675" CreationDate="2020-04-28T04:05:37.513" Score="0" Body="&lt;p&gt;No, the activation of the output layer should instead be tailored to the labels you're trying to predict. The network prediction can be seen as a distribution, for example a categorical for classification or a Gaussian (or something more flexible) for regression. The output of your network should predict the sufficient statistics of this distribution. For example, a softmax activation on the last layer ensures that the outputs are positive and sum up to one, as you would expect for a categorical distribution. When you predict a Gaussian with mean and variance, you don't need an activation for the mean but the variance has to be positive, so you could use exp as activation for that part of the output.&lt;/p&gt;&#xA;" OwnerUserId="30627" LastActivityDate="2020-04-28T04:05:37.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20714" PostTypeId="2" ParentId="20680" CreationDate="2020-04-28T04:58:36.503" Score="1" Body="&lt;p&gt;Speaking &lt;em&gt;very&lt;/em&gt; generally, I would say that with the current state of machine learning, a &quot;more complicated&quot; task requires &lt;em&gt;more trainable parameters&lt;/em&gt;. You can increase parameter count by either increasing width and also by increasing depth. Again, speaking &lt;em&gt;very generally&lt;/em&gt;, I would say that in practice, people have found more success by increasing depth than by increasing width.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this depends a lot on what you mean by &quot;more complicated&quot;. I would argue that generating something is a fundamentally more complicated problem than just identifying something. However, a GAN to generate a 4-pixel image will probably be far more shallow than the shallowest ImageNet network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could also make an argument that the &lt;em&gt;definition&lt;/em&gt; of complexity of a deep learning task is &quot;more layers needed == more complicated&quot;, in which case it's obvious that by definition, a more complicated task requires a deeper net.&lt;/p&gt;&#xA;" OwnerUserId="27092" LastEditorUserId="27092" LastEditDate="2020-04-28T05:05:21.250" LastActivityDate="2020-04-28T05:05:21.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20715" PostTypeId="1" CreationDate="2020-04-28T06:05:02.807" Score="1" ViewCount="25" Body="&lt;p&gt;There is a theorem that states that basically a neural network can approximate any function whatsoever. However, this does not mean that it can solve any equation. I have some notes where it states that backpropagation allows us to solve problems of the following kind &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ F(x_i, t) = y_i $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me to what exactly this means?&lt;/p&gt;&#xA;" OwnerUserId="36173" LastActivityDate="2020-04-28T06:05:02.807" Title="Class of functional equations that backpropagation can solve" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20717" PostTypeId="1" AcceptedAnswerId="20718" CreationDate="2020-04-28T06:40:59.453" Score="1" ViewCount="32" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1512.02325&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSD: Single Shot MultiBox Detector&lt;/a&gt;, under section 2.2 - (4), why do we add an offset of &lt;strong&gt;0.5&lt;/strong&gt; to x, y in generating the anchor boxes across feature maps? &lt;/p&gt;&#xA;" OwnerUserId="36534" LastEditorUserId="2444" LastEditDate="2020-04-28T14:21:10.907" LastActivityDate="2020-04-28T14:21:10.907" Title="Why do we set offset (0.5) in single shot detector?" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20718" PostTypeId="2" ParentId="20717" CreationDate="2020-04-28T08:46:18.707" Score="0" Body="&lt;p&gt;The integer coordinates refer to the top-left corner of a pixel. Adding 0.5 to both x and y gives you the center location of a pixel instead.&lt;/p&gt;&#xA;" OwnerUserId="22086" LastActivityDate="2020-04-28T08:46:18.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20720" PostTypeId="2" ParentId="20682" CreationDate="2020-04-28T09:16:08.597" Score="0" Body="&lt;p&gt;The tendency in literature in the last years (at least for computer vision problems) seems to point towards the single model option (I'll try to remember to come back and add some links to papers mentioning this when I find them), although this IMO is really data- and problem-dependent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, I would set up a network for the mapping &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt;, with a training-only auxiliary loss calculated on the mapping &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$f1(x)$&lt;/span&gt; and compare this with a model trained only on &quot;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="22086" LastActivityDate="2020-04-28T09:16:08.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20721" PostTypeId="2" ParentId="16874" CreationDate="2020-04-28T09:44:50.560" Score="1" Body="&lt;p&gt;From my perspective you should look at the concept of &lt;strong&gt;ontologies&lt;/strong&gt;, which might briefly be described as a set of axioms that formalize concepts such as &lt;code&gt;{Grass, Water, Green}&lt;/code&gt; and relations between those like &lt;code&gt;hasProperty(Grass, Green)&lt;/code&gt; and &lt;code&gt;needs(Grass, Water)&lt;/code&gt;. To describe such kind of knowledge the &lt;a href=&quot;https://en.wikipedia.org/wiki/Web_Ontology_Language&quot; rel=&quot;nofollow noreferrer&quot;&gt;Web Ontology Language&lt;/a&gt; was created. The theoretical framework on which it is built are different flavors of &lt;em&gt;description logics&lt;/em&gt;, which all are fragments of first order predicate logic, but come with different tradeoffs  between expressiveness and computational complexity for &lt;em&gt;automatic reasoning&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with other AI-topics this kind of stuff can get quite involved ‒ and interesting. I can recommend the open textbook: &lt;a href=&quot;https://people.cs.uct.ac.za/~mkeet/OEbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;An introduction to ontology engineering&lt;/a&gt; by Maria Keet (University of Cape Town).&lt;/p&gt;&#xA;" OwnerUserId="33714" LastActivityDate="2020-04-28T09:44:50.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20722" PostTypeId="1" CreationDate="2020-04-28T10:54:04.127" Score="1" ViewCount="25" Body="&lt;p&gt;Given some natural language sentences like&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I would like to talk to Mr. Smith&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would like to extract entities, like the person &amp;quot;Smith&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I know that frameworks, which are capable of doing so (f. e. RASA or spaCy), exist, but I would like to dive deeper and understand the theory behind all this.&lt;/p&gt;&#xA;&lt;p&gt;At university, I learned a few of the basic models like CRF's or SVM's used for this task, but I wonder if there are any good resources (preferably books) about this topic.&lt;/p&gt;&#xA;" OwnerUserId="32819" LastEditorUserId="2444" LastEditDate="2021-02-03T16:43:46.837" LastActivityDate="2021-02-03T16:43:46.837" Title="Are there any good resources (preferably books) about techniques used for entity extraction?" Tags="&lt;natural-language-processing&gt;&lt;resource-request&gt;&lt;books&gt;&lt;named-entity-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20724" PostTypeId="2" ParentId="8135" CreationDate="2020-04-28T11:08:29.890" Score="1" Body="&lt;p&gt;TensorFlow has a nice datasets package, which meets the requirements stated above.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tensorflow.org/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation link&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pypi.org/project/tensorflow-datasets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyPI link&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12345" LastActivityDate="2020-04-28T11:08:29.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20726" PostTypeId="1" CreationDate="2020-04-28T11:26:58.853" Score="1" ViewCount="48" Body="&lt;p&gt;I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us say I have a batch size of 2, to make it simple; I pass my input in the network and my y_pred would be a 2x3 tensor.&#xA;The same happens for y_true, 2x3 itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the thing I'm not sure of: I take first the difference, &lt;code&gt;diff = y_true - y_pred&lt;/code&gt;; this maintains the dimension.&#xA;Now, for MSE, I square diff, obtaining again a 2x3 tensor, is that right ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the tricky part (for me): I have to Mean. Which one should I consider:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mean all the (six) values obtaining thus a scalar ?&lt;/strong&gt; But in this case I do not understand how the backpropagation would make better on specific targets &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mean by rows, i.e, obtaining a 2x1 tensor, so that I have a mean for each example ?&lt;/strong&gt; Also here I cannot get how the optimization would work&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mean by columns, i.e, obtaining a 1x3 tensors, where I obtain thus an &quot;error&quot; for each target?&lt;/strong&gt; This seems the more logical to me, but I'm not so sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is clear &lt;/p&gt;&#xA;" OwnerUserId="36504" LastEditorUserId="36504" LastEditDate="2020-04-29T09:40:02.557" LastActivityDate="2020-04-29T09:40:02.557" Title="How MSE should be appliead with multi target deep network?" Tags="&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;regression&gt;&lt;loss&gt;&lt;mean-squared-error&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20728" PostTypeId="1" AcceptedAnswerId="20915" CreationDate="2020-04-28T12:58:54.420" Score="2" ViewCount="110" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In the standard Markov Decision Process (MDP) formalization of the reinforcement-learning (RL) problem (Sutton &amp;amp; Barto, 1998), a decision maker interacts with an environment consisting of &lt;strong&gt;finite state and action spaces&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is an extract from &lt;a href=&quot;http://carlosdiuk.github.io/papers/OORL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, although it has nothing to do with the paper's content per se (just a small part of the introduction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please explain why it makes sense to study finite state and action spaces? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the real world, we might not be able to restrict ourselves to a finite number of states and actions! Thinking of humans as RL agents, this really doesn't make sense.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-04-28T13:12:39.060" LastActivityDate="2020-05-05T21:07:02.903" Title="Why does it make sense to study MDPs with finite state and action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20729" PostTypeId="1" CreationDate="2020-04-28T13:23:04.370" Score="1" ViewCount="17" Body="&lt;p&gt;There are 11 objects of which 4 are &quot;Bad&quot; objects. So there are 7 &quot;Good&quot; objects.&#xA;You have to choose as many Good objects before proceeding to another set of objects of a different sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you train an AI that predicts the position of the Good objects based of previous sets of data?&lt;/p&gt;&#xA;" OwnerUserId="36548" LastActivityDate="2020-04-28T13:23:04.370" Title="Using AI to find the correct set of object/numbers based on previous data" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20730" PostTypeId="1" CreationDate="2020-04-28T13:43:49.627" Score="2" ViewCount="16" Body="&lt;p&gt;I suppose that picking an appropriate size for the bottleneck in Autoencoders is neither a trivial nor an intuitive task. After watching this &lt;a href=&quot;https://www.youtube.com/watch?v=9zKuYvjFFS8&quot; rel=&quot;nofollow noreferrer&quot;&gt;video about VAEs&lt;/a&gt;, I've been wondering: Do disentangled VAEs solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After all, if the network is trained to use as few latent space variables as possible, I might as well make the bottleneck large enough so that I don't run into any issues during training. Am I wrong somewhere?&lt;/p&gt;&#xA;" OwnerUserId="36550" LastEditorUserId="1671" LastEditDate="2020-04-28T21:50:27.593" LastActivityDate="2020-04-28T21:50:27.593" Title="Does bottleneck size matter in Disentangled Variational Autoencoders?" Tags="&lt;variational-autoencoder&gt;&lt;latent-variable&gt;&lt;bottlenecks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20731" PostTypeId="1" CreationDate="2020-04-28T14:39:44.580" Score="2" ViewCount="85" Body="&lt;p&gt;In RL (reinforcement learning) or MARL (multi-agent reinforcement learning), we have the usual tuple:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(state, action, transition_probabilities, reward, next_state)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In MORL (multi-objective reinforcement learning), we have two more additions to the tuple, namely, &quot;preferences&quot; and &quot;preference functions&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are they? What do we do with them? Can someone provide an intuitive example? &lt;/p&gt;&#xA;" OwnerUserId="25299" LastEditorUserId="2444" LastEditDate="2020-10-08T14:16:35.867" LastActivityDate="2020-10-08T14:16:35.867" Title="What are preferences and preference functions in multi-objective reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;multi-objective-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20732" PostTypeId="1" CreationDate="2020-04-28T14:42:14.507" Score="2" ViewCount="54" Body="&lt;p&gt;I've been reading the &lt;a href=&quot;http://carlosdiuk.github.io/papers/OORL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;attached paper&lt;/a&gt; - which aims to model entities in the world as objects, including the learning agent itself! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To say the least, the goal is to navigate through what seems like a maze (path-planning problem) - and drop off passengers in desired destinations, while avoiding walls in the map of the world (5x5 grid for now). The objects involved are, a taxi, passengers, walls and a destination. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, a particular paragraph says the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Whereas in the classical MDP&#xA;  model, the effect of encountering walls is felt as a property of specific locations in the grid, the OO-MDP view is that wall interactions are the same regardless of their location. As such, agents’ experience can transfer gracefully throughout the state space.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does this mean? &lt;strong&gt;How are the classical MDP and the object-oriented MDP views different?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't make sense of the above extract, at all. Any help would be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I did not consider posting parts of the extract as separate questions since my problem has more to do with understanding the extract as a whole which inevitably relies on understanding the parts. &lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-04-28T15:06:58.807" LastActivityDate="2020-04-28T19:08:07.257" Title="How are the classical MDP and the object-oriented MDP views different?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;markov-decision-process&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20733" PostTypeId="1" AcceptedAnswerId="20736" CreationDate="2020-04-28T14:43:15.900" Score="0" ViewCount="99" Body="&lt;p&gt;Let's say we have MDP where we have a state transition matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this state transition different from action value in reinforcement learning? Is the state transition in MDP stochastic transition, meaning transition to some other state without taking any action?&lt;/p&gt;&#xA;" OwnerUserId="36047" LastEditorUserId="2444" LastEditDate="2020-04-28T15:03:46.360" LastActivityDate="2020-04-28T15:48:46.520" Title="What is the difference between the state transition of an MDP and an action-value?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;markov-decision-process&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20735" PostTypeId="1" CreationDate="2020-04-28T15:03:52.557" Score="2" ViewCount="74" Body="&lt;p&gt;I'm trying to create a text recognition project using CNN. I need help regarding the text detection task. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the training images and bounding box details for them. But I'm unable to figure out how to create the loss function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me by telling how to take the output from the CNN model and compare it to the bounding box labels?&lt;/p&gt;&#xA;" OwnerUserId="36538" LastEditorUserId="2444" LastEditDate="2020-04-28T15:50:35.197" LastActivityDate="2021-04-27T03:10:36.143" Title="How should I define the loss function for a multi-object detection problem?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20736" PostTypeId="2" ParentId="20733" CreationDate="2020-04-28T15:48:46.520" Score="0" Body="&lt;p&gt;&lt;strong&gt;Transition Probabilities:&lt;/strong&gt; Consider that you are at state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and from that state take an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.Then there are some probability you will land up at state &lt;span class=&quot;math-container&quot;&gt;$s_{1}'$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$s_{2}'$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; indicate the next states). Those probabilities are called transition probabilities. In this example, the transition matrix is just a 3D array since it depends on your state and action(&lt;span class=&quot;math-container&quot;&gt;$p(s, a)$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Action value function &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(s, a)$&lt;/span&gt;:&lt;/strong&gt; It is the expected total reward you get from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and thereafter following the policy, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the state transition in MDP stochastic transition, meaning transition to some other state without taking any action?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The environment can be stochastic or deterministic. If the environment is stochastic then those transitions are stochastic. If the environment is deterministic then those transitions are deterministic.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-04-28T15:48:46.520" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20737" PostTypeId="2" ParentId="20676" CreationDate="2020-04-28T16:18:51.220" Score="0" Body="&lt;p&gt;Answer to my question is that values obtained from convolutions among different channels &lt;strong&gt;sum up&lt;/strong&gt; together, therefore 3 channels after convolution with one filter give one output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best explanation delivered by Andrew: &lt;a href=&quot;https://www.coursera.org/lecture/convolutional-neural-networks/convolutions-over-volume-ctQZz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.coursera.org/lecture/convolutional-neural-networks/convolutions-over-volume-ctQZz&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2020-04-28T16:18:51.220" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20738" PostTypeId="1" CreationDate="2020-04-28T18:04:46.480" Score="1" ViewCount="24" Body="&lt;p&gt;I have customer review texts. The data consists of the the raw and manually corrected texts of the reviews. I have aligned these pairs by using similarity algorithms and matched the words on them. Since there are some mis-matched words pairs, I have eliminated the pairs under a threshold value for their counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there are raw and corrected word pairs. What kind of machine learning model can I build for spellcheck by using the data mentioned as well?&lt;/p&gt;&#xA;" OwnerUserId="36188" LastActivityDate="2020-04-28T18:04:46.480" Title="Building a spell check model" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20739" PostTypeId="1" CreationDate="2020-04-28T18:20:08.313" Score="2" ViewCount="47" Body="&lt;p&gt;I would like to classify a dataset &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Credit Scoring&lt;/a&gt;, which is composed of 21 attributes, some of them are numeric and others are boolean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the output, I want to know if they have a good or bad credit based on those attributes, without calculating any numeric value for the credit score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Weka for this task. However, I am not sure what are the best/ideal classifiers for that kind of datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone here can put me in the right direction?&lt;/p&gt;&#xA;" OwnerUserId="36532" LastEditorUserId="2444" LastEditDate="2020-04-28T18:38:52.350" LastActivityDate="2020-04-28T18:53:37.183" Title="What are the best classifiers for this type of data?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20740" PostTypeId="2" ParentId="20739" CreationDate="2020-04-28T18:53:37.183" Score="1" Body="&lt;p&gt;Well, it depends on the structure of the data. The best way is to try all the intelligent models like naive bayes, random forest, svm with different parameters by grid search. There is no model works best all the time for classification. However,  neural network (named Multilayer Perceptron on weka) is supposed to be better if it is set correctly.&lt;/p&gt;&#xA;" OwnerUserId="36188" LastActivityDate="2020-04-28T18:53:37.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20744" PostTypeId="5" CreationDate="2020-04-28T21:04:10.017" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-28T21:04:10.017" LastActivityDate="2020-04-28T21:04:10.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20745" PostTypeId="4" CreationDate="2020-04-28T21:04:10.017" Score="0" Body="For questions related to stochastic gradient descent (SGD), which is stochastic gradient descent that uses stochastic (or noisy) gradients." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-28T21:04:10.017" LastActivityDate="2020-04-28T21:04:10.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20746" PostTypeId="5" CreationDate="2020-04-28T21:54:03.497" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The bottleneck in a neural network is just a layer with less neurons then the layer below or above it. Having such a layer encourages the network to compress feature representations to best fit in the available space, in order to get the best loss during training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;Source: &lt;a href=&quot;https://ai.stackexchange.com/questions/4864/what-is-the-concept-of-tensorflow-bottlenecks&quot;&gt;What is the concept of Tensorflow Bottlenecks? (SE:AI)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-04-28T21:54:03.497" LastActivityDate="2020-04-28T21:54:03.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20747" PostTypeId="4" CreationDate="2020-04-28T21:54:03.497" Score="0" Body="For questions about &quot;bottlenecks&quot; in dimensional reduction, typically in reference to Artificial Neural Networks and Autoencoders." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-04-28T22:24:00.773" LastActivityDate="2020-04-28T22:24:00.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20748" PostTypeId="1" CreationDate="2020-04-29T02:08:34.137" Score="2" ViewCount="34" Body="&lt;p&gt;A big class of problems that are relevant in today's society are full of uncertainty and are also sometimes computationally intractable. Along our lives we come to realize that we are solving the same type of problem multiple times, sometimes with different strategies and mixed results. I would like to close in on three main problem &lt;em&gt;types&lt;/em&gt;: pattern recognition, regression and density estimation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An agent (computer program or even a human) that identifies the &lt;em&gt;type&lt;/em&gt; of a problem and applies a &lt;em&gt;systematic procedure&lt;/em&gt; for finding its solution. A solution is understood in the classical sense for each of the problem types, thus, the solution does not have to be a global optima. This procedure &lt;strong&gt;must&lt;/strong&gt; be implementable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus points&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Uses metadata about the problem itself to 'gain insight' about the nature of the problem.&lt;/li&gt;&#xA;&lt;li&gt;Verifies that its solution is &lt;em&gt;correct&lt;/em&gt; in some sense. &lt;/li&gt;&#xA;&lt;li&gt;The types or classes of problems can be expanded later on.&lt;/li&gt;&#xA;&lt;li&gt;Works with very limited resources.&lt;/li&gt;&#xA;&lt;li&gt;Works with little information about the problem, or with &quot;small data&quot;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So far I've found Statistical Learning theory and Bayesian Inference as candidates that implement some of those ideas, but I was wondering if there's something else out there or I just need to take the best of both of those worlds.&lt;/p&gt;&#xA;" OwnerUserId="36173" LastActivityDate="2020-04-29T02:08:34.137" Title="Is there a theory that captures the following ideas?" Tags="&lt;theory&gt;&lt;models&gt;&lt;learning-algorithms&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20749" PostTypeId="1" AcceptedAnswerId="20753" CreationDate="2020-04-29T03:06:50.460" Score="1" ViewCount="51" Body="&lt;p&gt;My question is actually related to the addition of probabilities. I am reading on computational learning theory from &lt;a href=&quot;http://profsite.um.ac.ir/~monsefi/machine-learning/pdf/Machine-Learning-Tom-Mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tom Mitchell's machine learning book&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In chapter 7, when proving the upper bound of probabilities for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; exhausted version space (theorem 7.1), it says that the probability that at least one hypothesis out of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; hypotheses in the hypotheses space &lt;span class=&quot;math-container&quot;&gt;$|H|$&lt;/span&gt; being consistent with &lt;em&gt;m&lt;/em&gt; training examples is at most &lt;span class=&quot;math-container&quot;&gt;$k(1- \epsilon)^m$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the probability of a hypothesis, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, consistent with &lt;em&gt;m&lt;/em&gt; training examples is &lt;span class=&quot;math-container&quot;&gt;$(1-\epsilon)^m$&lt;/span&gt;. However, why is it possible to add the probabilities for &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; hypotheses? And might the probability be greater than 1 in this case?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-04-29T04:06:52.070" LastActivityDate="2020-04-29T23:17:25.600" Title="Why is probability that at least one hypothesis out of $k$ being consistent with $m$ training examples $k(1- \epsilon)^m$?" Tags="&lt;machine-learning&gt;&lt;proofs&gt;&lt;probability&gt;&lt;computational-learning-theory&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20750" PostTypeId="1" CreationDate="2020-04-29T03:41:46.813" Score="2" ViewCount="41" Body="&lt;p&gt;I have a mix of two deep models, as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if model A is YES --pass to B--&amp;gt; if model B is YES--&amp;gt; result = YES&#xA;if model A is NO ---&amp;gt; result = NO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So basically model B validates if A is saying &lt;code&gt;YES&lt;/code&gt;. My models are actually the same, but trained on two different feature sets of same inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is this mix called in machine learning terminology? I just call them master/slave architecture, or primary/secondary model.&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2020-05-01T18:37:43.703" LastActivityDate="2020-05-01T18:37:43.703" Title="How is an architecture composed of a second model that validates the first one called in machine learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;ensemble-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20751" PostTypeId="1" CreationDate="2020-04-29T03:43:03.337" Score="0" ViewCount="27" Body="&lt;p&gt;I have a data set of 1600 examples. I am using 1280 (80%) for training, 160 (10%) for testing, and 160 (10%) for validation. The training goes one of two ways no matter how I fine-tune the L2 parameter:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The validation and training error converge, albeit around 75% error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) The training error settles to around 0%, but the validation error stays around 75%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think my network is too large either. I have trained networks with two hidden layers, both with the same number of nodes as the input. I also tried dropout layers and that did not seem to help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this just mean that I need to add more training examples? Or how do I know that I have reached the limitations of what I am having the network learn?&lt;/p&gt;&#xA;" OwnerUserId="14811" LastActivityDate="2020-04-29T03:43:03.337" Title="Cannot fine-tune L2-regularization parameter" Tags="&lt;regularization&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20753" PostTypeId="2" ParentId="20749" CreationDate="2020-04-29T04:26:42.723" Score="1" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; be two events. In general, the probability that either &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; occurs is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are disjoint, i.e. they cannot happen at the same time, then &lt;span class=&quot;math-container&quot;&gt;$P(A \text{ and } B) = 0$&lt;/span&gt;, so the above formula becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(A \text{ or } B) = P(A) + P(B)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the probability of one &lt;strong&gt;arbitrary&lt;/strong&gt; hypothesis being consistent with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; training examples is &lt;span class=&quot;math-container&quot;&gt;$(1-\epsilon)^m$&lt;/span&gt;, then, given the rule above and assuming that only one hypothesis is consistent with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; training examples, the probability of &lt;strong&gt;one or more&lt;/strong&gt; (i.e. at least one) of the hypotheses being consistent with training examples is the sum of the probabilities, i.e. &lt;span class=&quot;math-container&quot;&gt;$k (1-\epsilon)^m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This probability can be bigger than one if more than one hypothesis is consistent with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; training examples. In that case, you have to take into account the probability that both hypotheses are consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. notes &lt;a href=&quot;https://faculty.math.illinois.edu/~hildebr/408/408general1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Probability, I: Rules of probability&lt;/a&gt; for more details about the union rule and other rules of probability.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-29T23:17:25.600" LastActivityDate="2020-04-29T23:17:25.600" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20754" PostTypeId="2" ParentId="20731" CreationDate="2020-04-29T05:50:34.613" Score="3" Body="&lt;p&gt;In MORL the reward component is a &lt;em&gt;vector&lt;/em&gt; rather than a scalar, with an element for each objective. So if we are using a multiobjective version of an algorithm like Q-learning, the Q-values stored for each state-action pair will also be vectors.&lt;/p&gt;&#xA;&lt;p&gt;Q-learning requires the agent to be able to identify the greedy action in any state (the action expected to lead to the highest long-term return). For scalar rewards this is easy, but for vector values it is more complicated as one vector may be higher for objective 1, while another is higher for objective 2, and so on.&lt;/p&gt;&#xA;&lt;p&gt;We need a means to &lt;em&gt;order&lt;/em&gt; the vector values in terms of how well they meet the user's desired trade-offs between the different objectives. That is the role of the &lt;em&gt;preference function&lt;/em&gt; and &lt;em&gt;preferences&lt;/em&gt;. The function defines a general operation for either converting the vector values to a scalar value so they can be compared, or for performing some sort of ordering of the vectors (some types of orderings such as lexicographic ordering can't readily be defined in terms of scalarisation). So, for example, our preference function might be a weighted sum of the components of the vector. The preferences specify the parameters of the preference function which define a specific ordering (i.e. based on the needs of the current user). So, in the case of a weighted sum for the preference function, the preferences would be specified in terms of the values of the weights.&lt;/p&gt;&#xA;&lt;p&gt;The choice of preference function can have implications for the types of solutions which can be found, or for whether additional information needs to be included in the state in order to ensure convergence.&lt;/p&gt;&#xA;&lt;p&gt;I'd suggest you read the following survey paper for an overview of MORL (disclaimer - I was a co-author on this, but I genuinely think it is a useful introduction to this area)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Roijers, D. M., Vamplew, P., Whiteson, S., &amp;amp; Dazeley, R. (2013). &lt;a href=&quot;https://www.jair.org/index.php/jair/article/download/10836/25862&quot; rel=&quot;nofollow noreferrer&quot;&gt;A survey of multi-objective sequential decision-making&lt;/a&gt;. Journal of Artificial Intelligence Research, 48, 67-113.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36575" LastEditorUserId="2444" LastEditDate="2020-10-08T14:13:29.233" LastActivityDate="2020-10-08T14:13:29.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20756" PostTypeId="2" ParentId="20555" CreationDate="2020-04-29T09:38:46.967" Score="1" Body="&lt;p&gt;It depends on whether the action is part of the input or output of a neural network estimating the Q-value(state, action).&lt;a href=&quot;https://i.stack.imgur.com/csAky.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/csAky.png&quot; alt=&quot;alternative architectures for Q-function neural network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network on the left has the state as input and outputs one scalar value for each of the categorical actions. It has the advantage of being easy to setup and only needs one network evaluation to predict the Q-value for all actions. If the action space is categorical and single-dimensional I would use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network on the right has both the state and a representation of the action as input and outputs one single scalar value. This architecture also allows to compute the Q-value for multi-dimensional and continuous action spaces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The action space of tic-tac-toe can be easily represented by a vector of length 9, thus I would recommend the left NN-architecture. However, if your game has continuous-valued variables in the action space (e.g. the position of your mouse pointer), you should use the NN-architecture on the right.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach to prevent illegal moves is only partially dependent on the choice of the Q-function architecture and covered by another question: &lt;a href=&quot;https://datascience.stackexchange.com/questions/37519/rl-agent-how-to-forbid-actions&quot;&gt;How to forbid actions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36560" LastEditorUserId="36560" LastEditDate="2020-04-30T18:18:31.040" LastActivityDate="2020-04-30T18:18:31.040" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20758" PostTypeId="1" CreationDate="2020-04-29T11:16:01.240" Score="2" ViewCount="329" Body="&lt;p&gt;I'm interested about using Reinforcement Learning in a setting that might seem more suitable for Supervised Learning. There's a dataset &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and for each sample &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; some decision needs to be made. Supervised Learning can't be used since there aren't any algorithms to solve or approximate the problem (so I can't solve it on the dataset) but for a given decision it's very easy to decide how good it is (define a reward). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you can think about the knapsack problem - let's say we have a dataset where each sample &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a list (of let's say size 5) of objects each associated with a weight and a value and we want to decide which objects to choose (of course you can solve the knapsack problem for lists of size 5, but let's imagine that you can't). For each solution the reward is the value of the chosen objects (and if the weight exceeds the allowed weight then the reward is 0 or something). So, we let an agent &quot;play&quot; with each sample &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; times, where play just means choosing some subset and training with the given value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th sample the step can be adjusted to be:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\theta = \theta + \alpha \nabla_{\theta}log \pi_{\theta}(a|x^i)v$$&lt;/span&gt; &#xA;for each &quot;game&quot; with &quot;action&quot; &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and value &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead of the original step:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\theta = \theta + \alpha \nabla_{\theta}log \pi_{\theta}(a_t|s_t)v_t$$&lt;/span&gt;&#xA;Essentially, we replace the state with the sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue with this is that REINFORCE assumes that an action also leads to some new state where here it is not the case. Anyway, do you think something like this could work?&lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="36083" LastEditDate="2020-04-29T13:54:32.713" LastActivityDate="2020-04-29T14:57:07.447" Title="Reinforcement Learning (and specifically REINFORCE algorithm) for one-round &quot;games&quot;" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20759" PostTypeId="2" ParentId="18654" CreationDate="2020-04-29T11:29:34.627" Score="0" Body="&lt;p&gt;Since no one answers, I will say the approach I used.&#xA;It was inspired by the successive halving algorithm. The concept is that we find good performing hyperparameters for one agent by playing manually against it. Then, we put the agent we want to tune to play against it. We assign different configurations. All of these configurations are tried for a predefined number of games per configuration. We keep the best performing half, and then go to the next round with the half we kept and repeat this process. The last configuration that remains is the best performing configuration.&lt;/p&gt;&#xA;" OwnerUserId="33307" LastActivityDate="2020-04-29T11:29:34.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20760" PostTypeId="1" CreationDate="2020-04-29T11:57:05.000" Score="1" ViewCount="37" Body="&lt;p&gt;Many Q-learning techniques have been developed to capture discrete state(observation), actions like a robot in a grid world, and even continuous (state or action) spaces. But I am wondering how we can model the states/space in a time-dependent environment. Please, consider the following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one smartphone (client) and five compute servers that are addressing/serving many clients (smartphones) at the same time. The smartphone transfers some raw data (e.g, sensor data) to one of those five servers (e.g., every t seconds) and gets the results. Suppose the server computes the stress-level of the client in real-time based on the collected data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, a q-learning agent should be deployed to the smartphone to be able to select the best server with minimum response time (i.e., the goal is to minimize the execution/response time). Note that servers are serving different clients and their load is a function of time and varies from time to time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the above scenario, I am wondering what would be our &quot;states&quot; and how we can model the &quot;environment&quot;? &lt;/p&gt;&#xA;" OwnerUserId="36581" LastActivityDate="2020-04-29T11:57:05.000" Title="Is Q-Learning suitable for time-dependent spaces?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20761" PostTypeId="1" CreationDate="2020-04-29T12:12:28.140" Score="1" ViewCount="87" Body="&lt;p&gt;What does the term &lt;strong&gt;&quot;easy negatives&quot;&lt;/strong&gt; exactly mean in the context of machine learning for a classification problem or any problem in general? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a quick google search, I think it means just negative examples in the training set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please elaborate a bit more on why the term &quot;easy&quot; is brought into the picture? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attached picture with black underline:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oLl7m.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oLl7m.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36582" LastEditorUserId="36582" LastEditDate="2020-04-29T13:42:56.433" LastActivityDate="2021-05-25T09:03:55.363" Title="What is the meaning of &quot;easy negatives&quot; in the context of machine learning?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20762" PostTypeId="2" ParentId="20758" CreationDate="2020-04-29T12:15:41.957" Score="1" Body="&lt;p&gt;This seems like a multi-armed bandit problem (no states involved here). I had the same problem some times ago and I was advised to sample the output distribution M times, calculate the rewards and then feed them to the agent, this was also explained in &lt;a href=&quot;https://arxiv.org/pdf/1801.07365.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; Algorithm 1 page 3 (but different problem &amp;amp; different context). I honestly don't know if this will work for your case. You could also take a look at &lt;a href=&quot;https://github.com/shshemi/NeuralKnapsack/blob/master/neural_knapsack.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36461" LastActivityDate="2020-04-29T12:15:41.957" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20763" PostTypeId="2" ParentId="20758" CreationDate="2020-04-29T12:21:12.647" Score="0" Body="&lt;p&gt;You should look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit#Contextual_bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;contextual bandits&lt;/a&gt;, and specifically &lt;a href=&quot;https://towardsdatascience.com/13-solutions-to-multi-arm-bandit-problem-for-non-mathematicians-1b88b4c0b3fc&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient bandit solvers&lt;/a&gt; (see section 13).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your derivation of the gradient seems correct to me. Instead of a sampled/bootstrapped value function (as in Actor-Critic) or sampled full return (in REINFORCE) you can use the sampled reward. You will probably want to subtract a baseline from &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, e.g. a rolling average reward for the current policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have &lt;a href=&quot;https://www.kaggle.com/slobo777/pytorch-giant-gradient-bandit&quot; rel=&quot;nofollow noreferrer&quot;&gt;successfully used a gradient bandit solver for one-shot optimisation problem&lt;/a&gt; with 5000 dimension actions. It was not as strong as a custom optimiser or SAT solver, but whether or not that is an issue for you will depend on the problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-04-29T12:37:55.323" LastActivityDate="2020-04-29T12:37:55.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20765" PostTypeId="2" ParentId="20758" CreationDate="2020-04-29T12:40:41.360" Score="0" Body="&lt;p&gt;Besides contextual Bandit or multi-armed Bandit perspective, if you want to use a dataset to train RL policy, I would recommend you &lt;a href=&quot;http://tgabel.de/cms/fileadmin/user_upload/documents/Lange_Gabel_EtAl_RL-Book-12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Batch RL&lt;/a&gt;, it is another RL working in a supervised learning way to train a policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your problem, I think you can still use one-state trajectories to train REINFORCE. For example, there is a trajectory, &lt;span class=&quot;math-container&quot;&gt;$\tau={(s, a, r, s^{\prime})}$&lt;/span&gt;, there &lt;span class=&quot;math-container&quot;&gt;$s^{\prime}$&lt;/span&gt; is NULL. By using REINFORCE, you can get the gradient &lt;span class=&quot;math-container&quot;&gt;$\theta = \theta + \alpha \nabla_{\theta}log \pi_{\theta}(a|s)r$&lt;/span&gt;, and you do not need &lt;span class=&quot;math-container&quot;&gt;$s^{\prime}$&lt;/span&gt; here.&lt;/p&gt;&#xA;" OwnerUserId="8415" LastEditorUserId="8415" LastEditDate="2020-04-29T12:46:19.693" LastActivityDate="2020-04-29T12:46:19.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20767" PostTypeId="2" ParentId="20750" CreationDate="2020-04-29T13:12:16.950" Score="1" Body="&lt;p&gt;Not in terms of models, but there is a terminology called &lt;strong&gt;'Hierarchical learning'&lt;/strong&gt;, wherein if your model has a task to classify disease, then, If it detects a presence of a disease (disease/ no disease), then it proceeds to further classify a disease(class A/B/C/...). Else it does not proceed.&#xA;This technique of hierarchical learning is very common amongst supervised learning tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now according to your question, you have two models and I assume that they have different tasks and provide a binary outcome(yes/no). Here, you can call it as &lt;strong&gt;'Multitask learning'&lt;/strong&gt;, where the output of task1 is given to task2 for processing.&#xA;If task1 detect the presence of disease, then task2 classifies disease into various classes / or segment it / localize it etc.&lt;/p&gt;&#xA;" OwnerUserId="35791" LastActivityDate="2020-04-29T13:12:16.950" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20769" PostTypeId="1" AcceptedAnswerId="20872" CreationDate="2020-04-29T13:57:32.883" Score="0" ViewCount="99" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;...for learning transition dynamics...in the &lt;strong&gt;KWIK framework&lt;/strong&gt;.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The above is part of a &lt;a href=&quot;http://carlosdiuk.github.io/papers/OORL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;'s conclusion - and I don't really seem to understand what the KWIK framework is. In the details of the paper, is a brief highlight of the KWIK conditions for a learning algorithm, which go as follows (I paraphrase):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;All predictions must be accurate (assuming a valid hypotheses class)&lt;/li&gt;&#xA;&lt;li&gt;However the learning algorithm may also return &lt;span class=&quot;math-container&quot;&gt;$\perp$&lt;/span&gt;, which indicates that it cannot yet predict the output for this input.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A quick Google Search brought me to &lt;a href=&quot;http://icml2008.cs.helsinki.fi/papers/627.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper from ICML 2008&lt;/a&gt;, but it is a little difficult to comprehend without a detailed read. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please &lt;em&gt;help me understand what the KWIK framework is&lt;/em&gt;, and what implication does it have for a &lt;em&gt;learning algorithm to satisfy KWIK conditions&lt;/em&gt;? An explanation that starts at simple and goes to fairly advanced discussions is appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-05-03T13:44:10.690" Title="What is the KWIK Framework?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20770" PostTypeId="1" AcceptedAnswerId="20776" CreationDate="2020-04-29T14:04:28.770" Score="3" ViewCount="72" Body="&lt;p&gt;We are building an AI to play a board game. Leaving aside the implementation, we noticed that it plays worse when we set an even (2,4,6,...) level of depth. We use a minimax depth-first strategy.&#xA;Do you have any ideas why it behaves like that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: for example if we set a game between an AI with 5 levels of depth and an AI with 6 levels of depth, the first one usually wins (and this is weird).&lt;/p&gt;&#xA;" OwnerUserId="36585" LastEditorUserId="36585" LastEditDate="2020-04-29T19:57:57.970" LastActivityDate="2020-04-29T19:57:57.970" Title="Why does our AI play worse at even levels of depth?" Tags="&lt;game-theory&gt;&lt;depth-first-search&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20771" PostTypeId="2" ParentId="5539" CreationDate="2020-04-29T14:21:55.400" Score="2" Body="&lt;p&gt;A random function cannot be learned efficiently by any algorithm, in particular, neural networks. However, if you are looking for function with (exponentially) smaller description size, I do not know but any function that is conjectured to be average-case hard probably cannot be learned efficiently by neural networks, for example,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Any cryptographic hard-core predicate (related: &lt;a href=&quot;https://datascience.stackexchange.com/q/16639/96045&quot;&gt;Could Deep Learning be used to crack encryption?&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Parity learning with noise (related: &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.36.5766&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cryptographic Primitive Based on Hard Learning Problems&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Combinatorial examples: planted SAT, planted clique (related: &lt;a href=&quot;https://eprint.iacr.org/2017/365.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barak's survey&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36586" LastActivityDate="2020-04-29T14:21:55.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20772" PostTypeId="2" ParentId="20758" CreationDate="2020-04-29T14:57:07.447" Score="1" Body="&lt;p&gt;I think the key to your problem may not the &lt;strong&gt;one-round&lt;/strong&gt;. Use RL to solve the knapsack problem is great related to the topic &lt;strong&gt;rl for combination optimization&lt;/strong&gt;. U can use &lt;a href=&quot;https://arxiv.org/pdf/1611.09940.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEURAL COMBINATORIAL OPTIMIZATION WITH REINFORCEMENT LEARNING&lt;/a&gt; to get some idea and find more related solutions.&lt;/p&gt;&#xA;" OwnerUserId="36587" LastActivityDate="2020-04-29T14:57:07.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20773" PostTypeId="1" CreationDate="2020-04-29T15:27:42.730" Score="2" ViewCount="334" Body="&lt;p&gt;In DDPG, if there are no &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy and no action noise, is DDPG an on-policy algorithm?&lt;/p&gt;&#xA;" OwnerUserId="8415" LastEditorUserId="2444" LastEditDate="2020-04-29T15:44:01.650" LastActivityDate="2021-05-09T23:13:58.287" Title="Why is DDPG an off-policy RL algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;ddpg&gt;&lt;off-policy-methods&gt;&lt;on-policy-methods&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20774" PostTypeId="1" CreationDate="2020-04-29T15:39:52.120" Score="1" ViewCount="35" Body="&lt;p&gt;Let's say I have a dataset with multiple types of multiple ingredients (salt1,salt2, etc). Each n-th variation of each ingredient vs flavor may be represented by an n×k matrix that where an ingredient corresponds with a particular value of &quot;flavor&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A recipe consists of a 1×n vector (where n is the number of ingredients) where each value corresponds to the quantity of ingredient in the recipe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A particular combination of ingredients, with particular weights, with some transformation, would result in a particular 1×k &quot;flavor&quot; profile, in this simple model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach could be to formulate this as a Probabilistic Matrix Factorization problem (I think), with k being the number of flavor parameters. And combining the recipe vector with the flavor matrix might do the trick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is, the flavor value of each ingredient (and each variation of the ingredient) in the ingredient-flavor matrix would be very very limited. The recipe flavor profile might have a corresponding flavor vector, that too would be limited, and would not be available, at the beginning. So in order to capture the relationship between the ingredients and the flavor, the system would be dependent on user-submitted data on recipe/ingredient flavors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I could create clusters of recipes based on user flavor ratings and extrapolate these to the constituent ingredients or vice versa? Could this be done via some unsupervised learning algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am quite new to this, I would appreciate some help or some pointers to which mathematical approaches I should be looking at to model this problem.&lt;/p&gt;&#xA;" OwnerUserId="36589" LastActivityDate="2020-04-29T15:39:52.120" Title="How do I approach this problem?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;unsupervised-learning&gt;&lt;clustering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20775" PostTypeId="2" ParentId="20773" CreationDate="2020-04-29T16:39:57.343" Score="0" Body="&lt;p&gt;If there was no action noise it would probably not explore enough to obtain a good estimate of Q or the policy gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of estimating Q of the target policy you could estimate Q of the behavior policy but then you have a stochastic policy and the deterministic policy gradient theorem does not work anymore as it is a special case of the stochastic policy gradient theorem (see Section 3.3 of the DPG paper, &lt;a href=&quot;http://proceedings.mlr.press/v32/silver14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proceedings.mlr.press/v32/silver14.pdf&lt;/a&gt;). You would have to use policy gradient theorem of Section 2.2 from the DPG paper.&lt;/p&gt;&#xA;" OwnerUserId="36590" LastEditorUserId="36590" LastEditDate="2020-04-29T17:43:34.243" LastActivityDate="2020-04-29T17:43:34.243" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="20776" PostTypeId="2" ParentId="20770" CreationDate="2020-04-29T17:37:16.257" Score="1" Body="&lt;p&gt;When the number of levels is odd, it means the first player can do one more extra movement on the board. As it is an extensive form game, when decide using backward induction, as the last and the first move are for the first player, so the first player can act better than the situation that the second player will make the last move.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2020-04-29T17:37:16.257" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20777" PostTypeId="1" AcceptedAnswerId="20804" CreationDate="2020-04-29T18:43:00.410" Score="1" ViewCount="61" Body="&lt;p&gt;What kinds of techniques do autopilots of autonomous cars (e.g. the ones of Tesla) use? Do they use reinforcement learning? Which types of neural network architecture do they use?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-04-30T12:34:04.053" LastActivityDate="2020-04-30T15:15:18.837" Title="What kinds of techniques do autopilots of autonomous cars use?" Tags="&lt;neural-networks&gt;&lt;applications&gt;&lt;reference-request&gt;&lt;autonomous-vehicles&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20778" PostTypeId="1" CreationDate="2020-04-29T22:24:48.947" Score="1" ViewCount="156" Body="&lt;p&gt;What are the current popular approaches to leveraging AI for stock price prediction? It seems like there could be several approaches and problem formulations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Supervised learning: &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Regression: predict the stock price directly&lt;/li&gt;&#xA;&lt;li&gt;Classification: predict whether the stock price goes up or down&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Unsupervised learning: find clusters of stocks that move together&lt;/li&gt;&#xA;&lt;li&gt;Reinforcement learning: let the agent directly maximize its stock market return &lt;/li&gt;&#xA;&lt;li&gt;Other AI methods: rules, symbolic systems etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which are most popular/performant? Are there other ways that people are using machine learning in stock trading (sentiment analysis on financial statments, news etc.). &lt;/p&gt;&#xA;" OwnerUserId="18086" LastActivityDate="2021-05-13T02:53:52.883" Title="Best ways of leveraging AI for stock market trading" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;symbolic-ai&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20779" PostTypeId="1" CreationDate="2020-04-30T00:00:24.280" Score="2" ViewCount="75" Body="&lt;p&gt;I have been recently studying Actor-Critic algorithms, and I ran into the following question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$Q_{\omega}$&lt;/span&gt; be the critic network, and &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta}$&lt;/span&gt; be the actor. It is known that in order to maximize the objective return &lt;span class=&quot;math-container&quot;&gt;$J(\theta)$&lt;/span&gt;, we follow the gradient direction, which could be estimated as follows &lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}J=\mathbb{E}[Q_{\omega}(s,a).\nabla_{\theta}log \pi_{\theta} (a|s)].$$&lt;/span&gt; But if we were to calculate the gradient of &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, what are the possible approaches to do so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More generally&lt;/strong&gt;, say we have a network &lt;span class=&quot;math-container&quot;&gt;$\phi_{\omega}$&lt;/span&gt; that is trained on data generated from another neural network, say a stochastic actor &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta}$&lt;/span&gt; like in classic reinforcement learning frameworks, how to find the gradient of &lt;span class=&quot;math-container&quot;&gt;$\phi_{\omega}$&lt;/span&gt; w.r.t &lt;span class=&quot;math-container&quot;&gt;${\theta}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="36603" LastEditorUserId="36603" LastEditDate="2020-04-30T02:22:58.517" LastActivityDate="2020-04-30T02:22:58.517" Title="What is the gradient of the Q function with respect to the policy's parameters?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20780" PostTypeId="2" ParentId="7601" CreationDate="2020-04-30T00:34:20.203" Score="0" Body="&lt;p&gt;I have recently watched &lt;a href=&quot;https://www.youtube.com/watch?v=STFcvzoxVw4&quot; rel=&quot;nofollow noreferrer&quot;&gt;a podcast by Lex Fridman where he interviews Vladimir Vapnik&lt;/a&gt;, who talks about this way of teaching with &amp;quot;predicates&amp;quot; and &amp;quot;invariants&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;If you were really able to teach a model with sentences like &amp;quot;a bird is an animal that is able to fly&amp;quot; but &amp;quot;not all birds fly, such as penguins&amp;quot;, that would probably represent one of the biggest milestones in machine learning, especially, if the machine was able to learn and apply this learned knowledge as &lt;em&gt;efficiently&lt;/em&gt; as humans do. However, we are not quite there yet!&lt;/p&gt;&#xA;&lt;p&gt;To know more about this new learning paradigm &lt;strong&gt;Learning Using Statistical Invariants (LUSI)&lt;/strong&gt;, you probably should read the paper &lt;a href=&quot;https://link.springer.com/article/10.1007/s10994-018-5742-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rethinking statistical learning theory: learning using statistical invariants&lt;/a&gt; (2019), by V. Vapnik and R. Izmailov, which will probably be difficult to follow if you have no knowledge of learning theory and your mathematical background is poor. However, there are some sections of this paper that are accessible to everyone. For example, section 6.6&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Suppose that the Teacher teaches Student to recognize digits by providing a number of examples and also suggesting the following heuristics: &amp;quot;In order to recognize the digit zero, look at the center of the picture — it is usually light; in order to recognize the digit 2, look at the bottom of the picture - it usually has a dark tail&amp;quot; and so on.&lt;/p&gt;&#xA;&lt;p&gt;From the theory above, the Teacher wants the Student to construct specific predicates &lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt; to use them for invariants. However, the Student does not necessarily construct exactly the same predicate that the Teacher had in mind (the Student's understanding of concepts &amp;quot;center of the picture&amp;quot; or &amp;quot;bottom of the picture&amp;quot;:  can be different). Instead of &lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt;, the Student constructs function &lt;span class=&quot;math-container&quot;&gt;$\hat{\psi}(x)$&lt;/span&gt;. However, this is acceptable, since any function from &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; can serve as a predicate for an invariant.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-04-30T00:34:20.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20782" PostTypeId="1" AcceptedAnswerId="20874" CreationDate="2020-04-30T03:53:39.437" Score="2" ViewCount="96" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Single-object tracking commonly uses &lt;strong&gt;Siamese networks, which can be seen as an RNN&lt;/strong&gt; unrolled over two time-steps.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1806.01794&quot; rel=&quot;nofollow noreferrer&quot;&gt;(from the SQAIR paper)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering how Siamese networks can be viewed as RNNs, as mentioned above. A diagrammatic explanation, or anything that helps understand the same, would help! Thank you!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-05-04T07:55:22.150" Title="How can Siamese Networks be viewed as RNNs?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;papers&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20783" PostTypeId="1" AcceptedAnswerId="22886" CreationDate="2020-04-30T04:23:46.487" Score="2" ViewCount="201" Body="&lt;p&gt;I'm trying to compare Glove, Fasttext, Bert on the basis of similarity between 2 words using Pre-trained Models. Glove and Fasttext had pre-trained models that could easily be used with gensim word2vec in python.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Does BERT have any such models?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is it possible to check the similarity between two words using BERT?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-08-06T04:54:18.243" Title="Similarity score between 2 words using Pre-trained BERT using Pytorch" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;&lt;similarity&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20784" PostTypeId="2" ParentId="13627" CreationDate="2020-04-30T04:50:57.687" Score="0" Body="&lt;p&gt;Not sure if you're using TensorFlow-Agents for this, but if you are, there's some nice functionality for this built out using the &lt;code&gt;ActorDistributionRnnNetwork&lt;/code&gt; and &lt;code&gt;ValueDistributionRnnNetwork&lt;/code&gt; that I've found to be very helpful.  The LSTM architecture is used in both the actor and critic networks.  See &lt;a href=&quot;https://www.tensorflow.org/agents/api_docs/python/tf_agents/networks/actor_distribution_rnn_network/ActorDistributionRnnNetwork&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; for more information - hope this helps!  &lt;/p&gt;&#xA;" OwnerUserId="36611" LastActivityDate="2020-04-30T04:50:57.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20785" PostTypeId="1" CreationDate="2020-04-30T05:14:46.093" Score="0" ViewCount="30" Body="&lt;p&gt;I'm trying to get a value for a correlation between a function input and its output. One brute force way to get this is to sample the entire space and find the standard deviation of the resulting outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a less expensive way to find this correlation, perhaps by differentiating the equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more context, I'm trying to see how much a particular input affects each output of a neural network (nothing to do with tuning weights).&lt;/p&gt;&#xA;" OwnerUserId="25574" LastEditorUserId="2444" LastEditDate="2020-04-30T15:48:20.113" LastActivityDate="2020-04-30T15:48:20.113" Title="How can I find the correlation between the input and output of a neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;statistical-ai&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20787" PostTypeId="2" ParentId="20761" CreationDate="2020-04-30T06:21:32.543" Score="0" Body="&lt;p&gt;OK, I think I understood what this means. Hard and easy negatives are ones which have a relatively large and small values for the loss function respectively. &lt;/p&gt;&#xA;" OwnerUserId="36582" LastActivityDate="2020-04-30T06:21:32.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20789" PostTypeId="2" ParentId="18701" CreationDate="2020-04-30T06:56:15.797" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What are the AI technologies currently used to fight the coronavirus pandemic?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You don't define what is AI for you. Let's suppose it means &lt;em&gt;advanced informatics&lt;/em&gt; (but perhaps also &lt;a href=&quot;https://en.wikipedia.org/wiki/Abstract_interpretation&quot; rel=&quot;nofollow noreferrer&quot;&gt;abstract interpretation&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In France we have polling websites like &lt;a href=&quot;https://covidnet.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://covidnet.fr/&lt;/a&gt; ... Does that count as AI?&lt;/p&gt;&#xA;&lt;p&gt;And we also have Deep Tech related Covid19 specific call for proposals, e.g. &lt;a href=&quot;https://systematic-paris-region.org/fr/actualite/appel-a-projets-structurants-pour-la-competitivite-specifique-a-la-crise-sanitaire-covid-19/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; ... does that count as AI?&lt;/p&gt;&#xA;&lt;p&gt;Does the French &lt;a href=&quot;https://stopcovid19.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;StopCovid&lt;/a&gt; platform count as AI? It is an open source digital project (using Bluetooth) to trace potential Covid-infected people. The current political debate is involving intense discussions about its legitimacy and deployement.&lt;/p&gt;&#xA;&lt;p&gt;At last, development of Covid specific ventilators (e.g. &lt;a href=&quot;https://github.com/Recovid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; open hardware / open software project) involves a lot of embedded software which could be analyzed by static analyzers such as &lt;a href=&quot;https://frama-c.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Frama-C&lt;/a&gt; ? Does that count as AI ?&lt;/p&gt;&#xA;&lt;p&gt;My employer &lt;a href=&quot;https://www.cea.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CEA&lt;/a&gt; (a French scaled down equivalent to &lt;a href=&quot;https://en.wikipedia.org/wiki/United_States_Department_of_Energy&quot; rel=&quot;nofollow noreferrer&quot;&gt;US DoE&lt;/a&gt;) is an applied research institution participating to several Covid related projects.&lt;/p&gt;&#xA;&lt;p&gt;And several French research institutions (e.g. &lt;a href=&quot;https://www.inserm.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;INSERM&lt;/a&gt;, &lt;a href=&quot;http://www.cnrs.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNRS&lt;/a&gt;, &lt;a href=&quot;https://www.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;INRIA&lt;/a&gt;, &lt;a href=&quot;https://www.lip6.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LIP6&lt;/a&gt;, etc...) are participating to Covid related research programmes. Also, major health organizations like &lt;a href=&quot;https://www.aphp.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AP-HP&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So is the &lt;a href=&quot;https://ec.europa.eu/info/index_en&quot; rel=&quot;nofollow noreferrer&quot;&gt;European Union&lt;/a&gt; which is partly funding several of them. Most of the Covid related research projects have strong digital aspects. In particular, related to genome decoding and bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;PS. My GitHub &lt;a href=&quot;https://github.com/bstarynk/helpcovid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;helpcovid&lt;/a&gt; ongoing project (a &lt;a href=&quot;https://www.gnu.org/philosophy/free-sw.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;free software&lt;/a&gt; web application related to Covid19)  does not claim to be AI, but certainly is a digital application. See also &lt;a href=&quot;http://refpersys.org/Starynkevitch-CAIA-RefPerSys-2020mar06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these&lt;/a&gt; slides about &lt;a href=&quot;https://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; (which &lt;em&gt;is&lt;/em&gt; an ongoing &lt;a href=&quot;https://www.gnu.org/philosophy/free-sw.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;free software&lt;/a&gt; ambitious AI project).&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="2444" LastEditDate="2020-09-29T22:01:48.643" LastActivityDate="2020-09-29T22:01:48.643" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20790" PostTypeId="1" AcceptedAnswerId="20822" CreationDate="2020-04-30T07:30:09.127" Score="0" ViewCount="49" Body="&lt;p&gt;In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform worse than the model without PCA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question comes from the fact that I want to add a PCA step prior to a model and hyperparameterize the PCA output dimension from 1 to N (N being the number of dimensions in the original dataset) and I wanted to know if there is any theoretical basis that there is no case in which performing this previous step could have a worse performance than the previous model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially my doubt comes if the best PCA case from a selection of dimensions from 1-N is always better than the best case without PCA.&lt;/p&gt;&#xA;" OwnerUserId="22930" LastEditorUserId="2444" LastEditDate="2020-04-30T15:44:05.400" LastActivityDate="2020-05-01T11:46:12.923" Title="Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?" Tags="&lt;theory&gt;&lt;models&gt;&lt;hyperparameter-optimization&gt;&lt;principal-component-analysis&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20791" PostTypeId="2" ParentId="16863" CreationDate="2020-04-30T07:30:41.150" Score="0" Body="&lt;p&gt;Hey I am currently studying First Order Logic right now and I think I can answer your question. Others please correct me if I am wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first case, you can generally substitute variables with constants. Hence, you can make the substitution &lt;span class=&quot;math-container&quot;&gt;$\theta \leftarrow \{x_1 \leftarrow A, y_1 \leftarrow B \}$&lt;/span&gt;. This is used very commonly when you want to infer some query &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; from your knowledge base. &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; usually is in the form &lt;span class=&quot;math-container&quot;&gt;$P(A,B)$&lt;/span&gt; as you have mentioned. When you unify you get &lt;span class=&quot;math-container&quot;&gt;$\neg L(A) \vee H(A)$&lt;/span&gt; and ur substitution has to stay the same throughout the resolution algorithm. I.e, you cannot substitute &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; for another constant / variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the second case, you cannot substitute a constant with a variable &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to another variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you can substitute a variable with a constant, or another variable. You can also substitute a variable with a skolem function, Eg. &lt;span class=&quot;math-container&quot;&gt;$x_1 \leftarrow G(y)$&lt;/span&gt;. However, for skolem functions, you cannot substitute &lt;span class=&quot;math-container&quot;&gt;$x \leftarrow F(x)$&lt;/span&gt;, in which the variable names are the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps ! &lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-04-30T07:30:41.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20793" PostTypeId="2" ParentId="14332" CreationDate="2020-04-30T08:21:56.490" Score="2" Body="&lt;p&gt;Both implementations may be closer than you think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In short:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PPO has both parts&lt;/strong&gt;: there is &lt;strong&gt;noisiness in draws during training&lt;/strong&gt; (with learned standard deviation), helping to explore new promising actions/policies. And there is a &lt;strong&gt;term added to the loss function aiming to prevent a collapse of the noisiness&lt;/strong&gt;, to help ensure exploration continues and we don't get stuck at a bad (local) equilibrium.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, for continuous action, the term for &lt;strong&gt;entropy in the loss function&lt;/strong&gt; you describe in Ex. 2, can &lt;strong&gt;make sense only when the actions are stochastic&lt;/strong&gt;, i.e. when there action choice has some standard deviation the way you describe in Ex. 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More detail:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On one hand, PPO (at least for continuous action), trains a central/deterministic value (say the mean policy or close to mean) targeting a most profitable action path. On the other hand, along with it, a standard deviation making the actions a random draw with noise around the deterministic value. This is the part you describe in &lt;em&gt;Example 1&lt;/em&gt;. The noise helps explore new paths and to update the policy according to the rewards on these sampled paths.&#xA;Entropy itself is a measure of the noisiness of the draws, an thus also an indirect indicator for the trained standard deviation value(s) of the policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, entropy tends to decay as training progresses, that is, the random draws become progressively less random. This can be good for reward maximization - really the best draws are taken for reward maximization - but it is bad for further improvements of policy: improvement may halt or slow down as exploration of new action paths fades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where entropy &lt;em&gt;encouragement&lt;/em&gt; comes in. PPO foresees the inclusion of entropy in the loss function: we reduce the loss by &lt;code&gt;x * entropy&lt;/code&gt;, with &lt;code&gt;x&lt;/code&gt; the entropy coefficient (e.g. 0.01), incentivizing the learning network to increase the standard deviations (or, to not let them drop too much). This part is what you describe in &lt;em&gt;Example 2&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further notes:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;During exploitation, we'd typically turn off the noise (implicitly assuming &lt;code&gt;action std = 0&lt;/code&gt;) and pick deterministic actions: in normal cases this increases the payoffs; we're choosing our best action estimate, rather than at a random value around it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;People are not always precise when referring to the model's &lt;code&gt;entropy&lt;/code&gt; vs. the &lt;code&gt;entropy coefficient&lt;/code&gt; added to the loss function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Other RL algorithms with continuous action tend to use noisy drwas with standard deviations/entropy too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="34179" LastEditorUserId="34179" LastEditDate="2020-04-30T10:42:27.763" LastActivityDate="2020-04-30T10:42:27.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20794" PostTypeId="1" CreationDate="2020-04-30T08:37:46.187" Score="1" ViewCount="36" Body="&lt;p&gt;I've been reading the &lt;a href=&quot;https://arxiv.org/abs/1806.01794&quot; rel=&quot;nofollow noreferrer&quot;&gt;SQAIR paper&lt;/a&gt; lately, and the mathematics involved seems a bit complicated.&lt;/p&gt;&#xA;&lt;p&gt;Some background, about the paper: &lt;em&gt;SQAIR stands for Sequential Attend, Infer, Repeat&lt;/em&gt; - the paper does generative modelling of moving objects. The idea of &lt;em&gt;Attend, Infer, Repeat&lt;/em&gt; is to decompose a static scene into constituent objects, where each object is represented by continuous latent variables. The latent variables, &lt;span class=&quot;math-container&quot;&gt;$z^{what}$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$z^{where}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z^{pres}$&lt;/span&gt; encode the appearance, position and presence of an object.&lt;/p&gt;&#xA;&lt;p&gt;Here's a screenshot of the first of many things I'm unable to understand -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6dRLa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6dRLa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why is &lt;span class=&quot;math-container&quot;&gt;$z^{pres,1:n+1}$&lt;/span&gt; a random vector of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; ones followed by a zero? Why do we need the zero? How does it help?&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, an explanation of equation &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt; as in the image above, would be great.&lt;/p&gt;&#xA;&lt;p&gt;P.S. I hope you all find the paper interesting. I'll ask other questions from the paper in separate posts, so as to not crowd one post with too many queries.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2021-03-04T17:07:40.870" Title="Why is this variable in equation 2 of the SQAIR paper a random vector of $n$ ones followed by a zero?" Tags="&lt;computer-vision&gt;&lt;papers&gt;&lt;generative-model&gt;&lt;random-variable&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20796" PostTypeId="1" CreationDate="2020-04-30T09:42:50.113" Score="1" ViewCount="22" Body="&lt;p&gt;I want to create an exercise suggester. Each day either has a routine or is a rest day. A routine has 4 slots. For each slot we select an exercise. We constrain the legal exercises, only do upper-body today, etc. We, therefore, restrict the number of available exercises. This seems easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how can I extend this model to take fatigue into account. Fatigue is a state such that every exercise reduces it (specific value for each exercise) and it recovers with time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this problem even be modeled as a constraint satisfaction problem? What's the best way of modeling and solving this problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to suggest all the legal exercises taking fatigue into account.&lt;/p&gt;&#xA;" OwnerUserId="23895" LastEditorUserId="2444" LastEditDate="2020-04-30T15:40:21.230" LastActivityDate="2020-04-30T15:40:21.230" Title="How can I design a system that suggests physical exercises to a person while keeping into account the fatigue?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20797" PostTypeId="1" CreationDate="2020-04-30T11:00:05.193" Score="1" ViewCount="22" Body="&lt;p&gt;I want to find out scenarios that useful to examine the performance of intra-group and inter-group cooperation in MARL. &#xA;Specifically, I prefer a board game (like sudoku) that is suitable for the cooperation evaluation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are some differences between the requirements and the go like game. Every grid on the board should be treated as an agent. They are designed to form a situation with the local utility and global utility. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the sudoku as an example, every grid should choose an appropriate value to reach the sudoku solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to my non-familiarity with traditional MARL scenarios, it will be a great help for showing me some keywords or lists.&lt;/p&gt;&#xA;" OwnerUserId="36587" LastEditorUserId="36587" LastEditDate="2020-04-30T16:34:29.463" LastActivityDate="2020-04-30T16:34:29.463" Title="Are there any board game appropriate to examine the performance of multiple agents that cooperate both inter-group and intra-group?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20798" PostTypeId="1" AcceptedAnswerId="20801" CreationDate="2020-04-30T11:32:12.423" Score="2" ViewCount="44" Body="&lt;p&gt;I'm currently working on a project where I am using a basic cellular automata and a genetic algorithm to create dungeon-like maps. Currently, I'm having an incredibly hard time understanding how exactly crossover works when my output can only be two states: DEAD or ALIVE (1 or 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand crossover conceptually - you find two fit members of the population and they exchange genetic material, hopefully producing a fitter offspring. I also understand this is usually done by performing k-point crossover on bit strings (but can also be done with real numbers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;However, even if I encode my DEAD/ALIVE cells into bits and cross them over, what do I end up with?&lt;/em&gt; The cell can &lt;em&gt;only&lt;/em&gt; be DEAD or ALIVE. &lt;em&gt;Will the crossover give me some random value that is outside this range?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;And even if I were to work on floating-point numbers, wouldn't I just end up with a 1 or 0 anyway?&lt;/em&gt; In that case, it seems like it would be better to just randomly mutate DEAD cells into ALIVE cells, or vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read several papers on the topic, but none seem to explain this particular issue (in a language I can understand, anyway). Intuitively, I thought maybe I can perform crossover on a &lt;em&gt;neighbourhood&lt;/em&gt; of cells - so I find 2 fit neighbourhoods, and then they exchange members (for example, neighbourhood A gives 4 of its neighbours to neighbourhood B). However, I have not seen this idea anywhere, which leads me to believe it must be fundamentally wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated, I'm really stuck on this one.&lt;/p&gt;&#xA;" OwnerUserId="36622" LastEditorUserId="2444" LastEditDate="2020-04-30T15:36:17.453" LastActivityDate="2020-04-30T15:36:17.453" Title="How does the crossover operator work when my output contains only 2 states?" Tags="&lt;genetic-algorithms&gt;&lt;crossover-operators&gt;&lt;genetic-operators&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20799" PostTypeId="1" CreationDate="2020-04-30T12:30:09.670" Score="0" ViewCount="63" Body="&lt;p&gt;I working on a classification problem that needs to detect patterns on a time serie. Basically, there's a catch-all class that means &quot;no pattern detected&quot;, the other are for the specific patterns. The data is imbalanced (ratio 1/10 at least), but I adapted the class weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm able to overfit successfully on a few days of data, but when I train on 2 years of data, the model seems stuck on class1 &quot;no pattern detected&quot; for a veeeery long time. I've tried several  learning rates, but it doesn't make the convergence happen significatively faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a better starting point for my training to use the overfitting model's weight as a starting point? Could this allow the model to converge faster?&lt;/p&gt;&#xA;" OwnerUserId="36624" LastActivityDate="2021-05-25T19:05:55.500" Title="Is it a good idea to overfit on a small part of your data for faster model convergence?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;overfitting&gt;&lt;convergence&gt;&lt;learning-rate&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20800" PostTypeId="1" CreationDate="2020-04-30T13:00:33.043" Score="1" ViewCount="60" Body="&lt;p&gt;In &lt;a href=&quot;https://www.youtube.com/watch?v=buptHUzDKcE&amp;amp;list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&amp;amp;index=6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Stanford lecture&lt;/a&gt; (minute 35:47 and 37:00), the professor says that Monte Carlo (MC) linear function approximation does not always converge, and she gives an example. In general, when does MC linear function approximation converge (or not)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do people use that MC linear function approximation if sometimes it doesn't converge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They also gave the definition of the stationary distribution of a policy, and I am not sure if using it for function approximation converges or not.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-04-30T14:31:44.667" LastActivityDate="2020-04-30T14:31:44.667" Title="When does Monte Carlo linear function approximation converge?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;monte-carlo-methods&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20801" PostTypeId="2" ParentId="20798" CreationDate="2020-04-30T13:28:15.513" Score="0" Body="&lt;p&gt;How do you calculate fitness for your organism if there are only two states? I would think it would be a solution to work on combinations of cells:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have blocks of, say, 4x4 cells, then each organism would be encoded using 16 bits. A crossover then is like overlaying two 4x4 blocks, picking the new block from either the first or the second parent. Effectively you are doing this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0011                   1111                 xx11&#xA;0001  combined with    0000  would become   000x&#xA;1001                   0000                 x00x&#xA;1111                   0000                 xxxx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(this would be a bottom-right corner combined with a top wall segment)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or, as a single bit-string:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0011000110011111 +&#xA;1111000000000000 =&#xA;xx11000xx00xxxxx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where 'x' can be either 0 or 1, as this is where the two organisms are different.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-04-30T13:28:15.513" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20802" PostTypeId="2" ParentId="20777" CreationDate="2020-04-30T14:34:53.350" Score="0" Body="&lt;p&gt;They definitely don't use RL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is a mix of NN for segmentation, lidar, radar and how for perception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once object detection has been performed they use a trajectory generator and MPC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an autonomous car class on Coursera that goes over this.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-04-30T14:34:53.350" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20803" PostTypeId="1" CreationDate="2020-04-30T14:45:32.223" Score="3" ViewCount="67" Body="&lt;p&gt;I was trying to understand the implementation of a basic policy gradient (REINFORCE) method using TensorFlow.&#xA;I think I got almost everything. The only thing that still bothers me is the loss function implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the theory, we have that after all the manipulation the gradient of the score function is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}J(\theta)=\mathop{\mathbb{E}}\left[\nabla_{\theta}(log(\pi(s,a,\theta)))R(\tau) \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this &lt;a href=&quot;https://github.com/simoninithomas/Deep_reinforcement_learning_Course/blob/master/Policy%20Gradients/Cartpole/Cartpole%20REINFORCE%20Monte%20Carlo%20Policy%20Gradients.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cartpole&lt;/a&gt; example the part relative to the loss function is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    neg_log_prob = tf.nn.softmax_cross_entropy_with_logits_v2(logits = NeuralNetworkOutputs, labels = actions)&#xA;    loss = tf.reduce_mean(neg_log_prob * discounted_episode_rewards_) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At this point, I do not understand how the definition from above translates into code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/49377483/about-tf-nn-softmax-cross-entropy-with-logits-v2&quot;&gt;As far as I understood&lt;/a&gt;, the functions &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tf.nn.softmax_cross_entropy_with_logits_v2(logits = NeuralNetworkOutputs, labels = actions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;log(softmax(NeuralNetworkOutputs))*actions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which is then multiplied by the discounted returns &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;log(softmax(NeuralNetworkOutputs))*actions*discounted_episode_rewards_&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Within this expression, I do not understand why should we multiply, an expression which looks like the loss function we want, by the value of the action.&lt;/p&gt;&#xA;" OwnerUserId="36629" LastEditorUserId="2444" LastEditDate="2020-04-30T15:06:00.740" LastActivityDate="2020-04-30T15:06:00.740" Title="Understanding the TensorFlow implementation of the policy gradient method" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;deep-rl&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20804" PostTypeId="2" ParentId="20777" CreationDate="2020-04-30T14:48:55.077" Score="0" Body="&lt;p&gt;I will focus on Tesla's autopilots in this answer (because that's the only specific autopilot you mention). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://www.tesla.com/autopilot&quot; rel=&quot;nofollow noreferrer&quot;&gt;their website&lt;/a&gt;, they mention the basic technologies underlying the current autopilots, which includes deep neural networks.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To make use of a camera suite this powerful, the new hardware introduces an entirely new and powerful set of vision processing tools developed by Tesla. Built on a &lt;strong&gt;deep neural network&lt;/strong&gt;, Tesla Vision deconstructs the car's environment at greater levels of reliability than those achievable with classical vision processing techniques.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, they are probably using convolutional neural networks for the computer vision tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also &lt;a href=&quot;https://en.wikipedia.org/wiki/Tesla_Autopilot&quot; rel=&quot;nofollow noreferrer&quot;&gt;a Wikipedia article&lt;/a&gt; completely dedicated to Tesla's autopilot.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-04-30T15:15:18.837" LastActivityDate="2020-04-30T15:15:18.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20806" PostTypeId="2" ParentId="20799" CreationDate="2020-04-30T15:56:40.033" Score="0" Body="&lt;p&gt;The first thing which you first have to understand is that does your trained model is working efficiently with both the training and testing data. If yes then its not overfitting. There is only one case where over-fitting doesn't matter and that is when your testing data is same as training data and dealing with is very important or it could be disastrous. Therefore, to avoid overfitting you can try several methods and they are followings: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Optimization Algorithms like sgd, Adam, RMSprop, Adagard, are few gradient descents.&lt;/li&gt;&#xA;&lt;li&gt;Loss function like Hinge loss could be useful.&lt;/li&gt;&#xA;&lt;li&gt;Parameter initialization greatly influence the optimization process.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Overall, Large datasets are better for building ML models but there are some methods there is an interesting article that i would recommend you to read: &lt;a href=&quot;https://towardsdatascience.com/breaking-the-curse-of-small-datasets-in-machine-learning-part-1-36f28b0c044d&quot; rel=&quot;nofollow noreferrer&quot;&gt;Article Link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I Hope this Helps!!!&lt;/p&gt;&#xA;" OwnerUserId="25685" LastEditorUserId="25685" LastEditDate="2020-04-30T16:56:57.503" LastActivityDate="2020-04-30T16:56:57.503" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20808" PostTypeId="1" CreationDate="2020-04-30T18:05:27.060" Score="6" ViewCount="1022" Body="&lt;p&gt;I would like to ask whether MCTS is usually chosen when the &lt;strong&gt;branching factor&lt;/strong&gt; for the states that we have available is large and not suitable for Minimax. Also, other than MCTS simluates actions, where Minimax actually 'brute-forces' all possible actions, what are some other benefits for using Monte Carlo for adversarial (2-player) games?&lt;/p&gt;&#xA;" OwnerUserId="36638" LastEditorUserId="36638" LastEditDate="2020-04-30T20:26:47.140" LastActivityDate="2020-07-11T11:51:34.107" Title="When should Monte Carlo Tree search be chosen over MiniMax?" Tags="&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20809" PostTypeId="1" CreationDate="2020-04-30T18:17:41.343" Score="1" ViewCount="37" Body="&lt;p&gt;I am at a very initial stage of my research so I will try to describe what I am trying to achieve: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to create an AI model which learns how to navigate the browser's component like clicking or creating new favorites and tabs, or navigating browser action menu, or bookmarking the website etc. In short, automating the testing for browser using selenium and an AI model so that over time the model learns itself to navigate the browser and test different functionality itself and eventually it test the functionalities that are not seen by the model before. For example: if I feed the AI model how browser is closed when &quot;x&quot; is clicked and minimize when &quot;-&quot; is clicked, next it can learn itself how to maximize the browser. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial input could be to record some videos of navigating the browser using selenium and then feed it to the model and with time it learns itself to go to different section of the browser which the model does not know and still test it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it even possible to combine AI and selenium together to create something like this? If yes, how can I achieve it and what is the best approach to develop such model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="36637" LastActivityDate="2020-04-30T18:17:41.343" Title="Automating browser actions using AI" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20810" PostTypeId="1" AcceptedAnswerId="20819" CreationDate="2020-04-30T18:18:59.843" Score="2" ViewCount="60" Body="&lt;p&gt;I'm trying to create a convolutional neural network without frameworks (such as PyTorch, TensorFlow, Keras, and so on) with Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a description of CNN taken from &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia article&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics. They have applications in image and video recognition, recommender systems, image classification, medical image analysis, natural language processing, and financial time series.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A CNN has different types of layers, such as convolution, pooling (max or average), flatten and dense (or fully-connected) layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Should we compute gradients (such as &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial A_i}$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial Z_i}$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial A_{i-1}}$&lt;/span&gt; and so on) in flatten layer or not?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If no, then how should I compute &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial A_i}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial Z_i}$&lt;/span&gt; of first layer of convolutional layer? With &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{[\frac{\partial g(A_i)}{\partial x}]}$&lt;/span&gt; or with &#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial dA_{i+2}}$&lt;/span&gt;(P.S. as you know iteration of BackPropagation is reverse, so I used i+n for denote the previous layer)? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or can I compute derivatives in Flatten layer with &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial J}{\partial A} = W_{i+1}^T Z_{i+1}$$&lt;/span&gt;(i+1 denotes prev.layer in BackProp)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial L}{\partial Z} = \frac{\partial L}{\partial A} *\frac{\partial g(A_i)}{\partial x} $$&lt;/span&gt;&#xA;and then reshape of Conv2D shape? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;P.S. I found questions like mine (names are same), but there're not answer to my question as I asking about formula.&lt;/p&gt;&#xA;" OwnerUserId="36639" LastEditorUserId="2444" LastEditDate="2020-05-01T12:38:08.850" LastActivityDate="2020-05-06T10:47:26.017" Title="Should I compute the gradients with respect to the flatten layer in a convolutional neural network?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20813" PostTypeId="2" ParentId="20790" CreationDate="2020-05-01T05:19:52.727" Score="1" Body="&lt;p&gt;PCA can make models worse,  imagine data points scattered along two elongated parallel rectangles. The axis with the greatest variation will be parallel to the rectangles but doesn't provide any benefit in classifying the points.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-01T05:19:52.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20817" PostTypeId="1" CreationDate="2020-05-01T06:07:42.070" Score="1" ViewCount="15" Body="&lt;p&gt;I am interested in doing some work in classification problems in music information retrieval. I know that there are some formats of datasets (such as MIDI, Spectrogram, Piano-roll, MusicXML, etc.) for this work but have been unable to find any nice large datasets for this. What are the best free datasets for such work? I am mainly looking into western classical music.&lt;/p&gt;&#xA;" OwnerUserId="36651" LastActivityDate="2020-05-01T06:07:42.070" Title="What are the best datasets available for music information retrieval?" Tags="&lt;natural-language-processing&gt;&lt;datasets&gt;&lt;data-science&gt;&lt;audio-processing&gt;&lt;structured-data&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20819" PostTypeId="2" ParentId="20810" CreationDate="2020-05-01T07:48:36.027" Score="1" Body="&lt;p&gt;I have found that you should compute derivatives &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial L}{\partial A}, \frac{\partial L}{\partial Z}$&lt;/span&gt; in Flatten layer and then reshape Conv2D input shape.&lt;/p&gt;&#xA;" OwnerUserId="36639" LastEditorUserId="36639" LastEditDate="2020-05-06T10:47:26.017" LastActivityDate="2020-05-06T10:47:26.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20820" PostTypeId="1" CreationDate="2020-05-01T08:53:03.383" Score="2" ViewCount="168" Body="&lt;p&gt;I am trying to understand the mathematics behind the forward and backward propagation of neural nets. To make myself more comfortable, I am testing myself with an arbitrarily chosen neural network. However, I am stuck at some point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a simple fully connected neural network with two hidden layers. For simplicity, choose linear activation function (&lt;span class=&quot;math-container&quot;&gt;${f(x) = x}$&lt;/span&gt;) at all layer. Now consider that this neural network takes two &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional inputs &lt;span class=&quot;math-container&quot;&gt;$X^{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X^{2}$&lt;/span&gt;. However, the first hidden layer only takes &lt;span class=&quot;math-container&quot;&gt;$X^1$&lt;/span&gt; as the input and produces the output of &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;. The second hidden layer takes &lt;span class=&quot;math-container&quot;&gt;$H^{1} $&lt;/span&gt;and &lt;span class=&quot;math-container&quot;&gt;$X^2$&lt;/span&gt; as the input and produces the output &lt;span class=&quot;math-container&quot;&gt;$H^{2}$&lt;/span&gt;. The output layer takes &lt;span class=&quot;math-container&quot;&gt;$H^{2}$&lt;/span&gt; as the input and produces the output &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}$&lt;/span&gt;. For simplicity, assume, we do not have any bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, we can write that, &lt;span class=&quot;math-container&quot;&gt;$H^1 = W^{x1}X^{1}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$H^2 = W^{h}H1 + W^{x2}X^{2} = W^{h}W^{x1}X^{1} + W^{x2}X^{2}$&lt;/span&gt; [substituting the value of &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat{Y} = W^{y}H^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$W^{x1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W^{x2}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W^{h}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W^{y}$&lt;/span&gt; are the weight matrix. Now, to make it more interesting, consider a sharing weight matrix &lt;span class=&quot;math-container&quot;&gt;$W^{x} = W^{x1} = W^{x2}$&lt;/span&gt;, which leads, &lt;span class=&quot;math-container&quot;&gt;$H^1 = W^{x}X^{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$H^2 = W^{h}W^{x}X^{1} + W^{x}X^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have any problem to do forward propagation by my hand; however, the problem arises when I tried to make backward propagation and update the &lt;span class=&quot;math-container&quot;&gt;$W^{x}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial loss}{\partial W^{x}} = \frac{\partial loss}{\partial H^{2}} . \frac{\partial H^{2}}{\partial W^{x}}$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting, &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial loss}{\partial H^{2}} = \frac{\partial Y}{\partial H^{2}}. \frac{\partial loss}{\partial Y}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$H^2 = W^{h}W^{x}X^{1} + W^{x}X^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial loss}{\partial W^{x}}= \frac{\partial Y}{\partial H^{2}}. \frac{\partial loss}{\partial Y} . \frac{\partial}{\partial W^{x}} (W^{h}W^{x}X^{1} + W^{x}X^{2})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I understand that, &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial Y}{\partial H^{2}} = (W^y)^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W^{x}} W^{x}X^{2} = (X^{2})^T$&lt;/span&gt; and we can also calculate &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial Y}{\partial H^{2}}$&lt;/span&gt;, if we know the loss function. But how do we calculate &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W^{x}} W^{h}W^{x}X^{1}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="33750" LastActivityDate="2021-04-22T15:40:38.740" Title="Backpropagation of neural nets with shared weight" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;weights&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20821" PostTypeId="2" ParentId="20820" CreationDate="2020-05-01T10:26:52.570" Score="0" Body="&lt;p&gt;If we write &lt;span class=&quot;math-container&quot;&gt;$ H^2 = W^{h}H1 + W^{x}X^{2} $&lt;/span&gt; then it will be better to understand the backward propagation step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W^{x}} W^{h}W^{x}X^{1}$&lt;/span&gt; can be written as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial H^2}{\partial H^1}\frac{\partial H^1}{\partial W^{x}} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial H^2}{\partial H^1} =  (W^h)^T$&lt;/span&gt; and &#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial H^1}{\partial W^{x}} = (X^{1})^T $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W^{x}} W^{h}W^{x}X^{1} = (W^h)^T(X^{1})^T  $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope it has solved your problem.&lt;/p&gt;&#xA;" OwnerUserId="36657" LastActivityDate="2020-05-01T10:26:52.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20822" PostTypeId="2" ParentId="20790" CreationDate="2020-05-01T11:46:12.923" Score="1" Body="&lt;p&gt;PCA works well where data sample space is linear. If data sample space is not linear or it is manifold data then model without PCA may perform better than model using PCA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FtQsj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FtQsj.png&quot; alt=&quot;Manifold Data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the given image you can see, data is manifold. In this type of data, PCA, which is based on projection technique does not work well. That's why we use manifold learning technique to handle these cases.&lt;/p&gt;&#xA;" OwnerUserId="36657" LastActivityDate="2020-05-01T11:46:12.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20823" PostTypeId="2" ParentId="18778" CreationDate="2020-05-01T13:06:35.037" Score="0" Body="&lt;p&gt;So this question is due to a misunderstanding I had with how the loss function works in A2C. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually loss functions are always positives, but in A2C it can be negative as well. I thought that minimizing the loss function means moving it closer to zero, but in fact reducing the loss is indifferent to the sign of the loss value and is about making the loss value smaller.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the loss for 0.1 is smaller than the loss for 0.9 prediction for a disadvantageous action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the advantage is positive, &lt;span class=&quot;math-container&quot;&gt;$-log (p) * advantage$&lt;/span&gt; will be smaller if &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is growing, and when the advantage is negative, &lt;span class=&quot;math-container&quot;&gt;$-log (p) * advantage$&lt;/span&gt; will be smaller if &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is decreasing.&lt;/p&gt;&#xA;" OwnerUserId="32950" LastActivityDate="2020-05-01T13:06:35.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20824" PostTypeId="1" CreationDate="2020-05-01T13:14:53.317" Score="1" ViewCount="87" Body="&lt;p&gt;I want to make a bot which clicks the fire button on the mobile screen upon seeing an enemies head.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pubg mobile which is an android game you have to control the fire button and the aim along with many other controls to kill an enemy or other players. I want to automate the fire button and everything else would be controlled by me, when I'll aim on a player's head the bot should click the fire button instantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are a few problems in this ,first one is that what if it shoots upon seeing my teammates head and other is what if there are  two players at once ,which will it shoot, for that it needs to shoot when my aim or the red crosshair is on the player's head.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to get started , I need to make an image recognition app and an autoclicker and combine them both. How do I get started? Assuming that I only know basic python.&lt;/p&gt;&#xA;" OwnerUserId="36664" LastActivityDate="2020-05-01T13:14:53.317" Title="How do i start building an autoclick bot for pubg mobile?" Tags="&lt;python&gt;&lt;image-recognition&gt;&lt;game-ai&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20825" PostTypeId="2" ParentId="20820" CreationDate="2020-05-01T13:23:52.473" Score="0" Body="&lt;p&gt;The product rule of partial derivative:&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial x} f g = g \frac{\partial}{\partial x} f + f \frac{\partial}{\partial x} g$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to this:  &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W^{x}} W^{h}W^{x}X^{1} = W^{h}X^{1}$&lt;/span&gt;, because derivative of other term with respect to &lt;span class=&quot;math-container&quot;&gt;$W^{x}$&lt;/span&gt; is zero. (I am not considering the transpose notation as it depends on how you organize your data.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, Your assumption of giving &lt;span class=&quot;math-container&quot;&gt;$H^{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X^{2}$&lt;/span&gt; as input to second hidden layer is not valid(they are called hidden layer for that reason). The output of first hidden layer (&lt;span class=&quot;math-container&quot;&gt;$H^{1}$&lt;/span&gt;)  will be fed to the input of second hidden layer.&#xA;Your output of second hidden layer would be &lt;span class=&quot;math-container&quot;&gt;$H^{2} = W^{h}  * H^{1}$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to fed your input &lt;span class=&quot;math-container&quot;&gt;$X^{1} X^{2}$&lt;/span&gt; to your network at once by means of looping or vectorization.&lt;/p&gt;&#xA;" OwnerUserId="36661" LastActivityDate="2020-05-01T13:23:52.473" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20826" PostTypeId="1" AcceptedAnswerId="20827" CreationDate="2020-05-01T13:49:18.937" Score="1" ViewCount="79" Body="&lt;p&gt;In the context of Reinforcement Learning, &lt;strong&gt;what does it mean to have a multi-dimensional continuous action space?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I came across the following in the &lt;a href=&quot;https://arxiv.org/abs/1905.09275&quot; rel=&quot;nofollow noreferrer&quot;&gt;COBRA Paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A method for learning a distribution over a &lt;strong&gt;multi-dimensional continuous action space.&lt;/strong&gt; This learned distribution can be sampled efficiently.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;During the initial exploration phase it explores its environment, in which it can move objects freely with a &lt;strong&gt;continuous action space&lt;/strong&gt; but is not rewarded for its actions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, what do the multi-dimensionality and the continuity of the action space refer to? It'd be great if someone could provide an explanation with examples!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2021-01-09T22:10:47.527" LastActivityDate="2021-01-09T22:23:26.697" Title="What is meant by a multi-dimensional continuous action space?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;action-spaces&gt;&lt;continuous-action-spaces&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20827" PostTypeId="2" ParentId="20826" CreationDate="2020-05-01T14:19:36.980" Score="2" Body="&lt;p&gt;Let me rephrase it a little - it's a multidimensional continuous space of actions. So, you assign each action some vector from &lt;span class=&quot;math-container&quot;&gt;$R^{n}$&lt;/span&gt;. For intuition -- imagine you have a robot arm with four joints. For every joint you could applied a rotation force from [-1, 1] and thus you get a 4-D vector with float numbers for each possible action.&lt;/p&gt;&#xA;" OwnerUserId="16940" LastEditorUserId="16940" LastEditDate="2020-05-01T14:28:46.420" LastActivityDate="2020-05-01T14:28:46.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20829" PostTypeId="2" ParentId="20826" CreationDate="2020-05-01T14:42:15.907" Score="1" Body="&lt;p&gt;The question has already been answered by Kirill, but I thought I'll add a good example of a multi-dimensional continuous action space too, namely the one I just encountered in the COBRA paper itself.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In all of our experiments we use a 2-dimensional virtual &amp;quot;touch-screen&amp;quot; environment that contains objects with configurable shape, position, and color. The agent can move any visible object by clicking on the object and clicking in a direction for the object to move. Hence the action space is &lt;strong&gt;continuous and 4-dimensional, namely a pair of clicks&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2021-01-09T22:23:26.697" LastActivityDate="2021-01-09T22:23:26.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20830" PostTypeId="1" CreationDate="2020-05-01T14:58:57.710" Score="2" ViewCount="60" Body="&lt;p&gt;I'm creating a RF Q-Learning agent for a two player fully-observable board game and wondered, if I was to train the Q Table using adversarial training, should I let both 'players' use, and update, the same Q Table? Or would this lead to issues?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastActivityDate="2021-05-27T13:05:40.477" Title="Adversarial Q Learning should use the same Q Table?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20831" PostTypeId="1" CreationDate="2020-05-01T16:10:05.723" Score="3" ViewCount="133" Body="&lt;p&gt;I'm building an agent for a racing game. In this game, there is a randomized map where there are speed boosts for the player to pick up and obstacles that act to slow the player down. The goal of the game is to reach the finishing line before the opponent.&lt;/p&gt;&#xA;&lt;p&gt;While working on this problem, I've realized that we can almost forget about the presence of our opponent and just focus on getting the agent to the finish line as quickly as possible.&lt;/p&gt;&#xA;&lt;p&gt;I started with a simple&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; reward for every timestep&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$+100$&lt;/span&gt; reward for winning, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$-100$&lt;/span&gt; for losing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;When I was experimenting with this, I felt like the rewards may be too &lt;em&gt;sparse&lt;/em&gt;, as my agent was converging to pretty poor average returns. I iterated to a function of speed and distance travelled (along with the &lt;span class=&quot;math-container&quot;&gt;$+100$&lt;/span&gt; reward), but, after some experimentation, I started feeling like the agent might be able to achieve high returns without necessarily being the fastest to the finish line.&lt;/p&gt;&#xA;&lt;p&gt;I'm thinking that I return to the first approach and possibly add in some reward for being in the first place (as a function of the opponent's distance behind the agent).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What else could I try? Should I try and spread the positive rewards out more for good behavior? should I create additional rewards/penalties for perhaps hitting obstacles and using boosts or can I expect the agent to learn the correlation?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="36670" LastEditorUserId="2444" LastEditDate="2020-10-07T22:49:00.397" LastActivityDate="2020-10-07T22:49:00.397" Title="How should I design the reward function for racing game (where the goal is to reach finishing line before the opponent)?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;rewards&gt;&lt;reward-design&gt;&lt;reward-shaping&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20832" PostTypeId="1" CreationDate="2020-05-01T17:41:45.043" Score="2" ViewCount="90" Body="&lt;p&gt;What is the original source of the TD Advantage Actor-Critic algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the following tutorial really helpful for learning the algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@asteinbach/actor-critic-using-deep-rl-continuous-mountain-car-in-tensorflow-4c1fb2110f7c&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@asteinbach/actor-critic-using-deep-rl-continuous-mountain-car-in-tensorflow-4c1fb2110f7c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what is the original source of this algorithm?&lt;a href=&quot;https://i.stack.imgur.com/ATMhi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATMhi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36673" LastActivityDate="2020-05-01T17:41:45.043" Title="Original source of the TD Advantage Actor-Critic algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20834" PostTypeId="2" ParentId="20830" CreationDate="2020-05-01T18:58:00.210" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;should I let both 'players' use, and update, the same Q Table? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes this works well for zero-sum games, when player 1 wants to maximise a result (often just +1 for &quot;player 1 wins&quot;) and player 2 wants to minimise a result (score -1 for &quot;player 2 wins&quot;). That alters algorithms such as Q-learning because the greedy choice switches beween min and max functions over action values - player 1's TD target becomes &lt;span class=&quot;math-container&quot;&gt;$R_{t+1} + \gamma \text{min}_{a'} [Q(S_{t+1}, a')]$&lt;/span&gt; because the greedy choice in the next state is taken by player 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if the states never overlap between players, then you could learn a Q function that always returns the future expected return according the current player. This can be forced if necessary by making part of the state record whose turn it is. With this you need a way to convert between player 1 and player 2 scores in order to use Q learning updates. For zero-sum games, then the Q value from a player 2 state is negative of player 1's value for that same state, and vice versa, so the TD target changes for both players to &lt;span class=&quot;math-container&quot;&gt;$R_{t+1} - \gamma \text{max}_{a'} [Q(S_{t+1}, a')]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first option can result in slightly less complexity in the learned function, that might be an issue if you are using function approximation and learning a Q function e.g. using neural networks instead of a Q table. That &lt;em&gt;may&lt;/em&gt; result in faster learning and generalisation, although it will depend on details of the game.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or would this lead to issues?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No major issues. I am performing this kind of training - a single Q function estimating a global score which P1 maximises and P2 minimises - for the &lt;a href=&quot;https://www.kaggle.com/c/connectx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle Connect X competition&lt;/a&gt;, and it works well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of a couple of minor things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You may still want to have the ability for each player to be using a different version of the table or learned Q function. This would allow you to have different versions of your agent (e.g. at different stages of learning) compete, to evaluate different agents against each other. To do this, you have to write code that allows for multiple tables or functions in any case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You need to keep track of how both players express and achieve their opposing goals when using the table, as you can already see by the modified TD targets above. This becomes more important when adding look-ahead planning, which is a common addition and can significantly improve the performance of an agent - during look-ahead you must switch between player views on what the best action choice is. It is possible to make off-by-one errors in some part of the code but not others and have the agent learn inconsistently.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-02T10:33:02.043" LastActivityDate="2020-05-02T10:33:02.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20836" PostTypeId="1" CreationDate="2020-05-01T19:31:56.960" Score="1" ViewCount="26" Body="&lt;p&gt;Assume we have a policy &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta}$&lt;/span&gt; in a classic reinforcement learning setting, and a reward function &lt;span class=&quot;math-container&quot;&gt;$R^{\pi}(s,a)$&lt;/span&gt; that changes as long as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; changes i.e. not only is it predefined by the environment itself, how can we model the popular algorithms (e.g. SAC) according to this change?&lt;/p&gt;&#xA;" OwnerUserId="36603" LastActivityDate="2020-05-01T19:31:56.960" Title="What if the rewards induced by an environment are related to the policy too?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;environment&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20839" PostTypeId="1" CreationDate="2020-05-01T23:16:29.457" Score="1" ViewCount="38" Body="&lt;p&gt;When using a trained Q-learning algorithm in an actual game, would I just use exploitation and no longer use exploration? Should I use exploration only during the training phase?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-05-01T23:23:45.433" LastActivityDate="2020-05-01T23:42:21.007" Title="Should I just use exploitation after I have trained the Q agent?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;q-learning&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20840" PostTypeId="5" CreationDate="2020-05-01T23:24:54.760" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-01T23:24:54.760" LastActivityDate="2020-05-01T23:24:54.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20841" PostTypeId="4" CreationDate="2020-05-01T23:24:54.760" Score="0" Body="For questions related to the exploration-exploitation trade-off (or dilemma) in the context of reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-01T23:24:54.760" LastActivityDate="2020-05-01T23:24:54.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20843" PostTypeId="2" ParentId="20839" CreationDate="2020-05-01T23:35:54.573" Score="1" Body="&lt;p&gt;Once you have estimated the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, you can derive the policy from it in different ways. For example, you can act greedily with respect to it (see &lt;a href=&quot;https://ai.stackexchange.com/a/13363/2444&quot;&gt;this answer&lt;/a&gt;), which can be formally denoted as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi(s) = \operatorname{argmax}_{a^*}Q(s, a), \; \forall s \in \mathcal{S}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; is your estimated value function and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; the policy greedily derived from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that you would just exploit your current knowledge of the return. This is probably a good thing to do if you believe your value function is optimal and the dynamics of the environment don't change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if your policy is not optimal, you may not want to always execute the greedy action. In that case, you could still perform some form of exploration (e.g. with the &lt;a href=&quot;https://ai.stackexchange.com/a/10495/2444&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if the dynamics (e.g. the reward function) of your environment change over time, you could continually train your RL agent. If you are interested in continual RL, the paper &lt;a href=&quot;https://arxiv.org/pdf/1802.07239.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continual Reinforcement Learning with Complex Synapses&lt;/a&gt; (2018) could potentially be useful.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-01T23:42:21.007" LastActivityDate="2020-05-01T23:42:21.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20844" PostTypeId="1" CreationDate="2020-05-02T03:59:38.023" Score="0" ViewCount="42" Body="&lt;p&gt;I have a very silly problem in hand. I have implemented 2 methods which give me the mask to separate the objects from the background. What I get from one method is the object encapsulated in the red Contour or boundary and the other one makes the background Red. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Keras to classify Trash. I wanted to use the output or the &lt;code&gt;EXTRACTED&lt;/code&gt; object as an input to the CNN model. Now I do not see any difference in the images and output. All there is extra boundary around the object and I fail to understand how it can help my model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could add an extra &lt;code&gt;alpha&lt;/code&gt; channel to second method to make the background transparent but Keras' &lt;code&gt;ImageDataGenerator&lt;/code&gt; do not work with &lt;code&gt;RGBA&lt;/code&gt; images. What shoul dI do to improve the model?&lt;/p&gt;&#xA;" OwnerUserId="36062" LastActivityDate="2021-01-30T21:03:50.470" Title="How to use 'Canny/Watershed' algorithm's output as an input for Image Classification Model" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;keras&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20845" PostTypeId="1" CreationDate="2020-05-02T04:41:17.237" Score="1" ViewCount="26" Body="&lt;p&gt;To give a little background, I've been reading the &lt;a href=&quot;https://arxiv.org/abs/1905.09275&quot; rel=&quot;nofollow noreferrer&quot;&gt;COBRA paper&lt;/a&gt;, and I've reached the section that talks about the exploration policy, in particular. We figure that a uniformly random policy won't do us any good, since the action space is sparsely populated with objects that the agent must act upon - and a random action is likely to result in no change (an object here occupies only about 1.7% of the space of the screen). Hence we need our agent to learn in the exploration phase a policy that clicks on and moves objects more frequently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get that a random policy won't work, but I've difficulty understanding how and why the transition model is trained adversarially. Following is the extract which talks about the same, and I've highlighted parts that I don't completely understand - &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Our approach is to &lt;strong&gt;train the transition model adversarially with an exploration policy that learns to take actions on which the transition model has a high error.&lt;/strong&gt; Such difficult-to-predict actions should be those that move objects (given that others would leave the scene unchanged). In this way the exploration policy and the transition model &lt;strong&gt;learn together in a virtuous cycle.&lt;/strong&gt; This approach is a form of curiosity-driven exploration, as previously described in both the psychology (Gopnik et al., 1999) and the reinforcement learning literature (Pathak et al., 2017; Schmidhuber, 1990a,b).&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How does it help to take actions on which the transition model has a&#xA;high error?&lt;/li&gt;&#xA;&lt;li&gt;I don't exactly see how a virtuous cycle is in action&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Could someone please explain? Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-05-02T04:41:17.237" Title="How can transition models in RL be trained adversarially?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;adversarial-ml&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20846" PostTypeId="1" CreationDate="2020-05-02T08:51:18.090" Score="1" ViewCount="66" Body="&lt;p&gt;I am wondering how can DDPG or DPG handle the discrete action space. There are some papers saying that use Gumbel softmax with DDPG can make the discrete action problem be solved. However, will the Gumbel softmax make the deterministic policy be the stochastic one? If not, how can that be achieved?&lt;/p&gt;&#xA;" OwnerUserId="36587" LastActivityDate="2020-05-02T08:51:18.090" Title="How can DDPG handle the discrete action space?" Tags="&lt;reinforcement-learning&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20847" PostTypeId="1" AcceptedAnswerId="20849" CreationDate="2020-05-02T11:56:09.327" Score="1" ViewCount="47" Body="&lt;p&gt;What are multi-hop relational paths in the context of knowledge graphs (KGs)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried looking it up online, but didn't find a simple explanation.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-05-02T17:58:31.337" LastActivityDate="2020-05-02T18:22:30.550" Title="What are multi-hop relational paths?" Tags="&lt;recommender-system&gt;&lt;knowledge-graph&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20849" PostTypeId="2" ParentId="20847" CreationDate="2020-05-02T13:49:12.940" Score="1" Body="&lt;p&gt;Before trying to explain this term in your context, let me briefly describe the term in other contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In computer networking, the term &quot;hop&quot; refers to a node (e.g. a router) that a packet goes through before reaching its destination from its source. In a multi-hop situation, you have several nodes involved in the process of sending the packet from the source to the destination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://arxiv.org/pdf/2003.02320.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;knowledge graph&lt;/a&gt; is a graph that accumulates and conveys knowledge of the real world, where nodes represent entities of interest and edges relations between those entities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, multi-hop relational paths are probably relational paths involving more than one node or edge in the knowledge graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;But what do we mean by &quot;relational&quot;?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are familiar with the basics of databases, the word &quot;relational&quot; shouldn't be so unfamiliar. In fact, there the so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Relational_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;relational databases&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Relational_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;relational models&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Relational_algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;relational algebra&lt;/a&gt;. Intuitively, the word &quot;relational&quot; is used to denote what you think it denotes, i.e. relations. See also &lt;a href=&quot;https://www.oracle.com/database/what-is-a-relational-database/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What a relational database is&lt;/a&gt; by Oracle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;And what is a path?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section 2.2.3 of the tutorial &lt;a href=&quot;https://arxiv.org/pdf/2003.02320.pdf#page=10&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge Graphs&lt;/a&gt;, Aidan Hogan et al. provide a description of a path (expressions) in the context of knowledge graphs&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Navigational graph patterns. A key feature that distinguishes graph query languages is the ability to include path expressions in queries. A path expression &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is a regular expression that allows matching arbitrary-length paths between two nodes, which is expressed as a regular path query &lt;span class=&quot;math-container&quot;&gt;$(x,r,y)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; can be variables or constants (or even the same term).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-02T18:22:30.550" LastActivityDate="2020-05-02T18:22:30.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20850" PostTypeId="1" AcceptedAnswerId="20861" CreationDate="2020-05-02T14:42:42.893" Score="3" ViewCount="169" Body="&lt;p&gt;If a research paper uses multi-armed bandits (either in their standard or contextual form) to solve a particular task, can we say that they solved this task using a reinforcement learning approach? Or should we distinguish between the two and use the RL term only when it is associated with an MDP formulation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, each RL course/textbook usually contains a section about bandits (especially when dealing with the exploration-exploitation tradeoff). Additionally, bandits also have the concept of actions and rewards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just want to make sure what the right terminology should be, when describing either approach.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-05-03T15:06:37.607" Title="Are bandits considered an RL approach?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;multi-armed-bandit&gt;&lt;contextual-bandit&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20852" PostTypeId="1" AcceptedAnswerId="20863" CreationDate="2020-05-02T15:09:08.137" Score="1" ViewCount="68" Body="&lt;p&gt;I've been reading &lt;a href=&quot;https://arxiv.org/pdf/1906.05237.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper on recommendation systems&lt;/a&gt; using reinforcement learning (RL) and knowledge graphs (KGs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give some background, the graph has several (finitely many) entities, of which some are user entities and others are item entities. The goal is to recommend items to users, i.e. to find a recommendation set of items for every user such that the user and the corresponding items are connected by one reasoning path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm attaching an example of such a graph for more clarity (from the paper itself) -&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&lt;a href=&quot;https://i.stack.imgur.com/zhdCC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zhdCC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the paper above, they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;First, we do not have pre-defined targeted items for any user, so it is &lt;strong&gt;not applicable to use a binary reward&lt;/strong&gt; indicating whether the user interacts with the item or not. A better design of the reward function is to &lt;em&gt;incorporate the uncertainty&lt;/em&gt; of how an item is relevant to a user based on the rich heterogeneous information given by the knowledge graph.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not able to understand the above extract, which talks about the reward function to use - binary, or something else. A detailed explanation of what the author is trying to convey in the above extract would really help. &lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-05-02T16:29:28.463" LastActivityDate="2020-05-03T05:50:05.840" Title="Which reward function works for recommendation systems using knowledge graphs?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;rewards&gt;&lt;recommender-system&gt;&lt;knowledge-graph&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20854" PostTypeId="5" CreationDate="2020-05-02T16:33:38.750" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-02T16:33:38.750" LastActivityDate="2020-05-02T16:33:38.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20855" PostTypeId="4" CreationDate="2020-05-02T16:33:38.750" Score="0" Body="For questions related to the concept of a knowledge graph, which is a knowledge base represented as a graph that accumulates and conveys knowledge of the real world, where nodes represent entities and edges represent relations between those entities." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-02T16:33:38.750" LastActivityDate="2020-05-02T16:33:38.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20858" PostTypeId="1" AcceptedAnswerId="20859" CreationDate="2020-05-02T20:18:04.490" Score="1" ViewCount="36" Body="&lt;p&gt;I am trying to apply &lt;code&gt;Eligibility Traces&lt;/code&gt; to a currently working Q-Learning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reference code for the Q-Learning algorithm was taken from &lt;a href=&quot;https://deeplizard.com/learn/video/HGeI30uATws&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; great blog by DeepLizard, but does not include &lt;code&gt;Eligibility Traces&lt;/code&gt;. &lt;a href=&quot;https://colab.research.google.com/drive/1CHRhxjW4XejFy-Ae8OAKyxd8se_rcx74&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to the code&lt;/a&gt; on Google Colab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to add the &lt;code&gt;Eligibility Traces&lt;/code&gt; by implementing this pseud code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Initialize Q(s,a) arbitrarily and e(s,a) = 0, for all s,a&#xA;Repeat (for each episode):&#xA;    Initialize s,a&#xA;    Repeat (for each step of episode):&#xA;        Take action a, observe r,s’&#xA;        Choose a’ from s’ using policy derived from Q (e.g., ϵ-greedy)&#xA;        δ ← r + γ Q(s’,a’) – Q(s,a)&#xA;        e(s,a) ← e(s,a) + 1&#xA;        For all s,a:&#xA;            Q(s,a) ← Q(s,a) + α δ e(s,a)&#xA;            e(s,a) ← γ λ e(s,a)&#xA;        s ← s’ ; a ← a’&#xA;    until s is terminal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Taken from &lt;a href=&quot;https://github.com/Hysperr/gridworld&quot; rel=&quot;nofollow noreferrer&quot;&gt;HERE&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code as I have implemented the pseudo-code - &lt;a href=&quot;https://colab.research.google.com/drive/1inWv4TUB6b81o0PShb1LDWYwz9pQS0Jf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The part that needs to be improved is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Q learning algorithem&#xA;for episode in range(num_episodes):&#xA;  state = env.reset()&#xA;  et_table = np.zeros((state_space_size,action_space_size))&#xA;  done = False&#xA;  reward_current_episode = 0&#xA;&#xA;  for steps in range(max_steps_per_episode):&#xA;    #Exploration-Explotation trade-off&#xA;    exploration_rate_thresh = random.uniform(0,1)&#xA;    if exploration_rate_thresh &amp;gt; exploration_rate:&#xA;      action = np.argmax(q_table[state,:])&#xA;    else:&#xA;      action = env.action_space.sample()&#xA;&#xA;    new_state, reward, done, info = env.step(action)&#xA;&#xA;    #Update Q-table and Eligibility table&#xA;    delta = reward + discount_rate * np.max(q_table[new_state,:]) - q_table[state,action]&#xA;    et_table[state, action] = et_table[state, action] + 1&#xA;&#xA;    for update_state in range(state_space_size):&#xA;      for update_action in range(action_space_size):&#xA;        q_table[update_state, update_action] = q_table[update_state, update_action] + learning_rate * delta * et_table[update_state, update_action]&#xA;        et_table[update_state, update_action] = discount_rate * gamma * et_table[update_state, update_action]&#xA;&#xA;    state = new_state&#xA;    reward_current_episode = reward&#xA;&#xA;    if done==True:&#xA;      break&#xA;&#xA;  #Exploration rate decay&#xA;  exploration_rate = min_exploration_rate + (max_exploration_rate - min_exploration_rate) * np.exp(-exploration_decay_rate*episode)&#xA;&#xA;  rewards_all_episodes.append(reward_current_episode)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a while, I was getting pure results (avg. rewards for 1000 episodes were around 0.14 while the original NON-ET algorithm was averaging 0.69 on the last 1000 episodes), but now I get these errors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:27: RuntimeWarning: overflow encountered in double_scalars&#xA;/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:22: RuntimeWarning: invalid value encountered in double_scalars&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36702" LastActivityDate="2020-05-02T20:35:08.377" Title="Applying Eligibility Traces to Q-Learning algorithm does not improve results (And might not function well)" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;unsupervised-learning&gt;&lt;eligibility-traces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20859" PostTypeId="2" ParentId="20858" CreationDate="2020-05-02T20:35:08.377" Score="0" Body="&lt;p&gt;The thing is, that while I was posting the question, I tried to tweak with the parameters and it seems that my &lt;code&gt;discount rate&lt;/code&gt; (set for 0.99) was causing these errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also - it seems that the &lt;code&gt;is_slippery&lt;/code&gt; argument passed to the environment (True - the agent's action will be fulfilled 33% of the times, the rest of the times will be in a random direction, False - the agent's action will be fulfilled 100% of the times) is crucial for &lt;code&gt;Eligibility Traces&lt;/code&gt;. The results improve from 0.39 to 0.993.&#xA;My assumption is that the randomness of the &quot;Slippery&quot; part of the environment is crucially hurting the Eligibility Traces due to the fact that it is relying on the actions that were taken (and assuming they were fulfilled and not randomly changed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After changing it to 0.9 I receive these results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;*** Average award per 1000 episodes ***&#xA;1000 :  0.2640000000000002&#xA;2000 :  0.7560000000000006&#xA;3000 :  0.9140000000000007&#xA;4000 :  0.9600000000000007&#xA;5000 :  0.9820000000000008&#xA;6000 :  0.9870000000000008&#xA;7000 :  0.9860000000000008&#xA;8000 :  0.9830000000000008&#xA;9000 :  0.9930000000000008&#xA;10000 :  0.9930000000000008&#xA;&#xA;&#xA;******* Q-Table *******&#xA;&#xA;[[0.53144572 0.59049007 0.59046218 0.53144103]&#xA; [0.53292711 0.         0.65610941 0.5864814 ]&#xA; [0.58892359 0.729037   0.59954581 0.6546992 ]&#xA; [0.65324248 0.         0.53612877 0.55540024]&#xA; [0.59049001 0.65610007 0.         0.53143728]&#xA; [0.         0.         0.         0.        ]&#xA; [0.         0.8099945  0.         0.65015572]&#xA; [0.         0.         0.         0.        ]&#xA; [0.65610004 0.         0.72900006 0.59049002]&#xA; [0.65610024 0.81       0.80996651 0.        ]&#xA; [0.73156036 0.9        0.         0.72830918]&#xA; [0.         0.         0.         0.        ]&#xA; [0.         0.         0.         0.        ]&#xA; [0.         0.8100008  0.9        0.72900011]&#xA; [0.81000015 0.89999998 1.         0.80998688]&#xA; [0.         0.         0.         0.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Comparing it to the original !-Learning algorithm by DeepLizard (&lt;a href=&quot;https://deeplizard.com/learn/video/HGeI30uATws&quot; rel=&quot;nofollow noreferrer&quot;&gt;LINK&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;*** Average award per 1000 episodes ***&#xA;1000 :  0.22900000000000018&#xA;2000 :  0.7250000000000005&#xA;3000 :  0.8990000000000007&#xA;4000 :  0.9610000000000007&#xA;5000 :  0.9860000000000008&#xA;6000 :  0.9900000000000008&#xA;7000 :  0.9870000000000008&#xA;8000 :  0.9970000000000008&#xA;9000 :  0.9870000000000008&#xA;10000 :  0.9890000000000008&#xA;&#xA;&#xA;******* Q-Table *******&#xA;&#xA;[[0.94148015 0.95099005 0.93206533 0.94148015]&#xA; [0.94148015 0.         0.71778349 0.84678222]&#xA; [0.88367728 0.43945604 0.0084652  0.27374571]&#xA; [0.15097126 0.         0.         0.        ]&#xA; [0.95099005 0.96059601 0.         0.94148015]&#xA; [0.         0.         0.         0.        ]&#xA; [0.         0.98009092 0.         0.36957636]&#xA; [0.         0.         0.         0.        ]&#xA; [0.96059599 0.         0.970299   0.95099004]&#xA; [0.96059598 0.98009932 0.9801     0.        ]&#xA; [0.97029894 0.99       0.         0.97021962]&#xA; [0.         0.         0.         0.        ]&#xA; [0.         0.         0.         0.        ]&#xA; [0.         0.89329522 0.99       0.91080933]&#xA; [0.98009934 0.98999961 1.         0.98009962]&#xA; [0.         0.         0.         0.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We see a bit better results (we can see that on the 10K episode our agent is making the right path at 0.993% of the times, in appose to 0.989% in the original algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottom line is that the &lt;code&gt;Eligibility Traces&lt;/code&gt; implementation can be done using these formulas:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a. create the et_table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;et_table = np.zeros((state_space_size,action_space_size))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;b. Set gamma for the decaying of the Eligibility Traces:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gamma = 0.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;c. At each step - apply these calculations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Update Q-table and Eligibility table&#xA;    delta = reward + discount_rate * np.max(q_table[new_state,:]) - q_table[state,action]&#xA;    et_table[state, action] = et_table[state, action] + 1&#xA;&#xA;    for update_state in range(state_space_size):&#xA;      for update_action in range(action_space_size):&#xA;        q_table[update_state, update_action] = q_table[update_state, update_action] + learning_rate * delta * et_table[update_state, update_action]&#xA;        et_table[update_state, update_action] = discount_rate * gamma * et_table[update_state, update_action]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;d. After each episode - reset the et_table to zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Full code can be found here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import gym&#xA;import random&#xA;import time&#xA;from IPython.display import clear_output&#xA;&#xA;env = gym.make(&quot;FrozenLake-v0&quot;, is_slippery=False)&#xA;&#xA;action_space_size = env.action_space.n&#xA;state_space_size = env.observation_space.n&#xA;&#xA;q_table = np.zeros((state_space_size,action_space_size))&#xA;et_table = np.zeros((state_space_size,action_space_size))&#xA;print(&quot;Q Table:\n&quot;, q_table)&#xA;print(&quot;Eligibility Traces:\n&quot;, et_table)&#xA;&#xA;num_episodes = 10000&#xA;max_steps_per_episode = 100&#xA;&#xA;learning_rate = 0.1&#xA;discount_rate = 0.9&#xA;gamma = 0.9&#xA;&#xA;exploration_rate = 1&#xA;max_exploration_rate = 1&#xA;min_exploration_rate = 0.01&#xA;exploration_decay_rate = 0.001&#xA;&#xA;rewards_all_episodes = []&#xA;q_table = np.zeros((state_space_size,action_space_size))&#xA;&#xA;#Q learning algorithem&#xA;for episode in range(num_episodes):&#xA;  state = env.reset()&#xA;  et_table = np.zeros((state_space_size,action_space_size))&#xA;  done = False&#xA;  reward_current_episode = 0&#xA;&#xA;  for steps in range(max_steps_per_episode):&#xA;    #Exploration-Explotation trade-off&#xA;    exploration_rate_thresh = random.uniform(0,1)&#xA;    if exploration_rate_thresh &amp;gt; exploration_rate:&#xA;      action = np.argmax(q_table[state,:])&#xA;    else:&#xA;      action = env.action_space.sample()&#xA;&#xA;    new_state, reward, done, info = env.step(action)&#xA;&#xA;    #Update Q-table and Eligibility table&#xA;    delta = reward + discount_rate * np.max(q_table[new_state,:]) - q_table[state,action]&#xA;    et_table[state, action] = et_table[state, action] + 1&#xA;&#xA;    for update_state in range(state_space_size):&#xA;      for update_action in range(action_space_size):&#xA;        q_table[update_state, update_action] = q_table[update_state, update_action] + learning_rate * delta * et_table[update_state, update_action]&#xA;        et_table[update_state, update_action] = discount_rate * gamma * et_table[update_state, update_action]&#xA;&#xA;    state = new_state&#xA;    reward_current_episode = reward&#xA;&#xA;    if done==True:&#xA;      break&#xA;&#xA;  #Exploration rate decay&#xA;  exploration_rate = min_exploration_rate + (max_exploration_rate - min_exploration_rate) * np.exp(-exploration_decay_rate*episode)&#xA;&#xA;  rewards_all_episodes.append(reward_current_episode)&#xA;&#xA;#Print average reward per thousend episodes&#xA;reward_per_thousend_episodes = np.split(np.array(rewards_all_episodes),num_episodes/1000)&#xA;count = 1000&#xA;print(&quot;*** Average award per 1000 episodes ***&quot;)&#xA;for r in reward_per_thousend_episodes:&#xA;  print(count, &quot;: &quot;, str(sum(r/1000)))&#xA;  count+=1000&#xA;&#xA;#Print Q-Table&#xA;print(&quot;\n\n******* Q-Table *******\n&quot;)&#xA;print(q_table)&#xA;&#xA;print(&quot;\n\n******* ET-Table *******\n&quot;)&#xA;print(et_table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36702" LastActivityDate="2020-05-02T20:35:08.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20860" PostTypeId="2" ParentId="18125" CreationDate="2020-05-02T21:34:10.770" Score="-1" Body="&lt;p&gt;It should be crystal clear to those who understand the word &amp;quot;stuck&amp;quot; that the term &amp;quot;get stuck&amp;quot; means, in this context, to either&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;reach an answer that appears to be an optimum before reaching an optimum and stopping or&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;never stopping.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The answer to the question is simply 1.&lt;/p&gt;&#xA;&lt;p&gt;If 4. is the global maximum, that it is a plateau is irrelevant.  The value (height) function finds all values on the surface of the plateau equal in value.&lt;/p&gt;&#xA;&lt;p&gt;If 3. is encountered but it is not a local maximum, the gradient would direct ascension around the ridge unless the ridge were perfectly perpendicular to the normal line.&lt;/p&gt;&#xA;&lt;p&gt;If a logical system were added to the inversion (gradient ascent) of gradient descent to augment it in some way, which is the only way 2. would make sense, it would be inadmissible to allow the insertion of rules, heuristic or not, that were inadmissible, so 2. applies only if the system designer was of the kind that does not understand fault resistant systems architecture.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="2444" LastEditDate="2020-11-24T02:35:24.453" LastActivityDate="2020-11-24T02:35:24.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20861" PostTypeId="2" ParentId="20850" CreationDate="2020-05-02T22:55:58.180" Score="2" Body="&lt;p&gt;Several important researchers distinguish between bandit problems and the general reinforcement learning problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=45&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning: an introduction&lt;/a&gt; by Sutton and Barto describes bandit problems as a special case of the general RL problem.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The first chapter of this part of the book describes solution methods for the &lt;strong&gt;special case of the reinforcement learning problem&lt;/strong&gt; in which there is &lt;strong&gt;only a single state&lt;/strong&gt;, called &lt;strong&gt;bandit problems&lt;/strong&gt;. The second chapter describes the &lt;strong&gt;general problem formulation&lt;/strong&gt; that we treat throughout the rest of the book — &lt;strong&gt;finite Markov decision processes&lt;/strong&gt; — and its main ideas including Bellman equations and value functions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf#page=21&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 1.1.2 of the book Bandit Algorithms&lt;/a&gt; (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One of the distinguishing features of all bandit problems studied in this book is that &lt;strong&gt;the learner never needs to plan for the future&lt;/strong&gt;. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. &lt;strong&gt;Problems that do require this kind of long-term planning fall into the realm of reinforcement learning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This definition is different than the one by Sutton and Barto. In this case, only bandit problems where the learner doesn't need to plan for the future are considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off and, in both cases, the underlying problem can be formulated as a Markov decision process.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-03T00:28:24.240" LastActivityDate="2020-05-03T00:28:24.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20863" PostTypeId="2" ParentId="20852" CreationDate="2020-05-03T05:50:05.840" Score="2" Body="&lt;p&gt;I found the answer further into the paper! I'll post it here for everyone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Given any user, there is no pre-known targeted item in the KGRE-Rec (Knowledge Graph Reasoning for Explainable Recommendation) problem, so it is unfeasible to consider binary rewards indicating whether the agent has reached a target or not.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, the agent is encouraged to explore as many “good” paths&#xA;as possible. In the context of recommendations, &lt;strong&gt;a “good” path is one that leads to an item that a user will interact with, with high probability.&lt;/strong&gt; To this end, a soft reward is given only for the terminal state &lt;span class=&quot;math-container&quot;&gt;$s_T = (u,e_T,h_T)$&lt;/span&gt; based on a scoring function &lt;span class=&quot;math-container&quot;&gt;$f(u,i)$&lt;/span&gt;. The terminal reward &lt;span class=&quot;math-container&quot;&gt;$R_T$&lt;/span&gt; is given as - &#xA;&lt;a href=&quot;https://i.stack.imgur.com/aXVC7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aXVC7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the value of &lt;span class=&quot;math-container&quot;&gt;$R_T$&lt;/span&gt; is normalized to the range &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;. Details about &lt;span class=&quot;math-container&quot;&gt;$f(u,i)$&lt;/span&gt;, the scoring function, can be found in the paper.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References:&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1906.05237&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Knowledge Graph Reasoning for Explainable&#xA;Recommendation&lt;/a&gt; [Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard&#xA;de Melo, Yongfeng Zhang]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-05-03T05:50:05.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20864" PostTypeId="1" AcceptedAnswerId="20867" CreationDate="2020-05-03T06:11:06.840" Score="3" ViewCount="39" Body="&lt;p&gt;I am learning RL for the first time. It may be naive, but it is a bit odd to grasp this idea that, if the goal of RL is to maximize the expected return, then shouldn't the expected return be calculated for some faraway time in the future (&lt;span class=&quot;math-container&quot;&gt;$t+n$&lt;/span&gt;) instead of current time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;? It is because we are building our system for the future using current information. ( I am coming from machine learning background and this makes more sense to me).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, the expected return is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[G_t] = \mathbb{E}[ R_{t+1} + R_{t+2} + R_{t+3}+... R_{t+n}]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, shouldn't the expected return be:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[G_{t+n}] = \mathbb{E}[R_{t+1} + R_{t+2} + R_{t+3}+... R_{t+n-1}]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="36710" LastEditorUserId="2444" LastEditDate="2020-05-03T12:00:34.960" LastActivityDate="2020-05-04T07:15:07.057" Title="Shouldn't expected return be calculated for some faraway time in the future $t+n$ instead of current time $t$?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;expectation&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20866" PostTypeId="1" CreationDate="2020-05-03T07:13:22.927" Score="1" ViewCount="55" Body="&lt;p&gt;I have been trying to write code to implement plain neural net without convolution from scratch. I took some help online &lt;a href=&quot;https://towardsdatascience.com/building-neural-network-from-scratch-9c88535bf8e9&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and added my code to my github &lt;a href=&quot;https://github.com/shekh01ml/mnist_ann.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;account&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why the prediction made by my code is only 88%-90% accurate after the 1st epoch, whereas his code is 95% accurate after 1st epoch with the same parameters (Same Xavier initialization for weights, biases are not initialized, same hidden layer neurons). While his architecture uses 2 hidden layers, my code performed worse with 2 hidden layers. For 1 hidden layer, his code performs similar (~96%). &lt;/p&gt;&#xA;" OwnerUserId="33029" LastActivityDate="2020-05-03T07:13:22.927" Title="MNIST Classification code performing with 88%-90% whereas other codes online perform 95% on first epoch" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20867" PostTypeId="2" ParentId="20864" CreationDate="2020-05-03T08:08:52.047" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;shouldn't the expected return be calculated for some faraway time in the future (+) instead of current time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is partly a notation issue, but &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; is &lt;em&gt;already&lt;/em&gt; the future sum of rewards as seen by the first (and correct) equation in your question. You don't actually know the value of any individual return &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt;* until after &lt;span class=&quot;math-container&quot;&gt;$t+n$&lt;/span&gt;. However, you can predict the expected value &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[G_t]$&lt;/span&gt; provided the environment and policy remain consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your second equation is pretty similar to the first one, which is why I say it is partly a notation issue. However, the point of calculating the forward-looking expectation for the return is to allow for assessment at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. This is to either predict likely outcome for future rewards knowing the system is in state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, or in control scenarios to choose an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, in both prediction and control scenarios, the past (from time steps &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t-1$&lt;/span&gt;) has already happened. Measurements of how well a system did in terms of gathering reward, looking backwards, might be useful metrics for e.g. &quot;how good is this agent?&quot;. However, they are not in general a guide to the future. In many sparse environments (e.g. a board game scoring +1 for a win), this data is basically useless for predicting the future and all you want to know is summarised by the current state and the acting policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of when you get to finally calculate a return, the start time step - the part of the trajectory that the return is calculated &lt;em&gt;from&lt;/em&gt;, is a key parameter. The end time step is a practical concern for implementation, but can often be considered to be at infinity for theoretical purposes (i.e. we are interested in measuring or optimising all future rewards). So if you are only going to display/use one parameter in the notation, the start time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the one to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are variants of notation, to show how the return is calculated, where the calculation &lt;em&gt;horizon&lt;/em&gt; is made explicit, e.g. &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}$&lt;/span&gt; for a &lt;em&gt;truncated&lt;/em&gt; return or &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+1}$&lt;/span&gt; when calculating a one step temporal difference target. All the ones I have seen still maintain the forward view that defines a value associated with a current timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, for the same reason as explained above - it is at timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; where this value is of most interest as a prediction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice during training you often wait until &lt;span class=&quot;math-container&quot;&gt;$t+n$&lt;/span&gt; before you know the correct value of &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt;* to apply as a training value - which is then used to update value estimates for &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(s_t)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s_t, a_t)$&lt;/span&gt;. It is possible to make partial updates before that ending time step using techniques such as &lt;a href=&quot;https://towardsdatascience.com/eligibility-traces-in-reinforcement-learning-a6b458c019d6&quot; rel=&quot;nofollow noreferrer&quot;&gt;eligibility traces&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Using notation of uppercase &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; for random variable, and lowercase &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; for a measured value.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-04T07:15:07.057" LastActivityDate="2020-05-04T07:15:07.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20868" PostTypeId="1" CreationDate="2020-05-03T08:15:07.343" Score="3" ViewCount="46" Body="&lt;p&gt;Recently I was reading &lt;a href=&quot;https://arxiv.org/abs/1909.06694&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt; based on a new evaluation metric SIMILE. In a section, validation loss comparison had been made for SIMILE and BLEU. The plot showed the expected BLEU cost when training with BLEU and SIMILE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm unable to understand is what is meant by the expected BLEU cost when training with BLEU and SIMILE? Are there any separate cost functions defined for these scores?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm attaching the image of the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A2m0N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A2m0N.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36711" LastEditorUserId="2212" LastEditDate="2020-05-03T20:26:25.067" LastActivityDate="2021-05-28T22:07:10.707" Title="What is meant by the expected BLEU cost when training with BLEU and SIMILE?" Tags="&lt;natural-language-processing&gt;&lt;training&gt;&lt;metric&gt;&lt;expectation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20869" PostTypeId="1" CreationDate="2020-05-03T09:55:09.727" Score="1" ViewCount="106" Body="&lt;p&gt;I have the following three rules &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If &lt;code&gt;A&lt;/code&gt; is medium and &lt;code&gt;B&lt;/code&gt; is low then quality is low&lt;/li&gt;&#xA;&lt;li&gt;If &lt;code&gt;A&lt;/code&gt; is medium and &lt;code&gt;B&lt;/code&gt; is medium then quality is medium&lt;/li&gt;&#xA;&lt;li&gt;If &lt;code&gt;A&lt;/code&gt; is medium and &lt;code&gt;B&lt;/code&gt; is high then quality is medium &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With this link as reference &lt;a href=&quot;https://pythonhosted.org/scikit-fuzzy/auto_examples/plot_tipping_problem.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pythonhosted.org/scikit-fuzzy/auto_examples/plot_tipping_problem.html&lt;/a&gt; and the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# We need the activation of our fuzzy membership functions at these values.&#xA;# The exact values 6.5 and 9.8 do not exist on our universes...&#xA;# This is what fuzz.interp_membership exists for!&#xA;qual_level_lo = fuzz.interp_membership(x_qual, qual_lo, 6.5)&#xA;qual_level_md = fuzz.interp_membership(x_qual, qual_md, 6.5)&#xA;qual_level_hi = fuzz.interp_membership(x_qual, qual_hi, 6.5)&#xA;&#xA;serv_level_lo = fuzz.interp_membership(x_serv, serv_lo, 9.8)&#xA;serv_level_md = fuzz.interp_membership(x_serv, serv_md, 9.8)&#xA;serv_level_hi = fuzz.interp_membership(x_serv, serv_hi, 9.8)&#xA;&#xA;# Now we take our rules and apply them. Rule 1 concerns bad food OR service.&#xA;# The OR operator means we take the maximum of these two.&#xA;active_rule1 = np.fmax(qual_level_lo, serv_level_lo)&#xA;&#xA;# Now we apply this by clipping the top off the corresponding output&#xA;# membership function with `np.fmin`&#xA;tip_activation_lo = np.fmin(active_rule1, tip_lo)  # removed entirely to 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I write the same kind of rules for &lt;strong&gt;AND&lt;/strong&gt; operator with respect to my rule?&lt;/p&gt;&#xA;" OwnerUserId="36495" LastEditorUserId="2444" LastEditDate="2020-05-03T16:16:57.843" LastActivityDate="2020-05-03T16:16:57.843" Title="How can I write these rules with the AND operator in skfuzzy?" Tags="&lt;python&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2020-05-03T22:41:48.990" ContentLicense="CC BY-SA 4.0" />
  <row Id="20870" PostTypeId="1" AcceptedAnswerId="21089" CreationDate="2020-05-03T12:54:02.367" Score="1" ViewCount="64" Body="&lt;p&gt;I'm looking at the Bernoulli naïve Bayes classifier on Wikipedia and I understand Bayes theorem along with Gaussian naïve Bayes, however when looking at how &lt;span class=&quot;math-container&quot;&gt;$P(x|c_k)$&lt;/span&gt; is calculated I don't understand it. The Wikipedia page says its calculated as follows &lt;span class=&quot;math-container&quot;&gt;$P(x|c_k) = \prod^{n}_{i=1} p^{x_i}_{ki} (1-p_{ki})^{(1-x_i)} $&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They mention that &lt;span class=&quot;math-container&quot;&gt;$p_{ki}$&lt;/span&gt; is the probability of class &lt;span class=&quot;math-container&quot;&gt;$c_k$&lt;/span&gt; generating the term &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, does that mean &lt;span class=&quot;math-container&quot;&gt;$P(x|c_k)$&lt;/span&gt;? Because if so then that doesn't make sense since to calculate that we need to have calculated it already. So what is &lt;span class=&quot;math-container&quot;&gt;$p_{ki}$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in the first part, after the product symbol, are they raising this probability to the power pf &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; or does that again just mean 'probability of class &lt;span class=&quot;math-container&quot;&gt;$c_k$&lt;/span&gt; generating the term &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;'?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also don't understand the intuition behind why or how this calculates &lt;span class=&quot;math-container&quot;&gt;$P(x|c_i)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help everyone.&lt;/p&gt;&#xA;" OwnerUserId="20736" LastEditorUserId="20736" LastEditDate="2020-05-10T08:36:59.093" LastActivityDate="2020-05-11T23:05:14.407" Title="Understanding how to calculate $P(x|c_k)$ for the Bernoulli naïve Bayes classifier" Tags="&lt;bayes&gt;&lt;naive-bayes&gt;&lt;bayesian-probability&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20871" PostTypeId="1" AcceptedAnswerId="20900" CreationDate="2020-05-03T13:09:01.787" Score="4" ViewCount="194" Body="&lt;pre&gt;&lt;code&gt; make_env = lambda: ptan.common.wrappers.wrap_dqn(gym.make(&quot;PongNoFrameskip-v4&quot;))&#xA; envs = [make_env() for _ in range(NUM_ENVS)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On/blob/master/Chapter10/02_pong_a2c.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; you can look at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two above lines of code create multiple environments for the game of Atari Pong with the A2C algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand why it is very useful to have multiple agents working on different instances of the same environment as it is presented in A3C (i.e. an asynchronous version of A2C). However, in the above code, it has a single agent working on different instances of the same environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the advantage of using more than one environment with a single agent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class GymEnvVec:&#xA;    def __init__(self, name, n_envs, seed):&#xA;        self.envs = [gym.make(name) for i in range(n_envs)]&#xA;        [env.seed(seed + 10 * i) for i, env in enumerate(self.envs)]&#xA;&#xA;    def reset(self):&#xA;        return [env.reset() for env in self.envs]&#xA;&#xA;    def step(self, actions):&#xA;        return list(zip(*[env.step(a) for env, a in zip(self.envs, actions)]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-11T14:26:18.507" LastActivityDate="2020-05-11T14:26:18.507" Title="What is the advantage of using more than one environment with the advantage actor-critic?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;environment&gt;&lt;advantage-actor-critic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20872" PostTypeId="2" ParentId="20769" CreationDate="2020-05-03T13:12:30.167" Score="1" Body="&lt;p&gt;Based on the &lt;a href=&quot;https://www.ics.uci.edu/~dechter/courses/ics-295/winter-2018/presentations/Praveen.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;presentation&lt;/a&gt; by Praveen Venkateswaran on the same paper, KWIK framework is any learning algorithm that makes the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;if it knows (by its learning) the answer already, it does tell that&lt;/li&gt;&#xA;&lt;li&gt;if it is uncertain, it tells the &quot;I don't know&quot; mark&lt;/li&gt;&#xA;&lt;li&gt;&quot;I don't know&quot; marks should have an upper limit, which is the accuracy/error term of the algorithm.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The idea is that algorithm has a trainer who knows the correct answers, like in any machine learning case, but the difference is that the algorithm agent should know when it has enough knowledge to give the result right away, and when is time to ask and learn. In learning phase the trainer does not calculate errors, only he gives right answers if asked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There I see two implications: algorithm should know when it has not all the necessary data to answer (when to use &quot;I don't know&quot; answer) and a mechanism to be sure you're correct when the answer is e.g. 1 or 0 in binary case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to put that in mathematics, it varies by your chosen algorithm. Also there is a formal equation set for this, but I leave here only this literal definition to give the basic idea.&lt;/p&gt;&#xA;" OwnerUserId="11810" LastEditorUserId="11810" LastEditDate="2020-05-03T13:44:10.690" LastActivityDate="2020-05-03T13:44:10.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20873" PostTypeId="2" ParentId="20850" CreationDate="2020-05-03T15:06:37.607" Score="2" Body="&lt;p&gt;Let's have a look at the introduction of &lt;em&gt;Chapter 2: Multi-armed Bandits&lt;/em&gt; in the&#xA;&lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction by Sutton, Barto&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;The most important feature distinguishing reinforcement learning from other types of learning is that it uses training information that evaluates the actions taken rather than instructs by giving correct&#xA;actions.&lt;/strong&gt; This is what creates the need for active exploration, for an explicit search for good behavior.&#xA;&lt;strong&gt;Purely evaluative feedback indicates how good the action taken was, but not whether it was the best or&#xA;the worst action possible. Purely instructive feedback, on the other hand, indicates the correct action&#xA;to take, independently of the action actually taken.&lt;/strong&gt; This kind of feedback is the basis of supervised&#xA;learning, which includes large parts of pattern classification, artificial neural networks, and system&#xA;identification. In their pure forms, these two kinds of feedback are quite distinct: evaluative feedback&#xA;depends entirely on the action taken, whereas instructive feedback is independent of the action taken.&#xA;In this chapter &lt;strong&gt;we study the evaluative aspect of reinforcement learning in a simplified setting&lt;/strong&gt;, one&#xA;that does not involve learning to act in more than one situation. This nonassociative setting is the&#xA;one in which most prior work involving evaluative feedback has been done, and it avoids much of the&#xA;complexity of the full reinforcement learning problem. Studying this case enables us to see most clearly&#xA;how evaluative feedback differs from, and yet can be combined with, instructive feedback.&#xA;The particular nonassociative, &lt;strong&gt;evaluative feedback problem that we explore is a simple version of&#xA;the k-armed bandit problem&lt;/strong&gt;. We use this problem to introduce a number of basic learning methods&#xA;which we extend in later chapters to apply to the &lt;strong&gt;full reinforcement learning problem&lt;/strong&gt;. At the end&#xA;of this chapter, we take a step closer to the full reinforcement learning problem by discussing what&#xA;happens when the bandit problem becomes associative, that is, when actions are taken in more than&#xA;one situation.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Since bandits involve &lt;em&gt;evaluative feedback&lt;/em&gt; they are indeed a type of a (simplified) reinforcement learning problem.&lt;/p&gt;&#xA;" OwnerUserId="22835" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-03T15:06:37.607" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20874" PostTypeId="2" ParentId="20782" CreationDate="2020-05-03T16:18:50.443" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mlDSR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mlDSR.png&quot; alt=&quot;RNN structure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, here in the picture we have the unrolled or unfold RNN on the right side. Siamese network is formed when it is said to be &quot;unrolled over two time-steps&quot;. So, take part where there is two first iterations of RNN and yes, you have kind of Siamese network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One take from the &lt;a href=&quot;http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source of the image&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unlike a traditional deep neural network, which uses different parameters at each layer, a RNN shares the same parameters (U, V, W above) across all steps. This reflects the fact that we are performing the same task at each step, just with different inputs. This greatly reduces the total number of parameters we need to learn.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sounds familiar to siamese network used on single-object tracking: there we take two signals (image and the tracked object), drive it through &lt;em&gt;identical&lt;/em&gt; paths and make some maths to get results. Just something the RNN makes to time separated values!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For proof of similarity, a take from a &lt;a href=&quot;https://medium.com/@reachraktim/object-tracking-with-siamese-networks-and-detectron2-572e04dac547&quot; rel=&quot;nofollow noreferrer&quot;&gt;site where siamese networks are nicely explained&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tEZBm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tEZBm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Side note:&lt;/strong&gt; &lt;em&gt;I don't know then, how closely those relate in real world (could a Siamese network in anyway be a RNN or vice versa), but supposedly so, because the comparison is made by researcher to say so. At diagrammatic level at least there would be no problem on that.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2020-05-03T16:18:50.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20875" PostTypeId="5" CreationDate="2020-05-03T16:21:21.563" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-03T16:21:21.563" LastActivityDate="2020-05-03T16:21:21.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20876" PostTypeId="4" CreationDate="2020-05-03T16:21:21.563" Score="0" Body="For questions related to the concept of metric, which is a function that defines the distance between pairs of elements in a set. However, it is often the case that the word &quot;metric&quot; refers to some measure that is not necessarily a metric in a mathematical sense." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T18:05:33.800" LastActivityDate="2020-12-13T18:05:33.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20877" PostTypeId="2" ParentId="20868" CreationDate="2020-05-03T19:38:33.037" Score="0" Body="&lt;p&gt;It looks like the method they use for training takes a set of candidate hypotheses &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}(x)$&lt;/span&gt;, along with associated probabilities, and then minimizes the expected value of the cost function over that distribution. Section 3 has the loss function being minimized: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \mathcal{L}_{Risk} = \sum\limits_{u \in \mathcal{U}(x)} cost(t, u) \frac{p(u|x)}{\sum_{u' \in \mathcal{U}(x)} p(u'|x)} $&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the cost functions used is &lt;span class=&quot;math-container&quot;&gt;$1 - \texttt{BLEU}(t, h)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the target and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the generated hypothesis. I'm not sure where &lt;span class=&quot;math-container&quot;&gt;$p(u|x)$&lt;/span&gt; is coming from, but &lt;span class=&quot;math-container&quot;&gt;$1 - \mathcal{L}_{Risk}$&lt;/span&gt; for the BLEU cost function is probably what they're refering to when they mention Expected BLEU. &lt;/p&gt;&#xA;" OwnerUserId="2212" LastActivityDate="2020-05-03T19:38:33.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20878" PostTypeId="1" CreationDate="2020-05-03T22:05:05.463" Score="3" ViewCount="84" Body="&lt;p&gt;I've created a Q Learning algorithm to play Connect Four against an opponent who just chooses a random free column. My Q Agent is currently only winning about 0.49 games on average (30,000 episodes). Will my Q Agent actually learn from these episodes, seeing as its opponent isn't 'trying' to beat it, as there's no strategy behind its random choices? Or should this not matter – if the Q Agent is playing enough games, it doesn't matter how good/bad its opponent is?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastActivityDate="2020-05-04T07:39:28.180" Title="Does Q Learning learn from an opponent playing random moves?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20879" PostTypeId="1" CreationDate="2020-05-03T22:44:43.560" Score="2" ViewCount="52" Body="&lt;p&gt;Is there any application of topology (as in math discipline) to deep learning? If so, what are some examples?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="1671" LastEditDate="2020-05-03T23:37:18.983" LastActivityDate="2020-05-03T23:37:18.983" Title="Is there any application of topology to deep learning?" Tags="&lt;deep-learning&gt;&lt;theory&gt;&lt;topology&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20881" PostTypeId="1" AcceptedAnswerId="20887" CreationDate="2020-05-04T02:41:33.243" Score="3" ViewCount="305" Body="&lt;p&gt;I have some gaps in my understanding regarding the performing of the gradient descent in Deep - Q networks. &lt;a href=&quot;https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The original deep q network for Atari&lt;/a&gt; performs a gradient descent step to minimise &lt;span class=&quot;math-container&quot;&gt;$y_j - Q(s_j,a_j,\theta)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$y_j = r_j + \gamma max_aQ(s',a',\theta)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example where I sample a single experience &lt;span class=&quot;math-container&quot;&gt;$(s_1,a_2,r_1,s_2)$&lt;/span&gt; and I try to conduct a single gradient descent step, then feeding in &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; to the neural network outputs an array of &lt;span class=&quot;math-container&quot;&gt;$Q(s_1,a_0), Q(s_1,a_1), Q(s_1,a_2), \dots$&lt;/span&gt; values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When doing gradient descent update for this single example, should the target output to set for the network be equivalent to &lt;span class=&quot;math-container&quot;&gt;$Q(s_1,a_0), Q(s_1,a_1), r_1 + \gamma max_{a'}Q(s_2,a',\theta), Q(s_1,a_3), \dots$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the inputs to the neural network to be &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt;, to give the corresponding Q values. However, I cannot concretize the target values that the network should be optimized. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-05-15T12:49:21.780" LastActivityDate="2020-05-15T12:49:21.780" Title="What should the target be when the neural network outputs multiple Q values in deep Q-learning?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20884" PostTypeId="2" ParentId="20881" CreationDate="2020-05-04T07:18:17.210" Score="1" Body="&lt;p&gt;You are looking for the best actions which minimize the loss function. You sample a batch of memory buffer uniformly and define a loss function based on that batch. The memory buffer consists of trajectories. Each trajectory consists of an state and the action taken in that state which results in next state and an immediate reward. &#xA;If the trajectory is shown by &lt;span class=&quot;math-container&quot;&gt;$(s,a,r,s\prime)$&lt;/span&gt;, the loss for this single state is simply defined as: &lt;span class=&quot;math-container&quot;&gt;$(r + max_a\prime Q(s\prime,a\prime,w^-)-Q(s,a,w))^2$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minus sign above the parameters means that you should fix the the target parameters to ensure the stability of learning. &#xA;So the loss function for a whole batch is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$L(w) = E_{(s,a,r,s\prime)\sim U(D)}(r + max_a\prime Q(s\prime,a\prime,w^-)-Q(s,a,w))^2$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="35633" LastActivityDate="2020-05-04T07:18:17.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20885" PostTypeId="2" ParentId="20878" CreationDate="2020-05-04T07:33:36.080" Score="2" Body="&lt;p&gt;It should be possible to train an agent using some variant of DQN to beat a random agent around 100% of the time within a few thousand games. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may require one or two more advanced techniques to get the learning time down to a low number of thousands. However, if your agent is winning ~50% of games against a random agent, something has gone wrong, since that is the performance you would expect of another random agent. Even simple policies, such as always play in same column, will beat a random agent a significant fraction of the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First thing to consider is that there are too many states in Connect 4 to use tabular Q learning. You have to use some variant of DQN. As a grid-based board game where winning patterns can repeat, some form of convolutional neural network (CNN) for the Q function is probably a good start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think for a first step, you should double-check that you have implemented DQN correctly. Check the TD target formula is correct, and that you have implemented experience replay. Ideally you will also have a delayed-update target network for calculating the TD targets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a second step, try some variations of hyper-parameters. The learning rate, exploration rate, size of replay table, number of games to play before starting learning etc. A discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; slightly below 1 can help (despite this being an episodic problem) - it makes the agent forget more of the initial bias for early time steps.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or should this not matter – if the Q Agent is playing enough games, it doesn't matter how good/bad its opponent is?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Up to a point this is true. It is hard to learn against a perfect agent in Connect 4, because it always wins as player one, which means all policies are equally good and there is nothing to learn. Other than that, if there is a way to win, &lt;em&gt;eventually&lt;/em&gt; a Q learning agent with exploration should find it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Against a random agent, you should be seeing &lt;em&gt;some&lt;/em&gt; improvement if your agent is correctly set up for the problem, after a few thousand games. As it happens I am currently training Connect 4 agents using variants of DQN for a Kaggle competition, and they consistently beat random agents with 100% measured success rate after 10,000 training games. I have added a few extras to my agents in order to achieve this - there are some discussions of approaches in the forums at &lt;a href=&quot;https://www.kaggle.com/c/connectx&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/c/connectx&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-04T07:39:28.180" LastActivityDate="2020-05-04T07:39:28.180" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20886" PostTypeId="2" ParentId="20782" CreationDate="2020-05-04T07:55:22.150" Score="0" Body="&lt;p&gt;Single object tracking using a Siamese Network is a Detect and compare approach, where an object of interest is detected and the one to be tracked is passed through a siam network with next consecutive frame to get a correlation between them, if you look at the related &lt;a href=&quot;https://arxiv.org/abs/1704.06036&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt;, it is a correlation-based tracking, ie. correlation between objects detected in one frame and the ones detected in next frame, which you can imagine as samples considered across 2 timesteps or an RNN unrolled to 2 timesteps&lt;/p&gt;&#xA;" OwnerUserId="27875" LastActivityDate="2020-05-04T07:55:22.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20887" PostTypeId="2" ParentId="20881" CreationDate="2020-05-04T08:10:31.350" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;When doing gradient descent update for this single example, should the target output to set for the network be equivalent to &lt;span class=&quot;math-container&quot;&gt;$Q(s_1,a_0), Q(s_1,a_1), r_2 + \gamma max_aQ(s',a',\theta) , Q(s_1,a_3),...$&lt;/span&gt; ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Other than what looks like a couple of small typos, then yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an implementation issue for DQN, where you have decided to create a function that outputs multiple Q functions at once. There is nothing about this in Q learning theory, so you need to figure out what will generate the correct error (and therefore gradients) for an update step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't know the TD targets for actions that were not taken, and cannot make any update for them, so the gradients for these actions must be zero. One way to achieve that is to feed back the network's own output for those actions. This is common practice because you can use built-in functions from neural network libraries to handle minibatches*. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some details worth clarifying:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You have substituted the third entry in the array with the calculated TD target because the action from experience replay is &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;. In general you substitute for the action taken. Looks like you have this correct.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You have &lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt; in your experience replay table, but put &lt;span class=&quot;math-container&quot;&gt;$r_2$&lt;/span&gt; in your TD target formula. Looks like a typo. Another typo is that you maximise over &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; but reference &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt;. Also, you reference &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; but don't define it anywhere. Fixing these issues gives &lt;span class=&quot;math-container&quot;&gt;$r_1 + \gamma \text{max}_{a'}Q(s_2,a',\theta)$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the TD target it is often worth using a dedicated &lt;em&gt;target&lt;/em&gt; network that every N steps is copied from the learning network. It helps with stability. This can be noted as a &quot;frozen copy&quot; of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; noted &lt;span class=&quot;math-container&quot;&gt;$\theta^-$&lt;/span&gt;, and the neural network approximate Q function often noted &lt;span class=&quot;math-container&quot;&gt;$\hat{q}$&lt;/span&gt; giving formula of &lt;span class=&quot;math-container&quot;&gt;$r_1 + \gamma \text{max}_{a'}\hat{q}(s_2,a',\theta^-)$&lt;/span&gt; for your example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* If you want you can also calculate the gradient more directly from the single action that was taken, and back propagate from there, knowing that all the other outputs will have a gradient componnet of zero. That requires implementing at least some of the back propagation yourself.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-04T08:16:27.857" LastActivityDate="2020-05-04T08:16:27.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20888" PostTypeId="1" AcceptedAnswerId="20909" CreationDate="2020-05-04T08:20:05.057" Score="1" ViewCount="89" Body="&lt;p&gt;In Appendix B of &lt;a href=&quot;https://arxiv.org/abs/1911.08265&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuZero&lt;/a&gt;, they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In two-player zero-sum games the value functions are assumed to be bounded within the &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt; interval.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm confused about the boundary: Shouldn't the value/utility function be in the range of [-1,1] for two-player zero-sum games?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="2444" LastEditDate="2020-05-04T12:07:01.603" LastActivityDate="2020-06-03T18:00:45.113" Title="Shouldn't the utility function of two-player zero-sum games be in the range $[-1, 1]$?" Tags="&lt;reinforcement-learning&gt;&lt;game-theory&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20889" PostTypeId="1" AcceptedAnswerId="20892" CreationDate="2020-05-04T08:34:50.007" Score="1" ViewCount="65" Body="&lt;p&gt;I'm new to reinforcement learning and trying to understand it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you train an agent using a reinforcement learning algorithm (discrete or continuous) on an environment (real or simulated), then how do you know if the agent has learnt its environment? Should it reach its goal on every run (episode)? (Any literature references are also welcome)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this related to the reward threshold defined in the environment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens if you continue training after the agent has learnt the environment? Will it perform by reaching its goal every time or will there be failed episodes?&lt;/p&gt;&#xA;" OwnerUserId="32237" LastActivityDate="2020-05-04T12:09:56.493" Title="How do you know if an agent has learnt its environment in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;intelligent-agent&gt;&lt;learning-algorithms&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20891" PostTypeId="1" AcceptedAnswerId="20895" CreationDate="2020-05-04T11:56:25.533" Score="4" ViewCount="201" Body="&lt;p&gt;I was watching a video in my online course where I'm learning about A.I. I am a very beginner in it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At one point in the course, the instructor says that reinforcement learning (RL) needs a deep learning model (NN) to perform an action. But for that, we need expected results in our model for the NN to learn how to predict the Q-values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, at the beginning of the course, they said to me that RL is an unsupervised learning approach because the agent performs the action, receives the response from the environment, and finally takes the more likely action, that is, with the highest Q value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I'm using deep learning in RL, for me, RL looks like a supervised learning approach. I'm a little confused about these things, could someone give me clarifications about them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JuKV7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JuKV7.png&quot; alt=&quot;Example from Online course&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36735" LastEditorUserId="2444" LastEditDate="2020-05-04T12:11:19.227" LastActivityDate="2020-05-05T15:44:27.053" Title="How can reinforcement learning be unsupervised learning if it uses deep learning?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20892" PostTypeId="2" ParentId="20889" CreationDate="2020-05-04T12:09:56.493" Score="0" Body="&lt;p&gt;This depends on the complexity of the environment being learned, and the purpose for learning it. There is no general answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the simple environments used to teach reinforcement learning (RL), often the optimal solution is obvious, or can be calculated and proven optimal. For instance, any environment that can be solved using policy iteration will have a known optimal policy and optimal value function. The goals of these environments are to teach, or to check correctness of agents - it helps in these cases to have a well known correct answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the next level up in terms of complexity are well-studied environments which can have achievable targets set for learning agents. The goals of these environments include getting useful metrics for learning agents, such as how many episodes it takes a particular implementation to learn it well enough. Defining &quot;well enough&quot; is a matter of experience with existing agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting more complex still, in general it is not possible to know whether an agent has fully optimised against its environment. The subject area of sequential decision making that includes RL agents can cover scenarios such as driving a car or playing a computer game. We don't know when any agent, whether they are based on RL or some other approach, has fully learning an environment, and instead must construct tests of behaviour - e.g. make an agent simulate driving in a set of scenarios, and expect at least safe behaviour in each of them, essentially a driving test similar to one a person might take. In these environments, often the tests are based on &quot;good enough to use&quot; goals. We can say an agent has learned to drive if it drives more safely than an average human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the special case of competitive games, we can score agents against each other or against human players. You might say that an agent has learned its environment if it beats some standard player, but also you can rank agents against each other and declare a particular agent as the current best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to mix and match these ideas. The Atari games learning suite &lt;a href=&quot;https://deepmind.com/blog/article/Agent57-Outperforming-the-human-Atari-benchmark&quot; rel=&quot;nofollow noreferrer&quot;&gt;has benchmark scores to reach&lt;/a&gt; that count as &quot;standard human&quot;, and recently agents have been published that beat all of those scores. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What happens if you continue training after the agent has learnt the environment? Will it perform by reaching its goal every time or will there be failed episodes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you include training episodes, then RL learns mainly by &quot;trial and error&quot;. So you should expect an agent to make &lt;em&gt;deliberate&lt;/em&gt; mistakes as it tests to see what happens. In some environments these could be critical mistakes leading to failed episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you ignore the training episodes and are interested only in performance without exploration - e.g. testing every few hundred episodes - then you can expect performance to vary depending on the type of agent and environment. Some agents even exhibit &quot;catastrophic forgetting&quot;, which as the name implies causes performance to drop significantly - this can be caused by a successful agent over-fitting to all the recent successful episodes without errors it just experienced, and losing the ability to predict the true lower value of incorrect actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither failed episodes during training nor catastrophic forgetting are inevitable. It depends on the environment and type of agent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-04T12:09:56.493" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20893" PostTypeId="1" AcceptedAnswerId="20974" CreationDate="2020-05-04T12:31:03.803" Score="2" ViewCount="70" Body="&lt;p&gt;In page 125 of &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; (second last paragraph) the proof for equality of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_*$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; soft policies is given. But I could not understand the statement explaining the proof:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider a new environment that is just like the original environment, except with the requirement that policies be &lt;span class=&quot;math-container&quot;&gt;$ε$&lt;/span&gt;-soft “moved inside” the environment. The new environment has the same action and state set as the original&#xA;  and behaves as follows. If in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, then with probability&#xA;  &lt;span class=&quot;math-container&quot;&gt;$1 − ε$&lt;/span&gt; the new environment behaves exactly like the old environment. With&#xA;  probability &lt;span class=&quot;math-container&quot;&gt;$ε$&lt;/span&gt; it repicks the action at random, with equal probabilities, and&#xA;  then behaves like the old environment with the new, random action. The best&#xA;  one can do in this new environment with general policies is the same as the&#xA;  best one could do in the original environment with &lt;span class=&quot;math-container&quot;&gt;$ε$&lt;/span&gt;-soft policies.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is the meaning of environment here? And what is this new thing/argument (provided above) the authors are describing to arrive at the proof?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-05-06T17:00:33.670" Title="Doubt regarding the proof of convergence of $\epsilon$ soft policies without exploring starts" Tags="&lt;reinforcement-learning&gt;&lt;sutton-barto&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20894" PostTypeId="1" AcceptedAnswerId="20899" CreationDate="2020-05-04T12:54:04.970" Score="1" ViewCount="553" Body="&lt;p&gt;xor is a non-linear dataset. It cannot be solved with any number of perceptron based neural network but when the perceptions are applied the sigmoid activation function, we can solve the xor dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I came across a source where the following statement is stated as &lt;code&gt;False&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A two layer (one input layer; one output layer; no hidden layer) neural network can represent the XOR function&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have trained a model with no hidden layers, gives the following result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[INFO] data=[0 0],ground-truth=0, pred=0.5161, step=1&#xA;[INFO] data=[0 1],ground-truth=1, pred=0.5000, step=1&#xA;[INFO] data=[1 0],ground-truth=1, pred=0.4839, step=0&#xA;[INFO] data=[1 1],ground-truth=0, pred=0.4678, step=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, in if I apply a softmax classifier, I can separate the xor dataset with a nn without any hidden layer. This makes the statement incorrect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it true that we cannot separate a non linear dataset without any hidden layers in a neural network? If yes, where am I wrong in my reasoning from the training of the nn I have done above&lt;/p&gt;&#xA;" OwnerUserId="35616" LastActivityDate="2020-05-04T13:36:14.007" Title="solving xor function using a neural network with no hidden layers" Tags="&lt;neural-networks&gt;&lt;non-linear-regression&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20895" PostTypeId="2" ParentId="20891" CreationDate="2020-05-04T13:06:01.620" Score="3" Body="&lt;h3&gt;Supervised learning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The supervised learning (SL) problem is formulated as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are given a dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D} = \{(x_i, y_i)_{i=1}^N$&lt;/span&gt;, which is assumed to be drawn i.i.d. from an &lt;strong&gt;unknown&lt;/strong&gt; joint probability distribution &lt;span class=&quot;math-container&quot;&gt;$p(x, y)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; represents the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th input and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is the corresponding label. You choose a loss function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}: V \times U \rightarrow \mathbb{R}$&lt;/span&gt;. Then your goal is to minimize the so-called &lt;em&gt;empirical risk&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R_{\mathcal{D}}[f]=\frac{1}{N} \sum_{i=1}^N \mathcal{L}(x_i, f(x_i)) \tag{0}\label{0}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with respect to &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. In other words, you want to find the &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that minimizes the average above, which can also be formally written as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f^* = \operatorname{argmin}_f R[f] \tag{1}\label{1}&#xA;$$&lt;/span&gt;&#xA;The problem \ref{1} is called the &lt;strong&gt;empirical risk minimization&lt;/strong&gt; because it is a proxy problem for the &lt;strong&gt;expected risk minimization&lt;/strong&gt; (but you can ignore this for now).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Reinforcement learning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In reinforcement learning, you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. At each time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent takes &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, receives a reward &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; from the environment and the agent and the environment move to another state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the agent is to maximize the &lt;strong&gt;expected return&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}\left[ G_t \right] = \mathbb{E}\left[ \sum_{i=t+1}^\infty R_i \right]$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the current time step (so we don't care about the past), &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; is a random variable that represents the probable reward at time step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{i=t+1}^\infty R_i $&lt;/span&gt; is the so-called &lt;em&gt;return&lt;/em&gt; (i.e. a sum of future rewards, in this case, starting from time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;), which is also a random variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, the most important job of the programmer is to define a function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(s, a)$&lt;/span&gt;, the reward function, which provides the &lt;em&gt;reinforcement&lt;/em&gt; (or reward) signal to the RL agent. &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(s, a)$&lt;/span&gt; will deterministically or stochastically determine the reward that the agent receives every time it takes action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. (Note that &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; is different from &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt;, which is a random variable that represents the reward at time step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is the difference between SL and RL?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In RL, you (the programmer) need to define the reward function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; and you want to maximize the expected return. On the other hand, in SL you are given (or you collect) a dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;, you choose &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; in \ref{0}, and the goal is to find the function &lt;span class=&quot;math-container&quot;&gt;$f^*$&lt;/span&gt; that minimizes the empirical risk.  So, these have different settings and goals, so they are different!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, every SL problem can be cast as an RL problem. See &lt;a href=&quot;https://ai.stackexchange.com/a/14168/2444&quot;&gt;this answer&lt;/a&gt;. Similarly, in certain cases, you can formulate an RL as an SL problem. So, although the approaches are different, they are related.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Is RL an unsupervised learning approach?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In RL, you do &lt;strong&gt;not&lt;/strong&gt; tell the agent what &lt;strong&gt;action&lt;/strong&gt; it needs to take. You only say that the action that was taken was &quot;bad&quot;, &quot;good&quot; or &quot;so so&quot;. The agent needs to figure out which actions to take based on your feedback. In SL, you explicitly say that, for this input &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, the output should be &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people may consider RL is an unsupervised learning approach, but I think this is wrong, because, in RL, the programmer still needs to define the reward function, so RL isn't totally unsupervised and it's also not totally supervised. For this reason, many people consider RL an approach that sits between UL and SL.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is deep learning?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The term/expression deep learning (DL) refers to the use of deep neural networks (i.e. neural networks with many layers, where &quot;many&quot; can refer to more than 1 or 1000, i.e. it depends on the context) in machine learning, either supervised, unsupervised, or reinforcement learning. So, you can apply deep learning to SL, RL and UL. So, DL is not only restricted to SL.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-05T14:49:37.570" LastActivityDate="2020-05-05T14:49:37.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20896" PostTypeId="2" ParentId="20891" CreationDate="2020-05-04T13:06:19.247" Score="3" Body="&lt;p&gt;In &lt;strong&gt;Supervised learning&lt;/strong&gt;, the goal is to learn a mapping from points in a feature space to labels. So that for any new input data point, we are able to predict its label. whereas in &lt;strong&gt;Unsupervised learning&lt;/strong&gt;  data set is composed only of points in a feature space, i.e. there are no labels &amp;amp; here the goal is to learn some inner structure or organization in the feature space itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement Learning is basically concerned with &lt;strong&gt;learning a policy&lt;/strong&gt; in a &lt;strong&gt;sequential decision problem&lt;/strong&gt;. There are some components in RL that are “unsupervised” and some that are “supervised”, &lt;strong&gt;but it is not a combination of “unsupervised learning” and “supervised learning”&lt;/strong&gt;, since those are terms used for very particular settings, and typically not used at all for sequential decision problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mnvVc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mnvVc.png&quot; alt=&quot;Reinforcement learning&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kHRkx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kHRkx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Reinforcement Learning, we have something called as &lt;strong&gt;Reward function&lt;/strong&gt; that the agent aims to &lt;strong&gt;maximize&lt;/strong&gt;. During the learning process, one typical intermediate step is to learn to predict the &lt;strong&gt;reward&lt;/strong&gt; obtained for a &lt;strong&gt;specific policy&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nutshell, we can say &lt;strong&gt;Reinforcement Learning&lt;/strong&gt; put a model in an environment where it learns everything on its own from data collection to model evaluation. It is about taking suitable action to maximize reward in a particular situation. There is no answer but the reinforcement agent decides what to do to perform the given task. In the absence of training dataset, it is bound to learn from its experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;To better understand let’s look at an analogy.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have a dog that is not so well trained, every time the dog messes up the living room you reduce the amount of tasty foods you give it (&lt;strong&gt;punishment&lt;/strong&gt;) and every time it behaves well you double the tasty snacks (&lt;strong&gt;reward)&lt;/strong&gt;. What will the dog eventually learn? Well, that messing up the living room is bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This simple concept is powerful. The dog is the agent, the living room the environment, you are the source of the reward signal (tasty snacks).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hO3sQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hO3sQ.jpg&quot; alt=&quot;RL analogy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To learn more on &lt;strong&gt;Reinforcement learning&lt;/strong&gt;, pls check this awesome Reinforcement Learning lecture that is freely available on &lt;strong&gt;youtube&lt;/strong&gt; by someone who actually leads the Reinforcement learning research group at &lt;strong&gt;DeepMind&lt;/strong&gt; and also a lead researcher on &lt;strong&gt;AlphaGo&lt;/strong&gt;, &lt;strong&gt;AlphaZero&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[&lt;strong&gt;&lt;em&gt;RL Course by David Silver&lt;/em&gt;&lt;/strong&gt;]&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;list=PLqYmG7hTraZBiG_XpjnPrSNw-1XQaM_gB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;list=PLqYmG7hTraZBiG_XpjnPrSNw-1XQaM_gB&lt;/a&gt;&quot;&lt;/p&gt;&#xA;" OwnerUserId="36737" LastEditorUserId="36737" LastEditDate="2020-05-05T15:44:27.053" LastActivityDate="2020-05-05T15:44:27.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20898" PostTypeId="1" CreationDate="2020-05-04T13:18:59.040" Score="3" ViewCount="545" Body="&lt;p&gt;I just read an article about the minimax algorithm. When you design the algorithm, you assume that your opponent is a perfect player, i.e. it plays optimally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider the game of chess. What happens if the opponent plays irrationally or sub-optimally? Do you still have a guarantee that you are going to win?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-04T13:53:09.473" LastActivityDate="2020-06-03T15:08:15.840" Title="What happens if the opponent doesn't play optimally in minimax?" Tags="&lt;reinforcement-learning&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20899" PostTypeId="2" ParentId="20894" CreationDate="2020-05-04T13:31:01.047" Score="2" Body="&lt;p&gt;Softmax is a probability distribution you use when you want probability for all multiple classes you are predicting which are not independent, ie, &lt;code&gt;exp(xi)/sum(exp(xj) for j in all x)&lt;/code&gt;, where &lt;code&gt;xi&lt;/code&gt; is the score of one neuron, so softmax is good if you have more than 1 neurons, but for just 1 neuron(in this case), the output of softmax will be 1, always (exp(xi)/exp(xi)). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets visualize your dataset, ie plot the points of XOR as a function of X,Y on coordinate space.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/13NUe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/13NUe.png&quot; alt=&quot;XOR plot image&quot;&gt;&lt;/a&gt;, [src: &lt;a href=&quot;https://www.researchgate.net/figure/The-exclusive-or-XOR-function-is-a-nonlinear-function-that-returns-0-when-its-two_fig4_322048911]&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/figure/The-exclusive-or-XOR-function-is-a-nonlinear-function-that-returns-0-when-its-two_fig4_322048911]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, whatever you do, you cannot separate the points with a single &lt;code&gt;wx+b&lt;/code&gt; straight line, and after passing it to sigmoid, you just squeeze the values for those, ie, a clipped &lt;code&gt;wx+b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, it is still a linear equation. Now when you add another layer, a hidden one, you can operate again on the 1st output, which if you squeeze between 0 and 1 or use something like relu activation, will produce some non linearity, otherwise it will just be &lt;code&gt;(w2(w1*x + b1)+b2&lt;/code&gt;, which again is a linear equation not able to separate the classes 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So after adding a hidden layer and passing it through activation function, you get non linear output, example, with sigmoid you get &lt;code&gt;w2*(1/+exp(w2*x+b1))+b2&lt;/code&gt; which has a fairly non linear term of x, and after passing it through sigmoid again, you are able to squeeze the outputs(otherwise they can be higher than 1 or lower than 0,but XOR outputs are binary) after fitting the curve so that both classes are separate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, food for thought, what will happen if you use 2 output neurons for XOR (represent 0 as [1,0] and 1 as [0,1])?&lt;/p&gt;&#xA;" OwnerUserId="27875" LastEditorUserId="27875" LastEditDate="2020-05-04T13:36:14.007" LastActivityDate="2020-05-04T13:36:14.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20900" PostTypeId="2" ParentId="20871" CreationDate="2020-05-04T14:14:48.187" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the advantage of using more than one environment with a single agent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are two main advantages to this approach:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The dataset for training is closer to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables&quot; rel=&quot;nofollow noreferrer&quot;&gt;independent, identically distributed (i.i.d.) ideal&lt;/a&gt;, important for theoretical and practical reasons when training a neural network. Samples taken from a single trajectory are not independent, but instead are correlated due to rules of the environment - so using a single trajectory is furthest from i.i.d. This is a similar motivation to use of experience replay tables for DQN variants of Q-learning. However, experience replay is inherently off-policy, so not a good fit to A2C or A3C that need samples taken when acting under the current policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Collecting experience is often a major bottleneck in training RL agents. Being able to do so in parallel in a distributed environment can significantly speed up the training process.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-11T06:50:36.017" LastActivityDate="2020-05-11T06:50:36.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20901" PostTypeId="2" ParentId="20898" CreationDate="2020-05-04T14:27:50.343" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What happens if the opponent plays irrationally or sub-optimally? Do you still have a guarantee that you are going to win?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If your search is deep enough to guarantee optimal play in all cases, then yes. Optimal play is such that the opponent's decision causes least impact to your agent. In fact, if the opponent makes a mistake, often that can make the search easier/faster, and the agent will win more convincingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it may mean is that optimisations you may have taken - e.g. pruning game tree segments that lead to non-optimal decisions by either player - might not be as useful. This might impact decision time if you keep some partial game tree or cache branch evaluations between moves to help speed up the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actual optimal agents for games as complex as chess are not possible. In these games, you will not have a truly optimal agent, but approximately optimal. You will be relying on some heuristic to guide the minimax search when it cannot force an end game win. If the opponent manages to control play into a state where the heuristics are not accurate, they could cause minimax search to fail and misdirect your agent into making mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A combination of effects is also possible if you have implemented a tree caching mechanism for performance improvements and made the system playable by limiting planning time - e.g. you limit computer search time to 3 seconds max - an irrational opponent may cause your agent's performance to degrade to the point where it too starts to make mistakes. Whether or not this is enough for a smart opponent to take advantage of it and beat an agent which is capable of otherwise playing a &quot;perfect&quot; game depends on details of the game, and how narrow the agent's measure of &quot;perfect&quot; is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An extreme case might be an agent that has memorised a single perfect game (by scoring high heuristics for any state on a single tarjectory through the game that has been pre-calculated for perfect moves by both players), and has poor heuristics otherwise - once state moves away from what it can evaluate directly, the agent will be limited by how well it can search for completed game wins, and can easily be manipulated by a more generally smart opponent into a losing position that is beyond the depth of its search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, well-coded agents will not suffer too much from this effect. If you make a mistake, or try a random probably bad move in an attempt to confuse the agent when playing against Stockfish, you &lt;em&gt;will&lt;/em&gt; lose.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-04T14:33:19.700" LastActivityDate="2020-05-04T14:33:19.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20902" PostTypeId="1" CreationDate="2020-05-04T14:29:46.277" Score="3" ViewCount="98" Body="&lt;p&gt;Apart from the vanishing or exploding gradient problems, what are other problems or pitfalls that we could face when training neural networks?&lt;/p&gt;&#xA;" OwnerUserId="36740" LastEditorUserId="2444" LastEditDate="2020-05-04T20:59:48.380" LastActivityDate="2020-05-04T21:14:44.880" Title="What are the common pitfalls that we could face when training neural networks?" Tags="&lt;neural-networks&gt;&lt;vanishing-gradient-problem&gt;&lt;exploding-gradient-problem&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20903" PostTypeId="1" AcceptedAnswerId="20957" CreationDate="2020-05-04T14:39:38.330" Score="4" ViewCount="801" Body="&lt;p&gt;In reinforcement learning (RL), what is the difference between training and testing an algorithm/agent? If I understood correctly, testing is also referred to as evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it, both imply the same procedure: select an action, apply to the environment, get a reward, and next state, and so on. But I've seen that, e.g., the &lt;a href=&quot;https://github.com/tensorforce/tensorforce&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorforce RL framework&lt;/a&gt; allows running with or without evaluation.&lt;/p&gt;&#xA;" OwnerUserId="32237" LastEditorUserId="2444" LastEditDate="2020-05-04T15:05:08.387" LastActivityDate="2020-11-23T15:42:19.127" Title="What is the difference between training and testing in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;comparison&gt;&lt;testing&gt;" AnswerCount="4" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20905" PostTypeId="5" CreationDate="2020-05-04T15:05:37.300" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-04T15:05:37.300" LastActivityDate="2020-05-04T15:05:37.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20906" PostTypeId="4" CreationDate="2020-05-04T15:05:37.300" Score="0" Body="For questions related to the concept of testing (or evaluating) machine learning models and algorithms, e.g. in terms of some performance measure (such as accuracy or cumulative reward)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-02T01:19:37.180" LastActivityDate="2021-02-02T01:19:37.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20907" PostTypeId="2" ParentId="20902" CreationDate="2020-05-04T15:45:18.797" Score="2" Body="&lt;p&gt;I can't say that it is the biggest problem of deep neural network but it is one of the big problem with deep neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other issue which happens a lot is overfitting on the training data hence network behaves badly on test set which can be solved using regression. So you have to make sure that network is generalised enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the network is trying to have a classification then if the outputs are mutually inclusive or exclusive, depending upon that you define your loss function which will be categorical cross entropy for mutually inclusive output and binary cross entropy for mutually exclusive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other thing which I can think is about &lt;code&gt;all zero initialization.&lt;/code&gt; Initializing the network weights as zero can lead to same gradient computation and due to which network does not learn most of the times.&lt;/p&gt;&#xA;" OwnerUserId="33835" LastEditorUserId="33835" LastEditDate="2020-05-04T21:14:44.880" LastActivityDate="2020-05-04T21:14:44.880" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20908" PostTypeId="1" CreationDate="2020-05-04T15:48:57.370" Score="3" ViewCount="66" Body="&lt;p&gt;I was wondering if there are companies that still use expert systems in their business processes?&lt;/p&gt;&#xA;" OwnerUserId="36744" LastActivityDate="2020-12-23T08:03:50.340" Title="Are there companies that still use expert systems?" Tags="&lt;expert-system&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2021-01-22T03:45:10.967" ContentLicense="CC BY-SA 4.0" />
  <row Id="20909" PostTypeId="2" ParentId="20888" CreationDate="2020-05-04T16:39:10.497" Score="2" Body="&lt;p&gt;it can be either. If you consider the lack of reward as &quot;penalty&quot; then getting 0 reward is bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you use a value estimator through a neural network, the range of rewards will dictate the squashing function you use for the output layer&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-05-04T16:39:10.497" CommentCount="19" ContentLicense="CC BY-SA 4.0" />
  <row Id="20910" PostTypeId="1" AcceptedAnswerId="20937" CreationDate="2020-05-04T16:52:36.420" Score="6" ViewCount="153" Body="&lt;p&gt;I am asking for a book (or any other online resource) where we can solve exercises related to neural networks, similar to the books or online resources dedicated to mathematics where we can solve mathematical exercises.&lt;/p&gt;&#xA;" OwnerUserId="4211" LastEditorUserId="2444" LastEditDate="2020-05-04T17:59:39.540" LastActivityDate="2020-05-05T18:56:19.450" Title="What are some resources with exercises related to neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reference-request&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20911" PostTypeId="1" CreationDate="2020-05-04T17:14:38.637" Score="3" ViewCount="70" Body="&lt;p&gt;Is there a good and modern book that focuses on word embeddings and their applications? It would also be ok to provide the name of a paper that provides a good overview of word embeddings.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="2444" LastEditDate="2020-05-22T18:04:46.053" LastActivityDate="2020-05-22T18:04:46.053" Title="Is there a good book or paper on word embeddings?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;word-embedding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20913" PostTypeId="1" CreationDate="2020-05-04T19:01:04.093" Score="2" ViewCount="32" Body="&lt;p&gt;From &lt;a href=&quot;https://arxiv.org/abs/1808.05377&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Architecture Search: A Survey&lt;/a&gt;, first published in 2018:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Moreover, common search spaces are also based on predefined building&#xA;  blocks, such as different kinds of convolutions and pooling, but do&#xA;  not allow identifying novel building blocks on this level; going&#xA;  beyond this limitation might substantially increase the power of NAS.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Has anyone tried that? If not, do you have any thoughts about the feasibility of this idea?&lt;/p&gt;&#xA;" OwnerUserId="36632" LastEditorUserId="2444" LastEditDate="2020-05-04T20:30:01.873" LastActivityDate="2020-05-04T20:30:01.873" Title="Can operations like convolution and pooling be discovered with a neural architecture search approach?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;neuroevolution&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20914" PostTypeId="1" CreationDate="2020-05-04T19:26:31.697" Score="2" ViewCount="37" Body="&lt;p&gt;I have taken some reference implementations of PPO algorithm and am trying to create an agent which can play space invaders . Unfortunately from the 2nd trial onwards (after training the actor and critic N Networks for the first time) , the probability distribution of the actions converges on only action and the PPO loss and the critic loss converges on only one value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wanted to understand the probable reasons why this might occur . I really cant run the code in my cloud VMs without being sure that I am not missing anything as the VMs are very costly to use . I would appreciate any help or advice in this regarding .. if required I can post the code as well . Hyperparameters used are as follows :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;clipping_val = 0.2 critic_discount = 0.5 entropy_beta = 0.01 gamma = 0.99 lambda = 0.95&lt;/p&gt;&#xA;&#xA;&lt;p&gt;code repo : github.com/superchiku/ReinforcementLearning . &lt;/p&gt;&#xA;" OwnerUserId="36749" LastActivityDate="2020-05-04T19:26:31.697" Title="PPO algorithm converges on only one action" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20915" PostTypeId="2" ParentId="20728" CreationDate="2020-05-04T19:41:39.477" Score="1" Body="&lt;p&gt;In addition to the reason outlined in the comment, also note that if the state-space and action-space are both finite and of feasible size, tabular methods can be used, and there are some advantages to them (like the existence of convergence guarantees and generally a smaller number of hyperparameters to tune).&lt;/p&gt;&#xA;" OwnerUserId="36748" LastActivityDate="2020-05-04T19:41:39.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20916" PostTypeId="1" CreationDate="2020-05-04T19:44:54.270" Score="1" ViewCount="128" Body="&lt;p&gt;I am trying to reproduce the recommender task experiment from &lt;a href=&quot;https://arxiv.org/pdf/1512.07679.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. &#xA;The paper suggests to embed discrete actions into continuous action space and then to use the proposed Wolpertinger agent. &#xA;The Wolpertinger agent is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DDPG produces so called proto action &lt;span class=&quot;math-container&quot;&gt;$f(s)$&lt;/span&gt;, then KNN finds k nearest embeddings of discrete actions to this proto action, and then we choose the one of these &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; embeddings, which has the highest Q-function value. The whole is a full policy, &lt;span class=&quot;math-container&quot;&gt;$\pi(\cdot)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While training we optimize the critic using only the full policy (DDPG + a choice of a neighbour).&#xA;The actor is optimized using proto action output in order to be differentiable, &lt;span class=&quot;math-container&quot;&gt;$Q(s, f_{\theta}(s)) \rightarrow \max_{\theta}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the critic does not know that it is used to optimize continuous output of the algorithm. It is trained only to value the embedded actions. As I understand, we hope that continuity of critic will help us with it, but what I have is that proto actions constantly appear to be in some corners with no real actions and where the Q-function unreasonably has greater values (because it is simply untrained in such domains). The DDPG output is normalized to match the embeddings bounds to make these empty spaces not so large. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems for me that there is a way to make embeddings more appropriate for the task and achieve higher reward. However, when I use &lt;span class=&quot;math-container&quot;&gt;$k = |&#xA;\mathcal{A}|$&lt;/span&gt;, proto actions are not considered and algorithm works pretty well.&#xA;Usually I use &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}| = 100$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k = 10$&lt;/span&gt;.&#xA;I have trained them with skip-gram, based on the users history.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are 2d projections of my embeddings to the first 10 axes (embeddings are from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{20}$&lt;/span&gt;). And undependently of the state, proto actions are about the same. The blue is a proto action for the some fixed state.&#xA;Having some state fixed, &lt;span class=&quot;math-container&quot;&gt;$Q(s, f(s))$&lt;/span&gt; value is always higher than &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; for any &lt;span class=&quot;math-container&quot;&gt;$a \in \mathcal{A}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2kmDo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2kmDo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would be glad to get any help, especially the help of people familiar with this algorithm.&#xA;Do I need to make embeddings fill proto actions range (some hyperrectangle in case we have the tanh activation it the actor)? What is the way to fill such a domain with embeddings?&lt;/p&gt;&#xA;" OwnerUserId="36690" LastEditorUserId="36690" LastEditDate="2020-05-04T19:50:48.593" LastActivityDate="2020-05-04T19:50:48.593" Title="A question about the Wolpertinger algorithm (Deep RL in Large Discrete Action Spaces paper)" Tags="&lt;reinforcement-learning&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20917" PostTypeId="2" ParentId="20902" CreationDate="2020-05-04T20:42:36.983" Score="1" Body="&lt;p&gt;There are several pitfalls or issues that require your attention when or before training or using neural networks. I will list some of them below, along with some questions you need to ask yourself before or while using neural networks.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Over-fitting and under-fitting problems, and the related generalization problem. Is your neural network generalizing to unseen data?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Availability of training and test data&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do you have enough data to train your neural network so that it generalizes well (i.e. it neither over-fits or under-fits)?&lt;/li&gt;&#xA;&lt;li&gt;Is your test dataset big enough to assess the generalization ability of your neural network?&lt;/li&gt;&#xA;&lt;li&gt;Is your data representative of the problem you are trying to solve?&lt;/li&gt;&#xA;&lt;li&gt;Do you need to augment or normalize your data?&lt;/li&gt;&#xA;&lt;li&gt;Do you need to use cross-validation?&lt;/li&gt;&#xA;&lt;li&gt;Is your data independent and identically distributed (i.i.d.)? If your data is correlated, training could be unstable. Shuffling your data may be a feasible solution when your data is initially correlated.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you have enough computational resources (i.e. GPUs) for training and testing your neural network?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are you solving a regression or classification problem? The type of the outputs and the loss function will typically be different in both cases&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you need explainability and transparency? If yes, neural networks aren't probably the best model to use, as the connections between the neurons are quite obscure and don't really represent any meaningful interaction. That's why neural networks are called black-boxes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you need uncertainty estimation? If yes, you may want to try Bayesian neural networks. Typical neural networks are not very appropriate for uncertainty estimation!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you use a neural network for function approximation (e.g. in reinforcement learning), you will lose certain convergence guarantees.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-04T20:49:06.963" LastActivityDate="2020-05-04T20:49:06.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20918" PostTypeId="2" ParentId="20910" CreationDate="2020-05-04T22:44:40.510" Score="4" Body="&lt;p&gt;There are actually quite a few. Personally I would say these courses have high quality and strong  focus on practice:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Standford computer vision cs231. Check the assignments materials on &lt;a href=&quot;http://cs231n.stanford.edu/2019/syllabus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;. This course has good explanation/exercises of how generally neural nets and backprop works.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/fastai/course-v3/tree/master/nbs/dl1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fastai course notebooks&lt;/a&gt;. You can listen to the lectures as well, but notebooks are quite self-containing&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/yandexdataschool/Practical_RL&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical reinforcement learning&lt;/a&gt; course, if you interested in NN application for RL&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16940" LastEditorUserId="16940" LastEditDate="2020-05-04T22:49:57.993" LastActivityDate="2020-05-04T22:49:57.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20919" PostTypeId="2" ParentId="20910" CreationDate="2020-05-04T23:11:29.243" Score="3" Body="&lt;p&gt;One of the most famous books dedicated to neural networks is &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks - A Systematic Introduction&lt;/a&gt; (1996) by &lt;a href=&quot;https://en.wikipedia.org/wiki/Ra%C3%BAl_Rojas&quot; rel=&quot;nofollow noreferrer&quot;&gt;Raul Rojas&lt;/a&gt;. &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/chapter/forword.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Most chapters end with a series of exercises&lt;/a&gt; that test your understanding of the material. For example, in chapter &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/chapter/K14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;14 Stochastic Networks&lt;/a&gt;, one of the exercises is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Solve the eight queens problem using a Boltzmann machine. Define the&#xA;  network's weights by hand.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This should give you a sense of the type of exercise that you will find in this book.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-04T23:11:29.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20921" PostTypeId="1" CreationDate="2020-05-05T03:44:03.907" Score="2" ViewCount="61" Body="&lt;p&gt;I was following a tutorial about Feed Forward Networks and wrote this code for a simple FFN :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class FirstFFNetwork:&#xA;&#xA;  #intialize the parameters&#xA;  def __init__(self):&#xA;    self.w1 = np.random.randn()&#xA;    self.w2 = np.random.randn()&#xA;    self.w3 = np.random.randn()&#xA;    self.w4 = np.random.randn()&#xA;    self.w5 = np.random.randn()&#xA;    self.w6 = np.random.randn()&#xA;    self.b1 = 0&#xA;    self.b2 = 0&#xA;    self.b3 = 0&#xA;&#xA;  def sigmoid(self, x):&#xA;    return 1.0/(1.0 + np.exp(-x))&#xA;&#xA;  def forward_pass(self, x):&#xA;    #forward pass - preactivation and activation&#xA;    self.x1, self.x2 = x&#xA;    self.a1 = self.w1*self.x1 + self.w2*self.x2 + self.b1&#xA;    self.h1 = self.sigmoid(self.a1)&#xA;    self.a2 = self.w3*self.x1 + self.w4*self.x2 + self.b2&#xA;    self.h2 = self.sigmoid(self.a2)&#xA;    self.a3 = self.w5*self.h1 + self.w6*self.h2 + self.b3&#xA;    self.h3 = self.sigmoid(self.a3)&#xA;    return self.h3&#xA;&#xA;  def grad(self, x, y):&#xA;    #back propagation&#xA;    self.forward_pass(x)&#xA;&#xA;    self.dw5 = (self.h3-y) * self.h3*(1-self.h3) * self.h1&#xA;    self.dw6 = (self.h3-y) * self.h3*(1-self.h3) * self.h2&#xA;    self.db3 = (self.h3-y) * self.h3*(1-self.h3)&#xA;&#xA;    self.dw1 = (self.h3-y) * self.h3*(1-self.h3) * self.w5 * self.h1*(1-self.h1) * self.x1&#xA;    self.dw2 = (self.h3-y) * self.h3*(1-self.h3) * self.w5 * self.h1*(1-self.h1) * self.x2&#xA;    self.db1 = (self.h3-y) * self.h3*(1-self.h3) * self.w5 * self.h1*(1-self.h1)&#xA;&#xA;    self.dw3 = (self.h3-y) * self.h3*(1-self.h3) * self.w6 * self.h2*(1-self.h2) * self.x1&#xA;    self.dw4 = (self.h3-y) * self.h3*(1-self.h3) * self.w6 * self.h2*(1-self.h2) * self.x2&#xA;    self.db2 = (self.h3-y) * self.h3*(1-self.h3) * self.w6 * self.h2*(1-self.h2)&#xA;&#xA;&#xA;  def fit(self, X, Y, epochs=1, learning_rate=1, initialise=True, display_loss=False):&#xA;&#xA;    # initialise w, b&#xA;    if initialise:&#xA;      self.w1 = np.random.randn()&#xA;      self.w2 = np.random.randn()&#xA;      self.w3 = np.random.randn()&#xA;      self.w4 = np.random.randn()&#xA;      self.w5 = np.random.randn()&#xA;      self.w6 = np.random.randn()&#xA;      self.b1 = 0&#xA;      self.b2 = 0&#xA;      self.b3 = 0&#xA;&#xA;    if display_loss:&#xA;      loss = {}&#xA;&#xA;    for i in tqdm_notebook(range(epochs), total=epochs, unit=&quot;epoch&quot;):&#xA;      dw1, dw2, dw3, dw4, dw5, dw6, db1, db2, db3 = [0]*9&#xA;      for x, y in zip(X, Y):&#xA;        self.grad(x, y)&#xA;        dw1 += self.dw1&#xA;        dw2 += self.dw2&#xA;        dw3 += self.dw3&#xA;        dw4 += self.dw4&#xA;        dw5 += self.dw5&#xA;        dw6 += self.dw6&#xA;        db1 += self.db1&#xA;        db2 += self.db2&#xA;        db3 += self.db3&#xA;&#xA;      m = X.shape[1]&#xA;      self.w1 -= learning_rate * dw1 / m&#xA;      self.w2 -= learning_rate * dw2 / m&#xA;      self.w3 -= learning_rate * dw3 / m&#xA;      self.w4 -= learning_rate * dw4 / m&#xA;      self.w5 -= learning_rate * dw5 / m&#xA;      self.w6 -= learning_rate * dw6 / m&#xA;      self.b1 -= learning_rate * db1 / m&#xA;      self.b2 -= learning_rate * db2 / m&#xA;      self.b3 -= learning_rate * db3 / m&#xA;&#xA;      if display_loss:&#xA;        Y_pred = self.predict(X)&#xA;        loss[i] = mean_squared_error(Y_pred, Y)&#xA;&#xA;    if display_loss:&#xA;      plt.plot(loss.values())&#xA;      plt.xlabel('Epochs')&#xA;      plt.ylabel('Mean Squared Error')&#xA;      plt.show()&#xA;&#xA;  def predict(self, X):&#xA;    #predicting the results on unseen data&#xA;    Y_pred = []&#xA;    for x in X:&#xA;      y_pred = self.forward_pass(x)&#xA;      Y_pred.append(y_pred)&#xA;    return np.array(Y_pred)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The data was generated as follows : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data, labels = make_blobs(n_samples=1000, centers=4, n_features=2, random_state=0)&#xA;labels_orig = labels&#xA;labels = np.mod(labels_orig, 2)&#xA;X_train, X_val, Y_train, Y_val = train_test_split(data, labels, stratify=labels, random_state=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I ran the program yesterday, I had gotten a training accuracy of about 98% and test accuracy of 94%. But when I ran it today, suddenly the accuracy dropped to 60-70%. I tried to scatter plot the result, and it looked like it behaved as if it were a single sigmoid instead of the Feed Forward Network.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ffn = FirstFFNetwork()&#xA;#train the model on the data&#xA;ffn.fit(X_train, Y_train, epochs=2000, learning_rate=.01, display_loss=False)&#xA;#predictions&#xA;Y_pred_train = ffn.predict(X_train)&#xA;Y_pred_binarised_train = (Y_pred_train &amp;gt;= 0.5).astype(&quot;int&quot;).ravel()&#xA;Y_pred_val = ffn.predict(X_val)&#xA;Y_pred_binarised_val = (Y_pred_val &amp;gt;= 0.5).astype(&quot;int&quot;).ravel()&#xA;accuracy_train_1 = accuracy_score(Y_pred_binarised_train, Y_train)&#xA;accuracy_val_1 = accuracy_score(Y_pred_binarised_val, Y_val)&#xA;#model performance&#xA;print(&quot;Training accuracy&quot;, round(accuracy_train_1, 2))&#xA;print(&quot;Validation accuracy&quot;, round(accuracy_val_1, 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not understand how this happened and cannot figure it out.&lt;/p&gt;&#xA;" OwnerUserId="36756" LastActivityDate="2020-05-19T09:33:09.403" Title="Accuracy dropped when I ran the program the second time" Tags="&lt;deep-learning&gt;&lt;feedforward-neural-network&gt;&lt;accuracy&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20925" PostTypeId="1" CreationDate="2020-05-05T06:35:01.597" Score="1" ViewCount="186" Body="&lt;p&gt;I've been trying to figure out how to compute the number of Flops in backward pass of ResNet. For forward pass, it seems straightforward: apply the conv filters to the input for each layer. But how does one do the Flops counts for gradient computation and update of all weights during the backward pass?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;how to compute Flops in gradient computations for each layer? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;what all gradients need to be computed so Flops for each of those can be counted? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How many Flops in computation of gradient for Pool, BatchNorm, and Relu layers?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I understand the chain rule for gradient computation, but having a hard time formulating how it'd apply to weight filters in conv layers of ResNet and how many Flops each of those would take. It'd be very useful to get any comments about method to compute total Flops for Backward pass. Thanks&lt;/p&gt;&#xA;" OwnerUserId="33580" LastActivityDate="2020-05-05T06:35:01.597" Title="Backward pass of CNN like Resnet: how to manually compute flops during backprop?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20926" PostTypeId="1" CreationDate="2020-05-05T07:24:26.437" Score="1" ViewCount="75" Body="&lt;p&gt;What would happen when an artificial general intelligence can improve itself over a long time, with limited resources?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assumption is that it has a large but finite amount of computing power, and can not escape that limit to find resources. Let's assume the limit to be on the order 100 human brains or so - the limits are not important, only that it is limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we let it run to self-improve as much as it can with the given resources, until it goes to some stable or periodic state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it can do is at least one of the following:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;converge to some state&lt;/li&gt;&#xA;&lt;li&gt;oscillate between two or more states&lt;/li&gt;&#xA;&lt;li&gt;go into a chaotic state&lt;/li&gt;&#xA;&lt;li&gt;stop doing anything&lt;/li&gt;&#xA;&lt;li&gt;disable itself in some other way than just stop doing anything&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are certainly more complex behaviors possible:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;become maximally happy, and then periodically redefine what happy means&lt;/li&gt;&#xA;&lt;li&gt;hacking its reward function in some other way&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would expect it to converges to a single state - but that seems naive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any ideas on how it would end up?&lt;/p&gt;&#xA;" OwnerUserId="2317" LastEditorUserId="2444" LastEditDate="2020-05-05T20:06:56.390" LastActivityDate="2020-05-05T21:09:46.713" Title="What does a self-improving artificial general intelligence with finite resources and infinite time do?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;superintelligence&gt;&lt;singularity&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20928" PostTypeId="1" CreationDate="2020-05-05T07:44:35.517" Score="1" ViewCount="59" Body="&lt;p&gt;Are there any cheap alternatives to NVIDIA hardware coming out in the near future?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like ASIC or FPGA based PCI-E cards, etc supporting popular libraries which have similar performance to high-end NVIDIA cards in training but are a lot cheaper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching the net is not providing me with any answers. How long will it take to launch such a product and how much can it cost?&lt;/p&gt;&#xA;" OwnerUserId="31307" LastActivityDate="2020-05-05T11:12:59.927" Title="Are there any cheap alternatives to NVIDIA hardware coming out in the near future?" Tags="&lt;hardware&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-05-05T10:56:45.153" ContentLicense="CC BY-SA 4.0" />
  <row Id="20929" PostTypeId="2" ParentId="3494" CreationDate="2020-05-05T08:04:58.973" Score="1" Body="&lt;p&gt;Many people who are interested in machine learning aren't professional programmers. For example there are mathematicians who work on differential equations and there are physicists who work on stochastic processes. These people aren't programmers. So using a language like C++ which is hard to learn is only detrimental to their works. And also creating a model in Python is much easier compared with C++ and Java. You have to use C++ when you want to create a game engine because the graphics is directly related to the hardware and if you want to be a professional Android programmer you have to learn Java. What are the benefits of choosing C++ and Java over Python when your work mainly consists of linear algebra and statistics?&lt;/p&gt;&#xA;" OwnerUserId="35633" LastActivityDate="2020-05-05T08:04:58.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20930" PostTypeId="1" CreationDate="2020-05-05T08:16:11.400" Score="2" ViewCount="48" Body="&lt;p&gt;Let's say I want to teach a neural to classify images, and, for some reason, I insist on using reinforcement learning rather than supervised learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a dataset of images and their matching classes. Then, for each image, I could define a reward function which is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for classifying it right and &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for classifying it wrong (or perhaps even define a more complicated reward function where some mistakes are less costly than others).  For each image &lt;span class=&quot;math-container&quot;&gt;$x^i$&lt;/span&gt;, I can loop through each class &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and use a vanilla REINFORCE step: &lt;span class=&quot;math-container&quot;&gt;$\theta = \theta + \alpha \nabla_{\theta}log \pi_{\theta}(c|x^i)r$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would that be different than using standard supervised learning methods (for example, the cross-entropy loss)? Should I expect different results? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method actually seems better since I could define a custom reward for each misclassification, but I've never seen anyone use something like that&lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="2444" LastEditDate="2020-05-05T11:03:10.690" LastActivityDate="2020-05-05T11:03:10.690" Title="Can a typical supervised learning problem be solved with reinforcement learning methods?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;supervised-learning&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20933" PostTypeId="2" ParentId="20921" CreationDate="2020-05-05T10:22:28.413" Score="3" Body="&lt;ul&gt;&#xA;&lt;li&gt;It is common during the training of &lt;strong&gt;Neural Networks&lt;/strong&gt; for &lt;strong&gt;accuracy to improve for a while&lt;/strong&gt; and &lt;strong&gt;then get worse&lt;/strong&gt; -- in general, This is caused by &lt;strong&gt;over-fitting.&lt;/strong&gt; It's also fairly common for the Neural Network to &lt;strong&gt;&quot;get UNLUCKY&lt;/strong&gt; and get knocked into a &lt;strong&gt;BAD sectors of parameter space&lt;/strong&gt; corresponding to a sudden decrease in accuracy -- sometimes it can recover from this quickly, but sometimes not.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cFiA1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cFiA1.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In general, lowering your &lt;strong&gt;learning rate&lt;/strong&gt; is a &lt;strong&gt;good approach&lt;/strong&gt; to this kind of problem. Also, setting a &lt;strong&gt;learning rate schedule&lt;/strong&gt; like FactorScheduler can help you achieve &lt;strong&gt;more stable convergence&lt;/strong&gt; by &lt;strong&gt;lowering the learning rate every few epochs&lt;/strong&gt;. In fact, this can sometimes cover up mistakes in picking an initial learning rate that is too high.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you can try using mini-batches.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The error (Entropy) with log functions must be used &lt;strong&gt;precisely&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/What-are-some-possible-reasons-why-training-accuracy-would-decrease-over-time&quot; rel=&quot;nofollow noreferrer&quot;&gt;What are some possible reasons why training accuracy would decrease over time?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check out this paper &lt;a href=&quot;https://arxiv.org/abs/1506.01186&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1506.01186&lt;/a&gt; a novel way to change the learning rate cyclically.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36737" LastEditorUserId="36737" LastEditDate="2020-05-19T09:33:09.403" LastActivityDate="2020-05-19T09:33:09.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20935" PostTypeId="1" CreationDate="2020-05-05T12:25:49.817" Score="2" ViewCount="171" Body="&lt;p&gt;I'm trying to make a bot to the famous &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Icy_Tower&quot; rel=&quot;nofollow noreferrer&quot;&gt;Icy Tower&lt;/a&gt;&amp;quot; game.&#xA;I rebuilt the game using pygame and I'm trying to build the bot using &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python-NEAT&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Every generation a population of 70 characters tries to jump to the next platform and increase their fitness. right now the fitness is the number of platforms they jumped on, each platform gives +10.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fYHPF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fYHPF.png&quot; alt=&quot;example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The problem&lt;/strong&gt; I'm facing is that the bot &lt;strong&gt;isn't learning&lt;/strong&gt; good enough after 1000 generations the best score was around 200 (it can get to 200 even at the first few generations by mistake. 200 means 20 platforms which is not a lot).&lt;/p&gt;&#xA;&lt;p&gt;when I look at the characters jumping it looks like they just always jump and go left or right and not deliberately aiming to the next platform.&lt;/p&gt;&#xA;&lt;p&gt;I tried several input configurations to make the bot perform better. but nothing really helped.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;these are the inputs I tried to mess around with&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pos.x, pos.y&lt;/li&gt;&#xA;&lt;li&gt;velocity.x, velocity.y&lt;/li&gt;&#xA;&lt;li&gt;isOnPlatform (bool)&lt;/li&gt;&#xA;&lt;li&gt;[plat.x, plat.y, plat.width] (list of the 3-7 next platforms locations, also tried distance from character in x,y)&lt;/li&gt;&#xA;&lt;li&gt;[prev.x, prev.y] (2-6 previous character positions)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm not so proficient with neuroevolution and I'm probably doing something wrong. glad if you could explain what's causing the bot to be so bad or what's not helping him to learn properly.&lt;/p&gt;&#xA;&lt;p&gt;Although I think that the fitness function and the inputs should be the only problem I'm attaching the python-NEAT config file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[NEAT]&#xA;fitness_criterion     = max&#xA;fitness_threshold     = 10000&#xA;pop_size              = 70&#xA;reset_on_extinction   = False&#xA;&#xA;[DefaultGenome]&#xA;# node activation options&#xA;activation_default      = tanh&#xA;activation_mutate_rate  = 0.0&#xA;activation_options      = tanh&#xA;&#xA;# node aggregation options&#xA;aggregation_default     = sum&#xA;aggregation_mutate_rate = 0.0&#xA;aggregation_options     = sum&#xA;&#xA;# node bias options&#xA;bias_init_mean          = 0.0&#xA;bias_init_stdev         = 1.0&#xA;bias_max_value          = 30.0&#xA;bias_min_value          = -30.0&#xA;bias_mutate_power       = 0.5&#xA;bias_mutate_rate        = 0.7&#xA;bias_replace_rate       = 0.1&#xA;&#xA;# genome compatibility options&#xA;compatibility_disjoint_coefficient = 1.0&#xA;compatibility_weight_coefficient   = 0.5&#xA;&#xA;# connection add/remove rates&#xA;conn_add_prob           = 0.5&#xA;conn_delete_prob        = 0.5&#xA;&#xA;# connection enable options&#xA;enabled_default         = True&#xA;enabled_mutate_rate     = 0.01&#xA;&#xA;feed_forward            = True&#xA;initial_connection      = full&#xA;&#xA;# node add/remove rates&#xA;node_add_prob           = 0.2&#xA;node_delete_prob        = 0.2&#xA;&#xA;# network parameters&#xA;num_hidden              = 6&#xA;num_inputs              = 11&#xA;num_outputs             = 3&#xA;&#xA;# node response options&#xA;response_init_mean      = 1.0&#xA;response_init_stdev     = 0.0&#xA;response_max_value      = 30.0&#xA;response_min_value      = -30.0&#xA;response_mutate_power   = 0.0&#xA;response_mutate_rate    = 0.0&#xA;response_replace_rate   = 0.0&#xA;&#xA;# connection weight options&#xA;weight_init_mean        = 0.0&#xA;weight_init_stdev       = 1.0&#xA;weight_max_value        = 30&#xA;weight_min_value        = -30&#xA;weight_mutate_power     = 0.5&#xA;weight_mutate_rate      = 0.8&#xA;weight_replace_rate     = 0.1&#xA;&#xA;[DefaultSpeciesSet]&#xA;compatibility_threshold = 3.0&#xA;&#xA;[DefaultStagnation]&#xA;species_fitness_func = max&#xA;max_stagnation       = 3&#xA;species_elitism      = 2&#xA;&#xA;[DefaultReproduction]&#xA;elitism            = 3&#xA;survival_threshold = 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; the previous character positions are the position in the previous frame, and if the game runs at 60 fps the previous position is not that different from the current one...&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Note2:&lt;/em&gt; the game score is a bit more complex than just jumping on platforms, the bot should also be rewarded for combos that can make him jump higher. the combo system is already implemented but I first want to see the bot aiming to the next platform before he learns to jump combo.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32953" LastEditorUserId="32953" LastEditDate="2021-03-18T06:30:28.007" LastActivityDate="2021-03-18T06:30:28.007" Title="How to select good inputs and fitness function to achive good results with NEAT for Icy Tower bot" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;evolutionary-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20937" PostTypeId="2" ParentId="20910" CreationDate="2020-05-05T13:58:32.923" Score="5" Body="&lt;p&gt;The book &lt;a href=&quot;http://www.hdip-data-analytics.com/_media/resources/pdf/s4/grokking_deep_learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grokking Deep Learning&lt;/a&gt;, by Andrew Trask (a PhD student at Oxford University and a research scientist at DeepMind), a wonderful, clean, and plain-English discussion of the basic mechanics that go on under the hood of neural networks - from data flow to updating of weights.  It is written without a slant on normally-wonky math, the concepts are presented and then advanced at a digestible pace for anyone. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few more possibly useful resources.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/introduction-to-neural-networks/playground-exercises&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks: Playground Exercises&lt;/a&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://cloudacademy.com/course/convolutional-neural-networks/exercise-1-solution/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Getting Started With Deep Learning: Convolutional Neural Networks&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://tensorchiefs.github.io/dl_course/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep learning focuses on practical aspects of deep learning&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.zemris.fer.hr/~ssegvic/du/lab1en.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;First lab assignment in Deep learning&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://deeplearning.stanford.edu/tutorial/supervised/ExerciseConvolutionalNeuralNetwork/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deeplearning stanford&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36737" LastEditorUserId="36737" LastEditDate="2020-05-05T18:56:19.450" LastActivityDate="2020-05-05T18:56:19.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20939" PostTypeId="1" CreationDate="2020-05-05T14:49:03.203" Score="3" ViewCount="109" Body="&lt;p&gt;I am training an autoencoder on (general) image data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use binary crossentropy loss function, but it is not very informative when I want to evaluate the performance of my autoencoder. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An obvious performance metric would be pixel-wise MSE, but it has its own downsides, shown on some toy examples in an image from &lt;a href=&quot;https://arxiv.org/pdf/2001.03444.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper from Pihlgren et al&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B0MlB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B0MlB.png&quot; alt=&quot;Examples where MSE is not a useful performance metric&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same paper, the authors suggest using perceptual loss, but it seems complicated and not well-studied. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/4ptq05/how_to_measure_the_quality_of_the_representations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;some&lt;/a&gt; &lt;a href=&quot;https://stats.stackexchange.com/questions/274123/evaluating-an-autoencoder-possible-approaches&quot;&gt;other&lt;/a&gt; &lt;a href=&quot;https://www.quora.com/How-can-I-understand-my-Autoencoder-or-RBM-is-good-without-having-any-labeled-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;instances&lt;/a&gt; of this question, but there doesn't seem to be a concensus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that it depends on the application, but &lt;strong&gt;I want to know if there are some general guidelines as to which performance metric to use when training autoencoders on image data&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36769" LastActivityDate="2020-07-24T13:02:00.243" Title="How to evaluate the performance of an autoencoder trained on image data?" Tags="&lt;autoencoders&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20940" PostTypeId="2" ParentId="20903" CreationDate="2020-05-05T15:07:04.587" Score="0" Body="&lt;p&gt;If you want, you can do training and testing in RL. &#xA;Exactly the same usage, training for building up a policy, and testing for evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In supervised learning, if you use test data in training, it is like cheating. You cannot trust the evaluation. That's why we separate train and test data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Objective of RL is a little different. &#xA;RL trying to find the optimal policy. Since RL collects the information by doing, while the agent explores the environment (for more information), there might be lost in the objective function. But, it might be inevitable for a better future gain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-arm bandit example,  If there are 10 slot machines. They will return random amounts of money. They have different expected returns. I want to find the best way to maximize my gain. easy, I have to find the machine with the greatest expected return and use only the machine. How to find the best machine?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have a training and testing (periods),&#xA;For example, I will give you an hour of the training period, so it doesn't matter if you lose or how much you earn. And in the testing period, I will evaluate your performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would you do? In the training period, you will try as much as possible, without considering the performance/gain. And in the testing period, you will use only the best machine you found. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not a typical RL situation. RL is trying to find the best way, Learning by doing. All the results while doing are considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;suppose...&#xA;I tried all 10 machines once each. And, the No.3 machine gave me the most money. But I am not sure that it is the best machine, because all the machines provide a RANDOM amount. If I keep using the No.3 machine, it might be a good idea, because according to the information so far, it is the best machine. However, You might miss the better machine if you don't try other machines due to randomness. But if you try other machines, you might lose an opportunity to earn more money. &#xA;What should I do?  This is a well-known Exploration and Exploitation trade-off in RL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL trying to maximize the gain including the gains right now and the gains in the future.&#xA;In other words, the performance during training also considered as its performance.&#xA;That's why RL is not unsupervised nor supervised learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in some situations, you might want to separate training and testing. &#xA;RL is designed for an agent who interacts with the environment. However,  in some cases, (for example), rather than having an interactive playground, you have data of interactions. The formulation would be a little different in this case. &lt;/p&gt;&#xA;" OwnerUserId="23788" LastActivityDate="2020-05-05T15:07:04.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20941" PostTypeId="1" AcceptedAnswerId="26839" CreationDate="2020-05-05T15:59:24.347" Score="1" ViewCount="386" Body="&lt;p&gt;I'm trying to design an OpenAI Gym environment in which multiple users/players perform actions over time. It's round based and each user needs to take an action before the round is evaluated and the next round starts. The action for one user can be model as a &lt;code&gt;gym.spaces.Discrete(5)&lt;/code&gt; space.&#xA;I want my RL agent to make decisions for all users. I'm wondering how to take multiple actions before progressing time and calculating the reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, what I want is:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;obs = env.reset()&#xA;user_actions = []&#xA;for each user:&#xA;    user_actions.append(agent.predict(obs))&#xA;obs, reward, done, _ = env.step(user_actions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the problem is that I don't immediately know the reward after getting an action since I need to collect all actions before evaluating the round.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could of course extend actions to include actions of all users in one go. But this would be problematic if I have a really large number of users or it even changes over time, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found these two (&lt;a href=&quot;https://ai.stackexchange.com/questions/18115/openai-gym-define-action-space-when-an-agent-can-take-multiple-sub-actions-in-a&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/questions/55539820/how-do-apply-q-learning-to-an-openai-gym-environment-where-multiple-actions-are&quot;&gt;2&lt;/a&gt;) related questions, but they didn't solve my problem.&lt;/p&gt;&#xA;" OwnerUserId="19928" LastActivityDate="2021-04-18T11:21:00.870" Title="OpenAI Gym: Multiple actions in one step" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;environment&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20942" PostTypeId="1" CreationDate="2020-05-05T16:17:44.210" Score="3" ViewCount="38" Body="&lt;p&gt;I had the same question when I am reading the RL textbook from Sutton Bartol &lt;a href=&quot;https://cstheory.stackexchange.com/q/38341/34637&quot;&gt;as posted here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/coSau.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/coSau.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we update &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{\mu (A_t | S_t)}$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi (A_t | S_t)}{\mu (A_t | S_t)}$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that, with the updating rule from the textbook, whatever action &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; decides to choose, we automatically assume that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; will choose it with 100% probability. But &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is greedy with respect to Q. How does this assumption make sense?   &lt;/p&gt;&#xA;" OwnerUserId="36120" LastEditorUserId="2444" LastEditDate="2020-05-05T17:22:07.610" LastActivityDate="2020-05-05T17:22:07.610" Title="Why do we update $W$ with $\frac{1}{\mu (A_t | S_t)}$ instead of $\frac{\pi (A_t | S_t)}{\mu (A_t | S_t)}$ in off-policy Monte Carlo control?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20944" PostTypeId="2" ParentId="20903" CreationDate="2020-05-05T16:23:06.487" Score="4" Body="&lt;h1&gt;Reinforcement Learning Workflow&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The general workflow for using and applying reinforcement learning to solve a task is the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RV6cf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RV6cf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Create the Environment&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Define the Reward&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Create the Agent&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Train and Validate the Agent&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Deploy the Policy&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Training&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Training in Reinforcement learning employs a &lt;strong&gt;system of rewards&lt;/strong&gt; and &lt;strong&gt;penalties&lt;/strong&gt; to compel the computer to solve a problem by itself.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Human involvement is &lt;strong&gt;limited&lt;/strong&gt; to &lt;strong&gt;changing&lt;/strong&gt; the environment and &lt;strong&gt;tweaking&lt;/strong&gt; the system of rewards and penalties.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As the computer &lt;strong&gt;maximizes the reward&lt;/strong&gt;, it is prone to seeking unexpected ways of doing it. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Human involvement is focused on &lt;strong&gt;preventing&lt;/strong&gt; it from &lt;strong&gt;exploiting the system&lt;/strong&gt; and &lt;strong&gt;motivating&lt;/strong&gt; the machine to perform the task in the way expected.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reinforcement learning is useful when there is no &lt;strong&gt;“proper way”&lt;/strong&gt; to perform a task, yet there are rules the model has to follow to perform its duties correctly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Example: By tweaking and seeking the optimal policy for deep reinforcement learning, we built an agent that in just &lt;strong&gt;20 minutes&lt;/strong&gt; reached a superhuman level in playing &lt;strong&gt;Atari games&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Similar algorithms, in principle, can be used to build AI for an &lt;strong&gt;autonomous car.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lplhT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lplhT.png&quot; alt=&quot;Car&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Testing&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Debugging RL algorithms is very hard. Everything runs and you are not sure where the problem is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To test if it worked well, if the trained agent is good at what it was trained for, &lt;strong&gt;you take your trained model and apply it to the situation it is trained for&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If it’s something like &lt;strong&gt;chess&lt;/strong&gt; or &lt;strong&gt;Go&lt;/strong&gt;, you could &lt;strong&gt;benchmark&lt;/strong&gt; it against other engines (say stockfish for chess) or human players.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can also define metrics for performance, ways of measuring the quality of the agent’s decisions. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In some settings (e.g a &lt;strong&gt;Reinforcement Learning Pacman player&lt;/strong&gt;), the game score literally defines the target outcome, so you can just evaluate your model’s performance based on that metric.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="36737" LastEditorUserId="2444" LastEditDate="2020-05-05T22:08:17.043" LastActivityDate="2020-05-05T22:08:17.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20945" PostTypeId="2" ParentId="20903" CreationDate="2020-05-05T16:47:43.480" Score="1" Body="&lt;p&gt;The goal of the reinforcement learning (RL) is to use data obtained via interaction with the environment to &lt;strong&gt;solve the underlying Markov Decision Process&lt;/strong&gt; (MDP). &quot;Solving the MDP&quot; is tantamount to finding the optimal policy (with respect to the MDP's underlying dynamics which are usually assumed to be stationary). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Training&lt;/strong&gt; is the process of using data in order to find the optimal policy. &#xA;&lt;strong&gt;Testing&lt;/strong&gt; is the process of evaluating the (final) policy obtained by training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, since we're generally testing the policy on the same MDP we used for training, the &lt;strong&gt;distinction between the training dataset and the testing set&lt;/strong&gt; is no longer as important as it is the case with say supervised learning. Consequently, classical notions of &lt;strong&gt;overfitting and generalization&lt;/strong&gt; should be approached from a different angle as well.&lt;/p&gt;&#xA;" OwnerUserId="36748" LastActivityDate="2020-05-05T16:47:43.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20946" PostTypeId="1" AcceptedAnswerId="20953" CreationDate="2020-05-05T16:48:59.533" Score="2" ViewCount="43" Body="&lt;p&gt;In pattern recognition systems, when no labeled data is available, what are some common unsupervised learning algorithms for pattern recognition, that can be used?&lt;/p&gt;&#xA;" OwnerUserId="36776" LastEditorUserId="36776" LastEditDate="2020-05-05T18:36:52.530" LastActivityDate="2020-05-05T18:49:58.450" Title="When labelled data is not available, what are some common unsupervised learning algorithms for pattern recognition that can be used?" Tags="&lt;algorithm&gt;&lt;reference-request&gt;&lt;unsupervised-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20948" PostTypeId="1" AcceptedAnswerId="20958" CreationDate="2020-05-05T17:01:18.590" Score="1" ViewCount="630" Body="&lt;p&gt;Let's consider this scenario. I have two conceptually different video datasets, for example a dataset A composed of videos about cats and a dataset B composed of videos about houses. Now, &lt;strong&gt;I'm able&lt;/strong&gt; to extract a feature vectors from both the samples of the datasets A and B, and I know that, each sample in the dataset A is related to one and only one sample in the dataset B and they belong to a specific class (there are only 2 classes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample x1 AND sample y1 ---&amp;gt; Class 1&#xA;Sample x2 AND sample y2 ---&amp;gt; Class 2&#xA;Sample x3 AND sample y3 ---&amp;gt; Class 1&#xA;and so on...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If I extract the feature vectors from samples in both datasets , which is the best way to combine them in order to give a correct input to the classifier (for example a neural network) ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;feature vector &lt;strong&gt;v1&lt;/strong&gt; extracted from &lt;strong&gt;x1&lt;/strong&gt; + feature vector &lt;strong&gt;v1'&lt;/strong&gt; extracted from &lt;strong&gt;y1&lt;/strong&gt; ---&gt; input for classifier&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I ask this because I suspect that neural networks only take one vector as input, while I have to combine two vectors&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="36363" LastEditorUserId="36363" LastEditDate="2020-05-05T17:06:21.857" LastActivityDate="2020-05-06T10:13:02.283" Title="Combine two feature vectors for a correct input of a neural network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20950" PostTypeId="2" ParentId="20728" CreationDate="2020-05-05T17:48:26.670" Score="1" Body="&lt;p&gt;&lt;strong&gt;Note: I assume you mean, countable Action and State Sets by 'Finite'.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;MDP(s) are not exclusive to finite spaces only. They can be used in Continuous/uncountable sets of Action and States too.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Markov Decision Process (MDP)&lt;/em&gt; is a tuple &lt;span class=&quot;math-container&quot;&gt;$(\mathcal S, \mathcal A, \mathcal P^a_s, \mathcal R^a_{ss'}, \gamma, \mathcal S_o)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathcal S$&lt;/span&gt; is a set of States, &lt;span class=&quot;math-container&quot;&gt;$\mathcal A$&lt;/span&gt; is the set of actions, &lt;span class=&quot;math-container&quot;&gt;$\mathcal P_{s}^a: \mathcal A \times \mathcal S \rightarrow [0, 1]$&lt;/span&gt; is a function that denotes Probability distribution over the states if action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is execuited at state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. [1][2]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where, Q-function is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Q^\pi (s,a) = \mathbb E_\pi \left [ \sum \limits_{t=0}^{+\infty} \gamma(t)r_t | s_o = s, a_o = a \right] \tag{*}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; is just special case of Reward function &lt;span class=&quot;math-container&quot;&gt;$\mathcal R^a_{ss'}$&lt;/span&gt;.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if states and actions are discrete, then, the Q-Table Method&lt;a href=&quot;https://towardsdatascience.com/simple-reinforcement-learning-q-learning-fcddc4b6fe56&quot; rel=&quot;nofollow noreferrer&quot;&gt;[3]&lt;/a&gt; which is a state-action matrix helps us to evaluate &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function and optimize efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas, in cases where the state/action sets are infinite or continuous, Deep Networks are preferred to Approximate &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function. &lt;a href=&quot;https://www.geeksforgeeks.org/deep-q-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[4]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q-Learning is Off-Policy method, doesn't require &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; policy function&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;References:&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;R. S. Sutton and A. G. Barto. &lt;em&gt;Reinforcement Learning: An Introduction&lt;/em&gt;. MIT Press,&#xA;1998.&lt;/li&gt;&#xA;&lt;li&gt;Alborz Geramifard, Thomas J. Walsh, Stefanie Tellex, Girish Chowdhary, Nicholas Roy and Jonathan P. How. &lt;em&gt;A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning&lt;/em&gt;. Foundations and Trends (R) in Machine Learning&#xA;Vol. 6, No. 4 (2013) 375–454&lt;/li&gt;&#xA;&lt;li&gt;Andre Violante. &lt;a href=&quot;https://towardsdatascience.com/simple-reinforcement-learning-q-learning-fcddc4b6fe56&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Simple Reinforcement Learning: Q-learning&lt;/em&gt;&lt;/a&gt;, &lt;em&gt;Create a q-table&lt;/em&gt;, &lt;a href=&quot;https://towardsdatascience.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com&lt;/a&gt;, 2019.&lt;/li&gt;&#xA;&lt;li&gt;Alind Gupta. &lt;a href=&quot;https://www.geeksforgeeks.org/deep-q-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Deep Q-Learning&lt;/em&gt;&lt;/a&gt;, Deep Q-Learning, &lt;a href=&quot;https://www.geeksforgeeks.org/deep-q-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.geeksforgeeks.org/deep-q-learning/&lt;/a&gt;, 2020.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: I'd like to thank @nbro for editing suggestions.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="34312" LastEditorUserId="34312" LastEditDate="2020-05-05T18:11:40.323" LastActivityDate="2020-05-05T18:11:40.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20952" PostTypeId="1" CreationDate="2020-05-05T18:42:49.657" Score="1" ViewCount="60" Body="&lt;p&gt;I want a model that outputs the pixel coordinates of the tip of my forefinger, and whether it's touching something or not. Those would be 3 output neurons: 2 for the X-Y coordinates and 1, with a sigmoid activation, wich predicts the probability whether it's touching or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do I need to change in the squeezenet model in order to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(PS: the trained model needs to be the fastest possible (in latency), that's why I wanted to use SqueezeNet)&lt;/p&gt;&#xA;" OwnerUserId="32751" LastEditorUserId="32751" LastEditDate="2020-05-06T06:58:32.143" LastActivityDate="2020-05-06T06:58:32.143" Title="Can SqueezeNet be used for regression?" Tags="&lt;deep-learning&gt;&lt;ai-design&gt;&lt;activation-function&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20953" PostTypeId="2" ParentId="20946" CreationDate="2020-05-05T18:49:58.450" Score="2" Body="&lt;p&gt;There are some unsupervised learning algorithms that can be used for &lt;a href=&quot;https://www.youtube.com/watch?v=cbZUnuyxcVs&quot; rel=&quot;nofollow noreferrer&quot;&gt;pattern recognition&lt;/a&gt; (i.e. the discovery of patterns in data). The most notable one is probably &lt;em&gt;k-means&lt;/em&gt;, which is a &lt;strong&gt;clustering&lt;/strong&gt; algorithm. In k-means, you cluster your unlabeled data into groups (or clusters) based on the distance (or similarity) between them. When a new data point arrives, you'll associate it with the most similar cluster. In this sense, you are performing pattern recognition in an unsupervised way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an excerpt from the famous book &lt;a href=&quot;http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern Recognition and Machine Learning&lt;/a&gt; (2006) by C. Bishop&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In other pattern recognition problems, the training data consists of a set of input vectors &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; without any corresponding target values. The goal in such unsupervised learning problems may be to discover groups of similar examples within the data, where it is called clustering, or to determine the distribution of data within the input space, known as density estimation, or to project the data from a high-dimensional space down to two or three dimensions for the purpose of visualization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, there are other problems, apart from the problem of clustering, that you may want to solve with unsupervised learning algorithms for the purpose of pattern recognition, such as &lt;strong&gt;density estimation&lt;/strong&gt; (see e.g. mixture models) or &lt;strong&gt;dimensionality reduction&lt;/strong&gt; (see e.g. PCA).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-05T18:49:58.450" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20954" PostTypeId="2" ParentId="20844" CreationDate="2020-05-05T19:52:56.323" Score="0" Body="&lt;p&gt;First and foremost, I have to say that this could (and likely will) be a &lt;strong&gt;very hard task&lt;/strong&gt;. Neural networks (&lt;strong&gt;NNs&lt;/strong&gt;) have excelled at computer vision tasks identifying everything from textures to complex objects but what you are trying to do goes beyond that. We (humans) identify trash using the context as much as the object. An object on a table and the same object in a trash bin can &lt;strong&gt;look identical&lt;/strong&gt; but the context tells us which one is trash. Also, &lt;strong&gt;anything&lt;/strong&gt; can be trash. Trash is not an object, its a state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said all that, it sounds like an interesting project and it would be a very useful model so I'll do my best to explain how you would go about trying this. As per your comment, you have outlined the two steps you need.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Identify and extract the trash object from the image.&lt;/li&gt;&#xA;&lt;li&gt;Classify the type of waste&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This could be accomplished using a single model but I'll explain how to do it with two models to make clear what is being done.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Identify and extract trash&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The objective of the first model is to identify and extract the region of interest (the part of the image containing the trash). This is an &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2019/04/introduction-image-segmentation-techniques-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;image segmentation&lt;/a&gt; task typically accomplished using an &lt;a href=&quot;https://arxiv.org/abs/1311.2524&quot; rel=&quot;nofollow noreferrer&quot;&gt;R-CNN&lt;/a&gt; - some guides explaining how they work can be found &lt;a href=&quot;https://towardsdatascience.com/computer-vision-instance-segmentation-with-mask-r-cnn-7983502fcad1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://towardsdatascience.com/r-cnn-3a9beddfd55a&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://cv-tricks.com/object-detection/faster-r-cnn-yolo-ssd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. These methods use supervised learning which requires masks delineating the object of interest to be used as the &lt;em&gt;ground truth&lt;/em&gt;. A mask is a binary image the same size as your input image where each pixel in the image representing your positive class (trash) is set to 1 and all others are set to 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of your Canny/Watershed algorithms could be used as the masks to train the model, &lt;strong&gt;however&lt;/strong&gt; your model will only ever be as good as your masks. Therefore, you might as well use your Canny/Watershed algorithms to accomplish task &lt;strong&gt;1&lt;/strong&gt;. If the masks generated by your algorithm are not of sufficient accuracy you will need to find another way to generate your masks - maybe even doing it manually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approximate rule of thumb for object detection with NNs is that you need 1,000 representative images per class, where class implies a specific object. In this case, unless you are attempting to identify very specific items of trash, you would likely need 100s of thousands of images to obtain a high degree of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Classify the type of waste&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This task should be easier, making the assumption that the segmented image from the first step is always trash. By assigning a label to each segmented image (paper, metal, glass, cardboard, etc) it becomes a normal &lt;strong&gt;multi-class classification&lt;/strong&gt; task which are well documented online with lots of explainations and tutorials.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Single model&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;These tasks could be combined into a single model by modifying the masks in step 1. Instead of using a single binary mask, you could create a multichannel mask of shape &lt;code&gt;m x n x o&lt;/code&gt; where &lt;code&gt;m x n&lt;/code&gt; is your input image size and &lt;code&gt;o&lt;/code&gt; is the number of kinds of waste you are attempting to identify. Each channel is a binary mask for a given type of waste. Therefore, it becomes a matter of not just identifying and segmenting trash but of segmenting each type of trash separately. Needless to say the complexity of the model would be a lot higher and so would the process to create the masks.&lt;/p&gt;&#xA;" OwnerUserId="31980" LastActivityDate="2020-05-05T19:52:56.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20955" PostTypeId="1" CreationDate="2020-05-05T20:12:39.927" Score="1" ViewCount="38" Body="&lt;p&gt;I'm trying to train a RL agent on a custom, highly stochastic environment (MDP). In order to do so I'm using existing implementations of state-of-the-art RL algorithms as provided by &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stable Baselines&lt;/a&gt;. However, no matter what algorithm I try out and despite extensive hyperparameter tuning, I'm failing to obtain any meaningful result. More precisely, a trivial &quot;always perform the same action (0.7,0.7) each time&quot; strategy works better than any of the obtained policies. The environment is highly stochastic (model of a financial market). &lt;strong&gt;How likely is it that the environment is simply &quot;too stochastic&quot; for any meaningful learning to take place?&lt;/strong&gt; If interested, here's the environment code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class environment1(gym.Env):&#xA;def __init__(self):&#xA;    self.t = 0.0 # initial time&#xA;    self.s = 100.0 # initial midprice value&#xA;    self.T = 1.0 # trading period length&#xA;    self.sigma = 2 # volatility constant&#xA;    self.dt = 0.005 # time step&#xA;    self.q = 0.0 # initial inventory&#xA;    self.oldq = 0 # initial old inventory&#xA;    self.x = 0 # initial wealth/cash&#xA;    self.gamma = 0.1 # risk aversion parameter&#xA;    self.k = 1.5 # intensity of arivals of orders&#xA;    self.A = 140 # constant&#xA;    self.done = False&#xA;    self.info = []&#xA;    high = np.array([np.finfo(np.float32).max,&#xA;                     np.finfo(np.float32).max,&#xA;                     np.finfo(np.float32).max],&#xA;                    dtype=np.float32)&#xA;    self.action_space = spaces.Discrete(100)&#xA;    self.observation_space = spaces.Box(-high, high, dtype=np.float32)&#xA;    self.seed()&#xA;    self.state = None&#xA;&#xA;def seed(self, seed=None):&#xA;    self.np_random, seed = seeding.np_random(seed)&#xA;    return [seed]&#xA;&#xA;def step(self, action):&#xA;    old_x, old_q, old_s = self.x, self.q, self.s # current becomes old&#xA;    self.t += 0.005 # time increment&#xA;    P1 = self.dt*self.A*np.exp(-self.k*(action//10)/10) # probabilities of execution&#xA;    P2 = self.dt*self.A*np.exp(-self.k*(action%10)/10)&#xA;    if random.random() &amp;lt; P1: # decrease inventory increase cash&#xA;        self.q -= 1&#xA;        self.x += self.s + (action//10)/10&#xA;    if random.random() &amp;lt; P2: # increase inventory decrease cash&#xA;        self.q += 1&#xA;        self.x -= self.s - (action%10)/10&#xA;    if random.random() &amp;lt; 0.5:&#xA;        self.s += np.sqrt(0.005)*self.sigma&#xA;    else:&#xA;        self.s -= np.sqrt(0.005)*self.sigma&#xA;    self.state = np.array([self.s-100,(self.q-34)/25,(self.t-0.5)/0.29011491975882037])&#xA;    reward = self.x+self.q*self.s-(self.oldx+self.oldq*self.olds)&#xA;    if np.isclose(self.t, self.T):&#xA;        self.done = True&#xA;    self.oldq = self.q&#xA;    self.oldx = self.x&#xA;    self.olds = self.s&#xA;    return self.state, reward, self.done, {}&#xA;&#xA;def reset(self):&#xA;    self.t = 0.0 # initial time&#xA;    self.s = 100.0 # initial midprice value&#xA;    self.T = 1.0 # trading period length&#xA;    self.sigma = 2 # volatility constant&#xA;    self.dt = 0.005 # time step&#xA;    self.q = 0.0 # initial inventory&#xA;    self.oldq = 0.0&#xA;    self.oldx = 0.0&#xA;    self.olds = 100.0&#xA;    self.x = 0.0 # initial wealth/cash&#xA;    self.gamma = 0.1 # risk aversion parameter&#xA;    self.k = 1.5 # intensity of arivals of orders&#xA;    self.A = 140 # constant&#xA;    self.done = False&#xA;    self.info = []&#xA;    self.state = np.array([self.s-100,(self.q-34)/25,(self.t-0.5)/0.29011491975882037])&#xA;    return self.state&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The state space is mostly normalized. The action space consists of 100 possible discrete actions (integers from 0 to 99 which are then transformed to (0.0,0.0),(0.0,0.1),...(1.0,1.0). The reward is simply given by the change in the portfolio value (cash+stock). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I've also tried transforming the action space into a continuous one in order to use DDPG but all to no avail.&lt;/p&gt;&#xA;" OwnerUserId="26195" LastActivityDate="2020-05-05T20:12:39.927" Title="State-of-the-art algorithms not working on a custom RL environment" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20956" PostTypeId="2" ParentId="20728" CreationDate="2020-05-05T21:07:02.903" Score="0" Body="&lt;p&gt;To my knowledge you can't compute or solve an uncountably large MDP numerically. It will need to be discretized in some capacity. The same applies for classic control: you can't optimize over the true functional so you use a discrete approximation to the system and solve that. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-05T21:07:02.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20957" PostTypeId="2" ParentId="20903" CreationDate="2020-05-05T23:40:33.497" Score="5" Body="&lt;h1&gt;What is reinforcement learning?&lt;/h1&gt;&#xA;&lt;p&gt;In reinforcement learning (RL), you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. On each time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent takes the action &lt;span class=&quot;math-container&quot;&gt;$a_t \in \mathcal{A}$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s_t \in \mathcal{S}$&lt;/span&gt;, receives a reward (or reinforcement) signal &lt;span class=&quot;math-container&quot;&gt;$r_t \in \mathbb{R}$&lt;/span&gt; from the environment and the agent and the environment move to another state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1} \in \mathcal{S}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is the action space and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; is the state space of the environment, which is typically assumed to be a Markov decision process (MDP).&lt;/p&gt;&#xA;&lt;h1&gt;What is the goal in RL?&lt;/h1&gt;&#xA;&lt;p&gt;The goal is to find a policy that maximizes the &lt;strong&gt;expected return&lt;/strong&gt;&#xA;(i.e. a sum of rewards starting from the current time step). The policy that maximizes the expected return is called the &lt;strong&gt;optimal policy&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Policies&lt;/h2&gt;&#xA;&lt;p&gt;A policy is a function that maps states to actions. Intuitively, the policy is the strategy that implements the behavior of the RL agent while interacting with the environment.&lt;/p&gt;&#xA;&lt;p&gt;A policy can be deterministic or stochastic. A deterministic policy can be denoted as &lt;span class=&quot;math-container&quot;&gt;$\pi : \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt;. So, a deterministic policy maps a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. A stochastic policy maps states to a probability distribution over actions. A stochastic policy can thus be denoted as &lt;span class=&quot;math-container&quot;&gt;$\pi(a \mid s)$&lt;/span&gt; to indicate that it is a conditional probability distribution of an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given that the agent is in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Expected return&lt;/h2&gt;&#xA;&lt;p&gt;The expected return can be formally written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}\left[ G_t \right] = \mathbb{E}\left[ \sum_{i=t+1}^\infty R_i \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the current time step (so we don't care about the past), &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; is a random variable that represents the probable reward at time step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{i=t+1}^\infty R_i $&lt;/span&gt; is the so-called &lt;em&gt;return&lt;/em&gt; (i.e. a sum of future rewards, in this case, starting from time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;), which is also a random variable.&lt;/p&gt;&#xA;&lt;h2&gt;Reward function&lt;/h2&gt;&#xA;&lt;p&gt;In this context, the most important job of the human programmer is to define a function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt;, the reward function, which provides the &lt;em&gt;reinforcement&lt;/em&gt; (or reward) signal to the RL agent while interacting with the environment. &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; will deterministically or stochastically determine the reward that the agent receives every time it takes action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. The reward function &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is also part of the environment (i.e. the MDP).&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt;, the reward function, is different from &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt;, which is a random variable that represents the reward at time step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. However, clearly, the two are very related. In fact, the reward function will determine the actual &lt;em&gt;realizations&lt;/em&gt; of the random variables &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; and thus of the return &lt;span class=&quot;math-container&quot;&gt;$G_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;How to estimate the optimal policy?&lt;/h2&gt;&#xA;&lt;p&gt;To estimate the optimal policy, you typically design optimization algorithms.&lt;/p&gt;&#xA;&lt;h3&gt;Q-learning&lt;/h3&gt;&#xA;&lt;p&gt;The most famous RL algorithm is probably Q-learning, which is also a numerical and iterative algorithm. Q-learning implements the interaction between an RL agent and the environment (described above). More concretely, it attempts to estimate a function that is closely related to the policy and from which the policy can be derived. This function is called the &lt;strong&gt;value function&lt;/strong&gt;, and, in the case of Q-learning, it's a function of the form &lt;span class=&quot;math-container&quot;&gt;$Q : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt;. The name &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning derives from this function, which is often denoted as &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Q-learning doesn't necessarily find the optimal policy, but there are cases where it is guaranteed to find the optimal policy (but I won't dive into the details).&lt;/p&gt;&#xA;&lt;p&gt;Of course, I cannot describe all the details of Q-learning in this answer. Just keep in mind that, to estimate a policy, in RL, you will typically use a numerical and iterative optimization algorithm (e.g. Q-learning).&lt;/p&gt;&#xA;&lt;h2&gt;What is training in RL?&lt;/h2&gt;&#xA;&lt;p&gt;In RL, training (also known as &lt;em&gt;learning&lt;/em&gt;) generally refers to the use of RL algorithms, such as Q-learning, to estimate the optimal policy (or a value function)&lt;/p&gt;&#xA;&lt;p&gt;Of course, as in any other machine learning problem (such as supervised learning), there are many practical considerations related to the implementation of these RL algorithms, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which RL algorithm to use?&lt;/li&gt;&#xA;&lt;li&gt;Which programming language, library, or framework to use?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These and other details (which, of course, I cannot list exhaustively) can actually affect the policy that you obtain. However, the basic goal during the learning or training phase in RL is to find a policy (possibly, optimal, but this is almost never the case).&lt;/p&gt;&#xA;&lt;h2&gt;What is evaluation (or testing) in RL?&lt;/h2&gt;&#xA;&lt;p&gt;During learning (or training), you may not be able to find the optimal policy, so how can you be sure that the learned policy to solve the actual real-world problem is &lt;em&gt;good enough&lt;/em&gt;? This question needs to be answered, ideally before deploying your RL algorithm.&lt;/p&gt;&#xA;&lt;p&gt;The evaluation phase of an RL algorithm is the &lt;strong&gt;assessment of the quality of the learned policy&lt;/strong&gt; and &lt;strong&gt;how much reward the agent obtains if it follows that policy&lt;/strong&gt;. So, a typical metric that can be used to assess the quality of the policy is to plot &lt;strong&gt;the sum of all rewards received so far&lt;/strong&gt; (i.e. cumulative reward or return) &lt;strong&gt;as a function of the number of steps&lt;/strong&gt;. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the &lt;strong&gt;generalization&lt;/strong&gt; of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [&lt;a href=&quot;http://proceedings.mlr.press/v97/cobbe19a/cobbe19a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;p&gt;The section &lt;a href=&quot;https://artint.info/2e/html/ArtInt2e.Ch12.S6.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;12.6 Evaluating Reinforcement Learning Algorithms&lt;/a&gt; of the book &lt;a href=&quot;https://artint.info/2e/html/ArtInt2e.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: Foundations of Computational Agents&lt;/a&gt; (2017) by Poole and Mackworth provides more details about the evaluation phase in reinforcement learning, so you should probably read it.&lt;/p&gt;&#xA;&lt;p&gt;Apart from evaluating the learned policy, you can also evaluate your RL algorithm, in terms of&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;resources used&lt;/strong&gt; (such as CPU and memory), and/or&lt;/li&gt;&#xA;&lt;li&gt;experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the &lt;strong&gt;data/sample efficiency&lt;/strong&gt; of your RL algorithm)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;robustness/sensitivity&lt;/strong&gt; (i.e., how the RL algorithm behaves if you change certain hyper-parameters); this is also important because RL algorithms can be very sensitive (from my experience)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;What is the difference between training and evaluation?&lt;/h2&gt;&#xA;&lt;p&gt;During training, you want to find the policy. During the evaluation, you want to assess the quality of the learned policy (or RL algorithm). You can perform the evaluation even during training.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T15:42:19.127" LastActivityDate="2020-11-23T15:42:19.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20958" PostTypeId="2" ParentId="20948" CreationDate="2020-05-06T00:27:33.683" Score="2" Body="&lt;p&gt;The easiest way can be the concatenation of the feature vectors to create a single feature vector for each sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume the first sample is made of the pair &lt;span class=&quot;math-container&quot;&gt;$X_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y_1$&lt;/span&gt;. Let the corresponding feature vectors for &lt;span class=&quot;math-container&quot;&gt;$X_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y_1$&lt;/span&gt; be &lt;span class=&quot;math-container&quot;&gt;$\textbf{v}_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\textbf{v}_2$&lt;/span&gt;, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\textbf{v}_1 = [f_1, f_2, \ldots , f_n],\\&#xA;\textbf{v}_2 = [g_1, g_2, \ldots , g_m].&#xA;$$&lt;/span&gt;&#xA;Then, the first sample's feature can be defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\textbf{v} = [f_1, f_2, \ldots , f_n, g_1, g_2, \ldots , g_m].&#xA;$$&lt;/span&gt;&#xA;Eventually, when you pass the latter feature vector to a machine learning model, it will try to capture the dependencies among all of these features, to learn a solution for your task of interest (i.e. classification).&lt;/p&gt;&#xA;" OwnerUserId="36785" LastActivityDate="2020-05-06T00:27:33.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20959" PostTypeId="1" CreationDate="2020-05-06T05:16:08.227" Score="1" ViewCount="77" Body="&lt;p&gt;This is an experiment in order to understand the working of Q table and Q learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the states as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;states = [0,1,2,3]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an arbitrary value for each of these states as shown below (assume index-based mapping) -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;arbitrary_values_for_states = [39.9,47.52,32.92,37.6]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the minimum of the state which will give me the minimum value.&#xA;So I have complimented the values to 50-arbitrary value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;inverse_values_for_states = [50-x for x in arbitrary_values_for_states]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I defined reward function as -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def reward(s,a,s_dash):&#xA;    if inverse_values_for_states[s]&amp;lt;inverse_values_for_states[s_dash]:&#xA;        return 1&#xA;    elif inverse_values_for_states[s]&amp;gt;inverse_values_for_states[s_dash]:&#xA;        return -1&#xA;    else:&#xA;        return 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Q table is initialized as - &#xA;&lt;code&gt;Q = np.zeros((4,4))&lt;/code&gt; (np is numpy)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The learning is carried out as -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;episodes = 5&#xA;steps = 10&#xA;for episode in range(episodes):&#xA;    s = np.random.randint(0,4)&#xA;    alpha0 = 0.05&#xA;    decay = 0.005&#xA;    gamma = 0.6&#xA;    for step in range(steps):&#xA;        a = np.random.randint(0,4)&#xA;        action.append(a)&#xA;        s_dash = a&#xA;        alpha = alpha0/(1+step*decay)&#xA;        Q[s][a] = (1-alpha)*Q[s][a]+alpha*(reward(s,a,s_dash)+gamma*np.max(Q[s_dash]))&#xA;&#xA;        s = s_dash&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is, the table doesn't converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example. For the above scenario - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.argmax(Q[0]) gives 3&#xA;np.argmax(Q[1]) gives 2&#xA;np.argmax(Q[2]) gives 2&#xA;np.argmax(Q[3]) gives 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All of the states should give argmax as 2 (which is actually the index[state] of the minimum value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;when I increase steps to 1000 and episodes to 50, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.argmax(Q[0]) gives 3&#xA;np.argmax(Q[1]) gives 0&#xA;np.argmax(Q[2]) gives 1&#xA;np.argmax(Q[3]) gives 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More, steps and episodes should assure convergence, but this is not visible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need help where I am going wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: This little experiment is needed to make Q-learning applicable to a larger combinatorial problem. Unless I understand this, I don't think I will be able to do that right.&#xA;Also, there is no terminal state because this is an optimization problem. (And I have heard that Q-learning doesn't necessarily needs a terminal state)&lt;/p&gt;&#xA;" OwnerUserId="21983" LastActivityDate="2021-02-15T21:03:45.380" Title="Q table not converging for an arbitrary experiment" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20960" PostTypeId="2" ParentId="12274" CreationDate="2020-05-06T05:46:56.737" Score="0" Body="&lt;h3&gt;Deterministic Policy  :&lt;/h3&gt;&#xA;&lt;p&gt;Its means that for every state you have clear defined action you will take&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;For Example:&lt;/strong&gt; We 100% know we will take action &lt;strong&gt;A&lt;/strong&gt; from state &lt;strong&gt;X&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Stochastic Policy :&lt;/h3&gt;&#xA;&lt;p&gt;Its mean that for every state you do not have clear defined action to take but you have probability distribution for actions to take from that state.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;For example&lt;/strong&gt; there are 10% chance of taking action &lt;strong&gt;A&lt;/strong&gt; from state &lt;strong&gt;S&lt;/strong&gt;, There are 20% chance of taking &lt;strong&gt;B&lt;/strong&gt; from State &lt;strong&gt;S&lt;/strong&gt; and there are 70% chance of taking action &lt;strong&gt;C&lt;/strong&gt; from state &lt;strong&gt;S&lt;/strong&gt;, Its mean we don't have clear defined action to take but we have some probability of taking actions.&lt;/p&gt;&#xA;" OwnerUserId="36104" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-06T06:00:59.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20961" PostTypeId="2" ParentId="12274" CreationDate="2020-05-06T08:11:02.477" Score="0" Body="&lt;p&gt;Apart from the answers above,&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Stochastic Policy function&lt;/em&gt;: &lt;span class=&quot;math-container&quot;&gt;$\pi (s_1s_2 \dots s_n, a_1 a_2 \dots a_n): \mathcal S \times \mathcal A \rightarrow [0,1]$&lt;/span&gt; is the probability distribution function, that, tells the probability that action sequence &lt;span class=&quot;math-container&quot;&gt;$a_1a_2 \dots a_n$&lt;/span&gt; may be chosen in state sequence &lt;span class=&quot;math-container&quot;&gt;$s_1 s_2 \dots s_n$&lt;/span&gt;&lt;a href=&quot;https://www.freecodecamp.org/news/an-introduction-to-policy-gradients-with-cartpole-and-doom-495b5ef2207f/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;&lt;a href=&quot;https://www.computing.dcu.ie/~humphrys/PhD/ch2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;[3]&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;em&gt;Markov Decision Process (MDP)&lt;/em&gt;, it's only &lt;span class=&quot;math-container&quot;&gt;$\pi (s, a)$&lt;/span&gt; following the assumptions&lt;a href=&quot;https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-825-techniques-in-artificial-intelligence-sma-5504-fall-2002/lecture-notes/Lecture20FinalPart1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \mathbb P(\omega_{t+1}| \omega_t, a_t) = \mathbb P(\omega_{t+1}| \omega_t,a_t, \dots \omega_o,a_o)$$&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$\omega \in \Omega$&lt;/span&gt; which is the set of Observations.  &lt;span class=&quot;math-container&quot;&gt;$\mathcal A, \mathcal S$&lt;/span&gt; denote the set of actions and states respectively. Since, the next observation is dependent only on present states and not the past, the policy function only needs the present state and action as parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next action is chosen as[2]: &lt;span class=&quot;math-container&quot;&gt;$$ a^* = \arg \max_a \pi(s_{t+1}, a) \quad\forall a \in \mathcal A $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Deterministic Policy function&lt;/em&gt; &lt;a href=&quot;https://www.computing.dcu.ie/~humphrys/PhD/ch2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;[3]&lt;/a&gt;: is a special case of Stochastic Policy function where for particular &lt;span class=&quot;math-container&quot;&gt;$a_o \in \mathcal A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi(s, a_n) = \delta^o_n$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$a_n \in \mathcal A$&lt;/span&gt;. Here, we are totally certain to choose particular action &lt;span class=&quot;math-container&quot;&gt;$a_o$&lt;/span&gt; in some arbitrary state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and no other. Here &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is &lt;a href=&quot;https://en.wikipedia.org/wiki/Kronecker_delta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kronecker delta&lt;/a&gt;. Since, the probability distribution here is discrete, it's often written in the form of &lt;span class=&quot;math-container&quot;&gt;$\pi(s): \mathcal S \rightarrow \mathcal A$&lt;/span&gt;, where the function takes arbitrary state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and maps it to an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; which is 100% probable.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;IMPORTANT&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;Stochastic Policy function&lt;/strong&gt; is not meant to be confused with the &lt;strong&gt;Transition Function&lt;/strong&gt;&lt;a href=&quot;https://www.freecodecamp.org/news/an-introduction-to-policy-gradients-with-cartpole-and-doom-495b5ef2207f/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt; (which is also a Probability Distribution Function), &lt;span class=&quot;math-container&quot;&gt;$T(s_t, a_t, s_{t+1}): \mathcal S \times \mathcal A \times \mathcal S \rightarrow [0, 1]$&lt;/span&gt; which tells the probability that - at state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, the action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; will lead us to next state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;References:&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ocw.mit.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ocw.mit.edu&lt;/a&gt;. &lt;a href=&quot;https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-825-techniques-in-artificial-intelligence-sma-5504-fall-2002/lecture-notes/Lecture20FinalPart1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;6.825 Techniques in Artificial Intelligence&lt;/em&gt;&lt;/a&gt;. &lt;em&gt;&lt;a href=&quot;https://ocw.mit.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ocw.mit.edu&lt;/a&gt;&lt;/em&gt;. Page Number - 6. Web. 6 May 2020&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Simonini, Thomas. &lt;a href=&quot;https://www.freecodecamp.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.freecodecamp.org&lt;/a&gt; .&lt;a href=&quot;https://www.freecodecamp.org/news/an-introduction-to-policy-gradients-with-cartpole-and-doom-495b5ef2207f/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An introduction to Policy Gradients with Cartpole and Doom&lt;/em&gt;&lt;/a&gt;. 9 May 2018. Web. 6 May 2020.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.computing.dcu.ie/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.computing.dcu.ie/&lt;/a&gt;. Reinforcement Learning. &lt;a href=&quot;https://www.computing.dcu.ie/~humphrys/PhD/ch2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;2.1.1 Special case - Deterministic worlds&lt;/em&gt;&lt;/a&gt;. Web. 6 May 2020 &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="34312" LastEditorUserId="34312" LastEditDate="2020-05-06T08:34:57.927" LastActivityDate="2020-05-06T08:34:57.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20965" PostTypeId="1" AcceptedAnswerId="20969" CreationDate="2020-05-06T10:06:07.347" Score="2" ViewCount="56" Body="&lt;p&gt;Is there any research on machine learning models that provide uncertainty estimation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I train a denoising autoencoder on words and put through a noised word, I'd like it to return a certainty that it is correct given the distribution of data it has been trained on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answering these questions or metrics for uncertainty are both things I am curious about. Just general ways for models to just say &quot;I'm not sure&quot; when it receives something far outside the inputs it's been trained to approximate.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2021-01-17T12:15:35.353" LastActivityDate="2021-01-17T12:15:35.353" Title="Is there any research on models that provide uncertainty estimation?" Tags="&lt;machine-learning&gt;&lt;bayesian-deep-learning&gt;&lt;bayesian-neural-networks&gt;&lt;uncertainty-quantification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20966" PostTypeId="2" ParentId="20948" CreationDate="2020-05-06T10:13:02.283" Score="1" Body="&lt;p&gt;&lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt;Note - Question is bit unclear, in case the answer doesn't addresses the question, please ask for edit/delete Request.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;GENERALIZATION&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Suppose there are multiple datasets denoted by &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt;. Datasets contain a set of Vectors &lt;span class=&quot;math-container&quot;&gt;$x_{j} $&lt;/span&gt;. Mathematically &lt;span class=&quot;math-container&quot;&gt;$A_i = \{ x_j\}_{j=0}^n$&lt;/span&gt;. We've to find an estimator function &lt;span class=&quot;math-container&quot;&gt;$\hat f$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$\hat f( \vec r) = y, \, \vec r \in X$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$X $&lt;/span&gt; is a special dataset created by combining all &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; which helps in classification into &lt;span class=&quot;math-container&quot;&gt;$y \in Y$&lt;/span&gt; which is the set of classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Amir Mentioned out, linearly separable feature can be easily separated by straight combination of vectors i.e. if &lt;span class=&quot;math-container&quot;&gt;$x_u \in A_i, w_v \in A_j \dots$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$r = [x_1 \,x_2 \, \dots  \, x_u \, w_1 \, \dots w_v \dots]$&lt;/span&gt;. Where, &lt;span class=&quot;math-container&quot;&gt;$r \in X$&lt;/span&gt; which is the required dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;There are cases where the features are not linearly separable&lt;/em&gt;, We use &lt;strong&gt;basis expansion methods&lt;/strong&gt;&lt;a href=&quot;https://towardsdatascience.com/non-linear-regression-basis-expansion-polynomials-splines-2d7adb2cc226&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt; to make required shape of hyperplane to separate the features. We create a new dataset combining &lt;span class=&quot;math-container&quot;&gt;$A_i \, \forall i \in C \subset \mathbb N$&lt;/span&gt;. Suppose that the new dataset is &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$r \in X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r = [r_0, r_1, \dots r_n].$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r_1 = u_1^2v_1^2 \\ r_2 = \sin(u_2)\sin(v_2) \\ r_3 = ae^{u_3 + v_3} \\ r_4 = a v_4 v_4 + a_2 u_4^2 v_4^2 + \dots \\ \dots$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$u_p \in A_i; \, v_q \in A_j$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you can use all the creativity to set &lt;span class=&quot;math-container&quot;&gt;$r = [r_1, r_2, \dots , r_n]$&lt;/span&gt; and make a new dataset. What equations and what functions you chose fully depends on the kind of hyperplane shape you want to obtain. Basis expansion is just one of the methods for feature extraction is certainly one of the most flexible too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you feed the newly created vectors into your trained estimator functions (which is Neural Net) which can classify things much easily now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In case of Regression/Classification without Neural Net needs some extra treatment to train the model&lt;/em&gt;&lt;a href=&quot;https://medium.com/datadriveninvestor/improve-your-classification-models-using-mean-target-encoding-a3d573df31e8&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://medium.com/datadriveninvestor/improve-your-classification-models-using-mean-target-encoding-a3d573df31e8&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;Note:&lt;/strong&gt; There is also a big role of encoding. For example, if you encode colors by numbers &lt;span class=&quot;math-container&quot;&gt;$1, 2, 3$&lt;/span&gt; for RGB or &lt;span class=&quot;math-container&quot;&gt;$10,01, 11$&lt;/span&gt; fully changes everything and your features too. In such cases, You may even need different equations to make your required dataset &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and vectors &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;REFERENCES:&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Oleszak, Michal. &lt;a href=&quot;https://towardsdatascience.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com&lt;/a&gt;. &lt;a href=&quot;https://towardsdatascience.com/non-linear-regression-basis-expansion-polynomials-splines-2d7adb2cc226&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Non-linear regression: basis expansion, polynomials &amp;amp; splines&lt;/em&gt;&lt;/a&gt;. Sep 30, 2019. Web. 6 May 2020.&lt;/li&gt;&#xA;&lt;li&gt;Sangarshanan. &lt;a href=&quot;https://medium.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com&lt;/a&gt;. &lt;a href=&quot;https://medium.com/datadriveninvestor/improve-your-classification-models-using-mean-target-encoding-a3d573df31e8&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Improve your classification models using Mean /Target Encoding&lt;/em&gt;&lt;/a&gt;. Jun 23, 2018. Web. 6 May 2020.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="34312" LastActivityDate="2020-05-06T10:13:02.283" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="20968" PostTypeId="1" CreationDate="2020-05-06T11:04:09.407" Score="3" ViewCount="61" Body="&lt;p&gt;Can you explain policy gradient methods and what it means for the policy to be parameterised? I am reading Sutton and Barto book on reinforcement learning and didn't understand well what it is, can you give some examples?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="36821" LastEditDate="2020-05-14T12:10:12.893" LastActivityDate="2020-05-14T14:19:02.627" Title="What does it mean to parameterise a policy in policy gradient methods?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20969" PostTypeId="2" ParentId="20965" CreationDate="2020-05-06T12:14:19.057" Score="2" Body="&lt;p&gt;Yes, there is some research on this topic. It's often called Bayesian machine learning or Bayesian deep learning (but I don't think this is a good name because there are models that aren't really based on a direct application of Bayesian statistics). Some ML/DL models that provide some kind of uncertainty estimation are, for example, &lt;a href=&quot;https://arxiv.org/abs/1506.02142&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Dropout (MC dropout)&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/pdf/1505.05424.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian neural networks&lt;/a&gt;. In theory, these techniques look promising. In practice, I don't think they are the ultimate solution to the problem of uncertainty estimation in deep learning. In fact, e.g. in the case of Bayesian neural networks, they have some disadvantages, such as more parameters to tweak and save.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-06T12:14:19.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20970" PostTypeId="1" CreationDate="2020-05-06T13:31:27.317" Score="1" ViewCount="27" Body="&lt;p&gt;How can I reduce the caputured movement data of a person in a way, that I have filtered the main features of the movement. Or how can I detect pattern/ main features in that data set?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I captured some data with a device (Inertial sensor and motion capturing (x,y,z)) attached to human body. So I have a huge data set. I want to prepare the data set so I have no noise and no &quot;unecessary&quot; data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end I just want to know for example that certain movement behaviour hints to a clueless guy or a person under stress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first thought approaches from association analysis could be useful but I think based on what I have read about the application of such algorithms they are not suitable for this data set.&lt;/p&gt;&#xA;" OwnerUserId="27777" LastActivityDate="2020-05-06T13:31:27.317" Title="How can raw data from a motion sensor (like an IMU) reduced to the main points of the data" Tags="&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20971" PostTypeId="1" CreationDate="2020-05-06T15:24:24.633" Score="0" ViewCount="68" Body="&lt;p&gt;Actually what is mean by 3D face recognition? In normal cases we are extracting face encoding s from a 2D image,right? &#xA;Is 3D face recognition is used for liveness detection? how its possible? &lt;/p&gt;&#xA;" OwnerUserId="31576" LastActivityDate="2021-05-02T00:07:35.707" Title="What is 3D face recognition? and how we can check liveness of a face image?" Tags="&lt;image-recognition&gt;&lt;object-detection&gt;&lt;facial-recognition&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20974" PostTypeId="2" ParentId="20893" CreationDate="2020-05-06T17:00:33.670" Score="2" Body="&lt;p&gt;Let's first clarify a couple of details:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; we're talking about is an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-soft policy (defined to mean that &lt;span class=&quot;math-container&quot;&gt;$\pi(a \vert s) \geq \frac{\epsilon}{\vert \mathcal{A}(s) \vert}$&lt;/span&gt; for all states and all actions).&lt;/li&gt;&#xA;&lt;li&gt;We're not trying to prove equality of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_*$&lt;/span&gt;, but of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tilde{v}_*$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\tilde{v}_*$&lt;/span&gt; denotes the optimal value function in this &quot;new environment&quot; that we're constructing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So, &quot;environment&quot; is basically the &quot;world&quot; that our agent &quot;lives&quot; and acts in. You can think of it as the &quot;rules&quot; that we &quot;play&quot; by. So, you could think of the definitions of our complete state and action spaces as part of the environment, and the function that tells us which successor state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; we end up in whenever we pick an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (i.e. the state transition dynamics) are a part of the environment. And the function that tells us what Rewards we'll obtain in what situations is also a part of the environment. The policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is &lt;strong&gt;not&lt;/strong&gt; a part of the environment; this is the &quot;brain&quot; of the agent itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, recall that here we're not interested in proving that &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; moves towards the true optimal value function &lt;span class=&quot;math-container&quot;&gt;$v_*$&lt;/span&gt; of the &quot;real&quot; environment. We know for a fact that we won't ever become completely equal to that, because we're forcing our policies to have exploratory behaviour by requiring them to be &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-soft, so it would be hopeless to prove such a thing. Instead, we're interested in proving that &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; will move towards whatever value function is the best one that we could possibly achieve &lt;strong&gt;under the restriction that we must have an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-soft policy&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we do in the book here is that we slightly &quot;transform&quot; our environment into a new environment (i.e. we change the rules that we play by just a little bit). This is done in a clever way, such that the thing I just described that we want to prove becomes mathematically equivalent to just proving that &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; moves towards (or becomes equal to) &lt;span class=&quot;math-container&quot;&gt;$\tilde{v}_*$&lt;/span&gt;. Now, if we can prove this for the new environment, we'll automatically have proven the thing that we actually wanted to prove for the &quot;real&quot; environment.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-05-06T17:00:33.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20975" PostTypeId="1" CreationDate="2020-05-06T18:20:48.833" Score="2" ViewCount="160" Body="&lt;pre&gt;&lt;code&gt;class AtariA2C(nn.Module):&#xA;    def __init__(self, input_shape, n_actions):&#xA;        super(AtariA2C, self).__init__()&#xA;&#xA;        self.conv = nn.Sequential(&#xA;            nn.Conv2d(input_shape[0], 32, kernel_size=8, stride=4),&#xA;            nn.ReLU(),&#xA;            nn.Conv2d(32, 64, kernel_size=4, stride=2),&#xA;            nn.ReLU(),&#xA;            nn.Conv2d(64, 64, kernel_size=3, stride=1),&#xA;            nn.ReLU(),&#xA;        )&#xA;&#xA;        conv_output_size = self. _get_conv_out(input_shape)&#xA;&#xA;        self.policy = nn.Sequential(&#xA;            nn.Linear(conv_output_size, 512),&#xA;            nn.ReLU(),&#xA;            nn.Linear(512, n_actions),&#xA;        )&#xA;&#xA;        self.value = nn.Sequential(&#xA;            nn.Linear(conv_output_size, 512),&#xA;            nn.ReLU(),&#xA;            nn.Linear(512, 1),&#xA;        )&#xA;&#xA;    def _get_conv_out(self, shape):&#xA;        o = self.conv(T.zeros(1, *shape))&#xA;        return int(np.prod(o.shape))&#xA;&#xA;    def forward(self, x):&#xA;        x = x.float() / 256&#xA;        conv_out = self.conv(x).view(x.size()[0], -1)&#xA;        return self.policy(conv_out), self.value(conv_out)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In Maxim Lapan's book &lt;code&gt;Deep Reinforcement Learning Hands-on&lt;/code&gt;, after implementing the above network model, it says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The forward pass through the network returns a tuple of two tensors:&#xA;  policy and value. Now we have a large and important function, which&#xA;  takes the batch of environment transitions and returns three tensors:&#xA;  batch of states, batch of actions taken, and batch of Q-values&#xA;  calculated using the formula &lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = \sum_{i=0}^{N-1} \gamma^i r_i&#xA; + \gamma^N V(s_N)$$&lt;/span&gt; This Q_value will be used in two places: to calculate mean squared error (MSE) loss to improve the value&#xA;  approximation, in the same way as DQN, and to calculate the advantage&#xA;  of the action.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am very confused about a single thing. How and why do we &lt;em&gt;calculate the mean squared error loss to improve the value approximation&lt;/em&gt; in Advantage Actor-Critic Algorithm?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-06T19:40:03.520" LastActivityDate="2020-05-07T15:18:21.250" Title="Why do we calculate the mean squared error loss to improve the value approximation in Advantage Actor-Critic Algorithm?" Tags="&lt;python&gt;&lt;q-learning&gt;&lt;actor-critic-methods&gt;&lt;mean-squared-error&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20977" PostTypeId="1" CreationDate="2020-05-06T19:43:47.153" Score="1" ViewCount="42" Body="&lt;p&gt;My task is to classify some texts. I have used word2vec to represent text words and I pass them to an LSTM as input. Taking into account that texts do not contain the same number of words, is it a good idea to create text features of fixed dimension using the word2vec word representations of the text and then classify the text using these features as an input of a neural network? And in general is it a good idea to create text features using this method?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-05-06T19:43:47.153" Title="Creating Text Features using word2vec" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;word2vec&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20978" PostTypeId="2" ParentId="6082" CreationDate="2020-05-06T23:39:03.320" Score="0" Body="&lt;p&gt;I think a nice back-of-the envelope calculation is the intuition for exploding/vanishing gradients in RNNs:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Simplifications&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;diagonalisable weights &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;no non-linearities&lt;/li&gt;&#xA;&lt;li&gt;1 layer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This gives a hidden state &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt; at timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; for input &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt;:  &lt;span class=&quot;math-container&quot;&gt;$h_t = W\cdot h_{t-1} + U\cdot x_t$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$L_t$&lt;/span&gt; be the loss at timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and the total loss &lt;span class=&quot;math-container&quot;&gt;$L = \sum_t L_t$&lt;/span&gt;. Then (eq. 3 -&amp;gt; 5 in the paper)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L_t}{\partial W} \sim = &#xA;\sum_{k=1}^{t} \frac{\partial L_t}{\partial h_t} \frac{\partial h_t}{\partial h_k} \frac{\partial h_k}{\partial W} = \sum_{k=1}^{t}\frac{\partial h_t}{\partial h_k}\times\alpha_{t, k}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's not care about terms regrouped in &lt;span class=&quot;math-container&quot;&gt;$\alpha_{t, k}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial h_t}{\partial h_k} = \prod_{k&amp;lt;i\leq t} \frac{\partial h_i}{\partial h_{i-1}} =  \prod_{k&amp;lt;i\leq t}  W =  \prod_{k&amp;lt;i\leq t} PDP^\top = PD^{t-k}P^{\top}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So you can easily see&lt;span class=&quot;math-container&quot;&gt;$^1$&lt;/span&gt; that if the eigen values of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; (in the diagonal matrix &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;) are larger than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, the gradient will explode with time, and if they are smaller than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, it will vanish.&lt;/p&gt;&#xA;&lt;p&gt;More detailed derivations in &lt;a href=&quot;http://proceedings.mlr.press/v28/pascanu13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the difficulty of training recurrent neural networks&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$^1$&lt;/span&gt; remember &lt;span class=&quot;math-container&quot;&gt;$\lim_{n \to +\infty}|x^n| = +\infty$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$|x|&amp;gt;1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$=0$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$|x| &amp;lt; 1$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="11351" LastEditorUserId="11351" LastEditDate="2020-10-05T05:38:52.627" LastActivityDate="2020-10-05T05:38:52.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20979" PostTypeId="1" CreationDate="2020-05-07T05:33:46.890" Score="1" ViewCount="35" Body="&lt;p&gt;My girlfriend has a masters degree in linguistics and would like to create an AI chatbot personal project to show potential employers her linguistics skills since she is struggling to find a job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately she doesn't know how to program except for extremely basic Python skills. She has been searching for weeks for tools to create a chatbot without needing to program, refusing to ask on forums for help so I'm asking the StackExchange community. Sort of like a plugin/widget for Slack, Facebook Messenger or website that you can just install on your website and just concentrate on the workflow/data/conversational design in a similar way to programming in Scratch or Node-Red.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know nothing about NLP, neural networks or anything like that and I can't understand for the life of me  what exactly it is a linguist needs to do in AI. Adversely, she doesn't have the computer knowledge to understand how an API works, or how to get some sort of service and a chat interface are needed to bootstrap her conversational designs with some code somewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Is there a way for a linguist to create a chatbot without knowing programming or going in too deep, all by themselves? We looked at tools like hubspot.com, where the chatbot design is either multiple choices questions with predefined answers or offers expensive paid solutions for companies. I'm sure there are free educational or community open-source platforms doing this.&lt;/p&gt;&#xA;" OwnerUserId="21790" LastActivityDate="2020-05-07T05:33:46.890" Title="Designing a chatbot personal project with zero coding experience, using an existing platform" Tags="&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20980" PostTypeId="1" AcceptedAnswerId="21207" CreationDate="2020-05-07T07:25:29.817" Score="2" ViewCount="68" Body="&lt;p&gt;According to &lt;a href=&quot;https://arxiv.org/abs/1906.05237&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Knowledge Graph Reasoning for Explainable Recommendation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;pure KG embedding methods lack the ability to discover multi-hop relational paths.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why is it so?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-05-16T11:21:14.633" LastActivityDate="2020-05-16T18:39:17.293" Title="Why can't pure KG embedding methods discover multi-hop relations paths?" Tags="&lt;reinforcement-learning&gt;&lt;recommender-system&gt;&lt;knowledge-graph&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20981" PostTypeId="1" CreationDate="2020-05-07T09:09:40.713" Score="1" ViewCount="27" Body="&lt;p&gt;So I have with me a data of rendered 2D images of a 3D object and along with that, I have the image projection coordinates &lt;code&gt;(X, Y)&lt;/code&gt; of all the voxels that are in the camera perspective in that image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rendered image&lt;a href=&quot;https://i.stack.imgur.com/p5wSM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p5wSM.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The voxel camera projections(This is just for visualization purposes)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ESMKF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ESMKF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to build a CNN which takes an input a rendered image and outputs all the voxel camera projection coordinates &lt;code&gt;(X, Y)&lt;/code&gt;.I was thinking to try out an encoder-decoder based network (like a &lt;code&gt;U-Net&lt;/code&gt;) but shouldn't the image obtained after decoding be the same dimensions as the input image. I want my decoder to output the tensor of coordinates but I am having a hard time thinking about how it would do so.&lt;/p&gt;&#xA;" OwnerUserId="36274" LastEditorUserId="36274" LastEditDate="2020-05-07T09:16:51.720" LastActivityDate="2020-05-07T09:16:51.720" Title="An Encoder-Decoder based CNN to predict a tensor of points" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20982" PostTypeId="1" AcceptedAnswerId="20985" CreationDate="2020-05-07T09:58:45.690" Score="2" ViewCount="107" Body="&lt;p&gt;I have been reading the Sutton and Barto textbook and going through David Silvers UCL lecture videos on YouTube and have a question on the equivalence of two forms of the state-action value function written in terms of the value function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Question 3.13 of the textbook I am able to write the state-action value function as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) = \sum_{s',r}p(s',r|s,a)(r + \gamma v_\pi(s')) = \mathbb{E}[r + \gamma v_\pi(s')|s,a]\;.$$&lt;/span&gt;&#xA;Note that the expectation is &lt;em&gt;not&lt;/em&gt; taken with respect to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the conditional probability of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Now, in David Silver's slides for the Actor-Critic methods of the Policy Gradient lectures, he says that &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\pi_\theta}[r + \gamma v_{\pi_\theta}(s')|s,a] = q_{\pi_\theta}(s,a)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these two definitions equivalent (in expectation)? &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-05-07T16:51:17.903" LastActivityDate="2020-05-07T17:20:47.120" Title="Are these two definitions of the state-action value function equivalent?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;value-functions&gt;&lt;expectation&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20983" PostTypeId="1" AcceptedAnswerId="21007" CreationDate="2020-05-07T11:09:19.920" Score="3" ViewCount="102" Body="&lt;p&gt;I was going through the &lt;a href=&quot;https://discovery.ucl.ac.uk/id/eprint/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero paper&lt;/a&gt; and I was trying to understand everything, but I just can't figure out this one formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0, b)^{\frac{1}{\tau}}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone decode how the policy makes decisions following this formula? I pretty much understood all of the other parts of the paper, and also the temperature parameter is clear to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be a simple question, but can't figure it out.&lt;/p&gt;&#xA;" OwnerUserId="36825" LastEditorUserId="2444" LastEditDate="2020-05-12T16:52:14.630" LastActivityDate="2021-02-07T11:50:37.740" Title="How does the AlphaGo Zero policy decide what move to execute?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;deepmind&gt;&lt;alphago-zero&gt;&lt;alphago&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20984" PostTypeId="1" CreationDate="2020-05-07T12:30:12.890" Score="2" ViewCount="39" Body="&lt;p&gt;I have a question for heuristic search with multiple agents. I know how heuristic search works with one agent (ex. one Pacman) but I don't really understand it with multiple agents. Let's say we have this problem where Worm A has to get to its goal state A and Worm B to B, knowing that the agents can move only in vertical and horizontal way:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wlKP4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wlKP4.png&quot; alt=&quot;Worm A should get to its goal state A and Worm B to its goal state B&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we had only Worm B, the optimal cost from starting position to the goal position would be &lt;code&gt;9&lt;/code&gt;, since one action costs &lt;code&gt;1&lt;/code&gt; and it'd follow the path &lt;code&gt;RIGHT-RIGHT-RIGHT-RIGHT-RIGHT-RIGHT-UP-UP-UP&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, if we have two worms, like in the picture, the optimal cost would be &lt;code&gt;9 + optimal cost for Worm A&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, strictly for this problem with 2 agents, if we use Manhattan distance as a heuristic for &lt;strong&gt;one agent&lt;/strong&gt;, would it be admissible if we take the &lt;strong&gt;average of Worm A and B heuristics&lt;/strong&gt; for a problem with &lt;strong&gt;two agents&lt;/strong&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question, I know for a fact that &lt;strong&gt;sum of two admissible heuristics&lt;/strong&gt; won't be admissible for one agent but would it be for the problem with &lt;strong&gt;two agents&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These two worms are dependent of each other. How? If one worm moves from position X to Y, the position X is marked as a wall and is not an available field to move in. So if one worm has been in a specific position, that position is no more free for moving in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we have something like &lt;code&gt;B^^^X^^^&lt;/code&gt;, where B is the Worm B, &lt;code&gt;^&lt;/code&gt; is an &lt;strong&gt;available&lt;/strong&gt; &lt;strong&gt;field&lt;/strong&gt; and &lt;code&gt;X&lt;/code&gt; is a &lt;strong&gt;wall&lt;/strong&gt;, after one RIGHT action it'll look like &lt;code&gt;XB^^X^^&lt;/code&gt;, after one more RIGHT: &lt;code&gt;XXB^X^^&lt;/code&gt; etc.&lt;/p&gt;&#xA;" OwnerUserId="36830" LastActivityDate="2020-05-07T12:30:12.890" Title="How does heuristic work with multiple agents?" Tags="&lt;heuristics&gt;&lt;multi-agent-systems&gt;&lt;a-star&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20985" PostTypeId="2" ParentId="20982" CreationDate="2020-05-07T13:13:04.883" Score="3" Body="&lt;p&gt;These two definitions are not exactly the same, even though they have a very similar formulation. David Silver's notation is probably an abuse of notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first difference between those two definitions is that, in the case of David Silver's slides, the policy is parametrized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (i.e. the policy could be represented e.g. by a neural network), while, in the first case, the policy is not parametrized, which is a more general way of writing the definition of the state-action value function, because you do not assume how you will represent the policy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second apparent difference (as you have noticed) is that, in the first case, the expectation is taken with respect to transition probability function, while, in the second case, it seems to be taken with respect to the parametrized policy. In the second case, you do not probably know the transition probability function, so you will compute your state-action value function based on your current best estimate of the policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as Brale pointed out in the comments (please, read the comments under this answer!), the definition of the state-action value function is always the same, even if you don't know the MDP (because that's a definition and it doesn't matter if you know the transition probability or not). However, in the case of David Silver's slides, by that notation, I think he means he will compute the state-action value function based on the current estimate of the policy. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-07T14:17:14.967" LastActivityDate="2020-05-07T14:17:14.967" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20986" PostTypeId="2" ParentId="20975" CreationDate="2020-05-07T15:04:02.470" Score="1" Body="&lt;p&gt;I believe that the author is referring to how the networks are trained in Deep RL. Consider Deep Q-Learning where the &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; is approximated using a neural network. Then the loss function used to train the network is &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[(r + \gamma \max_{a'} Q(s',a') - Q(s,a))^2]\;.$$&lt;/span&gt;&#xA;Here, &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \max_{a'} Q(s',a')$&lt;/span&gt; is your target, what you want your network to aim towards, and &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; is what your network predicted. (Note that I have left off some details that can be found in the &lt;a href=&quot;https://daiwk.github.io/assets/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nature paper&lt;/a&gt; for simplicity). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for actor-critic methods, most popular actor-critic methods will use the value function to 'replace' the action-value function by using the following relationship:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[r + \gamma v_\pi(s')] = Q_\pi(s,a)\;.$$&lt;/span&gt;&#xA;This relationship can be proved by looking at exercise 3.13 (or somewhere around there) in the Sutton and Barto textbook. This looks like what the author is doing in the textbook you are reading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on what I said at the start regarding how state-action value functions are trained, it is analogous to train a critic network that approximates the value function in the same way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;e1: spelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e2: added link to nature paper. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-05-07T15:18:21.250" LastActivityDate="2020-05-07T15:18:21.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20987" PostTypeId="1" AcceptedAnswerId="21010" CreationDate="2020-05-07T15:14:16.510" Score="1" ViewCount="84" Body="&lt;p&gt;If I have a convolutional neural network, and I convolve my input tensor with a kernel, the output is a feature map. Is an activation function then applied to this feature map?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If its an image that is a 2D tensor, would the activation function change every single value on this image? &lt;/p&gt;&#xA;" OwnerUserId="35615" LastEditorUserId="2444" LastEditDate="2020-05-07T17:16:31.733" LastActivityDate="2020-05-08T17:23:33.287" Title="Are activation functions applied to feature maps?" Tags="&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20988" PostTypeId="1" CreationDate="2020-05-07T15:18:02.540" Score="1" ViewCount="25" Body="&lt;p&gt;What would be the best way to create a vector representation of roadmap like scans? The goal I am trying to achieve is illustrated below. The left side represents the source image, the right side the output in the form of three vectors. The fussiness on the left is a simulation, not the actual source image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9l7B3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9l7B3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual source image would look more like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qYV42.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qYV42.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am looking at a combination of skeletonization and Hough transform. The result is rather messy though, and seems to warrant quite some extra engineering. Any other suggestions?&lt;/p&gt;&#xA;" OwnerUserId="36619" LastActivityDate="2020-07-07T07:42:36.663" Title="How to create vector representation of roadmap like scans" Tags="&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20989" PostTypeId="2" ParentId="9396" CreationDate="2020-05-07T16:28:21.783" Score="1" Body="&lt;p&gt;I will try to answer the question in a lesser mathematical (and hopefully correct way).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I have used &lt;span class=&quot;math-container&quot;&gt;$V_{\pi}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; interchangeably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We start from LHS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be written in terms of trajectories. Say the probability of observing a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; step trajectory (for a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; step return) is &lt;span class=&quot;math-container&quot;&gt;$p_j^{s}$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; . Thus we can write the expected return as sum of returns from all trajectories multiplied with the probability of the trajectory:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\pi}  [G_{t:t+n}|S_t = s] = \sum_j p_j^sG_{t:t+n}^j = \sum_j p_j^s [R_{t+1}^j + \gamma R_{t+2}^j.....\gamma^{n-1}R_{t+n}^j + \gamma^n V_{t+n-1}(S_{t + n})^j]$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use @Dennis's terminology for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; step rewards i.e&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$R_{t:t+n}^j \doteq R_{t + 1}^j + \gamma R_{t + 2}^j + \dots + \gamma^{n - 1} R_{t + n}^j$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we know &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; is nothing but &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}  [G_{t:t+n}^{\pi}|S_t = s]$&lt;/span&gt; where I have used &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}^{\pi}$&lt;/span&gt; to denote that the returns are actual returns if we have evaluated the policy completely (the value functions are consistent with the policy) for every state(using infinite episodes maybe) i.e &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}^{\pi} = R_{t:t+n} + \gamma^n V_{\pi}(S_{t + n})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now if we evaluate the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert = \max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - \mathbb{E}_{\pi}  [G_{t:t+n}^{\pi}|S_t = s]\Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is further written in form of trajectory probabilities (for easier comprehension):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \sum_j p_j^s(R_{t:t+n}^j+\gamma^n V_{t+n-1}^j(S_{t + n})) - \sum_j p_j^s(R_{t:t+n}^j+\gamma^n V_{\pi}^j(S_{t + n})) \Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the equation can be simplified by cancelling the reward terms (&lt;span class=&quot;math-container&quot;&gt;$R_{t:t+n}^j$&lt;/span&gt;) as they are the same for a trajectory, and thus is common to both the terms  we get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \gamma^n\sum_j p_j^s( V_{t+n-1}^j(S_{t + n})-V_{\pi}^j(S_{t + n}))\Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is basically the expectation of the deviation of each and every state at the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; th step (from its actual value &lt;span class=&quot;math-container&quot;&gt;$V_{\pi}$&lt;/span&gt;,), starting from &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; and multiplied by a discount factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now using the identity &lt;span class=&quot;math-container&quot;&gt;$E[X] \leq \max X$&lt;/span&gt; we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \gamma^n\sum_j p_j^s( V_{t+n-1}^j(S_{t + n})-V_{\pi}^j(S_{t + n}))\Bigr\rvert \leq \max \Bigl\lvert \gamma^n( V_{t+n-1}^j(S_{t + n})-V_{\pi}^j(S_{t + n}))\Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which can finally be written as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert \leq \max \Bigl\lvert \gamma^n( V_{t+n-1}^j(S_{t + n})-V_{\pi}^j(S_{t + n}))\Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the RHS is true for only those states reachable from &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt; via a trajectory, but since it has a maximizing operation we can include the whole state space in the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operation without any problem to finally write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert \leq \max_s \Bigl\lvert \gamma^n( V_{t+n-1}(s)-V_{\pi}(s))\Bigr\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which concludes the proof.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-05-08T04:07:34.307" LastActivityDate="2020-05-08T04:07:34.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20991" PostTypeId="1" CreationDate="2020-05-07T16:47:20.307" Score="1" ViewCount="31" Body="&lt;p&gt;I'm trying to process product data for an e-commerce platform. The goal is to understand products' size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to show you some examples on how messy product dimension description is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Overall Dimensions: 66 in W x 41 in D x 36 in H&#xA;Overall: 59 in W x 28.75 in D x 30.75 in H&#xA;92w 37d 32h&quot;,&#xA;86.6 in W x 33.9 in D x 24 in H&#xA;W: 95.75\&quot; D: 36.5\&quot; H: 28.75\&quot;&quot;,&#xA;W: 96\&quot; D: 39.25\&quot; H: 32\&quot;&quot;,&#xA;&quot;118\&quot;W x 35\&quot;D x 33\&quot;T.&quot;,&#xA;&quot;28 L x 95 W x 41 H&quot;&#xA;&quot;95\&quot; W x 26.5\&quot; H x 34.75\&quot; D&quot;&#xA;&quot;98\&quot;W x 39\&quot;D x 29\&quot;H&quot;&#xA;&quot;28\&quot; High x 80\&quot; Wide x 32\&quot; Deep&quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now assume that the product dimension description is short &amp;lt; 60 characters, I trained a two layer bidirectional LSTM, which can handle this task perfectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is, the above dimension is usually embedded in a long context (as a part of the product description). How can I extract the useful information from the long context and understand it? My LSTM can only accept context size of 60.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What language model is more suitable for this?&lt;/p&gt;&#xA;" OwnerUserId="33082" LastActivityDate="2020-05-07T16:47:20.307" Title="Which NLP model to use to handle long context?" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20993" PostTypeId="1" CreationDate="2020-05-07T17:57:12.583" Score="2" ViewCount="65" Body="&lt;p&gt;If I have the fitness of each genome, how do I determine which genome will crossover with which, and so on, so that I get a new population?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I can't find anything about it in the original paper, so I ask here?&lt;/p&gt;&#xA;" OwnerUserId="36844" LastEditorUserId="2444" LastEditDate="2020-05-07T18:02:52.950" LastActivityDate="2021-05-25T17:07:50.773" Title="How do I determine the genomes to use for crossover in NEAT?" Tags="&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;neuroevolution&gt;&lt;crossover-operators&gt;&lt;genetic-operators&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20994" PostTypeId="1" CreationDate="2020-05-07T18:30:37.047" Score="1" ViewCount="87" Body="&lt;p&gt;I am fairly new to reinforcement learning (RL) and deep RL. I have been trying to create my first agent (using A3C) that selects an optimal path with the reward being some associated completion time (the more optimal the path is, packets will be delivered in less time kind of).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in each episode/epoch, I do not have a certain batch size for updating my NN's parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make it more clear, let's say that, in my environment, on each step, I need to perform a request to the servers, and I have to select the optimal path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, each execution in my environment does not contain the same amount of files. For instance, I might have 3 requests as one run, and then 5 requests for the next one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The A3C code I have at hand has a batch size of a 100. In my case, that batch size is not known a priori.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that going to affect my training? Should I find a way to keep it fixed somehow? And how can one define an optimal batch size for updating?&lt;/p&gt;&#xA;" OwnerUserId="35978" LastEditorUserId="2444" LastEditDate="2020-05-20T12:16:51.203" LastActivityDate="2020-05-20T12:16:51.203" Title="How should I deal with variable batch size in A3C?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;a3c&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20995" PostTypeId="1" AcceptedAnswerId="21156" CreationDate="2020-05-07T20:03:29.633" Score="1" ViewCount="102" Body="&lt;p&gt;&lt;strong&gt;On-Policy Algorithms&lt;/strong&gt; like PPO directly maximize the performance objective or an approximation of it. They tend to be quite stable and reliable but are often sample inefficient. &lt;strong&gt;Off-Policy Algorithms&lt;/strong&gt; like TD3 improve the sample inefficiency by reusing data collected with previous policies, but they tend to be less stable. (Source: &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kinds of RL Algorithms - Spinning up - OpenAI&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at learning curves comparing SOTA algorithms, we see that off-policy algorithms quickly improve performance at the training's beginning. Here &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/bench.html#walker2d-tensorflow-versions&quot; rel=&quot;nofollow noreferrer&quot;&gt;an example&lt;/a&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kLViU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kLViU.png&quot; alt=&quot;3M timestep benchmark of different algorithms for Walker2d-v3&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we start training off-policy and after some time use the learned and quickly improved policy to init the policy network of an on-policy algorithm? &lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-05-14T13:41:02.223" Title="Can we combine Off-Policy with On-Policy Algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;open-ai&gt;&lt;off-policy-methods&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20997" PostTypeId="1" CreationDate="2020-05-07T21:14:24.493" Score="0" ViewCount="162" Body="&lt;p&gt;I wanted to train a model that recognizes sign language. I have found a dataset for this and was able to create a model that would get 94% accuracy on the test set. I have trained models before and my main goal is not to have the best model (I know 94% could easiy be tuned up). However these models where always for class exercises and thus were never used on 'real' new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I took a new picture of my hand that I know I wanted to be a certain letter (let's assume A).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my model was trained on 28x28 images, I needed to re-size my own image because it was larger. After that I fed this image to my model only to get a wrong classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://imgur.com/a/QE6snTa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imgur.com/a/QE6snTa&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are my pictures (upper-left = my own image (expected class A), upper-right = an image of class A (that my model correctly classifies as A), bottom = picture of class Z (the class my image was classified as)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can clearly see that my own image looks for more like the image of class A (that I wanted my model to predict), than the model it did predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be reasons that my model does not work on real-life images? (If code is wanted I can provide it ofcourse but since I don't know where I go wrong, it seemed out of line to copy all the code).&lt;/p&gt;&#xA;" OwnerUserId="34359" LastActivityDate="2020-06-08T15:56:17.367" Title="My CNN model performs bad on new (self-created) pictures, what are possible reasons?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20998" PostTypeId="2" ParentId="20997" CreationDate="2020-05-08T01:03:23.660" Score="1" Body="&lt;p&gt;I’m assuming that you used LeNet (our some other model with small number of parameters) since your training image size is 28x28. Note that LeNet doesn’t generalize well to new images. I think it performs fine (&gt;90%) on MNIST but not good on CIFAR10 (&gt;60%) albeit both datasets contain similar size image. (Just trying to remember the performance from PyTorch implementations). It’s more about if the model has capacity to learn the complexity of dataset. CIFAR10 is more complex and harder to model than MNIST. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;LeNet is a small image classification model (in terms of capacity) so it cannot nicely learn the correlation between pixels of input images well and therefore doesn’t perform well on unseen images. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case it seems like your model has overfit to training examples. It might perform well on test images because both training and test subsets are sampled from same data generating distribution but real-world images it might experience in future might be different (like your own hand). If it doesn’t perform well on unseen images we say it has not generalized well, which looks the case in your situation. In this case you need a validation set to validate that your model generalizes to unseen images. If you have it then you should use it in early stopping regularization technique. You can also add other regularizers to your model (the simplest one is weight decay). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But instead of inventing your network architecture why don’t you use models like ResNet. Just fine-tune the pre-trained ResNet on your own dataset. I’d prefer to fine-tune personally in this situation because data distribution it was trained on (ImageNet) is pretty different from your hand sign dataset. In other case if your dataset contained nature and surrounding images I’d rather freeze the parameters of fixed-feature extractor layers and trained only the last few layers of ResNet (or similar model). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps! &lt;/p&gt;&#xA;" OwnerUserId="5260" LastActivityDate="2020-05-08T01:03:23.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20999" PostTypeId="1" CreationDate="2020-05-08T01:25:22.773" Score="1" ViewCount="50" Body="&lt;p&gt;Are there any general guidelines for dealing with imbalanced data through upsampling/downsampling? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://developers.google.com/machine-learning/data-prep/construct/sampling-splitting/imbalanced-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google developer guide&lt;/a&gt; suggests performing downsampling with upweighting, but for the most part I've found upsampling usually works better in practice (&lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/6czlty/d_upsampling_vs_weighting_whats_the_best_way_to/&quot; rel=&quot;nofollow noreferrer&quot;&gt;some corroboration&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any clear consensus or empirical study of what works in practice, or when to use which?  Does it matter which classification algorithm you use?&lt;/p&gt;&#xA;" OwnerUserId="18086" LastEditorUserId="11539" LastEditDate="2020-05-09T22:17:23.410" LastActivityDate="2020-05-09T22:17:23.410" Title="Are there any general guidelines for dealing with imbalanced data through upsampling or downsampling?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;data-science&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21000" PostTypeId="1" CreationDate="2020-05-08T02:56:09.880" Score="0" ViewCount="349" Body="&lt;p&gt;I'm looking to implement a AI for the turn-based game Mastermind in Node.JS, using Google's Tensorflow library. Basically the AI needs to predict the 4D input for the optimal 2D output &lt;code&gt;[0,4]&lt;/code&gt; with a given list of 4D inputs and 2D outputs from previous turns in the form of &lt;code&gt;[input][output]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimal output would be &lt;code&gt;[0,4]&lt;/code&gt;, which would be the winning output. The training data looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[1,2,3,4][0,1] [0,5,2,6][3,1] [0,2,5,6][2,2] [6,5,2,0][4,0] [5,2,0,6][0,4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So given these previous turns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[1,2,3,4][0,1] [0,5,2,6][3,1] [0,2,5,6][2,2] [6,5,2,0][4,0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the AI would predict an input of &lt;code&gt;[5,2,0,6]&lt;/code&gt; for the output &lt;code&gt;[0,4]&lt;/code&gt;.&#xA;I've looked at &lt;a href=&quot;https://ai.stackexchange.com/questions/3463/neural-network-model-to-infer-inputs-given-an-output&quot;&gt;this post&lt;/a&gt; but it talks about only inferring input for a output without any context. In Mastermind, the context of previous guesses and results from them are critical&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My algorithm would need to use the information from previous turns to determine the best input for the winning output (&lt;code&gt;[0,4]&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: &lt;strong&gt;How can I implement AI for Mastermind?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="36853" LastEditorUserId="1847" LastEditDate="2020-05-09T11:48:37.333" LastActivityDate="2020-06-09T16:07:20.533" Title="How to implement AI strategy for Mastermind" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;prediction&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21002" PostTypeId="1" AcceptedAnswerId="21015" CreationDate="2020-05-08T08:14:17.277" Score="3" ViewCount="140" Body="&lt;p&gt;We all have heard about how beneficial AI can be in health. There are plenty of papers and research about confronting against diseases, like cancer. However in 2020 with COVID-19 be one of the most serious health problems that has caused thousands of deaths worldwide,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is AI already being used in the drug industry to combat the COVID-19? If yes, can you, please, provide a reference?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-05-08T14:51:52.647" LastActivityDate="2020-05-08T21:44:15.203" Title="Is AI already being used in the drug industry to combat the COVID-19?" Tags="&lt;reference-request&gt;&lt;healthcare&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21003" PostTypeId="1" CreationDate="2020-05-08T11:15:34.553" Score="2" ViewCount="138" Body="&lt;p&gt;How long should the state-dependent baseline be trained at each iteration? Or what baseline loss should we target at each iteration for use with policy gradient methods? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using this equation to compute the policy gradient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_{\theta} J\left(\pi_{\theta}\right)=\underset{\tau \sim \pi_{\theta}}{\mathrm{E}}\left[\sum_{t=0}^{T} \nabla_{\theta} \log \pi_{\theta}\left(a_{t} | s_{t}\right)\left(\sum_{t^{\prime}=t}^{T} R\left(s_{t^{\prime}}, a_{t^{\prime}}, s_{t^{\prime}+1}\right)-b\left(s_{t}\right)\right)\right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html#baselines-in-policy-gradients&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is mentioned to use one or more gradient steps, so is it a hyper-parameter to be found using random search? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way we can use an adaptive method to find out when to stop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an experiment to train Cartpole-v2 using a policy gradient with baseline, I found the results are better when applying 5 updates than when only a single update was applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am referring to the number of updates to take on a single batch of q values encountered across trajectories collected using current policy.&lt;/p&gt;&#xA;" OwnerUserId="36861" LastEditorUserId="2444" LastEditDate="2020-05-18T12:27:34.377" LastActivityDate="2021-02-12T14:08:05.767" Title="How long should the state-dependent baseline for policy gradient methods be trained at each iteration?" Tags="&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;&lt;adam&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21005" PostTypeId="2" ParentId="20988" CreationDate="2020-05-08T13:32:50.390" Score="1" Body="&lt;p&gt;It turned out that my intuition was not far off. The skeletonization is a good step. The Hough transform though is not a good way to create a graph of the roadmap. It seems that the Ramer–Douglas–Peucker algorithm can help out here. This algorithm first takes all the skeleton pixels as input, and sees this as a starting graph. The algorithm then proceeds to remove intermediary pixels that do not add information to the shape of the graph.&lt;/p&gt;&#xA;" OwnerUserId="36619" LastEditorUserId="36619" LastEditDate="2020-07-07T07:42:36.663" LastActivityDate="2020-07-07T07:42:36.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21006" PostTypeId="2" ParentId="20997" CreationDate="2020-05-08T14:06:59.953" Score="1" Body="&lt;p&gt;This is not an uncommon situation. The data set your model is trained on represents a certain probability distribution. Your test set is most likely a good representation from that distribution so your test results will be good. &#xA;However when you use real world images they may or may not have a similar distribution. Typically if the training set is large and diverse it is a good representation of the distribution and when the model is used to classify a real world image it will do so correctly.&#xA;I think I know of the data set you are working with and if I recall it is fairly large.&#xA;So the problem may be that your model is not complex enough to fully capture the complexity of the data. You can test that fairly simply by using transfer learning with a model that is known to be effective for image classification. I recommend using the MobileNet model. It contains only about 4 million parameters but is about as accurate as larger models containing 10 times as many parameters. So MobileNet is not computationally expensive to train. Documentation can be found &lt;a href=&quot;https://keras.io/api/applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-05-08T14:06:59.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21007" PostTypeId="2" ParentId="20983" CreationDate="2020-05-08T14:14:13.527" Score="1" Body="&lt;p&gt;The formula in question uses a function N(state, action) that defines a visit count of a state-action pair (introduced on page 3). To describe how it is used, lets first describe the steps of AlphaGo Zero as a whole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 4 &quot;phases&quot; to the Monte-Carlo tree search in AlphaGo Zero as depicted in Figure 2. The first 3 expand and update the tree and together are the &quot;search&quot; in Monte-Carlo tree &quot;search&quot; in AlphaGo Zero.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Select&lt;/strong&gt; an edge (action) in the tree with maximum action-value Q (plus upper confidence bound U) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Expand and evaluate&lt;/strong&gt; the leaf node using the network&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Backup&lt;/strong&gt; Action-values Q are updated to track the evaluations of the Value in the subtree.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Play&lt;/strong&gt; - After the &quot;search&quot; is complete, the search probabilities are returned proportional to the visitation count of the nodes of the tree.&lt;strong&gt;*&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt; This is where the formula in question comes into play (pun intended). During the &quot;search&quot;, nodes that looked good were expanded on and thus their visitation counts were updated. So the formula is essentially describing this logic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Good nodes have higher counts, so choose the nodes with higher counts often&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if there is a really good node but it hasn't been visited a lot so its not chosen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where the temperature parameter comes in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the temperature is 1, this selects moves proportionally to their visit counts.&lt;/li&gt;&#xA;&lt;li&gt;If the temperature is 0 (not actually but rather in infinitesimal that approaches 0), and with some added noise, it &quot;&lt;em&gt;ensures that all moves may be tried, but the search may still overrule bad moves&lt;/em&gt;.&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So altogether, the formula is saying : &lt;em&gt;Pick good things most of the time&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematical evaluation of the formula is described below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AlphaGo Zero defines the probability of each action (aka the policy) by that formula. If there are 3 nodes, A, B and C, and they have each been visited 10, 70, and 20 times (100 times in total), respectively, then the probability of taking those actions is: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;P(A) = 10/100 = .10&lt;/li&gt;&#xA;&lt;li&gt;P(B) = 70/100 = .7&lt;/li&gt;&#xA;&lt;li&gt;P(C) = 20/100 = .2 &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2020-05-12T15:02:12.470" LastActivityDate="2020-05-12T15:02:12.470" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21009" PostTypeId="1" CreationDate="2020-05-08T14:41:00.410" Score="3" ViewCount="63" Body="&lt;p&gt;I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to infinite. I mark this point in this picture:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4fHy2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4fHy2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book does not clearly mention the steps to simplify the above expression. I search on the web to find the solution but there is no clear explanation on that. Can anyone explain the marked point?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-10-06T01:05:00.463" Title="Simplification of expected reward under the limit in continuous tasks" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;statistical-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21010" PostTypeId="2" ParentId="20987" CreationDate="2020-05-08T17:23:33.287" Score="1" Body="&lt;p&gt;Yes, it is applied element-wise on every single value of the feature map.&#xA;Assuming ReLU as your nonlinearity function, all negative values of the image feature map are set to zero, and the rest of the elements stay unchanged.&lt;/p&gt;&#xA;" OwnerUserId="36785" LastActivityDate="2020-05-08T17:23:33.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21011" PostTypeId="1" CreationDate="2020-05-08T19:35:12.107" Score="1" ViewCount="31" Body="&lt;p&gt;I have hopefully a fundamental question of Do I understand things right.&#xA;(Thank you in advance and sorry for my English which might be not so good)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-Preambula 1:&#xA;I know that if we have 2 independent variables, both of a continuous type, it is ok to represent them as a 2d plane in a 3d space:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cFUgi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cFUgi.png&quot; alt=&quot;Figure1&amp;quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-Preambula 2:&#xA;I have seen that many times when we have to deal with continuous and categorical variables(male\female for example), we represent them like this(note the lines are parallel):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EyVJ5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EyVJ5.png&quot; alt=&quot;figure2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3-Assumption:&#xA;In the beginning I assumed that it is 2d representation of this 3d case:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Mqh8W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Mqh8W.png&quot; alt=&quot;figure3&quot;&gt;&lt;/a&gt;&#xA;4-Discussion 1: But If my assumption above was right, why do they always &quot;picture&quot;  it with parallel lines? After all this is a very specific situation. In most of cases both regression lines will not be parallel, further more, they may have different slope direction (one negative and another positive)For example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/c5f0X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c5f0X.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5-Discussion 2: On the other hand parallel models may be explained in such a way: if we will add a regression hyperplane which &quot;somehow&quot; fits both groups(male and female), we &lt;strong&gt;will&lt;/strong&gt; get the parallel lines:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7RdyQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7RdyQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6-Finally My questions are quite simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;question 5.1: Did I understand right the nature of &lt;strong&gt;parallel&lt;/strong&gt; lines as I show it above (in discussion2)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;question 5.2: If I was right in 5.1, I assume that in such cases hyperplane regression is a quite a bad predictor. Am I right?&lt;/p&gt;&#xA;" OwnerUserId="36453" LastActivityDate="2020-05-08T19:35:12.107" Title="3d representation of a regression with two independent variables one of them is categorical and another is continuous" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;regression&gt;&lt;linear-regression&gt;&lt;categorical-data&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21012" PostTypeId="1" CreationDate="2020-05-08T19:38:58.660" Score="4" ViewCount="123" Body="&lt;p&gt;Modern artificial neural networks use a lot more functions than just the classic sigmoid, to the point I'm having a hard time really seeing what classifies something as a &quot;neural network&quot; over other function approximators (such as Fourier series, Bernstein polynomials, Chebyshev polynomials or splines). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what makes something an artificial neural network? Is there a subset of theorems that apply only to neural networks? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backpropagation is classic, but that is the multi-variable chain rule, what else is unique to neural networks over other function approximators? &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-05-09T14:56:33.320" LastActivityDate="2020-06-15T23:05:10.423" Title="What are the differences between artificial neural networks and other function approximators?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21014" PostTypeId="2" ParentId="21002" CreationDate="2020-05-08T20:20:41.223" Score="2" Body="&lt;p&gt;I'm not sure if it is being used directly in the &lt;em&gt;industry&lt;/em&gt;, but &lt;a href=&quot;https://www.lancaster.ac.uk/news/uk-scientists-develop-new-rapid-smart-testing-device-for-coronavirus&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is an interesting article on research being done by 3 UK universities using AI. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-08T20:20:41.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21015" PostTypeId="2" ParentId="21002" CreationDate="2020-05-08T21:09:00.387" Score="5" Body="&lt;p&gt;A global race is underway to discover a vaccine, drug, or combination of treatments that can disrupt the SARS-CoV-2 virus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, there are more than a &lt;strong&gt;billion&lt;/strong&gt; such molecules. A researcher would conceivably want to test each one against the two dozen or so proteins in &lt;strong&gt;SARS-CoV-2&lt;/strong&gt; to see their effects. Such a project could use every wet lab in the world and still not be completed for centuries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer modelling is a common approach used by academic researchers and pharmaceutical companies as a preliminary, filtering step in drug discovery. However, in this case, even every supercomputer on Earth could not test those &lt;strong&gt;billion molecules&lt;/strong&gt; in a reasonable amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://foldingathome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Folding@home&lt;/strong&gt;&lt;/a&gt; is a distributed computing project run by Stanford University. The aim of the project is to examine how proteins fold and it does this using &lt;strong&gt;spare computing power&lt;/strong&gt;. however, there is a lot of research in progress that are harnessing the potential of &lt;strong&gt;Artificial intelligence&lt;/strong&gt; to develop the potential treatment to combat the &lt;strong&gt;COVID-19&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check this recent article By &lt;strong&gt;Tyler Orton&lt;/strong&gt; in &lt;strong&gt;biv&lt;/strong&gt; focus on how artificial intelligence is used to accelerate the process of Drug Discovery:&#xA;&lt;a href=&quot;https://biv.com/article/2020/05/drug-research-turns-artificial-intelligence-covid-19-fight&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Drug research turns to artificial intelligence in COVID-19 fight&lt;/strong&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the list some companies that are using &lt;strong&gt;AI-driven&lt;/strong&gt; approach for Drug discovery&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.blackthornrx.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BlackThorn Therapeutics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.exscientia.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;xscientia&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://insilico.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Insilico Medicine&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://insitro.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Insitro&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://notablelabs.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Notable Labs&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.standigm.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Standigm&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.recursionpharma.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recursion Pharmaceuticals&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Hong Kong-based company &lt;strong&gt;Insilico Medicine&lt;/strong&gt;, a developer of a comprehensive drug discovery and biomarker development platform &lt;strong&gt;GENTRL&lt;/strong&gt;, and a pioneer in the application of &lt;strong&gt;generative adversarial networks (GANs)&lt;/strong&gt; to drug discovery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insilico Medicine, Publishes a paper in September last year titled, &#xA;&lt;a href=&quot;https://www.nature.com/articles/s41587-019-0224-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Deep learning enables rapid identification of potent DDR1 kinase inhibitors&lt;/strong&gt;&lt;/a&gt;,&quot; in a most reputed journal &lt;strong&gt;Nature Biotechnology&lt;/strong&gt;. The paper describes a timed challenge, where the new artificial intelligence system called &lt;strong&gt;Generative Tensorial Reinforcement Learning (GENTRL)&lt;/strong&gt; designed &lt;strong&gt;six novel inhibitors of DDR1&lt;/strong&gt;, a &lt;strong&gt;kinase target implicated in fibrosis&lt;/strong&gt; &lt;strong&gt;and other diseases&lt;/strong&gt;, &lt;strong&gt;in 21 days&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Four compounds were active in biochemical assays, and two were validated in cell-based assays. One lead candidate was tested and demonstrated favourable pharmacokinetics in mice.&lt;/p&gt;&#xA;" OwnerUserId="36737" LastEditorUserId="36737" LastEditDate="2020-05-08T21:44:15.203" LastActivityDate="2020-05-08T21:44:15.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21016" PostTypeId="2" ParentId="20645" CreationDate="2020-05-08T22:19:56.180" Score="2" Body="&lt;p&gt;In the, presumably final, printed version the last two equal signs are approximations. This is just because over a large amount of weight updates where you have been sampling the expectation will be approximated by Monte Carlo.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-08T22:19:56.180" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21018" PostTypeId="1" AcceptedAnswerId="21025" CreationDate="2020-05-08T23:08:19.617" Score="4" ViewCount="81" Body="&lt;p&gt;From what I can find, reinforcement algorithms work on a grid or 2-dimensional environment.  How would I set up the problem for an approximate solution when I have a 1-dimensional signal from a light sensor.  The sensor sits some distance away from a lighthouse.  The intent would be to take the reading from the sensor to determine the orientation of the lighthouse beam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The environment would be a lighthouse beam, the state would be the brightness seen at the sensor for a given orientation, and the agent would be the approximate brightness/orientation?  What would the reward be?  What reinforcement learning algorithm would I use to approximate the lighthouse orientation given sensor brightnesses?&lt;/p&gt;&#xA;" OwnerUserId="36877" LastActivityDate="2020-05-09T14:57:19.623" Title="Is there 1-dimensional reinforcement learning?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21019" PostTypeId="1" CreationDate="2020-05-09T02:44:41.053" Score="2" ViewCount="85" Body="&lt;p&gt;I've implemented the Monte Carlo tree search (MCTS) algorithm for a connect four game I've built. The MCTS agent beats a random choice agent 90-100% of the time, but I’m still able to beat it pretty easily. It even misses obvious three in a row opportunities where it just needs to add one more token to win (but places it elsewhere instead). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this normal behavior, or should the MCTS agent be able to beat me consistently too? I'm allowing it to grow its tree for 2 seconds before getting it to return its chosen action - could it be that it needs longer to think?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-05-09T11:17:09.307" LastActivityDate="2020-05-09T11:17:09.307" Title="Should Monte Carlo tree search be able to consistently beat me in the connect four game?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="21020" PostTypeId="1" CreationDate="2020-05-09T04:44:23.200" Score="2" ViewCount="109" Body="&lt;p&gt;I am a beginner in machine learning and neural networks. I have only used neural networks for classification problems.  My aim is to modify it so that it can work for polynomial regression as well. In my problem, I have three inputs and three outputs. My aim is to predict these three outputs based on these three inputs. The outputs are real-valued, and can take positive and negative values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I choose the activation functions? I have only used sigmoid. &lt;/p&gt;&#xA;" OwnerUserId="36881" LastEditorUserId="2444" LastEditDate="2020-05-09T11:19:07.470" LastActivityDate="2020-05-09T11:19:07.470" Title="Which activation functions should I use for polynomial regression?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;ai-design&gt;&lt;activation-function&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21021" PostTypeId="1" AcceptedAnswerId="21026" CreationDate="2020-05-09T05:05:10.990" Score="1" ViewCount="480" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sutton barton's reinforcement learning textbook&lt;/a&gt; and have come across the finite Markov decision process (MDP) example of the blackjack game (Example 5.1). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't the environment constantly changing in this game? How would the transition probabilities be fixed in such an environment, when both you and the dealer draw cards?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-09T11:21:11.697" LastActivityDate="2020-05-09T11:37:28.120" Title="How can blackjack be formulated as a Markov decision process?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21022" PostTypeId="2" ParentId="21019" CreationDate="2020-05-09T09:13:28.107" Score="1" Body="&lt;p&gt;You should not let the tree grow for only two seconds rather you should use the simulation number equal to 1000 or something like that. I use the simulation number equal to 10000 for making a single move in the tictactoe game and it was working fine for me. Also, after the agent has chosen the move you do not have to start the statistics(N = visit count, V = expected reward, U = UCT score) from the beginning, you can use the current statistics and replace the root node with the chosen node. &lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-05-09T09:42:40.907" LastActivityDate="2020-05-09T09:42:40.907" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21025" PostTypeId="2" ParentId="21018" CreationDate="2020-05-09T09:56:40.790" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;From what I can find, reinforcement algorithms work on a grid or 2-dimensional environment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A lot of teaching materials use a &quot;grid world&quot; presentation to demonstrate basic reinforcement learning (RL). However, the underlying Markov Decision Process (MDP) theory works on an arbitrary &lt;em&gt;graph&lt;/em&gt; of connected states. This graph could be based on subdividing a metric space of any dimensions into a grid of the same dimensions (and using tiles of any shape that worked in that dimension). However, it is not limited to that, the state space does not need to be a metric that represents distances or physical properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the set of states can be arbitrary objects, connected via state transitions in any consistent way. Provided the transition probability function &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; is consistent, the environment could be used in a RL problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very common state description is that the state is a vector of numbers that capture all the variables relevant to the problem. The environment can then be measurements taken in the real world of those variables, or the same quantities provided by a simulation. That state vector can be of any size, and have arbitrary constraints on individual components. This is no different from numerical representations of other machine learning problems, such as the inputs allowed to a neural network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The environment would be a lighthouse beam, the state would be the brightness seen at the sensor for a given orientation, and the agent would be the approximate brightness/orientation? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Something not quite right about the description there. There does not seem to be any action that the agent takes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What would the reward be? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It would be whatever measure of reaching a goal or maintaining a &quot;good&quot; result that is appropriate for the problem. You do not give any information about the goal in your description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your goal is to light up a moving sensor with the highest brightness, then the brightness measured at the sensor, transformed into suitable units, would seem to be a good candidate for a reward function (you would also need the state to give information about the target - where it had been seen last for instance). Assuming the problem is continuous, you would also need a discount factor.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What reinforcement learning algorithm would I use to approximate the lighthouse orientation given sensor brightnesses?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generally RL algorithms estimate rewards, or generate policies. If the lighthouse orientation is the action you wanted to take, then pretty much all RL algorithms can do one or the other to allow you to do this. The differences are in things like complexity or speed of the algorithm, what approximations you are willing to take etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't give enough information about the problem to even nearly suggest a &quot;best&quot; algorithm. Before you start, you will need to determine a more thorough description of state, action and rewards, that will define the problem. Once you have a more formal description of the problem, that may suggest which algorithms would be good starting points.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-09T14:57:19.623" LastActivityDate="2020-05-09T14:57:19.623" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21026" PostTypeId="2" ParentId="21021" CreationDate="2020-05-09T11:37:28.120" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Isn't the environment constantly changing in this game?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The current state of the agent and the environment is constantly changing as you play, but not necessarily the transition probabilities. For simplicity, you may assume that the transition probabilities do not change (e.g. if the dealer and the deck are the same every time you play).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How would the transition probabilities be fixed in such an environment, when both you and the dealer draw cards?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The actions of the dealer will be incorporated into the transition probabilities of the environment. Whenever the player (or RL agent) takes an action, then it will receive a reward, according to the reward function of the environment (the rules of the game), and the agent and the environment will be moved to the next state, according to the transition probabilities, which do not have to change for stochasticity to occur in the environment. In fact, these transition probabilities already incorporate this stochasticity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if even the environment was changing, you could still model your problem as an MDP, but the MDP would change accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf#page=129&quot; rel=&quot;nofollow noreferrer&quot;&gt;example 5.1 of the book&lt;/a&gt; (that you mention) actually explains in detail how to formulate this game as a finite MDP.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-09T11:37:28.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21028" PostTypeId="2" ParentId="21012" CreationDate="2020-05-09T14:48:29.760" Score="2" Body="&lt;p&gt;First of all, neural networks are not (just) defined by the fact that they are typically trained with gradient descent and back-propagation. In fact, there are other ways of training neural networks, such as evolutionary algorithms and the Hebb's rule (e.g. Hopfield networks are typically associated with this Hebbian learning rule). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first difference between neural networks and other function approximators is &lt;strong&gt;conceptual&lt;/strong&gt;. In neural networks, you typically imagine that there are one or more computational units (often called &lt;em&gt;neurons&lt;/em&gt;) that are connected in different and often complex ways. The human can choose these connections (or they could also be learned) and the functions that these units compute given the inputs. So, there's a great deal of &lt;strong&gt;flexibility&lt;/strong&gt; and complexity, but, often, also a lack of rigorousness (from the mathematical point of view) while using and designing neuron networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other difference is that neural networks were originally inspired by the biological counterparts. See &lt;a href=&quot;https://link.springer.com/article/10.1007/BF02478259&quot; rel=&quot;nofollow noreferrer&quot;&gt;A logical calculus of the ideas immanent in nervous activity&lt;/a&gt; (1943) by Warren McCulloch and Walter Pitts, who proposed, inspired by neuroscience, the first mathematical model of an artificial neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other technical differences. For example, the &lt;a href=&quot;https://mathworld.wolfram.com/TaylorSeries.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taylor expansion of a function&lt;/a&gt; is &lt;em&gt;typically&lt;/em&gt; done only at a single value of the domain, it assumes that the function to be approximated is differentiable multiple times, and it makes uses of the derivatives of such a function. Fourier series typically approximate functions with a weighted sum of sinusoids. Given appropriate weights, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier series&lt;/a&gt; can be used to approximate an arbitrary function in a certain interval or the entire function (if the function you want to approximate is also periodic). On the other hand, neural networks attempt to approximate functions of the form &lt;span class=&quot;math-container&quot;&gt;$f: [0, 1]^n \rightarrow \mathbb{R}$&lt;/span&gt; (at least, this is the setup in &lt;a href=&quot;https://link.springer.com/article/10.1007/BF02551274&quot; rel=&quot;nofollow noreferrer&quot;&gt;the famous paper that proved the universality of neural networks&lt;/a&gt;) in many different ways (for example, weighted sums followed by sigmoids).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, neural networks are quite different from other function approximation techniques (such as Taylor or Fourier series) in the way they approximate functions and their purpose (i.e. which functions they were supposed to approximate and in which context).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-16T22:56:13.537" LastActivityDate="2020-05-16T22:56:13.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21030" PostTypeId="1" CreationDate="2020-05-09T15:06:00.653" Score="1" ViewCount="28" Body="&lt;p&gt;I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has a high dimension (about 16000), and I would like to classify these vectors using an RBF-ANN (there are only 2 possible classes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the high dimensionality of input vectors a problem for a radial basis function ANN? If yes, is there any way to deal with it?&lt;/p&gt;&#xA;" OwnerUserId="36363" LastEditorUserId="2444" LastEditDate="2020-05-09T22:26:21.737" LastActivityDate="2020-05-09T22:26:21.737" Title="Is the high dimensionality of input vectors a problem for a radial basis function neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21032" PostTypeId="1" CreationDate="2020-05-09T18:03:51.583" Score="2" ViewCount="68" Body="&lt;p&gt;Why can't DQN be used for self-driving cars?&#xA;Why can't DQN and similar RL algorithms be used for self-driving cars?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why I am curious is that it successfully plays go and other multistate games.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-11-23T13:30:27.490" LastActivityDate="2020-11-23T13:30:27.490" Title="Why can't DQN be used for self-driving cars?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;applications&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21033" PostTypeId="1" CreationDate="2020-05-09T19:49:04.690" Score="1" ViewCount="36" Body="&lt;p&gt;I am reading a paper implementing a deep deterministic policy gradient algorithm for portfolio management. My question is about a specific neural network implementation they depict in this picture (&lt;a href=&quot;https://arxiv.org/pdf/1706.10059v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, picture is on page 14).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nIX2S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nIX2S.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first three steps are convolutions. Once they have reduced the initial tensor into a vector, they add that little yellow square entry to the vector, called the cash bias, and then they do a softmax operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper does not go into any detail about what this bias term could be, they just say that they add this bias before the softmax. This makes me think that perhaps this is a standard step? But I don't know if this is a learnable parameter, or just a scalar constant they concatenate to the vector prior to the softmax.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) When they write softmax, is it safe to assume that this is just a softmax function, with no learnable parameters? Or is this meant to depict a fully connected linear layer, with a softmax activation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If it's the latter, then I can interpret the cash bias as being a constant term they concatenate to the vector before the fully connected layer, just to add one more feature for the cash assets. However, if softmax means just a function, then what is this cash bias? It must be a constant that they implement, but I don't see what the use of that would be, how can you pick a constant scalar that you are confident will have the intended impact on the softmax output to bias the network to put some weight on that feature (cash)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any comments/interpretations are appreciated!&lt;/p&gt;&#xA;" OwnerUserId="35809" LastActivityDate="2020-05-09T19:49:04.690" Title="What do the authors of this paper mean by the bias term in this picture of a neural network implementation?" Tags="&lt;convolutional-neural-networks&gt;&lt;papers&gt;&lt;algorithmic-bias&gt;&lt;softmax&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21034" PostTypeId="2" ParentId="18341" CreationDate="2020-05-09T20:04:23.237" Score="1" Body="&lt;p&gt;Probably to as many as possible. Average accept rate of papers is around 20%. You can find the best conferences on &lt;a href=&quot;https://aiml.events&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI &amp;amp; ML Event&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36893" LastActivityDate="2020-05-09T20:04:23.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21035" PostTypeId="1" CreationDate="2020-05-09T20:36:29.633" Score="1" ViewCount="27" Body="&lt;p&gt;Chess and starcraft games have been played by AI and sometimes won. Artificial intelligence had for these games a complex set of rules, database and decision-making tools to decide which actions it will do in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is said that the strategies applied by bot players are analyzed and compared with human activity at chess, especially chess experts such as Magnus Carlsen:&#xA;Jean-Marc Alliot, &lt;em&gt;When artificial intelligence evaluates chess champions&lt;/em&gt;, ScienceDaily, April 25th 2017.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But a chess bot is only giving a set of actions for a given game situation. So my question: How are AI's strategies identified from the &quot;optimized&quot; set of actions AI propose?&lt;/p&gt;&#xA;" OwnerUserId="26866" LastActivityDate="2020-05-09T20:36:29.633" Title="How are AI's strategies discovered from chess and starcraft games?" Tags="&lt;game-ai&gt;&lt;chess&gt;&lt;action-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21039" PostTypeId="1" CreationDate="2020-05-10T04:08:07.440" Score="1" ViewCount="69" Body="&lt;p&gt;For reference, see: &lt;a href=&quot;https://cs.stackexchange.com/questions/125485/jaccard-decay-jaccard-recall-and-class-agnostic-binary-segmentation&quot;&gt;Jaccard decay, Jaccard recall, and class-agnostic binary segmentation (SE:CS)&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know the meaning of the Jaccard index and the formula associated to it, but when it comes to Jaccard decay and Jaccard recall I am a bit confused.&lt;/p&gt;&#xA;&lt;p&gt;I see both of them used for evaluation in segmentation tasks (i.e.Video Object Segmentation on DAVIS ). Can someone help me understand the difference between the Jaccard index and those two metrics or point me to a link so I can read about it ?&lt;/p&gt;&#xA;" OwnerUserId="36901" LastEditorUserId="1671" LastEditDate="2020-10-05T00:26:05.597" LastActivityDate="2020-10-05T00:26:05.597" Title="What's the meaning of the Jaccard decay and the Jaccard recall?" Tags="&lt;machine-learning&gt;&lt;definitions&gt;&lt;object-detection&gt;&lt;metric&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2020-05-10T11:04:10.907" ContentLicense="CC BY-SA 4.0" />
  <row Id="21040" PostTypeId="1" CreationDate="2020-05-10T05:53:19.650" Score="1" ViewCount="24" Body="&lt;p&gt;I am trying to train a card game called Callbreak I tried inputs like all the opponents discarded cards all hands everything a human can see and calculate with &quot;common sense&quot; I fed it to the Agent but it's not learning the way it should after 100M steps it learned as basic as any new/bad player will do in the game.&#xA;I have never tried to train a card game before so Idk what should I feed and how much to feed any help will be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have trained 100m+ steps before it will just keep repeating these drops and will never get more rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;config I am using is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;default:&#xA;trainer: ppo&#xA;batch_size: 1024&#xA;beta: 5.0e-3&#xA;buffer_size: 10240&#xA;epsilon: 0.2&#xA;hidden_units: 256&#xA;lambd: 0.95&#xA;learning_rate: 3.0e-4&#xA;learning_rate_schedule: linear&#xA;max_steps: 100.0e5&#xA;memory_size: 128&#xA;normalize: false&#xA;num_epoch: 3&#xA;num_layers: 3&#xA;time_horizon: 64&#xA;sequence_length: 64&#xA;summary_freq: 100000&#xA;use_recurrent: false&#xA;vis_encode_type: simple&#xA;reward_signals:&#xA;    extrinsic:&#xA;        strength: 1.0&#xA;        gamma: 0.99&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Tensorboard&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6SIkb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6SIkb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;New Training Results&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;default:&#xA;trainer: ppo&#xA;batch_size: 1024&#xA;beta: 5.0e-3&#xA;buffer_size: 10240&#xA;epsilon: 0.2&#xA;hidden_units: 512&#xA;lambd: 0.95&#xA;learning_rate: 3.0e-4&#xA;learning_rate_schedule: linear&#xA;max_steps: 500.0e5&#xA;memory_size: 128&#xA;normalize: false&#xA;num_epoch: 3&#xA;num_layers: 3&#xA;time_horizon: 64&#xA;sequence_length: 64&#xA;summary_freq: 100000&#xA;use_recurrent: false&#xA;vis_encode_type: simple&#xA;reward_signals:&#xA;    extrinsic:&#xA;        strength: 1.0&#xA;        gamma: 0.99&#xA;    curiosity:&#xA;        strength: 0.02&#xA;        gamma: 0.99&#xA;        encoding_size: 256&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RCnbf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RCnbf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JTBE8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JTBE8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oCnOW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oCnOW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36902" LastEditorUserId="36902" LastEditDate="2020-05-11T04:21:07.457" LastActivityDate="2020-05-11T04:21:07.457" Title="Trying to Train Cards Game with RL" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21042" PostTypeId="1" AcceptedAnswerId="21048" CreationDate="2020-05-10T10:32:04.957" Score="2" ViewCount="624" Body="&lt;p&gt;I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have points &lt;span class=&quot;math-container&quot;&gt;$p_1, p_2, p_3, p_4$&lt;/span&gt; in the first image and &lt;span class=&quot;math-container&quot;&gt;$p'_1, p'_2, p'_3, p'_4$&lt;/span&gt; in the second. How am I going to generate the homography matix given these points.&lt;/p&gt;&#xA;" OwnerUserId="36484" LastEditorUserId="2444" LastEditDate="2020-05-10T11:05:51.500" LastActivityDate="2020-05-10T22:17:25.727" Title="How do you find the homography matrix given 4 points in both images?" Tags="&lt;computer-vision&gt;&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21043" PostTypeId="2" ParentId="21000" CreationDate="2020-05-10T10:32:22.910" Score="1" Body="&lt;p&gt;You could possibly apply neural networks, reinforcement learning to summarise results of previous choices (what you are calling &lt;em&gt;context&lt;/em&gt;) and use score predictions to suggest the next turn's guess. However, the game of Mastermind has a small search space and it is possible to process this &quot;context&quot; more directly by refining a set of guesses. This will be much more efficient and simpler to understand than a neural network approach. It would be very hard to make a neural network variant which was as efficient - either in terms of CPU time, or in terms of number of turns it takes to find a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, a Mastermind solver is much like a Hangman solver, or a Guess Who? solver. You have an initial large set of all possible answers, and need to narrow it down to a single correct answer. You do this by processing after each guess to reduce the set of answers that meet all the constraints that the game has given you so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The agent needs to know the score function that compares a target value with the guess and returns the score. Let's call that &lt;code&gt;score(guess, target)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(Opponent sets unknown_target)&#xA;Initialise possible_answers as list of all valid targets&#xA;&#xA;For each turn:&#xA;  Select one of possible_answers as this turn's guess&#xA;  Ask opponent for gscore = score(guess, unknown_target)&#xA;  If gscore is [0,4] then exit(win)&#xA;  If it was last guess then exit(lose)&#xA;  For each possible_answer in possible_answers:&#xA;    pscore = score(guess, possible_answer)&#xA;    If pscore != gscore, then remove possible_answer from possible_answers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can finesse this for the stage &lt;code&gt;Select one of possible_answers as this turns guess&lt;/code&gt; by trying to optimise either by psychological model of opponent or trying to find choices that are likely to cause the best reduction in size of &lt;code&gt;possible_answers&lt;/code&gt;. However, a simple random choice should do quite well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also worth noting is that the algorithm does not depend on the exact nature of the scoring function, so it is applicable for many variations of guessing games. It does rely on the score providing information that will reduce the remaining set of guesses. In some games that may mean taking more care about the precise nature of a guess, in order to maximise this effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of interest, I implemented this algorithm and tested it when there were 10 choices at each position (i.e. digits 0 to 9), maximum of 10 guesses allowed, and the target to guess was set randomly. Using random guesses and the algorithm exactly as written, the above approach guessed correctly 9,996 times out of 10,000, and on average the guesser won the game in 6.2 turns.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-10T12:40:08.560" LastActivityDate="2020-05-10T12:40:08.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21044" PostTypeId="1" CreationDate="2020-05-10T10:52:49.730" Score="10" ViewCount="555" Body="&lt;p&gt;I'm currently studying reinforcement learning and I'm having difficulties with question 6.12 in Sutton and Barto's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose action selection is greedy. Is Q-learning then exactly the same algorithm as SARSA? Will they make exactly the same action selections and weight updates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's true, because the main difference between the two is when the agent explores, and following the greedy policy it never explores, but I am not sure.&lt;/p&gt;&#xA;" OwnerUserId="36105" LastEditorUserId="2444" LastEditDate="2020-12-04T18:40:17.827" LastActivityDate="2020-12-06T12:26:10.680" Title="Are Q-learning and SARSA the same when action selection is greedy?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;sarsa&gt;&lt;greedy-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21045" PostTypeId="1" CreationDate="2020-05-10T12:01:15.010" Score="4" ViewCount="79" Body="&lt;p&gt;I read so many articles and the Fast R-CNN paper, but I'm still confused about how the region proposal method works in Fast R-CNN.&lt;/p&gt;&#xA;&lt;p&gt;As you can see in the image below, they say they used a &lt;em&gt;proposal method&lt;/em&gt;, but it is not specified how it works.&lt;/p&gt;&#xA;&lt;p&gt;What confuses me is, for example, in the VGGnet, the output of the last convolution layer is feature maps of shape 14x14x512, but what is the used algorithm to propose the regions and how does it propose them from the feature maps?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AQUnI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AQUnI.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36913" LastEditorUserId="2444" LastEditDate="2021-02-03T10:22:58.317" LastActivityDate="2021-02-04T10:46:56.180" Title="How does the region proposal method work in Fast R-CNN?" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;r-cnn&gt;&lt;selective-search&gt;&lt;faster-r-cnn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21046" PostTypeId="1" CreationDate="2020-05-10T13:01:32.923" Score="2" ViewCount="87" Body="&lt;p&gt;I am starting to get my head around convolutional neural networks, and I have been working with the CIFAR-10 dataset and some research papers that used it. In one of these papers, they mention a network architecture notation for a CNN, and I am not sure how to interpret that exactly in terms of how many layers are there and how many neurons in each.&lt;/p&gt;&#xA;&lt;p&gt;This is an image of their structure notation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wJZ8Q.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wJZ8Q.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Can some give me an explanation as to what exactly this structure looks like?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the CIFAR-10 dataset, each image is &lt;span class=&quot;math-container&quot;&gt;$32 \times 32$&lt;/span&gt; pixels, represented by 3072 integers indicating the red, green, blue values for each pixel.&lt;/p&gt;&#xA;&lt;p&gt;Does that not mean that my input layer has to be of size 3072? Or is there some way to group the inputs into matrices and then feed them into the network?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32477" LastEditorUserId="2444" LastEditDate="2020-06-22T16:37:13.940" LastActivityDate="2021-03-19T18:05:45.557" Title="Can you explain me this CNN architecture?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21048" PostTypeId="2" ParentId="21042" CreationDate="2020-05-10T13:30:50.760" Score="4" Body="&lt;p&gt;To understand homographies and how to find them, you will need a good dose of projective geometry. I will briefly describe some preliminary concepts that you need to know before trying to find the homography, but don't expect to understand all these concepts with one reading iteration and only by reading this answer, if you are not familiar with them, especially, if you don't even know what homogenous coordinates are. For more details, I suggest you read the book &lt;a href=&quot;http://cvrs.whu.edu.cn/downloads/ebooks/Multiple%20View%20Geometry%20in%20Computer%20Vision%20(Second%20Edition).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiple view geometry in computer vision&lt;/a&gt; (2004) by Richard Hartley and Andrew Zisserman, in particular, chapter 4.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The projective space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Projective_space&quot; rel=&quot;nofollow noreferrer&quot;&gt;projective space&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt;, so it is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; augmented with lines and &lt;a href=&quot;https://en.wikipedia.org/wiki/Point_at_infinity&quot; rel=&quot;nofollow noreferrer&quot;&gt;points at infinity&lt;/a&gt;. All &lt;strong&gt;points&lt;/strong&gt; and &lt;strong&gt;lines&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; actually belong to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt;, i.e. they are vectors of three components, because they are &lt;a href=&quot;https://en.wikipedia.org/wiki/Homogeneous_coordinates&quot; rel=&quot;nofollow noreferrer&quot;&gt;homogenous representations&lt;/a&gt; of the counterparts in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt;. To emphasize, in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt;, both points &lt;strong&gt;and&lt;/strong&gt; lines can be represented by a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt;, which is the homogenous representation of the counterpart vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; (if it exists, e.g. points at infinity do not exist in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What is a homography?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Homography&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;homography&lt;/strong&gt;&lt;/a&gt; (aka projectivity, collineation or projective transformation) is an invertible map &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; from the projective space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; to itself, &lt;span class=&quot;math-container&quot;&gt;$$h: \mathbb{P}^2 \rightarrow \mathbb{P}^2,$$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3$&lt;/span&gt; lie on the same &lt;strong&gt;straight-line&lt;/strong&gt; if and only if &lt;span class=&quot;math-container&quot;&gt;$h(\mathbf{x}_1), h(\mathbf{x}_2), h(\mathbf{x}_3)$&lt;/span&gt; do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This property is called &lt;strong&gt;collinearity&lt;/strong&gt;. In practice, this property means that projective transformations can map straight-lines in one image to straight-line to another image, but it cannot map e.g. straight-lines to parabolas (and vice-versa). So, if you have a distorted image, a homography cannot convert it to a non-distorted image. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;All and only linear maps in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; are homographies&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;All linear maps in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; are homographies and only linear maps can be homographies. Hence, when you want to find a homography, you are looking for a linear map in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;A homography can be represented as &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; matrix&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Given that homographies are linear maps they can be represented as an invertible matrix &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{H} &#xA;= &#xA;\begin{bmatrix} &#xA;h_{11} &amp;amp; h_{12} &amp;amp; h_{13} \\ &#xA;h_{21} &amp;amp; h_{22} &amp;amp; h_{23} \\ &#xA;h_{31} &amp;amp; h_{32} &amp;amp; h_{33} &#xA;\end{bmatrix} &#xA;\in \mathbb{R}^{3 \times 3},&#xA;$$&lt;/span&gt;&#xA;such that, &lt;span class=&quot;math-container&quot;&gt;$\forall \mathbf{x} \in \mathbb{P}^2$&lt;/span&gt;, the following equation holds&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;h(\mathbf{x}) = \mathbf{H}\mathbf{x}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Homographies map points in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; to points in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt;&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Given that homographies map points in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; to other points in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}^2$&lt;/span&gt; and there's a matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H} \in \mathbb{R}^{3 \times 3}$&lt;/span&gt; for each homography &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, then it follows that, &lt;span class=&quot;math-container&quot;&gt;$\forall \mathbf{x} \in \mathbb{P}^2$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{x}' &#xA;&amp;amp;= &#xA;\mathbf{H}\mathbf{x} \\&#xA;\begin{bmatrix} &#xA;\mathbf{x}'_1 \\ \mathbf{x}'_2 \\ \mathbf{x}'_3&#xA;\end{bmatrix}&#xA;&amp;amp;=&#xA;\begin{bmatrix} &#xA;h_{11} &amp;amp; h_{12} &amp;amp; h_{13} \\ &#xA;h_{21} &amp;amp; h_{22} &amp;amp; h_{23} \\ &#xA;h_{31} &amp;amp; h_{32} &amp;amp; h_{33} &#xA;\end{bmatrix} &#xA;\begin{bmatrix} &#xA;\mathbf{x}_1 \\ \mathbf{x}_2 \\ \mathbf{x}_3&#xA;\end{bmatrix}&#xA;\tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}' \in \mathbb{P}^2$&lt;/span&gt;, for some homography &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H} \in \mathbb{R}^{3 \times 3}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, this means that you can transform points in an image &lt;span class=&quot;math-container&quot;&gt;$I_1$&lt;/span&gt; to points in image &lt;span class=&quot;math-container&quot;&gt;$I_2$&lt;/span&gt; by matrix multiplication. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;A homography has &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; degrees of freedom&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Although the matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H}$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; entries, it has only &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; degrees of freedom, which means that, in practice, there are only &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; variables that you need to find. This property comes from the fact that only the ratio between the elements of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H}$&lt;/span&gt; actually counts. This means that equation \ref{1} can actually be written as &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{x}' = \lambda \mathbf{H}\mathbf{x} \tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&#xA;for all &lt;span class=&quot;math-container&quot;&gt;$\lambda \in \mathbb{R} \setminus \{0 \}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How to estimate a homography from point correspondences?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;We want to estimate a homography &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H} \in \mathbb{R}^{3 \times 3}$&lt;/span&gt; from point-to-point correspondences &lt;span class=&quot;math-container&quot;&gt;$\{(\mathbf{x}^i, \mathbf{x}'^i) \}_{i=1}^N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N \geq 4$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}'^i = \mathbf{H}\mathbf{x}^i, \forall i$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, recall that the matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H}$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; degrees of freedom (i.e. variables we want to find). &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Each point-to-point correspondence represents &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; constraints&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Each point-to-point correspondence &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}^i, \mathbf{x}'^i)$&lt;/span&gt; accounts for &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; constraints, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H}\mathbf{x}^i$&lt;/span&gt; maps to the point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}'^i$&lt;/span&gt;, which has &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; degrees of freedom (even if it's defined by 3 components) because it's represented in homogenous coordinates and so, as for equation \ref{2}, it's defined &quot;up to scale&quot;. In other words, these &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; degrees of freedom represent &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; constraints. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;&lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; point-to-point correspondences are necessary to estimate a homography&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; point-to-point correspondence represents &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; constraints, then &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; point-to-point correspondences corresponds to &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; constraints. Given this and given that homographies have &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; degrees of freedom, at least &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; point-to-point correspondences are necessary to estimate a homography.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation \ref{2} can actually be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{x}' \times \lambda \mathbf{H}\mathbf{x} = \mathbf{0} \label{3} \tag{3}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\times$&lt;/span&gt; is the cross-product.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Direct linear transformation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you manipulate equation \ref{3}, then you will end up with another equation (if you ignore the scaling factor &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;, which we can ignore because the homography converts points independently of their magnitude)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{A}_i \mathbf{h} = \mathbf{0} &#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}_i \in \mathbb{R}^{2 \times 9}$&lt;/span&gt; is the &lt;em&gt;design matrix&lt;/em&gt; (i.e. the matrix that contains the input data to estimate the homography) that contains the elements of the the point-to-point correspondence &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x}^i, \mathbf{x}'^i)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{h} \in \mathbb{R}^{9}$&lt;/span&gt; is a vector that contains the &lt;strong&gt;unknown&lt;/strong&gt; elements of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H}$&lt;/span&gt;. The details of this manipulation can be found in section 4.1 of the book &lt;a href=&quot;http://cvrs.whu.edu.cn/downloads/ebooks/Multiple%20View%20Geometry%20in%20Computer%20Vision%20(Second%20Edition).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiple view geometry in computer vision&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the idea is that you can vertically stack &lt;span class=&quot;math-container&quot;&gt;$N \geq 4$&lt;/span&gt; equations of the form &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}_i \mathbf{h} = \mathbf{0}$&lt;/span&gt; to build the final linear system that you would like to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{A} \mathbf{h} = \mathbf{0} \tag{4}\label{4}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A} \in \mathbb{R}^{2N \times 9}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to this system is the vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{h} \in \mathbb{R}^{9}$&lt;/span&gt;, that is, your homography!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know something about linear algebra, you know that the solutions to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A} \mathbf{h} = \mathbf{0}$&lt;/span&gt; are elements of the null space of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, to find &lt;span class=&quot;math-container&quot;&gt;$\mathbf{h}$&lt;/span&gt;, you will typically use &lt;em&gt;singular value decomposition&lt;/em&gt; (SVD). See e.g. &lt;a href=&quot;https://math.stackexchange.com/q/1771013/168764&quot;&gt;How is the null space related to singular value decomposition?&lt;/a&gt; for a possible explanation of why you can use SVD to find an element of the null space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm (which is described on page 91 of the cited book, i.e. Algorithm 4.1) is called &lt;strong&gt;direct linear transformation&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-10T22:17:25.727" LastActivityDate="2020-05-10T22:17:25.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21049" PostTypeId="1" CreationDate="2020-05-10T13:44:52.740" Score="2" ViewCount="423" Body="&lt;p&gt;I have this problem.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A basic wooden railway set contains the pieces shown in Figure 3.32.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ObU7H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ObU7H.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The task is to connect these pieces into a railway that has no overlapping tracks and no loose ends where a train could run off onto the floor.&lt;/p&gt;&#xA;&lt;p&gt;a. Suppose that the pieces fit together exactly with no slack. Give a precise formulation of the task as a search problem.&lt;/p&gt;&#xA;&lt;p&gt;b. Identify a suitable uninformed search algorithm for this task and explain your choice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I know I have to use a DFS for this problem. But how do I know all the pieces are connected. The last one and the first one are connected.&lt;/p&gt;&#xA;&lt;p&gt;Can someone help me with some tips on how to solve this problem and implement it (in Python)?&lt;/p&gt;&#xA;" OwnerUserId="36886" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-10T14:02:27.803" Title="Wooden railway search problem AI" Tags="&lt;ai-design&gt;&lt;python&gt;&lt;search&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21050" PostTypeId="1" CreationDate="2020-05-10T14:27:15.683" Score="1" ViewCount="62" Body="&lt;p&gt;I am currently learning policy gradient methods from the &lt;a href=&quot;https://www.youtube.com/watch?v=S_gwYj1Q-44&amp;amp;t=1241s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep RL boot camp by Pieter Abbeel&lt;/a&gt; in which he explains the actor-critic algorithm derivation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At around minute 39, he explains that the sum of the rewards from time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; onwards is actually an estimation of &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s,u)$&lt;/span&gt;. I understand the definition of &lt;span class=&quot;math-container&quot;&gt;$Q^\pi(s,u)$&lt;/span&gt; but I'm not sure why this is the case here. Is the reward following after time step &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; collected based on current policy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8DOfH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8DOfH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-10T18:41:00.893" LastActivityDate="2020-05-10T18:41:00.893" Title="Is the reward following after time step $t+1$ collected based on current policy?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21051" PostTypeId="1" CreationDate="2020-05-10T15:39:41.167" Score="3" ViewCount="159" Body="&lt;p&gt;I have seen this happening in implementations of state-of-the-art RL algorithms where the model converges to a single action over time after multiple training iterations. Are there some general loopholes or reasons why this kind of behavior is exhibited? &lt;/p&gt;&#xA;" OwnerUserId="36749" LastEditorUserId="2444" LastEditDate="2020-05-10T15:48:10.720" LastActivityDate="2020-05-10T18:34:18.537" Title="Why do RL implementations converge on one action?" Tags="&lt;reinforcement-learning&gt;&lt;convergence&gt;&lt;policies&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21052" PostTypeId="2" ParentId="17047" CreationDate="2020-05-10T15:53:43.380" Score="0" Body="&lt;p&gt;for regression, you can use a hidden layer with sigmoid, then a LINEAR output layer, where the weighted sum goes straight through, without modification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this way your output is not restricted to 0-1&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-05-10T15:53:43.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21053" PostTypeId="1" CreationDate="2020-05-10T16:01:31.993" Score="2" ViewCount="224" Body="&lt;p&gt;Why is it hard to prove the convergence of the DQN algorithm? We know that the tabular Q-learning algorithm converges to the optimal Q-values, and with a linear approximator convergence is proved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference of DQN compared to Q-Learning with linear approximator is using DNN, the experience replay memory, and the target network. Which of these components causes the issue and why?&lt;/p&gt;&#xA;" OwnerUserId="16912" LastEditorUserId="2444" LastEditDate="2021-03-23T09:08:53.140" LastActivityDate="2021-03-24T11:02:52.613" Title="Why is it hard to prove the convergence of the deep Q-learning algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;dqn&gt;&lt;convergence&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21054" PostTypeId="2" ParentId="21000" CreationDate="2020-05-10T16:02:21.767" Score="0" Body="&lt;p&gt;although I have seen RL solutions to this problem, (those I saw) fail to realize that the state of mastermind is not observable, as there is the &quot;secret&quot; we're trying to guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mastermind is best approached as a constraint satisfaction problem, along the lines described by Neil Slater. The whole trick is to realize that you can eliminate options from the &quot;current possible alternatives&quot; set by treating the latest guess as the target and eliminate any combinations that don't agree with it e.g. (using 3 digits for clarity)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;last guess=123, scores [2,0] i.e. 2 white, 0 black&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the current alternatives are eliminated if they don't score [2,0] against the &lt;strong&gt;last guess&lt;/strong&gt; 123:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;124&lt;/strike&gt; [0,2]&#xA;214 [2,0]&#xA;215 [2,0]&#xA;&lt;strike&gt;321&lt;/strike&gt; [2,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;let's say the secret is 215, you see that our method of elimination is correct, albeit we don't know the secret!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen lots of different approaches (genetic engineering, information theory etc), but the plain truth is that a 50-line matlab piece of code with random guessing will give a winning stragegy that averages 4.3 guesses for the standard mastermind game (1296 alternatives)&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-05-10T16:02:21.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21055" PostTypeId="1" CreationDate="2020-05-10T16:12:58.703" Score="2" ViewCount="28" Body="&lt;p&gt;The reservoir of the Liquid State Machine is an array of random numbers connected to each other with a probability depending on the distance between each other. Because of this connection with each other it apparently has &quot;recurrence&quot;. The reservoir is followed by a readout stage where the actual weight training is done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hidden layer of a FF-NN can be an array of random weights, which is exactly what an Extreme Learning Machine is. ELM has a closed form solution of the second-stage weight (Beta) calculation i.e. only the Beta needs to be trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in both cases you have a second-stage layer or readout layer where weights are trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, if the reservoir random weights are very much like the ELM random weights and both don't need to be trained, how are they any different than each other? In other words, both have a set of untrained random weights, so in LSM where is the recurrence exactly happening if the weights are just random? Can't the LSM be reduced to a FF-NN?&lt;/p&gt;&#xA;" OwnerUserId="36919" LastActivityDate="2020-05-10T16:12:58.703" Title="Reservoir of LSM vs. FF-NN or ELM" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21056" PostTypeId="2" ParentId="21051" CreationDate="2020-05-10T16:33:58.780" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do RL implementations converge on one action?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the optimal policy shouldn't always select the same action in the same state, i.e., if the optimal policy isn't &lt;a href=&quot;https://ai.stackexchange.com/a/12275/2444&quot;&gt;deterministic&lt;/a&gt; (e.g., in the case of the &lt;em&gt;rock paper scissors&lt;/em&gt;, the optimal policy cannot be deterministic because any intelligent player would easily memorize your deterministic policy, so, after a while, you would always lose again that player), then there are a few things that you can do to make your policy more stochastic&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Change the reward function. If your agent ends up selecting always the same action and you don't want that, it's probably because you're not giving it the right reinforcement signal (given that the agent selects the action that apparently will give it the highest reward in the long run).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to explore more during training. So, if you're using a behavior policy like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, you may want to increase your &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; (i.e. probability of selecting a random action).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you estimated the state-action value function (e.g. with Q-learning), maybe you derived the policy from it by selecting the best action, but, of course, that will make your policy deterministic. You may want to use e.g. softmax to derive the policy from the state-action value function (i.e. the probability of selecting an action is proportional to its value), although Q-learning assumes that your target policy is greedy with respect to the state-action value function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If the optimal policy is supposed to be deterministic, then, if you find the optimal policy (which isn't probably the case), you will end up with an agent that always selects the same action. In that case, obviously, it's not a problem that the RL agent selects always the same optimal action.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-10T18:34:18.537" LastActivityDate="2020-05-10T18:34:18.537" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21058" PostTypeId="1" CreationDate="2020-05-10T20:12:57.527" Score="2" ViewCount="44" Body="&lt;p&gt;When to do discretization to decrease the state/action space in RL? Can you give me some references that such a technique is used?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="2444" LastEditDate="2021-01-04T18:00:57.043" LastActivityDate="2021-01-04T18:00:57.043" Title="When to do discretization to decrease the state/action space in RL?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;state-spaces&gt;&lt;action-spaces&gt;&lt;discretization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21062" PostTypeId="1" AcceptedAnswerId="21064" CreationDate="2020-05-11T04:30:05.957" Score="1" ViewCount="64" Body="&lt;p&gt;AI is supposed to do anything human or traditional computer can do, that is what we expect AI to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So '&lt;strong&gt;generating random value&lt;/strong&gt;' is also a task included in the scope that AI should be able to do &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to generate random value using a single neuron but the outcome isn't much good.&#xA;Any suggestions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS.&lt;br/&gt;&#xA;Random weight initialisation is allowed coz weights are constants at start.&lt;br/&gt;&#xA;Using 'random' function is forbidden anywhere else.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-05-11T08:07:44.017" Title="Random value generator using a single neuron or DNN" Tags="&lt;machine-learning&gt;&lt;randomness&gt;&lt;generator&gt;&lt;neural-networks&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21063" PostTypeId="1" CreationDate="2020-05-11T06:44:33.520" Score="1" ViewCount="22" Body="&lt;p&gt;Consider that we want to create a very big neural network. If we consider to use dense layers, we might face some challenges. Now consider that we use sparse layers instead of dense layers. When using a really sparse model, we would have much less parameters and could create much bigger neural networks. Is there an efficient algorithm to parallelize the training of such networks? &lt;/p&gt;&#xA;" OwnerUserId="35633" LastActivityDate="2020-05-11T06:44:33.520" Title="Is there a parallelizable algorithm for training sparse neural networks?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21064" PostTypeId="2" ParentId="21062" CreationDate="2020-05-11T07:47:36.120" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;AI is supposed to do anything human or traditional computer can do, that is what we expect AI to be.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Technically you would need A&lt;strong&gt;G&lt;/strong&gt;I (Artifical General Intelligence) to do anything a human can do. This is not a technology that exists, but a goal of some AI research to perform more and more general tasks.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So 'generating random value' is also a task included in the scope that AI should be able to do&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Humans are actually very bad at generating random values directly. In fact &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3632045/&quot; rel=&quot;nofollow noreferrer&quot;&gt;it is possible to identify a person&lt;/a&gt; by getting them to generate random numbers from their mind. Indirectly, a human could pick up a die, roll it, and read the number - this is also something that is within capabilities of modern AI, but not something you could implement with a small neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer &lt;em&gt;software&lt;/em&gt; cannot generate random numbers directly, only pseudo-random numbers that are deterministic but follow a statistical pattern very similar to theoretical randomness. When combined with frequently changing seed data from the environment, this becomes very close to an ideal random source, but it does require some hardware input. Modern computer chips include an internal source of randomness to provide this seed data, but it is important to note that it is a piece of specialised hardware, not something that can be coded or simulated using a simple neural network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm trying to generate random value using a single neuron but the outcome isn't much good. Any suggestions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is going to be much harder than you assumed. Random number generators are always more complex than a single function, and require at least a little bit of architecture. The simplest architecture would be to add a state value to use as the next input to the generator, and which is somehow affected by the previous output. A simple feed-forward neural network does not have any architecture like this, you need to add it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first suggestion would be to learn how some simple software pseudo-random number generators (PRNGs) work. A good place to start might be &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_congruential_generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear congruential generators&lt;/a&gt; which are considered very poor quality PRNGs nowadays, but are the sort found in early computer systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be possible to create a rough approximation to a PRNG from a single neuron. Picking the weight, bias and non-linear functions carefully should allow you to produce something similar to the linear congruential generator, where it generates the next pseudo-random number when you input the previous one. A slightly more sophisticated approach would be to make this a simple recurrent neural network (RNN) so that the neural network has an internal state - such a network could still have a single neuron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example neuron that works like this might have a weight of 3, bias of 1 (to generate a &lt;em&gt;logit&lt;/em&gt; from input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$z= 3x+1$&lt;/span&gt;) and have activation function &lt;span class=&quot;math-container&quot;&gt;$y = 1000\text{sin}(z) \text{ mod } 1$&lt;/span&gt;. This is not a normal activation function that you would use to train a neural network - in fact this neural network probably could not be trained at all. It has been specially designed to achieve your goal of generating (pseudo-)random numbers using a single neuron. It is a valid neural network though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created the above example neuron, started it with a seed of 0, and got the sequence &lt;code&gt;0.4710, 0.8543, 0.1536, 0.9479, 0.1142, 0.0683, 0.7741, 0.2985, 0.7315, 0.0777, 0.5078&lt;/code&gt; by feeding back previous values - you can see it is working, although it is probably a very bad random number generator even compared to linear congruential generators. Possibly better than a human though.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-11T08:07:44.017" LastActivityDate="2020-05-11T08:07:44.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21065" PostTypeId="1" AcceptedAnswerId="21083" CreationDate="2020-05-11T08:58:13.647" Score="1" ViewCount="45" Body="&lt;p&gt;What are some conferences for publishing papers on graph convolutional networks?&lt;/p&gt;&#xA;" OwnerUserId="31672" LastEditorUserId="2444" LastEditDate="2020-05-11T20:18:42.923" LastActivityDate="2020-05-11T20:18:42.923" Title="What are some conferences for publishing papers on graph convolutional networks?" Tags="&lt;reference-request&gt;&lt;research&gt;&lt;geometric-deep-learning&gt;&lt;academia&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21067" PostTypeId="1" CreationDate="2020-05-11T09:35:45.850" Score="1" ViewCount="59" Body="&lt;p&gt;I have a convolutional neural network such as U-Net for a segmentation task and in input images with the same spatial resolution (256x256) but different pixel size due to the acquisition process. Specifically, every image has FoV 370x370mm 256/256 pixel, but a different zoom, for example an image might have 2.7/1.8 px/mm and another image 2.4/1.7 px/mm. &#xA;Considering the FoV the pixel size should be 370/256=1.44mm x pixel but with a zoom of 2.7/1.8 px/mm which is the pixel size in this case ? I thought 1.8/2.7= 0.67mm but I am not sure. &#xA;Why should I have the same in-plane resolution (pixel size) for each image when I train my CNN and not only the same spatial resolution (256x256 px) ? &lt;/p&gt;&#xA;" OwnerUserId="36939" LastActivityDate="2020-05-11T09:35:45.850" Title="How to manage the different pixel size for a CNN?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21068" PostTypeId="1" AcceptedAnswerId="21122" CreationDate="2020-05-11T11:59:18.823" Score="0" ViewCount="141" Body="&lt;p&gt;I am working on an app that generates heat/ thermal map given a picture. i have been able to get what i expected using python opencv builtin function &lt;code&gt;cv2.applyColorMap(img, cv2.COLORMAP_JET)&lt;/code&gt;. Everything works exactly as expected. But i want to understand how &lt;code&gt;applyColorMap&lt;/code&gt; works at the back end. I am aware how several image filters (like image, edge filters) work by convolution / cross correlation with appropriate kernals, but i can't seem to pull the same concept for color maps. &#xA;For this question lets consider a color map where we want :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brightest ones: (RED COLORED)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MEDIUM INTENSITY ONES: (YELLOW COLORED)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LOW INTENSITY ONES: (BLUE COLORED)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What i have done:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried dividing the pixels into 3 categories and replaced each pixel with the either of the colors (RED, YELLOW, BLUE ) depending upon it's value from gray scale image( 0-255). &#xA;This approach had a problem that there were solid 3 colors in the image with no variation in intensity of the individual color while in a good heat map there is blend of colors ( it decreases or increases ) based upon the intensity . I want to achieve that effect. I would appreciate any help or any lead to understand how heat maps work .&lt;/p&gt;&#xA;" OwnerUserId="36484" LastActivityDate="2020-05-13T09:56:42.797" Title="How does the math behind heat map filters work?" Tags="&lt;python&gt;&lt;computer-vision&gt;&lt;image-processing&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21069" PostTypeId="1" CreationDate="2020-05-11T13:08:39.250" Score="2" ViewCount="32" Body="&lt;p&gt;I have 2 different models with each model doing a separate function and have been trained with different weights. Is there any way I can merge these two models to get a single model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If it can be merged&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How should I go about it? Will the number of layers remain the same?&lt;/li&gt;&#xA;&lt;li&gt;Will it give me any performance gain?(Intuitively speaking, I should get a higher performance)&lt;/li&gt;&#xA;&lt;li&gt;Will the hardware requirements change when using the new model?&lt;/li&gt;&#xA;&lt;li&gt;Will I need to retrain the model? Can I somehow merge the trained weights?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If the models cannot be merged&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why so? After all, convolution is finding the correct pattern in data. &lt;/li&gt;&#xA;&lt;li&gt;Also, if CNN's cannot be merged, then how do skip-connections like ResNet50 work?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;EDIT:&lt;/em&gt;&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Representation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;What I currently have&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Image ---(model A) ---&amp;gt; Temporary image ---(Model B)---&amp;gt; Output image&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;What I want:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Image ---(model C) ---&amp;gt; Output image&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="36944" LastEditorUserId="36944" LastEditDate="2020-05-11T14:33:09.117" LastActivityDate="2020-05-11T14:33:09.117" Title="Merge two different CNN models into one" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21070" PostTypeId="2" ParentId="21044" CreationDate="2020-05-11T13:31:00.277" Score="7" Body="&lt;p&gt;If we write the pseudo-code for the SARSA algorithm we first initialise our hyper-parameters etc. and then initialise &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;, which we use to choose &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; from our policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;. Then for each &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the episode we do the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; and observe &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Choose &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt; using &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; in our policy&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(S_t, A_t) = Q(S_t, A_t) + \alpha [R_{t+1} + \gamma Q(S_{t+1},A_{t+1}) - Q(S_t, A_t)]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now, in Q-learning we replace &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1},A_{t+1})$&lt;/span&gt; in line 3 with &lt;span class=&quot;math-container&quot;&gt;$\max_aQ(S_{t+1},a)$&lt;/span&gt;. Recall that in SARSA we chose our &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt; using our policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; - if our policy is greedy with respect to the action value function then this simply means the policy is &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s) = \text{argmax}_aQ(s,a)$&lt;/span&gt; which is exactly how we choose our weight update in Q-learning.&lt;/p&gt;&#xA;&lt;p&gt;To answer the question - &lt;strong&gt;no&lt;/strong&gt;, they are not &lt;em&gt;always&lt;/em&gt; the same algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Consider where we transition from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s'=s$&lt;/span&gt;. I will outline the updates for SARSA and Q-learning indexing the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; functions with &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to demonstrate the difference.&lt;/p&gt;&#xA;&lt;p&gt;For each case, I will assume we are at the start of the episode, as this is the easiest way to illustrate the difference. Note that actions denoted by &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; are for actions taken explicitly in the environment -- in the Q-Learning update the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; action that is chosen for the update is not executed in the environment, the action taken in the environment is chosen by the policy after the update has happened.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SARSA&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We initialise &lt;span class=&quot;math-container&quot;&gt;$S_0 = s$&lt;/span&gt; and choose &lt;span class=&quot;math-container&quot;&gt;$A_0 = \text{argmax}_a Q_0(s,a)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Take action &lt;span class=&quot;math-container&quot;&gt;$A_0$&lt;/span&gt; and observe &lt;span class=&quot;math-container&quot;&gt;$R_{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S_{1} = s' = s$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Choose action &lt;span class=&quot;math-container&quot;&gt;$A_{1} = \text{argmax}_aQ_{0}(s,a)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Q_{1}(S_0,A_0) = Q_0(S_0,A_0) + \alpha [R_{1} + \gamma Q_0(s,A_1) - Q_0(S_0,A_0)]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q-Learning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialise &lt;span class=&quot;math-container&quot;&gt;$S_0 = s$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Choose action &lt;span class=&quot;math-container&quot;&gt;$A_0 = \text{argmax}_aQ_0(s,a)$&lt;/span&gt;, observe &lt;span class=&quot;math-container&quot;&gt;$R_{1}, S_{1} = s' = s$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Q_{1}(S_0,A_0) = Q_0(S_0,A_0) + \alpha [R_{1} + \gamma \max_aQ_0(s,a) - Q_0(S_0,A_0)]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Choose action &lt;span class=&quot;math-container&quot;&gt;$A_1 = \text{argmax}_aQ_1(s,a)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As you can see the next action for the updates in SARSA (line 4) and Q-learning (line 3) are taken with respect to the same &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, &lt;em&gt;but the key difference is that the actual next action taken in &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is taken with respect to the updated &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The key for understanding this edge case is that when we transition into the same state, &lt;strong&gt;the Q-Learning update will update the Q-function before choosing &lt;span class=&quot;math-container&quot;&gt;$A_1$&lt;/span&gt;&lt;/strong&gt;. I have indexed actions and Q-functions by the episode step - hopefully, it makes sense why I have done this for the Q-functions as, usually, this would not make sense, but, because we have two successive states that are the same, it is okay.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-12-06T12:26:10.680" LastActivityDate="2020-12-06T12:26:10.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21071" PostTypeId="2" ParentId="20588" CreationDate="2020-05-11T15:20:08.190" Score="0" Body="&lt;p&gt;well... simply put a &lt;code&gt;.split()&lt;/code&gt; at the end of your first two lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = &quot;This is a dog.&quot;.split()&#xA;b = &quot;This is a cat.&quot;.split()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your algorithm works with the iterables, and the string is broken into it's characters. You do the split, and &lt;code&gt;a,b&lt;/code&gt; would be a list of words, then your algorithm works on the word-level&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output on your example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[0. 1. 2. 3. 4.]&#xA; [1. 0. 1. 2. 3.]&#xA; [2. 1. 0. 1. 2.]&#xA; [3. 2. 1. 0. 1.]&#xA; [4. 3. 2. 1. 1.]]&#xA;&#xA;1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2020-05-11T15:20:08.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21072" PostTypeId="2" ParentId="20588" CreationDate="2020-05-11T15:29:04.943" Score="0" Body="&lt;p&gt;Maybe try this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from functools import lru_cache&#xA;from itertools import product&#xA;&#xA;@lru_cache(maxsize=4095)&#xA;def ld(s, t):&#xA;    &quot;&quot;&quot;&#xA;    Levenshtein distance memoized implementation from Rosetta code:&#xA;    https://rosettacode.org/wiki/Levenshtein_distance#Python&#xA;    &quot;&quot;&quot;&#xA;    if not s: return len(t)&#xA;    if not t: return len(s)&#xA;    if s[0] == t[0]: return ld(s[1:], t[1:])&#xA;    l1 = ld(s, t[1:])      # Deletion.&#xA;    l2 = ld(s[1:], t)      # Insertion.&#xA;    l3 = ld(s[1:], t[1:])  # Substitution.&#xA;    return 1 + min(l1, l2, l3)&#xA;&#xA;&#xA;a = &quot;this is a sentence&quot;.split()&#xA;b = &quot;yet another cat thing&quot;.split()&#xA;&#xA;# To get the triplets.&#xA;for i, j in product(a, b):&#xA;    print((i, j, ld(i, j)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get a matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.sparse import coo_matrix&#xA;import numpy as np&#xA;&#xA;a = &quot;this is a sentence&quot;.split()&#xA;b = &quot;yet another cat thing , yes&quot;.split()&#xA;&#xA;tripets = np.array([(i, j, ld(w1, w2)) for (i, w1) , (j, w2) in product(enumerate(a), enumerate(b))])&#xA;row, col, data = [np.squeeze(splt) for splt in np.hsplit(tripets, tripets.shape[-1])]&#xA;coo_matrix((data, (row, col))).toarray()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[out]:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[4, 5, 4, 2, 4, 3],&#xA;       [3, 7, 3, 4, 2, 2],&#xA;       [3, 6, 2, 5, 1, 3],&#xA;       [6, 7, 7, 7, 8, 7]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36951" LastEditorUserId="36951" LastEditDate="2020-05-11T15:37:10.960" LastActivityDate="2020-05-11T15:37:10.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21073" PostTypeId="1" AcceptedAnswerId="21107" CreationDate="2020-05-11T15:41:25.843" Score="1" ViewCount="134" Body="&lt;p&gt;My main purpose right now is to train an agent using the A2C algorithm to solve the Atari Breakout game. So far I have succeeded to create that code with a single agent and environment. To break the correlation between samples (i.i.d), I need to have an agent interacting with several environments. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class GymEnvVec():&#xA;&#xA;    def __init__(self, env_name, n_envs, seed=0):&#xA;        make_env = lambda: gym.make(env_name)&#xA;        self.envs = [make_env() for _ in range(n_envs)]&#xA;        [env.seed(seed + 10 * i) for i, env in enumerate(self.envs)]&#xA;&#xA;    def reset(self):&#xA;        return [env.reset() for env in self.envs]&#xA;&#xA;    def step(self, actions):&#xA;        return list(zip(*[env.step(a) for env, a in zip(self.envs, actions)]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can use the class &lt;code&gt;GymEnvVec&lt;/code&gt; to vectorize my environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I can set my environments with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;envs = GymEnvVec(env_name=&quot;Breakout-v0&quot;, n_envs=50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can get my first observations with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;observations = envs.reset()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pick some actions with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;actions = agent.choose_actions(observations)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;choose_actions&lt;/code&gt; method might look like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def choose_actions(self, states):&#xA;        assert isinstance(states, (list, tuple))&#xA;&#xA;        actions = []&#xA;        for state in states:&#xA;            probabilities  = F.softmax(self.network(state)[0])&#xA;            action_probs = T.distributions.Categorical(probabilities)&#xA;            actions.append(action_probs.sample())&#xA;&#xA;        return [action.item() for action in actions] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, the environments will spit the next_states, rewards and if it is done with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;next_states, rewards, dones, _ = env.step(actions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is at this point I am a bit confused. I think I need to gather immediate experiences, batch altogether and forward it to the agent. My problem is probably with the &quot;gather immediate experiences&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I propose a solution, but I am far from being sure it is a good answer. At each iteration, I think I must take a random number with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nb = random.randint(0, len(n_envs)-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and put the experience in history with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;history.append(Experience(state=states[nb], actions[nb], rewards[nb], dones[nb]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Am I wrong? Can you tell me what I should do?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-11T18:25:09.890" LastActivityDate="2020-05-18T09:58:31.163" Title="Once the environments are vectorized, how do I have to gather immediate experiences for the agent?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;pytorch&gt;&lt;actor-critic-methods&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21074" PostTypeId="2" ParentId="17047" CreationDate="2020-05-11T15:58:26.083" Score="1" Body="&lt;h1&gt;Lets mock some data up.&lt;/h1&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;100 numbers, each one is a parameter, they together define a number X(also given)&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# i.e. size of X_train -&amp;gt; [n x d]&#xA;# i.e. size of X_train -&amp;gt; [??? x 100]  , when d = 100&#xA;&#xA;# &quot;I have 20000 instances for training&quot;&#xA;# i.e. size of X_train -&amp;gt; [20000 x 100], when n = 20000&#xA;&#xA;import torch&#xA;import numpy as np&#xA;&#xA;X_train = torch.rand((20000, 100))&#xA;X_train = np.random.rand(20000, 100) # Or using numpy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;But what is your Y?&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Since the definition of a regression task, &#xA;# loosely means to predict an output real number&#xA;# given an input of d dimension&#xA;&#xA;# So the appropriate Y_train would &#xA;# be of dimension [n x 1] &#xA;# and look like this:&#xA;&#xA;y_train = torch.rand((20000, 1))&#xA;&#xA;y_train = np.random.rand(20000, 1) # Or using numpy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;What is a linear perceptron?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Taking definition from &lt;a href=&quot;https://github.com/alvations/tsundoku/blob/master/completed/Session%201%20-%20Hello%20DL%20World.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eEVd5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eEVd5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Thus, in picture:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZPM3A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZPM3A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Next we need to define training routine,&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;For now take it as biblical truth that this is an okay routine to train a neural net model (this isn't the only way but easiest or supervised learning):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Jt7M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Jt7M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;In code:&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;import numpy as np&#xA;np.random.seed(0)&#xA;&#xA;def sigmoid(x): # Returns values that sums to one.&#xA;    return 1 / (1 + np.exp(-x))&#xA;&#xA;def sigmoid_derivative(sx): &#xA;    # See https://math.stackexchange.com/a/1225116&#xA;    # Hint: let sx = sigmoid(x)&#xA;    return sx * (1 - sx)&#xA;&#xA;def cost(predicted, truth):&#xA;    return np.abs(truth - predicted)&#xA;&#xA;&#xA;num_epochs = 10000 # No. of times to iterate.&#xA;learning_rate = 0.03 # How large a step to take per iteration.&#xA;&#xA;# Lets standardize and call our inputs X and outputs Y&#xA;X = np.array(torch.rand((20000, 100)))&#xA;Y = or_output&#xA;&#xA;for _ in range(num_epochs):&#xA;    layer0 = X&#xA;&#xA;    # Step 2a: Multiply the weights vector with the inputs, sum the products, i.e. s&#xA;    # Step 2b: Put the sum through the sigmoid, i.e. f()&#xA;    # Inside the perceptron, Step 2. &#xA;    layer1 = sigmoid(np.dot(X, W))&#xA;&#xA;    # Back propagation.&#xA;    # Step 3a: Compute the errors, i.e. difference between expected output and predictions&#xA;    # How much did we miss?&#xA;    layer1_error = cost(layer1, Y)&#xA;&#xA;    # Step 3b: Multiply the error with the derivatives to get the delta&#xA;    # multiply how much we missed by the slope of the sigmoid at the values in layer1&#xA;    layer1_delta = layer1_error * sigmoid_derivative(layer1)&#xA;&#xA;    # Step 3c: Multiply the delta vector with the inputs, sum the product (use np.dot)&#xA;    # Step 4: Multiply the learning rate with the output of Step 3c.&#xA;    W +=  learning_rate * np.dot(layer0.T, layer1_delta)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now that we learn the model, i.e. the &lt;code&gt;W&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we see the data points that we need to use the model on, we apply the same forward propagation step, i.e. &lt;code&gt;layer1 = sigmoid(np.dot(X, W))&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we have:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have 5000 lines given, each containing the 100 numbers as parameters.My task is to predict the number X for these 5000 instances.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And in code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# If we mock up the data,&#xA;# it should be the same internal dimension. &#xA;X_test = np.random.rand(5000, 100)&#xA;&#xA;# The desired output just needs to pass through the W and the activation:&#xA;# the shape of `output` -&amp;gt; [5000 x 1] , &#xA;# where there's 1 output value for each input.&#xA;output = sigmoid(np.dot(X_test, W))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36951" LastActivityDate="2020-05-11T15:58:26.083" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21075" PostTypeId="2" ParentId="16599" CreationDate="2020-05-11T16:00:12.243" Score="3" Body="&lt;p&gt;Kaggle recently started adding 'Simulation' competitions, which are well-suited for reinforcement learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first competition that's live (no prizes) is &lt;a href=&quot;https://www.kaggle.com/c/connectx&quot; rel=&quot;nofollow noreferrer&quot;&gt;ConnectX&lt;/a&gt;, like a generalised Connect Four. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first competition with prize money is likely to be the next iteration of TwoSigma's Halite. There's a page for it here, but it hasn't been launched yet: &lt;a href=&quot;https://www.kaggle.com/c/halite/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/c/halite/overview&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created a site that lists &lt;a href=&quot;https://mlcontests.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ongoing machine learning competitions including Reinforcement Learning competitions&lt;/a&gt; - you can also sign up to the email list in case you want to get emails (roughly monthly) when new competitions launch. As of right now (May 2020) there are a few live RL competitions on there - the KDD cup, and AWS DeepRacer.&lt;/p&gt;&#xA;" OwnerUserId="36952" LastEditorUserId="36952" LastEditDate="2020-05-15T15:13:53.813" LastActivityDate="2020-05-15T15:13:53.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21076" PostTypeId="1" CreationDate="2020-05-11T16:04:54.853" Score="1" ViewCount="40" Body="&lt;p&gt;I finished working on a new algorithm in Reinforcement Learning, I need to compare it to some well-known algorithms. That's why I need to know the step-by-step procedures that RL researchers usually take in order to get their results and compare them to other papers' results. (e.g. running the algorithm multiple times with different random seeds, saving results to &lt;code&gt;.csv&lt;/code&gt; files, plotting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone can help? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am working on Pytorch, PyBullet Environments.)&lt;/p&gt;&#xA;" OwnerUserId="36603" LastEditorUserId="2444" LastEditDate="2020-05-11T17:39:00.327" LastActivityDate="2020-05-11T17:39:00.327" Title="What are the procedures to get RL paper results?" Tags="&lt;reinforcement-learning&gt;&lt;research&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2020-05-17T11:09:16.630" ContentLicense="CC BY-SA 4.0" />
  <row Id="21077" PostTypeId="1" CreationDate="2020-05-11T16:30:11.757" Score="1" ViewCount="79" Body="&lt;p&gt;During the training of DQN, I noticed that the model with prioritized experience replay (PER) had a smaller loss in general compared to a DQN without PER. The mean squared loss was an order of magnitude &lt;span class=&quot;math-container&quot;&gt;$10^{-5}$&lt;/span&gt; for the DQN with PER, whereas the mean squared loss was an order of magnitude &lt;span class=&quot;math-container&quot;&gt;$10^{-2}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do the smaller training errors have any effect on executing the final policy learned by the DQN?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-11T17:42:04.700" LastActivityDate="2021-03-08T02:03:45.507" Title="Do smaller loss values during DQN training produce better policies?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21078" PostTypeId="2" ParentId="17047" CreationDate="2020-05-11T18:32:46.193" Score="1" Body="&lt;p&gt;The quick answer is that you want to use an activation function on the output layer that does not compress values to &lt;span class=&quot;math-container&quot;&gt;$(0,1)$&lt;/span&gt;. Depending on your software, this might be called &quot;linear&quot; or &quot;identity&quot;. It looks like Keras just wants you to leave off the activation function: &lt;code&gt;model.add(Dense(1))&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical way of thinking of a neural network as a classifier (let's say a binary classifier) is just extending a logistic regression. In fact, when you use a sigmoid activation function on the output node, you're (sort of) running logistic regression on the final hidden layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A logistic regression is one type of generalized linear model. The gist of GLM is that some transformation of the the value of interest is a linear function of the feature space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; be the data matrix for the feature space. Let &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; be a parameter vector. Then &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = \mathbb{E}[y] = X\beta$&lt;/span&gt; is the linear model, and &lt;span class=&quot;math-container&quot;&gt;$g(\mathbb{E}[y]) = X\beta$&lt;/span&gt; is the generalized linear model (vectorized, so apply &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; to each &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we could extend this to a nonlinear transformation, and when a neural network is a binary classifier, this is precisely what we're doing. Instead of the transformation of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; being given by &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and thus linear, we apply some nonlinear transformation &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and get &lt;span class=&quot;math-container&quot;&gt;$g(\mathbb{E}[y]) = f(X)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terminology in GLM is &quot;link function&quot;, but that is essentially the activation function on the final node(s) of the neural network. Consequently, all of the GLM link functions are in play, and one of those link functions is the identity function. For a GLM, that's just linear regression. For your neural network, it will be a neural network (nonlinear) regression, which sounds like what you want.&lt;/p&gt;&#xA;" OwnerUserId="25529" LastEditorUserId="25529" LastEditDate="2020-05-11T18:41:10.433" LastActivityDate="2020-05-11T18:41:10.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21079" PostTypeId="2" ParentId="20681" CreationDate="2020-05-11T18:59:50.167" Score="2" Body="&lt;p&gt;Did you mean: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do you use a pre-trained BERT model in a &lt;strong&gt;feature-based&lt;/strong&gt; setting&#xA;  to get pre-trained &lt;strong&gt;word contextual embeddings&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is the BERT paper. I highly recommend you read it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, by &lt;em&gt;sentences&lt;/em&gt;, we mean a sequence of &lt;strong&gt;word embedding representations&lt;/strong&gt; of the words (or tokens) in the sentence. Word embeddings are the vectors that you mentioned, and so a (usually fixed) sequence of such vectors represent the sentence input. (We don't need the input to always be divided to individual sentences)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;strong&gt;mainly&lt;/strong&gt; two ways to train the modern neural network models for sequence modeling tasks such as language modelling, machine translation and question answering:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1. The pretrain - finetune based approach (also called Transfer Learning)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In this approach, a model is first pre-trained on some auxiliary dataset whose domain usually overlaps with the domain of the dataset corresponding to the &lt;strong&gt;downstream task&lt;/strong&gt;, that is, the actual task for which we need to build the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of Natural Language Processing (NLP) tasks, such as machine translation and question answering, the pre-training that is done is usually unsupervised (no labels in the dataset) where our training objective is to maximize the &lt;strong&gt;log likelihood&lt;/strong&gt; of the sentences. You can read more about Maximum Likelihood estimation &lt;a href=&quot;http://pages.cs.wisc.edu/~jerryzhu/cs838/LM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its almost always a good idea to pre-train models (whether it is for NLP tasks or for computer vision tasks etc). The more we pre-train a model, the closer its parameters are to the optimal ones, and the work needed in the &lt;strong&gt;fine-tuning&lt;/strong&gt; phase is minimized. This is why we use large scale datasets for pre-training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later, during the fine-tuning phase, we fine tune the parameters of the model to suit a downstream task, such as a task of translation between two &lt;strong&gt;specific&lt;/strong&gt; languages of text. Since the parameters of the model are already at a position in the error/loss space that is already &lt;strong&gt;sort of good enough&lt;/strong&gt; for the model (because we already pre-trained it with text in, say, 10 languages), the fine-tuning can, in a sense, tailor the model to suit the task a hand.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2. The feature based approach&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In this approach, we take an already pre-trained model (any model, e.g. a transformer based neural net such as BERT, which has been &lt;strong&gt;pre-trained&lt;/strong&gt; as described in the previous approach) and then we extract the activations from one or more layers of this pre-trained model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is done by simply inputting the word embedding sequence corresponding to a sentence to the pre-trained model and then extracting the activations from one or more layers (one or more of the last layers, since the features associated to the activations in these layers are far more complex and include more context) of this pre-trained model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These activations (also called &lt;strong&gt;contextual embeddings&lt;/strong&gt;) are used as input (similar to the word embeddings described earlier) to another model, such as an LSTM or even another BERT. That is, fixed features are extracted from the pre-trained model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually the size of these layers are the same as the size of the input layer, i.e. they are also a sequence, but a sequence of far more complex features instead of word embeddings. If the word embedding vector of the input to this model id &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and the length of the sentence which is input to the model is &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, then each of the hidden activation layers have the same length &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, but the hidden size of these &lt;strong&gt;vectors&lt;/strong&gt; is &lt;span class=&quot;math-container&quot;&gt;$H &amp;gt;&amp;gt; D$&lt;/span&gt;, i.e. far higher to incorporate more complex features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though the dimensionality of the hidden layers is far higher, these layers are equal in length to the input layer, and so we have a one to one correspondence between the input tokens in the sentence and each of these hidden layer activations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bonus in using these hidden representations is that we get to use representations of the input sentence that also incorporate &lt;strong&gt;CONTEXT&lt;/strong&gt;, dependency between the words of the sentence, an important feature that is to be modeled by all models used for NLP tasks. Models such as BERT, are based on Transformer models that use an attention mechanism in order to model these contextual features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Wu3Q4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wu3Q4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is a superb article describing to the last detail the attention mechanism and the Transformer model based on which models like BERT operate. For more about contextual embeddings I recommend taking a look at &lt;a href=&quot;http://jalammar.github.io/illustrated-bert/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; article. The figure used here is from this article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Models such as BERT make use of &lt;strong&gt;one half&lt;/strong&gt; of these Transformer models, called the &lt;strong&gt;encoder&lt;/strong&gt;, since the Transformer is a &lt;strong&gt;seq2seq&lt;/strong&gt; or &lt;strong&gt;encoder-decoder model&lt;/strong&gt;. &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is the link to the Transformer paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/nlp-extract-contextualized-word-embeddings-from-bert-keras-tf-67ef29f60a7b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a great blog on extracting contextual word embeddings from BERT using Tensorflow and Keras.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll also provide a link to a &lt;a href=&quot;https://www.kaggle.com/funtowiczmo/hugging-face-transformers-how-to-use-pipelines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle Python Notebook&lt;/a&gt; on using Pipelines functionality from the HuggingFace community repo on github that also is used for feature extraction (contextual embeddings).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if you have any questions, happy to help!&lt;/p&gt;&#xA;" OwnerUserId="26392" LastEditorUserId="26392" LastEditDate="2020-05-11T21:08:28.290" LastActivityDate="2020-05-11T21:08:28.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21080" PostTypeId="1" CreationDate="2020-05-11T19:40:14.393" Score="2" ViewCount="66" Body="&lt;p&gt;The first neural net I wrote was a classifier. After that, I learned that neural nets can be used for regression tasks, even quantile regression.&lt;/p&gt;&#xA;&lt;p&gt;It has become clear to me that the usual games with extensions of OLS linear regression can be applied to neural networks.&lt;/p&gt;&#xA;&lt;p&gt;What work has been done with Poisson-style regression via neural networks with log link functions (exponential activation function)?&lt;/p&gt;&#xA;" OwnerUserId="25529" LastEditorUserId="2444" LastEditDate="2020-06-27T23:46:08.427" LastActivityDate="2020-06-27T23:46:08.427" Title="What work has been done with Poisson-style regression via neural networks with exponential activation function?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21082" PostTypeId="2" ParentId="21077" CreationDate="2020-05-11T19:43:56.727" Score="0" Body="&lt;p&gt;I think it says something about the training progress, while another approach you can make sure is to look at the gradient norm. Sometimes, the training loss is really noisy while the gradient norm is much more clear.&lt;/p&gt;&#xA;" OwnerUserId="36787" LastActivityDate="2020-05-11T19:43:56.727" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21083" PostTypeId="2" ParentId="21065" CreationDate="2020-05-11T20:13:04.317" Score="2" Body="&lt;p&gt;Based on past publications, here are some journals and conferences where you can possibly publish or present a research paper on geometric deep learning or graph neural networks&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://nips.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Information Processing Systems (NIPS)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://iclr.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Conference on Learning Representations (ICLR)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cvpr2020.thecvf.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conference on Computer Vision and Pattern Recognition (CVPR)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://icml.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Conference on Machine Learning (ICML)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The website &lt;a href=&quot;http://geometricdeeplearning.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://geometricdeeplearning.com/&lt;/a&gt; also provides some information about the topic and links you to several workshops, papers, and tutorials. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some links to some of the past workshops on GDL. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://iclr.cc/Conferences/2019/Schedule?showEvent=631&quot; rel=&quot;nofollow noreferrer&quot;&gt;Representation Learning on Graphs and Manifolds&lt;/a&gt; (and &lt;a href=&quot;https://rlgm.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related homepage&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ipam.ucla.edu/programs/workshops/workshop-iv-deep-geometric-learning-of-big-data-and-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Workshop IV: Deep Geometric Learning of Big Data and Applications&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://nips.cc/Conferences/2017/Schedule?showEvent=8735&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geometric Deep Learning on Graphs and Manifolds&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-11T20:13:04.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21084" PostTypeId="1" AcceptedAnswerId="21094" CreationDate="2020-05-11T20:36:05.673" Score="0" ViewCount="70" Body="&lt;p&gt;For some environments taking an action may not update the environment state. For example, a trading RL agent may take an action to buy shares s. The state at time t which is the time of investing is represented as the interval of 5 previous prices of s. At t+1 the share price has changed but it may not be as a result of the action taken. Does this affect RL learning, if so how ? Is it required that state is updated as a result of taking actions for agent learning to occur ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In gaming environments it is clear how actions affect the environment. Can some rules of RL breakdown if no &quot;noticeable&quot; environment change takes place as a result of actions ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;actions influence the state transitions&quot;, is my understanding correct:&#xA;If transitioning to a new state is governed by epsilon greedy and epsilon is set to .1 then with .1 probability the agent will choose an action from the q table which has max reward reward for the given state. Otherwise the agent randomly chooses and performs an action then updates the q table with discounted reward received from the environment for the given action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've not explicitly modeled an MDP and just defined the environment and let the agent determine best actions over multiple episodes of choosing either a random action or the best action for the given state, the selection is governed by epsilon greedy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But perhaps I've not understood something fundamental in RL. I'm ignoring MDP in large part as I'm not modeling the environment explicitly. I don't set the probabilities of moving from each state to other states.&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="12964" LastEditDate="2020-05-12T11:10:54.680" LastActivityDate="2020-05-12T11:10:54.680" Title="Is it required that taking an action updates the state?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21085" PostTypeId="1" AcceptedAnswerId="21095" CreationDate="2020-05-11T21:23:37.377" Score="1" ViewCount="45" Body="&lt;p&gt;Does Google's deep dream project have a specific website so that it can be used to run edits and renderings on images and provide an output?&lt;/p&gt;&#xA;" OwnerUserId="36957" LastEditorUserId="2444" LastEditDate="2020-05-11T21:26:23.687" LastActivityDate="2020-05-12T02:53:12.040" Title="Does Google's deep dream project have a specific website for playing with it?" Tags="&lt;google&gt;&lt;deepdream&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21086" PostTypeId="1" AcceptedAnswerId="21090" CreationDate="2020-05-11T21:53:10.293" Score="1" ViewCount="52" Body="&lt;p&gt;Are there any reference papers where it is used a KMeans-like algorithm in state space quantization in Reinforcement Learning instead of range buckets?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-05-11T23:11:17.973" Title="Can I do state space quantization using a KMeans-like algorithm instead of range buckets?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;k-means&gt;&lt;dimensionality&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21089" PostTypeId="2" ParentId="20870" CreationDate="2020-05-11T22:59:47.150" Score="1" Body="&lt;p&gt;&lt;strong&gt;Bernoulli naïve Bayes&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x|c_k) = \prod^{n}_{i=1} p^{x_i}_{ki} (1-p_{ki})^{(1-x_i)}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's examine the example of document classification.&lt;br&gt;&#xA;Let K different text classes and n different terms that our vocabulary contains. &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;  are boolean variables (0, 1) expressing if the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; term exists in document &lt;strong&gt;x&lt;/strong&gt;. &lt;strong&gt;x&lt;/strong&gt; is a vector of dimension n.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x|c_k)$&lt;/span&gt; is the probability that given the class k, document &lt;strong&gt;x&lt;/strong&gt; to be generated. The equation uses a common trick to represent a multivariate Bernoulli event model, taking into account that when &lt;span class=&quot;math-container&quot;&gt;$x_i = 1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$1 - x_i = 0$&lt;/span&gt; and inversely. In other words, for each term it takes the probability that the document does contain this term or it does not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{ki}$&lt;/span&gt; is the probability of class &lt;span class=&quot;math-container&quot;&gt;$c_k$&lt;/span&gt; generating the term &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, that is it could be the prior probability a document that belongs to k class contains this term of the vocabulary.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-05-11T23:05:14.407" LastActivityDate="2020-05-11T23:05:14.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21090" PostTypeId="2" ParentId="21086" CreationDate="2020-05-11T23:11:17.973" Score="2" Body="&lt;p&gt;There is this paper &lt;a href=&quot;https://arxiv.org/pdf/1712.00654.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Representation and Reinforcement Learning for&#xA;Personalized Glycemic Control in Septic Patients&lt;/a&gt;, presented in the Machine Learning for Health Workshop in NIPS 2017. Here is a quote from the paper where the authors describe the clustering approach: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;After we generated the state representation,&#xA;  we used the k-means clustering algorithm to categorize millions of patient states into 500 clusters&#xA;  such that similar clinical states can collapse into the same cluster.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-05-11T23:11:17.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21091" PostTypeId="2" ParentId="17287" CreationDate="2020-05-11T23:36:07.290" Score="2" Body="&lt;p&gt;&lt;strong&gt;The Problem of Overfitting&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In most cases, when you increase a lot the number of epochs your model finally overfits. This is because your model reaches the point that it does not learn anymore but tries to remember what it has seen before. This is overfitting. So there is often a trade-off between the number of epochs and overfitting. In general a good way to avoid overfitting, except for fine-tuning, regularization, dropout etc, is to understand what you have from the learning curve. In most cases overfitting happens, after some epochs have passed, and, as a result the training error still decreases, whereas the validation error flactuates or increases. If so, you should save only the learning updates before overfitting appears and/or validation error is minimum.&lt;br /&gt;&#xA;Methods: early-stopping, checkpoint.&lt;/p&gt;&#xA;&lt;p&gt;useful link: &lt;a href=&quot;https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-10-10T12:29:34.737" LastActivityDate="2020-10-10T12:29:34.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21092" PostTypeId="1" CreationDate="2020-05-12T00:37:49.970" Score="1" ViewCount="222" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PcHLG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PcHLG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm predicting the used 9 pictures to predict the  last picture &lt;br/&gt;&#xA;so (40,40,9) -&gt; unet -&gt; (40,40,1) &lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but as you see the predict picture &lt;br/&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9nmxt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9nmxt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not just a mask(0or 1) its float &lt;br/&gt;&#xA;so which loss function should I define to achieve the best Unet result? and why?&lt;br/&gt;&lt;/p&gt;&#xA;" OwnerUserId="27529" LastActivityDate="2020-05-12T00:37:49.970" Title="what will be the best loss function for unet to predict the each pixel values?" Tags="&lt;loss-functions&gt;&lt;image-segmentation&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21093" PostTypeId="1" CreationDate="2020-05-12T02:21:45.787" Score="2" ViewCount="66" Body="&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$x_{t+1} \sim \mathbb{P}(\cdot | x_t, a_t)$&lt;/span&gt; denotes the state transition dynamics in a reinforcement learning (RL) problem. Let &lt;span class=&quot;math-container&quot;&gt;$y_{t+1} = \mathbb{P}(\cdot | x_{t+1})$&lt;/span&gt; denote the noisy observation or the imperfect state information. Let &lt;span class=&quot;math-container&quot;&gt;$H_{t}$&lt;/span&gt; denote the history of actions and observations &lt;span class=&quot;math-container&quot;&gt;$H_{t+1} = \{b_0,y_0,a_0,\cdots,y_{t+1}\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the RL Partially Observed Markov Decision Process (RL-POMDP), the summary of the history is contained in the &quot;belief state&quot;  &lt;span class=&quot;math-container&quot;&gt;$b_{t+1}(i) = \mathbb{P}(x_{t+1} = i | H_{t+1})$&lt;/span&gt;, which is the posterior distribution over the states conditioned on the history.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose the model is NOT known. Clearly, the belief state can't be computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we use a Gaussian Process (GP) to approximate the belief distribution &lt;span class=&quot;math-container&quot;&gt;$b_{t}$&lt;/span&gt; at every instant &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can Variational GP be adapted to such a situation? Can universal approximation property of GP be invoked here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there such results in the literature? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references and insights into this problem would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="36970" LastEditorUserId="2444" LastEditDate="2020-05-12T09:49:30.027" LastActivityDate="2020-05-12T09:58:16.373" Title="Can we use a Gaussian process to approximate the belief distribution at every instant in a POMDP?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;&lt;bayesian-optimization&gt;&lt;gaussian-process&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21094" PostTypeId="2" ParentId="21084" CreationDate="2020-05-12T02:42:53.013" Score="1" Body="&lt;p&gt;A very vague question. What's the objective?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reinforcement Learning (RL) typically uses the Markov Decision Process framework, which is a sequential decision making framework. In this framework, actions influence the state transitions. In other words, RL deals with controlling (via actions) a Markov chain. The objective in RL is figure out how to take actions in an optimal (in some sense) way!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, in the application you mentioned, the actions don't influence the state transitions and the objective is to predict states, RL is not required. It's just a regression/ time-series problem.   &lt;/p&gt;&#xA;" OwnerUserId="36970" LastActivityDate="2020-05-12T02:42:53.013" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21095" PostTypeId="2" ParentId="21085" CreationDate="2020-05-12T02:53:12.040" Score="1" Body="&lt;p&gt;Here are a few websites that help run the deep dream generator: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;a href=&quot;http://deepdream.psychic-vr-lab.com/deepdream/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://deepdream.psychic-vr-lab.com/deepdream/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;a href=&quot;https://dreamscopeapp.com/deep-dream-generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dreamscopeapp.com/deep-dream-generator&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; by Google AI explains the concept and &lt;a href=&quot;https://github.com/google/deepdream&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is the GitHub repo for the same. It has a python notebook for reproducing the results.&lt;/p&gt;&#xA;" OwnerUserId="36971" LastActivityDate="2020-05-12T02:53:12.040" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21096" PostTypeId="1" AcceptedAnswerId="22042" CreationDate="2020-05-12T06:55:31.077" Score="2" ViewCount="118" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1911.08265&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuZero&lt;/a&gt; seems to use two different methods to encode actions into planes for Atari games:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For the input action to the representation function, MuZero encodes historical actions as simple bias planes, scaled as &lt;span class=&quot;math-container&quot;&gt;$a/18$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$18$&lt;/span&gt; is the total number of valid actions in Atari.(from the appendix E of the paper)&lt;/li&gt;&#xA;&lt;li&gt;For the input action to the dynamics function, Muzero encode an action as a one-hot vector, which is tiled appropriately into planes(from the appendix F of the paper)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm not so sure about how to make of the term &quot;bias plane&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the second, my understanding is that, as an example, for action &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;, we first apply one-hot encoding, which gives us a zero vector of length &lt;span class=&quot;math-container&quot;&gt;$18$&lt;/span&gt; with one in the &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;-th position(as there are &lt;span class=&quot;math-container&quot;&gt;$18$&lt;/span&gt; actions). Then we tile it and get a zero vector of length &lt;span class=&quot;math-container&quot;&gt;$36$&lt;/span&gt;, with ones in the &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;-th and &lt;span class=&quot;math-container&quot;&gt;$23$&lt;/span&gt;-rd positions. At last, this vector is reshaped into a &lt;span class=&quot;math-container&quot;&gt;$6\times 6$&lt;/span&gt; plane as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;0, 0, 0, 0, 1, 0\\ 0, 0, 0, 0, 0, 0\\ 0, 0, 0, 0, 0, 0\\&#xA;0, 0, 0, 0, 1, 0\\ 0, 0, 0, 0, 0, 0\\ 0, 0, 0, 0, 0, 0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="8689" LastEditDate="2020-05-12T12:37:46.083" LastActivityDate="2020-06-20T16:28:32.963" Title="How's the action represented in MuZero for Atari?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21097" PostTypeId="1" AcceptedAnswerId="21099" CreationDate="2020-05-12T06:59:38.890" Score="0" ViewCount="54" Body="&lt;p&gt;I'm Spanish and I don't understand the meaning of &quot;non-held-out&quot;. I have tried Google Translator and online dictionaries like Longman but I can't find a suitable translation for this term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find these term using this &lt;a href=&quot;https://www.google.com/search?client=firefox-b-d&amp;amp;biw=1920&amp;amp;bih=901&amp;amp;ei=WUe6XtOICaqIlwTQ8Yy4BA&amp;amp;q=%22non-held-out%22%20meaning&amp;amp;oq=%22non-held-out%22%20meaning&amp;amp;gs_lcp=CgZwc3ktYWIQDFAAWABghCBoAHAAeACAAQCIAQCSAQCYAQCqAQdnd3Mtd2l6&amp;amp;sclient=psy-ab&amp;amp;ved=0ahUKEwiTmcCZ3q3pAhUqxIUKHdA4A0c4ChDh1QMICw&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Search&lt;/a&gt;, and in articles like this one:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&quot;&lt;strong&gt;computing SVD on the non-held-out data&lt;/strong&gt;&quot; from &lt;a href=&quot;https://link.springer.com/article/10.1007/s10618-019-00616-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&quot;&lt;strong&gt;The training set consists all the images and annotations containing non-held-out classes while held-out classes are masked as background during the training&lt;/strong&gt;&quot; from &lt;a href=&quot;http://bmvc2018.org/contents/papers/0255.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Few-Shot Semantic Segmentation with Prototype Learning&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&quot;&lt;strong&gt;A cross-validation procedure is that non held out data (meaning after holding out the test set) is splitted in k folds/sets&lt;/strong&gt;&quot; from &lt;a href=&quot;https://datasciencelab.its.uconn.edu/cross-validation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What is non-held-out data and held-out data or classes?&lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorUserId="4920" LastEditDate="2020-05-12T09:33:27.447" LastActivityDate="2020-05-12T09:33:27.447" Title="What are non-held-out data or non-held-out classes?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21099" PostTypeId="2" ParentId="21097" CreationDate="2020-05-12T07:47:05.713" Score="1" Body="&lt;p&gt;Held-out simply means &quot;not included&quot; particularly in the sense of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This part of the data was not included in this specific training run.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the context of all of these text &lt;strong&gt;non-held-out data/classes means the data that actually was included in a particular modeling exercise.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this excerpt from your first example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For instance, Owen and Perry (2009) show a method for &lt;strong&gt;holding out data&lt;/strong&gt;, computing SVD on the &lt;strong&gt;non-held-out data&lt;/strong&gt;, and selecting k so as to minimize the reconstruction error between the &lt;strong&gt;held-out data&lt;/strong&gt; and its SVD approximation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It actually means:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For instance, Owen and Perry (2009) show a method for &lt;strong&gt;excluding data&lt;/strong&gt;, computing SVD on the &lt;strong&gt;remaining data&lt;/strong&gt;, and selecting k so as to minimize the reconstruction error between the &lt;strong&gt;excluded data&lt;/strong&gt; and its SVD approximation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So it simply talks about a particular way of train-test-validation splitting the data.&lt;/p&gt;&#xA;" OwnerUserId="27665" LastActivityDate="2020-05-12T07:47:05.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21100" PostTypeId="1" CreationDate="2020-05-12T09:06:52.020" Score="1" ViewCount="51" Body="&lt;p&gt;I am trying to input text into my word processor to be split into sentences first and then into words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example paragraph:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;When the blow was repeated,together with an admonition in&#xA;childish sentences, he turned over upon his back, and held his paws in a peculiar manner.&#xA;&#xA;1) This a numbered sentence&#xA;2) This is the second numbered sentence&#xA;&#xA;At the same time with his ears and his eyes he offered a small prayer to the child.&#xA;&#xA;Below are the examples&#xA;- This an example of bullet point sentence&#xA;- This is also an example of bullet point sentence&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried the following codes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from nltk.tokenize import TweetTokenizer, sent_tokenize&#xA;&#xA;tokenizer_words = TweetTokenizer()&#xA;tokens_sentences = [tokenizer_words.tokenize(t) for t in &#xA;nltk.sent_tokenize(input_text)]&#xA;print(tokens_sentences)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import nltk&#xA;&#xA;sent_text = nltk.sent_tokenize(text) # this gives us a list of sentences&#xA;# now loop over each sentence and tokenize it separately&#xA;for sentence in sent_text:&#xA;    tokenized_text = nltk.word_tokenize(sentence)&#xA;    tagged = nltk.pos_tag(tokenized_text)&#xA;    print(tagged)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Output I got&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;['When', 'the', 'blow', 'was', 'repeated', ',', 'together', 'with', 'an', 'admonition', 'in', 'childish', 'sentences', ',', 'he', 'turned', 'over', 'upon', 'his', 'back', ',', 'and', 'held', 'his', 'paws', 'in', 'a', 'peculiar', 'manner', '.'], &#xA;['1', ')', 'This', 'a', 'numbered', 'sentence', '2', ')', 'This', 'is', 'the', 'second', 'numbered', 'sentence','At', 'the', 'same', 'time', 'with', 'his', 'ears', 'and', 'his', 'eyes', 'he', 'offered', 'a', 'small', 'prayer', 'to', 'the', 'child', '.']&#xA;['Below', 'are', 'the', 'examples', '-', 'This', 'an', 'example', 'of', 'bullet', 'point', 'sentence',&#xA;'-', 'This', 'also','an', 'example', 'of', 'bullet', 'point', 'sentence']&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Required Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;['When', 'the', 'blow', 'was', 'repeated', ',', 'together', 'with', 'an', 'admonition', 'in', 'childish', 'sentences', ',', 'he', 'turned', 'over', 'upon', 'his', 'back', ',', 'and', 'held', 'his', 'paws', 'in', 'a', 'peculiar', 'manner', '.'], &#xA;['1', ')', 'This', 'a', 'numbered', 'sentence']&#xA;['2', ')', 'This', 'is', 'the', 'second', 'numbered', 'sentence']&#xA;['At', 'the', 'same', 'time', 'with', 'his', 'ears', 'and', 'his', 'eyes', 'he', 'offered', 'a', 'small', 'prayer', 'to', 'the', 'child', '.']&#xA;['Below', 'are', 'the', 'examples']&#xA;['-', 'This', 'an', 'example', 'of', 'bullet', 'point', 'sentence']&#xA;['-', 'This', 'also','an', 'example', 'of', 'bullet', 'point', 'sentence']&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to split the sentence at Bullets and Numbering?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="30725" LastActivityDate="2020-05-12T09:06:52.020" Title="Sentence Segmentation for &quot;Bullets &amp; Numbering&quot;" Tags="&lt;natural-language-processing&gt;&lt;sentence-segmentation&gt;&lt;nltk&gt;&lt;spacy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21101" PostTypeId="1" AcceptedAnswerId="21104" CreationDate="2020-05-12T09:51:11.073" Score="1" ViewCount="111" Body="&lt;p&gt;Pieter Abbeel in his &lt;a href=&quot;https://www.youtube.com/watch?v=S_gwYj1Q-44&amp;amp;list=PLAdk-EyP1ND8MqJEJnSvaoUShrAWYe51U&amp;amp;index=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep rl bootcamp policy gradient lecture&lt;/a&gt; derived the gradient of the utility function with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\nabla U(\theta) \approx \hat{g} = 1/m\sum_{i=1}^m  \nabla_\theta logP(\tau^{(i)}; \theta)R(\tau^{(i)})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of rollouts, and &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; represents the trajectory of &lt;span class=&quot;math-container&quot;&gt;$s_0,u_0, ..., s_H, u_H$&lt;/span&gt; state action sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He also explains that the gradient increases the log probabilities of trajectories that have positive reward and decreases the log probabilities of trajectories with negative reward, as seen in the picture. From the equation, however, I don't see how the gradient tries to increase the probabilities of the path with positive R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the equation, what I understand is that we would want to update &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in a way that moves in the direction of &lt;span class=&quot;math-container&quot;&gt;$\nabla U(\theta)$&lt;/span&gt; so that the overall utility is maximised, and this entails computing the gradient log probability of a trajectory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, why is &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; omitted in &lt;span class=&quot;math-container&quot;&gt;$R(\tau^{(i)})$&lt;/span&gt;, since &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; depends on the policy which is dependent on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yfwre.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yfwre.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-12T09:53:55.647" LastActivityDate="2020-05-13T03:06:06.997" Title="How does the gradient increase the probabilities of the path with a positive reward in policy gradient?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21103" PostTypeId="1" CreationDate="2020-05-12T10:11:16.553" Score="1" ViewCount="35" Body="&lt;p&gt;I can't figure out what preprocessing of the image is needed before feeding it into the convolutional neural network. For example, I want to recognize circles on a 1000 by 1000 px photo. The learning process of a neural network occurs on 100 by 100 px (&lt;a href=&quot;https://www.kaggle.com/smeschke/four-shapes/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/smeschke/four-shapes/data&lt;/a&gt;). I'm having a little difficulty wrapping my head around the situation when the circle in the input image is much larger (or smaller) than 100x100 px. How then the convolution neural network determines that circle if it was learned on a dataset of a different picture's size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarity, I want to submit a 454 by 430 px image to the network input:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eAicY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eAicY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of the dataset for the learning process (100 by 100 px):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aZGJY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aZGJY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I want to recognize all the circles on the input image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iylkz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iylkz.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19347" LastActivityDate="2020-05-12T10:11:16.553" Title="What pre-processing of the image is needed before feeding it into the convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;image-recognition&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21104" PostTypeId="2" ParentId="21101" CreationDate="2020-05-12T10:14:59.933" Score="0" Body="&lt;p&gt;Think of a surface where z-axis is &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;, x and y axis are &lt;span class=&quot;math-container&quot;&gt;$\theta_{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_{2}$&lt;/span&gt; accordingly. Since you are following the gradient direction with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; vector, that means you are moving at the direction that increases the &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$R(\tau)$&lt;/span&gt; is positive, you are moving towards the uphill direction and vice-versa. More formally you will say the following:   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the policy gradient algorithm, our update step is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \theta_{new} = \theta_{old} + \alpha \nabla_{\theta}U(\theta) $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if we select a very bad trajectory the sum of all rewards (&lt;span class=&quot;math-container&quot;&gt;$R(\tau)$&lt;/span&gt;) will be negative and the following update will shift the &lt;span class=&quot;math-container&quot;&gt;$\theta_{new}$&lt;/span&gt; vector away from &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta}U(\theta)$&lt;/span&gt; vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we select a good trajectory means &lt;span class=&quot;math-container&quot;&gt;$R(\tau)$&lt;/span&gt; is positive, the update will shift &lt;span class=&quot;math-container&quot;&gt;$\theta_{new}$&lt;/span&gt; vector towards the &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta}U(\theta)$&lt;/span&gt; vector. So, it will increase the probability of selecting path with positive &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; &lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-05-12T10:22:37.547" LastActivityDate="2020-05-12T10:22:37.547" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21105" PostTypeId="1" CreationDate="2020-05-12T13:22:42.640" Score="3" ViewCount="33" Body="&lt;p&gt;How do reinforcement learning and collaborative learning overlap? What are the differences and similarities between these fields?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like the results I get via google do not make the distinction clear.&lt;/p&gt;&#xA;" OwnerUserId="36978" LastEditorUserId="2444" LastEditDate="2020-05-12T14:02:17.363" LastActivityDate="2020-05-12T14:02:17.363" Title="How do reinforcement learning and collaborative learning overlap?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;collaborative-filtering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21106" PostTypeId="1" CreationDate="2020-05-12T14:33:18.140" Score="2" ViewCount="53" Body="&lt;p&gt;The &lt;a href=&quot;https://www.youtube.com/watch?v=S_gwYj1Q-44&amp;amp;list=PLAdk-EyP1ND8MqJEJnSvaoUShrAWYe51U&amp;amp;index=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep RL bootcamp on policy gradient techniques&lt;/a&gt; gives the update equation for the policy network in A3C as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta_{i+1} = \theta_i + \alpha \times 1/m \sum_{k=1}^m\sum_{t=0}^{H-1}\nabla_{\theta}log\pi_{\theta_i}(u_t^{(k)} | s_t^{(k)})(Q(s_t^{(k)},u_t^{(k)}) - V_{\Phi_i}^\pi(s_t^{(k)}))    $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in the &lt;a href=&quot;https://arxiv.org/abs/1602.01783&quot; rel=&quot;nofollow noreferrer&quot;&gt;actual A3C paper&lt;/a&gt;, the gradient update is based on a single trajectory and there is no averaging of the gradient over &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; trajectories as defined in the video ? The simple &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html#a2c&quot; rel=&quot;nofollow noreferrer&quot;&gt;action-value actor-critic algorithm&lt;/a&gt; also does not seem to require an averaging over m trajectory. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2021-02-07T18:07:00.767" Title="Why a single trajectory can be used to update the policy network $\theta$ in A3C?" Tags="&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21107" PostTypeId="2" ParentId="21073" CreationDate="2020-05-12T14:43:40.343" Score="0" Body="&lt;pre&gt;&lt;code&gt;class ExperienceSource():&#xA;    def __init__(self, env, agent, reward_steps):&#xA;        self.env = env&#xA;        self.agent = agent&#xA;        self.reward_steps = reward_steps&#xA;&#xA;    def __iter__(self):&#xA;        histories = [deque(maxlen=self.reward_steps) for i in range(len(self.env.envs))]&#xA;        states = self.env.reset()&#xA;&#xA;        while True:&#xA;&#xA;            for idx, env in enumerate(self.env.envs):&#xA;                action = self.agent.choose_action(states[idx])&#xA;                state, reward, done, _ = env.step(action)&#xA;&#xA;                current_rewards[idx] += reward&#xA;                histories[idx].append(Experience(state, action, reward, done))&#xA;&#xA;                if len(histories[idx]) == self.reward_steps:&#xA;                    yield tuple(histories[idx])&#xA;&#xA;                if done: &#xA;                    yield tuple(histories[idx])&#xA;                    state = env.reset()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Be aware that &lt;code&gt;self.reward_steps&lt;/code&gt; is simply the value defined by &lt;code&gt;N-1&lt;/code&gt; in the following formula &lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = \sum_{i=0}^{N-1} \gamma^i r_i&#xA; + \gamma^N V(s_N)$$&lt;/span&gt; and &lt;code&gt;self.env&lt;/code&gt; is simply an instance of &lt;code&gt;GymEnvVec&lt;/code&gt; class from the question.&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="3012" LastEditDate="2020-05-18T09:58:31.163" LastActivityDate="2020-05-18T09:58:31.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21108" PostTypeId="1" CreationDate="2020-05-12T14:54:07.480" Score="1" ViewCount="48" Body="&lt;p&gt;In the paper &quot;Residual Energy-Based Models for Text Generation&quot; (&lt;a href=&quot;https://arxiv.org/abs/2004.11714&quot; rel=&quot;nofollow noreferrer&quot;&gt;arXiv&lt;/a&gt;), on page 5, they write that equation 5 is an instance of importance sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation 5 is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ P(x_t \mid x_{&amp;lt;t}) = P_{LM}(x_t \mid x_{&amp;lt;t}) \, \frac{\mathbb{E}_{x'_{&amp;gt;t} \sim P_{LM}(\cdot \mid x_{\leq t})}[\exp(-E_\theta (x_{&amp;lt;t}, \, x_t, \, x'_{&amp;gt;t}))]}{\mathbb{E}_{x'_{\geq t} \sim P_{LM}(\cdot \mid x_{\leq t-1})}[\exp(-E_\theta (x_{&amp;lt;t}, \, x'_t, \, x'_{&amp;gt;t}))]} \ \ .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to approximate sampling from a distribution from which sampling is intractable &lt;span class=&quot;math-container&quot;&gt;$P_\theta(Y \mid X) = P_{LM}(Y \mid X) \, \frac{\exp(-E_\theta (X, Y))}{Z_\theta(X)}$&lt;/span&gt;, by sampling from &lt;span class=&quot;math-container&quot;&gt;$P_{LM}$&lt;/span&gt;, from which sampling is cheaper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that they are marginalizing over &lt;span class=&quot;math-container&quot;&gt;$&amp;gt;t$&lt;/span&gt; in eq. 5, and I understand the basic idea of importance sampling to change &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{x \sim p}[f(x)]$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{x \sim q}[f(x) \frac{p(x)}{q(x)}]$&lt;/span&gt;. However, eq. 5 is not a mean or aggregate, it is a probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is happening? I don't see how eq. 5 fits in the importance sampling scheme (or a self-normalizing importance sampling scheme, &lt;a href=&quot;https://statweb.stanford.edu/~owen/mc/Ch-var-is.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;). Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="22545" LastActivityDate="2020-05-12T14:54:07.480" Title="Importance sampling eq. 5 in paper &quot;Residual Energy-based Models for Text Generation&quot;" Tags="&lt;probability&gt;&lt;text-generation&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21109" PostTypeId="1" AcceptedAnswerId="21186" CreationDate="2020-05-12T15:17:21.890" Score="3" ViewCount="185" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;One of the approaches to improving the stability of the Policy&#xA;Gradient family of methods is to use multiple environments in&#xA;parallel. The reason behind this is the fundamental problem we&#xA;discussed in Chapter 6, Deep Q-Network, when we talked about the&#xA;correlation between samples, which breaks the independent and&#xA;identically distributed (i.i.d) assumption, which is critical for&#xA;Stochastic Gradient Descent (SDG) optimization. The negative&#xA;consequence of such correlation is very high variance in gradients,&#xA;which means that our training batch contains very similar examples,&#xA;all of them pushing our network in the same direction. However, this&#xA;may be totally the wrong direction in the global sense, as all those&#xA;examples could be from one single lucky or unlucky episode. With our&#xA;Deep Q-Network (DQN), we solved the issue by storing a large amount of&#xA;previous states in the replay buffer and sampling our training batch&#xA;from this buffer. If the buffer is large enough, the random sample&#xA;from it is much better representation of the states distribution at&#xA;large. Unfortunately, this solution won't work for PG methods, at most&#xA;of them are on-policy, which means that we have to train on samples&#xA;generated by our current policy, so, remembering old transitions is&#xA;not possible anymore.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The above excerpt is from Maxim Lapan in the book &lt;em&gt;Deep Reinforcement Learning Hands-on&lt;/em&gt; page 284.&lt;/p&gt;&#xA;&lt;p&gt;How does being on-policy prevent us from using the replay buffer with the policy gradients? Can you explain to me mathematically why we can't use replay buffer with A3C for instance?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="2444" LastEditDate="2020-10-12T12:55:39.770" LastActivityDate="2020-10-12T12:55:39.770" Title="How does being on-policy prevent us from using the replay buffer with the policy gradients?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;experience-replay&gt;&lt;a3c&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21110" PostTypeId="1" CreationDate="2020-05-12T17:08:08.083" Score="3" ViewCount="59" Body="&lt;p&gt;I'm struggling with calculating accuracy&#xA;when I do cross-validation for a deep learning model.&#xA;I have two candidates for doing this.&#xA;1. Train a model with 10 different folds and get the best accuracy of them(so I get 10 best accuracies) and average them.&#xA;2. Train a model with 10 different folds and get 10 accuracy learning curves. Now, average these learning curves by calculating the mean of 10 accuracies of each epoch. So now we get one averaged accuracy learning curve and find the highest accuracy from this curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among these two candidates which one is correct??&lt;/p&gt;&#xA;" OwnerUserId="36987" LastActivityDate="2020-05-12T21:50:44.043" Title="Calculating accuracy for cross validation" Tags="&lt;deep-learning&gt;&lt;cross-validation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21111" PostTypeId="1" CreationDate="2020-05-12T19:31:25.090" Score="2" ViewCount="54" Body="&lt;p&gt;I have a neural network that connects &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; input variables to &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; output variables (qoi). By default, neural networks just give out point estimations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to plot some of the quantity of interests and produce also a prediction interval. To calculate the model uncertainty, I use the bootstrap method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma_{model}^2=\frac{1}{B-1}\sum_{i=1}^B(\hat{y}_i^b-\hat{y}_i)^2\qquad \text{with}\quad\hat{y}_i = \frac{1}{B}\sum_{b=1}^B\hat{y}_i^b$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; training datasets are resampled from original dataset with replacement.  &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i^b$&lt;/span&gt; is the preditcion of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; sample generated by the &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;th bootstrap model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understood it correctly, the model uncertainty (or epistemic uncertainty) is enough to create a confidence interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for the PI I also need the irreducible error &lt;span class=&quot;math-container&quot;&gt;$\sigma_{noise,\epsilon}^2$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$$\sigma_y^2= \sigma_{model}^2+\sigma_{noise,\epsilon}^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aleatoric uncertainty is explained in the following picture:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Nc3Ot.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nc3Ot.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a procedure to calculate this aleatoric uncertainty? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read the paper &lt;a href=&quot;https://arxiv.org/abs/1802.07167&quot; rel=&quot;nofollow noreferrer&quot;&gt;High-Quality Prediction Intervals for Deep Learning&lt;/a&gt; and watched the corresponding  &lt;a href=&quot;https://www.youtube.com/watch?v=JWrE1EKbR_c&amp;amp;t=1116s&quot; rel=&quot;nofollow noreferrer&quot;&gt;YouTube video&lt;/a&gt;. And I read the paper &lt;a href=&quot;https://ieeexplore.ieee.org/document/5966350&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Network-Based Prediction Intervals&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;I suggest the following algorithm to estimate the noise variance, but I am  not sure if this makes sense: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xKixJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xKixJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36989" LastEditorUserId="36989" LastEditDate="2020-05-19T19:55:11.310" LastActivityDate="2020-05-19T19:55:11.310" Title="How to calculate the data noise variance for a prediction interval?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;statistical-ai&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21112" PostTypeId="2" ParentId="21110" CreationDate="2020-05-12T21:13:07.043" Score="2" Body="&lt;p&gt;I guess you could train your model with 10 different folds and in each fold calculate the average accuracy. So you would have 10 values - one corresponding to each fold. And then take the mean of all of them to get the average accuracy of your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your first option doesn't seem great because you take the highest accuracy among folds. If for some reason, the variance between accuracies is high for a fold, this would bias your numbers. Taking mean or maybe median of accuracies might be more reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that help?&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-05-12T21:13:07.043" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21113" PostTypeId="2" ParentId="21032" CreationDate="2020-05-12T21:49:20.023" Score="2" Body="&lt;p&gt;I'm not familiar with the ins and outs of self-driving cars, but I imagine that the action space is not discrete. For instance, the car may want to decide what angle it needs to turn (rather than left or right). The update in Q-Learning involves taking &lt;span class=&quot;math-container&quot;&gt;$\max_aQ(s, a)$&lt;/span&gt;; this is theoretically possible for a continuous action space, but it would itself require some expensive optimisation at each time step to find the maximum. It is more likely that if RL were to be applied to self-driving cars it would be through a method that easily allows for a continuous action space, like the methods detailed in &lt;a href=&quot;https://arxiv.org/pdf/1509.02971.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://arxiv.org/pdf/2002.00444.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; survey of Deep RL for autonomous driving that you may want to look at.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-11-23T13:30:17.007" LastActivityDate="2020-11-23T13:30:17.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21114" PostTypeId="2" ParentId="21110" CreationDate="2020-05-12T21:50:44.043" Score="0" Body="&lt;p&gt;In most cases we choose to take the mean of the k accuracies of k-fold cross validation; that is each time take the one that corresponds to the fold and when every fold has been used as validation set, find the mean accuracy of them.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-05-12T21:50:44.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21115" PostTypeId="2" ParentId="21046" CreationDate="2020-05-12T23:31:53.673" Score="0" Body="&lt;p&gt;While it would certainly help if the link to the paper could also be posted, I will give it a shot based on what I understand from this picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) For any convolutional layer, there are few important things to configure, namely, the kernel (or, filter) size, number of kernels, stride. Padding is also important but it is generally defined to be zero unless mentioned otherwise. Let us consider the picture block-by-block. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first block contains 3 convolutional layers: (i) 2 conv layers with 96 filters each and the size of each filter is &lt;span class=&quot;math-container&quot;&gt;$ 3 \times 3$&lt;/span&gt; (and stride &lt;span class=&quot;math-container&quot;&gt;$=1$&lt;/span&gt; by default since it is not mentioned), and (ii) another conv layer with same configurations as above but with stride &lt;span class=&quot;math-container&quot;&gt;$=2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second block is pretty much the same as the previous except the number of filters is increased to 192 for each layer that is defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only considerable change in the third block is the introduction of &lt;span class=&quot;math-container&quot;&gt;$ 1 \times 1$&lt;/span&gt; convolutional filters instead of &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, a global average pooling layer is used (instead of a fully connected layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) As for your analysis, it is exactly the case in fully connected layers, wherein the number of units in the input layer must match the vectorized dimensions of the input data. But, in the case of CNN, we give the images directly as the input to the network. The whole idea of a CNN is to understand the spatial structure of the data by analyzing patches of the image at a time (which is what the filter size defines). This &lt;a href=&quot;https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch tutorial&lt;/a&gt; should give an idea as to how exactly the input is given to CNN.&lt;/p&gt;&#xA;" OwnerUserId="36971" LastActivityDate="2020-05-12T23:31:53.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21116" PostTypeId="1" AcceptedAnswerId="21132" CreationDate="2020-05-13T02:18:59.550" Score="4" ViewCount="95" Body="&lt;p&gt;I came across these slides &lt;a href=&quot;http://web.stanford.edu/class/cs224n/slides/cs224n-2020-lecture04-neuralnets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Natural Language Processing with Deep Learning CS224N/Ling284&lt;/a&gt;, in the context of natural language processing, which talk about the Jacobian as a generalization of the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is a lot of topic regarding this on the internet, and trust me, I've googled it. But things are getting more and more confused for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simple words, how is the Jacobian a generalization of the gradient? How can it be used in gradient descent?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-13T11:06:02.110" LastActivityDate="2020-05-13T18:08:03.590" Title="How is the Jacobian a generalisation of the gradient?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;math&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21117" PostTypeId="2" ParentId="21101" CreationDate="2020-05-13T03:06:06.997" Score="0" Body="&lt;p&gt;The grad log probability of the trajectory parameterised by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; tells us the direction &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; should move to increase the probability of that trajectory &lt;span class=&quot;math-container&quot;&gt;$P(\tau;\theta)$&lt;/span&gt; the most. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the reward is positive, &lt;span class=&quot;math-container&quot;&gt;$\nabla U(\theta)$&lt;/span&gt; tells us how much we want to increase/decrease the probability of that path &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;. The scalar quantity &lt;span class=&quot;math-container&quot;&gt;$R(\tau)$&lt;/span&gt; determines the magnitude and direction of shift. If &lt;span class=&quot;math-container&quot;&gt;$R(\tau)$&lt;/span&gt; is positive, and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is updated based on equation &lt;span class=&quot;math-container&quot;&gt;$\theta_{new}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\theta_{old} + \alpha\nabla_{\theta}U(\theta)$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; will move in the direction of it's steepest increase, leading to an increase in probability of &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$R(\tau)$&lt;/span&gt; is negative, then &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; moves in the direction of steepest decrease, leading to a decrease in probability of &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-05-13T03:06:06.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21118" PostTypeId="1" CreationDate="2020-05-13T05:53:45.773" Score="2" ViewCount="128" Body="&lt;p&gt;According to Brian Cantwell Smith&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;no calculation without representation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Therefore, computers depend on models. So, we can say that AI is limited internally by the model and externally by the environment. This problem is discussed &lt;a href=&quot;https://ai.stackexchange.com/q/17999/2444&quot;&gt;here&lt;/a&gt; in a previous question I have asked.&lt;/p&gt;&#xA;&lt;p&gt;Now, consider &lt;em&gt;Gödel's second incompleteness theorem&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a coherent theory does not demonstrate its own coherence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can we say that Gödel's second incompleteness theorem puts a limitation on artificial intelligence? How could AI bypass Gödel's second incompleteness theorem?&lt;/p&gt;&#xA;" OwnerUserId="21644" LastEditorUserId="2444" LastEditDate="2020-12-11T11:26:26.063" LastActivityDate="2020-12-11T18:12:18.183" Title="Does Gödel's second incompleteness theorem put a limitation on artificial intelligence systems?" Tags="&lt;philosophy&gt;&lt;math&gt;&lt;agi&gt;&lt;incompleteness-theorems&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21119" PostTypeId="1" CreationDate="2020-05-13T07:18:13.953" Score="3" ViewCount="41" Body="&lt;p&gt;I am trying to build a model for extractive text summarization using keras sequential layers. I am having a hard time trying to understand how to input my x data. Should it be an array of documents with each document containing an array of sentences? or should I further break it down to each sentence containing an array of words?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The y input is basically a binary classification of each sentence to check whether or not they belong to the summary of the document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first layer is an embedding layer and I'm using 100d Glove word embedding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.s: I am new to machine learning.&lt;/p&gt;&#xA;" OwnerUserId="37006" LastActivityDate="2020-10-10T10:04:55.497" Title="What should the dimension of the input be for text summarization?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;word-embedding&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21120" PostTypeId="1" AcceptedAnswerId="21123" CreationDate="2020-05-13T08:55:40.750" Score="3" ViewCount="47" Body="&lt;p&gt;When papers talk about the &quot;test time&quot;, does this mean the phase when the model is passed with new data instances to derive the accuracy of the test data set? Or is &quot;test time&quot; the phase when the model is fully trained and launched for real-world input data?&lt;/p&gt;&#xA;" OwnerUserId="32092" LastEditorUserId="2444" LastEditDate="2020-05-13T10:54:57.217" LastActivityDate="2020-05-13T11:19:34.510" Title="Is the test time the phase when the model's accuracy is calculated with test data set?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;definitions&gt;&lt;testing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21121" PostTypeId="2" ParentId="21119" CreationDate="2020-05-13T09:28:27.797" Score="1" Body="&lt;h3&gt;Briefly:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I think what you are looking for is an RNN network (Either LSTM or GRU) with the &lt;em&gt;many-to-many&lt;/em&gt; topology.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Explanation:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Clearly your input is the sentences (or to be more precise, the an embedding of your sentences, because you cannot feed the raw text to the network). then for each sentence you want to assign a value, which means for n inputs, you need n outputs. This is the many-to-many architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, you might want to check the &lt;em&gt;Bi-directional&lt;/em&gt; LSTM for your study. Not relevant to your question, but worth mentioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information, refer to &lt;a href=&quot;https://stackoverflow.com/questions/43034960/many-to-one-and-many-to-many-lstm-examples-in-keras&quot;&gt;this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2020-05-13T09:28:27.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21122" PostTypeId="2" ParentId="21068" CreationDate="2020-05-13T09:56:42.797" Score="1" Body="&lt;p&gt;About your question concerning ColorMaps: &#xA;A cv2 ColorMap is basically just a lookup table which directly maps the intensity values of the input image to a predefined RGB color. In its essences it is exactly what you did by categorizing and associating with a specific color value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the cv2 ColorMaps just have a little more detail, most of them have either 64( &quot;Rainbow&quot;, &quot;Hot&quot;, ...) or 256 (&quot;Jet&quot;, &quot;Magma&quot;, ...) steps, some like &quot;Winter&quot; have 11. They come from the GNU Octave or Matlab color palettes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to build this lookup table yourself the easiest way to achieve a high level of granularity for the individual steps is to use the HSV color space   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import cv2&#xA;&#xA;N_STEPS = 20&#xA;h = np.linspace(0, 180, N_STEPS, endpoint=False) # cv2 Hue range is [0, 179]  &#xA;s = np.ones_like(h)*255   # cv2 saturation range is from [0, 255]; adjust it to your liking&#xA;v = np.ones_like(h)*255   # cv2 value range is from [0, 255]; adjust it to your liking&#xA;hsv_colormap = np.dstack([h,s,v])&#xA;&#xA;rgb_colormap = cv2.cvtColor(np.uint8(hsv_colormap), cv2.COLOR_HSV2RGB)&#xA;&#xA;IMG_SIZE = 100&#xA;#intensities = np.random.random((IMG_SIZE,IMG_SIZE)) # from 0 to 1&#xA;intensities = np.array([np.linspace(0,1, IMG_SIZE, endpoint=False), ]*IMG_SIZE) # from 0 to 1&#xA;intensity_indices = np.uint8(N_STEPS*intensities) # map insity ranges to discretized intervals &#xA;&#xA;color_mapped_intensities = rgb_colormap[0,intensity_indices,:] &#xA;cv2.imshow('colormap', color_mapped_intensities)&#xA;&#xA;cv2.waitKey(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you can play around with the Hue values ranges to feed into the color map&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Happy color mapping to you !&lt;/p&gt;&#xA;" OwnerUserId="3132" LastActivityDate="2020-05-13T09:56:42.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21123" PostTypeId="2" ParentId="21120" CreationDate="2020-05-13T10:06:14.070" Score="2" Body="&lt;p&gt;If it is not defined otherwise, testing is the phase where the model is passed with new data instances to derive the score of the test set. It should not be confused with validation set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A validation dataset is a sample of data held back from training your model that is used to give an estimate of model skill while tuning model’s hyperparameters during training. There are a lot of methods of validation with k-fold cross-validation being one of the most popular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In k-fold cross-validation, the original training set is randomly partitioned into k equal sized subsamples. Of the k subsamples, a single subsample is retained as the validation data for validating the model, and the remaining k − 1 subsamples are used as training data. The cross-validation process is then repeated k times, with each of the k subsamples used exactly once as the validation data for the each epoch. The k results can then be averaged to produce a single estimation. The advantage of this method over repeated random sub-sampling is that all observations are used for both training and validation, and each observation is used for validation exactly once. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The validation dataset is different from the test set that is also held back from the training of the model, but is instead used to give an unbiased estimate of the skill of the final tuned model when comparing or selecting between final models.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-05-13T11:19:34.510" LastActivityDate="2020-05-13T11:19:34.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21124" PostTypeId="1" CreationDate="2020-05-13T10:32:45.627" Score="4" ViewCount="208" Body="&lt;p&gt;One way of understanding the difference between value function approaches, policy approaches and actor-critic approaches in reinforcement learning is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A critic explicitly models a value function for a policy.&lt;/li&gt;&#xA;&lt;li&gt;An actor explicitly models a policy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Value function approaches, such as Q-learning, only keep track of a value function, and the policy is directly derived from that (e.g. greedily or epsilon-greedily). Therefore, these approaches can be classified as a &quot;critic-only&quot; approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some policy search/gradient approaches, such as REINFORCE, only use a policy representation, therefore, I would argue that this approach can be classified as an &quot;actor-only&quot; approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, many policy search/gradient approaches also use value models in addition to a policy model. These algorithms are commonly referred to as &quot;actor-critic&quot; approaches (well-known ones are A2C / A3C).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keeping this taxonomy intact for model-based dynamic programming algorithms, I would argue that &lt;em&gt;value iteration&lt;/em&gt; is an actor-only approach, and &lt;em&gt;policy iteration&lt;/em&gt; is an actor-critic approach. However, not many people discuss the term actor-critic when referring to policy iteration. How come?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am not familiar with any model-based/dynamic programming like actor only approaches? Do these exist? If not, what prevents this from happening?&lt;/p&gt;&#xA;" OwnerUserId="34351" LastEditorUserId="2444" LastEditDate="2020-05-13T10:58:39.653" LastActivityDate="2020-05-13T13:41:00.513" Title="Would you categorize policy iteration as an actor-critic reinforcement learning approach?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;actor-critic-methods&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21125" PostTypeId="5" CreationDate="2020-05-13T10:39:46.693" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-13T10:39:46.693" LastActivityDate="2020-05-13T10:39:46.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21126" PostTypeId="4" CreationDate="2020-05-13T10:39:46.693" Score="0" Body="For questions related to the physical symbol system hypothesis (PSSH)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-13T10:39:46.693" LastActivityDate="2020-05-13T10:39:46.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21127" PostTypeId="2" ParentId="17790" CreationDate="2020-05-13T10:59:13.767" Score="0" Body="&lt;p&gt;I don't know if this comment will be helpful, but shouldn't the sum of the log determinant of the Jacobian (LDJ) have opposite sign in the forward and inverse pass? I'm not talking about the LDJ being sum of the positive scaling function in the forward and sum of the negative of the scaling function in the inverse, I'm talking about the LDJ itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, from the change of variable formula, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$p_z(z) \ dz = p_x(x) \ dx$$&lt;/span&gt;&#xA;differentiating w.r.t &lt;span class=&quot;math-container&quot;&gt;$\ dz$&lt;/span&gt; gives,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ p_z(z) = p_x(x) \left|\frac{dx}{dz}\right|$$&lt;/span&gt;&#xA;taking the log of both sides gives,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ ln(p_z(z)) = ln(p_x(x)) +  ln\left(\left|\frac{dx}{dz}\right|\right)$$&lt;/span&gt;&#xA;Now, if we repeat this process but differentitating w.r.t &lt;span class=&quot;math-container&quot;&gt;$\ dx$&lt;/span&gt;, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ p_x(x) = p_z(z) \left|\frac{dz}{dx}\right|$$&lt;/span&gt;&#xA;taking the log and rearranging for &lt;span class=&quot;math-container&quot;&gt;$ln(p_z(z))$&lt;/span&gt; gives,&#xA;taking the log of both sides gives,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ ln(p_z(z)) = ln(p_x(x)) -  ln\left(\left|\frac{dz}{dx}\right|\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A change of sign occurs when the Jacobian is reversed. Could this be your problem? Also, as @chris-cundy said you can have a probability density greater than 1 at a single point. Remember it is the integral over all space that cannot be greater than one, an individual point between greater than 1 is fine.&lt;/p&gt;&#xA;" OwnerUserId="36829" LastActivityDate="2020-05-13T10:59:13.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21128" PostTypeId="2" ParentId="21124" CreationDate="2020-05-13T11:34:24.517" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Keeping this taxonomy intact for model-based Dynamic programming algorithms, I would argue that value iteration is a Actor only approach, and policy iteration is a Actor-Critic approach. However, not many people discuss the term Actor-Critic when referring to Policy Iteration. How come?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both policy iteration and value iteration are value-based approaches. The policy in policy iteration is either arbitrary or derived from a value table. It is not modelled separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To count as an Actor, the policy function needs to modelled directly as a &lt;em&gt;parametric&lt;/em&gt; function of the state, not indirectly via a value assessment. You cannot use policy gradient methods to adjust an Actor's policy function unless it is possible to derive the gradient of the policy function with respect to parameters that control the relationship bewteen state and action. An Actor policy might be noted as &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s,\theta)$&lt;/span&gt; and the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are what make it possible to learn improvements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy iteration often &lt;em&gt;generates&lt;/em&gt; an explicit policy, from the current value estimates. This is not a representation that can be directly manipulated, instead it is a consequence of measuring values, and there are no parameters that can be learned. Therefore the policy seen in policy iteration cannot be used as an actor in Actor-Critic or related methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to state this is that the policy and value functions in DP are not separate enough to be considered as an actor/critic pair. Instead they are both views of the same measurement, with the value function being closer to raw measurements and policy being a mapping of the value function to policy space.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, I am not familiar with any model-based/dynamic programming like actor only approaches? Do these exist? If not, what prevents this from happening?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The main difference between model-based dynamic programming and model-free methods like Q-learning, or SARSA, is that the dynamic programming methods directly use the full distribution model (which can be expressed as &lt;span class=&quot;math-container&quot;&gt;$p(r, s'|s,a)$&lt;/span&gt;) to calculate expected bootstrapped returns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is nothing in principle stopping you substituting expected returns calculated in this way into REINFORCE or Actor-Critic methods. However, it may be computationally hard to do so - these methods are often chosen when action space is large for instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basic REINFORCE using model-based expectations would be especially hard as you need an expected value calculated over all possible trajectories from each starting state - if you are going to expand the tree of all possible results to that degree, then a simple tree search algorithm would perform better, and the algorithm then resolves to a one-off planning exhaustive tree search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actor-Critic using dynamic programming methods for the Critic should be viable, and I expect you could find examples of it being done in some situations. It may work well for some card or board games, if the combined action space and state space is not too large - it would behave a little like using Expected SARSA for the Critic component, except also run expectations over the state transition dynamics (whilst Expected SARSA only runs expectations over policy). You could vary the depth of this too, getting better estimates theoretically at the expense of extra computation (potentially a lot of extra computation if there is a large branching factor)&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-13T13:41:00.513" LastActivityDate="2020-05-13T13:41:00.513" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21129" PostTypeId="1" CreationDate="2020-05-13T13:32:01.890" Score="2" ViewCount="20" Body="&lt;p&gt;I have a quick question regarding the use of different latent spaces to represent a  distribution. Why is it that a Gaussian is usually used to represent the latent space of the generative model rather than say a hypercube? Is it because a Gaussian has most of its distribution centred around the origin rather than a uniform distribution which uniformly places points in a bounded region? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried modelling different distributions using a generative model with both a Gaussian and Uniform distribution in the latent space and the Uniform is always slightly restrictive when compared with a Gaussian. Is there a mathematical reason behind this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!  &lt;/p&gt;&#xA;" OwnerUserId="36829" LastActivityDate="2020-05-13T13:32:01.890" Title="Why do hypercube latent spaces perform poorer than Gaussian latent spaces in generative neural networks?" Tags="&lt;generative-model&gt;&lt;latent-variable&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21130" PostTypeId="2" ParentId="21106" CreationDate="2020-05-13T15:14:32.627" Score="1" Body="&lt;p&gt;I guess the gradient of the expectation of the Utility function, &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta}J(\theta)$&lt;/span&gt; in policy gradient methods where &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta}J(\theta) = E_{\tau \sim p(\tau ; \theta)}[r(\tau)\nabla_{\theta}log p(\tau;\theta)]$&lt;/span&gt; can be approximated using a single sample trajectory as shown in a &lt;a href=&quot;https://www.youtube.com/watchv=lvoHnicueoE&amp;amp;list=RDCMUCdKG2JnvPu6mY1NDXYFfN0g&amp;amp;index=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep reinforcement learning lecture by stanford&lt;/a&gt; where &lt;span class=&quot;math-container&quot;&gt;$J(\theta) \approx \sum_{t &amp;gt; 0}r(\tau)\nabla_{\theta}log\pi_{\theta}(a_t |s_t)$&lt;/span&gt; and an average over sampled trajectories is not needed to compute the gradient for &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; update in the direction of gradient.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-05-13T15:14:32.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21131" PostTypeId="1" CreationDate="2020-05-13T15:24:59.893" Score="2" ViewCount="32" Body="&lt;p&gt;In the decision tree algorithm, why do we use a weighted average of child entropies when we calculate information gain? What is wrong about using the arithmetic mean of entropies?&lt;/p&gt;&#xA;" OwnerUserId="37017" LastEditorUserId="2444" LastEditDate="2020-05-13T20:22:02.223" LastActivityDate="2020-05-13T20:24:22.047" Title="Why do we use a weighted average of child entropies when we calculate information gain?" Tags="&lt;decision-tree&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21132" PostTypeId="2" ParentId="21116" CreationDate="2020-05-13T15:38:49.637" Score="1" Body="&lt;p&gt;In short, &lt;strong&gt;the Jacobian matrix is a generalization of the gradient for vector-valued functions&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that the gradient is a vector of partial derivatives of a multi-variable function. So, consider a multi-variable function of the form &lt;span class=&quot;math-container&quot;&gt;$f: \mathcal{X}_1 \times \mathcal{X}_2 \times \dots \times \mathcal{X}_N \rightarrow \mathcal{Y}$&lt;/span&gt;. The output of this function is &lt;span class=&quot;math-container&quot;&gt;$f(x_1, x_2, \dots, x_N) = y$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x_i \in \mathcal{X}_i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y \in \mathcal{Y}$&lt;/span&gt;. And the gradient is &lt;span class=&quot;math-container&quot;&gt;$\nabla f = \left[ \frac{\partial f}{\partial x_1}, \dots, \frac{\partial f}{\partial x_N} \right] \in \mathbb{R}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Vector-valued_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;vector-valued function&lt;/a&gt; is a function whose output is a vector, i.e. a function of the form &lt;span class=&quot;math-container&quot;&gt;$f: \mathcal{X} \rightarrow \mathcal{Y}_1 \times \mathcal{Y}_2 \times \dots \times \mathcal{Y}_M$&lt;/span&gt; (I am not sure if this notation is rigorous enough!), so the output of this function is a vector &lt;span class=&quot;math-container&quot;&gt;$f(x) = [y_1, y_2, \dots, y_M]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x \in \mathcal{X}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i \in \mathcal{Y}_i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i = 1, \dots, M$&lt;/span&gt;. You can also view a vector-valued function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; as a vector of scalar-valued functions &lt;span class=&quot;math-container&quot;&gt;$[f_1, f_2, \dots, f_M]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f_i: \mathcal{X} \rightarrow \mathcal{Y}_i$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also have multi-variable vector-valued functions, i.e. functions of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f: \mathcal{X}_1 \times \mathcal{X}_2 \times \dots \times \mathcal{X}_N \rightarrow \mathcal{Y}_1 \times \mathcal{Y}_2 \times \dots \times \mathcal{Y}_M.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian matrix is an &lt;span class=&quot;math-container&quot;&gt;$N \times M$&lt;/span&gt; matrix with one partial derivative for each combination of inputs and outputs (i.e. &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to optimize a multi-variable vector-valued function, you can make use of the Jacobian, in a similar way that you make use of the gradient in the case of multi-variable functions, but, although I've seen it in the past, I can't provide now a concrete example of an application of the Jacobian (but &lt;a href=&quot;http://web.stanford.edu/class/cs224n/slides/cs224n-2020-lecture04-neuralnets.pdf#page=9&quot; rel=&quot;nofollow noreferrer&quot;&gt;the linked slides&lt;/a&gt; probably do that).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-13T18:08:03.590" LastActivityDate="2020-05-13T18:08:03.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21134" PostTypeId="2" ParentId="13544" CreationDate="2020-05-13T19:48:30.960" Score="0" Body="&lt;p&gt;That does not seem very useful to apply local minima search (as SGD) to another local minima search. Existing successful solutions combine global minima search techniques with local minima search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it's beneficial to combine simulated annealing with SGD to optimize it's learning rate and/or Nesterov momentum. In this case, you don't even need to spawn a population of SGD optimizers. But, you can also try population-based algorithms like evolution programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea to optimize optimizers is very curious, but it's rather useful to try it on global optimization algorithms.&lt;/p&gt;&#xA;" OwnerUserId="36779" LastActivityDate="2020-05-13T19:48:30.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21135" PostTypeId="1" CreationDate="2020-05-13T20:17:07.117" Score="2" ViewCount="59" Body="&lt;p&gt;I understand that L1 and L2 regularization helps to prevent overfitting. My question is then, does that mean they also help a neural network learn faster as a result?&lt;/p&gt;&#xA;&lt;p&gt;The way I'm thinking is that since the regularization techniques reduce weights (to 0 or close to 0 depending on whether it's L1 or L2) that are not important to the neural network, this would, in turn, result in &amp;quot;better values&amp;quot; for the output neurons right? Or perhaps I am completely wrong.&lt;/p&gt;&#xA;&lt;p&gt;For example, suppose I have a neural network that is to train a snake to move around a NxN environment. With regularization, the snake will learn faster in terms of survive longer in the game?&lt;/p&gt;&#xA;" OwnerUserId="33579" LastEditorUserId="2444" LastEditDate="2021-01-29T23:12:15.773" LastActivityDate="2021-03-01T00:06:40.573" Title="Does L1/L2 Regularization help reach an optimum result faster?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;regularization&gt;&lt;l2-regularization&gt;&lt;l1-regularization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21136" PostTypeId="5" CreationDate="2020-05-13T20:47:35.553" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-13T20:47:35.553" LastActivityDate="2020-05-13T20:47:35.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21137" PostTypeId="4" CreationDate="2020-05-13T20:47:35.553" Score="0" Body="For questions related to the quiescence search algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-13T20:47:35.553" LastActivityDate="2020-05-13T20:47:35.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21138" PostTypeId="5" CreationDate="2020-05-14T00:39:53.263" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-14T00:39:53.263" LastActivityDate="2020-05-14T00:39:53.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21139" PostTypeId="4" CreationDate="2020-05-14T00:39:53.263" Score="0" Body="For questions related to checkers (aka draughts) a group of strategy board games for two players which involve diagonal moves of uniform game pieces and mandatory captures by jumping over opponent pieces." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-14T00:39:53.263" LastActivityDate="2020-05-14T00:39:53.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21140" PostTypeId="1" CreationDate="2020-05-14T01:46:45.423" Score="1" ViewCount="65" Body="&lt;p&gt;I have retrospective data for a sort of &quot;behaviour policy&quot; which I will use to train a deep q network to learn a target greedy policy. After learning the Q values for this target policy, can we make the conclusion that because the Q value for the target policy, &lt;span class=&quot;math-container&quot;&gt;$Q(s,\pi_e(s))$&lt;/span&gt; is higher than the Q values for the behaviour policy, &lt;span class=&quot;math-container&quot;&gt;$Q(s,\pi_b(s))$&lt;/span&gt; at all states encountered, where &lt;span class=&quot;math-container&quot;&gt;$\pi_e$&lt;/span&gt; is the policy output by deep Q-learning and &lt;span class=&quot;math-container&quot;&gt;$\pi_b$&lt;/span&gt; is the behaviour policy, then this target policy has better performance than the behaviour policy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the proper way is to run the policy and do an empirical comparison of some sort. However, that is not possible in my case. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-14T10:40:03.043" LastActivityDate="2021-02-08T11:04:20.527" Title="Is it possible to prove that the target policy is better than the behavioural policy based on learned Q values?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21142" PostTypeId="1" AcceptedAnswerId="25979" CreationDate="2020-05-14T05:40:33.620" Score="4" ViewCount="310" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial intelligence (AI)&lt;/a&gt; refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. The term may also be applied to any machine that exhibits traits associated with a human mind such as learning and problem-solving.&lt;/p&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Swarm_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia article on swarm intelligence&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Swarm intelligence (SI) is the collective behavior of decentralized, self-organized systems, natural or artificial. The concept is employed in work on &lt;strong&gt;artificial intelligence&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The application of swarm principles to robots is called swarm robotics, while 'swarm intelligence' refers to the more general set of algorithms.&lt;/p&gt;&#xA;&lt;p&gt;SI systems consist typically of a population of simple agents or boids interacting locally with one another and with their environment. The inspiration often comes from nature, especially biological systems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These two terms seem to be related, especially in their application in computer science and software engineering. Is one a subset of another? Is one tool (SI) is used to build a system for the other(AI)? What are their differences and why are they significant?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2021-01-26T02:41:03.723" Title="What is the difference between artificial intelligence and swarm intelligence?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;swarm-intelligence&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21143" PostTypeId="2" ParentId="21140" CreationDate="2020-05-14T05:59:24.567" Score="-1" Body="&lt;p&gt;No, mainly because these are all stochastic approximations and may not represent the true values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost nothing good can be said about NN approximations to value and Q functions(at least according to a professor I have had). &lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-14T05:59:24.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21144" PostTypeId="2" ParentId="21142" CreationDate="2020-05-14T06:38:16.943" Score="2" Body="&lt;p&gt;Well, one of the simpler definitions for &lt;code&gt;SI&lt;/code&gt; sounds like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The emergent collective intelligence of groups of simple agents.”&#xA;  (Bonabeau et al, 1999)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, in order to get to the &lt;code&gt;SI&lt;/code&gt; you have to use some kind of algorithms/AI to get simple intelligent agents. It's just cooperative intelligence, or cooperative &lt;code&gt;AI&lt;/code&gt; if you wish. &lt;code&gt;SI&lt;/code&gt; just uses today's &lt;code&gt;AI/ML&lt;/code&gt; techniques to build the swarm, in same manners as reinforcement learning uses &lt;code&gt;AI/ML&lt;/code&gt; techniques to make agents that can behave reasonably in large spaces by approximating value functions &lt;code&gt;V(S)&lt;/code&gt; and policies &lt;code&gt;pi(S)&lt;/code&gt;. I hope this helps a little.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;code&gt;AI/ML&lt;/code&gt; is kinda of a tool plugged in &lt;code&gt;SI&lt;/code&gt;, as &lt;code&gt;SI&lt;/code&gt; is field with it's own algorithm definitions and theory.&lt;/p&gt;&#xA;" OwnerUserId="18676" LastActivityDate="2020-05-14T06:38:16.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21145" PostTypeId="1" CreationDate="2020-05-14T07:55:24.707" Score="2" ViewCount="47" Body="&lt;p&gt;I have a dataset with &lt;strong&gt;MRI&lt;/strong&gt; of patients with a specific disease that affects the &lt;strong&gt;brain&lt;/strong&gt; and another dataset with &lt;strong&gt;MRI&lt;/strong&gt; of healthy patients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to create a &lt;strong&gt;classifier&lt;/strong&gt; (using neural networks) to classify if the MRI of a new patient shows the presence of the illness or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, I &lt;strong&gt;extracted&lt;/strong&gt; the brain from all the MRIs (the so-called &lt;strong&gt;skull stripping&lt;/strong&gt;) using &lt;a href=&quot;https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/BET&quot; rel=&quot;nofollow noreferrer&quot;&gt;BET&lt;/a&gt; tool found in FSL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have three questions for you&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;As the input to the &lt;strong&gt;training&lt;/strong&gt; phase, I want to give the &lt;strong&gt;whole&lt;/strong&gt; extracted brains (possibly in the nii format). What kind of &lt;strong&gt;preprocessing&lt;/strong&gt; steps do I need to apply once I've &lt;strong&gt;extracted&lt;/strong&gt; the brains (before passing it to the classifier)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you know any better tool for &lt;strong&gt;skull stripping&lt;/strong&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you know a tool (or library) that takes as input a nii files and allows me to create a classifier that uses neural networks?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36363" LastEditorUserId="2444" LastEditDate="2020-05-14T10:28:33.493" LastActivityDate="2020-05-14T12:18:19.260" Title="Are my steps correct for a proper classification of a sick brain?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;brain&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21146" PostTypeId="2" ParentId="8607" CreationDate="2020-05-14T08:06:51.863" Score="0" Body="&lt;p&gt;Your network without regularization does not appear to be over fitting but rather it appears to be converging to a minima. I am actually a bit surprised it is doing as well as it is given that your data set is small. So You don't need regularization. If you want to improve the accuracy you might try using an adjustable learning rate. The Keras call back ReduceLROnPlateau can be used for this. Documentation is &lt;a href=&quot;https://keras.io/api/callbacks/reduce_lr_on_plateau/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Also use the callback ModelCheckpoint to save the model with the lowest validation loss. Documentation is &lt;a href=&quot;https://keras.io/api/callbacks/model_checkpoint/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It would help a lot if you posted your model code. I have found if you do encounter over fitting dropout works more effectively than regularization.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-05-14T08:06:51.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21147" PostTypeId="1" CreationDate="2020-05-14T08:21:24.000" Score="1" ViewCount="54" Body="&lt;p&gt;So far I have seen TD3 and DDPG benchmarks on Pybullet environments, but I am looking for SAC benchmarks on Pybullet too, anyone can help?&lt;/p&gt;&#xA;" OwnerUserId="36603" LastActivityDate="2020-05-14T08:21:24.000" Title="Benchmarking SAC on Pybullet" Tags="&lt;reinforcement-learning&gt;&lt;pytorch&gt;&lt;ddpg&gt;&lt;benchmarks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21148" PostTypeId="1" CreationDate="2020-05-14T08:24:28.973" Score="2" ViewCount="24" Body="&lt;p&gt;I have, like you see, just a general question about the combination of fuzziness and neural networks. I understood it as follows &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Fuzzy neural networks as a hybrid system: the neural network helps me to find the optimal parameters related to the fuzzy system, for example, the rules or the membership function &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adaptive neural fuzzy inference systems (ANFIS): the NN helps me to find the optimal parameters related to the fuzzy inference system. What are some examples here?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I cannot intuitively grasp the difference between these two.&lt;/p&gt;&#xA;" OwnerUserId="37039" LastEditorUserId="2444" LastEditDate="2020-05-14T10:07:18.260" LastActivityDate="2020-05-14T10:07:18.260" Title="What is the difference between fuzzy neural networks and adaptive neuro fuzzy inference systems?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21149" PostTypeId="1" AcceptedAnswerId="21151" CreationDate="2020-05-14T09:01:12.057" Score="3" ViewCount="97" Body="&lt;p&gt;I have implemented a simple Q-learning algorithm to minimize a cost function by setting the reward to the inverse of the cost of the action taken by the agent. The algorithm converges nicely, but there is some difference I get in the global cost convergence for different orders of the reward function. If I use the reward function as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{reward} = \frac{1}{(\text{cost}+1)^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;the algorithm converges better (lower global cost, which is the objective of the process) than when I use the reward as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{reward} = \frac{1}{(\text{cost}+1)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What could be the explanation for this difference? Is it the issue of optimism in the face of uncertainty?&lt;/p&gt;&#xA;" OwnerUserId="27231" LastEditorUserId="2444" LastEditDate="2020-10-08T11:46:43.007" LastActivityDate="2020-10-08T11:46:43.007" Title="Why is the reward function $\text{reward} = 1/{(\text{cost}+1)^2}$ better than $\text{reward} =1/(\text{cost}+1)$?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;rewards&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21150" PostTypeId="2" ParentId="7390" CreationDate="2020-05-14T09:11:50.747" Score="0" Body="&lt;p&gt;Though the word &quot;Advantage&quot; in the actor-critic realm has been used to refer to the difference between the state value and the state action value, A2C brings in the ideas of A3C. In A3C, several worker networks interact with different copies of the environment (asynchronous learning) and update a master network after a set if steps. This was meant to solve instability issues associated with both temporal difference update method and correlations within neural network generated prediction and target values. However it was noticed by OpenAI that there was no need of the asynchrony, i.e. there was no practical benefit of having different worker networks. Instead, they had the same copy of the network that interacted with different copies of the environment (one works from the beginning, another working backwards from the end) and they update at once without the master lagging behind like in A3C. The removal of asynchrony gave rise to A2C.&lt;/p&gt;&#xA;" OwnerUserId="27231" LastActivityDate="2020-05-14T09:11:50.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21151" PostTypeId="2" ParentId="21149" CreationDate="2020-05-14T09:21:46.807" Score="3" Body="&lt;p&gt;Reinforcement learning (RL) control maximises the expected sum of rewards. If you change the reward metric, it will change what counts as optimal. Your reward functions are not the same, so will in some cases change the priority of solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple example, consider a choice between trajectories with costs A(0,4,4,4) and B(1,1,1,1). In the original cost formula B is clearly better, with 4 total cost compared with A's cost of 12 - A just has one low cost at the beginning, which I put in deliberately as it exposes the problem with your conversion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your two reward formulae:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reward = 1/(cost+1)**2. &#xA;  A: 1.0 + 0.04 + 0.04 + 0.04 = 1.12&#xA;  B: 0.25 + 0.25 + 0.25 + 0.25 = 1.0&#xA;&#xA;reward = 1/(cost+1).&#xA;  A: 1.0 + 0.2 + 0.2 + 0.2 = 1.6&#xA;  B: 0.5 + 0.5 + 0.5 + 0.5 = 2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So with this example (numbers carefully chosen), maximising the total reward favours A for sum of inverse squares but B for sum of inverses, whilst B should be the clear preference for minimising sum of costs. It is possible to find examples for both of your formulae where the best sum of rewards does not give you the lowest cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, if you truly want to minimise total cost, then your conversion to rewards should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reward = -cost&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anything else is technically changing the nature of the problem, and will result in different solutions that may not be optimal with respect to your initial goal.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-14T09:48:11.583" LastActivityDate="2020-05-14T09:48:11.583" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21152" PostTypeId="1" CreationDate="2020-05-14T09:37:29.593" Score="2" ViewCount="40" Body="&lt;p&gt;I am actually working with the iris dataset from sklearn and try to understand the ANFIS-Package for python. But that does not really matter! I have a more general question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;During thinking about &lt;em&gt;adaptive neuro-fuzzy inference system&lt;/em&gt; (ANFIS), a general question came into my mind. I don't really understand: in general, why is ANFIS necessary? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for example, if I want to predict classes for this iris dataset, I also can use a supervised learning method or a neural network and I get the result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In ANFIS, I do nothing other than splitting the input attributes into linguistic terms and give membership functions to it. At the end of the day, I will receive &quot;predictions&quot; for the input values, which are classes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But - with the ANFIS-Package in Python - I cannot see, if my membership function has changed during the learning time or what rules the network constructed. So, I cannot really see why this is useful. Maybe it is just because I am usually using the iris dataset for supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="37039" LastEditorUserId="2444" LastEditDate="2020-05-14T10:08:08.067" LastActivityDate="2020-05-14T10:08:08.067" Title="Why is ANFIS important in general?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21153" PostTypeId="1" CreationDate="2020-05-14T10:22:22.107" Score="2" ViewCount="32" Body="&lt;p&gt;I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the model to output an action. That action would be a stepper motor, that either moves to the right or left. This process would be repeated until a given goal/position is reached. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't go to the lab at the moment, so I wrote a virtual environment and let the agent live in that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying a neural network with the cross-entropy function. For small environments, this works fine. However, when I increase the size of the environment, the computation becomes really really slow and the model needs a lot of data input until it starts to learn. Also, it only learns offline. But what I would rather want is a model that learns online and only takes a few tries until it understands the underlying pattern. That isn't a problem for the virtual environment, since I can easily get thousands of data samples. But in the real environment, it would take ages this way. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there an online reinforcement algorithm that could help me out (instead of training the neural network with the cross-entropy loss function)? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37042" LastEditorUserId="1671" LastEditDate="2020-05-14T22:02:52.427" LastActivityDate="2020-05-14T22:02:52.427" Title="Is there an online RL algorithm that receives as input a camera frame and produces an action as output?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;reference-request&gt;&lt;online-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21154" PostTypeId="2" ParentId="21145" CreationDate="2020-05-14T11:44:54.723" Score="2" Body="&lt;p&gt;It looks like everything you want is available with the &lt;a href=&quot;https://dltk.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Toolkit (DLTK) for Medical Imaging&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a blog: &lt;a href=&quot;https://blog.tensorflow.org/2018/07/an-introduction-to-biomedical-image-analysis-tensorflow-dltk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Biomedical Image Analysis with TensorFlow and DLTK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a DataCamp course that walks you through most of the process but instead of a classifier they use deep learning to reconstruct brain images.  They provide a link to their MNIST classifier example which should be easy to adapt for your purpose.  See: &lt;a href=&quot;https://www.datacamp.com/community/tutorials/reconstructing-brain-images-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reconstructing Brain MRI Images Using Deep Learning (Convolutional Autoencoder)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ResearchGate has a thread that may help:&#xA;&lt;a href=&quot;https://www.researchgate.net/post/What_is_the_appropriate_way_to_use_Nifti_files_in_deep_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is the appropriate way to use Nifti files in deep learning?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-05-14T12:18:19.260" LastActivityDate="2020-05-14T12:18:19.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21155" PostTypeId="1" CreationDate="2020-05-14T12:26:49.423" Score="5" ViewCount="595" Body="&lt;p&gt;What is the difference between the term 'prediction'/value estimation in RL as compared to the 'control' problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there scenarios in RL where the problem cannot be distinctly categorised into the aforementioned problems and is a mixture of the problems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples where the problem cannot be easily categorised into one of the aforementioned problems would be nice.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="2444" LastEditDate="2020-05-14T13:07:46.217" LastActivityDate="2020-09-04T22:51:17.470" Title="What is the difference between 'prediction' and 'control' problem in the context of Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21156" PostTypeId="2" ParentId="20995" CreationDate="2020-05-14T13:41:02.223" Score="1" Body="&lt;p&gt;In the DRL nanodegree in Udacity, the instructor says it is possible to combine on- and off-policy learning and suggests the following paper where this has been done: &lt;a href=&quot;https://arxiv.org/pdf/1611.02247.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic&lt;/a&gt; (ICLR 2017). &#xA;Citing the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The core idea is to use the first-order Taylor expansion of the critic&#xA;  as a control variate, resulting in an analytical gradient term through&#xA;  the critic and a Monte Carlo policy gradient term consisting of the&#xA;  residuals in advantage approximations. The method helps unify policy&#xA;  gradient and actor-critic methods: it can be seen as using the&#xA;  off-policy critic to reduce variance in policy gradient or using&#xA;  on-policy Monte Carlo returns to correct for bias in the critic&#xA;  gradient.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The authors provide an open source implementation of it in &lt;a href=&quot;https://github.com/shaneshixiang/rllabplusplus&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/shaneshixiang/rllabplusplus&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a follow-up paper by the same authors also addressing this problem: Interpolated Policy Gradient: &lt;a href=&quot;https://papers.nips.cc/paper/6974-interpolated-policy-gradient-merging-on-policy-and-off-policy-gradient-estimation-for-deep-reinforcement-learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning&lt;/a&gt; (NIPS 2017).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Related Work section in both papers might be also worth looking at.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-05-14T13:41:02.223" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21157" PostTypeId="2" ParentId="21155" CreationDate="2020-05-14T13:47:17.480" Score="6" Body="&lt;p&gt;&lt;strong&gt;Prediction&lt;/strong&gt; is the problem of predicting any feature of the environment. In reinforcement learning, the typical feature is the reward or return, but this doesn't have to be always the case. See &lt;a href=&quot;https://www.researchgate.net/publication/51969642_Multi-timescale_Nexting_in_a_Reinforcement_Learning_Robot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-timescale nexting in a reinforcement learning robot&lt;/a&gt; (2011) by Joseph Modayil et al.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Control&lt;/strong&gt; is the problem of estimating a policy. Clearly, the term &lt;em&gt;control&lt;/em&gt; is related to control theory. In fact, the term &lt;em&gt;control&lt;/em&gt; is often used as a synonym for &lt;em&gt;action&lt;/em&gt;. See &lt;a href=&quot;https://ai.stackexchange.com/q/13119/2444&quot;&gt;Is there any difference between a control and an action in reinforcement learning?&lt;/a&gt;. Similarly, the term &lt;em&gt;controller&lt;/em&gt; is also used as a synonym for &lt;em&gt;agent&lt;/em&gt;. For example, in the paper &lt;a href=&quot;https://arxiv.org/pdf/1705.02670.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metacontrol for Adaptive Imagination-Based Optimization&lt;/a&gt; (2017) by Jessica B. Hamrick et al. the term meta-controller is used to refer to an agent. A &lt;em&gt;controlled system&lt;/em&gt; can also refer to the &lt;em&gt;environment&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=364&quot; rel=&quot;nofollow noreferrer&quot;&gt;Section 14.1&lt;/a&gt; of the book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning: an introduction&lt;/a&gt; (2nd edition) provides more details about the distinction between prediction and control and how this distinction is related to psychological concepts.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-04T22:51:17.470" LastActivityDate="2020-09-04T22:51:17.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21158" PostTypeId="2" ParentId="20968" CreationDate="2020-05-14T14:19:02.627" Score="2" Body="&lt;p&gt;In the context of RL, for a policy to be parameterised it typically means we explicitly model the policy and is common in policy gradient methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider value based methods such as Q-learning where our policy is usually something like &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy where we choose our action using the following policy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\pi(a|s) = \left\{ \begin{array}{ll}&#xA;\arg \max_a Q(s,a) &amp;amp; \text{with probability } 1-\epsilon\;; \\ \text{random action} &amp;amp; \text{with probability } \epsilon\;.&#xA;\end{array}\right.&#xA;\end{align}&lt;/span&gt;&#xA;Here we have parameterised the policy with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; but the learning is done by learning the Q-functions. When we parameterise a policy we will explicitly model &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; by the following: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s|a,\boldsymbol{\theta}) = \mathbb{P}(A_t = a | S_t=s, \boldsymbol{\theta}_t = \boldsymbol{\theta})\;.$$&lt;/span&gt;&#xA;Learning is now done by learning the parameter &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; that &lt;em&gt;maximise&lt;/em&gt; some performance measure &lt;span class=&quot;math-container&quot;&gt;$J(\boldsymbol{\theta})$&lt;/span&gt; by doing approximate gradient ascent updates of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t + \alpha \hat{\Delta J(\boldsymbol{\theta}_t)}. $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, as per the Sutton and Barto textbook, &lt;span class=&quot;math-container&quot;&gt;$\hat{\Delta J(\boldsymbol{\theta}_t)}$&lt;/span&gt; is a noisy, stochastic estimate of &lt;span class=&quot;math-container&quot;&gt;$\Delta J(\boldsymbol{\theta}_t)$&lt;/span&gt; where the former approximates the latter in expectation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy can be parameterised in any way as long as it is differentiable with respect to the parameters. Commonly in Deep RL the policy is parameterised as a neural network so &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; would be the weights of the network.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-14T14:19:02.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21159" PostTypeId="1" CreationDate="2020-05-14T14:19:56.070" Score="21" ViewCount="3869" Body="&lt;p&gt;I'm a novice researcher, and as I started to read papers in the area of deep learning I noticed that the implementation is normally not added and is needed to be searched elsewhere, and my question is how come that's the case? The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation? Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?&lt;/p&gt;&#xA;" OwnerUserId="36083" LastEditorUserId="34157" LastEditDate="2020-05-17T22:42:50.407" LastActivityDate="2020-07-08T23:48:35.610" Title="Why do most deep learning papers not include an implementation?" Tags="&lt;deep-learning&gt;&lt;research&gt;&lt;papers&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21160" PostTypeId="1" CreationDate="2020-05-14T14:27:09.160" Score="1" ViewCount="27" Body="&lt;p&gt;I want to feed the Neural Net with examples of three images: 2 input and one output-result. Net shold learn by those examples and then generate the appropriate &quot;mixes&quot; for new pairs of pics. Is there any free open source solution for this task?&lt;/p&gt;&#xA;" OwnerUserId="37049" LastActivityDate="2020-05-14T14:27:09.160" Title="Solution for 'mixing' images" Tags="&lt;image-generation&gt;&lt;open-source&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21161" PostTypeId="2" ParentId="21159" CreationDate="2020-05-14T14:44:06.637" Score="15" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Some papers and authors actually provide a link to their own implementation, but most of the papers (that I have read) don't provide it, although some third-party implementations may already be available on Github (or other code-hosting sites) when you are reading the paper. There may be different reasons why the author(s) of a paper don't provide a reference implementation&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They use some closed-source software or maybe it makes use of other resources that cannot be shared&lt;/li&gt;&#xA;&lt;li&gt;Their implementation is a mess and, so, from a pedagogical point of view, it's quite useless&lt;/li&gt;&#xA;&lt;li&gt;This may encourage other people to try to reproduce their results with different implementations, so it may indirectly encourage people to do research on the same topic (but maybe not providing a reference implementation could actually have the opposite effect!)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Actually, that's a good question that only researchers that have already published something or some reviewers can say something about it. I would also like to know their take on this!&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless, from some reviews of papers I have read (e.g. on &lt;a href=&quot;https://openreview.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenReview&lt;/a&gt;), in most cases, the reviewers are interested in the consistency of the results, the novelty of the work, the clarity and structure of the paper, etc. I think that, in most cases, they probably trust the provided results, also because, often, for reproducibility, researchers are expected to describe their models and parameters in detail, provide plots, etc., but I don't exclude that there are cases of people that try to fool the reviewers. For example, watch &lt;a href=&quot;https://www.youtube.com/watch?v=bmZyJ2Dt8dw&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; where Phil Tabor comments on ridiculous attempts to fool people and plagiarism by Siraj Raval.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-08T23:48:35.610" LastActivityDate="2020-07-08T23:48:35.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21162" PostTypeId="1" AcceptedAnswerId="21265" CreationDate="2020-05-14T14:50:59.230" Score="5" ViewCount="76" Body="&lt;p&gt;If I replicated the results of a machine learning paper, is there a website that I can publish my code to that collects informal replication attempts?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://paperswithcode.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://paperswithcode.com/&lt;/a&gt; allows anyone to publish implementations, but I am looking for something similar that allows anyone to publish implementations along with replicated experimental results. Does such a website exist?&lt;/p&gt;&#xA;" OwnerUserId="37050" LastEditorUserId="1671" LastEditDate="2020-05-14T22:01:41.123" LastActivityDate="2020-05-18T15:33:18.070" Title="Where to publish replications of machine learning papers?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;research&gt;&lt;resource-request&gt;&lt;academia&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="0" ClosedDate="2021-02-03T16:41:40.540" ContentLicense="CC BY-SA 4.0" />
  <row Id="21163" PostTypeId="2" ParentId="21159" CreationDate="2020-05-14T15:02:21.677" Score="5" Body="&lt;p&gt;Someone can argue to some human adequate reasons, but there is a bad trend of falsified results in deep learning research papers that propose some nowel solutions or even update state-of-the-art model performance. And that's not just a few papers that lie, it's a large portion of them. And the reason for that is even more sad - most of so-called deep learning research papers just describing some empirical experiments, without any math and proving any theorem, and so it's easy to cheat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So objectively, if the only thing you propose in your paper is your empirical results - you must confirm them true by sharing source code. Otherwise, your work will be ignored.&lt;/p&gt;&#xA;" OwnerUserId="36779" LastEditorUserId="36779" LastEditDate="2020-05-14T15:08:03.793" LastActivityDate="2020-05-14T15:08:03.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21165" PostTypeId="1" CreationDate="2020-05-14T17:24:22.480" Score="1" ViewCount="20" Body="&lt;p&gt;I have read several papers about super-resolution with CNNs, where a low-resolution image is reconstructed to a high-resolution image.&#xA;What I don't understand is, why it is necessary to interpolate the low-resolution image at the beginning to a size that matches the high-resolution image target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the idea about that? If I have an image to image transformation, what are the benefits in a Neuronal network to have the same input size as the same output size? &lt;/p&gt;&#xA;" OwnerUserId="35615" LastActivityDate="2020-05-14T17:24:22.480" Title="Super-Resolution with Convolutional Neuronal Networks, why interpolation at the beginning?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21167" PostTypeId="2" ParentId="21159" CreationDate="2020-05-14T18:35:29.643" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The first reason described in &lt;a href=&quot;https://ai.stackexchange.com/a/21161/1641&quot;&gt;nbro's answer&lt;/a&gt; can definitely be an important one; authors may have implemented their software using code that they can't share. There's a lot of research coming out of companies (large and small), and they may use all sorts of proprietary libraries that were built in the company and cannot be distributed outside.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As described in &lt;a href=&quot;https://academia.stackexchange.com/a/10270/69376&quot;&gt;this answer&lt;/a&gt;, sometimes researchers prefer to keep the code to themselves because it may give them an &quot;advantage&quot; over other researchers fot future work / follow-up research in the same area. I'm not saying that I believe this is a &lt;em&gt;good&lt;/em&gt; reason, it definitely doesn't sound like it's good for the overall benefit of science... but it may be understandable in a &quot;publish or perish&quot; world where there's quite a bit of pressure to keep publishing frequently if you want your academic career to survive.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also described in more detail in the answer I linked above, research code is often messy, and not pretty. Nbro also mentioned this, though I personally don't feel like the rationale is &quot;it's too messy to be useful&quot;, and more often it's more along the lines of &quot;it's so messy that I'm too embarassed to share it&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some researchers, especially in larger teams, do not just work on a single paper at a time. They may have multiple papers they're working on simultaneously, and if they're closely related it can often be convenient to have them all in a single codebase. This is especially the case with longer review times; in the time between submission of a paper -- where it and anything related to it, such as source code, must remain private -- and an acceptance notification, there's plenty of time to start working on a next project. If the code for the previous project is mixed in with the code for the next project, and you can't / don't want to publish the code for the next project either yet... it may be easier to just not release anything.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In some cases, authors may feel it is &quot;dangerous&quot; to release their source code (or trained models). This is probably relatively uncommon, but can happen. Consider &lt;a href=&quot;https://openai.com/blog/better-language-models/&quot; rel=&quot;noreferrer&quot;&gt;the situation surrounding OpenAI's GPT-2 language model&lt;/a&gt;, for example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Not directly a response to your question, but it may also be useful to keep in mind that sometimes not all authors of a paper may agree on whether or not to open-source it. Legally, I suppose that usually all the authors (or all contributors to the source code) would be copyright holders, and it can only be released if they &lt;strong&gt;all&lt;/strong&gt; agree to release it. So if one of them feels (based on any of the reasons listed above, or maybe other reasons) that it shouldn't be released, it won't. In practice, I suppose that it would often primarily be the call of the more senior authors on a paper / principal investigators / supervisors.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Personally I wouldn't be concerned as much about forging results as just... &quot;accidental&quot; false positives. Yes, it's possible and it will happen. But the pay-off of successfully forging false results and getting a paper published seem REALLY low compared to the risk of your academic career ending if it gets out. If you really have to forge your results just to get your paper accepted, and it has zero other meaningful contributions (no &quot;unforgeable&quot; contributions like theoretical results or really new and useful insights).. it's unlikely to become a really impactful paper, a widely-cited one. The really highly impactful empirical papers only become highly impactful because people will immediately try to re-implement and reproduce it anyway, and if that turns out to be impossible, it will turn into a dead end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I'm not saying it can't be important to share source code. Especially in deep learning, and especially in deep reinforcement learning, &lt;a href=&quot;https://arxiv.org/abs/1709.06560&quot; rel=&quot;noreferrer&quot;&gt;it has indeed been shown that tiny implementation details can be massively important to empirical performance&lt;/a&gt;, and these tiny implementation details are rarely all available in papers. There has certainly been a push towards encouraging the publication of source code, and it &lt;strong&gt;is&lt;/strong&gt; important -- but unfortunately it's not always a black-and-white story, and there &lt;strong&gt;can&lt;/strong&gt; sometimes also be good reasons that make it difficult/impossible to do so. If it's good research, I'd personally still rather have it without source code, than not have it at all.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-05-14T18:35:29.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21168" PostTypeId="2" ParentId="21155" CreationDate="2020-05-14T19:42:42.033" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/21157/1641&quot;&gt;Nbro's answer&lt;/a&gt; already addresses the basic definitions, so I won't repeat that. Instead I'll try to elaborate a bit on the other parts of the question.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there scenarios in RL where the problem cannot be distinctly categorised into the aforementioned problems and is a mixture of the problems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure about cases where the &quot;problem&quot; can't be distinctly categories... but often, when we're actually interested in control as a problem, we still also actually deal with the prediction problem as a part of our training algorithm. Think of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, Sarsa, and all kinds of other algorithms related to the idea of &quot;Generalized Policy Iteration&quot;. Many of them work (roughly) like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialise (somehow, possibly randomly) a value function&lt;/li&gt;&#xA;&lt;li&gt;Express a policy in terms of that value function (greedy, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, etc.)&lt;/li&gt;&#xA;&lt;li&gt;Generate experience using that policy&lt;/li&gt;&#xA;&lt;li&gt;Train the value function to be more accurate for that policy (&lt;strong&gt;prediction problem&lt;/strong&gt; here)&lt;/li&gt;&#xA;&lt;li&gt;Go back to step 2 (&lt;strong&gt;control problem&lt;/strong&gt; here)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You could view these techniques in this way, as handling both of the problems at the same time, but there's also something to be said for the argument that they're really mostly just tackling the prediction problem. That's where all the &quot;interesting&quot; learning happens. The solution to the control problem is directly derived from the solution to the prediction problem in a single, small step. There are different algorithms, such as Policy Gradient methods, that directly aim to address the control problem instead.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;An interesting (in my opinion :)) tangent is that in some problems, one of these problems may be &lt;strong&gt;significantly&lt;/strong&gt; easier than the other, and this can be important to inform your selection of algorithm. For example, suppose you have a very long &quot;road&quot; where you can only move to the left or the right, you start on the left, and the goal is all the way to the right. In this problem, a solution to the control problem is trivial to express; just always go right. For the prediction problem, you need something much more powerful to be able to express all the predictions of values in all possible states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other problems, it may be much easier to quickly get an estimate of the value, but much more complicated to actually express how to obtain that value. For example, in StarCraft, if you have a much larger army, it is easy to predict that you will win. But you will still need to execute some very specific, long sequences of actions to achieve that goal.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-05-14T19:42:42.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21169" PostTypeId="1" CreationDate="2020-05-14T19:54:57.840" Score="2" ViewCount="122" Body="&lt;p&gt;Q-learning is a temporal-difference method and Monte Carlo tree search is a Monte Carlo method. In what category is MiniMax?&lt;/p&gt;&#xA;" OwnerUserId="27629" LastEditorUserId="2444" LastEditDate="2020-05-14T23:00:38.540" LastActivityDate="2020-05-14T23:00:38.540" Title="In what RL algorithm category is MiniMax?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;minimax&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21170" PostTypeId="1" AcceptedAnswerId="21376" CreationDate="2020-05-14T20:56:39.477" Score="4" ViewCount="98" Body="&lt;p&gt;I have a dataset with missing values, I would like to use machine learning methods to fill.&#xA;In more detail, there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&#xA;individuals, for which up to 10 properties are provided, all numerical. The fact is, there are no individuals for which all properties are given. The first rows (each row contains data for a given individual) do look as the following &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{bmatrix}&#xA;    1 &amp;amp; NA &amp;amp; 3.6 &amp;amp; 12.1 &amp;amp; NA \\&#xA;1.2 &amp;amp; NA &amp;amp; NA &amp;amp; 4 &amp;amp; NA \\&#xA;    NA &amp;amp; 4 &amp;amp; 5 &amp;amp; NA &amp;amp; 7&#xA;  \end{bmatrix}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What methods could be applicable in general? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some basic experience in classifiers and Random Forests. Modulo the obvious difference that this is not a classifying problem, what I struggle most with is that the same variable (described in the e.g &#xA;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-th column) is both an input and an output. Say I want to predict the value &lt;span class=&quot;math-container&quot;&gt;$A_{2,3}$&lt;/span&gt; in the dataset above. In this case, all the values in the third column could be used as input, excluded of course &lt;span class=&quot;math-container&quot;&gt;$A_{2,3}$&lt;/span&gt;&#xA;itself, which would be an output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be different than the more conventional set-up of predicting a property, given a set of other properties (e.g, predict income given education, work sector, seniority, etc.). In this case, sometimes the income is to be predicted, sometimes used for predicting another variable.&#xA;I am aware of methods which, given a vector &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt;, could approximate a function &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; and predict responses &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt; with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Y_i = F(X_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the scenario I described though, it looks like some implicit function &lt;span class=&quot;math-container&quot;&gt;$\Phi$&lt;/span&gt; is to be found, a function of all the variables &lt;span class=&quot;math-container&quot;&gt;$Z_i$&lt;/span&gt; (columns in the dataset above)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Phi (Z_i) = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What methods could handle this aspect? I understand the question is probably too general, but I could not find much and could do with a starting point. I would be already content with some hints for my further reading, but anything more would be gratefully welcomed, thanks.&lt;/p&gt;&#xA;" OwnerUserId="37059" LastEditorUserId="2444" LastEditDate="2020-05-15T10:23:21.643" LastActivityDate="2020-05-21T11:44:40.437" Title="How to fill missing values in a dataset where some properties can be inputs and outputs?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21171" PostTypeId="2" ParentId="21169" CreationDate="2020-05-14T21:19:50.093" Score="3" Body="&lt;p&gt;I think you are looking at it from the wrong direction, min-max is just a planning algorithm, decision strategy, in the sense that you are describing other algorithms/methods it does not have a category. For example, you have negamax algorithm which is in a sense the same thing the Monte Carlo Search Tree is to Monte Carlo. Min-max category is game theory really.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you should be thinking about RL algorithms in another way, and this is taxonomy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lAhlN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lAhlN.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you think about methods, you mentioned, let's put them in the right place:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;TD methods in general - model free&lt;/li&gt;&#xA;&lt;li&gt;Monte Carlo methods - model free&lt;/li&gt;&#xA;&lt;li&gt;MinMax - model-based (that could be discussed but it definitely needs access to a world model)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18676" LastActivityDate="2020-05-14T21:19:50.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21172" PostTypeId="1" CreationDate="2020-05-14T21:47:16.960" Score="2" ViewCount="185" Body="&lt;p&gt;Here is the &lt;a href=&quot;https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On/blob/master/Chapter10/02_pong_a2c.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; written by Maxim Lapan. I am reading his book (&lt;em&gt;Deep Reinforcement Learning Hands-on&lt;/em&gt;). I have seen a line in his code which is really weird. In the accumulation of the policy gradient &lt;span class=&quot;math-container&quot;&gt;$$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$&lt;/span&gt; we have to compute the advantage &lt;span class=&quot;math-container&quot;&gt;$R - V_{\theta}(s_i)$&lt;/span&gt;. In line 138, maxim uses &lt;code&gt;adv_v = vals_ref_v - value_v.detach()&lt;/code&gt;. Visually, it looks fine, but look at the shape of each term. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ipdb&amp;gt; adv_v.shape                                                                                                                            &#xA;torch.Size([128, 128])&#xA;&#xA;ipdb&amp;gt; vals_ref_v.shape                                                                                                                       &#xA;torch.Size([128])&#xA;&#xA;ipdb&amp;gt; values_v.detach().shape                                                                                                                &#xA;torch.Size([128, 1]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In a much simpler code, it is equivalent to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: import torch                                                            &#xA;&#xA;In [2]: t1 = torch.tensor([1, 2, 3])                                            &#xA;&#xA;In [3]: t2 = torch.tensor([[4], [5], [6]])                                      &#xA;&#xA;In [4]: t1 - t2                                                                 &#xA;Out[4]: &#xA;tensor([[-3, -2, -1],&#xA;        [-4, -3, -2],&#xA;        [-5, -4, -3]])&#xA;&#xA;In [5]: t1 - t2.detach()                                                        &#xA;Out[5]: &#xA;tensor([[-3, -2, -1],&#xA;        [-4, -3, -2],&#xA;        [-5, -4, -3]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have trained the agent with his code and it works perfectly fine. I am very confused why it is good practice and what it is doing. Could someone enlighten me on the line &lt;code&gt;adv_v = vals_ref_v - value_v.detach()&lt;/code&gt;? For me, the right thing to do was &lt;code&gt;adv_v = vals_ref_v - value_v.squeeze(-1)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the full algorithm used in his book : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aNOfJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aNOfJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see by the image, it is converging even though &lt;code&gt;adv_v = vals_ref_v - value_v.detach()&lt;/code&gt; looks wrongly implemented. It is not done yet, but I will update the question later.&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-15T18:47:18.190" LastActivityDate="2020-05-16T00:50:41.687" Title="Advantage computed the wrong way?" Tags="&lt;python&gt;&lt;policy-gradients&gt;&lt;pytorch&gt;&lt;actor-critic-methods&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21173" PostTypeId="1" CreationDate="2020-05-14T22:39:22.033" Score="1" ViewCount="23" Body="&lt;p&gt;I've been reading on Tacotron-2, a text-to-speech system, that generates speech just-like humans (indistinguishable from humans) using the GitHub &lt;a href=&quot;https://github.com/Rayhane-mamah/Tacotron-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Rayhane-mamah/Tacotron-2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm very confused about a simple aspect of text-to-speech even after reading &lt;a href=&quot;https://arxiv.org/pdf/1712.05884.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; several times. Tacotron-2 generates spectrogram frames for a given input-text. During training, the dataset is a text sentence and its generated spectrogram (it seems at a rate of 12.5 ms per spectogram frame). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the input is provided as a character string, then how many spectogram frames does it predict for each character?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does training supply which frames form the expected output from the dataset? Because the training dataset is simply a thousand of frames for a sentences, how does it know which frames are ideal output for a given character?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This basic aspect seems just not mentioned clearly anywhere and I'm having a hard time figuring this one out. &lt;/p&gt;&#xA;" OwnerUserId="33580" LastEditorUserId="2444" LastEditDate="2020-05-15T15:04:28.367" LastActivityDate="2020-05-15T15:04:28.367" Title="How many spectrogram frames per input character does text-to-speech (TTS) system Tacotron-2 generate?" Tags="&lt;recurrent-neural-networks&gt;&lt;word-embedding&gt;&lt;attention&gt;&lt;text-classification&gt;&lt;speech-synthesis&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21174" PostTypeId="1" CreationDate="2020-05-14T22:49:32.613" Score="0" ViewCount="88" Body="&lt;p&gt;In my discussion over my question on Math SE, I explained to a user, how I think AI works, I wrote that with the sigmoid(logistic) function, features of a data set are identified, many such iterations provide learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding of how this works correct?&lt;/p&gt;&#xA;" OwnerUserId="31307" LastEditorUserId="2444" LastEditDate="2020-05-14T22:59:06.817" LastActivityDate="2020-05-14T23:05:26.120" Title="Is my understanding of how AI works correct?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21175" PostTypeId="2" ParentId="21174" CreationDate="2020-05-14T23:05:26.120" Score="3" Body="&lt;p&gt;There's some useful information in your description, but that's just a very vague description of how neural networks with sigmoid activation functions are trained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, there are many other AI systems apart from neural networks (such as support vector machines, expert systems, etc.), which, of course, I cannot exhaustively list here. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is my understanding of how AI works correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would say it's not completely incorrect, but, as I said, it's a very vague description and it only refers to a subset of techniques in the AI field. With that description, no newbie would probably understand how neural networks are really trained, apart from knowing that you will train them iteratively and there are sigmoids involved.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-14T23:05:26.120" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21176" PostTypeId="1" CreationDate="2020-05-15T02:45:10.067" Score="3" ViewCount="51" Body="&lt;p&gt;I’ve coded a simple ELIZA chatbot for a high school coding competition. The chatbot is part of an app that’s designed to help its user cope with depression, anxiety, and similar mental health disorders. It uses sentiment analysis to identify signs of mental illness, and to track it's user's progress toward &quot;happiness&quot; over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what steps can I take to make it more realistic (without using some pre-existing software, library, etc, which isn't allowed)? Also, are there any existing tables of questions/responses I can add to my ELIZA bot's repertoire so that it can handle more conversations?&lt;/p&gt;&#xA;" OwnerUserId="37062" LastEditorUserId="2444" LastEditDate="2020-05-15T10:11:03.240" LastActivityDate="2020-05-15T10:19:37.447" Title="How can I make ELIZA more realistic?" Tags="&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;chat-bots&gt;&lt;natural-language-understanding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21178" PostTypeId="1" CreationDate="2020-05-15T05:46:53.160" Score="3" ViewCount="45" Body="&lt;p&gt;I'm implementing strided 2D convolution. My formula looks like this:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_{i, j} = \sum_{m=0}^{F_h - 1}\sum_{n=0}^{F_w - 1} x_{s\cdot i + m, s\cdot j + n}\,f_{m, n}, \tag{1}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the stride&#xA;(some sources might refer to this as 'cross-correlation' but 'convolution' is consistent with PyTorch's definition)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have calculated the gradient with respect to the filter as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial E}{\partial f_{m', n'}} = \sum_{i=0}^{(x_h - F_h) / s}\sum_{j=0}^{(x_w - F_w) / s} x_{s\cdot i + m', s\cdot j + n'} \frac{\partial E}{\partial y_{i, j}} \tag{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and some simple dummy index relabeling leads to:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial E}{\partial f_{i, j}} = \sum_{m=0}^{(x_h - F_h) / s}\sum_{n=0}^{(x_w - F_w) / s} x_{s\cdot m + i, s\cdot n + j} \frac{\partial E}{\partial y_{m, n}} \tag{3}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt; looks similar to the first, but not exactly (the &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is on the wrong term!). My objective is to convert the second equation into 'convolutional form' so that I can calculate it using my existing, efficient convolution algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please help me work this out, or point out any errors that I have made?&lt;/p&gt;&#xA;" OwnerUserId="36938" LastActivityDate="2020-05-15T05:46:53.160" Title="Conversion of strided filter gradient to convolutional form" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21181" PostTypeId="2" ParentId="21176" CreationDate="2020-05-15T10:19:37.447" Score="3" Body="&lt;p&gt;One 'easy' way would be to have some sort of conversational memory, where you track what the user has said already. I don't know how complex your patterns are, but if you could recognise names and track references, you could try and build up a mental model of the user's relationships with other people, and perhaps refer to that in your bots responses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latter will be quite advanced, but keeping track of things said earlier and referring back to them on occasion might make it appear a lot more capable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an added bonus, track changes in the user's sentiment scores, and see if you spot a pattern in the conversation (maybe over the course of multiple conversations) to see which bot utterance have the biggest (positive or negative) effect on the user's mood.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-05-15T10:19:37.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21182" PostTypeId="1" AcceptedAnswerId="21185" CreationDate="2020-05-15T11:48:43.013" Score="2" ViewCount="179" Body="&lt;p&gt;Good day, it's a pleasure having joined this Stack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my master thesis I have to expand a Deep Reinforcement Learning Network, to be precise a Deep Q-Network, which is used to control machines in an electrical grid for power quality management. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to evaluate if a network is doing a good job during training or not? Right now I have access to the reward function as well as the q_value function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rewards consist of 4 arrays, one for each learning criteria of the network. The first tuple is a &lt;code&gt;hard criteria&lt;/code&gt; (adherence mandatory) while the latter 3 are &lt;code&gt;soft criteria&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Episode: 1/3000 Step: 1/11 Reward: [[1.0, 1.0, -1.0], [0.0, 0.68, 1.0], [0.55, 0.55, 0.55], [1.0, 0.62, 0.79]]&#xA;Episode: 1/3000 Step: 2/11 Reward: [[-1.0, 1.0, 1.0], [0.49, 0.46, 0.67], [0.58, 0.58, 0.58], [0.77, 0.84, 0.77]]&#xA;Episode: 1/3000 Step: 3/11 Reward: [[-1.0, 1.0, 1.0], [0.76, 0.46, 0.0], [0.67, 0.67, 0.67], [0.77, 0.84, 1.0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The q_values are arrays which I do not fully understand yet. Could one of you explain them to me? I read the official definiton of &lt;code&gt;Q-Values&lt;/code&gt; &lt;code&gt;positive False Discovery Rate&lt;/code&gt;. Can these values be used to evaluate neural network training? These are the &lt;code&gt;Q-Values&lt;/code&gt; for &lt;code&gt;step 1&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q-Values: [[ 0.6934726  -0.24258053 -0.10599071 -0.44178435  0.5393113  -0.60132784&#xA;  -0.07680141  0.97968364  0.7707691   0.57855517  0.16273917  0.44632837&#xA;   0.00799532 -0.53355324 -0.45182624  0.9229134  -1.0455914  -0.0765233&#xA;   0.37784138  0.14711905  0.10986999  0.08918551 -0.8189287   0.14438646&#xA;   0.8869624  -0.43251887  0.7742889  -0.7671829   0.07737591  0.2569678&#xA;   0.5102049   0.5132051  -0.31643414 -0.0042788  -0.66071266 -0.18251896&#xA;   0.7762838   0.15322062 -0.06284399  0.18447408 -0.9609979  -0.4508798&#xA;  -0.07925312  0.7503184   0.6858963  -1.0436649  -0.03167241  0.87660617&#xA;  -0.43605536 -0.28459656 -0.5564517   1.2478396  -1.1418368  -0.9335588&#xA;  -0.72871417  0.04163677  0.30343965 -0.30024529  0.08418611  0.19429305&#xA;   0.44063848 -0.5541725   0.5740701   0.76789933 -0.9621064   0.0272104&#xA;  -0.44953588  0.13415053 -0.07738207 -0.16188647  0.6667519   0.31965214&#xA;   0.3241703  -0.27273563 -0.07130697  0.49683014  0.32996863  0.485767&#xA;   0.39242893  0.40508035  0.3413986  -0.5895434  -0.05772913 -0.6172271&#xA;  -0.12423459  0.2693861   0.32966745 -0.16036317 -0.36371914 -0.04342368&#xA;   0.22878243 -0.09400887 -0.1134861   0.07647536  0.04724833  0.2907955&#xA;  -0.70616114  0.71054566  0.35959414 -1.0539075   0.19137645  1.1948669&#xA;  -0.21796732 -0.583844   -0.37989947  0.09840107  0.31991178  0.56294084]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Are there other ways of evaluating DQNetworks? I would also appreciate literature about this subject. Thank you very much for your time.&lt;/p&gt;&#xA;" OwnerUserId="37079" LastActivityDate="2020-05-15T17:21:32.080" Title="How to evaluate a Deep Q-Network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;software-evaluation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21185" PostTypeId="2" ParentId="21182" CreationDate="2020-05-15T17:21:32.080" Score="2" Body="&lt;p&gt;Q-values represent expected return after taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, so they do tell you how good it is to take an action in the specific state. Better actions will have larger Q-values. Q-values can be used to compares actions but they are not very meaningful in representing performance of the agent since you have nothing to compare them with. You don't know the actual Q-values so you can't conclude if your agent is approximating well those Q-values or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better performance metric would be the average reward per episode/epoch, or average reward in last &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; timesteps for continuing tasks. If your agent is improving its performance then it's average reward should be increasing. You said that you have rewards per state and that some of them represent more important criteria then others. You could plot the average reward per episode by doing some kind of weighted linear combination of criteria rewards&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\bar R = \bar R_1 w_1 + \bar R_2 w_2 + \bar R_3 w_3 + \bar R_4 w_4&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\bar R_i$&lt;/span&gt; is the average episode reward for criteria &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;.That way you can provide more importance to some specific criteria in your evaluation.  &lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-05-15T17:21:32.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21186" PostTypeId="2" ParentId="21109" CreationDate="2020-05-15T19:32:00.710" Score="3" Body="&lt;p&gt;Let's say your old policy is &lt;span class=&quot;math-container&quot;&gt;$\pi_b$&lt;/span&gt; and your current one is &lt;span class=&quot;math-container&quot;&gt;$\pi_a$&lt;/span&gt;. If you collected trajectory by using policy &lt;span class=&quot;math-container&quot;&gt;$\pi_b$&lt;/span&gt; you would get return &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; whose expected value is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;E_{\pi_b}[G_t|S_t = s] &amp;amp;= E_{\pi_b}[R_{t+1} + G_{t+1}]\\&#xA;&amp;amp;= \sum_a \pi_b(a|s) \sum_{s', r} p(s', r|s, a) [r + E_{\pi_b}[G_{t+1}|S_{t+1} = s']]\\&#xA;\end{align}&lt;/span&gt;&#xA;You can see if you write out this recursively that this expectation depends on &lt;span class=&quot;math-container&quot;&gt;$\pi_b(a|s), \pi_b(a'|s'), \ldots$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you collect trajectory with policy &lt;span class=&quot;math-container&quot;&gt;$\pi_a$&lt;/span&gt; you would get expected return that depends on &lt;span class=&quot;math-container&quot;&gt;$\pi_a(a|s), \pi_a(a'|s'), \ldots$&lt;/span&gt; Since these are two different policies then &lt;span class=&quot;math-container&quot;&gt;$\pi_b(a|s) \neq \pi_a(a|s)$&lt;/span&gt; for some &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt;. That would mean that returns have different expected values and are sampled through different distributions. You cannot then use some return &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; sampled by following policy &lt;span class=&quot;math-container&quot;&gt;$\pi_b$&lt;/span&gt; to update policy &lt;span class=&quot;math-container&quot;&gt;$\pi_a$&lt;/span&gt; because it's not sampled according to the proper distribution, if we did, we would be updating policy &lt;span class=&quot;math-container&quot;&gt;$\pi_a$&lt;/span&gt; with biased gradient update that does not reflect how policy &lt;span class=&quot;math-container&quot;&gt;$\pi_a$&lt;/span&gt; performed.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-05-15T19:32:00.710" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21187" PostTypeId="1" AcceptedAnswerId="21308" CreationDate="2020-05-15T19:33:49.713" Score="2" ViewCount="553" Body="&lt;p&gt;I have two different implementations with PyTorch of the Atari Pong game using A2C algorithm. Both implementations are similar, but some portion are different.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://colab.research.google.com/drive/12YQO4r9v7aFSMqE47Vxl_4ku-c4We3B2?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/12YQO4r9v7aFSMqE47Vxl_4ku-c4We3B2?usp=sharing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The above code is from the following Github repository: &lt;a href=&quot;https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On/blob/master/Chapter10/02_pong_a2c.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On/blob/master/Chapter10/02_pong_a2c.py&lt;/a&gt; It converged perfectly well! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find an explanation in Maxim Lapan's book &lt;em&gt;Deep Reinforcement Learning Hands-on&lt;/em&gt; page 269&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the mean reward curve : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pGuDS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pGuDS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://colab.research.google.com/drive/1jkZtk_-kR1Mls9WMbX6l_p1bckph8x1c?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/1jkZtk_-kR1Mls9WMbX6l_p1bckph8x1c?usp=sharing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The above implementation has been created by me based on the Maxim Lapan's book. However, the code is not converging. There's a small portion of my code that is wrong, but I can't point out what it is. I've been working on that near a week now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the mean reward curve : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VBqYt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VBqYt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone tell me the problem portion of the code and how can I fix it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have decided to test my code with a simpler environment, i.e. &lt;code&gt;Cartpole-v0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code : &lt;a href=&quot;https://colab.research.google.com/drive/1zL2sy628-J4V1a_NSW2W6MpYinYJSyyZ?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/drive/1zL2sy628-J4V1a_NSW2W6MpYinYJSyyZ?usp=sharing&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even that code doesn't seem to converge. Still can't see where is my problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the bug might be in the &lt;a href=&quot;https://l.facebook.com/l.php?u=https%3A%2F%2Fgithub.com%2FJeremieGauthier%2FAI_Exercices%2Fblob%2Fmaster%2FAtari_Breakout%2FBreakout_A2C%2Fexperience.py%3Ffbclid%3DIwAR1wP3QhgYQ7Bd2rtYcTTzi5Ne0o9cJuxTJAOyTXhYXReDQq9JVAUCR6pyM&amp;amp;h=AT1CW-RwYDgt1g5EmMSgcNUwdjqxy9yDYkUMaBihpuL9dnAEJGkOWfwIQSdLoQYtqlCWo4PFWEDol8oLVu67oD2TWLqbn9EChoE_Sx5-7Aw5OnFj58Z5eKno1mpk7g&quot; rel=&quot;nofollow noreferrer&quot;&gt;ExperienceSource&lt;/a&gt; class or in the &lt;a href=&quot;https://l.facebook.com/l.php?u=https%3A%2F%2Fgithub.com%2FJeremieGauthier%2FAI_Exercices%2Fblob%2Fmaster%2FAtari_Breakout%2FBreakout_A2C%2Fagent.py%3Ffbclid%3DIwAR0RsFusMNhK0Qo2H4bE0_Szlu4rVU7imhsh1m3WYbb531pnc_xe2VCpFvY&amp;amp;h=AT1CW-RwYDgt1g5EmMSgcNUwdjqxy9yDYkUMaBihpuL9dnAEJGkOWfwIQSdLoQYtqlCWo4PFWEDol8oLVu67oD2TWLqbn9EChoE_Sx5-7Aw5OnFj58Z5eKno1mpk7g&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agent&lt;/a&gt; class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 3&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following &lt;a href=&quot;https://ai.stackexchange.com/questions/21073/once-the-environments-are-vectorized-how-do-i-have-to-gather-immediate-experien?noredirect=1#comment32363_21073&quot;&gt;question&lt;/a&gt; will help you understand the classes &lt;code&gt;ExperienceSource&lt;/code&gt; and &lt;code&gt;ExperienceSourceFirstLast&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-18T13:02:29.260" LastActivityDate="2020-05-20T00:43:35.723" Title="Why isn't my implementation of A2C for the the atari pong game converging?" Tags="&lt;reinforcement-learning&gt;&lt;pytorch&gt;&lt;actor-critic-methods&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21188" PostTypeId="2" ParentId="21172" CreationDate="2020-05-15T20:27:37.550" Score="1" Body="&lt;p&gt;I changed the line &lt;code&gt;adv_v = vals_ref_v - value_v.detach()&lt;/code&gt; to &lt;code&gt;adv_v = vals_ref_v - value_v.squeeze(-1).detach()&lt;/code&gt;. It seems the convergence is much faster. According to the A2C algorithm, it is just logic to apply &lt;span class=&quot;math-container&quot;&gt;$Q(a, s) - V(s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Q(a, s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; with the same shape.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The call to &lt;code&gt;detach()&lt;/code&gt; is important here as we don't want to propagate the PG into our value approximation head.&lt;/p&gt;&#xA;" OwnerUserId="35626" LastActivityDate="2020-05-15T20:27:37.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21189" PostTypeId="1" CreationDate="2020-05-15T20:55:33.027" Score="1" ViewCount="44" Body="&lt;p&gt;I've been trying to train a snake for the snake game in DQN. Which the snake can essentially just move up, down, left and right. I'm having a hard time getting the snake to stay alive longer. So my question is, what are some techniques that I can implement to get the snake to stay alive for longer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the things that I've attempted but doesn't seem to have done much after about 1000 episodes are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Implementing the L2 regularization&lt;/li&gt;&#xA;&lt;li&gt;Reduce the exploration decay rate so it give the snake more chance to explore&lt;/li&gt;&#xA;&lt;li&gt;Randomize the starting point for the snake for each episode to try to reduce &quot;local exploration&quot;&lt;/li&gt;&#xA;&lt;li&gt;I've tweeked some hyper parameters such as learning rate, policy/target network update rate&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The input neurons are fed with the state of the board. For example, if my board size is 12*12 then there are 144 input neurons each representing the space of the environment. I've checked that the loss decreases fairly quickly but no improvements on snake lasting longer in the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note my reward function is simply a +1 for every time step that the snake survives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm out of ideas of what I can do to get the snake to learn, maybe 1000 episodes is simply not enough? Or maybe my input is not providing good enough information to train the snake?&lt;/p&gt;&#xA;" OwnerUserId="33579" LastActivityDate="2020-05-15T20:55:33.027" Title="DQN not showing the agent is learning in a snake grid environment game" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21190" PostTypeId="2" ParentId="21162" CreationDate="2020-05-15T21:13:44.443" Score="1" Body="&lt;p&gt;OpenReview (&lt;a href=&quot;https://openreview.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openreview.net/&lt;/a&gt;) may be what you're looking for. It's both a platform for ML conferences to organize their reviewing process as well as an arXiv-like preprint repository.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One major machine learning conference (ICLR) has been holding &quot;reproducibility challenges,&quot; in which participants can submit replication reports of existing papers, but that probably only applies to papers submitted to that conference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;a href=&quot;https://openreview.net/group?id=OpenReview.net/Anonymous_Preprint&quot; rel=&quot;nofollow noreferrer&quot;&gt;general preprints section&lt;/a&gt;, which might be your best bet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I used to be on the OpenReview development team&lt;/p&gt;&#xA;" OwnerDisplayName="user37097" LastActivityDate="2020-05-15T21:13:44.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21191" PostTypeId="1" CreationDate="2020-05-16T00:33:19.943" Score="2" ViewCount="70" Body="&lt;p&gt;I’ve created an agent using MCTS to play Connect Four. It wins against humans pretty well, but I’d like to improve upon it. I decided to add domain knowledge to the MCTS rollout stage. My evaluation function checks how “good” an action is and returns the best/highest value action to the rollout policy as the action to use. &#xA;I created a “gym” application for one agent, who’s not using the evaluation function, to play against an agent who is using the evaluation function. &#xA;I would have expected the agent using the heuristics to perform better than the agent who isn’t, but the inclusion of the heuristics doesn’t seem to make any difference! Any ideas why this might be the case? &lt;/p&gt;&#xA;" OwnerUserId="27629" LastActivityDate="2021-02-12T00:01:30.173" Title="Why aren’t heuristics for Connect Four Monte Carlo tree search improving the agent?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;heuristics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21193" PostTypeId="2" ParentId="21172" CreationDate="2020-05-16T00:50:41.687" Score="2" Body="&lt;p&gt;Yeah, it seems like it's a wrong implementation.&#xA;vals_ref_v is a matrix of 1 row, and 128 columns.&#xA;value_v.detach() is a matrix of 128 row &lt;/p&gt;&#xA;" OwnerUserId="37099" LastActivityDate="2020-05-16T00:50:41.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21194" PostTypeId="1" CreationDate="2020-05-16T02:55:08.503" Score="1" ViewCount="70" Body="&lt;p&gt;When does it happen that a layer (either first or hidden) outputs negative values in order to justify the use of RELU? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, features are never negative or converted to negative in any other type of layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it that we can use the RELU with a different &quot;inflection&quot; point than zero, so we can make the neuron start describing a lineal response just after this &quot;new zero&quot;? &lt;/p&gt;&#xA;" OwnerUserId="36440" LastEditorUserId="2444" LastEditDate="2020-05-16T11:16:29.167" LastActivityDate="2020-05-16T11:16:29.167" Title="If features are always positives, why do we use RELU activation functions?" Tags="&lt;convolutional-neural-networks&gt;&lt;linear-regression&gt;&lt;relu&gt;&lt;non-linear-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21196" PostTypeId="2" ParentId="21194" CreationDate="2020-05-16T05:28:27.080" Score="3" Body="&lt;p&gt;The fact that features are always positive values don't guarantee that outputs of hidden layers are positive too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to multiplication, output of an hidden layer could contain negative values, i.e., a hidden layer can contain weights that have opposites signs as its input.&#xA;Remember that only layer outputs, not their weights, are passed through ReLu, so, weights of a model could contain negative values.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="32621" LastEditDate="2020-05-16T07:24:04.663" LastActivityDate="2020-05-16T07:24:04.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21197" PostTypeId="1" CreationDate="2020-05-16T07:02:40.860" Score="1" ViewCount="34" Body="&lt;p&gt;Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general property of deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a small batch size of 512 samples the walker achieves a near-optimal behavior after just 0.5 Mio steps. The optimized hyperparameters in the RL Zoo suggest using a batch size of 32k steps. This definitely leads to better performance after 5 Mio steps but takes 2 Mio steps until it reaches a near-optimal behavior. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore the question: &#xA;&lt;strong&gt;Shouldn't we schedule the batch-size to improve sample efficiency?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe it makes sense because after initialization the policy is far away from the optimal one and therefore should update quickly to get better. Even when the gradient estimates using small batches are very noisy, they still seem to bring the policy quickly in a quite good state. Thereafter we can increase the batch-size and make less but more precise gradient steps. &lt;strong&gt;Or am I missing an important point here?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-05-16T07:02:40.860" Title="Should we start with a small batch-size and increase during training to improve sample efficiency?" Tags="&lt;deep-learning&gt;&lt;deep-rl&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;sample-efficiency&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21198" PostTypeId="1" AcceptedAnswerId="21355" CreationDate="2020-05-16T09:03:48.750" Score="2" ViewCount="52" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1911.05954.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical Graph Pooling with Structure Learning&lt;/a&gt; (2019) introduces a distance measure between:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a graph's node-representation matrix &lt;span class=&quot;math-container&quot;&gt;$\text{H}$&lt;/span&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;an approximation of this constructed from each node's neighbours' information &lt;span class=&quot;math-container&quot;&gt;$\text{D}^{-1}\text{A}\text{H}$&lt;/span&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Here, we formally define the node information score as the Manhattan distance between the node representation itself and the one constructed from its neighbors:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{p} = \gamma(\mathcal{G}_i) = ||(\text{I}^{k}_{i} - (\text{D}^{k}_{i})^{-1}\text{A}^{k}_{i})\text{H}^{k}_{i}||  $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;sup&gt;(where &lt;span class=&quot;math-container&quot;&gt;$\text{A}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{D}$&lt;/span&gt; are the Adjacency and Diagonal matrices of the graph, respectively)&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;Expanding the product on the RHS we get (ignoring index notation for simplicity):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$||\text{H} - (\text{D}^{-1}\text{A}\text{H})||$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; I don't see how &lt;span class=&quot;math-container&quot;&gt;$\text{D}^{-1}\text{A}\text{H}$&lt;/span&gt; is a &amp;quot;node representation... constructed from its neighbors&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{I} - \text{D}^{-1}\text{A}$&lt;/span&gt; is clearly equivalent to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Laplacian_matrix#Random_walk_normalized_Laplacian&quot; rel=&quot;nofollow noreferrer&quot;&gt;Random Walk Laplacian&lt;/a&gt;, but it's not immediately obvious to me how multiplying this by &lt;span class=&quot;math-container&quot;&gt;$\text{H}$&lt;/span&gt; provides per-node information on how well one can reconstruct a node from its neighbours.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-21T07:52:52.327" Title="Understanding the node information score in the paper &quot;Hierarchical Graph Pooling with Structure Learning&quot;" Tags="&lt;papers&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;&lt;spectral-analysis&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21200" PostTypeId="1" CreationDate="2020-05-16T12:48:21.730" Score="3" ViewCount="133" Body="&lt;p&gt;I am trying to implement &lt;strong&gt;CNN&lt;/strong&gt; using python Numpy.&lt;br&gt; &#xA;I searched so much, but all I found was for one filter with one channel for Convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have an &lt;strong&gt;X&lt;/strong&gt; as Image with this shape: &lt;code&gt;(N_Height, N_Width, N_Channel) = (5,5,3)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And Let's say I have &lt;code&gt;16&lt;/code&gt; filters with this shape: &lt;code&gt;(F_Height, F_Width, N_Channel) = (3,3,3)&lt;/code&gt; , &lt;code&gt;stride=1&lt;/code&gt; and &lt;code&gt;padding=0&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Forward&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output shape after conv2d will be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;(&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;math.floor((N_Height - F_Height + 2*padding)/stride + 1 )),&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;math.floor((N_Width- F_Width + 2*padding)/stride + 1 )),&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;filter_count&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the output of this layer will be an array with this shape: &lt;code&gt;(Height, Width, Channel) =  (3, 3, 16)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BackPropagation&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$dL/dh$&lt;/span&gt; is the input for my layer in backpropagation with this shape: &lt;code&gt;(3,3,16)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I must find &lt;span class=&quot;math-container&quot;&gt;$dL/dw$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dL/dx$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$dL/dw$&lt;/span&gt; to update my filters params and &lt;span class=&quot;math-container&quot;&gt;$dL/dx$&lt;/span&gt; to pass it as input to the previous layer as Loss respect to the input &lt;strong&gt;X&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this answer &lt;a href=&quot;https://datascience.stackexchange.com/a/45978/89170&quot;&gt;Error respect to filters weights&lt;/a&gt; I found how to calculate &lt;span class=&quot;math-container&quot;&gt;$dL/dw$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I have in &lt;strong&gt;BackPropagation&lt;/strong&gt; is I don't know how to calculate &lt;span class=&quot;math-container&quot;&gt;$dL/dx$&lt;/span&gt; having this shape:&lt;code&gt;(5,5,3)&lt;/code&gt; and pass it to the prev layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read lots of articles in Medium and other sites but I don't get how to calculate it:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@pavisj/convolutions-and-backpropagations-46026a8f5d2c&quot; rel=&quot;nofollow noreferrer&quot;&gt;How Backpropagation works in a CNN&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://medium.com/technologymadeeasy/the-best-explanation-of-convolutional-neural-networks-on-the-internet-fbb8b1ad5df8&quot; rel=&quot;nofollow noreferrer&quot;&gt;The best explanation of Convolutional Neural Networks on the Internet!&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Backpropagation In Convolutional Neural Networks&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://datascience.stackexchange.com/questions/28593/how-to-propagate-error-back-to-previous-layer-in-cnn?rq=1&quot;&gt;How to propagate error back to previous layer in CNN?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance :)&lt;/p&gt;&#xA;" OwnerUserId="36905" LastEditorUserId="36905" LastEditDate="2020-05-17T10:47:52.240" LastActivityDate="2020-05-22T12:20:48.813" Title="How do I calculate the partial derivative with respect to $x$?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21201" PostTypeId="5" CreationDate="2020-05-16T14:23:41.043" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-16T14:23:41.043" LastActivityDate="2020-05-16T14:23:41.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21202" PostTypeId="4" CreationDate="2020-05-16T14:23:41.043" Score="0" Body="For questions related to the deep belief network (DBN), a generative deep neural network composed of multiple layers of latent variables (&quot;hidden units&quot;), with connections between the layers but not between units within each layer. DBNs can be viewed as a composition of simple, unsupervised networks such as restricted Boltzmann machines (RBMs or autoencoders, where each sub-network's hidden layer serves as the visible layer for the next." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-16T14:23:41.043" LastActivityDate="2020-05-16T14:23:41.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21203" PostTypeId="1" CreationDate="2020-05-16T16:04:39.783" Score="3" ViewCount="55" Body="&lt;p&gt;I am pretty new to the machine learning field. I want to use an &lt;span class=&quot;math-container&quot;&gt;$n \times m$&lt;/span&gt; matrix as the input of a model, in order to predict a vector &lt;span class=&quot;math-container&quot;&gt;$1 \times m$&lt;/span&gt;, both of real numbers. Input data are quite clean, with statistics of about 10000 items. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know a method that can handle that?&lt;/p&gt;&#xA;" OwnerUserId="37117" LastEditorUserId="2444" LastEditDate="2020-05-16T16:57:32.843" LastActivityDate="2020-05-16T16:57:32.843" Title="Which machine learning method can take a matrix as input?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21205" PostTypeId="2" ParentId="12133" CreationDate="2020-05-16T17:32:47.147" Score="0" Body="&lt;p&gt;The usage of the word &quot;kernel&quot; in the context of support vector machines probably comes from its usage in the context of &lt;a href=&quot;https://en.wikipedia.org/wiki/Integral_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;integral transforms&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the article &lt;a href=&quot;https://www.encyclopediaofmath.org/index.php/Kernel_of_an_integral_operator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kernel of an integral operator&lt;/a&gt;, and the questions &lt;a href=&quot;https://math.stackexchange.com/q/22133/168764&quot;&gt;What is the difference between a kernel and a function?&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/q/1919305/168764&quot;&gt;Why is the kernel of an integral transform called kernel?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The word &quot;kernel&quot; has been used in many other contexts, such as in computer vision, to refer to a certain function with a special purpose. See e.g. the paper that introduced &lt;a href=&quot;https://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-16T17:39:19.893" LastActivityDate="2020-05-16T17:39:19.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21206" PostTypeId="2" ParentId="17577" CreationDate="2020-05-16T18:22:44.650" Score="3" Body="&lt;p&gt;In a classification problem it's better to get higher error and higher error slope when we predict the label wrong.&lt;/p&gt;&#xA;&lt;p&gt;As you see in the graph by using cross-entropy you get high error when the algorithm predict a label wrong and small error when the prediacted label is close enough, so it helps us to separate the predicted classes better.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CwHzl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CwHzl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37123" LastEditorUserId="37123" LastEditDate="2020-07-23T12:56:51.987" LastActivityDate="2020-07-23T12:56:51.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21207" PostTypeId="2" ParentId="20980" CreationDate="2020-05-16T18:39:17.293" Score="1" Body="&lt;p&gt;To put this insert to context, we should take at least this much of text from the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One line of research focuses on making recommendations using knowledge graph embedding models, such as TransE [2] and node2vec [5]. These approaches align the knowledge graph in a regularized vector space and uncover the similarity between entities by calculating their representation distance [30]. However,pure KG embedding methods lack the ability to discover multi-hop relational paths.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my understanding the &lt;code&gt;pure KG embedding&lt;/code&gt; here refers to &lt;code&gt;TransE&lt;/code&gt; and &lt;code&gt;node2vec&lt;/code&gt; solutions. To learn more about those, we should read links [1] and [2]. From [1]:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Usually, we use a triple (head, relation, tail) to represent a knowledge. Here, head and tail are entities. For example, (sky tree, location, Tokyo). We can use the one-hot vector to represent this knowledge.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Later on same source there is an explanation on the end of definition section of &lt;code&gt;TransE&lt;/code&gt; solution:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But this model only can take care with one-to-one relation, not suitable for one-to-many/many-to-one relation, for example, there is two knowledge, (skytree, location, tokyo) and (gundam, location, tokyo). After training, the 'sky tree' entity vector will be very close with 'gundam' entity vector. But they do not have such similarity in real.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On other hand, [3] tells:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Knowledge graphs as described above represents a static snapshot of our knowledge. It does not reflect the process of it’s how the knowledge built up. In the real world, we learn by observing temporal patterns. While it’s possible to learn the similarity between nodes A and node B, it will be hard to see the similarity between node A and node C as it was 3 years ago.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Solution in paper states &quot;--,each recommended item is associated with around 1.6 reasoning paths.&quot; which is supposedly impossible for &lt;code&gt;TransE&lt;/code&gt; solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, knowledge graph, purely, is a static snapshot that can identify by great accuracy one to one findings. Actually according what [2] tells how &lt;code&gt;node2vec&lt;/code&gt; works, they can describe and combine also more information (&lt;code&gt;node2vec&lt;/code&gt; combines different types of similarities at same time), but anyways I think the main point is actually in one word in the citation: &lt;code&gt;discover&lt;/code&gt;! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model suggested on paper adds Reinforcement Learning principles to the KG modeling, so to say, the pure KG embedding always tells one distance based statistical solution, but a RL based solution may learn more aspects behind the scenes as it learns by try and error more complex paths underlying in the behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the paper says, when relating their solution to pure &lt;code&gt;TransE&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It can be regarded as a single-hop latent matching method, but the post-hoc explanations do not necessarily reflect the true reason of generating a recommendation. In contrast, our methods generate recommendations through an explicit path reasoning process over knowledge graphs, so that the explanations directly reflect how the decisions are generated, which makes the system transparent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, whether &lt;code&gt;TransE&lt;/code&gt; and those could actually recommend things in the given environment, the recommendation reasoning paths may stay obscure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://towardsdatascience.com/summary-of-translate-model-for-knowledge-graph-embedding-29042be64273&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/summary-of-translate-model-for-knowledge-graph-embedding-29042be64273&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://towardsdatascience.com/node2vec-embeddings-for-graph-data-32a866340fef&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/node2vec-embeddings-for-graph-data-32a866340fef&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://towardsdatascience.com/extracting-knowledge-from-knowledge-graphs-e5521e4861a0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/extracting-knowledge-from-knowledge-graphs-e5521e4861a0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11810" LastActivityDate="2020-05-16T18:39:17.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21209" PostTypeId="1" AcceptedAnswerId="21213" CreationDate="2020-05-16T20:12:54.310" Score="3" ViewCount="63" Body="&lt;p&gt;Here's the approximated value using weighted importance sampling&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;V_{n} \doteq \frac{\sum_{k=1}^{n-1} W_{k} G_{k}}{\sum_{k=1}^{n-1} W_{k}}, \quad n \geq 2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the incremental update rule for the approximated value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_{n+1} \doteq V_{n}+\frac{W_{n}}{C_{n}}\left[G_{n}-V_{n}\right], \quad n \geq 1$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the second equation derived from the first? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are used for the weighted importance sampling method of off-policy Monte Carlo control.&lt;/p&gt;&#xA;" OwnerUserId="37128" LastEditorUserId="2444" LastEditDate="2020-05-16T20:36:16.297" LastActivityDate="2020-05-16T22:09:16.120" Title="How is the incremental update rule derived from the weighted importance sampling in off-policy Monte Carlo control?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;value-functions&gt;&lt;importance-sampling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21211" PostTypeId="1" CreationDate="2020-05-16T20:34:07.090" Score="2" ViewCount="28" Body="&lt;p&gt;I am doing some research on the visual attention mechanism in remote sensing domain (where the features learnt from one layer are highlighted using the attention mask derived from another layer). From what I have observed, the attention mask is learnt in a similar fashion as any other branch in CNN. So, what is so special about the visual attention mask that makes it different from a regular two branch CNN? The reference papers are provided below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/document/8736024&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visual Attention-Driven Hyperspectral Image Classification (IEEE, 2019)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-01931435/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Two-Branch CNN Architecture for Land Cover&#xA;Classification of PAN and MS Imagery (MDPI, 2019)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15238" LastEditorUserId="1671" LastEditDate="2020-05-18T22:00:15.027" LastActivityDate="2020-05-18T22:00:15.027" Title="How is visual attention mechanism different from a two branch convolutional neural network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21212" PostTypeId="2" ParentId="19889" CreationDate="2020-05-16T21:04:32.817" Score="1" Body="&lt;h1&gt;Assumption&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In this answer it is assumed that with &quot;neurochips&quot; you mean chips made (using neuromorphic engineering) for neuromphic computing. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Related example&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;From what I currently understand from &lt;a href=&quot;https://arxiv.org/pdf/2004.06061.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; neuromorphic chips, in particular the TrueNorth chip, are being used (or emulated) for embedded systems related signals processing. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Doubt&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The signal processing performed by these (emulated) neuromorphic chips might be part of real-life applications involving control systems.&lt;/p&gt;&#xA;" OwnerUserId="4903" LastActivityDate="2020-05-16T21:04:32.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21213" PostTypeId="2" ParentId="21209" CreationDate="2020-05-16T22:09:16.120" Score="2" Body="&lt;p&gt;By definition of &lt;span class=&quot;math-container&quot;&gt;$V_{n+1}$&lt;/span&gt;, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = \frac{\sum_{k=1}^{n} W_{k} G_{k}}{\sum_{k=1}^{n} W_{k}} \; \tag{1}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, taking the &lt;span class=&quot;math-container&quot;&gt;$n^{th}$&lt;/span&gt; term out of the sum in the numerator, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = \frac{W_{n}G_{n} \; + \; \sum_{k=1}^{n-1} W_{k} G_{k}}{\sum_{k=1}^{n} W_{k}} \; \tag{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, from the definition of &lt;span class=&quot;math-container&quot;&gt;$V_n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_{n} = \frac{\sum_{k=1}^{n-1} W_{k} G_{k}}{\sum_{k=1}^{n-1} W_{k}}$&lt;/span&gt;, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{k=1}^{n-1} W_{k} G_{k} = V_{n}*\sum_{k=1}^{n-1} W_{k} \; \tag{3}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, substituting &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt; in the numerator of &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt;, we get: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = \frac{W_{n}G_{n} \; + \; V_{n}*\sum_{k=1}^{n-1} W_{k}}{\sum_{k=1}^{n} W_{k}} \; \tag{4}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, adding and subtracting &lt;span class=&quot;math-container&quot;&gt;$V_{n}W_{n}$&lt;/span&gt; in the numerator of &lt;span class=&quot;math-container&quot;&gt;$(4)$&lt;/span&gt;, we obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = \frac{W_{n}G_{n} \; + \; V_{n}*\sum_{k=1}^{n-1} W_{k} \; + \; V_n W_n \; - \; V_n W_n}{\sum_{k=1}^{n} W_{k}} \; \tag{5}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We factor &lt;span class=&quot;math-container&quot;&gt;$V_n$&lt;/span&gt; in the numerator of &lt;span class=&quot;math-container&quot;&gt;$(5)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = \frac{W_{n}G_{n} \; + \; V_{n}(W_n \; + \; \sum_{k=1}^{n-1} W_{k}) \; - \; V_n W_n}{\sum_{k=1}^{n} W_{k}} \; \tag{6}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We simplify, taking into account that the denominator &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=1}^{n} W_{k} = W_n + \sum_{k=1}^{n-1} W_{k}$&lt;/span&gt;, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = V_n + \frac{W_n G_n - W_n V_n}{\sum_{k=1}^{n} W_{k}} \; \tag{7} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further rearrangements of the terms give us:&#xA;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = V_n + \frac{W_n}{\sum_{k=1}^{n} W_{k}}[G_n - V_n] \; \tag{8}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, by definition of &lt;span class=&quot;math-container&quot;&gt;$C_n$&lt;/span&gt; as the cumulative sum of the weights up to time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, we get the desired incremental update equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$V_{n+1} = V_n + \frac{W_n}{C_n}[G_n - V_n] \; \tag{9}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-05-16T22:09:16.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21215" PostTypeId="2" ParentId="3101" CreationDate="2020-05-17T04:11:58.370" Score="-1" Body="&lt;p&gt;Not really, I mean at it's core machine learning from an application perspective often seeks produce human level results, but there isn't any theorem describing human understanding of reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like proving computer vision works well is essentially like proving you have a correct understanding of human perception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It becomes somewhat circular, and while there exists proofs for certain qualities of data, none of them are true. I mean think about trying to describing reality, it exists on a lower dimensional manifold but analytically describing it? Don't think so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even proving robustness ends up being somewhat futile since even if you correctly eliminate advasarial examples this doesn't mean you CV application will produce correct results in general, only that the classification is robust(robust and correct are two different things).&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-17T04:11:58.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21216" PostTypeId="1" CreationDate="2020-05-17T04:48:53.850" Score="4" ViewCount="378" Body="&lt;p&gt;Besides computer vision and image classification, what other use cases/applications are for few-shot learning?&lt;/p&gt;&#xA;" OwnerUserId="37137" LastEditorUserId="2444" LastEditDate="2021-01-07T17:24:21.437" LastActivityDate="2021-01-07T17:24:21.437" Title="What are some use cases of few-shot learning?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;applications&gt;&lt;few-shot-learning&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21218" PostTypeId="1" CreationDate="2020-05-17T06:55:00.937" Score="1" ViewCount="57" Body="&lt;p&gt;As far as I know, neural networks have hidden computational units and HMM has hidden states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hidden Markov Models can be used to generate a language, that is, list elements from a family of strings. For example, if you have an HMM that models a set of sequences, you would be able to generate members of this family, by listing sequences that would befall into the group of sequences we are modeling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://dl.acm.org/doi/pdf/10.1145/3379247.3379276&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://ieeexplore.ieee.org/document/5232417&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://www.researchgate.net/publication/33702866_Hidden_Markov_models_and_artificial_neural_networks_for_speech_and_speaker_recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, HMMs are combined with ANNs. But how exactly? What is a Hidden Markov Model - Artificial Neural Network (HMM-ANN)? Is HMM-ANN a hybrid algorithm? In simple words, how is this model or algorithm used?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-17T12:47:03.000" LastActivityDate="2020-05-17T12:47:03.000" Title="What is a Hidden Markov Model - Artificial Neural Network (HMM-ANN)?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;papers&gt;&lt;hidden-markov-model&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21219" PostTypeId="1" AcceptedAnswerId="21227" CreationDate="2020-05-17T07:24:48.927" Score="3" ViewCount="932" Body="&lt;p&gt;What is the difference between deep learning and &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fninf.2019.00053/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;shallow learning&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am interested in knowing is not the definition of deep learning &amp;amp; shallow learning, but understanding the actual difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links to other resources are also appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-17T13:22:39.573" LastActivityDate="2020-05-17T13:22:39.573" Title="What is the difference between deep learning and shallow learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;comparison&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21220" PostTypeId="1" CreationDate="2020-05-17T07:28:32.527" Score="2" ViewCount="58" Body="&lt;p&gt;A concept class &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is PAC-learnable if there exists an algorithm that can output a hypothesis with probability at least &lt;span class=&quot;math-container&quot;&gt;$(1-\delta)$&lt;/span&gt; (the &quot;probably&quot; part), and an error that is less than &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; (the &quot;approximately&quot; part), in time that is polynomial in &lt;span class=&quot;math-container&quot;&gt;$1/\epsilon$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1/\delta$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|C|$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.cmu.edu/~tom/mlbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tom Mitchell&lt;/a&gt; defines an upper bound for the sample complexity, &lt;span class=&quot;math-container&quot;&gt;$m &amp;gt;= 1/\epsilon (ln(|H|) + ln(1/\delta))$&lt;/span&gt; for the finite hypotheses. Based on this bound, he classifies whether target concepts are PAC-learnable or not. For example, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; conjunction boolean literal concept class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that PAC-learnability seeks to act more like a classification of certain concept classes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any practical purposes for knowing whether a concept class is PAC-learnable? &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-17T14:01:52.853" LastActivityDate="2020-05-18T13:54:49.370" Title="Is there any practical application of knowing whether a concept class is PAC-learnable?" Tags="&lt;applications&gt;&lt;computational-learning-theory&gt;&lt;pac-learning&gt;&lt;sample-complexity&gt;&lt;hypothesis-class&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21222" PostTypeId="5" CreationDate="2020-05-17T11:05:43.100" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-17T11:05:43.100" LastActivityDate="2020-05-17T11:05:43.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21223" PostTypeId="4" CreationDate="2020-05-17T11:05:43.100" Score="0" Body="For questions related to the sample complexity of a machine learning algorithm, which represents the number of training-samples that it needs in order to successfully learn a target function. More precisely, the sample complexity is the number of training-samples that we need to supply to the algorithm, so that the function returned by the algorithm is within an arbitrarily small error of the best possible function, with probability arbitrarily close to 1." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-17T11:05:43.100" LastActivityDate="2020-05-17T11:05:43.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21224" PostTypeId="5" CreationDate="2020-05-17T11:08:08.280" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-17T11:08:08.280" LastActivityDate="2020-05-17T11:08:08.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21225" PostTypeId="4" CreationDate="2020-05-17T11:08:08.280" Score="0" Body="For questions related to the concept of a hypothesis class in the context of computational learning theory. A hypothesis class can be defined as the set of hypotheses (i.e. functions) considered by the learning algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-17T11:08:08.280" LastActivityDate="2020-05-17T11:08:08.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21226" PostTypeId="2" ParentId="21216" CreationDate="2020-05-17T11:23:21.260" Score="2" Body="&lt;p&gt;Few-short learning (FSL) can be useful for many (if not all) machine learning problems, including supervised learning (regression and classification) and reinforcement learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1904.05046.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generalizing from a Few Examples: A Survey on Few-Shot Learning&lt;/a&gt; (2020) provides an overview (including examples of applications and use cases) of FSL. Their definition of FSL provided is based on Tom Mitchell's famous definition of machine learning.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Definition 2.1 (&lt;strong&gt;Machine Learning&lt;/strong&gt; [92, 94]). A computer program is said to learn from experience &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; with respect to some classes of task &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and performance measure &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; if its performance can improve with &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; measured by &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here's the definition of FSL.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Definition 2.2. &lt;strong&gt;Few-Shot Learning&lt;/strong&gt; (FSL) is a type of machine learning problems, specified by &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; contains only a limited number of examples with supervised information for the task &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Specific examples of applications of FSL are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;character generation&lt;/li&gt;&#xA;&lt;li&gt;drug toxicity discovery&lt;/li&gt;&#xA;&lt;li&gt;sentiment classification from short text&lt;/li&gt;&#xA;&lt;li&gt;object recognition&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-17T13:34:06.323" LastActivityDate="2020-05-17T13:34:06.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21227" PostTypeId="2" ParentId="21219" CreationDate="2020-05-17T12:57:56.797" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.edge.org/response-detail/26137&quot; rel=&quot;nofollow noreferrer&quot;&gt;That article&lt;/a&gt; only mentions &quot;shallow learning&quot; in the title and it mentions &quot;shallow&quot; to refer to the fact that deep learning models are not really learning any &quot;deep&quot; concepts, where &quot;deep&quot; here means &quot;philosophically deep&quot;. So, I think the title is just (fairly?) provocative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, in machine learning, the expression &quot;shallow learning&quot; isn't really standardized, as opposed to deep learning, which refers to learning, with gradient descent and back-propagation, from (typically) huge amounts of data with neural networks. Nevertheless, &quot;shallow learning&quot; may occasionally refer to everything that isn't deep learning (e.g. traditional machine learning models, such as support vector machines), but most likely it refers to learning in neural networks with only a small number (0-2) of hidden layers (i.e. non-deep neural networks). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the difference between deep and shallow neural networks isn't really clear. Some people may consider neural networks with only 1-2 hidden layers already deep, while others may consider only neural networks with e.g. 5-10 hidden layers deep. This also shows that deep learning isn't actually well-defined too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fninf.2019.00053/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;other linked article&lt;/a&gt; actually says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;CNN performance was compared to that of &lt;strong&gt;conventional (shallow) machine learning methods&lt;/strong&gt;, including ridge regression (RR) on the images’ principal components and support vector regression.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, in this article, they use &quot;shallow learning&quot; to refer to traditional (or conventional) machine learning models, which confirms what I said above.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-17T13:18:51.563" LastActivityDate="2020-05-17T13:18:51.563" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21228" PostTypeId="1" CreationDate="2020-05-17T16:15:48.917" Score="1" ViewCount="39" Body="&lt;p&gt;Is there an AI system (preferably, one that interacts with the human, such as a chatbot like &lt;a href=&quot;https://www.pandorabots.com/pandora/talk?botid=b8d616e35e36e881&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;) that, given some input (e.g. entered into the system by writing text), such as a person's physical history and symptoms (of a certain disease), produces a diagnosis of the disease and/or suggests medications or a treatment to improve the condition of the patient?&lt;/p&gt;&#xA;" OwnerUserId="36957" LastEditorUserId="2444" LastEditDate="2020-05-17T18:53:45.413" LastActivityDate="2020-05-17T18:53:50.190" Title="Is there an AI system that, given a patient's symptoms, produces a diagnosis and suggests a treatment?" Tags="&lt;applications&gt;&lt;reference-request&gt;&lt;chat-bots&gt;&lt;resource-request&gt;&lt;healthcare&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21229" PostTypeId="2" ParentId="21216" CreationDate="2020-05-17T17:17:48.897" Score="0" Body="&lt;p&gt;An interesting use case is IQ tests, or program synthesis from examples in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IQ tests often require you to derive a program from a few examples that can produce a certain output. See for instance &lt;a href=&quot;https://github.com/fchollet/ARC&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/fchollet/ARC&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37145" LastActivityDate="2020-05-17T17:17:48.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21230" PostTypeId="1" CreationDate="2020-05-17T17:44:55.300" Score="3" ViewCount="44" Body="&lt;p&gt;From the brief research I've done on the topic, it appears that the way Deepmind's Alphazero or Muzero makes decisions is through Monte Carlo tree searches, where in the randomized simulations allows for a more rapid way to make calculations than traditional alpha-beta pruning. As the simulation space increases, this search approaches that of a classical tree search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where exactly did Deepmind use neural networks? Was it in the evaluation portion? And if so, how did they make determinations on what makes a &quot;good&quot; or &quot;bad&quot; game state? If they deferred the evaluations of another chess engine like Stockfish, how do we see AlphaZero absolutely demolish Stockfish in head-to-head matches? &lt;/p&gt;&#xA;" OwnerUserId="37147" LastActivityDate="2020-05-17T17:44:55.300" Title="Where does reinforcement learning actually show up in Deepmind's game engines?" Tags="&lt;deep-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;chess&gt;&lt;alphazero&gt;&lt;deepmind&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21231" PostTypeId="1" CreationDate="2020-05-17T18:01:13.180" Score="1" ViewCount="38" Body="&lt;p&gt;I want to generate a confidence interval around my prediction (vector) &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt;. I have implemented the following procedure. However, I am not sure whether this makes sense in a statistical way:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I have a data set. First I split it into a 80% training set (2000 measurements), 10% valdidation set and 10% testing set (250 measurements)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I resample B (&lt;span class=&quot;math-container&quot;&gt;$\sim 100$&lt;/span&gt;) training sets from the original training set with replacement. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For each of the B training datasets I train the model &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and&lt;br&gt;&#xA;validate it &#xA;(everytime I use the same validation set).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I use the testset from point 1. and make a prediction &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i^b$&lt;/span&gt; &#xA;(so everytime I use the same test set, since I need the predictions for the same input values)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;I calculate the average of the &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; predictions. &lt;span class=&quot;math-container&quot;&gt;$$\bar{\hat{y}}_i=\frac{1}{B}\sum_{b=1}^B\hat{y}_i^b$$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I calculate the variance  (&lt;span class=&quot;math-container&quot;&gt;$i\in [1,250]$&lt;/span&gt;) &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma_{\hat{y}_i}^2=\frac{1}{B-1}\sum_{b=1}^B(\hat{y}_i^b -\bar{\hat{y}}_i)^2$$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I &lt;strong&gt;guess&lt;/strong&gt; the &lt;span class=&quot;math-container&quot;&gt;$95\%$&lt;/span&gt; confidence interval for the prediction &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$$\hat{y}_i\in \bar{\hat{y}}_i\pm z_{0.025}\frac{\sigma_{\hat{y}_i}}{\sqrt{B}}$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$z_{0.025}=1.96$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I sort the &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt; values and plot it together with the upper&#xA;and lower bound, I will get the prediction curve with a CI.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My biggest uncertainty relates to step 5). I read in a book &lt;a href=&quot;https://reader.elsevier.com/reader/sd/pii/B9780128117880000056?token=BDF247F6F2EAA467DDC6127D21F373AD489E541F3F7A96B8F821A56B5A165A70F45B047108765812F3559C20E3FEBDFC&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supervised Classification:Quite a Brief Overview, Marco Loog&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When the population standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is known and the parent population is normally distributed &lt;strong&gt;or &lt;span class=&quot;math-container&quot;&gt;$N&amp;gt;30$&lt;/span&gt;&lt;/strong&gt; the &lt;span class=&quot;math-container&quot;&gt;$100(1-\alpha)$&lt;/span&gt; CI for the population mean is given by the symmetrical distribution for the standardized normal distribution &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$\mu\in \bar{x}\pm z_{a/2}\frac{\sigma}{\sqrt{N}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is it correct to say here &lt;span class=&quot;math-container&quot;&gt;$N=B$&lt;/span&gt; (number of bootstrap models or number of resampled trainingsets or number of estimators &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i^b$&lt;/span&gt; ). Does the procedure make sense?&lt;/p&gt;&#xA;" OwnerUserId="36989" LastEditorUserId="36989" LastEditDate="2020-05-21T08:45:50.303" LastActivityDate="2020-05-21T08:45:50.303" Title="Confidence Interval around prediction with bootstrapping" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21232" PostTypeId="2" ParentId="21191" CreationDate="2020-05-17T21:15:29.010" Score="1" Body="&lt;p&gt;It might be the case that if you perform a large number of random rollouts, the &quot;best action&quot; as chosen by the agent without the domain knowledge, is same as the agent with the domain knowledge. I guess what you can do is try to reduce the number of rollouts and see if the performance changes.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-05-17T21:15:29.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21233" PostTypeId="1" CreationDate="2020-05-17T21:25:43.497" Score="2" ViewCount="73" Body="&lt;p&gt;I am currently implementing the very basic version (REINFORCE) of the Monte Carlo policy gradient algorithm. I was wondering if this is the correct gradient for the log of softmax. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \log \pi_{\theta}(s, a)&#xA;&amp;amp;=&#xA;\varphi(s, a)-\mathbb{E}\left[\varphi(s, a)_{\forall a \in A}\right] \\&#xA;&amp;amp;=&#xA;\left(\varphi(s)^T \cdot \theta_{a}\right)-\sum_{\forall a \in A}\left(\varphi(s)^T \cdot \theta_{a}\right) &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\varphi(s)$&lt;/span&gt; is the feature vector at state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if my interpretation of the equation is correct. I ask because, in my implementation, my weights (&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) blow up after a few iterations, and I have a feeling the problem is in this line. &lt;/p&gt;&#xA;" OwnerUserId="36404" LastEditorUserId="36404" LastEditDate="2020-05-18T18:05:09.973" LastActivityDate="2020-05-18T18:05:09.973" Title="Is this the correct gradient for log of softmax?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ClosedDate="2020-05-21T22:49:11.757" ContentLicense="CC BY-SA 4.0" />
  <row Id="21234" PostTypeId="1" CreationDate="2020-05-17T22:31:09.297" Score="2" ViewCount="100" Body="&lt;p&gt;I am working with the K-means clustering algorithm for unsupervised learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the following dataset suitable for the k-means clustering task or not? Why or why not? The dataset has only two features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gnc2O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gnc2O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pjzph.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pjzph.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33670" LastEditorUserId="2444" LastEditDate="2020-05-18T09:43:37.737" LastActivityDate="2020-05-18T09:43:37.737" Title="Is this dataset with only two features suitable for clustering with k-means?" Tags="&lt;datasets&gt;&lt;unsupervised-learning&gt;&lt;data-science&gt;&lt;clustering&gt;&lt;k-means&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21235" PostTypeId="1" CreationDate="2020-05-17T23:00:52.337" Score="2" ViewCount="27" Body="&lt;p&gt;I have the following time-series aggregated input for an LSTM-based model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x(0): {y(0,0): {a(0,0), b(0,0)}, y(0,1): {a(0,1), b(0,1)}, ..., y(0,n): {a(0,n), b(0,n)}}&#xA;x(1): {y(1,0): {a(1,0), b(1,0)}, y(1,1): {a(1,1), b(1,1)}, ..., y(1,n): {a(1,n), b(1,n)}}&#xA;...&#xA;x(m): {y(m,0): {a(m,0), b(m,0)}, y(m,1): {a(m,1), b(m,1)}, ..., y(m,n): {a(m,n), b(m,n)}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x(m)&lt;/code&gt; is a timestep, &lt;code&gt;a(m,n)&lt;/code&gt; and &lt;code&gt;b(m,n)&lt;/code&gt; are features aggregated by the non-temporal sequential key &lt;code&gt;y(m,n)&lt;/code&gt; which might be &lt;code&gt;0...1,000&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0: {90: {4, 4.2}, 91: {6, 0.2}, 92: {1, 0.4}, 93: {12, 11.2}}&#xA;1: {103: {1, 0.2}}&#xA;2: {100: {3, 0.1}, 101: {0.4, 4}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where 90-93, 103, and 100-101 are aggregation keys.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I feed this kind of input to LSTM?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to use non-aggregated data. In that case, I'd get the proper input for LSTM. Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aggregated input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0: {100: {3, 0.1}, 101: {0.4, 4}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Original input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0: 100, 1, 0.05&#xA;1: 101, 0.2, 2&#xA;2: 100, 1, 0&#xA;3: 100, 1, 0.05&#xA;4: 101, 0.2, 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But in that case, the aggregation would be lost, and the whole purpose of aggregation is to minimize the number of steps so that I get 500 timesteps instead of e.g. 40,000, which is impossible to feed to LSTM. If you have any ideas I'd appreciate it.&lt;/p&gt;&#xA;" OwnerUserId="36999" LastEditorUserId="36999" LastEditDate="2020-05-19T16:58:53.073" LastActivityDate="2020-05-19T16:58:53.073" Title="How to feed key-value features (aggregated data) to LSTM?" Tags="&lt;machine-learning&gt;&lt;long-short-term-memory&gt;&lt;data-preprocessing&gt;&lt;structured-data&gt;&lt;feature-engineering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21236" PostTypeId="1" CreationDate="2020-05-18T00:32:17.737" Score="1" ViewCount="46" Body="&lt;p&gt;Today, if you scan an object and want its CAD file (Solidworks/Autocad), you need to use reverse engineering software (Geomagic). This takes time and you need experience of the software tools. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an AI tool/app that does the job automatically? If not, is this a reasonable idea to develop an AI application capable of doing it? What would be the biggest challenges?&lt;/p&gt;&#xA;" OwnerUserId="37156" LastEditorUserId="1671" LastEditDate="2020-05-18T21:56:24.330" LastActivityDate="2020-05-18T21:56:24.330" Title="Is there an AI tool to reverse engineer scanned data to obtain its CAD file?" Tags="&lt;ai-design&gt;&lt;image-recognition&gt;&lt;resource-request&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21237" PostTypeId="1" AcceptedAnswerId="25057" CreationDate="2020-05-18T01:28:13.563" Score="2" ViewCount="387" Body="&lt;p&gt;In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wLI4m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wLI4m.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reason why is stated that this constrains the distribution of the weights of the output to have a standard deviation of 1.&lt;/p&gt;&#xA;&lt;p&gt;Quoted from &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/transformer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformer model for language understanding | TensorFlow&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For example, consider that &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; have a mean of 0 and variance of 1. Their matrix multiplication will have a mean of 0 and variance of &lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt;. Hence, square root of &lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt; is used for scaling (and not any other number) because the matmul of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; should have a mean of 0 and variance of 1, and you get a gentler softmax.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why does this multiplication have a variance of &lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;If I understand this, I will then understand why dividing by &lt;span class=&quot;math-container&quot;&gt;$\sqrt({d_k})$&lt;/span&gt; would normalize to 1.&lt;/p&gt;&#xA;&lt;p&gt;Trying this experiment on 2x2 arrays I get an output of 1.6 variance:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GCe3t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GCe3t.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36486" LastEditorUserId="32410" LastEditDate="2021-05-09T23:50:06.490" LastActivityDate="2021-05-09T23:50:06.490" Title="Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product attention?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21243" PostTypeId="2" ParentId="13959" CreationDate="2020-05-18T06:57:07.493" Score="0" Body="&lt;p&gt;From &lt;a href=&quot;https://heartbeat.fritz.ai/deep-learning-based-video-summarization-a-detailed-exploration-8b1a25946404&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Deep learning-based video summarization — A detailed exploration&amp;quot;&lt;/a&gt; by Surya Remanan:&lt;/p&gt;&#xA;&lt;p&gt;Video summarization can be considered as the process of distilling a raw video into a more compact form without losing much information. In a general video summarization system, image features of video frames are extracted, and then the most representative frames are selected through analyzing the visual variations among visual features. This is done either by taking a holistic view of the entire video or by identifying the local differentiation among the adjacent frames. Most of those attempts rely on global features such as colour, texture, motion information, etc. Clustering techniques are also used for summarization. Video summarization can be categorized into two forms:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Static video summarization (keyframing) and&lt;/li&gt;&#xA;&lt;li&gt;Dynamic video summarization (video skimming)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Static video summaries are composed of a set of keyframes extracted from the original video, while dynamic video summaries are composed of a set of shots and are produced taking into account the similarity or domain-specific relationships among all video shots.&lt;/p&gt;&#xA;&lt;p&gt;Following is an attention-based video summarization model - &lt;a href=&quot;https://github.com/ok1zjf/VASNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch implementation of the ACCV 2018-AIU2018 paper Video Summarization with Attention&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a video summarisation focused models using reinforcement learning - &lt;a href=&quot;https://github.com/KaiyangZhou/vsumm-reinforce&quot; rel=&quot;nofollow noreferrer&quot;&gt; Unsupervised video summarization with deep reinforcement learning (Theano)&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is an LSTM - GAN based approach to video summarization - &lt;a href=&quot;https://github.com/moabitcoin/Adversarial-video-summarization-pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;video summarization lstm-gan pytorch implementation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Microsoft Bing Search has come up with a video summarization technique using thumbnails. &lt;a href=&quot;https://blogs.bing.com/search-quality-insights/2018-08/Intelligent-Search-Video-summarization-using-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intelligent Search: Video summarization using machine learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12861" LastEditorUserId="12630" LastEditDate="2020-10-27T01:31:33.670" LastActivityDate="2020-10-27T01:31:33.670" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21244" PostTypeId="2" ParentId="21003" CreationDate="2020-05-18T07:29:12.990" Score="0" Body="&lt;p&gt;I went into the pytorch code for the spinning up implementation of vanilla policy gradient and from what I could understand, found that they use a learning rate of 1e-3 for training the baseline and do a gradient descent 80 times on the same dataset by default with no termination criteria. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also it is usually impossible to fit the value function completely as we are using a function approximator. The main point is not to get too carried away to get the lowest loss as the main improvement in the agent's performance will happen by doing policy gradient descents rather than trying to aggressively get the accurate value function for the baseline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link for the implementation:&#xA;&lt;a href=&quot;https://github.com/openai/spinningup/blob/master/spinup/algos/pytorch/vpg/vpg.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;spinning up implementation of vpg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36861" LastActivityDate="2020-05-18T07:29:12.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21245" PostTypeId="2" ParentId="7838" CreationDate="2020-05-18T08:50:05.093" Score="0" Body="&lt;p&gt;According to the book &lt;a href=&quot;https://www.cin.ufpe.br/%7Etfl2/artificial-intelligence-modern-approach.9780131038059.25368.pdf#page=27&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (section 1.1), artificial intelligence (AI) has been defined in &lt;em&gt;multiple&lt;/em&gt; ways, which can be organized into 4 categories.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Thinking Humanly&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Thinking Rationally&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Acting Humanly&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Acting Rationally&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The following picture (from the same book) provides 8 definitions of AI, where each box contains 2 definitions that fall into the same category. For example, the definitions in the top-left corner fall into the category &lt;strong&gt;thinking humanly&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/J9B2y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J9B2y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is also the &lt;a href=&quot;https://en.wikipedia.org/wiki/AI_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;AI effect&lt;/strong&gt;&lt;/a&gt;, which Pamela McCorduck describes (in her book &lt;a href=&quot;https://pdfs.semanticscholar.org/965b/39ad83c545849d473ce30cfc3d569f6e3828.pdf?_ga=2.48167655.1174938146.1610754221-826121516.1600962951#page=234&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machines Who Think: A Personal Inquiry into the History and Prospects of Artificial Intelligence&lt;/a&gt;, p. 204) as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;it's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something — play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, but &lt;em&gt;that's not thinking&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3005" LastEditorUserId="2444" LastEditDate="2021-01-15T23:55:02.310" LastActivityDate="2021-01-15T23:55:02.310" CommentCount="1" CommunityOwnedDate="2021-01-15T23:55:11.243" ContentLicense="CC BY-SA 4.0" />
  <row Id="21247" PostTypeId="2" ParentId="21234" CreationDate="2020-05-18T09:37:43.733" Score="2" Body="&lt;p&gt;One problem with clustering algorithms is that they will typically find you a solution, ie they will split your data set into clusters, but it will find you a structure even if there isn't one. Your data looks like it could consist of about 5 to 7 clusters, but it could equally well just be 2 or only 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need to do after the clustering is to assess the quality of the result. I recommend having a look at &lt;em&gt;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316801&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finding Groups in Data&lt;/a&gt;&lt;/em&gt; by Kaufman &amp;amp; Rousseeuw. They discuss various clustering algorithms and also a procedure that works out how cohesive your clusters are. Though it is 30 years old, it is an excellent book on the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also have the issue of choosing a value for &lt;em&gt;k&lt;/em&gt; in your clustering: I usually start with two, and increase it from there; at each step I compute the cohesion of the result using their method, until I get the best score. This is an objective way of finding a good value for &lt;em&gt;k&lt;/em&gt; and usually a reasonable clustering result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ultimate test, of course, is then if looking at the result makes sense to you. No cluster algorithm can do that for you.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-05-18T09:37:43.733" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21248" PostTypeId="2" ParentId="21237" CreationDate="2020-05-18T10:03:21.550" Score="0" Body="&lt;p&gt;It might help to take two small matrices that match the assumptions (mean of zero and variance of one) and just do the matrix multiplication.  The dimensionality of K scales Q in the multiplication, scaling the variance simultaneously.&lt;/p&gt;&#xA;" OwnerUserId="30426" LastActivityDate="2020-05-18T10:03:21.550" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21249" PostTypeId="1" CreationDate="2020-05-18T10:54:14.903" Score="1" ViewCount="68" Body="&lt;p&gt;How to best make use of learning rate scheduling in reinforcement learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, a low learning rate towards the end to fine-tune what you've learned with subtle updates makes sense. But I don't see why over training time this should be linearly brought down. Wouldn't this increase overfitting too, as it promotes an early adopted policy to get further and further finetuned for the rest of the training? &#xA;Wouldn't it be better to keep it constant over the entire training so that when the agent finds novel experiences later, it still has a high enough learning rate to update its model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also don't really know how these modern deep RL papers do it. The starcraft II paper by DeepMind, and the OpenAI hide and seek paper don't mention learning rate schedules for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or are there certain RL environments where it's actually best to use something like a linear learning rate schedule?&lt;/p&gt;&#xA;" OwnerUserId="31180" LastEditorUserId="2444" LastEditDate="2020-05-18T11:05:46.997" LastActivityDate="2021-02-26T23:02:50.233" Title="How to best make use of learning rate scheduling in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;learning-rate&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21250" PostTypeId="1" CreationDate="2020-05-18T11:25:13.807" Score="1" ViewCount="32" Body="&lt;p&gt;Can anyone list the differences between deep Belief network (DBN), restricted Boltzmann machine (RBM), deep Boltzmann machine (DBM) using simple examples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Links to other resources are also appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-18T12:22:25.113" LastActivityDate="2020-05-18T12:22:25.113" Title="What are the differences between a deep belief network, a restricted Boltzmann machine and a deep Boltzmann machine?" Tags="&lt;comparison&gt;&lt;boltzmann-machine&gt;&lt;deep-belief-network&gt;&lt;restricted-boltzmann-machine&gt;&lt;deep-boltzmann-machine&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21251" PostTypeId="5" CreationDate="2020-05-18T11:28:42.903" Score="0" Body="&lt;p&gt;A restricted Boltzmann machine (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-18T22:13:51.503" LastActivityDate="2020-05-18T22:13:51.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21252" PostTypeId="4" CreationDate="2020-05-18T11:28:42.903" Score="0" Body="For questions related to Restricted Boltzmann machine" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-18T22:16:48.930" LastActivityDate="2020-05-18T22:16:48.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21253" PostTypeId="5" CreationDate="2020-05-18T11:29:57.690" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-18T11:29:57.690" LastActivityDate="2020-05-18T11:29:57.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21254" PostTypeId="4" CreationDate="2020-05-18T11:29:57.690" Score="0" Body="For questions related to Deep belief network. In machine learning, a deep belief network is a generative graphical model, or alternatively a class of deep neural network, composed of multiple layers of latent variables, with connections between the layers but not between units within each layer" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-18T22:16:44.140" LastActivityDate="2020-05-18T22:16:44.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21255" PostTypeId="5" CreationDate="2020-05-18T11:32:44.217" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-18T11:32:44.217" LastActivityDate="2020-05-18T11:32:44.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21256" PostTypeId="4" CreationDate="2020-05-18T11:32:44.217" Score="0" Body="For questions related to the Deep Boltzmann machine (DBM). The deep Boltzmann machine (DBM) is an undirected deep network that connects the binary units of adjacent layers using the RBMs" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-18T22:16:33.377" LastActivityDate="2020-05-18T22:16:33.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21257" PostTypeId="5" CreationDate="2020-05-18T12:06:52.027" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-18T12:06:52.027" LastActivityDate="2020-05-18T12:06:52.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21258" PostTypeId="4" CreationDate="2020-05-18T12:06:52.027" Score="0" Body="For questions related to restricted Boltzmann machines, which is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-18T12:06:52.027" LastActivityDate="2020-05-18T12:06:52.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21259" PostTypeId="5" CreationDate="2020-05-18T12:11:02.460" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-18T12:11:02.460" LastActivityDate="2020-05-18T12:11:02.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21260" PostTypeId="4" CreationDate="2020-05-18T12:11:02.460" Score="0" Body="For questions related to Boltzmann machines (aka stochastic Hopfield network with hidden units), which are stochastic recurrent neural networks, more precisely, they are Markov random fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-18T12:11:02.460" LastActivityDate="2020-05-18T12:11:02.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21261" PostTypeId="1" CreationDate="2020-05-18T12:54:58.817" Score="2" ViewCount="53" Body="&lt;p&gt;In deep learning, the concept of validation loss is to ensure that the model being trained is not currently overfitting the data. Is there a similar concept of overfitting in deep q learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that I have a fixed number of experiences already in a replay buffer and I train a q network by sampling from this buffer, would computing the validation loss (separate from the experiences in the replay buffer) help me to decide whether I should stop training the network? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, If my validation loss increases even though my train loss continues to decrease, I should stop training the training. Does deep learning validation loss also apply in the deep q network case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to clarify again, no experiences are collected during the training of the DQN.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-18T13:27:23.557" LastActivityDate="2020-05-18T13:27:23.557" Title="Does the concept of validation loss apply to training deep Q networks?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21262" PostTypeId="2" ParentId="21220" CreationDate="2020-05-18T13:44:03.130" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any practical application of knowing whether a concept class is PAC-learnable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you know that a concept class is PAC-learnable (i.e. &lt;a href=&quot;https://dl.acm.org/doi/pdf/10.1145/76359.76371#page=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;its VC dimension is finite&lt;/a&gt;), then there's a possibility that you can design an algorithm that can find a function (or concept) that is arbitrarily close to your target (or desired) function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not really an application, but a consequence, which can lead to applications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, note that asking if PAC learning is useful in practice is like asking if special relativity is useful in practice. Yes, they are useful, but in the sense that they can be used to predict the outcomes of an experiment or explain the rules in their specific context. In the case of machine learning, PAC learning can be used to explain e.g. the probably required number of data points needed to learn a concept (a target function) approximately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://ai.stackexchange.com/q/17291/2444&quot;&gt;Are PAC learning and VC dimension relevant to machine learning in practice?&lt;/a&gt; for more concrete &quot;applications&quot; of PAC-learning and the VC dimension.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-18T13:54:49.370" LastActivityDate="2020-05-18T13:54:49.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21264" PostTypeId="1" AcceptedAnswerId="21290" CreationDate="2020-05-18T15:11:12.333" Score="3" ViewCount="101" Body="&lt;p&gt;I'm trying to understand RL applied to time series (so with infinite horizon) which have a continous state space and a discrete action space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, some preliminary questions: in this case, what is the optimal policy? Given the infinite horizon there is no terminal state but only an objective to maximise the rewards, so I can't run more than one episode, is it correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, what is the difference between on-policy and off-policy learning given this framework? &lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="1847" LastEditDate="2020-05-19T08:31:27.243" LastActivityDate="2020-05-19T19:46:15.063" Title="What is the difference between on-policy and off-policy for continuous environments?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;off-policy-methods&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21265" PostTypeId="2" ParentId="21162" CreationDate="2020-05-18T15:33:18.070" Score="1" Body="&lt;p&gt;I think you're looking for something like &lt;a href=&quot;https://www.comet.ml/site/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comet.ml&lt;/a&gt; or &lt;a href=&quot;https://github.com/IDSIA/sacred&quot; rel=&quot;nofollow noreferrer&quot;&gt;IDSIA's sacred&lt;/a&gt;, but bear in mind that these tools can also be combined with code-hosting services such as Github.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-18T15:33:18.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21266" PostTypeId="1" CreationDate="2020-05-18T15:37:59.747" Score="2" ViewCount="343" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/2405/2444&quot;&gt;This old question&lt;/a&gt; has no definitive answer yet, that's why I am asking it here again. I also asked this same question &lt;a href=&quot;https://stackoverflow.com/q/61829580/3924118&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If I'm doing policy gradient in Keras, using a loss of the form:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rewards*cross_entropy(action_pdf, selected_action_one_hot)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I manage negative rewards?&lt;/p&gt;&#xA;&lt;p&gt;I've had success with this form in cases where the reward is always positive, but it does not train with negative rewards. The failure mode is for it to drive itself to very confident predictions all the time, which results in very large negative losses due to induced deviation for exploration. I can get it to train by clipping rewards at zero, but this throws a lot of valuable information on the table (only carrots, no sticks).&lt;/p&gt;&#xA;" OwnerUserId="27616" LastEditorUserId="2444" LastEditDate="2020-11-01T22:48:11.860" LastActivityDate="2020-11-01T22:48:11.860" Title="How do you manage negative rewards in policy gradients?" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;policy-gradients&gt;&lt;rewards&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21267" PostTypeId="1" AcceptedAnswerId="21298" CreationDate="2020-05-18T16:18:00.477" Score="6" ViewCount="183" Body="&lt;p&gt;I am trying to understand the difference between a Bayesian Network and a Markov Chain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I search for this one the web, the unanimous solution seems to be that a Bayesian Network is directional (i.e. it's a DAG) and a Markov Chain is not directional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, often a Markov Chain example is overtime, where the weather today is impacting the weather tomorrow, but the weather tomorrow is not (obviously) impacting the weather today. So I am quite confused how is a Markov Chain not directional?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I seem to be missing something here. Can someone please help me understand?&lt;/p&gt;&#xA;" OwnerUserId="36997" LastEditorUserId="2444" LastEditDate="2020-05-18T19:55:07.270" LastActivityDate="2020-05-19T13:37:08.703" Title="What is the difference between a Bayesian Network and a Markov Chain?" Tags="&lt;markov-chain&gt;&lt;bayesian-network&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21268" PostTypeId="2" ParentId="21266" CreationDate="2020-05-18T17:22:31.803" Score="1" Body="&lt;p&gt;You don't need to manage negative rewards separately, if you implemented the algorithm correctly it will work regardless if the rewards are negative or not. You seem to be using rewards for the loss but you should be using the return which is the sum of the rewards for some state action pair from that point until the end of trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also seem to be missing &lt;span class=&quot;math-container&quot;&gt;$-$&lt;/span&gt; sign from the loss. The objective function for the vanilla policy gradient algorithm (REINFORCE) which we want to maximize is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;J = \sum_a \pi(a|s) q_{\pi}(s, a)&#xA;\end{equation}&lt;/span&gt;&#xA;It can be shown that the gradient sample for this policy gradient method is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\nabla J = G_t \nabla \log (\pi(A_t|S_t))&#xA;\end{equation}&lt;/span&gt;&#xA;so in TensorFlow you should define your loss as&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;J = - G_t \pi(A_t|S_t)&#xA;\end{equation}&lt;/span&gt;&#xA;We need the &lt;span class=&quot;math-container&quot;&gt;$-$&lt;/span&gt; because in TensorFlow you use minimizers, but we want to maximize this function so minimizing this loss is same as maximizing the objective function. In conclusion, the code similar to what you wrote, should be&lt;br&gt;&#xA;&lt;code&gt;-return * cross_entropy(action_pdf, selected_action_one_hot)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out in the comment we don't actually need &lt;span class=&quot;math-container&quot;&gt;$-$&lt;/span&gt; because it is already included in &lt;code&gt;cross_entropy&lt;/code&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2020-05-18T19:11:50.887" LastActivityDate="2020-05-18T19:11:50.887" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21270" PostTypeId="2" ParentId="21267" CreationDate="2020-05-18T17:57:25.787" Score="1" Body="&lt;p&gt;The main difference between a Bayesian network and a Markov chain is not that a Markov Chain is not directional, it is that the graph of the Bayesian network is not trivial whereas the graph of a Markov chain would be somewhat trivial, as all the previous &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; nodes would just point to the current node. To illustrate further why this would be trivial, we let each node represent a random variable &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt;. Then the nodes representing &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$ t-k \leq i &amp;lt; t$&lt;/span&gt; would be connected by a directed edge to &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt;. That is, the edges &lt;span class=&quot;math-container&quot;&gt;$(X_i, X_t) \in E$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$ t-k \leq i &amp;lt; t$&lt;/span&gt; and where &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the set of edges of the graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate this please see the examples below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that we have a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th order Markov chain, then by definition we have &lt;span class=&quot;math-container&quot;&gt;$\forall t &amp;gt; k$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X_t = x | X_0,...,X_{t-1}) = \mathbb{P}(X_t = x | X_{t-k},...,X_{t-1})$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference between the above definition and the definition of a Bayesian Network is that due to the direction of the graph we can have different dependencies for each &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt;. Consider the Bayesian Network in the Figure below &#xA;&lt;a href=&quot;https://i.stack.imgur.com/dsyLj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dsyLj.png&quot; alt=&quot;Bayesian Network&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would get that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X_4 = x| X_1, X_2, X_3) = \mathbb{P}(X_4 = x | X_2, X_3)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X_5 = x| X_1, X_2, X_3, X_4) = \mathbb{P}(X_5 = x | X_3)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the past events that the current random variable depends on don't have to have the same 'structure' in a Bayesian Network as in a Markov Chain. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-05-19T13:37:08.703" LastActivityDate="2020-05-19T13:37:08.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21271" PostTypeId="1" CreationDate="2020-05-18T20:29:46.563" Score="2" ViewCount="21" Body="&lt;p&gt;I am trying to understand the following statement taken from the paper &lt;a href=&quot;https://arxiv.org/pdf/1812.08434.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Neural Networks: A Review of Methods and Applications&lt;/a&gt; (2019). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Standard neural networks like CNNs and RNNs cannot handle the graph input properly in that they stack the feature of nodes by a specific order.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This statement is confusing to me. I have not used CNNs/RNNs for non-Euclidean data before, so perhaps that's where my understanding falls off. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do CNNs/RNNs stack the feature of nodes by a specific order?&lt;/p&gt;&#xA;" OwnerUserId="36486" LastEditorUserId="2444" LastEditDate="2020-05-18T21:25:20.563" LastActivityDate="2020-05-18T21:25:20.563" Title="How do CNNs or RNNs &quot;stack the feature of nodes by a specific order&quot;?" Tags="&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;terminology&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21272" PostTypeId="2" ParentId="21249" CreationDate="2020-05-18T20:36:25.353" Score="0" Body="&lt;p&gt;I have not used learning rate schedules, but I do have experience with adjustable learning rates.&lt;/p&gt;&#xA;&lt;p&gt;The Keras callback &lt;code&gt;ReduceLROnPlateau&lt;/code&gt; is useful for adjusting the learning rate. If you use it to monitor the validation loss versus training loss, you will avoid the danger of overfitting. Also, you can use the &lt;code&gt;ModelCheckpoint&lt;/code&gt; callback to save the model with the lowest validation loss and use that to make predictions. The documentation is &lt;a href=&quot;https://keras.io/api/callbacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I look at the validation loss as a deep valley in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; space, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of trainable parameters. As you progress down the valley, it becomes increasingly narrower, so it is best to reduce the learning rate to get further down the valley (closer to the minimum). With the adjustable learning rate, you can start with a larger initial rate that converges faster, then reduces as needed to achieve a minimum loss.&lt;/p&gt;&#xA;&lt;p&gt;I wrote a custom callback that initially monitors the training loss and adjusts the learning rate based on that until the training accuracy achieves 95%, then it switches to adjusting the learning rate based on validation loss.&lt;/p&gt;&#xA;&lt;p&gt;I also am experimenting with a slightly different approach to training. On a given epoch, assume the quantity you are monitoring does NOT improve. That means that you have moved to a point in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; space (value of the weights) that is NOT as &amp;quot;good&amp;quot; as the point you were at in the previous epoch. So, instead of training from the point you are at in the current epoch, I set the weights back to what they were for the previous (better) epoch, reduce the learning rate then continue training from there. This appears to work rather well.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-09-29T22:10:57.593" LastActivityDate="2020-09-29T22:10:57.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21273" PostTypeId="1" CreationDate="2020-05-18T21:18:18.600" Score="2" ViewCount="26" Body="&lt;p&gt;I am solving a problem of image classification of the image dataset for 3 classes. Dataset is highly imbalanced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How will sampling (either over- or under-sampling) work in that case? Should I remove (or add) any random number of images, or should I follow some pattern?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of CSV data, the general rule is to do PCA, and then remove the data points, but how to do it in the image dataset? Is there any other way to handle this problem?&lt;/p&gt;&#xA;" OwnerUserId="38341" LastEditorUserId="2444" LastEditDate="2020-05-18T21:35:59.780" LastActivityDate="2020-05-18T21:35:59.780" Title="How does sampling works in case of imbalanced image datasets?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21274" PostTypeId="1" AcceptedAnswerId="21352" CreationDate="2020-05-18T21:19:27.980" Score="4" ViewCount="58" Body="&lt;p&gt;Let's say you are training a neural network in an RL setting, where the state (i.e. features/input data) can be the same for multiple successive steps (~typically around 8 steps) of an episode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, an initial state might consist of the following values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[30, 0.2, 0.5, 1, 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then again the same state could be fed into the neural network for e.g. 6-7 times more, resulting in ultimately the following input arrays:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[30, 0.2, 0.5, 1, 0], &#xA; [30, 0.2, 0.5, 1, 0], &#xA; ..., &#xA; [30, 0.2, 0.5, 1, 0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that the value 0 in the feature set depicts that the weight for this feature results in insignificant value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what about the repetition of values? How does that affect learning, if it does at all? Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt; I am going to provide more information as requested in the comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I did not provide this information in the first place, is because I thought there would be similarities in such cases across problems/domains of application. But it is also fine to make it more specific.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The output of the network is a probability among two paths. Our network has to select an optimal path based on some gathered network statistics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I will be using A3C, as similar work in the bibliography has made progress.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The reason the agent is staying in the same state is the fact that the protocol &lt;strong&gt;&lt;em&gt;can also&lt;/em&gt;&lt;/strong&gt; make path selection decisions at the same time, without an actual update of network statistics. So in that case, you would have the same RTT for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i. This is a product of concurrency in the protocol&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii. It is expected behavior &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35978" LastEditorUserId="35978" LastEditDate="2020-05-20T19:53:27.713" LastActivityDate="2020-05-20T20:01:25.373" Title="How does the repetition of features across states at different time steps affect learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21275" PostTypeId="1" CreationDate="2020-05-18T21:25:39.777" Score="0" ViewCount="91" Body="&lt;p&gt;In graph neural networks, the Banach fixed-point theorem and Jacobi method it is described that the transition from one state to another be defined by a contraction map with a fixed-point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The autor states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The transition function in a GNN is assumed to be a contractive mapping with respect to the nodes’ state&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But in practice these output and transition functions are modeled using feed-forward networks, specifically an RNN. Which to my knowledge can represent any function and is not constrained to be a contraction map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My learning is from this article &lt;a href=&quot;https://towardsdatascience.com/graph-neural-networks-20d0f8da7df6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/graph-neural-networks-20d0f8da7df6&lt;/a&gt;, which describes what I said above but somewhat vaguely.&lt;/p&gt;&#xA;" OwnerUserId="36486" LastEditorUserId="36486" LastEditDate="2020-05-18T21:46:03.733" LastActivityDate="2020-05-18T21:46:03.733" Title="How can feedforward neural networks act as contraction maps?" Tags="&lt;feedforward-neural-network&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21276" PostTypeId="1" CreationDate="2020-05-18T23:59:32.193" Score="2" ViewCount="27" Body="&lt;ul&gt;&#xA;&lt;li&gt;Is there any published research on the information-carrying capacity of the human face?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here I mean &quot;how much information can be conveyed via facial expressions &amp;amp; micro-expressions&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a subject of interest because the human face is arguably &quot;the most interesting&quot; single thing for humans, since it's likely the first real pattern we recognize as infants, and conveys so much non-verbal communication than can relate to achievement of a goal or identification of a mortal threat.  (Dogs similarly are said to have co-evolved to read human faces. Film acting as &quot;the art of the closeup&quot; also validates this viewpoint.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, I'm tying to get a sense of how complex is the set of human facial expressions, what is the computational complexity of the range of problems related to identification of the range of possible expressions, and the emulation of such expressions to imitate a human agent.  (i.e. these techniques can be used to &quot;read&quot; a human subject or manipulate a human subject.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Well researched articles &amp;amp; blogs would also be welcome.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="2444" LastEditDate="2020-05-19T10:14:05.573" LastActivityDate="2020-05-19T10:14:05.573" Title="Is there any published research on the information-carrying capacity of the human face?" Tags="&lt;reference-request&gt;&lt;human-like&gt;&lt;facial-recognition&gt;&lt;complexity-theory&gt;&lt;information-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21277" PostTypeId="1" CreationDate="2020-05-19T01:32:57.263" Score="2" ViewCount="52" Body="&lt;p&gt;I've researched online and seen many papers on the use of RNNs (like LSTMs or GRUs) to autocomplete for, say, a search engine, character by character. Which makes sense since it inherently predicts character-by-character in a sequential manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to use the transformer architecture instead to do search autocomplete? If so, how might such a model be adapted? &lt;/p&gt;&#xA;" OwnerUserId="35867" LastEditorUserId="2444" LastEditDate="2020-05-19T09:45:20.013" LastActivityDate="2020-05-19T09:45:20.013" Title="Can you use transformer models to do autocomplete tasks?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21278" PostTypeId="5" CreationDate="2020-05-19T02:29:20.303" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-19T02:29:20.303" LastActivityDate="2020-05-19T02:29:20.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21279" PostTypeId="4" CreationDate="2020-05-19T02:29:20.303" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-19T02:29:20.303" LastActivityDate="2020-05-19T02:29:20.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21280" PostTypeId="1" AcceptedAnswerId="21289" CreationDate="2020-05-19T03:39:50.190" Score="4" ViewCount="79" Body="&lt;p&gt;I'm building a really simple experiment, where I let an agent move from the bottom-left corner to the upper-right corner of a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; grid world.&lt;/p&gt;&#xA;&lt;p&gt;I plan to use DQN to do this. I'm having trouble handling the starting point: what if the Q network's prediction is telling the agent to move downward (or leftward) at the beginning?&lt;/p&gt;&#xA;&lt;p&gt;Shall I program the environment to immediately give a &lt;span class=&quot;math-container&quot;&gt;$-\infty$&lt;/span&gt; reward and end this episode? Will this penalty make the agent &amp;quot;fear&amp;quot; of moving left again in the future, even if moving left is a possible choice?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="37178" LastEditorUserId="2444" LastEditDate="2020-11-14T17:55:34.493" LastActivityDate="2020-11-14T18:07:54.587" Title="How should I handle invalid actions in a grid world?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21281" PostTypeId="2" ParentId="10003" CreationDate="2020-05-19T04:05:27.760" Score="0" Body="&lt;p&gt;So there's pictures of low level activation maps, and some gradient based information where yoy take the deriviative of the output with respect to the input and generate a heatmap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I kind of have my doubts on how usefull this is in general, imo it kind of is creating a fallacious illusion of understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's some additional research using blurring to figure out the relevant features also but again I have my doubts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the most usefull is generating images by optimizing your class score. You can learn how badly your CNN actually labels things(doing this makes you realize quickly that CNN are garbage at actually understanding and incredibly easy to trick).&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-19T04:05:27.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21282" PostTypeId="1" AcceptedAnswerId="21387" CreationDate="2020-05-19T05:02:46.307" Score="2" ViewCount="92" Body="&lt;p&gt;The gradient of the softmax eligibility trace is given by the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \log(\pi_{\theta}(a|s)) &amp;amp;= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\&#xA;&amp;amp;= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a')&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this equation derived?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following relation is true:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_{\theta} \log(\pi_{\theta}(a|s)) &amp;amp;= \frac{\nabla_{\theta} \pi_{\theta}(a|s)}{\pi_{\theta}(a|s)} \tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the following relation must also be true:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{\nabla_{\theta} \pi_{\theta}(a|s)}{\pi_{\theta}(a|s)}  &amp;amp;=\phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a')&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, why would this be the case? Probably, you just need to answer my question above because \ref{1} is true and it's just &lt;a href=&quot;https://ai.stackexchange.com/a/20665/2444&quot;&gt;the rule to differentiate a logarithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36404" LastEditorUserId="2444" LastEditDate="2020-05-21T23:03:04.970" LastActivityDate="2020-05-21T23:03:04.970" Title="How do I derive the gradient with respect to the parameters of the softmax policy?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;eligibility-traces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21283" PostTypeId="1" CreationDate="2020-05-19T05:17:14.900" Score="1" ViewCount="50" Body="&lt;p&gt;Since the hidden layers of a CNN work as a trainable feature extractor, more detailed content based on a larger number of pixels shall require bigger filter sizes. But for cases where localized differences are to receive greater attention, smaller filter sizes are required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is a lot of topic on the internet regarding CNN and most of them have a simple explanation about Convolution Layer and what it is designed for, but they don’t explain&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;How many convolution layers are required?&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;What filters should I use in those convolution layers?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-19T10:05:27.497" LastActivityDate="2020-05-19T10:05:27.497" Title="How do we choose the filters for the convolutional layer of a convolution neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;convolution&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21284" PostTypeId="1" CreationDate="2020-05-19T05:29:00.697" Score="1" ViewCount="53" Body="&lt;p&gt;Let's say we have several vector points. My goal is to distinguish the vectors, so I want to make them far from each other.  Some of them are already far from each other, but some of them can be positioned very closely.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to get a certain mapping function that can separate such points that are close to each other, while still preserving the points that are already far away from each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not care what is the form of the mapping. Since the mapping will be employed as pre-processing, it does not have to be differentiable or even continuous.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this problem is somewhat similar to 'minimizing the maximum distance ratio between the points'. Maybe this problem can be understood as stretching the crushed graph to a sphere-like isotropic graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I googled it for an hour, but it seems that the people are usually interested in selecting the points that have such nice characteristics from a bunch of data, rather than mapping an existing vector points to a better one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in conclusion, I could not find anything useful.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you can think 'the neural network will naturally learn it while solving classification problem'. But it failed. Because it is already struggling with too many burdens. So, this is why I want to help my network with pre-processing.  &lt;/p&gt;&#xA;" OwnerUserId="18139" LastEditorUserId="2444" LastEditDate="2020-05-19T10:12:29.717" LastActivityDate="2021-02-14T16:05:15.467" Title="Can I find a mapping that minimizes the maximum distance ratio of certain vectors?" Tags="&lt;deep-learning&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21285" PostTypeId="1" CreationDate="2020-05-19T05:43:52.500" Score="0" ViewCount="91" Body="&lt;p&gt;In &lt;a href=&quot;http://www.itvictoria.edu.mx/alumnos/Programas_2016/Ing%20Electronica/asignaturas/Control%20Digital.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;, the terms &quot;Redes Neuronales estáticas monovariables&quot; and &quot;Redes Neuronales estáticas multivariables&quot; are mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are mono-variable and multi-variable neural networks? Is it the same as a multi-layer or uni-layer NN?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have searched about multivariable/mono-variable static/dynamic neural networks in some books, but at least in those books there's no information about these topics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the idea it refers to the inputs/outputs, but I'm not sure.&lt;/p&gt;&#xA;" OwnerUserId="37183" LastEditorUserId="2444" LastEditDate="2020-05-19T17:12:23.293" LastActivityDate="2021-05-14T21:01:50.227" Title="What are mono-variable and multi-variable neural networks?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21286" PostTypeId="1" AcceptedAnswerId="21288" CreationDate="2020-05-19T06:08:46.437" Score="4" ViewCount="200" Body="&lt;p&gt;In the textbook &quot;Reinforcement Learning: An Introduction&quot;, by Richard Sutton and Andrew Barto, the pseudo code for Policy Evaluation is given as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nGbDN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nGbDN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The update equation for &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; comes from the Bellman equation for &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; which is mentioned below (the update equation) for your convenience:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_{k+1}(s) = \sum_{a} \pi(a|s)\sum_{s',r}p(s',r|s,a)[r+\gamma v_{k}(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in Policy Iteration, the Policy Evaluation comes in stage 2, as mentioned in the following pseudo code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HEbLW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HEbLW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, in the policy Evaluation stage, &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; is updated using a different equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align} v_{k+1}(s) = \sum_{s',r}p(s',r|s,\pi (s))[r + \gamma v_{k}(s)] \end{align}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$a = \pi(s)$&lt;/span&gt; is used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please help me in understanding why this change is made in Policy Iteration? Are the two equations the same?&lt;/p&gt;&#xA;" OwnerUserId="37181" LastEditorUserId="37181" LastEditDate="2020-05-27T12:04:20.970" LastActivityDate="2020-05-27T12:04:20.970" Title="Why is update rule of the value function different in policy evaluation and policy iteration?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policy-iteration&gt;&lt;policy-evaluation&gt;&lt;dynamic-programming&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21288" PostTypeId="2" ParentId="21286" CreationDate="2020-05-19T07:38:56.250" Score="4" Body="&lt;p&gt;Yes, the two update equations are equivalent. As an aside, technically the equation you give is not the Bellman equation, but the update step re-written as an equation - in the Bellman equation instead of &lt;span class=&quot;math-container&quot;&gt;$v_{k+1}(s)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$v_{k}(s)$&lt;/span&gt; (showing iterations of approximate value functions), you would have &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; (representing the true value of a state under policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The difference between the equations is that&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In the first case of Policy Evaluation, in order to be general, a stochastic policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s): \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R} = Pr\{A_t = a|S_t =s\}$&lt;/span&gt; is used. That means to get the expected value, you must sum over all possible actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and weight them by the policy function output.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of Policy Iteration, a deterministic policy &lt;span class=&quot;math-container&quot;&gt;$\pi(s): \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt; is used. For that, you don't need to know all possible values of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; for probabilities, but use the output of the policy function directly as the action that is taken by the agent. That action therefore has a probability of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; of being chosen by the policy in the given state.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The equation used in Policy Iteration is simplified for a deterministic policy. If you want you could represent the policy using &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and use the same equation as for Policy Evaluation. If you do that, you would also need to alter the Policy Improvement policy update step to something like:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a_{max} \leftarrow \text{argmax}_a\sum_{r,s'}p(r,s'|s,a)[r + \gamma V(s')]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{ for each } a \in \mathcal{A(s)}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\qquad \pi(a|s) \leftarrow 1 \text{ if } a = a_{max}, 0 \text{ otherwise }$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Doing this will result in exactly the same value function and policy as before. The only reason to do this would be to show the equivalence between the two sets of update equations when dealing with a deterministic policy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-19T13:00:18.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21289" PostTypeId="2" ParentId="21280" CreationDate="2020-05-19T08:20:41.047" Score="2" Body="&lt;p&gt;In a toy environment, this is a choice you can make relatively freely, depending on what you want to achieve with the learning challenge.&lt;/p&gt;&#xA;&lt;p&gt;It may help if you think through what the actual consequences for making the &amp;quot;wrong&amp;quot; move are in your environment. There are a few self-consistent options:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The move simply cannot be made and count as playing the game as intended. In which case, do not allow the agent to make that choice. You can achieve that by filtering the list of choices that the agent is allowed to make. In DQN that will mean supplying an action mask to the agent based on the state so it does not include the action at the stage it makes a choice. This &amp;quot;available actions&amp;quot; function is usually coded as part of the environment.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The move can be attempted, but results in no change to state (e.g. the agent bumps into a wall). If the goal is to reach a certain state in shortest possible time, then you will typically have 0 reward and a discount factor, or negative reward for each attempted action. Either way, the agent should learn that the move was wasted and avoid it after a few iterations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The move can be attempted, but results in disaster (e.g. the agent falls off a cliff). This is the case where a large negative reward plus ending the episode is appropriate. However, don't use infinite rewards, positive or negative, because that will cause significant problems with numeric stability. Simply large enough to offset any interim positive rewards associated with that direction should be adequate. For a simple goal-seeking environment with no other positive rewards than reaching the goal, ending the episode early is already enough.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;When you don't have a toy environment where you get to decide, then the three basic scenarios above can still help. For instance, in most board games we are not interested in having the agent learn the rules for valid moves when they are already supplied by the environment, so the first scenario applies - only select actions from the valid ones as provided by the environment.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-14T18:07:54.587" LastActivityDate="2020-11-14T18:07:54.587" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21290" PostTypeId="2" ParentId="21264" CreationDate="2020-05-19T09:19:06.267" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;First, some preliminary questions: in this case, what is the optimal policy?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is the policy that maximises &lt;em&gt;return&lt;/em&gt; from any given time step &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt;. You need to be careful with your definition of &lt;em&gt;return&lt;/em&gt; with continuing environments. The simple expected sum of future rewards is likely to be positive or negative infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three basic approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Set an arbitrary finite horizon &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=1}^{h} R_{t+k}$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use discounting, with discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=0}^{\infty} \gamma^k R_{t+k+1}$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use average reward per time step,  &lt;span class=&quot;math-container&quot;&gt;$\bar{R} = \lim\limits_{h \to \infty}\frac{1}{h} \sum_{k=1}^{h} R_{t+k}$&lt;/span&gt;, which leads to thinking in terms of &lt;em&gt;differential return&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=1}^{\infty} (R_{t+k} - \bar{R})$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With large enough horizon (so that state is ergodic) or large enough &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; (close to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;), these approaches are similar and should result in approximately the same policy. The difference is in how you construct an agent in detail to solve the problem of maximising the value.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given the infinite horizon there is no terminal state but only an objective to maximise the rewards, so I can't run more than one episode, is it correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The term &lt;em&gt;episode&lt;/em&gt; becomes meaningless. It may be simpler to think of this differently though - you are trying to solve a non-episodic problem here, in that there is no natural separation of the process into separate meaningful episodes. No physical process is actually infinite, that's just a theoretical nicety.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, if you can run your environment in simulation, or multiple versions of it for training purposes, then you do start and stop pseudo-episodes. You don't treat them as episodes mathematically - i.e. there is no terminal state, and you can never obtain a simple episodic return value. However, you can decide to stop an environment and start a new one from a different state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if there is only one real environment, you can sample sections of it for training, or attempt to use different agents over time, each of which is necessarily finite in nature.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consequently, what is the difference between on-policy and off-policy learning given this framework?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The notions of on-policy and off-policy are entirely separate from episodic vs continuing environments. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On-policy agents use a single policy both to select actions (the behaviour policy), and as the learning target. When the learnt policy is updated with new information that immediately affects the behaviour of the agent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Off-policy agents use two or more policies. That is one or more &lt;em&gt;behaviour&lt;/em&gt; policies that select actions (the behaviour policy), and a target policy that is learned, typically the best guess at the optimal policy given data so far.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These things do not change between episodic and continuing tasks, and many algorithms remain identical when solving episodic vs continuing problems. For example, DQN requires no special changes to support continuing tasks, you can just set a high enough discount factor and use it as-is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You cannot wait until the end of an episode, so certain update methods won't work. However, value bootstrapping used in temporal difference (TD) learning still works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, you will want to address the differences in the definition of return. Using an average reward setting typically means looking at a differential return for calculating TD targets for example.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-19T19:46:15.063" LastActivityDate="2020-05-19T19:46:15.063" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21291" PostTypeId="1" CreationDate="2020-05-19T10:07:43.050" Score="1" ViewCount="47" Body="&lt;p&gt;Given e.g. 1M vectors of &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; floating points each, where every point in vectors is sampled from a uniform distribution between &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to have the bottleneck of the AE network with size 1? In other words, without caring about generalization, is it possible to train a network, where, given only 1 encoded value, it can recreate any of the 1M examples?&lt;/p&gt;&#xA;" OwnerUserId="37189" LastEditorUserId="37189" LastEditDate="2020-05-19T10:34:32.357" LastActivityDate="2020-05-19T12:53:44.570" Title="Is it possible to have the latent vector of an auto-encoder with size 1?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;autoencoders&gt;&lt;information-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21295" PostTypeId="2" ParentId="21291" CreationDate="2020-05-19T11:01:14.230" Score="1" Body="&lt;p&gt;According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But an issue may arise during decoding. If you're expecting that through one or two or maybe five layers in decoder you can achieve an accurate reconstruction, it is highly unlikely abd the result will turn out to be blurry. Maybe a great network with various parameters may help the reconstruction without considering generalization as asked by you.&lt;/p&gt;&#xA;" OwnerUserId="37180" LastEditorUserId="37180" LastEditDate="2020-05-19T12:53:44.570" LastActivityDate="2020-05-19T12:53:44.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21296" PostTypeId="1" CreationDate="2020-05-19T11:04:24.550" Score="1" ViewCount="38" Body="&lt;p&gt;Given e.g. &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;M vectors of &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; floating points each, where every point in vectors is sampled from a uniform distribution between &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, how to estimate the minimum network size required between input (&lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; units), bottleneck (preferably &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; unit), and output (&lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; units) which is cable of overfitting the training data perfectly?&lt;/p&gt;&#xA;" OwnerUserId="37189" LastEditorUserId="2444" LastEditDate="2020-05-19T20:26:14.010" LastActivityDate="2020-05-19T20:26:14.010" Title="How estimate the minimum size of an autoencoder to overfit the training data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;autoencoders&gt;&lt;overfitting&gt;&lt;computational-learning-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21298" PostTypeId="2" ParentId="21267" CreationDate="2020-05-19T12:06:46.647" Score="1" Body="&lt;p&gt;I am not an expert on this, but I'll try to explain my understnding of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Bayesian Network is a Directed Graphical Model (DGM) with the &lt;strong&gt;ordered Markov property&lt;/strong&gt; i.e the relationship of a node (random variable) depends only on its immediate parents and not its predecessors (generalized from first order Markov process).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Markov chain on the other hand can be of order &lt;span class=&quot;math-container&quot;&gt;$\geq 1$&lt;/span&gt;. Thus it may depends on not so immediate predecessors (although there can be no gap between predecessors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Defintions according to Kevin Murphy's book, &lt;a href=&quot;https://www.cs.ubc.ca/~murphyk/MLbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Probabilistic Approach to MAchine Learning&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the reason why such a confusion/ambiguity exists is, &lt;strong&gt;in my opinion&lt;/strong&gt;, due to the fact that Bayesian Nets has been generally used to model causal relationships (and hence directed cause &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; effect) between random variables, which are of different type, i.e the each random variable has a completely different state space (e.g sky is cloudy or sunny vs the ground is wet or dry). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas, we generally use Markov Chains to represent Stochastic Process (a collection of r.v's indexed by time: &lt;span class=&quot;math-container&quot;&gt;$X_0, X_1...$&lt;/span&gt;) having the Markov Property. And thus for Markov Chain's we have a state transition matrix. Thus, the collection of r.v's indexed with time has the same state space along with a transition matrix specifying the transition probabilities. The directed graph or DGM exists in Markov Chains to show you are moving forward in time, but the state space for each &lt;span class=&quot;math-container&quot;&gt;$X_k$&lt;/span&gt; remains the same, and hence no 'real parent' exists.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-05-19T12:39:09.527" LastActivityDate="2020-05-19T12:39:09.527" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21299" PostTypeId="1" AcceptedAnswerId="21314" CreationDate="2020-05-19T12:36:40.033" Score="3" ViewCount="249" Body="&lt;p&gt;Let's say I implemented a new deep learning model that pushed some SOTA a little bit further, and I wrote a new paper about for publication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does it work now? I pictured three options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Submit it to a conference&lt;/strong&gt;. Ok, that's the easy one, I submit it to something like NeurIPS or ICML and hope to get accepted. At that point, how do you make your paper accessible? Are there problems in uploading it to arXiv later, in order to get read by more people?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Upload it on arXiv directly&lt;/strong&gt;. If I do that it would not be peer-reviewed, and technically speaking it would be devoid of &quot;academic value&quot;. Right? It could easily be read by anyone, but there would be no formal &quot;proof&quot; of its &quot;scientific quality&quot;. Correct me if I'm wrong.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Submit it to a peer-reviewed journal&lt;/strong&gt;. Avoid desk rejection, avoid reviewers' rejection, after a long painful process it ends up on some international scientific journal. At that point, since the article is formally the editor's property, can you still upload it on arXiv, or on your blog, so that it can be accessible by many people?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do the big stars of deep learning research do when they have some hot new paper ready for publication? And what publications are the most valued in the professional and the academic world?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26580" LastEditorUserId="2444" LastEditDate="2020-05-20T11:41:23.443" LastActivityDate="2020-05-21T11:03:10.713" Title="How does publishing in the deep learning world work, with respect to journals and arXiv?" Tags="&lt;deep-learning&gt;&lt;research&gt;&lt;papers&gt;&lt;academia&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21300" PostTypeId="2" ParentId="20412" CreationDate="2020-05-19T12:51:15.070" Score="1" Body="&lt;p&gt;An interesting model I encountered in a course is Facebook Prophet. Prophet takes into account trends, seasonality, and holidays for its predictions. As you can probably guess, this is a model that fits Facebook's needs very well. I'll give a brief introduction then provide a link where you can read more. Prophet fits a couple of functions of time represented by a few terms. The general form of the timeseries predictions are, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y(t)=g(t)+s(t)+h(t)+\epsilon_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(t)$&lt;/span&gt; deals with the trends I mentioned above. This is exactly what you would think it is, and accounts for non-periodic features of the data. It takes the form of a piecewise linear or logistic function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$s(t)$&lt;/span&gt; accounts for seasonality - in other words these are periodic changes in our timeseries (maybe an increase of sunscreen purchases in the summer). As this is periodic, the natural way to model this term is with Fourier decomposition to identify important frequencies in the signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h(t)$&lt;/span&gt; deals with predictable changes in the timeseries but is for events like holidays (this can happen at different times year to year so this is not necessarily periodic, think Easter). The user provides a list of events and how they want to account for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon_t$&lt;/span&gt; is just an error term to deal with anything that cannot be addressed with the rest of the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2018/05/generate-accurate-forecasts-facebook-prophet-python-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; has a great explanation if you want more detail. I highly recommend you check it out because it is very cool!&lt;/p&gt;&#xA;" OwnerUserId="22373" LastActivityDate="2020-05-19T12:51:15.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21301" PostTypeId="5" CreationDate="2020-05-19T15:15:03.523" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-19T15:15:03.523" LastActivityDate="2020-05-19T15:15:03.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21302" PostTypeId="4" CreationDate="2020-05-19T15:15:03.523" Score="0" Body="For questions related to The Natural Language Toolkit, or more commonly NLTK, is a suite of libraries and programs for symbolic and statistical natural language processing for English written in the Python programming language." OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:38:00.817" LastActivityDate="2020-05-22T22:38:00.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21303" PostTypeId="5" CreationDate="2020-05-19T15:17:02.920" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-19T15:17:02.920" LastActivityDate="2020-05-19T15:17:02.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21304" PostTypeId="4" CreationDate="2020-05-19T15:17:02.920" Score="0" Body="For questions related to spaCy. spaCy is an open-source software library for advanced natural language processing, written in the programming languages Python and Cython. The library is published under the MIT license and its main developers are Matthew Honnibal and Ines Montani, the founders of the software company Explosion." OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:46:52.020" LastActivityDate="2020-05-22T22:46:52.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21305" PostTypeId="5" CreationDate="2020-05-19T15:18:21.373" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-19T15:18:21.373" LastActivityDate="2020-05-19T15:18:21.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21306" PostTypeId="4" CreationDate="2020-05-19T15:18:21.373" Score="0" Body="For questions related to pre-trained model.  A pre-trained model is a model that was trained on a large benchmark dataset to solve a problem similar to the one that we want to solve. Accordingly, due to the computational cost of training such models, it is common practice to import and use models from published literature (e.g. VGG, Inception, MobileNet)" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:39:36.973" LastActivityDate="2020-05-22T22:39:36.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21307" PostTypeId="1" CreationDate="2020-05-19T15:40:40.587" Score="1" ViewCount="67" Body="&lt;p&gt;Assume there exists a new and very efficient algorithm for calculating the polar decomposition of a matrix &lt;span class=&quot;math-container&quot;&gt;$A=UP$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; is a unitary matrix and &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is a positive-semidefinite Hermitian matrix. Would there be any interesting applications in Machine Learning? Maybe topic modeling? Or page ranking? I am interested in references to articles and books.&lt;/p&gt;&#xA;" OwnerUserId="15404" LastActivityDate="2020-05-19T15:40:40.587" Title="Applications of polar decomposition in Machine Learning" Tags="&lt;machine-learning&gt;&lt;graphs&gt;&lt;text-classification&gt;&lt;semantics&gt;&lt;singular-value-decomposition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21308" PostTypeId="2" ParentId="21187" CreationDate="2020-05-19T15:41:51.693" Score="2" Body="&lt;p&gt;Here is the &lt;a href=&quot;https://github.com/JeremieGauthier/AI_Exercices/commit/5a54cbcb53887321b272f0a37d149d3d9c3f7304&quot; rel=&quot;nofollow noreferrer&quot;&gt;commit&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fixed few minor errors, but the major one was when I saw what the line &lt;code&gt;histories = [deque(maxlen=self.reward_steps)] * len(self.env.envs)&lt;/code&gt; was doing. It was just repeating the same queue.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [2]: histories = [deque(maxlen=5)] * 4                                       &#xA;&#xA;In [3]: histories                                                               &#xA;Out[3]: [deque([]), deque([]), deque([]), deque([])]&#xA;&#xA;In [4]: histories[0].append(1)                                                  &#xA;&#xA;In [5]: histories                                                               &#xA;Out[5]: [deque([1]), deque([1]), deque([1]), deque([1])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I just replace it by &lt;code&gt;histories = [deque(maxlen=self.reward_steps) for i in range(len(self.env.envs))]&lt;/code&gt;. That fixed my problem.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [7]: histories = [deque(maxlen=5) for i in range(4)]                         &#xA;&#xA;In [8]: histories                                                               &#xA;Out[8]: [deque([]), deque([]), deque([]), deque([])]&#xA;&#xA;In [9]: histories[0].append(1)                                                  &#xA;&#xA;In [10]: histories                                                              &#xA;Out[10]: [deque([1]), deque([]), deque([]), deque([])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The curve representing the mean reward looks like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eDc8z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eDc8z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-20T00:43:35.723" LastActivityDate="2020-05-20T00:43:35.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21309" PostTypeId="2" ParentId="21285" CreationDate="2020-05-19T16:46:50.580" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What are mono-variable and multi-variable neural networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not sure about this, because most (if not all useful) neural networks are multivariable neural networks (i.e. they contain multiple parameters). Even the perceptron usually contains more than one parameter, so that terminology isn't clear even to me. Maybe they are referring to the number of inputs (sometimes called variables), but I don't see why this distinction in this context would make sense.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are static and dynamic neural networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To answer this question, I will first quote an excerpt from &lt;a href=&quot;http://sedici.unlp.edu.ar/bitstream/handle/10915/1374/2-_Redes_neuronales_artificiales.pdf?sequence=6&amp;amp;isAllowed=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt; (written in Spanish) to provide some context to Spanish speakers (I am not a Spanish speaker, but I understand 99% of it).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Un primer intento de clasificación puede separ &lt;strong&gt;estáticos&lt;/strong&gt; y &lt;strong&gt;dinámicos o recurrentes&lt;/strong&gt; (fig.2.1)&lt;/p&gt;&#xA;&lt;p&gt;Los modelos &lt;strong&gt;estáticos&lt;/strong&gt; realizan un mapeo entre entrada y salida. &lt;strong&gt;Despreciando el tiempo de procesamiento interno&lt;/strong&gt;, la salida se obtiene en forma inmediata en función de la entrada, no existe memoria ni dinámica de estados en el sistema neuronal.&lt;/p&gt;&#xA;&lt;p&gt;Por el contrario los sistemas &lt;strong&gt;recurrentes&lt;/strong&gt; si la poseen, son sistemas realimentados que ante un estimulo de entrada evolucionan hasta converger a una salida estable.&lt;/p&gt;&#xA;&lt;p&gt;Casos tipicos de ambos sistemas son el &lt;strong&gt;Perceptrón&lt;/strong&gt; (Rosemblatt, 1960a) (de una o múltiples capas) y la &lt;strong&gt;memoria asociativa de Hopfield&lt;/strong&gt;, respectivamente&#xA;(Tank, 1987).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, in this document, the word &amp;quot;dynamic&amp;quot; and &amp;quot;recurrent&amp;quot; are being used interchangeably. An example of a static (i.e. non-recurrent) neural network is the perceptron. An example of recurrent (or dynamic) neural network is the Hopfield network.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I recommend you contact the author of that article to ask for clarification (especially, about the mono-variable NNs)!&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-19T19:30:21.937" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21311" PostTypeId="2" ParentId="11617" CreationDate="2020-05-19T18:47:37.637" Score="0" Body="&lt;p&gt;AlphaGo uses MCTS.  AlphaZero does not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://discovery.ucl.ac.uk/id/eprint/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mastering the Game of Go without Human Knowledge&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37204" LastEditorUserId="1671" LastEditDate="2020-05-22T22:55:43.980" LastActivityDate="2020-05-22T22:55:43.980" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21312" PostTypeId="1" AcceptedAnswerId="21647" CreationDate="2020-05-19T20:29:55.280" Score="2" ViewCount="80" Body="&lt;p&gt;I'm working on creating an environment for a card game, which the agent chooses to discard certain cards in the first phase of the game, and uses the remaining cards to play with. (The game is Crib if you are familiar with it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I make an action space for these actions? For instance, in this game, we could discard 2 of 6 cards, then choose 1 of 4 remaining cards to play, then 1 of 3 remaining cards, then 1 of 2 remaining cards. How do I model this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read &lt;a href=&quot;https://ai.stackexchange.com/questions/18115/openai-gym-define-action-space-when-an-agent-can-take-multiple-sub-actions-in-a?newreg=d03169f4213c42939d4c25e2c8c275de&quot;&gt;this post on using MultiDiscrete spaces&lt;/a&gt;, but I'm not sure how to define this space based on the previous chosen action. Is this even the right approach to be taking?&lt;/p&gt;&#xA;" OwnerUserId="37205" LastEditorUserId="2444" LastEditDate="2020-05-20T10:53:16.610" LastActivityDate="2020-06-04T21:51:08.687" Title="What should the action space for the card game Crib be?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;open-ai&gt;&lt;environment&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21313" PostTypeId="1" CreationDate="2020-05-19T21:03:48.770" Score="1" ViewCount="47" Body="&lt;p&gt;From : &lt;a href=&quot;https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/&lt;/a&gt;&#xA;the following autoencoder is defined&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Autoencoder(nn.Module):&#xA;    def __init__(self):&#xA;        super(Autoencoder, self).__init__()&#xA;&#xA;        # encoder&#xA;        self.enc1 = nn.Linear(in_features=784, out_features=256)&#xA;        self.enc2 = nn.Linear(in_features=256, out_features=128)&#xA;        self.enc3 = nn.Linear(in_features=128, out_features=64)&#xA;        self.enc4 = nn.Linear(in_features=64, out_features=32)&#xA;        self.enc5 = nn.Linear(in_features=32, out_features=16)&#xA;&#xA;        # decoder &#xA;        self.dec1 = nn.Linear(in_features=16, out_features=32)&#xA;        self.dec2 = nn.Linear(in_features=32, out_features=64)&#xA;        self.dec3 = nn.Linear(in_features=64, out_features=128)&#xA;        self.dec4 = nn.Linear(in_features=128, out_features=256)&#xA;        self.dec5 = nn.Linear(in_features=256, out_features=784)&#xA;&#xA;    def forward(self, x):&#xA;        x = F.relu(self.enc1(x))&#xA;        x = F.relu(self.enc2(x))&#xA;        x = F.relu(self.enc3(x))&#xA;        x = F.relu(self.enc4(x))&#xA;        x = F.relu(self.enc5(x))&#xA;&#xA;        x = F.relu(self.dec1(x))&#xA;        x = F.relu(self.dec2(x))&#xA;        x = F.relu(self.dec3(x))&#xA;        x = F.relu(self.dec4(x))&#xA;        x = F.relu(self.dec5(x))&#xA;        return x&#xA;&#xA;net = Autoencoder()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From the &lt;code&gt;Autoencoder&lt;/code&gt; class, we can see that 784 features are passed through a series of transformations and are converted to 16 features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transformations (&lt;code&gt;in_features&lt;/code&gt; to &lt;code&gt;out_features&lt;/code&gt;) for each layer are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;784 to 256&#xA;256 to 128&#xA;128 to 64&#xA;64 to 32&#xA;32 to 16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why do we perform this sequence of operations? For example, why don't we perform the following sequence of operations instead?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;784 to 256&#xA;256 to 128&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or maybe&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;784 to 512&#xA;512 to 256&#xA;256 to 128&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or maybe just encode in two layers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;784 to 16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does the reduction of the dimensions over multiple layers (instead of a single layer) allow more details to be stored within the final representation? For example, if we used only the transformation &lt;span class=&quot;math-container&quot;&gt;$784 \rightarrow 16$&lt;/span&gt;, may this cause some detail not to be encoded? If so, why is this the case?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="2444" LastEditDate="2020-05-20T10:48:21.837" LastActivityDate="2020-05-20T10:48:21.837" Title="Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?" Tags="&lt;neural-networks&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21314" PostTypeId="2" ParentId="21299" CreationDate="2020-05-19T21:37:38.313" Score="3" Body="&lt;p&gt;Let me answer your questions one by one.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Submit it to a conference&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Let's start with the optimistic case. Say your paper gets accepted! You can upload your preprint on arXiv with the &quot;arXiv.org perpetual, non-exclusive license to distribute this article (Minimal rights required by arXiv.org)&quot;. It is a non-Creative Common License that does not provide any exclusive rights to arXiv as &lt;a href=&quot;https://academia.stackexchange.com/questions/103713/what-license-to-use-while-putting-papers-on-the-arxiv&quot;&gt;per&lt;/a&gt;. This is the default permission of arXiv (which does not interfere with the license of the conferences). You can easily find papers from ICML, NeurIPS, and AAAI on their websites as well as arXiv. For, e.g., NeurIPS 2017 began on 4th Dec 2017 and &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper has the latest submission on 6th Dec 2017.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you say your work was rejected or you are about to submit it, you need to check for the restrictions on uploading the manuscript online before submitting it to the conference. Some conference allows papers to be submitted only if they are not uploaded to sites like arXiv within say 30 days prior to the paper submission deadline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AAAI does not have restrictions on arXiv as per their &lt;a href=&quot;https://www.quora.com/Can-I-upload-a-paper-to-arXiv-before-submitting-to-AAAI&quot; rel=&quot;nofollow noreferrer&quot;&gt;policy&lt;/a&gt;. ACM also allows non-profit organizations like arXiv as per their &lt;a href=&quot;https://www.acm.org/publications/policies/copyright-policy#permanent%20rights&quot; rel=&quot;nofollow noreferrer&quot;&gt;policy&lt;/a&gt;.&#xA;Among all the info on the major publications I could find, I found a common phrase that the paper should not be uploaded to a for-profit digital library. ArXiv is certainly not among them. The reason why editors do not oppose arXiv papers could be that there are a lot of papers that are uploaded to arXiv in order to claim to be first, so banning them would be a big loss for the editors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you comply with the restrictions of the conference and have uploaded your preprint then it brings the risk that reviewers can know who the authors are. They might be biased once they know who the authors are. This works in the favor of renowned/superstars scientists of the field. The reviewer might be a Ph.D. who admires this superstar and hence would try to get it accepted. Or the author might be a known colleague of the author and would hence be biased into accepting it. On the other hand, if the reviewer has some personal bias towards the author then that may be reflected in their scores. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Upload it on arXiv directly&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Only uploading to arXiv is of not much academic value because it is not peer-reviewed as you pointed out. Anyone can write an absurd paper with false results and get it published on arXiv. That is why peer-reviewed papers are important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One reason to upload to arXiv first is to make your work safe from other people coming up with similar approaches. It is not rare at all for people coming up with similar ideas within a duration of 3 months. Deep Learning is a fast-paced domain as it is a hot research field. So getting your paper out first in the name of work in progress saves you from potential rejects from the conference pointing to a similar work done available online before you.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Submit it to a peer-reviewed journal&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Same thing as a conference. You can upload your preprint after acceptance of your manuscript with the default license of arXiv. For submitting it before on arXiv needs cross-checking with the rules of the journal. For, e.g., IEEE allows the work to be submitted to arXiv as &lt;a href=&quot;https://academia.stackexchange.com/questions/16831/is-it-legal-to-upload-a-paper-to-arxiv-when-it-is-under-double-blind-review-for&quot;&gt;per&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How do big stars publish?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Coming to your question of how big stars publish. They either publish directly in a conference and then upload to arXiv or they first upload on arXiv, get publicity and citations and then submit to a top conference. Having the preprint available online can also bias the reviewer into accepting their research (as mentioned before). This way they reduce their chance of rejection.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Fun fact&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Interestingly, if the preprints become very famous then the reviewer might get confused that your work is a derivative of a popular work which in reality is the preprint of your submitted work! In such cases, you would need to imply this diplomatically. For example, Music Transformer by the Google Brain team (which consisted of authors of the famous Transformer paper &quot;Attention is all you need&quot;) was available on arXiv in 2018 and was already being cited by others. When it was submitted to ICLR 2019, a reviewer mistakenly took it as a derivative work of the 2018 arXiv paper and suggested a rejection of the paper. However, after further inspection, he/she realized the confusion which blew their mind! Immediately the paper was given a suitable high score. Source: &lt;a href=&quot;https://openreview.net/forum?id=rJe4ShAcF7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openreview.net/forum?id=rJe4ShAcF7&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Where to publish?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Unlike in core electronics and other branches where a journal is much more important than conference papers, in machine learning, conference papers are at par or better than any journal out there. Mostly the quality depends on the Impact Factor of the journal or conference. For example, NeurIPS is a top venue to publish and it's a conference and not a journal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One reason for conferences to be so important is that they are more popular among researchers. Conferences are fast with their review as compared to the lengthy journal review. This allows researchers to meet and discuss the progress of the field with like-minded people much faster. This is crucial for a rapidly evolving field like artificial intelligence. This makes conference popular which attracts the best researchers to publish their cutting edge which in turn makes the conference more lucrative for other researchers to publish in the same venue as the big stars. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conferences also allow people from the industry to meet the researchers and hire them. The quick meeting opportunity provided by the conference is beneficial for both the academic researchers as well as the industry for attracting talent.&lt;/p&gt;&#xA;" OwnerUserId="37206" LastEditorUserId="37206" LastEditDate="2020-05-21T11:03:10.713" LastActivityDate="2020-05-21T11:03:10.713" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21315" PostTypeId="1" CreationDate="2020-05-20T05:34:07.067" Score="0" ViewCount="53" Body="&lt;p&gt;I was reading about the possibility of using Google's Coral for deep learning-based object detection and image classification. I heard it has a good speed in terms of frames/sec.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read that Google's Coral is only compatible with quantized models. What does this mean? How will this affect the performance of object detection or classification in terms of accuracy and speed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the advantage of using Google's Coral over Nvidia's Xavier?&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="2444" LastEditDate="2020-05-20T10:36:41.307" LastActivityDate="2020-05-21T03:26:00.770" Title="What is the advantage of using Google's Coral over Nvidia's Xavier?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21316" PostTypeId="1" CreationDate="2020-05-20T06:05:57.780" Score="1" ViewCount="47" Body="&lt;p&gt;I was wondering whether anyone could point me to some references that discuss &lt;strong&gt;Deep Learning&lt;/strong&gt; interview questions for &lt;strong&gt;Deep Learning Engineer Role&lt;/strong&gt;, from the most basic through the more advanced. &#xA;Answers are not necessary (although links to specific questions on this site would do well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be very interesting if somebody could give examples they were given themselves when interviewed&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-20T10:23:40.493" LastActivityDate="2020-05-20T14:57:34.997" Title="What are some references that discuss questions asked during deep learning interviews?" Tags="&lt;academia&gt;&lt;education&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21317" PostTypeId="5" CreationDate="2020-05-20T06:07:03.527" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-20T06:07:03.527" LastActivityDate="2020-05-20T06:07:03.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21318" PostTypeId="4" CreationDate="2020-05-20T06:07:03.527" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-20T06:07:03.527" LastActivityDate="2020-05-20T06:07:03.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21319" PostTypeId="5" CreationDate="2020-05-20T06:07:40.230" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-20T06:07:40.230" LastActivityDate="2020-05-20T06:07:40.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21320" PostTypeId="4" CreationDate="2020-05-20T06:07:40.230" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-20T06:07:40.230" LastActivityDate="2020-05-20T06:07:40.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21321" PostTypeId="1" CreationDate="2020-05-20T07:12:26.083" Score="1" ViewCount="44" Body="&lt;p&gt;I have the following assignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A2PnL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A2PnL.jpg&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't understand the b part of this question in my assignment. I have completed the 1st part and understand the maths behind it, but the 2nd part has me stumped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked up ridge functions and they basically map real vectors to a single real value, from what I understood. For that reason, I considered that the activation function has to be one that ranges over the real numbers, but that still doesn't clear my doubts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't need a full answer just an explanation of the question will be very helpful, here's some text from the book I'm referring( Russel and Norvig), though I couldn't really grasp how this would help me choose an activation function.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Before delving into learning rules, let us look at the ways in which networks generate complicated functions. First, remember that each unit in a sigmoid network represents a soft threshold in its input space, as shown in Figure 18.17(c) (page 726). With one hidden layer and one output layer, as in Figure 18.20(b), each output unit computes a soft-thresholded linear combination of several such functions. For example, by adding two opposite-facing soft threshold functions and thresholding the result, we can obtain a “ridge” function as shown in Figure 18.23(a). Combining two such ridges at right angles to each other (i.e., combining the outputs from four hidden units), we obtain a “bump” as shown in Figure 18.23(b).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="37216" LastEditorUserId="2444" LastEditDate="2020-05-20T10:22:06.733" LastActivityDate="2020-05-20T10:22:06.733" Title="If the output of a model is a ridge function, what should the activation functions at all the nodes be?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21323" PostTypeId="5" CreationDate="2020-05-20T10:35:47.473" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-20T10:35:47.473" LastActivityDate="2020-05-20T10:35:47.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21324" PostTypeId="4" CreationDate="2020-05-20T10:35:47.473" Score="0" Body="For questions related to Octave. GNU Octave is software featuring a high-level programming language, primarily intended for numerical computations. Octave helps in solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with MATLAB" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:38:19.227" LastActivityDate="2020-05-22T22:38:19.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21325" PostTypeId="5" CreationDate="2020-05-20T10:38:04.677" Score="0" Body="" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-10-23T16:17:52.533" LastActivityDate="2020-10-23T16:17:52.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21326" PostTypeId="4" CreationDate="2020-05-20T10:38:04.677" Score="0" Body="For questions about the contextual bandit problem, which is a generalization of the (context-free) multi-armed bandit problem, where there is more than one situation (or state) and the optimal action to take in one state may be different than the optimal action to take in another state, but where the actions do not affect states (as e.g. in the reinforcement learning problem), but only the rewards. " OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-10-23T16:17:52.533" LastActivityDate="2020-10-23T16:17:52.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21327" PostTypeId="5" CreationDate="2020-05-20T10:40:07.157" Score="0" Body="&lt;p&gt;Word Embeddings or Word vectorization is a methodology in NLP to map words or phrases from vocabulary to a corresponding vector of real numbers which used to find word predictions, word similarities/semantics. The process of converting words into numbers are called Vectorization&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:39:14.430" LastActivityDate="2020-05-22T22:39:14.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21328" PostTypeId="4" CreationDate="2020-05-20T10:40:07.157" Score="0" Body="For questions related to Vectorization." OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:37:56.963" LastActivityDate="2020-05-22T22:37:56.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21329" PostTypeId="5" CreationDate="2020-05-20T10:41:16.120" Score="0" Body="&lt;p&gt;Collaborative filtering is a technique used by recommender systems. Collaborative filtering has two senses, a narrow one and a more general one.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:38:06.990" LastActivityDate="2020-05-22T22:38:06.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21330" PostTypeId="4" CreationDate="2020-05-20T10:41:16.120" Score="0" Body="For questions related to Collaborative filtering. Collaborative filtering is a technique that can filter out items that a user might like on the basis of reactions by similar users. It works by searching a large group of people and finding a smaller set of users with tastes similar to a particular user." OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:39:29.097" LastActivityDate="2020-05-22T22:39:29.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21331" PostTypeId="5" CreationDate="2020-05-20T10:42:24.920" Score="0" Body="&lt;p&gt;Sentence segmentation is the process of determining the longer processing units consisting of one or more words. This task involves identifying sentence boundaries between words in different sentences.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:38:24.100" LastActivityDate="2020-05-22T22:38:24.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21332" PostTypeId="4" CreationDate="2020-05-20T10:42:24.920" Score="0" Body="For Questions related to Sentence segmentation. " OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:46:48.843" LastActivityDate="2020-05-22T22:46:48.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21333" PostTypeId="5" CreationDate="2020-05-20T10:44:00.170" Score="0" Body="" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-06-18T09:36:50.283" LastActivityDate="2020-06-18T09:36:50.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21334" PostTypeId="4" CreationDate="2020-05-20T10:44:00.170" Score="0" Body="For questions about the sample efficiency (or inefficiency) of learning algorithms, which is the amount of experience (or data) that the learning algorithm needs in order to reach a certain level of performance. In the case of reinforcement learning, this experience is represented by (transition) tuples of the form $(s, a, r, s')$." OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-10-13T11:17:50.230" LastActivityDate="2020-10-13T11:17:50.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21335" PostTypeId="5" CreationDate="2020-05-20T10:44:44.223" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-05-20T10:44:44.223" LastActivityDate="2020-05-20T10:44:44.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21336" PostTypeId="4" CreationDate="2020-05-20T10:44:44.223" Score="0" Body="For questions related to Spectral analysis or Spectrum analysis is a analysis in terms of a spectrum of frequencies or related quantities such as energies, eigenvalues, etc. In specific areas it may refer to: Spectroscopy in chemistry and physics, a method of analyzing the properties of matter from their electromagnetic interactions." OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:38:10.977" LastActivityDate="2020-05-22T22:38:10.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21337" PostTypeId="5" CreationDate="2020-05-20T12:17:13.867" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-20T12:17:13.867" LastActivityDate="2020-05-20T12:17:13.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21338" PostTypeId="4" CreationDate="2020-05-20T12:17:13.867" Score="0" Body="For questions related to the asynchronous advantage actor-critic (A3C) algorithm." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-20T12:17:13.867" LastActivityDate="2020-05-20T12:17:13.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21339" PostTypeId="1" AcceptedAnswerId="21346" CreationDate="2020-05-20T12:24:39.117" Score="1" ViewCount="188" Body="&lt;p&gt;I've read about &lt;a href=&quot;https://en.wikipedia.org/wiki/Knight%27s_tour&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Knight Tour problem&lt;/a&gt;. And I wanted to try to solve it with a reinforcement learning algorithm with OpenAI's gym.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I want to make a bot that can move on the chess table like the knight. And it is given a reward each time it moves and does not leave the table or step in an already visited place. So, it gets better rewards if it survives more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or there is a better approach to this problem? Also, I would like to display the best knight in each generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not very advanced at reinforcement learning (I'm still studying it), but this project really caught my attention. I know well machine learning and deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to start implementing a new OpenAI's gym environment and start all from scratch, or there is a better idea?&lt;/p&gt;&#xA;" OwnerUserId="37226" LastEditorUserId="2444" LastEditDate="2020-05-20T12:37:52.693" LastActivityDate="2020-05-20T15:12:55.823" Title="How can I model and solve the Knight Tour problem with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;game-ai&gt;&lt;q-learning&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21342" PostTypeId="1" CreationDate="2020-05-20T13:01:46.110" Score="2" ViewCount="21" Body="&lt;p&gt;Cat swarm optimization (CSO) is a novel metaheuristic for evolutionary optimization algorithms based on swarm intelligence which proposed in 2006. See &lt;a href=&quot;https://ieeexplore.ieee.org/document/6916353&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feature Selection of Support Vector Machine Based on Harmonious Cat Swarm Optimization&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://www.researchgate.net/publication/288068152_Modified_Cat_Swarm_Optimization_Algorithm_for_Feature_Selection_of_Support_Vector_Machines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Modified Cat Swarm Optimization Algorithm for Feature Selection of Support Vector Machines&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;CSO imitates the behavior of cats through two sub-modes: seeking and tracing. Previous studies have indicated that CSO algorithms outperform other well-known meta-heuristics, such as genetic algorithms and particle swarm optimization. This study presents a modified version of cat swarm optimization (MCSO), capable of improving search efficiency within the problem space. The basic CSO algorithm was integrated with a local search procedure as well as the feature selection of support vector machines (SVMs).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can someone explain how exactly Cat Swarm Algorithm (CSO) is used for feature selection?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-20T13:15:57.063" LastActivityDate="2020-05-20T13:15:57.063" Title="How can Cat Swarm Algorithm (CSO) used for feature selection?" Tags="&lt;machine-learning&gt;&lt;feature-selection&gt;&lt;swarm-intelligence&gt;&lt;cat-swarm-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21343" PostTypeId="5" CreationDate="2020-05-20T13:03:09.350" Score="0" Body="&lt;p&gt;Cat swarm optimization (CSO) algorithm is a computational intelligent algorithm, inspired from the behavior of cats. The CSO algorithm was introduced by Chu and Tsai (2006). &lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:37:01.010" LastActivityDate="2020-05-22T22:37:01.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21344" PostTypeId="4" CreationDate="2020-05-20T13:03:09.350" Score="0" Body="For questions related to Cat swarm optimization (CSO). " OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-05-22T22:37:54.543" LastActivityDate="2020-05-22T22:37:54.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21345" PostTypeId="2" ParentId="21284" CreationDate="2020-05-20T14:06:29.457" Score="0" Body="&lt;p&gt;An interesting question. I would start by finding &lt;code&gt;n&lt;/code&gt; nearest neighbors of each data point, then calculate their center of mass &lt;code&gt;c&lt;/code&gt; and the point's distance &lt;code&gt;d&lt;/code&gt; to its nth nearest neighbor. The smaller the &lt;code&gt;d&lt;/code&gt; is the larger the density is around a given point. You could then iteratively step every point away from their &lt;code&gt;c&lt;/code&gt; in an inverse proportion to the distance &lt;code&gt;d&lt;/code&gt; with a suitable step size. This would spread out the clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this won't help you transform any new points in the dataset, maybe you can learn this arbitrary mapping &lt;code&gt;R^n -&amp;gt; R^n&lt;/code&gt; by using an other neural network and apply it to new samples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the first ad-hoc idea which came to my mind. It would be interesting to see a 2D animation of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more rigorous approach might be a &lt;a href=&quot;https://blog.keras.io/building-autoencoders-in-keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;variational autoencoder&lt;/a&gt;, you can embed the data in a lower dimensional space with approximately normal distribution. But it doesn't guarantee that clusters would be as spread out as you'd like. An alternative loss function would help with that, for example every point's distance to their original nth closest neighbor should be as close to one as possible.&lt;/p&gt;&#xA;" OwnerUserId="32722" LastActivityDate="2020-05-20T14:06:29.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21346" PostTypeId="2" ParentId="21339" CreationDate="2020-05-20T14:18:04.563" Score="1" Body="&lt;h3&gt;Model your problem as an MDP&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;To solve a problem with reinforcement learning, you need to model your problem as a Markov decision process (MDP), so you need to define &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the state space, &lt;/li&gt;&#xA;&lt;li&gt;the action space, and&lt;/li&gt;&#xA;&lt;li&gt;the reward function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;of the MDP.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Understand your problem and the goal&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;To do define these, you need to understand your problem and define it as a goal-oriented problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the knight tour problem, there's a knight that needs to visit each square of a chessboard &lt;strong&gt;exactly once&lt;/strong&gt;. The knight can perform &lt;strong&gt;only &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;-shaped moves&lt;/strong&gt; (as for the rules of chess). See the animation below (taken from &lt;a href=&quot;https://en.wikipedia.org/wiki/Knight%27s_tour&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia article&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oEY0H.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oEY0H.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;goal&lt;/strong&gt; is then, by making &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; moves, to find a path through the squares such that each square is visited exactly once. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is the state space?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You could think that the state space &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; could be the set of all squares of the chessboard. So, if you have an &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; chessboard, then &lt;span class=&quot;math-container&quot;&gt;$|S| = n^2$&lt;/span&gt;, i.e. you will have &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this can be problematic because a square alone doesn't tell you all the information that you need to know to take the optimal action. So, you need to define the states such that all available information is available to the agent, i.e. you need to define a state as the position of the current square and the position of the other available squares.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is the action space?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The action space could be defined as the set of all actions that the knight can take across all states. Given that the knight can only take &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; moves, whenever the knight is in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, only &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;-shaped actions are available. Of course, it is possible that, for each state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, there's more thane one valid &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;-shaped action. That's fine. However, the chosen &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;-shaped action will definitely affect your next actions, so we need a way of guiding the knight. That's the purpose of the reward function!&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What is the reward function?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The reward function is typically the most crucial function that you need to define when modeling your problem as an MDP that needs to be solved with an RL algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you could give a reward of e.g. &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for each found path. More precisely, you will let your RL agent explore the environment. If it eventually finds a correct path (or solution), you will give it &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. You could also penalise the knight if it ends in a situation where it cannot take an &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;-shaped action anymore. Given that you don't really want this to happen, you could give it a very small reward e.g. &lt;span class=&quot;math-container&quot;&gt;$-100$&lt;/span&gt;. Finally, you could give it a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for each action taken, which could imply that you don't really care about the actions that the knight takes, as long as it reaches the goal, i.e. find a path through the chessboard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The design of the reward function will highly affect the behaviour and performance of your RL agent. The above-suggested reward function may actually not work well, so you may need to try different reward functions to get some satisfactory results. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Which RL algorithm to use?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Of course, you will also need to choose an RL algorithm to solve this problem numerically. The most common one is Q-learning. You can find its pseudocode &lt;a href=&quot;https://github.com/udacity/rl-cheatsheet/blob/master/cheatsheet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How to implement this with OpenAI's gym?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You probably need to create a custom environment and define the state and action spaces, as well as the reward function. I cannot tell you the details, but I think you can figure them out.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Is RL the right approach to solve this problem?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;RL isn't probably the most efficient approach to solve this problem. There are probably more efficient solutions. For example, there's a divide-and-conquer approach, which I am not familiar with, but that you may also try to use and compare with the RL approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also read the paper &lt;a href=&quot;https://core.ac.uk/download/pdf/81943311.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solution of the knight's Hamiltonian path problem on chessboards&lt;/a&gt; (1994), especially, if you are already familiar with the Hamiltonian path problem (HPP). Note that the knight tour problem is an instance of the HPP. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-20T15:12:55.823" LastActivityDate="2020-05-20T15:12:55.823" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="21347" PostTypeId="1" CreationDate="2020-05-20T15:23:57.633" Score="0" ViewCount="52" Body="&lt;p&gt;I want to use a pretrained model found in [BERT Embeddings] &lt;a href=&quot;https://github.com/UKPLab/sentence-transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/UKPLab/sentence-transformers&lt;/a&gt; and I want to add a layer to get the sentence embeddings from the model and pass on to the next layer. How do I approach this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inputs would be an array of documents and each document containing an array of sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input to the model itself is a list of sentences where it will return a list of embeddings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I've tried but couldn't solve the errors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_embeddings(input_data):&#xA;&#xA;    input_embed = []&#xA;    for doc in input_data:&#xA;      doc = tf.unstack(doc)&#xA;      doc_arr = asarray(doc)&#xA;      doc = [el.decode('UTF-8') for el in doc_arr]&#xA;      doc = list(doc)&#xA;      assert(type(doc)== list)&#xA;&#xA;      new_doc = []&#xA;      for sent in doc:&#xA;        sent = tf.unstack(sent)&#xA;        new_doc.append(str(sent))&#xA;        assert(type(sent)== str)&#xA;&#xA;      embedding= model.encode(new_doc)  # Accepts lists of strings to return BERT sentence embeddings&#xA;      input_embed.append(np.array(embedding))&#xA;&#xA;    return tf.convert_to_tensor(input_embed, dtype=float)&#xA;&#xA;&#xA;sentences = tf.keras.layers.Input(shape=(3,5)) #test shape&#xA;sent_embed = tf.keras.layers.Lambda(get_embeddings)&#xA;&#xA;&#xA;x = sent_embed(sentences)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1O8sY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1O8sY.png&quot; alt=&quot;Errors&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37006" LastActivityDate="2020-05-21T10:57:07.100" Title="How to add a pretrained model to my layers to get embeddings?" Tags="&lt;neural-networks&gt;&lt;word-embedding&gt;&lt;bert&gt;&lt;text-summarization&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21348" PostTypeId="1" AcceptedAnswerId="21349" CreationDate="2020-05-20T15:55:35.670" Score="0" ViewCount="38" Body="&lt;p&gt;Sometimes the agent learns a bit slow and you want to have multiple agents in one generation. And at each episode you'll draw on the screen only the best of them or all of them. How is that possible? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarification purposes, please watch this &lt;a href=&quot;https://www.youtube.com/watch?v=WSW-5m8lRMs&amp;amp;t=354s&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; on youtube at time 4:10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need just a theoretical approach, I'll try the coding myself :).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any answer! I really do appreciate it! :)&lt;/p&gt;&#xA;" OwnerUserId="37226" LastActivityDate="2020-05-20T16:40:14.430" Title="How to add more than 1 agent in one generation with Q Learning" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;q-learning&gt;&lt;open-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21349" PostTypeId="2" ParentId="21348" CreationDate="2020-05-20T16:25:11.063" Score="1" Body="&lt;p&gt;The video you linked is not using reinforcement learning (RL). It is using genetic algorithms (GA). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;GA is designed around using multiple agents and picking the best performing to move forward to next generation. With this approach, it is common to want to only view the best performing agents, as the learning mechanism uses the same selection process - the best agent is the output of the algorithm*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst you can run muliple agents to collect more data easily enough in RL, they typically won't perform better or worse other than by random chance. The best performance is not indicative of the agent's overall performance in the way that GA is, because there is only one agent. It would not be as meaningful to pick out the best ones for display. Instead, after a certain number of training episodes, you should take the agent so far, stop it running exploratory moves (set &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0$&lt;/span&gt; if you are using &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy exploration). Then render the behaviour of that agent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to compare RL versus GA for learning efficiency, then one arguably fair comparison would be to render the agent each time it has trained on a number of episodes in RL equal to the population size used in the GA version. If you are only showing results after every 100 generations of GA, then multiply number of training episodes by 100 to compare using the same amount of data input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also, separately to this concern, run RL with multiple exploring agents at once. If you want to have parallel training with multiple environments running in RL, then you will need a distributed environment. Each instance of the environment would run one agent, and collect training data. You have a rough choice between:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Feeding everything observed into one central experience replay memory and a single training loop routine samples from the whole memory and updates the agent. This should work OK for Q-learning.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate update gradients on each distributed environment, collate them centrally and update the agent on a mean gradient update step before sending out the updated agent to all the distributed systems. This is the approach typically used by A3C and A2C which benefit from having multiple agents running at once.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In both cases, the latest parameters of the agent (the neural network weights) need to be regularly copied out to each environment so that each instance can work as much as possible with a current policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setting up a distributed learning environment for RL is more work than for GA, because you need to move a lot more data between agents to complete learning, whilst for GA you only need to measure fitness. However, you should find that Q learning can be a lot more efficient (in terms of number of simulations required) than a GA-based approach for many control problems.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* As an aside, this can be a weakness of GA if you are running in an environment with random choices or events - the GA can select as it's &quot;best&quot; result at any stage something that is less optimal but that was lucky. On average over many generations this effect should be removed, because the same agent won't be luck every generation, but it does mean in some environments that you will get an over-estimate of performance unless your fitness assessment is very thorough (or perhaps run separately every so many generations, to double-check). Similar concerns occur in RL too, but do not affect which agent you select for assessment or display, since there is only ever one set of agent parameters at any time.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-20T16:40:14.430" LastActivityDate="2020-05-20T16:40:14.430" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21351" PostTypeId="1" CreationDate="2020-05-20T19:01:46.817" Score="1" ViewCount="60" Body="&lt;p&gt;I am working on a project to implement a collision avoidance algorithm on a real &lt;a href=&quot;https://en.wikipedia.org/wiki/Unmanned_aerial_vehicle&quot; rel=&quot;nofollow noreferrer&quot;&gt;unmanned aerial vehicle (UAV)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in understanding the process to set up a negative reward to account for scenarios wherein there is a UAV crash. This can be done very easily during the simulation (if the UAV touches any object, the episode stops giving a negative reward). In the real world, a UAV crash would usually entail it hitting a wall or an obstacle, which is difficult to model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial plan is to stop the RL episode and manually input a negative reward (to the algorithm) each time a crash occurs. Any improvements to this plan would be highly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2020-05-26T11:10:30.627" LastActivityDate="2020-05-26T11:10:30.627" Title="How do I set up rewards to account for unmanned aerial vehicle crashes?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;real-time&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21352" PostTypeId="2" ParentId="21274" CreationDate="2020-05-20T20:01:25.373" Score="1" Body="&lt;p&gt;In RL, neural networks may intuitively be thought of as using the input features as a representation that &quot;identifies&quot; the input state (or input state + action pair). Think back to the &quot;tabular&quot; RL setting that most people first study when they learn about RL. In tabular RL, you have a table of values (state values &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, or state-action values &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;), with unique entries in the table for every state. Such a table can &lt;strong&gt;perfectly identify&lt;/strong&gt; states or, in other words, &lt;strong&gt;perfectly disambiguate&lt;/strong&gt; different states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a non-tabular, function approximation setting, with function approximators such as Neural Networks, you can generally no longer uniquely identify every single state. Instead, you use approximate representations of these states, and the approximation implies that it's possible that you have multiple different states that look identical; they have identical input features. This is the case you're dealing with. Now, you specified explicitly that these multiple states with identical representations / input features follow each other up immediately in a single episode, but I don't think this detail is particularly important. You'd have exactly the same problems if these different states with identical representation showed up at different times within an episode. The only problem that you really have is a &lt;strong&gt;disambiguation problem&lt;/strong&gt;: you don't know how to disambiguate these states, since they look identical to the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How significant that problem is depends on your domain. Based on your domain knowledge, do you expect the optimal action, or the optimal values, to be kind of similar in all these states that have identical features? If so, no problem! Your network already thinks they're the same anyway, so it will learn that the same actions / same values are the best in those states. But do you expect the optimal actions / true value functions to be wildly different in these states despite the fact that a network can't disambiguate them? In this case, the problem will be more severe because you can't realistically expect your network to learn the optimal actions / value functions for all these different states. At best, it can learn a weighted average among them (weighted by how commonly they occur in your training episodes).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-05-20T20:01:25.373" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21353" PostTypeId="1" CreationDate="2020-05-20T20:23:41.290" Score="2" ViewCount="68" Body="&lt;p&gt;What are the most common feedforward neural networks? What kind of inputs do they receive? For example, do they receive binary numbers, real numbers, vectors, or matrics? Is there such a taxonomy? &lt;/p&gt;&#xA;" OwnerUserId="37183" LastEditorUserId="2444" LastEditDate="2020-05-20T21:11:34.353" LastActivityDate="2020-07-20T12:12:53.437" Title="What are the most common feedforward neural networks?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21354" PostTypeId="2" ParentId="16805" CreationDate="2020-05-20T20:54:00.833" Score="1" Body="&lt;p&gt;You can use &lt;strong&gt;&lt;a href=&quot;https://github.com/rusty1s/pytorch_geometric&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch_Geometric&lt;/a&gt;&lt;/strong&gt; library for your projects. Its &lt;strong&gt;supports weighted GCNs&lt;/strong&gt;. It is a rapidly evolving open-source library with easy to use syntax. It is mentioned in the landing page of &lt;a href=&quot;https://pytorch.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch&lt;/a&gt;. It is the most starred Pytorch github repo for geometric deep learning. Creating a GCN model which can process graphs with weights is as simple as:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn.functional as F&#xA;from torch_geometric.nn import GCNConv&#xA;&#xA;class Net(torch.nn.Module):&#xA;    def __init__(self):&#xA;        super(Net, self).__init__()&#xA;        self.conv1 = GCNConv(dataset.num_node_features, 16)&#xA;        self.conv2 = GCNConv(16, dataset.num_classes)&#xA;&#xA;    def forward(self, data):&#xA;&#xA;        # data has the following 3 attributes&#xA;        x, edge_index, edge_weight = data.x, data.edge_index, data.edge_weight&#xA;&#xA;        x = self.conv1(x, edge_index, edge_weight)&#xA;        x = F.relu(x)&#xA;        x = F.dropout(x, training=self.training)&#xA;        x = self.conv2(x, edge_index, edge_weight)&#xA;&#xA;        return F.log_softmax(x, dim=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; for getting started.&#xA;Check out its &lt;a href=&quot;https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html#module-torch_geometric.nn.conv.message_passing&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; on different variants of GCNs for further details. One of the best thing is that like Pytorch, its documentation are self-sufficient.&lt;/p&gt;&#xA;" OwnerUserId="37206" LastEditorUserId="37206" LastEditDate="2020-05-21T09:33:56.123" LastActivityDate="2020-05-21T09:33:56.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21355" PostTypeId="2" ParentId="21198" CreationDate="2020-05-20T21:15:41.697" Score="2" Body="&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$n * d$&lt;/span&gt; matrix where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of total nodes in the graph and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the dimension of embedding of each node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the notation in the question, the basic GNN formulation without self loop is: &lt;span class=&quot;math-container&quot;&gt;$\text{D}^{-1}\text{A}\text{H}$&lt;/span&gt;. If you study this equation closer then you will find that the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; row of &lt;span class=&quot;math-container&quot;&gt;$\text{A}\text{H}$&lt;/span&gt; generates the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; node's representation by summing the node representation of its neighboring nodes. Multiplying it with &lt;span class=&quot;math-container&quot;&gt;$\text{D}^{-1}$&lt;/span&gt; makes the aggregated representation normalized with respect to the degree of a node (number of neighbors). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By defining a metric called information score:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$||\text{H} - (\text{D}^{-1}\text{A}\text{H})||$$&lt;/span&gt;&#xA;we will be getting low values for nodes that are well represented by their local neighborhood nodes and high values for nodes that are having a hard time being represented/summarized by its neighboring nodes. &#xA;To approximate the graph information, the authors choose to preserve the nodes that can not be well represented by their neighbors, i.e., the nodes with relatively larger node information score will be preserved in the construction of the pooled graph, because the authors believe it can provide more information.&lt;/p&gt;&#xA;" OwnerUserId="37206" LastEditorUserId="37206" LastEditDate="2020-05-20T21:33:16.993" LastActivityDate="2020-05-20T21:33:16.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21356" PostTypeId="1" AcceptedAnswerId="21358" CreationDate="2020-05-20T21:48:43.617" Score="5" ViewCount="331" Body="&lt;p&gt;Here is my understanding of importance sampling. If we have two distributions &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;, where we have a way of sampling from &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; but not from &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;, but we want to compute the expectation wrt &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;, then we use importance sampling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula goes as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_q[x] = E_p\Big[x\frac{q(x)}{p(x)}\Big]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only limitation is that we need a way to compute the ratio. Now, here is what I don't understand. Without knowing the density function &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;, how can we compute the ratio &lt;span class=&quot;math-container&quot;&gt;$\frac{q(x)}{p(x)}$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because if we know &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;, then we can compute the expectation directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure I am missing something here, but I am not sure what. Can someone help me understand this?  &lt;/p&gt;&#xA;" OwnerUserId="36074" LastEditorUserId="2444" LastEditDate="2020-05-20T23:19:59.653" LastActivityDate="2020-05-21T09:33:40.807" Title="How can we compute the ratio between the distributions if we don't know one of the distributions?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;importance-sampling&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21357" PostTypeId="2" ParentId="18682" CreationDate="2020-05-20T22:03:53.197" Score="1" Body="&lt;p&gt;The advantage is basically a function of the actual return received and a baseline. The function of the baseline is to make sure that only the actions that are better than average receive a positive nudge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to estimate the baseline is to have a value function approximator. At every step, you train a NN, using the trajectories collected via the current policy, to predict the value function for states.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that answers your query.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastEditorUserId="36074" LastEditDate="2020-05-21T17:22:03.080" LastActivityDate="2020-05-21T17:22:03.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21358" PostTypeId="2" ParentId="21356" CreationDate="2020-05-20T22:18:21.540" Score="3" Body="&lt;p&gt;The rationale behind importance sampling is that &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt; is difficult to sample from but &lt;strong&gt;easy to evaluate&lt;/strong&gt;. Or at least you can easily evaluate some &lt;span class=&quot;math-container&quot;&gt;$\tilde{q}$&lt;/span&gt; such that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tilde{q}(z) = Zq(z)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; (scalar) might be unknown. The geometrical example would be here e.g. sampling uniformly from an area under the curve &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt; (in general it's not easy).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Because if we know &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;, then we can compute the expectation directly.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's the task we're trying to solve to begin with. And calculating expectation might be hard if we can't sample efficiently from &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you want to compute an expectation of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E[x]$&lt;/span&gt;. For this you need to calculate the following integral:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E[x] = \int{xq(x)dx}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is a probability distribution of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; for which you have an expression - so you can evaluate &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt; (up to the constant of proportionality). This integral might be hard to evaluate analytically so we need to use other methods such as Monte Carlo. Let's say it is hard to generate samples from &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (as per example above, e.g. generating samples from the area under the curve &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt; uniformly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can do is to calculate an expectation under a simple distribution &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (proposal distribution) which is a distribution of your choice that needs to allow you to easily sample from it (say Gaussian). Then you can rewrite your integral as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_q[x] = \int{xq(x)dx} = \int{xq(x) \color{blue}{\frac{p(x)}{p(x)}} dx}&#xA;= \int{x \frac{q(x)}{\color{blue}{p(x)}} \color{blue}{p(x)} dx}&#xA;= E_p \Big[{x\frac{q(x)}{p(x)}}\Big]&#xA;$$&lt;/span&gt;&#xA;(added index &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; to expectation to denote the sampling distribution). Now you can approximate the last expectation by Monte Carlo:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_p \Big[{x\frac{q(x)}{p(x)}}\Big] = \frac{1}{S} \sum_{s}{x^{(s)} \frac{q(x^{(s)})}{p(x^{(s)})} }, \ x^{(s)} \sim q(x)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="22835" LastEditorUserId="22835" LastEditDate="2020-05-21T08:26:13.260" LastActivityDate="2020-05-21T08:26:13.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21359" PostTypeId="2" ParentId="21353" CreationDate="2020-05-20T22:30:32.210" Score="1" Body="&lt;h2&gt;What is a neural network?&lt;/h2&gt;&#xA;&lt;p&gt;Many neural networks can be defined as a function &lt;span class=&quot;math-container&quot;&gt;$f: \mathbb{R}^n \rightarrow \mathbb{R}^m$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n, m \geq 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Equivalently, many neural networks can also be defined as a set of interconnected units (aka neurons or nodes) &lt;span class=&quot;math-container&quot;&gt;$f^i$&lt;/span&gt; that receive some input and produce output, i.e. &lt;span class=&quot;math-container&quot;&gt;$f^i(\mathbf{x}^i) = y^i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}^i \in \mathbb{R}^k$&lt;/span&gt;. The actual function &lt;span class=&quot;math-container&quot;&gt;$f^i$&lt;/span&gt; is variable and depends on the application or problem you want to solve. For example, &lt;span class=&quot;math-container&quot;&gt;$f^i$&lt;/span&gt; can just be a linear combination of the inputs, i.e. &lt;span class=&quot;math-container&quot;&gt;$f^i(\mathbf{x}^i) = \sum_{j} \mathbf{w}_j^i \mathbf{x}_{j}^i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}^i \in \mathbb{R}^k$&lt;/span&gt; is a vector of weights (aka parameters or coefficients). The linear combination can also be followed by a non-linear function, such as a sigmoid function.&lt;/p&gt;&#xA;&lt;p&gt;If the neural network (more precisely, its units) doesn't contain recurrent (aka cyclic) connections, then it can be called a feedforward neural network (FFNN).&lt;/p&gt;&#xA;&lt;h2&gt;What are the most common feedforward neural networks?&lt;/h2&gt;&#xA;&lt;h3&gt;Perceptron&lt;/h3&gt;&#xA;&lt;p&gt;The simplest (non-binary) FFNN is the &lt;strong&gt;perceptron&lt;/strong&gt;, where the inputs are directly connected to the outputs. The perceptron performs a linear combination of the inputs followed by a thresholding operation, so perceptrons can only represent straight-line functions, so they can only be used for classification or regression problems where your data is linearly separable. In fact, the perceptron cannot solve the XOR problem.&lt;/p&gt;&#xA;&lt;p&gt;Before the perceptron, &lt;a href=&quot;http://www.mind.ilstu.edu/curriculum/modOverview.php?modGUI=212&quot; rel=&quot;nofollow noreferrer&quot;&gt;McCulloch and Pitts&lt;/a&gt; had introduced simplified models of biological neurons, where all signals are binary, in an attempt to closely mimic their biological counterpart. The perceptron can actually be seen as an extension of this work. In fact, a perceptron can be viewed as a single artificial neuron.&lt;/p&gt;&#xA;&lt;h3&gt;Multi-layer perceptron&lt;/h3&gt;&#xA;&lt;p&gt;An FFNN with more layers (of units) between the input and the output is often called a &lt;strong&gt;multi-layer perceptron&lt;/strong&gt; (MLP). The layers in the middle are often denoted as &lt;em&gt;hidden layers&lt;/em&gt;. The MLP can represent not only linear functions (i.e. straight lines), but also more complicated functions by making use of non-linear functions, such as the sigmoid.&lt;/p&gt;&#xA;&lt;h3&gt;Convolution neural network&lt;/h3&gt;&#xA;&lt;p&gt;You can have other forms of FFNNs that perform other operations.&lt;/p&gt;&#xA;&lt;p&gt;For example, a &lt;strong&gt;convolution neural network&lt;/strong&gt; (CNN), provided it doesn't contain recurrent connections, is an FFNN that performs the &lt;a href=&quot;https://ai.stackexchange.com/a/22000/2444&quot;&gt;convolution operation&lt;/a&gt; (and often also a sub-sampling operation). For this reason, they are particularly suited to deal with images (and videos). (This shouldn't be surprising if you are familiar with the basics of image processing and computer vision, which I don't think it's the case)&lt;/p&gt;&#xA;&lt;p&gt;However, note that CNNs can also have recurrent connections, but this is not usually the case.&lt;/p&gt;&#xA;&lt;h3&gt;Residual neural network&lt;/h3&gt;&#xA;&lt;p&gt;There are also &lt;strong&gt;residual neural networks&lt;/strong&gt;, i.e. neural networks where a node in a certain layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; can be connected to other nodes in layers &lt;span class=&quot;math-container&quot;&gt;$l+j$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$j \geq 1$&lt;/span&gt;, as opposed to being connected only to the nodes in layer &lt;span class=&quot;math-container&quot;&gt;$l+1$&lt;/span&gt;, which is the typical case.&lt;/p&gt;&#xA;&lt;h3&gt;Auto-encoders&lt;/h3&gt;&#xA;&lt;p&gt;Auto-encoders are neural networks that compress the input and then decompress it. The answers to &lt;a href=&quot;https://ai.stackexchange.com/q/11405/2444&quot;&gt;this question&lt;/a&gt; may help you to understand why AEs would be useful.&lt;/p&gt;&#xA;&lt;h2&gt;What kind of inputs do they receive?&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What kind of inputs do they receive? For example, do they receive binary numbers, real numbers, vectors, or matrics?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In principle, each of these FFNNs can receive either binary or real numbers or vectors (either of real or binary numbers). However, certain NNs are more appropriate to deal with certain inputs. For example, CNNs are more appropriate for images (which are typically represented as matrices or tensors).&lt;/p&gt;&#xA;&lt;h2&gt;How can you further classify the NNs?&lt;/h2&gt;&#xA;&lt;p&gt;Based on &lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 2 of the book Neural Networks - A Systematic Introduction&lt;/a&gt; (1996) by Raul Rojas, you can also divide neural networks into other categories&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Unweighted (i.e. binary, such as the McCulloch and Pitts' model) vs weighted (e.g. the perceptron)&lt;/li&gt;&#xA;&lt;li&gt;Synchronous vs asynchronous (e.g. Hopfield networks, which are recurrent neural networks, though)&lt;/li&gt;&#xA;&lt;li&gt;Neural networks that store states vs NNs that don't store states&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You could also distinguish between FFNNs based on the learning algorithm. Nowadays, the widely used NNs are trained with gradient descent (and back-propagation to compute the gradients), but there are other approaches to train NNs, such as evolutionary algorithms or Hebbian learning. Moreover, you could also distinguish between neural networks that compute a deterministic function and neural networks that have some randomness or stochasticity inside them (e.g. Bayesian neural networks). There are probably many more possible subdivisions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-20T12:12:53.437" LastActivityDate="2020-07-20T12:12:53.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21360" PostTypeId="2" ParentId="21356" CreationDate="2020-05-20T22:45:08.730" Score="3" Body="&lt;p&gt;It is common in Bayesian statistics to only know the posterior up to a constant of proportionality. This means that we can't directly sample from the posterior. However, using importance sample we are able to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider our posterior density &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is only known up to some constant, i.e. &lt;span class=&quot;math-container&quot;&gt;$\pi(x) = K \tilde{\pi}(x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is some constant and we only have &lt;span class=&quot;math-container&quot;&gt;$\tilde{\pi}$&lt;/span&gt;. Then by importance sampling we can evaluate the expectation of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (or any function thereof) as follows by using a proposal density &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA; \mathbb{E}_\pi[X] &amp;amp; = \int_\mathbb{R} x \frac{\pi(x)}{q(x)}q(x)dx \; ; \\&#xA;&amp;amp; = \frac{\int_\mathbb{R} x \frac{\pi(x)}{q(x)}q(x)dx}{\int_\mathbb{R}\frac{\pi(x)q(x)}{q(x)}dx} \; ;\\&#xA;&amp;amp; = \frac{\int_\mathbb{R} x \frac{\tilde{\pi}(x)}{q(x)}q(x)dx}{\int_\mathbb{R}\frac{\tilde{\pi}(x)q(x)}{q(x)}dx} \; ; \\&#xA;&amp;amp; = \frac{\mathbb{E}_q[xw(x)]}{\mathbb{E}_q[w(x)]} \; ;&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$w(x) = \frac{\tilde{\pi}(x)}{q(x)}$&lt;/span&gt;. Note that on line two we have not done anything crazy - as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a density we know that it integrates to one and then we multiply the integral by &lt;span class=&quot;math-container&quot;&gt;$1 = \frac{q(x)}{q(x)}$&lt;/span&gt;. The thing to notice is that the if we were to write &lt;span class=&quot;math-container&quot;&gt;$\pi(x) = K \tilde{\pi}(x)$&lt;/span&gt; then the constants &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; in the integrals would cancel, and so we have our result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarise - we can sample from a distribution that is difficult/impossible to sample from (e.g. because we only know the density up to a constant of proportionality) by using importance sampling, as this allows us to calculate the importance ratio and use samples that are generated from a distribution of our choosing that is easier to sample from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that importance sampling isn't just used in Bayesian statistics - for instance it could be used in Reinforcement Learning as an off policy way of sampling from the environment whilst still evaluating the value of the policy you're interested in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: as requested I have added a concrete example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example to make things concrete - suppose we have &lt;span class=&quot;math-container&quot;&gt;$Y_i | \theta \sim \text{Poisson}(\theta)$&lt;/span&gt; and we are interested in &lt;span class=&quot;math-container&quot;&gt;$\theta \in (0, \infty)$&lt;/span&gt;. The likelihood function for the Poisson distribution is &#xA;    &lt;span class=&quot;math-container&quot;&gt;$$ f(\textbf{y} | \theta) = \prod\limits_{i=1}^n \frac{\theta^{y_i}\exp(-\theta)}{y_i\!}\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can then assign a gamma prior to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, that is we say that &lt;span class=&quot;math-container&quot;&gt;$\theta \sim \text{Gamma}(a,b)$&lt;/span&gt; with density &#xA;    &lt;span class=&quot;math-container&quot;&gt;$$\pi(\theta) \propto \theta^{a-1} \exp(-b\theta)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By applying Bayes rule our posterior is then &#xA;    &lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;    \pi(\theta|\textbf{y}) &amp;amp; \propto f(\textbf{y} | \theta) \pi(\theta) \\&#xA;    &amp;amp; = \prod\limits_{i=1}^n \frac{\theta^{y_i}\exp(-\theta)}{y_i\!} \times \theta^{a-1} \exp(-b\theta) \\&#xA;    &amp;amp; = \theta^{\sum\limits_{i=1}^n y_i + a - 1} \exp(-[n+b]\theta)\;.&#xA;\end{align}&lt;/span&gt;&#xA;Now we know that this is the kernel of a Gamma(&lt;span class=&quot;math-container&quot;&gt;$\sum\limits_{i=1}^n y_i + a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n+b$&lt;/span&gt;) distribution, but assume that we didn't know this and didn't want to calculate the normalising integral. This would mean that we are not able to calculate the mean of our posterior density, or even sample from it. This is where we can use importance sampling, for instance we could choose an Exponential(1) proposal distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would sample say 5000 times from the exponential distribution and then calculate the two expectations using MC integration and obtain an estimate for the mean of the posterior. NB that in this example &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; from earlier would be &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is some Python code to further demonstrate this. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;np.random.seed(1)&#xA;&#xA;# sample our data&#xA;y = np.random.poisson(lam=0.5,size = 100)&#xA;&#xA;# sample from proposal&#xA;samples_from_proposal = np.random.exponential(scale=1,size=5000)&#xA;&#xA;# set parameters for the prior&#xA;a = 5; b = 3&#xA;&#xA;def w(x, y, a, b):&#xA;    # calculates the ratio between our posterior kernel and proposal density&#xA;    pi = x ** (np.sum(y) + a - 1) * np.exp(-(len(y) + b) * x)&#xA;    q = np.exp(-x)&#xA;    return pi/q&#xA;&#xA;# calculate the top expectation&#xA;top = np.mean(samples_from_proposal * w(samples_from_proposal,y,a,b))&#xA;&#xA;# calculate the bottom expectation&#xA;bottom = np.mean(w(samples_from_proposal,y,a,b))&#xA;&#xA;print(top/bottom)&#xA;&#xA;# calculate the true mean since we knew the posterior was actually a gamma density&#xA;true_mean = (np.sum(y) + a)/(len(y) + b)&#xA;print(true_mean)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this you should see that the Expectation from importance sampling is 0.5434 whereas the true mean is 0.5436 (both of which are close to the true value of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; that I used to simulate the data from) so importance sampling approximates the expectation well. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-05-21T09:33:40.807" LastActivityDate="2020-05-21T09:33:40.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21361" PostTypeId="1" AcceptedAnswerId="21367" CreationDate="2020-05-21T02:02:30.193" Score="3" ViewCount="146" Body="&lt;p&gt;I have been reading about LSTMs and GRUs, which are recurrent neural networks (RNNs). The difference between the two is the number and specific type of gates that they have. The GRU has an &lt;em&gt;update&lt;/em&gt; gate, which has a similar role to the role of the &lt;em&gt;input&lt;/em&gt; and &lt;em&gt;forget&lt;/em&gt; gates in the LSTM.&lt;/p&gt;&#xA;&lt;p&gt;Here's a diagram that illustrates both units (or RNNs).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://miro.medium.com/max/1280/1*yBXV9o5q7L_CvY7quJt3WQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://miro.medium.com/max/1280/1*yBXV9o5q7L_CvY7quJt3WQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;With respect to the vanilla RNN, the LSTM has more &amp;quot;knobs&amp;quot; or parameters. So, why do we make use of the GRU, when we clearly have more control over the neural network through the LSTM model?&lt;/p&gt;&#xA;&lt;p&gt;Here are two more specific questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When would one use Long Short-Term Memory (LSTM) over Gated Recurrent Units (GRU)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the advantages/disadvantages of using LSTM over GRU?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2021-01-18T21:31:39.420" LastActivityDate="2021-01-18T21:31:39.420" Title="What's the difference between LSTM and GRU?" Tags="&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21362" PostTypeId="2" ParentId="2349" CreationDate="2020-05-21T02:54:03.893" Score="0" Body="&lt;p&gt;I tried to use 2 hidden ReLU-based unit, 1 output unit to solve the XOR problem and found that gradient will always become really small after training 1000 times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Loss vs training times:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5k4pz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5k4pz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the gradient looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gyT5T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gyT5T.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that means the units all dead. The robust way to solve this problem is increase the number of units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to 4 units, some times I will success, but sometimes not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And 5 units, I will fail but the rate decrease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s40oj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s40oj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so on. That is all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will try to use sigmoid + cross entropy instead of ReLU, I imagine linear function will work better in this case.&lt;/p&gt;&#xA;" OwnerUserId="37222" LastActivityDate="2020-05-21T02:54:03.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21363" PostTypeId="2" ParentId="21315" CreationDate="2020-05-21T03:26:00.770" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/lite/performance/model_optimization#quantization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantization&lt;/a&gt; is a technique used to make deep learning models smaller and faster to run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep learning models are essentially collections of real-valued numbers. Because there are infinitely many real numbers, computers represent them using a format call 'floating point' numbers, which are not completely accurate. For example, a 32-bit floating point number can only represent at most &lt;span class=&quot;math-container&quot;&gt;$2^{32}$&lt;/span&gt; distinct values. In contrast, a 64-bit floating point number can represent &lt;span class=&quot;math-container&quot;&gt;$2^{64}$&lt;/span&gt; distinct values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most CPUs and GPUs cannot operate directly on large floating point numbers. This means that to do something like multiply two floating point numbers, the CPU might have to work on half of each number at a time, and do some tricky work to combine the results. Some CPUs and GPUs can operate directly on large floating point numbers, but only by using more than one core to work on a single number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get around this, you might chose to take a model you have that was trained with high-precision floating point weights, and reduce it to lower precision. The weights won't be exactly the same, but they'll be very close. Doing this will make the models run much faster, but you might lose some accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the advantage of using a tool that only supports Quantized models is that models will run faster, but might be slightly less accurate.&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2020-05-21T03:26:00.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21365" PostTypeId="1" CreationDate="2020-05-21T06:36:41.000" Score="1" ViewCount="31" Body="&lt;p&gt;I am trying to use LSTM to do text classification and monitor the training process with tensorboard. But it seems that this model doesn't learn anything in early epochs. Is it normal for LSTM networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the definition of model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class RNN(nn.Module):&#xA;    &quot;&quot;&quot;&#xA;        RNN model for text classification&#xA;    &quot;&quot;&quot;&#xA;    def __init__(self, vocab_size, num_class, emb_dim, emb_droprate, rnn_cell_hidden, rnn_cell_type, birnn, num_layers, rnn_droprate, sequence_len):&#xA;        super().__init__()&#xA;        self.vocab_size = vocab_size                # vocab size&#xA;        self.emb_dim = emb_dim                      # embedding dimension&#xA;        self.emb_droprate = emb_droprate            # embedding droprate&#xA;        self.num_class = num_class                  # classes&#xA;        self.rnn_cell_hidden = rnn_cell_hidden      # hidden layer size&#xA;        self.rnn_cell_type = rnn_cell_type          # rnn cell type&#xA;        self.birnn = birnn                          # wheather use bidirectional rnn&#xA;        self.num_layers = num_layers                # number of rnn layers&#xA;        self.rnn_droprate = rnn_droprate            # rnn dropout rate before fc&#xA;        self.sequence_len = sequence_len            # fix sequence length, so we dont need loop&#xA;        pass&#xA;&#xA;    def build(self):&#xA;        self.embedding = nn.Embedding(self.vocab_size, self.emb_dim)&#xA;        self.emb_dropout = nn.Dropout(self.emb_droprate)&#xA;        if self.rnn_cell_type == &quot;LSTM&quot;:&#xA;            self.rnn = nn.LSTM(input_size=self.emb_dim, hidden_size=self.rnn_cell_hidden, num_layers=self.num_layers, bidirectional=self.birnn, batch_first=True)&#xA;        elif self.rnn_cell_type == &quot;GRU&quot;:&#xA;            self.rnn = nn.GRU(input_size=self.emb_dim, hidden_size=self.rnn_cell_hidden, num_layers=self.num_layers, bidirectional=self.birnn, batch_first=True)&#xA;        else:&#xA;            self.rnn = None&#xA;            print(&quot;unsupported rnn cell type, valid is [LSTM, GRU]&quot;)&#xA;        if self.birnn:&#xA;            self.fc = nn.Linear(2 * self.rnn_cell_hidden, self.num_class)&#xA;        else:&#xA;            self.fc = nn.Linear(self.rnn_cell_hidden, self.num_class)&#xA;&#xA;        self.rnn_dropout = nn.Dropout(self.rnn_droprate)&#xA;&#xA;    def forward(self, input_):&#xA;        batch_size = input_.shape[0]&#xA;&#xA;        x = self.embedding(input_)&#xA;        x = self.emb_dropout(x)&#xA;&#xA;        if self.rnn_cell_type == &quot;LSTM&quot;:&#xA;            if self.birnn:&#xA;                h_0 = torch.zeros(self.num_layers * 2, batch_size, self.rnn_cell_hidden, requires_grad=True).to(device)&#xA;                c_0 = torch.zeros(self.num_layers * 2, batch_size, self.rnn_cell_hidden, requires_grad=True).to(device)&#xA;            else:&#xA;                h_0 = torch.zeros(self.num_layers, batch_size, self.rnn_cell_hidden, requires_grad=True).to(device)&#xA;                c_0 = torch.zeros(self.num_layers, batch_size, self.rnn_cell_hidden, requires_grad=True).to(device)&#xA;            output, (h_n, c_n) = self.rnn(x, (h_0, c_0))&#xA;        elif self.rnn_cell_type == &quot;GRU&quot;:&#xA;            if self.birnn:&#xA;                h_0 = torch.zeros(self.num_layers * 2, batch_size, self.rnn_cell_hidden, requires_grad=True).to(device)&#xA;            else:&#xA;                h_0 = torch.zeros(self.num_layers, batch_size, self.rnn_cell_hidden, requires_grad=True).to(device)&#xA;            output, h_n = self.rnn(x, h_0)&#xA;&#xA;        if self.birnn:&#xA;            x = h_n.view(self.num_layers, 2, batch_size, self.rnn_cell_hidden)&#xA;            x = torch.cat((x[-1, 0, : , : ], x[-1, 1, : , : ]), dim = 1)&#xA;        else:&#xA;            x = h_n.view(self.num_layers, 1, batch_size, self.rnn_cell_hidden)&#xA;            x = x[-1, 0, : , : ]&#xA;&#xA;        x = x.view(batch_size, 1, -1)           # shape: [batch_size, 1, 2 or 1 * rnn_cell_hidden]&#xA;        x = self.rnn_dropout(x)&#xA;&#xA;        x = self.fc(x)&#xA;        x = x.view(-1, self.num_class)          # shape: [batch_size, num_class]&#xA;&#xA;        return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Parameters of this model:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;vocab size: 4805 &lt;/li&gt;&#xA;&lt;li&gt;number of classes: 27&lt;/li&gt;&#xA;&lt;li&gt;embedding dimension: 300&lt;/li&gt;&#xA;&lt;li&gt;embedding dropoutrate: 0.5&lt;/li&gt;&#xA;&lt;li&gt;rnn cell type: LSTM&lt;/li&gt;&#xA;&lt;li&gt;rnn cell hidden size: 1000&lt;/li&gt;&#xA;&lt;li&gt;bidirectional rnn: False&lt;/li&gt;&#xA;&lt;li&gt;number of lstm layers: 1&lt;/li&gt;&#xA;&lt;li&gt;dropout rate at last lstm layer hidden: 0.5&lt;/li&gt;&#xA;&lt;li&gt;padded sequence length: 64&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Optim:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;criterion = nn.CrossEntropyLoss().to(device)&#xA;optimizer = optim.Adam(model.parameters(), lr=args.lr, weight_decay=1e-6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;learning rate here is &lt;code&gt;0.001&lt;/code&gt;, batch size is &lt;code&gt;32&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tensorboard graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JvftL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JvftL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XUCFp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XUCFp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that this model starts learning after epoch 15. Is it normal?&lt;/p&gt;&#xA;" OwnerUserId="37246" LastActivityDate="2020-05-21T06:36:41.000" Title="My LSTM text classification model seems not learn anything in early epochs" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21367" PostTypeId="2" ParentId="21361" CreationDate="2020-05-21T08:17:16.437" Score="1" Body="&lt;p&gt;On the same problems, sometimes GRU is better, sometimes LSTM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, having more parameters (more &quot;knobs&quot;) is not always a good thing. The training process needs to learn those parameters. There is a higher chance of over-fitting, amongst other problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameters are assigned specific roles inside either GRU or LSTM, so if that role is less important for a specific learning challenge, then it can be wasteful or even counter-productive to have the system attempt to learn values for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to find out if LSTM is better than GRU on a problem is a hyperparameter search. Unfortunately, you cannot simply swap one for the other, and test that, because the number of cells that optimises a LSTM solution will be different to the number that optimises a GRU.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When would one use Long Short-Term Memory (LSTM) over Gated Recurrent Units (GRU)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When it proves better, experimentally. In some problem domains, this may be established and you can check. However, in other problem domains if either GRU or LSTM works well enough to solve a problem (and the superiority of either LSTM or GRU is not the main point of the work), then it may not so clear.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-21T08:17:16.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21368" PostTypeId="1" CreationDate="2020-05-21T08:25:22.417" Score="1" ViewCount="30" Body="&lt;p&gt;Let's assume I use convolutional networks for time-series prediction. Data I feed to the network have 1 channel depth, height of number of periods and number of features is the width, so the frame size is: &#xA;[1, periods, features]. Batch size is not relevant here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a difference between using 1d convolutions along time (height) dimension and 2d convolutional that will have a kernel size of for example (3, 1) or (5, 1), so that the larger number convolutes along the time dimension, and there is no convolution along features dimension?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2020-05-21T08:25:22.417" Title="Is there a difference between using 1d conv layers and 2d conv layers with kernel with size of 1 along other than time dimension?" Tags="&lt;convolutional-neural-networks&gt;&lt;prediction&gt;&lt;time-series&gt;&lt;convolution&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21369" PostTypeId="1" AcceptedAnswerId="21373" CreationDate="2020-05-21T08:42:21.550" Score="1" ViewCount="125" Body="&lt;p&gt;I ran a deep q learning algorithm (DQN) for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; number of epochs and got policy &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt;. &#xA;I reran the same script for the same &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; number of epochs and got policy &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt;. I expected &lt;span class=&quot;math-container&quot;&gt;$\pi_1 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; to be similar because i ran the same script. However, when computing the actions on the same test set, i realised the actions were very different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this supposed to be normal when training deep q networks or is there something that I am missing ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using prioritised experience replay when training the model.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-05-21T13:16:02.927" Title="Can deep reinforcement learning algorithms be deterministic in their reproducibility in results?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21370" PostTypeId="1" CreationDate="2020-05-21T09:14:28.637" Score="1" ViewCount="35" Body="&lt;p&gt;How can I increase the exploration in the Proximal Policy Optimation reinforcement learning algorithm? Is there a variable assigned for this purpose? I'm using the stable-baseline implementation: &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/modules/ppo2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stable-baselines.readthedocs.io/en/master/modules/ppo2.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34341" LastEditorUserId="2444" LastEditDate="2020-05-21T10:06:44.397" LastActivityDate="2020-05-21T10:06:44.397" Title="How can I increase the exploration in the Proximal Policy Optimation algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21373" PostTypeId="2" ParentId="21369" CreationDate="2020-05-21T10:37:14.853" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can deep reinforcement learning algorithms be deterministic in their reproducibility in results?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but only if you control all places in the code where stochastic methods are used (typically by seeding the affected RNGs):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Neural network weight initialisation&lt;/li&gt;&#xA;&lt;li&gt;Action choice for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy or other behaviour policy (does not apply in your case, because you work exclusively from experience replay)&lt;/li&gt;&#xA;&lt;li&gt;Minibatch sampling from experience replay&lt;/li&gt;&#xA;&lt;li&gt;Stochastic choices in the environment (does not apply in your case)&lt;/li&gt;&#xA;&lt;li&gt;Other stochastic parts of training that may be in use, such as dropout regularisation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Controlling all these should make your training process deterministic and repeatable. It won't necessarily make it correct.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I reran the same script for the same &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; number of epochs and got policy &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt;. I expected &lt;span class=&quot;math-container&quot;&gt;$\pi_1 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; to be similar because i ran the same script.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is subtly different. It seems you hoped that convergence of the algorithm would mean you got to the same approximately optimal policy. In principle this is possible, because Q-learning should find a deterministic policy. However, there are some details to bear in mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Many environments support multiple equivalent optimal policies. A simple grid world can have multiple equivalent paths from start to goal states. A Q-learning with approximation function will slightly prefer one or other path, resulting in very different, but still optimal, policies.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Q-learning with approximation can go wrong and learn incorrectly. The usual checks and balances against this are running large numbers of simulations and testing. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You don't have great options here, from your comments you are training purely offline from historic data. Your one sanity check - do I get the same policy if I re-try - has shown inconsistency. However, it doesn't necessarily mean you have a problem, perhaps the two policies are equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a couple of additional tests that may help:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of looking at the maximising action choice in the test data, look at how each Q function scores the behaviour policy action choice. If the scores are close (by some measure such as MSE), then the two Q-learners are basically agreeing and are more likely to have equivalent but different policies, as opposed to radically different end results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have each Q network score the other's Q function action choice over an arbitrary (but realistic) set of states. If the values are similar to each other, then again this points to successful convergence given the training data, but with different outcomes due to small details.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If either of these checks shows the networks are radically different, then you have a problem. Which run, if any, has found a viable policy, and which has failed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if the checks agree, it is circumstantial evidence that the Q learning process is stable, not proof that you have an agent that is better than the prevailing behaviour policy in your real world system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You won't know if the agent is truly better, unless you can find a more independent way to assess the agent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-21T13:16:02.927" LastActivityDate="2020-05-21T13:16:02.927" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21374" PostTypeId="1" AcceptedAnswerId="21379" CreationDate="2020-05-21T10:51:45.517" Score="2" ViewCount="54" Body="&lt;p&gt;What are the main differences and similarities between sparse autoencoders and convolution autoencoders?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When should one be preferred over the other? What are their applications?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(References are welcome. Somehow I was not able to find any comparisons of these autoencoders although I looked in a few textbooks and searched for material online. I was able to find the descriptions of each autoencoder separately, but what I am interested in is the comparison.)&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-21T11:08:38.703" LastActivityDate="2020-05-21T13:15:05.607" Title="What are the main differences between sparse autoencoders and convolution autoencoders?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21375" PostTypeId="2" ParentId="21347" CreationDate="2020-05-21T10:57:07.100" Score="1" Body="&lt;p&gt;I think you should use Keras embedding layer. It will be too easier than what you are doing.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Steps&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create Embedding Matrix&lt;/li&gt;&#xA;&lt;li&gt;add matrix to embedding layer while building model.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;You will find detailed article&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21393" LastActivityDate="2020-05-21T10:57:07.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21376" PostTypeId="2" ParentId="21170" CreationDate="2020-05-21T11:38:09.290" Score="1" Body="&lt;p&gt;As you mentioned in the comments about a possible problem of using mean, median type of imputations naively could lead to wrong predictions. In such cases, you need to first check whether you have &lt;em&gt;enough&lt;/em&gt; data. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;If you have enough data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can try using &lt;a href=&quot;https://www.rdocumentation.org/packages/mice/versions/3.8.0/topics/mice&quot; rel=&quot;nofollow noreferrer&quot;&gt;MICE&lt;/a&gt; (Multivariate Imputation By Chained Equations) algorithm on your missing data. The method is based on Fully Conditional Specification, where each incomplete variable is imputed by a separate model. The MICE algorithm can impute mixes of continuous, binary, unordered categorical and ordered categorical data. One note of caution with this method: It is a computationally expensive method, so it use it if you are not short on time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important thing to keep in mind is that in order to tackle such problems, you would be needing multiple iterations as a part of your algorithm. The conventional setup you are describing, does not seem to iterative in nature and hence you are running into the problem of features being input and output at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by any chance you insist on finding the missing values just for solving a downstream task like classification or regression, you can try the XgBoost algorithm. It can be used as a classifier or as a regressor. This algorithm can handle missing values inherently. Source: &lt;a href=&quot;https://stats.stackexchange.com/questions/235489/xgboost-can-handle-missing-data-in-the-forecasting-phase&quot;&gt;this answer&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;If you don't have enough data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In such a case you would need to introduce bias in your model using your insights or domain knowledge about the problem. For, e.g., in the a possible problem of estimating weights using heights, you had an insight that your data comprises more of short people. So instead of naively using median values of the total dataset for weight, you can try to bin the data according to their heights, say 'S', 'M', 'L', 'XL', and estimate the weights of each bin separately using the median values of their respective bins. The thing to keep in mind is that when data is low, you need to provide knowledge to the model by enforcing some bias using your insights and domain knowledge about the problem.&lt;/p&gt;&#xA;" OwnerUserId="37206" LastEditorUserId="37206" LastEditDate="2020-05-21T11:44:40.437" LastActivityDate="2020-05-21T11:44:40.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21379" PostTypeId="2" ParentId="21374" CreationDate="2020-05-21T13:15:05.607" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Sparse auto-encoders&lt;/strong&gt; (SAEs)&lt;/a&gt; are auto-encoders that impose constraints on the parameters so that they are &lt;em&gt;sparse&lt;/em&gt; (i.e. zero or close to zero). This can be achieved in different ways. For example, you can train an auto-encoder with &lt;a href=&quot;https://papers.nips.cc/paper/3313-sparse-deep-belief-net-model-for-visual-area-v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a loss function that includes a penalty term&lt;/a&gt; (to constraint the parameters to be close to zero or zero) or you e.g. &lt;a href=&quot;https://arxiv.org/pdf/1312.5663.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;set the smallest activations to zero&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convolution auto-encoders (CAEs)&lt;/strong&gt; are auto-encoders that use the convolution operation. So, they can be viewed as the auto-encoder version of convolutional neural networks. For this reason, they are particularly suited to compress and reconstruct images. The authors of the original paper, &lt;a href=&quot;http://people.idsia.ch/~juergen/icann2011stack.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction&lt;/a&gt;, train them with gradient descent and back-propagation by minimizing the mean squared error, so there's e.g. no penalty term, but you can probably combine SAEs with CAEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you could say that CAEs are &lt;em&gt;sparse&lt;/em&gt; with respect to the traditional auto-encoder (in the same way that you can say that CNNs are sparse with respect to fully connected neural networks), so, in this sense, CAEs are also sparse auto-encoders.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-21T13:15:05.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21380" PostTypeId="1" CreationDate="2020-05-21T13:38:33.120" Score="1" ViewCount="26" Body="&lt;p&gt;I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to find the non-zero connections that they have in common. Any idea how to visualize this or compute this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Python 3.7 and TensorFlow 2.0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the convergence of a neural network following the random weight initialization, some weights/connections increase (magnitude), while other weights decrease. You can then prune the smallest magnitude weights. I want to compare the remaining weights for say two networks having the same level of sparsity of say 50%. The idea is to have an idea of which weights were pruned away and which weights/connections remain.&lt;/p&gt;&#xA;" OwnerUserId="31215" LastEditorUserId="31215" LastEditDate="2020-05-22T01:43:54.387" LastActivityDate="2020-05-22T01:43:54.387" Title="How can I find the similar non-zero connections between different levels of sparsity of the same network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21381" PostTypeId="1" CreationDate="2020-05-21T14:17:07.917" Score="1" ViewCount="33" Body="&lt;p&gt;I understand that a tree-based variant will have nodes repeatedly added to the frontier. How do I craft an example where a particular goal node is never found. Is this example valid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T2kKS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T2kKS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, how do I explain that the graph-based version of the greedy best-first search is complete? &lt;/p&gt;&#xA;" OwnerUserId="37258" LastEditorUserId="2444" LastEditDate="2020-05-21T14:19:52.930" LastActivityDate="2020-05-21T14:19:52.930" Title="What is a example showing that the tree-based variant for the greedy best-first search is incomplete?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;best-first-search&gt;&lt;tree-search&gt;&lt;graph-search&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21382" PostTypeId="1" CreationDate="2020-05-21T15:40:18.533" Score="6" ViewCount="160" Body="&lt;p&gt;I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems in mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seeing how well recurrent neural networks can understand human language, I suppose that they could also be used to follow some simple mathematical statements and maybe even come up with some proofs. I know that computer-assisted proofs are more and more frequent and that some software can now understand simple mathematical language and verify proofs (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Coq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coq&lt;/a&gt;). Still, I've never heard of deep learning applied to mathematical research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can deep learning be used to help mathematical research? So, I am curious about whether systems like Coq could be combined with deep learning systems to help mathematical research. Are there some exciting results?&lt;/p&gt;&#xA;" OwnerUserId="37262" LastEditorUserId="4446" LastEditDate="2020-05-21T20:21:51.557" LastActivityDate="2020-06-10T13:14:01.577" Title="Can deep learning be used to help mathematical research?" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;proofs&gt;&lt;coq&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21383" PostTypeId="1" CreationDate="2020-05-21T16:13:27.670" Score="1" ViewCount="23" Body="&lt;p&gt;I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at &lt;a href=&quot;https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that the number of LSTMs in the encoder would have to be the same as number of words in the text (if word embeddings are being used) or characters in the text (if char embeddings are being used). For char embeddings, each embedding would correspond to 1 LSTM in 1 direction and 1 encoder hidden state. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this understanding correct? &#xA;E.g. If we have another model that uses encoder-decoder for a different application (say text-to-speech synthesis described here &lt;a href=&quot;https://ai.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html&lt;/a&gt;) tha uses 256 LSTMs in each direction of the bidirectional-encoder, does it mean the input to this encoder is limited to 256 characters of text?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can the decoder output has to be same length as the encoder input or can it be different? If different what factor describes what the decoder output length should be?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="33580" LastActivityDate="2020-05-21T16:13:27.670" Title="Is the number of bidirectional LSTMs in seq2seq model equal to the maximum length of input text/characters?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;bert&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21384" PostTypeId="2" ParentId="20959" CreationDate="2020-05-21T18:54:02.640" Score="0" Body="&lt;p&gt;If your intention is to learn make the agent learn which has the min arbitrary value, then you would need to modify your rewards a bit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current reward structure provides the incentive to just move to a stage where it gets a reward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if it is at state 0, it gets the same reward to go to either state 2 or state 3, since both of them have a higher inverse value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the agent learn to move to state 2, you would have to provide it with more incentives to go to state 2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def reward(s,a,s_dash):&#xA;    if s_dash == 2:&#xA;        return 5&#xA;    elif inverse_values_for_states[s]&amp;lt;inverse_values_for_states[s_dash]: &#xA;        return 1&#xA;    elif inverse_values_for_states[s]&amp;gt;inverse_values_for_states[s_dash]:&#xA;        return -1&#xA;    else:&#xA;        return 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried using this and it converges to 2. This is a hard-coded version, but I guess you get the idea.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-05-21T18:54:02.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21385" PostTypeId="1" CreationDate="2020-05-21T19:25:46.587" Score="1" ViewCount="71" Body="&lt;p&gt;I have a string of characters encoding a molecule. I want to regress some properties of those molecules. I tried using an LSTM that encodes all one hot encdoed characters, and then I take the last hidden state fed into a linear layer to regress the property. This works fine, but I wanted to see if transformers can do better, since they are so good in NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am not quiet sure about two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pytorch transformer encoder layer has two masking parameters: &quot;src_mask&quot; and &quot;src_key_padding_mask&quot;. The model needs the whole string to do the regression, so I dont think I need &quot;src_mask&quot;, but I do padding with 0 for parallel processing, is that what &quot;src_key_padding_mask&quot; is for?&lt;/li&gt;&#xA;&lt;li&gt;What output from the transformer do I feed into the linear regression layer? For the LSTM I took the last hidden output. For the transformer, since everything is processed in parallel, I feel like I should rather use the sum of all, but it doesn't work well. Instead using only the last state works better, which seems arbitrary to me. Any ideas on how to properly do this, how do sentiment analysis model do it?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31821" LastActivityDate="2020-05-21T19:25:46.587" Title="Transformer encoding for regression" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21387" PostTypeId="2" ParentId="21282" CreationDate="2020-05-21T19:31:41.313" Score="3" Body="&lt;p&gt;Softmax policy &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta(s,a)$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$\frac{\exp{(\phi(s,a)^T \theta})}{\Sigma \exp{(\phi(s,a) ^T \theta) }}$&lt;/span&gt;, where the summation is over the action space.&lt;br&gt;&#xA;Taking log, this becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\log \pi_\theta(s,a) = log(e^{\phi(s,a) ^T \theta}) - log({\Sigma e^{\phi(s,a) ^T \theta }}) \\&#xA;= \phi(s,a) ^T \theta - log({\Sigma e^{\phi(s,a)^T \theta }})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking derivative wrt &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, this becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\theta \log \pi_\theta(s,a) = \phi(s,a) - \nabla_\theta log({\Sigma e^{\phi(s,a) ^T \theta }})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can rewrite &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta log({\Sigma e^{\phi(s,a)^T \theta }})$&lt;/span&gt; as follows.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\theta log({\Sigma e^{\phi(s,a)^T \theta }}) &#xA;= \frac{\nabla_\theta \Sigma e^{\phi(s,a)^T \theta}}{\Sigma e^{\phi(s,a) ^T \theta}} &#xA;= \frac{\Sigma \phi(s,a) e^{\phi(s,a) ^T \theta}}{\Sigma e^{\phi(s,a) ^T \theta}}&#xA;= \Sigma \phi(s,a) \pi_\theta(s,a)&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final equation then becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_\theta \log \pi_\theta(s,a) = \phi(s,a) - \Sigma \phi(s,a) \pi_\theta(s,a)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="36074" LastEditorUserId="36074" LastEditDate="2020-05-21T20:08:46.850" LastActivityDate="2020-05-21T20:08:46.850" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21388" PostTypeId="1" AcceptedAnswerId="21408" CreationDate="2020-05-22T02:23:29.807" Score="5" ViewCount="356" Body="&lt;p&gt;In reinforcement learning, temporal difference seem to update the value function in each new iteration of experience absorbed from the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the conditions for temporal-difference learning to converge in the end? How is it guaranteed to converge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any intuitive understanding of those conditions that lead to the convergence?&lt;/p&gt;&#xA;" OwnerUserId="37275" LastEditorUserId="2444" LastEditDate="2020-05-22T10:40:15.060" LastActivityDate="2020-05-23T18:41:37.073" Title="What are the conditions of convergence of temporal-difference learning?" Tags="&lt;reinforcement-learning&gt;&lt;convergence&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21389" PostTypeId="1" AcceptedAnswerId="21404" CreationDate="2020-05-22T02:43:06.137" Score="4" ViewCount="508" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://medium.com/saarthi-ai/transformers-attention-based-seq2seq-machine-translation-a28940aaa4fe&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention&lt;/a&gt; idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to &quot;look back” at the complete input and extracts significant information that is useful in decoding.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am really having trouble understanding the intuition behind the attention mechanism. I mean how the mechanism works and how to configure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simple words (and maybe with an example), what is the intuition behind the attention mechanism?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some applications, advantages &amp;amp; disadvantages of attention mechanism?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-23T13:49:58.680" LastActivityDate="2020-05-23T13:49:58.680" Title="What is the intuition behind the attention mechanism?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21390" PostTypeId="1" CreationDate="2020-05-22T03:31:42.333" Score="1" ViewCount="22" Body="&lt;p&gt;I have a bunch of text documents, split into source documents and transformed documents. These text documents have multiple lines and are edited at specific locations, in a specific way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I make use of the &lt;code&gt;difflib&lt;/code&gt; package available in Python to identify the associated transformation, for each source document and the resulting transformed document. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to train and implement a ML technique which will help in identifying and automating this conversion activity.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is a sample result of how the transformation result looks like:&#xA;(&lt;em&gt;NOTE: This example contains only one line, but my actual use case contains several lines&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import difflib&#xA;&#xA;Initial = 'This is my initial state'&#xA;Final = 'This is what I transform into'&#xA;&#xA;diff = difflib.SequenceMatcher(None, Initial, Final)&#xA;&#xA;for tag,i1,i2,j1,j2 in diff.get_opcodes():&#xA;    print('{:7} Initial[{:}:{:}] --&amp;gt; Final[{:}:{:}] {:} --&amp;gt; {:}'.format(tag,i1,i2,j1,j2,Initial[i1:i2],Final[j1:j2]))&#xA;&#xA;#Result:&#xA;equal   Initial[0:8] --&amp;gt; Final[0:8] This is  --&amp;gt; This is &#xA;insert  Initial[8:8] --&amp;gt; Final[8:23]  --&amp;gt; what I transfor&#xA;equal   Initial[8:9] --&amp;gt; Final[23:24] m --&amp;gt; m&#xA;delete  Initial[9:10] --&amp;gt; Final[24:24] y --&amp;gt; &#xA;equal   Initial[10:13] --&amp;gt; Final[24:27]  in --&amp;gt;  in&#xA;delete  Initial[13:14] --&amp;gt; Final[27:27] i --&amp;gt; &#xA;equal   Initial[14:15] --&amp;gt; Final[27:28] t --&amp;gt; t&#xA;replace Initial[15:24] --&amp;gt; Final[28:29] ial state --&amp;gt; o&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This helps in outlining the transformation steps to transform &lt;code&gt;Initial&lt;/code&gt; to &lt;code&gt;Final&lt;/code&gt;. I wish to make use of ML to identify the common pattern in such transformation between a large collection of txt documents and train a model that I can use in future.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What will be the best method to approach this problem? I am not facing a problem in identifying and classifying text data, but in identifying the nature of editing and transformation of strings.&lt;/p&gt;&#xA;" OwnerUserId="37276" LastActivityDate="2020-05-22T03:31:42.333" Title="Training a model for text document transformation?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;training&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21392" PostTypeId="1" CreationDate="2020-05-22T05:20:47.083" Score="2" ViewCount="78" Body="&lt;p&gt;Typically it seems like reinforcement learning involves learning over either a discrete or a continuous action space. An example might be choosing from a set of pre-defined game actions in Gym Retro or learning the right engine force to apply in Continuous Mountain Car; some popular approaches for these problems are deep Q-learning for the former and actor-critic methods for the latter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about in the case where a single action involves picking both a discrete and a continuous parameter? For example, when choosing the type (discrete), pixel grid location (discrete), and angular orientation (continuous) of a shape from a given set to place on a grid and optimize for some reward.&#xA;Is there a well-established approach for learning a policy to make both types of decisions at once?&lt;/p&gt;&#xA;" OwnerUserId="37277" LastEditorUserId="37277" LastEditDate="2020-05-22T05:30:26.007" LastActivityDate="2020-05-25T11:51:43.977" Title="Learning policy where action involves discrete and continuous parameters" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21394" PostTypeId="1" AcceptedAnswerId="21401" CreationDate="2020-05-22T07:42:27.793" Score="2" ViewCount="227" Body="&lt;p&gt;Why do we need convolutional neural networks instead of feed-forward neural networks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification problem, then why is the CNN needed?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-12-11T14:49:27.683" LastActivityDate="2020-12-12T12:30:12.933" Title="Why do we need convolutional neural networks instead of feed-forward neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21395" PostTypeId="1" AcceptedAnswerId="21397" CreationDate="2020-05-22T08:55:35.400" Score="1" ViewCount="47" Body="&lt;p&gt;I have seen two deep Q-learning formulas:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q\left(S_{t}, A_{t}\right) \leftarrow Q\left(S_{t}, A_{t}\right)+\alpha\left[R_{t+1}+\gamma \max _{a} Q\left(S_{t+1}, a\right)-Q\left(S_{t}, A_{t}\right)\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a)=r(s, a)+\gamma \max _{a} Q\left(s^{\prime}, a\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one is correct?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-22T10:42:52.537" LastActivityDate="2020-05-22T10:42:52.537" Title="Do we have two Q-learning update formulas?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21396" PostTypeId="2" ParentId="18682" CreationDate="2020-05-22T09:22:18.367" Score="2" Body="&lt;p&gt;First let us note the definition of the advantage function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A(s,a) = Q(s,a) - V(s) \; ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; is the action-value function and &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; is the state-value function. In theory you could represent these by two different function approximators, but this would be quite inefficient. However, note that &#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = \sum_{s',r} \mathbb{P}(s',r|s,a)(r + V(s') = \mathbb{E}[r + V(s')|a,s]\;,$$&lt;/span&gt;&#xA;so we can actually use a single function approximation, for &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, to completely represent the advantage function. To optimise this function approximator you would use the returns at each step of the episode as in e.g. the REINFORCE algorithm like you mentioned. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-22T09:22:18.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21397" PostTypeId="2" ParentId="21395" CreationDate="2020-05-22T09:52:07.450" Score="2" Body="&lt;p&gt;The first one is the update rule that we use in the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second one is the &quot;definition&quot; of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values, although I would personally write it as follows, with an expectation around the reward, to also support cases where rewards might be non-deterministic;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a) \doteq \mathbb{E} \left[ r(s, a) \right] + \gamma \max_a Q(s', a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the &lt;span class=&quot;math-container&quot;&gt;$Q(s', a)$&lt;/span&gt; itself would also be similarly defined, and again that one is also assumed to be a &quot;ground-truth&quot; value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, when learning, we do not actually know what any of these &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values exactly are; that's why we're doing learning in the first place, we're trying to learn what these values are! More precisely, when we wish to assign a new value to &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;, the definition tells us that we should use &lt;span class=&quot;math-container&quot;&gt;$Q(s', a)$&lt;/span&gt; for that, but we can't because we don't know exactly what the correct &lt;span class=&quot;math-container&quot;&gt;$Q(s', a$&lt;/span&gt;) value is either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We do generally have some &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1}, a)$&lt;/span&gt; values, but they're generally going to be only &lt;strong&gt;approximations&lt;/strong&gt;, resulting from the previous steps of our own learning process (or maybe even randomly initialised values if we only just started training!). There may also be an additional approximation error in stochastic environments, where the true &lt;span class=&quot;math-container&quot;&gt;$Q(s', a)$&lt;/span&gt; value may be a weighted average over multiple different possible successor states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, but during the training we only observed a single concrete successor state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, we can follow the definition and use &lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1}, a)$&lt;/span&gt; as an approximation of &lt;span class=&quot;math-container&quot;&gt;$Q(s', a)$&lt;/span&gt;, but we know that it's just an approximation and not entirely reliable. Therefore, instead of fully applying the definition and completely replacing our &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; value based on it, we &lt;strong&gt;interpolate&lt;/strong&gt; between our current estimate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; and the new one that we should have according to the definition + our approximations. We only slightly shift towards it. The learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; determines how much we shift towards it. Normally we use a learning rate &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \alpha &amp;lt; 1$&lt;/span&gt;. Note that if you were to use &lt;span class=&quot;math-container&quot;&gt;$\alpha = 1$&lt;/span&gt;, you would actually recover the definition for &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values again (except it would still use some observations to estimate unknown quantities).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-05-22T09:52:07.450" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21398" PostTypeId="1" CreationDate="2020-05-22T09:54:31.133" Score="2" ViewCount="134" Body="&lt;p&gt;The technique for off-policy value evaluation comes from importance sampling, which states that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E_{x \sim q}[f(x)] \approx \frac{1}{n}\sum_{i=1}^n f(x_i)\frac{q(x_i)}{p(x_i)},$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is sampled from &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the application of importance sampling to RL, is the expectation of the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; equivalent to the value of the trajectories, which is represented by the trajectories &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distributions &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; represent the probability of sampling trajectories from the behavior policy and the distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; represents the probability of sampling trajectories from the target policy &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would the trajectories from distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; be better than that of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;? I know from the equation how it is better, but it is hard to understand intuitively why this could be so.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-23T00:12:42.270" LastActivityDate="2020-11-26T05:33:05.963" Title="What is the intuition behind importance sampling for off-policy value evaluation?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21399" PostTypeId="2" ParentId="21398" CreationDate="2020-05-22T11:16:45.160" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In the application of importance sampling to RL, is the expectation of the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; equivalent to the value of the trajectories, which is represented by the trajectories &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe what you are asking here is if when using importance sampling in the off-policy RL setting that we set &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; from the general importance sampling formula to be our returns - the answer to this is yes. As always we are interested in calculating our expected returns. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How would the trajectories from the distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; be better than that&#xA;  of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;? I know from the equation how it is better but it is hard to&#xA;  understand intuitively why this could be so.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think here you got your &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; the wrong way around as we are using samples from &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; to approximate our policy &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. We typically will use importance sampling to generate samples from a &lt;em&gt;different&lt;/em&gt; policy to our target policy for a few reasons - one reason might be that our target policy is hard to sample from whereas sampling from our behaviour policy &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; might be relatively easy to sample from. Another reason is that we generally want to learn an optimal policy, but this could be difficult to learn if we don't explore enough. So we can follow some other policy that will explore sufficiently and still learn about our optimal target policy through the importance sampling ratio.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-05-23T00:16:38.650" LastActivityDate="2020-05-23T00:16:38.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21400" PostTypeId="1" CreationDate="2020-05-22T11:41:05.723" Score="1" ViewCount="314" Body="&lt;p&gt;I am new in reinforcement learning. I started reading the PyTorch's documentation about the cart pole control. Whenever an agent fails, they restart the environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run the code, the time in the game is the same as time in real life. Can we train models quicker? Can we make the game faster so that model will be training faster?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-22T23:23:31.523" LastActivityDate="2020-05-22T23:53:33.060" Title="Can we increase the speed of training a reinforcement learning algorithm?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21401" PostTypeId="2" ParentId="21394" CreationDate="2020-05-22T11:51:45.143" Score="5" Body="&lt;h2&gt;Why are CNNs useful?&lt;/h2&gt;&#xA;&lt;p&gt;The main property of CNNs that make them more suitable than FFNNs to solve tasks where the inputs are images is that they perform &lt;strong&gt;convolutions&lt;/strong&gt; (or cross-correlations).&lt;/p&gt;&#xA;&lt;h2&gt;Convolution&lt;/h2&gt;&#xA;&lt;p&gt;The convolution is an operation (more precisely, a linear operator) that takes two functions &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and produces another function &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;. It's often denoted as &lt;span class=&quot;math-container&quot;&gt;$f \circledast h = g$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt; represents the convolution operation and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the function that results from the convolution of the functions &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the case of CNNs,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a multi-dimensional array (aka tensor) and it represents an &lt;strong&gt;image&lt;/strong&gt; (or a processed version of an image, i.e. a feature map)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is a multi-dimensional array and it is called &lt;strong&gt;kernel&lt;/strong&gt; (aka filter), which represents the learnable parameters of the CNN, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is a processed version (with &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;) of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and it is often called the &lt;strong&gt;feature map&lt;/strong&gt;, so it's also a multi-dimensional array&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Images as functions&lt;/h3&gt;&#xA;&lt;p&gt;To be consistent with the initial definition of the convolution, &lt;span class=&quot;math-container&quot;&gt;$f, h$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; can indeed be represented as functions.&lt;/p&gt;&#xA;&lt;p&gt;Suppose that the input image is a greyscale (so it is initially represented as a matrix), then we can represent it as a function as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f: [a, b] \times [c, d] \rightarrow [0, 1],$$&lt;/span&gt;&#xA;i.e. given two numbers &lt;span class=&quot;math-container&quot;&gt;$x \in [a, b]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y \in [c, d]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; outputs a number in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$f(x, y) = z$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is the grayscale intensity of the pixel at coordinates &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. Similarly, the kernel &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; can also be defined as a function &lt;span class=&quot;math-container&quot;&gt;$h: [a, b] \times [c, d] \rightarrow [0, 1]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g: [a, b] \times [c, d] \rightarrow [0, 1]$&lt;/span&gt;, respectively.&lt;/p&gt;&#xA;&lt;p&gt;To be more concrete, if the shape of the image &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$28 \times 28$&lt;/span&gt;, then it is represented as the function &lt;span class=&quot;math-container&quot;&gt;$f: [0, 28] \times [0, 28] \rightarrow [0, 1]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note that the domain of the images doesn't have to range from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$28$&lt;/span&gt; and the codomain doesn't have to range from &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;. For example, in the case of RGB images, the codomain can also equivalently range from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$255$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;RGB images can also be represented as functions, more precisely, vector-valued functions, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x, y) = &#xA;\begin{bmatrix}&#xA;r(x, y) \\&#xA;g(x, y) \\&#xA;b(x, y)&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r: [a, b] \times [c, d] \rightarrow [0, 1]$&lt;/span&gt; represents the red channel,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g: [a, b] \times [c, d] \rightarrow [0, 1]$&lt;/span&gt; represents the green channel, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$b: [a, b] \times [c, d] \rightarrow [0, 1]$&lt;/span&gt; represents the blue channel&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Or, equivalently, &lt;span class=&quot;math-container&quot;&gt;$f: [a, b] \times [c, d] \times [0, 1]^3$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Why is the convolution useful?&lt;/h2&gt;&#xA;&lt;p&gt;The convolution of an image with kernels (e.g. the median kernel) can be used to perform many operations.&lt;/p&gt;&#xA;&lt;p&gt;For example, the convolution of a noisy image with the median filter can be used to remove noise from that image.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kMrmO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kMrmO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a screenshot of an image from &lt;a href=&quot;https://ch.mathworks.com/help/images/noise-removal.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, which you should read if you want to understand more about noise removal. So, on the left, there's the noisy image, and, on the right, there's the convolution of the median filter with the noisy image, which removes (at least, partially) the initial noise (i.e. those dots, which are due to the so-called &amp;quot;pepper and salt&amp;quot; noise).&lt;/p&gt;&#xA;&lt;p&gt;The convolution of any image with the Sobel filter can be used to compute the derivatives of that image (both in the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; directions, from which you can compute the magnitude and orientation of the gradient at each pixel of the image). See &lt;a href=&quot;https://ch.mathworks.com/help/images/ref/imgradient.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&lt;p&gt;So, in general, the convolution of an image with a kernel processes the image and the results (i.e. another image, which, in the case of CNNs, is called a &lt;em&gt;feature map&lt;/em&gt;) can be different depending on the kernel.&lt;/p&gt;&#xA;&lt;p&gt;This is the same thing as in CNNs. The only difference is that, in CNNs, the kernels are the learnable (or trainable) parameters, i.e. they change during training so that the overall loss (that the CNN is making) reduces (in the case CNNs are trained with gradient descent and back-propagation). For this reason, people like to say that CNNs are &lt;em&gt;feature extractors&lt;/em&gt; or are performing feature extraction (aka feature learning or representation learning).&lt;/p&gt;&#xA;&lt;p&gt;(Moreover, note that the convolution and cross-correlation are the same operations when the kernels are symmetric (e.g. in the case of a Gaussian kernel). In the case of CNNs, the distinction between convolution and cross-correlation doesn't make much sense because the kernels are learnable. You can ignore this if you are a beginner, but you can find more details &lt;a href=&quot;https://ai.stackexchange.com/q/21999/2444&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;&lt;h2&gt;Other useful properties&lt;/h2&gt;&#xA;&lt;p&gt;There are other useful properties of CNNs, most of them are just a consequence of the use of the convolution&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Translation invariance (or equivariance), i.e. they can &lt;em&gt;potentially&lt;/em&gt; find the same features (if you think of them as feature extractors) in multiple places of the image independently of their position, orientation, etc. See &lt;a href=&quot;https://stats.stackexchange.com/a/208949/82135&quot;&gt;this answer&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The equivalent FFNN has a lot more parameters (so CNNs may be less prone to overfitting)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They often use a sub-sampling operation  (known as &lt;em&gt;pooling&lt;/em&gt;) to further reduce the number of parameters (which can possibly help to avoid overfitting) and introduce non-linearity.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Notes&lt;/h2&gt;&#xA;&lt;p&gt;Note that the FFNN can also be used to process images. It's just that the CNN is more suited to deal with images for the reasons described above.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T12:30:12.933" LastActivityDate="2020-12-12T12:30:12.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21402" PostTypeId="2" ParentId="21200" CreationDate="2020-05-22T12:20:48.813" Score="1" Body="&lt;p&gt;While this may not be the answer you were looking for, I hope this explanation will help you to understand applying backpropagation to a CNN. Fundamentally, convolutional layers are no different than dense layers, however there are restrictions. The key one is weight-sharing which allows a CNN to be much more efficient than a regular dense layer (as well as it being sparse due to locality). Imagine we are transforming a 4x4 image into a 2x2 image. Since we are inputting a 16-vector, and outputting a 4-vector, we need a weights matrix of 4x16:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YHJad.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YHJad.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has 64 parameters. In a convolutional layer, we can accomplish this by convolving a 3x3 kernel over the image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ K=&#xA;\begin{bmatrix}&#xA;k_{1,1} &amp;amp; k_{1,2} &amp;amp; k_{1,3} \\&#xA;k_{2,1} &amp;amp; k_{2,2} &amp;amp; k_{2,3} \\&#xA;k_{3,1} &amp;amp; k_{3,2} &amp;amp; k_{3,3}&#xA;\end{bmatrix} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This convolution is equivalent to multiplying by the weights matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jyY0V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jyY0V.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, this only requires 9 parameters and backpropagation can be applied to update these parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image Source: &lt;a href=&quot;https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22373" LastActivityDate="2020-05-22T12:20:48.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21403" PostTypeId="2" ParentId="21400" CreationDate="2020-05-22T12:42:20.383" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can we make the game faster so that model will be training faster?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It depends on how much processing is required to run the simulation, how efficient that is implemented in whichever library you have loaded, and whether there is anything non-necessary for training that you can disable. Some environments for instance deliberately run &quot;real time&quot; so humans can appreciate the video output, and that is not necessary for training purposes (unless you want to experiment with real-time robotics).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For OpenAI Gym, there is one thing you can usually do: Switch off the rendering. The rendering for environments like CartPole slows down each time step considerably, and unless you are learning using computer vision processing on the pixels output, the agent does not need the pictures. You may even notice during training that moving the rendering window so it is not visible will speed up the training process considerably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did for CartPole, LunarLander and a couple of similar environments is turn off rendering for 99 out of 100 episodes, and render just one of them in 100 to help me monitor progress. For Q learning, I also picked that to be a &quot;test&quot; episode where I stopped exploration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option for speeding up training is to run a distributed system with multiple simulations at once. You will need mechanisms to share collected data too, so it is more work, but it is another approach to take if the simulation steps are the bottleneck for training speed.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-05-22T23:53:33.060" LastActivityDate="2020-05-22T23:53:33.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21404" PostTypeId="2" ParentId="21389" CreationDate="2020-05-22T12:50:23.747" Score="2" Body="&lt;p&gt;Simply put, the attention mechanism is loosely inspired on well, attention. Consider we are attempting machine translation on the following sentence: &quot;The dog is a Labrador.&quot; If you were to ask someone to pick out the key words of the sentence, i.e. which ones encode the most meaning, they would likely say &quot;dog&quot; and &quot;Labrador.&quot; Articles like &quot;the&quot; and &quot;a&quot; are not as relevant in translation as the previous words (though they aren't completely insignificant). Therefore, we focus our attention on the important words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attention seeks to mimic this by adding attention weights to a model as trainable parameters to augment important parts of our input. Consider an encoder-decoder architecture such as the one Google Translate uses. Our encoder recurrent neural network (RNN) encodes our input sentence as a context vector in some vector space, which is then passed along to the decoder RNN which translates it into the target language. The attention mechanism scores each word in the input (via dot product with attention weights), then passes these scores through the softmax function to create a distribution. This distribution is then multiplied with the context vector to produce an attention vector, which is then passed to the decoder. In the example in the first paragraph, our attention weights for &quot;dog&quot; and &quot;Labrador&quot; would hopefully become larger in comparison to those for the other words during training. Note that all parts of the input are still considered since a distribution must sum to 1, just some elements have more effect on the output than others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a diagram from Towards Data Science that illustrates this concept very nicely in terms of an encoder-decoder architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TCwv1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TCwv1.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantages of attention is its ability to identify the information in an input most pertinent to accomplishing a task, increasing performance especially in natural language processing - Google Translate is a bidirectional encoder-decoder RNN with attention mechanisms. The disadvantage is the increased computation. In humans, attention serves to reduce our workload by allowing us to ignore unimportant features, however in a neural network, attention entails overhead as we are now  generating attention distributions and training our attention weights (we are not actually ignoring the unimportant features, just diminishing their importance).&lt;/p&gt;&#xA;" OwnerUserId="22373" LastEditorUserId="22373" LastEditDate="2020-05-23T09:26:06.343" LastActivityDate="2020-05-23T09:26:06.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21405" PostTypeId="2" ParentId="21398" CreationDate="2020-05-22T12:50:53.870" Score="4" Body="&lt;p&gt;Recall that our goal is to be able to accurately estimate the true value of each state by computing a sample average over returns starting from that state:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_{q}(s) \doteq \mathbb{E}_{q}\left[G_{t} | S_{t}=s\right] \approx \frac{1}{n} \sum_{i=1}^{n} Return_i $$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Return_i$&lt;/span&gt; is the return obtained from the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the &lt;span class=&quot;math-container&quot;&gt;$\approx $&lt;/span&gt; does not hold, since in off-policy learning, we got those returns by following the behavior policy, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, and not the target policy, &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address that, we have to correct each return in the sample average by multiplying by the importance sampling ratio. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{q}(s) \doteq \mathbb{E}_{q}\left[G_{t} | S_{t}=s\right] \approx \frac{1}{n} \sum_{i=1}^{n} \rho_i Return_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the importance sampling ratio is : &lt;span class=&quot;math-container&quot;&gt;$\rho=\frac{\mathbb{P}(\text { trajectory under } q)}{\mathbb{P}(\text { trajectory under } p)}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this multiplication does is that it increases the importance of returns that were more likely to be seen under the target policy &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; and it decreases those that were less likely. So, at the end, in expectation, it would be as if the returns were averaged following &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A side note: To avoid the risks of mixing &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, it might be a good idea to denote/think of the &lt;strong&gt;b&lt;/strong&gt;ehavior policy as &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and the target policy as &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, following the convention in Sutton and Barto's RL book.)&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="2444" LastEditDate="2020-05-23T00:18:37.807" LastActivityDate="2020-05-23T00:18:37.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21407" PostTypeId="1" AcceptedAnswerId="21409" CreationDate="2020-05-22T12:58:12.750" Score="1" ViewCount="66" Body="&lt;p&gt;Vanilla policy gradient algorithm (using baseline to reduce variance) acc to &lt;a href=&quot;http://joschu.net/docs/thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (page 16)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Initialize policy parameter θ, baseline b&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for iteration=1, 2, . . . do&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;Collect a set of trajectories by executing the current policy&lt;/p&gt;&#xA;    &#xA;    &lt;p&gt;At each timestep in each trajectory, compute&lt;/p&gt;&#xA;    &#xA;    &lt;blockquote&gt;&#xA;      &lt;p&gt;the return &lt;span class=&quot;math-container&quot;&gt;$R_{t}= \sum_{t'=t}^{T-1}\gamma^{t'-t}r_{t'}$&lt;/span&gt;&lt;/p&gt;&#xA;      &#xA;      &lt;p&gt;the advantage estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{A}_{t} = R_{t} - b(s_{t})$&lt;/span&gt;&lt;/p&gt;&#xA;    &lt;/blockquote&gt;&#xA;    &#xA;    &lt;p&gt;Re-fit the baseline, by minimizing &lt;span class=&quot;math-container&quot;&gt;$\lVert b(s_{t}) - R_{t} \rVert^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;    &#xA;    &lt;blockquote&gt;&#xA;      &lt;p&gt;summed over all trajectories and timesteps.&lt;/p&gt;&#xA;    &lt;/blockquote&gt;&#xA;    &#xA;    &lt;p&gt;Update the policy, using a policy gradient estimate &lt;span class=&quot;math-container&quot;&gt;$\hat{g}$&lt;/span&gt;,&lt;/p&gt;&#xA;    &#xA;    &lt;blockquote&gt;&#xA;      &lt;p&gt;which is a sum of terms &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta}log\pi(a_{t}|s_{t},\theta)\hat{A_{t}}$&lt;/span&gt;&lt;/p&gt;&#xA;    &lt;/blockquote&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At line 6, advantage estimate is computed by subtracting baseline from the returns&lt;/li&gt;&#xA;&lt;li&gt;At line 7, baseline is re-fit minimizing mean squared error between state dependent baseline and return&lt;/li&gt;&#xA;&lt;li&gt;At line 8, we update the policy using advantage estimate from line &lt;strong&gt;6&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So is the baseline expected to be used in the next iteration when our policy has &lt;em&gt;changed&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute the advantage we subtract the state value &lt;span class=&quot;math-container&quot;&gt;$V(s_{t})$&lt;/span&gt; from the action value &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t},a_{t})$&lt;/span&gt;, under the &lt;em&gt;same&lt;/em&gt; policy, then why is the old baseline used here in advantage estimation?&lt;/p&gt;&#xA;" OwnerUserId="36861" LastActivityDate="2020-05-22T14:23:22.427" Title="In vanilla policy gradient is the baseline lagging behind the policy?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21408" PostTypeId="2" ParentId="21388" CreationDate="2020-05-22T13:10:44.963" Score="4" Body="&lt;p&gt;There are different TD algorithms, e.g. Q-learning and SARSA, whose convergence properties have been studied separately (in many cases). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some convergence proofs, e.g. in the paper &lt;a href=&quot;http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence of Q-learning: A Simple Proof&lt;/a&gt; (by Francisco S. Melo), the required conditions for Q-learning to converge (in probability) are the Robbins-Monro conditions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{t} \alpha_t(s, a) = \infty$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{t} \alpha_t^2(s, a) &amp;lt; \infty,$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha_t(s, a)$&lt;/span&gt; is the learning rate at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (that can depend on the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;), and that each state is visited infinitely often. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The Robbins-Monro conditions (1 and 2) are due to Herbert Robbins and Sutton Monro, who started the field of &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic approximation&lt;/a&gt; in the 1950s, with the paper &lt;a href=&quot;https://projecteuclid.org/download/pdf_1/euclid.aoms/1177729586&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Stochastic Approximation Method&lt;/a&gt;. The fields of RL and stochastic approximation are related. See &lt;a href=&quot;https://ai.stackexchange.com/a/20273/2444&quot;&gt;this answer&lt;/a&gt; for more details.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, note again that the specific required conditions for TD methods to converge may vary depending on the proof and the specific TD algorithm. For example, the Robbins-Monro conditions are not assumed in &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007%2FBF00115009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Predict by the Methods of Temporal Differences&lt;/a&gt; by Richard S. Sutton (because this is not a proof of &lt;a href=&quot;https://en.wikipedia.org/wiki/Convergence_of_random_variables&quot; rel=&quot;nofollow noreferrer&quot;&gt;convergence in probability but in expectation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, note that the proofs mentioned above are only applicable to the tabular versions of Q-learning. If you use function approximation, Q-learning (and other TD algorithms) may not converge. Nevertheless, there are cases when Q-learning combined with function approximation converges. See &lt;a href=&quot;https://gaips.inesc-id.pt/~fmelo/pub/melo08icml.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Analysis of Reinforcement Learning with Function Approximation&lt;/a&gt; by Francisco S. Melo et al. and &lt;a href=&quot;https://arxiv.org/pdf/1712.10285.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation&lt;/a&gt; by Bo Dai et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-23T18:41:37.073" LastActivityDate="2020-05-23T18:41:37.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21409" PostTypeId="2" ParentId="21407" CreationDate="2020-05-22T14:23:22.427" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So is the baseline expected to be used in the next iteration when our policy has &lt;em&gt;changed&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To compute the advantage we subtract the state value &lt;span class=&quot;math-container&quot;&gt;$V(s_{t})$&lt;/span&gt; from the action value &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t},a_{t})$&lt;/span&gt;, under the &lt;em&gt;same&lt;/em&gt; policy, then why is the old baseline used here in advantage estimation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The precise value of the baseline is not that important. What is important is that the baseline does not depend on the action choice, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, so it does not impact the gradient estimations or update steps for the policy function you are trying to improve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could in theory use a fixed offset instead of &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt;, or any arbitrary function that does not depend on &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. In some settings the average reward &lt;span class=&quot;math-container&quot;&gt;$\bar{R}$&lt;/span&gt; seen so far is used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a rough approximation to &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; - and thus an approximate advantage function overall is useful as it removes a large source of variance in gradient estimates (the inherent value of the current state under the current policy, which is irrelevant to search for &lt;em&gt;adjustments&lt;/em&gt; to that  policy). The more accurate &lt;span class=&quot;math-container&quot;&gt;$V(s)$&lt;/span&gt; is, the lower variance, thus faster and more reliable convergence, so you do want it to be a good estimate. But a little bit of lag behind policy updates is acceptable and does not break the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more on this, see &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto, chapter 13, section 13.4&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-22T14:23:22.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21411" PostTypeId="1" CreationDate="2020-05-22T17:17:16.697" Score="1" ViewCount="65" Body="&lt;p&gt;I wanted to implement the Policy Gradient on Tic-Tac-Toe. &#xA;I tried to use the code that worked for any environment like CartPole-v0 to my Tic-Tac-To game. But it is not learning. There are no errors. Just the result is so bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RandomPlayer (&quot;Player X&quot;) vs PolicyAgent (&quot;Player O&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ctsIa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ctsIa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one can see that the Policy Agent is not learning after 500 battles. Each battle consists of 100games against the random player. Together 500 * 100 games. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone tell me the problem or the bug in my code. I can not figure it out. Or what I have to improve. It would be so great. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is also a project which did the same, which I want to do, but with success. &#xA;&lt;a href=&quot;https://github.com/fcarsten/tic-tac-toe/blob/master/tic_tac_toe/DirectPolicyAgent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/fcarsten/tic-tac-toe/blob/master/tic_tac_toe/DirectPolicyAgent.py&lt;/a&gt;&#xA;I did not get what I am making different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Packages:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch as T&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;import torch.optim as optim&#xA;&#xA;import numpy as np&#xA;import gym&#xA;from gym import wrappers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Neural Net:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class PolicyNetwork(nn.Module):&#xA;    def __init__(self, lr, input_dims, fc1_dims, fc2_dims, n_actions):&#xA;        super(PolicyNetwork, self).__init__()&#xA;        self.input_dims = input_dims&#xA;        self.lr = lr&#xA;        self.fc1_dims = fc1_dims&#xA;        self.fc2_dims = fc2_dims&#xA;        self.n_actions = n_actions&#xA;&#xA;        self.fc1 = nn.Linear(self.input_dims, self.fc1_dims)&#xA;        self.fc2 = nn.Linear(self.fc1_dims, self.fc2_dims)&#xA;        self.fc3 = nn.Linear(self.fc2_dims, self.n_actions)&#xA;&#xA;        self.optimizer = optim.Adam(self.parameters(), lr=lr)&#xA;&#xA;    def forward(self, observation):&#xA;        state = T.Tensor(observation)&#xA;        x = F.relu(self.fc1(state))&#xA;        x = F.relu(self.fc2(x))&#xA;        x = self.fc3(x)&#xA;        return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Policy Agent:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class PolicyAgent:&#xA;    def __init__(self, player_name):&#xA;        self.name = player_name&#xA;        self.value = PLAYER[self.name]&#xA;&#xA;    def board_to_input(self, board):&#xA;        input_ = np.array([0] * 27)&#xA;        for i, val in enumerate(board):&#xA;            if val == self.value:&#xA;                input_[i] = 1  &#xA;            if val == self.value * -1:&#xA;                input_[i+9] = 1&#xA;            if val == 0:&#xA;                input_[i+18] = 1&#xA;        return np.reshape(input_, (1,-1))&#xA;&#xA;&#xA;    def start(self, learning_rate=0.001, gamma=0.1):&#xA;        self.lr = learning_rate&#xA;        self.gamma = gamma&#xA;        self.all_moves = list(range(0,9))&#xA;        self.policy = PolicyNetwork(self.lr, 27, 243, 91, 9)&#xA;        self.reward_memory = []&#xA;        self.action_memory = []&#xA;&#xA;    def turn(self, board, availableMoves):&#xA;        state = self.board_to_input(board.copy())&#xA;        prob = F.softmax(self.policy.forward(state))&#xA;        action_probs = torch.distributions.categorical.Categorical(prob)&#xA;        action = action_probs.sample()&#xA;&#xA;        while action.item() not in availableMoves:&#xA;            state = self.board_to_input(board.copy())&#xA;            prob = F.softmax(self.policy.forward(state))&#xA;            action_probs = torch.distributions.categorical.Categorical(prob)&#xA;            action = action_probs.sample()&#xA;&#xA;        log_probs = action_probs.log_prob(action)&#xA;        self.action_memory.append(log_probs)&#xA;&#xA;        self.reward_memory.append(0)&#xA;        return action.item()&#xA;&#xA;    def learn(self, result):&#xA;        if result == 0:&#xA;            reward = 0.5&#xA;        elif result == self.value:&#xA;            reward = 1.0&#xA;        else:&#xA;            reward = 0&#xA;&#xA;        self.reward_memory.append(reward)&#xA;        #print(self.reward_memory)&#xA;&#xA;        self.policy.optimizer.zero_grad()&#xA;        #G = np.zeros_like(self.action_memory, dtype=np.float64)&#xA;        G = np.zeros_like(self.reward_memory, dtype=np.float64)&#xA;&#xA;&#xA;        #running_add = reward&#xA;        #for t in reversed(range(0, len(self.action_memory))):&#xA;        #    G[t] = running_add&#xA;        #    running_add = running_add * self.gamma&#xA;&#xA;        #'''&#xA;        running_add = 0&#xA;        for t in reversed(range(0, len(self.reward_memory))):&#xA;            if self.reward_memory[t] != 0:&#xA;                running_add = 0&#xA;            running_add = running_add * self.gamma + self.reward_memory[t]&#xA;            G[t] = running_add&#xA;        for t in range(len(self.reward_memory)):&#xA;            G_sum = 0&#xA;            discount = 1&#xA;            for k in range(t, len(self.reward_memory)):&#xA;                G_sum += self.reward_memory[k] * discount&#xA;                discount *= self.gamma&#xA;            G[t] = G_sum&#xA;        mean = np.mean(G)&#xA;        std = np.std(G) if np.std(G) &amp;gt; 0 else 1&#xA;        G = (G-mean)/std&#xA;        #'''&#xA;&#xA;        G = T.tensor(G, dtype=T.float)&#xA;&#xA;        loss = 0&#xA;        for g, logprob in zip(G, self.action_memory):&#xA;            loss += -g * logprob&#xA;&#xA;        loss.backward()&#xA;        self.policy.optimizer.step()&#xA;&#xA;        self.reward_memory = []&#xA;        self.action_memory = []&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37287" LastActivityDate="2020-05-22T17:17:16.697" Title="Policy Gradient on Tic-Tac-Toe not working" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21412" PostTypeId="2" ParentId="20283" CreationDate="2020-05-22T19:12:16.743" Score="0" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; denotes the &lt;span class=&quot;math-container&quot;&gt;$Q-value$&lt;/span&gt; for the state-action pair. It means the expected returns if we start from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, and act according to whatever policy we are currently following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we are in state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, take action &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt;. To compute the returns, we would need to follow our current policy from whatever state we land up after taking &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt;, till the end of the episode, and sum up the rewards (or discounted rewards) that we get along the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why average of returns?&lt;/em&gt;&lt;br&gt;&#xA;Because we would want to do this multiple times for a state-action pair and compute the average of all such episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why multiple times?&lt;/em&gt;&lt;br&gt;&#xA;Generally, the environments and the transition function would have some randomness and we don't get the same reward every time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why would you want to compute this?&lt;/em&gt;&lt;br&gt;&#xA;The idea is simple. Since our goal is to maximize the average return, if we compute Q-values for all the possible actions starting from state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, then we can compare between the values and decide which action is going to be most beneficial to take from state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is a tabular approach, when they say update the Q-function, they just mean to update the Q-values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, suppose we are in state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; and can take actions &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;. We first compute the Q-values for &lt;span class=&quot;math-container&quot;&gt;$(s_0, a_0), (s_0,a_1), (s_0, a_2)$&lt;/span&gt; pairs, and then we would choose the action which has the maximum Q-value out of these three. &lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-05-22T19:12:16.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21413" PostTypeId="1" CreationDate="2020-05-22T19:20:39.537" Score="3" ViewCount="61" Body="&lt;p&gt;I thought about an algorithm that twists the standard Q-learning slightly, but I am not sure whether convergence to the optimal Q-value could be guaranteed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm starts with an initial policy. &#xA;Within each episode, the algorithm conducts policy evaluation and does NOT update the policy. &#xA;Once the episode is done, the policy is updated using the greedy policy based on the current learnt Q-values.&#xA;The process then repeats. I attached the algorithm as a picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to emphasize that the updating policy does not change within each episode. The policy at each state is updated AFTER one episode is done, using the Q-tables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone seen this kind of Q-learning before? If so, could you please kindly guide me to some resources regarding the convergence? Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/54ynY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/54ynY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37291" LastEditorUserId="37291" LastEditDate="2020-05-23T00:53:43.167" LastActivityDate="2020-05-23T00:53:43.167" Title="Convergence of a delayed policy update Q-learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21415" PostTypeId="1" AcceptedAnswerId="21419" CreationDate="2020-05-23T00:28:46.417" Score="2" ViewCount="74" Body="&lt;p&gt;I am running into an issue in which the the target (label collums) of my dataset contain a mixture of binary label (yes/no) and some numeric value label. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yqfys.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yqfys.png&quot; alt=&quot;d&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of these numeric value (resource 1 and resource 2 collumns) experience a large variation margin. Sometime these numeric value can be like 0.389 but sometimes they can be 0.389 x 10^-4 or something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to predict the binary decision and the amount of resource allocated to a new user who have input feature 1 (numeric) and input feature 2 (numeric).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial though would be that the output neuron corresponding to the 0-1 decision would use logistic regression activation function. But for the neuron that corresponding to the resource I am not quite sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the appropriate way to tackle such situation in term of network structure or data pre-processing  strategy ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your enthusiasm !&lt;/p&gt;&#xA;" OwnerUserId="37297" LastEditorUserId="37297" LastEditDate="2020-05-24T06:47:05.190" LastActivityDate="2020-05-26T03:48:42.540" Title="How to train a neural network with a data set that in which the target is a mix of 0-1 label and numeric real value label?" Tags="&lt;neural-networks&gt;&lt;data-preprocessing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21417" PostTypeId="2" ParentId="21415" CreationDate="2020-05-23T01:24:33.303" Score="0" Body="&lt;p&gt;In Neural Networks the function that provides the largest interval while activation is &lt;strong&gt;Tanh&lt;/strong&gt; with a result &lt;strong&gt;between -1 and 1&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use it to train your model , when the label has value of false it should be -1 , and when true it should be 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In prediction , you'll see where the value is more close , &lt;em&gt;for example if you get 0.4 more close to 1 so it'll be true&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="37243" LastActivityDate="2020-05-23T01:24:33.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21418" PostTypeId="1" AcceptedAnswerId="21445" CreationDate="2020-05-23T04:39:36.507" Score="2" ViewCount="248" Body="&lt;p&gt;I am working on scheduling problem that has inherent randomness. The dimensions of action and state spaces are 1 and 5 respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using &lt;a href=&quot;https://arxiv.org/pdf/1509.02971.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG&lt;/a&gt;, but it seems extremely unstable, and so far it isn't showing much learning. I've tried to &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;adjust the learning rate, &lt;/li&gt;&#xA;&lt;li&gt;clip the gradients, &lt;/li&gt;&#xA;&lt;li&gt;change the size of the replay buffer, &lt;/li&gt;&#xA;&lt;li&gt;different neural net architectures, using SGD and Adam, &lt;/li&gt;&#xA;&lt;li&gt;change the &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; for the soft-update. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, I'd like to know what people's experience is with this algorithm, for the environments where it was tested on the paper but also for other environments. What values of hyperparameters worked for you? Or what did you do? How cumbersome was the fine-tuning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think my implementation is incorrect, because I pretty much replicated &lt;a href=&quot;https://github.com/nicehiro/ddpg/blob/master/ddpg_pytorch/agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, and every other implementation I found did exactly the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also, I am not sure this is necessarily the best website to post this kind of question, but I decided to give a shot.)&lt;/p&gt;&#xA;" OwnerUserId="36341" LastEditorUserId="2444" LastEditDate="2020-05-24T10:36:27.433" LastActivityDate="2020-05-26T13:35:12.200" Title="What made your DDPG implementation on your environment work?" Tags="&lt;reinforcement-learning&gt;&lt;implementation&gt;&lt;ddpg&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21419" PostTypeId="2" ParentId="21415" CreationDate="2020-05-23T05:10:54.433" Score="1" Body="&lt;p&gt;Your question are missing some details and i will assume some scenarios.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have a classification problem: you can try group the values in intervals that make sense (you should analyze and decide for this setup), if its possible. For example: 0.000-0.250 (0), 0.251-0.500 (1), 0.501-0.750 (2) and so on. Note that neural networks are sensible for distance between values (1 is closer to 0 than 2, so 1 is more similar to 0 than 2 and so on). If that is not your case, you should binarize the values in One Hot Encode manner.&lt;/li&gt;&#xA;&lt;li&gt;If you have a regression problem, you should be ok without anything else. You can try normalize your outputs and observe the results, but generally it's not necessary for regression problems.&lt;/li&gt;&#xA;&lt;li&gt;Be sure if your dataset are free of outliers and noisy data as much as possible. &lt;/li&gt;&#xA;&lt;li&gt;It's important choose activations functions that are adequate for the range of values in your attributes and output. This can depend on how do you treat and setup your dataset, the range of values, normalization etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update after more details in question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your neural network should have 3 neurons in the output layer, with linear activation. As said before, normalization usually is not necessary in regression problems, but if your values are too diferent (like the range in resource 1 and resource 2) maybe some kind of adjustment (normalization, standardization etc) can be helpful. But you need try and see the results. &lt;/p&gt;&#xA;" OwnerUserId="37300" LastEditorUserId="37300" LastEditDate="2020-05-26T03:48:42.540" LastActivityDate="2020-05-26T03:48:42.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21422" PostTypeId="1" CreationDate="2020-05-23T06:00:48.420" Score="3" ViewCount="45" Body="&lt;p&gt;After reading a lot of articles (for instance, this one &lt;a href=&quot;https://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding LSTM Networks&lt;/a&gt;), I know that the &lt;a href=&quot;https://arxiv.org/abs/1503.04069&quot; rel=&quot;nofollow noreferrer&quot;&gt;long short-term memory (LSTM)&lt;/a&gt; is an artificial recurrent neural network (RNN) architecture used in the field of deep learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does backpropagation work in the specific case of LSTMs?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-23T12:40:28.777" LastActivityDate="2020-05-23T12:40:28.777" Title="How does backpropagation work in LSTMs?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21424" PostTypeId="1" CreationDate="2020-05-23T07:29:04.767" Score="1" ViewCount="40" Body="&lt;p&gt;I'm using Python and tensorflow to implement a Deep Q-learning with experience replay in a continous action and state spaces and I have used two neural networks to approximate both the policy function and the Q-function. While for the Q-function I have a target so I can update the model minimizing the loss function, the policy update pseudocode does not include a target which I can use to fit the model but only a gradient ascent step, as the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DBDd2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DBDd2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use tensorflow and a sequential model in python to make this update?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="37169" LastEditDate="2020-05-23T10:10:21.983" LastActivityDate="2020-05-23T10:10:21.983" Title="How can I perform policy update in python?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;policy-gradients&gt;&lt;off-policy-methods&gt;" AnswerCount="0" CommentCount="4" ClosedDate="2020-05-23T10:45:29.963" ContentLicense="CC BY-SA 4.0" />
  <row Id="21425" PostTypeId="1" AcceptedAnswerId="21426" CreationDate="2020-05-23T09:28:06.870" Score="2" ViewCount="114" Body="&lt;p&gt;In a neural network, a neuron typically computes a linear function &lt;span class=&quot;math-container&quot;&gt;$f(x) = w*x$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the weight and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why not replacing the linear function with more complex functions, such as &lt;span class=&quot;math-container&quot;&gt;$f(x,w,a,b,c) = w*(x + b)^a + c$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will provide much more diversity into neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this have a name? Has this been used?&lt;/p&gt;&#xA;" OwnerUserId="36420" LastEditorUserId="2444" LastEditDate="2020-05-23T11:06:30.260" LastActivityDate="2020-05-23T11:06:30.260" Title="Why not replacing the simple linear functions that neurons compute with more complex functions?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21426" PostTypeId="2" ParentId="21425" CreationDate="2020-05-23T10:01:56.240" Score="4" Body="&lt;p&gt;It is definitley possible to make the links between neurons use more complex functions. Provided those functions are differentiable, backpropagation still works, and the resulting compound function might be able to learn something useful. The general name for such a thing is a &lt;a href=&quot;https://www.deepideas.net/deep-learning-from-scratch-i-computational-graphs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational graph&lt;/a&gt; and the standardised structures used in most neural networks are a subset of all possible (and maybe useful) computational graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When adding complex and non-linear functions into a neural network, this is usually alternated with simpler linear layers using the weights. A generalised function of a single neuron as used in most neural networks looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a = f(\sum_i w_i x_i + b)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; indexes all inputs to the neuron, &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; are the input values, &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; the weights associated with each input, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a bias term and &lt;span class=&quot;math-container&quot;&gt;$f()$&lt;/span&gt; is a differentiable non-linear &lt;em&gt;activation function&lt;/em&gt;. Training process learns &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. The output &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the the neuron's activation value, that may be taken as output of the neural network, or fed in to some other neuron as one of the next neuron's &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple feed-forward network using this basic neuron function, with at least one hidden layer which has a non-linear activation function can already learn approximations to any given function - a result proved in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The practical result of the universal approximation theorem is that you need motivation other than increasing diversity in order to make neural network functions more complex. If you were considering altering one of the &lt;span class=&quot;math-container&quot;&gt;$w_i x_i$&lt;/span&gt; multiplications, and replacing with a more complex learnable function, you can effectively achieve the same thing by adding another neuron whose output &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is used as &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; - or simply adding a layer in most neural network libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some situations there may be good reasons to make lower-level changes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you know the function you are learning relates to a theoretical model with a specific mathematical form, you can deliberately set up functions that mirror that with learnable parameters. Typcially that is done as transforms on inputs, but could also be part of a more complex computational graph if necessary.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In neural network architectures, you can consider things such as gate combinations in &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM cells&lt;/a&gt;, or &lt;a href=&quot;https://en.wikipedia.org/wiki/Residual_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;skip connections in residual networks&lt;/a&gt; as examples where the functions have deliberately be made more complex to achieve a specific goal - in both those cases in order to increase effectiveness of backpropagation in deep structures.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-23T10:01:56.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21427" PostTypeId="1" AcceptedAnswerId="21428" CreationDate="2020-05-23T14:39:13.500" Score="0" ViewCount="103" Body="&lt;p&gt;AIXI is a mathematical framework for artificial &lt;strong&gt;general&lt;/strong&gt; intelligence developed by Marcus Hutter since the year 2000. It's based on many concepts, such as reinforcement learning, Bayesian statistics, Occam's razor, or Solomonoff induction. The blog post &lt;a href=&quot;https://jan.leike.name/AIXI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is AIXI? — An Introduction to General Reinforcement Learning&lt;/a&gt; provides an accessible overview of the topic for those of you not familiar with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other &lt;strong&gt;mathematical&lt;/strong&gt; frameworks of artificial &lt;strong&gt;general&lt;/strong&gt; intelligence apart from AIXI? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of projects such as &lt;a href=&quot;https://wiki.opencog.org/w/The_Open_Cognition_Project&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCog&lt;/a&gt;, but that's not really a mathematical framework, but more a cognitive science framework.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T15:29:38.873" LastActivityDate="2021-01-17T16:04:37.560" Title="Are there other mathematical frameworks of artificial general intelligence apart from AIXI?" Tags="&lt;reference-request&gt;&lt;agi&gt;&lt;aixi&gt;&lt;godel-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21428" PostTypeId="2" ParentId="21427" CreationDate="2020-05-23T14:39:13.500" Score="1" Body="&lt;h3&gt;&lt;a href=&quot;https://arxiv.org/pdf/cs/0309048.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gödel machine&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;There is another mathematical framework for AGI: &lt;strong&gt;the Gödel machine&lt;/strong&gt;, which was proposed by Jürgen Schmidhuber (who also worked with Marcus Hutter). In the paper &lt;a href=&quot;https://arxiv.org/pdf/cs/0309048.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements&lt;/a&gt; (2003), Schmidhuber describes the Gödel machine as follows&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;They are universal problem solving systems that interact with some (partially observable) environment and can in principle modify themselves without essential limits apart from the limits of computability. Their initial algorithm is not hardwired; it can completely rewrite itself, but only if a proof searcher embedded within the initial algorithm can first prove that the rewrite is useful, given a formalized utility function reflecting computation time and expected future success (e.g., rewards). We will see that self-rewrites due to this approach are actually globally optimal (Theorem 4.1, Section 4), relative to Gödel's well-known fundamental restrictions of provability. These restrictions should not worry us; if there is no proof of some self-rewrite's utility, then humans cannot do much either.&lt;/p&gt;&#xA;&lt;p&gt;The initial proof searcher is &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt;()-optimal (has an optimal order of complexity) in the sense of Theorem 5.1, Section 5. Unlike hardwired systems such as Hutter's and Levin's (Section 6.4), however, a Gödel machine can in principle speed up any part of its initial software, including its proof searcher, to meet arbitrary formalizable notions of optimality beyond those expressible in the &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt;()-notation. Our approach yields the first theoretically sound, fully self-referential, optimal, general problem solvers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;&lt;a href=&quot;https://hal.inrae.fr/hal-02745675/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;Space-time embedded agents&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;This work combines &lt;a href=&quot;https://www.jair.org/index.php/jair/article/view/10134&quot; rel=&quot;nofollow noreferrer&quot;&gt;Russell's &lt;strong&gt;bounded rationality&lt;/strong&gt;&lt;/a&gt; with &lt;a href=&quot;https://doc.rero.ch/record/9503/files/2008INFO002.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Legg's and Hutter's definition of &lt;strong&gt;universal intelligence&lt;/strong&gt;&lt;/a&gt; to provide a framework for AGI that takes into account resource (space and time) constraints and other issues (such as that the environment can modify or even destroy the agent).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T16:04:37.560" LastActivityDate="2021-01-17T16:04:37.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21430" PostTypeId="1" AcceptedAnswerId="21431" CreationDate="2020-05-23T16:03:22.730" Score="1" ViewCount="98" Body="&lt;p&gt;Similarly to &lt;a href=&quot;https://ai.stackexchange.com/q/20411/2444&quot;&gt;What are the scientific journals dedicated to artificial general intelligence?&lt;/a&gt;, are there any conferences dedicated to artificial &lt;strong&gt;general&lt;/strong&gt; intelligence?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-13T12:19:27.013" Title="Are there any conferences dedicated to artificial general intelligence?" Tags="&lt;agi&gt;&lt;research&gt;&lt;academia&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21431" PostTypeId="2" ParentId="21430" CreationDate="2020-05-23T16:03:22.730" Score="2" Body="&lt;p&gt;There are several conferences dedicated to AGI or human-level intelligence, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://agi-conf.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI Conferences&lt;/a&gt; (organized by &lt;a href=&quot;http://www.agi-society.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI society&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bicasociety.org/meetings/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biologically Inspired Cognitive Architectures&lt;/a&gt; (organized by &lt;a href=&quot;https://bicasociety.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BICA Society&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cogsys.org/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advances in Cognitive Systems&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mini.pw.edu.pl/%7Emandziuk/cis_tf_thli/?page=activities&quot; rel=&quot;nofollow noreferrer&quot;&gt;IEEE Task Force on Towards Human-like Intelligence&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The conferences focus on topics such as cognitive architectures, autonomy, creativity, lifelong learning, and formal models of general intelligence. There are also journals associated with these conferences (see [&lt;a href=&quot;https://content.sciendo.com/view/journals/jagi/jagi-overview.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], [&lt;a href=&quot;https://www.sciencedirect.com/journal/biologically-inspired-cognitive-architectures&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;], [&lt;a href=&quot;http://www.cogsys.org/journal&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;]). If you want to know more about them, I suggest that you go to their websites.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-13T12:19:27.013" LastActivityDate="2020-07-13T12:19:27.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21432" PostTypeId="1" CreationDate="2020-05-23T17:57:59.923" Score="1" ViewCount="57" Body="&lt;p&gt;What is the relation between multi-agent learning and reinforcement learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is one a sub-field of the other? For instance, would it make sense to state that your research interest are multi-agent learning and reinforcement learning, or would that be weird as one includes most of the topics of the other?&lt;/p&gt;&#xA;" OwnerUserId="36116" LastEditorUserId="2444" LastEditDate="2020-05-23T18:17:33.717" LastActivityDate="2020-05-23T19:14:00.893" Title="What is the relation between multi-agent learning and reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21433" PostTypeId="1" CreationDate="2020-05-23T19:03:00.510" Score="2" ViewCount="202" Body="&lt;p&gt;I'm reading an article on reinforcement learning, and I don't understand why the agent's policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is not part of definition of Markov Decision process(MDP):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/giJ04.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/giJ04.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Bu, Lucian, Robert Babu, and Bart De Schutter. &amp;quot;A comprehensive survey of multiagent reinforcement learning.&amp;quot; IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38.2 (2008): 156-172.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why the policy is not a part of the MDP definition?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36175" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-24T10:24:13.213" Title="Why is the policy not a part of the MDP definition?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;markov-decision-process&gt;&lt;policies&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21434" PostTypeId="2" ParentId="21432" CreationDate="2020-05-23T19:14:00.893" Score="2" Body="&lt;p&gt;I think there is an intersection. There are problems that are in reinforcement learning and in learning in multi-agent systems. There are problems in reinforcement learning, but not exactly in multi-agent systems. And there is learning in multi-agent systems that is not through reinforcement learning. For sort you can say: multi-agent reinforcement learning. I recommend take a look at these references:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bu, Lucian, Robert Babu, and Bart De Schutter. &amp;quot;A comprehensive survey of multiagent reinforcement learning.&amp;quot; IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38.2 (2008): 156-172.&lt;/p&gt;&#xA;&lt;p&gt;Busoniu, Lucian, Robert Babuska, and Bart De Schutter. &amp;quot;Multi-agent reinforcement learning: A survey.&amp;quot; 2006 9th International Conference on Control, Automation, Robotics and Vision. IEEE, 2006.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36175" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-23T19:14:00.893" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21435" PostTypeId="2" ParentId="18567" CreationDate="2020-05-23T19:42:33.803" Score="0" Body="&lt;p&gt;Could you figure out a workaround for this problem of the steady-state error using DDPG?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently facing the same problem. My application is Satellite attitude control, and no matter which cost function variation I use, the resulting controlled system maintains a constant steady-state error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to be a limitation of the algorithm. But I can't understand why this happens. Hence, I've decided to use PPO for now. But would like to know if you could solve this problem using DDPG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the best,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wilson&lt;/p&gt;&#xA;" OwnerUserId="37316" LastActivityDate="2020-05-23T19:42:33.803" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21436" PostTypeId="2" ParentId="3889" CreationDate="2020-05-23T20:05:18.283" Score="1" Body="&lt;p&gt;In the context of reinforcement learning, the idea of modeling your goal-oriented problem as a hierarchy of multiple sub-problems is called &lt;strong&gt;hierarchical reinforcement learning&lt;/strong&gt;, which gives rise to concepts such as &lt;em&gt;semi-Markov decision processes&lt;/em&gt; and &lt;em&gt;options&lt;/em&gt; (aka macro actions). The article &lt;a href=&quot;https://thegradient.pub/the-promise-of-hierarchical-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Promise of Hierarchical Reinforcement Learning&lt;/a&gt; presents and describes the topic quite well, so I suggest you read it.&lt;/p&gt;&#xA;&lt;p&gt;However, this idea of solving multiple sub-tasks in order to solve a bigger task isn't limited to RL. For example, in the paper &lt;a href=&quot;https://arxiv.org/pdf/1511.06279.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Programmer-Interpreters&lt;/a&gt; (NPIs), without referring to the traditional RL topics, a model is proposed to write programs by composing simpler ones.&lt;/p&gt;&#xA;&lt;p&gt;Another example is genetic programming with &lt;em&gt;automatically defined functions&lt;/em&gt; (ADFs), where sub-programs (the ADFs) can be re-used in different parts of the program if that's beneficial according to the fitness.&lt;/p&gt;&#xA;&lt;p&gt;So, there are different ways you can design a system to solve a big task by solving multiple sub-tasks and then compose them. The approach that you choose depends on your use case. If you want to build programs, then NPIs can be a start. If you want to incorporate a time component in your system, then HRL is probably a viable approach. There are several HRL algorithms. Some of them (e.g. &lt;a href=&quot;https://github.com/wrighteagle2d/wrighteaglebase&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAXQ-OP&lt;/a&gt;) have been successfully used to solve the &lt;a href=&quot;https://www.robocup.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RoboCup challenge&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-17T22:34:01.487" LastActivityDate="2021-02-17T22:34:01.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21437" PostTypeId="2" ParentId="21433" CreationDate="2020-05-23T20:24:16.827" Score="7" Body="&lt;p&gt;The MDP defines the environment (which corresponds to the task that you need to solve), so it defines e.g. the states of the environment, the actions that you &lt;em&gt;can&lt;/em&gt; take in those states, the probabilities of transitioning from one state to the other and the probabilities of getting a reward when you take a certain action in a certain state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy corresponds to a strategy that the RL agent can follow to act in that environment. Note that the MDP doesn't define what the agent does in each state. That's why you need the policy! An optimal policy for a specific MDP corresponds to the strategy that, if followed, is guaranteed to give you the highest amount of reward in that environment. However, there are multiple strategies, most of them are not optimal. This should clarify why the policy is not part of the definition of the MDP.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-23T20:24:16.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21438" PostTypeId="1" CreationDate="2020-05-23T23:20:21.940" Score="2" ViewCount="28" Body="&lt;p&gt;I'm trying to detect an object in a video (with slight camera movement), and then augment another video on top of it. What is the simplest approach to do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, let's assume I have this simple video of a couch &lt;a href=&quot;https://streamable.com/dzh0at&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;HERE&lt;/strong&gt;&lt;/a&gt;. Now I want to augment the &lt;strong&gt;right cusion&lt;/strong&gt; with a human or a dog. The dog or human are video itself (let's assume transparency is not an issue).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the simplest approach to do that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T3juZl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T3juZl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2020-05-24T05:20:12.200" LastActivityDate="2020-05-24T05:20:12.200" Title="Detect object in video and augment another video on top of it" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;ai-basics&gt;&lt;object-detection&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21439" PostTypeId="1" AcceptedAnswerId="21446" CreationDate="2020-05-24T02:03:49.680" Score="3" ViewCount="61" Body="&lt;p&gt;There are many factors that cause the results of ML models to be different for every run of the same piece of code. One factor could be different initialization of weights in the neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since results might be stochastic, how would researchers know what their best performing model is? I know that a seed can be set to incorporate more determinism into the training. However, there could be other pseudo-random sequences that produce slightly better results?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-24T10:23:39.123" LastActivityDate="2020-05-24T12:31:15.113" Title="How would researchers determine the best deep learning model if every run of the code yields different results?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21440" PostTypeId="1" CreationDate="2020-05-24T06:54:57.207" Score="2" ViewCount="74" Body="&lt;p&gt;I read Q-learning algorithm and also I know value iteration (when you update action values).  I think the PyTorch example is value iteration rather than Q-learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link:&#xA;&lt;a href=&quot;https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="1847" LastEditDate="2020-05-24T10:19:48.260" LastActivityDate="2020-05-24T18:19:24.283" Title="Is the PyTorch official tutorial really about Q-learning?" Tags="&lt;q-learning&gt;&lt;value-iteration&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21441" PostTypeId="2" ParentId="21440" CreationDate="2020-05-24T08:50:26.437" Score="4" Body="&lt;p&gt;&lt;em&gt;TL;DR: It is Q learning. However Q learning is basically sample-based value iteration, so not surprising you see a similarity.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q learning* and value iteration are very strongly related. When considering action values, both approaches use the same Bellman equation for optimal policy, &lt;span class=&quot;math-container&quot;&gt;$q^*(s,a) = \sum_{r,s'}p(r,s'|s,a)(r+\gamma \text{max}_{a'} q^*(s', a'))$&lt;/span&gt; as the basis for update steps. The differences are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Value iteration makes updates using a model of the environment, Q learning works from samples from the environment made by an active agent.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By working from a simulated environment rather than a real one, it may not be clear when an agent is model-free or model-based (or planning rather than acting). However, the way that the simulated environment in the PyTorch example is used is consistent with a model-free method.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value iteration loops through all possible states and actions for updates independently of any action an agent might take (in fact the agent need not exist). Q learning works with whichever states the agent experienced.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By adding experience replay memory in DQN, Q learning becomes a little bit closer to value iteration, as you can frame the memory as a learned model, plus consider it to be a type of planning (or a &quot;sweep&quot; through states). This is how it is described for instance in DynaQ which is an almost identical algorithm to experience replay as used in DQN when both are used in the simplest versions - see Sutton &amp;amp; Barto chapter 8.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value iteration value update steps are over an &lt;em&gt;expectation&lt;/em&gt; of next states and rewards - it processes the weighted sum &lt;span class=&quot;math-container&quot;&gt;$\sum_{r,s'}p(r,s'|s,a)$&lt;/span&gt;. Q learning update steps are over &lt;em&gt;sampled&lt;/em&gt; next states and rewards - it ends up approximating the same expectation over many separate updates.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Even using large amounts of experience replay memory does not get Q learning the same as value iteration on this issue, samples are not guaranteed perfect. However, in a deterministic environment, this difference is not meaningful. So if you have a deterministic environment, Q learning and value iteration may also be considered a little closer in nature.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;* Technically this applies to single-step Q-learning. n-step Q-learning and Q(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;) use different estimates of future expected return, that are related but not the same as the single-step version shown here.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-24T18:19:24.283" LastActivityDate="2020-05-24T18:19:24.283" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21444" PostTypeId="2" ParentId="21433" CreationDate="2020-05-24T10:24:13.213" Score="2" Body="&lt;p&gt;Aside from the points raised in &lt;a href=&quot;https://ai.stackexchange.com/a/21437/1641&quot;&gt;nbro's answer&lt;/a&gt;, I'd like to point out that for a single MDP (a single instance of a &quot;problem&quot;), it may be sensible to study it from perspectives that include no policy at all, or multiple different policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if I have an MDP, I may be interested in studying it by looking at various inherent properties of the environment. And if I then have multiple different MDPs, all without any policies or anything like that, I could compare them based on those properties. For example, I might simply want to measure the sizes of the state and action spaces. Or write out something like a game tree, and measure properties like the branching factor and the average / min / max / median depth at which we can find a terminal state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, it can also be interesting sometimes to study multiple different policies all for the same MDP. A very common example would be any off-policy learning algorithm (like &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning): they all involve at least one &quot;target policy&quot; (for which they're learning the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values -- usually the greedy policy with respect to the values learned so far), and at least one &quot;behaviour policy&quot; (which they're using to generate experience -- often something like an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy). A more complex example would be population-based training setups, like the one &lt;a href=&quot;https://doi.org/10.1038/s41586-019-1724-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind used for their StarCraft 2 training&lt;/a&gt;; here they have a large population of different policies that they're all using in a complex training setup (and technically I suppose we should say they also have many different MDPs, where every combination of StarCraft 2 level + training opponent would formally be a different MDP).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-05-24T10:24:13.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21445" PostTypeId="2" ParentId="21418" CreationDate="2020-05-24T10:29:26.550" Score="1" Body="&lt;p&gt;Below are some tweaks that helped me accelerate the training of DDPG on a &lt;a href=&quot;https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#reacher&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reacher&lt;/a&gt;-like environment:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reducing the neural network size, compared to the original paper. Instead of: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2 hidden layers with 400 and 300 units respectively &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I used 128 units for both hidden layers. I see in your implementation that you used 256, maybe you could try reducing this. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As suggested in the paper, I added batch normalization:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... to manually scale the features so they are in similar ranges across&#xA;  environments and units. We address this issue by adapting a recent technique from deep learning&#xA;  called batch normalization (Ioffe &amp;amp; Szegedy, 2015). This technique normalizes each dimension&#xA;  across the samples in a minibatch to have unit mean and variance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The implementation you used does not seem to include this.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reducing the value of &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;, which is a parameter of the Ornstein-Uhlenbeck process used to enable exploration. Originally, it was &lt;span class=&quot;math-container&quot;&gt;$0.2$&lt;/span&gt;, I used &lt;span class=&quot;math-container&quot;&gt;$0.05$&lt;/span&gt;. (I can't find where this parameter is set in your code.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am not entirely sure if this will help in your environment, but it was just to give you some ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: &lt;a href=&quot;https://github.com/udacity/deep-reinforcement-learning/tree/master/ddpg-pendulum&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a link to the code I followed to build DDPG and &lt;a href=&quot;https://pasteboard.co/JaaaXuf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is a plot of rewards per episode.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="34010" LastEditDate="2020-05-26T13:35:12.200" LastActivityDate="2020-05-26T13:35:12.200" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21446" PostTypeId="2" ParentId="21439" CreationDate="2020-05-24T12:31:15.113" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I know that a seed can be set to incorporate more determinism into the training. However, there could be other pseudo-random sequences that produce slightly better results?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is correct. If you fix the seed for a process which inherently has stochastic behaviour by design (such as initialising neural network params), then what you know about the model is that it is the best one given the hyperparameters you have selected and &lt;em&gt;that specific seed&lt;/em&gt;. Sometimes the value of the seed is highly relevant, other times less so.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since results might be stochastic, how would researchers know what their best performing model is? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, as with any experiment where measurements are variable, by running the experiment multiple times and taking statistics over the set of results. This will give you a much better sense of how the algorithm does in general, independently of specific seeds. You can still fix your RNG seeds for repeatability, but you will need multiple sets of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For certain goals, such as making the best possible model that you can, independently of whether the approach you take is &quot;best in general&quot;, this is not necessary. A single run which creates a state-of-the-art performance is still of interest, for instance. Or if you are creating a model that you want to use in production, you may care less about the stability of the technique (and being &quot;lucky&quot;) than being in possession of a high performing agent for the task.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-24T12:31:15.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21448" PostTypeId="1" CreationDate="2020-05-24T12:37:55.473" Score="1" ViewCount="32" Body="&lt;p&gt;I'm looking for someone who can help me clarify a few details regarding the architecture of Bert model. Those details are necessary for me to come with a full understanding of Bert model, so your help would be really helpful. Here are the questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the self-attention layer of Bert model have parameters? Do the embeddings of words change ONLY according to the actual embeddings of other words when the sentence is passed through the self-attention layer?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are the parameters of the embedding layer of the model (the layer which transforms the sequence of indexes passed as input into a sequence of embeddings of size=size of the model) trainable or not?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37328" LastEditorUserId="30725" LastEditDate="2020-06-01T12:22:48.410" LastActivityDate="2020-06-01T12:22:48.410" Title="Two questions about the architecture of Google Bert model (in particular about parameters)" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21449" PostTypeId="1" CreationDate="2020-05-24T20:42:13.990" Score="1" ViewCount="104" Body="&lt;p&gt;I am using Q-learning to solve an engineering problem. The objective is to generate a Q-table associating state to Q-values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created a State vector &lt;code&gt;DS = [s1, s2, ..., sN]&lt;/code&gt; containing all &quot;desired&quot; states. So the Q-table has the form of &lt;code&gt;Q-table =[DS, Q-values]&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, my agent follows a trajectory. Playing action &lt;code&gt;a&lt;/code&gt; at state &lt;code&gt;s&lt;/code&gt; (which is a point of the trajectory) leads the agent to another state &lt;code&gt;s'&lt;/code&gt; (another point of the trajectory). However, I don't have the &lt;code&gt;s'&lt;/code&gt; state in the initially desired states vector &lt;code&gt;DS&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution is to add new states to the &lt;code&gt;DS&lt;/code&gt; vector while the Q-learning algorithm is running, but I do not want to add new states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other ideas on how to handle this problem? &lt;/p&gt;&#xA;" OwnerUserId="37337" LastEditorUserId="30725" LastEditDate="2020-05-31T13:38:06.240" LastActivityDate="2020-05-31T13:38:06.240" Title="Handle non-existing states in q-learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21451" PostTypeId="2" ParentId="8509" CreationDate="2020-05-25T04:50:46.797" Score="1" Body="&lt;p&gt;From what I understand, don't bother with a CNN, you have essentially perfectly structured images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can hand code detectors to measure how much filled in a circle is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically do template alignment and then search over the circles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex a simple detector would measure the average blackness of the circle which you could then threshold.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-25T04:50:46.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21453" PostTypeId="2" ParentId="16714" CreationDate="2020-05-25T09:42:52.657" Score="1" Body="&lt;p&gt;If you search &lt;a href=&quot;https://paperswithcode.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papers with Code&lt;/a&gt; for &lt;a href=&quot;https://paperswithcode.com/search?q_meta=&amp;amp;q=python+%22machine+learning%22&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;python &quot;machine learning&quot;&lt;/code&gt;&lt;/a&gt; (or a more specific query) you will get numerous results. Note these will be mostly scientific applications or methods.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastActivityDate="2020-05-25T09:42:52.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21454" PostTypeId="2" ParentId="7488" CreationDate="2020-05-25T09:53:47.330" Score="2" Body="&lt;p&gt;In addition to the ways the term topology is itself used generically to describe the &quot;shape&quot; of various aspects of Machine Learning, the term appears in the field &lt;a href=&quot;https://en.wikipedia.org/wiki/Topological_data_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Topological Data Analysis&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In applied mathematics, topological data analysis (TDA) is an approach to the analysis of datasets using techniques from topology. Extraction of information from datasets that are high-dimensional, incomplete and noisy is generally challenging. TDA provides a general framework to analyze such data in a manner that is insensitive to the particular metric chosen and provides dimensionality reduction and robustness to noise. Beyond this, it inherits functoriality, a fundamental concept of modern mathematics, from its topological nature, which allows it to adapt to new mathematical tools.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Some examples of its use in ML:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.intlpress.com/site/pub/files/_fulltext/journals/hha/2016/0018/0001/HHA-2016-0018-0001-a021.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The ring of algebraic functions on persistence bar codes&lt;/a&gt;, Homology, Homotopy and Applications&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1507.06217&quot; rel=&quot;nofollow noreferrer&quot;&gt;Persistence Images: A Stable Vector Representation of Persistent Homology&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.birs.ca/events/2012/5-day-workshops/12w5081&quot; rel=&quot;nofollow noreferrer&quot;&gt;Topological data analysis and machine learning theory&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/327427685_Topological_Kernel_Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Topological Kernel Learning: Discrete Morse Theory is used to connect kernel machine learning with topological data analysis.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-05-25T10:15:15.123" LastActivityDate="2020-05-25T10:15:15.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21456" PostTypeId="1" CreationDate="2020-05-25T10:28:15.963" Score="2" ViewCount="60" Body="&lt;p&gt;Reading through the &lt;a href=&quot;http://cs229.stanford.edu/notes/cs229-notes7b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS229 lecture notes&lt;/a&gt; on generalised linear models, I came across the idea that a linear regression problem can be modelled as a Gaussian distribution, which is a form of the exponential family. The notes state that &lt;span class=&quot;math-container&quot;&gt;$h_{\theta}(x)$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$E[y | x; \theta]$&lt;/span&gt;. However, how can &lt;span class=&quot;math-container&quot;&gt;$h_{\theta}(x)$&lt;/span&gt; be equal to the expectation of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; given input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, since the expectation would require a sort of an averaging to take place?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given x, our goal is to predict the expected value of &lt;span class=&quot;math-container&quot;&gt;$T(y)$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. In most of our examples, we will have &lt;span class=&quot;math-container&quot;&gt;$T(y) = y$&lt;/span&gt;, so this means we would like the prediction &lt;span class=&quot;math-container&quot;&gt;$h(x)$&lt;/span&gt; output by our learned hypothesis h to satisfy &lt;span class=&quot;math-container&quot;&gt;$h(x) = E[y|x]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To show that ordinary least squares is a special case of the GLM family of models, consider the setting where the target variable y (also called the response variable in GLM terminology) is continuous, and we model the conditional distribution of y given x as a Gaussian &lt;span class=&quot;math-container&quot;&gt;$N(\mu,\sigma^2)$&lt;/span&gt;. (Here, &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; may depend &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.) So, we let the ExponentialFamily(&lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;) distribution above be the Gaussian distribution. As we saw previously, in the formulation of the Gaussian as an exponential family distribution, we had μ = η. So, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$h_{\theta}(x) = E[y|x; \theta] = \mu = \eta = \theta^Tx.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Upon reading other sources, &lt;span class=&quot;math-container&quot;&gt;$y_i \sim N(\mu_i, \sigma^2)$&lt;/span&gt; meaning that each individual output has it's own normal distribution with mean &lt;span class=&quot;math-container&quot;&gt;$\mu_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_{\theta}(x_i)$&lt;/span&gt; is set as the mean of the normal distribution for &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;. In that case, then the hypothesis makes sense to be assigned the expectation.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-26T02:49:17.830" Title="Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?" Tags="&lt;models&gt;&lt;linear-regression&gt;&lt;logistic-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21457" PostTypeId="1" AcceptedAnswerId="21476" CreationDate="2020-05-25T11:19:46.343" Score="5" ViewCount="283" Body="&lt;p&gt;The AIMA book has an exercise about showing that an MDP with rewards of the form &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt; can be converted to an MDP with rewards &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt;, and to an MDP with rewards &lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt; with equivalent optimal policies.&lt;/p&gt;&#xA;&lt;p&gt;In the case of converting to &lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt; I see the need to include a post-state, as the author's solution suggests. However, my immediate approach to transform from &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$r(s,a)$&lt;/span&gt; was to simply take the expectation of &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; with respect to s' (*). That is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ r(s,a) = \sum_{s'} r(s,a,s') \cdot p(s'|s,a) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The authors however suggest a pre-state transformation, similar to the post-state one. I believe that the expectation-based method is much more elegant and shows a different kind of reasoning that complements the introduction of artificial states. However, &lt;a href=&quot;https://edge.edx.org/assets/courseware/v1/749efba05a2ecb46dc276aaa3dd252cf/c4x/BerkeleyX/CS188-FA14/asset/section_8_solutions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;another resource&lt;/a&gt; I found also talks about pre-states.&lt;/p&gt;&#xA;&lt;p&gt;Is there any flaw in my reasoning that prevents taking the expectation of the reward and allow a much simpler transformation? I would be inclined to say no since the accepted answer &lt;a href=&quot;https://ai.stackexchange.com/q/10442/2444&quot;&gt;here&lt;/a&gt; seems to support this. This answer mentions Sutton and Barto's book, by the way, which also seems to be fine with taking the expectation of &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is the kind of existential question that bugs me from time to time and I wanted to get some confirmation.&lt;/p&gt;&#xA;&lt;p&gt;(*) Of course, that doesn't work in the &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt; case, as we do not have a probability distribution over the actions (that would be a policy, in fact, and that's what we are after).&lt;/p&gt;&#xA;" OwnerUserId="37359" LastEditorUserId="2444" LastEditDate="2021-01-20T17:05:35.867" LastActivityDate="2021-01-20T17:05:35.867" Title="How do I convert an MDP with the reward function in the form $R(s,a,s')$ to and an MDP with a reward function in the form $R(s,a)$?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;proofs&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21458" PostTypeId="2" ParentId="21392" CreationDate="2020-05-25T11:51:43.977" Score="1" Body="&lt;p&gt;There is a recent paper: &lt;a href=&quot;https://arxiv.org/pdf/2001.00449.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continuous-Discrete Reinforcement Learning for&#xA;Hybrid Control in Robotics&lt;/a&gt; published by DeepMind that aims to solve this problem, as stated in the abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Many real-world control problems involve both discrete decision variables – such as the choice of control modes, gear switching or digital outputs – as&#xA;  well as continuous decision variables – such as velocity setpoints, control gains or&#xA;  analogue outputs. However, when defining the corresponding optimal control or reinforcement learning problem, it is commonly approximated with fully continuous&#xA;  or fully discrete action spaces. These simplifications aim at tailoring the problem&#xA;  to a particular algorithm or solver which may only support one type of action&#xA;  space. Alternatively, expert heuristics are used to remove discrete actions from&#xA;  an otherwise continuous space. In contrast, we propose to &lt;strong&gt;treat hybrid problems&#xA;  in their ‘native’ form&lt;/strong&gt; by solving them with &lt;strong&gt;hybrid reinforcement learning&lt;/strong&gt;, which&#xA;  &lt;strong&gt;optimizes for discrete and continuous actions simultaneously&lt;/strong&gt;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The idea is that they use a &lt;em&gt;hybrid policy&lt;/em&gt; that uses a Gaussian distribution for the continuous decision variables and a categorical distribution for the discrete decision variables. Then, they extend the &lt;a href=&quot;https://arxiv.org/pdf/1806.06920.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maximum a Posteriori Policy Optimisation (MPO)&lt;/a&gt; algorithm (also by DeepMind) to allow it to handle hybrid policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=eUqQDLQXb7I&amp;amp;feature=youtu.be&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a video showing how they used the resulting &lt;em&gt;hybrid MPO&lt;/em&gt; policy in a robotics task, where in addition to the continuous actions, the robot can choose a discrete action which is the control mode to be used (coarse vs. fine).&lt;/p&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-05-25T11:51:43.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21460" PostTypeId="2" ParentId="10644" CreationDate="2020-05-25T13:45:52.557" Score="2" Body="&lt;p&gt;I found someone that has done this thing! You can hear a good explanation in Marcus Hutter's answer to &lt;a href=&quot;https://www.youtube.com/watch?v=E1AxVXt2Gv4&amp;amp;feature=youtu.be&amp;amp;t=3924&quot; rel=&quot;nofollow noreferrer&quot;&gt;this question&lt;/a&gt; about rewards given to AIXI. He describes a work that seems to be referring to this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hutter1.net/publ/ksaprob.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Universal Knowledge-Seeking Agents for Stochastic Environments&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll edit this answer later with a full explanation of the approach, but essentially the idea is that you use an AIXI model that does optimal reinforcement learning, giving it a reward that is based on information gained (phrased in a careful way to avoid a few common pitfalls). As a result, it learns to choose actions that give it the most information possible to predict the impacts of it's actions. This results in a &quot;scientist&quot; like behaviour, and you could imagine it doing things like turing the entire earth into a supercollider to better understand some physics laws if it decides that is the best approach for gaining maximum information. It would probably also do plenty of very unethical psycology experiements, for example, if it ended up deciding that human actions were important to predict and understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not a &quot;safe&quot; singularity in that sense, but that's okay, I didn't require that. It's at least a formal definition. It requires doing some uncomputable things, but the hope of future research is that we can make close enough approximations to those uncomputable things to be good enough anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel this theory is lacking any explanation of how feasable such a task is since it uses uncomputable agents, so I won't accept it yet, but it's the best answer I've seen so far. And I'll be watching future research closely to see if they can get a better handle on feasability, there seems to be quite a bit of work that has gone on in finding computable approximations to AIXI. The reason I care about feasability is because it is very relevant for mathmatically deciding how plausable something like an &quot;intelligence explosion&quot; actually is. So if a theory doesn't talk about feasability, it is missing out on a big piece of this question. Still this theory seems hopeful. For example, maybe there are fundamental computational limits to maximizing some reward functions, and we can prove that even certain levels of approximations for this reward function aren't computable. That would be a really interesting negative result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, I think the idea of using reinforcement learning, then choosing a reward function that tries to capture something instrinsic (such as &quot;curiosity&quot;) is a very good approach to trying to formally define the singularity. I look forward to seeing other potential reward functions defined in the future, I don't expect this to be the only one.&lt;/p&gt;&#xA;" OwnerUserId="6378" LastEditorUserId="6378" LastEditDate="2020-05-25T14:23:03.530" LastActivityDate="2020-05-25T14:23:03.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21461" PostTypeId="1" CreationDate="2020-05-25T15:33:23.793" Score="1" ViewCount="23" Body="&lt;p&gt;Let's assume that we make forecasting of another metric partially based on forecasts of the weather forecast, e.g. of temperature, pressure, then we can potentially obtain those forecasts from one of public APIs and have some information about future values of these features and do more precise prediction of another parameter taken as a label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the approach to use in this case if one or more of the features in multivariate forecasting have some forecasted values for the predicted horizon? It looks that in this case not only values from historical data can be used but also predicted values, though not clear how to organize the model in this case, e.g. it can be multivariate LSTM.&lt;/p&gt;&#xA;" OwnerUserId="37364" LastActivityDate="2020-05-25T15:33:23.793" Title="How to make a multivariate forecasting if one of features becomes known for the future with some confidence level, e.g. weather forecast data" Tags="&lt;machine-learning&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21462" PostTypeId="2" ParentId="21449" CreationDate="2020-05-25T16:13:42.083" Score="2" Body="&lt;p&gt;You have two options, either interpolate or restrict the actions only to values that produce states which are in your state vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest interpolation scheme is a linear interpolation, which works as follows (assuming &lt;code&gt;DS&lt;/code&gt; contains a set of grid points in increasing order). For a state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; you can locate its closest neighbours from the array &lt;code&gt;DS&lt;/code&gt; and the value in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; will, then, be a weighted average of the values in those neighbouring states. Formally, &lt;span class=&quot;math-container&quot;&gt;$$Q(s') = \frac{s_{i+1} - s'}{s_{i+1}-s_i}Q(s_i) + \frac{s'-s_i}{s_{i+1}-s_i}Q(s_{i+1}),$$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$s_i &amp;lt; s'&amp;lt; s_{i+1}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_{i+1}$&lt;/span&gt; are the neighbours, &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th element of &lt;code&gt;DS&lt;/code&gt;(i.e. &lt;code&gt;si = DS[i] = Q-table[i,1]&lt;/code&gt;), and &lt;span class=&quot;math-container&quot;&gt;$Q(s_i)$&lt;/span&gt; is related to the Q-table as &lt;code&gt;Qsi = Q-table[i,2]&lt;/code&gt; (asuming array indexing starts from 1). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Restricting the actions would work as follows. For simplicity, assume that the agent chooses the next state directly, i.e. &lt;span class=&quot;math-container&quot;&gt;$s'=a$&lt;/span&gt;. Then, if you have an array of actions &lt;code&gt;A = [a1, a2, ... , aM]&lt;/code&gt;, &lt;span class=&quot;math-container&quot;&gt;$M \le N$&lt;/span&gt; then each &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; needs to be present in the state array &lt;code&gt;DS&lt;/code&gt; (i.e. &lt;code&gt;A&lt;/code&gt; is a subset of &lt;code&gt;DS&lt;/code&gt;, formally &lt;span class=&quot;math-container&quot;&gt;$A \subseteq S$&lt;/span&gt;). This may not be desirable but it is an option. &lt;/p&gt;&#xA;" OwnerUserId="11495" LastEditorUserId="11495" LastEditDate="2020-05-27T07:43:05.557" LastActivityDate="2020-05-27T07:43:05.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21464" PostTypeId="1" CreationDate="2020-05-25T18:21:50.963" Score="2" ViewCount="30" Body="&lt;p&gt;I am a beginner in TensorFlow as well as in AI. I am basically from Pharma background and learning AI from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data with 5038 input (Float64) and 826 output (Categorical - Multi Labels in each column). I have utilized one-hot encoding but the neural network tackles only one output at a time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]How to process all 826 output (which give 6689 one-hot output) at once in a neural network. Here is the code that I am using.&#xA;[2] I am getting only 31% accuracy. I get this accuracy just in the second epoch. From second or third epoch onwards accuracy and other parameters become constant. Am I doing the wrong code here?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dataset = df.values&#xA;X = dataset[:,0:5038]/220&#xA;Y_smile = dataset[:,5038 :5864]&#xA;&#xA;from sklearn.preprocessing import OneHotEncoder&#xA;enc = OneHotEncoder(handle_unknown='ignore')&#xA;enc.fit(Y_smile)&#xA;OneHotEncoder(handle_unknown='ignore')&#xA;enc.categories_&#xA;Y = enc.transform(Y_smile).toarray()&#xA;print(Y,Y.shape, Y.dtype)&#xA;&#xA;from sklearn.model_selection import train_test_split&#xA;X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, Y, test_size=0.3)&#xA;X_val, X_test, Y_val, Y_test = train_test_split(X_val_and_test, Y_val_and_test, test_size=0.5)&#xA;&#xA;import numpy as np&#xA;X_train = np.asarray(X_train).astype(np.float64)&#xA;X_val = np.asarray(X_val).astype(np.float64)&#xA;X_test = np.asarray(X_test).astype(np.float64)&#xA;&#xA;filepath = &quot;bestmodelweights.hdf5&quot;&#xA;checkpoint = [tf.keras.callbacks.ModelCheckpoint(filepath, monitor='val_accuracy', mode='auto', save_best_only=True, Save_weights_only = True, verbose = 1), &#xA;              tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=5, verbose =1)]&#xA;&#xA;model = tf.keras.Sequential([&#xA;                             tf.keras.layers.Dense(1024, activation='relu', input_shape=(5038,)),&#xA;                             tf.keras.layers.Dense(524, activation='relu'),&#xA;                             tf.keras.layers.Dense(524, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),&#xA;                             tf.keras.layers.Dense(1024, activation='relu'),                        &#xA;                             tf.keras.layers.Dense(6689, activation= 'softmax')])&#xA;&#xA;model.compile(optimizer=tf.keras.optimizers.Adam(), loss=tf.keras.losses.BinaryCrossentropy(from_logits = True), metrics=['accuracy'])&#xA;&#xA;hist = model.fit(X_train, Y_train, epochs= 200, callbacks=[checkpoint],validation_data=(X_val, Y_val))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="34540" LastEditorUserId="30725" LastEditDate="2020-05-30T11:04:14.217" LastActivityDate="2020-05-30T11:04:14.217" Title="How to use one-hot encoding for multiple columns (multi-class) with varying number of labels in each class?" Tags="&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21465" PostTypeId="1" CreationDate="2020-05-25T19:59:10.857" Score="1" ViewCount="35" Body="&lt;p&gt;I was working on a CNN for HDR image generation from LDR images. I used an encoder-decoder architecture and merged the input with the decoder output. However I'm getting some banding artifacts in the model prediction as shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)Input LDR image&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/licI4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/licI4.png&quot; alt=&quot;Input LDR Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Ground Truth HDR&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LaiSF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LaiSF.png&quot; alt=&quot;Ground Truth HDR&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Predicted output&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x3rjB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x3rjB.png&quot; alt=&quot;Predicted Output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the fine bands in the prediction. What might be causing these bands? Also I trained only for 20 epochs yet. Is the problem due to inadequate training?&#xA;Here's my model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; class TestNet2(nn.Module):&#xA;    def __init__(self):&#xA;        super(TestNet2, self).__init__()&#xA;&#xA;    def enclayer(nIn, nOut, k, s, p, d=1):&#xA;        return nn.Sequential(&#xA;            nn.Conv2d(nIn, nOut, k, s, p, d), nn.SELU(inplace=True)&#xA;        )&#xA;    def declayer(nIn, nOut, k, s, p):&#xA;        return nn.Sequential(&#xA;            nn.ConvTranspose2d(nIn, nOut, k, s, p), nn.SELU(inplace=True)&#xA;        )&#xA;&#xA;    self.encoder = nn.Sequential(&#xA;        enclayer(3,64,5,3,1),&#xA;        #nn.MaxPool2d(2, stride=2),&#xA;        enclayer(64,128,5,3,1),&#xA;        #nn.MaxPool2d(2, stride=2),&#xA;        enclayer(128,256,5,3,1),&#xA;        #nn.MaxPool2d(2, stride=2),&#xA;    )&#xA;    self.decoder = nn.Sequential(&#xA;        declayer(256,128,5,3,1),&#xA;        declayer(128,64,5,3,1),&#xA;        declayer(64,3,5,3,1),&#xA;&#xA;&#xA;    )&#xA;&#xA;  def forward(self, x):&#xA;      xc=x&#xA;      x = self.encoder(x)&#xA;      x = self.decoder(x)&#xA;      x = F.interpolate(&#xA;        x, (512, 512), mode='bilinear', align_corners=False&#xA;      )&#xA;      x=x+xc&#xA;      return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36767" LastActivityDate="2020-05-25T19:59:10.857" Title="Banding artifacts in CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21466" PostTypeId="1" CreationDate="2020-05-25T23:08:21.720" Score="1" ViewCount="38" Body="&lt;p&gt;I am working on stock price prediction project, I am using the &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;support vector regression (SVR) model&lt;/a&gt; for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I am splitting my data into train and test, I am getting high accuracy while predicting test data after fitting the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now when I am trying to use another data which I separate out from the used dataset before start doing anything, it gives me very bad results. Can anyone tell me what's happening?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bHKzL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bHKzL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking forward to your response.&lt;/p&gt;&#xA;" OwnerUserId="33670" LastEditorUserId="2444" LastEditDate="2020-05-25T23:47:49.887" LastActivityDate="2020-05-25T23:47:49.887" Title="Why is the accuracy of my model very low on a separate dataset from the training and test datasets?" Tags="&lt;machine-learning&gt;&lt;data-science&gt;&lt;support-vector-machine&gt;&lt;accuracy&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21468" PostTypeId="2" ParentId="21456" CreationDate="2020-05-26T02:49:17.830" Score="1" Body="&lt;p&gt;In generalised Linear models, each output variable &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is modelled as a distribution from the exponential family, with the hypothesis function &lt;span class=&quot;math-container&quot;&gt;$h_{\theta}(x)$&lt;/span&gt; for a given &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; as the expected value of &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; and maximum likelihood estimation is usually the method used to solve GLM's. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-05-26T02:49:17.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21469" PostTypeId="1" CreationDate="2020-05-26T02:56:54.747" Score="2" ViewCount="49" Body="&lt;p&gt;Can weighted importance sampling (WIS) and importance sampling (IS) be applied to off-policy evaluation for continuous state spaces MDPs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that I have trajectories of &lt;span class=&quot;math-container&quot;&gt;$(s_t,a_t)$&lt;/span&gt; pairs and the behavior policy distribution &lt;span class=&quot;math-container&quot;&gt;$\pi_b(a_t | s_t)$&lt;/span&gt; can be approximated with a neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;http://incompleteideas.net/papers/MHS-WISnips2014.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; I came across where they say that IS can be used for continuous states whereas WIS cannot be used for function approximation. I am not sure why WIS cannot be applied to the continuous case whereas IS can be. Both of these techniques seem similar. &lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-05-26T10:43:28.703" LastActivityDate="2020-05-26T10:43:28.703" Title="Can weighted importance sampling be applied to off-policy evaluation for continuous state space MDPs?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21470" PostTypeId="1" CreationDate="2020-05-26T05:12:25.093" Score="1" ViewCount="61" Body="&lt;p&gt;This is a general question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a general file type associated with AI projects?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Photoshop = .psd&#xA;Excel = csv&#xA;Artificial Intelligence = ?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37373" LastEditorUserId="2444" LastEditDate="2020-05-26T10:57:09.357" LastActivityDate="2020-05-26T10:57:09.357" Title="Is there a general file type associated with AI projects?" Tags="&lt;open-ai&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21472" PostTypeId="2" ParentId="21470" CreationDate="2020-05-26T06:25:26.913" Score="3" Body="&lt;p&gt;No, there is no file type associated with AI projects in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your examples of Photoshop and Excel are specific corporate branded products. These store bespoke data that only works with those products (plus maybe a few converters that can read the files for competitor products).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even more general examples such as &lt;code&gt;.jpg&lt;/code&gt; for images or &lt;code&gt;.txt&lt;/code&gt; for text documents are not a good match to AI in general. AI is such a broad field, it is next to impossible to define a standard set of components of an &quot;AI project&quot; in order to build a single file format that could handle the contents of all AI projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, practical work in AI is likely to include use of specific file formats and extensions, depending on what you do. For instance if you work with Python and TensorFlow you are likely to use &lt;code&gt;.py&lt;/code&gt; files for your source code, &lt;code&gt;.ckpt&lt;/code&gt; for neural network training checkpoints, &lt;code&gt;.pb&lt;/code&gt; for saved models. The last two extensions - ckpt and pb - are only semi-formal naming conventions though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This lack of a single file extension, and use of maybe a dozen ones that you would be familiar with on a project, is true of most coding and software development work. If you work with an integrated devlopment environment (IDE), then you &lt;em&gt;might&lt;/em&gt; have a single &quot;master&quot; file that allows you to load up all the resources in the project to work on them. That is entirely optional though, and may have any of a number of extensions depending on which IDE you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a quick example, the &lt;a href=&quot;https://github.com/huggingface/transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformers&lt;/a&gt; project on github is a popular resource for natural language processing, a topic often considered as part of AI. A quick look at that project shows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Various development tool configuration files either starting with &lt;code&gt;.&lt;/code&gt; or with various extensions - &lt;code&gt;.cfg&lt;/code&gt;, &lt;code&gt;.yaml&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;.py&lt;/code&gt; files for Python source code&lt;/li&gt;&#xA;&lt;li&gt;Script files with &lt;code&gt;.sh&lt;/code&gt; extension or with extension removed for convenience of using on command line&lt;/li&gt;&#xA;&lt;li&gt;Dcoumentation written with &lt;code&gt;.txt&lt;/code&gt;, &lt;code&gt;.md&lt;/code&gt; and &lt;code&gt;.rst&lt;/code&gt; extensions.&lt;/li&gt;&#xA;&lt;li&gt;Some &lt;code&gt;.ipynb&lt;/code&gt; files used for Jupyter notebooks - a format that bundles documentation, Python scripts and their output for sharing work.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-26T06:46:08.023" LastActivityDate="2020-05-26T06:46:08.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21474" PostTypeId="1" CreationDate="2020-05-26T09:43:00.153" Score="1" ViewCount="28" Body="&lt;p&gt;I am trying to implement an extractive text summarization model. I am using keras and tensorflow. I have used bert sentence embeddings and the embeddings are fed into an LSTM layer and then to a Dense layer with sigmoid activation function. I have used adam optimizer and binary crossentropy as the loss function. The input to the model are the sentence embeddings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training y labels is a 2d-array i.e [array_of_documents[array_of_biniary_labels_foreach_sentence]]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that during training, I am getting the training accuracy of around 0.22 and loss 0.6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I improve my accuracy for the model?&lt;/p&gt;&#xA;" OwnerUserId="37006" LastActivityDate="2020-05-26T09:43:00.153" Title="Low accuracy during training for text summarization" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;accuracy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21475" PostTypeId="1" CreationDate="2020-05-26T10:28:24.823" Score="1" ViewCount="55" Body="&lt;p&gt;I started thinking about the fairness of machine learning models recently. &#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fairness_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wiki page for Fairness_(machine_learning)&lt;/a&gt; defines fairness as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In machine learning, a given algorithm is said to be fair, or to have&#xA;  fairness if its results are independent of some variables we consider&#xA;  to be sensitive and not related to it (f.e.: gender, ethnicity,&#xA;  sexual orientation, &lt;strong&gt;etc.&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://fairmlclass.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UC Berkley CS 294&lt;/a&gt; in turn defines fairness as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;understanding and mitigating discrimination based on sensitive&#xA;  characteristics, &lt;strong&gt;such as&lt;/strong&gt;, gender, race, religion, physical ability,&#xA;  and sexual orientation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Many other resources, like Google in the &lt;a href=&quot;https://developers.google.com/machine-learning/fairness-overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;ML Fairness&lt;/a&gt; limit the fairness to these aforementioned categories and no other categories are considered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But fairness is a lot broader context than simply these few categories mentioned here, you could easily add another few like IQ, height, beauty anything that could have a real impact on your credit score, school application or job application. Some of these categories may not be popular in existing datasets nowadays but given the exponential growth of data, they will be soon, to the extent, that we will have an abundance of data about every individual with &lt;strong&gt;all their physical and mental categories&lt;/strong&gt; mapped into the datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the question would be how to define fairness given all these categories presented in the datasets. And will it even be possible to define fairness if all physical and mental dimensions are considered as it seems that when we do so, all our weights in, say, the neural nets should be exactly the same, i.e., giving no discriminator in any way or form towards or against any physical or mental category of a human being? That means that a machine learning system that is fair across all possible dimensions will have no way of distinguishing one human being from another which would render these machine learning models useless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To wrap it up, while it does make perfect sense to withdraw bias towards any individual given the categories like gender, ethnicity, sexual orientation, etc., the set is not closed and with the increasing number of categories being added to this set, we will inevitably arrive at a point where no discrimination (in a statistical sense) would be possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that's why my question, are fair machine learning models possible?&#xA;Or perhaps, the only possible fair machine learning models are those that arbitrarily include some categories but ignore other categories which, of course, if far from being fair.&lt;/p&gt;&#xA;" OwnerUserId="15780" LastEditorUserId="30725" LastEditDate="2020-06-01T12:22:53.030" LastActivityDate="2020-06-01T12:22:53.030" Title="Is it possible to create a fair machine learning system?" Tags="&lt;machine-learning&gt;&lt;explainable-ai&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21476" PostTypeId="2" ParentId="21457" CreationDate="2020-05-26T11:37:17.783" Score="3" Body="&lt;p&gt;I think I may be in position to answer my own question. The Bellman equation (for the optimal policy) for a MDP with &lt;span class=&quot;math-container&quot;&gt;$r(s,a,s')$&lt;/span&gt; rewards would look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) = \max_a \left\{ \sum_{s'} p(s'|s,a)(r(s,a,s') + \gamma V(s')) \right\} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(s) = \max_a \left\{ \sum_{s'} p(s'|s,a) \cdot r(s,a,s') + \gamma \sum_{s'} p(s'|a,s) \cdot V(s') \right\} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, &lt;span class=&quot;math-container&quot;&gt;$ \sum_{s'} p(s'|s,a) \cdot r(s,a,s') $&lt;/span&gt; is precisely &lt;span class=&quot;math-container&quot;&gt;$ \mathbb{E}\left[ r(s,a,s') | s,a \right] = r(s,a) $&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So all in all, the resulting Bellman equation looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) = \max_a \left\{r(s,a) + \gamma \sum_{s'} p(s'|s,a) \cdot V(s') \right\} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear, then, that a process with &lt;span class=&quot;math-container&quot;&gt;$ r(s,a,s') $&lt;/span&gt; rewards can be transformed to a &lt;span class=&quot;math-container&quot;&gt;$ r(s,a) $&lt;/span&gt; process without introducing artificial states and maintaining the optimal policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note unrelated to the question itself, that leads me to believe that &lt;span class=&quot;math-container&quot;&gt;$ r(s,a,s') $&lt;/span&gt; functions may be convenient in some scenarios, but they do not add &quot;expressive power&quot;, in the sense that they do not allow to model problems more compactly (as it happens when we extend &lt;span class=&quot;math-container&quot;&gt;$ r(s) $&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$ r(s,a) $&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="37359" LastEditorUserId="37359" LastEditDate="2020-05-26T13:41:29.293" LastActivityDate="2020-05-26T13:41:29.293" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21477" PostTypeId="1" AcceptedAnswerId="21668" CreationDate="2020-05-26T13:34:22.367" Score="1" ViewCount="322" Body="&lt;p&gt;I was reading &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/guide/rl_tips.html#tips-and-tricks-when-creating-a-custom-environment&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; tips &amp;amp; tricks for training in DRL and I noticed the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;always normalize your observation space when you can, i.e., when you know the boundaries&lt;/li&gt;&#xA;  &lt;li&gt;normalize your action space and make it symmetric when continuous (cf potential issue below) A good practice is to rescale your actions to lie in [-1, 1]. This does not limit you as you can easily rescale the action inside the environment&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am working on a discrete action space but it is quite difficult to normalize my states when I don't actually know the full range for each feature (only an estimation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this affect training? And more specifically, why on continuous action spaces we need to normalize also the action's values?&lt;/p&gt;&#xA;" OwnerUserId="35978" LastEditorUserId="2444" LastEditDate="2020-05-26T14:19:26.107" LastActivityDate="2020-06-05T16:53:01.207" Title="Why do we also need to normalize the action's values on continuous action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;deep-rl&gt;&lt;a3c&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21478" PostTypeId="2" ParentId="12411" CreationDate="2020-05-26T15:51:48.633" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Branch_and_bound#Generic_version&quot; rel=&quot;nofollow noreferrer&quot;&gt;Branch and Bound&lt;/a&gt; is similar to an &lt;a href=&quot;https://en.wikipedia.org/wiki/Brute-force_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;exhaustive search&lt;/a&gt;, except it incorporates a method for computing lower bounds on branches. If the lower bound on a given branch is greater than the upper bound on the problem (i.e. the current best solution encountered), that branch can be discarded since it will never produce an optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, since you explore all options except those you &lt;em&gt;know&lt;/em&gt; will produce values less optimal than your current best value, you are guaranteed to encounter the global optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note this is a generic algorithm, and you will need to reference a specific implementation if you want proof of why it satisfies these criteria.&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-05-26T16:26:50.077" LastActivityDate="2020-05-26T16:26:50.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21479" PostTypeId="2" ParentId="19849" CreationDate="2020-05-26T16:15:50.760" Score="2" Body="&lt;h1&gt;ReLU is non-linear by definition&lt;/h1&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In calculus and related areas, a &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear function&lt;/a&gt; is a function whose graph is a straight line, that is a polynomial function of degree one or zero.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since the graph of the ReLU function &lt;span class=&quot;math-container&quot;&gt;$f(x) = \max(0,x)$&lt;/span&gt; is not a straight line (equivalently, it cannot be expressed in the form &lt;span class=&quot;math-container&quot;&gt;$f(x) = mx + c$&lt;/span&gt;), by definition it is not linear.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;ReLU &lt;em&gt;is&lt;/em&gt; piecewise linear&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;ReLU &lt;em&gt;is&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Piecewise_linear_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;piecewise linear&lt;/a&gt; on the bounds &lt;span class=&quot;math-container&quot;&gt;$(-\inf,0]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[0,\inf)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x) = \max(0,x) = \begin{cases}&#xA;               0               &amp;amp; x \le 0\\&#xA;               x               &amp;amp; x \gt 0\\&#xA;       \end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is still non-linear on the entire domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gg4bV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gg4bV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-05-26T16:34:38.610" LastActivityDate="2020-05-26T16:34:38.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21480" PostTypeId="1" CreationDate="2020-05-26T20:41:46.303" Score="2" ViewCount="37" Body="&lt;p&gt;The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vhEO0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vhEO0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some related links:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://coggle.it/diagram/XsAi9ZnPdIGARTN0/t/reasons-for-sample-inefficiency-of-on-policy-deep-rl/fae93aa14ab81e96310145594778527a7cee5bee454782e355477fc83603278f&quot; rel=&quot;nofollow noreferrer&quot;&gt;interactive mindmap&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/reinforcementlearning/comments/gqtq40/have_we_caught_em_all_can_you_find_another_reason/?utm_source=share&amp;amp;utm_medium=web2x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit post&lt;/a&gt; asking the same question&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/18645/35821&quot;&gt;related SE question&lt;/a&gt; with only a few reasons all mentioned here&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="35821" LastActivityDate="2020-05-26T20:41:46.303" Title="Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;sample-efficiency&gt;&lt;sample-complexity&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21481" PostTypeId="2" ParentId="17883" CreationDate="2020-05-26T22:25:38.333" Score="0" Body="&lt;p&gt;My immediate suggestion would be to zero-fill the missing values, but I recalled the below comment suggesting a more sophisticated method:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Karim:&lt;/strong&gt; &lt;em&gt;How to deal with different size of feature vectors?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Nabila:&lt;/strong&gt; &lt;em&gt;That's a problem I'm actually working on. I've seen that you can create separate networks for each type of node feature, and sort of project them - so train them separately, and project them to the same size.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Or you can do concatenation so you don't have to worry about that, but at some point they all need to be the same size to do classification at the end.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://youtu.be/54ohmr2LYuI?t=2346&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Literature Review on Graph Neural Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-26T22:25:38.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21482" PostTypeId="1" CreationDate="2020-05-27T00:52:09.220" Score="1" ViewCount="51" Body="&lt;p&gt;There are several different angles we can classify Reinforcement Learning methods from. We can distinguish three main aspects :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Value-based and policy-based&lt;/li&gt;&#xA;&lt;li&gt;On-policy and off-policy&lt;/li&gt;&#xA;&lt;li&gt;Model-free and model-based&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Historically, due to their sample-efficiency, the model-based methods have been used in the robotics field and other industrial controls. That is happened due to the cost of the hardware and the physical limitations of samples that could be obtained from a real robot. Robots with a large number of degrees of freedom are not widely accessible, so RL researchers are more focused on computer games and other environments where samples are relatively cheap. However, the ideas from robotics are infiltrating, so, who knows, maybe the model-based methods will enter the focus quite soon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we know, &quot;model&quot; means the model of the environment, which could have various forms, for example, providing us with a new state and reward from the current state and action. From what I have seen so far, all the methods (i.e. A3C, DQN, DDPG) put zero effort into predicting, understanding, or simulating the environment. What we are interested in is proper behavior (in terms of the final reward), specified directly (a policy) or indirectly (a value), given the observation. The source of observations and reward is the environment itself, which in some cases could be very slow and inefficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a model-based approach, we're trying to learn the model of the environment to reduce the &quot;real environment&quot; dependency. If we have an accurate environment model, our agent can produce any number of trajectories that it needs, simply by using the model instead of executing the actions in the real world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in a day trading environment. Is it possible to create a model-based environment in order to build an accurate day trading environment model?&lt;/p&gt;&#xA;" OwnerUserId="35626" LastEditorUserId="35626" LastEditDate="2020-05-27T12:13:54.930" LastActivityDate="2020-05-27T12:13:54.930" Title="Using a model-based method to build an accurate day trading environment model" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;model-based-methods&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21484" PostTypeId="1" CreationDate="2020-05-27T09:50:13.723" Score="1" ViewCount="148" Body="&lt;p&gt;There are a lot of examples of balancing a pole (see image below) using reinforcement learning, but I find that almost all examples start close to the upright position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good source (or paper) for when the pole actually starts all the way at the bottom? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9NbPg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9NbPg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2020-05-27T11:40:04.687" LastActivityDate="2020-05-27T14:31:49.907" Title="Is there any good source for when the pole actually starts all the way at the bottom, in the cartpole problem?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21485" PostTypeId="1" AcceptedAnswerId="21486" CreationDate="2020-05-27T10:19:23.760" Score="1" ViewCount="419" Body="&lt;p&gt;I have recently watched David silver's course, and started implementing the deep Q-learning algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought I should make a switch between the Q-target and Q-current directly (meaning, every parameter of Q-current goes to Q-target), but I found a repository on GitHub where that guy updates Q-target as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{\text{target}} = \tau * Q_{\text{current}} + (1 - \tau)*Q_{\text{target}}$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is some number probably between 0 and 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that update correct or I miss something? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought after some iterations (e.g. 2000 iteration), we should update Q-target as: &lt;span class=&quot;math-container&quot;&gt;$Q_{\text{target}}=Q_{\text{current}}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-27T19:35:25.313" LastActivityDate="2020-05-27T19:35:25.313" Title="How and when should we update the Q-target in deep Q-learning?" Tags="&lt;deep-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21486" PostTypeId="2" ParentId="21485" CreationDate="2020-05-27T11:43:59.257" Score="4" Body="&lt;p&gt;The update form &lt;span class=&quot;math-container&quot;&gt;$\theta^{\prime} \leftarrow \tau \theta+(1-\tau) \theta^{\prime}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\theta'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; represent the weights of the target network and the current network, respectively) does exist and is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is called &lt;em&gt;soft update&lt;/em&gt; and it has been used in the &lt;a href=&quot;https://arxiv.org/pdf/1509.02971.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Deterministic Policy Gradient (DDPG)&lt;/a&gt; paper, which uses the concept of a target network like DQN. The authors of the paper state that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The weights of these target networks are then updated by having them &lt;strong&gt;slowly track the learned networks&lt;/strong&gt;: &lt;span class=&quot;math-container&quot;&gt;$\theta ' \leftarrow \tau \theta + (1 − \tau )\theta'$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\tau &amp;lt;&amp;lt; 1$&lt;/span&gt;. This means that the target values are constrained to change slowly, greatly&#xA;  improving the stability of learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This update will be made in &lt;em&gt;each time step&lt;/em&gt; as follows. For example, for &lt;span class=&quot;math-container&quot;&gt;$\tau= 0.001$&lt;/span&gt;, the new weights for the target network will take &lt;span class=&quot;math-container&quot;&gt;$0.1\%$&lt;/span&gt; of the main network’s weights and &lt;span class=&quot;math-container&quot;&gt;$99.9 \%$&lt;/span&gt; of the old target network weights. This does not go against the purpose of fixed target networks (which have been introduced to address the problem of “moving targets”). In fact, by keeping &lt;span class=&quot;math-container&quot;&gt;$99.9\%$&lt;/span&gt; of the old target network weights, they can still be considered as &lt;em&gt;fixed&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seeing that this resulted in improvements in DDPG, some DQN implementations/tutorials started to use soft updates for the target network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is opposed to the &lt;em&gt;hard update&lt;/em&gt; scheme used in the original DQN paper, i.e. the weights are copied every &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; steps. This means that the target network is kept fixed for &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; steps (10000 in the paper) and then gets a significant update.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="34010" LastEditDate="2020-05-27T12:33:03.113" LastActivityDate="2020-05-27T12:33:03.113" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21487" PostTypeId="2" ParentId="21484" CreationDate="2020-05-27T11:57:22.287" Score="2" Body="&lt;p&gt;It is difficult to prove a negative, but I doubt there will be a paper on that specific problem. It should be relatively easy to adjust the environment or write a new one that does this if you wished though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very similar environment that does have a &lt;em&gt;lot&lt;/em&gt; more written about it is Acrobot, &lt;a href=&quot;https://gym.openai.com/envs/Acrobot-v1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;which does have a OpenAI Gym version&lt;/a&gt;. Instead of a cart on a track with forces applied and a free join to the pole, there is a longer pole fixed to a free joint, with an active joint in the middle (that the agent can apply forces to). It can be thought of as a very simple model of an acrobat on a trapeze swing (with poles instead of chains, so the swing is stable balanced upside-down)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The degrees of freedom and difficulty of the Acrobot task are similar to CartPole - I would rate it as a harder problem overall, but if you started CartPole with the pole hanging down in fact the two problems are very similar. Usually the joint motor is made too weak to achieve the goal of balancing in a single clean action, and the agent must learn to build momentum over a few swings, before moving to the balance point. That also makes both your alternate start position CartPole and Acrobot similar to &lt;a href=&quot;https://gym.openai.com/envs/MountainCar-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MountainCar&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a low-effort look at a very similar problem you could try Acrobot. Otherwise you will likely need to do some custom work on CartPole.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-27T11:57:22.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21488" PostTypeId="1" AcceptedAnswerId="21489" CreationDate="2020-05-27T12:44:33.367" Score="6" ViewCount="183" Body="&lt;p&gt;The text book being referred to, in this question is &amp;quot;Reinforcement Learning: An introduction&amp;quot; by Richard Sutton and Andrew Barto (second edition, 2018). For your convenience, I have enclosed the following part of a paragraph about &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policies in the book, to convey my question with a better clarity. This paragraph can be found at the end of Pg 100, under section 5.4 .&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IXSf3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IXSf3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;So, the non-greedy actions are given the probability &lt;span class=&quot;math-container&quot;&gt;$\frac{\epsilon}{|\mathscr{A}(s)|}$&lt;/span&gt;, and the greedy action is given the probability &lt;span class=&quot;math-container&quot;&gt;$1-\epsilon+\frac{\epsilon}{|\mathscr{A}(s)|}$&lt;/span&gt;. All clear upto this point. However, I have a doubt in the policy improvement theorem that is mentioned in Pg 101, under section 5.4. I have enclosed a copy of this proof for your convenience:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_\pi(s, \pi'(s)) = \sum_a \pi'(a|s)q_\pi(s,a) \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ = \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) + (1-\epsilon)\max_a q_\pi(s,a) \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \geq \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) + (1-\epsilon)\sum_a\frac{\pi(a|s) - \frac{\epsilon}{|\mathscr{A}(s)|}}{1-\epsilon}q_\pi(s,a) \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \  \ \ = \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) - \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) + \sum_a \pi(a|s)q_\pi(s,a) \\ = v_\pi(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is, shoundn't the greedy action be chosen with a probability of &lt;span class=&quot;math-container&quot;&gt;$1-\epsilon + \frac{\epsilon}{|\mathscr{A}(s)|}$&lt;/span&gt;? The weighing factors do not add up to 1, as they are probability values. With this argument, the proof (with a slight modification) would be:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_\pi(s, \pi'(s)) = \sum_a \pi'(a|s)q_\pi(s,a) \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ = \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) + (1-\epsilon + \frac{\epsilon}{|\mathscr{A}(s)|})\max_a q_\pi(s,a) \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \geq \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) + (1-\epsilon + \frac{\epsilon}{|\mathscr{A}(s)|})\sum_a\frac{\pi(a|s) - \frac{\epsilon}{|\mathscr{A}(s)|}}{1-\epsilon + \frac{\epsilon}{|\mathscr{A}(s)|}}q_\pi(s,a) \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \  \ \ = \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) - \frac{\epsilon}{|\mathscr{A}(s)|}\sum_aq_\pi(s,a) + \sum_a \pi(a|s)q_\pi(s,a) \\ = v_\pi(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Though the end result isn't changed, I just want to know what I am conceptually missing, in order to understand the proof that is originally provided. I am extremely sorry, if this is something very elementary that I am not able to fathom.&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much for your time.&lt;/p&gt;&#xA;" OwnerUserId="37181" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-05-28T14:15:54.093" Title="Is this proof of $\epsilon$-greedy policy improvement correct?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;greedy-ai&gt;&lt;policy-improvement&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21489" PostTypeId="2" ParentId="21488" CreationDate="2020-05-27T13:29:25.450" Score="6" Body="&lt;p&gt;The weights &lt;em&gt;do&lt;/em&gt; sum to one. Note that in the second line where we have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\epsilon}{|\mathcal{A}(s)|} \sum_a q_{\pi}(s,a) + (1-\epsilon)\max_aq_{\pi}(s,a) \; ,$$&lt;/span&gt;&#xA;the sum is over the whole action space, including the greedy action, so the sum of the weights will be &lt;span class=&quot;math-container&quot;&gt;$\frac{\epsilon}{|\mathcal{A}(s)|} \times |\mathcal{A}(s)| + (1-\epsilon) = 1$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-05-28T13:39:00.260" LastActivityDate="2020-05-28T13:39:00.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21490" PostTypeId="1" CreationDate="2020-05-27T14:04:54.133" Score="2" ViewCount="29" Body="&lt;p&gt;I'm trying to look for a task that predicts a discrete label first (classification), and then predicts the multiple continuous attributes of the predicted class. I found some papers about &lt;a href=&quot;http://oa.upm.es/40804/1/INVE_MEM_2015_204213.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi-output regression&lt;/a&gt;, but it wasn't what I wanted. Perhaps such a task exists in robot control or in video games, but I haven't found it yet. At the same time I want to train within a supervised learning framework, not reinforcement learning.&lt;/p&gt;&#xA;" OwnerUserId="37401" LastActivityDate="2020-05-27T14:04:54.133" Title="Is there a classification task with multiple attribute regression?" Tags="&lt;classification&gt;&lt;regression&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21491" PostTypeId="2" ParentId="21484" CreationDate="2020-05-27T14:31:49.907" Score="2" Body="&lt;p&gt;Almost certainly, there is no such paper since that would be a trivial problem. The pole lying flat is the definition of failure, hence game over. If you started in that position, you would be permanently in the game-over state and you would never learn anything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is that if the pole is lying flat, then, if you apply a force on the cart (in the same direction as the pole is pointing, say), the pole head moves in exactly the same direction as the cart (i.e. the directional vectors of the cart and pole head movements are identical). Hence, the pole head never moves upwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I am fairly certain, that below certain angle with the surface, the pole can no longer be stabilized. This should be possible to prove from the dynamic equations governing the movement of the cart and the pole. This may not be easy, though, and definitely not easy for me (these are second-order differential equations). Anyway, with this in mind, you can see why one needs to start close enough to the stationary point for the problem to have a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the pole could go below the surface and swing, however, as &lt;a href=&quot;https://gereshes.com/2019/08/19/swing-up-of-an-inverted-pendulum-on-a-cart/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, it would be similar to the acrobot problem mentioned by Neil and you could start anywhere. &lt;/p&gt;&#xA;" OwnerUserId="11495" LastActivityDate="2020-05-27T14:31:49.907" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21493" PostTypeId="1" CreationDate="2020-05-27T18:17:46.637" Score="1" ViewCount="78" Body="&lt;p&gt;I'm new to working with neural networks and have recently began implementing neural networks for time series forecasting in some of my work. I've been particularly using Echo State Networks and have been doing some reading to understand how they work. For the most part, things seem pretty straight forward, but I'm confused as to why we use a 'delay' when feeding our input data (the delay concept mentioned in the paper &lt;a href=&quot;https://science.sciencemag.org/content/304/5667/78&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication&lt;/a&gt;)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking at &lt;a href=&quot;https://github.com/stefanonardo/echo-state-network&quot; rel=&quot;nofollow noreferrer&quot;&gt;some source code on Github&lt;/a&gt;, and they implement this delay as well (they feed two arrays inputData &amp;amp; targetData, into the network where one is delayed by one element relative to the other). I am noticing that the larger the delay, the worse the fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is this done? &#xA;My interest is eventually to forecast past sample data.&lt;/p&gt;&#xA;" OwnerUserId="37403" LastEditorUserId="2444" LastEditDate="2020-05-27T19:32:31.050" LastActivityDate="2021-03-04T18:06:35.657" Title="Why do we use a delay when feeding our input data to the echo state network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;time-series&gt;&lt;echo-state-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21494" PostTypeId="5" CreationDate="2020-05-27T19:28:01.700" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-27T19:28:01.700" LastActivityDate="2020-05-27T19:28:01.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21495" PostTypeId="4" CreationDate="2020-05-27T19:28:01.700" Score="0" Body="For questions related to the echo state network (ESN), which is a recurrent neural network with a sparsely connected hidden layer." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-27T19:28:01.700" LastActivityDate="2020-05-27T19:28:01.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21496" PostTypeId="1" CreationDate="2020-05-27T19:30:51.833" Score="1" ViewCount="33" Body="&lt;p&gt;I was thinking about training a neural network to colourize images. The input would be the luminosity/value for each pixel, and the output would be a hue and/or saturation. Training data would be easily obtained just by selecting the luminosity/value channel from a full colour image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose all channels are scaled to 0.0-1.0, there is a problem with pixels whose hue is nearly 0.0 or nearly 1.0.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The input data may have sharp discontinuities in hue which are not visible to the human eye. This is an unstable, illusory boundary which seems like it would destablilize the training.&lt;/li&gt;&#xA;&lt;li&gt;Also if the network outputs a value of 1.001 instead of 0.001 then this should NOT be penalised since.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Possible workarounds might be to preprocess the image to remap e.g. 0.99 to -0.01 if that pixel is near a region dominated by near-0 hues, or similarly to to remap e.g. 0.01 to 1.01 if that pixel is near a region dominated by near-1 hues. This has its own problems. Similarly, outputs could be wrapped to the range 0-1 before being scored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is there a better way to encode cyclic values such as hue so that they will naturally be continuous?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution I thought of would be to treat (hue,saturation) as a (theta,r) polar coordinate and translate this to Cartesian (x,y) and have that be the training target, but I don't know how this change of colour space will affect things (it might be find, I haven't tried it yet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there alternative colour representations which are better suited to machine learning?&lt;/p&gt;&#xA;" OwnerUserId="22279" LastActivityDate="2020-05-27T20:37:22.560" Title="How could a NN be trained to output a cyclic (e.g. hue) number?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21497" PostTypeId="2" ParentId="21496" CreationDate="2020-05-27T20:37:22.560" Score="1" Body="&lt;p&gt;Your solution is pretty much on spot. It corresponds to the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/YUV&quot; rel=&quot;nofollow noreferrer&quot;&gt;YUV scheme&lt;/a&gt; used in television and designed to match human perception characteristics. As you already noticed, such an encoding wouldn't suffer from discontinuities.&lt;/p&gt;&#xA;" OwnerUserId="22993" LastActivityDate="2020-05-27T20:37:22.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21498" PostTypeId="1" CreationDate="2020-05-28T02:33:02.680" Score="1" ViewCount="44" Body="&lt;p&gt;I am reading through a paper (&lt;a href=&quot;https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273&lt;/a&gt;) where they describe logloss as a ranking function and can be simplified to the margin of the training data &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. I am not sure what the transformation is in each step and could use a bit of help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A precursor to this ranking loss is standard logloss which may clarify my understanding as well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this loss I only get from step 2 to here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-\sum_{i=1}^n log(\frac{{e^{y_iF(x_i,\overline{a})}}}{1 + e^{yF(x,\overline{a})}})$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=-\sum_{i=1}^ny_iF(x_i, \overline{a}) - log(1 + e^{yF(x,\overline{a})})$$&lt;/span&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zPwlF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zPwlF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the full ranking loss I am having trouble on:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ymDux.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ymDux.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36486" LastActivityDate="2020-05-28T02:33:02.680" Title="Simplifying Log Loss" Tags="&lt;math&gt;&lt;loss-functions&gt;&lt;optimization&gt;&lt;linear-algebra&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21499" PostTypeId="1" CreationDate="2020-05-28T05:54:44.657" Score="1" ViewCount="53" Body="&lt;p&gt;I am trying to implement conditional GAN using GAN-CLS loss as described in paper: &lt;a href=&quot;https://arxiv.org/abs/1605.05396&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1605.05396&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, while training discriminator, I should I have three batches of data:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;[Real_Image, Embeddings] &lt;/li&gt;&#xA;&lt;li&gt;[Generated_Image, Embeddings]&lt;/li&gt;&#xA;&lt;li&gt;[Wrong_Image,Embeddings]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And, while training generator, I should have one batch of data i.e [Generated_Image, Embeddings].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct way to train the model?&lt;/p&gt;&#xA;" OwnerUserId="37412" LastActivityDate="2020-05-28T05:54:44.657" Title="Training Conditional DCGAN with GAN-CLS loss" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;discriminator&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21500" PostTypeId="1" AcceptedAnswerId="22845" CreationDate="2020-05-28T08:07:50.917" Score="2" ViewCount="51" Body="&lt;p&gt;We have different kinds of algorithms to optimize the loss like AdaGrad, SGD + Momentum, etc. Some are more commonly used than the others. In some algorithms, they usually range out before they converge, reach to the steepest slope and find the minima. But some of these algorithms are significantly fast. So my question is that the speed is more of a deciding factor here or the route is important too? Or is it just problem dependent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eL6Hh.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eL6Hh.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of what I mean by the Route.&lt;/p&gt;&#xA;" OwnerUserId="37414" LastEditorUserId="37414" LastEditDate="2020-05-31T05:47:40.760" LastActivityDate="2020-08-03T10:00:13.830" Title="Which one is more important in case of different loss optimization algorithms, Speed or the Route?" Tags="&lt;convolutional-neural-networks&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="21501" PostTypeId="1" CreationDate="2020-05-28T08:42:38.980" Score="1" ViewCount="90" Body="&lt;p&gt;I've found online this interesting algorithm:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/z9f73.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z9f73.png&quot; alt=&quot;r&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand reading this algorithm, I can't figure out why I should &quot;perform the opposite action&quot; and consequently storing that second experience as then it is never used to update or doing anything else. Is this algorithm incorrect?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastActivityDate="2020-05-28T08:42:38.980" Title="Understanding the role of the target network in this DQN algorithm" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21504" PostTypeId="1" AcceptedAnswerId="21505" CreationDate="2020-05-28T11:18:14.543" Score="2" ViewCount="210" Body="&lt;p&gt;How should I decay the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; in Q-learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am decaying epsilon as follows. I initialize &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; to be 1, then, after every episode, I multiply it by some &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; (let it be &lt;span class=&quot;math-container&quot;&gt;$0.999$&lt;/span&gt;), when it reaches &lt;span class=&quot;math-container&quot;&gt;$0.01$&lt;/span&gt;. After that, I keep &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; to be &lt;span class=&quot;math-container&quot;&gt;$0.01$&lt;/span&gt; all the time. I think this has a terrible consequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I need a &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; decay algorithm. I haven't found script or formula about it, so can you tell me?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-28T11:49:31.067" LastActivityDate="2020-05-28T11:49:31.067" Title="How should I decay $\epsilon$ in Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21505" PostTypeId="2" ParentId="21504" CreationDate="2020-05-28T11:31:25.427" Score="4" Body="&lt;p&gt;The way you have described tends to be the common approach. There are of course other ways that you could do this e.g. using an exponential decay, or to only decay after a 'successful' episode, albeit in the latter case I imagine you would want to start with a smaller &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; value and then decay by a larger amount. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-28T11:31:25.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21506" PostTypeId="1" AcceptedAnswerId="21507" CreationDate="2020-05-28T11:37:24.503" Score="1" ViewCount="76" Body="&lt;p&gt;I was reading the paper &lt;a href=&quot;https://www.aaai.org/ojs/index.php/AAAI/article/view/4227&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Combine Tree-Search Methods in Reinforcement Learning&lt;/a&gt; published in AAAI Conference 2019. It starts with the sentence &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is meant by &quot;finite horizon look-ahead&quot;?&lt;/p&gt;&#xA;" OwnerUserId="35679" LastEditorUserId="2444" LastEditDate="2020-05-28T11:48:53.880" LastActivityDate="2020-05-28T11:48:53.880" Title="What are finite horizon look-ahead policies in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21507" PostTypeId="2" ParentId="21506" CreationDate="2020-05-28T11:47:03.580" Score="1" Body="&lt;p&gt;Per &lt;a href=&quot;http://adp.princeton.edu/Papers/Powell_ADP_2ndEdition_Chapter%206.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; a look ahead policy is a policy that will make decisions based on some 'horizon'. Here horizon means some time steps into the future, and so a finite horizon is simply a finite amount of time steps into the future. For example, as we are typically concerned with maximising returns in RL, a 10-step look ahead policy would choose an action at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; that maximises the (expected) rewards at time &lt;span class=&quot;math-container&quot;&gt;$t+1, ... t+10$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-28T11:47:03.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21508" PostTypeId="2" ParentId="21142" CreationDate="2020-05-28T13:15:33.630" Score="2" Body="&lt;p&gt;Artificial Intelligence, as its name suggests, is intelligence made by humans. It's usually thought of as having human-like behaviors and characteristics. However, it doesn't have to resemble humans to be AI. It just has to be made by humans. Many common AI algorithms aren't even made to resemble humans, they may just have similarities. Reinforcement learning is present in humans, but also in the many creatures with intelligence.&lt;/p&gt;&#xA;&lt;p&gt;Swarm Intelligence is basically &lt;em&gt;a lot of small stupid things working together to do something complex&lt;/em&gt;. Take for example ants. Each individual ant only follows a few very simple &amp;quot;instructions&amp;quot; like &lt;code&gt;if has this chemical: follow&lt;/code&gt;. Like Evolutionary AI, it's taking mimicking features of nature. We humans just take a feature made by nature (Evolution/Swarming) and try to replicate some behaviors. Much like Evolutionary AI, Swarm Intelligence is a type of AI.&lt;/p&gt;&#xA;&lt;p&gt;tl;dr:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AI: intelligence made by humans&lt;/li&gt;&#xA;&lt;li&gt;SI: feature made by nature that humans are trying to copy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25130" LastEditorUserId="25130" LastEditDate="2021-01-26T02:41:03.723" LastActivityDate="2021-01-26T02:41:03.723" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21510" PostTypeId="1" CreationDate="2020-05-28T14:22:12.757" Score="1" ViewCount="31" Body="&lt;p&gt;I'm trying to optimize some reflective properties of curves of the form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$a_1x^n+a_2x^{n-1}+a_3x^{n-2} + ... + a_n + b_1y^n+b_2y^{n-1}+b_3y^{n-2} + ... + b_n = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is basically the curve that you get when you sum two polynomials of same degree in different variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) + g(y) = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, I was wondering what would be a good way to do crossover on two such curves? I tried averaging the curves and then mutating them, but the problem is that the entire population quickly becomes homogeneous and the fitness starts to drop. Another typical method I tried is taking a cutoff point somewhere on the expansion above and mixing the left and right part from both parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there are many ways to do a similar process. I could order the above expansion differently and then do a cutoff. I could seperate Xs and Ys and do two seperate cutoffs. Etc. The question is: in the context of algebraic curves, which method of generating offspring would be a good option considering I want to optimize some property of the curve (in this case I want it to have some reflective properties)?&lt;/p&gt;&#xA;" OwnerUserId="37422" LastEditorUserId="2444" LastEditDate="2020-05-29T19:42:36.103" LastActivityDate="2020-05-29T19:42:36.103" Title="How to effectively crossover mathematical curves?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;crossover-operators&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21511" PostTypeId="1" CreationDate="2020-05-28T14:56:25.563" Score="1" ViewCount="77" Body="&lt;p&gt;At every node, MAX would always move to maximise the minimum payoff while MIN choose to minimise the maximum payoff, hence there is nash equilibrium. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using backwards induction, at every node, MAX and MIN player would act optimally. Hence, there is subperfect nash equilibrium.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I formally prove this?   &lt;/p&gt;&#xA;" OwnerUserId="37258" LastEditorUserId="2444" LastEditDate="2020-05-28T15:34:14.623" LastActivityDate="2020-05-28T15:34:14.623" Title="How do you prove that minimax algorithm outputs a subgame-perfect Nash equilibrium?" Tags="&lt;proofs&gt;&lt;game-theory&gt;&lt;minimax&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21512" PostTypeId="1" CreationDate="2020-05-28T14:58:09.460" Score="2" ViewCount="37" Body="&lt;p&gt;I am currently trying to solve a classification task with a recurrent artificial neural network (RNN).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Situation&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt classes).&#xA;The sequence to predict is deterministic in the sense that only specific state transitions are allowed based on the past. A simplified Abstraction of my problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;y - Ground Truth: 01020&lt;/li&gt;&#xA;&lt;li&gt;y - Model Prediction: 01200&lt;/li&gt;&#xA;&lt;li&gt;Valid Transitions: 01, 10, 02, 20&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The predicted transition 12 is consequently not valid.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to optimize a model to make as less invalid transition predictions as possible (ideally none)?&#xA;(temporal shifts of the predictions in comparison to the ground truth are still acceptable)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By the integration of the knowledge about the valid transitions into the artificial neural network. Is this even possible to code hard restrictions into a RNN?&lt;/li&gt;&#xA;&lt;li&gt;By a custom loss function which penalizes these invalid transitions &lt;/li&gt;&#xA;&lt;li&gt;Another approach&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Current Approach&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;With a bidirectional recurrent network (one gated recurrent units layer (2000 neurons)) in a many to many fashion an accuracy of 99.5% on the training and 97.5% on the test set could be reached (Implemented with TensorFlow / keras 2.2)&lt;/p&gt;&#xA;" OwnerUserId="34155" LastActivityDate="2020-05-28T14:58:09.460" Title="Incorporating domain knowledge into recurrent network" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;loss-functions&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21514" PostTypeId="1" AcceptedAnswerId="21527" CreationDate="2020-05-28T15:55:34.237" Score="2" ViewCount="65" Body="&lt;p&gt;There is some sort of art to using the right loss function. However, I was wondering if there is a way to derive the loss function if I gave you a neural network model (the weights) as well as the training data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of this exercise is to see what family of loss functions we would get. And how that compares to the loss function that actually gave rise to the model.&lt;/p&gt;&#xA;" OwnerUserId="37423" LastEditorUserId="2444" LastEditDate="2020-05-29T21:01:14.067" LastActivityDate="2020-05-29T21:31:10.373" Title="Is there a way of deriving a loss function given the neural network and training data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21515" PostTypeId="1" AcceptedAnswerId="21517" CreationDate="2020-05-28T15:55:49.123" Score="4" ViewCount="607" Body="&lt;p&gt;I am new in reinforcement learning, but I already know deep Q-learning and Q-learning. Now, I want to learn about double deep Q-learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any good references for double deep Q-learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read some articles, but some of them don't mention what the loss is and how to calculate it, so many articles are not complete. Also, Sutton and Barto (in their book) don't describe that algorithm either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, help me to learn Double Q-learning.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-28T16:06:36.353" LastActivityDate="2020-05-28T20:09:08.107" Title="Is there any good reference for double deep Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;reference-request&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21516" PostTypeId="2" ParentId="21515" CreationDate="2020-05-28T16:27:59.663" Score="3" Body="&lt;p&gt;You should first read the introductory paper of Double DQN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1509.06461&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1509.06461&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, depending on what you would like to do, search for other relevant papers that use this method.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-05-28T19:38:23.367" LastActivityDate="2020-05-28T19:38:23.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21517" PostTypeId="2" ParentId="21515" CreationDate="2020-05-28T18:35:07.413" Score="5" Body="&lt;p&gt;If you're interested in the theory behind Double Q-learning (&lt;strong&gt;&lt;em&gt;not deep!&lt;/em&gt;&lt;/strong&gt;), the reference paper would be &lt;a href=&quot;https://papers.nips.cc/paper/3964-double-q-learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Double Q-learning&lt;/a&gt; by Hado van Hasselt (2010). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for Double &lt;strong&gt;&lt;em&gt;deep&lt;/em&gt;&lt;/strong&gt; Q-learning (also called DDQN, short for Double Deep Q-networks), the reference paper would be &lt;a href=&quot;https://arxiv.org/abs/1509.06461&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning with Double Q-learning&lt;/a&gt; by Van Hasselt et al. (2016), as pointed out in&#xA; &lt;a href=&quot;https://ai.stackexchange.com/a/21516/34010&quot;&gt;ddaedalus's answer&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how the loss is calculated, it is not explicitly written in the paper. But, you can find it in the &lt;a href=&quot;https://arxiv.org/pdf/1511.06581.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dueling DQN paper&lt;/a&gt;, which is a subsequent paper where Van Hasselt is a coauthor. In the appendix, the authors provide the pseudocode for Double DQN. The relevant part for you would be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y_{j}=\left\{\begin{array}{ll}r &amp;amp; \text { if } s^{\prime} \text { is terminal } \\ r+\gamma Q\left(s^{\prime}, a^{\max }\left(s^{\prime} ; \theta\right) ; \theta^{-}\right), &amp;amp; \text {otherwise}\end{array}\right.$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Do a gradient descent step with loss &lt;span class=&quot;math-container&quot;&gt;$ \left\|y_{j}-Q(s, a ; \theta)\right\|^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$y_j$&lt;/span&gt; is the target, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the parameters of the regular network and &lt;span class=&quot;math-container&quot;&gt;$\theta^{-}$&lt;/span&gt; are the target network parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important thing to note here is the difference with the DQN target:&#xA;&lt;span class=&quot;math-container&quot;&gt;$y_{i}^{D Q N}=r+\gamma \max _{a^{\prime}} Q\left(s^{\prime}, a^{\prime} ; \theta^{-}\right)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In DQN, we evaluate the Q-values based on parameters &lt;span class=&quot;math-container&quot;&gt;$\theta^{-}$&lt;/span&gt; and we take the max over actions based on these Q-values parametrized with the &lt;strong&gt;same&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$\theta^{-}$&lt;/span&gt;. The problem with this is that it leads to an overestimation bias, especially at the beginning of the training process, where the Q-values estimates are noisy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to address this issue, in double DQN, we instead take the max based on Q-values calculated using &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and we evaluate the Q-value of &lt;span class=&quot;math-container&quot;&gt;$a^{max}$&lt;/span&gt; based on a different set of parameters i.e. &lt;span class=&quot;math-container&quot;&gt;$\theta^{-}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to learn more about this, by watching a video lecture instead of reading a paper, I'd suggest you take a look at &lt;a href=&quot;https://www.youtube.com/watch?v=7Lwf-BoIu3M&amp;amp;t=39m20s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; lecture from UC Berkley's DRL course, where the professor (Sergey Levine) discusses this in detail with examples. &lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="34010" LastEditDate="2020-05-28T20:09:08.107" LastActivityDate="2020-05-28T20:09:08.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21518" PostTypeId="1" CreationDate="2020-05-28T19:47:50.697" Score="1" ViewCount="28" Body="&lt;p&gt;I have a question about the use of deep learning techniques with time-fixed features and images (setting 1) and time-dependent features (setting 2). (I am pretty new to the deep learning world so please excuse me if it's a basic question.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Setting 1:&lt;/em&gt;&#xA;Imagine having a training dataset composed of &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;some time-fixed features such as height, weight, and age of an individual at the first medical visit (these features are recorded once and therefore time-fixed, i.e., they do not change in time in the dataset). &lt;/li&gt;&#xA;&lt;li&gt;some medical images for each individual, such as for example a CT scan. &lt;/li&gt;&#xA;&lt;li&gt;a label defining if the patient has or not a specific disease.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Setting 2:&lt;/em&gt; &#xA;Same as setting 1 but with some features that are repeated over time (time-dependent, longitudinal), such as for example blood pressure recorded twice a day for each individual for several days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let say that the goal is to classify if an individual has or not a specific disease given the aforementioned features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen zillions of papers and blogs talking about convolutional neural networks to classify images and a few million about recurrent neural networks for time-dependent features. However, I am not very aware of what to use in case I have time-fixed, time-dependent, and imaging features altogether. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering how you would attach this problem.&lt;/p&gt;&#xA;" OwnerUserId="37426" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:17.793" LastActivityDate="2020-05-29T13:48:17.793" Title="Deep learning techniques with time-fixed, time-dependent and imaging data" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;structured-data&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21519" PostTypeId="1" CreationDate="2020-05-28T19:56:03.337" Score="2" ViewCount="22" Body="&lt;p&gt;I have a course assignment to use an LSTM to predict the movement directions of stock prices. One of the things I am asked to do is provide a visualization to compare the predictive powers of a set of N features (e.g. 1-day return, volatility, moving average, etc.). Let's assume that we use a window of 50 days as input to the LSTM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing that came to my mind is to use a RadViz plot (check below image taken from &lt;a href=&quot;https://www.scikit-yb.org/en/latest/api/features/radviz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scikit-yb.org/en/latest/api/features/radviz.html&lt;/a&gt;). &lt;a href=&quot;https://i.stack.imgur.com/uwr0r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uwr0r.png&quot; alt=&quot;RadViz for 5 features&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I soon realized this will not work for the features since each sample will have 50 values. So if we have M samples, the shape of the input data will be something like Mx50xN. This, unfortunately, is not something RadViz can deal with (it can handle 2D data). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this, I would be grateful if someone can point me to a viable way to visualize the data. Is it even possible when each feature comprises 50 values?&lt;/p&gt;&#xA;" OwnerUserId="37428" LastEditorUserId="30725" LastEditDate="2020-05-29T13:48:03.087" LastActivityDate="2020-05-29T13:48:03.087" Title="Visualisation for Features to Predict Timeseries Data" Tags="&lt;long-short-term-memory&gt;&lt;feature-selection&gt;&lt;feature-extraction&gt;&lt;feature-engineering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21521" PostTypeId="1" CreationDate="2020-05-29T10:07:16.630" Score="1" ViewCount="73" Body="&lt;p&gt;This GAN being trained with CelebA dataset doesn't seem to mode collapse, discriminator is not really over confident, and yet the quality is stuck on these rough Picasso-like generator images. Using Leaky-ReLU, strided conv instead of maxpool, and dampened truths helped a little, but still no better than this. Not sure what else to try. &lt;a href=&quot;https://www.youtube.com/watch?v=Eb007xdmM6Y&quot; rel=&quot;nofollow noreferrer&quot;&gt;training clip&lt;/a&gt; Discriminator feed is in top left corner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7OT0c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7OT0c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37443" LastActivityDate="2020-05-29T10:07:16.630" Title="Why is this GAN not converging?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21522" PostTypeId="2" ParentId="11679" CreationDate="2020-05-29T10:12:30.050" Score="2" Body="&lt;p&gt;There are three problems&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Limited capacity Neural Network (explained by John)&lt;/li&gt;&#xA;&lt;li&gt;Non-stationary Target&lt;/li&gt;&#xA;&lt;li&gt;Non-stationary distribution&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h1&gt;Non-stationary Target&lt;/h1&gt;&#xA;&lt;p&gt;In tabular Q-learning, when we update a Q-value, other Q-values in the table don't get affected by this. But in neural networks, one update to the weights aiming to alter one Q-value ends up affecting other Q-values whose states look similar (since neural networks learn a continuous function that is smooth)&lt;/p&gt;&#xA;&lt;p&gt;This is bad because when you are playing a game, two consecutive states of a game are always similar. Therefore, Q-value updates will increase or decrease Q-values for both states together. So, when you take one as the target for the other, the target becomes non-stationary since it moves along with you. This is analogous to a donkey running to catch a carrot that is attached to its head. Since the target is non-stationary, the donkey will never reach its target. And, in our case, in trying to chase, the Q-values will explode.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://daiwk.github.io/assets/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human-level control through deep reinforcement learning&lt;/a&gt;, this problem is addressed by caching an OLD copy of the DQN for evaluating the targets, &amp;amp; updating the cache every 100,000 steps of learning. This is called a &lt;em&gt;target network&lt;/em&gt;, and the targets remain stationary this way.&lt;/p&gt;&#xA;&lt;h1&gt;Non-stationary distribution&lt;/h1&gt;&#xA;&lt;p&gt;This is analogous to the &amp;quot;distribution drift&amp;quot; problem in imitation learning, which can be solved with the dataset aggregation technique called &lt;a href=&quot;https://www.ri.cmu.edu/pub_files/2011/4/Ross-AISTATS11-NoRegret.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAgger&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The idea is, as we train, our DQN gets better and better and our policy improves. And this causes our sampling distribution to change since we are doing online learning where we sample according to a policy with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; probability. This is a problem for supervised learning since it assumes stationary distribution or i.i.d. data.&lt;/p&gt;&#xA;&lt;p&gt;As an analogy, this is like training a Neural Network to identify cats and dogs but showing the network only dogs during the first 100 epochs, and then showing only cats for the remainder epochs. What happens is, the network learns to identify dogs, then forgets it and learns to identify cats.&lt;/p&gt;&#xA;&lt;p&gt;This is what happens when the distribution changes and we care only about the current distribution during training. So, in order to solve this,  &lt;a href=&quot;https://daiwk.github.io/assets/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;same paper&lt;/a&gt; starts aggregating data in a large buffer, and samples a mini-batch of both new data as well as old data every time during training. This is called &lt;em&gt;experience replay&lt;/em&gt;, since we don't throw away our past experience and keep re-using them in training.&lt;/p&gt;&#xA;" OwnerUserId="17954" LastEditorUserId="2444" LastEditDate="2020-12-19T13:14:29.677" LastActivityDate="2020-12-19T13:14:29.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21523" PostTypeId="1" AcceptedAnswerId="21524" CreationDate="2020-05-29T11:47:02.403" Score="3" ViewCount="49" Body="&lt;p&gt;In &lt;a href=&quot;https://youtu.be/ItMutbeOHtc?t=1542&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Silver's 8th lecture&lt;/a&gt; he talks about model learning and says that learning &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; is a regression problem whereas learning &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; is a kernel density estimation. His explanation for the difference is that if we are in a stochastic environment and we are in the tuple &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; then there might be a 30% chance the wind will blow me left, and a 70% chance the wind will blow me right, so we want to estimate these probabilities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the main difference between these two problems, and hence why one is regression and the other is kernel density estimation, because with the reward we are mainly concerned with the &lt;em&gt;expected&lt;/em&gt; reward (hence regression) whereas with the state transitioning, we want to be able to simulate this so we need the estimated density? &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-05-29T12:35:51.867" Title="Why is learning $s'$ from $s,a$ a kernel density estimation problem but learning $r$ from $s,a$ is just regression?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;model-based-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21524" PostTypeId="2" ParentId="21523" CreationDate="2020-05-29T12:14:17.047" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the main difference between these two problems, and hence why one is regression and the other is kernel density estimation, because with the reward we are mainly concerned with the expected reward (hence regression) whereas with the state transitioning, we want to be able to simulate this so we need the estimated density?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An expected reward function from &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; is all you need to construct valid Bellman equations for value functions. For example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) = r(s,a) + \gamma\sum_{s'}p(s'|s,a)\sum_{a'}\pi(a'|s')q(s',a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a valid way of writing the Bellman equation for action values. You can derive this from &lt;span class=&quot;math-container&quot;&gt;$r(s,a) = \sum_{r,s'}rp(r,s'|s,a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a) = \sum_{r,s'}p(r,s'|s,a)(r + \gamma\sum_{a'}\pi(a'|s')q(s',a'))$&lt;/span&gt; if you have the equations in that form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in general there is no such thing as an &quot;expected state&quot; when there is more than one possible outcome (i.e. in environments with stochastic state transitions). You can take a mean of the state vector representations over the samples you see for &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; but that is not the same thing at all and could easily be a representation of an unreachable/nonsense state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, the expectation &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}[x(S_{t+1})|S_t=s, A_t=a]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x(s)$&lt;/span&gt; creates a feature vector from any given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x(s): \mathcal{S} \rightarrow \mathbb{R}^d$&lt;/span&gt;, can be meaningful. The broadest and most trivial example of this is for deterministic environments. You may be able to construct stochastic environments where there is a good interpretation of such a vector, even if it does not represent any reachable state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple one-hot encoded states could maybe made to work like this by representing a probability distribution over states (this would also require re-interpretations of expected reward function and value functions). That is effectively a kernel density function over discrete state space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general knowing this &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}[x(S_{t+1})|S_t=s, A_t=a]$&lt;/span&gt; expected value does not help resolve future rewards, as they can depend arbitrarily on specific state transitions.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-05-29T12:35:51.867" LastActivityDate="2020-05-29T12:35:51.867" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21526" PostTypeId="1" CreationDate="2020-05-29T17:21:55.250" Score="3" ViewCount="42" Body="&lt;p&gt;There’s a lot of talk of undercover cops intentionally starting violence in otherwise peaceful protests. The evidence, primarily, are images like this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://images.app.goo.gl/4n3o2EXwFzMQfsKq6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://images.app.goo.gl/4n3o2EXwFzMQfsKq6&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks pretty convincing, but I’d like something more solid. Does anyone know of a model that can detect with a high level of certainty if the “mask” area of two photos represents the same person?&lt;/p&gt;&#xA;" OwnerUserId="37455" LastActivityDate="2020-05-29T17:21:55.250" Title="Does anyone know of a model for comparing the eyes of people in two images to see if they match?" Tags="&lt;image-recognition&gt;&lt;pattern-recognition&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21527" PostTypeId="2" ParentId="21514" CreationDate="2020-05-29T21:25:24.597" Score="1" Body="&lt;p&gt;I don't think there's a way of doing what you want, at least, I've never seen such a thing (and, currently, I am not seeing how this could be done in the general case). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same neural network model but with different (or same) weights could have been trained with the same loss function or not. For example, although it may not be a good idea, you can train a neural network for classification with the mean squared error, as opposed to the typical cross-entropy. Moreover, even if you know the loss function that the neural network is trained with, the training data alone may not lead to the same set of weights because the actual weights depend on different (possibly stochastic) factors, such as if (or how) you shuffle the data or the batch size.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-05-29T21:31:10.373" LastActivityDate="2020-05-29T21:31:10.373" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21528" PostTypeId="2" ParentId="5840" CreationDate="2020-05-29T22:32:38.743" Score="0" Body="&lt;p&gt;Note that &quot;algorithmically&quot; can refer to anything that uses an &lt;em&gt;algorithm&lt;/em&gt;. Currently, ML systems are trained with algorithms and neural networks can be seen as algorithms (although black-box ones), so ML is also algorithmic. Everything that runs on a computer (a concrete version of a Turing machine) can be seen as an algorithm (or program)! In fact, computers were invented exactly for this purpose: to perform some algorithmic operation (i.e. a set of instructions, like a recipe). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, by &lt;em&gt;algorithmic&lt;/em&gt;, I assume you're referring to techniques that are typically taught in an &quot;Algorithms and Data Structures&quot; course for a computer science student, such as the binary search (one of the most simple and yet beautiful and useful algorithms!), which is an algorithm that, given some constraints (a sorted array), gives you an exact correct solution in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\log n)$&lt;/span&gt; time. However, I think that you are also referring to every program that is primarily based on if-then statements and loops (e.g. desktop applications, websites, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, you first need to understand the scope of the machine learning field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning (like statistics) is a set of techniques that attempt to learn from data. So, every problem where data is available (and you can get insight from) can potentially be solved with a machine learning technique.  ML techniques typically produce approximative solutions and are typically used to solve problems where an exact solution is infeasible. However, note that machine learning isn't the only approach to solve hard problems (e.g. you can also use meta-heuristics, e.g. ant colony optimization algorithms).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an algorithm that produces an exact solution (without requiring data) in polynomial time (preferably, in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(n^2)$&lt;/span&gt; time), then machine learning (or any other technique that produces approximative solutions, e.g. heuristics) is quite useless. &lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-05-29T22:32:38.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21530" PostTypeId="1" CreationDate="2020-05-30T09:59:10.720" Score="1" ViewCount="162" Body="&lt;p&gt;What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of a pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nZDAE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nZDAE.png&quot; alt=&quot;example pattern&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37467" LastEditorUserId="2444" LastEditDate="2020-05-30T11:05:49.667" LastActivityDate="2020-05-30T11:05:49.667" Title="What is an auto-associator?" Tags="&lt;neural-networks&gt;&lt;math&gt;&lt;hebbian-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21531" PostTypeId="1" AcceptedAnswerId="21545" CreationDate="2020-05-30T10:08:30.587" Score="2" ViewCount="200" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The aim of weight initialization is to prevent layer activation outputs from exploding or vanishing during the course of a forward pass through a deep neural network&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am really having trouble understanding weights initialization technique and Xavier Initialization for deep neural networks (DNNs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simple words (and maybe with an example), what is the intuition behind the Xavier initialization for DNNs? When should we use Xavier's initialization?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-05-30T11:38:43.940" LastActivityDate="2020-05-31T15:45:54.997" Title="What is the intuition behind the Xavier initialization for deep neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21532" PostTypeId="1" AcceptedAnswerId="21534" CreationDate="2020-05-30T10:16:56.763" Score="3" ViewCount="275" Body="&lt;p&gt;I know how pooling works, and what effect it has on the input dimensions - but I'm not sure why it's done in the first place. It'd be great if someone could provide some intuition behind it - while explaining the following excerpt from a blog:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A problem with the output feature maps is that they are sensitive to the location of the features in the input. One approach to address this sensitivity is to down sample the feature maps. This has the effect of making the resulting down sampled feature maps more robust to changes in the position of the feature in the image, referred to by the technical phrase “&lt;em&gt;local translation invariance&lt;/em&gt;.”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What's local translation invariance here?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-05-30T11:12:43.903" LastActivityDate="2020-05-31T10:08:57.843" Title="What is the effect of using pooling layers in CNNs?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21533" PostTypeId="1" AcceptedAnswerId="21543" CreationDate="2020-05-30T12:47:52.217" Score="1" ViewCount="67" Body="&lt;p&gt;From the &lt;a href=&quot;https://www.youtube.com/watch?v=ItMutbeOHtc&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Silver's lecture 8: Integrating Learning and Planning&lt;/a&gt; - based on Sutton and Barto - he talks about using sample-based planning to use our model to take a sample of a state and then use model-free planning, such as Monte Carlo, etc, to run the trajectory and observe the reward. He goes on to say that this effectively gives us infinite data from only a few actual experiences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we only experience a handful of true state-action-rewards and then start sampling to learn more then we will surely end up with a skewed result, e.g., If I have 5 experiences but then create 10000 samples (as he says, infinite data). I am aware that as the experience set grows the Central Limit Theorem will come into play and the distribution of experience will more accurately represent the true environment's state-actions-rewards distribution but before this happens is sampled based planning still useful?&lt;/p&gt;&#xA;" OwnerUserId="36082" LastEditorUserId="2444" LastEditDate="2020-06-01T14:14:16.473" LastActivityDate="2020-06-01T14:38:38.287" Title="Is the distribution of state-action pairs from sample based planning accurate for small experience sets?" Tags="&lt;reinforcement-learning&gt;&lt;planning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21534" PostTypeId="2" ParentId="21532" CreationDate="2020-05-30T16:04:22.527" Score="5" Body="&lt;p&gt;Pooling has multiple benefits&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Robust feature detection.&lt;/li&gt;&#xA;&lt;li&gt;Makes it computationally feasible to have deeper CNNs&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Robust Feature Detection&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Think of max-pooling (most popular) for understanding this.&#xA; Consider a 2*2 box/unit in one layer which is mapped to only 1 box/unit in the next layer (Basically pooling). Let's say the feature map (kernel) detects a petal of a flower. Then qualifying a petal if any of the 4 units of the previous layer is fired makes the detection robust to noise. There is no strict requirement that all 4 units should be fired to detect a petal. Thus, the next layer (after pooling) captures the features with &lt;strong&gt;noise invariance&lt;/strong&gt;. We can also say it is &lt;strong&gt;local translation invariance&lt;/strong&gt; (in a close spatial sense) as a shifted feature will also be captured. But also remember &lt;strong&gt;translation invariance&lt;/strong&gt; in general is captured by the convolution with kernels in the first place. (See how 1 kernel is convolved with the whole image)&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Computational advantage&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The dimensions of the inputs in image classification are so huge that the number of the multiplication operation is in billions even with very few layers. Pooling the output layer reduces the input dimension for the next layer thus saving computation. But also now one can aim for really deep networks(number of layers) with the same complexity as before. &lt;/p&gt;&#xA;" OwnerUserId="26489" LastEditorUserId="35585" LastEditDate="2020-05-31T10:08:57.843" LastActivityDate="2020-05-31T10:08:57.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21535" PostTypeId="2" ParentId="21532" CreationDate="2020-05-30T17:32:33.337" Score="0" Body="&lt;p&gt;In addition in general it somewhat aides in detection as only the strongest feature feature filter is activated so in a sense it removes additional information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it obviously has draw backs resulting in combinations of features being detected which aren't actual.objects.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-05-30T17:32:33.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21536" PostTypeId="1" AcceptedAnswerId="22113" CreationDate="2020-05-30T23:50:37.257" Score="0" ViewCount="64" Body="&lt;p&gt;Here's the code in question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L491&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L491&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class BertOnlyNSPHead(nn.Module):&#xA;    def __init__(self, config):&#xA;        super().__init__()&#xA;        self.seq_relationship = nn.Linear(config.hidden_size, 2)&#xA;&#xA;    def forward(self, pooled_output):&#xA;        seq_relationship_score = self.seq_relationship(pooled_output)&#xA;        return seq_relationship_score&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think it was just ranking how likely one sentence would follow another? Wouldn't it be one score?&lt;/p&gt;&#xA;" OwnerUserId="18358" LastActivityDate="2020-06-23T14:38:55.440" Title="Why does the BERT NSP head linear layer have two outputs?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21537" PostTypeId="1" CreationDate="2020-05-31T07:39:33.603" Score="1" ViewCount="46" Body="&lt;p&gt;I am learning to use a LSTM model to predict time series data. Specifically, I hope the network should output a sequence (with multiple time steps) only after the input sequence has finished feeding in, as shown in the left figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mnD8D.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mnD8D.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, most of the LSTM sequence-to-sequence prediction tutorial I have read seems to be the right figure (i.e. each time step of the output sequence is generated after each time step of the input sequence). What's more, as far as I understand, the LSTM implementation in PyTorch (and probably Keras) can only return output sequence corresponding to each time step of the input sequence. It cannot make predictions after the input sequence is over. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope to know is there any way to make a sequence-to-sequence LSTM network which starts output only after the input sequence has finished feeding in? And it would be better if someone can show me an example implementation code.&lt;/p&gt;&#xA;" OwnerUserId="37482" LastActivityDate="2020-07-01T11:02:40.260" Title="How to make a LSTM network to predict sequence only after input sequence is finished?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21538" PostTypeId="1" AcceptedAnswerId="21539" CreationDate="2020-05-31T08:56:00.517" Score="1" ViewCount="74" Body="&lt;p&gt;I'm working on a deep q-learning model in an infinite horizon problem, with a continous state space and 3 possible actions. I'm using a neural network to approximate the action-value function. Sometimes it happens that, after a few steps, the algorithm starts choosing only one between the possible actions (apart from a few steps where I suppose it explores, given the epsilon-greedy policy it follows), leading to bad results in terms of cumulative rewards. Is this a sign that the algorithm diverged? &lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-05-31T11:13:10.350" LastActivityDate="2020-05-31T11:13:10.350" Title="If deep Q-learning starts to choose only one action, is this a sign that the algorithm diverged?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21539" PostTypeId="2" ParentId="21538" CreationDate="2020-05-31T09:48:25.073" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is this a sign that the algorithm diverged?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is a common sign of a problem with learning process. That includes divergence due to poor hyper-parameters, even just bad luck. But it can also point to a design/architecture problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other common causes of algorithm failing with a fixed action choice include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Neural network inputs not scaled before use.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Large reward values causing large initial squared errors (either re-scale rewards or reduce learning rate to fix)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;State representation too far from &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov property&lt;/a&gt; assumption&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A bug in code (almost anywhere, unfortunately)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Catastrophic_interference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Catastrophic forgetting&lt;/a&gt; due to focusing too much on specific results and generalising from them incorrectly. Your agent might be suffering from this if it starts to learn correctly, reaching some level of competence at the task before failing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-05-31T09:48:25.073" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21540" PostTypeId="1" CreationDate="2020-05-31T09:56:07.290" Score="3" ViewCount="103" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;SGD is able to jump out of local minima that would otherwise trap BGD&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't really understand the above statement. Could someone please provide a &lt;em&gt;mathematical explanation&lt;/em&gt; for &lt;strong&gt;why SGD (Stochastic Gradient Descent) is able to escape local minima, while BGD (Batch Gradient Descent) can't&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While searching online, I read that it has something to do with &quot;oscillations&quot; while taking steps towards the global minima. What's that?  &lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-06-02T15:41:50.857" LastActivityDate="2020-06-02T15:41:50.857" Title="How does SGD escape local minima?" Tags="&lt;optimization&gt;&lt;gradient-descent&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="0" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21542" PostTypeId="1" CreationDate="2020-05-31T10:22:47.387" Score="2" ViewCount="74" Body="&lt;p&gt;I have recently solved the Cartpole problem using double deep Q-learning. When I saw how the agent was doing, it used to go right every time, never left, and it did similar actions all the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did the model overfit the environment? It seems that the agent just memorized the environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the common techniques to prevent the agent to overfit like that? Is that a common problem?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-05-31T11:16:34.203" LastActivityDate="2020-05-31T11:16:34.203" Title="How to prevent deep Q-learning algorithms to overfit?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21543" PostTypeId="2" ParentId="21533" CreationDate="2020-05-31T11:23:22.013" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I am aware that as the experience set grows the Central Limit Theorem&#xA;  will come into play and the distribution of experience will more&#xA;  accurately represent the true environment's state-actions-rewards&#xA;  distribution&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe here you mean the &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_large_numbers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Law of Large Numbers&lt;/a&gt; which states that for a large enough sample (&lt;span class=&quot;math-container&quot;&gt;$n \rightarrow \infty$&lt;/span&gt;) the sample mean will converge to the true mean. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Central_limit_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;central limit theorem (CLT)&lt;/a&gt; states that if you take the sum/mean of a set of &lt;em&gt;independent&lt;/em&gt; random variables then the distribution of this new RV will be approximately normal as &lt;span class=&quot;math-container&quot;&gt;$n \rightarrow \infty$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;before this happens is sampled based planning still useful&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you mentioned, if you had only 5 full episodes of experience to choose from, then this likely would not represent true underlying distributions and so the approximations would not be good -- of course, this will depend on how complex your MDP is, if you had a trivially simple one then it may be enough to represent it well. As David Silver says in his lecture, then one of the disadvantages of planning with a model is that you introduce another set of uncertainty, mainly from when you approximate the properties of the model. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-01T14:38:38.287" LastActivityDate="2020-06-01T14:38:38.287" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21545" PostTypeId="2" ParentId="21531" CreationDate="2020-05-31T15:45:54.997" Score="1" Body="&lt;p&gt;Weight initialization is one of the most critical factors for successfully training a deep neural network. This &lt;a href=&quot;https://www.deeplearning.ai/ai-notes/initialization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;explanation&lt;/a&gt; by deeplearning.ai is probably the best that one could give for the need for initializing a DNN with Xavier initialization. Here is what it talks about in a nutshell:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem of exploding and vanishing gradients has been long-standing in the DL community. Initialize all the weights as zeros and the model learns identical features across all hidden layers, initialize random but large weights and the backpropagated gradients explode, initialize random but small weights and gradients vanish. The intuition is aptly captured by this simple mathematical observation: &lt;span class=&quot;math-container&quot;&gt;$1.1^{50} = 117.390$&lt;/span&gt;, while at the same time, &lt;span class=&quot;math-container&quot;&gt;$0.9^{50} = 0.00515$&lt;/span&gt;. Note that the difference between the two numbers is just &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; but it has a tremendous effect when multiplied repeatedly! A typical NN is a series of function compositions involving weight matrices and linear/non-linear activation functions. When stripped to a bare minimum, it essentially is a series of matrix multiplications. Therefore, the way in which the elements of these weight matrices are initialized plays a major role in how the network learns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard weight initialization methods come into the picture here. They reinforce what are the de-facto rules of thumb when it comes to weight initializations: (1) the mean of the activations should be zero, and (2) the variance of these activations across all the layers should be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The link given above has complete mathematical justification for why Xavier initialization works, along with an interactive visualization for the same.    &lt;/p&gt;&#xA;" OwnerUserId="36971" LastActivityDate="2020-05-31T15:45:54.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21546" PostTypeId="1" CreationDate="2020-05-31T17:46:18.967" Score="1" ViewCount="72" Body="&lt;p&gt;I've &lt;a href=&quot;https://github.com/Stephanehk/Reinforcement_Algorithm_Implementations/blob/master/actor_critic_lunarlander_NN.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;implemented a vanilla actor-critic&lt;/a&gt; and have run into a wall. My model does not seem to be learning the optimal policy. The red graph below shows its performance in cartpole, where the algorithm occasionally does better than random but for the most part lasts between 10-30 timesteps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cWCP1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cWCP1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am reasonably sure that the critic part of the algorithm is working. Below is a graph of the delta value (r + Q_w(s',a') - Q_w(s,a)), which seems to show that for the most part the predicted future reward is quite similar to the approximate future reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iHUQn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iHUQn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I am at a loss for what the problem could be. I have an inkling it lies within the actor, but I am not sure. I have double checked the loss function and that seems to be correct to me as well. I would appreciate any advice. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="36404" LastActivityDate="2020-05-31T17:46:18.967" Title="Actor-Critic implementation not learning" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21547" PostTypeId="1" CreationDate="2020-05-31T23:01:53.473" Score="1" ViewCount="43" Body="&lt;p&gt;Considering the paper &lt;a href=&quot;https://arxiv.org/abs/1802.05983&quot; rel=&quot;nofollow noreferrer&quot;&gt;Disentangling by Factorising&lt;/a&gt;, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution provided by the paper? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qvcTU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qvcTU.png&quot; alt=&quot;FactorVAE Architecture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="2444" LastEditDate="2020-06-01T14:37:22.893" LastActivityDate="2020-06-01T14:37:22.893" Title="What is the main contribution of the paper Disentangling by Factorising?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;variational-autoencoder&gt;&lt;disentangled-representation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21548" PostTypeId="2" ParentId="21547" CreationDate="2020-05-31T23:01:53.473" Score="1" Body="&lt;p&gt;One of the core contributions presented in the paper consists of understanding at a deeper level the objective function used in Beta VAE and improving it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, the authors started from Beta VAE OF&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{N} \sum_{i=1}^{N}\left[\mathbb{E}_{q\left(z | x^{(i)}\right)}\left[\log p\left(x^{(i)} | z\right)\right]-\beta K L\left(q\left(z | x^{(i)}\right) \| p(z)\right)\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which consists of the classical reconstruction loss and the regularization term which steers the Latent Code PDF towards a target PDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing the second term&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{p_{\text{data}}(x)}[K L(q(z | x) \| p(z))]=I(x ; z)+K L(q(z) \| p(z))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;they observed there are 2 terms which push in different directions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$I(x,z)$&lt;/span&gt; is the mutual information between the input data and the code, and we do not want to penalize this as in fact we want this to be as high as possible, so that the decode contains as much information as possible about the input&#xA;&lt;span class=&quot;math-container&quot;&gt;$KL(q(z) || p(z))$&lt;/span&gt; is the KL divergence term which if penalized pushes the latent code PDF &lt;span class=&quot;math-container&quot;&gt;$q(z)$&lt;/span&gt; towards the prior &lt;span class=&quot;math-container&quot;&gt;$p(z)$&lt;/span&gt; which we choose as factorized&#xA;So penalizing the second term in Beta VAE OF means also penalizing the mutual information hence ultimately loosing reconstruction capability&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors then propose a new OF which fixes this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{N} \sum_{i=1}^{N}\left[\mathbb{E}_{q\left(z | x^{(i)}\right)}\left[\log p\left(x^{(i)} | z\right)\right]-K L\left(q\left(z | x^{(i)}\right)|| p(z)\right)\right] -\gamma K L(q(z) \| \bar{q}(z))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the rest of the paper they run experiments and show they have improved the disentanglement of the representation (according to the new metric they propose in the paper) while preserving reconstruction hence improving the reconstruction vs disentanglement tradeoff with respect to the Beta VAE paper&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="2444" LastEditDate="2020-06-01T14:36:25.597" LastActivityDate="2020-06-01T14:36:25.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21550" PostTypeId="1" CreationDate="2020-06-01T04:03:40.193" Score="0" ViewCount="51" Body="&lt;p&gt;Is there any difference between &lt;em&gt;bounded sum&lt;/em&gt; and &lt;em&gt;bold union&lt;/em&gt; fuzzy set operations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about the difference between &lt;em&gt;bounded difference&lt;/em&gt; and &lt;em&gt;bold intersection&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some books, I found no difference, and, in others, there is a difference.&lt;/p&gt;&#xA;" OwnerUserId="22322" LastEditorUserId="2444" LastEditDate="2020-06-01T11:00:12.400" LastActivityDate="2020-06-01T11:00:12.400" Title="Is there any difference between bounded sum and bold union fuzzy set operations?" Tags="&lt;comparison&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21552" PostTypeId="2" ParentId="17324" CreationDate="2020-06-01T06:20:25.060" Score="1" Body="&lt;p&gt;From mathematical point of view you are correct as are your calculations. To catch all the patterns you need that many filters, but this is where a whole idea of a &lt;strong&gt;training&lt;/strong&gt; comes in. Main objective of the training in the CNNs is to find just a few good patterns from billions possible ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the direct answer to your question is: The standard layers of 64 to 1024 filters are only able to catch a small part of (perhaps) useful patterns, yes but this is assuming &lt;strong&gt;no training&lt;/strong&gt; taking place. If you conducted training on given data with given model, then 64 to 1024 filters could already extract a lot of useful patterns, perhaps more than needed. &lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2020-06-01T06:20:25.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21553" PostTypeId="1" AcceptedAnswerId="21555" CreationDate="2020-06-01T08:40:58.573" Score="1" ViewCount="112" Body="&lt;p&gt;My understanding of tabular Q-learning is that it essentially builds a dictionary of state-action pairs, so as to maximize the Markovian (i.e., step-wise, history-agnostic?) reward. This incremental update of the Q-table can be done by a trade-off exploration and exploitation, but the fact remains that one &quot;walks around&quot; the table until it converges to optimality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if we haven't &quot;walked around&quot; the whole table? Can the algorithm still perform well in those out-of-sample state-action pairs? &lt;/p&gt;&#xA;" OwnerUserId="30959" LastEditorUserId="2444" LastEditDate="2020-06-02T15:24:19.223" LastActivityDate="2020-06-02T17:36:34.637" Title="Can tabular Q-learning converge even if it doesn't explore all state-action pairs?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21554" PostTypeId="1" AcceptedAnswerId="21557" CreationDate="2020-06-01T09:23:27.983" Score="1" ViewCount="234" Body="&lt;p&gt;I have encountered the gym environment and decided to create AI that plays breakout. Here is the link: &lt;a href=&quot;https://gym.openai.com/envs/Breakout-ram-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gym.openai.com/envs/Breakout-ram-v0/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation says that the state is represented as a RAM state, but what is the RAM in this context? Is it the random access memory? What does the RAM state represent?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-06-01T10:47:10.973" LastActivityDate="2020-06-01T10:47:10.973" Title="What is a RAM state in the gym's breakout-ram environment?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;implementation&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21555" PostTypeId="2" ParentId="21553" CreationDate="2020-06-01T10:07:48.473" Score="1" Body="&lt;p&gt;In the tabular case, then the Q table will only converge if you &lt;em&gt;have&lt;/em&gt; walked around the whole of the table. Note that to guarantee convergence we need &lt;span class=&quot;math-container&quot;&gt;$\sum\limits_{n=1}^{\infty}\alpha_n(a) = \infty$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sum\limits_{n=1}^\infty \alpha_n^2(a) &amp;lt; \infty$&lt;/span&gt;. These conditions imply that in the limit each state-action pair will have been visited an infinite number of times, thus we will have walked around the whole table, so there are no out-of-sample state-action pairs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the case of function approximation, then convergence is no longer guaranteed. Generalisability is possible though - assuming we have an infinite state or action space then we will only ever visit the same state-action pair once, so the role of a function approximator is to allow us to generalise the state/action space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB that the convergence conditions I mentioned are only required in some proofs of convergence, depending on what type of convergence you are looking to prove. See &lt;a href=&quot;https://ai.stackexchange.com/questions/21388/what-are-the-conditions-of-convergence-of-temporal-difference-learning/21408#21408&quot;&gt;this answer&lt;/a&gt; for more details. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-06-02T17:36:34.637" LastActivityDate="2020-06-02T17:36:34.637" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21556" PostTypeId="2" ParentId="21537" CreationDate="2020-06-01T10:08:38.370" Score="1" Body="&lt;p&gt;You should try an architecture with an &lt;strong&gt;encoder&lt;/strong&gt; and a &lt;strong&gt;decoder&lt;/strong&gt;. The encoder will consume all the data you give as in put and decoder will give out the series of output.&lt;/p&gt;&#xA;" OwnerUserId="37504" LastActivityDate="2020-06-01T10:08:38.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21557" PostTypeId="2" ParentId="21554" CreationDate="2020-06-01T10:25:33.440" Score="2" Body="&lt;p&gt;Yes, it is the state of the memory; this would mainly involve variables, since the code would be in ROM. Since it is only 128 bytes in size, the screen memory would also not be included in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that all information relevant to the game is captured in these 128 bytes; they represent the &lt;em&gt;state&lt;/em&gt; of the game world at any given time. Movements of the ball, the game controller position, etc are all encoded there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For machine learning it is not actually relevant &lt;em&gt;which&lt;/em&gt; bytes are representing which value, as any optimisation of outcomes will treat all these 128 bytes as parameters describing the state. A machine learning system will pick up the optimal configuration through the learning process, eg that the 'racket' position should always near the ball x-coordinate. That will just be a correlation of two bytes, no matter which ones they actually are.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-06-01T10:35:39.413" LastActivityDate="2020-06-01T10:35:39.413" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21559" PostTypeId="1" CreationDate="2020-06-01T11:25:42.907" Score="1" ViewCount="32" Body="&lt;p&gt;I am using LSTM network for predicting IOT time-series data receiving from un-reliable devices and networks.&lt;br&gt;&#xA;This results in several multiple sections [continuous streak of bad data for several days until the problem is fixed].&lt;br&gt;&#xA;I need to exclude this bad data section before feeding it to model training.&lt;br&gt;&#xA;Since I am using LSTM-RNN network, it requires to do an un-roll data based on the previous records.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I properly exclude this bad data?&lt;br&gt;&#xA;I thought of an approach as training model separately using each batch of good data, and use subsequent good-data batch for fine-tuning the model.&lt;br&gt;&#xA;Please let me know if this is a good approach? or is there a better method?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;example data:&#xA;&quot;1-01&quot;,266.0&#xA;&quot;1-02&quot;,145.9&#xA;&quot;1-03&quot;,183.1&#xA;&quot;1-04&quot;,0  [bad data]&#xA;&quot;1-05&quot;,0  [bad data]&#xA;&quot;1-06&quot;,0  [bad data]&#xA;&quot;1-07&quot;,0  [bad data]&#xA;&quot;1-08&quot;,224.5&#xA;&quot;1-09&quot;,192.8&#xA;&quot;1-10&quot;,122.9&#xA;&quot;1-11&quot;,0  [bad data]&#xA;&quot;1-12&quot;,0  [bad data]&#xA;&quot;2-01&quot;,194.3&#xA;&quot;2-02&quot;,149.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37505" LastActivityDate="2020-07-06T13:28:38.153" Title="How to exclude sections of bad data from time-series data before training an LSTM network" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21560" PostTypeId="1" CreationDate="2020-06-01T11:42:58.087" Score="0" ViewCount="103" Body="&lt;p&gt;I stumbled across a question asking to draw a conceptual dependency for the following statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Place all ingredients in a bowl and mix thoroughly&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My attempt so far&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nAHgM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nAHgM.jpg&quot; alt=&quot;My working&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explanation: Both the sender and recipient are the same, except that the states of their contents are different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something feels like it isn't right. I would appreciate if you could correct errors, if any.&lt;/p&gt;&#xA;" OwnerUserId="37506" LastEditorUserId="2193" LastEditDate="2020-06-01T16:47:45.133" LastActivityDate="2021-02-27T00:06:50.613" Title="How can I draw a conceptual dependency for the statement &quot;Place all ingredients in a bowl and mix thoroughly&quot;?" Tags="&lt;knowledge-representation&gt;&lt;conceptual-dependency&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21562" PostTypeId="1" CreationDate="2020-06-01T14:11:47.477" Score="3" ViewCount="128" Body="&lt;p&gt;I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once deployed, humans review and check for correctness &lt;em&gt;only items predicted positive&lt;/em&gt; by the algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, once in production if I group predictions of the model on unseen examples in the confusion matrix &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+-----------+-----------------+&#xA;|           |   Ground-truth  |&#xA;|           +-----+-----------+&#xA;|           |     | Neg | Pos |&#xA;+-----------+-----+-----+-----+&#xA;|           | Neg | x11 | x12 |&#xA;| Predicted +-----+-----+-----+&#xA;|           | Pos | x21 | x22 |&#xA;+-----------+-----+-----+-----+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have access to all the ground-truth labels of the elements counted in &lt;span class=&quot;math-container&quot;&gt;$x_{21}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x_{22}$&lt;/span&gt; (the predicted-positive)&lt;/li&gt;&#xA;&lt;li&gt;I know the &lt;em&gt;sum&lt;/em&gt; of &lt;span class=&quot;math-container&quot;&gt;$x_{11}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_{12}$&lt;/span&gt;, but not their values&lt;/li&gt;&#xA;&lt;li&gt;I do &lt;em&gt;not&lt;/em&gt; have access to the ground-truth labels of the elements predicted-negative.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This (suboptimal) setup allows to measure precision &lt;span class=&quot;math-container&quot;&gt;$\frac{x_{22}}{x_{21} + x_{22}}$&lt;/span&gt;, while recall stays unknown as elements predicted negative are not examined at all (ground-truth labels of negatives can't be assigned due to resource constraints). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Information gathered from users about the (true and false) positive elements can be used to feed a retraining loop... but&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;are there any &quot;smart&quot; learning recipes that are expected to make the algorithm improve its overall performance (say, the F1 score for the positive class) in this setting?&lt;/li&gt;&#xA;&lt;li&gt;what's a meaningful metric to monitor to ensure that the performance of the model is not &lt;em&gt;degrading&lt;/em&gt;?* (given the constraint specified here, F1 score is unknown).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks for any hint on how to deal with this!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt; * One solution could be to continuously monitor the F1 score on a labeled evaluation set, but maybe there's more one can do?&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="33032" LastEditorUserId="33032" LastEditDate="2020-06-02T15:20:27.120" LastActivityDate="2020-06-03T19:02:46.640" Title="How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?" Tags="&lt;machine-learning&gt;&lt;online-learning&gt;&lt;algorithmic-bias&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21563" PostTypeId="2" ParentId="21560" CreationDate="2020-06-01T17:02:38.057" Score="0" Body="&lt;p&gt;As it's two clauses (&quot;place in bowl&quot; and &quot;mix&quot;), I would actually use two separate CD structures. The first one is a PTRANS (not ATRANS, as you don't change ownership, but location); the second one is a bit trickier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Mix&lt;/em&gt; can be paraphrased as &lt;em&gt;stir&lt;/em&gt;, or &lt;em&gt;move around&lt;/em&gt;. You would typically do this with a spoon or similar implement. I would do it as a PROPEL with the instrument spoon and the object the ingredients. PROPEL, according to Schank (1975) is the &quot;application of a physical force to an object&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the first one is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(PTRANS&#xA;  (ACTOR *you*)&#xA;  (OBJECT &quot;all ingredients&quot;)&#xA;  (TO &quot;bowl&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the second is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(PROPEL&#xA;  (ACTOR *you*)&#xA;  (OBJECT &quot;ingredients&quot;)&#xA;  (INSTRUMENT &quot;a spoon&quot;)&#xA;  (DIRECTION &quot;circular&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's debatable if you PROPEL the ingredients with the instrument spoon, or you PROPEL the object spoon with the target/location/direction ingredients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you draw them in a graphical representation, you can have both ACTs leading off the shared actor '*you*'; or you could have them as separate graphs linked by 'then' or some other indicator of sequentiality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Schank, Roger (1975) &lt;em&gt;The primitive ACTs of conceptual dependency&lt;/em&gt;, Proc of 1975 workshop on theoretical issues in natural language processing, 34-37&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-06-01T17:02:38.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21565" PostTypeId="1" AcceptedAnswerId="21568" CreationDate="2020-06-01T18:49:52.903" Score="2" ViewCount="86" Body="&lt;p&gt;From Sutton and Barto's book Reinforcement Learning (Adaptive Computation and Machine Learning series) (p. 99), the following definition for first-visit MC prediction, for estimating &lt;span class=&quot;math-container&quot;&gt;$V \sim V_\pi$&lt;/span&gt; is given:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/J5aP2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J5aP2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is determining the reward for each action a requirement of SARSA? Can the agent wait until the end of the episode to determine the reward? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the reward for the game tic-tac-toe is decided at the end of the episode, when the player wins, loses, or draws the match. The reward is not available at each step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean then that depending on the task it is not always possible to determine reward at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, and the agent must wait until the end of an episode? If the agent does not evaluate reward until the end of an episode, is the algorithm still SARSA?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="2444" LastEditDate="2020-06-01T18:54:29.197" LastActivityDate="2020-06-01T20:49:54.500" Title="Can the agent wait until the end of the episode to determine the reward in SARSA?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;rewards&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21566" PostTypeId="1" CreationDate="2020-06-01T19:03:22.820" Score="1" ViewCount="42" Body="&lt;p&gt;What are the real applications of &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_temporal_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;hierarchical temporal memory&lt;/a&gt; (HTM) in machine learning (ML) these days?&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2020-06-01T19:19:00.167" LastActivityDate="2020-06-02T17:52:01.623" Title="What are the real applications of hierarchical temporal memory?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;applications&gt;&lt;research&gt;&lt;htm&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21567" PostTypeId="2" ParentId="21566" CreationDate="2020-06-01T19:03:22.820" Score="1" Body="&lt;p&gt;The following figure [&lt;strong&gt;&lt;a href=&quot;https://numenta.com/blog/2019/10/24/machine-learning-guide-to-htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/strong&gt;] is inspiring:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pGyiq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pGyiq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It means HTM can be used as a memory unit in different neural networks such as RNN. In the same reference, we can find the following advantages and applications of HTM:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Sequence learning:&lt;/strong&gt;&#xA;  Being able to model temporally correlated patterns represents a key property of intelligence, as it gives both biological and artificial systems the essential ability to predict the future. It answers the basic question “what will happen next?” based on what it has seen before. Every machine learning algorithm should be able to provide valuable predictions not just based on static spatial information but also grounding it in time.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;High-order predictions:&lt;/strong&gt;&#xA;  Real-world sequences contain contextual dependencies that span multiple time steps, hence the ability to make high-order predictions becomes fundamental. The term “order” refers to Markov's order, specifically the minimum number of previous time steps the algorithm needs to consider in order to make accurate predictions. An ideal algorithm should learn the order automatically and efficiently.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Multiple simultaneous predictions:&lt;/strong&gt;&#xA;  For a given temporal context, there could be multiple possible future outcomes. With real-world data, it is often insufficient to only consider the single best prediction when information is ambiguous. A good sequence learning algorithm should be able to make multiple predictions simultaneously and evaluate the likelihood of each prediction online. This requires the algorithm to output a distribution of possible future outcomes.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Continual Learning:&lt;/strong&gt;&#xA;  Continuous data streams often have changing statistics. As a result, the algorithm needs to continuously learn from the data streams and rapidly adapt to changes. This property is important for processing continuous real-time perceptual streams, but has not been well studied in machine learning, especially without storing and reprocessing previously encountered data.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Online learning:&lt;/strong&gt;&#xA;  For real-time data streams, it is much more valuable if the algorithm can predict and learn new patterns on-the-fly without the need to store entire sequences or batch several sequences together as it normally happens when training gradient-based recurrent neural networks. The ideal sequence learning algorithm should be able to learn from one pattern at a time to improve efficiency and response time as the natural stream unfolds.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Noise robustness and fault tolerance:&lt;/strong&gt;&#xA;  Real world sequence learning deals with noisy data sources where sensor noise, data transmission errors and inherent device limitations frequently result in inaccurate or missing data. A good sequence learning algorithm should exhibit robustness to noise in the inputs.&lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;No hyperparameter tuning:&lt;/strong&gt;&#xA;  Learning in the cortex is extremely robust for a wide range of problems. In contrast, most machine-learning algorithms require optimizing a set of hyperparameters for each task. It typically involves searching through a manually specified subset of the hyperparameter space, guided by performance metrics on a cross-validation dataset. Hyperparameter tuning presents a major challenge for applications that require a high degree of automation, like data stream mining. An ideal algorithm should have acceptable performance on a wide range of problems without any task-specific hyperparameter tuning.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One of the applications of HTM is in &lt;strong&gt;Anomaly detection&lt;/strong&gt;. See, for example, this paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0925231217309864&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised real-time anomaly detection for streaming data&lt;/a&gt; (2017) which used from a network of HTMs to solve this problem in streaming data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/numenta/nupic&quot; rel=&quot;nofollow noreferrer&quot;&gt;The implementation of HTM&lt;/a&gt; (&lt;a href=&quot;http://nupic.docs.numenta.org/stable/quick-start/index.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;NuPIC&lt;/a&gt;) can help to see more applications of HTM in ML.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-06-02T17:52:01.623" LastActivityDate="2020-06-02T17:52:01.623" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21568" PostTypeId="2" ParentId="21565" CreationDate="2020-06-01T20:39:41.617" Score="1" Body="&lt;p&gt;There are a couple of things to break down here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing is to correct this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, the reward for the game tic-tac-toe is decided at the end of the episode, when the player wins, loses, or draws the match. The reward is not available at each step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a Markov Decision Process (MDP), there is always an immediate reward for each time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt; (terminal state). This is the reward distribution that the algorithms refer to as &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does not matter if nearly all the rewards are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. That is still a reward value, even if it is not an interesting or informative one. Some environments will have non-zero rewards on nearly all time steps. Some environments might have zero rewards for every transition apart from one or two important exceptions that define the goal of the agent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in tic-tac-toe, if the game has not ended, the reward will be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; because neither player has won or lost, and obtaining a win is a learning objective. If you use &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; reward value for all incomplete time steps in tic-tac-toe, and SARSA algorithm as written from Sutton &amp;amp; Barto, then it will work as expected. You don't need to wait until the end of the episode, and the algorithm will still learn to predict values of moves despite experiencing many &lt;span class=&quot;math-container&quot;&gt;$r_t = 0$&lt;/span&gt; during start and middle of the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You show the MC prediction algorithm, which &lt;em&gt;does&lt;/em&gt; wait until the end of each episode until calculating estimated values, using the experienced &lt;em&gt;return&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think one of the things that you might want to review is the difference between &lt;em&gt;immediate reward&lt;/em&gt; following a timestep (noted &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;) and &lt;em&gt;return&lt;/em&gt; (noted &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt;). There is a relationship between the two &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=0}^{\infty} \gamma^k R_{t+k+1}$&lt;/span&gt;. All the value based methods, such as MC prediction, or SARSA, are ways to estimate the expected return &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}[G_t]$&lt;/span&gt; given some context, such as the state or state and action at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that, this should answer your other questions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is determining the reward for each action a requirement of SARSA? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is a general requirement for MDPs, and is expected by all solvers. However, a default of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; is fine, if that's how the environment works. The reinforcement learning (RL) algorithms can all cope with delays to important reward values (such as those from winning or losing a game). Bigger delays - e.g. thousands of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; reward steps between more meaningful values - are often associated with harder problems to solve, so the simpler algorithms might not be practical. But they all cope in theory given enough experience.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the agent wait until the end of the episode to determine the reward?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, you should always be able calulate an &lt;em&gt;immediate reward&lt;/em&gt; value &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}$&lt;/span&gt; after taking each action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; from state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is OK in general to wait until you have summed up more of those rewards (e.g. &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$r_{t+n}$&lt;/span&gt;) before making an estimate of the return &lt;span class=&quot;math-container&quot;&gt;$g_t$&lt;/span&gt; to update a value estimate. Single step SARSA (sometimes called SARSA(0)) does not wait - it makes update steps using a single immediate reward value. However, other agent types including Monte Carlo (MC), and some variants of SARSA, can and do wait - sometimes just for some steps, sometimes until the end of each episode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a choice you can make when designing agents, there are consequences. Waiting until end of the episode (as in MC) means you have unbiased estimates of value, but takes longer, and leads to higher variability in updates. Using reward values after each step (as in SARSA(0)) means you get to make more frequent updates which may converge fatser, but you start with biased incorrect values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-01T20:49:54.500" LastActivityDate="2020-06-01T20:49:54.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21569" PostTypeId="1" CreationDate="2020-06-01T23:10:38.093" Score="1" ViewCount="29" Body="&lt;p&gt;I have the following problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am given a graph with a lot (&gt;30000) nodes. Nodes are associated with a low (&amp;lt;10)-dimensional feature vector, and edges are associated with a low (&amp;lt;10)-dimensional feature vector. In addition, all nodes start out having the color white.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At every time step until completion, I want to select a subset of the nodes in the graph and color them blue. Then I receive a reward based on my coloring. I continue until all nodes are colored blue, and the total reward is the sum (maybe with a gamma factor) of my total rewards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have suggestions of papers to read where the task was choosing an appropriate subgraph from a larger graph? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just doing a node classification task using a Graph Convolutional Network doesn't seem to do well, I suspect because, given that a good heuristic for reward is connectivity, it would need to learn to choose an optimal neighborhood in the graph and upweight only that neighborhood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To contextualize, each of the nodes of the graph represents a constraint that will be sent to an incremental &lt;a href=&quot;https://en.wikipedia.org/wiki/Satisfiability_modulo_theories&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMT solver&lt;/a&gt;, and edges represent shared variables or other relationships between the constraints.  I have found empirically that giving these constraints incrementally to the SMT solver when in a good order can be faster than just dumping the entire problem into an SMT solver, since the SMT solver doesn't have the best heuristics for this particular SMT problem.  However, eventually, I want to add all the constraints, i.e., color the entire graph. The cost is the amount of time the solver takes on each set, with a reward at the end for completing all the constraints. &lt;/p&gt;&#xA;" OwnerUserId="37521" LastEditorUserId="2444" LastEditDate="2020-06-02T17:04:24.843" LastActivityDate="2020-06-02T17:04:24.843" Title="How to learn how to select a subgraph via reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21570" PostTypeId="1" CreationDate="2020-06-01T23:40:19.250" Score="0" ViewCount="63" Body="&lt;p&gt;I'm trying to approximate the following function with a neural network (in Python).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Train a neural network so that y = my_function(x0, x1, ...) ~ NN(x0, x1, ...)&#xA;&#xA;def my_function(x0, x1, x2, x3, z1, z2, h1, h2):&#xA;    # Inputs are 1D arrays of type float64&#xA;    # And h1 and h2 are always &amp;gt;= 0.&#xA;&#xA;    # b is always &amp;gt;= 0&#xA;    b = np.maximum(z1 + h1, z2 + h2) - np.maximum(z1, z2) &#xA;&#xA;    c = (z1 + h1) - (z2 + h2)&#xA;    term1 = .9 * x0 + 0.05 * (x1 + x2)&#xA;    term2 = 4.9 * b * c&#xA;&#xA;    # if term1 * term2 &amp;lt; 0 then term1 = x0&#xA;    mask = term1 * term2 &amp;lt; 0 &#xA;    term1[mask] = x0[mask] &#xA;    x_norm = np.sqrt(x0**2 + x3**2)&#xA;    term3 = 1. + 0.0044 * x_norm / np.power(b, 7./3.)&#xA;    return (term1 + term2) / term3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I generated 3 million rows of data to train a neural network. However, I do not seem to achieve a good approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried many neural network structures using a combination of the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Transfer uniformly Radom samples of input datausing  (x-mu)/sigma&#xA; n_layers:  ranged between 3 to 5 layers&#xA; n_neurons: 30 to 60&#xA; Learning_rate: from 0.001 to 0.000001&#xA; Activation_function = tried relu and and then tanh &#xA; batch_size = 15,000 to 40,000 &#xA; loss_fnction = mean squared error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The lowest loss I get after around 9K epochs was 1.5464e-06. The following figure shows the residuals (&lt;code&gt;y_pred - y_obs&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gUxUE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gUxUE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something important here? What else I need to try to see if there will be an improvement?&lt;/p&gt;&#xA;" OwnerUserId="37523" LastEditorUserId="2444" LastEditDate="2020-06-02T14:55:26.430" LastActivityDate="2020-06-02T14:55:26.430" Title="Why is my neural network not able to approximate this function?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;feedforward-neural-network&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21571" PostTypeId="1" CreationDate="2020-06-01T23:40:56.923" Score="1" ViewCount="56" Body="&lt;p&gt;I have a simple question about model-free reinforcement. In a model I'm writing about, I want to know the value 'gain' we'd get for executing an action, relative to the current state. That is, what will I get if I moved from the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The measure I want is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G(s, a)=V(s^{\prime})-V(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the state that I would transition to if the underlying MDP was deterministic. If the MDP has a stochastic transition function, the model I want is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G(s, a)=\left[\sum_{s' \in S } P(s^{\prime} \mid a, s) V(s^{\prime})\right]-V(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a model-free environment, we don't have &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid a,s)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we had a Q-function &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt;, could we represent &lt;span class=&quot;math-container&quot;&gt;$G(s,a)$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: This is not the same as an 'advantage function' as first proposed by Baird (Leemon C Baird. Reinforcement learning in continuous time: Advantage updating. In Proceedings of 1994 IEEE International Conference on Neural Networks, pages 448–2453. IEEE, 1994.), which means the advantage of actions relative to the optimal action. What I'm looking for is the gain of actions relative to the current state.&lt;/p&gt;&#xA;" OwnerUserId="37522" LastEditorUserId="37522" LastEditDate="2020-06-02T10:52:34.227" LastActivityDate="2020-06-02T10:52:34.227" Title="Calculating the advantage 'gain' of actions in model-free reinforcement learning" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21572" PostTypeId="1" CreationDate="2020-06-02T00:41:27.743" Score="0" ViewCount="67" Body="&lt;p&gt;The available tutorials are most focused on time series prediction. I am wondering how shall we prepare the input data when it is an imbalanced data? Here is how data looks like.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|---------|--------------|-----------|-----------|&#xA;|    id   |  timestamp   |  feature1 |  feature2 |&#xA;|---------|--------------|-----------|-----------|&#xA;|  1      |      1       |    0.54   |     2     |&#xA;|---------|--------------|-----------|-----------|&#xA;|  1      |      2       |    0.67   |     3     |&#xA;|---------|--------------|-----------|-----------|&#xA;|  1      |      3       |    0.32   |     3     |&#xA;|------------------------|-----------|-----------|&#xA;|  1      |      4       |    0.43   |     4     |&#xA;|------------------------|-----------|-----------|&#xA;|  2      |      1       |    0.25   |     1     |&#xA;|------------------------|-----------|-----------|&#xA;|  2      |      2       |    0.93   |     2     |&#xA;|------------------------|-----------|-----------|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am aware of this &lt;a href=&quot;https://stackoverflow.com/questions/42694725/panel-data-in-keras-lstm&quot;&gt;post&lt;/a&gt; that discusses LSTM for balanced data. But how about an imbalanced dataset?&lt;/p&gt;&#xA;" OwnerUserId="23707" LastActivityDate="2020-06-02T00:41:27.743" Title="LSTM for imbalanced panel data" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21573" PostTypeId="1" CreationDate="2020-06-02T01:29:50.847" Score="1" ViewCount="44" Body="&lt;p&gt;The problem I currently have is that I want to train an AI to produce music, like music that contains voices etc... However, the problem is that with a WAV file, one second of audio can be up to 48,000 inputs, which is extremely detrimental to the ai's learning process and prevents it from really gaining any knowledge about context.&#xA;I've tried to do the fast Fourier transformation, but the amount of data coming in varies depending on what part of the song I'm training it on, which will not work since I can't know ahead of time what every single time unit of every single song will have!&#xA;And the max number of inputs is still 24,000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other way of compressing data down in a way that I can give my ai something within the range of a couple hundred inputs for a second or two?&lt;/p&gt;&#xA;" OwnerUserId="36808" LastActivityDate="2020-06-02T01:29:50.847" Title="What is the most compressed audio that I can feed an AI?" Tags="&lt;machine-learning&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21574" PostTypeId="1" CreationDate="2020-06-02T05:20:35.113" Score="0" ViewCount="18" Body="&lt;p&gt;While I attempt to learn AI/ML I have taken on the task to create a &lt;a href=&quot;https://en.wikipedia.org/wiki/Boggle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boggle&lt;/a&gt; solver. The idea is that a system could take an image of a Boggle arrangement of letters and identify the letters (and the relative placement/ordering of them) which could then be used to match with a dictionary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Blender, I created a script to auto-generate varying camera orientations, letter positions, etc. I also created corresponding segmentation masks and letter lists, as shown here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u4U3P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u4U3P.png&quot; alt=&quot;render&quot;&gt;&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/HkIi0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HkIi0.png&quot; alt=&quot;mask&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;letters: [20,  3, 23,  1, 23, 15, 24,  7, 22, 12, 11,  3, 24,  9, 18, 20,  1,&#xA;        0, 10, 23, 20, 25, 11,  8, 21]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I initially tried to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/U-Net&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net&lt;/a&gt; to try to automatically recreate the mask but that was unsuccessful. I am now thinking that re-creating the mask might be unnecessary and I would like to go straight from the render image to the list of letters. What might be a good approach for this type of problem? My current network is the left side of a U-Net that is then reduced using Dense layers to a one-hot tensor for the entire list of letters, but it fails to make any progress (hovers around .04 accuracy).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input_1 (InputLayer)         [(None, 256, 256, 3)]     0&#xA;_________________________________________________________________&#xA;lambda (Lambda)              (None, 256, 256, 3)       0&#xA;_________________________________________________________________&#xA;conv2d (Conv2D)              (None, 256, 256, 16)      448&#xA;_________________________________________________________________&#xA;dropout (Dropout)            (None, 256, 256, 16)      0&#xA;_________________________________________________________________&#xA;conv2d_1 (Conv2D)            (None, 256, 256, 16)      2320&#xA;_________________________________________________________________&#xA;max_pooling2d (MaxPooling2D) (None, 128, 128, 16)      0&#xA;_________________________________________________________________&#xA;conv2d_2 (Conv2D)            (None, 128, 128, 32)      4640&#xA;_________________________________________________________________&#xA;dropout_1 (Dropout)          (None, 128, 128, 32)      0&#xA;_________________________________________________________________&#xA;conv2d_3 (Conv2D)            (None, 128, 128, 32)      9248&#xA;_________________________________________________________________&#xA;max_pooling2d_1 (MaxPooling2 (None, 64, 64, 32)        0&#xA;_________________________________________________________________&#xA;conv2d_4 (Conv2D)            (None, 64, 64, 64)        18496&#xA;_________________________________________________________________&#xA;dropout_2 (Dropout)          (None, 64, 64, 64)        0&#xA;_________________________________________________________________&#xA;conv2d_5 (Conv2D)            (None, 64, 64, 64)        36928&#xA;_________________________________________________________________&#xA;max_pooling2d_2 (MaxPooling2 (None, 32, 32, 64)        0&#xA;_________________________________________________________________&#xA;conv2d_8 (Conv2D)            (None, 32, 32, 128)       73856&#xA;_________________________________________________________________&#xA;dropout_5 (Dropout)          (None, 32, 32, 128)       0&#xA;_________________________________________________________________&#xA;conv2d_9 (Conv2D)            (None, 32, 32, 128)       147584&#xA;_________________________________________________________________&#xA;flatten (Flatten)            (None, 131072)            0&#xA;_________________________________________________________________&#xA;dense_3 (Dense)              (None, 1024)              134218752&#xA;_________________________________________________________________&#xA;dropout_6 (Dropout)          (None, 1024)              0&#xA;_________________________________________________________________&#xA;dense_4 (Dense)              (None, 675)               691875&#xA;_________________________________________________________________&#xA;reshape (Reshape)            (None, 25, 27)            0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37527" LastActivityDate="2020-06-02T05:20:35.113" Title="Identifying and Labeling multiple letters in image" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21575" PostTypeId="1" CreationDate="2020-06-02T05:54:24.573" Score="2" ViewCount="32" Body="&lt;p&gt;Why do we need recurrent neural networks instead of feed-forward neural networks? What are the advantages of RNNs compared with FFNNs?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-06-02T10:18:15.057" LastActivityDate="2020-06-02T11:36:48.963" Title="Why do we need recurrent neural networks instead of feed-forward neural networks?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-06-02T11:32:00.443" ContentLicense="CC BY-SA 4.0" />
  <row Id="21576" PostTypeId="1" AcceptedAnswerId="21579" CreationDate="2020-06-02T09:16:47.187" Score="1" ViewCount="72" Body="&lt;p&gt;I am doing an online course on Reinforcement Learning from university of Alberta. &#xA;It focus too much on theory. I am engineering and I am interested towards applying RL to my applications directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there any website which has sample programmers for beginners. Small sample programs. &#xA;I have seen several websites for other machine learning topics such as CNN/RNN etc. But the resources for RL are either limited, or I couldn't find them&lt;/p&gt;&#xA;" OwnerUserId="36710" LastActivityDate="2020-06-02T11:11:33.583" Title="Is there any programming practice website for beginners in Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="2" ClosedDate="2020-06-02T11:31:33.343" ContentLicense="CC BY-SA 4.0" />
  <row Id="21577" PostTypeId="1" AcceptedAnswerId="21582" CreationDate="2020-06-02T09:40:27.923" Score="1" ViewCount="46" Body="&lt;p&gt;I've been reading this paper on Knowledge Graph Reasoning for Explainable Recommendation lately, and I don't understand a particular section:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Specifically, the scoring function &lt;span class=&quot;math-container&quot;&gt;$f((r,e)|u)$&lt;/span&gt; maps any edge &lt;span class=&quot;math-container&quot;&gt;$(r,e)$&lt;/span&gt; to a real-valued score conditioned on user &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;. Then, the user-conditional pruned action space of state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; denoted by &lt;span class=&quot;math-container&quot;&gt;$A_t(u)$&lt;/span&gt; is defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A_t(u) = \{(r,e)| rank(f((r,e)|u))) \leq \alpha ,(r,e) \in A_t\} $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a predefined integer that upper bounds the size of the action space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Details about the scoring function can be found in the attached paper - what I don't understand is - &lt;strong&gt;What does rank mean, here? Is the thing inside of it a matrix?&lt;/strong&gt;&#xA;It would be great if someone could explain the expression for the user conditional pruned action space in greater detail.&lt;/p&gt;&#xA;&lt;h2&gt;References:&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1906.05237.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Knowledge Graph Reasoning for Explainable&#xA;Recommendation&lt;/a&gt; [Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard&#xA;de Melo, Yongfeng Zhang]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-15T10:07:14.247" Title="What is meant by the rank of the scoring function here?" Tags="&lt;reinforcement-learning&gt;&lt;recommender-system&gt;&lt;knowledge-graph&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21579" PostTypeId="2" ParentId="21576" CreationDate="2020-06-02T11:11:33.583" Score="1" Body="&lt;p&gt;Firstly, since you are a beginner, I strongly recommend you start reading &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton's book&lt;/a&gt;. It is a really great book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, some tutorials:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/machine-learning-beginner-reinforcement-learning-in-python/?utm_source=adwords&amp;amp;utm_medium=udemyads&amp;amp;utm_campaign=DataScience_v.PROF_la.EN_cc.ROW_ti.5336&amp;amp;utm_content=deal4584&amp;amp;utm_term=_._ag_85469003954_._ad_437497334835_._kw__._de_c_._dm__._pl__._ti_dsa-774930036449_._li_9067687_._pd__._&amp;amp;matchtype=b&amp;amp;gclid=CjwKCAjw8df2BRA3EiwAvfZWaFozTSdALFp3qg_IFF5R_4PL6SGbeM5W_WGtGy5teJ5MW1Lef6dwBBoCqU0QAvD_BwE&quot; rel=&quot;nofollow noreferrer&quot;&gt;udemy rl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.udemy.com/course/deep-reinforcement-learning/?utm_source=adwords&amp;amp;utm_medium=udemyads&amp;amp;utm_campaign=DeepLearning_v.PROF_la.EN_cc.ROW_ti.5374&amp;amp;utm_content=deal4584&amp;amp;utm_term=_._ag_85479010714_._ad_437497336740_._kw__._de_c_._dm__._pl__._ti_dsa-774930039529_._li_9067687_._pd__._&amp;amp;matchtype=b&amp;amp;gclid=CjwKCAjw8df2BRA3EiwAvfZWaFHx42hKDJfUvGf_QPU7Bsub7wX1zGP6M8vz8maEP5EMAgEbu76rSRoCCiwQAvD_BwE&quot; rel=&quot;nofollow noreferrer&quot;&gt;udemy deep rl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;rl-with-tensorflow&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.learndatasci.com/tutorials/reinforcement-q-learning-scratch-python-openai-gym/&quot; rel=&quot;nofollow noreferrer&quot;&gt;learndatasci&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackabuse.com/introduction-to-reinforcement-learning-with-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;stackabuse&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-06-02T11:11:33.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21580" PostTypeId="1" CreationDate="2020-06-02T12:25:39.097" Score="0" ViewCount="29" Body="&lt;p&gt;I have read &lt;a href=&quot;https://link.springer.com/article/10.1007/s00779-017-1038-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper (&lt;a href=&quot;https://sci-hub.tw/10.1007/s00779-017-1038-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;its PDF&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sleep behavior assessment via smartwatch and stigmergic receptive fields&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At fuzzy classification part of paper said and referenced to &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0031320306002238?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper (19 REF.&lt;a href=&quot;https://sci-hub.tw/10.1016/j.patcog.2006.05.005&quot; rel=&quot;nofollow noreferrer&quot;&gt;its PDF File)&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ncGAb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ncGAb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see in this paper there are 16 sleep archetypes like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fAvIG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fAvIG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the authors used this kind of AI to find the input signal similarity by the one archetype:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VlWFk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VlWFk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SO after that, it used ANN to combine all archetypes sorting together:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4iwQh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4iwQh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above system output is one archetype series. And used as multi-layer structure with  sliced inputs to increase the robustness, like this system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MrhvR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MrhvR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the last system has had used the Fuzzy classification like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jW1zK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jW1zK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SO i don't know how this paper built the fuzzy rules and what is the membership function for the fuzzy classification inputs ( Does the inputs must be sorted by the 14 archetypes membership function said in paper as used as input for &lt;a href=&quot;https://se.mathworks.com/matlabcentral/fileexchange/66526-stigmergic-perceptron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stigmergic receptive fields&lt;/a&gt; cell? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or the fuzzy inputs are the last archetype series as the nights similarity matrix and made by the similarity matrix between the 20 day like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JJrCW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JJrCW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess this paper used dynamic fuzzy rules making like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UNGA9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UNGA9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i asked to know what is the &lt;strong&gt;fuzzy memberships function&lt;/strong&gt; for the inputs ( did made from &lt;strong&gt;nights&lt;/strong&gt; &lt;strong&gt;similarity&lt;/strong&gt; &lt;strong&gt;matrix&lt;/strong&gt;) and what is the &lt;strong&gt;fuzzy rules&lt;/strong&gt; and &lt;strong&gt;how&lt;/strong&gt; did those made?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Power point for this presentation could be found &lt;a href=&quot;https://www.slideshare.net/GSHCO/sleep-behavior-assessment-via-smartwatch-and-stigmergic-receptive-fields&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="33936" LastEditorUserId="33936" LastEditDate="2020-06-02T17:53:44.237" LastActivityDate="2020-06-02T17:53:44.237" Title="Fuzzy Rules making for sleep quality estimation" Tags="&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21582" PostTypeId="2" ParentId="21577" CreationDate="2020-06-02T12:33:33.880" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{rank}(f((r,e)|u))$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$A_t(u)$&lt;/span&gt; means to compute the value of scoring function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; for all pairs &lt;span class=&quot;math-container&quot;&gt;$(r,e)\in A_t$&lt;/span&gt; which are conditioned by &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, then sort them in a descending order. The rank of the &lt;span class=&quot;math-container&quot;&gt;$f((r,e)|u)$&lt;/span&gt; in this order is equal to &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(f((r,e)|u))$&lt;/span&gt;. Hence &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(f((r,e)|u)) \leqslant \alpha$&lt;/span&gt; means to select the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; top most scored pairs.  &lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-06-15T10:07:14.247" LastActivityDate="2020-06-15T10:07:14.247" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21583" PostTypeId="1" CreationDate="2020-06-02T14:10:10.190" Score="2" ViewCount="47" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://arxiv.org/pdf/1903.03332.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper and in algorithm 3 they describe an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step Q-Learning algorithm. Below is the pseudo-code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TrCEO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TrCEO.png&quot; alt=&quot;pseudo-code for n-step q-learning&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this pseudo-code, it looks as though the final tuples that they would visit in don't get added to the memory buffer &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;. They define a sample size &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, but also say in the paper that an episode terminates when &lt;span class=&quot;math-container&quot;&gt;$|S| = b$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leaves me two questions: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have I understood the episode termination correctly? It seems from the pseudocode they are just running an episode for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; time steps but also in the paper they have a definition for when an episode terminates, so I'm not sure why they would want to truncate the episode size. &lt;/li&gt;&#xA;&lt;li&gt;As I mentioned, it seems as though the final state &lt;span class=&quot;math-container&quot;&gt;$S_T$&lt;/span&gt; that you would be in won't get added to the experience buffer as we only append the &lt;span class=&quot;math-container&quot;&gt;$(T-n)$&lt;/span&gt;th tuple. Why would you want to exclude the information you get from the final tuples you visit? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-06-02T15:48:33.697" LastActivityDate="2020-06-02T15:48:33.697" Title="Are the final states not being updated in this $n$-step Q-Learning algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;dqn&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21584" PostTypeId="1" AcceptedAnswerId="21585" CreationDate="2020-06-02T16:25:08.113" Score="7" ViewCount="612" Body="&lt;p&gt;What is the difference between reinforcement learning (RL) and evolutionary algorithms (EA)?&lt;/p&gt;&#xA;&lt;p&gt;I am trying to understand the basics of RL, but I do not yet have practical experience with RL. I know slightly more about EAs, but not enough to understand the difference between RL and EA, and that's why I'm asking for their main differences.&lt;/p&gt;&#xA;" OwnerUserId="37533" LastEditorUserId="2444" LastEditDate="2021-01-20T19:11:00.897" LastActivityDate="2021-01-20T19:11:00.897" Title="What is the difference between reinforcement learning and evolutionary algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21585" PostTypeId="2" ParentId="21584" CreationDate="2020-06-02T16:56:04.550" Score="6" Body="&lt;p&gt;&lt;strong&gt;Evolutionary algorithms (EAs)&lt;/strong&gt; are a family of algorithms inspired by the biological evolution that can be used to solve (constrained or not) optimization problems where the function that needs to be optimized does not necessarily need to be differentiable (or satisfy any strong constraint). In EAs, you typically only need to define &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;an encoding of the solution (aka chromosome or individual)&lt;/li&gt;&#xA;&lt;li&gt;a fitness function that determines the &lt;em&gt;relative&lt;/em&gt; quality of each solution&lt;/li&gt;&#xA;&lt;li&gt;operations that stochastically change or combine solutions (e.g. the cross-over or the mutation operators, in genetic algorithms)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are other parameters that you need to defined (such as the number of solutions to consider at each generation or the number of generations to run the algorithms for), but these are three most important things to take into account when attempting to solve an optimization problem with EAs (in particular, GAs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reinforcement learning (RL)&lt;/strong&gt; is the field that studies how agents can sequentially take actions in a certain environment in order to maximize some notion of long-term reward (aka return).  The strategy that determines the behavior of the agent (i.e. which actions the agent takes) is called the policy. So, the goal of RL is to find a policy that maximizes the (expected) return, which depends on the reward function of the environment. For example, in the case of chess, a reward function may be any function that gives you a positive number if you win the game or a negative number if you lose it. The RL algorithms typically assume that the agent is able to interact with the environment in order to understand its dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL is thus concerned with a specific type of optimization problem, i.e. finding policies (strategies) that maximize the return, while an agent interacts with an environment in time steps. On the other hand, EAs can be applied to any optimization problem where you can encode solutions, define a fitness function that compares solutions and you can stochastically change those solutions. Essentially, EAs can be applied to almost any optimization problem. In principle, you could use EAs to find policies, as long as you're able to compare them with a fitness function (e.g. the amount of reward that you obtain by following these policies). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this does not mean that EAs are the most efficient and appropriate approach to solve all optimization problems! You typically use EAs when you need to solve certain problems where better approaches do not exist. For example, when your objective function is not differentiable, then you cannot apply gradient-based solutions, so, in that case, EAs may be a viable option (but there are also other alternatives to EAs, such as simulated annealing).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-02T17:24:04.610" LastActivityDate="2020-06-02T17:24:04.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21586" PostTypeId="1" CreationDate="2020-06-02T17:21:48.220" Score="2" ViewCount="77" Body="&lt;p&gt;I'm trying to understand DQN. I understand where the loss function comes from. I'm just unsure about why the target function works in practice. Given the loss function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ L_i(\theta_i) = [(y_i - Q(s,a;\theta_i))^2] $$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ y_i = r + \gamma * max_{a'}Q(s',a';\theta_{i-1}) $$&lt;/span&gt;&#xA;is the target value in the loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, experiences are pulled from the replay buffer, then the DQN is used to estimate the future sum of the discounted rewards for the next state (assuming it plays optimally) and adds this onto the current rewards &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; to create the target value. Then the DQN is used again to estimate &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value for the current state. Then the loss function is just the difference between the target and the estimated &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value for the current state. Afterward, you optimize the loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if the parameters of the DQN start off randomly, then surely the target value will be completely wrong since the parameters that define that target function are random. So, if the target function is wrong, then it will minimize the difference between the target value and the estimated value, but it will be learning to predict incorrect values, since the target values are wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why the target value works if the parameters of the DQN needed to create that target are completely random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What obvious mistake am I making here?&lt;/p&gt;&#xA;" OwnerUserId="37535" LastEditorUserId="2444" LastEditDate="2020-06-04T14:41:29.150" LastActivityDate="2020-06-04T14:41:29.150" Title="How can the target rely on untrained parameters?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;dqn&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21588" PostTypeId="1" CreationDate="2020-06-02T18:33:32.487" Score="2" ViewCount="45" Body="&lt;p&gt;Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original &quot;Location-based Attention&quot; in Bahadanau well-known paper at &lt;a href=&quot;https://arxiv.org/pdf/1506.07503.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1506.07503.pdf&lt;/a&gt;. (it seems this attention is used in various forms of GNMT and text-to-speech sythesizers like tacotron-2 &lt;a href=&quot;https://github.com/Rayhane-mamah/Tacotron-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Rayhane-mamah/Tacotron-2&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even after repeated readings of this paper and other articles about Attention-mechanism, I'm confused about the dimensions of the matrices used, as the paper doesn't seem to describe it. My understanding is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If I have decoder hidden dim 1024, that means &lt;span class=&quot;math-container&quot;&gt;${s_{i-1}}$&lt;/span&gt; vector is 1024 length.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I have encoder output dim 512, that means &lt;span class=&quot;math-container&quot;&gt;$h_{j}$&lt;/span&gt; vector is 512 length.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If total inputs to encoder is 256, then number of &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; can be from 1 to 256.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$W x S_{i-1}$&lt;/span&gt; is a matrix multiply, it seems &lt;span class=&quot;math-container&quot;&gt;$cols(W)$&lt;/span&gt; should match &lt;span class=&quot;math-container&quot;&gt;$rows(S_{i-1})$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$rows(W)$&lt;/span&gt; still remain undefined. Same seems true for matrices &lt;span class=&quot;math-container&quot;&gt;$V, U, w, b$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is page-3/4 from the paper above that describes Attention-layer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xKD2U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xKD2U.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unsure how to make sense of this. Am I missing something, or can someone explain this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't understand is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the dimension of previous alignment (denoted by &lt;span class=&quot;math-container&quot;&gt;$alpha_{i-1})$&lt;/span&gt;? Shouldn't it be total values of &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$h_{j}$&lt;/span&gt; (which is 256 and means total different encoder output states)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the dimension of &lt;span class=&quot;math-container&quot;&gt;$f_{i,j}$&lt;/span&gt; and convolution filter &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;? (the paper says &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; belongs to &lt;span class=&quot;math-container&quot;&gt;$kxr$&lt;/span&gt; shape but doesn't define &lt;span class=&quot;math-container&quot;&gt;$'r'$&lt;/span&gt; anywhere). What is &lt;span class=&quot;math-container&quot;&gt;$'r'$&lt;/span&gt; and what does &lt;span class=&quot;math-container&quot;&gt;$'k x r'$&lt;/span&gt; mean here?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How are these unknown dimensions for matrices &lt;span class=&quot;math-container&quot;&gt;$'V, U, w, b'$&lt;/span&gt; described above determined in this model?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="33580" LastEditorUserId="33580" LastEditDate="2020-06-02T20:09:32.340" LastActivityDate="2020-06-02T20:09:32.340" Title="How to understand the matrices used in the Attention layer?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21589" PostTypeId="1" CreationDate="2020-06-02T18:47:46.933" Score="2" ViewCount="37" Body="&lt;p&gt;Apart from its use in word embeddings (e.g word2vec algorithm), are there any other applications of hierarchical softmax? If yes, can you please give me some reference papers?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-06-02T18:47:46.933" Title="What are the applications of hierarchical softmax?" Tags="&lt;reference-request&gt;&lt;word2vec&gt;&lt;hierarchical-softmax&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21590" PostTypeId="1" AcceptedAnswerId="21619" CreationDate="2020-06-02T19:08:40.113" Score="0" ViewCount="48" Body="&lt;p&gt;To update the Q table Q-learning takes the arg max of the Q values - the state, value mappings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in tic tac toe the state &lt;code&gt;XOX OXO -X-&lt;/code&gt; contains two available positions, each marked by the &lt;code&gt;-&lt;/code&gt; character. In order to evaluate the arg max should &lt;code&gt;temporal difference&lt;/code&gt; calculate the arg max of just the available positions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the state &lt;code&gt;XOX OXO -X-&lt;/code&gt; the arg max should be taken at index 6 and 8 ? (assuming zero indexing) If not then how should the arg max indexes be updated? The indexes &lt;code&gt;0,1,2 3,4,5 7&lt;/code&gt; can be used as they are already been taken by X or O so should not be evaluated for their value ? This also means that indexes which are not available will not have their Q value updated, will this break the Q-learning procedure ?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="34010" LastEditDate="2020-06-03T22:20:33.630" LastActivityDate="2020-06-03T22:20:33.630" Title="What is correct update when the some indexes are not available?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21591" PostTypeId="1" CreationDate="2020-06-02T19:36:49.103" Score="0" ViewCount="98" Body="&lt;p&gt;For reinforcement learning implementations with a &lt;code&gt;gym.spaces.Box&lt;/code&gt; action space, which is the product of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; real closed intervals, it is common (actually more like ubiquitous) to use a diagonal Gaussian distribution to sample actions (see e.g. &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/common/distributions.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/baselines/blob/master/baselines/common/distributions.py&lt;/a&gt;). This means that, if we are sampling &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; real-valued continuous subactions, call them &lt;span class=&quot;math-container&quot;&gt;$(a_1, \dots, a_k)$&lt;/span&gt;, from some random vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A} = (A_1, \dots, A_k)$&lt;/span&gt;, then the &lt;span class=&quot;math-container&quot;&gt;$\mathrm{Cov}(A_i, A_j) = 0$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;. So they all have zero correlation (though they're not necessarily independent). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using torch as an example framework, is sampling from &lt;code&gt;torch.distributions.multivariate_normal.MultivariateNormal&lt;/code&gt; with a diagonal covariance matrix equivalent to sampling from &lt;code&gt;torch.distributions.normal.Normal&lt;/code&gt; with the same mean vector and the diagonal of the covariance matrix as your standard deviation vector? Also, in what situations would it be useful to learn the whole covariance matrix as parameters, allowing nonzero correlation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some mention of this being slightly more computationally intensive in the following answer: &lt;a href=&quot;https://ai.stackexchange.com/questions/4085/how-can-policy-gradients-be-applied-in-the-case-of-multiple-continuous-actions&quot;&gt;How can policy gradients be applied in the case of multiple continuous actions?&lt;/a&gt;. But I am curious if there are some problems where it might be necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also seems to be common practice to learn the &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\sigma}$&lt;/span&gt; vector as a parameter with no inputs, instead of having it be a function of the states/observations. Is there any motivation behind one approach or the other?&lt;/p&gt;&#xA;" OwnerUserId="33762" LastActivityDate="2020-06-02T19:36:49.103" Title="Is there any advantage to using a non-diagonal covariance matrix for a policy distribution?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21592" PostTypeId="2" ParentId="18026" CreationDate="2020-06-02T21:27:46.210" Score="0" Body="&lt;p&gt;The exhaustive nearest neighbor search performs an exhaustive search of the nearest neighbor (i.e. the closest image or matches, depending on your application). Here an exhaustive search means that you will compare your query image with any other image, or, in the case of feature matching, you will compare every feature of the query image with every other feature of the target/reference image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-nearest neighbor search finds the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; closest images. You typically do that by first building a KD-tree with your query (or target) image, so that to speed up the search for the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-nearest neighbors of your query image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Btw, in case you are interested in the concept of feature matching, in OpenCV, the class that you can use to perform an exhaustive search for the matches between two images is the &lt;code&gt;BFMatcher&lt;/code&gt; (which stands for Brute Force Matcher), while the class to perform the search with a KD-tree for the k-nearest neighbhours is &lt;code&gt;FlannBasedMatcher&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-02T22:11:44.267" LastActivityDate="2020-06-02T22:11:44.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21593" PostTypeId="1" CreationDate="2020-06-03T00:19:46.060" Score="14" ViewCount="1288" Body="&lt;p&gt;Say we have a machine and we give it a task to do (vision task, language task, game, etc.), how can one prove that a machine actually know's what's going on/happening in that specific task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To narrow it down, some examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conversation&lt;/strong&gt; - How would one prove that a machine actually knows what it's talking about or comprehending what is being said? The Turing test is a good start, but never actually addressed &lt;em&gt;actual comprehension&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Vision&lt;/strong&gt;: How could someone prove or test that a machine actually knows what it's seeing? Object detection is a start, but I'd say it's very inconclusive that a machine understands at any level what it is actually seeing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we prove comprehension in machines?&lt;/p&gt;&#xA;" OwnerUserId="22840" LastEditorUserId="2444" LastEditDate="2020-06-03T02:52:59.840" LastActivityDate="2020-06-04T10:05:26.260" Title="How does one prove comprehension in machines?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;artificial-consciousness&gt;&lt;chinese-room-argument&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21594" PostTypeId="1" CreationDate="2020-06-03T02:42:03.490" Score="0" ViewCount="197" Body="&lt;p&gt;I am having trouble making a reinforcement algorithm than can win the 2048 game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried with deep Q (which I think is the simplest algorithm that should be able to learn a winning strategy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Q function is given by a NN of two hidden layers 16 -&gt; 8 -&gt; 4. Weight initialization is XAVIER. Activation function is RELU. Loss function is cuadratic loss. Correction is via gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To train the NN I used a reward given by :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r_t = \frac{1}{1024} \sum_{i=0}^{n}{p^i r_{((t-n)+i)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where n is 20 or the amount of iterations since the last update if a game is lost and &lt;span class=&quot;math-container&quot;&gt;$p = 1.4$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an epsilon for discovery, set at 100% at the start and it decreases by 10% until it reaches 1%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to optimize the parameters but can't get better results than a &quot;256&quot; in the board. And the cuadratic loss seems to get stuck at 0.25:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/i9T42.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i9T42.png&quot; alt=&quot;DL4J web UI for neural net info&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something I am missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/EmmanuelMess/2048AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;public enum GameAction {&#xA;    UP, DOWN, LEFT, RIGHT&#xA;}&#xA;&#xA;public final class GameEnvironment {&#xA;&#xA;    public final int points;&#xA;    public final boolean lost;&#xA;    public final INDArray boardState;&#xA;&#xA;    public GameEnvironment(int points, boolean lost, int[] boardState) {&#xA;        this.points = points;&#xA;        this.lost = lost;&#xA;        this.boardState = new NDArray(boardState, new int[] {1, 16}, new int[] {16, 1});&#xA;    }&#xA;}&#xA;&#xA;public class SimpleAgent {&#xA;    private static final Random random = new Random(SEED);&#xA;&#xA;    private static final MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()&#xA;            .seed(SEED)&#xA;            .weightInit(WeightInit.XAVIER)&#xA;            .updater(new AdaGrad(0.5))&#xA;            .activation(Activation.RELU)&#xA;            .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)&#xA;            .weightDecay(0.0001)&#xA;            .list()&#xA;            .layer(new DenseLayer.Builder()&#xA;                    .nIn(16).nOut(8)&#xA;                    .build())&#xA;            .layer(new OutputLayer.Builder()&#xA;                    .nIn(8).nOut(4)&#xA;                    .lossFunction(LossFunctions.LossFunction.SQUARED_LOSS)&#xA;                    .build())&#xA;            .build();&#xA;    MultiLayerNetwork Qnetwork = new MultiLayerNetwork(conf);&#xA;&#xA;    private GameEnvironment oldState;&#xA;    private GameEnvironment currentState;&#xA;    private INDArray oldQuality;&#xA;&#xA;    private GameAction lastAction;&#xA;&#xA;    public SimpleAgent() {&#xA;        Qnetwork.init();&#xA;        ui();&#xA;    }&#xA;&#xA;    public void setCurrentState(GameEnvironment currentState) {&#xA;        this.currentState = currentState;&#xA;    }&#xA;&#xA;    private final ArrayList&amp;lt;INDArray&amp;gt; input = new ArrayList&amp;lt;&amp;gt;();&#xA;    private final ArrayList&amp;lt;INDArray&amp;gt; output = new ArrayList&amp;lt;&amp;gt;();&#xA;    private final ArrayList&amp;lt;Double&amp;gt; rewards = new ArrayList&amp;lt;&amp;gt;();&#xA;&#xA;    private int epsilon = 100;&#xA;&#xA;    public GameAction act() {&#xA;        if(oldState != null) {&#xA;            double reward = currentState.points - oldState.points;&#xA;&#xA;            if (currentState.lost) {&#xA;                reward = 0;&#xA;            }&#xA;&#xA;            input.add(oldState.boardState);&#xA;            output.add(oldQuality);&#xA;            rewards.add(reward);&#xA;&#xA;            if (currentState.lost || input.size() == 20) {&#xA;                for(int i = 0; i &amp;lt; rewards.size(); i++) {&#xA;                    double discount = 1.4;&#xA;                    double discountedReward = 0;&#xA;&#xA;                    for(int j = i; j &amp;lt; rewards.size(); j++) {&#xA;                        discountedReward += rewards.get(j) * Math.pow(discount, j - i);&#xA;                    }&#xA;&#xA;                    rewards.set(i, lerp(discountedReward, 1024));&#xA;                }&#xA;&#xA;                ArrayList&amp;lt;DataSet&amp;gt; dataSets = new ArrayList&amp;lt;&amp;gt;();&#xA;&#xA;                for(int i = 0; i &amp;lt; input.size(); i++) {&#xA;                    INDArray correctOut = output.get(i).putScalar(lastAction.ordinal(), rewards.get(i));&#xA;&#xA;                    dataSets.add(new DataSet(input.get(i), correctOut));&#xA;                }&#xA;&#xA;                Qnetwork.fit(DataSet.merge(dataSets));&#xA;&#xA;                input.clear();&#xA;                output.clear();&#xA;                rewards.clear();&#xA;            }&#xA;&#xA;            epsilon = Math.max(1, epsilon - 10);&#xA;        }&#xA;&#xA;        oldState = currentState;&#xA;        oldQuality = Qnetwork.output(currentState.boardState);&#xA;&#xA;        GameAction action;&#xA;&#xA;&#xA;        if(random.nextInt(100) &amp;lt; 100-epsilon) {&#xA;            action = GameAction.values()[oldQuality.argMax(1).getInt()];&#xA;        } else {&#xA;            action = GameAction.values()[new Random().nextInt(GameAction.values().length)];&#xA;        }&#xA;&#xA;        lastAction = action;&#xA;&#xA;        return action;&#xA;    }&#xA;&#xA;    private static double lerp(double x, int maxVal) {&#xA;        return x/maxVal;&#xA;    }&#xA;&#xA;    private void ui() {&#xA;        UIServer uiServer = UIServer.getInstance();&#xA;        StatsStorage statsStorage = new InMemoryStatsStorage();&#xA;        uiServer.attach(statsStorage);&#xA;        Qnetwork.setListeners(new StatsListener(statsStorage));&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14892" LastEditorUserId="14892" LastEditDate="2020-06-03T02:54:56.577" LastActivityDate="2020-06-13T07:50:50.290" Title="Help with deep Q learning for 2048 game getting stuck" Tags="&lt;q-learning&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21595" PostTypeId="2" ParentId="21593" CreationDate="2020-06-03T02:45:10.140" Score="14" Body="&lt;p&gt;This is one of the most important issues in the philosophy of artificial intelligence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most famous philosophical argument that attempts to address this issue is the &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/&quot; rel=&quot;noreferrer&quot;&gt;Chinese Room argument&lt;/a&gt; published by the philosopher John Searle in 1980. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The argument is quite simple. Suppose that you are inside a room and you need to communicate (in a written form) with people outside the room in a certain language that you do not &lt;em&gt;understand&lt;/em&gt; (in the particular example given by Searle, Chinese), but you are given the rules to manipulate the characters of this language (for a given input, you have the rules to produce the correct output). If you follow these rules, to the people outside the room, it will seem as if you understand this language, but you don't. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more concrete, when I say &quot;apple&quot;, you understand that it refers to a specific fruit because you have eaten apples and you have a model of the world. That's understanding, according to Searle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most famous mathematical model of computers, the Turing machine, is essentially a system that manipulates symbols, so the Chinese Room argument directly applies to computers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many replies or counterarguments to the CR argument have been discussed, such as&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/#SystRepl&quot; rel=&quot;noreferrer&quot;&gt;system reply&lt;/a&gt; (the symbol manipulator is only a part of the larger system).&lt;/li&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/#RoboRepl&quot; rel=&quot;noreferrer&quot;&gt;robot reply&lt;/a&gt; (the symbol manipulator does not understand the meaning of the symbols because it has not experienced the associated real-world objects, so it suggests that understanding requires a body with sensors and controllers)&lt;/li&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;https://plato.stanford.edu/entries/chinese-room/#BraiSimuRepl&quot; rel=&quot;noreferrer&quot;&gt;brain simulator reply&lt;/a&gt; (the symbol manipulator can actually simulate the activity in the brain of a person that understands the unknown language)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, can we prove that machines really understand? Even before Searle, Turing had already asked the question &lt;a href=&quot;https://academic.oup.com/mind/article/LIX/236/433/986238&quot; rel=&quot;noreferrer&quot;&gt;&quot;Can machines think?&quot;&lt;/a&gt;. To prove this, you need a rigorous definition of understanding and thinking that people agree on. However, many people do not want to agree on a &lt;a href=&quot;https://ai.stackexchange.com/a/13100/2444&quot;&gt;definition of intelligence&lt;/a&gt; and understanding (hence the many counterarguments to the CR argument). So, if you want to prove that machines understand, you need to provide a proof with respect to a specific definition of understanding. For example, if you think that understanding is just a side effect of symbol manipulation, you can easily prove that machines understand many concepts (it just follows from the definition of a Turing machine). However, even if understanding was just a side effect (what does a &lt;em&gt;side effect&lt;/em&gt; actually mean in this case?) of symbol manipulation, would a machine be able to understand the same concepts and in the same way that humans understand? It's harder to answer this question because we really do not know if humans only manipulate symbols in our brains.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-03T20:39:58.710" LastActivityDate="2020-06-03T20:39:58.710" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21596" PostTypeId="1" CreationDate="2020-06-03T07:51:32.197" Score="3" ViewCount="41" Body="&lt;p&gt;There are libraries for recognizing individual video frames, but I need to recognize an object in motion. I can recognize a person in every single frame, but I need to know if the person is running or waving. I can recognize a tree in every single frame, but I need to find out if the tree is swaying in the wind. I can recognize a wind turbine in every frame, but I need to know if it's spinning right now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is: do technologies, libraries, concepts, or algorithms exist for recognizing objects over a certain period of time? For example, I have a series of several pieces of frames, each of which exactly has a person, and need to find out if person are walking or waving their hands.&lt;/p&gt;&#xA;" OwnerUserId="37545" LastActivityDate="2020-06-03T07:51:32.197" Title="Video recognition (specifically video, not individual frames)" Tags="&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21597" PostTypeId="1" AcceptedAnswerId="21602" CreationDate="2020-06-03T08:50:46.637" Score="2" ViewCount="119" Body="&lt;p&gt;Should I use minimax or alpha-beta pruning (or both)? Apparently, alpha-beta pruning prunes some parts of the search tree. &lt;/p&gt;&#xA;" OwnerUserId="37258" LastEditorUserId="2444" LastEditDate="2020-06-03T22:46:33.737" LastActivityDate="2021-02-25T11:21:03.150" Title="Should I use minimax or alpha-beta pruning?" Tags="&lt;comparison&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21598" PostTypeId="1" CreationDate="2020-06-03T08:57:50.683" Score="0" ViewCount="310" Body="&lt;p&gt;First I would like to expand an issue I've been dealing with way too long: Creating a conditional Variational Autoencoder with continuous variables in non-image data ( more specifically, time series). Given one time series, I want to condition it in a continuous number, not discrete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, I know that theoretically is possible and do understand the ELBO formulation as written in (&lt;a href=&quot;https://ai.stackexchange.com/questions/13698/is-there-a-continuous-conditional-variational-autoencoder?newreg=34d4ddf910fc49f99d0e0fab9cf873ff&quot;&gt;Is there a Continuous Conditional Variational Autoencoder?&lt;/a&gt;) and (&lt;a href=&quot;https://stats.stackexchange.com/questions/419961/is-there-a-continuous-conditional-variational-autoencoder&quot;&gt;https://stats.stackexchange.com/questions/419961/is-there-a-continuous-conditional-variational-autoencoder&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However, from the ELBO formulation to an actual implementation I believe there is some gap since my results aren't great&lt;/strong&gt;. I manage to create the autoencoder, but, the decoder &lt;strong&gt;ignores the additional information&lt;/strong&gt; and produces the whole spectrum of the additional information in the generated data, that is, it works just like a VAE). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In images and with discrete labels there are tutorials such as (&lt;a href=&quot;https://towardsdatascience.com/understanding-conditional-variational-autoencoders-cd62b4f57bf8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/understanding-conditional-variational-autoencoders-cd62b4f57bf8&lt;/a&gt;) pointing out how to concatenate the additional information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried two approaches for concatenating the additional information: Just by concatenating the single continuous number in one of the possible architectures below. The second was by expanding the additional information with a linear/dense layer ( For example from 1D to 10D. My reasoning for trying this too was: Given an intermediate layer, let's say with 300 hidden neurons if I concatenate ONE additional number/information to it, is the network REALLY going to tell any difference and properly backpropagate towards its correction?)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;A vanilla CVAE&lt;/strong&gt;: I concatenate additional information after one pass of compressing the time series into an intermediate lower dimension. In the decoder, I concatenate additional information in the sampled data before it is expanded to intermediate hidden dimensions and later to the final, original dimension.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;A GRU CVAE&lt;/strong&gt;: I concatenate the additional information after one pass of a first GRU layer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;CNN-CVAE&lt;/strong&gt;: I concatenate additional information after all convolution passes, just before it is processed by linear layers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In all of these, the C-VAE does manage to generate output (as VAE) but ignoring the conditional vector. I also got suspicious of the posterior collapse issue and tried to follow what is recommended in Fixing the Broken ELBO (&lt;a href=&quot;https://arxiv.org/abs/1711.00464&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1711.00464&lt;/a&gt;), by adding a sigma term to force/control the rate term in the loss function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Iu19o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Iu19o.png&quot; alt=&quot;Traing-Val loss&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Kp0Hw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kp0Hw.png&quot; alt=&quot;Distortion loss: Here I used simple mean squared error as reconstruction error&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mfE3B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mfE3B.png&quot; alt=&quot;Rate loss: Simple KLD divergence loss form a normal distribution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I am getting a little frustrated by spending so much time in this, I was wondering if anyone could help by pointing out directions, papers or explaining &lt;strong&gt;how you actually dealt with the issue&lt;/strong&gt; in past or current experience of conditioning (&lt;strong&gt;implementing in code&lt;/strong&gt;) additional information of continuous nature to time series in a VAE framework. I would appreciate it!&lt;/p&gt;&#xA;" OwnerUserId="37548" LastEditorUserId="30725" LastEditDate="2020-06-03T09:41:55.363" LastActivityDate="2020-06-03T09:41:55.363" Title="Conditional Variational Autoencoder - NON Image Data" Tags="&lt;generative-model&gt;&lt;time-series&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21599" PostTypeId="2" ParentId="18567" CreationDate="2020-06-03T09:01:49.367" Score="0" Body="&lt;p&gt;After some research on the subject, I found a solution to my problem of steady state errors in continuous control using DDPG:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I added a reward component based on the integral of the error. This yields maximum reward if the error integral is zero and yields lower rewards down to 0 if the integral has some value in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This integral component in the reward was found to be very effective, but introduces some overshoot after set-point changes. By limiting the integral to quite small quantities, this behavior could be overcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All these findings are detailed out in the &quot;Reward Engineering&quot; section of my master's thesis. Please have a look into &lt;a href=&quot;https://github.com/opt12/Markov-Pilot/tree/master/thesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/opt12/Markov-Pilot/tree/master/thesis&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be glad to get feedback on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards, Felix&lt;/p&gt;&#xA;" OwnerUserId="25972" LastActivityDate="2020-06-03T09:01:49.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21600" PostTypeId="2" ParentId="17774" CreationDate="2020-06-03T09:05:12.910" Score="0" Body="&lt;p&gt;After some research on the subject, I found a possible solution to my problem of high frequency oscillations in continuous control using DDPG:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I added a reward component based on the actuator movement, i. e. the delta of actions from one step to the next.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excessive action changes are punished now and this could mitigate the tendency to oscillate. The solution is nnot really perfect, but it works for the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This finding is detailed out in the &quot;Reward Engineering&quot; section of my master's thesis. Please have a look into &lt;a href=&quot;https://github.com/opt12/Markov-Pilot/tree/master/thesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/opt12/Markov-Pilot/tree/master/thesis&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be glad to get feedback on it. And I'll be glad to hear better solutions than adding a delta-punishment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards, Felix&lt;/p&gt;&#xA;" OwnerUserId="25972" LastActivityDate="2020-06-03T09:05:12.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21601" PostTypeId="2" ParentId="21593" CreationDate="2020-06-03T09:22:36.153" Score="5" Body="&lt;p&gt;I recently came across a neat definition of understanding in Roger Schank's &lt;em&gt;Dynamic Memory&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you store everything you experience in your memory, but you need to index it in order to be able to use it for processing. Obviously, all experiences are slightly different, eg going to a restaurant is broadly the same, but the details vary. So you need to abstract away the details and store those only if necessary (eg if the food or service was particularly good or bad). Otherwise you just store a general template (or 'script') of the event.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your memory (note: this is modeled, not neurologically correct) you thus have a whole set of event scripts that you can retrieve. So currently I would be accessing my reply-to-stack-exchange-question script to guide me how to best write this answer without getting downvoted for ludicrous claims etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding, then, would be to receive (through sensory input, or language) an event, and putting it into the right area in your memory. So if I told you I just went to Burger King, you would understand it when this activates your fast-food-restaurant memory set. If I then told you I went there to wipe the floor, it should instead activate cleaning-job, rather than fast-food-restaurant. So you understand the sequence &quot;I went to Burger King to clean the floor&quot; by linking it to the correct memory region. If a computer then responded with &quot;What did you eat?&quot; it would clearly not have understood the input. But a response of &quot;Do you get free food for working there?&quot; would indicate some level of comprehension/understanding, as it might recognise that people working in food outlets might get free food as a work-related benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you experience something completely new, you recognise it as a new experience, and start a new cluster of experiences. For example, if you have been to restaurants before, but never to fast food ones. The first time it will be strange and different, but you remember it as differences to the existing restaurant script. Over time it becomes strong enough (assuming to go to more fast-food restaurants), and it will become its own area, still linked to restaurants, but also not quite the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I like about this is that it is a generic mechanism, rather than an explicit processing of content. It is based on learning and experience, which I believe are key aspects of intelligent behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; This answer is more concerned with trying to find a workable definition of what it means to &lt;em&gt;comprehend&lt;/em&gt; something, rather than trying to operationalise it in a dialogue system. You can probably pass the Turing test with some clever tricks, without any comprehension at all. But the point is, what does it mean to understand something? And in the current definition it means to classify related events together, and to recognise similarities and differences between similar experiences. The reaction (ie a response) is not the understanding itself, but only a reflection of the internal state that would demonstrate understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference to a neural network is, I would guess, that it can cope with a broad range of experiences, where a NN would need vast amounts of training data (as it doesn't comprehend). Comprehension involves compression of information through abstraction and evaluating differences. This is still a hard problem, and I'd think difficult to achieve just with automated machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 2:&lt;/strong&gt; With regards to the Turing Test, in a way it goes back to deep philosophical points about empiricism. How do you know the world around you exists? You can see it. But how do you know your eyes tell you the true picture? You can quickly descend into a Matrix-like scenario where you don't know anything for certain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Turing Test is a proxy for showing understanding. You don't know the computer understands what you say, so you observe its responses and interpret them accordingly. Just like at school: the teachers asks a question, and from the pupils' answers infers whether they show understanding. If you simply regurgitate a memorised answer, that's not understanding. If you paraphrase in different words, that shows some sort of comprehension. If you draw analogies to similar issues and analyse why and how they are distinct, now there you show that you really get it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because we cannot inspect the internal state of a pupil, we cannot measure objectively whether they understood something. We only have communication as an interface between our mind and theirs, and so far chatbots have focused on getting that right. But I think what we really need is to work on memory and memory processing to get further towards comprehension or understanding. And I say this as a computational linguist who specialises in the language parts...&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-06-04T10:05:26.260" LastActivityDate="2020-06-04T10:05:26.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21602" PostTypeId="2" ParentId="21597" CreationDate="2020-06-03T10:35:26.167" Score="5" Body="&lt;p&gt;Both algorithms should give the same answer. However, their main difference is that alpha-beta does not explore all paths, like minimax does, but prunes those that are guaranteed not to be an optimal state for the current player, that is max or min. So, alpha-beta is a better implementation of minimax.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the time complexities of both algorithms&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Minimax: &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^d)$&lt;/span&gt;,  &lt;/li&gt;&#xA;&lt;li&gt;Alpha-beta (best-case scenario): &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(b^{d/2})  = \mathcal{O}(\sqrt{b^d})$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is an average branching factor with a search depth of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; plies.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="2444" LastEditDate="2020-06-04T11:19:50.057" LastActivityDate="2020-06-04T11:19:50.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21603" PostTypeId="1" CreationDate="2020-06-03T11:23:44.193" Score="1" ViewCount="96" Body="&lt;p&gt;Is there any good tutorials about training reinforcement learning agent from raw pixels using PyTorch? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand the official PyTorch tutorial. I want to train the agent on the atari breakout environment. Unfortunately, I failed to train the agent on the RAM version. Now, I am looking for a way to train the agent from raw pixels.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-06-03T22:41:57.713" LastActivityDate="2020-06-05T06:12:52.183" Title="Are there any good tutorials about training RL agent from raw pixels using PyTorch?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21604" PostTypeId="1" CreationDate="2020-06-03T12:13:49.813" Score="0" ViewCount="36" Body="&lt;p&gt;I have an excel sheet filled with my own personal appreciations of movies I've watched, and I want to use it to train an AI model so that it can predict if I'll like a specific movie or not, based on the ones I've already seen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data is formatted as following (just a sample, the spreadsheet is filled with hundreds of movies):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/84df6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/84df6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I would like to use all the columns to train my model. Because I am going to say if I liked the movie or not, I know it will be Supervised Learning. I already cleaned the data so there's no blank or missing data, but I do not know how to train my model using every column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If required, I can be more specific on something, just ask and I'll edit the post.&lt;/p&gt;&#xA;" OwnerUserId="32862" LastActivityDate="2021-03-06T21:04:55.520" Title="Train a model using a multi-column text-filled excel sheet" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;supervised-learning&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21605" PostTypeId="2" ParentId="21603" CreationDate="2020-06-03T12:16:05.900" Score="1" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/dusty-nv/jetson-reinforcement&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jetson reinforcement&lt;/a&gt; for my quadcopter reinforcement in simulation. Maybe it will help you. because you can create AI agents that can learn from the interactive environment, gather experience, and system of reward with deep RL. You can also use end-to-end neural networks that translate raw pixels into action as per your need and use that RL-trained agent to complete complex tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best thing is you can easily transfer RL-agent which simulated in the simulator to real-world robots. and We are using multiple Nano's easily to perform the complex tasks of navigation and co-ordination of Our Quad-copter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nvidia.com/en-us/about-nvidia/webinar-portal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Webinar of Deep RL on Jetson&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://groups.google.com/d/forum/deep-reinforcement-learning-on-jetson-nano&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google group to get help for Deep RL on jetson nano&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps you.&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="32861" LastEditDate="2020-06-05T06:12:52.183" LastActivityDate="2020-06-05T06:12:52.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21606" PostTypeId="1" AcceptedAnswerId="21640" CreationDate="2020-06-03T12:22:39.937" Score="2" ViewCount="57" Body="&lt;p&gt;If a neural network has a limited number of neuron parameters to find, -let's say only 1000 parameters-, it is generally better to spend the parameters on weights or neuron bias?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if each neuron has 2 weights and one bias, it uses 3 parameters per neuron, so only 333 neurons would be available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if each neuron uses no bias parameter, then 500 neurons are available with 1000 parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm concerned with overfiting by using too many parameters, so I want to minimize the number of parameters meanwhile maximizing the quality of the result.&lt;/p&gt;&#xA;" OwnerUserId="37558" LastActivityDate="2020-06-04T17:26:01.147" Title="Is better to spend parameters on weights or bias?" Tags="&lt;neural-networks&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21607" PostTypeId="1" CreationDate="2020-06-03T12:25:28.967" Score="1" ViewCount="34" Body="&lt;p&gt;Deep belief networks (DBNs) are generative models, where, usually, you sample by thermalising the deepest layer (as it's a restricted Boltzmann machine), and then forward propagating a sample towards the visible layer to get a sample from the learned distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is less flexible sampling than in a single layer DBN: a restricted Boltzmann machine. There, we can start our sampling chain at any state we want, and get samples &quot;around&quot; that state. In particular, we can clamp some visible nodes &lt;span class=&quot;math-container&quot;&gt;$\{v_i\}$&lt;/span&gt; and get samples from the conditional probability &lt;span class=&quot;math-container&quot;&gt;$(v_j|\{v_i\})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do something similar in DBNs? When we interpret the non-RBM layers as RBMs by removing directionality, can we treat it as a deep Boltzmann machine and start sampling at e.g. a training example again?&lt;/p&gt;&#xA;" OwnerUserId="37557" LastEditorUserId="37557" LastEditDate="2020-06-04T11:57:47.753" LastActivityDate="2020-06-04T11:57:47.753" Title="How do I sample conditionally from deep belief networks?" Tags="&lt;generative-model&gt;&lt;conditional-probability&gt;&lt;restricted-boltzmann-machine&gt;&lt;deep-belief-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21608" PostTypeId="2" ParentId="21603" CreationDate="2020-06-03T12:55:49.150" Score="1" Body="&lt;p&gt;I always found that towards data science articles can be a good source for code for these types of problems. They usually have a git repo with everything you need and walk through the less obvious steps in the article. &lt;a href=&quot;https://towardsdatascience.com/beating-video-games-with-deep-q-networks-7f73320b9592&quot; rel=&quot;nofollow noreferrer&quot;&gt;This article may be of interest to you.&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-03T12:55:49.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21609" PostTypeId="1" CreationDate="2020-06-03T13:06:32.853" Score="0" ViewCount="53" Body="&lt;p&gt;I am trying to detect forgeries done after a document is scanned by a scanner.&lt;/p&gt;&#xA;&lt;p&gt;I already tried to access the metadata, and, if it is edited with any software after scanning, then it is easily detected by the metadata, like &lt;a href=&quot;https://exifdata.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;exif data&lt;/a&gt;, but the metadata can also be forged. I also tried &lt;a href=&quot;https://en.wikipedia.org/wiki/Error_level_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELA&lt;/a&gt; and &lt;a href=&quot;https://www.geeksforgeeks.org/image-splicing-set-1-introduction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;splicing&lt;/a&gt; and a trained neural network model on that, the result of ELA and splicing is best on images that are captured from Camera around 99.98% on Casia-1 and Casia-2 but for scanned documents like Exam result, degree certificate, etc..., a technique(Splicing + NN and ELA +NN) not working.&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-09T06:10:20.327" Title="How to detect forgery on scanned document images?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21610" PostTypeId="1" AcceptedAnswerId="21611" CreationDate="2020-06-03T15:50:26.320" Score="2" ViewCount="82" Body="&lt;p&gt;In the RL textbook by Sutton &amp;amp; Barto section 7.4, the author talked about the &quot;True online TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;)&quot;. The figure (7.10 in the book) below shows the algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of each step, &lt;span class=&quot;math-container&quot;&gt;$V_{old} \leftarrow V(S')$&lt;/span&gt; and also &lt;span class=&quot;math-container&quot;&gt;$S \leftarrow S'$&lt;/span&gt;. When we jump to next step, &lt;span class=&quot;math-container&quot;&gt;$\Delta \leftarrow V(S') - V(S')$&lt;/span&gt;, which is 0. It seems that &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; is always going to be 0 after step 1. If that is true, it does not make any sense to me. Can you please elaborate on how  &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; is updated? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vxG22.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vxG22.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36120" LastEditorUserId="2444" LastEditDate="2020-06-04T11:38:55.353" LastActivityDate="2020-06-04T11:38:55.353" Title="How is $\Delta$ updated in true online TD($\lambda$)?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;&lt;td-lambda&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21611" PostTypeId="2" ParentId="21610" CreationDate="2020-06-03T16:21:45.447" Score="4" Body="&lt;p&gt;Let us denote the state we are in at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt;. Then at iteration &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; we create a placeholder &lt;span class=&quot;math-container&quot;&gt;$V_{old} = V(S_{t+1})$&lt;/span&gt; for the state we will transition into. We then update the value function &lt;span class=&quot;math-container&quot;&gt;$V(s) \; \forall s \in \mathcal{S}$&lt;/span&gt; - i.e. we update the value function for &lt;em&gt;all&lt;/em&gt; states in our state space. Let us denote this updated value function by &lt;span class=&quot;math-container&quot;&gt;$V'(S)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At iteration &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; we calculate &lt;span class=&quot;math-container&quot;&gt;$\Delta = V'(S_{t+1}) - V_{old} = V'(S_{t+1}) - V(S_{t+1})$&lt;/span&gt;, which does not necessarily equal 0 because the placeholder &lt;span class=&quot;math-container&quot;&gt;$V_{old}$&lt;/span&gt; was created using the value function before the last update. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-03T18:41:12.397" LastActivityDate="2020-06-03T18:41:12.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21612" PostTypeId="1" CreationDate="2020-06-03T16:36:00.457" Score="0" ViewCount="29" Body="&lt;p&gt;I would like to build a neural network (using TensorFlow) that is able to take two animals, and replace a feature in the second with one in the first. For example, if given a dog and cat, the cat's ears will be replaced by dog ears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of using a CNN and photoshopping images to make the training set, but that is going to take forever. Maybe a deep neural network would work better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Followup: Is there a way to generalize this beyond animals?&lt;/p&gt;&#xA;" OwnerUserId="37569" LastEditorUserId="2444" LastEditDate="2020-06-03T16:55:31.850" LastActivityDate="2020-06-03T16:55:31.850" Title="How can I build a model that replaces a feature of one image with another feature?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21614" PostTypeId="1" CreationDate="2020-06-03T18:33:02.917" Score="1" ViewCount="36" Body="&lt;p&gt;I'm working on a time series forecasting task, and, in some specific cases, I don't need perfect accuracy, but the network cannot by any means miss by a lot. So, in detriment of a smaller mean error, I want to have fewer big mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions of loss functions or other methods to solve this issue? &lt;/p&gt;&#xA;" OwnerUserId="37572" LastEditorUserId="2444" LastEditDate="2020-06-03T22:37:42.927" LastActivityDate="2020-06-03T22:37:42.927" Title="What are some good loss functions used to minimize extreme errors in regression and time series forecasting?" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21615" PostTypeId="2" ParentId="21562" CreationDate="2020-06-03T19:02:46.640" Score="3" Body="&lt;p&gt;A first question that I think is important to consider is: &lt;strong&gt;do you expect the data that you're dealing with to be changing over time&lt;/strong&gt; (i.e. do you expect there to be &lt;em&gt;concept drift&lt;/em&gt;)? This could be any kind of change. Simply changes in how frequent certain inputs are, changes in how frequent positives/negatives are, or even changes in relations between inputs and ground-truth positive/negative labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do &lt;strong&gt;not&lt;/strong&gt; expect there to be concept drift, I'd almost consider suggesting that you may not have that big of a problem. It might be worth not doing anything at all with the data you receive online, and just stick to what you learned initially from offline data. Or you could try to use those few extra predicted-positive samples that you get for finetuning. You'd just have to be careful not too change your model too much based on this, because you know that you're not receiving a representative sample of &lt;strong&gt;all&lt;/strong&gt; the data anymore here, so you might bias your model if you pay too much attention to only this online data relative to the offline data.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I guess the question becomes much more interesting if you do expect there to be concept drift, and it also seems likely that you are indeed dealing with this in most of the situations that would match the problem description. In this case, you will indeed want to make good use of the new data that you get online, because it can allow you to adapt to changes in the data that you're dealing with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, one &quot;solution&quot; could be to just... ignore the problem that you're only learning online from a biased sample of all your data (only from the predicted-positives), and just learn anyway. This might actually not perform too badly. Unless your model is really incredibly good already, you'll likely still get false positives, and so also still be able to learn from some of those -- you're not learning exclusively from positives. Still, the false positives won't be representative of &lt;strong&gt;all&lt;/strong&gt; the negatives, so you still have bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only better solution I can think of is relaxing this assumption:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Once deployed, humans review and check for correctness &lt;em&gt;only items predicted positive&lt;/em&gt; by the algorithm. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can still have the humans focus on predicted positives, but maybe have them inspect a predicted-negative also sometimes. Not often, just a few times. You can think of this as doing &lt;strong&gt;exploration&lt;/strong&gt; like you would in reinforcement learning settings. You could do it randomly (randomly pick predicted negatives with some small probability), but you could also be smarter about it and explicitly target exploration of instances that your model is &quot;unsure&quot; about, or instances that are unlike data you've seen before (to specifically target concept drift).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a paper about something &lt;strong&gt;very similar&lt;/strong&gt; to this right here: &lt;a href=&quot;https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16183&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adapting to Concept Drift in Credit Card Transaction Data Streams Using Contextual Bandits and Decision Trees&lt;/a&gt;. Here the assumption is that we're dealing with (potentially fraudulent) transactions, of which we can pick out and manually inspect a very small sample online. The only real difference in this paper is that we assumed that different transactions also had different monetary &quot;rewards&quot; for getting correctly caught as positives, based on the transaction amount. So a transaction of a very high amount could be worth inspecting even if we predicted a low probability of being fraudulent, whereas a transaction of a very low amount might be ignored even if it had a higher predicted probability of being fraudulent.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;what's a meaningful metric to monitor to ensure that the performance of the model is not degrading?* (given the constraint specified here, F1 score is unknown).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Having a labelled evaluation set for this could be useful if possible... but it also might not be representative if &lt;strong&gt;concept drift&lt;/strong&gt; is expected to be a major issue in your problem setting (because I suppose that the concept drift that you deal with online would not be reflected in an older, labelled evaluation set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just keeping track of things that you can measure online, like precision, and how it changes over time, could be useful enough already. With some additional assumptions, you could get rough estimates of other metrics. For instance, if you assume that the ratio &lt;span class=&quot;math-container&quot;&gt;$\frac{TP + FN}{FP + TN}$&lt;/span&gt; between ground-truth-positives and ground-truth-negatives remains constant (remains the same as it was in your offline, labelled data), you could also try to extrapolate approximately how many positives you've missed out on. If your precision is dropping over time (your true positives are getting lower), you know -- assuming that fraction remains constant -- that your false negatives somewhere else in the dataset must be growing by approximately the same absolute number.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-06-03T19:02:46.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21617" PostTypeId="1" CreationDate="2020-06-03T20:22:10.297" Score="1" ViewCount="32" Body="&lt;p&gt;The Kantorovich-Rubinstein duality for the optimal transport problem implies that the Wasserstein distance between two distributions &lt;span class=&quot;math-container&quot;&gt;$\mu_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mu_2$&lt;/span&gt; can be computed as (equation 2 in section 3 in the &lt;a href=&quot;http://proceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGAN paper&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W(\mu_1,\mu_2)=\underset{f\in \text{1-Lip.}}{\sup}\left(\mathbb{E}_{x\sim \mu_1}\left[f\left(x\right)\right]-\mathbb{E}_{x \sim \mu_2}\left[f\left(x\right)\right]\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Under what conditions can one find the optimal &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that achieves the maximum? Is it possible to have an analytical expression for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that achieves the maximum in such scenarios?&lt;/p&gt;&#xA;&lt;p&gt;Any help is deeply appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28286" LastEditorUserId="2444" LastEditDate="2021-01-25T19:00:44.080" LastActivityDate="2021-01-25T19:00:44.080" Title="Under what conditions can one find the optimal critic in WGAN?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;wasserstein-metric&gt;&lt;wasserstein-gan&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21618" PostTypeId="1" CreationDate="2020-06-03T20:55:29.253" Score="1" ViewCount="95" Body="&lt;p&gt;Can we solve an &lt;span class=&quot;math-container&quot;&gt;$8 \times 8$&lt;/span&gt; sliding puzzle using a random-restart hill climbing technique (steepest-ascent)? If yes, how much computing power will this need? And what is the maximum &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; that can be solved normally (e.g. with a Google's colab instance)?&lt;/p&gt;&#xA;" OwnerUserId="37575" LastEditorUserId="37575" LastEditDate="2020-06-04T00:33:06.827" LastActivityDate="2020-06-04T00:33:06.827" Title="Can we solve an $8 \times 8$ sliding puzzle using hill climbing?" Tags="&lt;algorithm&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;local-search&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21619" PostTypeId="2" ParentId="21590" CreationDate="2020-06-03T22:13:00.043" Score="3" Body="&lt;p&gt;What you are referring to as the situation where &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;some indexes are not available&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is simply the situation where some actions are not available/valid in some state. So, yes, the &lt;span class=&quot;math-container&quot;&gt;${\arg \max }$&lt;/span&gt; will be calculated based only on the available actions in that state. More formally, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\underset{a \in \mathcal{A}(s)}{\arg \max } \, Q(s, a)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; has been initialized for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}^{+}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a \in \mathcal{A}(s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}(s)$&lt;/span&gt; is defined as the set of all actions available in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&#xA;See Sutton and Barto's &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intro to RL book&lt;/a&gt;, chapter 6 (the part on Q-learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same book (chapter 3), the authors state that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;To simplify notation&lt;/strong&gt;, we sometimes assume the special case in which&#xA;  the action set is the same in all states and write it simply as &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Therefore, in many sources, you may see the &lt;span class=&quot;math-container&quot;&gt;$\arg \max$&lt;/span&gt; expressed as &lt;span class=&quot;math-container&quot;&gt;$\underset{a \in \mathcal{A}} {\arg \max}$&lt;/span&gt; or even &lt;span class=&quot;math-container&quot;&gt;$\underset{a} {\arg \max}$&lt;/span&gt;. It's implicit that it is taken with respect to the set of actions &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}(s)$&lt;/span&gt; available in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-06-03T22:13:00.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21620" PostTypeId="1" CreationDate="2020-06-03T23:57:24.033" Score="1" ViewCount="115" Body="&lt;p&gt;In the appendix of &lt;a href=&quot;https://arxiv.org/abs/1807.03748&quot; rel=&quot;nofollow noreferrer&quot;&gt;Representation Learning with Contrastive Predictive Coding&lt;/a&gt;, van den Oord et al. prove that optimizing InfoNCE is equivalent to maximize the mutual information between input image &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; and the context latent &lt;span class=&quot;math-container&quot;&gt;$c_t$&lt;/span&gt; as follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FximA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FximA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x_{t+k}$&lt;/span&gt; is the image at time step &lt;span class=&quot;math-container&quot;&gt;$t+k$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X_{neg}$&lt;/span&gt; is a set of negative samples that do not appear in the sequence &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; belongs to, and &lt;span class=&quot;math-container&quot;&gt;$N-1$&lt;/span&gt; is the negative samples used to compute InfoNCE.&lt;/p&gt;&#xA;&lt;p&gt;I'm confused about Equation &lt;span class=&quot;math-container&quot;&gt;$(8)$&lt;/span&gt;. van den Oord et al. stressed that Equation &lt;span class=&quot;math-container&quot;&gt;$(8)$&lt;/span&gt; becomes more accurate as &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; increases, but I cannot see why. Here's my understanding, for &lt;span class=&quot;math-container&quot;&gt;$x_j\in X_{neg}$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$p(x_j|c_t)\le p(x_j)$&lt;/span&gt; . Therefore, &lt;span class=&quot;math-container&quot;&gt;$\sum_{x_j\in X_{neg}}{p(x_j|c_t)\over p(x_j)}\le N-1$&lt;/span&gt; and this does not become accurate as &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; increases. In fact, I think the gap between the left and right of &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; increases as &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; increases. Do I make any mistake?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-03T23:57:24.033" Title="Confusion about the proof that optimizing InfoNCE equals to maximizing mutual information" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21622" PostTypeId="5" CreationDate="2020-06-04T02:54:18.723" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-IDF&lt;/a&gt; (Term Frequency-Inverse Document Frequency) is a statistical measure that evaluates how relevant a word is to a document in a collection of documents. This is done by multiplying two metrics: how many times a word appears in a document, and the inverse document frequency of the word across a set of documents i.e, weight based on its term frequency (TF) and inverse document frequency (IDF). The terms with higher weight scores are considered to be more important.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-05T20:19:58.313" LastActivityDate="2020-06-05T20:19:58.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21623" PostTypeId="4" CreationDate="2020-06-04T02:54:18.723" Score="0" Body="For questions related to TF-IDF(Term Frequency — Inverse Document Frequency) a technique to quantify a word in documents" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-05T05:03:46.587" LastActivityDate="2020-06-05T05:03:46.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21624" PostTypeId="5" CreationDate="2020-06-04T02:58:06.957" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://coq.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coq&lt;/a&gt; is a formal proof management system. It provides a formal language to write mathematical definitions, executable algorithms, and theorems together with an environment for semi-interactive development of machine-checked proofs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Coq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coq&lt;/a&gt; implements a program specification and mathematical higher-level language called Gallina that is based on an expressive formal language called the Calculus of Inductive Constructions that itself combines both a higher-order logic and a richly-typed functional programming language.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-05T20:21:15.690" LastActivityDate="2020-06-05T20:21:15.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21625" PostTypeId="4" CreationDate="2020-06-04T02:58:06.957" Score="0" Body="For questions related to Coq, an environment for developing mathematical facts" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-05T20:21:10.943" LastActivityDate="2020-06-05T20:21:10.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21626" PostTypeId="5" CreationDate="2020-06-04T03:02:27.153" Score="0" Body="&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_machine#Deep_Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;deep Boltzmann machine (DBM)&lt;/a&gt; is a type of binary pairwise Markov random field (undirected probabilistic graphical model) with multiple layers of hidden random variables. It is a network of symmetrically coupled stochastic binary units. It comprises a set of visible units and layers of hidden units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like DBNs, DBMs can learn complex and abstract internal representations of the input in tasks such as object or speech recognition, using limited, labeled data to fine-tune the representations built using a large set of unlabeled sensory input data. &lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-05T20:20:06.970" LastActivityDate="2020-06-05T20:20:06.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21627" PostTypeId="4" CreationDate="2020-06-04T03:02:27.153" Score="0" Body="For questions related to deep Boltzmann machine (DBM) which is a type of binary pairwise Markov random field (undirected probabilistic graphical model) with multiple layers of hidden random variables." OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-05T20:20:02.563" LastActivityDate="2020-06-05T20:20:02.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21628" PostTypeId="1" AcceptedAnswerId="21629" CreationDate="2020-06-04T03:35:09.387" Score="4" ViewCount="227" Body="&lt;p&gt;I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in Section 5.6 of the book, 3rd line, first paragraph, it is written:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Whereas in Chapter 2 we averaged rewards, in Monte Carlo methods we average returns.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does it mean? Are rewards and returns different things?&lt;/p&gt;&#xA;" OwnerUserId="36710" LastEditorUserId="2444" LastEditDate="2020-06-04T11:14:18.527" LastActivityDate="2020-06-05T17:34:33.320" Title="Is there any difference between reward and return in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;rewards&gt;&lt;return&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21629" PostTypeId="2" ParentId="21628" CreationDate="2020-06-04T04:22:51.343" Score="2" Body="&lt;p&gt;&lt;strong&gt;Return&lt;/strong&gt; refers to the &lt;strong&gt;total discounted reward&lt;/strong&gt;, starting from the current timestep.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-06-04T04:22:51.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21632" PostTypeId="1" CreationDate="2020-06-04T07:18:40.760" Score="3" ViewCount="174" Body="&lt;p&gt;I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and the very famous &quot;Illustrated Transformer&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have a training example of &quot;I am a student&quot; and I have the respective French as &quot;Je suis etudient&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how these 3 words are converted to 4 words. What are the query, keys, values? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my understanding of the topic so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The encoder part is:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Query: a single word embedded in a vector form. such as &quot;I&quot; expressed as a vector of length 5 as &lt;span class=&quot;math-container&quot;&gt;$[.2, 0.1, 0.4, 0.9, 0.44]$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Keys: the matrix of all the vectors or in simple words, a matrix that has all the words from a sentence in the form of embeddings.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Values = Keys&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For decoder:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Query: the input word in the form of a vector (which is output given by the decoder from the previous pass).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Keys = values = outputs from the encoder's layers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BUT there are 2 different attention layers and one of which do not use the encoder's output at all&lt;/strong&gt;. So, what are the keys and values now? (I think they are just like encoder, but just the generated &lt;strong&gt;until that pass&lt;/strong&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="36062" LastEditorUserId="2444" LastEditDate="2020-11-20T12:55:46.150" LastActivityDate="2021-04-19T14:44:32.833" Title="What are the keys and values of the attention model for the encoder and decoder in the &quot;Attention Is All You Need&quot; paper?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21634" PostTypeId="5" CreationDate="2020-06-04T11:39:20.160" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-04T11:39:20.160" LastActivityDate="2020-06-04T11:39:20.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21635" PostTypeId="4" CreationDate="2020-06-04T11:39:20.160" Score="0" Body="For questions related to TD($\lambda$) family of algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-04T11:39:20.160" LastActivityDate="2020-06-04T11:39:20.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21637" PostTypeId="2" ParentId="7926" CreationDate="2020-06-04T16:06:08.963" Score="0" Body="&lt;p&gt;You guys are missing the point completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything I read assumes that being is based on science.  You are trying to fit the reality of being into a brain, that is, into a machine. You know that you exist. A billion sensors of every imaginable type cannot equate to reality. Even with a billion more neurons than what's in the brain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's like the mathematics you use. You believe you can be accurate when the truth is, you cannot be. The position between 2 numbers can never be accurate, they are infinite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's like time. You believe it to be organic when the truth is, there is only the now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try as you may, you will never fit being into an algorithm. The real illusion is that you believe you can.&lt;/p&gt;&#xA;" OwnerUserId="37600" LastActivityDate="2020-06-04T16:06:08.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21638" PostTypeId="1" CreationDate="2020-06-04T16:58:20.537" Score="0" ViewCount="53" Body="&lt;p&gt;I am running a MCTS as part of a game ML algorithm. The game that it is playing can reach the same state via multiple paths. It can also reach states again. This is similar to how a position on a chess board can be reached multiple ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I am doing the MCTS, sometimes it gets stuck in a loop where it wants to run the same moves over and return to a previously visited state. So, like chess, I decided to add draw by repetition. However, this seems to leave me with two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Drastically expand the Nodes of the MCTS and increase the state space. This is because the node is no longer defined by the game state and player turn alone. It also matters the path that was taken to it.&lt;/li&gt;&#xA;&lt;li&gt;Simply &quot;short circuit&quot; the MCTS process itself. When it reaches the same state (as defined by board/player turn) n times, return a value of 0 for simulation. I worry this will backfill along the tree and have unintended consequences. It also seems like bad practice to pollute the ML part with the rules of the game.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there any other solutions I'm missing? Does anyone know how folks handle it for chess?&lt;/p&gt;&#xA;" OwnerUserId="37603" LastEditorUserId="37603" LastEditDate="2020-06-04T20:39:56.610" LastActivityDate="2020-06-04T20:39:56.610" Title="How to set up MCTS and draw by repetition?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21639" PostTypeId="1" CreationDate="2020-06-04T17:05:41.123" Score="1" ViewCount="34" Body="&lt;p&gt;Most of the traditional machine learning algorithms need a feature vector of a constant dimension to predict the label. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithms can be used to predict a class label with a shorter or partial feature vector? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider a search engine. In search engines, when the user types a few letters, the search engine predicts the context of the query and suggests more queries to user. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, how can I predict a class label with an incomplete feature vector? I know one way is to pad the sequence, but I want a better solution.&lt;/p&gt;&#xA;" OwnerUserId="29645" LastEditorUserId="2444" LastEditDate="2020-06-05T23:55:18.567" LastActivityDate="2020-06-05T23:55:18.567" Title="How can I predict the label given a partial feature vector?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;prediction&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21640" PostTypeId="2" ParentId="21606" CreationDate="2020-06-04T17:26:01.147" Score="1" Body="&lt;p&gt;First of all, your estimates are a bit off. If you have 300 neurons, you won't have just 2 weights per neuron, but much more, assuming full connectivity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bias isn't just an extra parameter to fit, it is an important adjustable parameter that sets the offset of the separating hyperplane represented by each neuron. Think of the simple equation &lt;span class=&quot;math-container&quot;&gt;$ax+b$&lt;/span&gt;, there's no way to shift the line unless you use the &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; (bias) part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be especially important for small number of nodes and classification tasks (think perceptrons etc)&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-06-04T17:26:01.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21641" PostTypeId="1" AcceptedAnswerId="21650" CreationDate="2020-06-04T17:52:07.687" Score="4" ViewCount="133" Body="&lt;p&gt;My questions concern a particular formulation of the Bayes error rate from &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayes_error_rate&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;, summarized below.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For a multiclass classifier, the Bayes error rate may be calculated as follows:&#xA;  &lt;span class=&quot;math-container&quot;&gt;$$p = 1 - \sum_{C_i \ne C_{\max,x}} \int_{x \in H_i} P(C_i|x)p(x)\,dx$$&lt;/span&gt;&#xA;  where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an instance, &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is a class into which an instance is classified, &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt; is the area/region that a classifier function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; classifies as &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We are interested in the probability of &lt;em&gt;mis&lt;/em&gt;classifying an instance, so we wish to sum up the probability of each unlikely class label (hence we want to look at &lt;span class=&quot;math-container&quot;&gt;$C_i \ne C_{\max, x}$&lt;/span&gt;). However, the integral is confusing me. We want to integrate an area corresponding to the probability that we choose label &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. But we drew &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt;, the region covered/classified by &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;, so wouldn't &lt;span class=&quot;math-container&quot;&gt;$P(C_i|x) = 1$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think most of my confusion will be resolved if someone can help clarify the intention of the integral. Is it to draw random samples from the total space of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; (the classifier function), and then sum the probabilities from each classified &lt;span class=&quot;math-container&quot;&gt;$C_i \ne C_{\max, x}$&lt;/span&gt;? How does &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; exist in the outer summation before it has been sampled from &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt; in the integral?&lt;/p&gt;&#xA;" OwnerUserId="37604" LastActivityDate="2020-07-03T18:07:46.323" Title="Bayes error rate formula clarification" Tags="&lt;classification&gt;&lt;probability&gt;&lt;bayes&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21642" PostTypeId="1" CreationDate="2020-06-04T19:07:27.447" Score="0" ViewCount="22" Body="&lt;p&gt;I ran a simple experiment of training MNIST by varying the batch size, so each trial at training used a constant batch size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MLmVT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MLmVT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left plot is intuitive to me, since I expect training time to decrease rapidly as I increase the batch size. But the right plot is unintuitive. If I used a particular batchSize for training, then there are (60,000/batchSize) = numBatches for that training run, and the right plot indicates that training time increases &lt;em&gt;linearly&lt;/em&gt; with the number of batches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me this is very surprising, because this suggests that the computation time &lt;em&gt;per batch&lt;/em&gt; is nearly independent of size of each batch. I would have expected the time per batch&lt;/p&gt;&#xA;" OwnerUserId="37609" LastActivityDate="2020-06-04T19:07:27.447" Title="Why is MNIST training time linear with (1/batchSize)?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21643" PostTypeId="1" AcceptedAnswerId="21644" CreationDate="2020-06-04T19:27:43.360" Score="4" ViewCount="113" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf#page=68&quot; rel=&quot;nofollow noreferrer&quot;&gt;equation 3.17 of Sutton and Barto's book&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_*(s, a)=\mathbb{E}[R_{t+1} + \gamma v_*(S_{t+1}) \mid S_t = s, A_t = a]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; here have been replaced with &lt;span class=&quot;math-container&quot;&gt;$v_*(S_{t+1})$&lt;/span&gt;, but no reason has been provided for why this step has been taken. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone provide the reasoning behind why &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$v_*(S_{t+1})$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="2444" LastEditDate="2020-06-04T19:45:05.460" LastActivityDate="2020-06-05T17:08:02.957" Title="Why is $G_{t+1}$ is replaced with $v_*(S_{t+1})$ in the Bellman optimality equation?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;expectation&gt;&lt;return&gt;&lt;bellman-equations&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21644" PostTypeId="2" ParentId="21643" CreationDate="2020-06-04T20:45:36.110" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can someone provide the reasoning behind why &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$v_*(S_{t+1})$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The two things are not usually exactly equal, because &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; is a probability distribution over all possible future returns whilst &lt;span class=&quot;math-container&quot;&gt;$v_*(S_{t+1})$&lt;/span&gt; is a probability distribution derived over all possible values of &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;. These will be different distributions much of the time, but their &lt;em&gt;expectations&lt;/em&gt; are equal, provided the conditions of the expectation match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t+1} \neq v_*(S_{t+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[G_{t+1}] = \mathbb{E}[v_*(S_{t+1})]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;. . . when the conditions that apply to the expectations on each side are compatible. The relevant conditions are &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Same initial state or state/action at given timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (or you could pick any earlier timestep)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Same state progression rules and reward structure (i.e. same MDP)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Same policy&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More details&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; can be given as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s) = \mathbb{E}_\pi[G_t \mid S_t = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you substitute step s' and index &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; you get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s') = \mathbb{E}_\pi[G_{t+1} \mid S_{t+1} = s']$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is the same equation, true by definition, the substitution just shows you how it fits).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to put this into equation 3.17, you need to note that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is OK to substitute terms &lt;em&gt;inside&lt;/em&gt; an expectation if they are equal in separate expections, amd  the conditions &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; apply to both (or are irrelevant to either one or both). So if for example &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_c[Z] = \mathbb{E}_c[X \mid Y]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; are random variables, and you know &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is independent of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; then you can say &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_c[W + 2X \mid Y] = \mathbb{E}_c[W + 2Z \mid Y]$&lt;/span&gt; even if &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; are different distributions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A_{t+1} = a'$&lt;/span&gt; does not need to be specified because it is decided by the same &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; in both &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v(s')$&lt;/span&gt;, making the conditions on the expectation compatible already. So the condition of following &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is compatible with &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[G_{t+1} \mid S_{t} = s, A_{t}=a] = \mathbb{E}_\pi[v_*(S_{t+1}) \mid S_{t} = s, A_{t}=a]$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The expectation over possible &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[v_*(S_{t+1})|S_t=s, A_t=a] = \sum p(s'|s,a)v_*(s')$&lt;/span&gt; is already implied by conditions on the original expectation that the functions are evaluating the same environment - something that is not usually shown in the notation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also worth noting, in 3.17 &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;, but actually the equation holds for any fixed policy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-05T17:08:02.957" LastActivityDate="2020-06-05T17:08:02.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21645" PostTypeId="1" AcceptedAnswerId="21652" CreationDate="2020-06-04T20:57:55.577" Score="2" ViewCount="47" Body="&lt;p&gt;While I was studying the equations for the computation inside GRU and LSTM units, I realized that although the different gates have different Weight matrices, their overall structure is the same. They are all dot products of a weight matrix and their inputs, plus bias, followed by a learned gating activation. Now, the difference between computation depends on the weight matrices being different from each other, that is, those weight matrices are specifically for specializing in the particular tasks like forgetting/keeping etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these matrices are all initialized randomly, and it seems that there's no special tricks in the training scheme to make sure these weight matrices are learned in a manner that the associated gates specialize in their desired tasks. They are all random matrices that kept getting updated with gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how does, for example, a forget gate learn to function as a forgetting unit? Same question applies to others as well. Am I missing a part of the training for these networks? Can we ever say that these units learn truly disentangled functions from each other?&lt;/p&gt;&#xA;" OwnerUserId="37614" LastActivityDate="2020-06-05T04:21:08.853" Title="How do LSTM or GRU gates learn to specialize in their desired tasks?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21646" PostTypeId="1" CreationDate="2020-06-04T21:16:25.123" Score="1" ViewCount="75" Body="&lt;p&gt;Many of multi-armed bandit(MAB) algorithms are used when the total reward is the sum of all rewards. However, in RL, the discounted reward is mainly used. Why is the discounted reward not prevailing in MAB problem, and in what cases is this type of modeling valid and might be better?&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="40671" LastEditDate="2020-11-02T09:50:32.417" LastActivityDate="2021-03-05T10:23:59.113" Title="When discounted MAB is useful?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;multi-armed-bandit&gt;&lt;discount-factor&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21647" PostTypeId="2" ParentId="21312" CreationDate="2020-06-04T21:51:08.687" Score="1" Body="&lt;p&gt;I've actually implemented this game before using deep reinforcement learning. You are dealing with a &lt;em&gt;dynamic&lt;/em&gt; action space here, where the action space may change at each time step of the game (or more generally the MDP). First, let's discuss the actual action spaces in each one of the two phases of Crib (or Cribbage) and formalize the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Phase 1: The Discard&lt;/strong&gt;:In this phase, you are &lt;em&gt;concurrently&lt;/em&gt; discarding  2 cards without respect to order. Therefore, you have a &lt;em&gt;fixed&lt;/em&gt; discrete action space of size &lt;span class=&quot;math-container&quot;&gt;${{6}\choose{2}} = 15$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Phase 2: The Play&lt;/strong&gt;: In this phase, you and your opponent are &lt;em&gt;sequentially&lt;/em&gt; playing one of each of your remaining 4 cards (the original 6 minus the 2 discarded from phase 1). Therefore, you have a discrete action space of size &lt;span class=&quot;math-container&quot;&gt;$4! = 24$&lt;/span&gt;. Here's the catch - not every one of these actions is legal. The current state of the game restricts which cards you are allowed to play (the sum of all cards currently played must not be greater than 31). Since you do not know your opponent's cards and/or policy, you do not know which of these 24 actions are valid. To remedy this, the action space should dictate which of your remaining cards may be played &lt;strong&gt;at the current time step&lt;/strong&gt;. Thus, you have a &lt;em&gt;dynamic&lt;/em&gt; discrete action space of size 1, 2, 3, or 4 at each time step. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I make an action space for these actions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since you didn't specify any implementation standard (e.g. OpenAI Gym), there are multiple paths to take, and they usually depend on your implementation of the state feature vector. In my own implementation, I tinkered with two possible state representations, which are fairly simple to describe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Possibility 1: Separate State Representations for each Phase&lt;/strong&gt;: In phase 1, you need to know the cards in your hand and the scores; i.e., the state feature vector could be encoded as a list of &lt;code&gt;[card0, card1, card2, card3, card4, card5, your score, opponent score]&lt;/code&gt;. This state represents all information known about the game during phase 1 from a single player's viewpoint; after each time step, the current player may change, and the state must be updated according to the current player's viewpoint. Each card can be encoded as an integer from 1 to 52 (not starting from 0, as we will see in the next paragraph), and the score is an integer from 0 to 120 (tip: sort your cards for a reduced state space and faster convergence). Your action space can be the set of integers from 0 to 14 that maps to a 2-card combination in your hand. Alternatively, you could have a dynamic action space that sequentially asks for 1 of your 6 cards to discard and then for 1 of your remaining 5 cards to discard. The action space can be a subset of integers from 0 to 5 that maps to a single card in your hand. Be careful here - when choosing the second card to discard, your algorithm must know which card was discarded first. You can solve this by adding another component to your state vector that represents the first card that was discarded (set to 0 at the beginning of the phase), and therefore, make sure to update the state after the first discard. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In phase 2, you need to know the 4 cards in your hand, the cut card, the scores, and the cards currently played. Another helpful but unnecessary feature for learning is the current sum of played cards. A possible representation is a list of &lt;code&gt;[card0, card1, card2, card3, card played0, card played 1, card played 2, …,  card played 7, your score, opponent score, cut card, current sum of played cards]&lt;/code&gt;. The values of played cards should be initialized to 0 at the beginning of the phase. The state can be updated so that any card in your hand that you have played is set to 0, and any card played by your opponent can be set to its negative value. This will correctly encode which cards have been played from your hand, which cards played are your from your opponent, and all other available information. Consequently, the action space is dynamic and is a subset of integers from 0 to 3 that maps to a single card in your hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Possibility 2: Identical State Representations for each Phase&lt;/strong&gt; Another possibility is to have all of the above information for each phase encoded into a single state representation &lt;strong&gt;along with the phase number&lt;/strong&gt;. State features that are unique to phase 2 that aren't relevant for phase 1 can be set to 0 during phase 1 and vice versa. With this representation, the state feature vector is of the same length at all time steps of the game, and the action space will change as described above. The important ideas for the encoding are exactly the same as above and will change based on your particular implementation, so I won't include the details here.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I model this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you are going to implement something similar to Possibility 1, then you may need two agents that each learn a policy for a separate phase. For starters, you could use Q-learning or DQN and take the action with greatest q-value at each timestep, &lt;strong&gt;making sure that the chosen action is always a member of the current action space&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are going to implement something similar to Possibility 2, then you may only need a single agent that learns a policy for each phase, simply because the phase is a feature of the state. Essentially you are trading off a more complicated state representation for a simpler learning algorithm. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I've read this post on using MultiDiscrete spaces, but I'm not sure how to define this space based on the previous chosen action. Is this even the right approach to be taking?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After reading the OpenAI Gym documentation, it looks like the MultiDiscrete space is a product of Discrete spaces. Therefore, it is a &lt;em&gt;fixed&lt;/em&gt; action space and inherently not what you want here (a &lt;em&gt;dynamic&lt;/em&gt; action space). I don't believe that OpenAI Gym standards will support dynamic action spaces natively. You would need to do some extra work such as providing a method that returns the current action space of the environment. Alternatively, if you want to follow the &lt;code&gt;(state, reward, done, info)&lt;/code&gt; signal paradigm from OpenAI Gym, you may provide the current action space in the info dictionary. Finally, another idea is to allow the agent to always choose an action from a larger fixed action space (e.g. the set of integers from 0 to 3 for phase 2) and then penalize the agent through the reward signal whenever it chooses an action that is not a member of the current action space (e.g. if the chosen card was already played in phase 2). Afterward, you would return the current game state as the next state and make the agent try again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice is to first determine the state representation, and the rest of your implementation should follow, using the ideas above.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-04T21:51:08.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21648" PostTypeId="2" ParentId="8554" CreationDate="2020-06-04T22:00:35.907" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This means that the reward set is actually R={0,1,−3} (we assume that in each timestep, the robot can only collect one can).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;@riceissa While I agree with the rest of your demonstration, I wouldn't assume that the robot can only collect 0 or 1 can. As Neil Slater suggest, I think the robot could pick any number of cans between 0 and N.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is how I solve the problem for the more general case, assuming specific values for &lt;span class=&quot;math-container&quot;&gt;$s',s, a$&lt;/span&gt;. This generalization encompasses @riceissa answer.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$S_t=s'=\texttt{high}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$S_{t-1}=s=\texttt{high}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A_{t-1}=a=\texttt{search}$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We have the following equality : &#xA;&lt;span class=&quot;math-container&quot;&gt;$$r_{search}=\sum_{r \in R}r\cdot\frac{p(s', r|s, a)}{p(s'|s, a)}$$&lt;/span&gt;&#xA;For these values of &lt;span class=&quot;math-container&quot;&gt;$s', s, a$&lt;/span&gt; we also have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)=\alpha$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p(s', -3|s,a)=0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Writing &lt;span class=&quot;math-container&quot;&gt;$\eta_r:=p(s',r|s,a)$&lt;/span&gt; and for &lt;span class=&quot;math-container&quot;&gt;$R=\{0, 1, 2, \dots,N\}$&lt;/span&gt; (we omit &lt;span class=&quot;math-container&quot;&gt;$r=-3$&lt;/span&gt; since the proba is 0 for this case) we have then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*} &#xA;r_{\texttt{search}}&amp;amp;=\sum_{r=0}^N r\cdot\frac{\eta_r}{\alpha}\\&#xA;r_{\texttt{search}}\cdot\alpha&amp;amp;=\sum_{r=0}^N r\cdot\eta_r\\&#xA;r_{\texttt{search}}\cdot\alpha&amp;amp;=i\cdot\eta_i + \sum_{\substack{r=0 \\ r\neq i}}^N r\cdot\eta_r\\&#xA;i\cdot\eta_i&amp;amp;= r_{\texttt{search}}\cdot\alpha- \sum_{\substack{r=0 \\ r\neq i}}^N r\cdot\eta_r&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$i\neq 0$&lt;/span&gt; : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\eta_i= \frac{1}{i}\cdot\left(r_{\texttt{search}}\cdot\alpha- \sum_{\substack{r=0 \\ r\neq i}}^N r\cdot\eta_r\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$i=0$&lt;/span&gt;, we use that fact that &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;\alpha&amp;amp;=p(s'|s,a)=\sum_{r=0}^N p(s',r|s,a)=\sum_{r=0}^N\eta_r\\&#xA;&amp;amp;\Rightarrow\eta_0=\alpha - \sum_{r&amp;gt;0} \eta_r&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So substituting &lt;span class=&quot;math-container&quot;&gt;$\eta_r$&lt;/span&gt; by its probability formula we end up having, for these specific values of &lt;span class=&quot;math-container&quot;&gt;$s', s, a$&lt;/span&gt; : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    \begin{cases}&#xA;      p(s',i|s,a)= \frac{1}{i}\cdot\left(r_{\texttt{search}}\cdot\alpha- \sum_{\substack{r=0 \\ r\neq i}}^N r\cdot p(s',r|s,a)\right) &amp;amp; \texttt{$\forall i \in [1,N]$}\\&#xA;      p(s',0|s,a)=\alpha -\sum_{r&amp;gt;0} p(s',r|s,a) &amp;amp; \texttt{if $i=0$}&#xA;    \end{cases}    &#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37613" LastEditorUserId="37613" LastEditDate="2020-06-05T15:30:41.970" LastActivityDate="2020-06-05T15:30:41.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21649" PostTypeId="1" CreationDate="2020-06-04T22:14:01.853" Score="2" ViewCount="41" Body="&lt;p&gt;I've written a Double DQN-based stock trading bot using mainly time series stock data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've recently upgraded my Experience Replay(ER) code with a version of Prioritized Experience Replay (PER) similar to the one written by OpenAI. My DQN's reward function is the stock return over 30 days (the length of my test window).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strange thing is, once the bot has been trained using the same set of time series data and let free to trade on unseen stock data, the version that uses PER actually comes up with worse stock returns than the version using a regular ER.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not quite what I'd expected but it's very hard to debug and see what might have gone wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, will PER always perform better than a regular ER? If not, when/why not?&lt;/p&gt;&#xA;" OwnerUserId="37615" LastActivityDate="2020-06-04T22:14:01.853" Title="In a DQN, can Prioritized Experience Replay actually perform worse than a regular Experience Replay?" Tags="&lt;deep-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21650" PostTypeId="2" ParentId="21641" CreationDate="2020-06-04T23:49:33.480" Score="3" Body="&lt;p&gt;&lt;strong&gt;Bayes Error Rate&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the general case of K different classes, the probability of classifing x instance correctly is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} \label{eq1}&#xA;\begin{split}&#xA;P(correct) &amp;amp; = \sum_{i=1}^{K} p(x \in H_i, C_i) \\&#xA; &amp;amp; = \sum_{i=1}^{K} \int_{x \in H_i} p(x,C_i) \, dx\\&#xA; &amp;amp; = \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \\&#xA;\end{split}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt; is the region where class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; has the highest posterior. So the Bayes Error Rate is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} \label{eq2}&#xA;\begin{split}&#xA;P(error) &amp;amp; = 1 - p(correct) \\&#xA; &amp;amp; = 1 - \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx&#xA;\end{split}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Be careful if we drew &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt;, the region covered/classified by &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(C_i|x) \ne 1$&lt;/span&gt;, because that would mean that we surely predict correctly all the time. If &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; belongs to a decision area, this does not imply that it belongs to the corresponding class.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="1641" LastEditDate="2020-07-03T18:07:46.323" LastActivityDate="2020-07-03T18:07:46.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21651" PostTypeId="2" ParentId="21646" CreationDate="2020-06-05T03:57:42.837" Score="1" Body="&lt;p&gt;One of the reasons a discount factor is used, is to make sure the reward maximization is a well-defined problem and to make the sum of all rewards convergent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the MAB problem, the number of trials is typically finite owing to some sort of budget in the number of trials. Hence, this is less of problem. However, by all means discounts are still valid and helpful in cases where the analysis in asymptotic in the number of trials.&lt;/p&gt;&#xA;" OwnerUserId="37618" LastActivityDate="2020-06-05T03:57:42.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21652" PostTypeId="2" ParentId="21645" CreationDate="2020-06-05T04:21:08.853" Score="2" Body="&lt;p&gt;It comes down to the order they're computed in, and what they're used in. I will be referring to the LSTM in this answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the forget gate, you can see that it has the ability to manipulate the cell state. This gives it the ability to force a forget. Say (after training) it sees a super important input that means some previous data is irrelevant (say, like a full stop). This forget gate, while it might not force a forget, has the ability to force one, and will likely learn to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input gate ultimately adds to the cell state. This gate doesn't have direct influence over the cell state (it can't make it 0, like the forget gate can), but it can add to it and influence it that way. So it is an input gate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output gate is used to interpret the hidden state, and get it ready to be combined with the cell state for a final output at that time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While these gates all use sigmoid functions, are all initialised randomly and have the same dimensionality, what their output is used in, and the order they're computed in gives them a certain role to play. Initially, they won't conform to this role, but logically as they learn, they likely will. &lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-06-05T04:21:08.853" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21655" PostTypeId="1" CreationDate="2020-06-05T09:11:45.100" Score="2" ViewCount="56" Body="&lt;p&gt;I decided to train deep Q-learning agent based on getting raw pixels from environment.I have one particular problem:when I input stack of frames, suppose 4 consecutive frames, if action space is 6,then output is 4 by 6 matrix.So which one is real Q-value?I mean, I input batch of frames and it outputs batch of values and question is which is real Q-value out of those batch values?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastActivityDate="2020-07-05T10:03:05.353" Title="How to predict Q-values based on stack of frames" Tags="&lt;q-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21656" PostTypeId="2" ParentId="21655" CreationDate="2020-06-05T09:30:59.153" Score="2" Body="&lt;p&gt;You do not output the batch of Q-values. Input frame stacking is needed to gain full observability of the environment. In your case the output would be 6 elements for your current frame. If &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is a frame then you would stack 4 frames &lt;span class=&quot;math-container&quot;&gt;$[F_{k-3}, F_{k-2}, F_{k-1}, F_k]$&lt;/span&gt; and the output would be 6 Q-values for frame &lt;span class=&quot;math-container&quot;&gt;$F_k$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-06-05T09:30:59.153" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21658" PostTypeId="1" CreationDate="2020-06-05T11:09:10.250" Score="0" ViewCount="226" Body="&lt;p&gt;I'm looking for some indications about the tuning of hyper-parameters in building my double DQN.&lt;/p&gt;&#xA;&lt;p&gt;I have a time series problem (with about 2000 observations and no terminal state, I have to max the rewards), so I decided to define as an episode a subsequence of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; observations (from that dataset) starting from a random point (e.g. from the 245th period to the &lt;span class=&quot;math-container&quot;&gt;$245+N$&lt;/span&gt; period). So, here the first doubt: &lt;em&gt;should I run more episodes with a smaller &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; or vice versa?&lt;/em&gt; In a different context, I would make some tries, but here it takes a lot of time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What about the length of the batch and the step when updating? Are there any generally accepted rules?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The final hyperparameter that I don't know how to choose is the moment when updating the target model weights. I know I should do that rarely, but I do not find any specific rule.&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-11-04T22:39:20.593" LastActivityDate="2020-11-04T22:39:20.593" Title="How to choose hyperparameters in double DQN?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;hyperparameter-optimization&gt;&lt;double-dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21659" PostTypeId="2" ParentId="21643" CreationDate="2020-06-05T11:34:27.100" Score="2" Body="&lt;p&gt;Note that for a general policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; we have that &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a) = \mathbb{E}_{\pi}[G_t | S_t = s, A_t = a]$&lt;/span&gt;, where in state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; we take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and thereafter following policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Note that the expectation is taken with respect to the reward transition distribution &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(R_{t+1} = r, S_{t+1} = s' | A_t = a, S_t = s)$&lt;/span&gt; which I will denote as &lt;span class=&quot;math-container&quot;&gt;$p(s',r,|s,a)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can then rewrite the expectation as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align} q_{\pi}(s,a) &amp;amp;= \mathbb{E}_{\pi}[G_t | S_t = s, A_t = a] \\ &amp;amp; = \mathbb{E}_{\pi}[R_{t+1} + \gamma G_{t+1} | S_t = s, A_t = a] \\ &amp;amp; = \sum_{r,s'}p(s',r|s,a)(r + \gamma \mathbb{E}_\pi[G_{t+1} | S_{t+1} = s']) \\ &amp;amp; = \sum_{r,s'}p(s',r|s,a)(r + \gamma v_{\pi}(s')) \; .&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key thing to note is that these two terms, &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s')$&lt;/span&gt;, are only equal &lt;em&gt;in expectation&lt;/em&gt;, which is why in the equation you can exchange the terms because we are taking the expectation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I have shown this for a general policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; not just the optimal policy.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-05T11:34:27.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21660" PostTypeId="1" CreationDate="2020-06-05T12:09:32.123" Score="0" ViewCount="27" Body="&lt;p&gt;The data which I work with is sequential. It means that the size of batch equals one. My loss function is the negative log-likelihood. &#xA;What types of neural networks are suitable to train my model? The network must be an MLP.&lt;/p&gt;&#xA;" OwnerUserId="35633" LastActivityDate="2020-06-05T12:09:32.123" Title="Suitable networks for training sequential data?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21661" PostTypeId="1" CreationDate="2020-06-05T12:31:47.357" Score="0" ViewCount="387" Body="&lt;p&gt;We hear this many time for different problems&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Train a model to solve this problem!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What do we really mean by training a model?&lt;/p&gt;&#xA;" OwnerUserId="26489" LastEditorUserId="26489" LastEditDate="2020-06-05T15:22:58.010" LastActivityDate="2020-06-06T05:39:42.783" Title="What does it mean to train a model?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21662" PostTypeId="1" CreationDate="2020-06-05T12:43:23.847" Score="0" ViewCount="27" Body="&lt;p&gt;I want to obtain SHAP values with kernel SHAP without using python but I don't really understand the algorithm. If I have a kNN classifier, do I have to run the classifier for all the coalitions possible? For &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; variables, &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; predictions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how do I obtain the SHAP values after that? Linear regression?&lt;/p&gt;&#xA;" OwnerUserId="37628" LastEditorUserId="2444" LastEditDate="2020-06-05T14:58:52.340" LastActivityDate="2020-06-05T14:58:52.340" Title="How to obtain SHAP values" Tags="&lt;feature-selection&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21663" PostTypeId="1" AcceptedAnswerId="21664" CreationDate="2020-06-05T12:58:29.043" Score="3" ViewCount="278" Body="&lt;p&gt;I'm trying to solve the exercise 3.11 from the second-edition from the &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's book&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Exercise 3.11 If the current state is &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; , and actions are selected&#xA;  according to a stochastic policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, then what is the expectation of &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;&#xA;  in terms of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and the four-argument function &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (3.2)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here's my attempt. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, the expected immediate reward when taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is given in terms of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; by eq 3.5 in the book:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s,a) = \sum_{r \in R} r \, \sum_{s'\in S} p(s',r| s,a) = E[R_t | S_{t-1} = s, A_{t-1} = a]$&lt;/span&gt; (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a | s)$&lt;/span&gt; on the other hand gives the probability of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to express the expectation of the immediate reward over all actions &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; from the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; using (1) as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E[R_t | S_{t-1} = s, A] = \sum_{a \in A} \pi(a|s) r(a,s) \ \ \ \ \ \ \ \  (2) ?$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is valid, is this also valid in the next time step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E[R_{t+1} | S_{t} = s, A] = \sum_{a \in A} \pi(a|s) r(a,s) \ \ \ \ \ \ \ \ (3) ?$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If (2) and (3) are OK, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E[R_{t+1} | S_{t} = s, A] = \sum_{a \in A} \pi(a|s) \sum_{r \in R} r \, \sum_{s'\in S} p(s',r| s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37627" LastActivityDate="2020-06-07T10:29:39.763" Title="What is the solution to the exercise 3.11 in the RL book by Sutton and Barto" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21664" PostTypeId="2" ParentId="21663" CreationDate="2020-06-05T14:17:03.647" Score="3" Body="&lt;p&gt;First note that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[R_{t+1} |S_t=s] = \sum_{s',r}rm(s',r|s)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$m(\cdot)$&lt;/span&gt; is the mass function for the joint distribution of &lt;span class=&quot;math-container&quot;&gt;$S_{t+1},R_{t+1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are currently in state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; and we condition on taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; then the expected reward at time &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; is given as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align} \mathbb{E}[R_{t+1} | S_t = s, A_t=a] &amp;amp; = \sum_{s',r}rp(s'r|s,a)\;.&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; is taken according to some stochastic policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; so we need to marginalise this out of our expectation by using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_total_expectation&quot; rel=&quot;nofollow noreferrer&quot;&gt;tower law&lt;/a&gt; - i.e. we take &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{A_t \sim \pi}[\mathbb{E}[R_{t+1} | S_t = s, A_t=a]|S_t = s] = \sum_a \pi(a|s)\sum_{s',r}rp(s'r|s,a) = \mathbb{E}[R_{t+1} | S_t = s]\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why this holds we can re-write using some arbitrary mass functions &lt;span class=&quot;math-container&quot;&gt;$f(\cdot),h(\cdot),g(\cdot),m(\cdot)$&lt;/span&gt; as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(a|s)p(s'r|s,a) = \frac{f(a,s)}{g(s)} \times \frac{h(s',r,a,s)}{f(a,s)} = m(s',r,a|s)\;,$$&lt;/span&gt;&#xA;so when we would end up with (after re-arranging the summations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{s',r}r \sum_{a}m(s',r,a|s) = \sum_{s',r}r m(s',r|s) = \mathbb{E}[R_{t+1}|S_t = s]\;;$$&lt;/span&gt;&#xA;as required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: what you have done is mostly correct except be careful when going from (2) to (3). They are exactly the same equation except for the time stamps, which means you would have to change the time stamps in your &lt;span class=&quot;math-container&quot;&gt;$r(s,a)$&lt;/span&gt;. Note that when you are at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; you take your action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; from your current &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; to transition into state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; and then receive reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; (and the next state). &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-07T10:29:39.763" LastActivityDate="2020-06-07T10:29:39.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21665" PostTypeId="2" ParentId="21661" CreationDate="2020-06-05T14:37:08.987" Score="2" Body="&lt;p&gt;&lt;strong&gt;In machine learning, when you train a model, you adjust (or change) the parameters (or weights) of the model so that its performance in solving a certain task increases.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's little difference between the idea of training a model and the idea of training an animal. In fact, here's the dictionary definition of the verb &lt;em&gt;to train&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;teach (a person or animal) a particular skill or type of behaviour through practice and instruction over a period of time&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you train a model, you also teach a skill or type of behavior through practice and instruction. For example, if you train a model to solve an object classification problem, then you teach the model to classify certain objects according to their properties (which is the skill that the model learns).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different ways to train a model, depending on the problem you want to solve, the algorithms that you use to train the model, and the available data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a &lt;em&gt;labeled dataset&lt;/em&gt;, then you train a model with a supervisory signal (the labels), i.e. you explicitly tell the model the output that it is supposed to produce for each input, and, if it does not produce it, then you adjust its parameters so that next time it is more likely to produce the correct output for that input. This is called &lt;strong&gt;supervised learning&lt;/strong&gt; (or training).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In certain cases, you do not have the correct output that the model is supposed to produce for each input, but you only have a &lt;em&gt;reward&lt;/em&gt; (or reinforcement) signal. So, your training (or learning) algorithm needs to adjust the parameters of the model only based on the reward signal. This is called &lt;strong&gt;reinforcement learning&lt;/strong&gt; (or training).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there's also &lt;strong&gt;unsupervised learning&lt;/strong&gt; (or training), where you are given a dataset without labels or rewards, but you want to learn e.g. a probability distribution that this data was likely sampled from or separate this data into groups. For example, in k-means (a clustering algorithm), you want to split the data into groups so that similar objects belong to the same group and dissimilar objects belong to different groups. Note that k-means is a learning algorithm, so it's not a model, but you could consider the centroids of the clusters the parameters of the model (a clustering model).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are variations or subcategories of these learning paradigms and you can also combine them, so sometimes the difference between them is not so clear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also different types of models. There are &lt;a href=&quot;https://stats.stackexchange.com/q/268638/82135&quot;&gt;parametric (e.g. a linear regression model) and non-parametric (e.g. neural networks) models&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-05T15:35:17.480" LastActivityDate="2020-06-05T15:35:17.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21666" PostTypeId="1" CreationDate="2020-06-05T15:21:05.643" Score="2" ViewCount="162" Body="&lt;p&gt;For the problems that can be solved algorithmically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have very good formal literature for which problems can be solved in polynomial, exponential time and which cannot. &lt;strong&gt;P/NP/NP-hard&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But do we know some problems in machine learning paradigm for which no model can be trained? (With/without infinite computation capacity)&lt;/p&gt;&#xA;" OwnerUserId="26489" LastEditorUserId="2444" LastEditDate="2020-06-05T15:41:58.057" LastActivityDate="2020-06-06T16:40:33.027" Title="What kind of problems cannot be solved using machine learning techniques?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21667" PostTypeId="2" ParentId="21666" CreationDate="2020-06-05T16:33:42.007" Score="2" Body="&lt;p&gt;At least you should be aware of two points:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;P/NP/NP-hard (and all other class of complexities) are thoroughly valid for the machine learning area as well. Because these concepts are related to the fundamental of computations (&lt;a href=&quot;https://en.wikipedia.org/wiki/Theory_of_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;theory of computation&lt;/a&gt;), and machine learning is not an exception here. &lt;/li&gt;&#xA;&lt;li&gt;One of the useful concepts in the complexity of the learning problem is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;VC dimension&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Probably_approximately_correct_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAC learnability&lt;/a&gt;, and their related concepts (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Sample_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;sample complexity&lt;/a&gt;). Although these concepts can't be enough to measure the time complexity, they are useful for finding the learner model's capacity. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-06-05T16:39:05.553" LastActivityDate="2020-06-05T16:39:05.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21668" PostTypeId="2" ParentId="21477" CreationDate="2020-06-05T16:53:01.207" Score="2" Body="&lt;p&gt;Notably, these two tips/tricks are useful because we are assuming the context of &lt;strong&gt;deep&lt;/strong&gt; reinforcement learning here, as you pointed out. In DRL, the RL algorithm is guided in some fashion by a deep neural network, and the reasons for normalizing stem from the gradient descent algorithm and the architecture of the network.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How does this affect training?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An observation from the observation space is often used as an input to a neural network in DRL algorithms, and normalizing the input to neural networks is beneficial for many reasons (e.g. increases convergence speed, aids computer precision, prevents divergence of parameters, allows for easier hyperparameter tuning, etc.). These are standard results in DL theory and practice, so I won't provide details here.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;And more specifically, why on continuous action spaces we need to normalize also the action's values?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most popular discrete action space DRL algorithms (e.g. DQN) have one output node for each possible action in the neural net. The value of the output node may be a q-value (value-based algorithm) or a probability of taking that action (policy-based algorithm). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, a continuous action space DRL algorithm simply cannot have an output node for each possible action, as the action space is continuous. The output is usually the actual action to be taken by the agent or some parameters that could be used to construct the action (e.g. PPO outputs a mean and standard deviation and then an action is sampled from the corresponding Gaussian distribution - this phenomenon is mentioned in your linked reference). Therefore, normalizing the action space of a DRL algorithm is analogous to normalizing the outputs of the corresponding neural network, which is known to increase training speed and prevent divergence. Again, a quick search will yield some good resources if you are interested in these results.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-05T16:53:01.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21669" PostTypeId="1" AcceptedAnswerId="21671" CreationDate="2020-06-05T17:30:33.073" Score="3" ViewCount="73" Body="&lt;p&gt;I'm new to RL and to deep q-learning and I have a simple question about the architecture of the neural network to use in an environment with a continous state space a discrete action space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tought that the action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; should have been included as an input of the neural network, togheter with the state. It also made sense to me as when you have to compute the argmax or the max w.r.t. &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; it was like a &quot;standard&quot; function. Then I've seen some examples of networks that had as inputs only &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and that had as many outputs as the number of possible actions. I quite understand the logic behind this (replicate the q-values pairs of action-state) but is it  really the correct way? If so, how do you compute the &lt;span class=&quot;math-container&quot;&gt;$argmax$&lt;/span&gt; or the &lt;span class=&quot;math-container&quot;&gt;$max$&lt;/span&gt;? Do I have to associate to each output an action?&lt;/p&gt;&#xA;" OwnerUserId="37640" LastActivityDate="2020-06-05T17:54:27.773" Title="What's the right way of building a deep Q-network?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21670" PostTypeId="2" ParentId="21628" CreationDate="2020-06-05T17:34:33.320" Score="1" Body="&lt;p&gt;As the accepted answer states, the return at the current timestep is equal to the sum of discounted rewards from all future timesteps until the end of the episode. In Chapter 5 of Sutton and Barto, returns must be used to estimate the state-value and action-value functions because episode lengths are unrestricted and may be &lt;em&gt;greater than one&lt;/em&gt;. In contrast, Chapter 2 deals with the very special case of multi-armed bandits in which episode lengths are always &lt;em&gt;equal to one&lt;/em&gt;: The agent begins each episode in a fixed start state, takes an action, receives a reward, and then the episode terminates and the agent begins the next episode at the same start state. Therefore, &lt;strong&gt;a return is equivalent to a reward in Chapter 2 because all episodes have length one&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-05T17:34:33.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21671" PostTypeId="2" ParentId="21669" CreationDate="2020-06-05T17:54:27.773" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do I have to associate to each output an action?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You are absolutely correct! In DQN, each output node of the neural network will be associated with one of your possible actions (assuming a &lt;em&gt;finite&lt;/em&gt; discrete action space). After passing an input through the network, the value of each output node is the estimated q-value of the corresponding action. One benefit of this architecture is that you &lt;strong&gt;only need to pass the input through the neural network once to compute the q-value of each action&lt;/strong&gt;, which is constant in the number of actions. If you were to include an action as an input to the neural network along with an observation, then you would need to pass an input for each action, which scales linearly in the number of actions. This is mentioned in paragraph 2 of Section 4.1 in the original DQN paper (&lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1312.5602&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it really the correct way? If so, how do you compute the argmax or the max?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's one possible way that is used in many popular algorithms such as DQN. To find the argmax, you simply take the action corresponding to the output node with highest q-value after passing an input through the network.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-05T17:54:27.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21672" PostTypeId="1" AcceptedAnswerId="21673" CreationDate="2020-06-05T19:10:42.870" Score="2" ViewCount="395" Body="&lt;p&gt;How would you train a reinforcement learning agent from raw pixels? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you have 3 stacked images to sense motion, then how would you pass them to neural networks to output Q-learning values? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you pass that batch output, it would be a batch of values, so from here it is impossible to deduce which ones are the true Q-values for that state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am watching a YouTuber: &lt;a href=&quot;https://www.youtube.com/watch?v=RfNxXlO6BiA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning with Phil&lt;/a&gt;, and he did it very differently. On the 13th minute, he defined a network that outputs a batch of values rather than Q-values for 6 states. In short, he outputs a matrix rather than a vector.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-06-05T23:06:22.187" LastActivityDate="2020-06-05T23:06:22.187" Title="How to train a reinforcement learning agent from raw pixels?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21673" PostTypeId="2" ParentId="21672" CreationDate="2020-06-05T20:11:15.250" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How would you train a reinforcement learning agent from raw pixels? For example, if you have 3 stacked images to sense motion, then how would you pass them to neural networks to output Q-learning values?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A Convolutional Neural Network (CNN) structure is a standard neural network architecture when working with 2D pixel input in reinforcement learning, and it is the technique used in the original DQN paper (see paragraphs 1 &amp;amp; 3 of Section 4.1 of &lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1312.5602&lt;/a&gt;). CNNs typically take 3-dimensional input, where the first two dimensions are &lt;code&gt;height&lt;/code&gt; and &lt;code&gt;width&lt;/code&gt; of your images, and the third is &lt;code&gt;rgb color&lt;/code&gt;. The technique in the paper was to convert each RGB frame (or image) to greyscale format (so it has only 1 color channel/dimension instead of 3) and instead use the &lt;code&gt;rgb_color&lt;/code&gt; dimension as a &lt;code&gt;frames&lt;/code&gt; dimension that is indexed by each stacked frame.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Currently, I am watching a YouTuber: Machine Learning with Phil, and he did it very differently. On the 13th minute, he defined a network that outputs a batch of values rather than Q-values for 6 states. In short, he outputs a matrix rather than a vector.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Later in the tutorial series, he most likely will discuss the training of the neural network. During training, you need to find the q-values of a batch of sets of stacked frames. Specifically, &lt;strong&gt;each element of the batch is a set of stacked frames&lt;/strong&gt;. In other words, a set of stacked frames is treated as a single observation, so a batch of sets of stacked frames is a batch of observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find these q-values, you will perform a forward pass of the batch of observations through the neural network. A forward pass of a single observation (set of stacked frames) through the neural network yields a vector of q-values (one for each action). Thus, a forward pass of a batch of observations (batch of stacked frames) will yield a matrix of q-values (one vector of q-values for each observation (or set of stacked frames)). This technique is used because many standard neural network libraries are designed to &lt;strong&gt;perform a forward pass on a batch of inputs through the neural network much faster than performing a forward pass on each input separately&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastEditorUserId="37607" LastEditDate="2020-06-05T21:04:50.037" LastActivityDate="2020-06-05T21:04:50.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21674" PostTypeId="1" CreationDate="2020-06-05T20:32:22.853" Score="3" ViewCount="83" Body="&lt;p&gt;In deep Q learning, we execute the algorithm for each episode, and for each step within an episode, we take an action and record a reward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a situation where my action is 2-tuple &lt;span class=&quot;math-container&quot;&gt;$a=(a_1,a_2)$&lt;/span&gt;. Say, in episode &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, I have to take the first half of an action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;, then for each step of the episode, I have to take the second half of the action &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, say we are in episode &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and this episode has &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; timesteps. First, I have to take &lt;span class=&quot;math-container&quot;&gt;$a_1(i)$&lt;/span&gt;. (Where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is used to reference episode &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;.) Then, for each &lt;span class=&quot;math-container&quot;&gt;$t_i\in\{1,2,\ldots,T\}$&lt;/span&gt;, I have to take action &lt;span class=&quot;math-container&quot;&gt;$a_2(t_i)$&lt;/span&gt;. Once I choose &lt;span class=&quot;math-container&quot;&gt;$a_2(t_i)$&lt;/span&gt;, I get an observation and a reward for the global action &lt;span class=&quot;math-container&quot;&gt;$(a_1(i), a_2(t_i))$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to apply deep Q learning? If so, how? Should I apply the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy twice?&lt;/p&gt;&#xA;" OwnerUserId="37642" LastEditorUserId="37642" LastEditDate="2020-06-05T21:43:32.060" LastActivityDate="2020-06-05T21:43:32.060" Title="How to take actions at each episode and within each step of the episode in deep Q learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21675" PostTypeId="1" AcceptedAnswerId="21701" CreationDate="2020-06-05T22:28:31.917" Score="2" ViewCount="78" Body="&lt;p&gt;From &lt;a href=&quot;https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge&quot; rel=&quot;nofollow noreferrer&quot;&gt;the AlphaGo Zero paper&lt;/a&gt;, AlphaGo Zero uses an exponentiated visit count from the tree search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why use visit count instead of the mean action value &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="37645" LastEditorUserId="2444" LastEditDate="2020-06-05T22:57:55.423" LastActivityDate="2020-06-07T04:31:53.620" Title="Why does AlphaGo Zero select move based on exponentiated visit count?" Tags="&lt;reinforcement-learning&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21676" PostTypeId="1" CreationDate="2020-06-06T00:30:45.383" Score="1" ViewCount="142" Body="&lt;p&gt;Has anyone investigated ways to initialize a network so that everything is considered &quot;unknown&quot; at the start?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you consider the ways humans learn, if something doesn't fit a class well enough, it falls into an &quot;unknown category&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would argue we all ultimately do some type of correlation matching internally with a certain threshold existing for recognition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep networks don't currently have this ability, everything falls into a class. What I'm curious about is how might we force a network to initially classify things as &quot;unknown&quot; as the default state. &lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-06-06T00:41:14.950" LastActivityDate="2020-06-06T00:41:14.950" Title="Can we force the initial state of a neural network to produce an &quot;unknown&quot; class?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21679" PostTypeId="2" ParentId="21661" CreationDate="2020-06-06T05:39:42.783" Score="0" Body="&lt;p&gt;Training a model simply means learning good values for all the weights and the bias from labeled examples. In supervised learning, a machine learning algorithm builds a model by examining many examples and attempting to find a model that minimizes loss; this process is called empirical risk minimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loss is the penalty for a bad prediction. That is, loss is a number indicating how bad the model's prediction was on a single example. If the model's prediction is perfect, the loss is zero; otherwise, the loss is greater. The goal of training a model is to find a set of weights and biases that have low loss, on average, across all examples. shows a high loss model on the left and a low loss model on the right.&lt;/p&gt;&#xA;" OwnerUserId="32861" LastActivityDate="2020-06-06T05:39:42.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21680" PostTypeId="2" ParentId="5814" CreationDate="2020-06-06T05:47:31.500" Score="0" Body="&lt;p&gt;Typically, there are mainly three steps in an object bounding box detection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, a model or algo is used to generate ROI(region of interest) or region proposals. These region proposals are an all large set of bounding boxes spanning the full images. (that is, an object localization component).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second step, visual features(Face, Person, etc... using Convolution) are extracted for each of the bounding boxes, they are evaluated. It is determined whether and which objects are present in the interested area based on visual features (i.e. an object classification component).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the final post-processing step, overlapped boxes are combined into a single bounding box (that is, non-maximum suppression).&lt;/p&gt;&#xA;" OwnerUserId="32861" LastActivityDate="2020-06-06T05:47:31.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21681" PostTypeId="1" AcceptedAnswerId="21685" CreationDate="2020-06-06T07:34:06.170" Score="2" ViewCount="134" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf#page=68&quot; rel=&quot;nofollow noreferrer&quot;&gt;equation 4.9 of Sutton and Barto's book&lt;/a&gt; on page 79, we have(for policy iteration algo):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi ^{'}(s) = arg \max_{a}\sum_{s',r}p(s',r|s,a)[r+\gamma v_{\pi}(s')]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the previous policy and  &lt;span class=&quot;math-container&quot;&gt;$\pi ^{'}$&lt;/span&gt; is the new policy. Hence in iterations &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; it must mean&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi _{k+1}(s) = arg \max_{a}\sum_{s',r}p(s',r|s,a)[r+\gamma v_{\pi_{k}}(s')]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But in the example given in the same book on page 77 we have:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/EKr8W.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EKr8W.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now for the concerned state marked in red -&lt;/p&gt;&#xA;&lt;p&gt;So &lt;span class=&quot;math-container&quot;&gt;$v_{\pi_{1}}$&lt;/span&gt; = -1 for all four surrounding states&lt;/p&gt;&#xA;&lt;p&gt;r = -1 for all four surrounding states&lt;/p&gt;&#xA;&lt;p&gt;p(s',r|s,a) = 1 for all four surrounding states&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi _{2}(s) = arg \max_{a}[1*[-1+1*-1],1*[-1+1*-1],1*[-1+1*-1],1*[-1+1*-1]]$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\pi _{2}(s) = arg \max_{a}(-2,-2,-2,-2)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hence this should give us a criss-cross symbol(4 directional arrow) in &lt;span class=&quot;math-container&quot;&gt;$\pi_{2}$&lt;/span&gt;(s) but here a left arrow symbol is given.&lt;/p&gt;&#xA;&lt;p&gt;What's wrong with my calculations.&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-06T10:07:39.963" Title="Equation not satisfied in Policy Iteration Algorithm" Tags="&lt;value-functions&gt;&lt;policy-iteration&gt;&lt;bellman-equations&gt;&lt;optimal-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21682" PostTypeId="1" CreationDate="2020-06-06T07:57:34.000" Score="1" ViewCount="134" Body="&lt;p&gt;I have coded the breakout RAM version, but, unfortunately, its highest reward was 5. I trained it for about 2 hours and never reached a higher score. The code is huge, so I can't paste here, but, in short, I used &lt;em&gt;double deep Q-learning&lt;/em&gt;, and trained it like it was CartPole or lunar-lander environment. In CartPole, the observation was a vector of 4 components. In that case, my double deep Q-learning agent solved the environment, but in the breakout-ram version whose observation was a vector of 128 elements, it was not even close. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I miss something?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-06-06T10:28:34.927" LastActivityDate="2021-03-05T22:00:35.087" Title="How to implement RAM versions of Atari games" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;q-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21683" PostTypeId="1" CreationDate="2020-06-06T08:18:18.987" Score="1" ViewCount="60" Body="&lt;p&gt;While training a GAN-based model, every time the discriminator's loss gets a constant value of nearly 0.63 while the generator's loss keeps on changing from 0.5 to 1.5, so I am not able to understand if this thing is happening either due to the generator being successful in fooling the discriminator or some instability in training. I have been stuck in this confusion for so many days.&lt;/p&gt;&#xA;" OwnerUserId="37648" LastEditorUserId="2444" LastEditDate="2020-06-06T17:19:28.723" LastActivityDate="2020-06-06T17:19:28.723" Title="What does it mean when the discriminator's loss gets a constant value while the generator's loss keeps on changing?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21684" PostTypeId="1" AcceptedAnswerId="21687" CreationDate="2020-06-06T08:55:32.493" Score="7" ViewCount="171" Body="&lt;p&gt;I often see, the state-action value function is expressed as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s,a)=\mathbb{E}_{\pi}[R_{t+1}+\gamma G_{t+1} | S_t=s, A_t = a] = \mathbb{E}[R_{t+1}+\gamma v_{\pi}(s') |S_t = s, A_t =a]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does expressing the future return in the time &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; as a state value function &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; make the expected value under policy change to expected value in general?&lt;/p&gt;&#xA;" OwnerUserId="31324" LastEditorUserId="36821" LastEditDate="2020-06-12T13:55:22.490" LastActivityDate="2021-01-22T17:25:16.367" Title="Why state-action value function as an expected value of the return and state value function, does not need to follow policy?" Tags="&lt;reinforcement-learning&gt;&lt;bellman-equations&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21685" PostTypeId="2" ParentId="21681" CreationDate="2020-06-06T09:36:33.363" Score="3" Body="&lt;p&gt;Your calculations are correct, but you have misinterpreted the equations and the diagram. The index &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$v_k$&lt;/span&gt; for the diagram refers to the policy evaluation update iteration only, and is not related to the policy update step (which uses the notation &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; and does not mention &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Policy improvement consists of multiple sweeps through states to fully evaluate the current policy and estimate the value function for it. After that, it updates the policy in a separate policy improvement step. There are two loops - an &lt;em&gt;inner&lt;/em&gt; loop indexed by &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; in the equations and diagram, plus an &lt;em&gt;outer&lt;/em&gt; loop which is not given an index notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The diagram is not showing incremental &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; policies from outer loops over policy iteration. Instead it is showing &quot;Greedy Policy w.r.t. &lt;span class=&quot;math-container&quot;&gt;$v_k$&lt;/span&gt;&quot; steps in the inner loop - you can think of that as the policy &lt;span class=&quot;math-container&quot;&gt;$\pi'$&lt;/span&gt; that you would get inside the &lt;em&gt;first&lt;/em&gt; outer loop, if you terminated the policy evaluation stage after that iteration &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; of the inner loop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The diagram only shows behaviour of policy iteration for a single outer loop. It demonstrates at least two interesting things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of this very simple environment, if you ran a single outer loop with long enough policy evaluation stage (&lt;span class=&quot;math-container&quot;&gt;$k \ge 3$&lt;/span&gt;) you would find the optimal policy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Even before the value function estimate is close to convergence (with a high &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;), the policy that could be derived from the new estimates could be used to improve the policy. That leads to the idea of the &lt;em&gt;value iteration&lt;/em&gt; method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-06T10:07:39.963" LastActivityDate="2020-06-06T10:07:39.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21687" PostTypeId="2" ParentId="21684" CreationDate="2020-06-06T11:07:56.247" Score="8" Body="&lt;p&gt;Let's first write the state-value function as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) = \mathbb{E}_{s_{t},r_{t} \sim E,a_t \sim \pi}[r(s_t,a_t) + \gamma G_{t+1} | S_t = s, A_t = a]\; ;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$r(s_t,a_t)$&lt;/span&gt; is written to show that the reward gained at time &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; is a function of the state and action tuple we have at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (note here that &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; would be just the sum of future reward signals). This allows us to show that the expectation is taken under the joint distribution of &lt;span class=&quot;math-container&quot;&gt;$s,r\sim E$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the environment and actions are taken from our policy distribution.&lt;/p&gt;&#xA;&lt;p&gt;As we have conditioned on knowing &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; then the only random variable in the expectation that is dependent on our policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; because this is the  sum of future reward signals and so will depend on future state-action values. Thus, we can rewrite again as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) = \mathbb{E}_{s_{t},r_{t} \sim E}[r(s_t,a_t) + \gamma \mathbb{E}_{a_t\sim \pi}[ G_{t+1} |S_{t+1} = s'] | S_t = s, A_t = a]\;,$$&lt;/span&gt;&#xA;where the inner expectation (coupled with the fact its inside an expectation over the state and reward distributions) should look familiar to you as the state value function, i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{a_t\sim \pi}[ G_{t+1} |S_{t+1} = s'] = v_{\pi}(s')\;.$$&lt;/span&gt;&#xA;This leads us to get what you have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_{\pi}(s,a) = \mathbb{E}_{s_{t},r_{t} \sim E}[r(s_t,a_t) + \gamma v_{\pi}(s') | S_t = s, A_t = a]\;,$$&lt;/span&gt;&#xA;where the only difference is that we have made clear what our expectation is taken with respect to.&lt;/p&gt;&#xA;&lt;p&gt;The expectation is taken with respect to the conditional distribution &lt;span class=&quot;math-container&quot;&gt;$S_{t+1},R_{t+1}|A_t,S_t$&lt;/span&gt;, and we usually include the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; subscript to denote that they are also taking the expectation with respect to the policy, but here this does not effect the first term as we have conditioned on knowing &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; and only applies to the future reward signals.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-01-22T17:25:16.367" LastActivityDate="2021-01-22T17:25:16.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21688" PostTypeId="1" CreationDate="2020-06-06T11:47:49.497" Score="1" ViewCount="37" Body="&lt;p&gt;I'm trying to optimize a neural network. For that, I'm changing parameters like the batch size, learning rate, weight initialization, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A neural network is not a deterministic algorithm, so, in each training set, I train the neural network from scratch and I stop it when it's full converged.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After training is complete, I calculate the performance of the neural network in a test dataset. The problem is, I trained the neural network from scratch 2 times with the same parameters, but the difference in performance was almost 5%, which is a BIG DIFFERENCE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what's the reasonable number of training runs to obtain a credible performance number of a neural network?&lt;/p&gt;&#xA;" OwnerUserId="33545" LastEditorUserId="2444" LastEditDate="2020-06-06T11:54:40.047" LastActivityDate="2020-06-06T11:54:40.047" Title="How many training runs are needed to obtain a credible value for performance?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;optimization&gt;&lt;hyperparameter-optimization&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="21689" PostTypeId="1" CreationDate="2020-06-06T13:04:21.143" Score="0" ViewCount="24" Body="&lt;p&gt;i'm new to the topic of data science / AI and would like to start implementing a method which predicts sales planning data based on sales planning data from the past years. Hence, to do forecasting here. As i'm new to the topic i'm struggeling to go into the right direction, my initial guess would be neurol networks but is that correct? Where can i find a rough overview on the methods and which one to choose for what problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your patience and help&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers Mat&lt;/p&gt;&#xA;" OwnerUserId="37658" LastActivityDate="2020-06-06T13:04:21.143" Title="Sales Planning using Predictive Analytics, which method" Tags="&lt;neural-networks&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21692" PostTypeId="1" AcceptedAnswerId="21714" CreationDate="2020-06-06T14:32:49.413" Score="3" ViewCount="179" Body="&lt;p&gt;I recently read the DQN &lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; titled: Playing Atari with Deep Reinforcement Learning. My basic and rough understanding of the paper is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have two neural networks; one stays frozen for a duration of time steps and is used in the computation of the loss function with the neural network that is updating. The loss function is used to update the neural network using gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience replay is used, which basically creates a buffer of experiences. This buffer of experiences is randomly sampled and these random samples are used to update the non-frozen neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question pertains to the DQN algorithm illustrated in the paper: Algorithm 1, more specifically lines 4 and 9 of this algorithm. My understanding, which is also mentioned early on in the paper, is that the states are actually sequences of the game-play frames. I want to know, since the input is given to a CNN, how would we encode these frames to serve as input to the CNN? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to know since &lt;span class=&quot;math-container&quot;&gt;$s_{1}$&lt;/span&gt; is equal to a set, which can be seen in line 4 of the algorithm, then why is &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; equal to &lt;span class=&quot;math-container&quot;&gt;$s_{t}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_{t}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="30174" LastEditorUserId="37607" LastEditDate="2020-06-08T14:07:49.327" LastActivityDate="2020-06-08T14:07:49.327" Title="How to convert sequences of images into state in DQN?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;dqn&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21694" PostTypeId="2" ParentId="21666" CreationDate="2020-06-06T16:40:33.027" Score="0" Body="&lt;p&gt;Unsupervised disentanglement learning with arbitrary generative models is impossible without inductive biases [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, in general, any kind of learning is impossible without inductive biases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]: &lt;a href=&quot;https://arxiv.org/abs/1811.12359&quot; rel=&quot;nofollow noreferrer&quot;&gt;Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37618" LastActivityDate="2020-06-06T16:40:33.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21695" PostTypeId="1" AcceptedAnswerId="21697" CreationDate="2020-06-06T17:48:11.417" Score="2" ViewCount="375" Body="&lt;p&gt;I have a problem which I believe can be described as a contextual bandit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, in each round, I observe a context from the environment consisting of five continuous features, and, based on the context, I have to choose one of the ten available actions. The actions do not influence the next context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the above I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this a contextual bandit or an MDP with a discount equal to zero (one step RL)? I have read that, in contextual bandits, we receive a different context for each action and I am a little bit confused.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I use the DQN algorithm with TD Target only the observed reward instead of the reward plus the predicted value of the next state?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I use a policy gradient algorithm, like REINFORCE or A2C? If yes, should I use a baseline and what this baseline should be?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have seen in the literature that there are some algorithms for contextual bandits such as LinUCB, LinRel, NeuralBandit, etc. And I am wondering why the DQN, A2C and REINFORCE algorithms, which seem to work well in MDP setting, are not used in contextual bandits, given the fact that this problem can be described as an MDP with a discount equal to zero?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37663" LastEditorUserId="2444" LastEditDate="2020-06-06T19:42:15.303" LastActivityDate="2020-06-06T21:11:47.910" Title="Can I apply DQN or policy gradient algorithms in the contextual bandit setting?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;actor-critic-methods&gt;&lt;reinforce&gt;&lt;contextual-bandit&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21697" PostTypeId="2" ParentId="21695" CreationDate="2020-06-06T20:52:11.320" Score="4" Body="&lt;p&gt;MDPs are strict generalisations of contextual bandits, adding time steps and state transitions, plus the concept of return as a measure of agent performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, methods used in RL to solve MDPs will work to solve contextual bandits. You can either treat a contextual bandit as a series of 1-step episodes (with start state chosen randomly), or as a continuous problem with discount factor zero.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can I use the DQN algorithm with TD Target only the observed reward instead of the reward plus the predicted value of the next state?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. That's identical mathematically to having a discount of zero, or having 1-step episodes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can I use a policy gradient algorithm, like REINFORCE or A2C? If yes, should I use a baseline and what this baseline should be?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. Once converted to an MDP, you can use the same baselines in these algorithms as normal (A2C's use of advantage instead of action value is already a baseline). Generally adding a baseline can help reduce variance, so it may still help when applying RL to contextual bandit problems.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have seen in the literature that there are some algorithms for contextual bandits such as LinUCB, LinRel, NeuralBandit, etc. And I am wondering why the DQN, A2C and REINFORCE algorithms, which seem to work well in MDP setting, are not used in contextual bandits&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are a couple of reasons that contextual bandit problems are not solved using RL techniques more often:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The goal in contextual bandits is commonly focused on creating a highly efficient online learner that minimises &lt;em&gt;regret&lt;/em&gt;. Regret is the long term difference in total reward between always exploiting the best action choice compared to the exploration necessary to find it. Some RL solvers - e.g. DQN - are poor by this metric.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The lack of timesteps and state transitions can be used in algorithm design to be more efficient. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Improvements to RL methods designed to help with sparse rewards and the assignment problem in MDPs are pointless for contextual bandits and may be wasteful or even counter-productive.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some RL algorithms do resolve to be nearly identical to their contextual bandit counterparts, and have the same performance characteristics e.g. REINFORCE with baseline for 1-step episodes is essentially the &lt;a href=&quot;https://github.com/withai/Policy-Gradients-Contextual-Bandit-Problem/blob/master/Policy%20Gradient%20Contextual%20Bandits.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contextual Gradient Bandit algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also worth noting than many problem domains where contextual bandit algorithms do well - e.g. website recommendations and advertising - &lt;a href=&quot;https://dl.acm.org/doi/10.1145/3308558.3313616&quot; rel=&quot;nofollow noreferrer&quot;&gt;have research showing that&lt;/a&gt; that a more sophisticated MDP model and RL-like approach can do even better. Although that is not quite the same as your question, it typically means extending the model so that timesteps and state transitions are meaningful.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-06T21:11:47.910" LastActivityDate="2020-06-06T21:11:47.910" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21699" PostTypeId="1" AcceptedAnswerId="21703" CreationDate="2020-06-06T21:38:24.673" Score="1" ViewCount="146" Body="&lt;p&gt;In policy gradient algorithms the output is a stochastic policy - a probability for each action.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that if I follow the policy (sample an action from the policy) I make use of exploration because each action has a certain probability so I will explore all actions for a given state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it beneficial or is it common to use extra exploration strategies, like UCB, Thompson sampling, etc. with such algorithms?&lt;/p&gt;&#xA;" OwnerUserId="37663" LastEditorUserId="2444" LastEditDate="2020-06-16T16:19:48.120" LastActivityDate="2020-06-16T16:19:48.120" Title="Should I use exploration strategy in Policy Gradient algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;upper-confidence-bound&gt;&lt;thompson-sampling&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21700" PostTypeId="1" AcceptedAnswerId="21705" CreationDate="2020-06-07T02:17:51.880" Score="3" ViewCount="86" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf#page=81&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton-Barto's book&lt;/a&gt; on page 63 (81 of the pdf):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[R_{t+1} + \gamma v_\pi(S_{t+1}) \mid S_t=s,A_t=\pi'(s)] = \mathbb{E}_{\pi'}[R_{t+1} + \gamma v_\pi(S_{t+1}) \mid S_{t} = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}$&lt;/span&gt; suddenly change to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi'}$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$A_t = \pi'(s)$&lt;/span&gt; term disappears?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in general, in the conditional expectation, which distribution do we compute the expectation with respect to? From what I have seen, in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X \mid Y]$&lt;/span&gt;, we always calculate the expected value over distribution &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="2444" LastEditDate="2020-06-07T12:55:08.577" LastActivityDate="2020-06-07T13:33:12.080" Title="How does $\mathbb{E}$ suddenly change to $\mathbb{E}_{\pi'}$ in this equation?" Tags="&lt;reinforcement-learning&gt;&lt;probability&gt;&lt;probability-distribution&gt;&lt;expectation&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21701" PostTypeId="2" ParentId="21675" CreationDate="2020-06-07T04:31:53.620" Score="2" Body="&lt;p&gt;The answer is surprisingly hidden in the original AlphaGo &lt;a href=&quot;https://www.nature.com/articles/nature16961&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At the end of search AlphaGo selects the action with maximum visit count; this&#xA;  is less sensitive to outliers than maximizing action value.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, there did not appear to be further details in the paper or in the related &lt;a href=&quot;https://ieeexplore.ieee.org/document/5599855&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt;. The root child node (corresponding to an action) with maximum visit count is fittingly known as the &lt;em&gt;robust child&lt;/em&gt;, as described &lt;a href=&quot;https://project.dke.maastrichtuniversity.nl/games/files/msc/Fschadd_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and referenced in a MCTS survey &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/6145622&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-07T04:31:53.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21702" PostTypeId="1" CreationDate="2020-06-07T08:00:44.597" Score="0" ViewCount="38" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d9TkO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d9TkO.jpg&quot; alt=&quot;Photo of a product with visual defects&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a tool that could detect visual defects on a photo of a free flowing product like potato flakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Potato flakes is powder-like product that is essentially dedydrated mashed potatoes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defects are objects that are darker in color compared to light-yellow potato flakes. Theses are normally potato skin pieces, burned brown pieces, or other natural impurities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the attached photo defects are circled in red. Inside green circles are also some darker spots, but those are most likely due to shadows and do not look like defects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please suggest a tool that could help me identify the defects automatically.&#xA;I am new to this, but I heard there are some computer vision modules for python, such as Scikit-image, Outlook. Am I on the right path?&lt;/p&gt;&#xA;" OwnerUserId="37668" LastActivityDate="2020-06-07T08:00:44.597" Title="Which tool can I use to detect “visual defects” on a photo of a product?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21703" PostTypeId="2" ParentId="21699" CreationDate="2020-06-07T09:27:36.903" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I believe that if I follow the policy (sample an action from the policy) I make use of exploration because each action has a certain probability so I will explore all actions for a given state.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, having a stochastic policy function is the main way that a lot of policy gradient methods achieve exploration, including REINFORCE, A2C, A3C.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it beneficial or is it common to use extra exploration strategy like UCB, Thompson sampling etc. n such algorithms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It can be, but needs to be done carefully, as the gradient sampling for the policy function is different. Many policy gradient methods are strictly on-policy and will not work if you simply add extra exploration. It is relatively straightforward to adjust the critic part of actor-critic methods by using e.g.  Q learning update rules for it. However, the gradient of the policy function is trickier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some policy gradient methods that do work with a separate, tunable, exploration function. &lt;a href=&quot;https://towardsdatascience.com/deep-deterministic-policy-gradients-explained-2d94655a9b7b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Deterministic Policy Gradient&lt;/a&gt; (DDPG) may be of interest to you - as per the title, it works with a deterministic policy function, and exploration is achieved by adding a separate noise function on top. The sampling for policy gradient is then corrected for being off-policy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-06-07T09:27:36.903" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21704" PostTypeId="2" ParentId="21692" CreationDate="2020-06-07T10:23:02.933" Score="1" Body="&lt;p&gt;I read the DQN paper titled: Playing Atari with Deep Reinforcement Learning again&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read, in the pre-processing and model architecture section (section 4.1), that for each state that is input to the CNN, that this state is actually stacked frames of the game, so basically what has to be done, to my understanding, is that for each time step you stack 4 frames (current frame and 3 previous frames) and this will serve as input to the CNN as the dimensions would be side * side * 4, 4 because the frames are converted to grey-scale and 4 frames are being used. &lt;/p&gt;&#xA;" OwnerUserId="30174" LastEditorUserId="2444" LastEditDate="2020-06-07T13:09:00.493" LastActivityDate="2020-06-07T13:09:00.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21705" PostTypeId="2" ParentId="21700" CreationDate="2020-06-07T11:40:44.903" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, in general, in the conditional expectation, which distribution do we compute the expectation with respect to? From what I have seen, in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X|Y]$&lt;/span&gt;, we always calculate the expected value over distribution &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, for &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X|Y]$&lt;/span&gt; we take expectation of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; with respect to the conditional distribution &lt;span class=&quot;math-container&quot;&gt;$X|Y$&lt;/span&gt;, i.e. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[X|Y] = \int_\mathbb{R} x p(x|y)dx\;;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p(x|y)$&lt;/span&gt; is the density function of the conditional distribution. If your random variables are discrete then replace the integral with a summation. Also note that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X|Y]$&lt;/span&gt; is still a random variable in &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How does &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}$&lt;/span&gt; suddenly change to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi '}$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$A_t = \pi '(s)$&lt;/span&gt; term disappears? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is because in this instance &lt;span class=&quot;math-container&quot;&gt;$\pi '(s)$&lt;/span&gt; is a  deterministic policy, i.e. in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; the policy will take action &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; with probability 1 and all other actions with probability 0. NB: this is the convention used in Sutton and Barto to denote a deterministic policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without loss of generality, assume that &lt;span class=&quot;math-container&quot;&gt;$\pi'(s) = b$&lt;/span&gt;. The implication of this is that in the first expectation we have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[R_{t+1} + \gamma v(S_{t+1}) | S_t = s, A_t = \pi'(s) = b] = \sum_{s',r}p(s',r|s,a=b)(r + \gamma v(s'))\;,$$&lt;/span&gt;&#xA;and in the second expectation we have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\pi'}[R_{t+1} + \gamma v(S_{t+1}) | S_t = s] = \sum_a\pi'(a|s)\sum_{s',r}p(s',r|s,a)(r + \gamma v(s'))\;;$$&lt;/span&gt;&#xA;However, we know that &lt;span class=&quot;math-container&quot;&gt;$\pi'(a|s) = 0 \; \forall a \neq b$&lt;/span&gt;, so this sum over &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; would equal 0 for all &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; except when &lt;span class=&quot;math-container&quot;&gt;$a=b$&lt;/span&gt;, in which case we know that &lt;span class=&quot;math-container&quot;&gt;$\pi'(b|s) = 1$&lt;/span&gt;, and so the expectation becomes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\pi'}[R_{t+1} + \gamma v(S_{t+1}) | S_t = s] = \sum_{s',r}p(s',r|s,a=b)(r + \gamma v(s'))\;;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so we have equality of the two expectations. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-07T13:33:12.080" LastActivityDate="2020-06-07T13:33:12.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21706" PostTypeId="2" ParentId="21493" CreationDate="2020-06-07T14:41:03.427" Score="0" Body="&lt;p&gt;It is not a 'delay' but a 'look ahead' as some people call it.  This is how far in the future the model can forecast (or predict).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reference to the GitHub source, the code uses a look ahead ('delay') of one.  The echo state network learns on the 'inputData' using a future version of itself called 'targetData'.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an analogy.  It is early on Friday, June 5, and I have the last 51 days worth of Amazon stock prices.  I can use the first 50 days of prices, which go up to and include Wednesday, June 3, to train a network to predict Thursday, June 4.  If I have a lot of historical data on Amazon, I can train many 50 day windows.  Each 50 day window would have as its target a stock price one day in the future. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performance generally degrades the further into the future you try to forecast.  One reason is due to the build up of error over time.  This is true in life.  We usually know fairly accurately what we will do tomorrow, less so next week, and it would be a wild guess to predict what we will be doing in ten years.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-06-07T14:52:01.153" LastActivityDate="2020-06-07T14:52:01.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21707" PostTypeId="1" AcceptedAnswerId="21725" CreationDate="2020-06-07T15:19:46.003" Score="2" ViewCount="64" Body="&lt;p&gt;In Batch Normalisation, are the sample mean and standard deviation we normalise by the mean/sd of the original data put into the network, or of the inputs in the layer we are currently BN'ing over? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, suppose I have a mini-batch size of 2 which contains &lt;span class=&quot;math-container&quot;&gt;$\textbf{x}_1, \textbf{x}_2$&lt;/span&gt;. Suppose now we are at the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th layer and the outputs from the previous layer are &lt;span class=&quot;math-container&quot;&gt;$\tilde{\textbf{x}}_1,\tilde{\textbf{x}}_2$&lt;/span&gt;. When we perform batch norm at this layer would be subtract the sample mean of &lt;span class=&quot;math-container&quot;&gt;$\textbf{x}_1, \textbf{x}_2$&lt;/span&gt; or of &lt;span class=&quot;math-container&quot;&gt;$\tilde{\textbf{x}}_1,\tilde{\textbf{x}}_2$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition tells me that it must be the mean,sd of &lt;span class=&quot;math-container&quot;&gt;$\tilde{\textbf{x}}_1,\tilde{\textbf{x}}_2$&lt;/span&gt; otherwise I don't think it would be normalised to have 0 mean and sd of 1. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-07T18:39:29.920" LastActivityDate="2020-06-08T17:26:43.317" Title="In Batch Normalisation, are $\hat{\mu}$, $\hat{\sigma}$ the mean and stdev of the original mini-batch or of the input into the current layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;batch-normalization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21708" PostTypeId="1" CreationDate="2020-06-07T16:24:40.417" Score="2" ViewCount="86" Body="&lt;p&gt;I'm training a dueling double DQN agent with prioritized replay buffer and notice that the min Q values are decreasing, while the max Q values are increasing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a sign that it is diverging?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or should be just be looking at the mean Q value, which has a slight uptrend?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bMV3O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bMV3O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;sup&gt;1&lt;/sup&gt; There are 2 different colored lines because the initial training (in orange) was stopped at around 1.3M time steps, and resumed (in blue) from a checkpoint at around 1.1M time steps&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;sup&gt;2&lt;/sup&gt; Plots are from Tensorboard, visualizing data generated by Ray/RLlib&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;sup&gt;3&lt;/sup&gt; Epsilon starts at 1.0 and anneals to 0.02 over 10000 time steps. The sudden increase in magnitude of Q-values appear to come after resuming from checkpoint, but might just be a coincidence.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;After training for more steps...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/55v8O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/55v8O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37519" LastEditorUserId="37519" LastEditDate="2020-06-08T03:52:30.147" LastActivityDate="2020-06-08T03:52:30.147" Title="If the minimum Q value is decreasing and the maximum Q value increasing, is this a sign that dueling double DQN is diverging?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;dqn&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21713" PostTypeId="1" CreationDate="2020-06-07T19:35:55.130" Score="0" ViewCount="41" Body="&lt;p&gt;In ham radio, Morse code is used in CW transmissions (yes even today).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My project is to take already decoded Morse code text strings from an existing decoder and then recognize different conversations (QSOs) and their components such as call signs, signal reports, location and other aspects. Here’s the general structure of these conversations as background:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://naqcc.info/cw_qsos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://naqcc.info/cw_qsos.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, from this CW string decoded from the radio:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N3AQC DE K3WWP GM TNX CALL UR 599 599 IN KITTANNING PA KITTANNING PA NAME IS JOHN JOHN HW? AR N3AQC DE K3WWP K&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extract the following: &#xA;Conversation Type: &#xA;Call Reply &#xA;Call sign: N3AQC &#xA;Signal Rpt: 599 &#xA;Location: KITTANNING PA &#xA;Name: JOHN&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m new to NLP tools in Python but have some experience with deep learning networks. PyText looks interesting but with no NLP experience it’d be helpful to get some guidance at the outset around which NLP tools best match the requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is what NLP toolsets, ideally in Python, are best suited to quickly construct a parser for these ham radio Morse code conversations to identify the conversation type (contest, regular QSO, etc) and it’s key components?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While machine learning isn’t an absolute requirement, as there are many other forms of parsers around, it’s an area I’d like to apply to this problem as a ML learning exercise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rick, W5FCX&lt;/p&gt;&#xA;" OwnerUserId="37687" LastActivityDate="2020-06-07T19:35:55.130" Title="Which NLP Parser for Morse conversations?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21714" PostTypeId="2" ParentId="21692" CreationDate="2020-06-07T21:28:09.570" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to know, since the input is given to a CNN, how would we encode these frames to serve as input to the CNN? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As nbro mentioned in a comment to your answer, this question has very recently been asked and answered &lt;a href=&quot;https://ai.stackexchange.com/a/21673/37607&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I also want to know since &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; is equal to a set, which can be seen in line 4 of the algorithm, then why is &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; equal to &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The algorithm presented in the original DQN &lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; is relatively simple and written to express the main ideas of their approach (e.g. experience replay, preprocessing histories, gradient descent, etc.); in fact, it isn't even the exact algorithm used in the experiments! For example, the experiments use frame-skipping to reduce computation - this is not mentioned in Algorithm 1 in the paper. With that in mind, setting &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; equal to &lt;span class=&quot;math-container&quot;&gt;$s_t, a_t, x_{t+1}$&lt;/span&gt; in the algorithm signifies a &lt;strong&gt;general notion&lt;/strong&gt; of constructing the next raw state &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; from the previous preprocessed state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, previous action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;, and current frame &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt;. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the action space at the next timestep is constrained by the state, then the state may need additional parameters to encode the action space.&lt;/li&gt;&#xA;&lt;li&gt;The algorithm needs some indication if a state is terminal, and such an indication may need to be encoded in the state.&lt;/li&gt;&#xA;&lt;li&gt;If there is frame skipping, then multiple frames will be needed to construct the next state, possibly using the previous state as well.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The above examples should display how &lt;strong&gt;the encoding of the state cannot always simply be a stack of raw frames, or even a function of &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_{t+1}$&lt;/span&gt;, and therefore a more general approach is often required&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-07T21:28:09.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21715" PostTypeId="1" CreationDate="2020-06-07T22:13:15.413" Score="1" ViewCount="56" Body="&lt;p&gt;When I say template matching, I'm referring to finding occurrences of a small image (the template) in a larger image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The OpenCV library provides the trivial solution, that slides the template over every possible location of the image. While this implementation provides translation invariance, it provides no stretching/scaling invariance. To get stretch &lt;em&gt;and&lt;/em&gt; translation invariance, one would need to stretch the template (or shrink the main image) until iteratively, running the original template check over the image. This increases complexity to &lt;span class=&quot;math-container&quot;&gt;$O(S * n^2)$&lt;/span&gt; where S is the number of different resolutions one checks - if one want's to check every possible resolution, the overall complexity is &lt;span class=&quot;math-container&quot;&gt;$O(n^4)$&lt;/span&gt;. Effectively, you generate &lt;span class=&quot;math-container&quot;&gt;$O(n^4)$&lt;/span&gt; subsections and check if they're equal to the template.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I was taught, Image Segmentation Networks do just this - however, instead of using the basic template matching (i.e. checking the pixels match) the generated subsection is put through a classifier network - so this would be more expensive than standard templating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KURsI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KURsI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, are my calculations correct - for complete subsection generation, is the complexity &lt;span class=&quot;math-container&quot;&gt;$O(n^4)$&lt;/span&gt; and is there really no better algorithm for generating these subsections - used by both image detection algorithms?&lt;/p&gt;&#xA;" OwnerUserId="37690" LastEditorUserId="2444" LastEditDate="2020-06-08T20:01:09.933" LastActivityDate="2020-06-08T20:01:09.933" Title="Is subsection generation $O(n^4)$" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;algorithm&gt;&lt;image-processing&gt;&lt;time-complexity&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21717" PostTypeId="2" ParentId="21684" CreationDate="2020-06-08T03:27:02.593" Score="4" Body="&lt;p&gt;David Ireland gives a fantastic answer, and I will provide an intuitive and gentle (but less rigorous) answer for those who are unfamiliar with the relevant statistical concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Next reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;:&lt;/strong&gt; The next reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; is solely dependent on the current state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;. It is only dependent on the policy because the policy details the probability distribution of actions given a state. Since we assume that the current state and action are given when calculating the expectation &lt;span class=&quot;math-container&quot;&gt;$\left(S_t = s, A_t = a\right)$&lt;/span&gt;, then the policy does not give us any new information, and therefore the next reward is independent of the policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Return &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt;&lt;/strong&gt;: By definition, &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s') = \mathbb{E}_{\pi}[G_{t+1}|S_{t+1} = s']$&lt;/span&gt;. The value function is unaffected by sampling actions from the policy in the outer expectation &lt;span class=&quot;math-container&quot;&gt;$\left(\mathbb{E}_{\pi}[v_{\pi}(s')] = \mathbb{E}[v_{\pi}(s')]\right)$&lt;/span&gt; since the value function is an expectation under the policy, and hence samples actions from the policy already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Dropping &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}$&lt;/span&gt;&lt;/strong&gt;: The expectation under the current policy samples next states and rewards from the environment and also samples actions from our policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Because the next reward is independent of the policy given the current state and action, and because the value function is unaffected by sampling actions from the policy in the outer expectation, we can simply drop the policy from the outer expectation (the outer expectation will still sample next states and rewards from the environment). &lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-08T03:27:02.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21719" PostTypeId="1" AcceptedAnswerId="21727" CreationDate="2020-06-08T05:34:00.550" Score="4" ViewCount="1152" Body="&lt;p&gt;Lately, there are lots of posts on &lt;em&gt;one-shot&lt;/em&gt; learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to &lt;em&gt;transfer&lt;/em&gt; learning, in which we can use pre-trained model weights to create our own model. &lt;em&gt;Fine-tuning&lt;/em&gt; also seems a similar concept to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me and explain the differences between all three of them?&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="2444" LastEditDate="2020-06-08T14:22:27.833" LastActivityDate="2020-06-08T15:51:17.083" Title="What is the difference between one-shot learning, transfer learning and fine tuning?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;transfer-learning&gt;&lt;one-shot-learning&gt;&lt;fine-tuning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21723" PostTypeId="1" AcceptedAnswerId="21724" CreationDate="2020-06-08T12:26:36.383" Score="3" ViewCount="230" Body="&lt;p&gt;The task (exercise 3.13 in the RL book by Sutton and Barto) is to express &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt; as a function of &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt; is the action-value function, that states how good it is to be at some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in the Markov Decision Process (MDP), if at that state, we choose an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, and after that action, the policy &lt;span class=&quot;math-container&quot;&gt;$\pi(s,a)$&lt;/span&gt; determines future actions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say that we are at some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and we choose an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. The probability of landing at some other state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is determined by &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt;. Each new state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; then has a state-value function that determines how good is it to be at &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; if all future actions are given by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi(s',a)$&lt;/span&gt;, therefore: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_\pi(s,a) = \sum_{s' \in S} p(s',r|s,a) v_\pi(s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="2444" LastEditDate="2020-06-08T23:18:57.627" LastActivityDate="2020-06-08T23:18:57.627" Title="How do we express $q_\pi(s,a)$ as a function of $p(s',r|s,a)$ and $v_\pi(s)$?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21724" PostTypeId="2" ParentId="21723" CreationDate="2020-06-08T12:42:00.583" Score="5" Body="&lt;p&gt;Not quite. You are missing the reward at time step &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition you are looking for is (leaving out the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; subscripts for ease of notation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q(s,a) = \mathbb{E}[R_{t+1} + \gamma v(s') | S_t=s,A_t=a] = \sum_{r,s'}(r + v(s'))p(s',r|s,a)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; relates to expected returns at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, and returns are defined as &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{b = 0}^\infty \gamma ^b R_{t+b+1}$&lt;/span&gt;, thus &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; is also a random variable at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; that we need to take expectation with respect to, not just the state that we transition into. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-08T13:09:54.730" LastActivityDate="2020-06-08T13:09:54.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21725" PostTypeId="2" ParentId="21707" CreationDate="2020-06-08T12:42:52.353" Score="1" Body="&lt;p&gt;Your intuition is correct. We will be normalizing the inputs of the layer under consideration (just right before applying the activation function). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if this layer receives an input &lt;span class=&quot;math-container&quot;&gt;$\mathrm{x}=\left(x^{(1)} \ldots x^{(d)}\right)$&lt;/span&gt;, the formula for normalizing the &lt;span class=&quot;math-container&quot;&gt;$k^{th}$&lt;/span&gt; dimension of &lt;span class=&quot;math-container&quot;&gt;$\mathrm{x}$&lt;/span&gt; would look as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\widehat{x}^{(k)}=\frac{x^{(k)}-\mathrm{E}\left[x^{(k)}\right]}{\sqrt{\operatorname{Var}\left[x^{(k)}\right]}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in practice a constant &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is also added under the square root in the denominator to ensure stability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: The original &lt;a href=&quot;https://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Batch Normalization paper&lt;/a&gt; (Section 3). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Andrew Ng's &lt;a href=&quot;https://www.youtube.com/watch?v=em6dfRxYkYU&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; on this topic might also be useful for illustration. &lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="34010" LastEditDate="2020-06-08T17:26:43.317" LastActivityDate="2020-06-08T17:26:43.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21726" PostTypeId="1" AcceptedAnswerId="21733" CreationDate="2020-06-08T13:06:27.437" Score="5" ViewCount="199" Body="&lt;p&gt;I'm having difficulty understanding the distinction between a bandit problem and a non-bandit problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of the bandit problem is an agent playing &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; slot machines with the goal of discovering which slot machine is the most probable to return a reward. The agent learns to find the best strategy of playing and is allowed to pull the lever of one slot machine per time step. Each slot machine obeys a distinct probability of winning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my interpretation of this problem, there is no notion of state. The agent potentially can utilise the slot results to determine a state-action value? For example, if a slot machine pays when three apples are displayed, this is a higher state value than a state value where three apples are not displayed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is there just one state in the formulation of this bandit problem? As there is only one action (&quot;pulling the slot machine lever&quot; ), then there is one action. The slot machine action is to pull the lever, which starts the game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am taking this a step further now. An RL agent purchases &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; shares of an asset and its not observable if the purchase will influence the price. The next state is the price of the asset after the purchase of the shares. If &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is sufficiently large, then the price will be affected otherwise there is a minuscule if any effect on the share price. Depending on the number of shares purchased at each time step, it's either a bandit problem or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not a bandit problem if &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is large and the share price is affected? It is a bandit problem if &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is small and the share price is not affected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make sense to have a mix of a bandit and non-bandit states for a given RL problem? If so, then the approach to solving should be to consider the issue in its entirety as not being a bandit problem?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="2444" LastEditDate="2020-06-08T17:15:52.460" LastActivityDate="2020-06-08T17:18:59.423" Title="How do I recognise a bandit problem?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;multi-armed-bandit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21727" PostTypeId="2" ParentId="21719" CreationDate="2020-06-08T14:14:08.297" Score="5" Body="&lt;p&gt;They are all related terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From top to bottom:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;One-shot learning&lt;/strong&gt; aims to achieve results with one or very few examples. Imagine an image classification task. You may show an apple and a knife to a human and no further examples are needed to continue classifying. That would be the ideal outcome, but for algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to achieve one-shot learning (or close) we can rely on &lt;strong&gt;knowledge transfer&lt;/strong&gt;, just like the human in the example would do (we are trained to be amazing at image processing, but here we would also exploit other knowledge like abstract reasoning abilities, and so on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings us to &lt;strong&gt;transfer learning&lt;/strong&gt;. Generally speaking, transfer learning is a machine learning paradigm where we train a model on one problem and then try to apply it to a different one (after some adjustments, as we'll see in a second).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example above, classifying apples and knives is not at all trivial. However, if we are given a neural network that already excels at image classification, with super-human results in over 1000 categories... perhaps it is easy to adapt this model to our specific apples vs knives situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;adapting&quot;, those &quot;adjustments&quot;, are essentially what we call &lt;strong&gt;fine-tuning&lt;/strong&gt;. We could say that fine-tuning is the training required to adapt an already trained model to the new task. This is normally much less intensive than training from scratch, and many of the characteristics of the given model are retained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fine-tuning usually covers more steps. A typical pipeline in deep learning for computer vision would be this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get trained model (image classifier champion)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Note the head of our model does not match our needs (there's probably one output per category, and we only need two categories now!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Swap the very last layer(s) of the model, so that the output matches our needs, but keeping the rest of the architecture and already trained parameters intact.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train (fine-tune!) our model on images that are specific to our problem (only a few apples and knives in our silly example). We often only allow the last layers to learn at first, so they &quot;catch up&quot; with the rest of the model (in this case we talk about freezing and unfreezing and discriminative learning rates, but that's a bit beyond the question).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that some people may sometimes use fine-tuning as a synonym for transfer learning, so be careful about that!&lt;/p&gt;&#xA;" OwnerUserId="14744" LastEditorUserId="2444" LastEditDate="2020-06-08T15:51:17.083" LastActivityDate="2020-06-08T15:51:17.083" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21729" PostTypeId="5" CreationDate="2020-06-08T14:23:20.320" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-08T14:23:20.320" LastActivityDate="2020-06-08T14:23:20.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21730" PostTypeId="4" CreationDate="2020-06-08T14:23:20.320" Score="0" Body="For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-08T14:23:20.320" LastActivityDate="2020-06-08T14:23:20.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21731" PostTypeId="5" CreationDate="2020-06-08T14:24:22.057" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-08T14:24:22.057" LastActivityDate="2020-06-08T14:24:22.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21732" PostTypeId="4" CreationDate="2020-06-08T14:24:22.057" Score="0" Body="For questions related to the concept of one-shot learning, i.e. learning from one (or maybe a few) training examples (rather than big datasets)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-08T14:24:22.057" LastActivityDate="2020-06-08T14:24:22.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21733" PostTypeId="2" ParentId="21726" CreationDate="2020-06-08T14:48:51.483" Score="3" Body="&lt;p&gt;The bandit problem has one state, in which you are allowed to choose one lever among &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; levers to pull.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is there just one state in the formulation of this bandit problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is one state because the state does not change over time. Two notable consequences are that (i) pulling a lever does not change the internals of any slot machine (e.g. the distribution of rewards) and (ii) you are allowed to choose any lever without restrictions.  More generally, there is no &lt;em&gt;sequential&lt;/em&gt; aspect of the state in this problem, as future states are unaffected by past states, actions, and rewards. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is not a bandit problem if &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is large and the share price is affected?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Correct! If the share price is affected, then future states would be influenced by past actions. This is because the price per share is affected, which is one aspect of the state. Thus, you would need to plan a &lt;em&gt;sequential&lt;/em&gt; strategy for your purchases. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is a bandit problem if &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is small and the share price is not affected?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It all depends on the problem: as long as the state before buying shares remains completely the same after you purchase some shares, then yes. Share price being unaffected is only one of the requirements; another example requirement is that the maximum number of shares purchased is fixed at each time step, independent of the shares purchased previously. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it make sense to have a mix of a bandit and non-bandit states for a given RL problem? If so, then the approach to solving should be to consider the issue in its entirety as not being a bandit problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It makes sense to allow the share price to either be affected or unaffected based on &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in the same problem. Since some actions (large &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;) change the state, then there are multiple states, and actions &lt;em&gt;sequentially&lt;/em&gt; affect the next state. Hence it is not a bandit problem as a whole, as you correctly stated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The agent potentially can utilise the slot results to determine a state-action value?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Correct! I suggest reading Chapter 2 of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; to learn some fundamental algorithms of developing such strategies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nice work on analyzing this problem! To help solidify your understanding and formalize the arguments above, I suggest that you rewrite the variants of this problem as MDPs and determine which variants have multiple states (non-bandit) and which variants have a single state (bandit).&lt;/p&gt;&#xA;" OwnerUserId="37607" LastEditorUserId="2444" LastEditDate="2020-06-08T17:18:59.423" LastActivityDate="2020-06-08T17:18:59.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21734" PostTypeId="2" ParentId="20997" CreationDate="2020-06-08T15:56:17.367" Score="0" Body="&lt;p&gt;Are you sure the image quality in your test set and phone camera image similar?&#xA;I once trained a CNN model on poor quality images and with very good validation accuracy but when I tested on image from my camera it didn't work at all. I degraded the image quality by resizing the image from my camera to a small size then again back to required size and it worked perfectly.&lt;/p&gt;&#xA;" OwnerUserId="37512" LastActivityDate="2020-06-08T15:56:17.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21735" PostTypeId="2" ParentId="21682" CreationDate="2020-06-08T17:21:19.220" Score="0" Body="&lt;p&gt;To my knowledge from reading about model-based and model-free reinforcement learning,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DQN and Double DQN are model-free reinforcement learning methods. (Why am I mentioning this, see below):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/referenceworkentry/10.1007%2F978-1-4614-7320-6_674-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/referenceworkentry/10.1007%2F978-1-4614-7320-6_674-1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should see under the heading &quot;Definition&quot; (you may need to scroll down in the web-page) in the web-page (can be accessed by the above link), it states that: &quot;model-free techniques require extensive experience.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now extensive experience, depending of how fast you can go through states, can take a few days even weeks...model-free methods require a lot of samples to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also there are a lot of states: there are (256^128) states. That is a really big number (I'm just emphasizing that training may take a long while) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due implementation specifics not being supplied in question, I am assuming your implementation is correct, although...you are using RAM. DQN used image data, I very quickly skimmed the Double DQN paper: &lt;a href=&quot;https://arxiv.org/pdf/1509.06461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1509.06461.pdf&lt;/a&gt; . I am assuming they used image data as well because Double DQN was compared to DQN.&lt;/p&gt;&#xA;" OwnerUserId="30174" LastActivityDate="2020-06-08T17:21:19.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21737" PostTypeId="1" CreationDate="2020-06-09T01:28:12.383" Score="2" ViewCount="81" Body="&lt;p&gt;I have taken an algorithms course where we talked about LP significantly, and also many reductions to LPs. As I recall, normal LP is not NP-Hard. Integer LP is NP-Hard. I am currently taking an introduction to AI course, and I was wondering if CSP is the same as LP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems an awful lot of overlap, and I haven't been able to find anything that confirms or denies my suspicions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they are not the same (or one cannot be reduced to the other), what are the core differences in their concepts?&lt;/p&gt;&#xA;" OwnerUserId="37718" LastEditorUserId="2444" LastEditDate="2020-06-09T18:34:08.077" LastActivityDate="2020-07-09T19:00:40.347" Title="What are the differences between constraint satisfaction problems and linear programming?" Tags="&lt;comparison&gt;&lt;constraint-satisfaction-problems&gt;&lt;linear-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21738" PostTypeId="1" CreationDate="2020-06-09T02:25:58.773" Score="0" ViewCount="93" Body="&lt;p&gt;I have two sets of data, one training and one test set. I use the train set to train the deep q network model variant. I also continuously evaluate the agent Q values obtained on the test set every 5000 epochs I find that the agent Q values on the test set do not converge and neither do the policies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iteration &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;: Q values for the first 5 test data are &lt;code&gt;[15.271439, 13.013742, 14.137051, 13.96463, 11.490129]&lt;/code&gt; with policies: &lt;code&gt;[15, 0, 0, 0, 15]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iteration &lt;span class=&quot;math-container&quot;&gt;$x+10000$&lt;/span&gt;:&#xA;Q values for the first 5 test data are &lt;code&gt;[15.047309, 15.5233555, 16.786497, 16.100864, 13.066223]&lt;/code&gt; with policies: &lt;code&gt;[0, 0, 0, 0, 15]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the weights of the neural network are not converging. Although I can manually test each policy at each iteration and decide which of the policy performs best, I would like to know if correct training of the network would lead to weight convergence ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training loss plot:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ocSIG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ocSIG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that the loss decreases over time however, there are occasional spikes in the loss which does not seem to go away.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-06-09T12:50:53.450" LastActivityDate="2020-06-09T12:50:53.450" Title="Should the network weights converge when training Deep Q networks?" Tags="&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="21740" PostTypeId="1" CreationDate="2020-06-09T07:09:41.990" Score="0" ViewCount="39" Body="&lt;p&gt;Recently, I am working on an action recognition project where my input data is from the video stream. I read some of the concepts like ConvLstm, Convolutional Lstm, etc. I am looking for someone who already those kinds of staff already and can share his work with me that will be a really good help for me?  &lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-06-09T07:46:57.047" Title="Action recognition using video stream data" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21741" PostTypeId="1" CreationDate="2020-06-09T07:22:09.500" Score="1" ViewCount="61" Body="&lt;p&gt;I am building an &lt;code&gt;LSTM&lt;/code&gt; for predicting a price chart. &lt;code&gt;MAPE&lt;/code&gt; resulted in the best loss function compared to &lt;code&gt;MSE&lt;/code&gt; and &lt;code&gt;MAE&lt;/code&gt;. &lt;code&gt;MAPE&lt;/code&gt; function looks like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{N}\sum^N_{i=0}\frac{|P_i - A_i|}{A_i}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt; is the current predicted value and &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; is the corresponding actual value. In neural network, it is always advised to scale the data between a small range close to zeros such as &lt;code&gt;[0, 1]&lt;/code&gt;. In this case scaling range of &lt;code&gt;[0.001, 1]&lt;/code&gt; is imperative to remove a possible division by zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the &lt;code&gt;MAPE&lt;/code&gt; denominator, the close the scaling range is to zero the larger the loss function becomes for a given &lt;span class=&quot;math-container&quot;&gt;$|P_i - A_i|$&lt;/span&gt;. If on the other hand, the data is de-scaled just before it is inserted in the &lt;code&gt;MAPE&lt;/code&gt; function, the same &lt;span class=&quot;math-container&quot;&gt;$|P_i - A_i|$&lt;/span&gt; would give a smaller &lt;code&gt;MAPE&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a hypothetical example with a batch size of 1, &lt;span class=&quot;math-container&quot;&gt;$|P_i - A_i| = 2$&lt;/span&gt; (this is scale indipendent) and &lt;span class=&quot;math-container&quot;&gt;$A_i = 200$&lt;/span&gt;. Therefore scaled &lt;span class=&quot;math-container&quot;&gt;$A_i = 0.04$&lt;/span&gt;. The &lt;code&gt;MAPE&lt;/code&gt; error loss for the scaled version would be &lt;span class=&quot;math-container&quot;&gt;$\frac{2}{0.04} = 50$&lt;/span&gt;, and for the unscaled version &lt;span class=&quot;math-container&quot;&gt;$\frac{2}{200} = 0.01$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will mean that the derivative w.r.t each weight of the scaled version will also be larger, therefore making the weights themselves even smaller. Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am concluding that scaling the data when using &lt;code&gt;MAPE&lt;/code&gt; will effectively shrink the weights down more than necessary. Is that a good reason why I am seeing &lt;strong&gt;significantly&lt;/strong&gt; better performance with de-scaled &lt;code&gt;MAPE&lt;/code&gt; calculation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: I am not keeping the same hyperparameters for scaled and de-scaled &lt;code&gt;MAPE&lt;/code&gt; but a Bayesian optimization is performed with both runs, In the later a deeper network was preferred but in the scaled &lt;code&gt;MAPE&lt;/code&gt; more regularisation was preferred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some expertise on this would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="37728" LastActivityDate="2020-07-10T17:36:04.760" Title="LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation" Tags="&lt;long-short-term-memory&gt;&lt;loss-functions&gt;&lt;weights&gt;&lt;loss&gt;&lt;scalability&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21742" PostTypeId="2" ParentId="21740" CreationDate="2020-06-09T07:46:57.047" Score="1" Body="&lt;p&gt;If you want to build a stream recognition pipeline than you can use OpenCV, Kafka, and Spark together.&lt;a href=&quot;https://www.infoq.com/articles/video-stream-analytics-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;how to build pipeline of video stream&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for Action recognition using ConvLstm. This link will help you.&lt;a href=&quot;https://github.com/eriklindernoren/Action-Recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;action recognition using Convlstm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32861" LastActivityDate="2020-06-09T07:46:57.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21743" PostTypeId="1" AcceptedAnswerId="22577" CreationDate="2020-06-09T08:43:56.267" Score="1" ViewCount="79" Body="&lt;p&gt;Not sure if this is the right place, but I was wondering if someone could briefly explain to me the &lt;strong&gt;differences &amp;amp; similarities&lt;/strong&gt; between &lt;em&gt;simulated annealing&lt;/em&gt; and &lt;em&gt;deterministic annealing&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that both methods are used for optimization and both originate from statistical physics with the intuition of reaching a minimum energy (cost) configuration by &lt;em&gt;cooling&lt;/em&gt; (i.e. slowly reducing the temperature in the Boltzmann distribution to calculate probabilities for configurations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, Wikipedia has no article about deterministic annealing and the one about &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulated annealing&lt;/a&gt; does not mention any comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.intechopen.com/books/computational-optimization-in-engineering-paradigms-and-applications/deterministic-annealing-a-variant-of-simulated-annealing-and-its-application-to-fuzzy-clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; resource has a brief comparison section between the two methods, however, I do not understand why the search strategy of DA is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;based on the &lt;strong&gt;steepest descent algorithm&lt;/strong&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and how &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;it searches the local minimum &lt;strong&gt;deterministically&lt;/strong&gt; at each temperature.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any clarification appreciated.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="2444" LastEditDate="2020-06-11T13:26:54.700" LastActivityDate="2020-07-19T14:58:25.713" Title="What is the difference between simulated annealing and deterministic annealing?" Tags="&lt;comparison&gt;&lt;optimization&gt;&lt;simulated-annealing&gt;&lt;deterministic-annealing&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21746" PostTypeId="5" CreationDate="2020-06-09T10:53:00.553" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.iro.umontreal.ca/~lisa/pointeurs/hierarchical-nnlm-aistats05.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical softmax&lt;/a&gt; is a replacement for softmax which is must faster to evaluate. While softmax is  &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;  time, hierarchical softmax is  &lt;span class=&quot;math-container&quot;&gt;$O(logn)$&lt;/span&gt;  time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/What-is-hierarchical-softmax&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical Softmax (HS)&lt;/a&gt; is a solution to the enormous compute cost of training a neural network which uses the Softmax activation and has thousands of outputs. For example, in a word model like word2vec, you have one output for every word in your vocabulary, which can easily mean 100s of thousands of outputs. Without an approximation technique like HS, word2vec wouldn’t be feasible to train.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:33:44.880" LastActivityDate="2020-06-16T19:33:44.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21747" PostTypeId="4" CreationDate="2020-06-09T10:53:00.553" Score="0" Body="For questions related to Hierarchical softmax" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:28:14.807" LastActivityDate="2020-06-16T19:28:14.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21748" PostTypeId="5" CreationDate="2020-06-09T10:56:13.427" Score="0" Body="&lt;p&gt;In a structured representation of knowledge &lt;a href=&quot;http://artificialintelligence-notes.blogspot.com/2010/07/conceptual-dependency-cd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conceptual Dependency&lt;/a&gt; representation is used to place the real-world events or situations information on AI systems i.e to act as Intelligent Computer System., for that we using Conceptual Dependency primitives to represent the events information on systems knowledge-base. It is one of the procedures to place events information on the AI system and applying appropriate Procedural Mechanism on structured Representation of Knowledge., i.e we can exploit the represented structures by using procedural mechanisms either in LISP or PROLOG Language.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:28:11.963" LastActivityDate="2020-06-16T19:28:11.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21749" PostTypeId="4" CreationDate="2020-06-09T10:56:13.427" Score="0" Body="For questions related to Conceptual Dependency" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:30:28.087" LastActivityDate="2020-06-16T19:30:28.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21750" PostTypeId="5" CreationDate="2020-06-09T10:58:54.520" Score="0" Body="&lt;p&gt;A focused crawler focuses its indexing on topic-relevant and particularly current websites that collect Web pages that satisfy some specific property, by carefully prioritizing the crawl frontier and managing the hyperlink exploration process. Some predicates may be based on simple, deterministic, and surface properties.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:35:22.903" LastActivityDate="2020-06-16T19:35:22.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21751" PostTypeId="4" CreationDate="2020-06-09T10:58:54.520" Score="0" Body="For questions related to Focused Crawler which is a topic-specific and aims selectively to collect web pages that are relevant to a given topic from the Internet" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:35:19.997" LastActivityDate="2020-06-16T19:35:19.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21752" PostTypeId="5" CreationDate="2020-06-09T11:00:10.770" Score="0" Body="&lt;p&gt;Disentangled representation is an unsupervised learning technique that breaks down, or disentangles, each feature into narrowly defined variables and encodes them as separate dimensions. The goal is to mimic the quick intuition process of a human, using both “high” and “low” dimension reasoning.&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:28:07.790" LastActivityDate="2020-06-16T19:28:07.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21753" PostTypeId="4" CreationDate="2020-06-09T11:00:10.770" Score="0" Body="For questions related to Disentangled representation, an unsupervised learning technique" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:28:21.130" LastActivityDate="2020-06-16T19:28:21.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21754" PostTypeId="5" CreationDate="2020-06-09T11:13:57.770" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-09T11:13:57.770" LastActivityDate="2020-06-09T11:13:57.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21755" PostTypeId="4" CreationDate="2020-06-09T11:13:57.770" Score="0" Body="For questions related to the capsule neural network, which is a neural network that attempts to more closely mimic the biological neural organization." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-09T11:13:57.770" LastActivityDate="2020-06-09T11:13:57.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21756" PostTypeId="5" CreationDate="2020-06-09T11:56:46.453" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-09T11:56:46.453" LastActivityDate="2020-06-09T11:56:46.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21757" PostTypeId="4" CreationDate="2020-06-09T11:56:46.453" Score="0" Body="For questions related to the concept of a topic model, which is a type of statistical model for discovering the abstract &quot;topics&quot; that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden semantic structures in a text body." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-09T11:56:46.453" LastActivityDate="2020-06-09T11:56:46.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21758" PostTypeId="1" CreationDate="2020-06-09T12:02:24.310" Score="1" ViewCount="42" Body="&lt;p&gt;I am interested in capsule neural networks. I have already read the paper &lt;a href=&quot;https://arxiv.org/abs/1710.09829&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Routing Between Capsules&lt;/a&gt;, but it is a little bit difficult to follow. Is there a tutorial for beginners on capsule neural networks?&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="2444" LastEditDate="2020-06-09T14:51:33.730" LastActivityDate="2020-06-09T14:51:33.730" Title="Is there a tutorial for beginners on capsule neural networks?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;resource-request&gt;&lt;capsule-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21759" PostTypeId="2" ParentId="21737" CreationDate="2020-06-09T12:52:56.937" Score="1" Body="&lt;p&gt;LP is a mathematical problem to optimize a linear function subject to linear (in)equality constraints of the sort:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;min_x  w^t x&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Ax&amp;lt;b&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a continuous variable (vector). If the problem is feasible, then a unique global optimal solution exists for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A constraint satisfaction problem on the other hand is just the constraints part of the above LP. So we are not interested in the optimal solution, just a set of values that will satisfy the constraints. If the domain of your variables is continuous, then it can be recast as a dummy LP with a fake objective e.g. &lt;span class=&quot;math-container&quot;&gt;$min_x 0, s.t. Ax&amp;lt;b$&lt;/span&gt;, and then solved with an LP solver (e.g. simplex algorithm)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However many CSPs have discrete variables so they are not LP. Also CSP problems may have too many variables for plain LP algorithms to handle, so there are problem specific shortcut algorithms. We are often happy to find a feasible solution and forget about &quot;optimality&quot;&lt;/p&gt;&#xA;" OwnerUserId="36518" LastActivityDate="2020-06-09T12:52:56.937" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21760" PostTypeId="1" CreationDate="2020-06-09T13:09:46.643" Score="1" ViewCount="43" Body="&lt;p&gt;I have some trouble with the probability densities described in the original paper.  My question is based on Goodfellow's paper and tutorial, respectively: &lt;a href=&quot;https://arxiv.org/abs/1406.2661&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Adversarial Networks&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1701.00160&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIPS 2016 Tutorial: Generative Adversarial Networks&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Goodfellow et al. talk about probability distributions/densities in their paper, are they talking about discrete or continuous probability distributions? I don't think it's made clear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the continuous case, it would imply, for instance, that both &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt;  must be differentiable since the optimal discriminator (see Prop. 1) is essentially a function of their ratio and is assumed to be differentiable. Also, the existence of a continuous &lt;span class=&quot;math-container&quot;&gt;$p_g$&lt;/span&gt; is non-trivial. One sufficient condition would be that &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is a diffeomorphism (see normalising flows), but this is rarely the case. So it seems that much stronger assumptions are needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case that the answer is discrete distributions: the differentiability of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; implies continuous outputs of the generator. How can this work together with a discrete distribution of its outputs? Does the answer have something to do with the fact that we can only represent a finite set of numbers with computers anyway?&lt;/p&gt;&#xA;" OwnerUserId="37736" LastEditorUserId="2444" LastEditDate="2020-06-09T20:33:17.983" LastActivityDate="2020-06-09T20:33:17.983" Title="Is the generator distribution in GAN's continuous or discrete?" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;probability&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21761" PostTypeId="1" CreationDate="2020-06-09T13:52:20.430" Score="0" ViewCount="120" Body="&lt;p&gt;While implementing shi-tomasi corner detection algorithm i got stuck at deciding a suitable threshold for corner detection. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In shi-tomasi algorithm all those points that qualify &lt;code&gt;min(λ1 , λ2) &amp;gt; threshold&lt;/code&gt; are considered as corner points. (where &lt;code&gt;λ1, λ2&lt;/code&gt; are eigenvalues )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what is a suitable criteria to decide that &lt;code&gt;threshold&lt;/code&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="36484" LastActivityDate="2021-04-06T01:05:45.600" Title="How to choose a suitable threshold value for shi-tomasi corner detection algorithm?" Tags="&lt;computer-vision&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21762" PostTypeId="1" CreationDate="2020-06-09T15:02:35.630" Score="0" ViewCount="29" Body="&lt;p&gt;I have a problem where I know &lt;strong&gt;p&lt;/strong&gt; features which are each scalar values and the output of 1 set of those features is a time history. Is there a specific neural network (NN) type architecture that can handle this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my searching, a 1D convolutional NN can do the opposite of what I need where it goes from time history to scalar prediction. A recurrent NN can go from time history to time history so in theory I could just replicate my scalar features to be constant in time. However, that seems very inefficient and I'm not sure how well the model would learn with constant inputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also considered breaking the time history output into &lt;strong&gt;m&lt;/strong&gt; principal components (PCs) using functional PCA (FPCA). Then I could fit either &lt;strong&gt;m&lt;/strong&gt; different regular NNs to each PC or fit a single regular NN to all the PCs simultaneously. In the first case, I'd have to fit multiple networks, which seems less than ideal. In both cases, I'd have to represent my data using PCs, which means I'm only using a finite approximation to the data and not the actual data. However, in the absence of any other existing architectures, the second approach with a single NN and FPCA seems the most promising so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it matters, typical values for the size I'm working with are: number of features &lt;strong&gt;p&lt;/strong&gt;=5-20, number of data points &lt;strong&gt;n&lt;/strong&gt;=5000-25000, and when I've done FPCA before &lt;strong&gt;m&lt;/strong&gt;=3-7 to capture &gt;99% of the variance. Each time series output can be very general and does not necessarily follow a continuous function (&lt;em&gt;e.g.&lt;/em&gt; linear, polynomial, &lt;em&gt;etc.&lt;/em&gt;).&lt;/p&gt;&#xA;" OwnerUserId="37398" LastActivityDate="2020-06-09T15:02:35.630" Title="Neural Network architecture for going from scalar input to time series outputs?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;architecture&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21764" PostTypeId="1" CreationDate="2020-06-09T19:42:50.047" Score="3" ViewCount="433" Body="&lt;p&gt;In Reinforcement Learning, when reward function is not differentiable, a policy gradient algorithm is used to update the weights of a network. In &lt;a href=&quot;https://arxiv.org/abs/1611.01578&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper Neural Architecture Search with Reinforcement Learning&lt;/a&gt; they use accuracy of one neural network as the reward signal then choose a policy gradient algorithm to update weights of another network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot wrap my head around the concept of accuracy as a non-differentiable reward function. Do we need to find the function and then check if it is mathematically non-differentiable? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if I can use another value, for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Silhouette_(clustering)&quot; rel=&quot;nofollow noreferrer&quot;&gt;silhouette score&lt;/a&gt; (in a different scenario) as the reward signal?&lt;/p&gt;&#xA;" OwnerUserId="37744" LastEditorUserId="1847" LastEditDate="2020-06-10T13:03:20.893" LastActivityDate="2020-06-10T15:45:51.617" Title="Non-differentiable reward function to update a neural network" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21766" PostTypeId="1" CreationDate="2020-06-10T02:35:45.813" Score="0" ViewCount="54" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1901.03559&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Investigation of Model-Free Planning&lt;/a&gt;, the authors use ConvLSTM to learn a planning function. In particular, for each input &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; at time-step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the function approximator is run for 3 internal ticks. I wonder how these fake ticks help the learning. Also, when tested, the trained RL agent (with frozen weights) is given up to 10 no-op actions and this improves its predictions. Do you have an idea why that works? (It seems like magic to me.)&lt;/p&gt;&#xA;" OwnerUserId="2254" LastEditorUserId="2444" LastEditDate="2021-01-31T18:58:27.790" LastActivityDate="2021-01-31T18:58:27.790" Title="Why in RL function approximators with recurrent structures can learn planning?" Tags="&lt;reinforcement-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;papers&gt;&lt;planning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21767" PostTypeId="1" CreationDate="2020-06-10T02:46:22.760" Score="4" ViewCount="198" Body="&lt;p&gt;Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (&lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;page 131 of Barto and Sutton's book&lt;/a&gt; but not only), we the capitals are used &lt;span class=&quot;math-container&quot;&gt;$Q(S, A)$&lt;/span&gt;, while the Bellman equation it is &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt;? &lt;/p&gt;&#xA;" OwnerUserId="2254" LastEditorUserId="2444" LastEditDate="2020-06-10T10:13:10.623" LastActivityDate="2020-06-10T10:17:58.523" Title="Why are the value functions sometimes written with capital letters and other times with lower-case letters?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;notation&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21769" PostTypeId="2" ParentId="21767" CreationDate="2020-06-10T04:45:57.627" Score="1" Body="&lt;h1&gt;Ordinary variables vs Random Variables&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The difference is whether you're talking about a ordinary variable or a random variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the q-function (lowercase) is an expectation value (i.e. not a random variable), conditioned on a specific state-action pair:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;q(s,a)\ =\ \mathbb{E}_t\left\{ R_t+\gamma R_{t+1} + \gamma^2R_{t+2}+\dots\,\Big|\, S_t=s, A_t=a \right\}&#xA;$$&lt;/span&gt;&#xA;Then, in some case, some authors may abuse notation slightly by feeding in a random variable into the q-function, e.g. &lt;span class=&quot;math-container&quot;&gt;$q(S_t,a)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$q(s,A_t)$&lt;/span&gt; or even &lt;span class=&quot;math-container&quot;&gt;$q(S_t,A_t)$&lt;/span&gt;, thereby &lt;em&gt;undoing&lt;/em&gt; some or all of the conditioning in the definition of the q-function as an expectation value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feeding a random variable into a function like the q-function results in an output that is a random variable in its own right. It is for this reason that some authors choose to give the function itself an uppercase letter as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice would be to think to yourself, &lt;em&gt;is this a random variable?&lt;/em&gt; For the rest, I would interpret upper/lowercase as no more than a hint to the reader.&lt;/p&gt;&#xA;" OwnerUserId="37751" LastActivityDate="2020-06-10T04:45:57.627" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21771" PostTypeId="1" CreationDate="2020-06-10T08:12:03.210" Score="1" ViewCount="42" Body="&lt;p&gt;I have tried implementing a basic version of shi-tomasi corner detection algorithm. The algorithm works fine for corners but I came across a strange issue that the algorithm also gives high values for slanted(titled) edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what i did&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Took gray scale image&lt;/li&gt;&#xA;&lt;li&gt;computer dx, and dy of the image by convolving it with sobel_x and sobel_y&lt;/li&gt;&#xA;&lt;li&gt;Took a 3 size window and moved it across the image to compute the sum of the elements in the window.&lt;/li&gt;&#xA;&lt;li&gt;computed sum of the window elements from the dy image and sum of window elements from the dx image and saved it in sum_xx and sum_yy.&lt;/li&gt;&#xA;&lt;li&gt;created a new image (call it &lt;code&gt;result&lt;/code&gt;) where that pixel for which the window sum was computed was replaced with &lt;code&gt;min(sum_xx, sum_yy)&lt;/code&gt; as shi-tomasi algorithm requires.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I expected it to give maximum value for corners where dx and dy both are high, but i found it giving high values even for titled edges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the some outputs of the image i received: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/MMGTT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MMGTT.png&quot; alt=&quot;Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IPBvU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IPBvU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so far so good, corners have high values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another Image:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fu4r5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fu4r5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sLWNV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sLWNV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's where the problem lies. edges have high values which is not expected by the algorithm. I can't fathom how can edges have high values for both x and y gradients (sobel being close approximation of gradient). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask your help, if you can help me fix this issue for edges. I am open to any suggestions and ideas .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my code (if it helps):&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def shi_tomasi(image, w_size):&#xA;    ans = image.copy()&#xA;    dy, dx = sy(image), sx(image)&#xA;&#xA;    ofset = int(w_size/2)&#xA;    for y in range(ofset, image.shape[0]-ofset):&#xA;        for x in range(ofset, image.shape[1]-ofset):&#xA;&#xA;            s_y = y - ofset&#xA;            e_y = y + ofset + 1&#xA;&#xA;            s_x = x - ofset&#xA;            e_x = x + ofset + 1&#xA;&#xA;            w_Ixx = dx[s_y: e_y, s_x: e_x]&#xA;            w_Iyy = dy[s_y: e_y, s_x: e_x]&#xA;&#xA;            sum_xx = w_Ixx.sum()&#xA;            sum_yy = w_Iyy.sum()&#xA;&#xA;            ans[y][x] = min(sum_xx, sum_yy)&#xA;    return ans&#xA;&#xA;def sy(img):&#xA;    t = cv2.Sobel(img,cv2.CV_8U,0,1,ksize=3)&#xA;    return t&#xA;def sx(img):&#xA;    t = cv2.Sobel(img,cv2.CV_8U,1,0,ksize=3)&#xA;    return t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36484" LastActivityDate="2020-06-10T12:04:13.240" Title="Corner detection algorithm gives very high value for slanted edges?" Tags="&lt;python&gt;&lt;computer-vision&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21772" PostTypeId="2" ParentId="21767" CreationDate="2020-06-10T10:17:58.523" Score="4" Body="&lt;p&gt;In the Sutton and Barto book &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; is used to denote the &lt;em&gt;true&lt;/em&gt; expected value of taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, whereas capital &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; is used to denote an &lt;em&gt;estimate&lt;/em&gt; of &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt;. However, there is likely to be a lot of inconsistency in the literature as each author has their own preference on how to denote things. I would encourage you to consider whether the value you are reading is to denote an estimate or the true value. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-10T10:17:58.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21776" PostTypeId="2" ParentId="21771" CreationDate="2020-06-10T12:04:13.240" Score="1" Body="&lt;p&gt;I can't go into details of the algorithm but here's some intuition about what's apparently going wrong:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Sobel transformation identifies mostly-vertical and mostly-horizontal edges. For slanted edges, it also shows a response, just a bit weaker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using a window and taking the minimum of vertical and horizontal response, you identify points where you have both a vertical and a horizontal response. However, this is also the case at the slanted edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the structure of this simplistic corner detection algorithm, you can only expect to get meaningful results for strictly horizontal and vertical lines/edges and their intersections/corners. It is just not suitable for slanted edges.&lt;/p&gt;&#xA;" OwnerUserId="22993" LastActivityDate="2020-06-10T12:04:13.240" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21780" PostTypeId="2" ParentId="21382" CreationDate="2020-06-10T13:14:01.577" Score="2" Body="&lt;p&gt;Mathematical equations are generally expressed in a sequential form known as 'infix notation'.  It is characterised by the placement of operators between operands. To make the order of the operations in the Infix notation unambiguous, a lot of parenthesis are needed. Infix notation is more difficult to parse by computers than prefix notation (e.g. + 2 2) or postfix notation (e.g. 2 2 +). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a deep learning approach to symbolic mathematics recommended in the research paper by Guillaume Lample and François Charton. They have found an interesting approach to use deep neural networks for symbolic integration and differentiation equations. This paper proposes a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1912.01412&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning for Symbolic Mathematics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qyzIx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qyzIx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is essentially representing mathematical problems in prefix notation. First a symbolic syntax tree is constructed that captures the order and values of the operations in the expression. Second, the tree is traversed from top to bottom and from left to right. If the current node is a primitive value (a number), add it to the sequence string. If the current node is a binary operation, add the operations symbol to the sequence string. Then, add the representation of the left child node (could be recursive). Then, add the representation of the right child node. This procedure resulted in the following expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tBYUD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tBYUD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can expect further more advances in this area with the emergence of better symbolic learning models leveraging attention based transformers and other neural symbolic learning models. Recent work by MIT, DeepMind and IBM has shown the power of combining connectionist techniques like deep neural networks with symbolic reasoning. Please find the details in the following article. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://deepmind.com/research/publications/neuro-symbolic-concept-learner-interpreting-scenes-words-and-sentences-natural-supervision&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Neuro-Symbolic Concept Learner&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12861" LastActivityDate="2020-06-10T13:14:01.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21781" PostTypeId="5" CreationDate="2020-06-10T13:20:39.687" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-10T13:20:39.687" LastActivityDate="2020-06-10T13:20:39.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21782" PostTypeId="4" CreationDate="2020-06-10T13:20:39.687" Score="0" Body="For questions related to deep neural networks, which are artificial neural networks with &quot;many&quot; layers, where &quot;many&quot; can vary depending on the context." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-10T13:20:39.687" LastActivityDate="2020-06-10T13:20:39.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21783" PostTypeId="2" ParentId="21764" CreationDate="2020-06-10T13:21:22.900" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I cannot wrap my head around the concept of accuracy as a non-differentiable reward function. Do we need to find the function and then check if it is mathematically non-differentiable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In reinforcement learning (RL), the reward function is often not differentiable with respect to any learnable parameters. In fact it is quite common to not know the function at all, and apply a model-free learning method based purely on sampling many observations of state, action, reward and next state. For reward, you don't need to know the reward function as a function of your parameters, but you do need to know how to caclulate it from each &lt;em&gt;observation&lt;/em&gt;. That means that the reward is either provided as part of the environment, or it is clear how to calculate it from the initial state, action or next state. In the case of an agent that creates a neural network then reports the accuracy, you can probably view the NN training process and its results including accuracy on a validation set as a &quot;black box&quot; that reports back an arbitrary reward signal.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In Reinforcement Learning, when reward function is not differentiable, a policy gradient algorithm is used to update the weights of a network&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Policy gradient methods are one broad family of RL methods which are also often model-free. Alternatively, value-based methods are broadly the other choice (e.g. Q-learning) or can be used in combination with policy gradient methods (e.g. Actor-Critic). All of these can be model-free, and often are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you want to apply a model-based RL method, then you have no need to find the reward function in an explicit form based on any parameters. Even if you did want to use a model-based approach, the reward function does not need to be differentiable.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was wondering if I can use another value, for example silhouette score (in a different scenario) as the reward signal?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes this would likely be viable, and could be made to work similarly to the paper, provided there is some meaningful connection between the parameters you manipulate and the end result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RL can be made to work as a generic way of optimising numeric measure influenced due to a vector of parameters. That includes indirect optimisation of neural networks by non-differentiable metrics such as accuracy. There is nothing special about the choice of accuracy here, the metric just needs to be related to the parameters being learned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There &lt;em&gt;is&lt;/em&gt; a catch. Indirect learning using RL can produce very noisy training data, due to e.g. covariance between all the parameters, and it can take many samples to get a clear gradient signal through all the noise and make meaningful updates. It can be inefficient compared to other optimisation methods if they are available - however, the ability to extract a gradient for the parameters even when rules are complex and non-differentiable is a nice feature of RL.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-10T15:45:51.617" LastActivityDate="2020-06-10T15:45:51.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21784" PostTypeId="1" CreationDate="2020-06-10T13:38:53.023" Score="2" ViewCount="128" Body="&lt;p&gt;I am following the OpenAI's spinning up tutorial &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 3: Intro to Policy Optimization&lt;/a&gt;. It is mentioned there that the reward-to-go reduces the variance of the policy gradient. While I understand the intuition behind it, I struggle to find a proof in the literature.&lt;/p&gt;&#xA;" OwnerUserId="37770" LastEditorUserId="2444" LastEditDate="2020-10-10T15:51:28.350" LastActivityDate="2020-10-10T15:51:28.350" Title="What is the proof that &quot;reward-to-go&quot; reduces variance of policy gradient?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proofs&gt;&lt;reward-to-go&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21785" PostTypeId="1" CreationDate="2020-06-10T14:51:50.973" Score="0" ViewCount="24" Body="&lt;p&gt;I want to analyze queries and their differences between two different languages English and Spanish in this case. I'm aware of topic modeling. I'm in search of any corpus available or any algorithm to compare the differences in them. &#xA;P.S - I am trying to find fairness in queries for two languages. English and Spanish in this case. If I model the topics of the queries for 2 languages I could analyze the differences in the two languages. I thought Topic modeling would be a good direction for the problem &lt;/p&gt;&#xA;" OwnerUserId="10118" LastActivityDate="2020-06-10T14:51:50.973" Title="How to find difference between topics in two languages using NLP" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;latent-variable&gt;&lt;nltk&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21786" PostTypeId="2" ParentId="15371" CreationDate="2020-06-10T15:31:36.630" Score="0" Body="&lt;p&gt;Mask RCNN can be a very heavy function for a simple class classification. It is designed to handle multiple object in a single image. So I would suggest you could use much simpler models like VGGnet or Resnet which are backbones of the MaskRCNN. &#xA;The biggest hurdle you might face is the dataset. If you are trying to capture even small difference between knives and classify them you will need atleast 2000 images per knife type. which might be difficult to get. You may have to do a data augmentation.&lt;/p&gt;&#xA;" OwnerUserId="37773" LastActivityDate="2020-06-10T15:31:36.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21787" PostTypeId="1" CreationDate="2020-06-10T15:48:25.673" Score="1" ViewCount="30" Body="&lt;p&gt;I want to predict how open is the mouth given a face image. It's a regression problem (0= mouth not open, 1=mouth completely open). And something between 0 and 1 is also allowed. ConvNet works fine for one person. But when I train with many people with hope that it will generalize to an unseen person, the model suffers from not knowing the limit of a person's mouth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if a new person uses the model to predict, the model doesn't have a clue whether this person has completely opened the mouth or not. Because it's hard to know how much a person can open the mouth from one image. People's mouth openness capability is not the same. Some guys cannot open their mouth that much, but some guys can open the mouth like they can swallow an apple. The only way you can know how much a person can open the mouth is to look at multiple images of their mouth movements, especially when they open the mouth completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how to make the model know the limit of a person's mouth by using the info from past images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way for me to use a few unlabeled images of a new person in order to help the model calibrate its prediction? How do I do it?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should help the model know the min/max of the person's mouth and also knows the intermediate values between 0 and 1. If you run the model continuously on a webcam, I expect the prediction to be smooth (not noisy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to encode many images into an embedding that can be used as a calibration vector. The vector will be fed into the model along with the person's image. But I am not sure how to do it.&#xA;Any suggestions/tutorials would be welcomed.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2020-06-10T15:48:25.673" Title="How to calibrate model's prediction given past images?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21788" PostTypeId="1" CreationDate="2020-06-10T16:23:32.423" Score="0" ViewCount="24" Body="&lt;p&gt;How can I create a combination of a string that is very close to another combination?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a string like &quot;abc..xyz&quot;. Now I want a combination of 13 letters. According to:  &lt;span class=&quot;math-container&quot;&gt;$ {N}\choose{k}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;${26}\choose{13} $&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$10400600$&lt;/span&gt;&#xA;In order to actually generate these combinations one can use Knuth's Algorithm as it is described in &quot;The art of computer programming, Vol 4&quot;. Rearrangments are considered as equal combination and therefore aren't count: &lt;span class=&quot;math-container&quot;&gt;$&quot;abc&quot;=&quot;bca&quot;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I want to etimate the best combination toward a cost function using an optimization algorithm like genetic lgorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what these sort of algorithms do is taking a combination and comparing it to a cost function. If the solution is better they take it as the current solution and look for a better solution by alternating the solution a little bit. Alternating the solution is sometime sort of arbitrary.  How can a nearby combination be generate without repeating combinations like &lt;span class=&quot;math-container&quot;&gt;$&quot;abc&quot;=&quot;bca&quot;$&lt;/span&gt;? Is there a special graph based method or state space method?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastActivityDate="2020-06-10T16:23:32.423" Title="How can I combinate a string (no permutation allowed) without creating all combination" Tags="&lt;combinatorics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21789" PostTypeId="1" CreationDate="2020-06-10T18:17:49.183" Score="1" ViewCount="253" Body="&lt;p&gt;If my understanding of an LSTM is correct then the output from each LSTM unit is the hidden state from that layer. For the final layer if I wanted to predict e.g. a scalar real number, would I want to add a dense layer with 1 neuron or is it recommended to have a final LSTM layer where the output has just one hidden unit (i.e. the output dimension of the final hidden state is 1)? If we didn't add the dense layer, then the output from the hidden layer I believe would be between (-1,1), if you use the traditional activations in the LSTM unit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies if I've used wrong terminology, there seems to be some inconsistency with LSTM's when going between literature and definition in TensorFlow etc. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-10T18:17:49.183" Title="Do you have to add a dense layer onto the final layer of an LSTM?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21790" PostTypeId="2" ParentId="18068" CreationDate="2020-06-10T21:50:11.137" Score="2" Body="&lt;p&gt;If you used your five &lt;span class=&quot;math-container&quot;&gt;$X_{test}$&lt;/span&gt; sets multiple times (to measure the average AUC) to decide on the best set of hyperparameters (i.e. &lt;em&gt;optimizer, learning rate, batch size, dropout, activation&lt;/em&gt;) then &lt;strong&gt;yes, you successfully conducted hyper-parameter optimization&lt;/strong&gt;. However, the AUC you received for the best set of hyperparameters found (by manual tuning) &lt;strong&gt;is not representative of the real performance of your model&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because the fact of using a &lt;em&gt;test&lt;/em&gt; set to tune the parameters of your model degenerates it back to a &lt;em&gt;&quot;training&quot;&lt;/em&gt; set, because the data is not being used to measure the performance of the model but to improve it instead (although on a different level of abstraction, i.e. not to directly influence the parameters of the model, such as neural network weights), making the resulting AUC an overly optimistic biased estimator of the real AUC (that it could have resulted in for an unseen test dataset).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why, if you care both about hyperparameter optimisation &lt;strong&gt;and&lt;/strong&gt; being able to measure the &quot;real&quot; performance of your model, you need to split your dataset into three &quot;buckets&quot;: training set &lt;span class=&quot;math-container&quot;&gt;$X_{train}$&lt;/span&gt;, validation st &lt;span class=&quot;math-container&quot;&gt;$X_{val}$&lt;/span&gt; and test set &lt;span class=&quot;math-container&quot;&gt;$X_{test}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$X_{test}$&lt;/span&gt; should only be used once (after you've trained and tuned the model), and assuming it has enough samples and the samples are representative of the unseen data, you should get a good estimate of your model performance. &lt;span class=&quot;math-container&quot;&gt;$X_{val}$&lt;/span&gt;, on the other hand, is your &lt;em&gt;validation&lt;/em&gt; set which you can reuse as many times as you want to find the optimal set of hyperparameters that result in the highest performance (i.e. AUC).&lt;/p&gt;&#xA;" OwnerUserId="22835" LastActivityDate="2020-06-10T21:50:11.137" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21791" PostTypeId="2" ParentId="18111" CreationDate="2020-06-11T02:18:21.050" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How does policy evaluation work for continuous state space model-free approaches? ... Let's say you use a DQN to find another policy, how does model-free policy evaluation work then? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Policy evaluation is the process of determining state-value &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; or action-value &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}(s, a)$&lt;/span&gt; functions for the current policy. In the context of continuous state and action spaces without a model of the environment, &lt;strong&gt;policy evaluation must incorporate the agent's past experience instead of the model dynamics&lt;/strong&gt; and will generally use a function approximator such as a neural network to estimate the action-values. Many popular approaches apply online updates to the function approximator; e.g., DQN combines Temporal-Difference targets and gradient descent to change the weights of the neural network and the resultant action-value estimates. Since &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;we gradually change the weights of the neural network at each gradient descent step,&lt;/li&gt;&#xA;&lt;li&gt;the estimated action-values are solely dependent on the weights of the neural network,&lt;/li&gt;&#xA;&lt;li&gt;the current policy is solely dependent on the estimated action-values (e.g. DQN takes the action with largest action-value),&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then &lt;strong&gt;policy evaluation&lt;/strong&gt; (updating the estimated action-value function to better match the true action-value function under the current policy) &lt;strong&gt;and policy improvement&lt;/strong&gt; (greedily changing the current policy based on the new estimated action-value function) &lt;strong&gt;occur simultaneously at each gradient descent step&lt;/strong&gt;. In DQN, a gradient descent step occurs at each time step.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am thinking of Monte Carlo simulation, but that would require many many episodes. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After changing the action-value function, we may get a new policy. We assume that the &lt;strong&gt;action-value function of the old policy is similar to that of the new policy&lt;/strong&gt; (although it is not guaranteed) because the change in weights of the neural network was &lt;em&gt;small&lt;/em&gt;. Therefore, we use the estimated action-value function of the old policy as an initial estimate of the action-value function of the new policy. Specifically, &lt;strong&gt;we use the same neural network with the same weights as the initial approximation&lt;/strong&gt;. This is computationally convenient, as it &lt;strong&gt;prevents the need to start the next policy evaluation update from scratch&lt;/strong&gt; (e.g. with a Monte Carlo simulation over a painfully large number of episodes). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Theoretically, a model-based approach for the discrete state and action space can be computed via dynamic programming and solving the Bellman equation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This same technique of using the estimated action-values of the old policy as the initial estimates for the action-values of the new policy is employed by some Dynamic Programming methods such as &lt;em&gt;value iteration&lt;/em&gt;, albeit with known dynamics. &lt;em&gt;Generalized Policy Iteration&lt;/em&gt; (GPI) is the notion of letting policy evaluation and policy iteration interact on whatever granularity deemed necessary for the problem at hand. A consequence of adopting the GPI paradigm is the choice to halt policy evaluation before convergence of the action-value function. Many deep reinforcement learning algorithms take this to an extreme and perform policy evaluation and policy improvement simultaneously during a single gradient descent step. For reference, Chapter 4 of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; provides a brief summary of these ideas.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-11T02:18:21.050" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21792" PostTypeId="1" CreationDate="2020-06-11T03:31:32.027" Score="0" ViewCount="50" Body="&lt;p&gt;For traditional neural networks, I know that we can't constraint the output to be strict integers. My question is what technique does GANs use to produce integer outputs, that can be then converted to RGB colored pictures?&lt;/p&gt;&#xA;" OwnerUserId="37178" LastEditorUserId="2444" LastEditDate="2020-06-11T10:23:57.277" LastActivityDate="2020-06-11T10:23:57.277" Title="How GAN generator produce integer RGB colored picture?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21793" PostTypeId="1" AcceptedAnswerId="21796" CreationDate="2020-06-11T04:05:56.787" Score="1" ViewCount="80" Body="&lt;p&gt;I had read some blogs (like &lt;a href=&quot;https://medium.com/@narrativesci/natural-language-processing-vs-natural-language-generation-1b2d18dd0b67&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://towardsdatascience.com/nlp-vs-nlu-vs-nlg-know-what-you-are-trying-to-achieve-nlp-engine-part-1-1487a2c8b696&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; or &lt;a href=&quot;https://www.houseofbots.com/news-detail/11507-1-which-parameters-separates-nlp-nlu-and-nlg&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;) about what the difference between all three of them is. I am trying to build an open domain conversation agent using natural language AI. That agent can do casual conversation, like a friend. So, for that, I want to know what is the importance of NLP, NLG, and NLU, so that I can learn that part first.&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="2444" LastEditDate="2020-06-11T10:31:45.810" LastActivityDate="2020-06-11T10:31:45.810" Title="When to use NLP, NLG and NLU in conversation agents?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;chat-bots&gt;&lt;natural-language-understanding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21796" PostTypeId="2" ParentId="21793" CreationDate="2020-06-11T08:30:02.033" Score="1" Body="&lt;p&gt;They're all important. NLP is an umbrella term that includes the other two; NLG is only concerned with generating language, ie transforming some internal data structure into human language. NLU is about processing information contained in language, and putting it into relation with a knowledge base etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know anything about any of these fields, then I suggest your aim is far too optimistic. I work for a company that provides a conversational AI platform for businesses to develop their own agents, and it is a complex area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to have a quick go and pick up some experience, I suggest you start with &lt;a href=&quot;https://en.wikipedia.org/wiki/ELIZA&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELIZA&lt;/a&gt;. Even though this is 'ancient', many modern chatbots still work on the same principles. There are many implementations in a number of programming languages, so you should be able to find one that suits you and you can tinker around with it.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-06-11T08:30:02.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21797" PostTypeId="1" CreationDate="2020-06-11T10:16:11.540" Score="2" ViewCount="543" Body="&lt;p&gt;Throughout my readings, I have seen many authors using interchangeably the two terms to refer to the same thing. However, we all know about Google's first quotation of &quot;knowledge graph&quot; to refer to their &lt;em&gt;new&lt;/em&gt; way of making use of their knowledge base. Afterwards, other companies are claiming using knowledge graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the technical differences between the two? Concrete examples will be very useful to understand better the nuances.&lt;/p&gt;&#xA;" OwnerUserId="32217" LastEditorUserId="2444" LastEditDate="2020-06-11T10:27:12.657" LastActivityDate="2020-11-08T15:02:35.247" Title="What are the differences between a knowledge base and a knowledge graph?" Tags="&lt;comparison&gt;&lt;knowledge-graph&gt;&lt;knowledge-base&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21798" PostTypeId="1" CreationDate="2020-06-11T10:24:58.307" Score="0" ViewCount="42" Body="&lt;p&gt;I want to make a drone which can follow static and dynamic waypoints. I am a total beginner in the drone field so I can't figure out that should I use Reinforcement Learning or any other learning methods for the drone to make it follow both static and dynamic waypoints. If RL is the best choice for the task, then how would I go about training the model and upload it to the flight controller. And if RL is not required, then what should I use in order to achieve this task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know how should I begin with this task&lt;/p&gt;&#xA;" OwnerUserId="32071" LastActivityDate="2020-06-11T10:24:58.307" Title="Can Reinforcement Learning be used for UAV waypoint control?" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;learning-algorithms&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21800" PostTypeId="1" CreationDate="2020-06-11T11:24:55.413" Score="0" ViewCount="25" Body="&lt;p&gt;I have a dataset with patient information with discrete labels (labels are stages of a particular disease) which needs to be predicted (Basically a classification problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data set looks something like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Patient#|Visit#|Other medical features related to patient and visit|label (disease stage)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I am interested in using patient's past visit data inorder to predict the current disease stage. But, the problem is that all the patients don't have equal number of visits. So, I can't just append the past visit information to predict the future visit label like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;concat(Patient #n 1st visit (X = all input features)|label of this visit| Patient #n 2nd visit (X = all input features)) and then try to predict the label for 2nd visit using previous visit information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above problem, the number of visit =1, but I have a varying number of visit for each patient. How can I tackle this problem?&lt;/p&gt;&#xA;" OwnerUserId="4126" LastActivityDate="2020-06-11T11:24:55.413" Title="Input with variable length Classification problem" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;supervised-learning&gt;&lt;time-series&gt;&lt;feature-engineering&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21801" PostTypeId="2" ParentId="7088" CreationDate="2020-06-11T12:25:13.987" Score="3" Body="&lt;p&gt;***Take my answer as a side note to that given by cantordust:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one can verify that an activation function perform well in some cases, that good behavior often extrapolates to other problems.&#xA;Thus, by testing activation functions on a few different problems, one can often infer how well (or badly) it will perform on most problems.&#xA;The following video shows how different activation functions perform in different problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=Hb3vIYUQ_I8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=Hb3vIYUQ_I8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can verify that an activation function usually perform well in all cases, or the other way around: it does it poorly in all cases.&#xA;As cantordust says, I would recommend always starting with leaky ReLU: it is simple, efficient, and generally produces nice results in a wide variety of problems.&#xA;It also evades the dying ReLU problem, and does not suffer from the vanishing gradient problem.&#xA;The only thing to keep in mind is the exploding gradient problem if the neural network is too deep, or if it is a recurrent neural network, which are essentially the same concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The video shows that other activation functions worth trying (in addition to leaky ReLU) are Gaussian, Sinusoid, or Tanh.&lt;/p&gt;&#xA;" OwnerUserId="37800" LastEditorUserId="37800" LastEditDate="2020-06-11T17:41:09.913" LastActivityDate="2020-06-11T17:41:09.913" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21802" PostTypeId="2" ParentId="21797" CreationDate="2020-06-11T12:27:06.580" Score="1" Body="&lt;p&gt;Based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_base&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia&lt;/a&gt;, a knowledge base (KB) is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a technology used to store complex structured and unstructured information used by a computer system. The initial use of the term was in connection with expert systems which were the first knowledge-based systems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As there are different representation model for a KB, we can find different terminology in different domains. For example, in some AI articles, it's called &lt;strong&gt;ontology&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_Graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledeg graph&lt;/a&gt; (KG) is another object model to KB realization which is introduced by Google for its search engine (as you have mentioned). Hence, KG is a specification of KB. You can find more information in the paper &lt;a href=&quot;https://arxiv.org/pdf/2003.02320.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge Graphs&lt;/a&gt;, such as more history about the KG or a formal definition of that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;knowledge graph is a graph of data intended to accumulate and convey knowledge of the real world, whose nodes represent entities of interest and whose edges represent relations between these entities.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Moreover, you can find some articles about contextual KG (CKG) in the paper &lt;a href=&quot;https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1026&amp;amp;context=pacis2017&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Contextual Embeddings for Knowledge Graph Completion&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1805.12393.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;KG&lt;span class=&quot;math-container&quot;&gt;$^2$&lt;/span&gt;: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2020-06-11T13:59:24.293" LastActivityDate="2020-06-11T13:59:24.293" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21803" PostTypeId="1" CreationDate="2020-06-11T16:35:16.173" Score="1" ViewCount="64" Body="&lt;p&gt;When I was learning about neural networks, I saw that a complex neural network can understand the MNIST dataset and a simple convolution network can also understand the same. So I would like to know if we can achieve a CNN's functionality with just using a simple neural network without the convolution layer and if we can then how to convert a CNN into an ANN. &lt;/p&gt;&#xA;" OwnerUserId="17024" LastEditorUserId="2444" LastEditDate="2020-06-11T16:41:31.077" LastActivityDate="2020-06-12T07:59:24.990" Title="Can we achieve what a CNN can do with just a normal neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21804" PostTypeId="1" CreationDate="2020-06-11T16:41:53.130" Score="0" ViewCount="31" Body="&lt;p&gt;I am trying to build anomaly detection with low false positives .Dataset that i am using is a patient health sensor data. A number of parameters from the patient's sensors are collected hourly and I have roughly 7k parameters which can act as features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Issues i am facing are the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: For each patient, I gather hourly data for 8 days of data.Hence I get ~190 rows of these parameter. However I have 7000 parameters being collected for each hour as and hence my dataset for each patient is 190 rows* 7000columns. I feel the columns/attributes are really large compared to rows as attributes(columns) are 7000 vs 190 rows. Are there any recommendation on how to deal with such a big feature set. Should I do dimensionality reduction first and then pass to an isolation forest algorithm? Are there any better ways of dealing with such large attributes (columns) vs low rows?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: Any recommendations of any additional algorithms that I could try that might give lowest falses? I am currently using PCA, isolation forests, one class svm as the data currently represents only normal behavior.&lt;/p&gt;&#xA;" OwnerUserId="35979" LastActivityDate="2020-06-11T16:41:53.130" Title="How to deal with large number of features for Anomaly Detection" Tags="&lt;machine-learning&gt;&lt;anomaly-detection&gt;&lt;dimensionality-reduction&gt;&lt;principal-component-analysis&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21805" PostTypeId="2" ParentId="21803" CreationDate="2020-06-11T16:47:03.603" Score="1" Body="&lt;p&gt;The convolutional aspect of a CNN comes purely from the connections between layers. Instead of a fully-connected network, which can be difficult to train and tends to overfit more, the convolutional network utilizes hierarchical patterns in the data to limit the number of connections - a local edge detection feature in an image analysis network, for example, only needs input from a small number of local pixels, not the entire image. But in principle, you could assign weights to a fully-connected network to perfectly mimic a convolutional one - you just set the weights of the unneeded connections to zero. Because a general ANN has all the connections present in a CNN plus more, it can do anything a CNN can do plus more, although the training can be more difficult.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2020-06-11T16:47:03.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21807" PostTypeId="1" CreationDate="2020-06-11T19:19:05.600" Score="1" ViewCount="35" Body="&lt;p&gt;In section 6.1 of the paper &lt;a href=&quot;https://www.researchgate.net/publication/249900701_Neural_Networks_in_Economics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks in Economics&lt;/a&gt;, the authors say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;this leads to the problem, that no risk can be formulated which shall be minimized by a Neural Network learning algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, why can't neural networks be applied to &lt;em&gt;preference learning&lt;/em&gt; problems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See sections 6.0 of &lt;a href=&quot;https://www.researchgate.net/publication/249900701_Neural_Networks_in_Economics&quot; rel=&quot;nofollow noreferrer&quot;&gt;the same paper&lt;/a&gt; for a definition of preference learning.&lt;/p&gt;&#xA;" OwnerUserId="37807" LastEditorUserId="2444" LastEditDate="2020-06-11T23:43:32.757" LastActivityDate="2020-06-11T23:43:32.757" Title="Why can't neural networks be applied to preference learning problems?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21808" PostTypeId="2" ParentId="20831" CreationDate="2020-06-11T21:52:28.267" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt; state, &quot;The reward signal is your way of communicating to the robot [agent] &lt;em&gt;what&lt;/em&gt; you want it to achieve, not &lt;em&gt;how&lt;/em&gt; you want it achieved.&quot; Since you stated that the goal is to reach the finish line first, then a reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for winning, &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for losing, and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; at all other time steps seems to fit that narrative. If a draw is identical to a loss, then it should provide reward &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;; otherwise, a reward of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; seems reasonable. These rewards provide &lt;em&gt;model interpretability&lt;/em&gt;: an expected return of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (estimated with a state-value or action-value) at a certain state under the current policy would signify a &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; chance of winning. Also, keeping the rewards in absolute value at most 1 can aid in training speed and prevent divergence, but it often isn't necessary for deep reinforcement learning problems. You most certainly can add other rewards based on partial progress towards the goal, but as it seems you found out, they may lead to incorrect results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, I would focus on the &lt;strong&gt;training process instead of a finely-tuned reward signal&lt;/strong&gt;. Since there is a known goal state in the racing game (the finish line), I suggest training the RL agent by first initializing all racer agents only a few steps away from the goal state at the beginning of each episode. These episodes are shorter and therefore should provide a more dense reward signal. When your RL agent has learned a winning policy (e.g. wins more often than not), then initialize the agents slightly further from the goal state at the beginning of each episode. Also, &lt;strong&gt;continue to use and train the same neural network&lt;/strong&gt;. Since the neural network presumably knows a winning policy at states near the goal state, then by initializing the agents only a few states further back, the RL agent is given an warm start and only needs to learn a policy for a few more states. The policy encoded by the neural network essentially contains a refined reward signal for the states close to the goal state since it is based on a winning policy; this helps prevent the sparsity problem caused by only supplying a reward at episode completion. You can repeat this process by initializing the agents slightly further from the goal state once the RL agent has learned a winning policy while continuing to use and train the same neural network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your access to the environment internals, you may need other analogous approaches. For example, you could initialize the agents at the original starting line (i.e. not partway down the map) and then see which agent makes it &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; units down the map first to determine the winner. Once a winning policy is learned by the RL agent, then gradually increase &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; until &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; matches the distance from the starting line to the finish line. Since it seems like you have &lt;code&gt;distance traveled&lt;/code&gt; and &lt;code&gt;distance to the opponent&lt;/code&gt; as features, you may instead try this method if you are unable to initialize the agents wherever you want on the map and instead can only initialize them on the starting line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A notable benefit of the overall approach is that you can more quickly debug your algorithm on the easier environments (i.e. ones with shorter episode lengths) to be confident that the learning process is correct and focus your efforts elsewhere (e.g. the training process, including the reward signal).&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-11T21:52:28.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21809" PostTypeId="1" CreationDate="2020-06-11T22:02:31.657" Score="0" ViewCount="78" Body="&lt;p&gt;I've written a Double DQN-based stock trading bot using mainly time series stock data. The internal network of the Double DQN is a LSTM which handles the time series data. An Experience Replay buffer is also used. The objective function is cumulative stock return over the test period. My epsilon used for exploration is 0.1 (which I think is already very high).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My trading bot has a very simple action space, trade or no-trade. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- When it decides to trade, it sends a signal to buy and own a stock for a day. I'd get a positive return if stock price has gone up from today to tomorrow; equally would get a negative return if stock price has gone down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- When it decides to not trade, I own no stock and the daily return is 0 because there is no trading.&#xA;Strangely, my algorithm gives a daily 'no trade' signal most of the time when I run the algo through a number of different test periods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very often, after giving a 'no trade' signal for many days, the algo would finally give a 'trade' signal but the next day reverse back to giving 'no trade' right away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why am I getting this phenomenon? Most importantly, what can I do to make the algo not stuck in giving out 'no trade' signal most of the time?&lt;/p&gt;&#xA;" OwnerUserId="37615" LastActivityDate="2021-03-02T21:47:07.317" Title="My Double DQN with Experience Replay produces a no-action decision most of the time. Why?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21810" PostTypeId="1" AcceptedAnswerId="21824" CreationDate="2020-06-12T01:35:49.520" Score="2" ViewCount="3997" Body="&lt;p&gt;I was surveying some &lt;a href=&quot;https://arxiv.org/pdf/1512.08086.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;literature&lt;/a&gt; related to Fully Convolutional Networks and came across the following phrase, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A fully convolutional network is achieved by replacing the parameter-rich fully connected layers in standard CNN architectures by convolutional layers with &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; kernels.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have two questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is meant by &lt;em&gt;parameter-rich&lt;/em&gt;? Is it called parameter rich because the fully connected layers pass on parameters without any kind of &quot;spatial&quot; reduction? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, how do &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; kernels work? Doesn't &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; kernel simply mean that one is sliding a single pixel over the image? I am confused about this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="30910" LastEditorUserId="2444" LastEditDate="2020-06-12T15:47:18.907" LastActivityDate="2020-06-18T12:35:16.737" Title="What is a fully convolution network?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-segmentation&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21811" PostTypeId="1" CreationDate="2020-06-12T03:26:21.317" Score="1" ViewCount="166" Body="&lt;p&gt;I was wondering if the &lt;code&gt;BART&lt;/code&gt; or &lt;code&gt;T5&lt;/code&gt; models can do the task of generating sentences in English. Most of the models I have mentioned are trained to translate from English to German or French. Is is it possible that I can use the output of &lt;code&gt;BERT&lt;/code&gt; as an input to my Decoder? My theory is that when I already have the trained &lt;code&gt;Embeddings&lt;/code&gt;, I do not need to train the Encoder part. I can just add the outputs of sentences to the decoder to generate the sentences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In place of finding the loss value from the translated version, Can I compute loss on the reply of a sentence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me towards a tutorial where I can use the &lt;code&gt;BERT&lt;/code&gt; output as for the decoder part. I have a data of conversation with me. I want to build a &lt;code&gt;Chatbot&lt;/code&gt; from that data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already implemented &lt;code&gt;LSTM&lt;/code&gt; based &lt;code&gt;Sequence2sequence&lt;/code&gt; model but it is not providing me satisfactory answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some research, 2 such models are there as &lt;code&gt;T5&lt;/code&gt; and &lt;code&gt;BART&lt;/code&gt; which are based on the same idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible, can someone tell me how can I use &lt;code&gt;BART&lt;/code&gt; or &lt;code&gt;T5&lt;/code&gt; to make a conversational bot?&lt;/p&gt;&#xA;" OwnerUserId="36062" LastActivityDate="2021-03-09T12:05:50.500" Title="Can we use a pre trained Encoder (BERT, XLM ) with a Decoder (GPT, Transformer-XL) to build a Chatbot instead of Language Translation?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;deep-neural-networks&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21813" PostTypeId="2" ParentId="21811" CreationDate="2020-06-12T06:29:52.240" Score="0" Body="&lt;p&gt;=&gt; I don't have more ideas about &lt;strong&gt;BART&lt;/strong&gt; and &lt;strong&gt;T5&lt;/strong&gt; Right now. but I had created a chatbot Based on &lt;strong&gt;GPT-2&lt;/strong&gt; Model-based on &lt;a href=&quot;https://github.com/microsoft/DialoGPT&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft DialoGPT&lt;/a&gt;. Which is fine-tuned on millions of parameters of Reddit. you can fine-tune on your own data using DialoGPT. I had not found a public decoding method. So, We tried to generate diverse responses using simple &lt;strong&gt;nucleus sampling&lt;/strong&gt; at each time step. So, We used &lt;strong&gt;greedy nucleus sampling multiple times&lt;/strong&gt; in parallel to generate multiple candidates responses. We generated 30 such responses for each turn and also used the last 3 turn conversation history as extra context for generator. So, We can generate responses related to the current Context of Conversation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We used Code reference from &lt;a href=&quot;https://colab.research.google.com/drive/1-_KjlAV3J1IVDw_9KogjKDCzgFY7Jp7E#scrollTo=RZd4iXX5SqKC&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. you need to implement greedy nucleus sampling multiple on this code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=&gt;Now To Choose the best-matched response from response generated I created a Reranker.I used two sub-component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=&gt;Component one Counts &lt;strong&gt;Cosine similarity&lt;/strong&gt; between Response and Query using &lt;a href=&quot;https://github.com/PolyAI-LDN/polyai-models/blob/master/encoder_client.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;sentence Encoder&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=&gt;In Component Two we &lt;strong&gt;cross-entropy error&lt;/strong&gt; for generating original query from the response.For that I used reverse-generator of &lt;a href=&quot;https://github.com/microsoft/DialoGPT&quot; rel=&quot;nofollow noreferrer&quot;&gt;DialogueGPT&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We Combined Score of Both Component as &lt;code&gt;Score=norm(comp1Score)+norm(1-comp2Loss)&lt;/code&gt;. Using this score you can find best response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Tutorial My Favourite &lt;a href=&quot;https://huggingface.co/transformers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hugging Face&lt;/a&gt; they have &lt;strong&gt;BERT,GPT, GPT-2, XLNET,etc...&lt;/strong&gt;&#xA;and Second Favourite &lt;a href=&quot;https://github.com/facebookresearch/ParlAI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facebook ParlAI&lt;/a&gt;, &#xA;try &lt;a href=&quot;https://parl.ai/projects/recipes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blender Model&lt;/a&gt; also.&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="32861" LastEditDate="2020-06-12T11:09:10.447" LastActivityDate="2020-06-12T11:09:10.447" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21814" PostTypeId="2" ParentId="21803" CreationDate="2020-06-12T07:59:24.990" Score="0" Body="&lt;p&gt;It can be argued that CNN will outperform a fully connected network if they have the same structure (number of neurons).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normal neural networks can probably learn to detect things like CNNs, but the task would be a lot more computationally expensive. In a CNN, all neurons in a feature maps share the same parameters, so if CNN learns to recognize a pattern in one location, it can detect the pattern in any other location. Furthermore, CNNs take into account the fact that pixels that are closer in proximity with each other are more heavily related than the pixels that are further apart, this information is lost in a Normal neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read More &lt;a href=&quot;https://towardsdatascience.com/cnn-vs-fully-connected-network-for-image-processing-8c5b95d4e42f&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="34170" LastActivityDate="2020-06-12T07:59:24.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21815" PostTypeId="5" CreationDate="2020-06-12T08:14:37.467" Score="0" Body="&lt;p&gt;Deterministic annealing is used in order to avoid local minima of the given cost function which traps traditional techniques. A set of temperature parametrized Gibbs probability density functions relate each data point to each cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method is established in a probabilistic framework through basic information-theoretic techniques such as maximum entropy and random coding. It arises naturally in the context of statistical mechanics by the emulation of a physical process whereby a solid is slowly cooled and at zero temperature assumes its minimum energy configuration&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:28:05.000" LastActivityDate="2020-06-16T19:28:05.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21816" PostTypeId="4" CreationDate="2020-06-12T08:14:37.467" Score="0" Body="For questions related to deterministic annealing (DA), the  method used for the solution of several nonconvex problems offers the ability to avoid shallow local minima of a given cost surface and the ability to minimize the cost function even when there are many local minima" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-16T19:33:42.230" LastActivityDate="2020-06-16T19:33:42.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21817" PostTypeId="1" CreationDate="2020-06-12T10:03:32.587" Score="1" ViewCount="41" Body="&lt;p&gt;I was reading AlexNet paper and the authors quoted &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the kernels on one GPU were &quot;largely color agnostic,&quot; whereas the kernels on the other GPU were largely &quot;color-specific.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The upper GPU takes operates on filters on the top and lower GPU deals with the lower half. But what is the reason for each of them learning a different set of features, i.e. the top half of kernels learning the edges mostly and the bottom kernels learning color variation? Is there any reason behind it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r6LAb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r6LAb.png&quot; alt=&quot;alexnet-kernels&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37821" LastEditorUserId="2444" LastEditDate="2020-06-13T14:18:24.180" LastActivityDate="2020-06-13T14:18:24.180" Title="What is the reason for different learned features in upper and lower half in AlexNet?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;alexnet&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21819" PostTypeId="5" CreationDate="2020-06-12T11:00:10.147" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T11:00:10.147" LastActivityDate="2020-06-12T11:00:10.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21820" PostTypeId="4" CreationDate="2020-06-12T11:00:10.147" Score="0" Body="For questions related to the convolutional neural network denoted as AlexNet, designed by Alex Krizhevsky (hence the name AlexNet), and published with Ilya Sutskever and Krizhevsky's doctoral advisor Geoffrey Hinton." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T11:00:10.147" LastActivityDate="2020-06-12T11:00:10.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21822" PostTypeId="1" CreationDate="2020-06-12T12:57:49.927" Score="0" ViewCount="94" Body="&lt;p&gt;I am currently working on implementing &lt;a href=&quot;https://arxiv.org/pdf/1509.01549.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Giraffe&lt;/a&gt; chess algorithm. Following this paper, I designed a neural network similar to the one proposed by the author which I trained using TD-Leaf(lambda). The procedure that I followed is described in the extract of the paper shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OjEv1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OjEv1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding code that I wrote (using PyTorch) is the following (you can find it &lt;a href=&quot;https://github.com/denguir/deep-chess-bot/blob/master/td_leaf_training.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def self_play(batch, net, device, n_moves):&#xA;    '''Self play on n_moves of a given game'''&#xA;    boards = [chess.Board(b) for b in batch['board']]&#xA;    boards = list(map(push_random_move, boards))&#xA;    giraffe_move = partial(find_best_move, max_depth=0, &#xA;                                evaluator=partial(giraffe_evaluation, net=net, device=device))&#xA;    scores = []&#xA;    for _ in range(n_moves):&#xA;        moves_, scores_ = zip(*map(giraffe_move, boards))&#xA;        scores.append(torch.stack(scores_))&#xA;        boards = [push_move(board, move) for (board, move) in zip(boards, moves_)]&#xA;    #scores = list(zip(*scores))&#xA;    scores = torch.stack(scores)&#xA;    return torch.t(torch.squeeze(scores))&#xA;&#xA;&#xA;def td_loss(scores, td_lamdba):&#xA;    L, N = scores.size()&#xA;    err = torch.zeros((L, N))&#xA;    for t in range(N-1):&#xA;        discount = 1&#xA;        err_t = torch.zeros(L)&#xA;        for j in range(t, N-1):&#xA;            dj = scores[:, j+1] - scores[:, j]&#xA;            discount *= td_lamdba&#xA;            err_t += discount * dj&#xA;        err[:, t] = err_t&#xA;    # we include a minus sign because torch computes a gradient descent&#xA;    # by default, but we want to impose a custom update rule for the weights&#xA;    loss = torch.mean(torch.sum(-scores * err.detach(), dim=1))&#xA;    return loss&#xA;&#xA;&#xA;def self_learn(batch, net, device, n_moves, optimizer):&#xA;    scores = self_play(batch, net, device, n_moves)&#xA;    loss = td_loss(scores, 0.7)&#xA;    optimizer.zero_grad()&#xA;    loss.backward()&#xA;    optimizer.step()&#xA;    return loss.item()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The neural net is designed to receive an encoded chess board as input and outputs a score between -1 (Black wins) and 1 (White wins).&#xA;The problem is that, when I train the net using TD-Leaf, it learns to evaluate the board to 1 at every time step of the game, even when the game just started (which should be evaluated at 0 ideally). However, when I think back about it, this result is not surprising since most of the moves that happened during the self learning process are not end game moves (so no true score is available). This implies that even if the neural net always outputs 1, it will be wrong very rarely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know what do you think about this since I am running short of ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: Note that before using TD-Leaf, I pretrained my neural net on Stockfish engine to not start on a random weights initialization.&lt;/p&gt;&#xA;" OwnerUserId="37652" LastActivityDate="2020-06-12T12:57:49.927" Title="TD-Leaf struggles at learning chess" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reinforcement-learning&gt;&lt;chess&gt;&lt;td-lambda&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21823" PostTypeId="1" CreationDate="2020-06-12T13:17:14.340" Score="0" ViewCount="43" Body="&lt;p&gt;I have a dataset where for each country there is a name (string) and a multivariate time series (all integers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to use an Encoder-Decoder LSTM to forecast the next time steps in the time series for each country. Here is a crude diagram of what I'm trying to implement.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8KwgN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8KwgN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took the idea of using an Embedding layer from &lt;a href=&quot;https://stackoverflow.com/q/52627739/9506872&quot;&gt;here&lt;/a&gt;.&#xA;This way the model can learn how the time series for each country behaves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how to train it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am using a sliding window method based on the book &quot;Long Short-Term Memory Networks With Python&quot;  in chapter 3.4 is says the following (emphasis added):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sequence prediction problems &lt;strong&gt;must&lt;/strong&gt; be re-framed as supervised learning problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the data is divided into weeks and the model uses one week to predict one day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d1, d2, d3, d4, d5, d6, d7 -&amp;gt; d8&#xA;d2, d3, d4, d5, d6, d7, d8 -&amp;gt; d9&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is &lt;strong&gt;not&lt;/strong&gt; a seq2seq model as it only outputs one time step. I make predictions in a loop in order to get 21 days worth of forecasts. Ideally I would like to make the model produce X number of days worth of predictions e.g. 28 days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading some research papers I learnt about &lt;em&gt;Direct&lt;/em&gt; forecasting methods. Where the authors mention it is possible to directly predict a time step based on previous data without having to use the sliding window method.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is also known as moving window scheme in forecasting, and requires substantial data augmentation. Such method is not necessary in an RNN thanks to its temporal nature.&lt;br&gt;&#xA;  &lt;sub&gt;From &lt;a href=&quot;https://arxiv.org/pdf/1711.11053.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wen, R. et al 2018&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A brief summary of the ﬁndings is that the direct strategy generally has the smallest bias, and becomes superior for longer time series.&lt;br&gt;&#xA;  &lt;sub&gt;From &lt;a href=&quot;https://ieeexplore.ieee.org/document/7064712&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taieb, S. B. et al 2016&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From these two  papers it seems that using the &lt;em&gt;Direct&lt;/em&gt; method is better than my current Sliding Window method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My problem is that I don't know how to train this model without using the sliding window method.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An idea I  had was to use the last 28 days as the test data then use the last 28 days of the training data as the sample output. Then the model would use one big sample to produce 28 forecasts but this seems to cause a few problems.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The model will only be trained on one sample.&lt;/li&gt;&#xA;&lt;li&gt;Input data in the test will overlap with the train data.&lt;/li&gt;&#xA;&lt;li&gt;Most values in the training data are small compared to the target data.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7kjcy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7kjcy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This leads to a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does sequence prediction require supervised learning or is it possible to use unsupervised learning?&lt;/li&gt;&#xA;&lt;li&gt;Will the Embedding layer really help the model learn how each country behaves?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;How to train an Encoder-Decoder on direct forecasting, i.e. no sliding window?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32265" LastActivityDate="2020-06-12T13:17:14.340" Title="How to train an Encoder-Decoder LSTM for sequence to sequence prediction?" Tags="&lt;long-short-term-memory&gt;&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;&lt;time-series&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21824" PostTypeId="2" ParentId="21810" CreationDate="2020-06-12T13:25:16.283" Score="4" Body="&lt;h1&gt;Fully convolution networks&lt;/h1&gt;&#xA;&lt;p&gt;A &lt;a href=&quot;https://arxiv.org/pdf/1411.4038.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;fully convolution network (FCN)&lt;/strong&gt;&lt;/a&gt; is a neural network that only performs convolution (and subsampling or upsampling) operations. Equivalently, an FCN is a CNN without fully connected layers.&lt;/p&gt;&#xA;&lt;h2&gt;Convolution neural networks&lt;/h2&gt;&#xA;&lt;p&gt;The typical &lt;em&gt;convolution neural network (CNN)&lt;/em&gt; is not fully convolutional because it often contains &lt;em&gt;fully connected layers&lt;/em&gt; too (which do not perform the convolution operation), which are &lt;em&gt;parameter-rich&lt;/em&gt;, in the sense that they have many parameters (compared to their equivalent convolution layers), although &lt;a href=&quot;https://arxiv.org/pdf/1411.4038.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the fully connected layers can also be viewed as convolutions with kernels that cover the entire input regions&lt;/a&gt;, which is the main idea behind converting a CNN to an FCN. See &lt;a href=&quot;https://www.youtube.com/watch?v=XdsmlBGOK-k&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; by Andrew Ng that explains how to convert a fully connected layer to a convolutional layer.&lt;/p&gt;&#xA;&lt;h2&gt;An example of an FCN&lt;/h2&gt;&#xA;&lt;p&gt;An example of a fully convolutional network is the &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;U-net&lt;/strong&gt;&lt;/a&gt; (called in this way because of its U shape, which you can see from the illustration below), which is a famous network that is used for &lt;strong&gt;semantic segmentation&lt;/strong&gt;, i.e. classify pixels of an image so that pixels that belong to the same class (e.g. a person) are associated with the same label (i.e. person), aka pixel-wise (or dense) classification.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/938D2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/938D2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Semantic segmentation&lt;/h3&gt;&#xA;&lt;p&gt;So, in semantic segmentation, you want to associate a label with each pixel (or small patch of pixels) of the input image. Here's a more suggestive illustration of a neural network that performs semantic segmentation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CX1w0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CX1w0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Instance segmentation&lt;/h3&gt;&#xA;&lt;p&gt;There's also &lt;strong&gt;instance segmentation&lt;/strong&gt;, where you also want to differentiate different instances of the same class (e.g. you want to distinguish two people in the same image by labeling them differently). An example of a neural network that is used for instance segmentation is &lt;a href=&quot;https://arxiv.org/pdf/1703.06870.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;mask R-CNN&lt;/a&gt;. The blog post &lt;a href=&quot;https://glassboxmedicine.com/2020/01/21/segmentation-u-net-mask-r-cnn-and-medical-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Segmentation: U-Net, Mask R-CNN, and Medical Applications&lt;/a&gt; (2020) by Rachel Draelos describes these two problems and networks very well.&lt;/p&gt;&#xA;&lt;p&gt;Here's an example of an image where instances of the same class (i.e. person) have been labeled differently (orange and blue).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xi6po.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xi6po.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Both semantic and instance segmentations are &lt;em&gt;dense&lt;/em&gt; classification tasks (specifically, they fall into the category of &lt;a href=&quot;https://en.wikipedia.org/wiki/Image_segmentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;image segmentation&lt;/a&gt;), that is, you want to classify each pixel or many small patches of pixels of an image.&lt;/p&gt;&#xA;&lt;h2&gt;&lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolutions&lt;/h2&gt;&#xA;&lt;p&gt;In the U-net diagram above, you can see that there are only convolutions, copy and crop, max-pooling, and upsampling operations. There are no fully connected layers.&lt;/p&gt;&#xA;&lt;p&gt;So, how do we associate a label to each pixel (or a small patch of pixels) of the input? How do we perform the classification of each pixel (or patch) without a final fully connected layer?&lt;/p&gt;&#xA;&lt;p&gt;That's where the &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolution and upsampling operations are useful!&lt;/p&gt;&#xA;&lt;p&gt;In the case of the U-net diagram above (specifically, the top-right part of the diagram, which is illustrated below for clarity), &lt;em&gt;two&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$1 \times 1 \times 64$&lt;/span&gt; kernels are applied to the input volume (not the images!) to produce &lt;em&gt;two&lt;/em&gt; feature maps of size &lt;span class=&quot;math-container&quot;&gt;$388 \times 388$&lt;/span&gt;. They used two &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; kernels because there were two classes in their experiments (cell and not-cell). &lt;a href=&quot;https://glassboxmedicine.com/2020/01/21/segmentation-u-net-mask-r-cnn-and-medical-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The mentioned blog post&lt;/a&gt; also gives you the intuition behind this, so you should read it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rSiWZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rSiWZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you have tried to analyze the U-net diagram carefully, you will notice that the output maps have different spatial (height and weight) dimensions than the input images, which have dimensions &lt;span class=&quot;math-container&quot;&gt;$572 \times 572 \times 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That's fine because our general goal is to perform &lt;strong&gt;dense&lt;/strong&gt; classification (i.e. classify patches of the image, where the patches can contain only one pixel), although I said that we would have performed pixel-wise classification, so maybe you were expecting the outputs to have the same exact spatial dimensions of the inputs. However, note that, in practice, you could also have the output maps to have the same spatial dimension as the inputs: you would just need to perform a different upsampling (deconvolution) operation.&lt;/p&gt;&#xA;&lt;h3&gt;How &lt;span class=&quot;math-container&quot;&gt;$1\times 1$&lt;/span&gt; convolutions work?&lt;/h3&gt;&#xA;&lt;p&gt;A &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolution is just the typical 2d convolution but with a &lt;span class=&quot;math-container&quot;&gt;$1\times1$&lt;/span&gt; kernel.&lt;/p&gt;&#xA;&lt;p&gt;As you probably already know (and if you didn't know this, now you know it), if you have a &lt;span class=&quot;math-container&quot;&gt;$g \times g$&lt;/span&gt; kernel that is applied to an input of size &lt;span class=&quot;math-container&quot;&gt;$h \times w \times d$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the depth of the input volume (which, for example, in the case of grayscale images, it is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;), the kernel actually has the shape &lt;span class=&quot;math-container&quot;&gt;$g \times g \times d$&lt;/span&gt;, i.e. the third dimension of the kernel is equal to the third dimension of the input that it is applied to. This is always the case, except for 3d convolutions, but we are now talking about the typical 2d convolutions! See &lt;a href=&quot;https://ai.stackexchange.com/a/18669/2444&quot;&gt;this answer&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&lt;p&gt;So, in the case we want to apply a &lt;span class=&quot;math-container&quot;&gt;$1\times 1$&lt;/span&gt; convolution to an input of shape &lt;span class=&quot;math-container&quot;&gt;$388 \times 388 \times 64$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt; is the depth of the input, then the actual &lt;span class=&quot;math-container&quot;&gt;$1\times 1$&lt;/span&gt; kernels that we will need to use have shape  &lt;span class=&quot;math-container&quot;&gt;$1\times 1 \times 64$&lt;/span&gt; (as I said above for the U-net). The way you reduce the depth of the input with &lt;span class=&quot;math-container&quot;&gt;$1\times 1$&lt;/span&gt; is determined by the number of &lt;span class=&quot;math-container&quot;&gt;$1\times 1$&lt;/span&gt; kernels that you want to use. This is exactly the same thing as for any 2d convolution operation with different kernels (e.g. &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In the case of the U-net, the spatial dimensions of the input are reduced in the same way that the spatial dimensions of any input to a CNN are reduced (i.e. 2d convolution followed by downsampling operations). The main difference (apart from not using fully connected layers) between the U-net and other CNNs is that the U-net performs upsampling operations, so it can be viewed as an encoder (left part) followed by a decoder (right part).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-18T12:35:16.737" LastActivityDate="2020-06-18T12:35:16.737" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21825" PostTypeId="2" ParentId="21699" CreationDate="2020-06-12T15:15:15.963" Score="1" Body="&lt;p&gt;Neil Slater's answer is very nice, but I have a couple more suggestions: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use entropy regularization. Basically, you modify your loss function to penalize low policy entropy (so less loss for more entropy) which should prevent your policy from becoming &quot;too deterministic&quot; too early. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can also try maximum-entropy methods, like SAC, which employ a different strategy for promoting policy entropy. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-12T15:15:15.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21826" PostTypeId="5" CreationDate="2020-06-12T15:49:08.367" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T15:49:08.367" LastActivityDate="2020-06-12T15:49:08.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21827" PostTypeId="4" CreationDate="2020-06-12T15:49:08.367" Score="0" Body="For questions related to fully convolutional networks (FCNs), which is formally described in the paper &quot;Fully Convolutional Networks for Semantic Segmentation&quot; (2015) by Jonathan Long et al. An example of an FCN is the U-net (introduced in the paper &quot;U-Net: Convolutional Networks for Biomedical Image Segmentation&quot; by Olaf Ronneberger et al.)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T15:49:08.367" LastActivityDate="2020-06-12T15:49:08.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21828" PostTypeId="2" ParentId="13821" CreationDate="2020-06-12T15:56:14.047" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Are fully connected layers necessary in a CNN?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.  In fact, you can simulate a fully connected layer with convolutions. A convolutional neural network (CNN) that does not have fully connected layers is called a &lt;strong&gt;fully convolutional network (FCN)&lt;/strong&gt;. See &lt;a href=&quot;https://ai.stackexchange.com/a/21824/2444&quot;&gt;this answer&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of an FCN is the &lt;a href=&quot;https://arxiv.org/abs/1505.04597&quot; rel=&quot;nofollow noreferrer&quot;&gt;u-net&lt;/a&gt;, which does &lt;em&gt;not&lt;/em&gt; use any fully connected layers, but only convolution, downsampling (i.e. pooling), upsampling (deconvolution), and copy and crop operations. Nevertheless, u-net is used to classify pixels (more precisely, semantic segmentation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, you can use CNNs only for the purpose of feature extraction, and then feed these extracted features in another classifier (e.g. an SVM). In fact, transfer learning is based on the idea that CNNs extract &lt;em&gt;reusable&lt;/em&gt; features.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T16:03:06.430" LastActivityDate="2020-06-12T16:03:06.430" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21829" PostTypeId="1" AcceptedAnswerId="21830" CreationDate="2020-06-12T16:04:51.897" Score="0" ViewCount="134" Body="&lt;p&gt;I have two questions &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When we use our network to approximate our Q values, is the Q target a single value?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;During backpropagation, when the weights are updated, does it automatically update the Q values, shouldn’t the state be passed in the network again to update it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-12T16:15:15.903" LastActivityDate="2020-06-12T16:39:16.187" Title="When we use a neural network to approximate the Q values, is the Q target a single value?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21830" PostTypeId="2" ParentId="21829" CreationDate="2020-06-12T16:30:54.423" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;When we use our network to approximate our Q values,is the Q target a single value?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, the target Q value is a single value if you are just updating a single training example. The loss function of a vanilla DQN for a single experience tuple &lt;span class=&quot;math-container&quot;&gt;$(s_t,a_t,r_t,s_{t+1})$&lt;/span&gt; is calculated as &lt;span class=&quot;math-container&quot;&gt;$$L(\theta) = [r_t + \gamma \,max\,Q_{a_{t+1}}(s_{t+1},a_{t+1};\theta) - Q(s_t,a_t;\theta)]^2$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$r_t + \gamma \,max\,Q(s_{t+1},a_{t+1};\theta)$&lt;/span&gt; is the target Q value. However, when using mini-batch gradient descent, you would have to compute multiple target Q values equivalent to the batch size&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;During backpropagation, when the weights are updated, does it automatically update the Q values, shouldn’t the state be passed in the network again to update it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;During backpropagation of the loss function, the weights &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are automatically updated. You do not need to pass in the state again. Because in the first place, you would have computed &lt;span class=&quot;math-container&quot;&gt;$Q(s_t,a_t;\theta)$&lt;/span&gt; by passing in the state as input to the neural network. That is how backpropagation works for Deep Q networks.&lt;/p&gt;&#xA;&lt;p&gt;Training for the DQN is as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Collect experience tuples of &lt;span class=&quot;math-container&quot;&gt;$(s_t,a_t,r_t,s_{t+1})$&lt;/span&gt; and store them in a replay buffer.&lt;/li&gt;&#xA;&lt;li&gt;Sample mini-batch of experiences from the replay buffer.&lt;/li&gt;&#xA;&lt;li&gt;From these sampled batch of experiences, compute &lt;span class=&quot;math-container&quot;&gt;$Q(s_{t+1}.a_{t+1};\theta)$&lt;/span&gt; by passing &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; into the network and take the Q value with the maximum values&lt;/li&gt;&#xA;&lt;li&gt;Compute &lt;span class=&quot;math-container&quot;&gt;$Q(s_t,a_t;\theta)$&lt;/span&gt; by passing &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; into the network.&lt;/li&gt;&#xA;&lt;li&gt;Compute the Loss for this experience and propagate the loss back to the network, hence updating the weights.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Also, since the weights have changed after backpropagation, the Q values for the same state would also be updated if you pass the same state in to the network again. &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Check out this paper as it explains how Deep Q Network works&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-12T16:39:16.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21831" PostTypeId="2" ParentId="21829" CreationDate="2020-06-12T16:33:52.587" Score="0" Body="&lt;p&gt;What the network represents can be a little confusing - in tabular Q-learning you have a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function that you pass a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; into and receive a scalar value. In the &lt;a href=&quot;https://www.nature.com/articles/nature14236&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Level Control&lt;/a&gt; paper where the DQN gained its popularity, the network is a little different to the tabular function. You pass into the network your current state and it outputs values for every action in your action space. You then choose the action which has the highest value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way we then train it is as follows - first we get our scalar value which corresponds to the target &lt;span class=&quot;math-container&quot;&gt;$r + \gamma max_{a'} Q(s',a')$&lt;/span&gt;; I will denote this as &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; for short. As our network outputs a vector of dimension &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}|$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is our action space, we get our prediction for the current state which is a vector of the dimension I just mentioned. We will now assume our action space has two possible actions and say that for state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; our network outputs the vector &lt;span class=&quot;math-container&quot;&gt;$[\hat{y}_1,\hat{y}_2]$&lt;/span&gt;. Assume that when we calculated &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; earlier we took the first action, so we only want to update our prediction of &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_1$&lt;/span&gt;. To do this, if we were working in e.g. PyTorch I would put into the loss function, which is mean squared error, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;input = &lt;span class=&quot;math-container&quot;&gt;$[\hat{y}_1,\hat{y}_2$&lt;/span&gt;], target = &lt;span class=&quot;math-container&quot;&gt;$[y, \hat{y}_2]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;so essentially all we change is the position in the vector corresponding to the action we took with the Q-target we observed. The weights will thus be updated accordingly to move more into the direction of producing this output for state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to explicitly answer your first question, the Q-target is a scalar but the target we pass into the network is a vector. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-12T16:33:52.587" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21832" PostTypeId="2" ParentId="9934" CreationDate="2020-06-12T17:17:31.120" Score="2" Body="&lt;p&gt;It should not be much more difficult to predict a rotated rectangle compared to a bounding box.&lt;/p&gt;&#xA;&lt;p&gt;A bounding box can be parameterized with 4 floats: &lt;span class=&quot;math-container&quot;&gt;$x_c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_c$&lt;/span&gt;, width, height.&lt;/p&gt;&#xA;&lt;p&gt;A rotated rectangle can be parameterized with 5 floats: &lt;span class=&quot;math-container&quot;&gt;$x_c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_c$&lt;/span&gt;, width, height, angle.&lt;/p&gt;&#xA;&lt;p&gt;However, to avoid the wrap-around issue with predicting the angle with one value (0° is same as 360°), &lt;strong&gt;it should be better to predict sine and cosine instead&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It is actually useful to predict rotated rectangles for text detection (each text field is a rotated rectangle).  Indeed, in the wild, text can be in any orientation, and it is important to predict precisely rotated rectangles for OCR to work well. It is especially true for long text boxes near 45° (an axis-aligned bounding box around this would be useless because too big).&lt;/p&gt;&#xA;&lt;p&gt;Here are 2 links I found about this topic:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/ultralytics/yolov3/issues/345&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rotated Bounding boxes training&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1703.01086&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arbitrary-Oriented Scene Text Detection via Rotation Proposals&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37830" LastEditorUserId="2444" LastEditDate="2021-01-28T23:41:35.277" LastActivityDate="2021-01-28T23:41:35.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21835" PostTypeId="2" ParentId="21594" CreationDate="2020-06-12T18:18:20.587" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;There is an epsilon for discovery, set at 100% at the start and it decreases by 10% until it reaches 1%.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After looking through your code on the linked GitHub repository, I think that the annealing of the &lt;code&gt;epsilon&lt;/code&gt; parameter is a major issue.  As clarified in the above comments, the &lt;code&gt;act()&lt;/code&gt; method is called once per episode time step to determine the agent's choice of action. Within this method, it seems that &lt;code&gt;epsilon&lt;/code&gt; is decreased extremely rapidly. The code states that &lt;code&gt;epsilon = Math.max(1, epsilon - 10)&lt;/code&gt;, which means epsilon is decreased to 1 after 10 time steps. Also as clarified in the comments, &lt;code&gt;epsilon&lt;/code&gt; is never reset to a larger number. Therefore, it seems that &lt;code&gt;epsilon&lt;/code&gt; will be reduced from 100 to 1 after 10 time steps after (most likely) a single episode, which in my opinion is much too quick and &lt;strong&gt;will stifle exploration&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a first guess, I suggest annealing &lt;code&gt;epsilon&lt;/code&gt; more slowly from 100 to 1 after roughly one million time steps. If you want to anneal the parameter linearly, then each of the first million time steps could reduce epsilon by  &lt;code&gt;(starting epsilon - ending epsilon) / annealing time steps&lt;/code&gt;, where &lt;code&gt;starting epsilon&lt;/code&gt; = 100, &lt;code&gt;ending epsilon&lt;/code&gt; = 1, and &lt;code&gt;annealing time steps&lt;/code&gt; = 1000000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other issues may crop up after making the above change, but I think this is a good starting point. This seems like a very fun project, and it would be fun if you kept us posted about the results!&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-12T18:18:20.587" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21838" PostTypeId="1" CreationDate="2020-06-12T20:25:24.613" Score="0" ViewCount="30" Body="&lt;p&gt;I am currently studying this model speech generation known as &lt;a href=&quot;http://%20https://arxiv.org/pdf/1609.03499.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;WaveNet model by Google.&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/pdf/1609.03499.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1609.03499.pdf&lt;/a&gt; using the linked original paper and &lt;a href=&quot;https://github.com/Rayhane-mamah/Tacotron-2/blob/master/wavenet_vocoder/models/wavenet.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this implementation.&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the model very confusing in the input it takes and the output it generates, and some of the layer dimensions didn't seem to match based on what I understood from the wavenet paper, or am I misunderstanding something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What is the input to the WaveNet, isn't this a mel-spectrum input and not just 1  floating point value for raw audio? E.g. the input kernel layer shows as shaped 1x1x128. Isn't the input to the &lt;code&gt;input_convolution&lt;/code&gt; layer the mel-spectrum frames, which are 80 float values * 10,000 max_decoder_steps, so the in_channels for this conv1d layer should be 80 instead of 1?   &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;inference/input_convolution/kernel:0 (float32_ref 1x1x128) [128, bytes: 512]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;2) Is there reason for upsampling stride values to be [11, 25], like are the specific numbers 11 and 25 special or relevant in affecting other shapes/dimensions? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inference/ConvTranspose1D_layer_0/kernel:0 (float32_ref 1x11x80x80) [70400, bytes: 281600]&#xA;inference/ConvTranspose1D_layer_1/kernel:0 (float32_ref 1x25x80x80) [160000, bytes: 640000]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) Why is the input-channels in  residual_block_causal_conv 128 and residual_block_cin_conv 80? What exactly is their inputs? (e.g. is it mel-spectrum or just a raw floating point value?) Is the wavenet-vocoder generating just 1 float value per 1 input mel-spectrum frame of 80 floats?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inference/ResidualConv1DGLU_0/residual_block_causal_conv_ResidualConv1DGLU_0/kernel:0 (float32_ref 3x128x256) [98304, bytes: 393216]&#xA;inference/ResidualConv1DGLU_0/residual_block_cin_conv_ResidualConv1DGLU_0/kernel:0 (float32_ref 1x80x256) [20480, bytes: 81920]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was able to print the whole Wavenet network using the print(tf.trainable_variables()), but the model still seems very confusing.&lt;/p&gt;&#xA;" OwnerUserId="33580" LastActivityDate="2020-10-31T01:34:30.230" Title="Studying the speech-generation model and have question about the confusing nature of model input and outputs" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;speech-recognition&gt;&lt;speech-synthesis&gt;&lt;transpose-convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21839" PostTypeId="1" AcceptedAnswerId="26678" CreationDate="2020-06-12T20:34:03.733" Score="2" ViewCount="88" Body="&lt;p&gt;I was trying to solve an XOR problem, and the dataset seems like the one in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qVEHH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qVEHH.png&quot; alt=&quot;dataset&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I plotted the tree and got this result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iFopL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iFopL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand, the tree should have depth 2 and four leaves. The first comparison is annoying, because it is close to the right x border (0.887). I've tried other parameterizations, but the same result persists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.tree import DecisionTreeClassifier    &#xA;&#xA;clf = DecisionTreeClassifier(criterion='gini')&#xA;clf = clf.fit(X, y)&#xA;&#xA;fn=['V1','V2']&#xA;&#xA;fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (3,3), dpi=300)&#xA;&#xA;tree.plot_tree(clf, feature_names = fn, class_names=['1', '2'], filled = True);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would be grateful if anyone can help me to clarify this issue. &lt;/p&gt;&#xA;" OwnerUserId="37836" LastEditorUserId="37836" LastEditDate="2020-06-13T12:01:18.577" LastActivityDate="2021-03-06T06:10:18.033" Title="Why isn't my decision tree classifier able to solve the XOR problem properly?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;decision-tree&gt;&lt;scikit-learn&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21840" PostTypeId="1" CreationDate="2020-06-12T23:26:16.537" Score="0" ViewCount="84" Body="&lt;p&gt;Given equation 7.3 of &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=166&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto's book&lt;/a&gt; for convergence of TD(n): &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\max_s|\mathbb{E}_\pi[G_{t:t+n}|S_t = s] - v_\pi(s)| \leqslant \gamma^n \max_s|V_{t+n-1}(s) - v_\pi(s)|$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{PROBLEM 1}$&lt;/span&gt; : Why is this error &lt;span class=&quot;math-container&quot;&gt;$|\mathbb{E}_\pi[G_{t:t+n}|S_t = s] - v_\pi(s)|$&lt;/span&gt; compared with the error &lt;span class=&quot;math-container&quot;&gt;$|V_{t+n-1}(s) - v_\pi(s)|$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There can be two other logical comparisons for the convergence of algorithm(&lt;span class=&quot;math-container&quot;&gt;$TD(n)$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) If we compare and say that &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}(s)$&lt;/span&gt; is more close to &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; than &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-2}(s)$&lt;/span&gt; i.e we compare &lt;span class=&quot;math-container&quot;&gt;$|V_{t+n-1}(s) - v_\pi(s)|$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$|V_{t+n-2}(s) - v_\pi(s)|$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) We can also compare &lt;span class=&quot;math-container&quot;&gt;$|\mathbb{E}_\pi[G_{t:t+n}|S_t = s] - v_\pi(s)|$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$|\mathbb{E}_\pi[V_{t+n-1}(S_t)|S_t = s] - v_\pi(s)|$&lt;/span&gt; to show that  &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[G_{t:t+n}|S_t = s]$&lt;/span&gt; is better than &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[V_{t+n-1}(S_t)|S_t = s]$&lt;/span&gt; hence moving &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}(S_t)$&lt;/span&gt; towards &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}$&lt;/span&gt;(as done in eq 7.2) can lead to convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{PROBLEM 2}$&lt;/span&gt; : Are the above 2 methods of comparison for testing convergence correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equations for reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eq 7.1: &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n} = R_{t+1} + \gamma R_{t+2} +......+\gamma^{n-1}R_{t+n} + \gamma^{n}V_{t+n-1}(S_{t+n})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eq 7.2: &lt;span class=&quot;math-container&quot;&gt;$V_{t+n}(S_t) = V_{t+n-1}(S_t) + \alpha [G_{t:t+n} - V_{t+n-1}(S_t)]$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="37611" LastEditDate="2020-06-13T12:14:59.833" LastActivityDate="2020-09-10T19:41:01.603" Title="Problem in understanding equation given for convergence of TD(n) algorithm" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;&lt;value-iteration&gt;&lt;expectation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21841" PostTypeId="5" CreationDate="2020-06-12T23:55:40.713" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T23:55:40.713" LastActivityDate="2020-06-12T23:55:40.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21842" PostTypeId="4" CreationDate="2020-06-12T23:55:40.713" Score="0" Body="For questions related to the U-net, a neural network proposed in &quot;U-Net: Convolutional Networks for Biomedical Image Segmentation&quot; (2015) by Olaf Ronneberger et al. for semantic segmentation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-12T23:55:40.713" LastActivityDate="2020-06-12T23:55:40.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21844" PostTypeId="1" CreationDate="2020-06-13T06:29:21.597" Score="1" ViewCount="50" Body="&lt;p&gt;In the paragraph given between eq 7.12 and 7.13 in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=172&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto's book&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$G_{t:h} = R_{t+1} + G_{t+1:h} , t &amp;lt; h &amp;lt; T$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$G_{h:h} = V_{h-1}(S_h)$&lt;/span&gt;. (Recall that this return is used at time h, previously denoted t + n.) Now consider the effect of following a behavior policy &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; that is not the same as the target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. All of the resulting experience, including the ﬁrst reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; and the next state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; must be weighted by the importance sampling ratio for time &lt;span class=&quot;math-container&quot;&gt;$t, \rho_t = \frac{\pi(A_t|S_t)}{b(A_t|S_t)}$&lt;/span&gt; . One might be tempted to simply weight the righthand side of the above equation, but one can do better. Suppose the action at time t would never be selected by &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, so that &lt;span class=&quot;math-container&quot;&gt;$\rho_t$&lt;/span&gt; is zero. Then a simple weighting would result in the n-step return being zero, which could result in high variance when it was used as a target.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why does the n-step return being zero results in high variance?&lt;/p&gt;&#xA;&lt;p&gt;Also, why is the experience weighted by &lt;span class=&quot;math-container&quot;&gt;$\rho_t$&lt;/span&gt;, it should be weighted by &lt;span class=&quot;math-container&quot;&gt;$\rho_{t:t+h}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-13T13:44:17.677" Title="Why does the n-step return being zero result in high variance in off policy n-step TD?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;&lt;importance-sampling&gt;&lt;return&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21845" PostTypeId="1" CreationDate="2020-06-13T06:43:27.443" Score="1" ViewCount="44" Body="&lt;p&gt;Seung et.al recently published &lt;a href=&quot;http://arxiv.org/abs/2005.12126&quot; rel=&quot;nofollow noreferrer&quot;&gt;GameGAN&lt;/a&gt; paper, GameGAN learned and stored the whole Pacman game and was able to reproduce it without a game engine. The uniqueness of GameGAN is that it had added memory to its discriminator/generator which helped it to store the game states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iOb2m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iOb2m.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat{\boldsymbol{\theta}} = \mathop{\mathrm{argmax}} _ {\boldsymbol{\theta}} P(X \mid \boldsymbol{\theta})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will adding memory which can store prior information makes GameGAN a Bayesian learning system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can GameGAN or similar neural network with memory can be considered as a bayesian learning system. If yes, then which of these two equations(or something other) correctly explains this system (considering prior as memory)?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathop{\mathrm{argmax}} \frac{P(X \mid \boldsymbol{\theta})P(\boldsymbol{\theta})}{P(X)}$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathop{\mathrm{argmax}} \frac{P(X_t \mid \boldsymbol{X^{t+1}})P(\boldsymbol{X^{t+1}})}{P(X_t)}$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;PS: I understand GAN's are unsupervised learning systems, but we can assume discriminator and generator models separately trying to find weights that maximize their individual likelihood function.&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="2444" LastEditDate="2020-06-13T13:50:18.960" LastActivityDate="2020-06-13T13:50:18.960" Title="Will adding memory to a supervised learning system makes it into a Bayesian learning system?" Tags="&lt;generative-adversarial-networks&gt;&lt;bayesian-deep-learning&gt;&lt;bayesian-statistics&gt;&lt;game-gan&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21846" PostTypeId="2" ParentId="21594" CreationDate="2020-06-13T07:50:50.290" Score="2" Body="&lt;p&gt;Q learning on its own isn't enough to learn a winning strategy for a game like 2048. 2048 requires predictive thinking for possible outcomes and good positional awareness. Performance of the agent is heavily dependent on the reward function. The approach to give the reward proportional to the obtained points after every move is naive since it might sacrifice better positional play for short term rewards. The way it's posed it seems like the agent will try to maximize its performance for the last 20 moves or so. That could lead an agent to the positional situation which leads to a loss in couple of moves. Possibly better strategy would be to give positive reward when the agent actually completes the 2048 tiles and negative reward if it loses. Such parse rewards would add difficulty to training since it would require sophisticated exploration strategies which &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy certainly isn't. The stochastic nature of the game would pose difficulty to the agent as well. Similar positions might lead to different outcomes because in some cases the new tile would spawn in a bad position for the continuation while in other cases it might be beneficial. The suggested approach would be to definitely include Monte Carlo tree search approach along with some RL algorithm which was successfully applied to agents like AlphaZero and AlphaGo. MCTS would sample moves ahead and agent would get better representation of how good certain actions are.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-06-13T07:50:50.290" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21847" PostTypeId="1" CreationDate="2020-06-13T09:42:16.577" Score="2" ViewCount="25" Body="&lt;p&gt;I've been going through &quot;&lt;a href=&quot;https://arxiv.org/pdf/1708.03058.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms&lt;/a&gt;&quot; by Wang et al. and am unable to understand how the update equations for the hyperparameters (section 4.3, equation set (23)) were derived. I'd deeply appreciate it if anyone could provide a full or partial derivation of the updates. Any general suggestions regarding how to proceed with the derivation would also be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ICTR Graphical Model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nMVY6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nMVY6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variables are sampled as below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{p}_m|\lambda \sim \text{Dirichlet}(\lambda)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma^2_n|\alpha,\beta \sim \text{Inverse-Gamma}(\alpha,\beta)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{q}_n |\mu_{\mathbb{q}}, \Sigma_{\mathbb{q}}, \sigma_n^2 \sim \mathcal{N}(\mu_{\mathbb{q}}, \sigma_n^2\Sigma_{\mathbb{q}})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{\Phi}_k |\eta \sim \text{Dirichlet}(\eta)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$z_{m,t} | \mathbb{p}_m \sim \text{Multinomial}(\mathbb{p}_m)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_{m,t} | \mathbb{\Phi}_k \sim \text{Multinomial}(\mathbb{\Phi}_k) $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{m,t} \sim \mathcal{N}(\mathbb{p}_m^T\mathbb{q}_n, \sigma_n^2)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the update equations are below &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G5jcW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G5jcW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37848" LastEditorUserId="37849" LastEditDate="2020-06-13T10:29:39.293" LastActivityDate="2020-06-13T10:29:39.293" Title="Deriving hyperparameter updates in Online Interactive Collaborative Filtering" Tags="&lt;papers&gt;&lt;hyperparameter-optimization&gt;&lt;online-learning&gt;&lt;bayesian-network&gt;&lt;graphical-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21849" PostTypeId="1" AcceptedAnswerId="23500" CreationDate="2020-06-13T11:03:04.857" Score="2" ViewCount="324" Body="&lt;p&gt;In per-decison importance sampling given in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=136&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto's book&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Eq 5.12 &lt;span class=&quot;math-container&quot;&gt;$\rho_{t:T-1}R_{t+k} = \frac{\pi(A_{t}|S_{t})}{b(A_{t}|S_{t})}\frac{\pi(A_{t+1}|S_{t+1})}{b(A_{t+1}|S_{t+1})}\frac{\pi(A_{t+2}|S_{t+2})}{b(A_{t+2}|S_{t+2})}......\frac{\pi(A_{T-1}|S_{T-1})}{b(A_{T-1}|S_{T-1})}R_{t+k}$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Eq 5.13 &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}\left[\frac{\pi(A_{k}|S_{k})}{b(A_{k}|S_{k})}\right] = \displaystyle\sum_ab(a|S_k)\frac{\pi(A_{k}|S_{k})}{b(A_{k}|S_{k})} = \displaystyle\sum_a\pi(a|S_k) = 1$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Eq.5.14 &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[\rho_{t:T-1}R_{t+k}] = \mathbb{E}[\rho_{t:t+k-1}R_{t+k}]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As full derivation is not given, how do we arrive at Eq 5.14 from 5.12?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;From what i understand :&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;1) &lt;span class=&quot;math-container&quot;&gt;$R_{t+k}$&lt;/span&gt; is only dependent on action taken at &lt;span class=&quot;math-container&quot;&gt;$t+k-1$&lt;/span&gt; given state at that time i.e. only dependent on &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi(A_{t+k-1}|S_{t+k-1})}{b(A_{t+k-1}|S_{t+k-1})}$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi(A_{k}|S_{k})}{b(A_{k}|S_{k})}$&lt;/span&gt; is independent of &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi(A_{k+1}|S_{k+1})}{b(A_{k+1}|S_{k+1})}$&lt;/span&gt; , so &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}\left[\frac{\pi(A_{k}|S_{k})}{b(A_{k}|S_{k})}\frac{\pi(A_{k+1}|S_{k+1})}{b(A_{k+1}|S_{k+1})}\right] = \mathbb{E}\left[\frac{\pi(A_{k}|S_{k})}{b(A_{k}|S_{k})}\right]\mathbb{E}\left[\frac{\pi(A_{k+1}|S_{k+1})}{b(A_{k+1}|S_{k+1})}\right], \forall \, k\in [t,T-2]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[\rho_{t:T-1}R_{t+k}]= \mathbb{E}\left[\frac{\pi(A_{t}|S_{t})}{b(A_{t}|S_{t})}\frac{\pi(A_{t+1}|S_{t+1})}{b(A_{t+1}|S_{t+1})}\frac{\pi(A_{t+2}|S_{t+2})}{b(A_{t+2}|S_{t+2})}......\frac{\pi(A_{T-1}|S_{T-1})}{b(A_{T-1}|S_{T-1})}R_{t+k}\right] \\= \mathbb{E}\left[\frac{\pi(A_{t}|S_{t})}{b(A_{t}|S_{t})}\frac{\pi(A_{t+1}|S_{t+1})}{b(A_{t+1}|S_{t+1})}\frac{\pi(A_{t+2}|S_{t+2})}{b(A_{t+2}|S_{t+2})}....\frac{\pi(A_{t+k-2}|S_{t+k-2})}{b(A_{t+k-2}|S_{t+k-2})}\frac{\pi(A_{t+k}|S_{t+k})}{b(A_{t+k}|S_{t+k})}......\frac{\pi(A_{T-1}|S_{T-1})}{b(A_{T-1}|S_{T-1})}\right]\mathbb{E}\left[\frac{\pi(A_{t+k-1}|S_{t+k-1})}{b(A_{t+k-1}|S_{t+k-1})}R_{t+k}\right] \\= \mathbb{E}\left[\frac{\pi(A_{t}|S_{t})}{b(A_{t}|S_{t})}\right]\mathbb{E}\left[\frac{\pi(A_{t+1}|S_{t+1})}{b(A_{t+1}|S_{t+1})}\right]\mathbb{E}\left[\frac{\pi(A_{t+2}|S_{t+2})}{b(A_{t+2}|S_{t+2})}\right]....\mathbb{E}\left[\frac{\pi(A_{t+k-2}|S_{t+k-2})}{b(A_{t+k-2}|S_{t+k-2})}\right]\mathbb{E}\left[\frac{\pi(A_{t+k}|S_{t+k})}{b(A_{t+k}|S_{t+k})}\right]......\mathbb{E}\left[\frac{\pi(A_{T-1}|S_{T-1})}{b(A_{T-1}|S_{T-1})}\right]\mathbb{E}\left[\frac{\pi(A_{t+k-1}|S_{t+k-1})}{b(A_{t+k-1}|S_{t+k-1})}R_{t+k}\right] \\= \mathbb{E}[\frac{\pi_{t+k-1}}{b_{t+k-1}}R_{t+k}]\\=\mathbb{E}[\rho_{t+k-1}R_{t+k}]$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;which is not equal to eq 5.14. What's the mistake in the above calculations? Are 1 and 2 correct?&lt;/h3&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="37611" LastEditDate="2020-06-13T22:10:17.660" LastActivityDate="2020-09-11T10:54:21.943" Title="How is per-decision importance sampling derived in Sutton &amp; Barto's book?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;expectation&gt;&lt;importance-sampling&gt;&lt;conditional-probability&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21850" PostTypeId="1" CreationDate="2020-06-13T11:16:23.507" Score="0" ViewCount="36" Body="&lt;p&gt;I'm implementing an ecommerce search using nlp. there is a lack of clear documentation anywhere online about implementation and flow. this is what i'm doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Search query comes in&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do spellcheck&lt;/li&gt;&#xA;&lt;li&gt;Do lemmatization (like use men instead of husband,wed instead of wedding or marriage,have modified to include more than just lexical)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do Named entity recognition(NER)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.label brands,measurement units/symbols,numerics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.Recognize facets/filters among them (brand,size,color,length etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.Recognize model numbers (for this have to index all model number so is it a good idea to do under NER Only? or have a separate step to check it or just have model numbers inside product text indexed to search and match there)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.Symptoms and Fuzzy search again using NER only(like &quot;faulty&quot; pc,&quot;living room&quot; furniture)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;4.Split at stop words and look for additional faces/filter (laptop &quot;under&quot; 20000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5.After NER use recognized Entities for facets/filters and use non recognised for text search&lt;/p&gt;&#xA;&#xA;&lt;p&gt;another question is that some steps i'm doing based on manual logic if/else (like after NER step and splitting at stop words and querying word before and after if it's a numeral(then size or amount) or color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;also how to implement real time changes to model like for lemmatization is it better if i just index lemmas in lucene and add dynamically the new ones based on user searches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also i'm not using stemming since that i believe could lead to quite different results for synonyms which have same lemma but different stems&lt;/p&gt;&#xA;" OwnerUserId="37646" LastActivityDate="2020-06-13T11:16:23.507" Title="How to implement NLP for ecommerce product search" Tags="&lt;natural-language-processing&gt;&lt;search&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21851" PostTypeId="1" CreationDate="2020-06-13T12:34:32.943" Score="1" ViewCount="265" Body="&lt;p&gt;SegNet and U-Net are created for segmentation problem and EfficientNet is created for classification problem. I have a task and it is saying that train these models on the same dataset and compare results. Is it possible?&lt;/p&gt;&#xA;" OwnerUserId="37854" LastActivityDate="2020-06-13T12:34:32.943" Title="How to compare SegNet, U-Net and EfficientNet?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;tensorflow&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21852" PostTypeId="5" CreationDate="2020-06-13T13:54:21.203" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-13T13:54:21.203" LastActivityDate="2020-06-13T13:54:21.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21853" PostTypeId="4" CreationDate="2020-06-13T13:54:21.203" Score="0" Body="For questions related to the GameGAN, a generative model that learns to visually imitate a desired game by ingesting screenplay and keyboard actions during training. GameGAN was introduced in the paper &quot;Learning to Simulate Dynamic Environments with GameGAN&quot; (2020) by Seung Wook Kim et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-13T13:54:21.203" LastActivityDate="2020-06-13T13:54:21.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21854" PostTypeId="1" AcceptedAnswerId="21856" CreationDate="2020-06-13T16:10:33.907" Score="3" ViewCount="154" Body="&lt;p&gt;We assume infinite horizon and discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;. At each step, after the agent takes an action and gets its reward, there is a probability &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.2$&lt;/span&gt;, that agent will die. The assumed maze looks like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vTe5M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vTe5M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible actions are go left, right, up, down or stay in a square. The reward has a value 1 for any action done in the square (1,1) and zero for actions done in all the other squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, what is the value of a square (1,1)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct answer is supposed to be 5, and is calculated as &lt;span class=&quot;math-container&quot;&gt;$1/(1\cdot 0.2) = 5$&lt;/span&gt;. But why is that? I didn't manage to find any explanation on the net, so I am asking here. &lt;/p&gt;&#xA;" OwnerUserId="37858" LastEditorUserId="2444" LastEditDate="2020-06-14T15:53:57.957" LastActivityDate="2020-06-14T15:53:57.957" Title="What is the value of a state when there is a certain probability that agent will die after each step?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;value-functions&gt;&lt;value-iteration&gt;&lt;discount-factor&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21855" PostTypeId="2" ParentId="21854" CreationDate="2020-06-13T16:49:55.283" Score="3" Body="&lt;p&gt;The value of a state depends on the policy that you use, so I'll make the assumption here that you're talking about value &lt;strong&gt;using the optimal policy&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;According to the optimal policy, the agent would choose to stay in the square (1,1) every time&lt;/em&gt;, but since it has a 0.8 probability of actually staying (and 0.2 probability of dying), we can compute the value of the agent using the Bellman equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;V(1,1) = 1 + 0.8 V(1,1) + 0.2 V(\text{death state}) \\&#xA;\implies V(1,1) = 1 + 0.8 V(1,1) \\&#xA;\implies V(1,1) = \frac{1}{1 - 0.8} \\&#xA;\implies V(1,1) = 5&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other ways of deriving the same number (value function has multiple definitions) but they are equivalent.&lt;/p&gt;&#xA;" OwnerUserId="20714" LastEditorUserId="20714" LastEditDate="2020-06-13T17:03:19.367" LastActivityDate="2020-06-13T17:03:19.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21856" PostTypeId="2" ParentId="21854" CreationDate="2020-06-13T18:40:16.753" Score="2" Body="&lt;p&gt;I will fill in some details in shaabhishek's answer for people who are interested.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With this in mind, what is the value of a square (1,1)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;First of all, the value function is dependent on a policy. The supposed correct answer you provided is the value of &lt;span class=&quot;math-container&quot;&gt;$(1, 1)$&lt;/span&gt; under the &lt;em&gt;optimal&lt;/em&gt; policy, so from now on, we will assume that we are finding the value function under the optimal policy. Also, we will assume that the environment dynamics are deterministic: choosing to take an action will guarantee that the agent moves in that direction.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Possible actions are go left, right, up, down or stay in a square. Reward has a value 1 for any action done in square (1,1) and zero for actions done in all the other squares.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on this information, the optimal policy at &lt;span class=&quot;math-container&quot;&gt;$(1, 1)$&lt;/span&gt; should be to always stay in that square. The agent doesn't receive any reward for being in another square, and the probability of dying is the same for each square, so choosing the action to stay in square &lt;span class=&quot;math-container&quot;&gt;$(1, 1)$&lt;/span&gt; is best.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The correct answer is supposed to be 5, and is calculated as &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{1 \cdot 0.2} = 5$&lt;/span&gt;. But why is that?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By the Bellman Equation, the value function under the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi_*$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$(1,1)$&lt;/span&gt; can be written as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{\pi_*}((1, 1))  = \mathbb{E}_{\pi_*}\left[R_t + \gamma v_{\pi_{*}}(s') | s = (1,1)\right],$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; denotes the immediate reward, &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; denotes the current state, and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; denotes the next state. By the problem statement, &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;. The next state is the &lt;span class=&quot;math-container&quot;&gt;$\texttt{dead}$&lt;/span&gt; terminal state &lt;span class=&quot;math-container&quot;&gt;$\alpha = 20\%$&lt;/span&gt; of the time. Terminal states have value &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, as they do not accrue future rewards. The next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$(1, 1)$&lt;/span&gt; the remaining &lt;span class=&quot;math-container&quot;&gt;$(1-\alpha) = 80\%$&lt;/span&gt; of the time because our policy dictates to remain in the same state and we assumed the dynamics were deterministic. Since expectation is linear, we can rewrite the expectation as follows (replacing &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;v_{\pi_*}((1,1))  &amp;amp;= \mathbb{E}_{\pi_*}\left[R_t + v_{\pi_{*}}(s') | s = (1,1)\right]\\&#xA;&amp;amp;= \mathbb{E}_{\pi_*}\left[R_t |s=(1, 1)\right]+  \mathbb{E}_{\pi_*}\left[v_{\pi_{*}}(s') | s = (1,1)\right].\qquad (*)&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\pi_*}\left[R_t |s=(1, 1)\right] = 1\qquad (**)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because we are guaranteed an immediate reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; when taking an action in state &lt;span class=&quot;math-container&quot;&gt;$(1, 1)$&lt;/span&gt;. Also, from the comments above regarding the next state values and probabilities, we have the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}\mathbb{E}_{\pi_*}\left[v_{\pi_{*}}(s') | s = (1,1)\right] &amp;amp;= (1-\alpha) \cdot v_{\pi_{*}}((1,1)) + \alpha \cdot v_{\pi_*}(\texttt{dead})\\&#xA;&amp;amp;= 0.8 \cdot v_{\pi_{*}}((1,1)) + 0.2 \cdot 0\\&#xA;&amp;amp;= 0.8 \cdot v_{\pi_{*}}((1,1)).\qquad (***)&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting &lt;span class=&quot;math-container&quot;&gt;$(**)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(***)$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$(*)$&lt;/span&gt; yields the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;v_{\pi_*}((1,1)) &amp;amp;= 1 + 0.8 \cdot v_{\pi_{*}}((1,1))\\&#xA;v_{\pi_*}((1,1)) - 0.8 \cdot v_{\pi_{*}}((1,1)) &amp;amp;= 1\\&#xA;(1-0.8)v_{\pi_*}((1,1)) &amp;amp;= 1\\&#xA;v_{\pi_*}((1,1))  &amp;amp;= \frac{1}{1-0.8} = \frac{1}{0.2} = 5.&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-13T18:40:16.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21857" PostTypeId="5" CreationDate="2020-06-13T19:16:58.917" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-13T19:16:58.917" LastActivityDate="2020-06-13T19:16:58.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21858" PostTypeId="4" CreationDate="2020-06-13T19:16:58.917" Score="0" Body="For questions related to the family of models known as YOLO (which stands for &quot;You Only Look Once&quot;), which were proposed by Joseph Redmon et al. There are at least three YOLO models (versions 1, 2, and 3)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-13T19:16:58.917" LastActivityDate="2020-06-13T19:16:58.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21859" PostTypeId="1" AcceptedAnswerId="21882" CreationDate="2020-06-13T19:18:49.340" Score="3" ViewCount="222" Body="&lt;p&gt;Why don't we use an importance sampling ratio in Q-Learning, even though Q-Learning is an off-policy method? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Importance sampling is used to calculate expectation of a random variable by using data not drawn from the distribution. Consider taking a Monte Carlo average to calculate &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically an expectation is defined as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{x \sim p(x)}[X] = \sum_{x = \infty}^\infty x p(x)\;;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; denotes our probability mass function, and we can approximate this by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{x \sim p(x)}[X] \approx \frac{1}{n} \sum_{i=1}^nx_i\;;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; were simulated from &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can re-write the expectation from earlier as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{x \sim p(x)}[X] = \sum_{x = \infty}^\infty x p(x) = \sum_{x = \infty}^\infty x \frac{p(x)}{q(x)} q(x) = \mathbb{E}_{x\sim q(x)}\left[ X\frac{p(X)}{q(X)}\right]\;;$$&lt;/span&gt;&#xA;and so we can calculate the expectation using Monte Carlo averaging &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{x \sim p(x)}[X] \approx \frac{1}{n} \sum_{i=1}^nx_i \frac{p(x)}{q(x)}\;;$$&lt;/span&gt;&#xA;where the data &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; are now simulated from &lt;span class=&quot;math-container&quot;&gt;$q(x)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically importance sampling is used in RL when we use off-policy methods, i.e. the policy we use to calculate our actions is different from the policy we want to evaluate. Thus, I wonder why we don't use the importance sampling ratio in Q-learning, even though it is considered to be an off-policy method?&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-15T16:07:59.520" LastActivityDate="2020-06-15T16:07:59.520" Title="Why we don't use importance sampling in tabular Q-Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;importance-sampling&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21860" PostTypeId="5" CreationDate="2020-06-13T19:36:14.620" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-13T19:36:14.620" LastActivityDate="2020-06-13T19:36:14.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21861" PostTypeId="4" CreationDate="2020-06-13T19:36:14.620" Score="0" Body="For questions related to the concept of importance sampling (which comes up, for example, in reinforcement learning)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-13T19:36:14.620" LastActivityDate="2020-06-13T19:36:14.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21862" PostTypeId="2" ParentId="18157" CreationDate="2020-06-13T20:47:21.600" Score="0" Body="&lt;p&gt;FCNs can and typically have downsampling operations. For example, &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;u-net&lt;/a&gt; has downsampling (more precisely, max-pooling) operations. The difference between an FCN and a regular CNN is that the former does not have fully connected layers. See &lt;a href=&quot;https://ai.stackexchange.com/a/21824/2444&quot;&gt;this answer&lt;/a&gt; for more info. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, FCNs inherit the same properties of CNNs. There's nothing that a CNN (with fully connected layers) can do that an FCN cannot do. In fact, you can even simulate a fully connected layer with a convolution (with a kernel that has the same shape as the input volume).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-06-13T20:47:21.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21863" PostTypeId="2" ParentId="9569" CreationDate="2020-06-13T21:11:25.243" Score="0" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{X\thicksim P}[f(X)] = \sum P(X)f(X) = \sum Q(X)\frac{P(X)}{Q(X)}f(X) = \mathbb{E}_{X\thicksim Q}\left[\frac{P(X)}{Q(X)}f(X)\right]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$P = \pi, Q = b ,f = G_t|S_t $&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37611" LastActivityDate="2020-06-13T21:11:25.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21866" PostTypeId="1" CreationDate="2020-06-14T02:35:18.470" Score="1" ViewCount="70" Body="&lt;p&gt;I was reading this report: &lt;a href=&quot;https://www.theverge.com/2017/4/12/15271874/ai-adversarial-images-fooling-attacks-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.theverge.com/2017/4/12/15271874/ai-adversarial-images-fooling-attacks-artificial-intelligence&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researchers used noise to trick machine learning algorithms to misidentify or misclassify an image of a fish as a cat. I was wondering if something like that can be used to create subliminals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by subliminals: United Nations has defined subliminal messages as perceiving messages without being aware of them, it is unconscious perception, or perception without awareness. Like you may be aware of a message but cannot consciously perceive that message in the form of text, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the reports about the noise trick said the noise was so transparent that humans couldn't detect it. This can be changed to make it noticeable unconsciously but unnoticeable at a conscious level so a human can register the subliminal but not be aware of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to take an output from a hidden layer to construct such subliminal for humans, with trial and error one can find right combination? Can it be possible to come up with a pixel pattern or noise with ML which allows one to impose subliminals?&lt;/p&gt;&#xA;" OwnerUserId="31307" LastEditorUserId="2444" LastEditDate="2020-06-15T01:15:26.217" LastActivityDate="2020-11-12T04:03:25.357" Title="Can the addition of unnoticeable noise to images be used to create subliminals?" Tags="&lt;machine-learning&gt;&lt;social&gt;&lt;adversarial-ml&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21867" PostTypeId="1" CreationDate="2020-06-14T03:56:27.527" Score="2" ViewCount="56" Body="&lt;p&gt;I have a stream of data coming in like below (random numbers 0-9) &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;7, 7, 0, 0, 8, 9, 2, 7, 3, 8, 2, 8, 5, 7, 0, 8, 7, 8, 5, 3, 2, 6, 1, 9, 5, 7, 5, 3, 4, 9, 1, 3, 5, 5, 0, 7, 7, 5, 2, 8, 8, 7, 5, 5, 5, 2, 9, 7, 2, 1, 0, 0, 5, 7, 1, 4, 2, 7, 8, 8, 5, 2, 7, 5, 7, 1, 7, 2, 0, 5, 7, 5, 2, 6, 3, 6, 3, 6, 1, 9, 1, 9, 7, 2, 3, 9, 8, 8, 4, 9, 8, 2, 5, 3, 4, 0, 3, 1, 0, 7, 2, 3, 8, 7, 5, 7, 3, 6, 0, 3, 3, 3, 6, 3, 1, 3, 0, 6, 9, 8, 0, 1, 4, 4, 9, 9, 3, 7, 4, 1, 0, 5, 0, 6, 8, 8, 8, 1, 7, 6&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Ask: is to Predict the next numbers(at least 3-10). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which approach would be helpful in getting through this problem? &lt;/p&gt;&#xA;" OwnerUserId="37871" LastEditorUserId="2444" LastEditDate="2020-06-14T09:46:03.757" LastActivityDate="2020-06-14T09:46:03.757" Title="Which machine learning approach can be used to predict a univariate value?" Tags="&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21868" PostTypeId="1" CreationDate="2020-06-14T08:07:43.457" Score="1" ViewCount="17" Body="&lt;p&gt;I want to improve quality of translations for open-source projects in Ukrainian language. We have multiple translations from different authors. We can also translate messages using machine translations. Sometimes machine translation is even better than human translation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given multiple variants of translation of the same original text, I want to create AI which will be able to &quot;translate&quot; from Ukrainian to Ukrainian, using these multiple variants in parallel as the source, to produce one variant of higher quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in general, given multiple similar input sequences, the neural network needs to &quot;understand&quot; them, and produce a single output sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S_1, S_2, \dots \rightarrow S$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a simple example, we may want to train a NN to recognize a sequence of natural numbers: &lt;span class=&quot;math-container&quot;&gt;$1,2,3,4, \dots$&lt;/span&gt;. We give two sequences to NN: &lt;span class=&quot;math-container&quot;&gt;$23,4,24,6,8$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$3,65,5,6,23$&lt;/span&gt;, then trained the NN is expected to produce &lt;span class=&quot;math-container&quot;&gt;$3,4,5,6,7$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to modify an existing neural network to achieve that? Is it possible at all?&lt;/p&gt;&#xA;" OwnerUserId="37863" LastEditorUserId="2444" LastEditDate="2020-06-14T09:58:18.903" LastActivityDate="2020-06-14T09:58:18.903" Title="How to autocorelate multiple variants of same text into one?" Tags="&lt;natural-language-processing&gt;&lt;ai-design&gt;&lt;machine-translation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21869" PostTypeId="2" ParentId="21866" CreationDate="2020-06-14T08:30:05.503" Score="-1" Body="&lt;p&gt;I would say no, at least not now. To achieve that you need to have a model that represents the structure of a human brain at a very detailed level, so that inherits the same flaws.&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-14T08:30:05.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21870" PostTypeId="1" CreationDate="2020-06-14T09:05:27.043" Score="0" ViewCount="45" Body="&lt;p&gt;Pseudo code for Cross entropy method according to &lt;a href=&quot;https://youtu.be/aUrX-rP_ss4?t=1975&quot; rel=&quot;nofollow noreferrer&quot;&gt;youtube lecture 32:55&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize &lt;span class=&quot;math-container&quot;&gt;$\mu \in R^{d}, \sigma \in R^{d}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;iteration 1,2,...&lt;/li&gt;&#xA;&lt;li&gt;Collect n samples of &lt;span class=&quot;math-container&quot;&gt;$\theta_{i} \sim N(\mu,diag(\sigma))$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Perform Noisy evaluation &lt;span class=&quot;math-container&quot;&gt;$R_{i} \sim \theta_{i}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Select the top p% of samples (e.g. p = 20), which we'll call the &lt;strong&gt;elite set&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Fit a Gaussian distribution, with diagonal covariance, to the elite set, obtaining a new &lt;span class=&quot;math-container&quot;&gt;$\mu, \sigma$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Return the final &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is what I am doing:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create a neural network that takes in the state and gives out the action&lt;/li&gt;&#xA;&lt;li&gt;Create separate vectors for mean and std of all the network parameters. (i.e. a 1-d vector that contains mean and std for each parameter of neural net)&lt;/li&gt;&#xA;&lt;li&gt;Repeat n times:&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Initialize network parameters by sampling from gaussian distribution corresponding to all the parameters (using their mean and std defined in step 2) (let this be &lt;span class=&quot;math-container&quot;&gt;$\theta_{i}$&lt;/span&gt;)&lt;/li&gt;&#xA;  &lt;li&gt;Get a single trajectory from this neural network, and use this as the cost of the &lt;span class=&quot;math-container&quot;&gt;$\theta_{i}$&lt;/span&gt; (some number of timesteps)&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;Select top p% trajectories as the &lt;strong&gt;elite set&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Take mean and standard deviation of their parameter vectors.&lt;/li&gt;&#xA;&lt;li&gt;Save this new mean and standard deviation vector to be used in step 2 next time. (when finding action for another state)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this a valid cross entropy method algorithm if this is run at every query for getting action?&lt;/li&gt;&#xA;&lt;li&gt;Is taking just a single trajectory for each parameter enough in step 3.2?&lt;/li&gt;&#xA;&lt;li&gt;Given that the parameters come from a gaussian distribution all the time, is there an analytic solution to the problem of getting the best parameters or sampling is required? (Section 4 of this paper &lt;a href=&quot;https://www.aaai.org/Papers/ICML/2003/ICML03-068.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross Entropy Method for fast policy search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Can step 3 be parallelized?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36861" LastActivityDate="2020-06-14T09:05:27.043" Title="Model Based rl and cross entropy method with nonlinear function approximators" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;markov-decision-process&gt;&lt;model-based-methods&gt;&lt;cross-entropy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21871" PostTypeId="2" ParentId="21867" CreationDate="2020-06-14T09:16:41.107" Score="2" Body="&lt;p&gt;Assuming your data is &lt;strong&gt;not&lt;/strong&gt; purely random (otherwise it will be difficult to make useful predictions), you can try the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hidden Markov Models&lt;/a&gt;: Treating your discrete numbers as states, you can use HMMs to predict the next state (next number) by looking at the last or the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; last states.&lt;/li&gt;&#xA;&lt;li&gt;Recurrent Neural Networks: Assuming you have a lot of number sequences, you can treat your numbers as time series. Train a model and predict the next number(s). &lt;a href=&quot;https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a nice tutorial which uses LSTM cells.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Autoregressive_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autoregressive models&lt;/a&gt;: AR models are quite powerful as well and have already been successfully used in finance and signal processing before the rise of deep learning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Further this post (&lt;a href=&quot;https://stats.stackexchange.com/q/24521/264183&quot;&gt;Problem in discrete valued time series forecasting&lt;/a&gt;) on Cross Validated might answer your question as well.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-06-14T09:16:41.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21872" PostTypeId="1" AcceptedAnswerId="21878" CreationDate="2020-06-14T09:56:46.733" Score="1" ViewCount="115" Body="&lt;p&gt;Using the tutorial from: &lt;a href=&quot;https://pythonprogramming.net/q-learning-algorithm-reinforcement-learning-python-tutorial/?completed=/q-learning-reinforcement-learning-python-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SentDex - Python Programming&lt;/a&gt; I added Q Learning to my script that was previously just picking random actions. His script uses the MountainCar Environment so I had to amend it to the CartPole env I am using. Initially, the rewards seem sporadic but, after a while, they just drop off and oscillate between 0-10. Does anyone know why this is?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Learning_rate = 0.1&#xA;Discount_rate = 0.95&#xA;episodes = 200&#xA;&#xA;# Exploration settings&#xA;epsilon = 1  # not a constant, qoing to be decayed&#xA;START_EPSILON_DECAYING = 1&#xA;END_EPSILON_DECAYING = episodes//2&#xA;epsilon_decay_value = epsilon/(END_EPSILON_DECAYING - START_EPSILON_DECAYING)&#xA;&#xA;env = gym.make(&quot;CartPole-v0&quot;) #Create the environment. The name of the environments can be found @ https://gym.openai.com/envs/#classic_control&#xA;#Each environment has a number of possible actions. In this case there are two discrete actions, left or right&#xA;&#xA;#Each environment has some integer characteristics of the state.&#xA;#In this case we have 4:&#xA;&#xA;#env = gym.wrappers.Monitor(env, './', force=True)&#xA;&#xA;DISCRETE_OS_SIZE = [20, 20, 20, 20]&#xA;&#xA;discrete_os_win_size = (env.observation_space.high - env.observation_space.low)/ DISCRETE_OS_SIZE &#xA;&#xA;def get_discrete_state(state):&#xA;    discrete_state = (state - env.observation_space.low)/discrete_os_win_size&#xA;    return tuple(discrete_state.astype(np.int))&#xA;&#xA;q_table = np.random.uniform(low = -2, high = 0, size = (20, 20, 20, 20, env.action_space.n))&#xA;&#xA;plt.figure() #Instantiate the plotting environment&#xA;rewards_list = [] #Create an empty list to add the rewards to which we will then plot&#xA;for i in range(episodes):&#xA;    discrete_state = get_discrete_state(env.reset())&#xA;    done = False&#xA;    rewards = 0&#xA;    frames = []&#xA;&#xA;    while not done:&#xA;        #frames.append(env.render(mode = &quot;rgb_array&quot;))&#xA;&#xA;        if np.random.random() &amp;gt; epsilon:&#xA;            # Get action from Q table&#xA;            action = np.argmax(q_table[discrete_state])&#xA;&#xA;        else:&#xA;            # Get random action&#xA;            action = np.random.randint(0, env.action_space.n)&#xA;&#xA;        new_state, reward, done, info = env.step(action)&#xA;&#xA;        new_discrete_state = get_discrete_state(new_state)&#xA;&#xA;        # If simulation did not end yet after last step - update Q table&#xA;        if not done:&#xA;&#xA;            # Maximum possible Q value in next step (for new state)&#xA;            max_future_q = np.max(q_table[new_discrete_state])&#xA;&#xA;            # Current Q value (for current state and performed action)&#xA;            current_q = q_table[discrete_state, action]&#xA;&#xA;            # And here's our equation for a new Q value for current state and action&#xA;            new_q = (1 - Learning_rate) * current_q + Learning_rate * (reward + Discount_rate * max_future_q)&#xA;&#xA;            # Update Q table with new Q value&#xA;            q_table[discrete_state, action] = new_q&#xA;&#xA;        else:&#xA;            q_table[discrete_state + (action,)] = 0&#xA;&#xA;        discrete_state = new_discrete_state&#xA;&#xA;        rewards += reward&#xA;        rewards_list.append(rewards)&#xA;    #print(&quot;Episode:&quot;, i, &quot;Rewards:&quot;, rewards)&#xA;    #print(&quot;Observations:&quot;, obs)&#xA;&#xA;    # Decaying is being done every episode if episode number is within decaying range&#xA;    if END_EPSILON_DECAYING &amp;gt;= i &amp;gt;= START_EPSILON_DECAYING:&#xA;        epsilon -= epsilon_decay_value&#xA;&#xA;plt.plot(rewards_list)&#xA;plt.show()&#xA;env.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Abvo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Abvo.png&quot; alt=&quot;CartPoleRewards&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It becomes even more pronounced when I increase episodes to 20,000 so I don't think it's related to not giving the model enough training time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LaVby.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LaVby.png&quot; alt=&quot;20000 episodes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I set &lt;code&gt;START_EPSILON_DECAYING&lt;/code&gt; to say 200 then it only drops to &amp;lt; 10 rewards after episode 200 which made me think it was the epsilon that was causing the problem. However, if I remove the epsilon/exploratory then the rewards at every episode are worse as it gets stuck in picking the argmax value for each state.&lt;/p&gt;&#xA;" OwnerUserId="36082" LastEditorUserId="2444" LastEditDate="2020-06-14T14:40:15.657" LastActivityDate="2020-06-14T15:17:37.177" Title="Why do my rewards fall using tabular Q-learning as I perform more episodes?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="21873" PostTypeId="1" CreationDate="2020-06-14T12:09:13.400" Score="0" ViewCount="71" Body="&lt;p&gt;Is there any sanity check to know whether the Q functions learnt are appropriate in deep Q networks? I know that the Q values for end states should approximate the terminal reward. However, is it normal that Q values for the non-terminal states have higher values than those of the terminal states?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why I want to know whether Q values learnt are appropriate is because I want to apply the &lt;a href=&quot;https://arxiv.org/abs/1103.4601&quot; rel=&quot;nofollow noreferrer&quot;&gt;doubly robust estimator for off-policy value evaluation&lt;/a&gt;. Using doubly robust requires a good Q value estimate to be learnt for each state.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-06-14T13:04:15.580" LastActivityDate="2020-06-15T07:27:37.527" Title="How do I know that the DQN has learnt an appropriate Q function?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;off-policy-methods&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21874" PostTypeId="2" ParentId="11172" CreationDate="2020-06-14T12:51:56.600" Score="2" Body="&lt;p&gt;To show how the convolution (in the context of CNNs) can be viewed as matrix-vector multiplication, let's suppose that we want to apply a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; kernel to a &lt;span class=&quot;math-container&quot;&gt;$4 \times 4$&lt;/span&gt; input, with no padding and with unit stride.&lt;/p&gt;&#xA;&lt;p&gt;Here's an illustration of this convolutional layer (where, in blue, we have the input, in dark blue, the kernel, and, in green, the feature map or output of the convolution).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zviX9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zviX9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, let the kernel be defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{W} = &#xA;\begin{bmatrix}&#xA;w_{0, 0} &amp;amp; w_{0, 1} &amp;amp; w_{0, 2} \\&#xA;w_{1, 0} &amp;amp; w_{1, 1} &amp;amp; w_{1, 2} \\&#xA;w_{2, 0} &amp;amp; w_{2, 1} &amp;amp; w_{2, 2}&#xA;\end{bmatrix} &#xA;\in&#xA;\mathbb{R}^{3 \times 3}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Similarly, let the input be defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{I} = &#xA;\begin{bmatrix}&#xA;i_{0, 0} &amp;amp; i_{0, 1} &amp;amp; i_{0, 2} &amp;amp; i_{0, 3} \\&#xA;i_{1, 0} &amp;amp; i_{1, 1} &amp;amp; i_{1, 2} &amp;amp; i_{1, 3} \\&#xA;i_{2, 0} &amp;amp; i_{2, 1} &amp;amp; i_{2, 2} &amp;amp; i_{2, 3} \\&#xA;i_{3, 0} &amp;amp; i_{3, 1} &amp;amp; i_{3, 2} &amp;amp; i_{3, 3} \\ &#xA;\end{bmatrix} &#xA;\in&#xA;\mathbb{R}^{4 \times 4}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the convolution above (without padding and with stride 1) can be computed as a matrix-vector multiplication as follows. First, we redefine the kernel &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; as a sparse matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}' \in \mathbb{R}^{4 \times 16}$&lt;/span&gt; (which is a &lt;a href=&quot;https://ccrma.stanford.edu/%7Ejos/fp/Cyclic_Convolution_Matrix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;circulant matrix&lt;/em&gt;&lt;/a&gt; because of its circular nature) as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{\scriptscriptstyle&#xA;\mathbf{W}' = &#xA;\begin{bmatrix}&#xA;w_{0, 0} &amp;amp; w_{0, 1} &amp;amp; w_{0, 2} &amp;amp; 0 &amp;amp; w_{1, 0} &amp;amp; w_{1, 1} &amp;amp; w_{1, 2} &amp;amp; 0 &amp;amp;  w_{2, 0} &amp;amp; w_{2, 1} &amp;amp; w_{2, 2} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; w_{0, 0} &amp;amp; w_{0, 1} &amp;amp; w_{0, 2} &amp;amp; 0 &amp;amp; w_{1, 0} &amp;amp; w_{1, 1} &amp;amp; w_{1, 2} &amp;amp; 0 &amp;amp; w_{2, 0} &amp;amp; w_{2, 1} &amp;amp; w_{2, 2} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0  \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; w_{0, 0} &amp;amp; w_{0, 1} &amp;amp; w_{0, 2} &amp;amp; 0 &amp;amp; w_{1, 0} &amp;amp; w_{1, 1} &amp;amp; w_{1, 2} &amp;amp; 0 &amp;amp;  w_{2, 0} &amp;amp; w_{2, 1} &amp;amp; w_{2, 2} &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; w_{0, 0} &amp;amp; w_{0, 1} &amp;amp; w_{0, 2} &amp;amp; 0 &amp;amp; w_{1, 0} &amp;amp; w_{1, 1} &amp;amp; w_{1, 2} &amp;amp; 0 &amp;amp;  w_{2, 0} &amp;amp; w_{2, 1} &amp;amp; w_{2, 2}&#xA;\end{bmatrix} &#xA;}&#xA;$$&lt;/span&gt;&#xA;Similarly, we reshape the input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; as a 16-dimensional vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}' \in  \mathbb{R}^{16}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{\scriptstyle&#xA;\mathbf{I}' = &#xA;\begin{bmatrix}&#xA;i_{0, 0} &amp;amp; i_{0, 1} &amp;amp; i_{0, 2} &amp;amp; i_{0, 3} &amp;amp; i_{1, 0} &amp;amp; i_{1, 1} &amp;amp; i_{1, 2} &amp;amp; i_{1, 3} &amp;amp; i_{2, 0} &amp;amp; i_{2, 1} &amp;amp; i_{2, 2} &amp;amp; i_{2, 3} &amp;amp; i_{3, 0} &amp;amp; i_{3, 1} &amp;amp; i_{3, 2} &amp;amp; i_{3, 3} &#xA;\end{bmatrix}^T&#xA;}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the convolution of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt;, that is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{W} \circledast \mathbf{I} = \mathbf{O} \in \mathbb{R}^{2 \times 2},$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt; is the convolution operator, is equivalently defined as &lt;span class=&quot;math-container&quot;&gt;$$\mathbf{W}' \cdot \mathbf{I}' = \mathbf{O}' \in \mathbb{R}^{4},$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; is the matrix-vector multiplication operator. The produced vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{O}'$&lt;/span&gt; can then be reshaped as a &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; feature map.&lt;/p&gt;&#xA;&lt;p&gt;You can easily verify that this representation is correct by multiplying e.g. the 16-dimensional input vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}'$&lt;/span&gt; with the first row of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}'$&lt;/span&gt; to obtain the top-left entry of the feature map.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w_{0, 0} i_{0, 0} + w_{0, 1} i_{0, 1} + w_{0, 2} i_{0, 2} + 0 i_{0, 3} + w_{1, 0} i_{1, 0} + w_{1, 1} i_{1, 1} + w_{1, 2}i_{1, 2} + 0 i_{1, 3} +  w_{2, 0} i_{2, 0} + w_{2, 1}i_{2, 1} + w_{2, 2} i_{2, 2} + 0 i_{2, 3} + 0 i_{3, 0} + 0 i_{3, 1} + 0 i_{3, 2} + 0 i_{3, 3} = \\&#xA;w_{0, 0} i_{0, 0} + w_{0, 1} i_{0, 1} + w_{0, 2} i_{0, 2} + w_{1, 0} i_{1, 0} + w_{1, 1} i_{1, 1} + w_{1, 2}i_{1, 2} + w_{2, 0} i_{2, 0} + w_{2, 1}i_{2, 1} + w_{2, 2} i_{2, 2} = \\&#xA;\mathbf{O}'_{0}  \in \mathbb{R} ,$$&lt;/span&gt; which is equivalent to an element-wise multiplication of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; with the top-left &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; sub-matrix of the input followed by a summation over all elements (i.e. convolution), that is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum \left(&#xA;\begin{bmatrix}&#xA;w_{0, 0} &amp;amp; w_{0, 1} &amp;amp; w_{0, 2} \\&#xA;w_{1, 0} &amp;amp; w_{1, 1} &amp;amp; w_{1, 2} \\&#xA;w_{2, 0} &amp;amp; w_{2, 1} &amp;amp; w_{2, 2}&#xA;\end{bmatrix} &#xA;\odot&#xA;\begin{bmatrix}&#xA;i_{0, 0} &amp;amp; i_{0, 1} &amp;amp; i_{0, 2} \\&#xA;i_{1, 0} &amp;amp; i_{1, 1} &amp;amp; i_{1, 2}  \\&#xA;i_{2, 0} &amp;amp; i_{2, 1} &amp;amp; i_{2, 2}&#xA;\end{bmatrix} &#xA;\right) = \mathbf{O}_{0, 0} = \mathbf{O}'_{0} \in \mathbb{R},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; is the element-wise multiplication and &lt;span class=&quot;math-container&quot;&gt;$\sum$&lt;/span&gt; is the summation over all elements of the resulting matrix.&lt;/p&gt;&#xA;&lt;p&gt;The advantage of this representation (and computation) is that back-propagation can be computed more easily by just transposing &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}'$&lt;/span&gt;, i.e. with &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}'^T$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Take also a look at &lt;a href=&quot;https://github.com/alisaaalehi/convolution_as_multiplication&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Github repository&lt;/a&gt; that explains how the convolution can be implemented as matrix multiplication.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T00:33:53.540" LastActivityDate="2020-11-07T00:33:53.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21875" PostTypeId="1" CreationDate="2020-06-14T13:38:57.710" Score="0" ViewCount="137" Body="&lt;p&gt;I had done with different classification, regression and clustering approaches for predictions of values, etc. I was wondering if there is a machine learning approach for distribution of a whole based on some features (I do not know if there is an approach for that I just could not find one with my research).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An easy example might be lets consider we have height and weight data of many children and we have to distribute a given number of pizza slices amongst them so that skinny children get more pizza as compared to obese ones because pizza is more beneficial for skinny as compared to obese. So might have to find out the optimum number of slices for each child out of the total number of slices so that each child gets maximum possible nutrients. A more complex version could incorporate more features like age, overall health, blood sugar content, physical activity index, daily calorie consumption, and others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar example might be to find out the optimal value of fuel to be allocated to each vehicle if we have a total of 100 gallons. Features might be distance they have to travel, mpg, driver competency, engine horsepower, etc., so that all of them might travel the maximum distance possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can we achieve a task like this with machine learning/deep learning approaches? If not what are the hurdles achieving this?&lt;/p&gt;&#xA;" OwnerUserId="6635" LastEditorUserId="2444" LastEditDate="2020-06-15T10:23:47.460" LastActivityDate="2020-06-15T10:47:39.710" Title="Can a machine learning approach solve this constrained optimisation problem?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;optimization&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21876" PostTypeId="1" AcceptedAnswerId="21932" CreationDate="2020-06-14T13:55:02.290" Score="0" ViewCount="256" Body="&lt;p&gt;I'm trying to implement a soft actor-critic algorithm for financial data (stock prices), but I have trouble with losses: no matter what combination of hyper-parameters I enter, they are not converging, and basically it caused bad reward return as well. It sounds like the agent is not learning at all.&lt;/p&gt;&#xA;&lt;p&gt;I already tried to tune some hyperparameters (learning rate for each network + number of hidden layers), but I always get similar results.&#xA;The two plots below represent the losses of my policy and one of the value functions during the last episode of training.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r1xdn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r1xdn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3LtkW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3LtkW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is, &lt;strong&gt;would it be related to the data itself (nature of data) or is it something related to the logic of the code?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="37880" LastEditorUserId="2444" LastEditDate="2020-11-23T01:48:20.810" LastActivityDate="2020-11-23T01:48:20.810" Title="Why is my Soft Actor-Critic's policy and value function losses not converging?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;actor-critic-methods&gt;&lt;hyperparameter-optimization&gt;&lt;soft-actor-critic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21878" PostTypeId="2" ParentId="21872" CreationDate="2020-06-14T14:49:13.527" Score="1" Body="&lt;p&gt;The problem here is likely related to the state approximations you are using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, OpenAI's gym does not always give reasonable bounds when using &lt;code&gt;env.observation_space&lt;/code&gt;, and that seems to be the case for CartPole:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; env = gym.make('CartPole-v0')&#xA;&amp;gt;&amp;gt;&amp;gt; env.observation_space.high&#xA;array([4.8000002e+00, 3.4028235e+38, 4.1887903e-01, 3.4028235e+38],&#xA;      dtype=float32)&#xA;&amp;gt;&amp;gt;&amp;gt; env.observation_space.low&#xA;array([-4.8000002e+00, -3.4028235e+38, -4.1887903e-01, -3.4028235e+38],&#xA;      dtype=float32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Processing this, similarly to your code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; discrete_os_win_size = (env.observation_space.high - env.observation_space.low)/ DISCRETE_OS_SIZE&#xA;__main__:1: RuntimeWarning: overflow encountered in subtract&#xA;&amp;gt;&amp;gt;&amp;gt; discrete_os_win_size&#xA;array([0.48000002,        inf, 0.0418879 ,        inf])&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; discrete_state = (state - env.observation_space.low)/discrete_os_win_size&#xA;&amp;gt;&amp;gt;&amp;gt; discrete_state&#xA;array([11.27318768,  0.        , 19.50682776,  0.        ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That means that all the velocities will get squashed down to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; in your approximation. Your agent cannot tell the difference between a nearly static balancing position (generally the goal) and transitioning through it really fast - it will think that both are equally good. It is also not able to tell difference between moving towards balance point, or moving away from it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you check for what reasonable bounds are on the space (a quick look suggests +/- 2.0 might be a reasonable starting point) and use that instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approximation approach of discrete grid is also very crude, although it does allow you do use tabular approaches. If you want to stick with a linear system (and avoid trying neural networks and DQN) then the next step up would be &lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-tile-coding-implementation-7974b600762b&quot; rel=&quot;nofollow noreferrer&quot;&gt;some form of tile coding&lt;/a&gt;, which uses multiple offset grids to obtain smoother interpolation between states.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-14T15:17:37.177" LastActivityDate="2020-06-14T15:17:37.177" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21879" PostTypeId="1" CreationDate="2020-06-14T14:56:21.220" Score="0" ViewCount="123" Body="&lt;p&gt;Why is it useful to track loss while the model is being trained?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Loss is only useful as a final metric. It should not be evaluated while the model is being trained.&lt;/li&gt;&#xA;&lt;li&gt;Loss dictates how effective the model is.&lt;/li&gt;&#xA;&lt;li&gt;Loss can help understand how much the model is changing per iteration. When it converges, that's an indicator that further training will have little benefit.&lt;/li&gt;&#xA;&lt;li&gt;None of the above&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37882" LastEditorUserId="2444" LastEditDate="2020-06-14T15:04:13.197" LastActivityDate="2020-06-14T20:01:28.423" Title="Why is it useful to track loss while model is being trained?" Tags="&lt;machine-learning&gt;&lt;loss-functions&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21880" PostTypeId="1" AcceptedAnswerId="21884" CreationDate="2020-06-14T15:16:28.650" Score="3" ViewCount="178" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Deepfake#:~:text=Deepfakes%20(a%20portmanteau%20of%20%22deep,replaced%20with%20someone%20else&amp;#39;s%20likeness.&amp;amp;text=Deepfakes%20have%20garnered%20widespread%20attention,%2C%20hoaxes%2C%20and%20financial%20fraud.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deepfakes&lt;/a&gt; (a portmanteau of &quot;deep learning&quot; and &quot;fake&quot;) are synthetic media in which a person in an existing image or video is replaced with someone else's likeness.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Nowadays most of the news circulating in the news and social media are fake/gossip/rumors which may false-positives or false-negatives except  &lt;a href=&quot;https://en.wikipedia.org/wiki/WikiLeaks#:~:text=Authenticity,placed%20in%20the%20mainstream%20media.&quot; rel=&quot;nofollow noreferrer&quot;&gt;WikiLeaks&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there has been a &lt;a href=&quot;https://www.kaggle.com/c/deepfake-detection-challenge&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deepfake Detection Challenge&lt;/a&gt; Kaggle competition for a whooping sum $1,000,000 prize money.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know how deepfakes work and how they might be dangerous?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-14T15:59:40.063" LastActivityDate="2020-06-14T19:41:55.290" Title="How do deepfakes work and how they might be dangerous?" Tags="&lt;deep-learning&gt;&lt;deepfakes&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21881" PostTypeId="1" AcceptedAnswerId="21895" CreationDate="2020-06-14T15:29:57.433" Score="1" ViewCount="69" Body="&lt;p&gt;I'm using Q-learning (off-policy TD-control as specified in Sutton's book on pg 131) to train an agent to play connect four. My goal is to create a strong player (superhuman performance?) purely by self-play, without training models against other agents obtained externally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using neural network architectures with some convolutional layers and several fully connected layers. These train surprisingly efficiently against their opponent, either a random player or another agent previously trained through Q-learning. Unfortunately the resulting models don't generalise well. 5000 episodes seems enough to obtain a high (&gt; 90%) win rate against whichever opponent, but after &gt; 20 000 episodes, they are still rather easy to beat by myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this, I now train batches of models (~ 10 models per batch), which are then used in group as a new opponent, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I train a batch of models against a completely random agent (let's call them the generation one)&lt;/li&gt;&#xA;&lt;li&gt;Then I train a second generation of agents against this first generation&lt;/li&gt;&#xA;&lt;li&gt;Then I train a third generation against generation two&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So far this helped in creating a slightly stronger/more general connect four model, but the improvement is not as good as I was hoping for. Is it just a matter of training enough models/generations or are there better ways for using Q-learning in combination with self-play?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the most successful techniques (e.g. alpha zero) rely on MCTS, but I'm not sure how to integrate this with Q-learning? Neither how MCTS helps to solve the problem of generalisation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help!&lt;/p&gt;&#xA;" OwnerUserId="34059" LastActivityDate="2020-06-15T07:54:28.163" Title="Generalising performance of Q-learning agent through self-play in a two-player game (MCTS?)" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;self-play&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21882" PostTypeId="2" ParentId="21859" CreationDate="2020-06-14T16:21:39.347" Score="3" Body="&lt;p&gt;In Tabular Q-learning the update is as follows &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = Q(s,a) + \alpha \left[R_{t+1} + \gamma \max_aQ(s',a) - Q(s,a) \right]\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as we are interested in learning about the optimal policy, this would correspond to the &lt;span class=&quot;math-container&quot;&gt;$\max_aQ(s',a)$&lt;/span&gt; term in the TD target because that is how the optimal policy chooses its actions - i.e. &lt;span class=&quot;math-container&quot;&gt;$\pi_*(a|s) = \arg\max_aQ_*(s,a)$&lt;/span&gt;, so eventually the greedy TD update would be greedy with respect to the optimal state-action value function due to the guaranteed convergence of Q-learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the update rule, i.e. the action we chose in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to receive the reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;, was chosen according to some non-optimal policy, e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy. However, as the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function is defined as the expected returns assuming we are in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;em&gt;have taken action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;&lt;/em&gt; -- we thus don't need an importance sampling ratio for the &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt; term, even though it was generated from an action that the optimal policy might not have taken, because we are only updating the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function for state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, and by the definition of a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function it is assumed that we have taken action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; as we condition on this. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-14T16:21:39.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21883" PostTypeId="1" AcceptedAnswerId="21945" CreationDate="2020-06-14T17:28:28.307" Score="2" ViewCount="422" Body="&lt;p&gt;I'm currently trying to understand the difference between a vanilla LSTM and a fully connected LSTM. In &lt;a href=&quot;https://arxiv.org/abs/1506.04214&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt; I'm reading, the FC-LSTM gets introduced as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;FC-LSTM may be seen as a multivariate version of LSTM where the input, cell output and states are all 1D vectors&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But is not really expanded further upon. Google also didn't help me much in that regard as I can't seem to find anything under that keyword. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between the two? Also, I'm a bit confused by the quote - aren't inputs, outputs, etc. of a vanilla LSTM already 1D vectors?&lt;/p&gt;&#xA;" OwnerUserId="37883" LastEditorUserId="37883" LastEditDate="2020-06-15T14:10:22.287" LastActivityDate="2020-06-16T13:49:37.890" Title="What is the difference between LSTM and fully connected LSTM?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21884" PostTypeId="2" ParentId="21880" CreationDate="2020-06-14T17:32:30.417" Score="2" Body="&lt;p&gt;In general, deepfakes rely on advanced context-aware digital signal manipulations - usually image, video or audio - that allow for very natural looking modifications of content that previously have been costly or near impossible to produce in high quality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The AI models, often based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Generative_adversarial_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;generative adversarial networks (GANs)&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_Style_Transfer&quot; rel=&quot;nofollow noreferrer&quot;&gt;style transfer&lt;/a&gt;, &lt;a href=&quot;https://www.youtube.com/watch?v=F84jaIR5Uxc&quot; rel=&quot;nofollow noreferrer&quot;&gt;pose estimation&lt;/a&gt; and similar technologies, are capable of tasks such as transferring facial features from subject A to replace those of subject B in a still image or video, whilst copying subject B's pose, expression, and matching the scene's lighting. Similar technologies exist for voices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example of this might be &lt;a href=&quot;https://www.youtube.com/watch?v=5nDnlA1pv5U&quot; rel=&quot;nofollow noreferrer&quot;&gt;these Star Wars edits&lt;/a&gt;, where actors faces have been changed. It is not perfect, you can in a few shots see a little instability if you study the frames - but the quality is still pretty good, and it was done with a relatively inexpensive setup. The work was achieved using freely-available software, such as &lt;a href=&quot;https://github.com/iperov/DeepFaceLab&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepFaceLab on Github&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technology is not limited to simple replacements - other forms of puppet-like control over output are possible, where &lt;a href=&quot;https://www.youtube.com/watch?v=ohmajJTcpNk&quot; rel=&quot;nofollow noreferrer&quot;&gt;an actor can directly control the face of a target&lt;/a&gt; in real time using no more than a PC and webcam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, with the aid of deepfakes, it becomes possible to back up slanderous or libelous commentary with convincing media, at a low price point. Or the reverse, to re-word or re-enact an event that would otherwise be negative publicity for someone, in order to make it seem very different yet still naturally captured.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The danger of this technology is it puts tools for misinformation into a lot of people's hands. This leads to potential problems including:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Attacks on integrity of public figures, backed by realistic-looking &quot;evidence&quot;. Even with the knowledge that this fakery is possible (and perhaps likely given a particular context), then damage can still be done especially towards feeding people with already-polarised opinions with manufactured events, relying on confirmation bias.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Erosion of belief in any presented media as proof of anything. With deepfakes out in the wild, someone confronted with media evidence that went against any narrative can claim &quot;fake&quot; that much more easily.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Neither of these issues are new in the domains of reporting, political bias, propaganda etc. However, it adds another powerful tool for people willing to spread misinformation to support any agenda, alongside things such as selective statistics, quoting out of context, lies in media that is text-only or crudely photoshopped etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A search for papers studying impact of deep fakes should find academic research such as &lt;a href=&quot;https://journals.sagepub.com/doi/full/10.1177/2056305120903408&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deepfakes and Disinformation: Exploring the Impact of Synthetic Political Video on Deception, Uncertainty, and Trust in News&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Opinion: Video content, presented feasibly as a live capture or report, is especially compelling, as unlike text and still image media, it directly interfaces to two key senses that humans use to understand and navigate the world in real time. In short, it is more believable by default at an unconscious and emotional level compared to a newspaper article or even a photo. And that applies despite any academic knowledge of how it is produced that you might possess as a viewer.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-14T19:41:55.290" LastActivityDate="2020-06-14T19:41:55.290" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21885" PostTypeId="1" CreationDate="2020-06-14T18:21:00.140" Score="-1" ViewCount="220" Body="&lt;p&gt;After a GAN is trained, which parts of it are used to generate new outputs from data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Neither&lt;/li&gt;&#xA;&lt;li&gt;Discriminator&lt;/li&gt;&#xA;&lt;li&gt;Generator&lt;/li&gt;&#xA;&lt;li&gt;Both Generator and Discriminator&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37882" LastEditorUserId="2444" LastEditDate="2020-06-14T19:32:47.290" LastActivityDate="2020-11-12T11:00:30.053" Title="After a GAN is trained, which parts of it are used to generate new outputs from data?" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;homework&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21886" PostTypeId="2" ParentId="21879" CreationDate="2020-06-14T19:49:02.860" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss function&lt;/a&gt; (aka cost function) measures the correctness of the predictions of the model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a simple cost function could be &lt;span class=&quot;math-container&quot;&gt;$|y - f(x)|$&lt;/span&gt;, where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat{y} = f(x)$&lt;/span&gt; is the prediction of the model &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; when the input is &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the ground-truth label for input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (i.e. what the model is supposed to output when the input is &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;), and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$|\cdot|$&lt;/span&gt; is the absolute value&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The loss function is also used to understand whether a model is overfitting (and underfitting) or not. More specifically, if the loss of the model evaluated on the training data (the training loss) keeps decreasing while the loss of the model evaluated on the validation data (validation loss) starts to increase, that's a good sign that the model is overfitting (i.e. just memorizing the training data, but it will likely perform poorly on non-training data). If the training loss does not decrease, that's a good sign of underfitting (i.e. the model is not capable of learning the patterns in the training data), so you may need to use a model with a bigger capacity, change the loss, or do something else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss is typically not directly used as a measure of the &lt;a href=&quot;https://stats.stackexchange.com/q/385312/82135&quot;&gt;performance&lt;/a&gt; of the model because the loss doesn't directly represent the performance of the model. In fact, the loss can be relatively big (with respect to another model's loss) and the model still &lt;em&gt;performs&lt;/em&gt; well, but typically a lower loss corresponds to higher performance. The performance of the model is calculated differently depending on the task, model, and your goals. The most common performance measure is the &lt;em&gt;accuracy&lt;/em&gt; (the number of correct predictions over the total number of predictions), but there are many other performance measures, such as the precision, recall, f1 score, AUC, etc., that emphasize different behavior that you expect from your model.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-14T20:01:28.423" LastActivityDate="2020-06-14T20:01:28.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21887" PostTypeId="5" CreationDate="2020-06-14T22:13:22.283" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-14T22:13:22.283" LastActivityDate="2020-06-14T22:13:22.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21888" PostTypeId="4" CreationDate="2020-06-14T22:13:22.283" Score="0" Body="For questions related to the VGG neural networks, which were proposed in &quot;Very Deep Convolutional Networks for Large-Scale Image Recognition&quot; (2015) by Karen Simonyan and Andrew Zisserman." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-14T22:13:22.283" LastActivityDate="2020-06-14T22:13:22.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21889" PostTypeId="2" ParentId="21885" CreationDate="2020-06-15T02:20:51.940" Score="0" Body="&lt;p&gt;I think so you are new to GANs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the name itself implies, the generator task is to generate new images to match the distribution of the model and the input data, hence it is the generator that contributes in the generation of the data, while the discriminator just tries to differentiate between the generated images by the generator and real data.&lt;/p&gt;&#xA;" OwnerUserId="37648" LastEditorUserId="2444" LastEditDate="2020-06-15T10:04:51.797" LastActivityDate="2020-06-15T10:04:51.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21891" PostTypeId="1" CreationDate="2020-06-15T05:48:08.483" Score="0" ViewCount="68" Body="&lt;p&gt;I am able to train 345M parameter GPT-2 using &lt;a href=&quot;https://github.com/microsoft/DialoGPT,&quot; rel=&quot;nofollow noreferrer&quot;&gt;DialoGPT&lt;/a&gt; on Reddit data topics like AskReddit, Askmen, AskWomen, casualConvo, etc. And I using AWS p3dn.24xlarge instance with 256GB GPU. It is trained successfully. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I want to train a GPT-2 model on 1.5 billion parameters with 38GB of preprocessed data of Reddit, Blogspot, and other news sites. It is not getting trained on p3dn.24xlarge instance single instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how much amount of GPU do I need to train a DialoGPT2 with 1.5 billion parameters? Are there any techniques to train on Low 256GB GPU using some optimization technique?&lt;/p&gt;&#xA;" OwnerUserId="32861" LastEditorUserId="2444" LastEditDate="2020-06-15T10:10:15.503" LastActivityDate="2020-06-15T10:10:15.503" Title="What are the training and optimization technique to train GPT-2 with 1.5B parameters?" Tags="&lt;training&gt;&lt;transformer&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21893" PostTypeId="1" CreationDate="2020-06-15T06:27:58.247" Score="1" ViewCount="27" Body="&lt;p&gt;I have a chromosome where each gene contain s set of values. Like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;chromosome = [[A,B,C],[C,B,A],[C,D,],[],[E,F]]&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The order in each gene values matters. (A,B,C is different to A,C,B)&lt;/li&gt;&#xA;&lt;li&gt;Each value should not appear more than once in a gene. ([A,B,B] is not desirable, B is repeated.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In my current two-point crossover method. The genes values that are crossover is the whole set of values. (E.g the whole of [A,B,C] is crossed to another chromosome)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Soon, I realize my population lacks variations very quickly because the values within a gene always remain the same. Hence, my algorithm is evolving very slowly, and limited by the variation of gene values at initialization stage.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What crossover can I implement to cross values within the set as well? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty new to genetic algorithm. Any help will be much appreciated. Thank you. &lt;/p&gt;&#xA;" OwnerUserId="37898" LastActivityDate="2020-06-15T06:27:58.247" Title="Crossover method for gene value containing a set of values" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;crossover-operators&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21894" PostTypeId="2" ParentId="21873" CreationDate="2020-06-15T07:27:37.527" Score="1" Body="&lt;p&gt;DQN is famous for doing over-approximation on Q function. However, having over approximated Q does not imply that it does not perform well in the environment. (unless it looks ridiculously high) From my experience, high learning rate usually cause over approximated Q, or mistakes made in the code. Best way to check is to see plot of Q function when running on environment to see if it makes any sense. (e.g. It should go lower for bad states and vice versa) &lt;/p&gt;&#xA;" OwnerUserId="37899" LastActivityDate="2020-06-15T07:27:37.527" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21895" PostTypeId="2" ParentId="21881" CreationDate="2020-06-15T07:39:45.950" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;To solve this, I now train batches of models (~ 10 models per batch), which are then used in group as a new opponent,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems quite a reasonable approach on the surface, but possibly the agents will still lose generalisation if the solutions in each generation are too similar. It also looks like from your experiment that learning progress is too slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One simple thing you could do is progress through the generations faster. You don't need to train until agents win 90% of games before upping the generation number. Yuo could set the target as low as 60% or even 55%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For generalisation, it may also help to train against a mix of previous generations. E.g. if you use ten opponents, have five from previous generation, two from each of two iterations before that, and one even older one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the setup you have created plays an agent you are training against another agent that you have created, it is not quite self-play. In self-play, an agent plays against itself, and learns as both players simultaneously. This requires a single neural network function that can switch its evaluation to score for each player - you can either make it learn to take the current player into account and make the change in viewpoint itself, or in zero-sum games (which Connect 4 is one) it can be more efficient to have it evaluate the end result for player 1 and simply take the negative of that as the score for player 2. This is also equivalent to using &lt;span class=&quot;math-container&quot;&gt;$\text{max}_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}_a$&lt;/span&gt; for player 1's action choices and  &lt;span class=&quot;math-container&quot;&gt;$\text{min}_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{argmin}_a$&lt;/span&gt; for player 2's action choices - applying the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt; to Q learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can take minimax further to improve your algorithm's learning rate and performance during play. Essentially what Q learning and self-play does is learn a &lt;a href=&quot;https://en.wikipedia.org/wiki/Heuristic_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;heuristic&lt;/a&gt; for each state (or state/action pair) that can guide search. You can add search algorithms to your training and play in multiple ways. One simple approach during training is to perform some n-step look ahead using &lt;a href=&quot;https://en.wikipedia.org/wiki/Negamax#Negamax_with_alpha_beta_pruning&quot; rel=&quot;nofollow noreferrer&quot;&gt;negamax with alpha-beta pruning&lt;/a&gt; (an efficient variant of minimax in zero-sum games), and if it finds the end of the game:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;when training, use the result (win/draw/lose) as your ground truth value instead of the normal Q-learning TD target.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;when evaluating/playing vs human, prefer the action choice over anything the Q function returns. In practice, only bother with the Q function if look-ahead search cannot find a result.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the last few months, &lt;a href=&quot;https://www.kaggle.com/c/connectx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle have been running a &quot;Connect X&quot; challenge&lt;/a&gt; (which is effectively only Connect 4 at the moment). The forums and example scripts (called &quot;Kernels&quot;) are a good source of information for writing your own agents, and if you choose to compete, then the leaderboard should give you a sense for how well your agent is performing. The top agents are perfect players, as &lt;a href=&quot;https://www.youtube.com/watch?v=yDWPi1pZ0Po&quot; rel=&quot;nofollow noreferrer&quot;&gt;Connect 4 is a solved game&lt;/a&gt;. I am taking part in that competition, and have trained my agent using self-play Q-learning plus negamax search as above - it is not perfect, but is close enough that it can often beat a perfect playing opponent when playing as player 1. It was trained on around 100,000 games of self-play as I described above, plus extra training games versus previous agents.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I know the most successful techniques (e.g. alpha zero) rely on MCTS, but I'm not sure how to integrate this with Q-learning? Neither how MCTS helps to solve the problem of generalisation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;MCTS is a variant of search algorithm, and could be combined with Q-learning similarly to negamax, although in Alpha Zero it is combined with something more like Actor-Critic. The combination would be similar - from each position in play use MCTS to look ahead, and instead of picking the direct action with the best Q value, pick the one with the best MCTS score. Unlike negamax, MCTS is stochastic, but you can still use its evaluations as ground truth for training.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MCTS does not solve generalisation issues for neural networks, but like negamax it will improve the performance of a game-playing agent by looking ahead. Its main advantage over negamax in board games is a capability to scale to large branching factors. MCTS does work well for Connect 4. Some of the best agents in the Kaggle competition are using MCTS. Howver, it is not necessary for creating a &quot;superhuman&quot; Connect 4 agent, Q-learning plus negamax can do just as well.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-15T07:54:28.163" LastActivityDate="2020-06-15T07:54:28.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21896" PostTypeId="2" ParentId="21881" CreationDate="2020-06-15T07:41:39.847" Score="0" Body="&lt;p&gt;MCTS does not help with generalization directly, but it enables the agent to plan ahead (see depth-first search or breadth-first search). Having the state space search embedded in the algorithm is very important for playing zero sum games (we also plan ahead in our head when making moves right?). Now Q-learning is generally good for simple environments, but to achieve superhuman performance on board games you would need HUGE amounts of data without using any planning algorithm. I don't even know if practically achieving superhuman performance by only Q-learning is even possible.&lt;/p&gt;&#xA;" OwnerUserId="37899" LastActivityDate="2020-06-15T07:41:39.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21897" PostTypeId="1" AcceptedAnswerId="21904" CreationDate="2020-06-15T09:14:43.577" Score="1" ViewCount="167" Body="&lt;p&gt;The goal is to find an optimal deterministic policy for this MDP: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TegpT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TegpT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two possible policies: left (L) and right (R). What is the optimal policy, when different discounts are used: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.5$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi_* \ge \pi$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$v_{\pi^*}(s) \ge v_{\pi}(s), \forall s \in S$&lt;/span&gt;, so to find the optimal policy, the goal is to check which one of those results in the largest state value function for all states in the system given discount factors (A,B,C). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Bellman equation for the state value function is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v(s) = E_\pi[G_t | S_t= s] = E_\pi[R_{t+1} + \gamma v(S_{t+1}) | S_t = s]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The suffix &lt;span class=&quot;math-container&quot;&gt;$_n$&lt;/span&gt; marks the current iteration, and &lt;span class=&quot;math-container&quot;&gt;$_{n+1}$&lt;/span&gt; marks the next iteration. The following is valid if the value function is initialized to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or some random &lt;span class=&quot;math-container&quot;&gt;$x \ge 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A) &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{L,n+1}(S_0) = 1 + 0 v_{L,n}(S_L) = 1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{R,n+1}(S_0) = 0 + 0 v_{R,n}(S_R) = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is optimal in case A. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;B) &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{L,n+1}(S_0) = 1 + 0.9 v_{L,n}(S_L) = 1 + 0.9(0 + 0.9 v_{L,n}(S_0)) = 1 + 0.81v_{L,n}(S_0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{R,n+1}(S_0) = 0 + 0.9 v_{R,n}(S_R) = 0 + 0.9(2 + 0.9 v_{R,n}(S_0)) = 1.8 + 0.81v_{R,n}(S_0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is optimal in case B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C) &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.5$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{L,n+1}(S_0) = 1 + 0.5 v_{L,n}(S_L) = 1 + 0.5(0 + 0.9 v_{L,n}(S_0)) = 1 + 0.45v_{L,n}(S_0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{R,n+1}(S_0) = 0 + 0.5 v_{R,n}(S_R) = 0 + 0.5(2 + 0.9 v_{R,n}(S_0)) = 1 + 0.45v_{R,n}(S_0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; are optimal in case C. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Is this correct? &lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="37627" LastEditDate="2020-06-15T10:24:30.953" LastActivityDate="2020-06-15T12:57:18.727" Title="Solution to exercise 3.22 in the RL book by Sutton and Barto" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21898" PostTypeId="1" AcceptedAnswerId="21908" CreationDate="2020-06-15T09:41:14.660" Score="1" ViewCount="58" Body="&lt;p&gt;While transitioning from simple policy gradient to the actor-critic algorithm, most sources begin by replacing the &quot;reward to go&quot; with the state-action value function (see &lt;a href=&quot;http://rail.eecs.berkeley.edu/deeprlcourse-fa17/f17docs/lecture_5_actor_critic_pdf.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this slide 5&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not able to understand how this is mathematically justified. It seems intuitive to me that the &quot;reward to go&quot; when sampled through multiple trajectories should be estimated by the state-value function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel this way since nowhere in the objective function formulation or resulting gradient expression do we tie down the first action after reaching a state. Alternatively, when we sample a bunch of trajectories, these trajectories might include different actions being taken from the state reached in timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, why isn't the estimation/approximation for the &quot;reward to go&quot; the state value function, in which the expectation is also over all the actions that may be taken from that state as well?&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="2444" LastEditDate="2020-10-10T15:53:27.780" LastActivityDate="2020-10-10T15:53:27.780" Title="Why is the &quot;reward to go&quot; replaced by Q instead of V, when transitioning from PG to actor critic methods?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;reinforce&gt;&lt;advantage-actor-critic&gt;&lt;reward-to-go&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21899" PostTypeId="1" AcceptedAnswerId="21901" CreationDate="2020-06-15T09:53:13.960" Score="2" ViewCount="261" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.kdnuggets.com/2018/08/named-entity-recognition-practitioners-guide-nlp-4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Named entity recognition (NER)&lt;/a&gt;, also known as entity chunking/extraction, is a popular technique used in information extraction to identify and segment the named entities and classify or categorize them under various predefined classes.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Briefly, how does NER work? What are the main ideas behind it? And which algorithms are used to perform NER? &lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-06-15T10:30:10.490" LastActivityDate="2020-06-15T10:53:58.740" Title="What are the main ideas behind NER?" Tags="&lt;natural-language-processing&gt;&lt;named-entity-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21900" PostTypeId="2" ParentId="21875" CreationDate="2020-06-15T10:06:13.400" Score="1" Body="&lt;p&gt;Sounds more like a optimization problem than a deep learning / machine learning problem to me.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For machine learning you would have the features of every child / vehicle &lt;em&gt;and&lt;/em&gt; the optimal amount of pizza / fuel already given, but you don't know &lt;em&gt;how&lt;/em&gt; exactly the optimal amount is computed. So the target is to find a function which maps features to target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in your case you don't know the optimal value, you just have a number of constraints. So my suggestion would be to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear programming&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an simple example: drive the maximum distance with a given amount of fuel&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{max }\ m_1 \cdot x_1 + ... + m_n \cdot x_n$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{s.t. }\ x_1 + ... + x_n \leq F$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$m_i \cdot x_i \geq T_i\  \forall\  1 \leq i \leq n$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is the amount of fuel (in gallons) given to car &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m_i$&lt;/span&gt;'s are miles-per-gallon for car &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is the total amount of fuel available, this is our first constraint, we cannot distribute more fuel than we have. The second constraint say's that we like to drive at least &lt;span class=&quot;math-container&quot;&gt;$T_i$&lt;/span&gt; miles with car &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. This ensures we do not just give all the fuel to the most efficient car.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can come up with more constraints for drivers competency, engine horse power etc. In order so solve your problem with linear programming, you just have to make sure it is still linear and convex. Problems of this kind can be solved using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Simplex_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simplex algorithm&lt;/a&gt; (&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.optimize.linprog&lt;/a&gt;).&lt;br&gt;&#xA;If your objective or your constraints are more complex you can use numerical methods for non-linear &lt;a href=&quot;https://en.wikipedia.org/wiki/Constrained_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;constrained optimization&lt;/a&gt; problems (see &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.optimize&lt;/a&gt; for algorithms).&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="37120" LastEditDate="2020-06-15T10:47:39.710" LastActivityDate="2020-06-15T10:47:39.710" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21901" PostTypeId="2" ParentId="21899" CreationDate="2020-06-15T10:11:39.280" Score="3" Body="&lt;p&gt;There are different algorithms, each with their advantages and disadvantages.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Gazetteers: these have lists of the entities to be recognised, eg list of countries, cities, people, companies, whatever is required. They typically use a fuzzy matching algorithm to capture cases where the entity is not written in exactly the same way as in the list. For example, &lt;em&gt;USA&lt;/em&gt; or &lt;em&gt;U.S.A.&lt;/em&gt;, &lt;em&gt;United States&lt;/em&gt;, &lt;em&gt;United States of America&lt;/em&gt;, &lt;em&gt;US of A&lt;/em&gt;, etc. Advantage: generally good precision, ie can identify known entities. Disadvantage: can only find known entities&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Contextual Clues: here you have patterns that you find in the text, eg &lt;em&gt;[PERSON], chairman of [COMPANY]&lt;/em&gt;. In this case, sentences like &lt;em&gt;Jeff Bezos, chairman of Amazon,&lt;/em&gt; will match, even if you have never come across either Bezos or Amazon. Advantage: can find entities you didn't know about. Disadvantages: could end up with false positives, might be quite labour-intensive to come up with patterns; patterns depend on the domain (newspapers vs textbooks vs novels etc_&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Structural description: this is basically a 'grammar' describing what your entities look like, eg (in some kind of pseudo-regex): &lt;code&gt;title? [first_name|initial] [middle_name|initial]? surname&lt;/code&gt; would match &quot;Mr J. R. Ewing&quot; or &quot;Bob Smith&quot;. Similar descriptions could match typical company names; you'd probably still need lists of possible surnames or firstnames. Advantages: some flexibility, and potentially good precision. Disadvantages: patterns need to be developed and maintained.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Ideally you would want to combine all three approaches for a hybrid one to get the advantages of recognising unknown entities while keeping excess false positives in check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might also be other machine-learning approaches, but I'm not too familiar with those. The main problem is that they are hard to fine-tune or work out why they do what they do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; A good starting point would be to use a gazetteer-based approach to annotate some training data, and use that to identify contextual patterns. You can then use that data to train a machine learning approach (see OmG's answer on CRF) to broaden the approach; and then you add newly recognised entities to your list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally you would want to have a gazette as your main database to avoid false positives, and use machine-learning or contextual patterns to just capture previously unseen entities.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-06-15T10:53:58.740" LastActivityDate="2020-06-15T10:53:58.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21902" PostTypeId="2" ParentId="21899" CreationDate="2020-06-15T10:11:55.397" Score="3" Body="&lt;p&gt;One of the renowned learning algorithms for NER tagging is the conditional random field (&lt;a href=&quot;https://medium.com/data-science-in-your-pocket/named-entity-recognition-ner-using-conditional-random-fields-in-nlp-3660df22e95c&quot; rel=&quot;nofollow noreferrer&quot;&gt;CRF&lt;/a&gt;). As you can see in the provided link, s&lt;strong&gt;equence labeling&lt;/strong&gt; algorithms such as RNN with LSTM‌ can be used to named entity recognition as well. By the way, you can find an implementation of the CRF for NER tagging in &lt;a href=&quot;https://www.depends-on-the-definition.com/named-entity-recognition-conditional-random-fields-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this source&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that, the method of providing training data can be helpful to pass the data into the standard libraries of CRF without any extra preprocessing. One of the standard methods is BIO method(B (Begin), I (Interior), and E (End)). You can find more about it in &lt;a href=&quot;https://datascience.stackexchange.com/a/63400/27665&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-06-15T10:21:03.363" LastActivityDate="2020-06-15T10:21:03.363" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21904" PostTypeId="2" ParentId="21897" CreationDate="2020-06-15T11:38:13.493" Score="2" Body="&lt;p&gt;Your answer is correct but I am not sure exactly on how you arrived at it, as e.g. in the last case you don't know that &lt;span class=&quot;math-container&quot;&gt;$v_{L,n}(S_0) = v_{R,n}(S_0)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will show for case B when &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt; as case A is trivial and hopefully you can apply what I've done in case B to case C so that you get exact answers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as you stated &lt;span class=&quot;math-container&quot;&gt;$v(s) = \mathbb{E}[R_{t+1} + \gamma v(S_{t+1}) | S_t = s]$&lt;/span&gt;. Assuming that &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt; we can calculate the values for each state under the policy of taking the left action. Note that because we are looking for deterministic policies and the environment is deterministic then a lot of the expectations can be disregarded as nothing random is happening. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;'Left Policy'&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}v(s_0) &amp;amp;= 1 + 0.9 \times v(s_L) \\ v(s_L) &amp;amp;= 0 + 0.9 \times v(s_0) \\ v(s_R) &amp;amp;= 2 + 0.9 \times v(s_0)&#xA;\end{align}&lt;/span&gt;&#xA;We can solve this set of linear equations to get &lt;span class=&quot;math-container&quot;&gt;$v(s_0) = \frac{100}{19}, v(s_L) = \frac{90}{19}, v(s_R) = \frac{128}{19}\;.$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;'Right Policy'&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}v(s_0) &amp;amp;= 0 + 0.9 \times v(s_R) \\ v(s_L) &amp;amp;= 0 + 0.9 \times v(s_0) \\ v(s_R) &amp;amp;= 2 + 0.9 \times v(s_0)&#xA;\end{align}&lt;/span&gt;&#xA;We can again solve these to obtain &lt;span class=&quot;math-container&quot;&gt;$v(s_0) = \frac{180}{19}, v(s_L) = \frac{162}{19}, v(s_R) = \frac{200}{19}\;.$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we can see, for each of the states the value function is larger for &lt;em&gt;all&lt;/em&gt; of the states under the policy 'go right', thus this is the optimal policy for the case of &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to note that if we take the 'left' action in state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; then our policy would never take us to state &lt;span class=&quot;math-container&quot;&gt;$s_R$&lt;/span&gt;, and the same for the right action and state &lt;span class=&quot;math-container&quot;&gt;$s_L$&lt;/span&gt;, however due to the definition of an optimal policy requiring &lt;span class=&quot;math-container&quot;&gt;$v_{\pi ^*}(s) \geq v_{\pi}(s)\; \forall s \in \mathcal{S}$&lt;/span&gt; then we must evaluate the value function for all states, even ones that would not be visited under a policy you are evaluating. This means that the state for &lt;span class=&quot;math-container&quot;&gt;$s_R$&lt;/span&gt; will change whether we go right or left, because the value of this state depends on the value of &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, which clearly changes depending on whether we go right or left. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-15T12:57:18.727" LastActivityDate="2020-06-15T12:57:18.727" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21905" PostTypeId="1" CreationDate="2020-06-15T13:07:04.467" Score="1" ViewCount="32" Body="&lt;p&gt;I am working in the field of Machine Vision, where accuracy and performance both play a major factor in deciding the approach towards a problem. Traditional rule based approaches work quite well in such cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am gradually migrating towards deep learning, due to its umpteen advantages, where the results seem promising albeit with two huge caveats:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Lack of Training data in this field. To be precise, the lack of erroneous data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Performance issues on inference. Accuracy and speed are required in equal proportion, and cannot be compromised.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In industrial settings, Point 1 plays a strong factor. I have been dabbling with Transfer learning techniques and using pre-trained models to overcome this situation. For simpler applications such as classification, this suits and gives good results. In other cases such as detection and localization, I have tried using MaskRCNN, which gives really good results but poor inference speed, means it is not production-ready.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The worrying factor in both cases is how slow detection and inference is, compared to traditional vision algorithms. A solution would be to buy Machine Vision software specifically from companies such as Cognex, HALCON, etc, who sell deep learning bundles. They are quite expensive and are to be used out- of- box with minimal modifications, which does not suit me currently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Point 2, is highly necessary in production lines, where each iteration/image may take less than 500ms for execution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deep Learning gives a lot of opportunities in getting state of the art results with very less data in most of the situations, but in general without inference optimization in using apps such as TensorRT, the &quot;time&quot; metric does not give good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an approach in using open source that can solve both point 1 and point 2? Creating a CNN from scratch is out of the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This post is to discuss ideas if possible, I know a concrete solution is not really possible in the scope of this question. I am the only person working on this problem at my company, thus any discussion ideas would be highly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="37906" LastEditorUserId="2444" LastEditDate="2020-06-16T10:08:29.423" LastActivityDate="2020-06-16T10:08:29.423" Title="Overcome caveats on using Deep Learning for faster inference on limited performance availability" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21908" PostTypeId="2" ParentId="21898" CreationDate="2020-06-15T15:55:51.393" Score="0" Body="&lt;p&gt;When you say simple policy gradient, I assume you mean something like REINFORCE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference between actor-critic and REINFORCE-like algorithms is in how they estimate the reward to go: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;in REINFORCE, you wait until a trajectory terminates to make any updates, and your estimator of the reward to go is the actual reward to go that was observed in the trajectory. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;in actor critic algorithms, instead of using an entire trajectory to estimate a reward to go, you use the action value function (note that this way, you can compute updates at every time step). The reason why you use the action-value function is because at each transition, you have already committed to some action. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The benefit of the actor critic estimator is that it exhibits much less variance. REINFORCE estimators are Monte Carlo estimators, which are known to exhibit extremely high variance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the actor critic method in the slides you posted takes variance reduction one step further. Instead of estimating reward to go, it's estimating &lt;em&gt;advantage&lt;/em&gt; -- that is, the difference between the current action value and your estimated state value. The state value function term doesn't depend on the action, so it doesn't affect the expected value of the policy gradient. However, it serves as a 'baseline', which helps reduce the variance of the reward to go estimator even further.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-15T15:55:51.393" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21910" PostTypeId="1" AcceptedAnswerId="21912" CreationDate="2020-06-15T17:43:49.250" Score="1" ViewCount="153" Body="&lt;p&gt;From Sutton and Barto's book Reinforcement Learning (Adaptive Computation and Machine Learning series), are the following definitions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PIJ5B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PIJ5B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qwGI3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qwGI3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To aid my learning of RL and gain an intuition, I'm focusing on the differences between some algorithms. I've selected &lt;code&gt;Sarsa (on-policy TD control) for estimating Q ≈ q *&lt;/code&gt; and &lt;code&gt;Q-learning (off-policy TD control) for estimating π ≈ π *&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For conciseness I'll refer to &lt;code&gt;Sarsa (on-policy TD control) for estimating Q ≈ q *&lt;/code&gt; and &lt;code&gt;Q-learning (off-policy TD control) for estimating π ≈ π *&lt;/code&gt; as Sarsa and Q-learning respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are my following assertions correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary differences are how the Q values are updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sarsa Q value update:&#xA;&lt;span class=&quot;math-container&quot;&gt;$ Q ( S, A ) ← Q ( S, A ) + α [ R + \gamma Q ( S ′ , A ′ ) − Q ( S, A ) ] $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-learning Q value update:&#xA;&lt;span class=&quot;math-container&quot;&gt;$ Q ( S, A ) ← Q ( S, A ) + α [ R + \gamma \max_a Q ( S ′ , a ) − Q ( S, A ) ] $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sarsa, in performing the td update subtracts the discounted Q value of the next state and action, S', A' from the Q value of the current state and action S, A. &#xA;Q-learning, on the other hand, takes the discounted difference between the max action value for the Q value of the next state and current action S', a.  Within the Q-learning episode loop the &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; value is not updated, is an update made to &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; during Q-learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sarsa, unlike Q-learning, the current action is assigned to the next action at the end of each episode step. Q-learning does not assign the current action to the next action at the end of each episode step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sarsa, unlike Q-learning, does not include the arg max as part of the update to Q value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sarsa and Q learning in choosing the initial action for each episode both use a &quot;policy derived from Q&quot;, as an example, the epsilon greedy policy is given in the algorithm definition. But any policy could be used here instead of epsilon greedy?&#xA;Q learning does not utilise the next state-action pair in performing the td update, it just utilises the next state and current action, this is given in the algorithm definition as  &lt;span class=&quot;math-container&quot;&gt;$ Q ( S ′ , a ) $&lt;/span&gt; what is &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in this case ?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="2444" LastEditDate="2020-06-15T19:26:21.887" LastActivityDate="2020-06-15T19:28:32.940" Title="What are the differences between SARSA and Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21911" PostTypeId="1" AcceptedAnswerId="25635" CreationDate="2020-06-15T18:56:54.630" Score="3" ViewCount="327" Body="&lt;p&gt;In the textbook &quot;Reinforcement Learning: An Introduction&quot; by Richard Sutton and Andrew Barto, the concept of Maximization Bias is introduced in section 6.7, and how Q-learning &quot;over-estimates&quot; action-values is discussed using an example. However, a formal proof of the same is not presented in the textbook, and I couldn't get it anywhere on the internet as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading the paper on Double Q-learning by Hado van Hasselt (&lt;a href=&quot;https://papers.nips.cc/paper/3964-double-q-learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;), I could understand to some extent why Q-learning &quot;over-estimates&quot; action values. Here is my (vague, informal) construction of a mathematical proof:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that Temporal Methods (just like Monte Carlo methods), use sample returns instead of real expected returns as estimates, to find the optimal policy. These sample returns converge to the true expected returns over infinite trials, provided all the state-action pairs are visited. Thus the following notation is used, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[Q()] \rightarrow q_\pi()$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Q()$&lt;/span&gt; is calculated from the sample return &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; observed at every time-step. Over infinite trials, this sample return when averaged converges to it's expected value which is the true &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value under the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. Thus &lt;span class=&quot;math-container&quot;&gt;$Q()$&lt;/span&gt; is really an estimate of the true &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value &lt;span class=&quot;math-container&quot;&gt;$q_\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section 3 on page 4 of the paper, Hasselt describes how the quantity &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(s_{t+1}, a)$&lt;/span&gt; approximates &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[\max_a Q(s_{t+1}, a)]$&lt;/span&gt; which in turn approximates the quantity &lt;span class=&quot;math-container&quot;&gt;$\max_a(\mathbb{E}[Q(s_{t+1},a)])$&lt;/span&gt; in Q-learning. Now, we know that the &lt;span class=&quot;math-container&quot;&gt;$\max[]$&lt;/span&gt; function is a convex function (&lt;a href=&quot;https://math.stackexchange.com/questions/2435464/show-that-max-function-on-mathbb-rn-is-convex&quot;&gt;proof&lt;/a&gt;). From Jensen's inequality, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\phi(\mathbb{E}[X]) \leq \mathbb{E}[\phi(X)]$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a random variable, and the function &lt;span class=&quot;math-container&quot;&gt;$\phi()$&lt;/span&gt; is a convex function. Thus,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\max_a(\mathbb{E}[Q(s_{t+1},a)]) \leq \mathbb{E}[\max_a(Q(s_{t+1}, a)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\therefore \max_a Q(s_{t+1}, a) \approx \max_a(\mathbb{E}[Q(s_{t+1},a)]) \leq \mathbb{E}[\max_a(Q(s_{t+1}, a)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quantity on the LHS of the above equation appears (along with &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;) as an estimate of the next action-value in the Q-learning update equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(S_t,A_t) \leftarrow (1-\alpha)Q(S_t, A_t) + \alpha[R_{t+1} + \gamma\max_aQ(S_{t+1}, a)] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, we note that the bias of an estimate &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is given by:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$b(T) = \mathbb{E}[T] - T$$&lt;/span&gt;&#xA;Thus the bias of the estimate &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(s_{t+1},a)$&lt;/span&gt; will always be positive:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$b(\max_a Q(s_{t+1},a)) = \mathbb{E}[\max_a Q(s_{t+1},a)] - \max_a Q(s_{t+1},a) \geq 0$$&lt;/span&gt;&#xA;In statistics literature, any estimate whose bias is positive is said to be an &quot;over-estimate&quot;. Thus the action values are over-estimated by the Q-learning algorithm due to the &lt;span class=&quot;math-container&quot;&gt;$\max[]$&lt;/span&gt; operator, thus resulting in a &lt;span class=&quot;math-container&quot;&gt;$maximization$&lt;/span&gt;-&lt;span class=&quot;math-container&quot;&gt;$bias$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the arguments made above valid? I am a student, with no rigorous knowledge of random processes. Thus, please forgive me if any of the steps above are totally unrelated, and doesn't make sense in a more mathematically rigorous fashion. Please let me know, if there is a much better proof than this failed attempt. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much for your precious time. Any help/suggestions/corrections are greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="37181" LastEditorUserId="37181" LastEditDate="2020-06-16T09:47:42.390" LastActivityDate="2021-01-19T11:22:18.193" Title="Proof of Maximization Bias in Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;proofs&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21912" PostTypeId="2" ParentId="21910" CreationDate="2020-06-15T19:17:49.857" Score="1" Body="&lt;p&gt;The main difference between the two is that Q-learning is an &lt;em&gt;off policy&lt;/em&gt; algorithm. That is, we learn about an policy that is different to the one we choose to make actions. To see this, lets look at the update rule. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Q-Learning&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = Q(s,a) + \alpha (R_{t+1} + \gamma \max_aQ(s',a) - Q(s,a))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;SARSA&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = Q(s,a) + \alpha (R_{t+1} + \gamma Q(s',a') - Q(s,a))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SARSA we chose our &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; according to what our policy tells us to do when we are in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, so the policy we are learning about is also the policy that we choose to make our actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Q-learning, we learn about the greedy policy whilst following some other policy, such as &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy. This is because when we transition into state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; our TD-target becomes the maximum Q-value for whichever state we end up in, &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, where the max is taken over the actions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we have actually updated our Q-function and we need to choose an action to take in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, we do so from the policy we are using to generate our actions from -- thus we are learning about the greedy policy whilst following some other policy, hence &lt;em&gt;off policy&lt;/em&gt;. In SARSA when we move into &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; our TD-target is chosen by the Q-value for the state we transition into and then the action we would choose based on our policy. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Within the Q-learning episode loop the &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; value is not updated, is an update made to &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; during Q-learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It will be, because the policy we use to choose our actions is ensured to explore sufficiently around all of the state-action pairs and so it is guaranteed to be encountered at some point. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sarsa, unlike Q-learning, does not include the arg max as part of the update to Q value.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is not an &lt;span class=&quot;math-container&quot;&gt;$\arg \max$&lt;/span&gt;, it is a &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt;. This is defined as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\max_x f(x) = \{f(x) | \forall y\; : f(y) \leq f(x) \}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sarsa, unlike Q-learning, the current action is assigned to the next action at the end of each episode step. Q-learning does not assign the current action to the next action at the end of each episode step&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Kind of - the action that you chose for your TD-target in SARSA becomes the next action that you consider in the next step of the episode. This is natural because essentially you are in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, you take action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and observe a new state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, at which point you can use your policy to see which action you &lt;em&gt;will&lt;/em&gt; take, call this &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt;, and then perform the SARSA update, and then execute that action in the environment. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sarsa and Q learning in choosing the initial action for each episode both use a &quot;policy derived from Q&quot;, as an example, the epsilon greedy policy is given in the algorithm definition. But any policy could be used here instead of epsilon greedy? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, any policy can be used although you want to choose a policy that allows sufficient exploration of the state-space.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Q learning does not utilise the next state-action pair in performing&#xA;  the td update, it just utilises the next state and current action,&#xA;  this is given in the algorithm definition as &lt;span class=&quot;math-container&quot;&gt;$Q(S',a)$&lt;/span&gt; what is &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in&#xA;  this case ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;in the algorithm it actually has &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(S',a)$&lt;/span&gt;, which if you refer back to my earlier definition of what the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator does, should answer this question. &lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-15T19:28:32.940" LastActivityDate="2020-06-15T19:28:32.940" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21914" PostTypeId="1" AcceptedAnswerId="21915" CreationDate="2020-06-15T19:24:57.230" Score="1" ViewCount="509" Body="&lt;p&gt;In computer vision, what are bag-of-features (also known as bag-of-visual-words)? How do they work? What can they be used for? How are they related to the bag-of-words model in NLP?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-06-16T16:55:19.063" Title="What are bag-of-features in computer vision?" Tags="&lt;computer-vision&gt;&lt;feature-extraction&gt;&lt;features&gt;&lt;bag-of-features&gt;&lt;content-based-image-retrieval&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21915" PostTypeId="2" ParentId="21914" CreationDate="2020-06-15T19:24:57.230" Score="3" Body="&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/48190777_Introduction_to_the_Bag_of_Features_Paradigm_for_Image_Classificationand_Retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Bag-of-features&lt;/strong&gt;&lt;/a&gt; (BoF) (also known as &lt;strong&gt;bag-of-visual-words&lt;/strong&gt;) is a method to represent the features of images (i.e. a feature extraction/generation/representation algorithm). BoF is inspired by the bag-of-words model often used in the context of NLP, hence the name. In the context of computer vision, BoF can be used for different purposes, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Content-based_image_retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;content-based image retrieval (CBIR)&lt;/a&gt;, i.e. find an image in a database that is closest to a query image.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Steps&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The BoF can be divided into three different steps. To understand all the steps, consider a training dataset &lt;span class=&quot;math-container&quot;&gt;$D = \{x_1, \dots, x_N \}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; training images. Then BoF proceeds as follows.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;1. Feature extraction&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In this first step, we extract all the &lt;em&gt;raw&lt;/em&gt; features (i.e. keypoints and descriptors) from all images in the training dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. This can be done with &lt;a href=&quot;https://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT&lt;/a&gt;, where each descriptor is a &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt;-dimensional vector that represents the neighborhood of the pixels around a certain keypoint (e.g. a pixel that represents a corner of an object in the image). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not familiar with this extraction of &lt;em&gt;computer vision&lt;/em&gt; (sometimes known as &lt;em&gt;handcrafted&lt;/em&gt;) features, you should read the &lt;a href=&quot;https://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT paper&lt;/a&gt;, which describes a feature (more precisely, keypoint and descriptor) extraction algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that image &lt;span class=&quot;math-container&quot;&gt;$x_i \in D$&lt;/span&gt; may contain a different number of features (keypoints and descriptors) than image &lt;span class=&quot;math-container&quot;&gt;$x_j \neq x_i \in D$&lt;/span&gt;. As we will see in the third step, BoF produces a feature vector of size &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; for all images, so all images will be represented by a fixed-size vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$F= \{f_1, \dots, f_M\}$&lt;/span&gt; be the set of descriptors extracted from all training images in &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$M \gg N$&lt;/span&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt; may be a descriptor that belongs to any of the training examples (it does not matter which training image it belongs to).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2. Codebook generation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In this step, we cluster all descriptors &lt;span class=&quot;math-container&quot;&gt;$F= \{f_1, \dots, f_M\}$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; clusters using k-means (or another clustering algorithm). This is sometimes known as the &lt;em&gt;vector quantization&lt;/em&gt; (VQ) step. In fact,  the idea behind VQ is very similar to clustering and sometimes VQ is used interchangeably with clustering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, after this step, we will have &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; clusters, each of them associated with a centroid &lt;span class=&quot;math-container&quot;&gt;$C = \{ c_1, \dots, c_k\}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the set of centroids (and &lt;span class=&quot;math-container&quot;&gt;$c_i \in \mathbb{R}^{128}$&lt;/span&gt; in the case that SIFT descriptors have been used). These centroids represent the main features that are present in the whole training dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. In this context, they are often known as the &lt;strong&gt;codewords&lt;/strong&gt; (which derives from the vector quantization literature) or &lt;strong&gt;visual words&lt;/strong&gt; (hence the name &lt;em&gt;bag-of-visual-words&lt;/em&gt;). The set of codewords &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is often called &lt;strong&gt;codebook&lt;/strong&gt; or, equivalently, the &lt;strong&gt;visual vocabulary&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;3. Feature vector generation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In this last step, given a new (test) image &lt;span class=&quot;math-container&quot;&gt;$u \not\in D$&lt;/span&gt; (often called the &lt;em&gt;query image&lt;/em&gt; in this context of CBIR), then we will represent &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; as a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-dimensional vector (where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, if you remember, is the number of codewords) that will represent its &lt;em&gt;feature vector&lt;/em&gt;. To do that, we need to follow the following steps.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Extract the raw features from &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; with e.g. SIFT (as we did for the training images). Let the descriptors of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; be &lt;span class=&quot;math-container&quot;&gt;$U = \{ u_1, \dots, u_{|U|} \}$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a vector &lt;span class=&quot;math-container&quot;&gt;$I \in \mathbb{R}^k$&lt;/span&gt; of size &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; filled with zeros, where the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th element of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; corresponds to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th codeword (or cluster).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each &lt;span class=&quot;math-container&quot;&gt;$u_i \in U$&lt;/span&gt;, find the &lt;em&gt;closest&lt;/em&gt; codeword (or centroid) in &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. Once you found it, increment the value at the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th position of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; (i.e., initially, from zero to one), where &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is the found closest codeword to the descriptor &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt; of the query image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distance between &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt; and any of the codewords can be computed e.g. with the Euclidean distance. Note that the descriptors of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and the codewords have the same dimension because they have been computed with the same feature descriptor (e.g. SIFT).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of this process, we will have a vector &lt;span class=&quot;math-container&quot;&gt;$I \in \mathbb{R}^k$&lt;/span&gt; that represents the frequency of the codewords in the query image &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; (akin to the &lt;em&gt;term frequency&lt;/em&gt; in the context of the bag-of-words model), i.e. &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;'s feature vector. Equivalently, &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; can also be viewed as a histogram of features of the query image &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;. Here's an illustrative example of such a histogram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rx3XK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rx3XK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this diagram, we can see that there are &lt;span class=&quot;math-container&quot;&gt;$11$&lt;/span&gt; codewords (of course, this is an unrealistic scenario!). On the y-axis, we have the frequency of each of the codewords in a given image. We can see that the &lt;span class=&quot;math-container&quot;&gt;$7$&lt;/span&gt;th codeword is the most frequent in this particular query image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, rather than the codeword frequency, we can use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tf-idf&lt;/a&gt;. In that case, each image will be represented not by a vector that contains the frequency of the codewords but it will contain the frequency of the codewords weighted by their presence in other images. See &lt;a href=&quot;https://www.researchgate.net/publication/48190777_Introduction_to_the_Bag_of_Features_Paradigm_for_Image_Classificationand_Retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for more details (where they show how to calculate tf-idf in this context; specifically, section 4.1, p. 8 of the paper).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To conclude, BoF is a method to represent features of an image, which could then be used to train classifiers or generative models to solve different computer vision tasks (&lt;a href=&quot;http://vision.stanford.edu/documents/Fei-FeiPerona2005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;such as CBIR&lt;/a&gt;). More precisely, if you want to perform CBIR, you could compare your query's feature vector with the feature vector of every image in the database, e.g. using the cosine similarity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first two steps above are concerned with the creation of a &lt;strong&gt;visual vocabulary&lt;/strong&gt; (or codebook), which is then used to create the feature vector of a new test (or query) image.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A side note&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As a side note, the term &lt;em&gt;bag&lt;/em&gt; is used because the (relative) order of the features in the image is lost during this feature extraction process, and this can actually be a disadvantage.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For more info, I suggest that you read the following papers &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.robots.ox.ac.uk/~vgg/publications/papers/sivic03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video Google: A Text Retrieval Approach to Object Matching in Videos&lt;/a&gt; (2003) by Sivic and Zisserman&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://vision.stanford.edu/documents/Fei-FeiPerona2005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Bayesian Hierarchical Model for Learning Natural Scene Categories&lt;/a&gt; (2005) by Fei-Fei and Perona&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/48190777_Introduction_to_the_Bag_of_Features_Paradigm_for_Image_Classificationand_Retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to the Bag of Features Paradigm for Image Classification and Retrieval&lt;/a&gt; (2011) by O'Hara and Draper&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/258403856_Bag-of-Words_Representation_in_Image_Annotation_A_Review&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bag-of-Words Representation in Image Annotation: A Review&lt;/a&gt; (2012) by Tsai&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T16:55:19.063" LastActivityDate="2020-06-16T16:55:19.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21917" PostTypeId="1" CreationDate="2020-06-15T22:08:45.390" Score="0" ViewCount="332" Body="&lt;p&gt;I ran a test using 3 strategies for multi-armed bandit: UCB, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, and Thompson sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results for the rewards I got are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Thompson sampling had the highest average reward&lt;/li&gt;&#xA;&lt;li&gt;UCB was second&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy was third, but relatively close to UCB&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can someone explain to me why the results are like this?&lt;/p&gt;&#xA;" OwnerUserId="37918" LastEditorUserId="2444" LastEditDate="2020-06-16T11:41:11.667" LastActivityDate="2020-06-16T11:41:11.667" Title="Why am I getting better performance with Thompson sampling than with UCB or $\epsilon$-greedy in a multi-armed bandit problem?" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;thompson-sampling&gt;&lt;upper-confidence-bound&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="1" CommentCount="7" ClosedDate="2020-06-16T10:03:57.560" ContentLicense="CC BY-SA 4.0" />
  <row Id="21918" PostTypeId="1" CreationDate="2020-06-15T22:14:23.297" Score="0" ViewCount="17" Body="&lt;p&gt;I have an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional time-series to &lt;strong&gt;apply LSTM&lt;/strong&gt; to, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of features for each time point. These &lt;strong&gt;features can be clustered according to their concept&lt;/strong&gt;, for example &lt;span class=&quot;math-container&quot;&gt;$n_1, ..., n_4$&lt;/span&gt; are features related to let's say thermodynamic variables such as temperature and pressure, &lt;span class=&quot;math-container&quot;&gt;$n_5, ..., n_{10}$&lt;/span&gt; are features related to geographical variables and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what would be the &lt;strong&gt;best practice for designing an LSTM topology for this problem&lt;/strong&gt;? possible answers that comes to my mind are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use primary many-to-one LSTMs for each cluster of features, and feed the outputs to a Dense network &lt;/li&gt;&#xA;&lt;li&gt;use 1 single global LSTM (still many-to-&lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; being the number of labels) for all of the features, and let the model decide if it wants to decouple the implicit clusters of features &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the first method is simpler, with less trainable parameters, but with hard-coded features, and the second one, not simple and more parameters but still more able to capture complexities&lt;/p&gt;&#xA;" OwnerUserId="6258" LastActivityDate="2020-06-15T22:14:23.297" Title="Separated LSTMs or a global one for cluster of related features" Tags="&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;deep-neural-networks&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21920" PostTypeId="1" CreationDate="2020-06-15T22:50:51.853" Score="0" ViewCount="110" Body="&lt;p&gt;I am trying to do the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Install the OpenAI baseline algorithms from the following GitHub repository: github.com/openai/baselines by following the instructions in the readme file.&lt;/li&gt;&#xA;&lt;li&gt;Train an agent on the pendulum task by running DDPG: &quot;python -m baselines.ddpg.main --env-id Pendulum-&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Once I do that, I need to modify the DDPG code in baselines/ddpg/ to share the weights between the first layer of the actor network and the first layer of the critic network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I need to produce a graph where the y-axis is the average reward of each episode obtained by DDPG in the Pendulum-v0 environment and the x-axis is the # of episodes up to a maximum of 500 episodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure how to modify the DDPG code, as I am new to tensorflow. Can any one help me out here, or guide me as to how I can do that?&lt;/p&gt;&#xA;" OwnerUserId="37918" LastActivityDate="2020-06-15T22:50:51.853" Title="Using deep deterministic policy gradient in OpenAI Gym to solve problems with continuous actions" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;actor-critic-methods&gt;&lt;open-ai&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21923" PostTypeId="1" CreationDate="2020-06-16T05:17:16.167" Score="0" ViewCount="38" Body="&lt;p&gt;The simplest case of a deadlock:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;process 1             |  process 2&#xA;----------------------+------------------------&#xA;needs resources A, B  |  needs resources A, B&#xA;holds A               |  holds B&#xA;waits for B           |  waits for A&#xA;&#xA;          -=   D E A D L O C K   =-&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's possible to describe similar resource acquisition in the first order logic (FOL) in Prolog like some behavior of N processes and to verify is the scenario safe, i.e. it does not lead to a deadlock. I think it's possible to be done for race conditions too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made some simple experiments with &lt;a href=&quot;https://protege.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protege&lt;/a&gt; - it is an ontology editor which uses description logic (DL). The tool has reasoners (ELK, HermiT, jcel, Pellet, Fact++...), that can execute some logic: to infer some facts, to detect inconsistency... Is it possible to solve the same task with DL (let's say with Protege) i.e. to describe a behavior of N processes in relation to some resources and to verify that the behavior is safe (does not lead to deadlocks, race conditions) with DL reasoners? Or it's impossible even theoretically?&lt;/p&gt;&#xA;" OwnerUserId="37924" LastEditorUserId="37924" LastEditDate="2020-06-16T15:55:25.473" LastActivityDate="2020-06-16T15:55:25.473" Title="Is it possible to verify the processes/resources model for deadlocks with DL (Protege) as it's possible with FOL (Prolog)?" Tags="&lt;ontology&gt;&lt;prolog&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21925" PostTypeId="1" CreationDate="2020-06-16T07:26:47.960" Score="1" ViewCount="34" Body="&lt;p&gt;It is known that if &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is set to high, then the cost function of the model may not converge. &lt;a href=&quot;https://i.stack.imgur.com/KszUC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KszUC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &#xA;However, would a decaying of the learning rate provide some &quot;tuning&quot; of the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; value during training ? In the sense that if you set a high learning rate but you also have some form of learning rate decay, then eventually &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; value would fall within the &quot;just right&quot; and &quot;too low&quot; range eventually. Is it better to then set an initial learning rate that is more &quot;flexible&quot; in the higher ranges rather than a learning rate that is too low ?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-06-16T08:27:30.983" Title="Is it harmful to set the learning rate of training a model to be too high if there is some decay function for the learning rate?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;learning-rate&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21926" PostTypeId="1" CreationDate="2020-06-16T07:39:11.917" Score="0" ViewCount="87" Body="&lt;p&gt;I would like to build a model based on reinforcement learning (RL) for the following scenario&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Recommend the best route (of cities listed for a given country) that satisfies the required criteria (museum, beaches, food, etc) for a total budget of $2000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Based on the recommendation, the user will provide its feedback (as a reward), so the recommendations can be fine-tuned (by reinforcement learning) the next time. I modeled the system this way:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;States = (c,cr), where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the city and &lt;span class=&quot;math-container&quot;&gt;$cr$&lt;/span&gt; is the criteria (history, beach, food, etc)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Actions = (p) is the price of visiting the city&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reward: acceptance of the cities selected by end user as a route (1 or 0)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The objective is to decide which list of cities together satisfy the&#xA;given budget.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is this MDP model right and how can I implement this? May be the only option is using Monte Carlo methods and linear/dynamic programming.. Is there any other way?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="37930" LastEditorUserId="37930" LastEditDate="2020-06-17T14:55:32.380" LastActivityDate="2021-04-14T20:57:16.887" Title="How can I model this problem of delivering assets by choosing a route with reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;optimization&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21927" PostTypeId="1" AcceptedAnswerId="21935" CreationDate="2020-06-16T07:40:42.007" Score="1" ViewCount="33" Body="&lt;p&gt;I have 5 classes of pictures to classify:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 -&gt; ~&lt;strong&gt;3 200&lt;/strong&gt; (~800 initial number before interference and duplication)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 -&gt; ~&lt;strong&gt;9 000&lt;/strong&gt; (I reduced from ~90 000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 -&gt; ~&lt;strong&gt;8 000&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3 -&gt; ~&lt;strong&gt;3 000&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 -&gt; ~&lt;strong&gt;7 200&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to divide the data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have divided the data giving &lt;strong&gt;2 000&lt;/strong&gt; to test and &lt;strong&gt;2 000&lt;/strong&gt; to validation set &#xA;by taking a fixed number of images (&lt;strong&gt;400&lt;/strong&gt;) from each class. I don't have much knowledge so I don't know if this is a good division of data. &#xA;The attached picture shows the results on the test data after about 60 epoch of CNN with 15 layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kxCgs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kxCgs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network continues to overfiting, and the results of validation and test set do not improve. &#xA;I know that I could definitely improve my model but I would like to divide the data in some thoughtful and reasonable way. Pictures are spectrograms and are in RGB format.&lt;/p&gt;&#xA;" OwnerUserId="37928" LastEditorUserId="37928" LastEditDate="2020-06-16T09:46:12.373" LastActivityDate="2020-06-16T09:46:12.373" Title="How to split data into training validation and test set when the number of data in classes varies greatly?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;training&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21928" PostTypeId="1" CreationDate="2020-06-16T08:04:09.073" Score="1" ViewCount="139" Body="&lt;p&gt;I want to classify my corporate chat messages into a few categories such as question, answer, and report. I used a fine-tuned BERT model, and the result wasn't bad. Now, I started thinking about ways to improve it, and a rough idea came up, but I don't know what to do it exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I simply put chat text into the model, but don't use the speaker's information (who said the text, the speaker's ID in our DB). The idea is if I can use the speaker's information, the model might better understand the text and classify it better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, are there any examples or prior researches similar to what I want to achieve? I googled for a few hours, but couldn't find anything useful. (Maybe the keywords weren't good.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="37927" LastActivityDate="2020-06-17T16:36:18.267" Title="How to use speaker's information as well as text for fine-tuning BERT?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21929" PostTypeId="1" AcceptedAnswerId="21952" CreationDate="2020-06-16T08:18:17.213" Score="5" ViewCount="907" Body="&lt;p&gt;What is the cleanest, easiest way to explain someone who is a non-&lt;a href=&quot;https://en.wikipedia.org/wiki/Science,_technology,_engineering,_and_mathematics&quot; rel=&quot;nofollow noreferrer&quot;&gt;STEM&lt;/a&gt; work colleague the concept of Reinforcement Learning? What are the main ideas behind Reinforcement Learning?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-06-16T12:35:44.290" LastActivityDate="2020-06-17T18:52:52.140" Title="What is Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;academia&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21930" PostTypeId="2" ParentId="21925" CreationDate="2020-06-16T08:27:30.983" Score="1" Body="&lt;p&gt;Setting too high a learning rate will extend the time to get a good result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, it is better to set not too big a learning rate but to use learning with momentum. When the learning starts to be ineffective, increase the learning rate to find a better optimal result. It seems to me that this allows to get very good results in a faster time than setting a large value of the learning rate from the very beginning. At least in cases I've dealt with.&lt;/p&gt;&#xA;" OwnerUserId="37928" LastActivityDate="2020-06-16T08:27:30.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21931" PostTypeId="2" ParentId="21917" CreationDate="2020-06-16T08:34:08.777" Score="2" Body="&lt;p&gt;The first thing to note here is that your results seem aligned with the results commonly found in the bandit literature.&lt;/p&gt;&#xA;&lt;p&gt;Second thing to note would be that the performance of bandit algorithms is usually measured in terms of &lt;em&gt;regret&lt;/em&gt;. This is the difference between (i) the amount of rewards accumulated by an &lt;em&gt;oracle&lt;/em&gt; policy having prior knowledge about the &lt;em&gt;true&lt;/em&gt; rewards of the bandit arms and (ii) the amount of rewards accumulated by the bandit strategy you are evaluating. In other words, this measures the loss your incur due to not knowing what the optimal arm is.&lt;/p&gt;&#xA;&lt;p&gt;Ideally, the regret should be &lt;em&gt;logarithmic&lt;/em&gt;. In theory, all three algorithms achieve logarithmic regret.&#xA;See &lt;a href=&quot;https://link.springer.com/article/10.1023/A:1013689704352&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt; for the regret of decaying &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy and UCB and &lt;a href=&quot;http://proceedings.mlr.press/v23/agrawal12/agrawal12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt; for the regret of Thompson sampling.&lt;/p&gt;&#xA;&lt;p&gt;Despite their similar regret bounds, these algorithms may behave differently in practice.&#xA;For example, according to experiments in &lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/thompson.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[3]&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It appears that Thompson&#xA;sampling is more robust than UCB when the delay is long. Thompson sampling alleviates the&#xA;influence of delayed feedback&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt; by randomizing over actions; on the other hand, UCB is deterministic&#xA;and suffers a larger regret in case of a sub-optimal choice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For decaying &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, the fact that it obtained the lowest rewards may be due to the fact that the action selected for exploration is chosen &lt;em&gt;uniformly at random&lt;/em&gt; (even if it has a bad reward estimate). In addition, it will explore actions even if they have already been selected many times, unlike UCB which considers the number of action selections to refine the confidence bound.&lt;/p&gt;&#xA;&lt;p&gt;That said, besides these aspects, there may be other factors that could influence the results obtained by a given bandit algorithm. These may include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The reward distributions you chose to model your actions, and their&#xA;characteristics.&lt;/li&gt;&#xA;&lt;li&gt;The number of actions you have, and how far each&#xA;action’s reward is from the optimal action’s reward.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$^*$&lt;/span&gt; Delayed feedback occurs when the outcome of an action selection is not revealed immediately but after some time delay.&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-16T08:34:08.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21932" PostTypeId="2" ParentId="21876" CreationDate="2020-06-16T08:57:18.200" Score="0" Body="&lt;p&gt;I would say it is the nature of data. Generally speaking, you are trying to predict a random sequence, especially if you use the history data as an input and try to get the future value as an output.&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-16T08:57:18.200" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21935" PostTypeId="2" ParentId="21927" CreationDate="2020-06-16T09:08:33.080" Score="1" Body="&lt;p&gt;I would make the distribution of the classes in test and validation sets the same as in the training set (and as in the whole original set). Anyway all your metrics are relative, not absolute and designed to provide reasonable results when classes are not ideally balanced.&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-16T09:08:33.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21936" PostTypeId="2" ParentId="21929" CreationDate="2020-06-16T09:47:39.140" Score="5" Body="&lt;p&gt;Reinforcement Learning can be explained by a few equations. However I assume that this is not what you are looking at since the explanation should be for someone having a non-STEM background. Not to say non-STEM folks are not able to understand math equations, but intuition comes easier with words and examples in my opinion.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Reinforcement Learning is about &lt;em&gt;learning an optimal behavior&lt;/em&gt; by&#xA;repeatedly executing &lt;em&gt;actions&lt;/em&gt;, observing the &lt;em&gt;feedback&lt;/em&gt; from the&#xA;environment and adapting future actions based on that feedback.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Let's break down the last sentence by the concrete example of learning how to play chess:&lt;br /&gt;&#xA;Imagine you sit in front of a chess board, not knowing how to play. The &lt;em&gt;optimal behavior&lt;/em&gt; you'd like to learn is what moves to execute in order to win the game. So you start learning the game by playing a few moves (&lt;em&gt;actions&lt;/em&gt;) with some figures and &lt;em&gt;observing&lt;/em&gt; what is happening on the board (&lt;em&gt;environment&lt;/em&gt;) and identifying which moves bring you closer victory or give you a better position on the board (&lt;em&gt;feedback&lt;/em&gt;). Therefore in future games you will prefer moves which gave you a positive outcome in previous games.&lt;br /&gt;&#xA;Admittedly this is a very slow process of learning if you don't have a teacher which helps you in the beginning and you would have to play a lot of games until your first victory. But this essentially how computers (and sometimes humans in some sense) learn to do certain things by Reinforcement Learning. Behaviors which lead to positive experiences are collected, memorized and thus &lt;em&gt;reinforced&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="2444" LastEditDate="2020-06-17T18:52:52.140" LastActivityDate="2020-06-17T18:52:52.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21937" PostTypeId="1" CreationDate="2020-06-16T10:27:05.267" Score="1" ViewCount="54" Body="&lt;p&gt;There is a question already about applying RL to &lt;a href=&quot;https://ai.stackexchange.com/q/11809/37627&quot;&gt;&quot;large scale problems&quot;&lt;/a&gt;, where large scale refers to the problem of a &lt;em&gt;relatively small number of actions&lt;/em&gt; (that could be from a continous space) resulting in a very large number of states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good exapmle of this kind of large-scale probems is modeling a motion of a boat as a point on a plane, with an action being a displacement vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\delta}_b = (\delta_x, \delta_y)$&lt;/span&gt; and there are infinitely many states, because the next state is given by the next position of the boat, in a circle surrounding the boat &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}(\mathbf{x}_b, \mathbf{\delta}_{b,max})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x_b}$&lt;/span&gt; is the boat's current position, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\delta}_{b,max}$&lt;/span&gt; the maximal possible displacement. So here, the displacement as an action (move the boat) is from an infinite space because it is a 2D vector (&lt;span class=&quot;math-container&quot;&gt;$\delta_b \in \subset \mathbb{R}^2$&lt;/span&gt;) and so is the state space &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}$&lt;/span&gt;. Still, I just have two actions to apply to the boat in the end: move in x-directions this much, and in y-direction that much.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean, is something even larger. Considering the example of a boat, is it possible to apply reinforcement learning on a system that has 100 000 of such boats, and what would be the methods to look into to accomplish this. I do not mean to have 100 000 agents. The agent in this scenario is observing 100 000 boats, they are its environment, and let's say the agent is distributing them in a current on the sea in such a way that they have the least amount of resistance in the water (the wake of one ship influences the resistance of its downstream neighbors). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://ai.stackexchange.com/a/11821/37627&quot;&gt;this answer&lt;/a&gt; and from what I have read so far, I believe an approximation will be necessary for the displacements in &lt;span class=&quot;math-container&quot;&gt;$2D$&lt;/span&gt; space &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\delta}(x,y)$&lt;/span&gt; as well as for the states and rewards, because there are so many of them. However, before digging into this, I would like to know if there are some references out there where something like this has already been tried, or if this is simply something where RL cannot be applied. &lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="37627" LastEditDate="2020-06-17T07:50:59.467" LastActivityDate="2020-06-17T07:50:59.467" Title="Can reinforcement learning algorithms be applied on problems involving a very large number of possible actions?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21938" PostTypeId="1" AcceptedAnswerId="21944" CreationDate="2020-06-16T12:47:33.310" Score="2" ViewCount="69" Body="&lt;p&gt;I would love to know in detail, how exactly GPUs help, in technical terms, in training the deep learning models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my understanding, GPUs help in performing independent tasks simultaneously to improve the speed. For example, in calculation of the output through CNN, all the additions are done simultaneously, and hence improves the speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, what exactly happens in a basic neural network or in a LSTM type complex models in regard with GPU.&lt;/p&gt;&#xA;" OwnerUserId="37180" LastEditorUserId="37180" LastEditDate="2020-06-16T13:57:05.023" LastActivityDate="2020-06-16T14:04:29.503" Title="How do GPUs faciliate the training of a Deep Learning Architecture?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;gpu&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21940" PostTypeId="5" CreationDate="2020-06-16T13:23:01.327" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T13:23:01.327" LastActivityDate="2020-06-16T13:23:01.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21941" PostTypeId="4" CreationDate="2020-06-16T13:23:01.327" Score="0" Body="For questions related to the concept of bag-of-features (also known as bag-of-visual-words), which is a way of representing the features of images in a fixed-size vector." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T13:23:01.327" LastActivityDate="2020-06-16T13:23:01.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21942" PostTypeId="5" CreationDate="2020-06-16T13:24:23.877" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T13:24:23.877" LastActivityDate="2020-06-16T13:24:23.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21943" PostTypeId="4" CreationDate="2020-06-16T13:24:23.877" Score="0" Body="For questions related to content-based image retrieval (CBIR), i.e. the tasks of retrieving one or more images in a database that are close (in terms of content) to a query (or test) image." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T13:24:23.877" LastActivityDate="2020-06-16T13:24:23.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21944" PostTypeId="2" ParentId="21938" CreationDate="2020-06-16T13:46:44.857" Score="3" Body="&lt;p&gt;GPUs are able to execute a huge amount of similar and simple instructions (floating point operations like addition and multiplication) in parallel. In contrast to a CPU which is able to execute a few complex tasks sequentially very quick. Therefore GPUs are very good at doing vector &amp;amp; matrix operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the operations performed inside a single basic NN layer you will see that most of the operations are matrix-vector multiplications:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_{i+1} = \sigma(W_ix_i + b_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is the input vector, &lt;span class=&quot;math-container&quot;&gt;$W_i$&lt;/span&gt; the matrix of weights, &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; the bias vector in the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; layer, &lt;span class=&quot;math-container&quot;&gt;$x_{i+1}$&lt;/span&gt; the output vector and &lt;span class=&quot;math-container&quot;&gt;$\sigma(\cdot)$&lt;/span&gt; the elemtwise non-linear activation. The computational complexity here is governed by the matrix-vector multiplication. If you look at the architecture of a &lt;a href=&quot;https://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM cell&lt;/a&gt; you will notice that inside of it are multiple such operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being able to execute the matrix-vector operations quickly and efficiently in parallel will reduce execution time, this is where GPUs excel CPUs.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="37120" LastEditDate="2020-06-16T14:04:29.503" LastActivityDate="2020-06-16T14:04:29.503" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21945" PostTypeId="2" ParentId="21883" CreationDate="2020-06-16T13:49:37.890" Score="0" Body="&lt;p&gt;Based on the citations in the ConvLSTM paper I have come to the conclusion that they mean the Peephole LSTM when they say fully connected LSTM. In &lt;a href=&quot;https://arxiv.org/pdf/1502.04681.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; that they have taken the encoder-decoder-predictor model from, where they refer to a fully connected LSTM, a Peephole LSTM is used. Also they take their fully connected LSTM definition from &lt;a href=&quot;https://arxiv.org/pdf/1308.0850.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which again uses the Peephole LSTM.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that the difference would be the added &quot;peephole connections&quot;, that lets the gate layers look at the cell states and access the constant error carousel.&lt;/p&gt;&#xA;" OwnerUserId="37883" LastActivityDate="2020-06-16T13:49:37.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21946" PostTypeId="1" AcceptedAnswerId="21971" CreationDate="2020-06-16T14:15:42.207" Score="0" ViewCount="763" Body="&lt;p&gt;I am applying spacy lemmatization on my dataset, but already 20-30 mins passed and the code is still running. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anyway to make it faster? Is there any option to do this process using GPU?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataset size is 20k number of rows &amp;amp; 3 columns &lt;/p&gt;&#xA;" OwnerUserId="27505" LastEditorUserId="27505" LastEditDate="2020-06-17T06:12:09.653" LastActivityDate="2020-06-17T15:37:05.093" Title="How to make spacy lemmatization process fast?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;spacy&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21947" PostTypeId="1" AcceptedAnswerId="21948" CreationDate="2020-06-16T14:39:13.943" Score="1" ViewCount="76" Body="&lt;p&gt;During the first episode, it's 100% exploration, because all our Q values are 0. Suppose we have 1000 time steps, and it's terminated by meeting a reward. So, after the first episode, why can't we make it 100% exploitation? Why do we still need exploration?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-16T16:00:13.107" LastActivityDate="2020-06-17T16:04:35.247" Title="Why can't we fully exploit the environment after the first episode in Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21948" PostTypeId="2" ParentId="21947" CreationDate="2020-06-16T15:11:20.977" Score="1" Body="&lt;p&gt;You can't guarantee that you have taken every action from every state, even with  1000 time steps. There would be multiple outcomes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The episode terminates, either by success or failure before the 1000&#xA;time steps. The agent is trying to maximise reward, if this is achieved by taking less than 1000 steps then it will do. It won't just walk around until it hits an arbitrary number of time steps.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have more states than time steps then you will&#xA;never be unable to visit all states and so you cannot guarantee that&#xA;the policy you followed was optimal (and hence would still want to explore). Even if you have #states = #time-steps then you will almost certaintly have more state-action pairs than timesteps. The only time this would be equal is if from every state there is only one action, which would be a trivial problem that wouldn't need RL to solve.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36082" LastActivityDate="2020-06-16T15:11:20.977" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21950" PostTypeId="2" ParentId="21929" CreationDate="2020-06-16T17:10:48.057" Score="5" Body="&lt;p&gt;The famous book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning: an introduction&lt;/a&gt; by Sutton and Barto provides an intuitive description of reinforcement learning (that everyone is possibly able to understand).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Reinforcement learning is learning what to do — how to map situations to actions — so as to maximize a numerical reward signal. The learner is not told which actions to take, but instead must discover which actions yield the most reward by trying them.&lt;/p&gt;&#xA;&lt;p&gt;In the most interesting and challenging cases, actions may affect not only the immediate reward but also the next situation and, through that, all subsequent rewards. These two characteristics — trial-and-error search and delayed reward — are the two most important distinguishing features of reinforcement learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=69&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 3&lt;/a&gt;, the book also introduces the agent-environment interface, which summarises the cyclic interaction between the agent (aka policy) and the environment (which represents the task/problem that you need to solve).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/elWrq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/elWrq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Every RL algorithm implements a cyclic interaction between an agent and an environment (as illustrated above), where, on each time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent takes an action &lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;, the environment emits a reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+1}$&lt;/span&gt;, and the agent and the environment move from state &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; to the state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;. This interaction continues until some termination criterion is met (for example, the agent dies). While this interaction occurs, the agent is supposed to reinforce the actions that lead to better outcomes (i.e. higher reward).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="-1" LastEditDate="2020-06-17T09:57:20.710" LastActivityDate="2020-06-16T17:22:38.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21952" PostTypeId="2" ParentId="21929" CreationDate="2020-06-16T18:03:37.903" Score="5" Body="&lt;p&gt;Humans are set loose in the world and go about their days doing stuff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever they do specific things, their brain sends them good signals (endorphins, joy, etc.) or bad signals (pain, sadness, etc.). They learn through these signals which things they should be doing and which things they shouldn't be doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the signal is immediate and you know exactly what you're being &quot;rewarded&quot; or &quot;punished&quot; for (e.g. touch a hot stove and it hurts). Sometimes it takes a bit longer and there could be many possible reasons for the brain signal (even a combination of reasons), but you can hopefully figure out what caused it after it happens a few times (e.g. getting a stomach ache a few hours after eating a specific food).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's basically what Reinforcement Learning is.&lt;/p&gt;&#xA;" OwnerUserId="17490" LastActivityDate="2020-06-16T18:03:37.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21953" PostTypeId="5" CreationDate="2020-06-16T19:16:33.417" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T19:16:33.417" LastActivityDate="2020-06-16T19:16:33.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21954" PostTypeId="4" CreationDate="2020-06-16T19:16:33.417" Score="0" Body="For questions related to Kaggle, an online community of data scientists and machine learning practitioners that allows users to find and publish datasets, explore and build models, work with other people, and enter competitions to solve data science challenges." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-16T19:16:33.417" LastActivityDate="2020-06-16T19:16:33.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21955" PostTypeId="1" AcceptedAnswerId="21956" CreationDate="2020-06-16T19:32:05.997" Score="1" ViewCount="42" Body="&lt;p&gt;In a classic &lt;strong&gt;GridWorld Environment&lt;/strong&gt; where the possible &lt;strong&gt;actions&lt;/strong&gt; of an agent are (Up, Down, Left, Right), can another potential output of Action be &quot;x amount of steps&quot; where the agent takes 2,3,.. steps in the direction (U,D,L,R) that it chooses? If so, how would one go about doing it?&lt;/p&gt;&#xA;" OwnerUserId="37947" LastEditorUserId="37947" LastEditDate="2020-06-16T21:10:57.427" LastActivityDate="2020-06-16T21:24:37.297" Title="Additional (Potential) Action for Agent in MazeGrid Environment (Reinforcement Learning)" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21956" PostTypeId="2" ParentId="21955" CreationDate="2020-06-16T20:43:45.983" Score="0" Body="&lt;p&gt;You can definitely define an environment that accepts more types of action, including actions that take multiple steps in a direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing you would need to do is implement support for that action in the environment. That is not really a reinforcement learning issue, but like implementing the rules of a board game. You will need to decide things such as what happens if the move would be blocked - does the move succeed up the point of being blocked, does it fail completely, is the reward lower depending on how much the agent tries to overshoot, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you do that, you will want to write an agent that can choose the new actions. You have a few choices here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Simplest would be to enumerate all the choices separately and continue to use the same kind of agent as you already have. So instead of &lt;span class=&quot;math-container&quot;&gt;$\{U, D, L, R\}$&lt;/span&gt; you might have &lt;span class=&quot;math-container&quot;&gt;$\{U1, U2, U3, D1, D2, D3, L1, L2, L3, R1, R2, R3\}$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to take advantage of generalisation between similar actions (e.g. that action &lt;span class=&quot;math-container&quot;&gt;$U3$&lt;/span&gt; is similar to &lt;span class=&quot;math-container&quot;&gt;$U2$&lt;/span&gt; and also to &lt;span class=&quot;math-container&quot;&gt;$R3$&lt;/span&gt;), then you can use some form of coding for the action, such as the relative x,y movement that it is attempting. So you could express &lt;span class=&quot;math-container&quot;&gt;$U2$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$(0,2)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L3$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$(-3,0)$&lt;/span&gt;. For that to then work with Q values, you cannot easily use a table. Instead, you would need to use function approximation, for example a neural network, so you can implement &lt;span class=&quot;math-container&quot;&gt;$q(s,a)$&lt;/span&gt; as a parametric function - combining &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; into the input vector, and learn the parameters to that the neural network outputs the correct action value. This is what the Q learning variation DQN can do, as well as other similar RL algorithms that use neural networks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using a neural network, instead of tabular Q-learning, is not something you see often with grid world environments. It is a step up in complexity, but it is often required if state space or action space becomes large and might benefit from the generalisation possible from trainable function approximators.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-16T21:24:37.297" LastActivityDate="2020-06-16T21:24:37.297" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21957" PostTypeId="1" AcceptedAnswerId="21967" CreationDate="2020-06-17T04:20:42.210" Score="0" ViewCount="111" Body="&lt;p&gt;Given that this question has not yet been asked on this site, although similar questions have already been asked in the past (e.g. &lt;a href=&quot;https://ai.stackexchange.com/q/5546/2444&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://ai.stackexchange.com/q/21394/2444&quot;&gt;here&lt;/a&gt;), what is essentially a convolutional neural network (CNN)? Why are they heavily used in image recognition applications of machine learning?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-06-17T12:58:23.490" LastActivityDate="2020-06-17T12:58:23.490" Title="What is a convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;definitions&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21958" PostTypeId="2" ParentId="11949" CreationDate="2020-06-17T05:03:40.307" Score="0" Body="&lt;p&gt;Information outside of the bounding box could still be useful as context. So not to  lose it you can but bounding box as pixel mask into additional &quot;pseudo-color&quot; layer. That way you can also have many bounding boxes without changing input architecture. You will give network additional info without losing anything, so result shouldn't be worse at least.&lt;/p&gt;&#xA;" OwnerUserId="22745" LastActivityDate="2020-06-17T05:03:40.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21960" PostTypeId="1" CreationDate="2020-06-17T07:07:17.663" Score="1" ViewCount="30" Body="&lt;p&gt;Adam is known as an algorithm that has an adaptive learning rate for each parameter. I believe this is due to the division by the term &lt;span class=&quot;math-container&quot;&gt;$$v_t = \beta_2 \cdot v_{t-1} + (1-\beta_2) \cdot g_t^2 $$&lt;/span&gt; Hence, each weight will get updated differently based on the accumulated squared gradients in their respective dimensions, even though &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; might be constant. &lt;a href=&quot;https://stackoverflow.com/questions/39517431/should-we-do-learning-rate-decay-for-adam-optimizer#:%7E:text=Yes%2C%20absolutely.,learning%20rate%20decay%20using%20TensorFlow.&quot;&gt;There are other StackOverflow posts&lt;/a&gt; that have said that Adam has a built-in learning rate decay. In the original paper also, the &lt;a href=&quot;https://arxiv.org/abs/1412.6980&quot; rel=&quot;nofollow noreferrer&quot;&gt;authors of adam paper&lt;/a&gt; says that the learning rate at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; decays based on the equation &lt;span class=&quot;math-container&quot;&gt;$$\alpha_t = \alpha \cdot \frac{\sqrt{1-\beta_2^t}}{{1-\beta_1^t}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is the second equation the learning rate decay that has been built into the Adam algorithm?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-06-17T09:28:14.423" LastActivityDate="2020-06-17T09:28:14.423" Title="What is the equation of the learning rate decay in the Adam optimiser?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;&lt;learning-rate&gt;&lt;adam&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21961" PostTypeId="1" AcceptedAnswerId="21962" CreationDate="2020-06-17T07:11:19.363" Score="2" ViewCount="86" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hX1jB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hX1jB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the exercise 3.18 in Sutton and Barto's book.&lt;/p&gt;&#xA;&lt;p&gt;The task is to express &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; using &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Looking at the diagram above, the value of &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; for each &lt;span class=&quot;math-container&quot;&gt;$a \in A$&lt;/span&gt; we take gives us the value function at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; after taking the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and then following the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is probably wrong, but if&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = E_\pi[G_t | S_t = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q_\pi(s) = E_\pi[G_t | S_t = s, A_t = a]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;isn't then &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; just the expected action value function at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; over all actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that are given by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, namely&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = E_{a \sim \pi}[q_\pi(s,a) | S_t = s, A_t = a] = \sum_{a \in A}\pi(a|s) q_\pi(s,a)$$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="2444" LastEditDate="2020-06-17T09:30:36.420" LastActivityDate="2020-06-17T09:30:36.420" Title="How to express $v_\pi(s)$ in terms of $q_\pi(s,a)$?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21962" PostTypeId="2" ParentId="21961" CreationDate="2020-06-17T07:37:22.533" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;isn't then &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; just the expected action value function at &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; over all actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that are given by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, namely&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_\pi(s) = E_{a \sim \pi}[q_\pi(s,a) | S_t = s, A_t = a] = \sum_{a \in A}\pi(a|s) q_\pi(s,a)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes this is 100% correct.&lt;/p&gt;&#xA;&lt;p&gt;There is no &amp;quot;trick&amp;quot; to this or deeper thought needed. You have correctly isolated the key part of the MDP description that controls relationship between &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}$&lt;/span&gt; in that direction.&lt;/p&gt;&#xA;&lt;p&gt;Note that for a deterministic policy, with &lt;span class=&quot;math-container&quot;&gt;$\pi(s): \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt; then the relationship is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = q_\pi(s, \pi(s))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The related exercise in the book - expressing &lt;span class=&quot;math-container&quot;&gt;$q_{\pi}$&lt;/span&gt; in terms of &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}$&lt;/span&gt; and the MDP characteristics - is more complex because it involves a time step.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-06-17T09:29:40.613" LastActivityDate="2020-06-17T09:29:40.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21963" PostTypeId="1" AcceptedAnswerId="21964" CreationDate="2020-06-17T10:04:52.547" Score="1" ViewCount="73" Body="&lt;p&gt;When deriving the Bellman equation for &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt;, we have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a) = E_\pi[G_t | S_t = s, A_t = a] = E_\pi[R_{t+1} + \gamma G_{t+1} | S_t = s, A_t = a]$&lt;/span&gt; (1)&lt;/p&gt;&#xA;&lt;p&gt;This is what is confusing me, at this point, for the Bellman equation for &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt;, we write &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; as an expected value, conditioned on &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; &lt;em&gt;and&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; of the &lt;em&gt;action value function&lt;/em&gt; at &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, otherwise, there is no recursion with respect to &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt;, and therefore no Bellman equation. Namely,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ = \sum_{a \in A} \pi(a |s) \sum_{s' \in S} \sum_{r \in R} p(s',r|s,a)(r + \gamma E_\pi[G_{t+1}|S_{t+1} = s', A_{t+1} = a'])$&lt;/span&gt; (2)&lt;/p&gt;&#xA;&lt;p&gt;which introduces the recursion of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ = \sum_{a \in A} \pi(a |s) \sum_{s' \in S} \sum_{r \in R} p(s',r|s,a)(r + \gamma q_\pi(s',a'))$&lt;/span&gt; (3)&lt;/p&gt;&#xA;&lt;p&gt;which should be the Bellman equation for &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt;, right?&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, when connecting &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s')$&lt;/span&gt;, &lt;a href=&quot;https://ai.stackexchange.com/questions/21684/why-state-action-value-function-as-an-expected-value-of-the-return-and-state-val/21687#21687&quot;&gt;in this answer&lt;/a&gt;, I believe this is done&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a) = \sum_{a\in A} \pi(a |s) \sum_{s' \in S}\sum_{r \in R} p(s',r|s,a)(r + \gamma E_{\pi}[G_{t+1} | S_{t+1} = s'])$&lt;/span&gt; (4)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a) = \sum_{a\in A} \pi(a |s) \sum_{s' \in S}\sum_{r \in R} p(s',r|s,a)(r + \gamma v_\pi(s'))$&lt;/span&gt; (5)&lt;/p&gt;&#xA;&lt;p&gt;Is the difference between using the expectation &lt;span class=&quot;math-container&quot;&gt;$E_{\pi}[G_{t+1} | S_{t+1} = s', A_{t+1} = a']$&lt;/span&gt; in (3) and the expectation &lt;span class=&quot;math-container&quot;&gt;$E_{\pi}[G_{t+1} | S_{t+1} = s']$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$(4)$&lt;/span&gt; simply the difference in how we choose to express the expected return &lt;span class=&quot;math-container&quot;&gt;$G_{t+1}$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; in the definition of &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;, we express the total return at &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; using the action value function&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D8jyk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D8jyk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;leading to the recursion and the Bellman equation, and in &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;, the total return is expressed at &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; using the value function&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rlxHz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rlxHz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;leading to &lt;span class=&quot;math-container&quot;&gt;$q_\pi(s,a) = q_\pi(s,a,v_\pi(s'))$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="2444" LastEditDate="2020-10-04T10:54:47.637" LastActivityDate="2020-10-04T10:54:47.637" Title="Connection between the Bellman equation for the action value function $q_\pi(s,a)$ and expressing $q_\pi(s,a) = q_\pi(s, a,v_\pi(s'))$" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21964" PostTypeId="2" ParentId="21963" CreationDate="2020-06-17T10:28:28.830" Score="3" Body="&lt;p&gt;Your understanding of the Bellman equation is not quite right. The state-action value function is defined as the expected (discounted) returns when taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Now, in your equation (2) you have conditioned on taking action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; in the inner expectation - this is not what happens in the state-action value function, you &lt;em&gt;do not&lt;/em&gt; condition on knowing &lt;span class=&quot;math-container&quot;&gt;$A_{t+1}$&lt;/span&gt;, it is chosen according to the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; as per the definition of a bellman equation.&lt;/p&gt;&#xA;&lt;p&gt;If you want to see a 'recursion' between state action value functions, note that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \sum_a \pi(a|s)q_\pi(s,a)\;,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your equation (5) is incorrect -- you need to drop the outter sum over &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; as we have conditioned on knowing &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. I will drop the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; subscripts for ease on notation, and we can see a 'recursion' for state-action value functions as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$q(s,a) = \sum_{s',r}p(s',r|s,a)\left(r + \gamma \left[\sum_{a'} \pi(a'|s')q(s',a')\right]\right)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-17T11:43:04.100" LastActivityDate="2020-06-17T11:43:04.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21967" PostTypeId="2" ParentId="21957" CreationDate="2020-06-17T12:28:09.867" Score="2" Body="&lt;p&gt;(Of course, similar questions have been asked in the past and there are many sites, papers, video lessons, online that explain how CNNs work, but I think it's still a good idea to have a reference answer that hopefully will give you the main ideas behind CNNs.)&lt;/p&gt;&#xA;&lt;p&gt;A convolutional neural network (CNN) is a neural network that performs the &lt;strong&gt;convolution&lt;/strong&gt; (or cross-correlation) operation typically followed by some &lt;strong&gt;downsampling&lt;/strong&gt; (aka pooling) operations.&lt;/p&gt;&#xA;&lt;p&gt;The convolution operation comes from the mathematical equivalent operation, which is an operation that takes as inputs two functions &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and produces another function &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; as output, which is often denoted as &lt;span class=&quot;math-container&quot;&gt;$f \circledast h = g$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt; is the convolution operation (or operator).&lt;/p&gt;&#xA;&lt;p&gt;In image processing, the convolution is used to process images in multiples ways. For example, it is used to remove noise (e.g. the convolution of a noisy image with a Gaussian kernel produces a smoother image) or to compute derivatives of the image, which can then be used e.g. to detect edges or corners, which are usually the main features of an image. For example, the &lt;a href=&quot;http://www.bmva.org/bmvc/1988/avc-88-023.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Harris corner detector&lt;/em&gt;&lt;/a&gt; makes use of the partial derivatives (in the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; direction) of the image to find corners (or interest points) in the image.&lt;/p&gt;&#xA;&lt;p&gt;In the context of CNNs, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the &lt;strong&gt;image&lt;/strong&gt; (in the case of the first layer of the CNN) or a so-called &lt;em&gt;feature map&lt;/em&gt; (in the case of hidden layers), &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the &lt;strong&gt;kernel&lt;/strong&gt; (aka &lt;em&gt;filter&lt;/em&gt;) and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is also a &lt;strong&gt;feature map&lt;/strong&gt;. (In &lt;a href=&quot;https://ai.stackexchange.com/a/21401/2444&quot;&gt;this answer&lt;/a&gt;, I explain these concepts, including how an image can be viewed as a function, more in detail, so I suggest that you read it).&lt;/p&gt;&#xA;&lt;p&gt;Here's an illustrative example of how the convolution works.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8fjfx.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8fjfx.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\text{blue}}$&lt;/span&gt; grid is the image, the &lt;span class=&quot;math-container&quot;&gt;$\color{gray}{\text{gray}}$&lt;/span&gt; grid is the kernel and &lt;span class=&quot;math-container&quot;&gt;$\color{green}{\text{green}}$&lt;/span&gt; grid is the feature map (i.e. the output of the convolution between the image and the kernel). The white squares are around the images represent the padding that is added around the image so that the convolution operation produces a feature map of a specific dimension.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, the convolution is &lt;strong&gt;a series of dot products&lt;/strong&gt; between the kernel and different patches (or parts) of the image. &lt;a href=&quot;https://ai.stackexchange.com/q/11172/2444&quot;&gt;The convolution can even be represented as matrix multiplication&lt;/a&gt;, so the name &lt;em&gt;convolution&lt;/em&gt; shouldn't scare you anymore, if you are familiar with dot products and matrix multiplications.&lt;/p&gt;&#xA;&lt;p&gt;As opposed to many operations in image processing where the kernels are typically fixed, in the context of CNNs, the kernels are &lt;strong&gt;learnable&lt;/strong&gt; parameters, i.e. they change depending on the loss function, so they are supposed to represent functions that when convolved with their respective input functions are useful to extract meaningful information (i.e. features) to solve the task the CNN is being trained to solve. For this reason, the convolution is often thought of as an operation that extracts features from images. In fact, the output of the convolution, in the context of CNNs, is often called &lt;em&gt;feature map&lt;/em&gt;. Moreover, a CNN is typically thought of as a &lt;strong&gt;data-driven feature extractor&lt;/strong&gt; for the same reason.&lt;/p&gt;&#xA;&lt;p&gt;There are many different variations of the standard convolution operation. For example, there are &lt;strong&gt;transposed convolutions&lt;/strong&gt;, &lt;strong&gt;dilated convolutions&lt;/strong&gt;, etc., which are used to solve slightly different problems. For example, the dilated convolution can be used to increase the receptive field of an element of a feature map (which is typically a desirable property for several reasons). There are also &lt;strong&gt;upsampling&lt;/strong&gt; operations which are particularly useful in the context of image segmentation. These different convolutions and their arithmetic are explained very well in the paper &lt;a href=&quot;https://arxiv.org/pdf/1603.07285.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A guide to convolution arithmetic for deep learning&lt;/a&gt; by Vincent Dumoulin and Francesco Visin. &lt;a href=&quot;https://github.com/vdumoulin/conv_arithmetic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;'s also the associated Github repository that contains all the images in this paper that show how the convolution operations work (from which I also took the gif above).&lt;/p&gt;&#xA;&lt;p&gt;To conclude, CNNs are very useful to process images and extract features from them because they use convolution operations (and downsampling and upsampling operations). They can be used for &lt;strong&gt;image&lt;/strong&gt; (or object) &lt;strong&gt;classification&lt;/strong&gt;, &lt;strong&gt;object detection&lt;/strong&gt; (i.e. object localization with a bounding box + object classification), &lt;strong&gt;image segmentation&lt;/strong&gt; (including semantic segmentation and instance segmentation), and possibly many other tasks where you need to learn a function that takes as input images and needs to extract information from those images to get someone high-level (but also low-level) output (e.g. the name of the object in the image), given some training data.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-17T12:55:33.060" LastActivityDate="2020-06-17T12:55:33.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21968" PostTypeId="1" AcceptedAnswerId="22044" CreationDate="2020-06-17T14:07:48.640" Score="1" ViewCount="58" Body="&lt;p&gt;I think I'm misunderstanding the description of IDA* and want to clarify.&lt;/p&gt;&#xA;&lt;p&gt;IDA* works as follows (quoting from Wiki):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;At each iteration, perform a depth-first search, cutting off a branch when its total cost exceeds a given threshold. This threshold starts at the estimate of the cost at the initial state, and increases for each iteration of the algorithm. At each iteration, the threshold used for the next iteration is the minimum cost of all values that exceeded the current threshold.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Suppose that we have the following tree:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;branching factor = 5&lt;/li&gt;&#xA;&lt;li&gt;all cost are different&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Say we have expanded 1000 nodes. We pick the lowest cost of the nodes that we 'touched' but didn't expand. Since all costs are unique, there is now only one more node which satisfies this new cost bound, and so we expand 1001 nodes, and 'touch' 5 new ones. We now pick the smallest of these weights, and starting from the root expand 1002 nodes, and so on and so forth, 1003, 1004...&lt;/p&gt;&#xA;&lt;p&gt;I must be doing something wrong here right? If not, the complexity is &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt;, where n is the number of nodes with cost smaller than the optimum, compared to n for normal A*.&lt;/p&gt;&#xA;&lt;p&gt;Someone pointing out my misunderstanding would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="37970" LastEditorUserId="2444" LastEditDate="2020-06-17T14:16:43.573" LastActivityDate="2020-06-20T16:57:56.300" Title="Doesn't the number of explored nodes with IDA* increase linearly?" Tags="&lt;search&gt;&lt;a-star&gt;&lt;ida-star&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21969" PostTypeId="2" ParentId="17412" CreationDate="2020-06-17T14:23:56.760" Score="0" Body="&lt;p&gt;In Tensorflow 2.0 you can set random seed like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;tf.random.set_seed(221)&#xA;&#xA;&#xA;from tensorflow import keras&#xA;from tensorflow.keras import layers&#xA;&#xA;&#xA;model = keras.Sequential( [ &#xA;layers.Dense(2,name = 'one'),&#xA;layers.Dense(3,activation = 'sigmoid', name = 'two'),&#xA;layers.Dense(2,name = 'three')])&#xA;&#xA;x = tf.random.uniform((12,12))&#xA;model(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2773" LastActivityDate="2020-06-17T14:23:56.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21970" PostTypeId="1" AcceptedAnswerId="21977" CreationDate="2020-06-17T15:12:56.500" Score="4" ViewCount="1014" Body="&lt;p&gt;Nowadays, CV has really achieved great performance in many different areas. However, it is not clear what a CV algorithm is.&lt;/p&gt;&#xA;&lt;p&gt;What are some examples of CV algorithms that are commonly used nowadays and have achieved state-of-the-art performance?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-06-18T01:41:17.313" LastActivityDate="2020-06-19T08:03:09.947" Title="What are the main algorithms used in computer vision?" Tags="&lt;computer-vision&gt;&lt;terminology&gt;&lt;algorithm&gt;&lt;definitions&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21971" PostTypeId="2" ParentId="21946" CreationDate="2020-06-17T15:37:05.093" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://spacy.io/api/lemmatizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spacy.io/api/lemmatizer&lt;/a&gt; just uses lookup tables and the only upstream task it relies on is POS tagging, so it should be relatively fast. For large amounts of text, &lt;a href=&quot;https://spacy.io/usage/processing-pipelines#processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpaCy recommends&lt;/a&gt; using &lt;a href=&quot;https://spacy.io/api/language#pipe&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;nlp.pipe&lt;/code&gt;&lt;/a&gt;, which can work in batches and has built in support for multiprocessing (with the &lt;code&gt;n_process&lt;/code&gt; keyword), rather than than simply &lt;code&gt;nlp&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Also, make sure you disable any pipeline elements that you don't plan to use, as they'll just waste processing time. If you're only doing lemmatization, you'll pass &lt;code&gt;disable=[&amp;quot;parser&amp;quot;, &amp;quot;ner&amp;quot;]&lt;/code&gt; to the &lt;code&gt;nlp.pipe&lt;/code&gt; call.&lt;/p&gt;&#xA;&lt;p&gt;Example code that takes all of the above into account is below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import spacy&#xA;nlp = spacy.load(&amp;quot;en_core_web_sm&amp;quot;)&#xA;&#xA;docs = [&amp;quot;We've been running all day.&amp;quot;, &amp;quot;Let's be better.&amp;quot;]&#xA;&#xA;for doc in nlp.pipe(docs, batch_size=32, n_process=3, disable=[&amp;quot;parser&amp;quot;, &amp;quot;ner&amp;quot;]):&#xA;    print([tok.lemma_ for tok in doc])&#xA;&#xA;# ['-PRON-', 'have', 'be', 'run', 'all', 'day', '.']&#xA;# ['let', '-PRON-', 'be', 'well', '.']&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37972" LastActivityDate="2020-06-17T15:37:05.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21972" PostTypeId="1" AcceptedAnswerId="21983" CreationDate="2020-06-17T15:58:14.273" Score="7" ViewCount="2113" Body="&lt;p&gt;I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a bit. There are two parameters that I tweaked: the batch size and the gradient accumulation steps. However I am not sure about the effects of this modification so I would like to fully understand what is the relationship between the gradient accumulation steps parameter and the batch size.&lt;/p&gt;&#xA;&lt;p&gt;I know that when you accumulate the gradient you are just adding the gradient contributions for some steps before updating the weights. Normally, you would update the weights every time you compute the gradients (traditional approach):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w_{t+1} = w_t - \alpha \cdot \nabla_{w_t}loss$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But when accumulating gradients you compute the gradients several times before updating the weights (being &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; the number of gradient accumulation steps):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w_{t+1} = w_t - \alpha \cdot \sum_{0}^{N-1} \nabla_{w_t}loss$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is: What is the relationship between the batch size &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and the gradient accumulation steps &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;By example: are the following configurations equivalent?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$B=8, N=1$&lt;/span&gt;: No gradient accumluation (accumulating every step), batch size of 8 since it fits in memory.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$B=2, N=4$&lt;/span&gt;: Gradient accumuluation (accumulating every 4 steps), reduced batch size to 2 so it fits in memory.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My intuition is that they are but I am not sure. I am not sure either if I would have to modify the learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-06-18T01:48:25.027" Title="What is the relationship between gradient accumulation and batch size?" Tags="&lt;gradient-descent&gt;&lt;theory&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21973" PostTypeId="2" ParentId="21947" CreationDate="2020-06-17T16:04:35.247" Score="2" Body="&lt;p&gt;BlueTurtle's answer is good, but I'd like to add something.&lt;/p&gt;&#xA;&lt;p&gt;Your question realistically has nothing to do with Q Learning, in fact, you can ask the same thing about just about any RL algorithm. In fact, even in multi-armed bandits, you can easily see why your proposed method is suboptimal (please don't interpret this as a criticism, because I think your question is a very natural one). My suggestion to you is to read up on multi-armed bandits since they're much simpler to analyze. I think even the Sutton and Barto book deals with your proposed method explicitly, and mathematically proves that other strategies are better.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-17T16:04:35.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21974" PostTypeId="2" ParentId="21928" CreationDate="2020-06-17T16:36:18.267" Score="1" Body="&lt;p&gt;My answer assumes your fine-tuning architecture simply stacks a single fully-connected layer on top of the BERT &lt;code&gt;[CLS]&lt;/code&gt; output, as in Figure 4b of the &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Generally, when working with mixed data such as continuous and categorical features, the first step is to simply concatenate all the inputs into one long vector. In your case, you would concatenate a one-hot encoding of speaker ID to the BERT [CLS] output for each example. If you're using tensorflow, you'll need to use the functional API to create a multiple input model, as I outline below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow.keras as keras&#xA;&#xA;# bert_input is the wordpiece tokenized input text&#xA;# BERT is the BERT model, such as a tensorflow_hub.KerasLayer with a pretrained model from tfhub.dev&#xA;bert_out = BERT(bert_input)&#xA;&#xA;# spkr_input is the one-hot speaker ID encoding.&#xA;spkr_input = keras.Input(shape=(num_speakers,), name='spkr_input')&#xA;dense_input = keras.layers.concatenate([bert_out, spkr_input])&#xA;scores = keras.layers.Dense(num_classes, activation='softmax')(dense_input)&#xA;model = keras.Model(inputs=[bert_input , spkr_input], outputs=[scores])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could also try feeding the one-hot encoded speaker ID vectors through a separate fully-connected network first to obtain a continuous representation (i.e. a speaker embedding) and then concatenate that to the BERT [CLS] output and feed the result into your classification layer. Modifying my example above,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;spkr_emb = keras.layers.Dense(spkr_emb_size, activation=&amp;quot;relu&amp;quot;)(spkr_input)&#xA;dense_input = keras.layers.concatenate([bert_out, spkr_emb])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can find a more detailed guide to mixed input models with Keras at &lt;a href=&quot;https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37972" LastActivityDate="2020-06-17T16:36:18.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21975" PostTypeId="1" AcceptedAnswerId="22052" CreationDate="2020-06-17T16:39:32.947" Score="0" ViewCount="53" Body="&lt;p&gt;I'm in the middle of a project in which I want to generate a TV series script (characters answering to each other, scene by scene) using SOTA models, and I need some guidance to simplify my architecture.&lt;/p&gt;&#xA;&lt;p&gt;My current intuition is as follows: for a given character &lt;em&gt;C1&lt;/em&gt;, I have pairs of sentences from the original scripts where &lt;em&gt;C1&lt;/em&gt; answers other characters, for example, &lt;em&gt;C2&lt;/em&gt; (&lt;em&gt;C2&lt;/em&gt;-&amp;gt;&lt;em&gt;C1&lt;/em&gt;). These are used to fine-tune a data-driven chatbot. At inference time, the different chatbots simply answer each other, and, hopefully, the conversation will have some sense.&lt;/p&gt;&#xA;&lt;p&gt;This is however unpractical and will be kind of a mess with many characters, especially if I use heavy models.&lt;/p&gt;&#xA;&lt;p&gt;Is there an architecture out there that could be used for conversational purposes, which could be trained only once with the whole dataset while separating the different characters?&lt;/p&gt;&#xA;&lt;p&gt;I'm open to any ideas!&lt;/p&gt;&#xA;" OwnerUserId="37974" LastEditorUserId="2444" LastEditDate="2020-06-20T09:36:28.117" LastActivityDate="2020-06-21T07:54:12.877" Title="How to combine several chatbots into one?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;chat-bots&gt;&lt;generative-model&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21976" PostTypeId="1" CreationDate="2020-06-17T20:18:14.233" Score="1" ViewCount="44" Body="&lt;p&gt;I'm working on a project lately and I'm trying to solve a problem with reinforcement learning and I have serious issues with shaping the reward function.&lt;/p&gt;&#xA;&lt;p&gt;The problem is designing a device with maximum efficiency. So we simulated the problem as follows. There is a 4x4 grid (we defined a 4x4 matrix) and the elements of this matrix can be either 0 or 1 (value 0 means &amp;quot;air&amp;quot; and 1 means a certain material in reality), so there is a 2^16 possible configurations for this matrix. Our agent starts from the top left corner of this matrix and has 5 possible actions: move up, down, left, right and flip (which means flipping a 0 to 1 or vice versa). Based on flipping action, we get a new configuration and each configuration has an efficiency (which is calculated by maxwell equations in the background).&lt;/p&gt;&#xA;&lt;p&gt;Our goal is to find the best configuration so that the efficiency of the device is maximum.&lt;/p&gt;&#xA;&lt;p&gt;So far we have tried many reward functions and non of them seemed to work at all! I will mention some of them:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;reward = current_efficiency - previous_efficiency (the efficiency is being calculated in each time step)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;&lt;pre&gt;&lt;code&gt; if current_efficiency &amp;gt; previous_efficiency:&#xA;    reward = current_efficiency&#xA;    previous_efficiency = current_efficiency&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;&lt;pre&gt;&lt;code&gt; diff = current_efficiency - previous_efficiency&#xA; if diff &amp;gt; 0:&#xA;    reward = 1&#xA; else:&#xA;    reward = -2&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;and some other variations. Nothing is working for our problem and the agent doesn't learn at all! So far, we have used different approaches to DQN and also A2C method and so far no positive feedback. We tried different definitions of states as well, but we don't think that is the problem.&lt;/p&gt;&#xA;&lt;p&gt;So, can somebody maybe help me with this? It would be a huge help!&lt;/p&gt;&#xA;" OwnerUserId="37979" LastEditorUserId="2444" LastEditDate="2020-10-07T16:52:18.480" LastActivityDate="2020-10-07T16:52:18.480" Title="Designing a reward function for my reinforcement learning problem" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;reward-design&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21977" PostTypeId="2" ParentId="21970" CreationDate="2020-06-17T22:12:49.520" Score="3" Body="&lt;p&gt;There are many computer vision (CV) algorithms and models that are used for different purposes. So, of course, I cannot list all of them, but I can enumerate some of them based on my experience and knowledge. Of course, this answer will only give you a flavor of the type of algorithm or model that you will find while solving CV tasks.&lt;/p&gt;&#xA;&lt;p&gt;For example, there are algorithms that are used to extract &lt;strong&gt;keypoints&lt;/strong&gt; and &lt;strong&gt;descriptors&lt;/strong&gt; (which are often collectively called &lt;strong&gt;features&lt;/strong&gt;, although the descriptor is the actual feature vector and the keypoint is the actual feature, and in deep learning this distinction between keypoints and descriptors does not even exist, AFAIK) from images, i.e. &lt;strong&gt;feature extraction&lt;/strong&gt; algorithms, such as &lt;a href=&quot;https://www.cs.ubc.ca/%7Elowe/papers/ijcv04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;SIFT&lt;/strong&gt;&lt;/a&gt;, BRISK, FREAK, SURF or ORB. There are also edge and corner detectors. For example, the &lt;a href=&quot;http://www.bmva.org/bmvc/1988/avc-88-023.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Harris corner detector&lt;/strong&gt;&lt;/a&gt; is a very famous corner detector.&lt;/p&gt;&#xA;&lt;p&gt;Nowadays, &lt;strong&gt;convolutional neural networks&lt;/strong&gt; (CNNs) have basically supplanted all these algorithms in many cases, especially when enough data is available. Rather than extracting the typical features from an image (such as corners), CNNs extract features that are most useful to solve the task that you want to solve by taking into account the information in the training data (which probably includes corners too!). Hence CNNs are often called &lt;strong&gt;data-driven feature extractors&lt;/strong&gt;. There are different types of CNNs. For example, CNNs that were designed for &lt;strong&gt;semantic segmentation&lt;/strong&gt; (which is a CV task/problem), such as the &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;u-net&lt;/strong&gt;&lt;/a&gt;, or CNNs that were designed for &lt;strong&gt;instance segmentation&lt;/strong&gt;, such as &lt;a href=&quot;https://arxiv.org/pdf/1703.06870.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;mask R-CNN&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are also algorithms that can be used to normalize features, such as the &lt;a href=&quot;https://ai.stackexchange.com/q/21914/2444&quot;&gt;&lt;strong&gt;bag-of-features&lt;/strong&gt; algorithm&lt;/a&gt;, which can be used to create fixed-size feature vectors. This can be particularly useful for tasks like content-based image retrieval.&lt;/p&gt;&#xA;&lt;p&gt;There are many other algorithms that could be considered CV algorithms or are used to solve CV tasks. For example, &lt;a href=&quot;https://dl.acm.org/doi/10.1145/358669.358692&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;RanSaC&lt;/strong&gt;&lt;/a&gt;, which is a very general algorithm to fit models to data in the presence of outliers, can be used to fit homographies (matrices that are generally used to transform planes to other planes) that transform pixels of one image to another coordinate system of another image. This can be useful for the purpose of &lt;strong&gt;template matching&lt;/strong&gt; (which is another CV task), where you want to find a template image in another target image. This is very similar to &lt;strong&gt;object detection&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are also many &lt;strong&gt;image processing&lt;/strong&gt; algorithms and techniques that are heavily used in computer vision. For example, all the &lt;strong&gt;filters&lt;/strong&gt; (such as Gaussian, median, bilateral, non-local means, etc.) that can be used to smooth, blur or de-noise images. Nowadays, some deep learning techniques have also replaced some of these filters and image processing techniques, such as &lt;strong&gt;de-noising auto-encoders&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;All these algorithms and models have something in common: they are used to process images and/or get low- or high-level information from images. Most of them are typically used to extract features (i.e. regions of the images that are relevant in some way) from images, so that they can later be used to train a classifier or regressor to perform some kind of task (e.g. find and distinguish the objects, such people, cars, dogs, etc. in an image). The classifiers/regressors are typically machine learning (ML) models, such as SVMs or fully-connected neural networks, but there's a high degree of overlap between CV and ML because some ML tools are used to solve CV tasks (e.g. image classification).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-18T09:19:32.337" LastActivityDate="2020-06-18T09:19:32.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21978" PostTypeId="2" ParentId="21926" CreationDate="2020-06-17T22:32:59.283" Score="0" Body="&lt;p&gt;I do not see how you came to choose prices as actions. Normally, actions are something like &lt;code&gt;go left&lt;/code&gt;, &lt;code&gt;go right&lt;/code&gt;, &lt;code&gt;jump&lt;/code&gt;, &lt;code&gt;stay&lt;/code&gt; etc.&#xA;Analogously, I would say that in your case the actions are visiting a certain location, whereas locations are what you referred to as &lt;code&gt;states&lt;/code&gt;. I'd go for something like that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;locations = {location1=(c1,cr1), location2=(c1,cr2), ...}&#xA;Actions = {Visit location1, Visit location2, ...}&#xA;States = {--set of all the possible paths the model can generate until the budget is possibly exhausted--}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;reward&lt;/code&gt; function could then be a combination of both the &lt;em&gt;acceptance (vs. rejection) of a route/path by the user&lt;/em&gt; and the &lt;em&gt;inverse of the cost associated with the suggested path&lt;/em&gt; (because you want the model to favor cheap paths in order to keep your own business costs low).&#xA;How you balance these two terms is up to exploration.&lt;/p&gt;&#xA;&lt;p&gt;For rapid prototyping, check out &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;stable baselines&lt;/a&gt;, which offers a bunch of highly optimized RL algorithms.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-06-17T22:32:59.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21979" PostTypeId="1" CreationDate="2020-06-17T22:40:04.380" Score="1" ViewCount="44" Body="&lt;p&gt;I have developed an RPG in likeness to the features showcased in the Final Fantasy series; multiple character classes which utilise unique action sets, sequential turn-based combat, front/back row modifiers, item usage and a crisis mechanic which bears similarity to the limit break feature.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that the greater portion of my project depends on the use of some means of Machine Learning to, in some manner, act as an actor in the game environment, however, I do not know of my options under the bare-bones environment of a command line game; I am more familiar with the use of pixel data and a neural network for action selection on a frame-by-frame basis.&lt;/p&gt;&#xA;&lt;p&gt;Could I use reinforcement learning to learn a policy for action selection under a custom environment or should I apply a machine learning algorithm to character data, find the example outlined below, determine the best action to use on a particular turn state?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+&#xA;| Level | CurrHP | CurrMP | AtkStat | MagStat | StatusList | TargetsInEnemyParty | IsInCrisis | TurnNumber   | BattleParams       |ActionOutput  | DamageOutput |&#xA;+-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+&#xA;|    65 |   6500 |    320 |      47 |      56 |          0 | Behemoth            |0           | 7            | None               |ThiefAttack   |4254          |&#xA;|    92 |   8000 |    250 |      65 |      32 |          0 | Behemoth            |1           | 4            | None               |WarriorLimit  |6984          |&#xA;+-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to prioritise the ease of implementation of an algorithm over how optimal the potential algorithm could be, I just need a baseline to work towards. Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="37983" LastActivityDate="2020-06-17T22:40:04.380" Title="What would be the good choice of algorithm to use for character action selection in an RPG, implemented in Python?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;getting-started&gt;&lt;gaming&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21981" PostTypeId="2" ParentId="16824" CreationDate="2020-06-18T00:49:40.253" Score="0" Body="&lt;p&gt;In Convolutional Neural Networks (CNNs) you have small kernels (or filters) that you slide over an input (e.g. image). The value resulting from the convolution of the filter with a subset of the image over which the filter is currently positioned is then put into its respective cell in the output of that layer. Essentially, training CNNs boils down to training small filters, for example for detecting edges or corners etc. in input data, which most frequently happens to be images indeed. The assumption here is that features can be detected &lt;em&gt;locally&lt;/em&gt; in the input volume, which entails that the nature of the input data shall be coherent over the entire volume of input data.&lt;/p&gt;&#xA;&lt;p&gt;Recurrent Neural Networks (RNNs) do not work locally, but are applied to sequences of arbitrary input data, where one input node may receive sensor readings, while the next node receives the date on which the sensor reading was measured. Of such arbitrary data, you feed a sequence through the RNN, which always keeps its own internal state from processing the previous instance/sample in the sequence in memory while processing the next data point/sample in the sequence. Depending on the kind of recurrent cell type that is employed to construct a RNN layer, the memory of the previous internal state then affects the computation of the next internal state and/or output computed when working on the next data sample. So, information of past data points/samples is carried forward while iterating though a sequence.&lt;/p&gt;&#xA;&lt;p&gt;In short, CNNs are meant to detect local features in volume data, while RNNs preserve information over their previous internal state while processing the next data sample.&lt;/p&gt;&#xA;&lt;p&gt;Probably one of the best online resources walking you through all the related concepts step by step is the following &lt;a href=&quot;https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture series&lt;/a&gt; offered by the Stanford University.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-06-18T00:55:31.880" LastActivityDate="2020-06-18T00:55:31.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21982" PostTypeId="1" AcceptedAnswerId="22447" CreationDate="2020-06-18T01:18:03.113" Score="2" ViewCount="61" Body="&lt;p&gt;I am trying to understand &lt;a href=&quot;https://arxiv.org/pdf/1703.01365.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intagrated Gradients&lt;/a&gt;, but have difficulty in understanding the authors' claim (in section 3, page 3):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For most deep networks, it is possible to choose a baseline such that the prediction at the baseline is near zero (&lt;span class=&quot;math-container&quot;&gt;$F(x') \approx 0$&lt;/span&gt;). (For image models, the black image baseline indeed satisfies this property.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They are talking about a function &lt;span class=&quot;math-container&quot;&gt;$F : R^n \rightarrow [0, 1]$&lt;/span&gt; (in 2nd paragraph of section 3), and if you consider a deep learning classification model, the final layer would be a softmax layer. Then, I suspect for image models, the prediction at the baseline should be close to &lt;span class=&quot;math-container&quot;&gt;$1/k$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of categories. For CIFAR10 and MNIST, this would equal to &lt;span class=&quot;math-container&quot;&gt;$1/10$&lt;/span&gt;, which is not very close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. I have a binary classification model on which I am interested in applying the Integrated Gradients algorithm. Can the baseline output of &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; be a problem?&lt;/p&gt;&#xA;&lt;p&gt;Another related question is, why did they choose a black image as the baseline in the first place? The parameters in image classification models (in a convolution layer) are typically initialized around &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and the input is also normalized. Therefore, image classification models do not really care about the sign of inputs. I mean we could multiply all the training and test inputs with &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt;, and the model would learn the task equivalently. I guess I can find other neutral images other than a black one. I suppose we could choose a white image as the baseline, or maybe the baseline should be all zero after normalization?&lt;/p&gt;&#xA;" OwnerUserId="37987" LastEditorUserId="37987" LastEditDate="2020-06-22T04:29:21.547" LastActivityDate="2020-07-10T18:48:29.950" Title="Why should the baseline's prediction be near zero, according to the Integrated Gradients paper?" Tags="&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21983" PostTypeId="2" ParentId="21972" CreationDate="2020-06-18T01:48:25.027" Score="5" Body="&lt;p&gt;There isn't any explicit relation between the batch size and the gradient accumulation steps, except for the fact that gradient accumulation helps one to fit models with relatively larger batch sizes (typically in single-GPU setups) by cleverly avoiding memory issues. The core idea of gradient accumulation is to perform multiple backward passes &lt;em&gt;using the same model parameters&lt;/em&gt; before updating them all at once for multiple batches. This is unlike the conventional manner, where the model parameters are updated &lt;em&gt;once every&lt;/em&gt; batch-size number of samples. Therefore, finding the correct batch-size and accumulation steps is a design trade-off that has to be made based on two things: (i) how much increase in the batch-size can the GPU handle, and (ii) whether the gradient accumulation steps result in at least as much better performance than without accumulation.&lt;/p&gt;&#xA;&lt;p&gt;As for your example configurations, there are the same &lt;em&gt;in theory&lt;/em&gt;. But, there are a few important caveats that need to be addressed before proceeding with this intuition.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using Batch Normalization with gradient accumulation generally does not work well, simply because BatchNorm statistics cannot be accumulated. A better solution would be to use Group Normalization instead of BatchNorm.&lt;/li&gt;&#xA;&lt;li&gt;When performing a combined update in gradient accumulation, it must be ensured that the optimizer is not initialized to zero (i.e. &lt;code&gt;optimizer.zero_grad()&lt;/code&gt;) for every backward update (i.e. &lt;code&gt;loss.backward()&lt;/code&gt;). It is easy to include both statements in the same &lt;code&gt;for&lt;/code&gt; loop while training which defeats the purpose of gradient accumulation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here are some interesting resources to find out more about it detail.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/huggingface/training-larger-batches-practical-tips-on-1-gpu-multi-gpu-distributed-setups-ec88c3e51255&quot; rel=&quot;noreferrer&quot;&gt;Thomas Wolf's article on different ways of combating memory issues&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Kaggle discussions on the effect on learning rate: &lt;a href=&quot;https://www.kaggle.com/c/cdiscount-image-classification-challenge/discussion/40780&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.kaggle.com/c/understanding_cloud_organization/discussion/105614&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/62067400/understanding-accumulated-gradients-in-pytorch&quot;&gt;A comprehensive answer on gradient accumulation in PyTorch&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36971" LastActivityDate="2020-06-18T01:48:25.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21984" PostTypeId="1" AcceptedAnswerId="21998" CreationDate="2020-06-18T03:14:34.200" Score="2" ViewCount="746" Body="&lt;p&gt;In a Deep Q-learning algorithm, we perform a batch training every &lt;code&gt;train_freq&lt;/code&gt; and we update the parameters of the target network every &lt;code&gt;target_update_freq&lt;/code&gt;. Are &lt;code&gt;train_freq&lt;/code&gt; and &lt;code&gt;target_update_freq&lt;/code&gt; necessary related, e.g., one should be always greater than the other, or must they be independently optimized depending on the problem?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Changed the name of &lt;code&gt;batch_freq&lt;/code&gt; to &lt;code&gt;train_freq&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37642" LastEditorUserId="37642" LastEditDate="2020-06-18T14:04:38.407" LastActivityDate="2020-06-18T16:25:42.043" Title="In Deep Q-learning, are the target update frequency and the batch training frequency related?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21988" PostTypeId="2" ParentId="21975" CreationDate="2020-06-18T07:33:07.660" Score="0" Body="&lt;p&gt;Define features that describe C1 and C2 (like their ids) and add these features to inputs of your model. Thus your only model will be able to generate the next line for any pair of (C1, C2).&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-18T07:33:07.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21989" PostTypeId="1" CreationDate="2020-06-18T07:56:38.673" Score="0" ViewCount="47" Body="&lt;p&gt;I am doing human activity recognition project. I have total of 12 classes. The class distribution look like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z7iTi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z7iTi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\color{red}{If \ you \ watch \ carefully, you \ can \ see \ that \ I \ have \ no \ data \ points \ for \ class \ 11 \ and \ class \ 8.}$&lt;/span&gt; Also, the dataset is highly imbalanced. So, I took minimum data points (in this case 2028) for all of the classes. Now my balanced data look like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2Iakj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2Iakj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After doing this it looks like a balance data. &lt;span class=&quot;math-container&quot;&gt;$\color{red}{But \ still, \ I \ think \ it \ not, \ because \ I \ have \ zero \ datapoints \ for \ class \ 11 \ and \ class \ 8}$&lt;/span&gt;. In my opinion the classes are still imbalance.&lt;/p&gt;&#xA;&lt;p&gt;I am using CNN model to solve this activity project. My model summary is following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/abLdb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/abLdb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The main problem is, my model starts overfitting heavily when I train it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GPX9n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GPX9n.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it due to my imbalance data( class 8 and 11 has zero data points) or something else?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{Hyperperameter:}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{features:}$&lt;/span&gt; X, Y, Z of mobile accelerometer&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{frame size:}$&lt;/span&gt; 80&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{optimizer:}$&lt;/span&gt; Adam, &lt;span class=&quot;math-container&quot;&gt;$\textbf{Learning rate:}$&lt;/span&gt; 0.001&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{Loss:}$&lt;/span&gt; Sparse categorical cross-entropy&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-06-18T08:23:16.440" Title="Can imbalance data create overfitting?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21990" PostTypeId="1" CreationDate="2020-06-18T08:12:55.917" Score="0" ViewCount="27" Body="&lt;p&gt;I have read that propositional resolution works best for formulas in CNF but does it also work with DNF?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if &lt;strong&gt;yes&lt;/strong&gt; how?&lt;/li&gt;&#xA;&lt;li&gt;if &lt;strong&gt;no&lt;/strong&gt; why not?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15391" LastActivityDate="2020-06-18T08:12:55.917" Title="Can propositional resolution be done with DNF?" Tags="&lt;logic&gt;&lt;inference&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21991" PostTypeId="2" ParentId="21989" CreationDate="2020-06-18T08:23:16.440" Score="1" Body="&lt;p&gt;Your zero-classes just do not exist for the model. There is no information about them in training and test sets. I think the reason for overfitting is that your have very small size of the training set comparing to the number of parameters in your model. If it is easier for a model to remember all training entries - it will just do that. You need to prune your model to force it to make some generalizations about the data.&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-18T08:23:16.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21992" PostTypeId="1" AcceptedAnswerId="21996" CreationDate="2020-06-18T09:29:59.673" Score="7" ViewCount="387" Body="&lt;p&gt;I want to know if there is any metric to use for measuring sample-efficiency of a reinforcement learning algorithm? From reading research papers, I see claims that proposed models are more sample efficient but how does one reach this conclusion when comparing reinforcement learning algorithms?&lt;/p&gt;&#xA;" OwnerUserId="30174" LastActivityDate="2020-06-18T15:25:59.703" Title="How to measure sample efficiency of a reinforcement learning algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;sample-efficiency&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21993" PostTypeId="1" CreationDate="2020-06-18T10:47:33.567" Score="0" ViewCount="30" Body="&lt;p&gt;How do I calculate the points in a video sequence where the images are steady enough for a photo.  For example, I want to take maybe 20 photos for a facial recognition dataset.  Instead of asking the subject to hold still 20 times, I take a moving video of him and filter the images for 20 good photos.  The problem is that the subject face has to move into different poses and facial expressions and many times the movements blur the quality of the image.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="13058" LastEditorUserId="13058" LastEditDate="2020-06-19T00:39:32.757" LastActivityDate="2020-06-19T00:39:32.757" Title="How to determine when the image is steady enough in a video sequence to take photos?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21994" PostTypeId="1" AcceptedAnswerId="22013" CreationDate="2020-06-18T11:46:52.657" Score="1" ViewCount="47" Body="&lt;p&gt;For a kernel function, we have two conditions one is that it should be symmetric which is easy to understand intuitively because dot products are symmetric as well and our kernel should also follow this. The other condition is given below&lt;/p&gt;&#xA;&lt;p&gt;There exists a map &lt;span class=&quot;math-container&quot;&gt;$φ:R^d→H$&lt;/span&gt; called kernel feature map into some high dimensional feature space H such that &lt;span class=&quot;math-container&quot;&gt;$∀x,x'$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$R^d:k(x,x') = &amp;lt;φ(x),φ(x')&amp;gt;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I understand that this means that there should exist a feature map that will project the data from low dimension to any high dimension &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and kernel function will take the dot product in that space.&lt;/p&gt;&#xA;&lt;p&gt;For example, the Euclidean distance is given as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$d(x,y)=∑_i(x_i−y_i)^2=&amp;lt;x,x&amp;gt;+&amp;lt;y,y&amp;gt;−2&amp;lt;x,y&amp;gt;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I look this in terms of second condition how do we know that doesn't exist any feature map for euclidean distance? What exactly are we looking in feature maps mathematically?&lt;/p&gt;&#xA;" OwnerUserId="38000" LastEditorUserId="37965" LastEditDate="2020-06-18T14:38:49.487" LastActivityDate="2020-07-19T08:02:30.257" Title="How to understand mapping function of kernel?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;support-vector-machine&gt;&lt;mapping-space&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21995" PostTypeId="1" CreationDate="2020-06-18T12:52:03.177" Score="0" ViewCount="28" Body="&lt;p&gt;I have a bit of experience with training and using networks using tensorflow and pytorch.&lt;/p&gt;&#xA;&lt;p&gt;What I'd like to experiment with is to train networks for which the inference of the trained network can be implemented in pixel shaders, so that it can be applied as 'outputfilter' to a realtime game (for example, in a VR game this must happen like 180 times per second on 2k x 2k image buffers).&lt;/p&gt;&#xA;&lt;p&gt;For that it's not practical to output each frame to some separate inference engine, apply it and then send it back to the OpenGL or DirectX context.&lt;/p&gt;&#xA;&lt;p&gt;Which kind of networks are well suited for having the inference step directly implemented as GLSL or HLSL shaders?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastActivityDate="2020-06-18T12:52:03.177" Title="What set of deep learning networks is well suited for realtime inference in OpenGL and/or DirectX?" Tags="&lt;real-time&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21996" PostTypeId="2" ParentId="21992" CreationDate="2020-06-18T12:54:32.147" Score="5" Body="&lt;p&gt;First of all, let's recall some definitions.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;em&gt;sample&lt;/em&gt; in the context of (deep) RL is a tuple &lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, r_t, s_{t+1})$&lt;/span&gt; representing the information associated with a single interaction with the environment.&lt;/p&gt;&#xA;&lt;p&gt;As for &lt;em&gt;sample efficiency&lt;/em&gt;, it is defined as follows &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1364661319300610&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sample efficiency refers to the amount of data required for a learning system to attain any chosen target level of performance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, how you measure it is closely related to the way it is defined.&lt;/p&gt;&#xA;&lt;p&gt;For example, one way to do it would be as shown in the figure below:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On the y-axis you have the performance of your RL algorithm (e.g. in terms of average return over episodes as done in &lt;a href=&quot;https://arxiv.org/pdf/1611.02247.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;, or mean total episode reward across different environment runs as done in &lt;a href=&quot;https://papers.nips.cc/paper/8044-sample-efficient-reinforcement-learning-with-stochastic-ensemble-value-expansion.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[3]&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On the x-axis you have the number of samples that you took.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The dashed line corresponds to your performance baseline (e.g. the performance at which a certain game or any other RL environment is considered solved).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, you can measure sample efficiency at the intersection, where you get the number of samples needed to reach the performance baseline. So, an algorithm that requires less samples would be more sample-efficient.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rqce9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rqce9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another way to do it would be the other way around, i.e. the RL agent is provided with a limited &lt;em&gt;budget&lt;/em&gt; for the number of samples it could take. As a result, you can measure sample efficiency my measuring the area under the curve, as illustrated below. So, that would be how much performance you got just by using those samples in the budget. An algorithm that achieves a higher performance than another with the same amount of samples would then be more sample efficient.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3MIDR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3MIDR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not aware if there exist RL libraries that would provide you with this measure out-of-the-box. However, if you're using Python for example, I believe that using libraries like scipy or scikit-learn along with matplotlib could do the job.&lt;/p&gt;&#xA;&lt;p&gt;NB: Image credits go to the following presentation: &lt;a href=&quot;https://www.youtube.com/watch?v=DT2MbIeec4U&quot; rel=&quot;nofollow noreferrer&quot;&gt;DLRLSS 2019 - Sample Efficient RL - Harm Van Seijen&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34010" LastEditorUserId="34010" LastEditDate="2020-06-18T15:25:59.703" LastActivityDate="2020-06-18T15:25:59.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21997" PostTypeId="1" AcceptedAnswerId="22004" CreationDate="2020-06-18T14:46:48.870" Score="0" ViewCount="45" Body="&lt;p&gt;Let's say an RL trading system places trades based on pricing data.&lt;/p&gt;&#xA;&lt;p&gt;Each episode represents 1 hour of trading, and there are 24 hours of data available. The Q table represents for a given state, what is the most action with the highest utility.&lt;/p&gt;&#xA;&lt;p&gt;The state is a sequence of prices, and the action is either buy, hold, sell.&lt;/p&gt;&#xA;&lt;p&gt;Instead of &amp;quot;Loop for each episode&amp;quot; as per the Sarsa algorithm :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tG9O1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tG9O1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I add an additional outer loop. Now instead of just looping for each episode we have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for 1 to N:     &#xA;   &amp;quot;Loop for each episode&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Manually set N or exit out of the loop on convergence.&lt;/p&gt;&#xA;&lt;p&gt;Is this the correct approach? Iterating multiple times over the episodes will produce more valuable state-action pairs in the Q table, because e greedy is not deterministic and for each iteration may exploit an action to greater reward than other episode epochs.&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="12964" LastEditDate="2020-06-18T15:40:34.977" LastActivityDate="2020-06-18T18:48:52.900" Title="Looping over Sarsa algorithm for better Q values" Tags="&lt;reinforcement-learning&gt;&lt;algorithmic-trading&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21998" PostTypeId="2" ParentId="21984" CreationDate="2020-06-18T14:56:16.307" Score="2" Body="&lt;p&gt;It is fairly common in DQN to train a minibatch after &lt;em&gt;every&lt;/em&gt; observation received after the replay memory has enough data (how much is enough is yet another parameter). This is not necessary, and it is fine to collect more data between training steps, the alogrithm is still DQN. The value higher than 1 for &lt;code&gt;train_freq&lt;/code&gt; here might be related to use of prioritised replay memory sampling - I have no real experience with that.&lt;/p&gt;&#xA;&lt;p&gt;The update to target network generally needs to occur less frequently than training steps, it is intended to stabilise results numerically, so that over or under estimates of value functions do not result in runaway feedback.&lt;/p&gt;&#xA;&lt;p&gt;The parameters choices &lt;em&gt;will&lt;/em&gt; interact each other, most hyperparameters in machine learning do so unfortunately. Which makes searching for ideal values fiddly and time-consuming.&lt;/p&gt;&#xA;&lt;p&gt;In this case it is safe to say that &lt;code&gt;train_freq&lt;/code&gt; is expected to be much lower than &lt;code&gt;target_update_freq&lt;/code&gt;, probably by at least an order of magnitude, and more usually 2 or 3 orders of magnitude. However, that's not quite the same as saying there is a strong relationship between choices for those two hyperparameters. The value of &lt;code&gt;batch_size&lt;/code&gt; is also relevant here, as it shows the rate that memory is being used (and re-used) by the training process.&lt;/p&gt;&#xA;&lt;p&gt;The library you are using has these defaults:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    batch_size::Int64 = 32&#xA;    train_freq::Int64 = 4&#xA;    target_update_freq::Int64 = 500&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;They seem like sane starting points. You are relatively free to change them as if they were independent, as there is no simple rule like &amp;quot;target_update_freq should be 125 times train_freq&amp;quot;. As a very rough guide, you can expect that high values of &lt;code&gt;train_freq&lt;/code&gt;, low values of &lt;code&gt;batch_size&lt;/code&gt; and low values of &lt;code&gt;target_update_freq&lt;/code&gt; are likely to cause instability in the learning process, whilst going too far in the opposite direction may slow learning down. You might be able to set &lt;code&gt;train_freq&lt;/code&gt; to 1, but I am not completely certain about that either in combination with the prioritised replay memory sampling which seems to be the default in the library you are using.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-18T16:25:42.043" LastActivityDate="2020-06-18T16:25:42.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21999" PostTypeId="1" AcceptedAnswerId="22000" CreationDate="2020-06-18T17:13:00.543" Score="2" ViewCount="300" Body="&lt;p&gt;Typically, people say that convolutional neural networks (CNN) perform the convolution operation, hence their name. However, some people have also said that a CNN actually performs the cross-correlation operation rather than the convolution. How is that? Does a CNN perform the convolution or cross-correlation operation? What is the difference between the convolution and cross-correlation operations?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T10:00:19.323" LastActivityDate="2020-06-21T12:26:57.667" Title="Do convolutional neural networks perform convolution or cross-correlation?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;convolution&gt;&lt;cross-correlation&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22000" PostTypeId="2" ParentId="21999" CreationDate="2020-06-18T17:13:00.543" Score="6" Body="&lt;h1&gt;Short answer&lt;/h1&gt;&#xA;&lt;p&gt;Theoretically, convolutional neural networks (CNNs) can either perform the cross-correlation or convolution: it does not really matter whether they perform the cross-correlation or convolution because the kernels are learnable, so they can adapt to the cross-correlation or convolution given the data, although, in the typical diagrams, CNNs are shown to perform the cross-correlation because (in libraries like TensorFlow) they are typically &lt;em&gt;implemented&lt;/em&gt; with cross-correlations (and cross-correlations are conceptually simpler than convolutions). Moreover, in general, the kernels can or not be symmetric (although they typically won't be symmetric). In the case they are symmetric, the cross-correlation is equal to the convolution.&lt;/p&gt;&#xA;&lt;h1&gt;Long answer&lt;/h1&gt;&#xA;&lt;p&gt;To understand the answer to this question, I will provide two examples that show the similarities and differences between the convolution and cross-correlation operations. I will focus on the convolution and cross-correlation applied to 1-dimensional discrete and finite signals (which is the simplest case to which these operations can be applied) because, essentially, CNNs process finite and discrete signals (although typically higher-dimensional ones, but this answer applies to higher-dimensional signals too). Moreover, in this answer, I will assume that you are at least familiar with how the convolution (or cross-correlation) in a CNN is performed, so that I do not have to explain these operations in detail (otherwise this answer would be even longer).&lt;/p&gt;&#xA;&lt;h2&gt;What is the convolution and cross-correlation?&lt;/h2&gt;&#xA;&lt;p&gt;Both the convolution and the cross-correlation operations are defined as the dot product between a small matrix and different parts of another typically bigger matrix (in the case of CNNs, it is an image or a feature map). Here's the usual illustration (of the cross-correlation, but the idea of the convolution is the same!).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jhb6U.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jhb6U.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Example 1&lt;/h2&gt;&#xA;&lt;p&gt;To be more concrete, let's suppose that we have the output of a function (or signal) &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; grouped in a matrix &lt;span class=&quot;math-container&quot;&gt;$$f = [2, 1, 3, 5, 4] \in \mathbb{R}^{1 \times 5},$$&lt;/span&gt; and the output of a kernel function also grouped in another matrix &lt;span class=&quot;math-container&quot;&gt;$$h=[1, -1] \in \mathbb{R}^{1 \times 2}.$$&lt;/span&gt;  For simplicity, let's assume that we do not pad the input signal and we perform the convolution and cross-correlation with a stride of 1 (I assume that you are familiar with the concepts of padding and stride).&lt;/p&gt;&#xA;&lt;h3&gt;Convolution&lt;/h3&gt;&#xA;&lt;p&gt;Then the &lt;strong&gt;convolution&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, denoted as &lt;span class=&quot;math-container&quot;&gt;$f \circledast h = g_1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt; is the convolution operator, is computed as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f \circledast h = g_1&#xA;&amp;amp;=\\&#xA;[(-1)*2 + 1*1, \\&#xA;(-1)*1 + 1*3, \\&#xA;(-1)*3 + 1*5, \\&#xA;(-1)*5+1*4] &#xA;&amp;amp;=\\ &#xA;[-2 + 1, -1 + 3, -3 + 5, -5 + 4]&#xA;&amp;amp;=\\&#xA;[-1, 2, 2, -1] \in \mathbb{R}^{1 \times 4}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, the convolution of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is computed as a series of element-wise multiplications between the horizontally flipped kernel &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;, and each &lt;span class=&quot;math-container&quot;&gt;$1 \times 2$&lt;/span&gt; window of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, each of which is followed by a summation (i.e. a dot product). This follows from the definition of convolution (which I will not report here).&lt;/p&gt;&#xA;&lt;h3&gt;Cross-correlation&lt;/h3&gt;&#xA;&lt;p&gt;Similarly, the &lt;strong&gt;cross-correlation&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, denoted as &lt;span class=&quot;math-container&quot;&gt;$f \otimes h = g_2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\otimes$&lt;/span&gt; is the cross-correlation operator, is also defined as a dot product between &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and different parts of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, but without flipping the elements of the kernel before applying the element-wise multiplications, that is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f \otimes h = g_2&#xA;&amp;amp;=\\&#xA;[1*2 + (-1)*1, \\ &#xA;1*1 + (-1)*3, \\ &#xA;1*3 + (-1)*5, \\ &#xA;1*5 + (-1)*4] &#xA;&amp;amp;=\\ &#xA;[2 - 1, 1 - 3, 3 - 5, 5 - 4]&#xA;&amp;amp;=\\&#xA;[1, -2, -2, 1] \in \mathbb{R}^{1 \times 4}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The only difference between the convolution and cross-correlation operations is that, in the first case, the kernel is flipped (along all spatial dimensions) before being applied.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In both cases, the result is a &lt;span class=&quot;math-container&quot;&gt;$1 \times 4$&lt;/span&gt; vector. If we had convolved &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with a &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; vector, the result would have been a &lt;span class=&quot;math-container&quot;&gt;$1 \times 5$&lt;/span&gt; vector. Recall that we assumed no padding (i.e. we don't add dummy elements to the left or right borders of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;) and stride 1 (i.e. we shift the kernel to the right one element at a time). Similarly, if we had convolved &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with a &lt;span class=&quot;math-container&quot;&gt;$1 \times 3$&lt;/span&gt;, the result would have been a &lt;span class=&quot;math-container&quot;&gt;$1 \times 3$&lt;/span&gt; vector (as you will see from the next example).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The results of the convolution and cross-correlation, &lt;span class=&quot;math-container&quot;&gt;$g_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g_2$&lt;/span&gt;, are different. Specifically, one is the negated version of the other. So, the result of the convolution is generally different than the result of the cross-correlation, given the same signals and kernels (as you might have suspected).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;Example 2: symmetric kernel&lt;/h2&gt;&#xA;&lt;p&gt;Now, let's convolve &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with a &lt;span class=&quot;math-container&quot;&gt;$1 \times 3$&lt;/span&gt; kernel that is symmetric around the middle element, &lt;span class=&quot;math-container&quot;&gt;$h_2 = [-1, 2, -1]$&lt;/span&gt;. Let's first compute the convolution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f \circledast h_2 = g_3&#xA;&amp;amp;=\\&#xA;[(-1)*2 + 1*2 + (-1) * 3,\\ (-1)*1 + 2*3 + (-1) * 5,\\ (-1)*3 + 2*5 + (-1) * 4] &#xA;&amp;amp;=\\ &#xA;[-2 + 2 + -3, -1 + 6 + -5, -3 + 10 + -4] &#xA;&amp;amp;=\\&#xA;[-3, 0, 3]&#xA;\in \mathbb{R}^{1 \times 3}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, let's compute the cross-correlation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f \otimes h_2 = g_4&#xA;&amp;amp;=\\&#xA;[(-1)*2 + 1*2 + (-1) * 3, \\ (-1)*1 + 2*3 + (-1) * 5, \\ (-1)*3 + 2*5 + (-1) * 4] &#xA;&amp;amp;=\\&#xA;[-3, 0, 3]&#xA;\in \mathbb{R}^{1 \times 3}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yes, that's right! In this case, the result of the convolution and the cross-correlation is the same. This is because the kernel is symmetric around the middle element. This result applies to any convolution or cross-correlation in any dimension. For example, the convolution of the 2d Gaussian kernel (a centric-symmetric kernel) and a 2d image is equal to the cross-correlation of the same signals.&lt;/p&gt;&#xA;&lt;h2&gt;CNNs have learnable kernels&lt;/h2&gt;&#xA;&lt;p&gt;In the case of CNNs, the kernels are the learnable parameters, so we do not know beforehand whether the kernels will be symmetric or not around their middle element. They won't probably be. In any case, CNNs can perform either the cross-correlation (i.e. no flip of the filter) or convolution: it does not really matter if they perform cross-correlation or convolution because the filter is learnable and can adapt to the data and tasks that you want to solve, although, in the visualizations and diagrams, CNNs are typically shown to perform the cross-correlation (but this does not have to be the case in practice).&lt;/p&gt;&#xA;&lt;h2&gt;Do libraries implement the convolution or correlation?&lt;/h2&gt;&#xA;&lt;p&gt;In practice, certain libraries provide functions to compute both convolution and cross-correlation. For example, NumPy provides both the functions &lt;a href=&quot;https://numpy.org/doc/stable/reference/generated/numpy.convolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;convolve&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://numpy.org/doc/stable/reference/generated/numpy.correlate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;correlate&lt;/code&gt;&lt;/a&gt; to compute both the convolution and cross-correlation, respectively. If you execute the following piece of code (Python 3.7), you will get results that are consistent with my explanations above.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np &#xA;&#xA;f = np.array([2., 1., 3., 5., 4.])&#xA;&#xA;h = np.array([1., -1.])&#xA;h2 = np.array([-1., 2., 1.])&#xA;&#xA;g1 = np.convolve(f, h, mode=&amp;quot;valid&amp;quot;)&#xA;g2 = np.correlate(f, h, mode=&amp;quot;valid&amp;quot;)&#xA;&#xA;print(&amp;quot;g1 =&amp;quot;, g1) # g1 = [-1.  2.  2. -1.]&#xA;print(&amp;quot;g2 =&amp;quot;, g2) # g2 = [ 1. -2. -2.  1.]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, NumPy is not really a library that provides &lt;a href=&quot;https://numpy.org/doc/stable/reference/generated/numpy.correlate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;out-of-the-box&lt;/a&gt; functionality to build CNNs.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, TensorFlow's and &lt;a href=&quot;https://pytorch.org/docs/master/generated/torch.nn.Conv1d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch's&lt;/a&gt; functions to build the convolutional layers actually perform cross-correlations. As I said above, although it does not really matter whether CNNs perform the convolution or cross-correlation, this naming is misleading. Here's a proof that TensorFlow's &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/nn/conv1d&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tf.nn.conv1d&lt;/code&gt;&lt;/a&gt; actually implements the cross-correlation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf # TensorFlow 2.2&#xA;&#xA;f = tf.constant([2., 1., 3., 5., 4.], dtype=tf.float32)&#xA;h = tf.constant([1., -1.], dtype=tf.float32)&#xA;&#xA;# Reshaping the inputs because conv1d accepts only certain shapes.&#xA;f = tf.reshape(f, [1, int(f.shape[0]), 1])&#xA;h = tf.reshape(h, [int(h.shape[0]), 1, 1])&#xA;&#xA;g = tf.nn.conv1d(f, h, stride=1, padding=&amp;quot;VALID&amp;quot;)&#xA;print(&amp;quot;g =&amp;quot;, g) # [1, -2, -2, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;After having written this answer, I found the article &lt;a href=&quot;https://glassboxmedicine.com/2019/07/26/convolution-vs-cross-correlation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolution vs. Cross-Correlation&lt;/a&gt; (2019) by Rachel Draelos, which essentially says the same thing that I am saying here, but provides more details and examples.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-21T12:26:57.667" LastActivityDate="2020-06-21T12:26:57.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22001" PostTypeId="1" AcceptedAnswerId="22002" CreationDate="2020-06-18T18:26:54.243" Score="0" ViewCount="125" Body="&lt;p&gt;While I was playing with some hyperparameters, I came to a wired situation. My dataset is IRIS dataset to be specific. SVM algorithm has some hyperparameters that we can tune, such as Kernels, and C value. &lt;a href=&quot;https://i.stack.imgur.com/gEq4T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gEq4T.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(All accuracy calculations and SVM are from sklearn package to be specific)&lt;/p&gt;&#xA;&lt;p&gt;I made a comparison between kernels and noticed sigmoid kernel was performing way worse in terms of accuracy. It is more than 3 times less accuracy than RBF, Linear, and Polynomial. I do know that kernels are quite data-sensitive and data-specific, but I would like to know &lt;strong&gt;&amp;quot;Which types of data is sigmoid kernel good at any example? or is this my fault due to wrong C value for sigmoid kernel?&amp;quot;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="38008" LastEditorUserId="38008" LastEditDate="2020-06-19T01:46:19.417" LastActivityDate="2020-06-22T12:37:33.090" Title="Which kind of data does sigmoid kernel performance well?" Tags="&lt;algorithm&gt;&lt;performance&gt;&lt;support-vector-machine&gt;&lt;accuracy&gt;&lt;sigmoid&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22002" PostTypeId="2" ParentId="22001" CreationDate="2020-06-18T18:30:59.357" Score="0" Body="&lt;p&gt;The sigmoid kernel is better suited for binary classifications. As the IRIS dataset is for multi-class classification, its performance was not as good as other kernels. You can train with only 2 types of flowers to see if the sigmoid kernel can perform well.&lt;/p&gt;&#xA;" OwnerUserId="36446" LastEditorUserId="2444" LastEditDate="2020-06-22T12:37:33.090" LastActivityDate="2020-06-22T12:37:33.090" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22003" PostTypeId="1" AcceptedAnswerId="22005" CreationDate="2020-06-18T18:39:43.483" Score="1" ViewCount="54" Body="&lt;p&gt;I am trying to learn the theory behind first-order logic (FOL) and do some practice runs of converting statements into the form of FOL.&lt;/p&gt;&#xA;&lt;p&gt;One issue I keep running into is hesitating on whether to use an AND (&lt;span class=&quot;math-container&quot;&gt;$\land$&lt;/span&gt;) statement or an IMPLIES (&lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt;) statement.&lt;/p&gt;&#xA;&lt;p&gt;I have seen examples such as &amp;quot;Some boys are intelligent&amp;quot; turned into:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\exists x \text{boys}(x) \land \text{intelligent}(x)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can I make a general assumption that when I see &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is/are &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, I can use an AND?&lt;/p&gt;&#xA;&lt;p&gt;With a statement such as &amp;quot;All movies directed by M. Knight Shamalan have a supernatural character&amp;quot;, I feel that that statement can be translated to either:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall x, \exists y \; \text{directed}(\text{Shamalan}, x) \rightarrow \text{super-natural character}(y)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall x, \exists y \; \text{directed}(\text{Shamalan}, x) \land \text{super-natural character}(y)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a better way to distinguish between when to use one or the other?&lt;/p&gt;&#xA;" OwnerUserId="38009" LastEditorUserId="2444" LastEditDate="2020-06-19T10:42:41.240" LastActivityDate="2020-06-19T10:42:41.240" Title="When to use AND and when to use Implies in first-order logic?" Tags="&lt;proofs&gt;&lt;logic&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22004" PostTypeId="2" ParentId="21997" CreationDate="2020-06-18T18:48:52.900" Score="1" Body="&lt;p&gt;Given that your state space is continuous, then I would recommend using &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-Learning&lt;/a&gt;. As you say, running several episodes will definitely be beneficial so that the agent is able to explore the space more thoroughly.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-06-18T18:48:52.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22005" PostTypeId="2" ParentId="22003" CreationDate="2020-06-18T23:14:33.103" Score="0" Body="&lt;p&gt;When considering the sentence &amp;quot;Some boys are intelligent&amp;quot;, it makes sense to express it by &lt;code&gt;∃x boys(x) ∧ intelligent(x)&lt;/code&gt;. This is because the existential quantifier makes sure to express that at least some, but not necessarily all boys are intelligent. More specifically, you say that there exists something which is a boy and which is intelligent. But if such a something exists, then there are some boys which are intelligent. Your statement is satisfied.&lt;/p&gt;&#xA;&lt;p&gt;Using a simple implication &lt;code&gt;→&lt;/code&gt; here wouldn't work &lt;em&gt;semantically&lt;/em&gt; in that context. For example, &lt;code&gt;∃x boys(x) → intelligent(x)&lt;/code&gt; expresses that: &lt;code&gt;For some x, if x is a boy, then x is intelligent&lt;/code&gt;. By rewriting, we would get &lt;code&gt;∃x ¬boys(x) ∨ intelligent(x)&lt;/code&gt;. This statement would even be true if all &lt;code&gt;x&lt;/code&gt; were not boys, because &lt;code&gt;¬boys(x)&lt;/code&gt; would be true in these cases. So, &lt;code&gt;∃x boys(x) → intelligent(x)&lt;/code&gt; would hold as soon as something in your model wasn't a boy. In that case, &lt;code&gt;∃x boys(x) → intelligent(x)&lt;/code&gt; would be true, but &amp;quot;Some boys are intelligent&amp;quot; would not really be satisfied, since there are simply no boys in your model which could be intelligent. So, you will have to resort to something like &lt;code&gt;∃x boys(x) ∧ intelligent(x)&lt;/code&gt;, since otherwise there is no guarantee that boys (even intelligent ones) are present in the resulting model.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-06-18T23:14:33.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22007" PostTypeId="2" ParentId="21999" CreationDate="2020-06-19T04:59:34.363" Score="0" Body="&lt;p&gt;Just as a short and quick answer to build of nbros:&lt;/p&gt;&#xA;&lt;p&gt;The way CNNs are typically taught, they are taught using a correlation on the forward pass, rather than a convolution. In reality, Convolutional neural networks is a bit of a misleading name, but not entirely incorrect.&lt;/p&gt;&#xA;&lt;p&gt;CNNs do in fact use convolutions every time they are trained and run. If a &lt;em&gt;correlation&lt;/em&gt; is used on the forward pass, a &lt;em&gt;convolution&lt;/em&gt; is used on the backward pass. The opposite is true if a convolution is used on the forward pass (which is equally valid as using a correlation). I couldn't seem to find this information anywhere, so had to learn it myself the hard way.&lt;/p&gt;&#xA;&lt;p&gt;So to summarise, a typical CNN goes like this: Correlation forward, convolution backward.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-06-19T04:59:34.363" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22008" PostTypeId="2" ParentId="18417" CreationDate="2020-06-19T05:07:49.430" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The ROIs in the input space are mapped to the feature map space, by dividing it by the net stride at that layer. Say, in a network, after a sequence of four 2x2 pooling layers, your image is reduced to 1/16 of the original size. (A 32*32 image is reduced to 2x2) So, the bounding boxes in the original space are mapped to the feature space by dividing by the net stride, which is 16 here. But here's the catch, the ROI co-ordinates could be a floating point number when divided by 16, so, it is adjusted by either flooring it or ceiling it. This is why ROIPooling is &lt;strong&gt;quantized&lt;/strong&gt;. There is a loss of information when you are rounding off the co-ordinates of the ROI. Nevertheless, now, each region is pooled to a single-size feature map, and &lt;strong&gt;each ROI is fed one-by-one to the following layers&lt;/strong&gt;.&#xA;The &lt;a href=&quot;https://arxiv.org/abs/1703.06870&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mask-RCNN paper&lt;/a&gt; brings a change to the mapping to the feature space, by not rounding off the co-ordinates, and by &lt;a href=&quot;https://en.wikipedia.org/wiki/Bilinear_interpolation#:%7E:text=In%20mathematics%2C%20bilinear%20interpolation%20is,again%20in%20the%20other%20direction.&quot; rel=&quot;nofollow noreferrer&quot;&gt;bilinear interpolation&lt;/a&gt;, and the loss of information is reduced, thus, the ROIAlign algorithm(which has been described here) performs better at object detection, than the quantized ROIPool algorithm.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As said, each fixed-size feature map vector is processed one-by-one.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/understanding-region-of-interest-part-2-roi-align-and-roi-warp-f795196fc193&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reference&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36474" LastActivityDate="2020-06-19T05:07:49.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22010" PostTypeId="1" CreationDate="2020-06-19T06:48:17.510" Score="4" ViewCount="180" Body="&lt;p&gt;Currently, I'm only going through these two books&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction, by Sutton and Barto&lt;/a&gt;: RL explained on an engineering level (mathematical, but readable for a non-mathematician). Elementary notions from probability and statistics are required (conditional probability, total probability theorem, total expectation theorem, and similar. The &lt;a href=&quot;https://www.youtube.com/watch?v=1uW3qMFA9Ho&amp;amp;list=PLUl4u3cNGP60hI9ATjSFgLZpbNJ7myAg6&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIT RES.6-012 &amp;quot;Introduction to Probability&amp;quot;&lt;/a&gt; course is a great source of information for these topics.).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.manning.com/books/grokking-deep-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning, by Miguel Morales&lt;/a&gt;: this book introduces the main elements of reinforcement learning in a less formal way than Sutton and Barto (derivations for some equations are not given), using examples to describe the math.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What other introductory books to reinforcement learning do you know, and how do they approach this topic?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="2444" LastEditDate="2021-01-17T19:31:45.157" LastActivityDate="2021-01-17T19:31:45.157" Title="What introductory books to reinforcement learning do you know, and how do they approach this topic?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;books&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22011" PostTypeId="1" CreationDate="2020-06-19T06:52:12.437" Score="0" ViewCount="15" Body="&lt;p&gt;I have always had doubts about the necessity and intuitive/theoretical justification for capsule network in image classification and more recently nlp tasks. For the former, in order to address the global pose recognition problem, can’t we borrow idea from the LSTM/ attention to generalize the highly localized CNN by allowing more direct long Distance interactions between pixels far apart? Attention plus 2d position encoding seems a lot more natural and simpler than the elaborate dynamic routing approach advertised in the original capsule paper. Since I don’t work in the field of cv, this question may be naive, but really piques my mind and several others. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="38015" LastActivityDate="2020-06-19T06:52:12.437" Title="Can attention with 2d position encoding beat capsule on cv tasks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22013" PostTypeId="2" ParentId="21994" CreationDate="2020-06-19T07:36:12.400" Score="1" Body="&lt;p&gt;A kernel function &lt;span class=&quot;math-container&quot;&gt;$f : \mathcal{X} \times \mathcal{X} \rightarrow \mathbb{R}$&lt;/span&gt; is a valid support vector kernel if it is a Mercer kernel. Mercer's condition essentially ensures that the Gram matrix of the kernel is positive semi-definite. Interestingly, this ensures that the SVM objective is convex.&lt;/p&gt;&#xA;&lt;p&gt;The Euclidean distance function does not satisfy Mercer's condition since it's Gram matrix is not necessary positive semi-definite. Thus, it is not a valid kernel.&lt;/p&gt;&#xA;" OwnerUserId="5293" LastActivityDate="2020-06-19T07:36:12.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22014" PostTypeId="2" ParentId="21970" CreationDate="2020-06-19T08:03:09.947" Score="1" Body="&lt;p&gt;Computer vision is a wide field, and besides the fact that deep learning dominates, there are still many, many other algorithms that see widespread use in both academia and industry.&lt;/p&gt;&#xA;&lt;p&gt;For tasks such as image classification / object recognition, the typical paradigm is some CNN architecture such as a ResNet or VGG. There has been lots of works to extend and improves the CNNs, but the basic architecture has not really changed much over the years. Interestingly, there's been some work to encode more complex inductive biases / invariants into the deep learning modelling process, such as Spatial Transformer Networks and Group Equivariant Networks. More classical vision approaches to such problems typically include computing some form of hand-crafted feature (HOG, LBP), and training any off-the-shelf classifier.&lt;/p&gt;&#xA;&lt;p&gt;For object detection, the de-facto for many years was Viola-Jones for it's combination of performance and speed (even though there were more accurate systems at the time, but they were slower). More recently, object detection has been dominated by deep learning, with architectures such as SSD, YOLO, all the RCNN variants, etc.&lt;/p&gt;&#xA;&lt;p&gt;A related problem to object detection is segmentation. Deep learning again dominates in this area with algorithms such as Mask RCNN. However, many other approaches exist and see some use, such as superpixels (e.g. SLIC), watershed, and normalized cuts.&lt;/p&gt;&#xA;&lt;p&gt;For problems such as image search, vision approaches such as Fisher vectors and VLAD (computed from image descriptors such as SIFT or SURF) are still competitive. However, CNN features have also seen use in this domain.&lt;/p&gt;&#xA;&lt;p&gt;For video analysis, CNNs (typically, 3D CNNs) are popular. However, they often leverage other vision techniques such as optical flow. The most popular optical flow algorithms are Brox, TVL-1, KLT, and Farneback. There are more recent approaches which attempt to use deep learning to actually learn the optical flow, though.&lt;/p&gt;&#xA;&lt;p&gt;An overarching set of techniques that has so many varying applications are interest point detectors, image descriptors, and feature encoding techniques. Interest point detectors attempt to localise interest points in an image or video, and popular detectors include Harris, FAST, and MSER. Image descriptors are used to describe those interest points. Example descriptors include SIFT, SURF, KAZE, and ORB. The descriptors themselves can be used to do various things such as estimate homographies using the RANSAC algorithm (for applications such as panorama and camera stabilisation). However, the descriptors can also be encoded and pooled into a single fixed-length feature vector, which serves as the representation of the image. The most common approach to this encoding is bag of feature / bag of visual words. This is based on K-means. However, popular extensions / variants include Fisher vectors and VLAD.&lt;/p&gt;&#xA;&lt;p&gt;Self-supervised and semi-supervised learning is also very popular nowadays in academia, and seeks to get the most of out the abundant unlabelled data. In a computer vision context, popular techniques include MoCo and SimCLR, but new methods are released almost weekly!&lt;/p&gt;&#xA;&lt;p&gt;Another problem domain in computer vision is the ability to generate / synthesize images. The is not unique to computer vision, but the common algorithms for this are variational autoencoders (VAEs) and generative adversarial networks (GANs).&lt;/p&gt;&#xA;" OwnerUserId="5293" LastActivityDate="2020-06-19T08:03:09.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22015" PostTypeId="1" CreationDate="2020-06-19T09:08:16.870" Score="0" ViewCount="80" Body="&lt;p&gt;What would be the effect of normalizing each input patch going to Convolutional layer separately.&lt;/p&gt;&#xA;&lt;p&gt;Let's say our input is 64 channels of the size 224x224 (like is the case for some hidden layers in ResNet). And instead of using Batch Normalization or Instance Normalization, we use &amp;quot;Patch Normalization&amp;quot;. Our Convolutional layer is applying 3x3 kernel to the input. So we take each 64x3x3 input slice, calculate it's mean and std, use it to normalize, and then multiply it by the layer weights.&lt;/p&gt;&#xA;&lt;p&gt;It's rather obvious idea, so I assume it was tested multiple times. But I was unfortunate to find any info about it.&lt;/p&gt;&#xA;" OwnerUserId="31988" LastActivityDate="2020-06-19T09:08:16.870" Title="Normalization of every patch of CNN input separately" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22016" PostTypeId="1" CreationDate="2020-06-19T10:08:09.550" Score="0" ViewCount="29" Body="&lt;p&gt;I was running into a situation in which my input feature experience a very large variation in term of magnitude.&lt;/p&gt;&#xA;&lt;p&gt;Particularly, consider feature 1 belong to group 1 and feature 2 3 4 belong to group 2,&lt;/p&gt;&#xA;&lt;p&gt;Like this picture below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4FHOK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4FHOK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was really worried that in this case &lt;strong&gt;feature 1 might dominate feature 2,3,4 (group 2)&lt;/strong&gt; because its corresponding value is so large (I was trying to train this data set on a neural network).&lt;/p&gt;&#xA;&lt;p&gt;In this situation, what would be the appropriate scaling strategy ?&#xA;Update: I know for sure that the value of feature 1 is an integer that is uniform on the interval [22,42]&#xA;But for feature 2 ,3 ,4 I do not have any insight&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your enthusiast !&lt;/p&gt;&#xA;" OwnerUserId="37297" LastEditorUserId="37297" LastEditDate="2020-06-19T13:35:30.340" LastActivityDate="2021-03-16T14:04:55.497" Title="Feature scaling strategy for many feature with very large variation between them?" Tags="&lt;neural-networks&gt;&lt;features&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22017" PostTypeId="2" ParentId="22016" CreationDate="2020-06-19T10:23:24.577" Score="0" Body="&lt;p&gt;You should check the distribution of each feature and scale them accordingly, but in any case you should aim to roughly the same interval of values for every feature.&#xA;For example, if f1 has the standard distribution and f2 is close to the uniform one, then you can scale f1 to N(0,1) and f2 to U(-1,1).&#xA;In other words, try to have maximum, minimum and mean values for all features as close as possible, while keeping their original distributions.&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-19T10:23:24.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22019" PostTypeId="1" AcceptedAnswerId="22020" CreationDate="2020-06-19T12:00:12.957" Score="1" ViewCount="108" Body="&lt;p&gt;Is the VC dimension meaningful for reinforcement learning (RL), as a machine learning (ML) method? How?&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="2444" LastEditDate="2021-01-22T15:48:00.827" LastActivityDate="2021-01-22T15:48:00.827" Title="Is the VC Dimension meaningful in the context of Reinforcement Learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;computational-learning-theory&gt;&lt;vc-dimension&gt;&lt;vc-theory&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22020" PostTypeId="2" ParentId="22019" CreationDate="2020-06-19T12:00:12.957" Score="2" Body="&lt;p&gt;Yes, it is. &lt;a href=&quot;http://papers.nips.cc/paper/1664-approximate-planning-in-large-pomdps-via-reusable-trajectories.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This article&lt;/a&gt; (Approximate Planning in Large POMDPs via Reusable Trajectories) explain about it by means of the trajectory tree:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A trajectory tree is a binary tree in which each node is labeled by a state and observation&#xA;pair, and has a child for each of the two actions. Additionally, each link to a child is&#xA;labeled by a reward, and the tree's depth will be &lt;span class=&quot;math-container&quot;&gt;$H_\epsilon$&lt;/span&gt;, so it will have about &lt;span class=&quot;math-container&quot;&gt;$2^{H_\epsilon}$&lt;/span&gt; nodes. The root is labeled by &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; and the observation there, &lt;span class=&quot;math-container&quot;&gt;$o_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; will be defined like the following base on the trajectory tree:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For any deterministic strategy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and any trajectory tree &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; defines a path through&#xA;&lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; starts at the root, and inductively, if &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is at some internal node in &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, then we feed to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; the observable history along the path from the root to that node, and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; selects and moves to a child of the current node. This continues until a leaf node is reached, and we define &lt;span class=&quot;math-container&quot;&gt;$R(\pi, T)$&lt;/span&gt; to be the discounted sum of returns along the path taken. In the case that &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is stochastic, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; defines a distribution on paths in &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$R(\pi, T)$&lt;/span&gt; is the expected return according to this distribution.  Hence, given &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; trajectory trees &lt;span class=&quot;math-container&quot;&gt;$T_1 , \ldots , T_m$&lt;/span&gt;, a natural estimate for &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s_0)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s_0) = \frac{1}{m}\sum_{i=1}^mR(\pi, T_i)$&lt;/span&gt;. *Note that each tree can be used to evaluate any&#xA;strategy, much the way a single labeled example &lt;span class=&quot;math-container&quot;&gt;$\langle x, f(x)\rangle$&lt;/span&gt; can be used to evaluate any hypothesis &lt;span class=&quot;math-container&quot;&gt;$h(x)$&lt;/span&gt; in supervised learning. Thus in this sense, trajectory trees are reusable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now similar to definition of VC theory for classification methods:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our goal now is to establish uniform convergence results that bound the error of the estimates &lt;span class=&quot;math-container&quot;&gt;$V^\pi(s_0)$&lt;/span&gt; as a function of the &amp;quot;sample size&amp;quot; (number of trees) &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And finally, we have the following theorem:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt; be any finite class of deterministic strategies for an arbitrary two-action &lt;strong&gt;POMDP&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; trajectory trees be created using a generative model for &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, and&#xA;&lt;span class=&quot;math-container&quot;&gt;$\widehat{V}^\pi(s_0)$&lt;/span&gt; be the resulting estimates. If &lt;span class=&quot;math-container&quot;&gt;$m = O((V_{\max}/\epsilon)^2(\log(|\Pi|) + \log(1/\delta)))$&lt;/span&gt;, then with probability &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$|V^\pi(s_0) - \widehat{V}^\pi(s_0)|\leqslant \epsilon$&lt;/span&gt; holds simultaneously for all &lt;span class=&quot;math-container&quot;&gt;$\pi \in \Pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;About the VC dimension of &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt;, if we suppose we have two actions &lt;span class=&quot;math-container&quot;&gt;$\{a_1, a_2\}$&lt;/span&gt; (it can be generalized to more actions), we can say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt; is a (possibly infinite) set of deterministic strategies, then each strategy &lt;span class=&quot;math-container&quot;&gt;$\pi \in \Pi$&lt;/span&gt; is simply a deterministic function mapping from the set of observable histories to the set &lt;span class=&quot;math-container&quot;&gt;$\{a_1, a_2\}$&lt;/span&gt;,&#xA;and is thus a boolean function on observable histories. We can, therefore, write &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}(\Pi)$&lt;/span&gt; to&#xA;denote the familiar VC dimension of the set of binary functions &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt;. For example, if &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt; is&#xA;the set of all thresholded linear functions of the current vector of observations (a particular&#xA;type of memoryless strategy), then &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}(\Pi)$&lt;/span&gt; simply equals the number of parameters.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and the following theorem:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt; be any class of deterministic strategies for an arbitrary two-action&#xA;POMDP &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, and let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{VC}(\Pi)$&lt;/span&gt; denote its VC dimension. Let &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; trajectory trees be created using a generative model for &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\widehat{V}^\pi(s_0)$&lt;/span&gt; be the resulting estimates. If:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;m = O((V_{\max}/\epsilon)^2(H_\epsilon\mathcal{VC}(\Pi)\log(V_{\max}/\epsilon) + \log(1/\delta)))&#xA;$$&lt;/span&gt;&#xA;then with probability &lt;span class=&quot;math-container&quot;&gt;$1 - \delta$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$|V^\pi(s_0) - \widehat{V}^\pi(s_0)|\leqslant \epsilon$&lt;/span&gt; holds simultaneously for all &lt;span class=&quot;math-container&quot;&gt;$\pi \in \Pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-06-19T13:26:04.673" LastActivityDate="2020-06-19T13:26:04.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22021" PostTypeId="2" ParentId="22016" CreationDate="2020-06-19T12:20:45.990" Score="0" Body="&lt;p&gt;You should normalize every column individually. It will work just fine. Sum up the column and divide every element of that column by sum of that column. But as your feature 2,3,4 are of very small scale you should consider some transformation like log transformation as you might encounter numerical underflow.&lt;/p&gt;&#xA;" OwnerUserId="36446" LastActivityDate="2020-06-19T12:20:45.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22022" PostTypeId="2" ParentId="22010" CreationDate="2020-06-19T13:31:31.670" Score="3" Body="&lt;p&gt;In addition to the ones you mentioned, I would add &lt;a href=&quot;https://sites.ualberta.ca/%7Eszepesva/rlbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithms of Reinforcement Learning&lt;/a&gt; by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example &lt;a href=&quot;https://cs.uwaterloo.ca/%7Eppoupart/teaching/cs885-spring20/textbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;It generally follows the same outline as Sutton &amp;amp; Barto's book (except the part on bandits, it is included in the Chapter on &lt;em&gt;Control&lt;/em&gt;). In fact, it may be considered as a condensed version of Sutton &amp;amp; Barto (about 100 pages). In addition, it's freely available online.&lt;/p&gt;&#xA;&lt;p&gt;I like the author's justification as to why he wrote this book, so I'm just going to quote it:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why did I write this book? Good question! There exist a good number of&#xA;really great books on Reinforcement Learning. So why a new book? I had&#xA;selfish reasons: I wanted a &lt;strong&gt;short&lt;/strong&gt; book, which nevertheless contained&#xA;the major ideas underlying &lt;strong&gt;state-of-the-art RL algorithms&lt;/strong&gt; (back in&#xA;2010), a discussion of their relative &lt;strong&gt;strengths&lt;/strong&gt; and &lt;strong&gt;weaknesses&lt;/strong&gt;, with&#xA;hints on what is known (and not known, but would be good to know)&#xA;about these algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="34010" LastActivityDate="2020-06-19T13:31:31.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22023" PostTypeId="5" CreationDate="2020-06-19T16:34:01.180" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T16:34:01.180" LastActivityDate="2020-06-19T16:34:01.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22024" PostTypeId="4" CreationDate="2020-06-19T16:34:01.180" Score="0" Body="For questions related to the cross-correlation operation, a measure of similarity between two series, which is very similar to the convolution operation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T16:34:01.180" LastActivityDate="2020-06-19T16:34:01.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22025" PostTypeId="5" CreationDate="2020-06-19T16:39:46.350" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T16:39:46.350" LastActivityDate="2020-06-19T16:39:46.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22026" PostTypeId="4" CreationDate="2020-06-19T16:39:46.350" Score="0" Body="For questions related to the notion of batch size in machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T16:39:46.350" LastActivityDate="2020-06-19T16:39:46.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22027" PostTypeId="1" CreationDate="2020-06-19T18:56:05.627" Score="0" ViewCount="36" Body="&lt;p&gt;I have a background in OR and I am new to AI. I know the basics and I currently try to understand the article &amp;quot;&lt;a href=&quot;https://arxiv.org/abs/1704.01665&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Combinatorial Optimization Algorithms over Graphs&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So far I understand the overall procedure, however, part &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; of EQ (3),  is hard to grasp (I reformulated the formula in a more general way that does not require knowledge about the article)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mu = relu(\theta_1 a + \theta_2 b + \theta_3 \underbrace{\sum_{i \in I} relu(\theta_4 c_i)}_{x})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$\theta_1 \in \mathbb{R}^p, \theta_2,\theta_3 \in \mathbb{R}^{p\times p}, \theta_4 \in \mathbb{R}^p$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; being a set of varying cardinality for each input to be evaluated and &lt;span class=&quot;math-container&quot;&gt;$a,b,c_i$&lt;/span&gt; being given inputs.&lt;/p&gt;&#xA;&lt;p&gt;The authors claim to train the network end to end. If we replace &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by some input value (without &lt;span class=&quot;math-container&quot;&gt;$\theta_4$&lt;/span&gt;) I would be able to follow the idea.&lt;/p&gt;&#xA;&lt;p&gt;However as far as I understand &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;,  &lt;span class=&quot;math-container&quot;&gt;$relu(\theta_4 c_i)$&lt;/span&gt; is a little NN on its own with input &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; and output of length &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. Sure, we can compute the values and sum them up and use them as input, however, I do not understand how the end to end training will work for such a setup where basically two networks exist.&lt;/p&gt;&#xA;&lt;p&gt;I would be more than thankful for help on this topic.&lt;/p&gt;&#xA;" OwnerUserId="38032" LastEditorUserId="2444" LastEditDate="2020-06-19T20:21:35.907" LastActivityDate="2020-06-19T20:21:35.907" Title="How do I interpret the following parameters as a network" Tags="&lt;neural-networks&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22028" PostTypeId="1" CreationDate="2020-06-19T20:54:55.427" Score="0" ViewCount="37" Body="&lt;p&gt;PPO agent. The action space includes 3 actions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0: do nothing&lt;/li&gt;&#xA;&lt;li&gt;1: act (start)&lt;/li&gt;&#xA;&lt;li&gt;2: stop&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The agent has to perform thousands of steps doing nothing, then perform step 1 only once (act), then do nothing for N steps again (i.e. observe) for some time, then trigger action 2. Action 2 ends the episode.&lt;/p&gt;&#xA;&lt;p&gt;An example might be fishing: wait till good state conditions, cast a fishing rod, wait till it's time, pull the fish (or nothing). I give small negative rewards for waiting before casting, and explicit feedback reward after casting so the agent knows how it is doing.&lt;/p&gt;&#xA;&lt;p&gt;Problem: after 50K times of training, the agent decides that collecting negative rewards and doing nothing is 100% better than doing anything else, even though rewards would be higher.&lt;/p&gt;&#xA;&lt;p&gt;Tweaking rewards, including more severe punishment for doing nothing before acting and bigger rewards for acting and stopping, doesn't help.&lt;/p&gt;&#xA;&lt;p&gt;My guess is that it is caused by the fact that we have a disproportionately big number of cases when we do nothing, i.e. actions imbalance. Is there a way to deal with this kind of problem?&lt;/p&gt;&#xA;&lt;p&gt;If you think it is not the root cause, I'd highly appreciate other suggestions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/08yhI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/08yhI.png&quot; alt=&quot;metrics&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(P.S. please ignore the episode reward decline below -25; it is caused by increasing the &lt;code&gt;number of steps before acting&lt;/code&gt; the agent is allowed to use during an episode)&lt;/p&gt;&#xA;" OwnerUserId="36999" LastActivityDate="2020-06-19T20:54:55.427" Title="Patient PPO: how to handle imbalanced discrete action space?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22029" PostTypeId="2" ParentId="18480" CreationDate="2020-06-19T21:45:58.350" Score="0" Body="&lt;p&gt;&lt;strong&gt;Contrastive learning&lt;/strong&gt; is a framework that learns similar/dissimilar representations from data that are organized into similar/dissimilar pairs. This can be formulated as a &lt;strong&gt;dictionary look-up&lt;/strong&gt; problem.&lt;/p&gt;&#xA;&lt;p&gt;If I conceptually compare the loss mechanisms for:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SimCLR: A Simple Framework for Contrastive Learning of Visual Representations @ &lt;a href=&quot;https://arxiv.org/abs/2002.05709&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/2002.05709&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;MoCo: Momentum Contrast for Unsupervised Visual Representation Learning @ &lt;a href=&quot;https://arxiv.org/abs/1911.05722&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1911.05722&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both &lt;strong&gt;MoCo&lt;/strong&gt; and &lt;strong&gt;SimCLR&lt;/strong&gt; use varients of a contrastive loss function, like InfoNCE from the paper &lt;em&gt;Representation Learning with Contrastive Predictive Coding&lt;/em&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;\mathcal{L}_{q,k^+,\{k^-\}}=-log\frac{exp(q\cdot k^+/\tau)}{exp(q\cdot k^+/\tau)+\sum\limits_{k^-}exp(q\cdot k^-/\tau)}&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here q is a query representation, &lt;span class=&quot;math-container&quot;&gt;$k^+$&lt;/span&gt; is a representation of the positive (similar) key sample, and &lt;span class=&quot;math-container&quot;&gt;${k^−}$&lt;/span&gt; are representations of the negative (dissimilar) key samples. &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is a temperature hyper-parameter. In the instance discrimination pretext task (used by MoCo and SimCLR), a query and a key form a positive pair if they are data-augmented versions of the same image, and otherwise form a negative pair.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7F6DD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7F6DD.png&quot; alt=&quot;SimCLR vs MoCo&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The contrastive loss can be minimized by various mechanisms that differ in &lt;strong&gt;how the keys are maintained&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In an &lt;strong&gt;end-to-end&lt;/strong&gt; mechanism (Fig. 1a), the negative keys are from the same batch and updated end-to-end by back-propagation. &lt;strong&gt;SimCLR&lt;/strong&gt;, is based on this mechanism and requires a large batch to provide a large set of negatives.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;strong&gt;MoCo&lt;/strong&gt; mechanism i.e. &lt;strong&gt;Momentum Contrast&lt;/strong&gt; (Fig. 1b), the negative keys are maintained in a queue, and only the queries and positive keys are encoded in each training batch.&lt;/p&gt;&#xA;" OwnerUserId="32249" LastEditorUserId="32249" LastEditDate="2020-06-19T21:51:50.307" LastActivityDate="2020-06-19T21:51:50.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22030" PostTypeId="1" AcceptedAnswerId="22035" CreationDate="2020-06-19T22:19:03.553" Score="1" ViewCount="45" Body="&lt;p&gt;In this paper &lt;a href=&quot;https://arxiv.org/pdf/1104.3913.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fairness Through Awareness&lt;/a&gt;, the notation &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$&lt;/span&gt; is being used (page 5 top line), where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; denotes the set of individuals (so I guess set of feature vectors?) and the meaning of the other variables can be found in the paragraph above the mentioned notation. What does the &lt;span class=&quot;math-container&quot;&gt;$\sim$&lt;/span&gt; in the expectation stand for?&lt;/p&gt;&#xA;&lt;p&gt;Another notation that I do not know is &lt;span class=&quot;math-container&quot;&gt;$\Delta (A) $&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the set of outcomes, for instance, &lt;span class=&quot;math-container&quot;&gt;$A = \{ 0,1\}$&lt;/span&gt;. What does it stand for?&lt;/p&gt;&#xA;" OwnerUserId="36116" LastEditorUserId="2444" LastEditDate="2020-06-19T22:48:37.887" LastActivityDate="2020-06-20T00:09:52.940" Title="What do the notations $\sim$ and $\Delta (A) $ mean in the paper &quot;Fairness Through Awareness&quot;?" Tags="&lt;machine-learning&gt;&lt;papers&gt;&lt;notation&gt;&lt;explainable-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22031" PostTypeId="5" CreationDate="2020-06-19T22:44:48.007" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T22:44:48.007" LastActivityDate="2020-06-19T22:44:48.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22032" PostTypeId="4" CreationDate="2020-06-19T22:44:48.007" Score="0" Body="For questions related to the family of models known as R-CNN (such as the original R-CNN model, fast R-CNN, faster R-CNN and mask R-CNN)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T22:44:48.007" LastActivityDate="2020-06-19T22:44:48.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22033" PostTypeId="5" CreationDate="2020-06-19T23:00:37.223" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T23:00:37.223" LastActivityDate="2020-06-19T23:00:37.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22034" PostTypeId="4" CreationDate="2020-06-19T23:00:37.223" Score="0" Body="For questions related to the concept of pooling (e.g. max-pooling), aka downsampling, in the context of convolutional neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-19T23:00:37.223" LastActivityDate="2020-06-19T23:00:37.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22035" PostTypeId="2" ParentId="22030" CreationDate="2020-06-19T23:30:26.013" Score="2" Body="&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\sim$&lt;/span&gt; symbol means that a random variable is drawn from the given distribution, i.e. if I were to say &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; has a Standard Normal distribution I would write &lt;span class=&quot;math-container&quot;&gt;$X \sim \text{Normal}(0,1)$&lt;/span&gt;. They write two explicit expectations here because &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is a random variable with distribution &lt;span class=&quot;math-container&quot;&gt;$\mu_x$&lt;/span&gt; but &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is also a random variable with distribution &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;. I believe you are right that &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; would be analogous to a set of features. So we are saying that &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a random variable over the distribution over the features, or individuals in this context.&lt;/p&gt;&#xA;&lt;p&gt;As for the &lt;span class=&quot;math-container&quot;&gt;$\Delta(A)$&lt;/span&gt;, I have never seen this notation used before -- I am not sure if it is standard notation. I Googled to see if it was something that I just hadn't seen before, but there was no such answer. However, from the context of the paper they define &lt;span class=&quot;math-container&quot;&gt;$M: V \rightarrow \Delta (A)$&lt;/span&gt; to be mappings from individuals to probability distributions over &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, so I guess that &lt;span class=&quot;math-container&quot;&gt;$\Delta (A) $&lt;/span&gt; is probability distributions over &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-20T00:09:52.940" LastActivityDate="2020-06-20T00:09:52.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22036" PostTypeId="1" CreationDate="2020-06-20T02:44:25.867" Score="1" ViewCount="124" Body="&lt;p&gt;So, my company recently bought a big 4k HDR TV for our reception, where we keep showing some videos that were originally shot/created at 720p resolution. Before this, we had a relatively small HD TV, so not a problem. Because the old videos now look dated, my boss wanted to upscale them and enhance their coloring to avoid shooting or procuring new animated videos.&lt;/p&gt;&#xA;&lt;p&gt;This sounded like a fun project, but I know little about AI, and less so about video encoding/decoding. I've started researching and found some references, such as &lt;a href=&quot;http://ir.ia.ac.cn/bitstream/173211/14820/1/TPAMI_final.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video Super-Resolution via Bidirectional&#xA;Recurrent Convolutional Networks&lt;/a&gt;, so while it seems like I have homework to do, it's clearly &amp;quot;been done before&amp;quot;. Would be great to find some code that works on standard formatted videos though.&lt;/p&gt;&#xA;&lt;p&gt;What I'm struggling to find, but would need some good basis to answer in the negative, is: &lt;strong&gt;What about HDR?&lt;/strong&gt; I'm not finding the research terms nor any mention on result for improving dynamic range on videos. Is there any research done on that? Though actual HDR is a format, most of the shots and pictures used for our videos were taken on cameras with small color gamut and latitude, thus everything looks &amp;quot;washed&amp;quot; and the new TV really makes this obvious by comparing against demo videos.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;PS:&lt;/strong&gt;&#xA;Unlike much of the literature I'm finding, I'm not aiming at real-time super resolution, it would be great if it took less than one night to process for a 10 minute video though.&lt;/p&gt;&#xA;" OwnerUserId="38037" LastEditorUserId="38037" LastEditDate="2020-06-26T18:16:41.740" LastActivityDate="2020-06-26T18:16:41.740" Title="Creating 4k HDR video from 720p footage" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;applications&gt;&lt;research&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22037" PostTypeId="1" AcceptedAnswerId="22045" CreationDate="2020-06-20T03:37:41.063" Score="2" ViewCount="74" Body="&lt;p&gt;I have the following situation. An agent plays a game and wants to maximize the accumulated reward as usual, but it can choose its adversary. There are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; adversaries.&lt;/p&gt;&#xA;&lt;p&gt;In episode &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, the agent must first select an adversary. Then for each step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the episode &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, it plays the game against the chosen adversary. Every step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, it receives a reward following the chosen action in step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (for the chosen adversary). How to maximize the expected rewards using DQN? It is clear that choosing the &amp;quot;wrong&amp;quot; (the strongest) adversary won't be a good choice for the agent. Thus, to maximize the accumulated rewards, the agent must take two actions at two different timescales.&lt;/p&gt;&#xA;&lt;p&gt;I started solving it using two DQNs, one to decide the adversary to play against and one to play the game against the chosen adversary. I have two duplicate hyperparameters (&lt;code&gt;batch_size&lt;/code&gt;, &lt;code&gt;target_update_freq&lt;/code&gt;, etc), one for each DQN. Have you ever seen two DQNs like this? Should I train the DQNs simultaneously?&lt;/p&gt;&#xA;&lt;p&gt;The results that I am getting is not that good. The accumulated reward is decreasing, the loss isn't always decreasing...&lt;/p&gt;&#xA;" OwnerUserId="37642" LastActivityDate="2020-06-21T09:03:55.923" Title="Two DQNs in two different time scales" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22038" PostTypeId="1" AcceptedAnswerId="22053" CreationDate="2020-06-20T05:38:04.100" Score="3" ViewCount="76" Body="&lt;p&gt;I made a DQN that controls a traffic light. The observation states are the number of vehicles of each lane in the intersection. I trained it for 500 episodes and saved the model every 50th episode. I plotted the reward curve of the model after the training and found out that around the 460th episode, the reward curve become unstable. Does it mean that the optimized DQN model is the 450th model? If not, how do I know if the my DQN is really optimized?&lt;/p&gt;&#xA;" OwnerUserId="37382" LastActivityDate="2020-06-21T10:19:27.840" Title="How to know if my DQN is optimized?" Tags="&lt;machine-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22039" PostTypeId="5" CreationDate="2020-06-20T10:40:52.443" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-20T10:40:52.443" LastActivityDate="2020-06-20T10:40:52.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22040" PostTypeId="4" CreationDate="2020-06-20T10:40:52.443" Score="0" Body="For questions related to the credit assignment problem, i.e. determine which and how actions contribute to a certain outcome, which is the main problem that reinforcement learning algorithms attempt to solve." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-20T10:40:52.443" LastActivityDate="2020-06-20T10:40:52.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22041" PostTypeId="1" CreationDate="2020-06-20T15:03:13.713" Score="1" ViewCount="51" Body="&lt;p&gt;I'm going through &lt;a href=&quot;https://pytorch.org/docs/stable/distributions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the &lt;code&gt;distributions&lt;/code&gt; package&lt;/a&gt; on PyTorch's documentation and came across the term &lt;em&gt;stochastic computation graph&lt;/em&gt;. In layman's terms, what is it?&lt;/p&gt;&#xA;" OwnerUserId="33448" LastEditorUserId="2444" LastEditDate="2020-06-21T11:46:50.913" LastActivityDate="2020-06-21T11:46:50.913" Title="In layman's terms, what is stochastic computation graph?" Tags="&lt;reinforcement-learning&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22042" PostTypeId="2" ParentId="21096" CreationDate="2020-06-20T16:28:32.963" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The bias plane is a layer everywhere equal to the constant &lt;span class=&quot;math-container&quot;&gt;$a/18$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the action. So each of the 32 frames has three frames for RGB and a fourth frame which is the bias plane making for 128 input layers. This is explained in the Network Architecture section where it mentions that the actions are &amp;quot;broadcast&amp;quot; to the planes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For this I do not have conclusive evidence, but I think tiling a vector means arranging parallel copies of it into e.g. a grid shape. In other words, the input is 6x6x18, and action 1 is represented as all ones in the first plane with all zeros in the remaining planes. One problem with the way you've depicted is that the input is subject to convolutions, but there is no inherent reason why the fifth position and eleventh actions (which are next to each other vertically) should be included in the same 3x3 filter application, but the fifth and seventh (for example) actions should not.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="38039" LastActivityDate="2020-06-20T16:28:32.963" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22043" PostTypeId="1" CreationDate="2020-06-20T16:32:04.713" Score="0" ViewCount="97" Body="&lt;p&gt;Can I refine StyleGAN or StyleGAN2 without retraining it for many days, such that its pretrained model is trained to generate only faces similar to a (rather small) set of reference images?&lt;/p&gt;&#xA;&lt;p&gt;I would like to avoid creating a large dataset and training for many days to weeks, but use the existing model and just bias it towards a set of images.&lt;/p&gt;&#xA;" OwnerUserId="25798" LastActivityDate="2020-06-20T16:32:04.713" Title="Can StyleGAN be refined without a full training?" Tags="&lt;generative-adversarial-networks&gt;&lt;pretrained-models&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22044" PostTypeId="2" ParentId="21968" CreationDate="2020-06-20T16:57:56.300" Score="0" Body="&lt;p&gt;No misunderstanding. IDA* is indeed n^2 vs n for A* in certain situations.&lt;/p&gt;&#xA;&lt;p&gt;See&#xA;&lt;a href=&quot;https://pdfs.semanticscholar.org/388c/0a934934a9e60da1c22c050566dbcd995702.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdfs.semanticscholar.org/388c/0a934934a9e60da1c22c050566dbcd995702.pdf&lt;/a&gt;&#xA;for a reference.&lt;/p&gt;&#xA;&lt;p&gt;IDA* only works in certain (not uncommon) situations. If there are many states with the same costs, and this number grows exponentially with the cost, then IDA* will be O(n). An example of such a problem is the 8 sliding piece puzzle.&lt;/p&gt;&#xA;&lt;p&gt;IDA* will however, as noted, become exponential when all costs differ. Examples include continuous costs, or discrete values spanning a large range.&lt;/p&gt;&#xA;&lt;p&gt;If you really wanted to apply IDA* to e.g. a map, a workable approach would likely be to round all values - or perform a log+floor operation.&lt;/p&gt;&#xA;" OwnerUserId="37970" LastActivityDate="2020-06-20T16:57:56.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22045" PostTypeId="2" ParentId="22037" CreationDate="2020-06-20T17:14:48.040" Score="3" Body="&lt;p&gt;From comments, you say there is no &amp;quot;outer&amp;quot; goal for picking an adversary other than  scoring highly in an individual episode.&lt;/p&gt;&#xA;&lt;p&gt;You could &lt;em&gt;potentially&lt;/em&gt; model the initial adversary choice as a partially separate Markov Decision Process (MDP), where choosing the opponent is a single-step episode with return equal to whatever reward the secondary MDP - which played the game - obtains. However, this &amp;quot;outer&amp;quot; MDP is not much of an MDP at all, it is more like a contextual bandit. In addition, the performance of the inner game-playing agent will vary both with the choice of opponent, and over time as it learns to play better against each opponent. This makes the outer MDP non-stationary. It also requires the inner MDP to know what opponent it is facing in order to correctly predict correct choices and/or future rewards.&lt;/p&gt;&#xA;&lt;p&gt;That last part - the need for any &amp;quot;inner&amp;quot; agent to be aware of the opponent it is playing against - is likely to be necessary whatever structure you chooose. That choice of opponent needs to be part of the state for this inner agent, because it will have an impact on likely future rewards. A characterisation of the opponents also needs to be part of whatever predictive models you could use for the outer agent.&lt;/p&gt;&#xA;&lt;p&gt;A more natural, and probably more useful, MDP model for your problem is to have a single MDP where the first action &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt; is to select the opponent. This matches the language you use to describe the problem, and resolves your issue about trying to run a hierarchy of agents. &lt;a href=&quot;https://arxiv.org/abs/1910.04450#:%7E:text=Hierarchical%20Reinforcement%20Learning%20(HRL)%20is,to%20define%20low%2Dlevel%20rewards.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical reinforcement learning&lt;/a&gt; is a real thing, and very interesting for solving problems which can be broken down into meaningful sub-goals that an agent could discover autonomously, but it does not appear to apply for your problem.&lt;/p&gt;&#xA;&lt;p&gt;This leaves you with a practical problem of creating a model that can switch between choosing between two sets of radically different actions. The &lt;em&gt;select an opponent&lt;/em&gt; action only occurs at the first state of the game, and the two sets of actions do not overlap at all. However, in terms of the theoretical MDP model this is not an issue at all. It is only a practical issue of how you get to fit your Q function approximator to the two radically different action types. There a a few ways around that. Here are a couple that might work for you:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;One shared network&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Always predict for all kinds of action choice, so the agent still makes predictions for switching opponents all the way to the end of the game. Then filter the action choices down to only those available at any time step. When &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; only use the predictions for actions for selecting an opponent, for &lt;span class=&quot;math-container&quot;&gt;$t \ge 1$&lt;/span&gt; only use predictions relating to moves in the game.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Two separate approximators&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Have two function approximators in your agent, use one for predicting reward at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; that covers different opponent choices, and use the other for the rest of the game. If &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is small and there is no generalisation between opponents (i.e. no opponent &amp;quot;stats&amp;quot; that give some kind of clue towards the end results), then for the first approximator, you could even use a Q table.&lt;/p&gt;&#xA;&lt;p&gt;For update steps you need to know whether any particular action value was modelled in one or other of the Q functions - and this will naturally lead you to bootstrap&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat{q}_{o}(s_0, a_0, \theta_0) \leftarrow r_1 + \gamma \text{max}_{a'}\hat{q}_{p}(s_1, a', \theta_1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat{q}_{o}$&lt;/span&gt; is your approximate model for action values of selecting opponents (and &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt; must be an opponent choice) at the start of the game, and &lt;span class=&quot;math-container&quot;&gt;$\hat{q}_{p}$&lt;/span&gt; is the nodel you use for the rest of it (and &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; must be a position play in the game). I've misused &lt;span class=&quot;math-container&quot;&gt;$\leftarrow$&lt;/span&gt; here to stand in for whatever process used to update the action value towards the new estimate - a tabular method that would be a rolling average with current estimate, in neural networks of course that is gradient descent using backpropagation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-21T09:03:55.923" LastActivityDate="2020-06-21T09:03:55.923" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22046" PostTypeId="1" CreationDate="2020-06-20T17:42:24.363" Score="0" ViewCount="26" Body="&lt;p&gt;I read the manual for a multi-mode delay pedal, particularly how it &amp;quot;studies&amp;quot; your playing for reverse mode. I don't have the spec, but I doubt that is meant literally; if I had to guess, &amp;quot;study&amp;quot; just means &amp;quot;sample and hold &lt;em&gt;n&lt;/em&gt; milliseconds of forward play.&amp;quot; However, I got to thinking: &lt;strong&gt;are we at a point in tech where we can apply embedded machine learning to a guitar pedal?&lt;/strong&gt; I could think of two hitches, and there may be more: the size of a pedal (Line 6 DL4, anyone? No one has wanted big gear since the 70s) and the latency (I can afford to leave a cloud instance on and wait a day or two to wrangle lotsa pics of cars on a road, but a live guitarist doesn't; even .1 of a second is a problem, because that's almost a 32nd note at as little as 120 bpm). I do not know much about the state of embedded systems, or what can be done at present. What would it take to overcome these two constraints?&lt;/p&gt;&#xA;" OwnerUserId="35622" LastActivityDate="2020-06-20T17:42:24.363" Title="Is it possible to put embedded ML in something so low-latency-demanding as a guitar pedal?" Tags="&lt;hardware&gt;&lt;embedded-design&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22048" PostTypeId="2" ParentId="17608" CreationDate="2020-06-21T00:46:15.567" Score="0" Body="&lt;p&gt;Especially in continuous space, convergence of the value function is mainly a theoretical property. Without seeing enough of the state space, as you suggest, there's no way to ensure that your Q function will generalize to the whole state space. Convergence results for Q learning with function approximation generally show that in the limit of infinite data, your value function will converge to the desired fixed point -- note that this is only true when your agent explores occasionally, for an infinite amount of time.&lt;/p&gt;&#xA;&lt;p&gt;When your parameters have converged, this simply means that your Q function has fit the data you've collected. As you explore more, your agent may get &amp;quot;surprised&amp;quot; and your parameters may start to change again.&lt;/p&gt;&#xA;&lt;p&gt;Also, convergence of the parameters in function approximation can never guarantee that an optimal value function was found in practice -- the only guarantee you can wish for is that the optimal value function &lt;em&gt;that can be produced with your model&lt;/em&gt; has been found. For instance, the parameters of the linear Q function you posted can converge, even if the optimal Q function is not linear.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-21T00:46:15.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22049" PostTypeId="1" CreationDate="2020-06-21T04:21:23.867" Score="1" ViewCount="71" Body="&lt;p&gt;As the title says, in reinforcement learning, does the off-policy evaluation work for non-stationary policies?&lt;/p&gt;&#xA;&lt;p&gt;For example, IS (importance sampling)-based estimators, such as &lt;em&gt;weighted IS&lt;/em&gt; or &lt;em&gt;doubly robust&lt;/em&gt;, are still unbiased when they are used to evaluate UCB1, which is a non-stationary policy, as it chooses an action based on the history of rewards?&lt;/p&gt;&#xA;" OwnerUserId="30051" LastEditorUserId="2444" LastEditDate="2020-06-21T11:34:08.240" LastActivityDate="2020-06-21T11:34:08.240" Title="Does the off-policy evaluation work for non-stationary policies?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22050" PostTypeId="1" CreationDate="2020-06-21T05:24:02.143" Score="0" ViewCount="26" Body="&lt;p&gt;I've noticed if you have a small negative reward (e.g.,&lt;code&gt;-0.01&lt;/code&gt;) per step for idling and a series of idle steps, an agent seems to learn to trick GAE by learning a very low critic value target (&lt;code&gt;-0.99988735&lt;/code&gt; in this particular case) so when the GAE delta is calculated, the result is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;delta = -0.01 + 0.99 * 1 * -0.99988735 - -0.99988735 = -1.12649999e-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;, which results in GAE being next to 0.&lt;/p&gt;&#xA;&lt;p&gt;Formula (pretty standard):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;delta = np.array([reward + gamma * (1 - done) * next_predicted_value - predicted_value for reward, done, next_predicted_value, predicted_value in zip(rewards, dones, next_values, critic_predictions)], dtype=np.float32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Have you observed this kind of behavior? Any ideas on how to deal with it?&lt;/p&gt;&#xA;" OwnerUserId="36999" LastEditorUserId="36999" LastEditDate="2020-06-21T16:00:05.810" LastActivityDate="2020-06-21T16:00:05.810" Title="How to deal with GAE ineffectiveness because of critic value adaptation?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;reinforce&gt;&lt;advantage-actor-critic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22051" PostTypeId="1" CreationDate="2020-06-21T07:02:08.070" Score="2" ViewCount="107" Body="&lt;p&gt;Suppose that the transition time between two states is a random variable (for example, unknown exponential distribution); and between two arrivals, there is no reward. If &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; (real number not an integer number) shows the time between two arrivals, should I update Q-functions as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(s,a) = Q(s,a)+\alpha.(R+\gamma^{\tau} \max_{b \in A}Q(s^{\prime},b)-Q(s,a))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And, to compare different algorithms, total rewards (&lt;span class=&quot;math-container&quot;&gt;$TR=R_{1}+ R_2+R_{3}+...+R_{T}$&lt;/span&gt;) is used.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What measure should be used in the SMDP setting?&lt;/em&gt; I would be thankful if someone can explain the Q-Learning algorithm for the SMDP problem with this setting.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I am wondering when Q-functions are updated. For example, if a customer enters our website and purchases a product, we want to update the Q-functions. Suppose that the planning horizon (state &lt;span class=&quot;math-container&quot;&gt;$S_{0}$&lt;/span&gt;) starts at 10:00 am, and the first customer enters at 10:02 am, and we sell a product and gain &lt;span class=&quot;math-container&quot;&gt;$R_1$&lt;/span&gt; and the state will be &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;. The next customer enters at 10:04 am, and buy a product, and gain reward &lt;span class=&quot;math-container&quot;&gt;$R_2$&lt;/span&gt; (state &lt;span class=&quot;math-container&quot;&gt;$S_{2}$&lt;/span&gt;). &lt;em&gt;In this situation, should we wait until 10:02 to update the Q-function for state &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is the following formula correct?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(S_0)= R_1 \gamma^2+  \gamma^2V(S_1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this case, if I discretize the time horizon to 1-minute intervals, the problem will be a regular MDP problem. &lt;em&gt;Should I update Q-functions when no customer enters in a time interval (reward =0)?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="10191" LastEditorUserId="2444" LastEditDate="2020-06-27T21:47:05.300" LastActivityDate="2020-06-28T10:54:15.643" Title="Updating action-value functions in Semi-Markov Decision Process and Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;discount-factor&gt;&lt;semi-mdp&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22052" PostTypeId="2" ParentId="21975" CreationDate="2020-06-21T07:54:12.877" Score="0" Body="&lt;p&gt;What you want to do is you have some character like hulk, thor, caption America, iron man, etc. and you want to Train a response generator for each character like for thor on his dataset, for hulf on his dataset etc... and then you wanted to make a conversation. If I Understood you well then.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You can fine-tune GPT-2 small or GPT-2 Medium using your Dataset for each character. &lt;a href=&quot;https://github.com/microsoft/DialoGPT&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reference for Finetune&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can decode using simple nucleus sampling at each time step. Use greedy nucleus sampling multiple times in parallel to generate multiple responses. You can generate 30 such responses and also use the last 3-5 dialogue turn as a context or Short term memory. &lt;a href=&quot;https://github.com/JRC1995/Chatbot/blob/master/Generator/generator.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Decoder reference&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To Choose the best response out of all generated responses(30 responses as an example). you can use cosine similarity between generated responses and query + last 3-5 dialogue turn.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def cosine_similarity_nd(embd1, embd2):&#xA;numerator = np.multiply(embd1, embd2)&#xA;numerator = np.sum(numerator, axis=1)&#xA;eucli_norm_1 = np.sqrt(np.sum(np.power(embd1, 2), axis=1))&#xA;eucli_norm_2 = np.sqrt(np.sum(np.power(embd2, 2), axis=1))&#xA;denominator = np.multiply(eucli_norm_1, eucli_norm_2)&#xA;denominator = denominator + 1e-10  # remove zeros&#xA;cosine_similarity = np.divide(numerator, denominator)&#xA;return cosine_similarity.reshape((-1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or you can train a reverse model for either all character or mix all character dialogue and use this model to calculate loss of all responses with your query. &lt;a href=&quot;https://convaisharables.blob.core.windows.net/lsp/multiref/small_reverse.pkl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reference reverse model&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or you can Combine result of cosine and reverse model to find response out of all the responses.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32861" LastActivityDate="2020-06-21T07:54:12.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22053" PostTypeId="2" ParentId="22038" CreationDate="2020-06-21T08:46:56.760" Score="1" Body="&lt;p&gt;There is a good chance that your DQN is already optimized but you would have to take a look at its performance to really check and see whether its actions seem up to par.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reasons it may not be optimized:&lt;/strong&gt; If you are tracking the reward after every episode, unstable rewards are very common just due to random chance, but if you are averaging the reward over the past 50 episodes or so, then it may also be your learning rate or epsilon.&lt;/p&gt;&#xA;&lt;p&gt;If your learning rate is too high or low you may either never be able to reach a fully optimized DQN or be stuck in a local minimum. An easy way to solve a problem like this would be to just add a simple learning rate decay, so that the learning rate would start off high as to not get stuck in local minima yet decay to a small enough number where you know that the agent has found the global minimum.&lt;/p&gt;&#xA;&lt;p&gt;The other problem could be that your epsilon may be too high or low. A high epsilon will never allow the agent to fully optimize while a low epsilon doesn't allow the agent to explore and discover better strategies, so it may be a good idea to mess around with this as well.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The only way to really gauge the agent's performance would be to look at it making some decisions through a video or by analyzing some of its predictions. And, if it seems to be doing well, then it may very well be optimized, but, if the agent is not performing as well as it should, then it may be a good idea to try out some of the strategies above.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="38062" LastEditorUserId="2444" LastEditDate="2020-06-21T10:19:27.840" LastActivityDate="2020-06-21T10:19:27.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22054" PostTypeId="1" CreationDate="2020-06-21T10:52:02.907" Score="0" ViewCount="79" Body="&lt;p&gt;I'm trying to understand the DDPG algorithm shown at this &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt;. I don't know what should the result of the gradient at step 14 be.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vPmro.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vPmro.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it a scalar that I have to use to update all the weights (so all weights are updated with the same value)? Or is it a list with a different values to use for updating for each weight? I'm used to working with loss functions and an &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; target, but here I don't have them so I'm quite confused.&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-06-22T17:05:20.457" LastActivityDate="2021-03-19T20:06:25.587" Title="In Deep Deterministic Policy Gradient, are all weights of the policy network updated with the same or different value?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22055" PostTypeId="1" CreationDate="2020-06-21T14:27:22.673" Score="1" ViewCount="47" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/03JMT.png&quot; alt=&quot;on-policy (left) vs. off-policy (right) actor critic&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does this figure correctly represent the overall general idea about actor-critic methods for on-policy (left) and off-policy (right) case?&lt;/p&gt;&#xA;&lt;p&gt;I am a bit confused about the off-policy case (right figure). Why does the right figure represent the off-policy actor-critic methods?&lt;/p&gt;&#xA;" OwnerUserId="37381" LastEditorUserId="2444" LastEditDate="2020-06-22T12:08:33.813" LastActivityDate="2020-06-22T12:08:33.813" Title="Is this figure a correct representation of off-policy actor-critic methods?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;off-policy-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22056" PostTypeId="1" CreationDate="2020-06-21T14:41:13.537" Score="1" ViewCount="238" Body="&lt;p&gt;I am writing a couple of different reinforcement learning models based on Rainbow DQN or some PG models. All of them internally use an LSTM network because my project is using time series data.&lt;/p&gt;&#xA;&lt;p&gt;I wanted to test my models using OpenAI Gym before I add too many domain specific code to the models.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that, all of the Atari games seem to fall into the CNN area which I don't use.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to use OpenAI Gym to test any time series data driven RL models/networks?&lt;/p&gt;&#xA;&lt;p&gt;If not, is there any good environment that I can use to examine the validity of my models?&lt;/p&gt;&#xA;" OwnerUserId="37615" LastActivityDate="2020-11-18T16:03:45.310" Title="How do I test an LSTM-based reinforcement learning model using any Atari games in OpenAI gym?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;&lt;open-ai&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22057" PostTypeId="2" ParentId="22056" CreationDate="2020-06-21T15:10:04.220" Score="1" Body="&lt;p&gt;If I understand your problem correctly, you can test on just about any environment, and just omit parts of the observations to ensure your RNN is learning. For example, you can test on cartpole, ignoring the velocity and angular velocity states. This way the MDP isn't actually Markovian and you'll need the RNN to learn.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-21T15:10:04.220" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22058" PostTypeId="2" ParentId="22054" CreationDate="2020-06-21T15:13:49.083" Score="0" Body="&lt;p&gt;Each Q output is a scalar, so the sum of all those is a scalar. Thus, you're taking a gradient wrt your parameters of a scalar. The result is a vector with one entry per parameter.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-21T15:13:49.083" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22059" PostTypeId="1" CreationDate="2020-06-21T17:01:18.127" Score="0" ViewCount="49" Body="&lt;p&gt;In &lt;a href=&quot;https://towardsdatascience.com/policy-iteration-in-rl-an-illustration-6d58bdcb87a7&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, there is an explanation (with an example) of how policy iteration works.&lt;/p&gt;&#xA;&lt;p&gt;It seems that, if we replace all the probabilities of moves in the example by new probabilities where the best action is taken 100% of the time and the other moves are taken 0% of the time, then the final policy will end up to be (south, south, north) as in the example provided. However, if we are certain of our moves, we can go north and then south in the example to get most of the reward.&lt;/p&gt;&#xA;&lt;p&gt;In other words, it seems to be incorrect to calculate the value of a state by summing up rewards for all possible actions out of the state, because, like in the case I described above, or a case where one action gives you a huge penalty, you are 100% gonna avoid it, therefore the value state is unfairly weighted.&lt;/p&gt;&#xA;&lt;p&gt;Why care about the value of the action which I'm not gonna take?&lt;/p&gt;&#xA;" OwnerUserId="38069" LastEditorUserId="2444" LastEditDate="2020-06-22T20:06:39.740" LastActivityDate="2020-06-22T20:06:39.740" Title="Why care about the value of the action which I'm not gonna take in policy iteration?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;policy-iteration&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22060" PostTypeId="5" CreationDate="2020-06-21T17:21:34.067" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-21T17:21:34.067" LastActivityDate="2020-06-21T17:21:34.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22061" PostTypeId="4" CreationDate="2020-06-21T17:21:34.067" Score="0" Body="For questions related to the concept of data augmentation, where a dataset can be augmented in terms of number and diversity of the samples, which can be useful to avoid over-fitting, especially, when the available dataset(s) is relatively small." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-21T17:21:34.067" LastActivityDate="2020-06-21T17:21:34.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22062" PostTypeId="1" CreationDate="2020-06-21T17:53:10.537" Score="1" ViewCount="38" Body="&lt;p&gt;I made a DQN model and plot its reward curve. You can see intuitively that the curve already converged since its reward value now just oscillates. How can I show confidence that my DQN already reached its optimal other than by just showing the curve? Are there any way to validate that it is already optimized?&lt;/p&gt;&#xA;" OwnerUserId="37382" LastActivityDate="2020-06-21T17:53:10.537" Title="Is there a way to show convergence of DQN other than by eye observation?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22063" PostTypeId="1" CreationDate="2020-06-21T18:52:25.430" Score="0" ViewCount="185" Body="&lt;p&gt;My DQN model outputs the best traffic light state in an intersection. I used different values of batch size and learning rate to find the best model. How would I know if I got the optimal hyperparameter values?&lt;/p&gt;&#xA;" OwnerUserId="37382" LastActivityDate="2020-06-21T21:24:02.753" Title="How to validate that my DQN hyperparameters are the optimal?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22064" PostTypeId="1" AcceptedAnswerId="22197" CreationDate="2020-06-21T19:56:34.157" Score="1" ViewCount="405" Body="&lt;p&gt;I'm currently modeling DQN in Reinforcement Learning. My question is: what are the best practices related to Boltzmann Exploration? My current thoughts are: (1) Let the temperature decay through training and finally stop at 0.01, when the method will always select the best practice, with almost no randomness. (2) Standardize the predicted Q values before feeding into the softmax function.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I'm using (2), and the reward is suffering from high variance. I'm wondering whether it has something to do with the exploration method?&lt;/p&gt;&#xA;" OwnerUserId="37178" LastActivityDate="2020-06-26T18:55:18.503" Title="What's the best practice for Boltzmann Exploration temperature in RL?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22065" PostTypeId="2" ParentId="22063" CreationDate="2020-06-21T21:24:02.753" Score="1" Body="&lt;p&gt;If possible, I would try to calculate what the (theoretical) maximum  throughput through the intersection is for a given time interval. If the control behavior that the DQN produces comes empirically close to the maximally possible throughput score, the model is good. Otherwise, you could measure and compare the throughput of different models and choose the best performing one.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-06-21T21:24:02.753" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22067" PostTypeId="1" CreationDate="2020-06-22T05:13:35.490" Score="0" ViewCount="18" Body="&lt;p&gt;In classification, suppose you have 1 image labeled as cancer and 99 labeled as not cancer, you can just divide the loss weight of &amp;quot;not cancer&amp;quot; by 99. Then you can train the model as this will help fix the label imbalance problems somewhat.&lt;/p&gt;&#xA;&lt;p&gt;But in regression, it's not trivial what number I need to use to divide. Suppose you regress the probability of cancer from 0 to 1 instead of classifying it, and someone has collected the dataset with probabilities. The dataset has cancer around 0.1 about 99% of the time and you have cancer around 0.9 about 1% of the time. But the dataset also have labels like 0.26, 0.3634, 0.753, 0.5, etc.&#xA;It's not clear what loss weight you need to use for cancer=0.26 or any other cancer probability.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there any formula to figure out loss weight for regression problems that have imbalanced labels?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2020-06-22T05:13:35.490" Title="How to figure out loss weight for label-imbalanced regression problems?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22068" PostTypeId="1" CreationDate="2020-06-22T08:30:40.510" Score="0" ViewCount="22" Body="&lt;p&gt;I want to develop a real-time 3D shape measurement using convolutional neural network (CNN) for vehicle. Could you recommend me references for that? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="38082" LastActivityDate="2020-06-22T08:30:40.510" Title="Are there references for real-time 3D shape measurement using convolutional neural network?" Tags="&lt;convolutional-neural-networks&gt;&lt;autonomous-vehicles&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22069" PostTypeId="1" CreationDate="2020-06-22T09:02:14.030" Score="1" ViewCount="53" Body="&lt;p&gt;I have a paired dataset of binary images A and B: A1 paired with B1, A2-B2, etc., with simple shapes (rectangles, squares).&lt;/p&gt;&#xA;&lt;p&gt;The external software receives both images A and B and it returns a number that represents the error.&lt;/p&gt;&#xA;&lt;p&gt;I need a model that, given images A and B, can modify A into A' by adding or removing squares, so that the error from the software is minimized. I don't have access to the source code of the software so I don't know how it works.&lt;/p&gt;&#xA;&lt;p&gt;I tried to make a NN that copies the functionality of the software, and a generative NN to generate the modified image A' but I haven't got good results.&lt;/p&gt;&#xA;&lt;p&gt;The software can only receive binary images, so I cannot use a loss function because my last layer of the generator being a softmax, if I apply a threshold, I will lose the track of the gradients, so I cannot apply gradient descent.&lt;/p&gt;&#xA;&lt;p&gt;Someone told me that when you cannot calculate the gradient of the loss with respect to the weights, reinforcement learning with policy gradients is a good solution.&lt;/p&gt;&#xA;&lt;p&gt;I'm new to this field, so I want to be sure I'm going in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="38083" LastEditorUserId="2444" LastEditDate="2020-06-22T12:03:48.697" LastActivityDate="2020-06-22T12:03:48.697" Title="Is Reinforcement Learning what I need for this image to image translation problem?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22070" PostTypeId="1" CreationDate="2020-06-22T11:31:44.563" Score="1" ViewCount="105" Body="&lt;p&gt;I am a computer science student. I learned about programming languages recently, but I don't know much about artificial intelligence.&lt;/p&gt;&#xA;&lt;p&gt;I want to know, why don't we program something in a way that we could tell the program&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Hey! Do this for me!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And then just sit down and wait that the AI does the job?&lt;/p&gt;&#xA;&lt;p&gt;Is this currently possible to do?&lt;/p&gt;&#xA;" OwnerUserId="38086" LastEditorUserId="2444" LastEditDate="2020-06-22T14:00:37.600" LastActivityDate="2020-07-02T20:35:21.103" Title="Can we give a command to an AI and wait for it to do the job without explicitly telling it how to do it?" Tags="&lt;machine-learning&gt;&lt;state-of-the-art&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22071" PostTypeId="2" ParentId="22070" CreationDate="2020-06-22T12:37:44.017" Score="4" Body="&lt;p&gt;Normally when you write a program, you are acting like a boss that micromanages the job, telling the workers &lt;em&gt;how&lt;/em&gt; to accomplish a task, perhaps without even letting them know what the purpose is. What you are hoping to be is a boss that gives the workers a &lt;em&gt;goal&lt;/em&gt; and allows them to determine how to accomplish it.&lt;/p&gt;&#xA;&lt;p&gt;In many ways, that is one of the aims of AI.&lt;/p&gt;&#xA;&lt;p&gt;We already have small examples, such as a smart washing machine that weighs the clothes, monitors the amount of dirt in the water, and continually decides how much water to add or drain, when to agitate, when to rinse, and when to spin.  All you had to tell it was &amp;quot;&lt;em&gt;clean these&lt;/em&gt;&amp;quot;, and perhaps say what kind of material they are made of.&lt;/p&gt;&#xA;&lt;p&gt;As a much larger example, automobiles were traditionally operated by turning the steering wheel to cause it to change direction and by pressing the pedals to increase or decrease the speed, but now we are working on car controllers that can respond to &amp;quot;&lt;em&gt;go to Cleveland&lt;/em&gt;&amp;quot; by determining the best speed and direction by itself.&lt;/p&gt;&#xA;&lt;p&gt;But note that those two examples (the first requiring only a little &amp;quot;intellegence&amp;quot;, the second a lot) were for very specific requests that could be expressed in a few words.&#xA;As soon as the request becomes even slightly more complicated, the difficulty of creating a solution becomes much much more difficult.&lt;/p&gt;&#xA;&lt;p&gt;Ask your &amp;quot;&lt;em&gt;Do this for me!&lt;/em&gt;&amp;quot; request of a human being.&#xA;The first thing they'll do is ask &amp;quot;what does &lt;em&gt;this&lt;/em&gt; mean?&amp;quot;.&#xA;And then you'll have to give a lot of details.&#xA;And then they'll have questions about what you really want.&#xA;And so on.&#xA;Providing the requirements for &amp;quot;this&amp;quot; will be neither simple nor easy.&lt;/p&gt;&#xA;&lt;p&gt;Human intellegence is still vastly superior to current AI.&#xA;In particular, it is capable of not only recognizing that additional information is required, but of having the intuitive ability to know what that missing information must be like.&lt;/p&gt;&#xA;" OwnerUserId="36323" LastEditorUserId="36323" LastEditDate="2020-06-24T18:54:28.113" LastActivityDate="2020-06-24T18:54:28.113" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22072" PostTypeId="1" CreationDate="2020-06-22T12:40:47.963" Score="0" ViewCount="58" Body="&lt;p&gt;I am quite new to deep learning. I just finished the deep learning specialization by Professor Andrew NG and Deep Learning AI. Now, my professor (instructor) has advised me to look into some classic papers for aspect extraction and opinion mining from video. Could anyone suggest me some resources where I can get started? Can anyone suggest some papers I should read? Maybe a course or a book or some links to descriptive sessions. Your help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="38087" LastEditorUserId="38087" LastEditDate="2020-07-17T01:10:25.733" LastActivityDate="2021-04-13T16:02:55.260" Title="What are some good papers or resources for aspect extraction and opinion modelling from video or audio?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;feature-extraction&gt;&lt;sentiment-analysis&gt;&lt;data-mining&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22074" PostTypeId="1" AcceptedAnswerId="22076" CreationDate="2020-06-22T19:39:43.160" Score="1" ViewCount="63" Body="&lt;p&gt;The concept of experience replay is saving our experiences in our replay buffer. We select at random to break the correlation between consecutive samples, right?&lt;/p&gt;&#xA;&lt;p&gt;What would happen if we calculate our loss using just one experience instead of a mini-batch of experiences?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-22T19:52:58.743" LastActivityDate="2020-06-22T20:09:05.673" Title="What would happen if we sampled only one tuple from the experience replay?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22075" PostTypeId="1" AcceptedAnswerId="22077" CreationDate="2020-06-22T19:59:32.770" Score="2" ViewCount="51" Body="&lt;p&gt;If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the sizes and lengths of the filters are still the same, how can the number of parameter in a network increase?&lt;/p&gt;&#xA;" OwnerUserId="35615" LastEditorUserId="2444" LastEditDate="2020-06-22T20:13:32.797" LastActivityDate="2020-06-23T12:02:21.457" Title="Does the number of parameters in a convolutional neuronal network increase if the input dimension increases?" Tags="&lt;convolutional-neural-networks&gt;&lt;filters&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22076" PostTypeId="2" ParentId="22074" CreationDate="2020-06-22T20:09:05.673" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The concept of experience replay is saving our experiences in our replay buffer. We select at random to break the correlation between consecutive samples, right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes that is a major benefit of using a replay buffer.&lt;/p&gt;&#xA;&lt;p&gt;A secondary benefit is the ability to use the same sample more than once. This can lead to beter sample efficiency, although that is not guaranteed.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What would happen if we calculate our loss using just one experience instead of a mini-batch of experiences?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The algorithm is still valid, but the gradient estimate for the update step would be based on a single record of [state, action, reward, next state]. This would be a high variance update process, with many steps in wrong directions, but in expectation over many steps you should still see a correct gradient. You would probably need to compensate for the high variance per sample by reducing the learning rate.&lt;/p&gt;&#xA;&lt;p&gt;In addition, assuming the standard approach of collecting one time step then making one update to DQN neural network, each piece of experience would only be used once on average before being discarded.&lt;/p&gt;&#xA;&lt;p&gt;These two effects will likely combine such that the learning process would not be very sample efficient.&lt;/p&gt;&#xA;&lt;p&gt;The size of the minibatch is one of many hyperparameters you can change in DQN. It might be the case for some problems that choosing a low minibatch size is helpful, provided other adjustments (such as a lower learning rate) are made along with it. If you are not sure, you mostly have to try and see.&lt;/p&gt;&#xA;&lt;p&gt;In my experience on a small range of problems, a moderate size of minibatch - ranging from 10 to 100 - has worked the best in terms of end results of high scoring agents. However, I have not spent a long time trying to make low batch sizes work.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-06-22T20:09:05.673" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22077" PostTypeId="2" ParentId="22075" CreationDate="2020-06-22T20:23:26.313" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If the convolutional neural network (CNN) &lt;strong&gt;only&lt;/strong&gt; uses convolutional layers, then the number of parameters does not increase as a function of the &lt;strong&gt;spatial&lt;/strong&gt; dimensions (&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;) of the input. This is one of the advantages of CNNs!&lt;/p&gt;&#xA;&lt;p&gt;The reason is quite simple: the parameters of the convolutional layers are the &lt;em&gt;kernels&lt;/em&gt; (aka filters), which typically have fixed dimensions and can, nevertheless, be applied to inputs of different &lt;strong&gt;spatial&lt;/strong&gt; dimensions, provided that the necessary &lt;em&gt;padding&lt;/em&gt; is used. However, note that padding can create bigger &lt;em&gt;feature maps&lt;/em&gt;, but feature maps are not the parameters of the neural network: they are the output of the convolutional layers. That's probably what confuses you, when you see a diagram of a CNN, because you see bigger feature maps and you might think that the number of parameters increases.&lt;/p&gt;&#xA;&lt;p&gt;Here's a TensorFlow 2 (and Keras) program that shows that the number of parameters does not change as a function of the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; dimension of the input.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;input_shapes = [(2 * k, 2 * k, 3) for k in range(2, 6)]&#xA;&#xA;for input_shape in input_shapes:&#xA;    model = tf.keras.Sequential()&#xA;    model.add(tf.keras.layers.Input(shape=input_shape))&#xA;    model.add(tf.keras.layers.Conv2D(10, kernel_size=3, use_bias=True))&#xA;    model.summary() # The total number of parameters is always 280&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The parameters of a convolutional layer can increase if you increase the size of each kernel and the number of kernels, but this does not necessarily depend on the input.&lt;/p&gt;&#xA;&lt;p&gt;The parameters of the CNN can also increase if you increase the depth of the input, but that's typically fixed (either &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; for RGB images or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for grayscale images). The reason is quite simple too: the kernels in the first convolutional layer (connected to the input layer) will have the same depth as the depth of the input.&lt;/p&gt;&#xA;&lt;p&gt;If your CNN also has fully connected layers, then the number of parameters also depends on the dimensions of the inputs. This is because the parameters of the fully connected layers depend on the number and dimensions of the feature maps (remember the &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten?version=nightly&quot; rel=&quot;nofollow noreferrer&quot;&gt;flatten&lt;/a&gt; layer before the fully connected layers?), which, as I said, can increase as a function of the input.&lt;/p&gt;&#xA;&lt;p&gt;If you don't want to use fully connected layers, you may want to try &lt;a href=&quot;https://arxiv.org/pdf/1411.4038.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;fully convolutional networks&lt;/strong&gt;&lt;/a&gt; (FCNs), which do not make use of fully connected layers, but can, nonetheless, be used to solve classification (and other) tasks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-23T12:02:21.457" LastActivityDate="2020-06-23T12:02:21.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22079" PostTypeId="1" CreationDate="2020-06-22T20:51:52.507" Score="1" ViewCount="78" Body="&lt;p&gt;Model Description: Model based(assume known of the entire model) Markov decision process.&lt;/p&gt;&#xA;&lt;p&gt;Time(&lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;): Finite horizon discrete time with discounting factor&lt;/p&gt;&#xA;&lt;p&gt;State(&lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt;): Continuous multi-dimensional state&lt;/p&gt;&#xA;&lt;p&gt;Action(&lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;): Continuous multi-dimensional action (deterministic action)&lt;/p&gt;&#xA;&lt;p&gt;Feasible(possible action) (&lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt;): possible action is a continuous multi-dimensional space, no discretization for the action space.&lt;/p&gt;&#xA;&lt;p&gt;Transition kernel(&lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;): known and have some randomness associated with the current state and next state&lt;/p&gt;&#xA;&lt;p&gt;Reward function: known in explicit form and terminal reward is know.&lt;/p&gt;&#xA;&lt;p&gt;The method I tried to solve the model:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Discretize the state space and construct multi-dimensional grid for state space, starting from the terminal state, I used backward induction to reconstruct value function from the previous period. By using the Bellman equation, I need to solve an optimization problem, selecting the best action that gives me the best objective function.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V_t(x_t) = max_{a_t \in A_t}[R_t(x_t,a_t) + E[\tilde{V}_{t+1}(x_{t+1})|x_t, a_t]]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\tilde{V}_{t+1}$&lt;/span&gt; here is an approximation using interpolation method, since the discrete values are calculated from the previous time episode. In other words: &lt;span class=&quot;math-container&quot;&gt;$\tilde{V}_{t+1}$&lt;/span&gt; is approximated by some discrete value: &lt;span class=&quot;math-container&quot;&gt;$$V_{t+1}(x_0),V_{t+1}(x_1),V_{t+1}(x_2)\cdots V_{t+1}(x_N)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x_0,x_1,x_2\cdots x_N$&lt;/span&gt; are grid points from discretizing the state space.&lt;/p&gt;&#xA;&lt;p&gt;In this way, for every time steps t, I could have a value function for every grid point and my value function could be approximated by using some interpolation method(probably cubic spline). But here are some of the problems: 1. what kind of interpolation is suitable for high dimensional data. 2. Say we have five dimension for the state, then I discretize the state by giving 5 grid points to every dimension, then there are 5^5 = 3125 discrete state values I need to calculate through the optimization.(Curse of the dimensionality). 3. What kind of optimizer should I use? Since I do not know the shape of the objective function, I do not know if it is a smooth function and I do not know if the function is concave. So I may have to use a robust optimizer, probably some evolutionary optimizer. So eventually I end up with this computational complexity and it takes too long for the computation.&lt;/p&gt;&#xA;&lt;p&gt;And recently I learned the techniques of policy gradient from OpenAI: &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html&lt;/a&gt;&#xA;This method avoid using this backward induction and approximating the value function by using interpolation. And obtained the approximated policy by firstly guessing a functional form of the policy and take approximated gradient of the policy function by using sampling(simulation) method. Since the model is known, every time it could sample new trajectories and use that to update the policy by using the stochastic gradient assent method. In this way updating the policy until it gets some sort of convergency.&lt;/p&gt;&#xA;&lt;p&gt;I am wondering if this type of technique could potentially reduce the computational complexity significantly. Any advice helps, thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="38097" LastEditorUserId="38097" LastEditDate="2020-06-23T15:28:01.797" LastActivityDate="2021-03-20T17:03:55.350" Title="Continuous state and continuous action Markov decision process time complexity estimate: backward induction VS policy gradient method (RL)" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;time-complexity&gt;&lt;finite-markov-decision-process&gt;&lt;interpolation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22080" PostTypeId="1" CreationDate="2020-06-22T21:32:15.583" Score="0" ViewCount="192" Body="&lt;p&gt;Let's assume,  that we embedded a vector of length 49 into a matrix using 512-d embeddings. If we then multiply the matrix by it transposed version we receive a matrix of 49 by 49. Which is symmetric. Let's also assume we do not add the positional encoding and we only have only one attention head in the first layer of the transformer architecture. What would the result of the softmax on this 49 by 49 matrix look like? Is it still symmetric, or is the softmax correctly applied for each line of the matrix, resulting in a non symmetric matrix? My guess would be that the matrix should not be symmetric any more. But I'm unsure about that.&lt;/p&gt;&#xA;&lt;p&gt;I ask this, to verify if my implementation is wrong or not, and what the output should look like. I have seen so many sophisticated and different implementation of the transformer architecture with different frameworks, that I can't answer this question for myself right now (confusion). I still try to understand the basic building blocks of the transformer architecture.&lt;/p&gt;&#xA;" OwnerUserId="38099" LastEditorUserId="38099" LastEditDate="2020-06-23T13:35:43.370" LastActivityDate="2021-03-21T23:05:30.683" Title="Transformers - is the self attention matrix softmax output (layer 1) symmetric?" Tags="&lt;transformer&gt;&lt;attention&gt;&lt;softmax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22081" PostTypeId="1" CreationDate="2020-06-22T21:42:40.333" Score="0" ViewCount="40" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/490rR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/490rR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm implementing DDQN in my toy scenario. During training, I'm surprised to see that the total reward doesn't converge and have a tendency to degrade. What could be the problem?&lt;/p&gt;&#xA;&lt;p&gt;Here's the picture: episode reward vs # of episodes during the training process.&lt;/p&gt;&#xA;" OwnerUserId="37178" LastActivityDate="2020-06-22T21:42:40.333" Title="Reinforcement Learning Diagnostic: Total reward doesn't converge" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22082" PostTypeId="2" ParentId="17020" CreationDate="2020-06-22T22:04:04.613" Score="3" Body="&lt;p&gt;CNNs learn convolutional filters that get trained on finding local, recurring patterns in some kind of image/volume data. 1D convolution is actually a thing, but I think what would be more suitable for your case is using &lt;a href=&quot;https://www.tensorflow.org/guide/keras/rnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent Neural Nets&lt;/a&gt;. They are specifically designed for working on time series-es of heterogeneous data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to reconsider the answer I gave earlier.&#xA;First of all, in case of dealing with time series data where it is uncertain over which time span a given event to be detected lasts, I'd generally consider using Recurrent Neural Networks (RNNs) rather than CNNs, since RNNs maintain so-called hidden states that can carry potentially useful information over time, i.e. past inputs may influence present outputs. So, either if you know that the event to be detected spans many time steps (which might not even be consecutive time steps) or if you are uncertain about how long exactly the time span of an event to be detected is, then I'd suggest going for RNNs. If you decide to use RNNs, adding extra statistics to your data might make the learning task easier for the network, but is not strictly required since you would expect an RNN to learn relevant time-series statistics on its own.&lt;/p&gt;&#xA;&lt;p&gt;However, there is a case where using CNNs might actually suffice and yield good results as well. Consider the case of the figure shown below, where the white tiles are time series inputs and the blueish area is the area that a CNN filter/kernel (or multiple ones) is working on right now.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/50BGy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/50BGy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, there is a time series of data points and some statistics derived from that time series that have been appended to the input data stream. So, in the figure, the augmented time series input consists of the measurements themselves as well as mean, variance, and skewness values derived from the time series data (over consecutive time steps). Under the assumption that the type of event you are interested in is bounded, in terms of time steps, to the width of the CNN filter(s)/kernel(s) that you slide over your input data, then a simple CNN might suffice to detect the events you are interested in. In other words, &lt;strong&gt;if the width of your filter spans over at least as many time steps as an event of interest lasts, then your CNN will be able to learn to detect the event&lt;/strong&gt; (given proper training). In that case, adding further statistics to your input data might indeed help or even be necessary to detect certain events, since it implicitly widens the time window that your CNN filter can observe at any given time step, since a mean (or other statistics) may also considers past data that the filter itself (or multiple filters...) might not span over at a given time step where you want to detect an event.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2021-02-06T13:05:48.093" LastActivityDate="2021-02-06T13:05:48.093" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22083" PostTypeId="1" CreationDate="2020-06-23T00:29:09.303" Score="0" ViewCount="70" Body="&lt;p&gt;Following multiple guides for actor-critic, I have attempted to implement a single agent one-step version to see how well it would perform on Pong.&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that the actor provides probabilities on what action to perform out of the action pool. The critic estimates the value of a given state.&lt;/p&gt;&#xA;&lt;p&gt;The actor is trained by utilizing the advantage of the action taken at a given state. This advantage value is calculated as &lt;span class=&quot;math-container&quot;&gt;$R + G * V(S) - V(S^{+1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The critic, however, is trained as &lt;span class=&quot;math-container&quot;&gt;$R + G * V(S^{+1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For me, this seems to regularly result in overestimating a single action after 10-20 episodes. Once an action is overestimated this way and is chosen rather consistently, the advantage dwindles down close to zero over time as the critic is only obtaining the value of the single action taken for that sample.&lt;/p&gt;&#xA;&lt;p&gt;I assume this is a flaw in my implementation or a misunderstanding of the algorithm because if the value for the critic of a given state matches &lt;span class=&quot;math-container&quot;&gt;$R + G * V(S^{+1})$&lt;/span&gt; then the advantage calculation becomes zero when only a single action is consistently taken.&lt;/p&gt;&#xA;&lt;p&gt;So, have I misunderstood how it is to be implemented? If not, what does it sound like I am missing?&lt;/p&gt;&#xA;" OwnerUserId="29806" LastEditorUserId="2444" LastEditDate="2020-06-23T22:17:28.207" LastActivityDate="2020-06-23T22:17:28.207" Title="How to handle actor-critic when actor consistently overestimates one action?" Tags="&lt;actor-critic-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22084" PostTypeId="1" CreationDate="2020-06-23T00:29:39.470" Score="0" ViewCount="36" Body="&lt;p&gt;The EXP3 algorithm as given in the figure below (taken from &lt;a href=&quot;https://jeremykun.com/2013/11/08/adversarial-bandits-and-the-exp3-algorithm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial Bandits and the Exp3 Algorithm&lt;/a&gt;) is to solve the adversarial bandits for the single-player case.&lt;/p&gt;&#xA;&lt;p&gt;What happens if there are multiple players with common rewards? A quick search does not give me good results from the internet.&lt;/p&gt;&#xA;&lt;p&gt;The version I am interested in is where we want to maximize a common rewards say, the number of players that respect some condition. Say, once all players have chosen their actions &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}$&lt;/span&gt;, we count the number of players that respect the condition, say &lt;span class=&quot;math-container&quot;&gt;$x(\mathbf{a})$&lt;/span&gt;, and give a common reward to all players as &lt;span class=&quot;math-container&quot;&gt;$r=x(\mathbf{a})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I thought of simply modifying the original EXP3 given below by letting the players to play synchronously and then receive the common reward and update the probabilities. I have some issue with this: I think the probabilities distribution for each player will be identical and thus not interesting outcome.&lt;/p&gt;&#xA;&lt;p&gt;Is there a more interesting way to solve the adversarial multi-player multi-armed bandit problem?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NsrJh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NsrJh.png&quot; alt=&quot;Source: https://jeremykun.com/2013/11/08/adversarial-bandits-and-the-exp3-algorithm/&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37642" LastActivityDate="2020-06-23T00:29:39.470" Title="Is there a multi-agent version of EXP3?" Tags="&lt;multi-agent-systems&gt;&lt;multi-armed-bandit&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22085" PostTypeId="1" CreationDate="2020-06-23T02:23:24.920" Score="1" ViewCount="129" Body="&lt;p&gt;I have been creating sports betting algorithms for many years using Microsoft access and I am transitioning to the ML world and trying to get a grasp on determining the success of my algorithms. I have exported my algorithms as CSV files dating back to the 2013-14 NBA season and imported them into python via pandas.&lt;/p&gt;&#xA;&lt;p&gt;The purpose of importing these CSV files is to determine the future accuracy of these algorithms using ML. Here are the algorithm records based on the Microsoft access query:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A System: 471-344 (58%) +92.60&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;B System: 317-239 (57%) +54.10&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;C System: 347-262 (57%) +58.80&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have a total of 8,814 records in my database, however, the above systems are based on situational stats, e.g., Team A fits an algorithm if they have better Field Goal %, Played Last Game Home/Away, More Points Per Game, etc...&lt;/p&gt;&#xA;&lt;p&gt;​&lt;/p&gt;&#xA;&lt;p&gt;Here is some of the code that I wrote using Jupyter to determine the accuracy:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)&#xA;&#xA;clf = LinearSVC(C=1.0, penalty=&amp;quot;l2&amp;quot;, dual=False)&#xA;&#xA;clf.fit(X_train, y_train)&#xA;&#xA;pred_clf = clf.predict(X_test)&#xA;&#xA;scores = cross_val_score(clf, X, y, cv=10)&#xA;&#xA;rfe_selector = RFE(clf, 10)&#xA;&#xA;rfe_selector = rfe_selector.fit(X, y)&#xA;&#xA;rfe_values = rfe_selector.get_support()&#xA;&#xA;train = accuracy_score(y_train, clf.predict(X_train))&#xA;&#xA;test = accuracy_score(y_test, pred_clf)&#xA;&#xA;print(&amp;quot;Train Accuracy:&amp;quot;, accuracy_score(y_train, clf.predict(X_train)))&#xA;&#xA;print(&amp;quot;Test Accuracy:&amp;quot;, accuracy_score(y_test, pred_clf))&#xA;&#xA;print(classification_report(y_test, pred_clf, zero_division=1))&#xA;&#xA;print(confusion_matrix(y_test, pred_clf))&#xA;&#xA;print(&amp;quot;Accuracy: %0.2f (+/- %0.2f)&amp;quot; % (scores.mean(), scores.std() * 2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;​&lt;/p&gt;&#xA;&lt;p&gt;Here are the results from the code above by system:&lt;/p&gt;&#xA;&lt;p&gt;A System:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train Accuracy: 0.6211656441717791&lt;/li&gt;&#xA;&lt;li&gt;Test Accuracy: 0.5153374233128835&lt;/li&gt;&#xA;&lt;li&gt;F1 Score: 0.52&lt;/li&gt;&#xA;&lt;li&gt;CONFUSION MATRIX: [[16 50] [29 68]]&lt;/li&gt;&#xA;&lt;li&gt;Accuracy: 0.55 (+/- 0.10)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;B System:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train Accuracy: 0.6306306306306306&lt;/li&gt;&#xA;&lt;li&gt;Test Accuracy: 0.5178571428571429&lt;/li&gt;&#xA;&lt;li&gt;F1 Score: 0.52&lt;/li&gt;&#xA;&lt;li&gt;CONFUSION MATRIX: [[49 23] [31 9]]&lt;/li&gt;&#xA;&lt;li&gt;Accuracy: 0.55 (+/- 0.08)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;C System:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train Accuracy: 0.675564681724846&lt;/li&gt;&#xA;&lt;li&gt;Test Accuracy: 0.5409836065573771&lt;/li&gt;&#xA;&lt;li&gt;F1 Score: 0.54&lt;/li&gt;&#xA;&lt;li&gt;CONFUSION MATRIX: [[15 29] [27 51]]&lt;/li&gt;&#xA;&lt;li&gt;Accuracy: 0.57 (+/- 0.16)&#xA;​&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In order to have a profitable system, the accuracy only needs to be 52.5%. If I base my systems off of the Test Accuracy, only the C System is profitable. However, all are profitable if based on Accuracy (mean &amp;amp; standard deviation).&lt;/p&gt;&#xA;&lt;p&gt;My question is, can I rely on my Accuracy (mean &amp;amp; standard deviation) for future games even though my Testing Accuracy is lower than 52.5%?&lt;/p&gt;&#xA;&lt;p&gt;If not, any suggestions are greatly appreciated on how I can gauge the future results on these systems.&lt;/p&gt;&#xA;" OwnerUserId="38101" LastEditorUserId="2444" LastEditDate="2020-06-23T09:44:48.050" LastActivityDate="2021-03-20T13:01:45.157" Title="Is my 57% sports betting accuracy correct?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;support-vector-machine&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22086" PostTypeId="1" CreationDate="2020-06-23T04:08:03.560" Score="1" ViewCount="43" Body="&lt;p&gt;What are the main differences between a language model and a machine translation model?&lt;/p&gt;&#xA;" OwnerUserId="9863" LastEditorUserId="2444" LastEditDate="2020-06-23T09:45:37.647" LastActivityDate="2020-06-23T11:47:57.540" Title="What are the main differences between a language model and a machine translation model?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;models&gt;&lt;machine-translation&gt;&lt;language-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22087" PostTypeId="1" CreationDate="2020-06-23T05:40:23.750" Score="1" ViewCount="75" Body="&lt;p&gt;I'm new to relatively RNNs, and I'm trying to train &lt;em&gt;generative&lt;/em&gt; and &lt;em&gt;guessing&lt;/em&gt; neural networks to produce sequences of real numbers that look random. My architecture looks like this (each &amp;quot;circle&amp;quot; in the output is the adverserial network's guess for the generated circle vertically below it -- having seen only the terms before it):&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/KyA2RvA.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that the adverserial network is rewarded for predicting outputs &lt;em&gt;close&lt;/em&gt; to the true values, i.e. the loss function looks like &lt;code&gt;tf.math.reduce_max((sequence - predictions) ** 2)&lt;/code&gt; (I have also tried &lt;code&gt;reduce_mean&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I don't know if there's something obviously wrong with my architecture, but when I try to train this network (and I've added a reasonable number of layers), it doesn't really work very well.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://colab.research.google.com/drive/1Zd3l1tW64r9OizML_Vz_J3cYpPlZzrap?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colab Notebook&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you look at the result of the last code block, you'll see that my generative neural network produces things like&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;[0.9907787, 0.9907827, 0.9907827, 0.9907827, 0.9907827, 0.9907827, 0.9907827, 0.9907827, 0.9907827, 0.9907827]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But it could &lt;em&gt;easily&lt;/em&gt; improve itself by simply training to jump around more, since you'll observe that the adverserial network also predicts numbers very close to the given number (even when the sequence it is given to predict is one that jumps around a lot!).&lt;/p&gt;&#xA;&lt;p&gt;What am I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="38105" LastEditorUserId="38105" LastEditDate="2020-06-23T12:52:59.400" LastActivityDate="2021-03-20T20:02:00.133" Title="Why does my &quot;entropy generation&quot; RNN do so badly?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22088" PostTypeId="1" CreationDate="2020-06-23T06:46:33.567" Score="1" ViewCount="31" Body="&lt;p&gt;Are there any existing ontologies available for &amp;quot;engineering&amp;quot; data?&lt;/p&gt;&#xA;&lt;p&gt;By &amp;quot;engineering&amp;quot; I mean pertaining to the fields of electrical, mechanical, thermal, etc., engineering.&lt;/p&gt;&#xA;" OwnerUserId="38107" LastEditorUserId="2444" LastEditDate="2020-06-23T09:52:43.407" LastActivityDate="2020-06-23T09:52:43.407" Title="Are there any existing ontologies that model engineering data?" Tags="&lt;reference-request&gt;&lt;ontology&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22090" PostTypeId="1" CreationDate="2020-06-23T07:35:09.220" Score="1" ViewCount="27" Body="&lt;p&gt;In different books on reinforcement learning, policy-based methods are motivated by their ability to handle large (continuous) action spaces. Is this the only motivation for the policy-based methods? What if the action space is tiny (say, only 9 possible actions), &lt;em&gt;but&lt;/em&gt; each action costs a huge amount of resources and there is no model for the MDP, would this also be a good application of policy-based methods?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="2444" LastEditDate="2020-06-23T10:45:35.580" LastActivityDate="2020-06-23T10:45:35.580" Title="Are policy-based methods better than value-based methods only for large action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;policy-based-methods&gt;&lt;value-based-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22091" PostTypeId="2" ParentId="22085" CreationDate="2020-06-23T07:41:25.260" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;My question is, can I rely on my Accuracy (mean &amp;amp; standard deviation) for future games even though my Testing Accuracy is lower than 52.5%?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If by Accuracy you mean training accuracy, then absolutely you should not trust those values. For almost all machine learning algorithms there is a problem with &lt;a href=&quot;https://elitedatascience.com/overfitting-in-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;overfitting&lt;/a&gt; to training data, which will result in reporting over-estimates for metrics against the training data. This is &lt;em&gt;why&lt;/em&gt; you should always have a test data set of unseen data, because the performance on the training data is not what you truly care about - it is the performance on new unseen data, or how the model &lt;em&gt;generalises&lt;/em&gt;, that matters when you want to use it to predict new values.&lt;/p&gt;&#xA;&lt;p&gt;The test data set gives you a measure of how well your model will generalise, because it simulates the performance of the model against unseen data.&lt;/p&gt;&#xA;&lt;p&gt;However, the test data set is not perfect for all uses. Random chance will cause your test measurements to vary. If you use your test results to &lt;em&gt;select&lt;/em&gt; a model from a list of models based on performance, then:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The model you selected &lt;em&gt;does&lt;/em&gt; have the highest likelihood of being the best one, as you want. It is not guaranteed to be the best though.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The test result is likely to be an over-estimate of general performance. That is because you cannot separate the random fluctuations in test results from real performance improvements, you only see the combination.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The more tests you run, the more likely you will have an inflated view of the performance of your best model.&lt;/p&gt;&#xA;&lt;p&gt;The usual fix for this is to use cross validation. Cross validation uses yet another data set to help you with the first step of choosing the best model. After you have chosen your best performing model using cross validation, then you can use a test set that you have kept in reserve to measure the performance. Because you have not used that test set to &lt;em&gt;select&lt;/em&gt; a model, then it will give you an unbiased measure of performance. You should still bear in mind that this measure still comes with implied error bars (and with other caveats, such as any inherent sampling bias).&lt;/p&gt;&#xA;&lt;p&gt;When predicting future results from past data, you do also need to be concerned about population drift and non-stationary problems. This is a common issue in any data set that includes complex behaviour that can evolve over time. This is very likely to affect results from sports teams where many conditions affecting performance evolve over the same timescales that you are trying to predict. In practice this means you will want to feed in new data ad re-train your models constantly, and despite this your models will tend to lag behind reality. It is unlikely you will achieve even the test result accuracy in the long term.&lt;/p&gt;&#xA;&lt;p&gt;You can add one more thing to your testing routines to help assess the impact of non-stationarity - when reserving data for the test or cross-validation sets, don't do it at random, instead reserve all the latest results (e.g. last four weeks of data) for test only. You are likely to see a reduction in the metrics when you do this for a problem domain like sport, but that should give you a more realistic assessment of the model you are building.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-06-23T07:41:25.260" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="22093" PostTypeId="2" ParentId="16970" CreationDate="2020-06-23T08:02:21.957" Score="0" Body="&lt;p&gt;This is still Q-learning, remember Q-learning is off-policy value-based.&#xA;For Bellman optimality operator  &lt;span class=&quot;math-container&quot;&gt;$\mathcal{T}Q=r+max\ Q'$&lt;/span&gt;.&#xA;If you have enough exploration, it always takes &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; to the optimal fixed point.&lt;/p&gt;&#xA;" OwnerUserId="38110" LastActivityDate="2020-06-23T08:02:21.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22094" PostTypeId="1" AcceptedAnswerId="22635" CreationDate="2020-06-23T08:21:16.683" Score="2" ViewCount="57" Body="&lt;p&gt;In the proof of the policy gradient theorem in the &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL book of Sutton and Barto&lt;/a&gt; (that I shamelessly paste here):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ASU0q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ASU0q.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;there is the &amp;quot;unrolling&amp;quot; step that is supposed to be immediately clear&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;With just elementary calculus and re-arranging of terms&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Well, it's not. :) Can someone explain this step in more detail?&lt;/p&gt;&#xA;&lt;p&gt;How exactly is &lt;span class=&quot;math-container&quot;&gt;$Pr(s \rightarrow x, k, \pi)$&lt;/span&gt; deduced by &amp;quot;unrolling&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="37627" LastEditDate="2020-06-23T08:52:24.480" LastActivityDate="2020-07-23T10:23:18.860" Title="Understanding the &quot;unroling&quot; step in the proof of the policy gradient theorem" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22095" PostTypeId="2" ParentId="22094" CreationDate="2020-06-23T08:50:18.937" Score="-1" Body="&lt;p&gt;It looks like &amp;quot;v of s prime&amp;quot; is just substituted with the already derived value for &amp;quot;v of s&amp;quot;. You can call it a recursion of a kind.&#xA;In other words, v(s) is dependent on v(s') and that implies that v(s') is dependent on v(s''). So we can combine that and get the dependency of v(s) of v(s'').&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-23T08:50:18.937" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22096" PostTypeId="1" CreationDate="2020-06-23T09:40:22.223" Score="0" ViewCount="21" Body="&lt;p&gt;&lt;strong&gt;environment&lt;/strong&gt;&#xA;My agent needs to navigate in a 64x64x64 discrete 3d grid environment, and remove certain voxels. Voxels can be in a number of states: should-remove, should-not-remove, empty. It can move through empty voxels and can each step choose to delete all voxels within a selection of radii.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;agent input&lt;/strong&gt;&#xA;The input it receives from the environment are 2D 21x21 slices of the cube, centered around itself. Each voxel has a one-hot encodings of its state. Voxels falling outside of the bounds of the environment are in a state called empty.&lt;/p&gt;&#xA;&lt;p&gt;I started with a slice over each axis: x, y, z. This works okay, but there will be uncertainty to some areas within its voxel deletion radius. Any suggestions on how to get a better representation of state? I've tried doing ~ 7 slices over the x axis at different levels, but don't see improved performance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;architecture&lt;/strong&gt;&#xA;The CNNs interpreting each slice look like:&lt;/p&gt;&#xA;&lt;p&gt;conv1: kernel sz [8,8], stride [4,4], filters 16, elu activation&lt;/p&gt;&#xA;&lt;p&gt;conv2: kernel sz [4,4], stride [2,2], filters 32, elu activation&lt;/p&gt;&#xA;&lt;p&gt;All outputs are then concatenated over the channel dimension, and convolved over by a 1x1 conv layer with 32 filters, and encoded into a dense layer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;question&lt;/strong&gt;&#xA;Any obvious things that could be improved in my design? I'm kind of alone on this and could use a set of fresh/experienced eyes.&lt;/p&gt;&#xA;&lt;p&gt;p.s. the agent has recurrence, trains with PPO, gets reward every step, gets info about previous steps, and gets input about voxels outside of its 'visual' field as vector input, but that's not what I want to focus on for now.&lt;/p&gt;&#xA;" OwnerUserId="31180" LastEditorUserId="31180" LastEditDate="2020-06-23T09:53:06.970" LastActivityDate="2020-06-23T09:53:06.970" Title="Optimal RL agent's representation of 3D-grid data: 2D Slices and CNN encoding. Suggestions?" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22097" PostTypeId="1" CreationDate="2020-06-23T09:41:17.573" Score="1" ViewCount="37" Body="&lt;p&gt;I am considering using Reinforcement Learning to do optimal control of a complex process that is controlled by two parameters&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(n_O, n_I), \quad n_I = 1,2,3,\dots, M_I, n_O = 1,2,3,\dots, M_O$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this sense, the state of the system is represented &lt;span class=&quot;math-container&quot;&gt;$S_t = (n_{O,t}, n_{I,t})$&lt;/span&gt;. It is represented, because there is a relatively complex system, a solution of coupled Partial Differential Equations (PDES), actually in the background.&lt;/p&gt;&#xA;&lt;p&gt;Is this problem considered a partially observable Markov Decision Process (POMDP) because there is a whole mess of things behind &lt;span class=&quot;math-container&quot;&gt;$S_t = (n_{O,t}, n_{I,t})$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;The reward function has two parameters&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s) = (n_{lt}, \epsilon_\infty)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;that are results of the environment (solution of the PDEs).&lt;/p&gt;&#xA;&lt;p&gt;In a sense, using &lt;span class=&quot;math-container&quot;&gt;$S_t = (n_{O,t}, n_{I,t})$&lt;/span&gt; makes this problem similar to Gridworld, where the goal is to go from &lt;span class=&quot;math-container&quot;&gt;$S_0 = (M_O, M_I)$&lt;/span&gt; to a state with smaller &lt;span class=&quot;math-container&quot;&gt;$(n_O, n_I)$&lt;/span&gt;, given reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, where the reward changes from state to state and episode to episode.&lt;/p&gt;&#xA;&lt;p&gt;Available action operations are&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$inc(n) = n + 1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$dec(n) = n - 1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$id(n)  = n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; can be &lt;span class=&quot;math-container&quot;&gt;$n_I$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$n_O$&lt;/span&gt;. This means there are &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; possible actions&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A=\{(inc(n_O), inc(n_I)),(inc(n_O), dec(n_I)),(inc(n_O), id(n_I)),(dec(n_O), inc(n_I)), \dots\}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;to be taken, but there is no model for the state transition, and the state transition is extremely costly.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, as solving a kinematic equation for a point in space, solving coupled PDEs from fluid dynamics should have the Markov property (strongly if the flow is laminar, for turbulence, I have no idea). I've also found a handful of papers where a fluid dynamics problem is parameterized and a policy-gradient method is simply applied.&lt;/p&gt;&#xA;&lt;p&gt;I was thinking to use REINFORCE as a start, but the fact that &lt;span class=&quot;math-container&quot;&gt;$(n_O, n_I)$&lt;/span&gt; does not fully describe the state and questions like &lt;a href=&quot;https://ai.stackexchange.com/questions/11612/can-q-learning-be-used-in-a-pomdp&quot;&gt;this one on POMDP&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/questions/20118/isnt-a-simulation-a-great-model-for-model-based-reinforcement-learning&quot;&gt;this one about simulations&lt;/a&gt; make me suspicious. Could REINFORCE be used for such a problem, or is there something that prevents this?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="37627" LastEditDate="2020-06-23T09:58:37.863" LastActivityDate="2020-06-23T09:58:37.863" Title="How to choose an RL algorithm for a Gridworld that models a much more complex problem" Tags="&lt;reinforcement-learning&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22099" PostTypeId="2" ParentId="20118" CreationDate="2020-06-23T09:56:11.290" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Shouldn't this allow us to find better actions more quickly compared to policy gradient updates?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It depends on the nature of the simulation. If the simulation models a car as a solid body moving with three &lt;span class=&quot;math-container&quot;&gt;$(x,y,\theta)$&lt;/span&gt; degrees of freedom in a plane (hopefully, if it doesn't hit anything and propel vertically), the three ordinary differential equations of solid body motion can be solved quite quickly, compared to a simulation used to model the path of least resistance of a ship on wavy sea, where fluid dynamics equations must be solved, that require a huge amount of resources. OK, the response time needed for a ship is much longer, than for a car, yes, but to compute it predictively, one needs a huge amount of computational power.&lt;/p&gt;&#xA;" OwnerUserId="37627" LastActivityDate="2020-06-23T09:56:11.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22100" PostTypeId="1" AcceptedAnswerId="22110" CreationDate="2020-06-23T10:38:01.887" Score="1" ViewCount="100" Body="&lt;p&gt;Why don't we use a trigonometric function, such as &lt;span class=&quot;math-container&quot;&gt;$\tan(x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an element of the interval &lt;span class=&quot;math-container&quot;&gt;$[0,pi/2)$&lt;/span&gt;, instead of the sigmoid function for the output neurons (in the case of classification)?&lt;/p&gt;&#xA;" OwnerUserId="38113" LastEditorUserId="2444" LastEditDate="2020-06-23T11:02:45.043" LastActivityDate="2020-06-23T13:45:26.567" Title="Why don't we use trigonometric functions for the output neurons?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22101" PostTypeId="2" ParentId="21632" CreationDate="2020-06-23T10:46:10.127" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;BUT there are 2 different attention layers and one of which do not use the encoder’s output at all. So, what are the keys and values&#xA;now?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The first attention layer in the decoder is the &amp;quot;Masked Multi-Head Attention&amp;quot; layer and is the self-attention layer, calculating how much each word is related to each word in the same sentence. However, our aim in the decoder is to generate the next French word and so for any given output French word we can use all the English words but only the French words previously seen in the sentence. We, therefore, &amp;quot;mask&amp;quot; the words that appear later in the French sentence by transforming these to 0 so the attention network cannot use them.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;How these 3 words are converted to 4 words&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The second attention block in the decoder is where the English to French word mapping happens. We have a query for every output position in the French sentence and a key/value for every English input word. We calculate relevance scores from the dot product of the query and key and then obtain output scores for each predicted word from multiplying the relevance and value. The following diagram is useful to visualise how, for each predicted word, we can have relevance scores that can predict that one English word can be translated to multiple, or no French word.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tQgLu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tQgLu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In summary, the encoder discovers interesting things about the English sentence whilst the decoder predicts the next French word in the translation. It should be noted they use &amp;quot;Multi-Head Attention&amp;quot;, meaning that a number (8 in the original paper) of attention vectors are calculated to learn attention mechanisms to pay attention to different things, for example, grammar, vocabulary, tense, gender, and the output is a weighted average of these.&lt;/p&gt;&#xA;" OwnerUserId="9554" LastEditorUserId="2444" LastEditDate="2020-11-20T12:56:33.260" LastActivityDate="2020-11-20T12:56:33.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22102" PostTypeId="1" AcceptedAnswerId="22111" CreationDate="2020-06-23T10:59:33.633" Score="2" ViewCount="93" Body="&lt;p&gt;I created a virtual 2D environment where an agent aims to find a correct pose corresponding to a target image. I implemented a DQN to solve this task. When the goal is fixed, e.g. the aim is to find the pose for position (1,1), the agent is successful. I would now like to train an agent to find the correct pose while the goal pose changes after every episode. My research pointed me to the term &amp;quot;Multi-Objective Deep Reinforcement Learning&amp;quot;. As far as I understood, the aim here is to train one or multiple agents to achieve a policy approximation that fits all goals.&#xA;Am I on the right track or how should I deal with different goal states?&lt;/p&gt;&#xA;" OwnerUserId="37623" LastActivityDate="2020-06-23T13:46:58.957" Title="How to handle changing goals in a DQN?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22103" PostTypeId="2" ParentId="22086" CreationDate="2020-06-23T11:47:57.540" Score="2" Body="&lt;p&gt;The simple language model will give you the probability of a sequence of tokens(sentence) for that language. So lets say if you have trained a model for English language your model can give you the probability for any random english sentence.&lt;/p&gt;&#xA;&lt;p&gt;Consider some sentence&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt; &amp;quot;the quick brown fox jumps over the lazy dog&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt;  &lt;span class=&quot;math-container&quot;&gt;$x_1 \ x_2 \ x_3 \ ... \ x_n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;model will give you &lt;span class=&quot;math-container&quot;&gt;$P(X)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Moreover if the model has been trained properly, in the following scenario, where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt; &amp;quot;the quick brown fox jumps over the lazy dog&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt;  &lt;span class=&quot;math-container&quot;&gt;$x_1 \ x_2 \ x_3 \ ... \ x_n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt; &amp;quot;dog brown quick fox over the jumps lazy the&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt;  &lt;span class=&quot;math-container&quot;&gt;$y_1 \ y_2 \ y_3 \ ... \ y_m$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;model will always give &lt;span class=&quot;math-container&quot;&gt;$P(X) &amp;gt; P(Y)$&lt;/span&gt;, as it has learnt the structure of language.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand Machine translation model gives you the conditional probability of the next token given your source sentence and partial-target sentence.&#xA;So if&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X = $&lt;/span&gt; &amp;quot;I am a student&amp;quot; and &lt;span class=&quot;math-container&quot;&gt;$Y = $&lt;/span&gt; &amp;quot;je suis&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$=y_1, \ y_2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;model will give you &lt;span class=&quot;math-container&quot;&gt;$P(y_3 | X,y1_,y_2)$&lt;/span&gt; .&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a source sentence and &lt;span class=&quot;math-container&quot;&gt;$Y= y_1, y_2$&lt;/span&gt; is a partial target sentence. The probability of word/token &amp;quot;étudiant&amp;quot; would be maximum among all words of vocabulary.&lt;/p&gt;&#xA;" OwnerUserId="38114" LastActivityDate="2020-06-23T11:47:57.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22104" PostTypeId="1" AcceptedAnswerId="22109" CreationDate="2020-06-23T12:33:12.650" Score="0" ViewCount="43" Body="&lt;p&gt;Currently, I am reading &lt;a href=&quot;https://arxiv.org/pdf/1905.11946.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rethinking Model Scaling for Convolutional Neural Networks&lt;/a&gt;. The authors are talking about a different way of scaling convolutional neural networks by scaling all dimensions simultaneously and relative to each dimension. I understand the scaling methods regarding the depth of a network (# layers) and the resolution (size of the input image).&lt;/p&gt;&#xA;&lt;p&gt;What I was stumbling is the concept of the network's width (# channels). What is meant by the width or the number of channels of a network? I don't think it is the number of color channels, or is this the case? The number of color channels was the only link I found regarding the terms &amp;quot;ConvNets&amp;quot; and &amp;quot;number of channels&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="38119" LastEditorUserId="2444" LastEditDate="2020-06-23T16:58:43.783" LastActivityDate="2020-06-23T16:58:43.783" Title="What is meant by the number of channels of a network?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22105" PostTypeId="1" CreationDate="2020-06-23T12:43:40.593" Score="1" ViewCount="192" Body="&lt;p&gt;I am trying to understand the genetic algorithm in terms of feature selection and these features are extracted using a machine learning algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Let's suppose I have data of heart rate for 3 minutes collected from &lt;span class=&quot;math-container&quot;&gt;$50$&lt;/span&gt; subjects. From these 3-minute heart rate, I extracted &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; features, like the mean, standard deviation, variance, skewness and kurtosis. Now the shape of my feature set is &lt;code&gt;(50, 5)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I want to know what are gene, chromosome and population in genetic algorithm related to the above scenario.&lt;/p&gt;&#xA;&lt;p&gt;What I understand is each feature is a gene, and a set of all features for one subject &lt;code&gt;(1, 5)&lt;/code&gt; is the chromosome, and the whole feature set &lt;code&gt;(50, 5)&lt;/code&gt; is a population. But I think this concept is not correct. Because in the genetic algorithm, we take a random population, but according to my concept complete data is population, so how random data is selected.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me to understand it?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZKMxm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZKMxm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38120" LastEditorUserId="2444" LastEditDate="2020-06-23T16:40:59.177" LastActivityDate="2021-04-20T09:47:04.700" Title="What is meant by gene, chromosome, population in genetic algorithm in terms of feature selection?" Tags="&lt;ai-design&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22106" PostTypeId="1" CreationDate="2020-06-23T13:06:31.737" Score="2" ViewCount="236" Body="&lt;p&gt;I'm building a denoising autoencoder. I want to have the same input and output shape image.&lt;/p&gt;&#xA;&lt;p&gt;This is my architecture:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_img = Input(shape=(IMG_HEIGHT, IMG_WIDTH, 1))  &#xA;&#xA;x = Conv2D(32, (3, 3), activation='relu', padding='same')(input_img)&#xA;x = MaxPooling2D((2, 2), padding='same')(x)&#xA;x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)&#xA;encoded = MaxPooling2D((2, 2), padding='same')(x)&#xA;&#xA;&#xA;&#xA;x = Conv2D(32, (3, 3), activation='relu', padding='valid')(encoded)&#xA;x = UpSampling2D((2, 2))(x)&#xA;x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)&#xA;x = UpSampling2D((2, 2))(x)&#xA;decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)&#xA;&#xA;&#xA;# decodedSize = K.int_shape(decoded)[1:]&#xA;&#xA;# x_size = K.int_shape(input_img)&#xA;# decoded = Reshape(decodedSize, input_shape=decodedSize)(decoded)&#xA;&#xA;&#xA;autoencoder = Model(input_img, decoded)&#xA;autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My input shape is: &lt;strong&gt;1169x827&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is Keras output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Model: &amp;quot;model_6&amp;quot;&#xA;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;input_7 (InputLayer)         [(None, 1169, 827, 1)]    0         &#xA;_________________________________________________________________&#xA;conv2d_30 (Conv2D)           (None, 1169, 827, 32)     320       &#xA;_________________________________________________________________&#xA;max_pooling2d_12 (MaxPooling (None, 585, 414, 32)      0         &#xA;_________________________________________________________________&#xA;conv2d_31 (Conv2D)           (None, 585, 414, 64)      18496     &#xA;_________________________________________________________________&#xA;max_pooling2d_13 (MaxPooling (None, 293, 207, 64)      0         &#xA;_________________________________________________________________&#xA;conv2d_32 (Conv2D)           (None, 291, 205, 32)      18464     &#xA;_________________________________________________________________&#xA;up_sampling2d_12 (UpSampling (None, 582, 410, 32)      0         &#xA;_________________________________________________________________&#xA;conv2d_33 (Conv2D)           (None, 582, 410, 32)      9248      &#xA;_________________________________________________________________&#xA;up_sampling2d_13 (UpSampling (None, 1164, 820, 32)     0         &#xA;_________________________________________________________________&#xA;conv2d_34 (Conv2D)           (None, 1162, 818, 1)      289       &#xA;===============================================================&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I have the same input and output shape?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="38093" LastEditorUserId="2444" LastEditDate="2020-06-23T16:33:18.213" LastActivityDate="2020-07-24T09:01:55.190" Title="How can I have the same input and output shape in an auto-encoder?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;autoencoders&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22108" PostTypeId="2" ParentId="22100" CreationDate="2020-06-23T13:20:36.770" Score="2" Body="&lt;p&gt;The main reason why the sigmoid function is used is because it 'does not blow up' since it is between 0 and 1 always. As for the relu it is used because it is computationally cheap and even resolves the problem of vanishing gradient(and hence used more often than sigmoid).&lt;/p&gt;&#xA;&lt;p&gt;So a function like tan(x) will blow up for certain values of x. This can cause a problem of exploding gradients. So, I believe for this reason tan(x) cannot be a good non-linearity to be used.&lt;/p&gt;&#xA;&lt;p&gt;As for any other function, it is more because of the results that we have gotten over the years and sigmoid and relu have been promising.&lt;/p&gt;&#xA;" OwnerUserId="38118" LastActivityDate="2020-06-23T13:20:36.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22109" PostTypeId="2" ParentId="22104" CreationDate="2020-06-23T13:27:22.897" Score="1" Body="&lt;p&gt;It is exactly that - the number of color channels or any other analogue to color that you use.&lt;/p&gt;&#xA;" OwnerUserId="28041" LastActivityDate="2020-06-23T13:27:22.897" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22110" PostTypeId="2" ParentId="22100" CreationDate="2020-06-23T13:45:26.567" Score="0" Body="&lt;p&gt;Although it's true that if you use certain trigonometric functions, such as the tangent, you could have numerical problems (as suggested &lt;a href=&quot;https://ai.stackexchange.com/a/22108/2444&quot;&gt;in this answer&lt;/a&gt;), that's not the only reason for not using trigonometric functions.&lt;/p&gt;&#xA;&lt;p&gt;Trigonometric functions are periodic. In general, we may not want to convert a non-periodic function to a periodic one. To be more concrete, let's suppose we use the sine function as the activation function of the output neurons of a neural network. Assuming only one input, if the input to any of those output neurons is &lt;span class=&quot;math-container&quot;&gt;$360k$&lt;/span&gt;, for any integer &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, the result will always be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, but that may not be desirable.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-06-23T13:45:26.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22111" PostTypeId="2" ParentId="22102" CreationDate="2020-06-23T13:46:58.957" Score="1" Body="&lt;p&gt;The simplest thing you can do is to add data regarding the target pose to the state vector. This will allow any generalisations that the agent learns that apply to similar poses to be used directly.&lt;/p&gt;&#xA;&lt;p&gt;Clearly in normal use, where the target pose remains fixed during the episode, then that part of the state information will not change either during the episode. You will also need to train with a large variety of target poses - so training will take longer.&lt;/p&gt;&#xA;&lt;p&gt;Multi-Objective Deep Reinforcement Learning is slightly different in that it attempts to resolve prioritising between multiple sub-goals. It would also be a more complex solution, whilst augmenting the state vector should allow you to continue using a very similar DQN set up as you have already.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-06-23T13:46:58.957" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22112" PostTypeId="2" ParentId="22079" CreationDate="2020-06-23T14:03:14.700" Score="0" Body="&lt;p&gt;Discretizing state/action space will always be a very expensive strategy, in fact I don't think you can do better than exponential time in state/action dimension that way.&lt;/p&gt;&#xA;&lt;p&gt;Now, you still haven't really explained your algorithm with the discretized states and actions. Given your known models, it sounds like you're planning on doing dynamic programming. This would definitely be an exponential time algorithm. You also haven't explained how you'd use your knowledge of the model in your policy gradient implementation. If you're just using it to sample some trajectories to have more data for updates, policy gradient methods will be much more efficient (poly-time). Otherwise, if you're planning on using your model for some sort of MCTS planning, depending on how you implement that, it could be very inefficient as well.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-23T14:03:14.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22113" PostTypeId="2" ParentId="21536" CreationDate="2020-06-23T14:38:55.440" Score="1" Body="&lt;p&gt;This seems to be inherited from the original Google implementation, which also uses 2 outputs (&lt;a href=&quot;https://github.com/google-research/bert/blob/master/run_pretraining.py#L293&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/google-research/bert/blob/master/run_pretraining.py#L293&lt;/a&gt;). I can see two possible reasons that the original implementation uses 2 outputs:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;They are using the cross entropy loss, which typically works with log probabilities. To get probabilities they use softmax activation, which requires an output for each class. It is possible, of course, to compute cross entropy from sigmoid activations (which would correspond to a 1-output architecture), but &lt;a href=&quot;https://stackoverflow.com/questions/59058867/sigmoid-output-can-it-be-interpreted-as-probability&quot;&gt;there seems to be some confusion&lt;/a&gt; as to whether the output of the sigmoid function should be used as a probability.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using 2 outputs can simplify the computation of the binary cross entropy loss, which, in typical Google fashion, is computed using low-level tensorflow ops rather than with &lt;code&gt;tf.nn.softmax_cross_entropy_with_logits&lt;/code&gt;. Specifically,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;-tf.reduce_sum(one_hot_labels * log_probs, axis=-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;one_hot_labels&lt;/code&gt; and &lt;code&gt;log_probs&lt;/code&gt; are &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{N \times 2}$&lt;/span&gt;, is much easier to read than&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-tf.reduce_sum(binary_labels * tf.math.log(probs) + (1 - binary_labels) + tf.math.log(1 - probs))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;binary_labels&lt;/code&gt; and &lt;code&gt;probs&lt;/code&gt; are &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^N$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37972" LastActivityDate="2020-06-23T14:38:55.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22114" PostTypeId="2" ParentId="22106" CreationDate="2020-06-23T15:24:46.133" Score="1" Body="&lt;p&gt;If you look at Keras' output, there are various steps which lose pixels:&lt;/p&gt;&#xA;&lt;p&gt;Max pooling on odd sizes will always lose one pixel. Conv2D using 3x3 kernels will also lose 2pixels, although I'm puzzled that it doesn't seem to happen in the downsampling steps.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, padding the original images with enough border pixels to compensate for the pixel loss due to the various layers would be the simplest solution. At the moment I can't calculate how much it should be, but I suspect rounding up to a multiple of 4 should take care of the max pooling layers. For denoising, borders could be just copied from the outermost pixels, probably with some sort of low pass filtering to avoid artefacts.&lt;/p&gt;&#xA;" OwnerUserId="22993" LastEditorUserId="2444" LastEditDate="2020-06-23T16:35:15.173" LastActivityDate="2020-06-23T16:35:15.173" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22116" PostTypeId="5" CreationDate="2020-06-23T16:27:48.727" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-23T16:27:48.727" LastActivityDate="2020-06-23T16:27:48.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22117" PostTypeId="4" CreationDate="2020-06-23T16:27:48.727" Score="0" Body="For questions related to the concept of stride in the context of convolutional neural networks and the convolution operation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-23T16:27:48.727" LastActivityDate="2020-06-23T16:27:48.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22118" PostTypeId="1" AcceptedAnswerId="22129" CreationDate="2020-06-23T16:41:36.270" Score="2" ViewCount="274" Body="&lt;p&gt;I am watching DeepMind's video lecture series on reinforcement learning, and when I was watching the video of model-free RL, the instructor said the Monte Carlo methods have less bias than temporal-difference methods. I understood the reasoning behind that, but I wanted to know what one means when they refer to bias-variance tradeoff in RL.&lt;/p&gt;&#xA;&lt;p&gt;Is bias-variance trade-off used in the same way as in machine learning or deep learning?&lt;/p&gt;&#xA;&lt;p&gt;(I am just a beginner and have just started learning RL, so I apologize if it is a silly question.)&lt;/p&gt;&#xA;" OwnerUserId="37911" LastEditorUserId="2444" LastEditDate="2020-06-23T16:44:49.523" LastActivityDate="2020-06-24T10:49:20.970" Title="What is the bias-variance trade-off in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22119" PostTypeId="1" AcceptedAnswerId="22120" CreationDate="2020-06-23T16:47:51.683" Score="6" ViewCount="619" Body="&lt;p&gt;I understand the two major branches of RL are Q-Learning and Policy Gradient methods.&lt;/p&gt;&#xA;&lt;p&gt;From my understanding (correct me if I'm wrong), policy gradient methods have an inherent exploration built-in as it selects actions using a probability distribution.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, DQN explores using the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy. Either selecting the best action or a random action.&lt;/p&gt;&#xA;&lt;p&gt;What if we use a softmax function to select the next action in DQN? Does that provide better exploration and policy convergence?&lt;/p&gt;&#xA;" OwnerUserId="38127" LastEditorUserId="2444" LastEditDate="2020-12-04T18:32:40.900" LastActivityDate="2020-12-04T18:32:40.900" Title="What happens when you select actions using softmax instead of epsilon greedy in DQN?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;policy-gradients&gt;&lt;epsilon-greedy-policy&gt;&lt;softmax-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22120" PostTypeId="2" ParentId="22119" CreationDate="2020-06-23T16:59:49.110" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;DQN on the other hand, explores using epsilon greedy exploration. Either selecting the best action or a random action.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a very common choice, because it is simple to implement and quite robust. However, it is not a requirement of DQN. You can use other action choice mechanisms, &lt;em&gt;provided&lt;/em&gt; all choices are covered with a non-zero probability of being selected.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What if we use a softmax function to select the next action in DQN? Does that provide better exploration and policy convergence?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It might in some circumstances. A key benefit is that it will tend to focus on action choices that are close to its current best guess at optimal. One problem is that if there is a large enough error in Q value estimates, it can get stuck as the exploration could heavily favour a current best value estimate. For instance, if one estimate is accurate and relatively high, but another estimate is much lower but in reality would be a good action choice, then the softmax probabilities to resample the bad estimate will be very low and it could take a very long time to fix.&lt;/p&gt;&#xA;&lt;p&gt;A more major problem is that the Q values are not independent logits that define preferences (whilst they would be in a Policy Gradient approach). The Q values have an inherent meaning and scale based on summed rewards. Which means that differences between optimal and non-optimal Q value estimates could be at any scale, maybe just 0.1 difference in value, or maybe 100 or more. This makes plain softmax a poor choice - it might suggest a near random exploration policy in one problem, and a near determinitsic policy in another, irrespective of what exploration might be useful at the current stage of learning.&lt;/p&gt;&#xA;&lt;p&gt;A fix for this is to use Gibbs/Boltzmann action selection, which modifies softmax by adding a scaling factor - often called &lt;em&gt;temperature&lt;/em&gt; and noted as &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; - to adjust the relative scale between action choices:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(a|s) = \frac{e^{q(s,a)/T}}{\sum_{x \in \mathcal{A}} e^{q(s,x)/T}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can work nicely to focus later exploration towards refining differences between actions that are likely to be good whilst only rarely making obvious mistakes. However it comes at a cost - you have to decide starting &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, the rate to decay &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and an end value of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. A rough idea of min/max action value that the agent is likely to estimate can help.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-23T17:06:40.560" LastActivityDate="2020-06-23T17:06:40.560" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22121" PostTypeId="2" ParentId="22105" CreationDate="2020-06-23T17:11:55.970" Score="0" Body="&lt;p&gt;Genetic algorithms, also known as evolutionary search, provide a general technique to optimize an objective function. We also say that we are trying to maximize fitness. This means that we are trying to find an individual with the highest possible fitness. We start with a population, say 100 individuals, and using mutation and crossover  we generate offspring among whom we hope to find fitter individuals and as the generations progress, we get better and better.&lt;/p&gt;&#xA;&lt;p&gt;One way to start this all is to think about the &amp;quot;fitness&amp;quot; or objective function. What is it that we want in the best individuals? Can we model that? How do we model that?&lt;/p&gt;&#xA;&lt;p&gt;In your case, does a specific measurement (those 5 numbers you mention) say how fit an individual is? And that fitness can be one number, say from 1 to 100, or it could be unbounded (as in real life where things get better and better with temporary regressions).&lt;/p&gt;&#xA;&lt;p&gt;So the challenge is how to map the features to a number. That's a math function to design.&lt;/p&gt;&#xA;&lt;p&gt;Genes are what change from individual to individual and they mutate from parent to offspring and they are shared in crossover. Given a set of genes, what is the fitness? If you can answer that question (meaning a mathematical function to map the genes of each individual to a number), then you have a genetic algorithm to run and it will find the fittest individuals according to your (math) function.&lt;/p&gt;&#xA;" OwnerUserId="38131" LastActivityDate="2020-06-23T17:11:55.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22122" PostTypeId="2" ParentId="22087" CreationDate="2020-06-23T17:38:07.900" Score="0" Body="&lt;p&gt;I think there are two problems with your network.&#xA;The first one, always having very similar outputs, is the rather simple one. As it seems, your network suffers from the so-called, very common &lt;a href=&quot;https://developers.google.com/machine-learning/gan/problems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mode Collapse&lt;/a&gt; problem. The attached link provides both an explanation and some potential remedy to that problem.&lt;/p&gt;&#xA;&lt;p&gt;The second problem is more fundamental. You say that you want your network to produce random numbers. Or numbers that at least appear as such. However, once training is finished, your model is going to be a static function which will not change any further. Given the same input x, it will always produce the same output y. Consequently, unless the inputs to your network contain some randomness already or are, at least, always slightly dissimilar, you will not end up having a random generator. So, whether that is going to be useful will depend on your usecase. But if you make sure that the true random variable (like date&amp;amp;time) serves as input to the RNN and the RNN just translates this into some different format, that might work again. Just keep in mind that randomness can never arise out of a trained model.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-06-23T17:38:07.900" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22123" PostTypeId="1" CreationDate="2020-06-23T18:01:22.970" Score="1" ViewCount="44" Body="&lt;p&gt;I have a tree that represents a hierarchical ontology of computer science topics (such as AI, data mining, IR, etc). Each node is a topic, and its child nodes are its sub-topics. Leaf nodes are weighted based on their occurrence in a given document.&lt;/p&gt;&#xA;&lt;p&gt;Is there a well-known algorithm or function to calculate the weight of inner nodes based on the weights of leaf nodes? Or is it totally based on the application to decide mathematical calculation of the weights?&lt;/p&gt;&#xA;&lt;p&gt;In my application, the node's weights should be some sort of accumulation of its child nodes weights. Is there a better mathematical formula or function to do that than just summing up weights of child nodes?&lt;/p&gt;&#xA;&lt;p&gt;I am not asking about traversal, but rather about weighting function.&lt;/p&gt;&#xA;" OwnerUserId="38132" LastEditorUserId="2444" LastEditDate="2020-06-23T22:14:15.963" LastActivityDate="2020-06-23T22:14:15.963" Title="Is there an algorithm to calculate the weights of an ontology tree's inner nodes?" Tags="&lt;ontology&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22124" PostTypeId="1" CreationDate="2020-06-23T19:22:31.740" Score="4" ViewCount="121" Body="&lt;p&gt;I've read that ANNs are based on how the human brain works. Now, I am reading about dropout.&#xA;Is some kind of dropout used in the human brain? Can we say that the ability to forget is some kind of dropout?&lt;/p&gt;&#xA;" OwnerUserId="36900" LastEditorUserId="2444" LastEditDate="2020-06-23T22:05:21.070" LastActivityDate="2020-06-24T00:48:29.677" Title="Is some kind of dropout used in the human brain?" Tags="&lt;machine-learning&gt;&lt;dropout&gt;&lt;human-inspired&gt;&lt;neuroscience&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22126" PostTypeId="2" ParentId="9954" CreationDate="2020-06-23T20:40:59.590" Score="0" Body="&lt;p&gt;An algorithm's bias and variance can be thought of as its property, this can be tweaked with things that we call as hyperparameters, but every algorithm has its own set of assumptions that it makes which if fulfilled, the algorithm performs better.&lt;/p&gt;&#xA;&lt;p&gt;Some algorithms such as Logistic Regression, linear-SVMs (not the kernel SVMs, because they can be used for non-linear problems as well) etc are linear models and work well if the data is linearly separable. If the data can not be separated by a linear plane, then no matter how much you tweak and fine-tune them, it won't work, because the data simply can not be separated by a linear plane, and that is the bias everyone talks about for these kinds of algorithms.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, Decision Trees can split the whole space into several hypercubes and based on which hypercube a datapoint is in, they classify that datapoint. KNNs on the other hand, use the neighbours of a datapoint and their types/properties to make predictions. Thus, a change in the positions of those datapoints will largely affect both the decision boundary(s) of both of these algorithms, and that is why they can be very easily overfitted and have a high variance.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="37911" LastEditorUserId="37911" LastEditDate="2020-06-24T14:36:53.507" LastActivityDate="2020-06-24T14:36:53.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22127" PostTypeId="1" AcceptedAnswerId="22305" CreationDate="2020-06-24T00:38:10.667" Score="1" ViewCount="112" Body="&lt;p&gt;The following is a statement and I am trying to figure out if it's true or false and why.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given a non-admissible heuristic function, A* will always give a solution if one exists, but there is no guarantee it will be optimal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I know that a non-admissible function is &lt;span class=&quot;math-container&quot;&gt;$h(n) &amp;gt; h^*(n)$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$h^*(n)$&lt;/span&gt; is the real cost to the goal), but I do not know if there is a guarantee.&lt;/p&gt;&#xA;&lt;p&gt;Which heuristics guarantee the optimality of A*? Is the admissibility of the heuristic always a necessary condition for A* to produce an optimal solution?&lt;/p&gt;&#xA;" OwnerUserId="38009" LastEditorUserId="2444" LastEditDate="2020-11-07T14:38:07.960" LastActivityDate="2020-11-07T14:40:05.440" Title="Which heuristics guarantee the optimality of A*?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;a-star&gt;&lt;admissible-heuristic&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22128" PostTypeId="2" ParentId="22124" CreationDate="2020-06-24T00:48:29.677" Score="5" Body="&lt;p&gt;The human brain works by having neurons constantly fire at different rates. So, if the firing rate increases, the neuron is transmitting overly exciting or calming information to further neurons connected to it. How other neurons connected to the former neuron respond on the messages sent by it, depends on the strength of the connection between the connected neurons.&#xA;However, having true dropout in the brain would be like having a neuron stop firing entirely for some time. As far as I know, this is not gonna happen unless a neuron dies. So, no, there doesn't seem to be any real biological equivalent to dropout in the brain. If you forget something, that is just caused by a weakening of some connections in the brain. But such a weakening (or strengthening alternatively) is a gradual process and is a function of the degree of firing synchrony between any two connected nodes.&lt;/p&gt;&#xA;&lt;p&gt;It is true that the introductory texts to Neural Nets tell you about those nets' neuro-scientific inspiration, but this similarity holds only in the most abstract way. Yes, you have neurons in both biological and artificial neural networks. But communication in the brain works in an asynchronous, continuous bio-chemical way, while communication in an artificial neural net works by taking the numeric state of one node, scaling it by the weight of the connection between two nodes, and then adding the weighted contribution of the previous node to the activation of the receiving node. So, already the fundamentals are very different between the two approaches (with the biological variant being much more complex and elaborate).&lt;/p&gt;&#xA;&lt;p&gt;If you want to find something that possesses more biologically inspired properties, look at how Convolutional Neural Networks are theoretically inspired by the working of the (human) visual system. Besides that, dropout is a purely mechanical strategy to regularize a model well, in order to enhance the model's generalization ability, i.e. making the model work well also on unseen data. It's just part of an algorithmic optimization procedure.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-06-24T00:48:29.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22129" PostTypeId="2" ParentId="22118" CreationDate="2020-06-24T01:00:36.763" Score="2" Body="&lt;p&gt;The bias-variance trade-off that you're referring to has to do with the return estimator. Any RL algorithm you choose needs some estimate of the cumulative return, which is a random variable with many sources of randomness, such as stochastic transitions or rewards. Monte Carlo RL algorithms estimate returns by running full trajectories and literally averaging the return achieved for each state. This imposes very few assumptions on the system (in fact, you don't even need the markovian property for these methods), so bias is low. However variance is high since each estimate depends on the literal trajectories that you observe. As such, you'll need many, many trajectories to get a good estimate of the value function.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, with TD methods, you estimate returns as &lt;span class=&quot;math-container&quot;&gt;$R_t + \gamma V(S_{t+1})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is your estimate of the value function. Using &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; this imposes some bias (for instance, the initialization of the value function at the beginning of training affects your next value function estimates), with the benefit of reducing variance. In TD learning, you don't need full environment rollouts to make a return estimate, you just need one transition. This also lets you make much better use of what you've learned about the value function, because you're learning how to infer value &amp;quot;piecewise&amp;quot; rather than just via literal trajectories that you happened to witness.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastEditorUserId="36821" LastEditDate="2020-06-24T10:49:20.970" LastActivityDate="2020-06-24T10:49:20.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22130" PostTypeId="1" AcceptedAnswerId="22132" CreationDate="2020-06-24T02:59:18.853" Score="3" ViewCount="94" Body="&lt;p&gt;I'm using the DQN algorithm to train my agent to play a turn-based game. The memory replay buffer stores tuples of experiences &lt;span class=&quot;math-container&quot;&gt;$(s, a, r, s')$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; are consecutive states. At the last turn, the game ends, and the non-zero reward is given to the agent. There are no more observations to be made and there is no next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; to store in the experience tuple. How should the final states be handled?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-06-24T11:50:01.467" LastActivityDate="2020-06-24T11:50:39.207" Title="How to handle the final state in experience replay?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22131" PostTypeId="1" CreationDate="2020-06-24T03:52:05.803" Score="0" ViewCount="24" Body="&lt;p&gt;I am trying to build an OCR which can read the Mathematical equations just like &lt;a href=&quot;https://mathpix.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAthPix&lt;/a&gt; and &lt;a href=&quot;https://github.com/harvardnlp/im2markup&quot; rel=&quot;nofollow noreferrer&quot;&gt;im2markup&lt;/a&gt;. im2markup by HarvardNLP seems like a good model but the thing is that it is built using Torch and I want to implement it using the Keras or Pytorch.&lt;/p&gt;&#xA;&lt;p&gt;I am thinking about using a CNN most preferably reinforcement learning like VGG or ResNet for feature extraction from the images just like in the paper &lt;a href=&quot;https://arxiv.org/abs/1502.03044&quot; rel=&quot;nofollow noreferrer&quot;&gt; Show, Attend and Tell: Neural Image Caption Generation with Visual Attention&lt;/a&gt;  which has an implementation in &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/image_captioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;image captioning using Attention&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which method for building the network should give me the best results as Decoder? I believe the Attention Mechanism is supposed to give me the best results and second bet is with CRNN.&lt;/p&gt;&#xA;&lt;p&gt;Also, is there a good pre-built model with PyTorch or Keras that I can use instead of building my own which produces satisfactory results? Please do suggest.&lt;/p&gt;&#xA;" OwnerUserId="36062" LastActivityDate="2020-06-24T03:52:05.803" Title="Which model structure is best suited to build a Math OCR (img2latex)? RAM, DRAM, CRNN or Attention OCR?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;attention&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22132" PostTypeId="2" ParentId="22130" CreationDate="2020-06-24T07:04:57.743" Score="2" Body="&lt;p&gt;You do not store a terminal state as &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in the replay table because by definition its value is always &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and there is no action, reward or next state. There is literally nothing to learn.&lt;/p&gt;&#xA;&lt;p&gt;However, you may find it useful to store information that &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is actually a terminal state, in case this is not obvious. That is typically achieved by storing an additional &lt;strong&gt;done&lt;/strong&gt; boolean component. This is useful, because it allows you to branch when calculating the TD target &lt;strong&gt;g&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;s, a, r, next_s, done = replay_memory_sample()&#xA;if done:&#xA;  g = r&#xA;else:&#xA;  g = r + gamma * max( q(s') )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-06-24T11:50:39.207" LastActivityDate="2020-06-24T11:50:39.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22133" PostTypeId="1" CreationDate="2020-06-24T07:37:40.870" Score="0" ViewCount="44" Body="&lt;p&gt;I am using computer-vision for detecting objects on the measuring table. In my tests, I used two techniques/frameworks: semantic segmentation (framework TensorFlow) and object recognition (framework Halcon). Both techniques lead to good results, and in this case, every new object needs to be labeled and trained.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Could this process be simplified, i.e. could the machine be taught how the measuring table (background) looks like and not how objects look like?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It seems that this solution would have many advantages, it would be enough to teach the machine what the measuring table looks like only once. In this case, it would not be necessary to teach the machine what the objects look like (the number of these objects can theoretically be very large). The problem I see here is a small number of different/suitable measuring table images that I can provide. This table does not change at all and theoretically, only one image is available.&lt;/p&gt;&#xA;" OwnerUserId="38145" LastActivityDate="2020-06-24T07:37:40.870" Title="Computer vision and deep learning with a small number of training images" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;computer-vision&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22134" PostTypeId="1" CreationDate="2020-06-24T08:00:51.617" Score="1" ViewCount="23" Body="&lt;p&gt;For CNN image recognition tasks, like object recognition/face recognition/object segmentation/posture recognition, are there experiment results about how much will the performance be degraded with monochrome images?&lt;/p&gt;&#xA;&lt;p&gt;The imaginary experiment is like:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Take the existing frameworks, reduce the channel number in the framework to fit monochrome images&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transform the existing training data and testing data to monochrome images&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Train the model with monochrome training data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the model to test the monochrome testing data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compare the result with the original result.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25322" LastEditorUserId="2444" LastEditDate="2020-06-24T11:57:40.637" LastActivityDate="2020-06-24T11:57:40.637" Title="How is the performance of a CNN trained with monochrome images on image recognition tasks degraded?" Tags="&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;reference-request&gt;&lt;data-preprocessing&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22135" PostTypeId="2" ParentId="22106" CreationDate="2020-06-24T08:16:35.593" Score="0" Body="&lt;p&gt;I don't know if this is the right way of doing it but I solved the problem.&lt;/p&gt;&#xA;&lt;p&gt;Following the code from above I've added:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;img_size = K.int_shape(input_img)[1:]&#xA;&#xA;resized_image_tensor = tf.image.resize(decoded, list(img_size[:2]))****&#xA;&#xA;&#xA;autoencoder = Model(input_img, resized_image_tensor)&#xA;autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;I used tf.image.resize to synchronize the shape of reconstructed image and input image.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="38093" LastActivityDate="2020-06-24T08:16:35.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22136" PostTypeId="1" AcceptedAnswerId="22143" CreationDate="2020-06-24T09:48:13.607" Score="1" ViewCount="45" Body="&lt;p&gt;Neural network training problems are oftentimes formulated as probability estimation problems (such as autoregressive models).&lt;/p&gt;&#xA;&lt;p&gt;How does one intuitively understand this idea?&lt;/p&gt;&#xA;" OwnerUserId="37099" LastEditorUserId="2444" LastEditDate="2020-06-24T12:01:54.297" LastActivityDate="2020-06-24T17:03:01.380" Title="Intuitively, why can the training of a neural network be formulated as a probability estimation problem?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;training&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22138" PostTypeId="2" ParentId="20939" CreationDate="2020-06-24T12:37:34.743" Score="1" Body="&lt;p&gt;I will answer my own question to try and provide some insights.&lt;/p&gt;&#xA;&lt;p&gt;My research supervisor suggested that I should use the &lt;a href=&quot;https://www.cns.nyu.edu/%7Elcv/ssim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSIM metric&lt;/a&gt; or some other well-known image processing metric (see the book &lt;a href=&quot;https://www.morganclaypool.com/doi/abs/10.2200/s00010ed1v01y200508ivm003&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Modern Image Quality Assessment&amp;quot; by Wang and Bovik&lt;/a&gt;) for assessing the visual similarity of an images.&lt;/p&gt;&#xA;&lt;p&gt;Another way I evaluate the performance of an autoencoder is by simply visually comparing the input and output images taken from the test set. This is by no means very scientific, but it gives a good idea whether an autoencoder is able to reconstruct the input images. One thing I would add here is that even if an autoencoder can reconstruct images perfectly, it doesn't mean that the encoding it learned is useful. For example, when I wanted similar images to be mapped to similar encodings, the autoencoder that was able to do that better was outputting more blurred reconstructed images in comparison to the autoencoder that wasn't achieving this similarity preservation (but was outputting better reconstructions).&lt;/p&gt;&#xA;" OwnerUserId="36769" LastActivityDate="2020-06-24T12:37:34.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22139" PostTypeId="1" CreationDate="2020-06-24T13:20:36.460" Score="1" ViewCount="31" Body="&lt;p&gt;&lt;strong&gt;Is there a guide how much data do you need for making successful denoising model using autoencoders?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or the rule is, the more data, the better it is?&lt;/p&gt;&#xA;&lt;p&gt;I tried with small dataset 350 samples,  to see what I will get as an output. And I failed. :D&lt;/p&gt;&#xA;" OwnerUserId="38093" LastEditorUserId="2444" LastEditDate="2020-06-24T23:12:21.513" LastActivityDate="2020-06-24T23:12:21.513" Title="How much data do we need for making a successful de-noising auto-encoder?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;autoencoders&gt;&lt;denoising-autoencoder&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22140" PostTypeId="1" AcceptedAnswerId="22141" CreationDate="2020-06-24T13:25:33.800" Score="1" ViewCount="57" Body="&lt;p&gt;Suppose we have a small space state and that, after about 2000 episodes, we've accurately explored the environment and known the accurate &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values. In that case, why do we still leave a small probability for exploration?&lt;/p&gt;&#xA;&lt;p&gt;My guess is in the case of a dynamic environment where a bigger reward might pop up in another state. Is my assumption correct?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-24T14:17:38.077" LastActivityDate="2020-06-24T14:18:28.403" Title="Why do we explore after we have an accurate estimate of the value function?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22141" PostTypeId="2" ParentId="22140" CreationDate="2020-06-24T13:44:26.447" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Suppose we have a small space state and that, after about 2000 episodes, we've accurately explored the environment and known the accurate &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values. In that case, why do we still leave a small probability for exploration?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It will depend on the goal of the work:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the learning algorithm is off-policy (e.g. Q learning), it is normal to continue to explore at a moderate-to-low rate because it can accurately estimate an optimal deterministic target policy from a close-to-optimal stochastic behaviour policy.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perhaps it is engineered with a low tolerance and will keep going even when you don't need it to.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perhaps the code is for education and run so long that convergence is easily visible. Or for comparison with other methods which really do take that long to converge, and you would like data on the same axis.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For comparison with other methods for sample efficiency whilst learning and measuring &lt;a href=&quot;https://www.coursera.org/lecture/practical-rl/regret-measuring-the-quality-of-exploration-mNxVA&quot; rel=&quot;nofollow noreferrer&quot;&gt;regret&lt;/a&gt; (i.e. how much the exploration is costing you).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When environment is dynamic and could change, then continuous exploration is potentially useful to discover the changes, as you suggest in the question.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you do really have an ideal agent, then of course you could just stop and say &amp;quot;job done&amp;quot;. In practice for more interesting problems, you won't usually get small state spaces and perfect solutions inside 2000 episodes (or ever) - as a result if you are reading tutorials in reinforcement learning, they may just skip this point.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-06-24T14:18:28.403" LastActivityDate="2020-06-24T14:18:28.403" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22142" PostTypeId="2" ParentId="22140" CreationDate="2020-06-24T13:44:41.203" Score="0" Body="&lt;p&gt;When you are training a system using stochastic gradient descent, your system will converge towards some local minimum. If the local minimum was a good one, we would be fine with it. However, we cannot know how good a found solution is in comparison to other solutions of which we do not know their quality because they have been insufficiently explored. So, continuing to explore is a good way to escape comparatively bad local minima even if training has progressed already for quite a bit.&lt;/p&gt;&#xA;&lt;p&gt;Besides that, maybe even more importantly towards the end of training, one also wants the system to perform well, i.e. robustly, in the presence of noise and not just under ideal circumstances. So, introducing some randomness, i.e. noise, into the network's policy can also lead to more robust policies being learned since the agent gets trained on how to best recover failure/unforeseen transitions into unexpected states.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-06-24T13:44:41.203" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="22143" PostTypeId="2" ParentId="22136" CreationDate="2020-06-24T15:06:54.853" Score="1" Body="&lt;p&gt;Consider the case of binary classification, i.e. you want to classify each input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; into one of two classes: &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt;. For example, in the context of object classification, &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; could be &amp;quot;cat&amp;quot; and &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt; could be &amp;quot;dog&amp;quot;, and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an image that contains one main object.&lt;/p&gt;&#xA;&lt;p&gt;In certain cases, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; cannot be easily classified. For example, in object classification, if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a blurred image where there's some uncertainty about the object in the image, what should the output of the neural network be? Should it be &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt;, or maybe it should be an &lt;em&gt;uncertainty value&lt;/em&gt; (i.e. a probability) that lies between &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt;? The last option is probably the most reasonable, but also the most general one (in the sense that it can also be used in the case there's little or no uncertainty about what the object is).&lt;/p&gt;&#xA;&lt;p&gt;That's the reason why we can model or formulate this (or other) supervised learning problem(s) as the estimation of a probability value (or probability distribution).&lt;/p&gt;&#xA;&lt;p&gt;To be more concrete, you can formulate this binary classification problem as the estimation of the following probability&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;P(y_1 \mid x, \theta_i) \in [0, 1] \label{1}\tag{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; is the first class (or label), &lt;span class=&quot;math-container&quot;&gt;$(x, y) \in \mathcal{D}$&lt;/span&gt; is a labeled training example, where &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the ground-truth label for the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\theta_i$&lt;/span&gt; are the parameters of the neural network at iteration &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, so, intuitively, &lt;span class=&quot;math-container&quot;&gt;$P(y_1 \mid x, \theta_i) $&lt;/span&gt; is a probability that represents how likely the neural network thinks that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; belongs to the class &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt; given the current estimate of the parameters. The probability that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; belongs to the other class is just &lt;span class=&quot;math-container&quot;&gt;$1 - P(y_1 \mid x, \theta_i) = P(y_2 \mid x, \theta_i)$&lt;/span&gt;.  In this specific case, I have added a subscript to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; to indicate that this probability depends on the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th estimate of the parameters of the neural network.&lt;/p&gt;&#xA;&lt;p&gt;Once you have &lt;span class=&quot;math-container&quot;&gt;$P(y_1 \mid x, \theta_i)$&lt;/span&gt;, if you want to perform classification, you will actually need to choose a threshold value &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, such that, if &lt;span class=&quot;math-container&quot;&gt;$P(y_1 \mid x, \theta_i) &amp;gt; t$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is classified as &lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt;, else it is classified as &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt;. This threshold value &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; can be &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;, but it can also not be.&lt;/p&gt;&#xA;&lt;p&gt;Note that, in the case above, &lt;span class=&quot;math-container&quot;&gt;$P(y_1 \mid x, \theta_i)$&lt;/span&gt; is a number and not a probability distribution. However, in certain cases, you can also formulate your supervised learning problem so that the output is a probability distribution (rather just a probability). There are also other problems where you don't estimate a conditional probability but maybe a joint probability, but the case above is probably the simplest one that should give you the intuition behind the idea of formulating machine learning problems as the estimation of probabilities or probability distributions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-24T17:03:01.380" LastActivityDate="2020-06-24T17:03:01.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22144" PostTypeId="1" CreationDate="2020-06-24T17:21:20.247" Score="0" ViewCount="62" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1811.01458.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning&lt;/a&gt;, it is written&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We allow centralised training but require decentralised execution, from which follows that the policies &lt;span class=&quot;math-container&quot;&gt;$\pi^a$&lt;/span&gt; are known to all agents.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My confusion stems from e.g. the following scenario: In an &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; player game, all &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; players share parameters in a single agent network. In such a scenario, when we move on to decentralised execution, how does this take place if parameters for all &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; players were shared across a single network during training?&lt;/p&gt;&#xA;" OwnerUserId="33448" LastEditorUserId="2444" LastEditDate="2020-06-24T21:45:17.523" LastActivityDate="2020-06-24T21:45:17.523" Title="How is centralised training and decentralised execution in multi agent reinforcement learning implemented?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;deep-rl&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22145" PostTypeId="1" CreationDate="2020-06-24T17:47:00.280" Score="0" ViewCount="16" Body="&lt;p&gt;New to NN, I'm looking to get advice for the architectural implementation using tensorflow of a neural net for defect detection in the material as well as suggested image preprocessing to improve NN performance.&lt;/p&gt;&#xA;&lt;p&gt;Here some example of what I would like to detect:&lt;/p&gt;&#xA;&lt;p&gt;Cracks in cells:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nXOeP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nXOeP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;uneven intensity:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HvNhl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HvNhl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;micro cracks:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TY9se.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TY9se.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;hotspots:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1elEm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1elEm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;big cracks:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BjJNH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BjJNH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Breakage:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ir0vz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ir0vz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I intend to use a R-CNN network trained on the different type of failure.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would it be better to create different NN for different types of effect and running them independently, or should I try to make a model that can directly differentiate?&lt;/li&gt;&#xA;&lt;li&gt;Is a R-CNN network a good approach to this problem, is there better fitted NN ?&lt;/li&gt;&#xA;&lt;li&gt;The exposure (aka how bright is the image) can vary, would it be advised to try to correct for it or the NN wouldn't care?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="38153" LastActivityDate="2020-06-24T17:47:00.280" Title="NN for defect detection" Tags="&lt;convolutional-neural-networks&gt;&lt;materials-science&gt;&lt;r-cnn&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22146" PostTypeId="1" CreationDate="2020-06-24T18:48:44.990" Score="2" ViewCount="55" Body="&lt;p&gt;Is there a good website where I can learn about Deep Deterministic Policy Gradient?&lt;/p&gt;&#xA;" OwnerUserId="37947" LastEditorUserId="2444" LastEditDate="2020-06-24T20:24:49.663" LastActivityDate="2020-11-21T22:07:15.730" Title="Is there a good website where I can learn about Deep Deterministic Policy Gradient?" Tags="&lt;reinforcement-learning&gt;&lt;resource-request&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22147" PostTypeId="2" ParentId="22146" CreationDate="2020-06-24T18:57:53.067" Score="2" Body="&lt;p&gt;Spinning Up by Open Ai.&lt;/p&gt;&#xA;&lt;p&gt;Be sure to read up Part 3 (Intro to Policy Optimisation) before you move on to : &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33448" LastActivityDate="2020-06-24T18:57:53.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22148" PostTypeId="1" AcceptedAnswerId="22151" CreationDate="2020-06-24T20:36:26.980" Score="0" ViewCount="36" Body="&lt;p&gt;What is the advantage of having a stochastic/probabilistic classification procedure?&lt;/p&gt;&#xA;&lt;p&gt;The classifiers I have encountered so far are as follows. Suppose we have two outcomes &lt;span class=&quot;math-container&quot;&gt;$A = \{0,1\}$&lt;/span&gt;. Given a feature vector &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, we have calculated a probability for each outcome and return the outcome &lt;span class=&quot;math-container&quot;&gt;$a \in A$&lt;/span&gt; for which the probability is highest.&lt;/p&gt;&#xA;&lt;p&gt;Now, I encountered a classification procedure as follows: first, map each &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to a probability distribution on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; by a mapping &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;. To classify &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, choose an outcome &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; according to the distribution &lt;span class=&quot;math-container&quot;&gt;$H(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why not use the deterministic classification? Suppose &lt;span class=&quot;math-container&quot;&gt;$H(x)$&lt;/span&gt; is 1 with probability &lt;span class=&quot;math-container&quot;&gt;$0.75$&lt;/span&gt;. Then, the obvious choice for an outcome would be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36116" LastEditorUserId="2444" LastEditDate="2020-06-24T23:52:00.577" LastActivityDate="2020-06-24T23:53:11.147" Title="What is the advantage of having a stochastic classification procedure?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22149" PostTypeId="2" ParentId="22080" CreationDate="2020-06-24T21:08:25.627" Score="0" Body="&lt;p&gt;I compared my results visually to a second implementation known to be working - &amp;quot;The annotated transformer&amp;quot;.  I compared the pytorch calculation results of the attention-method to my implementation results.&lt;/p&gt;&#xA;&lt;p&gt;The answe is - the softmax is applied row by row. Therefore the resulting matrix p-attn is not equal to its transposed version.&lt;/p&gt;&#xA;" OwnerUserId="38099" LastActivityDate="2020-06-24T21:08:25.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22150" PostTypeId="1" CreationDate="2020-06-24T21:40:05.713" Score="2" ViewCount="35" Body="&lt;p&gt;&lt;a href=&quot;https://www.mitpressjournals.org/doi/pdf/10.1162/artl_a_00285&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geb is an alife simulation&lt;/a&gt; that as far as I know passes all of the tests we have tried to come up with in defining open endedness. However, when you actually run the code, the behavioral complexity certainly increases, but the physical bodies of the creatures never changes (and cannot change), and Geb bodies are the only thing present in the world.&lt;/p&gt;&#xA;&lt;p&gt;Tool development, or at least developing new physical capabilities and new “actions” seems to be a crucial part of what makes evolution open ended. I think the problem with Geb is that the evolution and progress all takes place in the networks and network production rules, which are systems outside the physical world. They are external systems that take in data from the world and output actions for the agents. So while this rich complexity and innovation is occurring, it’s not integrated with the agents actions and physical bodies.&lt;/p&gt;&#xA;&lt;p&gt;This leads to a simple question: is there an alife system that passes all the same tests Geb does, but is “fully embedded” in its world? In the sense that any mechanism agents use to make actions must be part of the physical world of those agents, and subject to the same rules as the bodies of the agents themselves.&lt;/p&gt;&#xA;&lt;p&gt;What I’m saying here is loose, you could come up with plenty of weird edge cases that meet what I ask exactly without meeting my intent. And perhaps being fully embodied isn’t necessary, just being &lt;em&gt;more&lt;/em&gt; embodied would be enough. What my intent is is to ask if we have any systems that pass the open endedness tests Geb have passed, but have the innovation occurring in a way that leads to emergent growth of “actions” and emergent growth of “bodies” because to evolve and do better those aspects must be improved as well.&lt;/p&gt;&#xA;" OwnerUserId="6378" LastEditorUserId="1847" LastEditDate="2020-06-25T09:23:11.127" LastActivityDate="2020-06-25T09:23:11.127" Title="Artificial life simulator that is fully embodied and passes open endedness tests" Tags="&lt;evolutionary-algorithms&gt;&lt;artificial-life&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22151" PostTypeId="2" ParentId="22148" CreationDate="2020-06-24T22:34:52.340" Score="1" Body="&lt;p&gt;There are multiple potential reasons for having stochastic predictions (instead of categorical/binary).&lt;/p&gt;&#xA;&lt;p&gt;First, it often simplifies training and improves the training outcome when training a classifier on producing probabilities per class. For example, it allows for the usage of many nice loss functions like the Mean Squared Error (MSE), which is compatible with the famous back-prop algorithm. Moreover, improving classification based on the loss computed from stochastic probabilities per class allows for improving the classifier even further when the counting loss computed on predicted class labels has reached 0 already. So, if you only have binary decisions and your classifier gets all classes correct, training can stop immediately since the loss becomes 0. However, when predicting probabilities per class and iteratively driving the probability of the correct class towards 1, training can continue for much longer, even after all classes have been classified correctly already. So, the classifier trained on making probabilistic predictions can in the end perform much better since training can progress for much longer, allowing for increasingly pronounced discrimination between classes with every update. A nice introduction is given in this Stanford &lt;a href=&quot;https://www.youtube.com/watch?v=h7iBpEHGVNc&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture recording&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Second, it is often nice for a user to know how much certainty versus uncertainty there is in a classification. If one class has 80% probability, this is a pretty clear decision in favor of the 80%-probability-class. However, if you only get categorical class labels returned by your classifier, you cannot possibly know whether the evidence in favor of the winning class was only marginally higher than that of the other class (as measured by the classifier) or whether there was a clear difference.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="2444" LastEditDate="2020-06-24T23:53:11.147" LastActivityDate="2020-06-24T23:53:11.147" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22152" PostTypeId="5" CreationDate="2020-06-24T23:13:04.840" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-24T23:13:04.840" LastActivityDate="2020-06-24T23:13:04.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22153" PostTypeId="4" CreationDate="2020-06-24T23:13:04.840" Score="0" Body="For questions related to the de-noising auto-encoder model." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-24T23:13:04.840" LastActivityDate="2020-06-24T23:13:04.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22154" PostTypeId="1" AcceptedAnswerId="22830" CreationDate="2020-06-24T23:55:48.977" Score="2" ViewCount="58" Body="&lt;p&gt;Everybody knows how successful transformers have been in NLP. Is there known work on other domains (e.g that also have a sequential natural way of occurring, such as stock price prediction or other problems)?&lt;/p&gt;&#xA;" OwnerUserId="36341" LastEditorUserId="2444" LastEditDate="2020-06-25T00:04:35.303" LastActivityDate="2020-08-02T16:11:35.317" Title="Do transformers have success in other domains different than NLP?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;applications&gt;&lt;sequence-modelling&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22156" PostTypeId="1" CreationDate="2020-06-25T03:55:41.967" Score="0" ViewCount="35" Body="&lt;p&gt;After reading up one Deep Deterministic Policy Gradient, I found this example on MATLAB:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/help/reinforcement-learning/ug/train-agent-to-control-flying-robot.html#TrainDDPGAgentToControlFlyingRobotExample-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/help/reinforcement-learning/ug/train-agent-to-control-flying-robot.html#TrainDDPGAgentToControlFlyingRobotExample-4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is the following: In DDPG, we plug in the Observation to our Actor to get our actions. The observations in the MATLAB environment are 7: x, y, dx, dy, sin, cos, dtheta. However, only x and y are assigned in the beginning. Does that mean that the rest are given random values before placed in the Critic Network? If my understanding is wrong, could someone please explain to me what is occurring in this model? Thank You&lt;/p&gt;&#xA;" OwnerUserId="37947" LastEditorUserId="37947" LastEditDate="2020-06-25T15:10:07.193" LastActivityDate="2020-06-25T15:10:07.193" Title="2 out of 7 Observations Defined in MATLAB DDPG Reinforcement Learning Environment. Are the rest given random values?" Tags="&lt;policy-gradients&gt;&lt;matlab&gt;&lt;ddpg&gt;&lt;deterministic-policy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22158" PostTypeId="2" ParentId="10644" CreationDate="2020-06-25T08:57:46.517" Score="0" Body="&lt;p&gt;I believe that mathematical theorems are social constructions which are formalised by virtue of rigorous proofs facilitated by an academic peer review process; in other words, I am not a mathematical Platonist. You ask: “Can we define the AI singularity mathematically?” I personally see no reason why the so-called AI singularity cannot be defined in mathematical terms. Let us pretend that a gifted mathematician is able to provide a convincing and logically consistent set of rigorous proofs for a mathematical conjecture pertaining to the AI singularity. The aforementioned mathematician then submits the paper to a highly prestigious mathematics journal, such as &lt;a href=&quot;https://www.ams.org/publications/journals/journalsframework/jams&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal of the American Mathematical Society&lt;/a&gt;. If this paper passes the stringent academic peer review process for publication, then this would constitute peer acceptance that the AI singularity can indeed be defined in mathematical terms.&lt;/p&gt;&#xA;" OwnerUserId="38152" LastActivityDate="2020-06-25T08:57:46.517" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22163" PostTypeId="1" CreationDate="2020-06-25T10:36:20.807" Score="0" ViewCount="14" Body="&lt;p&gt;In &lt;a href=&quot;https://papers.nips.cc/paper/5353-two-stream-convolutional-networks-for-action-recognition-in-videos.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;two-stream CNN paper&lt;/a&gt;, they have the following network architecture:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ukJqB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ukJqB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can use a simple average or multiclass linear SVM for the class score fusion. In the case of averaging the outputs of two steams, it is plausible that we can train spatial stream and temporal stream separately and in the test phase we will just average both outputs.&lt;/p&gt;&#xA;&lt;p&gt;But what happens if we have multiclass linear SVM as the class score fusion. Can we still train the spatial stream and temporal stream separately?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-06-25T11:38:01.490" LastActivityDate="2020-06-25T11:38:01.490" Title="Do we train the spatial net and temporal net separately in two stream CNN for action recognition from video frame?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22166" PostTypeId="1" AcceptedAnswerId="22168" CreationDate="2020-06-25T11:39:14.180" Score="0" ViewCount="101" Body="&lt;p&gt;Why is non-linearity desirable in a neural network?&lt;/p&gt;&#xA;&lt;p&gt;I couldn't find satisfactory answers to this question on the web. I typically get answers like &amp;quot;real-world problems require non-linear solutions, which are not trivial. So, we use non-linear activation functions for non-linearity&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="38172" LastEditorUserId="2444" LastEditDate="2020-06-25T11:55:29.563" LastActivityDate="2021-03-29T16:03:52.067" Title="Why is non-linearity desirable in a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22167" PostTypeId="1" CreationDate="2020-06-25T12:21:57.820" Score="0" ViewCount="34" Body="&lt;p&gt;I was running into a situation with a data set like this&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DQvYa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DQvYa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have 4 events and and they might happen together in pairs. I want to use 3 features to predict the coupling between event.&#xA;I am building a deep learning neural network using Keras and I am not sure what kind of loss function and metric is suitable for this kind of problem since the output is not one hot.&lt;/p&gt;&#xA;&lt;p&gt;Note that from line 2, (Event 1 and Event 3) and ( Event 2 and Event 4) can happen together !&lt;/p&gt;&#xA;" OwnerUserId="37297" LastEditorUserId="37297" LastEditDate="2020-06-26T10:44:04.100" LastActivityDate="2020-06-26T10:44:04.100" Title="Which loss function and evaluation metric should I use for a multiple output prediction problem?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;keras&gt;&lt;loss-functions&gt;&lt;metric&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22168" PostTypeId="2" ParentId="22166" CreationDate="2020-06-25T13:06:44.700" Score="0" Body="&lt;p&gt;Consider what happens if you intend to train a linear classifier on replicating something trivial as the XOR function. If you program/train the classifier (of arbitrary size) such that it outputs &lt;code&gt;XOR condition is met&lt;/code&gt; whenever feature &lt;code&gt;a&lt;/code&gt; or feature &lt;code&gt;b&lt;/code&gt; are present, then the linear classifier will also (incorrectly) output &lt;code&gt;XOR condition is met&lt;/code&gt; whenever both features together are present. That is because linear classifiers simply sum up contributions of all features and work with the total weighted inputs they receive. For our example, that means that when the weighted contribution of either feature is sufficient already to trigger the classifier to output &lt;code&gt;XOR condition is met&lt;/code&gt;, then obviously also the summed contributions of both features are sufficient to trigger the same response.&lt;/p&gt;&#xA;&lt;p&gt;To get a classifier that is capable of outputting &lt;code&gt;XOR condition is met&lt;/code&gt; &lt;em&gt;if and only if&lt;/em&gt; the summed contributions of all input features are above a lower threshold &lt;em&gt;and&lt;/em&gt; below an upper threshold, commonly non-linearities are introduced. You could of course also try to employ a quadratic function to solve the two-feature problem, but as soon as the number of variables/features increases again, you run into the same problem again, only in higher dimensions. Therefore, the most general approach to solving this problem of learning non-linear functions, like the &lt;code&gt;XOR&lt;/code&gt;, is by setting up large models with enough capacity to learn a given task and equipping them with non-linearities. That simplifies training since it allows for using stochastic gradient descent for training the system/classifier, preventing one from having to solve Higher-Degree polynomial equations analytically (which can get computationally quite expensive quite quickly) to solve some task.&lt;/p&gt;&#xA;&lt;p&gt;In case you are interested, &lt;a href=&quot;https://ieeexplore.ieee.org/document/1194667&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's one paper&lt;/a&gt; analyzing and dealing with the &lt;code&gt;XOR&lt;/code&gt; problem (as one concrete instance of a problem where purely linear models fail to solve some task).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can consider a layer in a network as a function &lt;span class=&quot;math-container&quot;&gt;$y = f(x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the input to some layer &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the output of the layer. As you propagate &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, being the network's input, through the network, you get something like &lt;span class=&quot;math-container&quot;&gt;$y = p(t(h(g(f(x)))))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the input layer and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; constitutes the output layer, i.e. a set of weights, by which the input to that respective layer gets multiplied.&#xA;If &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, for example, is some non-linear activation function, like &lt;code&gt;ReLU&lt;/code&gt; or &lt;code&gt;sigmoid&lt;/code&gt;, then &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, being the network's output, is a non-linear function of input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="36737" LastEditDate="2021-03-28T17:53:35.830" LastActivityDate="2021-03-28T17:53:35.830" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22169" PostTypeId="1" CreationDate="2020-06-25T13:27:52.517" Score="1" ViewCount="20" Body="&lt;p&gt;I am trying to create a Graph NN that will be able to predict the forces in truss elements of a space frame.&lt;/p&gt;&#xA;&lt;p&gt;The input for the NN will be a graph, where the nodes represent the nodes of the spaceframe. And the output should be the forces in the edges of the supplied graph/frame.&lt;/p&gt;&#xA;&lt;p&gt;The problem that I am facing is that for the NN to be beneficial I need to encode a lot of data per node:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3 floats for the position of the node,&lt;/li&gt;&#xA;&lt;li&gt;3 floats for the vector of the force applied to the node,&lt;/li&gt;&#xA;&lt;li&gt;a boolean/int to determine whether a node is a support.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am not sure how to design my Graph NN to allow for so many parameters per node.&lt;/p&gt;&#xA;&lt;p&gt;Maybe I should try a different approach?&lt;/p&gt;&#xA;&lt;p&gt;Any help is greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="38176" LastEditorUserId="2444" LastEditDate="2020-06-25T16:32:03.133" LastActivityDate="2020-06-25T16:32:03.133" Title="How to design a graph neural network to predict the forces in truss elements of a space frame?" Tags="&lt;ai-design&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22170" PostTypeId="1" CreationDate="2020-06-25T13:30:04.290" Score="1" ViewCount="38" Body="&lt;p&gt;Let's say we have a dynamic environment: a new state gets added after 2000 episodes have been done. So, we leave room for exploration, so that it can discover the new state.&lt;/p&gt;&#xA;&lt;p&gt;When it gets to that new state, it has no idea of the Q values, and, since we're past 2000 episodes, our exploration rate is very low. What happens if try to exploit when all Q values are 0?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-25T15:56:05.870" LastActivityDate="2021-03-22T18:00:40.137" Title="How to deal with the addition of a new state to the environment during training?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22171" PostTypeId="2" ParentId="22170" CreationDate="2020-06-25T14:12:49.923" Score="0" Body="&lt;p&gt;There are several ways to tackle this, although exploration is definitely not a solved problem yet ;)&lt;/p&gt;&#xA;&lt;p&gt;In general, I believe the right thing to do here is to measure the uncertainty of your policy or Q-value estimates and use that to construct some sort of exploration bonus. An intuitive example is given in &lt;a href=&quot;https://arxiv.org/abs/1810.12894&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Exploration by Random Network Distillation&quot;&gt;Exploration by Random Network Distillation&lt;/a&gt;. They make two randomly initialized neural nets, one of which is never updated. At every transition, they feed transition data through both neural nets and use the difference in output between them as an estimate of uncertainty, and this quantity is added to the reward. Then they update the modifiable neural net towards the other one. This way, on a completely novel transition, the two neural nets will likely have very different outputs so the reward will be augmented a lot. Of course, this will hopefully encourage the agent to explore.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-25T14:12:49.923" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22172" PostTypeId="1" AcceptedAnswerId="22174" CreationDate="2020-06-25T14:40:17.107" Score="1" ViewCount="46" Body="&lt;p&gt;Our length of feature representation decreases as we go deeper into the CNN, I mean to say that horizontal and vertical lengths decrease while depth(channels) increase. So, how will the input be preserved, since there won't be any data left at the end of the network, where we connect, to say Multi Layer Perceptrons?&lt;/p&gt;&#xA;" OwnerUserId="38060" LastEditorUserId="38060" LastEditDate="2020-07-06T07:46:47.617" LastActivityDate="2020-07-06T07:46:47.617" Title="How will the input be preserved as we go deeper in CNN, where dimensions decrease drastically?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22173" PostTypeId="1" CreationDate="2020-06-25T14:45:01.570" Score="0" ViewCount="25" Body="&lt;p&gt;I am training a NN for blobs vs non-blobs recognition.&lt;/p&gt;&#xA;&lt;p&gt;Blobs example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/huoIi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/huoIi.png&quot; alt=&quot;blobs example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Non-blobs:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dqkRq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dqkRq.png&quot; alt=&quot;non-blobs example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Keras architecture is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential() activation = &amp;quot;relu&amp;quot; model = Sequential([&#xA;    Convolution2D(8,(3,3), activation='relu', input_shape=input_shape),&#xA;    Convolution2D(8,(3,3), activation='relu'),&#xA;    MaxPooling2D(),&#xA;    Convolution2D(16,(3,3), activation='relu'),&#xA;    Convolution2D(16,(3,3), activation='relu'),&#xA;    MaxPooling2D(),&#xA;    Flatten(),&#xA;    Dense(1024, activation='relu'),&#xA;    Dense(128, activation='relu'),&#xA;    Dense(16, activation='relu'),&#xA;    Dense(1, activation='sigmoid')   ]) opt = SGD(lr=0.01) model.compile(loss=&amp;quot;binary_crossentropy&amp;quot;, optimizer=opt, metrics=[&amp;quot;accuracy&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Although training accuracy is about 96-97%, there are 25-30% of false-positive cases.&lt;/p&gt;&#xA;&lt;p&gt;Tried different things, like add more convolution layers, change number of filters, add dense layers - no difference, the mentioned model is pretty much the best.&lt;/p&gt;&#xA;&lt;p&gt;I appreciate any advice to improve the recognition.&lt;/p&gt;&#xA;" OwnerUserId="38177" LastEditorUserId="38177" LastEditDate="2020-06-25T16:49:56.023" LastActivityDate="2020-06-25T16:49:56.023" Title="Binary classification to recognize blobs on pictures generates many false-positive results" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;image-recognition&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22174" PostTypeId="2" ParentId="22172" CreationDate="2020-06-25T16:13:56.653" Score="2" Body="&lt;p&gt;You can also think of a convolutional neural network (CNN) as an encoder, i.e. a neural network that learns a smaller representation of the input, which then acts as the feature vector (input) to a fully connected network (or another neural network). In fact, there are CNNs that can be thought of as auto-encoders (i.e. an encoder followed by a decoder): for example, the &lt;a href=&quot;https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;u-net&lt;/a&gt; can indeed be thought of as an auto-encoder.&lt;/p&gt;&#xA;&lt;p&gt;Although it is (almost) never the case that you transform the input to an extremely small feature vector (e.g. a number), even a single float-pointing number can encode a lot of information. For example, if you want to classify the object in the image into one of two classes (assuming there is only one main object in the image), then a floating-point is more than sufficient (in fact, you just need one bit to encode that information).&lt;/p&gt;&#xA;&lt;p&gt;This smaller representation (the &lt;strong&gt;feature&lt;/strong&gt; vector) that is then fed to a fully connected network is learned based on the information in your given training data. In fact, CNNs are known as &lt;strong&gt;data-driven feature extractors&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am not aware of any theoretical guarantee that ensures that the learned representation is the best suited for your task (probably you need to look into &lt;em&gt;learning theory&lt;/em&gt; to know more about this). In practice, the quality of the learned feature vector will mainly depend on your available data and the inductive bias (i.e. the assumptions that you make, which are also affected by the specific neural network architecture that you choose).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-25T16:24:55.287" LastActivityDate="2020-06-25T16:24:55.287" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22177" PostTypeId="1" AcceptedAnswerId="22179" CreationDate="2020-06-25T17:51:37.737" Score="0" ViewCount="76" Body="&lt;p&gt;I am looking at a lecture on &lt;a href=&quot;https://youtu.be/I2uSCTUHsUI?t=3951&quot; rel=&quot;nofollow noreferrer&quot;&gt;POMDP&lt;/a&gt;, and the context is that, when the quadcopter can't see the landmarks, it has to use reckoning. And then he mentions the transition model is not deterministic, hence the uncertainty grows.&lt;/p&gt;&#xA;&lt;p&gt;Can transition models in MDP be deterministic?&lt;/p&gt;&#xA;" OwnerUserId="36047" LastEditorUserId="2444" LastEditDate="2020-06-25T23:36:54.420" LastActivityDate="2020-06-26T22:24:15.380" Title="Does &quot;transition model&quot; alone in an MDP imply it's non-deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;&lt;transition-model&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22179" PostTypeId="2" ParentId="22177" CreationDate="2020-06-25T18:34:36.780" Score="0" Body="&lt;p&gt;Yes, in which case, it will be more like a search problem, if it is not POMDP and have finite number of states. Or you can use the same framework (used for POMDP) with constrained (deterministic) transition matrix modeling for model-based systems.&lt;/p&gt;&#xA;&lt;p&gt;If you think about it after you train any model/agent with an MDP modeling, during test time the optimal strategy is generally deterministic, i.e., given a feature/state you will take a particular action even if it has more than one non-zero element in each row of the transition matrix.&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-06-25T18:34:36.780" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22180" PostTypeId="2" ParentId="11672" CreationDate="2020-06-25T20:22:04.743" Score="0" Body="&lt;p&gt;Differentiability for activation functions is desirable but not necessary since you can reformulate the derivatives at the non-differentiable points, like in the case of ReLU.&lt;/p&gt;&#xA;&lt;p&gt;Properties Needed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To take advantage of Universal Approximation Theorem, and to take advantage of the modeling capacity it promises, the activation functions need to be continuous or Borel measurable (if this term is confusing just think of this as common functions), discriminatory (similarly discriminatory means the function doesn't just produce 0 all the time when integrating w.r.t. another function) and nonpolynomial (nonlinear).&lt;/li&gt;&#xA;&lt;li&gt;Also with the recent research works it is beneficial to have it monotonically increasing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Recent theoretical analysis of the activation functions gives the edge to ReLU with some preliminary theoretical guarantees like this &amp;quot;Convergence Analysis of Two-layer Neural Networks with ReLU Activation&amp;quot; &lt;a href=&quot;https://arxiv.org/abs/1705.09886&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1705.09886&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-06-25T20:22:04.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22181" PostTypeId="5" CreationDate="2020-06-25T22:41:20.093" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-25T22:41:20.093" LastActivityDate="2020-06-25T22:41:20.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22182" PostTypeId="4" CreationDate="2020-06-25T22:41:20.093" Score="0" Body="For questions related to the transition model of a Markov decision process. This term is often used in reinforcement learning (RL) to distinguish between model-based and model-free RL algorithms, where model-based algorithms use the transition model while model-free don't use it." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-25T22:41:20.093" LastActivityDate="2020-06-25T22:41:20.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22183" PostTypeId="2" ParentId="16148" CreationDate="2020-06-26T09:55:10.123" Score="0" Body="&lt;p&gt;I have been reading a lot about Natural Gradient and its use to find a descent direction. I found that &lt;a href=&quot;https://wiseodd.github.io/techblog/2018/03/14/natural-gradient/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; was the most clear.&lt;/p&gt;&#xA;&lt;p&gt;Consider a model &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; parameterized by some parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and we want to maximize the likelihood of observing our data &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; under this model: &lt;span class=&quot;math-container&quot;&gt;$p(x|\theta)$&lt;/span&gt;. To optimise this likelihood we can take steps in the the &lt;strong&gt;distribution space&lt;/strong&gt;. Updating the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; we need to measure how our likelihood changes and this is measure using the KLK divergence.&lt;/p&gt;&#xA;&lt;p&gt;Even though the KL divergence is not a &amp;quot;proper&amp;quot; distance metric as it is not symmetric, it is still quite informative about the similarity between distributions. It's practical because it can capture differences between distributions that the Euclidean metric (parameter-dependent) could not (see &lt;a href=&quot;https://wiseodd.github.io/techblog/2018/03/14/natural-gradient/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the same post&lt;/a&gt; for a simple example).&lt;/p&gt;&#xA;&lt;p&gt;So answering your question is essentially answering which is the best between Natural Gradient Descent and &amp;quot;Normal&amp;quot; Gradient Descent in the Euclidean space where your loss is measured with a L2 norm.&#xA;You can train the same model using both methods and you will just find different descent directions.&lt;/p&gt;&#xA;&lt;p&gt;Hopefully though, both will converge but in my opinion Natural Gradient descent should be superior in nature. It is just very expensive to actually compute because to find the direction in distribution space you need to compute the inverse Fisher matrix &lt;span class=&quot;math-container&quot;&gt;$F^{-1}$&lt;/span&gt; or approximate it and that's quite costly as it is of size &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the size of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; which is typically high in neural networks.&lt;/p&gt;&#xA;" OwnerUserId="38197" LastActivityDate="2020-06-26T09:55:10.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22184" PostTypeId="1" CreationDate="2020-06-26T10:12:03.393" Score="2" ViewCount="137" Body="&lt;p&gt;Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category &amp;quot;self-supervised&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;Some say, self-supervised learning algorithms learn on a set of &lt;em&gt;auxiliary tasks&lt;/em&gt; [&lt;a href=&quot;https://stats.stackexchange.com/questions/434224/how-to-differentiate-auto-encoder-techniques-from-self-supervised-learning&quot;&gt;1&lt;/a&gt;], also named &lt;em&gt;pretext task&lt;/em&gt; [&lt;a href=&quot;https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/a/10743/38174&quot;&gt;3&lt;/a&gt;], instead of the task we are interested in. Further examples are word2vec or autoencoders [&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-46672-9_10&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;] or word2vec [&lt;a href=&quot;https://hackernoon.com/self-supervised-learning-gets-us-closer-to-autonomous-learning-be77e6c86b5a&quot; rel=&quot;nofollow noreferrer&quot;&gt;5&lt;/a&gt;]. Here it is sometimes mentioned that the goal is to &amp;quot;expose the inner structure of the data&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Others do not mention that, implying that some algorithms can be called to be &amp;quot;self-supervised learning algorithms&amp;quot; if they are directly learning the task we are interested in [&lt;a href=&quot;https://ai.stackexchange.com/a/10624/38174&quot;&gt;6&lt;/a&gt;, &lt;a href=&quot;https://medium.com/@behnamsabeti/various-types-of-supervision-in-machine-learning-c7f32c190fbe&quot; rel=&quot;nofollow noreferrer&quot;&gt;7&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is the &amp;quot;auxiliary tasks&amp;quot; a requirement for a training setup to be called &amp;quot;self-supervised learning&amp;quot; or is it just optional?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Research articles mentioning the auxiliary / pretext task:&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.09005&quot; rel=&quot;nofollow noreferrer&quot;&gt;Revisiting Self-Supervised Visual Representation Learning, 2019&lt;/a&gt;, mentioned by [&lt;a href=&quot;https://ai.stackexchange.com/a/10743/38174&quot;&gt;3&lt;/a&gt;]:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The self-supervised learning framework requires only unlabeled data in order to formulate a pretext learning task such as predicting context or image rotation, for which a target objective can be computed without supervision.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1803.07728&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised Representation Learning by Predicting Image Rotations,  ICLR, 2018&lt;/a&gt;, mentioned by&#xA;[&lt;a href=&quot;https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a prominent paradigm is the so-called self-supervised learning that defines an annotation free pretext task, using only the visual information present on the images or videos, in order to provide a surrogatesupervision signal for feature learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1505.05192&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unsupervised Visual Representation Learning by Context Prediction, 2016&lt;/a&gt;, mentioned by&#xA;[&lt;a href=&quot;https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This converts an apparently unsupervised problem (finding a good similarity metric between words) intoa “self-supervised” one:  learning a function from a givenword to the words surrounding it.  Here the context predic-tion  task  is  just  a  “pretext”  to  force  the  model  to  learn  agood word embedding, which, in turn, has been shown tobe useful in a number of real tasks, such as semantic wordsimilarity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1905.01235&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scaling and Benchmarking Self-Supervised Visual Representation Learning, 2019&lt;/a&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In discriminative self-supervised learning, which is the main focus of this work, a model is trained on an auxiliary or ‘pretext’ task for which ground-truth is available for free. In most cases, the pretext task involves predicting some hidden portion of the data (for example, predicting color for gray-scale images&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="38174" LastEditorUserId="2444" LastEditDate="2020-11-20T17:22:18.757" LastActivityDate="2021-04-20T22:57:02.220" Title="Does self-supervised learning require auxiliary tasks?" Tags="&lt;definitions&gt;&lt;transfer-learning&gt;&lt;feature-learning&gt;&lt;self-supervised-learning&gt;&lt;pretext-tasks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22186" PostTypeId="1" CreationDate="2020-06-26T11:52:51.607" Score="1" ViewCount="139" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_supervision&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weak supervision&lt;/a&gt; is supervised learning, with uncertainty in the labeling, e.g. due to automatic labeling or because non-experts labelled the data [&lt;a href=&quot;http://ai.stanford.edu/blog/weak-supervision/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;p&gt;Distant supervision [&lt;a href=&quot;http://deepdive.stanford.edu/distant_supervision&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://stats.stackexchange.com/a/47036/83252&quot;&gt;3&lt;/a&gt;] is a type of weak supervision that uses an auxiliary automatic mechanism to produce weak labels / reference output (in contrast to non-expert human labelers).&lt;/p&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://ai.stackexchange.com/a/10624/38174&quot;&gt;this answer&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Self-supervised learning (or self-supervision) is a supervised learning technique where the training data is automatically labelled.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the examples for self-supervised learning, I have seen so far, the labels were extracted from the input data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What is the difference between distant supervision and self-supervision?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it that for self-supervision, the labels must come from the input data and for distant supervision it &lt;em&gt;can&lt;/em&gt; come from anywhere (which would make self-supervision a type of distant supervision)?&lt;/li&gt;&#xA;&lt;li&gt;Or &lt;em&gt;must&lt;/em&gt; the labels from distant supervision come from somewhere else than the input data?&lt;/li&gt;&#xA;&lt;li&gt;If &lt;a href=&quot;https://ai.stackexchange.com/a/10624/38174&quot;&gt;&amp;quot;In robotics, this can be done by finding and exploiting the relations or correlations between inputs coming from different sensor modalities.&amp;quot;&lt;/a&gt; then for self-supervised learning, the labels do not even have to originate from the input data. (Or did I misinterpret the quote?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;(Setup mentioned in discussion:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GOcRN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GOcRN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38174" LastEditorUserId="38174" LastEditDate="2020-07-04T15:07:54.253" LastActivityDate="2020-07-04T15:07:54.253" Title="What is the difference between distant supervision and self-supervision?" Tags="&lt;comparison&gt;&lt;supervised-learning&gt;&lt;self-supervised-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22188" PostTypeId="1" AcceptedAnswerId="22191" CreationDate="2020-06-26T12:28:07.770" Score="1" ViewCount="58" Body="&lt;p&gt;I know we keep the target network constant during training to improve stability, but why exactly are we updating the weights of our target network? In particular, if we've already reached convergence, why exactly are we updating the weights of our target network?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-26T12:59:04.313" LastActivityDate="2020-06-26T14:08:48.767" Title="Why do we update the weights of the target network in deep Q learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22189" PostTypeId="1" CreationDate="2020-06-26T13:42:51.143" Score="0" ViewCount="45" Body="&lt;p&gt;Let's say we have a &amp;quot;monkey and banana&amp;quot; problem. Let's say we have 3 places: A, B and C. We also have the operator &lt;code&gt;move(X, Y)&lt;/code&gt;, which moves the monkey from &lt;code&gt;X&lt;/code&gt; to &lt;code&gt;Y&lt;/code&gt;, where &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;Y&lt;/code&gt; are variables. There is also the state &lt;code&gt;At(Y)&lt;/code&gt; (the monkey is at &lt;code&gt;Y&lt;/code&gt;) in his add list.&lt;/p&gt;&#xA;&lt;p&gt;Now, let's say that, on top of the stack, we have some unsatisfied goal such as &lt;code&gt;At(B)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;According to the STRIPS algorithm, must the operator that I choose be of the form &lt;code&gt;move(X, B)&lt;/code&gt;, or it doesn't matter which places from &lt;code&gt;A&lt;/code&gt;, &lt;code&gt;B&lt;/code&gt;, &lt;code&gt;C&lt;/code&gt; do I put in &lt;code&gt;X&lt;/code&gt; and in &lt;code&gt;Y&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="38202" LastEditorUserId="2444" LastEditDate="2020-06-28T11:44:50.547" LastActivityDate="2020-06-28T11:44:50.547" Title="choose operator in STRIPS algorithm" Tags="&lt;planning&gt;&lt;strips&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22190" PostTypeId="1" CreationDate="2020-06-26T13:57:40.870" Score="0" ViewCount="19" Body="&lt;p&gt;It looks like the music which is sampled and each sample has been transferred into frequency domain with FFT is in fact 2D object, capable to be dealed with CNN networks. So, it looks like if some pretrained network like Image net will be trained for this kind of 2D music data, it will be possible to try a style transfer from one music file to another.&lt;/p&gt;&#xA;&lt;p&gt;So, is there such kind of 2D models zoo?&lt;/p&gt;&#xA;" OwnerUserId="33491" LastActivityDate="2020-06-26T13:57:40.870" Title="Is there a 2D (sampled frequencies) music network in some kind of zoos like Imagenet to try it in order to get style transfer?" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22191" PostTypeId="2" ParentId="22188" CreationDate="2020-06-26T14:08:48.767" Score="2" Body="&lt;p&gt;If you are certain that you reached convergence then there is no point in continuing to train your agent, because of that there is also no point in discussing why is target network being updated after convergence is reached. You should simply stop training if you converged. During training we obviously need to keep updating target network to improve correctness of Q-value estimates.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-06-26T14:08:48.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22193" PostTypeId="2" ParentId="4748" CreationDate="2020-06-26T15:49:13.163" Score="0" Body="&lt;p&gt;You could vary an error coefficient in training. For example, if the expected output was negative and it gave some value in the positive you can train on C * ERROR and conversely if the expected output was positive and it gave some value in the negative you can train on just the error, that way false positives have more impact on the model as opposed to false negatives.&lt;/p&gt;&#xA;&lt;p&gt;Varying learning rates could as well help, however, increasing the learning rate and increase the error have different effects because changing the error will change the direction of the gradient whereas changing the learning rate will only change the gradient's magnitude of effect on the network, two slightly different things.&lt;/p&gt;&#xA;&lt;p&gt;(for the learning rate, split the data into two, positive and negative, then train them separately with the learning rate for negative cases larger than for positive cases)&lt;/p&gt;&#xA;" OwnerUserId="38204" LastActivityDate="2020-06-26T15:49:13.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22194" PostTypeId="1" CreationDate="2020-06-26T16:58:12.707" Score="3" ViewCount="105" Body="&lt;p&gt;During the learning phase, why don't we have a 100% exploration rate, to allow our agent to fully explore our environment and update the Q values, then during testing we bring in exploitation? Does that make more sense than decaying the exploration rate?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-26T23:23:08.740" LastActivityDate="2020-06-27T15:27:44.540" Title="Why is it not advisable to have a 100 percent exploration rate?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="2" CommentCount="0" ClosedDate="2020-06-29T19:47:43.617" ContentLicense="CC BY-SA 4.0" />
  <row Id="22195" PostTypeId="2" ParentId="22194" CreationDate="2020-06-26T17:03:08.920" Score="2" Body="&lt;p&gt;No - imagine if you were playing an Atari game and took completely random actions. Your games would not last very long and you would never get to experience all of the state space because the game would end too soon. This is why you need to combine exploration and exploitation to fully explore the state space.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-06-26T18:11:19.807" LastActivityDate="2020-06-26T18:11:19.807" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="22196" PostTypeId="1" CreationDate="2020-06-26T18:27:22.083" Score="1" ViewCount="32" Body="&lt;p&gt;Let's say I have two databases, &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x_i}, \mathbf{\hat{p_i}})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{x_j}, \mathbf{\hat{q_j}})$&lt;/span&gt;. A neural network with weights &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; can receive an input &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; and produce an output &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;. Mathematically, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} = f_{NN}(\mathbf{x},\theta)$&lt;/span&gt;. To compare the output of my neural network and the database, I need two wrappers, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}=g(\mathbf{y})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{q}=h(\mathbf{y})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The problem is: only &lt;span class=&quot;math-container&quot;&gt;$g(\cdot)$&lt;/span&gt; is differentiable while writing &lt;span class=&quot;math-container&quot;&gt;$h(\cdot)$&lt;/span&gt; in a differentiable manner would take a huge effort.&lt;/p&gt;&#xA;&lt;p&gt;Is there any efficient way to train my neural network to minimize the following loss function?&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{L}(\theta) = \sum_i \left\{g\left[f_{NN}(\mathbf{x_i}, \theta)\right] - \mathbf{\hat{p_i}}\right\}^2 + \sum_j \left\{h\left[f_{NN}(\mathbf{x_j}, \theta)\right] - \mathbf{\hat{q_j}}\right\}^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My thinking&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I am using gradient descent-type algorithm, I can only optimize the first part of the loss function while ignoring the second part.&#xA;If I am using evolutionary-type algorithm, I can optimize both parts, but it will take a long time and I don't make a full use of the differentiable property of &lt;span class=&quot;math-container&quot;&gt;$g(\cdot)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="38206" LastActivityDate="2020-06-26T18:27:22.083" Title="Algorithm to train a neural network against differentiable and non-differentiable databases?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22197" PostTypeId="2" ParentId="22064" CreationDate="2020-06-26T18:55:18.503" Score="1" Body="&lt;p&gt;Do you have to use Boltzmann exploration, strictly? There is a modification for Boltzmann exploration called Mellow-max. It, basically, provides an adaptive temperature for Boltzmann exploration.&lt;/p&gt;&#xA;&lt;p&gt;Here is the link for the paper for tuning mellow-max with deep reinforcement learning (DQN is often mentioned): &lt;a href=&quot;http://cs.brown.edu/people/gdk/pubs/tuning_mellowmax_drlw.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs.brown.edu/people/gdk/pubs/tuning_mellowmax_drlw.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the link for mellow-max implemented with SARSA (I recommend reading this first, to get an understanding of mellow-max): &lt;a href=&quot;https://arxiv.org/pdf/1612.05628.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1612.05628.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30174" LastActivityDate="2020-06-26T18:55:18.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22199" PostTypeId="1" AcceptedAnswerId="22203" CreationDate="2020-06-26T22:07:29.993" Score="2" ViewCount="54" Body="&lt;p&gt;Why isn't it wise for us to completely erase our old Q value and replace it with the calculated Q value? Why can't we forget the learning rate and temporal difference?&lt;/p&gt;&#xA;&lt;p&gt;Here's the update formula.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HcnEI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HcnEI.png&quot; alt=&quot;The formula&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-26T23:43:00.310" LastActivityDate="2020-06-27T04:42:42.193" Title="Why isn't it wise for us to completely erase our old Q value and replace it with the calculated Q value?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;temporal-difference-methods&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22200" PostTypeId="2" ParentId="22177" CreationDate="2020-06-26T22:24:15.380" Score="0" Body="&lt;p&gt;Yes , a transition model shows that our environment is stochastic in nature, and with that model we know the probability of entering a state when an action is taken.&lt;/p&gt;&#xA;" OwnerUserId="37831" LastActivityDate="2020-06-26T22:24:15.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22201" PostTypeId="1" AcceptedAnswerId="22273" CreationDate="2020-06-26T22:37:46.543" Score="2" ViewCount="83" Body="&lt;p&gt;q learning is defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sMDdm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sMDdm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is my implementation of q learning of the tic tac toe problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import timeit&#xA;from operator import attrgetter&#xA;import time&#xA;import matplotlib.pyplot&#xA;import pylab&#xA;from collections import Counter&#xA;import logging.handlers&#xA;import sys&#xA;import configparser&#xA;import logging.handlers&#xA;import unittest&#xA;import json, hmac, hashlib, time, requests, base64&#xA;from requests.auth import AuthBase&#xA;from pandas.io.json import json_normalize&#xA;from multiprocessing.dummy import Pool as ThreadPool&#xA;import threading&#xA;import time&#xA;from statistics import mean &#xA;import statistics as st&#xA;import os   &#xA;from collections import Counter&#xA;import matplotlib.pyplot as plt&#xA;from sklearn import preprocessing&#xA;from datetime import datetime&#xA;import datetime&#xA;from datetime import datetime, timedelta&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib.ticker as ticker&#xA;import matplotlib&#xA;import numpy as np&#xA;import pandas as pd&#xA;from functools import reduce&#xA;from ast import literal_eval&#xA;import unittest&#xA;import math&#xA;from datetime import date, timedelta&#xA;import random&#xA;&#xA;today = datetime.today()&#xA;model_execution_start_time = str(today.year)+&amp;quot;-&amp;quot;+str(today.month)+&amp;quot;-&amp;quot;+str(today.day)+&amp;quot; &amp;quot;+str(today.hour)+&amp;quot;:&amp;quot;+str(today.minute)+&amp;quot;:&amp;quot;+str(today.second)&#xA;&#xA;epsilon = .1&#xA;discount = .1&#xA;step_size = .1&#xA;number_episodes = 30000&#xA;&#xA;def epsilon_greedy(epsilon, state, q_table) : &#xA;    &#xA;    def get_valid_index(state):&#xA;        i = 0&#xA;        valid_index = []&#xA;        for a in state :          &#xA;            if a == '-' :&#xA;                valid_index.append(i)&#xA;            i = i + 1&#xA;        return valid_index&#xA;    &#xA;    def get_arg_max_sub(values , indices) : &#xA;        return max(list(zip(np.array(values)[indices],indices)),key=lambda item:item[0])[1]&#xA;    &#xA;    if np.random.rand() &amp;lt; epsilon:&#xA;        return random.choice(get_valid_index(state))&#xA;    else :&#xA;        if state not in q_table : &#xA;            q_table[state] = np.array([0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0])&#xA;        q_row = q_table[state]&#xA;        return get_arg_max_sub(q_row , get_valid_index(state))&#xA;    &#xA;def make_move(current_player, current_state , action):&#xA;    if current_player == 'X':&#xA;        return current_state[:action] + 'X' + current_state[action+1:]&#xA;    else : &#xA;        return current_state[:action] + 'O' + current_state[action+1:]&#xA;&#xA;q_table = {}&#xA;max_steps = 9&#xA;&#xA;def get_other_player(p):&#xA;    if p == 'X':&#xA;        return 'O'&#xA;    else : &#xA;        return 'X'&#xA;    &#xA;def win_by_diagonal(mark , board):&#xA;    return (board[0] == mark and board[4] == mark and board[8] == mark) or (board[2] == mark and board[4] == mark and board[6] == mark)&#xA;    &#xA;def win_by_vertical(mark , board):&#xA;    return (board[0] == mark and board[3] == mark and board[6] == mark) or (board[1] == mark and board[4] == mark and board[7] == mark) or (board[2] == mark and board[5] == mark and board[8]== mark)&#xA;&#xA;def win_by_horizontal(mark , board):&#xA;    return (board[0] == mark and board[1] == mark and board[2] == mark) or (board[3] == mark and board[4] == mark and board[5] == mark) or (board[6] == mark and board[7] == mark and board[8] == mark)&#xA;&#xA;def win(mark , board):&#xA;    return win_by_diagonal(mark, board) or win_by_vertical(mark, board) or win_by_horizontal(mark, board)&#xA;&#xA;def draw(board):&#xA;    return win('X' , list(board)) == False and win('O' , list(board)) == False and (list(board).count('-') == 0)&#xA;&#xA;s = []&#xA;rewards = []&#xA;def get_reward(state):&#xA;    reward = 0&#xA;    if win('X' ,list(state)):&#xA;        reward = 1&#xA;        rewards.append(reward)&#xA;    elif draw(state) :&#xA;        reward = -1&#xA;        rewards.append(reward)&#xA;    else :&#xA;        reward = 0&#xA;        rewards.append(reward)&#xA;        &#xA;    return reward&#xA;&#xA;def get_done(state):&#xA;    return win('X' ,list(state)) or win('O' , list(state)) or draw(list(state)) or (state.count('-') == 0)&#xA;    &#xA;reward_per_episode = []&#xA;            &#xA;reward = []&#xA;def q_learning():&#xA;    for episode in range(0 , number_episodes) :&#xA;        t = 0&#xA;        state = '---------'&#xA;&#xA;        player = 'X'&#xA;        random_player = 'O'&#xA;&#xA;&#xA;        if episode % 1000 == 0:&#xA;            print('in episode:',episode)&#xA;&#xA;        done = False&#xA;        episode_reward = 0&#xA;            &#xA;        while t &amp;lt; max_steps:&#xA;&#xA;            t = t + 1&#xA;&#xA;            action = epsilon_greedy(epsilon , state , q_table)&#xA;&#xA;            done = get_done(state)&#xA;&#xA;            if done == True : &#xA;                break&#xA;&#xA;            if state not in q_table : &#xA;                q_table[state] = np.array([0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0])&#xA;&#xA;            next_state = make_move(player , state , action)&#xA;            reward = get_reward(next_state)&#xA;            episode_reward = episode_reward + reward&#xA;            &#xA;            done = get_done(next_state)&#xA;&#xA;            if done == True :&#xA;                q_table[state][action] = q_table[state][action] + (step_size * (reward - q_table[state][action]))&#xA;                break&#xA;&#xA;            next_action = epsilon_greedy(epsilon , next_state , q_table)&#xA;            if next_state not in q_table : &#xA;                q_table[next_state] = np.array([0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0])&#xA;&#xA;            q_table[state][action] = q_table[state][action] + (step_size * (reward + (discount * np.max(q_table[next_state]) - q_table[state][action])))&#xA;&#xA;            state = next_state&#xA;&#xA;            player = get_other_player(player)&#xA;            &#xA;        reward_per_episode.append(episode_reward)&#xA;&#xA;q_learning()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The alogrithm player is assigned to 'X' while the other player is 'O':&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    player = 'X'&#xA;    random_player = 'O'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The reward per episode:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plt.grid()&#xA;plt.plot([sum(i) for i in np.array_split(reward_per_episode, 15)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;renders:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/p3Jyj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p3Jyj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Playing the model against an opponent making random moves:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Computer opponent that makes random moves against trained RL computer opponent&#xA;# Random takes move for player marking O position&#xA;# RL agent takes move for player marking X position&#xA;&#xA;def draw(board):&#xA;    return win('X' , list(board)) == False and win('O' , list(board)) == False and (list(board).count('-') == 0)&#xA;&#xA;x_win = []&#xA;o_win = []&#xA;draw_games = []&#xA;number_games = 50000&#xA;&#xA;c = []&#xA;o = []&#xA;&#xA;for ii in range (0 , number_games):&#xA;    &#xA;    if ii % 10000 == 0 and ii &amp;gt; 0:&#xA;        print('In game ',ii)&#xA;        print('The number of X game wins' , sum(x_win))&#xA;        print('The number of O game wins' , sum(o_win))&#xA;        print('The number of drawn games' , sum(draw_games))&#xA;&#xA;    available_moves = [0,1,2,3,4,5,6,7,8]&#xA;    current_game_state = '---------'&#xA;    &#xA;    computer = ''&#xA;    random_player = ''&#xA;    &#xA;    computer = 'X'&#xA;    random_player = 'O'&#xA;&#xA;    def draw(board):&#xA;        return win('X' , list(board)) == False and win('O' , list(board)) == False and (list(board).count('-') == 0)&#xA;        &#xA;    number_moves = 0&#xA;    &#xA;    for i in range(0 , 5):&#xA;&#xA;        randomer_move = random.choice(available_moves)&#xA;        number_moves = number_moves + 1&#xA;        current_game_state = current_game_state[:randomer_move] + random_player + current_game_state[randomer_move+1:]&#xA;        available_moves.remove(randomer_move)&#xA;&#xA;        if number_moves == 9 : &#xA;            draw_games.append(1)&#xA;            break&#xA;        if win('O' , list(current_game_state)) == True:&#xA;            o_win.append(1)&#xA;            break&#xA;        elif win('X' , list(current_game_state)) == True:&#xA;            x_win.append(1)&#xA;            break&#xA;        elif draw(current_game_state) == True:&#xA;            draw_games.append(1)&#xA;            break&#xA;            &#xA;        computer_move_pos = epsilon_greedy(-1, current_game_state, q_table)&#xA;        number_moves = number_moves + 1&#xA;        current_game_state = current_game_state[:computer_move_pos] + computer + current_game_state[computer_move_pos+1:]&#xA;        available_moves.remove(computer_move_pos)&#xA;     &#xA;        if number_moves == 9 : &#xA;            draw_games.append(1)&#xA;#             print(current_game_state)&#xA;            break&#xA;            &#xA;        if win('O' , list(current_game_state)) == True:&#xA;            o_win.append(1)&#xA;            break&#xA;        elif win('X' , list(current_game_state)) == True:&#xA;            x_win.append(1)&#xA;            break&#xA;        elif draw(current_game_state) == True:&#xA;            draw_games.append(1)&#xA;            break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;outputs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;In game  10000&#xA;The number of X game wins 4429&#xA;The number of O game wins 3006&#xA;The number of drawn games 2565&#xA;In game  20000&#xA;The number of X game wins 8862&#xA;The number of O game wins 5974&#xA;The number of drawn games 5164&#xA;In game  30000&#xA;The number of X game wins 13268&#xA;The number of O game wins 8984&#xA;The number of drawn games 7748&#xA;In game  40000&#xA;The number of X game wins 17681&#xA;The number of O game wins 12000&#xA;The number of drawn games 10319&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The reward per episode graph suggests the algorithm has converged? If the model has converged shouldnt the number of O game wins be zero ?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastActivityDate="2020-06-30T18:57:13.583" Title="q learning appears to converge but does not always win against random tic tac toe player" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22202" PostTypeId="2" ParentId="13460" CreationDate="2020-06-26T23:01:09.537" Score="1" Body="&lt;p&gt;In deep learning (and, in general, machine learning), tensors are multi-dimensional arrays. You can perform some operations on these multi-dimensional arrays (depending also on the specific implementations and libraries). These operations are similar to the operations you can apply to vectors or matrices, which are just specific examples of multi-dimensional arrays. Examples of these operations are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;indexing and slicing (if you are familiar with Python, these terms should not scare you)&lt;/li&gt;&#xA;&lt;li&gt;algebraic operations (such as multiplication of a tensor with another tensor, which includes numbers, vectors or matrices), which typically support &lt;a href=&quot;https://numpy.org/doc/stable/user/theory.broadcasting.html#array-broadcasting-in-numpy&quot; rel=&quot;nofollow noreferrer&quot;&gt;broadcasting&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;reshaping (i.e. change the shape of the tensor)&lt;/li&gt;&#xA;&lt;li&gt;conversion to or from another format (e.g. a string)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;TensorFlow provides &lt;a href=&quot;https://www.tensorflow.org/guide/tensor&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article&lt;/a&gt; that discusses these tensors, so I suggest that you read it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mathworld.wolfram.com/Tensor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;In mathematics&lt;/a&gt;, tensors are not just multi-dimensional arrays. They are multi-dimensional arrays that need to satisfy certain properties (in the same way that matrices need to satisfy certain properties to be called matrices) and are equipped with certain operations.&lt;/p&gt;&#xA;&lt;p&gt;The paper &lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;amp;arnumber=8884203&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey on Tensor Techniques and Applications in Machine Learning&lt;/a&gt; (2019) Yuwang Ji et al., published by IEEE, provides a comprehensive overview of tensors in mathematics. It is full of diagrams that illustrate the concepts and the explanations are concise. Some of the explanations in this paper may not be very useful to develop deep learning applications, but some of the illustrations and explanations (especially, in the first pages, which are the only ones that I read) will give you some intuition behind the tensors or multi-dimensional arrays used in deep learning).&lt;/p&gt;&#xA;&lt;p&gt;So, tensors in deep learning (DL) may not be exactly equivalent to the tensor objects in mathematics, because they may not satisfy all the required properties or some of the operations in the specific libraries may not be implemented, but it is fine to call them tensors because a tensor in mathematics is also a multi-dimensional array, to which you can apply operations (some of them are implemented in the DL libraries).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-26T23:14:52.693" LastActivityDate="2020-06-26T23:14:52.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22203" PostTypeId="2" ParentId="22199" CreationDate="2020-06-27T04:42:42.193" Score="3" Body="&lt;p&gt;Removing the learning rate will likely yield &lt;strong&gt;poor convergence&lt;/strong&gt; to the optimal policy and optimal Q-values. Note that the current policy is completely dependent on the Q-values, as we take the action with highest Q-value in a given state (with a few other considerations such as exploration, etc.). If we were to remove the learning rate, then we are making a relatively large change to our Q-values and possibly to our policy as well after only a single update. For example, if the sample rewards have great variance (e.g. in stochastic environments), then drastic updates to a single Q-value may occur simply by chance when a learning rate is not used. Due to the recursive definition of Q-values, a few poor updates can undo the work of many previous updates. If this phenomenon were to occur frequently, then the policy may take a long time to converge to the optimal policy, if at all.&lt;/p&gt;&#xA;&lt;p&gt;Underlying the temporal-difference update and many other reinforcement learning updates is the notion of policy iteration in which the estimated value function is updated to match the true value function of the current policy and the current policy is updated to be greedy with respect to the estimated value function. This process proceeds iteratively and &lt;strong&gt;gradually&lt;/strong&gt; until convergence to the optimal policy and optimal value function is achieved. Gradual changes such as setting a small learning rate (e.g. &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.1$&lt;/span&gt;) aim to speed up convergence by lessening the frequency of the phenomenon in the above paragraph. Sutton and Barto make comments on convergence throughout their &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;, with the remarks surrounding line 2.7 in Section 2.5 providing a summary.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-27T04:42:42.193" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22205" PostTypeId="1" CreationDate="2020-06-27T06:46:01.497" Score="3" ViewCount="31" Body="&lt;p&gt;The average return for trajectories, &lt;span class=&quot;math-container&quot;&gt;$V^{\pi_e}$&lt;/span&gt;(s) is often computed via the importance sampling estimate &lt;span class=&quot;math-container&quot;&gt;$$V^{\pi_e}(s) = \frac{1}{n}\sum_{i=1}^n\prod_{t=0}^{H}\frac{\pi_e(a_t | s_t)}{\pi_b(a_t|s_t)}G_i$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$G_i$&lt;/span&gt;is the reward observed for the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th trajectory. &lt;a href=&quot;https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barton gives an example whereby the variance could be infinite&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In general, however, why does this estimator suffer from high variance? Is it because &lt;span class=&quot;math-container&quot;&gt;$\pi_e(a_t|s_t)$&lt;/span&gt; is mainly deterministic and, therefore, the importance weight is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for most trajectories, rendering those sample trajectories useless?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-11-05T22:10:42.017" LastActivityDate="2020-11-05T22:10:42.017" Title="Why is it the case that off-policy evaluation using importance sampling suffers from high variance?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22207" PostTypeId="1" CreationDate="2020-06-27T08:30:42.513" Score="1" ViewCount="22" Body="&lt;p&gt;I stumbled upon a paper from P.Diehl and M.Cook with the title &amp;quot;Unsupervised learning of digit recognition using spike-timing-dependent plasticity&amp;quot; and I'm trying to understand the logic behind the network connection they made.&lt;/p&gt;&#xA;&lt;p&gt;The network is as follows. The inputs (size of an image 28x28) are connected to the usual all-to-all fashion with positive weights to an NxN layer of neurons. They are decoded using poisson random distribution in which the frequency of spikes of a pixel is set accordingly to the pixel value. The NxN layer is connected 1 on 1 to an NxN layer of inhibitory neurons. These neurons inhibit all other neurons except the one they are connected to. Thus they are connected all-to-all with an exception.&lt;/p&gt;&#xA;&lt;p&gt;According to the paper this provides competition among neurons. I cannot understand how, in this particular connection, competition is provided. How can different neurons inherit different properties? To me it seems that all neurons will inherit the same properties, thus no differences in weights will be made in the training session among all neurons. For example, if the input 5 is passed to the network all weights of all neurons will try to adjust according to 5. Then if input 7 is passed next, all the weights will be updated according to the new number (7). It is expected, though, that some weights will keep the previous adjustment ie that some weights will have the properties of 5 and the others the properties of 7.&lt;/p&gt;&#xA;" OwnerUserId="31817" LastEditorUserId="31817" LastEditDate="2020-06-28T06:05:08.437" LastActivityDate="2020-06-28T06:05:08.437" Title="How does Lateral Inhibition Provide Competition among Neurons?" Tags="&lt;spiking-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22208" PostTypeId="1" CreationDate="2020-06-27T13:00:37.823" Score="1" ViewCount="33" Body="&lt;p&gt;The following quote is taken from the beginning of the chapter on &amp;quot;Approximate Solution Methods&amp;quot; (p. 198) in &amp;quot;Reinforcement Learning&amp;quot; by Sutton &amp;amp; Barto (2018):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;reinforcement learning generally requires function approximation methods able to handle nonstationary target functions (target functions that change over time). In control methods based on GPI (generalized policy iteration) we often seek to learn &lt;span class=&quot;math-container&quot;&gt;$q_\pi$&lt;/span&gt; while &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; changes. Even if the policy [pi] remains the same, the target values of training examples are nonstationary if they are generated by bootstrapping methods (DP and TD learning).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Could someone explain why the same is not the case if we use non-bootstrapping methods (such as Monte Carlo that is not allowed infinite rollouts)?&lt;/p&gt;&#xA;" OwnerUserId="29670" LastEditorUserId="2444" LastEditDate="2020-06-27T13:54:43.417" LastActivityDate="2020-06-27T13:54:43.417" Title="Why do bootstrapping methods produce nonstationary targets more than non-bootstrapping methods?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;stationary-policy&gt;&lt;bootstrapping&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22209" PostTypeId="5" CreationDate="2020-06-27T14:05:24.200" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-27T14:05:24.200" LastActivityDate="2020-06-27T14:05:24.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22210" PostTypeId="4" CreationDate="2020-06-27T14:05:24.200" Score="0" Body="For questions related to the concept of bootstrapping in reinforcement learning (and statistics)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-27T14:05:24.200" LastActivityDate="2020-06-27T14:05:24.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22212" PostTypeId="2" ParentId="22194" CreationDate="2020-06-27T15:01:53.390" Score="0" Body="&lt;p&gt;While &lt;em&gt;theoretically&lt;/em&gt; you can do something like this if you're very confident you'll cover most of the state space in exploration, this is still a suboptimal strategy. Even in the case of multi-armed bandits, this strategy can be much less sample efficient than &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, and exploration is much easier in this case.&lt;/p&gt;&#xA;&lt;p&gt;So, even if your strategy miraculously works on a decently sized MDP, it'll be worse than combining exploration and exploitation.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastEditorUserId="36821" LastEditDate="2020-06-27T15:27:44.540" LastActivityDate="2020-06-27T15:27:44.540" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22213" PostTypeId="2" ParentId="16610" CreationDate="2020-06-27T16:58:20.107" Score="-2" Body="&lt;p&gt;The book sets this hypothesis up by laying out a few assumptions:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In reinforcement learning, the purpose or goal of the agent is formulated in terms of a special signal called the reward, passing from the environment to the agent. At each time step, the reward is a simple number.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We could think about what counterexamples to those assumptions might be:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The reward signal originates internally, instead of originating from the environment. (e.g. meditation, or abstract introspection)&lt;/li&gt;&#xA;&lt;li&gt;The signal is not received every time step, or isn't necessarily expected to be received at all. (e.g. seeking of transcendent experiences)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What might be common for these counterexamples is that the reinforcement learning mechanism itself undergoes spontaneous change. A signal that would have been positive before the spontaneous change might now be negative. The reward landscape itself might be completely different. From the agent's perspective, it might be impossible to evaluate what changed. The agent might have a 'subconscious' secondary algorithm that introduces changes in the learning algorithm itself, in a way that's decoupled from any reward-defined behavior.&lt;/p&gt;&#xA;" OwnerUserId="38221" LastEditorUserId="2444" LastEditDate="2020-07-01T16:34:08.057" LastActivityDate="2020-07-01T16:34:08.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22214" PostTypeId="1" AcceptedAnswerId="22216" CreationDate="2020-06-27T19:02:10.683" Score="2" ViewCount="89" Body="&lt;p&gt;I'm implementing DQN algorithm to train my agent to play a turn-based game. The action space for the game is small, but not all moves are available at all the states. Therefore, when deciding on which action to pick, agent sets Q-values to 0 for all the illegal moves while normalizing the values of the rest.&lt;/p&gt;&#xA;&lt;p&gt;During training, when the agent is calculating the loss between policy and target networks, should the illegal actions be ignored (set to 0) so that they don't affect the calculations?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastActivityDate="2020-06-27T22:36:44.717" Title="Should illegal moves be excluded from loss calculation in DQN algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22215" PostTypeId="2" ParentId="2231" CreationDate="2020-06-27T19:21:31.247" Score="1" Body="&lt;p&gt;Rules based on Gestalt psychology could be seen as a 'local minima' in terms of optimal image processing. Some of them could be surprisingly effective, but difficult to extend and improve upon, since they assume certain high level attributes that might not generalize well.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;If it's circular, then it's a fruit 90% of the time&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Modern methods like neural networks come at the problem from another direction, and try to build towards those high level attributes incrementally. You could say that a goal of these modern methods is to recreate a version of those 'assumptions' or 'Gestalt rules' empirically. That way, we can feel more confident that we're not using a locally optimal approach.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;This is the smallest algorithm that will accurately identify fruit 90% of the time&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Are such methods being used or worked on today?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yes, and not just in image processing. If you look at any industry where fast processing of ambiguous domain-specific data is needed, there are usually systems that were designed by experts in that field using 'hand-crafted' heuristics. A benefit of these approaches is that the 'Gestalt rules' are understandable apart from the system that uses it, and is therefore easier to trust.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Was any progress made on this? Or was this research program dropped?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Aside from trivial cases, employing the gestalt approach in a particular domain requires expertise. &amp;quot;Image Expert&amp;quot; is an expensive and hard to fill role. Even then, you're bounded by that expert's ability to effectively generalize an extremely complex problem. Over the past decades, machine learning has become a lot easier and less expensive to employ, and has been slowly replacing all those 'hand-crafted' approaches.&lt;/p&gt;&#xA;&lt;p&gt;However, one of the interesting things we're finding when we let the machine learning approaches loose is that they sometimes don't agree with our previous rules. Why they don't agree, and what this means about automation and our psychological biases, is one of the big open questions in AI.&lt;/p&gt;&#xA;&lt;p&gt;You might also be interested in Saliency Maps, and similar techniques intended to visualize the general rules used by neural networks: &lt;a href=&quot;https://en.wikipedia.org/wiki/Saliency_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Saliency_map&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://wayback.archive-it.org/219/20121229060421/http://www.cs.indiana.edu/%7Ejwmills/ANALOG.NOTEBOOK/klm/klm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wayback.archive-it.org/219/20121229060421/http://www.cs.indiana.edu/~jwmills/ANALOG.NOTEBOOK/klm/klm.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.sagepub.com/doi/pdf/10.1177/002029400403701001&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.sagepub.com/doi/pdf/10.1177/002029400403701001&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://maritime.org/doc/op1140/index.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://maritime.org/doc/op1140/index.htm&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://airandspace.si.edu/stories/editorial/inventing-apollo-spaceflight-biomedical-sensors&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://airandspace.si.edu/stories/editorial/inventing-apollo-spaceflight-biomedical-sensors&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38221" LastEditorUserId="38221" LastEditDate="2020-07-01T13:53:58.750" LastActivityDate="2020-07-01T13:53:58.750" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22216" PostTypeId="2" ParentId="22214" CreationDate="2020-06-27T22:36:44.717" Score="3" Body="&lt;p&gt;I've implemented this exact scenario before; your approach would most likely be successful, but I think it could be simplified.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Therefore, when deciding on which action to pick, agent sets Q-values to 0 for all the illegal moves while normalizing the values of the rest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In DQN, the Q-values are used to find the best action. To determine the best action in a given state, it suffices to look at the Q-values of all valid actions and then take the valid action with highest Q-value. Setting Q-values of invalid actions to 0 is unnecessary once you have a list of valid actions. Note that you would need that set of valid actions to set invalid Q-values to 0 in the first place, so the approach I'm suggesting is more concise without worsening the performance.&lt;/p&gt;&#xA;&lt;p&gt;Since the relative order of the Q-values is all that is required to find the best action, there is no need for normalization. Also, the original DQN &lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; uses &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy exploration. Keep in mind to only sample from valid actions in a given state when exploring this way.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;During training, when the agent is calculating the loss between policy and target networks, should the illegal actions be ignored (set to 0) so that they don't affect the calculations?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As noted in one of your previous &lt;a href=&quot;https://ai.stackexchange.com/q/22130/37607&quot;&gt;questions&lt;/a&gt;, we train on tuples of experiences &lt;span class=&quot;math-container&quot;&gt;$(s, a, r, s')$&lt;/span&gt;. The definition of the Q-learning update is as follows (taken from line 6.8 of &lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(S_t, A_t) \leftarrow Q(S_t, A_t) + \alpha \left[R_{t+1} + \gamma\max\limits_aQ(S_{t+1}, a) - Q(S_t, A_t)\right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The update requires taking a maximum over all valid actions in &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. Again, setting invalid Q-values to 0 is unnecessary extra work once you know the set of valid actions. Ignoring invalid actions is equivalent to leaving those actions out of the set of valid actions.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-06-27T22:36:44.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22217" PostTypeId="1" CreationDate="2020-06-27T23:37:30.840" Score="0" ViewCount="34" Body="&lt;p&gt;I am trying to duplicate and learn from example given on this &lt;a href=&quot;https://www.kaggle.com/basu369victor/image-colorization-basic-implementation-with-cnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt; . With my little modification, I am trying to simple exchange color for example like  red to orange in an image.  The original code does not work.  I have 1500 sample of different simple images.  So far I have accuracy stuck or converging to at 50%.  So far the learning rate is 0.008 for optimizer adam.  I noticed that if I increase the steps per epoch say 400, the accuracy and loss goes to close of 100%.  If I go for steps of 30 per epoch, the loss goes down accordingly but accuracy converges to 50%.  Below is my python set up code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;X_,y_ = ExtractInput(ImagePath)&#xA;K.clear_session()&#xA;def InstantiateModel(in_):&#xA;model_ = Conv2D(16,(3,3),padding='same',strides=1)(in_)&#xA;model_ = LeakyReLU()(model_)&#xA;#model_ = Conv2D(64,(3,3), activation='relu',strides=1)(model_)&#xA;model_ = Conv2D(32,(3,3),padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_)&#xA;model_ = MaxPooling2D(pool_size=(2,2),padding='same')(model_)&#xA;&#xA;model_ = Conv2D(64,(3,3),padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_)&#xA;model_ = MaxPooling2D(pool_size=(2,2),padding='same')(model_)&#xA;&#xA;model_ = Conv2D(128,(3,3),padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_)&#xA;&#xA;model_ = Conv2D(256,(3,3),padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_)&#xA;&#xA;model_ = UpSampling2D((2, 2))(model_)&#xA;model_ = Conv2D(128,(3,3),padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_)&#xA;&#xA;model_ = UpSampling2D((2, 2))(model_)&#xA;model_ = Conv2D(64,(3,3), padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_) # it was commented&#xA;&#xA;concat_ = concatenate([model_, in_]) &#xA;&#xA;model_ = Conv2D(64,(3,3), padding='same',strides=1)(concat_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_)&#xA;&#xA;model_ = Conv2D(32,(3,3),padding='same',strides=1)(model_)&#xA;model_ = LeakyReLU()(model_)&#xA;model_ = BatchNormalization()(model_) # it was commented&#xA;&#xA;model_ = Conv2D(2,(3,3), activation='tanh',padding='same',strides=1)(model_)&#xA;&#xA;return model_&#xA;Input_Sample = Input(shape=(HEIGHT, WIDTH,1))&#xA;Output_ = InstantiateModel(Input_Sample)&#xA;Model_Colourization = Model(inputs=Input_Sample, outputs=Output_)&#xA;&#xA;LEARNING_RATE = 0.008&#xA;Model_Colourization.compile(optimizer=Adam(lr=LEARNING_RATE),loss='mean_squared_error',metrics=  ['accuracy'])&#xA;Model_Colourization.summary()&#xA;def GenerateInputs(X_,y_):&#xA;for i in range(len(X_)):&#xA;    X_input = X_[i].reshape(1,224,224,1)&#xA;    y_input = y_[i].reshape(1,224,224,2)&#xA;    yield (X_input,y_input)&#xA;    &#xA;history = Model_Colourization.fit_generator(GenerateInputs(X_,y_),epochs=10,verbose=1,&#xA;                                        steps_per_epoch=24,shuffle=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the output is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Layer (type)                    Output Shape         Param #     Connected to                     &#xA; =========================================================================================&#xA; input_1 (InputLayer)            (None, 224, 224, 1)  0                                            &#xA; __________________________________________________________________________________________&#xA; conv2d_1 (Conv2D)               (None, 224, 224, 16) 160         input_1[0][0]                    &#xA; ____________________________________________________________________________________________&#xA; leaky_re_lu_1 (LeakyReLU)       (None, 224, 224, 16) 0           conv2d_1[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; conv2d_2 (Conv2D)               (None, 224, 224, 32) 4640        leaky_re_lu_1[0][0]              &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_2 (LeakyReLU)       (None, 224, 224, 32) 0           conv2d_2[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_1 (BatchNor (None, 224, 224, 32) 128         leaky_re_lu_2[0][0]              &#xA; _____________________________________________________________________________________________&#xA; max_pooling2d_1 (MaxPooling2D)  (None, 112, 112, 32) 0           batch_normalization_1[0][0]      &#xA; ______________________________________________________________________________________________&#xA; conv2d_3 (Conv2D)               (None, 112, 112, 64) 18496       max_pooling2d_1[0][0]            &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_3 (LeakyReLU)       (None, 112, 112, 64) 0           conv2d_3[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_2 (BatchNor (None, 112, 112, 64) 256         leaky_re_lu_3[0][0]              &#xA; _____________________________________________________________________________________________&#xA; max_pooling2d_2 (MaxPooling2D)  (None, 56, 56, 64)   0           batch_normalization_2[0][0]      &#xA; _____________________________________________________________________________________________&#xA; conv2d_4 (Conv2D)               (None, 56, 56, 128)  73856       max_pooling2d_2[0][0]            &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_4 (LeakyReLU)       (None, 56, 56, 128)  0           conv2d_4[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_3 (BatchNor (None, 56, 56, 128)  512         leaky_re_lu_4[0][0]              &#xA; _____________________________________________________________________________________________&#xA; conv2d_5 (Conv2D)               (None, 56, 56, 256)  295168      batch_normalization_3[0][0]      &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_5 (LeakyReLU)       (None, 56, 56, 256)  0           conv2d_5[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_4 (BatchNor (None, 56, 56, 256)  1024        leaky_re_lu_5[0][0]              &#xA; _____________________________________________________________________________________________&#xA; up_sampling2d_1 (UpSampling2D)  (None, 112, 112, 256 0           batch_normalization_4[0][0]      &#xA; _____________________________________________________________________________________________&#xA; conv2d_6 (Conv2D)               (None, 112, 112, 128 295040      up_sampling2d_1[0][0]            &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_6 (LeakyReLU)       (None, 112, 112, 128 0           conv2d_6[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_5 (BatchNor (None, 112, 112, 128 512         leaky_re_lu_6[0][0]              &#xA; _____________________________________________________________________________________________&#xA; up_sampling2d_2 (UpSampling2D)  (None, 224, 224, 128 0           batch_normalization_5[0][0]      &#xA; _____________________________________________________________________________________________&#xA; conv2d_7 (Conv2D)               (None, 224, 224, 64) 73792       up_sampling2d_2[0][0]            &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_7 (LeakyReLU)       (None, 224, 224, 64) 0           conv2d_7[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_6 (BatchNor (None, 224, 224, 64) 256         leaky_re_lu_7[0][0]              &#xA; _____________________________________________________________________________________________&#xA; concatenate_1 (Concatenate)     (None, 224, 224, 65) 0           batch_normalization_6[0][0]      &#xA;                                                             input_1[0][0]                    &#xA; _____________________________________________________________________________________________&#xA; conv2d_8 (Conv2D)               (None, 224, 224, 64) 37504       concatenate_1[0][0]              &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_8 (LeakyReLU)       (None, 224, 224, 64) 0           conv2d_8[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_7 (BatchNor (None, 224, 224, 64) 256         leaky_re_lu_8[0][0]              &#xA; _____________________________________________________________________________________________&#xA; conv2d_9 (Conv2D)               (None, 224, 224, 32) 18464       batch_normalization_7[0][0]      &#xA; _____________________________________________________________________________________________&#xA; leaky_re_lu_9 (LeakyReLU)       (None, 224, 224, 32) 0           conv2d_9[0][0]                   &#xA; _____________________________________________________________________________________________&#xA; batch_normalization_8 (BatchNor (None, 224, 224, 32) 128         leaky_re_lu_9[0][0]              &#xA; _____________________________________________________________________________________________&#xA; conv2d_10 (Conv2D)              (None, 224, 224, 2)  578         batch_normalization_8[0][0]      &#xA; =============================================================================================&#xA; Total params: 820,770&#xA; Trainable params: 819,234&#xA; Non-trainable params: 1,536&#xA;&#xA;  Epoch 1/10&#xA;  24/24 [==============================] - 520s 22s/step - loss: 0.1813 - acc: 0.5552&#xA;  Epoch 2/10&#xA;  24/24 [==============================] - 513s 21s/step - loss: 0.0125 - acc: 0.5076&#xA;  Epoch 3/10&#xA;  24/24 [==============================] - 517s 22s/step - loss: 0.0020 - acc: 0.5050&#xA;  Epoch 4/10&#xA;  24/24 [==============================] - 517s 22s/step - loss: 0.0016 - acc: 0.4796&#xA;  Epoch 5/10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;During the experimenting with the code I did something that I cannot replicate it.  I started with learning rate of 0.004, after two epochs, I interrupted kernel and started with 0.008. And I noticed that after two epoch, the acc was increasing.  However, when I restarted the entire program at learning rate of 0.008, the accuracy is decreasing.  Obviously it has something to do with the right amount of learning rate and steps per epochs.  Any suggestions on improving the accuracy?&lt;/p&gt;&#xA;" OwnerUserId="38228" LastActivityDate="2020-06-27T23:37:30.840" Title="CNN keras accuracy not improving" Tags="&lt;convolutional-neural-networks&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22218" PostTypeId="1" CreationDate="2020-06-28T01:03:34.500" Score="0" ViewCount="47" Body="&lt;p&gt;I am new to training VAEs and I am using it on some 16x16 images, that contains some images from a physics experiment with one or 2 events i.e. the images are mainly black, except for one or 2 regions where we have a Gaussian-like signal around several pixels. Below are the main components of my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class View(nn.Module):&#xA;    def __init__(self, size):&#xA;        super(View, self).__init__()&#xA;        self.size = size&#xA;&#xA;    def forward(self, tensor):&#xA;        return tensor.view(self.size)&#xA;&#xA;class Encoder(nn.Module):&#xA;    def __init__(self, lat_dim):&#xA;        super().__init__()&#xA;&#xA;        N = 32&#xA;        &#xA;        self.encoder = nn.Sequential(&#xA;            nn.Conv2d(1, N, 4, 2, 1),          # B, N, 8, 8&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(N, N, 4, 2, 1),          # B,  N, 4, 4&#xA;            nn.ReLU(True),&#xA;            nn.Conv2d(N, 2*N, 4, 1),          # B,  N, 4, 4&#xA;            nn.ReLU(True),&#xA;            View((-1, 2*N*1*1)),                 # B, 2*N&#xA;        )    &#xA;&#xA;        self.mu = nn.Linear(2*N, lat_dim)&#xA;        self.var = nn.Linear(2*N, lat_dim)&#xA;&#xA;    def forward(self, x):&#xA;        &#xA;        x = self.encoder(x)&#xA;        z_mu = self.mu(x)&#xA;        z_var = self.var(x)&#xA;        &#xA;        return z_mu, z_var&#xA;&#xA;class Decoder(nn.Module):&#xA;    def __init__(self, lat_dim):&#xA;        super().__init__()&#xA;        &#xA;        N = 32&#xA;&#xA;        self.decoder = nn.Sequential(&#xA;            nn.Linear(lat_dim, 2*N),               # B, 256&#xA;            View((-1, 2*N, 1, 1)),               # B, 256,  1,  1&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(2*N, N, 4),      # B,  64,  4,  4&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(N, N, 4, 2, 1), # B,  64,  8,  8&#xA;            nn.ReLU(True),&#xA;            nn.ConvTranspose2d(N, 1, 4, 2, 1), # B,  32, 16, 16&#xA;        )&#xA;    &#xA;    def forward(self, x):&#xA;        x = self.decoder(x)&#xA;&#xA;        return x&#xA;&#xA;class VAE(nn.Module):&#xA;    def __init__(self, enc, dec):&#xA;        super().__init__()&#xA;&#xA;        self.enc = enc&#xA;        self.dec = dec&#xA;&#xA;    def forward(self, x):&#xA;        # encode&#xA;        z_mu, z_var = self.enc(x)&#xA;&#xA;        # sample from the distribution having latent parameters z_mu, z_var&#xA;        # reparameterize&#xA;        std = torch.exp(z_var / 2)&#xA;        eps = torch.randn_like(std)&#xA;        x_sample = eps*std+z_mu &#xA;&#xA;        # decode&#xA;        predicted = self.dec(x_sample)&#xA;        return predicted, z_mu, z_var&#xA;&#xA;lat_dim = 10 &#xA;model_E = Encoder(lat_dim) &#xA;model_D = Decoder(lat_dim) &#xA;model = VAE(model_E, model_D).cuda()&#xA;&#xA;for epoch in range(801):&#xA;    model.train()&#xA;    &#xA;    if epoch%200==0 and epoch&amp;gt;1:&#xA;        lrs = lrs/10&#xA;    optimizer = optim.Adam(model.parameters(), lr = lrs, weight_decay=1e-3)&#xA;    &#xA;    for x, y in zip(train_data, train_data_labels):&#xA;        optimizer.zero_grad()&#xA;&#xA;        x = x.float().cuda()&#xA;        y = y.float().cuda()&#xA;        &#xA;        x_sample, z_mu, z_var = model(x)        &#xA;        recon_loss = F.mse_loss(x_sample, x)&#xA;        kl_loss = torch.mean(-0.5 * torch.sum(1 + z_var - z_mu ** 2 - torch.exp(z_var), dim = 1), dim = 0)&#xA;        &#xA;        beta = 0.1&#xA;        loss = recon_loss + beta*kl_loss&#xA;        &#xA;        loss.backward()&#xA;        optimizer.step()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The data is normalized to mean 0 and std of 1. When I train it however, the KL divergence loss goes almost instantly (I guess it starts there) to ~10^-5 and keeps going down to around 10^-10, while the reconstruction loss doesn't change from the starting values (around 1). I tried different values for the beta parameter (the weight of the kl loss) and i have to set that to 10^-4 or lower for the reconstruction loss to actually go down. But if I do that the KL loss doesn't change much (which makes sense I guess). But in most papers I read about VAE the beta parameter is between 0 and 1, so I guess I am doing something wrong. Also my data is a lot simpler in both structure and size compared to MNIST (or other complex data sets), so I guess that a VAE code should work. What am I doing wrong? I am using a batch size of 128 and 6000 images for training. The loss mentioned in the post is the training loss. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="22839" LastActivityDate="2020-06-28T01:03:34.500" Title="VAE KL divergence loss decreases really fast" Tags="&lt;variational-autoencoder&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22219" PostTypeId="2" ParentId="10812" CreationDate="2020-06-28T02:07:57.607" Score="0" Body="&lt;p&gt;For anyone coming across this question and wants a very intuitive understanding of first and every visit monte-carlo, look at the answer given in the link provided here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://amp.reddit.com/r/reinforcementlearning/comments/9zkdjb/d_help_need_in_understanding_monte_carlo_first/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://amp.reddit.com/r/reinforcementlearning/comments/9zkdjb/d_help_need_in_understanding_monte_carlo_first/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After looking at that intuition, then you can come back and look at nbroz answer provided above.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps anyone struggling with this idea&lt;/p&gt;&#xA;" OwnerUserId="38229" LastEditorUserId="38229" LastEditDate="2020-06-28T02:29:06.000" LastActivityDate="2020-06-28T02:29:06.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22220" PostTypeId="1" AcceptedAnswerId="22229" CreationDate="2020-06-28T02:19:59.697" Score="4" ViewCount="100" Body="&lt;p&gt;&lt;a href=&quot;https://coq.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coq&lt;/a&gt; exists, and there are other similar projects out there. Further, Reinforcement Learning has made splashes in the domain of playing games (a la Deepmind &amp;amp; OpenAI and other less well-known efforts).&lt;/p&gt;&#xA;&lt;p&gt;It seems to me that these two domains deserve to be married such that machine learning agents try to solve mathematical theorems. &lt;strong&gt;Does anyone know of any efforts in this area?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm a relative novice in both of these domains, but I'm proficient enough at both to take a stab at building a basic theorem solver myself and trying to make a simple agent have a go at solving some basic number theory problems. When I went to look for prior art in the area I was very surprised to find none. I'm coming here as an attempt to broaden my search space.&lt;/p&gt;&#xA;" OwnerUserId="38230" LastEditorUserId="2444" LastEditDate="2020-06-28T20:53:25.297" LastActivityDate="2020-06-28T20:53:25.297" Title="Has reinforcement learning been used to prove mathematical theorems?" Tags="&lt;reinforcement-learning&gt;&lt;automated-theorem-proving&gt;&lt;coq&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22221" PostTypeId="1" AcceptedAnswerId="22230" CreationDate="2020-06-28T06:59:30.497" Score="2" ViewCount="113" Body="&lt;p&gt;I am a bit confused as to how exactly I should be implementing SARSA (or Q-learning too) on what is a simple 2-stage Markov Decision Task. The structure of the task is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8RgU4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8RgU4.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Basically, there are three states &lt;span class=&quot;math-container&quot;&gt;$\{S_1,S_2,S_3\}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; is in the first stage for which the two possible actions are the two yellow airplanes. &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S_3$&lt;/span&gt; are the possible states for the second stage and the feasible actions are the blue and red background pictures, respectively. There is only a reward at the end of the second stage choice. If I call the two first stage actions &lt;span class=&quot;math-container&quot;&gt;$\{a_{11},a_{12}\}$&lt;/span&gt; and the four possible second stage actions &lt;span class=&quot;math-container&quot;&gt;$\{a_{21},a_{22},a_{23},a_{24}\}$&lt;/span&gt;, from left to right, then a sample trial/episode will look like:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$S_1, a_{11}, S_2, a_{22},R \quad \text{ or }\quad S_1, a_{11}, S_3, a_{24}, R.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the paper I am reading, where the figure is from, they used a complicated version of TD&lt;span class=&quot;math-container&quot;&gt;$(\lambda)$&lt;/span&gt; in which they maintained two action-value functions &lt;span class=&quot;math-container&quot;&gt;$Q_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q_2$&lt;/span&gt; for each stages. On the other hand, I am trying to implement a simple SARSA update for each episode &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q_{t+1}(s,a)= Q_t(s,a) + \alpha\left(r + \gamma\cdot Q_t(s',a') - Q_t(s,a)\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the first-stage, there is no reward so an actual realization will look like:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q_{t+1}(S_1, a_{11}) = Q_t(S_1,a_{11})+\alpha\left( \gamma\cdot Q_t(S_3,a_{23}) - Q_t(S_1,a_{11})\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I guess my confusion is then how should it look like for the second stage of an episode? That is, if we continue the above realization of the task above, &lt;span class=&quot;math-container&quot;&gt;$S_1, a_{11}, S_3, a_{23}, R$&lt;/span&gt;, would should fill in the &lt;span class=&quot;math-container&quot;&gt;$?$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q_{t+1}(S_3,a_{23}) = Q_t(S_3,a_{23}) + \alpha\left(R +\gamma\cdot Q_t(\cdot,\cdot)-Q_t(s_3,a_{23}) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;One on hand, it seems to me that since this is the end of an episode, we assign &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to the &lt;span class=&quot;math-container&quot;&gt;$Q_t(\cdot,\cdot).$&lt;/span&gt; On the other hand, the nature of this task is that it repeats the same episode over time for a total of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, a large number, times we need &lt;span class=&quot;math-container&quot;&gt;$Q_t(\cdot,\cdot) = Q_t(S_1,\cdot),$&lt;/span&gt; with the additional action-selection in the first stage there.&lt;/p&gt;&#xA;&lt;p&gt;I will greatly appreciate if someone can tell me what is the right way to go here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3077926/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The link to paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38234" LastEditorUserId="2444" LastEditDate="2020-06-28T20:49:51.270" LastActivityDate="2020-06-28T20:49:51.270" Title="Implementing SARSA for a 2-stage Markov Decision Process" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22222" PostTypeId="2" ParentId="22051" CreationDate="2020-06-28T10:54:15.643" Score="1" Body="&lt;p&gt;Personally, I find the best way to think of SMDPs intuitively by just imagining that you just discretise the time into such small steps (infinitesimally small steps if necessary) that you can treat it as a normal MDP again, but with some extra domain knowledge that you can exploit primarily for computational efficiency:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Only at time steps that really correspond to &amp;quot;events&amp;quot; in your SMDP can you observe non-zero rewards; at all other time steps you just get rewards equal to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Only at time steps that really correspond to &amp;quot;events&amp;quot; in your SMDP do you have an action space greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;; in all the &amp;quot;fake&amp;quot; time steps, you have no agency, you just have a single action available (say, a &amp;quot;dummy&amp;quot; or &amp;quot;null&amp;quot; action). So all of these &amp;quot;fake&amp;quot; time steps do not contribute in any way to the &amp;quot;credit assignment&amp;quot; problem in RL, and you can kind of ignore them in your learning steps; only the time spent in them can still be important for discount factors &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; (real number not an integer number) shows the time between two arrivals, should I update Q-functions as follows:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, an update rule like that looks correct to me. Let's take an example situation, where &lt;span class=&quot;math-container&quot;&gt;$\tau = 2.0$&lt;/span&gt;, and instead of using the update rule you suggest, we take the &amp;quot;proper&amp;quot; approach of discretising into smaller time steps, and treating it as a regular MDP. In this simple example case, it is sufficient to discretise by taking time steps that correspond to durations of &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the SMDP, we'll have only a single transition &lt;span class=&quot;math-container&quot;&gt;$s_0 \rightarrow s_2$&lt;/span&gt; (it will become clear why I use slightly strange time-indexing here soon), after which we observe a reward, and this transition takes time &lt;span class=&quot;math-container&quot;&gt;$\tau = 2.0$&lt;/span&gt;. In the corresponding MDP, we'll have two state transitions; &lt;span class=&quot;math-container&quot;&gt;$s_0 \rightarrow s_1$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$s_1 \rightarrow s_2$&lt;/span&gt;, with two reward observations &lt;span class=&quot;math-container&quot;&gt;$R_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_2$&lt;/span&gt;, where we know for sure that:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$R_2 = 0$&lt;/span&gt; (because it does not actually correspond to any event in the SMDP)&lt;/li&gt;&#xA;&lt;li&gt;We have a meaningful choice between multiple actions at &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, each of which can have different transition probabilities for taking us into different &amp;quot;dummy&amp;quot; states &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;, and yield possibly-non-zero rewards &lt;span class=&quot;math-container&quot;&gt;$R_1$&lt;/span&gt;. In the dummy state &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;, we'll always only have the choice for a single dummy/null action (because this state does not correspond to any event in the SMDP), which always yields &lt;span class=&quot;math-container&quot;&gt;$R_2 = 0$&lt;/span&gt; as mentioned above.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, the correct update rule for &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; where we picked a forced dummy action &lt;span class=&quot;math-container&quot;&gt;$\varnothing$&lt;/span&gt; and are doomed to receive a reward &lt;span class=&quot;math-container&quot;&gt;$R_2 = 0$&lt;/span&gt;, would be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_1, \varnothing) \gets Q(s_1, \varnothing) + \alpha \left( 0 + \gamma \max_{a'} Q(s_2, a') - Q(s_1, \varnothing) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the correct update rule for &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, where we picked a meaningful action &lt;span class=&quot;math-container&quot;&gt;$a_0$&lt;/span&gt; and may get a non-zero reward &lt;span class=&quot;math-container&quot;&gt;$R_1$&lt;/span&gt;, would be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_0, a_0) \gets Q(s_0, a_0) + \alpha \left( R_1 + \gamma \max_{a'} Q(s_1, a') - Q(s_0, a_0) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this last update rule, we know that &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; is a dummy state in which the dummy action &lt;span class=&quot;math-container&quot;&gt;$\varnothing$&lt;/span&gt; is the &lt;strong&gt;only&lt;/strong&gt; legal action. So, we can get rid of the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator there and simplify it to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_0, a_0) \gets Q(s_0, a_0) + \alpha \left( R_1 + \gamma Q(s_1, \varnothing) - Q(s_0, a_0) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since we know that &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; is a dummy state in which we are never able to make meaningful choices anyway, it seems a bit wasteful to actually keep track of &lt;span class=&quot;math-container&quot;&gt;$Q(s_1, \varnothing)$&lt;/span&gt; values for it. Luckily, we can easily express &lt;span class=&quot;math-container&quot;&gt;$Q(s_1, \varnothing)$&lt;/span&gt; directly in terms of &lt;span class=&quot;math-container&quot;&gt;$Q(s2, \cdot)$&lt;/span&gt; -- which is exactly the next set of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values that we would be interested in keeping track of again:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_1, \varnothing) = \mathbb{E} \left[ 0 + \gamma \max_{a'} Q(s_2, a') \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So if we want to skip learning &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values for &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; (because it's kind of a waste of effort), we can just use this definition and plug it straight into the update rule for &lt;span class=&quot;math-container&quot;&gt;$Q(s_0, a_0)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is inherently an algorithm that just concrete samples of experience to estimate expectations (and this is a major reason why it typically uses learning rates &lt;span class=&quot;math-container&quot;&gt;$\alpha &amp;lt; 1.0$&lt;/span&gt;, so we can simply get rid of the expectation operator when doing this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_0, a_0) \gets Q(s_0, a_0) + \alpha \left( R_1 + \gamma \left[ \gamma \max_{a'} Q(s_2, a') \right] - Q(s_0, a_0) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and this is basically the update rule that you suggested. Note; here I assumed that you receive your rewards directly when you do take actions in your SMDP, which is why I had &lt;span class=&quot;math-container&quot;&gt;$R_1$&lt;/span&gt; as a possibly-non-zero reward, and always &lt;span class=&quot;math-container&quot;&gt;$R_2 = 0$&lt;/span&gt;. I suppose you could also in some cases envision SMDPs where the reward only arrives on the next SMDP-time-step, and that the amount of time that ends up being elpased in between the two events is important to take into account via the discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;. So you could also choose to model a problem where &lt;span class=&quot;math-container&quot;&gt;$R_1 = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_2$&lt;/span&gt; may be non-zero, and this would yield a different update rule (I think one where the reward gets multiplied by &lt;span class=&quot;math-container&quot;&gt;$\gamma^{\tau - 1}$&lt;/span&gt;? not sure, would have to go through the steps again).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What measure should be used in the SMDP setting? I would be thankful if someone can explain the Q-Learning algorithm for the SMDP problem with this setting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think it would be important to involve the amount of time that you take somehow in your evaluation criterion. You could run episodes for a fixed amount of time, and then just evaluate agents based on the sum of rewards. If you don't run for a fixed amount of time (but instead for a fixed number of steps, each of which may take variable amounts of time, for example), you would probably instead want to evaluate agents based on the average rewards per unit of time. You could also include discount factors in your evaluation if you want, but probably don't have to.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Moreover, I am wondering when Q-functions are updated. For example, if a customer enters our website and purchases a product, we want to update the Q-functions. Suppose that the planning horizon (state &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;) starts at 10:00 am, and the first customer enters at 10:02 am, and we sell a product and gain &lt;span class=&quot;math-container&quot;&gt;$R_1$&lt;/span&gt; and the state will be &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt;. The next customer enters at 10:04 am, and buy a product, and gain reward &lt;span class=&quot;math-container&quot;&gt;$R_2$&lt;/span&gt; (state &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt;). In this situation, should we wait until 10:02 to update the Q-function for state &lt;span class=&quot;math-container&quot;&gt;$S_0$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This depends on you state representation, and how you model a &amp;quot;state&amp;quot;, and to what extent previous actions have influence the state you end up in. Keep in mind that the update rule for &lt;span class=&quot;math-container&quot;&gt;$Q(S_0)$&lt;/span&gt; also requires for &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; (or even &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$S_1$&lt;/span&gt; is a &amp;quot;dummy state&amp;quot; that you skip) to have been observed. So, if your state representation includes some features describing the &amp;quot;current customer&amp;quot; for which you want to pick an action (do you offer them a discount or not, for example?), then you can only update the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value for the previous customer when the next customer has arrived. This model does assume that your previous actions have some level of influence over the future states that you may end up in though. For example, you might assume that if your actions make the first customer very happy, you get a better reputation and are therefore more likely to end up in future states where other customers visit more frequently.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-06-28T10:54:15.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22224" PostTypeId="1" CreationDate="2020-06-28T14:10:16.097" Score="1" ViewCount="45" Body="&lt;p&gt;I am using Sutton and Barto's book for Reinforcement Learning.&lt;/p&gt;&#xA;&lt;p&gt;In Chapter 8, I am having difficulty in understanding the Trajectory Sampling.&lt;/p&gt;&#xA;&lt;p&gt;I have read the particular section on trajectory sampling (Sec 8.6) two times (plus 3rd time partially) but still, I do not get how it is different from the normal sampling update, and what are its benefits.&lt;/p&gt;&#xA;" OwnerUserId="36710" LastEditorUserId="2444" LastEditDate="2020-06-28T15:11:36.937" LastActivityDate="2020-06-28T15:11:36.937" Title="How is trajectory sampling different than normal (importance) sampling in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;importance-sampling&gt;&lt;dyna&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22225" PostTypeId="1" CreationDate="2020-06-28T14:12:56.333" Score="0" ViewCount="38" Body="&lt;p&gt;I am inspired by the paper &lt;a href=&quot;https://arxiv.org/abs/1611.01578&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Architecture Search with Reinforcement Learning&lt;/a&gt; to use reinforcement learning for optimizing a child network (learner). My meta-learner (controller or parent network) is an MLP and will take as the reward function a silhouette score. Its output is a vector of real numbers between 0 and 1. These values are k different possibilities for the number of clusters (the goal is to cluster the result of the child network which is an auto-encoder, embedded images are the input to the meta-learner).&lt;/p&gt;&#xA;&lt;p&gt;What I am confused about is the environment here and how to implement this network. I was reading &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; and the author has used &lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gym&lt;/a&gt; library to set the environment.&lt;/p&gt;&#xA;&lt;p&gt;Should I build an environment from scratch myself or it is not always needed?&lt;/p&gt;&#xA;&lt;p&gt;I appreciate any help or hints or links to a source that helps me understand better RL concepts. I am new to it and easily gets confused.&lt;/p&gt;&#xA;" OwnerUserId="37744" LastEditorUserId="2444" LastEditDate="2020-06-28T20:48:43.600" LastActivityDate="2020-07-28T21:02:35.197" Title="Should I build an environment from scratch myself or it is not always needed?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;environment&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22226" PostTypeId="2" ParentId="22225" CreationDate="2020-06-28T14:19:49.813" Score="1" Body="&lt;p&gt;I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway &lt;a href=&quot;https://github.com/openai/gym&quot; rel=&quot;nofollow noreferrer&quot;&gt;opensourced&lt;/a&gt;, you can always edit it to your liking.&lt;/p&gt;&#xA;&lt;p&gt;If you want a custom environment, you can add an environment to gym like &lt;a href=&quot;https://towardsdatascience.com/creating-a-custom-openai-gym-environment-for-stock-trading-be532be3910e&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-06-28T14:19:49.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22227" PostTypeId="2" ParentId="22224" CreationDate="2020-06-28T14:27:44.530" Score="1" Body="&lt;p&gt;Here is my understanding:&lt;/p&gt;&#xA;&lt;p&gt;In trajectory sampling as the book describes it, we use the current policy on the simulator to get (next-state, action) pairs. The advantage is that if some states occur more frequently than others in that environment, and if we take enough samples, the distribution among the samples would be similar to the actual distribution.&lt;/p&gt;&#xA;&lt;p&gt;On the another hand, you can sample in a different manner if you have access to the model. Suppose you have access to the transition distribution. Then you can sample your start state uniformly, and use the transition distribution to get the (next-state, action) pairs. This can be useful if you want to force your algorithm to look at all states evenly, even if that's not the actual case.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-06-28T14:27:44.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22228" PostTypeId="1" CreationDate="2020-06-28T14:49:36.407" Score="0" ViewCount="94" Body="&lt;p&gt;Hello I'm trying to implement DQN Agent to play Atari-Pong game. But still after 500.000 parameter updates, the model still scores -21. This is my code, do you have any idea what could be wrong?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Replaybuffer&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import random as r&#xA;from collections import deque&#xA;import sys&#xA;class ReplayBuffer():&#xA;    def __init__(self, buffer_size):&#xA;        self.buffer = deque([], maxlen=buffer_size)&#xA;&#xA;    def add(self, new_state):&#xA;        if len(new_state) != 5:&#xA;            raise Exception(&amp;quot;States must have: state, action, reward, next_state, done&amp;quot;)&#xA;        &#xA;        self.buffer.append(new_state)&#xA;&#xA;    def sample(self, batch_size):&#xA;        return r.sample(self.buffer, batch_size)&#xA;    &#xA;    def size(self):&#xA;        return len(self.buffer)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Preprocess frames&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def preprocess_frame(obs):&#xA;    processed_observe = np.uint8(&#xA;        resize(rgb2gray(obs), (84, 84), mode='constant') * 255)   &#xA;    return processed_observe&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Huber Loss&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def huber_loss(x, delta=1.0):&#xA;    &amp;quot;&amp;quot;&amp;quot;Reference: https://en.wikipedia.org/wiki/Huber_loss&amp;quot;&amp;quot;&amp;quot;&#xA;    return tf.where(&#xA;        tf.abs(x) &amp;lt; delta,&#xA;        tf.square(x) * 0.5,&#xA;        delta * (tf.abs(x) - 0.5 * delta)&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;DQN&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;class DQNetwork():&#xA;    def __init__(self, scope, state_size=(84, 84, 4), num_outputs=3, gamma=0.9, learning_rate=0.001):&#xA;        &#xA;        self.scope = scope&#xA;        &#xA;&#xA;        with tf.variable_scope(self.scope):&#xA;&#xA;            # ---------------------&#xA;            # Basic Deep Q-Network&#xA;            # ---------------------&#xA;            self.x = tf.placeholder(tf.float32, shape=[None, *state_size], name=&amp;quot;inputs&amp;quot;)&#xA;            &#xA;            # Input is 84x84x4&#xA;            self.conv1 = tf.layers.conv2d(inputs = self.x,&#xA;                                         filters = 32,&#xA;                                         kernel_size = [8,8],&#xA;                                         strides = [4,4],&#xA;                                         padding = &amp;quot;VALID&amp;quot;,&#xA;                                         name = &amp;quot;conv1&amp;quot;,&#xA;                                         activation=&amp;quot;relu&amp;quot;)&#xA;            &#xA;            self.conv2 = tf.layers.conv2d(inputs = self.conv1,&#xA;                                 filters = 64,&#xA;                                 kernel_size = [4,4],&#xA;                                 strides = [2,2],&#xA;                                 padding = &amp;quot;VALID&amp;quot;,&#xA;                                 name = &amp;quot;conv2&amp;quot;,&#xA;                                activation=&amp;quot;relu&amp;quot;)&#xA;&#xA;            &#xA;            &#xA;            &#xA;            self.conv3 = tf.layers.conv2d(inputs = self.conv2,&#xA;                                 filters = 64,&#xA;                                 kernel_size = [3,3],&#xA;                                 strides = [1,1],&#xA;                                 padding = &amp;quot;VALID&amp;quot;,&#xA;                                 name = &amp;quot;conv3&amp;quot;,&#xA;                                activation=&amp;quot;relu&amp;quot;)&#xA;            &#xA;            self.flatten = tf.layers.flatten(self.conv3)&#xA;            &#xA;            self.fc = tf.layers.dense(inputs = self.flatten,&#xA;                                  units = 512,&#xA;                                  activation = tf.nn.relu,&#xA;                                name=&amp;quot;fc1&amp;quot;)&#xA;            &#xA;            self.logits = tf.layers.dense(inputs = self.fc,&#xA;                                          units = num_outputs, &#xA;                                        activation=None)&#xA;            &#xA;&#xA;            self.best_action = tf.argmax(self.logits, name=&amp;quot;best_action&amp;quot;, axis=1)&#xA;            self.max_q = tf.reduce_max(self.logits, name=&amp;quot;max_q&amp;quot;, axis=1)&#xA;            # ------------------------&#xA;            # Q-Learning Calculations&#xA;            # ------------------------&#xA;               &#xA;            if scope == 'Target':&#xA;                self.rewards = tf.placeholder(tf.float32, shape=None, name=&amp;quot;rewards&amp;quot;)    &#xA;                self.gamma = tf.constant(gamma, name=&amp;quot;Gamma&amp;quot;)&#xA;                self.done = tf.placeholder(tf.int32, shape=None, name=&amp;quot;done_values&amp;quot;)&#xA;&#xA;                # if game is not done, then placeholder is 0, therefore tf.abs = 1&#xA;                # if game is done, then placeholder is 1, therefore tf.abs = 0 --&amp;gt; only reward is taken into account&#xA;                self.td_target = self.rewards + (self.gamma*self.max_q) * tf.cast( tf.abs(self.done -1 ), tf.float32)&#xA;              &#xA;                &#xA;            if scope == 'Q':&#xA;                self.target_placeholder = tf.placeholder(tf.float32, shape=None, name=&amp;quot;target_placeholder_q&amp;quot;)  &#xA;                # here we feed the actions we really have done, e.g. not 1 but 2&#xA;                # this results in the one-hot encoding: [0 1], if we mulitply this with, let's say the logit = [0.4, 0.7],&#xA;                # we get [0, 0.7] and tf.sum leads to 0.7. We therefore get the Q value of the desired action&#xA;                self.actions = tf.placeholder(tf.uint8, shape=None, name=&amp;quot;AllActions&amp;quot;)&#xA;                self.actions_onehot = tf.one_hot(self.actions, depth=num_outputs, name=&amp;quot;One_Hot&amp;quot;)  &#xA;                self.Q = tf.reduce_sum(tf.multiply(self.actions_onehot, self.logits))&#xA;                &#xA;                # Huber Loss&#xA;                &amp;quot;&amp;quot;&amp;quot;&#xA;                self.error = tf.abs(self.target_placeholder-self.Q, name=&amp;quot;td_error&amp;quot;)&#xA;                self.quadratic_part = tf.clip_by_value(self.error, 0.0, 1.0) &#xA;                self.linear_part = self.error - self.quadratic_part&#xA;                self.loss = tf.reduce_mean(0.5 * tf.square(self.quadratic_part) + self.linear_part)&#xA;                &amp;quot;&amp;quot;&amp;quot;&#xA;                self.huber_loss = huber_loss(self.target_placeholder-self.Q)&#xA;                self.loss = tf.reduce_mean(self.huber_loss)&#xA;                &#xA;                self.optimizer = tf.train.RMSPropOptimizer(learning_rate=learning_rate, epsilon=0.01)&#xA;                self.train = self.optimizer.minimize(self.loss, name=&amp;quot;minimize&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update Network&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from operator import attrgetter&#xA;&#xA;def get_update_target_ops(Q_network, Target_network):&#xA;&#xA;    # You code comes here     &#xA;    # 1. get the trainable variables per network&#xA;    Q_trainable = tf.trainable_variables(scope=Q_network.scope)&#xA;    Target_trainable = tf.trainable_variables(scope=Target_network.scope)&#xA;    &#xA;    # 2. sort them with sorted(list, key=attrgetter())&#xA;    Q_trainable  = sorted(Q_trainable, key=attrgetter(&amp;quot;name&amp;quot;))&#xA;    Target_trainable  = sorted(Target_trainable, key=attrgetter(&amp;quot;name&amp;quot;))&#xA;&#xA;    &#xA;    # 3.create a new list with all assign ops&#xA;    update_target_expr = []&#xA;    for q_var, t_var in zip(Q_trainable, Target_trainable):&#xA;        update_target_expr.append(t_var.assign(q_var))&#xA;&#xA;    return update_target_expr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Greedy action&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;obs_space = (84, 84, 4)&#xA;&#xA;def choose_egreedy_action(session, epsilon, network, state):&#xA;    &#xA;    &#xA;    if np.random.rand() &amp;lt;= epsilon:&#xA;        #return np.random.randint(0, n)&#xA;        return np.random.randint(0, n)&#xA;    else:&#xA;        #print(&amp;quot;best action taken&amp;quot;)&#xA;        state = np.float32(state / 255.0)&#xA;        state_reshaped = np.reshape(state, (1, *obs_space))&#xA;        best_action = session.run(network.best_action, feed_dict={network.x:state_reshaped})[0]&#xA;        return best_action&#xA;&#xA;import numpy as np&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Epsilon&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class LinearSchedule():&#xA; &#xA;    def __init__(self, start_epsilon, final_epsilon, pre_train_steps, decay, exploration_steps):&#xA;        &#xA;        self.start_epsilon = start_epsilon&#xA;        self.final_epsilon = final_epsilon&#xA;        self.pre_train_steps = pre_train_steps&#xA;        self.exploration_steps = exploration_steps&#xA;        &#xA;        self.decay = decay&#xA;        self.epsilon = start_epsilon&#xA;&#xA;    def value(self, t):&#xA;        if t &amp;lt;= self.pre_train_steps:&#xA;            return self.start_epsilon&#xA;        else:&#xA;            if (t-self.pre_train_steps) &amp;gt; self.exploration_steps:&#xA;                  return self.final_epsilon&#xA;            else:               &#xA;                return self.start_epsilon - (t-self.pre_train_steps)*self.decay&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Train function&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def train(sess, Q, Target, buffer, batch_size):&#xA;&#xA;    # You code comes here     &#xA;    # 1. Sample from the replay buffer&#xA;    mini_batches = buffer.sample(batch_size)&#xA;       &#xA;    observations, actions, rewards, next_observations, done,  = map(list, zip(*mini_batches))&#xA;    &#xA;    observations = np.array(observations)/255.&#xA;    next_observations =np.array(next_observations)/255.&#xA;    td_targets = sess.run( Target.td_target, feed_dict={Target.x : next_observations, Target.rewards:rewards, Target.done:done})&#xA;&#xA;    loss, _ = sess.run([Q.loss, Q.train], feed_dict={Q.x : observations, Q.target_placeholder:td_targets, Q.actions:actions})&#xA;    &#xA;    return loss&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Params&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;EPISODES = 30000&#xA;epsilon = 1.&#xA;epsilon_start, epsilon_end = 1.0, 0.1&#xA;exploration_steps = 500000&#xA;#exploration_steps = 100000.&#xA;&#xA;decay = (epsilon_start - epsilon_end) / exploration_steps&#xA;&#xA;batch_size = 128&#xA;train_start = 50000&#xA;update_target_rate = 10000&#xA;gamma = 0.99&#xA;buffer_size = 100000&#xA;no_op_steps = 10&#xA;&#xA;global_steps = 2000000&#xA;&#xA;&#xA;tf.reset_default_graph()&#xA;&#xA;Q_network  = DQNetwork(scope=&amp;quot;Q&amp;quot;, state_size=(84, 84, 4), num_outputs=3, gamma=gamma, learning_rate=0.00025)&#xA;T_network = DQNetwork(scope=&amp;quot;Target&amp;quot;,state_size=(84, 84, 4), num_outputs=3, gamma=gamma, learning_rate=0.00025)&#xA;update_target_network = get_update_target_ops(Q_network,T_network)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Training&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from tqdm import trange&#xA;import random&#xA;game = gym.make('PongDeterministic-v4')&#xA;scores, episodes = [], [], &#xA;&#xA;&#xA;with tf.Session() as sess:&#xA;    &#xA;        sess.run(tf.global_variables_initializer())&#xA;&#xA;        saver = tf.train.Saver()&#xA;        buffer = ReplayBuffer(buffer_size)&#xA;        &#xA;        global_step=0&#xA;&#xA;        sess.run(tf.global_variables_initializer())&#xA;&#xA;        epsilon_schedule = LinearSchedule(epsilon_start, epsilon_end, train_start, decay, exploration_steps)&#xA;        five_episode_score = 0&#xA;        &#xA;        done = False&#xA;        dead = False&#xA;&#xA;        # 1 episode = 5 lives&#xA;        step, score = 0, 0&#xA;        observe = game.reset()&#xA;&#xA;        for _ in range(random.randint(1, no_op_steps)):&#xA;            observe, _, _, _ = game.step(1)&#xA;&#xA;&#xA;        state = preprocess_frame(observe)&#xA;        history = np.stack((state, state, state, state), axis=2)&#xA;        history = np.reshape([history], (84, 84, 4))  &#xA;        &#xA;        eps = 0&#xA;        for global_step in trange(global_steps):&#xA;          &#xA;            step += 1&#xA;&#xA;            # get action for the current history and go one step in environment&#xA;            action = choose_egreedy_action(sess, epsilon, Q_network, history)&#xA;            if action == 0:&#xA;                real_action = 0&#xA;            elif action == 1:&#xA;                real_action = 2&#xA;            elif action == 2:&#xA;                real_action = 3&#xA;&#xA;            observe, reward, done, info = game.step(real_action)&#xA;&#xA;&#xA;            # pre-process the observation --&amp;gt; history&#xA;            next_state = preprocess_frame(observe)&#xA;            next_state = np.reshape([next_state], (84, 84, 1))&#xA;            #print(next_state.shape)&#xA;            next_history = np.append(next_state, history[ :, :, :3], axis=2)&#xA;&#xA;            reward = np.clip(reward, -1., 1.)&#xA;&#xA;            # save the sample &amp;lt;s, a, r, s'&amp;gt; to the replay memory&#xA;            buffer.add([history, action, reward, next_history, done])&#xA;&#xA;&#xA;            epsilon = epsilon_schedule.value(global_step)&#xA;&#xA;&#xA;            if global_step &amp;gt; train_start:&#xA;                train(sess, Q_network, T_network, buffer, batch_size)&#xA;&#xA;            # update the target model with model&#xA;            if global_step % update_target_rate == 0:&#xA;                sess.run(update_target_network) &#xA;&#xA;            score += reward&#xA;&#xA;&#xA;            history = next_history&#xA;&#xA;            # if done, plot the score over episodes&#xA;            if done:&#xA;                &#xA;                print(&amp;quot;episode:&amp;quot;, eps, &amp;quot; score:&amp;quot;, score, &amp;quot; global_step: &amp;quot;, global_step,&#xA;                     &amp;quot; epsilon: &amp;quot;, epsilon, &amp;quot; buffer: &amp;quot;, buffer.size())&#xA;&#xA;                eps += 1&#xA;                scores.append(score)&#xA;                episodes.append(step)&#xA;                &#xA;                done = False&#xA;&#xA;                # 1 episode = 5 lives&#xA;                step, score = 0, 0&#xA;                observe = game.reset()&#xA;&#xA;                for _ in range(random.randint(1, no_op_steps)):&#xA;                    observe, _, _, _ = game.step(1)&#xA;&#xA;&#xA;                state = preprocess_frame(observe)&#xA;                history = np.stack((state, state, state, state), axis=2)&#xA;                history = np.reshape([history], (84, 84, 4))  &#xA;&#xA;&#xA;&#xA;            if global_step % 500 == 0:&#xA;                saver.save(sess, f'models/pong/pong.ckpt')    &#xA;            &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've trained it for over 500.000 training steps, but still the reward average is almost -21. Regarding the paper, there should be an improvement already after 100.000 steps.&lt;/p&gt;&#xA;&lt;p&gt;Do you see any error in my code?&lt;/p&gt;&#xA;" OwnerUserId="38246" LastActivityDate="2020-06-28T14:49:36.407" Title="DQN not learning the game Pong" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22229" PostTypeId="2" ParentId="22220" CreationDate="2020-06-28T16:15:01.277" Score="4" Body="&lt;p&gt;Artificial Intelligence for Theorem Proving is an active research area as witnessed by the existence of the &lt;a href=&quot;http://aitp-conference.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AITP conference&lt;/a&gt; and of many publications on the topic. Some papers are mentioned in this thread: &lt;a href=&quot;https://coq.discourse.group/t/machine-learning-and-hammers-for-coq/303&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://coq.discourse.group/t/machine-learning-and-hammers-for-coq/303&lt;/a&gt;. I haven't read these papers myself, so I cannot point you to a paper using &lt;em&gt;reinforcement learning&lt;/em&gt; specifically, but given the important activity in this domain, I would be very surprised if it hadn't been attempted.&lt;/p&gt;&#xA;" OwnerUserId="38247" LastActivityDate="2020-06-28T16:15:01.277" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22230" PostTypeId="2" ParentId="22221" CreationDate="2020-06-28T18:56:36.023" Score="1" Body="&lt;p&gt;In this game you can view end of an episode two ways:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is an implied, terminal, fourth state &lt;span class=&quot;math-container&quot;&gt;$s_4$&lt;/span&gt; representing the end of the game.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could view the process as a continuous repeating one, where no matter what the choice is made in &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$s_3$&lt;/span&gt;, the following state is &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first, terminating, view is a simpler and entirely natural view since nothing that the agent does in one episode can influence the next. It will result in a Q table that predicts future rewards within a single episode for the current agent (as opposed to discounted view over multiple episodes).&lt;/p&gt;&#xA;&lt;p&gt;You are over-complicating things for yourself by ignoring that a zero reward is still a reward (of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;). There is no need to remove &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; from your initial update rule. In many environments there are rewards collected before the end of an episode.&lt;/p&gt;&#xA;&lt;p&gt;In addition, to complete the standard episodic view, you can note that &lt;span class=&quot;math-container&quot;&gt;$Q(s_4, \cdot) = 0$&lt;/span&gt; always by definition, hence so does &lt;span class=&quot;math-container&quot;&gt;$\text{max}_{a'}[Q(s_4, a'] = 0$&lt;/span&gt;. It is common here though to have a branch based on detecting a terminal state, and use a different update rule:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{t+1}(S_3,a_{23}) = Q_t(S_3,a_{23}) + \alpha\left(R - Q_t(s_3,a_{23}) \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In brief, most implementations of TD algorithms do this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Always assume a reward on each time step, which can be set to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Special case for end of episode with a simplified update rule, to avoid needing to store, look up or calculate the &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; value associated with terminal states&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;When implementing the environment, it is common to have a &lt;code&gt;step&lt;/code&gt; function that always returns reward, next state and whether or not it is terminal e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;reward, next_state, done = env.step(action)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Details may vary around this. If you are working with an environment that does not have such a function (many will not have an inherent reward), then it is common to implement a similar function as a convenient wrapper to the environment so that the agent code does not have to include calculations of what the reward should be or whether the state is terminal.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-28T19:01:51.453" LastActivityDate="2020-06-28T19:01:51.453" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22231" PostTypeId="1" AcceptedAnswerId="22277" CreationDate="2020-06-28T21:06:10.233" Score="0" ViewCount="31" Body="&lt;p&gt;Given network architecture, what are the possible ways to define fully connected layer &lt;code&gt;fc1&lt;/code&gt; to have a generalized structure such as &lt;code&gt;nn.Linear($size_of_previous_layer$, 50)&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;The main issue arising is due to &lt;code&gt;x = F.relu(self.fc1(x))&lt;/code&gt; in the forward function. After using the &lt;code&gt;flatten&lt;/code&gt;, I need to incorporate numerous dense layers. But to my understanding, &lt;code&gt;self.fc1&lt;/code&gt; must be initialized and hence, needs a size (to be calculated from previous layers). How can I declare the &lt;code&gt;self.fc1&lt;/code&gt; layer in a generalized manner?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;My Thought:&lt;/em&gt;&#xA;To get the size, I can calculate the size of the outputs from each of Convolution layer, and since I have just 3, it is feasible. But, in case of &lt;em&gt;n&lt;/em&gt; layers, how can you get the output size from the final convolutional layer?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class Net(nn.Module):&#xA;    def __init__(self):&#xA;        super(Net, self).__init__()&#xA;&#xA;        self.conv1 = nn.Conv2d(3, 10, kernel_size=3, padding = 1)&#xA;        self.conv2 = nn.Conv2d(10, 20, kernel_size=3, padding = 1)&#xA;        self.conv2_drop = nn.Dropout2d(0.4)&#xA;        self.conv3 = nn.Conv2d(20, 40, kernel_size=3, padding = 1)&#xA;        self.conv3_drop = nn.Dropout2d(0.4)&#xA;&#xA;        self.fc1 = nn.Linear(360, 50)  # self.fc1 = nn.Linear(&lt;span class=&quot;math-container&quot;&gt;$size_of_previous_layer$&lt;/span&gt;, 50)&#xA;&#xA;    def forward(self, x):&#xA;        x = F.relu(F.max_pool2d(self.conv1(x), 2))&#xA;        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))&#xA;        x = F.relu(F.max_pool2d(self.conv3_drop(self.conv3(x)), 2))&#xA;&#xA;        x = x.flatten(1)&#xA;&#xA;        x = F.relu(self.fc1(x))&#xA;        return F.log_softmax(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Input to the following architecture can assumed to be [3, 32, 32] (num_of_channels, height, width).&lt;/p&gt;&#xA;&lt;p&gt;PS:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Answers are expected in PyTorch.&lt;/li&gt;&#xA;&lt;li&gt;For single convolutional layer, it is quite easy. The question refers to solve for &lt;em&gt;n&lt;/em&gt; consecutive convolutional layers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37180" LastActivityDate="2020-06-30T21:58:19.827" Title="How is it possible to get the output size of `n` Consecutive Convolutional layers?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-07-01T02:09:00.900" ContentLicense="CC BY-SA 4.0" />
  <row Id="22232" PostTypeId="1" CreationDate="2020-06-28T22:51:35.317" Score="1" ViewCount="28" Body="&lt;p&gt;The intuition provided when introducing actor-critic algorithms is that the variance of its gradient estimates is smaller than in REINFORCE as, e.g., discussed &lt;a href=&quot;https://www.youtube.com/watch?v=EKqxumCuAAY&amp;amp;feature=youtu.be&amp;amp;t=278&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This intuition makes sense for the reasons outlined in the linked lecture.&lt;/p&gt;&#xA;&lt;p&gt;Is there a paper / lecture providing a formal proof of that claim for any type of actor-critic algorithm (e.g. the Q Actor-Critic)?&lt;/p&gt;&#xA;" OwnerUserId="38052" LastEditorUserId="2444" LastEditDate="2020-06-28T23:14:23.877" LastActivityDate="2020-06-28T23:14:23.877" Title="What is the proof that the variance of the gradient estimate in Actor-Critic is smaller than in REINFORCE?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;proofs&gt;&lt;actor-critic-methods&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22234" PostTypeId="1" CreationDate="2020-06-29T02:06:24.130" Score="1" ViewCount="38" Body="&lt;p&gt;Importance sampling is a common method for calculating off-policy estimates in RL. I have been reading through some of the original documentation (D.G. Horvitz and D.J. Thompson, Powell, M.J. and Swann, J) and cannot find any restrictions on the reward or value being estimated. However, it seems that there are constraints because the calculation is not what I would expect for RL environments that have negative rewards.&lt;/p&gt;&#xA;&lt;p&gt;For example, consider for a given action-state pair (&lt;span class=&quot;math-container&quot;&gt;$a_i, s_i$&lt;/span&gt;), &lt;span class=&quot;math-container&quot;&gt;$\pi_e(a|s) = 0.4$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_b(a|s) = 0.6,$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\pi_b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_e$&lt;/span&gt; are the behavioral and evaluation policies respectively. Also, assume the reward range is &lt;span class=&quot;math-container&quot;&gt;$[-1,0]$&lt;/span&gt;, and this action has a reward of &lt;span class=&quot;math-container&quot;&gt;$r_{\pi_b}=-0.5$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Under the IS definition, the expected reward under &lt;span class=&quot;math-container&quot;&gt;$\pi_b$&lt;/span&gt; would be &lt;span class=&quot;math-container&quot;&gt;$r_{\pi_e} = \frac{\pi_b(a|s)}{\pi_e(a|s)} r_{\pi_b}$&lt;/span&gt;. In this example, &lt;span class=&quot;math-container&quot;&gt;$r_{\pi_e}=-0.75$&lt;/span&gt; thus &lt;span class=&quot;math-container&quot;&gt;$r_{\pi_e} &amp;lt; r_{\pi_b}$&lt;/span&gt;. However, assuming a change of scale of the reward to be &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; which result in &lt;span class=&quot;math-container&quot;&gt;$r_{\pi_b}=0.5$&lt;/span&gt;, results in &lt;span class=&quot;math-container&quot;&gt;$r_{\pi_e} &amp;gt; r_{\pi_b}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;All examples of IS I have seen in reference focus on positive rewards. However, I find myself wondering if this formulation applies to negative rewards too. If this formulation does allow for negative reward structures, I'm not sure how to interpret this result. I'm wondering how changing the scale of the reward could change the order? Is there any documentation on the requirements of the value in IS? Any insight into this would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="38256" LastEditorUserId="2444" LastEditDate="2020-11-05T22:20:06.800" LastActivityDate="2020-11-05T22:20:06.800" Title="Does importance sampling for off-policy estimation also apply to the case of negative rewards?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22235" PostTypeId="1" AcceptedAnswerId="22240" CreationDate="2020-06-29T02:07:29.890" Score="2" ViewCount="75" Body="&lt;p&gt;Why can't we during the first 1000 episodes allow our agent to perform only exploration?&lt;/p&gt;&#xA;&lt;p&gt;This will give a better chance of covering the entire space state. Then, after the number of episodes, we can then decide to exploit.&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-06-29T12:14:43.843" LastActivityDate="2020-06-29T12:14:43.843" Title="Why is 100% exploration bad during the learning stage in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22236" PostTypeId="2" ParentId="18840" CreationDate="2020-06-29T02:10:06.720" Score="0" Body="&lt;p&gt;I ended up using a work around.&lt;/p&gt;&#xA;&lt;p&gt;I set up the network so that an C x C (i.e. 320 x 320) input would output a C x C mask for some constant C (in my case it was 320).&lt;/p&gt;&#xA;&lt;p&gt;I then resized the image I wanted to pass in to C x C, and then resized the output back to the original size of the Image.&lt;/p&gt;&#xA;" OwnerUserId="27240" LastActivityDate="2020-06-29T02:10:06.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22238" PostTypeId="1" CreationDate="2020-06-29T03:06:02.073" Score="1" ViewCount="60" Body="&lt;p&gt;In section 6.5.6 of the book &lt;a href=&quot;https://www.deeplearningbook.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; by Ian et. al. general backpropagation algorithm is described as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The back-propagation algorithm is very simple. To compute the gradient of some&#xA;scalar z with respect to one of its ancestors x in the graph, we begin by observing&#xA;that the gradient with respect to z is given by dz = 1. We can then compute dz&#xA;the gradient with respect to each parent of z in the graph by multiplying the current gradient by the Jacobian of the operation that produced z. We continue multiplying by Jacobians traveling backwards through the graph in this way until we reach x. For any node that may be reached by going backwards from z through two or more paths, we simply sum the gradients arriving from different paths at that node.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To be specific I don't get this part:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We can then compute dz the gradient with respect to each parent of z in the graph by multiplying the current gradient by the Jacobian of the operation that produced z.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can anyone help me understand this with some illustration? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="31749" LastEditDate="2020-06-30T03:04:58.893" LastActivityDate="2020-06-30T03:04:58.893" Title="I need help understanding general back propagation algorithm" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22240" PostTypeId="2" ParentId="22235" CreationDate="2020-06-29T07:53:26.070" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Why can’t we during the first 1000 episodes allow our agent perform only exploration&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can do this. It is fine to do so either to learn the value function of a simple random policy, or when performing off-policy updates. It is quite normal when learning an environment from scratch in a safe way - e.g. in simulation - to collect an initial set of data from behaving completely randomly. The amount of this random data varies, and usually the agent will not switch from fully random to fully deterministic based on calculated values as a single step, but will do so gradually.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;this will give a better chance of covering the entire space state&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That will depend on the nature of the problem. For really simple problems, it may explore the space sufficiently to learn from. However, for many problems it is just a starting point, and not sufficient to cover parts of the space that are of interest in optimal control.&lt;/p&gt;&#xA;&lt;p&gt;When behaving completely randomly, the agent may take a very long time to complete an episode, and may never complete its first episode. So you could be waiting for a long time to collect data for the first 1000 such episodes. An example of this sort of environment would be a large maze - the agent will move back and forth in the maze, revisiting same parts again and again, where in theory it could already be learning not to repeat its mistakes.&lt;/p&gt;&#xA;&lt;p&gt;In some environments, behaving completely randomly will result in early failure, and never experiencing postive rewards that are available in the environment. An example of this might be a robot learning to balance on a tightrope and get from one end to the other. It would fall off after a few random actions, gaining very little knowledge for 1000 episodes.&lt;/p&gt;&#xA;&lt;p&gt;The state space coverage you are looking for ideally should include the optimal path through the space - at least at some point during learning (not necessarily the start). This does not have to appear in one single perfect episode, because the update rules for value functions in reinforcement learning (RL) will eventually allocate the correct values and find this optimal path in the data. However, the collected data does need to include the information about this optimal path amongst all the alternatives so that the methods in RL can evaluate and select it. In simple environments acting randomly may be enough to gain this data, but becomes highly unlikely when the environments are more complex.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;then after the number of episodes, we can then decide to exploit&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Again this might work for very simple environments, where you have collected enough information through acting randomly to construct a useful value function. However, if acting randomly does not find enough of the optimal path, then the best that exploitation can do is find some local optimum based on the data that was collected.&lt;/p&gt;&#xA;&lt;p&gt;I suggest you experience this difference for yourself: Set up a toy example environment, and use it to compare different approaches for moving between pure exploration and pure exploitation. You will want to run many experiments (probably 100s for each combination, averaged) to smooth out the randomness, and you can plot the results to see how well each approach learns - e.g. how many time steps (count time steps, not episodes, if you are interested in sample efficiency) it takes for the agent to learn, and whether or not it actually finds the correct optimal behaviour. Bear in mind that the specific results will only apply in your selected environment - so you might also want to do this comparison on a small range of environments.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-06-29T08:13:05.090" LastActivityDate="2020-06-29T08:13:05.090" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22241" PostTypeId="1" CreationDate="2020-06-29T10:34:26.620" Score="0" ViewCount="36" Body="&lt;p&gt;I've written a Double DQN which can do either 1-step or multi-step learning. It also has a prioritised reply buffer. The internal network is an LSTM.&lt;/p&gt;&#xA;&lt;p&gt;My input data is a series of time series data and other features derived from them.&lt;/p&gt;&#xA;&lt;p&gt;I feed this model with a long set of time series data for training and let the trained model make its decisions on an independent test data set. I run this same test many times.&lt;/p&gt;&#xA;&lt;p&gt;The observation is that the model doesn't give consistent performance according to my measurement metric. Sometimes it performs better than a benchmark, other times it gives out downright wrong actions.&lt;/p&gt;&#xA;&lt;p&gt;I can only think of the following two possible causes of this problem:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The model has not been trained thoroughly enough. i.e. I haven't gone&#xA;through enough number of episodes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My current exploration epsilon is 0.1. It's too big and the model 'explores' too often.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I can test these theories, but, to tell the truth, I am not entirely convinced they are the reasons my model is not consistent.&lt;/p&gt;&#xA;&lt;p&gt;I would like to hear some opinions and advice.&lt;/p&gt;&#xA;" OwnerUserId="37615" LastEditorUserId="2444" LastEditDate="2020-06-29T22:29:21.557" LastActivityDate="2020-06-29T22:29:21.557" Title="Why doesn't my double deep Q network trained with the same training set give consistent performance?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;long-short-term-memory&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22242" PostTypeId="1" CreationDate="2020-06-29T12:42:35.660" Score="1" ViewCount="92" Body="&lt;p&gt;DQN for Atari takes considerable training time. For example, the 2015 &lt;a href=&quot;https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; in Nature notes that algorithms are trained for 50 million frames or equivalently around 38 days of game experience in total. One reason is that DQN for image data typically uses a CNN, which is costly to train.&lt;/p&gt;&#xA;&lt;p&gt;However, the main purpose of a CNN is to extract the image features. Note that the policy for DQN is represented by a CNN and an output layer equal to the number of discrete actions. Is it possible to use a pretrained DQN to accelerate the training process by fixing the weights of the underlying pretrained CNN, resetting the weights of the output layer, and then running another (possibly different) DQN algorithm to relearn the weights of the output layer? Both DQN algorithms would be run on the same underlying environment.&lt;/p&gt;&#xA;" OwnerUserId="31984" LastEditorUserId="37607" LastEditDate="2020-06-30T22:33:02.587" LastActivityDate="2020-06-30T22:33:02.587" Title="Atari Games: Pretrained CNN to accelerate training?" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22243" PostTypeId="1" CreationDate="2020-06-29T14:15:25.977" Score="1" ViewCount="118" Body="&lt;p&gt;I know that &lt;span class=&quot;math-container&quot;&gt;$G_t = R_{t+1} + G_{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt; and the reward sequence is &lt;span class=&quot;math-container&quot;&gt;$R_1 = 2$&lt;/span&gt; followed by an infinite sequence of &lt;span class=&quot;math-container&quot;&gt;$7$&lt;/span&gt;s. What is the value of &lt;span class=&quot;math-container&quot;&gt;$G_0$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;As it's infinite, how can we deduce the value of &lt;span class=&quot;math-container&quot;&gt;$G_0$&lt;/span&gt;? I don't see the solution. It's just &lt;span class=&quot;math-container&quot;&gt;$G_0 = 5 + 0.9*G_1$&lt;/span&gt;. And we don't know &lt;span class=&quot;math-container&quot;&gt;$G_1$&lt;/span&gt; value, and we don't know &lt;span class=&quot;math-container&quot;&gt;$R_2, R_3, R_4, ...$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="38264" LastEditorUserId="2444" LastEditDate="2020-06-30T12:06:07.967" LastActivityDate="2020-06-30T12:06:07.967" Title="How do I calculate the return given the discount factor and a sequence of rewards?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;rewards&gt;&lt;return&gt;&lt;discount-factor&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22244" PostTypeId="2" ParentId="22243" CreationDate="2020-06-29T14:42:41.703" Score="2" Body="&lt;p&gt;You know all the rewards. They're 5, 7, 7, 7, and 7s forever. The problem now boils down to essentially a geometric series computation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;G_0 = R_0 + \gamma G_1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;G_0 = 5 + \gamma\sum_{k=0}^\infty 7\gamma^k &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;G_0 = 5 + 7\gamma\sum_{k=0}^\infty\gamma^k &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;G_0 = 5 + \frac{7\gamma}{1-\gamma} = \frac{5 + 2\gamma}{1-\gamma}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-29T14:42:41.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22245" PostTypeId="2" ParentId="22243" CreationDate="2020-06-29T14:44:20.283" Score="1" Body="&lt;p&gt;There are a few ways to resolve values of infinite sums. In this case, we can use a simple technique of self-reference to create a solvable equation.&lt;/p&gt;&#xA;&lt;p&gt;I will show how to do it for the generic case here of an MDP with same reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; on each timestep:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t = \sum_{k=0}^{\infty} \gamma^k r$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can &amp;quot;pop off&amp;quot; the first item:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t = r + \sum_{k=1}^{\infty} \gamma^k r$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then we can note that the second term is just &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; times the orginal term:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t = r + \gamma G_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(There are situations where this won't work, such as when &lt;span class=&quot;math-container&quot;&gt;$\gamma \ge 1$&lt;/span&gt; - essentially we are taking advantage that the high order terms are arbitrarily close to zero, so can be ignored)&lt;/p&gt;&#xA;&lt;p&gt;Re-arrange it again:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t = \frac{r}{1 - \gamma}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This means that you can get the value for a return for the discounted sum of repeating rewards. Which allows you to calculate your &lt;span class=&quot;math-container&quot;&gt;$G_1$&lt;/span&gt;. I will leave that last part as an execrise for you, as you already figured the first part out.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-06-29T14:44:20.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22246" PostTypeId="1" AcceptedAnswerId="22248" CreationDate="2020-06-29T16:33:07.773" Score="2" ViewCount="49" Body="&lt;p&gt;I'm using the DQN algorithm to train my agent to play a turn-based game. The winner of the game can be known before the game is over. Once the winning condition is satisfied, it cannot be reverted. For example, the game might last 100 turns, but it's possible to know that one of the players won at move 80, because some winning condition was satisfied. The last 20 moves don't change the outcome of the game. If people were playing this game, they, would play it to the very end, but the agent doesn't have to.&lt;/p&gt;&#xA;&lt;p&gt;The agent will be using memory replay to learn from the experience. I wonder, is it helpful for the agent to have the experiences after the winning condition was satisfied for a more complete picture? Or is it better to terminate the game immediately, and why? How would this affect agent's learning?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="38076" LastEditDate="2020-06-29T19:50:42.130" LastActivityDate="2020-06-29T20:06:13.077" Title="Should the agent play the game until the end or until the winner is found?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22247" PostTypeId="1" CreationDate="2020-06-29T16:59:06.117" Score="1" ViewCount="65" Body="&lt;p&gt;I am still new to CNNs, but I would like to check my understanding between when to use convolutional layers versus fully connected layers.&lt;/p&gt;&#xA;&lt;p&gt;From what I have read, we can use convolutional layers with filters, rather than fully connected layers, with images, text, and audio. However, with regular data, for example, the iris dataset, a convolutional layer would not perform well because of the structure. As in the columns can be swapped, yet the record or sample itself does not change. For example we can swap the order of the Petal Length column with Petal Width and the record does not change. Where as in an image or audio, changing the column items would result in a different image or audio file.&lt;/p&gt;&#xA;&lt;p&gt;These convolutional layers are &lt;em&gt;&amp;quot;better&amp;quot;&lt;/em&gt; for images and audio because not all the features need to connect to the next layer. For example, we do not need the background of a car image to know it is a car, thus we do not need all the connections and we save computational costs.&lt;/p&gt;&#xA;&lt;p&gt;Is this the right way to think about when to use convolutional layers versus fully connected layers?&lt;/p&gt;&#xA;" OwnerUserId="38265" LastEditorUserId="38265" LastEditDate="2020-06-30T15:34:05.900" LastActivityDate="2020-06-30T15:34:05.900" Title="When to use convolutional layers as opposed to fully connected layers?" Tags="&lt;convolutional-neural-networks&gt;&lt;fully-connected-layer&gt;&lt;convolutional-layers&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22248" PostTypeId="2" ParentId="22246" CreationDate="2020-06-29T20:06:13.077" Score="1" Body="&lt;p&gt;You should probably grant reward at the point that the game is logically won. This will help the agent learn more efficiently, by reducing the number of timesteps over which return values need to be backed up.&lt;/p&gt;&#xA;&lt;p&gt;Stopping the episode at that point should also be fine, and may add some efficiency too, in that there will be more focused relevant data in the experience replay. It seems like on the surface that there is no benefit to exploring or discovering any policy &lt;em&gt;after&lt;/em&gt; the game is won, and from the comments no expectation from you as agent developer that the agent has any kind of behaviour - random actions would be fine.&lt;/p&gt;&#xA;&lt;p&gt;It is still possible that the agent could learn more from play &lt;em&gt;after&lt;/em&gt; a winning state. It would require certain things to be true about the environment and additional work from you as developer.&lt;/p&gt;&#xA;&lt;p&gt;For example, if the game has an end phase where a certain kind of action is more common and it gains something within the game (&amp;quot;victory points&amp;quot;, &amp;quot;gold&amp;quot; or some other numbered token that is part of the game mechanics and could be measured), then additional play where this happened could be of interest. Especially if the moves that gained this measure could also be part of winning moves in the earlier game. To allow the agent to learn this though, it would have to be something that it &lt;em&gt;predicted&lt;/em&gt; in addition to winning or losing.&lt;/p&gt;&#xA;&lt;p&gt;One way to achieve this is to have a secondary learning system as part of the agent, that learns to predict gains (or totals) of this resource. Such a prediction could either be learned separately (but very similarly to the action value) and fed into the q function as an input, or it could be a neural network that shares early layers with the q function (or policy function) but with a different head. Adding this kind of secondary function to the neural network can also have a regularising effect on the network, because the interim features have to be good for two types of prediction.&lt;/p&gt;&#xA;&lt;p&gt;You definitley do not &lt;em&gt;need&lt;/em&gt; to consider such an addition. It could be a lot more work. However, for some games it is possible that it helps. Knowing the game, and understanding whether there is any learning experience to be had as a human player to play on beyond winning or losing, might help you decide whether looking into trying to replicate this additional experience for a bot. Even if it works, the effect may be minimal and not worth the difference it makes. For instance running a more basic learning agent for more episodes may still result in a very good agent for the end game. That only costs you more run time for training, not coding effort.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-06-29T20:06:13.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22249" PostTypeId="1" CreationDate="2020-06-29T22:31:52.820" Score="1" ViewCount="68" Body="&lt;p&gt;I need help in understanding something basic.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://www.youtube.com/watch?v=ay3zYUeuyhU&amp;amp;list=PLkDaE6sCZn6Gl29AoE31iwdVwSG-KnDzF&amp;amp;index=11&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt;, Andrew Ng says, essentially, that convolutional layers are better than fully connected (FC) layers because they use fewer parameters.&lt;/p&gt;&#xA;&lt;p&gt;But I'm having trouble seeing when FC layers would/could ever be used for what convolutional layers are used for, specifically, feature detection.&lt;/p&gt;&#xA;&lt;p&gt;I always read that FC layers are used in the final, classification stages of a CNN, but could they ever be used for the feature detection part?&lt;/p&gt;&#xA;&lt;p&gt;In other words, is it even possible for a &amp;quot;feature&amp;quot; to be deciphered when the filter size is the same as the entire image?&lt;/p&gt;&#xA;&lt;p&gt;If not, it's hard for me to understand Andrew Ng's comparison---there aren't any parameter reduction &amp;quot;savings&amp;quot; if we're not going to use an FC &amp;quot;filter&amp;quot; in place of a CNN layer in the first place.&lt;/p&gt;&#xA;&lt;p&gt;A semi-related question: Can multi-layer perceptrons (which I understand to be fully connected neural networks) be used for feature detection? If so, how do image-sized &amp;quot;filters&amp;quot; make any sense?&lt;/p&gt;&#xA;" OwnerUserId="38271" LastEditorUserId="2444" LastEditDate="2020-06-29T22:41:00.777" LastActivityDate="2021-03-27T01:02:45.240" Title="Can fully connected layers be used for feature detection?" Tags="&lt;convolutional-neural-networks&gt;&lt;fully-connected-layer&gt;&lt;convolutional-layers&gt;&lt;feature-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22250" PostTypeId="5" CreationDate="2020-06-29T22:35:01.940" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-29T22:35:01.940" LastActivityDate="2020-06-29T22:35:01.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22251" PostTypeId="4" CreationDate="2020-06-29T22:35:01.940" Score="0" Body="For questions related to fully connected layers (which are often used as the last layers of a convolutional neural network)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-29T22:35:01.940" LastActivityDate="2020-06-29T22:35:01.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22252" PostTypeId="5" CreationDate="2020-06-29T22:35:38.797" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-29T22:35:38.797" LastActivityDate="2020-06-29T22:35:38.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22253" PostTypeId="4" CreationDate="2020-06-29T22:35:38.797" Score="0" Body="For questions related to convolutional layers, which are layers that perform the convolution (or cross-correlation) operation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-29T22:35:38.797" LastActivityDate="2020-06-29T22:35:38.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22254" PostTypeId="5" CreationDate="2020-06-29T22:41:49.267" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-29T22:41:49.267" LastActivityDate="2020-06-29T22:41:49.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22255" PostTypeId="4" CreationDate="2020-06-29T22:41:49.267" Score="0" Body="For questions related to feature detection in computer vision." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-29T22:41:49.267" LastActivityDate="2020-06-29T22:41:49.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22257" PostTypeId="2" ParentId="22249" CreationDate="2020-06-29T23:18:56.460" Score="0" Body="&lt;p&gt;First of all, (FC) Neural Networks (NN) are universal function approximators. This means, that, in theory, there must exist some NN of appropriate size that is capable of doing what a CNN can do as well. The only difference is that the standard NN would have to be considerably larger than a corresponding CNN with the same capabilities. Each input pixel would probably have to map to a few dozens to hundreds of input nodes that are trained each on detecting some part of a distinct feature in their respective input image location. Consecutive, hidden layers than have to combine the parts extracted by the input layer for any given location and finally act as the true feature detectors, detecting features as compositions of values in the first layer.&lt;/p&gt;&#xA;&lt;p&gt;So, for each position in the input image separately, all the filters have to be learned separately. This absence of weight sharing (since filters are not shared across input locations, but have to be trained for each separate input location) makes using a standard NN for feature extraction so inefficient.&lt;/p&gt;&#xA;&lt;p&gt;However, in theory, also a standard NN could be trained on solving an image classification task, thereby learning to extract features. Actually, I would not be surprised if someone had tested that already on the MNIST dataset.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="2444" LastEditDate="2020-06-29T23:42:26.357" LastActivityDate="2020-06-29T23:42:26.357" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22258" PostTypeId="1" CreationDate="2020-06-30T01:01:59.440" Score="0" ViewCount="20" Body="&lt;p&gt;Has anyone tried writing a 3-body gravitation tracker (lander would be too much to ask) game to see how various AIs would handle a chaotic dynamical system?&lt;/p&gt;&#xA;" OwnerUserId="26053" LastActivityDate="2020-06-30T01:01:59.440" Title="3-Body Gravitation Lander Game?" Tags="&lt;control-theory&gt;&lt;benchmarks&gt;&lt;atari-games&gt;&lt;chaos&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22259" PostTypeId="1" CreationDate="2020-06-30T01:38:03.537" Score="0" ViewCount="70" Body="&lt;p&gt;I am intrigued with the idea of &lt;a href=&quot;https://www.lesswrong.com/posts/NfdHG6oHBJ8Qxc26s/the-zettelkasten-method-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zettelkasten&lt;/a&gt; but unsatisfied with the current implementations. It seems to me that a machine learning and NLP approach could be productive by helpfully identifying “important” keywords on which to links could be created, with learning to help narrow the selection of keywords over time.&lt;/p&gt;&#xA;&lt;p&gt;My problem is that it’s been 30 years since AI classes in grad school and things have moved on. I’m sure I could become an nlp expert with study but I don’t wanna. So I’m looking for guidance: what are the right terms to describe identifying keywords in context, ideally with some semantic content; how would I apply ML with my training to improve the keyword identification.&lt;/p&gt;&#xA;&lt;p&gt;I’d love references, ideas, and packages references. Python is preferred, but not strongly; I write most common (and many uncommon, SNOBOL and COBOL anyone?) languages so language isn’t all that much of an issue.&lt;/p&gt;&#xA;" OwnerUserId="38273" LastActivityDate="2020-07-01T00:59:04.517" Title="NLP Identifying important key words in a corpus" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;text-classification&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22260" PostTypeId="1" CreationDate="2020-06-30T06:19:05.297" Score="0" ViewCount="33" Body="&lt;p&gt;I am trying to solve a MDP problem with almost 50 states and 60 actions with Q-Learning or SARSA. However, the performance of both algorithms is terrible and cannot find the optimal policy found by backward induction. Does it mean that these algorithms do not converge to the optimal policies or there are some other problems? I run the algorithms for 100k, 500k and 1m episodes but I could not find the optimal policy.&lt;/p&gt;&#xA;" OwnerUserId="10191" LastActivityDate="2020-06-30T06:19:05.297" Title="Why Q-Learning and SARSA have terrible performance?" Tags="&lt;q-learning&gt;&lt;markov-decision-process&gt;&lt;sarsa&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22262" PostTypeId="1" CreationDate="2020-06-30T08:32:22.240" Score="0" ViewCount="34" Body="&lt;p&gt;Is it possible to create a named entity recognition system without using POS tagging in the corpus?&lt;/p&gt;&#xA;" OwnerUserId="36258" LastEditorUserId="2444" LastEditDate="2020-06-30T12:07:08.767" LastActivityDate="2021-03-28T04:03:35.390" Title="Is it possible to create a named entity recognition system without using POS tagging in the corpus?" Tags="&lt;natural-language-processing&gt;&lt;named-entity-recognition&gt;&lt;pos-tagging&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22263" PostTypeId="1" CreationDate="2020-06-30T10:30:48.303" Score="2" ViewCount="63" Body="&lt;p&gt;There is often interest in the results of machine learning algorithms, specifically because they came from machine learning algorithms -- as opposed to interest in the results in and of itself. It seems similar to the 'from the mouths of babes' trope, where comments made by children are sometimes regarded as containing some special wisdom, due to the honest and innocent nature of their source. Similarly, people seem to think that the impassionate learning of a machine might extract some special insight from a data set.&lt;/p&gt;&#xA;&lt;p&gt;(Of course, anyone with such opinions has obviously never met either a machine learning algorithm or a child.)&lt;/p&gt;&#xA;&lt;p&gt;Has this effect been discussed or studied anywhere? Does it have a name?&lt;/p&gt;&#xA;" OwnerUserId="38253" LastEditorUserId="1641" LastEditDate="2021-02-03T17:51:37.280" LastActivityDate="2021-02-03T17:51:44.337" Title="Studies on interest in results from ML purely due to use of ML" Tags="&lt;machine-learning&gt;&lt;resource-request&gt;&lt;ethics&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22264" PostTypeId="5" CreationDate="2020-06-30T12:08:29.160" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-30T12:08:29.160" LastActivityDate="2020-06-30T12:08:29.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22265" PostTypeId="4" CreationDate="2020-06-30T12:08:29.160" Score="0" Body="For questions related to part-of-speech tagging (POS tagging or PoS tagging or POST), which is the process of marking up a word in a text (corpus) as corresponding to a particular part of speech, based on both its definition and its context." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-06-30T12:08:29.160" LastActivityDate="2020-06-30T12:08:29.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22267" PostTypeId="1" CreationDate="2020-06-30T13:32:12.963" Score="1" ViewCount="33" Body="&lt;p&gt;Does someone know a method to estimate / measure the total energy consumption during the test phase of the well-known CNN models? So with a tool or a power meter...&lt;/p&gt;&#xA;&lt;p&gt;MIT has already a tool to estimate the energy consumption but it only works on AlexNet and GoogleNet, I need something for more Architectures (VGG, MobileNet, ResNet...). And I also need a metric to evaluate the well-known architects in terms of energy consumption. So at first estimate or measure the energy consumption and then evaluate the results with a good metric.&lt;/p&gt;&#xA;&lt;p&gt;With a measuring device I would measure the power consumption before using the CNN and I will repeat this experiment a few times then I will average the results, and do the same thing while using the CNN and at the end I will compare the results. But I have three problems here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1. how can I know that nothing else is running on the PC that also consumes energy while using the CNN?&#xA;2. how can I increase the accuracy of the measurements?&#xA;3. I don't find any power meter that measures the energy consumption in short periodes (1s).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thats why I prefer a tool to estimate the energy consumption, the accuracy of the measurments will not be that good but I didn't find any another tool..&lt;/p&gt;&#xA;&lt;p&gt;Does someone have an Idea, papers, sites that can help me?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks in advance for your reply!&lt;/p&gt;&#xA;" OwnerUserId="38282" LastActivityDate="2020-06-30T13:32:12.963" Title="How to measure/estimate the energy consumption of CNN models during testing?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;pretrained-models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22268" PostTypeId="1" CreationDate="2020-06-30T13:37:32.867" Score="2" ViewCount="77" Body="&lt;p&gt;The associative property of multidimensional discrete convolution says that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y=(x \circledast h_1) \circledast h_2=x\circledast(h_1\circledast h_2)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; are the filters and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the input.&lt;/p&gt;&#xA;&lt;p&gt;I was able to do exploit this property in Keras with &lt;code&gt;Conv2D&lt;/code&gt;: first, I convolve &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt;, then I convolve the result with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (i.e. the rightmost part of the equation above).&lt;/p&gt;&#xA;&lt;p&gt;Up to this point, I don't have any problem, and I also understand that convolution is linear.&lt;/p&gt;&#xA;&lt;p&gt;The problem is when two Conv2D layers have a non-linear activation function after the convolution. For example, consider the following two operations&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y_1=\text{ReLU}(x \circledast h_1)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Y_2=\text{ReLU}(Y_1\circledast h_2)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is possible to apply the associative property if the first or both layers have a non-linear activation function (in the case above ReLU, but it could be any activation function)? I don't think so. Any idea or related paper or some kind of approach?&lt;/p&gt;&#xA;" OwnerUserId="38283" LastEditorUserId="2444" LastEditDate="2020-07-01T01:59:22.770" LastActivityDate="2021-03-28T05:04:50.450" Title="Is it possible to apply the associative property of the convolution operation when it is followed by a non-linearity?" Tags="&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22269" PostTypeId="1" CreationDate="2020-06-30T13:40:24.267" Score="3" ViewCount="202" Body="&lt;p&gt;I have studied linear algebra, probability, and calculus twice. But I don't understand how can I reach the level that I can read any AI paper and understand mathematical notation in it.&lt;/p&gt;&#xA;&lt;p&gt;What is your strategy when you see the mathematical expression that you can't understand?&lt;/p&gt;&#xA;&lt;p&gt;For example, in Wasserstein GAN article, there are many advanced mathematical notations. Also, some papers are written by people who have a master's in mathematics, and those people use advanced mathematics in some papers, but I have a CS background.&lt;/p&gt;&#xA;&lt;p&gt;When you come across this kind of problem, what do you do?&lt;/p&gt;&#xA;" OwnerUserId="27315" LastEditorUserId="2444" LastEditDate="2020-06-30T14:07:35.970" LastActivityDate="2020-07-01T15:54:19.340" Title="How can I read any AI paper?" Tags="&lt;research&gt;&lt;papers&gt;&lt;academia&gt;" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22270" PostTypeId="1" AcceptedAnswerId="24433" CreationDate="2020-06-30T14:05:35.657" Score="1" ViewCount="48" Body="&lt;p&gt;Is the &lt;a href=&quot;https://papers.nips.cc/paper/2016/file/cc7e2b878868cbae992d1fb743995d8f-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAIL&lt;/a&gt; applicable if the expert's trajectories (sample data) are for the same task but are in a different environment (modified but will not be completely different)?&lt;/p&gt;&#xA;&lt;p&gt;My gut feeling is, yes, otherwise we can just simply adopt behavioural cloning. Furthermore, since the expert's trajectories are from a different environment, the dimension/length of state-action pairs will most likely be different. Will those trajectories still be useful for GAIL training?&lt;/p&gt;&#xA;" OwnerUserId="33419" LastEditorUserId="2444" LastEditDate="2020-11-05T23:45:15.043" LastActivityDate="2020-11-05T23:45:50.413" Title="Is GAIL applicable if the expert's trajectories are for the same task but are in a different environment?" Tags="&lt;reinforcement-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;imitation-learning&gt;&lt;gail&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22271" PostTypeId="2" ParentId="22269" CreationDate="2020-06-30T15:04:29.643" Score="2" Body="&lt;p&gt;I think the best way to make reading papers easier is to practice (as in, read lots of papers, try implementing them, etc), and to discuss them with other students/researchers.&lt;/p&gt;&#xA;&lt;p&gt;Sometimes it's tough to avoid some obscure or really technical math, so you may just need to do extra reading. The Wasserstein metric, for example, is used a lot in ML but I kinda doubt most ML researchers have a good understanding of it. This metric comes from a branch of math called &amp;quot;optimal transportation theory&amp;quot;, which is super interesting, but very real-analysis-heavy. If you're really interested in learning about the Wasserstein metric, I recommend Cedric Villani's book &amp;quot;Optimal Transport: Old and New&amp;quot;. I also recommend &lt;a href=&quot;https://arxiv.org/abs/1705.10743&quot; rel=&quot;nofollow noreferrer&quot;&gt;this awesome paper&lt;/a&gt;. Nevertheless, learning analysis is likely gonna serve you very well for understanding a wide range of ML papers.&lt;/p&gt;&#xA;&lt;p&gt;Finally, as a beginning grad student, I have experienced your issue as well. I made a tool to help me with this at &lt;a href=&quot;https://github.com/harwiltz/sagerank&quot; rel=&quot;nofollow noreferrer&quot;&gt;this repo&lt;/a&gt;, which manages a library of papers you're interested in. It then uses a PageRank algorithm to recommend new papers to you that are commonly referred to by the papers you want to read, with the goal of helping you read up on the foundational &amp;quot;prerequisite&amp;quot; material.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-06-30T15:04:29.643" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22273" PostTypeId="2" ParentId="22201" CreationDate="2020-06-30T18:57:13.583" Score="3" Body="&lt;p&gt;The primary issue I see is that in the loop through time steps &lt;code&gt;t&lt;/code&gt; in every training episode, you select &lt;strong&gt;actions for both players&lt;/strong&gt; (who should have &lt;strong&gt;opposing goals&lt;/strong&gt; to each other), but update &lt;strong&gt;a single &lt;code&gt;q_table&lt;/code&gt;&lt;/strong&gt; (which can only ever be correct for the &amp;quot;perspective&amp;quot; of one of your two players) on &lt;strong&gt;both of those actions&lt;/strong&gt;, and updating both of them using &lt;strong&gt;a single, shared reward function&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, I guess this means that your learning algorithm assumes that your opponent will always be helping you win, rather than assuming that your opponent plays optimally towards its own goals. You can see that this is likely indeed the case from your plot; you use &lt;span class=&quot;math-container&quot;&gt;$30,000$&lt;/span&gt; training episodes, split up into &lt;span class=&quot;math-container&quot;&gt;$15$&lt;/span&gt; chunks of &lt;span class=&quot;math-container&quot;&gt;$2,000$&lt;/span&gt; episodes per chunk for your plot. In your plot, you also very quickly reach a score of about &lt;span class=&quot;math-container&quot;&gt;$1,950$&lt;/span&gt; per chunk, which is almost the maximum possible! Now, I'm not 100% sure what the winrate of an optimal player against random would be, but I think it's likely that that should be lower than 1950 out of 2000. Random players will occasionally achieve draws in Tic-Tac-Toe, especially taking into consideration that your learning agent itself is also not playing optimally (but &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedily)!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You should instead pick one of the following solutions (maybe there are more solutions, this is just what I come up with on the spot):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Keep track of &lt;strong&gt;two different tables&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values for the two different players, and update each of them only on &lt;strong&gt;half of the actions&lt;/strong&gt; (each of them pretending that actions selected by the opponent are just stochastic state transitions created by &amp;quot;the environment&amp;quot; or &amp;quot;the world&amp;quot;). See &lt;a href=&quot;https://ai.stackexchange.com/a/10033/1641&quot;&gt;this answer for more on what these scheme would look like&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Only keep track of a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value for your own agent (again only updating it on half the actions as described above -- specifically only on the actions your agent actually selected). Actions by the opposing player should then NOT be selected based on those same &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values, but instead by some different approach. You could for instance have opposing actions selected by a minimax or alpha-beta pruning search algorithm. Maybe selecting them to &lt;strong&gt;minimise&lt;/strong&gt; instead of &lt;strong&gt;maximise&lt;/strong&gt; values from the same &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-table could also work (didn't think this idea fully through, not 100% sure). You probably could also just pick opponent actions randomly, but then your agent will only learn to play well against random opponents, not necessarily against strong opponents.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;After looking into the above suggestions, you'll probably also want to look into making sure that your agent experiences games in which it starts as Player 1, as well as games in which it starts as Player 2, and trains for both of those possible scenarios and learns how to handle both of them. In your evaluation code (after training), I believe that you always make the Random opponent play first, and the trained agent play second? If you don't cover this scenario in your training episodes, your agent may not learn how to properly handle it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Finally, a couple of small notes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Your discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma 0.1$&lt;/span&gt; has an extremely small value. Common values in literature are values like &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.95$&lt;/span&gt;, or even &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.99$&lt;/span&gt;. Tic-Tac-Toe episodes tend to always be very short anyway, and we tend to not care too much about winning quickly rather than winning slowly (a win's a win), so I would tend to use a high value like &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.99$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;A small programming tip, not really AI-specific: your code contains various conditions of the form &lt;code&gt;if &amp;lt;condition&amp;gt; == True :&lt;/code&gt;, like: &lt;code&gt;if done == True :&lt;/code&gt;. The &lt;code&gt;== True&lt;/code&gt; part is redundant, and these conditions can be written more simply as just &lt;code&gt;if done:&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-06-30T18:57:13.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22274" PostTypeId="1" CreationDate="2020-06-30T21:20:54.733" Score="0" ViewCount="32" Body="&lt;p&gt;I found a paper titled &lt;a href=&quot;https://ieeexplore.ieee.org/document/7532765&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model&lt;/a&gt;. I am curious about how can Kneser-Ney Smoothing technique be integrated into a feed-forward neural language model with one linear hidden layer and a softmax activation. What is the purpose of the Kneser-Ney in such a neural network, and how can it be used for learning the conditional probability for the next word?&lt;/p&gt;&#xA;" OwnerUserId="38292" LastActivityDate="2020-06-30T21:20:54.733" Title="How can Kneser-Ney Smoothing be integrated into a neural language model?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;conditional-probability&gt;&lt;language-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22275" PostTypeId="1" CreationDate="2020-06-30T21:24:44.203" Score="1" ViewCount="29" Body="&lt;p&gt;I was reading the following paper: &lt;a href=&quot;http://cwnlab.eecs.ucf.edu/wp-content/uploads/2019/12/2019_MLSP_ANCS_NAZMUL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rl-Ncs: Reinforcement Learning Based Data-Driven Approach For Nonuniform Compressed Sensing&lt;/a&gt;, and my question is: how do they decide whether a signal is characterized as a region of interest coefficient or non-region of interest coefficient?&lt;/p&gt;&#xA;" OwnerUserId="37947" LastEditorUserId="2444" LastEditDate="2020-07-01T16:40:55.657" LastActivityDate="2020-07-01T16:40:55.657" Title="How are the coefficients of the Region of Interest being selected?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22276" PostTypeId="1" CreationDate="2020-06-30T21:50:35.177" Score="1" ViewCount="31" Body="&lt;p&gt;I am reading about GANs. I understand that GANs learn implicitly the probability distribution that generated the data. However, at the input we give a random noise vector. It seems that we can sample that random noise vector from whatever distribution we want.&lt;/p&gt;&#xA;&lt;p&gt;My question is: given that there is ONLY ONE possible distribution that could have generated the data, and that our GAN is trying to approximate that distribution, can I think that the GAN also learns how to map between that distribution that it needs to learn and the distribution from which we sample the random noise vector?&lt;/p&gt;&#xA;&lt;p&gt;I am thinking about this, as the random noise vector can be sampled from whatever distribution we want, so it can be different each time we start to train, so it can vary, but the GAN needs to be able to still imitate one unique distribution, so, in a way, it needs to be able to adapt to the distribution from which the noise comes.&lt;/p&gt;&#xA;" OwnerUserId="37919" LastEditorUserId="2444" LastEditDate="2020-07-01T00:50:49.080" LastActivityDate="2020-07-01T00:50:49.080" Title="Do GANs also learn to map between the distribution from which the random noise is sampled and the true distribution of the data?" Tags="&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22277" PostTypeId="2" ParentId="22231" CreationDate="2020-06-30T21:58:19.827" Score="0" Body="&lt;p&gt;The question was also asked in Pytorch Forums, where it has been sort of resolved and would love to have further discussions over there.&lt;/p&gt;&#xA;&lt;p&gt;Link to Pytorch thread: &lt;a href=&quot;https://discuss.pytorch.org/t/how-is-it-possible-to-get-the-output-size-of-n-consecutive-convolutional-layers/87300&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://discuss.pytorch.org/t/how-is-it-possible-to-get-the-output-size-of-n-consecutive-convolutional-layers/87300&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37180" LastActivityDate="2020-06-30T21:58:19.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22278" PostTypeId="1" CreationDate="2020-06-30T22:48:14.683" Score="0" ViewCount="21" Body="&lt;p&gt;I have some databases with information about exports and imports of a country, what are they importing, the times that an importation was made, the country from and target, everything related to Customs.&#xA;My goal is to create and A.I  which creates news based on the data, for example : &amp;quot;First time that country X exports item Y&amp;quot;, &amp;quot;Exports of item Z has been stopped since 1988&amp;quot;, &amp;quot;The main imported article of 2019 was Robots!&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;So, the problem is very complex, at first I have some functions that reads databases and returns some info like &amp;quot;Top exported item&amp;quot;, &amp;quot;First time exported item Y&amp;quot;, etc, later stel is to teach program to write a New. I used historical data to create some alerts from test database and then i'm using a listener to database to read every new record and compute with my functions if it generates some new information (new register, new country import, export, etc)&lt;/p&gt;&#xA;&lt;p&gt;Im stuck in the second part, and maybe my aproach is not the correct one. So, my question is &lt;strong&gt;How to deal with this problem?, how to teach to look for news in database&lt;/strong&gt;,&lt;/p&gt;&#xA;" OwnerUserId="38098" LastEditorUserId="38098" LastEditDate="2020-07-01T15:21:28.727" LastActivityDate="2020-07-01T15:21:28.727" Title="Looking for an Approach method for News maker" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;natural-language-understanding&gt;&lt;inference&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22279" PostTypeId="2" ParentId="17421" CreationDate="2020-07-01T00:31:43.213" Score="2" Body="&lt;p&gt;Perhaps you are getting checkerboard artifacts &lt;a href=&quot;https://distill.pub/2016/deconv-checkerboard/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explained here&lt;/a&gt;, solutions involve changing the kernel and stride size to prevent them from being not divisible. Besides that, a solution could be to apply Gaussian smoothing to minimize the artifacts.&lt;/p&gt;&#xA;&lt;p&gt;For example, using Gaussian smoothing in OpenCV with your image results in&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import cv2&#xA;img = cv2.imread('s.png') # I took a screen shot to see how it would look&#xA;blur = cv2.blur(img,(8,8))&#xA;plt.imshow(blur)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cBchd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cBchd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The artifact is gone using a kernel of size (8,8). I hope this can help&lt;/p&gt;&#xA;" OwnerUserId="38294" LastActivityDate="2020-07-01T00:31:43.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22280" PostTypeId="2" ParentId="22269" CreationDate="2020-07-01T00:38:38.833" Score="1" Body="&lt;p&gt;When I read papers in a new domain and when I started reading theoretical ML paper, I faced similar problems. I usually start with the introduction then related work and try to understand all the concepts and related papers cited that are relevant to understanding the paper.&lt;/p&gt;&#xA;&lt;p&gt;Specifically when it comes to difficult mathematical formulations, as @harwiltz said the more you read about it the easier it gets. There may be a set of papers with concepts that are similar to the paper you are reading but are well-explained I usually read them first (or if it is an important mathematical concept you can find some blogs describing the intuitions/basics behind it).&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-07-01T00:38:38.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22281" PostTypeId="2" ParentId="22262" CreationDate="2020-07-01T00:49:50.007" Score="0" Body="&lt;p&gt;Yes, both of them are different tasks. POS tagging helps NER systems but it is not necessary. You can get features (say BERT/ELMo embedding) for each word in the sentence and train a CRF NER model. This looks like simple example &lt;a href=&quot;https://www.pragnakalp.com/bert-named-entity-recognition-ner-tutorial-demo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pragnakalp.com/bert-named-entity-recognition-ner-tutorial-demo/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-07-01T00:49:50.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22282" PostTypeId="2" ParentId="22259" CreationDate="2020-07-01T00:59:04.517" Score="1" Body="&lt;p&gt;I am sure there are complex methods to extract keywords, but the standard one which should serve as a strong baseline is the RAKE graph algorithm &lt;a href=&quot;https://pypi.org/project/rake-nltk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pypi.org/project/rake-nltk/&lt;/a&gt;. It should work reasonably well in most text domains.&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-07-01T00:59:04.517" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22283" PostTypeId="2" ParentId="22268" CreationDate="2020-07-01T01:13:05.083" Score="0" Body="&lt;p&gt;Yes, when you have non-linearity it is not possible to combine your convolution steps.&lt;/p&gt;&#xA;&lt;p&gt;However, you can approximate the two layers network with one layer net according to the Universal Approximation Theorem. You will probably need to use something like a knowledge distillation technique to do it. Note that, the theorem doesn't say anything about the number of neurons required or if the learning techniques that we usually use will work well.&lt;/p&gt;&#xA;&lt;p&gt;Also, &lt;span class=&quot;math-container&quot;&gt;$ReLU(x)$&lt;/span&gt; is a linear mapping when &lt;span class=&quot;math-container&quot;&gt;$x \geq 0$&lt;/span&gt;, so if your input, weight, and biases are &lt;span class=&quot;math-container&quot;&gt;$\geq 0$&lt;/span&gt;, the net can be exactly modeled with a single layer network.&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-07-01T01:13:05.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22284" PostTypeId="2" ParentId="22269" CreationDate="2020-07-01T01:25:40.997" Score="1" Body="&lt;p&gt;From my experience (and I've been reading many research papers for a while), it's rare to find a research paper where you fully understand everything in one go, especially if the research paper was published or released recently or a very long time ago (because, back then, maybe people had a different writing style, used a different notation, or something like that), unless you are an expert on the topic, which is probably not the case, unless you are doing serious research on the topic (i.e. you're doing a Ph.D. and beyond; in that case, you probably don't need to ask questions on this site: hopefully, you have a qualified advisor to whom you can ask these questions!), or the paper is really easy and does not contain any formulas.&lt;/p&gt;&#xA;&lt;p&gt;Of course, if a paper is published, it must contain something novel, so that something novel could be one of the things that you need to spend some time to understand, but the hardest parts of a paper could also easily be the prerequisites (i.e. the concepts that the paper builds upon), because you may not have a very solid knowledge of those topics (as you probably have already experienced).&lt;/p&gt;&#xA;&lt;p&gt;There are at least three ways to proceed when you are stuck because you don't understand something&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you can ignore what you don't understand (i.e. you don't need it for your purposes because e.g. you just need to have a high-level understanding of the topics), ignore it (really!!)&lt;/li&gt;&#xA;&lt;li&gt;If it cannot be ignored (e.g. because you really need to know all the details of the paper because e.g. you need to give a presentation at your university), try to understand what you don't understand by picking up a resource on that topic that you don't understand, then read it; spend the time that you think is opportune (i.e. do not spend 6.5 days to understand a detail of a paper if you only have 7 days to read that paper and prepare a presentation or whatever you need to do)&lt;/li&gt;&#xA;&lt;li&gt;If you can afford it, stop reading that paper and go back to the basics.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In general, learning is not an easy process and, more specifically, reading research papers is not the easiest reading (because research papers are typically concise, i.e. there's a lot of information compression), so do not expect to understand everything of a paper in one go. In fact, the paper &lt;a href=&quot;https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Read a Paper&lt;/a&gt; by S. Keshav, which gives you some guidelines on how to read a paper, tells you to read a paper in three steps. For more details about these three steps, please, read the paper!&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-01T01:46:44.483" LastActivityDate="2020-07-01T01:46:44.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22285" PostTypeId="1" CreationDate="2020-07-01T01:57:16.453" Score="4" ViewCount="263" Body="&lt;p&gt;I am new to reinforcement learning. For my application, I have found out that if my reward function contains some negative and positive values, my model does not give the optimal solution, but the solution is not bad as it still gives positive reward at the end.&lt;/p&gt;&#xA;&lt;p&gt;However, if I just shift all readings by subtracting a constant until my reward function is all negative, my model can reach the optimal solution easily.&lt;/p&gt;&#xA;&lt;p&gt;Why is this happening?&lt;/p&gt;&#xA;&lt;p&gt;I am using DQN for my application.&lt;/p&gt;&#xA;&lt;p&gt;I feel that this is also the reason why the gym environment mountaincar-v0 uses &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for each time step and &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; at the goal, but correct me if I am wrong.&lt;/p&gt;&#xA;" OwnerUserId="38298" LastEditorUserId="2444" LastEditDate="2020-11-01T23:27:59.303" LastActivityDate="2020-11-01T23:27:59.303" Title="Why does shifting all the rewards have a different impact on the performance of the agent?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;rewards&gt;&lt;reward-shaping&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22286" PostTypeId="1" CreationDate="2020-07-01T02:34:30.487" Score="0" ViewCount="37" Body="&lt;p&gt;I just learnt the math behind neural networks so please bear with my ignorance. I wonder if there is a precise definition for DNN.&lt;/p&gt;&#xA;&lt;p&gt;Is it true that any neural network with more than 2 hidden layer can be named as a DNN, and training a NN with 2 hidden layers using Q-learning we are technically doing a type of deep reinforcement learning?&lt;/p&gt;&#xA;&lt;p&gt;PS: If it is conceptually that simple why do common people regard deep learning like something done by archmages in ivory towers.&lt;/p&gt;&#xA;" OwnerUserId="38299" LastActivityDate="2020-07-01T12:30:59.637" Title="A neural network with 2 or more hidden layers is a DNN?" Tags="&lt;neural-networks&gt;&lt;soft-question&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-07-01T12:29:23.790" ContentLicense="CC BY-SA 4.0" />
  <row Id="22288" PostTypeId="2" ParentId="22286" CreationDate="2020-07-01T04:23:24.090" Score="1" Body="&lt;p&gt;I don't think there is a fixed threshold that differentiates between Shallow and Deep Learning, but I would say that a 2 layer NN should not be considered deep. But now-a-days, almost all NN architectures are studied under the umbrella of Deep Learning.&lt;/p&gt;&#xA;&lt;p&gt;And yes, training a 2 hidden layers NN using Q-learning would technically mean doing deep RL.&lt;/p&gt;&#xA;&lt;p&gt;I guess it is conceptually simple but making NN perform optimally is an art. Tuning hyperparameters or debugging NN can be tough and one learns with experience. I guess others in the community would be much more suited to answer this question. But these were my 2 cents.&lt;/p&gt;&#xA;" OwnerUserId="36074" LastActivityDate="2020-07-01T04:23:24.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22289" PostTypeId="1" CreationDate="2020-07-01T07:24:41.207" Score="2" ViewCount="89" Body="&lt;p&gt;I know that there has been some discussion about this (e.g. &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/67gonq/d_batch_normalization_before_or_after_relu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://forums.fast.ai/t/correct-order-of-batchnorm-and-relu/41086&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), but I can't seem to find consensus.&lt;/p&gt;&#xA;&lt;p&gt;The crucial thing that I haven't seen mentioned in these discussions is that applying batch normalization before ReLU &lt;strong&gt;switches off half the activations&lt;/strong&gt;, on average. This may not be desirable.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the effect of batch normalization before ReLU is more than just &lt;em&gt;z-scaling&lt;/em&gt; activations.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, applying batch normalization after ReLU may feel unnatural because the activations are necessarily non-negative, i.e. not normally distributed. Then again, there's also no guarantee that the activations are normally distributed before ReLU clipping.&lt;/p&gt;&#xA;&lt;p&gt;I currently lean towards a preference to batch normalization after ReLU (which is also based on some empirical results).&lt;/p&gt;&#xA;&lt;p&gt;What do you all think? Am I missing something here?&lt;/p&gt;&#xA;" OwnerUserId="37751" LastEditorUserId="2444" LastEditDate="2020-07-01T16:43:23.927" LastActivityDate="2020-07-01T16:43:23.927" Title="Should batch normalisation be applied before or after ReLU?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;relu&gt;&lt;batch-normalization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22290" PostTypeId="2" ParentId="22285" CreationDate="2020-07-01T08:13:14.927" Score="1" Body="&lt;p&gt;You have some freedom to re-define reward schemes, whilst still describing the same goals for an agent. How this works depends to some degree on whether you are dealing with an episodic or continuing problem.&lt;/p&gt;&#xA;&lt;h2&gt;Episodic problems&lt;/h2&gt;&#xA;&lt;p&gt;An episodic problem ends, and once an agent reaches the terminal state, it is guaranteed zero rewards from that point on. The optimal behaviour can therefore depend quite critically on balance between positive and negative rewards.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If an environment contains many unavoidable negative rewards, and these outweigh total positive rewards, then the agent will be motivated to complete an episode sooner.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If an environment contains repeatable positive rewards, and these outweigh total negative rewards, then the agent will be motivated to loop through the postive rewards and not end the episode.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Scaling all rewards by the same positive factor makes no difference to the goals of an agent in an episodic problem. Adding a positive or negative offset to all rewards can make a difference though. It is likely to be most notable when such a change moves rewards from positive to negative or vice versa. In the MountainCar example, adding +2 to all rewards would mean the agent would gain +1 for each time step. As it would stop gaining any reward for reaching the goal, even though reaching that goal would score the highest possible +2.5 reward, the fact that this ends the episode means that it now becomes a poor choice. The best action for the car in this modified MountainCar is to stay at the bottom of the valley collecting the +1 reward per time step forever.&lt;/p&gt;&#xA;&lt;h2&gt;Continuing problems&lt;/h2&gt;&#xA;&lt;p&gt;In a continuing problem, there is no way for the agent to avoid the stream of new reward data. That means any positive scaling of all reward values or positive or negative offset, by the same amount, has no impact on what counts as the optimal policy. The calculated value of any state under the same policy, but with rewards all transformed with the same multiplier and offset, will be different, but the optimal policy in that environment will be the same.&lt;/p&gt;&#xA;&lt;p&gt;If you scale or offset rewards differently to each other, then that can change the goals of the agent and what the optimal policy is. The balance does not really depend on whether rewards are positive or negative in a continuing environment.&lt;/p&gt;&#xA;&lt;p&gt;There may be some exceptions to this for continuing problems when using a discount factor, and setting it relatively low (compared to the typical state &amp;quot;cycling&amp;quot; length in the problem). That can cause changes in behaviour due to offsets, similar to those seen in episodic problems. If you use an average reward setting this tends to be less relevant. Often in DQN, you will choose a high discount factor e.g. 0.99 or 0.999, and this will tend to behave close to an average reward setting provided rewards are not very sparse.&lt;/p&gt;&#xA;&lt;h2&gt;In general&lt;/h2&gt;&#xA;&lt;p&gt;In either case, if you change a reward system, and that results in an agent that consistently learns a different policy, that will usually mean one of two things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The original reward system was incorrect. It described a goal that you did not intend, or had &amp;quot;loopholes&amp;quot; that the agent could exploit to gain more reward in a way that you did not intend.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The implementation of the agent was sensitive in some way to absolute values of total reward. That could be due to a hyperparameter choice in something like a neural network for example, or maybe a bug.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Another possibility, that you may see if you only run a few experiments, is that the agent is not learning 100% consistently, and you are accidentally correlating your changes to reward scheme with the noise/randomness in the results. A DQN-based agent will usually have some variability in how well it solves a problem. After training, DQN is usually only approximately optimal, and by chance some approximations are closer than others.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-01T11:36:24.233" LastActivityDate="2020-07-01T11:36:24.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22291" PostTypeId="1" CreationDate="2020-07-01T08:34:14.240" Score="1" ViewCount="73" Body="&lt;p&gt;I know this is deceptively simple. Tic tac toe is a well studied game for RL.&lt;/p&gt;&#xA;&lt;p&gt;Assume your agent is playing aggainst a strong opponent.&lt;/p&gt;&#xA;&lt;p&gt;I know you deal in after states. I know that in Q learning the optimal policy should be converged on faster as the Q(S,A) is becoming closer to the optimal each step. While in SARSA the Q function will not be updated towards the optimal sometimes as it is exploring. If epislon is fixed SARSA will convereg to the epislon greedy policy.&lt;/p&gt;&#xA;&lt;p&gt;I came across the question above and I don't know the answer, is it that SARSA may play more conservatively? Opting for more draws rather than getting in board positions where one could is likely to either lose or win rather than draw.&lt;/p&gt;&#xA;" OwnerUserId="38303" LastActivityDate="2020-07-01T08:34:14.240" Title="Tic-tac-toe: How would standard SARSA and Q-learning yield different results in the agent's behaviour?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;sarsa&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22292" PostTypeId="1" CreationDate="2020-07-01T09:14:19.560" Score="0" ViewCount="56" Body="&lt;p&gt;I have been trying to understand how YOLO Darknet works, and for the most part reading the documentation and checking the code helps me understand. But when it comes to the weight file, I can't find the code where Darknet loads close to 250 MB of data.&lt;/p&gt;&#xA;&lt;p&gt;It seems to load 4 floats at the beginning (major, minor, revision and iseen) and then proceeds to iterate through all the layers, loading the respective weights and biases. I did the calculations and the number of MB it loads for the Yolov3.cfg file is close to 1 MB.&lt;/p&gt;&#xA;&lt;p&gt;Recently YoloV5 was released and they claim that their model weights 90% less than the Darknet model. This leads me to think that the reason the weights are so heavy has something to do with Darknet being written in C and CUDA.&lt;/p&gt;&#xA;&lt;p&gt;Does anybody know why the weights in Darknet are so ridiculously heavy? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="38041" LastActivityDate="2020-07-01T09:14:19.560" Title="Why are YOLO Darknet weights so heavy?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;yolo&gt;&lt;darknet&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22293" PostTypeId="1" CreationDate="2020-07-01T09:18:29.263" Score="0" ViewCount="84" Body="&lt;p&gt;I am trying to implement a Reinforcement Learning algorithm called &lt;a href=&quot;http://arxiv.org/pdf/1509.02971v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG&lt;/a&gt; in TensorFlow 2.x on a custom gym environment. I am new to TF. So, I started with the DDPG TF 1.x implementation from &lt;a href=&quot;https://github.com/pemami4911/deep-rl/blob/master/ddpg/ddpg.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;pemami4911&lt;/a&gt;. My TF 2.x implementation can be found in a &lt;a href=&quot;https://colab.research.google.com/drive/1QltD75hwesHixbUAMfETNUXAxB8J7Z-_?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;google-colab document&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For some reason, the agent would not learn. The output of the actor neural network is always +1 or -1 for the entire episode. My guess is that there is a mistake in one of the following&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the Actor and the Critic NN are not properly written&lt;/li&gt;&#xA;&lt;li&gt;For the Critic NN, there are two inputs. I would like one of them to skip the first layer. Todo this I used &lt;code&gt;Keras.layers.add&lt;/code&gt; (see lines &lt;a href=&quot;https://colab.research.google.com/drive/1QltD75hwesHixbUAMfETNUXAxB8J7Z-_#scrollTo=dP-FQ5tv5EA1&amp;amp;line=54&amp;amp;uniqifier=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;24-40&lt;/a&gt;). There might be a mistake in the implementation.&lt;/li&gt;&#xA;&lt;li&gt;I used Gradient tape to compute the gradient of the critic NN with respect to one of the inputs (Not the parameters). For this I use &lt;code&gt;var = tf.constant(input_actions)&lt;/code&gt; and Gradient Tape to compute the gradient using &lt;code&gt;self.tape.gradient(self.prediction, var)&lt;/code&gt;. This can be found at the line &lt;a href=&quot;https://colab.research.google.com/drive/1QltD75hwesHixbUAMfETNUXAxB8J7Z-_#scrollTo=dP-FQ5tv5EA1&amp;amp;line=54&amp;amp;uniqifier=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;54-57&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31445" LastEditorUserId="2444" LastEditDate="2020-07-01T15:22:52.030" LastActivityDate="2020-07-01T15:22:52.030" Title="Why is my DDPG agent (implemented in TensorFlow) not learning?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;deep-rl&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22294" PostTypeId="1" CreationDate="2020-07-01T14:48:41.870" Score="0" ViewCount="21" Body="&lt;p&gt;Is there a (large) indoor scene understanding datasets (providing instance segmentation masks) which can be used &lt;strong&gt;commercially&lt;/strong&gt; ?&lt;/p&gt;&#xA;&lt;p&gt;All large scene understanding datasets (SceneNet, ScanNet, InteriorNet, ...) cannot be used commercially.&#xA;An overview of datasets can be found at &lt;a href=&quot;https://arxiv.org/pdf/1803.03352.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1803.03352.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38308" LastActivityDate="2020-07-01T14:48:41.870" Title="Indoor scene understanding dataset which can be used commercially?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;&lt;scene-classification&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22295" PostTypeId="5" CreationDate="2020-07-01T15:07:17.063" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-01T15:07:17.063" LastActivityDate="2020-07-01T15:07:17.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22296" PostTypeId="4" CreationDate="2020-07-01T15:07:17.063" Score="0" Body="For questions related to reward shaping, which is a technique where supplemental rewards are provided to make a problem easier to learn. In general, there is usually an obvious natural reward for any problem. For games, this is usually a win or loss. For financial problems, the reward is usually profit. Reward shaping augments the natural reward signal by adding additional rewards for making progress toward a good solution." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-01T15:07:17.063" LastActivityDate="2020-07-01T15:07:17.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22297" PostTypeId="5" CreationDate="2020-07-01T15:15:47.260" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-01T15:15:47.260" LastActivityDate="2020-07-01T15:15:47.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22298" PostTypeId="4" CreationDate="2020-07-01T15:15:47.260" Score="0" Body="For questions related to inverse reinforcement learning (IRL), the problem of recovering the reward function from the observed behavior (or policy) of an agent. It's called IRL because it's the inverse problem of RL, i.e. the problem of finding optimal policies given the reward function." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-01T15:15:47.260" LastActivityDate="2020-07-01T15:15:47.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22299" PostTypeId="2" ParentId="16610" CreationDate="2020-07-01T15:30:22.670" Score="-1" Body="&lt;p&gt;The closest counterexamples I can think of are cases where reward shaping is required to learn a good policy but ends up having unintended consequences.&lt;/p&gt;&#xA;&lt;p&gt;Reward shaping is usually used in cases we want to encourage a particular behavior or when the reward is sparse or when capturing exactly what you want is not straightforward or infeasible. But it is not a good practice to rely too much on it as it can have unintended consequences. A simple example of this is described here &lt;a href=&quot;https://openai.com/blog/faulty-reward-functions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/faulty-reward-functions/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="38188" LastActivityDate="2020-07-01T15:30:22.670" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22300" PostTypeId="2" ParentId="22269" CreationDate="2020-07-01T15:54:19.340" Score="3" Body="&lt;p&gt;I think the answer depends very much on &lt;strong&gt;why&lt;/strong&gt; you are reading the paper, what are you trying to get out of it? There are plenty of papers that I &amp;quot;read&amp;quot; (or often really just quickly skim through) where I'll definitely not understand all the math. More often than not, this will be because I don't actually care to deeply understand it.&lt;/p&gt;&#xA;&lt;p&gt;There is plenty of more &amp;quot;practical&amp;quot; research to be done in AI, which definitely doesn't always require a deep understanding of all the math. Intuition can often be enough, at least to get started, for meaningful practical contributions. If this is the sort of research that you're interested in doing, you probably don't need to understand as many of the mathematical parts of AI papers as you do if you're really trying to do research directly in that theoretical area.&lt;/p&gt;&#xA;&lt;p&gt;Personally, when I write &amp;quot;math-heavy&amp;quot; parts in my own papers (and that will often already be restricted to a rather simple level of math in comparison to the &amp;quot;real theory&amp;quot; ML papers), I always try to make sure to include intuitive, English descriptions of what we're doing around it. Even if you don't immediately understand a full equation, just having the intuitive explanation around it to tell you what it means can be enough for a broad understanding of the paper. Then you only have to dive deep into the details of the equations if -- based on the English text -- you decide that you're actually really interested. So, if there are sufficient, intuitive explanations surrounding the equations, I'd recommend to focus heavily on that first. Not every paper does this though, sometimes there's very little text and very much math, and then this can be difficult.&lt;/p&gt;&#xA;&lt;p&gt;Even if it turns out that you do have to understand math, you may not have to understand ALL of it right away though. The important parts that I would try to focus on understanding first are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A mathematical description of the &amp;quot;problem&amp;quot;. This could be an objective function, a metric to be optimised/minimised/maximised, or an existing equation from previous literature that the authors take as a starting point and inspect some detail of in greater detail.&lt;/li&gt;&#xA;&lt;li&gt;Mathematical descriptions of the outcomes/results. These could be equations that they actually use in concrete algorithms (see if you can relate them to any pseudocode that may be present), or the final equations stated in theorems / at the end of proofs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;All the complex parts in between are probably less important. Just a vague idea of what the starting point is, and a vague understanding of the final outcome, can be enough to at least know what the paper is about. Then you can decide for yourself whether you really need to know more about the details in between, or if they're maybe not relevant to you / your work / your research.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-07-01T15:54:19.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22301" PostTypeId="1" CreationDate="2020-07-01T22:17:44.237" Score="0" ViewCount="22" Body="&lt;p&gt;Conceptually, if you had an internal 3d model of all objects in CV you could do a scan matching algorithm.&lt;/p&gt;&#xA;&lt;p&gt;This algorithm would be ridiculously computationally intensive, but it would have a high degree of accuracy (I believe).&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if ANN can be used to approximate this internal scan matching in a way that greatly reduces computational load?&lt;/p&gt;&#xA;&lt;p&gt;For example let's say you have a 3d scan of a car. This car can take a variety of orientations.&lt;/p&gt;&#xA;&lt;p&gt;One way to do scan matching is pick a bunch of orientations, project to 2d and do a sliding scan match.&lt;/p&gt;&#xA;&lt;p&gt;Could this entire operation be approximated by a NN which ultimately would have a reduced computational load?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-07-01T22:17:44.237" Title="Computational difference between ANN and Pattern Matching" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22302" PostTypeId="1" AcceptedAnswerId="22310" CreationDate="2020-07-02T00:06:25.233" Score="0" ViewCount="52" Body="&lt;p&gt;I want to use reinforcement learning in an environment I made. The exact environment doesn't really matter, but it comes down to this: The amount of different states in the environment is infinite e.g. amount of ways you can put 4 cars at an intersection, but the amount of different actions is only 3 e.g. go forward, right or left. The state exists out of five numbers. My question is: what algorithm should I use or at least what kind of algorithm?&lt;/p&gt;&#xA;" OwnerUserId="38316" LastEditorUserId="2444" LastEditDate="2020-07-02T18:24:09.123" LastActivityDate="2020-07-02T18:24:09.123" Title="What reinforcement learning algorithm should I use in continuous states?" Tags="&lt;reinforcement-learning&gt;&lt;algorithm&gt;&lt;dqn&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22303" PostTypeId="1" CreationDate="2020-07-02T03:07:49.060" Score="0" ViewCount="39" Body="&lt;p&gt;I am trying to implement a DQN agent for playing standard 3x3 Tic-Tac-Toe (it is a double DQN with experience replay, and using a target network). I got the hyperparameters to the point&#xA;where the agent tends to reach a 92-93% win percentage against a random opponent, which is not quite optimal (a tabular Q-learner wins 99.5% and ties the remaining 0.5%, as does minimax). Soon after reaching the peak, it tends to crash down to abysmal performance, and never quite recovers. I am just wondering how typical these results are. Should I expect that it is possible to obtain an agent that plays optimally using DQN, or is the algorithm inherently so noisy that it can only be expected to get good but never quite perfect?&lt;/p&gt;&#xA;&lt;p&gt;Thank you :)&lt;/p&gt;&#xA;" OwnerUserId="38318" LastActivityDate="2020-07-02T03:07:49.060" Title="DQN Tic-Tac-Toe does not quite become optimal" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22304" PostTypeId="1" AcceptedAnswerId="22313" CreationDate="2020-07-02T05:27:25.177" Score="1" ViewCount="98" Body="&lt;p&gt;In a course that I am attending, the cost function of a support vector machine is given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum_{j=1}^{n} \Theta_{j}^{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\operatorname{cost}_{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\operatorname{cost}_{0}$&lt;/span&gt; look like this (in Magenta):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z2xaT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z2xaT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QSIqh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QSIqh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What are the values of the functions &lt;span class=&quot;math-container&quot;&gt;$\operatorname{cost}_{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\operatorname{cost}_{0}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;For example, if using logistic regression the values of &lt;span class=&quot;math-container&quot;&gt;$\operatorname{cost}_{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\operatorname{cost}_{0}$&lt;/span&gt; would be &lt;span class=&quot;math-container&quot;&gt;$-\log* \operatorname{sigmoid}(-z)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-\log*(1-\operatorname{sigmoid}(-z))$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="2444" LastEditDate="2020-07-03T13:11:53.397" LastActivityDate="2020-11-01T22:20:26.683" Title="What is the definition of the &quot;cost&quot; function in the SVM's objective function?" Tags="&lt;math&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22305" PostTypeId="2" ParentId="22127" CreationDate="2020-07-02T05:32:45.687" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Given a non-admissible heuristic function, A* will always give a solution if one exists, but there is no guarantee it will be optimal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I won't duplicate the proof here, but it isn't too hard to prove that any best-first search will find a solution for any measure of best, given that a path to the solution exists and infinite memory. A* is a best-first search algorithm, so it will always find a solution if one exists.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Which heuristics guarantee the optimality of A*? Is the admissibility of the heuristic always a necessary condition for A* to produce an optimal solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Admissibility is not a necessary condition. Take any admissible heuristic &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; and make a new function &lt;span class=&quot;math-container&quot;&gt;$h(n) = h_1(n)+5$&lt;/span&gt;. This heuristic is not admissible, but if you run A* on it, it will still find optimal solutions.&lt;/p&gt;&#xA;&lt;p&gt;But, we also have to ask what you mean by &amp;quot;the optimality of A*&amp;quot;, because optimality can have two senses here. My point in the previous paragraph is in the sense of returning optimal paths. An alternate interpretation is that no algorithm performs fewer expansions that A* with the same information. This is probably not what was meant, and the answer in that context is far more complicated. But, with an inconsistent (but admissible) heuristic, A* can perform exponentially more expansions than other known algorithms, and thus is not the optimal algorithm to use.&lt;/p&gt;&#xA;" OwnerUserId="17493" LastEditorUserId="2444" LastEditDate="2020-11-07T14:40:05.440" LastActivityDate="2020-11-07T14:40:05.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22307" PostTypeId="1" AcceptedAnswerId="22325" CreationDate="2020-07-02T07:55:41.800" Score="1" ViewCount="354" Body="&lt;p&gt;Feature scaling, in general, is an important stage in the data preprocessing pipeline.&lt;/p&gt;&#xA;&lt;p&gt;Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature scaling. Why is that?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2021-02-15T23:03:04.750" Title="Why are decision trees and random forests scale invariant?" Tags="&lt;decision-tree&gt;&lt;random-forest&gt;&lt;feature-engineering&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22308" PostTypeId="1" AcceptedAnswerId="22311" CreationDate="2020-07-02T08:44:24.823" Score="4" ViewCount="92" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In contrast to L2 regularization, L1 regularization usually yields sparse feature vectors and most feature weights are zero.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What's the reason for the above statement - could someone explain it mathematically, and/or provide some intuition (maybe geometric)?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2021-01-29T22:24:37.303" LastActivityDate="2021-01-29T22:24:37.303" Title="Why does L1 regularization yield sparse features?" Tags="&lt;machine-learning&gt;&lt;regularization&gt;&lt;l2-regularization&gt;&lt;l1-regularization&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22309" PostTypeId="2" ParentId="18000" CreationDate="2020-07-02T09:28:35.467" Score="1" Body="&lt;p&gt;Inverse Reinforcement Learning (IRL) is a technique that attempts to recover the reward function that the expert is implicitly maximising based on expert demonstrations. When solving reinforcement learning problems, the agent maximises a reward function specified by the designer, and in the process of reward maximisation, accomplishes some task that it had set out to do. However, reward functions for certain tasks are sometimes difficult to specify by hand. For example the task of driving takes into consideration many different factors such as the distance of the car in front of him, the road conditions and whether or not the person needs to get to his destination quickly. A reward function can be hand specified based on these features. However, when there exists trade offs between these different features, it is difficult to know how to specify the different desiderata of these tradeoffs.&lt;/p&gt;&#xA;&lt;p&gt;Instead of specifying the trade offs manually, it would be easier to recover a reward function from expert demonstrations using IRL. Such a reward function can lead to better generalisations to unseen states as long as the features of driving do not change.&lt;/p&gt;&#xA;&lt;p&gt;In the case where reward shaping fails to learn a task (such as driving), it would be better to have someone demonstrate a task and learn a reward function from these demonstrations. Solving the MDP with the learnt reward function will thus yield a policy that should resemble the demonstrated behaviour. The reward function learnt should also generalise to unseen states and the agent acting in unseen states should be able to perform actions that an expert would take when he is placed in the same conditions, assuming that the unseen states come from the same distribution as the training states.&lt;/p&gt;&#xA;&lt;p&gt;While Reward Shaping might be able to perform the same task as well, IRL might be able to do better, based on some performance metric that will differ from problem to problem.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-07-02T09:28:35.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22310" PostTypeId="2" ParentId="22302" CreationDate="2020-07-02T10:17:17.193" Score="0" Body="&lt;p&gt;I would recommend looking at &lt;a href=&quot;https://www.nature.com/articles/nature14236&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-Learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-07-02T10:17:17.193" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22311" PostTypeId="2" ParentId="22308" CreationDate="2020-07-02T10:48:36.283" Score="3" Body="&lt;p&gt;In &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/regularizers/l1&quot; rel=&quot;nofollow noreferrer&quot;&gt;L1 regularization&lt;/a&gt;, the penalty term you compute for every parameter is a function of the absolute value of a given weight (times some regularization factor).&#xA;Thus, irrespective of whether a weight is positive or negative (due to the absolute value) and irrespective of how large the weight is, there will be a penalty incurred as long as weight is unequal 0. So, the only way how a training procedure can considerably reduce the L1 regularization penalty is by driving all (unnecessary) weights towards 0, which results in a sparse representation.&lt;/p&gt;&#xA;&lt;p&gt;Of course, the &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/regularizers/l2&quot; rel=&quot;nofollow noreferrer&quot;&gt;L2 regularization&lt;/a&gt; will also only be strictly 0 when all weights are 0. However, in L2, the contribution of a weight to the L2 penalty is proportional to the &lt;em&gt;&lt;strong&gt;squared&lt;/strong&gt;&lt;/em&gt; value of the weight. Therefore, a weight whose absolute value is smaller than 1, i.e. &lt;span class=&quot;math-container&quot;&gt;$abs(weight) &amp;lt; 1$&lt;/span&gt;, will be much less punished by L2 than it would be by L1, which means that L2 puts less emphasis on driving all weights towards exactly 0. This is because squaring a some value in (0,1) will result in a value of lower magnitude than taking the un-squared value itself: &lt;span class=&quot;math-container&quot;&gt;$x^2 &amp;lt; x\ for\ all\ x\ with\ abs(x) &amp;lt; 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, while both regularization terms end up being 0 only when weights are 0, the L1 term penalizes small weights with &lt;span class=&quot;math-container&quot;&gt;$abs(x) &amp;lt; 1$&lt;/span&gt; much more strongly than L2 does, thereby driving the weight more strongly towards 0 than L2 does.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-07-02T17:12:04.663" LastActivityDate="2020-07-02T17:12:04.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22312" PostTypeId="1" CreationDate="2020-07-02T12:45:22.237" Score="0" ViewCount="28" Body="&lt;p&gt;I have a problem with a subset of my data which is as follows:&lt;/p&gt;&#xA;&lt;p&gt;I can train a model (doesn't matter what, xgboost, BERT, etc., it is a text classification problem), on my data and get a decent performance. Now it turns out that there is a specific subset I can access where the performance is explicitly worse. On the contrary I can obviously exclude this data and get a better performance.&lt;/p&gt;&#xA;&lt;p&gt;Now, I know that the final input distribution will obviously contain samples which would fall into this &amp;quot;bad&amp;quot; subset, so simply excluding it in the training will probably not suffice. A model trained on data which doesn't follow the &amp;quot;real-world&amp;quot; distribution will obviously be worse in the &amp;quot;real-world&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way I can deal with this knowledge? Maybe weighting the subset in the training? Or preprocessing in a way? Or is there a specific technical term for this so I can do research myself?&lt;/p&gt;&#xA;&lt;p&gt;I'll gladly provide more information if needed, but would also like to keep the question as general as possible.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-07-02T12:45:22.237" Title="accessible subset of data impacts performance" Tags="&lt;classification&gt;&lt;datasets&gt;&lt;text-classification&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22313" PostTypeId="2" ParentId="22304" CreationDate="2020-07-02T13:09:35.713" Score="2" Body="&lt;p&gt;That is the &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Hinge_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;hinge loss&lt;/a&gt;&lt;/em&gt;, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\ell(y)=\max (0,1-t \cdot y),&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which, in your use case, is something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{cost}\left(h_{\theta}(x), y\right)=\left\{\begin{array}{ll}&#xA;\max \left(0,1-\theta^{T} x\right) &amp;amp; \text { if } y=1 \\&#xA;\max \left(0,1+\theta^{T} x\right) &amp;amp; \text { if } y=0&#xA;\end{array}\right.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Check &lt;a href=&quot;https://towardsdatascience.com/optimization-loss-function-under-the-hood-part-iii-5dff33fa015d&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; on it.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastEditorUserId="2444" LastEditDate="2020-11-01T22:20:26.683" LastActivityDate="2020-11-01T22:20:26.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22314" PostTypeId="2" ParentId="4714" CreationDate="2020-07-02T14:28:51.283" Score="1" Body="&lt;p&gt;You can do that pretty easily using Posenet or Openpose. Train the keypoints for Squats and then count it. :)&lt;/p&gt;&#xA;" OwnerUserId="38330" LastActivityDate="2020-07-02T14:28:51.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22315" PostTypeId="1" CreationDate="2020-07-02T15:28:19.617" Score="0" ViewCount="31" Body="&lt;p&gt;I have time series data for every single day from last 5 years with seasonal variation and a general increase in trend. This is what my data looks like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zGT0S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zGT0S.png&quot; alt=&quot;Time series forecast for everyday for till a distant future&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And I am trying to predict for every single day for 4-5 years in future. Approaches I have used currently:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;LSTM, GRU (but they are extremely prone to overfitting ad I am trying&#xA;successive predictions which results in massive error accumulation&#xA;and often flat over time for smaller lookbacks and sine wave driven&#xA;patterns for long lookbacks.&lt;/li&gt;&#xA;&lt;li&gt;SARIMAX and other seasonal forecast&#xA;models but they tend to scale very bad for a large period of&#xA;predictions. Mostly they eventually result in a curve that follows&#xA;trend(increase/decrease) but values are almost linear or&#xA;unrealistically close to each other.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What I am currently thinking of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Separate same day of each year and fit a model to those values and&#xA;then have successive predictions for the same day for subsequent&#xA;years. Thus having 365 different models and then concatenating the&#xA;resultant values to have the whole year and future years predictions.&#xA;This way I'll perfectly preserve the seasonality and yet accumulation&#xA;errors will be minimized.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So my question is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is this a good idea (and if it is, is there a method to do it feasibly without assessing each of the 365 models) or I should try any other approach?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6635" LastActivityDate="2020-07-02T15:28:19.617" Title="Time series forecast for everyday for till a distant future" Tags="&lt;long-short-term-memory&gt;&lt;time-series&gt;&lt;forecasting&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22316" PostTypeId="1" CreationDate="2020-07-02T19:46:49.127" Score="1" ViewCount="68" Body="&lt;p&gt;When using an on-policy method in reinforcement learning, like &lt;em&gt;advantage actor-critic&lt;/em&gt;, you shouldn't use old data from an &lt;em&gt;experience buffer&lt;/em&gt;, since a new policy requires new data. Does this mean that to apply batching to an on-policy method you have to have multiple parallel environments?&lt;/p&gt;&#xA;&lt;p&gt;As an extension of this, if only one environment is available when using on-policy methods, does that mean batching isn't possible? Doesn't that limit the power of such algorithms in certain cases?&lt;/p&gt;&#xA;" OwnerUserId="38335" LastEditorUserId="2444" LastEditDate="2020-10-12T12:53:06.013" LastActivityDate="2021-03-12T00:01:45.350" Title="Do we need multiple parallel environments to train in batches an on-policy algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;policies&gt;&lt;experience-replay&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22317" PostTypeId="2" ParentId="22070" CreationDate="2020-07-02T20:28:36.360" Score="0" Body="&lt;p&gt;This is still a long way off and would require what is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial General Intelligence&lt;/a&gt;, which is probably the best descriptive term for AI with cognitive capacity akin to humans.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;(Another term often used is &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Superintelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;superintelligence&lt;/a&gt;&amp;quot;, but it's less precise in regard to general intelligence, in that it could be understood to connote superior intelligence that is not general.  The same problem exists for Searle's &amp;quot;Strong AI&amp;quot;, an AI that matches or exceeds humans capacity, since AI can now be &amp;quot;strong but narrow&amp;quot;, exceeding human capability in a single task or a set of tasks, such as a game, or managing fully definable systems such as air-conditioning.)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Part of the problem is we still don't really understand how human cognition works, so the famous projection by an incredibly important figure in AI, &lt;a href=&quot;https://en.wikipedia.org/wiki/Herbert_A._Simon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Herbert A. Simon&lt;/a&gt;, that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Machines will be capable, within twenty years, of doing any work a man can do.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;is still, like fusion power, potentially always 20 years in the future.  (Worth noting that if a general superintelligence is developed, it could not only do anything humans could do, but things we can't even conceive of, because its intelligence would greatly exceed our own, and would likely be capable of continual self-optimization.)&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-07-02T20:35:21.103" LastActivityDate="2020-07-02T20:35:21.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22318" PostTypeId="1" AcceptedAnswerId="22324" CreationDate="2020-07-03T04:37:45.397" Score="0" ViewCount="39" Body="&lt;p&gt;Text classification of equal length texts works without padding, but in reality, practically, texts never have the same length.&lt;/p&gt;&#xA;&lt;p&gt;For example, spam filtering on blog article:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;thanks for sharing    [3 tokens] --&amp;gt; 0 (Not spam)&#xA;this article is great [4 tokens] --&amp;gt; 0 (Not spam)&#xA;here's &amp;lt;URL&amp;gt;          [2 tokens] --&amp;gt; 1 (Spam)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should I pad the texts on the right:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;thanks for     sharing --&#xA;this   article is      great&#xA;here's URL     --      --&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, pad on the left:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--   thanks  for    sharing&#xA;this article is     great&#xA;--   --      here's URL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What are the pros and cons of either pad left or right?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-07-03T07:35:07.677" Title="Text classification of non-equal length texts, should I pad left or right?" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;text-classification&gt;&lt;padding&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22319" PostTypeId="1" CreationDate="2020-07-03T05:22:04.260" Score="1" ViewCount="34" Body="&lt;p&gt;I want to collect training samples from images.&lt;/p&gt;&#xA;&lt;p&gt;That can mean different things depending on the context. I think of the simplest case, which should be most commonly required. Because it is so common, there may be a standard tool for it.&lt;/p&gt;&#xA;&lt;p&gt;An example would be to have a collection of images of random street scenes and manually collect images of nonoccluded cars from them into separate files.&lt;/p&gt;&#xA;&lt;p&gt;What is a common way or tool to do this:&lt;/p&gt;&#xA;&lt;p&gt;For a large number of images, select one or more rectangles (of arbitrary size and with edges parallel to the image edges) in the image and save them to separate image files.&lt;/p&gt;&#xA;&lt;p&gt;Of course, it can be done with any general image editing program, but in this case, most of the work time would be used for opening new images, closing old images, saving sample images and the most time-consuming part of entering a non-conflicting file name for the individual sample image files.&lt;br /&gt;&#xA;For small numbers of samples per input file, this may need about an order of magnitude more time, and also more complex interaction.&lt;/p&gt;&#xA;&lt;p&gt;I would prefer a tool running on Linux/Ubuntu.&lt;br /&gt;&#xA;If this does not exist, I'd be curious why.&lt;/p&gt;&#xA;" OwnerUserId="2317" LastEditorUserId="2317" LastEditDate="2020-07-03T06:13:27.833" LastActivityDate="2020-07-14T09:20:46.813" Title="How to manually collect rectangular training data samples from images?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22320" PostTypeId="1" CreationDate="2020-07-03T05:28:10.090" Score="2" ViewCount="52" Body="&lt;p&gt;I've been reading about &lt;strong&gt;Fisher's Linear Discriminant Analysis&lt;/strong&gt; lately, and I noticed that the objective function (particularly for &lt;em&gt;two-class&lt;/em&gt; classification) to be maximized contains &lt;strong&gt;scatter&lt;/strong&gt; terms instead of &lt;strong&gt;variance&lt;/strong&gt;, in the denominator. Why is that?&lt;/p&gt;&#xA;&lt;p&gt;To clarify, the scatter of a sample is just the variance multiplied by the number of data points in the sample.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-07-14T07:37:29.527" Title="Why is 'scatter' used instead of variance in LDA?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22322" PostTypeId="1" CreationDate="2020-07-03T05:53:26.910" Score="0" ViewCount="49" Body="&lt;p&gt;I wanted to understand back-propagation so I made a basic neural network library. I used momentum, with learning rate = &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt;, beta = &lt;span class=&quot;math-container&quot;&gt;$0.99$&lt;/span&gt;, epochs = &lt;span class=&quot;math-container&quot;&gt;$200$&lt;/span&gt;, batch size = &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, loss function is cross entropy and model structure is &lt;span class=&quot;math-container&quot;&gt;$784$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; and all layers use sigmoid. It performed terribly at first, so I initialized all the weights and biases in the range &lt;span class=&quot;math-container&quot;&gt;$[10^{-9}, 10^{-8}]$&lt;/span&gt; and it worked. I am quite new to deep learning and I find TensorFlow doesn't seem as friendly to beginners who want to play around with hyper-parameters. How do you find the right hyper-parameters? I trained it on 100 digits (which took 10 minutes), tweaked hyper-parameters, chose the best set and trained the model using that set on the entire data set of &lt;span class=&quot;math-container&quot;&gt;$60,000$&lt;/span&gt; images. I also found that halving the epochs and doubling the training set size gave better results. Are there fool proof heuristics to find good hyper-parameters? What is the best set of hyper-parameters (without regularization, dropout, etc) for MNIST digits? &lt;a href=&quot;https://github.com/coderkarthiko/CustomNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is the code for those who want to take a look.&lt;/p&gt;&#xA;" OwnerUserId="38343" LastEditorUserId="38343" LastEditDate="2020-07-03T09:38:32.217" LastActivityDate="2020-07-03T09:38:32.217" Title="96.91% accuracy on MNIST after 2 hours of training using custom made neural net library. Ways to improve?" Tags="&lt;deep-learning&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22323" PostTypeId="1" AcceptedAnswerId="22346" CreationDate="2020-07-03T06:09:24.657" Score="3" ViewCount="90" Body="&lt;p&gt;Assuming the definition of an agent to be:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;An entity that perceives its environment, processes the perceived information, and acts on the environment such that some goal is fulfilled.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Are there any agents that are based on quantum processing/computing (i.e. implemented by a network of &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_logic_gate&quot; rel=&quot;nofollow noreferrer&quot;&gt;quantum gates&lt;/a&gt;)?&lt;/p&gt;&#xA;&lt;p&gt;Is there any work done towards this end? If so, could someone provide references?&lt;/p&gt;&#xA;" OwnerUserId="34403" LastEditorUserId="2444" LastEditDate="2020-07-05T00:02:30.317" LastActivityDate="2020-07-06T01:24:44.087" Title="Are there any agents that are based on quantum computing?" Tags="&lt;intelligent-agent&gt;&lt;quantum-computing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22324" PostTypeId="2" ParentId="22318" CreationDate="2020-07-03T07:35:07.677" Score="1" Body="&lt;p&gt;For any model that does not take a time series approach like an RNN does, the padding shouldn't make a difference.&lt;/p&gt;&#xA;&lt;p&gt;I prefer padding right simply because there also might be text you need to cut-off. Then padding is more intuitive as you either cut-off a text if it's too long or pad a text when it's too short.&lt;/p&gt;&#xA;&lt;p&gt;Either way, when a model is trained a certain way, it shouldn't make a difference so long the testing is also padded the way it was presented in training.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-07-03T07:35:07.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22325" PostTypeId="2" ParentId="22307" CreationDate="2020-07-03T07:39:16.543" Score="2" Body="&lt;p&gt;Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.&lt;/p&gt;&#xA;&lt;p&gt;Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing.&lt;/p&gt;&#xA;&lt;p&gt;Now on the other hand NN have some kind of activation function (leaving RELu aside) that react differently to input that is above 1. Here Normalization, putting every feature between 0 and 1 makes sense.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-07-03T07:39:16.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22326" PostTypeId="2" ParentId="22322" CreationDate="2020-07-03T08:05:49.297" Score="1" Body="&lt;p&gt;There is no singular best set of hyperparameters. Even more, there is no real search algorithm for hyperparameters. You can do a grid search, but this obviously will take some time. Most people either do that or will try to handpick their parameters.&lt;/p&gt;&#xA;&lt;p&gt;A few other things to note: Initializing your weights at &lt;code&gt;[10^9,10^8]&lt;/code&gt; doesn't seem right to me. They should be centered and close to zero initially. You should also take a look again how to set up train/test/val splits.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-07-03T08:05:49.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22327" PostTypeId="1" CreationDate="2020-07-03T09:11:53.473" Score="1" ViewCount="41" Body="&lt;p&gt;I'm using U-Net for image segmentation.&lt;/p&gt;&#xA;&lt;p&gt;The model was trained with images that could contain up to 4 different classes. The train classes are never overlapping.&lt;/p&gt;&#xA;&lt;p&gt;The output of the UNet is a heatmap (with float values between 0 and 1) for each of these 4 classes.&lt;/p&gt;&#xA;&lt;p&gt;Now, I have 2 problems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;for a certain class, how do I segment (draw contours) in the original image only for the points where the heatmap has significant values? (In the image below an example: the values in the centre are significant, while the values on the left aren't. If I draw the segmentation of the entire image without any additional operation, both are considered.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6t3HP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6t3HP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;downstream of the first point, how do I avoid that in the original image the contours of two superimposed classes are drawn? (maybe by drawing only the one that has higher values in the corresponding heatmap)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16136" LastActivityDate="2020-07-03T09:11:53.473" Title="Suppress heatmap non-maxima in segmentation with UNet" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22328" PostTypeId="1" AcceptedAnswerId="22338" CreationDate="2020-07-03T10:07:58.203" Score="2" ViewCount="52" Body="&lt;p&gt;I was reading this &lt;a href=&quot;https://arxiv.org/abs/1911.08415?fbclid=IwAR06xkxojSP-Tb7xemUQWByzmmu_EM2WMorqVZOCw44JhB_pMA357E1OMoI#:%7E:text=GMAN%20adapts%20an%20encoder%2Ddecoder,temporal%20factors%20on%20traffic%20conditions&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; where they are stating the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We also use the T-Test to test the significance of GMAN in 1 hour ahead prediction compared to Graph WaveNet. The p-value is less than 0.01, which demonstrates that GMAN statistically outperforms Graph WaveNet.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does &amp;quot;Model A statistically outperforms B&amp;quot; mean in this context? And how should the p-value threshold be selected?&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2020-07-03T22:55:30.583" LastActivityDate="2020-07-04T07:11:04.450" Title="What does it mean when a model &quot;statistically outperforms&quot; another?" Tags="&lt;deep-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22329" PostTypeId="1" AcceptedAnswerId="24787" CreationDate="2020-07-03T10:21:53.273" Score="1" ViewCount="50" Body="&lt;p&gt;When describing the model architecture for a deep recurrent q network, the authors of the paper &lt;a href=&quot;https://arxiv.org/pdf/1605.06676.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Communicate with Deep Multi-Agent Reinforcement Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;each agent consists of a recurrent neural network (RNN), unrolled for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; time-steps, that maintains an internal state &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, an input network for producing a task embedding &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, and an output network for the Q-values and the messages &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. The input for agent &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is defined as a tuple of &lt;span class=&quot;math-container&quot;&gt;$\left(o_{t}^{a}, m_{t-1}^{a^{\prime}}, u_{t-1}^{a}, a\right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can someone explain what the purpose of the embedding layer is in this specific context?&lt;/p&gt;&#xA;&lt;p&gt;Implementation can be found &lt;a href=&quot;https://colab.research.google.com/gist/MJ10/2c0d1972f3dd1edcc3cd17c636aac8d2/dial.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33448" LastEditorUserId="2444" LastEditDate="2020-11-21T23:03:25.550" LastActivityDate="2020-11-22T14:58:57.960" Title="What is the role of embeddings in a deep recurrent Q network?" Tags="&lt;reinforcement-learning&gt;&lt;recurrent-neural-networks&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;embeddings&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22330" PostTypeId="1" AcceptedAnswerId="22332" CreationDate="2020-07-03T12:12:34.183" Score="2" ViewCount="90" Body="&lt;p&gt;In the course of a scientific work, I will discuss the different types of reinforcement learning. However, I have difficulties to find these different types.&lt;/p&gt;&#xA;&lt;p&gt;So, into which subcategories can reinforcement learning be divided? For example, the following subdivisions seem to be useful&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Model-free and Model-based&lt;/li&gt;&#xA;&lt;li&gt;Dynamic Programming, Monte Carlo and Temporal Difference&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any others?&lt;/p&gt;&#xA;" OwnerUserId="38349" LastEditorUserId="2444" LastEditDate="2020-07-03T13:40:35.550" LastActivityDate="2020-07-03T13:40:35.550" Title="Into which subcategories can reinforcement learning be divided?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22331" PostTypeId="1" CreationDate="2020-07-03T12:17:31.510" Score="0" ViewCount="44" Body="&lt;p&gt;In looking at an algorithm in the paper &lt;a href=&quot;https://arxiv.org/abs/1605.06676&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Communicate with Deep Multi-Agent Reinforcement Learning&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is the full algorithm:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g1vq2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g1vq2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What does the notation &lt;code&gt;for t=T to 1,−1 do:&lt;/code&gt; refer to in terms of &lt;strong&gt;time steps&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;The network structure is a deep recurrent q network.&lt;/p&gt;&#xA;&lt;p&gt;Secondly, why do the gradients need to be reset to zero?&lt;/p&gt;&#xA;" OwnerUserId="33448" LastEditorUserId="2444" LastEditDate="2020-07-03T13:37:13.257" LastActivityDate="2020-07-03T13:37:13.257" Title="What does the notation &quot;for t=T to 1,−1 do&quot; in terms of time steps, in deep recurrent q network?" Tags="&lt;reinforcement-learning&gt;&lt;recurrent-neural-networks&gt;&lt;dqn&gt;&lt;notation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22332" PostTypeId="2" ParentId="22330" CreationDate="2020-07-03T12:49:31.310" Score="4" Body="&lt;p&gt;Your two suggestions are not mutually exclusive. If you go by this process, you'll have to do a &amp;quot;Cartesian product&amp;quot; of a bunch of different RL categorizations which would get out of hand. I recommend, if you can, to describe some sort of &amp;quot;RL taxonomy&amp;quot; instead. By this I mean describing different RL characterizations without assuming they're mutually exclusive.&lt;/p&gt;&#xA;&lt;p&gt;To add to your list :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;On-policy or off-policy&lt;/li&gt;&#xA;&lt;li&gt;Value based or policy gradient&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-03T12:49:31.310" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22333" PostTypeId="1" AcceptedAnswerId="22334" CreationDate="2020-07-03T14:41:33.120" Score="3" ViewCount="104" Body="&lt;p&gt;I want to train ResNet50 model using resistor images like below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dFRVt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dFRVt.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried it by collecting data from google images and there were quite few. So accuracy was very low (around %10) but I wonder If it is due lack of images or is it really possible to classify these images? Because as it is seen the object to be classified is very small and its value as color coded. I thought maybe this is not a good idea. Searched it on google but could not find anybody tried to do it before. I have also tried data augmentation and changing to other models but still its accuracy was quite low.&lt;/p&gt;&#xA;&lt;p&gt;P.S: I have also tried changin epoch numbers, optimizers and all other parameters. So I want to make sure If it is due low data or is it just very hard task to complete for a computer vision model.&lt;/p&gt;&#xA;&lt;p&gt;And Is it rational to crop the image by using a mask before classifying it to make sure all color codes are bigger and easily valuable by model?&lt;/p&gt;&#xA;" OwnerUserId="38344" LastEditorUserId="38344" LastEditDate="2020-07-03T14:47:08.020" LastActivityDate="2020-07-03T19:34:08.647" Title="Is it possible to classify resistors using ResNet50?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;datasets&gt;&lt;categorical-data&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22334" PostTypeId="2" ParentId="22333" CreationDate="2020-07-03T19:34:08.647" Score="3" Body="&lt;p&gt;Yes it should be possible. You may have a bug in your code, or the wrong hyperparameters. Training ResNet-50 will take a long time. Try training on other sets of images and see what accuracy you get to check if your approach is correct. Or, try loading a pretrained model, and training from that.&lt;/p&gt;&#xA;" OwnerUserId="26838" LastActivityDate="2020-07-03T19:34:08.647" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22335" PostTypeId="1" CreationDate="2020-07-03T21:31:47.700" Score="1" ViewCount="61" Body="&lt;p&gt;Suppose I have a private set of images containing some objects.&lt;/p&gt;&#xA;&lt;p&gt;How do i&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Make it very hard for the neural networks such as ImageNet to recognize these objects, while allowing humans to do it at the same time?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose I label these private images - a picture of a cat with a label &amp;quot;cat&amp;quot; - how do I make it hard for the attacker to train his neural network on my labels? Is it possible to somehow fool a neural network so that they couldn't easily train it to recognize it?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Like random transforms etc, so that they couldn't use a neural network to recognize these objects, or even train it on my dataset if they had labels.&lt;/p&gt;&#xA;" OwnerUserId="38359" LastActivityDate="2020-07-04T02:13:04.327" Title="How to prevent image recognition of my dataset with neural networks and make it hard to train them?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;captcha&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22336" PostTypeId="2" ParentId="22335" CreationDate="2020-07-04T02:13:04.327" Score="2" Body="&lt;p&gt;If the model is trained and held constant, then there are so-called adversarial attacks to modify images such that the model classifies them incorrectly (see &lt;em&gt;&lt;a href=&quot;https://openai.com/blog/adversarial-example-research/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attacking Machine Learning&#xA;with Adversarial Examples&lt;/a&gt;&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;However, if you want to make images that are untrainable, you are probably out of luck. Deep neural networks can learn to recognize even random images with random labels (see &lt;em&gt;&lt;a href=&quot;https://arxiv.org/abs/1611.03530&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding deep learning requires rethinking generalization&lt;/a&gt;&lt;/em&gt;), though if there's no rhyme or reason to the randomness, they won't generalize in meaningful ways.&lt;/p&gt;&#xA;" OwnerUserId="11155" LastActivityDate="2020-07-04T02:13:04.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22337" PostTypeId="1" CreationDate="2020-07-04T04:50:44.890" Score="0" ViewCount="1210" Body="&lt;p&gt;How does &lt;em&gt;best-first search&lt;/em&gt; differ from &lt;em&gt;hill-climbing&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="38362" LastEditorUserId="2444" LastEditDate="2020-07-04T21:44:25.327" LastActivityDate="2020-07-04T21:44:25.327" Title="How does best-first search differ from hill-climbing?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;hill-climbing&gt;&lt;best-first-search&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22338" PostTypeId="2" ParentId="22328" CreationDate="2020-07-04T07:01:25.480" Score="4" Body="&lt;p&gt;Most model-fitting is stochastic, so you get different parameters every time you train, and you usually can't say that one algorithm will &lt;em&gt;always&lt;/em&gt; give you a better-performing model.&lt;/p&gt;&#xA;&lt;p&gt;However, since you can retrain many times to get a distribution of models, you can use a statistical test like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Student%27s_t-test&quot; rel=&quot;nofollow noreferrer&quot;&gt;T-Test&lt;/a&gt; to say &amp;quot;algorithm A usually produces a better model than algorithm B,&amp;quot; which is what they mean by &amp;quot;statistically outperforms.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;p-value is usually set by consensus in the field. The higher the p-value, the less confidence you have that there's a statistical difference between the distribution of values being compared. 0.1 might be normal in a field where data is very expensive to collect (like risky, long-term studies of humans), but in machine learning, it's usually easy enough to retrain a model that 0.01 is common, and demonstrates very high confidence. To know more about selecting and interpreting the values, I recommend &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_significance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia's page on statistical significance&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11155" LastEditorUserId="11155" LastEditDate="2020-07-04T07:11:04.450" LastActivityDate="2020-07-04T07:11:04.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22340" PostTypeId="1" AcceptedAnswerId="22344" CreationDate="2020-07-04T08:26:01.267" Score="3" ViewCount="120" Body="&lt;p&gt;Let's say we have a captcha system that consists of a greyscale picture (of a part of a street or something akin to re-captcha), divided into 9 blocks, with 2 missing pieces.&lt;/p&gt;&#xA;&lt;p&gt;You need to choose the appropriate missing pieces from over 15 possibilities to complete the picture.&lt;/p&gt;&#xA;&lt;p&gt;The puzzle pieces have their edges processed with glitch treatment as well as they have additional morphs such as heavy jpeg compression, random affine transform, and blurred edges.&lt;/p&gt;&#xA;&lt;p&gt;Every challenge picture is unique - pulled from a dataset of over 3 million images.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible for the neural network to reliably (above 50%) predict the missing pieces? Sometimes these are taken out of context and require human logic to estimate the correct piece.&lt;/p&gt;&#xA;&lt;p&gt;The chance of selecting two answers in correct order is 1/15*1/14.&lt;/p&gt;&#xA;" OwnerUserId="38359" LastEditorUserId="2444" LastEditDate="2020-07-05T23:51:21.577" LastActivityDate="2020-07-05T23:51:21.577" Title="Is such a captcha AI-resistant?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;image-segmentation&gt;&lt;feature-extraction&gt;&lt;captcha&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22341" PostTypeId="2" ParentId="22186" CreationDate="2020-07-04T11:36:30.057" Score="1" Body="&lt;p&gt;The main difference between distant supervision (as described in the link you provided) and self-supervision lies on the task the network is trained on.&lt;/p&gt;&#xA;&lt;p&gt;Distant supervision focuses on generating weak labels for the very same task that would be tackled with supervised labels, and the final result could be directly used for that matter.&lt;/p&gt;&#xA;&lt;p&gt;Self-supervision is a means for learning a data representation. It does so by learning a surrogate task, which is defined by inputs and labels derived exclusively from the original input data.&lt;/p&gt;&#xA;&lt;p&gt;I can imagine cases in which an implementation of self-supervision could be considered distant supervision (if the task casually matches the target task). On the other hand, if external data sources would be employed for training on a surrogate task, that would be a case of representation learning (that could incorporate self-supervision too).&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2020-07-04T11:36:30.057" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="22342" PostTypeId="1" CreationDate="2020-07-04T13:28:04.040" Score="2" ViewCount="24" Body="&lt;p&gt;I'm reading this paper: &lt;a href=&quot;https://arxiv.org/pdf/1602.07576.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1602.07576.pdf&lt;/a&gt;. I'll quote the relevant bits:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Deep neural networks produce a sequence of progressively more abstract representations by mapping the input through a series of parameterized functions.  In the current generation of neural networks, the representation spaces are usually endowed with very minimal internal&#xA;structure, such as that of a linear space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;^n.&lt;/p&gt;&#xA;&lt;p&gt;In this paper we construct representations that have the structure of a linear &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;-space, for some chosen group &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. This means that each vector in the representation space has&#xA;a &lt;em&gt;pose&lt;/em&gt; associated with it, which can be transformed by the elements of some group of transformations &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. This additional structure allows us to model data more efficiently: A&#xA;filter in a &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;-CNN detects co-occurrences of features that have the preferred relative pose [...]&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A representation space can obtain its structure from other representation spaces to which it is connected. For this to work, the network or layer &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; that maps one representation&#xA;to another should be structure preserving. For &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;-spaces this means that &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; has to be equivariant: &lt;span class=&quot;math-container&quot;&gt;$$\phi(T_gx)=T'_g\phi(x)$$&lt;/span&gt;That is, transforming an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by a transformation &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; (forming &lt;span class=&quot;math-container&quot;&gt;$T_gx$&lt;/span&gt;) and then passing it through the learned map &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; should give the same result as first mapping &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; through &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; and then transforming the representation.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Equivariance can be realized in many ways, and in particular the operators &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T'$&lt;/span&gt; need not be the same. The only requirement for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T'$&lt;/span&gt; is that for any two transformations &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$T(gh) = T (g)T (h)$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is a linear representation of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I didn't understand the paragraph in bold. A structure preserving map is something that preserves some operation between elements in the underlying set. A simple example: if &lt;span class=&quot;math-container&quot;&gt;$f:\mathbb{R}^3\to\mathbb{R}$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$(x,y,z)^T\mapsto x+y+z$&lt;/span&gt;, then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(r+s)=f((r_1,r_2,r_3)^T+(s_1,s_2,s_3)^T)=f((r_1+s_1,r_2+s_2,r_3+s_3))&#xA;\\=r_1+s_1+r_2+s_2+r_3+s_3=f(r)+f(s)$$&lt;/span&gt;&#xA;where the addition in the far left term is in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt; and addition in the far right is in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;. So the map &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; preserves the additional structure of addition.&lt;/p&gt;&#xA;&lt;p&gt;In the quoted paragraph, &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is the structure preserving map, but what's the structure being preserved exactly? And why is the operator on the right different from one on the left? i.e. &lt;span class=&quot;math-container&quot;&gt;$T'$&lt;/span&gt; on RHS instead of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="38372" LastActivityDate="2020-07-04T13:28:04.040" Title="Structure-preserving layer in a network with respect to a transformation" Tags="&lt;convolutional-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22344" PostTypeId="2" ParentId="22340" CreationDate="2020-07-04T17:51:54.540" Score="0" Body="&lt;p&gt;Well to give you a short answer, I would say that &lt;strong&gt;YES,&lt;/strong&gt; it would be &lt;strong&gt;&lt;em&gt;MORE&lt;/em&gt; resistant&lt;/strong&gt; than a more standard captcha approach...&lt;/p&gt;&#xA;&lt;p&gt;This being said, I would still go as far as to predict something like a 75-80% successful prediction rates, for a custom model which is designed specifically for defeating a mechanism such as what you describe. The reason why I am fairly confident in such an appraisal, is primarily because of the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;New techniques which researchers have begun to explore, which are intended to be &lt;em&gt;&lt;strong&gt;&amp;quot;Structure Preserving Convolutions&amp;quot;&lt;/strong&gt;&lt;/em&gt; which utilize a higher dimensional filter to store the extra correlation data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I think that the obfuscation efforts that you mention will definitely help to some degree, although they can be easily defeated by training the model on a dataset which you pull out some portion of the samples during pre-processing and inject the same sort of noise and glitch treatments, etc.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An idea that would be worth exploring would be to process your dataset with an adversarial model which you could then use to generate &lt;em&gt;&lt;strong&gt;Adversarial Noise&lt;/strong&gt;&lt;/em&gt; that could then be fed into a pre-process step for your images and replace &lt;em&gt;(or extend)&lt;/em&gt; the obfuscation efforts!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; If you cant beat 'em, then join 'em! Just train a model to defeat your captcha implementation, and then use the model to generate adversarial examples and then apply obfuscations to your data set accordingly!&lt;/p&gt;&#xA;&lt;p&gt;For more information on what I am talking about in my suggestion for further obfuscation efforts explore some of the papers you can find on Google Scholar - &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C5&amp;amp;as_vis=1&amp;amp;q=adversarial+training+examples&amp;amp;btnG=&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensemble Adversarial Training Examples&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30631" LastActivityDate="2020-07-04T17:51:54.540" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22345" PostTypeId="2" ParentId="22337" CreationDate="2020-07-04T21:16:56.117" Score="1" Body="&lt;h2&gt;Best-first search&lt;/h2&gt;&#xA;&lt;p&gt;BFS is a search approach and not just a single algorithm, so there are many best-first (BFS) algorithms, such as greedy BFS, A* and B*. BFS algorithms are &lt;strong&gt;informed&lt;/strong&gt; search algorithms, as opposed to uninformed search algorithms (such as breadth-first search, depth-first search, etc.), i.e. BFS algorithms make use of domain knowledge that can be encoded into a so-called &lt;strong&gt;heuristic function&lt;/strong&gt; (that's why they are informed!).&lt;/p&gt;&#xA;&lt;p&gt;Every BFS algorithm defines a so-called &lt;strong&gt;evaluation function&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; of the following form&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(n) = g(n) + h(n)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for all nodes &lt;span class=&quot;math-container&quot;&gt;$n \in V$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is the set of nodes or states of the search space), where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the cost from the start node (of the search) to the node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; (the so-called heuristic, which is the way of including domain knowledge to solve the search problem) is an estimate of the cost of the cheapest path from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal node.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Depending on how you define &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and, in particular, the heuristic function &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, you get different BFS algorithms. For instance, A* is a BFS algorithm where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is an &lt;em&gt;admissible heuristic&lt;/em&gt; (i.e. it never overestimates the cost to the goal node). Because of this admissibility property, A* is guaranteed to find the globally optimal solution (i.e. the cheapest path from a start node to the goal node among all paths), but you can ignore this detail.&lt;/p&gt;&#xA;&lt;p&gt;To apply a BFS algorithm, you need to define the evaluation function and the &lt;strong&gt;search space&lt;/strong&gt;, i.e. the states (or nodes) and their connections. For example, if you want to find the cheapest path from Paris to Madrid, you need to define that Paris is the start node, Madrid the goal node and then you need to define all intermediate nodes between Paris and Madrid, but you also need to define &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Hill climbing&lt;/h2&gt;&#xA;&lt;p&gt;Hill climbing (HC) is a &lt;em&gt;general&lt;/em&gt; search strategy (so it's also not just an algorithm!). HC algorithms are &lt;strong&gt;greedy local search&lt;/strong&gt; algorithms, i.e. they typically only find local optima (as opposed to global optima) and they do that greedily (i.e. they do not look ahead). The idea behind HC algorithms is that of moving (or climbing) in the direction of increasing value. HC algorithms can be used to solve optimization problems and not just well-defined search problems, i.e. you start from some solution and you move to the best neighboring solution, and then loop.&lt;/p&gt;&#xA;&lt;h2&gt;Best-first search vs hill climbing&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BFS algorithms are informed search algorithms (as opposed to uninformed)&lt;/li&gt;&#xA;&lt;li&gt;BFS algorithms need to define the search space and the evaluation function&lt;/li&gt;&#xA;&lt;li&gt;Some BFS algorithms (such as A*) are guaranteed to find the best global solution&lt;/li&gt;&#xA;&lt;li&gt;HC algorithms are general (i.e. widely applicable) but local and greedy search and optimization algorithms; consequently, they are not generally guaranteed to find the global optimum (but, in practice, they may work well, depending also on the problem).&lt;/li&gt;&#xA;&lt;li&gt;HC algorithms do not need to define the search space explicitly (i.e. you do not need to define the start and goal nodes, and so on), but you just need to a way of determining the best neighboring solution&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;The book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; by Stuart Russell and Peter Norvig provides more details about these two search approaches. (You can find free copies of this book online).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-04T21:40:36.530" LastActivityDate="2020-07-04T21:40:36.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22346" PostTypeId="2" ParentId="22323" CreationDate="2020-07-05T00:33:05.573" Score="3" Body="&lt;p&gt;I think you are looking for &lt;a href=&quot;https://arxiv.org/pdf/1611.09347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;quantum machine learning&lt;/em&gt; (QML)&lt;/a&gt;, which is a relatively new field that sits at the intersection of quantum computing and machine learning.&lt;/p&gt;&#xA;&lt;p&gt;If you are not familiar with quantum computing (QC) and you are interested in QML, I suggest that you follow &lt;a href=&quot;https://www.youtube.com/watch?v=VPsl_5RQe1A&amp;amp;list=PLnhoxwUZN7-6hB2iWNhLrakuODLaxPTOG&quot; rel=&quot;nofollow noreferrer&quot;&gt;this course&lt;/a&gt; by prof. Umesh Vazirani and read the book &lt;a href=&quot;https://modphys.hosted.uark.edu/ETC/MISC/Quantum_Computing_for_Computer_Scientists_-_Yanofsky-cup-2008.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Computing for Computer Scientists&lt;/a&gt; (2008) by Yanofsky and Mannucci. If you have a solid knowledge of linear algebra, you should not encounter big problems while learning QC. Be prepared to deal with the weirdness and beauty of qubits, quantum entanglement, and so on.&lt;/p&gt;&#xA;&lt;p&gt;If you want to directly dive into QML (although I am not familiar with the details of QML, I suggest that you first get familiar with the basics of QC, which I am familiar with) there are already several courses, &lt;a href=&quot;https://arxiv.org/pdf/1611.09347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;papers&lt;/a&gt;, &lt;a href=&quot;https://www.tensorflow.org/quantum/concepts&quot; rel=&quot;nofollow noreferrer&quot;&gt;overviews&lt;/a&gt;, and libraries (such as &lt;a href=&quot;https://www.tensorflow.org/quantum&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow Quantum&lt;/a&gt;) on quantum machine learning.&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in quantum reinforcement learning, maybe have a look at the paper &lt;a href=&quot;https://arxiv.org/pdf/0810.3828.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Reinforcement Learning&lt;/a&gt; (2008) by Daoyi Dong et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-06T01:24:44.087" LastActivityDate="2020-07-06T01:24:44.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22347" PostTypeId="1" AcceptedAnswerId="22360" CreationDate="2020-07-05T05:39:39.613" Score="2" ViewCount="43" Body="&lt;p&gt;I have trouble understanding how &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/model_selection/plot_nested_cross_validation_iris.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;nested cross-validation&lt;/a&gt; works - I understand the need for two loops (one for selecting the model, and another for training the selected model), but &lt;strong&gt;why are they nested?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;From what I understood, we need to select the model before training it, which points toward non-nested loops.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please explain what's wrong (or right?) with my line of reasoning, and also explain nested cross-validation in greater detail? A representative example would be great.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-07-06T12:22:11.310" LastActivityDate="2020-07-06T12:22:11.310" Title="How exactly does nested cross-validation work?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;hyper-parameters&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22348" PostTypeId="1" AcceptedAnswerId="22349" CreationDate="2020-07-05T09:25:12.933" Score="2" ViewCount="54" Body="&lt;p&gt;I've found online some DQN algorithms that (in a problem with a continuous state space and few actions, let's say 2 or 3), at each time step, compute and store (in the memory used for updating) all the possible actions (so all the possible rewards). For example, on page 5 of the paper &lt;a href=&quot;http://cslt.riit.tsinghua.edu.cn/mediawiki/images/5/5f/Dtq.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-trading&lt;/a&gt;, they say&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This means that we don't need a random exploration to sample an action as in many reinforcement learning tasks; instead we can emulate all the three actions to update the Q-network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How can this be compatible with the exploration-exploitation dilemma, which states that you have to balance the time steps of exploring with the ones of exploiting?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-07-05T23:44:07.713" LastActivityDate="2020-07-05T23:44:07.713" Title="Why do some DQN implementations not require random exploration but instead emulate all actions?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22349" PostTypeId="2" ParentId="22348" CreationDate="2020-07-05T12:55:04.447" Score="1" Body="&lt;p&gt;The example that you linked is using a model (emulation) in order to look ahead at all possible actions from any state. It essentially explores off-policy and offline using that model. This is not an option that is available in all environments, but if possible it resolves the exploration/eploitation dilemma for a &lt;em&gt;single&lt;/em&gt; time step nicely by investigating all options.&lt;/p&gt;&#xA;&lt;p&gt;Longer term the agent proposed by the link does not sufficiently explore for general use in my opinion. It appears to always choose a single action deterministically based on maximising action value. In other words it always attempts to exploit the training data so far, even though it augments the training data with short-term knowledge about exploration. However, this appears to be sufficient in the problem domain that it is used in. I suspect this is for a couple of reasons:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The environment is non-stationary, making long-term state predictions unreliable in any case. An agent that learns to exploit in the short term (i.e. over only a few time steps into the future) is likely to be approximately optimal already.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;State transitions may be highly stochastic, meaning that state space will still be adequately explored even using a deterministic policy. This feature of the environment is also used by other well-known Q learning approaches with deterministic behaviour policies, such as &lt;a href=&quot;https://cling.csd.uwo.ca/cs346a/extra/tdgammon.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD Gammon&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I think you have correctly identified a weakness of the approach used in the linked paper that means it may not make a strong general algorithm. The algorithm avoids addressing the exploration/exploitation balance in full, and instead relies on features of the environment to work well despite this. If you find yourself working in similar environments for your own projects, then it may well be worth trying the same approach. However, if you find yourself working in a more deterministic environment with more stationary behaviour and sparse rewards, the lack of state space exploration would be a serious limitation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-05T13:02:15.067" LastActivityDate="2020-07-05T13:02:15.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22350" PostTypeId="2" ParentId="22340" CreationDate="2020-07-05T15:18:00.333" Score="0" Body="&lt;p&gt;This is not resistant at all. A simple comparison on the similarity of edge pixels between borders should be very sufficient to break this method completely.&lt;/p&gt;&#xA;&lt;p&gt;We can do a very simple calculation. Assume the picture is 8bit black and white with each border being 50x50 pixels. Also assume the distribution is continuously uniform between 0-255 (should probably be normally distributed, but whatever). You have a total of 200 pixels that border each other in between borders. Assume that the &lt;em&gt;naturally generated image&lt;/em&gt;  is continuous in brightness with respect to dimension in at least 10% of the image with +/- 10 units of brightness is acceptable. Thus we have 20 pixels to work with.&lt;/p&gt;&#xA;&lt;p&gt;In the case where the image is incorrect, we assume the pixel brightness to be iid in [0-255], thus giving us roughly a 8% (21/256) chance of each pixel around the border to be of acceptable similarity. Which gives us about a 10^-22 chance of this algorithm being fooled. You might disagree with my assumed parameters, but to be frank I am probably being too generous in estimating a lower bound.&lt;/p&gt;&#xA;&lt;p&gt;There are two lessons here: 1. Just because you and others can't think of a way to break your secure system doesn't mean it's actually secure. 2. Modern ML techniques are not strictly stronger than handcrafted algorithms, though I would also imagine that a simple NN would be able to solve this problem easily.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-07-05T15:56:11.640" LastActivityDate="2020-07-05T15:56:11.640" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22351" PostTypeId="1" CreationDate="2020-07-05T16:49:19.937" Score="0" ViewCount="223" Body="&lt;p&gt;Any simple example of a self-learning model (any business use case, banking)? I have found the terms &lt;a href=&quot;https://www.spotintelligence.com/blog/how-to-implement-self-learning-systems/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.absolutdata.com/blog/4-ways-self-learning-algorithms-are-impacting-business-and-customers-dont-even-realize-it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="38392" LastEditorUserId="2444" LastEditDate="2020-07-06T12:19:46.073" LastActivityDate="2020-07-06T12:19:46.073" Title="What is a self-learning model?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;self-supervised-learning&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22353" PostTypeId="1" AcceptedAnswerId="22363" CreationDate="2020-07-05T21:07:30.973" Score="0" ViewCount="87" Body="&lt;p&gt;Suppose that we want to train a car to drive in the real world and decide to use Reinforcement Learning (specifically, DQN) for that. I am a bit confused about how training generally works.&lt;/p&gt;&#xA;&lt;p&gt;Is it that we are exploring the environment at the same time that we are training the Q network? If so, is there not a way to train the Q network before actually going out into the real world? And then, aren't there millions of possible states in the real world? So, how does RL or I guess the neural network generalize so that it can function during rush hour, empty roads, etc.&lt;/p&gt;&#xA;" OwnerUserId="37947" LastEditorUserId="2444" LastEditDate="2020-07-05T23:38:57.170" LastActivityDate="2020-07-06T15:14:30.390" Title="How does training for DQN work if messing up in the environment in costly?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22354" PostTypeId="1" CreationDate="2020-07-06T03:30:24.667" Score="1" ViewCount="32" Body="&lt;p&gt;Say we have a simple gray scale image. If we use a filter which is just the 3x3 identity matrix (or more pointedly the identity matrix but with -1 instead of the 0 entries), it is fairly easy to see how applying this filter with stride length 1 and padding of 1 would produce an image of the same size that represents the presence of north-west diagonals in the input image.&lt;/p&gt;&#xA;&lt;p&gt;As I am reading more about the generative networks in GAN paradigms, I am learning that 'transposed convolutions' are used to turn gaussian noise into meaningful images like a human face. However, when I try to look at sources for transposed convolutions, most articles address the upscaling use of these convolutions, rather than their 'generative' properties. Also it is not clear to me that upscaling is even necessary in these applications, since we could start with noise that has the same resolution as our desired output.&lt;/p&gt;&#xA;&lt;p&gt;I am asking for an example, article, or paper that can provide me with more understanding as to the feature generation aspect of transposed convolutions. I have found &lt;a href=&quot;https://www.machinecurve.com/index.php/2019/09/29/understanding-transposed-convolutions/#applications-of-transposed-convolutions&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; interesting article that relates the word 'transpose' to the transpose of a matrix. I have a good background in linear algegra, and I understand how the transpose would swap the dimensions of the input/output. This has obvious relation to upscaling/downscaling, but this effect would happen if we replaced the m x n matrix with any other n x m matrix, not specifically just the transpose. Essentially, I'm not sure how actual transpose functor can go from detecting a given feature associated to a convolutional filter, to &lt;em&gt;producing&lt;/em&gt; that same feature&lt;/p&gt;&#xA;&lt;p&gt;EDIT: I've done some thinking at it is clear to me now how the transpose matrix will produce an 'input image' that has the features specified by a given feature map. That is if &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is the matrix given by the convultion operation, and &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is a feature map then&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$M^T F$$&lt;/span&gt;&#xA;will produce an image with the corresponding features. Its obviously not a perfect inverse operation, but it works. However I still don't yet see how to interpret this transposed matrix as a convolution of its own&lt;/p&gt;&#xA;" OwnerUserId="38407" LastEditorUserId="38407" LastEditDate="2020-07-06T04:34:25.827" LastActivityDate="2020-07-06T04:34:25.827" Title="Concrete example of how transposed convolutions are able to *add* features to an image" Tags="&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;image-generation&gt;&lt;transpose-convolution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22355" PostTypeId="1" AcceptedAnswerId="22555" CreationDate="2020-07-06T04:04:53.040" Score="0" ViewCount="122" Body="&lt;p&gt;Theoretically, number of units for a LSTM layer is the number of hidden states or the max length of sequences as per my practice.&lt;/p&gt;&#xA;&lt;p&gt;For example, in Keras:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Lstm1 = LSTM(units=MAX_SEQ_LEN, return_sequences=False);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, with &lt;strong&gt;lots of sequences to train&lt;/strong&gt;, should I add more LSTM layers? because increasing MAX_SEQ_LEN is not the way as it doesn't help make the network better since the extra number of hidden states isn't useful any more.&lt;/p&gt;&#xA;&lt;p&gt;I'm considering increasing number of LSTM layers, but &lt;strong&gt;how many are enough?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example, 3 of them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Lstm1 = LSTM(units=MAX_SEQ_LEN, return_sequences=True);&#xA;Lstm2 = LSTM(units=MAX_SEQ_LEN, return_sequences=True);&#xA;Lstm3 = LSTM(units=MAX_SEQ_LEN, return_sequences=False);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-07-18T01:19:14.920" Title="Number of LSTM layers needed to learn a certain number of sequences" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;hidden-layers&gt;&lt;network-design&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22356" PostTypeId="1" CreationDate="2020-07-06T07:39:27.250" Score="0" ViewCount="38" Body="&lt;p&gt;Do you know any lab in the world (the most reliable one) where one can apply for a Ph.D. in the domain of innovation using AI for education and human learning purpose?&lt;/p&gt;&#xA;&lt;p&gt;I heard about Apple and Google invested this new field but it's hard to find a research lab. I also took a look at Cambridge and Oxford Google research didn't provide any specific lab.&lt;/p&gt;&#xA;" OwnerUserId="37267" LastEditorUserId="2444" LastEditDate="2020-07-06T12:25:04.130" LastActivityDate="2020-07-06T12:25:04.130" Title="Is there a lab where one can apply for a Ph.D. in the domain of innovation using AI for education and human learning purpose?" Tags="&lt;education&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22358" PostTypeId="1" CreationDate="2020-07-06T09:53:56.790" Score="4" ViewCount="2010" Body="&lt;p&gt;I wish to train two domain-specific models:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Domain 1: Constitution and related Legal Documents&lt;/li&gt;&#xA;&lt;li&gt;Domain 2: Technical and related documents.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For Domain 1, I've access to a text-corpus with texts from the constitution and no question-context-answer tuples. For Domain 2, I've access to Question-Answer pairs.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to fine-tune a light-weight BERT model for Question-Answering using just the data mentioned above?&lt;/p&gt;&#xA;&lt;p&gt;If yes, what are the resources to achieve this task?&lt;/p&gt;&#xA;&lt;p&gt;Some examples, from the huggingface/models library would be mrm8488/bert-tiny-5-finetuned-squadv2, sshleifer/tiny-distilbert-base-cased-distilled-squad, /twmkn9/albert-base-v2-squad2.&lt;/p&gt;&#xA;" OwnerUserId="38411" LastEditorUserId="2444" LastEditDate="2021-01-26T15:34:58.237" LastActivityDate="2021-01-26T15:34:58.237" Title="How to fine tune BERT for question answering?" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;&lt;fine-tuning&gt;&lt;question-answering&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22360" PostTypeId="2" ParentId="22347" CreationDate="2020-07-06T10:17:53.160" Score="1" Body="&lt;p&gt;&amp;quot;Selecting the model&amp;quot; in this case refers to selecting the hyperparameters of the model. The reason to use a nested CV is simply to avoid overfitting training data.&lt;/p&gt;&#xA;&lt;p&gt;Consider the example in the link. First you like to select the best hyperparameters of your &lt;code&gt;svm&lt;/code&gt; model by &lt;code&gt;GridSearchCV()&lt;/code&gt;. This is done by 4-fold CV. Now the &lt;code&gt;clf.best_score_&lt;/code&gt; will be the mean cross-validated score of the best estimator (the model with the best hyperparameters). However now you used the same data for training and reporting the performance, although you used CV. Keep in mind that the folds are not independent. Therefore the hyperparameters might be too data specific, i.e. your generalization error estimates are too optimistic. Therefore we like to evaluate our final model performance outside / independent of the hyperparameter selection loop / process (the call &lt;code&gt;cross_val_score()&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In the provided plot, you can clearly see that the reported performance by &lt;code&gt;GridSearchCV()&lt;/code&gt; is most of the time better than the performance reported by &lt;code&gt;cross_val_score()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-07-06T10:17:53.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22361" PostTypeId="1" CreationDate="2020-07-06T11:27:49.380" Score="2" ViewCount="87" Body="&lt;p&gt;I am learning computer vision. When I was going through implementations of various computer vision projects, some OCR problems used GRU or LSTM, while some did not. I understand that RNNs are used only in problems where input data is a sequence, like audio or text.&lt;/p&gt;&#xA;&lt;p&gt;So, in &lt;a href=&quot;https://www.kaggle.com/c/digit-recognizer/notebooks&quot; rel=&quot;nofollow noreferrer&quot;&gt;kernels of MNIST on kaggle&lt;/a&gt; almost no kernel has used RNNs and almost every repository for &lt;a href=&quot;https://github.com/search?q=iam%20handwriting&quot; rel=&quot;nofollow noreferrer&quot;&gt;OCR on IAM dataset on GitHub&lt;/a&gt; has used GRU or LSTMs. Intuitively, written text in an image is a sequence, so RNNs were used. But, so is the written text in MNIST data. So, when exactly is it that RNNs(or GRUs or LSTMs) need to be used in computer vision and when don't?&lt;/p&gt;&#xA;" OwnerUserId="38060" LastEditorUserId="38060" LastEditDate="2020-07-06T12:34:11.903" LastActivityDate="2020-07-07T12:57:29.543" Title="Why are RNNs used in some computer vision problems?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;recurrent-neural-networks&gt;&lt;r-cnn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22362" PostTypeId="2" ParentId="22358" CreationDate="2020-07-06T12:10:24.743" Score="1" Body="&lt;p&gt;The answer is yes but 'lightweight' will require a 'lightweight' model.&lt;/p&gt;&#xA;&lt;p&gt;Your application for 'domain one' is called open domain question answering (ODQA).  Here is a demonstration of ODQA using BERT: &lt;a href=&quot;https://www.pragnakalp.com/demos/BERT-NLP-QnA-Demo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pragnakalp.com/demos/BERT-NLP-QnA-Demo/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your application for 'domain two' is a little different.  It is about learning sequences from sequences.  More specifically these are called sequence to sequence models.  &lt;a href=&quot;https://mccormickml.com/2020/03/10/question-answering-with-a-fine-tuned-BERT/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is an example using a pre-trained BERT model fine-tuned on the Stanford Question Answering (SQuAD) dataset.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;a href=&quot;https://rajpurkar.github.io/SQuAD-explorer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford Question Answering Dataset (SQuAD)&lt;/a&gt; is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In both applications, the resources required are going to rely on the performance you require.  There are many sizes of BERT models.  Generally, the larger the model, the larger the GPU memory requirements, and the higher the performance (i.e. accuracy, precision, recall, F1-score, etc.).  For example, I can run BERT Base on a particular data set on a GTX 1080Ti and a RTX 2080Ti but not BERT Large.&lt;/p&gt;&#xA;&lt;p&gt;This article, &lt;a href=&quot;https://blog.exxactcorp.com/nvidia-quadro-rtx-6000-bert-large-fine-tune-benchmarks-with-squad-dataset/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NVIDIA Quadro RTX 6000 BERT Large Fine-tune Benchmarks with SQuAD Dataset&lt;/a&gt; shows performance for BERT using TensorFlow on four NVIDIA Quadro RTX 6000 GPUs.&lt;/p&gt;&#xA;&lt;p&gt;There is a 'mobile' version of BERT called MobileBERT for running on small devices like smartphones.  Here is an article on using that with SQuAD: &lt;a href=&quot;https://www.tensorflow.org/lite/models/bert_qa/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/lite/models/bert_qa/overview&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;cdQA-suite is a good package.  The following should help in fine-tuning on your own corpus:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/how-to-create-your-own-question-answering-system-easily-with-python-2ef8abc8eb5&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to create your own Question-Answering system easily with python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/cdqa-suite/cdQA/tree/master/examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;cdQA examples&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-07-06T14:07:59.307" LastActivityDate="2020-07-06T14:07:59.307" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="22363" PostTypeId="2" ParentId="22353" CreationDate="2020-07-06T12:32:26.987" Score="1" Body="&lt;p&gt;In general, you need to actively explore the environment to gather data to train your Q network. However, especially in your self-driving car example, you might be looking for Batch RL. In Batch RL you start with a given, fixed dataset of transitions (state, action, reward, next state) and you learn a policy (or Q function) based on the dataset without exploring. The Batch-Constrained Q-learning algorithm (BCQ) is a good example of this.&lt;/p&gt;&#xA;&lt;p&gt;With regards to acquiring the data necessary for Batch RL, that's another story. I suppose one could collect data through sensors in a car, and label datapoints with rewards according to some reward function.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastEditorUserId="37829" LastEditDate="2020-07-06T15:14:30.390" LastActivityDate="2020-07-06T15:14:30.390" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22364" PostTypeId="2" ParentId="21559" CreationDate="2020-07-06T13:28:38.153" Score="1" Body="&lt;p&gt;Your idea is a good one.&lt;/p&gt;&#xA;&lt;p&gt;Another idea is to upsample or aggregate your data.  For example, average by week if you generally have a couple of missing days in every week.&lt;/p&gt;&#xA;&lt;p&gt;A similar question on Stack Exchange:&#xA;&lt;a href=&quot;https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task&quot;&gt;https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-07-06T13:28:38.153" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22365" PostTypeId="1" CreationDate="2020-07-06T14:47:11.363" Score="0" ViewCount="35" Body="&lt;p&gt;Where can I find a training text for some specific intents in English? For example, the intent of asking for an appointment, or asking for contact info exchange?&lt;/p&gt;&#xA;&lt;p&gt;Is there any recommended source of such classified text?&lt;/p&gt;&#xA;" OwnerUserId="33135" LastEditorUserId="2444" LastEditDate="2020-07-06T16:34:39.197" LastActivityDate="2020-07-06T16:34:39.197" Title="Where can I find a training text for some specific intents in English?" Tags="&lt;training&gt;&lt;spacy&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22366" PostTypeId="1" CreationDate="2020-07-06T15:43:48.843" Score="0" ViewCount="41" Body="&lt;p&gt;I am having a training data set for a time-series dataset like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;train_df = pd.DataFrame(np.random.randint(0,100,size=(100, 16)))&#xA;train_df.columns=['var1(t-3)','var2(t-3)','var3(t-3)','var4(t-3)','var1(t-2)','var2(t-2)','var3(t-2)','var4(t-2)','var1(t-1)','var2(t-1)','var3(t-1)','var4(t-1)','var1(t)','var2(t)','var3(t)','var4(t)']&#xA;train_X=train_df.drop(['var1(t)'],axis=1)&#xA;train_y=train_df[['var1(t)']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So as you see for training I am inputting the network with past three timesteps of the all 4 variables and the current timestep of the three remaining variables which makes up to 15 variables.&lt;/p&gt;&#xA;&lt;p&gt;I want to train this with an LSTM with functional API in Keras because I cannot use sequential API in my case due to different input size. So I tried the below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow&#xA;from tensorflow.keras.utils import plot_model&#xA;from tensorflow.keras.models import Model&#xA;from tensorflow.keras.layers import Input&#xA;from tensorflow.keras.layers import Dense&#xA;from tensorflow.keras.layers import LSTM&#xA;visible = Input(shape=(100,15))&#xA;hidden1 = LSTM(10)(visible)&#xA;hidden2 = Dense(10, activation='relu')(hidden1)&#xA;output = Dense(1, activation='sigmoid')(hidden2)&#xA;model = Model(inputs=visible, outputs=output)&#xA;model.compile(loss='mae', optimizer='adam')&#xA;&#xA;&#xA;history = model.fit(train_X, train_y, batch_size=64, epochs=2, validation_split=0.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this throws the error like below:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ValueError: Input 0 of layer lstm_2 is incompatible with the layer:&#xA;expected ndim=3, found ndim=2. Full shape received: [None, 15]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then I reshape the train and test like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;train_X=train_X.reshape(train_X.shape[0],1,train_X.shape[1])&#xA;train_y = train_y.reshape(train_y.shape[0],1)&#xA;&#xA;&#xA;train_X shape is (100,15)&#xA;train_y shape is (100,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this gave a very huge MAPE and RMSE.So I think my reshaping is wrong here.&lt;/p&gt;&#xA;&lt;p&gt;Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="38418" LastActivityDate="2020-07-06T15:43:48.843" Title="How to feed the LSTM with different length for the latest time step?" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22367" PostTypeId="1" AcceptedAnswerId="22406" CreationDate="2020-07-06T16:32:41.237" Score="0" ViewCount="56" Body="&lt;p&gt;I have a confusion on where exactly is the L2 regularization (weight decay) is added.&lt;/p&gt;&#xA;&lt;p&gt;In various resources I have come across, I find two equations where L2 regularization is applied.&lt;/p&gt;&#xA;&lt;p&gt;Adding R(W) to loss function makes sense because it tries decrease large weights. Also, I have seen equations where we add R(W) to the weight update term, 2nd equation in 2nd line as shown in this image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n0EdM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n0EdM.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the above image, using the weight update rule that&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;W(final) = W(initial) + (alpha) * (Gradient of W),&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I obtain a different equation as compared to the other equation which is commonly written in various resources.&lt;/p&gt;&#xA;&lt;p&gt;Where exactly is the regularization term added, I previously thought it was only added in Loss function but that gives me a different weight update equation from what is commonly presented in resources.( Or is my interpretation of the equation wrong? )&lt;/p&gt;&#xA;&lt;p&gt;I presume it is also added in weight update equation because while constructing models, we add regularization term.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.add(Conv2D(256, (5,5), padding=&amp;quot;same&amp;quot;, kernel_regularizer=l2(reg))) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Would be grateful for any help.&lt;/p&gt;&#xA;" OwnerUserId="35616" LastEditorUserId="35616" LastEditDate="2020-07-08T17:01:29.650" LastActivityDate="2020-07-08T19:12:05.910" Title="Where is L2-regularization term applied" Tags="&lt;regularization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22368" PostTypeId="1" CreationDate="2020-07-06T18:47:37.357" Score="0" ViewCount="27" Body="&lt;p&gt;I am trying to see whether there are differences in sentences between the topics in 2 languages ie English and Spanish. Eg. (Face masks are mandatory, la mascarilla es obligatoria) but this stretched out to n number of queries. The final goal is to find whether a document in English talks about 1 topic and the other corpus in Spanish talks about a different one.&lt;/p&gt;&#xA;&lt;p&gt;I tried using BiLDA for generating topics in both the languages simultaneously but my data is not super clean for the model to work properly and was giving me vague results.&lt;/p&gt;&#xA;&lt;p&gt;I then went on towards BERT and TRANSFORMERS by using this wrapper &lt;a href=&quot;https://github.com/amaiya/ktrain&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/amaiya/ktrain&lt;/a&gt; for applying the pre-trained models to classify text by giving my own topics to the model only realizing that it would not work for Spanish. I also tried to look at FACEBOOK'S LASER and thought about comparing the embedding space but again I am relying on their pre-trained space and would not get my intended goal.&lt;/p&gt;&#xA;&lt;p&gt;Tried using transformers from huggingface. I used the automodelforsequenceclassification with zero shot learning which totally works for English, But it uses Facebook's bart mnli which only supports 1 language. I need support for multi language. If the same was working for spanish my job was done.&lt;/p&gt;&#xA;&lt;p&gt;Zero-shot learnings helps me classify text into custom topics although it only works with English language.no support for other languages.&lt;/p&gt;&#xA;&lt;p&gt;If anybody has done anything like that or can help me direct this it would be really helpful.&lt;/p&gt;&#xA;" OwnerUserId="10118" LastActivityDate="2020-07-06T18:47:37.357" Title="Comparing English and Spanish Queries using NLP" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;bert&gt;&lt;machine-translation&gt;&lt;language-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22369" PostTypeId="1" CreationDate="2020-07-06T23:30:18.850" Score="1" ViewCount="149" Body="&lt;p&gt;I am trying to build a recurrent neural network from scratch. It's a very simple model. I am trying to train it to predict two words (dogs and gods). While training, the value of cost function starts to increase for some time, after that, the cost starts to decrease again, as can be seen in the figure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ak7oj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ak7oj.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using the gradient descend method for optimization. Decreasing the step size/learning rate does not change the behavior. I have checked the code and math, again and again, I don't think there is an error (I could be wrong).&lt;/p&gt;&#xA;&lt;p&gt;Why is the cost function not decreasing monotonically? Could there be a reason other than an error in my code/math? If there is an error, do you think that it is just a coincidence that each time the system finally converges to a very small value of error?&lt;/p&gt;&#xA;&lt;p&gt;I am a beginner in the field of machine learning, hence, many questions I have asked may seem foolish to you. I am saving the values after every 100 iterations so the figure is actually for 15000 iterations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;About training:&lt;/strong&gt;&#xA;I am using one-hot encoding. As the training data has only two samples (&amp;quot;gods&amp;quot; and &amp;quot;dogs&amp;quot;), where each alphabet is represented as d=[1,0,0,0],o=[0,1,0,0],g=[0,0,1,0],s=[0,0,0,1]. The recurrent neural network (RNN) goes back to a maximum of 3 time units, (e.g for dogs, the first input is 'd', then 'o', followed by 'g' and s). So, for the second input, the RNN goes back to 1 input, for the third input the RNN observes both previous inputs and so on. After calculating the gradients for the word &amp;quot;dogs&amp;quot;, the values of the gradients are saved and the process is repeated for the word &amp;quot;gods&amp;quot;. The gradients calculated for the second input &amp;quot;gods&amp;quot; are summed with the gradients calculated for &amp;quot;dogs&amp;quot; at the end of each epoch/iteration, and then the sum is used to updated all the weights. In each epoch, the inputs remain the same i.e &amp;quot;gods&amp;quot; and &amp;quot;dogs&amp;quot;. In mini-batch training, in some epoch the RNN may encounter new inputs, hence, the loss may increase. However, I do not think that what I am doing qualifies as mini-batch training as there are only two inputs, both inputs are used in each epoch, and the sum of calculated gradients is used to update the weights.&lt;/p&gt;&#xA;" OwnerUserId="38423" LastEditorUserId="38423" LastEditDate="2020-07-07T18:11:51.283" LastActivityDate="2021-04-03T21:01:30.230" Title="Why the cost/loss starts to increase for some iterations during the training phase?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22370" PostTypeId="2" ParentId="22369" CreationDate="2020-07-07T00:24:18.207" Score="0" Body="&lt;p&gt;In general, there's nothing wrong with training loss to increase from time-to-time during training.&lt;/p&gt;&#xA;&lt;p&gt;This is because GD with minibatch is a stochastic process and doesn't guarantee that the loss will decrease at each step.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-07-07T00:24:18.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22371" PostTypeId="1" CreationDate="2020-07-07T02:16:39.217" Score="0" ViewCount="34" Body="&lt;p&gt;For learning a single sequence, LSTM only should suffice.&lt;/p&gt;&#xA;&lt;p&gt;However, my situation is different here. I have a &lt;strong&gt;list of sequences&lt;/strong&gt; to learn:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The sale volumes of 12 months, these are the sequences&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And &lt;strong&gt;each sequence above belongs to a category&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'm trying it out by consider [category,sequence] as a sequential sample, the loss can be reduced to 1% but it gives wrong values in inferring real data.&lt;/p&gt;&#xA;&lt;p&gt;The second try is considering [category,sequence] as a sample of 2 inputs:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;X1 = sequence&lt;/li&gt;&#xA;&lt;li&gt;X2 = category&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Feed the sequence thru' LSTM layers to get &lt;strong&gt;H&lt;/strong&gt;, and then concat with &lt;strong&gt;X2&lt;/strong&gt;, and feed again the pair &lt;strong&gt;[H,X2]&lt;/strong&gt; thru' some &lt;strong&gt;dense layers&lt;/strong&gt;, the results aren't better.&lt;/p&gt;&#xA;&lt;p&gt;Any popular solutions (network shape, network design) for learning this kind of data: &lt;strong&gt;sequential data in different categories&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-07-07T02:16:39.217" Title="Network design to learn multiple sequences of multiple categories" Tags="&lt;neural-networks&gt;&lt;long-short-term-memory&gt;&lt;sequence-modelling&gt;&lt;categorical-data&gt;&lt;network-design&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22372" PostTypeId="1" CreationDate="2020-07-07T08:39:14.497" Score="1" ViewCount="25" Body="&lt;p&gt;I'm studying RL with Sutton and Barto's book. I'd like to ask about the order of execution of a statement in the algorithm below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ryuYd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ryuYd.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; (importance sampling ratio) is updated at the end of the &lt;code&gt;For&lt;/code&gt; loop.&lt;/p&gt;&#xA;&lt;p&gt;But, I think that updating should be located after calculating &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; (the return) and before updating &lt;span class=&quot;math-container&quot;&gt;$C(s,a)$&lt;/span&gt; (cumulative of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;). This seems to be right considering the second picture below, which I found in &lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node56.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://incompleteideas.net/book/first/ebook/node56.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xmJQu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xmJQu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is Sutton and Barto's book wrong? Or the two algorithms in Sutton and Barto's book and seconds picture are actually the same, and I am wrong? Is there any difference between the two when implemented? If I am wrong, can you explain the reason?&lt;/p&gt;&#xA;" OwnerUserId="38430" LastEditorUserId="2444" LastEditDate="2020-07-07T12:35:44.323" LastActivityDate="2020-07-07T12:35:44.323" Title="Should the importance sampling ratio be updated at the end of the for loop in the off-policy Monte Carlo control algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22373" PostTypeId="1" CreationDate="2020-07-07T09:22:15.887" Score="1" ViewCount="24" Body="&lt;p&gt;I'm reading this paper &lt;a href=&quot;https://www.aclweb.org/anthology/P18-1135.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Global-Locally Self-Attentive Dialogue State Tracker&lt;/a&gt; and follow through the implementation published in &lt;a href=&quot;https://github.com/salesforce/glad&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLAD&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if someone can clarify what variable or score is used to calculate the global and local self-attention scores in Figure 4 (the heatmap).&lt;/p&gt;&#xA;&lt;p&gt;For me, it is not really clear how to derive these scores. The only score that would match the given dimension would be in the scoring module  &lt;span class=&quot;math-container&quot;&gt;$p_{utt}=softmax(a_{utt})$&lt;/span&gt;. However, I do not see in their implementation that anything is done with this value.&lt;/p&gt;&#xA;&lt;p&gt;So, what I did was the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; q_utts = []&#xA; a_utts=[]&#xA; for c_val in C_vals:&#xA;     q_utt, a_utt = attend(H_utt, c_val.unsqueeze(0).expand(len(batch), *c_val.size()), lens=utterance_len)&#xA;     q_utts.append(q_utt)&#xA;     a_utts.append(a_utt)&#xA;attention_score= torch.mean(torch.stack(a_utts,dim=1),dim=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the resulting attention score differs very much from what I expect.&lt;/p&gt;&#xA;" OwnerUserId="35910" LastEditorUserId="2444" LastEditDate="2020-07-07T12:38:37.323" LastActivityDate="2020-07-07T12:38:37.323" Title="What is the score used to visualize attention in this paper?" Tags="&lt;natural-language-processing&gt;&lt;papers&gt;&lt;implementation&gt;&lt;dialogue-systems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22375" PostTypeId="1" CreationDate="2020-07-07T10:11:22.407" Score="1" ViewCount="83" Body="&lt;p&gt;I have many text documents and I want to identify concepts in these documents in an unsupervised manner. One of my problems is that the concepts can be bigrams, trigrams, or even longer.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, out of all the bigrams, how can I identify the ones that are more likely to represent a concept?&lt;/p&gt;&#xA;&lt;p&gt;A concept could be &amp;quot;machine learning&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Are you aware of any standard approaches to solve this problem?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; The corpus I am working with consists of papers accessed from web of science. That is, they are all in some given domain niche. I want to extract words, bigrams, trigrams... that represent common concepts/buzzwords from these papers. These could be &amp;quot;Automated machine learning&amp;quot;, &amp;quot;natural language processing&amp;quot; et cetera. I need to be able to distinguish these from other common n-grams such as &amp;quot;New York&amp;quot;, &amp;quot;Barack Obama&amp;quot;,...&lt;/p&gt;&#xA;&lt;p&gt;I know that I could do this using a NER approach but this would require hand-labelling. Are you aware of any unsupervised ways to approach this problem? Or even a semi-superised method with little labelled data?&lt;/p&gt;&#xA;" OwnerUserId="38434" LastEditorUserId="38434" LastEditDate="2020-07-08T11:38:31.437" LastActivityDate="2021-04-04T13:02:45.263" Title="How can I identify bigrams and trigrams that represent concepts?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22376" PostTypeId="2" ParentId="22361" CreationDate="2020-07-07T12:57:29.543" Score="3" Body="&lt;p&gt;There are tasks in computer vision where recurrent neural networks (RNNs) can be useful because there's some sequential sub-task in the main task.&lt;/p&gt;&#xA;&lt;p&gt;For instance, in the paper &lt;a href=&quot;https://openaccess.thecvf.com/content_cvpr_2015/html/Donahue_Long-Term_Recurrent_Convolutional_2015_CVPR_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Long-Term Recurrent Convolutional Networks for Visual Recognition and Description&lt;/a&gt;, the authors investigate the use of a neural network that is both recurrent and convolutional to solve certain computer vision tasks that also have a sequential component/part, such as &lt;strong&gt;video recognition tasks&lt;/strong&gt;, &lt;strong&gt;image to sentence generation problems&lt;/strong&gt;, and &lt;strong&gt;video narration challenges&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are other papers that investigate the combination of convolutional and recurrent layers, such as &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01551/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition&lt;/a&gt;, which also has a biological motivation.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-07T12:57:29.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22377" PostTypeId="2" ParentId="22369" CreationDate="2020-07-07T13:22:21.393" Score="0" Body="&lt;p&gt;Since neural networks rely on stochasticity (i.e. randomness) to initialize their parameters and gradient descent selects random batches of training data at each iteration, is perfectly normal if the value of loss function fluctuates instead of decreasing monotonically.&lt;/p&gt;&#xA;" OwnerUserId="38411" LastActivityDate="2020-07-07T13:22:21.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22378" PostTypeId="1" CreationDate="2020-07-07T14:29:38.917" Score="1" ViewCount="333" Body="&lt;p&gt;I am confused as to why the sequence length is the first dimension of the input tensor for an RNN, while the batch size is the first dimension for any other kind of network (linear, CNN, etc.).&lt;/p&gt;&#xA;&lt;p&gt;This makes me think that I haven't fully grasped the concept of RNN batches. Is each independent batch a different sequence? And is the same hidden state across batches? Is the hidden state maintained between timesteps for a given sequence (for vanilla/truncated BPTT)?&lt;/p&gt;&#xA;" OwnerUserId="38445" LastEditorUserId="2444" LastEditDate="2020-12-05T03:16:58.807" LastActivityDate="2021-01-04T04:04:00.283" Title="In PyTorch, why does the sequence length need to be provided as the first dimension of the input tensor for an RNN?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22379" PostTypeId="2" ParentId="20585" CreationDate="2020-07-07T15:06:29.760" Score="3" Body="&lt;p&gt;Deep Q Learning is a model-free algorithm. In the case of Go (and chess for that matter) the model of the game is very simple and deterministic. It's a perfect information game, so it's trivial to predict the next state given your current state and action (this is the model). They take advantage of this with MCTS to speed up training. I suppose Deep Q Learning would also work, but it would be at a huge disadvantage.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-07T15:06:29.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22380" PostTypeId="1" AcceptedAnswerId="22396" CreationDate="2020-07-07T15:10:02.380" Score="3" ViewCount="161" Body="&lt;p&gt;How is the notion of immediate &lt;em&gt;reward&lt;/em&gt; used in the reinforcement learning different from the notion of a &lt;em&gt;label&lt;/em&gt; we find in the supervised learning problems?&lt;/p&gt;&#xA;" OwnerUserId="38447" LastEditorUserId="38447" LastEditDate="2020-07-07T15:38:00.963" LastActivityDate="2020-07-08T10:28:31.437" Title="How is the reward in reinforcement learning different from the label in supervised learning problems?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;rewards&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22381" PostTypeId="1" CreationDate="2020-07-07T16:25:28.417" Score="3" ViewCount="57" Body="&lt;p&gt;I'm trying to train some deep RL agents using policy gradient methods like AC and PPO. While training, I have a ton of different metrics being monitored.&lt;/p&gt;&#xA;&lt;p&gt;I understand that the ultimate goal is to maximize the reward or return per episode.&#xA;But there are a ton of other metrics that I don't understand what they are used for.&lt;/p&gt;&#xA;&lt;p&gt;In particular, how should one interpret the mean and standard deviation curves of the policy loss, value, value loss, entropy, and reward/return over time while training?&lt;/p&gt;&#xA;&lt;p&gt;What does it mean when these values increase or decrease over time? Given these curves, how would one decide how to tune hyperparameters, see where the training is succeeding and failing, and the like?&lt;/p&gt;&#xA;" OwnerUserId="38448" LastEditorUserId="2444" LastEditDate="2020-07-07T22:47:09.670" LastActivityDate="2020-08-07T16:00:40.047" Title="How should we interpret all the different metrics in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;policy-gradients&gt;&lt;metric&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22382" PostTypeId="2" ParentId="22375" CreationDate="2020-07-07T16:46:03.170" Score="0" Body="&lt;p&gt;One of the renowned methods is using the TF-IDF method. You can define a lower bound for the TF-IDF score of the bigram or trigram to identify it as a keyword. &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;, you can find more information about it.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2020-07-07T16:46:03.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22383" PostTypeId="1" CreationDate="2020-07-07T17:52:47.697" Score="0" ViewCount="102" Body="&lt;p&gt;In his paper introducing SHA-RNN (&lt;a href=&quot;https://arxiv.org/pdf/1911.11423.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1911.11423.pdf&lt;/a&gt;) Stephen Merity states that neglecting one direction of research (in this case LSTMs) over another (transformers) merily because the SOTA in transformers are due to using more computing power is not the way to go.&lt;/p&gt;&#xA;&lt;p&gt;I agree that finding neat tricks in AI/ML is equally (if not more) important than just throwing more computing power at the problem. However I am a little bit confused.&lt;/p&gt;&#xA;&lt;p&gt;The main difference (since they both use attention units) between his SHA-RNN and transformers seem to be the fact that SHA-RNN uses LSTMs to &amp;quot;encode the position of words&amp;quot;, where transformers do position encoding by using cosine and sine functions.&lt;/p&gt;&#xA;&lt;p&gt;My confusion comes from the fact that LSTMs need to be handled sequentially and thus they cannot use this large advantage of GPUs, being able to compute things in parallel, whilst transformers can. Wouldn't this mean that (assuming LSTMs and positional encoding are able to acquire the same results), training using LSTMs would take longer than transformers and thus need more computing power, thus defeating the initial puporse of this paper? Or am I misinterpreting this?&lt;/p&gt;&#xA;&lt;p&gt;Basically my question comes down to &amp;quot;Why would an SHA-RNN be less computationally expensive than a transformer?&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="34359" LastActivityDate="2021-04-04T01:07:05.617" Title="What is the big fuzz about SHA-RNN versus Transformers?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22384" PostTypeId="2" ParentId="20585" CreationDate="2020-07-07T20:03:42.870" Score="6" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning (and also its deep variant, and most of the other well-known reinforcement learning algorithms) are inherently learning approaches for &lt;strong&gt;single-agent environments&lt;/strong&gt;. The entire problem setting that these algorithms are developed for (Markov decision processes, or MDPs) is always framed in terms of a single agent situated in some environment, where that agent can take actions that have some level of influence over the states that they lead to, and rewards may be observed.&lt;/p&gt;&#xA;&lt;p&gt;If you have a problem that is, in reality, a multi-agent environment, there is a way to translate this environment to a single-agent setting; you simply have to assume that all other agents (i.e. your opponent in Go) are an inherent part of &amp;quot;the world&amp;quot; or &amp;quot;the environment&amp;quot;, and that all the states in which these other agents make moves are not really states (not visible to your agent), but just intermediate steps where these part-of-the-environment-agents cause the environment to change and, as a result, create state transitions.&lt;/p&gt;&#xA;&lt;p&gt;The primary issue with this approach is; we still need to model the decision-making of these agents in order to implement this new view of &amp;quot;the world&amp;quot;, where our opponents are really a part of the world. Whatever implementation we give them, that is what our single-agent RL algorithm will learn to play against. We can just implement our opponents to be random agents, and run a single-agent RL algorithm like DQN, and then we will likely learn to play well against random agents. We'll probably still be very bad against strong opponents though. &lt;strong&gt;If we want to use a single-agent RL algorithm to learn to play well against strong opponents, we need to have an implementation for those strong opponents first.&lt;/strong&gt; But if we already have that... why even bother with learning at all? We've already got the strong Go player, so we're already done and don't need to learn!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;MCTS&lt;/strong&gt; is a tree search algorithm, one that actively takes into account the fact that there is an opponent with opposing goals, and tries to model the choices that this opponent can make, and can do so better the more computation time we give it. &lt;strong&gt;This algorithm, and learning approaches built around it, are inherently designed to tackle the multi-agent setting&lt;/strong&gt; (with agents having opposing goals).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-07-07T20:03:42.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22385" PostTypeId="1" CreationDate="2020-07-07T21:31:44.717" Score="1" ViewCount="95" Body="&lt;p&gt;I am a student learning machine learning recently, and one thing is keep confusing me, I tried multiple sources and failed to find the related answer.&lt;/p&gt;&#xA;&lt;p&gt;As following table shows (this is from some paper):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Cf2qZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Cf2qZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is it possible that every class has a higher recall than precision for multi-class classification?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Recall can be higher than precision over some class or overall performance which is common, but is it possible to keep recall greater than precision for every class?&lt;/p&gt;&#xA;&lt;p&gt;The total amount of test data is fixed, so, to my understanding, if the recall is greater than the precision for one class, it is a must that the recall must be smaller than the precision for some other classes.&lt;/p&gt;&#xA;&lt;p&gt;I tried to make a fake confusion matrix to simulate the result, but I failed. Can someone explain it to me?&lt;/p&gt;&#xA;&lt;h3&gt;this is a further description:&lt;/h3&gt;&#xA;&lt;p&gt;Assume we have classified 10 data into 3 classes, and we have a confusion matrix like this,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sQpQt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sQpQt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&#xA;if we want to keep recall bigger than precision over each class (this case 0,1,2) respectively, we need to keep:&lt;br /&gt;&#xA;x1+x2 &amp;lt; x3+x5&lt;br /&gt;&#xA;x3+x4 &amp;lt; x1+x6&lt;br /&gt;&#xA;x5+x6 &amp;lt; x2+x4&lt;/p&gt;&#xA;&lt;p&gt;There is a conflict, because the sum of the left side equals to the sum of the right side in these inequalities, and the sum(x1...x6) = 10 - sum(a,b,c) in this case.&lt;/p&gt;&#xA;&lt;p&gt;Hence, I think to get recall higher than precision on all classes is not feasible, because the quantity of the total classification is fixed.&lt;/p&gt;&#xA;&lt;p&gt;I don't know am I right or wrong, please tell me if I made a mistake.&lt;/p&gt;&#xA;" OwnerUserId="38455" LastEditorUserId="38455" LastEditDate="2020-07-08T01:44:11.713" LastActivityDate="2021-04-04T04:05:55.540" Title="Is it possible that every class has a higher recall than precision for multi-class classification?" Tags="&lt;machine-learning&gt;&lt;text-classification&gt;&lt;metric&gt;&lt;precision&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22386" PostTypeId="2" ParentId="22383" CreationDate="2020-07-07T23:07:38.797" Score="0" Body="&lt;p&gt;Transformers can be trained with parallelization. So you might have 100 cores training the transformer for an hour (100 total hours of CPU time), whereas the LSTM has one core training for ten hours (10 total hours of CPU time). Even though the LSTM takes longer, it is less expensive to train.&lt;/p&gt;&#xA;" OwnerUserId="26838" LastActivityDate="2020-07-07T23:07:38.797" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22387" PostTypeId="2" ParentId="22385" CreationDate="2020-07-07T23:32:34.270" Score="0" Body="&lt;p&gt;In the case of binary classification, i.e. when you have two classes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, the recall is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle {\text{Recall}}={\frac {tp}{tp+fn}}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Similarly, the precision as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle {\text{Precision}}={\frac {tp}{tp+fp}}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where, &lt;strong&gt;if you choose and fix a class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; as your main class (or class of interest)&lt;/strong&gt;,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$tp$&lt;/span&gt; is the number of true positives (i.e. the number of observations that have been classified as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and actually belong to class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, i.e. they have been correctly classified)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$fn$&lt;/span&gt; is the number of false negatives (i.e. the number of observations that have been &lt;em&gt;incorrectly&lt;/em&gt; classified as &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, so, in this case of binary classification, they actually belong to class &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$fp$&lt;/span&gt; is the number of false positives (i.e. the number of observations that have been &lt;em&gt;incorrectly&lt;/em&gt; classified as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In this context, the confusion matrix will be a &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; matrix because you have two predictions (one for each of the two classes) compared against the two ground-truth labels (or classes).&lt;/p&gt;&#xA;&lt;p&gt;In the context of multi-class classification, where you have &lt;span class=&quot;math-container&quot;&gt;$N &amp;gt; 2$&lt;/span&gt; classes, you will have a &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt; confusion matrix, for the same reason. In this case, the recall and precision are just an extension of the binary classification problem. More precisely, let &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; represent your confusion matrix, so &lt;span class=&quot;math-container&quot;&gt;$M_{ij}$&lt;/span&gt; is the entry of this confusion matrix at row &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and column &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. Then the recall, &lt;strong&gt;for class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;&lt;/strong&gt; (i.e. your main class, as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; above), is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{Recall}_{~i} = \cfrac{M_{ii}}{\sum_j M_{ij}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{Precision}_{~i} = \cfrac{M_{ii}}{\sum_j M_{ji}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In other words, the recall for a certain class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is computed as the fraction of the diagonal element &lt;span class=&quot;math-container&quot;&gt;$M_{ii}$&lt;/span&gt; and the sum of all elements in that row &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. Similarly, the precision is computed as the fraction of &lt;span class=&quot;math-container&quot;&gt;$M_{ii}$&lt;/span&gt; and the sum of all elements in column &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If the true positives (i.e. the diagonal elements &lt;span class=&quot;math-container&quot;&gt;$M_{ii}$&lt;/span&gt;) of all classes are indeed bigger compared to the denominator, the recall can indeed be bigger than the precision for all classes.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-07T23:32:34.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22388" PostTypeId="2" ParentId="22378" CreationDate="2020-07-08T00:04:28.017" Score="1" Body="&lt;p&gt;As it says in the &lt;a href=&quot;https://pytorch.org/docs/master/generated/torch.nn.RNN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, you can simply reverse the order of dimensions by providing the argument &lt;code&gt;batch_first=True&lt;/code&gt; when constructing the RNN.&#xA;Then, the dimensionality will be: &lt;code&gt;(batch, seq, feature)&lt;/code&gt;, i.e. batch-size &lt;em&gt;times&lt;/em&gt; sequence length &lt;em&gt;times&lt;/em&gt; the dimension of your input (however dimensional that may be). Then, everything is gonna work as you are used to it.&lt;/p&gt;&#xA;&lt;p&gt;To answer the second part of your question, normally, each sequence in a batch is independent of the others (since they commonly get sampled at random). So, there is no direct dependence between any two inputs in a batch (except, of course, for the fact that they are commonly expected to stem from some underlying shared data generating process which you want to approximate by the RNN).&lt;/p&gt;&#xA;&lt;p&gt;And a hidden state is commonly maintained per batch element, i.e. there is one hidden state per batch element (i.e. per sequence).&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-07-08T00:04:28.017" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22390" PostTypeId="1" CreationDate="2020-07-08T00:45:12.953" Score="1" ViewCount="325" Body="&lt;p&gt;I have seen two different representations of neural networks when it comes to bias. Consider a &amp;quot;simple&amp;quot; neural network, with just an input layer, a hidden layer and an output layer. To compute the value of a neuron in the hidden layer, the weights and neurons from the input layer are multiplied, shifted by a bias and then activated by the activation function. To compute the values in the output layer, you may choose not to have a bias and have an identity activation function on this layer, so that this last calculation is just &amp;quot;scaling&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Is it standard to have a &amp;quot;scaling&amp;quot; layer? You could say that there is a bias associated with each neuron, except those in the input layer correct (and those in the output layer when it is a scaling layer)? Although I suppose you could immediately shift any value you're given. Does the input layer have a bias?&lt;/p&gt;&#xA;&lt;p&gt;I have seen bias represented as an extra unchanging neuron in each layer (except the last) having value 1, so that the weights associated with the connections from this neuron correspond to the biases of the neurons in the next layer. Is this the standard way of viewing bias? Or is there some other way to interpret what bias is that is more closely described by &amp;quot;a number that is added to the weighted sum before activation&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="38458" LastEditorUserId="2444" LastEditDate="2020-07-08T12:12:58.367" LastActivityDate="2021-04-05T00:07:10.710" Title="Does the input layer have bias and are there bias neurons?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22391" PostTypeId="1" CreationDate="2020-07-08T01:37:34.027" Score="0" ViewCount="41" Body="&lt;p&gt;I am learning the expectation-maximization algorithm from the article &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.89.5223&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt; Semi-Supervised Text Classification Using&#xA;EM&lt;/a&gt;. The algorithm is very interesting. However, the algorithm looks like doing a circular inference here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W4eFM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W4eFM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't know am I understanding the description right or wrong, what I perceived is:&lt;/p&gt;&#xA;&lt;p&gt;Step 1: train NB classifier on labeled data.&lt;br /&gt;&#xA;&lt;strong&gt;Repeat&lt;/strong&gt;&lt;br /&gt;&#xA;Step 2 (E-step): use trained NB to add label to unlabeled data.&lt;br /&gt;&#xA;Step 3 (M-step): train NB classifier by using labeled data and unlabeled data(with tags from step 2) to get a new classifier&lt;br /&gt;&#xA;&lt;strong&gt;Until convergent&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is the question:&lt;/p&gt;&#xA;&lt;p&gt;In step 2, the label is tagged by the classifier trained on the labeled data, which is the only source containing the knowledge on a correct prediction. And the step-3 (M-step) is actually updating the classifier on the labels generated from step 2. The whole process is relying on the labeled data, so how can the EM classifier improve the classification? Can someone explain it to me?&lt;/p&gt;&#xA;" OwnerUserId="38455" LastEditorUserId="2444" LastEditDate="2020-07-08T13:53:28.730" LastActivityDate="2020-07-08T13:53:28.730" Title="How can the expectation-maximization improve the classification?" Tags="&lt;machine-learning&gt;&lt;naive-bayes&gt;&lt;semi-supervised-learning&gt;&lt;expectation-maximization&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22392" PostTypeId="2" ParentId="1742" CreationDate="2020-07-08T02:42:39.227" Score="2" Body="&lt;p&gt;First, in most condition &lt;strong&gt;machine learning&lt;/strong&gt; actually &lt;strong&gt;refers&lt;/strong&gt; &lt;strong&gt;traditional/classical machine learning&lt;/strong&gt;, and deep learning is specifically referring multi-layered neural network, and &lt;strong&gt;neural network&lt;/strong&gt; &lt;strong&gt;is&lt;/strong&gt; one of the &lt;strong&gt;machine learning&lt;/strong&gt; approach.&lt;/p&gt;&#xA;&lt;p&gt;Second, Machine learning especially supervised &lt;strong&gt;machine learning requires engineers to design and predefine features manually&lt;/strong&gt;, which are used to represent the data in numerical way. Such as we can represent animals with three features such as the number of eyes, the number of legs and the number of heads. The data [2,4,1] representing an animal with 2 eyes, 4 legs and 1 head. In this scenario, the feature is extracted by us, because we have knowledge on animals, and we think these features can represent animals.  However, instead of hand-crafting features the &lt;strong&gt;deep learning learn the features automatically.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Third, when someone say &lt;strong&gt;machine learning&lt;/strong&gt; he &lt;strong&gt;is&lt;/strong&gt; saying &lt;strong&gt;algorithm&lt;/strong&gt;, such as naive bayes, decision tree, linear regression etc. However, the &lt;strong&gt;deep learning is&lt;/strong&gt; more &lt;strong&gt;related to&lt;/strong&gt; the framework and &lt;strong&gt;architecture&lt;/strong&gt; such as RNN, CNN, Transformer etc.&lt;/p&gt;&#xA;&lt;p&gt;Fourth, &lt;strong&gt;it is possible to start deep learning without knowing machine learning&lt;/strong&gt;, sources from internet like Andrew Ng's course usually covers most topic you should know in deep learing. Try search Andrew Ng, I think he is really good!&lt;/p&gt;&#xA;" OwnerUserId="38455" LastActivityDate="2020-07-08T02:42:39.227" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22393" PostTypeId="1" CreationDate="2020-07-08T05:45:38.653" Score="0" ViewCount="33" Body="&lt;p&gt;I have images of questions from the domain of mathematics, where the image can be a mixture of the English language and mathematical formulas. I want to build and train an OCR model like &lt;a href=&quot;https://github.com/harvardnlp/im2markup&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harvard NLP's Im2Markup&lt;/a&gt;, but the problem is that, even if I am able to replicate and train such a model, how will I be able to extract from &lt;strong&gt;MY TYPE OF IMAGES&lt;/strong&gt;? Because the &lt;a href=&quot;https://zenodo.org/record/56198#.XwVVoXUzZuR&quot; rel=&quot;nofollow noreferrer&quot;&gt;im2latex-100k training data&lt;/a&gt; has pure equations of mathematical formulas and my image has Texts and Formulas.&lt;/p&gt;&#xA;&lt;p&gt;Let us even say that by using &lt;a href=&quot;https://mathpix.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MathPix&lt;/a&gt;, I am able to extract the Latex for a lot of images say 100000 or so, how can I then approach the training model?&lt;/p&gt;&#xA;&lt;p&gt;One approach is that I try to classify whether a question has mathematical formulas or not using a classification model. But even if I classify if there is a formula presented, how am I able to extract the latex along with the Text?&lt;/p&gt;&#xA;" OwnerUserId="36062" LastEditorUserId="2444" LastEditDate="2020-07-08T14:00:10.570" LastActivityDate="2020-07-08T14:00:10.570" Title="Approaches for OCR building which can extract latex from the image as mathematical formulas" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22394" PostTypeId="1" CreationDate="2020-07-08T07:14:54.567" Score="0" ViewCount="25" Body="&lt;p&gt;I have the following problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I1VHl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I1VHl.png&quot; alt=&quot;The problem&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I create a fuzzy set in &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; using the given information?&lt;/p&gt;&#xA;&lt;p&gt;I clearly have no idea where to start. Any tips or solutions would be appreciated&lt;/p&gt;&#xA;" OwnerUserId="38459" LastEditorUserId="2444" LastEditDate="2020-07-08T14:02:49.043" LastActivityDate="2020-07-08T14:02:49.043" Title="How to find a fuzzy set with a given relation matrix?" Tags="&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22395" PostTypeId="1" CreationDate="2020-07-08T08:23:37.693" Score="0" ViewCount="30" Body="&lt;p&gt;I got two classes namely &lt;code&gt;positive&lt;/code&gt; and &lt;code&gt;negative&lt;/code&gt; with 1500 samples on each a total of 3k. A sample sequence is like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Period = 1min   Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;S2&amp;gt;M2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 5min   Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;M2&amp;gt;S2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 10min  Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;S2&amp;gt;M2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 15min  Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;S2&amp;gt;M2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 20min  Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;S2&amp;gt;M2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 30min  Pattern = M1&amp;gt;S1&amp;gt;T1&amp;gt;B2&amp;gt;M2&amp;gt;S2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 60min  Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;S2&amp;gt;M2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;Period = 120min Pattern = S1&amp;gt;M1&amp;gt;T1&amp;gt;B2&amp;gt;S2&amp;gt;M2&amp;gt;S3&amp;gt;T3&amp;gt;M3&amp;gt;B3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So for each sample, we have a total of 8 sequences and within each sequence there are 10 elements which are used only once. I assigned a number for each of the elements from &lt;code&gt;0 to 9&lt;/code&gt;. Later, stored &lt;code&gt;positive&lt;/code&gt; and &lt;code&gt;negative&lt;/code&gt; samples into numpy arrays with shapes of &lt;code&gt;(1500,8,10)&lt;/code&gt;. By using LSTM's, I was able to achieve around 96% accuracy, which I'm glad and sufficient for the task.&lt;/p&gt;&#xA;&lt;p&gt;For now, my task is to create artificial samples and make the LSTM's classify it. I should mention that there are some rules for sequence generation like &lt;code&gt;M1&amp;gt;T1&amp;gt;M2&amp;gt;T3&amp;gt;M3&lt;/code&gt;. By applying all of these rules as permutation constraints, I was able to get down to a &lt;code&gt;12960&lt;/code&gt; sequences that could be used in any of the periods. This makes a total of &lt;code&gt;12960^8~= Multiple orders of Petabytes of data&lt;/code&gt;. Not feasible to store all samples. So I gave up generating each sample even by this means.&lt;/p&gt;&#xA;&lt;p&gt;What I'm looking for now is a way to generate artificial data, by means of some GAN style LSTM or whatever I don't know. Which will take &lt;code&gt;(1500,8,10) arrays of positive and negative samples&lt;/code&gt; and spit out a positive or negative candidate with a label. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="30841" LastEditorUserId="1641" LastEditDate="2020-07-08T19:36:13.917" LastActivityDate="2020-07-08T19:36:13.917" Title="Generating artificial data by means of LSTM" Tags="&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22396" PostTypeId="2" ParentId="22380" CreationDate="2020-07-08T08:29:18.687" Score="3" Body="&lt;p&gt;Reward in reinforcement learning (RL) is entirely different from a supervised learning (SL) label, but can be related to it indirectly.&lt;/p&gt;&#xA;&lt;p&gt;In a RL control setting, you can imagine that you had a data oracle that gave you SL training example and label pairs &lt;span class=&quot;math-container&quot;&gt;$x_i, y_i$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; represents a state and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; represents the correct action to take in that state in order to maximise the expected return. For simplicity I will use &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=1}^{\infty} \gamma^k R_{t+k+1}$&lt;/span&gt; for return here (where &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; are random variables), there are other definitions, but the argument that follows doesn't change much for them.&lt;/p&gt;&#xA;&lt;p&gt;You can use the oracle to reduce the RL training process to SL, creating a policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(s): \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt; learned from a dataset that the oracle output. This clearly relates SL with RL, but how do &lt;span class=&quot;math-container&quot;&gt;$x_i, y_i$&lt;/span&gt; from SL relate to &lt;span class=&quot;math-container&quot;&gt;$s_t, a_t$&lt;/span&gt; from RL in terms of reward values?&lt;/p&gt;&#xA;&lt;p&gt;The states can relate directly (as input):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_i \equiv s_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The action from the policy function is more indirect, if you want to see how reward is involved:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_i \equiv \pi^*(s_t) = \text{argmax}_a \mathbb{E}_{A \sim \pi^*}[\sum_{k=1}^{\infty} \gamma^k R_{t+k+1} | S_t=s_t, A_t=a]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note the oracle is represented by the optimal policy function &lt;span class=&quot;math-container&quot;&gt;$\pi^*(s_t)$&lt;/span&gt;, and the expectation is conditional both on the start conditions of state and action plus following the optimal policy from then on (which is what &lt;span class=&quot;math-container&quot;&gt;$A \sim \pi^*$&lt;/span&gt; is representing).&lt;/p&gt;&#xA;&lt;p&gt;In practice the optimal policy function is unknown when starting RL, so the learning process &lt;em&gt;cannot&lt;/em&gt; be reduced to a SL problem. However, you can get close in some circumstances by creating a dataset of action choices made by an expert at the problem. In that case a similar relationship applies - the label (of which action to take) and immediate reward are different things but can be related by noting that the expert behaviour is close to the &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}$&lt;/span&gt; over actions of expected sums of future reward.&lt;/p&gt;&#xA;&lt;p&gt;Another way to view the difference:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In SL, the signal from the label is an &lt;em&gt;instruction&lt;/em&gt; - &amp;quot;associate these two values&amp;quot;. Data is supplied to the learning process by some other independent process, and can be learned from directly&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In RL, the signal from the reward is a &lt;em&gt;consequence&lt;/em&gt; - &amp;quot;this is the value, in context, of what you just did&amp;quot;, and needs to be learned from indirectly. Data is not supplied separately from the learning process, but must be actively collected by it - deciding which state, action pairs to learn from is part of the agent's learning task&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-08T10:28:31.437" LastActivityDate="2020-07-08T10:28:31.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22398" PostTypeId="1" CreationDate="2020-07-08T10:51:18.027" Score="0" ViewCount="22" Body="&lt;p&gt;Suppose I have a neural network &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; that produces the output probabilities &lt;span class=&quot;math-container&quot;&gt;$[0.3, 0.8]$&lt;/span&gt;. Normally, I would specify a threshold of 0.5 for the argmax of the prediction, let's say, second arg &amp;gt; 0.5 means that the image is attractive, and if both probabilities are lesser than 0.5 we don't have that good of a prediction.&lt;/p&gt;&#xA;&lt;p&gt;My question is, can we plot this threshold on a ROC curve so we can figure out the best value?&lt;/p&gt;&#xA;" OwnerUserId="20934" LastEditorUserId="2444" LastEditDate="2020-07-08T14:09:35.050" LastActivityDate="2020-07-08T14:09:35.050" Title="Best ROC threshold for classifier?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22399" PostTypeId="2" ParentId="22390" CreationDate="2020-07-08T12:37:32.347" Score="0" Body="&lt;p&gt;The purpose of the input layer is just to conceptually represent the input and, in case it is necessary, define the dimensions of the input that the neural network expects. In fact, some neural networks, such as multi-layer perceptrons, expect a fixed-size input, but not all of them: fully convolutional networks can deal with inputs of different dimensions.&lt;/p&gt;&#xA;&lt;p&gt;The input layer doesn't contain neurons (although in the diagrams that you will come across they are usually represented as circles, like the neurons, and that's probably why you are confused!), so it also does not contain biases, linear transformations, and non-linearities. In fact, in the context of neural networks, you could define a neuron as some unit/entity that performs a linear or non-linear transformation (to which you can add a bias). Note that the hidden and output layers can contain biases because they contain neurons that perform a linear or non-linear transformation.&lt;/p&gt;&#xA;&lt;p&gt;However, although I have never seen it (or I don't recall having seen it), I would not exclude the existence of an input layer that transforms or augments the inputs before passing them to the next layer. For example, one could implement a neural network that first scales the input to a certain range, and the input layer could do this, although, in practice, this is typically done by some object/class that does not belong to the neural network (e.g. &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/data/Dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tf.data.Dataset&lt;/code&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-08T22:48:21.777" LastActivityDate="2020-07-08T22:48:21.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22400" PostTypeId="1" CreationDate="2020-07-08T13:15:25.007" Score="0" ViewCount="23" Body="&lt;p&gt;I am starting to learn about Neural Network and I have come into one problem that I am still learning how to figure it out.&lt;/p&gt;&#xA;&lt;p&gt;I have a dataset with shape (105,96) (105 samples and 95 first column as features and last column for label of each sample). In short data = (105,95) and label = (105,1).&lt;/p&gt;&#xA;&lt;p&gt;I have build a model for my dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(200, input_dim = 95, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))&#xA;model.add(Dense(30, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))&#xA;model.add(Dense(3, activation = 'softmax'))&#xA;&#xA;# Compile model&#xA;model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I use this model on 2 purpose and get a curious result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;1. K-Cross Validation:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I set &lt;strong&gt;k = 5, batch_size = 7 and epoch = 150&lt;/strong&gt; the result I got is: 94.29% (+-7%)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def baseline_model():&#xA;    # create model&#xA;    model = Sequential()&#xA;    model.add(Dense(first_layer_neurons, input_dim=input_neurons, activation='relu', bias_initializer = 'ones', kernel_initializer = 'normal'))&#xA;    model.add(Dense(second_layer_neurons, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))&#xA;    model.add(Dense(classes, activation='softmax'))&#xA;    model.summary()&#xA;    # Compile model&#xA;    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])&#xA;    return model&#xA;&#xA;estimator = KerasClassifier(build_fn=baseline_model, epochs= epoch, batch_size = batch, verbose=1)&#xA;kfold = KFold(n_splits=k, shuffle=True)&#xA;results = cross_val_score(estimator, X, y, cv=kfold)&#xA;print(&amp;quot;Baseline: %.2f%% (%.2f%%)&amp;quot; % (results.mean()*100, results.std()*100))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;2. Train-Validation-Test split:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I split dataset to &lt;strong&gt;train = (90,95); validation_data = (10,95) and test = (5,95) with epoch = 150 and batch_size = 10&lt;/strong&gt; and the result I got is: Train = 99%, Validation = 90% and Test is 100%&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(200, input_dim = 95, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))&#xA;model.add(Dense(30, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))&#xA;model.add(Dense(3, activation = 'softmax'))&#xA;&#xA;# Compile model&#xA;model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])&#xA;&#xA;history = model.fit(X_train, y_train, validation_split = 0.1, epochs = 150, batch_size = 10)&#xA;&#xA;_ , acc_train = model.evaluate(X_train, y_train)&#xA;_ , acc_test = model.evaluate(X_test, y_test)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I have notice that maybe my model got overfitting because when I plot the Accuracy, Loss Plot and it like this:&lt;a href=&quot;https://i.stack.imgur.com/0luWa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0luWa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So my question is how I encounter this problem, my model have a very good result but it seem Overfitting, how I can get over the Overfiting for my model (Dropout or add some hidden layer will help?)&lt;/p&gt;&#xA;&lt;p&gt;I am appreciate all of answers, comment for my post.&lt;/p&gt;&#xA;&lt;p&gt;Thank you guys very much.&lt;/p&gt;&#xA;" OwnerUserId="38468" LastActivityDate="2020-07-08T13:15:25.007" Title="Different result from k-cross validation model and Train-Validation-Test split model ? (AI fresher question)" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;keras&gt;&lt;overfitting&gt;&lt;cross-validation&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22401" PostTypeId="1" AcceptedAnswerId="22402" CreationDate="2020-07-08T14:15:23.210" Score="2" ViewCount="60" Body="&lt;p&gt;Recently, I followed the open course CS229,&#xA;&lt;a href=&quot;http://cs229.stanford.edu/notes/cs229-notes1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs229.stanford.edu/notes/cs229-notes1.pdf&lt;/a&gt;&lt;br /&gt;&#xA;This lecturer introduces an alternative approach to gradient descent that is called &amp;quot;Normal Equation&amp;quot; and the equation is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta=\left(X^{T} X\right)^{-1} X^{T} \vec{y}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The normal equation can directly compute the &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If the normal equation works, why do we need gradient descent? What is the trade-off between these two methods?&lt;/p&gt;&#xA;" OwnerUserId="38455" LastEditorUserId="2444" LastEditDate="2020-07-08T14:23:34.573" LastActivityDate="2020-07-08T14:31:12.243" Title="If the normal equation works, why do we need gradient descent?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22402" PostTypeId="2" ParentId="22401" CreationDate="2020-07-08T14:31:12.243" Score="1" Body="&lt;p&gt;That &lt;a href=&quot;https://mathworld.wolfram.com/NormalEquation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;normal equation&lt;/a&gt; is sometimes called the &lt;a href=&quot;https://mathworld.wolfram.com/Closed-FormSolution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;closed-form solution&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The short answer to your question is that the closed-form solution may be impractical or unavailable in certain cases or the iterative numerical method (such as gradient descent) may be more efficient (in terms of resources).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/a/23132/82135&quot;&gt;This answer&lt;/a&gt; gives you more details and an example.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-08T14:31:12.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22403" PostTypeId="2" ParentId="22381" CreationDate="2020-07-08T15:06:36.337" Score="1" Body="&lt;p&gt;As you said, generally the most important one is reward per episode. If this isn't increasing overall, there's a problem (of course this metric can fluctuate, I mean to say that macroscopically it should increase).&lt;/p&gt;&#xA;&lt;p&gt;Policy loss (I assume you mean the &amp;quot;actor loss&amp;quot;?) is generally harder to interpret. You should think of this more as a source of gradients and not necessarily a good indicator of how well your agent is performing.&lt;/p&gt;&#xA;&lt;p&gt;I'm not really sure why you'd be monitoring the value during training. Value loss, however, is basically equivalent to value loss in value based methods like Q-learning, for example. So this one should be decreasing overall. Otherwise the baselines you compute to reduce variance in the policy gradient will either be less effective, or even harmful.&lt;/p&gt;&#xA;&lt;p&gt;Entropy is a nice quantity to measure, because it's a good indicator of how much your agent is exploring. If you see that your agent is not achieving high returns and the entropy is really low, this means that your policy has converged to a suboptimal one. If the entropy is really high, this means the agent is acting fairly randomly (so it's basically exploring a lot). Ideally the entropy should decrease over time, so your policy becomes more deterministic (less exploration) as it reaches an optimum.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-08T15:06:36.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22405" PostTypeId="2" ParentId="16033" CreationDate="2020-07-08T16:08:30.253" Score="1" Body="&lt;p&gt;The brat annotation tool has provided some useful scripts for converting the annotations format including standoff format to CoNLL. Please see this source code from brat GitHub repo for converting the .ann and .txt inputs (standoff format) to .conll file: &lt;a href=&quot;https://github.com/nlplab/brat/blob/master/tools/anntoconll.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nlplab/brat/blob/master/tools/anntoconll.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38474" LastEditorUserId="38474" LastEditDate="2020-07-15T00:07:58.513" LastActivityDate="2020-07-15T00:07:58.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22406" PostTypeId="2" ParentId="22367" CreationDate="2020-07-08T19:12:05.910" Score="1" Body="&lt;p&gt;The regularization terms are applied to the loss functions by default.&#xA;However, their gradients do appear in the update step as the gradient of loss appears in the update step.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-07-08T19:12:05.910" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22407" PostTypeId="1" CreationDate="2020-07-08T20:16:34.383" Score="0" ViewCount="23" Body="&lt;p&gt;I am facing the following problem. I need to create a model to predict the product groups from product title. For each word in sentence I need to predict position of a word marked as product. My dataset looks like this.&lt;/p&gt;&#xA;&lt;p&gt;Input: product title (max 10 words), eg.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;“red adidas sneakers” (this is when there is one product in product title) or&lt;/li&gt;&#xA;&lt;li&gt;“red adidas sneakers + white t-shirt” (this is an exmaple when there are two products in title)&#xA;there may be more products in title&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Output: position of product for each word in title.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;E. g. for title:  “red adidas sneakers” word “”sneakers” indicates that’s a product. Other words are sneaker’s features. It’s on 3rd position do so the output would be a vector [3,3,3,0,0,0,0,0,0,0].  So I am mapping features to the product (word to position of product)&lt;/li&gt;&#xA;&lt;li&gt;For title with 2 products:  “red adidas sneakers + white t-shirt” the output would look like this: [3,3,3,0,6,6,0,0,0,0]. mapping goes like this: (red=3, adidas=3, sneakers=3, +=0, white=6, t-shirt=6)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It seems like category prediction problem but it is not. Two similar titles make different outputs: e.g. “adiadas sneakers red t-shirt white” gets [2,2,2,4,4,0,0,0,0,0] and if differs from [3,3,3,0,6,6,0,0,0,0]. The same word “red” gets different prediction (2 and 3).&lt;/p&gt;&#xA;&lt;p&gt;Therefore I have a two question.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a known model that solves this kind of problem?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I came with idea inspired by attention mechanism to create a model with 2 encoders from the same title. Use some CNN architecture to embrace the word context.  And then make dot product of two encoded inputs and predict word positions.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;embeddings = tf.keras.layers.Embedding(max_tokens, dimension)&#xA;input1_embeddings = token_embedding(embeddings)&#xA;input2_embeddings = token_embedding(embeddings)&#xA;&#xA;cnn_layer = tf.keras.layers.Conv1D(&#xA;    filters=100,&#xA;    kernel_size=3,&#xA;    # Use 'same' padding so outputs have the same shape as inputs.&#xA;    padding='same')&#xA;encoded1 = cnnlayer(input1_embeddings)&#xA;encoded2 = cnnlayer(input2_embeddings)&#xA;l = tf.keras.layers.Dot(axes=(1, 2))([encoded1, encoded2])&#xA;output = Dense(10, activation=”softmax”)(l)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this a reasonable way to do it?&lt;/p&gt;&#xA;" OwnerUserId="38475" LastActivityDate="2020-07-08T20:16:34.383" Title="Time distributed word position prediction" Tags="&lt;keras&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22408" PostTypeId="1" CreationDate="2020-07-09T01:38:43.430" Score="1" ViewCount="66" Body="&lt;p&gt;When I label images for semantic segmentation (using u-net, if that matters), is labeling the background (anything I am not interested in) necessary? Will it improve the network's performance?&lt;/p&gt;&#xA;" OwnerUserId="38295" LastEditorUserId="2444" LastEditDate="2020-07-09T17:18:43.793" LastActivityDate="2020-07-09T17:18:43.793" Title="Is it necessary to label the background when generating the labelled dataset for semantic segmentation?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;training&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22409" PostTypeId="1" CreationDate="2020-07-09T03:45:43.010" Score="1" ViewCount="44" Body="&lt;p&gt;For practical applications, like autonomous driving, depth perception is needed to make useful decisions.&lt;/p&gt;&#xA;&lt;p&gt;How is this normally addressed without using a LIDAR or RADAR unit (but using a camera)?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-07-09T11:48:09.037" LastActivityDate="2020-07-09T11:48:09.037" Title="How is depth perception (e.g. in autonomous driving) addressed without using a Lidar or Radar unit?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22410" PostTypeId="1" CreationDate="2020-07-09T05:10:56.730" Score="1" ViewCount="51" Body="&lt;p&gt;Given a hand-drawn shape, I'd like to generate the corresponding symmetrical polished shapes such as circle, rectangle, triangle,  trapezoid, square, parallelogram, etc.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.lynda.com/OneNote-tutorials/Convert-hand-drawn-shapes-polished-symmetrical-shapes/574673/603787-4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;A short video demonstration&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here below we can see a parallelogram, trapezoid, triangle and a circle.&#xA;I was wondering how can I transform it into symmetrical polished shapes?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ccI5C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ccI5C.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;At first, I tried a simple approach with traditional computer vision algorithms, with OpenCV (no neural networks were involved), by counting the number of corners, but it failed miserably, since there are many edge cases within a user's doodler.&lt;/p&gt;&#xA;&lt;p&gt;So, I was thinking to delve into CNN specifically U-Net for segmentation.&lt;/p&gt;&#xA;&lt;p&gt;Can somebody please give me some suggestions on how to approach this kind of problem? I'd like to read some relevant articles and code about this subject for getting a better grasp of this kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="25338" LastEditorUserId="2444" LastEditDate="2020-07-09T11:51:35.317" LastActivityDate="2021-04-07T03:07:05.683" Title="How to quickly change hand-drawn shapes to symmetrical polished shapes?" Tags="&lt;convolutional-neural-networks&gt;&lt;object-recognition&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22412" PostTypeId="1" CreationDate="2020-07-09T12:51:05.703" Score="1" ViewCount="50" Body="&lt;p&gt;My problem has a single state and an infinite amount of actions on a certain interval (0,1). After quite some time of googling I found a few paper about an algorithm called zooming algorithm which can solve problems with a continous action space. However my implementation is bad at exploiting. Therefore I'm thinking about adding an epsilon-greedy kind of behavior.&lt;/p&gt;&#xA;&lt;p&gt;Is it reasonable to combine different methods?&lt;/p&gt;&#xA;&lt;p&gt;Do you know other approaches to my problem?&lt;/p&gt;&#xA;" OwnerUserId="38494" LastActivityDate="2020-07-09T12:51:05.703" Title="Solving multi-armed bandit problems with continuous action space" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22413" PostTypeId="1" CreationDate="2020-07-09T15:04:15.103" Score="3" ViewCount="47" Body="&lt;p&gt;For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some vector representation. I understand the gist, but I can't find any deep explanations or tutorials that don't start with importing TensorFlow. All I'm really told is that it works by context using a few surrounding words, but I don't understand exactly how.&lt;/p&gt;&#xA;&lt;p&gt;Is it much different from a classic network, with weights and biases? How does it figure out the loss?&lt;/p&gt;&#xA;&lt;p&gt;If someone could point me towards a guide to how these things work exactly I would be very grateful.&lt;/p&gt;&#xA;" OwnerUserId="38497" LastEditorUserId="2444" LastEditDate="2020-07-09T19:57:47.647" LastActivityDate="2021-04-05T22:02:50.267" Title="How can I create an embedding layer to convert words to a vector space from scratch?" Tags="&lt;neural-networks&gt;&lt;implementation&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22414" PostTypeId="1" CreationDate="2020-07-09T15:05:03.133" Score="1" ViewCount="39" Body="&lt;p&gt;Sample-based algorithms, like Monte Carlo Algorithms and TD-Learning, are often presented as useful since they do not require a transition model.&lt;/p&gt;&#xA;&lt;p&gt;Assuming I do have access to a transition model, are there any reasons one might want to use sample-based methods instead of performing a full Bellman update?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditDate="2020-07-09T17:23:54.243" LastActivityDate="2020-07-09T17:23:54.243" Title="If the transition model is available, why would we use sample-based algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;bellman-equations&gt;&lt;dynamic-programming&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22415" PostTypeId="2" ParentId="22414" CreationDate="2020-07-09T15:09:29.047" Score="1" Body="&lt;p&gt;A full Bellman update can be intractable. For instance, if your state space or action space are continuous, the full Bellman update is intractable. You can try to solve this by discretizing, but if your state space is large this will also be intractable.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-09T15:09:29.047" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22416" PostTypeId="1" AcceptedAnswerId="22420" CreationDate="2020-07-09T16:49:17.693" Score="3" ViewCount="89" Body="&lt;p&gt;I'm training a robot to walk to a specific &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt; point using TD3, and, for simplicity, I have something like  &lt;code&gt;reward = distance_x + distance_y + standing_up_straight&lt;/code&gt;, and then it adds this reward to the replay buffer. However, I think that it would be more efficient if it can break the reward down by category, so it can figure out &amp;quot;that action gave me a good distance &lt;code&gt;distance_x&lt;/code&gt;, but I still need work on &lt;code&gt;distance_y&lt;/code&gt; and &lt;code&gt;standing_up_straight&lt;/code&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Are there any existing algorithms that add rewards this way? Or have these been tested and proven not to be effective?&lt;/p&gt;&#xA;" OwnerUserId="34441" LastEditorUserId="2444" LastEditDate="2020-12-20T16:40:13.550" LastActivityDate="2021-01-13T13:19:58.590" Title="Can rewards be decomposed into components?" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;multi-objective-rl&gt;&lt;reward-hypothesis&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22417" PostTypeId="2" ParentId="22413" CreationDate="2020-07-09T17:16:13.710" Score="0" Body="&lt;p&gt;Word2vec embedding are trained by simple auto-encoder model that takes a word and tries to predict one word form the window of surrounding words.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2VUM3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2VUM3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You could define it like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;num_of_words = 50000&#xA;# one hot encoded word&#xA;input = Input(num_of_words)&#xA;# You could use non linear activation&#xA;w2v = Dense(300, activation=”linear”)(input)&#xA;output = Dense(num_of_words, activation=”softmax”)(w2v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But in practice, the model is redefined and takes two words as input and predicts next words. And it outputs a probability score for all the words it knows (the model’s “vocabulary”, which can range from a few thousand to over a million words).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9h4VT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9h4VT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is trained both ways from beginning to end of sentence and reverse. Loss is a regular categorical_crossentropy Detailed explenation can be found here [http://jalammar.github.io/illustrated-word2vec/]&lt;/p&gt;&#xA;" OwnerUserId="38475" LastActivityDate="2020-07-09T17:16:13.710" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22419" PostTypeId="1" CreationDate="2020-07-09T18:34:45.570" Score="0" ViewCount="46" Body="&lt;p&gt;I am a complete beginner in the area. I implemented my first neural network following the online book &amp;quot;Neural Networks and Deep Learning&amp;quot; by Micheal Nielsen. It works fine with classifying handwritten digits. Achieving ~9500/10000 accuracy on the test data.&#xA;I am trying to train the network to determine whether &lt;span class=&quot;math-container&quot;&gt;$x &amp;gt; 5$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is in the interval &lt;span class=&quot;math-container&quot;&gt;$[0,10)$&lt;/span&gt;, which should be a much simpler task than classifying handwritten digits. However, no learning happens and the accuracy ever the test data stays exactly the same with every epoch. I tried different structures and different learning rates but always the same thing happened. Here is the code I wrote that uses libraries in Nielsen's book:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import networkCopy&#xA;import numpy as np&#xA;# Creating training data&#xA;x = []&#xA;y = []&#xA;for n in range(1000):&#xA;    to_add = 100*np.random.rand()&#xA;    x.append(np.array([to_add]).reshape(1,1))&#xA;    y.append(np.array([float(to_add &amp;gt; 50)]).reshape(1,1))&#xA;training_data = zip(x, y)&#xA;# Creating test data&#xA;tx = []&#xA;ty = []&#xA;for n in range(1000):&#xA;    to_add = 100*np.random.rand()&#xA;    tx.append(np.array([to_add]).reshape(1,1))&#xA;    ty.append(np.array([float(to_add &amp;gt; 50)]).reshape(1,1))&#xA;test_data = zip(tx, ty)&#xA;&#xA;# Creating and training the network&#xA;net = networkCopy.Network([1, 5, 1])  # [1, 5, 1] contains the number of neurons for each layer&#xA;net.SGD(training_data, 300, 100, 5.0, test_data=test_data)&#xA;# 300 is the number of epochs, 100 is the mini batch size&#xA;#5.0 is the learning rate &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The way I generated the data may not be optimal, it is an ad hoc solution to make the data in the proper form for the network. This is my first question so I apologize for any mistakes that might be in the format of the question.&lt;/p&gt;&#xA;" OwnerUserId="38499" LastActivityDate="2020-07-09T18:34:45.570" Title="Neural Network is not learning a very simple task" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22420" PostTypeId="2" ParentId="22416" CreationDate="2020-07-09T18:36:10.527" Score="5" Body="&lt;p&gt;If I understood correctly you're looking at a &lt;a href=&quot;https://ewrl.files.wordpress.com/2015/02/ewrl12_2015_submission_2.pdf&quot; rel=&quot;noreferrer&quot;&gt;Multi-Objective Reinforcement Learning&lt;/a&gt; (MORL). Keep in mind however  that many scientist will often follow the &lt;em&gt;reward hypothesis&lt;/em&gt; (Sutton and Barto) which says that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;All of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The argument for a scalar reward could be that even if you define your policy using some objective &lt;em&gt;vector&lt;/em&gt; (as in MORL) - you will find a &lt;em&gt;pareto bound&lt;/em&gt; of optimal policies, some of which favour one component of the objective over the other - leaving you (the scientist) responsible for making the ultimate decision concerning the objectives' tradeoff - thus eventually degenerating the reward objective into scalar.&lt;/p&gt;&#xA;&lt;p&gt;In your example there might be two different &amp;quot;optimal&amp;quot; policies - one which results in a very high value of &lt;code&gt;distance_x&lt;/code&gt; but relatively poor &lt;code&gt;distance_y&lt;/code&gt; and a one that favours &lt;code&gt;distance_y&lt;/code&gt; instead. It'll be up to you to find the sweet spot and collapse a reward function back to a scalar.&lt;/p&gt;&#xA;" OwnerUserId="22835" LastActivityDate="2020-07-09T18:36:10.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22421" PostTypeId="1" CreationDate="2020-07-09T19:37:22.937" Score="1" ViewCount="33" Body="&lt;p&gt;Isn't it true that using &lt;a href=&quot;https://pytorch.org/docs/master/generated/torch.max.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;max&lt;/a&gt; over a softmax will be much slower because there is not a smooth gradient?&lt;/p&gt;&#xA;&lt;p&gt;Max basically zeros out the gradients of all the non-maximum values. Especially at the beginning of training, this means it is zeroing out potentially useful features simply because of random weight initialization. Wouldn't this drastically slow down the training in the beginning?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2020-07-09T19:58:27.313" LastActivityDate="2020-07-09T19:58:27.313" Title="Isn't it true that using max over a softmax will be much slower because there is not a smooth gradient?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;training&gt;&lt;gradient-descent&gt;&lt;softmax&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22422" PostTypeId="1" AcceptedAnswerId="22435" CreationDate="2020-07-09T20:50:03.967" Score="1" ViewCount="74" Body="&lt;p&gt;I am training a neural network that takes an input &lt;code&gt;(H, W, 3)&lt;/code&gt; and has the output of size &lt;code&gt;(H', W', C)&lt;/code&gt;. Now, to augment my dataset, since I only have 45k images, I am using the following in my custom data generator&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def Generator():&#xA;img=cv2.imread(trainDir+'\'+imgpath)&#xA;img=tf.keras.preprocessing.image.random_rotation(img,20m,row_axis=0,col_axis=1,channel_axis=2)&#xA;&#xA;output_mask=np.load(trainDir+'\'+maskpath)&#xA;&#xA;yield(img/255-.5,output_mask)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since I am rotating my input images and the output_masks are generated from information about the input (specifically, heat maps around the joint locations) do I also need to rotate the masks as well?&lt;/p&gt;&#xA;" OwnerUserId="32434" LastEditorUserId="2444" LastEditDate="2020-07-10T11:58:30.210" LastActivityDate="2020-07-10T11:58:30.210" Title="Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;data-preprocessing&gt;&lt;data-augmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22423" PostTypeId="1" AcceptedAnswerId="22425" CreationDate="2020-07-09T22:12:14.443" Score="0" ViewCount="99" Body="&lt;p&gt;I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and halt (adding target position to the NN inputs); 2. Walk straight at a specified velocity (adding a target velocity to NN inputs).&lt;/p&gt;&#xA;&lt;p&gt;Now let's say I retrain the original model again to walk properly after changing, say, the mass of the robot. How can I approach &amp;quot;forwarding&amp;quot; this update to the two transfer-learned models? The purpose of this is to minimize re-training time for all future models transfer-learned from the original.&lt;/p&gt;&#xA;&lt;p&gt;(What strikes me as particularly challenging is the fact that the input layer of the transfer-learned models have additional features, so this may re-wire the majority of the NN, making a &amp;quot;forwarded update&amp;quot; completely incompatible...)&lt;/p&gt;&#xA;" OwnerUserId="34441" LastActivityDate="2020-07-10T09:49:10.957" Title="How to &quot;forward&quot; updated NN model to a transferred model?" Tags="&lt;reinforcement-learning&gt;&lt;transfer-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22424" PostTypeId="1" AcceptedAnswerId="22464" CreationDate="2020-07-09T22:34:56.163" Score="2" ViewCount="62" Body="&lt;p&gt;I have recently made a work about the application of neural networks to time series forecasting, and I treated this as a supervised learning (regression) problem. I have come across the suggestion of treating this problem as an unsupervised, semi-supervised, or reinforcement learning problem. The ones that made this suggestion didn't know how to explain this approach and I haven't found any paper of this. So I found myself now trying to figure it out without any success. To my understanding:&lt;/p&gt;&#xA;&lt;p&gt;Unsupervised learning problems (clustering and segmentation reduction) and semi-supervised learning problems (semi-supervised clustering and semi-supervised classification) can be used to decompose the time series but not forecast it.&lt;/p&gt;&#xA;&lt;p&gt;Reinforcement learning problems (model-based and non-model-based on/off-policy) is to decision taken problems, not to forecast.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to treat forecasting time series with neural networks as an unsupervised, semi-supervised, or reinforcement learning problem? How it is done?&lt;/p&gt;&#xA;" OwnerUserId="38501" LastEditorUserId="2444" LastEditDate="2020-07-11T11:23:24.667" LastActivityDate="2020-08-10T22:02:55.307" Title="Should forecasting with neural networks only be treated as a supervised learning (regression) problem?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;unsupervised-learning&gt;&lt;forecasting&gt;&lt;semi-supervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22425" PostTypeId="2" ParentId="22423" CreationDate="2020-07-09T23:14:46.163" Score="1" Body="&lt;p&gt;I think there is no simple way to transfer knowledge changes between different models.&lt;/p&gt;&#xA;&lt;p&gt;If you take your initial model and create a new version of it which you use to learn some other task (like &amp;quot;Walk to a specific location&amp;quot;), then the values copied from the first (original) model change in the second model.&#xA;From that moment on, training the former model on another task will have different effects on its weights than continuing the training of the second model, whose parameter have been changed already.&lt;/p&gt;&#xA;&lt;p&gt;Consider, for example, that you had changed the mass of the robot and trained the initial model on that new task already. Then, if you took all the re-trained parameters from the first model and implanted them into the second model (trained on walking to a certain location), then you would essentially overwrite the additional knowledge the second model had gained already during its initial transfer-learning-process (not even taking into consideration any additional parameters appended to the list of parameters in the second model).&lt;/p&gt;&#xA;&lt;p&gt;So, you will have to re-train all three models (the original one and the two transfer-learning models) if you change the mass of the robot.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There might be an option to apply the same knowledge changes to another model architecture if you refrain from pure transfer learning. This can be achieved with a more modularized model architecture.&lt;/p&gt;&#xA;&lt;p&gt;Consider that you train your first model on walking straight head. Let's call this model &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then you intend to recycle &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; for another task, like walking straight ahead to a given location. Such a model architecture could be realized in two ways:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You apply real transfer learning, retraining a copy of &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; to walk straight ahead until it reaches a certain location&lt;/li&gt;&#xA;&lt;li&gt;You take &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt;, don't change it, but add a second model (let's call it &lt;span class=&quot;math-container&quot;&gt;$m_{navigator}$&lt;/span&gt;), which is trained on predicting &lt;span class=&quot;math-container&quot;&gt;$go$&lt;/span&gt; vs. &lt;span class=&quot;math-container&quot;&gt;$stop$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In the second case, your overall model architecture (let's call it &lt;span class=&quot;math-container&quot;&gt;$m_{go\_to}$&lt;/span&gt;) consists of both one model used for walking (i.e. &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt;) and one model architecture which is used for predicting &lt;span class=&quot;math-container&quot;&gt;$go$&lt;/span&gt; vs. &lt;span class=&quot;math-container&quot;&gt;$stop$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$m_{navigator}$&lt;/span&gt;).&#xA;The idea then is that the robot executes the actions suggested by &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; until &lt;span class=&quot;math-container&quot;&gt;$m_{navigator}$&lt;/span&gt; suggests stopping, upon which prediction the suggestions by &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; will be ignored.&lt;/p&gt;&#xA;&lt;p&gt;Then, whenever you retrain your model &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; (e.g. because the mass of some robot changes), you can simply apply the changes to &lt;span class=&quot;math-container&quot;&gt;$m_{go\_to}$&lt;/span&gt; by replacing &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; by a new version, leaving the rest of &lt;span class=&quot;math-container&quot;&gt;$m_{go\_to}$&lt;/span&gt; intact.&lt;/p&gt;&#xA;&lt;p&gt;If you generalize &lt;span class=&quot;math-container&quot;&gt;$m_{walk}$&lt;/span&gt; to not only walk straight ahead, but also to take turns etc. and you generalize &lt;span class=&quot;math-container&quot;&gt;$m_{navigator}$&lt;/span&gt; to predict going &lt;span class=&quot;math-container&quot;&gt;$left$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$right$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$straight\ ahead$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$back$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$stop$&lt;/span&gt; (being predicted when a certain destination has been reached), you can generalize &lt;span class=&quot;math-container&quot;&gt;$m_{go\_to}$&lt;/span&gt; to walk where-ever you want it to go.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-07-10T09:49:10.957" LastActivityDate="2020-07-10T09:49:10.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22426" PostTypeId="1" AcceptedAnswerId="22427" CreationDate="2020-07-10T00:37:40.403" Score="2" ViewCount="138" Body="&lt;p&gt;Nowadays, the softmax function is widely used in deep learning and, specifically, classification with neural networks. However, the origins of this term and function are almost never mentioned anywhere. So, which paper introduced this term?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-13T22:29:39.100" Title="Which paper introduced the term &quot;softmax&quot;?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;history&gt;&lt;softmax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22427" PostTypeId="2" ParentId="22426" CreationDate="2020-07-10T00:37:40.403" Score="3" Body="&lt;p&gt;The paper that appears to have introduced the &lt;em&gt;term&lt;/em&gt; &amp;quot;softmax&amp;quot; is &lt;a href=&quot;https://papers.nips.cc/paper/195-training-stochastic-model-recognition-algorithms-as-networks-can-lead-to-maximum-mutual-information-estimation-of-parameters.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters&lt;/a&gt; (1989, NIPS) by John S. Bridle.&lt;/p&gt;&#xA;&lt;p&gt;As a side note, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;softmax function&lt;/a&gt; (with base &lt;span class=&quot;math-container&quot;&gt;$b = e^{-\beta}$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma (\mathbf {z} )_{i}={\frac {e^{-\beta z_{i}}}{\sum _{j=1}^{K}e^{-\beta z_{j}}}}{\text{ for }}i=1,\dotsc ,K {\text{ and }}\mathbf {z} =(z_{1},\dotsc ,z_{K})\in \mathbb {R} ^{K}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is very similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boltzmann (or Gibbs) distribution&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p_i=\frac{e^{- {\varepsilon}_i / k T}}{\sum_{j=1}^{M}{e^{- {\varepsilon}_j / k T}}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which was formulated by Ludwig Boltzmann in 1868, so the idea and formulation of the softmax function is quite old.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-13T22:29:39.100" LastActivityDate="2020-07-13T22:29:39.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22428" PostTypeId="2" ParentId="21761" CreationDate="2020-07-10T00:54:26.987" Score="0" Body="&lt;p&gt;&lt;strong&gt;The threshold is typically chosen empirically, so there is no exact answer.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's dependent on how many corners you wish to select, and how strict you want the detection, which could depend on the use case, the dataset, and the block size of the algorithm. If you’re not sure what to choose for the threshold, I would suggest using a scheme relative to your images to deduce the best threshold, based on what performs best for your use case.&lt;/p&gt;&#xA;&lt;p&gt;For example, let the criteria for selection, or scoring function, be &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, which in the Shi-Tomasi case is &lt;span class=&quot;math-container&quot;&gt;$R = min(\lambda_1, \lambda_2)$&lt;/span&gt;. We could choose some value &lt;span class=&quot;math-container&quot;&gt;$q \in (0,1)$&lt;/span&gt;, so that the threshold becomes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$t = q\max_p{R}$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where the max &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is calculated over all points &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, an approach similar to &lt;a href=&quot;https://docs.opencv.org/master/d4/d8c/tutorial_py_shi_tomasi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV Good Features to Track&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You could examine the corners detected and adjust &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; based on how stringently you wanted to select corners. If you don't want to waste the time manually fine-tuning the threshold, you could check out some automated approaches in the literature, such as &lt;a href=&quot;https://ieeexplore.ieee.org/document/6880164&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automated thresholding for low-complexity corner detection&#xA;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38505" LastActivityDate="2020-07-10T00:54:26.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22429" PostTypeId="1" CreationDate="2020-07-10T01:56:37.410" Score="1" ViewCount="95" Body="&lt;p&gt;I implemented a MCES for 2048 (the game), with a quality function implemented as a neural net of a single layer.&lt;/p&gt;&#xA;&lt;p&gt;The starts are created with 6 cells filled with values between 64 and 1024, two cells are 1024 an ther other 8 cells are filled with 0. The game is then progressed until the AI loses or wins and another start is created.&lt;/p&gt;&#xA;&lt;p&gt;After 10 wins the max cell created in the start is reduced in half. Thus, after the first 10 wins, the max cell created in the start is 512.&lt;/p&gt;&#xA;&lt;p&gt;The issue I am having is that after the first 10 wins, the AI gets stuck, it can run around 3 million steps but doesn't get any more wins.&lt;/p&gt;&#xA;&lt;p&gt;How should I create the starts for it to actually learn?&lt;/p&gt;&#xA;&lt;p&gt;Code for reward (&lt;a href=&quot;https://github.com/EmmanuelMess/2048AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;complete code here&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        ArrayList&amp;lt;DataSet&amp;gt; dataSets = new ArrayList&amp;lt;&amp;gt;();&#xA;        double gain = 0;&#xA;&#xA;        for(int i = rewards.size()-1; i &amp;gt;= 0; i--) {&#xA;            gain = gamma * gain + rewards.get(i);&#xA;&#xA;            double lerpGain = reward(gain);&#xA;            INDArray correctOut = output.get(i).putScalar(actions.get(i).ordinal(), lerpGain);&#xA;            dataSets.add(new DataSet(input.get(i), correctOut));&#xA;        }&#xA;&#xA;        Qnetwork.fit(DataSet.merge(dataSets));  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/EmmanuelMess/2048AI&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;public class SimpleAgent {&#xA;    private static final Random random = new Random(SEED);&#xA;&#xA;    private static final MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()&#xA;            .seed(SEED)&#xA;            .weightInit(WeightInit.XAVIER)&#xA;            .updater(new AdaGrad(0.5))&#xA;            .activation(Activation.RELU)&#xA;            .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)&#xA;            .weightDecay(0.0001)&#xA;            .list()&#xA;            .layer(new DenseLayer.Builder()&#xA;                    .nIn(16).nOut(4)&#xA;                    .build())&#xA;            .layer(new OutputLayer.Builder()&#xA;                    .nIn(4).nOut(4)&#xA;                    .lossFunction(LossFunctions.LossFunction.SQUARED_LOSS)&#xA;                    .build())&#xA;            .build();&#xA;&#xA;&#xA;    public SimpleAgent() {&#xA;        Qnetwork.init();&#xA;        ui();&#xA;    }&#xA;&#xA;    private static final double gamma = 0.02;&#xA;&#xA;    private final ArrayList&amp;lt;INDArray&amp;gt; input = new ArrayList&amp;lt;&amp;gt;();&#xA;    private final ArrayList&amp;lt;INDArray&amp;gt; output = new ArrayList&amp;lt;&amp;gt;();&#xA;    private final ArrayList&amp;lt;Double&amp;gt; rewards = new ArrayList&amp;lt;&amp;gt;();&#xA;    private final ArrayList&amp;lt;GameAction&amp;gt; actions = new ArrayList&amp;lt;&amp;gt;();&#xA;&#xA;    private MultiLayerNetwork Qnetwork = new MultiLayerNetwork(conf);&#xA;    private GameEnvironment oldState;&#xA;    private GameEnvironment currentState;&#xA;    private INDArray oldQuality;&#xA;    private double epsilon = 1;&#xA;&#xA;    public void setCurrentState(GameEnvironment currentState) {&#xA;        this.currentState = currentState;&#xA;    }&#xA;&#xA;    public GameAction act() {&#xA;        if(oldState != null) {&#xA;            double reward = currentState.points - oldState.points;&#xA;&#xA;            if (currentState.lost) {&#xA;                reward = 0;&#xA;            }&#xA;&#xA;            input.add(oldState.boardState);&#xA;            output.add(oldQuality);&#xA;            rewards.add(reward);&#xA;&#xA;            epsilon -= (1 - 0.01) / 1000000.;&#xA;        }&#xA;&#xA;        oldState = currentState;&#xA;        oldQuality = Qnetwork.output(currentState.boardState);&#xA;&#xA;        GameAction action;&#xA;&#xA;        if(random.nextDouble() &amp;lt; 1-epsilon) {&#xA;            action = GameAction.values()[oldQuality.argMax(1).getInt()];&#xA;        } else {&#xA;            action = GameAction.values()[new Random().nextInt(GameAction.values().length)];&#xA;        }&#xA;&#xA;        actions.add(action);&#xA;&#xA;        return action;&#xA;    }&#xA;&#xA;    private final int WINS_TO_NORMAL_GAME = 100;&#xA;    private int wonTimes = 0;&#xA;&#xA;    public void setHasWon(boolean won) {&#xA;        if(won) {&#xA;            wonTimes++;&#xA;        }&#xA;    }&#xA;&#xA;    public boolean playNormal() {&#xA;        return wonTimes &amp;gt; WINS_TO_NORMAL_GAME;&#xA;    }&#xA;&#xA;    public boolean shouldRestart() {&#xA;        if (currentState.lost || input.size() == 20) {&#xA;            ArrayList&amp;lt;DataSet&amp;gt; dataSets = new ArrayList&amp;lt;&amp;gt;();&#xA;            double gain = 0;&#xA;&#xA;            for(int i = rewards.size()-1; i &amp;gt;= 0; i--) {&#xA;                gain = gamma * gain + rewards.get(i);&#xA;&#xA;                double lerpGain = reward(gain);&#xA;                INDArray correctOut = output.get(i).putScalar(actions.get(i).ordinal(), lerpGain);&#xA;                dataSets.add(new DataSet(input.get(i), correctOut));&#xA;            }&#xA;&#xA;            Qnetwork.fit(DataSet.merge(dataSets));&#xA;&#xA;            input.clear();&#xA;            output.clear();&#xA;            rewards.clear();&#xA;            actions.clear();&#xA;&#xA;            return true;&#xA;        }&#xA;&#xA;        return false;&#xA;    }&#xA;&#xA;    public Game2048.Tile[] generateState() {&#xA;        double lerped = lerp(wonTimes, WINS_TO_NORMAL_GAME);&#xA;        int filledTiles = 8;&#xA;&#xA;        List&amp;lt;Integer&amp;gt; values = new ArrayList&amp;lt;&amp;gt;(16);&#xA;&#xA;        for (int i = 0; i &amp;lt; 16-filledTiles; i++) {&#xA;            values.add(0);&#xA;        }&#xA;&#xA;        for (int i = 16-filledTiles; i &amp;lt; 14; i++) {&#xA;            values.add((int) (7-7*lerped) + random.nextInt((int) (2- 2*lerped)));&#xA;        }&#xA;&#xA;        values.add((int) ceil(10-10*lerped));&#xA;        values.add((int) ceil(10-10*lerped));&#xA;&#xA;        Collections.shuffle(values);&#xA;&#xA;        return values&#xA;                .stream()&#xA;                .map((value) -&amp;gt; (value == 0? 0: 1 &amp;lt;&amp;lt; value))&#xA;                .map(Game2048.Tile::new)&#xA;                .toArray(Game2048.Tile[]::new);&#xA;    }&#xA;&#xA;    private static double reward(double x) {&#xA;        return x/ 2048;&#xA;    }&#xA;&#xA;    private static double lerp(double x, int maxVal) {&#xA;        return x/maxVal;&#xA;    }&#xA;&#xA;    private void ui() {&#xA;        UIServer uiServer = UIServer.getInstance();&#xA;        StatsStorage statsStorage = new InMemoryStatsStorage();&#xA;        uiServer.attach(statsStorage);&#xA;        Qnetwork.setListeners(new StatsListener(statsStorage));&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14892" LastActivityDate="2020-07-13T18:13:59.943" Title="Monte Carlo Exploring Starts broke for 2048 game AI" Tags="&lt;game-ai&gt;&lt;monte-carlo-methods&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22430" PostTypeId="1" CreationDate="2020-07-10T02:23:15.363" Score="0" ViewCount="59" Body="&lt;p&gt;I want to use a neural network to predict the refractive index of a solution. My thinking is, instead of immediately training on many samples, I will first find the 'ultimate resolution' of the network given the experimental apparatus I am using. What I mean is I will make two different solutions which have refractive indices near the middle of the range of which I am interested. Then I will train the network to classify these two solutions based on reflectance measured from the solution. If it works, say with at least 95% accuracy, then I will make two different solutions in which the difference in refractive index is smaller than before. I will repeat this until the ANN classifies, say below 95%.&lt;/p&gt;&#xA;&lt;p&gt;Will this method of finding the 'resolution' by classification extrapolate well to regression with many more training examples?&lt;/p&gt;&#xA;" OwnerUserId="14811" LastEditorUserId="14811" LastEditDate="2020-07-13T00:24:49.077" LastActivityDate="2020-07-13T00:24:49.077" Title="Finding the 'ultimate resolution' of an ANN" Tags="&lt;classification&gt;&lt;training&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22432" PostTypeId="2" ParentId="11226" CreationDate="2020-07-10T03:47:23.347" Score="0" Body="&lt;p&gt;As far as I understand, the concept of &lt;em&gt;non-Euclidean space&lt;/em&gt; doesn't bring the ordinality or hierarchy among the features, compared to that with the data formed in the Euclidean space.&lt;/p&gt;&#xA;&lt;p&gt;The difference between both these techniques is not remarkable for discriminative tasks like classification. But, for generative modeling, the non-Euclidean techniques helps in defining the latent manifold space for the given data distribution. This can further help in traversing the manifold from the same distribution (to generate similar samples from the same or underlying manifold) even with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; degrees of freedom in the latent space. This is not possible with Euclidean techniques. One cannot fully traverse/generate samples from or outside the manifold without the minimal change in the Euclidean space. More precisely, it can, but it will only present it as noisy data.&lt;/p&gt;&#xA;" OwnerUserId="28451" LastEditorUserId="2444" LastEditDate="2020-07-10T11:51:44.223" LastActivityDate="2020-07-10T11:51:44.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22433" PostTypeId="1" AcceptedAnswerId="22438" CreationDate="2020-07-10T04:10:08.553" Score="1" ViewCount="118" Body="&lt;p&gt;I actually went through the Keras' batch normalization tutorial and the description there puzzled me more.&lt;/p&gt;&#xA;&lt;p&gt;Here are some facts about batch normalization that I read recently and want a deep explanation on it.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you froze all layers of neural networks to their random initialized weights, except for batch normalization layers, you can still get 83% accuracy on CIFAR10.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When setting the trainable layer of batch normalization to false, it will run in inference mode and will not update its mean and variance statistics.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28451" LastEditorUserId="2444" LastEditDate="2020-07-12T13:26:39.097" LastActivityDate="2020-08-11T14:02:00.213" Title="How does batch normalisation actually work?" Tags="&lt;machine-learning&gt;&lt;transfer-learning&gt;&lt;batch-normalization&gt;" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22434" PostTypeId="2" ParentId="22433" CreationDate="2020-07-10T06:08:41.893" Score="0" Body="&lt;p&gt;A batch normalisation layer is like a standard FC layer but instead of learning weights and bias', you learn means and variances and scale the whole layer by said means and variances.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Fact 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Because it behaves just like a normal layer, and can learn, with the right structure it will learn to get a high enough accuracy.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Fact 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Disabling learning on a batch norm layer is just like disabling learning on any other layer. It will not updated any of its parameters, and in this case the parameters are the means and variances, and so these will not be updated.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-07-10T06:08:41.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22435" PostTypeId="2" ParentId="22422" CreationDate="2020-07-10T08:08:05.053" Score="1" Body="&lt;p&gt;Yes! This is crucial.&lt;/p&gt;&#xA;&lt;p&gt;If you rotate your input images for segmentation, you need to rotate the output masks as well. Otherwise the loss of your network will not be correctly calculated and your network will not learn how to generalize to rotated input images.&lt;/p&gt;&#xA;&lt;p&gt;If you use keras, you can use two &lt;code&gt;ImageDataGenerator&lt;/code&gt; classes, one for the images and one for the masks, with the &lt;strong&gt;same&lt;/strong&gt; random seed and augmentation parameters. It looks something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data_gen_args = dict(rotation_range=45)&#xA;&#xA;image_datagen = ImageDataGenerator(**data_gen_args)&#xA;mask_datagen = ImageDataGenerator(**data_gen_args)&#xA;&#xA;image_generator = image_datagen.flow(train,seed=SEED)&#xA;mask_generator = mask_datagen.flow(y_train,seed=SEED)&#xA;train_generator = zip(image_generator, mask_generator)&#xA;&#xA;model.compile(...)&#xA;model.fit_generator(train_generator, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-07-10T08:08:05.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22436" PostTypeId="1" AcceptedAnswerId="22439" CreationDate="2020-07-10T08:33:16.017" Score="1" ViewCount="30" Body="&lt;p&gt;&amp;quot;simple&amp;quot; issue:&#xA;I'm building a model, where from a feature set A I want to predict a target set C; I need to understand if another feature set B, together with A, can improve my model performances, instead of using only A.&lt;/p&gt;&#xA;&lt;p&gt;Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.&lt;/p&gt;&#xA;&lt;p&gt;If I get good predictions when testing the model A -&amp;gt; B, is it true then that adding B to A in predicting C is completely useless ?&lt;/p&gt;&#xA;&lt;p&gt;And furthermore, are there smarter ways to decide if/when a feature is useless ?&lt;/p&gt;&#xA;" OwnerUserId="36504" LastActivityDate="2020-07-10T09:21:49.937" Title="When adding a feature is useless?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;prediction&gt;&lt;feature-selection&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22437" PostTypeId="1" CreationDate="2020-07-10T08:48:32.693" Score="0" ViewCount="30" Body="&lt;p&gt;What AI method can I use to detect known pattern in a large signal?&lt;/p&gt;&#xA;&lt;p&gt;This topic is a bit theoretical, but I will make it easier to understand given the following example:&lt;/p&gt;&#xA;&lt;p&gt;DGAWILLYOUEVERDGQLYNOTICEITPKL&lt;/p&gt;&#xA;&lt;p&gt;For a human it is quite simple to detect in the given sequence of letters a short sentence. Even if there is noise: &amp;quot;WILL YOU EVER NOTICE IT&amp;quot;. So the focus is not really on a string (CTRL+F solution is not what I am looking for). This is just an example, I actually wan't to apply it on a signal? And what if there are small mistakes like: &amp;quot;WILL YO EVER NOTICE IT&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastActivityDate="2020-07-10T08:48:32.693" Title="How to detect known pattern in a huge signal" Tags="&lt;signal-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22438" PostTypeId="2" ParentId="22433" CreationDate="2020-07-10T08:52:10.530" Score="0" Body="&lt;p&gt;I'm not sure how just training the batch normalisation layer, you can get an accuracy of 83%. The batch normalisation layer parameters &lt;span class=&quot;math-container&quot;&gt;$\gamma^{(k)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta^{(k)}$&lt;/span&gt;, are used to scale and shift the normalised batch outputs. These parameters are learnt during the back-propagation step. For the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th layer, &lt;span class=&quot;math-container&quot;&gt;$$y^{(k)} = \gamma^{(k)}\hat{x}^{(k)} + \beta^{(k)}$$&lt;/span&gt; The scaling and shifting are done to ensure a non - linear activation being outputted by each layer. Because batch normalisation scales outputs between 0-1, some activation functions are linear within that range (Eg. &lt;span class=&quot;math-container&quot;&gt;$tahh$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$sigmoid$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Regarding the second fact however, the difference between training and inference mode is this. During training mode, the statistics of each batch norm layer &lt;span class=&quot;math-container&quot;&gt;$\mu_B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2_B$&lt;/span&gt; is computed. This statistic is used in scaling and normalising the outputs of the batch norm layer to have 0 mean and unit variance. At the same time, the current batch statistic computed is also used to update the running mean and running variance of the population. &lt;span class=&quot;math-container&quot;&gt;$\mu_B[t]$&lt;/span&gt; represents the current batch mean, &lt;span class=&quot;math-container&quot;&gt;$\sigma^2_B[t]$&lt;/span&gt; represents the current batch variance, while &lt;span class=&quot;math-container&quot;&gt;$\mu'_B[t]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma'_B[t]$&lt;/span&gt; represent the accumulated means and variance from the previous batches. The running mean and variance of the population is then updated as &lt;span class=&quot;math-container&quot;&gt;$$\mu'_B[t]=\mu'_B[t]× momentum+ \mu_B[t]×(1−momentum)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma'^2_B[t]=\sigma'^2_B[t] × momentum + \sigma^2_B[t]×(1−momentum)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In inference mode, the batch normalisation uses the running mean and variance computed during training mode to scale and normalise inputs in the batch norm layer instead of the current batch mean and variance.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-07-10T08:52:10.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22439" PostTypeId="2" ParentId="22436" CreationDate="2020-07-10T08:59:44.310" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This will help inform you how much redundancy there is between A and B. However, even if you can predict B with 100% accuracy from A, you &lt;em&gt;may&lt;/em&gt; still be better off using A+B (instead of A alone) to predict C.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If I get good predictions when testing the model A -&amp;gt; B, is it true then that adding B to A in predicting C is completely useless ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is an indicator that adding B &lt;em&gt;probably&lt;/em&gt; won't make great improvements to your prediction of C.&lt;/p&gt;&#xA;&lt;p&gt;The only way to be sure is to make the model that uses A+B and compare its performance against a model that uses only A. If collecting B costs time or other resources, then perform this check by limiting both models to only learn from the subset of data where you have all of A, B, C available.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And furthermore, are there smarter ways to decide if/when a feature is useless ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Another thing you can do is to try and predict C from B alone. It doesn't need to score well, but may indicate that something useful is in the data if you get better than chance results. Scoring badly unfortunately doesn't rule it out for working well in combination with A.&lt;/p&gt;&#xA;&lt;p&gt;Generally, if you cannot reason it clearly one way or another from theory, the accepted method is to build variations of your model with and without a feature and test them. You do have to be aware of the &lt;a href=&quot;https://www.tylervigen.com/spurious-correlations&quot; rel=&quot;nofollow noreferrer&quot;&gt;chances of spurious correlation&lt;/a&gt; though, so usually you need to have some basis or motivation for considering any new feature, from your domain knowledge that applies to the model.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-10T09:21:49.937" LastActivityDate="2020-07-10T09:21:49.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22440" PostTypeId="1" CreationDate="2020-07-10T10:14:57.753" Score="2" ViewCount="92" Body="&lt;p&gt;I've been using &lt;a href=&quot;https://github.com/matterport/Mask_RCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;matterport's Mask R-CNN&lt;/a&gt; to train on a custom dataset. However, there seem to be some parameters that i failed to correctly define because on practically all of the images, the bottom or top of the object's mask is cut off:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YiGbq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YiGbq.png&quot; alt=&quot;innacurate prediction: mask is cut-off on the bottom&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, the bounding box is fine since it covers the whole blade, but the mask seems to suddenly stop in a horizontal line on the bottom.&lt;/p&gt;&#xA;&lt;p&gt;On another hand, there is a stair-like effect on masks of larger and curvier objects such as this one (in addition to the bottom and top cut-offs):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/riTf8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/riTf8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The original images are downscaled to &lt;code&gt;IMAGE_MIN_DIM = IMAGE_MAX_DIM = 1024&lt;/code&gt; using the &amp;quot;square&amp;quot; mode.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;USE_MINI_MASK&lt;/code&gt; is set to true with &lt;code&gt;MINI_MASK_SHAPE = (512, 512)&lt;/code&gt; (somehow if i set it off, RAM gets filled and training chrashes).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;RPN_ANCHOR_SCALES = (64, 128, 256, 512, 1024)&lt;/code&gt; since the objects occupy a large space of the image.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It doesn't feel like the problem comes from the amount of training. These two predictions come from 6 epochs of 7000 steps per epoch (took around 17 hours). And the problem appears from early stage and persists along all the epochs.&lt;/p&gt;&#xA;&lt;p&gt;I posted the same &lt;a href=&quot;https://stackoverflow.com/questions/62810854/inaccurate-masks-with-mask-rcnn-stairs-effect-and-sudden-stops&quot;&gt;question&lt;/a&gt; on stack overflow, and an answer pointed out that this issue is common when using mask r-cnn. It also suggested to look at &lt;a href=&quot;https://research.fb.com/wp-content/uploads/2020/06/PointRend-Image-Segmentation-as-Rendering.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PointRend&lt;/a&gt;, an implementation to mask r-cnn that addresses this issue.&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless I feel like I could still optimize my model and use the full potential of mask r-cnn before looking for an alternative.&lt;/p&gt;&#xA;&lt;p&gt;Any idea on what changes to make ?&lt;/p&gt;&#xA;" OwnerUserId="19155" LastEditorUserId="19155" LastEditDate="2020-07-13T07:40:45.217" LastActivityDate="2020-07-13T07:40:45.217" Title="Inaccurate masks with Mask-RCNN: Stairs effect and sudden stops" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;keras&gt;&lt;r-cnn&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22441" PostTypeId="1" CreationDate="2020-07-10T11:44:46.517" Score="0" ViewCount="49" Body="&lt;h3&gt;Context&lt;/h3&gt;&#xA;&lt;p&gt;I'm trying to dive into deep learning for tasks on images, and trying to figure out how to reuse some well-known &lt;em&gt;structures&lt;/em&gt;* that have been published, mainly on github.&lt;br /&gt;&#xA;( *Here, &lt;em&gt;structure&lt;/em&gt; can be replaced by here by one or more of the concepts used hereafter.)&lt;/p&gt;&#xA;&lt;h3&gt;Struggling&lt;/h3&gt;&#xA;&lt;p&gt;But while reading articles, blog posts and watching videos or paper presentations on deep learning, especially about &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;ConvNets&lt;/a&gt; and researches applied on images (classification, object detection, semantic segmentation or scene understanding) I'm struggling with these concepts; backbones, frontends, models, networks, and architectures.&lt;/p&gt;&#xA;&lt;p&gt;For me, they are almost interchangeable, except maybe for the model, which is according to my current knowledge, the resulting weights matrix of the learning process (which undoubtedly has to be associated with the network used for this training phase).&lt;/p&gt;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&lt;p&gt;I would be very kind if someone can define these concepts and explain their differences thoroughly and rigorously (maybe with links to papers if they have been commonly accepted in the scientific literature).&lt;/p&gt;&#xA;" OwnerUserId="20761" LastEditorUserId="2444" LastEditDate="2020-07-10T12:02:31.957" LastActivityDate="2020-07-10T12:02:31.957" Title="What are the differences between backbones, frontends, models and architectures in applied deep learning?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;models&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22442" PostTypeId="1" CreationDate="2020-07-10T13:46:21.413" Score="0" ViewCount="42" Body="&lt;p&gt;I'm defining a deep network to emulate a multitarget regression.&lt;/p&gt;&#xA;&lt;p&gt;When I costruct my training set, I take information from a graph; without going into too much detail, it could happen that I take 2 times the same value (maybe arriving on the same node from different points).&lt;/p&gt;&#xA;&lt;p&gt;Thus I have duplicates in my input.&lt;/p&gt;&#xA;&lt;p&gt;I have read that this &amp;quot;condition&amp;quot; could adversely alter the result of the training, and I would like to understand why.&lt;/p&gt;&#xA;&lt;p&gt;Just intuitively, in my understanding, when my input vector has 2 equal values, once reached the output and computed the error, its backpropagation kind of influence equally the weights related to these 2 input neurons which had the same value.&lt;/p&gt;&#xA;&lt;p&gt;Is the poor training due to the fact that the &amp;quot;correction&amp;quot; is split into the two neurons instead of focusing on only one?&lt;/p&gt;&#xA;" OwnerUserId="36504" LastEditorUserId="36082" LastEditDate="2020-07-11T19:50:41.627" LastActivityDate="2020-07-11T19:50:41.627" Title="Why is it that having a duplicate in features set makes training to work bad" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;features&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22443" PostTypeId="1" CreationDate="2020-07-10T14:18:09.760" Score="1" ViewCount="82" Body="&lt;p&gt;What is the idea behind double DQN?&lt;/p&gt;&#xA;&lt;p&gt;The target in double DQN is computed as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Y_{t}^{\text {DoubleQ }} \equiv R_{t+1}+\gamma Q\left(S_{t+1}, \underset{a}{\operatorname{argmax}} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}\right) ; \boldsymbol{\theta}_{t}^{\prime}\right),&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}_{t}^{\prime}$&lt;/span&gt; are the weights of the &lt;em&gt;target network&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}_{t}$&lt;/span&gt; are the weights of the online value network&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the discount factor&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;On the other hand, the target in DQN is computed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y_{t}^{\mathrm{DQN}} \equiv R_{t+1}+\gamma \max _{a} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}^{-}\right),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}_{t}^{-}$&lt;/span&gt; are the weights of the &lt;em&gt;target network&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The target network for evaluating the action is updated using weights of the online network and the value fed to the target value is basically the old q value of the action.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas on how or why adding another network based on weights from the first network helps? Any example?&lt;/p&gt;&#xA;" OwnerUserId="38519" LastEditorUserId="2444" LastEditDate="2020-11-04T21:32:00.377" LastActivityDate="2020-11-04T21:32:00.377" Title="Why does adding another network help in double DQN?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-11-04T21:35:19.077" ContentLicense="CC BY-SA 4.0" />
  <row Id="22445" PostTypeId="1" CreationDate="2020-07-10T17:39:54.843" Score="0" ViewCount="68" Body="&lt;p&gt;I am trying to learn backpropagation and this is what I know so far.&lt;/p&gt;&#xA;&lt;p&gt;To update the weights of the neural network you have to figure out the partial derivative of each of the parameters on the loss function using the chain rule. List all of these partial derivatives in a column vector and you have your gradient vector of your current parameter's on the loss function. Then by taking the negative of the gradient vector to descend the loss function and multiplying it by the learning rate (step size) and adding it to your original gradient vector, you have your new weights.&lt;/p&gt;&#xA;&lt;p&gt;Is my understanding correct? Also, how can this be done in iterations over training examples?&lt;/p&gt;&#xA;" OwnerUserId="38523" LastEditorUserId="2444" LastEditDate="2020-07-10T19:59:46.597" LastActivityDate="2020-07-10T20:18:18.937" Title="Is my understanding of back-propogation correct?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22446" PostTypeId="1" CreationDate="2020-07-10T18:46:38.427" Score="0" ViewCount="23" Body="&lt;p&gt;So most of AI scripts that output something (GANs generating image files for example)  use CUDA and thus require a GPU to run them.&lt;/p&gt;&#xA;&lt;p&gt;I have a 7 year old Nvidia GPU which does not support that and thus I can run only a tiny fraction of all the fun stuff that has been released lately.&lt;/p&gt;&#xA;&lt;p&gt;I did some research and found out there is something called Ocelot which allows the CPU to be used instead.&lt;/p&gt;&#xA;&lt;p&gt;How would I go about this? Can I just take any script and run it through Ocelot? Or would the developer have needed to build something into the script to support it being run on a CPU?&lt;/p&gt;&#xA;" OwnerUserId="38524" LastActivityDate="2020-07-10T18:46:38.427" Title="How to use Ocelot to run an AI script on the CPU which would normally use GPU?" Tags="&lt;ai-design&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22447" PostTypeId="2" ParentId="21982" CreationDate="2020-07-10T18:48:29.950" Score="2" Body="&lt;p&gt;You are right that the baseline score is near zero only when there are a large number of label classes, i.e., when k is large. We should have qualified this line in the paper more carefully.&lt;/p&gt;&#xA;&lt;p&gt;In this sense, formally, the technique explains the *difference in prediction between the input score and the baseline score, as is made clear elsewhere in the paper (see Remark 1 and Proposition 1 for instance.)&lt;/p&gt;&#xA;" OwnerUserId="38525" LastActivityDate="2020-07-10T18:48:29.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22448" PostTypeId="2" ParentId="22445" CreationDate="2020-07-10T19:58:42.060" Score="2" Body="&lt;p&gt;Your understanding seems to be correct (although your explanation isn't completely precise), apart from &amp;quot;adding it to your original gradient vector&amp;quot;. You add the gradient vector to the parameters/weights vector.&lt;/p&gt;&#xA;&lt;p&gt;(Note that back-propagation is just the algorithm that computes the gradient vector. The update of the parameters with the gradient of the loss function is the gradient descent/ascent step, even though it's true that some people, at least in the context of deep learning, refer to the combination of gradient descent and the computation of the gradients as &lt;em&gt;the back-propagation algorithm&lt;/em&gt;, but this is just a terminology issue, which you should not dwell too much on.)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Also, how can this be done in iterations over training examples?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I understand this question correctly, you want to know how we would update the parameters when there's more than one training example. In that case, you compute a gradient vector for each training example. Then the average of the gradient vectors (which is also a vector) is the actual gradient vector that you use to update the parameters. See &lt;a href=&quot;https://ai.stackexchange.com/a/20380/2444&quot;&gt;this answer&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-10T20:18:18.937" LastActivityDate="2020-07-10T20:18:18.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22449" PostTypeId="1" CreationDate="2020-07-10T20:52:43.937" Score="1" ViewCount="238" Body="&lt;p&gt;I would like to ask a question about the relationship of accuracy with the loss function.&lt;/p&gt;&#xA;&lt;p&gt;My experiment is a multiclass text classification problem, and I have built a Keras neural network to tackle it. My labels are something like&lt;/p&gt;&#xA;&lt;p&gt;array([array([0, 0, 0, 0, 0, 1, 0, 1]), array([0, 1, 1, 0, 0, 0, 0, 1])])&lt;/p&gt;&#xA;&lt;p&gt;For the final output layer I use the 'sigmoid' activation function and for loss the 'binary crossentropy', however, I am a bit confused about the metric. I am using the F1_score metric because Accuracy it's not a metric to count on when there are many more negative labels than positive labels. So, since the problem is multilabel classification shall I use the multi-label mode like tfa.metrics.F1_score(mode=&amp;quot;micro&amp;quot;). Is that correct? or since I use binary_crossentropy and sigmoid activation function should I use the standard binary f1-score because every label-tag Is independent to the others and has a different Bernoulli distribution?&lt;/p&gt;&#xA;&lt;p&gt;I would really like to get your input on this. My humble opinion is that I should you the binary standard mode of binary f1-score and not the multi-label micro approach even though my experiment is multi-label text classification.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My current approach (using micro F1-score since my y_train is Multi-label&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model_for_pruning.compile(optimizer='adam',&#xA;                          loss='binary_crossentropy',&#xA;                          metrics=[tfa.metrics.F1Score(y_train[0].shape[-1], average=&amp;quot;micro&amp;quot;)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;My alternative approach (based on the binary_crossentropy and the sigmoid activation function, despite I have multi-label y_train)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model_for_pruning.compile(optimizer='adam',&#xA;                          loss='binary_crossentropy',&#xA;                          metrics=[tfa.metrics.F1Score(y_train[0].shape[-1], average=None)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Τhe reason why I use sigmoid and not softmax as the output layer&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/towards-artificial-intelligence/keras-for-multi-label-text-classification-86d194311d0e&quot; rel=&quot;nofollow noreferrer&quot;&gt;relevant link&lt;/a&gt;&#xA;Why Sigmoid and not Softmax in the final dense layer?&#xA;In the final layer of the above architecture, sigmoid function as been used instead of softmax. The advantage of using sigmoid over Softmax lies in the fact that one synopsis may have many possible genres. Using the Softmax function would imply that the probability of occurrence of one genre depends on the occurrence of other genres. But for this application, we need a function that would give scores for the occurrence of genres, which would be independent of occurrences of any other movie genre.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.pyimagesearch.com/2018/05/07/multi-label-classification-with-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;relevant link 2&lt;/a&gt;&#xA;Binary cross-entropy rather than categorical cross-entropy.&#xA;This may seem counterintuitive for multi-label classification; however, the goal is to treat each output label as an independent Bernoulli distribution and we want to penalize each output node independently.&lt;/p&gt;&#xA;&lt;p&gt;Please check my reasoning behind this and I would be happy if we can contradict this explanation. To better explain my experiment, I want to predict movie genres. So a movie can belong to 1 or more genres ['Action', 'Comedy', 'Children'], so when I use softmax the probability sum to 1, while when I use sigmoid its single probability of a class has a range between (0,1). Thus, if the predictions are correct the genres with the highest probabilities are those assigned to the movie. So imagine that my vector of prediction probabilities are something like [0.15, 0.12, 0.54, 0.78, 0.99] sum()&amp;gt; 1, and not something like [0.12, 0.43, 11, 0.32,  0.01, 0.01) sum() = 1.&lt;/p&gt;&#xA;" OwnerUserId="38530" LastEditorUserId="38530" LastEditDate="2020-07-11T13:39:58.673" LastActivityDate="2021-04-07T15:06:50.740" Title="Binary mode or Multi-label mode is correct when using binary crossentropy and sigmoid output function on multi-label classification" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22450" PostTypeId="1" CreationDate="2020-07-10T22:03:36.810" Score="2" ViewCount="67" Body="&lt;p&gt;I have been working on understanding how &lt;a href=&quot;https://openaccess.thecvf.com/content_ECCV_2018/papers/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CornerNet&lt;/a&gt; works, but I couldn't figure out a few parts about the architecture.&lt;/p&gt;&#xA;&lt;p&gt;First, the authors mention that there are 3 distinct parts to be predicted as a heatmap, embedding, and offset.&lt;/p&gt;&#xA;&lt;p&gt;Also, in &lt;a href=&quot;https://openaccess.thecvf.com/content_ECCV_2018/papers/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;, it is stated that the network was trained on the COCO dataset, which has bounding box and class annotations.&lt;/p&gt;&#xA;&lt;p&gt;As far as I am concerned, since CornerNet is based on detecting the top-left and bottom-right corners, the ground-truth labels for heatmap should be composed of top-left and bottom-right pixel locations of bounding boxes with the class score (but I might be wrong). What is the heatmap used for?&lt;/p&gt;&#xA;&lt;p&gt;Moreover, for the embedding part, authors used the pull&amp;amp;push loss at the ground-truth pixel locations to find out which corner pairs belong to which object, but I don't understand how to backpropagate this loss. How do I back-propagate the embedding loss?&lt;/p&gt;&#xA;" OwnerUserId="38533" LastEditorUserId="2444" LastEditDate="2020-07-11T18:33:32.907" LastActivityDate="2021-05-06T11:22:10.073" Title="What is a heatmap in the CornerNet paper?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;papers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22451" PostTypeId="2" ParentId="22410" CreationDate="2020-07-11T00:06:06.513" Score="0" Body="&lt;p&gt;From how it looks, the most reliable method to try out is using Hough transform.&lt;/p&gt;&#xA;&lt;p&gt;The Hough transform can be used to detect e.g. lines and circles in images (depending on which variant you are using; in this case it would amount to a combination of variants for both lines and circles obviously).&lt;/p&gt;&#xA;&lt;p&gt;So, given some input image, the Hough transform tells you what the line/circle parameters are that have created a line in the input image. For example, given a line, it would tell you the intersection with the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-axis and the &lt;span class=&quot;math-container&quot;&gt;$slope$&lt;/span&gt; of the line detected in the image.&#xA;Then, you could use these parameter information to reconstruct lines and circles detected in the image.&lt;/p&gt;&#xA;&lt;p&gt;The last remaining problem to be solved then is to check where a detected line starts and stops (since this is not obvious from line parameters like &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; (=&lt;span class=&quot;math-container&quot;&gt;$slope$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; (=intersection with &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-axis) in the equation &lt;span class=&quot;math-container&quot;&gt;$y = mx+b$&lt;/span&gt; describing some line).&lt;/p&gt;&#xA;&lt;p&gt;But for that, you could &amp;quot;walk&amp;quot; along a line in the image space and check where the line is present or not. Then, you can draw line segments in the reconstruction image when the corresponding elements are also present in the original image.&lt;/p&gt;&#xA;&lt;p&gt;The problem with (C)NNs would be that they are sensitive to rotation and scale etc. You could of course take a tremendously large number of filters to account for shapes of different rotation and scale, but that would increase the demand for labeled training data again (which could of course be automatically be generated a priori in this simple case).&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I'd suggest checking out Hough transform. To get some feeling for it, there are lots of libraries available implementing it for &lt;a href=&quot;https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python&lt;/a&gt; or &lt;a href=&quot;https://www.mathworks.com/help/images/ref/hough.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MatLab&lt;/a&gt;, for example.&lt;/p&gt;&#xA;&lt;p&gt;For further information, check out &lt;a href=&quot;https://en.wikipedia.org/wiki/Hough_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; or &lt;a href=&quot;https://www.youtube.com/watch?v=hYcugbbf9ug&amp;amp;feature=youtu.be&amp;amp;t=756&quot; rel=&quot;nofollow noreferrer&quot;&gt;YouTube&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-07-11T00:06:06.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22452" PostTypeId="1" AcceptedAnswerId="22459" CreationDate="2020-07-11T08:25:32.980" Score="2" ViewCount="53" Body="&lt;p&gt;In the book of Barto and Sutton, there are 3 methods presented that solve an RL problem: DP, Monte Carlo, and TD. But which category does policy gradient methods (or actor-only methods) classify in? Should I classify them as the 4th method of solving a reinforcement learning problem?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-07-11T11:25:33.667" LastActivityDate="2020-07-11T14:55:22.110" Title="How can I classify policy gradient methods in RL?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22453" PostTypeId="2" ParentId="22433" CreationDate="2020-07-11T09:47:55.823" Score="0" Body="&lt;p&gt;The original paper by Sergey Ioffe and Christian Szegedy; &lt;a href=&quot;https://arxiv.org/abs/1502.03167&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1502.03167&lt;/a&gt; &amp;quot;Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift&amp;quot; is very good. Make sure to go through the paper slowly and make annotations to truly understand it.&lt;/p&gt;&#xA;" OwnerUserId="36082" LastActivityDate="2020-07-11T09:47:55.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22454" PostTypeId="2" ParentId="22443" CreationDate="2020-07-11T11:21:30.287" Score="1" Body="&lt;p&gt;As the authors of &lt;a href=&quot;https://cs.brown.edu/people/kasadiat/pdf/deep_mellow.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; state it:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, the agent updates the value of executing an action in the current state, using the values of executing actions in a successive state. This procedure often results in an instability because the values change simultaneously on both sides of the update equation.  A target network is a copy of the estimated value function that is held fixed to serve as a stable target for some number of steps.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I remember it correctly, the main concern is that the network could end up in a positive feedback loop, making sufficient exploration of various action and state combinations less likely to occur, which could be detrimental to the learning task.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-07-11T11:21:30.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22455" PostTypeId="2" ParentId="20808" CreationDate="2020-07-11T11:51:34.107" Score="3" Body="&lt;p&gt;Some basic advantages of MCTS over Minimax (and its many extensions, like Alpha-Beta pruning and all the other extensions over that) are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;MCTS does not &lt;strong&gt;need&lt;/strong&gt; a heuristic evaluation function for states. It can make meaningful evaluations just from random playouts that reach terminal game states where you can use the loss/draw/win outcome. So if you're faced with a domain where you have absolutely no heuristic domain knowledge that you can plug in, MCTS is likely a better choice. Minimax &lt;strong&gt;must&lt;/strong&gt; have a heuristic evaluation function for states (&lt;strong&gt;exception&lt;/strong&gt;: if you game is so simple that you can afford to compute the complete game tree and reach all terminal game states immediately from the the initial game state, you don't need heuristics). If you do have strong evaluation functions, you can still incorporate them and use them to improve MCTS too; they're just not strictly necessary for MCTS.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MCTS has simpler &lt;strong&gt;anytime&lt;/strong&gt; behaviour; you can just keep running iterations until you run out of computing time, and then return the best move. Typically we expect the performance level of MCTS to grow with computatinon time / iteration count relatively smoothly (not always 100% true, but intuitively you can usually expect something like this). You can sort of achieve anytime behaviour in minimax with &lt;strong&gt;iterative deepening&lt;/strong&gt;, but that's usually a bit less &amp;quot;smooth&amp;quot;, a bit more &amp;quot;bumpy&amp;quot;; this is because every time you increase the search depth, you need &lt;em&gt;significantly&lt;/em&gt; more processing time than you did for the previous depth limit. If you run out of time and have to abort your current search at your current depth limit, that last search will be completely useless; you'll have to discard it and stick to the results from the previous search with the previous depth limit.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A difference, which is not necessarily an advantage or disadvantage either way in the general case (but can be in specific cases):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The computation time of MCTS is generally dominated by running (semi-)random playouts. This means that functions for computing legal move lists, and applying moves to game states, typically dictate how fast or slow your MCTS runs; making these functions faster will generally make your MCTS faster. On the other hand, the computation time of Minimax is generally dominated by copying game states (or &amp;quot;undoing&amp;quot; moves, which is an operation that in most games will require additional memory usage for game states to be possible) and heuristic evaluation functions (though the latter are likely to also become important in terms of computation cost in MCTS if you choose to include them there). In some games it will be easier to provide efficient implementations for one of these, and in other games it may be different.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A basic advantage of Minimax over MCTS:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In settings where MCTS can only run &lt;strong&gt;very few&lt;/strong&gt; iterations relative to the branching factor (or in the extreme case, fewer iterations than there are actions available in the root node), MCTS will perform extremely poorly / close to random play. We've noticed this being the case for quite a decent number of games in our general game system Ludii (where the &amp;quot;general game system&amp;quot; often implies that games are implemented less efficiently than they could be in a dedicated single-game-specific program) with low time controls (like 1 second per move). This same general game setting often makes it difficult to find super strong heuristics, but it's generally still possible to come up with some relatively simple ones (like just a simple material heuristic in chess). An alpha-beta search with just a couple of search plies and a basic, simple heuristic will often outperform a close-to-random MCTS if the MCTS can't manage to run significantly more iterations than it has legal moves in the root node.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-07-11T11:51:34.107" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22456" PostTypeId="1" CreationDate="2020-07-11T12:55:47.423" Score="0" ViewCount="48" Body="&lt;p&gt;I am trying to predict nursing activity using mobile accelerometer data. My dataset is a CSV file containing x, y, z component of acceleration. Each frame contains 20-second data. The dataset is highly imbalance, so I perform data augmentation and balance the data. In the data augmentation technique, I only use scaling and my assumption is, if I scale down or up a signal the activity remains the same. Using this assumption I augmented the data and my validation set not only contain the original signals but also the augmented (scaling) signals. Using this process, I am getting quite a good accuracy that I never being expected using only data augmentation. So, I am thinking that I performed a terrible mistake somewhere. I check the code, everything is right. So now I think, since my validation set has augmented data, that's the reason of getting this high accuracy (maybe the augmented data is really easy to classify).&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-07-11T13:22:40.320" Title="Can I use augmented data in the validation set?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22457" PostTypeId="2" ParentId="22449" CreationDate="2020-07-11T13:21:01.130" Score="0" Body="&lt;p&gt;Since you have a multiclass classification problem rather than a binary classification problem (i.e. a two-class problem), I recommend to adjust your architecture and use &lt;code&gt;softmax&lt;/code&gt; instead of &lt;code&gt;sigmoid&lt;/code&gt; as final activation function and &lt;code&gt;categorcal_crossentropy&lt;/code&gt; instead of &lt;code&gt;binary_crossentropy&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Softmax will ensure all your outputs are valid probabilities. This is crucial in calculating the loss of your model. Using sigmoid will only ensure this if you have binary classes. Further categorical crossentropy is used for multiclass problems, again this is to make sure you calculate the correct loss. This is just a general model characteristic:&lt;br /&gt;&#xA;&lt;em&gt;binary&lt;/em&gt; -&amp;gt; &lt;code&gt;sigmoid&lt;/code&gt; &amp;amp; &lt;code&gt;binary_crossentropy&lt;/code&gt;&lt;br /&gt;&#xA;&lt;em&gt;multi-class&lt;/em&gt; -&amp;gt; &lt;code&gt;softmax&lt;/code&gt; &amp;amp; &lt;code&gt;(sparse_)categorical_crossentropy&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;To answer you original question, I think you should use &lt;code&gt;average = micro&lt;/code&gt;, &lt;code&gt;macro&lt;/code&gt; or &lt;code&gt;weighted&lt;/code&gt; when calculating the F1-score. This depends on what you like your model to put more emphasis on and how severe your class imbalances are.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-07-11T13:21:01.130" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22458" PostTypeId="2" ParentId="22456" CreationDate="2020-07-11T13:22:40.320" Score="2" Body="&lt;p&gt;You should &lt;strong&gt;not&lt;/strong&gt; use augmented data in the validation nor in the test set.&lt;/p&gt;&#xA;&lt;p&gt;Validation and test set are purely used for hyperparameter tuning and estimating the final performance, i.e. estimating the generalization error. These two data sets should be as close as possible to other data, which you could have acquired, but you actually haven not, i.e. your true data distribution.&lt;/p&gt;&#xA;&lt;p&gt;It's fine to augment training data, since it mimics other samples from the true distribution by applying transformations, noise, etc. and thus helps to increase generalization performance (assuming your augmentation assumptions are right). But evaluation should always be performed on original data.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-07-11T13:22:40.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22459" PostTypeId="2" ParentId="22452" CreationDate="2020-07-11T14:55:22.110" Score="2" Body="&lt;p&gt;DP, Monte Carlo, and TD are methods of estimating returns. Policy gradient describes methods of learning a policy. So policy gradients serve a different purpose than the other things you mentioned. For clarity, you can use Monte Carlo or TD methods to estimate returns to construct the loss that you get your policy gradient from.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-11T14:55:22.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22460" PostTypeId="1" CreationDate="2020-07-11T17:01:47.077" Score="0" ViewCount="168" Body="&lt;p&gt;I have developed a basic feedforward neural network from scratch to classify whether image is of cat or not cat. It works fine, but after 2500 iterations, my cost function is not reducing properly.&lt;/p&gt;&#xA;&lt;p&gt;The loss function which I am using is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L(\hat{y},y) = -ylog\hat{y}-(1-y)log(1-\hat{y})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can you please point out where I am going wrong the link to the notebook is&#xA;&lt;a href=&quot;https://www.kaggle.com/sidcodegladiator/catnoncat-nn&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/sidcodegladiator/catnoncat-nn&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="38546" LastEditorUserId="38546" LastEditDate="2020-07-12T03:37:20.330" LastActivityDate="2021-04-15T06:56:45.620" Title="Why isn't the loss of my neural network reduced after 2500 iterations?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22462" PostTypeId="1" CreationDate="2020-07-11T18:13:58.677" Score="2" ViewCount="163" Body="&lt;p&gt;I'm trying to understand the DDPG algorithm using Keras&lt;/p&gt;&#xA;&lt;p&gt;I found the &lt;a href=&quot;https://keras.io/examples/rl/ddpg_pendulum/&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt; and started analyzing the code, I can't understand 2 things.&lt;/p&gt;&#xA;&lt;p&gt;The algorithm used to write the code presented on the page&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OnnPC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OnnPC.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the algorithm image, updating the critic's network does not require gradient&lt;/p&gt;&#xA;&lt;p&gt;But the gradient is implemented in the code, why?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with tf.GradientTape() as tape:&#xA;    target_actions = target_actor(next_state_batch)&#xA;    y = reward_batch + gamma * target_critic([next_state_batch, target_actions])&#xA;    critic_value = critic_model([state_batch, action_batch])&#xA;    critic_loss = tf.math.reduce_mean(tf.math.square(y - critic_value))&#xA;&#xA;critic_grad = tape.gradient(critic_loss, critic_model.trainable_variables)&#xA;critic_optimizer.apply_gradients(zip(critic_grad, critic_model.trainable_variables))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second question is why in the photo of the algorithm when calculating the actor's policy gradient are 2 gradients multiplied by themselves and in the code only one gradient is calculated for the critic's network and it's not multiplied by the second gradient?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with tf.GradientTape() as tape:&#xA;    actions = actor_model(state_batch)&#xA;    critic_value = critic_model([state_batch, actions])&#xA;    # Used `-value` as we want to maximize the value given&#xA;    # by the critic for our actions&#xA;    actor_loss = -tf.math.reduce_mean(critic_value)&#xA;&#xA;actor_grad = tape.gradient(actor_loss, actor_model.trainable_variables)&#xA;actor_optimizer.apply_gradients(zip(actor_grad, actor_model.trainable_variables))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38547" LastEditorUserId="2444" LastEditDate="2020-07-13T13:55:42.207" LastActivityDate="2020-08-06T13:48:34.493" Title="Understand the DDPG algorithm in Keras" Tags="&lt;keras&gt;&lt;algorithm&gt;&lt;math&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22463" PostTypeId="1" CreationDate="2020-07-11T19:13:54.060" Score="2" ViewCount="43" Body="&lt;p&gt;One of my friends sent me a problem he was working on lately, and I couldn't help but I wonder how could it be solved using Q-learning. The statement is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given the following datasets, the objective is to find a suitable strategy per customer contract to maximize Gain and minimize Cost according to the characteristics of the customer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;train.csv: 5000 independent rows, 33 columns.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Columns description:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Day (1, 2 or 3): on which day the strategy was applied.&lt;/li&gt;&#xA;&lt;li&gt;28 variables (A, B, C, ..., Z, AA, BB): characteristics of the individual;&lt;/li&gt;&#xA;&lt;li&gt;Gain: the gain for this individual for the corresponding strategy;&lt;/li&gt;&#xA;&lt;li&gt;Cost: the cost for this individual for the corresponding strategy;&lt;/li&gt;&#xA;&lt;li&gt;Strategy (0, 1 or 2): the strategy applied on this individual;&lt;/li&gt;&#xA;&lt;li&gt;Success: 1 if the strategy succeeded, 0 otherwise.&lt;/li&gt;&#xA;&lt;li&gt;If Success is 1, then the net gain is Gain - Cost, and if Success is 0, consider a standardized cost of 5.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;test.csv: 2 000 independent rows, 31 columns.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Columns description:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Index: 0 to 1999, unique for each row.&lt;/li&gt;&#xA;&lt;li&gt;Day (4): on which day the strategy will be applied.&lt;/li&gt;&#xA;&lt;li&gt;28 variables (A, B, C, ..., Z, AA, BB): characteristics of the client;&lt;/li&gt;&#xA;&lt;li&gt;Gain: the gain for this individual for the corresponding strategy;&lt;/li&gt;&#xA;&lt;li&gt;Cost: the cost for this individual for the corresponding strategy;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;From what I understood, the &lt;code&gt;train.csv&lt;/code&gt; file is used to build a Q-Learning model, and the test one for generating a strategy and predicting a Success.&lt;/p&gt;&#xA;&lt;p&gt;My main question is:&lt;/p&gt;&#xA;&lt;p&gt;How to formulate this problem as an RL problem? How to define an episode? Since the training data is labeled, this could be clearly a classification problem (predicting the strategy), but I have no idea how to solve it using RL (Q-learning ideally). Any ideas will be helpful.&lt;/p&gt;&#xA;" OwnerUserId="38550" LastEditorUserId="2444" LastEditDate="2020-07-12T10:56:51.227" LastActivityDate="2020-07-12T10:56:51.227" Title="How can I formulate a prediction problem (given labeled data) as an RL problem and solve it with Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;q-learning&gt;&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22464" PostTypeId="2" ParentId="22424" CreationDate="2020-07-11T21:13:50.297" Score="1" Body="&lt;p&gt;I think the choice of technique strongly depends on how fine-grained your forecast-predictions need to be.&lt;/p&gt;&#xA;&lt;p&gt;When it comes to forecasting by Reinforcement Learning (RL), one prominent example is the stock-trading RL agent. The agent must decide which stock to buy or sell, thereby drawing upon predictions concerting the expected future development of some stock.&#xA;Given this approach, you would not necessarily let the RL agent explicitly generate estimates of how stock prices are gonna develop at any point, but instead you would only observe the predicted decision concerning whether to buy or sell etc.&lt;/p&gt;&#xA;&lt;p&gt;But if you think hard enough, I am certain that you could come up with setups of RL agents that would allow you to explicitly generate future estimates of values to be forecast. In this case, the final buy/sell decision would have to depend on the explicit future stock price predictions to enforce accurate predictions.&lt;/p&gt;&#xA;&lt;p&gt;Concerning unsupervised learning, you could cluster data points (training samples) with respect to how some value(s) of interest changed &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; time steps in the future (after having observed the training sample). Then, you could associate clusters with rough forecast-estimates. &lt;em&gt;After all, you would treat the forecast value as a label associated with data points.&lt;/em&gt;&#xA;Afterwards, you could use some kind of nearest neighbor approach to determine which cluster is closest to some novel data sample. Then, you take as a prediction for the new data sample the forecast prediction (i.e. label) that is associated with the closest cluster/prototype etc. But strictly speaking, as soon as you start turning forecast values (which were previously part of some unlabeled time-series dataset) into labels, you turn the training procedure of course into a supervised technique again.&lt;/p&gt;&#xA;&lt;p&gt;How well especially the latter training approach would work, I can't tell since I have never heard anyone using this method. But if training data is too scarce to employ some deep learning method, why not giving it at least a try if accuracy doesn't have to be too precise?&lt;/p&gt;&#xA;&lt;p&gt;After all, it's just a matter of creativity and testing which method works best given your specific machine learning problem at hand.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-07-11T21:13:50.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22465" PostTypeId="1" CreationDate="2020-07-11T21:19:40.777" Score="1" ViewCount="40" Body="&lt;p&gt;I am trying to learn more about text-independent writer identification and was hoping for some advice.&lt;/p&gt;&#xA;&lt;p&gt;I have a folder with 100k images, each of them with a different handwritten sentence. All of the images have sentences of different lengths. They range from about 30 to 80 English characters. The file names start at &lt;code&gt;1.png&lt;/code&gt; and go up to &lt;code&gt;100k.png&lt;/code&gt;. That's it, as far as input data. 95% of the sentences are written by different writers. 5% are written by the same writers. Some writers might have written 2 sentences, while others 300+.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of an identification method that would be able to determine what images were written by the same writer?&lt;/p&gt;&#xA;&lt;p&gt;I know that most methods require each writer to have provided a full page of sample writing for training, but, of course, I do not have that.&lt;/p&gt;&#xA;" OwnerUserId="38553" LastEditorUserId="2444" LastEditDate="2020-07-13T20:01:45.217" LastActivityDate="2020-07-13T20:01:45.217" Title="Can text-independent writer identification be done without multi-sentence training datasets for each writer?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22466" PostTypeId="1" CreationDate="2020-07-12T01:13:26.603" Score="0" ViewCount="39" Body="&lt;p&gt;Where can I place an attention module for different computer vision tasks?&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;https://github.com/kobiso/CBAM-tensorflow-slim/issues/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this Github issue&lt;/a&gt;, the author is not sure where to place the attention module. Are there any rules or best practices to follow?&lt;/p&gt;&#xA;" OwnerUserId="38557" LastEditorUserId="2444" LastEditDate="2020-07-12T13:07:39.003" LastActivityDate="2020-07-12T13:07:39.003" Title="Where can I place an attention module for different computer vision tasks?" Tags="&lt;convolutional-neural-networks&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22467" PostTypeId="1" CreationDate="2020-07-12T02:50:14.303" Score="1" ViewCount="68" Body="&lt;p&gt;I tried the first neural network architecture and the second one, but keeping all other variables constants, I am getting better results with the second architecture. Why are these same neural network architecture giving different results? Or am I making some mistakes?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;First one:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def __init__(self, state_size, action_size, seed, hidden_advantage=[512, 512],&#xA;             hidden_state_value=[512,512]):&#xA;    super(DuelingQNetwork, self).__init__()&#xA;    self.seed = torch.manual_seed(seed)&#xA;    hidden_layers = [state_size] + hidden_advantage&#xA;    self.adv_network = nn.Sequential(nn.Linear(hidden_layers[0], hidden_layers[1]),&#xA;                                     nn.ReLU(),&#xA;                                     nn.Linear(hidden_layers[1], hidden_layers[2]),&#xA;                                     nn.ReLU(),&#xA;                                     nn.Linear(hidden_layers[2], action_size))&#xA;&#xA;    hidden_layers = [state_size] + hidden_state_value&#xA;    self.val_network = nn.Sequential(nn.Linear(hidden_layers[0], hidden_layers[1]),&#xA;                                     nn.ReLU(),&#xA;                                     nn.Linear(hidden_layers[1], hidden_layers[2]),&#xA;                                     nn.ReLU(),&#xA;                                     nn.Linear(hidden_layers[2], 1))                                                           &#xA;def forward(self, state):&#xA;    &amp;quot;&amp;quot;&amp;quot;Build a network that maps state -&amp;gt; action values.&amp;quot;&amp;quot;&amp;quot;&#xA;    # Perform a feed-forward pass through the networks&#xA;    advantage = self.adv_network(state)&#xA;    value = self.val_network(state)&#xA;    return advantage.sub_(advantage.mean()).add_(value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second one:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def __init__(self, state_size, action_size, seed, hidden_advantage=[512, 512],&#xA;             hidden_state_value=[512,512]):&#xA;    super(DuelingQNetwork, self).__init__()&#xA;    self.seed = torch.manual_seed(seed)&#xA;&#xA;    hidden_layers = [state_size] + hidden_advantage&#xA;    advantage_layers = OrderedDict()&#xA;    for idx, (hl_in, hl_out) in enumerate(zip(hidden_layers[:-1],hidden_layers[1:])):&#xA;        advantage_layers['adv_fc_'+str(idx)] = nn.Linear(hl_in, hl_out)&#xA;        advantage_layers['adv_activation_'+str(idx)] = nn.ReLU()&#xA;&#xA;    advantage_layers['adv_output'] = nn.Linear(hidden_layers[-1], action_size)&#xA;&#xA;    self.network_advantage = nn.Sequential(advantage_layers)&#xA;&#xA;    value_layers = OrderedDict()&#xA;    hidden_layers = [state_size] + hidden_state_value&#xA;&#xA;    # Iterate over the parameters to create the value network&#xA;    for idx, (hl_in, hl_out) in enumerate(zip(hidden_layers[:-1],hidden_layers[1:])):&#xA;        # Add a linear layer&#xA;        value_layers['val_fc_'+str(idx)] = nn.Linear(hl_in, hl_out)&#xA;        # Add an activation function&#xA;        value_layers['val_activation_'+str(idx)] = nn.ReLU()&#xA;&#xA;    # Create the output layer for the value network&#xA;    value_layers['val_output'] = nn.Linear(hidden_layers[-1], 1)&#xA;&#xA;    # Create the value network&#xA;    self.network_value = nn.Sequential(value_layers)&#xA;&#xA;def forward(self, state):&#xA;    &amp;quot;&amp;quot;&amp;quot;Build a network that maps state -&amp;gt; action values.&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    # Perform a feed-forward pass through the networks&#xA;    advantage = self.network_advantage(state)&#xA;    value = self.network_value(state)&#xA;    return advantage.sub_(advantage.mean()).add_(value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38540" LastEditorUserId="8448" LastEditDate="2020-07-13T18:02:18.540" LastActivityDate="2020-07-13T18:02:18.540" Title="Why are these same neural network architecture giving different results?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22468" PostTypeId="1" CreationDate="2020-07-12T03:25:59.810" Score="1" ViewCount="34" Body="&lt;h2&gt;Forward propagation in Deep Neural Networks&lt;/h2&gt;&#xA;&lt;p&gt;In the &amp;quot;Forward Propagation in a Deep Network&amp;quot; video on Coursera, Andrew NG mentions that there's no way to avoid a for loop to loop through the different layers of the network during forward propagation.&lt;/p&gt;&#xA;&lt;p&gt;See image showing a deep network with 4 layers, and the requirement of a forloop to compute activations for each layer during forward propagation: &lt;a href=&quot;https://nimb.ws/CkRVLT&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nimb.ws/CkRVLT&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This makes intuitive sense since each layer's activation depends on the previous layer's output.&lt;/p&gt;&#xA;&lt;h2&gt;Warning: start of speculation&lt;/h2&gt;&#xA;&lt;p&gt;My rudimentary understanding of quantum computing is that it somehow &amp;quot;magically&amp;quot; can bypass computing intermediate states -&amp;gt; this is why supposedly quantum computers can break cryptography... or something like that.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if a quantum computer could perform vectorized forward propagation on an L layer deep neural network.&lt;/p&gt;&#xA;" OwnerUserId="38558" LastEditorUserId="2444" LastEditDate="2020-07-12T10:49:00.403" LastActivityDate="2020-07-12T10:49:00.403" Title="Could a quantum computer perform vectorized forward propagation in deep networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;quantum-computing&gt;&lt;forward-pass&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22469" PostTypeId="1" AcceptedAnswerId="22476" CreationDate="2020-07-12T10:00:24.727" Score="0" ViewCount="69" Body="&lt;p&gt;Just came across &lt;a href=&quot;https://www.gwern.net/newsletter/2020/05#gpt-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article on GPT-3&lt;/a&gt;, and that lead me to the question:&lt;/p&gt;&#xA;&lt;p&gt;In order to make a certain kind of neural network architecture smarter all one needs to do is to make it bigger?&lt;/p&gt;&#xA;&lt;p&gt;Also, if that is true, how does the importance of computer power relates with the importance of fine-tuning/algorithmic improvement?&lt;/p&gt;&#xA;" OwnerUserId="10135" LastActivityDate="2020-07-13T00:23:05.657" Title="Is the size of a neural network directly linked with an increase in its inteligence?" Tags="&lt;neural-networks&gt;&lt;algorithm&gt;&lt;architecture&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22471" PostTypeId="2" ParentId="20411" CreationDate="2020-07-12T11:11:11.420" Score="1" Body="&lt;p&gt;There's also the journal &lt;strong&gt;Advances in Cognitive Systems&lt;/strong&gt;. According to &lt;a href=&quot;http://www.cogsys.org/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;their website&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cogsys.org/journal&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advances in Cognitive Systems (ISSN 2324-8416)&lt;/a&gt; publishes research articles, review papers, and essays on the computational study of &lt;strong&gt;human-level intelligence&lt;/strong&gt;, &lt;strong&gt;integrated intelligent systems&lt;/strong&gt;, &lt;strong&gt;cognitive architectures&lt;/strong&gt;, and related topics. Research on cognitive systems is distinguished by a focus on high-level cognition, reliance on rich, structured representations, a systems-level perspective, use of heuristics to handle complexity, and incorporation of insights about human thinking. Advances in Cognitive Systems reviews submissions within approximately three months and publishes accepted papers on the journal Web site immediately upon receipt of final versions. Articles are distributed freely over the internet by the Cognitive Systems Foundation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-12T11:11:11.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22472" PostTypeId="2" ParentId="20411" CreationDate="2020-07-12T13:40:23.743" Score="1" Body="&lt;p&gt;The Institute of Electrical and Electronic Engineers (IEEE) recently announced a new &lt;strong&gt;&amp;quot;IEEE Transaction on Artificial Intelligence&lt;/strong&gt;&amp;quot;.  Although the topics listed do not specifically mention AGI they do not limit it.  I think this will be an interesting journal to keep an eye on as there could be some interesting AGI papers.  Below is from their &lt;a href=&quot;https://cis.ieee.org/publications/ieee-transactions-on-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;web page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Welcome to the page for the IEEE Transactions on Artificial Intelligence (IEEE TAI). IEEE has established the new journal our community has been waiting for to publish our work on Artificial Intelligence! The submission site for manuscripts opens on April 1, 2020, and the inaugural issue is set to publish August 2020. Don’t miss out on this opportunity and join the community of authors who started already preparing their papers for the IEEE TAI! In one month, you will be able to submit your paper. It is time to prepare your best piece of work for IEEE TAI to increase your visibility in the international AI community and to be among the first authors to publish in IEEE TAI.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is also the &lt;a href=&quot;https://cis.ieee.org/publications/t-cognitive-and-developmental-systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;IEEE Transactions on Cognitive and Developmental Systems (TCDS)&lt;/strong&gt;&lt;/a&gt; which is not strictly AGI but publishes topics that pertain to AGI.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS (TCDS) focuses on advances in the study of development and cognition in natural (humans, animals) and artificial (robots, agents) systems. It welcomes contributions from multiple related disciplines including cognitive systems, cognitive robotics, developmental and epigenetic robotics, autonomous and evolutionary robotics, social structures, multi-agent and artificial life systems, computational neuroscience, and developmental psychology. Articles on theoretical, computational, application-oriented, and experimental studies as well as reviews in these areas are considered.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TCDS is co-sponsored by the Computational Intelligence Society, the Robotics and Automation Society, and the Consumer Electronics Society. TCDS is technically co-sponsored by the Computer Society.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="2444" LastEditDate="2020-07-12T14:44:31.863" LastActivityDate="2020-07-12T14:44:31.863" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22473" PostTypeId="1" AcceptedAnswerId="22474" CreationDate="2020-07-12T14:15:52.503" Score="3" ViewCount="115" Body="&lt;p&gt;From the book:&lt;/p&gt;&#xA;&lt;p&gt;Sutton, Richard S.,Barto, Andrew G.. Reinforcement Learning (Adaptive Computation and Machine Learning series) (p. 100). The MIT Press. Kindle Edition. &amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;following is stated:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;On-policy methods attempt to evaluate or improve the policy that is used to make decisions, whereas off-policy methods evaluate or improve a policy different from that used to generate the data.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Looking at off policy:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d9AI0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d9AI0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and on-policy:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f6liK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f6liK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What is meant by &amp;quot;generate the data&amp;quot;? I'm confused as to what 'data' means in this context.&lt;/p&gt;&#xA;&lt;p&gt;Does &amp;quot;generate the data&amp;quot; translate to the actions generated by the policy ? or Does &amp;quot;generate the data&amp;quot; translate to the Q data state action mappings?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="12964" LastEditDate="2020-07-12T17:57:47.807" LastActivityDate="2020-07-12T17:57:47.807" Title="What is meant by &quot;generate the data&quot; in describing the difference between on-policy and off-policy?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22474" PostTypeId="2" ParentId="22473" CreationDate="2020-07-12T17:20:47.823" Score="3" Body="&lt;p&gt;In the book, the phrase &amp;quot;generate the data&amp;quot; refers to the data from observations about states, actions, next states and rewards, that then get used to make value estimate updates.&lt;/p&gt;&#xA;&lt;p&gt;In both the SARSA and Q learning pseudocode from the book, there is a &lt;em&gt;behaviour&lt;/em&gt; policy that selects the next action to take. Other than the initial start state, this policy drives the observations that the learning process must handle - every action is directly selected by it, and due to how MDPs work, every next state and reward are indirectly influenced by it. It is this &lt;em&gt;behaviour&lt;/em&gt; policy therefore that must &amp;quot;generate the data&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;You can see from the pseudocode that both algorithms describe this behaviour policy as &amp;quot;policy derived from Q (e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy)&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The difference in the two algorithms is in which &lt;em&gt;target&lt;/em&gt; polcy is being used:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In SARSA, the update is based on the Bellman equation for the same policy that generated the most recent set of data for S, A, R, S' and A' - all of these values will be directly or indirectly caused by the behaviour policy. So the whole equation uses the same data that was generated. Another way to put that is behaviour policy and target policy are the same.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In Q learning, the &lt;span class=&quot;math-container&quot;&gt;$\text{max}_a Q(S',a)$&lt;/span&gt; from the Bellman optimality equation removes the need to use A' in the update. Effectively it is a local search for actions to find one that is potentially better than A', and runs the update as if that one was the one that was taken. This revision of A' is what makes the off-learning evaluate a different target policy to the behaviour policy that generated the rest of the data used in the update.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-07-12T17:20:47.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22475" PostTypeId="1" CreationDate="2020-07-12T19:40:22.510" Score="1" ViewCount="57" Body="&lt;p&gt;Where can I find a machine learning library that implements &lt;code&gt;loss functions&lt;/code&gt; measuring the &lt;a href=&quot;http://www.scholarpedia.org/article/Algorithmic_information_theory#:%7E:text=Algorithmic%20information%20theory%20(AIT)%20is,computation%2C%20information%2C%20and%20randomness.&amp;amp;text=presumably%20has%20no%20simple%20description%20other%20than%20writing%20down%20the%20string%20itself.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithmic Information Theoretic&lt;/a&gt;-friendly quantity &amp;quot;&lt;em&gt;bits of information&lt;/em&gt;&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;To illustrate the difference between &lt;em&gt;entropy&lt;/em&gt;, in the &lt;em&gt;Shannon information&lt;/em&gt; sense of &amp;quot;bits&amp;quot; and the &lt;em&gt;algorithmic information&lt;/em&gt; sense of &amp;quot;bits&amp;quot;, consider the way these two measures treat a 1 million character string representing &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Shannon entropy&lt;/em&gt; &amp;quot;bits&amp;quot; (&lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt; for the '.'):  &lt;span class=&quot;math-container&quot;&gt;$\lceil 1e6*\log_2(10) \rceil+6$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Algorithmic&lt;/em&gt; &amp;quot;bits&amp;quot;:  The length, in bits, of the shortest &lt;a href=&quot;https://www.angio.net/pi/pi-programs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;program that outputs 1e6 digits of &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; &lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;All &lt;em&gt;statistical&lt;/em&gt; measures of information, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;KL divergence&lt;/a&gt;, are based on &lt;em&gt;Shannon information&lt;/em&gt;.  By contrast, &lt;em&gt;algorithmic information&lt;/em&gt; permits representations that are fully &lt;em&gt;dynamical&lt;/em&gt; as in &lt;a href=&quot;https://en.wikipedia.org/wiki/Unrestricted_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chomsky type 0&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_completeness&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing Complete&lt;/a&gt;, etc. languages.  Since the world in which we live is &lt;em&gt;dynamical&lt;/em&gt;, algorithmic models are at least &lt;em&gt;plausibly&lt;/em&gt; more valid in many situations than are &lt;em&gt;statistical&lt;/em&gt; models.  (I recognize that recursive neural nets can be dynamical and that they can be trained with statistical loss functions.)&lt;/p&gt;&#xA;&lt;p&gt;For a more authoritative and formal description of these distinctions see the Hutter Prize FAQ questions &lt;a href=&quot;http://www.hutter1.net/prize/hfaq.htm#xvalid&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why aren't cross-validation or train/test-set used for evaluation?&lt;/a&gt; and &lt;a href=&quot;http://www.hutter1.net/prize/hfaq.htm#reg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why is Compressor Length superior to other Regularizations?&lt;/a&gt; For a paper-length exposition on the same see &amp;quot;&lt;a href=&quot;https://www.mdpi.com/1099-4300/22/6/612&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Review of Methods for Estimating Algorithmic Complexity: Options, Challenges, and New Directions&lt;/a&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;From what I can see, machine learning makes it difficult to relate &lt;code&gt;loss&lt;/code&gt; to &lt;em&gt;algorithmic&lt;/em&gt; information.  Such an AIT-friendly &lt;code&gt;loss function&lt;/code&gt; must, by definition, measure the number of bits required to reconstruct, &lt;em&gt;without loss&lt;/em&gt;, the original training dataset.&lt;/p&gt;&#xA;&lt;p&gt;Let me explain with examples of what I mean by AIT-friendly &lt;code&gt;loss functions&lt;/code&gt;, starting with the baby-step of &lt;code&gt;classification loss&lt;/code&gt; (usually measured as &lt;a href=&quot;https://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-entropy&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;Let's say your training set consists of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; patterns belonging to &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; classes. You can then construct a partial AIT loss function providing the length of the corrections to the model's classifications with a &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;-length vector, each element containing a &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if the model was correct for that pattern, or the class if not.  These elements would each have a bit-length of &lt;span class=&quot;math-container&quot;&gt;$\lceil \log_2(C+1) \rceil$&lt;/span&gt;, and be prefixed by a &lt;a href=&quot;https://en.wikipedia.org/wiki/Variable-length_quantity&quot; rel=&quot;nofollow noreferrer&quot;&gt;variable length integer&lt;/a&gt; storing &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;.  The more &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; elements, the more compressible this correction vector until, in the limit, a single &lt;code&gt;run-length code&lt;/code&gt; for &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;'s is stored as the correction, prefixed by &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and the length of the binary for the &lt;code&gt;RLE&lt;/code&gt; algorithm itself. The bit-length of these, taken together, would comprise this partial &lt;code&gt;loss function&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is a reasonable first cut at an AIT-friendly loss function for &lt;code&gt;classification error&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So now let's go one step further to outputs that are numeric, the typical approach is a summation of a function of individual error measures, such as squaring or taking their absolute value or whatever -- perhaps taking their mean.  None of these are in units of bits of information.  To provide the correction on the outputs to reproduce the actual training values requires, again, a vector of corrections.  This time it would be deltas, the precision of which must be adequate to the original data being &lt;em&gt;losslessly&lt;/em&gt; represented, hence requiring some sort of adaptive variable length quantity representation(s).  These deltas would likely have a non-uniform distribution so they can be arithmetically encoded.  That seems like a reasonable approach to another AIT-friendly loss function.&lt;/p&gt;&#xA;&lt;p&gt;But now we get to the &amp;quot;model parameters&amp;quot; and find ourselves in the &lt;em&gt;apparently&lt;/em&gt; well-defined but ill-founded notions like &amp;quot;&lt;a href=&quot;https://towardsdatascience.com/l1-and-l2-regularization-methods-ce25e7fc831c&quot; rel=&quot;nofollow noreferrer&quot;&gt;L2 regularization&lt;/a&gt;&amp;quot;, which are defined in terms of ill-defined &amp;quot;parameters&amp;quot;, e.g. &amp;quot;parameter counts&amp;quot; aren't given in bits.&lt;/p&gt;&#xA;&lt;p&gt;I'll grant that &lt;code&gt;L2 regularization&lt;/code&gt; sounds like it is heading in the right direction by squaring the weights and summing them up, but when one looks at what is actually being done, it is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;applying additional functions to the sum such as mean&lt;/li&gt;&#xA;&lt;li&gt;asking for a scaling factor to apply&lt;/li&gt;&#xA;&lt;li&gt;applying the regularization on a per-layer basis rather than the entire model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm sure I missed some of the many ways &lt;code&gt;L2 regularization&lt;/code&gt; fails to be AIT-friendly.&lt;/p&gt;&#xA;&lt;p&gt;Finally, there is the model's pseudo-invariance, measured, not simply in terms of its &lt;code&gt;hyperparameters&lt;/code&gt; but in terms of the length of the (compressed archive of the) actual executable binary running on the hardware.  I say 'pseudo' because there is nothing that says one cannot vary, say, the number of neurons in a neural network during learning -- nor even change to another learning paradigm than neural networks during learning (in the most general case).&lt;/p&gt;&#xA;&lt;p&gt;So that's pretty much the complete loss function down to the Universal Turing Machine iron, but I'd be happy to see just a reference to an existing TensorFlow or another library that tries to do even a partial &lt;code&gt;loss&lt;/code&gt; function for AIT-theoretic learning.&lt;/p&gt;&#xA;" OwnerUserId="26053" LastEditorUserId="26053" LastEditDate="2020-07-12T23:11:25.387" LastActivityDate="2020-07-12T23:11:25.387" Title="Loss Function In Units Of Bits?" Tags="&lt;loss-functions&gt;&lt;information-theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22476" PostTypeId="2" ParentId="22469" CreationDate="2020-07-13T00:23:05.657" Score="2" Body="&lt;p&gt;First of all, there is no real 'intelligence' innate to artificial Neural Networks (NNs).&#xA;All they do is trying to approximate a mathematical function with a certain degree of generalization (hopefully without learning a given dataset by heart, i.e. hopefully without &lt;em&gt;overfitting&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The more nodes (or &lt;em&gt;neurons&lt;/em&gt;) you include into the network, the more complex a function can be that a network can learn to approximate. It's similar to high-school math: The higher the degree of some polynomial, the better the polynomial can be adjusted to fit some observation to be modeled; with the only difference being that NNs commonly include non-linearities and are trained via some kind of stochastic gradient descent.&lt;/p&gt;&#xA;&lt;p&gt;So, yes. The more nodes a model possesses, the higher the so-called &lt;em&gt;model capacity&lt;/em&gt;, i.e. the higher the degree of freedom a NN-model has to fit some function. After all, NN are said to be universal function approximators - given they have enough internal nodes in their hidden layer(s) to fit some given function.&lt;/p&gt;&#xA;&lt;p&gt;In practice, however, you don't want to blow up a model architecture unnecessarily, since this commonly results in overfitting if it doesn't cause some instabilities of the training procedure instead.&lt;/p&gt;&#xA;&lt;p&gt;Generally, the larger the model to be trained, the higher the computational cost to train the network.&lt;/p&gt;&#xA;&lt;p&gt;A common suggestion is to reduce the number of nodes in a network at the expense of increasing a network's depth, i.e. the number of hidden layers. Often, that can help reduce the demand for excessively many nodes.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-07-13T00:23:05.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22478" PostTypeId="1" AcceptedAnswerId="22479" CreationDate="2020-07-13T03:07:55.687" Score="1" ViewCount="49" Body="&lt;p&gt;I am new to working with neural networks. However, I have built some linear regression models in the past. My question is, is it worth looking for features with a correlation to my target variable as I would normally do in a linear regression or is it better to feed the neural network with all the data I have?&lt;/p&gt;&#xA;&lt;p&gt;Assuming that the data I have is all related to my target variable of course. I am working with this dataset and building a neural network regressor for it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0101EN/labs/data/concrete_data.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0101EN/labs/data/concrete_data.csv&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is a snippet of the data. The target variable is the concrete strength rate given a certain combination of materials for that concrete sample.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0xaPT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0xaPT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I greatly appreciate any tips and explanations. I excuse me if this is too noob of a question but unfortunately I did not find any info about it on google. Thanks again!&lt;/p&gt;&#xA;" OwnerUserId="33342" LastActivityDate="2020-07-13T09:56:56.137" Title="Do correlations matter when building neural networks?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22479" PostTypeId="2" ParentId="22478" CreationDate="2020-07-13T09:25:34.530" Score="0" Body="&lt;p&gt;If there is some correlation between features, that is what the network will ideally find out on its own and learn to utilize. So, in general, don't take correlated samples or features out of the training loop only because they look correlated. After all, they could convey a lot of valuable information.&lt;/p&gt;&#xA;&lt;p&gt;When it comes to correlation between data samples during training, this correlation is commonly broken up by training a network on randomly selected mini-batches of training data samples. So, you randomly sample e.g. 16 or 32 (or so) training examples based on which you apply a single update of the weights using some Stochastic Gradient Descent variant. Since the members of a mini-batch are sampled at random, chances for finding highly correlated training samples in some mini-batch shall be sufficiently minimized in order not to negatively affect the training outcome.&lt;/p&gt;&#xA;&lt;p&gt;Having said that, if you are concerned about overfitting of your model or weights that would overly weight just a small subset of all available input features, you could try applying regularization techniques like L1 (encouraging sparse representations) or L2 (encouraging low weights in general) regularization or dropout.&#xA;In your particular case, since the main concern is an excessive contribution of only a small set of input features, L2 shall yield better results (avoiding excessively large weights that would be required to excessively much weight just a small number of features).&lt;/p&gt;&#xA;&lt;p&gt;Besides that: Commonly, you split your training dataset into 3 parts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Data used for fitting the model (actual training data)&lt;/li&gt;&#xA;&lt;li&gt;Data used for assessing the training progress &amp;amp; possibly for determining when to apply early stopping (validation data)&lt;/li&gt;&#xA;&lt;li&gt;Test data used to assess the performance of the system after all training &amp;amp; intermediate testing is done&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The final evaluation on the test dataset shall reveal then the generalization ability of your trained model to novel data.&lt;/p&gt;&#xA;&lt;p&gt;So, with regularization in place during training and relatively low error rates on the validation and test datasets, you are pretty much save even without checking for correlated data beforehand. Only when you really struggle decreasing the validation loss, it might be worth to further inspect what exactly is going wrong in terms of correlations and such.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-07-13T09:56:56.137" LastActivityDate="2020-07-13T09:56:56.137" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22480" PostTypeId="1" CreationDate="2020-07-13T09:28:31.823" Score="0" ViewCount="30" Body="&lt;p&gt;I have a Deep Convolutional Generative Adversarial Network (DCGAN) that trains on the CIFAR dataset. When I finish the training (100k epochs), how can I make my network generate 1000 fake images? I can't find anything online. I need to do this to calculate the fid score.&lt;/p&gt;&#xA;" OwnerUserId="38576" LastEditorUserId="2444" LastEditDate="2020-07-13T11:11:31.607" LastActivityDate="2020-07-13T11:11:31.607" Title="How do I get my DCGAN to generate a number of fake images?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;pytorch&gt;&lt;dc-gan&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22481" PostTypeId="1" CreationDate="2020-07-13T09:42:29.813" Score="1" ViewCount="247" Body="&lt;p&gt;From the article &lt;a href=&quot;https://towardsdatascience.com/dangerous-feedback-loops-in-ml-e9394f2e8f43&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dangerous Feedback Loops in ML&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Let’s say our model has leads from Facebook, Google, and Bing. If our first model decides that the probability of conversion is 3%, 5%, and 1% from these given sources, and we have finite amount of callbacks we can make, we will only callback the 5% probability. Now fast forward two months. The second model finds these probabilities are now: 0.5%, 8.5%, and 0%. What happened?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Because we started only calling Google leads back, we increased our chances of converting these leads, and likewise, because we stopped calling Facebook and Bing leads, these leads never converted because we never called them. This is an example of a real world feedback loop&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How can we solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="38392" LastEditorUserId="2444" LastEditDate="2020-07-13T11:23:36.067" LastActivityDate="2021-04-09T13:04:25.490" Title="How to solve the &quot;dangerous feedback loops&quot; in machine learning?" Tags="&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22482" PostTypeId="2" ParentId="22481" CreationDate="2020-07-13T11:00:34.300" Score="1" Body="&lt;p&gt;I read the article you linked, and what you are missing are that the given conversion probabilities are assessed pre-callback - i.e. they include an assessment of whether you will even call them back or not. So of course the probabilities change if you change your behaviour. The writer of the article has created a bit of a straw man argument by defining a model and decision process that don't go well together. They should have used a model that predicted conversion rates after callback, then it could be used as they wanted.&lt;/p&gt;&#xA;&lt;p&gt;So the sales example is pretty easy to solve. Make the model accept the lead source as a feature, and predict the conversion rate &lt;em&gt;after&lt;/em&gt; callback. That will give you the &amp;quot;action value&amp;quot; of choosing to callback, which is much more useful value to have if you are deciding whether or not to callback.&lt;/p&gt;&#xA;&lt;p&gt;To cover the possibility that probabilities change over time, you have to be willing to test that hypothesis by calling back at least some leads that the model predicts have a low probability of success, in order to update the model. This is related to &lt;a href=&quot;https://towardsdatascience.com/reinforcement-learning-demystified-exploration-vs-exploitation-in-multi-armed-bandit-setting-be950d2ee9f6&quot; rel=&quot;nofollow noreferrer&quot;&gt;exploration vs exploitation trade off in bandit problems&lt;/a&gt; or in reinforcement learning. The callback optimisation problem looks a lot like a bandit problem in fact.&lt;/p&gt;&#xA;&lt;p&gt;Many problems of choosing how to act optimally, based on an updating statistical model of results, can be re-framed successfully as bandit problems or reinforcement learning problems. This is one way to try and address the feedback loops issue, because these include theory around decision making. It is not a magic fix for all these problems, but contextual bandits are very strong in advertising strategies that are very similar to the example you give.&lt;/p&gt;&#xA;&lt;p&gt;To be able to address feedback loops like this well does require that source data is unbiased, so that adding conditional actions to your model helps obtain better ground truth. So the other examples that the linked article gives - recidivism and recruitment - are much harder to address, because biased data is inserted as ground truth even if you add the conditions (e.g. for exploring and tracking results in a statistically neutral way). For some, adding these conditions may be enough to help, it will make the model accurate enough that it starts to address bias. For others, there is no really good solution other than to be aware of weakness of the model.&lt;/p&gt;&#xA;&lt;p&gt;You can attempt to de-bias the model by using some abstract ideal. A common technique used in recruiting for instance is to &lt;em&gt;remove&lt;/em&gt; features that might be causing bias: Literally remove race, sex and related determiners from the CV on the grounds that these should not be part of any fair model, regardless of whether or not a ML model could make of use them to predict results. This might accidentally throw out data that would improve accuracy too, but can be used to achieve a decision process that observers agree should be free of contentious soruces of bias.&lt;/p&gt;&#xA;&lt;p&gt;Which features to remove or adjust is an open and subjective question. Unfortunately ML models can often determine protected characteristics from other data, and with &amp;quot;black box&amp;quot; ones such as neural networks it is not clear whether or not they are doing so. It's still an open question, and the main defence against biased models is basically awareness that it is a problem and that feeding all your data into a computer to get a statistical model does not in any way &amp;quot;purify&amp;quot; it or achieve some higher level of objective truth.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-13T11:07:41.673" LastActivityDate="2020-07-13T11:07:41.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22484" PostTypeId="1" CreationDate="2020-07-13T15:24:21.633" Score="1" ViewCount="42" Body="&lt;p&gt;I've seen a few mentions in papers that neural network parameters can be found using REINFORCE algorithm. It was mentioned in the context of nondifferentiable operations involving e.g. step function which appears in &lt;a href=&quot;https://arxiv.org/abs/1502.03044&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;hard attention&amp;quot;&lt;/a&gt; or weight pruning. Unfortunately, I haven't seen how to really do this. In Markov Decision Process we have, states (S), actions (A) and rewards (R) so what is what in case of neural net? I don't see how can we find parameters or neural net if the gradient is not well defined.&#xA;Any code sample or explanation?&lt;/p&gt;&#xA;" OwnerUserId="38583" LastActivityDate="2020-07-14T14:20:14.897" Title="How to optimize neural network parameters with REINFORCE" Tags="&lt;reinforcement-learning&gt;&lt;optimization&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22486" PostTypeId="1" CreationDate="2020-07-13T18:52:09.740" Score="1" ViewCount="422" Body="&lt;p&gt;Should the training data be the same in each epoch?&lt;/p&gt;&#xA;&lt;p&gt;If the training data is generated on the fly, for example, is there a difference between training 1000 samples with 1 epoch or training 1000 epochs with 1 sample each?&lt;/p&gt;&#xA;&lt;p&gt;To elaborate further, samples do not need to be saved or stay in memory if they are never used again. However, if training performs best by training over the same samples repeatedly, then data would have to be stored to be reused in each epoch.&lt;/p&gt;&#xA;&lt;p&gt;More samples is generally considered advantageous. Is there a disadvantage to never seeing the same sample twice in training?&lt;/p&gt;&#xA;" OwnerUserId="10957" LastEditorUserId="10957" LastEditDate="2020-07-14T16:26:20.303" LastActivityDate="2020-07-14T16:26:20.303" Title="Should the training data be the same in each epoch?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22488" PostTypeId="2" ParentId="22486" CreationDate="2020-07-13T21:21:58.563" Score="4" Body="&lt;p&gt;Let's quickly get out our copies of &lt;a href=&quot;https://www.deeplearningbook.org/contents/optimization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Deep Learning&lt;/strong&gt;&lt;/a&gt; by &lt;em&gt;Goodfellow et al.&lt;/em&gt; (2016). More specifically, I'm referring to &lt;a href=&quot;https://www.deeplearningbook.org/contents/optimization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;page 276&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;On this page, the authors argue for a relatively small minibatch size, since there are &lt;em&gt;less than linear returns&lt;/em&gt; for estimating the gradient when increasing the minibatch size. &lt;em&gt;Returns&lt;/em&gt; here refer to the reduction of the standard error of the mean (gradient per weight) computed over a minibatch.&lt;/p&gt;&#xA;&lt;p&gt;So, yes. In &lt;em&gt;theory&lt;/em&gt;, having unlimited resources, you will get the best performance when averaging the loss over all samples in your dataset. In practice, however, the larger the size of minibatches, the slower the training procedure, and consequently the less the total number of weight updates that can be afforded. Reversely, in &lt;em&gt;practice&lt;/em&gt;, the cheaper the weight updates, the quicker the training procedure can converge to a (subjectively) satisfactory result.&lt;/p&gt;&#xA;&lt;p&gt;Eventually, also Goodfellow et al. state that &lt;em&gt;rapidly&lt;/em&gt; computing gradients leads to &lt;em&gt;much&lt;/em&gt; faster convergence (in terms of total computations) for most optimization algorithms than when training them more slowly on exact gradients.&lt;/p&gt;&#xA;&lt;p&gt;So, to summarize: If the main concern is to get to a specific level of accuracy at all, go for rather low minibatch sizes, whereas you could go up to a few hundreds (as the Goodfellow et al. state as a reasonable upper bound on page 148) if you are interested in more accurate gradients for your weight updates.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="2444" LastEditDate="2020-07-14T00:21:23.303" LastActivityDate="2020-07-14T00:21:23.303" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22489" PostTypeId="2" ParentId="22486" CreationDate="2020-07-13T23:27:34.000" Score="0" Body="&lt;p&gt;This would be more suitable as a comment but I don't have enough points; but here's my opinion.&lt;/p&gt;&#xA;&lt;p&gt;Optimisation algorithms like gradient descent are iterative algorithms. So it is rarely possible that they arrive at the minima in 1 epoch. A single epoch means that all data points have been visited once or a certain number of data samples have been taken from a distribution. However more passes might be necessary.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;generated on the fly&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am assuming that the data is being generated as a part of a fixed distribution. Hence multiple epochs of multiple samples is still the ideal scenario.&lt;/p&gt;&#xA;&lt;p&gt;1000 samples 1 epoch: Not enough training.&lt;br /&gt;&#xA;1 sample 1000 epoch: Overfitting or possibly not enough training.&lt;/p&gt;&#xA;" OwnerUserId="31827" LastEditorUserId="2444" LastEditDate="2020-07-14T00:24:06.287" LastActivityDate="2020-07-14T00:24:06.287" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22490" PostTypeId="2" ParentId="13317" CreationDate="2020-07-14T01:22:11.070" Score="0" Body="&lt;p&gt;Just wanted to add that the new text &lt;em&gt;Deep Learning Architectures&#xA;A Mathematical Approach&lt;/em&gt; mentions this result, but I'm not sure if it gives a proof. It does mention an improved result by Hanin (&lt;a href=&quot;http://arxiv.org/abs/1708.02691&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://arxiv.org/abs/1708.02691&lt;/a&gt;) for which I think it does give at least a partial proof. The original paper by Hanin seems to omit some proofs as well, but the published version (&lt;a href=&quot;https://www.mdpi.com/2227-7390/7/10/992/htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mdpi.com/2227-7390/7/10/992/htm&lt;/a&gt;) may be more complete.&lt;/p&gt;&#xA;" OwnerUserId="38596" LastActivityDate="2020-07-14T01:22:11.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22491" PostTypeId="1" CreationDate="2020-07-14T07:21:15.833" Score="2" ViewCount="44" Body="&lt;p&gt;In this blog post: &lt;a href=&quot;http://www.argmin.net/2016/04/18/bottoming-out/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.argmin.net/2016/04/18/bottoming-out/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Prof Recht shows two plots:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IKNxh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IKNxh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T0njq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T0njq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;He says one of the reasons the plot below has a lower train-test gap is because that model was trained with a lower learning rate (and he also manually drops the learning rate at 120 epoch).&lt;/p&gt;&#xA;&lt;p&gt;Why would a lower learning rate reduce overfitting?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2020-07-14T11:59:33.120" LastActivityDate="2020-07-14T11:59:33.120" Title="Why does learning rate reduce train-test generalization gap?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;generalization&gt;&lt;learning-rate&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22492" PostTypeId="1" AcceptedAnswerId="22494" CreationDate="2020-07-14T08:04:50.097" Score="1" ViewCount="58" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;...Designing such a likelihood function is typically challenging; however, we observe that features like spectrogram are effective when &lt;strong&gt;latent variables have limited degrees of freedom.&lt;/strong&gt; This motivates us to infer latent variables via methods like Gibbs sampling, where we focus on approximating the conditional probability of a single variable given the others.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Above is an excerpt from &lt;a href=&quot;https://papers.nips.cc/paper/6727-shape-and-material-from-sound.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper I've been reading&lt;/a&gt;, and I don't understand what the author means by degrees of freedom of latent variables. Could someone please explain with an example, or add more details?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://papers.nips.cc/paper/6727-shape-and-material-from-sound.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shape and Material from Sound&lt;/a&gt; &lt;em&gt;(31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-07-14T12:05:43.997" LastActivityDate="2020-07-15T14:14:05.560" Title="What is meant by degrees of freedom of latent variables?" Tags="&lt;terminology&gt;&lt;papers&gt;&lt;generative-model&gt;&lt;latent-variable&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22493" PostTypeId="2" ParentId="22319" CreationDate="2020-07-14T09:20:46.813" Score="1" Body="&lt;p&gt;Maybe &lt;a href=&quot;https://github.com/tzutalin/labelImg&quot; rel=&quot;nofollow noreferrer&quot;&gt;LabelImg&lt;/a&gt; is what you are looking for?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;LabelImg is a graphical image annotation tool and label object&#xA;bounding boxes in images.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If not, maybe you can find other options for your problem on &lt;a href=&quot;https://lionbridge.ai/articles/image-annotation-tools-for-computer-vision/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; summary of computer vision tools.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-07-14T09:20:46.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22494" PostTypeId="2" ParentId="22492" CreationDate="2020-07-14T10:12:46.807" Score="1" Body="&lt;p&gt;A good example is the degree of freedom in &lt;a href=&quot;https://en.wikipedia.org/wiki/Student%27s_t-distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Student's distribution&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;‌ The degrees of freedom refers to the number of independent observations in a set of data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For &lt;a href=&quot;https://stattrek.com/probability-distributions/t-distribution.aspx#:%7E:text=The%20degrees%20of%20freedom%20refers,in%20a%20set%20of%20data.&amp;amp;text=Hence%2C%20the%20distribution%20of%20the,or%207%20degrees%20of%20freedom.&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When estimating a mean score or a proportion from a single sample, the number of independent observations is equal to the sample size minus one.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;e.g, if we have 100 observation &lt;span class=&quot;math-container&quot;&gt;$X_1, \ldots, X_{100}$&lt;/span&gt; and we want to estimate their mean &lt;span class=&quot;math-container&quot;&gt;$\bar{X}$&lt;/span&gt;, as &lt;span class=&quot;math-container&quot;&gt;$\bar{X} = \frac{X_1 + \cdots + X_{100}}{100}$&lt;/span&gt;, if we know the mean, just we need to find the value of 99 variables from &lt;span class=&quot;math-container&quot;&gt;$X_1, \ldots, X_{100}$&lt;/span&gt;. Hence, here the degree of freedom is 99.&lt;/p&gt;&#xA;&lt;p&gt;Your referenced paragraph is a general explanation in the paper as well. However, base on the above example, the degree of freedom in the paragraph depends on the likelihood function and number of observations that we have from spectrograms.&lt;/p&gt;&#xA;&lt;p&gt;Now, as the DoF of latent variables is not high, using Gibbs sampling we will approximate some observations, and then using them we will compute the value of the latent variables.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-07-15T14:14:05.560" LastActivityDate="2020-07-15T14:14:05.560" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22495" PostTypeId="2" ParentId="22484" CreationDate="2020-07-14T14:20:14.897" Score="1" Body="&lt;p&gt;The term REINFORCE actually corresponds to a method of estimating gradients, it is not particular to reinforcement learning. The paper you linked doesn't appear to deal with RL at all, so the issue they're describing is not one that you should expect to find in a policy gradient application.&lt;/p&gt;&#xA;&lt;p&gt;If you're using REINFORCE to estimate policy gradients in RL (this is the common use case), your parameters are parameterizing a policy function. The inputs are states and the outputs are actions. The issue with estimating the gradient of the parameters is as follows. In RL the objective is to maximize expected rewards. This is an expectation with respect to the policy of the cumulative reward of a trajectory. If you take the gradient of this wrt the parameters, since the expectation is wrt the policy which depends on the parameters, you can't move the gradient inside the expectation so you can't estimate the gradient from samples. Using REINFORCE however, you use the &amp;quot;log-derivative&amp;quot; trick to rewrite the objective as an expected gradient, which &lt;em&gt;can&lt;/em&gt; be estimated from samples.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-14T14:20:14.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22496" PostTypeId="1" CreationDate="2020-07-14T17:09:56.573" Score="0" ViewCount="30" Body="&lt;p&gt;I've been training a snake for the snake game with Dueling Double DQN as well as the Double DQN. I've noticed that I can see progress being made when the snake starts off from the same location every time. However, I noticed that if I am to randomise the starting location of the snake then the snake has a very hard time learning. In fact I have yet to see the snake make any progress after running for 80k episodes. This is surprising to me as I've reduced the environment size from 15x15 to 6x6, but it is still struggling to learn.&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain how I can improve the learning when the starting position is random? Do I simply just need to train the snake longer? Will increasing the layer neuron counts help with the increased complexity?&lt;/p&gt;&#xA;&lt;p&gt;Does a neural network that is trained based on a fixed starting point becomes more rigid such that it only able to move the snake more intelligently provided the starting point is the same? Would the neural network become incompetent if I now randomise the starting point after having being trained with a fixed starting position?&lt;/p&gt;&#xA;" OwnerUserId="38609" LastActivityDate="2020-07-14T17:09:56.573" Title="Training a neural network for games with dynamic starting locations" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22497" PostTypeId="1" AcceptedAnswerId="22498" CreationDate="2020-07-14T20:11:35.197" Score="4" ViewCount="87" Body="&lt;p&gt;I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the &lt;span class=&quot;math-container&quot;&gt;$|A(s)|$&lt;/span&gt; mean. I guess &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the action set but I'm not sure about that notation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\varepsilon}{|\mathcal{A}(s)|} \sum_{a} Q^{\pi}(s, a)+(1-\varepsilon) \max _{a} Q^{\pi}(s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.incompleteideas.net/book/first/ebook/node54.html&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt; is the source of the formula.&lt;/p&gt;&#xA;&lt;p&gt;I also want to clarify that I understand the idea behind the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy approach and the motivation behind the on-policy methods. I just had a problem understanding this notation (and also some other minor things). The author there omitted some stuff, so I feel like there was a continuity jump, which is why I didn't get the notation, etc. I'd be more than glad if I can be pointed towards a better source where this is detailed.&lt;/p&gt;&#xA;" OwnerUserId="34516" LastEditorUserId="2444" LastEditDate="2020-07-14T20:38:20.040" LastActivityDate="2020-07-14T20:44:55.347" Title="What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;notation&gt;&lt;on-policy-methods&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22498" PostTypeId="2" ParentId="22497" CreationDate="2020-07-14T20:35:00.197" Score="6" Body="&lt;p&gt;This expression: &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}(s)|$&lt;/span&gt; means&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$|\quad|$&lt;/span&gt; the size of&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}(s)$&lt;/span&gt; the set of actions in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;or more simply the number of actions allowed in the state.&lt;/p&gt;&#xA;&lt;p&gt;This makes sense in the given formula because &lt;span class=&quot;math-container&quot;&gt;$\frac{\epsilon}{|\mathcal{A}(s)|}$&lt;/span&gt; is then the probability of taking each exploratory action in an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy. The overall expression is the expected return when following that policy, summing expected results from the exploratory and greedy action.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-14T20:44:55.347" LastActivityDate="2020-07-14T20:44:55.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22499" PostTypeId="1" AcceptedAnswerId="22508" CreationDate="2020-07-15T04:49:28.313" Score="1" ViewCount="45" Body="&lt;p&gt;Do the test dataset feature order and inference (real world) feature order have to be the same as the training dataset? For example, if features are in the order (a,c,b,e,d) for the training dataset, does that particular order have to match for the inference and test dataset?&lt;/p&gt;&#xA;" OwnerUserId="38618" LastActivityDate="2020-07-15T21:33:20.273" Title="Do the order of the features ie channel matter for a 1d convolutional network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22501" PostTypeId="2" ParentId="10738" CreationDate="2020-07-15T12:29:04.330" Score="0" Body="&lt;p&gt;The current version of the document &lt;a href=&quot;https://www.gov.uk/government/publications/code-of-conduct-for-data-driven-health-and-care-technology/initial-code-of-conduct-for-data-driven-health-and-care-technology#principle-7-show-what-type-of-algorithm-is-being-developed-or-deployed-the-ethical-examination-of-how-the-data-is-used-how-its-performance-will-be-validated-and-how-it-will-be-integrated-into-health-and-care-provision&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code of conduct for data-driven health and care technology&lt;/a&gt; provides more details about all principles, including principle number 7, which I will quote here.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Consider how the introduction of AI will change relationships in&#xA;health and care provision, and the implications of these changes for&#xA;responsibility and liability. Use current best practice on how to&#xA;explain algorithms to those taking actions based on their outputs.&lt;/p&gt;&#xA;&lt;p&gt;When building an algorithm, be it a stand-alone product or integrated&#xA;within a system, show it clearly and be transparent of the learning&#xA;methodology (if any) that the algorithm is using. Undertake ethical&#xA;examination of data use specific to this use-case. Achieving&#xA;transparency of algorithms that have a higher potential for harm or&#xA;unintended decision-making, can ensure the rights of the data subject&#xA;as set out in the Data Protection Act 2018 are met, to build trust in&#xA;users and enable better adoption and uptake.&lt;/p&gt;&#xA;&lt;p&gt;Work collaboratively with partners, specify the context for the&#xA;algorithm, specify potential alternative contexts and be transparent&#xA;on whether the model is based on active, supervised or unsupervised&#xA;learning. Show in a clear and transparent specification:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the functionality of the algorithm&lt;/li&gt;&#xA;&lt;li&gt;the strengths and limitations of the algorithm (as far as they are known)&lt;/li&gt;&#xA;&lt;li&gt;its learning methodology&lt;/li&gt;&#xA;&lt;li&gt;whether it is ready for deployment or still in training&lt;/li&gt;&#xA;&lt;li&gt;how the decision has been made on the acceptable use of the algorithm in the context it is being used (for example, is there a&#xA;committee, evidence or equivalent that has contributed to this&#xA;decision?)&lt;/li&gt;&#xA;&lt;li&gt;the potential resource implications&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This specification and transparency in development will build trust in&#xA;incorporating machine-led decision-making into clinical care.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-15T12:29:04.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22502" PostTypeId="1" CreationDate="2020-07-15T15:21:38.493" Score="2" ViewCount="46" Body="&lt;p&gt;While reading a paper about Q-learning in network energy consumption, I came across the section on convergence analysis. Does anyone know what convergence analysis is, and why is convergence analysis needed in reinforcement learning?&lt;/p&gt;&#xA;" OwnerUserId="38632" LastEditorUserId="2444" LastEditDate="2020-07-15T16:14:02.650" LastActivityDate="2020-07-16T01:31:05.193" Title="What is convergence analysis, and why is it needed in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;papers&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22503" PostTypeId="1" CreationDate="2020-07-15T16:04:53.270" Score="1" ViewCount="48" Body="&lt;p&gt;I'm in the process of trying to learn more about RL by shadowing a course offered collaboratively by UCL and DeepMind that has been made available to the public. I'm most of the way through the course, which for auditors consists of a Youtube playlist, copies of the Jupyter notebooks used for homework assigments (thanks to some former students making them public on Github), and reading through Sutton and Barto's wonderful book &lt;em&gt;Reinforcement Learning: An Introduction (2nd edition)&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've gone a little more than half of the book and corresponding course material at this point, thankfully with the aid of public solutions for the homework assignments and textbook exercises which have allowed me to see which parts of my own work that I've done incorrectly. Unfortunately, I've been unable to find such a resource for the last homework assignment offered and so I'm hoping one of the many capable people here might be able to explain parts of the following question to me.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We are given a simple Markov reward process consisting of two states and with a reward of zero everywhere.  When we are in state &lt;span class=&quot;math-container&quot;&gt;$s_{0}$&lt;/span&gt;, we always transition to &lt;span class=&quot;math-container&quot;&gt;$s_{1}$&lt;/span&gt;.  If we are in state &lt;span class=&quot;math-container&quot;&gt;$s_{1}$&lt;/span&gt;, there is a probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (which is set to 0.1 by default) of terminating, after which the next episode starts in &lt;span class=&quot;math-container&quot;&gt;$s_{0}$&lt;/span&gt; again.  With a probability of &lt;span class=&quot;math-container&quot;&gt;$1 - p$&lt;/span&gt;, we transition from &lt;span class=&quot;math-container&quot;&gt;$s_{1}$&lt;/span&gt; back to itself again.  The discount is &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; on non-terminal steps.&lt;/p&gt;&#xA;&lt;p&gt;Instead of a tabular representation, consider a single feature &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, which takes the values &lt;span class=&quot;math-container&quot;&gt;$\phi(s_0) = 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi(s_1) = 4$&lt;/span&gt;.  Now consider using linear function approximation, where we learn a value &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$v_{\theta}(s) = \theta \times \phi(s) \approx v(s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; is the true value of state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$\theta_{0} = 1$&lt;/span&gt;, and suppose we update this parameter with TD(0) with a step size of &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.1$&lt;/span&gt;.  What is the expected value of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[ \theta_T ]$&lt;/span&gt; if we step through the MRP until it terminates after the first episode, as a function of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;?  (Note that &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is random.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My real point of confusion surrounds &lt;span class=&quot;math-container&quot;&gt;$\theta_{0}$&lt;/span&gt; being given as 1. My understanding was that the dimensionality of the parameter vector should be equal to that of the feature vector, which I've understood as being (1, 4) and thus two-dimensional. I also don't grok the idea of evaluating &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[ \theta_T ]$&lt;/span&gt; should &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; be a scalar (as an aside I attempted to simply brute-force simulate the first episode using a scalar parameter of 1 and, unless I made errors, found the value of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; to not depend on &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; whatsoever). If &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is two-dimensional, would that be represented as (1, 0), (0, 1), or (1, 1)?&lt;/p&gt;&#xA;&lt;p&gt;Neither the 1-d or 2-d options make intuitive sense to me so I hope there's something clear and obvious that someone might be able to point out. For more context or should someone just be interested in the assignment, here is a link to the Jupyter notebook:&#xA;&lt;a href=&quot;https://github.com/chandu-97/ADL_RL/blob/master/RL_cw4_questions.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/chandu-97/ADL_RL/blob/master/RL_cw4_questions.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38633" LastEditorUserId="38633" LastEditDate="2020-07-16T12:04:53.327" LastActivityDate="2020-07-16T12:04:53.327" Title="Correct dimensionality of parameter vector for solving an MRP with linear function approximation?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22504" PostTypeId="1" CreationDate="2020-07-15T17:03:01.307" Score="1" ViewCount="506" Body="&lt;p&gt;I already know deep RL, but to learn it deeply I want to know why do we need 2 networks in deep RL. What does the target network do? I now there is huge mathematics into this, but I want to know deep Q-learning deeply, because I am about to make some changes in the deep Q-learning algorithm (i.e. invent a new one). Can you help me to understand what happens during executing a deep Q-learning algorithm intuitively?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-07-15T22:32:37.643" LastActivityDate="2020-07-15T22:33:13.610" Title="Why do we need target network in deep Q learning?" Tags="&lt;deep-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ClosedDate="2020-07-16T15:46:37.100" ContentLicense="CC BY-SA 4.0" />
  <row Id="22506" PostTypeId="1" CreationDate="2020-07-15T19:25:00.960" Score="0" ViewCount="75" Body="&lt;p&gt;I recently trained &lt;a href=&quot;https://www.kaggle.com/c/house-prices-advanced-regression-techniques&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggles &amp;quot;Advanced Housing Prices&amp;quot;-Competition&lt;/a&gt; using Catboost. For training i used a compute-instance on Google Cloud Platform (GCP) (CPU: Xeon Quad-Core, RAM: 15GB, GPU: Tesla V100) and trained the model using the CPU first, then on the GPU.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;reg = CatBoostRegressor(iterations=4000, learning_rate=0.06)&#xA;reg.fit(X, y, cat_features=categorical, verbose=400)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;0:  learn: 76301.1754682    total: 85.8ms   remaining: 5m 43s&#xA;400:    learn: 12187.9081510    total: 6.74s    remaining: 1m&#xA;800:    learn: 8249.9456584 total: 13.5s    remaining: 53.8s&#xA;1200:   learn: 5816.9481262 total: 20.2s    remaining: 47.1s&#xA;1600:   learn: 4371.4781446 total: 26.8s    remaining: 40.2s&#xA;2000:   learn: 3331.5702252 total: 33.4s    remaining: 33.4s&#xA;2400:   learn: 2600.9734000 total: 40s  remaining: 26.6s&#xA;2800:   learn: 2066.7154652 total: 46.5s    remaining: 19.9s&#xA;3200:   learn: 1680.6404763 total: 53.2s    remaining: 13.3s&#xA;3600:   learn: 1373.1095875 total: 59.9s    remaining: 6.64s&#xA;3999:   learn: 1146.4845213 total: 1m 6s    remaining: 0us&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;reg_gpu = CatBoostRegressor(iterations=4000, learning_rate=0.04, loss_function='RMSE', task_type='GPU')&#xA;reg_gpu.fit(X, y, cat_features=categorical, verbose=400)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;0:  learn: 78013.3893895    total: 39.9ms   remaining: 2m 39s&#xA;400:    learn: 24004.7793662    total: 15.1s    remaining: 2m 15s&#xA;800:    learn: 21142.6899404    total: 29.9s    remaining: 1m 59s&#xA;1200:   learn: 19752.2511301    total: 44s  remaining: 1m 42s&#xA;1600:   learn: 18967.4797535    total: 58.3s    remaining: 1m 27s&#xA;2000:   learn: 18201.5697225    total: 1m 12s   remaining: 1m 12s&#xA;2400:   learn: 17599.8105474    total: 1m 26s   remaining: 57.6s&#xA;2800:   learn: 17162.3720952    total: 1m 40s   remaining: 43s&#xA;3200:   learn: 16706.0578195    total: 1m 55s   remaining: 28.7s&#xA;3600:   learn: 16351.7230276    total: 2m 9s    remaining: 14.3s&#xA;3999:   learn: 16073.2521291    total: 2m 23s   remaining: 0us&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I experienced a learning slowdown of about 5x compared to training on the CPU. I gave the same notebook to a colleague to train on his machine (CPU: Ryzen 1700x, RAM: 16GB, GPU: GTX1080) and he experienced a learning-speedup of about 30%.&lt;/p&gt;&#xA;&lt;p&gt;I know, that according to &lt;a href=&quot;https://github.com/catboost/catboost/issues/1034&quot; rel=&quot;nofollow noreferrer&quot;&gt;this issue&lt;/a&gt; GPU-training on small datasets should always be relatively slow and since the mentioned dataset is a table of 1460x82 values it is in fact a relatively small dataset. However, this doesn't explain why my colleague experienced such a speedup.&lt;/p&gt;&#xA;&lt;p&gt;Is the GPU shared when using GCP? How can i accelerate GPU-training on GCP?&lt;/p&gt;&#xA;" OwnerUserId="31589" LastActivityDate="2020-07-15T19:25:00.960" Title="GPU-training on Google Cloud Platform slower than CPU-training" Tags="&lt;regression&gt;&lt;gpu&gt;&lt;google-cloud&gt;&lt;gradient-boosting&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22507" PostTypeId="1" CreationDate="2020-07-15T19:34:24.010" Score="1" ViewCount="39" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What would be the good choice of algorithm to use for character action selection in an RPG, implemented in Python?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I had previously asked this question in the hopes of getting headway on the AI portion of a project I have been working on, only to realized that I had the obstacle of converting the text based game I had created to a custom gym-environment before I could consider algorithm selection. I have found few papers relating to the task I am looking to achieve, hence I've come for advice on how to get started.&lt;/p&gt;&#xA;&lt;p&gt;As my code for that particular task is extensive and rather messy, I have tried to create a minimized version which contains most of the core features for the development of an environment that, from my limited understanding, should be relatable to the larger program which looks to explore the possibility of an RL agent being a second player in a game akin to a simple Pokemon-esque RPG.&lt;/p&gt;&#xA;&lt;p&gt;In this example, the player, or the agent, can choose a class prior to the start of the battle and on each turn, an action from a set can be chosen; for ease of access, I did not add differing action sets for each character class and to use the last 'Special' action, there is a requirement of a specific amount of 'mp' which cannot be increased.&lt;/p&gt;&#xA;&lt;p&gt;I assume that if this game is converted to a compatible environment, an agent can learn how to optimally play this game.&lt;/p&gt;&#xA;&lt;p&gt;I have a few questions aside from the major 'how-to' just because I lack the understanding in this area and for more directly transferable information to the larger project:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I prevent an agent from selecting a particular action when the condition has not been met (e.g not enough MP)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I define the agent's action set after it's character class selection or I have to give the complete list of actions usable in the game and then set probabilities based on the chosen character class?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I handle the agent being unable to act when it's HP is 0, but the game has not ended. Due to the existence of an allied player in the party, for example.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To follow up, how to I allow action selection once it's HP has been restored. &lt;em&gt;(Not currently applicable to the example shown below)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can the agent learn to interact with another player in the game, can I reward the agent for using particular types of actions on a particular type of player? &lt;em&gt;(Not currently applicable to the example shown below)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If, for example, a particular condition was applied to the game environment for one episode but not another, such as healing instead harms, can the agent deduce that such a condition has been met under a particular episode and choose not to use healing-based actions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If I need to make extensions of amendments to the code, just let me know.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Specified code is as follows:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Based of original source-code by users: Citrus-Code and AlexV on Code Review Stack Exchange&#xA;&#xA;import random&#xA;&#xA;def classSelector(enemySelection = False):&#xA;    if enemySelection:&#xA;        classSelection = random.randint(1,3)&#xA;        if classSelection is 1:&#xA;            return 1700&#xA;        elif classSelection is 2:&#xA;            return 1750&#xA;        elif classSelection is 3:&#xA;            return 1300&#xA;        &#xA;    else:&#xA;        print(&amp;quot;Choose a class: \n1. Thief\n2. Warrior\n3. Mage&amp;quot;)&#xA;        classSelection = int(input())&#xA;        print(&amp;quot;\nPlayer Class selection complete! \n&amp;quot;)&#xA;        if classSelection is 1:&#xA;            return 1000&#xA;        elif classSelection is 2:&#xA;            return 1200&#xA;        elif classSelection is 3:&#xA;            return 900&#xA;def battle_simulation():&#xA;    &amp;quot;&amp;quot;&amp;quot;Run a simple interactive RPGChar battle simulation&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    class RPGChar:&#xA;        def __init__(self, health):&#xA;            self.hp = health&#xA;            self.mp = 100&#xA;            self.maxHealth = health&#xA;            # used to denote if a character is defending&#xA;            self.defenseState = 0&#xA;&#xA;        def getDefenseModifier(self):&#xA;            if self.defenseState is 0:&#xA;                return 1&#xA;            else:&#xA;                return 0.75&#xA;&#xA;        def heal(self, heal_amount):&#xA;            self.hp += heal_amount&#xA;            if self.hp &amp;gt; self.maxHealth:&#xA;&#xA;                self.hp = self.maxHealth&#xA;            return heal_amount&#xA;        def cast(self, mpCost):&#xA;            self.mp -= mpCost&#xA;            if self.mp &amp;lt;0:&#xA;                self.mp = 0&#xA;&#xA;        def attack(self, target, damage):&#xA;            target.hp -= int(damage * 1 - target.getDefenseModifier())&#xA;            if target.hp &amp;lt; 0:&#xA;                target.hp = 0&#xA;            return damage&#xA;&#xA;        def defend(self):&#xA;            self.defenseState = -self.defenseState + 1&#xA;&#xA;        def defendReset(self):&#xA;            self.defenseState = 0&#xA;&#xA;    enemy = RPGChar(classSelector(enemySelection=True))&#xA;    player = RPGChar(classSelector())&#xA;    while True:&#xA;        print(&amp;quot;\nATTACK CHOICES\n1. Attack\n2. Defend\n3. Heal\n4. Special&amp;quot;)&#xA;        attack_choice = int(input(&amp;quot;\nSelect an attack: &amp;quot;))&#xA;&#xA;        # The enemy selects an attack by random, but will always attack if hp is full&#xA;        enemy_choice = random.randint(1, 2 if enemy.hp == enemy.maxHealth else 4 )&#xA;&#xA;        if attack_choice is 2:&#xA;            print(f&amp;quot;You defend yourself from incoming attacks!&amp;quot;)&#xA;            player.defend()&#xA;&#xA;        if enemy_choice is 2:&#xA;            print(f&amp;quot;Enemy defends from incoming attacks!&amp;quot;)&#xA;            enemy.defend()&#xA;&#xA;        if attack_choice is 1:&#xA;            print(f&amp;quot;You dealt {player.attack(enemy, 275)} damage.&amp;quot;)&#xA;&#xA;        if enemy_choice is 1:&#xA;            print(f&amp;quot;Mew dealt {enemy.attack(player,250)} damage.&amp;quot;)&#xA;&#xA;        if attack_choice is 3:&#xA;            print(&#xA;                f&amp;quot;You healed {player.heal(random.randint(int(player.maxHealth * 0.1),int(player.maxHealth * 0.2)))} health points.&amp;quot;&#xA;            )&#xA;&#xA;        if enemy_choice is 3:&#xA;            print(&#xA;                f&amp;quot;Mew healed {enemy.heal(random.randint(int(player.maxHealth * 0.1),int(player.maxHealth * 0.15)))} health points.&amp;quot;&#xA;            )&#xA;        if attack_choice is 4:&#xA;            if player.mp &amp;gt; 0:&#xA;                print(f&amp;quot;You dealt {enemy.attack(player,450)} damage.&amp;quot;)&#xA;            else:&#xA;                print(&amp;quot;You do not have MP to use this action!&amp;quot;) &#xA;                print(&amp;quot;You do nothing on this turn.&amp;quot;)&#xA;        if enemy_choice is 4:&#xA;            if enemy.mp &amp;gt; 0:&#xA;                print(f&amp;quot;You dealt {enemy.attack(player,450)} damage.&amp;quot;)&#xA;            else:&#xA;                print(&amp;quot;Enemy does not have MP to use this action!&amp;quot;) &#xA;                print(&amp;quot;Enemy does nothing on this turn.&amp;quot;)&#xA;        if enemy.hp is 0 or player.hp is 0:&#xA;            break&#xA;&#xA;        print(f&amp;quot;Your current health is {player.hp}&amp;quot;)&#xA;        print(f&amp;quot;Mew's current health is {enemy.hp}&amp;quot;)&#xA;&#xA;        enemy.defendReset()&#xA;        player.defendReset()&#xA;&#xA;        print(&amp;quot;\n Next Turn!&amp;quot;)&#xA;&#xA;    print(f&amp;quot;Your final health is {player.hp}&amp;quot;)&#xA;    print(f&amp;quot;Mew's final health is {enemy.hp}&amp;quot;)&#xA;&#xA;    if player.hp &amp;lt; enemy.hp:&#xA;        print(&amp;quot;\nYou lost! Better luck next time!&amp;quot;)&#xA;    else:&#xA;        print(&amp;quot;\nYou won against Mew!&amp;quot;)&#xA;&#xA;&#xA;def Main():&#xA;    battle_simulation()&#xA;&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    Main()&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37983" LastActivityDate="2020-07-15T19:34:24.010" Title="How can I convert a simple CLI RPG to a compatible environment for training an RL agent via stable-baselines?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;getting-started&gt;&lt;open-ai&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22508" PostTypeId="2" ParentId="22499" CreationDate="2020-07-15T21:25:20.657" Score="1" Body="&lt;p&gt;Generally, order matters. A (trained) Neural Network (NN) is just a mathematical function trained on taking some given input and producing the corresponding output. So, if you train a certain node on producing large output if (and only if) an animal is present in a picture (for example), but later you give it the numeric evidence for a car being present in the image, it will still produce large output, indicating an animal being present in the image. This is simply because a node doesn't know what it is receiving or supposed to detect. It just follows its standard mathematical procedure.&lt;/p&gt;&#xA;&lt;p&gt;So, if you train your network on one kind of input data, you must provide the same kind of input data during testing (or at least you cannot simply exchange inputs or outputs of certain nodes without manually correcting for that in the remainder of the network).&lt;/p&gt;&#xA;&lt;p&gt;In the most simple case, consider whether you could simply swap the inputs to that simple function: &lt;span class=&quot;math-container&quot;&gt;$f(x,y) = x^2 + y$&lt;/span&gt;. If you swap the inputs, the output will be different. The exactly same applies to NNs.&lt;/p&gt;&#xA;&lt;p&gt;Addition:&#xA;I think &lt;a href=&quot;https://stackoverflow.com/a/44628011/11478452&quot;&gt;this post&lt;/a&gt; explains and especially illustrates the intuition nicely.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-07-15T21:33:20.273" LastActivityDate="2020-07-15T21:33:20.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22509" PostTypeId="2" ParentId="22504" CreationDate="2020-07-15T21:53:38.770" Score="2" Body="&lt;p&gt;In DQN that was presented in the &lt;a href=&quot;https://www.nature.com/articles/nature14236&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; the update target for the Q-Network is &lt;span class=&quot;math-container&quot;&gt;$\left(r_t + \max_aQ(s_{t+1},a;\theta^-) - Q(s_t,a_t; \theta)\right)^2$&lt;/span&gt; were &lt;span class=&quot;math-container&quot;&gt;$\theta^-$&lt;/span&gt; is some old version of the parameters that gets updated every &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; updates, and the Q-Network with these parameters is the target network.&lt;/p&gt;&#xA;&lt;p&gt;If you didn't use this target network, i.e. if your update target was &lt;span class=&quot;math-container&quot;&gt;$\left(r_t + \max_aQ(s_{t+1},a;\theta) - Q(s_t,a_t; \theta)\right)^2$&lt;/span&gt;,  then learning would become unstable because the target, &lt;span class=&quot;math-container&quot;&gt;$r_t + \max_aQ(s_{t+1},a;\theta)$&lt;/span&gt;, and the prediction, &lt;span class=&quot;math-container&quot;&gt;$Q(s_t,a_t; \theta)$&lt;/span&gt;, are not independent, as they both rely on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A nice analogy I saw once was that it is akin to a dog chasing it's own tail - it will never catch it because the target is non-stationary; this non-stationarity is exactly what the dependence between the target and the prediction cause.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-07-15T22:33:13.610" LastActivityDate="2020-07-15T22:33:13.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22510" PostTypeId="1" CreationDate="2020-07-15T22:02:19.617" Score="1" ViewCount="44" Body="&lt;p&gt;I have an image dataset of about 400 images. 70% of these data points were used for training, 15% for validation, and 15% for testing. I am using the 70% to train a CNN-based binary classifier. I augmented the &lt;em&gt;training&lt;/em&gt; data to around 8000 images. That makes my test set really small in comparison. Is that ok, and what is considered a decent size of images for a test set?&lt;/p&gt;&#xA;" OwnerUserId="38639" LastEditorUserId="2444" LastEditDate="2020-07-15T22:31:50.233" LastActivityDate="2020-07-15T22:31:50.233" Title="What is the amount of test data needed to evaluate a CNN?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;testing&gt;&lt;data-augmentation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22511" PostTypeId="2" ParentId="22502" CreationDate="2020-07-16T01:31:05.193" Score="3" Body="&lt;p&gt;Convergence analysis is about proving that your policy and/or value function converge to some desired value, which is usually the fixed-point of an operator or an extremum. So it essentially proves that theoretically the algorithm achieves the desired function. Without convergence, we have no guarantees that the value function will be accurate or the policy will be any good, so in other words the proposed RL algorithm can completely fail at serving its purpose even in simple cases.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-16T01:31:05.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22513" PostTypeId="1" CreationDate="2020-07-16T02:47:50.897" Score="1" ViewCount="61" Body="&lt;p&gt;During a Machine Learning course which I have done I have learnt about the K means algorithm. Is it possible to use the principals of K means within a neural network?&lt;/p&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="31949" LastEditDate="2020-07-16T07:03:51.567" LastActivityDate="2020-07-16T07:03:51.567" Title="Would it be possible to implement the principals of the K means clustering algorithm in a Neural Network" Tags="&lt;neural-networks&gt;&lt;clustering&gt;&lt;k-means&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22514" PostTypeId="1" CreationDate="2020-07-16T07:40:16.523" Score="0" ViewCount="46" Body="&lt;p&gt;Do we have to use the IOB format on labels in the NER dataset (such as B-PERSON, I-PERSON, etc.) instead of using the usual format (PERSON, ORGANIZATION, etc.)? If so, why? How will it affect the performance of the model?&lt;/p&gt;&#xA;" OwnerUserId="36258" LastActivityDate="2021-01-31T17:06:45.623" Title="Do we have to use the IOB format on labels in the NER dataset? If so, why?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;named-entity-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22515" PostTypeId="2" ParentId="22460" CreationDate="2020-07-16T08:04:33.927" Score="0" Body="&lt;p&gt;You may try to adjust the learning rate first. As the learning rate has a great effect on changing the weights and the bias value.&lt;/p&gt;&#xA;&lt;p&gt;See if the results has changed after adjusting the learning rate.&lt;/p&gt;&#xA;" OwnerUserId="38650" LastActivityDate="2020-07-16T08:04:33.927" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22516" PostTypeId="1" AcceptedAnswerId="22519" CreationDate="2020-07-16T08:54:45.820" Score="0" ViewCount="34" Body="&lt;p&gt;I would like to train a model that serializes a table of nutrition facts into it's values.&#xA;The tables can vary in form and colour, but always contain the same set of keys (e.g. carbs, fats).&#xA;Examples for these tables can be found &lt;a href=&quot;https://www.google.de/search?q=n%C3%A4hrwerttabelle&amp;amp;tbm=isch&amp;amp;tbs=rimg%3ACWwAb8YC7ysGYe8IH4zwI7P5&amp;amp;hl=de&amp;amp;ved=0CAIQrnZqFwoTCMDMpr2w0eoCFQAAAAAdAAAAABAG&amp;amp;biw=1903&amp;amp;bih=938&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;The end goal is to be able to take a picture of such a table and have it's values added to a database.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;My initial idea was to train a model on finding subpictures of the individual key/value pairs and then using OCR to find out which value it actually is.&lt;/p&gt;&#xA;&lt;p&gt;As I am relatively new to ML, I would love to have some ideas about how one could try to build this, so I can do further research on it.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="38651" LastActivityDate="2020-07-16T12:30:07.797" Title="Detect data in tables of roughly the same structure" Tags="&lt;image-recognition&gt;&lt;object-detection&gt;&lt;models&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22517" PostTypeId="1" CreationDate="2020-07-16T10:43:17.897" Score="0" ViewCount="59" Body="&lt;p&gt;I need to generate a 3D plane given a set of feature inputs. Most inputs are a range of values between 0 and 1 (sigmoidial), except a few. For example a rectangle:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;width = 10 &amp;quot;&amp;quot;&amp;quot;in meters&amp;quot;&amp;quot;&amp;quot;&#xA;height = 10 &amp;quot;&amp;quot;&amp;quot;meters as above&amp;quot;&amp;quot;&amp;quot;&#xA;thickness = 1.3 &amp;quot;&amp;quot;&amp;quot;meters as above&amp;quot;&amp;quot;&amp;quot;&#xA;chamfers = [0.2, 0.6, 0.3, 0, 0.12, 0.54, 0.2, 0.8] &amp;quot;&amp;quot;&amp;quot;edge chamfers left to right and top to bottom ordered according to origin coords, presumably 0,0&amp;quot;&amp;quot;&amp;quot;&#xA;radiuses = [0.3, 0.7, 0.8, 0.1, 0.9, 0.23, 0.87, 0] &amp;quot;&amp;quot;&amp;quot;same as above, but for corners&amp;quot;&amp;quot;&amp;quot;&#xA;materials = [&amp;quot;iron&amp;quot;, &amp;quot;lead&amp;quot;, &amp;quot;carbon&amp;quot;] &amp;quot;&amp;quot;&amp;quot;what this plane should be made of, both materials and their comb ratios are open to curiosity&amp;quot;&amp;quot;&amp;quot;&#xA;rigidity = 0.45 &amp;quot;&amp;quot;&amp;quot;0 least rigid, 1 max rigidity&amp;quot;&amp;quot;&amp;quot;&#xA;tensile = 0.2 &amp;quot;&amp;quot;&amp;quot;0 least strength, 1 max strength&amp;quot;&amp;quot;&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;also some interesting stuff besides pure material eng&amp;quot;&amp;quot;&amp;quot;&#xA;reflection = 0.68 &amp;quot;&amp;quot;&amp;quot;how reflective surface is&amp;quot;&amp;quot;&amp;quot;&#xA;heights = [[3,2,-0.54], [4,2,-0.21]] &amp;quot;&amp;quot;&amp;quot;optional map of x,y coords respective to width and height, to adjust the surface bumps&amp;quot;&amp;quot;&amp;quot;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So it should be clear that the new plane generation using a predefined random seed data is not optimal. However it seems very difficult to determine how exactly a ML network can address this challenge.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What kinds of networks are more suitable for these inputs?&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I've looked at GANs, but they seem to work best for &amp;quot;pixels&amp;quot; and &amp;quot;image patterns&amp;quot;. CNNs also seem to address images, while both need to be trained using an existing dataset which tunes the outputs to the training dataset.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;What kinds of trainings should I be looking at?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I understand you may have a lot of questions, I'll add edits addressing them. All help is appreciated, I'm beyond my skills here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1&lt;/strong&gt;&#xA;There will be many expected outputs, but to start off answering above questions we'd expect render data as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vertices = [[1,2,3], [5,2,2], ...] &amp;quot;&amp;quot;&amp;quot;vector3 vertices for rendering&amp;quot;&amp;quot;&amp;quot;&#xA;composition_attr = {&amp;quot;conductivity&amp;quot;: &amp;lt;sigmoid&amp;gt;, &amp;quot;weight&amp;quot;: &amp;lt;kg/m3&amp;gt;, ...} &amp;quot;&amp;quot;&amp;quot;for analysis, recording and passing on to other systems&amp;quot;&amp;quot;&amp;quot;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38654" LastEditorUserId="-1" LastEditDate="2020-07-19T10:15:19.987" LastActivityDate="2020-07-19T10:15:19.987" Title="Need to determine a ML solution for the given graphical problem" Tags="&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22518" PostTypeId="1" AcceptedAnswerId="22520" CreationDate="2020-07-16T11:33:12.990" Score="1" ViewCount="433" Body="&lt;p&gt;I'm looking for intuition in simple words but also some simple insights (I don't know if the latter is possible). Can anybody shed some light on the Turing test?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-07-16T16:52:54.403" LastActivityDate="2020-07-16T17:02:38.893" Title="What is the Turing test?" Tags="&lt;agi&gt;&lt;definitions&gt;&lt;intelligence-testing&gt;&lt;turing-test&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22519" PostTypeId="2" ParentId="22516" CreationDate="2020-07-16T12:30:07.797" Score="0" Body="&lt;p&gt;Assuming all of the tables will be oriented in similar ways (label and value running horizontally) and that all writing will be printed rather than handwritten, one solution method would be to use an image segmentation method such as &lt;a href=&quot;https://docs.opencv.org/trunk/da/d22/tutorial_py_canny.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;edge detection&lt;/a&gt; to segregate these horizontal (label, value) pairs and then use a library like &lt;a href=&quot;https://guides.nyu.edu/c.php?g=884390&amp;amp;p=6355021&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tesseract&lt;/a&gt; for OCR.&lt;/p&gt;&#xA;&lt;p&gt;There are &lt;a href=&quot;https://missinglink.ai/guides/computer-vision/image-segmentation-deep-learning-methods-applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;many types&lt;/a&gt; of image segmentation methods that may all have value, but if my assumption holds regarding the neat, structured nature of the tables, then I think simple edge detection methods could be sufficient.&lt;/p&gt;&#xA;" OwnerUserId="38633" LastActivityDate="2020-07-16T12:30:07.797" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22520" PostTypeId="2" ParentId="22518" CreationDate="2020-07-16T12:56:28.450" Score="3" Body="&lt;p&gt;The &lt;strong&gt;Turing test&lt;/strong&gt; is a test proposed by Alan Turing (one of the founders of computer science and artificial intelligence), described in section 1 of paper &lt;a href=&quot;https://academic.oup.com/mind/article/LIX/236/433/986238&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing Machinery and Intelligence&lt;/a&gt; (1950), to answer the question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Can machines think?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;More precisely, the Turing test was originally framed as an interactive quiz (denoted as the &lt;strong&gt;imitation game&lt;/strong&gt; by Turing) where a human interrogator &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; asks multiple questions to two entities, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (a computer) and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (a human), which stay in different rooms than the room of the interrogator, so the interrogator cannot see them, in order to figure out which one is &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (the computer) and which one is &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (the human). &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;  can only communicate in written form or any form that avoids them being easily recognized by &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. The goal of the computer is to fool the interrogator and make him/her believe that it is a human and the goal of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is to somehow help him and make him believe that he/she is the actual human.&lt;/p&gt;&#xA;&lt;p&gt;If the computer is able to fool the interrogator and make him/her believe that it is a human, then that would be an indication that machines can think. However, note that even Turing called this game the &lt;em&gt;&lt;strong&gt;imitation&lt;/strong&gt; game&lt;/em&gt;, so Turing was aware of the fact that this game would only really show that a machine can &lt;strong&gt;imitate&lt;/strong&gt; a human (unless he was using the term &amp;quot;imitation&amp;quot; differently than its current meaning).&lt;/p&gt;&#xA;&lt;p&gt;Nowadays, there are different variations of the Turing test and some people use the term &lt;em&gt;Turing test&lt;/em&gt; to refer to any test that attempts to tell humans and computers apart. For example, some people consider the &lt;strong&gt;CAPTCHA&lt;/strong&gt; test a Turing test. In fact, &lt;a href=&quot;http://www.captcha.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAPTCHA stands for &amp;quot;&lt;em&gt;Completely Automated Public &lt;strong&gt;Turing Test&lt;/strong&gt; To Tell Computers and Humans Apart&lt;/em&gt;&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The Turing test also has different interpretations and meanings. Some people think that the Turing test is sufficient to test that a machine can actually think and possesses consciousness, other people think that this only tests human-like intelligence (and there could be other intelligences) and some people (like me) think that this test is limited and only tests the conversational skills (and maybe other properties too) of the machine. Even Turing attempted to address these issues in the same paper (section 2), where he discusses some advantages and disadvantages of his imitation game. In any case, we can all agree that, if machines (in particular, programs like Siri, Google Home, Cortana, or Alexa) were always able to pass the Turing test, they would be a lot more useful, interesting and entertaining than they are now.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-16T16:57:49.813" LastActivityDate="2020-07-16T16:57:49.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22521" PostTypeId="1" CreationDate="2020-07-16T13:41:39.910" Score="4" ViewCount="112" Body="&lt;p&gt;I'm doing some introductory research on classical (stochastic) MABs. However, I'm a little confused about the common notation (e.g. in the popular paper of &lt;a href=&quot;https://homes.di.unimi.it/cesa-bianchi/Pubblicazioni/ml-02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Auer (2002)&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/pdf/1204.5721.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bubeck and Cesa-Bianchi (2012)&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;As in the latter study, let us consider an MAB with a finite number of arms &lt;span class=&quot;math-container&quot;&gt;$i\in\{1,...,K\}$&lt;/span&gt;, where an agent choses at every timestep &lt;span class=&quot;math-container&quot;&gt;$t=1,...,n$&lt;/span&gt; an arm &lt;span class=&quot;math-container&quot;&gt;$I_t$&lt;/span&gt; which generates a reward &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,t}$&lt;/span&gt; according to a distribition &lt;span class=&quot;math-container&quot;&gt;$v_{I_t}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In my understanding, each arm has an inherent distribution, which is unknown to the agent. Therefore, I'm wondering why the notation &lt;span class=&quot;math-container&quot;&gt;$v_{I_t}$&lt;/span&gt; is used instead of simply using &lt;span class=&quot;math-container&quot;&gt;$v_{i}$&lt;/span&gt;? Isn't the distribution independent of the &lt;strong&gt;time&lt;/strong&gt; the arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; was chosen?&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, I ask myself: Why not simply use &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,t}$&lt;/span&gt; (in terms of rewards). Is it because the chosen arm at step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (namely &lt;span class=&quot;math-container&quot;&gt;$I_t$&lt;/span&gt;) is a random variable and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; depends on it? If I am right, why is &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; used twice in the index (namely &lt;span class=&quot;math-container&quot;&gt;$I_t,t$&lt;/span&gt;)?&#xA;Shouldn't &lt;span class=&quot;math-container&quot;&gt;$X_{I_t}$&lt;/span&gt; be sufficient, since &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,m}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,n}$&lt;/span&gt; are drawn from the same distribution?&lt;/p&gt;&#xA;" OwnerUserId="38657" LastEditorUserId="2444" LastEditDate="2020-12-22T14:57:16.427" LastActivityDate="2020-12-22T14:57:16.427" Title="Why do we use $X_{I_t,t}$ and $v_{I_t}$ to denote the reward received and the at time step $t$ and the distribution of the chosen arm $I_t$?" Tags="&lt;papers&gt;&lt;notation&gt;&lt;multi-armed-bandit&gt;&lt;upper-confidence-bound&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22522" PostTypeId="2" ParentId="22518" CreationDate="2020-07-16T16:29:14.853" Score="1" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Turing_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The &amp;quot;standard interpretation&amp;quot; of the Turing test, in which player C, the interrogator, is given the task of trying to determine which player – A or B – is a computer and which is a human. The interrogator is limited to using the responses to written questions to make the determination.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BqThf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BqThf.png&quot; alt=&quot;Turing test diagram&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32076" LastEditorUserId="2444" LastEditDate="2020-07-16T17:02:38.893" LastActivityDate="2020-07-16T17:02:38.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22524" PostTypeId="2" ParentId="22521" CreationDate="2020-07-16T17:03:25.090" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Isn't the distribution independent of the time the arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; was chosen?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, but you don't know which arm was chosen at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, that is what &lt;span class=&quot;math-container&quot;&gt;$I_t$&lt;/span&gt; represents. &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; would represent the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th arms distribution, whereas you want the distribution of the arm that was chosen at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, which is &lt;span class=&quot;math-container&quot;&gt;$v_{I_t}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X_{I_t,t}$&lt;/span&gt; is used to represent the arm you chose at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and the time you chose it. Imagine if we chose the following arms at respective time steps {1,2,5,2,4}, then the reward you get at time 3 (assuming in my example time started at 1) would be &lt;span class=&quot;math-container&quot;&gt;$X_{5,3}$&lt;/span&gt;. You need this notation because every time you pull arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; you get a different reward, because the reward is a random variable (unless the assumption was a deterministic reward but that would not be interesting).&lt;/p&gt;&#xA;&lt;p&gt;It has been a while since I read the papers but I assume that the distributions of the arms are stationary, however this notation is more general and would allow for non-stationary distributions.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-07-16T21:14:58.073" LastActivityDate="2020-07-16T21:14:58.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22525" PostTypeId="1" CreationDate="2020-07-16T17:14:43.570" Score="1" ViewCount="123" Body="&lt;p&gt;I'm confused as to the purpose of training a neural network (NN) for reinforcement learning (RL) tasks such as Gridworld. In RL tasks, namely q-learning, we have a q-learning update rule, which is designed to take some state and action and compute the value of that state-action pair.&lt;/p&gt;&#xA;&lt;p&gt;Performing this process several times will eventually produce a table of states and what action will likely lead to a high reward.&lt;/p&gt;&#xA;&lt;p&gt;In RL examples, I've seen them train a neural network to output q-values and a loss function like MSE to compute the loss between the q-learning update rule q-value and the NN's q value.&lt;/p&gt;&#xA;&lt;p&gt;So:&lt;/p&gt;&#xA;&lt;p&gt;(a) Q-learning update rule-&amp;gt; outputs target Q-values&lt;/p&gt;&#xA;&lt;p&gt;(b) NN -&amp;gt; outputs Q values&lt;/p&gt;&#xA;&lt;p&gt;MSE to compute the loss between (a) and (b)&lt;/p&gt;&#xA;&lt;p&gt;So, given we already know what the target Q-value is from a, why do we need to train a NN?&lt;/p&gt;&#xA;" OwnerUserId="26159" LastActivityDate="2020-07-16T17:45:52.003" Title="What is the purpose of a Neural Network in Reinforcement Learning when we have a Q-learning update rule?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22526" PostTypeId="2" ParentId="22525" CreationDate="2020-07-16T17:45:52.003" Score="3" Body="&lt;p&gt;I don't think people generally do use neural nets for grid world. As long as the state and action spaces are small enough, you should be able to store Q values in a table like you suggested. Neural nets come in handy when the state space is very large (or even continuous), so you can't afford to store a table of Q values. Also, neural nets have the ability to generalize across &amp;quot;similar&amp;quot; states -- for instance, if two states are very similar the neural net would likely produce similar values for those states, whereas a tabular implementation might not have seen enough data to accurately estimate the Q values of both.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-16T17:45:52.003" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22528" PostTypeId="2" ParentId="22521" CreationDate="2020-07-16T21:08:48.407" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Isn't the distribution independent of the time the arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; was chosen?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Each one of the two references you describe assumes the context of the random bandit problem proposed by &lt;a href=&quot;https://projecteuclid.org/euclid.bams/1183517370&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robbins (1952)&lt;/a&gt; where the underlying reward distributions of each bandit are fixed. Therefore, &lt;strong&gt;yes&lt;/strong&gt;, the underlying distributions are independent of the current time.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is it because the chosen arm at step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (namely &lt;span class=&quot;math-container&quot;&gt;$I_t$&lt;/span&gt;) is a random variable and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; depends on it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The reward is a random variable which is dependent on the chosen arm at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Since each arm has an underlying reward distribution, the index &lt;span class=&quot;math-container&quot;&gt;$I_t$&lt;/span&gt; is a random variable that designates the specific arm we are pulling, and the index &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; denotes the time step when we pull the arm.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why is &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; used twice in the index (namely &lt;span class=&quot;math-container&quot;&gt;$I_t,t$&lt;/span&gt;)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is used twice, but the observed value of &lt;span class=&quot;math-container&quot;&gt;$I_t$&lt;/span&gt; &lt;strong&gt;does not encode any information about the time it was chosen&lt;/strong&gt;. For example, if &lt;span class=&quot;math-container&quot;&gt;$I_m = 5$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$X_{I_m,\ m} = X_{5,\ m}$&lt;/span&gt;. If we drop the second subscript, then we have no way to distinguish &lt;span class=&quot;math-container&quot;&gt;$X_{5,\ m}$&lt;/span&gt; notationally from &lt;span class=&quot;math-container&quot;&gt;$X_{5,\ n}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$I_{n\ \neq\ m} = 5$&lt;/span&gt;). Two distinct rewards &lt;span class=&quot;math-container&quot;&gt;$X_{5,\ m}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{5,\ n}$&lt;/span&gt; would map to the same reward &lt;span class=&quot;math-container&quot;&gt;$X_5$&lt;/span&gt; notationally. At first glance, this introduces numerous potential notational problems, such as losing the count of the number of times each arm was pulled.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why not simply use &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,\ t}$&lt;/span&gt; (in terms of rewards)? Shouldn't &lt;span class=&quot;math-container&quot;&gt;$X_{I_t}$&lt;/span&gt; be sufficient, since &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,\ m}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X_{I_t,\ n}$&lt;/span&gt; are drawn from the same distribution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Admittedly, there are probably ways to get around the extra subscript for certain algorithms. For example, maybe you are using an algorithm where past rewards from each arm are averaged to yield an estimate of each arm's expected reward (see Section 2.2 of &lt;a href=&quot;http://www.incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt;). This may require a collection of lists that store the past rewards for each arm, or it may require the count of each arm being pulled and an associated current estimate of the expected reward (see Section 2.4 of Sutton and Barto). However, &lt;strong&gt;these methods introduce more parameters that would be unnecessary had we initially included a second subscript for time in our notation&lt;/strong&gt; (e.g. the counts of each arm pulled, the current estimate of expected reward for each arm, the labels of each reward list corresponding to an arm, etc.). Most of the fundamental equations regarding multi-armed bandits that I have seen are either &lt;strong&gt;heavily or solely dependent on the reward random variable&lt;/strong&gt; (e.g. the definition of regret). Keeping the time index in a single random variable &lt;strong&gt;promotes concision and consistency&lt;/strong&gt; among various sources by &lt;strong&gt;preventing the need for delegating the time index to another random variable, data structure, etc.&lt;/strong&gt;, even though specific implementations or contexts may profit from other notations.&lt;/p&gt;&#xA;&lt;p&gt;The dual-subscript notation also has the benefit of generalizing to other contexts aside from the one posed by Robbins (1952). These include nonstationary reward distributions (see Section 2.5 of Sutton and Barto), time discounting, and families of alternative bandit processes, among others (see Sections 2.2-2.4 of &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/book/10.1002/9780470980033&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt; for info on the last two extensions).&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-07-16T21:08:48.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22529" PostTypeId="1" CreationDate="2020-07-17T01:08:03.587" Score="1" ViewCount="41" Body="&lt;p&gt;I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take the under-fitted random forest with little depth and try to boost it?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-07-17T12:23:38.670" LastActivityDate="2020-07-17T12:23:38.670" Title="Can I apply AdaBoost on a random forest?" Tags="&lt;machine-learning&gt;&lt;decision-tree&gt;&lt;random-forest&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22530" PostTypeId="1" CreationDate="2020-07-17T01:17:09.320" Score="2" ViewCount="302" Body="&lt;p&gt;Why is the expected return in Reinforcement Learning (RL) computed as a sum of cumulative rewards?&lt;/p&gt;&#xA;&lt;p&gt;Would it not make more sense to compute &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}(R \mid s, a)$&lt;/span&gt; (the expected return for taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;) as the average of all rewards recorded for being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;In many examples, I've seen the value of a state computed as the expected return computed as the cumulative sum of rewards multiplied by a discount factor:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V^π(s)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}(R \mid s)$&lt;/span&gt; (the value of state s, if we follow policy π is equal to the expected return given state s)&lt;/p&gt;&#xA;&lt;p&gt;So, &lt;span class=&quot;math-container&quot;&gt;$V^π(s)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}(r_{t+1}+ γr_{t+2}+ (γ^2)_{t+3} + ...    \mid s) = {E}(∑γ^kr_{t+k+1}\mid s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;as &lt;span class=&quot;math-container&quot;&gt;$R=r_{t+1}+ γr_{t+2}+ {γ^2}r_{t+3}, + ...   $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Would it not make more sense to compute the value of a state as the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V^π(s)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$(r_{t+1} + γr_{t+2} + (γ^2)_{t+3}, + ...    \mid s)/k = {E}(∑γ^kr_{t+k+1}\mid s)/k $&lt;/span&gt; where k is the number of elements in the sum, thus giving us the average reward for being in state s.&lt;/p&gt;&#xA;&lt;p&gt;Reference for cumulative sum example: &lt;a href=&quot;https://joshgreaves.com/reinforcement-learning/understanding-rl-the-bellman-equations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://joshgreaves.com/reinforcement-learning/understanding-rl-the-bellman-equations/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="26159" LastEditDate="2020-07-17T12:18:37.710" LastActivityDate="2020-07-17T16:46:03.407" Title="Why is the expected return in Reinforcement Learning (RL) computed as a sum of cumulative rewards?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;rewards&gt;&lt;value-functions&gt;&lt;return&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22531" PostTypeId="1" CreationDate="2020-07-17T03:17:28.277" Score="1" ViewCount="39" Body="&lt;p&gt;It happened to my neural network, when I use a learning rate of &amp;lt;0.2 everything works fine, but when I try something above 0.4 I start getting &amp;quot;nan&amp;quot; errors because the output of my network keeps increasing.&lt;/p&gt;&#xA;&lt;p&gt;From what I understand, what happens is that if I choose a learning rate that is too large, I overshoot the local minimum. But still, I am getting somewhere, and from there I'm moving in the correct direction. At worst my output should be random, I don't understand what is the scenario that causes my output and error to approach infinity every time I run my NN with a learning rate that is too large (and it's not even that large)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oYC2z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oYC2z.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;How does the red line go to infinity ever? I kind of understand it could happen if we choose a crazy high learning rate, but if the NN works for 0.2 and doesn't for 0.4, I don't understand that&lt;/p&gt;&#xA;" OwnerUserId="38668" LastActivityDate="2020-07-17T03:17:28.277" Title="How can a learning rate that is too large cause the output of the network (and the error) to go to infinity?" Tags="&lt;learning-rate&gt;&lt;mean-squared-error&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22532" PostTypeId="1" CreationDate="2020-07-17T05:47:11.613" Score="1" ViewCount="39" Body="&lt;p&gt;For example, in &lt;a href=&quot;https://arxiv.org/pdf/1612.00593.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PointNet&lt;/a&gt;, you see the 1D convolutions with the following channels &lt;code&gt;64 -&amp;gt; 128 -&amp;gt; 1024&lt;/code&gt;. Why not e.g. &lt;code&gt;64 -&amp;gt; 1024 -&amp;gt; 1024&lt;/code&gt; or &lt;code&gt;1024 -&amp;gt; 1024 -&amp;gt; 1024&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2020-07-17T12:03:53.990" LastActivityDate="2020-07-17T12:03:53.990" Title="Why does the number of channels in the PointNet increase as we go deeper?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;filters&gt;&lt;convolutional-layers&gt;&lt;1d-convolution&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22533" PostTypeId="1" AcceptedAnswerId="22540" CreationDate="2020-07-17T05:51:10.757" Score="0" ViewCount="130" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;There was a lot of Negative news on &lt;a href=&quot;https://www.lasserouhiainen.com/7-questions-artificial-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence&lt;/a&gt;. Most people were first exposed to the idea of artificial intelligence from Hollywood movies, long before they ever started seeing it in their day-to-day lives. This means that many people misunderstand the technology. When they think about common examples that they’ve seen in movies or television shows, they may not realize that the killer robots they’ve seen were created to sell emotional storylines and drive the entertainment industry, rather than to reflect the actual state of AI technology.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are few questions on our SE on how AI impacts/harms humankind. For example, &lt;a href=&quot;https://ai.stackexchange.com/a/15478/2444&quot;&gt;How could artificial intelligence harm us?&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/10504/2444&quot;&gt;Could artificial general intelligence harm humanity?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, now, I'm looking for the positive impacts of AI on humans. How could AI help humankind?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-07-17T11:40:41.773" LastActivityDate="2020-12-14T13:02:55.320" Title="How is AI helping humanity?" Tags="&lt;philosophy&gt;&lt;social&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22534" PostTypeId="2" ParentId="22533" CreationDate="2020-07-17T07:08:19.850" Score="0" Body="&lt;p&gt;We've already seen significant progress in fields that we could not even come close to prior to the explosion in AI research. For example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6895901/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the automated identification of cancerous tumours in lung tissues&lt;/a&gt; could save countless lives, as a computer never tires and has no bias. Incredible advances in &lt;a href=&quot;https://deepmind.com/blog/article/wavenet-generative-model-raw-audio&quot; rel=&quot;nofollow noreferrer&quot;&gt;speech synthesis&lt;/a&gt; can allow people who lost or never had a voice to have their own human-sounding voice. Personally, I also predict soon we will be able to synthesise music to an individual's tastes.&lt;/p&gt;&#xA;&lt;p&gt;Advances in rapid image processing, thanks to the polynomial complexity of machine learning techniques, allows for advanced guidance systems and autonomous cars that are only improving. Ranking algorithms for listings allow for finely tuned results for a user's search. Unsupervised techniques can help us separate suspicious activity in banking records to catch fraudulent transactions.&lt;/p&gt;&#xA;&lt;p&gt;There is a very, very long list here and this only scratches the surface. AI is already benefiting us enormously, to the point where very soon, if not already, I would say we will become so dependant on learning techniques that if they were outlawed, entire systems may collapse.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2020-07-17T11:53:21.593" LastActivityDate="2020-07-17T11:53:21.593" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22535" PostTypeId="1" CreationDate="2020-07-17T07:09:59.120" Score="2" ViewCount="52" Body="&lt;p&gt;I'm using Experience Replay based on the original &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prioritized Experience Replay (PER)&lt;/a&gt; paper. In the paper authors show ~ an order of magnitude increase in data efficiency from prioritized sampling. There is space for further improvement, since PER remembers all experiences, regardless of their importance.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to extend PER so it remembers selectively based on some &lt;strong&gt;metric&lt;/strong&gt;, which would determine whether the experience is worth remembering or not. The time of sampling and re-adjusting the importance of the experiences increases with the number of samples remembered, so being smart about remembering should at the very least speed-up the replay, and hopefully also show some increase in data efficiency.&lt;/p&gt;&#xA;&lt;p&gt;Important design constrains for this &lt;strong&gt;remembering metric&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;compatibility with Q-Learning, such as DQN&lt;/li&gt;&#xA;&lt;li&gt;computation time, to speed up the process of learning and not trade off one type of computation for another&lt;/li&gt;&#xA;&lt;li&gt;simplicity&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What considerations would you make for designing such a metric?&lt;/li&gt;&#xA;&lt;li&gt;Do you know about any articles addressing the prioritized experience memorization for Q-Learning?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="38671" LastEditorUserId="38671" LastEditDate="2020-07-17T07:51:34.023" LastActivityDate="2020-07-17T07:51:34.023" Title="Prioritised Remembering in Experience Replay (Q-Learning)" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22536" PostTypeId="1" CreationDate="2020-07-17T07:28:09.220" Score="0" ViewCount="51" Body="&lt;p&gt;I'm trying to train the most popular Models (mobileNet, VGG16, ResNet...) with the CIFAR10-dataset but the accuracy can't get above 9,9%. I want to do that with the completely model (include_top=True) and without the weights from imagenet.&lt;/p&gt;&#xA;&lt;p&gt;I have tried increasing/decreasing dropout and learning rate and I changed the optimizers but I become always the same accuracy.&lt;/p&gt;&#xA;&lt;p&gt;with weights='imagenet' and include_top=False I achieve an accuracy of over 90% but I want to train the model without those parameters.&lt;/p&gt;&#xA;&lt;p&gt;Is there any solution to solve this? It is possible, that the layers of those Models are not set to be trainable?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;train_generator = ImageDataGenerator(&#xA;                                    rotation_range=2, &#xA;                                    horizontal_flip=True,&#xA;                                    zoom_range=.1 )&#xA;val_generator = ImageDataGenerator(&#xA;                                    rotation_range=2, &#xA;                                    horizontal_flip=True,&#xA;                                    zoom_range=.1)&#xA;&#xA;train_generator.fit(x_train)&#xA;val_generator.fit(x_val)&#xA;&#xA;base_model_1 = MobileNet(include_top=True,weights=None,input_shape=(32,32,3),classes=y_train.shape[1])&#xA;&#xA;batch_size= 100&#xA;epochs=50&#xA;&#xA;learn_rate=.001&#xA;&#xA;sgd=SGD(lr=learn_rate,momentum=.9,nesterov=False)&#xA;adam=Adam(lr=learn_rate, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)&#xA;&#xA;model_1.compile(optimizer=adam,loss='sparse_categorical_crossentropy',metrics=['accuracy'])&#xA;&#xA;model_1.fit_generator(train_generator.flow(x_train,y_train,batch_size=batch_size),&#xA;                      epochs=epochs,&#xA;                      steps_per_epoch=x_train.shape[0]//batch_size,&#xA;                      validation_data=val_generator.flow(x_val,y_val,batch_size=batch_size),validation_steps=250,&#xA;                      verbose=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results of MobileNet:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Epoch 1/50&#xA;350/350 [==============================] - 17s 50ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1021&#xA;Epoch 2/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1030&#xA;Epoch 3/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016&#xA;Epoch 4/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1014&#xA;Epoch 5/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1040&#xA;Epoch 6/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1009&#xA;Epoch 7/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1035&#xA;Epoch 8/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1013&#xA;Epoch 9/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1029&#xA;Epoch 10/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1023&#xA;Epoch 11/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1017&#xA;Epoch 12/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1020&#xA;Epoch 13/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1020&#xA;Epoch 14/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1033&#xA;Epoch 15/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1011&#xA;Epoch 16/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016&#xA;Epoch 17/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1024&#xA;Epoch 18/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1024&#xA;Epoch 19/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1041&#xA;Epoch 20/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1010&#xA;Epoch 21/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022&#xA;Epoch 22/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1014&#xA;Epoch 23/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1035&#xA;Epoch 24/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1032&#xA;Epoch 25/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1012&#xA;Epoch 26/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1018&#xA;Epoch 27/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022&#xA;Epoch 28/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1031&#xA;Epoch 29/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022&#xA;Epoch 30/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1015&#xA;Epoch 31/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1028&#xA;Epoch 32/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1015&#xA;Epoch 33/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1030&#xA;Epoch 34/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1003&#xA;Epoch 35/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1044&#xA;Epoch 36/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1012&#xA;Epoch 37/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022&#xA;Epoch 38/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1021&#xA;Epoch 39/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1028&#xA;Epoch 40/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1012&#xA;Epoch 41/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1035&#xA;Epoch 42/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1009&#xA;Epoch 43/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1034&#xA;Epoch 44/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1024&#xA;Epoch 45/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016&#xA;Epoch 46/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1028&#xA;Epoch 47/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016&#xA;Epoch 48/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1033&#xA;Epoch 49/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1018&#xA;Epoch 50/50&#xA;350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1023&#xA;&#xA;&amp;lt;tensorflow.python.keras.callbacks.History at 0x7fa30b188e48&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38282" LastEditorUserId="38282" LastEditDate="2020-07-17T07:46:10.067" LastActivityDate="2020-07-17T07:46:10.067" Title="CIFAR-10 can't get above 10% Accuracy with MobileNet, VGG16 and ResNet on Keras" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22537" PostTypeId="1" CreationDate="2020-07-17T07:59:35.873" Score="1" ViewCount="46" Body="&lt;p&gt;What is the reason to train a Neural Network to estimate a task's success (i.e. robotic grasp planning) using a simulator that is based on analytic grasp quality metrics?&lt;/p&gt;&#xA;&lt;p&gt;Isn't a perfectly trained NN going to essentially output the same probability of task success as the analytic grasp quality metrics that were used to train it? What benefits does this NN have with respect to just directly using said analytic grasp quality metrics to determine whether a certain grasp candidate is good or bad? Analytic metrics are by definition deterministic, so I fail to understand the reason for using them to train a NN that will ultimately output the same result.&lt;/p&gt;&#xA;&lt;p&gt;This approach is used in high-caliber works like the &lt;a href=&quot;https://arxiv.org/abs/1703.09312&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dex-Net2&lt;/a&gt; from &lt;a href=&quot;https://berkeleyautomation.github.io/dex-net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berkeley Automation&lt;/a&gt;. I am rather new to the field and the only reason I can think of is computational efficiency in production?&lt;/p&gt;&#xA;" OwnerUserId="38673" LastEditorUserId="38673" LastEditDate="2020-07-18T14:15:38.473" LastActivityDate="2020-07-18T14:15:38.473" Title="Advantages of training Neural Networks based on analytic success criteria" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22539" PostTypeId="1" CreationDate="2020-07-17T08:21:44.240" Score="0" ViewCount="37" Body="&lt;p&gt;I am training a multilabel text neural network and the model metric I chose, to measure the performance of the training and the validation sets, is the f1 score (Micro average, None average). However, I have a hard time trying to increase the value achieved by most of the model estimators. Specifically, the majority of my neural networks achieve an f1 score (micro averaged) between 0.69 - 0.71 points while a simple f1 score around 0.48 - 0.50. However, the loss of the neural network continued to decrease despite the f1 score being stuck around 0.70 units.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if my models fall into a local minimum and this causes the performance metric of the algorithm can no longer increase. Or it's just my imagination and I simply can't get more of 0.70 units of f1 score due to other reasons, like the data, the multi labeled target, etc.&lt;/p&gt;&#xA;&lt;p&gt;My neural network has the following specs:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 dense (hidden layer)&lt;/li&gt;&#xA;&lt;li&gt;32 - 128 batch size (hidden units/neurons)&lt;/li&gt;&#xA;&lt;li&gt;Drop out: 0.1&lt;/li&gt;&#xA;&lt;li&gt;Activation function of the last layer: 'sigmoid'&lt;/li&gt;&#xA;&lt;li&gt;loss metric: 'binary cross-entropy'&lt;/li&gt;&#xA;&lt;li&gt;model metric: [fi_score(average=micro), fi_score(average=None)]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Structure of the neural network&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3P8NT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3P8NT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Training per epoch - As you can see the f1 score (micro) cannot increase more than 0.71 units.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ATZPH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATZPH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;F1 score (None average) per genre tag - You can see that some of my genres cannot achieve a very high f1 score&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xTeje.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xTeje.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What can I do to tackle this? I would really like to increase the f1 score above 0.71 units but it would be helpful to know if this is feasible and the ways to achieve this.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for any proposal and please let me know in the comments if you would like any additional information or you want me to share with you my colab notebook.&lt;/p&gt;&#xA;" OwnerUserId="38530" LastEditorUserId="38530" LastEditDate="2020-07-17T08:50:04.390" LastActivityDate="2020-07-17T08:50:04.390" Title="Micro average f1_score, None average f1_score cannot increase more than 0.71, 0.50 respectively for the best model estimator" Tags="&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22540" PostTypeId="2" ParentId="22533" CreationDate="2020-07-17T08:23:38.417" Score="1" Body="&lt;p&gt;For good or bad, AI is the next step in automation. The impact which is already visible, and trends show will continue in the future, is the eradication of repetitive and body-straining labor.&lt;/p&gt;&#xA;&lt;p&gt;Hopefully, the transformation will be gradual enough for the global labor market to re-adjust, otherwise, we'll face a problem of growing unemployment. It seems to me that we've become aware enough to foresee bad outcomes of our inventions, hence in almost every dimension affected by AI, a plausible and either positive or negative future can be presented, depending on the sentiment of the storyteller.&lt;/p&gt;&#xA;&lt;p&gt;Regardless of what different experts and sci-fi writers tell us about the future, actually predicting it is a futile endeavour. Considering that predictions made for a dynamic system, even when we have a lot of data and good models (like about the weather), become unreliable just for a few weeks ahead.&lt;/p&gt;&#xA;" OwnerUserId="38671" LastEditorUserId="2444" LastEditDate="2020-07-17T11:56:30.247" LastActivityDate="2020-07-17T11:56:30.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22541" PostTypeId="2" ParentId="11949" CreationDate="2020-07-17T09:24:23.853" Score="0" Body="&lt;p&gt;One could imagine using a segmentation network as a first step of processing. Then feeding an area corresponding to a bounding box of each segmented object to the classifier.&lt;/p&gt;&#xA;&lt;p&gt;Potentially that could yield an increase in performance in classifying objects in an image, but not without a cost of training time, sine suddenly there are two networks to train instead of just one.&lt;/p&gt;&#xA;" OwnerUserId="38671" LastActivityDate="2020-07-17T09:24:23.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22542" PostTypeId="5" CreationDate="2020-07-17T12:04:56.977" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-17T12:04:56.977" LastActivityDate="2020-07-17T12:04:56.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22543" PostTypeId="4" CreationDate="2020-07-17T12:04:56.977" Score="0" Body="For questions about the 1d convolution in convolutional neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-17T12:04:56.977" LastActivityDate="2020-07-17T12:04:56.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22544" PostTypeId="2" ParentId="22072" CreationDate="2020-07-17T12:14:23.577" Score="0" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/2005.13362.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Multi-modal Approach to Fine-grained Opinion Mining on Video Reviews&lt;/a&gt; could be useful for your purposes, although I have read only the abstract&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Despite the recent advances in opinion mining for written reviews, few works have tackled the problem on other sources of reviews. In light of this issue, &lt;strong&gt;we propose a multimodal approach for mining fine-grained opinions from video reviews&lt;/strong&gt; that is able to determine the aspects of the item under review that are being discussed and the sentiment orientation towards them. Our approach works at the sentence level without the need for time annotations and uses features derived from the audio, video and language transcriptions of its contents. We evaluate our approach on two datasets and show that leveraging the video and audio modalities consistently provides increased performance over text-only baselines, providing evidence these extra modalities are key in better understanding video reviews.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and looked at some of the diagrams and figures in the paper.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-17T12:14:23.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22545" PostTypeId="1" CreationDate="2020-07-17T12:41:40.930" Score="0" ViewCount="158" Body="&lt;p&gt;I am working on an OCR project and I wonder how I can calculate the confidence score of my OCR system.&lt;/p&gt;&#xA;&lt;p&gt;I have digital multi meter images. There are some measurement results on the screens of devices in images. I want to recognize those values. However, according to my research, I am not sure which OCR confidence calculation technique is fit for my system.&lt;/p&gt;&#xA;&lt;p&gt;As I understand OCR confidence scores can calculate in character, word and sentence wise. Actually, the last two methods build on character confidence scores. In my case, character wise calculation might be wrong or not sufficient.&lt;/p&gt;&#xA;&lt;p&gt;For example, I have &amp;quot;40.245 V&amp;quot; text. I got two different recognition result like &amp;quot;40.247 V&amp;quot; and &amp;quot;70.245 V&amp;quot;. If I am not wrong both of the result will have same or close confidence scores. Yet &amp;quot;40.247 V&amp;quot; prediction is acceptable and &amp;quot;70.245 V&amp;quot; is not acceptable in my case.&lt;/p&gt;&#xA;&lt;p&gt;Is there any idea how to calculate confidence score for this case?&lt;/p&gt;&#xA;" OwnerUserId="37854" LastActivityDate="2020-07-17T12:41:40.930" Title="How to calculate confidence score of OCR system?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22546" PostTypeId="2" ParentId="22530" CreationDate="2020-07-17T12:50:09.103" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Why is the expected return in Reinforcement Learning (RL) computed as a sum of cumulative rewards?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That is the definition of return.&lt;/p&gt;&#xA;&lt;p&gt;In fact when applying a discount factor this should formally be called &lt;em&gt;discounted&lt;/em&gt; return, and not simply &amp;quot;return&amp;quot;. Usually the same symbol is used for both (&lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; in your case, &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; in e.g. Sutton &amp;amp; Barto).&lt;/p&gt;&#xA;&lt;p&gt;There are also other variations, such as &lt;em&gt;truncated&lt;/em&gt; return (sum up to a given time horizon). They all share the feature that a return is a sum of reward values. You cannot really change that and keep the formal term &amp;quot;return&amp;quot;, that's how it has been defined.&lt;/p&gt;&#xA;&lt;p&gt;You can however define the value function to be something other than the expected return. Rather than looking for alternative definitions of return as your title suggests, you could be looking for alternative metrics to use as value functions.&lt;/p&gt;&#xA;&lt;p&gt;You do go on to ask about computing &amp;quot;the value of a state&amp;quot; without mentioning the word &amp;quot;return&amp;quot;, but it is not 100% clear whether you are aware that the way to resolve this is to not use return, but something else.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would it not make more sense to compute the value of a state as the following:&#xA;&lt;span class=&quot;math-container&quot;&gt;$V^π(s)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$(r_{t+1} + γr_{t+2} + (γ^2)_{t+3}, + ...    \mid s)/k = {E}(∑γ^kr_{t+k+1}\mid s)/k $&lt;/span&gt; where k is the number of elements in the sum, thus giving us the average reward for being in state s.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your example would nearly always result in zero for long-running or non-episodic problems, as you are summing a decreasing geometric series possibly up to very large &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, then dividing by the maximum &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. Notation-wise you are also using &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; to be an iterator and the maximum value of the same iterator, that would need fixing.&lt;/p&gt;&#xA;&lt;p&gt;However, this is very close to a real value metric used in reinforcement learning, called the &lt;a href=&quot;https://link.springer.com/article/10.1007/BF00114727&quot; rel=&quot;nofollow noreferrer&quot;&gt;average reward setting&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The expected average reward value function for a non-episodic problem is typically given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V^\pi(s) = \mathbb{E}[\lim_{h \to \infty}\frac{1}{h}\sum_{k=0}^{h}r_{t+k+1}|s_t = s]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note there is no discount factor, it is not usually possible to combine a discount factor with the average reward setting.&lt;/p&gt;&#xA;&lt;p&gt;Sutton &amp;amp; Barto point out in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; chapter 10, section 10.4, that when using function approximation on continuing tasks, then a discount factor is not a useful part of the setting. Instead average reward is a more natural approach. It is also not so different, and quite easy to modify the Bellman equations and update rules. However, many DQN implementations still use discounted return to solve continuing tasks. That is because with high enough discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, e.g. &lt;span class=&quot;math-container&quot;&gt;$0.99$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$0.999$&lt;/span&gt;, then the end result is likely to be the same optimal solution - the discount factor has moved from being part of the problem formulation to being a solution hyperparameter.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-17T16:46:03.407" LastActivityDate="2020-07-17T16:46:03.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22547" PostTypeId="1" CreationDate="2020-07-17T14:02:38.373" Score="2" ViewCount="60" Body="&lt;p&gt;I'm trying to find out what kind of policy improvement and policy evaluation AlphaGo, AlphaGo Zero, and AlphaZero are using. By looking into their respective paper and SI, I can conclude that it is a kind of policy gradient actor-critic approach, where the policy is evaluated by a critic and is improved by and actor. Yet still can't fit it to any of the known policy gradient algorithms.&lt;/p&gt;&#xA;" OwnerUserId="31324" LastActivityDate="2020-07-17T14:02:38.373" Title="What kind of policy evaluation and policy improvement AlphaGo, AlphaGo Zero and AlphaZero are using" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;alphazero&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22548" PostTypeId="1" AcceptedAnswerId="22549" CreationDate="2020-07-17T14:55:25.737" Score="3" ViewCount="294" Body="&lt;p&gt;I am working on OpenAI's &amp;quot;MountainCar-v0&amp;quot; environment.  In this environment, each step that an agent takes returns (among other values) the variable named &lt;code&gt;done&lt;/code&gt; of type boolean. The variable gets a &lt;code&gt;True&lt;/code&gt; value when the episode ends. However, I am not sure how each episode ends. My initial understanding was that an episode should end when the car reaches the flagpost. However, that is not the case.&lt;/p&gt;&#xA;&lt;p&gt;What are the states/actions under which the episode terminates in this environment?&lt;/p&gt;&#xA;" OwnerUserId="36710" LastEditorUserId="36710" LastEditDate="2020-07-18T17:09:59.940" LastActivityDate="2020-07-18T17:09:59.940" Title="How does an episode end in OpenAI Gym's &quot;MountainCar-v0&quot; environment?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;gym&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2021-03-16T13:42:13.703" ContentLicense="CC BY-SA 4.0" />
  <row Id="22549" PostTypeId="2" ParentId="22548" CreationDate="2020-07-17T15:08:00.153" Score="3" Body="&lt;p&gt;The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the &lt;code&gt;_max_episode_steps&lt;/code&gt; attribute of the environment.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-17T15:08:00.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22550" PostTypeId="2" ParentId="22548" CreationDate="2020-07-17T15:45:40.113" Score="3" Body="&lt;p&gt;To answer your question, the specifics of some of the OpenAI Gym environments can be found on their &lt;a href=&quot;https://github.com/openai/gym/wiki/MountainCar-v0&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The episode ends when you reach 0.5 position, or if 200 iterations are reached.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is a deeper question in what you asked, though:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My initial understanding was that an episode should end when the Car reaches the flagpost.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The environment certainly could be set up that way. Limiting the number of steps per episode has the immediate benefit of forcing the agent to reach the goal state in a fixed amount of time, which often results in a speedier trajectory by the agent (MountainCar-v0 further penalizes long trajectories through the reward signal). Also, the underlying learning algorithm may only perform policy updates after completion of the episode. If the agent will never reach the goal state under its current policy (i.e. the policy is very bad, lacks much randomness, etc.), then terminating the episode after a fixed amount of time will ensure that the agent is able to perform a policy update and try a new policy on the next episode (alternatively, the learning algorithm could perform a policy update during the episode). There is a branch of tasks called &lt;em&gt;continuing tasks&lt;/em&gt; which never terminate (see Section 3.3 of &lt;a href=&quot;http://www.incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton and Barto&lt;/a&gt;), so the choice to limit the number of steps per episode is heavily dependent on the task at hand and the choice of learning algorithm.&lt;/p&gt;&#xA;" OwnerUserId="37607" LastActivityDate="2020-07-17T15:45:40.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22551" PostTypeId="1" AcceptedAnswerId="22552" CreationDate="2020-07-17T16:36:05.920" Score="1" ViewCount="81" Body="&lt;p&gt;I'm working on recognizing the numbers 3 and 7 using the MNIST data set. I'm using &lt;code&gt;cnn_learner()&lt;/code&gt; function from &lt;a href=&quot;https://docs.fast.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastai library&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When I plotted the learning rate, the curve started going backward after a certain value on X-axis. Can someone please explain what does it signify?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SxIdz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SxIdz.png&quot; alt=&quot;learning rate graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31799" LastEditorUserId="2444" LastEditDate="2020-07-18T12:16:04.547" LastActivityDate="2020-08-17T15:03:45.470" Title="Why would the learning rate curve go backwards?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;learning-rate&gt;&lt;fast-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22552" PostTypeId="2" ParentId="22551" CreationDate="2020-07-17T17:44:28.310" Score="1" Body="&lt;p&gt;I have not used fastai library but this also happens on tensorboard when you have more than one training being recorded on the same plot.&lt;/p&gt;&#xA;&lt;p&gt;Looking at the picture, I think this is a very special type of graph because for a single LR value you have 2 loss values associated. Put in other words, you have the same LR value for different loss values. My guess it that some time-dependent issue is messing things here.&lt;/p&gt;&#xA;&lt;p&gt;Another intuition that may help to solve the issue is representing data differently. If you rotate the graph 90º left-wise you could see how the LR is evolving with different loss values. LR should be decreasing along with loss value, but in this case it is not like that either. So review how you are setting the LR too!&lt;/p&gt;&#xA;&lt;p&gt;My list of to-check would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Check how you update LR according to the loss value&lt;/li&gt;&#xA;&lt;li&gt;Check the LR recorder instance is only being used by one training at a time&lt;/li&gt;&#xA;&lt;li&gt;Try plotting the value of LR (y) against your loss values (x)&lt;/li&gt;&#xA;&lt;li&gt;Try plotting the value of LR (y) against the current epoch number (x)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Hope it helps&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-07-17T17:44:28.310" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22554" PostTypeId="1" AcceptedAnswerId="22568" CreationDate="2020-07-18T01:13:28.213" Score="2" ViewCount="48" Body="&lt;p&gt;For Keras on TensorFlow, a layer class constructor comes with these:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;kernel_regularizer&lt;/code&gt;=...&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bias_regularizer&lt;/code&gt;=...&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;activity_regularizer&lt;/code&gt;=...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For example, Dense layer:&lt;br/&gt;&#xA;&lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense#arguments_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense#arguments_1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first one, &lt;code&gt;kernel_regularizer&lt;/code&gt; is easy to understand, it regularises weights, makes weights smaller to avoid overfitting on training data only.&lt;/p&gt;&#xA;&lt;p&gt;Is &lt;code&gt;kernel_regularizer&lt;/code&gt; enough? When should I use &lt;code&gt;bias_regularizer&lt;/code&gt; and &lt;code&gt;activity_regularizer&lt;/code&gt; too?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-07-18T12:18:31.177" LastActivityDate="2020-07-19T00:50:58.523" Title="When would bias regularisation and activation regularisation be necessary?" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;regularization&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22555" PostTypeId="2" ParentId="22355" CreationDate="2020-07-18T01:19:14.920" Score="0" Body="&lt;p&gt;From my personal experience, the &lt;code&gt;units&lt;/code&gt; hyperparam in LSTM is not necessary to be the same as max sequence length. Add more units to have the loss curve dive faster.&lt;/p&gt;&#xA;&lt;p&gt;And about the number of LSTM layers, trying out a single LSTM layer is a good start point, the model trains better with more LSTM layers.&lt;/p&gt;&#xA;&lt;p&gt;For example, MAX_SEQ_LEN=10, in Keras:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Lstm1 = LSTM(units=30, return_sequences=True); #Time sequence, to feed to next layer&#xA;Lstm2 = LSTM(units=20, return_sequences=True); #Time sequence, to feed to next layer&#xA;Lstm3 = LSTM(units=10, return_sequences=False);&#xA;&#xA;Output = Lstm3(Lstm2(Lstm1));&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2844" LastActivityDate="2020-07-18T01:19:14.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22556" PostTypeId="2" ParentId="22460" CreationDate="2020-07-18T07:45:02.980" Score="0" Body="&lt;p&gt;Since after a number of iterations the cost function is not reducing, this may be able to be diagnosed as a vanishing gradient problem. A solution to this is the use of a Residual neural network.&lt;/p&gt;&#xA;&lt;p&gt;Another solution is that you carefully initialise your weights as throughout your neural network your gradient may exponentially explode or exponentially vanish.&lt;/p&gt;&#xA;&lt;p&gt;Watch this video on how to initialise weights truly randomly:&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=s2coXdufOzE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=s2coXdufOzE&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Another possible cause for your issue is that your algorithm is having an high bias problem. This is due to your algorithm not performing well on the training set. In your case one of the best solutions would be to make your network deeper and so it shall be able to conduct more complex functions and so perform better on your training set.&lt;/p&gt;&#xA;" OwnerUserId="31949" LastEditorUserId="31949" LastEditDate="2020-07-19T05:06:28.237" LastActivityDate="2020-07-19T05:06:28.237" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22558" PostTypeId="1" CreationDate="2020-07-18T09:52:16.273" Score="2" ViewCount="259" Body="&lt;p&gt;I understand that, in tree search, an admissible heuristic implies that &lt;span class=&quot;math-container&quot;&gt;$A*$&lt;/span&gt; is optimal. The intuitive way I think about this is as follows:&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; be two costs from any respective nodes &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; to the goal. Assume &lt;span class=&quot;math-container&quot;&gt;$P&amp;lt;Q$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$P'$&lt;/span&gt; be an estimation of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$P'\le P \Rightarrow P'&amp;lt;Q$&lt;/span&gt;. It follows from uniform-cost-search that the path through &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; must be explored.&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand, is why the idea of an admissible heuristic does not apply as well to &amp;quot;graph-search&amp;quot;. If a heuristic is admissible but inconsistent, would that imply that &lt;span class=&quot;math-container&quot;&gt;$A*$&lt;/span&gt; is not optimal? Could you please provide an example of an admissible heuristic that results in a non-optimal solution?&lt;/p&gt;&#xA;" OwnerUserId="25671" LastEditorUserId="2444" LastEditDate="2020-07-18T19:54:43.970" LastActivityDate="2020-11-08T16:43:15.040" Title="Is A* with an admissible but inconsistent heuristic optimal?" Tags="&lt;search&gt;&lt;heuristics&gt;&lt;a-star&gt;&lt;admissible-heuristic&gt;&lt;consistent-heuristic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22559" PostTypeId="1" CreationDate="2020-07-18T10:08:58.977" Score="0" ViewCount="31" Body="&lt;p&gt;I'm curious if there is current research or a common classification model for determining whether an image is a photo (as in taken with a camera) or something else (such as a vector, screenshot, or other media).&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to filter out non-photos from photos from a large dataset and curious if there is something that already exists or is commonly used in this space. You may assume for this problem that no common metadata would able to obviate this need.&lt;/p&gt;&#xA;" OwnerUserId="38694" LastActivityDate="2020-07-18T10:08:58.977" Title="Is there a developed model for image classification of photo or not photo?" Tags="&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22560" PostTypeId="1" CreationDate="2020-07-18T11:44:28.267" Score="0" ViewCount="59" Body="&lt;p&gt;I have the following problem.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The following descriptions best describe what:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Value that has to be assigned manually.&lt;/li&gt;&#xA;&lt;li&gt;The K value in K-nearest neighbor is an example of this.&lt;/li&gt;&#xA;&lt;li&gt;Value is set before the training.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The options are&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Model Parameter&lt;/li&gt;&#xA;&lt;li&gt;Centroid&lt;/li&gt;&#xA;&lt;li&gt;Model hyperparameter&lt;/li&gt;&#xA;&lt;li&gt;Significance&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Which of the above is the solution?&lt;/p&gt;&#xA;" OwnerUserId="38695" LastEditorUserId="2444" LastEditDate="2020-07-18T12:26:34.093" LastActivityDate="2020-07-18T12:26:34.093" Title="What is the thing that is assigned manually, set before training and is equal to K in K-NN?" Tags="&lt;machine-learning&gt;&lt;homework&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22561" PostTypeId="1" CreationDate="2020-07-18T13:27:17.247" Score="4" ViewCount="430" Body="&lt;p&gt;Is there an upper limit to the maximum cumulative reward in a deep reinforcement learning problem? For example you want to train an DQN agent in an environment and you want to know what is the highest possible value you can get from the cumulative reward, so you can compare this with your agents performance.&lt;/p&gt;&#xA;" OwnerUserId="38696" LastActivityDate="2020-07-19T09:47:42.777" Title="Upper limit to the maximum cumulative reward in a deep reinforcement learning problem" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;rewards&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22563" PostTypeId="1" CreationDate="2020-07-18T16:42:24.300" Score="1" ViewCount="66" Body="&lt;p&gt;I have a question about implementing policy gradient methods for problems with continuous action spaces.&lt;/p&gt;&#xA;&lt;p&gt;Assume that actions are sampled from a diagonal Gaussian distribution with mean vector &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and standard deviation vector &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;. As far as I understand, we can define a neural network that takes the current state as the input and returns a &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; as its output. According to &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Spinning Up&lt;/a&gt;, the standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; can be represented in two different ways:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xkgln.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xkgln.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't completely understand the first method. Does it mean that we must set the log standard deviations to fix numbers? Then how do we choose these numbers?&lt;/p&gt;&#xA;" OwnerUserId="38699" LastEditorUserId="38700" LastEditDate="2020-07-19T01:00:43.953" LastActivityDate="2020-07-19T01:00:43.953" Title="In continuous action spaces, how is the standard deviation, associated with Gaussian distribution from which actions are sampled, represented?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22564" PostTypeId="1" CreationDate="2020-07-18T19:32:53.513" Score="0" ViewCount="275" Body="&lt;p&gt;I am learning with the OpenAI gym's cart pole environment.&lt;/p&gt;&#xA;&lt;p&gt;I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [&lt;span class=&quot;math-container&quot;&gt;$&#xA;-\infty, \infty$&lt;/span&gt;] to some finite upper and lower limits. (By the way, these states are velocity and pole velocity at the tip).&lt;/p&gt;&#xA;&lt;p&gt;How can I change these limits in the actual gym's environment?&#xA;Any other suggestions are also welcome.&lt;/p&gt;&#xA;" OwnerUserId="36710" LastActivityDate="2020-07-19T10:22:12.727" Title="How can I change observation states' values in OpenAI gym's cartpole environment?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22565" PostTypeId="2" ParentId="22561" CreationDate="2020-07-18T19:39:32.353" Score="2" Body="&lt;p&gt;In any reinforcement learning problem, not just Deep RL, then there is an upper bound for the cumulative reward, provided that the problem is episodic and not continuing.&lt;/p&gt;&#xA;&lt;p&gt;If the problem is episodic and the rewards are designed such that the problem has a natural ending, i.e. the episode will end regardless of how well the agent does in the environment, then then you could work it out by calculating the max possible reward in each step of the episode; however this is potentially non-trivial depending on your environment.&lt;/p&gt;&#xA;&lt;p&gt;For an example in a trivial setting, however, imagine the problem of cartpole -- I could define the MDP to have a reward of +1 for every time step that the agent is able to balance the pole upright, and 0 when the pole falls. If I also defined that the problem terminates after 200 time steps then the upper bound on cumulative rewards for this problem would be 200.&lt;/p&gt;&#xA;&lt;p&gt;In general, if the problem is continuing then in theory the problem goes on infinitely and so there is no upper bound, as the episode never ends -- this is partly why we use the discount factor, to ensure that &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0} \gamma^k R_{t+k}$&lt;/span&gt; converges.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-07-19T09:47:42.777" LastActivityDate="2020-07-19T09:47:42.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22566" PostTypeId="2" ParentId="22561" CreationDate="2020-07-18T20:48:30.583" Score="2" Body="&lt;p&gt;My answer to:&lt;code&gt;Is there an upper limit to the maximum cumulative reward in a deep reinforcement learning problem?&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yes but depending on the environment, if dealing with the theoretical environment, where there are infinite number of time steps.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Calculating the upper bound&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In reinforcement learning (deep RL inclusive), we want to maximize the discounted cumulative reward i.e. Find the upper bound of:  &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^\infty \gamma^kR_{t+k+1}, where$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$[0, 1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Before we find the upper bound of the series above, we need to find out if the upper bound exists i.e. whether it converges according to the environment specifications such as the reward function.&lt;/p&gt;&#xA;&lt;p&gt;I will provide one example environment where the series converges. It is an environment that has simple rules and goes on for infinite time steps. It's reward function definition is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-&amp;gt; A reward of +2 for every favorable action.&#xA;&#xA;-&amp;gt; A reward of 0 for every unfavorable action.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, our path through the MDP that gives us the upper bound is where we only get 2's.&lt;/p&gt;&#xA;&lt;p&gt;Let's say &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is a constant, example &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.5$&lt;/span&gt;, note that &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$[0, 1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, we have a geometric series which converges:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^\infty \gamma^kR_{t+k+1}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=1}^\infty (1)(2\gamma^{k-1})$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=1}^\infty 2\gamma^{k-1}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\frac{2}{1 - 0.5}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Thus the upper bound is 4.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;For environments that go on for a finite number of time steps the the upper bound does exist but for certain environments, likewise for the infinite time step environments, it may be a bit difficult to calculate but not necessarily impossible, the environments I speak of are ones with complicated reward functions and environments i.e. the environments are stochastic or the reward function's possible values are dependent on the state, they always are but we can loosely say that a reward function is independent of state when all possible reward values for an environment can be given in any state, obviously with regards to the actions taken though.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="30174" LastEditorUserId="30174" LastEditDate="2020-07-18T21:07:36.207" LastActivityDate="2020-07-18T21:07:36.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22567" PostTypeId="2" ParentId="22561" CreationDate="2020-07-18T21:24:13.673" Score="2" Body="&lt;p&gt;Lets assume &lt;span class=&quot;math-container&quot;&gt;$\sup_{s,a} r(s,a)&amp;lt;b$&lt;/span&gt;. Then for continuing problems the upper bound can be obtained by&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\sum_{t=0}^{\infty} \gamma^{t}r(s_t,a_t) &#xA;&amp;amp;\le \sum_{t=0}^{\infty} \gamma^{t} \sup_{s,a}r(s,a) \nonumber \\&#xA;&amp;amp;=\sum_{t=0}^{\infty} \gamma^{t} b = \frac{b}{1-\gamma}.&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can use the same bound for episodic tasks with discounted return.&#xA;For episodic tasks without discounting (&lt;span class=&quot;math-container&quot;&gt;$\gamma=1$&lt;/span&gt;) the above sum goes to infinity. However, if we know the episode length &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, we can use &lt;span class=&quot;math-container&quot;&gt;$Tb$&lt;/span&gt; as an upper bound.&lt;/p&gt;&#xA;" OwnerUserId="38700" LastActivityDate="2020-07-18T21:24:13.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22568" PostTypeId="2" ParentId="22554" CreationDate="2020-07-19T00:50:58.523" Score="2" Body="&lt;p&gt;Regularizer's are used as a means to combat over fitting.They essentially create a cost function penalty which tries to prevent quantities from becoming to large.  I have primarily used kernel regularizers. First I try to control over fitting using dropout layers. If that does not do the job or leads to poor training accuracy I try the Kernel regularizer. I usually stop at that point. I think activity regularization would be my next option to prevent outputs from becoming to large. I suspect weight regularization effectively can pretty much achieve the same result.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-07-19T00:50:58.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22569" PostTypeId="1" CreationDate="2020-07-18T21:21:23.330" Score="1" ViewCount="149" Body="&lt;p&gt;I have recently been exposed to the concept of decentralized applications,&#xA;I know that neural networks require a lot of parallel computing infra for training.&lt;/p&gt;&#xA;&lt;p&gt;What are the technical difficulties one may face for training neural networks in a p2p manner?&lt;/p&gt;&#xA;" OwnerDisplayName="ram bharadwaj" LastEditorUserId="2444" LastEditDate="2020-07-19T12:21:38.683" LastActivityDate="2020-07-21T11:37:51.087" Title="Why can't we train neural networks in a peer-to-peer manner?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;distributed-computing&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22570" PostTypeId="1" CreationDate="2020-07-19T08:50:47.477" Score="0" ViewCount="36" Body="&lt;p&gt;I want to use the DQN algorithm from &lt;a href=&quot;https://www.tensorflow.org/agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow Agent API&lt;/a&gt; for a custom environment. In my environment, the agent's observations are image sequences.&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, images should be encoded by a neural network before they are used. Thereafter, the DQN network is able to produce action. I wonder how can I do this with TensorFlow Agent API.&lt;/p&gt;&#xA;&lt;p&gt;Besides, I know that I have to override some function in order to build an environment in a custom fashion based on this &lt;a href=&quot;https://towardsdatascience.com/creating-a-custom-environment-for-tensorflow-agent-tic-tac-toe-example-b66902f73059&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;All the examples that I have seen in TensorFlow Agent API have discrete observation space. For example, in the &lt;strong&gt;CartPole&lt;/strong&gt; environment, observation space consists of two vectors describing velocity and angle.&lt;/p&gt;&#xA;" OwnerUserId="38541" LastEditorUserId="2444" LastEditDate="2020-07-20T12:09:01.327" LastActivityDate="2020-07-20T12:09:01.327" Title="How to define image as observation in tensorflow_agent?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22571" PostTypeId="2" ParentId="22564" CreationDate="2020-07-19T10:22:12.727" Score="1" Body="&lt;p&gt;I don't recommend changing the rules of the environment.&lt;/p&gt;&#xA;&lt;p&gt;What you could do:&lt;/p&gt;&#xA;&lt;p&gt;Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation.&lt;/p&gt;&#xA;&lt;p&gt;e.g. Say I do have a continuous state space with one variable in range &lt;span class=&quot;math-container&quot;&gt;$[-\infty,\infty]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The buckets can be as follows:&lt;/p&gt;&#xA;&lt;p&gt;0). x &amp;lt; -1000&lt;/p&gt;&#xA;&lt;p&gt;1). -1000 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; -500&lt;/p&gt;&#xA;&lt;p&gt;2). -500 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; -100&lt;/p&gt;&#xA;&lt;p&gt;3). -100 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; -50&lt;/p&gt;&#xA;&lt;p&gt;4). -50 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; 0&lt;/p&gt;&#xA;&lt;p&gt;5). 0 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; 50&lt;/p&gt;&#xA;&lt;p&gt;6). 50 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; 100&lt;/p&gt;&#xA;&lt;p&gt;7). 100 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; 500&lt;/p&gt;&#xA;&lt;p&gt;8). 500 &lt;span class=&quot;math-container&quot;&gt;$\le$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;$&lt;/span&gt; 1000&lt;/p&gt;&#xA;&lt;p&gt;9). x &amp;gt; 1000&lt;/p&gt;&#xA;&lt;p&gt;Therefore in this example scenario there are 9 buckets. Hence, the observations can be in range [0, 9] discretely.&lt;/p&gt;&#xA;" OwnerUserId="30174" LastActivityDate="2020-07-19T10:22:12.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22572" PostTypeId="1" CreationDate="2020-07-19T10:43:38.220" Score="1" ViewCount="58" Body="&lt;p&gt;Is there a way to get landmark features automatically learned by a neural network without having to manually pre-label them in the images that are being fed into the network?&lt;/p&gt;&#xA;" OwnerUserId="38711" LastEditorUserId="2444" LastEditDate="2020-07-19T12:29:52.717" LastActivityDate="2020-08-19T11:03:15.333" Title="Is there a way to get landmark features automatically learned by a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;self-supervised-learning&gt;&lt;automated-machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22573" PostTypeId="2" ParentId="7182" CreationDate="2020-07-19T10:46:27.660" Score="0" Body="&lt;p&gt;You could say that the class of the data (e.g. spam vs not spam) is a hidden quality that can be inferred through the observable features (e.g. message subject contains &amp;quot;bitcoin&amp;quot;),&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C \; | \; F)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which says that the probability of the class &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is conditioned on the visibility of the feature &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using Bayes' theorem we can write&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C \; | \; F) = \frac{P(F \; | \; C) \, P(C)}{P(F)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your problem is that you are not taking into account the &lt;strong&gt;evidence&lt;/strong&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(F)$&lt;/span&gt;, denominator in this expression.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3ujvJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3ujvJ.jpg&quot; alt=&quot;Bayes' theorem label factors&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can have multiple features in your &lt;strong&gt;likelihood&lt;/strong&gt; function&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(F \; | \; C) = P(F_1,F_2,F_3,\ldots \; | \; C)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and these can be independent&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\prod_k P(F_k \; | \; C)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and then you have your &lt;strong&gt;prior&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C) \prod_k P(F_k \; | \; C)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;but what you are missing is the &lt;strong&gt;evidence&lt;/strong&gt; factor in the denominator&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C \; | \; F) = \frac{P(C) \prod_k P(F_k \; | \; C)}{\sum_C \scriptsize{P(C) \prod_k P(F_k \; | \; C)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the general view of the problem. Specifically, to fix your issue, evaluate the probabilities for all your classes, add them together, then divide each probability by that sum.&lt;/p&gt;&#xA;&lt;p&gt;For instance, if you have two classes and the probabilities are &lt;span class=&quot;math-container&quot;&gt;$p_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt;, then write&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C=0) = \frac{p_0}{p_0 + p_1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C=1) = \frac{p_1}{p_0 + p_1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which guarantees that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(C=0) + P(C=1) = 1$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="38712" LastActivityDate="2020-07-19T10:46:27.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22574" PostTypeId="1" CreationDate="2020-07-19T11:02:34.883" Score="1" ViewCount="69" Body="&lt;p&gt;Is there a place where people can share (or buy) ready made neural networks instead of creating them themselves?&#xA;Something like a Wikipedia for DNNs?&lt;/p&gt;&#xA;" OwnerUserId="38713" LastEditorUserId="16565" LastEditDate="2020-07-19T12:03:03.283" LastActivityDate="2020-07-19T15:01:43.290" Title="Is there a place where people can share (or buy) ready made neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22575" PostTypeId="2" ParentId="22574" CreationDate="2020-07-19T11:33:09.550" Score="2" Body="&lt;p&gt;It's not a specific web for sell or share neural network models, but actually you can easily find other people models in Github! Just search it! For example, this is a random repo I've found for &lt;a href=&quot;https://github.com/batogov/cat-breeds-classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cat Classification&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But.. the problem is everyone have different problems. So you can't easily use other people neural network models and then use it for your problem. That's why there's a &amp;quot;trick&amp;quot; called &lt;a href=&quot;https://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfer learning or fine tuning&lt;/a&gt;. That method show you how to train other people models (it's called: pre-trained models) from different problems for your specific problem.&lt;/p&gt;&#xA;&lt;p&gt;Tensorflow and PyTorch provide a lot of pre-trained model for common problems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pytorch.org/docs/stable/torchvision/models.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch Image Classification Pre-Trained models&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/tensorflow/models/blob/r1.12.0/research/object_detection/g3doc/detection_model_zoo.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow Object Detection Model Zoo&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/facebookresearch/detectron2/blob/master/MODEL_ZOO.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch Detectron Model Zoo&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You can also find other pre-trained models in Github! This is &lt;a href=&quot;https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;an official PyTorch tutorial&lt;/a&gt; how to use that pre-trained model to your specific case.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2020-07-19T15:01:43.290" LastActivityDate="2020-07-19T15:01:43.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22576" PostTypeId="1" CreationDate="2020-07-19T14:42:16.277" Score="0" ViewCount="35" Body="&lt;p&gt;I'm training a sklearn.neural_network.mlpregressor by a large data of students performance (an excel file with 740 students and 27 columns that are their qualities) and I want to predict their grades. I'm using mlpregressor with these parameters:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mlp = MLPRegressor(activation='relu',&#xA;                       hidden_layer_sizes=(128, 64, 32), learning_rate='constant',&#xA;                       learning_rate_init=0.01, max_iter=50000, shuffle=True,&#xA;                       solver='adam')&#xA;    mlp.fit(train_data, train_target)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I'm getting 0.6 score on training set and 0.3 on test set.&#xA;How do you think i should change the parameters to get a better score? is mlp a good network to do this?&lt;/p&gt;&#xA;" OwnerUserId="37393" LastActivityDate="2020-07-19T14:42:16.277" Title="Problem fitting data using mlpregressor" Tags="&lt;python&gt;&lt;regression&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22577" PostTypeId="2" ParentId="21743" CreationDate="2020-07-19T14:58:25.713" Score="1" Body="&lt;p&gt;After diving deeper into the material I am able to answer my own question:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Simulated Annealing&lt;/em&gt; tries to optimize a energy (cost) function by &lt;em&gt;stochastically&lt;/em&gt; searching for minima at different temparatures via a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Chain Monte Carlo&lt;/a&gt; method. The stochasticity comes from the fact that we always accept a new state &lt;span class=&quot;math-container&quot;&gt;$c'$&lt;/span&gt; with lower energy (&lt;span class=&quot;math-container&quot;&gt;$\Delta E &amp;lt; 0$&lt;/span&gt;), but a new state with higher energy (&lt;span class=&quot;math-container&quot;&gt;$\Delta E &amp;gt; 0$&lt;/span&gt;) only with a certain probability&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(c \to c') = \text{min}\{1, \exp(-\frac{\Delta E}{T}) \},$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta E = E(c') - E(c).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where we used the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_measure&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gibbs distribution&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$p(c) = \frac{1}{Z}\text{exp}(\frac{-E(c)}{T})$&lt;/span&gt; to calculate probabilities for each state, with &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; being the partition sum. The temperature &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; plays the role of a scaling factor for the probability distribution. If &lt;span class=&quot;math-container&quot;&gt;$T \to \infty $&lt;/span&gt; we have a uniform distribution and all states are equally possible. If &lt;span class=&quot;math-container&quot;&gt;$T \to 0$&lt;/span&gt; we have a Dirac delta function around the global optimum. By starting with a high &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, sampling states and gradually decreasing it, we can make sure to sample enough states from the state space and accepting energetic higher states in order to escape local minima on the way to the global optimum. After sampling long enough while slowly decreasing the temperature, we theoretically arrive at the global optimum.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Deterministic Annealing&lt;/em&gt; on the other hand directly minimizes the &lt;em&gt;free energy&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$F(T)$&lt;/span&gt; of the system deterministically at each temperature, e.g. by Expectation-Maximization (&lt;a href=&quot;https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;EM-algorithm&lt;/a&gt;). The intuition behind it is that we like to find an optimum at a high temperature (where it is easier to find one because there are fewer local minima), accept this as intermediate solution, lower the temperature, thus scaling the cost function such it is more peaked around it's optima (making optimization a bit more difficult) and start deterministically looking for an optimum again. This is repeated until the temperature is low enough and we (hopefully) found a global solution to our problem. Major drawback is that there is no guarantee to arrive at a global optimum in contrast to simulated annealing. The whole idea of scaling the energy function is based on the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Homotopy&quot; rel=&quot;nofollow noreferrer&quot;&gt;homotopy&lt;/a&gt;: &lt;em&gt;&amp;quot;Two continuous functions [...] can be &amp;quot;continuously deformed&amp;quot; into each other.&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-07-19T14:58:25.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22578" PostTypeId="1" CreationDate="2020-07-19T16:06:02.997" Score="0" ViewCount="39" Body="&lt;p&gt;Recently I finished a project which combined stm32f030r8 microcontroller and MPU9250 sensor to create a system which would detect orientation on all 3 axis using a combination of accel data, gyro data and magnetometer data.&lt;/p&gt;&#xA;&lt;p&gt;Now, I got an idea to expand the project by building a neural network and training it using Python on a PC and then transfer the trained neural network onto a microcontroller in order to be PC-independent and actually be able to recognize the written stuff on it's own. I am still contemplating how to build a database from the microcontroller data and what the actual neural network structure should be.&lt;/p&gt;&#xA;&lt;p&gt;My first thought was to build a neural network which would have n inputs (since I'm still figuring out what the database should look like, I don't know n), 5-10 hidden single-layer neurons and an output for each letter and number so that I can recognize for example letter 'A' as as 100...0.&lt;/p&gt;&#xA;&lt;p&gt;Is this idea feasible on a microcontroller since I'm not actually training the neural network on it? What do you think of the idea so far and could you suggest some sources to read upon, or some courses to help me with this?&lt;/p&gt;&#xA;&lt;p&gt;Cross-posted to EE&lt;/p&gt;&#xA;" OwnerUserId="38716" LastEditorUserId="30178" LastEditDate="2020-07-19T18:22:13.967" LastActivityDate="2020-07-19T18:22:13.967" Title="Using a neural network in a microcontroller to recognize air-written letters and numbers" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;object-recognition&gt;&lt;feedforward-neural-network&gt;&lt;neurons&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22579" PostTypeId="2" ParentId="14098" CreationDate="2020-07-19T17:57:52.073" Score="0" Body="&lt;p&gt;Check this page out, it describes how to develop apply triplet loss to a network: &lt;a href=&quot;https://towardsdatascience.com/image-similarity-using-triplet-loss-3744c0f67973&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/image-similarity-using-triplet-loss-3744c0f67973&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38711" LastActivityDate="2020-07-19T17:57:52.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22580" PostTypeId="1" CreationDate="2020-07-19T18:46:19.693" Score="2" ViewCount="69" Body="&lt;p&gt;I'm currently implementing the NEAT algorithm. But problems occur when testing it with problems which don't have a linear solution(for example xor). My xor only produces 3 correct outputs once at a time:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1, 0 -&amp;gt; 0.99&#xA;0, 0 -&amp;gt; 0&#xA;1, 1 -&amp;gt; 0&#xA;0, 1 -&amp;gt; 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My genome class works fine, so I guess that the problem occurs on breeding or that my config is wrong.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Config&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;const size_t population_size = 150;&#xA;const size_t inputs = 3 (2 inputs + bias);&#xA;const size_t outputs = 1;&#xA;double compatibility_threshold = 3;&#xA;double steps = 0.01;&#xA;double perturb_weight = 0.9;&#xA;double mutate_connection = 0.05;&#xA;double mutate_node = 0.03;&#xA;double mutate_weights = 0.8;&#xA;double mutate_disable = 0.1;&#xA;double mutate_enable = 0.2;&#xA;double c_excess = 1;&#xA;double c_disjoint = 1;&#xA;double c_weight = 0.4;&#xA;double crossover_chance = 0.75;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone has an idea what the problem might be? I proof read my code multiple times, but wasnt able the figure it out.&lt;/p&gt;&#xA;&lt;p&gt;Here is the github link to my code(not documented): &lt;a href=&quot;https://github.com/Creepsy/neat&quot; rel=&quot;nofollow noreferrer&quot;&gt;click&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38720" LastEditorUserId="38720" LastEditDate="2020-07-20T05:25:14.093" LastActivityDate="2020-07-20T05:25:14.093" Title="NEAT can't solve XOR completely" Tags="&lt;machine-learning&gt;&lt;evolutionary-algorithms&gt;&lt;unsupervised-learning&gt;&lt;neat&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22581" PostTypeId="1" CreationDate="2020-07-19T19:23:11.663" Score="4" ViewCount="210" Body="&lt;p&gt;I've been hearing a lot about &lt;a href=&quot;https://arxiv.org/abs/2005.14165&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT-3 by OpenAI&lt;/a&gt;, and that it's a simple to use API with text in text out and has a big neural network off 175B parameters.&lt;/p&gt;&#xA;&lt;p&gt;But how did they achieve this huge number of parameters, and why is it being predicted as one of the greatest innovations?&lt;/p&gt;&#xA;" OwnerUserId="38719" LastEditorUserId="2444" LastEditDate="2020-09-14T13:34:25.570" LastActivityDate="2020-09-14T13:34:25.570" Title="Why is GPT-3 such a game changer?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22583" PostTypeId="1" CreationDate="2020-07-19T23:59:41.170" Score="0" ViewCount="48" Body="&lt;p&gt;My dear friends,&lt;/p&gt;&#xA;&lt;p&gt;In the past couple of years I read a lot about AI with JS and some libraries like TensorFlow. I have great interest in the subject but never used it on a serious project. However, after struggling a lot with linear regression to solve an optimization problem I have, I think that finally I will get much better results, with greater performance, using AI. I work for 12 years with web development and lots of server side, but never worked with any AI library, so please, have a little patience with me if I say something stupid!&lt;/p&gt;&#xA;&lt;p&gt;My problem is this: every user that visits our platform (website) we save the Hour, Day of the week, if the device requesting the page was a smartphone or computer... and such of the FIRST access the user made. If the user keeps visiting other pages, we dont care, we only save the data of the FIRST visit. And if the user anytime does something that we consider a conversion, we assign that conversion to the record of the first access that user made. So we have almost 3 millions of lines like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SESSION     HOUR    DAY_WEEK    DEVICE       CONVERSION&#xA;9847        7       MONDAY      SMARTPHONE   NO&#xA;2233        13      TUESDAY     COMPUTER     YES&#xA;5543        19      SUNDAY      COMPUTER     YES&#xA;3721        8       FRIDAY      SMARTPHONE   NO&#xA;1849        12      SUNDAY      COMPUTER     NO&#xA;6382        0       MONDAY      SMARTPHONE   YES&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I would like to do is this: next time a user visits our platform, we wanna know the probability of that user making a conversion. If a user access now, our website, depending on their device, day of week, hour... we wanna know the probability of that user making a future conversion. With that, we can show very specific messages to the user while he is using our platform and a different price model according to that probability.&lt;/p&gt;&#xA;&lt;p&gt;CURRENTLY we are using a liner regression, and it predicts if the user will make a conversion with an accuracy of around 30%. It's pretty low but so far, it's the best we got it, and this linear regression generates almost 18% increase in conversions when we use it to show specific messages/prices to that specific user compaired to when we dont use it. SO, with a 30% accuracy our linear regression already provides 18% better conversions (and with that, higher revenues and so on).&lt;/p&gt;&#xA;&lt;p&gt;In case you are curious, our linear regression model works like this: we generate a linear equation to every first user access on our system with variables that our system tries to find in order to minimize error &lt;code&gt;sqr(expected value - value)&lt;/code&gt;. Using the data above, our model would generate these equations below (SUNDAY = 0, MONDAY = 1...COMPUTER = 0, SMARTPHONE = 1... CONVERSION YES = 1 and NO = 0)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A*7 + B*1 + C*1 = 0&#xA;A*13 + B*2 + C*0 = 1&#xA;A*19 + B*0 + C*0 = 1&#xA;A*8 + B*6 + C*1 = 0&#xA;A*12 + B*0 + C*1 = 0&#xA;A*0 + B*1 + C*0 = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, our system find the best A, B and C that generates the minimizes error. How can we do that with AI? If possible, it would be nice if we could use TensorFlow or anything with JS! I know there are several AI models, and I have no idea which one would best fit what we need!&lt;/p&gt;&#xA;" OwnerUserId="38724" LastActivityDate="2020-07-19T23:59:41.170" Title="Predict probability of user making a conversion" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;&lt;linear-regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22585" PostTypeId="2" ParentId="1987" CreationDate="2020-07-20T03:31:27.763" Score="0" Body="&lt;p&gt;You can increase no of hidden layers. Following is an example (But not very efficient)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W3TPj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W3TPj.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38728" LastActivityDate="2020-07-20T03:31:27.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22586" PostTypeId="1" CreationDate="2020-07-20T03:49:35.500" Score="1" ViewCount="65" Body="&lt;p&gt;I am trying to implement a paper on Image tempering detection and localization, the paper is &lt;a href=&quot;https://ieeexplore.ieee.org/document/8554259&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Manipulation Detection and Localization Based on the Dual-Domain Convolutional Neural Networks&lt;/a&gt;, I was able to implement the SCNN, the one surrounded by red dots, I could not quite understand the FCNN, the one that is surrounded with blue dots.&lt;/p&gt;&#xA;&lt;p&gt;The problem I am facing is: How the network made features vector from (1048 x 100) to (523 x 100) through max-pooling (instead of 524 x 100), and from (523 x 100) to (260 x 100) and then (260 x 100) to (256, ).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dz3hl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dz3hl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It appears that the given network diagram might be wrong, but, if it is wrong, how could it be published in IEEE. Please, help me understand how the FCNN is constructed.&lt;/p&gt;&#xA;" OwnerUserId="38727" LastEditorUserId="2444" LastEditDate="2020-07-20T12:01:11.683" LastActivityDate="2020-07-20T12:01:11.683" Title="How can the FCNN reduce the dimensions of the input from $1048 \times 100$ to $523 \times 100$ with max-pooling?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;&lt;convolutional-layers&gt;&lt;pooling&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22587" PostTypeId="1" CreationDate="2020-07-20T05:13:45.203" Score="1" ViewCount="39" Body="&lt;p&gt;I'm looking at some baseline implementations of RL agents on the &lt;a href=&quot;https://gym.openai.com/envs/Pendulum-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pendulum&lt;/a&gt; environment. My guess was to use a relatively small neural net (~100 parameters).&lt;/p&gt;&#xA;&lt;p&gt;I'm comparing my solution with some baselines, e.g. the top entry on the &lt;a href=&quot;https://github.com/openai/gym/wiki/Leaderboard#pendulum-v0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pendulum leaderboard&lt;/a&gt;. The models for these solutions are typically &lt;strong&gt;huge&lt;/strong&gt;, i.e. ~120k parameters. What's more, they use very large replay buffers as well, like ~1M transitions. Such model sizes seem warranted for Atari-like environments, but for something as small as the Pendulum, this seems like complete overkill to me.&lt;/p&gt;&#xA;&lt;p&gt;Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?&lt;/p&gt;&#xA;" OwnerUserId="37751" LastEditorUserId="2444" LastEditDate="2020-07-20T11:57:53.260" LastActivityDate="2020-08-19T22:07:00.690" Title="Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?" Tags="&lt;reinforcement-learning&gt;&lt;architecture&gt;&lt;environment&gt;&lt;ddpg&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22588" PostTypeId="2" ParentId="22587" CreationDate="2020-07-20T05:20:32.963" Score="1" Body="&lt;p&gt;Actually, I just started inspecting the entries further down in the leaderboard list, and there are in fact more modest architectures, e.g. &lt;a href=&quot;https://gist.github.com/heerad/1983d50c6657a55298b67e69a2ceeb44#file-ddpg-pendulum-v0-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;, which uses 3 hidden layers with 8 units each.&lt;/p&gt;&#xA;" OwnerUserId="37751" LastEditorUserId="2444" LastEditDate="2020-07-20T21:16:14.750" LastActivityDate="2020-07-20T21:16:14.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22589" PostTypeId="1" AcceptedAnswerId="22601" CreationDate="2020-07-20T07:07:19.860" Score="5" ViewCount="243" Body="&lt;p&gt;Everybody is implementing and using DNN with, for example, TensorFlow or PyTorch.&lt;/p&gt;&#xA;&lt;p&gt;I thought IBM's Deep Blue was an ANN-based AI system, but &lt;a href=&quot;https://analyticsindiamag.com/understanding-difference-symbolic-ai-non-symbolic-ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; says that IBM's Deep Blue was symbolic AI.&lt;/p&gt;&#xA;&lt;p&gt;Are there any special features in symbolic AI that explain why it was used (instead of ANN) by IBM's Deep Blue?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-07-21T18:41:45.940" LastActivityDate="2020-07-28T00:30:12.330" Title="Why is symbolic AI not so popular as ANN but used by IBM's Deep Blue?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;&lt;symbolic-ai&gt;&lt;deep-blue&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22590" PostTypeId="1" CreationDate="2020-07-20T07:36:02.887" Score="1" ViewCount="41" Body="&lt;p&gt;Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say &amp;quot;&lt;em&gt;Price of a ticket&lt;/em&gt;&amp;quot; good. More features can be then be engineered to explain the target variable even better.&lt;/p&gt;&#xA;&lt;p&gt;I have heard people say, that there is no need to hand-engineer features when working with CNNs or RNNs or Deep Neural Networks, provided all the advancements in AI and computation. So, my question is, how would one know, before training, why a particular architecture worked(or would work) when it did or why it didn't when the performance isn't acceptable or very bad. And also that not all of us would have the time to try out all possible architectures, how can one know or at least be sure that something would work for the problem in hand. Or to say, what are the things one needs to follow when designing an architecture to train for a problem, to ensure that an architecture will work?&lt;/p&gt;&#xA;" OwnerUserId="38060" LastEditorUserId="2444" LastEditDate="2020-07-20T11:43:54.233" LastActivityDate="2020-07-20T11:43:54.233" Title="How can one be sure that a particular neural network architecture would work?" Tags="&lt;convolutional-neural-networks&gt;&lt;ai-design&gt;&lt;recurrent-neural-networks&gt;&lt;deep-neural-networks&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22591" PostTypeId="1" AcceptedAnswerId="23443" CreationDate="2020-07-20T09:53:50.560" Score="4" ViewCount="362" Body="&lt;p&gt;I'm creating a CNN network without other frameworks such as PyTorch, Keras, Tensorflow, and so on.&lt;/p&gt;&#xA;&lt;p&gt;During the forward pass, the &lt;code&gt;Flatten&lt;/code&gt; layer reshapes the previous layer's activation.  I know there are a lot of questions about it, but what should I do with the &lt;code&gt;Flatten&lt;/code&gt; layer during back-propagation? Should I compute the derivative of &lt;span class=&quot;math-container&quot;&gt;$dA$&lt;/span&gt; and reshape it for the next layer or just reshape &lt;span class=&quot;math-container&quot;&gt;$dA$&lt;/span&gt; of the previous layer?&lt;/p&gt;&#xA;" OwnerUserId="38736" LastEditorUserId="2444" LastEditDate="2020-07-20T11:41:27.020" LastActivityDate="2020-09-10T21:38:13.503" Title="What should I do with the flatten layer during back-propagation?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-12-16T15:03:25.933" ContentLicense="CC BY-SA 4.0" />
  <row Id="22593" PostTypeId="2" ParentId="22572" CreationDate="2020-07-20T10:05:50.383" Score="1" Body="&lt;p&gt;No, you can't. In CNN, if you want to detect landmark, you need to prepare data with region box, it's coordinates, width, height, than number of points that should be detected and points coordinates. Then your target vector should be,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pZPp7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pZPp7.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is your target vector. Optionally you can use YOLO algorithm.&lt;/p&gt;&#xA;" OwnerUserId="38736" LastActivityDate="2020-07-20T10:05:50.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22594" PostTypeId="1" CreationDate="2020-07-20T10:44:34.557" Score="0" ViewCount="38" Body="&lt;p&gt;I'm working on a problem to identify whether a person's image matches with the one in his ID proof. The inputs are the real-time face image and scanned ID proof.&lt;/p&gt;&#xA;&lt;p&gt;Options that I'm thinking of&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Prepare a data set of ID's from original images using GAN's&lt;/li&gt;&#xA;&lt;li&gt;Face recognition using transfer learning &amp;amp; some sort of similarity matching&lt;/li&gt;&#xA;&lt;li&gt;Use 3rd party face verification/comparison solution&lt;/li&gt;&#xA;&lt;li&gt;Any data set which suits this task&lt;/li&gt;&#xA;&lt;li&gt;Any other viable approach&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Looking forward to suggestions.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="28466" LastEditorUserId="28466" LastEditDate="2020-07-20T12:22:24.640" LastActivityDate="2020-07-20T12:22:24.640" Title="How to match faces with images in ID proofs" Tags="&lt;computer-vision&gt;&lt;generative-adversarial-networks&gt;&lt;facial-recognition&gt;&lt;image-generation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22595" PostTypeId="1" CreationDate="2020-07-20T12:17:52.247" Score="0" ViewCount="53" Body="&lt;p&gt;When we are doing multi-label segmentation tasks, our &lt;code&gt;y_true&lt;/code&gt; (the mask) will be &lt;code&gt;(w, h, 3)&lt;/code&gt;, but, in our model, at the last layer, we will be getting &lt;code&gt;(w, h, number of classes)&lt;/code&gt; as output.&lt;/p&gt;&#xA;&lt;p&gt;How do we make our outputs to have the same size as the true mask so that to apply the loss function, given that, currently, the shapes are not equal? Also, if we are done with applying the loss function and trained the model, how do I make results in the shape of &lt;code&gt;(w, h, 3)&lt;/code&gt; from &lt;code&gt;(w, h, number of classes)&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="38737" LastEditorUserId="2444" LastEditDate="2020-07-20T21:09:27.523" LastActivityDate="2021-04-17T01:48:02.823" Title="How do we make our outputs to have the same size as the true mask?" Tags="&lt;deep-learning&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22596" PostTypeId="2" ParentId="22595" CreationDate="2020-07-20T12:56:56.860" Score="0" Body="&lt;p&gt;You can create a mapping from classes to colors for a simple one is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y # y.shape = (W, H, n_classes)&#xA;_, y_color = y.max(dim=-1, keepdim=True) / n_classes # y_color.shape = (W, H, 1)&#xA;y_color = torch.cat([y_color] * 3, dim=-1) # y_color.shape = (W, H, 3)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(using pytorch like code)&lt;/p&gt;&#xA;&lt;p&gt;This mapping is visualizable, of course, you may get nicer visualizations if your class to color mapping is more sophisticated.&lt;/p&gt;&#xA;" OwnerUserId="38741" LastActivityDate="2020-07-20T12:56:56.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22599" PostTypeId="1" AcceptedAnswerId="22611" CreationDate="2020-07-20T21:11:43.370" Score="2" ViewCount="67" Body="&lt;p&gt;I have implemented several policy gradient algorithms (REINFORCE, A2C, and PPO) and am finding that the resultant policy's action probability distributions can be rather extreme. As a note, I have based my implementations on OpenAI's baselines. I've been using NNs as the function approximator followed by a Softmax layer. For example, with Cartpole I end up with action distributions like &lt;span class=&quot;math-container&quot;&gt;$[1.0,3e-17]$&lt;/span&gt;. I could understand this for a single action, potentially, but sequential trajectories end up having a probability of 1. I have been calculating the trajectory probability by &lt;span class=&quot;math-container&quot;&gt;$\prod_i \pi(a_i|s_i)$&lt;/span&gt;. Varying the learning rate changes how fast I arrive at this distribution, I have used learning rates of &lt;span class=&quot;math-container&quot;&gt;$[1e-6, 0.1]$&lt;/span&gt;. It seems to me that a trajectory's probability should never be 1.0 or 0.0 consistently, especially with a stochastic start. This also occurs for environments like LunarLander.&lt;/p&gt;&#xA;&lt;p&gt;For the most part, the resulting policies are near-optimal solutions that pass the criteria for solving the environments set by OpenAI. Some random seeds are sub-optimal&lt;/p&gt;&#xA;&lt;p&gt;I have been trying to identify a bug in my code, but I'm not sure what bug would be across all 3 algorithms and across environments.&lt;/p&gt;&#xA;&lt;p&gt;Is it common to have such extreme policy's probabilities? Is there a common way to handle an update so the policy's probabilities do not end up so extreme? Any insight would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="38256" LastEditorUserId="2444" LastEditDate="2020-07-22T11:22:42.773" LastActivityDate="2020-07-22T11:22:42.773" Title="Is it common to have extreme policy's probabilities?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;policies&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22600" PostTypeId="2" ParentId="22558" CreationDate="2020-07-20T23:03:07.703" Score="1" Body="&lt;p&gt;It depends on what you mean by optimal.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A* will always find the optimal solution (that is, the algorithm is &lt;em&gt;admissible&lt;/em&gt;) as long as the heuristic is admissible. (Note that the definition of admissible is overloaded and means something slightly different for an algorithm and a heuristic.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you talk about the &lt;em&gt;set&lt;/em&gt; of nodes expanded by A*, then it expands the minimal &lt;em&gt;set&lt;/em&gt; of nodes up to tie-breaking even with an inconsistent heuristic.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you talk about the number of &lt;em&gt;expansions&lt;/em&gt;, then A* is not optimal. A* can perform up to &lt;span class=&quot;math-container&quot;&gt;$2^N$&lt;/span&gt; expansions of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; states with an inconsistent heuristic. This result comes from &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/0004370277900029&quot; rel=&quot;nofollow noreferrer&quot;&gt;Martelli, 1977&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Algorithm B has worst-case &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt; expansions and &lt;a href=&quot;https://webdocs.cs.ualberta.ca/%7Enathanst/papers/IBEX.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;budgeted graph search (BGS)&lt;/a&gt; has worst-case &lt;span class=&quot;math-container&quot;&gt;$N \log C$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the optimal solution cost.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can see a demo that shows the worst-case performance of A* as well as the performance of BGS here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.movingai.com/SAS/INC/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.movingai.com/SAS/INC/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17493" LastEditorUserId="17493" LastEditDate="2020-11-08T16:43:15.040" LastActivityDate="2020-11-08T16:43:15.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22601" PostTypeId="2" ParentId="22589" CreationDate="2020-07-21T05:43:47.847" Score="2" Body="&lt;p&gt;ANNs as used today need 1. a lot of data 2. a lot of computational power. Before we had any of the above two, we didn't really know how to properly build ANNs since we didn't quite have the means to train the network, and thus couldn't evaluate it.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Symbolic AI&amp;quot; on the other hand, is very much just a bunch of if-else/logical conditions, much like regular programming. You don't need to think too much about the whole &amp;quot;symbolic&amp;quot; part of it. The main/big breakthrough is that you had a lot of clever &amp;quot;search algorithms&amp;quot; and a lot of computation power relative to before.&lt;/p&gt;&#xA;&lt;p&gt;Point being, is just that symbolic AI was the main research program at the time, and people didn't really bother with &amp;quot;connectionist&amp;quot; methods.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2020-07-21T05:43:47.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22602" PostTypeId="1" AcceptedAnswerId="23179" CreationDate="2020-07-21T08:49:03.870" Score="1" ViewCount="83" Body="&lt;p&gt;I have questions on the way AlphaGo Zero is trained.&#xA;From original AlphaGo Zero &lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, I knew that AlphaGo Zero agent learns a policy, value functions by the gathered data &lt;span class=&quot;math-container&quot;&gt;$\{(s_t, \pi_t, z_t)\}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$z_t = r_T \in \{-1,1\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, the fact that the agent tries to learn a policy distribution when &lt;span class=&quot;math-container&quot;&gt;$z_t = -1$&lt;/span&gt; seems to be counter-intuitive (at least to me).&lt;/p&gt;&#xA;&lt;p&gt;My assertion is that &lt;em&gt;the agent should not learn the policy distribution of when it loses (i.e, gets &lt;span class=&quot;math-container&quot;&gt;$z_t=-1$&lt;/span&gt;), since such a policy will guide it to lose.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think I have missed some principles and resulted in that assertion. Or is my assertion reasonable, either?&lt;/p&gt;&#xA;" OwnerUserId="38753" LastEditorUserId="2444" LastEditDate="2020-07-21T13:45:51.330" LastActivityDate="2020-08-21T19:59:10.493" Title="How AlphaGo Zero is learning from $\pi_t$ when $z_t = -1$?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;alphago-zero&gt;&lt;alphago&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22603" PostTypeId="1" AcceptedAnswerId="22605" CreationDate="2020-07-21T09:59:44.620" Score="3" ViewCount="291" Body="&lt;p&gt;In reinforcement learning, an agent can receive a positive reward for correct actions and a negative reward for wrong actions, but does the agent also receive rewards for every other step/action?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-11-02T22:00:31.913" LastActivityDate="2020-12-20T16:59:50.490" Title="Is a reward given at every step or only given when the RL agent fails or succeeds?" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;reward-shaping&gt;&lt;dense-rewards&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22604" PostTypeId="1" CreationDate="2020-07-21T10:29:31.883" Score="2" ViewCount="130" Body="&lt;p&gt;With over 100 papers published in the area of artificial intelligence, machine learning and their subfields every day (&lt;a href=&quot;http://data-mining.philippe-fournier-viger.com/too-many-machine-learning-papers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;), accounting for ~3% of all publications world wide per year (&lt;a href=&quot;https://hai.stanford.edu/sites/default/files/ai_index_2019_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;) and dozens of annual conferences like NeurIPS, ICML, ICLR, ACL, ... I wonder how you keep up with the current state of the art and latest developments? The field is progressing very fast, models that were considered SOTA not even a decade ago are now (almost) outdated (&lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt;). A lot of this progress is driven by big tech companies (&lt;a href=&quot;https://hai.stanford.edu/sites/default/files/ai_index_2019_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;), e.g. 12% of all papers accepted at NeurIPS 2019 have at least one author from Google and DeepMind (&lt;a href=&quot;https://huyenchip.com/2019/12/18/key-trends-neurips-2019.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;My strategy is to read blogs and articles to maintain a general overview and not to miss any important breakthroughs. To be up to date in subfields of my own interest, I read specific papers once in a while. What are your personal strategies? Continuous education is a big keyword here. It's not about understanding every detail and being able to reproduce results, but rather maintaining a bird's eye view, having an idea about the direction of research and knowing whats already possible.&lt;/p&gt;&#xA;&lt;p&gt;To name a few of my preferred sources there are the research blogs of the big players: &lt;a href=&quot;https://openai.com/blog/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI&lt;/a&gt;, &lt;a href=&quot;https://deepmind.com/blog&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind&lt;/a&gt;, &lt;a href=&quot;https://ai.googleblog.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google AI&lt;/a&gt;, &lt;a href=&quot;https://ai.facebook.com/blog/?page=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAIR&lt;/a&gt;. Further there are very good personal blogs with a more educational character, like the well known one of &lt;a href=&quot;https://colah.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Christopher Olah&lt;/a&gt;, the recently started one of &lt;a href=&quot;https://yoshuabengio.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yoshua Bengio&lt;/a&gt; and the one from &lt;a href=&quot;http://jalammar.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jay Alammar&lt;/a&gt;. Unfortunately finding personal blogs is hard, it often depends on luck and referrals, also the update frequency is generally lower since these people have (understandably) other important things to do in life as well.&lt;/p&gt;&#xA;&lt;p&gt;Therefore I'm always looking for new sources, which I can bookmark and read later, if I like to avoid doing other stuff.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Can you name any other personal / corporate research blogs or news websites that publish latest advances in ML &amp;amp; AI?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="37120" LastEditDate="2020-07-21T11:41:12.350" LastActivityDate="2020-07-21T11:41:12.350" Title="Ways to keep up with the latest developments in Machine Learning and AI?" Tags="&lt;machine-learning&gt;&lt;research&gt;&lt;papers&gt;&lt;state-of-the-art&gt;&lt;ai-development&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22605" PostTypeId="2" ParentId="22603" CreationDate="2020-07-21T10:40:28.477" Score="2" Body="&lt;p&gt;In reinforcement learning (RL), an immediate reward value must be returned after each action, along with the next state. This value can be zero though, which will have no direct impact on optimality or setting goals.&lt;/p&gt;&#xA;&lt;p&gt;Unless you are modifying the reward scheme to try and make an environment easier to learn (called &lt;a href=&quot;https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-30164-8_731&quot; rel=&quot;nofollow noreferrer&quot;&gt;reward shaping&lt;/a&gt;), then you should be aiming for a &amp;quot;natural&amp;quot; reward scheme. That means granting reward based directly on the goals of the agent.&lt;/p&gt;&#xA;&lt;p&gt;Common reward schemes might include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;+1 for winning a game or reaching a goal state granted only at the end of an episode, whilst all other steps have a reward of zero. You might also see 0 for a draw and -1 for losing a game.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;-1 per time step, when the goal is to solve a problem in minimum time steps.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a reward proportional to the amount of something that the agent produces - e.g. energy, money, chemical product, granted on any stop where this product is obtained, zero otherwise. Potentially a negative reward based on something else that the agent consumes in order to produce the product, e.g. fuel.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-12-20T16:59:50.490" LastActivityDate="2020-12-20T16:59:50.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22607" PostTypeId="1" AcceptedAnswerId="22610" CreationDate="2020-07-21T11:17:58.000" Score="1" ViewCount="45" Body="&lt;p&gt;In &lt;a href=&quot;http://jmlr.csail.mit.edu/papers/volume7/evendar06a/evendar06a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; - &amp;quot;Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems&amp;quot;, on page 1083, on the 6th line from the bottom, the authors define expectation of the empirical model as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat{\mathbb{E}}_{s,s',a}[V(s')] = \sum_{s' \in S} \hat{P}^{a}_{s, s'}V(s').$$&lt;/span&gt;&#xA;I didn't understand the significance of this quantity since it puts &lt;span class=&quot;math-container&quot;&gt;$V(s')$&lt;/span&gt; inside an expectation while assuming the knowledge of &lt;span class=&quot;math-container&quot;&gt;$V(s')$&lt;/span&gt; in the definition on the right.&lt;/p&gt;&#xA;&lt;p&gt;A clarification in this regard would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;The paper defines &lt;span class=&quot;math-container&quot;&gt;$\hat{P}^{a}_{s, s'}$&lt;/span&gt; as,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat{P}^{a}_{s, s'} = \frac{|(s, a, s', t)|}{|(s, a, t)|}.$$&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$|(s, a, t)|$&lt;/span&gt; is the number of times state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; was visited and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; was taken and &lt;span class=&quot;math-container&quot;&gt;$|(s, a, s', t)|$&lt;/span&gt; as the number of times among the &lt;span class=&quot;math-container&quot;&gt;$|(s, a, t)|$&lt;/span&gt; times &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; was visited when the next state landed in was &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; during model learning.&lt;/p&gt;&#xA;&lt;p&gt;No explicit definition for &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is provided however, &lt;span class=&quot;math-container&quot;&gt;$V^{\pi}$&lt;/span&gt; is defined as the usual expected discounted return, using the same definition as Sutton and Barto or other sources.&lt;/p&gt;&#xA;" OwnerUserId="28384" LastEditorUserId="28384" LastEditDate="2020-07-21T13:39:08.163" LastActivityDate="2020-07-21T14:53:24.033" Title="What is the expectation of an empirical model in model based RL?" Tags="&lt;model-based-methods&gt;&lt;pac-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22608" PostTypeId="2" ParentId="22569" CreationDate="2020-07-21T11:32:14.947" Score="1" Body="&lt;p&gt;Data management and bandwidth are key issues for interconnecting multiple GPUs.  These are such big issues that it is hard to think about other challenges like neural network architecture, metrics, etc.  The key to success for interconnecting multiple GPUs on a single computer is NVIDIA's &lt;a href=&quot;https://www.nvidia.com/en-us/data-center/nvlink/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NVLink&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;NVLink is a wire-based communications protocol for near-range semiconductor communications developed by Nvidia that can be used for data and control code transfers in processor systems between CPUs and GPUs and solely between GPUs. NVLink specifies a point-to-point connection with data rates of 20 and 25 Gbit/s (v1.0/v2.0) per differential pair.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Compare 25 Gbit/s to a typical peer to peer connection over the web of 100Mbps.  NVLINK provides a 250x advantage assuming everything else is equal which it is not.  This means that, considering bandwidth only, a neural network which takes one day to train on a computer with two GPUs connected with NVLINK could take 250 days over the internet using two computers with the same GPU!&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-07-21T11:37:51.087" LastActivityDate="2020-07-21T11:37:51.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22609" PostTypeId="1" AcceptedAnswerId="22617" CreationDate="2020-07-21T14:20:27.200" Score="2" ViewCount="110" Body="&lt;p&gt;I understand the fact that the neural network is used to take the states as inputs and it outputs the Q-value for state-action pairs. However, in order to compute this and update its weights, we need to calculate the maximum Q-value for the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;. In order to get that, in the DDQN case, we input that next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; in the target network.&lt;/p&gt;&#xA;&lt;p&gt;What I'm not clear on is: how do we train this target network itself that will help us train the other NN? What is its cost function?&lt;/p&gt;&#xA;" OwnerUserId="34516" LastEditorUserId="2444" LastEditDate="2020-11-05T01:04:23.933" LastActivityDate="2020-11-05T01:19:27.783" Title="How does the target network in double DQNs find the maximum Q value for each action?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22610" PostTypeId="2" ParentId="22607" CreationDate="2020-07-21T14:53:24.033" Score="1" Body="&lt;p&gt;If I understand your question correctly, the significance of this is due to the fact that &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is random. In the RHS of the equation it is assumed that &lt;span class=&quot;math-container&quot;&gt;$V(\cdot)$&lt;/span&gt; is known for each state, but the quantity is measuring the expected value of the next state given the current state and action.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-07-21T14:53:24.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22611" PostTypeId="2" ParentId="22599" CreationDate="2020-07-21T16:11:28.853" Score="2" Body="&lt;p&gt;Your policy gradient algorithms appear to be working as intended. All standard MDPs have one or more &lt;em&gt;deterministic&lt;/em&gt; optimal solutions, and those are the policies that solvers will converge to. Making any of these policies more random will often reduce their effectiveness, making them sub-optimal. So once consistently good actions are discovered, the learning process will reduce exploration naturally as a consequence of the gradients, much like a softmax classifier with a clean dataset.&lt;/p&gt;&#xA;&lt;p&gt;There are some situations where a stochastic policy can be optimal, and you could check your implementations can find those:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A partially observable MDP (POMDP) where one or more key states requiring different optimal actions are indistinguishable to the agent. For example, the state could be available exits in a corridor trying to get the end in a small maze, where one location secretly (i.e. without the agent having any info in the state representation that the location is different) reverses all directions, so that progressing along it is not possible for a deterministic agent, but a random agent would eventually get through.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In opposing guessing games where a Nash equilibrium occurs for specific random policies. For example scissor, paper, stone game where the optimal policy in self-play should be to choose each option randomly with 1/3 chance.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first example is probably easiest to set up a toy environment to show that your implementations can find stochastic solutions when needed. A concrete example of that kind of environment is in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto: Reinforcement Learning, An Introduction&lt;/a&gt; chapter 13, example 13.1 on page 323.&lt;/p&gt;&#xA;&lt;p&gt;Setting up opposing agents in self-play is harder, but if you can get it to work and discover the Nash equilibrium point for the policies, it would be further proof that you have got something right.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-21T16:16:37.910" LastActivityDate="2020-07-21T16:16:37.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22612" PostTypeId="1" CreationDate="2020-07-21T16:57:25.537" Score="0" ViewCount="36" Body="&lt;p&gt;We are building an enterprise search engine for about 130,000 documents. It includes FAQs, Small articles to 3-4 page documents, all of them are stored in structured JSON.&#xA;We were originally planning to use Amazon Kendra to do the same, but unfortunately it was not approved by our enterprise cloud team.&lt;/p&gt;&#xA;&lt;p&gt;Now we have to move to elasticsearch, looking to experiment several ways to improve search results with the use of machine learning/nlp.&lt;/p&gt;&#xA;&lt;p&gt;Ex: search query correction with model trained using previous search data and it's results, generate alternative queries with same meaning to enhance the result etc..&#xA;Looking for ideas from here to experiment with&lt;/p&gt;&#xA;" OwnerUserId="25366" LastActivityDate="2020-07-21T16:57:25.537" Title="Suggestion for an Enterprise search engine: What are all the way I can use machine learning/dnn/nlp to enhance elasticsearch results?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22614" PostTypeId="1" AcceptedAnswerId="22615" CreationDate="2020-07-21T18:18:34.200" Score="3" ViewCount="412" Body="&lt;p&gt;While learning RL, I came across some problems where the Q-matrix that I need to make is very very large. I am not sure if it is ever practical. Then I research and came to this conclusion that using the tabular method is not the only way, in fact, it is a very less powerful tool as compared to other methods such as deep RL methods.&lt;/p&gt;&#xA;&lt;p&gt;Am I correct in this understanding that with the increasing complexity of problems, tabular RL methods are getting obsolete?&lt;/p&gt;&#xA;" OwnerUserId="36710" LastEditorUserId="2444" LastEditDate="2020-07-21T18:37:23.793" LastActivityDate="2020-07-21T20:07:24.420" Title="Are tabular reinforcement learning methods obsolete (or getting obsolete)?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22615" PostTypeId="2" ParentId="22614" CreationDate="2020-07-21T20:07:24.420" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Am I correct in this understanding that with the increasing complexity of problems, tabular RL methods are getting obsolete?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Individual problems don't get any more complex, but the scope of &lt;em&gt;solvable&lt;/em&gt; environments increases due to research and discovery of better or more apt methods.&lt;/p&gt;&#xA;&lt;p&gt;Using deep RL methods with large neural nets can be a lot less efficient for solving simple problems. So tabular methods still have their place there.&lt;/p&gt;&#xA;&lt;p&gt;Practically, if your state/action space (number of states times number of actions) is small enough to fit a Q table in memory, and it is possible to visit all relevant state/action pairs multiple times in a relatively short time, then tabular methods offer guarantees of convergence that approximate methods cannot. So tabular approaches are often preferred if they are appropriate.&lt;/p&gt;&#xA;&lt;p&gt;Many interesting, cutting edge problems that are relevant to AI, such as autonomous robots acting in the real world, do not fit the tabular approach. In that sense, the approach is &amp;quot;obsolete&amp;quot; in that it no longer provides challenging research topics for practical AI (there are still unanswered theoretical questions, such as proof of convergence for Monte Carlo control).&lt;/p&gt;&#xA;&lt;p&gt;It is still worth understanding tabular value-based methods in detail, because they form the foundations of the more complex deep learning methods. In some sense they represent ideal solutions that deep RL tries to approximate, and the design of tabular solutions can be the inspiration for changes and adjustments to neural-network methods.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-07-21T20:07:24.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22616" PostTypeId="2" ParentId="22589" CreationDate="2020-07-21T21:32:17.597" Score="3" Body="&lt;p&gt;You might also ask if there's any particular reason why we &lt;em&gt;would&lt;/em&gt; use a neural net. If we're to train a neural net to play chess, we need to be able to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; Feed it positions as input vectors (easy enough),&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Decide on an output format. Perhaps a distribution over possible moves (but then, how to represent that such that the meaning of a specific output cell doesn't change drastically based on the board state?  Or perhaps instead, we let the resulting board state after a candidate move be the input, and let the output be a score that represents the desirability of that state. That'll require exponentially more forward/backprop passes, though.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; Provide it with an error signal to whatever output vector it produces. This is the really tricky bit, since we don't &lt;em&gt;know&lt;/em&gt; whether a given move will result in victory until the very end.&lt;/p&gt;&#xA;&lt;p&gt;Do we play the game to the very end, storing decisions as we go, and then at the end, replay each input, feeding it an error signal if we lost? This will give the same error to the good moves as to the ones that actually lost the game. With enough games, this will work, since the good moves will get positive feedback a bit more often than negative, and vice versa for the bad ones. But it'll take a lot of games. More than a human is going to be willing to play.  We can have different networks learn by playing against each other, but not on 1996 hardware.&lt;/p&gt;&#xA;&lt;p&gt;Do we instead provide a score based on another heuristic of the board state? In that case, why not just use &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt;? It's provably optimal for a given heuristic up to however many moves deep we look, and it doesn't need training.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Add to this the fact that if we don't choose a good representation at each of these steps, there's a good chance that the network will only learn the positions it's specifically been trained on, rather than generalizing to unseen states, which is the main reason for using a neural network in the first place.&lt;/p&gt;&#xA;&lt;p&gt;It's certainly possible to use neural nets to learn chess (DeepMind's approach can be found &lt;a href=&quot;https://arxiv.org/abs/1712.01815&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, for instance), but they're not a natural fit to the problem by any means. Minimax, by contrast, fits the problem very well, which is why it was one of the techniques used by Deep Blue. Neural nets are an amazing tool, but they're not always the right tool for the job.&lt;/p&gt;&#xA;&lt;p&gt;Addendum:&#xA;I didn't stress this point much, since K.C. already brought it up, but training large neural nets require us to perform a huge number of matrix vector multiplications, and this wasn't especially practical before GPUs got powerful and cheap.&lt;/p&gt;&#xA;" OwnerUserId="2212" LastEditorUserId="2212" LastEditDate="2020-07-22T14:19:05.840" LastActivityDate="2020-07-22T14:19:05.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22617" PostTypeId="2" ParentId="22609" CreationDate="2020-07-22T00:08:08.423" Score="1" Body="&lt;p&gt;Both in DQN and in DDQN, the target network starts as an exact copy of the Q-network, that has the same weights, layers, input and output dimensions, etc., as the Q-network.&lt;/p&gt;&#xA;&lt;p&gt;The main idea of the DQN agent is that the Q-network predicts the Q-values of actions from a given state and selects the maximum of them and uses the mean squared error (MSE) as its cost/loss function. That is, it performs gradient descent steps on&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left(Y_{t}^{\mathrm{DQN}} -Q\left(s_t, a_t;\boldsymbol{\theta}\right)\right)^2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the target &lt;span class=&quot;math-container&quot;&gt;$Y_{t}^{\mathrm{DQN}}$&lt;/span&gt; is defined (in the case of DQN) as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Y_{t}^{\mathrm{DQN}} \equiv R_{t+1}+\gamma \max _{a} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}^{-}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; are the Q-network weights and &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta^-}$&lt;/span&gt; are the target network weights.&lt;/p&gt;&#xA;&lt;p&gt;After a usually fixed number of timesteps, the target network updates its weights by copying the weights of the Q-network. So, basically, the target network never performs a feed-forward training phase and, thus, ignores a cost function.&lt;/p&gt;&#xA;&lt;p&gt;In the case of DDQN, the target is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Y_{t}^{\text {DDQN}} \equiv R_{t+1}+\gamma Q\left(S_{t+1}, \underset{a}{\operatorname{argmax}} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}\right) ; \boldsymbol{\theta}_{t}^{-}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This target is used to decouple the selection of the action (i.e. the &lt;em&gt;argmax&lt;/em&gt; part) from its evaluation (i.e. the computation of the Q value at the next state with this selected action), as stated &lt;a href=&quot;https://arxiv.org/pdf/1509.06461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper that introduced the DDQN&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The max operator in standard Q-learning and DQN, in (2) and (3), uses the same values both to select and to evaluate an action. This makes it more likely to select overestimated values, resulting in overoptimistic value estimates. To prevent this, we can decouple the selection from the evaluation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="2444" LastEditDate="2020-11-05T01:19:27.783" LastActivityDate="2020-11-05T01:19:27.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22618" PostTypeId="1" AcceptedAnswerId="22624" CreationDate="2020-07-22T01:18:42.457" Score="1" ViewCount="67" Body="&lt;p&gt;I am trying to implement the DDPG algorithm based on this &lt;a href=&quot;https://arxiv.org/abs/1509.02971&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The part that confuses me is the actor network's update.&#xA;I don't understand why the policy loss is simply the mean of &lt;span class=&quot;math-container&quot;&gt;$-Q(s, \mu(s))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is the critic network and &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the policy network.&#xA;How does one arrive at this?&lt;/p&gt;&#xA;" OwnerUserId="30632" LastEditorUserId="2444" LastEditDate="2020-07-22T11:28:20.953" LastActivityDate="2020-07-22T12:49:17.607" Title="Why is the policy loss the mean of $-Q(s, \mu(s))$ in the DDPG algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;pytorch&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22619" PostTypeId="1" AcceptedAnswerId="22666" CreationDate="2020-07-22T05:57:01.547" Score="1" ViewCount="70" Body="&lt;p&gt;XOR data, without labels:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[0,0],[0,1],[1,0],[1,1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm using this network for auto-classifying XOR data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;H1  &amp;lt;-- Dense(units=2, activation=relu)    #any activation here&#xA;Z   &amp;lt;-- Dense(units=2, activation=softmax) #softmax for 2 classes of XOR result&#xA;Out &amp;lt;-- Dense(units=2, activation=sigmoid) #sigmoid to return 2 values in (0,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There's a logical problem in the network, that is, Z represents 2 classes,&#xA;however, the 2 classes can't be decoded back to 4 samples of XOR data.&lt;/p&gt;&#xA;&lt;p&gt;How to fix the network above to auto-classify XOR data, in unsupervised manner?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-07-25T13:57:22.700" LastActivityDate="2020-07-25T16:35:11.637" Title="What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;unsupervised-learning&gt;&lt;autoencoders&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22622" PostTypeId="1" CreationDate="2020-07-22T08:00:14.133" Score="3" ViewCount="120" Body="&lt;p&gt;I'm trying to decide which policy improvement algorithm to use in the context of my problem. But let me emerge you into the problem&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to move a set of points in a 3D space. Depending on how the points move, the environment gives a positive or negative reward. Further, the environment does not split up into episodes, so it is a continuing problem. The state space is high-dimensional (a lot of states are possible) and many states can be similar (so state aliasing can appear), also states are continuous. The problem is dense in rewards, so for every transition, there will be a negative or positive reward, depending on the previous state.&lt;/p&gt;&#xA;&lt;p&gt;A state is represented as a vector with dimension N (initially it will be something like ~100, but in the future, I want to work with vectors up to 1000).&lt;/p&gt;&#xA;&lt;p&gt;In the case of action, it is described by a matrix 3xN, where N is the same as in the case of the state. The first dimension comes from the fact, that action is 3D displacement.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I have done so far&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since actions are continuous, I have narrowed down my search to policy gradient methods. Further, I researched methods, that work with continuous state spaces. I found a deep deterministic policy gradient (DDPG) and the Proximal Policy Gradient (PPO) would fit here. Theoretically, they should work but I'm unsure and any advice would be gold here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Would those algorithms be suitable for the problem (PPO or DDPG)?&#xA;There are other policy improvement algorithms that would work here or a family of policy improvement algorithms?&lt;/p&gt;&#xA;" OwnerUserId="31324" LastEditorUserId="31324" LastEditDate="2020-07-22T08:19:09.657" LastActivityDate="2020-07-28T10:16:43.780" Title="Choosing a policy improvement algorithm for a continuing problem with continuous action and state-space" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22623" PostTypeId="1" AcceptedAnswerId="22628" CreationDate="2020-07-22T12:02:21.933" Score="5" ViewCount="115" Body="&lt;p&gt;AFAIK, GANs are used for generating/synthesizing near-perfect human faces (deepfakes), gallery arts, etc., but can GANs be used to generate something other than images?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-07-22T20:49:47.977" LastActivityDate="2020-07-23T00:20:39.300" Title="Can GANs be used to generate something other than images?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;generative-adversarial-networks&gt;&lt;deepfakes&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22624" PostTypeId="2" ParentId="22618" CreationDate="2020-07-22T12:49:17.607" Score="2" Body="&lt;p&gt;This is not quite the loss that is stated in the paper.&lt;/p&gt;&#xA;&lt;p&gt;For standard policy gradient methods the objective is to maximise &lt;span class=&quot;math-container&quot;&gt;$v_{\pi_\theta}(s_0)$&lt;/span&gt; -- note that this is analogous to minimising &lt;span class=&quot;math-container&quot;&gt;$-v_{\pi_\theta}(s_0)$&lt;/span&gt;. This is for a stochastic policy. In DDPG the policy is now assumed to be deterministic.&lt;/p&gt;&#xA;&lt;p&gt;In general, we can write&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \mathbb{E}_{a\sim\pi}[Q(s,a)]\;;$$&lt;/span&gt;&#xA;to see this note that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) = \mathbb{E}[G_t | S_t = s, A_t=a]\;;$$&lt;/span&gt;&#xA;so if we took expectation over this with respect to the distribution of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; we would get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{a\sim\pi}[\mathbb{E}[G_t|S_t=s, A_t=a]] = \mathbb{E}[G_t|S_t=s] = v_\pi(s)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, if our policy is deterministic then &lt;span class=&quot;math-container&quot;&gt;$\pi(\cdot|s)$&lt;/span&gt; is a point mass (a distribution which has probability 1 for a specific point and 0 everywhere else) for a certain action, so &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{a\sim\pi}[ Q(s,a)] = Q(s,a=\pi(s)) = v_\pi(s)$&lt;/span&gt;. Thus the objective is still to maximise &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; it is just that now we know the policy is deterministic we say we want to maximise &lt;span class=&quot;math-container&quot;&gt;$Q(s,a=\pi(s))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The policy gradient of this term was shown to be&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla_\theta Q(s,a=\pi_\theta(s)) &amp;amp; \approx \mathbb{E}_{s \sim \mu}[\nabla_\theta Q(s,a=\pi_\theta(s))]\;; \\ &amp;amp; =  \mathbb{E}_{s\sim\mu}[\nabla_aQ(s,a=\pi(s)) \nabla_\theta \pi_\theta(s)]\;;&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where if we put a minus at the front of this term then we would arrive at the loss from the paper. Intuitively this makes sense, you want to know how much the action-value function changes with respect to the parameter of the policy, but this would be difficult to directly calculate so you use the chain rule to see how much the action-value function changes with &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and in term how much &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (i.e. our policy) changes with the parameter of the policy.&lt;/p&gt;&#xA;&lt;p&gt;I realise I have changed notation from the paper you are reading so here &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is our policy as opposed to &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and here where I have used &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; I take this to be the state distribution function.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-07-22T12:49:17.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22627" PostTypeId="1" AcceptedAnswerId="24429" CreationDate="2020-07-22T20:41:50.807" Score="3" ViewCount="126" Body="&lt;p&gt;For the past few days, I am trying to learn graph convolutional networks. I saw some of the lectures on youtube. But I can not able to get any clear concept of how those networks are trained. I have a vague understanding of how to perform convolution, but I can not understand how we train them. I want a solid mathematical understanding of graph convolutional networks. So, can anyone please suggest me how to start learning graph convolutional network from start to expert level?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-07-22T20:46:19.607" LastActivityDate="2021-01-15T11:24:24.827" Title="What is the best resources to learn Graph Convolutional Neural Networks?" Tags="&lt;resource-request&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22628" PostTypeId="2" ParentId="22623" CreationDate="2020-07-23T00:20:39.300" Score="2" Body="&lt;p&gt;They can indeed. Although generally they are kept to images because at the moment, they are the best at that, but not the best in other areas that you might consider.&lt;/p&gt;&#xA;&lt;p&gt;GANs can be used for audio generation, with many examples such as &lt;a href=&quot;https://magenta.tensorflow.org/gansynth&quot; rel=&quot;nofollow noreferrer&quot;&gt;GANsynth&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1802.06840&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN voice generation&lt;/a&gt;. But each of these tasks are outperformed by other methods. With music generation, &lt;a href=&quot;https://deepmind.com/blog/article/wavenet-generative-model-raw-audio&quot; rel=&quot;nofollow noreferrer&quot;&gt;WaveNet&lt;/a&gt; is the best (last I checked, and it also performs very well at speech synthesis), and a more powerful model for voice generation is &lt;a href=&quot;https://arxiv.org/pdf/1806.04558.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;achieved through the use of a VAE&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is only looking at one area that you could use GANs for, because in reality you could use them for any kind of generation if you wanted to, but at the moment the vast majority of the research into GANs is into image generation, and as such other areas do not compete with the current SOTA techniques, unless there's some big paper I've missed within the last few months.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-07-23T00:20:39.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22629" PostTypeId="1" CreationDate="2020-07-23T02:05:34.473" Score="0" ViewCount="53" Body="&lt;p&gt;I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?&lt;/p&gt;&#xA;" OwnerUserId="30884" LastEditorUserId="2444" LastEditDate="2020-07-25T13:47:53.033" LastActivityDate="2020-07-25T13:47:53.033" Title="Can I train a neural network with multiple datasets (e.g. 25)?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;datasets&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22630" PostTypeId="1" AcceptedAnswerId="22646" CreationDate="2020-07-23T04:56:05.627" Score="1" ViewCount="80" Body="&lt;p&gt;AI is the emerging field and biggest business opportunity of the next decade. It's already automating manual and repetitive tasks. And in some areas, it can learn faster than humans, if not yet as deeply.&lt;/p&gt;&#xA;&lt;p&gt;From the Forbes article&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://www.forbes.com/sites/cognitiveworld/2019/10/17/the-ai-enabled-future/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AI-enabled future&lt;/a&gt;, humans will be able to converse and interact with each other in the native language of choice, not having to worry about miscommunicating intentions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would like to know more about how artificial intelligence will change the future?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastActivityDate="2020-07-24T06:41:01.667" Title="How artificial intelligence will change the future?" Tags="&lt;philosophy&gt;&lt;theory&gt;&lt;social&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22631" PostTypeId="1" AcceptedAnswerId="22663" CreationDate="2020-07-23T06:14:16.183" Score="3" ViewCount="86" Body="&lt;p&gt;In many diagrams, as seen below, residual neural networks are only depicted with ReLU activation functions, but can residual NNs also use other activation functions, such as the sigmoid, hyperbolic tangent, etc.?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G42j3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G42j3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="2444" LastEditDate="2020-07-25T02:50:56.113" LastActivityDate="2020-07-25T03:38:16.443" Title="Can residual neural networks use other activation functions different from ReLU?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;relu&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22632" PostTypeId="1" AcceptedAnswerId="22688" CreationDate="2020-07-23T07:10:58.130" Score="0" ViewCount="63" Body="&lt;p&gt;Normalisation transform data into a range:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X_i = \dfrac{X_i - Min}{Max-Min}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of standardisation (another formula shown below).&lt;/p&gt;&#xA;&lt;p&gt;Before training a neural net, data are usually standardised or normalised. Standardising seems good as it makes the model generalise better, while normalisation may make the model not working with values out of training data range.&lt;/p&gt;&#xA;&lt;p&gt;So I'm using standardisation for input data (X), however, I'm confusing whether I should standardise the expected output values too?&lt;/p&gt;&#xA;&lt;p&gt;For a column in input data:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X_i = \dfrac{(X_i - Mean)}{Standard\ Deviation\ of\ the\ Column}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Should I apply this formula to the expected output values (labels) too?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2020-07-26T03:26:15.283" LastActivityDate="2020-07-27T04:18:22.410" Title="Is it necessary to standardise the expected output" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;normalisation&gt;&lt;standardisation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22633" PostTypeId="1" CreationDate="2020-07-23T07:52:57.733" Score="4" ViewCount="82" Body="&lt;p&gt;I'm wondering, has anyone seen any paper where one trains a network but biases it to produce similar outputs to a given model (such as one given from expert opinion or it being a previously trained network).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Formally, I'm looking for a paper doing the following:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$g:\mathbb{R}^d\rightarrow \mathbb{R}^D$&lt;/span&gt; be a model (not necessarily, but possibly, a neural network) trained on some input/output data pairs &lt;span class=&quot;math-container&quot;&gt;$\{(x_n,y_n)\}_{n=1}^N$&lt;/span&gt; and train a neural network &lt;span class=&quot;math-container&quot;&gt;$f_{\theta}(\cdot)$&lt;/span&gt; on&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\underset{\theta}{\operatorname{argmin}}\sum_{n=1}^N \left\|&#xA;f_{\theta}(x_n) - y_n&#xA;\right\| + \lambda \left\|&#xA;f_{\theta}(x_n) - g(x_n)&#xA;\right\|,&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; represents all the trainable weight and bias parameters of the network &lt;span class=&quot;math-container&quot;&gt;$f_{\theta}(\cdot)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So put another way...&lt;span class=&quot;math-container&quot;&gt;$f_{\theta}(\cdot)$&lt;/span&gt; is being regularized by the outputs of another model...&lt;/p&gt;&#xA;" OwnerUserId="34185" LastActivityDate="2020-07-23T07:52:57.733" Title="Forcing a neural network to be close to a previous model - Regularization through given model" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;reference-request&gt;&lt;regularization&gt;&lt;algorithmic-bias&gt;" AnswerCount="0" CommentCount="7" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22634" PostTypeId="1" CreationDate="2020-07-23T09:33:59.227" Score="2" ViewCount="358" Body="&lt;p&gt;I have used a different setting, but DDPG is not learning and it does not converge. I have used these codes &lt;a href=&quot;https://github.com/liampetti/DDPG/blob/master/ddpg.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;,&lt;a href=&quot;https://github.com/maxmax1992/DDPG-MADDPG/blob/b8bc4c75dc5ba765c91f82b351b32b28a0211f3d/DDPG_agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, and &lt;a href=&quot;https://github.com/agakshat/tensorflow-ddpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt; and I used different optimizers, activation functions, and learning rate but there is no improvement.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    parser.add_argument('--actor-lr', help='actor network learning rate', default=0.001)&#xA;    parser.add_argument('--critic-lr', help='critic network learning rate', default=0.0001)&#xA;    parser.add_argument('--gamma', help='discount factor for critic updates', default=0.95)&#xA;&#xA;    parser.add_argument('--tau', help='soft target update parameter', default=0.001)&#xA;    parser.add_argument('--buffer-size', help='max size of the replay buffer', default=int(1e5))&#xA;    parser.add_argument('--minibatch-size', help='size of minibatch for minibatch-SGD', default=64)&#xA;&#xA;    # run parameters&#xA;    # parser.add_argument('--env', help='choose the gym env- tested on {Pendulum-v0}', default='MountainCarContinuous-v0')&#xA;    parser.add_argument('--random-seed', help='random seed for repeatability', default=1234)&#xA;    parser.add_argument('--max-episodes', help='max num of episodes to do while training', default=200)&#xA;    parser.add_argument('--max-episode-len', help='max length of 1 episode', default=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sm6Qj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sm6Qj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have trained in the same environment with A2C and it converged.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FGXPI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FGXPI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which parameters should I change to make the DDPG converge? Can anyone help me with this?&lt;/p&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="2444" LastEditDate="2020-07-25T11:57:21.857" LastActivityDate="2020-07-29T12:51:47.783" Title="Why is DDPG not learning and it does not converge?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;convergence&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22635" PostTypeId="2" ParentId="22094" CreationDate="2020-07-23T10:23:18.860" Score="3" Body="&lt;p&gt;The unrolling step is due to the fact you end up with an equation that you can keep expanding indefinitely.&lt;/p&gt;&#xA;&lt;p&gt;Note that we start with calculating &lt;span class=&quot;math-container&quot;&gt;$\nabla v_\pi(s)$&lt;/span&gt; and arrive at&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla v_\pi(s) = \sum_a\left[ \nabla \pi(a|s) q_\pi(s,a) + \pi(a|s) \sum_{s'}p(s'|s,a) \nabla v_\pi (s') \right]\;,$$&lt;/span&gt;&#xA;which contains a term for &lt;span class=&quot;math-container&quot;&gt;$\nabla v_\pi(s')$&lt;/span&gt;. This is a recursive relationship, similar to the bellman equation, so we can substitute in a term for &lt;span class=&quot;math-container&quot;&gt;$\nabla v_\pi(s')$&lt;/span&gt; which will be a term similar just with &lt;span class=&quot;math-container&quot;&gt;$\nabla v_\pi(s'')$&lt;/span&gt;. As I mentioned, we can do this indefinitely which leads us to&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla v_\pi(s) = \sum_{x \in \mathcal{S}} \sum_{k=0}^\infty \mathbb{P}(s\rightarrow x, k, \pi) \sum_a \nabla \pi(a|x) q_\pi(x,a)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We need the term &lt;span class=&quot;math-container&quot;&gt;$\sum_{x \in \mathcal{S}} \sum_{k=0}^\infty \mathbb{P}(s\rightarrow x, k, \pi)$&lt;/span&gt; because we want to take an average over the state space, however due to unrolling there are many different &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;'s that we need to average over (this comes from the &lt;span class=&quot;math-container&quot;&gt;$s',s'',s''',...$&lt;/span&gt; in the unrolling) so we also need to add the probability state of transitioning from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; time steps, where we sum over an infinite horizon due to the repeated unrolling.&lt;/p&gt;&#xA;&lt;p&gt;If you are wondering what happens to the terms &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; terms and why they are not explicitly shown in this final form, it is because this is exactly what the &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(s\rightarrow x, k, \pi)$&lt;/span&gt; represents. The average over all possible states accounts for the &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; and the fact that we follow policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; in the probability statement accounts for the &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-07-23T10:23:18.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22636" PostTypeId="1" CreationDate="2020-07-23T10:44:28.770" Score="3" ViewCount="79" Body="&lt;p&gt;Suppose I have a model that was trained with a dataset that contains the features &lt;code&gt;(f1, f2, f3, f4, f5, f6)&lt;/code&gt;. However, my test dataset does not contain all features of the training dataset, but only &lt;code&gt;(f1, f2, f3)&lt;/code&gt;. How can I predict the true label of the entries of this test dataset without all features?&lt;/p&gt;&#xA;" OwnerUserId="38808" LastEditorUserId="2444" LastEditDate="2020-07-25T12:29:47.227" LastActivityDate="2020-07-27T10:13:51.037" Title="How can I predict the true label for data with incomplete features based on the trained model with data with more features?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;datasets&gt;&lt;testing&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22637" PostTypeId="1" CreationDate="2020-07-23T13:55:08.583" Score="1" ViewCount="84" Body="&lt;p&gt;In the &lt;a href=&quot;https://www.youtube.com/watch?v=LdK9HzBAR8c&amp;amp;list=PL-Y8zK4dwCrQyASidb2mjj_itW2-YYx6-&amp;amp;index=8&amp;amp;t=3295s&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic variant of GCN we have the following&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m4Iz9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m4Iz9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here we aggregate the information from the adjacent node and pass it to a neural network, then transform our own information and add them all.&lt;/p&gt;&#xA;&lt;p&gt;But the main question is: how can we ensure that &lt;span class=&quot;math-container&quot;&gt;$W_{k}(\sum(\frac{h_k}{N(V)})$&lt;/span&gt; will be the same size as &lt;span class=&quot;math-container&quot;&gt;$B_{k}h_{v}$&lt;/span&gt; and does &lt;span class=&quot;math-container&quot;&gt;$B_{k}$&lt;/span&gt; emply another neural network?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2021-03-13T09:31:17.800" LastActivityDate="2021-03-13T09:31:17.800" Title="How Graph Convolutional Neural Networks forward propagate?" Tags="&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22638" PostTypeId="2" ParentId="17734" CreationDate="2020-07-23T14:01:53.993" Score="2" Body="&lt;p&gt;It is not so much the problem of using Reinforcement Learning to train the neural networks, it is the assumptions made about the data given to standard Neural Networks. They are not capable of handling strongly correlated data which is one of the motivations for introducing Recurrent Neural Networks, as they can handle this correlated data well.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-07-23T14:01:53.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22639" PostTypeId="2" ParentId="22637" CreationDate="2020-07-23T14:14:45.070" Score="2" Body="&lt;p&gt;I think the picture you're presenting is mostly for educational purposes and that's why they are excluding the node itself from it's neighbors and using two distinct networks (most of the papers I've read they are using the same network for the neighbors and for the center node). But you are right the two networks needs to have the same input and output shapes otherwise the point-wise summation between the two terms is not possible.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2020-07-23T14:14:45.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22640" PostTypeId="2" ParentId="12311" CreationDate="2020-07-23T14:22:46.667" Score="0" Body="&lt;p&gt;Get a raspberry with a camera, power it by a battery bank, attach it to the EV3 and run a 2 one program on raspberry and another on ev3 communicating with each other via MQTT&lt;/p&gt;&#xA;" OwnerUserId="38815" LastActivityDate="2020-07-23T14:22:46.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22641" PostTypeId="1" CreationDate="2020-07-23T17:26:11.440" Score="1" ViewCount="120" Body="&lt;p&gt;DQN implemented at &lt;a href=&quot;https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py&lt;/a&gt; uses the mean square error loss function for the neural network to learn the state -&amp;gt; action mapping :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.criterion=torch.nn.MSELoss()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could cross-entropy be used instead as the loss function? Cross entropy is typically used for classification, and mean squared error for regression.&lt;/p&gt;&#xA;&lt;p&gt;As the actions are discrete (the example utilises the mountain car environment - &lt;a href=&quot;https://github.com/openai/gym/wiki/MountainCar-v0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openai/gym/wiki/MountainCar-v0&lt;/a&gt;) and map to [0,1,2] can cross-entropy loss be used instead of mean squared error? Why use regression as the state -&amp;gt; action function approximator for deep Q learning instead of classification?&lt;/p&gt;&#xA;&lt;p&gt;Entire DQN src from &lt;a href=&quot;https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;Source codes for PyTorch 1.0 Reinforcement Learning (Packt Publishing)&#xA;Chapter 7: Deep Q-Networks in Action&#xA;Author: Yuxi (Hayden) Liu&#xA;'''&#xA;&#xA;import gym&#xA;import torch&#xA;&#xA;from torch.autograd import Variable&#xA;import random&#xA;&#xA;&#xA;env = gym.envs.make(&amp;quot;MountainCar-v0&amp;quot;)&#xA;&#xA;&#xA;&#xA;class DQN():&#xA;    def __init__(self, n_state, n_action, n_hidden=50, lr=0.05):&#xA;        self.criterion = torch.nn.MSELoss()&#xA;        self.model = torch.nn.Sequential(&#xA;                        torch.nn.Linear(n_state, n_hidden),&#xA;                        torch.nn.ReLU(),&#xA;                        torch.nn.Linear(n_hidden, n_action)&#xA;                )&#xA;        self.optimizer = torch.optim.Adam(self.model.parameters(), lr)&#xA;&#xA;&#xA;    def update(self, s, y):&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Update the weights of the DQN given a training sample&#xA;        @param s: state&#xA;        @param y: target value&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        y_pred = self.model(torch.Tensor(s))&#xA;        loss = self.criterion(y_pred, Variable(torch.Tensor(y)))&#xA;        self.optimizer.zero_grad()&#xA;        loss.backward()&#xA;        self.optimizer.step()&#xA;&#xA;&#xA;    def predict(self, s):&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Compute the Q values of the state for all actions using the learning model&#xA;        @param s: input state&#xA;        @return: Q values of the state for all actions&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        with torch.no_grad():&#xA;            return self.model(torch.Tensor(s))&#xA;&#xA;&#xA;&#xA;def gen_epsilon_greedy_policy(estimator, epsilon, n_action):&#xA;    def policy_function(state):&#xA;        if random.random() &amp;lt; epsilon:&#xA;            return random.randint(0, n_action - 1)&#xA;        else:&#xA;            q_values = estimator.predict(state)&#xA;            return torch.argmax(q_values).item()&#xA;    return policy_function&#xA;&#xA;&#xA;def q_learning(env, estimator, n_episode, gamma=1.0, epsilon=0.1, epsilon_decay=.99):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Deep Q-Learning using DQN&#xA;    @param env: Gym environment&#xA;    @param estimator: DQN object&#xA;    @param n_episode: number of episodes&#xA;    @param gamma: the discount factor&#xA;    @param epsilon: parameter for epsilon_greedy&#xA;    @param epsilon_decay: epsilon decreasing factor&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    for episode in range(n_episode):&#xA;        policy = gen_epsilon_greedy_policy(estimator, epsilon, n_action)&#xA;        state = env.reset()&#xA;        is_done = False&#xA;&#xA;        while not is_done:&#xA;            action = policy(state)&#xA;            next_state, reward, is_done, _ = env.step(action)&#xA;            total_reward_episode[episode] += reward&#xA;&#xA;            modified_reward = next_state[0] + 0.5&#xA;&#xA;            if next_state[0] &amp;gt;= 0.5:&#xA;                modified_reward += 100&#xA;            elif next_state[0] &amp;gt;= 0.25:&#xA;                modified_reward += 20&#xA;            elif next_state[0] &amp;gt;= 0.1:&#xA;                modified_reward += 10&#xA;            elif next_state[0] &amp;gt;= 0:&#xA;                modified_reward += 5&#xA;&#xA;            q_values = estimator.predict(state).tolist()&#xA;&#xA;            if is_done:&#xA;                q_values[action] = modified_reward&#xA;                estimator.update(state, q_values)&#xA;                break&#xA;&#xA;            q_values_next = estimator.predict(next_state)&#xA;&#xA;            q_values[action] = modified_reward + gamma * torch.max(q_values_next).item()&#xA;&#xA;            estimator.update(state, q_values)&#xA;&#xA;            state = next_state&#xA;&#xA;&#xA;        print('Episode: {}, total reward: {}, epsilon: {}'.format(episode, total_reward_episode[episode], epsilon))&#xA;&#xA;        epsilon = max(epsilon * epsilon_decay, 0.01)&#xA;&#xA;n_state = env.observation_space.shape[0]&#xA;n_action = env.action_space.n&#xA;n_hidden = 50&#xA;lr = 0.001&#xA;dqn = DQN(n_state, n_action, n_hidden, lr)&#xA;&#xA;&#xA;n_episode = 1000&#xA;&#xA;total_reward_episode = [0] * n_episode&#xA;&#xA;q_learning(env, dqn, n_episode, gamma=.9, epsilon=.3)&#xA;&#xA;&#xA;&#xA;import matplotlib.pyplot as plt&#xA;plt.plot(total_reward_episode)&#xA;plt.title('Episode reward over time')&#xA;plt.xlabel('Episode')&#xA;plt.ylabel('Total reward')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12964" LastActivityDate="2020-07-23T17:26:11.440" Title="Classification or regression for deep Q learning" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;loss-functions&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22642" PostTypeId="1" CreationDate="2020-07-23T17:32:14.873" Score="6" ViewCount="163" Body="&lt;p&gt;It is proved that the Bellman update is a contraction (1).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here is the Bellman update that is used for Q-Learning:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{t+1}(s, a) = Q_{t}(s, a) + \alpha*(r(s, a, s') + \gamma  \max_{a^*} (Q_{t}(s',&#xA;   a^*)) - Q_t(s,a)) \tag{1} \label{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The proof of (\ref{1}) being contraction comes from one of the facts (the relevant one for the question) that max operation is non expansive; that is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lvert \max_a f(a)- \max_a g(a) \rvert \leq \max_a \lvert f(a) -  g(a) \rvert \tag{2}\label{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is also proved in a lot of places and it is pretty intuitive.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Consider the following Bellman update:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Q_{t+1}(s, a) = Q_{t}(s, a) + \alpha*(r(s, a, s') + \gamma  SAMPLE_{a^*} (Q_{t}(s', a^*)) - Q_t(s,a)) \tag{3}\label{3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$SAMPLE_a(Q(s, a))$&lt;/span&gt; samples an action with respect to the Q values (weighted by their Q values) of each action in that state.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is this new Bellman operation still a contraction?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is the SAMPLE operation non-expansive? It is, of course, possible to generate samples that will not satisfy equation (\ref{2}). I ask &lt;strong&gt;is it non-expansive in expectation?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My approach is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lvert\,\mathbb{E}_{a \sim Q}[f(a)] - \mathbb{E}_{a \sim Q}[g(a)]\, \rvert \leq \,\,\mathbb{E}_{a \sim Q}\lvert\,\,[f(a) - g(a)]\,\,\rvert \tag{4} \label{4} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Equivalently:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lvert\,\mathbb{E}_{a \sim Q}[f(a) - g(a)] \, \rvert \leq \,\,\mathbb{E}_{a \sim Q}\lvert\,\,[f(a) - g(a)]\,\,\rvert$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(\ref{4}) is true since:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lvert\,\mathbb{E}[X] \, \rvert \leq \,\,\mathbb{E} \,\,\lvert\,\,[X]\,\,\rvert $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;But, I am not sure if proving (\ref{4}) proves the theorem. Do you think that this is a legit proof that (\ref{3}) is a contraction.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;(If so; this would mean that stochastic policy q learning theoretically converges and we can have stochastic policies with regular q learning; and this is why I am interested.)&lt;/p&gt;&#xA;&lt;p&gt;Both intuitive answers and mathematical proofs are welcome.&lt;/p&gt;&#xA;" OwnerUserId="38818" LastEditorUserId="9947" LastEditDate="2020-07-25T08:28:13.937" LastActivityDate="2020-07-25T08:28:13.937" Title="Is the Bellman equation that uses sampling weighted by the Q values (instead of max) a contraction?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;proofs&gt;&lt;convergence&gt;&lt;bellman-equations&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22643" PostTypeId="1" AcceptedAnswerId="22644" CreationDate="2020-07-23T20:44:58.063" Score="3" ViewCount="229" Body="&lt;p&gt;Why does every neuron in a hidden layer of a multi-layer perceptron (MLP) typically have the same activation function as every other neuron in the same or other hidden layers (so I exclude the output layer, which typically has a different activation function) of the MLP? Is this a requirement, are there any advantages, or maybe is it just a rule of thumb?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2021-01-17T16:24:11.677" LastActivityDate="2021-01-17T16:24:11.677" Title="Why does every neuron in hidden layers of a multi-layer perceptron typically have the same activation function?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;activation-function&gt;&lt;multilayer-perceptron&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-02-03T11:25:06.030" ContentLicense="CC BY-SA 4.0" />
  <row Id="22644" PostTypeId="2" ParentId="22643" CreationDate="2020-07-23T21:08:58.547" Score="3" Body="&lt;p&gt;As you stated, it's popular to have some form of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;rectified linear unit (ReLU) activation&lt;/a&gt; in hidden layers and the output layer is often a &lt;a href=&quot;https://en.wikipedia.org/wiki/Softmax_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;softmax&lt;/a&gt; or sigmoid (depending also on the problem: multi-class or binary classification, respectively), which provides an output that can be viewed as a probability distribution.&lt;/p&gt;&#xA;&lt;p&gt;You could generalize this further to blocks of different activation functions within the same layer. This is something I've thought about, haven't done, but imagine has been attempted. In some sense, the idea here would be to allow for a subsection of the network to develop a representation that may not be feasible otherwise. These different representations within the same layer would then be unified by subsequent layers as we move closer to the output.&lt;/p&gt;&#xA;" OwnerUserId="5210" LastEditorUserId="2444" LastEditDate="2021-01-17T16:21:04.440" LastActivityDate="2021-01-17T16:21:04.440" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22645" PostTypeId="1" CreationDate="2020-07-24T04:35:24.853" Score="0" ViewCount="74" Body="&lt;p&gt;I want to build an open Source OCR just like MathPix. There is already a model to extract LaTex from the image by &lt;a href=&quot;https://github.com/harvardnlp/im2markup&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harverd NLP's im2markup&lt;/a&gt; but the problem is that their data has been trained and tested only for the &lt;strong&gt;Cropped Portion of Latex Only&lt;/strong&gt;. Somehow, I might be able to implement the code using their idea but the problem is that I have hybrid images of text as well as mathematical equation.&lt;/p&gt;&#xA;&lt;p&gt;My idea is that if I am able to find the portion/bounding box/ region where the equation is present, I can crop and then apply Harvard's model to get the latex. But the main problem is that How am I supposed to find the area of interest where equations are present??&lt;/p&gt;&#xA;&lt;p&gt;All the object recognition algorithms like &lt;code&gt;YOLO, Fast,Faster RCNN, MTCNN&lt;/code&gt; etc won't work here in this case as I am dealing with the text within text than any object in text  and vice versa.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please suggest a method hoe could I get the area where I can get te bounding box of equation and crop the image to get the LaTex from that image using &lt;code&gt;im2markup&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="36062" LastActivityDate="2020-07-24T04:35:24.853" Title="What could be the possible strategy and Deep Learning method that MathPix might be using for LaTex detection?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22646" PostTypeId="2" ParentId="22630" CreationDate="2020-07-24T06:41:01.667" Score="1" Body="&lt;p&gt;Here is something I've noticed about humans: We're bad at projecting the future with all of its 2nd, 3rd ... &lt;em&gt;N&lt;/em&gt; order effects, and we're REALLY bad a projecting and quantifying risk. So, I'm not sure that you'll get an answer that is anything more than either trivially true (&amp;quot;Chatbots will be commonplace&amp;quot;) or a correct but wasn't justified (&amp;quot;We'll explore the stars by sending out swarms of AI-enhanced satellites that will use their computing power to improve themselves since they are always on but have copious amounts of down-time&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;But if you want to limit the timeframe to the next decade and don't mind the previous caveat, here goes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chatbots will be commonplace. This will include ones that can handle conversation, like &lt;a href=&quot;https://www.youtube.com/watch?v=Xu0OKD3UL_k&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gridspace&lt;/a&gt; but can also handle multiple languages and &lt;a href=&quot;https://en.wikipedia.org/wiki/Code-switching&quot; rel=&quot;nofollow noreferrer&quot;&gt;code-switching&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;In professional use, AI will augment professionals (such as lawyers, physicians, teachers, and accountants) at the upper end (who will add value with strategy, experience, and research) and replace them at the lower end.&lt;/li&gt;&#xA;&lt;li&gt;In military use, they will be used for target identification, automated assessment of damage, threat notification, and forecasting.&lt;/li&gt;&#xA;&lt;li&gt;We'll explore the stars by sending out swarms of AI-enhanced satellites that will use their computing power to improve themselves since they are always on but have copious amounts down-time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I don't hold a lot of hope out for communication without error of to speakers of different languages as you can have two native fluent speakers who have known each other for years speak to each and still have them miscommunicate.&lt;/p&gt;&#xA;" OwnerUserId="30750" LastActivityDate="2020-07-24T06:41:01.667" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22647" PostTypeId="2" ParentId="22105" CreationDate="2020-07-24T07:02:27.107" Score="1" Body="&lt;p&gt;A chromosome in this case could be a set of filters, each extracting a different feature (analogous to Convolutional Neural Network). Your question doesn't say what you want to do with these features, so this solution is made under the assumption that there is a fitness function which would take these features as an input and output a score. Then, each gene is a parameter for a filter, each chromosome defines a set of such filters, which makes up an individual. A population is a set of such individuals.&lt;/p&gt;&#xA;" OwnerUserId="38671" LastActivityDate="2020-07-24T07:02:27.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22648" PostTypeId="1" CreationDate="2020-07-24T07:14:30.750" Score="0" ViewCount="44" Body="&lt;p&gt;I train a fully convoluted network for semantic segmentation. To each convolution blocks, I associate a module pruning feature maps to reduce the quantity of information generated by the network. From these modules, I extract a sparsity ratio, i.e. the number of feature maps containing information over the total number of feature maps in the given layer. Averaging this ratio at the network scale (given all modules), I obtain a &lt;em&gt;&lt;strong&gt;total_sparsity_ratio&lt;/strong&gt;&lt;/em&gt; which is added to the loss function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;total_sparsity_ratio&lt;/strong&gt;&lt;/em&gt; is differentiable, but since I use a step function to obtain calculate it, the gradient is 0.&lt;/p&gt;&#xA;&lt;p&gt;Is it acceptable to use a &amp;quot;non-trainable&amp;quot; regularizer in the loss function? Training results show slightly better results.&lt;/p&gt;&#xA;" OwnerUserId="28171" LastActivityDate="2020-07-24T07:14:30.750" Title="Non-trainable regularizer in loss function" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;regularization&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22649" PostTypeId="1" CreationDate="2020-07-24T08:54:46.553" Score="1" ViewCount="49" Body="&lt;p&gt;I want to teach a neural network to distinguish between different types of defects. For that, I generated images of fake-defects. The images of the fake-defect types are attached.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I0GO0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I0GO0.jpg&quot; alt=&quot;border&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iKkdP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iKkdP.jpg&quot; alt=&quot;Border&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UDJvo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UDJvo.jpg&quot; alt=&quot;Single Dirt&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b636R.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b636R.jpg&quot; alt=&quot;Scratch&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S9rqQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S9rqQ.jpg&quot; alt=&quot;Dust&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lNUBV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lNUBV.jpg&quot; alt=&quot;Dot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried many different network architectures now:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;resnet18&lt;/li&gt;&#xA;&lt;li&gt;squeezenet&lt;/li&gt;&#xA;&lt;li&gt;own architectures: a narrow network with broad layers and high dropout rates.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have to say that some of these defects have really random shapes, like the type single-dirt or multi-dirt. I imagine that the classification should not be as easy as I thought before, due to the lack of repetitive features within the defects. But I always feel like the network is learning some &amp;quot;weird&amp;quot; features, which do not occur in the test set, and the results are really frustrating. I felt like teaching binary images had way better results, which should IMO be not the case.&lt;/p&gt;&#xA;&lt;p&gt;Still, I feel like a neural network should be able to learn to distinguish them.&lt;/p&gt;&#xA;&lt;p&gt;Which kind of network architecture would you recommend to classify the images in the attachment?&lt;/p&gt;&#xA;" OwnerUserId="26857" LastEditorUserId="2444" LastEditDate="2020-07-25T12:22:38.973" LastActivityDate="2020-07-25T12:22:38.973" Title="Which neural network should I use to distinguish between different types of defects?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22650" PostTypeId="1" CreationDate="2020-07-24T08:58:11.450" Score="1" ViewCount="73" Body="&lt;p&gt;Kipf et al described in his paper that we can write graph convolution operation like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H_{t+1} = AH_tW_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the normalized adjacency matrix, &lt;span class=&quot;math-container&quot;&gt;$H_t$&lt;/span&gt; is the embedded representation of the nodes and &lt;span class=&quot;math-container&quot;&gt;$W_t$&lt;/span&gt; is the weight matrix.&lt;/p&gt;&#xA;&lt;p&gt;Now, can I imagine the same formula as first performing 2D convolution with fixed-size kernel over the whole feature space then multiply the result with the adjacency matrix? If this is the case, I think I can create graph convolution operation just using the Conv2D layer then performing simple matrix multiplication with adjacency matrix using PyTorch.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-07-24T20:17:48.580" LastActivityDate="2020-07-24T20:17:48.580" Title="Can I think graph convolution as 2D convolution like images?" Tags="&lt;deep-learning&gt;&lt;geometric-deep-learning&gt;&lt;convolution&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22651" PostTypeId="1" CreationDate="2020-07-24T09:54:10.420" Score="2" ViewCount="49" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/1609.04938v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;What You Get Is What You See: A Visual Markup Decompiler&lt;/a&gt;, the authors have proposed a method to extract the features from the CNN and then &lt;strong&gt;arrange those extracted features in a grid to pass into an RNN encoder&lt;/strong&gt;. Here's an illustration.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nX6jy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nX6jy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can easily extract features from either the existing model, like ResNet, VGG, or make a new CNN model easily as they have described in the paper.&lt;/p&gt;&#xA;&lt;p&gt;For example, let us suppose, I do this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;features = keras.applications.ResNet()(images_array) # just hypothetical&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I convert these images to the grid?? I am supposed to feed the output of the changed grid to an LSTM Encoder as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;keras.layers.LSTM()(grid) # again, hypothetical type&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I just want to know what the author means from changing the output in the grid format.&lt;/p&gt;&#xA;" OwnerUserId="36062" LastEditorUserId="2444" LastEditDate="2020-07-25T12:18:15.450" LastActivityDate="2020-07-25T12:18:15.450" Title="What is meant by &quot;arranging the final features of CNN in a grid&quot; and how to do it?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;terminology&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22653" PostTypeId="1" AcceptedAnswerId="22659" CreationDate="2020-07-24T11:13:54.367" Score="3" ViewCount="64" Body="&lt;p&gt;When training a relatively small DL model, which takes several hours to train, I typically start with some starting points from literature and then use a trial-and-error or grid-search approach to fine-tune the values of the hyper-parameters, in order to prevent overfitting and achieve sufficient performance.&lt;/p&gt;&#xA;&lt;p&gt;However, it is not uncommon for large models to have training time measured in days or weeks [&lt;a href=&quot;https://discovery.ucl.ac.uk/id/eprint/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], [&lt;a href=&quot;https://arxiv.org/pdf/1812.04948.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;], [&lt;a href=&quot;https://arxiv.org/pdf/1301.3781.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;p&gt;How are hyperparameters determined in such cases?&lt;/p&gt;&#xA;" OwnerUserId="38829" LastEditorUserId="2444" LastEditDate="2021-01-18T15:15:32.467" LastActivityDate="2021-01-18T15:15:32.467" Title="How are training hyperparameters determined for large models?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22654" PostTypeId="1" CreationDate="2020-07-24T12:33:55.693" Score="1" ViewCount="59" Body="&lt;p&gt;I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text?&lt;/p&gt;&#xA;&lt;p&gt;Just for reference, I am using the library &lt;a href=&quot;https://spacy.io/universe/project/neuralcoref&quot; rel=&quot;nofollow noreferrer&quot;&gt;spacy-neuralcoref&lt;/a&gt; in Python that is based on &lt;a href=&quot;https://www.aclweb.org/anthology/D16-1245.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning for Mention-Ranking Coreference Models&lt;/a&gt; by Kevin Clark and Christopher D. Manning, EMNLP 2016.&lt;/p&gt;&#xA;&lt;p&gt;Why am I asking about splitting the text?&lt;/p&gt;&#xA;&lt;p&gt;I am applying coreference to chapters of books (roughly 30 pages of text). All the examples I have seen show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right?&lt;/p&gt;&#xA;&lt;p&gt;I didn't check all databases that people use to test coreference but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A test Example&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;TST1-MUC3-0001&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;GUATEMALA CITY, 4 FEB 90 (ACAN-EFE) -- [TEXT] THE GUATEMALA ARMY&#xA;DENIED TODAY THAT GUERRILLAS ATTACKED THE &amp;quot;SANTO TOMAS&amp;quot; PRESIDENTIAL&#xA;FARM, LOCATED ON THE PACIFIC SIDE, WHERE PRESIDENT CEREZO HAS BEEN&#xA;STAYING SINCE 2 FEBRUARY.&lt;/p&gt;&#xA;&lt;p&gt;A REPORT PUBLISHED BY THE &amp;quot;CERIGUA&amp;quot; NEWS AGENCY -- MOUTHPIECE OF&#xA;THE GUATEMALAN NATIONAL REVOLUTIONARY UNITY (URNG) -- WHOSE MAIN&#xA;OFFICES ARE IN MEXICO, SAYS THAT A GUERRILLA COLUMN ATTACKED THE FARM&#xA;2 DAYS AGO.&lt;/p&gt;&#xA;&lt;p&gt;HOWEVER, ARMED FORCES SPOKESMAN COLONEL LUIS ARTURO ISAACS SAID&#xA;THAT THE ATTACK, WHICH RESULTED IN THE DEATH OF A CIVILIAN WHO WAS&#xA;PASSING BY AT THE TIME OF THE SKIRMISH, WAS NOT AGAINST THE FARM, AND&#xA;THAT PRESIDENT CEREZO IS SAFE AND SOUND.&lt;/p&gt;&#xA;&lt;p&gt;HE ADDED THAT ON 3 FEBRUARY PRESIDENT CEREZO MET WITH THE&#xA;DIPLOMATIC CORPS ACCREDITED IN GUATEMALA.&lt;/p&gt;&#xA;&lt;p&gt;THE GOVERNMENT ALSO ISSUED A COMMUNIQUE DESCRIBING THE REBEL REPORT&#xA;AS &amp;quot;FALSE AND INCORRECT,&amp;quot; AND STRESSING THAT THE PRESIDENT WAS NEVER&#xA;IN DANGER.&lt;/p&gt;&#xA;&lt;p&gt;COL ISAACS SAID THAT THE GUERRILLAS ATTACKED THE &amp;quot;LA EMINENCIA&amp;quot;&#xA;FARM LOCATED NEAR THE &amp;quot;SANTO TOMAS&amp;quot; FARM, WHERE THEY BURNED THE&#xA;FACILITIES AND STOLE FOOD.&lt;/p&gt;&#xA;&lt;p&gt;A MILITARY PATROL CLASHED WITH A REBEL COLUMN AND INFLICTED THREE&#xA;CASUALTIES, WHICH WERE TAKEN AWAY BY THE GUERRILLAS WHO FLED TO THE&#xA;MOUNTAINS, ISAACS NOTED.&lt;/p&gt;&#xA;&lt;p&gt;HE ALSO REPORTED THAT GUERRILLAS KILLED A PEASANT IN THE CITY OF&#xA;FLORES, IN THE NORTHERN EL PETEN DEPARTMENT, AND BURNED A TANK TRUCK.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="38831" LastEditorUserId="2444" LastEditDate="2020-07-25T12:08:36.163" LastActivityDate="2020-07-25T12:08:36.163" Title="Is there an optimal way to split the text into small parts when working with co-reference resolution?" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22655" PostTypeId="1" CreationDate="2020-07-24T13:51:16.690" Score="1" ViewCount="48" Body="&lt;p&gt;I am trying to classify tampered, pristine images from set of images, in that I have built a network in which I would divide the image into multiple overlapping patches and then classify them into pristine or fake(based on the probability outputs), but now I want extend the same to Image level. That is I want to build some model or some rule over output probabilities of patches of each image to get probability that the image is fake or pristine.&lt;/p&gt;&#xA;&lt;p&gt;ways I am thinking to do is -&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Build a shallow network over the probabilities of the patch probabilities. In this case problem is all images are of different shape&lt;/li&gt;&#xA;&lt;li&gt;Apply a ML classifier (something like Logistic Regression), with output probabilities by appending zeros to the output probability vector generated so that all image has same sized probability vector as input&lt;/li&gt;&#xA;&lt;li&gt;generate a mask using patches and then build  a simple classification network over the masks using original image labels.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I can't really say which among the above three is better or worse, I don't even know the possibility of the above three. (Kind of hit a roadblock in thinking)&lt;/p&gt;&#xA;&lt;p&gt;Now the question am I thinking in right direction, what would be better among the ideas I am considering and why. Is there anything better than what I am thinking. It would helpful in suggesting some resources.&lt;/p&gt;&#xA;" OwnerUserId="38727" LastActivityDate="2020-07-24T13:51:16.690" Title="Extending patch based image classification into image classification" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;image-segmentation&gt;&lt;feature-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22657" PostTypeId="1" CreationDate="2020-07-24T17:54:52.897" Score="2" ViewCount="86" Body="&lt;p&gt;Recently I was reading this paper &lt;a href=&quot;https://arxiv.org/pdf/1801.07455.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Skeleton Based Action RecognitionUsing Spatio Temporal Graph Convolution&lt;/a&gt;. In this paper, the authors claim (below equation (\ref{9})) that we can perform graph convolution with the following formula&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{f}_{o u t}=\mathbf{\Lambda}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I}) \mathbf{\Lambda}^{-\frac{1}{2}} \mathbf{f}_{i n} \mathbf{W} \label{9}\tag{9}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;using the standard 2d convolution with kernels of shape &lt;span class=&quot;math-container&quot;&gt;$1 \times \Gamma$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\Gamma$&lt;/span&gt; is defined under equation 6 of the paper), and then multiplying it with the normalised adjacency matrix&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{\Lambda}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I}) \mathbf{\Lambda}^{-\frac{1}{2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the past few days, I was thinking about his claim but I can't find an answer. Does anyone read this paper and can help me to find it out, please?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-07-24T20:28:47.310" LastActivityDate="2020-11-01T01:29:13.677" Title="Why can we perform graph convolution using the standard 2d convolution with $1 \times \Gamma$ kernels?" Tags="&lt;computer-vision&gt;&lt;geometric-deep-learning&gt;&lt;convolution&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="25" ContentLicense="CC BY-SA 4.0" />
  <row Id="22658" PostTypeId="1" CreationDate="2020-07-24T17:57:25.503" Score="0" ViewCount="45" Body="&lt;p&gt;I'm training a deep learning model. After each epoch I measure the performance of the model on validation set. Here is how the performance looks like while training:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OghOW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OghOW.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's a binary classification task with cross entropy loss function. I use argmax at the last layer to do the prediction and measure precision and recall. Note, the number of positive and negative samples within each mini batch are almost the same (mini-batches are balanced). Any idea about possible reasons that the model is behaving like this? And how I can improve the recall as well as making it more stable like the precision?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="38844" LastActivityDate="2020-07-24T17:57:25.503" Title="Possible reasons that validation recall is fluctuating across different epochs but the precision is stable?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22659" PostTypeId="2" ParentId="22653" CreationDate="2020-07-24T19:11:29.807" Score="3" Body="&lt;p&gt;In general, it is definitely very computationally expensive, so an exhaustive search is not performed in practice. However, there are some recent approaches for determining whether the architecture is &amp;quot;fine&amp;quot; without training the neural network first - by looking at the covariance matrix after forwarding the data, for example, in a recent paper &lt;a href=&quot;https://arxiv.org/abs/2006.04647&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Architecture Search without Training&lt;/a&gt;. However, such an approach is very limited.&lt;/p&gt;&#xA;" OwnerUserId="38846" LastEditorUserId="2444" LastEditDate="2021-01-18T15:12:32.217" LastActivityDate="2021-01-18T15:12:32.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22661" PostTypeId="2" ParentId="22589" CreationDate="2020-07-25T01:22:33.073" Score="3" Body="&lt;ul&gt;&#xA;&lt;li&gt;I'm not sure any intelligent mechanism can be entirely free of symbolic logic.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Even where a decision is statistically based, a machine that takes actions must include some form of:&lt;br&gt;&lt;br&gt;&#xA;IF {some condition} &lt;br&gt;&#xA;THEN {some action}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As to the popularity of newly proven statistical AI methods (ANN and genetic algorithms), this derives from the greater utility they demonstrate at ever more complex problems compared to expert systems (&amp;quot;good old fashioned AI&amp;quot;) for problems that do not have a mathematical solution.&lt;/p&gt;&#xA;&lt;p&gt;(i.e. the statistical approach for 3x3 Tic-tac-toe is overkill and unnecessary b/c the 3x3 form is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot; rel=&quot;nofollow noreferrer&quot;&gt;solved game&lt;/a&gt;. But for larger-order gameboards &lt;span class=&quot;math-container&quot;&gt;$m*m$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$m*n$&lt;/span&gt;, the n-dimensional game, &lt;span class=&quot;math-container&quot;&gt;$m^n$&lt;/span&gt;, barring a mathematical solution that applies to every variation, ANN is way to go.)&lt;/p&gt;&#xA;&lt;p&gt;The main issue with expert systems, no matter how complex, is &amp;quot;brittleness&amp;quot;—inability to adapt to changes without human programmer intervention.  As conditions change, the mechanism demonstrates diminishing utility, or simply &amp;quot;breaks&amp;quot; (invalid input as an example.)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The amount of human effort required to create DeepBlue was monumental, which is why it took decades to achieve it goal, funded by a large corporation with a history of basic research.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Compare to a simple ANN that can be trained to achieve the same goal in an extremely short timeframe.&lt;/p&gt;&#xA;&lt;p&gt;It's possible future artificial general intelligences of whatever strength would involve statistical AI programming and adapting its own symbolic functions.&lt;/p&gt;&#xA;&lt;p&gt;Finally, symbolic AI is still vastly more widely implemented than statistical AI, in that all of the basic functions of modern computing, all of the mathematical functions, all traditional software and apps, utilize symbolic logic, even if the high level function is statistically driven. This will likely always be the case.&lt;/p&gt;&#xA;&lt;p&gt;Thus, in terms of what method is best for a given problem, it really depends on the nature/structure of the problem, it's solvability or even &lt;a href=&quot;https://en.wikipedia.org/wiki/Decidability_(logic)&quot; rel=&quot;nofollow noreferrer&quot;&gt;decidability&lt;/a&gt;, as well as its &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability&quot; rel=&quot;nofollow noreferrer&quot;&gt;tractability&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-07-28T00:30:12.330" LastActivityDate="2020-07-28T00:30:12.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22662" PostTypeId="1" CreationDate="2020-07-25T03:12:13.437" Score="1" ViewCount="40" Body="&lt;p&gt;I came across a &lt;a href=&quot;https://arxiv.org/abs/1903.10077&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; that describes its model architecture in the following way.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our TRIL network is a &lt;strong&gt;two-channel network&lt;/strong&gt; jointly trained to predict the expert’s action given state and the system’s next state transition given state and expert action. The training procedure of TRIL is similar to that of a multi- channel supervised classifier with regularization. Let &lt;span class=&quot;math-container&quot;&gt;$\theta_{π_0}$&lt;/span&gt; be the parameters of TRIL and &lt;span class=&quot;math-container&quot;&gt;$L_{ce}$&lt;/span&gt; be the cross entropy loss for predicting expert’s action and &lt;span class=&quot;math-container&quot;&gt;$L_{mse}$&lt;/span&gt; be the mean squared error loss on predicting next state given current state and the expert’s action&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The loss function is given in the following manner&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L(\theta_{\pi_0}) = L_{ce}(a, \pi_0(s)) + \lambda L_{mse}(T_{\pi_0}(s,a),s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TRIL is a dual- channel network that &lt;strong&gt;shares certain hidden layers&lt;/strong&gt; and &lt;strong&gt;jointly predicts&lt;/strong&gt; expert action(a) and state transitions(s’)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not sure what a dual channel network means and what does it mean when it is able to jointly predict two outputs ? It seems something similar to a multi-task learning since there is shared hidden layers and different &amp;quot;task&amp;quot; prediction but i am not too sure of that either.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-07-25T03:12:13.437" Title="What is a multi channel supervised classifier?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22663" PostTypeId="2" ParentId="22631" CreationDate="2020-07-25T03:32:36.850" Score="2" Body="&lt;p&gt;The problem with certain activation functions, such as the sigmoid, is that they squash the input to a finite interval (i.e. they are sometimes classified as &lt;em&gt;saturating activation functions&lt;/em&gt;). For example, the sigmoid function has codomain &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, as you can see from the illustration below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fhg3s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fhg3s.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This property/behaviour can lead to the vanishing gradient problem (which was one of the problems that Sepp Hochreiter, the author of the LSTM, was trying to solve in the context of recurrent neural networks, when developing the LSTM, along with his advisor, Schmidhuber).&lt;/p&gt;&#xA;&lt;p&gt;Empirically, people have noticed that ReLU can avoid this vanishing gradient problem. See e.g. &lt;a href=&quot;https://machinelearningmastery.com/how-to-fix-vanishing-gradients-using-the-rectified-linear-activation-function/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;. The paper &lt;a href=&quot;http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Sparse Rectifier Neural Networks&lt;/a&gt; provides more details about the advantage of ReLUs (aka rectifiers), so you may want to read it. However, ReLUs can also suffer from another (opposite) problem, i.e. the exploding gradient problem. Nevertheless, there are several ways to combat this issue. See e.g. &lt;a href=&quot;https://machinelearningmastery.com/exploding-gradients-in-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That being said, I am not an expert on residual networks, but I think that they used the ReLU to further avoid the vanishing gradient problem. &lt;a href=&quot;https://ai.stackexchange.com/a/17785/2444&quot;&gt;This answer&lt;/a&gt; (that I gave some time ago) should give you some intuition about why residual networks can avoid the vanishing gradient problem.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-25T03:38:16.443" LastActivityDate="2020-07-25T03:38:16.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22664" PostTypeId="1" CreationDate="2020-07-25T07:58:24.607" Score="2" ViewCount="95" Body="&lt;p&gt;Is there a type of neural network that can be fed patterns to train itself on to complete new patterns that it has not seen before?&lt;/p&gt;&#xA;&lt;p&gt;What I'm trying to do is train a neural network to transform an image into another image. The image may be slightly different each time (denoted with different lines in the shapes) but a human would get the idea of how the new images should look. I'd like to make a network that can learn how to learn what comes next and then predict the rest of the sequence from the first part of a new sequence.&lt;/p&gt;&#xA;&lt;p&gt;Taking the picture below as an example. The neural network would be fed the patterns in grey and learn how to predict the next ones in the sequence. Then the user would put the blue shapes into the network and hope to get the green ones out.&lt;/p&gt;&#xA;&lt;p&gt;Is there a neural network that could perform this type of function of completing a pattern based on only a small number of examples to start the pattern based on the other patterns it has seen?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hrKO3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hrKO3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; Corrected image and added more context&lt;/p&gt;&#xA;" OwnerUserId="38857" LastEditorUserId="36578" LastEditDate="2020-11-10T14:08:36.403" LastActivityDate="2020-11-10T14:08:36.403" Title="What kind of neural network can be trained to recognise patterns?" Tags="&lt;neural-networks&gt;&lt;pattern-recognition&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22665" PostTypeId="1" CreationDate="2020-07-25T11:31:15.277" Score="1" ViewCount="57" Body="&lt;p&gt;Reading the paper 'Reinforcement Learning for FX trading 'at &lt;a href=&quot;https://stanford.edu/class/msande448/2019/Final_reports/gr2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stanford.edu/class/msande448/2019/Final_reports/gr2.pdf&lt;/a&gt; it states:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While our end goal is to be able to make decisions on a universal time&#xA;scale, in order to apply a reinforcement learning approach to this&#xA;problem with rewards that do not occur at each step, we formulate the&#xA;problem with a series of episodes. In each episode, which we designate&#xA;to be one hour long, the agent will learn the make decisions to&#xA;maximize the reward (return) in that episode, given the time series&#xA;features we have.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This may be a question for the authors but is it not better in RL to apply rewards at each time step instead of &amp;quot;rewards that do not occur at each step&amp;quot;? If apply rewards at each time step then the RL algorithm will achieve better convergence properties as a result of learning at smaller time intervals rather than waiting for &amp;quot;one hour&amp;quot;. Why not apply rewards at each time step?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastActivityDate="2020-07-25T11:31:15.277" Title="When to apply reward for time series data?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22666" PostTypeId="2" ParentId="22619" CreationDate="2020-07-25T16:35:11.637" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;How to fix the network above to auto-classify XOR data, in unsupervised manner?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This cannot be done, except accidentally.&lt;/p&gt;&#xA;&lt;p&gt;Unsupervised learning cannot replace or emulate supervised learning.&lt;/p&gt;&#xA;&lt;p&gt;As a thought experiment, consider why you would expect the network to discover XOR, when simply considering outputs rounded to binary, you could equally find AND, OR, NAND, NOR or any of the 16 possible mapping functions from input to output. All of the possible maps are equally valid functions, and there is no reason why a discovered function mapping should become any one of them by preference.&lt;/p&gt;&#xA;&lt;p&gt;Unsupervised learning approaches typically find patterns that optimise some measure across the dataset without using labelled data. Clustering is a classic example, and auto-encoding is sometimes considered unsupervised because there is no separate label (although the term &lt;em&gt;self-supervised&lt;/em&gt; is also used, because there is still technically a label used in training, it happens to equal the input).&lt;/p&gt;&#xA;&lt;p&gt;You cannot use auto-encoding approaches here anyway, because XOR needs to map &lt;span class=&quot;math-container&quot;&gt;$\{0,1\} \times \{0,1\} \rightarrow \{0,1\}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You could potentially use a loss function based on how close to a 0 &lt;em&gt;or&lt;/em&gt; 1 any output is. That should cause the network to converge to one of the 16 possible binary functions, based on random initialisation. For example, you could use &lt;span class=&quot;math-container&quot;&gt;$y(1-y)$&lt;/span&gt; as the loss.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-07-25T16:35:11.637" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22667" PostTypeId="1" CreationDate="2020-07-25T17:19:03.437" Score="3" ViewCount="155" Body="&lt;p&gt;&lt;a href=&quot;https://lilianweng.github.io/lil-log/2017/08/20/from-GAN-to-WGAN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this article&lt;/a&gt; I am reading:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$D_{KL}$&lt;/span&gt; gives us inifity when two distributions are disjoint. The value of &lt;span class=&quot;math-container&quot;&gt;$D_{JS}$&lt;/span&gt; has sudden jump, not differentiable at &lt;span class=&quot;math-container&quot;&gt;$\theta=0$&lt;/span&gt;. Only Wasserstein metric provides a smooth measure, which is super helpful for a stable learning process using gradient descents.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why is this important for a stable learning process? I have also the feeling this is also the reason for mode collapse in GANs, but I am not sure.&lt;/p&gt;&#xA;&lt;p&gt;The Wasserstein GAN paper also talks about it obviously, but I think I am missing a point. Does it say JS does not provide a usable gradient? What exactly does that mean?&lt;/p&gt;&#xA;" OwnerUserId="38812" LastEditorUserId="2444" LastEditDate="2021-01-25T19:04:51.297" LastActivityDate="2021-01-25T19:04:51.297" Title="What is the reason for mode collapse in GAN as opposed to WGAN?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;kl-divergence&gt;&lt;wasserstein-metric&gt;&lt;wasserstein-gan&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22668" PostTypeId="1" CreationDate="2020-07-25T20:45:30.330" Score="0" ViewCount="97" Body="&lt;p&gt;I've been trying to implement a Multilayer Perceptron Network using java language with the ultimate goal of creating and teaching a neural network to recognize handwritten digits. Pretty simple and straight forward. &lt;a href=&quot;https://github.com/ark1375/op-charmander&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is the link to the repository of the project. I tried to write everything as simple as I could. When the class was ready to test, I provided the network with suitable inputs (around 3000 training samples) but the network doesn't seem to converge to a minimum point. Instead for every input it just gives a vector output of 1s.&amp;quot;&#xA;I think that the problem is with the implementation of the backpropagation algorithm, though I'm not 100% sure of it. I use a book called &lt;em&gt;&lt;strong&gt;Fundamentals of Computational Intelligence&lt;/strong&gt;&lt;/em&gt; by &lt;em&gt;Keller, Liu, and Fogel&lt;/em&gt; as the source for all the maths behind ANN. I try to explain my code and my understanding of the math as vividly as I can.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Backpropagation code:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qAAzW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qAAzW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sorry that I didn't type the code. I needed to refer to some specific lines and I couldn't if I wrote the code down.&lt;/p&gt;&#xA;&lt;p&gt;So the algorithm begins from line 392 with a loop that calculates the error for each neuron. &lt;code&gt;this.weigths.length&lt;/code&gt; is the number of network's layers. So I'm iterating in layers with the first loop and as you can see I'm starting from the output layer and work my way back to input layer.&lt;br /&gt;&#xA;In line 394 I'm referring that if the program is iterating on the neurons of the output layer, the error of each node is calculatable by subtracting the desired output of the network from the actual output, After that, when the iteration of layers gets to the hidden layers of the network, the error for each neuron is calculatable by the following rule:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle\sum_{l}^{\#(l)} e_l \phi_l'(v_{l,k})w_{lj}$$&lt;/span&gt;&#xA;In which &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; represents a node in the next layer of the network, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; represents the input data (for a given input data &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; represents the current node that we are calculating the error for.&#xA;Likewise &lt;span class=&quot;math-container&quot;&gt;$\phi(*)$&lt;/span&gt; is the activation function so &lt;span class=&quot;math-container&quot;&gt;$\phi'(*)$&lt;/span&gt; is its derevitive, &lt;span class=&quot;math-container&quot;&gt;$e_l$&lt;/span&gt; is the error of node &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$w_{lj}$&lt;/span&gt; is the weight of the connection which connects node &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; to node &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; and finally &lt;span class=&quot;math-container&quot;&gt;$v_{l,k}$&lt;/span&gt; is the signal of node &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; given a sample &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; in the input.&lt;br /&gt;&#xA;I'm saving all of these in a double array called &lt;code&gt;errors&lt;/code&gt;.&lt;br /&gt;&#xA;Using this error array, from line 414 forward, the program is calculating &lt;span class=&quot;math-container&quot;&gt;$\Delta w_{ij}$&lt;/span&gt; which is a correction for the weight of the connection that connects node &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta w_{ij} = \alpha e_{(j,k)} \phi'_j(v_{(j,k)}) y_{(i,k)} $$&lt;/span&gt;&#xA;note that &lt;span class=&quot;math-container&quot;&gt;$y_{(i,k)}$&lt;/span&gt; represents the signal output of neuron &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; for a given sample &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;&lt;br /&gt;&#xA;In the program &lt;code&gt;outputs&lt;/code&gt; is &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, &lt;code&gt;fields&lt;/code&gt; is &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, &lt;code&gt;errors&lt;/code&gt; is &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, &lt;code&gt;activation_function_derivative&lt;/code&gt; is &lt;span class=&quot;math-container&quot;&gt;$\phi'$&lt;/span&gt; , &lt;code&gt;learningRate&lt;/code&gt; is &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;code&gt;weights&lt;/code&gt; are &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;&lt;br /&gt;&#xA;I personally found these two pictures verry helpful&#xA;&lt;a href=&quot;https://i.stack.imgur.com/40t6a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/40t6a.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/duFKU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/duFKU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now with all of this said, I don't know where I made a mistake in understanding the MLP's math that my program is not working.&#xA;It'll be of much help if you could help me a little bit. I've been going blind for 3 days staring at monitors and textbooks.&lt;/p&gt;&#xA;" OwnerUserId="38866" LastActivityDate="2020-07-25T20:45:30.330" Title="Backpropagation implementation with Java" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22669" PostTypeId="5" CreationDate="2020-07-26T02:57:25.127" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-26T02:57:25.127" LastActivityDate="2020-07-26T02:57:25.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22670" PostTypeId="4" CreationDate="2020-07-26T02:57:25.127" Score="0" Body="For questions related to the Kullback–Leibler (KL) divergence, which is a measure (that is not a metric, but it is pre-metric, because it does not satisfy all properties of metrics, i.e. it is not symmetric) of divergence (or distance) between two probability measures (density functions, or mass functions), which is commonly used in many machine learning settings, e.g. in the context of variational auto-encoders (VAES)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-07-26T02:57:25.127" LastActivityDate="2020-07-26T02:57:25.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22671" PostTypeId="1" AcceptedAnswerId="22672" CreationDate="2020-07-26T04:39:28.857" Score="4" ViewCount="768" Body="&lt;p&gt;AIs like Siri and Alexa respond to their names being called. How does the system recognize the name by ignoring all the other words that have been said before their name? For example, &amp;quot;Hey Siri&amp;quot; would trigger Siri to start listening for commands, but if a user said &amp;quot;hey how are you hey Siri&amp;quot; the system will ignore &amp;quot;hey how are you&amp;quot; but trigger the system to &amp;quot;hey Siri&amp;quot;. Is it because their listening function reloads in milliseconds or even nanoseconds, or is there a different way it works?&lt;/p&gt;&#xA;" OwnerUserId="38873" LastEditorUserId="-1" LastEditDate="2020-07-28T00:25:29.133" LastActivityDate="2020-07-28T00:25:29.133" Title="How do AIs like Siri and Alexa respond to their names being called?" Tags="&lt;applications&gt;&lt;speech-recognition&gt;&lt;siri&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22672" PostTypeId="2" ParentId="22671" CreationDate="2020-07-26T07:39:29.583" Score="10" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is it because their listening function reloads in milliseconds or even nanoseconds&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, it expects the keyword to start every moment of time and it ignores the rest.&lt;/p&gt;&#xA;&lt;p&gt;Overall, the algorithm is described here, you can read for details:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearning.apple.com/research/hey-siri&quot; rel=&quot;noreferrer&quot;&gt;https://machinelearning.apple.com/research/hey-siri&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3459" LastEditorUserId="2444" LastEditDate="2020-07-26T13:04:38.743" LastActivityDate="2020-07-26T13:04:38.743" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22673" PostTypeId="1" AcceptedAnswerId="22715" CreationDate="2020-07-26T08:12:32.787" Score="5" ViewCount="1419" Body="&lt;p&gt;When I studied neural networks, parameters were learning rate, batch size etc. But even GPT3's ArXiv paper does not mention anything about what exactly the parameters are, but gives a small hint that they might just be sentences.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LWEEQ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LWEEQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Even tutorial sites like &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2019/07/openai-gpt2-text-generator-python/&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt; start talking about the usual parameters, but also say &lt;code&gt;&amp;quot;model_name: This indicates which model we are using. In our case, we are using the GPT-2 model with 345 million parameters or weights&amp;quot;&lt;/code&gt;. So are the 175 billion &amp;quot;parameters&amp;quot; just neural weights? Why then are they called parameters? &lt;a href=&quot;https://arxiv.org/pdf/2005.14165.pdf&quot; rel=&quot;noreferrer&quot;&gt;GPT3's paper&lt;/a&gt; shows that there are only 96 layers, so I'm assuming it's not a very deep network, but extremely fat. Or does it mean that each &amp;quot;parameter&amp;quot; is just a representation of the encoders or decoders?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dthvC.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dthvC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;An excerpt from &lt;a href=&quot;https://minimaxir.com/2019/09/howto-gpt2/&quot; rel=&quot;noreferrer&quot;&gt;this website&lt;/a&gt; shows tokens:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this case, there are two additional parameters that can be passed&#xA;to gpt2.generate(): truncate and include_prefix. For example, if each&#xA;short text begins with a &amp;lt;|startoftext|&amp;gt; token and ends with a&#xA;&amp;lt;|endoftext|&amp;gt;, then setting prefix='&amp;lt;|startoftext|&amp;gt;',&#xA;truncate=&amp;lt;|endoftext|&amp;gt;', and include_prefix=False, and length is&#xA;sufficient, then gpt-2-simple will automatically extract the shortform&#xA;texts, even when generating in batches.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So are the parameters various kinds of tokens that are manually created by humans who try to fine-tune the models? Still, 175 billion such fine-tuning parameters is too high for humans to create, so I assume the &amp;quot;parameters&amp;quot; are auto-generated somehow.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;noreferrer&quot;&gt;attention-based&lt;/a&gt; paper mentions the &lt;a href=&quot;http://jalammar.github.io/illustrated-gpt2/&quot; rel=&quot;noreferrer&quot;&gt;query-key-value weight&lt;/a&gt; matrices as the &amp;quot;parameters&amp;quot;. &lt;strong&gt;Even if it is these weights, I'd just like to know what kind of a process generates these parameters, who chooses the parameters and specifies the relevance of words? If it's created automatically, how is it done?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9268" LastEditorUserId="9268" LastEditDate="2020-07-26T09:04:44.753" LastActivityDate="2020-07-28T09:50:31.090" Title="What exactly are the &quot;parameters&quot; in GPT-3's 175 billion parameters and how are they chosen/generated?" Tags="&lt;recurrent-neural-networks&gt;&lt;open-ai&gt;&lt;transformer&gt;&lt;attention&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22674" PostTypeId="1" CreationDate="2020-07-26T08:38:22.170" Score="2" ViewCount="56" Body="&lt;p&gt;In my research, I remember to have read that, in case of an environment which can be modeled by partially observable MDP, there are no convergence guarantees (unfortunately, I do not find the paper anymore and I would appreciate if someone can post the link to the reference).&lt;/p&gt;&#xA;&lt;p&gt;If the performance of an RL agent in a partially observable environment is &amp;quot;good&amp;quot; (i.e. the agent does pretty well in achieving its goal), is this likely only accidental or due to chance?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-07-26T22:10:46.187" LastActivityDate="2020-07-26T22:10:46.187" Title="If the performance of an RL agent in a partially observable environment is &quot;good&quot;, is this likely only accidental?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;convergence&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22676" PostTypeId="1" CreationDate="2020-07-26T14:13:41.207" Score="4" ViewCount="456" Body="&lt;p&gt;I have an NLP model for answer-extraction. So, basically, I have a paragraph and a question as input, and my model extracts the span of the paragraph that corresponds to the answer to the question.&lt;/p&gt;&#xA;&lt;p&gt;I need to know how to compute the F1 score for such models. It is the standard metric (along with Exact Match) used in the literature to evaluate question-answering systems.&lt;/p&gt;&#xA;" OwnerUserId="23350" LastEditorUserId="2444" LastEditDate="2021-01-26T15:35:32.910" LastActivityDate="2021-05-02T04:38:03.230" Title="How is the F1 score calculated in a question-answering system?" Tags="&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;metric&gt;&lt;question-answering&gt;&lt;f1-score&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22678" PostTypeId="1" AcceptedAnswerId="22680" CreationDate="2020-07-26T15:43:39.957" Score="2" ViewCount="71" Body="&lt;p&gt;I'm new to reinforcement learning. I have a problem where an action is composed of an order (rod with a required length) and an item from a warehouse (an existing rod with a certain length, which will be cut to the desired length and the remainder put back to the warehouse).&lt;/p&gt;&#xA;&lt;p&gt;I imagine my state as two lists of a defined size: orders and warehouse, and my action as an index from the first list and an index from the second list. However, I have only worked with environments where it was only possible to pick single action and I'm not sure how to deal with two indexes. I'm not sure how DQN architecture should look like to give me such action.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone validate my general idea and help me find a solution? Or maybe just point me to some papers where similar problems are described?&lt;/p&gt;&#xA;" OwnerUserId="38881" LastEditorUserId="2444" LastEditDate="2020-07-26T22:06:10.913" LastActivityDate="2020-07-26T22:08:20.720" Title="Reinforcement learning with action consisting of two discrete values" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;markov-decision-process&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22680" PostTypeId="2" ParentId="22678" CreationDate="2020-07-26T19:04:33.077" Score="3" Body="&lt;p&gt;You would still be picking a single action. Your action space is now &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A} = \mathcal{O} \times \mathcal{I}$&lt;/span&gt; where I've chosen &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}$&lt;/span&gt; to be the set of possible orders from your problem and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{I}$&lt;/span&gt; to be the set of possible items.&lt;/p&gt;&#xA;&lt;p&gt;Provided both of these sets are finite, then you should still be able to approach this problem with DQN. Theoretically, this should be easy to see, as any element from &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is still a single element it just happens that this element is now a tuple.&lt;/p&gt;&#xA;&lt;p&gt;From a programming point of view, let's consider the simple example of cartpole, where the possible actions are left and right. Your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function obviously won't know the meanings of 'left' and 'right', you just assign it to an element of a vector, i.e. your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function would output a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; with e.g. the first element corresponding to the score for 'left' and the second element corresponding to the score for 'right'. This is still the case in your problem formulation, you will just have a &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function that outputs a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^d$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$d = |\mathcal{A}|$&lt;/span&gt; - you would just have to make sure you know which element corresponds to which action.&lt;/p&gt;&#xA;&lt;p&gt;Also, there is the possibility that this approach could leave you with a large dimensional vector output, which I imagine would probably mean you'd need more simulations to properly explore the action space.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2020-07-26T22:08:20.720" LastActivityDate="2020-07-26T22:08:20.720" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22681" PostTypeId="1" CreationDate="2020-07-26T22:05:34.390" Score="1" ViewCount="44" Body="&lt;p&gt;I built a DRL model to trade stocks in the financial market but the number of observations is relatively small and I would like to increase it by training the same model with stocks from several different companies. My problem is that I don't know what is the correct way to do this since the price series is a time series. Someone to enlighten me? I have read articles that show that this is possible but none that say how.&lt;/p&gt;&#xA;" OwnerUserId="38886" LastEditorUserId="2444" LastEditDate="2020-07-26T22:17:34.490" LastActivityDate="2020-07-26T22:17:34.490" Title="How can I build a deep reinforcement learning model that can be trained with multiple time series datasets" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22682" PostTypeId="1" AcceptedAnswerId="22683" CreationDate="2020-07-26T22:41:37.287" Score="4" ViewCount="89" Body="&lt;p&gt;I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? e.g. ReLU, ELUs, etc. I'd appreciate any statistic on this or insight.&lt;/p&gt;&#xA;" OwnerUserId="36341" LastEditorUserId="38892" LastEditDate="2020-07-27T04:08:21.643" LastActivityDate="2020-07-27T04:08:21.643" Title="What activation functions are currently popular?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22683" PostTypeId="2" ParentId="22682" CreationDate="2020-07-27T02:32:52.417" Score="2" Body="&lt;p&gt;Currently both ReLU and ELUs are the most popular activation functions (AF) used in neural nets (NNs). This is because they eliminate the vanishing gradient problem that causes major problems in the training process and degrades the accuracy and performance of NN models.&lt;/p&gt;&#xA;&lt;p&gt;Also these AFs, more specifically ReLU, are very fast learning AF which make them even more useful in research.&lt;/p&gt;&#xA;&lt;p&gt;However, depending on type of NN you working on, it's always a good practice to pay some attentions to new studies.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T02:32:52.417" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22684" PostTypeId="2" ParentId="22676" CreationDate="2020-07-27T02:50:07.093" Score="0" Body="&lt;p&gt;It really depends on what you are looking for your model to do. For example do false negative or false positive really cost your research (or your business)? Also it's very important to consider your label (class) distribution.&lt;/p&gt;&#xA;&lt;p&gt;If you just want to acheive the highest accuracy, and you don't have any issue with your class distribution (that I beleive you probably don't in your case) then accuracy works pretty good.&lt;/p&gt;&#xA;&lt;p&gt;F1 score might be a better option to use if you need to seek a balance between precision and recall and there is an uneven class distribution.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T02:50:07.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22685" PostTypeId="2" ParentId="22664" CreationDate="2020-07-27T03:20:10.950" Score="1" Body="&lt;p&gt;The idea is simple, but it requires some time to develop.&lt;/p&gt;&#xA;&lt;p&gt;Assumption: I am assuming in your problem the final model will have seen all possible shapes.&lt;/p&gt;&#xA;&lt;p&gt;What your algorithm needs is a convolutional NN to understand each shape by extracting features, but you just need to be very careful with pooling.&lt;/p&gt;&#xA;&lt;p&gt;Then what you need is a recurrent NN. In the example you showed (the image of shapes) we have bigrams (sequence of 2) which means we have the first shape as an input and the second shape is a target. In this case normal RNN should work.&lt;/p&gt;&#xA;&lt;p&gt;But if you have sequence of many shapes, for example 10 shapes and let's say the final shape, 10th, is the target; And also if the sequence is in the way that 10th shape could be more depend on initial shapes (e.g. 1st or 2nd shape) then what you also need to consider beside RNN is long-short-term-memory (LSTM).&lt;/p&gt;&#xA;&lt;p&gt;I cannot think of a simpler solution than this.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T03:20:10.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22686" PostTypeId="1" CreationDate="2020-07-27T03:51:42.390" Score="0" ViewCount="58" Body="&lt;p&gt;According to deep Q learning, we want to learn &lt;span class=&quot;math-container&quot;&gt;$Q^*(s,a)$&lt;/span&gt;, which is the optimal action-value function. It does make sense because we assume there is only one optimal function so the algorithm will converge supposedly.&lt;/p&gt;&#xA;&lt;p&gt;But when it comes to actor-critic method, we use critic network (also called  value network) to estimate &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt;. This is what confused me. Since our policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; will change through time, the target &lt;span class=&quot;math-container&quot;&gt;$Q_\pi(s, a)$&lt;/span&gt; of value network will also change. What will happen for a network to learn a changing function?&lt;/p&gt;&#xA;" OwnerUserId="38894" LastActivityDate="2020-07-27T09:47:39.823" Title="Why can we use a network to estimate $Q_\pi(s, a)$ in Actor-Critic Method?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22687" PostTypeId="2" ParentId="22636" CreationDate="2020-07-27T04:00:26.930" Score="0" Body="&lt;p&gt;I assume you trained your model on &lt;code&gt;(f1, f2, f3, f4, f5, f6)&lt;/code&gt; and in your test data you sometimes have &lt;code&gt;(f1, f2, f3)&lt;/code&gt; and sometimes have for example &lt;code&gt;(f1, f2, f3, f4, f5, f6)&lt;/code&gt;, right? Because if your test data always have &lt;code&gt;(f1, f2, f3)&lt;/code&gt;, then isn't it better to just train a model on available features?&lt;/p&gt;&#xA;&lt;p&gt;So if my assumption is correct what I would do is to manipulate the training set a bit, keeping some training set with &lt;code&gt;(f1, f2, f3, f4, f5, f6)&lt;/code&gt; and some others with &lt;code&gt;(f1, f2, f3)&lt;/code&gt; with replacement of real values in their &lt;code&gt;(f4, f5, f6)&lt;/code&gt; by e.g. mean of respective feature. So all training set still have &lt;code&gt;(f1, f2, f3, f4, f5, f6)&lt;/code&gt; but some of them have manipulated &lt;code&gt;(f4, f5, f6)&lt;/code&gt;. Then finally when testing, do the same manipulation to those test data that have a smaller number of features.&lt;/p&gt;&#xA;&lt;p&gt;I think like this your model learn how to predict base on &lt;code&gt;(f1, f2, f3)&lt;/code&gt; when other features are not available. but at the same time, take advantage of all features if they are all available.&lt;/p&gt;&#xA;&lt;p&gt;It's probably not the best approach but it worth to try.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T04:00:26.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22688" PostTypeId="2" ParentId="22632" CreationDate="2020-07-27T04:18:22.410" Score="2" Body="&lt;p&gt;It depends, as mentioned in comments, on your model and labels. For example how would you use standardisation on multi classification problem?&lt;/p&gt;&#xA;&lt;p&gt;Generally, standardisation is more flavourable for input data as its mean is around 0.&lt;/p&gt;&#xA;&lt;p&gt;I assume you have a regression model and in that case using standardisation could be better than normalisation.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T04:18:22.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22689" PostTypeId="1" AcceptedAnswerId="22692" CreationDate="2020-07-27T04:40:57.510" Score="3" ViewCount="562" Body="&lt;p&gt;What is the difference between vanilla policy gradient (VPG) with a baseline as value function and advantage actor-critic (A2C)?&lt;/p&gt;&#xA;&lt;p&gt;By vanilla policy gradient I am specifically referring to spinning up's explanation of VPG.&lt;/p&gt;&#xA;" OwnerUserId="38895" LastEditorUserId="2444" LastEditDate="2020-07-27T13:05:40.813" LastActivityDate="2020-07-27T13:05:40.813" Title="What is the difference between vanilla policy gradient and advantage actor-critic?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;advantage-actor-critic&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22690" PostTypeId="2" ParentId="22686" CreationDate="2020-07-27T09:36:02.220" Score="1" Body="&lt;p&gt;The policy doesn't change over time. That is, the values will change, otherwise we would not be learning anything, but our rules for action selection don't. I.e. we always take action according to the distribution postulated to our current estimate of the policy &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta(a|s)$&lt;/span&gt;, we don't suddenly start taking &lt;span class=&quot;math-container&quot;&gt;$\max_a \pi_\theta(a|s)$&lt;/span&gt;, which would be a true change in policy and would make learning both the actor and the critic unstable.&lt;/p&gt;&#xA;&lt;p&gt;Because NN's are able to handle noisy target distributions, this is how they can deal with the changing data. If you think of how Actor-Critic methods work, you would initially start to shift your NN to some unfeasible values (due to random initialisation and the Actor-Critic not having any information about the environment), but as you start to interact with the environment you will start to update the agent towards the 'true' policy.&lt;/p&gt;&#xA;&lt;p&gt;An analogy in supervised learning would be to have some noisy data which is incorrect and some true data. If you trained your network on the noisy data for a small number of epochs and then never showed it to the network again and trained it solely on the correct data, it would forget it has ever seen the noisy data and only represent the new, true data.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-07-27T09:47:39.823" LastActivityDate="2020-07-27T09:47:39.823" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22691" PostTypeId="1" CreationDate="2020-07-27T10:00:31.183" Score="1" ViewCount="47" Body="&lt;p&gt;I wanted to plot a graph to show the effect of increasing the batch size on loss calculated (MNIST dataset). But I am not able to decide if I should show change in loss over training time of the neural network or number of updates made to weights and biases (iterations and epochs basically, but for large differences in batch sizes, I think number of updates made makes more sense?). I am confused about what makes more sense (or neither of them makes sense, idk). &lt;a href=&quot;https://i.stack.imgur.com/xnkMi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xnkMi.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;With Loss vs training time graph, I can show that for any training time, the loss for large batch is more. From what I have read on wiki, with Loss vs number of updates made graph, I can show that change in loss is smoother for larger batches (rate of convergence). But can't the same conclusion be made when plotted against time? (Smooth convergence means lesser spikes right?)&lt;/p&gt;&#xA;" OwnerUserId="33029" LastEditorUserId="33029" LastEditDate="2020-07-27T10:06:46.703" LastActivityDate="2020-07-27T10:06:46.703" Title="Plotting loss vs number of updates made and plotting loss vs run time" Tags="&lt;machine-learning&gt;&lt;plotting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22692" PostTypeId="2" ParentId="22689" CreationDate="2020-07-27T10:07:14.377" Score="3" Body="&lt;p&gt;The difference between Vanilla Policy Gradient (VPG) with a baseline as Value function and Advantage Actor Critic (A2C) is very similar to the difference between Monte Carlo Control and SARSA:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The value estimates used in updates for VPG are based on full sampled returns, calculated at the end of episodes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The value estimates used in updates for A2C are based on temporal difference &lt;em&gt;bootstrapped&lt;/em&gt; from e.g. a single step difference, and the Bellman function.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This leads to following practical differences:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A2C can learn &lt;em&gt;during&lt;/em&gt; an episode which can lead to faster refinements in policy than with VPG.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A2C can learn in continuing environments, whilst VPG cannot.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A2C relies on initially biased value estimates, so can take more tuning to find hyperparameters for the agent that allow for stable learning. Whilst VPG typically has higher variance and can require more samples to achieve the same degree of learning.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-07-27T10:12:54.467" LastActivityDate="2020-07-27T10:12:54.467" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22693" PostTypeId="2" ParentId="22636" CreationDate="2020-07-27T10:13:51.037" Score="1" Body="&lt;p&gt;Assuming that you have access to the training data set, you could use an &lt;a href=&quot;https://medium.com/pytorch/implementing-an-autoencoder-in-pytorch-19baa22647d1&quot; rel=&quot;nofollow noreferrer&quot;&gt;autoencoder network&lt;/a&gt; to predict what features f4, f5, f6 'could be' for the test data set. The way to do this is to train the autoencoder on the training data set with features f1, f2, f3 as inputs, and then use f1,f2,f3,f4,f5,f6 as the output of the network. The autoencoder then effectively learns to map any input samples with (f1,f2,f3) to (f1,f2,f3,f4,f5,f6). By passing your test data through the autoencoder, you can then use the output and pass it to your model.&lt;/p&gt;&#xA;" OwnerUserId="34180" LastActivityDate="2020-07-27T10:13:51.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22694" PostTypeId="1" CreationDate="2020-07-27T10:46:56.330" Score="1" ViewCount="76" Body="&lt;p&gt;Let's suppose that our RL agent needs to play a game with different levels. If we train our RL agent sequentially or with sequential data, our agent will learn how to play level 1, but then it will learn to play level 2 differently, because our agent learns how to play level 2 and forgets how to play level 1, since now our model is fitted using only experiences from level 2.&lt;/p&gt;&#xA;&lt;p&gt;How does an experience replay buffer change this? Can you explain this in simple terms?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-07-27T13:11:18.837" LastActivityDate="2020-07-27T13:11:18.837" Title="What is the advantage of using experience replay (as opposed to feeding it sequential data)?" Tags="&lt;reinforcement-learning&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22695" PostTypeId="1" CreationDate="2020-07-27T11:18:03.670" Score="1" ViewCount="89" Body="&lt;p&gt;I have been reading a few papers in this area recently and I keep coming across these two terms. As far as I'm aware, Belief-MDPs are when you cast a POMDP as a regular MDP with a continuous state space where the state is a belief (distribution) with some unknown parameters.&lt;/p&gt;&#xA;&lt;p&gt;Are they not the same thing?&lt;/p&gt;&#xA;" OwnerUserId="38899" LastEditorUserId="2444" LastEditDate="2020-07-27T13:13:53.430" LastActivityDate="2021-02-21T23:00:25.560" Title="What is the difference between Bayes-adaptive MDP and a Belief-MDP in Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22696" PostTypeId="1" AcceptedAnswerId="22699" CreationDate="2020-07-27T11:51:00.447" Score="4" ViewCount="232" Body="&lt;p&gt;Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I use level 1 experiences, my model &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is fitted to learn how to play that level.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-07-27T13:16:31.910" LastActivityDate="2020-07-27T13:16:31.910" Title="Why do DQNs tend to forget?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;experience-replay&gt;&lt;catastrophic-forgetting&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22697" PostTypeId="1" CreationDate="2020-07-27T12:02:41.493" Score="0" ViewCount="83" Body="&lt;p&gt;I'm trying to reproduce the work in the paper &lt;a href=&quot;https://ieeexplore.ieee.org/document/8681422&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demand Response for Home Energy Management Using Reinforcement Learning and Artificial Neural Network&lt;/a&gt;. I want to optimize the power consumption for home appliances. The action space is a different power rating for home appliances. My reward function is = -(power rating *electricity price).&lt;/p&gt;&#xA;&lt;p&gt;I have trained an RL agent using DQN algorithm on Matlab. I have action space that the agent should select from, but my agent always takes the same action irrespective of state. I have checked my reward function and the algorithm does not select the action with the highest reward. Anyone can think of why is the agent behaving this way?&lt;/p&gt;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ihyBY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ihyBY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Db59v.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Db59v.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I'm getting while training:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HFJCn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HFJCn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And my agent always takes the same power rating regardless of the state (electricity price). Why?&lt;/p&gt;&#xA;" OwnerUserId="38901" LastEditorUserId="2444" LastEditDate="2020-07-27T13:22:18.290" LastActivityDate="2020-07-27T13:22:18.290" Title="Why does my agent always takes a same action in RL?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22698" PostTypeId="1" CreationDate="2020-07-27T12:12:34.933" Score="1" ViewCount="47" Body="&lt;p&gt;&lt;strong&gt;Problem (Sketch):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; seeks to learn a new network &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; whose last few layers behave very similarly to &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.  Thus, the &amp;quot;knowledge of f&amp;quot; is encoded into the output layers of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, but only approximately.&lt;/p&gt;&#xA;&lt;p&gt;This is essentially what is described in &lt;a href=&quot;https://keras.io/guides/transfer_learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the &amp;quot;fine-tuning&amp;quot; section of this Keras page&lt;/a&gt;.  However, I'm looking for an academic reference..&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&#xA;Does anyone know of some article which use this kind of setup?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Formal Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Suppose that we have a feed-forward network&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x):=f_n\circ \dots \circ f_1(x) \mbox{ where }&#xA;f_i(x)=\sigma(A_ix +b_i),\, f_n(x)=A_n x+ b_n&#xA;,&#xA;$$&lt;/span&gt;&#xA;for &lt;span class=&quot;math-container&quot;&gt;$i=1,\dots,n-1$&lt;/span&gt;, trained on some given learning task.  Given some &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0$&lt;/span&gt;, and some loss functions &lt;span class=&quot;math-container&quot;&gt;$L_1,L_2$&lt;/span&gt;, I'm interested in transferring the &amp;quot;knowledge&amp;quot; of the network &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; to a different learning task by training a new network&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;g(x):=g_N\circ \dots \circ g_1(x) \mbox{ where }&#xA;g_i(x)=\sigma(A_i'x +b_i'),\, g_n(x)=A_n' x+ b_n;&#xA;,&#xA;$$&lt;/span&gt;&#xA;for &lt;span class=&quot;math-container&quot;&gt;$i=1,\dots,N-1$&lt;/span&gt;; by optimizing the following&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{aligned}&#xA;\min &amp;amp; \, L_2(g) &#xA;\\&#xA;\mbox{ such that: } &amp;amp;&#xA;L_1(f,g_N\circ\dots \circ g_{N-2} )&#xA;&amp;lt;\epsilon&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Related Post&lt;/strong&gt; &lt;a href=&quot;https://ai.stackexchange.com/questions/22633/forcing-a-neural-network-to-be-close-to-a-previous-model-regularization-throug?noredirect=1#comment34593_22633&quot;&gt;Here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="34185" LastEditorUserId="34185" LastEditDate="2020-07-27T13:55:02.067" LastActivityDate="2020-07-27T13:55:02.067" Title="Reference for Transfer Learning via Final Layers of a Neural Network" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reference-request&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22699" PostTypeId="2" ParentId="22696" CreationDate="2020-07-27T12:13:32.620" Score="3" Body="&lt;p&gt;You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this &lt;a href=&quot;https://www.pnas.org/content/114/13/3521&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T12:13:32.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22700" PostTypeId="1" CreationDate="2020-07-27T13:12:35.457" Score="1" ViewCount="72" Body="&lt;p&gt;I have developed a multi label classifier using BERT. I'm leveraging Hugging Face Pytorch implementation for transformers.&lt;/p&gt;&#xA;&lt;p&gt;I have saved the pretrained model into the file directory in dev environment. Now, the application is ready to be moved the production environment.&lt;/p&gt;&#xA;&lt;p&gt;Is it a good practice to save the models into file system in prod ?&#xA;Can I serialize the model files and word embeddings into any DB and read again ?&lt;/p&gt;&#xA;" OwnerUserId="22195" LastActivityDate="2020-07-27T13:12:35.457" Title="Is it good practice to save NLP Transformer based pre-trained models into file system in production environment" Tags="&lt;models&gt;&lt;pytorch&gt;&lt;word-embedding&gt;&lt;transformer&gt;&lt;pretrained-models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22701" PostTypeId="1" CreationDate="2020-07-27T13:48:40.853" Score="1" ViewCount="50" Body="&lt;p&gt;I'm looking to encode PDF documents for deep learning such that an image representation of the PDF refers to word embeddings instead of graphic data&lt;/p&gt;&#xA;&lt;p&gt;So I've indexed a relatively small vocabulary (88 words). I've generated images that replace graphic data with word indexed (1=cat, 2=dog, etc) data. Now I'm going to my NN model&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;right_input = Input((width, height, 1), name='right_input')&#xA;right = Flatten()(right_input)&#xA;right = Embedding(wordspaceCount, embeddingDepth)(right)&#xA;right = Reshape((width, height, embeddingDepth))(right)&#xA;right = vgg16_model((width, height, embeddingDepth))(right)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Image data is positive-only and embedding outputs negative values though so I'm wondering if it is necessary to normalize the embedding layer with something like this after the &lt;code&gt;Embedding&lt;/code&gt; layer&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;right = Lambda(lambda x: (x + 1.)/2.)(right)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The word indexed image looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d3bHD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d3bHD.png&quot; alt=&quot;Word Encoded Document&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, is this a problematic concept generally?&lt;/p&gt;&#xA;" OwnerUserId="10957" LastEditorUserId="10957" LastEditDate="2020-07-27T13:53:44.267" LastActivityDate="2020-07-27T13:53:44.267" Title="Embedding Layer into Convolution Layer" Tags="&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;convolution&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22702" PostTypeId="1" AcceptedAnswerId="22703" CreationDate="2020-07-27T14:14:20.140" Score="1" ViewCount="65" Body="&lt;p&gt;I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?&lt;/p&gt;&#xA;" OwnerUserId="38906" LastActivityDate="2020-07-29T20:40:17.857" Title="In what situations ELUs should be used instead of RELUs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;activation-function&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22703" PostTypeId="2" ParentId="22702" CreationDate="2020-07-27T14:37:26.807" Score="2" Body="&lt;p&gt;ELU does not suffer from from dying neurons issue, unlike ReLU. While ELU can help you to acheive a better accuracy, it is a slower than ReLU because of its non linearity in its negative range.&lt;/p&gt;&#xA;&lt;p&gt;Choosing a right activation function totally depends on situations but you need to also consider other similar types of activation function such as leaky ReLU.&lt;/p&gt;&#xA;&lt;p&gt;Check this &lt;a href=&quot;https://mlfromscratch.com/activation-functions-explained/#/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; out. It could be useful.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-27T14:37:26.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22704" PostTypeId="1" CreationDate="2020-07-27T15:19:16.963" Score="1" ViewCount="77" Body="&lt;p&gt;In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-encoders with this anomalous images.&lt;/p&gt;&#xA;&lt;p&gt;And, during testing, they take and pass an anomalous image and get the output, take their pixel-wise difference, and, based on a threshold, they detect if it is an anomaly or not.&lt;/p&gt;&#xA;&lt;p&gt;If we are adding noise or anomalies to the training set, are we generalizing the model's capability to recreate the original normal input?&lt;/p&gt;&#xA;&lt;p&gt;How does it help to detect the anomaly?&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that we should train using only normal data without adding any noise, then give an anomaly image at test, take the loss as a threshold.&lt;/p&gt;&#xA;" OwnerUserId="38908" LastEditorUserId="2444" LastEditDate="2020-07-28T12:56:53.023" LastActivityDate="2020-07-29T13:53:50.717" Title="How can a de-noising auto-encoder act as an anomaly detection model?" Tags="&lt;neural-networks&gt;&lt;papers&gt;&lt;autoencoders&gt;&lt;anomaly-detection&gt;&lt;denoising-autoencoder&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22705" PostTypeId="1" CreationDate="2020-07-27T15:21:55.363" Score="0" ViewCount="51" Body="&lt;p&gt;I have some images with anomalies, like small cracks, but it's an imbalanced dataset.&lt;/p&gt;&#xA;&lt;p&gt;Please, suggest some effective scalable approaches. Should I consider convolutional auto-encoders? It's supposed to work in realtime, like video output.&lt;/p&gt;&#xA;" OwnerUserId="38908" LastEditorUserId="2444" LastEditDate="2020-07-28T12:51:36.480" LastActivityDate="2020-07-28T12:51:36.480" Title="What are some scalable approaches to perform anomaly detection (for images with small cracks) with unsupervised learning?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22706" PostTypeId="1" AcceptedAnswerId="22712" CreationDate="2020-07-27T17:57:19.977" Score="1" ViewCount="108" Body="&lt;p&gt;Why L2 loss is more commonly used in Neural Networks than other loss functions?&#xA;What is the reason to L2 being a default choice in Neural Networks?&lt;/p&gt;&#xA;" OwnerUserId="37414" LastActivityDate="2020-07-28T03:58:09.017" Title="Why L2 loss is more commonly used in Neural Networks than other loss functions?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;regularization&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22707" PostTypeId="1" CreationDate="2020-07-27T18:19:47.750" Score="1" ViewCount="80" Body="&lt;p&gt;Is there any recent work on combining clustering approaches (k-means, or gaussian mixture or PGM) with deep learning for computer vision?&lt;/p&gt;&#xA;&lt;p&gt;In particular I'm interested in if anyone has used the first few layers of a deep learning network as feature extractors in conjunction with clustering algorithms which have been engineered to induce things like translation and rotation invariance while preserving basic object structure?&lt;/p&gt;&#xA;&lt;p&gt;Taking the max value out of the output of each feature and fitting them to a gaussian mixture is the easiest approach but I'm interested in seeing other ways you could structure the clustering algorithm. For example I'm interested in seeing how you might learn structure between features that includes position information.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-07-27T18:19:47.750" Title="Combining clustering and deep learning for computer vision" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;clustering&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22710" PostTypeId="1" CreationDate="2020-07-27T22:11:58.247" Score="1" ViewCount="72" Body="&lt;p&gt;My OpenAI CartPole-v0 problem's implementation using basic Q-learning does not learn at all.&#xA;I am a beginner and have implemented my first ever Q-learning from scratch after learning from tutorials.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggest what is going wrong?&lt;/p&gt;&#xA;&lt;p&gt;I have seen through testing that the problem may be that most of the states are remain unvisited even after 10,000 runs. Hence, Q-table remains mostly unchanged at the end of all episodes. I have seen other things in the implementation and they all seem fine to me, at least. Any tip where I should start looking at?&lt;/p&gt;&#xA;&lt;p&gt;The reward is -200 flat, for all the episodes! which suggests that the improvement is NILL/NADDA/NONE!&lt;/p&gt;&#xA;&lt;p&gt;Some relevant images are given at the end.&lt;/p&gt;&#xA;&lt;p&gt;The q-learning part of code is given below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;env.reset()&#xA;while not done:    &#xA;    current_state = current_state_to_string(assign_obs_to_bins(obs, bins))&#xA;&#xA;    if np.random.uniform() &amp;lt; EPSILON:&#xA;        act = env.action_space.sample()&#xA;        best_q_value = return_max_from_dict(q[current_state], action = act)&#xA;    else:&#xA;        act, best_q_value = return_max_from_dict(q[current_state])&#xA;&#xA;    obs, reward, done, _  = env.step(act)&#xA;    q[current_state][act] += LEARNING_RATE * (reward + DISCOUNT_FACTOR * best_q_value - q[current_state][act])&#xA;    cnt+=1&#xA;    total_reward += reward&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pbcpG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pbcpG.jpg&quot; alt=&quot;States' Histogram&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O5RGs.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O5RGs.jpg&quot; alt=&quot;Reward after all episodes&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36710" LastActivityDate="2020-07-27T22:11:58.247" Title="OpenAI gym's CartPole problem system does not learn" Tags="&lt;q-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22712" PostTypeId="2" ParentId="22706" CreationDate="2020-07-28T03:58:09.017" Score="4" Body="&lt;p&gt;I'll cover both L2 regularized loss, as well as Mean-Squared Error (MSE):&lt;/p&gt;&#xA;&lt;p&gt;MSE:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;L2 loss is continuously-differentiable across any domain, unlike L1 loss.  This makes training more stable and allows for gradient-based optimization, as opposed to combinatorial optimization.&lt;/li&gt;&#xA;&lt;li&gt;Using L2 loss (without any regularization) corresponds to the Ordinary Least Squares Estimator, which, if you're able to invoke Gauss-Markov assumptions, can lead to some beneficial theoretical guarantees about your estimator/model (e.g. that it is the &amp;quot;Best Linear Unbiased Estimator&amp;quot;).  Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;L2 Regularization:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Using L2 regularization is equivalent to invoking a Gaussian prior (see &lt;a href=&quot;https://stats.stackexchange.com/questions/163388/why-is-the-l2-regularization-equivalent-to-gaussian-prior&quot;&gt;https://stats.stackexchange.com/questions/163388/why-is-the-l2-regularization-equivalent-to-gaussian-prior&lt;/a&gt;) on your model/estimator.  If modeling your problem as a Maximum A Posteriori Inference (MAP) problem, if your likelihood model (p(y|x)) is Gaussian, then your posterior distribution over parameters (p(x|y)) will also be Gaussian.  From Wikipedia: &amp;quot;If the likelihood function is Gaussian, choosing a Gaussian prior over the mean will ensure that the posterior distribution is also Gaussian&amp;quot; (source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_prior&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Conjugate_prior&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As in the case above, L2 loss is continuously-differentiable across any domain, unlike L1 loss.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36611" LastActivityDate="2020-07-28T03:58:09.017" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22714" PostTypeId="1" CreationDate="2020-07-28T06:49:18.347" Score="2" ViewCount="70" Body="&lt;p&gt;In building my first Q-learning algorithm for OpenAI gym's CartPole problem, many of my states remain unvisited. I believe it is the reason that my agent does not learn.&lt;/p&gt;&#xA;&lt;p&gt;Can I be told of the reasons I can look into why that may happen? I have read and seen tutorials and I know a lot has already been done for this problem. My goal here is to learn and hence this simple implementation with Q-learning.&lt;/p&gt;&#xA;&lt;p&gt;PS. The specific question to my problem is asked &lt;a href=&quot;https://ai.stackexchange.com/questions/22710/openai-gyms-cartpole-problem-system-does-not-learn?noredirect=1#comment34679_22710&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;PSS. As an edit, I am inserting my whole code in the following.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import gym&#xA;import matplotlib.pyplot as plt&#xA;&#xA;env = gym.make('MountainCar-v0')&#xA;&#xA;EPISODES = 5000&#xA;LEARNING_RATE = 0.1&#xA;SHOW_AFTER = 1500&#xA;DISCOUNT_FACTOR = 0.95&#xA;EPSILON = 0.1&#xA;NUMBER_OF_BINS = 10&#xA;OBSERVATION_SPACE = 2&#xA;MAX_STATES = 100 # e.g. 23 means obs one is 2, obs two is 3&#xA;&#xA;'''This function breaks the continuous states into discrete form'''&#xA;def digitize_states():&#xA;    bins = np.zeros((OBSERVATION_SPACE,NUMBER_OF_BINS))&#xA;    bins[0] = np.linspace(-1.2, 0.6, NUMBER_OF_BINS)&#xA;    bins[1] = np.linspace(-.07, 0.07, NUMBER_OF_BINS)&#xA;    return bins&#xA;&#xA;'''This function assign the observations into discrete bins using &#xA;   digitize function and the bins that we created using digitize_states()&#xA;'''&#xA;def assign_obs_to_bins(obs, bins):&#xA;    states = np.zeros((OBSERVATION_SPACE))&#xA;    states[0] = np.digitize(obs[0], bins[0]) &#xA;    states[1] = np.digitize(obs[1], bins[1]) &#xA;    return states&#xA;&#xA;'''This function merely make the states in form of the strings so that we can&#xA;   later use those strings (i.e. number of states) as the KEYs in our q-table &#xA;   dictionary.&#xA;'''&#xA;def get_all_states_as_strings():&#xA;    states = []&#xA;    for i in range(MAX_STATES):&#xA;        states.append(str(i).zfill(OBSERVATION_SPACE))&#xA;    return states&#xA;&#xA;'''Convert the current state into string so that it can be used as key for dictionary '''&#xA;def current_state_to_string(state):&#xA;    current_state = ''.join(str(int(e)) for e in state)&#xA;    return current_state&#xA;&#xA;'''This function iniquation the q-table to zeros'''&#xA;def initialize_q():&#xA;    states = get_all_states_as_strings()&#xA;    q = {}&#xA;    for state in states:&#xA;        q[state] = {}&#xA;        for action in range(env.action_space.n):&#xA;            q[state][action] = 0 &#xA;    return q&#xA;&#xA;def initialize_Q():&#xA;    Q = {}&#xA;&#xA;    all_states = get_all_states_as_strings()&#xA;    for state in all_states:&#xA;        Q[state] = {}&#xA;        for action in range(env.action_space.n):&#xA;            Q[state][action] = np.random.uniform(-.5, 0.5, 1)&#xA;    return Q&#xA;&#xA;'''This function returns the maximum Q-value from Q-table'''&#xA;def return_max_from_dict(dict_var, action = None):&#xA;    '''    Arguments&#xA;    # dict_var: Dictionary variable, which represent the q-table.&#xA;    &#xA;    # Return&#xA;    # max_key: Best Action&#xA;    # max_val: max q-value for the current state, taking best action&#xA;    '''&#xA;    if(action == None):&#xA;        max_val = float('-Inf')&#xA;        for key, val in dict_var.items():&#xA;            if val &amp;gt; max_val:&#xA;                max_val = val&#xA;                max_key = key&#xA;        return max_key, max_val&#xA;    else:&#xA;        return dict_var[action]   &#xA;        &#xA;&#xA;'''Main code starts here'''&#xA;&#xA;bins = digitize_states()&#xA;all_states = get_all_states_as_strings()&#xA;&#xA;q = initialize_Q()    &#xA;&#xA;Total_reward_matrix = []&#xA;_testing_action_matrix = []&#xA;_testing_state_matrix = [] &#xA;_testing_states = []&#xA;_testing_random = 0&#xA;_testing_greedy = 0&#xA;&#xA;for episode in range(EPISODES):&#xA;    &#xA;    done = False&#xA;    cnt = 0&#xA;    &#xA;    # Reset the observations -&amp;gt; then assign them to bins&#xA;    obs = env.reset()&#xA;    &#xA;    if episode%SHOW_AFTER == 0:&#xA;        print(episode)&#xA;    &#xA;    total_reward = 0&#xA;    &#xA;    while not done:&#xA;        current_state = current_state_to_string(assign_obs_to_bins(obs, bins))&#xA;        _testing_state_matrix.append(int(current_state))&#xA;&#xA;        if np.random.uniform() &amp;lt; EPSILON:&#xA;            act = env.action_space.sample()&#xA;            best_q_value = return_max_from_dict(q[current_state], action = act)&#xA;            _testing_random+=1&#xA;        else:&#xA;            act, best_q_value = return_max_from_dict(q[current_state])&#xA;            _testing_greedy+=1&#xA;        &#xA;        obs, reward, done, _  = env.step(act)&#xA;        _testing_action_matrix.append(act)&#xA;        &#xA;        q[current_state][act] = (1-LEARNING_RATE)*q[current_state][act] + LEARNING_RATE * (reward + DISCOUNT_FACTOR * best_q_value)&#xA;        cnt+=1&#xA;        total_reward += reward&#xA;    &#xA;        if done and cnt &amp;gt; 200:&#xA;            print(f'reached at episode: {episode} in count {cnt}') &#xA;            Total_reward_matrix.append(total_reward)&#xA;        elif done:&#xA;#             print('Failed to reach flag in episode ', episode)&#xA;            Total_reward_matrix.append(total_reward)&#xA;        &#xA;    &#xA;env.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36710" LastEditorUserId="36710" LastEditDate="2020-07-28T20:05:02.197" LastActivityDate="2020-07-28T20:05:02.197" Title="Most of state-action pairs remain unvisited in the q-table" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22715" PostTypeId="2" ParentId="22673" CreationDate="2020-07-28T09:50:31.090" Score="4" Body="&lt;p&gt;Parameters is a synonym for weights, which is the term most people use for a neural networks parameters (and indeed in my experience it is a term that machine learners will use in general whereas parameters is more often found in statistics literature). Batch size, learning rate etc. are &lt;em&gt;hyper-parameters&lt;/em&gt; which basically means they are user specified, whereas weights are what the learning algorithm will learn through training.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-07-28T09:50:31.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22716" PostTypeId="2" ParentId="22622" CreationDate="2020-07-28T10:10:53.170" Score="1" Body="&lt;p&gt;I'm using my own implementation of A2C (Advantage Actor Critic) in an industrial application based on Markov Process (present state alone provides sufficient knowledge to make an optimal decision). It's simple and versatile, its performance proven in many different applications. The results so far have been promising.&lt;/p&gt;&#xA;&lt;p&gt;One of my colleagues had issues with solving a simple task of mapping images to coordinates with OpenAI's Stable Baselines implementations of PPO and TRPO. Hence I'm biased against this framework.&lt;/p&gt;&#xA;&lt;p&gt;My suggestion is to try the simplest model and if that doesn't satisfy your expectations for performance, then try something fancier. Once you've made a pipeline for learning, switching to a different algorithm is relatively time inexpensive.&lt;/p&gt;&#xA;&lt;p&gt;Here is a list of algorithms for continuous action and state space from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Reinforcement_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt; article about RL:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RHIrT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RHIrT.png&quot; alt=&quot;list of RL algorithms with continuous action and state space&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38671" LastEditorUserId="38671" LastEditDate="2020-07-28T10:16:43.780" LastActivityDate="2020-07-28T10:16:43.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22717" PostTypeId="1" CreationDate="2020-07-28T10:26:44.140" Score="0" ViewCount="154" Body="&lt;p&gt;As what the title said. Does not deepAR require the time series being stationary?&lt;/p&gt;&#xA;" OwnerUserId="38926" LastActivityDate="2021-04-26T15:02:10.303" Title="Why does not the deepAR model of Amazon require the time series being stationary, as opposed to ARMA model?" Tags="&lt;deep-neural-networks&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22721" PostTypeId="1" CreationDate="2020-07-28T14:31:41.377" Score="1" ViewCount="81" Body="&lt;p&gt;I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers?&lt;/p&gt;&#xA;&lt;p&gt;What are formal definitions for these two?&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2021-01-28T16:53:32.557" Title="What is the difference between artificial neural network (ANN) and deep learning?" Tags="&lt;deep-learning&gt;&lt;definitions&gt;&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22722" PostTypeId="1" AcceptedAnswerId="22730" CreationDate="2020-07-28T15:19:44.927" Score="2" ViewCount="68" Body="&lt;p&gt;If a policy maps states to actions in reinforcement learning, then for a path planning with obstacles, can't we simply use &lt;a href=&quot;https://www.youtube.com/watch?v=r9FD7P76zJs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Potential Field&lt;/a&gt; fields for path planning and model policy mathematically as a field where the obstacles form repulsive field and goal form attractive field?&lt;/p&gt;&#xA;&lt;p&gt;So, technically, is a policy simply a field?&lt;/p&gt;&#xA;" OwnerUserId="36047" LastEditorUserId="2444" LastEditDate="2020-07-29T22:56:49.127" LastActivityDate="2020-07-29T22:56:49.127" Title="Is a policy in reinforcement learning analogous to a field such as APF?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;policies&gt;&lt;path-planning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22723" PostTypeId="2" ParentId="15374" CreationDate="2020-07-28T15:28:57.583" Score="1" Body="&lt;p&gt;In general it's better to not use sigmoid function in any hidden layer. There are many other great options such as ReLU and ELU. However, if for any reason you have to use sigmoid-like function, then go with Tanh function, at least it has ~0 mean.&lt;/p&gt;&#xA;" OwnerUserId="38892" LastActivityDate="2020-07-28T15:28:57.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22724" PostTypeId="1" CreationDate="2020-07-28T15:31:37.560" Score="0" ViewCount="95" Body="&lt;p&gt;The game of cribbage &lt;a href=&quot;https://en.wikipedia.org/wiki/Cribbage&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Cribbage&lt;/a&gt; is a two-player card game played over a series of deal with the goal to reach 121 points.&lt;/p&gt;&#xA;&lt;p&gt;The game's elements are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the discard. There are three hands, one for each player of 4 cards selected from six. The discards go face down to a hand which belongs to the dealer (the crib). On average, because non-dealer is optimising one hand, his hand scores slightly more points, say 8 points vs 7.8 for dealer, while the crib scores around 4 points, because one player aims to toss trash (while not damaging his own hand), while the other aims to toss good cards (while likewise maximising his own hand)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;pegging. Here the dealer expects to peg on average around 4 points (he plays second, so has an advantage in responding to non-dealer's lead), and non-dealer around 2 points&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;the cut. This is a fifth card which belongs to all three hands. If the cut is a Jack, dealer scores 2 points, for free.&lt;/li&gt;&#xA;&lt;li&gt;the order of play. This is:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the cut. Averages 2/13 = 0.15 points for dealer (if a Jack is cut)&lt;/li&gt;&#xA;&lt;li&gt;pegs (played one card at a time, scoring for each). Dealer will peg at least 1 point, and non-dealer will peg 0 or more. Averages 4 and 2 points.&lt;/li&gt;&#xA;&lt;li&gt;non-dealer shows his hand. Averages 8 points&lt;/li&gt;&#xA;&lt;li&gt;dealer shows his hand. Averages 8 points.&lt;/li&gt;&#xA;&lt;li&gt;dealer shows his crib. Average 4 points.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As should be obvious the hands are generally the main source of points, so players will tend to optimise their hands to maximise points there. This problem is OUTSIDE OF THE SCOPE of this post, since there are exhaustive analyses that have solved this problem reasonably well. E.g.,  &lt;a href=&quot;https://cliambrown.com/cribbage/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cliambrown.com/cribbage/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are 6C4 = 15 discards possible for each hand, and often the correct play will be unambiguous. For example, if we had the hand A49QKK with mixed suits (no more than 3 of any 1 suit) and we are non-dealer, then the correct hold is obvious - A4KK (this hand has two ways to make 15 A4K and A4K for 2 points each, and a pair for 2 points, plus it improve swith cuts of A, 4, 5, T, J, Q K)&lt;/p&gt;&#xA;&lt;p&gt;After we've held the card we must then 'play the pegging game'. &lt;strong&gt;The scope of this post/question is therefore limited to how to play a given 4-card hand that we have already selected using an exhaustive combinatorial analysis, hence I'm assuming that the bots inputs are a given 4 card hand, as well as the three dead cards (discard and cut), and the replies to each play from our opponent.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The scoring for pegging is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;each pair scores 2 for the player playing the second card, 6 for 3 of a kind, 12 for 4 of a kind&lt;/li&gt;&#xA;&lt;li&gt;each time the count reaches 15, the player playing the card scores 2&lt;/li&gt;&#xA;&lt;li&gt;each time the count reaches 31, the player reaching scores 2 (it is not permitted to exceed 31, and if you have no card that will keep the count at/below 31, you must call 'go', and if both players call 'go', the last player scores 1 point). After this the cards are turned over, and play continues from zero&lt;/li&gt;&#xA;&lt;li&gt;each time a run of 3 or more cards is visible (between the cards played by both players), then you score 3, 4, 5, 6, 7, or even 8 points according to the length of the run.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Follows discussion about pegging strategy, which I have blockquoted for those who find this tl;dr&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The optimum play for a given hand is not necessarily clear-cut. For&#xA;example, with the hand A4KK, then we could lead:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;K - on the basis that if dealer replies with another K (for 2 points), then we reply with the third K (for 6 points), and then most&#xA;likely dealer does not have an Ace, so we score 2 points for the 31 (K&#xA;= 10, A = 1), AND dealer must lead the next round of pegging, which is a considerable disadvantage.&lt;/li&gt;&#xA;&lt;li&gt;not K - because there's a bias to holding 5s, so if dealer was dealt a 5, then he can reply with the 5 scoring 2 points for the 15, to&#xA;which we have no scoring reply. In addition, non-dealer generally has&#xA;a bias to leading from a pair, so if we lead the K, then in general a&#xA;smart dealer says 'he is likely to have a pair of Ks'. So even if&#xA;dealer has a K, he may decline to peg it, especially if he doesn't&#xA;hold the A himself&lt;/li&gt;&#xA;&lt;li&gt;4 - this is the most popular lead in cribbage because 15 cannot be scored on it. As indicated above there are 16 10 cards (TJQK), as&#xA;opposed to 4 of every other denomination, so it's more likely to&#xA;receive a ten back than any other denomination, and we hold an Ace, so&#xA;we would score 2 points following a ten card reply&lt;/li&gt;&#xA;&lt;li&gt;A - this has the same advantage that dealer cannot reach 15, and again if we get a ten card back we can score 15-2.&lt;/li&gt;&#xA;&lt;li&gt;It might be that the A is better to lead because the 4 is such a popular lead that dealer is more likely to pair it (risking that we&#xA;hold the third 4 for 6 points), in which case we have no good reply.&#xA;If we lead the A, then the dealer is [probably!] less likely to reply&#xA;with an A (which doesn't help us).&lt;/li&gt;&#xA;&lt;li&gt;We might prefer to lead the 4 if we think that the A is more likely to allow us to play the last card before 31, in which case dealer is&#xA;forced to lead first next round, which is a disadvantage.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;During pegging we have considerable information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;which cards we hold and discarded. For example, clearly if we hold 3 4s, and the 4th was cut, then there is no chance that we get one&#xA;played against us.&lt;/li&gt;&#xA;&lt;li&gt;which cards have been played and therefore which cards are likely to remain, based on the hand selection process. For example, if a 4 is&#xA;led then x% of the time that will be from A4 TJQK, y% it will be from&#xA;4456, etc. As more cards are played, then this becomes more obvious.&#xA;For example, if we've seen the 3 cards 5TK, then it's VERY likely that&#xA;the 4th card is another TJQK card, and somewhat likely that it's a 5.&#xA;It could be ANY card because maybe of the six cards the player was&#xA;dealt he ended up with an unconnected card. But we can say that the&#xA;chance of this is low.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In terms of exactly what cards are held, then if we analysed millions&#xA;of games, we could calculate in general knowing one, two or three&#xA;cards what the remaining cards could be.&lt;/p&gt;&#xA;&lt;p&gt;Although there are in theory 270,725 4 card hands, in pegging terms&#xA;(ignoring suits), there are only 1820  distinct hands&#xA;(&lt;a href=&quot;https://docs.google.com/spreadsheets/d/1fxkLBkWC2LA6J06zhku21jcG2ATHqE1RNHlPSDcc4fQ/edit#gid=834958733&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/spreadsheets/d/1fxkLBkWC2LA6J06zhku21jcG2ATHqE1RNHlPSDcc4fQ/edit#gid=834958733&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;For any given hand, if we for example held the cards 358TTT and we&#xA;were non-dealer, then we would choose to hold 5TTT and toss 38. We&#xA;would then lead the T. In this spot, clearly of the 1820 possible&#xA;hands, then combinations such as TT xy are no longer possible.&lt;/p&gt;&#xA;&lt;p&gt;As another example if we were dealt A778JK, then we'd toss the JK.&#xA;Here we'd likely lead the 7.&lt;/p&gt;&#xA;&lt;p&gt;Before we play the 7 it's relatively simple to calculate the odds that&#xA;dealer was dealt two 7s. Since he had six cards, that is&#xA;2/46&lt;em&gt;1/45&lt;/em&gt;6C2, which is 1.45%. The chance that he chose to hold those&#xA;two 7s is a different number, and we could theoretically calculate it&#xA;by figuring out for each of the 45C6 (but simplifiable!) combinations,&#xA;which hold he would make. However it won't be too far from this number&#xA;of 1.45%.&lt;/p&gt;&#xA;&lt;p&gt;HOWEVER, once we have played a 7 and dealer replies with a second 7,&#xA;then this number is now very far away from accurate. Firstly because&#xA;it's now a simple conditional probability where one condition is&#xA;already satisfied, so the chance of any of 5 unknown cards being a&#xA;specific card (e.g., the 7 of diamonds) is now almost 1/9. Of course&#xA;dealer has 3 cards, not 5, so again the chances of this are not that&#xA;number, but not too far off, because of those hands where dealer was&#xA;dealt 77, and he holds one seven, then he is seldom going to toss the&#xA;other one.&lt;/p&gt;&#xA;&lt;p&gt;In terms of our possible plays, we have at the count of 14:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;play the Ace and score 2 points for the count of 15.&lt;/li&gt;&#xA;&lt;li&gt;play the 7 and bring the count to 21, which is 6 points. However, dealer is fairly likely to have a TJQK (bringing the count to 31,&#xA;scoring 2 points, and making us lead the next deal). In addition,&#xA;given that dealer has a similar analysis process to us, in hands where&#xA;he holds say 78 then he probably replies to the 7 lead with the 8, as&#xA;it doesn't allow us to score 6 points. So given that dealer has&#xA;replied with 7, this increases the chance that he holds the fourth 7&#xA;as well.  It's hard to say what this chance is, but for example there&#xA;is an x% chance of 31-2, a y% chance of 28-12, and a z% chance of&#xA;something else. Or we could consider the chance of any given card.&lt;/li&gt;&#xA;&lt;li&gt;probably not play the 8, because at 778 the count is 22, and dealer has many scoring replies: 6 is a run (3 points), 9 is a run AND 31 (5&#xA;points), and 8 is a pair (2 points)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In general it can be seen that there are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;up to 1820 unique 4-card hands&lt;/li&gt;&#xA;&lt;li&gt;up to 455 3-card hands&lt;/li&gt;&#xA;&lt;li&gt;up to 91 2-card hands&lt;/li&gt;&#xA;&lt;li&gt;up to 13 1-card hands&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Clearly we could weight each hand by the chance it has to be held, and&#xA;this would work well BEFORE a card has been played.&lt;/p&gt;&#xA;&lt;p&gt;However after 1 or more cards has been played this approach is going&#xA;to be hopelessly naive. For example, let's say we led the 4. If dealer&#xA;is holding a hand like 5JJK then he's definitely not going to reply&#xA;with the 5, because we can play a 3 for 3 points (run), or a 6 for 5&#xA;points (run and count of 15).&lt;/p&gt;&#xA;&lt;p&gt;Further, against non beginner-level players, it will be obvious that&#xA;we often hold hands like A4 JK or 23 KK, etc. So if we lead the 3, and&#xA;dealer replies with a ten card (TJQK), then it's likely he's NOT&#xA;holding cards such as 789, which a non-beginner player would likely&#xA;prefer to reply with here. It's possible of course that dealer has the&#xA;same cards. For example, if we hold 23JK, dealer might be holding&#xA;23QQ. In this case, the Q reply might be preferred by dealer, because&#xA;if the play goes 3-Q-2, then dealer can pair our 2, and perhaps dealer&#xA;doesn't like to pair our 3 lead, for fear we have a third 3, while he&#xA;lacks the fourth.&lt;/p&gt;&#xA;&lt;p&gt;In addition, while for pegging terms suits don't matter at all, suit&#xA;information is during important, so long as the player has a flush.&#xA;For example if we were dealt 2468h Jd Js, we'd hold the 2468 of&#xA;hearts, because that's a flush worth 4 points. So if during we play,&#xA;we've seen 468h from our opponent, then the probability distribution&#xA;of possible hands is going to contain a good weight for every heart&#xA;card. Whereas if we've seen 4h 6d 8h, then we know after two cards&#xA;that he does NOT have a flush, so this weights likely hands towards&#xA;hand such as 4568, 4678 and so on, and it's unlikely that the&#xA;remaining card is, say, a K.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In general the goal of a pegging bot could be seen as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;score most points or&lt;/li&gt;&#xA;&lt;li&gt;concede fewest points or&lt;/li&gt;&#xA;&lt;li&gt;maximise net points&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Nuance: In early game it's likely max net points is the best approach.&#xA;But if we are at a score like 117-117 (121 is the winning mark) as&#xA;dealer, then it's (let's say) 80% likely that non-dealer has 4 points&#xA;in his hand, which means he wins if we do not peg 4 points. In this&#xA;case non-dealer would try to hold cards that score 4 points (as a&#xA;hand), and if there are multiple options, try to hold cards that&#xA;reduce dealer's chance of pegging. Meanwhile dealer's realistic route&#xA;to victory would be to hold the 4 cards that give him the best chance&#xA;of pegging 4 points (remembering that dealer will on average score&#xA;more pegs, while non-dealer scores his hand first). If the score was&#xA;113-113, then dealer would play differently as he has NO chance of&#xA;pegging 8 points, but there's perhaps a 40% chance that non-dealer&#xA;fails to score 8 points from pegging and his hand. So in this case&#xA;dealer would try to stop non-dealer pegging anything.&#xA;So it seems that an AI would need to take into account the current score&#xA;to decide how to peg.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have read a couple of papers on cribbage AI, but they have been superficial and tended to concentrate on the discard game, which can be optimised at least for discarding purposes (without considering the relative pegging value), by simply iterating through the possible hands.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://r6.ca/cs486/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://r6.ca/cs486/&lt;/a&gt; (does not cover pegging)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://helda.helsinki.fi/bitstream/handle/10138/273545/SeanLang_MS_thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://helda.helsinki.fi/bitstream/handle/10138/273545/SeanLang_MS_thesis.pdf&lt;/a&gt; (addresses pegging very superficially by maximising the score of the next card, not considering that that might result in very poor total scores)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now the question is to what extent this is a problem of machine learning, and to what extent this is an exhaustive analysis? If we ignore the discard problem, and we say that we will input four-card hands to our bot using the output of a process similar to the one here &lt;a href=&quot;https://cliambrown.com/cribbage/methodology.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cliambrown.com/cribbage/methodology.php&lt;/a&gt; then for example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;we could calculate the exact probability that our opponent holds any given four card hand, by iterating through the 45C6 hands (subject to appropriate simplifications) that our opponent could have, and then producing a weight for each of the (up to) 1820 possible hands, and for the 715 different hands of 4 unique denominations, the further chance for each to be a flush&lt;/li&gt;&#xA;&lt;li&gt;I am not quite clear how computationally expensive this is, but it seems to me that should be calculate the weights in a reasonable amount of time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So we have 4 cards, and we have weighted possibilities for each of 1820 hands.&lt;/p&gt;&#xA;&lt;p&gt;Clearly it's not appropriate for us to simply randomly iterate through the hands. I.e. there are four choices for our first card, likewise four for our opponent, then 3 for the next. Roughly there will be 4!*4! choices in this way (roughly because the issue of the count resetting at 31 means that the card order after the third card is not always the samme). But our opponent's reply is not random.  If we lead the 5, then he will certainly reply with a TJQK if he has one. He won't possibly reply with a 5.&lt;/p&gt;&#xA;&lt;p&gt;So it seems to me that some kind of learning process is appropriate. But I am not really familiar with AI to say how this should work. Do we need a pool of human training data? Or do we allow the bot to play against itself?&lt;/p&gt;&#xA;&lt;p&gt;And what role do probability tables have in this process? Is it going to be productive on a hand level to iterate through  possible hands that our opponent might have, or is a Monte Carlo process just as good?&lt;/p&gt;&#xA;&lt;p&gt;I should note that humans might make sub-optimal play. So for example, if we play PERFECTLY, then if we hold the hand 6699, and the 4 is led, then we SHOULD reply with the 9, rather than risk a 5 on our 6, conceding 5 points. So the play of a 6 on a 4 SHOULD indicate that we hold a hand such as 5566, in which case then we are informed accordingly. But clearly the chance that we hold 6699 and just made a blunder is not zero. So the bot cannot ever be completely certain about our holds.&lt;/p&gt;&#xA;&lt;p&gt;Likewise it might be we choose to make 'sub-optimal' plays in order to avoid becoming predictable ourselves. For example, the question 'will our opponent pair our lead' is an important one - if we have a pair, we generally want him to. But sometimes we will hold a hand such as A4 JK, in which case we don't want him to pair our lead. Some players will play more aggressively than others, and against an aggressive player, he might pair our lead every time, and a more defensive player might almost never pair our lead.&lt;/p&gt;&#xA;" OwnerUserId="38920" LastEditorUserId="38920" LastEditDate="2020-07-30T11:41:55.783" LastActivityDate="2021-04-26T13:07:01.263" Title="Do I need a large pool of training data to train a bot to play the 'pegging' game in the cribbage card game" Tags="&lt;ai-design&gt;&lt;game-ai&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22725" PostTypeId="2" ParentId="22721" CreationDate="2020-07-28T15:42:09.713" Score="1" Body="&lt;p&gt;A few years ago, deep learning was a buzz word, but now is de-facto a standard term or expression, and it's widely used in all research papers, although &lt;em&gt;deep learning&lt;/em&gt; is almost never defined rigorously (but this doesn't seem to be a big problem!).&lt;/p&gt;&#xA;&lt;p&gt;From my experience (this is not just an opinion, of course!), after having read so many papers on the topic, deep learning typically refers to the subset of machine learning techniques (specifically, gradient descent and back-propagation) applied to neural networks, and, in particular, neural networks with multiple layers. However, there is no consensus on what &lt;em&gt;multiple&lt;/em&gt; refers to. In fact, Schmidhuber (co-author of the &lt;a href=&quot;https://www.bioinf.jku.at/publications/older/2604.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt;), in [&lt;a href=&quot;https://arxiv.org/pdf/1404.7828.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], writes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;At which problem depth does &lt;em&gt;Shallow Learning&lt;/em&gt; end, and &lt;em&gt;Deep Learning&lt;/em&gt; begin? Discussions with DL experts have not yet yielded a conclusive response to this question. Instead of committing myself to a precise answer, let me just define for the purposes of this overview: problems of depth &amp;gt; 10 require &lt;em&gt;Very Deep Learning&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In other words, although there is typically no agreement on the definition of &lt;em&gt;deep&lt;/em&gt; or &lt;em&gt;multiple&lt;/em&gt;, there are several problems (such as the &lt;em&gt;vanishing&lt;/em&gt; or &lt;em&gt;exploding gradient problems&lt;/em&gt;) that arise as the number of layers in a neural network increases, so there are reasons behind the distinction of deep and non-deep learning.&lt;/p&gt;&#xA;&lt;p&gt;Note that some research papers (e.g. [&lt;a href=&quot;https://arxiv.org/pdf/1404.7828.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]) distinguish between deep learning applied to neural networks and other types of deep learning, so that not to exclude the layered composition of other models (e.g. SVMs) other than neural networks and the use of machine learning algorithms (e.g. gradient descent) to train them.&lt;/p&gt;&#xA;&lt;p&gt;To answer your question more directly, neural networks are models or function approximators, while deep learning is the set of machine learning techniques applied to the layered composition of models (typically, neural networks).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-28T16:53:32.557" LastActivityDate="2021-01-28T16:53:32.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22726" PostTypeId="1" AcceptedAnswerId="22727" CreationDate="2020-07-28T17:56:25.200" Score="1" ViewCount="77" Body="&lt;p&gt;I found that the regret in &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/papers/OLsurvey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Online Machine Learning&lt;/a&gt; is stated as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\operatorname{Regret}_{T}(h)=\sum_{t=1}^{T} l\left(p_{t}, y_{t}\right)-\sum_{t=1}^{T} l\left(h(x), y_{t}\right),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p_t$&lt;/span&gt; is the answer of my algorithm to the question &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_t$&lt;/span&gt; is the right answer, while &lt;span class=&quot;math-container&quot;&gt;$h()$&lt;/span&gt; is one of the hypotheses in the hypothesis space. Intuitively, as denoted in the paper, our objective is to minimize this Regret in order to optimize our algorithm, but in the following formula&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{Regret}_{T}(\mathcal{H})=\max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;they maximize this value. Am I interpreting the &lt;span class=&quot;math-container&quot;&gt;$max$&lt;/span&gt; wrongly?&lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2020-07-29T12:36:28.693" LastActivityDate="2020-07-29T12:36:28.693" Title="Does this $\max$ mean that we need to maximize the regret in this regret formula?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;computational-learning-theory&gt;&lt;online-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22727" PostTypeId="2" ParentId="22726" CreationDate="2020-07-28T19:49:07.880" Score="1" Body="&lt;p&gt;Yes, you're interpreting the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; there wrongly. In your second formula&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{Regret}_{T}(\mathcal{H})=\max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right) \label{1}\tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The sign &lt;span class=&quot;math-container&quot;&gt;$=$&lt;/span&gt; means &amp;quot;is defined as&amp;quot;, so maybe the following notation is less confusing&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{Regret}_{T}(\mathcal{H}) \triangleq \max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In fact, in &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/papers/OLsurvey.pdf#page=8&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 2.1 of the same paper&lt;/a&gt;, there's a similar but more detailed formula that should clarify the meaning of &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\operatorname{Regret}_{T}(\mathcal{H})=\max _{h \in \mathcal{H}}\left(\sum_{t=1}^{T}\left|p_{t}-y_{t}\right|-\sum_{t=1}^{T}\left|h\left(\mathbf{x}_{t}\right)-y_{t}\right|\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$\operatorname{Regret}_{T}(\mathcal{H})$&lt;/span&gt; is defined for &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt;, while your first formula is defined for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. So, \ref{1} is the regret for the whole hypotheses class &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt;, which is thus the maximum regret that you can have across all possible hypotheses &lt;span class=&quot;math-container&quot;&gt;$h \in \mathcal{H}$&lt;/span&gt;. This should make sense.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-28T19:49:07.880" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22728" PostTypeId="2" ParentId="16801" CreationDate="2020-07-28T22:04:09.327" Score="4" Body="&lt;p&gt;I don’t know for certain, but I can make a guess. This is just my opinion, some others may disagree.&lt;/p&gt;&#xA;&lt;p&gt;The field of ALife has four branches that I’m aware of:&lt;/p&gt;&#xA;&lt;p&gt;Self-Organizing/self assembly behavior. This is the application you refer to, another context it’s useful is swarm control (for drone swarms, for example). While this is technically ALife, as far as I’m aware it’s not really where most of the emphasis is. Swarm control and self assembly are seen as “different” problems, as machines that can work together and also build more of themselves is interesting (and potentially dangerous), but is missing out on the diversity, the open-endedness that life on earth has. Much of ALife research is focused on trying to formally define this open-endedness and coming up with systems that achieve that. Self assembly and swarm control are interesting and difficult problems, just different. This leads to the other three sides of ALife research:&lt;/p&gt;&#xA;&lt;p&gt;Coming up with environments, and running tests on them. This is a constant game of coming up with a definition that seems to capture open-endedness, then coming up with ALife sims that meet that criteria but fall short of our expectations. So new definitions are made and we repeat. Geb is a classic example: Geb has passed pretty much every test so far, but it’s fairly uninspiring to watch. Most of those programs you reference chose a particular ALife paradigm, but that paradigm may not be the right one, and is often disappointing. Because we still haven’t found something that really “looks like life”, new paradigms and programs are constantly being created and abandoned when they fail to work (Or perhaps some would have already worked, but the computing time is too much). That’s what you’re seeing. Without any unifying theory or sim that is really convincing, I suspect it’ll stay this way for a while. And because:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;we still haven’t made much progress since Karl Sims in the 90s, or since Geb (this point is debatable)&lt;/li&gt;&#xA;&lt;li&gt;these sorts of sims don’t really have much commercial use aside from games&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;the direction of making new simulators seems to be lacking funding and research interest, as far as I can tell. Commercial sim games seem to push the boundary these days.&lt;/p&gt;&#xA;&lt;p&gt;Fortunately there’s a sub field of cellular automata life that’s pretty interesting, its software is slightly more developed due to the overlap with cellular automata and ease of implementation, and research seems to be progressing there at an okay rate.&lt;/p&gt;&#xA;&lt;p&gt;Realistically, there seem to be two things people want: novel behavior, and novel bodies. My two cents is that these are separate problems, and achieving both is more expensive than just achieving one. But most of these sims end up not balancing development happening in both of these factors (doing this is very difficult), so one factor develops much further than the other, and this disconnect is disappointing to the sim creator. For example, Geb does behavioural diversity really well, while Karl Sims does body diversity well. Sensitivity to small details like mutation rate or genetic encoding also can be quite frustrating. Fortunately, eventually we’ll sorta get behavioural diversity for free in any sim once RL/AI is really understood well.&lt;/p&gt;&#xA;&lt;p&gt;The third piece of ALife research I’m aware of is the theoretical side, which right now mostly isn’t really far enough along to warrant practical implementation. One big branch of this is the learning theory side, represented by Valiant’s Evolvability theory and followups. Essentially this talks about what functions are possible to evolve, and using stuff like PAC Learning theory they are able to prove some things. Some of these models are more natural than others, but it’s an interesting perpendicular approach to coming up with sims and seeing if they do what we want. Maybe eventually these two approaches will meet in the middle at some point, but they haven’t yet.&lt;/p&gt;&#xA;&lt;p&gt;The fourth piece is Artificial Chemistry. I recommend &lt;a href=&quot;http://web.cs.mun.ca/%7Ebanzhaf/papers/alchemistry_review_MIT.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; as a somewhat dated overview. While this is technically a field of ALife, and is centered around understanding a chemical system that has the necessary emergent properties, it has broken off into applications that may have industrial relevance. For example, robust &lt;a href=&quot;https://arxiv.org/pdf/1707.07617.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;self repairing and self assembling electronic systems&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1811.10361&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA computing&lt;/a&gt; (DNA is capable of simulating arbitrary chemical reaction networks which are capable of arbitrary computing), and &lt;a href=&quot;http://dl.ifip.org/db/conf/seus/seus2007/BrinkschultePR07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;artificial hormone systems for automatic task assignment&lt;/a&gt;. This has some software developed, but much of that software isn’t really considered ALife anymore since it has branched off into its own domain.&lt;/p&gt;&#xA;" OwnerUserId="6378" LastEditorUserId="6378" LastEditDate="2020-07-30T16:01:28.723" LastActivityDate="2020-07-30T16:01:28.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22729" PostTypeId="1" CreationDate="2020-07-28T23:24:27.757" Score="1" ViewCount="64" Body="&lt;p&gt;I've been reading on neural networks, but for me, seems like the easiest way for me to learn is seeing some code.  I am curious about what is the exact structure within a node of a hidden layer and the areas of customization?&lt;/p&gt;&#xA;&lt;p&gt;From what I have gathered, each node receives some inputs (either features of your original data or outputs from the previous layer).  The first step is, we form a linear combination of the inputs with coefficients in the node.  Here's my first few questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For a NODE within a layer, if we have p inputs, we would multiply a 1xp coefficient matrix with a px1 input vector, correct?&lt;/li&gt;&#xA;&lt;li&gt;In practice, for that LAYER, we would multiply a mxp coefficient matrix by a px1 input vector, where m is the number of nodes within that layer?&lt;/li&gt;&#xA;&lt;li&gt;If we have a bias term, then it would be a mx(p+1) coefficient matrix and a (p+1)x1 input vector?&lt;/li&gt;&#xA;&lt;li&gt;In terms of customization, for most out of the box NNs, do we choose to have a bias term or not?  Or for the most part, it automatically includes one.&lt;/li&gt;&#xA;&lt;li&gt;In terms of customization, this combination of inputs is always linear?  There are no other ways to combine the inputs (such as including polynomials, interaction terms, etc...)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;After we have generated a scalar via the linear combination, the second step is we feed that value to our &amp;quot;activation function&amp;quot; which is most often a non-linear function like ReLU.  Here obviously the customizable aspect is the choice of &amp;quot;activation&amp;quot; function.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each node within the same hidden layer, are the activation functions usually the same, or differ among each node?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="29801" LastActivityDate="2020-08-28T20:04:15.550" Title="What is the exact structure within the nodes of a hidden layer?" Tags="&lt;activation-function&gt;&lt;artificial-neuron&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-09-09T10:18:12.990" ContentLicense="CC BY-SA 4.0" />
  <row Id="22730" PostTypeId="2" ParentId="22722" CreationDate="2020-07-29T00:53:40.260" Score="0" Body="&lt;p&gt;Since most policies depend solely on actions and states/observations, then if you model the space of this field as the Cartesian Product of your state and action spaces, then the policy learns a surface over this combined space, similar to the way a field is parameterized.&lt;/p&gt;&#xA;&lt;p&gt;The policy an agent learns could exhibit the same behavior as the field you describe above (obstacles form an repulsive field, and goal(s) form an attractive field).  However, unlike the field described above, it is not guaranteed that the learned policy will capture this behavior - the policy learned depends on:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The learning algorithm used&lt;/li&gt;&#xA;&lt;li&gt;The approximators (e.g. neural networks) used for learning, and their respective hyperparameters&lt;/li&gt;&#xA;&lt;li&gt;The formulation of the reward function&lt;/li&gt;&#xA;&lt;li&gt;The number of episodes/total steps the policy/agent is trained over.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To sum this answer up, I believe you could train the policy in such a way (using the mechanisms above) such that it resembles the field you describe.&lt;/p&gt;&#xA;" OwnerUserId="36611" LastActivityDate="2020-07-29T00:53:40.260" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22731" PostTypeId="2" ParentId="8554" CreationDate="2020-07-29T03:33:49.027" Score="0" Body="&lt;p&gt;My understanding is that it will be the same as p(s' | s, a) for any s, a, s', r combination.&lt;/p&gt;&#xA;&lt;p&gt;The reward r(s, a, s') is already defined in terms of s, a, s'.&lt;/p&gt;&#xA;&lt;p&gt;Since p(s', r| s, a) = p(r|s', a, s)* p(s'| a, s). For each case the p(r | s', a, s) is equal to 1 by definition. Thus, the column for p(s', r| a, s) = p(s'| s, a)&lt;/p&gt;&#xA;&lt;p&gt;For instance,&lt;/p&gt;&#xA;&lt;p&gt;If,&#xA;S= high, A=search and S'=High, then the reward is always alpha.&lt;/p&gt;&#xA;&lt;p&gt;p(r=alpha| s=high, a = search , s'=high) = 1 (you always get that reward for this case) etc.&lt;/p&gt;&#xA;" OwnerUserId="38874" LastActivityDate="2020-07-29T03:33:49.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22732" PostTypeId="1" CreationDate="2020-07-29T04:56:21.203" Score="1" ViewCount="50" Body="&lt;p&gt;I know there are quite a few good deep learning books out there, but most explain neural networks and deep learning via application on images.  If there are examples/code, they are often done on the MNIST data set.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if there's a book out there that goes in depth into neural networks and is equally well written but explains it on non-image data.  I am particularly interested in time series application, although some talk on cross-sectional application would be helpful as well.  I am particularly interested in learning about:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What types of layers/functions/structure are better suited for time-series data&lt;/li&gt;&#xA;&lt;li&gt;Various models for time series and pros/cons/applications of each (Convoluted NN, LTSM, etc...)&lt;/li&gt;&#xA;&lt;li&gt;Typical structures of your neural network (depth, sparse connections, etc...) that seem to work well on time series data&lt;/li&gt;&#xA;&lt;li&gt;Special considerations or settings you should have in your neural network when working with time series data&lt;/li&gt;&#xA;&lt;li&gt;Maybe some talk/examples on how time series prediction using traditional models like ARIMA, can be reproduced or done better using neural networks.  Or side by side comparison of pros/cons of using one vs the other.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="29801" LastEditorUserId="29801" LastEditDate="2020-07-29T06:49:47.620" LastActivityDate="2020-07-29T06:49:47.620" Title="Recommendations or resources for neural network/deep learning for time series application?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22733" PostTypeId="2" ParentId="22732" CreationDate="2020-07-29T05:43:50.693" Score="1" Body="&lt;p&gt;I'm currently working with Temporal Convolution Networks (TCNs) for making predictions with time series data (link to article here: &lt;a href=&quot;https://medium.com/@raushan2807/temporal-convolutional-networks-bfea16e6d7d2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@raushan2807/temporal-convolutional-networks-bfea16e6d7d2&lt;/a&gt;).  These types of networks, like other types of convolutional networks for time series, use a dilated convolution operation, which, unlike the standard convolution in these networks, is a causal operation that preserves the causality of the time series input.&lt;/p&gt;&#xA;&lt;p&gt;I've also seen CNNs used in Temporal Differencing applications, but that may be more useful in the context of video processing rather than time series analysis.&lt;/p&gt;&#xA;" OwnerUserId="36611" LastActivityDate="2020-07-29T05:43:50.693" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22734" PostTypeId="1" CreationDate="2020-07-29T06:36:26.360" Score="1" ViewCount="133" Body="&lt;p&gt;I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material.&lt;/p&gt;&#xA;&lt;p&gt;How large should the corpus be to optimally retrain the GPT-2 model? And what is the bare minimum size? Should it simply be as large as possible or can it flip over and make the model worse in some way?&lt;/p&gt;&#xA;&lt;p&gt;I am also not certain how many steps you should let the retraining run. I have been using 6000 steps when testing, and it seems not much happens after that, loss only moved from 0.2 to 0.18 last 1000 steps.&lt;/p&gt;&#xA;" OwnerUserId="38955" LastEditorUserId="38955" LastEditDate="2020-07-30T05:49:25.853" LastActivityDate="2021-04-30T12:02:40.337" Title="How large should the corpus be to optimally retrain the GPT-2 model?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22735" PostTypeId="1" CreationDate="2020-07-29T06:39:03.267" Score="1" ViewCount="71" Body="&lt;p&gt;I wanted to build a digit recognition neural network using MATLAB ANFIS kit.&lt;/p&gt;&#xA;&lt;p&gt;I started by using the MNIST database and I figured out it's almost impossible to classify 784 dimension data using ANFIS. So, I reduced my data dimension from 784 to 13, using an autoencoder in Python. With the new data, I had about 80 percent accuracy in classification using a sequential model. I implemented my data in MATLAB too.&lt;/p&gt;&#xA;&lt;p&gt;Since MATLAB treats the problem as a regression problem, I had about 1.5 RMSE after 10 epoch of learning, in both grid partitioning and subtractive clustering, and also the error curve almost seems constant in the process.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way that I can have less error?&lt;/p&gt;&#xA;" OwnerUserId="38957" LastEditorUserId="2444" LastEditDate="2020-07-29T12:53:03.360" LastActivityDate="2020-07-29T12:53:03.360" Title="Is there a way to reduce the RMSE error when training a neural network to recognise MNIST digits using ANFIS?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;matlab&gt;&lt;mnist&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22736" PostTypeId="1" CreationDate="2020-07-29T06:46:34.143" Score="1" ViewCount="43" Body="&lt;p&gt;I see many papers in AAMAS talk about artificial intelligence and mechanism design simultaneously. I was wondering, for the sake of being pedantic, is mechanism design could be classified under AI.&lt;/p&gt;&#xA;" OwnerUserId="8895" LastActivityDate="2020-07-29T06:46:34.143" Title="Does Algorithmic Mechanism Design come under the field of AI?" Tags="&lt;classification&gt;&lt;terminology&gt;&lt;game-theory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22737" PostTypeId="2" ParentId="20993" CreationDate="2020-07-29T09:48:10.853" Score="0" Body="&lt;p&gt;The good thing about genetic algorithms is that they are exchangeable.&#xA;If you have the fitness of each individual, any algorithm (i.e. roulette, rank, tournament) will do.&lt;/p&gt;&#xA;" OwnerUserId="15530" LastActivityDate="2020-07-29T09:48:10.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22738" PostTypeId="1" CreationDate="2020-07-29T10:27:50.280" Score="2" ViewCount="94" Body="&lt;p&gt;When I read the paper, &lt;a href=&quot;https://arxiv.org/pdf/2001.04451.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reformer: The Efficient Transformer&lt;/a&gt;, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-product, memory efficient, and LSH attention.&lt;/p&gt;&#xA;&lt;p&gt;The memory complexity of the memory-efficient method is as follow:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max \left(b n_{h} l d_{k}, b n_{h} l^{2}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;: batch size&#xA;&lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;: sequence length&#xA;&lt;span class=&quot;math-container&quot;&gt;$n_h$&lt;/span&gt;: the number of attention head&#xA;&lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt;: the dimension of query or key&lt;/p&gt;&#xA;&lt;p&gt;To the best of my knowledge, the memory-efficient method will do a loop for each query, therefore, the whole attention matrix will not show up.&lt;/p&gt;&#xA;&lt;p&gt;So, shouldn't the memory complexity be &lt;span class=&quot;math-container&quot;&gt;$\max(b n_h l d_k, b n_h l)=(b n_h l d_k)$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\max(b n_h l d_k,b n_h l^2)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="38960" LastEditorUserId="38960" LastEditDate="2020-07-30T02:25:04.687" LastActivityDate="2020-07-30T02:25:04.687" Title="What is the memory complexity of the memory-efficient attention in Reformer?" Tags="&lt;deep-learning&gt;&lt;papers&gt;&lt;transformer&gt;&lt;space-complexity&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22739" PostTypeId="1" CreationDate="2020-07-29T10:44:26.123" Score="0" ViewCount="56" Body="&lt;p&gt;I am writing a AlphaGo Zero clone, and sometimes in the training the policy head loss and value head loss would both be decreasing, but the total loss is increasing?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/stVXX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/stVXX.png&quot; alt=&quot;screenshot of training progress&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How is this possible? I am using Adam, and here's the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    input_tensor = Input((self.input_board_size, self.input_board_size, 2))&#xA;    x = input_tensor&#xA;    x = self.convolution_block(x)&#xA;    for _ in range(self.number_of_residual_block):&#xA;        x = self.residual_block(x)&#xA;    self.model = Model(inputs=input_tensor, outputs=[self.policy_head(x), self.value_head(x)])['categorical_crossentropy', 'mean_squared_error'])&#xA;    self.model.compile(Adam(lr=1e-4), ['categorical_crossentropy', 'mean_squared_error'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37645" LastActivityDate="2020-07-29T10:44:26.123" Title="Total loss increasing, but loss components are decreasing?" Tags="&lt;reinforcement-learning&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22740" PostTypeId="1" CreationDate="2020-07-29T11:54:33.117" Score="0" ViewCount="49" Body="&lt;p&gt;I am trying to make an AI that plays the Snake Game using the NEAT library in Python. I have most things figured out. I have chosen 6 inputs for my neural network -&amp;gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it clear in forward direction ? (1-YES/0-NO)&lt;/li&gt;&#xA;&lt;li&gt;Is it clear in left direction ? (1-YES/0-NO)&lt;/li&gt;&#xA;&lt;li&gt;Is it clear in right direction ? (1-YES/0-NO)&lt;/li&gt;&#xA;&lt;li&gt;Is food there in forward direction ? (1-YES/0-NO)&lt;/li&gt;&#xA;&lt;li&gt;Is food there in left direction ? (1-YES/0-NO)&lt;/li&gt;&#xA;&lt;li&gt;Is food there in right direction ? (1-YES/0-NO)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The output is simply an array of three number -&amp;gt; (LEFT, FORWARD, RIGHT) which tell the snake whether to continue straight, or move to left or right.&lt;/p&gt;&#xA;&lt;p&gt;I tried using different activation functions like relu, tanh, sigmoid and so on. Still I get the output was the same -&amp;gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;[0.0, 0.0, 0.0]&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I trained my network for 150 generations still no use. At times, it generated a network that is able to score at least 5 points, but that network doesn't seem to reproduce further. Like I never see it again.&lt;/p&gt;&#xA;&lt;p&gt;Here is my NEAT Config File -&amp;gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;[NEAT]&#xA;fitness_criterion     = max&#xA;fitness_threshold     = 1000&#xA;pop_size              = 10&#xA;reset_on_extinction   = False&#xA;&#xA;[DefaultGenome]&#xA;# node activation options&#xA;activation_default      = tanh&#xA;activation_mutate_rate  = 0.0&#xA;activation_options      = tanh&#xA;&#xA;# node aggregation options&#xA;aggregation_default     = random&#xA;aggregation_mutate_rate = 0.05&#xA;aggregation_options     = sum product min max mean median maxabs&#xA;&#xA;# node bias options&#xA;bias_init_mean          = 0.05&#xA;bias_init_stdev         = 1.0&#xA;bias_max_value          = 30.0&#xA;bias_min_value          = -30.0&#xA;bias_mutate_power       = 0.5&#xA;bias_mutate_rate        = 0.7&#xA;bias_replace_rate       = 0.1&#xA;&#xA;# genome compatibility options&#xA;compatibility_disjoint_coefficient = 1.0&#xA;compatibility_weight_coefficient   = 0.5&#xA;&#xA;# connection add/remove rates&#xA;conn_add_prob           = 0.5&#xA;conn_delete_prob        = 0.1 &#xA;#Ändra till 0.5?&#xA;&#xA;# connection enable options&#xA;enabled_default         = False&#xA;enabled_mutate_rate     = 0.2 &#xA;&#xA;feed_forward            = False&#xA;initial_connection      = full&#xA;#initial_connection      = full_nodirect 0.5&#xA;&#xA;# node add/remove rates&#xA;node_add_prob           = 0.5&#xA;node_delete_prob        = 0.1&#xA;&#xA;# network parameters&#xA;num_hidden              = 0 &#xA;num_inputs              = 6&#xA;num_outputs             = 3&#xA;&#xA;# node response options&#xA;response_init_mean      = 1.0&#xA;response_init_stdev     = 0.05&#xA;response_max_value      = 30.0&#xA;response_min_value      = -30.0&#xA;response_mutate_power   = 0.1&#xA;response_mutate_rate    = 0.75&#xA;response_replace_rate   = 0.1&#xA;&#xA;# connection weight options&#xA;weight_init_mean        = 0.1&#xA;weight_init_stdev       = 1.0&#xA;weight_max_value        = 30&#xA;weight_min_value        = -30&#xA;weight_mutate_power     = 0.5&#xA;weight_mutate_rate      = 0.8&#xA;weight_replace_rate     = 0.1&#xA;&#xA;[DefaultSpeciesSet]&#xA;compatibility_threshold = 2.5&#xA;&#xA;[DefaultStagnation]&#xA;species_fitness_func = max&#xA;max_stagnation       = 20&#xA;species_elitism      = 1&#xA;&#xA;[DefaultReproduction]&#xA;elitism            = 8&#xA;survival_threshold = 0.3&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to make an AI for Tic-Tac-Toe game but faced same issue. I am new to this AI World, so please do help me out. Thank You in advance !!&lt;/p&gt;&#xA;" OwnerUserId="38963" LastActivityDate="2020-07-29T11:54:33.117" Title="ALL OUTPUTS for my neural network return SAME as [0.0, 0.0, 0.0] all of time. Trying to make an AI that plays the Snake Game in Python using NEAT" Tags="&lt;python&gt;&lt;genetic-algorithms&gt;&lt;neat&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22741" PostTypeId="2" ParentId="17203" CreationDate="2020-07-29T12:11:24.683" Score="0" Body="&lt;p&gt;Some notes about &lt;strong&gt;VQ-VAE&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In the paper, they used &lt;strong&gt;PixelCNN&lt;/strong&gt; to learn the prior.&#xA;&lt;strong&gt;PixelCNN&lt;/strong&gt; is trained on images.&lt;/li&gt;&#xA;&lt;li&gt;The discrete latent variables are just the indices of the&#xA;embedding vectors. For example, you can put your embedding vectors&#xA;in an array.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For a single input image, the number of the output channels of the encoder before quantization equals the dimensionality of the embedding vectors. For an analogy, the output of the encoder is like an image but with a number of channels that equals to the dimensions in the embedding space. With that, each pixel is a vector in the space of the embedding vectors&lt;sup&gt;1&lt;/sup&gt;. The quantization is done on each pixel by mapping each pixel to the nearest embedding vector &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, each pixel can be represented by a single integer number which equals to the index of its nearest embedding (what you call the discrete latent variables). Thus, you have two representations for the quantized output, one is a large tensor with many channels (embedding vectors), and another is a simple discrete image with one channel (discrete latent variables).&lt;/p&gt;&#xA;&lt;p&gt;The one channel discrete images are used to train &lt;strong&gt;PixelCNN&lt;/strong&gt;. This is great because they are small size images. while the large tensor with the embedding vectors is used for the decoder as it holds the information needed for reconstruction.&lt;/p&gt;&#xA;&lt;p&gt;So, the discrete latent variables for the prior model and the embedding vectors for the encoder.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;sub&gt;This is just an analogy, pixels values belong to a closed interval.&lt;/sub&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="38964" LastEditorUserId="38964" LastEditDate="2020-07-30T13:07:34.110" LastActivityDate="2020-07-30T13:07:34.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22742" PostTypeId="1" CreationDate="2020-07-29T13:29:55.170" Score="2" ViewCount="125" Body="&lt;h2&gt;For example, if I have the following architecture:&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I1SWw.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I1SWw.gif&quot; alt=&quot;3 layer neural network&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each neuron in the hidden layer has a connection from each one in the&#xA;input layer.&lt;/li&gt;&#xA;&lt;li&gt;3 x 1 Input Matrix and a 4 x 3 weight matrix (for the backpropagation we have of course the transformed version 3 x 4)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But until now, I still don't understand what the point is that a neuron has 3 inputs (in the hidden layer of the example). It would work the same way, if I would only adjust one weight of the 3 connections.&lt;/p&gt;&#xA;&lt;p&gt;But in the current case the information flows only distributed over several &amp;quot;channels&amp;quot;, but what is the point?&lt;/p&gt;&#xA;&lt;p&gt;With backpropagation, in some cases the weights are simply adjusted proportionally based on the error.&lt;/p&gt;&#xA;&lt;p&gt;Or is it just done that way, because then you can better mathematically implement everything (with matrix multiplication and so on)?&lt;/p&gt;&#xA;&lt;p&gt;Either my question is stupid or I have an error in my thinking and assume wrong ideas. Can someone please help me with the interpretation.&lt;/p&gt;&#xA;&lt;p&gt;In tensorflow playground for example, I cut the connections (by setting the weight to 0), it just compansated it by changing the other still existing connection a bit more:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lNHq4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lNHq4.png&quot; alt=&quot;TensorflowImage&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38966" LastEditorUserId="38966" LastEditDate="2020-07-29T22:17:30.263" LastActivityDate="2020-08-29T04:05:15.567" Title="Why does a neuron in a multi-layer network need several input connections?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;architecture&gt;&lt;fully-connected-layer&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22744" PostTypeId="2" ParentId="22742" CreationDate="2020-07-29T15:06:25.903" Score="0" Body="&lt;p&gt;&lt;strong&gt;It doesn't.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Whether or not this is useful is another story, but it is totally fine to do that neural net you have with just one input value. Perhaps you choose one pixel of the photo and make your classification based on the intensity in that one pixel (I guess I'm assuming a black-and-white photo), or you have some method to condense an entire photograph into one value that summarizes the photo. Then each neuron in the hidden layer only has one input connection.&lt;/p&gt;&#xA;&lt;p&gt;Likewise, you are allowed to decide that the top neuron in the hidden layer should have only one input connection; just drop the other two.&lt;/p&gt;&#xA;&lt;p&gt;Again, this might not give useful results, but they're still neural networks.&lt;/p&gt;&#xA;" OwnerUserId="25529" LastActivityDate="2020-07-29T15:06:25.903" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="22745" PostTypeId="1" AcceptedAnswerId="22746" CreationDate="2020-07-29T15:58:38.247" Score="0" ViewCount="81" Body="&lt;p&gt;I want to implement this function on a voice searching application:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q(S, A) \leftarrow Q(S, A)+\alpha\left(R+\gamma Q\left(S^{\prime}, A^{\prime}\right)-Q(S, A)\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And also restricted to use epsilon-greedy policy based on a given Q-function and epsilon. I simply need a &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy for updating my q-table.&lt;/p&gt;&#xA;" OwnerUserId="38931" LastEditorUserId="22079" LastEditDate="2020-07-29T23:02:21.317" LastActivityDate="2020-07-29T23:02:21.317" Title="How can I update my Q-table in Python?" Tags="&lt;q-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22746" PostTypeId="2" ParentId="22745" CreationDate="2020-07-29T16:01:12.840" Score="2" Body="&lt;p&gt;Just try returning a function that takes the state as an input and returns the probabilities for each action in the form of a numpy array of length of the action space (set of possible actions). Here, is one attempt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def EpsilonGreedyPolicy(Q, epsilon, no_of_actions):&#xA;&#xA;    def policy(state): &#xA;        probabilities = np.ones(no_of_actions, dtype = float) * &#xA;                    epsilon / num_actions &#xA;        best_action = np.argmax(Q[state]) &#xA;        probabilities[best_action] += (1.0 - epsilon) &#xA;        return probabilities &#xA;   &#xA;    return policy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38803" LastEditorUserId="38803" LastEditDate="2020-07-29T16:20:27.873" LastActivityDate="2020-07-29T16:20:27.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22747" PostTypeId="2" ParentId="15685" CreationDate="2020-07-29T16:29:11.730" Score="0" Body="&lt;p&gt;I eventually used the keep-efficient function from &lt;a href=&quot;https://stackoverflow.com/a/32793059/453673&quot;&gt;this answer&lt;/a&gt; to calculate the Pareto front and used the k-means function to calculate the centroid of the front. This gave me the approximate knee-point of the front, which is usually the optimal solution. One of the calculations was to maximise the distance moved in x direction (dx) vs. minimising the energy consumed (e), so since the x axis needed positive maximization and the y axis needed minimization, I inverted the y axis, since &lt;code&gt;min(f(y)) = - max(-f(y))&lt;/code&gt;. This helped get the pareto front toward the top right side of the graph and both the x and y axes were maximization objectives. The optimal point calculated was the robot that had the best fitness.&lt;/p&gt;&#xA;" OwnerUserId="9268" LastActivityDate="2020-07-29T16:29:11.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22748" PostTypeId="1" AcceptedAnswerId="22749" CreationDate="2020-07-29T16:44:02.450" Score="1" ViewCount="79" Body="&lt;p&gt;I have done creating the virtual environment, creating the Q-table, initializing the q-parameters, then I made a training module and stored it in a &lt;code&gt;numpy&lt;/code&gt; array. After completion of training, I have updated the q-table and now I get the plots for the explorations But how can I code for rate decay? Here is my sample code for every step of the training module,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for step in range(max_steps): &#xA;        exploration_rate_threshold = random.uniform(0,1)&#xA;&#xA;        if exploration_rate_threshold &amp;gt; exploration_rate:&#xA;            action = np.argmax(q_table[state,:])&#xA;        else:&#xA;            action = env.action_space.sample()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38931" LastEditorUserId="22079" LastEditDate="2020-07-29T23:01:56.083" LastActivityDate="2020-07-29T23:01:56.083" Title="How can I fetch ​exploration decay rate of an iterable Q-table in Python?" Tags="&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22749" PostTypeId="2" ParentId="22748" CreationDate="2020-07-29T16:47:10.663" Score="1" Body="&lt;p&gt;Here is one way to calculate the exploration rate decay:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;exploration_rate = min_exploration_rate + \ (max_exploration_rate - min_exploration_rate) * np.exp(-exploration_decay_rate*episode)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38803" LastActivityDate="2020-07-29T16:47:10.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22750" PostTypeId="2" ParentId="17450" CreationDate="2020-07-29T18:30:55.510" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;However, I’m not sure which policy is saved&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The policy from the Monte Carlo tree search is stored, as we can get the policy estimate from the network later by passing the given state through the network, which is used to calculate the cross entropy loss to update the network's policy (summed with Mean squared error loss between value head's prediction and the actual value/reward).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Wouldn’t it be more logical to choose the move with the highest probability calculated by the policy head?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Depends on the number of searches you've performed, after thousands of simulations the MCTS would give better results, as it approximates the minimax tree.&lt;/p&gt;&#xA;" OwnerUserId="37657" LastActivityDate="2020-07-29T18:30:55.510" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22751" PostTypeId="1" CreationDate="2020-07-29T18:35:14.397" Score="3" ViewCount="53" Body="&lt;p&gt;I have an optimization problem that I'm looking for the right algorithm to solve.&lt;/p&gt;&#xA;&lt;p&gt;What I have: A large set of low-res 360 images that were taken on a regular grid within a certain area. each of these images is quite sparsely sampled and each of these 360 images has an accurate XYZ position assigned of its center. There are millions of these small images, clusters of close-by images obviously share a lot of information while images farther apart can be completely different.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is to compress these small 360 images.&lt;/p&gt;&#xA;&lt;p&gt;If two 360 images are close by each other, they can be 'warped' into each other by projecting it onto a sphere of finite distance and then moving that sphere (so a closeby 360 image can be a good aproximation of another 360 image when it has been warped that way).&lt;/p&gt;&#xA;&lt;p&gt;Based on this idea, I want to compress these small low-res 360 images by replacing each of them with:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;N (N being something like 2-5) indices into an archive of M (M being something like 50-500) different 'prototype' images (of possibly higher resolution than the low res 360 images), each of which has an XZY location assigned plus a radius&lt;/li&gt;&#xA;&lt;li&gt;N blend weights&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Such that if I want to reconstruct one of the small, sparsely sampled 360 images I take the N indices stored for this image, look-up the corresponding prototype images from the archive, warp them based on the radius of the archive image and the delta vector of archive XZY and compressed image XYZ location, and then blend the N prototype images based on the N blend weights (and possibly scale down in the prototype images are higher res)&lt;/p&gt;&#xA;&lt;p&gt;I guess this goes into the direction of Eigen Faces, but with Eigen faces each compressed face has a weight stored for each eigen-face, whereas I want that each compressed sphere only has N non-zero weights.&lt;/p&gt;&#xA;&lt;p&gt;So my input is:&#xA;a lot of small 360 images plus a XYZ location each&lt;/p&gt;&#xA;&lt;p&gt;my output should be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;an archive of M &amp;quot;prototype&amp;quot; images, each assigned an XYZ location and a projection radius&lt;/li&gt;&#xA;&lt;li&gt;all compressed spheres, with each sphere compressed to N indices and N weights&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This seems to be some non-linear least squares problem, but I wonder if someone can point me into the right direction on how to solve this?&lt;/p&gt;&#xA;&lt;p&gt;As a completely alternative approach I also looked into spherical harmonics, but with those I only get enough high-frequency details at l=6 which takes 36 coefficients which is too much and also too slow to decompress.&lt;/p&gt;&#xA;" OwnerUserId="32111" LastActivityDate="2020-07-29T18:35:14.397" Title="Looking for the proper algorithm to compress many lowres images of nearby locations" Tags="&lt;autoencoders&gt;&lt;principal-component-analysis&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22752" PostTypeId="1" CreationDate="2020-07-29T18:37:36.497" Score="0" ViewCount="48" Body="&lt;p&gt;I'm trying to generate rhymes, so it would be very helpful to have a language model where I could input a final word, and have it output a sequence of words that ends with that word.&lt;/p&gt;&#xA;&lt;p&gt;I could train my own model and reverse the direction of the mask, but I was wondering if there was any way I could use a pretrained model but apply a different mask to achieve this goal.&lt;/p&gt;&#xA;&lt;p&gt;If this isn't possible, what's the best way to sample a forward-predicting model to achieve the highest probability sentence ending with a particular word?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="37521" LastActivityDate="2020-07-29T18:37:36.497" Title="Using transformer but masking in reverse direction/smart sampling for desired final word?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22753" PostTypeId="2" ParentId="22724" CreationDate="2020-07-29T19:36:28.453" Score="0" Body="&lt;p&gt;Deep reinforcement learning might be suitable for your requirement. If possible I would recommend you to add DRL hashtag to your thread, so that it can reach to larger number of  DRL researchers and practitioners, to help you out.&lt;/p&gt;&#xA;&lt;p&gt;Thanks ,&#xA;Durga&lt;/p&gt;&#xA;" OwnerUserId="33878" LastActivityDate="2020-07-29T19:36:28.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22754" PostTypeId="2" ParentId="22729" CreationDate="2020-07-29T20:01:41.173" Score="0" Body="&lt;p&gt;Let's go part-by-part.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For a NODE within a layer, if we have p inputs, we would multiply a 1xp coefficient matrix with a px1 input vector, correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In practice, for that LAYER, we would multiply a mxp coefficient matrix by a px1 input vector, where m is the number of nodes within that layer?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These two are correct. No issues here. The only differences may be in implementation, the logic here is sound.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If we have a bias term, then it would be a mx(p+1) coefficient matrix and a (p+1)x1 input vector?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is correct. Intuitively, it is analogous to &amp;quot;offest&amp;quot; term in the equation of a line, &lt;code&gt;y = mx + c&lt;/code&gt;. &lt;code&gt;c&lt;/code&gt; here plays the same role as the bias.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In terms of customization, for most out of the box NNs, do we choose to have a bias term or not? Or for the most part, it automatically includes one.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Most out-of-the box NN include a bias term by default, since they do provide flexibility to the network, no matter how small. Practically, their utility varies with architecture and application. For example, Feed Forward networks have a good boost in performance with biases when used for regression. CNNs used for classification usually aren't affected much if biases are removed.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In terms of customization, this combination of inputs is always linear? There are no other ways to combine the inputs (such as including polynomials, interaction terms, etc...)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Non-linear combinations are definately possible. But using such combinations basically kills the purpose of ANNs - to have simple stacked operations approximate a complex function. Non-linear combinations may help in rare cases, but more often than not, will probably lead to underfit or overfit models. By convention, any non-linearity is introduced into the model on top of the linear combination, as the activation function.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For each node within the same hidden layer, are the activation functions usually the same, or differ among each node?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Conventionally, all nodes in the same layer have the same activation. This makes vectorization easy, and ultimately doesn't really affect preformance. Instead of varying activations of individual neurons, complex architectures use parallel layers having different activations but with the same inputs. Usually, all stacked layers in models share the same activations, except maybe the last layer and in some cases, the penultimate layer.&lt;/p&gt;&#xA;" OwnerUserId="33346" LastActivityDate="2020-07-29T20:01:41.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22755" PostTypeId="2" ParentId="22702" CreationDate="2020-07-29T20:19:06.420" Score="0" Body="&lt;p&gt;The answer above makes some great comparisons/trade-offs. To help address the non-linearity issue with eLU units that the previous answer brings up, you can also use Leaky-ReLU units, which are linear in both the positive and negative range, and piecewise-linear across the whole real domain.&lt;/p&gt;&#xA;&lt;p&gt;Please see the link here for more details: &lt;a href=&quot;https://qr.ae/pNsatH&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://qr.ae/pNsatH&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36611" LastEditorUserId="2444" LastEditDate="2020-07-29T20:40:17.857" LastActivityDate="2020-07-29T20:40:17.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22756" PostTypeId="2" ParentId="16214" CreationDate="2020-07-29T21:45:32.013" Score="0" Body="&lt;p&gt;I think you are looking for the field known as &lt;strong&gt;explainable artificial intelligence&lt;/strong&gt;. The book &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning: A Guide for Making Black Box Models Explainable&lt;/a&gt; will surely help you to understand the issues and existing techniques. See also the following question &lt;a href=&quot;https://ai.stackexchange.com/q/12870/2444&quot;&gt;Which explainable artificial intelligence techniques are there?&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-07-29T21:45:32.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22757" PostTypeId="2" ParentId="22742" CreationDate="2020-07-29T22:35:51.733" Score="0" Body="&lt;p&gt;If you adopt a slightly different point-of-view, then a neural network of this static kind is just a big function with parameters, &lt;span class=&quot;math-container&quot;&gt;$y=F(x,P)$&lt;/span&gt;, and the task of training the network is a non-linear fit of this function to the data set.&lt;/p&gt;&#xA;&lt;p&gt;That is, training the network is to reduce all of the residuals &lt;span class=&quot;math-container&quot;&gt;$y_k-F(x_k,P)$&lt;/span&gt; simultaneously. This is a balancing act, just tuning one weight to adjust one residual will in general worsen some other residuals. Even if that is taken into account, methods that adjust one variable at a time are usually much slower than methods that adjust all variables simultaneously along some gradient or Newton direction.&lt;/p&gt;&#xA;&lt;p&gt;The usual back-propagation algorithm sequentializes the gradient descent method for the square sum of the residuals. Better variants improve that to a Newton-like method by some estimate of the Hessean of this square sum or following along the idea of the Gauß-Newton method.&lt;/p&gt;&#xA;" OwnerUserId="38980" LastActivityDate="2020-07-29T22:35:51.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22759" PostTypeId="2" ParentId="22742" CreationDate="2020-07-30T02:23:13.043" Score="2" Body="&lt;p&gt;There's a few reasons I can think of, though I have not read an explicit description of why it is done this way. It's likely that people just started doing it this way because it's most logical, and people who have attempted to try your method of having reduced connections have seen a performance hit and so no change was made.&lt;/p&gt;&#xA;&lt;p&gt;The first reason is that if you allow all nodes from one layer to connect to all others in the next, the network will optimise unnecessary connections out. Essentially, the weighting of these connections will become 0. This, however, does not mean you can trim these connections, as ignoring them in this local minima might be optimal, but later it might be really important these connections remain. As such, you can never truly know if a connection between one layer and the next is necessary, so it's just better to leave it in case it helps improve network performance.&lt;/p&gt;&#xA;&lt;p&gt;The second reason is it's just simpler mathematically. Networks are implemented specifically so it's very easy to apply a series of matrix calculations to perform all computations. Trimming connections means either:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A matrix must contain 0 values, wasting computation time&lt;/li&gt;&#xA;&lt;li&gt;A custom script must be written to calculate this networks structure, which in the real world can take a very long time as it must be implemented using something like CUDA (on a GPU level, making it very complicated)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Overall, it's just a lot simpler to have all nodes connected between layers, rather than on connection per node.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-07-30T02:23:13.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22760" PostTypeId="1" AcceptedAnswerId="22762" CreationDate="2020-07-30T04:03:38.467" Score="0" ViewCount="82" Body="&lt;p&gt;Similarly to the question, &lt;a href=&quot;https://ai.stackexchange.com/questions/7838/what-is-artificial-intelligence&quot;&gt;What is artificial intelligence?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://luca-d3.com/data-speaks/technology-dictionary/cognitive-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cognitive Intelligence&lt;/a&gt;, as well as being a part of Artificial Intelligence, is an area that mainly covers the technology and tools that allow our apps, websites, and bots to see, hear, speak and understand the needs of the user through natural language&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What is the definition of Cognitive Intelligence?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2020-07-31T03:38:18.213" LastActivityDate="2020-07-31T03:38:18.213" Title="What is Cognitive Intelligence?" Tags="&lt;terminology&gt;&lt;definitions&gt;&lt;history&gt;&lt;academia&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22761" PostTypeId="1" CreationDate="2020-07-30T06:06:19.590" Score="1" ViewCount="132" Body="&lt;p&gt;I am working on an object detection model and have thought of looking into stratified splits for the dataset.&lt;/p&gt;&#xA;&lt;p&gt;Now since I am doing object detection I have a variable number of &amp;quot;labels&amp;quot; for every image because in each image there is variable number of occurrences for each object I am looking for (car, truck, motorbike, etc.).&lt;/p&gt;&#xA;&lt;p&gt;Obviously single-label stratification does not apply.&lt;/p&gt;&#xA;&lt;p&gt;From what I understand multi-label stratification is only applicable if there are basically label &amp;quot;features&amp;quot; that we know are always present, which does not seem the case here.&lt;/p&gt;&#xA;&lt;p&gt;My question is... is there a way to perform stratified split in this case so that in each split there is roughly the same number of cars/trucks/bikes/etc.? (Or is it going to actually improve the results at all?)&lt;/p&gt;&#xA;" OwnerUserId="22885" LastActivityDate="2020-07-30T06:06:19.590" Title="Multilabel stratified split for images/object detection" Tags="&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22762" PostTypeId="2" ParentId="22760" CreationDate="2020-07-30T06:48:21.893" Score="1" Body="&lt;p&gt;The term it comes from &lt;strong&gt;cognitive science&lt;/strong&gt; - depending on the paradigm, it can have many meanings correlated with &lt;strong&gt;neuroscience&lt;/strong&gt; as well as the brain.&lt;/p&gt;&#xA;&lt;p&gt;Activities based on this sphere is :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;language&lt;/li&gt;&#xA;&lt;li&gt;learning&lt;/li&gt;&#xA;&lt;li&gt;thinking&lt;/li&gt;&#xA;&lt;li&gt;perception&lt;/li&gt;&#xA;&lt;li&gt;awareness&lt;/li&gt;&#xA;&lt;li&gt;decision making&lt;/li&gt;&#xA;&lt;li&gt;intelligence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The mentioned cognitive processes - activate the appropriate regions in the brain narrow cognitive intelligence - it's an effective response to in particular in unforeseen and uncertain situations.&lt;/p&gt;&#xA;&lt;p&gt;However, it requires the ability to &lt;strong&gt;set goals&lt;/strong&gt;, &lt;strong&gt;learn&lt;/strong&gt;, &lt;strong&gt;plan&lt;/strong&gt; and &lt;strong&gt;think about your own thinking&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="32352" LastActivityDate="2020-07-30T06:48:21.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22763" PostTypeId="1" CreationDate="2020-07-30T07:00:11.330" Score="0" ViewCount="97" Body="&lt;p&gt;I have a dataset where each of the training instances is different in the length and the data is sequential. So, I design an LSTM but I am thinking about how to train the LSTM. In fixed-length data, we just keep all of the input in an array and pass it to the network, but here the case is different. I can not store varying length data in an array and I do not want to use padding to make it fixed length. Now, should I train the LSTM where each training instance are varying in length?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-07-30T09:08:12.317" Title="How to train an LSTM with varying length input?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;tensorflow&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22764" PostTypeId="1" CreationDate="2020-07-30T07:37:22.307" Score="0" ViewCount="65" Body="&lt;p&gt;I built a simple X*Y grid world environment to learn and then trained my agent over it. All worked fine and the agent learned as well. Let me give some detail about the environment.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Environment&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A 4x4 grid world with episode starting at (0,0) and terminal state (3,3)&lt;/li&gt;&#xA;&lt;li&gt;Four actions: Left, Up, Right, Down&lt;/li&gt;&#xA;&lt;li&gt;The reward of -1 for moving into new state from the previous state to a new state. The reward of 0 when reaching the terminal state. The reward of -2 for bouncing off of the boundary&lt;/li&gt;&#xA;&lt;li&gt;Epsilon-greedy scheme for action selection.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;All works fine, and the following are the learning results of the agent.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mw96G.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mw96G.jpg&quot; alt=&quot;Mean reward of QL-agent after 50 simulation runs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Later I ran a test run of my &lt;strong&gt;TRAINED&lt;/strong&gt; QL-agent where I used greedy action selection. All I could see in every episode was that my agent start from (0,0), take right to move to (1,0), and then take left to move back to (0,0) again and this goes on and on and on... I check the Q table and it makes sense because the Q-values for these actions justifies such behaviour. But this is not a practical agent should be doing.&lt;/p&gt;&#xA;" OwnerUserId="36710" LastActivityDate="2020-07-30T07:37:22.307" Title="Strange behavior of Q-learning agent after being trained" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;environment&gt;&lt;testing&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22765" PostTypeId="2" ParentId="22763" CreationDate="2020-07-30T08:48:25.230" Score="2" Body="&lt;p&gt;You could sequentially pass in each element of your sequential data and save the hidden and cell states in a separate buffer. In a typical LSTM implementation, you input the entire sequence and the hidden and cell states are propagated internally. In the end, the final hidden and cell states returned as the output. This works if your input is all the same length. Instead, you can handle sequentially giving the next element to the LSTM as well as the hidden and cell state yourself. To keep it efficient you can batch your inputs by the batch dimension (batch_first=True in the &lt;a href=&quot;https://pytorch.org/docs/master/generated/torch.nn.LSTM.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pytorch LSTM implementation&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;For example, propose you have 5 sequences of length 5, 4, 3, 2, and 1. You initialize your hidden and cell state buffers for each of the sequences and pass the first batch containing the first element of all 5 sequences. You save the output hidden and cell states in the buffers for each sequence. Next, you input the batch of the second element of the 4 sequences with length &amp;gt; 1, and save the states in the respective sequence buffers and so and so forth until you exhaust the sequence of greatest length.&lt;/p&gt;&#xA;" OwnerUserId="38985" LastEditorUserId="38985" LastEditDate="2020-07-30T09:08:12.317" LastActivityDate="2020-07-30T09:08:12.317" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22766" PostTypeId="1" AcceptedAnswerId="22771" CreationDate="2020-07-30T09:25:14.900" Score="1" ViewCount="93" Body="&lt;p&gt;Afaik, investigating meta reinforcement learning algorithms requires a collection of two or more environments which have similar structure but are still different enough.&#xA;When I read &lt;a href=&quot;https://arxiv.org/abs/1703.03400&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; it was unclear to me what the meta-training and meta-testing environments were.&lt;/p&gt;&#xA;&lt;p&gt;For eg., a graph is given for Ant-Fwd-Bkwd showing its performance with number of gradient steps. I'm guessing these are the meta-testing performances. So, which environment was it 'meta-trained' on?&lt;/p&gt;&#xA;&lt;p&gt;Was it meta-trained on the same Ant-Fwd-Bkwd environment?&lt;/p&gt;&#xA;" OwnerUserId="35679" LastActivityDate="2020-08-29T15:05:00.437" Title="How are mujoco environments used for meta-rl?" Tags="&lt;reinforcement-learning&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22767" PostTypeId="1" CreationDate="2020-07-30T10:40:25.393" Score="0" ViewCount="48" Body="&lt;p&gt;I have a dataset of 180k images of license plates (so, not necessary to localize the license plate at first) for which I try to recognize the characters on the images (License plate recognition). All of these license plates contain seven characters and 35 characters are possible, so the output vector y is of shape (7, 35). I therefore onehot-encoded every license plate label.&lt;/p&gt;&#xA;&lt;p&gt;I applied the bottom of the EfficicentNet-B0 model (&lt;a href=&quot;https://keras.io/api/applications/efficientnet/#efficientnetb0-function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/api/applications/efficientnet/#efficientnetb0-function&lt;/a&gt;) together with a customized top, which is divided in 7 branches (because of seven characters per license plate). I used the weights of the imagenet and freezed the bottom layers of efnB0_model:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efnB0_model = efn.EfficientNetB0(include_top=False, weights=&amp;quot;imagenet&amp;quot;, input_shape=(224, 224, 3))&#xA;efnB0_model.trainable = False&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using Transfer Learning because the authors in this paper did this as well and got very good results regarding accuracy: &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-981-13-1733-0_6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/chapter/10.1007/978-981-13-1733-0_6&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The top of the model is constructed similar to some works in literature following their segmentation-free approaches (so, the step segmentation is not necessary). This are the two papers I used as starting points for the top of my model: &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8078501&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/abstract/document/8078501&lt;/a&gt; &amp;amp; &lt;a href=&quot;https://dl.acm.org/doi/abs/10.1145/3009977.3010052&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dl.acm.org/doi/abs/10.1145/3009977.3010052&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def create_model(input_shape = (224, 224, 3)):&#xA;    input_img = Input(shape=input_shape)&#xA;    model = efnB0_model (input_img)&#xA;    model = GlobalAveragePooling2D(name='avg_pool')(model)&#xA;    model = Dropout(0.2)(model)&#xA;    backbone = model&#xA;&#xA;    branches = []&#xA;    for i in range(7):&#xA;        branches.append(backbone)&#xA;        branches[i] = Dense(360, name=&amp;quot;branch_&amp;quot;+str(i)+&amp;quot;_Dense_16000&amp;quot;)(branches[i])&#xA;        branches[i] = BatchNormalization()(branches[i])&#xA;        branches[i] = Activation(&amp;quot;relu&amp;quot;) (branches[i])&#xA;        branches[i] = Dropout(0.2)(branches[i])&#xA;          &#xA;        branches[i] = Dense(35, activation = &amp;quot;softmax&amp;quot;, name=&amp;quot;branch_&amp;quot;+str(i)+&amp;quot;_output&amp;quot;)(branches[i])&#xA;&#xA;    output = Concatenate(axis=1)(branches)&#xA;    output = Reshape((7, 35))(output)&#xA;    model = Model(input_img, output)&#xA;&#xA;    return model&#xA;&#xA;opt = keras.optimizers.Adam(learning_rate=0.0001)&#xA;model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=[&amp;quot;accuracy&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For training and validating the model I only use 10.000 training images and 3.000 validation images due to the technical constraints in Colab and the huge number of data which would make my training very, very slow (one epoch lasts 45 minutes).&lt;/p&gt;&#xA;&lt;p&gt;I use this DataGenerator to feed batches to my model and extend the code when I add &lt;code&gt;GaussianNoise&lt;/code&gt; or &lt;code&gt;Snowflakes&lt;/code&gt; (but I will not paste the code here due to redability):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; class DataGenerator(Sequence):&#xA;&#xA;    def __init__(self, x_set, y_set, batch_size):&#xA;        self.x, self.y = x_set, y_set&#xA;        self.batch_size = batch_size&#xA;&#xA;    def __len__(self):&#xA;        return math.ceil(len(self.x) / self.batch_size)&#xA;&#xA;    def __getitem__(self, idx):&#xA;        batch_x = self.x[idx*self.batch_size : (idx + 1)*self.batch_size]&#xA;        batch_x = np.array([resize(imread(file_name), (224, 224)) for file_name in batch_x])&#xA;        batch_x = batch_x * 1./255&#xA;        batch_y = self.y[idx*self.batch_size : (idx + 1)*self.batch_size]&#xA;        batch_y = np.array(batch_y)&#xA;&#xA;        return batch_x, batch_y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I fit the model using this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.fit_generator(generator=training_generator,&#xA;                    validation_data=validation_generator,&#xA;                    steps_per_epoch = num_train_samples // 16,&#xA;                    validation_steps = num_val_samples // 16,&#xA;                    epochs = 10, workers=6, use_multiprocessing=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are my results after trying different approaches:&#xA;When I apply Transfer Learning to the model illustrated above I get overfitting, so huge differences regarding between training and validation accuracy (training accuracy increases but validation accuracy stops at around 0.18x.&#xA;When I only use the EfficientNet architecture + customized top without any other technique, I get training accuracy and validation accuracy around 0.18x.&#xA;When I add &lt;code&gt;GaussianNoise&lt;/code&gt; or &lt;code&gt;Snowflakes&lt;/code&gt;, I also get training and validation accuracy around 0.18x.&#xA;When I use a more simple model, I also get values around 0.18x.&lt;/p&gt;&#xA;&lt;p&gt;So, I am wondering, what I should check or what I can improve regarding my model? Do you think, there is something completely wrong with my architecture?&lt;/p&gt;&#xA;&lt;p&gt;I would say, four reasons are possible:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;regarding the overfitting: data volume too low and model too complex&lt;/li&gt;&#xA;&lt;li&gt;preprocessing (failures, but i do not think so) or inadequate (regarding the scaling of the images and a distortion of the characters)&lt;/li&gt;&#xA;&lt;li&gt;data quality: different data than in other approaches and lower data quality than in some other papers&lt;/li&gt;&#xA;&lt;li&gt;Architecture is not adequate: positioning is misaligned&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What do you think?&lt;/p&gt;&#xA;" OwnerUserId="38119" LastActivityDate="2020-07-30T10:40:25.393" Title="ConvNet - What to improve regarding architecture, procedure and technique?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;overfitting&gt;&lt;accuracy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22768" PostTypeId="1" CreationDate="2020-07-30T12:15:20.420" Score="2" ViewCount="60" Body="&lt;p&gt;What I mean to say is&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For example, if I give the meaning of Apple from the dictionary as input to the program, it should give output as Apple.&lt;/li&gt;&#xA;&lt;li&gt;Or I say My day to day job involves monitoring and managing the resources - the output should be Project management.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The meaning and the word could be a dictionary or it could be custom. I am looking for ideas and tools to go further on this.&#xA;Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="37468" LastActivityDate="2020-08-29T22:04:00.393" Title="How to predict the &quot;word&quot; based on the meaning in a document?" Tags="&lt;natural-language-processing&gt;&lt;training&gt;&lt;python&gt;&lt;natural-language-understanding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22769" PostTypeId="1" CreationDate="2020-07-30T13:31:15.327" Score="4" ViewCount="134" Body="&lt;p&gt;What is the difference between eager learning and lazy learning?&lt;/p&gt;&#xA;&lt;p&gt;How does eager learning or lazy learning help me build a neural network system? And how can I use it for any target function?&lt;/p&gt;&#xA;" OwnerUserId="38931" LastEditorUserId="2444" LastEditDate="2020-07-31T14:13:50.097" LastActivityDate="2020-07-31T14:13:50.097" Title="What is eager learning and lazy learning?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22771" PostTypeId="2" ParentId="22766" CreationDate="2020-07-30T13:36:01.240" Score="0" Body="&lt;p&gt;According to &lt;a href=&quot;https://arxiv.org/abs/1903.08254&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (PEARL):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;These locomotion task families require adaptation across reward&#xA;functions (walking direction for Half-CheetahFwd-Back, Ant-Fwd-Back,&#xA;Humanoid-Direc-2D, target velocity for Half-Cheetah-Vel, and goal&#xA;location for Ant-Goal2D) or across dynamics (random system parameters&#xA;for Walker-2D-Params).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It looks like different versions of the same environment with differing reward functions are used. For eg., Forward direction might be rewarded positively in one version, Negative in another, Both directions rewarded positively in yet another.&lt;/p&gt;&#xA;" OwnerUserId="35679" LastActivityDate="2020-07-30T13:36:01.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22772" PostTypeId="2" ParentId="22717" CreationDate="2020-07-30T13:36:20.130" Score="0" Body="&lt;p&gt;I have have been part of a project where we implemented Amazon Forecast service in production. As per my practical experience, we need not worry about stationarity while applying DeepAR.&lt;/p&gt;&#xA;&lt;p&gt;As per my understanding as in ARIMA the prediction function is a function around the time series moving average, but in DeepAR we are more relying on the backtest window and leg-horizon length.&lt;/p&gt;&#xA;&lt;p&gt;DeepAR uses recurrent neural Networks (RNN) and gated recurrent unit (GRU) without any assumption on their probability distribution to learn the sequence for prediction.&lt;/p&gt;&#xA;&lt;p&gt;For a detailed explanation, you can refer. &lt;a href=&quot;https://aws.amazon.com/blogs/machine-learning/forecasting-time-series-with-dynamic-deep-learning-on-aws/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://aws.amazon.com/blogs/machine-learning/forecasting-time-series-with-dynamic-deep-learning-on-aws/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25978" LastActivityDate="2020-07-30T13:36:20.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22773" PostTypeId="1" CreationDate="2020-07-30T13:41:35.270" Score="1" ViewCount="57" Body="&lt;p&gt;I understand that the actor-critic method is probably where I want to start because of how it works with continuous action spaces.&lt;/p&gt;&#xA;&lt;p&gt;However, the problem I am trying to solve would require the action be a vector of 11 continuous values. When I go to design my training environment and the architecture of my DRL network, I am not sure how to map a vector of values to the state for the state-action pairs.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to use this article as a jumping off point, but I am not sure where to go: &lt;a href=&quot;https://medium.com/@asteinbach/actor-critic-using-deep-rl-continuous-mountain-car-in-tensorflow-4c1fb2110f7c&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@asteinbach/actor-critic-using-deep-rl-continuous-mountain-car-in-tensorflow-4c1fb2110f7c&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38992" LastActivityDate="2020-07-30T13:41:35.270" Title="What is the best way to make a deep reinforcement learning environment with a continuous 2D action space?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22774" PostTypeId="1" CreationDate="2020-07-30T13:51:18.330" Score="1" ViewCount="54" Body="&lt;p&gt;If I want to find a (linear) subspace onto which a data-set projects well, I can simply use PCA. However, often the data can project with much smaller error if I first separate it into a couple of classes and then perform PCA for each class individually. But what If I don't know what kind of classes there might be in my data and into how many classes it would make sense to split the data? What kind of machine learning algorithm can do this well?&lt;/p&gt;&#xA;&lt;p&gt;Example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ti4f4.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ti4f4.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I'd just cluster first based on distance in the high-dimensional space, I would arrive at the bad clustering. There are 5 clusters and the green and red clusters don't project very well onto a 2D subspace.&lt;/p&gt;&#xA;&lt;p&gt;As a human looking at the data, I see however that if I separate the data as indicated, red and blue will project very well onto a plane each and green will project very well onto a line, so I can run PCA for each group individually.&lt;/p&gt;&#xA;&lt;p&gt;How can I automate this clustering based on how well it will project onto as low-imensional subspaces as possible?&lt;/p&gt;&#xA;&lt;p&gt;Something like minimize E = SumOverClusters(SumOverPoints(SquaredDist(projected_point, original_point)) * (number_dims_projected / number_dims_original)) + C * number_of_clusters&lt;/p&gt;&#xA;&lt;p&gt;What technique is well suited to do that?&lt;/p&gt;&#xA;&lt;p&gt;(edit: while the example shows a 3d space, I'm more interested in doing that in about 64dimensional spaces)&lt;/p&gt;&#xA;" OwnerUserId="32111" LastEditorUserId="32111" LastEditDate="2020-07-30T13:56:31.527" LastActivityDate="2020-07-30T13:56:31.527" Title="How to cluster data points such that the number of clusters is kept minimal and each cluster projects well onto a lower-dimensional subspace?" Tags="&lt;clustering&gt;&lt;dimensionality-reduction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22776" PostTypeId="1" AcceptedAnswerId="22777" CreationDate="2020-07-30T19:40:30.153" Score="2" ViewCount="326" Body="&lt;p&gt;I started looking into the &lt;em&gt;double DQN&lt;/em&gt; (DDQN). Apparently, the difference between DDQN and DQN is that in DDQN we use the main value network for action selection and the target network for outputting the Q values.&lt;/p&gt;&#xA;&lt;p&gt;However, I don't understand why would this be beneficial, compared to the standard DQN. So, in simple terms, what exactly is the advantage of DDQN over DQN?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-11-04T17:15:24.143" LastActivityDate="2020-11-04T17:15:24.143" Title="What exactly is the advantage of double DQN over DQN?" Tags="&lt;comparison&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22777" PostTypeId="2" ParentId="22776" CreationDate="2020-07-30T20:08:55.683" Score="2" Body="&lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning there is what is known as a maximisation bias. That is because the update target is &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \max_a Q(s,a)$&lt;/span&gt;. If you slightly overestimate your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value then this error gets compounded (there is a nice example in the Sutton and Barto book that illustrates this). The idea behind tabular double &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is to have two &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-networks, &lt;span class=&quot;math-container&quot;&gt;$Q_1,Q_2$&lt;/span&gt;, and you choose an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from them, e.g. from &lt;span class=&quot;math-container&quot;&gt;$Q_1 + Q_2$&lt;/span&gt;. You then flip a coin to decide which to update. If you choose to update &lt;span class=&quot;math-container&quot;&gt;$Q_1$&lt;/span&gt; then the update target becomes &lt;span class=&quot;math-container&quot;&gt;$r + \gamma Q_2(s', \arg\max_a Q_1(s',a))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The idea is that if you overshoot your estimate on one &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; network then having the second will hopefully control this bias when you would take the max.&lt;/p&gt;&#xA;&lt;p&gt;In Deep Double &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning the idea is essentially the same but instead of having to maintain and train two &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-networks, they use the target network from vanilla DQN to provide the target. To make this more concrete, the update target they use is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$r + \gamma Q(s', \arg\max_aQ(s',a;\theta);\theta^-)\;,$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Q(s,a;\theta^-)$&lt;/span&gt; denotes the target network whose parameters are only updated to the current networks every &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; time steps.&lt;/p&gt;&#xA;&lt;p&gt;As before, the idea is that if we have overestimated our value of being state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; in our current network when taking the max action, using the target network to provide the target will help control for this bias.&lt;/p&gt;&#xA;&lt;h2&gt;Maximisation Bias&lt;/h2&gt;&#xA;&lt;p&gt;I will here explain maximisation bias from the simple example given from the Sutton and Barto book.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d3B7w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d3B7w.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Markov Decision Process in the image is defined as follows: we start in state A and can take the 'right' action which gives us 0 reward and immediately leads to termination. If we choose 'left' we get 0 immediate reward where we then move to state B. From there, we have an arbitrary number of action we can take where they all lead to the terminal state and the reward is drawn from a Normal(-0.1,1) distribution.&lt;/p&gt;&#xA;&lt;p&gt;Clearly, the optimal action is always to move to the right from state A as this gives 0 expected future returns. Taking the left action will give a &lt;span class=&quot;math-container&quot;&gt;$\gamma \times -0.1$&lt;/span&gt; expected future returns (the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is our discount factor).&lt;/p&gt;&#xA;&lt;p&gt;Now, if we got into state &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and took some random action our initial reward could be bigger than 0 -- after all it is drawn from a Normal(-0.1,1) distribution.&lt;/p&gt;&#xA;&lt;p&gt;Now, consider we are updating our &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-function for state A and taking the left action. Our update target will be &lt;span class=&quot;math-container&quot;&gt;$0 + \gamma \max_a Q(B,a)$&lt;/span&gt;. Because we are taking the max over all possible actions, this will lead to a positive reward and so we are backing up the belief of our expected future rewards from taking action left in state A to be something positive -- clearly this is wrong since we know it should be -0.1. This is what is known as the maximisation bias, because it gives us a kind of 'optimistic' estimate of the action value!&lt;/p&gt;&#xA;&lt;p&gt;I've attached an image below that shows the %age of time the agent chose the left action, which it &lt;em&gt;shouldn't&lt;/em&gt; be choosing). As you can see, it takes normal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning along time to even start to correct itself, whereas double &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning corrects the mistake almost immediately.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4hepx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4hepx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-07-31T13:26:07.227" LastActivityDate="2020-07-31T13:26:07.227" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22779" PostTypeId="2" ParentId="22768" CreationDate="2020-07-30T21:52:41.230" Score="1" Body="&lt;p&gt;While your question has some ambiguities, I try to answer.&lt;/p&gt;&#xA;&lt;p&gt;From my understanding you want your model to predict “topic” of a sentence or a description. It’s just a classification problem with huge possible number of output classes.&lt;/p&gt;&#xA;&lt;p&gt;The first initial issue is very short length of documents (sentences). Most of topic modelling algorithms such as LDA have statistical approach and do not work very well with very short documents (less than 50 words could be a good definition of very short document).&lt;/p&gt;&#xA;&lt;p&gt;The second issue is how do you want to collect enough data to train your model that supposed to predict target out of extremely large number of output classes? Dictionaries are not enough because they offer a single definition for each word. Examples of words in dictionaries don’t help much and they will probably affect your model adversely. How can your model be generalised by a single (or few) example(s) for each class?&lt;/p&gt;&#xA;&lt;p&gt;So, it’s not possible, but maybe having some innovations can help.&lt;/p&gt;&#xA;&lt;p&gt;Here is definition of “apple” in oxford dictionary: ”a round fruit with shiny red or green skin that is fairly hard and white inside”. There are just two nouns in the definition: &amp;quot;fruit&amp;quot; and &amp;quot;Skin&amp;quot;, if we just read the definition without considering these two words, even we, as human, struggle to guess.&lt;/p&gt;&#xA;&lt;p&gt;Consider nouns in input data and use them to build up a natural graph. You just consider main classes such as &amp;quot;fruit&amp;quot;. If you’re getting some good results, consider other words, adj, adv, ...&lt;/p&gt;&#xA;" OwnerUserId="38892" LastEditorUserId="38892" LastEditDate="2020-07-30T21:58:58.803" LastActivityDate="2020-07-30T21:58:58.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22782" PostTypeId="1" CreationDate="2020-07-31T02:29:51.773" Score="0" ViewCount="44" Body="&lt;p&gt;For instance, consider the following piece of text:&lt;/p&gt;&#xA;&lt;p&gt;'The father of Richard is a very nice guy. He was born in a poor family. Because of that, Richard learnt very good values. Richard is also a very nice guy. However, Richard's mother embarasses the family. She was born rich and she does not know the real value of the money. She did not have to be a hard worker to succeed in life.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;How should a perfect coreference should work? Is this solution the perfect solution?&lt;/p&gt;&#xA;&lt;p&gt;Cluster 1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'The father of Richard' (first sentence) &amp;lt;-&amp;gt; 'He' (second sentence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Cluster 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'Richard' (third sentence) &amp;lt;-&amp;gt; 'Richard' (forth sentence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Cluster 3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'Richard\'s mother' (fifth sentence) &amp;lt;-&amp;gt; She (sixth sentence) &amp;lt;-&amp;gt; she (sixth sentence) &amp;lt;-&amp;gt; She (seventh sentence)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I use the coreference library of spacy (neuralcoref), I get these clusters:&lt;/p&gt;&#xA;&lt;p&gt;Clusters:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[Richard: [The father of Richard, He, Richard, Richard, Richard], a poor family: [a poor family, the family], Richard's mother: [Richard's mother, She, she, She]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this output says that &amp;quot;Richard&amp;quot; is the same for sentences that is true. However, &amp;quot;He&amp;quot; in the second sentence is not related to &amp;quot;Richard&amp;quot;, but to his father. Also &amp;quot;Richard&amp;quot; and the &amp;quot;Father of Richard&amp;quot; are together in the same cluster. Furthermore, &amp;quot;poor family&amp;quot; and &amp;quot;family&amp;quot; should not come together. However, this is realy difficulty since in this case there is some level of ambiguity.'&lt;/p&gt;&#xA;&lt;p&gt;I know that this is a very difficult problem. The point is not criticize this fantastic library. I am just trying to understand what I should expect as perfect result.&lt;/p&gt;&#xA;&lt;p&gt;If I change a little the text:&lt;/p&gt;&#xA;&lt;p&gt;'The mother of Richard is a very nice woman. She was born in a poor family. Because of that, Richard learnt very good values. Richard is a very nice guy. However, Richard's father embarasses the family. He was born rich and he does not know the real value of the money. He did not have to be a hard worker to succeed in life.'&lt;/p&gt;&#xA;&lt;p&gt;The clusters are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[Richard: [The mother of Richard, She, Richard, Richard, Richard, He, he, He], a poor family: [a poor family, the family]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this case, the clusters become stranger, since &amp;quot;She&amp;quot; and &amp;quot;Richard&amp;quot; are in the same cluster. Furthermore, the &amp;quot;He&amp;quot; related to the &amp;quot;father of Richard&amp;quot; belongs to the cluster, but not &amp;quot;Richard's father&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;So, my question is:&lt;/p&gt;&#xA;&lt;p&gt;What is the perfect result that I should expect from a &amp;quot;perfect&amp;quot; coreference system?&lt;/p&gt;&#xA;" OwnerUserId="38831" LastActivityDate="2021-05-06T21:02:00.223" Title="NLP: What is expected from the output of a perfect coreference system?" Tags="&lt;natural-language-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22783" PostTypeId="1" CreationDate="2020-07-31T02:48:34.320" Score="5" ViewCount="151" Body="&lt;p&gt;In &lt;a href=&quot;https://rlchina.org/lectures/lecture2.pdf#page=17&quot; rel=&quot;noreferrer&quot;&gt;these slides&lt;/a&gt;, it is written&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\left\|T^{\pi} V-T^{\pi} U\right\|_{\infty} &amp;amp; \leq \gamma\|V-U\|_{\infty} \tag{9} \label{9} \\&#xA;\|T V-T U\|_{\infty} &amp;amp; \leq \gamma\|V-U\|_{\infty} \tag{10} \label{10}&#xA;\end{align}&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is the space of functions on domain &lt;span class=&quot;math-container&quot;&gt;$\mathbb{S}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$T^{\pi}: \mathbb{F} \mapsto \mathbb{F}$&lt;/span&gt; is the Bellman&#xA;&lt;em&gt;policy&lt;/em&gt; operator&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$T: \mathbb{F} \mapsto \mathbb{F}$&lt;/span&gt; is the Bellman&#xA;&lt;strong&gt;optimality&lt;/strong&gt; operator&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://rlchina.org/lectures/lecture2.pdf#page=19&quot; rel=&quot;noreferrer&quot;&gt;slide 19&lt;/a&gt;, they say that equality &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; follows from&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;{\scriptsize&#xA;\left\| &#xA;T^{\pi} V-T^{\pi} U &#xA;\right\|_{\infty}&#xA;= &#xA;\max_{s} \gamma \sum_{s^{\prime}} \operatorname{Pr}&#xA;\left(&#xA;s^{\prime} \mid s, \pi(s)&#xA;\right)&#xA;\left|&#xA;V\left(s^{\prime}\right) - U &#xA;\left(s^{\prime}\right)&#xA;\right| \\&#xA;\leq \gamma \left(\sum \operatorname{Pr} \left(s^{\prime} \mid s, \pi(s)\right)\right) \max _{s^{\prime}}\left|V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right| \\&#xA;\leq \gamma\|U-V\|_{\infty}&#xA;}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why is that? Can someone explain to me this derivation?&lt;/p&gt;&#xA;&lt;p&gt;They also write that inequality \ref{10} follows from&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;{\scriptsize&#xA;\|T V-T U\|_{\infty}&#xA;= \max_{s} &#xA;\left| &#xA;\max_{a}&#xA;\left\{ &#xA;R(s, a) + \gamma \sum_{s^{\prime}} \operatorname{Pr}&#xA;\left(&#xA;s^{\prime} \mid s, a&#xA;\right) V&#xA;\left(&#xA;s^{\prime}&#xA;\right)&#xA;\right\}&#xA;-\max_{a} \left\{R(s, a)+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right) U\left(s^{\prime}\right)\right\} \right| \\&#xA;\leq \max _{s, a}\left|R(s, a)+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right) V\left(s^{\prime}\right)&#xA;-R(s, a)-\gamma \sum \operatorname{Pr}\left(s^{\prime} \mid s, a\right) V\left(s^{\prime}\right) \right| \\&#xA;= &#xA;\gamma \max _{s, a}\left|\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right)\left(V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right)\right| \\&#xA;\leq \gamma\left(\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right)\right) \max _{s^{\prime}}\left|\left(V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right)\right| \\&#xA;\leq &#xA;\gamma\|V-U\|_{\infty}&#xA;}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain to me also this derivation?&lt;/p&gt;&#xA;" OwnerUserId="20581" LastEditorUserId="2444" LastEditDate="2020-08-10T13:01:13.533" LastActivityDate="2020-08-14T11:06:19.043" Title="Why are the Bellman operators contractions?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;policy-iteration&gt;&lt;bellman-equations&gt;&lt;bellman-operators&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22784" PostTypeId="1" AcceptedAnswerId="22794" CreationDate="2020-07-31T03:53:49.427" Score="1" ViewCount="126" Body="&lt;p&gt;AFAIK, deep learning became popular in 2012 with the victory of &lt;a href=&quot;http://www.image-net.org/challenges/LSVRC/2012/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ImageNet Competition - Large Scale Visual Recognition Challenge 2012&lt;/a&gt; where winners of this contest actually used deep learning techniques for optimizing the solution for object recognition.&lt;/p&gt;&#xA;&lt;p&gt;Who first coined the term deep learning? Is there any published research paper that first used that term?&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="2444" LastEditDate="2020-07-31T14:18:59.277" LastActivityDate="2020-08-01T03:19:33.563" Title="Who first coined the term &quot;deep learning&quot;?" Tags="&lt;deep-learning&gt;&lt;terminology&gt;&lt;history&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22786" PostTypeId="1" AcceptedAnswerId="22823" CreationDate="2020-07-31T08:34:01.520" Score="1" ViewCount="56" Body="&lt;p&gt;Normally, in practice, people use those loss functions with minima, e.g. &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; mean absolute loss, &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; mean squared error, etc. All those come with a minimum to optimize to.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/RSccA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RSccA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, there's another thing, logistic loss, I'm reading about, but don't get it why the logistic function could be used as a loss function, given that it has the so-called minimum at infinity, but that isn't a normal minimum. Logistic loss function (black curve):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0pREW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0pREW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can an optimizer minimize the logistic loss?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2020-08-02T12:54:14.483" LastActivityDate="2020-08-02T12:54:14.483" Title="How could logistic loss be used as loss function for an ANN?" Tags="&lt;loss-functions&gt;&lt;optimization&gt;&lt;logistic-regression&gt;&lt;neural-networks&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22787" PostTypeId="1" CreationDate="2020-07-31T09:02:03.270" Score="1" ViewCount="71" Body="&lt;p&gt;When we use DDQN, we often use the target network in case our online network overestimates a value, but this doesn't make sense to me, because&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What happens if our target network is the one that overestimated a value, then we’d keep using that overestimated value&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why can't we use both target network for selection and evaluation&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-07-31T12:32:25.327" LastActivityDate="2020-07-31T12:32:25.327" Title="What happens if our target network overestimates the value?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22788" PostTypeId="1" CreationDate="2020-07-31T09:57:39.023" Score="0" ViewCount="35" Body="&lt;p&gt;I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something.&#xA;Let's say you have pathology/biopsy tissue images from patients dying from a disease A and patients died from other causes (whatever causes actually but not related to disease A).&lt;/p&gt;&#xA;&lt;p&gt;To date, I think we can say that nobody actually really knows what causes at the level of a biopsy the disease A. My idea, as my group could have actually a lot of these biopsies for both groups, would be to use them to fuel a neural network. Why would I do that? Biopsies images are rather complex images, and maybe some fine details are hard to guess for a human being, or maybe the sum of some details is actually important to tell whether disease A kills the patient or not. But again, I don't think anybody could come and say: on those tissue biopsy, the sign(s) for disease A are x, y, z.&lt;/p&gt;&#xA;&lt;p&gt;My question then becomes a bit more theoretical: &lt;em&gt;given the fact that you have enough data to actually give chances to the algorithm to find differences, is it a good idea to train a neural network without having actually any idea of what could differentiate the two groups?&#xA;Do you know examples of such a strategy? How hard is it afterwards - in the case of a rather good accuracy - to understand what makes it so recognisable?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="40012" LastEditorUserId="2444" LastEditDate="2020-07-31T20:55:50.357" LastActivityDate="2020-07-31T20:55:50.357" Title="Is it a good idea to train a neural network to classify images without base-hypothesis?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;research&gt;&lt;theory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22789" PostTypeId="1" CreationDate="2020-07-31T10:02:27.017" Score="3" ViewCount="138" Body="&lt;p&gt;I know we should scale the input and output (assuming regression task) before we feed it to the neural network. Then the gradient descent will give the better minima much faster. But I have subtle confusion whether gradient descent with feature scale and without feature scale gives the same result or just gradient descent is not scale-invariant.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-07-31T10:41:31.227" LastActivityDate="2021-04-27T14:34:48.513" Title="Is gradient descent scale invariant or not?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22791" PostTypeId="2" ParentId="22789" CreationDate="2020-07-31T10:29:05.647" Score="-1" Body="&lt;p&gt;What are you hoping to get out of the answer for this question?&#xA;Feature scaling is a method you CAN(but don't have to) use, so that your algorithm performs faster and reaches better general accuracy.&#xA;I would say that on a simple regression task, where the feature value ranges do not vary a lot, the output would probably be almost the same, but as soon as you introduce data that has one feature of range 1-10, and other range of 10000-100000, that is where you would notice that you NEED to standardize/normalize your features in order to reach optimal results.&#xA;That's why it's almost a general rule to just scale your data, so your algorithm can generalize better, and you don't have to worry about your algorithm giving higher importance to a feature with higher values, instead of the one with lower values(just an example).&lt;/p&gt;&#xA;" OwnerUserId="38919" LastActivityDate="2020-07-31T10:29:05.647" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22792" PostTypeId="2" ParentId="22769" CreationDate="2020-07-31T11:16:15.687" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is eager learning or lazy learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Eager learning is when a model does all its computation &lt;strong&gt;before&lt;/strong&gt; needing to make a prediction for unseen data. For example, Neural Networks are eager models.&lt;/p&gt;&#xA;&lt;p&gt;Lazy learning is when a model doesn't require any &lt;em&gt;training&lt;/em&gt;, but all of its computation &lt;strong&gt;during inference&lt;/strong&gt;. An example of such a model is k-NN. Eager learning is also known as instance-based learning [&lt;a href=&quot;https://www.researchgate.net/publication/228075681_Hybrid_algorithms_for_instance-based_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://link.springer.com/content/pdf/10.1023/A:1007613203719.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;http://www.floodsite.net/html/partner_area/project_docs/Solomatine_et_al.,_2006b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How does eager and lazy learning help me build a neural network system? And how can I use it for any target function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To answer your second question, you can't employ &lt;em&gt;lazy learning&lt;/em&gt; to train a neural network, because they are inherently &lt;em&gt;eager models&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastEditorUserId="2444" LastEditDate="2020-07-31T14:11:13.933" LastActivityDate="2020-07-31T14:11:13.933" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22794" PostTypeId="2" ParentId="22784" CreationDate="2020-07-31T12:23:12.810" Score="0" Body="&lt;p&gt;The term was introduced to the machine learning and computer science community by &lt;a href=&quot;https://scholar.google.com/citations?user=PI_3bqEAAAAJ&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rina Dechter&lt;/a&gt; in &lt;a href=&quot;https://www.aaai.org/Library/AAAI/1986/aaai86-029.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning While Searching in Constraint-Satisfaction-Problems&lt;/a&gt; (1986) [&lt;a href=&quot;http://www.scholarpedia.org/article/Deep_Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], where she writes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Discovering all minimal conflict-sets amounts to acquiring all the possible information out of a dead-end. Yet, such &lt;strong&gt;deep learning&lt;/strong&gt; may require considerable amount of work.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When &lt;strong&gt;deep learning&lt;/strong&gt; is used in conjunction with restricting the level of learning we get deep first-order learning (identifying minimal conflict sets of size 1) and deep second-order learning (i.e. identifying minimal conflict-sets of sizes 1 and 2).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our experiments (implemented in LISP on a Symbolits LISP Machine) show that in most cases both performance measures improve as we move from &lt;strong&gt;shallow learning&lt;/strong&gt; to &lt;strong&gt;deep learning&lt;/strong&gt; and from first-order to second order.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, note that the term &lt;em&gt;deep learning&lt;/em&gt; was used before 1986 in other contexts, for example, in [&lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/BF00155616.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]. Moreover, note that Rina Dechter did not use the term in the context of neural networks, which was probably used later.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-01T03:19:33.563" LastActivityDate="2020-08-01T03:19:33.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22795" PostTypeId="1" CreationDate="2020-07-31T14:27:14.333" Score="0" ViewCount="27" Body="&lt;p&gt;I am working on a problem similar to this one:(supervised, artificial data)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x=np.random.uniform(-100,100,10000)&#xA;y=np.random.uniform(-100,100,10000)&#xA;z=np.random.uniform(-100,100,10000)&#xA;a=np.random.uniform(-100,100,10000)&#xA;b=np.random.uniform(-100,100,10000)&#xA;&#xA;i= x**2/1000 + 2*x*y/1000 + 4*np.sqrt(abs(z)) + z*a + 4*a + abs(b)**1.5 -1/((b+1)**2) * a + 10*y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since I am not creating the data myself I want to make sure, that my customer provided all the relevant input features. Is there a way to find out, whether the input is complete and not lacking a feature, say &amp;quot;a&amp;quot;? Obviously if the input is the same and the output differs it would be evidence of missing data but it isn't guaranteed that any two input samples are the same. Another way I thought of would be to use an autoencoder to find the dimension of the dataset(including the output) and hope it is exactly the input dimension but in my case it is also possible that there are redundant features. Is there any other way to check whether a function is computable from the given inputs?&lt;/p&gt;&#xA;" OwnerUserId="40019" LastActivityDate="2020-07-31T14:27:14.333" Title="Finding whether an input column is missing" Tags="&lt;machine-learning&gt;&lt;supervised-learning&gt;&lt;dimensionality&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22796" PostTypeId="1" CreationDate="2020-07-31T15:12:02.400" Score="0" ViewCount="49" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/76S3y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/76S3y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know the reason why there's a FCNN after the self-attention layer? Or at least some intuition for it?&lt;/p&gt;&#xA;" OwnerUserId="40020" LastActivityDate="2020-07-31T15:12:02.400" Title="What is the reason for implementing position-wise Feed Forward Network in Transformer?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22797" PostTypeId="1" CreationDate="2020-07-31T15:34:48.043" Score="0" ViewCount="126" Body="&lt;p&gt;I decided to start learning neural networks by creating a bot for the game. One of the intermediate steps is to create a global map from a series of inaccurate overlapping sub-maps. This task can be solved using OpenCV, but this solution will be too limited and sensitive (in the future I intend to complicate the task and work directly with the map image, instead of binary masks).&lt;/p&gt;&#xA;&lt;p&gt;I've tried the following options:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;predict the position of a new map area within the global map. (as a probability distribution)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;predict the new state of the global map from the old and new minimap.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I've tried a lot of options formulation of the problem of network architecture, including the idea of conjoined networks, but nothing gave any relevant results.&lt;/p&gt;&#xA;&lt;p&gt;Some articles about solving similar problems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1877050918315278&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automatic Field-of-View Expansion using Deep Features and Image Stitching&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1809.04137&quot; rel=&quot;nofollow noreferrer&quot;&gt;JigsawNet: Shredded Image Reassembly using Convolutional Neural Network and Loop-based Composition&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.mdpi.com/1424-8220/19/23/5310/htm&quot; rel=&quot;nofollow noreferrer&quot;&gt; Robust Cylindrical Panorama Stitching for Low-Texture Scenes Based on Image Alignment Using Deep Learning and Iterative Optimization&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is an example of one of the options statement of the problem:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VCITr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VCITr.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38036" LastEditorUserId="38036" LastEditDate="2020-08-02T22:05:46.297" LastActivityDate="2020-08-02T22:05:46.297" Title="How to choice CNN architecture for stitching images" Tags="&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22799" PostTypeId="1" CreationDate="2020-08-01T01:48:10.950" Score="1" ViewCount="45" Body="&lt;p&gt;Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. Those tasks usually comes with an easily and clearly defined value function, which is the objective function for the AI to optimize.&lt;/p&gt;&#xA;&lt;p&gt;My question is, how is DNN reinforcement learning or related techniques to be applied to AI system that are designed to just imitate human but not outperform human? Note this is different from a human-like system. Our objective here is to let the AI become a human rather than a super intelligence. For example, if human consistently makes a mistake in image identification, then the AI system must also make the same mistake. Another example is the classic chatbot to pass the Turing test. Is DNN reinforcement learning useful in these kind of tasks? I find it is really hard to start with because the value function cannot be easily calculated. What are some theory behind this?&lt;/p&gt;&#xA;" OwnerUserId="38299" LastActivityDate="2020-12-30T10:03:30.297" Title="The are some fundamental learning theories for developing an AI that imitates human behavior" Tags="&lt;deep-learning&gt;&lt;theory&gt;&lt;human-like&gt;&lt;value-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22800" PostTypeId="1" AcceptedAnswerId="22822" CreationDate="2020-08-01T02:33:50.033" Score="0" ViewCount="61" Body="&lt;p&gt;I am trying to create a CNN model that classifies if a person is wearing a seatbelt or not to verify they drive safely. I know to get images of people wearing seatbelts and people not wearing seatbelts, but I have a problem.&lt;/p&gt;&#xA;&lt;p&gt;What if the person doesn’t submit a picture of them in a car at all? How do I construct the rest of the dataset to determine if that picture is an actual picture of a person wearing a seatbelt?&lt;/p&gt;&#xA;&lt;p&gt;Do I insert completely random pictures in a different category? Do I classify images that don’t have a high confidence score as &amp;quot;wrong&amp;quot; images? Or leave it?&lt;/p&gt;&#xA;" OwnerUserId="40029" LastEditorUserId="2444" LastEditDate="2020-08-02T20:52:46.300" LastActivityDate="2020-08-02T20:52:46.300" Title="How to handle images that don’t pertain to image classifier at all?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22804" PostTypeId="2" ParentId="22800" CreationDate="2020-08-01T08:25:39.863" Score="1" Body="&lt;p&gt;Yes, a category &amp;quot;no person&amp;quot; or &amp;quot;random image&amp;quot; would make sense. Binary classification is only helpful if you know that your input always belongs to one or the other category, for example by pre-filtering the inputs.&lt;/p&gt;&#xA;" OwnerUserId="22993" LastActivityDate="2020-08-01T08:25:39.863" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22805" PostTypeId="1" CreationDate="2020-08-01T09:28:50.910" Score="1" ViewCount="89" Body="&lt;p&gt;The genetic algorithm consists of 5 phases of which 4 are repeated:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initial population (initially)&lt;/li&gt;&#xA;&lt;li&gt;Fitness function&lt;/li&gt;&#xA;&lt;li&gt;Selection&lt;/li&gt;&#xA;&lt;li&gt;Crossover&lt;/li&gt;&#xA;&lt;li&gt;Mutation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In the selection phase, the number of solutions decreases. How is it avoided to run out of the population before reaching a suitable solution?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2021-01-30T21:54:16.373" LastActivityDate="2021-01-30T21:54:16.373" Title="How to avoid running out of solutions in genetic algorithm due to selection?" Tags="&lt;genetic-algorithms&gt;&lt;genetic-operators&gt;&lt;selection-operators&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22806" PostTypeId="1" CreationDate="2020-08-01T09:34:42.580" Score="0" ViewCount="43" Body="&lt;p&gt;I have two questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Since &lt;a href=&quot;https://towardsdatascience.com/understanding-region-of-interest-part-2-roi-align-and-roi-warp-f795196fc193&quot; rel=&quot;nofollow noreferrer&quot;&gt;both use bilinear interpolation&lt;/a&gt;, why is RoI Align &lt;strong&gt;discrete&lt;/strong&gt; while PrRoI Pooling &lt;strong&gt;continuous&lt;/strong&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Tn6TE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Tn6TE.jpg&quot; alt=&quot;comparison&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Could anyone explain the intuition behind the derivative of PrPool()?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7sh24.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7sh24.png&quot; alt=&quot;PrRoI derivative&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20581" LastEditorUserId="2444" LastEditDate="2020-08-02T14:00:03.760" LastActivityDate="2020-08-02T14:00:03.760" Title="Since both RoI Align and PrRoI Pooling use bilinear interpolation, why is RoI Align discrete while PrRoI Pooling continuous?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;r-cnn&gt;&lt;pooling&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22807" PostTypeId="1" CreationDate="2020-08-01T10:36:17.417" Score="0" ViewCount="39" Body="&lt;p&gt;I’ve looked at various articles and I’m still very confused, I understand the normal double Q learning about having two Action value estimates that use two different set of samples&lt;/p&gt;&#xA;&lt;p&gt;But coming to neural networks I’m confused&#xA;The normal DQN algorithm uses our target network for both action selection and evaluation when performing updates&lt;/p&gt;&#xA;&lt;p&gt;I was told that they initialize them to small weights to prevent overestimation, if that’s the case why exactly are we now using the online network to select actions&lt;/p&gt;&#xA;&lt;p&gt;I mean our target network can handle that since it’ll never overestimate values&lt;/p&gt;&#xA;&lt;p&gt;Can someone shed more light on this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="37831" LastActivityDate="2020-08-01T10:36:17.417" Title="Having trouble understanding how Double deep Q networks work" Tags="&lt;deep-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22808" PostTypeId="1" CreationDate="2020-08-01T12:13:08.553" Score="2" ViewCount="71" Body="&lt;p&gt;Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?&lt;/p&gt;&#xA;" OwnerUserId="36000" LastEditorUserId="2444" LastEditDate="2020-12-31T21:28:42.947" LastActivityDate="2020-12-31T21:28:42.947" Title="What are some applications where tree models perform better than neural networks?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;decision-tree&gt;&lt;random-forest&gt;&lt;gradient-boosting&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22809" PostTypeId="2" ParentId="22805" CreationDate="2020-08-01T14:33:32.750" Score="2" Body="&lt;p&gt;There are multiple ways to interpret those steps. The most common standard approaches are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;select two parents and produce two offspring; repeat until child population is the same size as parent population, and let the children replace their parents unconditionally (generational GA)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;same as the above, but allow a few parents to live on instead of a few children if the parents have higher fitness (elitism)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;each iteration, select two parents, produce one child, let the child replace a member of the parent population if it is better (steady state GA)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But there are other ways to go. There's an algorithm called CHC that lets the child population get smaller over time, and when it reaches zero, the algorithm triggers a smart restart. The point is there's no single definition for what makes an evolutionary algorithm. It's up to you to decide how to make something that works well for your problem. When you're a beginner though, it's handy to start from known points, like the three I mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastActivityDate="2020-08-01T14:33:32.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22810" PostTypeId="1" AcceptedAnswerId="22876" CreationDate="2020-08-01T16:05:07.527" Score="3" ViewCount="92" Body="&lt;p&gt;I am trying to predict the number of likes an article or a post will get using a NN.&lt;/p&gt;&#xA;&lt;p&gt;I have a dataframe with ~70,000 rows and 2 columns: &amp;quot;text&amp;quot; (predictor - strings of text) and &amp;quot;likes&amp;quot; (target - continuous int variable). I've been reading on the approaches that are taken in NLP problems, but I feel somewhat lost as to what the input for the NN should look like.&lt;/p&gt;&#xA;&lt;p&gt;Here is what I did so far:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Text cleaning: removing html tags, stop words, punctuation, etc...&lt;/li&gt;&#xA;&lt;li&gt;Lower-casing the text column&lt;/li&gt;&#xA;&lt;li&gt;Tokenization&lt;/li&gt;&#xA;&lt;li&gt;Lemmatization&lt;/li&gt;&#xA;&lt;li&gt;Stemming&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I assigned the results to a new column, so now I have &amp;quot;clean_text&amp;quot; column with all the above applied to it. However, I'm not sure how to proceed.&lt;/p&gt;&#xA;&lt;p&gt;In most NLP problems, I have noticed that people use word embeddings, but from what I have understood, it's a method used when attempting to predict the next word in a text. Learning word embeddings creates vectors for words that are similar to each other syntax-wise, and I fail to see how that can be used to derive the weight/impact of each word on the target variable in my case.&lt;/p&gt;&#xA;&lt;p&gt;In addition, when I tried to generate a word embedding model using the Gensim library, it resulted in more than 50k words, which I think will make it too difficult or even impossible to onehot encode. Even then, I will have to one hot encode each row and then create a padding for all the rows to be of similar length to feed the NN model, but the length of each row in the new column I created &amp;quot;clean_text&amp;quot; varies significantly, so it will result in very big onehot encoded matrices that are kind of redundant.&lt;/p&gt;&#xA;&lt;p&gt;Am I approaching this completely wrong? and what should I do?&lt;/p&gt;&#xA;" OwnerUserId="38887" LastEditorUserId="38887" LastEditDate="2020-08-09T18:21:53.983" LastActivityDate="2020-08-09T18:21:53.983" Title="How to use text as an input for a neural network - regression problem? How many likes/claps an article will get" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22811" PostTypeId="2" ParentId="22805" CreationDate="2020-08-01T18:08:07.860" Score="1" Body="&lt;p&gt;This is a more complex question than it might initially seem. A genetic algorithm models a biological process,namely population genetics. No biological population evolves to a single cloned individual, a process in genetic algorithms referred to as premature convergence where the population converges to a single non optimal, though possibly locally optimal, solution. The avoidance of premature convergence or the maintenance of population diversity is an important aspect of the genetic model that is often not well addressed, and one that the five step model you detail definitely does not.&lt;/p&gt;&#xA;&lt;p&gt;The one operator that will maintain diversity is mutation, since it is a purely random operator. However, what the mutation rate should be is highly argued over. A general consensus is that if each chromosome is of length N then the mutation rate should be 1/N. Likewise, the consensus is that 60% of the population should be replaced in each breeding cycle.&lt;/p&gt;&#xA;&lt;p&gt;However, these settings do not emerge directly from biological reality and premature convergence remains problematic. A more realistic model is to reflect the fact that in biology resources are finite, and to adjust the fitness of individuals proportionate to the number of similar individuals on the assumption that similar individuals are chasing the same resource. The fitness landscape is thus dynamically warped by  the changing distribution of the population. You will still have to retain memory of the fittest solution before adjustment. A common solution is to apply cluster analysis to the population, reducing the individual’s fitness by the size of the cluster to which it is allotted. A seminal paper is by &lt;a href=&quot;https://www.semanticscholar.org/paper/A-Fast-Genetic-Algorithm-with-Sharing-Scheme-Using-Yin-Germay/87e16bb2c15dbe699b84ef07722661c6acdff88e&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yin and Germay A Fast Genetic Algorithm with Sharing Scheme Using Cluster Analysis Methods in Multimodal Function Optimization &lt;/a&gt;`. The assumption is still made that the population is modelling a single biological species. How diversity does not merely maintain diversity but results in a population dividing into separate reproductively isolated species is a question for another day, and one that divides biologists to the current day.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastActivityDate="2020-08-01T18:08:07.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22812" PostTypeId="1" CreationDate="2020-08-01T18:56:51.520" Score="0" ViewCount="82" Body="&lt;p&gt;&lt;em&gt;In the concept of the vanilla policy gradient algorithm, is it possible for our trajectory size to be fixed?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example, my environment is the space of embedded images (using a pre-trained encoder to take images in a space with smaller dimensions), the action I am performing is clustering via k-means algorithm and the reward is the silhouette score metric applied on the clustered images.&lt;/p&gt;&#xA;&lt;p&gt;I am thinking to set batches of size 100 (dataset is MNIST and its trainset size is 60000). Then take the mean of them and consider this as one observation. Then feed this into the policy network to give me its logits which is an array of size 20 (20 discrete actions). These actions tell me the number of k clusters in the k-means clustering algorithm. One value for k is sampled and the k-means algorithm is applied on these 100 images and then reward is calculated.&lt;/p&gt;&#xA;&lt;p&gt;I can set a constant number for trajectory sizes, for example, 20 and sum the rewards to get the R(trajectory). &lt;em&gt;Is this possible in the context of the RL and policy gradient, or the trajectory's size cannot be fixed?&lt;/em&gt; Also, the action that our policy gives us must lead us to the next observation in the environment, but here images are independent of the policy network's parameters.&lt;/p&gt;&#xA;&lt;p&gt;I wonder if I can utilize RL to implement this. I appreciate any hints.&lt;/p&gt;&#xA;" OwnerUserId="37744" LastEditorUserId="2444" LastEditDate="2020-08-02T11:51:05.087" LastActivityDate="2020-12-30T14:04:25.363" Title="Is it possible to have a fixed trajectory size in the vanilla policy gradient algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22813" PostTypeId="2" ParentId="22805" CreationDate="2020-08-01T20:56:25.897" Score="2" Body="&lt;p&gt;It is not true that the number of solutions necessarily decreases during the selection phase (if by solutions you mean the number of individuals in the population). The number of solutions is usually constant, i.e., you can start with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; individuals, then, every iteration (or generation), you can e.g. select two individuals from the population (typically, the fittest ones, but you can have some more sophisticated selection criteria), then you merge them to create two new individuals (i.e. crossover), which will then replace (with a certain probability) the two least fit individuals from the current population, so the population's size remains constant.&lt;/p&gt;&#xA;&lt;p&gt;If you are talking about reaching a local minimum, i.e. none of the solutions in the population are &amp;quot;good enough&amp;quot;, then, as someone has already suggested, there are potentially multiple ways to address this issue, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;increase the population size&lt;/li&gt;&#xA;&lt;li&gt;run the genetic algorithm for a longer time (if you have the resources)&lt;/li&gt;&#xA;&lt;li&gt;change your genetic operators (i.e. the mutation and crossover) so that to introduce more diversity&lt;/li&gt;&#xA;&lt;li&gt;tweak the replacement, mutation, and crossover rates&lt;/li&gt;&#xA;&lt;li&gt;change your selection strategy (there are many selection strategies)&lt;/li&gt;&#xA;&lt;li&gt;make sure that the representation of the solutions is suitable (e.g. once, by mistake, I was using an array of integers rather than floating-point numbers, so I couldn't ever find the correct solution, which was an array of floating-point numbers)&lt;/li&gt;&#xA;&lt;li&gt;use something like &lt;a href=&quot;http://people.idsia.ch/%7Etino/papers/cuccu.evostar11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;novelty search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The correct approach will probably depend on the context.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-01T22:24:43.487" LastActivityDate="2020-08-01T22:24:43.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22814" PostTypeId="1" CreationDate="2020-08-01T22:07:21.563" Score="0" ViewCount="24" Body="&lt;p&gt;Consider the network with N neurons, each of which takes a &lt;span class=&quot;math-container&quot;&gt;$2 \times k$&lt;/span&gt; input specified by the tuple &lt;span class=&quot;math-container&quot;&gt;$(\vec c_t, \vec \theta_t)$&lt;/span&gt; to produce output &lt;span class=&quot;math-container&quot;&gt;$\vec{R}_t$&lt;/span&gt; through an update rule on the pairwise weights between the neurons &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W_t}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;g(x) &amp;amp;= \lfloor x \rfloor^2, \forall x \in R,\\&#xA;\vec r_t &amp;amp;= g\left(\begin{bmatrix}\vec c_t &amp;amp; b\end{bmatrix}^\intercal \begin{bmatrix}\mathbf{f}(\vec\theta_t) \\ \mathbf{1}\end{bmatrix}\right),\\&#xA;           \vec R_t &amp;amp;=  \vec r_t^\intercal  [\mathrm{diag}(\sigma^2\mathbf{1} + \mathbf{W_t} \vec r_t/n)]^{-1},\\&#xA;\Delta \mathbf{W_{t+1}} &amp;amp;= \mathbf{W_{t+1}} - \mathbf{W_{t}} = \alpha (\vec R_t^\intercal \vec R_t - \mathbf{C}) + \beta (\mathbf{W_{t}} - \mathbf{C}).&#xA;  \end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\vec c \in R^k, \vec \theta \in R^k, \vec r \in R^n, \alpha, \beta, b \in [0,1]$&lt;/span&gt;, nonlinear activation &lt;span class=&quot;math-container&quot;&gt;$\mathbf{f} \in [c_0, 1]^{nk}$&lt;/span&gt;, convex map &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; truncates all negative elements to 0 then squares each element of a vector, &lt;span class=&quot;math-container&quot;&gt;$\mathrm{diag}$&lt;/span&gt; converts its input vector to a diagonal matrix. At &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;, network is initialized using inputs &lt;span class=&quot;math-container&quot;&gt;$(\vec c_0, \vec \theta_0)$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W_0}=\mathbf{1}, C:=\vec R_0^\intercal \vec R_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Premise question on existence of stable state(s): What initial conditions of &lt;span class=&quot;math-container&quot;&gt;$b, \alpha, \beta, \sigma$&lt;/span&gt; and inputs &lt;span class=&quot;math-container&quot;&gt;$(\vec c_t, \vec\theta_t)$&lt;/span&gt; lead to stable states? That is, when does Eq.4 lead to no update at &lt;span class=&quot;math-container&quot;&gt;$t\to+\infty$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Main question&lt;/strong&gt;: Suppose the update rule Eq. 4 is applying some &amp;quot;backprop&amp;quot; trying to &amp;quot;drive&amp;quot; some network state &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; to an &amp;quot;optimal&amp;quot; state &lt;span class=&quot;math-container&quot;&gt;$E^*$&lt;/span&gt; computed by a cost function &lt;span class=&quot;math-container&quot;&gt;$E(R, ...)$&lt;/span&gt; from the network outputs &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and the network state &lt;span class=&quot;math-container&quot;&gt;$W, C, b, \sigma, \alpha, \beta$&lt;/span&gt; [1]. Can we write &lt;span class=&quot;math-container&quot;&gt;$E(R, ...)$&lt;/span&gt; explicitly in terms of the output &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; (and the initial conditions)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;My attempts&lt;/h2&gt;&#xA;&lt;p&gt;The first question requires solving &lt;span class=&quot;math-container&quot;&gt;$\vec R_t^\intercal \vec R_t = \mathbf{C},\mathbf{W_{t}} = \mathbf{C}$&lt;/span&gt;. Plugging in Eq.3 doesn't seem to simplify much. I tried taking svd of &lt;span class=&quot;math-container&quot;&gt;$\vec R_t^\intercal \vec R_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W_{t}}$&lt;/span&gt; etc., but didn't lead to much progress. I am not sure how to solve for the equilibrium states of a dynamic system specified by matrices like this.&lt;/p&gt;&#xA;&lt;p&gt;For the second question, suppose Eq.4 describes traveling along the steepest gradient in the state space &lt;span class=&quot;math-container&quot;&gt;$E(R, ...)$&lt;/span&gt;. Then it should correspond to the maximizing the differential &lt;span class=&quot;math-container&quot;&gt;$\left|\frac{\partial E_t}{\partial\mathbf{W_t}}\right|$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathrm{argmax}_{\mathcal{E}}\left|\frac{\partial E_t}{\partial\mathbf{W_t}}\right|=\alpha (\vec R_t^\intercal \vec R_t - \mathbf{C}) + \beta (\mathbf{W_{t}} - \mathbf{C})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;[1]: Inspired by &lt;a href=&quot;https://cs.nyu.edu/%7Eyann/research/ebm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cs.nyu.edu/~yann/research/ebm/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40038" LastEditorUserId="40038" LastEditDate="2020-08-01T22:20:41.113" LastActivityDate="2020-08-01T22:20:41.113" Title="Finding the energy function given update rule of a single layer non-linear neural network" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22815" PostTypeId="2" ParentId="8894" CreationDate="2020-08-02T00:10:48.800" Score="0" Body="&lt;p&gt;GANs are usually trained in a &lt;a href=&quot;https://ai.stackexchange.com/a/10624/2444&quot;&gt;&lt;strong&gt;self-supervised&lt;/strong&gt; fashion&lt;/a&gt;, i.e. they use the unlabelled data as the supervisory signal. Note that some self-supervised learning methods are unsupervised learning techniques, given that no human-annotated data is needed. However, not all SSL techniques are used for solving an unsupervised learning task. In fact, there are SSL techniques that are specifically used to generate labeled data, which can then be used to train a model in a supervised fashion.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-08-02T00:10:48.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22817" PostTypeId="1" CreationDate="2020-08-02T04:45:10.910" Score="0" ViewCount="35" Body="&lt;p&gt;I made a simple obstacle collision game in Python and made a neural network which will play the game using the NEAT (NeuroEvolution of Augmenting Topologies) and I got a final network as follows -&amp;gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; FINAL NETWORK -&amp;gt;&#xA;&#xA; Nodes:&#xA;    0 DefaultNodeGene(key=0, bias=-0.9866681151537819, response=1.0, activation=tanh, aggregation=sum)&#xA;    1 DefaultNodeGene(key=1, bias=-0.19684707599816334, response=1.0, activation=tanh, aggregation=sum)&#xA;    2 DefaultNodeGene(key=2, bias=-0.023412587732523353, response=1.0, activation=tanh, aggregation=sum)&#xA; Connections:&#xA;    DefaultConnectionGene(key=(-3, 0), weight=0.413094212700832, enabled=True)&#xA;    DefaultConnectionGene(key=(-3, 1), weight=0.38493758445483867, enabled=True)&#xA;    DefaultConnectionGene(key=(-3, 2), weight=0.3326930938849062, enabled=True)&#xA;    DefaultConnectionGene(key=(-2, 0), weight=-1.4191333690437578, enabled=True)&#xA;    DefaultConnectionGene(key=(-2, 1), weight=-1.604381939943581, enabled=True)&#xA;    DefaultConnectionGene(key=(-2, 2), weight=-1.8972709882162406, enabled=True)&#xA;    DefaultConnectionGene(key=(-1, 1), weight=-2.030443212728862, enabled=True)  &#xA;    DefaultConnectionGene(key=(-1, 2), weight=-1.0869417163203305, enabled=True)&#xA;    DefaultConnectionGene(key=(2, 2), weight=0.6586795750845641, enabled=True)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I don't know how to use this neural network to directly play the game, like how to implement this neural network. Please do help . Thank You in advance !!&lt;/p&gt;&#xA;&lt;p&gt;GitHub Link to my project - &lt;a href=&quot;https://github.com/NirajSawant136/Simple-Obstacle-Game-AI-using-NEAT-Python&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38963" LastActivityDate="2020-08-02T04:45:10.910" Title="How to interpret the final neural network generated in NEAT-Python?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;neat&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22818" PostTypeId="1" CreationDate="2020-08-02T05:21:15.000" Score="0" ViewCount="27" Body="&lt;p&gt;This is the equation where n denotes the current state, (n-1) denotes the state in the previous step etc.&lt;/p&gt;&#xA;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cbar%7By%7D(n)=b_%7B0%7D*net(n)&amp;plus;b_1*net(n-1)&amp;plus;b2*net(n-2)-a1*%5Cbar%7By%7D(n-1)-a2*%5Cbar%7By%7D(n-2)&quot; title=&quot;\bar{y}(n)=b_{0}*net(n)+b_1*net(n-1)+b2*net(n-2)-a1*\bar{y}(n-1)-a2*\bar{y}(n-2)&quot; /&gt;&#xA;&lt;p&gt;And to do back-propagation I need to find partial derivatives over each of the variables. For now let's just focus on &lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cfrac%7B%5Cpartial&amp;space;%5Cbar%7By%7D(n)%7D%7B%5Cpartial&amp;space;b_0%7D&quot; title=&quot;\frac{\partial \bar{y}(n)}{\partial b_0}&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;The term &lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cbar%7By%7D(n-1)&quot; title=&quot;\bar{y}(n-1)&quot; /&gt; in the above equation can be written as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cbar%7By%7D(n-1)=b_%7B0%7D*net(n-1)&amp;plus;b_1*net(n-2)&amp;plus;b2*net(n-3)-a1*%5Cbar%7By%7D(n-2)-a2*%5Cbar%7By%7D(n-3)&quot; title=&quot;\bar{y}(n-1)=b_{0}*net(n-1)+b_1*net(n- &#xA;2)+b2*net(n-3)-a1*\bar{y}(n-2)-a2*\bar{y}(n-3)&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since it also contains &lt;img src=&quot;https://latex.codecogs.com/gif.latex?b_%7B0%7D&quot; title=&quot;b_{0}&quot; /&gt;, it needs to be substituted into the first equation. But this is where the issue starts. I also need to then substitute &lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cbar%7By%7D(n-2)&quot; title=&quot;\bar{y}(n-2)&quot; /&gt; in the equation above with this:&lt;/p&gt;&#xA;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cbar%7By%7D(n-2)=b_%7B0%7D*net(n-2)&amp;plus;b_1*net(n-3)&amp;plus;b2*net(n-4)-a1*%5Cbar%7By%7D(n-3)-a2*%5Cbar%7By%7D(n-4)&quot; title=&quot;\bar{y}(n-2)=b_{0}*net(n-2)+b_1*net(n-3)+b2*net(n-4)-a1*\bar{y}(n-3)-a2*\bar{y}(n-4)&quot; /&gt;&#xA;&lt;p&gt;Which contains &lt;img src=&quot;https://latex.codecogs.com/gif.latex?b_%7B0%7D&quot; title=&quot;b_{0}&quot; /&gt;, and so does &lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cbar%7By%7D(n-3)&quot; title=&quot;\bar{y}(n-3)&quot; /&gt; . And I end up in a never ending loop.&lt;/p&gt;&#xA;&lt;p&gt;So how to do this?&lt;/p&gt;&#xA;" OwnerUserId="38668" LastEditorUserId="38668" LastEditDate="2020-08-02T07:13:47.223" LastActivityDate="2020-08-02T07:13:47.223" Title="How to find the derivative of a dynamic neuron model, which depends on previous states of the neuron?" Tags="&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22819" PostTypeId="2" ParentId="22808" CreationDate="2020-08-02T07:45:09.773" Score="1" Body="&lt;p&gt;Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see &lt;a href=&quot;https://ndres.me/kaggle-past-solutions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.kaggle.com/rajiao/winning-solutions-of-kaggle-competitions&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), I would say tree-based models e.g. Random Forests, Decision Trees, Gradient Boosting are favorable over neural networks when working with &lt;strong&gt;low-dimensional data&lt;/strong&gt; and easy &lt;strong&gt;interpretable features&lt;/strong&gt; (usually simple tabular data with numeric, ordinal or categorical features).&lt;/p&gt;&#xA;&lt;p&gt;Whereas when working with everything high-dimensional like images, text, time series or other data with non-trivial features, I would recommend neural networks.&lt;/p&gt;&#xA;&lt;p&gt;Of course there might be exceptions and the future may prove me wrong.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-08-02T07:45:09.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22820" PostTypeId="1" CreationDate="2020-08-02T07:46:25.900" Score="0" ViewCount="21" Body="&lt;p&gt;It is well known, that CNN have advantage with respect to the Dense neural networks in the image classification and other pattern recognition tasks, because they have a translationall invariance built in, so they do require less parameters to fit, which is advantegeous from the point of view of computational cost and overfitting.&lt;/p&gt;&#xA;&lt;p&gt;However, for example, in task of number or other enitiy recognition, from the human point of view there is a larger group of deformations, under which we still regard the object as belonging to the same class - namely rotations, rescalings, for the case, when image is located on sphere, there is also a global inversion (special conformal transform) :&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x_\mu \rightarrow \frac{x_\mu}{x^2} &#xA;$$&lt;/span&gt;&#xA;Are there any neural network architectures with a kernels, or some architecture, with activations invariant under such transformations? Such a network would further reduce drastically the number of weights to optimize and there will be no need for any data augmentation, which definitely leads to the significant reduction in training time.&lt;/p&gt;&#xA;&lt;p&gt;Even more powerful architecture would be the one, that is not invariant only under global transformations (which act with the same parameters at each point of the sample), but under a local (where the parameters of distortion depend on location on the image).&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate links to the papers which investigate the following issue!&lt;/p&gt;&#xA;" OwnerUserId="38846" LastActivityDate="2020-08-02T07:46:25.900" Title="The convolutional network architectures with enhanced invariance" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;theory&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22821" PostTypeId="2" ParentId="22799" CreationDate="2020-08-02T09:03:13.647" Score="1" Body="&lt;p&gt;There is an interesting discussion the progress achieved in this field by far in the paper of Francois Chollet - &lt;a href=&quot;https://arxiv.org/abs/1911.01547&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1911.01547&lt;/a&gt;. At the present time, many architectures are able to outperform the human in particular tasks, because they have a strong priors coded into them and ability to process a huge amount of data.&lt;/p&gt;&#xA;&lt;p&gt;However, when it comes to generalization, or even more difficult task of doing the things, that the coder has not put into the model, the present algorithms do no perform well. There is a rather sophisticated and developed mathematical definition of what is required from the system to be called intelligent. And in a nutshell, it is ability to develop a sensible behaviour and enough accuracy for new tasks without putting strong priors and large amount of experience.&lt;/p&gt;&#xA;&lt;p&gt;In the end of the paper, there is proposed a benchmark to measure the intelligence of AI system.&lt;/p&gt;&#xA;" OwnerUserId="38846" LastActivityDate="2020-08-02T09:03:13.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22822" PostTypeId="2" ParentId="22800" CreationDate="2020-08-02T11:01:17.997" Score="1" Body="&lt;p&gt;From what I understood, you want to be able to determine whether the input to your classifier is a valid picture or not. Where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Valid picture: image of a person wearing or not wearing a seatbelt&lt;/li&gt;&#xA;&lt;li&gt;Not valid picture: unrelated images (say a kitchen picture) or noise, or a black image (no input at all)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For that you could build a Bayesian model from your current deep-learning model. Check out Pyro (from pytorch). The main idea behind it, is that the model will always predict a class: person wearing or not wearing a seatbelt. But since it is a Bayesian model it will also tell you the confidence of the prediction in terms of the similarity of the input with the input distribution for which the model was trained. In other words, it will also tell you &amp;quot;how valid&amp;quot; is that prediction, or &amp;quot;how valid&amp;quot; is the input for that prediction.&lt;/p&gt;&#xA;&lt;p&gt;Check out &lt;a href=&quot;https://towardsdatascience.com/making-your-neural-network-say-i-dont-know-bayesian-nns-using-pyro-and-pytorch-b1c24e6ab8cd&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; where it is very well explained. Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-08-02T11:01:17.997" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22823" PostTypeId="2" ParentId="22786" CreationDate="2020-08-02T11:24:35.600" Score="2" Body="&lt;p&gt;I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$CE = -[y \log(p) + (1 - y) \log(1 - p)]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This tells basically if the predicted class &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; was right &lt;span class=&quot;math-container&quot;&gt;$y=1$&lt;/span&gt; then the loss is &lt;span class=&quot;math-container&quot;&gt;$CE=-\log(p)$&lt;/span&gt;, if the predicted class was not the right one then the loss is &lt;span class=&quot;math-container&quot;&gt;$CE=-\log(1-p)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If we look at the function as a pure math concept we see that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$CE = f(x) = - \log(x)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And as you point out, that function is minima-unbounded as its domain is &lt;span class=&quot;math-container&quot;&gt;$D(f(x)) = [0, +\infty]$&lt;/span&gt;. You can check that in here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dWM0J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dWM0J.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However the trick is that the inputs must be bounded, meaning, the inputs to the loss function must be in range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;. This bounding is achieved by applying a sigmoid activation function as the final &amp;quot;layer&amp;quot; of the network. Then if the inputs to the loss function are bounded the function has a clear minima.&lt;/p&gt;&#xA;&lt;p&gt;Check how the function looks in reality from one of the most important papers in loss function in the AI world: &lt;a href=&quot;https://arxiv.org/abs/1708.02002&quot; rel=&quot;nofollow noreferrer&quot;&gt;Focal Loss&lt;/a&gt; (I really encourage you to read it as the first section explains in detail the cross-entropy loss). The blue curve is the one you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/U0AbV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U0AbV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, you might want to review your log-loss/CE function since it should have an asymptote for &lt;span class=&quot;math-container&quot;&gt;$f(x=0) = \infty$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-08-02T11:24:35.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22824" PostTypeId="2" ParentId="21045" CreationDate="2020-08-02T11:36:09.927" Score="3" Body="&lt;p&gt;Yes, it is not specified because the region proposal algorithm did not change from R-CNN (the previous version from Fast R-CNN, however, in the next verion, Faster R-CNN, this algorithm is replaced by a CNN).&lt;/p&gt;&#xA;&lt;p&gt;The region proposal algorithm you are looking for is called selective search. You can find in the R-CNN paper that the algorithm is described in &amp;quot;Selective Search for Object Recognition&amp;quot;, I found a copy &lt;a href=&quot;http://www.huppelen.nl/publications/selectiveSearchDraft.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The algorithm is based on a series of segmentation and aggregation techniques of the input image for generating the proposed regions. Check it out 4 iterations of segmentation &amp;amp; aggregation over the same input image to build the proposed regions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K9vNZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K9vNZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;All the algorithm is doing is just iterating over 4 steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initial regions based on segmentation by pixel light intensity are obtained by applying a segmentation algorithm described in the paper. For example, given a picture of a shepherd with his sheep in the mountain, it is segmented by light intensity, and the image of Figure (a) is obtained.&lt;/li&gt;&#xA;&lt;li&gt;Different regions are proposed based on the previous segmentation, Figure (e)&lt;/li&gt;&#xA;&lt;li&gt;The similarity between the proposed regions is calculated using the formula proposed in equation 6&#xA;in Section 3.2 in the paper which is nothing more than an aggregate metric of the similarity of two regions based in 4 metrics: similarity in color, texture, size and fill (measures how well a region within another)&lt;/li&gt;&#xA;&lt;li&gt;Add the regions based on similarity and get Figure (b). Then return to step two.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;That is how iteratively you get all the images depicted.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastEditorUserId="26882" LastEditDate="2021-02-04T10:46:56.180" LastActivityDate="2021-02-04T10:46:56.180" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22825" PostTypeId="2" ParentId="20694" CreationDate="2020-08-02T11:47:42.740" Score="1" Body="&lt;p&gt;As you ask, &amp;quot;in general...&amp;quot;, I will answer generally, however this changes a lot from model to model and the way they handle close objects.&lt;/p&gt;&#xA;&lt;p&gt;In general, yes, they would do a poor job detecting very close objects, switch to segmentation models for that (for class or better, instance segmentation).&lt;/p&gt;&#xA;&lt;p&gt;In general, objects detectors learn to tell an object from other based in 2 criterion:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intersection over union: for object of the same class&lt;/li&gt;&#xA;&lt;li&gt;Class probability: for objects of different class&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, if two objects of the same class are very close, the 2 detected bounding boxes will be highly overlapping, then, the Non Maximal Suppression filter will remove one of them. This is where objects detector, in general, perform worse.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, if two objects belong to different classes the 2 detected bounding boxes will be highly overlapping but the NMS filter won't remove them (again, in general, NMS is set only for same class objects). However when 2 objects are very close, there is a high chance they are partially occluded. Objects detectors, in general, don't handle occlusions very well.&lt;/p&gt;&#xA;&lt;p&gt;So, in conclusion, objects detectors will perform better detecting far-away objects.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-08-02T11:47:42.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22826" PostTypeId="2" ParentId="8274" CreationDate="2020-08-02T12:51:15.903" Score="1" Body="&lt;p&gt;The paper referenced by Martin Thoma is the go-to for semantic segmentation. However I will also like to add the Panoptic Segmentation metric as an aggregated method to measure both the detection task and segmentation task of the model.&lt;/p&gt;&#xA;&lt;p&gt;It is a very well-known and widely used metric since it is the standard metric for COCO dataset (segmentation)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1801.00868&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is the paper&lt;/a&gt; where the metric is proposed.&lt;/p&gt;&#xA;&lt;p&gt;And here is the metric:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DEJa8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DEJa8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-08-02T12:51:15.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22827" PostTypeId="1" AcceptedAnswerId="22911" CreationDate="2020-08-02T13:02:45.957" Score="3" ViewCount="96" Body="&lt;p&gt;Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?&lt;/p&gt;&#xA;&lt;p&gt;I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each other.&lt;/p&gt;&#xA;" OwnerUserId="40042" LastActivityDate="2020-08-07T10:02:31.700" Title="Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?" Tags="&lt;deep-learning&gt;&lt;deep-rl&gt;&lt;multi-agent-systems&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22828" PostTypeId="1" CreationDate="2020-08-02T13:16:38.423" Score="1" ViewCount="43" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Suppose, instead of playing against a random opponent, the reinforcement learning algorithm described above played against itself, with both sides learning. What do you think would happen in this case? &lt;strong&gt;Would it learn a different policy for selecting moves?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Above is an extract from &lt;em&gt;Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton&lt;/em&gt;, and I wasn't quite sure about what the answer to the question would be, so thought of posting it here. The algorithm being referred to is the one for playing the game tic-tac-toe.&lt;/p&gt;&#xA;&lt;p&gt;In my opinion, if the same algorithm plays both sides, it may end up assisting itself to win every time - and not really learn anything. What do you think?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-08-02T20:59:52.647" LastActivityDate="2020-08-02T20:59:52.647" Title="What does self-play in reinforcement learning lead to?" Tags="&lt;reinforcement-learning&gt;&lt;self-play&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22829" PostTypeId="2" ParentId="22812" CreationDate="2020-08-02T13:55:18.150" Score="2" Body="&lt;p&gt;Trajectory size can be fixed, but in this case problem would be formulated as something similar to the multi-armed bandit problem where there is a single state and a set of actions to choose from. There is no sequential decision making since samples are not correlated, they are picked at random. So, if you take a batch of 20 examples then you would basically have 20 single state trajectories. For each of those trajectories you can calculate policy gradient and average it over the batch size (which would be 20 in this case).&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-08-02T13:55:18.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22830" PostTypeId="2" ParentId="22154" CreationDate="2020-08-02T16:11:35.317" Score="1" Body="&lt;p&gt;When it talks to other domains such as image or music, using transformer will always face a problem of sequence length limitation. To the best of my knowledge, the bottleneck of self-attention which uses a &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; matrix quite limits transformer being applied to other domains. For example, a 32x32 pixel image, means a sequence of 1024 tokens.&lt;/p&gt;&#xA;&lt;p&gt;OpenAI did some related research, as the followings.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://openai.com/blog/sparse-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Modeling with Sparse Transformers&lt;/a&gt;:&#xA;In the paper, transformers with sparse attention are applied to image and waveform.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://openai.com/blog/image-gpt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ImageGPT&lt;/a&gt;:&#xA;A large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. (Abstract from the blog)&lt;/p&gt;&#xA;" OwnerUserId="38960" LastActivityDate="2020-08-02T16:11:35.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22831" PostTypeId="1" CreationDate="2020-08-02T17:18:20.507" Score="1" ViewCount="32" Body="&lt;p&gt;In deep Q-learning, &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q'(s, a)$&lt;/span&gt; are predicted or estimated by the neural network itself. In supervised learning, the target value is a true unbiased value. However, this isn't the case in reinforcement learning. So, how can we be sure that deep Q-learning converges? How do we know that the target Q values are accurate?&lt;/p&gt;&#xA;" OwnerUserId="37831" LastEditorUserId="2444" LastEditDate="2020-08-02T21:03:53.320" LastActivityDate="2020-08-02T21:03:53.320" Title="How can deep Q-learning converge if the targets may not be correct?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22832" PostTypeId="1" AcceptedAnswerId="22849" CreationDate="2020-08-02T19:12:55.517" Score="0" ViewCount="36" Body="&lt;p&gt;I'm trying to learn to use AI, and so I've followed some basic tutorials like training an MLP to predict the price of a car given properties like its age and manufacturer. Now I want to see if I can do it myself, so I thought it'd be fun to predict what score I would give a movie given some data scraped off IMDB.&lt;/p&gt;&#xA;&lt;p&gt;I immediately got stuck, because how do you deal with the cast? A single property with multiple values, where a particular actor may impact the final score (or a combination of actors - that's for the neurons to suss out).&lt;/p&gt;&#xA;&lt;p&gt;I haven't found a way to do this when googling, but it may just be that I'm unfamiliar with the terminology. Or have I accidentally chosen a really difficult problem?&lt;/p&gt;&#xA;&lt;p&gt;Note that I'm completely new to all of this, so if you have suggestions, please try to put it as simply as possible.&lt;/p&gt;&#xA;" OwnerUserId="40048" LastEditorUserId="22079" LastEditDate="2020-08-05T12:39:18.320" LastActivityDate="2020-08-05T12:39:18.320" Title="How to input dataset with multi-value properties?" Tags="&lt;neural-networks&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22833" PostTypeId="1" CreationDate="2020-08-02T20:34:09.553" Score="0" ViewCount="30" Body="&lt;p&gt;I've been following the tutorial &lt;a href=&quot;https://arxiv.org/pdf/1604.00772.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The CMA Evolution Strategy: A Tutorial&lt;/a&gt; to try and understand the CMA-ES, but I'm having trouble understanding how the step size and the covariance matrix are been updated. Is there any intuitive and simpler explanation for that?&lt;/p&gt;&#xA;" OwnerUserId="26654" LastEditorUserId="2444" LastEditDate="2020-08-02T21:11:17.640" LastActivityDate="2020-08-02T21:11:17.640" Title="How are the step size and covariance matrix updated in CMA-ES?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22834" PostTypeId="1" CreationDate="2020-08-02T21:13:33.927" Score="1" ViewCount="57" Body="&lt;p&gt;From Wikipedia:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;According to the most popular version of the singularity hypothesis, called intelligence explosion, an upgradable intelligent agent will eventually enter a &amp;quot;runaway reaction&amp;quot; of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly, causing an &amp;quot;explosion&amp;quot; in intelligence and resulting in a powerful superintelligence that qualitatively far surpasses all human intelligence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But what if the complexity of the problem of self-improving the software grows at a faster rate than the AGI intelligence self-improvement?&lt;/p&gt;&#xA;&lt;p&gt;From experience we know that problems tend to be harder to solve at every iteration, with diminishing returns. Take as an example the theory of gravitation. Newtonian physics is relatively easy to formulate and covers the majority of high level gravitation phenomenas.&#xA;A more refined picture, like General Relativity, fill few holes in the theory with a huge increase in complexity. To describe black holes and primordial cosmology we need a theory of quantum gravity, which appears to require a further step in complexity.&lt;/p&gt;&#xA;&lt;p&gt;What &amp;quot;saved&amp;quot; us so far is the economic growth of our civilisation, which allowed more and more scientist to focus on solving the next problems. It's true that the first AGI will have the luxury of being duplicated, being a software base intelligence, but at the same time is likely that the first AGI will be extremely compute intensive. But even assuming that we (or maybe it would better to say, they) have the hardware to run &lt;span class=&quot;math-container&quot;&gt;$10^{2}$&lt;/span&gt; instances, if the complexity of every substantial self-improvement grows say by &lt;span class=&quot;math-container&quot;&gt;$10^{d}$&lt;/span&gt;x with &lt;span class=&quot;math-container&quot;&gt;$d=3$&lt;/span&gt; while the improvement to the intelligence is only &lt;span class=&quot;math-container&quot;&gt;$10^{l}$&lt;/span&gt;x with &lt;span class=&quot;math-container&quot;&gt;$l=1$&lt;/span&gt;, the self-improvement cycle will quickly slow down.&lt;/p&gt;&#xA;&lt;p&gt;So is increasing software complexity the most likely bottleneck to the AI singularity? And what are likely values for &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="16363" LastEditorUserId="1671" LastEditDate="2020-08-19T01:21:48.033" LastActivityDate="2020-08-19T01:21:48.033" Title="Is increasing software complexity the most likely bottleneck to the AI singularity?" Tags="&lt;agi&gt;&lt;singularity&gt;&lt;kolmogorov-complexity&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22835" PostTypeId="1" CreationDate="2020-08-02T21:26:15.513" Score="0" ViewCount="36" Body="&lt;p&gt;Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online network for Evaluation and our target network for selection?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="40050" LastActivityDate="2020-08-02T21:26:15.513" Title="Why do we use the Target Network for action evaluation in Double deep Q networks" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22836" PostTypeId="1" CreationDate="2020-08-02T21:59:18.640" Score="0" ViewCount="28" Body="&lt;p&gt;I have a model with TD3 + lstm in both actor and critic.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to make it learn to predict some specific actions based on the environment conditions.&#xA;However i see that the AI predicts very nearly the same actions for all the states.Moreover my problem statement has many negative rewards and there is only one kind of scenario is which the the reward is positive. In this scenario the AI needs to get the state and past state and a particular action gives high rewards. rest all gives small negative reward.&lt;/p&gt;&#xA;&lt;p&gt;Also the action it is giving is not the optimal negative loss avoidance action&lt;/p&gt;&#xA;&lt;p&gt;Can someone please help to let know the intuition what is wrong happening here.&lt;/p&gt;&#xA;" OwnerUserId="40051" LastEditorUserId="40051" LastEditDate="2020-08-03T05:58:05.677" LastActivityDate="2020-08-03T05:58:05.677" Title="TD3+lstm predicting the same output for varying states" Tags="&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22837" PostTypeId="1" AcceptedAnswerId="22840" CreationDate="2020-08-02T22:40:08.727" Score="3" ViewCount="284" Body="&lt;p&gt;I have been reading about deterministic and stochastic environments, when I came up with an article that states that tic-tac-toe is a non-deterministic environment.&lt;/p&gt;&#xA;&lt;p&gt;But why is that?&lt;/p&gt;&#xA;&lt;p&gt;An action will lead to a known state of the game and an agent has full knowledge of the board and of its enemies' past moves.&lt;/p&gt;&#xA;" OwnerUserId="40052" LastEditorUserId="2444" LastEditDate="2020-08-03T20:37:38.543" LastActivityDate="2020-08-03T20:37:38.543" Title="Why is tic-tac-toe considered a non-deterministic environment?" Tags="&lt;game-ai&gt;&lt;environment&gt;&lt;multi-agent-systems&gt;&lt;stochastic-policy&gt;&lt;deterministic-policy&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22839" PostTypeId="1" CreationDate="2020-08-03T00:45:36.980" Score="0" ViewCount="27" Body="&lt;p&gt;I’m trying to train a classifier to recognize if people are wearing seatbelts. What if the person submitted a picture unrelated to a seatbelt classifier?&lt;/p&gt;&#xA;&lt;p&gt;Would I create an image label that is full of random images to the other labels of seatbelt/ no seatbelt? Or should I train a BNN CNN to create a confidence threshold? Which would provide more accuracy?&lt;/p&gt;&#xA;" OwnerUserId="40029" LastActivityDate="2020-08-03T00:45:36.980" Title="Is Bayesian NN vs adding random data more accurate?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;bayes&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22840" PostTypeId="2" ParentId="22837" CreationDate="2020-08-03T08:29:41.793" Score="3" Body="&lt;p&gt;The game of TIC-TAC-TOE can be modelled as a non-deterministic Markov decision process (MDP) if, and only if:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The opponent is considered part of the environment. This is a reasonable approach when the goal is to solve playing against a specific opponent.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The opponent is using a stochastic policy. Stochastic policies are a  generalisation that include deterministic policies as a special case, so this is a reasonable default assumption.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;An action will lead to a known state of the game and an agent has full knowledge of the board and of Its enemies past moves.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Whilst this is true, the next state and reward as observed by an agent may not be due to the postion it plays in (with the exception being if it wins or draws on that move), but the position after the &lt;em&gt;opponent&lt;/em&gt; plays.&lt;/p&gt;&#xA;&lt;p&gt;It is also possible to frame TIC-TAC-TOE as a partially observed MDP (POMDP) if you consider the opponent to not have a fixed policy, but to be reacting to play so far, perhaps even learning from past games. In which case, the internal state of the opponent is the unknown part of the state. In standard game playing engines and in games of perfect information, this is resolved by assuming the opponent will make the best possible (or &lt;em&gt;rational&lt;/em&gt;) move, which can be determined using a search process such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax&lt;/a&gt;. When there is imperfect information, such as in poker, it becomes much harder to allow for an opponent's action.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-03T14:59:52.240" LastActivityDate="2020-08-03T14:59:52.240" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22841" PostTypeId="2" ParentId="7875" CreationDate="2020-08-03T08:45:14.823" Score="1" Body="&lt;p&gt;Yes, it must be taken seriously. There are two main reasons:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There is no sharp argument or no-go theorem against the existence of a singularity. It's unclear how fast the singularity could develop, but many authors given a non zero probability to this event (see this &lt;a href=&quot;https://www.springer.com/gp/book/9783642325595&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt;, it contains different points of view on the singularity by leading experts).&lt;/li&gt;&#xA;&lt;li&gt;The consequences of a singularity would be dramatic. So even if the perceived probability in point 1 is very small, it's worth studying possible mitigations routes.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16363" LastActivityDate="2020-08-03T08:45:14.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22842" PostTypeId="1" CreationDate="2020-08-03T08:45:55.547" Score="0" ViewCount="21" Body="&lt;p&gt;Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic intersection. It only records when something occurs, like a car passes, a pedestrian crosses, etc... But otherwise doesn't record information.&lt;/p&gt;&#xA;&lt;p&gt;I want to produce a LTSM NN (or some other memory based NN for time series data), but since my features don't occur at even time intervals, I am not sure how having a memory would help. For example let's consider two sequences:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 1&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At 1PM, I recorded a car passing.&lt;/li&gt;&#xA;&lt;li&gt;At 105, some people cross.&lt;/li&gt;&#xA;&lt;li&gt;At 150 some more people cross.&lt;/li&gt;&#xA;&lt;li&gt;At 2PM, another car passes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 2&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At 1 PM a car passes&lt;/li&gt;&#xA;&lt;li&gt;At 2 PM a car passes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the first scenario, the last car passed 3 observations ago. In the second scenario, the last car passed 1 observation ago. Yet in both scenarios, the last car passed 1 hour ago. I am afraid that any model would treat the last car passing in scenario 1 as 4 time periods ago and the last car passing in scenario 2 as 1 time period ago, even though in reality, the time difference is the same. My hypothesis is that the time difference is a very important feature, probably more so than the intermediate activity between the two car passing. In other words, knowing that the last car passed 1 hour ago is equal or likely more important than knowing that there were some people crossing in the last hour. With that said, knowing that people crossed is important too so I can't just remove that feature.&lt;/p&gt;&#xA;&lt;p&gt;Another example of my issue can be scene below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1PM Car passes&lt;/li&gt;&#xA;&lt;li&gt;2PM Car passes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1PM Car passes&lt;/li&gt;&#xA;&lt;li&gt;10PM Car passes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Once again, in my data set, this would be treated as adjacent observations, but in reality, the time gap is vastly different and thus, the two scenarios should be viewed as very dissimilar.&lt;/p&gt;&#xA;&lt;p&gt;What are some ways to solve these issues?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I've thought of just expanding the data set by creating a row for every possible time stamp, but I don't think this is the right choice as it would make my dataset humongous and most rows would have 0s across the board. I have observations that occur in microseconds so it would just become a very sparse dataset.&lt;/li&gt;&#xA;&lt;li&gt;It would be nice to include time difference as a feature, but I am not sure if there's a way to include a dynamic feature in your data set. For example, in the first scenario, at 105, the 1PM observation needs a feature that says this occurred 5 minute ago. But at 150, that feature needs to be changed to this occurred 50 minutes ago, and then at 2PM, that feature needs to now say that it occurred 1 hour ago.&lt;/li&gt;&#xA;&lt;li&gt;Would the solution to just give the NN the raw data and not worry about it? When building a NN on word prediction, I guess if you give the model enough data, it'll learn the language even if the relevant word happened 10 paragraphs ago...  However, I am not sure if there are enough examples of the exact same sequences (even with the amount of data) for it to obtain the predictability I want.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Any ideas on ways to solve this problem while keeping in mind that the goal is to build a NN?  Another way to think about it is, the time when a data point occurred relative to when the prediction will be made, in my situation, is a crucial piece of information for prediction.&lt;/p&gt;&#xA;" OwnerUserId="29801" LastActivityDate="2020-08-03T08:45:55.547" Title="What are some solutions for dealing with time series data that are recorded at uneven intervals?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;time-series&gt;&lt;feature-engineering&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22843" PostTypeId="1" CreationDate="2020-08-03T08:50:47.350" Score="1" ViewCount="59" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1805.10002.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;label propagation algorithm&lt;/a&gt; in section 3.2.3, we know the label of some nodes and we want to predict the label for the rest of the nodes whose labels we don't know. The update formula for this is the following:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F(t+1) = \alpha SF(t) + (1-\alpha)Y $$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$F(t)$&lt;/span&gt; is predicted label from timestep t and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; can be considered as an adjacency matrix, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is the label for both the unlabeled data and labeled data. In the case of labeled data, we initialize &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; with ground truth and for the unlabeled data, we randomly initialize their label and assign it to &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;.&#xA;Now, the most problematic part is I think the &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; matrix. Since I do not know the label of some nodes, so we initialize with some random value and keep Y as a constant throughout this iterative process.&#xA;We can calculate the optimal value of F directly using:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F^{*} = (I - \alpha S)^{-1}Y$$&lt;/span&gt;&#xA;But my question is, if we keep Y as a constant ( assign random numbers to unknown nodes as labels) what kind of sense does it make?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="9947" LastEditDate="2020-08-05T17:55:35.140" LastActivityDate="2020-08-05T17:55:35.140" Title="How to make sense of label propagation formula in graph neural networks?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;graph-theory&gt;&lt;graphical-model&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22844" PostTypeId="1" CreationDate="2020-08-03T09:28:44.553" Score="3" ViewCount="85" Body="&lt;p&gt;I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine whether the input audio (the voice of a person) contains the word &amp;quot;hello&amp;quot;  or not.&lt;/p&gt;&#xA;&lt;p&gt;I do not know what kind of network to use. I used the SOM network, but I did not get the desired result. My training data contains a large number of voices that contain the word &amp;quot;hello&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Is there any python code for dis problem?&lt;/p&gt;&#xA;" OwnerUserId="23216" LastEditorUserId="2444" LastEditDate="2020-08-04T12:15:24.353" LastActivityDate="2021-01-27T09:02:23.273" Title="How can I find a specific word in an audio file?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;python&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22845" PostTypeId="2" ParentId="21500" CreationDate="2020-08-03T10:00:13.830" Score="1" Body="&lt;p&gt;The route/trajectory followed by the optimization algorithm basically depends your dataset and the loss function. However, what really matters, for the purpose of final accuracy performance, is the final point which the trajectory converges to.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-08-03T10:00:13.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22846" PostTypeId="2" ParentId="16922" CreationDate="2020-08-03T10:48:41.473" Score="0" Body="&lt;p&gt;If I understood well you have 2 questions.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How to get the bounding box given the network output&lt;/li&gt;&#xA;&lt;li&gt;What Smooth L1 loss is&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The answer to your first question lies in the equation (2) in the section 3.2.1 from the &lt;a href=&quot;https://arxiv.org/pdf/1506.01497.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faster R-CNN paper&lt;/a&gt;. As all anchor based object detector (Faster RCNN, YOLOv3, EfficientNets, FPN...) the regression output from the network are not the bounding box coordinates. The regression output predicts the shift of the predicted bounding box with respect to the selected anchor (all of these networks, use more than 1 anchor per location, check section 3.1.1 from the paper).&lt;/p&gt;&#xA;&lt;p&gt;So basically what your network predict is &lt;span class=&quot;math-container&quot;&gt;$t_x, t_y, t_w, t_h$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hf2VM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hf2VM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the bounding box coordinates are &lt;span class=&quot;math-container&quot;&gt;$x, y, w, h$&lt;/span&gt;, and the anchor coordinates are &lt;span class=&quot;math-container&quot;&gt;$x_a, y_a, w_a, h_a$&lt;/span&gt;. So in order to compute &lt;span class=&quot;math-container&quot;&gt;$x, y, w, h$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$t_x, t_y, t_w, t_h$&lt;/span&gt;, you just have to invert the equations above. However I think you could gain more intuition about it if you take your time and read the whole section 3.1 from the paper. I know sometimes is a pain, but you will grasp the high level concept.&lt;/p&gt;&#xA;&lt;p&gt;With regard to your second question. Yes the loss is computed with the output from the network and the &amp;quot;coded&amp;quot; ground truth, meaning you compute the loss with the paramters &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (predicted) against &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt; (coded ground truth) instead for computing loss with the real coordinates of the bounding boxes (decoded output from the network). For the equation on Smooth L1 loss check &lt;a href=&quot;https://pytorch.org/docs/master/generated/torch.nn.SmoothL1Loss.html#torch.nn.SmoothL1Loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;this wonderful documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastEditorUserId="26882" LastEditDate="2020-08-03T10:53:47.730" LastActivityDate="2020-08-03T10:53:47.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22847" PostTypeId="1" CreationDate="2020-08-03T12:34:07.580" Score="0" ViewCount="15" Body="&lt;p&gt;From what I understand, (Faster/Mask-)RCNN is fully convolutional. The backbone is fully convolutional, and the region proposal network (RPN) creates anchors on the feature map with a fixed stride.&lt;/p&gt;&#xA;&lt;p&gt;This means that a forward pass on a crop of a larger image should give the same output (barring any edge issues) as a forward pass on the full image for that cropped area, since feature sizes are the same. Alternatively, padding an image with zeros shouldn't affect the output (again, barring edge issues).&lt;/p&gt;&#xA;&lt;p&gt;However, that doesn't seem to be the case when I try it with &lt;a href=&quot;https://github.com/pytorch/vision/blob/d2c763e14efe57e4bf3ebf916ec243ce8ce3315c/torchvision/models/detection/mask_rcnn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch's Mask-RCNN implementation&lt;/a&gt;. What am I missing here?&lt;/p&gt;&#xA;&lt;p&gt;I've already turned off the resizing that the PyTorch Mask-RCNN implementation does internally. Is there another resolution-dependent process at play here? Is this specific to this implementation, or am I misunderstanding something more general about how the RCNN architecture works?&lt;/p&gt;&#xA;" OwnerUserId="40060" LastActivityDate="2020-08-03T12:34:07.580" Title="Is RCNN resolution-independent, if keeping feature size constant?" Tags="&lt;convolutional-neural-networks&gt;&lt;r-cnn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22848" PostTypeId="1" CreationDate="2020-08-03T13:04:29.593" Score="1" ViewCount="40" Body="&lt;p&gt;I'm training a neural network on some input data. I know that loss increasing may be related to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;overfitting, if the loss increases on test data (while still decreases on training data)&lt;/li&gt;&#xA;&lt;li&gt;oscillations near the optimal point, if the learning rate is too big&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, I find that while for some input data the net makes good predictions, for other data the loss continues to increase, even if I only train on &lt;em&gt;one data point&lt;/em&gt; and if the learning rate is fairly low. To me, it's quite strange that performing the training on only one point the loss continues to increase and not decreases; in fact, the only reason I can find for this is a big learning rate.&lt;/p&gt;&#xA;&lt;p&gt;Can you think about some other reason?&lt;/p&gt;&#xA;" OwnerUserId="32915" LastEditorUserId="2444" LastEditDate="2020-08-06T11:18:49.733" LastActivityDate="2020-08-06T11:18:49.733" Title="Why would the loss increase on a single fixed input?" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22849" PostTypeId="2" ParentId="22832" CreationDate="2020-08-03T14:08:44.167" Score="0" Body="&lt;p&gt;You could use scikit-learn's &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MultiLabelBinarizer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MultiLabelBinarizer&lt;/a&gt;. It's essentially the multi-label equivalent of one-hot encoding. For each movie, create a vector of zeros, where each zero is associated with a particular actor. If an actor is in that movie, change their zero to a one. In the context of a neural network, think of it as each actor having their own input neuron, which will fire only if they are in that movie.&lt;/p&gt;&#xA;&lt;p&gt;The caveat is that to represent all actors, you'd need a rediculously long vector. In such cases, it's often sufficient to only look at the most common, say, 100 and ignore the rest. It intuitively makes sense that having a big actor who's done a lot of movies says more about the quality of a movie than whoever's playing Unimportant Bystander #3. This is actually how natural language processing represents words in the English language - take the top &lt;code&gt;n&lt;/code&gt; and ignore the rest.&lt;/p&gt;&#xA;" OwnerUserId="40060" LastActivityDate="2020-08-03T14:08:44.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22850" PostTypeId="1" CreationDate="2020-08-03T14:20:41.593" Score="0" ViewCount="24" Body="&lt;p&gt;I am building an AI with TD3 and lstm in both actor and critic.&#xA;By LSTM size is 5,5 with 3 layers and hidden layers with 400 and 300 neurons respectively.&#xA;I have states dimension of 5 with each value is a float tensor of values b/w -1 and 100.&#xA;During initial training it is trying to avoid negative losses. The actions also depend on the previous state..like in breakout...so LSTM is an important part&lt;/p&gt;&#xA;&lt;p&gt;Can someone please help to let know what should be the ideal training episodes after which I should judge whether the model is predicting correctly or not.&lt;/p&gt;&#xA;&lt;p&gt;PS.. I am new to AI an NN ..I have read this all depends on the reward function env etc... but still I would like to know how continuous states are trained.&lt;/p&gt;&#xA;" OwnerUserId="40051" LastActivityDate="2020-08-03T14:20:41.593" Title="Descent Training episodes for LSTM + TD3" Tags="&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22851" PostTypeId="1" AcceptedAnswerId="22854" CreationDate="2020-08-03T16:30:18.907" Score="4" ViewCount="315" Body="&lt;p&gt;Generally speaking, is there a best-practice procedure to follow when trying to define a reward function for a reinforcement-learning agent? What common pitfalls are there when defining the reward function, and how should you avoid them? What information from your problem should you take into consideration when going about it?&lt;/p&gt;&#xA;&lt;p&gt;Let us presume that our environment is fully observable MDP.&lt;/p&gt;&#xA;" OwnerUserId="25560" LastEditorUserId="2444" LastEditDate="2020-11-02T21:47:49.860" LastActivityDate="2021-01-20T12:55:33.830" Title="What are some best practices when trying to design a reward function?" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;reward-shaping&gt;&lt;inverse-rl&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22853" PostTypeId="1" AcceptedAnswerId="24680" CreationDate="2020-08-03T19:10:12.367" Score="1" ViewCount="59" Body="&lt;p&gt;I have an ensemble of 231 time series, the largest among them being 14 lines long. The task at hand is to try to predict these time-series. But I'm finding this difficult due to the very small size of the data. Any suggestions about what algorithm to use? I'm thinking about going for a hidden markov model, but I don't know if that's a wise choice.&lt;/p&gt;&#xA;" OwnerUserId="38261" LastActivityDate="2020-11-17T20:55:12.830" Title="How to deal with very, very small time-series?" Tags="&lt;python&gt;&lt;time-series&gt;&lt;hidden-markov-model&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22854" PostTypeId="2" ParentId="22851" CreationDate="2020-08-03T22:13:30.340" Score="4" Body="&lt;h3&gt;Designing reward functions&lt;/h3&gt;&#xA;&lt;p&gt;Designing a reward function is sometimes straightforward, if you have knowledge of the problem. For example, consider the game of chess. You know that you have three outcomes: win (good), loss (bad), or draw (neutral). So, you could reward the agent with &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; if it wins the game, &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; if it loses, and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if it draws (or for any other situation).&lt;/p&gt;&#xA;&lt;p&gt;However, in certain cases, the specification of the reward function can be a difficult task [&lt;a href=&quot;https://ai.stanford.edu/%7Eang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://es.mathworks.com/help/reinforcement-learning/ug/define-reward-signals.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=491&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;] because there are many (often unknown) factors that could affect the performance of the RL agent. For example, consider the driving task, i.e. you want to teach an agent to drive e.g. a car. In this scenario, there are so many factors that affect the behavior of a driver. How can we incorporate and combine these factors in a reward function? How do we deal with unknown factors?&lt;/p&gt;&#xA;&lt;p&gt;So, often, designing a reward function is a &lt;strong&gt;trial-and-error&lt;/strong&gt; and engineering process (so there is no magic formula that tells you how to design a reward function in all cases). More precisely, you define an initial reward function based on your knowledge of the problem, you observe how the agent performs, then tweak the reward function to achieve greater performance (for example, in terms of observable behavior, so &lt;strong&gt;not&lt;/strong&gt; in terms of the collected reward; otherwise, this would be an easy problem: you could just design a reward function that gives infinite reward to the agent in all situations!). For example, if you have trained an RL agent to play chess, maybe you observed that the agent took a lot of time to converge (i.e. find the best policy to play the game), so you could design a new reward function that &lt;a href=&quot;https://ai.stackexchange.com/q/24375/2444&quot;&gt;penalizes the agent for every non-win move (maybe it will hurry up!)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Of course, this trial-and-error approach is not ideal, and it can sometimes be impractical (because maybe it takes a lot of time to train the agent) and lead to misspecified reward signals.&lt;/p&gt;&#xA;&lt;h3&gt;Misspecification of rewards&lt;/h3&gt;&#xA;&lt;p&gt;It is well known that the misspecification of the reward function can have unintended and even dangerous consequences [&lt;a href=&quot;https://openai.com/blog/faulty-reward-functions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;5&lt;/a&gt;]. To overcome the misspecification of rewards or improve the reward functions, you have some options, such as&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Learning from demonstrations&lt;/strong&gt; (aka &lt;em&gt;apprenticeship learning&lt;/em&gt;), i.e. do not specify the reward function directly, but let the RL agent imitate another agent's behavior, either to&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;learn the policy directly (known as &lt;strong&gt;imitation learning&lt;/strong&gt; [&lt;a href=&quot;https://papers.nips.cc/paper/2016/file/cc7e2b878868cbae992d1fb743995d8f-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;8&lt;/a&gt;]), or&lt;/li&gt;&#xA;&lt;li&gt;learn a reward function first to later learn the policy (known as &lt;strong&gt;inverse reinforcement learning&lt;/strong&gt; [&lt;a href=&quot;https://ai.stanford.edu/%7Eang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] or sometimes known as &lt;strong&gt;reward learning&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Incorporate &lt;strong&gt;human feedback&lt;/strong&gt; [&lt;a href=&quot;https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/&quot; rel=&quot;nofollow noreferrer&quot;&gt;9&lt;/a&gt;] in the RL algorithms (in an interactive manner)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Transfer the information&lt;/strong&gt; in the policy learned in another but similar environment to your environment (i.e. use some kind of transfer learning for RL [&lt;a href=&quot;https://www.jmlr.org/papers/volume10/taylor09a/taylor09a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;10&lt;/a&gt;])&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Of course, these solutions or approaches can also have their shortcomings. For example, interactive human feedback can be tedious.&lt;/p&gt;&#xA;&lt;h3&gt;Reward shaping&lt;/h3&gt;&#xA;&lt;p&gt;Regarding the common pitfalls, although &lt;strong&gt;reward shaping&lt;/strong&gt; (i.e. augment the natural reward function with more rewards) is often suggested as a way to improve the convergence of RL algorithms, [&lt;a href=&quot;https://hal.archives-ouvertes.fr/file/index/docid/331752/filename/matignon2006ann.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;] states that reward shaping (and progress estimators) should be used cautiously.  If you want to perform reward shaping, you should probably be using &lt;a href=&quot;https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;potential-based reward shaping&lt;/strong&gt;&lt;/a&gt; (which is guaranteed not to change the optimal policy).&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;The MathWorks' article &lt;a href=&quot;https://es.mathworks.com/help/reinforcement-learning/ug/define-reward-signals.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Define Reward Signals&lt;/a&gt; discusses &lt;strong&gt;continuous&lt;/strong&gt; and &lt;strong&gt;discrete&lt;/strong&gt; reward functions (this is also discussed in [&lt;a href=&quot;https://hal.archives-ouvertes.fr/file/index/docid/331752/filename/matignon2006ann.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;]), and addresses some of their advantages and disadvantages.&lt;/p&gt;&#xA;&lt;p&gt;Last but not least, the 2nd edition of the RL bible contains a section (&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=491&quot; rel=&quot;nofollow noreferrer&quot;&gt;17.4 Designing Reward Signals&lt;/a&gt;) completely dedicated to this topic.&lt;/p&gt;&#xA;&lt;p&gt;Another similar question was also asked &lt;a href=&quot;https://ai.stackexchange.com/q/12264/2444&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T12:55:33.830" LastActivityDate="2021-01-20T12:55:33.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22855" PostTypeId="2" ParentId="22851" CreationDate="2020-08-03T22:14:40.630" Score="4" Body="&lt;p&gt;If your objective is for the agent to attain some goal (say, reaching a target), then a valid reward function is to assign a reward of 1 when the goal is attained and 0 otherwise. The problem with this reward function is that it's too &lt;em&gt;sparse&lt;/em&gt;, meaning the agent has little guidance on how to modify their behavior to become better at attaining said goal, especially if the goal is hard to attain through a random policy in the first place (which is probably roughly what the agent starts with).&lt;/p&gt;&#xA;&lt;p&gt;The practice of modifying the reward function to guide the learning agent is called &lt;em&gt;reward shaping&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A good start is &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.48.345&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Policy invariance under reward transformations: Theory and application to reward shaping&lt;/em&gt;&lt;/a&gt; by Ng et al. The idea is to create a &lt;em&gt;reward potential&lt;/em&gt; (see Theorem 1) on top of the existing reward. This reward potential should be an approximation of the &lt;em&gt;true value&lt;/em&gt; of a given state. For instance, if you have a gridworld scenario where the goal is for the agent to reach some target square, you could create a reward potential based on the Manhattan distance to this target (without accounting for obstacles), which is an &lt;em&gt;approximation&lt;/em&gt; to the true value of a given position.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, creating a reward potential that is close to the true values makes the job easier for the learning agent because it reduces the disadvantage of being myopic, and the agent more quickly gets closer to a &amp;quot;somewhat good&amp;quot; policy from which it is easier to crawl toward the optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, reward potentials have the property that they are &lt;em&gt;consistent&lt;/em&gt; with the optimal policy. That is, the optimal policy to the &lt;em&gt;true&lt;/em&gt; problem will not become suboptimal under the new, modified problem (with the new reward function).&lt;/p&gt;&#xA;" OwnerUserId="3373" LastActivityDate="2020-08-03T22:14:40.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22856" PostTypeId="1" CreationDate="2020-08-04T05:59:42.123" Score="1" ViewCount="26" Body="&lt;p&gt;I am trying to build an AI that needs to have some information about the past states as well. Therefore, LSTMs are suitable for this.&lt;/p&gt;&#xA;&lt;p&gt;Now, I want to know that for a problem/game like Breakout, where we require previous states as well, does A3C perform better than &lt;a href=&quot;https://www.mathworks.com/help/reinforcement-learning/ug/td3-agents.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TD3&lt;/a&gt;, given that TD3 does not have LSTM?&lt;/p&gt;&#xA;&lt;p&gt;Or without the LSTM TD3 should perform better than A3C despite the fact that A3C has LSTM in it.&lt;/p&gt;&#xA;" OwnerUserId="40051" LastEditorUserId="2444" LastEditDate="2020-08-04T11:44:09.783" LastActivityDate="2020-08-04T11:44:09.783" Title="When past states contain useful information, does A3C perform better than TD3, given that TD3 does not use an LSTM?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;long-short-term-memory&gt;&lt;actor-critic-methods&gt;&lt;a3c&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22857" PostTypeId="1" AcceptedAnswerId="22872" CreationDate="2020-08-04T07:10:33.170" Score="5" ViewCount="103" Body="&lt;p&gt;I would like to understand the difference between the &lt;a href=&quot;https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard policy gradient theorem&lt;/a&gt; and the &lt;a href=&quot;http://proceedings.mlr.press/v32/silver14-supp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;deterministic policy gradient theorem&lt;/a&gt;. These two theorem are quite different, although the &lt;em&gt;only&lt;/em&gt; difference is whether the policy function is deterministic or stochastic.  I summarized the relevant steps of the theorems below. The policy function is &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; which has parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Standard Policy Gradient&lt;/strong&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\dfrac{\partial V}{\partial \theta} &amp;amp;= \dfrac{\partial}{\partial \theta} \left[ \sum_a \pi(a|s) Q(a,s) \right]  \\&#xA;&amp;amp;= \sum_a \left[ \dfrac{\partial \pi(a|s)}{\partial \theta}  Q(a,s) + \pi(a|s) \dfrac{\partial Q(a,s)}{\partial \theta} \right] \\&#xA;&amp;amp;= \sum_a \left[ \dfrac{\partial \pi(a|s)}{\partial \theta}  Q(a,s) + \pi(a|s) \dfrac{\partial}{\partial \theta} \left[ R + \sum_{s'} \gamma p(s'|s,a) V(s') \right] \right]  \\&#xA;&amp;amp;= \sum_a \left[ \dfrac{\partial \pi(a|s)}{\partial \theta}  Q(a,s) + \pi(a|s) \gamma \sum_{s'} p(s'|s,a) \dfrac{\partial V(s') }{\partial \theta} \right]&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&#xA;When one now expands next period's value function &lt;span class=&quot;math-container&quot;&gt;$V(s')$&lt;/span&gt; again one can eventually reach the final policy gradient:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\dfrac{\partial J}{\partial \theta} = \sum_s \rho(s) \sum_a \dfrac{\pi(a|s)}{\partial \theta}  Q(s,a)&#xA;$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; being the stationary distribution. What I find particularly interesting is that there is no derivative of &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and also not of the probability distribution &lt;span class=&quot;math-container&quot;&gt;$p(s'|s,a)$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. The derivation of the deterministic policy gradient theorem is different:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Deterministic Policy Gradient Theorem&lt;/strong&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\dfrac{\partial V}{\partial \theta} &amp;amp;= \dfrac{\partial}{\partial \theta} Q(\pi(s),s)  \\&#xA;&amp;amp;= \dfrac{\partial}{\partial \theta} \left[ R(s, \pi(s)) + \gamma \sum_{s'} p(s'|a,s) V(s') \right] \\&#xA;&amp;amp;= \dfrac{R(s, a)}{\partial a}\dfrac{\pi(s)}{\partial \theta} + \dfrac{\partial}{\partial \theta} \left[\gamma \sum_{s'} p(s'|a,s) V(s')   \right]  \\&#xA;&amp;amp;= \dfrac{R(s, a)}{\partial a}\dfrac{\pi(s)}{\partial \theta} + \gamma \sum_{s'}  \left[p(s'|\mu(s),s) \dfrac{V(s')}{\partial \theta} + \dfrac{\pi(s)}{\partial \theta} \dfrac{p(s'|s,a)}{\partial a} V(s')   \right]  \\&#xA;&amp;amp;= \dfrac{\pi(s)}{\partial \theta} \dfrac{\partial}{\partial a} \left[ R(s, a) +  p(s'|s,a) V(s') \right] +  \gamma  p(s'|\pi(s),s) \dfrac{V(s')}{\partial \theta}  \\&#xA;&amp;amp;= \dfrac{\pi(s)}{\partial \theta} \dfrac{\partial Q(s, a)}{\partial a} +  \gamma  p(s'|\pi(s),s) \dfrac{V(s')}{\partial \theta}  \\&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&#xA;Again, one can obtain the finaly policy gradient by expanding next period's value function. The policy gradient is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\dfrac{\partial J}{\partial \theta} = \sum_s \rho(s) \dfrac{\pi(s)}{\partial \theta} \dfrac{\partial Q(s,a))}{\partial a}&#xA;$$&lt;/span&gt;&#xA;In contrast to the standard policy gradient, the equations contain derivatives of the reward function &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and the conditional probability &lt;span class=&quot;math-container&quot;&gt;$p(s'|s, a,)$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why do the two theorems differ in their treatment of the derivatives of &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and the conditional probability? Does determinism in the policy function make such a difference for the derivatives?&lt;/p&gt;&#xA;" OwnerUserId="34041" LastEditorUserId="34041" LastEditDate="2020-08-04T15:40:51.817" LastActivityDate="2020-08-06T10:01:15.673" Title="Comparing the derivation of the Deterministic Policy Gradient Theorem with the standard Policy Gradient Theorem" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22858" PostTypeId="1" CreationDate="2020-08-04T08:34:22.443" Score="2" ViewCount="472" Body="&lt;p&gt;I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image.&lt;/p&gt;&#xA;&lt;p&gt;Let's say that I have an image classification task, and I have very little labeled data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;How can I generate the target label from the other data in the dataset?&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="38093" LastEditorUserId="2444" LastEditDate="2020-11-20T12:56:48.310" LastActivityDate="2020-11-20T12:56:48.310" Title="How to generate labels for self-supervised training?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;supervised-learning&gt;&lt;self-supervised-learning&gt;&lt;feature-learning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22859" PostTypeId="1" CreationDate="2020-08-04T11:28:55.873" Score="0" ViewCount="64" Body="&lt;p&gt;What if I have some data, let's say I'm trying to answer if education level and IQ affect earnings, and I want to analyze this data and put in a regression model to predict earnings based on the IQ and education level. My confusion is, what if the data is not linear or polynomial? What if it's a mess but there are still patterns that the linear plane algorithm can't capture? How do I figure out if plotting all of the independent variables will form a line or a polynomial curve like here?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LZ5Bd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LZ5Bd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I mean, with one dependent and one independent variable it's easy because you can plot it and see, but in a situation with multiple independent variables... how do I figure out if the relationship is linear or something like this? How do I figure out if I should use a regression model?&lt;/p&gt;&#xA;&lt;p&gt;Let's say I want to predict a store's daily revenue based on the day of the week, weather and the number of people arrived in the city. My data would look something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+-----------+---------+----------------+---------+&#xA;| DAY       | WEATHER | PEOPLE ARRIVED | REVENUE |&#xA;+-----------+---------+----------------+---------+&#xA;| Monday    | Sunny   | 1115           | &lt;span class=&quot;math-container&quot;&gt;$500    |&#xA;+-----------+---------+----------------+---------+&#xA;| Tuesday   | Cloudy  | 808            | $&lt;/span&gt;250    |&#xA;+-----------+---------+----------------+---------+&#xA;| Wednesday | Sunny   | 450            | $300    |&#xA;+-----------+---------+----------------+---------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm a bit confused about what ML algorithm I should use in such a scenario. I can represent the days of the week as (Monday - 1, Tuesday - 2, Wednesday - 3, etc.) and the weather as (Sunny - 1, Cloudy - 2, Normal - 3, etc.) but would a regression model work? I'm skeptical because I'm not sure if there's a linear relationship between the variables and I'm not sure if a hyperplane can create accurate representation of what's going on.&lt;/p&gt;&#xA;" OwnerUserId="32539" LastEditorUserId="32539" LastEditDate="2020-08-04T12:44:25.640" LastActivityDate="2021-02-01T18:03:55.393" Title="What ML algorithm should I use that suits this data?" Tags="&lt;prediction&gt;&lt;regression&gt;&lt;linear-regression&gt;&lt;non-linear-regression&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22860" PostTypeId="2" ParentId="22858" CreationDate="2020-08-04T11:38:19.113" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;How can I generate the target label from the other data in the dataset?&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you are asking how you can create the learning signal in SSL, when given an unlabelled dataset, for learning representations of these unlabelled data, then there is &lt;strong&gt;no general answer&lt;/strong&gt;. The answer depends on the type of data that you have (which can be e.g. textual or visual), and which features do you think you want to learn or can be learned from your unlabelled data. &lt;a href=&quot;https://arxiv.org/pdf/1902.06162.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; and other answers to &lt;a href=&quot;https://ai.stackexchange.com/q/10623/2444&quot;&gt;this question&lt;/a&gt; provide some examples of how that can be done (depending on the type of data). Below, I also provide an example.&lt;/p&gt;&#xA;&lt;p&gt;Let me try to explain this more in detail.&lt;/p&gt;&#xA;&lt;p&gt;Let's assume that you have &lt;em&gt;both&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;an &lt;strong&gt;unlabelled&lt;/strong&gt; dataset &lt;span class=&quot;math-container&quot;&gt;$U = \{ u_i \}_{i=1}^m$&lt;/span&gt; and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a &lt;strong&gt;labelled&lt;/strong&gt; dataset &lt;span class=&quot;math-container&quot;&gt;$D = \{(x_i, y_i) \}_{i=1}^n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;where we may have &lt;span class=&quot;math-container&quot;&gt;$m \gg n$&lt;/span&gt; (although this is not a strict requirement), i.e. you may have a lot more unlabelled data than labelled data (this can easily be the case, given that, in general, manual data annotation is expensive/laborious). Let's say that your ultimate task is to perform object recognition (or classification). Let's call this task the &lt;strong&gt;downstream task&lt;/strong&gt;. So, you may think that &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt; are images and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; are labels, like &amp;quot;cat&amp;quot; or &amp;quot;dog&amp;quot; (let's say that you want to differentiate between cats and dogs).&lt;/p&gt;&#xA;&lt;p&gt;You want to solve this downstream task by supervised learning with &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. However, given that your labeled dataset is not big enough, you may think that training a neural network from scratch (i.e. by randomly initializing its weights) with &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; may not lead to good performance. So, you think that it could be useful to start training from a pre-trained model that already contains useful representations of data similar to your labeled data, i.e. to perform &lt;a href=&quot;https://www.tensorflow.org/tutorials/images/transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;transfer learning&lt;/strong&gt;&lt;/a&gt;. To pre-train such a model, you could use SSL.&lt;/p&gt;&#xA;&lt;p&gt;So, to solve your downstream task with SSL, there are &lt;strong&gt;2 different steps&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Self-supervised learning (SSL)&lt;/strong&gt;: learn representations of your images &lt;span class=&quot;math-container&quot;&gt;$u_i \in U$&lt;/span&gt; by training a neural network &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; to solve a so-called &lt;strong&gt;pretext&lt;/strong&gt; (or &lt;strong&gt;auxiliary&lt;/strong&gt; task); there are many pre-text tasks: you can find many examples &lt;a href=&quot;https://arxiv.org/pdf/1902.06162.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/a/20393/2444&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/10623/2444&quot;&gt;here&lt;/a&gt; (see example below too);&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Supervised learning (SL)&lt;/strong&gt; by &lt;strong&gt;transfer learning&lt;/strong&gt;: fine-tune &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; (the labeled dataset), in a supervised way; this task is known as &lt;strong&gt;downstream task&lt;/strong&gt; (as stated above)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In this process, there are &lt;strong&gt;2 different labels&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In step 1, you have the labels that are generated automatically. But how are these labels generated? As I said, there are many ways. Let me describe one way (among many others!). Let's say that your &lt;strong&gt;unlabelled&lt;/strong&gt; dataset &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; contains high-resolution images (i.e. &lt;span class=&quot;math-container&quot;&gt;$u_i \in U$&lt;/span&gt; are high-resolution images), then you could define your pre-text task as follows. You lower the resolution of your high-resolution images to create other images. Let &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; be the low-resolution image created from the high-resolution image &lt;span class=&quot;math-container&quot;&gt;$u_i \in U$&lt;/span&gt;, then the training pair to your neural network &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$(v_i, u_i) \in U'$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt; is the label (which is the original high-resolution image) and &lt;span class=&quot;math-container&quot;&gt;$U'$&lt;/span&gt; the labeled dataset automatically generated (i.e. with the algorithm I've just explained).&lt;/p&gt;&#xA;&lt;p&gt;So, these labels &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt; (high-resolution images) are semantically different than &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; (&amp;quot;cat&amp;quot; or &amp;quot;dog&amp;quot;) in the pairs &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i) \in D$&lt;/span&gt;. They are different because, here, we want to learn representations and not to perform object recognition/classification: the idea is that, by solving this pre-text task, your final trained neural network, should have learned features of the images in the unlabelled data (i.e. representation learning). These learned features can then be used to bootstrap training in the downstream task.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In step 2, you use the labeled dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, which has been typically annotated (or labeled) by a human. As stated above, this dataset contains pairs &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is, for example, the label &amp;quot;cat&amp;quot; or &amp;quot;dog&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In this step, the pre-trained model &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, with the SSL technique,  can be fine-tuned with &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; in a supervised fashion. Given that we start with a pre-trained model &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, we are effectively performing transfer learning.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that SSL can also refer to something (slightly) different than what has been explained in this answer. See &lt;a href=&quot;https://ai.stackexchange.com/a/10624/2444&quot;&gt;my other answer&lt;/a&gt; for more details. Moreover, note that you can perform representation learning with SSL without necessarily solving a downstream task later, which may also not be an SL task (in the example above, I've described a downstream task that is an SL task only for simplicity).&lt;/p&gt;&#xA;&lt;p&gt;If this answer is still unclear, maybe you should have a look at existing implementations of SSL techniques (such as &lt;a href=&quot;https://github.com/google/revisiting-self-supervised&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;) for more inspiration.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T12:47:33.780" LastActivityDate="2020-11-20T12:47:33.780" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22861" PostTypeId="1" CreationDate="2020-08-04T15:44:08.490" Score="1" ViewCount="25" Body="&lt;p&gt;I am a programmer, but just now attempting to enter the world of ML. I'm eyeballing a potential project/problem related to foosball.&lt;/p&gt;&#xA;&lt;p&gt;Pro foosball is a thing believe it or not and I'm wondering if I can use decades worth of game footage to determine where defensive holes are most likely to be.&lt;/p&gt;&#xA;&lt;p&gt;The way shooting works in pro foosball is you front pin the ball and walk it back and forth in front of the goal. The defense meanwhile is attempting to randomly move two men in front of the goal. Of course, our human brains are not truly random and this I'd like ML to help me understand and exploit.&lt;/p&gt;&#xA;&lt;p&gt;Questions like, if I walk the ball left, then step right, historically where is the open hole likely to be?&lt;/p&gt;&#xA;&lt;p&gt;If you'd like to better understand the nature of shooting, here is video of pro foosball: &lt;a href=&quot;https://www.youtube.com/watch?v=uOdnqmwOQhA&amp;amp;t=16s&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=uOdnqmwOQhA&amp;amp;t=16s&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So what ML topics should I research and what strategies and tools do you recommend for creating such a model?&lt;/p&gt;&#xA;" OwnerUserId="40088" LastActivityDate="2020-08-04T15:44:08.490" Title="Can I use ML to discover via videos the best place to shoot in foosball?" Tags="&lt;machine-learning&gt;&lt;video-classification&gt;&lt;games-of-chance&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22862" PostTypeId="2" ParentId="20778" CreationDate="2020-08-04T15:56:29.663" Score="0" Body="&lt;p&gt;Your list is complete for what is considered 'popular' by most practitioners who apply AI for stock trading.  Supervised learning and rule learning are at the top for accuracy.  There are more academic papers published on classifiers than on regression approaches;&#xA;classifiers are typically more accurate than regressors.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-08-04T15:56:29.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22863" PostTypeId="1" CreationDate="2020-08-04T17:00:46.140" Score="1" ViewCount="29" Body="&lt;p&gt;I am trying to get a better grasp of how object detection works. I (almost) completely understand the concept behind RPNs. However, I am a little bit confused with the selective search algorithm part. This algorithm does not really learn anything, as far as I understand.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, when I have an image containing people (even though my network does not need to classify these), will the selective search still propose these people to my network?&lt;/p&gt;&#xA;&lt;p&gt;Of course, my CNN  has not learned to classify a human and will output a very low probability for every class (it did learn), and thus, this way, the human will not contain a bounding box.&lt;/p&gt;&#xA;&lt;p&gt;Also, in further iterations of the R-CNN model, they proposed using regressors to improve the bounding box.&lt;/p&gt;&#xA;&lt;p&gt;Does this mean that this part of the model got the CNNs feature maps, and, based on this, learned to output a bounding box (this way smaller instances of a detected object would get a smaller bounding box)?&lt;/p&gt;&#xA;&lt;p&gt;So, in this first iteration, they probably did not need bounding boxes in the training data (since there was no way to learn the size of the bounding boxes and thus no need to find a loss function for this problem)?&lt;/p&gt;&#xA;&lt;p&gt;Lastly, I understand that the selective search algorithm is an improvement on the sliding window algorithm. It tries to have a high recall, so having false positives is not bad, as long as we have all the true positives. Again, I do not seem to understand HOW this algorithm knows when it has the object it needs without really learning. Any intuïtive explanation or visual (I am a visual learner at first) on how this algorithm works is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="34359" LastEditorUserId="2444" LastEditDate="2021-01-19T17:25:56.377" LastActivityDate="2021-01-19T17:25:56.377" Title="Does the selective search algorithm in object detection learn?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;r-cnn&gt;&lt;selective-search&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22865" PostTypeId="2" ParentId="20971" CreationDate="2020-08-04T20:39:55.780" Score="0" Body="&lt;h2&gt;1. What is typically meant by 3D-face recognition? We are usually extracting the face encoding from 2D-images, right?&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Yes. The goal is to reconstruct the three-dimensional shape, as well as the texture of a face from a single or multiple images of that person.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In recent years, &amp;quot;the performance of 2D face recognition algorithms has significantly increased with the use of &lt;strong&gt;deep neural networks&lt;/strong&gt; and the use of large-scale labeled training data.&amp;quot; &lt;a href=&quot;https://arxiv.org/pdf/1703.10714.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep 3D Face Ifentification&lt;/a&gt; with the latter still being a notable problem because there are so few of them and they are usually needed to better measure and hence train the deep networks. &lt;em&gt;(The referenced paper gives you a nice overview.)&lt;/em&gt; Even with enough data, occlusions, lens distortions or possible variations of expressions, etc. make the problem tough. So many other tricks and augmentation techniques are needed.&lt;/p&gt;&#xA;&lt;h2&gt;2. How can 3D face recognition used for liveness detection?&lt;/h2&gt;&#xA;&lt;p&gt;3D face recognition is used to differentiate 2D spoofs from actual present people. Using multiple time-delayed images one could possibly track if the face moves like a living person as well, or if it is static or stretching weird like a mask, puppet, or moving display.&lt;/p&gt;&#xA;&lt;p&gt;Nonetheless, often even advanced 2D image-based techniques fall flat in real-world applications because of the used cameras. They might not have enough resolution to recognize if the input is a video of a face on a high-resolution display as explained &lt;a href=&quot;https://www.liveness.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40095" LastActivityDate="2020-08-04T20:39:55.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22867" PostTypeId="1" CreationDate="2020-08-04T22:31:23.197" Score="0" ViewCount="17" Body="&lt;p&gt;I have written this post in medium:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@soheil.dating/opensource-online-group-dancing-with-motion-capture-and-95b59a5e93f8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Opensource Online group Dancing with Motion capture and …&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I have was working on motion analysis and seen &lt;a href=&quot;https://github.com/derikon/awesome-human-motion&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GitHub page&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=AGm3hF_BlYM&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; (&lt;a href=&quot;https://compvis.github.io/hbugen2018/images/rerender.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;related paper could be seen here&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;But because i'm newbie in this area and don't tried any module codes in my PC to test creating online Avatar and ..., i asked here to know for having one live avatar only by few picture of on person how mush is hard to create 10 live human dancer's avatar which are synced and combined  in 3D environment.&lt;/p&gt;&#xA;&lt;p&gt;Also asked this question in &lt;a href=&quot;https://www.youtube.com/watch?v=AGm3hF_BlYM&quot; rel=&quot;nofollow noreferrer&quot;&gt;that video page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How is hard to make live avatar  for 10 person dancing online, and&#xA;create some live environment like SIMS game by these avatars ( CPU and&#xA;Memory which is needed?) and the main module to do it , it it written&#xA;by &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenAI&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;i thinks it must be fast enough as human eye scan time ( 25 frame in 1 second) which i dont know it is possible now to do it? also if possible could you say, how much CPU and memory needs to do this simulation online ?)&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="33936" LastActivityDate="2020-08-04T22:31:23.197" Title="About synced and combined, 10 live human dancer's avatar in 3D environment" Tags="&lt;image-processing&gt;&lt;pose-estimation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22869" PostTypeId="1" CreationDate="2020-08-05T02:17:18.710" Score="0" ViewCount="40" Body="&lt;p&gt;Let's say that I want to classify whether a document is a legal document or not. I have a list of keywords that will be presented only in legal documents.&lt;/p&gt;&#xA;&lt;p&gt;What is the proper way or algorithm to calculate probability based on this list?&lt;/p&gt;&#xA;" OwnerUserId="40100" LastEditorUserId="40100" LastEditDate="2020-08-05T02:26:16.900" LastActivityDate="2021-05-04T17:05:41.570" Title="How do I classify whether a document is legal or not given a set of keywords that appear only in legal documents?" Tags="&lt;machine-learning&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22871" PostTypeId="1" CreationDate="2020-08-05T05:12:23.190" Score="0" ViewCount="40" Body="&lt;p&gt;To best of my knowledge, images are usually fed in pixel format to ML models.&#xA;Is there any work that does image classification where the image format is SVG?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastActivityDate="2020-08-05T05:12:23.190" Title="Image classification on SVG format" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22872" PostTypeId="2" ParentId="22857" CreationDate="2020-08-05T10:23:24.770" Score="4" Body="&lt;p&gt;In the policy gradient theorem, we don't need to write &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; as a function of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; because the only time we explicitly 'see' &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is when we are taking the expectation with respect to the policy. For the first couple lines of the PG theorem we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla v_\pi(s) &amp;amp;= \nabla \left[ \sum_a \pi(a|s) q_\pi (s,a) \right] \;, \\ &amp;amp;= \sum_a \left[ \nabla \pi(a|s) q_\pi(s,a) + \pi(a|s) \nabla\sum_{s',r} p(s',r|s,a)(r+ v_\pi(s')) \right] \; ;&#xA;\end{align}&lt;/span&gt;&#xA;you can see that we are taking expectation of &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; with respect to the policy, so we don't need to write something like &lt;span class=&quot;math-container&quot;&gt;$r(s,\pi(a|s))$&lt;/span&gt; (especially because this notation doesn't really make sense for a stochastic policy). This is why we don't need to take the derivative of &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; with respect to the policy parameters. Now, the next line of the PG theorem is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla v_\pi(s) = \sum_a \left[ \nabla \pi(a|s) q_\pi(s,a) + \pi(a|s)\sum_{s'} p(s'|s,a) \nabla v_\pi(s') \right] \; ;$$&lt;/span&gt;&#xA;so now we have an equation similar to the bellman equation in terms of the &lt;span class=&quot;math-container&quot;&gt;$\nabla v_\pi(s)$&lt;/span&gt;'s, so we can unroll this repeatedly meaning we never have to take an explicit derivative of the value function.&lt;/p&gt;&#xA;&lt;p&gt;For the deterministic gradient, this is a bit different. In general we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \mathbb{E}_\pi[Q(s,a)] = \sum_a \pi(a|s) Q(s,a)\;,$$&lt;/span&gt;&#xA;so for a deterministic policy (denoted by &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; which represents the action taken in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;) this becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = Q(s,\pi(s))$$&lt;/span&gt;&#xA;because the deterministic policy has 0 probability for all actions except one, where it has probability one.&lt;/p&gt;&#xA;&lt;p&gt;Now, in the deterministic policy gradient theorem we can write&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla v_\pi(s) = \nabla Q(s,\pi(s)) = \nabla \left(r(s, \pi(s)) + \sum_{s'} p(s'|s,a)v(s') \right)\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We have to write &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; explicitly as a function of &lt;span class=&quot;math-container&quot;&gt;$s,a$&lt;/span&gt; now because we are not taking an expectation with respect to the actions because we have a deterministic policy. Now, if you replace where I have written &lt;span class=&quot;math-container&quot;&gt;$\nabla$&lt;/span&gt; with the notation you have used for the derivatives you will arrive at the same result and you'll see why you need to use the chain rule, which I believe you understand because your question was more why &lt;em&gt;don't&lt;/em&gt; we use the chain rule for the normal policy gradient, which I have hopefully explained -- it is essentially because of how an expectation over the action space works with a deterministic policy vs. a stochastic policy.&lt;/p&gt;&#xA;&lt;p&gt;Another way to think of this is as follows -- the term you're concerned with is obtained by expanding &lt;span class=&quot;math-container&quot;&gt;$\nabla q_\pi(s,a) = \nabla \sum_{s', r}p(s',r|s,a)(r(s,a) + v_\pi(s'))$&lt;/span&gt;. Because, by definition of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function, we have conditioned on knowing &lt;span class=&quot;math-container&quot;&gt;$a,s$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is completely independent of the policy in this scenario - we could even condition on an action that the policy would have 0 probability for - thus the derivative of &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; with respect to the policy parameters is 0.&lt;/p&gt;&#xA;&lt;p&gt;However, in the deterministic policy gradient we are taking &lt;span class=&quot;math-container&quot;&gt;$\nabla q_\pi(s, \pi(s)) = \nabla \left(r(s, \pi(s)) + \sum_{s'} p(s'|s,a) v_\pi(s')\right)$&lt;/span&gt; -- here &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; clearly depends on the policy parameters because the action taken was the deterministic action given by the policy in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, thus the derivative wrt the policy parameters is not necessarily 0!&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-08-06T10:01:15.673" LastActivityDate="2020-08-06T10:01:15.673" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="22873" PostTypeId="1" CreationDate="2020-08-05T10:32:18.637" Score="0" ViewCount="45" Body="&lt;p&gt;I've been playing with the SV2TTS voice cloning implementation provided on Github: &lt;a href=&quot;https://github.com/CorentinJ/Real-Time-Voice-Cloning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/CorentinJ/Real-Time-Voice-Cloning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It was straight forward to use as a TTS engine with clearly recognizable voice characteristics of my target speaker voice. Unfortunately, it has no vocal emotions and inflections.&lt;/p&gt;&#xA;&lt;p&gt;So, I've been wondering, if it's possible to use my own voice recording of the phrases and then apply the vocal timbre of the target speaker? Essentially, I don't even need the TTS synthesis part, I just want to &amp;quot;transcode&amp;quot; my voice timbre to the target speaker's timbre.&lt;/p&gt;&#xA;" OwnerUserId="40105" LastActivityDate="2020-08-05T10:32:18.637" Title="How to add voice inflections to an existing SV2TTS voice cloning implementation?" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22875" PostTypeId="2" ParentId="22859" CreationDate="2020-08-05T15:42:07.440" Score="0" Body="&lt;p&gt;Regression Model will definitely work on That problem.&lt;/p&gt;&#xA;&lt;p&gt;You only need to change shape of predicting variables like(Day, Weather, people arrived) into 1D array if you got error..&#xA;Otherwise you can simply apply Linear Regression, SVM etc to get your output with good accuracy.&lt;/p&gt;&#xA;" OwnerUserId="40109" LastActivityDate="2020-08-05T15:42:07.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22876" PostTypeId="2" ParentId="22810" CreationDate="2020-08-05T16:46:44.050" Score="3" Body="&lt;p&gt;I'll answer in a couple of stages.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I feel somewhat lost as to what the input for the NN should look like.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your choices boil down to two options, each with their own multitude of variants:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Vector Representation: Your input is a vector of the same size as your vocabulary where the elements represent the tokens in the input example. The most basic version of this is a bag-of-words (BOW) encoding with a 1 for each word that occurs in the input example and a 0 otherwise. Some other variants are (normalized) word counts or &lt;a href=&quot;http://tfidf.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF-IDF&lt;/a&gt; values. With this representation padding will not be necessary as each example will be encoded as a vector of the same size as the vocabulary. However, it suffers from a variety of issues: the input is high-dimensional and very sparse making learning difficult (as you note), it does not encode word order, and the individual word representations have little (TF-IDF) to no (BOW, counts) semantic information. It also limits your NN architecture to a feed-forward network, as more &amp;quot;interesting&amp;quot; architectures such a RNNs, CNNs, and transformers assume a matrix-like input, described below.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Matrix Representation: Here your input representation is a matrix with each row being a vector (i.e. embedding) representation of the token at that index in the input example. How you actually get the pretrained embeddings into the model depends on a number of implementation-specific factors, but &lt;a href=&quot;https://stackoverflow.com/questions/49710537/pytorch-gensim-how-to-load-pre-trained-word-embeddings&quot;&gt;this stackoverflow question&lt;/a&gt; shows how to load embeddings from gensim into PyTorch. Here padding is necessary because the input examples will have variable numbers of tokens. &lt;a href=&quot;https://stackoverflow.com/a/52242625/2544396&quot;&gt;This stackoverflow answer&lt;/a&gt; shows how to add zero padding in PyTorch.&#xA;This representation will be significantly better than the vector representation as it is relatively low-dimensional and non-sparse, it maintains word order, and using pretrained word-embeddings means your model will have access to semantic information. In fact, this last point leads to your next question.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;em&gt;Learning word embeddings creates vectors for words that are similar to each other syntax-wise, and I fail to see how that can be used to derive the weight/impact of each word on the target variable in my case.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Word embeddings are based on the assumptions of &lt;a href=&quot;https://en.wikipedia.org/wiki/Distributional_semantics&quot; rel=&quot;nofollow noreferrer&quot;&gt;distributional semantics&lt;/a&gt;, the core tenet of which is often quoted as &amp;quot;a word is characterized by the company it keeps&amp;quot;. That is, the meaning of a word &lt;em&gt;is&lt;/em&gt; how it relates to other words. In the context of NLP, models can make better decisions because similar words are treated similarly from the get-go.&lt;/p&gt;&#xA;&lt;p&gt;For example, say that articles about furry pets get a lot of likes (entirely plausible if you ask me). However, the mentions of furry pets in these articles will be varied, including words like &amp;quot;dog&amp;quot;, &amp;quot;cat&amp;quot;, &amp;quot;chinchilla&amp;quot;, &amp;quot;poodle&amp;quot;, &amp;quot;doggo&amp;quot;, &amp;quot;good boy&amp;quot;, etc. An input representation that treats these mentions as completely distinct (such as BOW) will need to learn individual correlations between each word and the number of likes (that's a lot of learning). A well-trained word embedding, on the other hand, will be able to immediately group these mentions together and learn general correlations between groups of similar words and likes. Fair warning, this is a very imprecise description of why word embeddings work, but I hope it gives you some intuitive understanding.&lt;/p&gt;&#xA;&lt;p&gt;Finally, since you're doing regression, make sure you choose your objective function accordingly. Mean squared error would be my first try.&lt;/p&gt;&#xA;" OwnerUserId="37972" LastActivityDate="2020-08-05T16:46:44.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22877" PostTypeId="1" AcceptedAnswerId="22881" CreationDate="2020-08-05T17:35:44.830" Score="2" ViewCount="1079" Body="&lt;p&gt;I know it cost around $4.3 million dollars to train, but how much computing power does it cost to run the finished program? IBM Watson chatbot AI only costs a few cents per chat message to use, OpeenAI Five seemed to run on a single gaming PC setup. So I'm wondering how much computing power does it need to run the finished ai program.&lt;/p&gt;&#xA;" OwnerUserId="40112" LastActivityDate="2020-08-09T16:30:45.497" Title="How much computing power does it cost to run GPT-3?" Tags="&lt;gpu&gt;&lt;gpt&gt;" AnswerCount="2" CommentCount="2" ClosedDate="2020-08-10T16:56:39.070" ContentLicense="CC BY-SA 4.0" />
  <row Id="22878" PostTypeId="1" CreationDate="2020-08-05T19:37:00.497" Score="1" ViewCount="58" Body="&lt;p&gt;I am currently working on a public project for the National Weather Model. We are experimenting with using a recurrent neural network to replace the output of a quadratic formula that is in use. The aim of the experiment is to get a speedup in the computation by using a neural network to essentially mimic the output of the quadratic formula. We have achieved an accuracy of about +-.02 but would like to see that improve to +-.001 or so in order to make the outputs indiscernible from a usage standpoint. Despite changing or increasing the training data size, validation data size, number of layers, size of layers, optimizer, batch size, epoch number, normalizations, etc. we cannot seem to move past this level of accuracy. We have changed and tested every standard metric we can find on how to improve the model, but nothing improves the accuracy beyond that threshold.&lt;/p&gt;&#xA;&lt;p&gt;The main question we have is whether or not Keras is rounding at some point between each layer or has some limiting factor on the backend limiting the model's significant figures in the output. The training data resolution should allow for a finer level of accuracy, but as stated before, any changes made the model cannot improve past what has been achieved. Any insight on what is holding the model back would be greatly appreciated and could help with applying this method elsewhere. The Github has a readme file explaining what is occurring in each file and how to run the model as this is still a work in progress. I would be happy to dive deeper into any aspect of the model as well.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/NOAA-OWP/t-route/tree/testing/src/lookup_routing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NOAA-OWP/t-route/tree/testing/src/lookup_routing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40114" LastEditorUserId="40114" LastEditDate="2020-08-07T13:57:01.907" LastActivityDate="2020-08-07T13:57:01.907" Title="Keras model accuracy not improving beyond threshold" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;recurrent-neural-networks&gt;&lt;accuracy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22879" PostTypeId="1" CreationDate="2020-08-05T19:39:07.447" Score="0" ViewCount="45" Body="&lt;p&gt;I have found that the K-means algorithm with K=20, has been mentioned as a solution for the below question,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A new mobile phone service chain store would like to open 20 service&#xA;centers in your city. Each service center should cover at least one&#xA;shopping center and 5000 households of annual income over $75000.&#xA;Design a scalable algorithm that decides locations of service centere&#xA;by taking all the aforementioned constraints into consideration.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, I think Local Search algorithm, similar to solving N-Queen problem by local searches, can also be an alternate solution. In fact, below is the algorithm that I have in mind,&lt;/p&gt;&#xA;&lt;p&gt;My Algorithm:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;//Define constraints &#xA;//  CT1 as - should cover at at least one shopping center, and &#xA;//  and CT2 as - should cover 5000 households with min income 75K/anum&#xA;&#xA;//Initialize locations of the service centers C_i for i=1,..,20&#xA;for i=1 to 20&#xA;    C_i := randomly select location from map&#xA;end-for&#xA;//Iterate until cut-off T&#xA;for t=1 to T //T may be initialized with 1000&#xA;    C_r := randomly select any of {C_1,..,C_20} such that they are currently not satisfying the constraints CT_1 and CT_2&#xA;    if C_r is NULL$&#xA;         return success //current assignments of C_1 to C_20 are valid&#xA;    else&#xA;         Assign C_r a new location that has lowest conflict w.r.t. CT_1 and CT_2&#xA;    end-if&#xA;end-for&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Further, in support of my suggestion, I have found quite a few papers that are suggesting that the local search is a better approximation. Few of such papers are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.umd.edu/%7Emount/Projects/KMeans/kmlocal-cgta.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Local Search Approximation Algorithm for k-Means Clustering, Tapas&#xA;Kanungo et al&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v97/lattanzi19a/lattanzi19a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Better k-means++ Algorithm via Local Search,&#xA;Silvio Lattanzi &amp;amp; Christian Sohler&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Would you share your thoughts on the feasibility of the local search algorithm provided above, as a potential solution.&lt;/p&gt;&#xA;" OwnerUserId="31550" LastActivityDate="2020-08-05T19:39:07.447" Title="Local Search vs K-means Clustering" Tags="&lt;k-means&gt;&lt;local-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22880" PostTypeId="1" AcceptedAnswerId="22885" CreationDate="2020-08-05T21:09:22.017" Score="0" ViewCount="62" Body="&lt;p&gt;I am quite new to neural networks. I am trying to implement in Python a neural network having only one hidden layer with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; neurons and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; output layer.&lt;/p&gt;&#xA;&lt;p&gt;The point is that I am analyzing time series and would like to use the output layer as the input of the next unit: by feeding the network with the input at time &lt;span class=&quot;math-container&quot;&gt;$t-1$&lt;/span&gt; I obtain the output &lt;span class=&quot;math-container&quot;&gt;$O_{t-1}$&lt;/span&gt; and, in the next step, I would like to use both the input at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$O_{t-1}$&lt;/span&gt;, introducing a sort of auto-regression. I read that &lt;em&gt;recurrent neural network&lt;/em&gt; are suitable to address this issue.&lt;/p&gt;&#xA;&lt;p&gt;Anyway I cannot imagine how to implement a network in Keras that involves &lt;em&gt;multilayer recurrence&lt;/em&gt;: all the references I found are linked to using the output of a layer as input of the same layer in the next step. Instead, I would like to include the output of the last layer (the output layer) in the inputs of the first hidden layer.&lt;/p&gt;&#xA;" OwnerUserId="40115" LastEditorUserId="40115" LastEditDate="2020-08-05T22:30:26.803" LastActivityDate="2020-08-05T22:30:26.803" Title="Is there a neural network that accepts both the current input and previous output?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22881" PostTypeId="2" ParentId="22877" CreationDate="2020-08-05T21:35:00.387" Score="2" Body="&lt;p&gt;I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB of memory). The biggest gpu has 48 GB of vram &lt;br/&gt; I've read that gtp-3 will come in eigth sizes, 125M to 175B parameters. So depending upon which one you run you'll need more or less computing power and memory.&lt;br/&gt; (&lt;a href=&quot;https://lambdalabs.com/blog/demystifying-gpt-3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lambdalabs.com/blog/demystifying-gpt-3/&lt;/a&gt;) &lt;br/&gt;&#xA;For an idea of the size of the smallest, &amp;quot;The smallest GPT-3 model is roughly the size of BERT-Base and RoBERTa-Base.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="30100" LastEditorUserId="30100" LastEditDate="2020-08-08T14:26:39.677" LastActivityDate="2020-08-08T14:26:39.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22883" PostTypeId="2" ParentId="22880" CreationDate="2020-08-05T21:44:33.040" Score="1" Body="&lt;p&gt;You want to look at recurrent neural networks.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-05T21:44:33.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22885" PostTypeId="2" ParentId="22880" CreationDate="2020-08-05T21:55:13.400" Score="0" Body="&lt;p&gt;You could just do this; concatenate your input_vector with zero's vector that has the size of your output. Then in the first pass you concatenate with the output instaid of the zero's vector. After that repeat.. At the end just compare (compute the loss) your entire output from t0 to t1 to your target and backprop.&lt;/p&gt;&#xA;&lt;p&gt;You might want to look into recurrent layers, these are layers that have connections back to themselves so that the network can learn what to &amp;quot;remember&amp;quot;.&#xA;These have some problems with longer sequences, so the &amp;quot;newer&amp;quot; versions try to deal with that. (LSTM and GRU)&#xA;You can also use attention mechanisms if you're dealing with sequences. (basically you learn what parts of your input sequence to look at given a certain &amp;quot;query&amp;quot;, in your case maybe the last timestep) (generally used in natural language processing) But it's a bit more exotic and complicated.&lt;/p&gt;&#xA;" OwnerUserId="30100" LastActivityDate="2020-08-05T21:55:13.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22886" PostTypeId="2" ParentId="20783" CreationDate="2020-08-06T04:54:18.243" Score="0" Body="&lt;p&gt;&lt;strong&gt;1. Does BERT have any such models?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you mean the &lt;strong&gt;pretrained model&lt;/strong&gt;, then the answer is YES.&#xA;BERT has Cased and Uncased models and also models for other language (&lt;code&gt;BERT-Base, Multilingual Cased&lt;/code&gt; and &lt;code&gt;BERT-Base, Multilingual Uncased&lt;/code&gt;, trained in 104 languages). You can check those models &lt;a href=&quot;https://github.com/google-research/bert#pre-trained-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2. Is it possible to check similarity between two words using BERT?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Refer to &lt;a href=&quot;https://colab.research.google.com/drive/1ZQvuAVwA3IjybezQOXnrXMGAnMyZRuPU#scrollTo=BokW7CAgMxCB&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Google Colab Notebook&lt;/a&gt;, it is &lt;strong&gt;not appopriate&lt;/strong&gt; with BERT Embeddings for &lt;strong&gt;Word-Level similarity comparisons&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, doing &lt;strong&gt;sentence embedding similarity comparisons&lt;/strong&gt; are still valid with BERT.&lt;/p&gt;&#xA;&lt;p&gt;That's it, hope it helps you :)&lt;/p&gt;&#xA;" OwnerUserId="40119" LastActivityDate="2020-08-06T04:54:18.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22887" PostTypeId="1" CreationDate="2020-08-06T05:48:04.593" Score="0" ViewCount="24" Body="&lt;p&gt;i'm a student and i'm new to NLP.&lt;/p&gt;&#xA;&lt;p&gt;I want to build an Automated Scoring system which is in Indonesian Language using BERT. The system is expected to be able to measure the similarity of an answer(e.g: student answer) with a reference answer and give the value of similarity.&lt;/p&gt;&#xA;&lt;p&gt;But i'm still not clear about how the right &lt;strong&gt;step-to-step process&lt;/strong&gt; to build the system using BERT.&#xA;Since my system is going to &lt;strong&gt;process texts that are in Indonesian&lt;/strong&gt; and i'm going to build and test the system on &lt;strong&gt;Google Colab&lt;/strong&gt;, my main questions are :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For the preprocessing, is it necessary for using a library such as NLTK or is it enough just to use BERT's default Tokenizer?&lt;/li&gt;&#xA;&lt;li&gt;Should i build my own vocabulary and my own sentence vectors? (For later, i will use &lt;a href=&quot;https://colab.research.google.com/drive/1ZQvuAVwA3IjybezQOXnrXMGAnMyZRuPU#scrollTo=mQaco6jRLkXn&quot; rel=&quot;nofollow noreferrer&quot;&gt;this google colab tutorial&lt;/a&gt; to build my own sentence vectors if it's really necessary to build them on my own)&lt;/li&gt;&#xA;&lt;li&gt;Will my system work properly if i only use the &lt;code&gt;BERT-Base Multilingual Uncased&lt;/code&gt; as the language model?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;And maybe there's any complete article/tutorial about this?&#xA;I appreciate any helpful answers.&#xA;Thank you so much in advance.&lt;/p&gt;&#xA;" OwnerUserId="40119" LastActivityDate="2020-08-06T05:48:04.593" Title="Automated Scoring (non-english language) Using BERT" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;pytorch&gt;&lt;word-embedding&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22888" PostTypeId="1" AcceptedAnswerId="22896" CreationDate="2020-08-06T07:42:47.230" Score="1" ViewCount="85" Body="&lt;p&gt;Could someone please help me gain some &lt;strong&gt;intuition&lt;/strong&gt; as to why the optimal policy for a Markov Decision Process in the infinite horizon case (agent acts forever) is deterministic?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2021-03-19T01:04:27.070" LastActivityDate="2021-03-19T01:04:27.070" Title="Why is the optimal policy for an infinite horizon MDP deterministic?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;policies&gt;&lt;optimal-policy&gt;&lt;optimality&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22889" PostTypeId="1" CreationDate="2020-08-06T08:09:14.450" Score="1" ViewCount="256" Body="&lt;p&gt;Is it okay if I label my images with their original size and then resize them, or should I first resize them and then label them?&lt;/p&gt;&#xA;&lt;p&gt;I mean do I need to recalibrate my labels if I resized my images?&lt;/p&gt;&#xA;" OwnerUserId="37414" LastActivityDate="2020-08-06T08:09:14.450" Title="Can I resize my images after labeling them?" Tags="&lt;object-detection&gt;&lt;object-recognition&gt;&lt;data-preprocessing&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22890" PostTypeId="1" CreationDate="2020-08-06T08:30:44.903" Score="3" ViewCount="43" Body="&lt;p&gt;I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of &lt;a href=&quot;https://stats.stackexchange.com/q/126238/82135&quot;&gt;RELU advantages&lt;/a&gt; on faster computation and no &lt;strong&gt;vanishing gradient&lt;/strong&gt; problem. But about vanishing gradient, the main problem is about the backpropagation algorithm going to zero quickly if using sigmoid or tanh. So I would like to try to compensate this effect that affects deep layers with a variable &lt;code&gt;learning rate&lt;/code&gt; for every layer, increasing the coefficient every time you go a layer deeper to compensate the vanishing gradient.&lt;/p&gt;&#xA;&lt;p&gt;I have read about &lt;a href=&quot;https://machinelearningmastery.com/learning-rate-for-deep-learning-neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;adaptive learning rate&lt;/a&gt;, but it seems to refer to a technique to change the learning rate on every epoch, I'm looking for a different learning rate for every layer, into any epoch.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on your experience, do you think that is a good effort to try?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you know some libraries I can use that already let you define the learning rate as a function and not a constant?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If such function exists, it will be better to define a simple function &lt;code&gt;lr=(a*n)*0.001&lt;/code&gt; where &lt;code&gt;n&lt;/code&gt; is layer number, and &lt;code&gt;a&lt;/code&gt; a multiplier based on experience, of we will need the inverse of the activation function to compensate enough the gradient vanishing?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25257" LastEditorUserId="2444" LastEditDate="2020-08-06T11:31:51.273" LastActivityDate="2020-08-06T11:31:51.273" Title="Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;activation-function&gt;&lt;learning-rate&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22892" PostTypeId="1" CreationDate="2020-08-06T10:38:21.487" Score="1" ViewCount="43" Body="&lt;p&gt;Is it a good idea to change the learning rate at each training step as a function of the loss? i.e. for points with high loss value, put a high learning rate and for low loss value a low learning rate (using a tailored function)?&lt;/p&gt;&#xA;&lt;p&gt;I know that the update of the parameters is done via &lt;span class=&quot;math-container&quot;&gt;$\gamma \nabla L$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\nabla L$&lt;/span&gt; is the gradient and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; the learning rate, and that points with high loss should correspond to a high gradient. Hence the dependency of the update of the parameters on the value of the loss should be already contained, although in a more indirect way. Is doing what I propose dangerous and/or useless?&lt;/p&gt;&#xA;" OwnerUserId="32915" LastEditorUserId="2444" LastEditDate="2020-08-06T11:35:50.280" LastActivityDate="2020-08-06T11:35:50.280" Title="Is it a good idea to change the learning rate at each training step as a function of the loss?" Tags="&lt;machine-learning&gt;&lt;learning-rate&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="22895" PostTypeId="2" ParentId="22462" CreationDate="2020-08-06T13:48:34.493" Score="1" Body="&lt;p&gt;The answer to your first question is because the line 'update the critic by minimising the loss &lt;span class=&quot;math-container&quot;&gt;$L = \frac{1}{N} \sum_i \left( y_i - Q(s_i, a_i |\theta^Q)\right)^2$&lt;/span&gt; is implying that you will do this by using a gradient, i.e. you calculate the gradient of the loss wrt the parameters and perform a gradient descent step.&lt;/p&gt;&#xA;&lt;p&gt;For the second question, I am not 100% sure because I don't use tensorflow but I would assume that in the automatic differentiation used it will do the multiplication automatically behind the scenes. In PyTorch I know that it automatically calculates the chain rule when you take the derivative, so it is likely to be a reason similar to this.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-06T13:48:34.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22896" PostTypeId="2" ParentId="22888" CreationDate="2020-08-06T14:31:00.597" Score="3" Body="&lt;p&gt;Suppose you learned your action-value function perfectly. Recall that the action-value function measures the expected return after taking a given action in a given state. Now, the goal when solving an MDP is to find a policy that maximizes expected returns. Suppose you're in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. According to your action-value function, let's say actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; maximizes the expected return. So, according to the goal of solving an MDP, the only action you would ever take from state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. In other words &lt;span class=&quot;math-container&quot;&gt;$\pi(a'\mid s) = \mathbf{1}[a'=a]$&lt;/span&gt;, which is a deterministic policy.&lt;/p&gt;&#xA;&lt;p&gt;Now, you might argue that your action-value function will never be perfect. However, this just means you need more exploration, which can manifest itself as stochasticity in the policy. However, in the limit of infinite data, the optimal policy will be deterministic since the action-value function will have converged to the optimum.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-08-06T14:31:00.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22897" PostTypeId="1" CreationDate="2020-08-06T14:47:51.503" Score="0" ViewCount="181" Body="&lt;p&gt;I've read the paper &lt;em&gt;A Neural Algorithm of Artistic Style&lt;/em&gt; by Gatys et. al. and I find the application of neural style transfer very fun.&lt;/p&gt;&#xA;&lt;p&gt;I also read that &lt;em&gt;Exploring the structure of a real-time, arbitrary neuralartistic stylization network&lt;/em&gt; by Ghiasi et. al. is a more modern approach to NST.&lt;/p&gt;&#xA;&lt;p&gt;My question is whether the above paper by Ghiasi et. al. is still the state-of-the-art method in NST, or maybe new algorithms perform even more efficiently.&lt;/p&gt;&#xA;&lt;p&gt;I shall precise that my goal is to deploy some NST algorithm on a web page as a fun project to apply some deep learning and learn about backend-frontend interactions.&lt;/p&gt;&#xA;" OwnerUserId="38660" LastActivityDate="2021-03-15T00:01:30.163" Title="What is the state-of-the-art algorithm for neural style transfer?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;deep-neural-networks&gt;&lt;image-processing&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22898" PostTypeId="1" CreationDate="2020-08-06T17:31:56.710" Score="0" ViewCount="13" Body="&lt;p&gt;Suppose I observe a set of states &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X} = \{X_{1}, X_{2}, \ldots, X_{K}\}$&lt;/span&gt; over time.&lt;/p&gt;&#xA;&lt;p&gt;I assume that there exist &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; cycles &lt;span class=&quot;math-container&quot;&gt;$\mathbf{C} = \{C_{1}, C_{2}, \ldots, C_{M} \}$&lt;/span&gt; in the observed state data of different lengths, for instance &lt;span class=&quot;math-container&quot;&gt;$C_{1} = \{X_{1}, X_{2}, X_{4} \}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_{2} = \{X_{3}, X_{5}, X_{6}, X_{9}\}$&lt;/span&gt;. However, a priori, I do not know what these cycles actually are, but I know that their constitutent states are disjoint.&lt;/p&gt;&#xA;&lt;p&gt;If I observe the sequence &lt;span class=&quot;math-container&quot;&gt;$\{X_{i}\}_{i \in K}$&lt;/span&gt; where the cycles may overlap or interleave with each other, how can I determine:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How many cycles exist in the sequence?&lt;/li&gt;&#xA;&lt;li&gt;What those cycles are?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Idea 1:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Train a markov model on the state sequence &lt;span class=&quot;math-container&quot;&gt;$\{X_{i}\}_{i \in K}$&lt;/span&gt;. This means finding the transition matrix. If I view the transition matrix as a graph, with the weight from state i to state j as &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{a_{ij}}$&lt;/span&gt;, then I can use Dijkstra's algorithm to find the shortest cycle. Once found, I remove the states, and do it again. I do this until no more cycles are detected.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;-- The problem with that is -- how do I know the cycle is a 'valid' cycle? Some edges may have very low probability.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Idea 2:&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Train a hidden Markov model on the data. The hidden states will be the cycles.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;-- The problem with that is -- how do I choose the number of cycles to train for?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Are these good ideas? How do I address those problems? Are there any other methods that I am missing?&lt;/p&gt;&#xA;&lt;p&gt;Any references, or ideas are welcome!&lt;/p&gt;&#xA;" OwnerUserId="40133" LastActivityDate="2020-08-06T17:31:56.710" Title="Finding Cycles in a State Sequence" Tags="&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;unsupervised-learning&gt;&lt;markov-chain&gt;&lt;hidden-markov-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22899" PostTypeId="1" CreationDate="2020-08-06T17:45:59.027" Score="1" ViewCount="61" Body="&lt;p&gt;When designing CNN for image recogition a commonly used sainty check to see if a model is working/designed fine is to see if we are able to overfit the model with a very small subset of images.&lt;/p&gt;&#xA;&lt;p&gt;I am trying out GANs. While designing GAN I took a dataset with just one image(full black image). I used the DCGAN implementation in pytorch website&lt;a href=&quot;https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;code link&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I tried training the model with this just one black image and even after training for 100s-1000 epochs. I am not able to overfit the model ie generate a black (or something close). All what is generated are random noise image as below&lt;a href=&quot;https://i.stack.imgur.com/YdjLZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YdjLZ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However the model does work well for celeba dataset(the one used in the tutorial). Which means the model is good. Can anybody help me why overfitting is very difficult/impossible when using a single image.&lt;/p&gt;&#xA;" OwnerUserId="40134" LastActivityDate="2020-08-06T17:45:59.027" Title="How to overfit GANs with a single image" Tags="&lt;generative-adversarial-networks&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22900" PostTypeId="1" AcceptedAnswerId="22948" CreationDate="2020-08-06T22:06:40.557" Score="9" ViewCount="398" Body="&lt;p&gt;I'm reading Reinforcement Learning by Sutton &amp;amp; Barto, and in section 3.2 they state that the reward in a Markov decision process is always a scalar real number. At the same time, I've heard about the problem of assigning credit to an action for a reward. Wouldn't a vector reward make it easier for an agent to understand the effect of an action? Specifically, a vector in which different components represent different aspects of the reward. For example, an agent driving a car may have one reward component for driving smoothly and one for staying in the lane (and these are independent of each other).&lt;/p&gt;&#xA;" OwnerUserId="40138" LastEditorUserId="2444" LastEditDate="2020-10-08T14:15:59.193" LastActivityDate="2021-01-15T19:16:31.817" Title="Why is the reward in reinforcement learning always a scalar?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;reward-functions&gt;&lt;multi-objective-rl&gt;" AnswerCount="3" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22901" PostTypeId="1" CreationDate="2020-08-06T23:01:37.490" Score="0" ViewCount="92" Body="&lt;p&gt;I am trying to train an AI with an environment where the states are continuous but the actions are discrete, that means I can not apply DDPG or TD3.&lt;/p&gt;&#xA;&lt;p&gt;Can someone please help to let know what should be the best algorithm for discrete action spaces and is there any version of DDPG or TD3 which can be applied to discrete action spaces on partially observable MDPs.&lt;/p&gt;&#xA;" OwnerUserId="40051" LastEditorUserId="2444" LastEditDate="2020-08-06T23:31:00.387" LastActivityDate="2020-08-06T23:31:00.387" Title="Which is the best RL algo for continuous states but discrete action spaces problem" Tags="&lt;ai-design&gt;&lt;deep-rl&gt;&lt;ddpg&gt;&lt;td3&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22902" PostTypeId="5" CreationDate="2020-08-06T23:32:05.580" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-06T23:32:05.580" LastActivityDate="2020-08-06T23:32:05.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22903" PostTypeId="4" CreationDate="2020-08-06T23:32:05.580" Score="0" Body="For questions related to the Twin Delayed Deep Deterministic policy gradient algorithm (TD3)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-06T23:32:05.580" LastActivityDate="2020-08-06T23:32:05.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22904" PostTypeId="2" ParentId="8844" CreationDate="2020-08-06T23:46:16.913" Score="1" Body="&lt;h3&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Upgrade_(film)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Upgrade&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;p&gt;This film depicts a very plausible near future when drones oversee our lives (e.g. the police use them to fight crime) and common people possess self-driving cars.&lt;/p&gt;&#xA;&lt;p&gt;This is definitely one of the best science fiction movies I have ever watched in my entire life, and I have watched many, such as &lt;strong&gt;2001&lt;/strong&gt;, &lt;strong&gt;Blade Runner&lt;/strong&gt;, or &lt;strong&gt;The Matrix&lt;/strong&gt;. In fact, these are the four best science fiction movies ever made, in my opinion (and I have some knowledge of cinema, cinematography, directing, etc.)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-08-06T23:46:16.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22905" PostTypeId="1" CreationDate="2020-08-07T01:57:23.523" Score="0" ViewCount="33" Body="&lt;p&gt;To be more specific I have a dataset of 2400 images with unbalanced classes, 1 object per image and sometimes some objects are repeated  along the dataset but in a different position and rotation of the object.&lt;/p&gt;&#xA;&lt;p&gt;So the problem is, will the neural network trained on this data set recognize objects in different scenarios?&lt;/p&gt;&#xA;" OwnerUserId="40140" LastEditorUserId="40140" LastEditDate="2020-08-07T02:02:51.737" LastActivityDate="2020-08-07T02:02:51.737" Title="Can I train an object detection model with images with a white background?" Tags="&lt;computer-vision&gt;&lt;datasets&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22906" PostTypeId="1" CreationDate="2020-08-07T03:25:55.650" Score="0" ViewCount="19" Body="&lt;h1&gt;How to develop a system which detects if a object crosses a line.&lt;/h1&gt;&#xA;&lt;p&gt;I am developing a system, basically which tells a object crossed a line. Do I need to get the camera calibration matrix ? Do I need to understand camera calibration or we can use annotation line points(using CVAT) to calculate the math ? If yes and mandatory, I will go on that path. Please advice.&lt;/p&gt;&#xA;&lt;h3&gt;Camera calibration matrix is required(mandatory) for below use case?&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ghRJH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ghRJH.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;I guess, No Camera Calibration matrix required, since camera is just looking from top view&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S3stH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S3stH.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;My approach,&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Draw lines using CVAT&lt;/li&gt;&#xA;&lt;li&gt;Write a program using opencv which uses object bouding box center and trip line intersection to calculate the crossing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CV7I9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CV7I9.jpg&quot; alt=&quot;I draw lines using CVAT &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;I got these points, can we use these without calibaration ? &#xA;&amp;lt;polyline label=&amp;quot;entry_out&amp;quot; occluded=&amp;quot;0&amp;quot; source=&amp;quot;manual&amp;quot; points=&amp;quot;774.68,499.77;444.65,646.45&amp;quot;&amp;gt; &amp;lt;/polyline&amp;gt;&#xA;&amp;lt;polyline label=&amp;quot;boundary&amp;quot; occluded=&amp;quot;0&amp;quot; source=&amp;quot;manual&amp;quot; points=&amp;quot;406.80,349.70;1031.37,720.00&amp;quot;&amp;gt; &amp;lt;/polyline&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37741" LastActivityDate="2020-08-07T03:25:55.650" Title="How to develop a model to detect a point crossed a line?" Tags="&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22907" PostTypeId="1" AcceptedAnswerId="22910" CreationDate="2020-08-07T03:57:03.260" Score="1" ViewCount="86" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;If a model is not available, then it is particularly useful to estimate action values (the&#xA;values of state-action pairs) rather than state values. With a model, state values alone are&#xA;sufficient to determine a policy; one simply looks ahead one step and chooses whichever&#xA;action leads to the best combination of reward and next state, as we did in the chapter on&#xA;DP. &lt;strong&gt;Without a model, however, state values alone are not sufficient. One must explicitly&#xA;estimate the value of each action in order for the values to be useful in suggesting a policy.&lt;/strong&gt;&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The above extract is from Sutton and Barto's Reinforcement Learning, Section 5.2 - part of the chapter on Monte Carlo Methods.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please explain in some more detail, as to &lt;strong&gt;why it is necessary to determine the value of each action&lt;/strong&gt; (i.e. state-values alone are not sufficient) for suggesting a policy in a &lt;strong&gt;model-free setting&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;P.S.&lt;/p&gt;&#xA;&lt;p&gt;From what I know, state-values basically refer to the expected return one gets when starting from a state (we know that we'll reach a terminal state, since we're dealing with Monte Carlo methods which, at least in the book, look at only episodic MDPs). That being said, why is it not possible to suggest a policy solely on the basis of state-values; why do we need state-action values? I'm a little confused, it'd really help if someone could clear it up.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-08-07T04:06:18.143" LastActivityDate="2020-08-07T09:24:55.150" Title="Why are state-values alone not sufficient in determining a policy (without a model)?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22908" PostTypeId="1" AcceptedAnswerId="22909" CreationDate="2020-08-07T05:19:43.647" Score="4" ViewCount="65" Body="&lt;p&gt;One of my friends and I were discussing the differences between Dynamic Programming, Monte-Carlo, and Temporal Difference (TD) Learning as policy evaluation methods - and we agreed on the fact that &lt;strong&gt;Dynamic Programming&lt;/strong&gt; &lt;strong&gt;requires&lt;/strong&gt; the Markov assumption while &lt;strong&gt;Monte-Carlo&lt;/strong&gt; policy evaluation does &lt;strong&gt;not&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, he also pointed out that &lt;strong&gt;Temporal Difference Learning cannot handle non-Markovian domains&lt;/strong&gt;, i.e. it depends on the Markov assumption. Why is it so?&lt;/p&gt;&#xA;&lt;p&gt;The way I understand it, the TD learning update is, in essence, the same as the Monte-Carlo update, except for the fact that the return instead of being calculated using the entire trajectory, is bootstrapped from the previous estimate of the value function, i.e. we can update the value as soon as we encounter a &lt;span class=&quot;math-container&quot;&gt;$(s,a,r,s')$&lt;/span&gt; tuple, we don't have to wait for the episode (if finite) to terminate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Where is the &lt;em&gt;Markov assumption&lt;/em&gt; being used here, i.e the &lt;em&gt;future is independent of the past given the present&lt;/em&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-08-07T15:34:24.030" Title="Why does TD Learning require Markovian domains?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;&lt;markov-property&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22909" PostTypeId="2" ParentId="22908" CreationDate="2020-08-07T08:34:44.840" Score="5" Body="&lt;p&gt;The Markov assumption is used when deriving the Bellman equation for state values:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v(s) = \sum_a \pi(a|s)\sum_{r,s'} p(r,s'|s,a)(r + \gamma v(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;One requirement for this equation to hold is that &lt;span class=&quot;math-container&quot;&gt;$p(r,s'|s,a)$&lt;/span&gt; is consistent. The current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is a key argument of that function. There is no adjustment for history of previous states, actions or rewards. This is the same as requiring the Markov trait for state, i.e. that &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; holds all information necessary to predict outcome probabilities of the next step.&lt;/p&gt;&#xA;&lt;p&gt;The one step TD target that is sampled in basic TD learning is simply the inner part of this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_{t:t+1} = R_{t+1} + \gamma \hat{v}(S_{t+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which when sampled is equal to &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt; in &lt;em&gt;expectation&lt;/em&gt; *, when &lt;span class=&quot;math-container&quot;&gt;$S_t = s$&lt;/span&gt;. That is, when you measure a single instance of the TD target and use it to update a value function, you implicitly assume that the values or &lt;span class=&quot;math-container&quot;&gt;$r_{t+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; that you observed occur with probabilities determined by &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(r,s'|s,a)$&lt;/span&gt; as shown by the Bellman equation.&lt;/p&gt;&#xA;&lt;p&gt;So the theory behind TD learning uses the Markov assumption, otherwise the sampled TD targets would be incorrect.&lt;/p&gt;&#xA;&lt;p&gt;In practice you can get away with slightly non-Markov environments - most measurements of state for machinery are approximations that ignore details at some level, for instance, and TD learning can solve optimal control in many robotics environments. However, Monte Carlo methods are more robust against state representations that are not fully Markov.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;* Technically this sample is biased because &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(S_{t+1})$&lt;/span&gt; is not correct when learning starts. The bias reduces over time and multiple updates. So the expected value during learning is &lt;em&gt;approximately&lt;/em&gt; the same as the true value as shown by the Bellman equation.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-07T15:34:24.030" LastActivityDate="2020-08-07T15:34:24.030" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22910" PostTypeId="2" ParentId="22907" CreationDate="2020-08-07T08:55:59.810" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;why is it not possible to suggest a policy solely on the basis of state-values; why do we need state-action values?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A policy function takes state as an argument and returns an action &lt;span class=&quot;math-container&quot;&gt;$a = \pi(s)$&lt;/span&gt;, or it may return a probability distribution over actions &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Pr}\{A_t=a|S_t=s \} =\pi(a|s)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In order to do this rationally, an agent needs to use the knowledge it has gained to select the best action. In value-based methods, the agent needs to identify the action that has the highest expected return. As an aside, whilst learning it may not take that action because it has decided to explore, but if it is not capable of even identifying a best action then there is no hope of it ever finding an optimal policy, and it cannot even perform &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy action selection, which is a very basic exploration approach.&lt;/p&gt;&#xA;&lt;p&gt;If you use an action value estimate, then the agent can select the greedy action simply:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a Q(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you have state values, then the agent can select the greedy action directly only if it knows the model distribution &lt;span class=&quot;math-container&quot;&gt;$p(r,s'|s,a)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a \sum_{r,s'}p(r,s'|s,a)(r + \gamma V(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In other words, to find the best action to take the agent needs to look ahead a time step to find out what the distribution of next states would be following that action. If the only values the agent knows are state values, this is the only way the agent can determine the impact of any specific action.&lt;/p&gt;&#xA;&lt;p&gt;Although there are alternatives to this specific equation, there is no alternative that does not use a model in some form. For instance, if you can simulate the environment, you could simulate taking each action in turn, and look over multiple simulation runs to see which choice ends up with the best &lt;span class=&quot;math-container&quot;&gt;$(r + \gamma V(s'))$&lt;/span&gt; on average. That would be a type of planning, and perhaps the start of a more sophisticated approach such as MCTS. However, that simulation &lt;em&gt;is&lt;/em&gt; a model - it needs access to the transition probabilities in some form in order to correctly run.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to have an entirely &lt;em&gt;separate&lt;/em&gt; policy function that you train alongside a state value function. This is the basis of Actor-Critic methods, which make use of policy gradients to adjust the policy function, and one of the value-based methods, such as TD learning, to learn a value function that assists with calculating the updates to the policy function. In that case you would not be using a value-based method on its own, so the quote from that part of Sutton &amp;amp; Barto does not apply.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-07T09:24:55.150" LastActivityDate="2020-08-07T09:24:55.150" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22911" PostTypeId="2" ParentId="22827" CreationDate="2020-08-07T10:02:31.700" Score="3" Body="&lt;h1&gt;Depends on perspective.&lt;/h1&gt;&#xA;&lt;p&gt;On one hand, you have an agent playing in an environment with another agent also evolving. This falls under the definition of Multi-Agent Learning, as can be seen with works such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning&#xA;rate. Artificial Intelligence, 136(2):215 – 250, 2002.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Michael Bowling. Convergence and no-regret in multiagent learning. In Proceedings of&#xA;the 17th International Conference on Neural Information Processing Systems, NIPS’04,&#xA;pages 209–216, Cambridge, MA, USA, 2004. MIT Press.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;M. D. Awheda and H. M. Schwartz. Exponential moving average q-learning algorithm. In&#xA;2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning&#xA;(ADPRL), pages 31–38, April 2013.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sherief Abdallah and Victor Lesser. A multiagent reinforcement learning algorithm with&#xA;non-linear dynamics. Journal of Artificial Intelligence Research, 33:521–549, 2008.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, you can also claim that you simply have a single agent learning on a non-stationary environment (the environment contains both the game rules and the opponent), and you simply learn on that basis. From this perspective, there is no multi-agent learning at all.&lt;/p&gt;&#xA;" OwnerUserId="7496" LastActivityDate="2020-08-07T10:02:31.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22912" PostTypeId="1" CreationDate="2020-08-07T11:48:21.100" Score="2" ViewCount="39" Body="&lt;p&gt;I have two signals that I want to use to model my reward.&lt;/p&gt;&#xA;&lt;p&gt;The first one is the &lt;strong&gt;CPU TIME: running mean&lt;/strong&gt; from this diagram:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m3Ijz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m3Ijz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The second one is the &lt;strong&gt;MAX RESIDUAL&lt;/strong&gt; from this diagram:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h8zOg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h8zOg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since they are both equally important, I can weight them together like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r = w_\rho \rho + w_\tau \tau$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the reward function, &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the &lt;strong&gt;CPU TIME: running mean&lt;/strong&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is the &lt;strong&gt;MAX RESIDUAL&lt;/strong&gt;. The problem is, how to set the weights &lt;span class=&quot;math-container&quot;&gt;$w_\tau,w_\rho$&lt;/span&gt; to make the contributions equally important if &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; are on very different scales?&lt;/p&gt;&#xA;&lt;p&gt;RL algorithms will learn policies based on increases/decreases of the reward, and if one signal has values that are much smaller than the other, it will influence the reward less, if this is done in a wrong way.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, if the algorithm converges, they must be on different scales, as I want the CPU time to go &lt;em&gt;ideally&lt;/em&gt; to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and residuals &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; to be minimized as well.&lt;/p&gt;&#xA;&lt;p&gt;Modeling the reward function is a crucial RL step, because it decides what the algorithm will in fact optimize. How are examples like these handled? Are there any best practices for this? Also, what happens when there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; such signals, that have to be combined with &amp;quot;equally important&amp;quot; weighting into a reward function?&lt;/p&gt;&#xA;&lt;p&gt;Basing the weights &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; on the &lt;em&gt;current&lt;/em&gt; signal values is possible to define their reward, but then the reward contributions won't see &lt;span class=&quot;math-container&quot;&gt;$\max(\rho), \min(\rho)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\max(\tau), \min(\tau)$&lt;/span&gt; over time.&lt;/p&gt;&#xA;&lt;p&gt;So, how do you do feature scaling for reward signals?&lt;/p&gt;&#xA;" OwnerUserId="37627" LastEditorUserId="2444" LastEditDate="2020-11-02T21:46:10.833" LastActivityDate="2020-11-02T21:46:10.833" Title="How to combine two differently equally important signals into the reward function, that have different scales?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22913" PostTypeId="1" CreationDate="2020-08-07T12:55:09.513" Score="1" ViewCount="1952" Body="&lt;p&gt;How do I determine the computational complexity (big-O notation) of the forward pass of a &lt;strong&gt;convolutional&lt;/strong&gt; neural network?&lt;/p&gt;&#xA;&lt;p&gt;Let's assume for simplicity that we use zero-padding such that the input size and the output size are the same.&lt;/p&gt;&#xA;" OwnerUserId="40147" LastEditorUserId="2444" LastEditDate="2020-08-08T14:31:26.483" LastActivityDate="2020-08-08T14:44:59.433" Title="What is the computational complexity of the forward pass of a convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;time-complexity&gt;&lt;computational-complexity&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22914" PostTypeId="1" CreationDate="2020-08-07T13:19:44.373" Score="3" ViewCount="404" Body="&lt;p&gt;For MCTS there is an expansion phase where we make a move and list down all the next states. But this is complicated by the fact that for some games, after making the move, there is a stochastic change to the environment. Consider the game 2048, after I make a move, random tile is generated. So the state of the world after my next move is a mix of possibilities!&lt;/p&gt;&#xA;&lt;p&gt;How does MCTS work in a stochastic environment? I am having trouble understanding how to keep track of the expansion, do I expand all stochastic possibilities and weight the return via their chance of happening?&lt;/p&gt;&#xA;" OwnerUserId="20234" LastActivityDate="2020-08-13T16:14:41.833" Title="How to run a Monte Carlo Tree Search MCTS for stochastic environment?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22915" PostTypeId="1" CreationDate="2020-08-07T14:48:40.400" Score="0" ViewCount="10" Body="&lt;p&gt;The idea of using AI to automatically propose new designs for physical objects seems very interesting. Looking at some proofs-of-concept, e.g. &lt;a href=&quot;https://www.autodesk.com/solutions/generative-design&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.autodesk.com/solutions/generative-design&lt;/a&gt;, I am wondering,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is that still an area where some companies use custom and proprietary software to consult on particular projects, or&lt;/li&gt;&#xA;&lt;li&gt;are there some general frameworks/high-level languages out there that allow anyone to dabble with their own toy-projects?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40150" LastActivityDate="2020-08-07T14:48:40.400" Title="Framework/language for automated design of physical objects" Tags="&lt;automation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22916" PostTypeId="2" ParentId="22869" CreationDate="2020-08-07T15:40:22.710" Score="0" Body="&lt;p&gt;Maybe this is what you are looking for:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Basically you would&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;build and store a finite-state machine that resembles a trie with additional links between the various internal nodes using the given keywords.&lt;/li&gt;&#xA;&lt;li&gt;for the candidate document, go through it with the above finite-state machine.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Then based on the result&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If there is any match, then the document is valid.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise it is possible that the document is invalid. However, in terms of the probability, I do not have any good idea yet.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17133" LastActivityDate="2020-08-07T15:40:22.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22917" PostTypeId="2" ParentId="13848" CreationDate="2020-08-07T16:50:09.903" Score="0" Body="&lt;p&gt;I've been working with a TD3 implementation for bipedal hardcore. It solved the easy version (v2 and v3) in about 300 epochs (&lt;a href=&quot;https://github.com/QasimWani/policy-value-methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/QasimWani/policy-value-methods&lt;/a&gt;). I've been training it for hardcore and even after about 1200 episodes, it's no where close to convergence.&#xA;Did you end up solving, and if so, what algorithm did you end up going with?&#xA;Cheers,&#xA;Q.&lt;/p&gt;&#xA;" OwnerUserId="40155" LastActivityDate="2020-08-07T16:50:09.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22918" PostTypeId="1" CreationDate="2020-08-07T17:53:34.797" Score="1" ViewCount="47" Body="&lt;p&gt;In supervised learning we have an unbiased target value, but in reinforcement learning this isn’t the case&lt;/p&gt;&#xA;&lt;p&gt;The network predicts its own target value, now how exactly does it converge if the network predicts its target value&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain this to me ??&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pe3vt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pe3vt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40049" LastActivityDate="2020-08-07T17:53:34.797" Title="How does DQN convergence work in reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="22919" PostTypeId="2" ParentId="18378" CreationDate="2020-08-07T18:06:48.280" Score="0" Body="&lt;p&gt;In framing the problem as an &lt;em&gt;episodic&lt;/em&gt; reinforcement learning problem, the goal is to find a policy that optimizes &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[\sum_{t=0}^\tau r(s_t)],$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the random time at which the robot leaves the maze. This implicitly assigns a reward of 0 to the out-of-maze state, &lt;span class=&quot;math-container&quot;&gt;$s_{terminal}$&lt;/span&gt;. If you include this state then the transformation &lt;span class=&quot;math-container&quot;&gt;$r\rightarrow 1\cdot r - 1$&lt;/span&gt; does not change the optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;If we rewrite this episodic objective accounting for &lt;span class=&quot;math-container&quot;&gt;$s_{terminal}$&lt;/span&gt; (and a horizon &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;) we get the following objective:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}\mathbb{E}\left[\sum_{t=0}^\tau r(s_t) + \sum_{\tau}^H r(s_{terminal})\right] &amp;amp;= \mathbb{E}\left[(H-\tau) r(s_{terminal}) + \sum_{t=0}^\tau r(s_t)\right]\\&#xA;&amp;amp;= \mathbb{E}\left[(H-\tau) r(s_{terminal}) + r(s_{goal}) + (\tau-1) r(s_{maze}) \right]\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$s_{goal}$&lt;/span&gt; is the exit state from the maze, the goal, and &lt;span class=&quot;math-container&quot;&gt;$s_{maze}$&lt;/span&gt; represents the other states of the maze. In the question, 1 is subtracted from &lt;span class=&quot;math-container&quot;&gt;$s_{goal}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_{maze}$&lt;/span&gt;, but not &lt;span class=&quot;math-container&quot;&gt;$s_{terminal}$&lt;/span&gt;. Thus, this is not a positive affine transformation of the reward function. In effect, this changes the relative value of &lt;span class=&quot;math-container&quot;&gt;$s_{terminal}$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\min_s r(s)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\max_s r(s)$&lt;/span&gt;, and that changes the optimal policy.&lt;/p&gt;&#xA;" OwnerUserId="40157" LastActivityDate="2020-08-07T18:06:48.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22920" PostTypeId="1" CreationDate="2020-08-08T04:10:48.320" Score="0" ViewCount="16" Body="&lt;p&gt;Is there a way to estimate odometry using a single camera without a depth map? It somewhat seems like a circular problem since to estimate movement you need to know the change in object size, but you don't know the size of the object without either movement or a stereo camera setup.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-08-08T10:58:02.003" LastActivityDate="2020-08-08T10:58:02.003" Title="Is there a way to estimate odometry using a single camera without a depth map?" Tags="&lt;computer-vision&gt;&lt;odometry&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22921" PostTypeId="1" CreationDate="2020-08-08T05:44:21.740" Score="0" ViewCount="29" Body="&lt;p&gt;Just to check if everything is working properly in my neural net, I set my validation data to be the same as my training data, expecting to achieve a better NRMS for validation data (since it uses only the most recently updated weights for calculating the NRMS). But that didn't happen. More often than not, the NRMS was larger for the validation data than for training data.&#xA;Am I missing something or is this a normal thing? Cause it doesn't make sense to me.&lt;/p&gt;&#xA;" OwnerUserId="38668" LastActivityDate="2020-08-08T05:44:21.740" Title="What would happen if validation set was the same as the training set?" Tags="&lt;training&gt;&lt;cross-validation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22922" PostTypeId="2" ParentId="8496" CreationDate="2020-08-08T06:01:09.053" Score="0" Body="&lt;p&gt;According to &lt;a href=&quot;http://tianlinliu.com/files/notes_exercise_RL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://tianlinliu.com/files/notes_exercise_RL.pdf&lt;/a&gt;, MDP may not be feasible to multi-target tasks.&lt;/p&gt;&#xA;&lt;p&gt;In contrast, EA-based methods like NSGA-II, NSGA-III, can solve the multi-target tasks.&lt;/p&gt;&#xA;&lt;p&gt;And also, tasks that need more than one state to predict the next action are also not suitable to use MDP. For example, when we predict the next action a stranger who just meets you at a party will do, we need to consider all the behaviors he did in the past minutes. It will be more suitable than using the MDP way, which will simply label the stranger is a &amp;quot;good&amp;quot; guy or &amp;quot;bad&amp;quot; guy (or a continuous number between good or bad).&lt;/p&gt;&#xA;" OwnerUserId="40164" LastEditorUserId="40164" LastEditDate="2020-08-08T06:17:23.123" LastActivityDate="2020-08-08T06:17:23.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22923" PostTypeId="1" CreationDate="2020-08-08T06:54:43.033" Score="0" ViewCount="29" Body="&lt;p&gt;I am trying to implement the DDPG algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Why is the actor's loss calculated as the negative mean of the model's predicted Q values in the states we are in? Shouldn't it be like the difference of the Q values when random action is taken and Q values of the model's predicted actions in that state?&lt;/p&gt;&#xA;&lt;p&gt;Basically, if a random action is generating a better Q value, only then the actor Q value should be updated, else we already have the value and no need to propagate actor loss separately. It will be taken care of in the value loss.&lt;/p&gt;&#xA;&lt;p&gt;As I understand it, there may be a case when we want to maximize the actor Q values with a constraint given by backpropagation of value loss in the critic network. Is it correct?&lt;/p&gt;&#xA;" OwnerUserId="40051" LastEditorUserId="2444" LastEditDate="2020-08-08T13:20:44.277" LastActivityDate="2020-08-08T13:20:44.277" Title="Why is actor's loss the mean of the predicted Q values and not an error in calculating Q values, in the DDPG algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22924" PostTypeId="1" CreationDate="2020-08-08T08:55:39.730" Score="0" ViewCount="24" Body="&lt;p&gt;Can someone please explain to me the complete architecture of cGANs as mentioned &lt;a href=&quot;https://affinelayer.com/pix2pix/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I am confused as &amp;quot;ref&amp;quot; is not defined in the whole article. I am sorry if you find the question silly. I don't have a ML background (Just trying to solve some inverse problems in applied optics using ML). Thanks in Advance!&lt;/p&gt;&#xA;" OwnerUserId="40167" LastEditorUserId="40167" LastEditDate="2020-08-08T11:13:19.613" LastActivityDate="2020-08-08T11:13:19.613" Title="How Discriminator and Generator weights are adjusted in conditional Generative adversarial networks proposed by Isola et al. ? (in simple terms)" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22925" PostTypeId="1" CreationDate="2020-08-08T11:47:55.787" Score="0" ViewCount="6" Body="&lt;p&gt;I have a need to classify documents in a set of documents, which grows over time from a small tagged training set. The classification is a binary classification. Training on the tagged set produces good results, but as the set increases in size as does the size of the vocabulary the model degrades. The model is a Boosted NaiveBayes applied to a tfidf representation of the text. Each document is a reasonably sized news report.&lt;/p&gt;&#xA;&lt;p&gt;What is the state of the art solution for such a problem? I was thinking a semi-supervised approach might be a way forward, tagging new data using the previous model to create a new model but it doesn’t seem successful.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastActivityDate="2020-08-08T11:47:55.787" Title="What is the state of the art solution for text classification for large corpora" Tags="&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22929" PostTypeId="2" ParentId="22913" CreationDate="2020-08-08T14:18:28.727" Score="2" Body="&lt;h2&gt;What is the time complexity?&lt;/h2&gt;&#xA;&lt;p&gt;The time complexity of an algorithm is the number of basic operations, such as multiplications and summations, that the algorithm performs. The time complexity is usually expressed as a function of the input's size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (but this does not always have to be the case: for instance, you can express the time complexity as a function of the output's size).&lt;/p&gt;&#xA;&lt;h2&gt;Example&lt;/h2&gt;&#xA;&lt;p&gt;Rather than giving you a full answer to your question, I will try to help you by explaining, with the simplest example, how you should calculate the time complexity.&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, let's assume that we have a kernel &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H} \in \mathbb{R}^{3 \times 3}$&lt;/span&gt; and input image &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I} \in \mathbb{R}^{3 \times 3}$&lt;/span&gt; (i.e. the kernel has the same dimensions as the input), we use a stride of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and no padding. If we convolve &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H}$&lt;/span&gt;, how many operations will we perform? The convolution is defined as a scalar product, so it is composed of multiplications and summations, so we need to count both of them. We have &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; multiplications and &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; summations, for a total of &lt;span class=&quot;math-container&quot;&gt;$17$&lt;/span&gt; operations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{I} \circledast \mathbf{H} &#xA;&amp;amp;=&#xA;\begin{bmatrix}&#xA;i_{11} &amp;amp; i_{12} &amp;amp; i_{13} \\&#xA;i_{21} &amp;amp; i_{22} &amp;amp; i_{23} \\&#xA;i_{31} &amp;amp; i_{32} &amp;amp; i_{33}&#xA;\end{bmatrix}&#xA;\odot&#xA;\begin{bmatrix}&#xA;h_{11} &amp;amp; h_{12} &amp;amp; h_{13} \\&#xA;h_{21} &amp;amp; h_{22} &amp;amp; h_{23} \\&#xA;h_{31} &amp;amp; h_{32} &amp;amp; h_{33}&#xA;\end{bmatrix}\\&#xA;&amp;amp;=&#xA;\sum_{ij}&#xA;\begin{bmatrix}&#xA;i_{11} h_{11} &amp;amp; i_{12} h_{12} &amp;amp; i_{13} h_{13} \\&#xA;i_{21} h_{21} &amp;amp; i_{22}  h_{22} &amp;amp; i_{23} h_{23} \\&#xA;i_{31} h_{31} &amp;amp; i_{32} h_{32} &amp;amp; i_{33} h_{33}&#xA;\end{bmatrix}\\&#xA;&amp;amp;=&#xA;i_{11} h_{11} + i_{12} h_{12} + i_{13} h_{13} + &#xA;i_{21} h_{21} + i_{22}  h_{22} + i_{23} h_{23} + &#xA;i_{31} h_{31} + i_{32} h_{32} + i_{33} h_{33}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Time complexity&lt;/h3&gt;&#xA;&lt;p&gt;What is the time complexity of this convolution? To answer this question, you first need to know the input's size, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. The input contains &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; elements, so its size is &lt;span class=&quot;math-container&quot;&gt;$n = 9$&lt;/span&gt;. How many operations did we perform with respect to the input's size? We performed &lt;span class=&quot;math-container&quot;&gt;$17$&lt;/span&gt; operations, so the time complexity &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(2*n) = \mathcal{O}(n)$&lt;/span&gt;, i.e. this operation is linear. If you are not familiar with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Big_O_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;big-O notation&lt;/a&gt;, I suggest that you get familiar with it, otherwise, you will not understand anything about computational complexity.&lt;/p&gt;&#xA;&lt;p&gt;To calculate the time complexity in the case the input's dimensions are different than the kernel's dimensions, you will need to calculate the number of times you slide the kernel over the input. You can't ignore this (as I ignored the constant &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; above) because the number of times you slide the kernel over the input depends on the input's size, so it's a function of the input. Anyway, the paper &lt;a href=&quot;https://arxiv.org/pdf/1603.07285.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A guide to convolution arithmetic for deep learning&lt;/a&gt; contains a lot of information about convolution arithmetic, so it will be helpful.&lt;/p&gt;&#xA;&lt;h3&gt;Non-linearities, pooling and fully connected layers&lt;/h3&gt;&#xA;&lt;p&gt;Note that, in the above example, I ignored the non-linearities and pooling layer. You can easily extend my reasoning to include these operations too. I also ignored the operations in the final fully connected layers. You can find how to calculate the number of operations in an MLP &lt;a href=&quot;https://ai.stackexchange.com/a/13626/2444&quot;&gt;in this answer&lt;/a&gt;. If you also perform other operations or have other layers other than convolutional, pooling and fully connected, of course, you will also need to take them into account.&lt;/p&gt;&#xA;&lt;h3&gt;Forward pass&lt;/h3&gt;&#xA;&lt;p&gt;Moreover, the time complexity of the forward pass of a CNN depends on all these operations in these different layers, so you need to compute the number of operations in each layer first. However, once you know how to compute the number of operations for one convolutional layer, one pooling layer, and one fully connected layer, you can easily compute the number of operations for the other convolutional, pooling, and fully connected layers. Then you just need to sum all these operations and express your time complexity as a function of the input (and probably number of layers).&lt;/p&gt;&#xA;&lt;h3&gt;Space complexity&lt;/h3&gt;&#xA;&lt;p&gt;If you also want to compute the space complexity, you just need to do the same thing, but as a function of the space that you use, i.e. how many variables you use to perform the convolution.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-08T14:44:59.433" LastActivityDate="2020-08-08T14:44:59.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22930" PostTypeId="1" AcceptedAnswerId="22935" CreationDate="2020-08-08T22:58:14.970" Score="0" ViewCount="29" Body="&lt;p&gt;I am somewhat a novice at the topic of Neural Netoworks and PyTorch.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to create a model that takes a word (that I have modified very slightly) and a 'window' of context around it and predicts one of 5 tags (the tags relate to what sort of action I should perform on that word to get its correct form).&lt;/p&gt;&#xA;&lt;p&gt;For example, here's what I would call a window of size 7 and it's tag (what it means isn't too important, it's just the 'target'):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        Sentence                    Label&#xA;here is a sentence for my network     N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;sentence&lt;/code&gt; is the word that I want the network to predict the label for, but the 3 words on either side provide contextual meaning. My problem is, how would I get a network to know I want it to predict for that central word but not outright ignore the others? I am familiar with more normal NLP tasks such as NMT and character level classification.&lt;/p&gt;&#xA;&lt;p&gt;I have already gotten my dataset 'padded' out so they're all of equal size.&lt;/p&gt;&#xA;&lt;p&gt;Any help is appreciated&lt;/p&gt;&#xA;" OwnerUserId="40177" LastActivityDate="2020-08-09T02:11:21.803" Title="Get Neural Network to predict a tag/class on a certain word using the surrounding words as context [PyTorch]?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22931" PostTypeId="1" AcceptedAnswerId="22937" CreationDate="2020-08-08T23:43:56.357" Score="0" ViewCount="169" Body="&lt;p&gt;What are some books on reinforcement learning (RL) and deep RL for beginners?&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for something as friendly as the head first series, that breaks down every single thing.&lt;/p&gt;&#xA;" OwnerUserId="40049" LastEditorUserId="2444" LastEditDate="2020-08-09T10:43:49.667" LastActivityDate="2020-08-10T04:07:44.707" Title="What are some (deep) reinforcement learning books for beginners?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;reference-request&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ClosedDate="2020-08-14T11:45:13.147" ContentLicense="CC BY-SA 4.0" />
  <row Id="22932" PostTypeId="5" CreationDate="2020-08-09T00:09:33.903" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-08-09T00:09:33.903" LastActivityDate="2020-08-09T00:09:33.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22933" PostTypeId="4" CreationDate="2020-08-09T00:09:33.903" Score="0" Body="For questions about odometry, the use of data from motion sensors to estimate change in position over time. This can be used with the (computer-vision) and (image-recognition) tags." OwnerUserId="40178" LastEditorUserId="40178" LastEditDate="2020-08-13T20:59:33.023" LastActivityDate="2020-08-13T20:59:33.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22934" PostTypeId="1" CreationDate="2020-08-09T00:38:50.303" Score="1" ViewCount="16" Body="&lt;p&gt;I am currently working with some models aimed at predicting time series (89 days for training, 22 for testing), including a CNN LSTM and a convLSTM.&lt;/p&gt;&#xA;&lt;p&gt;When training these models, I had the following scenario:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0ZyyJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0ZyyJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TjaXN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TjaXN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the first case, it is possible to see the val loss moving more sharply away from the train loss. In the second case, it seems to me that this also happens, but in a much smoother way.&lt;/p&gt;&#xA;&lt;p&gt;What do these graphs mean? What causes these situations to occur? If they are problematic situations, is it possible to correct them? If so, how?&lt;/p&gt;&#xA;" OwnerUserId="38864" LastActivityDate="2020-08-09T00:38:50.303" Title="Understanding graphs of the mean square error: relationships between val loss and train loss" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22935" PostTypeId="2" ParentId="22930" CreationDate="2020-08-09T02:01:47.573" Score="1" Body="&lt;p&gt;You may want to take a look at &lt;a href=&quot;https://towardsdatascience.com/identifying-the-right-meaning-of-the-words-using-bert-817eef2ac1f0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, but I'll summarize. You can use BERT (or some other tool) to make embeddings of every word in every sentence. Then for each word, make a contextualized embedding vector using the rest of the sentence. &lt;a href=&quot;https://github.com/negedng/bert-embedding/tree/master/bert_embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;bert-embedding&lt;/a&gt; does all of this itself. Then keep the embedding vector for the important words.&lt;/p&gt;&#xA;&lt;p&gt;For each important word, you would then have two pieces of information: the embedding vector and the correct label (which could easily be made into an integer from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;). Depending on the size of the embedding vectors, you could use PCA to reduce the size, although this may not be needed. Using this data, you can then train a neural network or use a k-nearest neighbors classifier.&lt;/p&gt;&#xA;&lt;p&gt;There is more information in the article, which I suggest that you read. They do a better job explaining than me and also have some actual code you may want to look at.&lt;/p&gt;&#xA;" OwnerUserId="40178" LastEditorUserId="40178" LastEditDate="2020-08-09T02:11:21.803" LastActivityDate="2020-08-09T02:11:21.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22936" PostTypeId="1" CreationDate="2020-08-09T03:14:05.647" Score="0" ViewCount="66" Body="&lt;p&gt;Can a computer solve the following problem, i.e. make a proof by induction? And why?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Prove by &lt;strong&gt;induction&lt;/strong&gt; that &lt;span class=&quot;math-container&quot;&gt;$$\sum_{k=1}^nk^3=\left(\frac{n(n+1)}{2}\right)^2, \, \, \, \forall n\in\mathbb N .$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm doing a Ph.D. in pure maths. I love coding when I wanna have some fun, but I've never got too far in this field. I say my background because maybe there's someone who wants to explain this in a more abstract language there's a chance that I will understand it.&lt;/p&gt;&#xA;" OwnerUserId="40179" LastEditorUserId="2444" LastEditDate="2020-08-09T10:41:45.833" LastActivityDate="2020-08-09T11:24:00.577" Title="Can a computer make a proof by induction?" Tags="&lt;math&gt;&lt;proofs&gt;&lt;automated-theorem-proving&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22937" PostTypeId="2" ParentId="22931" CreationDate="2020-08-09T06:19:23.777" Score="2" Body="&lt;p&gt;&lt;strong&gt;Reinforcement Learning: An Introduction&lt;/strong&gt; by &lt;em&gt;Richard Sutton and Andrew Barto&lt;/em&gt; is undoubtedly one of the best books, to begin with. Despite its age, the book is still the canonical introduction to reinforcement learning. It does require some patience, but I think it's very approachable and rigorous at the same time!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-08-09T06:19:23.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22938" PostTypeId="1" CreationDate="2020-08-09T09:13:51.313" Score="0" ViewCount="22" Body="&lt;p&gt;I trained a model with my data in the range (0, 50000) and I used the &lt;em&gt;mean absolute percentage error (MAPE)&lt;/em&gt; as my metric. Now, the model gave me MAPE &amp;gt; 10 million, but on my test data I have the following plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JuPKb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JuPKb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Could you explain to me which one (MAPE or results of prediction(plot)) is wrong? Or why this is happening?&lt;/p&gt;&#xA;" OwnerUserId="40183" LastEditorUserId="2444" LastEditDate="2020-08-09T23:40:45.383" LastActivityDate="2020-08-09T23:40:45.383" Title="Why MAPE (mean absolute percentage error) is so large?" Tags="&lt;neural-networks&gt;&lt;metric&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22939" PostTypeId="2" ParentId="11528" CreationDate="2020-08-09T09:58:11.827" Score="0" Body="&lt;p&gt;GA is well suited to optimize a non-linear fitness function with a lot of variables. Each vector that is a possible solution is evaluated with the fitness function that we want to optimize.&lt;/p&gt;&#xA;&lt;p&gt;So, GA is well suited to optimize schedules for a lot of people, optimizing resources, etc.&lt;/p&gt;&#xA;&lt;p&gt;Your mission is to define all variables that you can move-tune and that affect your objectives, and assign an estimated weight on every variable.&lt;/p&gt;&#xA;&lt;p&gt;For example, create a total project cost as the fitness function, to minimize. One variable will be the number of programmers, multiplied by cost and total months. If you don't want delays over 3 months, add a correction factor: +10000*max(0, months-3), penalizing the fitness function's results where months &amp;gt; 3, etc.&lt;/p&gt;&#xA;&lt;p&gt;When you apply your GA algorithm to the fitness function, you will find some minimum for your function, if any exists, because the solution may not exist if you added too many constraints.&lt;/p&gt;&#xA;" OwnerUserId="25257" LastEditorUserId="2444" LastEditDate="2021-01-06T21:44:31.733" LastActivityDate="2021-01-06T21:44:31.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22940" PostTypeId="1" CreationDate="2020-08-09T10:15:42.783" Score="1" ViewCount="17" Body="&lt;p&gt;I wonder if there is research, patents, or libraries using Genetic algorithms (GA) to improve Neural Networks. I don't find anything in the subject. For example:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;use GA to find better parameters in a NN. So the chromosome will be [learning rate, activation function, layers number, layers size, dropout factor] and the fit function minimize computational cost to reach NN 95% accuracy.&lt;/li&gt;&#xA;&lt;li&gt;use GA to mix your NN input data and generate new data to adjust.&lt;/li&gt;&#xA;&lt;li&gt;use GA to mix several small NN, different types, and find the perfect mix for better predictions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25257" LastActivityDate="2020-08-09T10:15:42.783" Title="experiences on using genetic algorithms as a way to improve neural networks?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22941" PostTypeId="1" CreationDate="2020-08-09T10:31:23.317" Score="0" ViewCount="89" Body="&lt;p&gt;I am trying to code a MCTS agent for tic tac toe and i have some theoretical questions regarding MCTS.&lt;/p&gt;&#xA;&lt;p&gt;1)I am using the UCB1 MCTS &lt;span class=&quot;math-container&quot;&gt;$UCB(Si)=average value + 2*sqrt(ln(N)/ni)$&lt;/span&gt; .&lt;/p&gt;&#xA;&lt;p&gt;Considering the image below that describes a state diagram,i would like to ask if the UCB(s3)(for state S3) is equal to&#xA;&lt;span class=&quot;math-container&quot;&gt;$3/3 + 2*sqrt(ln(6)/3)$&lt;/span&gt;  (i used the total number N of iterations of So ) OR it is &lt;span class=&quot;math-container&quot;&gt;$3/3 + 2*sqrt(ln(4)/3)$&lt;/span&gt; (here i used the n1 total number of iterations of the parent node of s3,s4.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Og1U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Og1U.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;I am wondering what's happening if during the expansion phase of the UCB1 MCTS algorithm i find a state that already exists in the tree. For example: Consider again the image (class diagram below) .During expansion of S1 i find again State s2 .What should i do in this case ? Should i treat s2 as i already visited it and copy its information or treat it like a completely new state ? And when i propagate it after the rollout should i update its stats everywhere?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/glnz7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/glnz7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;3)I would like to ask  specifically for tic-tac-toe what would be ideal number for the evaluation function.For example considering a ucb1 monte carlo UCB(si)&lt;span class=&quot;math-container&quot;&gt;$=average value + 2*sqrt(ln(N)/ni)$&lt;/span&gt; what would be typical numbers for a rollout for a win , a lost and a draw for each player 'x','o'? Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="40116" LastEditorUserId="40116" LastEditDate="2020-08-09T11:00:42.673" LastActivityDate="2020-08-09T11:00:42.673" Title="Monte Carlo Tree search UCB1 for Tic-Tac-Toe help" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22943" PostTypeId="1" CreationDate="2020-08-09T15:22:31.803" Score="0" ViewCount="21" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;&lt;strong&gt;The basic idea of TD methods are to make state-next state pairs fit the constraints of the Bellman equation on average.&lt;/strong&gt;&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is this statement true? If yes, why, and if not, why not? I'm not able to make complete sense of it or analyze it rigorously. I would appreciate any help!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://web.stanford.edu/class/cs234/slides/lecture5_post.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Slide 2/55&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-08-10T02:45:28.670" LastActivityDate="2020-08-10T02:45:28.670" Title="Do TD methods involve $(s,s')$ pairs fitting the Bellman equation on average?" Tags="&lt;reinforcement-learning&gt;&lt;temporal-difference-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22944" PostTypeId="1" AcceptedAnswerId="22947" CreationDate="2020-08-09T15:35:20.917" Score="4" ViewCount="249" Body="&lt;p&gt;Here's another interesting multiple-choice question that puzzles me a bit.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In tabular MDPs, if using a decision policy that visits all states an &lt;strong&gt;infinite&lt;/strong&gt; number of times, and in each state, &lt;strong&gt;randomly&lt;/strong&gt; selects an action, then:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Q-learning will converge to the optimal Q-values&lt;/li&gt;&#xA;&lt;li&gt;SARSA will converge to the optimal Q-values&lt;/li&gt;&#xA;&lt;li&gt;Q-learning is learning off-policy&lt;/li&gt;&#xA;&lt;li&gt;SARSA is learning off-policy&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;My thoughts, and question:&lt;/em&gt; Since the actions are being sampled randomly from the action space, learning definitely seems to be &lt;strong&gt;off-policy&lt;/strong&gt; (correct me if I'm wrong, please!). So that rules 3. and 4. as incorrect. Coming to the first two options, I'm not quite sure whether Q-learning and/or SARSA would converge in this case. All that I'm able to understand from the question is that the agent &lt;strong&gt;explores more than it exploits&lt;/strong&gt;, since it visits all states (an infinite number of times) and also takes random actions (and not the best action!). How can this piece of information help me deduce if either process converges to the optimal Q-values or not?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://web.stanford.edu/class/cs234/slides/lecture5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Slide 2/55&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-08-09T17:48:43.087" Title="When do SARSA and Q-Learning converge to optimal Q values?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;convergence&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22945" PostTypeId="1" AcceptedAnswerId="22965" CreationDate="2020-08-09T15:50:26.347" Score="1" ViewCount="335" Body="&lt;p&gt;I am trying to implement Deep Deterministic policy gradient algorithm by referring to the paper &lt;em&gt;Continuous Control using Deep Reinforcement Learning&lt;/em&gt; on the MountainCarContinuous-v0 gym environment. I am using 2 hidden Linear layers of size 32 for both the actor and the critic networks with ReLU activations and a Tanh activation for the output layer of the actor network. However, for some reason, algorithm doesn't seem to converge for some reason. I tried tuning the hyperparameters to no success.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Code&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;import copy&#xA;import random&#xA;from collections import deque, namedtuple&#xA;&#xA;import matplotlib.pyplot as plt&#xA;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Hyperparameters:&#xA;&#xA;actor_layer_sizes&#xA;critic_layer_sizes&#xA;max_buffer_size&#xA;polyak_constant&#xA;max_time_steps&#xA;max_episodes&#xA;actor_lr&#xA;critic_lr&#xA;GAMMA&#xA;update_after&#xA;batch_size&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;device = torch.device(&amp;quot;cpu&amp;quot;)&#xA;dtype = torch.double&#xA;&#xA;Transition = namedtuple(&#xA;    &amp;quot;Transition&amp;quot;, (&amp;quot;state&amp;quot;, &amp;quot;action&amp;quot;, &amp;quot;reward&amp;quot;, &amp;quot;next_state&amp;quot;, &amp;quot;done&amp;quot;)&#xA;)&#xA;&#xA;&#xA;class agent:&#xA;    def __init__(&#xA;        self,&#xA;        env,&#xA;        actor_layer_sizes=[32, 32],&#xA;        critic_layer_sizes=[32, 32],&#xA;        max_buffer_size=2500,&#xA;    ):&#xA;        self.env = env&#xA;        (&#xA;            self.actor,&#xA;            self.critic,&#xA;            self.target_actor,&#xA;            self.target_critic,&#xA;        ) = self.make_models(actor_layer_sizes, critic_layer_sizes)&#xA;        self.replay_buffer = deque(maxlen=max_buffer_size)&#xA;        self.max_buffer_size = max_buffer_size&#xA;&#xA;    def make_models(self, actor_layer_sizes, critic_layer_sizes):&#xA;        actor = (&#xA;            nn.Sequential(&#xA;                nn.Linear(&#xA;                    self.env.observation_space.shape[0],&#xA;                    actor_layer_sizes[0],&#xA;                ),&#xA;                nn.ReLU(),&#xA;                nn.Linear(actor_layer_sizes[0], actor_layer_sizes[1]),&#xA;                nn.ReLU(),&#xA;                nn.Linear(&#xA;                    actor_layer_sizes[1], self.env.action_space.shape[0]&#xA;                ), nn.Tanh()&#xA;            )&#xA;            .to(device)&#xA;            .to(dtype)&#xA;        )&#xA;&#xA;        critic = (&#xA;            nn.Sequential(&#xA;                nn.Linear(&#xA;                    self.env.observation_space.shape[0]&#xA;                    + self.env.action_space.shape[0],&#xA;                    critic_layer_sizes[0],&#xA;                ),&#xA;                nn.ReLU(),&#xA;                nn.Linear(critic_layer_sizes[0], critic_layer_sizes[1]),&#xA;                nn.ReLU(),&#xA;                nn.Linear(critic_layer_sizes[1], 1),&#xA;            )&#xA;            .to(device)&#xA;            .to(dtype)&#xA;        )&#xA;&#xA;        target_actor = copy.deepcopy(actor)    # Create a target actor network&#xA;&#xA;        target_critic = copy.deepcopy(critic)   # Create a target critic network&#xA;&#xA;        return actor, critic, target_actor, target_critic&#xA;&#xA;    def select_action(self, state, noise_factor):         # Selects an action in exploratory manner&#xA;      with torch.no_grad():&#xA;        noisy_action = self.actor(state) + noise_factor * torch.randn(size = self.env.action_space.shape, device=device, dtype=dtype)&#xA;        action = torch.clamp(noisy_action, self.env.action_space.low[0], self.env.action_space.high[0])&#xA;&#xA;        return action&#xA;&#xA;    def store_transition(self, state, action, reward, next_state, done):             # Stores the transition to the replay buffer with a default maximum capacity of 2500&#xA;        if len(self.replay_buffer) &amp;lt; self.max_buffer_size:&#xA;            self.replay_buffer.append(&#xA;                Transition(state, action, reward, next_state, done)&#xA;            )&#xA;        else:&#xA;            self.replay_buffer.popleft()&#xA;            self.replay_buffer.append(&#xA;                Transition(state, action, reward, next_state, done)&#xA;            )&#xA;&#xA;    def sample_batch(self, batch_size=128):                                            # Samples a random batch of transitions for training&#xA;      return Transition(&#xA;            *[torch.cat(i) for i in [*zip(*random.sample(self.replay_buffer, min(len(self.replay_buffer), batch_size)))]]&#xA;        )&#xA;&#xA;&#xA;    def train(&#xA;        self,&#xA;        GAMMA=0.99,&#xA;        actor_lr=0.001,&#xA;        critic_lr=0.001,&#xA;        polyak_constant=0.99,&#xA;        max_time_steps=5000,&#xA;        max_episodes=200,&#xA;        update_after=1,&#xA;        batch_size=128,&#xA;        noise_factor=0.2,&#xA;    ):&#xA;        &#xA;        self.train_rewards_list = []&#xA;        actor_optimizer = optim.Adam(self.actor.parameters(), lr=actor_lr)&#xA;        critic_optimizer = optim.Adam(&#xA;            self.critic.parameters(), lr=critic_lr&#xA;        )&#xA;        print(&amp;quot;Starting Training:\n&amp;quot;)&#xA;        for e in range(max_episodes):&#xA;            state = self.env.reset()&#xA;            state = torch.tensor(state, device=device, dtype=dtype).unsqueeze(0)&#xA;            episode_reward = 0&#xA;            for t in range(max_time_steps):&#xA;                #self.env.render()&#xA;                action = self.select_action(state, noise_factor)               &#xA;                next_state, reward, done, _ = self.env.step(action[0])         # Sample a transition&#xA;                episode_reward += reward&#xA;&#xA;                next_state = torch.tensor(next_state, device=device, dtype=dtype).unsqueeze(0)&#xA;                reward = torch.tensor(&#xA;                    [reward], device=device, dtype=dtype&#xA;                ).unsqueeze(0)&#xA;                done = torch.tensor(&#xA;                    [done], device=device, dtype=dtype&#xA;                ).unsqueeze(0)&#xA;&#xA;                self.store_transition(                               &#xA;                    state, action, reward, next_state, done&#xA;                )                # Store the transition in the replay buffer&#xA;&#xA;                state = next_state&#xA;                &#xA;                sample_batch = self.sample_batch(128)&#xA;&#xA;                with torch.no_grad():                 # Determine the target for the critic to train on&#xA;                  target = sample_batch.reward + (1 - sample_batch.done) * GAMMA * self.target_critic(torch.cat((sample_batch.next_state, self.target_actor(sample_batch.next_state)), dim=1))&#xA;                &#xA;                # Train the critic on the sampled batch&#xA;                critic_loss = nn.MSELoss()(&#xA;                    target,&#xA;                    self.critic(&#xA;                        torch.cat(&#xA;                            (sample_batch.state, sample_batch.action), dim=1&#xA;                        )&#xA;                    ),&#xA;                )&#xA;&#xA;                critic_optimizer.zero_grad()&#xA;                critic_loss.backward()&#xA;                critic_optimizer.step()&#xA;&#xA;                actor_loss = -1 * torch.mean(&#xA;                  self.critic(torch.cat((sample_batch.state, self.actor(sample_batch.state)), dim=1))&#xA;                  )&#xA;&#xA;                #Train the actor  &#xA;                actor_optimizer.zero_grad()&#xA;                actor_loss.backward()&#xA;                actor_optimizer.step()&#xA;                &#xA;&#xA;                #if (((t + 1) % update_after) == 0):&#xA;                for actor_param, target_actor_param in zip(self.actor.parameters(), self.target_actor.parameters()):&#xA;                  target_actor_param.data = polyak_constant * actor_param.data + (1 - polyak_constant) * target_actor_param.data&#xA;                  &#xA;                for critic_param, target_critic_param in zip(self.critic.parameters(), self.target_critic.parameters()):&#xA;                  target_critic_param.data = polyak_constant * critic_param.data + (1 - polyak_constant) * target_critic_param.data&#xA;&#xA;                if done:&#xA;                    print(&#xA;                        &amp;quot;Completed episode {}/{}&amp;quot;.format(&#xA;                            e + 1, max_episodes&#xA;                        )&#xA;                    )&#xA;                    break&#xA;&#xA;            self.train_rewards_list.append(episode_reward)&#xA;&#xA;        self.env.close()&#xA;        print(self.train_rewards_list)&#xA;&#xA;    def plot(self, plot_type):&#xA;        if (plot_type == &amp;quot;train&amp;quot;):&#xA;            plt.plot(self.train_rewards_list)&#xA;            plt.show()&#xA;        elif (plot_type == &amp;quot;test&amp;quot;):&#xA;            plt.plot(self.test_rewards_list)&#xA;            plt.show()&#xA;        else:&#xA;            print(&amp;quot;\nInvalid plot type&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Train code snippet&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gym&#xA;&#xA;env = gym.make(&amp;quot;MountainCarContinuous-v0&amp;quot;)&#xA;&#xA;myagent = agent(env)&#xA;myagent.train(max_episodes=150)&#xA;myagent.plot(&amp;quot;train&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The figure below shows the plot for episode reward vs episode number:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nipus.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nipus.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38895" LastEditorUserId="38895" LastEditDate="2020-08-09T17:45:15.830" LastActivityDate="2020-08-10T09:17:48.393" Title="DDPG doesn't converge for MountainCarContinuous-v0 gym environment" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;policy-gradients&gt;&lt;ddpg&gt;&lt;gym&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22946" PostTypeId="2" ParentId="22877" CreationDate="2020-08-09T15:56:44.757" Score="1" Body="&lt;p&gt;I think it is premature to answer your question as OpenAI has not made GPT-3 available yet other than via a web-based API.  For more information see &lt;a href=&quot;https://openai.com/blog/openai-api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI API&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://www.theverge.com/2020/6/11/21287966/openai-commercial-product-text-generation-gpt-3-api-customers&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI will start selling its text-generation tech, and the first customers include Reddit&lt;/a&gt;, by James Vincent:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Access to the GPT-3 API is invitation-only, and pricing is undecided.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can join the OpenAI wait list here: &lt;a href=&quot;https://beta.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://beta.openai.com/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I read somewhere that to load GPT-3 for inferencing requires 300GB if using half-precision floating point (FP16).  There are no GPU cards today that even in a set of four will provide 300GB of video RAM.  For example, the best I believe you can do in a single desktop box is four NVLinked Nvidia RTX 8000 cards on a single motherboard.  Each card has 48GB of VRAM each.  That would only provide a total of 192GB of VRAM.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-08-09T16:30:45.497" LastActivityDate="2020-08-09T16:30:45.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22947" PostTypeId="2" ParentId="22944" CreationDate="2020-08-09T17:48:43.087" Score="4" Body="&lt;p&gt;The true answers are 1 and 3. 1 because the required conditions for tabular Q-learning to converge is that each state action pair will be visited infinitely often, and Q-learning learns directly about the greedy policy, &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s) := \arg \max_a Q_\pi(s,a)$&lt;/span&gt;,  and because Q-learning converges to the optimal Q-value function we know that the policy will be optimal (because the optimal policy is the greedy policy wrt the optimal Q-function).&lt;/p&gt;&#xA;&lt;p&gt;3 is true because Q-learning is by definition an off-policy algorithm, because we learn about the greedy policy whilst following some arbitrary policy.&lt;/p&gt;&#xA;&lt;p&gt;2 is false because SARSA is on-policy, so it will be learning the Q-function under the random policy, and 3 is false because SARSA is strictly on-policy, for reasons analogous to why Q-learning is off-policy.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-09T17:48:43.087" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22948" PostTypeId="2" ParentId="22900" CreationDate="2020-08-09T17:50:43.570" Score="4" Body="&lt;p&gt;If you have multiple types of rewards (say, R1 and R2), then it is no longer clear what would be the optimal way to act: it can happen that one way of acting would maximize R1 and another way would maximize R2.  Therefore, optimal policies, value functions, etc., would all be undefined.  Of course, you could say that you want to maximize, for example, R1+R2, or 2R1+R2, etc.  But in that case, you're back at a scalar number again.&lt;/p&gt;&#xA;&lt;p&gt;It can still be helpful for other purposes to split up the reward into multiple components as you suggest, e.g., in a setup where you need to learn to predict these rewards.  But for the purpose of determining optimal actions, you need to boil it down into a single scalar.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2020-08-09T17:50:43.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22949" PostTypeId="1" CreationDate="2020-08-09T19:15:23.720" Score="2" ViewCount="18" Body="&lt;p&gt;Consider a problem where the agent must learn to control a hierarchy of agents acting against another such agent in a competitive environment.  The agents on each team need to learn cooperate in order to compete with the other agents.&lt;/p&gt;&#xA;&lt;p&gt;A hierarchical RL algorithm would seem to be ideal for such a problem, learning a policy that includes sub-policies for sub-agents.  But are there are other types of algorithms that could be used for this kind of task, perhaps ones that are involved centralized cooperation but aren't considered hierarchical RL?&lt;/p&gt;&#xA;" OwnerUserId="40189" LastActivityDate="2020-08-09T19:15:23.720" Title="Alternatives to Hierarchical RL for centralized control tasks?" Tags="&lt;reinforcement-learning&gt;&lt;learning-algorithms&gt;&lt;hierarchical-rl&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22950" PostTypeId="2" ParentId="22782" CreationDate="2020-08-09T19:31:04.437" Score="0" Body="&lt;p&gt;The clusters you mentioned at the beginning seem right. A &amp;quot;perfect&amp;quot; coreference system should be able to find all words/phrases that refer to the same object.&lt;/p&gt;&#xA;&lt;p&gt;However, in language, there will always be some ambiguity. For example, consider the sentences &amp;quot;Bob and John like building. He has fun doing it.&amp;quot;. It is ambiguous whether &amp;quot;He&amp;quot; refers to &amp;quot;Bob&amp;quot; or &amp;quot;John&amp;quot;. In that sense, a perfect coreference system may not even exist theoretically. If it did, it would either avoid classifying &amp;quot;He&amp;quot; or recognize that it could go with either &amp;quot;Bob&amp;quot; or &amp;quot;John&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="40178" LastActivityDate="2020-08-09T19:31:04.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22951" PostTypeId="1" CreationDate="2020-08-09T20:01:28.683" Score="1" ViewCount="69" Body="&lt;p&gt;I would like to add a short ~1-3 minute video to a presentation, to demonstrate how Reinforcement Learning is used to solve problems. I am thinking something like a short gif of an agent playing an Atari game, but for my audience it would probably be better to have something more manufacturing/industry based.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know any good sources where I could find some stuff like this?&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-09-13T17:00:50.090" Title="Where can I find short videos of examples of RL being used?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22954" PostTypeId="1" CreationDate="2020-08-09T21:58:00.803" Score="2" ViewCount="59" Body="&lt;p&gt;I know this may be a question of semantics but I always see different articles explain forward pass slightly different. e.g. Sometimes they represent a forward pass to a hidden layer in a standard neural network as &lt;code&gt;np.dot(x, W)&lt;/code&gt; and sometimes I see it as &lt;code&gt;np.dot(W.T, x)&lt;/code&gt; and sometimes &lt;code&gt;np.dot(W, x)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Take this image for example. They represent the input data as a matrix of &lt;code&gt;[NxD]&lt;/code&gt; and weight data as &lt;code&gt;[DxH]&lt;/code&gt; where H is the number of neurons in the hidden layer. This seems the most natural since input data will often be in tabular format with rows as samples and columns as features.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wUNV7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wUNV7.png&quot; alt=&quot;Example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now an example from the CS231n course notes. They talk about this below example and cite the code used to compute it as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f = lambda x: 1.0/(1.0 + np.exp(-x)) # activation function (use sigmoid)&#xA;x = np.random.randn(3, 1) # random input vector of three numbers (3x1)&#xA;h1 = f(np.dot(W1, x) + b1) # calculate first hidden layer activations (4x1)&#xA;h2 = f(np.dot(W2, h1) + b2) # calculate second hidden layer activations (4x1)&#xA;out = np.dot(W3, h2) + b3 # output neuron (1x1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &lt;code&gt;W&lt;/code&gt; is &lt;code&gt;[4x3]&lt;/code&gt; and x is &lt;code&gt;[3x1]&lt;/code&gt;. I would expect the weight matrix to have dimensions equal to [n_features, n_hidden_neurons] but in this example it just seems like they transposed it naturally before it was used.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FHJ5p.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FHJ5p.jpg&quot; alt=&quot;Example2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I guess I am just confused about general nomenclature in how data should be shaped and used consistently when computing neural network forward passes. Sometimes I see transpose, sometimes I don't. Is there a standard, preferred way to represent data in accordance to a diagram like these This question may be silly but I just wanted to discuss it a bit. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="40192" LastEditorUserId="2444" LastEditDate="2020-10-10T22:07:25.743" LastActivityDate="2020-10-10T22:07:25.743" Title="What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;feedforward-neural-network&gt;&lt;forward-pass&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22955" PostTypeId="1" CreationDate="2020-08-09T23:02:33.973" Score="1" ViewCount="15" Body="&lt;p&gt;I've been considering starting a project for some time on sound source identification.&lt;/p&gt;&#xA;&lt;p&gt;To be more specific, my goal is to be able to identify the &amp;quot;sources&amp;quot; for sound in videos. Moving parts clanging, lips speaking, hands clapping etc. I'd like to think a model trained to be able to do this might be helpful for:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Identifying who is saying what in a crowd&lt;/li&gt;&#xA;&lt;li&gt;Discovering noise sources caught on video (ex. a carpenter's saw as he is talking to someone)&lt;/li&gt;&#xA;&lt;li&gt;Extending this to design a model for reading lips, to discern speech in silent video.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EOs87.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EOs87.png&quot; alt=&quot;Helen Keller talking&quot; /&gt;&lt;/a&gt;&#xA;(Taken from &lt;a href=&quot;https://www.youtube.com/watch?v=8ch_H8pt9M8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=8ch_H8pt9M8&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;You might think of this task like grounding in NLP, expect for sound/speech instead. I'm sure this has been done before, and I'd like to conduct a literature review. So is there a name for this kind of sound-source identification?&lt;/p&gt;&#xA;&lt;p&gt;I've tried Googling &amp;quot;Sound Source Identification&amp;quot;, but it only returns Speech Classification results (Is this sound a car or a truck etc.)&lt;/p&gt;&#xA;" OwnerUserId="35768" LastActivityDate="2020-08-09T23:02:33.973" Title="Is there a problem for &quot;Sound Source Identification in Video Footage&quot;?" Tags="&lt;classification&gt;&lt;computer-vision&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22956" PostTypeId="2" ParentId="22954" CreationDate="2020-08-10T03:54:05.653" Score="1" Body="&lt;p&gt;I don't think there's a &amp;quot;standard way&amp;quot; of expressing the forward pass: you use the transpose when you need to use it, and this depends on how you define the weights and inputs matrices, and on the architecture of your neural network. For example, in a fully connected feedforward neural network, you know that every neuron in the previous layer is connected to every neuron in the current layer, so, as long as this is satisfied when you multiply the matrices, it does not matter whether you use transposes or not, and I don't think that, in computational terms, it makes any difference if you use transposes or not. (By the way, if you are writing something, I suggest that you always specify the dimensions of your matrices and your conventions).&lt;/p&gt;&#xA;&lt;p&gt;Of course, if you want to use a library like TensorFlow, you will probably need to follow the conventions of the library, but this is another story.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-08-10T03:54:05.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22957" PostTypeId="1" AcceptedAnswerId="22960" CreationDate="2020-08-10T04:01:13.780" Score="10" ViewCount="1409" Body="&lt;p&gt;The transformer, introduced in the paper &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt;, is a popular new neural network architecture that is commonly viewed as an alternative to recurrent neural networks, like LSTMs and GRUs.&lt;/p&gt;&#xA;&lt;p&gt;However, having gone through the paper, as well as several online explanations, I still have trouble wrapping my head around how they work. How can a non-recurrent structure be able to deal with inputs of arbitrary length?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditDate="2020-09-17T13:56:31.700" LastActivityDate="2020-09-17T13:56:31.700" Title="How can Transformers handle arbitrary length input?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22958" PostTypeId="2" ParentId="22931" CreationDate="2020-08-10T04:07:44.707" Score="0" Body="&lt;p&gt;If you are looking for a book that is more beginner friendly than the Sutton and Barto book (which you should of course check out also), try out:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1838826998&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning Hands On&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12201" LastActivityDate="2020-08-10T04:07:44.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22959" PostTypeId="1" AcceptedAnswerId="22991" CreationDate="2020-08-10T05:26:52.783" Score="4" ViewCount="121" Body="&lt;p&gt;I recently read some introductions to AI alignment, AIXI and decision theory things.&lt;/p&gt;&#xA;&lt;p&gt;As far as I understood, one of the main problems in &lt;em&gt;AI alignment&lt;/em&gt; is how to define a &lt;em&gt;utility function&lt;/em&gt; well, not causing something like the paperclip apocalypse.&lt;/p&gt;&#xA;&lt;p&gt;Then a question comes to my mind that whatever the utility function would be, we need a computer to compute the utility and reward, so that there is no way to prevent AGI from seeking it to manipulate the utility function to always give the maximum reward.&lt;/p&gt;&#xA;&lt;p&gt;Just like we humans know that we can give happiness to ourselves in chemical ways and some people actually do so.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to prevent this from happening? Not just protecting the utility calculator physically from AGI (How can we sure it works forever?), but preventing AGI from thinking of it?&lt;/p&gt;&#xA;" OwnerUserId="40196" LastEditorUserId="2444" LastEditDate="2020-08-12T23:37:42.983" LastActivityDate="2020-08-12T23:37:42.983" Title="How can we prevent AGI from doing drugs?" Tags="&lt;reinforcement-learning&gt;&lt;agi&gt;&lt;rewards&gt;&lt;aixi&gt;&lt;reward-hacking&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22960" PostTypeId="2" ParentId="22957" CreationDate="2020-08-10T05:43:22.993" Score="7" Body="&lt;p&gt;Actually, there is usually an upper bound for inputs of transformers, due to the inability of handling long-sequence. Usually, the value is set as 512 or 1024 at current stage.&lt;/p&gt;&#xA;&lt;p&gt;However, if you are asking handling the various input size, adding padding token such as [PAD] in BERT model is a common solution. The position of [PAD] token could be masked in self-attention, therefore, causes no influence. Let's say we use a transformer model with 512 limit of sequence length, then we pass a input sequence of 103 tokens. We padded it to 512 tokens. In the attention layer, positions from 104 to 512 are all masked, that is, they are not attending or being attended.&lt;/p&gt;&#xA;" OwnerUserId="38960" LastActivityDate="2020-08-10T05:43:22.993" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22961" PostTypeId="1" CreationDate="2020-08-10T05:48:21.117" Score="2" ViewCount="40" Body="&lt;p&gt;I have been reading more about computer vision and I'm bothered by YOLO and similar deep learning architectures.&lt;/p&gt;&#xA;&lt;p&gt;The thing I am confused about is how non-class image sections are dealt with. In particular, it's not clear to me at all why YOLO doesn't consider every part of an image a possible class.&lt;/p&gt;&#xA;&lt;p&gt;What actually sets the cutoff for detection and then classification?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2021-01-29T00:03:17.817" LastActivityDate="2021-02-28T03:02:50.377" Title="How does YOLO handle non-class objects?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22962" PostTypeId="1" CreationDate="2020-08-10T06:06:56.967" Score="2" ViewCount="63" Body="&lt;p&gt;I am new to reinforcement learning, but, for a finite horizon application problem, I am considering using the &lt;em&gt;average reward&lt;/em&gt; instead of the &lt;em&gt;sum of rewards&lt;/em&gt; as the objective. Specifically, there are a total of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; maximally possible time steps (e.g., the usage rate of an app in each time-step), in each time-step, the reward may be 0 or 1. The goal is to maximize the daily average usage rate.&lt;/p&gt;&#xA;&lt;p&gt;Episode length (&lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;) is maximally 10. &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is the maximum time window the product can observe about a user's behavior of the chosen data. There is an indicator value in the data indicating whether an episode terminates. From the data, it is offline learning, so in each episode, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is given in the data. As long as an episode doesn't terminate, there is a reward of &lt;span class=&quot;math-container&quot;&gt;$\{0, 1\}$&lt;/span&gt; in each time-step.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I heard if I use an average reward for the finite horizon, the optimal policy is no longer a stationary policy, and optimal &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; function depends on time. I am wondering why this is the case.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I see normally, the objective is defined maximizing&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_t^T \gamma^t r_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And I am considering two types of average reward definition.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$1/T(\sum^_{t=0}\gamma^t r_t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; varies is in each episode.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$1/(T-t)\sum^T_{i=t-1}\gamma^i r_i$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40193" LastEditorUserId="2444" LastEditDate="2020-08-11T10:34:50.887" LastActivityDate="2020-08-11T10:34:50.887" Title="Should I use the discounted average reward as objective in a finite-horizon problem?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;rewards&gt;&lt;stationary-policy&gt;" AnswerCount="0" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22963" PostTypeId="1" AcceptedAnswerId="22973" CreationDate="2020-08-10T08:05:12.857" Score="3" ViewCount="235" Body="&lt;p&gt;Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem.&lt;/p&gt;&#xA;&lt;p&gt;In the Transfer Learning, we take layers from a previously trained model and freeze them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why is this layer freezing required and what are the effects of layer freezing?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="30725" LastActivityDate="2020-09-22T22:10:23.127" Title="What is layer freezing in transfer learning?" Tags="&lt;transfer-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22964" PostTypeId="2" ParentId="22959" CreationDate="2020-08-10T08:31:58.227" Score="1" Body="&lt;p&gt;You make a lot of assumptions about AGI, namely that 'we need a computer to compute the utility and reward AGI'. It not clear to me that (1) we can achieve AGI, (2) AGI will be on a computer as we know it and (3) AGI will work with a utility / reward function as we know them.&lt;/p&gt;&#xA;&lt;p&gt;One thing I am sure though is that ML is known for &amp;quot;cheating&amp;quot; (see for &lt;a href=&quot;https://arxiv.org/abs/2004.07780&quot; rel=&quot;nofollow noreferrer&quot;&gt;exemple&lt;/a&gt;). Avoiding such cheating is part of the building process. So, when you assume that we can achieve AGI, it means that you assume we can build an AGI that do not &amp;quot;cheat&amp;quot;. Thus, the answer is mostly contained in your assumptions.&lt;/p&gt;&#xA;&lt;p&gt;Whether we are able to build an AGI, what &amp;quot;cheating&amp;quot; we would have to overcome to do so and how we will be able to do so is mostly undetermined.&lt;/p&gt;&#xA;" OwnerUserId="22654" LastActivityDate="2020-08-10T08:31:58.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22965" PostTypeId="2" ParentId="22945" CreationDate="2020-08-10T09:17:48.393" Score="1" Body="&lt;p&gt;I had to change the actions selection function for this and tune some hyper-parameters. Here's what I did to make it converge:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sampled the noise from a standard normal distribution instead of sampling randomly.&lt;/li&gt;&#xA;&lt;li&gt;Changed the polyak constant (tau) from 0.99 to 0.001 (I didn't have an idea of what it should be, so I had just set it randomly in the first try)&lt;/li&gt;&#xA;&lt;li&gt;Changed the hidden layer sizes of the critic network to [64, 64].&lt;/li&gt;&#xA;&lt;li&gt;Removed the ReLU activation after the second layer in the critic network. Earlier the layer were stacked as (Linear, ReLU, Linear, ReLU, Linear). I changed it to (Linear, ReLU, Linear, Linear).&lt;/li&gt;&#xA;&lt;li&gt;Changed max buffer size to 1000000&lt;/li&gt;&#xA;&lt;li&gt;Changed the size of the batch_size to be sampled from 128 to 64&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is the plot that I get now after training it for 75 episodes :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q84yc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q84yc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38895" LastActivityDate="2020-08-10T09:17:48.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22966" PostTypeId="1" CreationDate="2020-08-10T12:13:35.000" Score="1" ViewCount="33" Body="&lt;p&gt;I was reading the book &amp;quot;Reinforcement Learning: An Introduction&amp;quot; by Sutton and Barto. In section 7.3, they write the formula for n-step off-policy TD as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(S_t) = V(S_{t-1}) + \alpha \rho_{t:t+n-1}[G_{t:t+n} - V(S_{t-1})],$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$V(S_{t})$&lt;/span&gt; is state value function of the state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ G_{t:t+n} \doteq \sum_{i=t}^{t+n-1}\gamma^{i-t}R_{i+1} + \gamma^n V(S_{t+n})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_{t:t+n-1}$&lt;/span&gt; is the importance sampling ratio.&lt;/p&gt;&#xA;&lt;p&gt;I tried to prove this equation for &lt;span class=&quot;math-container&quot;&gt;$n = 1$&lt;/span&gt; using the incremental update of the value function. Now I end up with this formula:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(S_t) = \frac{1}{t} \sum_{j=1}^{t} \rho_{j}G_{j} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(S_t)= \frac{1}{t}(\rho_{t}G_{t} + \sum_{j=1}^{t-1}\rho_{j}G_{j}) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(S_t) = \frac{1}{t}(\rho_t G_t + (t-1)V(S_{t-1}))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(S_t)=V(S_{t-1}) + \frac{1}{t}(\rho_{t}G_{t} - V(S_{t-1}))$$&lt;/span&gt;&#xA;I know I'm wrong because this does not match with the above equation. But can anyone please show me where I am wrong?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-11-05T22:39:51.907" LastActivityDate="2020-11-05T22:39:51.907" Title="How can I derive n-step off-policy temporal difference formula?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;temporal-difference-methods&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22967" PostTypeId="1" CreationDate="2020-08-10T13:04:08.283" Score="0" ViewCount="42" Body="&lt;p&gt;I playing around with a stock trading agent trained via (deep) reinforcement learning, including memory replay. The agent is trained for 1000 episodes, where each episodes consists of 180 timesteps (e.g. daily stock prices). My question is concerning the sampling of episodes for training. Assuming I've got daily stock prices going back 3 years, that's about 750 trading days/prices. How should I sample this data set to get enough episodes for training. With a episode length of 180 and an episode count of 1000 I'd need 180k &amp;quot;days&amp;quot; to choose from, if I wouldn't want any duplication. Do I even need to sample 1000 non-overlapping windows from my dataset or can I sample my episodes using a sliding window approach? Could I even just randomly sample the dataset for episodes? E.g. calculate a random date and build the episode from the 180 days following that random starting date?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;The reward for each action is calculated as follows, &lt;code&gt;p&lt;/code&gt; are the prices and &lt;code&gt;t&lt;/code&gt; the current timestep of the episode:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Rewards per action:&#xA;- CASH: 0&#xA;- BUY:  p(t+1) - p(t) - fee&#xA;- HOLD: p(t+1) - p(t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40202" LastEditorUserId="40202" LastEditDate="2020-08-10T16:00:50.780" LastActivityDate="2020-08-10T16:00:50.780" Title="Select training data for episodic reinforcement learning (stock trading agent)" Tags="&lt;training&gt;&lt;deep-rl&gt;&lt;algorithmic-trading&gt;" AnswerCount="0" CommentCount="10" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22969" PostTypeId="1" AcceptedAnswerId="23079" CreationDate="2020-08-10T13:42:03.550" Score="1" ViewCount="147" Body="&lt;p&gt;I am trying to determine the complexity of the neural network we use. The neural network is a U-net generator with an input shape of NxN (not an image but image-like data) and output of the same shape. There is 7x downsampling and 7x upsampling. Downsampling is a simple convolutional layer, where I have no problem to determine complexity as &lt;a href=&quot;https://arxiv.org/pdf/1412.1710.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;stated here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;O\left(\sum_{l=1}^{d} n_{l-1} \cdot s_{l}^{2} \cdot n_{l} \cdot m_{l}^{2}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I however cannot find what is big O complexity for the upsampling stage, where the UpSampling2D layer is used before convolution.&lt;/p&gt;&#xA;&lt;p&gt;Any idea what is the time complexity of the upsampling convolutional layer, or where I might find information? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="40205" LastEditorUserId="2444" LastEditDate="2020-08-10T14:56:07.100" LastActivityDate="2020-08-17T07:51:00.977" Title="What is the time complexity of the upsampling stage of the U-net?" Tags="&lt;convolutional-neural-networks&gt;&lt;time-complexity&gt;&lt;u-net&gt;&lt;computational-complexity&gt;&lt;pooling&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22970" PostTypeId="2" ParentId="22783" CreationDate="2020-08-10T14:38:29.897" Score="2" Body="&lt;p&gt;The inequality&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\left\|T^{\pi} V-T^{\pi} U\right\|_{\infty} &amp;amp; \leq \gamma\|V-U\|_{\infty} \label{1}\tag{1},&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; are two value functions, follows from the definition of &lt;em&gt;Bellman &lt;strong&gt;policy&lt;/strong&gt; operator&lt;/em&gt; (at &lt;a href=&quot;https://rlchina.org/lectures/lecture2.pdf#page=16&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide  16&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;T^{\pi} V(s) &#xA;&amp;amp;\triangleq &#xA;R(s, a)+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right) V\left(s^{\prime}\right) \\&#xA;&amp;amp;=R(s, \pi(s))+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s)\right) V\left(s^{\prime}\right), \; \forall s \in S &#xA;\tag{2}\label{2},&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\triangleq$&lt;/span&gt; means &amp;quot;defined as&amp;quot;. Note the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; in the definition, hence the name &lt;em&gt;Bellman &lt;strong&gt;policy&lt;/strong&gt; operator&lt;/em&gt; (BPO), and note that the BPO holds for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To prove (\ref{1}), first recall that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\left\|\mathbf {x} \right\|_{\infty } \triangleq \max _{i}\left|x_{i}\right|.$$&lt;/span&gt;&#xA;In the case of value functions &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\left\|V - U \right\|_{\infty } \triangleq \max_{s \in S}\left|V(s) - U(s) \right|.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note also that &lt;span class=&quot;math-container&quot;&gt;$Pr$&lt;/span&gt; is always non-negative (specifically, between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Successively, we expand the left-hand side of (\ref{1}) by applying the definition (\ref{2}) and using the properties just mentioned&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;{\scriptsize&#xA;\left\|T^{\pi} V-T^{\pi} U\right\|_{\infty}&#xA;= &#xA;\left\|&#xA;\left(&#xA;R(s, \pi(s))+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) V\left(s^{\prime}\right) \right) - \left( &#xA;R(s, \pi(s))+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) U\left(s^{\prime}\right) \right)&#xA;\right\|_{\infty} &#xA;}% &#xA;=\\&#xA;{\scriptsize&#xA;\max_{s \in S}&#xA;\left|&#xA;\left(&#xA;R(s, \pi(s))+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) V\left(s^{\prime}\right) \right) - \left( &#xA;R(s, \pi(s))+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) U\left(s^{\prime}\right) \right)&#xA;\right|&#xA;}% &#xA;= &#xA;\\&#xA;\max_{s \in S}&#xA;\left|&#xA;\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) V\left(s^{\prime}\right) - \gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) U\left(s^{\prime}\right)&#xA;\right|&#xA;= &#xA;\\&#xA;\gamma &#xA;\max_{s \in S}&#xA;\left|&#xA;\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) V\left(s^{\prime}\right) - \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s) \right) U\left(s^{\prime}\right)&#xA;\right|&#xA;= &#xA;\\&#xA;\gamma \max_{s \in S} \left|&#xA;\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s)\right) \left ( V\left(s^{\prime}\right) - U\left(s^{\prime}\right) \right)&#xA;\right|&#xA;= &#xA;\\&#xA;\gamma \max_{s \in S} &#xA;\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, \pi(s)\right) \left| V\left(s^{\prime}\right) - U\left(s^{\prime}\right) \right|&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, note that&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\max _{s^{\prime}}\left|V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right| = \| V - U \|_{_{\infty}}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha x + (1- \alpha) x = x$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\alpha \in [0, 1]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\left| V\left(s^{\prime}\right) - U\left(s^{\prime}\right) \right| \leq \max_{s' \in S} \left| V\left(s^{\prime}\right) - U\left(s^{\prime}\right) \right|$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: I am not completely sure about all these steps. I need to think about them a little bit more (but later). I encourage you to try to spot wrong derivations!!&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-10T14:43:55.663" LastActivityDate="2020-08-10T14:43:55.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22971" PostTypeId="2" ParentId="22783" CreationDate="2020-08-10T14:59:28.307" Score="0" Body="&lt;p&gt;I am assuming you are aware of the meaning of the notations. I will provide an informal explanation.&lt;/p&gt;&#xA;&lt;p&gt;From your comment I am guessing you have difficulty in this portion in the 1st equation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;{\scriptsize&#xA;\max_{s} \gamma \sum_{s^{\prime}} \operatorname{Pr}&#xA;\left(&#xA;s^{\prime} \mid s, \pi(s)&#xA;\right)&#xA;\left|&#xA;V\left(s^{\prime}\right) - U &#xA;\left(s^{\prime}\right)&#xA;\right| \\&#xA;\leq \gamma \left(\sum \operatorname{Pr} \left(s^{\prime} \mid s, \pi(s)\right)\right) \max _{s^{\prime}}\left|V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right| \\&#xA;\leq \gamma\|U-V\|_{\infty}&#xA;}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first inequality arises simply due to the fact that you are assigning a probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to the succesor state which has the maximum difference under the &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; value functions, whereas previously you wee maximizing the entire equation with respect to a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and hence certain probabilities get assigned to low value diiference states as well (i.e &lt;span class=&quot;math-container&quot;&gt;$|U(s') - V(s')|$&lt;/span&gt; is small compared to the largest value difference), whereas now you just pick the maximum difference between a succesor state, under the 2 value functions &lt;span class=&quot;math-container&quot;&gt;$V,U$&lt;/span&gt; and assign the entire probability to it i.e (&lt;span class=&quot;math-container&quot;&gt;$(\sum_{s'}Pr(s'|s, \pi(s))) = 1$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The second inequality is due to the fact, that now instead of selecting from a successor state, you select the maximum difference under the 2 value functions (&lt;span class=&quot;math-container&quot;&gt;$U(s),V(s)$&lt;/span&gt;) from the entire state space.&lt;/p&gt;&#xA;&lt;p&gt;In the 2nd equation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;{\scriptsize&#xA; \gamma \max _{s, a}\left|\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right)\left(V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right)\right| \\&#xA;\leq \gamma\left(\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right)\right) \max _{s^{\prime}}\left|\left(V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right)\right| \\&#xA;\leq &#xA;\gamma\|V-U\|_{\infty}&#xA;}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first inequality is again due to the same reasoning as above, that you assign the entire probability to the succesor state with highest value difference (under &lt;span class=&quot;math-container&quot;&gt;$U,V$&lt;/span&gt;) the maximum probability. And the second inequality is also due to the same reasoning as the 1st equation. You look for the maximum difference in the entire state space instead of just among successor states.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; In general succesor states can be the entire state space with those unreachable from state having &lt;span class=&quot;math-container&quot;&gt;$Pr(s'|s) = 0$&lt;/span&gt;, in that case the last inequality will become equality in both the equations.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-08-14T11:06:19.043" LastActivityDate="2020-08-14T11:06:19.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22973" PostTypeId="2" ParentId="22963" CreationDate="2020-08-10T20:05:45.707" Score="1" Body="&lt;p&gt;&lt;strong&gt;Why is this layer freezing required?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's not.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What are the effects of layer freezing?&lt;/strong&gt;&#xA;The consequences are:&lt;/p&gt;&#xA;&lt;p&gt;(1) Should be &lt;em&gt;faster&lt;/em&gt; to train (the gradient will have far less components)&lt;/p&gt;&#xA;&lt;p&gt;(2) Should require &lt;em&gt;less&lt;/em&gt; data to train on&lt;/p&gt;&#xA;&lt;p&gt;If you do unfreeze the weights, I'd think your performance would be better because you are adjusting (i.e., fine-tuning) the parameters to your specific problem at hand. I am not sure what the marginal improvements are in practice, as I have not experiemented much with fine-tuning (like are the improvements typically a 0.01% reduction in error rate? Not sure.)&lt;/p&gt;&#xA;" OwnerUserId="40213" LastActivityDate="2020-08-10T20:05:45.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22974" PostTypeId="1" CreationDate="2020-08-10T23:36:07.480" Score="1" ViewCount="55" Body="&lt;p&gt;The definition I see for representational capacity is &amp;quot;the family of functions the learning algorithm can choose from when varying the parameters in order to reduce a training objective.&amp;quot; (Goodfellow's Deep learning book).&lt;/p&gt;&#xA;&lt;p&gt;However, to me this seems to be the same as the definition of the hypothesis space. Is the key difference the &amp;quot;in order to reduce a training objective&amp;quot; in that some functions may not be chosen in reducing a training objective? Or are these identical definitions.&lt;/p&gt;&#xA;" OwnerUserId="40216" LastEditorUserId="2444" LastEditDate="2020-08-11T01:23:42.183" LastActivityDate="2020-08-11T01:23:42.183" Title="What is the representational capacity of a learning algorithm?" Tags="&lt;deep-learning&gt;&lt;terminology&gt;&lt;computational-learning-theory&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-08-12T23:17:21.687" ContentLicense="CC BY-SA 4.0" />
  <row Id="22975" PostTypeId="2" ParentId="22974" CreationDate="2020-08-11T00:51:13.947" Score="0" Body="&lt;p&gt;Learning algorithms (some others too, like search) aim at generating functions that get as close as possible to the &amp;quot;shape&amp;quot; of the training data (so we can then feed values to the generated functions and get outputs like, say, a prediction).&lt;/p&gt;&#xA;&lt;p&gt;In 2D, the &amp;quot;shape&amp;quot; may be easy to visualize. If the data in 2D seems to line up, learning algorithms generating linear/affine functions (e.g. &lt;code&gt;y = ax + b&lt;/code&gt;), should fit fairly well. Their representational capacity extends to lines. If the data seems to form a parabola, a representational capacity bound to lines will do poorly. We then need more &amp;quot;capable&amp;quot; representations, which can cope with the quadratic terms.&lt;/p&gt;&#xA;&lt;p&gt;So this should be what the &amp;quot;family of functions&amp;quot; Goodfellow's book refers to. The &amp;quot;training objective&amp;quot; should not be in the definition (to me), as a line fitting solution is unable to represent, say, a parabola, whatever the training objective is. However the book definition may mean a line can be fit to a parabola, strictly speaking, although it will do so very poorly.&lt;/p&gt;&#xA;&lt;p&gt;The hypothesis space resembles the definition, perhaps (I have not checked it), but what matters is the &amp;quot;mindset&amp;quot;. I would draw a parallel with prior and posterior distributions in Statistics, where we usually make some hypothesis on the shape of the distribution.&lt;/p&gt;&#xA;&lt;p&gt;Concrete examples like linear regressions generate linear functions, as hinted by their names. Decision trees generate sets of linear functions, thus more complex and &amp;quot;capable&amp;quot;. SVM generates functions depending on its kernel. The &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RBF kernel&lt;/a&gt;, for example, allows generating functions that can map quite complex, non-linear data &amp;quot;shapes&amp;quot;. And arbitrary neural networks can map &amp;quot;arbitrary&amp;quot; data &amp;quot;shape&amp;quot; (easier written than actually done).&lt;/p&gt;&#xA;" OwnerUserId="169" LastEditorUserId="169" LastEditDate="2020-08-11T01:05:34.680" LastActivityDate="2020-08-11T01:05:34.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22976" PostTypeId="1" AcceptedAnswerId="23844" CreationDate="2020-08-11T05:16:50.373" Score="2" ViewCount="48" Body="&lt;p&gt;I have been doing a course which teaches you about Deep Neural Networks, during one of the exercises I was made to make an RNN for sentiment classification which I did, but I did not understand how an RNN is able to deal with sentences of different lengths while conducting sentiment classification.&lt;/p&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="32636" LastEditDate="2020-08-11T09:14:08.667" LastActivityDate="2020-10-30T18:04:05.427" Title="How do RNN's for sentiment classification deal with different sentence lengths?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;text-classification&gt;&lt;sentiment-analysis&gt;&lt;multi-label-classification&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22977" PostTypeId="1" CreationDate="2020-08-11T07:44:22.893" Score="0" ViewCount="13" Body="&lt;p&gt;What is the current way robots can assess and traverse difficult obstacles in 3d space?&lt;/p&gt;&#xA;&lt;p&gt;I could see manual feature extraction using stereo vision (for example, the height of the obstacle, &amp;quot;smoothness&amp;quot;, etc.), but I am wondering if Deep RL can work?&lt;/p&gt;&#xA;&lt;p&gt;Or even something involving model-based planning, where you internally run an RRT algorithm on the obstacle in real-time using a physics engine (but it's not clear to me how well a physics engine could actually model the environment).&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-08-11T10:08:57.980" LastActivityDate="2020-08-11T10:08:57.980" Title="What is the current way robots can assess and traverse difficult obstacles in 3d space?" Tags="&lt;ai-design&gt;&lt;computer-vision&gt;&lt;robotics&gt;&lt;path-planning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22978" PostTypeId="1" CreationDate="2020-08-11T09:06:02.343" Score="0" ViewCount="39" Body="&lt;p&gt;I am making an RNN for sentiment classification. What activation functions would you use in order to achieve this goal (excluding the one present in the output layer)?&lt;/p&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="32410" LastEditDate="2021-04-23T00:10:53.997" LastActivityDate="2021-04-23T00:10:53.997" Title="What are the most used and effective activation functions for sentiment classification with an recurrent neural network?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;activation-function&gt;&lt;text-classification&gt;&lt;sentiment-analysis&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22979" PostTypeId="5" CreationDate="2020-08-11T10:13:11.180" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-11T10:13:11.180" LastActivityDate="2020-08-11T10:13:11.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22980" PostTypeId="4" CreationDate="2020-08-11T10:13:11.180" Score="0" Body="For questions related to Bellman operators in the context of reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-11T10:13:11.180" LastActivityDate="2020-08-11T10:13:11.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22981" PostTypeId="2" ParentId="3847" CreationDate="2020-08-11T10:25:57.063" Score="1" Body="&lt;p&gt;The question is based on two concepts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First artificial intelligence (AI)&lt;/li&gt;&#xA;&lt;li&gt;The transistor is an intelligent device.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Let us talk about the first AI, why transistors, the same definition of intelligence can be applied to Vaccum tubes, and they definitely existed before transistors. So no matter what definition you decide for intelligence, &lt;em&gt;&lt;strong&gt;transistors are not the first AI&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now we come to the next part, &lt;strong&gt;what is artificial intelligence,&lt;/strong&gt; like intelligence, the definition of artificial intelligence has undergone changes, in the last 6 decades.&lt;/p&gt;&#xA;&lt;p&gt;If you use the definition loosely, almost any device can be intelligent even an electric bulb.&lt;/p&gt;&#xA;" OwnerUserId="12211" LastActivityDate="2020-08-11T10:25:57.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22982" PostTypeId="1" CreationDate="2020-08-11T11:26:17.283" Score="0" ViewCount="13" Body="&lt;p&gt;My dataset has 3 columns - a,b,c.&#xA;Using b (and its history), I wish to predict c.&#xA;Using list function and converting to array, I can tell python to look at last 20 b's for any b, as input to predict c.&#xA;Now, I wish to tell the code, to look at those last 20 b's where the 'a' value is same, and in case it doesn't find 20 such entries with same 'a' value, then it should look at as many entries as there are available with same value of 'a'&lt;/p&gt;&#xA;" OwnerUserId="40221" LastActivityDate="2020-08-11T11:26:17.283" Title="Python code in LSTM to look at selective history" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22983" PostTypeId="1" CreationDate="2020-08-11T12:03:55.967" Score="0" ViewCount="33" Body="&lt;p&gt;What is the purpose of a fully connected multi layer perceptron in which every input is connected to every output by a weight?&lt;/p&gt;&#xA;&lt;p&gt;After all, the information is only distributed over several channels, but it would be sufficient to change only one weight per neuron in the next layer.&#xA;If I have a 3 x 3 architecture, each neuron in the hidden layer has 3 inputs (3 channels).&#xA;It would also be enough to change one weight of the 3 inputs...&#xA;Is there any book that goes into detail about the beginnings of topology and simplifies the interpretation?&lt;/p&gt;&#xA;" OwnerUserId="38966" LastActivityDate="2020-08-11T12:03:55.967" Title="What is the purpose to have fully connected layers?" Tags="&lt;neural-networks&gt;&lt;multilayer-perceptron&gt;&lt;fully-connected-layer&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22985" PostTypeId="1" AcceptedAnswerId="22990" CreationDate="2020-08-11T15:31:42.437" Score="1" ViewCount="66" Body="&lt;p&gt;I'm following the guide as outlined at this link: &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://neuralnetworksanddeeplearning.com/chap2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the purposes of this question, I've written a basic network 2 hidden layers, one with 2 neurons and one with one neuron. For a very basic task, the network will learn how to compute an OR logic gate so the training data will be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;X = [[0, 0], [0, 1], [1, 0], [1, 1]]&#xA;Y = [0, 1, 1, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the diagram:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/E4aCO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E4aCO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For this example, the weights and biases are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;w = [[0.3, 0.4], [0.1]]&#xA;b = [[1, 1], [1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The feedforward part was pretty easy to implement so I don't think I need to post that here. The tutorial I've been following summarises calculating the errors and the gradient descent algorithm with the following equations:&lt;/p&gt;&#xA;&lt;p&gt;For each training example &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, compute the output error &lt;span class=&quot;math-container&quot;&gt;$\delta^{x, L}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$L =$&lt;/span&gt; Final layer (Layer 1 in this case). &lt;span class=&quot;math-container&quot;&gt;$\delta^{x, L} = \nabla_aC_x \circ \sigma'(z^{x, L})$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\nabla_aC_x$&lt;/span&gt; is the differential of the cost function (basic MSE) with respect to the Layer 1 activation output, and &lt;span class=&quot;math-container&quot;&gt;$\sigma'(z^{x, L})$&lt;/span&gt; is the derivative of the sigmoid function of the Layer 1 output i.e. &lt;span class=&quot;math-container&quot;&gt;$\sigma(z^{x, L})(1-\sigma(z^{x, L}))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That's all good so far and I can calculate that quite straightforwardly. Now for &lt;span class=&quot;math-container&quot;&gt;$l = L-1, L-2, ...$&lt;/span&gt;, the error for each previous layer can be calculated as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\delta^{x, l} = ((w^{l+1})^T \delta^{x, l+1}) \circ \sigma(z^{x, l})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which again, is pretty straight forward to implement.&lt;/p&gt;&#xA;&lt;p&gt;Finally, to update the weights (and bias), the equations are for &lt;span class=&quot;math-container&quot;&gt;$l = L, L-1, ...$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w^l \rightarrow w^l - \frac{\eta}{m}\sum_x\delta^{x,l}(a^{x, l-1})^T$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b^l \rightarrow b^l - \frac{\eta}{m}\sum_x\delta^{x,l}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand is how this works with vectors of different numbers of elements (I think the lack of vector notation here confuses me).&lt;/p&gt;&#xA;&lt;p&gt;For example, Layer 1 has one neuron, so &lt;span class=&quot;math-container&quot;&gt;$\delta^{x, 1}$&lt;/span&gt; will be a scalar value since it only outputs one value. However, &lt;span class=&quot;math-container&quot;&gt;$a^{x, 0}$&lt;/span&gt; is a vector with two elements since layer 0 has two neurons. Which means that &lt;span class=&quot;math-container&quot;&gt;$\delta^{x, l}(a^{x, l-1})^T$&lt;/span&gt; will be a vector even if I sum over all training samples &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. What am I supposed to do here? Am I just supposed to sum the components of the vector as well?&lt;/p&gt;&#xA;&lt;p&gt;Hopefully my question makes sense; I feel I'm very close to implementing this entirely and I'm just stuck here.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;&lt;p&gt;[edit] Okay, so I realised that I've been misrepresenting the weights of the neurons and have corrected for that.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;weights = [np.random.randn(y, x) for x, y in zip(sizes[:-1], sizes[1:])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which has the output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[array([[0.27660583, 1.00106314],&#xA;   [0.34017727, 0.74990392]])&#xA;array([[ 1.095244  , -0.22719165]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which means that layer0 has a weight matrix with shape 2x2 representing the 2 weights on neuron01 and the 2 weights on neuron02.&lt;/p&gt;&#xA;&lt;p&gt;My understanding then is that &lt;span class=&quot;math-container&quot;&gt;$\delta^{x,l}$&lt;/span&gt; has the same shape as the weights array because each weight gets updated indepedently. That's also fine.&lt;/p&gt;&#xA;&lt;p&gt;But the bias term (according to the link I sourced) has 1 term for each neuron, which means layer 0 will has two bias terms (b00 and b01) and layer 1 has one bias term (b10).&lt;/p&gt;&#xA;&lt;p&gt;However, to calculate the update for the bias terms, you sum the deltas over x i.e &lt;span class=&quot;math-container&quot;&gt;$\sum_x \delta^{x, l}$&lt;/span&gt;; if delta has the size of the weight matrix, then there are too many terms to update the bias terms. What have I missed here?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="40230" LastEditorUserId="40230" LastEditDate="2020-08-12T11:42:39.343" LastActivityDate="2020-08-12T15:00:40.680" Title="Implementing Gradient Descent Algorithm in Python, bit confused regarding equations" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22986" PostTypeId="1" AcceptedAnswerId="23062" CreationDate="2020-08-11T15:42:37.790" Score="5" ViewCount="232" Body="&lt;p&gt;I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it should work, but the agent is not learning. I will really really really appreciate if someone can pinpoint where I am doing wrong.&lt;/p&gt;&#xA;&lt;p&gt;Note that I have a target neuaral network and a policy network already there. The code is as below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import gym&#xA;import random&#xA;from keras.optimizers import Adam&#xA;from keras.models import Sequential&#xA;from keras.layers import Dense&#xA;from collections import deque&#xA;&#xA;env = gym.make('CartPole-v0')&#xA;&#xA;EPISODES = 2000&#xA;BATCH_SIZE = 32&#xA;DISCOUNT = 0.95&#xA;UPDATE_TARGET_EVERY = 5&#xA;STATE_SIZE = env.observation_space.shape[0]&#xA;ACTION_SIZE = env.action_space.n&#xA;SHOW_EVERY = 50&#xA;&#xA;class DQNAgents:&#xA;    &#xA;    def __init__(self, state_size, action_size):&#xA;        self.state_size = state_size&#xA;        self.action_size = action_size&#xA;        self.replay_memory = deque(maxlen = 2000)&#xA;        self.gamma = 0.95&#xA;        self.epsilon = 1&#xA;        self.epsilon_decay = 0.995&#xA;        self.epsilon_min = 0.01&#xA;        self.model = self._build_model()&#xA;        self.target_model = self.model&#xA;        &#xA;        self.target_update_counter = 0&#xA;        print('Initialize the agent')&#xA;        &#xA;    def _build_model(self):&#xA;        model = Sequential()&#xA;        model.add(Dense(20, input_dim = self.state_size, activation = 'relu'))&#xA;        model.add(Dense(10, activation = 'relu'))&#xA;        model.add(Dense(self.action_size, activation = 'linear'))&#xA;        model.compile(loss = 'mse', optimizer = Adam(lr = 0.001))&#xA;        &#xA;        return model&#xA;&#xA;    def update_replay_memory(self, current_state, action, reward, next_state, done):&#xA;        self.replay_memory.append((current_state, action, reward, next_state, done))&#xA;        &#xA;    def train(self, terminal_state):&#xA;        &#xA;        # Sample from replay memory&#xA;        minibatch = random.sample(self.replay_memory, BATCH_SIZE)&#xA;        &#xA;        #Picks the current states from the randomly selected minibatch&#xA;        current_states = np.array([t[0] for t in minibatch])&#xA;        current_qs_list= self.model.predict(current_states) #gives the Q value for the policy network&#xA;        new_state = np.array([t[3] for t in minibatch])&#xA;        future_qs_list = self.target_model.predict(new_state)&#xA;        &#xA;        X = []&#xA;        Y = []&#xA;        &#xA;        # This loop will run 32 times (actually minibatch times)&#xA;        for index, (current_state, action, reward, next_state, done) in enumerate(minibatch):&#xA;            &#xA;            if not done:&#xA;                new_q = reward + DISCOUNT * np.max(future_qs_list)&#xA;            else:&#xA;                new_q = reward&#xA;                &#xA;            # Update Q value for given state&#xA;            current_qs = current_qs_list[index]&#xA;            current_qs[action] = new_q&#xA;            &#xA;            X.append(current_state)&#xA;            Y.append(current_qs)&#xA;        &#xA;        # Fitting the weights, i.e. reducing the loss using gradient descent&#xA;        self.model.fit(np.array(X), np.array(Y), batch_size = BATCH_SIZE, verbose = 0, shuffle = False)&#xA;        &#xA;       # Update target network counter every episode&#xA;        if terminal_state:&#xA;            self.target_update_counter += 1&#xA;            &#xA;        # If counter reaches set value, update target network with weights of main network&#xA;        if self.target_update_counter &amp;gt; UPDATE_TARGET_EVERY:&#xA;            self.target_model.set_weights(self.model.get_weights())&#xA;            self.target_update_counter = 0&#xA;    &#xA;    def get_qs(self, state):&#xA;        return self.model.predict(np.array(state).reshape(-1, *state.shape))[0]&#xA;            &#xA;&#xA;''' We start here'''&#xA;&#xA;agent = DQNAgents(STATE_SIZE, ACTION_SIZE)&#xA;&#xA;for e in range(EPISODES):&#xA;    &#xA;    done = False&#xA;    current_state = env.reset()&#xA;    time = 0 &#xA;    total_reward = 0&#xA;    while not done:&#xA;        if np.random.random() &amp;gt; agent.epsilon:&#xA;            action = np.argmax(agent.get_qs(current_state))&#xA;        else:&#xA;            action = env.action_space.sample()&#xA;        &#xA;        next_state, reward, done, _ = env.step(action)&#xA;&#xA;        agent.update_replay_memory(current_state, action, reward, next_state, done)&#xA;        &#xA;        if len(agent.replay_memory) &amp;lt; BATCH_SIZE:&#xA;            pass&#xA;        else:&#xA;            agent.train(done)&#xA;            &#xA;        time+=1    &#xA;        current_state = next_state&#xA;        total_reward += reward&#xA;        &#xA;    print(f'episode : {e}, steps {time}, epsilon : {agent.epsilon}')&#xA;    &#xA;    if agent.epsilon &amp;gt; agent.epsilon_min:&#xA;        agent.epsilon *= agent.epsilon_decay&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results for first 40ish iterations are below (look for the number of steps, they should be increasing and should reach a maximum of 199)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;episode : 0, steps 14, epsilon : 1&#xA;episode : 1, steps 13, epsilon : 0.995&#xA;episode : 2, steps 17, epsilon : 0.990025&#xA;episode : 3, steps 12, epsilon : 0.985074875&#xA;episode : 4, steps 29, epsilon : 0.9801495006250001&#xA;episode : 5, steps 14, epsilon : 0.9752487531218751&#xA;episode : 6, steps 11, epsilon : 0.9703725093562657&#xA;episode : 7, steps 13, epsilon : 0.9655206468094844&#xA;episode : 8, steps 11, epsilon : 0.960693043575437&#xA;episode : 9, steps 14, epsilon : 0.9558895783575597&#xA;episode : 10, steps 39, epsilon : 0.9511101304657719&#xA;episode : 11, steps 14, epsilon : 0.946354579813443&#xA;episode : 12, steps 19, epsilon : 0.9416228069143757&#xA;episode : 13, steps 16, epsilon : 0.9369146928798039&#xA;episode : 14, steps 14, epsilon : 0.9322301194154049&#xA;episode : 15, steps 18, epsilon : 0.9275689688183278&#xA;episode : 16, steps 31, epsilon : 0.9229311239742362&#xA;episode : 17, steps 14, epsilon : 0.918316468354365&#xA;episode : 18, steps 21, epsilon : 0.9137248860125932&#xA;episode : 19, steps 9, epsilon : 0.9091562615825302&#xA;episode : 20, steps 26, epsilon : 0.9046104802746175&#xA;episode : 21, steps 20, epsilon : 0.9000874278732445&#xA;episode : 22, steps 53, epsilon : 0.8955869907338783&#xA;episode : 23, steps 24, epsilon : 0.8911090557802088&#xA;episode : 24, steps 14, epsilon : 0.8866535105013078&#xA;episode : 25, steps 40, epsilon : 0.8822202429488013&#xA;episode : 26, steps 10, epsilon : 0.8778091417340573&#xA;episode : 27, steps 60, epsilon : 0.8734200960253871&#xA;episode : 28, steps 17, epsilon : 0.8690529955452602&#xA;episode : 29, steps 11, epsilon : 0.8647077305675338&#xA;episode : 30, steps 42, epsilon : 0.8603841919146962&#xA;episode : 31, steps 16, epsilon : 0.8560822709551227&#xA;episode : 32, steps 12, epsilon : 0.851801859600347&#xA;episode : 33, steps 12, epsilon : 0.8475428503023453&#xA;episode : 34, steps 10, epsilon : 0.8433051360508336&#xA;episode : 35, steps 30, epsilon : 0.8390886103705794&#xA;episode : 36, steps 21, epsilon : 0.8348931673187264&#xA;episode : 37, steps 24, epsilon : 0.8307187014821328&#xA;episode : 38, steps 33, epsilon : 0.8265651079747222&#xA;episode : 39, steps 32, epsilon : 0.8224322824348486&#xA;episode : 40, steps 15, epsilon : 0.8183201210226743&#xA;episode : 41, steps 20, epsilon : 0.8142285204175609&#xA;episode : 42, steps 37, epsilon : 0.810157377815473&#xA;episode : 43, steps 11, epsilon : 0.8061065909263957&#xA;episode : 44, steps 30, epsilon : 0.8020760579717637&#xA;episode : 45, steps 11, epsilon : 0.798065677681905&#xA;episode : 46, steps 34, epsilon : 0.7940753492934954&#xA;episode : 47, steps 12, epsilon : 0.7901049725470279&#xA;episode : 48, steps 26, epsilon : 0.7861544476842928&#xA;episode : 49, steps 19, epsilon : 0.7822236754458713&#xA;episode : 50, steps 20, epsilon : 0.778312557068642&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36710" LastActivityDate="2020-08-16T04:37:40.397" Title="My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-neural-networks&gt;&lt;open-ai&gt;&lt;gym&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22989" PostTypeId="1" CreationDate="2020-08-11T22:53:36.750" Score="0" ViewCount="20" Body="&lt;p&gt;I'm looking at the beam search psuedo code for the RNN-T decoder &lt;a href=&quot;https://arxiv.org/pdf/1211.3711.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1211.3711.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why are lines 5-7 needed?&lt;/p&gt;&#xA;" OwnerUserId="40237" LastActivityDate="2020-08-11T22:53:36.750" Title="Why does the RNN-T beam search algorithm need to loop over the prefixes of y" Tags="&lt;sequence-modelling&gt;&lt;speech-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22990" PostTypeId="2" ParentId="22985" CreationDate="2020-08-12T00:04:10.137" Score="0" Body="&lt;p&gt;There seems to be a mismatch between the weights you provide and your network diagram. Since &lt;code&gt;w[0]&lt;/code&gt; (the yellow connections) is meant to transform &lt;span class=&quot;math-container&quot;&gt;$ x \in \mathbb{R}^2 $&lt;/span&gt; into the layer 0 activations which are &lt;span class=&quot;math-container&quot;&gt;$ \mathbb{R}^2 $&lt;/span&gt;, &lt;code&gt;w[0]&lt;/code&gt; should be a matrix &lt;span class=&quot;math-container&quot;&gt;$ \in \mathbb{R}^{2 \times 2} $&lt;/span&gt;, not a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2 $&lt;/span&gt; as you have. Likewise, your &lt;code&gt;w[1]&lt;/code&gt; (the red connections) should be a vector &lt;span class=&quot;math-container&quot;&gt;$ \in \mathbb{R^2} $&lt;/span&gt; and not a scalar. Finally, if you are indeed scaling the output of layer 1 (the blue connection), then you'll need an additional scalar value. However, the blue connection confuses me a bit as usually the activated output is used directly in the loss function, not a scaled version of it. Unless the blue connection stands for the loss function.&lt;/p&gt;&#xA;&lt;p&gt;In short, I believe if you change the shapes of your weight matrices to actually represent your network diagram, your update equations will work. I'll go through the network below to make sure I illustrate my point.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ x \in \mathbb{R}^{2} $&lt;/span&gt;, an input example&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ W^0 \in \mathbb{R}^{2 \times 2} $&lt;/span&gt;, the yellow connections&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ W^1 \in \mathbb{R}^2 $&lt;/span&gt;, the red connections&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ z^0 = xW^0 \in \mathbb{R}^{2} $&lt;/span&gt;, the weighted inputs to the layer 0 nodes. The dimensions of this should match the number of nodes at layer 0.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ a^0 = \sigma(z^0) \in \mathbb{R}^{2} $&lt;/span&gt;, the output of the layer 0 nodes. The dimensions of this should match the number of nodes at layer 0.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ z^1 = a^0 W^1 \in \mathbb{R} $&lt;/span&gt;, the weighted inputs to the layer 1 nodes. The dimensions of this should match the number of nodes at layer 1.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ a^1 = \sigma(z^1) \in \mathbb{R} $&lt;/span&gt;, the output of the layer 1 nodes and thus the output of the network. The dimensions of this should match the number of nodes at layer 1.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Weight Updates&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you say before your edit, &lt;span class=&quot;math-container&quot;&gt;$\delta^1$&lt;/span&gt;, as the product of two scalars &lt;span class=&quot;math-container&quot;&gt;$\nabla_a C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma'(z^1)$&lt;/span&gt;, is also a scalar. Since &lt;span class=&quot;math-container&quot;&gt;$a^0$&lt;/span&gt; is a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt;, then&#xA;&lt;span class=&quot;math-container&quot;&gt;$\delta^1(a^0)^T$&lt;/span&gt; is also a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt;. This matches what we expect, as it should match the dimensions of &lt;span class=&quot;math-container&quot;&gt;$W^1$&lt;/span&gt; to allow the element-wise subtraction in the weight update equation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NB.&lt;/strong&gt; It is not the case, as you say in your edit, that the shape of &lt;span class=&quot;math-container&quot;&gt;$\delta^l$&lt;/span&gt; should match the shape of &lt;span class=&quot;math-container&quot;&gt;$W^l$&lt;/span&gt;. It should instead match the number of nodes, and it is the shape of &lt;span class=&quot;math-container&quot;&gt;$\delta^l(a^{l-1})^T$&lt;/span&gt; that should match the shape of &lt;span class=&quot;math-container&quot;&gt;$W^l$&lt;/span&gt;. You had this right in your original post.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Bias Updates&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This brings us to the bias updates.&#xA;There should be one bias term per node in a given layer, so the shapes of your biases are correct (i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2$&lt;/span&gt; for layer 0 and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; for layer 1). Now, we saw above that the shape of &lt;span class=&quot;math-container&quot;&gt;$\delta^l$&lt;/span&gt; also matches the number of nodes in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, so again the element-wise subtraction in your original bias update equation works.&lt;/p&gt;&#xA;&lt;p&gt;I also tried using this book to learn backprop, but I had a hard time connecting the variables with the different parts of the network and the corresponding code. I finally understood the algorithm in depth only after deriving all the update equations by hand for a very small network (2 inputs, one output, no hidden layers) and working my way up to larger networks, making sure to keep track of the shapes of the inputs and outputs along the way. If you're having trouble with the update equations I highly recommend this.&lt;/p&gt;&#xA;&lt;p&gt;A final piece of advice that helped me: drop the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and the summations over input examples from your formulations and just treat everything as matrices (e.g. a scalar becomes a matrix in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{1 \times 1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a matrix in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{N \times D}$&lt;/span&gt;). First, this allows you to better interpret matrix orientations and debug issues such as a missing transpose operation. Second, this is (in my limited understanding) how backprop should actually be implemented in order to take advantage of optimized linalg libraries and GPUs, so it's perhaps a bit more relevant.&lt;/p&gt;&#xA;" OwnerUserId="37972" LastEditorUserId="37972" LastEditDate="2020-08-12T15:00:40.680" LastActivityDate="2020-08-12T15:00:40.680" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22991" PostTypeId="2" ParentId="22959" CreationDate="2020-08-12T01:08:21.350" Score="1" Body="&lt;p&gt;This is known as &lt;em&gt;reward hacking&lt;/em&gt; in the literature; see, e.g., &lt;a href=&quot;https://medium.com/@deepmindsafetyresearch/designing-agent-incentives-to-avoid-reward-tampering-4380c1bb6cd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@deepmindsafetyresearch/designing-agent-incentives-to-avoid-reward-tampering-4380c1bb6cd&lt;/a&gt; for discussion and further links.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2020-08-12T01:08:21.350" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22992" PostTypeId="2" ParentId="18290" CreationDate="2020-08-12T04:52:50.653" Score="0" Body="&lt;p&gt;The term &amp;quot;intentionality&amp;quot; has two quite different senses. One is a very technical concept in philosophy of AI and means (roughly) aboutness in the sense that beliefs, desires, fears, etc., are about things (snakes, tax, chocolate). That's the sense central to searle's Chinese room argument. The other sense is the common idea of intending to do (or not do) something. A really key issue for AI according to Searle is how can a computer have intentionality in the first sense.&lt;/p&gt;&#xA;" OwnerUserId="17709" LastActivityDate="2020-08-12T04:52:50.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22993" PostTypeId="1" AcceptedAnswerId="22995" CreationDate="2020-08-12T12:39:45.807" Score="9" ViewCount="1099" Body="&lt;p&gt;I'm new to the artificial intelligence field. In our first chapters, there is one topic called &amp;quot;problem-solving by searching&amp;quot;. After searching for it on the internet, I found the &lt;em&gt;depth-first search&lt;/em&gt; algorithm. The algorithm is easy to understand, but no one explains why this algorithm is included in the artificial intelligence study.&lt;/p&gt;&#xA;&lt;p&gt;Where do we use it? What makes it an artificial intelligence algorithm? Is every search algorithm is an AI algorithm?&lt;/p&gt;&#xA;" OwnerUserId="40245" LastEditorUserId="2444" LastEditDate="2020-08-12T13:59:10.847" LastActivityDate="2020-08-12T23:20:56.690" Title="Why is depth-first search an artificial intelligence algorithm?" Tags="&lt;search&gt;&lt;ai-field&gt;&lt;depth-first-search&gt;" AnswerCount="2" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22994" PostTypeId="1" AcceptedAnswerId="22996" CreationDate="2020-08-12T12:53:08.957" Score="2" ViewCount="67" Body="&lt;p&gt;I'm slightly confused about the experience replay process. I understand why we use batch processing in reinforcement learning, and from my understanding, a batch of states is input into the neural network model.&lt;/p&gt;&#xA;&lt;p&gt;Suppose there are 2 valid moves in the action space (UP or DOWN)&lt;/p&gt;&#xA;&lt;p&gt;Suppose the batch size is 5, and the 5 states are this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[s_1, s_2, s_3, s_4, s_5]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We put this batch into the neural network model and output Q values. Then we put &lt;span class=&quot;math-container&quot;&gt;$[s_1', s_2', s_3', s_4', s_5']$&lt;/span&gt; into a target network.&lt;/p&gt;&#xA;&lt;p&gt;What I'm confused about is this:&lt;/p&gt;&#xA;&lt;p&gt;Each state in &lt;span class=&quot;math-container&quot;&gt;$[s_1, s_2, s_3, s_4, s_5]$&lt;/span&gt; is different.&lt;/p&gt;&#xA;&lt;p&gt;Are we computing Q values for UP and DOWN for ALL 5 states after they go through the neural network?&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;span class=&quot;math-container&quot;&gt;$$[Q_{s_1}(\text{UP}), Q_{s_1}(\text{DOWN})],&#xA;\\ [Q_{s_2} (\text{UP}), Q_{s_2}(\text{DOWN})], \\ &#xA;[Q_{s_3}(\text{UP}), Q_{s_3}(\text{DOWN})], \\ &#xA;[Q_{s_4}(\text{UP}), Q_{s_4}(\text{DOWN})], \\ &#xA;[Q_{s_5}(\text{UP}), Q_{s_5}(\text{DOWN})]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="2444" LastEditDate="2020-08-12T21:07:20.923" LastActivityDate="2020-08-12T21:07:20.923" Title="When using experience replay in reinforcement learning, which state is used for training?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22995" PostTypeId="2" ParentId="22993" CreationDate="2020-08-12T13:13:49.470" Score="14" Body="&lt;p&gt;This is a fundamentally a philosophical question. What makes AI AI? But first things, why would DFS be considered an AI algorithm?&lt;/p&gt;&#xA;&lt;p&gt;In its most basic form, DFS is a very general algorithm that is applied to wildly different categories of problems: topological sorting, finding all the connected components in a graph, etc. It may be also used for searching. For instance, you could use DFS for finding a path in a 2D maze (although not necessarily the shortest one). Or you could use it to navigate through more abstract state spaces (e.g. between configuration of chess or in the towers of Hanoi). And this is where the connection to AI arises. DFS can be used on its own for navigating such spaces, or as a basic subroutine for more complex algorithms. I believe that in the book &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt; (which you may be reading at the moment) they introduce DFS and Breadth-First Search this way, as a first milestone before reaching more complex algorithms like A*.&lt;/p&gt;&#xA;&lt;p&gt;Now, you may be wondering why such search algorithms should be considered AI. Here, I'm speculating, but maybe the source of the confusion comes from the fact that DFS does not learn anything. This is a common misconception among new AI practitioners. Not every AI technique has to revolve around learning. In other words, AI != Machine Learning. ML is one of the many subfields within AI. In fact, early AI (around the 50s-60s) was more about logic reasoning than it was about learning.&lt;/p&gt;&#xA;&lt;p&gt;AI is about making an artificial system behave &amp;quot;intelligently&amp;quot; in a given setting, whatever it takes to reach that intelligent behavior. If what it takes is applying well-known algorithms from computer science like DFS, then so be it. Now, what is it that intelligent means? This is where we enter more philosophical grounds. My interpretation is that &amp;quot;intelligence&amp;quot; is a broad term to define the large set of techniques that we use to approach the immense complexity that reality and certain puzzle-like problems have to offer. Often, &amp;quot;intelligent behavior&amp;quot; revolves around heuristics and proxy methods away from the perfect, provable algorithms that work elsewhere in computer science. While certain algorithms (like DFS or A*) may be proven to give optimal answers if infinitely many resources can be devoted to the task at hand, only in sufficiently constrained settings would such techniques be affordable. Fortunately, we can make them work in many situations (like A* for chess or for robot navigation, or Monte Carlo Tree Search for Go), but only if reasonable assumptions and constraints over the state space are imposed. For all the rest is where learning techniques (like Markov Random Fields for image segmentation, or Neural Nets paired with Reinforcement Learning for situated agents) may come handy.&lt;/p&gt;&#xA;&lt;p&gt;Funny enough, even if intelligence is often regarded as a good thing, my interpretation can be summed up as &lt;em&gt;imperfect modes of behavior to address immensely complex problems for which no known perfect solution exists (with rare exceptions in sufficiently bounded problems)&lt;/em&gt;. If we had a huge table that, for each chess position, gives the best possible move you can make, and put that table inside a program, would this program be intelligent? Maybe you'd think so, but in any case it seems more arguable than a program that makes real-time reasoning and spits a decision after some reasonable time, even if it's not the best one. Similarly, do you consider sorting algorithms intelligent? Again, the answer is arguable, but the fact is that algorithms exist with optimal time and memory complexities, we know that we can't do better than what those algorithms do, and we do not have to resort to any heuristic or any learning to do better (disclaimer: I haven't actually checked if there's some madman out in the wild applying learning to solve sorting with better average times).&lt;/p&gt;&#xA;" OwnerUserId="37359" LastEditorUserId="37359" LastEditDate="2020-08-12T15:33:21.430" LastActivityDate="2020-08-12T15:33:21.430" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22996" PostTypeId="2" ParentId="22994" CreationDate="2020-08-12T15:05:57.593" Score="1" Body="&lt;p&gt;The way the states are used is as follows:&lt;/p&gt;&#xA;&lt;p&gt;Typically your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-network will state a state as input and output scores over the action space. I.e. &lt;span class=&quot;math-container&quot;&gt;$Q : \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}|}$&lt;/span&gt;. So, in your replay buffer you should store &lt;span class=&quot;math-container&quot;&gt;$s_t, a_t, r_{t+1}, s_{t+1}, \mbox{done}$&lt;/span&gt; (note that done just represents where the episode ended on this transition and I add for completeness.&lt;/p&gt;&#xA;&lt;p&gt;Now, when you are doing your batch updates you sample uniformly at random from this replay buffer. This means you get &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; tuples of &lt;span class=&quot;math-container&quot;&gt;$s_t, a_t, r_{t+1}, s_{t+1}, \mbox{done}$&lt;/span&gt;. Now, I will assume &lt;span class=&quot;math-container&quot;&gt;$B=1$&lt;/span&gt; as it is easier to explain and the extension to &lt;span class=&quot;math-container&quot;&gt;$B &amp;gt; 1$&lt;/span&gt; should be easy to see.&lt;/p&gt;&#xA;&lt;p&gt;For our state-action tuple &lt;span class=&quot;math-container&quot;&gt;$s_t, a_t$&lt;/span&gt; we want to shift what the network predicts for this pair to be closer to &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma \arg\max_a Q(s,a)$&lt;/span&gt;. However, our neural network only takes the state as input, and outputs a vector of scores for each action. That means we want to shift the output of our network for the state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; towards the target I just mentioned, &lt;em&gt;but only for the action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; that we took&lt;/em&gt;. To do this we just calculate the target, i.e. we calculate &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma \arg\max_a Q(s,a)$&lt;/span&gt;, and then we do gradient ascent like we would a normal neural network where the target vector is the same as the predicted vector everywhere except the &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;th element, which we will change to &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma \arg\max_a Q(s,a)$&lt;/span&gt;. This way, our network moves closer to our Q-learning update for only the action we want, in line with how Q-learning works.&lt;/p&gt;&#xA;&lt;p&gt;It is also worth nothing that you &lt;em&gt;can&lt;/em&gt; parameterise your Neural Network to be a function &lt;span class=&quot;math-container&quot;&gt;$Q: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt; which would make training more in line with how tabular Q-learning but is seldom used in practice as it becomes much more expensive to compute (you have to do a forward pass for each action, rather than one forward pass per state).&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-08-12T15:53:31.813" LastActivityDate="2020-08-12T15:53:31.813" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="22997" PostTypeId="1" CreationDate="2020-08-12T15:52:57.503" Score="2" ViewCount="56" Body="&lt;p&gt;In Convolutional Neural Networks, do all filters of the same convolutional layer need to have the same dimensions and stride?&lt;/p&gt;&#xA;&lt;p&gt;If they don't, then it would seem the channel produced by each filter would have different sizes. Or is there some way to get around that?&lt;/p&gt;&#xA;" OwnerUserId="40250" LastEditorUserId="2444" LastEditDate="2020-08-12T23:47:50.077" LastActivityDate="2021-01-10T14:05:15.527" Title="Do all filters of the same convolutional layer need to have the same dimensions and stride?" Tags="&lt;convolutional-neural-networks&gt;&lt;filters&gt;&lt;convolutional-layers&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22999" PostTypeId="1" AcceptedAnswerId="23004" CreationDate="2020-08-12T18:15:41.780" Score="0" ViewCount="135" Body="&lt;p&gt;I am trying to re-implement the &lt;a href=&quot;https://www.kdd.org/kdd2016/papers/files/rfp0191-wangAemb.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SDNE algorithm&lt;/a&gt; for graph embedding by PyTorch.&lt;/p&gt;&#xA;&lt;p&gt;I get stuck at some issues about evaluation metric &lt;strong&gt;Precision@K&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;precision@k&lt;/em&gt; is a metric which gives equal weight to the returned instance. It is defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$precision@k(i) = \frac{\left| \, \{ j \, | \, i, j \in V, index(j) \le k, \Delta_i(j) = 1 \} \, \right|}{k}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is the vertex set, &lt;span class=&quot;math-container&quot;&gt;$index(j)$&lt;/span&gt; is the ranked index of the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;-th vertex and &lt;span class=&quot;math-container&quot;&gt;$\Delta_i(j) = 1$&lt;/span&gt; indicates that &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_j$&lt;/span&gt; have a link.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't understand what &amp;quot;ranked index of the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;-th vertex&amp;quot; means.&lt;/p&gt;&#xA;&lt;p&gt;Beside, I am also confused about the &lt;strong&gt;MAP&lt;/strong&gt; metric in &lt;strong&gt;section 4.3&lt;/strong&gt;. I don't understand how to calculate it.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;Mean Average Precision (MAP)&lt;/em&gt; is a metric with good discrimination and stability. Compared with &lt;em&gt;precision@k&lt;/em&gt;, it is&#xA;more concerned with the performance of the returned items ranked ahead. It is calculated as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$AP(i) = \frac{\sum_j precision@j(i) \cdot \Delta_i(j)}{\left| \{ \Delta_i(j) = 1 \} \right|}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$MAP = \frac{\sum_{i \in Q} AP(i)}{|Q|}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is the query set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If anyone is familiar with these metrics, could you help me to explain them?&lt;/p&gt;&#xA;" OwnerUserId="25645" LastEditorUserId="40178" LastEditDate="2020-08-13T03:27:53.870" LastActivityDate="2021-05-16T15:12:35.343" Title="What is Precision@K for link prediction in graph embedding meaning?" Tags="&lt;deep-learning&gt;&lt;graphs&gt;&lt;precision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23000" PostTypeId="1" CreationDate="2020-08-12T18:22:14.033" Score="0" ViewCount="12" Body="&lt;p&gt;I'm lately writing a chat-bot to answer questions about children's fantasy books I save to my database .&lt;/p&gt;&#xA;&lt;p&gt;When a user opens a book it loads the chat-bot for the given book . I don't need it to work on all books globally .&lt;/p&gt;&#xA;&lt;p&gt;I wrote a tf-idf model to retrieve a set of possible sections from the book that could answer my question . I understood that this model won't scale well since i'll be adding lots of books ever day and my vocab will increase reasonably with new fantasy words . I was wondering if this was a problem on itself.&lt;/p&gt;&#xA;&lt;p&gt;I was advised to use transformer models like BERT but i don't have a proper dataset with questions so i found a &lt;a href=&quot;https://github.com/facebookresearch/UnsupervisedQA&quot; rel=&quot;nofollow noreferrer&quot;&gt;facebook repo&lt;/a&gt; to generate possible QA for some documents and train . I was wondering if it could be a good idea to train my model on another model output . I feel that creating a model to approximate another one is a stupid approach since we could just use the first model in the first place but the facebook model is too time consuming and i want mine to give fast results .&lt;/p&gt;&#xA;&lt;p&gt;I want to deploy the chatbot as a web app so a fast model would be really great but i think that this has more to do with software engineering and deployment that artificial intelligence .&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering what approach would be best for my problem . I'm open to any new approach .&lt;/p&gt;&#xA;" OwnerUserId="40254" LastActivityDate="2020-08-12T18:22:14.033" Title="What is the best NLP model for handeling a rapid extension of the production data for a question answeing task" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;bert&gt;&lt;tf-idf&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23001" PostTypeId="1" CreationDate="2020-08-12T19:11:28.057" Score="1" ViewCount="82" Body="&lt;p&gt;I understand we use a target network because it helps resolve issues regarding stability, however, that's not what I'm here to ask.&lt;/p&gt;&#xA;&lt;p&gt;What I would like to understand is why a target network is used as a measure of ground truth as opposed to the expectation equation.&lt;/p&gt;&#xA;&lt;p&gt;To clarify, here is what I mean. This is the process used for DQN:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In DQN, we begin with a state &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;We then pass this state through a neural network which outputs Q values for each action in the action space&lt;/li&gt;&#xA;&lt;li&gt;A policy e.g. epsilon-greedy is used to take an action&lt;/li&gt;&#xA;&lt;li&gt;This subsequently produces the next state &lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$S_{t+1}$&lt;/span&gt; is then passed through a target neural network to produce target Q values&lt;/li&gt;&#xA;&lt;li&gt;These target Q values are then injected into the Bellman equation which ultimately produces a target Q value via the Q-learning update rule equation&lt;/li&gt;&#xA;&lt;li&gt;MSE is used on 6 and 2 to compute the loss&lt;/li&gt;&#xA;&lt;li&gt;This is then back-propagated to update the parameters for the neural network in 2&lt;/li&gt;&#xA;&lt;li&gt;The target neural network has its parameters updated every X epochs to match the parameters in 2&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Why do we use a target neural network to output Q values instead of using statistics. Statistics seems like a more accurate way to represent this. By statistics, I mean this:&lt;/p&gt;&#xA;&lt;p&gt;Q values are the expected return, given the state and action under policy π.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(S_{t+1},a) = V^π(S_{t+1})$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}(r_{t+1}+ γr_{t+2}+ (γ^2)_{t+3} + ...    \mid S_{t+1}) = {E}(∑γ^kr_{t+k+1}\mid S_{t+1})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can then take the above and inject it into the Bellman equation to update our target Q value:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(S_{t},a_t) + α*(r_t+γ*max(Q(S_{t+1},a))-Q(S_{t},a))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, why don't we set the target to the sum of diminishing returns? Surely a target network is very inaccurate, especially since the parameters in the first few epochs for the target network are completely random.&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="26159" LastEditDate="2020-08-12T21:03:10.173" LastActivityDate="2020-08-13T08:52:25.560" Title="Why are Target Networks used in Deep Q-Learning as opposed to the Expected Value equation?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23003" PostTypeId="1" AcceptedAnswerId="23014" CreationDate="2020-08-12T22:01:21.603" Score="1" ViewCount="127" Body="&lt;p&gt;I'm aware that we back-propagate after computing the loss between:&lt;/p&gt;&#xA;&lt;p&gt;The Neural Network Q values and the Target Network Q values&lt;/p&gt;&#xA;&lt;p&gt;However, all this is doing is updating the parameters of the Neural Network to produce an output that matches the Target Q values as close as possible.&lt;/p&gt;&#xA;&lt;p&gt;Suppose one epoch is run and the reward is +10, surely we need to update the parameters using this too to tell the Network to push the probability of these actions, given these parameters up.&lt;/p&gt;&#xA;&lt;p&gt;How does the algorithm know +10 is good? Suppose the reward range is -10 for loss and +10 for win.&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="26159" LastEditDate="2020-08-13T13:30:34.170" LastActivityDate="2020-08-13T13:30:34.170" Title="In DQN, when do the parameters in the Neural Network update based on the reward received?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23004" PostTypeId="2" ParentId="22999" CreationDate="2020-08-12T22:44:24.137" Score="0" Body="&lt;p&gt;These measures are used for evaluating how &amp;quot;good&amp;quot; an embedding of a graph is or how &amp;quot;good&amp;quot; the graph reconstructed from the embedding resembles the original.&lt;/p&gt;&#xA;&lt;p&gt;Given the embedding and vertex &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, it seems to be that the rank of the vertices is dependent on the probability of there being a link between vertex &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and vertex &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in the original graph. If there is a higher probability of there being a link between &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in the original graph, &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; has a lower rank.&lt;/p&gt;&#xA;&lt;p&gt;In other words, &lt;span class=&quot;math-container&quot;&gt;$precision@k(i)$&lt;/span&gt; is the proportion of vertices &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; that vertex &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; has a link to in the original graph out of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; vertices for which vertex &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; has the highest probability of having a link to, recovered from the embedding.&lt;/p&gt;&#xA;&lt;p&gt;This matches up with the common definition of &lt;span class=&quot;math-container&quot;&gt;$precision@n$&lt;/span&gt; used in evaluating information/document retrieval, defined as the proportion of relevant documents out of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; best retrieved documents.&lt;/p&gt;&#xA;&lt;p&gt;The average precision of a vertex, &lt;span class=&quot;math-container&quot;&gt;$AP(i)$&lt;/span&gt;, is the average of &lt;span class=&quot;math-container&quot;&gt;$precision@j$&lt;/span&gt; over all &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; such that there is a link between vertex &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and vertex &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. Perhaps a more clear definition would have been &lt;span class=&quot;math-container&quot;&gt;$$AP(i) = \frac{\sum_{j \in S_i} precision@j(i)}{\left| S_i \right|}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$S_i = \{j \, |\,  \Delta_i(j) = 1 \}$&lt;/span&gt;, the set of all &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; such that there is a link from &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$MAP$&lt;/span&gt; for a query set &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is then the mean of the average precision (&lt;span class=&quot;math-container&quot;&gt;$AP$&lt;/span&gt;) over all vertices in &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40178" LastEditorUserId="40178" LastEditDate="2020-08-19T20:49:49.320" LastActivityDate="2020-08-19T20:49:49.320" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="23005" PostTypeId="2" ParentId="7683" CreationDate="2020-08-12T23:20:29.090" Score="2" Body="&lt;p&gt;A &lt;strong&gt;hypothesis space/class&lt;/strong&gt; is the &lt;strong&gt;set&lt;/strong&gt; of functions that the learning algorithm considers when picking one function to minimize some risk/loss functional.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;capacity&lt;/strong&gt; of a hypothesis space is a &lt;strong&gt;number&lt;/strong&gt; or &lt;strong&gt;bound&lt;/strong&gt; that quantifies the size (or richness) of the hypothesis space, i.e. the number (and type) of functions that can be represented by the hypothesis space. So a hypothesis space &lt;strong&gt;has&lt;/strong&gt; a capacity. The two most famous measures of capacity are VC dimension and Rademacher complexity.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the hypothesis class is the &lt;em&gt;object&lt;/em&gt; and the capacity is a &lt;em&gt;property&lt;/em&gt; (that can be measured or quantified) of this object, but there is not a big difference between hypothesis class and its capacity, in the sense that a hypothesis class naturally defines a capacity, but two (different) hypothesis classes could have the same capacity.&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;em&gt;representational capacity&lt;/em&gt; (not &lt;em&gt;capacity&lt;/em&gt;, which is common!) is not a standard term in computational learning theory, while &lt;em&gt;hypothesis space/class&lt;/em&gt; is commonly used. For example, &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this famous book&lt;/a&gt; on machine learning and learning theory uses the term &lt;em&gt;hypothesis class&lt;/em&gt; in many places, but it never uses the term &lt;em&gt;representational capacity&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Your book's definition of &lt;em&gt;representational capacity&lt;/em&gt; is &lt;em&gt;bad&lt;/em&gt;, in my opinion, if &lt;em&gt;representational capacity&lt;/em&gt; is supposed to be a synonym for &lt;em&gt;capacity&lt;/em&gt;, given that that definition also coincides with the definition of hypothesis class, so your confusion is understandable.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-14T11:34:39.377" LastActivityDate="2020-08-14T11:34:39.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23006" PostTypeId="2" ParentId="22993" CreationDate="2020-08-12T23:20:56.690" Score="1" Body="&lt;p&gt;&lt;code&gt;DFS&lt;/code&gt; on its own would not typically be considered &lt;code&gt;AI&lt;/code&gt; imo.  It is a standard computer science deterministic algorithm.  Instead an intelligent agent might use &lt;code&gt;DFS&lt;/code&gt; to inform its decision making as part of an AI package.&lt;/p&gt;&#xA;" OwnerUserId="33059" LastActivityDate="2020-08-12T23:20:56.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23007" PostTypeId="5" CreationDate="2020-08-12T23:40:03.540" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-12T23:40:03.540" LastActivityDate="2020-08-12T23:40:03.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23008" PostTypeId="4" CreationDate="2020-08-12T23:40:03.540" Score="0" Body="For questions related to reward hacking, i.e. when an agent manipulates the reward system to get a lot of reward in unintended ways. The paper &quot;Concrete Problems in AI Safety&quot; by Dario Amodei et al. provides more details about this topic." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-12T23:46:09.543" LastActivityDate="2020-08-12T23:46:09.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23009" PostTypeId="2" ParentId="7683" CreationDate="2020-08-13T03:57:40.150" Score="2" Body="&lt;p&gt;A &lt;strong&gt;hypothesis space&lt;/strong&gt; is defined as the set of functions &lt;span class=&quot;math-container&quot;&gt;$\mathcal H$&lt;/span&gt; that can be chosen by a learning algorithm to minimize loss (in general).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal H = \{h_1, h_2,....h_n\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The hypothesis class can be finite or infinite, for example a discrete set of shapes to encircle certain portion of the input space is a finite hypothesis space, whereas hpyothesis space of parametrized functions like neural nets and linear regressors are infinite.&lt;/p&gt;&#xA;&lt;p&gt;Although the term representational capacity is not in the vogue a rough definition woukd be: The &lt;strong&gt;representational capacity&lt;/strong&gt; of a model, is the ability of its &lt;strong&gt;hypothesis space&lt;/strong&gt;  to approximate a complex function, with 0 error, which can only be approximated by &lt;strong&gt;infinitely&lt;/strong&gt; many hypothesis spaces whose representational capacity is equal to or exceed the representational capacity required to approximate the complex function.&lt;/p&gt;&#xA;&lt;p&gt;The most popular measure of representational capacity is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal V$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathcal C$&lt;/span&gt; Dimension&lt;/a&gt; of a model. The upper bound for VC dimension (&lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;) of a model is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$d \leq \log_2| \mathcal H|$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$|H|$&lt;/span&gt; is the cardinality of the set of hypothesis space.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-08-13T03:57:40.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23011" PostTypeId="1" CreationDate="2020-08-13T06:55:54.493" Score="1" ViewCount="30" Body="&lt;p&gt;I am newbie to Reinforcement Learning, this is my idea - Agent(food provider) has to select a food based on the environment(based on the user profile). Here the reward will be given to the agent based on the user's feedback. This is for single person, what if I wanted to make it for multiple persons. I wanted the system to learn on its own and create a policy such that it can identify certain group of people based on their profile and what type of food will be suitable for them.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this possible to implement in Reinforcement learning?&lt;/li&gt;&#xA;&lt;li&gt;If so what type of problem is this and what type of solution I can use to solve this.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40262" LastActivityDate="2020-08-13T06:55:54.493" Title="Customized food for persons based on their profile using Reinforcement learning" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;policy-gradients&gt;&lt;markov-decision-process&gt;&lt;multi-armed-bandit&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23012" PostTypeId="1" AcceptedAnswerId="23013" CreationDate="2020-08-13T07:05:23.650" Score="3" ViewCount="519" Body="&lt;p&gt;From what I understand, if the rewards are &lt;strong&gt;sparse&lt;/strong&gt; the agent will have to &lt;strong&gt;explore more&lt;/strong&gt; to get rewards and learn the optimal policy, whereas if the rewards are &lt;strong&gt;dense&lt;/strong&gt; in time, the agent is &lt;strong&gt;quickly&lt;/strong&gt; guided towards its learning goal.&lt;/p&gt;&#xA;&lt;p&gt;Are the above thoughts correct, and are there any other pros and cons of the two contrasting settings? On a side-note, I feel that the inability to specify rewards that are dense in time is what makes imitation learning useful.&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-11-02T21:42:31.633" LastActivityDate="2020-11-02T21:42:31.633" Title="What are the pros and cons of sparse and dense rewards in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;reward-functions&gt;&lt;sparse-rewards&gt;&lt;dense-rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23013" PostTypeId="2" ParentId="23012" CreationDate="2020-08-13T07:34:27.970" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What are the pros and cons of sparse and dense rewards in reinforcement learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is unusual to refer to this difference as &amp;quot;pros and cons&amp;quot; because that term is often used to make comparisons between difference &lt;em&gt;choices&lt;/em&gt;. Assuming you have a specific problem to solve, then whether or not the rewards are &lt;em&gt;naturally&lt;/em&gt; sparse or dense is not a choice. You cannot say &amp;quot;I want to solve MountainCar, I will use a dense reward setting&amp;quot;, because MountainCar has (relatively, for a starting problem) sparse rewards. You can only say &amp;quot;I won't attempt MountainCar, it is too difficult&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In short however, your assessment is correct:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;if the rewards are sparse the agent will have to explore more to get rewards and learn the optimal policy, whereas if the rewards are dense in time, the agent is quickly guided towards its learning goal&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is not really any other difference at the top level. Essentially, sparser rewards make for a harder problem to solve. All RL algorithms can cope with sparse rewards to some degree, the whole concept of returns and value backup is designed to deal with sparseness at a theoretical level. In practical terms however, it may take some algorithms an unreasonable amount of time to determine a good policy beyond certain levels of sparseness.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;On a side-note, I feel that the inability to specify rewards that are dense in time is what makes imitation learning useful.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Imitation learning is one of many techniques available to work around or deal with problems that have sparse reward structure. Others include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Reward shaping, which attempts to convert a sparse reward scheme to a dense one using domain knowledge of the researcher.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Eligibility traces, which back up individual TD errors across multiple time steps.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Prioritised sweeping, which focuses updates on &amp;quot;surprising&amp;quot; reward data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Action selection planning algorithms that look ahead from the current state.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;Curiousity&amp;quot; driven reinforcement learning that guides exploration to new state spaces independently of any reward signal.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-08-13T07:34:27.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23014" PostTypeId="2" ParentId="23003" CreationDate="2020-08-13T07:47:05.433" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;However, all this is doing is updating the parameters of the Neural Network to produce an output that matches the Target Q values as close as possible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. That is all it needs to do because we have defined the policy around the Q values like so:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a \hat{q}(s,a,\theta)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is the neural network weights.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, if the estimates of Q are approximately the same as the action value of the optimal policy, the policy in DQN is approximately the optimal policy.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How does the algorithm know +10 is good?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It does not, at least not directly. The algorithm knows, approximately, what the action values are &lt;strong&gt;if&lt;/strong&gt; it acts consistently with their current estimates by always choosing the maximising action at each step.&lt;/p&gt;&#xA;&lt;p&gt;The learning process will learn that +10 is &lt;em&gt;relatively&lt;/em&gt; good in your scenario because it never finds anything better when exploring.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-08-13T07:47:05.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23015" PostTypeId="2" ParentId="23001" CreationDate="2020-08-13T08:30:08.907" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Why are Target Networks used in Deep Q-Learning as opposed to the Expected Value equation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In short, because for many problems, this learns more efficiently.&lt;/p&gt;&#xA;&lt;p&gt;It is the difference between Monte Carlo (MC) methods and Temporal Difference (TD) learning.&lt;/p&gt;&#xA;&lt;p&gt;You &lt;em&gt;can&lt;/em&gt; use MC estimates for expected return in deep RL. They are slower for two reasons:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It takes far more experience to collect enough data to train a neural network, because to fully sample a return you need a whole episode. You cannot just use one episode at a time because that presents the neural network with correlated data. You would need to collect multiple episodes and fill a large experience table.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As an aside, you would also need to discard all the experience after each update, because sampled full returns are on-policy data. Or you could implement importance sampling for off-policy Monte Carlo control and re-calculate the correct updates when the policy starts to improve, which is added complexity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Samples of full returns have a higher variance, so the sampled data is noisier.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In comparison, TD learning starts with biased samples. This bias reduces over time as estimates become better, but it is the reason why a target network is used (otherwise the bias would cause runaway feed back).&lt;/p&gt;&#xA;&lt;p&gt;So you have a bias/variance trade off with TD representing high bias and MC representing high variance.&lt;/p&gt;&#xA;&lt;p&gt;It is not clear theoretically which is better in general, because it depends on the nature of MDPs that you are solving with each method. In practice, on the types of problems Deep RL has been tried on, single-step TD learning appears to do better than MC sampling of returns, in terms of goals such as sample efficiency and learning time.&lt;/p&gt;&#xA;&lt;p&gt;You can compromise between TD and MC using eligibility traces, resulting in TD(&lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;). However, this is awkward to implement in Deep RL due to the experience replay table. A simpler compromise is to use &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns e.g. &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma r_{t+2} + \gamma^2 r_{t+3} + \gamma^3 \text{max}_a(Q(s_{t+4},a))$&lt;/span&gt;, which was one of the refinements used in the &lt;a href=&quot;https://arxiv.org/abs/1710.02298&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Rainbow&amp;quot; DQN paper&lt;/a&gt; - note that even though strictly in their case, this handled off-policy incorrectly (it &lt;em&gt;should&lt;/em&gt; use importance sampling, but they didn't bother), it still worked well enough for low &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; on the Atari problems.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-13T08:52:25.560" LastActivityDate="2020-08-13T08:52:25.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23016" PostTypeId="1" AcceptedAnswerId="23020" CreationDate="2020-08-13T09:15:28.017" Score="2" ViewCount="75" Body="&lt;p&gt;I've been reading &lt;a href=&quot;https://arxiv.org/abs/1011.0686&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Reduction of Imitation Learning and Structured Prediction&#xA;to No-Regret Online Learning&lt;/em&gt;&lt;/a&gt; lately, and I can't understand what they mean by the &lt;strong&gt;surrogate loss function.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some relevant notation from the paper -&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$d_\pi$&lt;/span&gt; = average distribution of states if we follow policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; timesteps&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$C(s,a)$&lt;/span&gt; = the expected immediate cost of performing action a in state s for the task we are considering (assume &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is bounded in [0,1]&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$C_\pi(s) = \mathbb{E}_{a\sim\pi(s)}[C(s,a)]$&lt;/span&gt; is the expected immediate cost of &lt;span class=&quot;math-container&quot;&gt;$π$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$J(π) = T\mathbb{E}_{s\sim d_\pi}[C_\pi(s)]$&lt;/span&gt; is the total cost of executing policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; timesteps&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In imitation learning, we may not necessarily know or observe true costs &lt;span class=&quot;math-container&quot;&gt;$C(s,a)$&lt;/span&gt; for the particular task. Instead, we observe expert demonstrations and seek to bound &lt;span class=&quot;math-container&quot;&gt;$J(π)$&lt;/span&gt;&#xA;for any cost function &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; based on how well &lt;span class=&quot;math-container&quot;&gt;$π$&lt;/span&gt; mimics the expert’s policy &lt;span class=&quot;math-container&quot;&gt;$π^{*}$&lt;/span&gt;. Denote &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; the &lt;strong&gt;observed surrogate loss function&lt;/strong&gt; we minimize &lt;strong&gt;instead&lt;/strong&gt; of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. For instance, &lt;span class=&quot;math-container&quot;&gt;$l(s,π)$&lt;/span&gt; may be the expected 0-1 loss of &lt;span class=&quot;math-container&quot;&gt;$π$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$π^{*}$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, or a squared/hinge loss of &lt;span class=&quot;math-container&quot;&gt;$π$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$π^{*}$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Importantly, in many instances, &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; may be the same function&lt;/strong&gt; – for instance, if we are interested in &lt;strong&gt;optimizing the learner’s ability to predict the actions chosen by an expert&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't understand &lt;em&gt;how exactly&lt;/em&gt; the surrogate loss is &lt;strong&gt;different&lt;/strong&gt; from the true costs, and &lt;em&gt;what are the possible cases&lt;/em&gt; in which both are the &lt;strong&gt;same&lt;/strong&gt;. It'd be great if someone could throw some light on this. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-08-13T10:40:15.167" LastActivityDate="2020-08-13T14:08:51.650" Title="What is the surrogate loss function in imitation learning, and how is it different from the true cost?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;imitation-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23017" PostTypeId="5" CreationDate="2020-08-13T10:58:50.577" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-13T10:58:50.577" LastActivityDate="2020-08-13T10:58:50.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23018" PostTypeId="4" CreationDate="2020-08-13T10:58:50.577" Score="0" Body="For questions related to imitation learning (IL), a reinforcement learning technique where a policy is learned from examples (represented as trajectories) of an (optimal) agent's behavior. IL is similar to inverse reinforcement learning (IRL), where a reward function is learned from examples of the (optimal) agent's behavior, which can then be used to solve the RL problem (i.e. find the policy)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-06T01:35:36.150" LastActivityDate="2020-11-06T01:35:36.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23019" PostTypeId="1" AcceptedAnswerId="24519" CreationDate="2020-08-13T11:01:08.400" Score="6" ViewCount="82" Body="&lt;p&gt;I just read the following points about the number of required expert demonstrations in imitation learning, and I'd like some clarifications. For the purpose of context, I'll be using a linear reward function throughout this post (i.e. the reward can be expressed as a weighted sum of the components of a state's feature vector)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The number of expert demonstrations required &lt;strong&gt;scales with the number of features in the reward function&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't think this is obvious at all - why is it true? Intuitively, I think that as the number of features rises, the complexity of the problem does too, so we may need more data to make a better estimate of the expert's reward function. Is there more to it?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The number of expert demonstration required does not depend on -&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Complexity of the expert’s optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^{*}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Size of the state space&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't see how the complexity of the expert's optimal policy plays a role here - which is probably why it doesn't affect the number of expert demonstrations we need; but how do we &lt;strong&gt;quantify the complexity of a policy&lt;/strong&gt; in the first place?&lt;/p&gt;&#xA;&lt;p&gt;Also, I think that the number of expert demonstrations &lt;strong&gt;should depend on the size of the state space&lt;/strong&gt;. For example, if the train and test distributions don't match, we can't do behavioral cloning without falling into problems, in which case we use the &lt;em&gt;DAGGER algorithm&lt;/em&gt; to repeatedly query the expert and make better decisions (take better actions). I feel that a larger state space means that we'll have to query the expert more frequently, i.e. to figure out the expert's optimal action in several states.&lt;/p&gt;&#xA;&lt;p&gt;I'd love to know everyone's thoughts on this - &lt;strong&gt;the dependence of the number of expert demonstrations on the above, and if any, other factors&lt;/strong&gt;. Thank you!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa12/slides/inverseRL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Slide 20/75&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-08-13T20:55:52.897" LastActivityDate="2020-11-10T13:44:05.670" Title="What does the number of required expert demonstrations in Imitation Learning depend on?" Tags="&lt;reinforcement-learning&gt;&lt;apprenticeship-learning&gt;&lt;inverse-rl&gt;&lt;imitation-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23020" PostTypeId="2" ParentId="23016" CreationDate="2020-08-13T11:18:25.173" Score="2" Body="&lt;p&gt;A surrogate loss is a loss than you use &amp;quot;instead of&amp;quot;, &amp;quot;in place of&amp;quot;, &amp;quot;as a proxy for&amp;quot; or &amp;quot;as a substitute for&amp;quot; another loss, which is typically the &amp;quot;true&amp;quot; loss.&lt;/p&gt;&#xA;&lt;p&gt;Surrogate losses are actually common in machine learning (although almost nobody realizes that they are surrogate losses). For example, the &lt;strong&gt;empirical risk&lt;/strong&gt; (which the &lt;em&gt;mean squared error&lt;/em&gt; is an instance of) is a surrogate for the &lt;strong&gt;expected risk&lt;/strong&gt;, which is &lt;strong&gt;incomputable&lt;/strong&gt; in almost all cases, given that you do not know the underlying probability distribution. See &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.356&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Overview of Statistical Learning Theory&lt;/a&gt; by V. N. Vapnik for more details. In fact, discussions on &lt;em&gt;generalization&lt;/em&gt; arise because of this issue, i.e. you use surrogate losses rather than true losses.&lt;/p&gt;&#xA;&lt;p&gt;The term &amp;quot;surrogate&amp;quot; is also used in conjunction with the term &amp;quot;model&amp;quot;, i.e. &amp;quot;surrogate model&amp;quot;, for example, in the context of &lt;a href=&quot;https://distill.pub/2020/bayesian-optimization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian optimization&lt;/a&gt;, where a Gaussian process is the surrogate model for the unknown model/function that you want to know about, i.e. you use the Gaussian process to approximate the unknown function/model.&lt;/p&gt;&#xA;&lt;p&gt;Regarding the excerpt you are quoting and your specific concerns, although I didn't read the paper and I am not an expert in imitation learning, let me try to explain what I understand from this excerpt. Essentially, in imitation learning, you use the expert's policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt; to train the agent, rather than letting him just explore and exploit the environment. So, what you know is &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt; and you can calculate the &amp;quot;loss&amp;quot; between &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (the current agent's policy), denoted by &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;. However, this loss &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; that you calculate is &lt;strong&gt;not necessarily&lt;/strong&gt; the &amp;quot;true&amp;quot; loss (i.e. it is a surrogate loss), given that our goal is not really to imitate the &amp;quot;expert&amp;quot; but to learn an optimal policy to behave in the environment. If the goal was to just imitate the &amp;quot;expert&amp;quot;, then &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; would coincide, because, in that case, &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; would represent the &amp;quot;discrepancy&amp;quot; or &amp;quot;loss&amp;quot; between &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and the expert's policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-13T14:08:51.650" LastActivityDate="2020-08-13T14:08:51.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23021" PostTypeId="1" CreationDate="2020-08-13T13:31:43.800" Score="5" ViewCount="128" Body="&lt;p&gt;In Sutton and Barto's book about reinforcement learning, policy iteration and value iterations are presented as separate/different algorithms.&lt;/p&gt;&#xA;&lt;p&gt;This is very confusing because policy iteration includes an update/change of value and value iteration includes a change in policy. They are the same thing, as also shown in the Generalized Policy Iteration method.&lt;/p&gt;&#xA;&lt;p&gt;Why then, in many papers as well, they (i.e. policy and value iterations) are considered two separate update methods to reach an optimal policy?&lt;/p&gt;&#xA;" OwnerUserId="38958" LastEditorUserId="2444" LastEditDate="2020-08-13T23:00:42.440" LastActivityDate="2020-08-14T18:36:01.103" Title="Why are policy iteration and value iteration studied as separate algorithms?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23022" PostTypeId="2" ParentId="6329" CreationDate="2020-08-13T13:55:49.107" Score="0" Body="&lt;p&gt;I also struggled with this when I was implementing NEAT.&lt;/p&gt;&#xA;&lt;p&gt;What worked for me was cycle detection using DFS search in this video &lt;a href=&quot;https://www.youtube.com/watch?v=tg96sZqhXyU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=tg96sZqhXyU&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Simply put, I did DFS on all my input nodes recording all the nodes visited if I encounter a node I've already visited then its a cycle thereby I for my neat to discard this and attempt to make another connection.&lt;/p&gt;&#xA;" OwnerUserId="40272" LastActivityDate="2020-08-13T13:55:49.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23023" PostTypeId="2" ParentId="22997" CreationDate="2020-08-13T13:57:06.387" Score="1" Body="&lt;p&gt;It seems that a similar question has been raised here: &lt;a href=&quot;https://stackoverflow.com/questions/57438922/different-size-filters-in-the-same-layer-with-tensorflow-2-0&quot;&gt;https://stackoverflow.com/questions/57438922/different-size-filters-in-the-same-layer-with-tensorflow-2-0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Like answered in the link above, you could combine severall Conv2D ops with different kernel sizes on the same input. You would have to adapt each output with padding, or cropping, so that you could concatenate all of them.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="34409" LastActivityDate="2020-08-13T13:57:06.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23024" PostTypeId="2" ParentId="23021" CreationDate="2020-08-13T14:11:09.473" Score="3" Body="&lt;p&gt;Policy iteration is made up of two steps. The first is a full policy evaluation, where a value function is calculated for the current policy. The second is policy improvement, where the policy is made greedy with respect to the value function.&lt;/p&gt;&#xA;&lt;p&gt;Value iteration looks to speed things up by stopping policy evaluation after one iteration, make the policy greedy with respect to that value function, and repeat until convergence.&lt;/p&gt;&#xA;&lt;p&gt;Clearly, these are two different algorithms, hence why they are considered to be different. They are, however, very closely linked, which is why you might consider them to be 'the same thing'. I guess you could say they belong to the same family of algorithm.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-13T14:11:09.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23025" PostTypeId="1" AcceptedAnswerId="23033" CreationDate="2020-08-13T14:15:15.523" Score="1" ViewCount="40" Body="&lt;p&gt;This is an open-ended question.Suppose I have a reinforcement learning task that is being solved using many different fixed policies, one of which is optimal. The goal of the agent is not to figure out what the optimal policy is but rather which policy (from a set of predefined fixed policies) is the optimal one.&lt;/p&gt;&#xA;&lt;p&gt;Are there any algorithms/methods that handle this?&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if meta learning is the right area to look into?&lt;/p&gt;&#xA;" OwnerUserId="40275" LastActivityDate="2020-08-13T19:03:22.860" Title="Finding the optimal policy from a set of fixed policies in reinforcement learning" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;meta-learning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23026" PostTypeId="1" CreationDate="2020-08-13T14:24:47.487" Score="3" ViewCount="95" Body="&lt;p&gt;I've already read the original paper about double DQN but I do not find a clear and practical explanation of how the target &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is computed, so here's how I interpreted the method (let's say I have 3 possible actions (1,2,3)):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For each experience &lt;span class=&quot;math-container&quot;&gt;$e_{j}=(s_{j},a_{j},r_{j},s_{j+1})$&lt;/span&gt; of the mini-batch (consider an experience where &lt;span class=&quot;math-container&quot;&gt;$a_{j}=2$&lt;/span&gt;) I compute the output through the main network in the state &lt;span class=&quot;math-container&quot;&gt;$s_{j+1}$&lt;/span&gt;, so I obtain 3 values.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I look which of the three is the highest so: &lt;span class=&quot;math-container&quot;&gt;$a^*=arg\max_{a}Q(s_{j+1},a)$&lt;/span&gt;, let's say &lt;span class=&quot;math-container&quot;&gt;$a^*=1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I use the target network to compute the value in &lt;span class=&quot;math-container&quot;&gt;$a^*=1$&lt;/span&gt; , so  &lt;span class=&quot;math-container&quot;&gt;$Q_{target}(s_{j+1},1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I use the value at point 3 to substitute the value in the target vector associeted with the known action &lt;span class=&quot;math-container&quot;&gt;$a_{j}=2$&lt;/span&gt;, so: &lt;span class=&quot;math-container&quot;&gt;$Q_{target}(s_{j+1},2)\leftarrow r_{j}+\gamma Q_{target}(s_{j+1},1)$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$Q_{target}(s_{j+1},1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q_{target}(s_{j+1},3)$&lt;/span&gt;, which complete the target vector &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, remain the same.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is there anything wrong?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-11-04T21:13:46.130" LastActivityDate="2020-11-04T21:13:46.130" Title="How to compute the target for double Q-learning update step?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23027" PostTypeId="1" CreationDate="2020-08-13T15:05:54.023" Score="0" ViewCount="20" Body="&lt;p&gt;I have data containing 50 lines like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 1 0 0 1 1 0 0  E0&#xA;1 0 1 0 1 1 0 0  E1&#xA;0 1 0 1 0 1 0 1  E2&#xA;0 0 1 1 0 1 0 1  E3&#xA;1 1 1 1 0 0 0 0  E4&#xA;0 0 0 1 1 1 0 1  E5&#xA;0 1 1 1 1 0 0 0  E6&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each ordering system contains 8 numbers, which 4 numbers of them are 0, and 4 other numbers must be 0. I would like to find an ordering which gives the lowest value of E. Can you suggest to me which algorithm in machine learning can predict the minimum best? Can linear regression perform it well?&lt;/p&gt;&#xA;" OwnerUserId="40273" LastActivityDate="2020-08-13T15:05:54.023" Title="How to find lowest value value of ordering" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23028" PostTypeId="2" ParentId="22914" CreationDate="2020-08-13T16:14:41.833" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I am having trouble understanding how to keep track of the expansion, do I expand all stochastic possibilities and weight the return via their chance of happening?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is indeed one option you can take. This would be very similar in spirit to the idea of &amp;quot;Expectimax&amp;quot; as a variant of minimax for non-deterministic games, in the sense that you'll include explicit &amp;quot;chance nodes&amp;quot; in your tree. When running into such a chance node later on again during a Selection phase, of a later MCTS iteration, you can just select a path of the tree to follow based on a &amp;quot;dice roll&amp;quot;. &lt;strong&gt;Importantly&lt;/strong&gt;, note that this option is only actually available if you have explicit knowledge of exactly when chance events occur, which states they can lead to, and with which probabilities they lead to different states. We also assume that this is feasible, i.e. that you don't have a crazy high (or infinite) number of slightly different game states you could reach.&lt;/p&gt;&#xA;&lt;p&gt;An alternative option is to use an &amp;quot;open-loop&amp;quot; variant of MCTS. Your nodes would no longer represent game states, but only be representative of the sequence of actions leading to them. You would no longer store any game states in any nodes, but always regenerate them from scratch when traversing the tree, starting from the root node. You would no longer have any explicit chance nodes, but instead have states being representative of larger &lt;strong&gt;sets of states&lt;/strong&gt; that could possibly be reached by following the corresponding path from the root node. For more on this, see &lt;a href=&quot;https://ai.stackexchange.com/a/13919/1641&quot;&gt;my answer to this other question&lt;/a&gt;. The advantage of this approach is that it does not require explicit knowledge of all the possible states you can reach due to chance events, do not need explicit knowledge of the probabilities, and can just sample instead of explicitly enumerating every possible outcome.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-08-13T16:14:41.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23029" PostTypeId="2" ParentId="17540" CreationDate="2020-08-13T16:19:46.153" Score="1" Body="&lt;p&gt;I might be able to help with the theory, but the coding... it is a non standard API such as Tensorflow or Pytorch (it might be custom code for what I can tell).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;The key element here is that the bouding boxes are removed only if they hold a prediciton for the same class that the box that is overlapping with&lt;/strong&gt;&lt;/em&gt; (but with less confidence, that is why it gets removed).&lt;/p&gt;&#xA;&lt;p&gt;Here is an example, where we have:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Two classes &lt;span class=&quot;math-container&quot;&gt;$c \in [c_1, c_2] = [$&lt;/span&gt; &amp;quot;star&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$, $&lt;/span&gt; &amp;quot;moon&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Three bounding boxes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The blue bounding boxes holds prediction for the class &lt;span class=&quot;math-container&quot;&gt;$c_1$&lt;/span&gt; so their predictions are &lt;span class=&quot;math-container&quot;&gt;$p(c_1)_{box1} = 0.8$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ p(c_1)_{box2} = 0.9$&lt;/span&gt;. On the other hand, the green box holds a prediction for the class &lt;span class=&quot;math-container&quot;&gt;$c_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The three boxes are highly overlaping so the overlap between any box &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; with any box &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; will be above the IoU threshold: &lt;span class=&quot;math-container&quot;&gt;$IoU(box_x, box_y) &amp;gt; 0.5$&lt;/span&gt;. So in principle all boxes are suceptible to be removed.&lt;/p&gt;&#xA;&lt;p&gt;However the NMS only applies for boxes predicting the same class (in the case the blue one). So the NMS algorihtm is: if the boxes are overlapping, &lt;span class=&quot;math-container&quot;&gt;$IoU(box_1, box_2) &amp;gt; 0.5$&lt;/span&gt;, which is true, remove all non maximal class probability boxes. Said differently, take just the box with highest &lt;span class=&quot;math-container&quot;&gt;$p(c_1)$&lt;/span&gt; and remove the rest. So the &lt;span class=&quot;math-container&quot;&gt;$box_1$&lt;/span&gt; with class probability &lt;span class=&quot;math-container&quot;&gt;$p(c_1) = 0.8$&lt;/span&gt; would be removed.&lt;/p&gt;&#xA;&lt;p&gt;So what happens with green box? Isn't it overlapping as well? Yes, but consider that the green box is not trying to predict the same object, is trying to predict another object, &lt;span class=&quot;math-container&quot;&gt;$c_2$&lt;/span&gt;, which happens to be very close to the first object, &lt;span class=&quot;math-container&quot;&gt;$c_1$&lt;/span&gt;. This way object detectors support detection of different overlapping objects.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6MJ1l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6MJ1l.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-08-13T16:19:46.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23031" PostTypeId="1" CreationDate="2020-08-13T17:53:19.207" Score="0" ViewCount="17" Body="&lt;p&gt;Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The &lt;a href=&quot;https://www.tensorflow.org/guide/keras/transfer_learning#fine-tuning&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommended way&lt;/a&gt; would be:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Freeze the backbone and train the model&lt;/li&gt;&#xA;&lt;li&gt;Unfreeze the backbone and train the model with a low learning rate&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My question now is which of these configurations is better.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Config 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Freeze the backbone and train the model with all 10,000 images of the dataset&lt;/li&gt;&#xA;&lt;li&gt;Unfreeze the backbone and train the model with a low learning rate with all 10,000 images.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Config 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Freeze the backbone and train the model, e.g. 2000 images of the dataset, so that we get rid of the initial randomness.&lt;/li&gt;&#xA;&lt;li&gt;Unfreeze the backbone and train the model with a low learning rate with all 10,000 images.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Config 3&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Freeze the backbone and train the model, e.g. 2000 images of the dataset, so that we get rid of the initial randomness.&lt;/li&gt;&#xA;&lt;li&gt;Unfreeze the backbone and train the model with a low learning rate with the rest of images (8000).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My intuition says that config 1 could be good for when the backbone was pretrained for cats, dog, bus, car, ship, etc. (like ImageNet or COCO) and my dataset is similar. But I think when I have a complete different dataset like (bio-)medical images or other kind of special and very different images, then config 2 or 3 could be a better choice since you get the weight in the right direction in the first step and then have more space for information to store in the second step.&lt;/p&gt;&#xA;&lt;p&gt;I am curious to know if there is a preferred way of doing and whether there are good reasons for or against.&lt;/p&gt;&#xA;" OwnerUserId="33494" LastActivityDate="2020-08-13T17:53:19.207" Title="Transfer Learning: Finetune a model with a splitted dataset?" Tags="&lt;deep-learning&gt;&lt;transfer-learning&gt;&lt;fine-tuning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23033" PostTypeId="2" ParentId="23025" CreationDate="2020-08-13T18:39:25.733" Score="0" Body="&lt;p&gt;The quickest way to do this would be to use policy evaluation methods. Most of the standard optimal control algorithms consist of policy evaluation plus a rule for updating the policy.&lt;/p&gt;&#xA;&lt;p&gt;It may not be possible to rank arbitrary policies by performance when considering &lt;em&gt;all&lt;/em&gt; states. So you will want to rank them according to some fixed distribution of state values. The usual distribution of start states would be a natural choice (this is also the objective when learning via policy gradients in e.g. Actor-Critic).&lt;/p&gt;&#xA;&lt;p&gt;One simple method would be to run multiple times for each policy, starting each time according to the distribution of start states, and calculate the return (discounted sum of rewards) from each one. A simple Monte Carlo run from each start state would be fine, and is very simple to code. Take the mean value as your estimate, and measure the variance too so you can establish a confidence for your selection.&lt;/p&gt;&#xA;&lt;p&gt;Then simply select the policy with the best average value in start states. You can use the variance to calculate a standard error for this, so you will have a feel for how robust your selection is.&lt;/p&gt;&#xA;&lt;p&gt;If have a large number of policies to select between, you could do a first pass through with a relatively low number of samples, and try to rule out policies that perform badly enough that even adding say 3 standard errors to the estimated value would not cause them to be preferred. Other than that, the more samples you can take, the more accurate your estimates of mean starting value for each policy will be, and the more likely you will be to select the right policy.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I was wondering if meta learning is the right area to look into?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In general no, but you might want to consider meta learning if:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You have too many policies to select between by testing them all thoroughly.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The policies have some meaningful low dimension representation that is driving their behaviour. The policy function itself would normally be too high dimensional.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You could then use some form of meta-learning to predict policy performance directly from the representation, and start to skip evaluations from non-promising policies. You may need your fixed policies to number in the thousands or millions before this works though (depending on the number of parameters in the representation and complexity of mapping between parameters and policy function), plus you will still want to thoroughly estimate performance of candidates selected as worth evaluating by the meta-learning.&lt;/p&gt;&#xA;&lt;p&gt;In comments you suggest treating the list of policies as context-free bandits, using a bandit solver to pick the policy that scores the best on average. This might offer some efficiency over evaluating each policy multiple times in sequence. A good solver will try to find best item in the list using a minimal number of samples, and you could use something like UCB or Gibbs distribution to focus more on the most promising policies. I think the main problem with this will be finding the right hyperparameters for the bandit algorithm. I would suggest if you do that to seed the initial estimates with an exhaustive test of each policy multiple times, so you can get a handle on variance and scale of the mean values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-13T19:03:22.860" LastActivityDate="2020-08-13T19:03:22.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23034" PostTypeId="1" CreationDate="2020-08-13T19:13:57.887" Score="0" ViewCount="27" Body="&lt;p&gt;I want to dive into Reinforcement Learning and therefore as a little project I am trying to swing up and balance a Furuta pendulum in Simulink. Unfortunately I don´t really know where to start, when it comes to the network architecture. Are there standard architectures for my actor and critics networks that I can use to try out or is there good literature for beginners summerizing the most important steps or tips?&#xA;I am quite unsure how many and what kind of layers with how many neurons each is appropriate?&lt;/p&gt;&#xA;" OwnerUserId="40282" LastActivityDate="2020-08-13T19:13:57.887" Title="Neural network architecture for DDPG agent in Matlab - standard networks?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;matlab&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23035" PostTypeId="1" AcceptedAnswerId="23039" CreationDate="2020-08-13T20:00:29.993" Score="1" ViewCount="74" Body="&lt;p&gt;In section &lt;a href=&quot;http://www.incompleteideas.net/book/4/node5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;4.4 Value Iteration&lt;/a&gt;, the authors write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;One important special case is when policy evaluation is stopped after just one sweep (&lt;strong&gt;one update of each state&lt;/strong&gt;). This algorithm is called value iteration.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;After that, they provide the following pseudo-code&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kvMKn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kvMKn.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is clear from the code that updates of each state occur until &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; is sufficiently small. Not &lt;strong&gt;one update of each state&lt;/strong&gt; as the authors write in the text. Where is the mistake?&lt;/p&gt;&#xA;" OwnerUserId="40285" LastEditorUserId="2444" LastEditDate="2020-08-13T22:54:06.003" LastActivityDate="2020-08-13T22:54:06.003" Title="Is value iteration stopped after one update of each state?" Tags="&lt;reinforcement-learning&gt;&lt;value-iteration&gt;&lt;policy-evaluation&gt;&lt;pseudocode&gt;&lt;policy-improvement&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23036" PostTypeId="1" AcceptedAnswerId="23046" CreationDate="2020-08-13T20:08:19.157" Score="3" ViewCount="63" Body="&lt;p&gt;I am trying to find the best algorithm to create a list of recommendations for a user based on the interests of all other users.&lt;/p&gt;&#xA;&lt;p&gt;Say I have a list of of samples:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$samples = [&#xA;    ['hot dog', 'big mac', 'whopper'],&#xA;    ['hot dog', 'big mac'],&#xA;    ['hot dog', 'whopper'],&#xA;    ['big mac', 'dave single'],&#xA;    ['whopper', 'mcnuggets', 'mcchicken'],&#xA;    ['mcchicken', 'original chicken sandwich'],&#xA;    ['mcchicken', 'mcrib']&#xA;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And we will say each array in the sample list is unique user's food preferences.&lt;/p&gt;&#xA;&lt;p&gt;Let's say now I have a user with this food preference:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['hot dog', 'mcchicken']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to be able to recommend to this user other foods that other users have in their preferences.&lt;/p&gt;&#xA;&lt;p&gt;So in the simplest terms, it should return:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['whopper', 'big mac', 'original chicken sandwich', 'mcrib', 'mcnuggets']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Obviously I will also introduce other variables such as how each user rates each item in their preference list and also the percentage of users that need to have that item in order to use their other food items as recommendations.&lt;/p&gt;&#xA;&lt;p&gt;But I would like to find the best algorithm to start working on it.&lt;/p&gt;&#xA;&lt;p&gt;At first I thought &lt;code&gt;Apriori&lt;/code&gt; might be my best guess, but I wasn't having luck once I introduced multiple items.&lt;/p&gt;&#xA;" OwnerUserId="40284" LastEditorUserId="40284" LastEditDate="2020-08-13T21:52:04.293" LastActivityDate="2020-08-14T08:56:08.033" Title="What is the most appropriate ML algorithm for creating recommendations" Tags="&lt;machine-learning&gt;&lt;recommender-system&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23037" PostTypeId="1" CreationDate="2020-08-13T20:19:46.490" Score="1" ViewCount="68" Body="&lt;p&gt;Suppose you have a ground plane and can use a stereo vision system to detect things that are possibly separate objects.&lt;/p&gt;&#xA;&lt;p&gt;Suppose also your robot or agent can attempt to pick up and move these objects around in real-time.&lt;/p&gt;&#xA;&lt;p&gt;Is there any current system in computer vision that allows new objects to be learned in real-time?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-08-15T09:47:09.840" LastActivityDate="2020-08-18T14:48:53.567" Title="Is there any real-time computer vision system that can learn to detect new objects of new classes?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;online-learning&gt;&lt;incremental-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23038" PostTypeId="1" CreationDate="2020-08-13T20:45:41.447" Score="2" ViewCount="156" Body="&lt;p&gt;2015 was a milestone year for AI--&amp;quot;deep learning&amp;quot; was validated in a very public way with AlphaGo.  However, at the time, the question was raised: &amp;quot;What else is deep learning good for?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;5 years later, I want to gauge:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How is deep learning applied to real world problems in 2020?  What real world applications is it currently used for?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2020-08-14T10:51:38.910" Title="What is the scope of real-world deep learning applications in 2020?" Tags="&lt;deep-learning&gt;&lt;applications&gt;&lt;real-world&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23039" PostTypeId="2" ParentId="23035" CreationDate="2020-08-13T21:01:20.183" Score="2" Body="&lt;p&gt;Where the author mentions the policy evaluation being stopped after one state, they are referring to the part of the algorithm that evaluates the policy -- the pseudocode you have listed is the pseudocode for &lt;em&gt;Value Iteration&lt;/em&gt;, which consists of iterating between policy evaluation and policy improvement.&lt;/p&gt;&#xA;&lt;p&gt;In normal policy evaluation, you would apply the update &lt;span class=&quot;math-container&quot;&gt;$v_{k+1}(s) = \mathbb{E}_\pi[R_{t+1} + \gamma v_k(S_{t+1})|S_t = s]$&lt;/span&gt; until convergence. In the &lt;em&gt;policy iteration&lt;/em&gt; algorithm, you perform policy evaluation until the value functions converge in each state, then apply policy improvement, and repeat. &lt;em&gt;Value iteration&lt;/em&gt; will perform &lt;em&gt;policy evaluation&lt;/em&gt; for one update, i.e. not until convergence, and then improve the policy, and repeat this until the value functions converge.&lt;/p&gt;&#xA;&lt;p&gt;The line&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \leftarrow \max_a \sum_{s', r} p(s',r|s,a)[r + \gamma V(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;perform both the early stopping policy evaluation and policy improvement. Lets examine how:&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\sum_{s', r} p(s',r|s,a)[r + \gamma V(s')]$&lt;/span&gt; is the same as the expectation I wrote earlier, so we can see clearly that is policy evaluation for just one iteration. Then, we take a max over the actions -- this is policy improvement. Policy improvement is defined as (for a deterministic policy)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\pi'(s) &amp;amp;= \arg\max_a q_\pi(s,a) \\ &amp;amp;= \arg\max_a \sum_{s', r} p(s',r|s,a)[r + \gamma V(s')]\;.&#xA;\end{align}&lt;/span&gt;&#xA;Here, we assign the action that satisfies the &lt;span class=&quot;math-container&quot;&gt;$\mbox{argmax}$&lt;/span&gt; to the improved policy in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. This is essentially what we are doing in the   line from your pseudo when we take the max. We are evaluating our value function for a policy that is greedy with respect to said value function.&lt;/p&gt;&#xA;&lt;p&gt;If you keep applying the line from the pseudocode of value iteration it will eventually converge to the optimal value function as it will end up satisfying the Bellman Optimality Equation.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-08-13T21:51:58.877" LastActivityDate="2020-08-13T21:51:58.877" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23040" PostTypeId="5" CreationDate="2020-08-13T22:54:26.963" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-13T22:54:26.963" LastActivityDate="2020-08-13T22:54:26.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23041" PostTypeId="4" CreationDate="2020-08-13T22:54:26.963" Score="0" Body="For questions that involve the pseudocode of artificial intelligence algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-13T22:54:26.963" LastActivityDate="2020-08-13T22:54:26.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23042" PostTypeId="2" ParentId="23038" CreationDate="2020-08-13T23:48:20.517" Score="4" Body="&lt;p&gt;Deep learning is used to perform language translation in &lt;a href=&quot;https://translate.google.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Google Translate&lt;/strong&gt;&lt;/a&gt; [&lt;a href=&quot;https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. Specifically, Google Translate now uses &lt;a href=&quot;https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformers&lt;/a&gt; and RNNs rather than the original &lt;a href=&quot;https://ai.googleblog.com/2016/09/a-neural-network-for-machine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNMT system&lt;/a&gt; (proposed in 2016), which was also based on neural networks. Deep learning is also used in &lt;a href=&quot;https://www.deepl.com/en/translator&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;DeepL&lt;/strong&gt;&lt;/a&gt; (though I cannot find a good resource to cite apart from Wikipedia [&lt;a href=&quot;https://en.wikipedia.org/wiki/DeepL_Translator&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;] given that the system is closed-source), a strong alternative to Google Translate. However, note that, in general, machine translation is still far from perfect and it is probably not adopted to perform serious translations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Tesla's autopilot&lt;/strong&gt; also uses neural networks [&lt;a href=&quot;https://www.tesla.com/autopilot&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Built on a deep neural network, Tesla Vision deconstructs the car's environment at greater levels of reliability than those achievable with classical vision processing techniques.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;DeepFakes&lt;/strong&gt; are also developed using deep learning techniques [&lt;a href=&quot;https://en.wikipedia.org/wiki/Deepfake&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;].&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-14T10:51:38.910" LastActivityDate="2020-08-14T10:51:38.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23043" PostTypeId="1" CreationDate="2020-08-14T04:49:50.107" Score="0" ViewCount="24" Body="&lt;p&gt;I am attempting to forecast a time series using tensorflow with the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;X = mytimeseries&#xA;scaler = MinMaxScaler()&#xA;scaled = scaler.fit_transform(X)&#xA;&#xA;length = len(X)-1&#xA;generator = TimeseriesGenerator(scaled,scaled,&#xA;                            length=length,batch_size=1)&#xA;&#xA;model = Sequential()&#xA;model.add(LSTM(units=100,activation='relu',input_shape=(length,n_features)))&#xA;model.add(Dense(units=100))&#xA;model.add(Dense(units=1))&#xA;&#xA;model.fit(generator,epochs=20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I just run a loop to forecast, but it's giving me nothing more than a straight line after a few points, as observed below.&lt;/p&gt;&#xA;&lt;p&gt;Obviously there is a trend for the data to go down, and I would expect to see that.&lt;/p&gt;&#xA;&lt;p&gt;Is this because my architecture is not sophisticated enough / not the right one to pick up on the general decline of the known data? Have I inappropriately chosen any parameters?&lt;/p&gt;&#xA;&lt;p&gt;I have tried increasing the number of neurons in the dense layer, units in the LSTM cell, etc. At the moment, the thing that looks like to most effect the resultant curve is to change the &lt;code&gt;length&lt;/code&gt; parameter in my code above. But all this does is make the predictions more sinusoidal.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/k20Tr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k20Tr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your help!&lt;/p&gt;&#xA;" OwnerUserId="40293" LastActivityDate="2020-08-14T04:49:50.107" Title="Time Series Forecasting - Recurrent Neural Networks (tensorflow)" Tags="&lt;recurrent-neural-networks&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23044" PostTypeId="1" CreationDate="2020-08-14T06:59:09.167" Score="3" ViewCount="95" Body="&lt;p&gt;I have a neural network (which I am making from scratch). In order to make the neural network &amp;quot;learn&amp;quot; I need to conduct back-propagation. Using the code at the below how would I conduct back-propagation.&lt;/p&gt;&#xA;&lt;p&gt;FYI: I have tried using the &lt;code&gt;tf.train&lt;/code&gt; functions and &lt;code&gt;.minimize(loss)&lt;/code&gt; but I got this error message: &lt;em&gt;The f passed in variableGrads(f) must be a function&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;var X = tf.tensor([[1,2,3], [4,5,6], [7,8,9]])&#xA;var Y = tf.tensor([[0,0,1]])&#xA;var m = X.shape[0]&#xA;var a0 = tf.zeros([1,3])&#xA;&#xA;var parameters = {&#xA;    &amp;quot;Wax&amp;quot;: tf.randomUniform([3,3]),&#xA;    &amp;quot;Waa&amp;quot;: tf.randomUniform([3,3]),&#xA;    &amp;quot;ba&amp;quot;: tf.zeros([1,3]),&#xA;    &amp;quot;Wya&amp;quot;: tf.randomUniform([3,3]),&#xA;    &amp;quot;by&amp;quot;: tf.zeros([1,3])&#xA;}&#xA;&#xA;&#xA;&#xA;function RNN_cell_Foward(xt, a_prev, parameters){&#xA;    var Wax = parameters[&amp;quot;Wax&amp;quot;]&#xA;    var Waa = parameters[&amp;quot;Waa&amp;quot;]&#xA;    var ba = parameters[&amp;quot;ba&amp;quot;]&#xA;&#xA;    var a_next = tf.sigmoid(tf.add(tf.add(tf.matMul(xt, Wax), tf.matMul(a_prev , Waa)),ba) )&#xA;&#xA;    return a_next&#xA;}&#xA;function RNN_FowardProp(X, a0, parameters){&#xA;    var T_x  = X.shape[0]&#xA;    var a_next = a0&#xA;    var i = 1&#xA;    var Wya = parameters[&amp;quot;Wya&amp;quot;]&#xA;    var by = parameters[&amp;quot;by&amp;quot;]&#xA;    &#xA;    for(; i &amp;lt;= T_x; i++){&#xA;        var xt = X.slice([i-1,0],[1,-1])&#xA;        a_next = RNN_cell_Foward(xt, a_next, parameters)&#xA;    }&#xA;    var y_pred = tf.sigmoid(tf.add(tf.matMul(a_next, Wya), by))&#xA;  return y_pred&#xA;}&#xA;const learningRate = 0.01;&#xA;var optimizer = tf.train.sgd(learningRate);&#xA;var model = RNN_FowardProp(X, a0, parameters)&#xA;var loss = tf.losses.meanSquaredError(Y, model)&#xA;for (let i = 0; i &amp;lt; 10; i++) {&#xA;    optimizer.minimize(loss)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="32636" LastEditDate="2020-08-14T09:32:41.103" LastActivityDate="2020-08-16T07:00:11.290" Title="Doing backpropagation in an Tensorflow.js Neural Network" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;&lt;javascript&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23046" PostTypeId="2" ParentId="23036" CreationDate="2020-08-14T08:56:08.033" Score="3" Body="&lt;p&gt;You can use &lt;strong&gt;Collaborative Filtering&lt;/strong&gt;, and specifically its &lt;em&gt;&lt;code&gt;memory based approach&lt;/code&gt;&lt;/em&gt;.&#xA;The problem that you have discussed in the question should probably be solved using &lt;em&gt;User-Item collaborative filtering&lt;/em&gt; which will calculate similarity between users and then recommend the item. The similarity can be be calculated using cosine similarity or Pearson's similarity formulae. &lt;br&gt;&lt;br&gt;&#xA;The best thing about this approach is that no training is required here but if you wish to have very large data, this approach's performance decreases.&lt;/p&gt;&#xA;" OwnerUserId="35612" LastActivityDate="2020-08-14T08:56:08.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23049" PostTypeId="2" ParentId="22010" CreationDate="2020-08-14T10:42:11.753" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.pearson.com/us/higher-education/program/Graesser-Foundations-of-Deep-Reinforcement-Learning-Theory-and-Practice-in-Python/PGM2027228.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations of Deep Reinforcement Learning: Theory and Practice in Python (Addison-Wesley Data &amp;amp; Analytics Series) 1st Edition&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This book does not give a detailed background information on Markov Decision Processes, different Bellman equations and relationships between the value function and action-value function, etc. It focuses on Deep Reinforcement Learning and goes straight to Policy and Value - based algorithms using neural networks. It might be good for someone trying to quickly understand what Deep RL algorithms are out there and apply them.&lt;/p&gt;&#xA;" OwnerUserId="37627" LastActivityDate="2020-08-14T10:42:11.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23050" PostTypeId="1" CreationDate="2020-08-14T11:01:10.740" Score="0" ViewCount="43" Body="&lt;p&gt;I am trying to understand RPN network in Faster RCNN.&lt;/p&gt;&#xA;&lt;p&gt;I understand the concept of RPN network,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pass the input images to the pre trained CNN, and get the output as feature maps&lt;/li&gt;&#xA;&lt;li&gt;Make fixed size of the feature maps&lt;/li&gt;&#xA;&lt;li&gt;Extract anchors (3 different scales and ratio for every sliding window) from the fixed size feature maps.&lt;/li&gt;&#xA;&lt;li&gt;Use two 1×1 Fully connected NN to find the background or object and the bounding box coordinates (4 values)&lt;/li&gt;&#xA;&lt;li&gt;Calculate IOU for Anchors bounding box with Ground Truth bounding box, if IOU&amp;gt;0.7, then the anchor has object, otherwise, the anchor has background.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The theme for RPN is to give the region proposals which have objects.&lt;/p&gt;&#xA;&lt;p&gt;But, I do not understand the input and the output structure.&lt;/p&gt;&#xA;&lt;p&gt;For example, I have 50 images, each images having 5 to 6 objects, and labeling informations(coordinates of each objects).&lt;/p&gt;&#xA;&lt;p&gt;How do I generate target values, to train PRN Network...&lt;/p&gt;&#xA;&lt;p&gt;In all the blogs, they shows the architecture as feed the entire image to the pre trained CNN.&lt;/p&gt;&#xA;&lt;p&gt;And, the output of RPN, the model has to tell whether the anchor has object or not, and also predict the bounding box for the object in the anchor.&lt;/p&gt;&#xA;&lt;p&gt;For this, how to prepare the input and target/output values like we do in dog/cat or dog/cat/car classification problem.&lt;/p&gt;&#xA;&lt;p&gt;Let me correct if I am not correct,&lt;/p&gt;&#xA;&lt;p&gt;Is that, we have to crop all the objects in every image and do binary classification as object vs background for classifying the anchor has object or not&lt;/p&gt;&#xA;&lt;p&gt;And, Is that, we have to give the ground truth value as target for every cropped objects from all images in the dataset, so that the RPN network trained well to predict the bounding box for the object in every anchor.&lt;/p&gt;&#xA;&lt;p&gt;Hope, I clearly explained my doubts.&lt;/p&gt;&#xA;&lt;p&gt;Help me to learn this concept, Thank you&lt;/p&gt;&#xA;" OwnerUserId="40298" LastActivityDate="2020-08-14T11:01:10.740" Title="Faster RCNN-RPN Network Training" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23051" PostTypeId="2" ParentId="22951" CreationDate="2020-08-14T16:57:43.480" Score="1" Body="&lt;p&gt;You can take some ideas from &lt;a href=&quot;https://www.youtube.com/watch?v=JgvyzIkgxF0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; YouTube video .&lt;/p&gt;&#xA;&lt;p&gt;In addition, you should consider &lt;a href=&quot;http://karpathy.github.io/2016/05/31/rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;that&lt;/a&gt; page which is about Deep Reinforcement Learning used in a game (Pong from Pixels) .&lt;/p&gt;&#xA;" OwnerUserId="32076" LastActivityDate="2020-08-14T16:57:43.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23052" PostTypeId="1" CreationDate="2020-08-14T17:01:35.953" Score="0" ViewCount="39" Body="&lt;p&gt;I am working on making an app that would require the ratio of the height and the largest width of a person, in order to group the individual into a certain body fat percentage category.&#xA;In short, I want the body measurements of them by inputting their full body photos.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me out with what technology I should study and research on to implement this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="40305" LastActivityDate="2021-02-13T11:05:50.657" Title="Technology for predicting body measurements of a person, with a full body photo of them" Tags="&lt;computer-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23054" PostTypeId="2" ParentId="23021" CreationDate="2020-08-14T18:36:01.103" Score="1" Body="&lt;p&gt;Policy iteration is based on the insight that for a &lt;em&gt;given&lt;/em&gt; policy, it is straightforward to compute the value function (the long-run expected discounted value of being in a given stage) &lt;em&gt;exactly&lt;/em&gt; -- it is a set of linear equations at that point.  So, we update the policy, then calculate the &lt;em&gt;exact&lt;/em&gt; values of the states for always following that particular policy, and based on that we update the policy again, etc.&lt;/p&gt;&#xA;&lt;p&gt;Value iteration, in contrast, does not use that insight.  It just updates estimates of the values of being in the states one step at a time.  If these values are initialized at 0, you can think of this of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th iteration computing the value of what would be the optimal policy &lt;em&gt;if we knew the MDP would end after &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; iterations&lt;/em&gt;.  We never really have to think explicitly about policies (though we are in effect computing a policy each iteration), and never directly calculate the infinite sum of expected discounted rewards.&lt;/p&gt;&#xA;&lt;p&gt;These are just the vanilla variants and it is possible to mix and match these ideas -- e.g., you might not evaluate a policy by explicitly solving a system of linear equations but rather just do some iterations -- but the vanilla variants are clearly distinct.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2020-08-14T18:36:01.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23055" PostTypeId="1" CreationDate="2020-08-14T20:42:44.307" Score="0" ViewCount="19" Body="&lt;p&gt;I have seen conditional GANs often applied to easier datasets like MNIST and CIFAR-10 to reasonable success, but at the same time these datasets are simple enough that naïve CNNs can fairly easily max out performance on the training set. Are there any examples of training GANs to generate features and labels for more sophisticated datasets like CIFAR-100, in which even a decent-sized vanilla CNN may converge to 70-80% performance? What kinds of discriminator/generator architectures does this involve?&lt;/p&gt;&#xA;" OwnerUserId="37277" LastActivityDate="2020-08-14T20:42:44.307" Title="Conditional GAN on harder datasets" Tags="&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23056" PostTypeId="1" CreationDate="2020-08-15T09:46:39.343" Score="0" ViewCount="43" Body="&lt;p&gt;I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW parameters, but the same (HxWxC) form.&lt;/p&gt;&#xA;&lt;p&gt;The scenario I have is that I have 4 gray pictures of bulb samples (yielding similar images from each side), which I overlay as channels, but a possible failure that needs to be detected might only appear on one side (a bulb has 4 images and a single classification). The rotation of the object when the picture is taken is arbitrary. Now I solve this by shuffling the channels at training, but it would be more efficient if I could just bind the kernel parameters. Pytorch and Tensorflow solutions are both welcome.&lt;/p&gt;&#xA;" OwnerUserId="40315" LastActivityDate="2021-05-13T15:07:15.690" Title="How can I implement 2D CNN filter with channelwise-bound kernel weights?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23057" PostTypeId="1" AcceptedAnswerId="23058" CreationDate="2020-08-15T12:30:38.393" Score="3" ViewCount="126" Body="&lt;p&gt;Here's a screenshot of the popular policy-gradient algorithm from Sutton and Barto's book -&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YsOS1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YsOS1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I understand the mathematical derivation of the update rule - but I'm not able to build intuition as to why this algorithm should work in the first place. What really bothers me is that we start off with an &lt;strong&gt;incorrect policy&lt;/strong&gt; (i.e. we don't know the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; yet), and we use this policy to generate episodes and do consequent updates.&lt;/p&gt;&#xA;&lt;p&gt;Why should &lt;strong&gt;REINFORCE work at all?&lt;/strong&gt; After all, the episode it uses for the gradient update is generated using the policy that is parametrized by parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; &lt;em&gt;which are yet to be updated&lt;/em&gt; (the episode isn't generated using the &lt;strong&gt;optimal policy&lt;/strong&gt; - there's no way we can do that).&lt;/p&gt;&#xA;&lt;p&gt;I hope that my concern is clear and I request y'all to provide some intuition as to why this works! I suspect that, &lt;strong&gt;somehow&lt;/strong&gt;, even though we are sampling an episode from the wrong policy, we get closer to the right one after each update (monotonic improvement). Alternatively, we could be going closer to the optimal policy (optimal set of parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) on average.&lt;/p&gt;&#xA;&lt;p&gt;So, what's really going on here?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-08-15T12:54:54.427" LastActivityDate="2020-08-15T21:52:54.493" Title="Why does REINFORCE work at all?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;reinforce&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23058" PostTypeId="2" ParentId="23057" CreationDate="2020-08-15T14:16:04.453" Score="4" Body="&lt;p&gt;The key to REINFORCE working is the way the parameters are shifted towards &lt;span class=&quot;math-container&quot;&gt;$G \nabla \log \pi(a|s, \theta)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$ \nabla \log \pi(a|s, \theta) = \frac{ \nabla \pi(a|s, \theta)}{\pi(a|s, \theta)}$&lt;/span&gt;. This makes the update quite intuitive - the numerator shifts the parameters in the direction that gives the highest increase in probability that the action will be repeated, given the state, proportional to the returns - this is easy to see because it is essentially a gradient ascent step. The denominator controls for actions that would have an advantage over other actions because they would be chosen more frequently, by inversely scaling with respect to the probability of the action being taken; imagine if there had been high rewards but the action at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; has low probability of being selected (e.g. 0.1) then this will multiply the returns by 10 leading to a larger update step in the direction that would increase the probability of this action being selected the most (which is what the numerator controls for, as mentioned).&lt;/p&gt;&#xA;&lt;p&gt;That is for the intuition -- to see why it &lt;em&gt;does&lt;/em&gt; work, then think about what we've done. We defined an objective function, &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;, that we are interested in maximising with respected to our parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. We find the derivative of this objective with respect to our parameters, and then we perform gradient ascent on our parameters to maximise our objective, i.e. to maximise &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt;, thus if we keep performing gradient ascent then our policy parameters will converge (eventually) to values that maximise &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; and thus our policy would be optimal.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-08-15T21:52:54.493" LastActivityDate="2020-08-15T21:52:54.493" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23059" PostTypeId="2" ParentId="3397" CreationDate="2020-08-15T15:56:27.760" Score="0" Body="&lt;p&gt;Amazingly, I just found a claim about that. I just read it in twitter and the model is simply a GPT-2 WITH 355M params trained with 200,000 raw title and body-based jokes. what is amazing is that GPT-2 is the most advanced text generating model it even can translate or answer math questions if trained well.&lt;/p&gt;&#xA;&lt;p&gt;Let's see example output from twitter.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;quot;I asked my girlfriend if she knew what sex was like || she said that&#xA;you can kiss her and she'll think you're a queer.&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;Why does the teacher have her own car? || She's a car company for Santa.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://i.stack.imgur.com/rMEYN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rMEYN.png&quot; alt=&quot;Examples&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/lgbtinethiopia/status/1294644776772472834?s=20&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/lgbtinethiopia/status/1294644776772472834?s=20&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40320" LastActivityDate="2020-08-15T15:56:27.760" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23060" PostTypeId="2" ParentId="22986" CreationDate="2020-08-15T15:57:09.080" Score="0" Body="&lt;p&gt;I think the problem is with openAI gym CartPole-v0 environment reward structure.&#xA;The reward is always +1 for each time step. So if pole falls reward is +1 itself.&#xA;So we need to check and redefine the reward for this case.&#xA;So in the train function try this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if not done:&#xA;    new_q = reward + DISCOUNT * np.max(future_qs_list)&#xA;else:&#xA;    # if done assign some negative reward&#xA;    new_q = -20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Or change the reward during replay buffer update)&lt;/p&gt;&#xA;&lt;p&gt;Check the lines 81 and 82 in Qlearning.py code in &lt;a href=&quot;https://github.com/girishdhegde/reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; repo for further clarification.&lt;/p&gt;&#xA;" OwnerUserId="40321" LastEditorUserId="40321" LastEditDate="2020-08-16T04:37:40.397" LastActivityDate="2020-08-16T04:37:40.397" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23061" PostTypeId="1" CreationDate="2020-08-15T19:32:02.320" Score="0" ViewCount="47" Body="&lt;p&gt;I am trying to implement &lt;a href=&quot;https://www.kdd.org/kdd2016/papers/files/rfp0191-wangAemb.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SDNE&lt;/a&gt;, a algorithm uses deep auto encoder to map a graph to latent representation &lt;code&gt;d&lt;/code&gt; dimension.&lt;/p&gt;&#xA;&lt;p&gt;The idea is kind of simple, SDNE uses the adjacency matrix as input and output is embedding layer (also node embedding of graph). The loss function tries to minimize the reconstruction layer and input layer which defined &lt;span class=&quot;math-container&quot;&gt;$\hat{X}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. To keeping node embedding &lt;code&gt;Y&lt;/code&gt; (embedding layer) is close together like the original node in graph input, loss function continue minimizes &lt;span class=&quot;math-container&quot;&gt;$||y_i - y_j||*s_{ij}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$s_{ij}$&lt;/span&gt; is weight of node &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_j$&lt;/span&gt;, if not link &lt;span class=&quot;math-container&quot;&gt;$s_{ij} = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To sum up, loss function like this:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;   L = \sum_{i=1}^{n}{||(\hat{x_i} -x_i) \odot{b_i}||_2^2} + \sum_{i,j=1}^{n} {s_{i,j}||y_i -y_j||_2^2}&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Leftrightarrow L = ||(\hat{X} - X) \odot B||_F^2 + 2tr(Y^TLY)&#xA;$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$L=D-S, D\in \mathbb{R}^{n\times n}$&lt;/span&gt; is diagonal matrix, &lt;span class=&quot;math-container&quot;&gt;$D_{i,i} = \sum_{j}{s_{ij}}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is call Laplacian matrix.&lt;/p&gt;&#xA;&lt;p&gt;When I implement code in PyTorch, I got &lt;strong&gt;negative&lt;/strong&gt; loss value although the loss function does not negative value.&lt;/p&gt;&#xA;&lt;p&gt;This is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def _compute_loss(self, x, x_hat, y, L):&#xA;        def loss_1st(Y, L):&#xA;            Y_ = torch.matmul(torch.transpose(Y, 0, 1), L)&#xA;            return 2 * torch.trace(torch.matmul(Y_, Y))&#xA;&#xA;        def loss_2nd(X_hat, X, beta):&#xA;            B = torch.ones_like(X).to(device)&#xA;            B[X != 0] = beta&#xA;            return torch.sum(torch.square((X_hat - X) * B))&#xA;&#xA;        batch_size = x.shape[0]&#xA;        # TODO: check if divide batch_size&#xA;        loss_1 = loss_1st(y, L)&#xA;        loss_2 = loss_2nd(x_hat, x, self.beta)&#xA;        loss = loss_2 + self.alpha * loss_1&#xA;        return loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I trained on all batch dataset which means &lt;code&gt;batch_size = len(Adjancency_matrix) = len(A)&lt;/code&gt;.&#xA;Actually, I'm not sure about this algorithm can train on &lt;code&gt;batch_size&lt;/code&gt; = 64, 128,.. because I can not calcualte &lt;code&gt;L&lt;/code&gt; on mini batch and mini batch also can not represent for the whole graph.&lt;/p&gt;&#xA;&lt;p&gt;So, can you see in my code what reason why my loss function is &lt;strong&gt;negative&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="25645" LastActivityDate="2020-08-15T19:32:02.320" Title="Why does my loss value of autoencoder in PyTorch is negative?" Tags="&lt;deep-learning&gt;&lt;pytorch&gt;&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23062" PostTypeId="2" ParentId="22986" CreationDate="2020-08-15T22:54:23.440" Score="3" Body="&lt;p&gt;There is a really small mistake in here that causes the problem: &lt;br/&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;for index, (current_state, action, reward, next_state, done) in enumerate(minibatch):&#xA;            if not done:&#xA;                new_q = reward + DISCOUNT * np.max(future_qs_list) #HERE &#xA;            else:&#xA;                new_q = reward&#xA;                &#xA;            # Update Q value for given state&#xA;            current_qs = current_qs_list[index]&#xA;            current_qs[action] = new_q&#xA;            &#xA;            X.append(current_state)&#xA;            Y.append(current_qs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since np.max(future_qs_list) should be np.max(future_qs_list[index]) since you're now getting the highest Q of the entire batch.&#xA;Instead of the getting the highest Q from the current next state.&lt;/p&gt;&#xA;&lt;p&gt;It's like this after changing that (remember an epsilon of 1 means that you get 100% of your actions taken by the a dice roll so I let it go for a few more epochs, also tried it with the old code but indeed didn't get more then 50 steps (even after 400 epochs/episodes))&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;episode : 52, steps 16, epsilon : 0.7705488893118823&#xA;episode : 53, steps 25, epsilon : 0.7666961448653229&#xA;episode : 54, steps 25, epsilon : 0.7628626641409962&#xA;episode : 55, steps 36, epsilon : 0.7590483508202912&#xA;episode : 56, steps 32, epsilon : 0.7552531090661897&#xA;episode : 57, steps 22, epsilon : 0.7514768435208588&#xA;episode : 58, steps 55, epsilon : 0.7477194593032545&#xA;episode : 59, steps 24, epsilon : 0.7439808620067382&#xA;episode : 60, steps 46, epsilon : 0.7402609576967045&#xA;episode : 61, steps 11, epsilon : 0.736559652908221&#xA;episode : 62, steps 14, epsilon : 0.7328768546436799&#xA;episode : 63, steps 13, epsilon : 0.7292124703704616&#xA;episode : 64, steps 113, epsilon : 0.7255664080186093&#xA;episode : 65, steps 33, epsilon : 0.7219385759785162&#xA;episode : 66, steps 33, epsilon : 0.7183288830986236&#xA;episode : 67, steps 39, epsilon : 0.7147372386831305&#xA;episode : 68, steps 27, epsilon : 0.7111635524897149&#xA;episode : 69, steps 22, epsilon : 0.7076077347272662&#xA;episode : 70, steps 60, epsilon : 0.7040696960536299&#xA;episode : 71, steps 40, epsilon : 0.7005493475733617&#xA;episode : 72, steps 67, epsilon : 0.697046600835495&#xA;episode : 73, steps 115, epsilon : 0.6935613678313175&#xA;episode : 74, steps 61, epsilon : 0.6900935609921609&#xA;episode : 75, steps 43, epsilon : 0.6866430931872001&#xA;episode : 76, steps 21, epsilon : 0.6832098777212641&#xA;episode : 77, steps 65, epsilon : 0.6797938283326578&#xA;episode : 78, steps 45, epsilon : 0.6763948591909945&#xA;episode : 79, steps 93, epsilon : 0.6730128848950395&#xA;episode : 80, steps 200, epsilon : 0.6696478204705644&#xA;episode : 81, steps 200, epsilon : 0.6662995813682115&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30100" LastEditorUserId="30100" LastEditDate="2020-08-15T23:26:19.590" LastActivityDate="2020-08-15T23:26:19.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23063" PostTypeId="2" ParentId="23026" CreationDate="2020-08-15T23:51:14.987" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y_{t}^{\text {DoubleDQN }} \equiv R_{t+1}+\gamma Q\left(S_{t+1}, \underset{a}{\operatorname{argmax}} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}\right), \boldsymbol{\theta}_{t}^{-}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only difference between the &amp;quot;original&amp;quot; DQN and this one is that you use your &lt;span class=&quot;math-container&quot;&gt;$Q_\text{est}$&lt;/span&gt; with the next state to get your action (by choosing the action with the highest Q).&lt;/p&gt;&#xA;&lt;p&gt;Afterward, you just figure out what the target &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is given that action, by selecting the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; belonging to that action from the target_network (instead of using the argmax a directly on the target Q network).&lt;/p&gt;&#xA;&lt;p&gt;About the formula&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta_{t}^{-}$&lt;/span&gt; above it means frozen weights, so it represents the &lt;em&gt;target Q network&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the other &lt;span class=&quot;math-container&quot;&gt;$\theta_{t}$&lt;/span&gt; represents the &amp;quot;learnable weights&amp;quot; so the &lt;em&gt;estimate Q network&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30100" LastEditorUserId="2444" LastEditDate="2020-08-17T00:13:03.097" LastActivityDate="2020-08-17T00:13:03.097" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23064" PostTypeId="2" ParentId="23019" CreationDate="2020-08-16T04:55:21.363" Score="-2" Body="&lt;p&gt;since the reward is a sum of the states, the amount of times you need to query the expert is only dependent on the number of reward features (# of sum totals) whether it summed 5 or 10 states doesnt affect the number of reward values you need to query since once summed that is the value you are getting your loss from. succinctly once summed 5 or 100 states for 5 features is still just 5 values&lt;/p&gt;&#xA;" OwnerUserId="20044" LastActivityDate="2020-08-16T04:55:21.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23065" PostTypeId="1" CreationDate="2020-08-16T09:30:18.880" Score="6" ViewCount="126" Body="&lt;p&gt;I have two functions &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt;, and each of them can be computed with a neural network &lt;span class=&quot;math-container&quot;&gt;$\phi_f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi_g$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is, how can I write a neural net for &lt;span class=&quot;math-container&quot;&gt;$f(x)g(x)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;So, for example, if &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; is constant and equal to &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;  and &lt;span class=&quot;math-container&quot;&gt;$\phi_f = ((A_1,b_1),...(A_L,b_L))$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\phi_{fg} = ((A_1,b_1),...,(cA_L,cb_L))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Actually, I need to show it for &lt;span class=&quot;math-container&quot;&gt;$f(x)=x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)=x^2$&lt;/span&gt; if this make something easier.&lt;/p&gt;&#xA;" OwnerUserId="40330" LastEditorUserId="2444" LastEditDate="2020-08-17T23:46:33.460" LastActivityDate="2020-08-23T17:03:06.497" Title="Given two neural networks that compute two functions $f(x)$ and $g(x)$, how can I create a neural network that computes $f(x)g(x)$?" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23066" PostTypeId="2" ParentId="23065" CreationDate="2020-08-16T11:44:27.610" Score="1" Body="&lt;p&gt;Use another network &lt;em&gt;&lt;strong&gt;h&lt;/strong&gt;&lt;/em&gt; which takes &lt;em&gt;&lt;strong&gt;f(x)&lt;/strong&gt;&lt;/em&gt; and &lt;em&gt;&lt;strong&gt;g(x)&lt;/strong&gt;&lt;/em&gt; as input i.e. &lt;em&gt;&lt;strong&gt;h(f(x), g(x))&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Training psuedo code(pytorch):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for epoch in epochs:&#xA;    for batch, x in dataset:&#xA;        train(f)&#xA;        train(g)&#xA;&#xA;&#xA;for epoch in epochs:&#xA;    for batch, x in dataset:&#xA;        # freeze f and g (using torch.no_grad in pytorch)&#xA;        with torch.no_grad:&#xA;            fx = f(x)&#xA;            gx = g(x)&#xA;        pred = h(fx, gx)&#xA;        loss = loss_fn(pred, (fx . gx))&#xA;        backpropagate()&#xA;        optimise()&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40321" LastActivityDate="2020-08-16T11:44:27.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23067" PostTypeId="1" CreationDate="2020-08-16T12:47:40.163" Score="2" ViewCount="67" Body="&lt;p&gt;Recently I came along the paper  &lt;a href=&quot;https://proceedings.icml.cc/static/paper_files/icml/2020/5945-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robust and Stable Black Box Explanations&lt;/a&gt;, which discusses a nice framework for global model-agnostic explanations.&lt;/p&gt;&#xA;&lt;p&gt;I was thinking to recreate the experiments performed in the paper, but unfortunately authors haven't provided the code. The summary of the experiments are :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;use LIME, SHAP and MUSE as baseline models, and compute fidelity score on test data. (All the 3 datasets are used for classification problems)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;since LIME and SHAP give local explanations, for a particular data point, the idea is to use K points from the training dataset, and create K explanations using LIME. LIME is supposed to return a local linear explanation. Now, for a new test data point, using the nearest point from K points used earlier and use the corresponding explanation to classify this new point.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;measure the performance, using fidelity score (% of points for which E(x) = B(x), where E(x) is the explanation of the point and B(x) is the classification of the point using black box.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now, the issue is, I am using LIME and SHAP packages in Python to achieve the results on baseline models. But, I am not sure how I'll get linear explanation for a point (one from the set K), and use to to classify a new test point in the neighborhood. Every tutorial on YouTube and Medium, discusses about visualizing the explanation for a given point but none talks about how to get the linear model itself and use it for newer points.&lt;/p&gt;&#xA;" OwnerUserId="40332" LastEditorUserId="11539" LastEditDate="2020-10-27T01:29:17.513" LastActivityDate="2020-10-27T01:29:17.513" Title="Black Box Explanations: Using LIME and SHAP in python" Tags="&lt;machine-learning&gt;&lt;explainable-ai&gt;&lt;black-box&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23068" PostTypeId="2" ParentId="23056" CreationDate="2020-08-16T13:41:12.503" Score="0" Body="&lt;p&gt;Assuming you want HxWx1 kernel to perform convolution on hxwxc images.&lt;/p&gt;&#xA;&lt;p&gt;Here's sample code which uses single channel kernel to operate on multichannel feature: maps&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;&#xA;&#xA;class model(nn.Module):&#xA;    def __init__(self, in_ch=4):&#xA;        super().__init__()&#xA;        self.in_ch  = in_ch&#xA;        # single channel kernel initialization&#xA;        self.kernel = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=1, stride=1)&#xA;&#xA;    def forward(self, x):&#xA;        bs, ch, w, h = x.shape&#xA;        x   = x.view((bs, ch, 1, w, h))&#xA;        out = self.kernel(x[:, 0])&#xA;        # reusing of same kernel&#xA;        for i in range(1, self.in_ch):&#xA;            out = torch.cat((out, self.kernel(x[:, i])), 1)&#xA;        return out&#xA;&#xA;net = model(4)&#xA;print(net)&#xA;inp = torch.randn((10, 4, 100, 100))&#xA;out = net(inp)&#xA;print(out.shape)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(The main hack is in the &lt;code&gt;forward&lt;/code&gt; function)&lt;/p&gt;&#xA;" OwnerUserId="40321" LastActivityDate="2020-08-16T13:41:12.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23071" PostTypeId="1" CreationDate="2020-08-16T21:45:52.253" Score="1" ViewCount="21" Body="&lt;p&gt;I have an RBM model which takes extremely long to train and evaluate because of the large number of free parameters and the large amount of input data. What would be the most efficient way of tuning its hyperparameters (batch size, number of hidden units, learning rate, momentum and weight decay)?&lt;/p&gt;&#xA;" OwnerUserId="37354" LastActivityDate="2020-08-16T21:45:52.253" Title="Best/quickest approach for tuning the hyperparameters of a restricted boltzmann machine" Tags="&lt;hyperparameter-optimization&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23072" PostTypeId="1" AcceptedAnswerId="23074" CreationDate="2020-08-16T21:58:50.113" Score="3" ViewCount="1284" Body="&lt;p&gt;Q-learning seems to be related to A*. I am wondering if there are (and what are) the differences between them.&lt;/p&gt;&#xA;" OwnerUserId="10135" LastEditorUserId="2444" LastEditDate="2020-08-16T22:10:31.200" LastActivityDate="2020-08-17T14:56:43.540" Title="What are the differences between Q-Learning and A*?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;a-star&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23073" PostTypeId="2" ParentId="50" CreationDate="2020-08-16T23:41:29.197" Score="1" Body="&lt;p&gt;Error Estimation is a subject with a long history. The test-set method is only one way to estimate generalization error. Others include resubstitution, cross-validation, bootstrap, posterior-probability estimators, and bolstered estimators. These and more are reviewed, for instance, in the book: Braga-Neto and Dougherty, &amp;quot;Error Estimation for Pattern Recognition,&amp;quot; IEEE-Wiley, 2015.&lt;/p&gt;&#xA;" OwnerUserId="40338" LastActivityDate="2020-08-16T23:41:29.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23074" PostTypeId="2" ParentId="23072" CreationDate="2020-08-17T00:26:37.047" Score="16" Body="&lt;p&gt;Q-learning and A* can both be viewed as search algorithms, but, apart from that, they are not very similar.&lt;/p&gt;&#xA;&lt;p&gt;Q-learning is a &lt;strong&gt;reinforcement learning&lt;/strong&gt; algorithm, i.e. an algorithm that attempts to find a &lt;em&gt;policy&lt;/em&gt; or, more precisely, &lt;strong&gt;value function&lt;/strong&gt; (from which the policy can be derived) by taking stochastic moves (or actions) with some policy (which is different from the policy you want to learn), such as the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy, given the current estimate of the &lt;em&gt;value function&lt;/em&gt;. Q-learning is a &lt;strong&gt;numerical&lt;/strong&gt; (and stochastic optimization) algorithm that can be shown to converge to the optimal solution in the tabular case (but it does not necessarily converge when you use a function approximator, such as a neural network, to represent the value function). Q-learning can be viewed as a search algorithm, where the solutions are value functions (or policies) and the search space is some space of value functions (or policies).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, A* is a general &lt;strong&gt;search algorithm&lt;/strong&gt; that can be applied to any search problem where the search space can be represented as a &lt;strong&gt;graph&lt;/strong&gt;, where nodes are positions (or locations) and the edges are the weights (or costs) between these positions. A* is an &lt;strong&gt;informed&lt;/strong&gt; search algorithm, given that you can use an (informed) heuristic to guide the search, i.e. you can use domain knowledge to guide the search. A* is a &lt;em&gt;best-first search&lt;/em&gt; (BFS) algorithm, which is a family of search algorithms that explore the search space by following the next best location according to some objective function, which varies depending on the specific BFS algorithm. For example, in the case of A*, the objective function is &lt;span class=&quot;math-container&quot;&gt;$f(n) = h(n) + g(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is a node, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the heuristic function and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; the function that calculates the cost of the path from the starting node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. A* is also known to be optimal (provided that the heuristic function is &lt;em&gt;admissible&lt;/em&gt;)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-17T14:56:43.540" LastActivityDate="2020-08-17T14:56:43.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23075" PostTypeId="1" CreationDate="2020-08-17T03:17:59.787" Score="4" ViewCount="99" Body="&lt;p&gt;I'm trying to get a better understanding of Multi-Arm Bandits, Contextual Multi-Arm Bandits and Markov Decision Process.&lt;/p&gt;&#xA;&lt;p&gt;Basically, Multi-Arm Bandits is a special case of Contextual Multi-Arm Bandits where there is no state(features/context). And Contextual Multi-Arm Bandits is a special case of Markov Decision Process, where there is only one state (features, but no transitions).&lt;/p&gt;&#xA;&lt;p&gt;However, since MDP has Markov property, I wonder if every MDP problem can also be converted into a Contextual Multi-Arm Bandits problem, if we simply treat each state as a different input context (features)?&lt;/p&gt;&#xA;" OwnerUserId="40340" LastEditorUserId="2444" LastEditDate="2020-08-17T23:54:11.937" LastActivityDate="2021-01-16T19:02:30.220" Title="Can you convert a MDP problem to a Contextual Multi-Arm Bandits problem?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;markov-decision-process&gt;&lt;multi-armed-bandit&gt;&lt;contextual-bandit&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23076" PostTypeId="1" AcceptedAnswerId="23078" CreationDate="2020-08-17T03:34:16.450" Score="1" ViewCount="30" Body="&lt;p&gt;I have made an RNN from scratch in Tensorflow.js. In order to update my weights (without needing to calculate the derivatives), I thought of using the normal equation to find the optimal values for my RNN's weights. Would you recommend this approach and if not why?&lt;/p&gt;&#xA;" OwnerUserId="32636" LastEditorUserId="2444" LastEditDate="2020-08-18T10:11:46.850" LastActivityDate="2020-08-18T10:11:46.850" Title="Can the normal equation be used to optimise the RNN's weights?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23077" PostTypeId="2" ParentId="23052" CreationDate="2020-08-17T06:13:41.963" Score="0" Body="&lt;p&gt;If the user will be imputting the body height/largest width measures, and you want to have a couple of groups where you final results will be stored at, you should look at the classification algorithms(Logistic Regression, KNN, Random Forests).&#xA;If on the other hand, you want your program to detect the user from an image and take the measurements itself, you would need to look into Convolutional Neural Networks, along with person detection/segmentation&lt;/p&gt;&#xA;" OwnerUserId="38919" LastActivityDate="2020-08-17T06:13:41.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23078" PostTypeId="2" ParentId="23076" CreationDate="2020-08-17T07:20:07.450" Score="2" Body="&lt;p&gt;Unfortunately, this is not possible. The normal equation can only directly optimise a single layer that connects input and output. There is no equivalent for multiple layers such as those in any neural network architecture.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-08-17T07:20:07.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23079" PostTypeId="2" ParentId="22969" CreationDate="2020-08-17T07:51:00.977" Score="0" Body="&lt;p&gt;After further investigating the problem I have found the answer:&lt;/p&gt;&#xA;&lt;p&gt;U-net generators' up-sampling stage consists of two steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use &lt;code&gt;UpSampling2D&lt;/code&gt; layer&lt;/li&gt;&#xA;&lt;li&gt;Apply convolution on the output&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The &lt;code&gt;UpSampling2D&lt;/code&gt; layer is in the keras documentation described as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Repeats the rows and columns of the data by size[0] and size[1] respectively.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;From this information, we can calculate the time cost for &lt;code&gt;UpSampling2D&lt;/code&gt; alone. Lets set size to &lt;code&gt;(2,2)&lt;/code&gt;, as is set in basic configuration of the U-net generator. The output of the &lt;code&gt;UpSampling2D&lt;/code&gt; is then doubled. In case we started with &lt;code&gt;(4,4,3)&lt;/code&gt;, where the last index corresponds to number of channels, the output shape will be &lt;code&gt;8,8,3&lt;/code&gt;. We can see that each row and column need to be copied twice in each channel. From this we can define time complexity of a single up-sampling as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;O\left(2 \cdot c \cdot n \cdot s\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;em&gt;c&lt;/em&gt; corresponds to number of channels, &lt;em&gt;n&lt;/em&gt; corresponds to input length (one side of a matrix) and &lt;em&gt;s&lt;/em&gt; is equal to filter size. Assuming that length and filter size have square shape, the complexity is multiplied by 2. Since in this case the the filter size is known, equal to &lt;code&gt;(2,2)&lt;/code&gt;, the notation can be simplified to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;O\left(4 \cdot c \cdot n \right) = O\left(c \cdot n \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In my case, with only 1 channel, the complexity is simply&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;O\left(n \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which means the up-sampling stage is linear, and the only important feature is input size, which is negligible to the complexity of the following convolutional layer and can be ignored.&lt;/p&gt;&#xA;" OwnerUserId="40205" LastActivityDate="2020-08-17T07:51:00.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23080" PostTypeId="2" ParentId="23075" CreationDate="2020-08-17T10:44:18.813" Score="1" Body="&lt;p&gt;The main difference between an MDP and contextual bandit setting is time steps and state progression. If those are important to the problem you want to solve, then it is not possible to convert.&lt;/p&gt;&#xA;&lt;p&gt;Essentially MDPs are a strict generalisation of contextual bandits. You can model a CB as an MDP but not vice-versa.&lt;/p&gt;&#xA;&lt;p&gt;In some very specific cases you can convert MDP to CB - any one of these situations means that the MDP can be simplified to CB, and then you could use bandit solving algorithms to optimise it:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;When there is only one time step for an episodic problem.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When the discount factor is zero.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When state transition rules are completely independent of action choice, but reward is not.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-19T17:14:45.487" LastActivityDate="2020-08-19T17:14:45.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23081" PostTypeId="1" CreationDate="2020-08-17T12:17:39.610" Score="0" ViewCount="21" Body="&lt;p&gt;While working on open source Time Series Data for Grocery sales where my aim is to perform Demand forecast of Grocery Products, I completed the cleaning of data of Grocery sales data, then in order to check whether the Grocery dataset is Stationary or not, I conducted Dickey Fuller statistical test on Historical Sales column of Grocery Dataset&lt;br /&gt;&#xA;and got result that data is Stationary, Pls refer below output of Python code :-&lt;/p&gt;&#xA;&lt;p&gt;ADF Statistic: -29.450181&#xA;p-value: 0.000000&#xA;Critical Values:&#xA;1%: -3.710&#xA;5%: -2.971&#xA;10%: -2.678&#xA;Reject Ho - Time Series is Stationary&lt;/p&gt;&#xA;&lt;p&gt;Now when i visualize the historical Sales on which i conducted Statistical Dickey Fuller Test which gave result as Time Series is Stationary,But Visually it is giving output like a Curve going up and down at regular intervals demonstrating Seasonality.&lt;/p&gt;&#xA;&lt;p&gt;Visually it shows seasonality but when conducted statistical test it says time series data is stationary, so can i proceed ahead without removing Seasonality as i will be using Regression based Models and not Time Series Models to predict the Demand Forecast of Grocery Products.&lt;/p&gt;&#xA;&lt;p&gt;Please Advice.&lt;/p&gt;&#xA;" OwnerUserId="33564" LastEditorUserId="33564" LastEditDate="2020-08-23T07:40:09.623" LastActivityDate="2020-08-23T07:40:09.623" Title="Do we need to conduct statistical time series test if we are using Regression Models(Linear Regression) for Forecasting Demand?" Tags="&lt;time-series&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23082" PostTypeId="1" CreationDate="2020-08-17T12:25:05.197" Score="0" ViewCount="27" Body="&lt;p&gt;When training policies, is there a reason we need on-policy samples? For expensive simulations, it makes sense to try and reuse samples. Say we're interested in hyperparameter tuning. Can we collect a bunch of episodes using randomly sampled actions (or maybe by following an old policy) one time, and train multiple policies using this set of samples to find the most effective hyperparameters? Every time we train a new policy, does it make sense to replay all the episodes generated by the previous policy? I'm mostly interested in actor-critic methods.&lt;/p&gt;&#xA;" OwnerUserId="40354" LastEditorUserId="2444" LastEditDate="2020-08-17T23:50:38.333" LastActivityDate="2020-08-17T23:50:38.333" Title="Learning only using off-policy samples" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;off-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23084" PostTypeId="1" CreationDate="2020-08-17T13:06:22.027" Score="0" ViewCount="19" Body="&lt;p&gt;&lt;strong&gt;1. Context&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm studying Health-Monitoring techniques, and I practice on the &lt;a href=&quot;https://data.nasa.gov/dataset/C-MAPSS-Aircraft-Engine-Simulator-Data/xaut-bemq&quot; rel=&quot;nofollow noreferrer&quot;&gt;C-MAPSS dataset&lt;/a&gt;. The goal is to predict the Remaining Useful Life (RUL) of an engine given sensor measurements series. There's a wide litterature about the C-MAPSS dataset, including both classical (non-DL) ML techniques and DL-based approaches. A few years ago, LSTM-based networks showed promising results (see &lt;a href=&quot;https://www.researchgate.net/publication/318890794_Long_Short-Term_Memory_Network_for_Remaining_Useful_Life_estimation&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Long Short-Term Memory Network for Remaining Useful Life estimation, Zheng et al&lt;/em&gt;, 2017&lt;/a&gt;), and I'm trying to reproduce these results.&lt;/p&gt;&#xA;&lt;p&gt;The C-MAPSS dataset contains a low amount of data. The FD001 subset has for instance only 100 run-to-failure series. When I pre process it to get fixed-length time series, I can get up to ~20 000 framed series. In the article mentioned above using LSTM, they use two hidden LSTM layers with 64 units each, and two fully-connected layers with 8 neurons each (~55 000 parameters).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2. Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;LSTMs induce a great number of parameter, so overfitting may be encountered when training such a network. I can use L1 or L2 regularization, dropouts, the net will still be largely oversized regarding to the dataset. Keeping the same architecture, I can't reach the scores and RMSE in the paper in the validation set, and overfitting is always here.&lt;/p&gt;&#xA;&lt;p&gt;However, one thing that works is reducing the number of units of the LSTM layers. Expectedly, with only 24 units instead of 64 per layer, the net has much less parameters (~9000), and it presents no overfitting. The scores and RMSE are a bit worse than the one in the paper, but it's the best I can get so far. Although these results are fine for me, I'm curious about how it was possible for the authors of the paper to avoid overfitting on their LSTM(64,64) net.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;3. Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;LSTM are great, but they induce a lot of parameters that hinder a correct learning on small dataset : I wonder if there is any method to tackle this specific issue. &lt;strong&gt;Would you have any advice on how to avoid overfitting with a LSTM-based net on a small dataset ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;4. Infos&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I provide here below more infos about my net and results :&lt;/p&gt;&#xA;&lt;p&gt;Network architecture&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = keras.models.Sequential([&#xA;    keras.layers.LSTM(24, return_sequences=True, kernel_regularizer=keras.regularizers.l1(0.01),&#xA;                      input_shape=input_shape),&#xA;    keras.layers.Dropout(0.2),&#xA;    keras.layers.LSTM(24, return_sequences=False, kernel_regularizer=keras.regularizers.l1(0.01)),&#xA;    keras.layers.Dropout(0.2),&#xA;    keras.layers.Dense(8, activation='relu', kernel_regularizer=keras.regularizers.l2()),&#xA;    keras.layers.Dropout(0.2),&#xA;    keras.layers.Dense(8, activation='relu', kernel_regularizer=keras.regularizers.l2(), bias_regularizer=keras.regularizers.l2()),&#xA;    keras.layers.Dense(1, activation='relu')&#xA;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Scores (Validation set)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Paper:  Score = 16.14 ; RMSE = 338&lt;/li&gt;&#xA;&lt;li&gt;My LSTM(64, 64): Score = 26.47; RMSE = 3585 (overfits)&lt;/li&gt;&#xA;&lt;li&gt;My LSTM(24, 24): Score = 16.82; RMSE = 515&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2020-08-17T15:00:22.797" LastActivityDate="2020-08-17T15:00:22.797" Title="Avoiding Overfitting with a large LSTM net on a small amount of data" Tags="&lt;long-short-term-memory&gt;&lt;overfitting&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23086" PostTypeId="2" ParentId="23082" CreationDate="2020-08-17T13:46:51.447" Score="1" Body="&lt;p&gt;What you're describing is off-policy learning. A classic example is &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning, where you follow some policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; whilst learning about the greedy policy.&lt;/p&gt;&#xA;&lt;p&gt;If you're interested in actor-critic methods then a popular off-policy method is the &lt;a href=&quot;https://arxiv.org/pdf/1509.02971.pdf?source=post_page---------------------------&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Deterministic Policy Gradient&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-17T13:46:51.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23087" PostTypeId="1" AcceptedAnswerId="25440" CreationDate="2020-08-17T14:02:43.823" Score="1" ViewCount="199" Body="&lt;p&gt;I am relatively new to reinforcement learning, and I am trying to implement a reinforcement learning algorithm that can do continuous control in a custom environment. The state of the environment is composed of 5 sequential observations of 11 continuous values that represent cells (making the observation space 55 continuous values total), and the action is 11 continuous values representing multipliers of those cell totals.&lt;/p&gt;&#xA;&lt;p&gt;After preliminary research, I decided to use Deep Deterministic Policy Gradient (DDPG) as my control algorithm because of its ability to deal with both discrete states and actions. However, most of the examples, including &lt;a href=&quot;https://keras.io/examples/rl/ddpg_pendulum/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the one that I am basing my implementation off of&lt;/a&gt;, have only a single continuously valued action as the output. I have tried to naively change the agent network from outputting a single value to output a vector of values, but the agent does not improve as all, and the  set of outputs seems to split into two groups near either the maximum value or the minimum value (I believe the tanh activation on the output has something to do with it) with the values in those groups changing in unison.&lt;/p&gt;&#xA;&lt;p&gt;I have two questions about my problems.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;First, is it even possible to use DDPG for multi-dimensional continuous action spaces? My research leads me to believe it is, but I have not found any code examples to learn from and many of the papers I have read are near the limit of my understanding in this area.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Second, why might my actor network be outputting values clustered near its max/min values, and why would the values in either cluster all be the same?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Again, I am fairly new to reinforcement learning, so any advice or recommendations would be greatly appreciated, thanks.&lt;/p&gt;&#xA;" OwnerUserId="38992" LastActivityDate="2020-12-29T10:31:50.990" Title="Using DDPG for control in multi-dimensional continuous action space?" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;ddpg&gt;&lt;control-problem&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23088" PostTypeId="1" CreationDate="2020-08-17T14:03:57.040" Score="0" ViewCount="31" Body="&lt;p&gt;I am building a regression network to predict gravity strength based on meteorological data. I'm getting a fairly good fit, but quite a bit of noise in my output. I can find a lot of info on how to deeal with noisy input, but not so much regarding the output. What methods could I employ to reduce the noise in my network?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/D5G6g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D5G6g.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40358" LastActivityDate="2020-08-17T14:03:57.040" Title="Noise reduction in output from regression network" Tags="&lt;neural-networks&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23089" PostTypeId="1" CreationDate="2020-08-17T14:44:07.343" Score="1" ViewCount="45" Body="&lt;p&gt;I want to use RNN for classifying whole sequences of events, generated by website visitors. Each event has some categorical properties and a Unix timestamp:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence1 = [{'timestamp': 1597501183, 'some_field': 'A'}, {'timestamp': 1597681183, 'some_field': 'B'}]&#xA;sequence2 = [{'timestamp': 1596298782, 'some_field': 'B'}]&#xA;sequence3 = [{'timestamp': 1596644362, 'some_field': 'A'}, {'timestamp': 1596647951, 'some_field': 'C'}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately, they can't be treated as classic time series, because they're of variable length and irregular, so timestamps contain essential information and cannot be ignored. While categorical features can be one-hot encoded or made into embeddings, I'm not sure what to do with the timestamps. It doesn't look like a good idea to use them raw. I've come up with two options so far:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Subtract the minimum timestamp from every timestamp in the sequence, so that all sequences start at 0. But in this case the numbers can still be high, because the sequences run over a month.&lt;/li&gt;&#xA;&lt;li&gt;Use offsets from previous event instead of absolute timestamps.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm wondering if there are common ways to deal with this? I haven't found much on this subject.&lt;/p&gt;&#xA;" OwnerUserId="40359" LastActivityDate="2021-04-16T22:45:55.727" Title="Unix timestamps for Recurrent Neural Networks" Tags="&lt;recurrent-neural-networks&gt;&lt;time-series&gt;&lt;feature-extraction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23090" PostTypeId="1" CreationDate="2020-08-17T16:28:20.970" Score="0" ViewCount="31" Body="&lt;p&gt;I am new to Object Detection with Yolo and I have questions regarding the labeling (for custom objects): Is there any guideline or tips on how to label images to have high accuracy at the end? Anything I have to take care of?&lt;/p&gt;&#xA;&lt;p&gt;For example, what if I have one object twice next to each other like in the following picture:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.manchestereveningnews.co.uk/news/greater-manchester-news/greater-manchester-28-new-buses-17777605.amp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.manchestereveningnews.co.uk/news/greater-manchester-news/greater-manchester-28-new-buses-17777605.amp&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How would you label the black bus? Just the black part or would you assume the whole bus and thus create a box that would include the blue bus as well?&lt;/p&gt;&#xA;" OwnerUserId="40364" LastEditorUserId="2444" LastEditDate="2020-08-17T23:58:44.453" LastActivityDate="2020-08-17T23:58:44.453" Title="How should I label images to get high accuracy with YOLO?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;datasets&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23091" PostTypeId="1" CreationDate="2020-08-17T17:16:59.297" Score="3" ViewCount="426" Body="&lt;p&gt;I have been dealing with a problem that I'm trying to solve with DQN. A general question that I have is regarding the target's update frequency. How should it change? Depending on what factor do we increase or decrease this hyperparameter?&lt;/p&gt;&#xA;" OwnerUserId="40367" LastEditorUserId="2444" LastEditDate="2020-08-18T10:21:23.223" LastActivityDate="2020-08-18T10:21:23.223" Title="How should I choose the target's update frequency in DQN?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23096" PostTypeId="1" CreationDate="2020-08-17T18:43:46.947" Score="1" ViewCount="85" Body="&lt;p&gt;I came across the term '&lt;em&gt;principal angle between subspaces&lt;/em&gt;' as a tool for comparing objects in images. All material that I found on the internet seems to deal with this idea in a highly mathematical way and I couldn't understand the real physical meaning behind the term.&lt;/p&gt;&#xA;&lt;p&gt;I have some knowledge of linear algebra. Any help to understand the physical significance of this term and its application in object recognition would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="35576" LastEditorUserId="2444" LastEditDate="2021-01-15T11:36:07.153" LastActivityDate="2021-02-14T13:03:45.297" Title="What do we mean by 'principal angle between subspaces'?" Tags="&lt;computer-vision&gt;&lt;terminology&gt;&lt;object-detection&gt;&lt;linear-algebra&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23097" PostTypeId="1" CreationDate="2020-08-17T18:45:38.490" Score="0" ViewCount="29" Body="&lt;p&gt;I have read an article talking about binary clustering using Matrix factorization(see attached), but i would like to understand some optimization concepts:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it reasonable to use a Frobenius norm in such optimization?&lt;/li&gt;&#xA;&lt;li&gt;What does it mean the centroid constraint: C^T*1=0, and why it is equivalent to K-means especially in this case (When rho is large)?&lt;/li&gt;&#xA;&lt;li&gt;Is there other constraints that could improve clustering optimization?&lt;/li&gt;&#xA;&lt;li&gt;Is there other optimization technique in place of DPLM(Discrete Proximal Linearized Minimization)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8LSqh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8LSqh.png&quot; alt=&quot;Binary data clustering by Matrix factorization&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40370" LastActivityDate="2020-08-17T18:45:38.490" Title="Binary data clustering by Matrix factorization" Tags="&lt;optimization&gt;&lt;clustering&gt;&lt;k-means&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23098" PostTypeId="1" CreationDate="2020-08-17T19:10:11.577" Score="0" ViewCount="262" Body="&lt;p&gt;&lt;strong&gt;Here is the Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Describe the PEAS descriptions for the following agents:&lt;/p&gt;&#xA;&lt;p&gt;a) A grocery store scanner that digitally scans a fruit or vegetable and&#xA;identifies it.&lt;/p&gt;&#xA;&lt;p&gt;b) A GPS system for an automobile. Assume that the destination has been&#xA;preprogrammed and that there is no ongoing interaction with the driver.&#xA;However, the agent might need to update the route if the driver misses a turn.&lt;/p&gt;&#xA;&lt;p&gt;c) A credit card fraud detection agent that monitors an individual’s transactions&#xA;and reports suspicious activity.&lt;/p&gt;&#xA;&lt;p&gt;d) A voice activated mobile-phone assistant&lt;/p&gt;&#xA;&lt;p&gt;For each of the agents described above, categorize it with respect to the six dimensions&#xA;of task environments as described on pages 41-45 (Section 2.3.2 of AIMA). Be sure&#xA;that your choices accurately reflect the way you have specified your environment,&#xA;especially the sensors and actuators. Give a short justification for each property&lt;/p&gt;&#xA;&lt;p&gt;Here is what i thinks that the answers of above questions might be this. Can you guyz correct me if i answered wrong at any point.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qaWhb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qaWhb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40371" LastActivityDate="2020-08-17T19:10:11.577" Title="Need some reviews in PEAS descriptions" Tags="&lt;intelligent-agent&gt;&lt;environment&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23099" PostTypeId="1" CreationDate="2020-08-17T19:54:38.823" Score="0" ViewCount="11" Body="&lt;p&gt;I am doing an internship project regarding deep learning, and it is a totally new topic for me as I have never studied machine learning in the bachelor's degree courses. I have to implement a GAN that generates a middle image starting from two images, the previous and the next. In particular, the network must work with TACs, therefore it must generate the middle TAC starting from two successive ones. I am studying the basic theory concerning deep learning, neural networks and in particular GANs, but I have not yet fully understood where the network takes in the two starting images. Does the generating network still have to take in only one noise vector or must it also know the two images? Can you recommend any architecture that I can use to get ideas? Do you have any advice?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="40372" LastActivityDate="2020-08-17T19:54:38.823" Title="Generative Adversarial Network with two images in input" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23100" PostTypeId="1" CreationDate="2020-08-17T19:55:19.713" Score="0" ViewCount="40" Body="&lt;p&gt;I have started learning about and understanding intelligent agents and how agents perform in environments, like in games.&lt;/p&gt;&#xA;&lt;p&gt;However, I am a little bit confused about the question below. Basically, I have picked this question from &lt;a href=&quot;https://www.chegg.com/homework-help/questions-and-answers/question-2-table-driven-agent-performs-well-table-s-size-relative-small-however-size-table-q55212659&quot; rel=&quot;nofollow noreferrer&quot;&gt;the internet&lt;/a&gt;. This question is confusing me because in this scenario I thought that we have to consider the entire &amp;quot;Percept Sequence&amp;quot; when the agent is a model-based agent. I might be wrong because I am just starting the Agents chapter recently. If anyone can correct me or tell me the right answer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here is the question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Table-Driven agent performs well when the table's size is relative small. However, as the size of the table grows the efficiency of the agent decreases. We can reduce the table size by only considering the current &amp;quot;Percept&amp;quot; and not the entire &amp;quot;Percept Sequence&amp;quot;. Under what circumstances would this agent be rational? When considering the entire &amp;quot;Percept Sequence&amp;quot; useful?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="40371" LastEditorUserId="2444" LastEditDate="2020-08-20T10:30:21.323" LastActivityDate="2020-08-20T10:30:21.323" Title="Under which circumstances would an agent that only considers the current percept be rational?" Tags="&lt;intelligent-agent&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23101" PostTypeId="2" ParentId="23096" CreationDate="2020-08-17T23:23:27.923" Score="0" Body="&lt;p&gt;Let's consider the case where you have two photos, one base photo and one other photo which is a scaled version of the base photo. Consider then that you could create a 'mapping' from the base photo to the scaled photo as defined by a set of vector changes for each pixel in the base photo.&lt;/p&gt;&#xA;&lt;p&gt;That is to say, if you had a pixel at point (0,0) in the base photo it would be in position (0,0) on the scaled photo. If you had a pixel at point (0,1) on the base photo, it would be at point (0,s) on the scaled photo, where s is the scale factor between them. The same would be true for (1,0) mapping to (s,0) and (1,1) mapping to (s,s).&lt;/p&gt;&#xA;&lt;p&gt;We can understand each of these pixels as a subspace on the vector space of &amp;quot;all possible pixels&amp;quot; (possibly with an included vector for the 'value' of each pixel as a third+ dimension, based on how it's represented). Note though, that they are different subspaces. There are vectors that exist in the scaled picture for scale factors greater than 1 which are not in the original picture. For scale values less than one, there are vectors which are in the original picture which are not in the scaled picture. For scale vector equal to one, it's basically the identity and they're the same picture.&lt;/p&gt;&#xA;&lt;p&gt;What's more, we can do the same for rotation. If you have base photo and rotated photo, then there is a mapping from one pixel on the base photo to one pixel on the rotated photo, I'll leave the math up to you. The same is further true for translating, skewing, and even changing the colors of the image. Each one denotes a type of mapping that you could make from one subspace to another.&lt;/p&gt;&#xA;&lt;p&gt;So what does this mean? Well, one thing that you could do is take two images, one &amp;quot;base&amp;quot; photo and two &amp;quot;scaled&amp;quot; photos and determine &amp;quot;how different are these two photos from the base photo&amp;quot;? That is, if the first scaled photo has a scaling factor of 10 and the second has a scaling factor of 100, then you could say that the larger scaled-photo is &amp;quot;further away&amp;quot; from the base photo. That is, we can compare the two mappings together to determine which mapping is closer to the original.&lt;/p&gt;&#xA;&lt;p&gt;What's more, these mapping functions are often composable and commutative. That is if you do a &amp;quot;scale-then-rotate&amp;quot;, that is very similar if not exactly identical to doing &amp;quot;rotate-then-scale&amp;quot;. Let's take a look at why this is. Let's consider that we take a unit vector from either one. We then apply &amp;quot;scale&amp;quot; which scales both the x and y components, and then rotate this unit vector about some axis (let's assume the origin, though maybe there's a translation that happens as well). You will then yield a &lt;em&gt;new&lt;/em&gt; vector which represents all of these applied changes. What's more, we can do this to &lt;em&gt;both&lt;/em&gt; of the &amp;quot;scale-then-rotate&amp;quot; function and the &amp;quot;rotate-then-scale&amp;quot; function.&lt;/p&gt;&#xA;&lt;p&gt;Now we have two vectors, and we want to determine if they are the same. Well, one way to do that is to calculate the angle between two vectors. If the angle is 0, then we can say that the vectors are the same (with the assumption that they are also of the same magnitude). In this case, both would yield the same vector, the angle between them is 0, so we can say they are the same.&lt;/p&gt;&#xA;&lt;p&gt;So then how do we get back to the question of &amp;quot;how different are these two objects&amp;quot;? Well, one way that we could do it is to attempt to answer the question &amp;quot;What set of composed functions maps from one object onto the other one?&amp;quot;. If we can create a new &amp;quot;vector&amp;quot; which is the representation of the full set of changes from one picture to the other, we can then attempt to determine how different these two objects are, as represented in the dimensional spaces of their image.&lt;/p&gt;&#xA;&lt;p&gt;But the question that you may then ask is &amp;quot;why is this useful&amp;quot;? Well, imagine that you are a neural network. You've been given a picture and tasked with answering the question &amp;quot;Is this a cat?&amp;quot;. How would you go about answering this? Well, one thing as a neural net that you've learned is &amp;quot;cats have ears&amp;quot;. You have a canonical representation of a &amp;quot;cat ear&amp;quot; encoded in the weights of your neural network, so you go about looking at different subsets of the picture to say &amp;quot;is this a cat ear&amp;quot;? You can do this, for example, by taking a cropped section of the picture, determining the angle and dimensions apart that cropped section is from your canonical &amp;quot;cat ear&amp;quot; is, and &lt;strong&gt;if the angle between your canonical &amp;quot;cat ear&amp;quot; and your sample is below some amount&lt;/strong&gt;, you can say with confidence that what you have found is a &amp;quot;cat ear&amp;quot;. A couple cat ears, eyes, a nose, some fur, and a tail (though maybe the tail is occluded), and you can be confident that you have found a &amp;quot;cat&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;(Note: This is an oversimplification and this is usually done in deep networks by breaking pictures out into a continually composing series of features, for instance first &amp;quot;lines&amp;quot; vs &amp;quot;curves&amp;quot; vs &amp;quot;textures&amp;quot;, which then yield &amp;quot;circles&amp;quot; vs &amp;quot;rectangles&amp;quot; vs &amp;quot;cross-hashes&amp;quot;, which then yield &amp;quot;eyes&amp;quot; vs &amp;quot;fur&amp;quot; vs &amp;quot;skin&amp;quot;, etc. Each different layer could be implemented with its own comparison in this same method though!)&lt;/p&gt;&#xA;" OwnerUserId="16857" LastActivityDate="2020-08-17T23:23:27.923" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23102" PostTypeId="1" CreationDate="2020-08-18T02:50:23.650" Score="0" ViewCount="12" Body="&lt;p&gt;I have a short video where a TV is shown. I need to detect the TV there (let's assume it's all pure black or it's covered with a green screen), and overlay a video on top of it on the same area. I guess I need to detect the angle and its scale to overlay the content correctly and nicely.&lt;/p&gt;&#xA;&lt;p&gt;Is there any ways to do it with AI or computer vision? We can detect the TV with object detection, but I don't know how to compute the tilt and angle for best fit when it's not a retangle.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wnOdb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wnOdb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;    &lt;a href=&quot;https://i.stack.imgur.com/SfTVO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SfTVO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-08-18T02:50:23.650" Title="Detect and overlay a black TV on a video" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;object-detection&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23104" PostTypeId="2" ParentId="16004" CreationDate="2020-08-18T04:16:46.933" Score="0" Body="&lt;p&gt;There is a commonly used method, that is also used in machine learning: Independent Component Analysis. (ICA). This is commonly used to find specific noises in the data, however, you need to have some EEG knowledge to do this because automatic rejection is not completely solved at this time. Software like EEGLab is available (as a standalone and Matlab toolbox)&lt;/p&gt;&#xA;&lt;p&gt;Now to do this in real-time it is also not impossible after you collect initial data for a while and don't have too many channels. You can isolate relatively constant noises with ICA, like heart-beating, other temporal noises can be rejected globally (on all channels) because EEG normally does not exceed certain levels.&lt;/p&gt;&#xA;&lt;p&gt;Useful documentation is EEGLabs artifacts Wikipedia page:&#xA;&lt;a href=&quot;https://sccn.ucsd.edu/wiki/Chapter_01:_Rejecting_Artifacts&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sccn.ucsd.edu/wiki/Chapter_01:_Rejecting_Artifacts&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40315" LastActivityDate="2020-08-18T04:16:46.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23105" PostTypeId="2" ParentId="15504" CreationDate="2020-08-18T05:53:37.207" Score="1" Body="&lt;p&gt;I would suggest to convert the documents into &lt;strong&gt;TF-IDF(use Gensim&lt;/strong&gt;) vectors and then compare them using various similarity calculating techniques like &lt;strong&gt;cosine similarity&lt;/strong&gt;.&lt;br&gt;&lt;br&gt;&#xA;You should read this amazing article for the same. I once used it while working on my project.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://medium.com/@adriensieg/text-similarities-da019229c894&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@adriensieg/text-similarities-da019229c894&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35612" LastActivityDate="2020-08-18T05:53:37.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23106" PostTypeId="1" CreationDate="2020-08-18T06:22:30.417" Score="1" ViewCount="35" Body="&lt;p&gt;I have run into a strange behavior of my &lt;strong&gt;multi label&lt;/strong&gt; classification ANN&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(Dense(6, input_shape=(input_size,), activation='elu'))&#xA;#model.add(BatchNormalization(axis=-1))&#xA;model.add(Dropout(0.2))&#xA;#model.add(BatchNormalization(axis=-1))&#xA;model.add(Dense(6, activation='elu'))&#xA;model.add(Dropout(0.2))&#xA;#model.add(BatchNormalization(axis=-1))&#xA;model.add(Dense(6, activation='elu'))&#xA;model.add(Dropout(0.2))&#xA;&#xA;# model.add(keras.layers.BatchNormalization(axis=-1))&#xA;model.add(Dense(6, activation='sigmoid'))&#xA;model.compile(loss='binary_crossentropy',&#xA;              optimizer='nadam',&#xA;              metrics=['accuracy'])&#xA;history = model.fit(X_train, Y_train,batch_size=64 ,epochs=300,&#xA;                    validation_data = (X_test, Y_test), verbose=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result is quite strange, I have a feeling that my model could not improve any more. Why does the loss and the accuracy does not change overtime ?&lt;/p&gt;&#xA;&lt;p&gt;P/S For clarification, I have 6 output and the value of each output is 0 or 1 that is&#xA;output 1: can be 0 or 1&lt;/p&gt;&#xA;&lt;p&gt;output 2: can be 0 or 1&lt;/p&gt;&#xA;&lt;p&gt;output 3: can be 0 or 1&lt;/p&gt;&#xA;&lt;p&gt;output 4: can be 0 or 1&lt;/p&gt;&#xA;&lt;p&gt;output 5: can be 0 or 1&lt;/p&gt;&#xA;&lt;p&gt;output 6: can be 0 or 1&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1rUqs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1rUqs.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40383" LastEditorUserId="40383" LastEditDate="2020-08-18T07:50:00.023" LastActivityDate="2020-08-18T07:50:00.023" Title="Why does loss and accuracy for a multi label classification ann does not change overtime?" Tags="&lt;neural-networks&gt;&lt;accuracy&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23107" PostTypeId="1" AcceptedAnswerId="23108" CreationDate="2020-08-18T06:36:10.197" Score="1" ViewCount="62" Body="&lt;p&gt;I trying to use DDPG augmented with Hindsight Experience Replay (HER) on &lt;a href=&quot;https://github.com/bulletphysics/bullet3/blob/master/examples/pybullet/gym/pybullet_envs/bullet/kukaGymEnv.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;pybullet's KukaGymEnv&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To formulate the feature vector for the goal state, I need to know what the features of the state of the environment represent. To be precise, a typical state vector of KukaGymEnv is an object of the &lt;code&gt;numpy.ndarray&lt;/code&gt; class with a shape of &lt;code&gt;(9,)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What do each of these 8 elements represent, and how can I formulate the goal state vector for this environment? I tried going through the &lt;a href=&quot;https://github.com/bulletphysics/bullet3/blob/master/examples/pybullet/gym/pybullet_envs/bullet/kukaGymEnv.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; of the KukaGymEnv, but was unable to understand anything useful.&lt;/p&gt;&#xA;" OwnerUserId="38895" LastEditorUserId="2444" LastEditDate="2020-11-21T12:52:53.237" LastActivityDate="2020-11-21T12:52:53.237" Title="What do the state features of KukaGymEnv represent?" Tags="&lt;reinforcement-learning&gt;&lt;ddpg&gt;&lt;hindsight-experience-replay&gt;&lt;state-spaces&gt;&lt;pybullet&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23108" PostTypeId="2" ParentId="23107" CreationDate="2020-08-18T07:09:42.423" Score="1" Body="&lt;p&gt;Here's an incomplete answer, but it may help.&lt;/p&gt;&#xA;&lt;p&gt;Your state is read by the function &lt;code&gt;getExtendedObservation()&lt;/code&gt;. This function makes two things : it calls the function &lt;code&gt;getObservation()&lt;/code&gt; from &lt;a href=&quot;https://github.com/bulletphysics/bullet3/blob/master/examples/pybullet/gym/pybullet_envs/bullet/kuka.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this source code&lt;/a&gt;, gets a state, and extend this state with three components :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;relative x,y position and euler angle of block in gripper space&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But what are the 5 first components returned by &lt;code&gt;getObservation()&lt;/code&gt;? From what I read, there are positions, then euler angles describing the orientation. But that would make 6 + 3 = 9 features, so there is either only 2 positions, or only 2 euler angles. You may know kuka better than me and know the answer of this one :).&lt;/p&gt;&#xA;&lt;p&gt;So, to sum up :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;state = [X, Y, (Z, ) , Alpha, Gamma, (Beta, ), gripX, gripY, gripAlpha]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Either Z or Beta is absent)&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2020-08-23T12:31:33.933" LastActivityDate="2020-08-23T12:31:33.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23110" PostTypeId="2" ParentId="23091" CreationDate="2020-08-18T08:05:56.707" Score="1" Body="&lt;p&gt;As you said yourself, it is a hyperparameter. Hence, no one (even you) can say what is the ideal update frequency. You have to test and try.&lt;/p&gt;&#xA;&lt;p&gt;Having said that, remember one thing the target NN should mimic the actual network as closely as possible. Hence if you update it after a long number runs, then I think you will start losing the accuracy. On the contrary, if you update it too often, then you lose the benefit of using the target network (which is to boost the training rate and reduce training time) and the training will take a larger amount of time.&lt;/p&gt;&#xA;&lt;p&gt;My suggestion is to try updating after every 5 to 8 episodes.&lt;/p&gt;&#xA;" OwnerUserId="36710" LastActivityDate="2020-08-18T08:05:56.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23111" PostTypeId="1" CreationDate="2020-08-18T09:32:53.853" Score="2" ViewCount="25" Body="&lt;p&gt;In reinforcement learning, when we talk about the principle of optimality, do we assume the policy to be deterministic?&lt;/p&gt;&#xA;" OwnerUserId="40229" LastEditorUserId="2444" LastEditDate="2020-08-18T10:18:38.280" LastActivityDate="2020-08-18T10:18:38.280" Title="Do we assume the policy to be deterministic when proving the optimality?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;policies&gt;&lt;deterministic-policy&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23112" PostTypeId="1" AcceptedAnswerId="23661" CreationDate="2020-08-18T12:33:00.667" Score="1" ViewCount="109" Body="&lt;p&gt;I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling.&lt;/p&gt;&#xA;&lt;p&gt;The size of the model that indicates the number of hidden layers and units should not be too much or too few for the model can capture useful features from the dataset.&lt;/p&gt;&#xA;&lt;p&gt;So, how do I choose the right size of the deep autoencoder model is enough to good?&lt;/p&gt;&#xA;" OwnerUserId="25645" LastEditorUserId="2444" LastEditDate="2020-10-05T21:20:34.760" LastActivityDate="2020-10-05T21:20:34.760" Title="How to determine the number of hidden layers and units of a deep auto-encoder?" Tags="&lt;deep-learning&gt;&lt;autoencoders&gt;&lt;hyperparameter-optimization&gt;&lt;hidden-layers&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23113" PostTypeId="2" ParentId="23037" CreationDate="2020-08-18T14:48:53.567" Score="1" Body="&lt;p&gt;You are probably looking for &lt;strong&gt;incremental learning&lt;/strong&gt; (sometimes known as &lt;em&gt;lifelong learning&lt;/em&gt;) techniques, i.e. machine learning techniques that attempt to address the &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;&lt;strong&gt;catastrophic forgetting&lt;/strong&gt;&lt;/a&gt; effect of neural networks when trained incrementally, i.e. as new classes or data are added to the original training data.&lt;/p&gt;&#xA;&lt;p&gt;There are different techniques and some of them store (or compress) the old data in order to fully or partially re-train the neural network with the new classes or data.  However, note that this is a relatively new area of research and significant progress still needs to be made to produce serious tools. If you are specifically interested in incremental class learning, maybe have a look at this paper &lt;a href=&quot;https://arxiv.org/pdf/1903.07864.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Class-incremental Learning via Deep Model Consolidation&lt;/a&gt; (2020).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-08-18T14:48:53.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23114" PostTypeId="1" CreationDate="2020-08-18T15:27:59.317" Score="0" ViewCount="24" Body="&lt;p&gt;I'm trying to get some TensorFlow on Android code working that is using a model from a Model Zoo. One of the models in the zoo is YOLOV2 (&lt;a href=&quot;https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/yolov2&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;). But when I pass the model an image I'm getting back what I think to be garbage results. Ex: some of the propability fields have negative numbers, and I expected these fields to only have positive results from 0..1.&lt;/p&gt;&#xA;&lt;p&gt;According to the description for this model the input should have the following shape.&lt;/p&gt;&#xA;&lt;p&gt;(1x3x416x416)&lt;/p&gt;&#xA;&lt;p&gt;I want to make sure I am understanding this correctly. I believe it means that I must rearrange my pixels. I take the 1 to mean that it works on one image at a time. The 416x416 means that the image should be resized to be 416 pixels tall and 416 pixels wide. The 3 is throwing me off a little. Since it is coming before the pixel dimensions, does this mean that all of the pixels for the red channel are together, followed by all of the pixels for the blue channel, and then all of the pixels for the blue channel? In other words, if this were a 2x2 image the pixels would be saved in a typical file like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; R1G1B1 R2G2B2 R3G3B3 R4G4B4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But, if I am interpreting the above correctly, I think it would be like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; R1R2R3R4 B1B2B3B4 G1G2G3G4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this a correct interpretation? Is it necessary to rearrange my pixels like the above?&lt;/p&gt;&#xA;" OwnerUserId="40395" LastActivityDate="2020-08-18T15:27:59.317" Title="What modifications do I need to make to an Image for YOLO recognition" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23115" PostTypeId="1" CreationDate="2020-08-18T16:36:09.173" Score="1" ViewCount="25" Body="&lt;p&gt;I'm evaluating the state of the art techniques to translate legal text to simple text, what are the best approaches for a non-English language (Portuguese)?&lt;/p&gt;&#xA;" OwnerUserId="40396" LastEditorUserId="40396" LastEditDate="2020-08-18T22:43:57.023" LastActivityDate="2020-08-18T22:43:57.023" Title="What are the best techniques to perform text simplification?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;text-summarization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23116" PostTypeId="2" ParentId="16610" CreationDate="2020-08-18T17:23:48.657" Score="3" Body="&lt;p&gt;What if a scalar reward is insufficient, or its unclear on how to collapse a multi-dimensional reward to a single dimension. Example, for someone eating a burger, both taste and cost are important. Agents may prioritize taste and cost differently, so its not clear on how to aggregate the two. It is also not clear on how a subjective categorical taste value can be combined with a numerical cost.&lt;/p&gt;&#xA;" OwnerUserId="40398" LastActivityDate="2020-08-18T17:23:48.657" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23117" PostTypeId="1" CreationDate="2020-08-18T17:24:32.113" Score="2" ViewCount="77" Body="&lt;p&gt;In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="40397" LastEditorUserId="2444" LastEditDate="2020-11-10T12:59:05.903" LastActivityDate="2020-11-10T12:59:05.903" Title="What's the difference between domain randomization and domain adaptation?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;transfer-learning&gt;&lt;domain-adaptation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23118" PostTypeId="2" ParentId="22834" CreationDate="2020-08-19T01:19:25.487" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What if the complexity of the problem of self-improving the software grows at a faster rate than the AGI intelligence self-improvement?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If this turns out to be the case, then the complexity would indeed be the bottleneck.&lt;/p&gt;&#xA;&lt;p&gt;There's a formal term for this: &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;kolmogrov complexity or &amp;quot;k-complexity&amp;quot;&lt;/a&gt;.  (The best formal definition of intelligence I've come across is Hutter &amp;amp; Legg's, which is broken down in the article &amp;quot;&lt;a href=&quot;https://nbro.github.io/blogging/2020/05/20/on-the-definition-of-intelligence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the definition of intelligence&lt;/a&gt;&amp;quot; by our own nbro.  It will shed shed some light on the formal structure of the question at hand.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Any attempt to provide values to your d &amp;amp; l would speculative, since we're still so far from AGI. The question would be what is the effect of any given (d,l)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The subject can definitely be treated formally in a mathematical sense the and values analyzed.  (There may even been some formulae out there, although, they are likely to be more complex, per the Hutter&amp;amp;Legg.)&lt;/p&gt;&#xA;&lt;p&gt;Sorry I can't give you a more formal answer--it's a good question!&lt;/p&gt;&#xA;&lt;br&gt;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&lt;p&gt;Another bottleneck would be the speed of light, essentially, &amp;quot;bits and bytes across a pipeline&amp;quot;.  This would be a factor in both self-replication and networked processing and memory.  You can expand bandwidth/throughput, but not infinitely.&lt;/p&gt;&#xA;&lt;p&gt;This relates to space complexity (how much volume/memory does the algorithm require?) in addition to the k-complexity, which itself is likely to be non-trivial for anything approaching AGI.&lt;/p&gt;&#xA;&lt;p&gt;AI Threat:&lt;/p&gt;&#xA;&lt;p&gt;From the standpoint of neoluddism, I'm skeptical that &amp;quot;infinitely&amp;quot; self-optimizing superintelligence is the greatest threat.  Evolution, and economic theory, suggest that the greater danger would come from not what is most expensive, complex, &amp;amp; capable, but that which is &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Satisficing&quot; rel=&quot;nofollow noreferrer&quot;&gt;just good enough&lt;/a&gt;&amp;quot;, cheaper and more easily replicable.  This is because decision time becomes a critical factor in a competitive environment, as does replication time.&lt;/p&gt;&#xA;&lt;p&gt;I always bring up &lt;a href=&quot;https://en.wikipedia.org/wiki/Gray_goo&quot; rel=&quot;nofollow noreferrer&quot;&gt;grey goo&lt;/a&gt; because it only has to be good at 1 thing to eat everything else!&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2020-08-19T01:19:25.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23119" PostTypeId="5" CreationDate="2020-08-19T01:27:44.457" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In algorithmic information theory (a subfield of computer science and mathematics), the Kolmogorov complexity of an object, such as a piece of text, is the length of a shortest computer program (in a predetermined programming language) that produces the object as output. It is a measure of the computational resources needed to specify the object, and is also known as algorithmic complexity, Solomonoff–Kolmogorov–Chaitin complexity, program-size complexity, descriptive complexity, or algorithmic entropy. It is named after Andrey Kolmogorov, who first published on the subject in 1963.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;sub&gt;Source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kolmogorov complexity (wiki)&lt;/a&gt;&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://people.eecs.berkeley.edu/%7Eluca/cs172/notek.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Notes on Kolmogorov Complexity (U.C. Berkeley — CS172: Automata, Computability and Complexity)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-08-19T01:27:44.457" LastActivityDate="2020-08-19T01:27:44.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23120" PostTypeId="4" CreationDate="2020-08-19T01:27:44.457" Score="0" Body="Use for questions involving the k-complexity of an algorithm, the &quot;shortest computer program&quot; or set of instructions that produce the desired output." OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-08-19T01:27:44.457" LastActivityDate="2020-08-19T01:27:44.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23121" PostTypeId="1" CreationDate="2020-08-19T03:43:32.620" Score="0" ViewCount="51" Body="&lt;p&gt;I am curious to know how neural networks are built in practice.&#xA;Are they hand coded using weight matrices, activation functions etc OR are there ways to build the NN by mentioning the number of layers, number of neurons in each layer, activation to be used, etc as parameters?&lt;/p&gt;&#xA;&lt;p&gt;Similar question on training, once built is there a ‘fit’ method or does the training need to be hand coded?&lt;/p&gt;&#xA;&lt;p&gt;Any reference for understanding these basics will be of great help.&lt;/p&gt;&#xA;" OwnerUserId="15267" LastEditorUserId="9947" LastEditDate="2020-08-19T04:00:29.167" LastActivityDate="2020-08-21T04:09:26.730" Title="How are neural networks built in practice?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23122" PostTypeId="1" CreationDate="2020-08-19T04:28:46.577" Score="3" ViewCount="34" Body="&lt;p&gt;On page 27 of the &lt;a href=&quot;http://airesearch.com/wp-content/uploads/2016/01/deepmind-mastering-go.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind AlphaGo paper&lt;/a&gt; appears the following sentence:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The first hidden layer zero pads the input into a &lt;span class=&quot;math-container&quot;&gt;$23 \times 23$&lt;/span&gt; image, then convolves &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; filters of kernel size &lt;span class=&quot;math-container&quot;&gt;$5 \times 5$&lt;/span&gt; with stride &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; with the input image and applies a rectifier nonlinearity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does &amp;quot;convolves &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; filters&amp;quot; mean here?&lt;/p&gt;&#xA;&lt;p&gt;Does it mean the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The first hidden layer is a convolutional layer with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; groups of &lt;span class=&quot;math-container&quot;&gt;$(19 \times 19)$&lt;/span&gt; neurons, where there is a kernel of &lt;span class=&quot;math-container&quot;&gt;$(5 \times 5 \times numChannels + 1)$&lt;/span&gt; parameters (input weights plus a bias term) used by all the neurons of each group. &lt;span class=&quot;math-container&quot;&gt;$numChannels$&lt;/span&gt; is 48 (the number of feature planes in the input image stack).&lt;/p&gt;&#xA;&lt;p&gt;All &lt;span class=&quot;math-container&quot;&gt;$(19 \times 19 \times k)$&lt;/span&gt; neurons' outputs are available to the second hidden layer (which happens to be another convolutional layer, but could in principle be fully connected).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;" OwnerUserId="40405" LastEditorUserId="2444" LastEditDate="2020-08-21T11:36:49.690" LastActivityDate="2020-08-21T11:36:49.690" Title="What does &quot;convolve k filters&quot; mean in the AlphaGo paper?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;alphago&gt;&lt;filters&gt;&lt;convolutional-layers&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23123" PostTypeId="2" ParentId="23121" CreationDate="2020-08-19T06:14:38.193" Score="3" Body="&lt;p&gt;A &amp;quot;software library&amp;quot; is a codebase that includes many commonly used functions/algorithms, you do not need to write the same function again if you &amp;quot;import&amp;quot; the library.&lt;/p&gt;&#xA;&lt;p&gt;In practice, practitioners/researchers don't often code the nitty gritty details of well-used algorithms, they just use existing libraries such as sklearn or TensorFlow that already has an existing implementation.&lt;/p&gt;&#xA;&lt;p&gt;In the case of sklearn, ignoring some details, you can create a muti-layer perceptron in this line of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; clf = MLPClassifier(activation = 'relu', alpha=1e-5, hidden_layer_sizes((5,2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;...in which &lt;code&gt;clf&lt;/code&gt; is an &amp;quot;object&amp;quot; that refers to the neural network, &lt;code&gt;MLPClassifier&lt;/code&gt; is a function provided by sklearn that &amp;quot;creates&amp;quot; a neural net depending on the parameters provided.&lt;/p&gt;&#xA;&lt;p&gt;In this case, the parameters provided are that the activation function is ReLU, where the network has two layers, with the first having 5 nodes, and the second has 2 nodes.&lt;/p&gt;&#xA;&lt;p&gt;All other parameters that are not explicitly provided are set to a default value, in this case, the learning rate, among others.&lt;/p&gt;&#xA;&lt;p&gt;It should be noted that this is not complete yet, we still have not completed training. Luckily, this is as simple as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; clf.fit(X, y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; are some array-shaped data/target set.&lt;/p&gt;&#xA;&lt;p&gt;I recommend sklearn as i think its more beginner friendly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://scikit-learn.org/stable/getting_started.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/getting_started.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-08-21T04:09:26.730" LastActivityDate="2020-08-21T04:09:26.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23124" PostTypeId="1" CreationDate="2020-08-19T06:38:39.940" Score="2" ViewCount="39" Body="&lt;p&gt;I have an image dataset, which is composed of 113695 images for training and 28424 images for validation. Now, when I use &lt;code&gt;ImageDataGenerator&lt;/code&gt; and &lt;code&gt;flow_from_dataframe&lt;/code&gt;, it as the parameter &lt;code&gt;batch_size&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How can I take the correct number for &lt;code&gt;batch_size&lt;/code&gt; because both numbers cannot be divided by the same number? Should I need to drop four images in the validation data to make them &lt;code&gt;batch_size&lt;/code&gt; of 5? Or is there another way?&lt;/p&gt;&#xA;" OwnerUserId="38737" LastEditorUserId="2444" LastEditDate="2020-08-21T11:14:49.920" LastActivityDate="2020-08-21T11:14:49.920" Title="How to take the optimal batch_size for training a model?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;tensorflow&gt;&lt;data-preprocessing&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23125" PostTypeId="1" AcceptedAnswerId="23129" CreationDate="2020-08-19T06:53:02.550" Score="2" ViewCount="186" Body="&lt;p&gt;I'm training a DQN in a real environment where I do not have a natural terminal state, so I've built the episode in an artificial way (i.e. it starts in a random condition and after T steps it ends). My question is about the terminal state: should I consider it when I have to compute &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (so using only the reward) or not?&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="37169" LastEditDate="2020-08-21T07:02:30.763" LastActivityDate="2020-08-21T07:02:30.763" Title="How should I compute the target for updating in a DQN at the terminal state if I have pseudo-episodes?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23127" PostTypeId="2" ParentId="23124" CreationDate="2020-08-19T07:41:49.417" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/a/153535/264183&quot;&gt;This&lt;/a&gt; Cross Validated post might answer your question.&lt;/p&gt;&#xA;&lt;p&gt;In a nutshell:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A single batch (that is all your data in one batch) will result in a smooth trajectory on the loss surface. The drawback is that all your data might not fit into your memory. Which is highly likely for ~100k images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;One image per batch (batch size = no. examples) will result in a more stochastic trajectory since the gradients are calculated on a single example. Advantages are of computational nature and faster training time.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The middle way is to choose the batch size in such a way that your batch fits into memory and gradients behave less 'noisy'. To be honest there is no 'golden' number, personally I like to choose powers of two.&lt;/p&gt;&#xA;&lt;p&gt;Don't worry that your data is not divisible by the batch size. Libraries will take care about that internally, the last batch will just be a smaller than the defined batch size (&lt;span class=&quot;math-container&quot;&gt;$N \text{ mod } b$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-08-19T07:41:49.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23128" PostTypeId="2" ParentId="16067" CreationDate="2020-08-19T07:57:32.923" Score="0" Body="&lt;p&gt;I could have given you the specification suggestions directly. But I think I need to share some of my experiences with you. Based on that following are my suggestions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;First of all&lt;/strong&gt; try some good free alternatives. If it meets your requirement then you are done. You can try Google Colab for this purpose. From my experience I can say that it will be enough for you if your data set is not too big. If it is too big then you are likely to face the following problems (as I myself already faced them very well):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The Notebook usage quota will be a nightmare. The notebook's session will not last for more than 12 hours.&lt;/li&gt;&#xA;&lt;li&gt;If you are using your google drive for hosting data set you will likely face another nightmare. The Google Drive has also limitations how much you can read and write within a certain length of time. Personally I ran out of read-right quota after reading 15 GB of training data from drive and therefore could not read files from drive around for around 4 days. I was blocked. There might be some bypasses but for me I just could not avail them.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But you should have a try on this free option initially to verify if it is enough for you.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Secondly&lt;/strong&gt; you can look for some paid cloud services. But if you need to use them for days or months then it might not be a better solution for you.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Finally&lt;/strong&gt; if you are planning for buying a GPU then there are various options for you. If you are planning to buy just for your current research purpose then you might compromise with the GPU capacity if it meets your need. But if you are very serious about deep learning then you may follow the suggestions in the others' answers, discuss with people who are using those GPUs and lastly, I recommend you to have a look on &lt;a href=&quot;https://lambdalabs.com/blog/choosing-a-gpu-for-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Choosing the Best GPU for Deep Learning in 2020 | Lambda Labs&lt;/a&gt;. It describes many aspects of various GPUs and also provided benchmarks. Additionally I am just quoting a portion of the summarization here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;RTX 2060 (6 GB): if you want to explore deep learning in your spare&#xA;time.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;RTX 2070 or 2080 (8 GB): if you are serious about deep learning, but  your GPU budget is $600-800.Eight GB of VRAM can fit the majority of models.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;RTX 2080 Ti (11 GB): if you are serious about deep learning and your GPU budget is ~$1,200. The RTX 2080 Ti is ~40% faster than the&#xA;RTX 2080.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Titan RTX and Quadro RTX 6000 (24 GB): if you are working on SOTA models extensively, but don't have budget for the future-proofing&#xA;available with the RTX 8000.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Quadro RTX 8000 (48 GB): you are investing in the future and might even be lucky enough to research SOTA deep learning in 2020.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27355" LastEditorUserId="27355" LastEditDate="2020-08-19T08:22:03.513" LastActivityDate="2020-08-19T08:22:03.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23129" PostTypeId="2" ParentId="23125" CreationDate="2020-08-19T08:20:42.750" Score="3" Body="&lt;p&gt;If the episode does not terminate naturally, then if you are breaking it up into pseudo-episodes for training purposes, the one thing you should not do is use the TD target &lt;span class=&quot;math-container&quot;&gt;$G_{T-1} = R_T$&lt;/span&gt; used for an end of episode, which assumes a return of 0 from any terminal state &lt;span class=&quot;math-container&quot;&gt;$S_{T}$&lt;/span&gt;. Of course that is because it is &lt;em&gt;not&lt;/em&gt; the end of the episode.&lt;/p&gt;&#xA;&lt;p&gt;You have two &amp;quot;natural&amp;quot; options to tweak DQN to match to theory at the end of a pseudo-episode:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Store the state, action, reward, next_state tuple as normal and use the standard one step TD target &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+1} = R_{t+1} + \gamma \text{max}_{a'} Q(S_{t+1}, a')$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Completely ignore the last step and don't store it in memory. There is no benefit to this as opposed to the above option, but it might be simpler to implement if you are using a pre-built RL library.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both these involve ignoring any &lt;code&gt;done&lt;/code&gt; flag returned by the environment for the purposes of calculating TD targets. You still can use that flag to trigger the end of a loop and a reset to new starting state.&lt;/p&gt;&#xA;&lt;p&gt;You should also take this approach if you terminate an episodic problem early after hitting a time step limit, in order to reset for training purposes.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As an aside (and mentioned in comment by Swakshar Deb), you can also look into the &lt;a href=&quot;https://link.springer.com/article/10.1007/BF00114727&quot; rel=&quot;nofollow noreferrer&quot;&gt;average reward setting for non-episodic environments&lt;/a&gt;. This solves the problem of needing to pick a value for &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;. If you have no reason to pick a specific &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; in a continuing problem, then it is common to pick a value close to 1 such as 0.99 or 0.999 in DQN - this is basically an approximation to average reward.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-19T09:35:39.410" LastActivityDate="2020-08-19T09:35:39.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23131" PostTypeId="1" CreationDate="2020-08-19T09:09:39.227" Score="0" ViewCount="25" Body="&lt;p&gt;This is more of a &lt;strong&gt;conceptual&lt;/strong&gt; question that I am asking here. I am using an encoder-decoder model with Bidirectional LSTMs used as a time series regression problem. I train it with a sufficiently low learning rate (1e-5) and an early-stop callback with a patience of 25 epochs, meaning that training of the model stops automatically when the validation loss is stuck at a constant trend for 25 extra epochs (My validation loss does decrease significantly from 1 to around 0.06). This brings my total trained epochs to around 50-80. I am sure that I am using sufficient layers of LSTMs and the TimeDistributed Dense layers. Despite this, I am getting some errors in prediction.&lt;/p&gt;&#xA;&lt;p&gt;My question to you with respect to the above scenario is: Does the low and constant validation loss mean that the model cannot give a better prediction performance if trained for further epochs? In other words, is it possible that the model still needs more epochs to learn to give better model prediction performance, even though the validation loss has decreased as much as it could?&lt;/p&gt;&#xA;" OwnerUserId="40410" LastActivityDate="2020-08-19T09:09:39.227" Title="Does a constant low validation loss mean great model accuracy?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23133" PostTypeId="1" CreationDate="2020-08-19T10:52:16.303" Score="1" ViewCount="33" Body="&lt;p&gt;Since the discriminator defines how the generator is updated, then building a discriminator with a higher number of parameters/more layers should lead to a better quality of generated samples. So, assuming that it won't lead to overwhelming the generator (discriminator loss toward 0) or mode collapse, when engineering a GAN, I should build a discriminator as good as possible?&lt;/p&gt;&#xA;" OwnerUserId="31324" LastEditorUserId="2444" LastEditDate="2020-08-20T10:09:13.663" LastActivityDate="2021-05-17T15:01:50.293" Title="Does a better discriminator in GANs mean better sample generation by the generator?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23134" PostTypeId="1" CreationDate="2020-08-19T10:59:04.033" Score="0" ViewCount="57" Body="&lt;p&gt;I am trying to implement safe exploration technique in [Ref.1]. I am using Soft Actor-Critic algorithm to teach an agent to introduce a bias between 0 and 1 to a specific state of interest in my environment.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I would like to ask for your help&lt;/strong&gt; in order to modify the critic update equation -which is originally  based on the return or the RL cost function- &lt;strong&gt;from this&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LyKs1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LyKs1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is based on the return functions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/96yfU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/96yfU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;  =&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ku8J5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ku8J5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;to be based on the following cost function&lt;/strong&gt; to make the RL objective risk-sensitive and avoid large-valued actions (bias values) at the start of the agent's learning,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gQqjM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gQqjM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I include the second part of the cost function - in which the variance of the reward is evaluated- in the update equation?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[Ref.1] &lt;em&gt;Heger, M., Consideration of risk in reinforcement learning. 11th International Machine Learning Conference (1994)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="40409" LastEditorUserId="40409" LastEditDate="2020-09-02T12:42:02.970" LastActivityDate="2020-09-02T12:42:02.970" Title="How to modify the Actor-Critic policy gradient algorithm to perform Safe exploration in Reinforcement Learning" Tags="&lt;actor-critic-methods&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23136" PostTypeId="1" CreationDate="2020-08-19T12:52:46.927" Score="0" ViewCount="16" Body="&lt;p&gt;I want to do a multiclass segmentation task using deep learning (in python). Here, is a summary of vgg_unet model that is mainly collected from &lt;a href=&quot;https://github.com/divamgupta/image-segmentation-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;. So, in my dataset 8 labels are available. So, at the last convolution layer, there are 8 channels for the categorical classification of every class.&#xA;The summary of my model is here,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__________________________________________________________________________________________________&#xA;Layer (type)                    Output Shape         Param #     Connected to                     &#xA;==================================================================================================&#xA;input_1 (InputLayer)            [(None, 512, 512, 3) 0                                            &#xA;__________________________________________________________________________________________________&#xA;block1_conv1 (Conv2D)           (None, 512, 512, 64) 1792        input_1[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;block1_conv2 (Conv2D)           (None, 512, 512, 64) 36928       block1_conv1[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block1_pool (MaxPooling2D)      (None, 256, 256, 64) 0           block1_conv2[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block2_conv1 (Conv2D)           (None, 256, 256, 128 73856       block1_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;block2_conv2 (Conv2D)           (None, 256, 256, 128 147584      block2_conv1[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block2_pool (MaxPooling2D)      (None, 128, 128, 128 0           block2_conv2[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block3_conv1 (Conv2D)           (None, 128, 128, 256 295168      block2_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;block3_conv2 (Conv2D)           (None, 128, 128, 256 590080      block3_conv1[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block3_conv3 (Conv2D)           (None, 128, 128, 256 590080      block3_conv2[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block3_pool (MaxPooling2D)      (None, 64, 64, 256)  0           block3_conv3[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block4_conv1 (Conv2D)           (None, 64, 64, 512)  1180160     block3_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;block4_conv2 (Conv2D)           (None, 64, 64, 512)  2359808     block4_conv1[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block4_conv3 (Conv2D)           (None, 64, 64, 512)  2359808     block4_conv2[0][0]               &#xA;__________________________________________________________________________________________________&#xA;block4_pool (MaxPooling2D)      (None, 32, 32, 512)  0           block4_conv3[0][0]               &#xA;__________________________________________________________________________________________________&#xA;zero_padding2d (ZeroPadding2D)  (None, 34, 34, 512)  0           block4_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;conv2d (Conv2D)                 (None, 32, 32, 512)  2359808     zero_padding2d[0][0]             &#xA;__________________________________________________________________________________________________&#xA;batch_normalization (BatchNorma (None, 32, 32, 512)  2048        conv2d[0][0]                     &#xA;__________________________________________________________________________________________________&#xA;up_sampling2d (UpSampling2D)    (None, 64, 64, 512)  0           batch_normalization[0][0]        &#xA;__________________________________________________________________________________________________&#xA;concatenate (Concatenate)       (None, 64, 64, 768)  0           up_sampling2d[0][0]              &#xA;                                                                 block3_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;zero_padding2d_1 (ZeroPadding2D (None, 66, 66, 768)  0           concatenate[0][0]                &#xA;__________________________________________________________________________________________________&#xA;conv2d_1 (Conv2D)               (None, 64, 64, 256)  1769728     zero_padding2d_1[0][0]           &#xA;__________________________________________________________________________________________________&#xA;batch_normalization_1 (BatchNor (None, 64, 64, 256)  1024        conv2d_1[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;up_sampling2d_1 (UpSampling2D)  (None, 128, 128, 256 0           batch_normalization_1[0][0]      &#xA;__________________________________________________________________________________________________&#xA;concatenate_1 (Concatenate)     (None, 128, 128, 384 0           up_sampling2d_1[0][0]            &#xA;                                                                 block2_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;zero_padding2d_2 (ZeroPadding2D (None, 130, 130, 384 0           concatenate_1[0][0]              &#xA;__________________________________________________________________________________________________&#xA;conv2d_2 (Conv2D)               (None, 128, 128, 128 442496      zero_padding2d_2[0][0]           &#xA;__________________________________________________________________________________________________&#xA;batch_normalization_2 (BatchNor (None, 128, 128, 128 512         conv2d_2[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;up_sampling2d_2 (UpSampling2D)  (None, 256, 256, 128 0           batch_normalization_2[0][0]      &#xA;__________________________________________________________________________________________________&#xA;concatenate_2 (Concatenate)     (None, 256, 256, 192 0           up_sampling2d_2[0][0]            &#xA;                                                                 block1_pool[0][0]                &#xA;__________________________________________________________________________________________________&#xA;zero_padding2d_3 (ZeroPadding2D (None, 258, 258, 192 0           concatenate_2[0][0]              &#xA;__________________________________________________________________________________________________&#xA;conv2d_3 (Conv2D)               (None, 256, 256, 64) 110656      zero_padding2d_3[0][0]           &#xA;__________________________________________________________________________________________________&#xA;batch_normalization_3 (BatchNor (None, 256, 256, 64) 256         conv2d_3[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;up_sampling2d_3 (UpSampling2D)  (None, 512, 512, 64) 0           batch_normalization_3[0][0]      &#xA;__________________________________________________________________________________________________&#xA;conv2d_4 (Conv2D)               (None, 512, 512, 64) 36928       up_sampling2d_3[0][0]            &#xA;__________________________________________________________________________________________________&#xA;batch_normalization_4 (BatchNor (None, 512, 512, 64) 256         conv2d_4[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_5 (Conv2D)               (None, 512, 512, 8)  4616        batch_normalization_4[0][0]      &#xA;__________________________________________________________________________________________________&#xA;activation (Activation)         (None, 512, 512, 8)  0           conv2d_5[0][0]                   &#xA;==================================================================================================&#xA;Total params: 12,363,592&#xA;Trainable params: 12,361,544&#xA;Non-trainable params: 2,048&#xA;__________________________________________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But, in the main &lt;a href=&quot;https://github.com/divamgupta/image-segmentation-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt; page he, reshape the output of conv2d_5 layer (last convolution layer in my model) to a single dimension, that is given below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__________________________________________________________________________________________________&#xA;conv2d_5 (Conv2D)               (None, 512, 512, 8)  4616        batch_normalization_4[0][0]      &#xA;__________________________________________________________________________________________________&#xA;reshape (Reshape)               (None, 262144, None) 0           conv2d_5[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;activation (Activation)         (None, 262144, None) 0           reshape[0][0]                    &#xA;==================================================================================================  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is why this type of reshaping is used here and what is its purpose and benefit? Also when I predict and visualize any image I need to reshape it to (512,512,8) and further processed it. So, what is the benefit such type of reshaping(reshape layer in the above summary) and if I don't use this reshape what will the drawback in my model?&lt;/p&gt;&#xA;" OwnerUserId="40414" LastEditorUserId="40414" LastEditDate="2020-08-22T11:27:57.077" LastActivityDate="2020-08-22T11:27:57.077" Title="Why it is reshaped the last layers of VGG_UNet segmentation model?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23139" PostTypeId="1" CreationDate="2020-08-19T18:15:45.957" Score="0" ViewCount="52" Body="&lt;p&gt;When an AI is trained to play an opposing game, such as chess or go, it can become very strong.&lt;/p&gt;&#xA;&lt;p&gt;I have read in an article (non-scientific) the claim that AI strategies were identified by scientists while an AI was bound to play go games, as well as starcraft games. However it did not tell what these strategies actually were, how they were identified, nor did it explain the configuration in which AI played (AI vs AI? AI vs human?)&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain it to me? I am familiar with go, not with starcraft, so an explanation about go is appreciated.&lt;/p&gt;&#xA;&lt;p&gt;I also note that the chess game is not mentioned. Is there any specific feature for chess that makes them inappropriate for strategies? Or is it the behavior of an AI in the chess game that does not allow to identify strategy?&lt;/p&gt;&#xA;&lt;p&gt;I understand there are plenty of definitions for strategy, and the article did not give one. So let's focus on following significance: Strategy is a group of principles that tell which fields are important to fight for and which are not. A strategy gives long term rewards, which is opposite to tactics with short term rewards obtained thanks to calculation on a specific issue. With this definition, go game stand for a strategic game with a few, well known tactical situations such as line versus line.&lt;/p&gt;&#xA;" OwnerUserId="26866" LastEditorUserId="2444" LastEditDate="2020-08-20T09:49:54.630" LastActivityDate="2020-08-20T09:49:54.630" Title="Were AI strategies identified at go or starcraft games and how?" Tags="&lt;game-ai&gt;&lt;chess&gt;&lt;go&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23140" PostTypeId="2" ParentId="23139" CreationDate="2020-08-19T22:11:25.950" Score="1" Body="&lt;p&gt;Truth be told, I have no idea how to play Go, but luckily this is a AI forum and not a Go forum. Addressing your questions about the specific strategies that AI discovered, there's a paper released by OpenAI titled &amp;quot;Mastering the game of Go without human knowledge&amp;quot; (&lt;a href=&quot;https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge&lt;/a&gt;). Here, there's a section titled &amp;quot;Knowledge learned by AlphaGo Zero.&amp;quot; In it, they mention that AlphaGo Zero discovered some variants of common corner sequences. I think it may be worth checking out. The section also lists some other common Go strategies that AlphaGo Zero learned. I believe during the training process, AlphaGo Zero was trained using self-play, where the AI played itself over and over again to get better.&lt;/p&gt;&#xA;&lt;p&gt;To address your questions about chess. I don't think there's anything limiting AI from learning strategy in chess or anything about chess that limits AI from learning an effective strategy. Check out this paper for more: &lt;a href=&quot;https://deepmind.com/research/publications/Mastering-Atari-Go-Chess-and-Shogi-by-Planning-with-a-Learned-Model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/research/publications/Mastering-Atari-Go-Chess-and-Shogi-by-Planning-with-a-Learned-Model&lt;/a&gt;&#xA;Here the authors present a general reinforcement learning model capable of mastering multiple common board games. I'm not too familiar with strategies in Star Craft, however, here's a link containing some other projects where strategies that emerge by the AI are shown: &lt;a href=&quot;https://openai.com/blog/competitive-self-play/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/competitive-self-play/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40428" LastActivityDate="2020-08-19T22:11:25.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23141" PostTypeId="1" AcceptedAnswerId="23144" CreationDate="2020-08-19T23:27:21.167" Score="2" ViewCount="91" Body="&lt;p&gt;Let's say I have pairs of keys and values of the form &lt;span class=&quot;math-container&quot;&gt;$(x_1, y_1), \dots, (x_N, y_N)$&lt;/span&gt;. Then I give a neural net a key and a value, &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;. For example, &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; could be &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; could be &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;, but this does not have to be the case.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to teach the neural net to output the &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; variable every time it receives the corresponding &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;By the way, how do our brains perform this function?&lt;/p&gt;&#xA;" OwnerUserId="4744" LastEditorUserId="2444" LastEditDate="2020-08-20T12:22:52.380" LastActivityDate="2020-08-23T13:33:07.687" Title="How can we teach a neural net to make arbitrary data associations?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23142" PostTypeId="1" CreationDate="2020-08-20T03:53:55.233" Score="0" ViewCount="15" Body="&lt;p&gt;Are there any deep learning tools so that can be used to estimate long-tail distributions?&lt;/p&gt;&#xA;&lt;p&gt;update: my context is there are is there is a long-tail distribution (probability distribution) in my data of some labels, and I want to estimate it given the data.&lt;/p&gt;&#xA;" OwnerUserId="8415" LastEditorUserId="8415" LastEditDate="2020-08-20T10:07:59.333" LastActivityDate="2020-08-20T10:07:59.333" Title="Are there any deep learning tools so that can be used to estimate long-tail distributions?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;probability-distribution&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23143" PostTypeId="1" CreationDate="2020-08-20T07:14:16.930" Score="0" ViewCount="9" Body="&lt;p&gt;When SMOTE or ADASYN create synthetic sample and the corresponding label, is the quality of the label trust worthy  OR should I should  double check the label by myself ?&lt;/p&gt;&#xA;&lt;p&gt;Particularly in a hypothetical situation of a heavy imbalance digit recognition datasets (10 000 image of digit 9 and 1000 image of digit 10) can it actually create 9000 synthetic sample of the minority digit 10 and correctly label it owns 9000 synthetic sample ?&lt;/p&gt;&#xA;" OwnerUserId="37297" LastActivityDate="2020-08-20T07:14:16.930" Title="Quality of synthetic sample and corresponding label generated by SMOTE or ADASYN?" Tags="&lt;datasets&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23144" PostTypeId="2" ParentId="23141" CreationDate="2020-08-20T08:11:48.963" Score="3" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;In a nutshell : Memorizing is not Learning&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, first let's just remind the classical use of a neural net, in &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supervised Learning&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You have a set of &lt;span class=&quot;math-container&quot;&gt;$(x_{train}, y_{train}) \in X \times Y$&lt;/span&gt; pairs, and you want to extract a general mapping law from &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;You use a neural net function &lt;span class=&quot;math-container&quot;&gt;$f_{\theta} : x \rightarrow f_{\theta}(x)$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; the weights (parameters) of your net.&lt;/li&gt;&#xA;&lt;li&gt;You optimise &lt;span class=&quot;math-container&quot;&gt;$f_{\theta}$&lt;/span&gt; by minimizing the prediction error, represented by the loss function.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Can this solve your question ? Well, I don't think so. With this scheme, your neural net will &lt;em&gt;learn&lt;/em&gt; an appropriate mapping from the set &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to the set &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, but this mapping is appropriate according to your loss function , not to your &lt;span class=&quot;math-container&quot;&gt;$(x_{train}, y_{train})$&lt;/span&gt; pairs.&lt;/p&gt;&#xA;&lt;p&gt;Imagine that a small part of the data is wrongly labelled. A properly trained net learns to extract relevant features and thus will predict correctly the label, not like you did. So the net doesn't memorize your pairs, it infers a general law from the data, and this law may not respect each &lt;span class=&quot;math-container&quot;&gt;$(x_{train}, y_{train})$&lt;/span&gt;.&#xA;So classical Supervised Deep Learning should not memorize &lt;span class=&quot;math-container&quot;&gt;$(x_{train}, y_{train})$&lt;/span&gt;  pairs.&lt;/p&gt;&#xA;&lt;p&gt;However, you could &lt;em&gt;memorize&lt;/em&gt; using a net with too many parameters :  it's &lt;a href=&quot;https://en.wikipedia.org/wiki/Overfitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overfitting&lt;/a&gt; !&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In this case, you set up the net with too many parameters. That gives too much degrees of freedom to your net, and the net will use these DoFs to exactly fit rightly each &lt;span class=&quot;math-container&quot;&gt;$(x_{train}, y_{train})$&lt;/span&gt;  pair you feed during training.&lt;/li&gt;&#xA;&lt;li&gt;However, for an input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; that it never saw during training, &lt;span class=&quot;math-container&quot;&gt;$f_{\theta}(x)$&lt;/span&gt; would have no meaning. That's why we say an overfitted net did not learn, and overfitting is feared by many DL practitioner.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But as long as you want only to memorize, and not to learn, a overfitted net may be the a solution. An other solution for memorization may be &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expert Systems&lt;/a&gt;, I don't know them enough to explain them, but you may check that if you want.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What about the brain ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The matter in answering this question is that we don't really know how does the brain work. I highly recommend &lt;a href=&quot;https://www.semanticscholar.org/paper/The-recent-excitement-about-neural-networks-Crick/66752d19577b2007d4b5e3a20c6ddeb8f1d1e600&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; discussing neural networks and the brain.&lt;/p&gt;&#xA;&lt;p&gt;Some thoughts to start :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The brain has an incredibly huge amount of &lt;em&gt;parameters&lt;/em&gt;, and has a great plasticity. In that sense, we could draw a parallel with overfitted neural networks :  so the brain could be also able to overfit, and thus to memorize by this mean.&lt;/li&gt;&#xA;&lt;li&gt;Our brain is not a feed forward network at all, we can't delimitate any &lt;em&gt;layer&lt;/em&gt;, just some rough zones where we know that some specific information is processed. This makes any parallel between neural nets and the brain difficult.&lt;/li&gt;&#xA;&lt;li&gt;It's still unclear how our brain updates itself. There's no backpropagation for instance. Our overfitted networks also stem from the update processes (for instance, adding regularization to the loss helps avoiding underfitting), but we have no idea of how this works in the brain, so that's another hurdle to drawing parallels !&lt;/li&gt;&#xA;&lt;li&gt;A more personal thought : the brain is able to both learn and memorize (&lt;em&gt;&amp;quot;The exception that proves the rule&amp;quot;&lt;/em&gt; motto shows that I think), while learning and memorizing are antonyms for neural nets...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2020-08-23T13:33:07.687" LastActivityDate="2020-08-23T13:33:07.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23145" PostTypeId="2" ParentId="23133" CreationDate="2020-08-20T08:33:32.850" Score="0" Body="&lt;p&gt;If you start to train a GAN in a way in which the discriminator becomes very powerful than the generator, then the training of generator will not be very much successful. In the same way, if the discriminator is too low in comparison with generator, it would let it produce any thing. So, either way, it will affect the training of the entire system.&lt;br&gt;&lt;br&gt;&#xA;So, if you see here, &lt;code&gt;both generator and discriminator and competing against each other&lt;/code&gt; and on the other side they are dependent on each other for efficient training.&lt;br&gt;&lt;br&gt;&#xA;But I've read in a research paper that adding noise to discriminator is good for the overall stability. And sometimes people also change the labelling system as well. For instance, the value for YES is 1 and people change it to 0.9. This actually keeps the discriminator from being over confident.&lt;/p&gt;&#xA;" OwnerUserId="35612" LastActivityDate="2020-08-20T08:33:32.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23146" PostTypeId="1" CreationDate="2020-08-20T09:44:34.910" Score="1" ViewCount="101" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Additionally, by default, the UpSampling2D layer will use a &lt;strong&gt;nearest neighbor algorithm&lt;/strong&gt; to fill in the new rows and columns. This has the effect of simply doubling rows and columns, as described and is specified by the ‘&lt;strong&gt;interpolation&lt;/strong&gt;‘ argument set to ‘nearest‘. Alternately, a &lt;strong&gt;bilinear interpolation&lt;/strong&gt; method can be used which draws upon multiple surrounding points. This can be specified via setting the ‘interpolation‘ argument to ‘bilinear‘.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How exactly does the &lt;strong&gt;nearest neighbor&lt;/strong&gt; algorithm mentioned above work? Also, what does &lt;strong&gt;interpolation&lt;/strong&gt; mean in this context (&lt;em&gt;&lt;strong&gt;nearest&lt;/strong&gt;&lt;/em&gt; and &lt;em&gt;&lt;strong&gt;bilinear&lt;/strong&gt;&lt;/em&gt;)?&lt;/p&gt;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://machinelearningmastery.com/upsampling-and-transpose-convolution-layers-for-generative-adversarial-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Section on Upsampling2D layer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-08-20T09:44:34.910" Title="What's the nearest neighbor algorithm used for upsampling?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;interpolation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23147" PostTypeId="2" ParentId="20773" CreationDate="2020-08-20T10:59:13.013" Score="3" Body="&lt;p&gt;DDPG is an off-policy algorithm simply because of the objective taking expectation with respect to some other distribution that we are not learning about, i.e. the deterministic policy gradient can be expressed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla _{\theta^\mu} J \approx \mathbb{E}_{s_t \sim \rho^\beta} \left[ \nabla _{\theta^\mu} Q(s,a|\theta^Q) | s=s_t, a=\mu(s_t ; \theta ^\mu)  \right]\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We are interested in learning about the policy parameters of &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, denoted by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, but we take expected with respect to some discounted state distribution induced by a policy &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;, which we will denote as &lt;span class=&quot;math-container&quot;&gt;$\rho^\beta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To summarise, we are learning off-policy as the expectation of the gradient is taken with respect to some state distribution that occurs under some policy that we are not learning about.&lt;/p&gt;&#xA;&lt;p&gt;Given that on-policy learning is a special case of off-policy learning, if the replay buffer had a size of one, i.e. we use only the most recent experience tuple to perform parameter updates, then DDPG would be on-policy.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-05-09T23:13:58.287" LastActivityDate="2021-05-09T23:13:58.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23148" PostTypeId="1" CreationDate="2020-08-20T11:07:43.970" Score="1" ViewCount="17" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/3058/how-do-recommendation-systems-work&quot;&gt;This&lt;/a&gt; does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the application of SVD. Like movie recommendation system, Google page ranking system, etc.&lt;/p&gt;&#xA;&lt;p&gt;Now in the case of movie recommendation system, what I had as a mental picture is...&lt;/p&gt;&#xA;&lt;p&gt;The SVD is a technique that falls under collaborative filtering. And what the SVD does is factor a big data matrix into two smaller matrix. And as an input to the SVD we give an incomplete data matrix. And SVD gives us a probable complete data matrix. Here, in the case of a movie recommendation system we try to predict ratings of users. Incomplete input data matrix means some users didn't give ratings to certain movies. So the SVD will help to predict users' ratings. I still don't know how the SVD breaks down a large matrix to smaller pieces. I don't how the SVD determines the dimensions of the smaller matrices.&lt;/p&gt;&#xA;&lt;p&gt;It would be helpful if anyone could judge my understanding. And I will very much appreciate any resources which can help me to understand the SVD from scratch to its application to Netflix recommendation systems. Also for the Google Page ranking system or for other applications.&lt;/p&gt;&#xA;&lt;p&gt;I am looking forward to seeing an explanation more from human-intuition level and from a linear-algebra point of view. Because I am interested in using this algorithm in my research, I need to understand as soon as possible: how does the SVD work deep down from the core?&lt;/p&gt;&#xA;" OwnerUserId="40437" LastActivityDate="2020-08-20T11:07:43.970" Title="Human intuition behind SVD in case of recommendation system" Tags="&lt;machine-learning&gt;&lt;recommender-system&gt;&lt;linear-algebra&gt;&lt;singular-value-decomposition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23150" PostTypeId="1" CreationDate="2020-08-20T14:59:42.830" Score="0" ViewCount="35" Body="&lt;p&gt;I want to develop a neural network for my data, but I have no idea if my problem should be solved with curve fitting, classification or maybe something else.&lt;/p&gt;&#xA;&lt;p&gt;Here is my problem.&lt;/p&gt;&#xA;&lt;p&gt;My input data are some points like &lt;code&gt;[1.32 1.42 0.21 0.42 0.7 .91 1.23 1.3]&lt;/code&gt;, which I want them to turn into &lt;code&gt;[1 0 0 0 0 1 0 0]&lt;/code&gt;. I can guess what kind of transformation functions should I use, but, when it comes to developing the network, I've got no idea which approach should I follow.&lt;/p&gt;&#xA;" OwnerUserId="40443" LastEditorUserId="2444" LastEditDate="2020-08-21T11:30:14.660" LastActivityDate="2020-08-21T11:30:14.660" Title="What kind of ANN should I use for this problem?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;pattern-recognition&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23151" PostTypeId="1" CreationDate="2020-08-20T15:14:52.807" Score="0" ViewCount="23" Body="&lt;p&gt;I have been reading the research paper &lt;a href=&quot;https://arxiv.org/pdf/1802.10171v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tell Me Where to Look: Guided Attention Inference Network&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In this paper, they calculate the attention loss, but I didn't understand how to calculate it. Do we have to calculate it like &lt;code&gt;outcome[c]&lt;/code&gt;? If it is so, then why do arrows connect with each other from the middle FC and the last FC?&lt;/p&gt;&#xA;&lt;p&gt;Here is the image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YagSc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YagSc.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-08-21T11:48:06.390" LastActivityDate="2020-08-21T11:48:06.390" Title="How to calculate the attention loss in the paper &quot;Tell Me Where to Look: Guided Attention Inference Network&quot;?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;loss-functions&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23154" PostTypeId="1" CreationDate="2020-08-20T17:45:01.883" Score="1" ViewCount="256" Body="&lt;p&gt;I am designing a deep autoencoder for graph embedding (exactly node embedding) following this paper &lt;a href=&quot;https://www.kdd.org/kdd2016/papers/files/rfp0191-wangAemb.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SDNE&lt;/a&gt;. In the original paper, they used the &lt;em&gt;sigmoid&lt;/em&gt; activation for all hidden layers in the autoencoder model, even for the embedding layer.&lt;/p&gt;&#xA;&lt;p&gt;However, I think the embedding layer should use the &lt;em&gt;tanh&lt;/em&gt; activation and the reconstruction layer should be used &lt;em&gt;ReLU&lt;/em&gt; activation. Because, embedding is in the range &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt; and reconstruction layer is in the range &lt;span class=&quot;math-container&quot;&gt;$[0, x]$&lt;/span&gt;, which generates better results due to a larger range for representation and directed graph. Instead of in the range &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; from &lt;em&gt;sigmoid&lt;/em&gt; will lead to a lack of embedding information.&lt;/p&gt;&#xA;&lt;p&gt;So, what is the best activation function for deep autoencoders to capture good information about the structure of graph?&lt;/p&gt;&#xA;" OwnerUserId="25645" LastEditorUserId="2444" LastEditDate="2020-08-21T11:52:10.870" LastActivityDate="2021-02-28T18:05:50.613" Title="What is the best activation function for the embedding layer in a deep auto-encoder?" Tags="&lt;machine-learning&gt;&lt;activation-function&gt;&lt;autoencoders&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23155" PostTypeId="2" ParentId="23124" CreationDate="2020-08-20T18:04:10.807" Score="0" Body="&lt;p&gt;From Andrew lesson on Coursera, &lt;code&gt;batch_size&lt;/code&gt; should be the power of 2, ex: 512, 1024, 2048. It will faster for training.&lt;/p&gt;&#xA;&lt;p&gt;And you don't need to drop your last images to &lt;code&gt;batch_size&lt;/code&gt; of 5 for example. The library likes Tensorflow or Pytorch, the last &lt;code&gt;batch_size&lt;/code&gt; will be &lt;code&gt;number_training_images % 5&lt;/code&gt; which 5 is your &lt;code&gt;batch_size&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Last but not least, batch_size need to fit your memory training (CPU or GPU). You can try several large batch_size to know which value is not out of memory. The smaller &lt;code&gt;number_mini_batch = number_training_image//batch_size + 1&lt;/code&gt;, the faster for training time.&lt;/p&gt;&#xA;&lt;p&gt;Hope they can help you!&lt;/p&gt;&#xA;" OwnerUserId="25645" LastActivityDate="2020-08-20T18:04:10.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23156" PostTypeId="1" CreationDate="2020-08-20T18:27:29.260" Score="0" ViewCount="16" Body="&lt;p&gt;I am using Mask-RCNN to detect human bodies in photos, to get a rough approximation of the ratio of their heights to the length of their chests. I want to decrease the time for making the mask of the human being.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way I could alter its code so that it only focuses on detecting people and hence, it's running time decreases?&lt;/p&gt;&#xA;&lt;p&gt;I don't want it to waste its time to check for other classes.&lt;/p&gt;&#xA;" OwnerUserId="40305" LastEditorUserId="2444" LastEditDate="2020-08-21T11:55:48.293" LastActivityDate="2020-08-21T11:55:48.293" Title="How can I decrease the time to compute the mask in the Mask-RCNN for human body detection?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;deep-neural-networks&gt;&lt;r-cnn&gt;&lt;mask-rcnn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23157" PostTypeId="1" CreationDate="2020-08-20T20:57:30.040" Score="0" ViewCount="22" Body="&lt;p&gt;I've used &lt;a href=&quot;https://github.com/tzutalin/labelImg&quot; rel=&quot;nofollow noreferrer&quot;&gt;LabelImg&lt;/a&gt; to create labels for my images using YOLO. After that, I would like to input the images and labels into a CNN model, like a VGG or ResNet. I've searched a lot and have not found anything like this.&lt;/p&gt;&#xA;&lt;p&gt;Can someone tell me if it's doable or not? And if yes, is there a tutorial or code I can read to help? I would like to use a model other than YOLO and R-CNN&lt;/p&gt;&#xA;" OwnerUserId="40451" LastEditorUserId="2444" LastEditDate="2020-08-21T11:57:13.810" LastActivityDate="2020-08-21T11:57:13.810" Title="Object detection using CNN model architectures" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23158" PostTypeId="1" CreationDate="2020-08-20T21:32:54.600" Score="1" ViewCount="42" Body="&lt;p&gt;When I build a convolution layer for image processing, the filter parameters should have 3 dimensions, &lt;code&gt;(filter_length, filter_width, color_depth)&lt;/code&gt; is that correct?&lt;/p&gt;&#xA;&lt;p&gt;Why does this convolution layer call &lt;code&gt;Conv2D&lt;/code&gt;? Where does the &lt;code&gt;2&lt;/code&gt; come from?&lt;/p&gt;&#xA;" OwnerUserId="37178" LastEditorUserId="2444" LastEditDate="2020-08-21T12:05:25.577" LastActivityDate="2020-08-21T12:05:25.577" Title="Why is the convolution layer called Conv2D?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;image-recognition&gt;&lt;keras&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23159" PostTypeId="1" AcceptedAnswerId="23168" CreationDate="2020-08-21T01:08:31.910" Score="1" ViewCount="646" Body="&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential([&#xA;    tf.keras.layers.Embedding(1000, 16, input_length=20), &#xA;    tf.keras.layers.Dropout(0.2),                           # &amp;lt;- How does the dropout work?&#xA;    tf.keras.layers.Conv1D(64, 5, activation='relu'),&#xA;    tf.keras.layers.MaxPooling1D(pool_size=4),&#xA;    tf.keras.layers.LSTM(64),&#xA;    tf.keras.layers.Dense(1, activation='sigmoid')&#xA;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can understand when dropout is applied between &lt;code&gt;Dense layers&lt;/code&gt;, which randomly drops and prevents the former layer neurons from updating parameters. I don't understand how dropout works after an embedding layer.&lt;/p&gt;&#xA;&lt;p&gt;Let's say the output shape of the Embedding layer is &lt;code&gt;(batch_size,20,16)&lt;/code&gt; or simply &lt;code&gt;(20,16)&lt;/code&gt; if we ignore the batch size. How is dropout applied to the embedding layer's output?&lt;/p&gt;&#xA;&lt;p&gt;Randomly dropout rows or columns?&lt;/p&gt;&#xA;" OwnerUserId="37178" LastEditorUserId="2444" LastEditDate="2020-08-21T12:08:19.880" LastActivityDate="2020-08-21T13:45:28.780" Title="How is dropout applied to the embedding layer's output?" Tags="&lt;natural-language-processing&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;word-embedding&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23160" PostTypeId="1" CreationDate="2020-08-21T04:27:51.967" Score="1" ViewCount="22" Body="&lt;p&gt;Suppose the following properties of a board game:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;High branching factor in the beginning of the game (~500) which slowly tends towards 0 at the end of the game&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Evaluation of the any given board state isn't hard to create and can be quite accurate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;And that we want to create an AI to play such board game.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;What method of tree searching should be applied for the AI?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Considering the absurd branching factor (at least for most of the game), the Monte Carlo method of search is appealing. The problem is that from what I've seen usually monte carlo search methods are used on games with both high branching factor and &lt;em&gt;no easy evaluation function&lt;/em&gt;. However that is not the case for this board game as previously stated.&lt;/p&gt;&#xA;&lt;p&gt;I'm simply curious how this property of evaluation should influence my decision. For example: Should I replace simulations and playouts with an evaluation function? At that point, would alpha-beta pruning minimax work better? Is there some hybrid which would be optimal?&lt;/p&gt;&#xA;" OwnerUserId="40455" LastActivityDate="2020-08-21T04:27:51.967" Title="Which method of tree searching should be used for this board game?" Tags="&lt;algorithm&gt;&lt;optimization&gt;&lt;search&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23161" PostTypeId="1" AcceptedAnswerId="23163" CreationDate="2020-08-21T05:01:32.977" Score="4" ViewCount="50" Body="&lt;p&gt;From &lt;a href=&quot;https://arxiv.org/abs/1406.2661&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goodfellow et al. (2014)&lt;/a&gt;, we have the adversarial loss:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \min_G \, \max_D V (D, G) = \mathbb{E}_{x∼p_{data}(x)} \, [\log \, D(x)] \\&#xA;\quad\quad\quad\quad\quad\quad\quad  + \, \mathbb{E}_{z∼p_z(z)} \, [\log \, (1 − D(G(z)))] \, \text{.} \quad$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In practice, the expectation is computed as a mean over the minibatch. For example, the discriminator loss is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_{\theta_{d}} \frac{1}{m} \sum_{i=1}^{m}\left[\log D\left(\boldsymbol{x}^{(i)}\right)+\log \left(1-D\left(G\left(\boldsymbol{z}^{(i)}\right)\right)\right)\right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is: &lt;strong&gt;why is the mean used to compute the expectation&lt;/strong&gt;? Does this imply that &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; is &lt;strong&gt;uniformly distributed&lt;/strong&gt;, since every sample must be drawn from &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; with equal probability?&lt;/p&gt;&#xA;&lt;p&gt;The expectation, expressed as an integral, is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;V(G, D) &amp;amp;=\int_{\boldsymbol{x}} p_{\text {data }}(\boldsymbol{x}) \log (D(\boldsymbol{x})) d x+\int_{\boldsymbol{z}} p_{\boldsymbol{z}}(\boldsymbol{z}) \log (1-D(g(\boldsymbol{z}))) d z \\&#xA;&amp;amp;=\int_{\boldsymbol{x}} p_{\text {data }}(\boldsymbol{x}) \log (D(\boldsymbol{x}))+p_{g}(\boldsymbol{x}) \log (1-D(\boldsymbol{x})) d x&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So, how do we go from an integral involving a continuous distribution to summing over discrete probabilities, and further, that all those probabilities are the same?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The best I could find from other StackExchange posts is that the mean is just an approximation, but I'd really like a more rigorous explanation.&lt;/p&gt;&#xA;&lt;p&gt;This question isn't exclusive to GANs, but is applicable to any loss function that is expressed mathematically as an expectation over some sampled distribution, which is not implemented directly via the integral form.&lt;/p&gt;&#xA;&lt;p&gt;(All equations are from the Goodfellow paper.)&lt;/p&gt;&#xA;" OwnerUserId="40197" LastEditorUserId="2444" LastEditDate="2020-08-21T19:05:27.940" LastActivityDate="2020-08-21T19:05:27.940" Title="Why is the mean used to compute the expectation in the GAN loss?" Tags="&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;&lt;probability-distribution&gt;&lt;expectation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23162" PostTypeId="1" AcceptedAnswerId="23250" CreationDate="2020-08-21T05:37:41.317" Score="2" ViewCount="74" Body="&lt;p&gt;When training a network using word embeddings, it is standard to add an embedding layer to first convert the input vector to the embeddings.&lt;/p&gt;&#xA;&lt;p&gt;However, assuming the embeddings are pre-trained and frozen, there is another option. We could simply preprocess the training data prior to giving it to the model so that it is already converted to the embeddings. This will speed up training, since this conversion need only be performed once, as opposed to on the fly for each epoch.&lt;/p&gt;&#xA;&lt;p&gt;Thus, the second option seems better. But the first choice seems more common. Assuming the embeddings are pre-trained and frozen, is there a reason I might choose the first option over the second?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="12201" LastEditDate="2020-08-21T14:12:04.043" LastActivityDate="2020-08-25T13:36:46.010" Title="When to convert data to word embeddings in NLP" Tags="&lt;natural-language-processing&gt;&lt;data-preprocessing&gt;&lt;word-embedding&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23163" PostTypeId="2" ParentId="23161" CreationDate="2020-08-21T10:15:56.273" Score="4" Body="&lt;p&gt;It seems your question is concerned with how an empirical mean works.&lt;/p&gt;&#xA;&lt;p&gt;It is indeed true that, if all &lt;span class=&quot;math-container&quot;&gt;$x^{(i)}$&lt;/span&gt; are independent identically distributed realisations of a random variable &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\lim_{n \rightarrow \infty} \frac{1}{n}\sum_{i=1}^n f(x^{(i)}) = \mathbb{E}[f(X)]$&lt;/span&gt;. This is a standard result in statistics known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_large_numbers#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;the law of large numbers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-21T10:15:56.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23164" PostTypeId="5" CreationDate="2020-08-21T11:32:39.210" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-21T11:32:39.210" LastActivityDate="2020-08-21T11:32:39.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23165" PostTypeId="4" CreationDate="2020-08-21T11:32:39.210" Score="0" Body="For questions related to the filters (also known as kernels) of a convolutional layer (of a convolutional neural network) or, in general, used in a convolution operation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-21T11:32:39.210" LastActivityDate="2020-08-21T11:32:39.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23166" PostTypeId="1" CreationDate="2020-08-21T11:33:47.787" Score="0" ViewCount="15" Body="&lt;p&gt;I want to make a covariance matrix that incorporates my belief of how correlated the various dimensions are. The reason why I want to incorporate my belief is that in my modelling, the dimensions are food items and I want to manually correlate which food items are similar to which other in my covariance matrix. Is there any way to generate a valid covariance matrix(ensuring positive definiteness, though symmetry is easy)? After making such a covariance matrix, I intend to do a Cholesky decomposition and use it to uncorrelate my data-points. Are there any ways to do an approximate Cholesky decomposition, given a non-positive definite symmetric matrix?&lt;/p&gt;&#xA;&lt;p&gt;Any help would be appreciated!!!&lt;/p&gt;&#xA;" OwnerUserId="35576" LastActivityDate="2020-08-21T11:33:47.787" Title="How to decompose a non-positive definite matrix in the same manner as Cholseky decomposition?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;linear-algebra&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23167" PostTypeId="2" ParentId="23158" CreationDate="2020-08-21T12:02:32.007" Score="2" Body="&lt;p&gt;A 2D convolution is a convolution where the kernel has the same depth as the input, so, in theory, you do not need to specify the depth of the kernel, if you know the depth of the input.&lt;/p&gt;&#xA;&lt;p&gt;I don't know which library you are referring to (although you tagged your post with TensorFlow and Keras), but, in TensorFlow, you only need to specify the &lt;em&gt;width&lt;/em&gt; and &lt;em&gt;height&lt;/em&gt; of the kernel in the &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Conv2D&lt;/code&gt;&lt;/a&gt; class, given that the depth is automatically calculated from the depth of the input.&lt;/p&gt;&#xA;&lt;p&gt;Thus, the &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; comes from the fact that you slide across two dimensions (i.e. width and height).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, in a 3D convolution, the depth of the kernel does not necessarily have the same depth of the input, so, in that case, you also slide across the depth. In TensorFlow, you need to specify the width, height and depth of the &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Conv3D&lt;/code&gt;&lt;/a&gt; class.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-08-21T12:02:32.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23168" PostTypeId="2" ParentId="23159" CreationDate="2020-08-21T13:45:28.780" Score="1" Body="&lt;p&gt;It doesn't drops rows or columns, it acts directly on scalars. The &lt;a href=&quot;https://keras.io/api/layers/regularization_layers/dropout/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout Layer keras documentation&lt;/a&gt; explains it and illustrates it with an example :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The Dropout layer randomly sets input units to 0 with a frequency of rate&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;After an Dense Layer, the Dropout inputs are directly the outputs of the Dense layer neurons, as you said. After your embedding layer, in your case, you should have &lt;code&gt;rate * (16 * input_length) = 0.2 * 20 * 16 = 64&lt;/code&gt; inputs set to 0 out of the 320 scalars inputs. These 64 &lt;em&gt;dropped&lt;/em&gt; inputs are randomly selected in the 20x16 grid. Note that the Dropout rescales the non dropped inputs by multiplicating them by a factor &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{1-rate}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17759" LastActivityDate="2020-08-21T13:45:28.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23169" PostTypeId="1" CreationDate="2020-08-21T14:27:47.450" Score="1" ViewCount="57" Body="&lt;p&gt;This is a generic question. Still posting it to get insights from experts in the field.&lt;/p&gt;&#xA;&lt;p&gt;I am interested in knowing if Neural Networks are used in general apart from specific hi-tech organizations.&lt;/p&gt;&#xA;&lt;p&gt;If so, which type of NN is used in which industry and for what purpose?&lt;/p&gt;&#xA;" OwnerUserId="15267" LastEditorUserId="2444" LastEditDate="2020-08-21T17:30:14.257" LastActivityDate="2020-09-20T18:00:50.183" Title="Are neural networks really used apart from specific hi-tech organisations?" Tags="&lt;neural-networks&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23171" PostTypeId="1" CreationDate="2020-08-21T16:00:00.063" Score="0" ViewCount="22" Body="&lt;p&gt;In most of the cases the embedding vectors of last 4 layers of BERT are summed up to represent the tokens embedding. I've tried to explore but haven't found any strong reason/resource on why we should be considering only the last 4 layers.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="25676" LastEditDate="2020-08-21T17:44:06.993" LastActivityDate="2020-08-21T17:44:06.993" Title="Why BERT last 4 Layers should be considered to extract word embeddings?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23172" PostTypeId="1" CreationDate="2020-08-21T16:59:03.197" Score="0" ViewCount="38" Body="&lt;p&gt;I am practicing with an image dataset which is having different dimensions. &lt;a href=&quot;https://i.stack.imgur.com/FnnVo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FnnVo.png&quot; alt=&quot;here is the sample images&#xA;&quot; /&gt;&lt;/a&gt;&#xA;If I simply crop and pad them to 1024X1024(the original images having smallest width is around 300 and largest is around 2400 and widths and heights of the images are not the same) I am not getting good val_accuracy. It's just giving 49% accuracy.&#xA;How to do image processing to these images because the brightness of the images is also changing. My task is to classify them into 5 classes.&lt;/p&gt;&#xA;" OwnerUserId="38737" LastActivityDate="2021-05-20T08:08:07.010" Title="How to train the images of various sizes?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23173" PostTypeId="2" ParentId="23169" CreationDate="2020-08-21T17:02:30.537" Score="2" Body="&lt;p&gt;Before putting up my experience I'd like to provide some facts about AI startups and established Hi-Tech companies.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.thequint.com/tech-and-auto/tech-news/40-percent-european-ai-startups-not-actually-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Most of companies claiming to use AI doesn't actually use ML/DL&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;And there are products by some companies that are classified to be using AI since they just use linear regression&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Coming to my experience I can say that AI is passively as well as actively utilized in product development and services. Though Neural Networks are the driving force behind the current AI tech, the thing that we study in the text books are just basic building blocks that are used to designing complex network architectures. Though most of High-Tech companies as you have mentioned might not be willing to share their secrets, the publications they make in conferences are good enough. I would suggest one to go through the papers that are published by companies like NVIDIA, Microsoft, and Google to get understanding on how complex networks are designed to address specific problems.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastEditorUserId="25676" LastEditDate="2020-08-21T17:52:06.180" LastActivityDate="2020-08-21T17:52:06.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23175" PostTypeId="1" CreationDate="2020-08-21T17:56:45.743" Score="1" ViewCount="34" Body="&lt;p&gt;I am trying to solve the following problem it is to classify the the red points&#xA;and green points in image 1 into two cases.  The cluster of green or&#xA;red points can be anywhere and there can be any number of green or red&#xA;clusters; different coloured clusters do not mix or bleed into each other;&#xA;at least one green and one red cluster always exists.&#xA;An example of points to classify is given in image 1.&lt;/p&gt;&#xA;&lt;p&gt;So I guess there are two ways to do this&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Classify then with boundaries as shown in image 2, with some algorithm, then use a some post&#xA;processing step to link the separate classes that have the same colour.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use some algorithm to directly find the classification boundaries as shown in image 3.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So my question is what is the  algorithm or algorithm can I use for 1) and 2)?&lt;/p&gt;&#xA;&lt;p&gt;It seems it can be solved using the MLE algorithm in some way.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0b0XG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0b0XG.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4ijWu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4ijWu.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yjF1p.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yjF1p.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40467" LastEditorUserId="40467" LastEditDate="2020-08-21T18:03:00.467" LastActivityDate="2020-08-21T18:03:00.467" Title="What Classification Algorithm Do I need to Use to Solve this Problem?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23176" PostTypeId="5" CreationDate="2020-08-21T19:02:17.467" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-21T19:02:17.467" LastActivityDate="2020-08-21T19:02:17.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23177" PostTypeId="4" CreationDate="2020-08-21T19:02:17.467" Score="0" Body="For theoretical questions that involve the Theano library, but note that programming questions are off-topic here." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-21T19:02:17.467" LastActivityDate="2020-08-21T19:02:17.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23178" PostTypeId="1" CreationDate="2020-08-21T19:52:04.467" Score="0" ViewCount="28" Body="&lt;p&gt;My purpose is to apply deep learning for planning. To do so, I decided to use a similar approach as AlphaGo.&lt;/p&gt;&#xA;&lt;p&gt;But my &amp;quot;game state&amp;quot; is very different. Instead of considering some planes representing the board, my &amp;quot;game state&amp;quot; is a list of different work packages with their attributes (type, mode ...).&lt;/p&gt;&#xA;&lt;p&gt;Just as AlphaGo, the purpose of the neural network is to predict the probability of choosing each work package and the value of the state which is a score which reflects the total cost and the total duration of my final plan.&lt;/p&gt;&#xA;&lt;p&gt;I've already code all the &amp;quot;game rules&amp;quot;, the MCTS is also already ready.&lt;/p&gt;&#xA;&lt;p&gt;I've got 2 issues about that :&lt;/p&gt;&#xA;&lt;p&gt;1- I don't have the same number of activity for each state. That means that I need a flexible size input for the NN&lt;/p&gt;&#xA;&lt;p&gt;2-(which is linked) I really don't know which type of structure could be interesting for my neural network.&lt;/p&gt;&#xA;&lt;p&gt;Have you got some tips to help me for this ?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="40473" LastActivityDate="2020-08-21T19:52:04.467" Title="Structure for neural network" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;alphago-zero&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23179" PostTypeId="2" ParentId="22602" CreationDate="2020-08-21T19:59:10.493" Score="0" Body="&lt;p&gt;Intuitively I think there's definitely something to be said for your idea, but it's not a 100% clear case, and there are also some arguments to be made for the case that we should also be training the policy from data where &lt;span class=&quot;math-container&quot;&gt;$z_t = -1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, first let's establish that if we do indeed choose to discard any and all data where &lt;span class=&quot;math-container&quot;&gt;$z_t = -1$&lt;/span&gt;, we are in fact discarding a really significant part of our data; &lt;strong&gt;we're discarding 50% of all the data we generate&lt;/strong&gt; in games like Go where there are no draws (less than that in games like Chess where there are many draws, but still a significant amount of data). So this is not a decision to be made lightly (it has a major impact on our sample efficiency), and we should probably only do it if we really believe that policy learning from any data where &lt;span class=&quot;math-container&quot;&gt;$z_t = -1$&lt;/span&gt; is actually harmful.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The primary idea behind the self-play learning process in AlphaGo Zero / AlphaZero may intuitively be explained as:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When we run an MCTS search biased by a trained policy &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;, we expect the resulting distribution of visits to be slightly better than what was produced by &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt; alone.&lt;/li&gt;&#xA;&lt;li&gt;According to the expectation from point 1., we may use the visit counts of MCTS as a training target for the policy &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;, and hence we expect to get a slight improvement in the quality of that trained policy.&lt;/li&gt;&#xA;&lt;li&gt;If we were to now run a new MCTS search biased by the updated policy in the same situation again, we would expect that to perform even better than it previously did because it is now biased by a new policy which has improved in comparison to the policy we previously used.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Of course, there can be exceptions to point 1. if we get unlucky, but on average we expect that to be true. Crucially for your question, we don't expect this to only be true in games where we actually won, but still also be true in games that we ultimately end up losing. &lt;strong&gt;Even if we still end up losing the game played according to the MCTS search, we expect that we at least put up a slightly better fight with the MCTS + &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt; combo than we would have done with just &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;&lt;/strong&gt;, and so it may still be useful to learn from it (to at least lose less badly).&lt;/p&gt;&#xA;&lt;p&gt;On top of this, it is important to consider that &lt;strong&gt;we intentionally build in exploration mechanisms in the self-play training process&lt;/strong&gt;, which may &amp;quot;pollute&amp;quot; the signal &lt;span class=&quot;math-container&quot;&gt;$z_t$&lt;/span&gt; without having polluted the training target for the policy. In self-play, we do not always pick the action with the maximum visit count (as we would in an evaluation match / an important tournament game), but we pick actions proportionally to the MCTS visit counts. This is done for exploration, to introduce extra variety in the experience that we generate, to make sure that we do not always learn from exactly the same games. This can clearly affect the &lt;span class=&quot;math-container&quot;&gt;$z_t$&lt;/span&gt; signal (because sometimes we knowingly make a very very bad move just for the sake of exploration), but it does not affect the policy training targets encountered throughout that game; MCTS still tries to make the best that it can out of the situations it faces. So, these policy training targets are still likely to be useful, even if we &amp;quot;intentionally&amp;quot; made a mistake somewhere along the way which caused us to lose the game.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-08-21T19:59:10.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23180" PostTypeId="1" CreationDate="2020-08-21T20:00:04.873" Score="4" ViewCount="686" Body="&lt;p&gt;In section 3 of the paper &lt;a href=&quot;https://arxiv.org/pdf/1509.02971.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continuous control with deep reinforcement learning&lt;/a&gt;, the authors write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As detailed in the supplementary materials we used an &lt;strong&gt;Ornstein-Uhlenbeck process&lt;/strong&gt; (Uhlenbeck &amp;amp; Ornstein, 1930) to generate temporally correlated exploration for exploration efficiency in physical control problems with inertia (similar use of autocorrelated noise was introduced in (Wawrzynski, 2015)).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In section 7, they write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For the exploration noise process we used temporally correlated noise in order to explore well in physical environments that have momentum. We used an &lt;strong&gt;Ornstein-Uhlenbeck process&lt;/strong&gt; (Uhlenbeck &amp;amp; Ornstein, 1930) with θ = 0.15 and σ = 0.2. The Ornstein-Uhlenbeck process models the velocity of a Brownian particle with friction, which results in temporally correlated values centered around 0.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In a few words, what is the Ornstein-Uhlenbeck process? How does it work? How exactly is it used in DDPG?&lt;/p&gt;&#xA;&lt;p&gt;I want to implement the Deep Deterministic Policy Gradient algorithm, and, in the initial actions, noise has to be added. However, I cannot understand how this Ornstein-Uhlenbeck process works. I have searched the internet, but I have not understood the information that I found.&lt;/p&gt;&#xA;" OwnerUserId="40477" LastEditorUserId="2444" LastEditDate="2020-08-21T22:23:08.060" LastActivityDate="2020-08-23T06:08:52.340" Title="How does the Ornstein-Uhlenbeck process work, and how it is used in DDPG?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;papers&gt;&lt;ddpg&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23181" PostTypeId="1" CreationDate="2020-08-21T20:02:19.910" Score="0" ViewCount="34" Body="&lt;p&gt;I built a CNN model for extracting features from its dense layer. The extracted features are then used for classification using KNN and Random Forest classifier.&lt;/p&gt;&#xA;&lt;p&gt;My question is, can I use the whole dataset to train the CNN feature extractor as it is not used for the classification purpose?&lt;/p&gt;&#xA;" OwnerUserId="40474" LastEditorUserId="2444" LastEditDate="2020-08-23T17:23:36.643" LastActivityDate="2020-08-23T17:23:36.643" Title="Can I use the full dataset for training the CNN feature extractor?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23184" PostTypeId="1" CreationDate="2020-08-22T04:01:05.723" Score="1" ViewCount="36" Body="&lt;p&gt;If the title wan not very clear, I want a method to take an input image like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[0, 0, 0, 0],&#xA; [1, 1, 1, 0],&#xA; [1, 1, 1, 0],&#xA; [0, 1, 1, 0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and output the 2D coordinates of the &lt;code&gt;1&lt;/code&gt;s of the image (So that I can recreate the image)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The application is a robot creating the image using some kind of building blocks, placing one block after the other&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want the output to be sequential because I need to reconstruct the input image pixel by pixel and there are some conditions on the image construction order (e.g. You cannot place a &lt;code&gt;1&lt;/code&gt; somewhere when it is surrounded by &lt;code&gt;1&lt;/code&gt;s)&lt;/p&gt;&#xA;&lt;p&gt;The image can change and the number of &lt;code&gt;1&lt;/code&gt;s in the image too.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is an appropriate AI method to apply in this case?&lt;/li&gt;&#xA;&lt;li&gt;How should I feed the image to the network? (Will flattening it to 2D affect my need of an output order?)&lt;/li&gt;&#xA;&lt;li&gt;Should I get the output coordinates one by one or as an ordered 2xN matrix?&lt;/li&gt;&#xA;&lt;li&gt;If one by one, should I feed the same image for each output or the image without the &lt;code&gt;1&lt;/code&gt;s already filled?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I have tried to apply &amp;quot;NeuroEvolution of Augmenting Topologies&amp;quot; for this using &lt;a href=&quot;https://neat-python.readthedocs.io/en/latest/neat_overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;neat-python&lt;/a&gt; but was unsuccessful. I am currently looking at RNNs but I am not sure if it is the best choice either.&lt;/p&gt;&#xA;" OwnerUserId="40482" LastActivityDate="2020-08-22T09:41:34.157" Title="Choosing an AI method to recreate a given binary 2D image" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;recurrent-neural-networks&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23185" PostTypeId="1" CreationDate="2020-08-22T05:32:13.460" Score="0" ViewCount="21" Body="&lt;p&gt;I wanted to have a common critic for all my agents so I defined it as following but because the critic will be changing while training in every iteration, I am not sure if all agents still share the same critic network or they will eventually have the different critic in the process?&#xA;Also, If they will eventually have different critics, how can I implement the critic such that all the agents share the same critic weights even after training every iteration?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import random&#xA;import copy&#xA;from collections import namedtuple, deque&#xA;&#xA;from model import Actor, Critic&#xA;&#xA;import torch&#xA;import torch.nn.functional as F&#xA;import torch.optim as optim&#xA;&#xA;BUFFER_SIZE = int(1e6)  # replay buffer size&#xA;BATCH_SIZE = 128        # minibatch size&#xA;LR_ACTOR = 1e-3         # learning rate of the actor&#xA;LR_CRITIC = 1e-3        # learning rate of the critic&#xA;WEIGHT_DECAY = 0        # L2 weight decay&#xA;LEARN_EVERY = 1         # learning timestep interval&#xA;LEARN_NUM = 5           # number of learning passes&#xA;GAMMA = 0.99            # discount factor&#xA;TAU = 8e-3              # for soft update of target parameters&#xA;OU_SIGMA = 0.2          # Ornstein-Uhlenbeck noise parameter, volatility&#xA;OU_THETA = 0.15         # Ornstein-Uhlenbeck noise parameter, speed of mean reversion&#xA;EPS_START = 5.0         # initial value for epsilon in noise decay process in Agent.act()&#xA;EPS_EP_END = 300        # episode to end the noise decay process&#xA;EPS_FINAL = 0           # final value for epsilon after decay&#xA;&#xA;device = torch.device(&amp;quot;cuda:0&amp;quot; if torch.cuda.is_available() else &amp;quot;cpu&amp;quot;)&#xA;&#xA;#Critic Network (w/ Target Network)(every agent share common critic)&#xA;**critic_local = Critic(24, 2, 0).to(device)&#xA;critic_target = Critic(24, 2, 0).to(device)&#xA;critic_optimizer = optim.Adam(critic_local.parameters(), lr=LR_CRITIC, &#xA;                       weight_decay=WEIGHT_DECAY)**&#xA;&#xA;class Agent():&#xA;&amp;quot;&amp;quot;&amp;quot;Interacts with and learns from the environment.&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA;    def __init__(self, state_size, action_size, num_agents, random_seed):&#xA;    &amp;quot;&amp;quot;&amp;quot;Initialize an Agent object.&#xA;&#xA;    Params&#xA;    ======&#xA;        state_size (int): dimension of each state&#xA;        action_size (int): dimension of each action&#xA;        num_agents (int): number of agents&#xA;        random_seed (int): random seed&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    self.state_size = state_size&#xA;    self.action_size = action_size&#xA;    self.num_agents = num_agents&#xA;    self.seed = random.seed(random_seed)&#xA;    self.eps = EPS_START&#xA;    self.eps_decay = 1/(EPS_EP_END*LEARN_NUM)  # set decay rate based on epsilon end target&#xA;    self.timestep = 0&#xA;&#xA;    # Actor Network (w/ Target Network)&#xA;    self.actor_local = Actor(state_size, action_size, random_seed).to(device)&#xA;    self.actor_target = Actor(state_size, action_size, random_seed).to(device)&#xA;    self.actor_optimizer = optim.Adam(self.actor_local.parameters(), lr=LR_ACTOR)&#xA;&#xA;    # Noise process&#xA;    self.noise = OUNoise((num_agents, action_size), random_seed)&#xA;&#xA;    # Replay memory&#xA;    self.memory = ReplayBuffer(action_size, BUFFER_SIZE, BATCH_SIZE, random_seed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38540" LastActivityDate="2020-08-22T05:32:13.460" Title="Are my agents sharing common critic in the following Multiagent DDPG implementation?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23186" PostTypeId="2" ParentId="23184" CreationDate="2020-08-22T07:07:26.083" Score="1" Body="&lt;p&gt;This sounds like a problem that's best solved with a simple non-AI algorithm. If you just enumerate the coordinates in a regular order (rows, colums, zig-zag, hilbert curve) and emit the coordinates where the image has a '1' you're meeting your requirements. Is there any specific reason you want to use an AI algorithm which is most likely worse than this?&lt;/p&gt;&#xA;" OwnerUserId="22993" LastEditorUserId="22993" LastEditDate="2020-08-22T09:41:34.157" LastActivityDate="2020-08-22T09:41:34.157" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23187" PostTypeId="1" CreationDate="2020-08-22T09:58:16.660" Score="2" ViewCount="73" Body="&lt;p&gt;In Goodfellow et al. book Deep Learning &lt;a href=&quot;https://www.deeplearningbook.org/contents/applications.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 12.1.4&lt;/a&gt; they write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;These large models learn some function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;, but do so using many more parameters than are necessary for the task. Their size is necessary only due to the limited number of training examples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not able to understand this. Large models are expressive, but if you train them on few examples they should also overfit.&lt;/p&gt;&#xA;&lt;p&gt;So, what do the authors mean by saying large models are necessary precisely because of the limited number of training examples?&lt;/p&gt;&#xA;&lt;p&gt;This seems to go against the spirit of using more bias when training data is limited.&lt;/p&gt;&#xA;" OwnerUserId="21511" LastEditorUserId="21511" LastEditDate="2020-08-23T16:12:13.903" LastActivityDate="2021-04-22T20:02:55.283" Title="Why are large models necessary when we have a limited number of training examples?" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;&lt;bias-variance-tradeoff&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23188" PostTypeId="1" CreationDate="2020-08-22T10:02:41.873" Score="0" ViewCount="12" Body="&lt;p&gt;In the experiments section of the &lt;a href=&quot;https://arxiv.org/abs/1707.01495&quot; rel=&quot;nofollow noreferrer&quot;&gt;HER&lt;/a&gt; paper, the authors mention that they added the square depth penetration term to the reward so as to penalize such behavior.&#xA;Where do you get this quantity in the gym?&#xA;&lt;br&gt;&#xA;I've gone through the source code of the Fetch environments from OpenAI's Gym github page, but still can't find what I'm looking for. Is there some way to calculate it using some other variables?&lt;/p&gt;&#xA;" OwnerUserId="30632" LastActivityDate="2020-08-22T10:02:41.873" Title="How to obtain the penetration term for the Fetch Environments in the OpenAI gym?" Tags="&lt;reinforcement-learning&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23190" PostTypeId="1" CreationDate="2020-08-22T12:52:11.850" Score="2" ViewCount="27" Body="&lt;p&gt;I was going through the Sutton book and they said the update formula for Q learning comes from the weighted average of the returns&#xA;I.e&lt;/p&gt;&#xA;&lt;p&gt;New estimate= old estimate +alpha*[returns- old estimate]&lt;/p&gt;&#xA;&lt;p&gt;So by the law of large numbers this will converge to the optimal true q value&lt;/p&gt;&#xA;&lt;p&gt;Now when we go to Deep Q networks,how exactly is the weighted average computed, all they simply did was try to reduce the error between the target and the estimate, and keep in mind this isn’t the true target, it’s just an unbiased estimate,since it’s an unbiased estimate how is the weighted average computed , which is the expectation?&lt;/p&gt;&#xA;&lt;p&gt;Can someone help me out here??&#xA;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="40049" LastActivityDate="2020-08-22T12:58:57.710" Title="How is weighted average computed in Deep Q networks" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23191" PostTypeId="2" ParentId="23190" CreationDate="2020-08-22T12:58:57.710" Score="2" Body="&lt;p&gt;Let's say &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is the old estimate, &lt;span class=&quot;math-container&quot;&gt;$Q'$&lt;/span&gt; the new estimate, and &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the return.&lt;/p&gt;&#xA;&lt;p&gt;We have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;Q' = Q + \alpha(R-Q)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can be rewritten as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;Q' = (1-\alpha)Q + \alpha R&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is a constant, this is an exponential weighted average of returns. If &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of samples we get and &lt;span class=&quot;math-container&quot;&gt;$\alpha=1/n$&lt;/span&gt; (so it decreases with each sample), we get&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q' = \frac{n-1}{n}Q + \frac{1}{n}R&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This simply represents the average return. So, playing with &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; tunes the weighting of the estimate.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-08-22T12:58:57.710" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="23192" PostTypeId="1" CreationDate="2020-08-22T14:03:38.923" Score="0" ViewCount="33" Body="&lt;p&gt;I want to create a large 2D gridworld environment(10000x10000) with multiple agents. Most of environments lacked documentation or were small or camera can't move around or select agents.&lt;/p&gt;&#xA;&lt;p&gt;How do I approach this without using game engines or with minimal training?&lt;/p&gt;&#xA;&lt;p&gt;I think I can build a numpy array but I only can render it using matplotlib. how can render it so I can move around or select agents?&lt;/p&gt;&#xA;" OwnerUserId="40495" LastActivityDate="2020-08-22T14:03:38.923" Title="How to create a gridworld environment?" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23193" PostTypeId="1" CreationDate="2020-08-22T16:24:43.103" Score="0" ViewCount="50" Body="&lt;p&gt;What is the error function? Is it the same as the cost function?&lt;/p&gt;&#xA;&lt;p&gt;Is the error function known or unknown?&lt;/p&gt;&#xA;&lt;p&gt;When I get the outcome of a neural net I compare it with the target value. The difference between both is called the error. When I get mutiple error values e.g. when I pass a batch through the NN I will get as many error value as the size of my batch. Is the error function the plot of the points? If yes, to me the error function would be unknown. I would only know some point on the graph of the error function.&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-08-22T17:14:38.423" LastActivityDate="2020-08-23T13:30:53.610" Title="Is the error function known or unknown?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23194" PostTypeId="2" ParentId="23193" CreationDate="2020-08-22T17:26:39.680" Score="2" Body="&lt;p&gt;In deep learning, the &lt;strong&gt;error function&lt;/strong&gt; is sometimes known as &lt;strong&gt;loss function&lt;/strong&gt; or &lt;strong&gt;cost function&lt;/strong&gt; (but I do not exclude the possibility that these terms/expressions have also been used to refer to different although related functions, so you should take into account your context).&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.356&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical learning theory&lt;/a&gt;, the function that you want to minimize is known as &lt;strong&gt;expected risk&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R(\theta)&#xA;&amp;amp;=&#xA;\int L(y, f_{\theta}(x)) d P(x, y) \\&#xA;&amp;amp;=&#xA;\mathbb{E}_{P(x, y)} \left[ L(y, f_{\theta}(x)) \right],&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the parameters of your model &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, and the function that you actually minimize is the &lt;strong&gt;empirical risk&lt;/strong&gt; given the dataset &lt;span class=&quot;math-container&quot;&gt;$D = \{ (x_i, y_i) \}_{i=1}^n$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_{\mathrm{emp}}(\theta)=\frac{1}{n} \sum_{i=1}^{n}L(y_i, f_{\theta}(x_i)),&#xA;$$&lt;/span&gt;&#xA;which is a &lt;em&gt;generalization&lt;/em&gt; of the commonly used cost functions, such as &lt;em&gt;mean squared error (MSE)&lt;/em&gt; or &lt;em&gt;binary cross-entropy&lt;/em&gt;. For example, in the case of MSE, &lt;span class=&quot;math-container&quot;&gt;$L = \left(y_{i}-f_\theta(x_i)\right)^{2}$&lt;/span&gt;, which can be called the &lt;em&gt;loss function&lt;/em&gt; (though this terminology may not be standard).&lt;/p&gt;&#xA;&lt;p&gt;You optimize the empirical risk, a &lt;a href=&quot;https://ai.stackexchange.com/a/23020/2444&quot;&gt;proxy&lt;/a&gt; for the expected risk, because the expected risk is &lt;em&gt;incomputable&lt;/em&gt; (given that &lt;span class=&quot;math-container&quot;&gt;$P(x, y)$&lt;/span&gt; is generally &lt;em&gt;unknown&lt;/em&gt;), so, in this sense, the expected risk is unknown.&lt;/p&gt;&#xA;&lt;p&gt;If you use the term &lt;em&gt;error function&lt;/em&gt; to refer to the expected risk, then, yes, the error function is typically unknown, but &lt;em&gt;error function&lt;/em&gt; is typically used to refer to an instance of the empirical risk, so, in that case, it is known and computable.&lt;/p&gt;&#xA;&lt;p&gt;Note that I purposely used the term &lt;em&gt;loss function above&lt;/em&gt; to refer to &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and the term &lt;em&gt;cost function&lt;/em&gt; to refer to the &lt;em&gt;empirical risks&lt;/em&gt;, such as the MSE (i.e., in this case, I did not use &lt;em&gt;loss function&lt;/em&gt; and &lt;em&gt;cost function&lt;/em&gt; as synonyms), which shows that terminology is not always used consistently, so take into account your context.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-23T13:30:53.610" LastActivityDate="2020-08-23T13:30:53.610" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23195" PostTypeId="1" CreationDate="2020-08-22T22:05:01.343" Score="0" ViewCount="14" Body="&lt;p&gt;I trained a GRU model on some data and then created a bunch of predictions on a test set.&lt;/p&gt;&#xA;&lt;p&gt;The predictions are really bad, as indicated by a near zero R2 score.&lt;/p&gt;&#xA;&lt;p&gt;I notice that the variance of the model predictions are much smaller than the actual training data. i.e it seems like the model is overfit to the mean:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/k96Gs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k96Gs.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But why is this? I made sure to stop training/use hyperparameters where there was no model overfitting, so why are model predictions centred around the mean and less dispersed than the actual variance of the data set?&lt;/p&gt;&#xA;&lt;p&gt;My model, if it is relevant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;model.add(GRU(100, activation='relu', input_shape=(3, 280), recurrent_dropout = 0.2, dropout = 0.2))&#xA;model.add(Dense(1))&#xA;model.compile(optimizer='adam', loss='mse', metrics=['mae'])&#xA;model.summary()&#xA;&#xA;# fit model&#xA;history = model.fit(X_train, y_train, epochs=40, verbose=1, validation_split=0.33)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40503" LastActivityDate="2020-08-22T22:05:01.343" Title="Why is the variance of my model predictions much smaller than the training data?" Tags="&lt;keras&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23196" PostTypeId="1" CreationDate="2020-08-22T23:36:52.027" Score="2" ViewCount="34" Body="&lt;p&gt;I have to deal with a non-episodic task, where there is addittionally a continuous state space and more specifically in each time step there is always a new state that has never been seen before. I want to use DQN algorithm. As it is referred in Sutton's book (Chapter 10), the average reward setting, that is the undiscounted setting with differential function, should be preferred for non-episodic tasks with function approximation.&lt;/p&gt;&#xA;&lt;p&gt;(a) Are there any reference papers that use DQN with the average reward setting?&lt;/p&gt;&#xA;&lt;p&gt;(b) Why should the classic discounted setting (with no average reward) fail in such tasks, comparing to the average reward setting, taking into account that the highest reward that my agent can gain in a time step is 1.0 and thus the max &lt;span class=&quot;math-container&quot;&gt;$G_t = \frac{1}{1-γ}$&lt;/span&gt; and not infinite ?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-08-23T00:08:55.847" LastActivityDate="2020-08-23T00:08:55.847" Title="Combine DQN with the Average Reward setting" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;reference-request&gt;&lt;discount-factor&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23197" PostTypeId="2" ParentId="23187" CreationDate="2020-08-23T03:40:04.047" Score="1" Body="&lt;p&gt;If you read the relevant section. it also says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Model compression is applicable when the size of the original model is driven&#xA;primarily by a need to prevent overfitting. In most cases, the model with the&#xA;lowest generalization error is an ensemble of several independently trained models.&#xA;Evaluating all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; ensemble members is expensive. Sometimes, even a single model&#xA;generalizes better if it is large (for example, if it is regularized with dropout).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The keyword (I think) here is dropout. Dropout Learning in the referred book has been intepreted as training an ensemble of models, with a model probability same as the probability of a particular dropout architecture of the large Neural Network. Thus, this effectively makes the training as training multiple smaller Neural Nets. According to this &lt;a href=&quot;https://arxiv.org/pdf/1207.0580.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on dropout, by the original authors, dropout prevents co-adaptation which effectively means you are just training an ensemble of Neural Nets. But this intuition lacks any theoretical justification.&lt;/p&gt;&#xA;&lt;p&gt;Another &lt;a href=&quot;https://arxiv.org/pdf/1602.04484.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; (understanding the paper might require familiarity with certain statistical ideas of ML) claims this is not true, and dropout doesn't reduce co-adaptation but more likely reduces the variance over dropout patterns. They have provided better empirical and theoretical justifications to this end. So it is still up for debate what actually happens.&lt;/p&gt;&#xA;&lt;p&gt;But in general the generalization error upper bound &lt;strong&gt;very roughly&lt;/strong&gt; is directly proportional to the size of the Neural Nets. So yes the authors statement in face value is oversimplified and most likely wrong in the general case.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-08-23T03:40:04.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23198" PostTypeId="2" ParentId="23172" CreationDate="2020-08-23T04:06:52.457" Score="0" Body="&lt;p&gt;I would use the ImageDataGenerator.flow_from_directory. Documentation is [here.][1]Make a train directory. Within it create 5 subdirectories one for each class and give them the desired class name. Place the associated images into the 5 class sub directories. Then use something like the code below. Set the image size to something standard particularly is you are using transfer learning. For MobileNet standard size is 224 X 224. You can use the Image Data Generator to augment your data for example set horizontal flip =True. You can also vary the brightness and rotation of the images.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tf.keras.preprocessing.image.ImageDataGenerator(&#xA;    featurewise_center=False,&#xA;    samplewise_center=False,&#xA;    featurewise_std_normalization=False,&#xA;    samplewise_std_normalization=False,&#xA;    zca_whitening=False,&#xA;    zca_epsilon=1e-06,&#xA;    rotation_range=0,&#xA;    width_shift_range=0.0,&#xA;    height_shift_range=0.0,&#xA;    brightness_range=None,&#xA;    shear_range=0.0,&#xA;    zoom_range=0.0,&#xA;    channel_shift_range=0.0,&#xA;    fill_mode=&amp;quot;nearest&amp;quot;,&#xA;    cval=0.0,&#xA;    horizontal_flip=True,&#xA;    vertical_flip=False,&#xA;    rescale=None,&#xA;    preprocessing_function=None,&#xA;    data_format=None,&#xA;    validation_split=0.0,&#xA;    dtype=None)&#xA;#for flow from directory&#xA;.flow_from_directory(&#xA;    directory,&#xA;    target_size=(256, 256),&#xA;    color_mode=&amp;quot;rgb&amp;quot;,&#xA;    classes=None,&#xA;    class_mode=&amp;quot;categorical&amp;quot;,&#xA;    batch_size=32,&#xA;    shuffle=True,&#xA;    seed=None,&#xA;    save_to_dir=None,&#xA;    save_prefix=&amp;quot;&amp;quot;,&#xA;    save_format=&amp;quot;png&amp;quot;,&#xA;    follow_links=False,&#xA;    subset=None,&#xA;    interpolation=&amp;quot;nearest&amp;quot;,&#xA;).&#xA;&#xA;&#xA;  [1]: https://keras.io/api/preprocessing/image/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="33976" LastActivityDate="2020-08-23T04:06:52.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23200" PostTypeId="2" ParentId="23180" CreationDate="2020-08-23T06:02:45.270" Score="3" Body="&lt;p&gt;The Ornstein Ulhenebck Process is defined as (in the continuous setting) :&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$dX_t = -\beta(X_t - \alpha)dt + \sigma dW_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The analogue for this process in the discrete time case which I assume will be applicable in the RL case will be:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X_{t+1} = X_t -\beta(X_t - \alpha) + \sigma \{W_{t+1}-W_t\}=$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X_{t+1} = (1 -\beta)X_t - \alpha + \sigma \{W_{t+1}-W_t\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the RL seting the terms in the equation probably means:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt; will stand for a state in RL i.e. the state is the number &lt;span class=&quot;math-container&quot;&gt;$\in \mathbb R$&lt;/span&gt; where the particle moves to at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; are just constants which decide certain movement characteristics of the particle. Check &lt;a href=&quot;http://web.math.ku.dk/%7Esusanne/StatDiff/Overheads1b&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for graphs plotted for various &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W_t$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Wiener_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weiner process&lt;/a&gt; which starts at &lt;span class=&quot;math-container&quot;&gt;$W_0 = 0$&lt;/span&gt; and then adds &lt;strong&gt;independent&lt;/strong&gt; increments of &lt;span class=&quot;math-container&quot;&gt;$\mathcal N(\mu,\sigma)$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$W_{t+1} = W_t+\mathcal N(\mu,\sigma)$&lt;/span&gt; which is basically a radom walk. More generally we use &lt;span class=&quot;math-container&quot;&gt;$\mathcal N(0,1)$&lt;/span&gt;. This is formulated as &lt;span class=&quot;math-container&quot;&gt;$W_t-W_s = \sqrt{t-s} \mathcal N(0,1)$&lt;/span&gt;. This is because of the fact, &lt;span class=&quot;math-container&quot;&gt;$W_t$&lt;/span&gt; can be written recursively as &lt;span class=&quot;math-container&quot;&gt;$W_t = \mathcal N(0,1)+W_{t-1} = \mathcal N(0,1) + \mathcal N(0,1) + ...W_s$&lt;/span&gt; and since the samplings are independent at each step the &lt;strong&gt;mean&lt;/strong&gt; get added as &lt;span class=&quot;math-container&quot;&gt;$\mu_t+\mu_{t-1}...$&lt;/span&gt; and the &lt;strong&gt;variances&lt;/strong&gt; as &lt;span class=&quot;math-container&quot;&gt;$\sigma_t^2 + \sigma_{t-1}^2...$&lt;/span&gt;. SInce her the means and variances are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; respectively, the final mean &lt;span class=&quot;math-container&quot;&gt;$\mu = 0$&lt;/span&gt; and variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2 = (t-s)$&lt;/span&gt;. And hence, byproperties of Gaussian random variables you can write (it is easy to show this via variable tranformation) &lt;span class=&quot;math-container&quot;&gt;$W_t-W_s = \sqrt{t-s} \mathcal N(0,1)$&lt;/span&gt;.  &lt;a href=&quot;https://sites.me.ucsb.edu/%7Emoehlis/APC591/tutorials/tutorial7/node2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;, is the formulation of standard Weiner process.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; will be the weighting factor of the Weiner process, which just  means the amount of noise being added to the process.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Another useful &lt;a href=&quot;https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=3214&amp;amp;context=etd&quot; rel=&quot;nofollow noreferrer&quot;&gt;resource&lt;/a&gt; on discrete Ornstein Ulhenbeck process, much less generalized.&#xA;I think now you can extend this to whatever scenario you are intereted in RL setting.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-08-23T06:08:52.340" LastActivityDate="2020-08-23T06:08:52.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23202" PostTypeId="1" CreationDate="2020-08-23T10:37:41.883" Score="1" ViewCount="123" Body="&lt;p&gt;In normal Q-learning, the update rule is an implementation of the exponential moving average, which then converges to the optimal true Q values. However, looking at DQN, how exactly is the exponential moving average implemented in deep networks?&lt;/p&gt;&#xA;" OwnerUserId="40049" LastEditorUserId="2444" LastEditDate="2020-08-23T11:11:24.047" LastActivityDate="2021-05-20T14:05:30.613" Title="How is exponential moving average computed in deep Q networks?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23204" PostTypeId="2" ParentId="23202" CreationDate="2020-08-23T11:39:49.253" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;However, looking at DQN, how exactly is the exponential moving average implemented in deep networks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is not implemented directly as exponential moving average.&lt;/p&gt;&#xA;&lt;p&gt;Instead, the ability of neural networks to learn online and incrementally forget older input/output associations is used to achieve the same goal.&lt;/p&gt;&#xA;&lt;p&gt;If you use the simplest mini-batch stochastic gradient descent methods - i.e. just a simple gradient step &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} \leftarrow \mathbf{w} - \alpha \nabla_{\mathbf{w}}\sum_i(g_i - \hat{q}_i)^2$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$g_i$&lt;/span&gt; is measured (or bootstrap estimated) discounted return for a single state/action pair and &lt;span class=&quot;math-container&quot;&gt;$\hat{q}_i$&lt;/span&gt; is the current estimate, then the learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is analagous to the same factor in exponential moving average approach, and in fact would be the same thing mathematically if you one-hot-encoded the states and only had a single layer in the neural network.&lt;/p&gt;&#xA;&lt;p&gt;Typical implementations of DQN will have deeper networks, will not one-hot-encode the entire state space, and will typically use some gradient modifier such as momentum or Adam to improve performance. So the match to exponential moving average is not exact. But the behaviour is similar in the most important aspect for RL - the ability to learn online and forget older values as the target distribution of expected returns changes due to changes in policy.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-08-23T11:39:49.253" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23205" PostTypeId="1" CreationDate="2020-08-23T11:42:10.700" Score="0" ViewCount="11" Body="&lt;p&gt;I want to know which kind of machine learning algorithms or techniques shall I use to solve such a problem&lt;/p&gt;&#xA;&lt;p&gt;for example, I have this text below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Hello, how are you doing welcome to a new day. You should make diet to have good health. This was the health topic today see you tomorrow.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;also for another example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;This is john doe and welcome to a new day. You should eat vegtables to be more healthy everyday. health topic ended see you tomorrow. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so, there is a certain topic or sentence inside the bigger discussion with a certain pattern for beginning and ending I want to be able to identify the start and end of that text talking about health for example.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;making a training set that says if there is health advice and given the beginning of the health talk and the end of it. ex: &lt;code&gt;beginning: &amp;quot;You should&amp;quot;, ending: &amp;quot;health topic ended&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Expected output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is john doe and welcome to a new day. &lt;strong&gt;You should eat vegetables to be more healthy every day. health topic ended&lt;/strong&gt; see you tomorrow.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;start_word: You&#xA;end_word: ended&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so, basically I just need to output the beginning and end of the sentence or just the sentence it would be even better.&lt;/p&gt;&#xA;&lt;p&gt;I know it needs like classification and maybe NLP but, I don't know how to do that would you please advise on how to solve such a problem with what algorithms in python.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="40513" LastActivityDate="2020-08-23T11:42:10.700" Title="How to identify a certain part of a text based on some criteria given the beginning and ending info as training data?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;classification&gt;&lt;text-classification&gt;&lt;multi-label-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23206" PostTypeId="2" ParentId="67" CreationDate="2020-08-23T12:05:32.803" Score="3" Body="&lt;p&gt;One actual example of SAT solvers is finding the set of compatible package versions in python conda package manager.&lt;/p&gt;&#xA;&lt;p&gt;(see, for example, &lt;a href=&quot;https://www.anaconda.com/blog/understanding-and-improving-condas-performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.anaconda.com/blog/understanding-and-improving-condas-performance&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;The practical applications of SAT solvers involve as well (see &lt;a href=&quot;http://www.carstensinz.de/talks/RISC-2005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.carstensinz.de/talks/RISC-2005.pdf&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Product Configuration&lt;/li&gt;&#xA;&lt;li&gt;Hardware Verification&lt;/li&gt;&#xA;&lt;li&gt;Bounded Model Checking&lt;/li&gt;&#xA;&lt;li&gt;(Hardware) Equivalence Checking&lt;/li&gt;&#xA;&lt;li&gt;Asynchronous circuit synthesis (IBM)&lt;/li&gt;&#xA;&lt;li&gt;Software-Verification&lt;/li&gt;&#xA;&lt;li&gt;Expert system verification&lt;/li&gt;&#xA;&lt;li&gt;Planning (air-traffic control, telegraph routing)&lt;/li&gt;&#xA;&lt;li&gt;Scheduling (sport tournaments)&lt;/li&gt;&#xA;&lt;li&gt;Finite mathematics (quasigroups)&lt;/li&gt;&#xA;&lt;li&gt;Cryptanalysis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40514" LastActivityDate="2020-08-23T12:05:32.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23207" PostTypeId="1" CreationDate="2020-08-23T13:07:01.373" Score="0" ViewCount="75" Body="&lt;p&gt;The 'by the book' method of delivering final machine learning models is to include all data in the final training (including validation and test sets). To check robustness of my model I use randomly chosen population for training and validation sets with each training (no set random seed). The results on validation and then test sets are pretty satisfactory for my case however they are always different each time, precision spans between 0.7 and 0.9. This is due to fact that each time different data points fall to set with which model is trained.&lt;/p&gt;&#xA;&lt;p&gt;My question is: how do I know that final training will also generate good model and how to estimate its precision when I do not have anymore unseen data?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="22659" LastEditDate="2020-08-24T10:45:49.143" LastActivityDate="2020-10-24T04:02:10.333" Title="How can I be sure that the final model, trained on all data, is correct?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;models&gt;&lt;testing&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23208" PostTypeId="1" CreationDate="2020-08-23T13:13:59.473" Score="0" ViewCount="115" Body="&lt;p&gt;Recently, I started working on time-series models and would mention that I am very new to python and ML as a whole.&lt;/p&gt;&#xA;&lt;p&gt;I tried to implement a time-series model on wind speed data. Being a newbie, I followed the steps given in this article: &lt;a href=&quot;https://kanoki.org/2020/04/30/time-series-analysis-and-forecasting-with-arima-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://kanoki.org/2020/04/30/time-series-analysis-and-forecasting-with-arima-python/&lt;/a&gt; and it's a great one to start with but somehow I am unable to &lt;strong&gt;forecast&lt;/strong&gt; my data (or I would say the forecasted data is constant around the 5.88 value!)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OeqBl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OeqBl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# FORECAST&#xA;n=39&#xA;forecast,err,ci = results.forecast(steps=n, alpha= 0.05)&#xA;windspeed_forecast = pd.DataFrame({'forecast':forecast},index=pd.date_range(start='22/8/2020 01:00:00', periods=n, freq='MS'))&#xA;windspeed_forecast &#xA;&#xA; #plot for forecast&#xA;ax = windspeed[19:].SPEED.plot(label='observed', figsize=(20, 15))&#xA;windspeed_forecast.plot(ax=ax,label='Forecast',color='r')&#xA;ax.fill_between(windspeed_forecast.index, ci[:,0], ci[:,1], color='b', alpha=.005)&#xA;ax.set_xlabel('DATE')&#xA;ax.set_ylabel('SPEED')&#xA;&#xA;plt.legend()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;-What I think is the high AIC value of 700 might be the problem!&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bzRbl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bzRbl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;-Also, I am unable to figure out how can I create the column of Date-time for the forecasted values same as that of the original data(i.e. hourly based data of a specific date) [As shown in the ss number 1 and as shown in ss below - I need a column starting from 22/8/2020 with hourly gaps and so on].&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OE0A0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OE0A0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, PFA the ss of my data in jupyter notebook (out of 191 total data, 152 used as train data and rest as test data)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/U8xIU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U8xIU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UoU1o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UoU1o.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Any suggestion/help regarding the same will be appreciated :)&lt;/p&gt;&#xA;" OwnerUserId="40515" LastActivityDate="2021-05-20T18:05:15.367" Title="Wind speed forecasting using ARIMA model in Python3" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;regression&gt;&lt;time-series&gt;&lt;forecasting&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23209" PostTypeId="2" ParentId="7755" CreationDate="2020-08-23T15:12:56.313" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Change the action space at each step, depending on the internal_state. I assume this is nonsense. &lt;br&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, this seems overkill and makes the problem unnecessarily complex, there could be other things you can do.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do nothing : let the model understand that choosing an unavailable action has no impact.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;While this will not harm your model negatively, in any way the negative thing about this is that the model could take a long time to understand that some actions don't matter for some values of state.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do -almost- nothing : impact slightly negatively the reward when the model chooses an unavailable action.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Same response as for the previous point except that this could harm your model negatively (but not sure if this is significant). Assume you give a reward of -100 for every illegal action it takes. Looking at only the negative rewards, we have: &lt;br&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;-100 when initial_state == 0&lt;/li&gt;&#xA;&lt;li&gt;-200 when initial state == 1&#xA;By doing this, you might be implicitly favoring situations where state == 0. Plus, I dont see the point of the -100 reward anyway since once they come to state 0 they will have to choose a value for the illegal actions too (it's not like they can ignore the values for the action and escape -100 reward)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Help the model : by incorporating an integer into the state/observation space that informs the model what's the internal_state value + bullet point 2 or 3&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To me, this seems like the most ideal thing to do. You could branch out the final output layer (of your ActorCritic Agent) into 2 instead of 1.&lt;/p&gt;&#xA;&lt;p&gt;Like: Input layer, fc1 (from input layer), output1(from fc1), output2(from fc1)&#xA;Based on initial_state you can get the output from output1 or output2.&lt;/p&gt;&#xA;" OwnerUserId="40518" LastActivityDate="2020-08-23T15:12:56.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23210" PostTypeId="2" ParentId="23208" CreationDate="2020-08-23T15:39:50.813" Score="0" Body="&lt;p&gt;First things first, time series is probably the most difficult of all things to predict. If it wasn't I'd be sitting on nice warm beach or on yacht cruising the Mediterranean.&lt;/p&gt;&#xA;&lt;p&gt;Besides that here's a few hints. First, always visualise your data first. Tables of numbers are fine for computers by the MK1 eyeball is still your best weapon for time series. Is there any obvious trends or seasonal behaviour? If not then ARIMA is unlikely to be of much utility. Secondly, visualise your data after forecasting. Don't just rely on error measures like AIC, use your eyes. Most time series forecasts are useful only in the short term. So you may see the earlier part of your forecast make sense while the latter doesn't.&lt;/p&gt;&#xA;&lt;p&gt;Finally ARIMA is paramterised by by p, d, and q. The value of d is the minimum number of differencing needed to make the series stationary. If the time series is already stationary, then d = 0. ‘p’ is the order of the ‘Auto Regressive’ (AR) term. It refers to the number of lags of Y to be used as predictors. And ‘q’ is the order of the ‘Moving Average’ (MA) term. It refers to the number of lagged forecast errors that should go into the ARIMA Model.&lt;/p&gt;&#xA;&lt;p&gt;You have only shown results for p,d,q = 1,0,1. Much of ML is done by trial and error, trying different parameters. So I'd suggest you try varying your values of p, d and q. AIC can then be used to compare models, with smaller values indicating better models.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastActivityDate="2020-08-23T15:39:50.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23211" PostTypeId="2" ParentId="23065" CreationDate="2020-08-23T16:56:10.623" Score="1" Body="&lt;p&gt;You need to express the product of the activation functions of your neurons via some combination of individual activation functions.&lt;/p&gt;&#xA;&lt;p&gt;For example, if the product of your activation functions is equal to the linear combination of such functions then the network for &lt;span class=&quot;math-container&quot;&gt;$f(x)g(x)$&lt;/span&gt; can be derived from the networks for &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; exactly.&lt;/p&gt;&#xA;&lt;p&gt;Let's assume for simplicity that &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; are approximated with trained networks that consist of just one neuron,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = A(w_f x + b_f)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$g(x) = A(w_g x + b_g)$$&lt;/span&gt;&#xA;Then, if the activation function &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$A(w_f x + b_f)A(w_g x + b_g) = \sum\limits_{i=1}^N \alpha_iA(w_i x + b_i)$$&lt;/span&gt;&#xA;then &lt;span class=&quot;math-container&quot;&gt;$f(x)g(x)$&lt;/span&gt; is approximated by a neural network consisting of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; neurons:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x)g(x) = \sum\limits_{i=1}^N \alpha_iA(w_i x + b_i)$$&lt;/span&gt;&#xA;This is easily generalized to a network that consists of multiple neurons.&lt;/p&gt;&#xA;&lt;p&gt;An example is when some of your neurons have activation functions &lt;span class=&quot;math-container&quot;&gt;$A(z)=\cos(z)$&lt;/span&gt; and others are &lt;span class=&quot;math-container&quot;&gt;$A(z)=\sin(z)$&lt;/span&gt;. Then you can build the network computing &lt;span class=&quot;math-container&quot;&gt;$f(x)g(x)$&lt;/span&gt; from the networks computing &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; using the product-to-sum identities, like &lt;span class=&quot;math-container&quot;&gt;$2\sin(\theta)\sin(\phi) = cos(\theta-\phi)-cos(\theta+\phi)$&lt;/span&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;If the product of your activation functions cannot be expressed as a linear combination of such functions (as in the case of sigmoids) then the network for &lt;span class=&quot;math-container&quot;&gt;$f(x)g(x)$&lt;/span&gt; can be derived from the networks for &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; approximately. You need to figure out the parameters of the approximation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$A(z_f)A(z_g) \approx \sum\limits_{i=1}^N \alpha_iA(z_i)$$&lt;/span&gt;&#xA;Such parameters are: the number of terms, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, the coefficients &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt;, and the range &lt;span class=&quot;math-container&quot;&gt;$[z_{min}, z_{max}]$&lt;/span&gt; where this approximation works with a desired accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Linear combination is not the only opportunity. Say, your activation is &lt;span class=&quot;math-container&quot;&gt;$A(z)=a^z$&lt;/span&gt;. Then, &lt;span class=&quot;math-container&quot;&gt;$$A(w_f x + b_f)A(w_g x + b_g)=a^{(w_f+w_g) x + (b_f+b_g)}=a^{w_{fg} x + b_{fg}}$$&lt;/span&gt; so the weights and the biases of the product are exactly expressed via the weights and the biases of the multiplicands.&lt;/p&gt;&#xA;" OwnerUserId="15524" LastEditorUserId="15524" LastEditDate="2020-08-23T17:03:06.497" LastActivityDate="2020-08-23T17:03:06.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23212" PostTypeId="1" AcceptedAnswerId="23214" CreationDate="2020-08-23T17:49:31.347" Score="3" ViewCount="109" Body="&lt;p&gt;I know that the notation &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(\mu, \sigma)$&lt;/span&gt; stands for a normal distribution.&#xA;But I'm reading the book &amp;quot;An Introduction to Variational Autoencoders&amp;quot; and in it, there is this notation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{N}(z; 0, I)$$&lt;/span&gt;&#xA;What does it mean?&lt;/p&gt;&#xA;&lt;p&gt;picture of the book:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/34JES.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/34JES.png&quot; alt=&quot;picture of the book An Introduction to Variational Autoencoders&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23811" LastActivityDate="2020-08-23T19:19:47.283" Title="What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?" Tags="&lt;terminology&gt;&lt;variational-autoencoder&gt;&lt;notation&gt;&lt;random-variable&gt;&lt;bayesian-statistics&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23213" PostTypeId="1" AcceptedAnswerId="23215" CreationDate="2020-08-23T18:27:38.103" Score="1" ViewCount="77" Body="&lt;p&gt;TensorFlow allows users to save the weights and the model architecture, however, that will be insufficient unless the values of certain other variables are also stored. For instance, in DQN, if &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is not stored the model will start exploring from scratch and a new model will have to be trained.&lt;/p&gt;&#xA;&lt;p&gt;What are the variables that need to be saved and loaded, so that a DQN model starts where it left off? Some pseudocode will be highly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Here is my current model with code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Slightly modified from the following repository - https://github.com/gsurma/cartpole&#xA;&#xA;from __future__ import absolute_import, division, print_function, unicode_literals&#xA;&#xA;import os&#xA;import random&#xA;import gym&#xA;import numpy as np&#xA;import tensorflow as tf&#xA;&#xA;from collections import deque&#xA;from tensorflow.models import Sequential&#xA;from tensorflow.layers import Dense&#xA;from tensorflow.optimizers import Adam&#xA;&#xA;&#xA;ENV_NAME = &amp;quot;CartPole-v1&amp;quot;&#xA;&#xA;GAMMA = 0.95&#xA;LEARNING_RATE = 0.001&#xA;&#xA;MEMORY_SIZE = 1000000&#xA;BATCH_SIZE = 20&#xA;&#xA;EXPLORATION_MAX = 1.0&#xA;EXPLORATION_MIN = 0.01&#xA;EXPLORATION_DECAY = 0.995&#xA;&#xA;checkpoint_path = &amp;quot;training_1/cp.ckpt&amp;quot;&#xA;&#xA;&#xA;class DQNSolver:&#xA;&#xA;    def __init__(self, observation_space, action_space):&#xA;        self.exploration_rate = EXPLORATION_MAX&#xA;&#xA;        self.action_space = action_space&#xA;        self.memory = deque(maxlen=MEMORY_SIZE)&#xA;&#xA;        self.model = Sequential()&#xA;        self.model.add(Dense(24, input_shape=(observation_space,), activation=&amp;quot;relu&amp;quot;))&#xA;        self.model.add(Dense(24, activation=&amp;quot;relu&amp;quot;))&#xA;        self.model.add(Dense(self.action_space, activation=&amp;quot;linear&amp;quot;))&#xA;        self.model.compile(loss=&amp;quot;mse&amp;quot;, optimizer=Adam(lr=LEARNING_RATE))&#xA;&#xA;    def remember(self, state, action, reward, next_state, done):&#xA;        self.memory.append((state, action, reward, next_state, done))&#xA;&#xA;    def act(self, state):&#xA;        if np.random.rand() &amp;lt; self.exploration_rate:&#xA;            return random.randrange(self.action_space)&#xA;        q_values = self.model.predict(state)&#xA;        return np.argmax(q_values[0])&#xA;&#xA;    def experience_replay(self):&#xA;        if len(self.memory) &amp;lt; BATCH_SIZE:&#xA;            return&#xA;        batch = random.sample(self.memory, BATCH_SIZE)&#xA;        for state, action, reward, state_next, terminal in batch:&#xA;            q_update = reward&#xA;            if not terminal:&#xA;                q_update = (reward + GAMMA * np.amax(self.model.predict(state_next)[0]))&#xA;            q_values = self.model.predict(state)&#xA;            q_values[0][action] = q_update&#xA;            self.model.fit(state, q_values, verbose=0)&#xA;        self.exploration_rate *= EXPLORATION_DECAY&#xA;        self.exploration_rate = max(EXPLORATION_MIN, self.exploration_rate)&#xA;&#xA;&#xA;def cartpole():&#xA;    env = gym.make(ENV_NAME)&#xA;    #score_logger = ScoreLogger(ENV_NAME)&#xA;    observation_space = env.observation_space.shape[0]&#xA;    action_space = env.action_space.n&#xA;    dqn_solver = DQNSolver(observation_space, action_space)&#xA;    checkpoint = tf.train.get_checkpoint_state(os.getcwd()+&amp;quot;/saved_networks&amp;quot;)&#xA;    print('checkpoint:', checkpoint)&#xA;    if checkpoint and checkpoint.model_checkpoint_path:&#xA;        dqn_solver.model = keras.models.load_model('cartpole.h5')&#xA;        dqn_solver.model = model.load_weights('cartpole_weights.h5')&#xA;        &#xA;    run = 0&#xA;    i = 0&#xA;    while i&amp;lt;5:&#xA;        i = i + 1&#xA;        #total = 0&#xA;        run += 1&#xA;        state = env.reset()&#xA;        state = np.reshape(state, [1, observation_space])&#xA;        step = 0&#xA;        while True:&#xA;            step += 1&#xA;            #env.render()&#xA;            action = dqn_solver.act(state)&#xA;            state_next, reward, terminal, info = env.step(action)&#xA;            #total += reward&#xA;            reward = reward if not terminal else -reward&#xA;            state_next = np.reshape(state_next, [1, observation_space])&#xA;            dqn_solver.remember(state, action, reward, state_next, terminal)&#xA;            state = state_next&#xA;            dqn_solver.model.save('cartpole.h5')&#xA;            dqn_solver.model.save_weights('cartpole_weights.h5')&#xA;            if terminal:&#xA;                print(&amp;quot;Run: &amp;quot; + str(run) + &amp;quot;, exploration: &amp;quot; + str(dqn_solver.exploration_rate) + &amp;quot;, score: &amp;quot; + str(step))&#xA;                #score_logger.add_score(step, run)&#xA;                break&#xA;            dqn_solver.experience_replay()&#xA;&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    cartpole()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2020-08-23T21:20:51.973" LastActivityDate="2020-09-01T11:44:24.047" Title="What are the variables that need to be saved and loaded, so that a DQN model starts where it left off?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;implementation&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23214" PostTypeId="2" ParentId="23212" CreationDate="2020-08-23T19:19:47.283" Score="5" Body="&lt;p&gt;It means that &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; has a (multivariate) normal distribution with 0 mean and identity covariance matrix. This essentially means each individual element of the vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; has a standard normal distribution.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-08-23T19:19:47.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23215" PostTypeId="2" ParentId="23213" CreationDate="2020-08-23T21:49:21.257" Score="2" Body="&lt;p&gt;Typically you would need to save the network weights, hyper-parameters and the replay buffer if you wanted to stop training and then come back at a later date and carry on training. Usually, I do this by writing it all as a class in Python (the agent, the memory buffer, hyper-parameters etc.) and saving the final object with Pickle.&lt;/p&gt;&#xA;&lt;p&gt;Looking at your code, the only thing I would personally have done different would be to define the model outside of the class and have the class take as input a network; however I usually use PyTorch as opposed to Keras/Tensorflow so I'm not sure which method works better.&lt;/p&gt;&#xA;&lt;p&gt;As per OP's request in the comments, here is a snippet of code I used for Car-pool.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class DQN:&#xA;&#xA;def __init__(self, observation_space, action_space):&#xA;    self.exploration_rate = epsilon_max&#xA;    self.observation_space = observation_space&#xA;    self.batch_size = batch_size&#xA;    self.gamma = gamma&#xA;&#xA;    self.action_space = action_space&#xA;&#xA;    self.memory = deque(maxlen=max_memory)&#xA;&#xA;    self.model = Net()&#xA;    self.loss_fn = nn.MSELoss()&#xA;    self.optimizer = optim.Adam(self.model.parameters(), lr=0.0001)&#xA;&#xA;def remember(self, state, action, reward, next_state, done):&#xA;    self.memory.append((state, action, reward, next_state, done))&#xA;&#xA;def act(self, state):&#xA;    self.model.eval()&#xA;    state = torch.from_numpy(state).type(torch.FloatTensor)&#xA;    self.exploration_rate *= exploration_decay&#xA;    self.exploration_rate = max(epsilon_min, self.exploration_rate)&#xA;    if np.random.uniform() &amp;lt; self.exploration_rate:&#xA;        return np.random.randint(0, self.action_space)&#xA;    q_values = self.model(state)&#xA;    action = torch.argmax(q_values, dim=1)&#xA;    return int(action)&#xA;&#xA;def experience_replay(self):&#xA;    if len(self.memory) &amp;lt; batch_size:&#xA;        return&#xA;&#xA;    self.optimizer.zero_grad()&#xA;    states,actions,rewards,next_states,dones = self.get_batch()&#xA;    states = torch.FloatTensor(states).reshape((self.batch_size,4))&#xA;    next_states = torch.FloatTensor(next_states).reshape((self.batch_size,4))&#xA;    rewards = torch.FloatTensor(rewards).reshape((self.batch_size,1))&#xA;    dones = torch.FloatTensor(dones).reshape((self.batch_size,1))&#xA;&#xA;    # get the q update ready&#xA;    # first take max and set up target&#xA;    max_vals,argmax = torch.max(self.model(next_states),axis=1)&#xA;    q_target = rewards + self.gamma * max_vals.reshape((self.batch_size,1)) * (1-dones)&#xA;&#xA;    q_values = self.model(states)&#xA;    for _ in range(self.batch_size):&#xA;        q_values[_][actions[_]] = q_target[_][0]&#xA;    input = self.model(states)&#xA;    q_values = q_values.detach()&#xA;    loss = self.loss_fn(input, q_values)&#xA;    loss.backward()&#xA;    self.optimizer.step()&#xA;&#xA;def get_batch(self):&#xA;    batch = random.sample(self.memory, self.batch_size)&#xA;&#xA;    states = []&#xA;    actions = []&#xA;    rewards = []&#xA;    next_states = []&#xA;    dones = []&#xA;    for state,action,reward,next_state,done in batch:&#xA;        states.append(state)&#xA;        actions.append(action)&#xA;        rewards.append(reward)&#xA;        next_states.append(next_state)&#xA;        dones.append(done)&#xA;    return states,actions,rewards,next_states,dones&#xA;&#xA;&#xA;def tensor_max(self, ten):&#xA;    ten = ten.numpy()&#xA;    ten = np.amax(ten, 1).reshape(1, 1)&#xA;    ten = torch.from_numpy(ten).type(torch.FloatTensor)&#xA;    return ten&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-09-01T11:44:24.047" LastActivityDate="2020-09-01T11:44:24.047" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23216" PostTypeId="1" CreationDate="2020-08-24T06:04:44.063" Score="0" ViewCount="29" Body="&lt;p&gt;I have camera feed (in the form of RGB images) from 3 cameras with overlapping FOV e.g. for the time stamp 0 to 100, I have synchronized RGB frames for each camera.&lt;/p&gt;&#xA;&lt;p&gt;The object (Robot) is moving from one place to another place. I don't know about the camera locations but for each frame &amp;amp; for each camera, I have 8-corner points of bounding-box which are just 2D projections of corresponding 3D camera points on an image. For example, in frames, depending upon the time stamp, if the object appears then I have 2D coordinates of the 8 corners of blue bounding box. Note that, in the below image, I have shown only an example object not the entire frame!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/959JA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/959JA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.esko.com/docs/en-us/artioscad/12/userguide/assets/ac/se_ac_120_3DExtendToolExample.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;image source&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Apart from this, I have an Intrinsic Matrix which is the same for all the cameras. I also know the object (Robot) dimensions (length, breadth, &amp;amp; height).&lt;/p&gt;&#xA;&lt;p&gt;So, using these RGB frames from all the cameras, 2D coordinates of 8 corner-points and object (Robot) dimensions, how can I calibrate 3 cameras as well as how can I find the poses of each camera w.r.t. the first camera in the form of a 4x4 transformation matrix [SE(3)]?&lt;/p&gt;&#xA;" OwnerUserId="40524" LastEditorUserId="40524" LastEditDate="2020-08-24T16:05:07.140" LastActivityDate="2020-08-24T16:05:07.140" Title="How can I calibrate 3 cameras without knowing global pose of the object &amp; camera locations? How can I find the pose of each camera wrt the first one?" Tags="&lt;computer-vision&gt;&lt;linear-algebra&gt;&lt;pose-estimation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23217" PostTypeId="1" CreationDate="2020-08-24T07:53:03.697" Score="1" ViewCount="52" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;A &lt;em&gt;homograph&lt;/em&gt; - is a word that shares  the same written form as another word but has a different meaning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They can be even different parts of speech. For example:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;close(verb) - close(adverb)&lt;/li&gt;&#xA;&lt;li&gt;lead(verb) - lead(noun)&lt;/li&gt;&#xA;&lt;li&gt;wind(noun) - wind(verb)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;And there is rather a big list &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_English_homographs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/List_of_English_homographs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As far as I understand, after processing the text data in any conventional way, lemmatization, building an embedding, these words, despite having different meaning, and appearing in different contexts, would be absolutely the same for the algorithm, and in the end we would get some averaged context between two or more meainings of the word. And this embedding would be meaningless.&lt;/p&gt;&#xA;&lt;p&gt;How is this problem treated or these words are regarded to be too rare to have a significant impact on the quality of resulting embedding?&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate comments and references to the papers or sources&lt;/p&gt;&#xA;" OwnerUserId="38846" LastActivityDate="2021-03-26T08:21:13.103" Title="How homographs is an NLP task can be treated?" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23219" PostTypeId="1" CreationDate="2020-08-24T12:17:04.240" Score="1" ViewCount="49" Body="&lt;p&gt;I have devised an gridworld-like environment where a RL agent is tasked to cover all the blank squares by passing through them. Possible actions are up, down, left, right. The reward scheme is the following: +1 for covering a blank cell, and -1 per step. So, if the cell was colored after a step, the summed reward is (+1) + (-1) = 0, otherwise it is (0) + (-1) = -1. The environment is a tensor whose layers encode the positions to be covered and the position of the agent.&lt;/p&gt;&#xA;&lt;p&gt;Under this reward scheme, DQN fails to find a solution (implementation: &lt;a href=&quot;https://github.com/DLR-RM/stable-baselines3&quot; rel=&quot;nofollow noreferrer&quot;&gt;stable_baselines3&lt;/a&gt;). However, when the rewards are reduced by a factor of 10 to +0.1/-0.1, then the algorithm learns an optimal path.&lt;/p&gt;&#xA;&lt;p&gt;I wonder why that happens. I have tried reducing learning rate and gradient clipping (by norm) for the first case to see whether it will improve the learning, but it does not.&lt;/p&gt;&#xA;&lt;p&gt;The activation function used is ReLU&lt;/p&gt;&#xA;" OwnerUserId="2254" LastEditorUserId="2254" LastEditDate="2020-08-24T17:58:44.233" LastActivityDate="2020-08-24T17:58:44.233" Title="Why scaling reward drastically affects performance?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="0" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23221" PostTypeId="1" AcceptedAnswerId="23683" CreationDate="2020-08-24T14:56:35.433" Score="3" ViewCount="1173" Body="&lt;p&gt;As far as I can tell, BERT is a type of Transformer architecture. What I do not understand is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How is Bert different from the original transformer architecture?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What tasks are better suited for BERT, and what tasks are better suited for the original architecture?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditDate="2020-09-21T16:48:20.787" LastActivityDate="2020-10-21T22:02:45.387" Title="How is BERT different from the original transformer architecture?" Tags="&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;transformer&gt;&lt;bert&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23222" PostTypeId="1" CreationDate="2020-08-24T19:15:00.600" Score="0" ViewCount="61" Body="&lt;p&gt;Researcher here. I just read this piece about &lt;a href=&quot;https://chooch.ai/computer-vision/medical-imaging-ai-recent-breakthroughs-with-object-recognition/&quot; rel=&quot;nofollow noreferrer&quot;&gt;medical imaging ai with object recognition&lt;/a&gt; and it left me wondering why there are still 100,000+ deaths a year in the US due to misdiagnosis - anyone out there working on these problems? Vinod Khosla famously said that he'd rather get surgery from AI than from a human - so where are we at with that?&lt;/p&gt;&#xA;" OwnerUserId="40545" LastActivityDate="2020-10-17T12:04:05.370" Title="Why isn't medical imaging improving faster with AI?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23223" PostTypeId="1" CreationDate="2020-08-24T19:33:11.853" Score="1" ViewCount="19" Body="&lt;p&gt;How can edge detection algorithms, which are not based on deep learning, such as the canny edge detector, be implemented on a GPU? For example, how are non-edge pixels removed from an image once it detects all the edges?&lt;/p&gt;&#xA;&lt;p&gt;The reason why I am asking this question is that when writing data to memory the GPU cores can't see what memory locations the other cores are writing to, so I am interested in knowing how traditional edge detectors can be implemented in on GPU.&lt;/p&gt;&#xA;" OwnerUserId="40546" LastEditorUserId="2444" LastEditDate="2020-08-24T21:01:36.810" LastActivityDate="2020-08-24T21:01:39.647" Title="How can traditional edge detection algorithms be implemented on a GPU?" Tags="&lt;computer-vision&gt;&lt;image-processing&gt;&lt;gpu&gt;&lt;edge-detection&gt;&lt;canny-edge-detector&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23224" PostTypeId="5" CreationDate="2020-08-24T20:01:29.097" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-24T20:01:29.097" LastActivityDate="2020-08-24T20:01:29.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23225" PostTypeId="4" CreationDate="2020-08-24T20:01:29.097" Score="0" Body="For questions related to edge detection algorithms in image processing and computer vision." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-24T20:01:29.097" LastActivityDate="2020-08-24T20:01:29.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23226" PostTypeId="1" AcceptedAnswerId="23227" CreationDate="2020-08-24T20:21:11.760" Score="12" ViewCount="1637" Body="&lt;p&gt;The definitions for these two appear to be very similar, and frankly, I've been only using the term &amp;quot;active learning&amp;quot; the past couple of years. What is the actual difference between the two? Is one a subset of the other?&lt;/p&gt;&#xA;" OwnerUserId="40250" LastEditorUserId="23503" LastEditDate="2020-10-30T13:01:02.370" LastActivityDate="2020-10-30T13:01:02.370" Title="What is the difference between active learning and online learning?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;online-learning&gt;&lt;active-learning&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23227" PostTypeId="2" ParentId="23226" CreationDate="2020-08-24T20:44:43.903" Score="12" Body="&lt;p&gt;&lt;strong&gt;Active learning (AL)&lt;/strong&gt; is a &lt;em&gt;weakly supervised learning&lt;/em&gt; (WSL) technique where you can have both labelled and unlabelled data [&lt;a href=&quot;https://academic.oup.com/nsr/article/5/1/44/4093912&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;]. The main idea behind AL is that the learner (or learning algorithm) can query an &amp;quot;oracle&amp;quot; (e.g. a human) to label some unlabelled instances. AL is similar to &lt;em&gt;semi-supervised learning&lt;/em&gt; (SSL), which is also a WSL technique, given that both deal with unlabelled and labeled data, but do that differently (i.e. SSL does not use an oracle).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Online_machine_learning&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Online learning&lt;/strong&gt;&lt;/a&gt; are machine learning techniques that update the models as new data is collected or arrives sequentially, as opposed to &lt;strong&gt;batch learning&lt;/strong&gt; (or &lt;em&gt;offline learning&lt;/em&gt;), where you first collect a dataset of multiple instances and then you train a model once (although you can later update it when you update your dataset). Batch learning is currently the common way of training machine learning models, given that it avoids problems like the known &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;&lt;strong&gt;catastrophic interference&lt;/strong&gt;&lt;/a&gt; (aka &lt;em&gt;catastrophic forgetting&lt;/em&gt;) problem, which can occur if you learn online. For example, neural networks are known to face this problem when learning online. There are &lt;strong&gt;incremental learning&lt;/strong&gt; (aka &lt;em&gt;lifelong learning&lt;/em&gt;) algorithms that attempt to address this &lt;em&gt;catastrophic interference&lt;/em&gt; problem.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-25T11:09:38.147" LastActivityDate="2020-08-25T11:09:38.147" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23228" PostTypeId="2" ParentId="23226" CreationDate="2020-08-24T21:03:19.223" Score="4" Body="&lt;p&gt;As it is referred in the survey paper &amp;quot;Active Learning Literature Survey&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The key idea behind &lt;strong&gt;active learning&lt;/strong&gt; is that a machine learning algorithm can&#xA;achieve greater accuracy with fewer training labels if it is allowed to choose the&#xA;data from which it learns. An active learner may pose queries, usually in the form&#xA;of unlabeled data instances to be labeled by an &lt;em&gt;oracle&lt;/em&gt; (e.g., a human annotator).&#xA;Active learning is well-motivated in many modern machine learning problems,&#xA;where unlabeled data may be abundant or easily obtained, but labels are difficult,&#xA;time-consuming, or expensive to obtain.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Online learning&lt;/strong&gt; uses data which become available in a sequential order. It's main goal is to update the best predictor for future data at each step.&lt;/p&gt;&#xA;&lt;p&gt;So, online learning is a more general method of machine learning that is opposed to &lt;strong&gt;offline learning&lt;/strong&gt;, or &lt;strong&gt;batch learning&lt;/strong&gt;, where the whole dataset has already been generated and used for training / updating the model's parameters. Moreover, a common technique for training Machine Learning models is to first perform online learning, in order to acquire an adequate data size, and then perform offline learning on the whole dataset and finaly compare the results generated by the two learning processes.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, active learning can be performed both with online learning[&lt;a href=&quot;https://www.eecs.tufts.edu/%7Edsculley/papers/activeSpam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] and offline learning, in order to reduce manual annotation effort during the annotation of training data for machine learning classifiers. That is, independently of how data have been generated and with what order, active learning should make the least queries, to an Oracle, needed for annotation of a subset of the data.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-08-24T21:12:00.230" LastActivityDate="2020-08-24T21:12:00.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23229" PostTypeId="5" CreationDate="2020-08-24T21:09:43.013" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-24T21:09:43.013" LastActivityDate="2020-08-24T21:09:43.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23230" PostTypeId="4" CreationDate="2020-08-24T21:09:43.013" Score="0" Body="For questions related to the Canny edge detector, which is a well-known edge detector developed by John F. Canny in 1986." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-24T21:09:43.013" LastActivityDate="2020-08-24T21:09:43.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23231" PostTypeId="2" ParentId="23162" CreationDate="2020-08-24T21:29:49.587" Score="0" Body="&lt;p&gt;Assuming that the dictionary of the words, that your model comes up with, is a subset of the pretrained embeddings, for example of Google's pretrained word2vec, then it is maybe a better option following these embeddings, if your model can handle that size of dimension.&lt;/p&gt;&#xA;&lt;p&gt;However, sometimes that would not always be the best solution, taking into account the nature of the problem. For example, if you are trying to use NLP on medical texts that contain rare and special words, then maybe you should use your embedding layer, assuming that you have an adequate data size, or both of them. That is just a thought of mine. For sure, there can be several other use cases which should propose the embedding layer.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-08-24T21:29:49.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23232" PostTypeId="2" ParentId="23207" CreationDate="2020-08-24T23:56:53.697" Score="1" Body="&lt;p&gt;The purpose of the test set is to test your model before deploying, otherwise, you would not need the test set in the first place. If you retrain your model by also including the validation and test datasets, of course, you cannot test your model anymore. You need to leave the test dataset separate and not use it for retraining, unless you have more data for testing.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-08-24T23:56:53.697" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23233" PostTypeId="2" ParentId="20342" CreationDate="2020-08-25T01:16:23.563" Score="0" Body="&lt;p&gt;To anyone who reads this, I still haven't solved this completely.  At the moment I'm doing a lot better with much cleaner data, using a loss metric that matches what I'm after (F1_Score), using a very deep learning model (a custom Inception Resnet V2 model), use a custom learning rate function that depends on the training round's F1 Score, and every training round computing a F1 Score for various dB of signal/noise test sets with which I compute a model wellness score with which I determine if the model is good enough.   Pretty close.&lt;/p&gt;&#xA;" OwnerUserId="36092" LastActivityDate="2020-08-25T01:16:23.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23234" PostTypeId="1" CreationDate="2020-08-25T01:32:30.693" Score="0" ViewCount="12" Body="&lt;p&gt;I'm not sure how to describe this in the most accurate way but I'll give it a shot.&lt;/p&gt;&#xA;&lt;p&gt;I've developed a Inception-Resnet V2 model for detecting audio signals via spectrogram.  It does a pretty good job but is not exactly the way I'd like it to be.&lt;/p&gt;&#xA;&lt;p&gt;Some details:  I use 5 sets of data to evaluate my model during training.  They are all similar but slightly different.  Once I get to a certain threshold of F1 Scores for each training set I stop training.  My overall threshold is pretty hard to get to.  Every time training develops a model that produces a &amp;quot;best yet&amp;quot; of one of these data sets I save the model.&lt;/p&gt;&#xA;&lt;p&gt;What I've noticed is that, during training, some round will produce a high F1 Score for one particular set while the other sets languish as mediocre.  Then, several dozen rounds later, another data set will peak while the others are mediocre.  Overall the entire model gets better but there are always some models that work better for some data sets.&lt;/p&gt;&#xA;&lt;p&gt;What I would like to know is, given I might have 5 different models that each work better for a particular subset of data, is there a way that I can combine these models (either as a whole or better yet their particular layers) to produce a single model that works the best for all my data validation subsets?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.  Mecho&lt;/p&gt;&#xA;" OwnerUserId="36092" LastActivityDate="2020-08-25T01:32:30.693" Title="How to combine specific CNN models that work better at slightly different tasks?" Tags="&lt;convolutional-neural-networks&gt;&lt;models&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23236" PostTypeId="2" ParentId="23162" CreationDate="2020-08-25T01:40:45.277" Score="1" Body="&lt;p&gt;There are multiple ways to get word embedding from a corpus.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Count Vectorizer:&lt;/strong&gt; You can use the &lt;code&gt;CountVectorizer()&lt;/code&gt; from &lt;code&gt;sklearn.feature_extraction.text&lt;/code&gt; and then use the &lt;code&gt;fit_transform()&lt;/code&gt; if the corpus has been converted into a list of sentences&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;TF-IDF Vectorizer:&lt;/strong&gt; You can use the &lt;code&gt;TfidfVectorizer&lt;/code&gt; from &lt;code&gt;sklearn.feature_extraction.text&lt;/code&gt; and then again use the &lt;code&gt;fit_transform()&lt;/code&gt; on a list of sentences&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;word2vec:&lt;/strong&gt; You can make a &lt;code&gt;word2vec&lt;/code&gt; model from &lt;code&gt;gensim.models&lt;/code&gt; by using  &lt;code&gt;word2vec.Word2vec&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-08-25T01:40:45.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23237" PostTypeId="1" CreationDate="2020-08-25T02:23:25.627" Score="0" ViewCount="29" Body="&lt;p&gt;Would anybody share the experience on how to train a hierarchical DQN to play the Montezuma's Revenge game? How should I design the reward function? How should I balance the anneal rate of the two-level?&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to train an agent to solve this game. The agent is with 6 lives. So, every time when the agent fetches the key and loses his life for the instability or power of the sub-goal network, the agent restart at the original location and simply go through the door, thus gets a huge reward. With an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy rate 0.1, the agent is possible to choose the subgoal key network and fetch the key, so the agent always chooses the door as the subgoal.&lt;/p&gt;&#xA;&lt;p&gt;Would anyone show me how to train this agent in the setting of one life?&lt;/p&gt;&#xA;" OwnerUserId="40552" LastEditorUserId="2444" LastEditDate="2020-08-25T10:27:36.960" LastActivityDate="2020-08-25T10:27:36.960" Title="How to train a hierarchical DQN to play the Montezuma's Revenge game?" Tags="&lt;dqn&gt;&lt;atari-games&gt;&lt;hierarchical-rl&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23238" PostTypeId="1" AcceptedAnswerId="23244" CreationDate="2020-08-25T03:59:23.073" Score="1" ViewCount="45" Body="&lt;p&gt;Based on the Turing test, what would be the criteria for an agent to be considered smart?&lt;/p&gt;&#xA;" OwnerUserId="40555" LastEditorUserId="2444" LastEditDate="2020-08-25T10:29:36.130" LastActivityDate="2020-08-25T15:08:50.297" Title="Based on the Turing test, what would be the criteria for an agent to be considered smart?" Tags="&lt;intelligence-testing&gt;&lt;turing-test&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23239" PostTypeId="2" ParentId="20658" CreationDate="2020-08-25T04:23:36.663" Score="1" Body="&lt;p&gt;Perhaps it could be due to the lack of inputs? Many videos of Snake AIs that use NN and GA to learn seem to have more than double or triple the amount of inputs you're feeding to your neural network (See &lt;a href=&quot;https://www.youtube.com/watch?v=vhiO4WsHA6c&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.youtube.com/watch?v=3bhP7zulFfY&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). I would recommend that you add more inputs by giving the NN the distance to the snake part and the wall for every direction you're looking in.&lt;/p&gt;&#xA;" OwnerUserId="40556" LastActivityDate="2020-08-25T04:23:36.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23241" PostTypeId="1" CreationDate="2020-08-25T05:41:41.407" Score="0" ViewCount="158" Body="&lt;p&gt;I'm working in &lt;strong&gt;A2C&lt;/strong&gt; and I have an environment where there is increasing or decreasing in the number of agents. The action space in the environment will not change but the state will change when new agents join or leave the game.&#xA;I have tried&#xA;&lt;strong&gt;encoder-decoder model with attention&lt;/strong&gt; but the problem is that the state and the model will change when the number of agents is changing.&#xA;I also tried &lt;a href=&quot;https://github.com/livey/scalable_maddpg/blob/master/env/envs.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this way&lt;/a&gt; where they use LSTM to get the Q value for the agent but I got this message&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cannot interpret feed_dict key as Tensor: Tensor Tensor(&amp;quot;state:0&amp;quot;, shape=(137,), dtype=float32) is not an element of this graph.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or error like this because of changing of state size&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Cannot feed value of shape (245,) for Tensor 'state:0', which has shape '(161,)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(1) Are there any reference papers that deal with such a problem?&lt;/p&gt;&#xA;&lt;p&gt;(2) What is the best way to deal with the new agents that join or leave the game?&lt;/p&gt;&#xA;&lt;p&gt;(3) How to deal with the changing of state space?&lt;/p&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="21181" LastEditDate="2020-08-27T08:41:00.870" LastActivityDate="2020-08-29T18:28:29.923" Title="How to handle a changing in the Reinforcement Learning environment where there is increasing or decreasing in number of agents?" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;actor-critic-methods&gt;&lt;environment&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23242" PostTypeId="1" CreationDate="2020-08-25T05:43:09.870" Score="0" ViewCount="58" Body="&lt;p&gt;I have implemented a 3D convolutional neural network and I was not able to find resources for interpretation for my model. I have found some techniques such as GradCam and GradCam++ but these generate class activation maps in 2D only. Is there any way in which I could generate 3d interactive models plotting the activation map on the 3d model.&lt;/p&gt;&#xA;" OwnerUserId="40557" LastActivityDate="2020-08-25T05:43:09.870" Title="Class activation maps for 3D Convolutional neural network?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23243" PostTypeId="1" CreationDate="2020-08-25T09:57:38.277" Score="0" ViewCount="161" Body="&lt;p&gt;How would one extract the feature vector from a given input image using &lt;a href=&quot;https://github.com/Tianxiaomo/pytorch-YOLOv4&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLOv4&lt;/a&gt; and pass that data into an LSTM to generate captions for the image?&lt;/p&gt;&#xA;&lt;p&gt;I am trying to make an image captioning software in PyTorch using YOLO as the base object classifier and an LSTM as the caption generator.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me figure out what part of the code I would need to call and how I would achieve this?&lt;/p&gt;&#xA;&lt;p&gt;Any help is much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="40561" LastEditorUserId="40561" LastEditDate="2020-08-25T10:20:40.043" LastActivityDate="2021-05-28T12:00:13.493" Title="Feeding YOLOv4 image data into LSTM layer?" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;pytorch&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23244" PostTypeId="2" ParentId="23238" CreationDate="2020-08-25T10:47:48.333" Score="1" Body="&lt;p&gt;If an artificial agent (AA) passes the (standard) Turing test (i.e. where you have to imitate a human that speaks), then, on average, the AA should be able to imitate any human in any situation that &lt;strong&gt;mainly&lt;/strong&gt; requires the conversation abilities and common-sense knowledge of a human, without being ever recognized as an AA.&lt;/p&gt;&#xA;&lt;p&gt;For example, if you want to talk about football, you don't expect the AA only to say &amp;quot;I don't know&amp;quot; or to clearly avoid a topic (e.g. by redirecting you to a search engine) when it doesn't know something (although some humans behave in this way), but you expect it to have &lt;strong&gt;common-sense knowledge&lt;/strong&gt;, such as that Messi, Cristiano Ronaldo, Pelé, Maradona, etc., are among the best players of all time, and this should be precious information that the AA should have in any case, even if it doesn't know much about football.&lt;/p&gt;&#xA;&lt;p&gt;You also expect it to &lt;strong&gt;be emotional&lt;/strong&gt; and have a &lt;strong&gt;personality&lt;/strong&gt;, given that humans are emotional and have personalities. So, in the example above, maybe the AA could say that Maradona is its favorite player, and then it could explain why in an emotional way (e.g. by changing the tone of the voice).&lt;/p&gt;&#xA;&lt;p&gt;The AA should also be able to keep track of (almost) everything you and it said, and it should be &lt;strong&gt;able to contextualize&lt;/strong&gt; very well, as humans do. Some personal assistants already take context into account, but they don't do this very well or just do it to a little extent.&lt;/p&gt;&#xA;&lt;p&gt;The AA should also be &lt;strong&gt;able to reason&lt;/strong&gt; given the current situation. For example, if you explain something to the AA, you expect it to infer or predict something based on the information it has acquired and the common-sense knowledge.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, when you speak or write to the AA, you don't expect it to regularly hear badly or not understand what you say or ask (and ask you to repeat), but you expect it to understand well what you say almost always, provided you don't talk or write trash. You also &lt;strong&gt;don't expect big delays&lt;/strong&gt; and you expect the AA to at least say something while it searches for a more appropriate answer, although not all humans behave in this way, but I think that &lt;strong&gt;interjections&lt;/strong&gt; or words such as &amp;quot;hm&amp;quot;, &amp;quot;well&amp;quot;, &amp;quot;let me think...&amp;quot;, etc., will be very important to make the AA look or sound like a human. In general, the AA should be as &lt;strong&gt;interactive&lt;/strong&gt; as a human.&lt;/p&gt;&#xA;&lt;p&gt;These are some traits that the AA absolutely needs to have in order to pass the Turing test (and be considered intelligent according to that test), but there are probably many others.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-25T15:08:50.297" LastActivityDate="2020-08-25T15:08:50.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23245" PostTypeId="1" AcceptedAnswerId="23247" CreationDate="2020-08-25T12:27:47.983" Score="1" ViewCount="39" Body="&lt;p&gt;Basic deep reinforcement learning methods use as input an image for the current state, do some convolutions on that image, apply some reinforcement learning algorithm and it is solved.&lt;/p&gt;&#xA;&lt;p&gt;Let us take the game Breakout or Pong as an example. What I do not understand is, how does the agent understand when an object is moving towards it or away from it ? I believe that the action it chooses must be different in these two scenarios and from a single image as input, there is no notion of motion.&lt;/p&gt;&#xA;" OwnerUserId="36447" LastActivityDate="2020-08-25T12:57:11.813" Title="Deep Reinforcement Learning Atari: how does the agent understand motion?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;atari-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23246" PostTypeId="1" AcceptedAnswerId="23251" CreationDate="2020-08-25T12:35:26.587" Score="2" ViewCount="55" Body="&lt;p&gt;I was looking at the Bellman equation, and I noticed a difference between the equations used in policy evaluation and value iteration.&lt;/p&gt;&#xA;&lt;p&gt;In policy evaluation, there was the presence of &lt;span class=&quot;math-container&quot;&gt;$\pi(a \mid s)$&lt;/span&gt;, which indicates the probability of choosing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, under policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. But this probability seemed to be omitted in the value iteration formula. What might be the reason? Maybe an omission?&lt;/p&gt;&#xA;" OwnerUserId="40049" LastEditorUserId="2444" LastEditDate="2020-08-25T15:13:49.037" LastActivityDate="2020-08-25T15:13:49.037" Title="Why doesn't value iteration use $\pi(a \mid s)$ while policy evaluation does?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23247" PostTypeId="2" ParentId="23245" CreationDate="2020-08-25T12:50:52.573" Score="3" Body="&lt;p&gt;In the article &lt;a href=&quot;https://arxiv.org/pdf/1312.5602.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Playing Atari with Deep Reinforcement Learning&lt;/a&gt;, &lt;em&gt;Mnih et al, 2013&lt;/em&gt;, which was a major outbreak in Deep Reinforcement learning (especially in Deep Q learning), they don't feed only the last image to the network. They stack the 4 last images :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For the experiments in this paper, the function φ from algorithm 1&#xA;applies this preprocessing to the last 4 frames of a history and stacks&#xA;them to produce the input to the Q-function&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So they add the motion through sequentiality. From various articles and own coding experiences, this seems to me to be the main common approach. I don't know if other techniques have been implemented.&lt;/p&gt;&#xA;&lt;p&gt;One thing we could imagine would be to compute the Cross-correlation between a previous frame and the last one, and then feed the cross correlation product to the net.&lt;/p&gt;&#xA;&lt;p&gt;Another idea would be to train previously a CNN to extract motion features from a sequence of frames, and feed these extracted features to your net. &lt;a href=&quot;https://www.duo.uio.no/bitstream/handle/10852/58552/2/ArticlePIVANN.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This article (Performing Particle Image Velocimetry using Artificial Neural Networks: a proof-of-concept)&lt;/a&gt;, &lt;em&gt;Rabault et al, 2017&lt;/em&gt; is an example of a CNN to extract motion features.&lt;/p&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="17759" LastEditDate="2020-08-25T12:57:11.813" LastActivityDate="2020-08-25T12:57:11.813" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23248" PostTypeId="1" CreationDate="2020-08-25T13:11:54.907" Score="0" ViewCount="28" Body="&lt;p&gt;I have a video where multiple persons are seated. I need to keep track of the emotions they show throughout the video. My final result should be a csv file with all the emotions depicted by each person for every 1 minute.&lt;/p&gt;&#xA;&lt;p&gt;So it would look like this :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Person 1 : Angry Happy Sad ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used &lt;a href=&quot;https://github.com/StanislasBertrand/RetinaFace-tf2&quot; rel=&quot;nofollow noreferrer&quot;&gt;RetinaFace&lt;/a&gt; to detect all the faces present in selected frames from the video. &lt;a href=&quot;https://i.stack.imgur.com/qeYp9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qeYp9.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It worked well for the frames detecting almost all the faces. However I would like to keep track of the faces detected throughout the video, as I want to track the emotion depicted by each person throughout the video. But since the locations of faces change throughout the video, I'm a little confused as to how I can accomplish this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Some Ideas :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;These are the annotation files for the first 2 frames used :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0,&amp;quot;[1811, 850, 1948, 1013]&amp;quot;,0.999666452407836,&amp;quot;[[1828, 911], [1887, 913], [1841, 942], [1832, 974], [1876, 976]]&amp;quot;&#xA;1,&amp;quot;[346, 1285, 503, 1468]&amp;quot;,0.9996420145034791,&amp;quot;[[365, 1361], [424, 1348], [385, 1395], [390, 1426], [439, 1416]]&amp;quot;&#xA;2,&amp;quot;[1543, 1418, 1702, 1618]&amp;quot;,0.9995224475860591,&amp;quot;[[1578, 1514], [1647, 1498], [1619, 1554], [1610, 1585], [1658, 1572]]&amp;quot;&#xA;3,&amp;quot;[1191, 838, 1323, 1006]&amp;quot;,0.9994902610778801,&amp;quot;[[1215, 889], [1273, 898], [1228, 930], [1212, 959], [1257, 966]]&amp;quot;&#xA;4,&amp;quot;[2153, 857, 2293, 1047]&amp;quot;,0.9993738532066341,&amp;quot;[[2170, 928], [2227, 925], [2182, 956], [2175, 996], [2219, 992]]&amp;quot;&#xA;5,&amp;quot;[303, 860, 423, 1031]&amp;quot;,0.9985873699188231,&amp;quot;[[315, 929], [354, 922], [319, 962], [327, 995], [355, 988]]&amp;quot;&#xA;6,&amp;quot;[1596, 356, 1691, 467]&amp;quot;,0.998558819293975,&amp;quot;[[1617, 398], [1659, 397], [1633, 418], [1621, 438], [1654, 438]]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;0,&amp;quot;[391, 1246, 541, 1430]&amp;quot;,0.9996455907821651,&amp;quot;[[409, 1316], [469, 1311], [423, 1349], [419, 1383], [466, 1379]]&amp;quot;&#xA;1,&amp;quot;[1556, 1377, 1708, 1577]&amp;quot;,0.9996392726898191,&amp;quot;[[1583, 1466], [1650, 1455], [1610, 1491], [1597, 1533], [1648, 1526]]&amp;quot;&#xA;2,&amp;quot;[1806, 862, 1940, 1031]&amp;quot;,0.999588668346405,&amp;quot;[[1827, 928], [1887, 926], [1848, 963], [1836, 992], [1884, 990]]&amp;quot;&#xA;3,&amp;quot;[678, 876, 812, 1063]&amp;quot;,0.99949049949646,&amp;quot;[[691, 954], [743, 946], [707, 989], [712, 1019], [751, 1011]]&amp;quot;&#xA;4,&amp;quot;[2192, 873, 2338, 1047]&amp;quot;,0.999396681785583,&amp;quot;[[2213, 950], [2260, 936], [2232, 983], [2244, 1010], [2283, 998]]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From these files, we can see that &lt;code&gt;[1543, 1418, 1702, 1618]&lt;/code&gt; and &lt;code&gt;[1556, 1377, 1708, 1577]&lt;/code&gt;, &lt;code&gt;[346, 1285, 503, 1468]&lt;/code&gt; and &lt;code&gt;[391, 1246, 541, 1430]&lt;/code&gt; are similar to each other, so they might be the same faces. Also throughout the video, all the persons will be seated, so their positions might not differ much. But I do not know what margin I could use to find this.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if there are models for keeping track of the faces throughout the video.&lt;/p&gt;&#xA;" OwnerUserId="16881" LastActivityDate="2020-08-25T13:11:54.907" Title="Keeping track of multiple faces throughout a video" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;facial-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23249" PostTypeId="1" CreationDate="2020-08-25T13:27:05.043" Score="0" ViewCount="29" Body="&lt;p&gt;I ultimately want to build an application like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Inputs:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a list of written phrases of 1-5 words each&lt;/li&gt;&#xA;&lt;li&gt;a bunch of audio files like mp3, with total duration in the 100 to&#xA;1000 hours range, and potentially thousands of separate files, all in&#xA;English, moderate audio quality&lt;/li&gt;&#xA;&lt;li&gt;specify whether to check for the presence or absence of each phrase&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A list where a True/False result is given for each phrase / audio&#xA;file combination&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The obvious approach to me is to use a speech processing API, generate a transcript for every file, then search for each phrase in each transcript.&lt;/p&gt;&#xA;&lt;p&gt;I don't have any experience with speech processing so I'm looking to be pointed in the right direction, to know if there are best practices for my specific use case etc, is &amp;quot;audio phrase detection&amp;quot; a thing or does it just come under speech processing? Any answers to the following questions would also be very helpful:&lt;/p&gt;&#xA;&lt;p&gt;Are there existing APIs/services out there that already do what I described (the transcribing but also handle the phrase searching component)&lt;/p&gt;&#xA;&lt;p&gt;Are there specific speech processing algorithms best suited to long durations like this? What sort of runtime can I expect?&lt;/p&gt;&#xA;&lt;p&gt;What sort of error rates should I expect? False positives vs False negatives&lt;/p&gt;&#xA;&lt;p&gt;Any impact from using different filetypes (mp3 vs wav) on accuracy, speed etc?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="40559" LastActivityDate="2020-08-25T13:27:05.043" Title="What is the best approach for phrase detection in speech audio?" Tags="&lt;machine-learning&gt;&lt;audio-processing&gt;&lt;speech-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23250" PostTypeId="2" ParentId="23162" CreationDate="2020-08-25T13:36:46.010" Score="2" Body="&lt;p&gt;If you have to move a lot of data around during training (like retrieving batches from disk/network/what have you), it's much faster to do so as a rank-3 tensor of [batches, documents, indices] than as a rank-4 tensor of [batches, documents, indices, vectors]. In this case, while the embedding is O(1) wherever you put it, it's more efficient to do so as part of the graph.&lt;/p&gt;&#xA;" OwnerUserId="29873" LastActivityDate="2020-08-25T13:36:46.010" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23251" PostTypeId="2" ParentId="23246" CreationDate="2020-08-25T14:15:47.700" Score="3" Body="&lt;p&gt;You appear to comparing the value table update steps in &lt;em&gt;policy iteration&lt;/em&gt; and &lt;em&gt;value iteration&lt;/em&gt;, which are both derived from Bellman equations.&lt;/p&gt;&#xA;&lt;h2&gt;Policy iteration&lt;/h2&gt;&#xA;&lt;p&gt;In &lt;em&gt;policy iteration&lt;/em&gt;, a policy lookup table is generated, which can be arbitrary. It usually maps a deterministic policy &lt;span class=&quot;math-container&quot;&gt;$\pi(s): \mathcal{S} \rightarrow \mathcal{A}$&lt;/span&gt;, but can also be of the form &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s): \mathcal{A} \times  \mathcal{S} \rightarrow \mathbb{R} = Pr\{A_t = a |S_t =s\}$&lt;/span&gt;. Policy iteration then alternately evaluates then improves that policy, with the improvement always being to act greedily with respect to expected return. Because the policy function can be arbitrary, and also the current value estimates during evaluation might not relate to it directly, the function &lt;span class=&quot;math-container&quot;&gt;$\pi(s)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; needs to be shown.&lt;/p&gt;&#xA;&lt;p&gt;Typically with policy iteration, you will see this update rule:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \leftarrow \sum_{r,s'} p(r,s'|s,\pi(s))(r + \gamma V(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above rule is for evaluating a deterministic policy, and is probably more commonly used. There is no real benefit in policy iteration to working with stochastic policies.&lt;/p&gt;&#xA;&lt;p&gt;For completeness, the update rule for an arbitrary stochastic policy is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \leftarrow \sum_a \pi(a|s) \sum_{r,s'} p(r,s'|s,a)(r + \gamma V(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Value iteration&lt;/h2&gt;&#xA;&lt;p&gt;In &lt;em&gt;value iteration&lt;/em&gt;, the current policy to evaluate is to always take the greedy action with respect to the current evaluations. As such, it does not need to be explicity written, because it can be derived from the value function, and so can the terms in the Bellman equation (specifically the Bellman equation for the optimal value function is used here, which usually does not refer the policy). What you would typically write for the update step is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \leftarrow \text{max}_a \sum_{r,s'} p(r,s'|s,a)(r + \gamma V(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, you can write this out as if there was a policy table:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) \leftarrow \text{argmax}_a \sum_{r,s'} p(r,s'|s,a)(r + \gamma V(s'))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$a \leftarrow \pi(s)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \leftarrow \sum_{r,s'} p(r,s'|s,a)(r + \gamma V(s'))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is not the usual way to implement it though, because of the extra maximum value search required to identify the action. In simple value iteration it does not matter what the interim action choices and policies actually are, and you can always derive them from the value function if you want to know.&lt;/p&gt;&#xA;&lt;h2&gt;Other value-based methods&lt;/h2&gt;&#xA;&lt;p&gt;You will find other algorithms that drive the current policy direct from a value function, and when they are described in pseudo-code they might not have an explicit policy function. It is still there, only the Bellman update is easily calculated directly from the value function, so the policy is not shown in the update step. Descriptions of SARSA and Q-learning are often like that.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-25T14:44:04.807" LastActivityDate="2020-08-25T14:44:04.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23252" PostTypeId="1" CreationDate="2020-08-25T14:29:35.050" Score="0" ViewCount="52" Body="&lt;p&gt;Past few days I am doing some research on Federated Learning. I got many solutions with MNIST dataset using Nural Net but I am thinking to solve some common Machine Learning problem like Churn Prediction etc. using Federated Learning or implementing some General ML algorithms like Logistic &amp;amp; Linear Regression, SVM, Random Forest etc. using Federated Learning. Can you guys please share me some resource on it any source code or article anything???&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="13007" LastActivityDate="2020-08-25T14:29:35.050" Title="Logistic Regression or General Machine Learning Model using Federated Learning" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;federated-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23254" PostTypeId="1" AcceptedAnswerId="23302" CreationDate="2020-08-25T22:16:33.010" Score="3" ViewCount="158" Body="&lt;p&gt;I am training an RL agent using Deep-Q learning with experience replay. At each frame, I am currently sampling 32 random transitions from a queue which stores a maximum of 20000 and training as described in the Atari with Deep RL paper. All is working fine, but I was wondering whether there is any logical way to select the proper batch size for training, or if simply using a grid search is best. At the moment, I’m simply using 32, for its small enough that I can render the gameplay throughout training at a stunning rate of 0.5fps. However, I’m wondering how much of an effect batch size has, and if there is any criteria we could generalize across all Deep Q-learning tasks.&lt;/p&gt;&#xA;" OwnerUserId="40575" LastActivityDate="2020-08-27T23:33:19.387" Title="Is there a logical method of deducing an optimal batch size when training a Deep Q-learning agent with experience replay?" Tags="&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23256" PostTypeId="1" CreationDate="2020-08-25T23:30:53.537" Score="1" ViewCount="144" Body="&lt;p&gt;In the convolutional layer for CNNs, when you specify the stride of a filter, typical notes show some examples of this but only for the horizontal panning. Is this same stride applied for the vertical direction too when you're done with the current row?&lt;/p&gt;&#xA;&lt;p&gt;In other words, say our input volume is 7x7, and we apply a stride of 1 for a 3x3 filter. Is the output volume 5x5? (which would mean you applied the stride in both the horizontal and vertical panning).&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to apply a different stride for each direction?&lt;/p&gt;&#xA;" OwnerUserId="20358" LastEditorUserId="2444" LastEditDate="2021-01-24T01:55:26.327" LastActivityDate="2021-01-24T01:55:26.327" Title="Is the stride applied both in the horizontal and vertical directions in convolutional neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolutional-layers&gt;&lt;stride&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23257" PostTypeId="2" ParentId="23256" CreationDate="2020-08-26T01:35:19.113" Score="1" Body="&lt;p&gt;Yes, in Keras this is simply implemented by using a tuple for the stride argument of a convolutional layer, with each element of the tuple corresponding to the stride of each dimension.&lt;/p&gt;&#xA;" OwnerUserId="40575" LastActivityDate="2020-08-26T01:35:19.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23258" PostTypeId="1" CreationDate="2020-08-26T04:10:01.610" Score="1" ViewCount="35" Body="&lt;p&gt;I know the training of neural nets involves some sort of dimension manipulation to separate classes of different features.&lt;/p&gt;&#xA;&lt;p&gt;If there is no variation of features, no matter for neural nets or simple dimension reduction methods (e.g. PCA, LDA) + clustering, neither of them are going to distinguish different classes.&lt;/p&gt;&#xA;&lt;p&gt;In such sense, I would like to know the true power of neural nets:&lt;/p&gt;&#xA;&lt;p&gt;How classification neural nets are different from simple dimension reduction + clustering?&lt;/p&gt;&#xA;&lt;p&gt;or rephrase the question:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What value do neural nets add to solving classification problems in terms of its algorithmic architecture compared with simple dimension reduction + clustering?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="40581" LastActivityDate="2020-08-26T04:50:47.433" Title="How classification neural nets are different from simple dimension reduction + clustering?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;dimensionality-reduction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23259" PostTypeId="2" ParentId="23258" CreationDate="2020-08-26T04:50:47.433" Score="2" Body="&lt;p&gt;PCA is linear, NN are nonlinear, more generally it's a universal function approximator.&lt;/p&gt;&#xA;&lt;p&gt;That said basic NN are not terribly usefull, the real value of NN is for structured data which is obvious for us but hard to describe analytically. Basically NN architecture has been designed to learn this structure using operations like convolution and maxpooling.&lt;/p&gt;&#xA;&lt;p&gt;There are still problems of course, and in some ways I think clustering algorithms still have value for things like anomaly detection.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-08-26T04:50:47.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23260" PostTypeId="1" CreationDate="2020-08-26T08:27:42.267" Score="1" ViewCount="53" Body="&lt;p&gt;In Simulated Annealing, a worse solution is accepted with this probability:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p=e^{-\frac{E(y)-E(x)}{kT}}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If that understanding is correct: Why is this probability function used? This means that, the bigger the energy difference, the smaller the probability of accepting the new solution. I would say the bigger the difference the more we want to escape a local minimum. I plotted that function in Matlab in two dimensions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x1P0p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x1P0p.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2021-01-23T18:51:13.607" LastActivityDate="2021-01-23T18:51:13.607" Title="Why does Simulated Annealing not take worse solution if the energy difference becomes higher?" Tags="&lt;optimization&gt;&lt;simulated-annealing&gt;&lt;meta-heuristics&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23261" PostTypeId="1" AcceptedAnswerId="23271" CreationDate="2020-08-26T09:16:28.173" Score="0" ViewCount="64" Body="&lt;p&gt;&lt;strong&gt;Problem Description&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as &lt;code&gt;case&lt;/code&gt; or &lt;code&gt;control&lt;/code&gt; using gene expression values (from &lt;code&gt;Gene-A&lt;/code&gt;, &lt;code&gt;Gene-B&lt;/code&gt; and &lt;code&gt;Gene-C&lt;/code&gt;) using decision tree classifier. I am using the &lt;code&gt;entropy&lt;/code&gt; criteria for node splitting and is implementing the algorithm in python. The classifier is easily able to differentiate the samples.&lt;/p&gt;&#xA;&lt;p&gt;Below is the &lt;em&gt;sample data&lt;/em&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;sample training set&lt;/strong&gt; with labels&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene-A    Gene-B    Gene-C    Sample&#xA;   1        0         38       Case&#xA;   0        7         374      Case&#xA;   1        6         572      Case&#xA;   0        2         538      Control&#xA;   33       5         860      Control&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;sample testing set&lt;/strong&gt; labels&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene-A    Gene-B    Gene-C    Sample&#xA;  1         6        394       Case&#xA;  13        4        777       Control&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have gone through a lot of resources and have learned, how to mathematically calculate &lt;code&gt;Gini-impurity&lt;/code&gt;, &lt;code&gt;entropy&lt;/code&gt; and &lt;code&gt;information gain&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am not able to comprehend how the actual training and testing work. It would be really helpful if someone can show the calculation for training and testing with my &lt;strong&gt;sample datasets&lt;/strong&gt; or provide an online resource?&lt;/p&gt;&#xA;&lt;p&gt;I have asked this question initially on &lt;a href=&quot;https://math.stackexchange.com/questions/3803702/mathematical-calculation-behind-decision-tree-classifier-with-continuous-variabl?noredirect=1#comment7839606_3803702&quot;&gt;Mathematics Stack Exchange&lt;/a&gt; but was redirected here.&lt;/p&gt;&#xA;" OwnerUserId="40584" LastActivityDate="2020-08-26T19:13:41.287" Title="Mathematical calculation behind decision tree classifier with continuous variables" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;decision-tree&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23262" PostTypeId="1" CreationDate="2020-08-26T10:56:52.623" Score="3" ViewCount="84" Body="&lt;p&gt;I'm trying to understand how to calculate the strength of every arc in a Bayesian Network.&lt;/p&gt;&#xA;&lt;p&gt;I came across this report &lt;a href=&quot;https://smartech.gatech.edu/bitstream/handle/1853/14331/GT-IIC-07-01.pdf?sequence=1&amp;amp;isAllowed=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Measuring Connection Strengths and Link Strengths in Discrete Bayesian Networks&lt;/a&gt;, but I got lost in the calculation.&lt;/p&gt;&#xA;&lt;p&gt;In particular, how are the values of Link Strength true, Link Strength blind, and Mutual Information computed in Table 1?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UMRTd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UMRTd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40586" LastEditorUserId="2444" LastEditDate="2021-05-24T00:39:02.027" LastActivityDate="2021-05-24T00:39:02.027" Title="How are the &quot;Link Strength true&quot;, &quot;Link Strength blind&quot; and &quot;Mutual Information&quot; calculated in this report on Bayesian networks?" Tags="&lt;papers&gt;&lt;bayesian-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23263" PostTypeId="1" CreationDate="2020-08-26T12:11:58.467" Score="1" ViewCount="29" Body="&lt;p&gt;Is there any effort made to compress text (and maybe other media) using prediction of next word and thus sending only the order number of the word/token which will be predicted on the client side&#xA;i.e&lt;br&gt;&#xA;&lt;strong&gt;Server text&lt;/strong&gt;: This is an example of a long text example, custom word flerfom inserted to confuse, that may appear on somewhere&lt;br&gt;&#xA;&lt;strong&gt;Compressed Text transmitted&lt;/strong&gt; : This [choice no 3] [choice no 4] [choice no 1] [choice no 6] [choice no 1] [choice no 3] [choice no 1], custom word flerfom [choice no 4] inserted [choice no 4] confuse [choice no 5] [choice no 4] [choice no 6] [choice no 5] on somewhere&#xA;&lt;br&gt;&lt;br&gt;(Note: of course [choice no 3] will be shortened to [3] to save bytes and also maybe we can do much better in some cases by sending the first letter of the word)&#xA;&lt;br&gt;&lt;br&gt;of course it means that the client side neural network has to be static or only updated in a predictable fasion, so the server knows for sure that the client neural network's predictions will follow the given choice orders. I tried example with &lt;a href=&quot;https://demo.allennlp.org/next-token-lm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://demo.allennlp.org/next-token-lm&lt;/a&gt;, but the prediction is not that good. maybe gpt-3 can do better . but its too heavy for use in a normal pc / mobile device&lt;/p&gt;&#xA;&lt;p&gt;In more details, the process is&lt;/p&gt;&#xA;&lt;p&gt;Deploy the same model on both sides&lt;br&gt;&#xA;Predict the next word after the starting word&lt;br&gt;&#xA;Keep the prediction limit say 100&lt;br&gt;&#xA;For any word which have more than 2 characters we do the prediction&lt;br&gt;&#xA;If the current word is predicted within the top 100 predictions of the model , we can essentially replace it with a numeric char between 0-99 (inclusive) so we are replacing a say , 5 character word with a 2 character numerical char..&lt;br&gt;&#xA;And if the word is not predicted in top 100 we send the word as it is..&lt;br&gt;&#xA;As much better the model predicts, that much better the compression&lt;br&gt;&#xA;And under no scenario it will work worse than the existing method..&lt;br&gt;&lt;/p&gt;&#xA;" OwnerUserId="40588" LastEditorUserId="1671" LastEditDate="2020-08-27T00:08:24.007" LastActivityDate="2020-08-27T00:08:24.007" Title="Compressing text using AI by sending only prediction rank of next word" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;prediction&gt;&lt;game-theory&gt;&lt;information-theory&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23264" PostTypeId="2" ParentId="23263" CreationDate="2020-08-26T12:37:31.830" Score="1" Body="&lt;p&gt;If you have a fixed predictor, then yes. If the predictor is not fixed but deterministic, the feasibility depends on the effort needed to update the predictor, and ensuring that messages include a time stamp to ensure the correct version of the predictor is used for compression and inflation.&lt;/p&gt;&#xA;&lt;p&gt;You get a really nice property if the prediction order is in order of probability that the word occurs, and if you use fewer bits for lower numbers. You would get something that is pretty close to &lt;a href=&quot;https://en.wikipedia.org/wiki/Shannon_coding&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shannon coding&lt;/a&gt;, which is not optimal but is still valid.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-26T12:37:31.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23265" PostTypeId="1" AcceptedAnswerId="23273" CreationDate="2020-08-26T15:18:39.967" Score="0" ViewCount="73" Body="&lt;p&gt;I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once.&lt;/p&gt;&#xA;&lt;p&gt;Assuming I have an NN with a single &lt;a href=&quot;https://keras.io/api/layers/core_layers/dense/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Dense&lt;/code&gt;&lt;/a&gt; layer. This &lt;code&gt;Dense&lt;/code&gt; layer of &lt;code&gt;20 units&lt;/code&gt; has an input shape &lt;code&gt;(10, 3)&lt;/code&gt;. This means that I am feeding the NN 10 examples at once, with every example being represented by 3 values. This &lt;code&gt;Dense&lt;/code&gt; layer will have an output shape of &lt;code&gt;(10, 20)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I understand that the 20 in the 2nd dimension comes from the number of units in the &lt;code&gt;Dense&lt;/code&gt; layer. However, what does the &lt;code&gt;10 (Batch Size)&lt;/code&gt; in the first dimension mean? Does this mean that the NN learns 10 separate sets of weights (with each set of weights corresponding to one example, and one set of weights being a matrix of 60 values:3 features x 20 units)?&lt;/p&gt;&#xA;" OwnerUserId="40592" LastEditorUserId="2444" LastEditDate="2020-08-30T12:23:28.703" LastActivityDate="2020-08-30T12:23:28.703" Title="Why does the output shape of a Dense layer contain a batch size?" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;hidden-layers&gt;&lt;fully-connected-layer&gt;&lt;batch-learning&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23266" PostTypeId="1" AcceptedAnswerId="23281" CreationDate="2020-08-26T15:31:02.673" Score="0" ViewCount="51" Body="&lt;p&gt;Currently, I'm working on 6-axis IMU(Inertial Measurment Unit) dataset. &lt;a href=&quot;https://www.kaggle.com/dasmehdixtr/carla-driver-behaviour-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;This dataset&lt;/a&gt; contain 6 axis IMU data of 7 different drivers. The Imu sensor attached on vehicle. The drivers drives same path. So, the dataset include 6 feature columns and a label column.&lt;/p&gt;&#xA;&lt;p&gt;I tried multiple neural network models.The sensor data is a sequential data so I tried LSTM(Long Short Term Memory) &amp;amp; classical fully-connected layers.&#xA;Some of my architecture(in keras framework):&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;Layer (type)                 Output Shape              Param #   &#xA;&#xA;lstm_4 (LSTM)                (None, 1, 128)            69120     &#xA;_________________________________________________________________&#xA;lstm_5 (LSTM)                (None, 1, 64)             49408     &#xA;_________________________________________________________________&#xA;lstm_6 (LSTM)                (None, 1, 32)             12416     &#xA;_________________________________________________________________&#xA;dense_8 (Dense)              (None, 1, 64)             2112      &#xA;_________________________________________________________________&#xA;dropout_2 (Dropout)          (None, 1, 64)             0         &#xA;_________________________________________________________________&#xA;dense_9 (Dense)              (None, 1, 7)              455       &#xA;&#xA;&#xA;2nd Architecture:&#xA;&#xA;=================================================================&#xA;dense_10 (Dense)             (None, 32)                224       &#xA;_________________________________________________________________&#xA;dense_11 (Dense)             (None, 64)                2112      &#xA;_________________________________________________________________&#xA;dense_12 (Dense)             (None, 128)               8320      &#xA;_________________________________________________________________&#xA;dense_13 (Dense)             (None, 256)               33024     &#xA;_________________________________________________________________&#xA;dropout_3 (Dropout)          (None, 256)               0         &#xA;_________________________________________________________________&#xA;dense_14 (Dense)             (None, 512)               131584    &#xA;_________________________________________________________________&#xA;dense_15 (Dense)             (None, 256)               131328    &#xA;_________________________________________________________________&#xA;dense_16 (Dense)             (None, 128)               32896     &#xA;_________________________________________________________________&#xA;dense_17 (Dense)             (None, 64)                8256      &#xA;_________________________________________________________________&#xA;dropout_4 (Dropout)          (None, 64)                0         &#xA;_________________________________________________________________&#xA;dense_18 (Dense)             (None, 128)               8320      &#xA;_________________________________________________________________&#xA;dense_19 (Dense)             (None, 7)                 903     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The best accuracy in my models was %70 which is not good. How style of layers should I use to handle this data? Or, which type of model would increase accuracy?&lt;/p&gt;&#xA;" OwnerUserId="28129" LastActivityDate="2020-08-27T00:14:38.397" Title="What type of model should I fit to increase accuracy?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23267" PostTypeId="2" ParentId="23260" CreationDate="2020-08-26T15:50:03.213" Score="0" Body="&lt;p&gt;Nice question!&lt;/p&gt;&#xA;&lt;p&gt;My guess is that, if the probability of acceptance increases the bigger the difference between the current and new solutions is, then there's the risk that you need to search a lot again to find a good solution, i.e. you may oscillate between different subspaces or you could actually often end up in subspaces where there are only bad solutions. Your reasoning probably makes sense at the beginning of the search when the initial solutions may not be good enough or maybe if you have parallel searches (and you want to explore the search space at different search subspaces), but once you have a good solution, you don't want to completely discard it and replace it with a quite worse solution.&lt;/p&gt;&#xA;&lt;p&gt;If you perform some experiments with your idea, I would like to see the results.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-26T16:03:48.660" LastActivityDate="2020-08-26T16:03:48.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23268" PostTypeId="2" ParentId="23260" CreationDate="2020-08-26T16:16:12.427" Score="1" Body="&lt;p&gt;Note that you can't really predict whether your escape from a local minimum will work or not - you might just wind up in another, worse local minimum. The probability function you describe increases the likelihood of this happening. By upweighting the likelihood of allowing small energy differences, you allow for the possibility of escaping local minima, while ensuring that whatever new minimum you find can't be that much worse that where you started. If you make the acceptance of large energy differences more likely, you can escape local minima more often, but you increase the likelihood that you'll just wind up in a region with an &lt;em&gt;even higher&lt;/em&gt; local minimum.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2020-08-26T16:16:12.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23269" PostTypeId="2" ParentId="23262" CreationDate="2020-08-26T18:10:50.180" Score="0" Body="&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$MI$&lt;/span&gt; is easy enough.&#xA;For &lt;span class=&quot;math-container&quot;&gt;$MI(X \to Z)$&lt;/span&gt;, we get &lt;span class=&quot;math-container&quot;&gt;$U([0.5, 0.5]) - 0.5 U([0.9, 0.1]) - 0.5 U([0.1, 0.9])$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$[0.5, 0.5]$&lt;/span&gt; comes from &lt;span class=&quot;math-container&quot;&gt;$0.5 [0.9, 0.1] + 0.5[0.1, 0.9]$&lt;/span&gt; -- this is how to calculate &lt;span class=&quot;math-container&quot;&gt;$P(Z)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$P(Z|X)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(X)$&lt;/span&gt;.&#xA;For &lt;span class=&quot;math-container&quot;&gt;$MI(X \to Y)$&lt;/span&gt;, we need to marginalize out &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;. This takes a bit of work, but is not too hard. I manually verified all of the reported values in the figure, and they are all correct.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\mathrm{LS}{\mathrm{true}}$&lt;/span&gt; values are a bit trickier, but still feasible. I managed to get the same values. It is important when calculating &lt;span class=&quot;math-container&quot;&gt;$\mathrm{LS}{\mathrm{true}}(X \to Z)$&lt;/span&gt; that &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is not a parent, so it does not come into it -- which is why the MI gives exactly the same value.&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;blind&amp;quot; version took me a bit more effort to implement, but again, I am getting the same values as reported in the table.&#xA;I used the &amp;quot;simple formula&amp;quot; at the top of page 5.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-26T18:10:50.180" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="23270" PostTypeId="1" AcceptedAnswerId="23277" CreationDate="2020-08-26T18:52:41.560" Score="2" ViewCount="114" Body="&lt;p&gt;I was reading &lt;a href=&quot;https://www.unil.ch/files/live/sites/ln/files/shared/Revonsuo_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; about the question &amp;quot;Why do we dream?&amp;quot; in which the author discusses dreams as a form of rehearsal for future threats, and presents it as an evolutive advantage. My question is &lt;strong&gt;whether this idea has been explored in the context of RL&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, in a competition between AIs on a shooter game, one could design an agent that, besides the behavior it has learned in a &amp;quot;normal&amp;quot; training, seeks for time in which is out of danger, to then use its computation time in the game to produce simulations that would further optimize its behavior. As the agent still needs to be somewhat aware of its environment, it could alternate between processing the environment and this kind of simulation. Note that this &amp;quot;in-game&amp;quot; simulation has an advantage with respect to the &amp;quot;pre-game&amp;quot; simulations used for training; the agent in the game experiences the behavior of the other agents, which could not have been predicted beforehand, and then simulates on top of these experiences, e.g. by slightly modifying them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;For more experienced folks, does this idea make sense? has something similar been explored?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have absolutely no experience in the field, so I apologize if this question is poorly worded, dumb or obvious. I would appreciate suggestions on how to improve it if this is the case.&lt;/p&gt;&#xA;" OwnerUserId="40596" LastEditorUserId="2444" LastEditDate="2020-08-26T22:49:32.240" LastActivityDate="2020-08-26T23:19:44.113" Title="Have agents that &quot;dream&quot; been explored in Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;model-based-methods&gt;&lt;imagination&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23271" PostTypeId="2" ParentId="23261" CreationDate="2020-08-26T19:13:41.287" Score="0" Body="&lt;p&gt;Of course, it depends on what algorithm you use. Typically, a top-down algorithm is used.&lt;/p&gt;&#xA;&lt;p&gt;You gather all the training data at the root. The base decision is going to be whatever class you have most of. Now, we see if we can do better.&lt;/p&gt;&#xA;&lt;p&gt;We consider all possible splits. For categorical variables, every value gets its own node. For continuous variables, we can use any possible midpoint between two values (if the values were sorted). For your example, possible splits are &lt;code&gt;Gene-A &amp;lt; 0.5&lt;/code&gt;, &lt;code&gt;Gene-A &amp;lt; 17&lt;/code&gt;, &lt;code&gt;Gene-B &amp;lt; 1&lt;/code&gt;, &lt;code&gt;Gene-B &amp;lt; 3.5&lt;/code&gt;, and so on. There is a total of 10 possible splits.&lt;/p&gt;&#xA;&lt;p&gt;For each of those candidate splits, we measure how much the entropy decreases  (or whatever criterion we selected) and, if this decrease looks significant enough, we introduce this split.&#xA;For example. Our entropy in the root node is &lt;span class=&quot;math-container&quot;&gt;$-0.4 \log_2 0.4 - 0.6 \log_2 0.6 \approx 0.97$&lt;/span&gt;. If we introduce the split &lt;code&gt;Gene-A &amp;lt; 0.5&lt;/code&gt;, we get one leaf with entropy &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (with 2 data points in it), and one leaf with entropy &lt;span class=&quot;math-container&quot;&gt;$0.918$&lt;/span&gt; (with 3 data points). The total decrease of entropy is &lt;span class=&quot;math-container&quot;&gt;$0.97 - (\frac25 \times 1 + \frac35 \times 0.918) \approx 0.02$&lt;/span&gt;.&#xA;For the split &lt;code&gt;Gene-A &amp;lt; 17&lt;/code&gt; we get a decrease of entropy of about &lt;span class=&quot;math-container&quot;&gt;$0.3219$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The best splits for the root are &lt;code&gt;Gene-B &amp;lt; 5.5&lt;/code&gt; and &lt;code&gt;Gene-C &amp;lt; 456&lt;/code&gt;. These both reduce the entropy by about &lt;span class=&quot;math-container&quot;&gt;$0.42$&lt;/span&gt;, which is a substantial improvement.&lt;/p&gt;&#xA;&lt;p&gt;When you choose a split, you introduce a leaf for the possible outcomes of the test. Here it's just 2 leaves: &amp;quot;yes, the value is smaller than the threshold&amp;quot; or &amp;quot;no, it is not smaller&amp;quot;. In every leaf, we collect the training data from the parent that corresponds to this choice. So, if we select &lt;code&gt;Gene-B &amp;lt; 5.5&lt;/code&gt; as our split, the &amp;quot;yes&amp;quot; leaf will contain the first, fourth and fifth data points, and the &amp;quot;no&amp;quot; leaf will contain the other data points.&lt;/p&gt;&#xA;&lt;p&gt;Then we continue, by repeating the process for each of the leaves. In our example, the &amp;quot;yes&amp;quot; branch can still be split further. A good split would be &lt;code&gt;Gene-C &amp;lt; 288&lt;/code&gt;, which results in pure leaves (they have 0 entropy).&lt;/p&gt;&#xA;&lt;p&gt;When a leaf is &amp;quot;pure enough&amp;quot; (it has very low entropy) or we don't think we have enough data, or the best split for a leaf is not a significant improvement, or we have reached a maximum depth, you stop the process for that leaf. In this leaf you can store the count for all the classes you have in the training data.&lt;/p&gt;&#xA;&lt;p&gt;If you have to make a prediction for a new data point (from the test set), you start at the root and look at the test (the splitting criterion). For example, for the first test point, we have that &lt;code&gt;Gene-B &amp;lt; 5.5&lt;/code&gt; is false, so we go to the 'no' branch. You continue until you get to a leaf.&lt;/p&gt;&#xA;&lt;p&gt;In a leaf, you would predict whatever class you have most of. If the user wants, you can also output a probability by giving the proportion.&#xA;For the first test point, we go to the &amp;quot;no&amp;quot; branch of the first test, and we end up in a leaf; our prediction would be &amp;quot;&lt;code&gt;Case&lt;/code&gt;&amp;quot;. For the second test point, we go to the &amp;quot;yes&amp;quot; branch of the first test. Here we test whether &lt;code&gt;777 &amp;lt; 288&lt;/code&gt;, which is false, so we go to the &amp;quot;no&amp;quot; branch, and end up in a leaf. This leaf contains only &amp;quot;&lt;code&gt;Control&lt;/code&gt;&amp;quot; cases, so our prediction would be &amp;quot;&lt;code&gt;Control&lt;/code&gt;&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-26T19:13:41.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23272" PostTypeId="2" ParentId="23270" CreationDate="2020-08-26T19:34:41.377" Score="0" Body="&lt;p&gt;Model-based RL is obviously the correct approach. Mainly because it lets you simulate the environment internally without having direct interaction.&lt;/p&gt;&#xA;&lt;p&gt;And all successful RL algorithms essentially are model-based because nobody has done real-time RL and been successful.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-08-26T22:07:58.053" LastActivityDate="2020-08-26T22:07:58.053" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23273" PostTypeId="2" ParentId="23265" CreationDate="2020-08-26T19:41:28.677" Score="0" Body="&lt;p&gt;The Dense layers outputs 20 values per example. And since you have 10 examples in the batch the output is &lt;code&gt;(10, 20)&lt;/code&gt; (one set of 20 values per example in the batch).&#xA;The nn doesn't learn 10 separate sets of weights. Each set of 20 values is computed with the same weight (and bias if you have any). So if say example 2 and 5 had the same input values, they'll always have the same output values.&lt;/p&gt;&#xA;" OwnerUserId="40597" LastActivityDate="2020-08-26T19:41:28.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23276" PostTypeId="1" CreationDate="2020-08-26T20:02:03.287" Score="2" ViewCount="78" Body="&lt;p&gt;I have a conceptual question for you all that hopefully I can convey clearly. I am building an RL agent in Keras using continuous PPO to control a laser attached to a pan/tilt turret for target tracking. My question is how the new policy gets updated. My current implementation is as follows&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make observation (distance from laser to target in pan and tilt)&lt;/li&gt;&#xA;&lt;li&gt;Pass observation to actor network which outputs a mean (std for now is fixed)&lt;/li&gt;&#xA;&lt;li&gt;I sample from a gaussian with the mean output from step 2&lt;/li&gt;&#xA;&lt;li&gt;Apply the command and observe the reward (1/L2 distance to target)&lt;/li&gt;&#xA;&lt;li&gt;collect N steps of experience, compute advantage and old log probabilities,&lt;/li&gt;&#xA;&lt;li&gt;train actor and critic&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My question is this. I have my old log probabilities (probabilities of the actions taken given the means generated by the actor network), but I dont understand how the new probabilities are generated. At the onset of the very first minibatch my new policy is identical to my old policy as they are the same neural net. Given that in the model.fit function I am passing the same set of observations to generate 'y_pred' values, and I am passing in the actual actions taken as my 'y_true' values, the new policy should generate the exact same log probabilities as my old one. The only (slight) variation that makes the network update is from the entropy bonus, but my score&#xA;np.exp(new_log_probs-old.log_probs)  is nearly identically 1 because the policies are the same.&lt;/p&gt;&#xA;&lt;p&gt;Should I be using a pair of networks similar to DDQN so there are some initial differences in the policies between the one used to generate the data and the one used for training?&lt;/p&gt;&#xA;" OwnerUserId="40600" LastActivityDate="2020-08-27T07:30:26.420" Title="Generation of 'new log probabilities' in continuous action space PPO" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;loss-functions&gt;&lt;policy-gradients&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23277" PostTypeId="2" ParentId="23270" CreationDate="2020-08-26T22:17:21.330" Score="2" Body="&lt;p&gt;Yes, the concept of &lt;em&gt;dreaming&lt;/em&gt; or &lt;em&gt;imagining&lt;/em&gt; has already been explored in reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;For example, have a look at &lt;a href=&quot;https://arxiv.org/pdf/1705.02670.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metacontrol for Adaptive Imagination-Based Optimization&lt;/a&gt; (2017) by Jessica B. Hamrick et al., which is a paper that I gave a talk/presentation on 1-2 years ago (though I don't remember well the details anymore).&lt;/p&gt;&#xA;&lt;p&gt;There is also a blog post about the topic &lt;a href=&quot;https://deepmind.com/blog/article/agents-imagine-and-plan&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agents that imagine and plan&lt;/a&gt; (2017) by DeepMind, which discusses two more recent papers and also mentions Hamrick's paper.&lt;/p&gt;&#xA;&lt;p&gt;In 2018, another related and interesting paper was also presented at NIPS, i.e. &lt;a href=&quot;https://worldmodels.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Models&lt;/a&gt;, by Ha and Schmidhuber.&lt;/p&gt;&#xA;&lt;p&gt;If you search for &amp;quot;imagination/dreaming in reinforcement learning&amp;quot; on the web, you will find more papers and articles about this interesting topic.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-26T23:19:44.113" LastActivityDate="2020-08-26T23:19:44.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23278" PostTypeId="5" CreationDate="2020-08-26T22:49:52.820" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-26T22:49:52.820" LastActivityDate="2020-08-26T22:49:52.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23279" PostTypeId="4" CreationDate="2020-08-26T22:49:52.820" Score="0" Body="For questions related to the concept of imagination in the context of artificial intelligence and cognitive science." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-26T22:49:52.820" LastActivityDate="2020-08-26T22:49:52.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23280" PostTypeId="1" AcceptedAnswerId="23311" CreationDate="2020-08-26T22:52:13.137" Score="0" ViewCount="50" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt; In C51 DQNs you must specify a v-min/max to be used during training. The way this is generally done is you take the max score possible for the game and set that to v-max, then v-min is just negative v-max. For a game like Pong deciding the v-min/max is simple because the max score possible is 20, therefore, &lt;code&gt;v_min=-20&lt;/code&gt; and &lt;code&gt;v_max=20&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; In a game like Space Invaders, there is no max score, so how would I calculate the v-min/max for a C51 DQN?&lt;/p&gt;&#xA;" OwnerUserId="38062" LastEditorUserId="38062" LastEditDate="2020-08-28T01:37:26.940" LastActivityDate="2020-08-28T14:49:55.860" Title="How to calculate v min and v max for C51 DQN" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;probability-distribution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23281" PostTypeId="2" ParentId="23266" CreationDate="2020-08-27T00:14:38.397" Score="2" Body="&lt;p&gt;after reading some literature in the area I'd recommend the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Try using Convolution Neural Networks (CNNs) &lt;a href=&quot;https://wiki.tum.de/download/attachments/242090417/IMU_Sensor_Fusion_With_ML.pdf?version=1&amp;amp;modificationDate=1562329774913&amp;amp;api=v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper outlines some really good points on why should you use CNNs.&lt;/li&gt;&#xA;&lt;li&gt;Try a Combination of the different layers in the same model. Start with some Convolution Layers, then some LSTMs and then a couple of Dense Layers followed by Dropout&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-08-27T00:14:38.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23282" PostTypeId="2" ParentId="23256" CreationDate="2020-08-27T00:21:37.847" Score="1" Body="&lt;p&gt;Yes, in Keras you can apply different strides by giving a &lt;strong&gt;tuple/list&lt;/strong&gt;, specifying the value of strides along the height and width. If you just give a &lt;strong&gt;single&lt;/strong&gt; value the API assumes the &lt;strong&gt;same value&lt;/strong&gt; for &lt;strong&gt;all spatial dimensions&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can find the official documentation &lt;a href=&quot;https://keras.io/api/layers/convolution_layers/convolution2d/#:%7E:text=strides%3A%20An%20integer%20or%20tuple,Specifying%20any%20stride%20value%20!%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In Pytorch, too you can specify the values in a tuple for the &lt;code&gt;stride&lt;/code&gt;&#xA;argument. Link to Pytorch Documentation for &lt;a href=&quot;https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;stride&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-08-27T00:21:37.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23283" PostTypeId="2" ParentId="23276" CreationDate="2020-08-27T07:30:26.420" Score="2" Body="&lt;p&gt;The idea in PPO is that you want to reuse the batch many times to update the current policy. However, you cannot update mindlessly in a regular actor-critic fashion, because your policy might stray too far away from the optimal point.&lt;/p&gt;&#xA;&lt;p&gt;This means you repeat your step 6. &lt;code&gt;epoch&lt;/code&gt; amount of times for the same batch of trajectories. Usually &lt;code&gt;epoch&lt;/code&gt; is somewhere between 3 and 30 but it is a hyper-parameter you need to adjust. For the first &lt;em&gt;repeat&lt;/em&gt;, the old and the new policy are the same, so their ratio should be 1. After the first update, the new probabilities will change due to the updated policy, whereas you will still need to use the old probabilities giving you a ratio different from 1. The old probabilities will stay the same during these &lt;code&gt;epoch&lt;/code&gt; update steps, whereas your new probabilities will keep changing.&lt;/p&gt;&#xA;" OwnerUserId="8448" LastActivityDate="2020-08-27T07:30:26.420" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23286" PostTypeId="1" CreationDate="2020-08-27T10:16:55.680" Score="2" ViewCount="33" Body="&lt;p&gt;Hado van Hasselt, a researcher at DeepMind, mentioned in one of his &lt;a href=&quot;https://www.youtube.com/watch?v=bRfUxQs6xIM&quot; rel=&quot;nofollow noreferrer&quot;&gt;videos&lt;/a&gt; (from 7:20 to 8:20) on Youtube (about policy gradient methods) that there are cases when the policy is very simple compared to the value function - and it makes more sense to learn the policy directly rather than first learning the value function and then doing control.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What are some real-life examples of simple policies but complex value functions?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-08-27T21:25:40.237" LastActivityDate="2020-08-27T21:25:40.237" Title="What's an example of a simple policy but a complex value function?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;policies&gt;&lt;value-functions&gt;&lt;real-world&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23287" PostTypeId="1" AcceptedAnswerId="23294" CreationDate="2020-08-27T10:22:00.807" Score="3" ViewCount="319" Body="&lt;p&gt;A deterministic policy in the rock-paper-scissors game can be easily exploited by the opponent - by doing just the right sequence of moves to defeat the agent. More often than not, I've &lt;em&gt;heard&lt;/em&gt; that &lt;strong&gt;a random policy&lt;/strong&gt; is the &lt;strong&gt;optimal&lt;/strong&gt; policy in this case - but the argument seems a little informal.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please expound on this, possibly adding more mathematical details and intuition? I guess the case I'm referring to is that of a game between &lt;strong&gt;two RL agents&lt;/strong&gt;, but I'd be happy to learn about other cases too. Thanks!&lt;/p&gt;&#xA;&lt;p&gt;EDIT: When would a random policy be optimal in this case?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-08-27T16:02:56.980" LastActivityDate="2020-08-27T16:22:39.593" Title="What's the optimal policy in the rock-paper-scissors game?" Tags="&lt;reinforcement-learning&gt;&lt;game-theory&gt;&lt;optimal-policy&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23288" PostTypeId="1" AcceptedAnswerId="23316" CreationDate="2020-08-27T10:36:32.770" Score="3" ViewCount="76" Body="&lt;p&gt;I came across the following proof of what's commonly referred to as the &lt;strong&gt;log-derivative trick&lt;/strong&gt; in policy-gradient algorithms, and I have a question -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tqATW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tqATW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;While transitioning from the first line to the second, the gradient with respect to policy parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; was pushed into the summation. What bothers me is how it skipped over &lt;span class=&quot;math-container&quot;&gt;$\mu (s)$&lt;/span&gt;, the distribution of states - which (the way I understand it), &lt;strong&gt;is induced by the policy &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt; itself!&lt;/strong&gt; Why then does it not depend on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Let me know what's going wrong! Thank you!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-09-04T00:11:03.963" Title="Why does (not) the distribution of states depend on the policy parameters that induce it?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;proofs&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23289" PostTypeId="1" CreationDate="2020-08-27T10:41:22.080" Score="0" ViewCount="7" Body="&lt;p&gt;This seems like it'd be a common issue in speech to text systems (but I'm not very familiar with the area):&lt;/p&gt;&#xA;&lt;p&gt;If you're speaking long paragraphs, it's hard to discern sentence boundaries (e.g. if I pause in the middle of a sentence)&lt;/p&gt;&#xA;&lt;p&gt;Right now I'm using pauses for sentence boundaries. That causes outputs like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;My friend worked. On a cool hackathon project&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What are some techniques I could use to perhaps post-process this STT output to convert it to the following?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;My friend worked worked on a cool hackathon project.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16637" LastActivityDate="2020-08-27T10:41:22.080" Title="Discerning sentence boundaries in speech-to-text of long paragraphs" Tags="&lt;natural-language-processing&gt;&lt;speech-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23290" PostTypeId="1" AcceptedAnswerId="23376" CreationDate="2020-08-27T11:05:48.977" Score="3" ViewCount="101" Body="&lt;p&gt;I can't seem to understand why we need importance sampling in prioritized experience replay (PER). The authors of the &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; write on page 5:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The estimation of the expected value with stochastic updates relies on those updates corresponding to the same distribution as its expectation. Prioritized replay introduces bias because it changes this distribution in an uncontrolled fashion, and therefore changes the solution that the estimates will converge to (even if the policy and state distribution are fixed).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My understanding of this statement is that sampling non-uniformly from the replay memory is an issue.&lt;/p&gt;&#xA;&lt;p&gt;So, my question is: Since we are working 1-step off-policy, why is it an issue? I thought that in an off-policy setting we don't care how transitions are sampled (at least in the 1-step case).&lt;/p&gt;&#xA;&lt;p&gt;The one possibility for an issue that came to my mind is that in the particular case of PER, we are sampling transitions according to the errors and rewards, which does seem a little fishy.&lt;/p&gt;&#xA;&lt;p&gt;A somewhat related question was asked &lt;a href=&quot;https://datascience.stackexchange.com/questions/32873/prioritized-replay-what-does-importance-sampling-really-do&quot;&gt;here&lt;/a&gt;, but I don't think it answers my question.&lt;/p&gt;&#xA;" OwnerUserId="40603" LastActivityDate="2020-09-01T19:35:35.687" Title="Why is sampling non-uniformly from the replay memory an issue? (Prioritized experience replay)" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;&lt;experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23291" PostTypeId="2" ParentId="23288" CreationDate="2020-08-27T13:12:03.463" Score="3" Body="&lt;p&gt;The reason you are confused is because this is not the full derivation of the Policy Gradient Theorem. You are correct in thinking that &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; depends on the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; which in turn depends on the policy parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, and so there &lt;em&gt;should&lt;/em&gt; be a derivative of &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; wrt &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, however the Policy Gradient Theorem doesn't require you to take this derivative.&lt;/p&gt;&#xA;&lt;p&gt;In fact, the great thing about the Policy Gradient Theorem is that the final result &lt;em&gt;does not&lt;/em&gt; require you to take a derivative of the state distribution with respect to the policy parameters. I would encourage you to read and go through the derivation of the Policy Gradient Theorem from e.g. Sutton and Barto to see why you don't need to take the derivative.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5IN1C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5IN1C.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Above is an image of the Policy Gradient Theorem proof from the Sutton and Barto book. If you carefully go through this line by line you will see that you are not required to take a derivative of the state distribution anywhere in the proof.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-08-28T09:24:18.113" LastActivityDate="2020-08-28T09:24:18.113" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23292" PostTypeId="1" AcceptedAnswerId="23293" CreationDate="2020-08-27T14:17:21.020" Score="3" ViewCount="119" Body="&lt;p&gt;I have a scheduling problem in which there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; slots and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; clients. I am trying to solve the problem using Q-learning so I have made the following state-action model.&lt;/p&gt;&#xA;&lt;p&gt;A state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; is given by the current slot &lt;span class=&quot;math-container&quot;&gt;$t=1,2,\ldots,n$&lt;/span&gt; and an action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; at slot &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is given by one client, &lt;span class=&quot;math-container&quot;&gt;$a_t\in\{1,2,\ldots,m\}$&lt;/span&gt;. In my situation, I do not have any reward associated with a state-action pair &lt;span class=&quot;math-container&quot;&gt;$(s_t,a_t)$&lt;/span&gt; until the terminal state which is the last slot. In other words, for all &lt;span class=&quot;math-container&quot;&gt;$s_t\in\{1,2,\ldots,n-1\}$&lt;/span&gt;, the reward is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and for &lt;span class=&quot;math-container&quot;&gt;$s_t=n$&lt;/span&gt; I can compute the reward given &lt;span class=&quot;math-container&quot;&gt;$(a_1,a_2,\ldots,a_n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In this situation, the Q table, &lt;span class=&quot;math-container&quot;&gt;$Q(s_t,a_t)$&lt;/span&gt;, will contain only zeros except for the last row in which it will contain the updated reward.&lt;/p&gt;&#xA;&lt;p&gt;Can I still apply Q-learning in this situation? Why do I need a Q table if I only use the last row?&lt;/p&gt;&#xA;" OwnerUserId="37642" LastEditorUserId="2444" LastEditDate="2021-01-24T03:36:26.143" LastActivityDate="2021-01-24T03:36:26.143" Title="How to apply Q-learning when rewards is only available at the last state?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;reward-functions&gt;&lt;sparse-rewards&gt;&lt;combinatorial-optimization&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23293" PostTypeId="2" ParentId="23292" CreationDate="2020-08-27T14:59:50.013" Score="2" Body="&lt;p&gt;Having only a non-zero reward at the very end is not uncommon. When rewards are sparse, it becomes a bit harder to learn compared to having lots of different rewards along the way, but for your problem, the goal state is always reached, so that should not be a problem. (The real problem with sparse rewards is that, if an agent can do a lot of exploration without every finding the goal, it essentially receives no feedback and will behave randomly, until it happens to stumble upon the very rare reward state.)&lt;/p&gt;&#xA;&lt;p&gt;What concerns me more about your problem is that the final reward depends not just on the last state visited, but also on the chain of actions taken so far. That means that, to make this a proper MDP, you need to keep the chain of action in the state. So, your state would be something of the type &lt;span class=&quot;math-container&quot;&gt;$(s_k, [a_1, a_2, \ldots, a_{k-1}])$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This kind of combinatorial problem is not what RL is really great at. RL is really good when the state and action together give a lot of information about the next state. Here it seems that, in your formulation, the next state is independent of the previous action.&lt;/p&gt;&#xA;&lt;p&gt;Instead of seeing this as a RL problem, you might want to express this as sequences of actions with an associated reward, and look at it as a combinatorial optimization problem.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-27T14:59:50.013" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23294" PostTypeId="2" ParentId="23287" CreationDate="2020-08-27T16:22:39.593" Score="5" Body="&lt;p&gt;For this, we will need game theory.&lt;/p&gt;&#xA;&lt;p&gt;In game theory, an optimal strategy is one that cannot be exploited by the opponent &lt;em&gt;even if they know your strategy&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Let's say you want a strategy where your move selection is not based on what happened before (so you are not trying to model your opponent, or trick them into believing you will always play scissors and then throw them off, anything like that).&#xA;A strategy will look like &lt;span class=&quot;math-container&quot;&gt;$(P, S, R)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$P, S, R \in [0, 1], P+S+R = 1$&lt;/span&gt;. You select paper with probability &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, scissors with probability &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, rock with probability &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;.&#xA;Now, if your probabilities are a bit uneven (for example &lt;span class=&quot;math-container&quot;&gt;$(0.5, 0.2, 0.3)$&lt;/span&gt;) an opponent can abuse that strategy. If your opponent plays with probabilities &lt;span class=&quot;math-container&quot;&gt;$(p, s, r)$&lt;/span&gt;, their expected reward (counting +1 for win, -1 for loss, 0 for draw) would be &lt;span class=&quot;math-container&quot;&gt;$0.5(s - r) + 0.2(r - p) + 0.3(p - s) = 0.1p + 0.2s - 0.3r$&lt;/span&gt;. If they wish to maximize their wins, they would play scissors all the time against you, and expect to have a distinct advantage over you.&lt;/p&gt;&#xA;&lt;p&gt;In general, for a strategy &lt;span class=&quot;math-container&quot;&gt;$(P, S, R)$&lt;/span&gt; for you and &lt;span class=&quot;math-container&quot;&gt;$(p, s, r)$&lt;/span&gt; for your opponent, your opponent's winnings would be &lt;span class=&quot;math-container&quot;&gt;$P(s - r) + S(r - p) + R(p - s) =  p(R-S) + s(P-R) + r(S - P)$&lt;/span&gt;. If all the partial derivatives of this, with respect to &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; are 0, the opponent has no way to maximize his winnings; they would have no incentive to play a particular move over any other move.&#xA;This occurs when &lt;span class=&quot;math-container&quot;&gt;$P = S = R = \frac13$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That's basically how to approach game theory: find a strategy so your opponent has no incentive to choose one action over another. The approach seems a bit counter-intuitive at first (you're trying to find the optimal strategy for your opponent instead of for yourself) but it works for many similar problems.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-27T16:22:39.593" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23295" PostTypeId="2" ParentId="19873" CreationDate="2020-08-27T16:41:24.263" Score="2" Body="&lt;p&gt;Yes, it should be no problem.&lt;/p&gt;&#xA;&lt;p&gt;When you decide to use a CNN, you have to make sure that this makes sense. Another answer mentioned using &lt;code&gt;3x3&lt;/code&gt; convolutions -- which I would recommend &lt;em&gt;against&lt;/em&gt;. For that to work, you would need to turn your vector into a rectangular array, and you would be implying a structure that isn't there.&lt;/p&gt;&#xA;&lt;p&gt;Use one-dimensional convolutions instead.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-27T16:41:24.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23296" PostTypeId="1" CreationDate="2020-08-27T17:30:01.857" Score="6" ViewCount="241" Body="&lt;p&gt;Are there any examples of single player games that use modern ML technique in its games? By this I mean AI that plays with or against the human player, and not just play the game by itself (like Atari).&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Modern ML techniques&amp;quot; is a vague term, but for example, Neural Networks, Reinforcement Learning, or probabilistic methods. Basically anything that goes above and beyond traditional search methods that most games use nowadays.&lt;/p&gt;&#xA;&lt;p&gt;Ideally, the AI would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;widely available (i.e. not like the OpenAI Five, which was only available for a limited amount of time and requires a high amount of computational power)&lt;/li&gt;&#xA;&lt;li&gt;human level (not overpowered)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Ideally, the game would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;symmetrical (the AI has the same agent capabilities as the player, though answers similar to &lt;a href=&quot;https://left4dead.fandom.com/wiki/The_Director&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Director&lt;/a&gt; would be very interesting as well)&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;complex environment&amp;quot; (more complex than, say, a board game, but a CIV5 game might work)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But any answer would be appreciated, as some of the criteria above are quite vauge.&lt;/p&gt;&#xA;&lt;p&gt;Edit: the ideal cases listed above are not meant to discourage other answers, nor are they intended to be of strictly inclusionary (ie: any game would need to satisfy all of the above requirements)&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-08-29T23:07:50.643" LastActivityDate="2020-09-03T15:23:39.340" Title="Examples of single player games that use modern ML techniques in the AI?" Tags="&lt;game-ai&gt;&lt;reference-request&gt;" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23297" PostTypeId="2" ParentId="23292" CreationDate="2020-08-27T17:47:51.427" Score="1" Body="&lt;p&gt;If by,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I can compute the reward given &lt;span class=&quot;math-container&quot;&gt;$(a_1, a_2, \dots, a_n)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;you simply mean that your game is deterministic, this is absolutely fine. I feel another answer had assumed you were implying your terminal reward is a matter of some sequence. RL does, however, struggle more greatly in games with indeterminable reward until the terminal state, however, it is perfectly normal, but a significant challenge.&lt;/p&gt;&#xA;&lt;p&gt;In terms of implementation, simply record the reward at the end of each match, and perform a train step only after each terminal state, assigning this reward to each transition recorded in that game. Instead of updating your target network after a given number of steps, update it after some number of terminal states. I suggest 20 games as a starter for the frequency of updating your target network.&lt;/p&gt;&#xA;" OwnerUserId="40575" LastActivityDate="2020-08-27T17:47:51.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23298" PostTypeId="2" ParentId="23296" CreationDate="2020-08-27T17:56:05.330" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1702.06230.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beating the World’s Best at Super Smash Bros. Melee with Deep Reinforcement&#xA;Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Firoiu, Whitney, Tenenbaum created a RL agent that plays and defeats professional players in Super Smash Bros Melee. The RL agent first played against the built-in AI, and then via self-play.&lt;/p&gt;&#xA;&lt;p&gt;Only one character playing on a single stage was trained. The character picked (Captain Falcon) has no &amp;quot;projectile attacks&amp;quot; to simplify training.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastEditorUserId="6779" LastEditDate="2020-08-30T04:18:21.740" LastActivityDate="2020-08-30T04:18:21.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23299" PostTypeId="1" AcceptedAnswerId="23310" CreationDate="2020-08-27T19:39:51.300" Score="1" ViewCount="52" Body="&lt;p&gt;I am training an RL agent with Deep Q-learning + Experience Replay on the Q*bert Atari environment. After 400,000 frames, my agent appears to have learned strategic information about the game, but none about the environment. It has learned that a good immediate strategy is to simply jump down both diagonals and fall of the board, thus completing a large portion of the first level. However, it remains to understand neither the boundaries of the board to prevent jumping off, nor anything about avoiding enemies. I’m asking this here, instead of Stack Overflow because it is a more general question with less of a need in terms of programming understanding. Simply, I am asking whether or not this is a matter of a pore exploration policy (which I presume). If you agree, what should be a better exploration policy for Q*bert that would facilitate my agent’s learning experience?&lt;/p&gt;&#xA;&lt;p&gt;As per the request of a comment:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Could you add what your current exploration approach is, and what options you are using for your Deep Q Learning implementation (e.g. replay size, batch size, NN architecture, steps per target network copy, or if you are using a different update mechanism for the target network). Also if you are using any other approach different to the classic DQN paper such as in state representation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here are my parameters:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Exploration policy: epsilon = &lt;code&gt;min(1.0, 1000 / (frames + 1))&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Replay Memory = 20,000 frames&lt;/li&gt;&#xA;&lt;li&gt;Batch size = 32 transitions&lt;/li&gt;&#xA;&lt;li&gt;NN architecture: Conv2D(64, 3, 2), Dropout(0.2), Dense(32, relu), Dense(32, relu), Dense(num_actions, linear)&lt;/li&gt;&#xA;&lt;li&gt;Steps per target network copy: 100&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40575" LastEditorUserId="40575" LastEditDate="2020-08-28T13:22:44.683" LastActivityDate="2020-08-28T13:22:44.683" Title="What is the optimal exploration-exploitation trade-off in Q*bert?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23300" PostTypeId="1" CreationDate="2020-08-27T20:29:35.593" Score="2" ViewCount="57" Body="&lt;p&gt;In the average reward setting the quality of a policy is defined as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ r(\pi) = \lim_{h\to\infty}\frac{1}{h} \sum_{j=1}^{h}E[R_j] $$&lt;/span&gt;&#xA;When we reach the steady state distribution we can write the above equation as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ r(\pi) = \lim_{t\to\infty}E[R_t | A \sim \pi] $$&lt;/span&gt;&#xA;We can use the incremental update method to find &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ r(\pi) = \frac{1}{t} \sum_{j=1}^{t} R_j = \bar R_{t-1} + \beta (R_t - \bar R_{t-1})$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$ \bar R_{t-1}$&lt;/span&gt; is the estimate of the average reward &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt; at timestep &lt;span class=&quot;math-container&quot;&gt;$t-1$&lt;/span&gt;.&#xA;We use this incremental update rule in the SARSA algorithm:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BFJor.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BFJor.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, In this above algorithm, we can see that the policy will change with respect to time. But to calculate the &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt;, the agent should follow the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; for a long period of time. Then how we are using &lt;span class=&quot;math-container&quot;&gt;$r(\pi)$&lt;/span&gt; if the policy changes with respect to time?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-08-27T21:17:20.497" Title="How we are calculating average reward ($r(\pi)$) if the policy changes over time?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23301" PostTypeId="2" ParentId="23300" CreationDate="2020-08-27T21:17:20.497" Score="1" Body="&lt;p&gt;You are correct: to evaluate a policy, we need to fix it.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We can &lt;em&gt;temporarily&lt;/em&gt; fix it, just to evaluate it over a number of test cases. For a fair comparison, we should fix the start states and random seeds used for the transitions.&lt;/li&gt;&#xA;&lt;li&gt;We can wait until convergence / until we are satisfied. The resulting policy would be what we implement in the &amp;quot;true&amp;quot;, trained agent. This is important when exploration might be harmful in the &amp;quot;real world&amp;quot; domain where the agent will be operating.&lt;/li&gt;&#xA;&lt;li&gt;We can also measure average reward of the &amp;quot;non-stationary&amp;quot; policy, and assume that, once the agent is doing well, this should be close enough to evaluating the fixed policy. This is not ideal, but on the other hand it is trivial to implement, and is often used to track the learning process. If you have a life-long learning agent, this might be the best you can do.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-27T21:17:20.497" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23302" PostTypeId="2" ParentId="23254" CreationDate="2020-08-27T23:33:19.387" Score="1" Body="&lt;p&gt;There is no special calculation you can do to determine the optimal batch size for any situation, so you kinda have to do a bit of testing to determine what batch size will work best. But there are some common trends you can take into account to make your testing easier.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;How to choose your batch size&lt;/h1&gt;&#xA;&lt;p&gt;According to the paper &lt;a href=&quot;https://arxiv.org/pdf/1803.02811.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accelerated Methods for Deep Reinforcement Learning&lt;/a&gt; you get the best performance from DQNs (on average) with a batch size of 512. The problem with this is that is is much slower than the usual batch size of 32 and most of the time the performance improvement doesn't warrant it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://storage.googleapis.com/groundai-web-prod/media%2Fusers%2Fuser_14%2Fproject_256499%2Fimages%2Fdqn_catdqn_scale.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://storage.googleapis.com/groundai-web-prod/media%2Fusers%2Fuser_14%2Fproject_256499%2Fimages%2Fdqn_catdqn_scale.png&quot; alt=&quot;Performance vs. Batch Size&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you are just trying to test out your agents it is generally best to stick with a batch size of 32 or 64 so that you can train the agent quickly yet still get an idea of what it is capable of. But if getting the best performance is your top priority and waiting longer isn't a problem, then you should go for a batch size of 512 (higher can actually lead to worse performance) or something near that.&lt;/p&gt;&#xA;" OwnerUserId="38062" LastActivityDate="2020-08-27T23:33:19.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23304" PostTypeId="2" ParentId="18659" CreationDate="2020-08-28T03:11:12.750" Score="-2" Body="&lt;p&gt;To begin from the scratch, and in order to keep approach simple we have to analyze the input text(clinical narration) for the following data:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is input a word or a group of words or a sentence?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is input a meaningful sentence? By meaningful, I mean grammatically correct.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the word, group of words or a sentence contain symptoms or health issues?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the sentence contain data about a person’s age and gender?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the sentence contain data about a person’s diet, medical history, work routine,  travelling history or getting in contact with&#xA;any ill person?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If there are any other attributes that one has to look for then I would be keen to find out from the subject matter experts.&lt;/p&gt;&#xA;" OwnerUserId="34306" LastActivityDate="2020-08-28T03:11:12.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23305" PostTypeId="1" AcceptedAnswerId="23306" CreationDate="2020-08-28T04:26:44.007" Score="1" ViewCount="47" Body="&lt;p&gt;Like making a bed, washing dishes, taking out the garbage, etc., by training it on the video of specific individuals doing those cores in their own unique environments?&lt;/p&gt;&#xA;&lt;p&gt;I have researched what machine learning is capable of doing at this point in time, and it seems this may be now feasible when done on a customer-specific basis and enable by an A.I. enhanced, full articulated, robot along the lines of an enhanced InMoov. &lt;a href=&quot;https://en.wikipedia.org/wiki/InMoov&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/InMoov&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If it's feasible, what are the AI algorithms I should be considering to train my robot to do these tasks? Isn't deep learning the most promising of these selections:  &lt;a href=&quot;https://www.ubuntupit.com/machine-learning-algorithms-for-both-newbies-and-professionals/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ubuntupit.com/machine-learning-algorithms-for-both-newbies-and-professionals/&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="40288" LastEditorUserId="2444" LastEditDate="2020-08-29T00:57:12.420" LastActivityDate="2020-08-29T00:57:12.420" Title="Is it feasible using today's technology to use an AI training algorithm to custom teach a robot to do common household cores?" Tags="&lt;neural-networks&gt;&lt;robotics&gt;&lt;state-of-the-art&gt;&lt;humanoid-robots&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23306" PostTypeId="2" ParentId="23305" CreationDate="2020-08-28T05:03:58.003" Score="1" Body="&lt;p&gt;I would suggest using a neural network with back propagation. From what I know, they can be applied to many different circumstances and work well. For your more simpler and repetitive tasks like moving an object, you can just use simpler regression methods.&lt;/p&gt;&#xA;" OwnerUserId="40622" LastEditorUserId="40622" LastEditDate="2020-08-28T05:21:41.140" LastActivityDate="2020-08-28T05:21:41.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23307" PostTypeId="1" AcceptedAnswerId="23309" CreationDate="2020-08-28T06:37:45.787" Score="0" ViewCount="25" Body="&lt;p&gt;I am new in machine learning and learning linear regression concept. Please help with answers to below queries.&#xA;I want to understand effect on existing independent variable(X1) if I add a new independent variable(X2) in my model.&#xA;This new variable is highly correlated with dependent variable(Y)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Will it have any effect on beta coefficient of X1?&lt;/li&gt;&#xA;&lt;li&gt;Will relationship between X1 and Y become insignificant?&lt;/li&gt;&#xA;&lt;li&gt;Can adjusted R-square value decrease?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40626" LastActivityDate="2020-08-28T08:44:11.633" Title="Effect of adding an Independent Variable in Multiple Linear Regression" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23308" PostTypeId="2" ParentId="18659" CreationDate="2020-08-28T07:21:24.637" Score="-1" Body="&lt;p&gt;So for Medical Prognosis, there are some variables that commonly come up like Age, Sex, Ascites, Hepato, Spider, Status of the disease and many others but it depends on the disease. You'll commonly encounter these variables if you're doing &lt;strong&gt;regression&lt;/strong&gt; or &lt;strong&gt;classification&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Also, if you're reading Radiology Reports for getting the input for the model then you also have to take care of jargons. The same symptoms can be written in various ways but all point towards the same prognosis i.e., there can be &lt;strong&gt;synonyms for labels&lt;/strong&gt;. Try reading &lt;a href=&quot;https://arxiv.org/pdf/1901.07031.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; to get more information on how we can do information extraction from Radiology Reports. This is the famous &lt;strong&gt;CheXpert&lt;/strong&gt; paper&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-08-28T07:21:24.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23309" PostTypeId="2" ParentId="23307" CreationDate="2020-08-28T08:44:11.633" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;Yes, the addition of the independent variable (&lt;span class=&quot;math-container&quot;&gt;$X_2$&lt;/span&gt;) will have an impact on the beta coefficient of &lt;span class=&quot;math-container&quot;&gt;$X_1$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Maybe, it depends on the relationship between &lt;span class=&quot;math-container&quot;&gt;$X_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, and whether you are using any regularization or not.&lt;/li&gt;&#xA;&lt;li&gt;If the independent variable &lt;span class=&quot;math-container&quot;&gt;$X_2$&lt;/span&gt; is more correlated than &lt;span class=&quot;math-container&quot;&gt;$X_1$&lt;/span&gt;, then the value of &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt; should be higher (but I am not sure on this)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31016" LastActivityDate="2020-08-28T08:44:11.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23310" PostTypeId="2" ParentId="23299" CreationDate="2020-08-28T10:28:08.677" Score="1" Body="&lt;p&gt;I can spot three, maybe four, things in your implementation that could be contributing to incomplete learning that you are observing.&lt;/p&gt;&#xA;&lt;h2&gt;More exploration in long term&lt;/h2&gt;&#xA;&lt;p&gt;I think you have correctly identified that exploration could be an issue. In off-policy learning (which Q-learning is an instance of), it is usual to set a minimum exploration rate. It is a hyperparameter that you need to manage. Set too high, the agent will never experience the best rewards as it will make too many mistakes. Set too low, the agent will not explore enough to find the correct alternative actions when the opportunity to learn them occurs.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest for you something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epsilon = max(min(1.0, 1000 / (frames + 1)), 0.01)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can choose numbers other than 0.01, but I think that is a reasonable start for many Atari games. You could try higher, up to 0.1 in games which are more forgiving of mistakes.&lt;/p&gt;&#xA;&lt;h2&gt;Remove dropout&lt;/h2&gt;&#xA;&lt;p&gt;I am not sure why, but I always have problems with dropout in RL neural networks. Try removing the dropout layer.&lt;/p&gt;&#xA;&lt;h2&gt;More convolutional layers&lt;/h2&gt;&#xA;&lt;p&gt;Convolutional layers are very efficient generalisers for vision and grid-based problems. You won't really benefit much from having a single layer though. I would add another two, increase the number of output channels.&lt;/p&gt;&#xA;&lt;h2&gt;Maybe state representation?&lt;/h2&gt;&#xA;&lt;p&gt;It is not clear from your description whether you are using a single colour frame for the state representation, or stacked greyscale frames for the last 3 inputs. It should be the latter, and if you want to more closely replicate the orginal DQN Atari paper, you should take the previous 4 frames as input.&lt;/p&gt;&#xA;&lt;p&gt;In addition, you should be normalising the input into range &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt;. The native image range &lt;span class=&quot;math-container&quot;&gt;$[0,255]$&lt;/span&gt; is tricky for neural networks to process, and quite common for value functions to get stuck if you don't normalise.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-08-28T10:28:08.677" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23311" PostTypeId="2" ParentId="23280" CreationDate="2020-08-28T14:49:55.860" Score="1" Body="&lt;p&gt;If you're using a discount factor less than 1, you should be able to compute a maximum return (likewise, a minimum return) based on the max (min) reward you can earn at each timestep. However, this issue you bring up is usually cited as a difficulty with C51. I think people tend to simply use fixed values for the min/max return (or just make rough estimates). If you want to avoid this, I recommend looking into the QR-DQN algorithm which circumvents this issue altogether and is more theoretically sound.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-08-28T14:49:55.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23312" PostTypeId="1" CreationDate="2020-08-28T15:19:45.613" Score="3" ViewCount="196" Body="&lt;p&gt;I was going through Sutton's book and, using sample-based learning for estimating the expectations, we have this formula&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{new estimate} = \text{old estimate} + \alpha(\text{target} - \text{old estimate})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I don't quite understand is why it's called the target, because since it's the sample, it’s not the actual target value, so why are we moving towards a wrong value?&lt;/p&gt;&#xA;" OwnerUserId="40049" LastEditorUserId="2444" LastEditDate="2020-08-28T19:13:57.993" LastActivityDate="2020-08-28T19:40:28.757" Title="Why is the target called &quot;target&quot; in Monte Carlo and TD learning if it is not the true target?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23314" PostTypeId="2" ParentId="20993" CreationDate="2020-08-28T15:49:42.957" Score="0" Body="&lt;p&gt;The original work on NEAT(Neuroevolution of augmenting topologies) was by Ken Stanley in 2002 at The University of Texas at Austin. The web page for the project is &lt;a href=&quot;http://nn.cs.utexas.edu/?stanley:ec02&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; I suggest you download and read the paper linked from that page. As for selection of genome pairs, NEAT makes use of a speciation model so  the selection of such pairs is constrained to at least prefer pairs from the same 'species', on the assumption that the species has evolved such that species population is isolated under reproduction. The innovation that has been 'bred' into the species is thus preserved under reproduction.  Selection by fitness alone is insufficient in such models. This differs from the simple GA where pair selection is unconstrained.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastActivityDate="2020-08-28T15:49:42.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23315" PostTypeId="2" ParentId="23241" CreationDate="2020-08-28T15:50:55.973" Score="1" Body="&lt;p&gt;I depends on your overall model architecture (and problem specification). As I understand it, you take the observations of all agents together and feed it into one model, a central controller, which then predicts the action per available agent.&lt;/p&gt;&#xA;&lt;p&gt;I believe that this varying number of applicable observations (depending on the number of currently present agents) is what you mean by &lt;em&gt;changing state spaces&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;One option that works in these kinds of settings is to use padding. So, a fixed number of individual observations gets predetermined before training, being set to the average number of agents expected to be present in the environment at any timestep.&lt;/p&gt;&#xA;&lt;p&gt;Then, during training, local, individual observations of all agents (for the current time step) get stacked together, resulting in an overall observation being fed into central controller.&lt;/p&gt;&#xA;&lt;p&gt;Then, the controller predicts (concurrently/in a single pass) one action (or multiple, depends on what you want it to predict per agent...) per agent.&lt;/p&gt;&#xA;&lt;p&gt;In the case that the number of present agents currently (=in a given time step) is smaller than the predetermined number of expected agents, then padding is used to fill the empty observation slots by zero-pseudo-observations. The quality of the predicted actions is consequently only assessed for the present agents, neglecting predictions for absent agents, indicated by their empty observation slots.&#xA;If the number of controlled agents happens to exceed the expected maximal number of agents present in the simulation, excess agents are selected at random and randomly controlled for the next time step. Then, this (randomized control of a random selection of some agents) continues until no excess agents are present in the simulation anymore. Alternatively, if available, other reasonable controllers might be used to temporarily steer excess agents.&lt;/p&gt;&#xA;&lt;p&gt;I have seen researchers doing this in the context of traffic control, where a variable number of cars was controlled by a central controller.&#xA;See &lt;a href=&quot;http://proceedings.mlr.press/v87/vinitsky18a/vinitsky18a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (Section 3.2) for an example of the aforementioned technique.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, there exist approaches where each agent gets a periodically updated local copy of some central controller and then contributes to jointly updating the central controller. &lt;a href=&quot;http://proceedings.mlr.press/v80/liang18b/liang18b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; shortly summarizes a lot of different options and possibilities to choose from in terms of different RL algorithms, where different options might apply depending on the concrete task at hand. Also, the paper discusses different options with respect to algorithms implemented in the python Reinforcement Learning (RL) library RLlib, which might already provide a suitable implementation for a class of algorithm you want to work with.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, a lot of papers don't reveal explicitly whether their (partly) distributed RL training procedures are more suitable, i.e. converge stably to a good solution, for single-agent environments or equally suitable for multi-agent environments as well.&lt;/p&gt;&#xA;&lt;p&gt;When it comes to introducing new agents to the simulation, this depends entirely on your concrete problem at hand. In the context of vehicle control, you could position a vehicle at the start of the simulated road, steer it (safely) a few time steps straight ahead (to initialize its observation space) and then hand over control to the RL controller. For other types of agents (e.g. soccer playing robots etc.) that might look entirely different. In that case, you might have them sitting or standing next to the field and then have them enter the play ground or so. So, that really depends on your task at hand.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-08-29T18:28:29.923" LastActivityDate="2020-08-29T18:28:29.923" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="23316" PostTypeId="2" ParentId="23288" CreationDate="2020-08-28T16:36:40.527" Score="2" Body="&lt;p&gt;The proof you are given in the above post is not wrong. It's just they skip some of the steps and directly written the final answer. Let me go through those steps:&lt;/p&gt;&#xA;&lt;p&gt;I will simplify some of the things to avoid complication but the generosity remains the same. Like I will think of the reward as only dependent on the current state, &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, and current action, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$r = r(s,a)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;First, we will define the average reward as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$r(\pi) = \sum_s \mu(s)\sum_a \pi(a|s)\sum_{s^{\prime}} P_{ss'}^{a} r $$&lt;/span&gt;&#xA;We can further simplify average reward as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$r(\pi) = \sum_s \mu(s)\sum_a \pi(a|s)r(s,a) $$&lt;/span&gt;&#xA;My notation may be slightly different than the aforementioned slides since I'm only following Sutton's book on RL. Our objective function is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ J(\theta) = r(\pi) $$&lt;/span&gt;&#xA;We want to prove that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \nabla_{\theta} J(\theta) = \nabla_{\theta}r(\pi) = \sum_s \mu(s) \sum_a \nabla_{\theta}\pi(a|s) Q(s,a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now let's start the proof:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}V(s) = \nabla_{\theta} \sum_{a} \pi(a|s) Q(s,a)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}V(s) = \sum_{a} [Q(s,a) \nabla_{\theta} \pi(a|s)  + \pi(a|s) \nabla_{\theta}Q(s,a)]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}V(s) = \sum_{a} [Q(s,a) \nabla_{\theta} \pi(a|s)  + \pi(a|s) \nabla_{\theta}[R(s,a) - r(\pi) + \sum_{s^{\prime}}P_{ss^{\prime}}^{a}V(s^{\prime})]]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}V(s) = \sum_{a} [Q(s,a) \nabla_{\theta} \pi(a|s)  + \pi(a|s) [- \nabla_{\theta}r(\pi) + \sum_{s^{\prime}}P_{ss^{\prime}}^{a}\nabla_{\theta}V(s^{\prime})]]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}V(s) = \sum_{a} [Q(s,a) \nabla_{\theta} \pi(a|s)  + \pi(a|s) \sum_{s^{\prime}}P_{ss^{\prime}}^{a}\nabla_{\theta}V(s^{\prime})] - \nabla_{\theta}r(\pi)\sum_{a}\pi(a|s)$$&lt;/span&gt;&#xA;Now we will rearrange this:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}r(\pi) = \sum_{a} [Q(s,a) \nabla_{\theta} \pi(a|s)  + \pi(a|s) \sum_{s^{\prime}}P_{ss^{\prime}}^{a}\nabla_{\theta}V(s^{\prime})] - \nabla_{\theta}V(s)$$&lt;/span&gt;&#xA;Multiplying both sides by &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; and summing over &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}r(\pi) \sum_{s}\mu(s)= \sum_{s}\mu(s) \sum_{a} Q(s,a) \nabla_{\theta} \pi(a|s)  + \sum_{s}\mu(s) \sum_a \pi(a|s) \sum_{s^{\prime}}P_{ss^{\prime}}^{a}\nabla_{\theta}V(s^{\prime}) - \sum_{s}\mu(s) \nabla_{\theta}V(s)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}r(\pi) = \sum_{s}\mu(s) \sum_{a} Q(s,a) \nabla_{\theta} \pi(a|s)  + \sum_{s^{\prime}}\mu(s^{\prime})\nabla_{\theta}V(s^{\prime}) - \sum_{s}\mu(s) \nabla_{\theta}V(s)$$&lt;/span&gt;&#xA;Now we are there:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{\theta}r(\pi) = \sum_{s}\mu(s) \sum_{a} Q(s,a) \nabla_{\theta} \pi(a|s)$$&lt;/span&gt;&#xA;This is the policy gradient theoram for average reward formulation (ref. &lt;a href=&quot;https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Policy gradient&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="35585" LastEditDate="2020-09-04T00:11:03.963" LastActivityDate="2020-09-04T00:11:03.963" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="23317" PostTypeId="1" AcceptedAnswerId="23319" CreationDate="2020-08-28T17:00:40.643" Score="2" ViewCount="48" Body="&lt;p&gt;I am trying to train an agent to explore an unknown two-dimensional map while avoiding circular obstacles (with varying radii). The agent has control over its steering angle and its speed. The steering angle and speed are normalized in a &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt; range, where the sign encodes direction (i.e. a speed of &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; means that it is going backwards at the maximum units/second).&lt;/p&gt;&#xA;&lt;p&gt;I am familiar with similar problems where the agent must navigate to a waypoint, and in which case the reward is the successful arrival to the target position. But, in my case, I can't really reward the agent for that, since there is no direct 'goal'.&lt;/p&gt;&#xA;&lt;h2&gt;What I have tried&lt;/h2&gt;&#xA;&lt;p&gt;The agent is penalised when it hits an obstacle; however, I am not sure how to motivate the agent to move. Initially, I was thinking of having the agent always move forward, meaning that it only has control over the steering angle. But, I want the ability for the agent to control its speed and be able to reverse (since I'm trying to model a car).&lt;/p&gt;&#xA;&lt;p&gt;What I have tried is to reward the agent for moving and to penalise it for remaining stationary. At every timestep, the agent is rewarded &lt;span class=&quot;math-container&quot;&gt;${1}/{t_\text{max}}$&lt;/span&gt; if the absolute value of the speed is above some epsilon, or penalised that same amount if otherwise. But, as expected, this doesn't work. Rather than motivating the agent to move, it simply causes it to jitter back and forth. This makes sense since 'technically' the most optimal strategy if you want to avoid obstacles is to remain stationary. If the agent can't do that then the next best thing is to make small adjustements in the position.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So my question: how can I add in an exploration incentive to my agent?&lt;/strong&gt; I am using proximal policy optimization (PPO).&lt;/p&gt;&#xA;" OwnerUserId="40635" LastEditorUserId="2444" LastEditDate="2020-10-07T22:50:24.900" LastActivityDate="2020-10-07T22:50:24.900" Title="How do I design the rewards and penalties for an agent whose goal it is to explore a map" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;reward-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23318" PostTypeId="1" CreationDate="2020-08-28T17:22:58.937" Score="0" ViewCount="47" Body="&lt;p&gt;I have a question about generalized advantage estimation in multi agent environment. According to the formula I've computed GAE in this way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for t in reversed(range(len(rewards))):&#xA;    delta = rewards[t] + self.discount_factor * state_estimated_value[t + 1] * not_dones[t] - \&#xA;                        state_estimated_value[t]&#xA;    gaes[t] = future_gae = delta + self.discount_factor * self.lmbda * not_dones[t] * future_gae&#xA;    returns.insert(0, gaes[t] + state_estimated_value[t])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm working in a multi-agent environment. Let's consider an agent which, at the end of episode, it doesn't reach its goal: in this case its done = False. Do I have to impose done = True at the end of the episode, even for agents who have not reached their goal?&lt;/p&gt;&#xA;" OwnerUserId="40636" LastEditorUserId="40583" LastEditDate="2020-08-30T15:51:08.543" LastActivityDate="2020-08-30T15:51:08.543" Title="Generalized advantage estimation DRL PPO" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23319" PostTypeId="2" ParentId="23317" CreationDate="2020-08-28T17:39:45.303" Score="1" Body="&lt;p&gt;Measure what you want to achieve as directly as possible, and reward that. Later you can add more sophisticated incentives for the type of motion etc, but the key to a good reward signal is that it measures the quality of a solution at a high level, without specifying how to achieve that solution.&lt;/p&gt;&#xA;&lt;p&gt;If you want your simulated car to explore, you will want to give it a reward signal based on it encountering new unexplored areas. There are lots of reasonable choices here. I suspect a good one will depend on what sensors you can reasonably code for the car, and what you consider to count as exploration - e.g. is it a thorough search of an area, moving far from the original position, experiencing different &amp;quot;views&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;One likely component you will need to give your agent and incorporate into the state representation is a memory. In order to understand whether the agent is exploring, something will need to know whether the agent has experienced something before and how much. A very simple kind of memory would be to add counters to a grid map and allow the agent to know how many time steps it has spent in each position on the map. The reward signal can then be higher when the agent enters a point on the map that it has not been in recently. If you want a non-episodic or repeating tour of exploration you might decay the values over time, so that an area that has not been visited for a long time counts the same as a non-visited one.&lt;/p&gt;&#xA;&lt;p&gt;A related concept that you might be able to borrow ideas from is curiousity. There have been some interesting attempts to encourage an agent to seek new/interesting states by modifying action selection. &lt;a href=&quot;https://pathak22.github.io/noreward-rl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Curiosity-driven Exploration by&#xA;Self-supervised Prediction&lt;/a&gt; is one such attempt, and might be of interest to you. In that case, the authors use an intrinsic model of curiousity that can solve some environments even when there is no external reward signal at all!&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, if you don't care to get involved in technical solution, you could create a maybe acceptable behaviour for your vehicle by setting a random goal position, then granting a reward and moving it to a new random location each time the car reaches it.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-28T17:45:08.830" LastActivityDate="2020-08-28T17:45:08.830" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23320" PostTypeId="2" ParentId="23312" CreationDate="2020-08-28T17:46:53.627" Score="5" Body="&lt;p&gt;It is our &amp;quot;current&amp;quot; target. We assume that the value we get now is at least a closer approximation to the &amp;quot;true&amp;quot; target.&lt;/p&gt;&#xA;&lt;p&gt;We're not so much moving towards a wrong value as we are moving away from a more wrong value.&lt;/p&gt;&#xA;&lt;p&gt;Of course, it is all base on random trials, so saying anything definite (such as: &amp;quot;we are guaranteed to improve at each step&amp;quot;) is hard to show without working probabilistically. The &lt;em&gt;expectation&lt;/em&gt; of the error of the value function (as compared to the true value function) will decrease, that is all we can say.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-08-28T17:46:53.627" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23321" PostTypeId="2" ParentId="23312" CreationDate="2020-08-28T18:10:20.437" Score="0" Body="&lt;p&gt;It would be heplful for me if you specify the section and page number of the Sutton's book.&#xA;But as far as I understand your question I will try explain this.&#xA;Think of TD update. The sample contains &lt;span class=&quot;math-container&quot;&gt;$(s_t,a_t,r_{t+1},s_{t+1})$&lt;/span&gt;. Using incremental update we can write:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ v_{t}(s) = \frac{1}{t} \sum_{j=1}^{t}(r_{t+1} + \gamma v_{s_{t+1}})$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ v_{t}(s) = v_{t-1}(s) + \alpha (r_{t+1} + \gamma v_{t-1}(s_{t+1}) - v_{t-1}(s_t))$$&lt;/span&gt;&#xA;We are calling this &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma v_{t-1}(s_{t+1})$&lt;/span&gt; as the TD target. From the above equation you can already see that &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma v_{t-1}(s_{t+1})$&lt;/span&gt; is actually the unbaised estimate for &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;. We are calling &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma v_{t-1}(s_{t+1})$&lt;/span&gt; an unbiased estimate since &lt;span class=&quot;math-container&quot;&gt;$E[r_{t+1} + \gamma v_{t-1}(s_{t+1})] = v_t(s_t)$&lt;/span&gt;. That means expectation over &lt;span class=&quot;math-container&quot;&gt;$r_{t+1} + \gamma v_{t-1}(s_{t+1})$&lt;/span&gt; lead us to true state value function, &lt;span class=&quot;math-container&quot;&gt;$v_t(s)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For the monte carlo update same explain will be applied. I hope that this answer your question.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-08-28T19:40:28.757" LastActivityDate="2020-08-28T19:40:28.757" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="23322" PostTypeId="1" CreationDate="2020-08-28T19:25:14.637" Score="1" ViewCount="113" Body="&lt;p&gt;I was going through &lt;a href=&quot;https://www.cse.iitb.ac.in/%7Eshivaram/teaching/cs747-a2020/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; course on reinforcement learning (the course has two lecture videos and corresponding slides) and I had a doubt. On slide 18 of &lt;a href=&quot;https://drive.google.com/file/d/1iAQC4mW0itfrfxPPg_0by64Lx1tk-LI4/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; pdf, it states following condition for an algorithm to have regret sublinear in T (T being number of pulls of multi arm bandit).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;C2 - Greedy in the Limit:&lt;/strong&gt; Let exploit(T) denote the number of pulls that are that are greedy w.r.t the empirical mean up to horizon &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;. For sub-linear regret, we need&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{T\rightarrow\infty}\frac{\mathbb{E}[exploit(T)]}{T}=1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$exploit(T)$&lt;/span&gt; denote the total number of &amp;quot;exploit&amp;quot; rounds performed in the first &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; pulls. Given that expectation is defined as &amp;quot;the weighted sum of the outcome values, where the weights correspond to the probabilities of realizing that value&amp;quot;,&lt;/p&gt;&#xA;&lt;p&gt;(&lt;strong&gt;Q1&lt;/strong&gt;) how exactly mathematically we define &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[exploit(T)]$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;In second video (at 24:44), instructor has said that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[exploit(T)]$&lt;/span&gt; is the number of exploit steps.&lt;/p&gt;&#xA;&lt;p&gt;(&lt;strong&gt;Q2&lt;/strong&gt;) Then how it equals &amp;quot;weighted sum of outcome values&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;(note that instructor assumes that the pulling of arm may give reward which correspond to outcome value of 1 and may not give reward which correspond to ourcome value of 0)&lt;/p&gt;&#xA;&lt;p&gt;Also in slide 27, for GLIE-ifying &lt;span class=&quot;math-container&quot;&gt;$\epsilon_T$&lt;/span&gt;-first strategy, he selects &lt;span class=&quot;math-container&quot;&gt;$\epsilon_T=\frac{1}{\sqrt{T}}$&lt;/span&gt;. Then, the instructor counts &lt;span class=&quot;math-container&quot;&gt;$\sqrt{T}$&lt;/span&gt; exploratory pulls and &lt;span class=&quot;math-container&quot;&gt;$T-\sqrt{T}$&lt;/span&gt; exploitory pulls. Then to show that this satisfies condition C2, instructor states &lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[exploit(T)]\geq \frac{T-\sqrt{T}}{T}$$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$\frac{T-\sqrt{T}}{T}$&lt;/span&gt; is a fraction of exploitory pulls.&lt;/p&gt;&#xA;&lt;p&gt;(&lt;strong&gt;Q3&lt;/strong&gt;) So, by above equation does the instructor mean, number of exploitory pulls is greater than equal to fraction of number of exploitory pulls?&lt;/p&gt;&#xA;&lt;p&gt;(&lt;strong&gt;Q4&lt;/strong&gt;) How can we put 2nd equation in first equation and still prove limit in first equation still holds, that is, how following is the case:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{T=\rightarrow\infty}\frac{\frac{T-\sqrt{T}}{T}}{T}=1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I guess I am missing some basic concept of expectation here.&lt;/p&gt;&#xA;" OwnerUserId="40640" LastEditorUserId="36821" LastEditDate="2021-02-21T15:46:04.287" LastActivityDate="2021-02-21T15:46:04.287" Title="Understanding GLIE conditions for epsilon greedy approach" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23327" PostTypeId="2" ParentId="13741" CreationDate="2020-08-29T14:00:54.067" Score="0" Body="&lt;p&gt;LSTM can be tricky, I'll give my $0.02.&lt;/p&gt;&#xA;&lt;p&gt;LSTM input layer defines the shape so it would be something like this.&lt;/p&gt;&#xA;&lt;p&gt;If I am understanding your question correctly, your data can be framed as 184 samples with 2 time steps and 70 features?&lt;/p&gt;&#xA;&lt;p&gt;So the start of the code might look like this.&lt;/p&gt;&#xA;&lt;p&gt;model = Sequential()&#xA;model.add(LSTM(184, input_shape=(50, 2)))&lt;/p&gt;&#xA;" OwnerUserId="34095" LastActivityDate="2020-08-29T14:00:54.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23328" PostTypeId="1" AcceptedAnswerId="23358" CreationDate="2020-08-29T14:09:13.053" Score="3" ViewCount="87" Body="&lt;p&gt;I am training an agent to do object avoidance. The agent has control over its steering angle and its speed. The steering angle and speed are normalized in a &lt;span class=&quot;math-container&quot;&gt;$[−1,1]$&lt;/span&gt; range, where the sign encodes direction (i.e. a speed of −1 means that it is going backwards at the maximum units/second).&lt;/p&gt;&#xA;&lt;p&gt;My reward function penalises the agent for colliding with an obstacle and rewards it for moving away from its starting position. At a time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the reward, &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;, is defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_t=&#xA;\begin{cases}&#xA;r_{\text{collision}},&amp;amp;\text{if collides,}\\&#xA;\lambda^d\left(\|\mathbf{p}^{x,y}_t-\mathbf{p}_0^{x,y}\|_2-\|\mathbf{p}_{t-1}^{x,y}-\mathbf{p}_0^{x,y}\|_2 \right),&amp;amp;\text{otherwise,}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\lambda_d$&lt;/span&gt; is a scaling factor and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}_t$&lt;/span&gt; gives the pose of the agent at a time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. The idea being that we should reward the agent for moving away from the inital position (and in a sense 'exploring' the map—I'm not sure if this is a good way of incentivizing exploration but I digress).&lt;/p&gt;&#xA;&lt;p&gt;My environment is an unknown two-dimensional map that contains circular obstacles (with varying radii). And the agent is equipped with a sensor that measures the distance to nearby obstacles (similar to a 2D LiDAR sensor). The figure below shows the environment along with the agent.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OScwx.png?s=256&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OScwx.png?s=256&quot; alt=&quot;environment&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since I'm trying to model a car, I want the agent to be able to go forward and reverse; however, when training, the agent's movement is very jerky. It quickly switches between going forward (positive speed) and reversing (negative speed). &lt;a href=&quot;https://puu.sh/Gn1yO/fbf333421a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is what I'm talking about.&lt;/p&gt;&#xA;&lt;p&gt;One idea I had was to penalise the agent when it reverses. While that did significantly reduce the jittery behaviour, it also caused the agent to collide into obstacles on purpose. In fact, over time, the average episode length decreased. I think this is the agent's response to the reverse penalties. Negative rewards incentivize the agent to reach a terminal point as fast as possible. In our case, the only terminal point is obstacle collision.&lt;/p&gt;&#xA;&lt;p&gt;So then I tried rewarding the agent for going forward instead of penalising it for reversing, but that did not seem to do much. &lt;strong&gt;Evidently, I don't think trying to correct the jerky behaviour directly through rewards is the proper approach. But I'm also not sure how I can do it any other way.&lt;/strong&gt; Maybe I just need to rethink what my reward signal wants the agent to achieve?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I rework the reward function to have the agent move around the map, covering as much distance as possible, while also maintaining smooth movement?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="40635" LastEditorUserId="40635" LastEditDate="2020-08-30T15:00:18.167" LastActivityDate="2020-08-31T15:21:31.330" Title="How can I fix jerky movement in a continuous action space" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;deep-rl&gt;&lt;rewards&gt;&lt;reward-shaping&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23329" PostTypeId="1" CreationDate="2020-08-29T14:50:51.470" Score="1" ViewCount="15" Body="&lt;p&gt;i am trying to train an A3C algorithm but I am getting same output in the multinomial function.&lt;/p&gt;&#xA;&lt;p&gt;can I train the A3C with random actions as in below code.&lt;/p&gt;&#xA;&lt;p&gt;can someone expert comment.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while count&amp;lt;max_timesteps-1:&#xA;            value, action_values, (hx, cx) = model((Variable(state.unsqueeze(0)), (hx, cx)))&#xA;            prob = F.softmax(action_values,dim = -1)&#xA;            log_prob = F.log_softmax(action_values, dim=-1)&#xA;            print(log_prob.shape)&#xA;            print(&amp;quot;log_prob: &amp;quot;,log_prob)&#xA;            entropy = -(log_prob * prob).sum(1, keepdim=True)&#xA;            entropies.append(entropy)&#xA;            actn = np.random.randn(3)&#xA;            action = actn.argmax()&#xA;            log_prob = log_prob[0,action]&#xA;            # print(&amp;quot;log_prob &amp;quot;,log_prob)&#xA;            # print(&amp;quot;action &amp;quot;,action)&#xA;            state, reward, done = env.step(action)&#xA;            done = (done or count == max_timesteps-2)&#xA;            reward = max(min(reward, 1), -1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40051" LastActivityDate="2020-08-29T14:50:51.470" Title="is it ok to take random actions while training a3c as in below code" Tags="&lt;reinforcement-learning&gt;&lt;pytorch&gt;&lt;a3c&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23330" PostTypeId="1" AcceptedAnswerId="23333" CreationDate="2020-08-29T15:55:25.767" Score="4" ViewCount="82" Body="&lt;p&gt;I am trying to make a classifier.&lt;/p&gt;&#xA;&lt;p&gt;I am new to AI (even if I know the definition and all such a bit) , and also I have no idea of how to implement it properly by myself even if I know a bit of Python coding (in fact, I am fifteen years old !), but my passion for this has made me ask this (silly, probably) question.&lt;/p&gt;&#xA;&lt;p&gt;Are there neural networks where nodes are randomly selected from among a set of nodes (in random orders and a random number of times)? I know this is from ML (or maybe deep learning, I suppose), but I have no idea how to recognize such a thing from the presently available algorithms. It will be great if you all could help me, because I am preparing to release an API for programming a model which I call the 'Insane Mind' on GitHub, and I want some help to know if my effort was fruitless.&lt;/p&gt;&#xA;&lt;p&gt;And for reference, here's the code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from math import *&#xA;from random import *&#xA; &#xA;class MachineError(Exception):&#xA;    '''standard exception in the API'''&#xA;    def __init__(self, stmt):&#xA;        self.stmt = stmt&#xA;def sig(x):&#xA;    '''Sigmoid function'''&#xA;    return (exp(x) + 1)/exp(x)&#xA;&#xA;class Graviton:&#xA;    def __init__(self, weight, marker):&#xA;        '''Basic unit in 'Insane Mind' algorithm&#xA;           -------------------------------------&#xA;           Graviton simply refers to a node in the algorithm.&#xA;           I call it graviton because of the fact that it applies a weight&#xA;           on the input to transform it, besides using the logistic function '''&#xA;        self.weight = weight # Weight factor of the graviton&#xA;        self.marker = marker # Marker to help in sorting&#xA;        self.input = 0 # Input to the graviton&#xA;        self.output = 0 # Output of the graviton&#xA;        self.derivative = 0 # Derivative of the output&#xA;&#xA;    def process(self, input_to_machine):&#xA;        '''processes the input (a bit of this is copied from the backprop algorithm'''&#xA;        self.input = input_to_machine&#xA;        self.output = (sig(self.weight * self.input) - 1)/(self.marker + 1)&#xA;        self.derivative = (sig(self.input * self.weight) - 1) * self.input *self.output * (1- self.output) &#xA;        return self.output&#xA;    &#xA;    def get_derivative_at_input(self):&#xA;        '''returns the derivative of the output'''&#xA;        return self.derivative&#xA;&#xA;    def correct_self(self, learning_rate, error):&#xA;        '''edits the weight'''&#xA;        self.weight += -1 * error * learning_rate * self.get_derivative_at_input() * self.weight&#xA;        &#xA;class Insane_Mind:&#xA;&#xA;    def __init__(self, number_of_nodes):&#xA;        '''initialiser for Insane_Mind class.&#xA;           arguments : number_of_nodes : the number of nodes you want in the model'''&#xA;        self.system = [Graviton(random(),i) for i in range(number_of_nodes)] # the actual system&#xA;        self.system_size = number_of_nodes # number of nodes , or 'system size'&#xA;        &#xA;    def  output_sys(self, input_to_sys):&#xA;        '''system output'''&#xA;        self.output = input_to_sys&#xA;        for i in range(self.system_size):&#xA;            self.output = self.system[randint(0,self.system_size - 1 )].process(self.output)&#xA;        return self.output&#xA;    &#xA;    def train(self, learning_rate, wanted):&#xA;        '''trains the system'''&#xA;        self.cloned = [] # an array to keep the sorted elements during the sorting process below&#xA;        order = [] # the array to make out the order of arranging the nodes&#xA;        temp = {} # a temporary dictionary to pick the nodes from&#xA;        for graviton in self.system:&#xA;            temp.update({str(graviton.derivative): graviton.marker})&#xA;        order = sorted(temp)&#xA;        i = 0&#xA;        error = wanted - self.output&#xA;        for value in order:&#xA;            self.cloned.append(self.system[temp[value]])&#xA;            self.cloned[i].correct_self(learning_rate, error)&#xA;            error *= self.cloned[i].derivative&#xA;            i += 1&#xA;        self.system = self.cloned&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sorry for not using that &lt;code&gt;MachineError&lt;/code&gt; exception anywhere in my code (I will use it when I am able to deploy this API).&lt;/p&gt;&#xA;&lt;p&gt;To tell more about this algorithm, this gives randomized outputs (as if guessing). The number of guesses vary from 1 (for a system with one node), 2 (for two nodes) and so on to an infinite number of guesses for an infinite number of nodes.&lt;/p&gt;&#xA;&lt;p&gt;Also, I wanna try and find how much it can be of use (if this is something that has never been discovered, if it is something that can find a good place in the world of ML or Deep Learning) and where it can be used.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;Criticisms (with a clear reason) are also accepted.&lt;/p&gt;&#xA;" OwnerUserId="40583" LastEditorUserId="1847" LastEditDate="2020-08-29T18:12:36.670" LastActivityDate="2020-08-30T10:33:18.080" Title="Are there neural networks where nodes are randomly selected from among a set of nodes (in random orders and a random number of times)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23331" PostTypeId="1" AcceptedAnswerId="23344" CreationDate="2020-08-29T16:21:13.770" Score="6" ViewCount="107" Body="&lt;p&gt;Consider a neural network, e.g. as presented by Nielsen &lt;a href=&quot;http://neuralnetworksanddeeplearning.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Abstractly, we just construct some function &lt;span class=&quot;math-container&quot;&gt;$f: \mathbb{R}^n \to [0,1]^m$&lt;/span&gt; for some &lt;span class=&quot;math-container&quot;&gt;$n,m \in \mathbb{N}$&lt;/span&gt; (i.e. the dimensions of the input and output space) that depends on a large set of parameters, &lt;span class=&quot;math-container&quot;&gt;$p_j$&lt;/span&gt;. We then just define the cost function &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; and calculate &lt;span class=&quot;math-container&quot;&gt;$\nabla_p C$&lt;/span&gt; and just map &lt;span class=&quot;math-container&quot;&gt;$p \to p - \epsilon \nabla_p C$&lt;/span&gt; repeatedly.&lt;/p&gt;&#xA;&lt;p&gt;The question is why do we choose &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; to be what it is in standard neural networks, e.g. a bunch of linear combinations and sigmoids? One answer is that there a theorem saying any suitably nice function can be approximated using neural networks. But the same is true of other types of functions &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The Stone-Weierstrass theorem gives that we could use polynomials in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; variables: &lt;span class=&quot;math-container&quot;&gt;$$f(x) =  c^0_0 + (c^1_1 x_1 + c^1_2 x_2 + \cdots + c^1_n x_n) + (c^2_{11}x_1 x_1 + c^2_{12} x_1x_2 + \cdots + c^2_{1n} x_1 x_2 + c^2_{21} x_2x_1 + c^2_{22} x_2x_2 + \cdots) + \cdots,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and still have a nice approximation theorem. Here the gradient would be even easier to calculate. Why not use polynomials?&lt;/p&gt;&#xA;" OwnerUserId="40653" LastEditorUserId="2444" LastEditDate="2020-08-30T02:34:26.040" LastActivityDate="2021-01-21T23:38:42.803" Title="Why are neural networks preferred to other classification functions optimized by gradient decent" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23332" PostTypeId="1" CreationDate="2020-08-29T16:36:47.383" Score="2" ViewCount="251" Body="&lt;p&gt;I've been looking into self-attention lately, and in the articles that I've been seeing, they all talk about &amp;quot;weights&amp;quot; in attention. My understanding is that the weights in self-attention are not the same as the weights in a neural network.&lt;/p&gt;&#xA;&lt;p&gt;From this article, &lt;a href=&quot;http://peterbloem.nl/blog/transformers&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://peterbloem.nl/blog/transformers&lt;/a&gt;, in the additional tricks section, it mentions,&lt;/p&gt;&#xA;&lt;p&gt;The query is the dot product of the query weight matrix and the word vector,&#xA;&lt;code&gt;ie, q = W(q)x&lt;/code&gt; and the key is the dot product of the key weight matrix and the word vector, &lt;code&gt;k = W(k)x&lt;/code&gt; and similarly for the value it is &lt;code&gt;v = W(v)x&lt;/code&gt;. So my question is, where do the weight matrices come from?&lt;/p&gt;&#xA;" OwnerUserId="33579" LastEditorUserId="2444" LastEditDate="2020-08-30T00:58:43.297" LastActivityDate="2021-05-28T16:04:50.467" Title="What is the weight matrix in self-attention?" Tags="&lt;neural-networks&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23333" PostTypeId="2" ParentId="23330" CreationDate="2020-08-29T18:29:46.463" Score="2" Body="&lt;p&gt;It is difficult to prove a negative, but I do not think there is any classifier (neural network or otherwise) that fully matches to your idea.&lt;/p&gt;&#xA;&lt;p&gt;I suspect that you will not be able to take the idea of random connections and loops at run time, and make a useful classifier out of it. That's not to say the idea is completely without merit, sometimes it is good to explore blue sky ideas and just see what happens. However, I think it might be a frustrating excercise to build anything on top of your idea without some basic foundation work first. I recommend that you look into the &lt;a href=&quot;https://towardsdatascience.com/introduction-to-logistic-regression-66248243c148&quot; rel=&quot;nofollow noreferrer&quot;&gt;theory and implementation of logistic regression&lt;/a&gt; as a starting point, which is a good stepping stone to understanding neural networks.&lt;/p&gt;&#xA;&lt;p&gt;There are some neural network components and architectures that make use of random behaviour at the activation level:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.cs.toronto.edu/%7Ehinton/absps/JMLRdropout.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dropout&lt;/a&gt;. This is a method used during training which zeroes outputs from randomly selected neurons. It often gives an effective boost to neural network stability (acting to prevent overfitting to input data) and can improve accuracy of classifiers too due to behaving similarly to having multiple simpler classifiers.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boltzmann machines&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;restricted Boltzmann machines (RBMs)&lt;/a&gt; output 0 or 1 randomly from each &amp;quot;neuron&amp;quot; unit, with the probability decided by sum of inputs. They are used to create &lt;em&gt;generative&lt;/em&gt; models, not classifiers though. Another difference is that the randomness is applied both during training and during inference, whilst dropout is most often applied to augment training. Early on in the days of deep learning, RBMs were used to pre-train layers in a deep neural network. This was effective, but other simpler methods were discovered later and are nowadays preferred in most cases.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A variant of dropout call called &lt;a href=&quot;https://arxiv.org/pdf/1506.02142.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo dropout&lt;/a&gt; is used at inference time. This can be used to measure &lt;em&gt;uncertainty&lt;/em&gt; in a model's individual predictions, which is otherwise hard to obtain.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Although not quite as freeform as your random connections on a per neuron basis. If you &lt;a href=&quot;https://medium.com/@bingobee01/a-review-of-dropout-as-applied-to-rnns-72e79ecd5b7b&quot; rel=&quot;nofollow noreferrer&quot;&gt;applied dropout to a recurrent neural network&lt;/a&gt;, that might be quite close to your idea, because the existence of loops between neurons in each time step would be random. This could be applied in language modelling and classifiers for sequence data. The same motivations apply here as for dropout in simpler feed forward classifiers - it can in theory make a classifier more robust against noise in the inputs and more accurate.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-08-30T10:33:18.080" LastActivityDate="2020-08-30T10:33:18.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23334" PostTypeId="1" CreationDate="2020-08-29T21:56:28.467" Score="2" ViewCount="30" Body="&lt;p&gt;I don't get how the training of the RPN works. From the forward propagation, I have &lt;span class=&quot;math-container&quot;&gt;$W \times H \times k$&lt;/span&gt; outputs from the RPN.&lt;/p&gt;&#xA;&lt;p&gt;How is the training data labeled such that I can use the loss function and update the weights through bach propagation?  Is the training data labeled in the same shape of the output, as there are &lt;span class=&quot;math-container&quot;&gt;$W \times H \times k$&lt;/span&gt; anchor boxes and we use the loss function directly or what?&lt;/p&gt;&#xA;" OwnerUserId="40659" LastEditorUserId="2444" LastEditDate="2020-08-30T01:02:10.647" LastActivityDate="2021-04-26T20:09:44.497" Title="How is the data labelled in order to train a region proposal network?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;r-cnn&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23335" PostTypeId="1" CreationDate="2020-08-29T23:09:31.493" Score="2" ViewCount="224" Body="&lt;p&gt;In general, how do I calculate the GPU memory need to run a deep learning network?&lt;/p&gt;&#xA;&lt;p&gt;I'm asking this question because my training for some network configuration is getting out of memory.&lt;/p&gt;&#xA;&lt;p&gt;If the TensorFlow only store the memory necessary to the tunable parameters, and if I have around 8 million, I supposed the RAM required will be:&lt;/p&gt;&#xA;&lt;p&gt;RAM = 8.000.000 * (8 (float64)) / 1.000.000 (scaling to MB)&lt;/p&gt;&#xA;&lt;p&gt;RAM = 64 MB, right?&lt;/p&gt;&#xA;&lt;p&gt;The TensorFlow requires more memory to store the image at each layer?&lt;/p&gt;&#xA;&lt;p&gt;By the way, these are my GPU Specifications:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Nvidia GeForce 1050 4GB&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Networking topology&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Unet&lt;/li&gt;&#xA;&lt;li&gt;Input Shape (256,256,4)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;Model: &amp;quot;functional_1&amp;quot;&#xA;__________________________________________________________________________________________________&#xA;Layer (type)                    Output Shape         Param #     Connected to                     &#xA;==================================================================================================&#xA;input_1 (InputLayer)            [(None, 256, 256, 4) 0                                            &#xA;__________________________________________________________________________________________________&#xA;conv2d (Conv2D)                 (None, 256, 256, 64) 2368        input_1[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;dropout (Dropout)               (None, 256, 256, 64) 0           conv2d[0][0]                     &#xA;__________________________________________________________________________________________________&#xA;conv2d_1 (Conv2D)               (None, 256, 256, 64) 36928       dropout[0][0]                    &#xA;__________________________________________________________________________________________________&#xA;max_pooling2d (MaxPooling2D)    (None, 128, 128, 64) 0           conv2d_1[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_2 (Conv2D)               (None, 128, 128, 128 73856       max_pooling2d[0][0]              &#xA;__________________________________________________________________________________________________&#xA;dropout_1 (Dropout)             (None, 128, 128, 128 0           conv2d_2[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_3 (Conv2D)               (None, 128, 128, 128 147584      dropout_1[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;max_pooling2d_1 (MaxPooling2D)  (None, 64, 64, 128)  0           conv2d_3[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_4 (Conv2D)               (None, 64, 64, 256)  295168      max_pooling2d_1[0][0]            &#xA;__________________________________________________________________________________________________&#xA;dropout_2 (Dropout)             (None, 64, 64, 256)  0           conv2d_4[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_5 (Conv2D)               (None, 64, 64, 256)  590080      dropout_2[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;max_pooling2d_2 (MaxPooling2D)  (None, 32, 32, 256)  0           conv2d_5[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_6 (Conv2D)               (None, 32, 32, 512)  1180160     max_pooling2d_2[0][0]            &#xA;__________________________________________________________________________________________________&#xA;dropout_3 (Dropout)             (None, 32, 32, 512)  0           conv2d_6[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_7 (Conv2D)               (None, 32, 32, 512)  2359808     dropout_3[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_transpose (Conv2DTranspo (None, 64, 64, 256)  524544      conv2d_7[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;concatenate (Concatenate)       (None, 64, 64, 512)  0           conv2d_transpose[0][0]           &#xA;                                                                 conv2d_5[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_8 (Conv2D)               (None, 64, 64, 256)  1179904     concatenate[0][0]                &#xA;__________________________________________________________________________________________________&#xA;dropout_4 (Dropout)             (None, 64, 64, 256)  0           conv2d_8[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_9 (Conv2D)               (None, 64, 64, 256)  590080      dropout_4[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_transpose_1 (Conv2DTrans (None, 128, 128, 128 131200      conv2d_9[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;concatenate_1 (Concatenate)     (None, 128, 128, 256 0           conv2d_transpose_1[0][0]         &#xA;                                                                 conv2d_3[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_10 (Conv2D)              (None, 128, 128, 128 295040      concatenate_1[0][0]              &#xA;__________________________________________________________________________________________________&#xA;dropout_5 (Dropout)             (None, 128, 128, 128 0           conv2d_10[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_11 (Conv2D)              (None, 128, 128, 128 147584      dropout_5[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_transpose_2 (Conv2DTrans (None, 256, 256, 64) 32832       conv2d_11[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;concatenate_2 (Concatenate)     (None, 256, 256, 128 0           conv2d_transpose_2[0][0]         &#xA;                                                                 conv2d_1[0][0]                   &#xA;__________________________________________________________________________________________________&#xA;conv2d_12 (Conv2D)              (None, 256, 256, 64) 73792       concatenate_2[0][0]              &#xA;__________________________________________________________________________________________________&#xA;dropout_6 (Dropout)             (None, 256, 256, 64) 0           conv2d_12[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_13 (Conv2D)              (None, 256, 256, 64) 36928       dropout_6[0][0]                  &#xA;__________________________________________________________________________________________________&#xA;conv2d_14 (Conv2D)              (None, 256, 256, 1)  65          conv2d_13[0][0]                  &#xA;==================================================================================================&#xA;Total params: 7,697,921&#xA;Trainable params: 7,697,921&#xA;Non-trainable params: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;This is the error given.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---------------------------------------------------------------------------&#xA;ResourceExhaustedError                    Traceback (most recent call last)&#xA;&amp;lt;ipython-input-17-d4852b86b8c1&amp;gt; in &amp;lt;module&amp;gt;&#xA;     23 # Train the model, doing validation at the end of each epoch.&#xA;     24 epochs = 30&#xA;---&amp;gt; 25 result_model = model.fit(train_gen, epochs=epochs, validation_data=val_gen, callbacks=callbacks)&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py in _method_wrapper(self, *args, **kwargs)&#xA;    106   def _method_wrapper(self, *args, **kwargs):&#xA;    107     if not self._in_multi_worker_mode():  # pylint: disable=protected-access&#xA;--&amp;gt; 108       return method(self, *args, **kwargs)&#xA;    109 &#xA;    110     # Running inside `run_distribute_coordinator` already.&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)&#xA;   1096                 batch_size=batch_size):&#xA;   1097               callbacks.on_train_batch_begin(step)&#xA;-&amp;gt; 1098               tmp_logs = train_function(iterator)&#xA;   1099               if data_handler.should_sync:&#xA;   1100                 context.async_wait()&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\def_function.py in __call__(self, *args, **kwds)&#xA;    778       else:&#xA;    779         compiler = &amp;quot;nonXla&amp;quot;&#xA;--&amp;gt; 780         result = self._call(*args, **kwds)&#xA;    781 &#xA;    782       new_tracing_count = self._get_tracing_count()&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\def_function.py in _call(self, *args, **kwds)&#xA;    838         # Lifting succeeded, so variables are initialized and we can run the&#xA;    839         # stateless function.&#xA;--&amp;gt; 840         return self._stateless_fn(*args, **kwds)&#xA;    841     else:&#xA;    842       canon_args, canon_kwds = \&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py in __call__(self, *args, **kwargs)&#xA;   2827     with self._lock:&#xA;   2828       graph_function, args, kwargs = self._maybe_define_function(args, kwargs)&#xA;-&amp;gt; 2829     return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access&#xA;   2830 &#xA;   2831   @property&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py in _filtered_call(self, args, kwargs, cancellation_manager)&#xA;   1846                            resource_variable_ops.BaseResourceVariable))],&#xA;   1847         captured_inputs=self.captured_inputs,&#xA;-&amp;gt; 1848         cancellation_manager=cancellation_manager)&#xA;   1849 &#xA;   1850   def _call_flat(self, args, captured_inputs, cancellation_manager=None):&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py in _call_flat(self, args, captured_inputs, cancellation_manager)&#xA;   1922       # No tape is watching; skip to running the function.&#xA;   1923       return self._build_call_outputs(self._inference_function.call(&#xA;-&amp;gt; 1924           ctx, args, cancellation_manager=cancellation_manager))&#xA;   1925     forward_backward = self._select_forward_and_backward_functions(&#xA;   1926         args,&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py in call(self, ctx, args, cancellation_manager)&#xA;    548               inputs=args,&#xA;    549               attrs=attrs,&#xA;--&amp;gt; 550               ctx=ctx)&#xA;    551         else:&#xA;    552           outputs = execute.execute_with_cancellation(&#xA;&#xA;~\Anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)&#xA;     58     ctx.ensure_initialized()&#xA;     59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,&#xA;---&amp;gt; 60                                         inputs, attrs, num_outputs)&#xA;     61   except core._NotOkStatusException as e:&#xA;     62     if name is not None:&#xA;&#xA;ResourceExhaustedError:  OOM when allocating tensor with shape[8,64,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc&#xA;     [[node gradient_tape/functional_1/conv2d_14/Conv2D/Conv2DBackpropInput (defined at &amp;lt;ipython-input-17-d4852b86b8c1&amp;gt;:25) ]]&#xA;Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.&#xA; [Op:__inference_train_function_17207]&#xA;&#xA;Function call stack:&#xA;train_function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any type of mistake in the network definition? How could I improve the network to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="40662" LastEditorUserId="2444" LastEditDate="2020-09-07T12:37:00.783" LastActivityDate="2021-03-11T06:02:46.980" Title="How to calculate the GPU memory need to run a deep learning network?" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;training&gt;&lt;memory&gt;" AnswerCount="3" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23336" PostTypeId="2" ParentId="23296" CreationDate="2020-08-30T00:43:52.517" Score="5" Body="&lt;p&gt;There is &lt;a href=&quot;https://ai.googleblog.com/2019/06/introducing-google-research-football.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Research Football&lt;/a&gt;, which is an open-source platform to develop reinforcement learning algorithms to play a game similar to FIFA or PES, although the football simulation is not as realistic as the current versions of FIFA or PES. You can play this game against different RL agents (e.g. DQN or IMPALA) and, of course, you can even develop your own RL agents and play against them. &lt;a href=&quot;https://www.youtube.com/watch?v=F8DcgFDT9sc&amp;amp;feature=emb_title&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a video that illustrates the environment. &lt;a href=&quot;https://github.com/google-research/football&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is the code and instructions to use it&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, there isn't yet an AI that plays simulated football at a human-level (i.e. as good as the best human players). For example, I can regularly (although not always) beat the legendary level-AI at FIFA, but I also don't know the details about this AI (which could also be rule-based).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-30T13:18:41.313" LastActivityDate="2020-08-30T13:18:41.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23337" PostTypeId="1" CreationDate="2020-08-30T02:58:15.380" Score="1" ViewCount="189" Body="&lt;p&gt;I am having a difficult time trying to find out if spacy lists the gender or for that matter the inflection of any word in German.  The software was started in Berlin so I find it very difficult to believe that this would not be the case. When you input the word 'gender' into &lt;a href=&quot;https://spacy.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spacy.io/&lt;/a&gt; search box, nothing comes back.  When you input the search into google 'gender' site:&lt;a href=&quot;https://spacy.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spacy.io/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You actually get back the result that several other languages do have something like gender in their algorithm, for example, Spanish has&lt;/p&gt;&#xA;&lt;p&gt;ADJ , ADJ__AdpType=Prep , ADJ__AdpType=Preppron|Gender=Masc|Number=Sing , ADJ__AdvType=Tim , ADJ__Degree=Abs|Gender=Fem|Number=Plur&lt;/p&gt;&#xA;&lt;p&gt;The most detailed info on German at spacy found here &lt;a href=&quot;https://spacy.io/models/de&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spacy.io/models/de&lt;/a&gt; is very brief with very little info.&lt;/p&gt;&#xA;&lt;p&gt;When I parse the sentence:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;'Wir glauben, daß diese Vorgänge nur empirisch-psychologisch&#xA;untersucht werden können und mit Logik wenig zu tun haben.'&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;And I look at the noun 'Vorgänge' I would expect to see something that looks like it is a plural noun and perhaps also nominative but I only see the following in my namespace (of course the namespace is larger than this but you get the idea):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lWQJx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lWQJx.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GJaaP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GJaaP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38616" LastActivityDate="2021-05-28T13:01:30.273" Title="How do you find the gender of a noun in German using spacy" Tags="&lt;natural-language-processing&gt;&lt;spacy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23338" PostTypeId="1" CreationDate="2020-08-30T07:50:51.177" Score="1" ViewCount="27" Body="&lt;p&gt;I've been reading non-local neural networks as explained in &lt;a href=&quot;https://arxiv.org/pdf/1711.07971&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt;. My understanding is that they solve the restrained reception of local filters. I see how they are different from convolutions and fully connected networks.&lt;/p&gt;&#xA;&lt;p&gt;How do they relate to attention (specifically self-attention)? How do they integrate this attention?&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="2444" LastEditDate="2020-09-07T12:38:57.153" LastActivityDate="2020-09-07T12:38:57.153" Title="How do non-local neural networks relate to attention and self-attention?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;papers&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23339" PostTypeId="5" CreationDate="2020-08-30T12:26:00.583" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-30T12:26:00.583" LastActivityDate="2020-08-30T12:26:00.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23340" PostTypeId="4" CreationDate="2020-08-30T12:26:00.583" Score="0" Body="For questions about machine learning algorithms that learn in batches of data rather than one example at a time (i.e. online learning). Batch learning can also be called offline learning and it is the common way of training machine learning models." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-08-30T12:26:00.583" LastActivityDate="2020-08-30T12:26:00.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23341" PostTypeId="1" CreationDate="2020-08-30T13:59:54.970" Score="1" ViewCount="31" Body="&lt;p&gt;I was reading a paper &lt;a href=&quot;https://arxiv.org/pdf/2006.09066.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-Agent Reinforcement Learning for Adaptive&#xA;User Association in Dynamic mmWave Networks&lt;/a&gt; and I was stuck understanding the deep neural network architecture that was used. The authors gave it in Fig. 3 (on top of page 6) and they state the following (on page 9):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This architecture comprises 2 multi-layers perceptron (MLP) of 32 hidden units, one RNN layer (a long short memory term - LSTM) layer with 64 memory cells followed by another 2 MLPs of 32 hidden units. The network then branches off in two MLPs of 16 hidden units to construct the duelling network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;According to Fig. 3 there is one MLP, one RNN and one MLP. So why the authors said 2 MLPs?&lt;/p&gt;&#xA;&lt;p&gt;Assuming it is 2 MLPs, does this mean we have 2 hidden layers of 32 neurons each? So, at the end we will have:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;one input layer - one hidden layer with 32 neurons - another hidden layer with 32 neurons - one RNN layer with 64 cells - one hidden layer with 32 neurons - another hidden layer with 32 neurons - one hidden layer with 16 neurons - another hidden layer with 16 neurons - one output layer.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="37642" LastEditorUserId="2444" LastEditDate="2020-08-30T14:44:57.527" LastActivityDate="2020-08-30T14:44:57.527" Title="How to understand this NN architecture?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-neural-networks&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23342" PostTypeId="1" CreationDate="2020-08-30T15:19:18.823" Score="0" ViewCount="33" Body="&lt;p&gt;I am trying to create an A3C but it is giving same action for all the states during the training.&lt;/p&gt;&#xA;&lt;p&gt;The same action is also not an obvious way to maximise the reward.&lt;/p&gt;&#xA;&lt;p&gt;However I donot know if normal a3c with multinomail sampling of actions can be used for continuous state space.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions for best RL algo to be used for continuous state space and discrete action action space.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        while count&amp;lt;max_timesteps-1:&#xA;            value, action_values, (hx, cx) = model((Variable(state.unsqueeze(0)), (hx, cx)))&#xA;            prob = F.softmax(action_values,dim = -1)&#xA;            log_prob = F.log_softmax(action_values, dim=-1)&#xA;            entropy = -(log_prob * prob).sum(1, keepdim=True)&#xA;            entropies.append(entropy)&#xA;            cdist = categorical.Categorical(prob)&#xA;            action = cdist.sample()&#xA;            log_prob = log_prob[0, Variable(action)].data&#xA;            state, reward, done = env.step(action)&#xA;            done = (done or count == max_timesteps-2)&#xA;            reward = max(min(reward, 1), -1)&#xA;            &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40051" LastEditorUserId="40051" LastEditDate="2020-08-30T15:26:39.547" LastActivityDate="2020-08-30T15:26:39.547" Title="Does normal A3C works well for continuous state space?" Tags="&lt;reinforcement-learning&gt;&lt;a3c&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23343" PostTypeId="1" CreationDate="2020-08-30T16:18:46.797" Score="0" ViewCount="21" Body="&lt;p&gt;I have camera feed (in the form of RGB images) from 3 cameras with partially overlapping Field-of-view i.e. for the time stamp 0 to 100, I have total 300 frames or say synchronized 100 RGB frames for each camera.&lt;/p&gt;&#xA;&lt;p&gt;Apart from this, the only available information is the Intrinsic Matrix which is same for all the cameras.&lt;/p&gt;&#xA;&lt;p&gt;So, using these RGB frames from 3 cameras, how can I calibrate (extrinsic calibration) 3 cameras and calculate the global 3-DOF pose of an object (Robot - which is moving from one place to another)? For the object pose, assume a global coordinate system is defined by the first camera pose. (I might have access to the robot dimension: length, breadth, &amp;amp; height!)&lt;/p&gt;&#xA;&lt;p&gt;If any direct method is not possible then I'm also open to using machine learning based method(s)?&lt;/p&gt;&#xA;&lt;p&gt;Note: this question is an extension of my other question: &lt;a href=&quot;https://robotics.stackexchange.com/questions/21084/how-can-i-calibrate-3-cameras-without-knowing-global-pose-of-the-object-camera&quot;&gt;How can I calibrate 3 cameras without knowing global pose of the object &amp;amp; camera locations? How can I find the pose of each camera wrt the first one?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40524" LastActivityDate="2020-08-30T16:18:46.797" Title="How can I calibrate 3 cameras and track the object using only synchronized cameras feeds from all the cameras?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;linear-algebra&gt;&lt;pose-estimation&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23344" PostTypeId="2" ParentId="23331" CreationDate="2020-08-30T17:02:05.430" Score="3" Body="&lt;p&gt;You can indeed fit a polynomial to your labelled data, which is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Polynomial_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;polynomial regression&lt;/a&gt; (which can e.g. be done with the function &lt;a href=&quot;https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;numpy.polyfit&lt;/code&gt;&lt;/a&gt;). One apparent limitation of polynomial regression is that, in practice, you need to assume that your data follows some specific polynomial of some degree &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, i.e. you assume that your data has the form of the polynomial that you choose, which may not be true.&lt;/p&gt;&#xA;&lt;p&gt;When you use a neural network to solve a classification or regression problem, you also need to choose the activation functions, the number of neurons, how they are connected, etc., so you also need to limit the number and type of functions that you can learn with neural networks, i.e. the &lt;em&gt;hypothesis space&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, it is not necessarily a bad thing to limit the hypothesis space. In fact, &lt;em&gt;learning&lt;/em&gt; is generally an &lt;em&gt;ill-posed problem&lt;/em&gt;, i.e. in simple terms, there could be multiple solutions or no solutions at all (&lt;a href=&quot;https://www.mit.edu/%7E9.520/spring12/slides/class02/class02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;and other problems&lt;/a&gt;), so, actually, you often need to limit the hypothesis space to find some useful solutions (e.g. solutions that generalise better to unseen data). &lt;em&gt;Regularisations techniques&lt;/em&gt; are ways of constraining the learning problem, and the hypothesis space (i.e. the set of functions that your learning algorithm can choose from), and thus making the learning problem well-posed.&lt;/p&gt;&#xA;&lt;p&gt;Neural networks are not preferred over polynomial regression because they are theoretically more powerful. In fact, both can approximate any continuous function [&lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/BF02551274.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], but these are just theoretical results, i.e. these results do not give you the magical formula to choose the most appropriate neural network or polynomial that best approximates the desired unknown function.&lt;/p&gt;&#xA;&lt;p&gt;In practice, neural networks have been proven to &lt;strong&gt;effectively solve many tasks&lt;/strong&gt; (e.g. translation of natural language, playing go or atari games, image classification, etc.), so I would say that this is the main reason they are widely studied and there is a lot of interest in them. However, neural networks typically require large datasets to approximate well the desired but unknown function, it can be computationally expensive to train or perform inference with them, and there are other limitations (see &lt;a href=&quot;https://ai.stackexchange.com/a/18578/2444&quot;&gt;this&lt;/a&gt;), so neural networks are definitely not perfect tools, and there is the need to improve them to make them more efficient and useful in certain scenarios (e.g. scenarios where uncertainty estimation is required).&lt;/p&gt;&#xA;&lt;p&gt;I am not really familiar with research on polynomial regression, but it is possible that this and other tools have been overlooked by the ML community. You may want to have a look at &lt;a href=&quot;https://arxiv.org/abs/1806.06850&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which states that NNs are essentially doing polynomial regression, though I have not read it, so I don't know the details about the main ideas and results in this paper.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T23:38:42.803" LastActivityDate="2021-01-21T23:38:42.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23347" PostTypeId="1" CreationDate="2020-08-30T20:42:09.047" Score="0" ViewCount="18" Body="&lt;p&gt;My model deals with videos, and I want to calculate how fast it can process frames as in frames per second or processing time for 1 frame.&lt;/p&gt;&#xA;&lt;p&gt;I have made a single function to get predictions, it takes in raw frames as input, does all the preprocessing, and returns the classification. One of the preprocessing steps is sampling the frames from the video, basically, it reduces the number of frames which go into the deep learning model by 1/5. Without all the preprocessing, the model won't perform as expected.&lt;/p&gt;&#xA;&lt;p&gt;So my question is, should I consider the preprocessing time aswell? And, most importantly, is this processing time for all frames or just for the frames the model actually sees?&lt;/p&gt;&#xA;&lt;p&gt;Example: Input is of 50 frames, and total time taken to preprocess them and make predictions is 1 second.&#xA;So, the processing time for 1 frame is 1/50seconds. &lt;strong&gt;OR&lt;/strong&gt; Should it be 1/10seconds, as model only gets to process 10 frames, others simply get skipped in preprocessing.&lt;/p&gt;&#xA;&lt;p&gt;Which way is the standard way or the right way?&lt;/p&gt;&#xA;" OwnerUserId="40475" LastActivityDate="2020-08-30T20:42:09.047" Title="Calculating processing time of a deep learning model" Tags="&lt;deep-learning&gt;&lt;algorithm&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23348" PostTypeId="1" AcceptedAnswerId="23353" CreationDate="2020-08-30T21:21:30.330" Score="1" ViewCount="49" Body="&lt;p&gt;I have a dataset with a number of houses, for each house, I have a description. For example &amp;quot;The house is luxuriously renovated&amp;quot; or &amp;quot;The house is nicely renovated&amp;quot;. My aim is to identify for each house whether it is luxuriously, well or poorly renovated. I am new to NLP so any tips on how to approach this problem would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="40688" LastEditorUserId="2444" LastEditDate="2020-08-31T12:14:18.037" LastActivityDate="2020-08-31T12:14:18.037" Title="How can I classify houses given a dataset of houses with descriptions?" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23349" PostTypeId="1" CreationDate="2020-08-30T21:33:58.520" Score="0" ViewCount="110" Body="&lt;p&gt;What are examples of problems where &lt;strong&gt;neural networks&lt;/strong&gt; have been used and have achieved human-level or higher performance?&lt;/p&gt;&#xA;&lt;p&gt;Each answer can contain one or more examples. Please, provide links to research papers or reliable articles that validate your claims.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-11-17T23:58:36.977" Title="What are examples of problems where neural networks have achieved human-level or higher performance?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;applications&gt;&lt;social&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23350" PostTypeId="2" ParentId="23349" CreationDate="2020-08-30T21:33:58.520" Score="2" Body="&lt;p&gt;Here is an initial list of AI systems that used neural networks and have achieved human-level or superhuman performance. All of these systems are reinforcement learning systems that play videogames.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/nature16961.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt; and &lt;a href=&quot;https://discovery.ucl.ac.uk/id/eprint/10045895/1/agz_unformatted_nature.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo Zero&lt;/a&gt; (an improved version of AlphaGo that does not use human knowledge but learns by playing against itself) have achieved superhuman performance in the game of &lt;strong&gt;go&lt;/strong&gt; and, in the case of &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero&lt;/a&gt; (a generalized version of AlphaGo Zero), also in the games of &lt;strong&gt;chess&lt;/strong&gt; and &lt;strong&gt;shogi&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN&lt;/a&gt; has achieved human-level or superhuman performance in many &lt;strong&gt;Atari games&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://science.sciencemag.org/content/356/6337/508&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepStack&lt;/a&gt; has achieved human-level performance in poker&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaStar&lt;/a&gt; defeated a top professional player in the real-time strategy game &lt;strong&gt;StarCraft 2&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1912.06680.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI Five&lt;/a&gt; defeated world champions in the game of &lt;strong&gt;Dota&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-17T23:58:36.977" LastActivityDate="2020-11-17T23:58:36.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23351" PostTypeId="2" ParentId="23296" CreationDate="2020-08-31T08:21:01.483" Score="0" Body="&lt;p&gt;There are many games where AI involved but less of them against the human player such as playing the video game. For example in &lt;a href=&quot;https://science.sciencemag.org/content/364/6443/859&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, they proposed a three-dimensional multiplayer first-person video game called Quake III Arena in Capture the Flag mode. Also, &lt;a href=&quot;https://link.springer.com/article/10.1007/s13218-020-00647-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and &lt;a href=&quot;https://uhra.herts.ac.uk/handle/2299/22881&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; show many games where AI involved and they show which kind of games that a human player can play against. I recommend also &lt;a href=&quot;https://github.com/datamllab/awesome-game-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Awesome-Game-AI&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EdmKr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EdmKr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21181" LastEditorUserId="21181" LastEditDate="2020-09-03T15:23:39.340" LastActivityDate="2020-09-03T15:23:39.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23352" PostTypeId="2" ParentId="23243" CreationDate="2020-08-31T08:22:14.513" Score="0" Body="&lt;p&gt;You can save the extracted features coming out of YOLOv4 and save it to pickle file and use them for later.&lt;/p&gt;&#xA;&lt;p&gt;You can also find related information in this project made by Jason Brownlee&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40565" LastActivityDate="2020-08-31T08:22:14.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23353" PostTypeId="2" ParentId="23348" CreationDate="2020-08-31T09:49:00.833" Score="0" Body="&lt;p&gt;It all depends on what kind of annotations or other variables are present in your dataset. I see 2 possible scenarios here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;your dataset is made only of houses descriptions, without any indication of their luxury level.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you have annotations regarding the luxury level or another similar variables from which you can infer the luxury level (like the house price for example).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the first case there's not much you can do except for trying to apply some unsupervised algorithms or transfer learning. Usually in NLP unsupervised techniques are used for tasks like Topic or language modeling, both of which are not really helpful for your specific application since they work at a really abstract level trying to learn relationships between words or documents in corpora containing huge variety of texts. The best you could try could be preprocessing the data to extract specific terms like entities (cities names, furnitures names, etc.) and adjectives from each description, and then try cluster them into n clusters were n is the number of classes you're interest in by applying for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Latent Dirichlet Allocation&lt;/a&gt;. Even though everything is possible with the right time and patience, I would never follow this road, especially because it relies on a perfect preprocessing, that involves already transfer learning, for the name entity recognition part for example. And even tough libraries like &lt;a href=&quot;https://spacy.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpaCy&lt;/a&gt; offer really good models to perform these tasks, once you have these kind of annotations a rule based approach would probably be faster and easier to build than another unsupervised model, e.g. creating a simple dictionary contacting names of luxurious furnitures name and adjectives that indicate if the house is expensive would probably be sufficiently good.&lt;/p&gt;&#xA;&lt;p&gt;In the second case, the story change completely because if you have annotations you could rely on supervised learning. If you already have explicit annotations like &amp;quot;luxuriously renovated&amp;quot; &amp;quot;nicely renovated&amp;quot; nothing stops you from trying to train whatever architecture you feel more comfortable with on this classification task. Even simple architecture like CNN that are easy and fast to train usually achieve good results in classification, and you definitely want to leverage some pre-trained embedding vectors like &lt;a href=&quot;https://nlp.stanford.edu/projects/glove/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GloVe&lt;/a&gt; as an input feature (every deep learning framework like tensorflow or pytorch already implement the possibility to use them).&lt;/p&gt;&#xA;&lt;p&gt;To conclude, if you don't have annotations you might try LDA just to check if you're lucky but if I were you I would start annotating data as quick as I can. A good practice in this case is to also ask someone else to perform the annotations, not only to be faster in the creation of the dataset but also to then calculate the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inter-rater_reliability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inter Annotator Agreement&lt;/a&gt; score, that gives an indication about the quality of the annotations (if the score is low, the dataset quality is poor and no model will be able to learn something from it).&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-08-31T09:49:00.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23354" PostTypeId="1" CreationDate="2020-08-31T09:56:39.337" Score="0" ViewCount="14" Body="&lt;p&gt;I am trying to build a chatbot that can act as a real patient and indulge with a doctor in a conversation. Does anyone have any clue how will it be possible? I am currently doing Rasa stack to build it.&lt;/p&gt;&#xA;" OwnerUserId="40565" LastActivityDate="2020-08-31T09:56:39.337" Title="How can I build a chatbot where patient is a bot and doctor is a user?" Tags="&lt;chat-bots&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23355" PostTypeId="2" ParentId="23337" CreationDate="2020-08-31T10:41:01.197" Score="0" Body="&lt;p&gt;It seems from this old thread &lt;a href=&quot;https://github.com/explosion/spaCy/issues/390&quot; rel=&quot;nofollow noreferrer&quot;&gt;German lemmatization and noun gender&lt;/a&gt; that they never actually implemented this feature. But in the same thread you can find links to these two GitHub projects that should provide you an alternative:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Liebeck/IWNLP&quot; rel=&quot;nofollow noreferrer&quot;&gt;IWNLP&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/DuyguA/DEMorphy&quot; rel=&quot;nofollow noreferrer&quot;&gt;DEMorphy&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-08-31T10:41:01.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23357" PostTypeId="2" ParentId="23332" CreationDate="2020-08-31T14:29:28.267" Score="0" Body="&lt;p&gt;The answer is actually really simple: they are all randomly initialised. So they are to all intents and purposes &amp;quot;normal&amp;quot; weights of a neural network.&lt;/p&gt;&#xA;&lt;p&gt;This is also the reason why in the &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; the authors tested several setting with single and multiple attention heads. If these matrices were somehow &amp;quot;special&amp;quot; or predetermined they would all serve the same purpose. Instead, because of their random initialisation, each attention head learn to contribute to solve a different task, like they show in Figure 3 and 4.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-08-31T14:29:28.267" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23358" PostTypeId="2" ParentId="23328" CreationDate="2020-08-31T15:15:30.857" Score="1" Body="&lt;p&gt;I think you should try to reason in terms of total &amp;quot;area&amp;quot; explored by the agent rather than &amp;quot;how far&amp;quot; it moves from the initial point, and also you should add some reward terms to push the agent steering more often. I think that the problem with your setting is more or less this: The agent go as straight as it can because you're rewarding it for it, it start sensing an obstacle so it stops, there is no reward for steering so the best strategy to go away from the obstacle and not end the episode is just to go backwards.&lt;/p&gt;&#xA;&lt;p&gt;Considering that you have information about the grid points at any time you could rewrite the reward function in terms of grid squared explored by checking at each move if the agent end up in a new square grid:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_t=&#xA;\begin{cases}&#xA;r_{\text{collision}}\\&#xA;\lambda^d\left(\|\mathbf{p}^{x,y}_t-\mathbf{p}_0^{x,y}\|_2-\|\mathbf{p}_{t-1}^{x,y}-\mathbf{p}_0^{x,y}\|_2 \right) + r_{new-squared-explored}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Moreover it would be useful to add some reward terms also related to how the agent avoid the obstacle, for example a penalisation when the sensor goes and remain under a certain threshold (to make the agent learn to not go and stay too close to an obstacle) but also a rewarding term when an obstacle is detected and the agent manage to maintain a certain distance from it (even though if not well tuned this term could lead the agent to learn to just run in circles around a single obstacle, but if tuned properly I think it might help to make the agent movements smoother).&lt;/p&gt;&#xA;" OwnerUserId="34098" LastEditorUserId="34098" LastEditDate="2020-08-31T15:21:31.330" LastActivityDate="2020-08-31T15:21:31.330" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23360" PostTypeId="1" AcceptedAnswerId="23363" CreationDate="2020-08-31T16:30:27.263" Score="1" ViewCount="196" Body="&lt;p&gt;For an upcoming project I'm trying to write a text classifier for the IMDb sentiment analysis dataset. This needs to vectorize words using an embedding layer and then reduce the dimensions of the output with global average pooling. This is proving however to be very difficult for my low experience level, and I am struggling to wrap my head around the dimensionality involved, bearing in mind I must avoid libraries such as tensorflow that would make it very basic exercise. I am hoping that I could make it easier by encoding each word in the reviews as a one-hot vector, and passing it through a few regular dense layers. Would this work and yield decent results?&lt;/p&gt;&#xA;" OwnerUserId="38497" LastActivityDate="2020-08-31T19:49:33.603" Title="Can I use one-hot vectors for text classification?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;text-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23361" PostTypeId="1" CreationDate="2020-08-31T17:39:54.100" Score="1" ViewCount="24" Body="&lt;p&gt;I designed a DQN architecture for some problem. The problem has a parameter &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; as the number of clients. In my situation, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is large, &lt;span class=&quot;math-container&quot;&gt;$m\in\{100,200,\ldots,1000\}$&lt;/span&gt;. For this situation, the number of input ports of the DQN is some few thousand, &lt;span class=&quot;math-container&quot;&gt;$\{1000, 2000, \ldots, 10000\}$&lt;/span&gt;. For some fixed &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, I would like to see the performance of deep Q learning on the performance. So I have to train the DQN for every change that occurs on &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and this should handle thousands of inputs ports for each training. Is this situation familiar in DQN and if not how to solve this issue?&lt;/p&gt;&#xA;" OwnerUserId="37642" LastActivityDate="2020-08-31T17:39:54.100" Title="Is it feasible to train a DQN with thousands of input ports?" Tags="&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;curse-of-dimensionality&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23362" PostTypeId="1" AcceptedAnswerId="23407" CreationDate="2020-08-31T18:45:53.633" Score="6" ViewCount="147" Body="&lt;p&gt;I have watched Stanford's lectures about artificial intelligence, I currently have one question: why don't we use autoencoders instead of GANs?&lt;/p&gt;&#xA;&lt;p&gt;Basically, what GAN does is it receives a random vector and generates a new sample from it. So, if we train autoencoders, for example, on cats vs dogs dataset, and then cut off the decoder part and then input random noise vector, wouldn't it do the same job?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-09-02T23:42:16.157" LastActivityDate="2020-09-27T19:21:54.213" Title="Why don't we use auto-encoders instead of GANs?" Tags="&lt;computer-vision&gt;&lt;generative-adversarial-networks&gt;&lt;autoencoders&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23363" PostTypeId="2" ParentId="23360" CreationDate="2020-08-31T19:49:33.603" Score="1" Body="&lt;p&gt;One hot encoding is a good strategy to apply with categorical variables that assume few possible values. The problem with text data is that you easily end up with corpora with a really large vocabulary. If I remember correctly the IMDb dataset contains around 130.000 unique words, which means that you should create a network with an input matrix of size 130.000 x max_length where max_length is the fixed maximum length allowed for each review. Apart from the huge size, this matrix would also be extremely sparse, and that's another big issue in using one-hot encoding with text.&lt;/p&gt;&#xA;&lt;p&gt;For these reasons, I really doubt you would achieve any good results with a simple one-hot encoding. Embeddings where actually designed precisely to overcome all these issues, they have fixed reasonable size, they assume continue values between 0 and 1, which is desirable for deep neural networks, and they can be treated as &amp;quot;extra&amp;quot; trainable weights of a network.&lt;/p&gt;&#xA;&lt;p&gt;If you really want to avoid embeddings I would suggest you to use (or implement, I don't think it will be so hard) a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tf%E2%80%93idf&quot; rel=&quot;nofollow noreferrer&quot;&gt;term frequency–inverse document frequency&lt;/a&gt; vectoriser. It is closer to one-hot encoding in the fact that it is based on the creation of a huge co-occurances matrix between words, but at least the values are continuous and not dichotomous. Nevertheless I would not expect high performances with the tf-idf either, simply because this type of encoding works best with shallow models like the Naive Bayes rather than deep models.&lt;/p&gt;&#xA;" OwnerUserId="34098" LastActivityDate="2020-08-31T19:49:33.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23364" PostTypeId="1" CreationDate="2020-08-31T20:35:40.093" Score="1" ViewCount="38" Body="&lt;p&gt;I have a binary classification problem. I have variables (features) var1, var2, var3, ..., var14.&lt;/p&gt;&#xA;&lt;p&gt;Using these variables (aka features) in a logistic regression, I get their weights.&lt;/p&gt;&#xA;&lt;p&gt;If I use the same set of variables in a neural network:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I get a different output?&lt;/p&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I get the same output?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I developed a ROC Curve, and I have both lines overlaying on each other. I am not sure if I am missing something here.&lt;/p&gt;&#xA;" OwnerUserId="40706" LastEditorUserId="2444" LastEditDate="2020-09-07T10:58:37.833" LastActivityDate="2020-09-07T10:58:37.833" Title="Given the same features, do logistic regression and neural networks produce the same output?" Tags="&lt;neural-networks&gt;&lt;comparison&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23365" PostTypeId="2" ParentId="7416" CreationDate="2020-08-31T21:08:37.917" Score="1" Body="&lt;p&gt;I've published an article with the corresponding new method based on the generative grammars of first-order theories:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.academia.edu/43984841/Thoughts_on_generative_grammars_and_their_use_in_automated_theorem_proving_based_on_neural_networks_with_a_type_0_grammar_of_Nicods_propositional_logic_system_added_&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thoughts on generative grammars and their use in automated theorem proving based on neural networks&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This approach allows not to use previous data but to generate it as much as it's needed in machine learning. In the article, you may find necessary theory on logic, grammars, and neural networks. You'll also find examples of the python-functions generating proofs literally. I've added a grammar for the propositional logic that can be naturally enlarged to the &amp;quot;real&amp;quot; cases of first-order theories (say, group or number theory).&lt;/p&gt;&#xA;" OwnerUserId="40703" LastEditorUserId="40703" LastEditDate="2020-09-02T21:36:32.653" LastActivityDate="2020-09-02T21:36:32.653" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23367" PostTypeId="1" AcceptedAnswerId="23379" CreationDate="2020-09-01T00:02:24.690" Score="2" ViewCount="52" Body="&lt;p&gt;I read the following from a book:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You can intuitively understand the dimensionality of your representation space as “how much freedom you’re allowing the model to have when learning internal representations.” Having more units (a higher-dimensional representation space) allows your model to learn more-complex representations, but it makes the model more computationally expensive and may lead to learning unwanted patterns (patterns that will improve performance on the training data but not on the test data).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why does using a higher representation space lead to performance increase on the training data but not on the test data?&lt;/p&gt;&#xA;&lt;p&gt;Surely the representations/patterns learnt in the training data will be found too in the test data.&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="2444" LastEditDate="2020-09-01T11:22:55.857" LastActivityDate="2020-09-01T12:01:35.060" Title="Why does using a higher representation space lead to performance increase on the training data but not on the test data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23368" PostTypeId="1" CreationDate="2020-09-01T06:54:14.883" Score="1" ViewCount="54" Body="&lt;p&gt;I am trying to train a DDPG agent augmented with Hindsight Experience Replay (HER) to solve &lt;a href=&quot;https://i.stack.imgur.com/gT1Rh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;the KukaGymEnv environment&lt;/a&gt;.  The actor and critic are simple neural networks with two hidden layers (as in the HER paper).&lt;/p&gt;&#xA;&lt;p&gt;More precisely, the hyper-parameters I am using are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The actor's hidden layer's sizes: [256, 128] (using ReLU activations and a tanh activation after the last layer)&lt;/li&gt;&#xA;&lt;li&gt;Critic's hidden layer's sizes: [256, 128] (Using ReLU activations)&lt;/li&gt;&#xA;&lt;li&gt;Maximum Replay buffer size: 50000&lt;/li&gt;&#xA;&lt;li&gt;Actor learning rate: 0.000005 (Adam Optimizer)&lt;/li&gt;&#xA;&lt;li&gt;Critic learning rate: 0.00005 (Adam Optimizer)&lt;/li&gt;&#xA;&lt;li&gt;Discount rate: 0.99&lt;/li&gt;&#xA;&lt;li&gt;Polyak constant : 0.001&lt;/li&gt;&#xA;&lt;li&gt;The transitions are sampled in batches of 32 from the replay buffer for training&lt;/li&gt;&#xA;&lt;li&gt;Update rate: 1 (target networks are updated after each time step)&lt;/li&gt;&#xA;&lt;li&gt;The action selection is stochastic with the noise being sampled from a normal distribution of mean 0 and standard deviation of 0.7&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I trained the agent for 25 episodes with a maximum of 700 time-steps each and got the following reward plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gT1Rh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gT1Rh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reward shoots up to a very high value of about 8000 for the second episode and steeply falls to -2000 in the very next time step, never to rise again. What could be the reason for this behavior and how can I get it to converge?&lt;/p&gt;&#xA;&lt;p&gt;PS : One difference I observed while training this agent from while training a simple DDPG agent is that for simple DDPG, the episode would usually terminate at around 450 time-steps, thus never reaching the maximum specified timesteps. However, here, no episode terminated before the specified 700 maximum time steps. This might have something to do with the performance.&lt;/p&gt;&#xA;" OwnerUserId="38895" LastEditorUserId="2444" LastEditDate="2020-11-20T19:12:32.753" LastActivityDate="2020-11-20T19:12:32.753" Title="Why would DDPG with Hindsight Experience Replay not converge?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;rewards&gt;&lt;ddpg&gt;&lt;hindsight-experience-replay&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23369" PostTypeId="1" CreationDate="2020-09-01T08:03:21.840" Score="0" ViewCount="152" Body="&lt;p&gt;I have a data set with  36 rows and 9 columns. I am trying to make a model to predict the 9th column&lt;/p&gt;&#xA;&lt;p&gt;I have tried modeling the data using  a range of models using caret to perform cross-validation and hyper parameter tuning: 'lm', random forrest (ranger) and GLMnet, with range of different folds and hyper-parameter tuning, but the modeling has not been very successful.&lt;/p&gt;&#xA;&lt;p&gt;Next I have tried to use some of the neural-network models. I tried the 'monmlp'. During hyper parameter tuning I could see that the RMSE drops to a level when using ~ 6 hidden units. The problem I observe using this model is&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Prediction is almost equal to data&lt;/li&gt;&#xA;&lt;li&gt;When doing a &amp;quot;manual&amp;quot; cross validation by removing a single datapoint and using the trained model to predict, it has no predictive power&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have tried to use a range of different hidden units, but i think the problem is that the model is overfitted despite using  caret cross validation feature.&lt;/p&gt;&#xA;&lt;p&gt;There two feedbacks I would appreciate&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a way to prevent overfitting, by chosen optimal number of training iterations ( optimal RMSE on out of sample ). Can this by done using caret or some other package&lt;/li&gt;&#xA;&lt;li&gt;Am I using the right model?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am relatively unexperienced with ML and choosing a good model is tough: when you look at the available packages it is overwhelming:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://topepo.github.io/caret/train-models-by-tag.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://topepo.github.io/caret/train-models-by-tag.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40709" LastActivityDate="2021-02-12T17:08:40.850" Title="How to avoid over-fitting using early stopping when using R cross validation package caret" Tags="&lt;machine-learning&gt;&lt;overfitting&gt;&lt;cross-validation&gt;&lt;r&gt;&lt;early-stopping&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23370" PostTypeId="1" CreationDate="2020-09-01T08:29:41.013" Score="2" ViewCount="122" Body="&lt;p&gt;How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, but not this case.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{maximize} &amp;amp; \mathrm{Tr}\left( \mathbf{w}^T\mathbf{X}\mathbf{X}^T\mathbf{w} \right)\\ \text{subject to} &amp;amp; \mathbf{w}^T\mathbf{w} = 1\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="40710" LastEditorUserId="3171" LastEditDate="2020-11-15T13:35:11.073" LastActivityDate="2020-11-15T13:35:11.073" Title="How does PCA work when we reduce the original space to 2 or higher-dimensional space?" Tags="&lt;math&gt;&lt;optimization&gt;&lt;linear-algebra&gt;&lt;principal-component-analysis&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23371" PostTypeId="1" AcceptedAnswerId="23374" CreationDate="2020-09-01T08:55:48.847" Score="3" ViewCount="149" Body="&lt;p&gt;As it is discussed &lt;a href=&quot;https://www.reddit.com/r/latin/comments/6akqdi/why_is_google_translate_so_bad_for_latin_a/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and I saw it on other Latin language forums too, everybody complains about how Google Translate fails to translate the Latin language. From my personal experience, it is not that much bad on other languages, including romance languages.&lt;/p&gt;&#xA;&lt;p&gt;So, what makes Google Translate fail so much to translate the Latin language? Is it about its syntax and grammar or lack of data?&lt;/p&gt;&#xA;" OwnerUserId="38344" LastEditorUserId="2444" LastEditDate="2020-11-10T10:55:31.860" LastActivityDate="2020-11-10T10:55:31.860" Title="What makes Google Translate fail on the Latin language?" Tags="&lt;natural-language-processing&gt;&lt;google-translate&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23372" PostTypeId="2" ParentId="23370" CreationDate="2020-09-01T09:05:31.870" Score="2" Body="&lt;p&gt;You might want to have a look at the wikipedia article of &lt;a href=&quot;https://en.wikipedia.org/wiki/Principal_component_analysis#Further_components&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCA&lt;/a&gt;, where it says:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;The &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th component can be found by subtracting the first &lt;span class=&quot;math-container&quot;&gt;$k − 1$&lt;/span&gt; principal components from &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt;:&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat{\mathbf{X}}_k = \mathbf{X} - \sum_{s=1}^{k-1}\mathbf{X}\mathbf{w}_s\mathbf{w}_s^T$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then you repeat the process to find the next component:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{w}_k = \arg\max \mathbf{w}^T\mathbf{\hat{X}}^T_k\mathbf{\hat{X}}_k\mathbf{w}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{s.t. } \mathbf{w}_k^T\mathbf{w}_k = 1$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-09-01T09:05:31.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23373" PostTypeId="2" ParentId="23362" CreationDate="2020-09-01T09:07:57.300" Score="4" Body="&lt;p&gt;In fact, autoencoders are used for generative tasks. Have a look at &lt;a href=&quot;https://arxiv.org/pdf/1606.05908.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tutorial on Variational Autoencoders&lt;/a&gt; (VAEs).&lt;/p&gt;&#xA;&lt;p&gt;The coolest thing about VAE is that abstract features can be easily amplified or suppressed based on extracted vectors from the latent space. Let's imagine a model trained on MNIST to generate digits. If you take two images of the same digit which only vary in thickness, encode both of them and subtract the two vectors, the resulting vector will be a description of thickness in the latent space. Now you can generate an arbitrary digit and incrementally adjust its thickness based on that vector.&lt;/p&gt;&#xA;" OwnerUserId="38671" LastActivityDate="2020-09-01T09:07:57.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23374" PostTypeId="2" ParentId="23371" CreationDate="2020-09-01T09:48:24.510" Score="4" Body="&lt;p&gt;I don't know what model Google is using for their translations, but it's highly likely that they're using one of today's SOTA deep learning models.&lt;/p&gt;&#xA;&lt;p&gt;The latest NLP models are trained on data scraped from the web, e.g. OpenAI's &lt;a href=&quot;https://openai.com/blog/better-language-models/#fn1&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT-2&lt;/a&gt; was trained on a dataset of 8 million web pages, Google's &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt; was trained on the BookCorpus (800M words) and English Wikipedia (2.500M words) pages.&lt;/p&gt;&#xA;&lt;p&gt;Now think about the amount of latin web pages and notice that there are over 6 million english wikipedia articles but less than 135.000 in latin (see &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Wikipedias#Detailed_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;As you can see, massive amounts of data are crucial for neural machine translation and I assume there is simply not enough out there for latin. Plus latin is one of the most complex and complicated languages, this makes the task not easier. Maybe Google and Co also focus less on a 'dead' language which is not spoken anymore and has it's right to exist more for educational purposes.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-09-01T09:48:24.510" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23375" PostTypeId="1" AcceptedAnswerId="23393" CreationDate="2020-09-01T10:00:23.143" Score="0" ViewCount="38" Body="&lt;p&gt;As I am curious on music theory I would like to know that If is there any such network that analyse like labeling chords, or doing a roman numeral analysis.&lt;/p&gt;&#xA;&lt;p&gt;Like an example below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tE45W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tE45W.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Roman_numeral_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It does not seem to be a difficult task.&lt;/p&gt;&#xA;&lt;p&gt;Some other examples are given &lt;a href=&quot;http://bachchoraleharmony.com/wp-content/uploads/2014/08/J.S.Bach-413-Chorales-Analyzed_Preview-4.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;[external link]&lt;/p&gt;&#xA;&lt;p&gt;Also I am curious that If it is a possible task for AI to accomplish.&lt;/p&gt;&#xA;" OwnerUserId="38344" LastEditorUserId="38344" LastEditDate="2020-09-01T10:05:30.057" LastActivityDate="2020-09-02T11:05:20.270" Title="Is there any network/paper used to analyse music scores?" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23376" PostTypeId="2" ParentId="23290" CreationDate="2020-09-01T10:00:30.477" Score="2" Body="&lt;p&gt;The problem is not that we need importance sampling because the learning is off-policy -- you are correct in that for one step off-policy algorithms such as &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning we don't need importance sampling, &lt;a href=&quot;https://ai.stackexchange.com/questions/21859/why-we-dont-use-importance-sampling-in-tabular-q-learning&quot;&gt;see e.g. here for an explanation why&lt;/a&gt;. The reason we need the importance sampling is due to the loss used to train the network.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://daiwk.github.io/assets/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original DQN paper&lt;/a&gt;, the loss is defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_i(\theta_a) = \mathbb{E}_{(s,a,r,s') \sim \mbox{U}(D)} \left[ \left( r + \gamma \max_{a'} Q(s',a' ; \theta_i^-) - Q(s,a;\theta_i) \right)^2 \right ]\;.$$&lt;/span&gt;&#xA;You can see here the expectation over the loss is taken according to a uniform distribution over the replayed buffer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. If we started randomly sampling non-uniformly, as is the case in PER, then the expectation wouldn't be satisfied and would introduce bias. Importance sampling is used to correct this bias.&lt;/p&gt;&#xA;&lt;p&gt;Note that in the paper they mention that the bias isn't &lt;em&gt;as much&lt;/em&gt; of an issue at the start of learning and hence they use a decaying &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; that only makes the importance sampling weights the 'correct' weights to use at the end of learning - this means that the estimate of the loss is asymptotically unbiased.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-09-01T19:35:35.687" LastActivityDate="2020-09-01T19:35:35.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23377" PostTypeId="1" CreationDate="2020-09-01T11:04:38.000" Score="1" ViewCount="97" Body="&lt;p&gt;This question is in relation to a previous doubt of mine :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/23330/are-there-neural-networks-where-nodes-are-randomly-selected-from-among-a-set-of&quot;&gt;Are there neural networks where nodes are randomly selected from among a set of nodes (in random orders and a random number of times)?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have made a bit of progress from there, refurbished my code, and got things ready.&lt;/p&gt;&#xA;&lt;p&gt;What I intend to make is 'Insane Mind', a model which forms random linear neural networks from a set of nodes at random times ( I made out the 'linear neural network' part from a bit of Google searches).&lt;/p&gt;&#xA;&lt;p&gt;The basic process involved is :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The system forms nodes of random weights . These nodes also have the Sigmoid function (the logistic fuction : &lt;span class=&quot;math-container&quot;&gt;$f(x) = \frac{1}{1 + e^{-x}}$&lt;/span&gt; ) , and I termed these 'Gravitons' (because of the usage of the word 'weights' in them - sorry if my terminology work seems ambiguous...)&lt;/li&gt;&#xA;&lt;li&gt;The input enters the system via one of the gravitons.&lt;/li&gt;&#xA;&lt;li&gt;The node processes it and either passes the output to the next node or to itself .&lt;/li&gt;&#xA;&lt;li&gt;Step 3 is repeated a certain number of times as the number of gravitons made for use.&lt;/li&gt;&#xA;&lt;li&gt;The output of the final graviton is given as the output of the whole system.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;One thing I'm sure of this model is that this model can transform an input vector into an output vector.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure whether this is ambiguous or similar to  previously discovered model. Plus, I'd like to know if this will be effective in any situation (I believe it will be of help in classification problems).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note : I made this out of my imagination , which means this may be useless one way or the other, but still it seemed to work.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's the training algorithm I made for this model :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In my Python implementation of this model, I had added a provision in the 'Graviton' class to store the derivative of the output of the graviton. Using this, the gravitons are ordered in the increasing order of the derivatives of their outputs.&lt;/li&gt;&#xA;&lt;li&gt;The first graviton is taken, and its weight is modified by the error in the output.&lt;/li&gt;&#xA;&lt;li&gt;The error is modified by the product of the graviton's output derivative and its weight after editing.&lt;/li&gt;&#xA;&lt;li&gt;Steps 2 through 3 are done for the other gravitons as well. The final error (given by the &lt;code&gt;error&lt;/code&gt; variable ) will be the product of the derivatives, the edited weights and the error in the output.&lt;/li&gt;&#xA;&lt;li&gt;The set of gravitons thus formed is the next set subjected to this training.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For extra reference, here's the code:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Insane_Mind.py :&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;from math import *&#xA;from random import *&#xA; &#xA;class MachineError(Exception):&#xA;    '''standard exception in the API'''&#xA;    def __init__(self, stmt):&#xA;        self.stmt = stmt&#xA;        &#xA;def sig(x):&#xA;    '''Sigmoid function'''&#xA;    try :&#xA;        return exp(x)/(exp(x) + 1)&#xA;    except OverflowError:&#xA;        if x &amp;gt; 0 :&#xA;            return 1&#xA;        elif x &amp;lt; 0:&#xA;            return &#xA;&#xA;class Graviton:&#xA;    def __init__(self, weight, marker):&#xA;        '''Basic unit in 'Insane Mind' algorithm'''&#xA;        self.weight = weight&#xA;        self.marker = marker + 1&#xA;        self.input = 0&#xA;        self.output = 0&#xA;        self.derivative = 0&#xA;&#xA;    def process(self, input_to_machine):&#xA;        '''processes the input'''&#xA;        self.input = input_to_machine&#xA;        self.output = sig(self.weight * self.input)&#xA;        self.derivative = self.input * self.output * (1- self.output) &#xA;        return self.output&#xA;    &#xA;    def get_derivative_at_input(self):&#xA;        '''returns the derivative of the output'''&#xA;        return self.derivative&#xA;&#xA;    def correct_self(self, learning_rate, error):&#xA;        '''edits the weight'''&#xA;        self.weight += -1 * error * learning_rate * self.get_derivative_at_input() * self.weight&#xA;        &#xA;class Insane_Mind_Base:&#xA;    '''Insane_Mind base class - this is what we're gonna use to build the actual machine'''&#xA;    def __init__(self, number_of_nodes):&#xA;        '''initialiser for Insane_Mind_Base class.&#xA;           arguments : number_of_nodes : the number of nodes you want'''&#xA;        self.system = [Graviton(random(),i) for i in range(number_of_nodes)] # the actual system&#xA;        self.system_size = number_of_nodes # number of nodes , or 'system size'&#xA;        &#xA;    def  output_sys(self, input_to_sys):&#xA;        '''system output'''&#xA;        self.output = input_to_sys&#xA;        for i in range(self.system_size):&#xA;            self.output = self.system[randint(0,self.system_size - 1 )].process(self.output)&#xA;        return self.output&#xA;    &#xA;    def train(self, learning_rate, wanted):&#xA;        '''trains the system'''&#xA;        self.cloned = []&#xA;        order = []&#xA;        temp = {}&#xA;        for graviton in self.system:&#xA;            temp.update({str(graviton.derivative): self.system.index(graviton)})&#xA;        order = sorted(temp)&#xA;        i = 0&#xA;        error = wanted - self.output&#xA;        for value in order:&#xA;            self.cloned.append(self.system[temp[value]])&#xA;            self.cloned[i].correct_self(learning_rate, error)&#xA;            error *= self.cloned[i].derivative * self.cloned[i].weight&#xA;            i += 1&#xA;        self.system = self.cloned&#xA;&#xA;    def details(self):&#xA;        '''gets the weights of each graviton'''&#xA;        for graviton in self.system:&#xA;            print(&amp;quot;Node : {0}, weight : {1}&amp;quot;.format(graviton.marker , graviton.weight))&#xA;&#xA;class Insane_Mind:&#xA;    &#xA;    '''Actaul Insane_Mind class'''&#xA;    def __init__(self, number_of_gravitons):&#xA;        '''initialiser'''&#xA;        self.model = Insane_Mind_Base(number_of_gravitons)&#xA;        self.size = number_of_gravitons&#xA;        &#xA;    def get(self, input):&#xA;        '''processes the input'''&#xA;        return self.model.output_sys(input)&#xA;    &#xA;    def train_model(self, lrate, inputs, outputs, epoch):&#xA;        '''train the model'''&#xA;        if len(inputs) != len(outputs):&#xA;            raise MachineError(&amp;quot;Unequal sizes for training input and output vectors&amp;quot;)&#xA;        epoch = str(epoch)&#xA;        if epoch.lower() == 'sys_size':&#xA;            epoch = int(self.model.system_size)&#xA;        else:&#xA;            epoch = int(epoch)&#xA;        for k in range(epoch):&#xA;            for j in range(len(inputs)):&#xA;                    val = self.model.output_sys(inputs[j])&#xA;                    self.model.train(1/val if str(lrate).lower() == 'output' else lrate, outputs[j])&#xA;    &#xA;    def details(self):&#xA;        '''details of the machine'''&#xA;        self.model.details()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Insane_Mind_Test.py :&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Insane_Mind import *&#xA;from statistics import *&#xA;&#xA;input_data = [3,4,3,5,4,4,3,6,5,4] # list of forces using which the coin is tossed&#xA;output_data = [1,0,0,1,1,0,0,0,1,1] # head or tails in binary form (0 = tail (= not head), 1 = head)&#xA;wanteds = output_data.copy()&#xA;model = Insane_Mind(2) # Insane Mind model&#xA;print(&amp;quot;Before Training:&amp;quot;)&#xA;print(&amp;quot;----------------&amp;quot;)&#xA;model.details() # fetches you weights of the model&#xA;&#xA;def normalize(x):&#xA;    cloned = x.copy()&#xA;    meanx = mean(x)&#xA;    stdevx = stdev(x)&#xA;    for i in range(len(x)):&#xA;        cloned[i] = (cloned[i] - meanx)/stdevx&#xA;    return cloned&#xA;&#xA;def random_catch(range_of_catches, sample_length):&#xA;    # sample data generator. I named it random catch as part of using it in testing whether my model &#xA;    # ' catches the correct guess'. :)&#xA;    return [randint(range_of_catches[0], range_of_catches[1]) for i in range(sample_length)]&#xA;&#xA;input_data = normalize(input_data)&#xA;output_data = normalize(output_data)&#xA;&#xA;model.train_model('output', input_data, output_data, 'sys_size')&#xA;# the argument 'output' for the argument 'lrate' (learning rate) was to specify that the learning rate at # each step is the inverse of the output, and the use of 'sys_size' for the number of times to be trained&#xA;# is used to tell the machine that the required number of epochs is equal to the size of the system or &#xA;# the number of nodes in it.&#xA;&#xA;print(&amp;quot;After Training:&amp;quot;)&#xA;print(&amp;quot;----------------&amp;quot;)&#xA;model.details() # fetches you weights of the model&#xA;&#xA;predictions = [model.get(i) for i in input_data]&#xA;&#xA;threshold = mean(predictions)&#xA;predictions = [1 if i &amp;gt;= threshold else 0 for i in predictions]&#xA;&#xA;print(&amp;quot;Predicted : {0}&amp;quot;.format(predictions))&#xA;print(&amp;quot;Actual:{0}&amp;quot;.format(wanteds))&#xA;mse_array = [(wanteds[j] - predictions[j])**2 for j in range(len(input_data))]&#xA;print(&amp;quot;Mean squared error:{0}&amp;quot;.format(mean(mse_array)))&#xA;&#xA;accuracy = 0&#xA;for i in range(len(predictions)):&#xA;    if predictions[i] == wanteds[i]:&#xA;        accuracy += 1&#xA;&#xA;print(&amp;quot;Accuracy:{0}({1} out of {2} predictions correct)&amp;quot;.format(accuracy/len(wanteds), accuracy, len(predictions)))&#xA;&#xA;print(&amp;quot;______________________________________________&amp;quot;)&#xA;&#xA;print(&amp;quot;Random catch test&amp;quot;)&#xA;print(&amp;quot;-----------------&amp;quot;)&#xA;&#xA;times = int(input(&amp;quot;No. of tests required : &amp;quot;))&#xA;catches = int(input(&amp;quot;No. of catches per test&amp;quot;))&#xA;mse = {}&#xA;for m in range(times):&#xA;    wanted = random_catch([0,1] , catches)&#xA;    forces = random_catch([1,10], catches)&#xA;    predictions = [model.get(k) for k in forces]&#xA;    threshold = mean(predictions)&#xA;    predictions = [1 if value &amp;gt;= threshold else 0 for value in predictions]&#xA;    mse_array = [(wanted[j] - predictions[j])**2 for j in range(len(predictions))]&#xA;    print(&amp;quot;Mean squared error:{0}&amp;quot;.format(mean(mse_array)))&#xA;    mse.update({(m + 1):mean(mse_array)})&#xA;    accuracy = 0&#xA;    for i in range(len(predictions)):&#xA;        if predictions[i] == wanted[i]:&#xA;            accuracy += 1&#xA;    print(&amp;quot;Accuracy:{0}({1} out of {2} predictions correct)&amp;quot;.format(accuracy/len(wanteds), accuracy, len(predictions)))&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried running 'Insane_Mind_Test.py', and the results I got are :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6OWPM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6OWPM.png&quot; alt=&quot;Results from 'Insane_Mind_Test.py'&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The formula I used from MSE is (please correct me if I was wrong):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ MSE = \frac{\sum_{i = 1}^n (x_i - x'_i)^2}{n}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ x_i = \text{Intended output}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ x'_i = \text{Output predicted}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ n = \text{Number of outputs}$$&lt;/span&gt;&#xA;My main intention was to make a guess system.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note : Here, I had to think differently. I decided to classify the forces as those yielding a head and those that yield a tail (unlike what I say in the comments in the program).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thanks for all help in advance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Here's the training data :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Forces         Head(1) or not head(0)[rather call it tail] &#xA;_______        ______________________&#xA;3                  1&#xA;4                  0&#xA;3                  0&#xA;5                  1&#xA;4                  1&#xA;4                  0&#xA;3                  0&#xA;6                  0&#xA;5                  1&#xA;4                  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40583" LastEditorUserId="40583" LastEditDate="2020-09-01T16:16:18.090" LastActivityDate="2021-01-31T18:06:55.603" Title="Is my &quot;Insane Mind&quot; design for a classifier novel or effective?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23378" PostTypeId="2" ParentId="5399" CreationDate="2020-09-01T11:19:24.927" Score="1" Body="&lt;p&gt;There is a hardware based reasoning. Matrix multiplication is one of the central computations in deep learning. SIMD operations in CPUs happen in batch sizes, which are powers of 2.&lt;/p&gt;&#xA;&lt;p&gt;Here is a good reference about speeding up neural networks on CPUs by leveraging SIMD instructions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37631.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving the speed of neural networks on CPUs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You will notice batch sizes that are powers of 2. This is a good paper to read about implementing neural networks using SIMD instructions.&lt;/p&gt;&#xA;" OwnerUserId="40714" LastEditorUserId="40714" LastEditDate="2021-05-07T15:56:04.170" LastActivityDate="2021-05-07T15:56:04.170" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23379" PostTypeId="2" ParentId="23367" CreationDate="2020-09-01T11:44:08.597" Score="1" Body="&lt;p&gt;The answer to your question is that the &lt;strong&gt;capacity&lt;/strong&gt; of your model (i.e. the number and type of function that your model can compute) generally increases with the number of parameters. So, a bigger model can potentially &lt;strong&gt;approximate better&lt;/strong&gt; the function represented by your training data, but, at the same time, it may not take into account the test data, a phenomenon known as &lt;strong&gt;over-fitting&lt;/strong&gt; the training data (i.e. fitting &amp;quot;too much&amp;quot; the training data).&lt;/p&gt;&#xA;&lt;p&gt;In theory, you want to fit the training data perfectly, so over-fitting should not make sense, right? The problem is that, if we just fit all the (training) data, there is no way of &lt;em&gt;empirically&lt;/em&gt; checking that our model will perform well on &lt;strong&gt;unseen data&lt;/strong&gt;, i.e. will it &lt;strong&gt;generalize&lt;/strong&gt; to data not seen during training?&#xA;We split our data into training and test data because of this: we want to understand whether our model will perform well also on unseen data or not.&lt;/p&gt;&#xA;&lt;p&gt;There are also some &lt;strong&gt;theoretical bounds&lt;/strong&gt; that ensure you that, probabilistically and approximately, you can generalize: if you have more training data than a certain threshold, the probability that you perform badly is small. However, these theoretical bounds are often not taken into account in practice because, for example, we may not be able to collect more data to ensure that the bounds are satisfied.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Surely the representations/patterns learnt in the training data will be found too in the test data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is possibly the wrong assumption and the reason why you are confused. You may assume that both your training data and test data come from the same distribution &lt;span class=&quot;math-container&quot;&gt;$p(x, y)$&lt;/span&gt;, but it does not necessarily mean that they have the same patterns. For example, I can sample e.g. 13 numbers from a Gaussian &lt;span class=&quot;math-container&quot;&gt;$N(0, 1)$&lt;/span&gt;, the first 10 numbers could be very close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and the last &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; could be close to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. If you split this data so that your training data contains different patterns than the test data, then it is not guaranteed that you will perform well also on the test data.&lt;/p&gt;&#xA;&lt;p&gt;Finally, note that, in supervised learning, our ultimate goal when we fit models to labeled data is to learn a function (or a probability distribution over functions), where we often assume that both the training and test data are input-output pairs from our unknown target function, i.e. &lt;span class=&quot;math-container&quot;&gt;$y_i = f(x_i)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i) \in D$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is your labelled dataset), and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the &lt;strong&gt;unknown target function&lt;/strong&gt; (i.e. the function we want to compute with our model), so, if our model performs well on the training data but not on the test data and we assume that both training and test data come from the same function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, there is no way that our model is computing our target function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; if it performs badly on the test data.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-01T12:01:35.060" LastActivityDate="2020-09-01T12:01:35.060" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23380" PostTypeId="2" ParentId="23369" CreationDate="2020-09-01T13:03:16.413" Score="0" Body="&lt;p&gt;You are handling a &lt;em&gt;very&lt;/em&gt; small dataset. The only way to prevent overfitting then is to choose a very restrictive model search space. The simpler the better, and you should prefer models involving some regularization. Even tuning hyperparameters will be hard, so avoid families with many hyperparameters. Neural networks are definitely a no-go, IMHO.&lt;/p&gt;&#xA;&lt;p&gt;Cross-validation is important in this case, a.o. for tuning the regularization hyperparameter. You were right to perform the extra leave-one-out cross-validation. But remember checking the confidence intervals, which will be very broad.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest to use some prior business knowledge to further reduce the number of candidate predictor variables to 2-4. Then you can use a linear regression or a ridge regression (i.e. linear regression with L2 regularization, amount to be tuned as hyperparameter) using only these variables. If you have reasons to believe that the relation is very non-linear, you'll need to find another family.&lt;/p&gt;&#xA;&lt;p&gt;Another possibility is to perform a data-driven feature selection (see &lt;a href=&quot;https://topepo.github.io/caret/feature-selection-overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://topepo.github.io/caret/feature-selection-overview.html&lt;/a&gt;), but it is harder to properly implement &lt;strong&gt;and&lt;/strong&gt; cross-validate. I wouldn't recommend it to a beginner. Remember that this step can also introduce overfit or instability.&lt;/p&gt;&#xA;" OwnerUserId="27606" LastActivityDate="2020-09-01T13:03:16.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23381" PostTypeId="1" AcceptedAnswerId="23391" CreationDate="2020-09-01T13:06:27.527" Score="0" ViewCount="25" Body="&lt;p&gt;I have IMU (Inertial Measurment Unit- 6 axis) sensor data. The sensor attached on a car and 7 different drivers wipe on same path. I want to extract features and classify drivers. Which type of feature extractor do you guys suggest? I am planning to use PCA and Autoencoders but what do you think about classical signal properties to classify drivers?&lt;/p&gt;&#xA;" OwnerUserId="28129" LastActivityDate="2020-09-02T10:48:03.590" Title="Which type of feature extractor do you suggest to classify sensor data?" Tags="&lt;classification&gt;&lt;autoencoders&gt;&lt;time-series&gt;&lt;feature-extraction&gt;&lt;signal-processing&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23382" PostTypeId="1" CreationDate="2020-09-01T14:52:12.980" Score="0" ViewCount="16" Body="&lt;p&gt;I am fairly new to transformers and deep learning in general so please be kind,&lt;/p&gt;&#xA;&lt;p&gt;I am currently working on a project that will caption images using either Transformer-XL or BERT, however, I am not sure how to pass the image tensor that is [608, 608, 3] from my CNN to the transformer model for text generation, can anyone help?&lt;/p&gt;&#xA;&lt;p&gt;Please feel free to ask questions, I would be glad to assist in any way I can.&lt;/p&gt;&#xA;" OwnerUserId="40561" LastActivityDate="2020-09-01T14:52:12.980" Title="How to use an image tensor for caption generation with Transformer-XL or BERT?" Tags="&lt;natural-language-processing&gt;&lt;pytorch&gt;&lt;transformer&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23383" PostTypeId="1" CreationDate="2020-09-01T15:21:35.390" Score="0" ViewCount="27" Body="&lt;p&gt;I'm using CFR to solve a large imperfect-information game. One important technique for optimizing performance of this algorithm is &amp;quot;&lt;a href=&quot;https://www.cs.cmu.edu/%7Enoamb/papers/15-NIPS-Regret-Based.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;partial pruning&lt;/a&gt;&amp;quot;, which allows the algorithm to skip&#xA;updates for a player in a sequence if the other player’s current strategy does not reach the sequence&#xA;with positive probability.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me understand how to implement this? The problem I'm having is that the utility of the information set must still be computed recursively, even if there is no regret accumulated.&lt;/p&gt;&#xA;&lt;p&gt;For example, using the implementation provided &lt;a href=&quot;https://justinsermeno.com/posts/cfr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, the relevant section of the code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Utility of information set.&#xA;util = sum(action_utils * strategy)&#xA;regrets = action_utils - util&#xA;if is_player_1:&#xA;    info_set.regret_sum += pr_2 * pr_c * regrets&#xA;else:&#xA;    info_set.regret_sum += pr_1 * pr_c * regrets&#xA;return util&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If &lt;code&gt;pr_2&lt;/code&gt; is &lt;code&gt;0.0&lt;/code&gt; (at a node belonging to player 1), then it's true that &lt;code&gt;pr_2 * regrets&lt;/code&gt; will also be zero, making the value of &lt;code&gt;regrets&lt;/code&gt; irrelevant. However, we still need to compute and return &lt;code&gt;util&lt;/code&gt; for the node, which requires each child node to have been visited recursively (giving us &lt;code&gt;action_utils&lt;/code&gt;, which is needed to compute the final &lt;code&gt;util&lt;/code&gt; result).&lt;/p&gt;&#xA;&lt;p&gt;I must be missing something. What can I do to actually prune this subtree? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="14479" LastActivityDate="2020-09-01T15:21:35.390" Title="Partial pruning in counterfactual regret minimization (CFR)" Tags="&lt;optimization&gt;&lt;imperfect-information&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23384" PostTypeId="1" AcceptedAnswerId="23388" CreationDate="2020-09-01T18:46:47.300" Score="1" ViewCount="98" Body="&lt;p&gt;I wrote a Python program for a simple inventory control problem where decision epochs are equally divided (every morning) and there is no lead time for orders (the time between submitting an order until receiving the order). I use the Bellman equations, and solve them by policy iteration (Dynamic Programming).&lt;/p&gt;&#xA;&lt;p&gt;Now I want to consider lead time (between 1 to 3 days with equal probabilities). As I understand, the problem is defined by Semi Markov Decision Process for considering sojourn time in each state. I am confused about the Bellman equations in this scenario because we don't know exactly when the order will be received and is it necessary to discount the reward for day two or three?&lt;/p&gt;&#xA;" OwnerUserId="40719" LastEditorUserId="2444" LastEditDate="2020-09-04T18:13:43.630" LastActivityDate="2020-09-04T18:13:43.630" Title="Bellman optimality equation in semi Markov decision process" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;dynamic-programming&gt;&lt;semi-mdp&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23385" PostTypeId="1" CreationDate="2020-09-01T18:59:48.817" Score="0" ViewCount="41" Body="&lt;p&gt;I am implementing a simple backpropagation neural network for classifying images. One set of images are cars another set of images are buildings (houses). So far I have used Sobel Edge detector after converting the images from black and white. I need a way to remove the offset (in other words normalise the input) of where the car or where the house is in the image.&lt;/p&gt;&#xA;&lt;p&gt;Will taking the discrete Fourier cosine transform remove the offset? (so the input to the neural network will be the coefficients of the discrete cosine Fourier transform). To be clear, when I mean offset I mean a pair of values (across the number of pixels, and vertically the number of pixels) determining where the car or the building is in the 2D image from the origin.&lt;/p&gt;&#xA;" OwnerUserId="40467" LastEditorUserId="2444" LastEditDate="2020-09-04T15:55:24.327" LastActivityDate="2020-09-04T15:55:24.327" Title="How to normalise image input to backpropogation algorithm?" Tags="&lt;computer-vision&gt;&lt;backpropagation&gt;&lt;image-processing&gt;&lt;edge-detection&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23386" PostTypeId="2" ParentId="23370" CreationDate="2020-09-01T20:08:00.697" Score="0" Body="&lt;p&gt;You can also understand the logic from the view of constrained optimisation.&#xA;Introduce a Lagrange function:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{L} = \text{Tr} (w^{T} X X^{T} w) - \lambda w^{T} w&#xA;$$&lt;/span&gt;&#xA;And take the derivative with respect to &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial \mathcal{L}}{\partial w} = 2 (X X^{T} - \lambda) w&#xA;$$&lt;/span&gt;&#xA;For the general case of dimension &lt;span class=&quot;math-container&quot;&gt;$\geqslant 1$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is a set of vectors &lt;span class=&quot;math-container&quot;&gt;$w = (w_1 w_2 \ldots w_n)$&lt;/span&gt;.&#xA;This expression vanishes, if for some index &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; is an of eigenvector of &lt;span class=&quot;math-container&quot;&gt;$XX^{T}$&lt;/span&gt; with the eigenvalue &lt;span class=&quot;math-container&quot;&gt;$\lambda_i$&lt;/span&gt;, and all other components are set to zero. In other words, stationary points are the eigenvectors of &lt;span class=&quot;math-container&quot;&gt;$X X^{T}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The condititon &lt;span class=&quot;math-container&quot;&gt;$w^T w = 1$&lt;/span&gt; imposes the orthogonality condition on the eigenvectors. In fact, going back to the initial functional, one sees, that &lt;span class=&quot;math-container&quot;&gt;$w_i X X^{T} w_j = \lambda_j w_i^{T} w_j = 0$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;. Therefore, we have finally:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{L} =\sum \lambda_i - \lambda&#xA;$$&lt;/span&gt;&#xA;Which is maximized for any &lt;span class=&quot;math-container&quot;&gt;$k \geq 1$&lt;/span&gt;, by taking &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; largest eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="38846" LastEditorUserId="9947" LastEditDate="2020-09-01T23:15:52.020" LastActivityDate="2020-09-01T23:15:52.020" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23387" PostTypeId="2" ParentId="23296" CreationDate="2020-09-02T08:23:22.650" Score="0" Body="&lt;p&gt;I don't know about specific game titles but in terms of research University of Malta has a strong team working with application of machine learning to games. The key figure there used to be &lt;a href=&quot;http://yannakakis.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Georgios N. Yannakakis&lt;/a&gt; who published a lot of good papers and even wrote a &lt;a href=&quot;https://www.springer.com/gp/book/9783319635187&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; about content generation, smart game agents and, imho the most interesting, player modelling.&lt;/p&gt;&#xA;" OwnerUserId="38671" LastActivityDate="2020-09-02T08:23:22.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23388" PostTypeId="2" ParentId="23384" CreationDate="2020-09-02T09:53:02.330" Score="1" Body="&lt;p&gt;The core problem here is state representation, not estimating return due to delayed response to actions on the original state representation (which is no longer complete for the new problem). If you fix that, then you can solve your problem as a normal MDP, and base calculations on single timesteps. This allows you to continue using dynamic programming to solve it, provided the state space remains small enough.&lt;/p&gt;&#xA;&lt;p&gt;What needs to change is the state representation and state transitions. Instead of orders resulting in immediate change of stock levels, they become pending changes, and for each item you will have state representation for the amount of current stock, plus amount of stock in each lead time category. State transitions will modify expected lead time for each amount of pending stock as well as amount of current stock.&lt;/p&gt;&#xA;&lt;p&gt;Your lead time categories will depend on whether the agent knows the lead time immediately after making an order:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If lead times are known, track remaining time until items arrive 1,2 or 3 days. These categories will be assigned by the enviroment following the order, then lead time will transition down on each day deterministically. A 1 day lead time will transition to in stock, 2 day lead will transition to 1 day etc.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If lead times are not known, but probabilities of them are, track time since the order was made. This will be 0, 1 or 2 days. Although you don't know when an order will arrive, you know the probabilities for state transition - e.g. items in 0 days have a 1 in 3 chance of transitioning to &amp;quot;in stock&amp;quot; and a 2 in 3 chance of transitioning to 1 days.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This makes the state space larger, but is less complex than moving to the Semi MDP representation. For instance, doing it this way means that you can still work with single time step transitions and apply dynamic programming in a standard way.&lt;/p&gt;&#xA;&lt;p&gt;In general, if the environment has a delayed response to actions, then the  best way to maintain Markov trait is to add relevant history of actions taken to the state. The added state variables can either be a direct list of the relevant actions, or something that tracks the logical consequence of those actions.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-02T12:27:20.617" LastActivityDate="2020-09-02T12:27:20.617" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="23389" PostTypeId="5" CreationDate="2020-09-02T10:24:59.150" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-02T10:24:59.150" LastActivityDate="2020-09-02T10:24:59.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23390" PostTypeId="4" CreationDate="2020-09-02T10:24:59.150" Score="0" Body="For questions related to principal component analysis (PCA), which is commonly used in machine learning for dimensionality reduction." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-02T10:24:59.150" LastActivityDate="2020-09-02T10:24:59.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23391" PostTypeId="2" ParentId="23381" CreationDate="2020-09-02T10:48:03.590" Score="1" Body="&lt;p&gt;There could be multiple possible ways to extract the features. One would be to use RNNs for a temporal relationship as the input data is time-series.&lt;/p&gt;&#xA;" OwnerUserId="31016" LastActivityDate="2020-09-02T10:48:03.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23393" PostTypeId="2" ParentId="23375" CreationDate="2020-09-02T11:05:20.270" Score="1" Body="&lt;p&gt;I have not come across music labeling algorithms but upon a google &lt;a href=&quot;https://core.ac.uk/download/pdf/301311636.pdf#page=40&quot; rel=&quot;nofollow noreferrer&quot;&gt;scholar&lt;/a&gt; search, I found a couple of papers that aim to do quite the same task.&lt;/p&gt;&#xA;&lt;p&gt;In general, if you have a labeled dataset then you can take an approach of a general speech recognition model. It should work fine for music labeling too, but you might need to tweak certain parameters.&lt;/p&gt;&#xA;" OwnerUserId="31016" LastActivityDate="2020-09-02T11:05:20.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23394" PostTypeId="1" CreationDate="2020-09-02T13:57:06.573" Score="0" ViewCount="37" Body="&lt;p&gt;I'm trying to white board the different mechanisms behind a convolutional neural network.&#xA;I have on question regarding the dimension of my volume after using a max pooling layer.&#xA;Let's suppose I have a (21,21,#filtres) volume's dimension. If Max Pooling divide by 2 the height and width of my volume, what will be the dimension after the Max Pooling layer ?&#xA;If odd numbers are a problem when using max pooling layer, How do I fix it ?&lt;/p&gt;&#xA;&lt;p&gt;Thank you !&lt;/p&gt;&#xA;" OwnerUserId="40730" LastEditorUserId="40730" LastEditDate="2020-09-03T10:42:25.267" LastActivityDate="2021-01-31T14:05:10.510" Title="What is the dimension of my output of the form (2n + 1, 2n + 1, #filters) after a MaxPooling layer" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23395" PostTypeId="2" ParentId="23394" CreationDate="2020-09-02T14:27:47.330" Score="1" Body="&lt;p&gt;The result from applying a max pooling layer with a stride that does not exactly fit to the input will be dependent on the implementation in your library.&lt;/p&gt;&#xA;&lt;p&gt;Assuming stride 2, and pool size (2,2), in your case the most likely things are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The result will round up, so you will have a feature map layer with dimensions &lt;code&gt;(11, 11, num_filters)&lt;/code&gt; although the right edge pixels will be a max over 2 pixels in the input, and the right bottom corner will just be a copy of the right bottom corner in the input (counting from top left as &lt;span class=&quot;math-container&quot;&gt;$(0, 0)$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is an error condition for your libary.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If it is not an error, then the max pooling should still perform the task it was intended to. If important features are often at the right or bottom edge, then they may generalise slightly less well, but you probably won't notice a measurable effect.&lt;/p&gt;&#xA;&lt;p&gt;You could experiment with different sizes of pooling, different strides, or padding the previous convolutional layer so that the max pooling fits exactly. You will have to do something like this if the library has errors when the pooling does not fit exactly. You can test your experiments using cross validation, to see if there is any measurable difference.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-03T13:48:34.753" LastActivityDate="2020-09-03T13:48:34.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23396" PostTypeId="1" CreationDate="2020-09-02T14:52:36.123" Score="1" ViewCount="15" Body="&lt;p&gt;I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, for example, solving problems related to business processes?&lt;/p&gt;&#xA;&lt;p&gt;I want to know what characteristics of the data determine the applicability or non-applicability with respect to generating models for prediction where multiple domains are involved within an enterprise information database?&lt;/p&gt;&#xA;" OwnerUserId="40731" LastEditorUserId="2444" LastEditDate="2020-09-04T14:55:55.830" LastActivityDate="2020-09-04T14:55:55.830" Title="Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;applications&gt;&lt;transfer-learning&gt;&lt;domain-adaptation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23397" PostTypeId="2" ParentId="23296" CreationDate="2020-09-02T15:32:39.447" Score="0" Body="&lt;p&gt;Not sure if this fits your requirements of the AI playing &lt;em&gt;with&lt;/em&gt; the player, but I still wanted to mention it because to me it is the quintessential AI-based game:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://play.aidungeon.io/main/landing&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIDungeon&lt;/a&gt;, which is a text based story-telling game, where you can do literally anything. It's using GPT-2/GPT-3(paid) and has blown my mind several times. You've probably heard of it, but in case you haven't give it a try, only takes a couple of minutes to see what it can do.&lt;/p&gt;&#xA;" OwnerUserId="40732" LastActivityDate="2020-09-02T15:32:39.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23400" PostTypeId="1" CreationDate="2020-09-02T19:23:20.970" Score="0" ViewCount="61" Body="&lt;p&gt;What prerequisite knowledge should I have and what are the best resources to learn about neuromorphic chips/computing on a deep, technical level?&lt;/p&gt;&#xA;&lt;p&gt;I would like to understand everything from the transistor physics level all the way up to the high-level learning algorithms that such chips aim to execute.&lt;/p&gt;&#xA;&lt;p&gt;And after learning the prerequisites, what are some good resources to learn and understand the technical concepts behind this new technology?&lt;/p&gt;&#xA;" OwnerUserId="40734" LastEditorUserId="2444" LastEditDate="2020-09-02T23:27:54.623" LastActivityDate="2020-09-02T23:27:54.623" Title="What prerequisite knowledge should I have to learn about neuromorphic chips and computing?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;hardware&gt;&lt;neuromorphic-engineering&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="23402" PostTypeId="1" CreationDate="2020-09-02T20:04:51.040" Score="0" ViewCount="25" Body="&lt;p&gt;I am trying to create an evaluation function for a general game player based on the research from this article &lt;a href=&quot;http://www.mini.pw.edu.pl/%7Emandziuk/dynamic/wp-content/uploads/TCIAIG-2014-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Automatically-Generated Evaluation Function in General Game Playing&lt;/a&gt;. I can't understand the 2nd phase of this process.&lt;/p&gt;&#xA;&lt;p&gt;The evaluation function in this article consists of 4 phases.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Features identification (generalization and specialization)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Features analysis.&lt;/p&gt;&#xA;&lt;p&gt;More specifically, in the 2nd phase, we will make 100 random short game sequences from a starting position (let's suppose we play tic-tac-toe) and 10 rollouts from every sequence. Then we calculate the variance and stability on each feature based on their values, average values, etc.&lt;/p&gt;&#xA;&lt;p&gt;I would like to mention I don't even understand what the score of the feature supposed to be? How am I going to find it, and then how I will find the variance of the feature?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40116" LastEditorUserId="2444" LastEditDate="2021-02-06T20:43:41.540" LastActivityDate="2021-02-06T20:43:41.540" Title="How does the second phase of the evaluation function described in this article work?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;search&gt;&lt;monte-carlo-tree-search&gt;&lt;evaluation-functions&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23404" PostTypeId="1" CreationDate="2020-09-02T22:20:25.680" Score="0" ViewCount="70" Body="&lt;p&gt;What I want to achieve is this: If my desired outputs are [1, 2, 3, 4] I would rather have my network produce this output:&lt;/p&gt;&#xA;&lt;p&gt;[0.99, 2.01, 999, 4.01]&lt;/p&gt;&#xA;&lt;p&gt;than say this:&lt;/p&gt;&#xA;&lt;p&gt;[0.94, 1.88, 3.12, 4.1]&lt;/p&gt;&#xA;&lt;p&gt;So I'd rather have a few very accurate outputs and the rest completely off, than have them all be decent but no more than that. My question is, is there a known way to do this? If not, would it make sense to remove the inputs that produce poor outputs, and redo the learning phase?&lt;/p&gt;&#xA;" OwnerUserId="38668" LastActivityDate="2020-09-03T19:25:54.827" Title="Is there a way to make my neural network discard inputs with bad results from learning?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23405" PostTypeId="5" CreationDate="2020-09-02T23:24:45.263" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-02T23:24:45.263" LastActivityDate="2020-09-02T23:24:45.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23406" PostTypeId="4" CreationDate="2020-09-02T23:24:45.263" Score="0" Body="For theoretical questions about Google Translate." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-02T23:24:45.263" LastActivityDate="2020-09-02T23:24:45.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23407" PostTypeId="2" ParentId="23362" CreationDate="2020-09-02T23:37:39.500" Score="4" Body="&lt;p&gt;&lt;strong&gt;Auto-encoders are widely used&lt;/strong&gt; and maybe even more used than GANs (in fact, auto-encoders are older than GANs, although the main general idea behind GANs is quite old). For example, auto-encoders are used in &lt;a href=&quot;https://worldmodels.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Models&lt;/a&gt;, for drug design (e.g. see &lt;a href=&quot;http://proceedings.mlr.press/v80/jin18a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;) and many other tasks that involve data compression or generation.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So, if we train autoencoders, for example, on cats vs dogs dataset, and then cut off the decoder part and then input random noise vector, wouldn't it do the same job?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, the encoder part of the auto-encoder produces a vector that represents the input in a compressed form. You may also be interested in &lt;strong&gt;denoising auto-encoders&lt;/strong&gt;, but there are other variations, such as convolutional auto-encoders or variational auto-encoders.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-27T19:21:54.213" LastActivityDate="2020-09-27T19:21:54.213" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23408" PostTypeId="2" ParentId="23404" CreationDate="2020-09-03T03:34:02.173" Score="0" Body="&lt;p&gt;Use genetic algorithm. Run like 25 neural networks at once and choose the most successful  one. This method is similar to evolution, which is why it is very effective. I created a model like this with similar sized training data as yours and it reached an overall error rate of 0.06% in a second. Don’t get rid of nodes. Instead, eliminate the bad networks. However, this doesn’t produce extremely high error rates if that is what you want.&lt;/p&gt;&#xA;" OwnerUserId="40622" LastEditorUserId="40622" LastEditDate="2020-09-03T04:06:27.830" LastActivityDate="2020-09-03T04:06:27.830" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23409" PostTypeId="1" AcceptedAnswerId="23411" CreationDate="2020-09-03T09:00:55.203" Score="1" ViewCount="58" Body="&lt;p&gt;I am very new to the field of AI so please bear with me.&#xA;Say there is a dice with three sides, -1,0 and 1, and I want to predict which side it lands on (so only one output is needed I guess). The input variables are numerous but not that many, maybe 7-10.&lt;/p&gt;&#xA;&lt;p&gt;These input variables are certain formulae that involve calculations to do with wind, time, angle, momentum etc, and each formula returns which side it thinks the dice will like roll. Let's say that intuitively, by looking at these variables, I can make a very good guess at which side the dice lands on. If for example 6 out of 7 input variables say it likely that the dice will land on 1 but the 7th input suggests that it will land on 0, I would guess it lands on 1. As a human, I'm essentially consulting these inputs as a kind of &amp;quot;brains trust&amp;quot;, and I act as a judge to make the final decision based on the brains trust. Of course in that example, my logic as a judge was simply majority rules, but what if some other more complicated non-linear method of judging was needed?&lt;/p&gt;&#xA;&lt;p&gt;I essentially want my neural network to take this role as a judge. I have read that feedforward nns have limitations regarding control flow and loops, so I'm not sure if that structure will be appropriate. I'm not sure if recurrent nn will be appropriate either as I don't care what the previous inputs were.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="40744" LastActivityDate="2020-09-03T10:21:22.037" Title="What's a good neural network for this problem?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23410" PostTypeId="2" ParentId="23371" CreationDate="2020-09-03T09:33:01.650" Score="1" Body="&lt;p&gt;Simple old Latin is different from Latin and in the language words are added to written language that are not spoken as well as reverse order of words to have forward meaning.&lt;/p&gt;&#xA;" OwnerUserId="40746" LastActivityDate="2020-09-03T09:33:01.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23411" PostTypeId="2" ParentId="23409" CreationDate="2020-09-03T10:21:22.037" Score="3" Body="&lt;p&gt;A simple feed-forward neural network with at least one hidden layer would suffice in your problem, and can deal with arbitrary non-linear relationships between input and output. If you expect relationships to be highly non-linear then additional layers might be required, but from your description of the problem, I would be surprised if you needed more than few layers, and a relatively small network.&lt;/p&gt;&#xA;&lt;p&gt;However, I note that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The input variables are numerous but not that many, maybe 7-10.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This gives you &lt;span class=&quot;math-container&quot;&gt;$3^{10} = 59049$&lt;/span&gt; possible inputs. That's not much in terms of amount of data needed for ML statistical models. Assuming that even the best predictions are still probabilistic, then you may only need a million or so examples to create a reasonably accurate lookup table, not needing a neural network at all.&lt;/p&gt;&#xA;&lt;p&gt;The strength of a neural network is to be able to generalise well from less examples than that. Of course, this is not perfect, but it would be able to do things such as notice if inputs 1,2 and 3 agree then that is always the most likely answer. If that turns out to be true (and not an accident of having low numbers of samples), then the NN could learn that useful pattern using far less data than a table-based approach.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have read that feedforward nns have limitations regarding control flow and loops, so I'm not sure if that structure will be appropriate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is true, but does not impact your situation, because there is no control flow or loops involved. You have described a simple function. Whilst you or I might inspect the data and look backwards and forwards across it before coming to a decision, a neural network approximating a function does not need to do that, and in simple cases there is usually no benefit to doing so - a statistical summary of the correct mapping from input to output is more than sufficient and likely the best that can be done.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I'm not sure if recurrent nn will be appropriate either as I don't care what the previous inputs were.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As all your inputs represent the same kind of thing, you could implement as a RNN with a single input, -1, 0 or +1, always feeding in the predictions by type in the same order. It might resemble how you are thinking about the problem as a human (at least a better analogy than the direct statistical match in a feed-forward network), especially if you implemented a variant of attention. However, I don't think there would be any benefit to that in improved accuracy, and it would be a significant challenge to build that if you are new to AI.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-09-03T10:21:22.037" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23412" PostTypeId="1" CreationDate="2020-09-03T15:37:02.850" Score="2" ViewCount="46" Body="&lt;p&gt;I am studying computer vision for the past 3 months. I have come across the object identification problem, where given an image, CV would identify various parts in the image.&lt;/p&gt;&#xA;&lt;p&gt;If I give an image, and a rectangle coordinates, can CV identify the parts' names within that rectangle? For example, can I train a model to identify the parts in the below image (mountain, river, in this case)? The model should not identify other parts like flowers, sky, etc., as they come outside the rectangle).&lt;/p&gt;&#xA;&lt;p&gt;I tried searching but could not find similar problems. Can anyone give me a direction to solve this problem?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sjbNp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sjbNp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40754" LastEditorUserId="2444" LastEditDate="2020-09-04T18:49:20.157" LastActivityDate="2021-05-19T15:01:04.020" Title="Can we identify only the objects in specific parts of an image with computer vision?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23413" PostTypeId="2" ParentId="23377" CreationDate="2020-09-03T17:13:19.137" Score="0" Body="&lt;p&gt;Well, here's what I ended up with of late :&lt;/p&gt;&#xA;&lt;p&gt;My model is completely psychological (or philosophical - no idea which to choose) and hence is theoretically possible but needn't be so mathematically (I am not sure of that part, so I leave it to you to verify that in your upcoming answers).&lt;/p&gt;&#xA;&lt;p&gt;Therefore, there must be some changes in the terminology as well as procedures:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Instead of 'gravitons' , the nodes can preferably be called 'thought centers' , taking them to be the centers of the models perception of what class the object may belong to.&lt;/li&gt;&#xA;&lt;li&gt;Editing weights randomly while training (&lt;strong&gt;NB:I decided to edit the training algorithm&lt;/strong&gt;) and using them in randomly during the transformation of the input to the output - which I call 'shuffle' (number of times a weight used needn't be taken into consideration - &lt;em&gt;&lt;strong&gt;'freedom of thought granted to AI !'&lt;/strong&gt;&lt;/em&gt;) - is something which makes the model really vague when it comes to the flow of data through it, but still I prefer it, as it is basically a guesswork machine and since the guess choices can be limited by using at most &lt;em&gt;&lt;strong&gt;2 'thought centers' (4 choices - if you go for a single node, it will always yield &lt;span class=&quot;math-container&quot;&gt;$\varphi(wx)$&lt;/span&gt; for a constant input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; [ &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; = weight, &lt;span class=&quot;math-container&quot;&gt;$\varphi(x) = \frac{1}{1 + e^{-x}}$&lt;/span&gt;])&lt;/strong&gt;&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The code I posted here was a bit faulty, so I decided to rebuild it. The code will be posted in my GitHub profile, and if any of you would like to try it, you may download it from there (I am not good at making Python packages and I haven't made one till now, I'll be putting it up there in the &lt;code&gt;master&lt;/code&gt; branch itself. I will tell you of the release as an edit in this answer).I built that code from scratch, integrated it into a class (&lt;code&gt;Insane_Mind&lt;/code&gt;) and tried that on the iris dataset (not the testing part, but the training part at this time of development)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Again, if anything seems ambiguous, please tell me.&lt;/p&gt;&#xA;" OwnerUserId="40583" LastActivityDate="2020-09-03T17:13:19.137" CommentCount="17" ContentLicense="CC BY-SA 4.0" />
  <row Id="23415" PostTypeId="1" CreationDate="2020-09-03T19:07:51.527" Score="2" ViewCount="40" Body="&lt;p&gt;I am attempting to solve a timetabling problem using deep Q learning. It could be thought of as a resource allocation problem to obtain some certificate of 'optimality'. However, how to define and access the action space is alluding me. Any help, thoughts, or direction towards the literature would be appreciated. Thanks!&lt;/p&gt;&#xA;&lt;p&gt;The problem is entirely deterministic, the pair of the current state and action is isomorphic to the resulting state. The Q network is therefore being set up to approximate a Q value (a scalar) for the resulting state, i.e. for the current state and proposed action.&lt;/p&gt;&#xA;&lt;p&gt;I have so far assumed that the action space should be randomly sampled during training to generate some approximation of the Q table. This seems highly inefficient.&lt;/p&gt;&#xA;&lt;p&gt;I am open to reinterpretations of the action space. The problem involves a set of n individuals and at any given state a maximum of b can be 'active' and, of the remaining 'inactive' individuals, f can be made 'active' by an action. An action will need to involve making some reallocation to active individuals made up of those who are already active and the other f available people.&lt;/p&gt;&#xA;&lt;p&gt;To give you a sense over the numbers that I will ultimately use, &lt;span class=&quot;math-container&quot;&gt;$n=17, b=7$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; will hover somewhere around 7-10 (but depends on the allocations). At first this sounds tractable, but a (very) rough approximation of the cardinality of the set of actions is 17 choose 7 = 19448.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know a more efficient way to encode this action space? If not, is there a more sensible way to sample it (as is my current plan) than uniformly extracting actions from the space? Also when sampling the space is it valid to enforce some cap on the number of samples drawn (say 500). Please feel free to ask for further clarification.&lt;/p&gt;&#xA;" OwnerUserId="40758" LastActivityDate="2020-09-03T19:07:51.527" Title="Handling a Large Discrete Action Space in Deep Q Learning" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23416" PostTypeId="1" CreationDate="2020-09-03T19:20:07.363" Score="0" ViewCount="50" Body="&lt;p&gt;I would like to ask if &lt;em&gt;Multi-Agent Deep Deterministic Policy Gradient (MADDPG)&lt;/em&gt; works fine with discrete action space.&lt;/p&gt;&#xA;&lt;p&gt;DDPG works only with continuous action space, but I have read that MADDPG can also work with discrete actions.&lt;/p&gt;&#xA;&lt;p&gt;Is that true? Does that make any sense?&lt;/p&gt;&#xA;" OwnerUserId="40477" LastEditorUserId="2444" LastEditDate="2020-09-03T21:51:46.587" LastActivityDate="2020-09-03T21:51:46.587" Title="Does Multi-Agent Deep Deterministic Policy Gradient also work with discrete action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;markov-decision-process&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23417" PostTypeId="2" ParentId="23404" CreationDate="2020-09-03T19:25:54.827" Score="3" Body="&lt;p&gt;I assume [1, 2, 3, 4] are the desired outputs for different examples for a regression task. Sound like you need a different loss function. From your description it seems you don't care how big the error is if it's bigger than some value. Try the &lt;a href=&quot;https://en.wikipedia.org/wiki/Huber_loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;Huber loss&lt;/a&gt;(in &lt;a href=&quot;https://pytorch.org/docs/stable/generated/torch.nn.SmoothL1Loss.html#smoothl1loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch&lt;/a&gt; and &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/losses/Huber&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;). Examples that are far from the expected value won't produce big gradients (:&lt;/p&gt;&#xA;" OwnerUserId="40597" LastActivityDate="2020-09-03T19:25:54.827" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23418" PostTypeId="1" CreationDate="2020-09-03T19:40:30.553" Score="3" ViewCount="46" Body="&lt;p&gt;Does anyone know of research involving the GPT models to learn not only regular texts, but also learn from physics books with the equations written in latex format?&lt;/p&gt;&#xA;&lt;p&gt;My intuition is that the model might learn the rules relating equations and deductions, as they can learn statistically what correlates with what. I understand that the results can also be a little nonsensical, like the sometimes surreal paragraphs written by these models.&lt;/p&gt;&#xA;&lt;p&gt;Have there been any attempts to do this?&lt;/p&gt;&#xA;" OwnerUserId="30433" LastEditorUserId="5763" LastEditDate="2020-09-06T01:27:27.700" LastActivityDate="2020-09-06T01:27:27.700" Title="Can in principle GPT language models learn physics?" Tags="&lt;gpt&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23419" PostTypeId="1" CreationDate="2020-09-03T23:58:01.983" Score="1" ViewCount="35" Body="&lt;p&gt;Do you know of some paper mentioning a &lt;em&gt;running&lt;/em&gt; AI system (preferably open source) that helps finding funding for AI software development?&lt;/p&gt;&#xA;&lt;p&gt;Obviously, be funded for working on AI is not easy, and could be a problem approached with AI techniques.&lt;/p&gt;&#xA;&lt;p&gt;As a concrete example, calls like &lt;a href=&quot;https://en.wikipedia.org/wiki/Framework_Programmes_for_Research_and_Technological_Development#Horizon_2020&quot; rel=&quot;nofollow noreferrer&quot;&gt;H2020&lt;/a&gt; could be approached by some kind of AI tools (e.g. NLP based ones)&lt;/p&gt;&#xA;&lt;p&gt;PS. The system I have in mind is &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; - I am trying to get funded on it, and it is a difficult problem, which suggest some AI approach.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-11-17T15:23:29.367" LastActivityDate="2020-11-17T15:23:29.367" Title="Any AI software to help finding funding for AI projects?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;software-development&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23420" PostTypeId="1" CreationDate="2020-09-04T00:32:55.703" Score="1" ViewCount="31" Body="&lt;p&gt;I've been working on the Punctuation Restoration Problem for my Master's Thesis, however, me being primarily a programmer at heart, I wish I could use some of my NLP skills to solve issues related to programming in general.&lt;/p&gt;&#xA;&lt;p&gt;I know Microsoft does lots of research in NLP and I think after they acquired Github, they have an immense dataset to work with for any problems related to programming they want to tackle. Most recently I think they did a great job on their new python suggestion extension on VSCode.&lt;/p&gt;&#xA;&lt;p&gt;So, could you suggest to me some issues you think are interesting research topics? This is something that I would like to work with, but I have no idea where to start yet.&lt;/p&gt;&#xA;" OwnerUserId="22621" LastActivityDate="2020-09-04T00:32:55.703" Title="What are some programming related topics that can be solved using NLP?" Tags="&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23424" PostTypeId="1" CreationDate="2020-09-04T14:08:25.093" Score="1" ViewCount="50" Body="&lt;p&gt;Training neural networks takes a while. My question is, how efficient is a neural network that is completely trained (assuming it's not a model that is constantly learning)?&lt;/p&gt;&#xA;&lt;p&gt;I understand that this is a vague and simply difficult question to answer, so let me be more specific: Imagine we have a trained Deep Neural Net, and even to be more specific it's a GPT-3 model.&lt;/p&gt;&#xA;&lt;p&gt;Now, we put the whole thing on a Raspberry Pi. No internet access. The whole process takes place locally.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Will it run at all? Will it have enough RAM?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now let's say we give it some text to analyze. Then we ask it a question. Will it take milliseconds to answer? Or is it going to be in the seconds? Minutes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What I'm trying to understand, once a model is trained is it fairly performant because it's just essentially a bunch of very simple function calls on top of each other, or is it very heavy to execute? (perhaps due to the sheer number of these simple function calls)&lt;/p&gt;&#xA;&lt;p&gt;Please correct any misunderstanding about how the whole process works if you spot any. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="40771" LastEditorUserId="2444" LastEditDate="2020-09-07T10:28:32.650" LastActivityDate="2020-09-07T10:28:32.650" Title="What is the efficiency of trained neural networks?" Tags="&lt;neural-networks&gt;&lt;gpt&gt;&lt;efficiency&gt;&lt;computational-complexity&gt;&lt;benchmarks&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23425" PostTypeId="2" ParentId="5318" CreationDate="2020-09-04T17:06:31.720" Score="1" Body="&lt;p&gt;Let's start with understanding what over-fitting means. Your model is over-fitting if during training your training loss continues to decrease but (in the later epochs) your validation loss begins to increase. That means the model can not generalize well to images it has not previously encountered.&lt;/p&gt;&#xA;&lt;p&gt;Naturally, you do not want this situation. What you want is a high training accuracy and a very low validation loss, which implies a high validation accuracy.&lt;/p&gt;&#xA;&lt;p&gt;The first task is to ensure that your model gets a high training accuracy. Once that is accomplished, you can work on getting a low validation loss.&lt;/p&gt;&#xA;&lt;p&gt;If your model is overfitting, there are several ways to mitigate the problem. First, start out with a simple model. If you have a lot of dense layers with a lot of neurons, reduce the hidden dense layers to a minimum. Typical just leave the top dense layer used for final classification. Then see how the model trains. If it trains well, look at the validation loss and see if it is reducing in the later epochs. If the model does not train well, add a dense layer followed by a dropout layer. Use the level of dropout to adjust for overfitting. If it still trains poorly, increase the number of neurons, and train again. If that fails, add another dense hidden layer with fewer neurons than the previous layer followed by another dropout layer.&lt;/p&gt;&#xA;&lt;p&gt;Another method to combat overfitting is to add regularizers to the dense layers. Documentation for that is &lt;a href=&quot;https://keras.io/api/layers/regularizers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="33976" LastEditorUserId="2444" LastEditDate="2020-10-04T20:25:43.173" LastActivityDate="2020-10-04T20:25:43.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23427" PostTypeId="2" ParentId="11375" CreationDate="2020-09-04T21:22:03.667" Score="2" Body="&lt;p&gt;The same book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=36&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement learning: an introduction&lt;/a&gt; (2nd edition, 2018) by Sutton and Barto has a section, &lt;strong&gt;1.7 Early History of Reinforcement Learning&lt;/strong&gt;, that describes what &lt;strong&gt;optimal control&lt;/strong&gt; is and how it is related to reinforcement learning. I will quote the most relevant part to answer your question, but you should read all that section to have a full understanding of the relationship between optimal control and reinforcement learning.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The term &amp;quot;optimal control&amp;quot; came into use in the late 1950s to describe the &lt;strong&gt;problem of designing a controller to minimize or maximize a measure of a dynamical system's behavior over time&lt;/strong&gt;. One of the &lt;strong&gt;approaches&lt;/strong&gt; to this problem was developed in the mid-1950s by &lt;strong&gt;Richard Bellman&lt;/strong&gt; and others through extending a nineteenth-century theory of Hamilton and Jacobi. This approach uses the concepts of a dynamical system's &lt;strong&gt;state&lt;/strong&gt; and of a &lt;strong&gt;value function&lt;/strong&gt;, or &amp;quot;optimal return function&amp;quot;, to define a functional equation, now often called the &lt;strong&gt;Bellman equation&lt;/strong&gt;. The &lt;strong&gt;class of methods&lt;/strong&gt; for solving optimal control problems by solving this equation came to be known as &lt;strong&gt;dynamic programming&lt;/strong&gt; (Bellman, 1957a). Bellman (1957b) also introduced the &lt;strong&gt;discrete stochastic version of the optimal control problem&lt;/strong&gt; known as &lt;strong&gt;Markov decision processes (MDPs)&lt;/strong&gt;. Ronald Howard (1960) devised the &lt;strong&gt;policy iteration&lt;/strong&gt; method for MDPs. &lt;strong&gt;All of these are essential elements underlying the theory and algorithms of modern reinforcement learning&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To answer your specific questions.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In optimal control we have, controllers, sensors, actuators, plants, etc, as elements. Are these different names for similar elements in deep RL? For example, would an optimal control plant be called an environment in deep RL?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. In reinforcement learning (see the first footnote of the cited book on page 48), the term &lt;a href=&quot;https://ai.stackexchange.com/q/13119/2444&quot;&gt;&lt;em&gt;control&lt;/em&gt;&lt;/a&gt; is often used as a synonym for &lt;em&gt;action&lt;/em&gt;. Similarly, the term &lt;em&gt;controller&lt;/em&gt; (or &lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf#page=539&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;decision maker&lt;/em&gt;&lt;/a&gt;) is used as a synonym for &lt;em&gt;agent&lt;/em&gt; (and &lt;a href=&quot;https://ai.stackexchange.com/a/26296/2444&quot;&gt;sometimes also a synonym for &lt;em&gt;policy&lt;/em&gt;&lt;/a&gt;, given that the policy usually defines and &lt;em&gt;controls&lt;/em&gt; the agent, although the concept of the agent is more abstract and we could associate more than one policy with the same agent). The term &lt;em&gt;environment&lt;/em&gt; is also used as a synonym for &lt;em&gt;controlled system&lt;/em&gt; (or &lt;em&gt;plant&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf#page=539&quot; rel=&quot;nofollow noreferrer&quot;&gt;section &lt;strong&gt;38.8 Notes&lt;/strong&gt; (page 530)&lt;/a&gt; of the book &lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf#page=539&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit Algorithms&lt;/a&gt; by Csaba Szepesvari and Tor Lattimore.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-11T19:18:00.817" LastActivityDate="2021-02-11T19:18:00.817" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23428" PostTypeId="1" CreationDate="2020-09-04T21:47:59.250" Score="2" ViewCount="38" Body="&lt;p&gt;I have not found a lot of information on this, but I am wondering if there is a standard way to apply the outputs of a Bert model being used for sentiment analysis, and connect them back to the initial tokenized string of words, to gain an understanding of which words impacted the outcome of the sentiment most.&lt;/p&gt;&#xA;&lt;p&gt;For example, the string &amp;quot;this coffee tastes bad&amp;quot; outputs a negative sentiment.  Is it possible to analyze the output of the hidden layers to then tie those results back to each token to gain an understanding of which words in the sentence had the most influence on the negative sentiment?&lt;/p&gt;&#xA;&lt;p&gt;The below chart is a result at my attempt to explore this, however I am not sure it makes sense and I do not think I am interpreting it correctly. I am basically taking the outputs of the last hidden layer, which in this case has shape (1, 7, 768), [CLS] + 5 word tokens + [SEP], and looping through each token summing up their values (768) and computing the average. The resulting totals are outputted in the below graph.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wSYIU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wSYIU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any thoughts around if there is any meaning to this or if i am way off on approach, would be appreciated.  Might be my misunderstanding around the actual output values themselves.&lt;/p&gt;&#xA;&lt;p&gt;Hopefully this is enough to give someone the idea of what i am trying to do and how each word can be connected to positive or negative associations that contributed to the final classification.&lt;/p&gt;&#xA;" OwnerUserId="40782" LastEditorUserId="40782" LastEditDate="2020-09-04T23:50:43.797" LastActivityDate="2020-09-04T23:50:43.797" Title="Bert for Sentiment Analysis - Connecting final output back to the input" Tags="&lt;bert&gt;&lt;sentiment-analysis&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23429" PostTypeId="1" AcceptedAnswerId="23430" CreationDate="2020-09-05T07:56:55.750" Score="1" ViewCount="176" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=106&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example 4.3:Gambler's Problem of Sutton and Barto's book&lt;/a&gt; whose code is given &lt;a href=&quot;https://github.com/ShangtongZhang/reinforcement-learning-an-introduction/blob/master/chapter04/gamblers_problem.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;In this code the value function array is initialized as &lt;code&gt;np.zeros(states)&lt;/code&gt; where &lt;code&gt;states&lt;/code&gt; &lt;span class=&quot;math-container&quot;&gt;$\in[0,100]$&lt;/span&gt; and the value function for optimal policy which is returned after solving it with value iteration is same as the one given in the book, but, if we only change the initialization of the value function in the code, suppose to &lt;code&gt;np.ones(states)&lt;/code&gt; then the optimal value function returned changes too, which means that the value iteration algorithm converges in both the cases but to different optimal value functions,but two different optimal value function is impossible in a MDP. So why is the value iteration algorithm not converging to optimal value function?&lt;/p&gt;&#xA;&lt;p&gt;PS: If we change the initialization of value function array to &lt;code&gt;-1*np.random.rand(states)&lt;/code&gt;, then the converged optimal value function also contains negative numbers which should be impossible as &lt;code&gt;rewards&amp;gt;=0&lt;/code&gt;, hence value iteration fails to converge to optimal value function.&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="40583" LastEditDate="2020-09-06T11:45:20.640" LastActivityDate="2020-09-06T11:45:20.640" Title="Value Iteration failing to converge to optimal value function in Sutton-Barto's Gambler problem" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;sutton-barto&gt;&lt;value-iteration&gt;&lt;numpy&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23430" PostTypeId="2" ParentId="23429" CreationDate="2020-09-05T08:33:29.110" Score="3" Body="&lt;p&gt;So, naturally, if you've observed something that contradicts the theoretical properties of Value Iteration, something's wrong, right?&lt;/p&gt;&#xA;&lt;p&gt;Well, the code you've linked, as it is, is fine. It works as intended when all the values are initialized to zero. HOWEVER, my guess is that you're the one introducing an (admittedly very subtle) error. I think you're changing this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;state_value = np.zeros(GOAL + 1)&#xA;state_value[GOAL] = 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;state_value = np.ones(GOAL + 1)&#xA;state_value[GOAL] = 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, you see, this is wrong. And the reason why it's wrong is that both GOAL (which is 100 in the example) and 0 must have an immutable and fixed values, because they're terminal states, and their values are not subject to estimation. The value for GOAL is 1.0, as you can see in the original code. If you want initial values other than 0, then you must do this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;state_value = np.ones(GOAL + 1)&#xA;state_value[GOAL] = 1.0&#xA;state_value[0] = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the first case (changing the initial values to 1) what you were seeing was, essentially, an &amp;quot;I don't care policy&amp;quot;. Whatever you do, you'll end with a value of 1. In the second case, with the random values, you saw the classic effects of &amp;quot;garbage in, garbage out&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="37359" LastActivityDate="2020-09-05T08:33:29.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23433" PostTypeId="1" CreationDate="2020-09-05T12:32:16.680" Score="0" ViewCount="75" Body="&lt;p&gt;I know we have developed some mathematical tools to understand deep neural networks, gradient descent for optimization, and basic calculus. Recently, I encountered arxiv paper that describes higher mathematics for neural networks, such as functional analysis. For example, I remember universal approximation theorem was proved with the Hann-Banach theorem, but I lost the link of that article, so I need to find similar papers or articles to develop my understanding of neural networks mathematically (like with functional analysis, in short, I need to learn more advanced math for research), can you suggest some books or arxiv papers or articles or any other source that describes mathematics for deep neural networks?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-09-05T21:26:57.667" LastActivityDate="2020-09-05T21:26:57.667" Title="What are the mathematical prerequisites needed to understand research papers on neural networks?" Tags="&lt;neural-networks&gt;&lt;deep-neural-networks&gt;&lt;research&gt;&lt;resource-request&gt;" AnswerCount="1" CommentCount="11" ClosedDate="2020-09-05T19:45:02.860" ContentLicense="CC BY-SA 4.0" />
  <row Id="23434" PostTypeId="2" ParentId="23433" CreationDate="2020-09-05T15:24:01.277" Score="1" Body="&lt;p&gt;Knowing you want to focus on the theory, I think that a good choice is &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning Book&lt;/a&gt; from Ian Goodfellow et al., which is publicly available. It has three main parts. On the first one the author presents the math/ statistic tools that will be needed to understand the following parts. On the second part, the author explains the current state of the art in Deep Learning and on the last part more advanced topics are introduced. The author also uses references so as to facilitate extra resources to dive more into the theory.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand I strongly recommend you Google Scholar, there you have plenty of information given by articles/ papers of the current state of the art techniques in the field. There you can also find papers related on what that you mentioned about the &lt;a href=&quot;https://scholar.google.es/scholar?hl=es&amp;amp;as_sdt=0,5&amp;amp;q=Hahn-Banach+theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hahn-Banach Theorem&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40612" LastActivityDate="2020-09-05T15:24:01.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23435" PostTypeId="1" CreationDate="2020-09-05T16:10:27.817" Score="1" ViewCount="102" Body="&lt;p&gt;I have been reading :&#xA;&lt;a href=&quot;http://incompleteideas.net/book/the-book-2nd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction by Sutton and Barto&lt;/a&gt;.&#xA;I admit it's a good read for learning RL whereas it's more theoretical with detailed algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Now, &lt;strong&gt;I want something more programming oriented resource(s)&lt;/strong&gt; maybe a course, book, etc.&#xA;I have been exploring Kaggle, Open-source RL projects.&lt;/p&gt;&#xA;&lt;p&gt;I need this to learn and grasp a deeper understanding of RL from the perspective of a developer i.e optimized way of writing code, explanation about using the latest RL libraries, cloud services, etc.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="40485" LastEditorUserId="2444" LastEditDate="2020-09-05T19:36:54.207" LastActivityDate="2020-09-08T08:50:34.753" Title="What are some programming-oriented resources for reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;resource-request&gt;" AnswerCount="2" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23438" PostTypeId="2" ParentId="23435" CreationDate="2020-09-05T19:33:29.860" Score="2" Body="&lt;p&gt;Arthur Juliani has some interesting Medium articles on reinforcement learning with TensorFlow backed up with code on &lt;a href=&quot;https://github.com/awjuliani/DeepRL-Agents&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 0 — Q-Learning Agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/super-simple-reinforcement-learning-tutorial-part-1-fd544fab149&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 1 — Two-Armed Bandit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-1-5-contextual-bandits-bff01d1aad9c&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 1.5 — Contextual Bandits&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/super-simple-reinforcement-learning-tutorial-part-2-ded33892c724&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 2 — Policy-Based Agents&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-3-model-based-rl-9a6fe0cce99&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 3 — Model-Based RL&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-4-deep-q-networks-and-beyond-8438a3e2b8df&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 4 — Deep Q-Networks and Beyond&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-5-visualizing-an-agents-thoughts-and-actions-4f27b134bb2a&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 5 — Visualizing an Agent’s Thoughts and Actions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-6-partial-observability-and-deep-recurrent-q-68463e9aeefc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 6 — Partial Observability and Deep Recurrent Q-Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-7-action-selection-strategies-for-exploration-d3a97b7cceaf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 7 — Action-Selection Strategies for Exploration&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-8-asynchronous-actor-critic-agents-a3c-c88f72a5e9f2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Part 8 — Asynchronous Actor-Critic Agents (A3C)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As &lt;a href=&quot;https://ai.stackexchange.com/users/2444/nbro&quot;&gt;nbro&lt;/a&gt; pointed out Denny Britz has a good repository: &lt;a href=&quot;https://github.com/dennybritz/reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dennybritz/reinforcement-learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you have seen with Sutton &amp;amp; Barto's book the code is mostly in Lisp.  Shangtong Zhang has replicated the code in Python: &lt;a href=&quot;https://github.com/ShangtongZhang/reinforcement-learning-an-introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ShangtongZhang/reinforcement-learning-an-introduction&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sudharsan and Ravichandiran wrote a book &amp;quot;Hands-On Reinforcement Leraning with Python&amp;quot; which uses OpenAI Gym and TensorFlow.  You can find more information on the book along with their code repository at &lt;a href=&quot;https://github.com/sudharsan13296/Hands-On-Reinforcement-Learning-With-Python&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hands-On Reinforcement Learning with Python&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-09-06T00:24:47.317" LastActivityDate="2020-09-06T00:24:47.317" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23440" PostTypeId="1" CreationDate="2020-09-06T03:45:41.807" Score="2" ViewCount="37" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wQGit.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wQGit.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Above is the algorithm for Policy Iteration from Sutton's RL book. So, step 2 actually looks like value iteration, and then, at step 3 (policy improvement), if the policy isn't stable it goes back to step 2.&lt;/p&gt;&#xA;&lt;p&gt;I don't really understand this: it seems like, if you do step 2 to within a small &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt;, then your estimate of the value function should be pretty close to optimal for each state.&lt;/p&gt;&#xA;&lt;p&gt;So, why would you need to visit it again after policy improvement?&lt;/p&gt;&#xA;&lt;p&gt;It seems like policy improvement only improves the policy function, but that doesn't affect the value function, so I'm not sure why you'd need to go back to step 2 if the policy isn't stable.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-09-07T10:18:06.947" LastActivityDate="2020-09-08T08:04:44.350" Title="Why do we need to go back to policy evaluation after policy improvement if the policy is not stable?" Tags="&lt;reinforcement-learning&gt;&lt;value-iteration&gt;&lt;policy-iteration&gt;&lt;policy-evaluation&gt;&lt;policy-improvement&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23441" PostTypeId="1" CreationDate="2020-09-06T06:12:50.670" Score="1" ViewCount="55" Body="&lt;p&gt;Let's take a 32 x 32 x 3 NumPy array and convolve with 10 filters of size 2 x 2 x 3 with stride 2 to produce feature maps of volume 16 x 16 x 10. The total number of operations - 16 * 16 * 10 * 2 * 2 * 2 * 3 = 61440 operations. Now, let's take an input array of length 3072 (flattening the 32 * 32 * 3 array) and dot it with a weight matrix of size 500 x 3072. The total number of operations - 500 * 3072 * 2 = 3072000 operations. The convolution takes 4-5 times longer than &lt;code&gt;np.dot(w, x)&lt;/code&gt; even though number of operations is less.&lt;/p&gt;&#xA;&lt;p&gt;Here's my code for the convolution operation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(16):&#xA;    for j in range(16):&#xA;        for k in range(10):&#xA;            v[i, j, k] = np.sum(x[2 * i:2 * i + 2, 2 * j:2 * j + 2] * kernels[k]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is &lt;code&gt;np.dot(w, x)&lt;/code&gt; optimized or something? Or are my calculations wrong? Sorry if this is a silly question...&lt;/p&gt;&#xA;" OwnerUserId="38343" LastActivityDate="2020-09-06T20:36:44.023" Title="Why does CNN forward pass take longer compared to MLP forward pass?" Tags="&lt;convolutional-neural-networks&gt;&lt;time-complexity&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-09-07T13:22:24.557" ContentLicense="CC BY-SA 4.0" />
  <row Id="23442" PostTypeId="1" AcceptedAnswerId="23445" CreationDate="2020-09-06T06:21:56.487" Score="0" ViewCount="47" Body="&lt;p&gt;The topologies (or architectures) of the neural networks that I have seen so far are only 2-dimensional. So, are there neural networks whose topology is 3-dimensional (i.e. they have a width, height, and depth)?&lt;/p&gt;&#xA;" OwnerUserId="40583" LastEditorUserId="2444" LastEditDate="2020-09-07T13:55:44.843" LastActivityDate="2020-09-07T13:55:44.843" Title="Are there neural networks with 3-dimensional topologies?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23443" PostTypeId="2" ParentId="22591" CreationDate="2020-09-06T07:10:25.853" Score="2" Body="&lt;p&gt;The &lt;code&gt;Flatten&lt;/code&gt; layer has no learnable parameters in itself (the operation it performs is fully defined by construction); still, it has to propagate the gradient to the previous layers.&lt;/p&gt;&#xA;&lt;p&gt;In general, the &lt;code&gt;Flatten&lt;/code&gt; operation is well-posed, as whatever is the input shape you know what the output shape is.&lt;/p&gt;&#xA;&lt;p&gt;When you backpropagate, you are supposed to do an &amp;quot;Unflatten&amp;quot;, which maps a flattened tensor into a tensor of a given shape, and you know what that specific shape is from the forward pass, so it is also a well-posed operation.&lt;/p&gt;&#xA;&lt;p&gt;More formally&lt;/p&gt;&#xA;&lt;p&gt;Say you have &lt;code&gt;Img1&lt;/code&gt; in input of your &lt;code&gt;Flatten&lt;/code&gt; layer&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;f_{1,1}(x; w_{1,1}) &amp;amp; f_{1,2}(x; w_{1,2}) \\ &#xA;f_{2,1}(x; w_{2,1}) &amp;amp; f_{2,2}(x; w_{2,2})&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, in the output you have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;f_{1,1}(x; w_{1,1}) &amp;amp; f_{1,2}(x; w_{1,2}) &amp;amp; f_{2,1}(x; w_{2,1}) &amp;amp; f_{2,2}(x; w_{2,2})&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When you compute the gradient you have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{df_{i,j}(x; w_{i,j})}{dw_{i,j}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and everything in the same position as in the forward pass, so the unflatten maps from the &lt;code&gt;(1, 4)&lt;/code&gt; tensor to the &lt;code&gt;(2, 2)&lt;/code&gt; tensor.&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="2444" LastEditDate="2020-09-10T21:38:13.503" LastActivityDate="2020-09-10T21:38:13.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23444" PostTypeId="2" ParentId="23440" CreationDate="2020-09-06T09:13:47.020" Score="3" Body="&lt;p&gt;There is a difference between &lt;em&gt;accurate&lt;/em&gt; value function estimates, and &lt;em&gt;optimal&lt;/em&gt; value functions. An optimal value function is more specifically the value function of an optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;Value functions are always specific to some policy, which is why you will often see the subscript &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; in e.g. &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt; when there is a defined policy.&lt;/p&gt;&#xA;&lt;p&gt;The policy evaluation step (step 2) in policy iteration converges to an accurate value function estimate for whatever the current policy is. In general this will not be an optimal value function, except on the last time that step 2 is used, and there is no change to the policy in the next stage policy improvement (step 3).&lt;/p&gt;&#xA;&lt;p&gt;The policy improvement stage (step 3) can only usefully be run once for any value function estimate. The policy is updated to be greedy with respect to the value function from step 2 - that will always give the same results from the same value function estimate. If the value function is accurate then this new policy is guaranteed to be as good or better than the previous policy. Once step 3 is done, further improvements are only possible if the &lt;em&gt;new&lt;/em&gt; policy is accurately evaluated.&lt;/p&gt;&#xA;&lt;h3&gt;Comparison with value iteration&lt;/h3&gt;&#xA;&lt;p&gt;The difference with value iteration is that it never accurately evaluates any interim policies. In value iteration, the implied policy changes every time the maximising action changes due to new value estimates. In the later stages, when the optimal policy has been found and is stable, then the value function will converge to the optimal value function. In value iteration, most of the interim value functions are not accurate, but when it becomes accurate it will also be the optimal value function.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-08T08:04:44.350" LastActivityDate="2020-09-08T08:04:44.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23445" PostTypeId="2" ParentId="23442" CreationDate="2020-09-06T10:17:42.220" Score="1" Body="&lt;p&gt;Yes. &lt;em&gt;Convolutional neural networks&lt;/em&gt; are usually 3-dimensional. In fact, they usually deal with images (e.g. RGB images), which can already be 3-dimensional.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-09-06T10:17:42.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23446" PostTypeId="1" CreationDate="2020-09-06T10:22:23.737" Score="1" ViewCount="86" Body="&lt;p&gt;I've read through the Alpha(Go)Zero paper and there is only one thing I don't understand.&lt;/p&gt;&#xA;&lt;p&gt;The paper on page 1 states:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The MCTS search outputs probabilities π of playing each move. These search probabilities usually select much stronger moves than the raw move probabilities p of the neural network fθ(s);&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My question: Why is this the case? Why is &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; usually better than &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;? I think I can imagine why it's the case but I'm looking for more insight.&lt;/p&gt;&#xA;&lt;p&gt;what &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; are:&lt;/p&gt;&#xA;&lt;p&gt;Say we are in state &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;. We have a network that takes the state and produces &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; (probabilities for actions) and &lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; (a value for the state).&#xA;We then run MCTS from this state and extract a policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s_1) = \frac{N(s_1,a)^{1/\tau}}{\sum_b N(s_1,b)^{1/\tau}}$&lt;/span&gt;.&#xA;The paper is saying that &lt;span class=&quot;math-container&quot;&gt;$\pi(-|s_1)$&lt;/span&gt; is usually better than &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40603" LastEditorUserId="40603" LastEditDate="2020-09-08T13:04:57.803" LastActivityDate="2020-09-08T19:32:59.070" Title="In Alpha(Go)Zero, why is the policy extracted from MCTS better than the network one?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23447" PostTypeId="1" CreationDate="2020-09-06T10:35:04.353" Score="1" ViewCount="77" Body="&lt;p&gt;Inverse Reinforcement Learning based on &lt;a href=&quot;https://arxiv.org/abs/1606.03476&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAIL&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1603.00448&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAN-Guided Cost Learning(GAN-GCL)&lt;/a&gt;, uses a discriminator to classify between expert demos and policy generated samples.&#xA;&lt;a href=&quot;https://arxiv.org/abs/1710.11248&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial iRL&lt;/a&gt;, build upon GAN-GCL, has its discriminator &lt;span class=&quot;math-container&quot;&gt;$D_{\theta, \phi}$&lt;/span&gt; as a function of a state-only reward approximator &lt;span class=&quot;math-container&quot;&gt;$f_{\theta, \phi}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{\theta, \phi}\left(s, a, s^{\prime}\right)=\frac{\exp \left\{f_{\theta, \phi}\left(s, a, s^{\prime}\right)\right\}}{\exp \left\{f_{\theta, \phi}\left(s, a, s^{\prime}\right)\right\}+\pi(a \mid s)},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f_{\theta,\phi}$&lt;/span&gt; is expressed as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_{\theta,\phi} = g_{\theta} (s) + γh_φ (s\prime ) − h_φ (s).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The optimal &lt;span class=&quot;math-container&quot;&gt;$*g(s)$&lt;/span&gt; tries to recover the optimal reward function &lt;span class=&quot;math-container&quot;&gt;$r^*(s)$&lt;/span&gt;. While the &lt;span class=&quot;math-container&quot;&gt;$h(s)$&lt;/span&gt; tries to recover the optimal value funtion &lt;span class=&quot;math-container&quot;&gt;$V^*(s)$&lt;/span&gt;, which makes &lt;span class=&quot;math-container&quot;&gt;$f_{\theta,\phi}$&lt;/span&gt; interpretable as the advantage.&lt;/p&gt;&#xA;&lt;p&gt;My question comes from the network architecture used for &lt;span class=&quot;math-container&quot;&gt;$h(s)$&lt;/span&gt; in the original paper.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;... we use a &lt;strong&gt;2-layer ReLU network for the shaping term h&lt;/strong&gt;. For the policy, we use a &lt;strong&gt;two-layer (32 units) ReLU gaussian policy&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What is meant by the quoted text in &lt;em&gt;bold&lt;/em&gt;, because my interpretation of that text, (shown below) doesn't seem viable&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;h = nn.Sequential([nn.ReLu(), nn.ReLu()])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="2444" LastEditDate="2020-09-07T09:54:10.390" LastActivityDate="2020-11-09T15:43:49.723" Title="Can entire neural networks be composed of only activation functions?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;activation-function&gt;&lt;inverse-rl&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23448" PostTypeId="1" CreationDate="2020-09-06T12:31:41.123" Score="0" ViewCount="40" Body="&lt;p&gt;If the accumulated reward increases, the loss increases and vice versa. This is a strange behaviour. See the figure below for an example. What is the possibility of having this behaviour in DQN?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jUKa2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jUKa2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have &lt;span class=&quot;math-container&quot;&gt;$3000$&lt;/span&gt; episodes. An episode lasts &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; time steps. So, in total, I have &lt;span class=&quot;math-container&quot;&gt;$60000$&lt;/span&gt; time steps. I am using the ADAM optimizer with a learning rate of &lt;span class=&quot;math-container&quot;&gt;$5\times10^{-6}$&lt;/span&gt;. The target update frequency is once every &lt;span class=&quot;math-container&quot;&gt;$2000$&lt;/span&gt; time steps (once every &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; episodes). The training happens every episode (once every &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt; time steps). I am using double DQN with the duelling architecture and the prioritized experience replay buffer. The size of the buffer is &lt;span class=&quot;math-container&quot;&gt;$300$&lt;/span&gt; and the batch size is &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I tried to modify some hyperparameters but for now I still have this strange behaviour.&lt;/p&gt;&#xA;" OwnerUserId="37642" LastActivityDate="2020-09-06T12:31:41.123" Title="Reward and loss follow the same shape in DQN" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23449" PostTypeId="1" AcceptedAnswerId="23684" CreationDate="2020-09-06T13:31:51.477" Score="1" ViewCount="83" Body="&lt;p&gt;As a software engineer, I am searching for an existing solution or, if none exists, willing to create one that will be able to process texts (e.g. news from online media) to extract/paraphrase &lt;strong&gt;dry facts&lt;/strong&gt; from them, leaving all opinions, analysis, speculations, humor, etc., behind.&lt;/p&gt;&#xA;&lt;p&gt;If no such solution exists, what would be a good way to start creating it (considering that I have zero experience in AI/machine learning)?&lt;/p&gt;&#xA;&lt;p&gt;It would be no problem to manually create a set of examples (pairs of original news + dry facts extracted), but is that basically what it takes? I doubt so.&lt;/p&gt;&#xA;&lt;p&gt;(This knowledge domain is already huge, so which parts of it need to be learned first and foremost to figure out how to achieve the goal?)&lt;/p&gt;&#xA;" OwnerUserId="40800" LastEditorUserId="2444" LastEditDate="2021-01-15T00:46:19.673" LastActivityDate="2021-01-15T00:46:19.673" Title="How could facts be distinguished from opinions?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;natural-language-understanding&gt;&lt;text-summarization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23450" PostTypeId="1" CreationDate="2020-09-06T13:36:07.450" Score="1" ViewCount="19" Body="&lt;p&gt;I was watching a Youtube video in which the problem of trying to predict the last word in  a sentence was posed. The sentence was &amp;quot;I took my cat for a&amp;quot; and the last word was &amp;quot;walk&amp;quot;. The lecturer in this video stated that whilst sentences (the sequence) can be of varying lengths, if we take a really large fixed window we can model the whole sentence. In essence she said that we can convert any sentence into a fixed size vector and still preserve the order of the sentence (sequence). I was then wondering why do we need RNNs if we can just use FFNNs? Also does a fixed size vector really preserve sequential order information?&lt;/p&gt;&#xA;&lt;p&gt;Thank You for any help!&lt;/p&gt;&#xA;" OwnerUserId="40801" LastActivityDate="2020-09-06T13:36:07.450" Title="What's the difference between RNNs and Feed Forward Neural Networks if a fixed size vector can preserve sequential information?" Tags="&lt;recurrent-neural-networks&gt;&lt;time-series&gt;&lt;feedforward-neural-network&gt;&lt;sequence-modelling&gt;&lt;word2vec&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23452" PostTypeId="2" ParentId="23441" CreationDate="2020-09-06T20:36:44.023" Score="0" Body="&lt;p&gt;From the NumPy Linear algebra &lt;a href=&quot;https://numpy.org/doc/stable/reference/routines.linalg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The NumPy linear algebra functions rely on &lt;a href=&quot;https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAS&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/LAPACK&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAPACK&lt;/a&gt; to provide&#xA;efficient low level implementations of standard linear algebra&#xA;algorithms. Those libraries may be provided by NumPy itself using C&#xA;versions of a subset of their reference implementations but, when&#xA;possible, highly optimized libraries that take advantage of&#xA;specialized processor functionality are preferred.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In your case, &lt;code&gt;np.dot()&lt;/code&gt; is a matrix-vector multiplication which internally calls  such a highly optimized BLAS routine. Those libraries are implemented in low-level languages like Fortan or C, they exist for many years now and are still unbeaten in terms of speed.&lt;br /&gt;&#xA;Your &lt;em&gt;naive&lt;/em&gt; implementation of the convolution operation in python on the other hand is not optimized for execution speed and will be executed by the python interpreter (which is much slower than the compiled C functions).&lt;/p&gt;&#xA;&lt;p&gt;You could try to replace your manual convolution by the SciPy's equivalent &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.convolve2d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.signal.convolve2d()&lt;/code&gt;&lt;/a&gt;, to make advantage of optimized libraries as well and get a speed up.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-09-06T20:36:44.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23453" PostTypeId="1" CreationDate="2020-09-07T06:22:49.980" Score="1" ViewCount="81" Body="&lt;p&gt;There is emerging effort for Third Wave Artificial Intelligence (Artificial General Intelligence) (&lt;a href=&quot;http://hlc.doc.ic.ac.uk/3AI_HLC_2019.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hlc.doc.ic.ac.uk/3AI_HLC_2019.html&lt;/a&gt; and &lt;a href=&quot;https://www.darpa.mil/work-with-us/ai-next-campaign&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.darpa.mil/work-with-us/ai-next-campaign&lt;/a&gt;) and it covers the open-ended life-long machine learning as well. Currently machine learning agents are being run on quite immutable and simple games like Atari and Go. But what about the efforts to build and run machine learning adaptable agents (or even teams of them) in the virtual worlds (like Second Life) which are complex, expanding and in which the interaction with human representatives happens? Are there efforts to do that?&lt;/p&gt;&#xA;&lt;p&gt;I have found some articles from 2005-2009, but Google gives no recent literature on queries like &lt;code&gt;Reinforcement Learning Second Life&lt;/code&gt; etc.&lt;/p&gt;&#xA;&lt;p&gt;So - maybe there are some efforts to do this, but I can not just Google it.&lt;/p&gt;&#xA;&lt;p&gt;My question is - are there references for machine learning agents for virtual worlds and if not - what are the obstacles for trying to build them? There are little risks or costs for building them for virtual worlds?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://meta-guide.com/embodiment/secondlife-npc-artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://meta-guide.com/embodiment/secondlife-npc-artificial-intelligence&lt;/a&gt; is some bibliography and it is lacking recent research, for example.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="2444" LastEditDate="2020-09-07T22:23:20.600" LastActivityDate="2020-09-07T22:23:20.600" Title="Use of virtual worlds (e.g. Second Life) for training Artificial General Intelligence agents?" Tags="&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;agi&gt;&lt;intelligent-agent&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23454" PostTypeId="1" CreationDate="2020-09-07T07:07:28.877" Score="2" ViewCount="403" Body="&lt;p&gt;I have got numerous frames and I've detected all the faces in all the frames using Retinaface. However I need to track the faces of people over frames.&lt;/p&gt;&#xA;&lt;p&gt;For this purpose, I assumed I could try finding the landmarks from the face using libraries like &lt;code&gt;dlib&lt;/code&gt; and maybe compare these landmarks to check if they are infact the face of the same person.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if there are other methods or some useful resources I could refer for the same. Thanks a lot in advance.&lt;/p&gt;&#xA;" OwnerUserId="16881" LastActivityDate="2020-09-07T17:30:07.067" Title="How to identify if 2 faces contain the same person?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;facial-recognition&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23456" PostTypeId="2" ParentId="23454" CreationDate="2020-09-07T08:05:48.250" Score="2" Body="&lt;p&gt;The topic of your problem is person re-identification. You can check &lt;a href=&quot;https://github.com/bismex/Awesome-person-re-identification&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28129" LastActivityDate="2020-09-07T08:05:48.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23457" PostTypeId="2" ParentId="23435" CreationDate="2020-09-07T10:43:48.980" Score="1" Body="&lt;p&gt;I suggest you to have a look at &lt;a href=&quot;https://github.com/aikorea/awesome-rl&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; repo. It contains state-of-the art algorithms, papers, frameworks, courses and some implementations. You can also check &amp;quot;Deep Reinforcement Learning Hands On&amp;quot; book examples written by Max Lapan &lt;a href=&quot;https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This repo contains many programming and reinforcement learning examples with PyTorch framework.&lt;/p&gt;&#xA;" OwnerUserId="28129" LastEditorUserId="28129" LastEditDate="2020-09-08T08:50:34.753" LastActivityDate="2020-09-08T08:50:34.753" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23459" PostTypeId="1" AcceptedAnswerId="23479" CreationDate="2020-09-07T12:14:22.997" Score="0" ViewCount="53" Body="&lt;p&gt;I attempt to understand the formulation of dictionary learning for this paper:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nextcenter.org/wp-content/uploads/2018/02/Depression-Detection-via-Harvesting-Social-Media-A-Multimodal-Dictionary-Learning-Solution.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1502.01094.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multimodal Task-Driven Dictionary Learning for Image Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Both papers used the exact formulation in two different domains.&lt;/p&gt;&#xA;&lt;p&gt;Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.&lt;/p&gt;&#xA;&lt;p&gt;Given a matrix, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{array}{lcccccc} &#xA;&amp;amp; p_1 &amp;amp; p_2 &amp;amp;  p_3 &amp;amp;  p_4 &amp;amp;  p_5 &amp;amp; \text { label } \\&#xA;o_1 &amp;amp; 1 &amp;amp; 2 &amp;amp; 3 &amp;amp; 4 &amp;amp; 1 &amp;amp; 1 \\&#xA;o_2 &amp;amp; 2 &amp;amp; 3 &amp;amp; 4 &amp;amp; 5 &amp;amp; 2 &amp;amp; 1 \\&#xA;o_3 &amp;amp; 3 &amp;amp; 4 &amp;amp; 5 &amp;amp; 6 &amp;amp; 2 &amp;amp; 0 \\&#xA;o_4 &amp;amp; 4 &amp;amp; 5 &amp;amp; 6 &amp;amp; 7 &amp;amp; 3 &amp;amp; 0&#xA;\end{array}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, using a math notation and excluding the label, I can define this matrix, &lt;span class=&quot;math-container&quot;&gt;$A = [o_1, o_2, o_3, o_4]  ∈ R^{4×5}$&lt;/span&gt;, as &lt;span class=&quot;math-container&quot;&gt;$A = [{(1, 2, 3, 4, 1), (2, 3, 4, 5, 2), (3, 4, 5, 6, 2), (4, 5, 6, 7, 3)}]$&lt;/span&gt;, and in numpy:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;A = np.array([[1, 2, 3, 4, 1],&#xA;              [2, 3, 4, 5, 2],&#xA;              [3, 4, 5, 6, 2],&#xA;              [4, 5, 6, 7, 3]])&#xA;&#xA;A.shape&#xA;# (4, 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Am I right?&lt;/p&gt;&#xA;" OwnerUserId="27796" LastEditorUserId="2444" LastEditDate="2021-02-05T14:55:47.680" LastActivityDate="2021-02-05T14:55:47.680" Title="Do the rows of the design matrix refer to the observations or predictors?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;papers&gt;&lt;data-preprocessing&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23461" PostTypeId="1" AcceptedAnswerId="23464" CreationDate="2020-09-07T17:14:09.677" Score="0" ViewCount="112" Body="&lt;p&gt;For the purposes of training a Convolutional Neural Network Classifier, should image augmentation be done before or after resizing the training images?&lt;/p&gt;&#xA;&lt;p&gt;To reduce file size and speed up training time, developers often resize training images to a set height and width using something like PIL (Python Imaging Library).&lt;/p&gt;&#xA;&lt;p&gt;If the images are augmented (to increase training set size), should it be done before or after resizing the members of the set?&lt;/p&gt;&#xA;&lt;p&gt;For simplicity sake, it would probably be faster to augment the images after resizing, but I am wondering if any useful data is lost in this process. I assume it may depend on the method used to resize the images (cropping, scaling technique, etc.)&lt;/p&gt;&#xA;" OwnerUserId="40821" LastActivityDate="2020-09-07T19:26:25.287" Title="Should image augmentation be applied before or after image resizing?" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23462" PostTypeId="2" ParentId="23454" CreationDate="2020-09-07T17:30:07.067" Score="1" Body="&lt;p&gt;You can try using something called as a Siamese network if you are willing to train the network on your own using something called as triplet loss(if you have lots of face images).&lt;/p&gt;&#xA;&lt;p&gt;Another approach would we something called a one-shot using FaceNet(transfer learning approach)&#xA;FaceNet uses deep convolutional neural network (CNN). The network is trained such that the squared L2 distance between the embeddings correspond to face similarity. The images used for training are scaled, transformed and are tightly cropped around the face area.&lt;/p&gt;&#xA;&lt;p&gt;Another important aspect of FaceNet is its loss function . It is already trained using triplet loss function.In this case you could just feed to face images and you would get a thershold score for the similarities.&lt;/p&gt;&#xA;" OwnerUserId="40823" LastActivityDate="2020-09-07T17:30:07.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23463" PostTypeId="2" ParentId="14028" CreationDate="2020-09-07T17:35:19.307" Score="0" Body="&lt;p&gt;You could try building Siamese network and train it on a large set of faces.&lt;/p&gt;&#xA;&lt;p&gt;Two identical networks are used; one taking the known signature for the person, and another taking a candidate signature. The outputs of both networks are combined and scored to indicate whether the candidate signature is real or a forgery. The deep CNN are first trained to discriminate between examples of each class. The models are then re-purposed for verification to predict whether new examples match a template for each class. Specifically, each network produces a feature vector for an input image, which are then compared using the L1 distance and a sigmoid activation. Similar goes with face.&lt;/p&gt;&#xA;" OwnerUserId="40823" LastEditorUserId="1641" LastEditDate="2020-09-13T10:23:17.590" LastActivityDate="2020-09-13T10:23:17.590" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23464" PostTypeId="2" ParentId="23461" CreationDate="2020-09-07T19:26:25.287" Score="0" Body="&lt;p&gt;As it is told in &lt;a href=&quot;https://pillow.readthedocs.io/en/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PIL documentation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It uses some filters to resize images.And those filters are explained &lt;a href=&quot;https://pillow.readthedocs.io/en/stable/handbook/concepts.html#concept-filters&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&#xA;uses mostly numerical methods as I see. So it is &lt;em&gt;approximating&lt;/em&gt; the image data input. Which means you are right about data loss. But here might be the question would it change the data so much If it is done after augmentation or before?&lt;/p&gt;&#xA;&lt;p&gt;Since in numerical methodic approaches the more values means more valid approximation in general.&lt;/p&gt;&#xA;&lt;p&gt;So it might be beneficial to augment and then resize.&lt;/p&gt;&#xA;&lt;p&gt;But one should look for its &lt;em&gt;real&lt;/em&gt; answer. Mine is just a thought experiment. You will get better &lt;em&gt;proven&lt;/em&gt; answers&lt;/p&gt;&#xA;" OwnerUserId="38344" LastActivityDate="2020-09-07T19:26:25.287" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23465" PostTypeId="2" ParentId="11822" CreationDate="2020-09-07T19:33:40.000" Score="0" Body="&lt;p&gt;Addition to dear Oliver Mason's answer&lt;/p&gt;&#xA;&lt;p&gt;You can check &lt;a href=&quot;https://github.com/topics/text-summarizer&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to see how people uses machine learning tools to summarize texts and also check some articles from &lt;a href=&quot;https://towardsdatascience.com/tagged/text-summarization&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to understand its background If you are curious about.&lt;/p&gt;&#xA;" OwnerUserId="38344" LastActivityDate="2020-09-07T19:33:40.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23467" PostTypeId="1" CreationDate="2020-09-07T22:43:52.497" Score="2" ViewCount="65" Body="&lt;p&gt;I am learning about &lt;em&gt;incremental learning&lt;/em&gt; and read that &lt;em&gt;rehearsal&lt;/em&gt; learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? You train a model by passing in batches of data and redo this with a set number of epochs.&lt;/p&gt;&#xA;&lt;p&gt;If I'm understanding rehearsal learning correctly, you do the exact same thing but with &amp;quot;new&amp;quot; data. Thus, the only difference is inconsistencies in the epoch number across data batches.&lt;/p&gt;&#xA;" OwnerUserId="26900" LastEditorUserId="2444" LastEditDate="2020-09-29T19:47:46.260" LastActivityDate="2021-02-26T21:07:45.817" Title="Is batch learning with gradient descent equivalent to &quot;rehearsal&quot; in incremental learning?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;gradient-descent&gt;&lt;incremental-learning&gt;&lt;batch-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23469" PostTypeId="1" CreationDate="2020-09-08T01:57:31.797" Score="1" ViewCount="69" Body="&lt;p&gt;I understand that I can draw a state-space graph for any problem. However, here is the problem: I can't really figure out how to make production systems.&lt;/p&gt;&#xA;&lt;p&gt;I am solving the FWGC (Farmer, Wolf, Goat, Cabbage) River Crossing Puzzle using a state-space search. So, my tasks are that:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Represent the state-space graph (which I know how to do)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Write production systems.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My questions: &lt;strong&gt;How do I write production systems?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The thing that confused me, was the production system example in Rich's book (about the water jug problem), where he has imagined all the states possible and wrote the next state for them.&lt;/p&gt;&#xA;&lt;p&gt;Here in the FWGC problem, I see some problems while writing the production system.&lt;/p&gt;&#xA;&lt;p&gt;For instance, for a given state, there are multiple possible next states, i.e. a farmer can take Goat, Cabbage, Wolf, or go alone to the other side (assuming that all states are safe, just for the sake of simplicity).&lt;/p&gt;&#xA;&lt;p&gt;So, how would I represent the same state going to multiple next states in production systems?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I have tried-:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, I googled a pdf&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.unm.edu/%7Eluger/ai-final2/CH4_Depth-.%20Breadth-,%20and%20Best-first%20Search.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.unm.edu/~luger/ai-final2/CH4_Depth-.%20Breadth-,%20and%20Best-first%20Search.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9XbGU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9XbGU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is that what I call the production system for this case?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;But, here are my reasons why it should not be called a production system:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There might be other possible states as well.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is showing only 1 solution.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, how do I actually learn to create production rules (I know how to make state-space representation better as I have read that J.Nillison's book which was GOLD in this matter)? And, what would be the production rules in this case?&lt;/p&gt;&#xA;" OwnerUserId="40828" LastEditorUserId="2444" LastEditDate="2020-09-09T16:18:07.440" LastActivityDate="2020-09-09T16:18:07.440" Title="How do I write production systems?" Tags="&lt;expert-system&gt;&lt;prolog&gt;&lt;production-systems&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23471" PostTypeId="2" ParentId="14028" CreationDate="2020-09-08T05:43:35.543" Score="0" Body="&lt;p&gt;&lt;strong&gt;For the Construction of Deep Learning Models&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Backbone Deep Learning models which can be applied to a variety of deep learning tasks (including facial recognition) have been implemented in a range of libraries available in Python. I'm assuming by constructing your own algorithm you mean a novel implementation of the model structure. Taking the PyTorch framework as an example, some common pretrained models are available here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/pytorch/vision/tree/master/torchvision/models&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pytorch/vision/tree/master/torchvision/models&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To train a novel face recognition model you could follow the tutorial for object detection available here:&#xA;&lt;a href=&quot;https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html&lt;/a&gt;&#xA;and make changes to the model.&lt;/p&gt;&#xA;&lt;p&gt;In the tutorial they use model features from the library in the following section of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load a pre-trained model for classification and return&#xA;# only the features&#xA;backbone = torchvision.models.mobilenet_v2(pretrained=True).features&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For the simplest example torchvision.models.AlexNet.features look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.features = nn.Sequential(&#xA;            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),&#xA;            nn.ReLU(inplace=True),&#xA;            nn.MaxPool2d(kernel_size=3, stride=2),&#xA;            nn.Conv2d(64, 192, kernel_size=5, padding=2),&#xA;            nn.ReLU(inplace=True),&#xA;            nn.MaxPool2d(kernel_size=3, stride=2),&#xA;            nn.Conv2d(192, 384, kernel_size=3, padding=1),&#xA;            nn.ReLU(inplace=True),&#xA;            nn.Conv2d(384, 256, kernel_size=3, padding=1),&#xA;            nn.ReLU(inplace=True),&#xA;            nn.Conv2d(256, 256, kernel_size=3, padding=1),&#xA;            nn.ReLU(inplace=True),&#xA;            nn.MaxPool2d(kernel_size=3, stride=2),&#xA;        )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Adding or subtracting layers from this backbone feature extractor would result in a new &amp;quot;algorithm&amp;quot; for object detection.&#xA;If you want to know exactly what mathematical operation each of these layers is performing you can look at the PyTorch documentation. For example, in the case of nn.Relu layer:&#xA;&lt;a href=&quot;https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Applies the rectified linear unit function element-wise:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ ReLU(x)=(x)^{+}=max(0,x)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="40833" LastActivityDate="2020-09-08T05:43:35.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23472" PostTypeId="1" AcceptedAnswerId="23476" CreationDate="2020-09-08T07:45:33.783" Score="2" ViewCount="78" Body="&lt;p&gt;After having chosen the number of layers for a convolutional neural network, we must also choose the number of filters/channels for each convolutional layer.&lt;/p&gt;&#xA;&lt;p&gt;The intuition behind the filter's spatial dimension is the number of pixels in the image that must be considered to perform the recognition/detection task.&lt;/p&gt;&#xA;&lt;p&gt;However, I still can't find the intuition behind the number of filters. The numbers 128 and 256 are often used in the literature, but why?&lt;/p&gt;&#xA;" OwnerUserId="40839" LastEditorUserId="2444" LastEditDate="2020-11-06T01:56:41.267" LastActivityDate="2020-11-06T01:56:41.267" Title="What is the intuition behind the number of filters/channels for each convolutional layer?" Tags="&lt;convolutional-neural-networks&gt;&lt;hyper-parameters&gt;&lt;filters&gt;&lt;convolutional-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23473" PostTypeId="1" CreationDate="2020-09-08T07:53:50.003" Score="1" ViewCount="14" Body="&lt;p&gt;I have multiple FFT's taken from a sample at different pressures, through different analysis I can see that the resonant frequencies are shifting in the spectrum for each FFT at a different pressure.&lt;/p&gt;&#xA;&lt;p&gt;Using conventional peak tracking has been difficult as the peaks increase/decrease in magnitude within the FFT as well as shifting in the spectrum.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible for a neural network to 'detect'/'pick out' these frequency values?&lt;/p&gt;&#xA;&lt;p&gt;Any help or guidance is appreciated :)&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="40840" LastActivityDate="2020-09-08T07:53:50.003" Title="Neural Network for locating shifting resonant frequencies" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23474" PostTypeId="1" CreationDate="2020-09-08T08:36:45.027" Score="0" ViewCount="30" Body="&lt;p&gt;I attempt to understand the formulation of dictionary learning for this paper:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nextcenter.org/wp-content/uploads/2018/02/Depression-Detection-via-Harvesting-Social-Media-A-Multimodal-Dictionary-Learning-Solution.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1502.01094.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multimodal Task-Driven Dictionary Learning for Image Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Part 2: Confusion on math notation definitions (Conflict with my understanding)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;(Part 1 is in the &lt;a href=&quot;https://ai.stackexchange.com/questions/23459/confusion-on-vector-matrix-notation-on-research-papers-and-the-need-of-row-norm&quot;&gt;other post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;From both papers that were published in reputable venues, they defined the dictionary learning as follow:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given the original feature representation &lt;span class=&quot;math-container&quot;&gt;$X = [x_1, ..., x_N] ∈&#xA; R^{M×N}$&lt;/span&gt; , dictionary learning aims to learn a set of latent concepts&#xA;or feature patterns, &lt;span class=&quot;math-container&quot;&gt;$D = [d_1,..., d_D] ∈ R^{M×D}$&lt;/span&gt; and a latent&#xA;sparse representation &lt;span class=&quot;math-container&quot;&gt;$A = [α_1, ..., α_N] ∈ R^{D×N}$&lt;/span&gt;, with the&#xA;following empirical cost:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_D \frac1N \sum_{n=1}^N \ell(x_n, D), s.t., ||d_k||_{\ell_{2}} \le 1, \forall k = 1, ... , D $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And in (2), they defined:&lt;/p&gt;&#xA;&lt;p&gt;Let  &lt;span class=&quot;math-container&quot;&gt;$X = [x_1, x_2, . . . , x_N ] ∈ Rn×N$&lt;/span&gt;  be the collection of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (normalized) training samples that are assumed to be statistically independent. Using the example in part 1, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; here is the observation which means the matrix looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         &lt;span class=&quot;math-container&quot;&gt;$o_1$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$o_2$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$o_3$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$o_4$&lt;/span&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt;      1     2     3     4     &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt;      2     3     4     5     &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_3$&lt;/span&gt;      3     4     5     6    &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_4$&lt;/span&gt;      4     5     6     7    &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_5$&lt;/span&gt;      1     2     2     3 &#xA;label      1     1     0     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But since this is the dictionary learning problem, instead of a conventional machine learning one, it is okay to represent the matrix in any form. So basically, if I want it to be my conventional way, I can just transpose the matrix to the one I familiar, from &lt;span class=&quot;math-container&quot;&gt;$R^{5×4}$&lt;/span&gt; transpose to &lt;span class=&quot;math-container&quot;&gt;$R^{4×5}$&lt;/span&gt;, right?&lt;/p&gt;&#xA;&lt;p&gt;Their formulation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$X_{M\times N} = D_{M \times D} \times A_{D \times N}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;To replicate their experiments in my own setting, the formulation becomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$X_{N \times M} = A_{N \times D} \times D_{D \times M}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$X^T = (DA)^T = A^T \times D^T$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X = [x_1, ..., x_N] ∈ R^{N×M}$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; represents the number of observations, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; represents the number of features / predictors.&lt;/p&gt;&#xA;&lt;p&gt;Right?&lt;/p&gt;&#xA;" OwnerUserId="27796" LastActivityDate="2020-09-08T08:36:45.027" Title="Confusion on Math Notation Definition" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;&lt;math&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23475" PostTypeId="2" ParentId="23335" CreationDate="2020-09-08T09:22:39.753" Score="1" Body="&lt;p&gt;In fact, I do not know how to calculate GPU memory to run a neural network but I have a solution for allocation problems in GPUs while using tensorflow framework.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;gpus = tf.config.experimental.list_physical_devices('GPU')&#xA;if gpus:&#xA;    # Restrict TensorFlow to only allocate 2GB * 2 of memory on the first GPU&#xA;    try:&#xA;        tf.config.experimental.set_virtual_device_configuration(&#xA;            gpus[0],&#xA;            [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=2048 * 2)])&#xA;        logical_gpus = tf.config.experimental.list_logical_devices('GPU')&#xA;        print(len(gpus), &amp;quot;Physical GPUs,&amp;quot;, len(logical_gpus), &amp;quot;Logical GPUs&amp;quot;)&#xA;    except RuntimeError as e:&#xA;        # Virtual devices must be set before GPUs have been initialized&#xA;        print(e)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can set a memory limit on GPU which sometimes solves memory allocation problems. As shown above, you can set &amp;quot;memory_limit&amp;quot; parameter as your configuration requires.&lt;/p&gt;&#xA;&lt;p&gt;Also be careful about using correct framework. If you want to use above code to set memory, you have to build your neural network from tensorflow with keras backend.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from tensorflow.python.keras.models import Sequential&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28129" LastActivityDate="2020-09-08T09:22:39.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23476" PostTypeId="2" ParentId="23472" CreationDate="2020-09-08T10:43:53.210" Score="1" Body="&lt;p&gt;The channel sizes 32, 128, etc. are used because of memory and efficiency. There is nothing holy about these numbers.&lt;/p&gt;&#xA;&lt;p&gt;The intuition behind choosing the number of channels is as follows-&#xA;The initial layers extract low-level features- they consist of edge detectors, etc. There aren't many such features. So, we won't gain much by adding a lot of filters(of course, if we use a 3x3 filters on an RGB image, we would have 2^27 different filters even if our neurons have only 0 and 1 as their values. However, most of them are quite similar/meaningless for our job). &lt;strong&gt;Using a lot of filters might even lead to overfitting.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The latter layers are responsible for detecting more nuanced features, like elbows/nose shape from the lower level features extracted previously. So, we might do better if we increase the number of channels. Also, note that the resultant layers become more and more sparse as we go deeper.&lt;/p&gt;&#xA;&lt;p&gt;Though it might differ in applications like super resolution image, &lt;strong&gt;in general, the number of channels stays the same or increases when we go deeper.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A nice experiment would be to try and increase the number of channels until you get no more benefit from it. I believe there was a paper that did exactly this(please cite it if someone remembers). You could even try to visualise the filters at this stage and see if the filters are similar or not.&lt;/p&gt;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="40843" LastActivityDate="2020-09-08T10:43:53.210" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23479" PostTypeId="2" ParentId="23459" CreationDate="2020-09-08T12:01:52.343" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The rows (or, in general, the first dimension of your tensor) are typically the observations. For example, in TensorFlow, the first dimension of the input tensor typically refers to the batch size, i.e. the number of observations. If you are using Pandas (a Python library to manipulate data), the rows are typically the observations and the columns are the predictors.&lt;/p&gt;&#xA;&lt;p&gt;However, in general, it does not really matter which convention you use, provided that you use one of the conventions consistently in your implementation (i.e. you choose one of the conventions and you stick with it throughout all your code, to avoid complexity), and make it clear in the documentation. So, you can have a matrix where either the rows or columns are observations and, consequently, the columns or, respectively, rows are the features (aka predictors or independent variables).&lt;/p&gt;&#xA;&lt;p&gt;Anyway, it is probably a good idea to be consistent with existing literature and implementations/libraries, so you should probably use the rows for the observations.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-08T12:11:52.723" LastActivityDate="2020-09-08T12:11:52.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23480" PostTypeId="1" CreationDate="2020-09-08T12:14:16.500" Score="1" ViewCount="66" Body="&lt;p&gt;I am wondering how much I should extend my training set with data augmentation. Is there somewhere a pre-defined number I can go with?&lt;/p&gt;&#xA;&lt;p&gt;Suppose I have 10000 images, can I go as far as 10x or 20x times, to get 100000 and 200000, respectively, images? I am wondering how will this impact model training. I am using a mask R-CNN.&lt;/p&gt;&#xA;" OwnerUserId="18147" LastEditorUserId="2444" LastEditDate="2020-09-09T23:26:52.090" LastActivityDate="2020-09-09T23:26:52.090" Title="How much should we augment our training data?" Tags="&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;data-augmentation&gt;&lt;mask-rcnn&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23481" PostTypeId="1" CreationDate="2020-09-08T14:00:28.657" Score="1" ViewCount="40" Body="&lt;p&gt;I was working on a project involving the search for biosignatures (signs of life) on exoplanets and the probability of that planet harboring life. In this case, we know that Earth is the only planet confirmed to have life on it. So the parameters of atmospheric conditions, radius, temperature, distance from the star for planets confirmed to have life is one (Earth).&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to use NNs to predict the probability of an exoplanet harboring life if we have the data of all these parameters for that planet?&lt;/p&gt;&#xA;" OwnerUserId="40849" LastEditorUserId="11539" LastEditDate="2020-09-11T12:33:11.037" LastActivityDate="2020-09-11T12:33:11.037" Title="Is there any way where you can train a Neural Network with only one data point in the dataset?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23482" PostTypeId="1" CreationDate="2020-09-08T14:38:27.730" Score="1" ViewCount="473" Body="&lt;p&gt;I am comparing different CNN architectures for edge implementation. Some papers describing architectures refer to &lt;em&gt;mult-adds&lt;/em&gt;, like &lt;a href=&quot;https://arxiv.org/pdf/1704.04861.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the MobileNet V1 paper&lt;/a&gt;, where it is claimed that this net has 569M mult-adds, and others refer to floating-point operations (FLOPs), like &lt;a href=&quot;https://arxiv.org/pdf/1711.09224.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the CondenseNet paper&lt;/a&gt; claims 274M FLOPs.&lt;/p&gt;&#xA;&lt;p&gt;Are these comparable? Is 1 multiply-add equivalent to 2 floating-point operations? Any direction will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="40753" LastEditorUserId="2444" LastEditDate="2020-09-08T22:19:27.883" LastActivityDate="2021-02-06T05:03:35.327" Title="Are mult-adds and FLOPs equivalent?" Tags="&lt;convolutional-neural-networks&gt;&lt;terminology&gt;&lt;papers&gt;&lt;complexity-theory&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23483" PostTypeId="1" CreationDate="2020-09-08T18:16:42.847" Score="0" ViewCount="39" Body="&lt;p&gt;I am building a recommendation system that recommends relevant articles to the user. I am doing this using simple similarity-based techniques (with the Jaccard similarity) using as features the page title, the tags, and the article content.&lt;/p&gt;&#xA;&lt;p&gt;Now my problem is I have different &amp;quot;adult articles&amp;quot; and some are articles that expire (for example, an article about a movie in Jan 2019 would not be relevant in Dec 2019).&lt;/p&gt;&#xA;&lt;p&gt;I want to keep these adult articles separate, as a person who is reading about history does not want to be led to an adult article and not recommend articles that have expired or would not be relevant in the present moment.&lt;/p&gt;&#xA;&lt;p&gt;Should I just improve the quality of my features or tags? Or is there any other way to achieve this?&lt;/p&gt;&#xA;" OwnerUserId="10118" LastEditorUserId="2444" LastEditDate="2020-09-10T22:04:48.073" LastActivityDate="2020-09-10T22:04:48.073" Title="How can I build a recommendation system that takes into account some constraints or the context?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;recommender-system&gt;&lt;jaccard-similarity&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23484" PostTypeId="1" CreationDate="2020-09-08T19:24:19.207" Score="0" ViewCount="15" Body="&lt;p&gt;I am using an encoder-decoder architecture, with 2 layers each in the encoder and decoder and 128 neurons in each hidden layer. The inputs are in a 2D form: one column has the days and the other column has the time series dependent on the days (shape:(5780, 100, 2)). The output is a single value, that represents a particular day when the breaking point occurs (shape:(5780, 1, 1)). The breaking point is one of the time dependent values that is not of concern to me.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that after training, the output on all test data is almost &lt;strong&gt;exactly the same&lt;/strong&gt;. I have experimented with high and low learning rates (ranging 1e-2 to 1e-5), the number of training epochs (300 to 3000).&lt;/p&gt;&#xA;&lt;p&gt;What I haven't done is batch normalization or any kind of normalization, but I had done some operations with a similar model where the outputs are the a sequence of points in the second feature column, and it worked perfectly fine.&lt;/p&gt;&#xA;&lt;p&gt;What could be going wrong here?&lt;/p&gt;&#xA;&lt;p&gt;In addition, is there another model that is more suited to predicting this kind of an output with the test data that I have?&lt;/p&gt;&#xA;" OwnerUserId="40410" LastActivityDate="2020-09-08T19:24:19.207" Title="Is there a model suitable to predict one correct value based on a 2D input series?" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23485" PostTypeId="2" ParentId="23446" CreationDate="2020-09-08T19:32:59.070" Score="1" Body="&lt;p&gt;The most important word for answering your question from that quote from the paper is probably the word &amp;quot;usually&amp;quot;: These search probabilities &lt;strong&gt;usually&lt;/strong&gt; select much stronger moves than the raw move probabilities &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; of the neural network. It's not always going to be true, but more often than not / most of the time / &amp;quot;on average&amp;quot;, we intuitively expect it to be true. In theory there could be pathological cases, especially with low MCTS iteration counts and/or a poorly-trained network, where it may not be true. But even if we just find it to be true most of the time, that can be good enough for the algorithm to work well in practice.&lt;/p&gt;&#xA;&lt;p&gt;Recall that in the Selection phase of MCTS, Alpha(Go) Zero always selects the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that maximises the following expression for a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; when traversing the tree (see supplementary material of Alpha Zero paper):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a) + C(s) \frac{P(s, a) \sqrt{N(s)}}{1 + N(s, a)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values here are the value estimates resulting from the MCTS search; these are the results produced by our tree search, and these generally become more and more accurate the longer our tree search runs. If we run our tree search for an infinite amount of time, we expect these to converge to the true minimax values.&lt;/p&gt;&#xA;&lt;p&gt;These same &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values are the things that can cause the final distribution of visit counts &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; to shift away from the prior neural network distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. Suppose that we have a hypothetical case where for &lt;em&gt;all the different actions&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values end up staying identical. In this (unlikely) hypothetical case, the visit counts would continue getting distributed proportionally to &lt;span class=&quot;math-container&quot;&gt;$P(s, a)$&lt;/span&gt;, causing the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; distribution to stay equal to the &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; one (barring minor potential differences due to &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; being derived from discrete, integer visit counts which may not be able to &lt;strong&gt;exactly&lt;/strong&gt; replicate the real-valued probabilities in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;). Only if the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values resulting from a (hopefully smart!) tree search algorithm give us reason to shift away from the prior distribution &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; do we actually really start shifting away from it.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-09-08T19:32:59.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23486" PostTypeId="1" CreationDate="2020-09-09T04:08:28.537" Score="0" ViewCount="46" Body="&lt;p&gt;I'm experimenting with music and transformers, and I have sequences &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; of shape: &lt;span class=&quot;math-container&quot;&gt;$(B,L,N)$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is the batch size, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the sequence length, and &lt;span class=&quot;math-container&quot;&gt;$N=12$&lt;/span&gt; are the number of musical notes with each &lt;span class=&quot;math-container&quot;&gt;$N_i , i\in[0,11]$&lt;/span&gt; representing the probability of each note. i.e. &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=0}^{N-1}N_i=1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also have embeddings &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; for each note of shape: &lt;span class=&quot;math-container&quot;&gt;$(N,512)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Then I take &lt;span class=&quot;math-container&quot;&gt;$\text{argmax}_N (S)$&lt;/span&gt; which gives most probable note of shape: &lt;span class=&quot;math-container&quot;&gt;$(B,L)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;then pick the corresponding embedding (&lt;code&gt;gather&lt;/code&gt; in tensorflow), resulting in shape: &lt;span class=&quot;math-container&quot;&gt;$(B,L,512)$&lt;/span&gt; which I feed to the transformer, I'll call this &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I also take the probability &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; of the most probable note by taking &lt;span class=&quot;math-container&quot;&gt;$\text{max}_N (S)$&lt;/span&gt;, shape: &lt;span class=&quot;math-container&quot;&gt;$(B,L)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now I feed &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; to a transformer (or an RNN), but the problem is the transformer (or an RNN) doesnt take the probability of occurrence of that note into account.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I was thinking of ways to include the probability aswell along with the embeddings, and I'm thinking of taking the direct product of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; (resulting shape: &lt;span class=&quot;math-container&quot;&gt;$(B,L,512)$&lt;/span&gt;) and then feed this to the transformer, i.e. multiplying each note embedding with its occurrence probability&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Question: It is okay to give the multiplied embedding (&lt;span class=&quot;math-container&quot;&gt;$XP$&lt;/span&gt;) to the transformer or an RNN?&lt;/p&gt;&#xA;&lt;p&gt;I'm guessing it is okay because the dot products &lt;span class=&quot;math-container&quot;&gt;$(QK^T)$&lt;/span&gt; in a transformer is simply scaled to the given probability&lt;/p&gt;&#xA;&lt;p&gt;P.S: I'm note sure whether this question fit in AI S.E or DataScience S.E, feel free to move this question, if required.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="20795" LastEditorUserId="20795" LastEditDate="2020-09-09T07:33:23.777" LastActivityDate="2020-09-09T07:33:23.777" Title="How to input a given sequence to a transformer (or an RNN) with probability of occurrence?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;recurrent-neural-networks&gt;&lt;transformer&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23487" PostTypeId="2" ParentId="23482" CreationDate="2020-09-09T04:33:12.050" Score="0" Body="&lt;p&gt;According to &lt;a href=&quot;https://github.com/sovrasov/flops-counter.pytorch/issues/16&quot; rel=&quot;nofollow noreferrer&quot;&gt;this source&lt;/a&gt;, one FLOP is roughly equal to two MAC (multiply accumulate). My guess/understanding would be that the distinction is made because neural nets spend compute overwhelmingly on multiply-accumulate operations, and thus optimizations and statistics over MAC operations would be more significant than FLOPs.&lt;/p&gt;&#xA;" OwnerUserId="6779" LastActivityDate="2020-09-09T04:33:12.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23491" PostTypeId="1" CreationDate="2020-09-09T10:04:13.830" Score="1" ViewCount="143" Body="&lt;p&gt;In &lt;a href=&quot;https://cdac.uchicago.edu/research/learned-emulators-of-physics-simulations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, the term &amp;quot;learned emulator&amp;quot; is used.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Recently, scientists have started creating &amp;quot;learned emulators&amp;quot; using&#xA;AI neural network approaches, but have not yet fully explored the&#xA;advantages and potential pitfalls of these surrogates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What is a &amp;quot;learned emulator&amp;quot;?  I believe it is related to neural networks. Where can I read more?&lt;/p&gt;&#xA;" OwnerUserId="29801" LastEditorUserId="2444" LastEditDate="2021-01-07T00:32:17.910" LastActivityDate="2021-01-07T00:32:17.910" Title="What is a &quot;learned emulator&quot;?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23492" PostTypeId="2" ParentId="23491" CreationDate="2020-09-09T10:50:14.957" Score="1" Body="&lt;p&gt;In a typical situation, for the emulation of physical environments, you need to define all physical rules and forces. In the &amp;quot;learned emulators&amp;quot;, they use some machine learning techniques to learn those rules by supervising and interacting (instead of formalizing all of them). In this case, they do not need any exact formulation of the physical environment to emulate it.&lt;/p&gt;&#xA;&lt;p&gt;An instance of this simulation can be this article &lt;a href=&quot;https://pubs.acs.org/doi/abs/10.1021/acs.jpclett.8b00902&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Realistic Atomistic Structure of Amorphous Silicon from Machine-Learning-Driven Molecular Dynamics&amp;quot;&lt;/a&gt;, i.e., a machine-learning-driven simulation instead of exact formulation of the environment.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-09-09T13:15:01.340" LastActivityDate="2020-09-09T13:15:01.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23493" PostTypeId="1" CreationDate="2020-09-09T12:12:57.673" Score="0" ViewCount="69" Body="&lt;p&gt;Geometric interpretation of Logistic Regression and Linear regression is considered here.&lt;/p&gt;&#xA;&lt;p&gt;I was going through Logistic regression and Linear regression. In the optimization equation of both following term is used. &lt;span class=&quot;math-container&quot;&gt;$$W^{T}.X$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;W is a vector which holds weights of the hyper-plane.&lt;/p&gt;&#xA;&lt;p&gt;I realized following about the dimensions of the fitted hyper-plane. Want to confirm it.&lt;/p&gt;&#xA;&lt;p&gt;Let,&#xA;d = Number of features for both Logistic Regression and Linear Regression.&lt;/p&gt;&#xA;&lt;p&gt;Logistic Regression case:&#xA;Fitted hyper-plane is d-dimensional.&lt;/p&gt;&#xA;&lt;p&gt;Linear Regression case:&#xA;Fitted hyper-plane is (d + 1) dimensions.&lt;/p&gt;&#xA;&lt;p&gt;Example&lt;/p&gt;&#xA;&lt;p&gt;d = 2&lt;/p&gt;&#xA;&lt;p&gt;feature 1 : weight,&#xA;feature 2 : height&lt;/p&gt;&#xA;&lt;p&gt;Logistic Regression:&#xA;Its a 2 class classification.&#xA;y : {obsess, normal)&lt;/p&gt;&#xA;&lt;p&gt;Linear Regression:&#xA;y: blood pressure (real value)&lt;/p&gt;&#xA;&lt;p&gt;Here,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Logistic Regression will fit a 2-D line.&lt;/li&gt;&#xA;&lt;li&gt;Linear Regression will fit a 3-D plane.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Please confirm if this understanding is correct and same happens even in higher dimensions.&lt;/p&gt;&#xA;" OwnerUserId="40856" LastActivityDate="2021-02-07T23:05:45.523" Title="Hyper-plane in logistic regression vs linear regression for same number of features" Tags="&lt;linear-regression&gt;&lt;logistic-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23494" PostTypeId="5" CreationDate="2020-09-09T16:16:05.143" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-09T16:16:05.143" LastActivityDate="2020-09-09T16:16:05.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23495" PostTypeId="4" CreationDate="2020-09-09T16:16:05.143" Score="0" Body="For questions about production systems in the context of artificial intelligence. A production system combines a working memory of facts (also called atomic sentences) with condition-action rules (also called production rules, if-then rules, or just plain rules). " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-09T16:25:34.240" LastActivityDate="2020-09-09T16:25:34.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23496" PostTypeId="1" CreationDate="2020-09-09T20:43:33.953" Score="0" ViewCount="47" Body="&lt;p&gt;In &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;example 5.5 of Sutton and Barto's book&lt;/a&gt; for proving infinite variance of first visit monte carlo ordinary importance sampling estimator,&#xA;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[(\Pi_t\frac{\pi(A_t|S_t)}{b(A_t|S_t)}G_0)^2]$&lt;/span&gt; is proven to be infinite by taking values from the example, but is there any way we can prove this in general without using any example?&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="37611" LastEditDate="2020-09-09T21:59:09.770" LastActivityDate="2020-09-09T21:59:09.770" Title="How to prove variance infinite of monte carlo ordinary importance sampling estimator" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;off-policy-methods&gt;&lt;sutton-barto&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23497" PostTypeId="1" AcceptedAnswerId="23509" CreationDate="2020-09-09T21:36:01.080" Score="0" ViewCount="46" Body="&lt;p&gt;Let's suppose we have calculated the gradient and it came out to be &lt;span class=&quot;math-container&quot;&gt;$f(WX)(1-f(W X))X$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f()$&lt;/span&gt; is the sigmoid function, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; of order &lt;span class=&quot;math-container&quot;&gt;$2\times2$&lt;/span&gt; is the weight matrix, and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is an input vector of order &lt;span class=&quot;math-container&quot;&gt;$2\times 1$&lt;/span&gt;. For ease let &lt;span class=&quot;math-container&quot;&gt;$f(WX)(1-f(W X))=\Bigg[&#xA;\begin{array}{c}&#xA;0.3 \\&#xA;0.8 \\&#xA;\end{array}\Bigg]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X=\Bigg[&#xA;\begin{array}{c}&#xA;1 \\&#xA;0 \\&#xA;\end{array}\Bigg]$&lt;/span&gt;. When we multiply these vectors we will multiply them as &lt;span class=&quot;math-container&quot;&gt;$f(WX)(1-f(W X))\times X^T$&lt;/span&gt; i.e &lt;span class=&quot;math-container&quot;&gt;$\Bigg[&#xA;\begin{array}{c}&#xA;0.3 \\&#xA;0.8 \\&#xA;\end{array}\Bigg]\times[1 \quad0]$&lt;/span&gt;. I do this because I know that we need this gradient to update a &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; weight matrix, hence, the gradient should have size &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt;. But, I don't know the law/rule behind this, if I was just given the values and had no knowledge that we need the solution to update the weight matrix, then, I might have done something like &lt;span class=&quot;math-container&quot;&gt;$[0.3 \quad 0.8]\times\Bigg[&#xA;\begin{array}{c}&#xA;1 \\&#xA;0 \\&#xA;\end{array}\Bigg]$&lt;/span&gt; which will return a scalar. For a long chain of such operations (multiple derivatives in applying chain rule, resulting in many vectors), how do we know if the multiplication of two vectors should return a vector or matrix (dot or cross product)?&lt;/p&gt;&#xA;" OwnerUserId="38423" LastActivityDate="2020-09-10T21:13:15.407" Title="What are the rules behind vector product in gradient?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;&lt;deep-neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23498" PostTypeId="1" AcceptedAnswerId="23501" CreationDate="2020-09-10T08:06:02.687" Score="3" ViewCount="72" Body="&lt;p&gt;I am modelling a ride-hailing system where passenger requests continuously arrive into the system. An RL model is developed to learn how to match those requests with drivers efficiently.&lt;/p&gt;&#xA;&lt;p&gt;Basically, the system can run infinitely as long as there are requests arriving (infinite horizon reality). However, in order for the RL training to conduct, the episode length should be restricted to some finite duration, say &lt;span class=&quot;math-container&quot;&gt;$[0,T]$&lt;/span&gt; (finite horizon training).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My question is how to implement the learned policy based on finite horizon &lt;span class=&quot;math-container&quot;&gt;$[0,T]$&lt;/span&gt; to the real system with infinite horizon &lt;span class=&quot;math-container&quot;&gt;$[0,\infty]$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I expect there would be a conflict of objectives. The value function near &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is partially cut off in a finite horizon and would become an underestimate and affect policy performance in an infinite horizon implementation. To this end, I doubt the applicability of the learned policy.&lt;/p&gt;&#xA;" OwnerUserId="32145" LastEditorUserId="32145" LastEditDate="2020-09-10T08:12:29.230" LastActivityDate="2020-09-10T13:00:27.650" Title="How to implement RL policies learned on a finite horizon?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23500" PostTypeId="2" ParentId="21849" CreationDate="2020-09-10T12:22:44.333" Score="1" Body="&lt;p&gt;As mentioned in the comments your assumption about independence is wrong. Here's why. To prove independence we need to show the following holds:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(X=x, Y=y) = P(X=x)P(Y=y)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;in the case of RL this becomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(X=a, X=a') = P(X=a)P(Y=a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The left hand side has the value:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(X=a, Y=a') = b(A_t = a| S_t = s) p(s'|a,s) b(A_{t+1} = a'|, S_{t+1} = s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;while the right hand side has the value:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(X=a)P(Y=a') = b(A_t = a| S_t = s)b(A_{t+1} = a'| S_{t+1} = s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And hence not independent.&lt;/p&gt;&#xA;&lt;p&gt;Now let use look at why the following expression holds:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Eq.5.14:   &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[\rho_{t:T-1}R_{t+k}] = \mathbb{E}[\rho_{t:t+k-1}R_{t+k}]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I will not derive the exact expressions, but I hope you can form the reasoning I provide. By the rules of probability we know that sum of joint probability is equal to 1 i.e.:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{X_1..X_n} P(X_1=a_1, X_2=a_2,...X_n = a_n) = 1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have alredy showed above, the trajectory is not independent. So &lt;span class=&quot;math-container&quot;&gt;$R_{t+k}$&lt;/span&gt; will depend on the trajectory &lt;span class=&quot;math-container&quot;&gt;$S_{t:t+k-1}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$S_{t:t+k-1}$&lt;/span&gt; is a particular trajectory. At the end of this trajectory we get a reward &lt;span class=&quot;math-container&quot;&gt;$R_{t+k}$&lt;/span&gt; and thus &lt;span class=&quot;math-container&quot;&gt;$R_{t+k}$&lt;/span&gt; is exclusively a function of &lt;span class=&quot;math-container&quot;&gt;$S_{t:t+k-1}$&lt;/span&gt; i.e. &lt;span class=&quot;math-container&quot;&gt;$R_{t+k} = f(S_{t:t+k-1})$&lt;/span&gt;. The trajectory after this &lt;span class=&quot;math-container&quot;&gt;$S_{t+k:T-1}$&lt;/span&gt; irrelevant since it will always sum up of to 1. i.e once you have reached a particular state at time step &lt;span class=&quot;math-container&quot;&gt;$t+k-1$&lt;/span&gt; you are now conditioning based on that &lt;span class=&quot;math-container&quot;&gt;$P(S_{t+k:T-1}|S_{t:t+k-1})$&lt;/span&gt; and taking the expected value over all trajectories possible from thereon i.e. &lt;span class=&quot;math-container&quot;&gt;$\sum_{S_{t+k:T-1}} P(S_{t+k:T-1}|S_{t:t+k-1})$&lt;/span&gt; which is 1 by probability rules. Thus, what you are really doing is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(S_{t:t+k-1})R_{t+k}(\sum_{S_{t+k:T-1}} P(S_{t+k:T-1}|S_{t:t+k-1}))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and hence the remaining trajectory has no contribution.&lt;/p&gt;&#xA;&lt;p&gt;Another way of thinking this is you are taking weighted trajectories till time step &lt;span class=&quot;math-container&quot;&gt;$t+k-1$&lt;/span&gt; weighted by rewards &lt;span class=&quot;math-container&quot;&gt;$R_{t+k}$&lt;/span&gt; and hence you cannot sum up to 1. The rest of the trajectory after &lt;span class=&quot;math-container&quot;&gt;$t+k-1$&lt;/span&gt; will sum up to 1.&lt;/p&gt;&#xA;&lt;p&gt;I hope this qualitative description suffices. You can do the maths, but you must be careful with the notations and the assumptions you make.&lt;/p&gt;&#xA;&lt;p&gt;Also all the equations are correct, I hope you can indirectly see it from my reasoning.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2020-09-10T12:56:15.340" LastActivityDate="2020-09-10T12:56:15.340" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23501" PostTypeId="2" ParentId="23498" CreationDate="2020-09-10T12:53:08.453" Score="2" Body="&lt;p&gt;A normal way to deal with training on infinite horizon (aka &amp;quot;continuing&amp;quot; or &amp;quot;non-episodic&amp;quot;) problems is to use TD learning or other bootstrapping methods (of which Q-learning in DQN is one example), and to treat the cutoff at &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; for pseudo-episodes as a training artefact.&lt;/p&gt;&#xA;&lt;p&gt;If the state at time &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; was really a terminal state, the TD target would be just &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$q(s^T,\cdot) = 0$&lt;/span&gt; by definition, but that doesn't apply in your case.&lt;/p&gt;&#xA;&lt;p&gt;So always use the bootstrapped TD target - e.g. &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \text{max}_{a'} \hat{q}(s',a',\theta)$&lt;/span&gt; for single step TD target with a Q estimate having &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; as learned params - and don't treat the horizon data any differently.&lt;/p&gt;&#xA;&lt;p&gt;If you do this, then your concerns about under-estimates should not be an issue. Your pseudo-episodes do need to be long enough to observe the impact of multiple requests in the long term is the main issue (setting &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; too low so that the system does not reach any kind of equilibrium would be a problem).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/BF00114727&quot; rel=&quot;nofollow noreferrer&quot;&gt;You could also use an average reward setting&lt;/a&gt; and differential value functions. It is slightly better from a theoretical standpoint, but Q-learning and DQN is fine if you don't want to be bothered with that. The same basic rule applies - ignore &amp;quot;episode end&amp;quot; for constructing TD targets because it is just a training artefact. Also ensure you set &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; high enough that the long term impacts of a policy are observable.&lt;/p&gt;&#xA;&lt;p&gt;If your starting state is special (e.g. cars all in fixed places, and no requests in progress) this could also be an issue, because the real world system will rarely be in that state but you will have many episode starts with it. If you cannot start the system in a reasonable random initial state for your problem, you may also want to discard data from episode starts and allow a run-in time before using data for training from the pseudo-episodes.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-10T13:00:27.650" LastActivityDate="2020-09-10T13:00:27.650" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23502" PostTypeId="1" CreationDate="2020-09-10T14:29:43.663" Score="0" ViewCount="18" Body="&lt;p&gt;I'm using a transfert-style based deep learning approach that use VGG (neural network). The latter works well with images of small size (512x512pixels), however it provides distorted results when input images are large (size &amp;gt; 1500px). The author of the approach suggested to divide the input large image to portions and perform style-transfert to portion1 and then to portion2 and finally concatenate the two portions to have a final large result image, because VGG was made for small images... The problem with this method is that the resulting image will have some inconsistent regions at the level of areas where the portions were &amp;quot;glued&amp;quot;. How can I correct these areas ? Is there an alternative approach to this dividing method ?&#xA;thanks&lt;/p&gt;&#xA;" OwnerUserId="40901" LastActivityDate="2020-09-10T14:29:43.663" Title="Strategy to input and get large images in VGG neural networks" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;vgg&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23504" PostTypeId="1" CreationDate="2020-09-10T18:46:36.457" Score="1" ViewCount="47" Body="&lt;p&gt;I wonder if we can use Natural Language Processing (NLP) to process programming code:&#xA;Given a piece of code, can we&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Translate&lt;/strong&gt; it to human language to &lt;em&gt;understand&lt;/em&gt; what it does? The input could be a function definition（normally lack of documentation) in Python and the output could be the documentation for that function.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Compile&lt;/strong&gt; or translate it to another programming language? Compile Python code to C or machine code, or translate C code to Python code?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40884" LastActivityDate="2020-10-10T20:01:00.117" Title="Can we use NLP to understand/parse/compile programming code?" Tags="&lt;natural-language-processing&gt;&lt;machine-translation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23505" PostTypeId="2" ParentId="23504" CreationDate="2020-09-10T19:17:25.530" Score="1" Body="&lt;p&gt;Yes, and that is the ambition of the &lt;a href=&quot;https://www.decoder-project.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Decoder&lt;/a&gt; project (H2020 funded in Europe). Analyzing with NLP techniques the comments in e.g. C or C++ source code (of open source projects, and perhaps their &lt;a href=&quot;https://git-scm.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;git&lt;/code&gt;&lt;/a&gt; logs).&lt;/p&gt;&#xA;&lt;p&gt;I even happen to be on the photo.&lt;/p&gt;&#xA;&lt;p&gt;My dream is to try similar things, in a few years, in &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt;. You could join that project, BTW.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-09-10T19:22:32.080" LastActivityDate="2020-09-10T19:22:32.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23506" PostTypeId="1" CreationDate="2020-09-10T20:08:19.390" Score="0" ViewCount="191" Body="&lt;p&gt;I want to create a CNN in Python, specifically, only with NumPy, if possible. For optimizing the time of convolution (actually correlation) in the network, I wanna try to use FFT-based convolution. The data that needs to be convoluted (correlated) is a 4D image tensor with shape &lt;code&gt;[batch_size, width, height, channels]&lt;/code&gt; and 4D filter tensor &lt;code&gt;[filter_width, filter_height, in_channel, out_channel]&lt;/code&gt;. I read a lot of articles about FFT-based convolution, but they aren't doing it in my way. Thus, I need your help.&lt;/p&gt;&#xA;&lt;p&gt;How could I fft-convolve a 4D image and a 4D filter with stride?&lt;/p&gt;&#xA;" OwnerUserId="38736" LastEditorUserId="2444" LastEditDate="2020-09-10T21:34:20.643" LastActivityDate="2021-05-08T17:02:50.733" Title="How could I convolve a 4D image and a 4D filter with stride?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;convolution&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23507" PostTypeId="1" AcceptedAnswerId="23520" CreationDate="2020-09-10T20:20:58.943" Score="4" ViewCount="664" Body="&lt;p&gt;I am looking for a book about machine learning that would suit my physics background. I am more or less familiar with classical and complex analysis, theory of probability, сcalculus of variations, matrix algebra, etc. However, I have not studied topology, measure theory, group theory, and other more advanced topics. I try to find a book that is written neither for beginners, nor for mathematicians.&lt;/p&gt;&#xA;&lt;p&gt;Recently, I have read the great book &amp;quot;Statistical inference&amp;quot; written by Casella and Berger. They write in the introduction that &amp;quot;The purpose of this book is to build theoretical statistics (as different from mathematical statistics) from the first principles of probability theory&amp;quot;. So, &lt;em&gt;I am looking for some &amp;quot;theoretical books&amp;quot; about machine learning&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are many online courses and brilliant books out there that focus on the practical side of applying machine learning models and using the appropriate libraries. It seems to me that there are no problems with them, but I would like to find a book on theory.&lt;/p&gt;&#xA;&lt;p&gt;By now I have skimmed through the following books&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0387310738&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern Recognition And Machine Learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It looks very nice. The only point of concern is that the book was published in 2006. So, I am not sure about the relevance of the chapters considering neural nets, since this field is developing rather fast.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0387848576&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;The elements of statistical learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This book also seems very good. It covers most of the topics as well as the first book. However, I am feeling that its style is different and I do not know which book will suit me better.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0136042597&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence. A Modern Approach&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This one covers more recent topics, such as natural language processing. As far as I understand, it represents the view of a computer scientist on machine learning.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0262018020&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning A Probabilistic Perspective&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Maybe it has a slight bias towards probability theory, which is stated in the title. However, the book looks fascinating as well.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I think that the first or the second book should suit me, but I do not know what decision to make.&lt;/p&gt;&#xA;&lt;p&gt;I am sure that I have overlooked some books.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Are there some other ML books that focus on theory?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="40905" LastEditorUserId="2444" LastEditDate="2021-01-16T19:37:17.240" LastActivityDate="2021-01-24T21:12:37.150" Title="What are other examples of theoretical machine learning books?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;computational-learning-theory&gt;&lt;books&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23508" PostTypeId="2" ParentId="23507" CreationDate="2020-09-10T21:07:50.897" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://users.isr.ist.utl.pt/%7Ewurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern Recognition And Machine Learning&lt;/a&gt; is a great theoretical book. I don't know anything better on standard ML. I read several pages from it myself and all my colleagues researchers suggest to look there if you are not sure about some concepts. The 2 problems with it are that it's huge and it doesn't cover almost all deep learning models known for today.&lt;/p&gt;&#xA;&lt;p&gt;So, in addition, I'd suggest you look at &lt;a href=&quot;https://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; by Ian Goodfellow et al.&lt;/p&gt;&#xA;&lt;p&gt;Your concerns about not studying topology, measure theory and group theory are groundless. These sections of math aren't prerequisites in any way, they aren't even discussed anywhere I know.&lt;/p&gt;&#xA;&lt;p&gt;Actually, ML theory is more like probability theory and statistics. Especially, statistical learning theory (which is nothing more than probability theory and statistics). I haven't read any books on SLT so have a look at &lt;a href=&quot;https://ai.stackexchange.com/a/20358/2444&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastEditorUserId="2444" LastEditDate="2021-01-16T19:34:15.397" LastActivityDate="2021-01-16T19:34:15.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23509" PostTypeId="2" ParentId="23497" CreationDate="2020-09-10T21:13:15.407" Score="0" Body="&lt;p&gt;It helps to think of each output dimension separately.&lt;/p&gt;&#xA;&lt;p&gt;You have &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; which is an &lt;span class=&quot;math-container&quot;&gt;$(2 \times 1)$&lt;/span&gt; vector, and &lt;span class=&quot;math-container&quot;&gt;$W_1$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$(1 \times 2 )$&lt;/span&gt; vector. Their product is a scalar, of which we then take the sigmoid to get our output &lt;span class=&quot;math-container&quot;&gt;$Y_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The gradient of this w.r.t. &lt;span class=&quot;math-container&quot;&gt;$W_1$&lt;/span&gt; will be &lt;span class=&quot;math-container&quot;&gt;$f(WX) (1 - f(WX)) X^T$&lt;/span&gt;, which has the appropriate dimensions.&lt;/p&gt;&#xA;&lt;p&gt;Then, you just stack these for all your outputs, giving you the shape you got.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-10T21:13:15.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23511" PostTypeId="2" ParentId="23506" CreationDate="2020-09-10T21:14:56.453" Score="1" Body="&lt;p&gt;I think this should work for you:&#xA;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.correlate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.signal.correlate | SciPy&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I used it myself while I was writing a CNN in numpy.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastEditorUserId="32410" LastEditDate="2021-05-08T17:02:50.733" LastActivityDate="2021-05-08T17:02:50.733" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23513" PostTypeId="2" ParentId="23493" CreationDate="2020-09-10T21:57:57.417" Score="0" Body="&lt;p&gt;Mm, I understand what you want to say, but I think you're slightly wrong with terminology. If you consider a d-dimensional vector space, people usually a hyper-plane (d - 1)-dimensional, because it's a subspace of lower dimensionality. So for your example d = 2 the logistic regression would fit a 1-dimentional line (which would separate 2 classes) and the linear regression would fit a 2-dimentional plane (for every combination of 2 features the prediction would lay on some plane in 3-D space). So the general rule would be logreg -&amp;gt; (d-1)-dimensional hyperplane in d-dimensional vector space, linear regression -&amp;gt; d-dimensional hyperplane in (d+1)-dimensional vector space.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastActivityDate="2020-09-10T21:57:57.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23514" PostTypeId="5" CreationDate="2020-09-10T22:05:13.780" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-10T22:05:13.780" LastActivityDate="2020-09-10T22:05:13.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23515" PostTypeId="4" CreationDate="2020-09-10T22:05:13.780" Score="0" Body="For questions related to the Jaccard similarity in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-10T22:05:13.780" LastActivityDate="2020-09-10T22:05:13.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23517" PostTypeId="1" AcceptedAnswerId="23519" CreationDate="2020-09-11T06:22:39.670" Score="0" ViewCount="41" Body="&lt;p&gt;I am using pretraining code from &lt;a href=&quot;https://github.com/NVIDIA/DeepLearningExamples&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NVIDIA/DeepLearningExamples&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Pretrain parameters:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 15:47:02,534: INFO tensorflow 140678508230464   init_checkpoint: bertbase3layer-extract-from-google&#xA; 15:47:02,534: INFO tensorflow 140678508230464   optimizer_type: lamb&#xA; 15:47:02,534: INFO tensorflow 140678508230464   max_seq_length: 64&#xA; 15:47:02,534: INFO tensorflow 140678508230464   max_predictions_per_seq: 5&#xA; 15:47:02,534: INFO tensorflow 140678508230464   do_train: True&#xA; 15:47:02,535: INFO tensorflow 140678508230464   do_eval: False&#xA; 15:47:02,535: INFO tensorflow 140678508230464   train_batch_size: 32&#xA; 15:47:02,535: INFO tensorflow 140678508230464   eval_batch_size: 8&#xA; 15:47:02,535: INFO tensorflow 140678508230464   learning_rate: 5e-05&#xA; 15:47:02,535: INFO tensorflow 140678508230464   num_train_steps: 10000000&#xA; 15:47:02,535: INFO tensorflow 140678508230464   num_warmup_steps: 10000&#xA; 15:47:02,535: INFO tensorflow 140678508230464   save_checkpoints_steps: 1000&#xA; 15:47:02,535: INFO tensorflow 140678508230464   display_loss_steps: 10&#xA; 15:47:02,535: INFO tensorflow 140678508230464   iterations_per_loop: 1000&#xA; 15:47:02,535: INFO tensorflow 140678508230464   max_eval_steps: 100&#xA; 15:47:02,535: INFO tensorflow 140678508230464   num_accumulation_steps: 1&#xA; 15:47:02,535: INFO tensorflow 140678508230464   allreduce_post_accumulation: False&#xA; 15:47:02,535: INFO tensorflow 140678508230464   verbose_logging: False&#xA; 15:47:02,535: INFO tensorflow 140678508230464   horovod: True&#xA; 15:47:02,536: INFO tensorflow 140678508230464   report_loss: True&#xA; 15:47:02,536: INFO tensorflow 140678508230464   manual_fp16: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464   amp: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464   use_xla: True&#xA; 15:47:02,536: INFO tensorflow 140678508230464   init_loss_scale: 4294967296&#xA; 15:47:02,536: INFO tensorflow 140678508230464   ?: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464   help: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464   helpshort: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464   helpfull: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464   helpxml: False&#xA; 15:47:02,536: INFO tensorflow 140678508230464 **************************&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Pretrain loss: (I remove nsp_loss)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'throughput_train': 1196.9646684552622, 'mlm_loss': 0.9837073683738708, 'nsp_loss': 0.0, 'total_loss': 0.9837073683738708, 'avg_loss_step': 1.200513333082199, 'learning_rate': '0.00038143058'}&#xA;{'throughput_train': 1230.5063662500734, 'mlm_loss': 1.3001925945281982, 'nsp_loss': 0.0, 'total_loss': 1.3001925945281982, 'avg_loss_step': 1.299936044216156, 'learning_rate': '0.00038143038'}&#xA;{'throughput_train': 1236.4348949169155, 'mlm_loss': 1.473339319229126, 'nsp_loss': 0.0, 'total_loss': 1.473339319229126, 'avg_loss_step': 1.2444063007831574, 'learning_rate': '0.00038143017'}&#xA;{'throughput_train': 1221.2668264552692, 'mlm_loss': 0.9924975633621216, 'nsp_loss': 0.0, 'total_loss': 0.9924975633621216, 'avg_loss_step': 1.1603020071983337, 'learning_rate': '0.00038142994'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Fine-tune code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.train_op = tf.train.AdamOptimizer(0.00001).minimize(self.loss, global_step=self.global_step)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Fine-tune accuracy: (restore from my ckpt pretrained from &lt;a href=&quot;https://github.com/NVIDIA/DeepLearningExamples&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NVIDIA/DeepLearningExamples&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epoch 1:&#xA;training step 895429, loss 4.98, acc 0.079&#xA;dev loss 4.853, acc 0.092&#xA;&#xA;epoch 2:&#xA;training step 895429, loss 4.97, acc 0.080&#xA;dev loss 4.823, acc 0.092&#xA;&#xA;epoch 3:&#xA;training step 895429, loss 4.96, acc 0.081&#xA;dev loss 4.849, acc 0.092&#xA;&#xA;epoch 4:&#xA;training step 895429, loss 4.95, acc 0.082&#xA;dev loss 4.843, acc 0.092&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Without restore the pretrained ckpt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epoch 1:&#xA;training step 10429, loss 2.48, acc 0.606&#xA;dev loss 1.604, acc 0.8036&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Restore the google's BERT-Base pretrained ckpt. Or restore from a pretrained ckpt pretrained from &lt;a href=&quot;https://github.com/guotong1988/BERT-GPU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/guotong1988/BERT-GPU&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epoch 1:&#xA;training loss 1.89, acc 0.761&#xA;dev loss 1.351, acc 0.869&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40765" LastEditorUserId="40765" LastEditDate="2020-09-14T02:40:13.807" LastActivityDate="2020-09-14T11:02:01.323" Title="BERT: After pretraining 880000 step, why fine-tune not work?" Tags="&lt;transformer&gt;&lt;bert&gt;&lt;transfer-learning&gt;&lt;pretrained-models&gt;&lt;fine-tuning&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-09-14T11:02:07.267" ContentLicense="CC BY-SA 4.0" />
  <row Id="23518" PostTypeId="1" CreationDate="2020-09-11T08:23:08.893" Score="1" ViewCount="50" Body="&lt;p&gt;I am working on a deep Q-learning project. My project is different than normal deep Q-learning. The rewards of my neural network must be positive because I need their values to importance sample actions. I know that I can't use ReLU as the activation function of my neural network. So the only suitable functions which I know are sigmoid, softmax and exponential function. I tried working with sigmoid and softmax but they generate wrong results and the loss function diverges.&#xA;There are two terminal states in my model. Their rewards are 1 and 0. All other states don't have any immediate rewards.&lt;/p&gt;&#xA;" OwnerUserId="35633" LastActivityDate="2021-02-09T17:03:30.427" Title="What are some suitable positive functions as activations of neural networks?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23519" PostTypeId="2" ParentId="23517" CreationDate="2020-09-11T09:47:58.747" Score="0" Body="&lt;p&gt;change&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bert_output = bert_model.get_pooled_output()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bert_output = tf.reduce_mean(bert_model.get_sequence_output()[:,1:,:],1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40765" LastActivityDate="2020-09-11T09:47:58.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23520" PostTypeId="2" ParentId="23507" CreationDate="2020-09-11T10:21:58.013" Score="1" Body="&lt;p&gt;Some of the books that you mention are often used as reference books in introductory courses to machine learning or artificial intelligence.&lt;/p&gt;&#xA;&lt;p&gt;For example, if I remember correctly, in my introductory course to machine learning, the professor suggested the book &lt;a href=&quot;http://users.isr.ist.utl.pt/%7Ewurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern Recognition And Machine Learning&lt;/a&gt; (2006) by Bishop, although we never used it during the lessons. This is a good book, but, in my opinion, it covers many topics, such as variational inference or sampling methods, that are not suited for an introductory course.&lt;/p&gt;&#xA;&lt;p&gt;The book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence. A Modern Approach&lt;/a&gt;, by Norvig and Russell, definitely does not focus on machine learning, but it covers many other aspects of &lt;strong&gt;artificial intelligence&lt;/strong&gt;, such as search, planning, knowledge representation, machine learning, robotics, natural language processing or computer vision. This is probably the book that you should read and use if you want to have an extensive overview of the AI field. Although I never fully read it, I often used it as a reference, as I use the other mentioned book. For instance, during my bachelor's and, more specifically, an introductory course to artificial intelligence, we had used this book as the reference book, but note that there are &lt;a href=&quot;https://ai.stackexchange.com/a/25799/2444&quot;&gt;other books that provide an extensive overview of the AI field&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The other two books are not as famous as these two, but they are probably also good books, although their focus may be different.&lt;/p&gt;&#xA;&lt;p&gt;There are at least three other books that I think you should also be aware of, given that they also cover the actual &lt;em&gt;theory of learning&lt;/em&gt;, aka &lt;strong&gt;(computational) learning theory&lt;/strong&gt;, before diving into more specific topics, such as kernel methods.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://profsite.um.ac.ir/%7Emonsefi/machine-learning/pdf/Machine-Learning-Tom-Mitchell.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt; (1997) by Tom Mitchell&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://mitpress.ublish.com/ereader/7093/?preview=#page/Cover&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations of Machine Learning&lt;/a&gt; (2nd edition, 2018) by Mehryar Mohri et al.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Understanding Machine Learning: From Theory to Algorithms&lt;/a&gt; (2014) by Shai Shalev-Shwartz et al.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You can find more books on learning theory &lt;a href=&quot;https://ai.stackexchange.com/a/20358/2444&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-24T21:12:37.150" LastActivityDate="2021-01-24T21:12:37.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23521" PostTypeId="2" ParentId="21849" CreationDate="2020-09-11T10:41:44.953" Score="0" Body="&lt;p&gt;&lt;strong&gt;First Part&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can reduce variance in off-policy importance sapling, even in the absence of discounting (&lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt;). Notice that the off-policy estimators are made up of terms like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{t:T-1}G_t = \rho_{t:T-1} (R_{t+1} + \gamma R_{t+2} + \dots+ \gamma^{T-t-1}R_{T})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and consider the second term, imagine &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;=&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{t:T-1}R_{t+2} = \frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1})......\pi(A_{T-1}|S_{T-1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1})...... b(A_{T-1}|S_{T-1})} R_{t+2}$$&lt;/span&gt;&#xA;In above equation, the term &lt;span class=&quot;math-container&quot;&gt;$\pi(A_t|S_t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi(A_{t+1}|S_{t+1})$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R_{t+2}$&lt;/span&gt; are correleated, all the other terms are independent of each other.&lt;/p&gt;&#xA;&lt;p&gt;Notice the very import property of expectation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$E[ab] = E[a] E[b]$&lt;/span&gt; if and only if &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; are independent random variables.&lt;/p&gt;&#xA;&lt;p&gt;Now:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$  E[\frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1}).....\pi(A_{T-1}|S_{T-1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1}).....b(A_{T-1}|S_{T-1})} R_{t+2}]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$  = E[\frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1})} R_{t+2}]  E[\frac{\pi(A_{t+2}|S_{t+2})}{b(A_{t+2}|S_{t+2})}] .....  E[\frac{\pi(A_{T-1}|S_{T-1})}{b(A_{T-1}|S_{T-1})}]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ = E[\frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1})} R_{t+2}] \sum_a b(a|s_{t+2}) \frac{\pi(a|s_{t+2}}{b(a|s_{t+2}}.....\sum_a b(a|s_{T-1}) \frac{\pi(a|s_{T-1}}{b(a|s_{T-1}} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ = E[\frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1})} R_{t+2}] \sum_a \pi(a|s_{t+2}).....\sum_a \pi(a|s_{T-1})$$&lt;/span&gt;&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ = E[\frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1})} R_{t+2}] 1 * 1  $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ = E[\frac{\pi(A_t|S_t) \pi(A_{t+1}|S_{t+1})}{b(A_t|S_t) b(A_{t+1}|S_{t+1})} R_{t+2}]  $$&lt;/span&gt;&#xA;therefore&#xA;&lt;span class=&quot;math-container&quot;&gt;$$  E[\rho_{t:T-1}R_{t+2}] = E[\rho_{t:t+1} R_{t+2}]$$&lt;/span&gt;&#xA;If we repeat this analysis for the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th term, we will get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$E[\rho_{t:T-1}R_{t+k}] = E[\rho_{t:t+k-1} R_{t+k}]$$&lt;/span&gt;&#xA;It follows that the expectation of our original term can be written:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$E[\rho_{t:T-1}G_{t}] = E[\tilde{G_{t}}]$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\tilde{G}_t \doteq \rho_{t:t}R_{t+1} + \gamma \rho_{t:t+1}R_{t+2} + \gamma^{2} \rho_{t:t+2}R_{t+3} + ...... + \gamma^{T-t-1} \rho_{t:T-1}R_{T}$$&lt;/span&gt;&#xA;We call this idea per reward importance sampling. It follows immediately that there is an alternative importance sampling estimate, with the same unbiased expectation as the ordinary importance sampling estimate:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V(s) \doteq \frac{\sum_{t\in\mathcal{T}(s)} \tilde{G}_t}{|\mathcal{T}(s)|}$$&lt;/span&gt;&#xA;which we might expect to sometimes be of lower variance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second Part&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The reward &lt;span class=&quot;math-container&quot;&gt;$R_{k+1}$&lt;/span&gt; depends on the previous &lt;span class=&quot;math-container&quot;&gt;$\pi(a_1|s_1)$&lt;/span&gt; up to &lt;span class=&quot;math-container&quot;&gt;$\pi(a_{k-1}|s_{k-1})$&lt;/span&gt;. So, you can't seperate them and treat them as independent variable just you did on the aforementioned example.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-09-11T10:54:21.943" LastActivityDate="2020-09-11T10:54:21.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23523" PostTypeId="1" CreationDate="2020-09-11T12:34:48.487" Score="0" ViewCount="38" Body="&lt;p&gt;Each person probably uses an app that tracks his/her position periodically and sends it to our servers. What I want is to use these data to train a model to predict the rush hours of each bus-stop on the map, so we can send extra buses to handle the predicted cumulation before it happens.&lt;/p&gt;&#xA;&lt;p&gt;I have no experience in AI nor machine learning. So, which model should I use to do this?&lt;/p&gt;&#xA;" OwnerUserId="40917" LastEditorUserId="2444" LastEditDate="2020-09-11T14:28:35.643" LastActivityDate="2021-02-12T05:07:55.677" Title="How to train a model to predict the number of people at a certain bus stop before they cumulate in large numbers?" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;prediction&gt;&lt;ai-development&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23524" PostTypeId="1" CreationDate="2020-09-11T12:39:05.683" Score="0" ViewCount="43" Body="&lt;p&gt;I am trying to make a NN(probably with dense layers) to map a &lt;strong&gt;specific&lt;/strong&gt; input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and predict the output of any other input I give it.&lt;/p&gt;&#xA;&lt;p&gt;I have 2 files - one with the inputs and another with all the corresponding outputs and would probably use a bunch of Dense Layers with word embeddings to vectorize it into higher dimensions. However, I cannot find any good resources out there for that.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know how to accomplish such an NN? Which architectures are best for pattern matching? examples, links, and other resources would be very welcome. I was considering using RNN's but found them not very good in the pattern matching tasks so had ditched them. I would still consider them if someone can provide a plausible explanation...&lt;/p&gt;&#xA;" OwnerUserId="36322" LastActivityDate="2021-05-08T20:04:55.503" Title="How to use a NN for seq2seq tasks?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;prediction&gt;&lt;pattern-recognition&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23525" PostTypeId="1" CreationDate="2020-09-11T16:25:49.117" Score="0" ViewCount="44" Body="&lt;p&gt;I deal with a text classification problem, where there are only two classes; relevant and irrelevant. That is, a text might be relevant / irrelevant with a predefined topic. I have a dataset that consists of only relevant text instances. I am thinking of using zero-shot learning. A way to represent the semantic space (prototype) for the relevant instances is to use word embeddings and to encode the text in a fixed size vector. But how can I represent the semantic space (prototype) for the irrelevant class which should represent all instances that are not relevant ?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-09-11T21:34:13.070" LastActivityDate="2020-09-11T21:34:13.070" Title="Is there Binary Zero-Shot Learning with no defined prototypes for the unseen class?" Tags="&lt;classification&gt;&lt;text-classification&gt;&lt;zero-shot-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23527" PostTypeId="1" CreationDate="2020-09-11T20:56:36.993" Score="0" ViewCount="55" Body="&lt;p&gt;As we all know, zero shot learning involves a model predicting classes that it has not seen. But we are given all the attributes each class might have.&lt;/p&gt;&#xA;&lt;p&gt;Is it fair to assume that we are &amp;quot;aware&amp;quot; of all the class labels a dataset might have ? (Including the test set)&lt;/p&gt;&#xA;" OwnerUserId="40928" LastActivityDate="2020-10-13T14:03:35.420" Title="Zero shot learning available labels in testing set" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;zero-shot-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23528" PostTypeId="1" AcceptedAnswerId="23532" CreationDate="2020-09-12T01:26:48.367" Score="1" ViewCount="65" Body="&lt;p&gt;This is a bit of a weird question.&lt;/p&gt;&#xA;&lt;p&gt;I am hoping to create an online reference since I have some downtime.  I know some about statistics but very little about computer science.  As a result, the reference guide I am hoping to create will be very statistics oriented - even though I wish that it could be a reference for someone who wants to start from scratch and work their way to AI.&lt;/p&gt;&#xA;&lt;p&gt;While I would love to be involved with AI, from what I have read about ML and AI, seems like AI does not involve much statistics.  (A lot of statistical theory is based off normal assumption and math, and ML seems to bypass that by not requiring strong assumptions nor analytical results). CS seems to be more relevant.&lt;/p&gt;&#xA;&lt;p&gt;And so my question is, since my guide will mostly cover statistics, how relevant would it be for someone who wants to get into AI?  If it's not relevant, then I guess I'll just make my guide for someone who wants to get into stats/data science, as opposed to someone who wants to be an AI researcher.&lt;/p&gt;&#xA;&lt;p&gt;I guess another way to phrase my question is, as an AI researcher, when you &amp;quot;google&amp;quot; stuff, wikipedia things, or go to your notes, what subjects are you looking at and what exactly are you googling?  Are you getting a refresher on how to code back propagation?  Or are you getting a refresher on the pros and cons of L1 vs. L2?  Do you ever look at how to implement a boosting tree or NN using a pre-existing package?&lt;/p&gt;&#xA;&lt;p&gt;Basically, I know that what I can provide will be relevant to HS/college stats and data science students.  But what really want to do is create something useful for aspiring/current AI researchers.  The former is realistic, the latter is a dream.  I want to see if my dream is realistic.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="29801" LastEditorUserId="29801" LastEditDate="2020-09-12T01:31:53.037" LastActivityDate="2020-09-12T12:46:41.820" Title="As an AI researcher, what subjects do you find yourself referring to most often?" Tags="&lt;research&gt;" AnswerCount="1" CommentCount="5" ClosedDate="2020-09-12T12:48:31.867" ContentLicense="CC BY-SA 4.0" />
  <row Id="23529" PostTypeId="1" CreationDate="2020-09-12T02:54:16.143" Score="3" ViewCount="110" Body="&lt;p&gt;Genetic algorithms are used to solve many optimization tasks.&lt;/p&gt;&#xA;&lt;p&gt;If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset?&lt;/p&gt;&#xA;&lt;p&gt;We could consider each feature of the initial dataset as a chromosome (or individual), which is then combined with other chromosomes (features) to find more features. Is this possible? Has this been done?&lt;/p&gt;&#xA;&lt;p&gt;I will like to edit the details with an example so that it is easier to understand.&lt;/p&gt;&#xA;&lt;p&gt;Example: In practice cyber-security attacks evolve over time since it finds a new way to breach a system. The main draw-back of intrusion detection model is that it needs to be trained every time attack evolves. So I was hoping if genetic algorithm can be used on the present benchmarked datasets (like NSL-KDD) to come up with a futuristic type dataset maybe after X-number of generations.&#xA;And check if a model is able to classify that generated dataset as well.&lt;/p&gt;&#xA;" OwnerUserId="40929" LastEditorUserId="40929" LastEditDate="2020-09-13T14:43:18.113" LastActivityDate="2020-09-13T14:43:18.113" Title="Can we use genetic algorithms to evolve datasets?" Tags="&lt;datasets&gt;&lt;genetic-algorithms&gt;&lt;genetic-operators&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23530" PostTypeId="2" ParentId="23447" CreationDate="2020-09-12T05:57:21.627" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://pytorch.org/tutorials/beginner/examples_nn/two_layer_net_nn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch docs&lt;/a&gt; define a fully connected ReLU network as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;torch.nn.Sequential(&#xA;    torch.nn.Linear(D_in, H),&#xA;    torch.nn.ReLU(),&#xA;    torch.nn.Linear(H, D_out),&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Neural networks are called are made of neurons. Activation functions only help determine which of these neurons to fire up, meaning they have no learnable nodes themselves through which we can back-propagate gradients. A network with no learnable parameters is therefore not a neural net. So neural nets can't be composed of activation functions only.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What is meant by the quoted text in bold, because my interpretation of that text, (shown below) doesn't seem viable&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, what's given here is not a network that can approximate the said function &lt;span class=&quot;math-container&quot;&gt;$h(s)$&lt;/span&gt;. A two layer ReLU network would resemble:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   x = nn.ReLU(nn.Linear(d_IN, H))&#xA;   x = nn.ReLU(nn.Linear(H, H))&#xA;   out = nn.Linear(x, d_OUT)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Another way to see it is that a network must have an input and ouput layer, and optional hidden layers. It's not possible to use an activation funtion as an input layer, because then you'll have &lt;strong&gt;no way of configuring the number of features to represent your input data&lt;/strong&gt;. In this context, a &lt;code&gt;ReLU&lt;/code&gt; can't represent the features of the observation input &lt;code&gt;s&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To show activation functions have no learnable nodes in them and that this interpretation&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;h = nn.Sequential([nn.ReLu(), nn.ReLu()])&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;is not what the authors are driving across, here a script that counts the number of parameters in a network.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;import torch.nn as nn&#xA;import numpy as np&#xA;&#xA;activation = nn.ReLU&#xA;&#xA;&#xA;def count_params(module):&#xA;    return np.sum([np.prod(x.shape) for x in module.parameters()])&#xA;&#xA;&#xA;one_linear = nn.Sequential(nn.Linear(32, 10), nn.Linear(10, 1))&#xA;linear_act = nn.Sequential(nn.Linear(32, 10), activation(), nn.Linear(10, 1))&#xA;act_only = nn.Sequential(activation(), activation())&#xA;&#xA;t_lin = count_params(linear_act)&#xA;lin = count_params(one_linear)&#xA;act = count_params(act_only)&#xA;&#xA;print(f'Linear only: {lin}, Linear + Activation: {t_lin},' +&#xA;        f'Activation only: {act}')&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;[Out]: Linear only: 341, Linear + Activation: 341, Activation only: 0.0&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The activation-function-only module has zero learnable parameters. Likewise, an activation function adds no parameters to the fully connected layers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update: Links to implementations&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To confirm this answer's interpretation is correct, here are links to GAIL and GAN-GCL example implementations&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;GAIL : &lt;a href=&quot;https://github.com/justinjfu/inverse_rl/blob/9609933389459a3a54f5c01d652114ada90fa1b3/inverse_rl/models/imitation_learning.py#L222&quot; rel=&quot;nofollow noreferrer&quot;&gt;discriminator prediction&lt;/a&gt; (Forward call), &lt;a href=&quot;https://github.com/justinjfu/inverse_rl/blob/9609933389459a3a54f5c01d652114ada90fa1b3/inverse_rl/models/architectures.py#L15&quot; rel=&quot;nofollow noreferrer&quot;&gt;discriminator architecture&lt;/a&gt; (The ReLU net):&lt;/li&gt;&#xA;&lt;li&gt;GAN-GCL : &lt;a href=&quot;https://github.com/justinjfu/inverse_rl/blob/9609933389459a3a54f5c01d652114ada90fa1b3/inverse_rl/models/imitation_learning.py#L415&quot; rel=&quot;nofollow noreferrer&quot;&gt;discriminator prediction&lt;/a&gt;, &lt;a href=&quot;https://github.com/justinjfu/inverse_rl/blob/9609933389459a3a54f5c01d652114ada90fa1b3/inverse_rl/models/architectures.py#L29&quot; rel=&quot;nofollow noreferrer&quot;&gt;discriminator architecture&lt;/a&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="40671" LastEditDate="2020-11-09T15:43:49.723" LastActivityDate="2020-11-09T15:43:49.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23531" PostTypeId="1" CreationDate="2020-09-12T06:23:38.427" Score="0" ViewCount="35" Body="&lt;p&gt;I am novice in machine learning and try to implement &lt;code&gt;Linear regression algorithm&lt;/code&gt; using &lt;code&gt;Gradient Descent&lt;/code&gt;. I am using Google Co-lab to implement the algorithm.&#xA;First step: Implementing Y=mX + C where I have to find out m(slope). I am using Kaggle dataset for predicting No. of rooms available depending upon price of house.&lt;/p&gt;&#xA;&lt;p&gt;My code block is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;import io&#xA;import matplotlib.pyplot as plt&#xA;plt.rcParams['figure.figsize']=(20.0,10.0)&#xA;from google.colab import files &#xA;uploaded = files.upload()&#xA;df = pd.read_csv(io.BytesIO(uploaded['datasets_46927_85203_data.csv'])) &#xA;house_price=df.price&#xA;No_of_bedrooms=df.bedrooms&#xA;&#xA;# Mean of house_price and No_of_bedrooms&#xA;mean_x= np.mean(house_price)&#xA;mean_y=np.mean(No_of_bedrooms)&#xA;&#xA;#Total number of datapoints&#xA;n=len(house_price)&#xA;numerator = 0;&#xA;denominator = 0;&#xA;for i in range (n):&#xA;   numerator += (house_price[i] - mean_x) * (No_of_bedrooms[i] - mean_y)&#xA;   denominator += (house_price[i] - mean_x) ** 2&#xA;slope= numerator / denominator&#xA;constant = mean_y - (slope*mean_x)&#xA;print(slope, constant) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got the Gradient decent implementation from &lt;a href=&quot;https://machinelearningmastery.com/linear-regression-tutorial-using-gradient-descent-for-machine-learning/#:%7E:text=Gradient%20Descent%20is%20the%20process,downhill%20towards%20the%20minimum%20value.&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Gradient Descent Iteration #1&#xA;Let’s start with values of 0.0 for both coefficients.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note: As per my code slope and constant are B1 and B0. However, I did not understand as per the tutorial from where it got B1=0, B0=0 The tutorial already provides a dataset&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x   y&#xA;1   1&#xA;2   3&#xA;4   3&#xA;3   2&#xA;5   5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;If we follow this dataset then slope and constant should be something else.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;B0 = 0.0&lt;/p&gt;&#xA;&lt;p&gt;B1 = 0.0&lt;/p&gt;&#xA;&lt;p&gt;y = 0.0 + 0.0 * x&lt;/p&gt;&#xA;&lt;p&gt;We can calculate the error for a prediction as follows:&lt;/p&gt;&#xA;&lt;p&gt;error = p(i) – y(i)&lt;/p&gt;&#xA;&lt;p&gt;Where p(i) is the prediction for the i’th instance in our dataset and y(i) is the i’th output variable for the instance in the dataset.&lt;/p&gt;&#xA;&lt;p&gt;We can now calculate he predicted value for y using our starting point coefficients for the first training instance:&lt;/p&gt;&#xA;&lt;p&gt;x=1, y=1&lt;/p&gt;&#xA;&lt;p&gt;p(i) = 0.0 + 0.0 * 1&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;This portion is fine for me. They simply use y(i)= m+ x(i)+ c to predict the value of y(i)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;p(i) = 0&lt;/p&gt;&#xA;&lt;p&gt;Using the predicted output, we can calculate our error:&lt;/p&gt;&#xA;&lt;p&gt;error = 0 – 1&lt;/p&gt;&#xA;&lt;p&gt;error = -1&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here is my next doubt: to find out the error we have to subtract the predicted value from the original value. As per the table original value of y=1 but from where they get predicted value 0?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can now use this error in our equation for gradient descent to update the weights. We will start with updating the intercept first, because it is easier.&lt;/p&gt;&#xA;&lt;p&gt;We can say that B0 is accountable for all of the error. This is to say that updating the weight will use just the error as the gradient. We can calculate the update for the B0 coefficient as follows:&lt;/p&gt;&#xA;&lt;p&gt;B0(t+1) = B0(t) – alpha * error&lt;/p&gt;&#xA;&lt;p&gt;Where B0(t+1) is the updated version of the coefficient we will use on the next training instance, B0(t) is the current value for B0, alpha is our learning rate and error is the error we calculate for the training instance. Let’s use a small learning rate of 0.01 and plug the values into the equation to work out what the new and slightly optimized value of B0 will be:&lt;/p&gt;&#xA;&lt;p&gt;B0(t+1) = 0.0 – 0.01 * -1.0&lt;/p&gt;&#xA;&lt;p&gt;B0(t+1) = 0.01&lt;/p&gt;&#xA;&lt;p&gt;Now, let’s look at updating the value for B1. We use the same equation with one small change. The error is filtered by the input that caused it. We can update B1 using the equation:&lt;/p&gt;&#xA;&lt;p&gt;B1(t+1) = B1(t) – alpha * error * x&lt;/p&gt;&#xA;&lt;p&gt;Where B1(t+1) is the update coefficient, B1(t) is the current version of the coefficient, alpha is the same learning rate described above, error is the same error calculated above and x is the input value.&lt;/p&gt;&#xA;&lt;p&gt;We can plug in our numbers into the equation and calculate the updated value for B1:&lt;/p&gt;&#xA;&lt;p&gt;B1(t+1) = 0.0 – 0.01 * -1 * 1&lt;/p&gt;&#xA;&lt;p&gt;B1(t+1) = 0.01&lt;/p&gt;&#xA;&lt;p&gt;We have just finished the first iteration of gradient descent and we have updated our weights to be B0=0.01 and B1=0.01. This process must be repeated for the remaining 4 instances from our dataset.&lt;/p&gt;&#xA;&lt;p&gt;One pass through the training dataset is called an epoch.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I am confused about the learning rate(alpha) as well. From where I get the value of alpha? They are using 0.01 as learning rate. Can I use it as my example?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am also went through the link &lt;a href=&quot;https://ai.stackexchange.com/questions/12666/calculating-parameter-value-using-gradient-descent-for-linear-regression-model&quot;&gt;Calculating Parameter value Using Gradient Descent for Linear Regression Model&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I also watched this &lt;a href=&quot;https://www.youtube.com/watch?v=E5RjzSK0fvY&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now I am exhausted thinking of what do to do and what not to do?&lt;/p&gt;&#xA;&lt;p&gt;Any stepwise suggestion to solve the linear regression using Gradient decent will be helpful for me.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="18384" LastActivityDate="2020-09-12T06:23:38.427" Title="linear regression using gradient decent" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;linear-regression&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23532" PostTypeId="2" ParentId="23528" CreationDate="2020-09-12T08:16:21.590" Score="1" Body="&lt;p&gt;I think AI researcher mostly google some new papers because now there's just a crazy amount of them published. Sometimes people just forget the new concept which was introduced in a paper they read several months ago and they google that concept. Sometimes I forget some loss functions (and intuition behind them) used in a specific area like computer vision, natural language processing or audio processing. Like dice loss or contrastive losses. So these are more advanced things than pros and cons of L1 and L2 losses and how to code backprop. Usually I find the answers on &lt;a href=&quot;https://distill.pub/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://distill.pub/&lt;/a&gt; or &lt;a href=&quot;https://towardsdatascience.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/&lt;/a&gt;. Have a look at those, I think they represent correctly current interests and topics for refreshment in AI research community. From my experience I can say not much statistics is used in contemporary AI research (unless you're doing research in statistical learning theory). Sometimes I google some statistical tests to prove results of my experiments are statistically significant and I think that's it.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastEditorUserId="40906" LastEditDate="2020-09-12T10:34:19.573" LastActivityDate="2020-09-12T10:34:19.573" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23533" PostTypeId="5" CreationDate="2020-09-12T13:43:57.753" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T13:43:57.753" LastActivityDate="2020-09-12T13:43:57.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23534" PostTypeId="4" CreationDate="2020-09-12T13:43:57.753" Score="0" Body="For questions related to zero-shot learning, where training and test classes are disjoint. This problem can be tackled by solving related sub-problems, e.g. learning intermediate attribute classifiers." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T13:43:57.753" LastActivityDate="2020-09-12T13:43:57.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23535" PostTypeId="2" ParentId="23524" CreationDate="2020-09-12T13:50:09.640" Score="0" Body="&lt;p&gt;I think you meant pattern recognition instead of pattern matching in you question. Because pattern matching has nothing to do with NNs as far as I know. So RNNs is the easiest architecture you could use for this task. Have a look at this post. &lt;a href=&quot;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://karpathy.github.io/2015/05/21/rnn-effectiveness/&lt;/a&gt;&#xA;It's long, but it's very well written and explains why RNNs work well with sequential data. In brief RNNs accumulate information about what was fed in them previously and store it in the hidden state vector. That information is a summary of a sequence which helps it to predict the output sequence. RNNs can be applied to variable length input and catch dependencies among different parts of the input (as opposed to dense layers that can be applied to a sequence only point-wise).&#xA;Here's how exactly you can apply reccurent layers (&lt;a href=&quot;https://www.tensorflow.org/guide/keras/rnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/guide/keras/rnn&lt;/a&gt;, &lt;a href=&quot;https://pytorch.org/docs/stable/generated/torch.nn.RNN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pytorch.org/docs/stable/generated/torch.nn.RNN.html&lt;/a&gt;).&#xA;There are more advanced architectures like LSTMs. Also you can try transformers (&lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jalammar.github.io/illustrated-transformer/&lt;/a&gt;) which use attention mechanism (&lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="40906" LastActivityDate="2020-09-12T13:50:09.640" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23536" PostTypeId="5" CreationDate="2020-09-12T14:13:30.517" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T14:13:30.517" LastActivityDate="2020-09-12T14:13:30.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23537" PostTypeId="4" CreationDate="2020-09-12T14:13:30.517" Score="0" Body="For questions about the Kalman filter, which is e.g. used in robotics." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T14:13:30.517" LastActivityDate="2020-09-12T14:13:30.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23538" PostTypeId="5" CreationDate="2020-09-12T15:02:20.457" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T15:02:20.457" LastActivityDate="2020-09-12T15:02:20.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23539" PostTypeId="4" CreationDate="2020-09-12T15:02:20.457" Score="0" Body="For questions about the ADADELTA optimization algorithm used to train neural networks. ADADELTA adaptively changes the learning rate during training. It was proposed in the paper &quot;ADADELTA: An Adaptive Learning Rate Method&quot; (2012) by Matthew D. Zeiler." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T15:02:20.457" LastActivityDate="2020-09-12T15:02:20.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23540" PostTypeId="5" CreationDate="2020-09-12T15:05:40.470" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T15:05:40.470" LastActivityDate="2020-09-12T15:05:40.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23541" PostTypeId="4" CreationDate="2020-09-12T15:05:40.470" Score="0" Body="For questions about Adam, a gradient-based optimization algorithm widely used to train neural networks. It was proposed in the paper &quot;Adam: A Method for Stochastic Optimization&quot; (2014) by Diederik P. Kingma and Jimmy Ba." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T15:05:40.470" LastActivityDate="2020-09-12T15:05:40.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23544" PostTypeId="2" ParentId="23518" CreationDate="2020-09-12T16:08:14.353" Score="0" Body="&lt;p&gt;First of all: an activation function is usually placed after a linear operation and you can have a lot of them (maybe different) in your nn. That's why it would be better to say &lt;span class=&quot;math-container&quot;&gt;$\bf a$&lt;/span&gt; activation function of the neural net and not &lt;span class=&quot;math-container&quot;&gt;$\bf the$&lt;/span&gt; activation function. So if you meant by activation function the last operation which is going to make your outputs non-negative, then you're right, ReLU isn't a good choice. Usually when people need to output some positive values, they take exponent as the last operation. I used it several times and everything was just fine.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastActivityDate="2020-09-12T16:08:14.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23545" PostTypeId="5" CreationDate="2020-09-12T17:32:10.813" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T17:32:10.813" LastActivityDate="2020-09-12T17:32:10.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23546" PostTypeId="4" CreationDate="2020-09-12T17:32:10.813" Score="0" Body="For questions about non-max suppression in the context of object detection." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-12T17:32:10.813" LastActivityDate="2020-09-12T17:32:10.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23547" PostTypeId="1" AcceptedAnswerId="23548" CreationDate="2020-09-12T19:01:13.877" Score="2" ViewCount="118" Body="&lt;p&gt;I read an &lt;a href=&quot;https://blog.coast.ai/five-video-classification-methods-implemented-in-keras-and-tensorflow-99cad29cc0b5&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; about captioning videos and I want to use solution number 4 (extract features with a CNN, pass the sequence to a separate RNN) in my own project.&lt;/p&gt;&#xA;&lt;p&gt;But for me, it seems really strange that in this method we use the Inception model without any retraining or something like that. Every project has different requirements and even if you use pretrained model instead of your own, you should do some training.&lt;/p&gt;&#xA;&lt;p&gt;And I wonder how to do this? For example, I created a project where I use the network with CNN layers and then LSTM and Dense layers. And in every epoch, there is feed-forward and backpropagation through the whole network, all layers. But what if you have CNN network to extract features and LSTM network that takes sequences as inputs. How to train CNN network if there is no defined output? This network should only extract features but the network doesn't know what features. So the question is: How to train CNN to extract relevant features and then passing these features to LSTM?&lt;/p&gt;&#xA;" OwnerUserId="40943" LastEditorUserId="32410" LastEditDate="2021-04-24T10:11:27.207" LastActivityDate="2021-04-24T10:11:27.207" Title="Extract features with CNN and pass as sequence to RNN" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;long-short-term-memory&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23548" PostTypeId="2" ParentId="23547" CreationDate="2020-09-12T19:27:48.653" Score="2" Body="&lt;p&gt;The approach that you don't train the whole net, but just the latter part of it (all starting with lstm in our case), can actually work. The idea is that the inception was already pretrained a very large dataset (imagenet for instance). And it's capable of extracting some useful information from it. Actually there are different domains of images in the imagenet and the inception net needed to capture a vast variety of input information to classify images well. The idea is that the pretrained inception is already capable to extract almost everything what could possibly be useful (unless your images aren't something completely different from imagenet, but that a rare case). Then you adapt the lstm layers and the fully connected layers to correctly process that information. Maybe you aren't going to get the perfect score with this approach and maybe it's better to train the whole large net including the inception part on the new data to lower the distributional shift and that's what people usually do in fact, but it takes more time to train and if you don't have enough data you won't be able to achieve results that are significantly better than those with a frozen CNN part.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastActivityDate="2020-09-12T19:27:48.653" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23549" PostTypeId="1" CreationDate="2020-09-12T20:01:56.940" Score="1" ViewCount="29" Body="&lt;p&gt;I found Keras API that allows to add additional time dimension.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential()&#xA;# define CNN model&#xA;model.add(TimeDistributed(Conv2D(...))&#xA;model.add(TimeDistributed(MaxPooling2D(...)))&#xA;model.add(TimeDistributed(Flatten()))&#xA;# define LSTM model&#xA;model.add(LSTM(...))&#xA;model.add(Dense(...))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I thought that this is only something that allows to write shorter code because we don't need to use loop over images and extracting features.&#xA;But I read somewhere that it is not the same as extracting features with a CNN, ad then passing the sequence to a separate RNN.&lt;/p&gt;&#xA;&lt;p&gt;So what is the difference between these two solutions? What is better in what situations?&lt;/p&gt;&#xA;" OwnerUserId="40943" LastEditorUserId="32410" LastEditDate="2021-04-24T10:12:15.453" LastActivityDate="2021-04-24T10:12:15.453" Title="What is the difference between TimeDistributed CNN+RNN and passing sequence of features to RNN" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23550" PostTypeId="1" CreationDate="2020-09-12T20:52:07.287" Score="0" ViewCount="36" Body="&lt;p&gt;I am training an A3C with stacked LSTM.&lt;/p&gt;&#xA;&lt;p&gt;During initial training, my model was giving descent +ve reward. However, after many episodes, its reward just goes to zero and is continuing for a long time. Is it because of LSTM?&lt;/p&gt;&#xA;&lt;p&gt;Is it normal?&lt;/p&gt;&#xA;&lt;p&gt;Should I expect it to work after the training is over or just terminate the training and increase the density of my network?&lt;/p&gt;&#xA;" OwnerUserId="40051" LastEditorUserId="2444" LastEditDate="2020-09-12T21:26:52.733" LastActivityDate="2021-02-10T03:08:20.823" Title="Why would the reward of A3C with LSTM suddenly drop off after many episodes?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;pytorch&gt;&lt;actor-critic-methods&gt;&lt;a3c&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23551" PostTypeId="1" CreationDate="2020-09-12T21:14:26.737" Score="3" ViewCount="74" Body="&lt;p&gt;Sorry if this is too noob question, I'm just a beginner.&lt;/p&gt;&#xA;&lt;p&gt;I have a data set with companies' info. There are 2 kinds of features: financial (revenue and so on) and general info (like the number of employees and date of registration)&lt;/p&gt;&#xA;&lt;p&gt;I have to predict the probability of default. And the data has gaps: about the half of the companies have no financial data at all. But general features are 100% filled.&lt;/p&gt;&#xA;&lt;p&gt;What is the best practice for such a situation?&lt;/p&gt;&#xA;&lt;p&gt;Will be great if you can give some example links to read.&lt;/p&gt;&#xA;" OwnerUserId="40944" LastEditorUserId="2444" LastEditDate="2020-09-12T23:21:27.677" LastActivityDate="2020-09-12T23:21:27.677" Title="How to perform prediction when some features have missing values?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;data-preprocessing&gt;&lt;feature-engineering&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23552" PostTypeId="2" ParentId="12020" CreationDate="2020-09-12T21:28:52.663" Score="0" Body="&lt;p&gt;There is a lot of related research out there.&lt;/p&gt;&#xA;&lt;p&gt;You can look at contextual bandit problems, which is the basis for &amp;quot;Monte Carlo Tree Search&amp;quot;. Here some clever bookkeeping is used to make sure that branches that looked bad but haven't been explored recently will still get explored eventually. This results in the UCT algorithm, which then got used in combination with deep learning in AlphaGo, but you can use it without deep learning too, if that makes more sense for your particular problem.&lt;/p&gt;&#xA;&lt;p&gt;The exploration / exploitation trade-off is crucial for this kind of problem. Your original estimates of the value of a position will be very uninformed, so it should not be used to prune the search tree too aggressively. This is exactly what UTC does, with provable theoretical guarantees.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-12T21:28:52.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23553" PostTypeId="2" ParentId="23551" CreationDate="2020-09-12T21:58:43.070" Score="3" Body="&lt;p&gt;You should look into &amp;quot;missing values&amp;quot;. This is an entire research field in itself.&lt;/p&gt;&#xA;&lt;p&gt;First, you need to identify the type of missing values:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;They can be missing purely at random.&lt;/li&gt;&#xA;&lt;li&gt;Whether they are missing or not is itself a useful feature, and should be treated as a class of its own.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;(Those two are the best case scenarios.)&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Whether they are missing or not depends on the underlying (unknown) value. For example, a thermometer might fail occasionally if the temperatures get too high. In your case, certain types of companies might be more likely to not share their information.&lt;/li&gt;&#xA;&lt;li&gt;Information might be missing specifically to mislead you, the data analyst. This is the worst possible scenario, and there is not much you can do.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, what do you do about it? A few typical options:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Throw out all the rows with missing data: we do not have enough information about these companies.&lt;/li&gt;&#xA;&lt;li&gt;Throw out all the columns with missing data: this field is not reliably measurable and we shouldn't use it.&lt;/li&gt;&#xA;&lt;li&gt;Try to guess the missing values. This can be done if the amount of missing data is small. Either you train a predictive model based on the non-missing data, or you fill in the median for that type of row, or you fill in the value of the &amp;quot;closest&amp;quot; matching row. This can be dangerous.&lt;/li&gt;&#xA;&lt;li&gt;Some algorithms are OK with missing data. Check the documentation for your models and algorithms to see how they deal with missing values.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-12T21:58:43.070" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23554" PostTypeId="2" ParentId="23529" CreationDate="2020-09-12T22:21:50.660" Score="2" Body="&lt;p&gt;This question raises a lot more questions. It seems like a solution looking for a problem, instead of the other way round.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do you measure the fitness of a feature?&lt;/li&gt;&#xA;&lt;li&gt;What would one of the &amp;quot;evolved datasets&amp;quot; &lt;em&gt;mean&lt;/em&gt;? What does it represent?&lt;/li&gt;&#xA;&lt;li&gt;What would your overal purpose be? If you just wish to generate simulated datasets, there are easier ways to do this, with more control over the various aspects of the resulting datasets.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you want to compute a new set of features to &amp;quot;better&amp;quot; describe a given dataset, there are many approaches to this, such as PCA, ISOMAP, self-organizing maps, ... If this is the kind of thing you're thinking about, I would recommend starting there.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-12T22:21:50.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23556" PostTypeId="2" ParentId="23550" CreationDate="2020-09-13T01:03:39.327" Score="0" Body="&lt;p&gt;The thing you're explaining is not impossible for a RL model, but it's rare. That's a known thing that some RL algorithms work or don't work depending on a random seed. I implemented the same model once to play KungFuMaster-v0. It was during a university RL course and the code seemed fine (actually 2 people including teacher looked at it very carefully and they didn't find any bugs). I remember calling it 10 times a row and one time out of ten it showed that nasty behavior. There were teacher's notes in the task: if the reward suddenly drops to 0 and stay there for a long time, check your code, there's a high probability of a bug. So I'd say if your net works just fine 9 times out of 10, probably there are no bugs, otherwise if I were you, I'd carefully check the code.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastActivityDate="2020-09-13T01:03:39.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23557" PostTypeId="1" AcceptedAnswerId="23566" CreationDate="2020-09-13T01:19:04.667" Score="3" ViewCount="96" Body="&lt;p&gt;Suppose that we want to generate a sentence made of words according to language &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W_1 W_2 \ldots W_n&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; What is the perfect language model?&lt;/p&gt;&#xA;&lt;p&gt;I ask about &lt;em&gt;perfect&lt;/em&gt; because I want to know the concept fundamentally at its fullest extent. I am not interested in knowing heuristics or shortcuts that reduce the complexity of its implementation.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;1. My thoughts so far&lt;/h1&gt;&#xA;&lt;h2&gt;1.1. Sequential&lt;/h2&gt;&#xA;&lt;p&gt;One possible way to think about it is moving from left to right.  So, 1st, we try to find out value of &lt;span class=&quot;math-container&quot;&gt;$W_1$&lt;/span&gt;. To do so, we choose the specific word &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; from the space of words &lt;span class=&quot;math-container&quot;&gt;$\mathcal{W}$&lt;/span&gt; that's used by the language &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;.  Basically:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_1 = \underset{w \in \mathcal{W}}{\text{arg max }} \Pr(W_1 = w)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, we move forward to find the value of the next word &lt;span class=&quot;math-container&quot;&gt;$W_2$&lt;/span&gt; as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_2 = \underset{w \in \mathcal{W}}{\text{arg max }} \Pr(W_2 = w | W_1 = w_1)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Likewise for &lt;span class=&quot;math-container&quot;&gt;$W_3, \ldots, W_n$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_3 = \underset{w \in \mathcal{W}}{\text{arg max }} \Pr(W_3 = w | W_1 = w_1, W_2=w_2)&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vdots&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_n = \underset{w \in \mathcal{W}}{\text{arg max }} \Pr(W_n = w | W_1 = w_1, W_2=w_2, \ldots W_{n-1}=w_{n-1})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But is this really &lt;em&gt;perfect&lt;/em&gt;?  I personally doubt.  I think while language is read and written usually from a given direction (e.g. left to right), it is not always done so, and in many cases language is read/written possibly in a funny order as we always do.  E.g. even when I wrote this question, I jumped back and forth, then went to edit it (as I'm doing now).  So I clearly didn't write it from left to right!  Similarly, &lt;em&gt;you&lt;/em&gt;, the reader; you won't really read it in a single pass from left to right, will you?  You will probably read it in some funny order and go back and forth for awhile until you conclude an understanding.  So I personally really doubt that the sequential formalism is perfect.&lt;/p&gt;&#xA;&lt;h2&gt;1.2. Joint&lt;/h2&gt;&#xA;&lt;p&gt;Here we find all the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; words jointly. Of course ridiculously expensive computationally (if implemented), but our goal here is to only know what is the problem at its fullest.&lt;/p&gt;&#xA;&lt;p&gt;Basically, we get the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; words as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(w_1, w_2, \ldots, w_n) = \underset{(w_1,w_2,\ldots,w_n) \in \mathcal{W}^n}{\text{arg max }} \Pr(W_1 = w_1, W_2=w_2, \ldots W_n=w_n)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a &lt;em&gt;perfect&lt;/em&gt; representation of language model in my opinion, because its answer is gauranteed to be correct.  But there is this annoying aspect which is that its &lt;em&gt;words candidates space&lt;/em&gt; is needlessly large!&lt;/p&gt;&#xA;&lt;p&gt;E.g. this formalism is basically saying that the following is a candidate words sequence: &lt;span class=&quot;math-container&quot;&gt;$(., Hello, world, !)$&lt;/span&gt; even though we know that in (say) English a sentence cannot start by a dot &amp;quot;.&amp;quot;.&lt;/p&gt;&#xA;&lt;h2&gt;1.3. Joint but &lt;em&gt;slightly&lt;/em&gt; smarter&lt;/h2&gt;&#xA;&lt;p&gt;This is very similar to &lt;strong&gt;1.2 Joint&lt;/strong&gt;, except that it deletes the single bag of all words &lt;span class=&quot;math-container&quot;&gt;$\mathcal{W}$&lt;/span&gt;, and instead introduces several bags &lt;span class=&quot;math-container&quot;&gt;$\mathcal{W}_1, \mathcal{W}_2, \ldots, \mathcal{W}_n$&lt;/span&gt;, which work as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{W}_1$&lt;/span&gt; is a bag that contains words that can &lt;em&gt;only&lt;/em&gt; appear as &lt;em&gt;1&lt;sup&gt;st&lt;/sup&gt;&lt;/em&gt; words.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{W}_2$&lt;/span&gt; is a bag that contains words that can &lt;em&gt;only&lt;/em&gt; appear as &lt;em&gt;2&lt;sup&gt;nd&lt;/sup&gt;&lt;/em&gt; words.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vdots$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{W}_n$&lt;/span&gt; is a bag that contains words that can &lt;em&gt;only&lt;/em&gt; appear as &lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&lt;sup&gt;th&lt;/sup&gt;&lt;/em&gt; words.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This way, we will avoid the &lt;em&gt;stupid&lt;/em&gt; candidates that &lt;strong&gt;1.2. Joint&lt;/strong&gt; evaluated by following this:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(w_1, w_2, \ldots, w_n) = \underset{w_1 \in \mathcal{W}_1,w_2 \in \mathcal{W}_2,\ldots,w_n \in \mathcal{W}_n) \in \mathcal{W}^n}{\text{arg max }} \Pr(W_1 = w_1, W_2=w_2, \ldots W_n=w_n)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This will also guarantee being a &lt;em&gt;perfect&lt;/em&gt; representation of a language model, yet it its &lt;em&gt;candidates&lt;/em&gt; space is smaller than one in &lt;strong&gt;1.2. Joint&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;1.4. Joint but &lt;em&gt;fully&lt;/em&gt; smart&lt;/h2&gt;&#xA;&lt;p&gt;Here is where I'm stuck!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question rephrase (in case it helps):&lt;/strong&gt; Is there any formalism that gives the perfect correctness of &lt;strong&gt;1.2.&lt;/strong&gt; and &lt;strong&gt;1.3.&lt;/strong&gt;, except for also being &lt;em&gt;fully&lt;/em&gt; smart in that its candidates space is smallest?&lt;/p&gt;&#xA;" OwnerUserId="2361" LastEditorUserId="2361" LastEditDate="2020-09-14T14:21:03.897" LastActivityDate="2020-09-14T14:21:03.897" Title="Fundamentally, what is a perfect language model?" Tags="&lt;language-model&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="23558" PostTypeId="2" ParentId="23529" CreationDate="2020-09-13T10:59:18.390" Score="2" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1907.13508.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Dataset Optimisation: learning algorithm quality through evolution&lt;/a&gt; (2019), by Henry Wilde et al., proposes a method to generate datasets with a genetic algorithm. Their goal is to generate data for which a particular algorithm performs well, in terms of a certain metric, so that to get more insights about this algorithm and why it performs well. The individuals of the population are &lt;em&gt;datasets&lt;/em&gt; (so not features of the dataset!), which can be combined with a crossover operator or mutated. The details are explained in section 2 (page 4) and they also provide nice diagrams that summarise their descriptions.&lt;/p&gt;&#xA;&lt;p&gt;The authors evaluate their approach on k-means (section 3, page 12) and they use the k-means objective function as the fitness function of the genetic algorithm.&lt;/p&gt;&#xA;&lt;p&gt;They also developed a library &lt;a href=&quot;https://github.com/daffidwilde/edo&quot; rel=&quot;nofollow noreferrer&quot;&gt;edo&lt;/a&gt; that is freely available, so you can start to play with their approach.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-13T11:07:03.410" LastActivityDate="2020-09-13T11:07:03.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23559" PostTypeId="1" CreationDate="2020-09-13T11:56:10.630" Score="2" ViewCount="44" Body="&lt;p&gt;I'm about to write a non-player character (NPC). I wonder how much the AI should know about the game's world. So, my question isn't about the amount of training data the AI has to collect. I'm interested in how much the AI is &lt;strong&gt;allowed&lt;/strong&gt; to know about what's going on in the game's world.&#xA;For example, can (shall) it have knowledge about the build queue of the player?&lt;/p&gt;&#xA;&lt;p&gt;To provide more details: while a human plays a game against another human, not all information of what the opponent is doing is available (e.g. the queue of the units your opponent is building). This could give you an advantage (so that you can prepare for a rush, when he's building many cheap units). Theoretically, an NPC could access and make use of that knowledge and, in addition, spare resources for scouting/spying/exploring.&lt;/p&gt;&#xA;&lt;p&gt;But is this the way of constructing an NPC AI? Or should this data also be restricted? I have never done anything like this before.&lt;/p&gt;&#xA;&lt;p&gt;I don't know where to ask else wise or what more information I could provide. So, if something in my question is unclear or unfit, please let me know what exactly.&lt;/p&gt;&#xA;" OwnerUserId="40950" LastEditorUserId="2444" LastEditDate="2020-09-13T13:00:37.447" LastActivityDate="2020-11-17T06:03:35.387" Title="How much can/should the non-player character know about the game's world?" Tags="&lt;game-ai&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23560" PostTypeId="1" CreationDate="2020-09-13T12:45:40.990" Score="1" ViewCount="28" Body="&lt;p&gt;I am trying to create a language generation model to generate very short sentences/words, like a rapper name generator. The sentences in my dataset are anywhere between 1 word and 15 words (3-155 characters). So far, I have tried LSTM's with 1-3 layers and inputs as subwords and characters. The results so far are not that great, I am getting ~0.5 crossentropy loss and ~50% accuracy.&lt;/p&gt;&#xA;&lt;p&gt;My inputs are like a sliding window with prepadding, (eg. (for a batch) &lt;code&gt;Inputs = [[0,0,0,1], [0,0,1,2]...[n-4,..n-1]]&lt;/code&gt;, &lt;code&gt;outputs=[[0,0,1,2], ...[n-3,n-2,n-1,n]]&lt;/code&gt;) where 0 is padding, &lt;code&gt;1&lt;/code&gt; is the start token and &lt;code&gt;n&lt;/code&gt; is the end token. Outputs are 1 hot encoded.&lt;/p&gt;&#xA;&lt;p&gt;The model is an embedding layer, few lstm and dropout layers, followed by time distributed dense and then a dense layer.&lt;/p&gt;&#xA;&lt;p&gt;My doubt is, is accuracy a right metric, I am using it because at the end, I am making a classification for 4 output values. Another one is, will a transformer be suitable for this, since I want to generate small sentences, (which are nouns) and models like GPT/ Bert are more suitable for capturing dependency between long sentences.&lt;/p&gt;&#xA;" OwnerUserId="27875" LastEditorUserId="27875" LastEditDate="2020-09-13T15:27:19.653" LastActivityDate="2020-09-13T15:47:54.627" Title="Appropriate metric and approach for natural language generation for small sentences" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;&lt;attention&gt;&lt;language-model&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23561" PostTypeId="1" CreationDate="2020-09-13T12:57:23.703" Score="16" ViewCount="2825" Body="&lt;p&gt;As we all know, &lt;a href=&quot;https://en.wikipedia.org/wiki/%22Hello,_World!%22_program&quot; rel=&quot;noreferrer&quot;&gt;&amp;quot;Hello World&amp;quot;&lt;/a&gt; is usually the first program that any programmer learns/implements in any language/framework.&lt;/p&gt;&#xA;&lt;p&gt;As Aurélien Géron mentioned in his &lt;a href=&quot;https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/&quot; rel=&quot;noreferrer&quot;&gt;book&lt;/a&gt; that &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;noreferrer&quot;&gt;MNIST&lt;/a&gt; is often called the &lt;em&gt;Hello World of Machine Learning&lt;/em&gt;, &lt;strong&gt;is there any &amp;quot;Hello World&amp;quot; problem of Reinforcement Learning?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A few candidates that I could think of are &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;noreferrer&quot;&gt;multi armed bandits problem&lt;/a&gt; and &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0/&quot; rel=&quot;noreferrer&quot;&gt;Cart Pole Env&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40485" LastEditorUserId="2444" LastEditDate="2020-09-13T13:08:27.533" LastActivityDate="2020-09-15T06:32:51.710" Title="What is the &quot;Hello World&quot; problem of Reinforcement Learning?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23562" PostTypeId="2" ParentId="23527" CreationDate="2020-09-13T13:18:59.287" Score="1" Body="&lt;p&gt;The formal definition of zero-shot learning is that given labeled training instances &lt;span class=&quot;math-container&quot;&gt;$D_{tr}$&lt;/span&gt; belonging to the seen classes &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, the aim is to learn a classifier &lt;span class=&quot;math-container&quot;&gt;$f^u(·):X→U$&lt;/span&gt; that can classify testing instances &lt;span class=&quot;math-container&quot;&gt;$X_{te}$&lt;/span&gt; (i.e., to predict &lt;span class=&quot;math-container&quot;&gt;$Y_{te}$&lt;/span&gt;) belonging to the unseen classes &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The general idea of zero-shot learning is to transfer the knowledge contained in the training instances &lt;span class=&quot;math-container&quot;&gt;$D_{tr}$&lt;/span&gt; to the task of testing instance classification. So it is considered a transfer learning method, and, more specifically, heterogeneous transfer learning with different label spaces.&lt;/p&gt;&#xA;&lt;p&gt;However, since no labeled instances belonging to the unseen classes are available,&#xA;to solve the zero-shot learning problem, some auxiliary information is necessary. Such auxiliary information should contain information about all of the unseen classes. This auxiliary information usually contains some semantic information about the unseen classes, and its representation belongs to a space that is often referred as &lt;span class=&quot;math-container&quot;&gt;$semantic$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$space$&lt;/span&gt;. In the semantic space, each class has a corresponding vector representation, which is referred to as the class prototype or &lt;span class=&quot;math-container&quot;&gt;$prototype$&lt;/span&gt; for short.&lt;/p&gt;&#xA;&lt;p&gt;So, you should be aware of classes of your problem, in order to construct the semantic space and their class prototypes. I also suggest you read the survey paper &amp;quot;A survey of zero-shot learning: settings, methods and applications&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-09-13T13:42:58.713" LastActivityDate="2020-09-13T13:42:58.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23563" PostTypeId="2" ParentId="23561" CreationDate="2020-09-13T13:28:29.507" Score="9" Body="&lt;p&gt;MNIST (along with CIFAR) may be the &amp;quot;Hello World&amp;quot; of &lt;strong&gt;supervised&lt;/strong&gt; learning for image classification, but it is definitely not the &amp;quot;Hello World&amp;quot; of all machine learning techniques, given that RL is also part of ML and MNIST is definitely not the &amp;quot;Hello World&amp;quot; of RL.&lt;/p&gt;&#xA;&lt;p&gt;I don't think there is a single &amp;quot;Hello World&amp;quot; problem for RL. However, if you are looking for simple problems (or environments) that are usually used as baselines to assess the quality of RL agents, then I would say that the simple &lt;strong&gt;grid worlds&lt;/strong&gt; where you need to move from one place to the other, the &lt;strong&gt;CartPole&lt;/strong&gt;, &lt;strong&gt;MountainCar&lt;/strong&gt;, &lt;strong&gt;Pendulum&lt;/strong&gt; or other environments listed &lt;a href=&quot;https://github.com/openai/gym/wiki/Leaderboard&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; are often used.&lt;/p&gt;&#xA;&lt;p&gt;The environment that you choose to train and test your RL agent depends on your goals. For example, if you designed an algorithm that is supposed to deal with continuous action spaces, then an environment where you can take only a discrete number of actions may not be a good option.&lt;/p&gt;&#xA;&lt;p&gt;The mentioned environments are very simple (i.e. toy problems). In my opinion, we need more serious environments that can show the applicability of RL to other areas other than (relatively simple) games.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-13T20:26:37.147" LastActivityDate="2020-09-13T20:26:37.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23565" PostTypeId="2" ParentId="23560" CreationDate="2020-09-13T15:07:07.023" Score="1" Body="&lt;p&gt;I wouldn't say accuracy in the next word prediction is a good global metric. It would depend on the length of sentences. It's always difficult to predict the first word, cos you don't have any context. And having at least one word in the context it's easier. As long as your error rate is averaged among all predicted words, the accuracy could be higher if the lengths of sentences are higher. So the value of 0.5 doesn't tell much. The fact that you improved accuracy by 0.1 for instance on the same dataset means one method is better than another. Also, people measure perplexity and it's more sensitive to small changes of loss than accuracy. That's why I suggest you to measure perplexity. This metric also depends on lengths of sentences, that's why you should only compare 2 models trained on the same dataset.&#xA;What about GPT and Transformer, they were invented to solve one of the issues of LSTMs (they're not capable to memorize a very large context), that's why transformers would be better for long sentences, but they would also be better for short I think due to their attention mechanist, which has many useful properties.&lt;/p&gt;&#xA;" OwnerUserId="40906" LastEditorUserId="40906" LastEditDate="2020-09-13T15:47:54.627" LastActivityDate="2020-09-13T15:47:54.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23566" PostTypeId="2" ParentId="23557" CreationDate="2020-09-13T18:37:13.790" Score="1" Body="&lt;p&gt;One of your hypothesis is very close to the truth, it's 1.2&lt;/p&gt;&#xA;&lt;p&gt;So, a language model measures the probability of a given sentence in a language &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;. The sentences can have any length and the sum of probabilities of all the sentences in the language &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is 1. It's very difficult to compute, thus people use some simplifications, like say if the words are located far enough from each other, then the occurrence of a current word doesn't depend on a word which was occurred far away in the past.&#xA;Each sentence is a sequence &lt;span class=&quot;math-container&quot;&gt;$w_1, \dots, w_n$&lt;/span&gt; and a language model computes the probability of the sequence &lt;span class=&quot;math-container&quot;&gt;$p([w_1, \dots w_n])$&lt;/span&gt; (it's not joint distrribution yet). It can be decomposed into a joint distribution with some special tokens added &lt;span class=&quot;math-container&quot;&gt;$p(BOS, w_1, \dots w_n, EOS])$&lt;/span&gt;. BOS is begin of the sentence and EOS is end of sentence. Then this joint distribution can be decomposed using the chain rule &lt;span class=&quot;math-container&quot;&gt;$p(BOS, w_1, \dots w_n, EOS]) = p(BOS) p(w_1 | BOS) \Big[ \prod\limits_{i=1}^n p(w_i | BOS, w_1, \dots, w_{i-1}) \Big] p(EOS | BOS, w_1, \dots, w_n)$&lt;/span&gt;. There are 2 types of probabilities that are usually modelled differently: a prior probability &lt;span class=&quot;math-container&quot;&gt;$p(BOS)$&lt;/span&gt; which is always equal to 1, because you always have BOS as the first token in the augmented sequence. Then conditional probabilities can be computed as follows &lt;span class=&quot;math-container&quot;&gt;$p(w_i | BOS, w_1, \dots, w_{i-1}) = \frac{c(BOS, w_1, \dots, w_{i-1}, w_i)}{\sum_{w_i \in W} c(BOS, w_1, \dots, w_{i-1}, w_i)}$&lt;/span&gt;. Where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is a counter function that measures how many times a given sequence occured in the dataset you specified to train your model. You can notice it's a maximum likelihood estimate of the unknown conditional probabilities. Obviously if you're using a certain dataset you compute a model of that dataset, not of a language, but that's the to approximate true probabilities of sentences in a language.&#xA;The EOS token is needed to make difference between a probability of a non-finished yet sequence and that which has finished, because if you take those counters from above and forget about adding the EOS into your dataset in the end of all sentences, you'll get probabilities that don't sum into 1 (which is bad).&lt;/p&gt;&#xA;" OwnerUserId="40906" LastActivityDate="2020-09-13T18:37:13.790" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="23567" PostTypeId="1" AcceptedAnswerId="24529" CreationDate="2020-09-13T20:35:39.700" Score="3" ViewCount="183" Body="&lt;p&gt;In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?&lt;/p&gt;&#xA;&lt;p&gt;In my case, I want to use a convolutional neural network as a classifier of heartbeat types. The ECG signal is split, and a color image is created using feature extraction. These photos (the inputs) are fed into a deep CNN, but they must be labeled by someone first.&lt;/p&gt;&#xA;&lt;p&gt;Are there ways to implement continuous learning in a deep neural network for image recognition? Does such an implementation make sense if the labels have to be specially prepared in advance?&lt;/p&gt;&#xA;" OwnerUserId="37928" LastEditorUserId="2444" LastEditDate="2020-11-10T10:39:25.660" LastActivityDate="2020-11-10T12:48:38.913" Title="Is continuous learning possible with a deep convolutional neural network, without changing its topology?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;incremental-learning&gt;&lt;online-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23568" PostTypeId="1" CreationDate="2020-09-13T20:58:37.250" Score="1" ViewCount="43" Body="&lt;p&gt;i was watching this video: &lt;a href=&quot;https://www.youtube.com/watch?v=4cCS8rrYT14&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=4cCS8rrYT14&lt;/a&gt;&#xA;and I tried to solve this question:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Given the variables A, B, C and D, with domain {1, 2, 3, 4} in each of them and restrictions A&amp;gt; B, B = C and D ≠ A, use the AC algorithm.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;But the teacher told me that my answer was wrong! He gave me a tip, the tip is: Domain D will not be changed! Below, I will leave my answer step by step, if someone can help me find the error, I appreciate it!&lt;/p&gt;&#xA;&lt;p&gt;My answer is:&lt;/p&gt;&#xA;&lt;p&gt;To solve this exercise, it is first necessary to organize the data in order to separate what is the domain, agenda and arc.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mHp2I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mHp2I.png&quot; alt=&quot;Step 1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Soon after, we will analyze the first item on the agenda “A&amp;gt; B” with domain A, in order to eliminate unnecessary elements from the domain.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/SIk5Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SIk5Q.png&quot; alt=&quot;step 2.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain B with the agenda item “B &amp;lt;A”&#xA;&lt;a href=&quot;https://i.stack.imgur.com/R5Ucq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R5Ucq.png&quot; alt=&quot;3 step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain B with the agenda item “B = C” and add the constraint “A&amp;gt; B”&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ja65I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ja65I.png&quot; alt=&quot;4 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain D with the agenda item “D ≠ A” and add the constraint “B &amp;lt;A”&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tfoXc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tfoXc.png&quot; alt=&quot;5 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain A with the agenda item “A ≠ D”&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mIpBs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mIpBs.png&quot; alt=&quot;6 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain A with the agenda item “A&amp;gt; B”&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XKi22.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XKi22.png&quot; alt=&quot;7 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain B with the agenda item “B = C”&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DmTPE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DmTPE.png&quot; alt=&quot;8 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Analyze domain B with the agenda item “B &amp;lt;A”&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gNuDq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gNuDq.png&quot; alt=&quot;9 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Result&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DzNCk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DzNCk.png&quot; alt=&quot;10 Step&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40555" LastEditorUserId="40555" LastEditDate="2020-09-13T21:51:32.130" LastActivityDate="2020-09-13T21:51:32.130" Title="Somone can help me to solve this question about the AC-3 algorithm" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;intelligent-agent&gt;&lt;constraint-satisfaction-problems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23570" PostTypeId="1" CreationDate="2020-09-14T02:24:30.847" Score="2" ViewCount="87" Body="&lt;p&gt;I am confused about the training part in &lt;a href=&quot;https://openaccess.thecvf.com/content_cvpr_2018/papers/Xu_AttnGAN_Fine-Grained_Text_CVPR_2018_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AttnGan&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you observe page 3. There are two types of losses for generator network: one involving the &lt;em&gt;Deep Attentional&#xA;Multimodal Similarity Model (DAMSM)&lt;/em&gt; loss &lt;span class=&quot;math-container&quot;&gt;$(L_{DAMSM})$&lt;/span&gt; and the others for individual generator &lt;span class=&quot;math-container&quot;&gt;$(L_{G_i})$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i= 1, 2, 3$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My doubt is: if each generator has its own loss function that is useful in training, what is the purpose in using &lt;span class=&quot;math-container&quot;&gt;$L_G$&lt;/span&gt;, i.e., with DAMSM loss function? Is my assumption wrong?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="18758" LastEditDate="2021-04-05T06:37:03.953" LastActivityDate="2021-04-05T06:37:03.953" Title="What is the purpose of the DAMSM loss for the generators in AttnGAN?" Tags="&lt;loss-functions&gt;&lt;papers&gt;&lt;generative-adversarial-networks&gt;&lt;attn-gan&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23571" PostTypeId="1" CreationDate="2020-09-14T03:29:47.943" Score="1" ViewCount="50" Body="&lt;p&gt;I am constructing a convolutional variational autoencoder for images, starting out with mnist digits. Typically I would specify convolutional layers in the following way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_img = layers.Input(shape=(28,28,1))&#xA;conv1 = keras.layers.Conv2D(32, (3,3), strides=2, padding='same', activation='relu')(input_img)&#xA;conv2 = keras.layers.Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(conv1) &#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I would also like to construct a convolutional filter/kernel that is fixed BUT dependent on some content related to the input, which we can call an auxiliary label. This could be a class label or some other piece of relevant information corresponding to the input. For example, for MNIST I can use the class label as auxiliary information and map the digit to a (3,3) kernel and essentially generate a distinct kernel for each digit. This specific filter/kernel is not learned through the network so it is fixed, but it is class dependent. This filter will then be concatenated with the traditional convolutional filters shown above.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_img = layers.Input(shape=(28,28,1))&#xA;conv1 = keras.layers.Conv2D(32, (3,3), strides=2, padding='same', activation='relu')(input_img)&#xA;&#xA;# TODO: add a filter/kernel that is fixed (not learned by model) but is class label specific&#xA;# Not sure how to implement this?&#xA;# auxiliary_conv = keras.layers.Conv2D(1, (3,3), strides=2, padding='same', activation='relu')(input_img)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know there are kernel initializers to specify initial weights &lt;a href=&quot;https://keras.io/api/layers/initializers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/api/layers/initializers/&lt;/a&gt;, but I'm not sure if this is relevant and if so, how to make this work with a class specific initialization.&lt;/p&gt;&#xA;&lt;p&gt;In summary, I want a portion of the model's weights to be input content dependent so that some of the trained model's weights vary based on the auxiliary information such as class label, instead of being completely fixed regardless of the input. Is this even possible to achieve in Keras/Tensorflow? I would appreciate any suggestions or examples to get started with implementation.&lt;/p&gt;&#xA;" OwnerUserId="40781" LastActivityDate="2021-02-12T06:07:50.640" Title="How to construct input dependent convolutional filter?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;variational-autoencoder&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23572" PostTypeId="1" CreationDate="2020-09-14T03:52:45.823" Score="1" ViewCount="42" Body="&lt;p&gt;Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.&lt;/p&gt;&#xA;&lt;p&gt;I'm new in this world of the AI but my problem is that many child groups contain the same elements or even different combinations to result the parent data but in this case more to avoid duplication is get those duplications and the some way add up the data.&lt;/p&gt;&#xA;&lt;p&gt;This is an example of what secondary data can look like and what I want to get from grouping it.&lt;/p&gt;&#xA;&lt;p&gt;Parent data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  ID        FIELD1       FIELD2 FIELD3  FIELD4      FIELD5&#xA;  90148001  BLABLA       40     0       35896.89479 35896.89479&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Child data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  ID        FIELD1       FIELD2 FIELD3  FIELD4      FIELD5&#xA;* 90148001  BLABLA       1      1770    1769.572665 1769.572665&#xA;* 90148001  DESCRIPTION2 1      13146   13146.45284 13146.45284&#xA;* 90148001  BLABLA       1      2176    2176.435074 2176.435074&#xA;* 90148001  BLABLA       1      2306    2305.716285 2305.716285&#xA;* 90148001  BLABLA       1      2531    2531.271196 2531.271196&#xA;* 90148001  BLABLA       1      1147    1146.803622 1146.803622&#xA;* 90148001  BLABLA       1      1991    1990.613246 1990.613246&#xA;* 90148001  BLABLA       1      3641    3641.394446 3641.394446&#xA;* 90148001  BLABLA       1      2471    2470.8253   2470.8253&#xA;* 90148001  BLABLA       1      2247    2246.984815 2246.984815&#xA;* 90148001  BLABLA       1      2471    2470.8253   2470.8253&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could a neural network be able to process, aggregate, and group those quantities?&lt;/p&gt;&#xA;" OwnerUserId="25333" LastEditorUserId="25333" LastEditDate="2020-09-14T15:34:06.143" LastActivityDate="2020-09-14T15:34:06.143" Title="Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?" Tags="&lt;neural-networks&gt;&lt;datasets&gt;&lt;math&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23573" PostTypeId="1" AcceptedAnswerId="23578" CreationDate="2020-09-14T06:29:05.967" Score="5" ViewCount="349" Body="&lt;p&gt;As a followup to &lt;a href=&quot;https://ai.stackexchange.com/questions/23561/what-is-the-hello-world-problem-of-reinforcement-learning&quot;&gt;this question&lt;/a&gt;, I'm interested in what the typical &amp;quot;Hello World&amp;quot; problem (first easy example problem) is for unsupervised learning.&lt;/p&gt;&#xA;&lt;p&gt;A quick Google search didn't find any obvious answers for me.&lt;/p&gt;&#xA;" OwnerUserId="40968" LastActivityDate="2020-09-14T09:48:56.537" Title="What is the “Hello World” problem of Unsupervised Learning?" Tags="&lt;unsupervised-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23574" PostTypeId="1" AcceptedAnswerId="23583" CreationDate="2020-09-14T07:18:16.130" Score="1" ViewCount="71" Body="&lt;p&gt;Taken from section 2.1 in the article:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the environment is fully observable. An environment is described by a set of states &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, a set of actions &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, a distribution of initial states &lt;span class=&quot;math-container&quot;&gt;$p(s_0)$&lt;/span&gt;, a reward function &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt;&lt;/strong&gt;, transition probabilities &lt;span class=&quot;math-container&quot;&gt;$p(s_{t+1} \mid s_t, a_t)$&lt;/span&gt;, and a discount factor &lt;span class=&quot;math-container&quot;&gt;$\gamma \in [0, 1]$&lt;/span&gt;.*&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How should one interpret the maths behind it?&lt;/p&gt;&#xA;" OwnerUserId="40971" LastEditorUserId="2444" LastEditDate="2020-12-02T00:26:53.097" LastActivityDate="2020-12-02T00:26:53.097" Title="What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;papers&gt;&lt;hindsight-experience-replay&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23575" PostTypeId="1" CreationDate="2020-09-14T07:20:44.050" Score="2" ViewCount="57" Body="&lt;p&gt;Model based RL attempts to learn a function &lt;span class=&quot;math-container&quot;&gt;$f(s_{t+1}|s_t, a_t)$&lt;/span&gt; representing the environment transitions, otherwise known as a model of the system. I see linear functions are still being used in model-based RL &lt;a href=&quot;https://arxiv.org/abs/2002.09093&quot; rel=&quot;nofollow noreferrer&quot;&gt;such as in robotic manipulation&lt;/a&gt; to learn system dynamics, and can work effectively well. (Here, I mean in learning the model, not as an optimization method for the controller selecting the best actions).&lt;/p&gt;&#xA;&lt;p&gt;In model-based RL, are there situations where a learning a linear model such as using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Lyapunov_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lyapunov function&lt;/a&gt; would be better suited than using a neural network, or are the examples of problems framed to use linear models when addressing them using model-based RL?&lt;/p&gt;&#xA;" OwnerUserId="40671" LastActivityDate="2021-02-11T14:04:45.397" Title="Are linear approximators better suited to some tasks compared to complex neural net functions?" Tags="&lt;reinforcement-learning&gt;&lt;control-problem&gt;&lt;model-based-methods&gt;&lt;linear-programming&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23577" PostTypeId="2" ParentId="23561" CreationDate="2020-09-14T08:56:56.680" Score="2" Body="&lt;p&gt;While there's no simple Hello World problem of RL, if your aim is to understand the basic working of Reinforcement Learning and see it at play while using as few moving parts as possible, a simple suggestion would be using &lt;strong&gt;Tabular&lt;/strong&gt; Q-Learning in a toy environment (like your suggested Cart-Pole Env).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here's the reasoning behind this suggestion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say we interpret MNIST's label as a &lt;em&gt;Hello World&lt;/em&gt; of Supervised Learning to mean something showing the basic steps of doing Supervised Learning: Create a model, load the data, then train.&lt;/p&gt;&#xA;&lt;p&gt;If that interpretation is not far off, we can say a simple introductory problem to Reinforcement Learning (RL) should focus on easily demonstrating a working &lt;a href=&quot;https://www.coursera.org/lecture/fundamentals-of-reinforcement-learning/markov-decision-processes-8T0GQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process (MDP)&lt;/a&gt; which is the backbone of the RL decision making process. As such, this minimal working would involve: Observing the world, selecting an action, as shown in this loop:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rrgVV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rrgVV.jpg&quot; alt=&quot;Simple MDP&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This picture is missing two important steps in an RL algorithm learning loop:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Estimating the rewards or Fitting the model&lt;/li&gt;&#xA;&lt;li&gt;Improving how you select actions. (Updating your policy)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.&lt;/p&gt;&#xA;&lt;p&gt;So a suggested first problem would be one that helps you &lt;strong&gt;see the MDP in action&lt;/strong&gt;, while keeping steps 1 and 2 simple enough so that you &lt;strong&gt;understand how the agent learns&lt;/strong&gt;. Tabular Q-Learning seems clear enough for this because it uses a &lt;a href=&quot;https://observablehq.com/@frynd/q-table-reinforcement-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-table&lt;/a&gt; represented as a 2D array to do the two steps. This should not suggest Q-learning is a &amp;quot;Hello World&amp;quot; RL algorithm because of the said relative ease in understanding it :)&lt;/p&gt;&#xA;&lt;p&gt;You will be unable to use it's Tabular version anywhere else than in a toy environment though, typically Frozen-Lake and CartPole. An improvement would be &lt;a href=&quot;https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;using a neural network&lt;/a&gt; instead of a table to estimate Q values.&lt;/p&gt;&#xA;&lt;p&gt;Here are a few useful resources:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Learning with Tables&lt;/a&gt; (Guide)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gist.github.com/awjuliani/9024166ca08c489a60994e529484f7fe#file-q-table-learning-clean-ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-learning jupyter notebook&lt;/a&gt; (Code ~25 lines)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mugoh/reinforcementLearning/tree/master/algorithms/qlearning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Learning with Frozen-Lake and Taxi&lt;/a&gt; (Code)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.freecodecamp.org/news/diving-deeper-into-reinforcement-learning-with-q-learning-c18d0db58efe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning with Q-Learning&lt;/a&gt; (Guide)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;A multi-armed bandit would also be great in introducing you to &lt;a href=&quot;https://or.stackexchange.com/questions/4598/difference-between-exploration-and-exploitation-in-simulated-annealing-algorithm&quot;&gt;exploration-exploitation trade-off&lt;/a&gt; (which Q-learning does too), though it wouldn't be considered a full RL algorithm since it has no context.&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="40671" LastEditDate="2020-09-15T06:32:51.710" LastActivityDate="2020-09-15T06:32:51.710" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23578" PostTypeId="2" ParentId="23573" CreationDate="2020-09-14T09:48:56.537" Score="5" Body="&lt;p&gt;I disagree with the context that MNIST is the &amp;quot;hello world&amp;quot; of supervised learning. It is definitely, though, the &amp;quot;hello world&amp;quot; of &lt;strong&gt;image classification&lt;/strong&gt;, which is a very specific sub-field of supervised learning.&lt;/p&gt;&#xA;&lt;p&gt;I'd consider the &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/iris&quot; rel=&quot;noreferrer&quot;&gt;Iris&lt;/a&gt; dataset a better candidate for the &amp;quot;hello world&amp;quot; of supervised learning, with other close candidates such as the &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/Wine&quot; rel=&quot;noreferrer&quot;&gt;Wine&lt;/a&gt;, &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29&quot; rel=&quot;noreferrer&quot;&gt;Wisconsin breast cancer&lt;/a&gt; or &lt;a href=&quot;https://www.kaggle.com/uciml/pima-indians-diabetes-database&quot; rel=&quot;noreferrer&quot;&gt;Pima indians&lt;/a&gt; datasets. However, as an even simpler and more alternative choice, a lot of people prefer generating &lt;a href=&quot;https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html&quot; rel=&quot;noreferrer&quot;&gt;their own 2-dimensional datasets&lt;/a&gt; so that can more intuitively understand what the different algorithms are doing. An example of this is &lt;a href=&quot;https://playground.tensorflow.org&quot; rel=&quot;noreferrer&quot;&gt;TensorFlow playground&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Equivalently, in unsupervised learning there are a lot of different tasks. I personally think that &lt;strong&gt;clustering&lt;/strong&gt; is probably the task that is easier for people to understand and as such the most common intro to unsupervised learning. Here there are, as well, two options:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using an already established dataset, e.g. &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/iris&quot; rel=&quot;noreferrer&quot;&gt;Iris&lt;/a&gt; (without the labels).&lt;/li&gt;&#xA;&lt;li&gt;Generating your own synthetic 2-dimensional data, to better understand how the algorithms work. An example is &lt;a href=&quot;https://github.com/djib2011/python_ml_tutorial/blob/master/notebooks/26_clustering.ipynb&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-09-14T09:48:56.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23579" PostTypeId="1" CreationDate="2020-09-14T10:12:38.863" Score="0" ViewCount="34" Body="&lt;p&gt;I'm writing a General Game Playing (GGP) AI in python. I'd like to test it on some GGP games. So are there any python implementations of GGP games?&lt;/p&gt;&#xA;&lt;p&gt;I found in &lt;a href=&quot;http://games.ggp.org/base&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://games.ggp.org/base&lt;/a&gt; games written in Game Description Language (GDL). How to use them in python if it is possible to do so?&lt;/p&gt;&#xA;" OwnerUserId="40975" LastEditorUserId="40975" LastEditDate="2020-09-14T10:26:56.830" LastActivityDate="2020-09-14T10:26:56.830" Title="Are there any python implementations of GGP games or how to use game logic written in GDL in python?" Tags="&lt;python&gt;&lt;game-ai&gt;&lt;combinatorial-games&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23580" PostTypeId="1" CreationDate="2020-09-14T10:35:56.700" Score="0" ViewCount="24" Body="&lt;p&gt;The following is definition of reinforcement comparison, which updates an average reward and a preference for each action&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/first/ebook/node22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://incompleteideas.net/book/first/ebook/node22.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to know if this method finds optimal policy? What are necessary conditions for it?&lt;/p&gt;&#xA;" OwnerUserId="37733" LastActivityDate="2020-09-14T10:35:56.700" Title="Reinforcement comparison optimality" Tags="&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23581" PostTypeId="2" ParentId="22581" CreationDate="2020-09-14T12:23:26.180" Score="2" Body="&lt;p&gt;The main point in GPT-3 and already in 2 was the observation that performance was steadily increasing with increasing model size (As seen in Figure 1.2 in your linked paper). So it seems that while all progress made in NLP was definitely useful, it also is important to just scale up the model size.&lt;/p&gt;&#xA;&lt;p&gt;This may not seem like surprising point, but it actually kind of is. Normally, performance would saturate or at least the gain would slope off, but this is not the case! So the main innovation may not be that big and is kind off brute-force but the point still stands: Bigger models are better.&lt;/p&gt;&#xA;&lt;p&gt;Another point to mention is the way they did the training. Such a large model needs some tricks to be actually trained (and fast at that). You also want to make use of multiple GPUs for parallel training. This means they also had to develop new structures for training.&lt;/p&gt;&#xA;&lt;p&gt;Why exactly it is predicted as a huge innovation may only be contained to some twitter demonstration, there are no real sources on this as far as I know. Especially because the model is not openly available.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-09-14T12:23:26.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23583" PostTypeId="2" ParentId="23574" CreationDate="2020-09-14T13:14:46.827" Score="3" Body="&lt;p&gt;This answer assumes that you only have a problem with this notation from the article:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a standard notation, used in many disciplines, for defining a function and its input and output &lt;em&gt;domains&lt;/em&gt;. It is a bit like the method signature for the function - it does not fully define it, but does enough to show how it can interact with other expressions.&lt;/p&gt;&#xA;&lt;p&gt;All functions can be thought of as maps between the input domain and output domain. You provide an input value, and it returns an output value. The values can be arbitrary mathematical objects. To show what kind of objects the inputs and outputs are allowed to be, the &lt;a href=&quot;https://www.mathsisfun.com/sets/symbols.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;notation for sets&lt;/a&gt; is used.&lt;/p&gt;&#xA;&lt;p&gt;Importantly the symbol &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; at the end does not refer to the set of possible rewards in the environment (although it is a reward function, and that will be its output), but the set of all real numbers, because a reward is always a real number*.&lt;/p&gt;&#xA;&lt;p&gt;As a concrete example, if you had the function &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2 - 2x + 7$&lt;/span&gt; defined for a real number &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, then its equivalent notation might be &lt;span class=&quot;math-container&quot;&gt;$f : \mathbb{R} \rightarrow \mathbb{R}$&lt;/span&gt;. If you allowed &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to be complex then it would be &lt;span class=&quot;math-container&quot;&gt;$f : \mathbb{C} \rightarrow \mathbb{C}$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$\mathbb{C}$&lt;/span&gt; is the standard symbol for the set of all complex numbers.&lt;/p&gt;&#xA;&lt;p&gt;So now we can break down the notation &lt;span class=&quot;math-container&quot;&gt;$r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The function is called &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$:$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It has an input domain of . . .&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{S} \times \mathcal{A}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The cartesian product of the set of all possible states &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; and the set of possible actions &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That is much the same as saying the function has a signature &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a \in \mathcal{A}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It has an output domain of . . .&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;any single real number.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;* This choice (of declaring the more general &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; instead of specific &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt;) is made partly because operators like &lt;span class=&quot;math-container&quot;&gt;$+$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\times$&lt;/span&gt; are well defined for real numbers. This is a useful thing to assert about the behaviour of the reward function output when defining how value functions work for instance. Of course you could be more specific, defining &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; as some subset of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;, that would be correct and more precise definition, but it is not needed for general theory in reinforcement learning. The less precise definition is fine for nearly all purposes.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-14T16:02:58.903" LastActivityDate="2020-09-14T16:02:58.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23584" PostTypeId="2" ParentId="23575" CreationDate="2020-09-14T13:21:06.587" Score="0" Body="&lt;p&gt;This is just a case of supervised learning. You are trying to predict &lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt;, so the answer to your question depends on how complex your state dynamics are.&lt;/p&gt;&#xA;&lt;p&gt;For example, if the state space is really complex, e.g. if your state space is an image and you want to predict the next image given the current image and an action, then linear methods are unlikely to work well.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-09-14T13:21:06.587" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23585" PostTypeId="1" CreationDate="2020-09-14T13:25:07.877" Score="2" ViewCount="60" Body="&lt;p&gt;I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse 16, the marker will say 16-18. I wonder if, given I have a lot of other versions that separate them discretely, I can train an NLP model (I have about 30 versions that could act as a training set which would constitute  to separate those combined verses into discrete verses. I'm fairly new at deep learning, having done a few toy projects. I wonder how to think about this problem? What kind of problem is it? I think it might be similar to auto-punctuation problems and it seems the options there are seq2seq and classifier. This makes more sense to me as a classification problem, but maybe my inexperience is what drives me that direction. Can people suggest ways to think about this problem and resources I might use?&lt;/p&gt;&#xA;&lt;p&gt;In answer to questions in the comment, I am dealing only with text, not images. An example might be like this:&lt;/p&gt;&#xA;&lt;p&gt;Genesis 2, New Revised Standard Version:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;5 when no plant of the field was yet in the earth and no herb of the field had yet sprung up—for the Lord God had not caused it to rain upon the earth, and there was no one to till the ground; 6 but a stream would rise from the earth, and water the whole face of the ground— 7 then the Lord God formed man from the dust of the ground, and breathed into his nostrils the breath of life; and the man became a living being.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Genesis 2, The message version:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;5-7 At the time God made Earth and Heaven, before any grasses or shrubs had sprouted from the ground—God hadn’t yet sent rain on Earth, nor was there anyone around to work the ground (the whole Earth was watered by underground springs)—God formed Man out of dirt from the ground and blew into his nostrils the breath of life. The Man came alive—a living soul!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The goal then would be to divide the message version into discrete verses in the way that the NRSV is. Certainly, a part of the guide would be that a verse always ends in some kind of punctuation, though while necessary it is not sufficient to assign a distinct verse.&lt;/p&gt;&#xA;" OwnerUserId="40982" LastEditorUserId="40982" LastEditDate="2020-09-15T12:36:17.687" LastActivityDate="2020-09-15T12:36:17.687" Title="NLP Bible verse division problem: Whats the best model/method?" Tags="&lt;natural-language-processing&gt;&lt;sequence-modelling&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23588" PostTypeId="5" CreationDate="2020-09-14T20:31:54.107" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-14T20:31:54.107" LastActivityDate="2020-09-14T20:31:54.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23589" PostTypeId="4" CreationDate="2020-09-14T20:31:54.107" Score="0" Body="For questions about the Attentional Generative Adversarial Network (AttnGAN) proposed in the paper &quot;AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks&quot; (2018) by Tao Xu et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-14T20:31:54.107" LastActivityDate="2020-09-14T20:31:54.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23590" PostTypeId="1" AcceptedAnswerId="23591" CreationDate="2020-09-15T01:22:19.240" Score="1" ViewCount="69" Body="&lt;p&gt;I have roughly 30,000 images of two categories, which are 'crops' and 'weeds.' An example of what I have can be found below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZIwtM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZIwtM.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The goal will use my training images to detect weeds among crops, given an orthomosaic GIS image of a given field. I guess you could say that I'm trying to detect certain objects in the field.&lt;/p&gt;&#xA;&lt;p&gt;As I'm new to deep learning, how would one go about generating training labels for this task? Can I just label the entire photo as a 'weed' using some type of text file, or do I actually have to draw bounding boxes (around weeds) on each image that will be used for training? If so, is there an easier way than going through all 30,000 of my images?&lt;/p&gt;&#xA;&lt;p&gt;I'm very new to this, so any specific details would really help a lot!&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-09-24T06:08:39.530" Title="How do I label images for deep learning classification?" Tags="&lt;deep-learning&gt;&lt;object-detection&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23591" PostTypeId="2" ParentId="23590" CreationDate="2020-09-15T01:41:13.167" Score="2" Body="&lt;p&gt;If each photo is intended to show a photo of weed or crops you should give one label. If your task is different where you also try to localize weed or crops in the image, then you need to label accordingly. My understanding is you are trying to do the first case, therefore, there should be one label for each image.&lt;/p&gt;&#xA;" OwnerUserId="40957" LastActivityDate="2020-09-15T01:41:13.167" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23592" PostTypeId="2" ParentId="23590" CreationDate="2020-09-15T01:45:37.557" Score="2" Body="&lt;p&gt;This is really a semantic segmentation problem if OP wants to pinpoint the weeds.&lt;/p&gt;&#xA;&lt;p&gt;If OP wants to have such a segmentation he will need to hand segment every.single picture.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-09-15T01:45:37.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23593" PostTypeId="2" ParentId="23523" CreationDate="2020-09-15T01:53:23.607" Score="0" Body="&lt;p&gt;Maybe you can use a recurrent neural network on saved data to train a predictive model based on past data.&lt;/p&gt;&#xA;" OwnerUserId="40957" LastActivityDate="2020-09-15T01:53:23.607" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23594" PostTypeId="2" ParentId="23523" CreationDate="2020-09-15T02:36:45.893" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;H2O's AutoML is the thing that you are looking for&lt;/a&gt; believe me it will make your life super easy.&lt;/p&gt;&#xA;&lt;p&gt;So how it generally works is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You have data and want to make sense out of it by making a prediction/classification and whole other array of things, in your case prediction.&lt;/li&gt;&#xA;&lt;li&gt;Suppose there are 50 prediction algorithms out there, it's highly unlikely that an individual knows all of them. That's where AutoML comes into the picture.&lt;/li&gt;&#xA;&lt;li&gt;You give AutoML some part of your processed data and tell it to find a perfect algorithm that he thinks you should use on this data for this type of prediction. See the usage of the AutoML's API in the doc and videos on youtube.&lt;/li&gt;&#xA;&lt;li&gt;It then gives you a list of best algorithms that AutoML thinks is best based on the loss function that you specify. There are many other parameters that you can specify in the AutoML's API.&lt;/li&gt;&#xA;&lt;li&gt;Pick top 1-2 and tune the hyperparameters of the algorithm.&lt;/li&gt;&#xA;&lt;li&gt;That's it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Even then if you are not happy with the performance of your system try out &lt;a href=&quot;https://en.wikipedia.org/wiki/Ensemble_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensemble learning&lt;/a&gt; before you jump for the power of neural nets which comes with lots of complexities and performance issues.&lt;/p&gt;&#xA;" OwnerUserId="40485" LastEditorUserId="40485" LastEditDate="2020-09-15T02:44:50.840" LastActivityDate="2020-09-15T02:44:50.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23595" PostTypeId="2" ParentId="23571" CreationDate="2020-09-15T04:18:02.783" Score="0" Body="&lt;p&gt;Not a tensorflow expert but I may be able to offer some conceptual advice. Since you do not care to learn the filter, but instead want to fix a discrete set of possible values for a discrete set of cases, you can use tensor operations (i.e convolutions) rather than neural network layer operations. Essentially, in framework-agnostic pseudocode this would look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# layers with learned parameters&#xA;output1 = layers1(input)&#xA;&#xA;# apply unlearned but changeable layer convolutions&#xA;kernel_val = kernel_val_selection_function(output1)&#xA;output2 = convolve_2D(output1,kernel_val)&#xA;&#xA;# more layers with learned parameters&#xA;output3 = layers3(output2)&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The function graph will treat &lt;code&gt;kernel_val&lt;/code&gt; as a constant for purposes of backpropagation, so as long as your convolution operations are done within the framework used to create the function graph (i.e. tensorflow) you shouldn't have any problems with backprop.&lt;/p&gt;&#xA;" OwnerUserId="33839" LastActivityDate="2020-09-15T04:18:02.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23596" PostTypeId="2" ParentId="23349" CreationDate="2020-09-15T04:52:48.767" Score="1" Body="&lt;p&gt;&lt;strong&gt;AlphaDogfight&lt;/strong&gt; - from &lt;a href=&quot;https://en.wikipedia.org/wiki/DARPA&quot; rel=&quot;nofollow noreferrer&quot;&gt;Defense Advanced Research Projects Agency (DARPA)&lt;/a&gt; a programme that pitted computers using F-16 flight simulators against one another and later went on to defeat Air Force’s top F-16 fighter pilots.&lt;/p&gt;&#xA;&lt;p&gt;Check out &lt;a href=&quot;https://www.darpa.mil/news-events/2020-08-26&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://www.darpa.mil/news-events/2020-08-07&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; news and events by DARPA.&lt;/p&gt;&#xA;" OwnerUserId="40485" LastEditorUserId="40485" LastEditDate="2020-09-15T05:02:32.470" LastActivityDate="2020-09-15T05:02:32.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23597" PostTypeId="2" ParentId="23506" CreationDate="2020-09-15T05:13:15.153" Score="0" Body="&lt;p&gt;I think what you need to use is 3D convolution operation. Your data is 3D, width, height, and num_channels. Your data is similar to color images with RGB channels. However, since you are trying to consider the correlation amongst channels 2D convolution will not work for you. You can use 3D convolution which is available to use with deep learning tools such as Tensorflow.&lt;/p&gt;&#xA;" OwnerUserId="40957" LastActivityDate="2020-09-15T05:13:15.153" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23599" PostTypeId="1" AcceptedAnswerId="23600" CreationDate="2020-09-15T07:26:33.550" Score="0" ViewCount="23" Body="&lt;p&gt;I am using a dataset from Google which contains 1,27,000 data points on simulated concentrations of the atmosphere of exoplanets which can sustain life. So, the output label of all these data points is 1 i.e, probability of life existing there is 1. If I train my neural network on this data, and test it on data points with concentrations other than these, can I expect to get probability values at the output? Asking because the model knows no false labelled value.&lt;/p&gt;&#xA;" OwnerUserId="40849" LastActivityDate="2020-09-15T08:39:54.970" Title="Can a neural network be trained on a dataset containing only values for true output for a classification problem?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23600" PostTypeId="2" ParentId="23599" CreationDate="2020-09-15T08:39:54.970" Score="0" Body="&lt;p&gt;Yes, you can and the answer is &lt;strong&gt;One-Class Classification&lt;/strong&gt;. A well-written resource to understand is &lt;a href=&quot;https://machinelearningmastery.com/one-class-classification-algorithms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40485" LastActivityDate="2020-09-15T08:39:54.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23601" PostTypeId="2" ParentId="22900" CreationDate="2020-09-15T08:54:42.830" Score="5" Body="&lt;p&gt;Markov decision problems are usually defined with a reward function &lt;span class=&quot;math-container&quot;&gt;$r:\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R}$&lt;/span&gt;, and in these cases the rewards are expected to be scalar real values. This makes reinforcement learning (RL) easier, for example when defining a policy &lt;span class=&quot;math-container&quot;&gt;$\pi(s,a)=\arg\max_a Q(s,a)$&lt;/span&gt;, it is clear what is the maximum of the Q-factors in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As you might have also realized, in practice however, problems often have multiple objectives that we wish to optimize at the same time. This is called multiobjective optimization and the related RL field is multiobjective reinforcement learning (MORL). If you have access to the paper &lt;em&gt;Liu, Xu, Hu: Multiobjective Reinforcement Learning: A Comprehensive Overview (2015)&lt;/em&gt; you might be interested in reading it. (Edit: as Peter noted in his answer, the original version of this paper was found to be a plagiarism of various other works. Please refer to his answer for better resources.)&lt;/p&gt;&#xA;&lt;p&gt;The above-mentioned paper categorizes methods for dealing with multiple rewards into two categories:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;single objective strategy&lt;/strong&gt;, where multiple rewards are somehow aggregated into one scalar value. This can be done by giving weights to rewards, making some of the objectives a constraint and optimize the others, ranking the objectives and optimize them in order etc. (Note: in my experience, weighted sum of rewards is not a good objective as it might combine two completely unrelated objectives in a very forced way.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Pareto strategy&lt;/strong&gt;, where the goal is to find Pareto-optimal strategies or a Pareto front. In this case we keep the rewards a vector and may compute a composite Q-factor, eg.: &lt;span class=&quot;math-container&quot;&gt;$\bar{Q}(s,a)=[Q_1(s,a), \ldots, Q_N(s,a)]$&lt;/span&gt; and may have to modify the &lt;span class=&quot;math-container&quot;&gt;$\arg\max_a$&lt;/span&gt; function to select the maximum in a Pareto sense.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Finally, I believe it is important to remind you that all these methods really depend on the use-case and on what you really want to achieve and that there is no one solution that fits all. Even after finding an appropriate method you might find yourself spending time tweaking hyper-parameters just so that your RL agent would do what you would like it to do in one specific scenario and do something else in a slightly different scenario. (Eg. taking over on a highway vs. taking over on a country road).&lt;/p&gt;&#xA;" OwnerUserId="8448" LastEditorUserId="8448" LastEditDate="2021-01-15T15:45:52.660" LastActivityDate="2021-01-15T15:45:52.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23602" PostTypeId="1" AcceptedAnswerId="23606" CreationDate="2020-09-15T10:19:26.177" Score="1" ViewCount="69" Body="&lt;p&gt;My simple understanding of AI is that it is based on a mathematical model of a problem. If I understood correctly, the model is a polynomial equation and its weights are calculated by training the model with data sets.&lt;/p&gt;&#xA;&lt;p&gt;I am interested to see a few example polynomial equations (trained models) which are used in certain problem areas. I tried to search it, but so far could not find any simple answers.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone list a few examples here?&lt;/p&gt;&#xA;" OwnerUserId="41003" LastEditorUserId="2444" LastEditDate="2020-09-15T13:38:07.397" LastActivityDate="2020-09-15T14:23:14.770" Title="What are some examples of functions that machine learning models compute?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;datasets&gt;&lt;models&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23604" PostTypeId="1" AcceptedAnswerId="23608" CreationDate="2020-09-15T11:06:38.880" Score="3" ViewCount="104" Body="&lt;p&gt;Equation 7.3 of &lt;a href=&quot;http://incompleteideas.net/book/bookdraft2017nov5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton Barto&lt;/a&gt; book:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{Equation: } max_s|\mathbb{E}_\pi[G_{t:t+n}|S_t = s] - v_\pi| \le \gamma^nmax_s|V_{t+n-1}(s) - v_\pi(s)| $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{where }G_{t:t+n} = R_{t+1} + \gamma R_{t+2} + .....+\gamma^{n-1} R_{t+n} + \gamma^nV_{t+n-1}(S_{t+n})$$&lt;/span&gt;&#xA;Here &lt;span class=&quot;math-container&quot;&gt;$V_{t+n-1}(S_{t+n})$&lt;/span&gt; is the estimate of &lt;span class=&quot;math-container&quot;&gt;$V_\pi(S_{t+n})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But the Left Hand Side of the above equation should be zeros as, for any state s, &lt;span class=&quot;math-container&quot;&gt;$G_{t:t+n}$&lt;/span&gt; is an unbiased estimate of &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; hence &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[G_{t:t+n}|S_t = s] = v_\pi(s)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="2444" LastEditDate="2020-09-20T10:05:10.257" LastActivityDate="2020-09-20T10:05:10.257" Title="What is wrong with equation 7.3 in Sutton &amp; Barto's book?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;sutton-barto&gt;&lt;expectation&gt;&lt;return&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23605" PostTypeId="1" AcceptedAnswerId="23624" CreationDate="2020-09-15T11:59:54.363" Score="2" ViewCount="48" Body="&lt;p&gt;I've been reading through the research literature for image processing, computer vision, and convolutional neural networks. For image classification and object recognition, I know that convolutional neural networks deliver state-of-the-art performance when large amounts of data are available. Furthermore, I know that Hinton et al. created &amp;quot;capsule networks&amp;quot; to try and overcome some of the fundamental limitations of CNN architecture (such as them not being rotationally invariant). However, my understanding is that capsule networks have been a failure (so far), and most people expect them to go nowhere. And CNNs have progressively been improved in various ways (Bayesian optimisation for hyper parameter tuning, new convolution kernels, etc.). It seems to me that, at the moment, and for the foreseeable future, CNNs are the best architecture available for image-related stuff.&lt;/p&gt;&#xA;&lt;p&gt;But, as I said, CNNs, like other Deep Learning architectures, require large amounts of data. So my question is as follows:&lt;/p&gt;&#xA;&lt;p&gt;What are the research areas/topics for improving CNNs in the sense of making them work more effectively (that is, have greater performance) with less data (working with small datasets)?&lt;/p&gt;&#xA;&lt;p&gt;I know that there is various research looking at approaches to &lt;em&gt;increasing&lt;/em&gt; data (such as data augmentation, generative networks, etc.), but I am primarily interested in fundamental modifications to CNNs themselves, rather than purely focusing on changes to the data itself.&lt;/p&gt;&#xA;&lt;p&gt;And to expand upon my question, using my above definition of &amp;quot;performance&amp;quot;, I am interested in these two categories:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;Computational methods&amp;quot; for increasing CNN performance. This would be the non-mathematical stuff that I've read about, such as just increasing the number of layers and making the CNN deeper/wider (and I think another one had to do with just making the size of the convolution kernel smaller, so that it looks at smaller pieces of the image at any one time, or something like that?).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;Mathematical methods&amp;quot; for increasing CNN performance. This would be the cutting-edge mathematical/statistical stuff that I've read about: things like algorithms (such as Bayesian optimization); I've come across a lot of geometric stuff; and I guess the cutting-edge convolution kernels created by the image processing people would also fall under this category.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Obviously, this &amp;quot;list&amp;quot; is not exhaustive, and it's probably incorrect; I'm a novice to this research, so I'm trying to find my way around.&lt;/p&gt;&#xA;&lt;p&gt;I am interested in studying &lt;em&gt;both&lt;/em&gt; of the above categories, but I will primarily be working from the mathematical/statistical side. And I want to work on research that is still &lt;em&gt;practical&lt;/em&gt; and can be put to use in industry for improved performance (even if it might still be &amp;quot;advanced&amp;quot;/complex for most people in industry) – not the the highly theoretical stuff related.&lt;/p&gt;&#xA;&lt;p&gt;Related (but unanswered): &lt;a href=&quot;https://ai.stackexchange.com/q/20521/16521&quot;&gt;Are there any good research papers on image identification with limited data?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16521" LastActivityDate="2020-09-16T14:41:40.057" Title="Research paths/areas for improving the performance of CNNs when faced with limited data" Tags="&lt;convolutional-neural-networks&gt;&lt;math&gt;&lt;research&gt;&lt;image-processing&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23606" PostTypeId="2" ParentId="23602" CreationDate="2020-09-15T13:33:27.087" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If I understood correctly, the model is a polynomial equation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, it's not true that &lt;em&gt;all&lt;/em&gt; machine learning (ML) models compute (or represent) a polynomial function. For example, a sigmoid is not a polynomial, but, for example, in a neural network, you can combine many sigmoids to build complicated functions that may not necessarily be polynomials.&lt;/p&gt;&#xA;&lt;p&gt;We usually distinguish between &lt;strong&gt;linear&lt;/strong&gt; (straight-lines) and &lt;strong&gt;non-linear&lt;/strong&gt; &lt;em&gt;functions&lt;/em&gt; (rather than polynomials and non-polynomials). In some cases, it is straightforward to visualize the function that your model computes: for example, in the case of &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear regression&lt;/a&gt;, once you learned the coefficients (i.e. the slope and y-intercept), you can plot the learned straight-line function. In other cases, for example, in the case of &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;neural networks&lt;/a&gt;, it is not fully clear how to visualize the function that your model computes, given that it is the composition of many non-linear functions (typically, ReLUs, sigmoids or hyperbolic tangents).&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in solving problems with polynomials, take a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Polynomial_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;polynomial regression&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;and its weights are calculated by training the model with data sets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, in machine learning, we want to find a function that &amp;quot;fits the given data&amp;quot;, and the specific meaning of &amp;quot;fitting the data&amp;quot; depends on the specific machine learning technique.&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, let's focus on &lt;strong&gt;supervised&lt;/strong&gt; learning, a machine learning technique where we are given a &lt;strong&gt;labelled&lt;/strong&gt; dataset, i.e. a dataset of pairs &lt;span class=&quot;math-container&quot;&gt;$D = \{(x_1, y_1), \dots, (x_N, y_N)\}$&lt;/span&gt;, where we assume that &lt;span class=&quot;math-container&quot;&gt;$f(x_i) = y_i$&lt;/span&gt;, for some typically unknown function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;s are the &lt;strong&gt;labels&lt;/strong&gt; (the outputs of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; the inputs of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The goal is to find function &lt;span class=&quot;math-container&quot;&gt;$g_{\theta}$&lt;/span&gt; that approximates well &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. I will soon describe what the subscript &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; represents.&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, let's &lt;strong&gt;assume&lt;/strong&gt; that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is a linear function (i.e. a straight-line). So, we can define a linear model &lt;span class=&quot;math-container&quot;&gt;$g_{\theta}$&lt;/span&gt; that we can use to find a function that approximates well &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. Here is the linear model&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$g_{\theta}(x) = ax + b,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g_{\theta}(x)$&lt;/span&gt; is the output&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the input&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$ax + b$&lt;/span&gt; is the linear function (a straight-line)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is the slope (a parameter, aka weight)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-intercept (another parameter)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta = \{ a, b \}$&lt;/span&gt; (the set of parameters of the linear model)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;Why is this a model?&lt;/em&gt; I call this a model because, depending on the &lt;strong&gt;specific&lt;/strong&gt; values of the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, we have different specific functions. So, I am using the term &amp;quot;model&amp;quot; as a synonym for a set of functions, which, in this case, are limited by the definition &lt;span class=&quot;math-container&quot;&gt;$ax + b$&lt;/span&gt; and the specific values that &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) can take.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;So, what do we do with this linear model?&lt;/em&gt; We want to find a specific set of parameters &lt;span class=&quot;math-container&quot;&gt;$\hat{\theta}$&lt;/span&gt; (note that I use the &lt;span class=&quot;math-container&quot;&gt;$\hat{ }$&lt;/span&gt; to emphasize that this is a specific configuration of the variable &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;) that corresponds to a linear function (a straight-line) that approximates &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; well.  In other words, we need to find the parameters &lt;span class=&quot;math-container&quot;&gt;$\hat{\theta}$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$g_\hat{\theta} \approx f$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\approx$&lt;/span&gt; means &amp;quot;approximately computes&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;How do we do that?&lt;/em&gt; We typically don't know &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, but we know (or assumed) that &lt;span class=&quot;math-container&quot;&gt;$f(x_i) = y_i$&lt;/span&gt;, so the labeled dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; contains information about our unknown function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. So, the idea is that we can use the dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to find a specific set of parameters &lt;span class=&quot;math-container&quot;&gt;$\hat{\theta}$&lt;/span&gt; that corresponds to some function that approximates &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; according to the information in &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This process of finding &lt;span class=&quot;math-container&quot;&gt;$\hat{\theta}$&lt;/span&gt; based on &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is often denoted as &amp;quot;fitting the model to the data&amp;quot;. There different ways of fitting the model to the data that differ in the way they compute some notion of distance between the information in &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g_{\hat{\theta}}$&lt;/span&gt;. I will not explain them here because this answer is already quite long. If you want to know more about it, you should take a book about the topic and read it.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are some examples of functions that machine learning models compute?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't have specific examples, but you can easily try to fit a linear regression model to some labelled data, then plot the function that you found. You could use the Python library &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sklearn&lt;/code&gt;&lt;/a&gt; to do that.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-15T14:23:14.770" LastActivityDate="2020-09-15T14:23:14.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23607" PostTypeId="1" CreationDate="2020-09-15T13:36:50.983" Score="1" ViewCount="72" Body="&lt;p&gt;After the state of the art Deep Learning techniques/algorithms being implemented in low-level languages like Objective-C, C++, etc to high-level languages like Python, JS, etc. and with the help of huge libraries like &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow&lt;/a&gt;, &lt;a href=&quot;https://pytorch.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch&lt;/a&gt;, &lt;a href=&quot;https://scikit-learn.org/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scikit-Learn&lt;/a&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;Now, &lt;a href=&quot;https://swift.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swift&lt;/a&gt;: Google's bet on &lt;a href=&quot;https://en.wikipedia.org/wiki/Differentiable_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;differentiable programming&lt;/a&gt;, they are making Swift differential programming ready see &lt;a href=&quot;https://github.com/apple/swift/blob/master/docs/DifferentiableProgramming.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;this manifesto&lt;/a&gt; and they are building TensorFlow from the ground up in swift &lt;a href=&quot;https://www.tensorflow.org/swift&quot; rel=&quot;nofollow noreferrer&quot;&gt;S4TF&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, how &lt;em&gt;differentiable programming&lt;/em&gt; and programming language supporting it will &lt;em&gt;potentially&lt;/em&gt; help the development towards AGI?&lt;/p&gt;&#xA;" OwnerUserId="40485" LastEditorUserId="2444" LastEditDate="2020-09-16T16:28:09.437" LastActivityDate="2020-09-16T16:28:09.437" Title="How differentiable programming and programming language supporting it will potentially help the development towards AGI?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;agi&gt;&lt;research&gt;&lt;programming-languages&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23608" PostTypeId="2" ParentId="23604" CreationDate="2020-09-15T14:54:36.707" Score="3" Body="&lt;p&gt;In general, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[G_{t:t+n}|S_t = s] \neq v_\pi(s)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_\pi[\sum_{k=0}^{\infty} \gamma^k R_{t+k+1} | S_t = s]$&lt;/span&gt;, so you should be able to see why the two are not equal when the LHS is an expectation of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th step return. They would only be equal as &lt;span class=&quot;math-container&quot;&gt;$n \rightarrow \infty$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-09-15T14:54:36.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23609" PostTypeId="2" ParentId="23369" CreationDate="2020-09-15T15:43:33.577" Score="0" Body="&lt;p&gt;Thanks for your answer. I have tried to assess other models using cross validation and could see that the complex neural network models did not perform very well on out of sample data.. what tricked me was that i looked at the predicted data from the training on the full data set ( after optimization of hyper parameter tuning). This predicted data alwasys looked extremely well correlated with actual data. I did not look at the magnitude of the RMSE after hyperparamter tuning.Because. When I then did an extra &amp;quot;leave-one-out&amp;quot; validation i could get random results.&lt;/p&gt;&#xA;&lt;p&gt;In the end I used caret's resampling method  which compares RMSE of different models.and found out that svmPoly (support vector machine) or random forrest had the best out of sample performance - better than lm and GLMnet.&lt;/p&gt;&#xA;" OwnerUserId="40709" LastActivityDate="2020-09-15T15:43:33.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23611" PostTypeId="1" CreationDate="2020-09-15T16:16:37.950" Score="4" ViewCount="49" Body="&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt; encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of the input length of the text, which makes it more cumbersome to use as input to downstream neural networks that take only fixed-size inputs.&lt;/p&gt;&#xA;&lt;p&gt;Are there any transformer-based neural network architectures that can encode a piece of text into a fixed-size feature vector more suitable for downstream tasks?&lt;/p&gt;&#xA;" OwnerUserId="9220" LastEditorUserId="2444" LastEditDate="2020-09-20T18:02:25.537" LastActivityDate="2020-09-20T18:02:25.537" Title="Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;autoencoders&gt;&lt;transformer&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23613" PostTypeId="1" CreationDate="2020-09-15T18:43:29.633" Score="1" ViewCount="23" Body="&lt;p&gt;I want to use frames from video game and analyze them using CNN and LSTM. But when I have the model defined like that&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;frames, channels, rows, columns = 5,3,224,224&#xA;&#xA;video = Input(shape=(frames,&#xA;                     rows,&#xA;                     columns,&#xA;                     channels))&#xA;cnn_base = VGG16(input_shape=(rows,&#xA;                              columns,&#xA;                              channels),&#xA;                 weights=&amp;quot;imagenet&amp;quot;,&#xA;                 include_top=False)&#xA;cnn_base.trainable = False&#xA;&#xA;cnn_out = GlobalAveragePooling2D()(cnn_base.output)&#xA;cnn = Model(cnn_base.input, cnn_out)&#xA;encoded_frames = TimeDistributed(cnn)(video)&#xA;encoded_sequence = LSTM(256)(encoded_frames)&#xA;hidden_layer = Dense(1024, activation=&amp;quot;relu&amp;quot;)(encoded_sequence)&#xA;outputs = Dense(10, activation=&amp;quot;softmax&amp;quot;)(hidden_layer)&#xA;&#xA;model = Model(video, outputs)&#xA;model.summary()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it is very slow. Because you must feed 5 frames to the model and it works like that:&lt;/p&gt;&#xA;&lt;p&gt;You feed 1,2,3,4 and 5 frame&#xA;Then you feed 2,3,4,5 and 6 frame&#xA;Then you feed 3,4,5,6 and 7 frame&lt;/p&gt;&#xA;&lt;p&gt;And every time the CNN calculations are done for every frame instead of only the new one. It's terrible waste of time. I think it should work that for every new frame there are CNN calculations and then you feed extracted features from previous frames (and the newest frame) to the LSTM. So only LSTM part must be done from the beginning (because it is the new sequence).&lt;/p&gt;&#xA;&lt;p&gt;I think one solution would be to have two separate networks - CNN and LSTM. So for every frame you can calculate features using CNN that takes 1 image as input and then you feed features to LSTM.&#xA;Is it ok? If yes, how to train such CNN to extract useful features? In order to train CNN to extract useful features you must define some output for this CNN (without defined output you cannot train any network). So how I should define output of the CNN? The same way as the output of the LSTM or CNN-LSTM network?&#xA;I want to use later the last pooling layer from trained CNN to feed features from this layer to the LSTM.&#xA;But the important question is: how to define output of the CNN network? The same way as the output of the LSTM network?&lt;/p&gt;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&lt;p&gt;My task is autonomous driving but I think it doesn't matter here. The code is just an example. I want to train my own network from scratch, not use pretrained network like VGG.&lt;/p&gt;&#xA;" OwnerUserId="40943" LastActivityDate="2020-09-15T18:43:29.633" Title="Duplicating calculations in CNN-LSTM architecture" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23614" PostTypeId="1" CreationDate="2020-09-15T21:45:54.853" Score="2" ViewCount="91" Body="&lt;p&gt;In GPT-2, the large achievement was being able to generate coherent text over a long-form while maintaining context. This was very impressive but for GPT-2 to do new language tasks, it had to be explicitly fine-tuned for the new task.&lt;/p&gt;&#xA;&lt;p&gt;In GPT-3 (From my understanding), this is no longer the case. It can perform a larger array of language tasks from translation, open domain conversation, summarization etc. with only a few examples. No explicit fine-tuning is needed.&lt;/p&gt;&#xA;&lt;p&gt;The actual theory behind GPT-3 is &lt;em&gt;fairly&lt;/em&gt; simple, which would not suggest any level of ability other than what would be found in common Narrow Intelligence systems. However, Looking past the media hype and the news coverage, GPT-3 not explicitly programmed to &amp;quot;know&amp;quot; how to do these wider arrays of tasks. In fact, with limited examples, it can perform many language tasks quite well and &amp;quot;learn on the fly&amp;quot; so to speak. To me, this does seem to align fairly well with what most people would consider strong AI but in a narrow context, which is language tasks.&lt;/p&gt;&#xA;&lt;p&gt;Thoughts? Is GPT-3 an early example of strong AI but in a narrower context?&lt;/p&gt;&#xA;" OwnerUserId="22840" LastActivityDate="2020-09-16T02:41:34.507" Title="Is GPT-3 an early example of Strong AI in a Narrow setting?" Tags="&lt;philosophy&gt;&lt;agi&gt;&lt;theory&gt;&lt;gpt&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23615" PostTypeId="2" ParentId="23614" CreationDate="2020-09-16T02:41:34.507" Score="2" Body="&lt;p&gt;GPT-3 is based on &lt;strong&gt;in-context learning&lt;/strong&gt;. It’s common wisdom one can hope that bigger models will yield better in-context capabilities. And indeed, this holds true, in the case of GPT-3 175B or &amp;quot;GPT-3&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Neverthless GPT-3 is more powerful than it's predecessors. In some of the tasks, GPT-3 failed miserably. This might be due to the choice to use an autoregressive LM, instead of incorporating bidirectional information (similarly to Bert).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;While in-context learning is more straightforward with autoregressive LMs, bidirectional models are known to be better at downstream tasks after fine-tuning&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the end, training a bidirectional model at the scale of GPT-3 or trying to make bidirectional models work with few-shot learning is a promising direction for future research.&lt;/p&gt;&#xA;&lt;p&gt;Check out &lt;a href=&quot;https://www.kdnuggets.com/2020/06/gpt-3-deep-learning-nlp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://towardsdatascience.com/gpt-3-primer-67bc2d821a00&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/2001.08361.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper on Scaling Laws for Neural Language Models&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="40485" LastActivityDate="2020-09-16T02:41:34.507" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23617" PostTypeId="2" ParentId="23052" CreationDate="2020-09-16T10:59:22.830" Score="0" Body="&lt;p&gt;There are measuring machines which, based on sensors, are able to determine specific parameters of the body shape or state.&#xA;(proteins, muscle mass, adipose tissue mass, body-weight, height)&#xA;and separate into segments. (right hand, left hand etc.)&lt;/p&gt;&#xA;&lt;p&gt;Hovewer the same issue seems relatively difficult.&lt;/p&gt;&#xA;&lt;p&gt;With my humble opinion, I can recommend the process to you:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Prepare a position pattern - all photos should be in a similar&#xA;pattern based on one algorithm.&lt;/li&gt;&#xA;&lt;li&gt;Reading height or weight itself seems to be a complicated topic.&lt;/li&gt;&#xA;&lt;li&gt;I suggest Initially start on the algorithm - &lt;strong&gt;image recognition&lt;/strong&gt;(You can reuse existing solutions (Pytorch || TensorFlow) - (CNN + seq2seq with visual attention will be usefull here)&lt;/li&gt;&#xA;&lt;li&gt;Then use some methods - which can be seen in the photo - after obtaining (x1, x2, y1, y2) from the person - measure the correlation between y and y axis (the greater the higher the weight - although not necessarily)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;While exploring, more variables will appear for analysis&#xA;Initially recommend 2 repositories:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/sarthakagarwal18/Image-Recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sarthakagarwal18/Image-Recognition&lt;/a&gt;&#xA;&lt;a href=&quot;https://github.com/Kidel/Deep-Learning-CNN-for-Image-Recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Kidel/Deep-Learning-CNN-for-Image-Recognition&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32352" LastActivityDate="2020-09-16T10:59:22.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23618" PostTypeId="1" CreationDate="2020-09-16T11:28:29.667" Score="4" ViewCount="123" Body="&lt;p&gt;That is, if AGI were an existing technology, how much would it be valued to?&lt;/p&gt;&#xA;&lt;p&gt;Obviously it would depend on its efficiency, if it requires more than all the existing hardware to run it, it would be impossible to market.&lt;/p&gt;&#xA;&lt;p&gt;This question is more about getting a general picture of the economy surrounding this technology.&lt;/p&gt;&#xA;&lt;p&gt;Assuming a specific definition of AGI and that we implemented that AGI, what is its potential economical value?&lt;/p&gt;&#xA;&lt;p&gt;Current investments in this research field are also useful data.&lt;/p&gt;&#xA;" OwnerUserId="41025" LastEditorUserId="41025" LastEditDate="2020-09-16T13:13:26.840" LastActivityDate="2020-10-21T22:56:43.323" Title="What is the current artificial general intelligence technology valuation?" Tags="&lt;agi&gt;&lt;economics&gt;" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23619" PostTypeId="1" CreationDate="2020-09-16T12:58:03.157" Score="1" ViewCount="34" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I have tried to fit a logistic regression model - written using a forward / back propagation approach (as part of Andrew Ng's deep learning course) - to a very non-linear data set (see picture below). Of course, it totally fails; in Andrew Ng's course, the failure of logistic regression to fit to this motivates developing a neural net - which works quite nicely.  But my question concerns what my logistic model is doing and why.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EUG4P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EUG4P.png&quot; alt=&quot;non-linear data set&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The problem&lt;/strong&gt; My logistic regression model's cost increases, even after massively reducing the learning rate. But at the same time my accuracy (slowly) increases. I simply cannot see why.&lt;/p&gt;&#xA;&lt;p&gt;To confuse matters even more - if I resort to a negative learning rate (essentially trying to force the calibration to higher cost values) the cost then decreases for a time until the accuracy hits 50%. After this point, the cost then inexorably increases - but the accuracy stays equal to 50%. The solution so found is to set all points to either red or blue (a reasonable fit given logistic regression simply cannot work on this data).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My questions and thoughts on answers&lt;/strong&gt; I have reproduced the Python code below - hopefully it's clear. My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a mistake in the model that explains why negative learning rates seem to work better?&lt;/li&gt;&#xA;&lt;li&gt;On the topic of why the cost increases even as accuracy asymptotes to 50%: is the issue that once the model has discovered the &amp;quot;all points equal to either red or blue&amp;quot; solution the parameters &amp;quot;w&amp;quot; and &amp;quot;b&amp;quot; just get larger and larger (in absolute terms) - driving all of the predictions closer to 1 (or conversely if it predicts all points are 0)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To explain this second question a bit more: imagine red points are defined by y = 1. Suppose parameters w, b are chosen such that the probability for every point equals 0.9. Then the model predicts all points are red - which is correct for half the points. The model can then improve half the predictions by driving w and b up (so that sigmoid ( w*x + b) --&amp;gt; 1). But of course, this makes half the predictions (the blue points) more and more wrong - which causes the cost function for those points - log(1 - prob) - to diverge. I don't truly see why gradient descent would do this but it's all I can think of for the peculiar behaviour of the algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Hope this all makes sense. Hit me up if not.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;# function to create a flower-like arrangement of 1s and 0s&#xA;def load_planar_dataset():&#xA;    np.random.seed(1)&#xA;    m = 400 # number of examples&#xA;    N = int(m/2) # number of points per class&#xA;    D = 2 # dimensionality / i.e. work in 2d plane - so X is a set of (x,y) coordinate points&#xA;    X = np.zeros((m,D)) # data matrix where each row is a single example&#xA;    Y = np.zeros((m,1), dtype='uint8') # labels vector (0 for red, 1 for blue)&#xA;    a = 4 # maximum ray of the flower&#xA;&#xA;    for j in range(2):&#xA;        ix = range(N*j,N*(j+1))&#xA;        t = np.linspace(j*3.12,(j+1)*3.12,N) + np.random.randn(N)*0.2 # theta / random element mixes up some of the petals so you get mostly blue with some red petals and vice-versa&#xA;        r = a*np.sin(4*t) + np.random.randn(N)*0.2 # radius / again random element alters  shape of flower slightly&#xA;        X[ix] = np.c_[r*np.sin(t), r*np.cos(t)]&#xA;        Y[ix] = j&#xA;        &#xA;    X = X.T # transpose so columns = training example as per standard in lectures&#xA;    Y = Y.T&#xA;&#xA;    return X, Y&#xA;&#xA;# function to plot the above data plus a modelled decision boundary - works by applying model to grid of points and colouring accordingly&#xA;def plot_decision_boundary(model, X, y):&#xA;    # Set min and max values and give it some padding&#xA;    x_min, x_max = X[0, :].min() - 1, X[0, :].max() + 1&#xA;    y_min, y_max = X[1, :].min() - 1, X[1, :].max() + 1&#xA;    h = 0.01&#xA;    # Generate a grid of points with distance h between them&#xA;    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))&#xA;    # Predict the function value for the whole grid&#xA;    Z = model(np.c_[xx.ravel(), yy.ravel()])&#xA;    Z = Z.reshape(xx.shape)&#xA;    # Plot the contour and training examples&#xA;    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)&#xA;    plt.ylabel('x2')&#xA;    plt.xlabel('x1')&#xA;    plt.scatter(X[0, :], X[1, :], c=y, cmap=plt.cm.Spectral)&#xA;&#xA;&#xA;# sigmoid function as per sandard linear regression&#xA;def sigmoid(z):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Compute the sigmoid of z&#xA;&#xA;    Arguments:&#xA;    z -- A scalar or numpy array of any size.&#xA;&#xA;    Return:&#xA;    s -- sigmoid(z)&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    s = 1. / (1. + np.exp(-z))&#xA;    &#xA;    return s&#xA;&#xA;&#xA;# &#xA;def propagate(w, b, X, Y):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Implement the cost function and its gradient for the propagation explained above&#xA;&#xA;    Arguments:&#xA;    w -- weights, a numpy array of size (num_px * num_px * 3, 1)&#xA;    b -- bias, a scalar&#xA;    X -- data of size (num_px * num_px * 3, number of examples)&#xA;    Y -- true &amp;quot;label&amp;quot; vector (containing 0 if non-cat, 1 if cat) of size (1, number of examples)&#xA;&#xA;    Return:&#xA;    cost -- negative log-likelihood cost for logistic regression&#xA;    dw -- gradient of the loss with respect to w, thus same shape as w&#xA;    db -- gradient of the loss with respect to b, thus same shape as b&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    m = X.shape[1];&#xA;&#xA;    # forward prop&#xA;    Z = np.dot(w.T, X) + b; &#xA;    A = sigmoid(Z); # activiation = the prediction of the model&#xA;&#xA;&#xA;    # compute cost&#xA;    cost =  - 1. / m * np.sum( (Y * np.log(A) + (1. - Y) * np.log(1. - A)  ) )&#xA;&#xA;    #back prop for gradient descent&#xA;&#xA;    da = - Y / A + (1. - Y) / (1. - A)  &#xA;    dz = da * A * (1. - A) # = - Y (1-A) + (1. - Y) A =  A - Y  &#xA;    dw = 1. / m * np.dot( X, dz.T )&#xA;    db = 1. / m * np.sum(dz)&#xA;&#xA;    grads = {&amp;quot;dw&amp;quot;: dw,&#xA;                &amp;quot;db&amp;quot;: db}&#xA;&#xA;    return grads, cost&#xA;&#xA;&#xA;def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost = False):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    This function optimizes w and b by running a gradient descent algorithm&#xA;    &#xA;    Arguments:&#xA;    w -- weights, a numpy array of size (num_px * num_px * 3, 1)&#xA;    b -- bias, a scalar&#xA;    X -- data of shape (num_px * num_px * 3, number of examples)&#xA;    Y -- true &amp;quot;label&amp;quot; vector (containing 0 if non-cat, 1 if cat), of shape (1, number of examples)&#xA;    num_iterations -- number of iterations of the optimization loop&#xA;    learning_rate -- learning rate of the gradient descent update rule&#xA;    print_cost -- True to print the loss every 100 steps&#xA;    &#xA;    Returns:&#xA;    params -- dictionary containing the weights w and bias b&#xA;    grads -- dictionary containing the gradients of the weights and bias with respect to the cost function&#xA;    costs -- list of all the costs computed during the optimization, this will be used to plot the learning curve.&#xA;    &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    costs = []&#xA;&#xA;    for i in range(num_iterations):&#xA;        &#xA;        # cost /gradient calculation&#xA;        grads, cost = propagate(w, b, X, Y)&#xA;&#xA;        #retrieve derivatives&#xA;        dw = grads[&amp;quot;dw&amp;quot;]&#xA;        db = grads[&amp;quot;db&amp;quot;]&#xA;&#xA;        # update values according to gradient descent algorithm&#xA;        w = w - learning_rate * dw&#xA;        b = b - learning_rate * db&#xA;&#xA;        # record the costs&#xA;        if i % 100 == 0:&#xA;            costs.append(cost)&#xA;&#xA;            # Print the cost every 100 training iterations&#xA;            if print_cost:&#xA;                print(&amp;quot;Cost after iteration %i: %f&amp;quot; %(i, cost))&#xA;&#xA;&#xA;&#xA;    params = {  &amp;quot;w&amp;quot;: w,&#xA;                &amp;quot;b&amp;quot;: b}&#xA;&#xA;    grads = {   &amp;quot;dw&amp;quot;: dw,&#xA;                &amp;quot;db&amp;quot;: db}&#xA;&#xA;    return params, grads, costs&#xA;&#xA;&#xA;def predict(w, b, X):&#xA;    '''&#xA;    Predict whether the label is 0 or 1 using learned logistic regression parameters (w, b)&#xA;    &#xA;    Arguments:&#xA;    w -- weights, a numpy array of size (num_px * num_px * 3, 1)&#xA;    b -- bias, a scalar&#xA;    X -- data of size (num_px * num_px * 3, number of examples)&#xA;    &#xA;    Returns:&#xA;    Y_prediction -- a numpy array (vector) containing all predictions (0/1) for the examples in X&#xA;    '''&#xA;    Z = np.dot(w.T, X) + b&#xA;    A = sigmoid(Z)&#xA;&#xA;&#xA;    Y_prediction = (A &amp;gt;= 0.5).astype(int)&#xA;&#xA;    return Y_prediction&#xA;&#xA;&#xA;&#xA;&#xA;np.random.seed(1) # set a seed so that the results are consistent&#xA;&#xA;X, Y = load_planar_dataset()&#xA;&#xA;# Visualize the data:&#xA;&#xA;plt.scatter(X[0, :], X[1, :], c=Y, s=40, cmap=plt.cm.Spectral); # s = size of points; cmap are nicer colours&#xA;plt.show()&#xA;&#xA;shape_X = X.shape&#xA;shape_Y = Y.shape&#xA;m = shape_Y[1]  # training set size&#xA;n = shape_X[0] # number of features (2)&#xA;&#xA;&#xA;# initialise parameters&#xA;w = np.random.rand(n, 1)&#xA;b = 0&#xA;&#xA;# print accuracy of initial parameters by comparing prediction to &#xA;print(&amp;quot;train accuracy: {} %&amp;quot;.format(100 - np.mean(np.abs(predict(w, b, X) - Y)) * 100))&#xA;&#xA;&#xA;# fit model and print out costs every 100 iterations of the forward / back prop&#xA;parameters, grads, costs = optimize(w, b, X, Y, num_iterations = 10000, learning_rate = 0.000005, print_cost = True)&#xA;&#xA;&#xA;# return the prediction&#xA;Y_prediction = predict(parameters[&amp;quot;w&amp;quot;], parameters[&amp;quot;b&amp;quot;], X)&#xA;&#xA;# print accuracy of fitted model&#xA;print(&amp;quot;train accuracy: {} %&amp;quot;.format(100 - np.mean(np.abs(Y_prediction - Y)) * 100))&#xA;&#xA;&#xA;# print parameters for interest&#xA;print( parameters[&amp;quot;w&amp;quot;] , parameters[&amp;quot;b&amp;quot;] )&#xA;&#xA;# plot decision boundary&#xA;plot_decision_boundary(lambda x: predict(parameters[&amp;quot;w&amp;quot;], parameters[&amp;quot;b&amp;quot;], x.T), X, Y)&#xA;plt.show()&#xA;&#xA;List item&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41027" LastActivityDate="2020-09-16T15:11:33.587" Title="Back propagation approach to logistic regression: why is cost diverging but accuracy increasing?" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;logistic-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23620" PostTypeId="1" CreationDate="2020-09-16T13:06:47.257" Score="4" ViewCount="249" Body="&lt;p&gt;How much is currently invested in artificial general intelligence research and development &lt;em&gt;worldwide&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Feel free to add company or VC names, but this is not the point. The point is to get an idea of the economics around artificial general intelligence.&lt;/p&gt;&#xA;" OwnerUserId="41025" LastEditorUserId="2444" LastEditDate="2021-05-21T22:06:27.240" LastActivityDate="2021-05-21T22:06:27.240" Title="How much is currently invested in artificial general intelligence research and development?" Tags="&lt;reference-request&gt;&lt;agi&gt;&lt;research&gt;&lt;economics&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23622" PostTypeId="2" ParentId="23620" CreationDate="2020-09-16T13:49:32.780" Score="5" Body="&lt;p&gt;In the last years, there have been big investments in AI technologies. For an overview, maybe take a look at this article &lt;a href=&quot;http://documents1.worldbank.org/curated/ar/617511573040599056/pdf/Artificial-Intelligence-Investment-Trends-and-Selected-Industry-Uses.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: Investment Trends and Selected Industry Uses&lt;/a&gt; (2019).&lt;/p&gt;&#xA;&lt;p&gt;A few companies that have the &lt;strong&gt;long-term&lt;/strong&gt; goal of creating an AGI, although, currently, they mainly do research on specific problems (e.g. video games) or AI techniques (e.g. reinforcement learning), have received many funds. I will only list a few (maybe the most well-known ones) of these companies below, but there are probably many other companies that have this long-term goal and have been funded by other companies or people.&lt;/p&gt;&#xA;&lt;h3&gt;DeepMind&lt;/h3&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://deepmind.com/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;their site&lt;/a&gt;, they write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Like the Hubble telescope that helps us see deeper into space, we aim to build advanced AI - sometimes known as &lt;strong&gt;Artificial General Intelligence (AGI)&lt;/strong&gt; - to expand our knowledge and find new answers. By solving this, we believe we could help people solve thousands of problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;DeepMind was acquired by Google in 2015 for $500 million, given its success in playing games at a superhuman performance, which is a promising step towards the development of more AI techniques and maybe AGI.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/DeepMind&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on DeepMind&lt;/a&gt; contains some information about people or companies that have invested in DeepMind, which includes companies Horizons Ventures and Founders Fund, and people  Scott Banister, Peter Thiel, Elon Musk, and Jaan Tallinn, although I cannot give you the exact numbers in terms of capital. In any case, serious investments have been done in DeepMind, which is definitely one of the promising companies that could develop good insights into the development of AGI systems.&lt;/p&gt;&#xA;&lt;h3&gt;OpenAI&lt;/h3&gt;&#xA;&lt;p&gt;Another company that has a similar goal and is doing research on similar topics (such as reinforcement learning or natural language processing) is OpenAI, which also has the long-term goal of creating AGI systems, as they write in &lt;a href=&quot;https://openai.com/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;their website&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;OpenAI’s mission is to ensure that &lt;strong&gt;artificial general intelligence (AGI)&lt;/strong&gt; — by which we mean highly autonomous systems that outperform humans at most economically valuable work — benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://openai.com/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Their investors include Microsoft, Reid Hoffman's charitable foundation, and Khosla Ventures&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Vicarious&lt;/h3&gt;&#xA;&lt;p&gt;They write in &lt;a href=&quot;https://www.vicarious.com/company/&quot; rel=&quot;nofollow noreferrer&quot;&gt;their site&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Artificial general intelligence&lt;/strong&gt; is the finish line on our journey toward progressively more capable robots. Our approach leverages deep expertise in neuroscience and is shaped by a decade of research.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.vicarious.com/company/&quot; rel=&quot;nofollow noreferrer&quot;&gt;They are apparently backed by more than 150 million dollars&lt;/a&gt; from people like Jeff Bezos, Elon Musk and Mark Zuckerberg and companies like Samsung.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-16T15:26:53.837" LastActivityDate="2020-09-16T15:26:53.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23624" PostTypeId="2" ParentId="23605" CreationDate="2020-09-16T14:41:40.057" Score="2" Body="&lt;p&gt;Some research areas that come to mind which can be useful when faced with a limited amount of data:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Regularization: Comprises different methods to prevent the network from overfitting, to make it perform better on the validation data but not necessarily on the training data. In general, the less training data you have, the stronger you want to regularize. Common types include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Injecting noise in the network, e.g., &lt;a href=&quot;https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dropout&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adding regularization terms to the training loss, e.g., L1 and L2 regularization of the weights, but also &lt;a href=&quot;https://arxiv.org/abs/1701.06548&quot; rel=&quot;nofollow noreferrer&quot;&gt;confident output distributions&lt;/a&gt; can be penalized.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reducing the number of parameters in the network to make it unable to fit the training data completely and thus unable to overfit badly. Interestingly, &lt;a href=&quot;https://openai.com/blog/deep-double-descent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;increasing&lt;/em&gt;&lt;/a&gt; the number of parameters for large models can also improve the validation performance.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Early stopping of training. For example, if one part of the training set is set aside and not used to update the weights, training can be stopped when the observed loss on this part of the training set is observed to start to increase.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generating new training data:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Data augmentation: Ways to augment existing training examples without removing the semantics, e.g., slight rotations, crops, translations (shifts) of images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Data interpolation, e.g., &lt;a href=&quot;https://arxiv.org/abs/1806.05236&quot; rel=&quot;nofollow noreferrer&quot;&gt;manifold mixup&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using synthetic data, e.g., frames from video games or other CGI.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transfer learning: When you take a neural network that has already been trained on another, much larger dataset of the same modality (images, sounds, etc.) as your dataset and fine-tune it on your data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Multitask learning: Instead of training your network to perform one task, you give it multiple output heads and train it to perform many tasks at once, given that you have that the labels for the additional tasks. While it may seem that this is a more difficult for the network, the extra tasks have a regularizing effect.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Semi-supervised learning: If you have much unlabeled data that labeled data, you can combine supervised learning with unsupervised learning. Much like with multitask learning, the extra task introduced by the unsupervised learning also has a regularizing effect.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Other interesting methods can be found in systems that perform &lt;a href=&quot;https://arxiv.org/abs/1606.04080&quot; rel=&quot;nofollow noreferrer&quot;&gt;one-shot learning&lt;/a&gt;, which inherently implies very little training data. These systems often uses slightly modified network architectures. For example, facial recognition systems can learn to recognize a face from only a single photo, and usually use a &lt;a href=&quot;https://arxiv.org/abs/1503.03832&quot; rel=&quot;nofollow noreferrer&quot;&gt;triplet loss&lt;/a&gt; (or similar) of a vector encoding of the face, instead of cross-entropy loss of the output of a softmax layer normally used for image classification.&lt;/p&gt;&#xA;&lt;p&gt;Zero-shot learning also exists (e.g., &lt;a href=&quot;https://ai.googleblog.com/2016/11/zero-shot-translation-with-googles.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;zero-shot machine translation&lt;/a&gt;), but this is a completely different type of problem setup and requires multiple data modalities.&lt;/p&gt;&#xA;" OwnerUserId="9220" LastActivityDate="2020-09-16T14:41:40.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23625" PostTypeId="1" AcceptedAnswerId="23628" CreationDate="2020-09-16T15:09:11.747" Score="5" ViewCount="190" Body="&lt;p&gt;I understand that this is the update for the parameters of a policy in REINFORCE:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Delta \theta_{t}=\alpha \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) v_{t}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;sub&gt;&lt;/sub&gt; is usually the discounted future reward and  &lt;sub&gt;&lt;/sub&gt;(&lt;sub&gt;&lt;/sub&gt;|&lt;sub&gt;&lt;/sub&gt;) is the probability of taken the action that the agent took at time . (Tell me if something is wrong here)&lt;/p&gt;&#xA;&lt;p&gt;But I don’t understand how that is passed into a neural network for back propagation.&lt;br /&gt;&#xA;I have this pseudocode&lt;br /&gt;&#xA;&lt;code&gt;probs = policy.feedforward(state)&lt;/code&gt;&lt;br /&gt;&#xA;This returns the probabilities if taking each actions, like: &lt;code&gt;[0.6,0.4]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;action = choose_action_from(probs)&lt;/code&gt; this will return the index of the probability chosen. For example, if it chose 0.6, the action would be 0.&lt;/p&gt;&#xA;&lt;p&gt;Then later when it is time to update, is it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gradient = policy.backpropagate(total_discounted_reward*log(probs[action])&#xA;policy.weights += gradient&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is this the right way to calculate the derivative of the loss and backpropagate it?&lt;/strong&gt; And I only backpropagate this through one output neuron?&lt;/p&gt;&#xA;&lt;p&gt;If you need more explanation, I have this &lt;a href=&quot;https://stackoverflow.com/q/63602222/14170431&quot;&gt;question&lt;/a&gt; on SO.&lt;/p&gt;&#xA;" OwnerUserId="41026" LastEditorUserId="41026" LastEditDate="2020-09-16T16:45:59.557" LastActivityDate="2020-09-16T17:23:18.817" Title="What, exactly, does the REINFORCE update equation mean?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;policy-gradients&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23626" PostTypeId="2" ParentId="23619" CreationDate="2020-09-16T15:11:33.587" Score="1" Body="&lt;p&gt;Just realised the issue - super subtle (at least for a Python novice like me) - I implemented numerical gradient checking (as I should have done from the start) and saw the gradient descent was working incorrectly.&lt;/p&gt;&#xA;&lt;p&gt;The dataset Y was created as a datatype &amp;quot;uint8&amp;quot;. In the line of the back-propagation where I calculate the derivative with respect to &amp;quot;a&amp;quot; (&amp;quot;da&amp;quot;) the statement &amp;quot;-Y&amp;quot; was returning 255 for those values of Y that were 1 - this appears to be a feature of these unsigned binary numbers.&lt;/p&gt;&#xA;&lt;p&gt;By replacing &amp;quot;uint8&amp;quot; with &amp;quot;int8&amp;quot; in the definition of Y - or else forcing the correct behaviour by writing &amp;quot;-1.*Y&amp;quot; in the calculation of &amp;quot;da&amp;quot; (which I think casts &amp;quot;Y&amp;quot; as an integer rather than an unsigned integer) I managed to produce the correct behaviour.&lt;/p&gt;&#xA;&lt;p&gt;This in turn means the gradient descent is doing what it should and the logistic regression converges to a stable value of the cost. Accuracy then tracks cost as well.&lt;/p&gt;&#xA;" OwnerUserId="41027" LastActivityDate="2020-09-16T15:11:33.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23627" PostTypeId="1" AcceptedAnswerId="23642" CreationDate="2020-09-16T16:45:06.560" Score="1" ViewCount="46" Body="&lt;p&gt;I am training an algorithm to identify weeds within crops using the YOLOv5 algorithm. This algorithm will  be used in the future to identify weeds in images collected by unmanned aircraft (drones) after making an orthomosaic images. Using the open-source LabelImg software, I am labeling images for object detection that were collected with both UAV and hand-held digital cameras. Using both platforms, I collected many images of weeds that will need to be identified.&lt;/p&gt;&#xA;&lt;p&gt;My question is this: Does it make sense to collect training samples from the hand-held digital camera, since it will be of much higher resolution than the UAV imagery (and thus not used for future imagery collections after the model is trained)? My initial thought is that it would be best to only use the UAV imagery, since it will be the most similar to what will be collected in the future. However, I do not want to throw out the hand-held digital imagery if it could help in the image classification process.&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-09-18T12:57:49.417" Title="Does it make sense to train images (for object detection algorithms) with cameras that will not be used to collect future data?" Tags="&lt;deep-learning&gt;&lt;training&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23628" PostTypeId="2" ParentId="23625" CreationDate="2020-09-16T17:23:18.817" Score="3" Body="&lt;p&gt;The loss function you are looking for is &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross entropy loss&lt;/a&gt;. The 'label' that you use is the action you took at the time point you are updating for.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-09-16T17:23:18.817" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23629" PostTypeId="1" CreationDate="2020-09-16T23:12:54.117" Score="1" ViewCount="19" Body="&lt;p&gt;Straying from the current trends in deep learning, there is an, arguably, interesting idea of &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuronal_ensemble&quot; rel=&quot;nofollow noreferrer&quot;&gt;neuronal ensembles&lt;/a&gt; possibly providing an alternative to the current &amp;quot;layered feature detectors&amp;quot; framework for neural network construction by being considered a basic computational unit instead of one feature detecting neuron. This idea certainly has at least some presence in neuroscience circles, but I found it hard to find studies, attempting to obtain a working computational ensemble based model, which would try to solve any of the existing computer vision/NLP tasks or anything of the sort. This may be just due to me looking in the wrong places, but in any case, I would appreciate any references to papers, exploring building neural network architectures with neuronal ensemblies involvement.&lt;/p&gt;&#xA;&lt;p&gt;Just to be clear, I would be interested in &lt;em&gt;any&lt;/em&gt; papers on computational modelling of ensemblies even if they are not trying to solve any particular ML task, but it would be better, if the topic of the research is more closely aligned with computer science instead of neurobiology even if CS connection is of a more exotic kind; for example, paper trying to see, whether you can store different concepts and their relations in the ensemble based network is more desirable than paper, trying to accurately model individual neuron and synaptic plasticity dynamics and see, that ensembles emerge, if you scale the system. But again, I would be glad to get references to research in both of these example topics and many more.&lt;/p&gt;&#xA;" OwnerUserId="2672" LastEditorUserId="2672" LastEditDate="2020-09-18T14:39:06.190" LastActivityDate="2020-09-18T14:39:06.190" Title="Literature on computational modelling involving neuronal ensemblies" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;&lt;models&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23630" PostTypeId="1" AcceptedAnswerId="23639" CreationDate="2020-09-17T00:44:54.133" Score="2" ViewCount="45" Body="&lt;p&gt;If I am attempting to train a CNN on some image data to perform image classification, but some of the images have pieces of text overlaying them (for the purpose of description to humans), then is it better for the CNN to remove the text? And if so, then how do I remove the text? Furthermore, is it a good idea to use both the images with text overlaying them &lt;em&gt;and&lt;/em&gt; the images with the removed text for training, since it might act as a form of data augmentation?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="2444" LastEditDate="2020-09-20T10:02:09.840" LastActivityDate="2020-09-20T10:02:09.840" Title="Should I remove the text overlaying some images in the dataset before training the CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;&lt;data-preprocessing&gt;&lt;data-augmentation&gt;&lt;image-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23633" PostTypeId="2" ParentId="23222" CreationDate="2020-09-17T11:33:55.797" Score="1" Body="&lt;p&gt;AI in healthcare is already playing a big role in diagnosing the diseases, assisting patients or helping the medical staff in supplying various things or performing actions.&lt;/p&gt;&#xA;&lt;p&gt;The medical imaging helps AI to diagnosis the diseases without help radiologist. And to develop the AI model using medical images to diagnosis various disease need training datasets for machine learning algorithm helping to detect with accuracy.&lt;/p&gt;&#xA;&lt;p&gt;But just like AI, medical imaging is not improving faster making sometimes difficult or more time taking to diagnosis the diseases. Actually, while developing such highly sensitive models, high-quality training data is required and there is lack of such annotated data to train the models.&lt;/p&gt;&#xA;&lt;p&gt;The more training data is feed into the model, it will learn the detection process with more accuracy resulting predicting the faster diagnosis. And to improve the quality and quantity of data, more dedicated and fast labeling process is required.&lt;/p&gt;&#xA;&lt;p&gt;Hence, data annotation companies now use the &lt;strong&gt;AI-assisted labeling&lt;/strong&gt; process to annotate the medical images at faster speed with better accuracy.  Compare to manual annotation, AI companies can get multiple times faster annotated data for machine learning algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Once the training data will start available in large quantity for medical imaging analysis process through AI will also improve. The AI-assisted data annotation process can only help to produce the medical imaging datasets in large quantity for better predictions in healthcare sector.&lt;/p&gt;&#xA;" OwnerUserId="32316" LastActivityDate="2020-09-17T11:33:55.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23637" PostTypeId="1" CreationDate="2020-09-17T17:45:27.433" Score="1" ViewCount="42" Body="&lt;p&gt;When I run a meta-heuristics, like a Genetic Algorithm or a Simulated Annealing, I want to have a termination criterion that stops the algorithms when there is not any significant fitness improvement.&lt;/p&gt;&#xA;&lt;p&gt;What are good methods for that?&lt;/p&gt;&#xA;&lt;p&gt;I tried something like&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$improvement=\frac{fit(Solution_{new})}{fit(Solution_{old})}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$improvement={fit(Solution_{new})}-{fit(Solution_{old})}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Both options don't seem to be good, because as the old solutions get better and newer solutions even if they are good don't improve so much compare to the old.&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-09-18T12:59:36.683" LastActivityDate="2021-02-15T14:07:10.657" Title="What are most commons methods to measure improvement rate in a meta-heuristic?" Tags="&lt;genetic-algorithms&gt;&lt;simulated-annealing&gt;&lt;meta-heuristics&gt;&lt;stopping-conditions&gt;&lt;numerical-algorithms&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23638" PostTypeId="1" CreationDate="2020-09-17T21:02:13.873" Score="3" ViewCount="35" Body="&lt;p&gt;I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I want to encourage these outputs to be sparse through a regularization term in the loss function.&lt;/p&gt;&#xA;&lt;p&gt;One way to do this is to add an L1 regularization term that penalizes the sum of the absolute value of pixel intensities. While this is a good start, is there any penalization that take adjacency and spatial contiguity into account? It doesn't have to be a commonly used constraint/regularization term, but even potential concepts or papers that go in this direction would be extremely helpful. In natural images, sparse pixels tend to form regions or patches as opposed to being dispersed or scattered. Are there ways to encourage regions of contiguous pixels to be sparse as opposed to individual pixels?&lt;/p&gt;&#xA;" OwnerUserId="40781" LastActivityDate="2020-09-17T21:02:13.873" Title="Enforcing sparsity constraints that make use of spatial contiguity" Tags="&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;image-processing&gt;&lt;autoencoders&gt;&lt;regularization&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23639" PostTypeId="2" ParentId="23630" CreationDate="2020-09-18T04:54:36.153" Score="1" Body="&lt;p&gt;Removing the overlayed text might increase accuracy, but you'd need to train a different model to do this, and that is an entirely different task as it is no longer classification, but generation. There are easier ways to augment your data and probably get similar benefits to your accuracy. However, if you would still like to do this, there is a lot of examples you can find by simple searching &amp;quot;Watermark removal machine learning&amp;quot; in google. &lt;a href=&quot;https://github.com/marcbelmont/cnn-watermark-removal&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's&lt;/a&gt; an example I found.&lt;/p&gt;&#xA;&lt;p&gt;Overall, a CNN will be able to look past the overlayed text without issue, and perform classification like it would without the overlayed text. There is the possibility that it actually learns relationships between the overlayed text and the expected output, but that depends on the data, and is likely a harder task then simply identifying features.&lt;/p&gt;&#xA;&lt;p&gt;The only issue you might run into is if the real data this model will be used on is different to the data provided, as in the real word images do not contain overlayed text describing what the image is.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-09-18T04:54:36.153" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23640" PostTypeId="1" CreationDate="2020-09-18T09:18:27.273" Score="1" ViewCount="40" Body="&lt;p&gt;I'm planning an RL project and I have to decide which RL framework do I use if any at all. The project has a highly custom environment, and testing different algorithms will be required to obtain optimal results. Furthermore, it will use a custom neural network, not implemented in the popular TensorFlow/PyTorch ML frameworks. Therefore, the framework should allow for customization with regard to approximation function (1) and the environment (2). The problem is that to my current knowledge, most of the framework allows only to work with a built-in environment. Does anybody know a framework that meets the two conditions (1) and (2)? Or anybody knows a review that contains information about framework in the context of those conditions?&lt;/p&gt;&#xA;" OwnerUserId="31324" LastActivityDate="2021-03-17T20:12:11.257" Title="What framework for a project with a custom environment?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23641" PostTypeId="2" ParentId="23637" CreationDate="2020-09-18T12:55:34.613" Score="0" Body="&lt;p&gt;You can use one of your suggested methods to calculate the relative improvement, but you need also to define a threshold value &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; that determines when a relative improvement is negligible, and so you can terminate the algorithm. To be more concrete, you could terminate the genetic algorithm when, for example, the following condition is met&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|f(x_{t+1}) - f(x_{t})| &amp;lt; \epsilon, \tag{1}\label{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the fitness function&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the iteration number of your iterative algorithm&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; is a solution at iteration &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$|\cdot|$&lt;/span&gt; is the absolute value&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is the threshold value (a hyper-parameter of your algorithm), which is typically a small number (e.g. &lt;span class=&quot;math-container&quot;&gt;$10^{-6}$&lt;/span&gt;), but this also depends on the magnitude of the fitness of the solutions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This stopping criterion in \ref{1} (sometimes known as the &lt;a href=&quot;https://mathworld.wolfram.com/AbsoluteError.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;absolute error&lt;/em&gt;&lt;/a&gt;) is not the only possible one. For example, you can stop the genetic algorithm when it has run for a certain maximum number of iterations. The stopping criterion (or criteria, i.e. you can use more than one stopping criteria) that you choose probably depends on what you want to achieve and this is not just specific to genetic algorithms, but, in general, this applies to any iterative numerical algorithm.&lt;/p&gt;&#xA;&lt;p&gt;The MathWorks article &lt;a href=&quot;https://www.mathworks.com/help/gads/how-the-genetic-algorithm-works.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;How the Genetic Algorithm Works&lt;/a&gt; enumerates several stopping criteria for genetic algorithms.&lt;/p&gt;&#xA;&lt;p&gt;If you want to know more about the topic, maybe take a look at this paper &lt;a href=&quot;https://www.researchgate.net/profile/Martin_Safe/publication/220974459_On_Stopping_Criteria_for_Genetic_Algorithms/links/5720cbef08aead26e7213007.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Stopping Criteria for Genetic Algorithms&lt;/a&gt; (2004) by Martín Safe et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-18T13:07:11.480" LastActivityDate="2020-09-18T13:07:11.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23642" PostTypeId="2" ParentId="23627" CreationDate="2020-09-18T12:57:49.417" Score="1" Body="&lt;p&gt;I think this can only be used for pretraining/some kind of transfer learning. This would be useful if the ratio of real training data to pretraining data is really low. You could then pretrain on digital data, and fine-tune on your UAV data.&lt;/p&gt;&#xA;&lt;p&gt;How much of this is really useful I can't really say, this depends on how close the digital data is to UAV data. If it is significantly different, you are training on a different distribution and sample space than your UAV images, which is pointless.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-09-18T12:57:49.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23643" PostTypeId="5" CreationDate="2020-09-18T13:00:55.983" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-18T13:00:55.983" LastActivityDate="2020-09-18T13:00:55.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23644" PostTypeId="4" CreationDate="2020-09-18T13:00:55.983" Score="0" Body="For questions about stopping conditions (also known as stopping criteria or termination criteria) of numerical algorithms, but, in particular, the ones used in artificial intelligence, such as simulated annealing, genetic algorithms, reinforcement learning algorithms (such as Q-learning), and so on." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-18T13:00:55.983" LastActivityDate="2020-09-18T13:00:55.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23645" PostTypeId="5" CreationDate="2020-09-18T13:04:43.820" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-18T13:04:43.820" LastActivityDate="2020-09-18T13:04:43.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23646" PostTypeId="4" CreationDate="2020-09-18T13:04:43.820" Score="0" Body="For questions about numerical algorithms used in artificial intelligence. Examples of those algorithms are Q-learning, simulated annealing, or any of the genetic algorithms. You should use this tag if you want to ask something about the numerical nature or analysis of those algorithms and you don't have more specific tags." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-18T13:04:43.820" LastActivityDate="2020-09-18T13:04:43.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23647" PostTypeId="2" ParentId="10620" CreationDate="2020-09-18T13:58:13.070" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Given that model-based RL algorithms do not necessarily estimate or compute the transition model or reward function, in the case these are unknown, how can they be computed or estimated (so that they can be used by the model-based algorithms)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A generally reliable approach to creating learned models from interacting with the environment, then using those models internally for planning or explicitly model-based learning, is still something of a holy grail in RL. An agent that can do this across multiple domains might be considered a significant step in autonomous AI. Sutton &amp;amp; Barto write in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (Chapter 17.5):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;More work is needed before planning with learned models can be effective. For example,the learning of the model needs to be selective because the scope of a model strongly affects planning efficiency. If a model focuses on the key consequences of the most important options, then planning can be efficient and rapid, but if a model includes details of unimportant consequences of options that are unlikely to be selected, then planning may be almost useless. Environment models should be constructed judiciously with regard to both their states and dynamics with the goal of optimizing the planning process. The various parts of the model should be continually monitored as to the degree to which they contribute to, or detract from, planning efficiency. &lt;strong&gt;The field has not&#xA;yet addressed this complex of issues or designed model-learning methods that take into account their implications.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;[Emphasis mine]&lt;/p&gt;&#xA;&lt;p&gt;This was written in 2019, so as far as I know still stands as a summary of state-of-the-art. There is ongoing research into this - for instance the paper &lt;a href=&quot;https://arxiv.org/pdf/1809.05214.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model-Based Reinforcement Learning via Meta-Policy Optimization&lt;/a&gt; considers using multiple learned models to assess reliability. I have seen a similar recent paper which also assesses reliability of the learned model and chooses how much it should trust it over a simpler model-free prediction, but cannot recall the name or find it currently.&lt;/p&gt;&#xA;&lt;p&gt;One very simple form of learned model is to memorise transitions that have been experienced already. This is functionally very similar to the experience replay table used in DQN. The &lt;a href=&quot;https://medium.com/@ranko.mosic/online-planning-agent-dyna-q-algorithm-and-dyna-maze-example-sutton-and-barto-2016-7ad84a6dc52b&quot; rel=&quot;nofollow noreferrer&quot;&gt;classic RL algorithm for this kind of model is Dyna-Q&lt;/a&gt;, where the data stored about known transitions is used to perform background planning. In it's simplest form the algorithm is almost indistinguishable from experience replay in DQN. However, this memorised set of transition records &lt;em&gt;is&lt;/em&gt; a learned model, and is used as such in Dyna-Q.&lt;/p&gt;&#xA;&lt;p&gt;The basic Dyna-Q approach creates a tabular model. It does not generalise to predicting outcomes from previously unseen state, action pairs. However, this is relatively easy to fix - simply feed experience so far as training data into a function approximator and you can create a learned model of the environment that attempts to generalise to new states. This idea has been around a long time. Unfortunately, it has problems - planning accuracy is strongly influenced by the accuracy of the model. This applies for both background planning and looking forward from current state. Approximate models like this to date &lt;a href=&quot;https://arxiv.org/abs/1906.05243&quot; rel=&quot;nofollow noreferrer&quot;&gt;typically perform worse than simple replay-based approaches&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This general approach - learn the model statistically from observations - can be refined and may work well if there is any decent prior knowledge that restricts the model. For example if you want to model a physical system that is influenced by current air pressure and local gravity, you could have free parameters for those unknowns starting with some standardised guesses, and then refine the model of dynamics when observations are made, with strong constraints about the form it will take.&lt;/p&gt;&#xA;&lt;p&gt;Similarly in games of chance with hidden state, you may be able to model the unknowns within a broader well-understood model, and use e.g. Bayesian inference to add constraints and best guesses. This is typically what you would do for a POMDP with a &amp;quot;belief state&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Both of the domain-specific approaches in the last two paragraphs can be made to work better than model-free algorithms alone, but they require deep understanding/analysis of the problem being solved by the researcher to set up a parametric model that is both flexible enough to match the environment being learned, but also constrained enough that it cannot become too inaccurate.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-27T13:42:38.780" LastActivityDate="2020-09-27T13:42:38.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23648" PostTypeId="2" ParentId="23547" CreationDate="2020-09-18T14:40:22.397" Score="1" Body="&lt;p&gt;you could also just use a &lt;strong&gt;Task-agnostic CNN&lt;/strong&gt; as an &lt;strong&gt;encoder&lt;/strong&gt; to get extract features like in (1) and then use the output of the last global pooling layer and then feed that as an input to the LSTM layer or any other downstream task. Add another small Neural Network (&lt;strong&gt;projection head&lt;/strong&gt;) after the CNN. And then use contrastive loss on output of this projection head to improve upon the model.&lt;/p&gt;&#xA;&lt;p&gt;(1) Big Self-Supervised Models are Strong Semi-Supervised Learners (Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton ) Link: &lt;a href=&quot;https://arxiv.org/abs/2006.10029&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/2006.10029&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-09-18T14:40:22.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23650" PostTypeId="1" CreationDate="2020-09-18T17:33:54.990" Score="1" ViewCount="27" Body="&lt;p&gt;I've been researching the topic of Cognitive Load Measurement through pupil dilation measurement. All solutions to pupil dilation measurement require some kind of special hardware setup. I was wondering if it would be possible to use AI on a regular webcam record and do those measurements later. If yes, I'd love some pointers to resources of what I need to know to be able to implement it.&lt;/p&gt;&#xA;" OwnerUserId="41069" LastActivityDate="2020-09-18T17:33:54.990" Title="Would it be possible to use AI to measure pupil dilation diameters and fluctuation, on video films on a regular webcam?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23651" PostTypeId="2" ParentId="12023" CreationDate="2020-09-19T03:58:23.480" Score="1" Body="&lt;p&gt;Image Caption Generation is an interesting problem to work on. I think your question was to know if there are any open-source libraries with built-in functions for Image Captioning. You can build Image Caption Generation models using Frameworks like &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Tensorflow&lt;/strong&gt;&lt;/a&gt;, &lt;a href=&quot;https://pytorch.org/docs/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;PyTorch&lt;/strong&gt;&lt;/a&gt;, and &lt;a href=&quot;https://trax-ml.readthedocs.io/en/latest/trax.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Trax&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'd also recommend you to read the following papers:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Show and Tell: A Neural Image Caption Generator. &lt;a href=&quot;https://arxiv.org/abs/1411.4555&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Transfer learning from language models to image caption generators: Better models may not transfer better. &lt;a href=&quot;https://arxiv.org/abs/1901.01216&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Image Captioning with Unseen Objects. &lt;a href=&quot;https://arxiv.org/abs/1908.00047&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Also, here are a couple of blog posts you can read:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Develop a Deep Learning Photo Caption Generator from Scratch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://data-flair.training/blogs/python-based-project-image-caption-generator-cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learn to Build Image Caption Generator with CNN &amp;amp; LSTM&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/image-captioning-with-keras-teaching-computers-to-describe-pictures-c88a46a311b8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Captioning with Keras&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-09-19T03:58:23.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23652" PostTypeId="1" CreationDate="2020-09-19T06:23:36.540" Score="1" ViewCount="14" Body="&lt;p&gt;I am a newbie to Machine Learning field as I am engaging to a personal project that I am trying to use the 6 degree of freedom Inertial Measurement Units(IMUs) measuring the Acceleration acting on 3 axes(x-y-z) and the Angular velocity around the same 3 axis(x-y-z). One sensor generates a set of 6 raw variables of: Acc_x, Acc_y, Acc_z, Gyro_x, Gyro_y, Gyro_z.&lt;/p&gt;&#xA;&lt;p&gt;Initially I have 2 of those sensors that used to be attached on to the arm (one to the part above the elbow and one to the part bellow the elbow) together they spit out a dataset of 12 raw variables that represent a specific movement of the arm, I save them as a the csv file. This is the point where I really get overwhelmed with a huge amount of data that I don't know how to process this kind of data and extract the features to differentiate the gestures.&lt;/p&gt;&#xA;&lt;p&gt;My dataset of the first movement I recorded looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KO1ha.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KO1ha.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I denoted 1 for the first sensor above the elbow and 2 for the sensor below the elbow.&lt;/p&gt;&#xA;&lt;p&gt;Looking forward to hearing the opinions from the experts and seniors on this.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advanced.&lt;/p&gt;&#xA;&lt;p&gt;Let me know if my question is inappropriate and lack of information as it is my first time.&lt;/p&gt;&#xA;" OwnerUserId="41076" LastActivityDate="2020-09-19T06:23:36.540" Title="Multiple Inertia sensors system based for gestures recognition" Tags="&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23653" PostTypeId="1" CreationDate="2020-09-19T12:23:40.640" Score="1" ViewCount="66" Body="&lt;p&gt;For imbalanced datasets (either in the context of computer vision or NLP), from what I learned, it is good to use a weighted log loss. However, in competitions, the people who are in top positions are not using weighted loss functions, but treating the classification problem as a regression problem, and using MSE as the loss function. I want to know which one should I use for imbalanced datasets? Or maybe should I combine both?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;the weighted loss I am talking is:: &#xA;neg_weights=[]&#xA;pos_weights=[]&#xA;for i in tqdm(range(5)):##range(num_classes)&#xA;    neg_weights.append(np.sum(y_train[:,i],axis=0)/y_train.shape[0])&#xA;    pos_weights.append(np.sum(1-y_train[:,i],axis=0)/y_train.shape[0])&#xA;def customloss(y_true,y_pred):&#xA;    y_true=tf.cast(y_true,dtype=y_pred.dtype)&#xA;    loss=0.0&#xA;    loss_pos=0.0&#xA;    loss_neg=0.0&#xA;    for i in range(5):&#xA;        loss_pos+=-1*(K.mean(pos_weights[i]*y_true[:,i]*K.log(y_pred[:,i]+1e-8)))&#xA;        loss_neg+=-1*(K.mean(neg_weights[i]*(1-y_true[:,i])*K.log(1-y_pred[:,i]+1e-8)))&#xA;    loss=loss_pos+loss_neg&#xA;    return loss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the competition I was talking about is &lt;a href=&quot;https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/109594&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/109594&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38737" LastEditorUserId="38737" LastEditDate="2020-09-20T11:25:56.927" LastActivityDate="2020-09-20T11:25:56.927" Title="Which loss function to choose for imbalanced datasets?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;loss-functions&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23654" PostTypeId="1" AcceptedAnswerId="23663" CreationDate="2020-09-19T12:34:47.970" Score="2" ViewCount="29" Body="&lt;p&gt;My doubt is like this :&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Suppose we have an MLP. In an MLP, as per the backprop algorithm (back-propagation algorithm), the correction applied to each weight is :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ w_{ij} := -\eta\frac{\partial E}{\partial w_{ij}}$$&lt;/span&gt;&#xA;(&lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt; = learning rate, &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; = error in the output, &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$i^{\text{th}}$&lt;/span&gt; neuron in the &lt;span class=&quot;math-container&quot;&gt;$j^{\text{th}}$&lt;/span&gt; row or layer)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now, if we put an extra factor in the correction as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ w_{ij} := -k\eta  \frac{\partial E}{\partial w_{ij}}$$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; denotes the number of iterations at the time of correction)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;how much will that factor affect the learning of the network ? Will it affect the convergence of the  network such that it takes time to fit to the data ?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;NB : I am only asking this as a doubt. I haven't tried any ML projects recently, so this is not related to anything I am doing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="40583" LastActivityDate="2020-09-20T03:21:11.897" Title="How much can an inclusion of the number of iterations have on the training of an MLP?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;convergence&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23655" PostTypeId="1" CreationDate="2020-09-19T12:50:14.013" Score="4" ViewCount="48" Body="&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Occam%27s_razor&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia article on Occam's razor&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Another contentious aspect of the razor is that a theory can become more complex in terms of its structure (or syntax), while its ontology (or semantics) becomes simpler, or vice versa. Quine, in a discussion on definition, referred to these two perspectives as &amp;quot;economy of practical expression&amp;quot; and &amp;quot;economy in grammar and vocabulary&amp;quot;, respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Within AI, are there some examples that can help understand where the above occurs or where this may be of importance?&lt;/p&gt;&#xA;&lt;p&gt;I have tried to form a possible example:&lt;/p&gt;&#xA;&lt;p&gt;Suppose an AI software program has a high cyclomatic complexity. This would mean it has high syntactical complexity (is this correct?). Further, suppose the program operates at a high abstraction level then its semantic complexity could be low (again, is this correct?).&lt;/p&gt;&#xA;" OwnerUserId="37533" LastEditorUserId="2444" LastEditDate="2020-09-19T20:49:45.970" LastActivityDate="2020-09-19T20:49:45.970" Title="What are examples of AI theories that are structurally complex but ontologically simple?" Tags="&lt;machine-learning&gt;&lt;semantics&gt;&lt;ontology&gt;&lt;occam-razor&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23656" PostTypeId="1" CreationDate="2020-09-19T20:24:03.210" Score="0" ViewCount="82" Body="&lt;p&gt;I cannot find reliable sources but someone says it is 40 moves and someone else says it is 50+ moves. I read their papers and they use value function (NN) and policy function to trim the tree, so more layers can be searched while spending less time searching less different positions.&lt;/p&gt;&#xA;&lt;p&gt;My question is, is the search depth a fixed preset parameter? If so, approximately how much is it back to 2016 (AlphaGo) and 2018 (AlphaGo Zero)?&lt;/p&gt;&#xA;" OwnerUserId="38299" LastEditorUserId="2444" LastEditDate="2020-09-19T20:46:01.043" LastActivityDate="2020-09-22T16:50:42.370" Title="What is the search depth of AlphaGo and AlphaGo Zero?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;alphago-zero&gt;&lt;alphago&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23657" PostTypeId="5" CreationDate="2020-09-19T20:50:08.183" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-19T20:50:08.183" LastActivityDate="2020-09-19T20:50:08.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23658" PostTypeId="4" CreationDate="2020-09-19T20:50:08.183" Score="0" Body="For questions about the Occam's razor in the context of artificial intelligence and machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-19T20:50:08.183" LastActivityDate="2020-09-19T20:50:08.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23659" PostTypeId="2" ParentId="20523" CreationDate="2020-09-19T20:54:42.243" Score="0" Body="&lt;p&gt;It depends on your image size and the size of the compression you want! Usually deep learning algorithms are not so fast as why they run on GPU, and we have highly optimized frameworks like TensorFlow! Something I can say for sure is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compressing video using autoencoders means compressing each frame one by one! However, video compressions usually contain the calculation of the deference of every frame with the previous frame. This means the compressing video is much more time consuming than compressing just a single image.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The encoder is half part of the autoencoder, so the compression is faster than training the whole autoencoder.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use GPU! It really makes much different!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try Google Colab! You can choose between CPU and GPU and then make a decision.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="35757" LastEditorUserId="35757" LastEditDate="2020-09-22T06:30:28.350" LastActivityDate="2020-09-22T06:30:28.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23660" PostTypeId="2" ParentId="9333" CreationDate="2020-09-19T21:12:30.643" Score="0" Body="&lt;p&gt;You can also check &lt;a href=&quot;https://www.researchgate.net/publication/328820423_Deep_Active_Learning_for_Text_Classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which discusses the use of LSTM and GRU with Active Learning and word embeddings (word2vec).&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-09-19T21:12:30.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23661" PostTypeId="2" ParentId="23112" CreationDate="2020-09-19T21:27:24.750" Score="0" Body="&lt;p&gt;You are right!&lt;/p&gt;&#xA;&lt;p&gt;1- the number of hidden layers shouldn't be too high! Because of the gradient descent when the number of layers is too large, the gradient effect on the first layers become too small! This is why the Resnet model was introduced.&lt;/p&gt;&#xA;&lt;p&gt;2- the number of hidden layers shouldn't be too small to extracts good features. It's proved that in CNN networks the first layers extract very simple elements like lines and curves but last layers extracts more complex features.&lt;/p&gt;&#xA;&lt;p&gt;3- number of hidden units is a hyper-parameters and usually you should find it by testing or based on your background knowledge.&lt;/p&gt;&#xA;&lt;p&gt;But what can you do at all?&#xA;As you can tests different parameters and compare their results there is some other options! One option is grid search you can check this tutorial &lt;a href=&quot;https://towardsdatascience.com/grid-search-for-model-tuning-3319b259367e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/grid-search-for-model-tuning-3319b259367e&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35757" LastActivityDate="2020-09-19T21:27:24.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23662" PostTypeId="1" CreationDate="2020-09-20T00:48:03.453" Score="2" ViewCount="25" Body="&lt;p&gt;Why is there no upper confidence bound algorithm for linear stochastic bandits that uses lasso regression in the case that the regression parameters are sparse in the features?&lt;/p&gt;&#xA;&lt;p&gt;In particular, I don't understand what is hard about lasso regression that makes it hard to be used in a UCB type algorithm whereas there is a lot of work on ridge regression based UCB algorithms see e.g. &lt;a href=&quot;https://papers.nips.cc/paper/4417-improved-algorithms-for-linear-stochastic-bandits.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yadkori et al&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I looked up some works e.g. &lt;a href=&quot;http://web.stanford.edu/%7Ebayati/papers/lassoBandit.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bastani and Bayati&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1907.11362v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kim and Paik&lt;/a&gt; but they all do not a UCB-type algorithm, instead, they propose forced or probabilistic sampling to satisfy the compatibility condition (see Lemma EC.6. of &lt;a href=&quot;http://web.stanford.edu/%7Ebayati/papers/lassoBandit.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bastani and Bayati&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="27277" LastEditorUserId="27277" LastEditDate="2020-09-20T01:27:28.580" LastActivityDate="2020-09-20T01:27:28.580" Title="Is there a UCB type algorithm for linear stochastic bandit with lasso regression?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;linear-regression&gt;&lt;multi-armed-bandit&gt;&lt;contextual-bandit&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23663" PostTypeId="2" ParentId="23654" CreationDate="2020-09-20T03:21:11.897" Score="2" Body="&lt;p&gt;If anything, you want the learning rate to &lt;em&gt;decrease&lt;/em&gt; as the number of iterations increases.&lt;/p&gt;&#xA;&lt;p&gt;When you're looking for a good spot and you're clueless, take large steps. When you've found a pretty good spot, take small steps, so you don't end up far away.&lt;/p&gt;&#xA;&lt;p&gt;In other fields of machine learning, there are studies of how the learning rate should scale. For example, in traditional reinforcement learning methods, if &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; is the learning rate at step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, then we want to have the following two criteria, to make sure we get convergence to the optimal policy:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{i=0}^{\infty} \alpha_i = \infty$&lt;/span&gt;. This makes sure that, no matter how bad our initial experience was, we can eventually forget it and replace it with better information.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{i=0}^{\infty} \alpha_i^2 &amp;lt; \infty$&lt;/span&gt;. This guarantees eventual convergence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;A typical choice here is &lt;span class=&quot;math-container&quot;&gt;$\alpha_i = \frac{1}{1+i}$&lt;/span&gt;, which fits both criteria.&lt;/p&gt;&#xA;&lt;p&gt;I am unaware of similar criteria for MLPs, but if you're going to modify the step sizes, I would follow a similar approach. Make the step sizes decrease, but not too fast.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-20T03:21:11.897" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23664" PostTypeId="5" CreationDate="2020-09-20T09:59:26.170" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-20T09:59:26.170" LastActivityDate="2020-09-20T09:59:26.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23665" PostTypeId="4" CreationDate="2020-09-20T09:59:26.170" Score="0" Body="For questions about image classification, aka object classification, i.e. given an image containing one main object, we want to classify it into 1 of N classes." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-20T09:59:26.170" LastActivityDate="2020-09-20T09:59:26.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23666" PostTypeId="1" CreationDate="2020-09-20T10:21:05.780" Score="2" ViewCount="69" Body="&lt;p&gt;I am trying to create a simple Deep Q-Network with 2d convolutional layers.&lt;/p&gt;&#xA;&lt;p&gt;I can't figure out what I am doing wrong, and the only thing I can see that doesn't seem right is when I get the model prediction for a state after the optimizer step it doesn’t seem to get closer to the target.&lt;/p&gt;&#xA;&lt;p&gt;I am using pixels from pong in OpenAI's gym with single-channel 90x90 images, a batch size of 32, and replay memory.&lt;/p&gt;&#xA;&lt;p&gt;As an example, if I try with a batch size of 1, and try running &lt;code&gt;self(states)&lt;/code&gt; again right after the optimizer step the output is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;current_q_values -&amp;gt; -0.16351485  0.29163417  0.11192469 -0.08969332  0.11081569  0.37215832&#xA;q_target -&amp;gt;         -0.16351485  0.5336551   0.11192469 -0.08969332  0.11081569  0.37215832&#xA;self(states) -&amp;gt;     -0.8427617   0.6415581   0.44988257 -0.43897176  0.8693738   0.40007943&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does this look as what would be expected for a single step?&lt;/p&gt;&#xA;&lt;p&gt;The network with loss and optimizer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    self.in_layer = Conv2d(channels, 32, 8)&#xA;    self.hidden_conv_1 = Conv2d(32, 64, 4)&#xA;    self.hidden_conv_2 = Conv2d(64, 128, 3)&#xA;    self.hidden_fc1 = Linear(128 * 78 * 78, 64)&#xA;    self.hidden_fc2 = Linear(64, 32)&#xA;    self.output = Linear(32, action_space)&#xA;&#xA;    self.loss = torch.nn.MSELoss()&#xA;    self.optimizer = torch.optim.Adam(&#xA;        self.parameters(), lr=learning_rate) # lr is 0.001&#xA;&#xA;def forward(self, state):&#xA;    in_out = fn.relu(self.in_layer(state))&#xA;    in_out = fn.relu(self.hidden_conv_1(in_out))&#xA;    in_out = fn.relu(self.hidden_conv_2(in_out))&#xA;    in_out = in_out.view(-1, 128 * 78 * 78)&#xA;    in_out = fn.relu(self.hidden_fc1(in_out))&#xA;    in_out = fn.relu(self.hidden_fc2(in_out))&#xA;    return self.output(in_out)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then the learning block:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        self.optimizer.zero_grad()&#xA;&#xA;        sample = self.sample(self.batch_size)&#xA;        states = torch.stack([i[0] for i in sample])&#xA;        actions = torch.tensor([i[1] for i in sample], device=device)&#xA;        rewards = torch.tensor([i[2] for i in sample], dtype=torch.float32, device=device)&#xA;        next_states = torch.stack([i[3] for i in sample])&#xA;        dones = torch.tensor([i[4] for i in sample], dtype=torch.uint8, device=device)&#xA;&#xA;        current_q_vals = self(states)&#xA;        next_q_vals = self(next_states)&#xA;        q_target = current_q_vals.clone()&#xA;        q_target[torch.arange(states.size()[0]), actions] = rewards + (self.gamma * next_q_vals.max(dim=1)[0]) * (~dones).float()&#xA;&#xA;        loss = fn.smooth_l1_loss(current_q_vals, q_target)&#xA;        loss.backward()&#xA;&#xA;        self.optimizer.step()&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41097" LastEditorUserId="2444" LastEditDate="2021-03-18T10:59:30.983" LastActivityDate="2021-03-28T18:32:55.563" Title="DQN not learning and step not stepping towards target" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;pytorch&gt;&lt;convolutional-layers&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23667" PostTypeId="1" CreationDate="2020-09-20T14:18:54.747" Score="0" ViewCount="171" Body="&lt;p&gt;I know that if you use an ReLU activation function at a node in the neural network, the output of that node will be non-negative. I am wondering if it is possible to have a negative output in the final layer, provided that you do not use any activation functions in the final layer, and all the activation functions in the previous hidden layers are ReLU?&lt;/p&gt;&#xA;" OwnerUserId="20358" LastActivityDate="2020-09-21T18:19:51.063" Title="Is it possible to have a negative output using only ReLU activation functions, but not in the final layer?" Tags="&lt;neural-networks&gt;&lt;relu&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23668" PostTypeId="2" ParentId="23667" CreationDate="2020-09-20T14:45:18.817" Score="2" Body="&lt;p&gt;Yes, if there's no activation function in the last layer, the weights could simply be negative there, so the network would multiply a positive value with a negative weight, therefore outputting a negative value.&lt;/p&gt;&#xA;&lt;p&gt;There is still an activation function, but it is the identity.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-09-20T14:45:18.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23669" PostTypeId="1" AcceptedAnswerId="24074" CreationDate="2020-09-20T14:53:35.783" Score="2" ViewCount="110" Body="&lt;p&gt;The understanding I have is that they somehow adjust the objective to make it easier to meet, without changing the reward function.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;... the observed proxy reward function is the approximate solution to a reward design problem&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;(source: &lt;a href=&quot;https://arxiv.org/pdf/1711.02827.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inverse Reward Design&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;But I have trouble getting how they fit the overall reward objective and got confused by some examples of them.&#xA;I had the idea of them being small reward functions (as in the case of solving for sparse rewards) eventually leading to the main goal. But the statement below, from &lt;a href=&quot;https://ai-alignment.com/thoughts-on-reward-engineering-82b193ec03f6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;, made me question that.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Typical examples of proxy reward functions include “partial credit” for behaviors that look promising; artificially high discount rates and careful reward shaping;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are they, and how would one go about identifying and integrating proxy rewards in an RL problem?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the examples above, how would high discount rates form a proxy reward?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm also curious about how they are used as a source of multiple rewards&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="2444" LastEditDate="2020-10-08T11:48:04.640" LastActivityDate="2020-11-13T17:03:05.323" Title="What are proxy reward functions?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;papers&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23670" PostTypeId="1" CreationDate="2020-09-20T15:07:46.743" Score="1" ViewCount="74" Body="&lt;p&gt;I have been studying about auto-encoders and variational auto-encoders. I would like to know how many variants of VAEs are there today.&lt;/p&gt;&#xA;&lt;p&gt;If there are many variants, can they be used for feature extraction for complex reinforcement learning tasks like self-driving cars?&lt;/p&gt;&#xA;" OwnerUserId="41103" LastEditorUserId="2444" LastEditDate="2020-09-20T17:53:42.450" LastActivityDate="2020-12-22T18:01:55.163" Title="How many types of variational auto-encoders are there?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;variational-autoencoder&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23671" PostTypeId="2" ParentId="23618" CreationDate="2020-09-20T15:12:50.130" Score="1" Body="&lt;p&gt;I will try to give some sense to this question.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Artificial general intelligence (AGI) is the hypothetical[1] intelligence of a machine that has the capacity to understand or learn any intellectual task that a human being can. It is a primary goal of some artificial intelligence research and a common topic in science fiction and futures studies. AGI can also be referred to as strong AI,[2][3][4] full AI,[5] or general intelligent action.[6] Some academic sources reserve the term &amp;quot;strong AI&amp;quot; for machines that can experience consciousness.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These are the first sentences on AGI on wikipedia (&lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_general_intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;), and the softest limit there is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[learn] any intellectual task that a human being can.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Even taking only this, it would mean that any AGI has infinite economic value. As soon as there is something that can learn any human task and has the speed of current GPUs/CPUs it could potentially immediatly replace every human in every task. There are certainly enough computers with CPUs and GPUs out there.&lt;/p&gt;&#xA;&lt;p&gt;This question is still a little flawed because you not only have to constrain the definition of AGI but also how it would actually be implemented.&lt;/p&gt;&#xA;" OwnerUserId="38328" LastActivityDate="2020-09-20T15:12:50.130" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23672" PostTypeId="1" AcceptedAnswerId="23689" CreationDate="2020-09-20T16:30:59.723" Score="3" ViewCount="92" Body="&lt;p&gt;There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer learning from AlexNet to GoogLeNet, etc.? Or even just from a &amp;quot;conventional&amp;quot; CNN to one of these other architectures, or the other way around? Is this possible in general?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: My understanding is that &lt;em&gt;all&lt;/em&gt; machine learning models have the ability to perform transfer learning. If this is true, then I guess the question is, as I said, whether we can transfer between two &lt;em&gt;different&lt;/em&gt; CNN architectures – for instance, what was learned by a conventional CNN to a different CNN architecture.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="16521" LastEditDate="2020-09-21T15:27:14.373" LastActivityDate="2021-04-18T20:53:54.790" Title="Can we apply transfer learning between any two different CNN architectures?" Tags="&lt;convolutional-neural-networks&gt;&lt;transfer-learning&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23676" PostTypeId="1" AcceptedAnswerId="23679" CreationDate="2020-09-21T07:38:01.377" Score="4" ViewCount="281" Body="&lt;p&gt;While exploration is an integral part of reinforcement learning (RL), it does not pertain to supervised learning (SL) since the latter is already provided with the data set from the start.&lt;/p&gt;&#xA;&lt;p&gt;That said, can't hyperparameter optimization (HO) in SL be considered as exploration? The more I think about this the more I'm confused as to what exploration really means. If it means exploring the environment in RL and exploring the model configurations via HO in SL, isn't its end goal &amp;quot;mathematically&amp;quot; identical in both cases?&lt;/p&gt;&#xA;" OwnerUserId="30959" LastEditorUserId="2444" LastEditDate="2020-09-21T10:53:25.963" LastActivityDate="2020-10-08T23:12:51.177" Title="What is the meaning of &quot;exploration&quot; in reinforcement and supervised learning?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;supervised-learning&gt;&lt;hyperparameter-optimization&gt;&lt;exploration-exploitation-tradeoff&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23678" PostTypeId="2" ParentId="23656" CreationDate="2020-09-21T10:28:19.150" Score="1" Body="&lt;p&gt;For easier visualization, I recommend this video:&#xA;&lt;a href=&quot;https://twitter.com/i/status/1257053365424578565&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/i/status/1257053365424578565&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The more detailed article about GO algorithms:&#xA;&lt;a href=&quot;https://deepmind.com/blog/article/alphago-zero-starting-scratch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/blog/article/alphago-zero-starting-scratch&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;With its breadth of &lt;span class=&quot;math-container&quot;&gt;$250$&lt;/span&gt; possible moves each turn (go is played on a &lt;span class=&quot;math-container&quot;&gt;$19$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$19$&lt;/span&gt; board, compared to the much smaller &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; chess field) and a typical game depth of &lt;span class=&quot;math-container&quot;&gt;$150$&lt;/span&gt; moves, there are about &lt;span class=&quot;math-container&quot;&gt;$250^{150}$&lt;/span&gt;, or&#xA;&lt;span class=&quot;math-container&quot;&gt;$10^{360}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;After &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; moves in go, it's &lt;span class=&quot;math-container&quot;&gt;$130000$&lt;/span&gt; possible combinations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Decision tree pruning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bR68K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bR68K.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32352" LastEditorUserId="2444" LastEditDate="2020-09-22T16:50:42.370" LastActivityDate="2020-09-22T16:50:42.370" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23679" PostTypeId="2" ParentId="23676" CreationDate="2020-09-21T10:41:10.660" Score="5" Body="&lt;p&gt;In reinforcement learning, &lt;strong&gt;exploration&lt;/strong&gt; has a specific meaning, which is in contrast with the meaning of &lt;strong&gt;exploitation&lt;/strong&gt;, hence the so-called &lt;strong&gt;exploration-exploitation dilemma&lt;/strong&gt; (or trade-off). You &lt;strong&gt;explore&lt;/strong&gt; when you decide to visit states that you have not yet visited or to take actions you have not yet taken. On the other hand, you &lt;strong&gt;exploit&lt;/strong&gt; when you decide to take actions that you have already taken and you know how much reward you can get. It's like in life: maybe you like cereals &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, but you never tried cereals &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, which could be tastier. What are you going to do: continue to eat cereals &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (exploitation) or maybe try once &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (exploration)? Maybe cereals &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are as tasty as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, but, in the long run, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are healthier than &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;More concretely, recall that, in RL, the goal is to collect as much reward as you can. Let's suppose that you are in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and, in the past, when you were in that state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, you had already taken the action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;, but not the other actions &lt;span class=&quot;math-container&quot;&gt;$a_2, a_3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_4$&lt;/span&gt;. The last time you took action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;, you received a reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, which is a good thing, but what if you take action &lt;span class=&quot;math-container&quot;&gt;$a_2, a_3$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_4$&lt;/span&gt;? Maybe you will get a higher reward, for example, &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt;, which is better. So, you need to decide whether to choose again action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; (i.e. whether to &lt;strong&gt;exploit&lt;/strong&gt; your current knowledge) or try another action that may lead to a higher (or smaller) reward (i.e. you &lt;strong&gt;explore&lt;/strong&gt; the environment). The problem with exploration is that you don't know what's going to happen, i.e. you are risking if you already get a nice amount of reward if you take an action already taken, but sometimes exploration is the best thing to do, given that maybe the actions you have taken so far have not led to any good reward.&lt;/p&gt;&#xA;&lt;p&gt;In hyper-parameter optimization, you do not need to collect any reward, unless you formulate your problem as a reinforcement learning problem (&lt;a href=&quot;https://arxiv.org/pdf/1906.11527.pdf&quot; rel=&quot;noreferrer&quot;&gt;which is possible&lt;/a&gt;). The goal is to find the best set of hyper-parameters (e.g. the number of layers and neurons in each layer of the neural network) that performs well, typically, on the validation dataset. Once you have found a set of hyper-parameters, you usually do not talk about exploiting it, in the sense that you will not continually receive any type of reward if you use that set of hyper-parameters, unless you conceptually decide that this is the case, i.e., whenever you use that set of hyper-parameters you are exploiting that model to get good performance on the test sets that you have. You could also say that when you are searching for new sets of hyper-parameters you are exploring the search space, but, again, the distinction between exploitation and exploitation, in this case, is typically not made, but you can well talk about it.&lt;/p&gt;&#xA;&lt;p&gt;It makes sense to talk about the exploitation-exploration trade-off when there is stochasticity involved, but in the case of the hyper-parameter optimization there may not be such a stochasticity, but it's usually a deterministic search, which you can, if you like, call exploration.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-21T11:02:13.463" LastActivityDate="2020-09-21T11:02:13.463" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23681" PostTypeId="1" CreationDate="2020-09-21T13:03:10.537" Score="0" ViewCount="28" Body="&lt;p&gt;I read about the spatial pyramid pooling concept, it's really cool! Now, my doubt is how to find the number of layers to use, and in each layer what should be the grid sizes when using spp. But, I haven't seen people much implementing these thing even when they have very size-varying images in their data. Can someone explain why? and what are its drawbacks?&lt;/p&gt;&#xA;" OwnerUserId="38737" LastEditorUserId="38737" LastEditDate="2020-09-21T13:40:18.630" LastActivityDate="2020-09-21T13:40:18.630" Title="How to use SPP-net and what are its drawbacks?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23682" PostTypeId="1" CreationDate="2020-09-21T16:02:43.007" Score="0" ViewCount="22" Body="&lt;p&gt;I used OCR to extract text from an image, but there are some spelling mistakes in it :&lt;/p&gt;&#xA;&lt;p&gt;The text is as follows :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'gaRBOMATED WATER\n\nSFMEETENED CARBONATED 6\nBSREDERTS: CARBONATED WATER,\nSUGAR. ACIOITY REGULATOR (338),\n\nCFFENE. CONTAINS PERMITTED NATURAL\nCOLOUR (1506) AMD ADDED FLAVOURS QUcTURAL,\nSATIRE: OENTICAL AND ARTIFICIAL PLIVOUREE\n\nCOLA\nl 1187.3 PIRANGUT, TAL. MULSHI,\nGBST. PUME 612111, MAHARASHTRA.\nHELPLINE: 1800- 180-2653\ntet indishetptine@cocs-cola.com\nAUTHORITY OF THE COCA-COLA\n‘COCA-COLA PLAZA, ATLANTA, GA 36313, USA\nme DATE OF MANUFACTURE. BATCH NO. &amp;amp;\nLP CNL. OF ae TAXES}:\nSE BOTTOM OF CAN.\n\nTST Fone Sor MOTHS FROM\nWe, RE WHEN STORED ft.\n\nY PLACE.\nChe coca conn\nnee\n\n| BRA License uo:\n‘ eS wo:\n\n \n\x0c'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to know if there are some NLP models/libraries that I can use to correct spelling mistakes(like correcting &lt;code&gt;gaRBOMATED&lt;/code&gt; to &lt;code&gt;CARBONATED&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="16881" LastActivityDate="2020-09-21T16:02:43.007" Title="What are some good models to use for spelling corrections?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23683" PostTypeId="2" ParentId="23221" CreationDate="2020-09-21T16:04:08.137" Score="4" Body="&lt;h3&gt;What is a transformer?&lt;/h3&gt;&#xA;&lt;p&gt;The original &lt;strong&gt;transformer&lt;/strong&gt;, proposed in the paper &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is all you need&lt;/a&gt; (2017), is an &lt;strong&gt;encoder-decoder&lt;/strong&gt;-based neural network that is mainly characterized by the use of the so-called &lt;a href=&quot;https://arxiv.org/pdf/1902.02181.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;attention&lt;/strong&gt;&lt;/a&gt; (i.e. a mechanism that determines the importance of words to other words in a sentence or which words are more likely to come together) and the &lt;strong&gt;non&lt;/strong&gt;-use of recurrent connections (or recurrent neural networks) to solve tasks that involve sequences (or sentences), even though RNN-based systems were becoming the standard practice to solve natural language processing (NLP) or understanding (NLU) tasks. Hence the name of the paper &amp;quot;&lt;strong&gt;Attention&lt;/strong&gt; is all you need&amp;quot;, i.e. you only need attention and you don't need recurrent connections to solve NLP tasks.&lt;/p&gt;&#xA;&lt;p&gt;Both the encoder-decoder architecture and the attention mechanism are not novel proposals. In fact, previous neural network architectures to solve many NLP tasks, such as machine translation, had already used these mechanisms (for example, take a look at &lt;a href=&quot;https://arxiv.org/pdf/1409.0473.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;).  The novelty of the transformer and &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this cited paper&lt;/a&gt; is that it shows that we can simply use attention to solve tasks that involve sequences (such as machine translation) and we do not need recurrent connections, which is an advantage, given that recurrent connections can hinder the parallelization of the training process.&lt;/p&gt;&#xA;&lt;p&gt;The original transformed architecture is depicted in figure 1 of &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the cited paper&lt;/a&gt;. Both the encoder and decoder are composed of&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;attention modules&lt;/li&gt;&#xA;&lt;li&gt;feed-forward (or fully connected) layers&lt;/li&gt;&#xA;&lt;li&gt;residual (or skip) connections&lt;/li&gt;&#xA;&lt;li&gt;normalization layers&lt;/li&gt;&#xA;&lt;li&gt;dropout&lt;/li&gt;&#xA;&lt;li&gt;label smoothing&lt;/li&gt;&#xA;&lt;li&gt;embedding layers&lt;/li&gt;&#xA;&lt;li&gt;positional encoding&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The decoder part is also composed of a linear layer followed by a softmax to solve the specific NLP task (for example, predict the next word in a sentence).&lt;/p&gt;&#xA;&lt;h3&gt;What is BERT?&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;BERT&lt;/strong&gt;&lt;/a&gt; stands for &lt;em&gt;Bidirectional &lt;strong&gt;Encoder Representations&lt;/strong&gt; from&#xA;&lt;strong&gt;Transformers&lt;/strong&gt;&lt;/em&gt;, so, as the name suggests, it is a way of learning &lt;strong&gt;representations&lt;/strong&gt; of a language that uses a &lt;strong&gt;transformer&lt;/strong&gt;, specifically, the &lt;strong&gt;encoder&lt;/strong&gt; part of the transformer.&lt;/p&gt;&#xA;&lt;h3&gt;What is the difference between the transformer and BERT?&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;BERT is a language model, i.e. it represents the statistical relationships of the words in a language, i.e. which words are more likely to come after another word and stuff like that. Hence the part &lt;strong&gt;Representations&lt;/strong&gt; in its name, &lt;em&gt;Bidirectional Encoder &lt;strong&gt;Representations&lt;/strong&gt; from&#xA;Transformers&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;BERT can be trained in an unsupervised way for &lt;em&gt;representation learning&lt;/em&gt;, and then we can &lt;em&gt;fine-tune&lt;/em&gt; BERT on the so-called downstream tasks in a supervised fashion (i.e. transfer learning). There are pre-trained versions of BERT that can be already fine-tuned (e.g. &lt;a href=&quot;https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;) and used to solve your specific supervised learning task. You can play with &lt;a href=&quot;https://www.tensorflow.org/official_models/fine_tuning_bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;this TensorFlow tutorial&lt;/a&gt; to use a pre-trained BERT model.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, the original transformed was not originally conceived to be a language model, but to solve sequence transduction tasks (i.e. converting one sequence to another, such as machine translation) without recurrent connections (or convolutions) but only attention.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;BERT is only an encoder, while the &lt;em&gt;original&lt;/em&gt; transformer is composed of an encoder and decoder. Given that BERT uses an encoder that is very similar to the original encoder of the transformer, we can say that BERT is a transformer-based model. So, BERT does not use recurrent connections, but only attention and feed-forward layers. There are other transformed-based neural networks that use only the decoder part of the transformer, for example, &lt;a href=&quot;https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the GPT model&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;BERT uses different hyper-parameters than the ones used in &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is all you need&lt;/a&gt; to achieve the best performance. For example, it uses 12 and 16 &amp;quot;attention heads&amp;quot; (please, read the &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer paper&lt;/a&gt; to know more about these &amp;quot;attention heads&amp;quot;) rather than 8 (although in the original transformer paper the authors experimented with a different number of heads).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;BERT also uses &lt;em&gt;segment embeddings&lt;/em&gt;, while the original transformer only uses word embeddings and positional encodings.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There are probably other small differences that I missed, but, after having read the paper &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is all you need&lt;/a&gt; and quickly read some parts of &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the BERT paper&lt;/a&gt;, these seem to be the main differences.&lt;/p&gt;&#xA;&lt;h3&gt;When to use BERT and the transformer?&lt;/h3&gt;&#xA;&lt;p&gt;Although I never used them, I would say that you want to use BERT whenever you want to solve an NLP task in a supervised fashion, but your labeled training dataset is not big enough to achieve good performance. In that case, you start with a pre-trained BERT model, then fine-tune it with your small labeled dataset. You probably need to add specific layers to BERT to solve your task.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-21T21:29:16.003" LastActivityDate="2020-09-21T21:29:16.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23684" PostTypeId="2" ParentId="23449" CreationDate="2020-09-21T17:37:56.520" Score="5" Body="&lt;p&gt;I will be starting my PhD in natural language processing in a few days and this is very similar to my proposed topic. It's an open problem that ties NLP and AI into philosophy of science and epistemology and is, I think, extremely interesting. I say all this to drive home the point that this is &lt;em&gt;not&lt;/em&gt; a simple problem.&lt;/p&gt;&#xA;&lt;p&gt;Two major theoretical concerns come to my mind:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is a &amp;quot;fact&amp;quot;? Is it a universal truth, if there is such a thing? Or is it a generally accepted theory, and if so how do you measure acceptance? That is, accepted by whom, where, when?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any linguistic markers of opinions vs. facts? Only in rare cases, such as when the speaker prefaces their statement with something like &amp;quot;I believe&amp;quot;. In most cases, I think, opinions will be stated linguistically similarly to facts. For example, compare &amp;quot;Cats are felines.&amp;quot; (a &amp;quot;fact&amp;quot;) with &amp;quot;Cats are aliens.&amp;quot; (an opinion some may hold). They have the exact same syntactic structure. The difference here is deeply semantic, and probably relates to the speaker's intention. I'd venture that often people state their opinions with the intention of communicating a &amp;quot;fact&amp;quot;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Some more practical concerns are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Information extraction (also called relationship extraction, text mining, etc.), which for the most part assumes that the &amp;quot;facts&amp;quot; given in the labeled datasets are correct, is far from a solved problem. E.g. the &lt;a href=&quot;https://paperswithcode.com/sota/relation-extraction-on-2010-i2b2-va&quot; rel=&quot;noreferrer&quot;&gt;state of the art model&lt;/a&gt; developed for a task released in 2010 has an F1 of only 76! What you propose adds significant uncertainty to these types of tasks.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I suspect that even if you were able to compile a dataset of facts and opinions with corresponding labels you would encounter a number of modeling problems. Given the linguistic similarity between the statements of facts and opinions, I'd guess that your model will simply memorize the dataset, making it generalize poorly to your test set. Either that or it would would pick up on random, hidden correlations in the data to solve the problem (neural nets are really good at this), perhaps generalizing to the test set, but failing to apply to any other data.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fact vs. opinion is something that is embedded in a cultural milieu, so a model would, I think, need access to some proxy for what is culturally accepted in order to make this distinction, perhaps a via knowledge base. This may be feasible for limited, highly curated domains (e.g. biomedicine), but there is currently nothing suitable for a general-purpose fact finder.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;tldr&lt;/strong&gt;: No, it is not enough to simply create a dataset of facts vs. opinions. This problem poses major theoretical concerns related to epistemology, linguistics, and cognitive science. Additionally, there are more mundane (but non-trivial!) modeling issues to consider. @Sceptre is right that it will be impossible to start this without knowledge of AI/ML/NLP, especially a rather deep knowledge of what current AI systems are &lt;em&gt;really&lt;/em&gt; capable of.&lt;/p&gt;&#xA;" OwnerUserId="37972" LastActivityDate="2020-09-21T17:37:56.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23685" PostTypeId="2" ParentId="23676" CreationDate="2020-09-21T17:40:20.837" Score="-3" Body="&lt;p&gt;Just to add up to the answer above.&lt;/p&gt;&#xA;&lt;p&gt;In fact if the reward that you get are not stochastic in RL then you simply take a step into your parameter space that guaranteed you the best reward so far (after the evaluation of all other states). So for example if action up is the best one so far, nothing motivates you to try an other one.&lt;/p&gt;&#xA;&lt;p&gt;When you are doing naïve HO it could be seen as an exploration of the space. The environment is not stochastic but the reward (loss decrease) that you will get are not known by the agent beforehand. That's enough to make the exploration step mandatory. So let's say the combination (up, up, down) has got you the best loss so far, you need to actually try other combinations to know if they are the best above all others. In that sense you are exploring too.&lt;/p&gt;&#xA;&lt;p&gt;So when are you not exploring ? If the next step in your HO is given by an optimization step, let's say by a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, then you are not exploring anymore. You are progressing toward the objective given by &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Thus, you have to make sure that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; correctly gives you the best combination of parameters - mathematically &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is converging to a global optimum.&lt;/p&gt;&#xA;&lt;p&gt;So grid search could be viewed as exploration, Bayesian optimization HO not that much.&lt;/p&gt;&#xA;" OwnerUserId="41123" LastEditorUserId="-1" LastEditDate="2020-10-08T23:12:51.177" LastActivityDate="2020-10-08T23:12:51.177" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23686" PostTypeId="2" ParentId="16014" CreationDate="2020-09-21T18:08:09.753" Score="1" Body="&lt;p&gt;Nothing is written on stone in here, but as a rule of thumb linear activation is not very common.&#xA;A linear activation function in a hidden layer can collapse more neurons in more layers.&#xA;Linear activation can be implemented in the last layer if a scale of the outputs is not used. (This is the most common use I have seen.)&lt;/p&gt;&#xA;" OwnerUserId="41126" LastEditorUserId="40573" LastEditDate="2020-09-25T02:45:28.143" LastActivityDate="2020-09-25T02:45:28.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23687" PostTypeId="1" CreationDate="2020-09-21T18:08:22.097" Score="2" ViewCount="49" Body="&lt;p&gt;One of the main arguments against &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-gram models is that, as &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; increases, there is no way to compute &lt;span class=&quot;math-container&quot;&gt;$P(w_n|w_1,\cdots,w_{n-1})$&lt;/span&gt; from training data (since the chance of visiting &lt;span class=&quot;math-container&quot;&gt;$w_n,...,w_1$&lt;/span&gt; is practically zero).&lt;/p&gt;&#xA;&lt;p&gt;Wondering why we cannot estimate &lt;span class=&quot;math-container&quot;&gt;$P(w_n|w_1,\cdots,w_{n-1})$&lt;/span&gt; using the following:&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$P_i(u|v)$&lt;/span&gt; be the probability of having sequences where word &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; comes exactly &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; words after word &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; (This is easy to compute).&lt;/p&gt;&#xA;&lt;p&gt;Then we can esitmate &lt;span class=&quot;math-container&quot;&gt;$P(w_n|w_1,\cdots,w_{n-1})$&lt;/span&gt; as a function of &lt;span class=&quot;math-container&quot;&gt;$P_i(u|v)$&lt;/span&gt;. I could not find any reference to such approach in the literature. The most similar approach is the smoothing/backoff methods.&lt;/p&gt;&#xA;&lt;p&gt;Is there any reason why no-one used this approach? Or if one can share some previous work about this approach.&lt;/p&gt;&#xA;&lt;p&gt;P.S.1. The disadvantage of this approach, comparing with standard &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-gram model, is its running time.&lt;/p&gt;&#xA;&lt;p&gt;P.S.2. We could use bucketing idea: Instead of computing/storing/using &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt;, for every &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, we can compute/store/use &lt;span class=&quot;math-container&quot;&gt;$PB_{i}=P_{2^i}$&lt;/span&gt; . Then &lt;span class=&quot;math-container&quot;&gt;$P_i(u|v) \approx PB_{\log i}(u|v)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41124" LastEditorUserId="41124" LastEditDate="2021-04-23T17:22:34.447" LastActivityDate="2021-04-23T17:22:34.447" Title="Estimating an $n$-Gram model using on bigrams" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23688" PostTypeId="2" ParentId="23667" CreationDate="2020-09-21T18:19:51.063" Score="0" Body="&lt;p&gt;I guess you are using NN for Regresions.&#xA;In the most common aplication a scale of the outputs is implemented.&#xA;This is recommended. Specialy if you have more than one output with diferent scales.&#xA;Otherwise, you will remunerate the neural network for correcting the error of one variable over the other.&#xA;If you still want to avoid a scale of the outputs. Yes. You can use the identity function in the output layer or a linear function (tha same with different slope).&#xA;The weights and bias of some conections will become negative and the hidden neurons are going to work as always.&lt;/p&gt;&#xA;" OwnerUserId="41126" LastActivityDate="2020-09-21T18:19:51.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23689" PostTypeId="2" ParentId="23672" CreationDate="2020-09-21T20:06:58.720" Score="1" Body="&lt;p&gt;No, transfer learning cannot be applied &amp;quot;between&amp;quot; different architectures, as transfer learning is the practice of taking a neural network that has already been trained on one task and retraining it on another task with the same input modality, which means that only the weights (and other trainable parameters) of the network change during transfer learning but not the architecture.&lt;/p&gt;&#xA;&lt;p&gt;In my understanding, transfer learning is also only really effective in deep learning, but I could be wrong, considering that &lt;a href=&quot;https://www.google.com/search?q=random+forest+transfer+learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Google search&lt;/a&gt; seems to yield some results.&lt;/p&gt;&#xA;&lt;p&gt;You might otherwise be thinking of knowledge distillation, which is a related but different concept, where an already trained network acts as a teacher and teaches another network (a student network) with possibly a different architecture (or a machine learning model not based on neural networks at all) the correct outputs for a bunch of input examples.&lt;/p&gt;&#xA;" OwnerUserId="9220" LastActivityDate="2020-09-21T20:06:58.720" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23690" PostTypeId="1" AcceptedAnswerId="23717" CreationDate="2020-09-21T23:10:51.437" Score="4" ViewCount="116" Body="&lt;p&gt;This is a simple question. I know the weights in a neural network can be initialized in many different ways like: random uniform distribution, normal distribution, and Xavier initialization. &lt;strong&gt;But what is the weight initialization trying to achieve?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it trying to allow the gradients to be large so it can quickly converge? Is it trying to make sure there is no symmetry in the gradients? Is it trying to make the outputs as random as possible to learn more from the loss function? Is it only trying to prevent exploding and vanishing gradients? Is it more about speed or finding a global maximum? What would the perfect weights (without being learned parameters) for a problem achieve? What makes them perfect? What are the properties in an initialization that makes the network learn faster?&lt;/p&gt;&#xA;" OwnerUserId="41026" LastEditorUserId="41026" LastEditDate="2020-09-22T16:50:55.577" LastActivityDate="2020-09-24T03:55:03.840" Title="What is the goal of weight initialization in neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;weights&gt;" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23691" PostTypeId="2" ParentId="23690" CreationDate="2020-09-21T23:15:10.173" Score="1" Body="&lt;p&gt;The most important thing we achieve is indeed making sure the weights are not all equal. If they were, every layer would behave as if it were a single cell.&lt;/p&gt;&#xA;&lt;p&gt;We typically want weights that are near zero (so unimportant connections will not accidentally dominate) but non-zero.&lt;/p&gt;&#xA;&lt;p&gt;The different types of initialization all have different motivations, including those mentioned in the question.&lt;/p&gt;&#xA;&lt;p&gt;If you're curious what the motivation for each one is, I would recommend you check the documentation and try to find the original papers where they were first introduced.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-21T23:15:10.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23692" PostTypeId="1" CreationDate="2020-09-21T23:23:44.520" Score="1" ViewCount="41" Body="&lt;p&gt;I have a deep learning configuration in which I obtain good results on the validation set but even better results in the training set. From my understanding this means that there is overfitting to some extent. What does this mean in practice? Does it mean that my model is not good and that I should not use it? If I decrease the gap between the validation and training accuracy (decreasing the overfitting) but at the same time decrease the validation accuracy, which of the two models is better?&lt;/p&gt;&#xA;&lt;p&gt;Below are some images to illustrate the two situations outlined previously:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/moq2x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/moq2x.png&quot; alt=&quot;High validation accuracy and even higher training accuracy &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cCWmy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cCWmy.png&quot; alt=&quot;Smaller gap between validation and training accuracy&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41131" LastActivityDate="2020-09-22T01:49:47.883" Title="How much overfitting is acceptable?" Tags="&lt;machine-learning&gt;&lt;overfitting&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23693" PostTypeId="2" ParentId="23692" CreationDate="2020-09-22T01:49:47.883" Score="1" Body="&lt;p&gt;Validation results will almost never be as good as training results; that's just natural. As long as they are not &lt;em&gt;too&lt;/em&gt; different, you should be fine. What &amp;quot;too different&amp;quot; means depends on the particular data set and model you're using.&lt;/p&gt;&#xA;&lt;p&gt;If you plot the curves for varying parameter values, when the training error keeps going down but the validation error starts going up again, that's when you know there is overfitting. In your second graph, after 14 epochs, we might see the start of overfitting. If you continue this until 20 epochs or so, it should be even more clear. I would guess that 12 is probably a good value for the number of epochs for that problem.&lt;/p&gt;&#xA;&lt;p&gt;In the first graph, we don't see that happening yet. The model might not be well-suited (the gap between training and validation results is a bit larger)  but that can also be because of too little data, or other factors. Perhaps that's just the best you can do; there might be noise in the data or something.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-22T01:49:47.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23696" PostTypeId="5" CreationDate="2020-09-22T10:19:07.877" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-22T10:19:07.877" LastActivityDate="2020-09-22T10:19:07.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23697" PostTypeId="4" CreationDate="2020-09-22T10:19:07.877" Score="0" Body="For theoretical questions that involve the OpenCV library. Note that asking about programming issues or about the software libraries is generally OFF-TOPIC here." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-22T10:19:07.877" LastActivityDate="2020-09-22T10:19:07.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23698" PostTypeId="1" CreationDate="2020-09-22T11:25:13.367" Score="0" ViewCount="91" Body="&lt;p&gt;What is the time complexity for training a single-hidden layer auto-encoder, for 1 epoch?&lt;/p&gt;&#xA;&lt;p&gt;You can assume that there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; training examples, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; features, and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; neurons in the hidden layer, and that we use gradient descent and back-propagation to train the auto-encoder.&lt;/p&gt;&#xA;" OwnerUserId="41141" LastEditorUserId="2444" LastEditDate="2020-09-22T12:09:06.447" LastActivityDate="2020-09-22T12:09:06.447" Title="What is the time complexity for training a single-hidden layer auto-encoder?" Tags="&lt;training&gt;&lt;autoencoders&gt;&lt;time-complexity&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23699" PostTypeId="1" CreationDate="2020-09-22T11:42:23.870" Score="1" ViewCount="36" Body="&lt;p&gt;Convolutions can be expressed as a matrix-multiplication (see e.g. &lt;a href=&quot;https://ai.stackexchange.com/q/11172/2444&quot;&gt;this post&lt;/a&gt;) and as an element-wise multiplication using the Fourier domain (&lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Convolution_theorem&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Attention utilizes matrix multiplications, and is as such &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt;. So, my question is, is it possible to exploit the Fourier domain for attention mechanisms by turning the matrix multiplication of attention into a large convolution between the query and the key matrices?&lt;/p&gt;&#xA;" OwnerUserId="33058" LastEditorUserId="2444" LastEditDate="2020-09-23T11:47:56.033" LastActivityDate="2020-09-23T11:47:56.033" Title="Is it possible to express attention as a Fourier convolution?" Tags="&lt;convolution&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23700" PostTypeId="1" CreationDate="2020-09-22T13:25:16.130" Score="2" ViewCount="46" Body="&lt;p&gt;I have an environment that is computationally heavy (takes several seconds to get a reward and next state). This limits reinforcement capability, due to poor sampling of the problem. There is any strategy that could be used to address the problem (e.g. If I can use the environment in parallel, then I could use a multi-agent approach)&lt;/p&gt;&#xA;" OwnerUserId="31324" LastEditorUserId="31324" LastEditDate="2020-09-22T13:31:50.367" LastActivityDate="2021-03-17T18:05:30.527" Title="What are the strategies for computationally heavy environments or long-time waiting environments?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23701" PostTypeId="1" CreationDate="2020-09-22T14:42:45.517" Score="0" ViewCount="10" Body="&lt;p&gt;I am trying to train a model that can detect a 3D object and give me a 3D bounding box around it. For this, I have a RGBD camera and a 2D LiDAR.&lt;/p&gt;&#xA;&lt;p&gt;Most of the research is done for cars/cyclists/pedestrians and I ponder how to create a model for a custom object (a toy for example).&lt;/p&gt;&#xA;&lt;p&gt;If anyone has experience with this, I need a helping hand.&lt;/p&gt;&#xA;" OwnerUserId="26452" LastActivityDate="2020-09-22T14:42:45.517" Title="Train 3D object detection model for custom object" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23703" PostTypeId="1" CreationDate="2020-09-22T19:02:56.837" Score="0" ViewCount="61" Body="&lt;p&gt;When it comes to using Transformers for image captioning is there any reason to use masking?&lt;/p&gt;&#xA;&lt;p&gt;I currently have a resnet101 encoder and am trying to use the features as the input for a transformer model in order to generate a caption for the image, is there any need to use masking? and what would I mask if I did need to?&lt;/p&gt;&#xA;&lt;p&gt;Any help would be much appreciated&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="40561" LastActivityDate="2020-12-01T14:59:08.630" Title="How to implement or avoid masking for transformer?" Tags="&lt;convolutional-neural-networks&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;pytorch&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23704" PostTypeId="5" CreationDate="2020-09-22T20:13:40.510" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-22T20:13:40.510" LastActivityDate="2020-09-22T20:13:40.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23705" PostTypeId="4" CreationDate="2020-09-22T20:13:40.510" Score="0" Body="For questions that involve software development, but note that programming questions (such as &quot;How to fix this bug?&quot; or &quot;How to use this library's API?&quot;) are OFF-TOPIC here." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-22T20:13:40.510" LastActivityDate="2020-09-22T20:13:40.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23706" PostTypeId="2" ParentId="22963" CreationDate="2020-09-22T22:10:23.127" Score="0" Body="&lt;p&gt;Layer freezing means that the layer weights of the trained model do not change when reused on a subsequent downstream mission, they remain frozen. Basically, when backpropagation is performed during training, these layer weights aren't compromised.&lt;/p&gt;&#xA;" OwnerUserId="17348" LastActivityDate="2020-09-22T22:10:23.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23708" PostTypeId="1" CreationDate="2020-09-23T09:39:29.120" Score="1" ViewCount="115" Body="&lt;p&gt;Recently I have come up with a VGG16 model for my binary classification task. I have relatively simple signal images&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/w6K40.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w6K40.png&quot; alt=&quot;this&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Therefore (maybe?) other deeper models like &lt;code&gt;resnet18&lt;/code&gt; and &lt;code&gt;Inceptionv3&lt;/code&gt; were not as good. As known, VGG uses &lt;code&gt;3x3&lt;/code&gt; filters for convolving the images to make feature maps. I have tried several hyper-parameters to get a desired performance. However, there are still some things I need to do. I was thinking of replacing the &lt;code&gt;3x3&lt;/code&gt; &lt;code&gt;conv&lt;/code&gt; filters with &lt;code&gt;3x1&lt;/code&gt; followed by &lt;code&gt;1x3&lt;/code&gt; filters to reduce the compute. I think it will definitely do so considering the multiplications (9 operations for &lt;code&gt;3x3&lt;/code&gt;and 6 for &lt;code&gt;3x1&lt;/code&gt; followed by &lt;code&gt;1x3&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Then I came to think: &lt;em&gt;&lt;strong&gt;If I replace all the &lt;code&gt;3x3&lt;/code&gt; filters with separable filters, will I get any performance improvement?&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;What are the benefits of replacing &lt;code&gt;3x3&lt;/code&gt; filters with separable ones?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="31870" LastActivityDate="2021-03-23T13:03:45.373" Title="Does replacing 3x3 filters with 3x1 and 1x3 filters improve the performance?" Tags="&lt;deep-learning&gt;&lt;filters&gt;&lt;vgg&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23709" PostTypeId="1" CreationDate="2020-09-23T12:55:16.413" Score="0" ViewCount="12" Body="&lt;p&gt;So this question pertains to Statistical Relational Learning. Specifically in domains where you have a Knowledge base with probabilistic facts, you may think something like a Markov Logic Networks (MLN). But then you have continuous information as well, although one could say that Hybrid MLN's can deal with such tasks, but they don't really seem as general as a Bayesian Network.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if anyone knows of problems where hybrid inference task is clearly specified and exploits expressivity of both logic and probabilistic distributions ?&lt;/p&gt;&#xA;" OwnerUserId="37803" LastActivityDate="2020-09-23T12:55:16.413" Title="What are some good use cases of Inference in Hybrid Domains (Logical + Continuous)?" Tags="&lt;markov-decision-process&gt;&lt;logic&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23710" PostTypeId="1" CreationDate="2020-09-23T14:25:23.813" Score="0" ViewCount="8" Body="&lt;p&gt;I am having trouble understanding implementation of a research paper. &lt;a href=&quot;https://drive.google.com/file/d/13_Xd8RfDZyQRGuF2-_b6lFx0v2wMhu6d/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I need assistance in the &lt;strong&gt;3.6 Final Classification Model&lt;/strong&gt; on page 3. How exactly should the pixels be discretized into 50 bins? Also, how will the model behave as classification model with 50 classes? what will be the shape of the output? Can someone please help me understand the model?&lt;/p&gt;&#xA;" OwnerUserId="20512" LastEditorUserId="20512" LastEditDate="2020-09-23T14:36:20.913" LastActivityDate="2020-09-23T14:36:20.913" Title="Need some help in understanding a Research paper on Auto Image Colorization" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;image-processing&gt;&lt;vgg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23711" PostTypeId="1" CreationDate="2020-09-23T16:12:06.680" Score="0" ViewCount="43" Body="&lt;p&gt;I've started to work on time series. I was wondering what would be the best data normalizing and pre-processing technique for non-linear models, specifically, neural networks.&lt;/p&gt;&#xA;&lt;p&gt;One I can think of is min-max normalization&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$z = \frac{x - min(x)}{max(x) - min(x)}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="41110" LastEditorUserId="2444" LastEditDate="2020-10-03T10:16:14.307" LastActivityDate="2021-04-16T23:47:34.800" Title="What would be a typical pre-processing and data normalization pipeline for time series data (for non-linear models such as neural networks)?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;time-series&gt;&lt;data-preprocessing&gt;&lt;normalisation&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23712" PostTypeId="2" ParentId="2214" CreationDate="2020-09-23T18:07:42.307" Score="0" Body="&lt;p&gt;Imho - manual testers - yes.&lt;/p&gt;&#xA;&lt;p&gt;Developers &amp;amp; automated testers are safety.&lt;/p&gt;&#xA;&lt;p&gt;All tasks that can be described in a sequence of steps - algorithmically&#xA;can be automated.&lt;/p&gt;&#xA;&lt;p&gt;eg. Tricentis - the company already has technology - that can automate the work of manual testers.&lt;/p&gt;&#xA;&lt;p&gt;Let's analyze this piece of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;public class LoginTest {&#xA;&#xA;    private LoginPage loginPage;&#xA;&#xA;    @Before&#xA;    public void setUpTest() {&#xA;        loginPage = new LoginPage(DRIVER);&#xA;    }&#xA;&#xA;    @Test&#xA;    public void shouldLogin() {&#xA;        loginPage.login(getProperty(&amp;quot;ai.stack.username&amp;quot;), getProperty(&amp;quot;ai.stack.password&amp;quot;));&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;functions of this type can be easily presented in UI the form of a diagrams -  adding algorithms to it - they will find generic use for many applications:&lt;/p&gt;&#xA;&lt;p&gt;__&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cDwEf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cDwEf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;strong&gt;source:&lt;/strong&gt; &lt;a href=&quot;https://www.researchgate.net/publication/338441733_Usage_of_Machine_Learning_in_Software_Testing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/338441733_Usage_of_Machine_Learning_in_Software_Testing&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where are the individual pieces are dockerized EC2 instances?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The more detailed algorithm, which can be use in automating software testing:&lt;/p&gt;&#xA;&lt;p&gt;**Static Attributes&#xA;Imbalance learning&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Ensemble learning&lt;/strong&gt; - is a process using which multiple machine learning models (such as classifiers) are strategically constructed to solve a particular problem. (eg.&lt;em&gt;data passed by KNN + SVR + RF - and aggregate prediction&lt;/em&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Multiple Classiﬁers&lt;/strong&gt; - aggregate clasiffier algorithms*(eg. Decision tree classifier, SVM, KNN,)*&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Resampling&lt;/strong&gt;** - mainly refers to &lt;strong&gt;Cross-validation&lt;/strong&gt; and &lt;strong&gt;Bootstrap&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Cross-Validation - evaluating models by splitting into &lt;strong&gt;k&lt;/strong&gt; smaller sets&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Bootstrap - bagging - It also reduces variance and helps to avoid overfitting.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="32352" LastEditorUserId="32352" LastEditDate="2020-09-24T10:02:24.487" LastActivityDate="2020-09-24T10:02:24.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23713" PostTypeId="1" CreationDate="2020-09-23T21:48:05.787" Score="3" ViewCount="21" Body="&lt;p&gt;Consider following problem statement:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You have given &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of actions to get maximum successes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here, we can perform actions and slowly decide upon possible probabilities of each action for success. I have no doubts in this problem.&lt;/p&gt;&#xA;&lt;p&gt;Now consider following variant of the problem:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You have given &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; actions. You can perform any of them. Each action gives you success with some probability. Also you are given set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; probabilities, but you are not told which probability is associated with which action. The challenge is to utilise this additional information to perform given finite number of actions to get maximum successes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have doubt in this problem that how we can map probabilities to actions? I can do some enough number of actions to gather empirical probability and them try to associate given probabilties with actions having closest empirical probabities. But, is there any algorithm for such problem in the literature?&lt;/p&gt;&#xA;" OwnerUserId="41169" LastActivityDate="2020-09-23T21:48:05.787" Title="Mapping given probabilities to empirical probabilities" Tags="&lt;reinforcement-learning&gt;&lt;math&gt;&lt;statistical-ai&gt;&lt;multi-armed-bandit&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23714" PostTypeId="1" CreationDate="2020-09-23T22:23:20.977" Score="0" ViewCount="36" Body="&lt;p&gt;Ridgid body simulation is a well known field with well established methods. It's still fairly computationally expensive to simulate things.&lt;/p&gt;&#xA;&lt;p&gt;I am interested in approaches to training deep learning networks to predict rigid body dynamics and interactions to reduce the computational load associated with simulations.&lt;/p&gt;&#xA;&lt;p&gt;Has this been done before and what approaches have been used?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-09-23T22:23:20.977" Title="Deep learning based physics engine" Tags="&lt;deep-learning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23715" PostTypeId="2" ParentId="2214" CreationDate="2020-09-24T00:04:26.393" Score="3" Body="&lt;p&gt;Artificial intelligence and, in particular, machine learning (ML) and evolutionary algorithms are already being used to automate the task of software development and, in particular, software testing. For example, take a look at the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0950584917304858&quot; rel=&quot;nofollow noreferrer&quot;&gt;An empirical evaluation of evolutionary algorithms for unit test suite generation&lt;/a&gt; (2018). There are other examples, such as the &lt;a href=&quot;https://arxiv.org/pdf/1511.06279.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Programmer-Interpreter&lt;/a&gt; (2016), which is an ML model that can learn to write (very simple) programs or algorithms (e.g. it learns to perform addition).&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless, we are far from creating an AI system that is as capable as an average human software developer of developing software. In fact, I conjecture that this AI needs to be an &lt;em&gt;artificial general intelligence&lt;/em&gt; (AGI), an AI that can solve as many tasks a human (at least approximately). In other words, software development is probably an &lt;a href=&quot;https://ai.stackexchange.com/a/12147/2444&quot;&gt;AI-complete problem&lt;/a&gt; (i.e. a problem that probably requires an AGI to be solved, i.e. a narrow AI is not sufficient). Why do I say that? Because software development is a very complex and high-level task, even for humans, which not only includes writing code, but it requires the analyses of a problem (that is often very complex or novel), finding possible solutions, devising a plan to execute those solutions, etc., so it requires some form of creativity, and also common-sense knowledge, given that this artificial software developer would need to interact with other general intelligences (in particular, humans), which share some knowledge and assumptions.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When will an artificial intelligence system do the job of a software developer or tester? Is this possible (for example, in the next 10 years)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, AI techniques are already being developed to automate some tasks that a human software developer or tester does. However, it is very improbable that in the next 10-20 years an AI will completely replace the human in this very complex task. Maybe in 100-200 years an AI system will completely replace the human, but some radically important innovations or breakthroughs need to be made for this to happen. In any case, I do not think that machine learning (i.e. a data-driven approach) is sufficient to solve this problem because software development is not just about pattern recognition and stuff like that, but it also requires logical reasoning and planning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-09-24T00:04:26.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23716" PostTypeId="2" ParentId="23207" CreationDate="2020-09-24T03:35:43.680" Score="1" Body="&lt;p&gt;We usually divide the dataset into multiple subsets namely (training, validation and test sets). During training, we validate the model against the validation set. And during testing, we use the test dataset to obtain metrics for the model. We should make sure the subsets are taken from the same sample. Once you've tested it against the test subset, there's nothing really we can do.&lt;/p&gt;&#xA;&lt;p&gt;You can also increase your dataset, by using multiple data sources if the problem statement allows you to.&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-09-24T03:35:43.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23717" PostTypeId="2" ParentId="23690" CreationDate="2020-09-24T03:38:59.967" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Is it trying to make sure there is no symmetry in the gradients&lt;/strong&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The aim of weight initialization is to make sure that we don't converge to a trivial solution. That's why we have different kinds of initialization depending on the dataset type. So, Yes it is trying to avoid symmetry.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Is it trying to allow the gradients to be large so it can quickly converge?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The time it takes to converge, is I think a property of the optimizer and not of the weights initialization. Of course, the manner in which we initialize our weights matters but I think Optimization Algorithms contribute more towards convergence&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;What are the properties in an initialization that makes the network learn faster?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Glorot and Bengio believed that Xavier weight initialization would maintain the variance of activations and back-propagated gradients all the way up or down the layers of a network. Incidentally, when they trained deeper networks that used ReLUs, it was found that a 30-layer CNN using Xavier initialization stalled completely and didn’t learn at all. Thus, it depends on the particular problem at hand.&lt;/p&gt;&#xA;" OwnerUserId="40434" LastEditorUserId="40434" LastEditDate="2020-09-24T03:55:03.840" LastActivityDate="2020-09-24T03:55:03.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23718" PostTypeId="1" CreationDate="2020-09-24T03:52:40.513" Score="0" ViewCount="63" Body="&lt;p&gt;While researching why we need non linear activation functions, all the explanations revolve around neural network being able to separate values that aren't linearly separable.&#xA;So I wonder, if we have a neural network whose task is something else, say predicting an output value of a time series, is it still important to have an activation function that is non linear?&lt;/p&gt;&#xA;" OwnerUserId="38668" LastActivityDate="2020-09-24T04:08:15.607" Title="Do we need non-linear activation function in neural networks whose task isn't classification?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;activation-function&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23719" PostTypeId="2" ParentId="23718" CreationDate="2020-09-24T04:08:15.607" Score="2" Body="&lt;h1&gt;Yes, definitely.&lt;/h1&gt;&#xA;&lt;p&gt;In the simplest example, predicting an output value for a time series &lt;em&gt;is&lt;/em&gt; classification. You take in the previous time steps and &lt;em&gt;classify&lt;/em&gt; what is the most likely next value. You could do this with a &lt;strong&gt;RNN&lt;/strong&gt; (&lt;strong&gt;R&lt;/strong&gt;ecurrent &lt;strong&gt;N&lt;/strong&gt;eural &lt;strong&gt;N&lt;/strong&gt;etwork) for example.&lt;/p&gt;&#xA;&lt;p&gt;If the activation functions are all linear, the nerual network is just a glorified linear regression. Think of it like this: a neural network is trying to approximate a complicated function in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; dimensional space. It does this by combining operations on a series of known functions, to get a resultant function that hopefully mimics the desired function. The issue with combining linear functions is the only thing you'll ever get at the end is a linear function.&lt;/p&gt;&#xA;&lt;p&gt;As a concrete example, try and approximate the function &lt;span class=&quot;math-container&quot;&gt;$y = x^3 + x^2 -x -1$&lt;/span&gt; by adding a series of linear functions together. You'll find pretty quickly this is useless. However, if you use a non-linear function, such as a &lt;strong&gt;ReLU&lt;/strong&gt; (&lt;strong&gt;Re&lt;/strong&gt;ctified &lt;strong&gt;L&lt;/strong&gt;inear &lt;strong&gt;U&lt;/strong&gt;nit) you can quite easily approximate this function. &lt;a href=&quot;https://www.desmos.com/calculator/cfvtjusqmq&quot; rel=&quot;nofollow noreferrer&quot;&gt;See this implementation on desmos&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If a problem has any sort of complexity to it, the function it follow is likely incredibly complicated, and futile to approximate using linear equations.&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-09-24T04:08:15.607" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23720" PostTypeId="2" ParentId="23708" CreationDate="2020-09-24T04:16:05.273" Score="-1" Body="&lt;p&gt;First of all, Keep in mind that maths operations aren’t the only thing that contribute to performance. Memory bandwidth can also be a factor.&lt;/p&gt;&#xA;&lt;p&gt;And most importantly, we want to capture as much area as we can in lowest possible number of operations. So in 3x3 kernel case, we can capture 9 cells in one shot, but with 3x1 followed by 1x3, we have to compute 6 times to capture 9 cells. Which clearly states that 3x3 kernel is far more efficient than these two sequential kernels.&lt;/p&gt;&#xA;&lt;p&gt;So, answer to your question will be no, it will not improve performance, instead it will increase computation overhead for your system.&lt;/p&gt;&#xA;" OwnerUserId="41110" LastActivityDate="2020-09-24T04:16:05.273" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23723" PostTypeId="1" AcceptedAnswerId="23855" CreationDate="2020-09-24T08:50:23.183" Score="1" ViewCount="49" Body="&lt;p&gt;I would like to classify the subject of a conversation. I could classify each messages of the conversation, but I will loose some imformation because of related messages.&lt;/p&gt;&#xA;&lt;p&gt;I also need to do it gradually and not at the end of the conversation.&lt;/p&gt;&#xA;&lt;p&gt;I searched near &lt;strong&gt;recurrent neural network&lt;/strong&gt; and &lt;strong&gt;connectionist classification&lt;/strong&gt; but I'm not sure it answer really well my issue.&lt;/p&gt;&#xA;" OwnerUserId="41176" LastEditorUserId="2444" LastEditDate="2020-09-24T15:53:05.847" LastActivityDate="2020-10-01T14:58:00.797" Title="Is it possible to classify the subject of a conversation?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;text-summarization&gt;&lt;natural-language-understanding&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23724" PostTypeId="1" CreationDate="2020-09-24T11:08:53.300" Score="1" ViewCount="15" Body="&lt;p&gt;Suppose I have a dataset with hand images. Hand completely opened is labeled as 0 and hand completely closed (fist) are labeled as 1. I also have a bunch of unlabeled images of hands which, if properly labeled would have values between 0 and 1. Because they are not completely opened and not completely closed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CBFyl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CBFyl.png&quot; alt=&quot;hand open&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dtQK3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dtQK3.jpg&quot; alt=&quot;hand close&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Extra info I have is the ordering between all the pairs of unlabeled images. For example, given image A and B, I can tell you which image should be predicted with higher value, but I cannot tell you what exactly is the value. The unlabeled dataset is collected by recording a video of a closing hand from completely opened to completely closed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What are some machine learning techniques that I can use to give the label or predict the values of hands not completely closed and not completely opened?&lt;/strong&gt; I expect it to be based on ordering or ranking system. If it doesn't even require the ordering label (A &amp;gt; B?) then that would be a very smart algorithm.&lt;/p&gt;&#xA;&lt;p&gt;I want the values between 0 and 1. If there's a code for it that would be a plus. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="20819" LastActivityDate="2020-09-24T11:08:53.300" Title="How do you make a regression model from a binary labeled dataset?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23726" PostTypeId="1" CreationDate="2020-09-24T11:40:05.960" Score="1" ViewCount="35" Body="&lt;p&gt;In a deep connected network, when every unit gets all the input features(X) so it has one parameter for every feature and every unit tweaks its parameters for loss optimization. What if we use only one unit and that one unit will have all the parameters which it can tweak for loss optimization. Is there a reason or benefit of using multiple units in every layer except the output layer?&lt;/p&gt;&#xA;" OwnerUserId="41178" LastEditorUserId="2444" LastEditDate="2020-09-25T13:43:23.000" LastActivityDate="2020-09-25T15:54:34.570" Title="Why one unit in the layers of neural network is not enough?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;artificial-neuron&gt;&lt;hyper-parameters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23728" PostTypeId="1" CreationDate="2020-09-24T12:57:39.407" Score="2" ViewCount="41" Body="&lt;p&gt;For example, if I use some iterative solvers to find a solution to a non-linear least squares problem, is that already considered machine learning?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastEditorUserId="4446" LastEditDate="2020-09-24T13:41:29.467" LastActivityDate="2020-09-24T15:02:13.503" Title="What's the threshold to call something 'machine learning'?" Tags="&lt;machine-learning&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23729" PostTypeId="2" ParentId="23723" CreationDate="2020-09-24T13:13:57.490" Score="1" Body="&lt;p&gt;This is a difficult problem.&lt;/p&gt;&#xA;&lt;p&gt;First, how do you define 'subject'? Do you have a (closed) lists of labels you want to assign? What about subjects that overlap, or don't occur in your list? What even is a subject? This is a non-trivial issue.&lt;/p&gt;&#xA;&lt;p&gt;Second, and this is even harder, how do you want to recognise subjects? A simple solution could be using a list of associated keywords, but this is problematic as many words have multiple meanings, and words are not really a good indicator of a conversation topic in the first place.&lt;/p&gt;&#xA;&lt;p&gt;Instead of jumping to an implementation method, be clear about how you want to tackle these two items first. Start by annotating a conversation transcript by hand. You will then get a feeling for the problems and possible solutions. After you have done this, you can think about how to get a machine to do it efficiently.&lt;/p&gt;&#xA;&lt;p&gt;UPDATE: For a scheme to annotate the functions of lines within a conversation have a look at Francis &amp;amp; Hunston (1992) &lt;em&gt;Analysing Everyday Conversation&lt;/em&gt;.  In Coulthard, M. (ed.) &amp;quot;Advances in Spoken Discourse Analysis&amp;quot;. London: Routledge. pp.123-161.  This is more oriented towards linguistics, but might give you some ideas on how to proceed.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-09-25T16:03:42.200" LastActivityDate="2020-09-25T16:03:42.200" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23730" PostTypeId="2" ParentId="23728" CreationDate="2020-09-24T13:34:55.540" Score="2" Body="&lt;p&gt;T. Mitchell defines machine learning in &lt;a href=&quot;https://books.google.be/books?id=EoYBngEACAAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Machine Learning&amp;quot; book&lt;/a&gt; as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a computer program is said to learn from experience  concerning some class of tasks  and performance measure , if its performance at tasks in , as measured by , improves with experience &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Hence, based on the above definition, we can't say a machine learning method to every iterative method.&#xA;In your specific example, it is just a non-linear solver such as the Newton method to finding roots.&lt;/p&gt;&#xA;&lt;p&gt;However, you should notice that a non-specific machine learning method can be used in the learning process. For example, you might need some numerical methods to compute the measure &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;‌ (in the above definition). But, we can't say that the specified method is a machine learning method.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-09-24T15:02:13.503" LastActivityDate="2020-09-24T15:02:13.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23732" PostTypeId="1" AcceptedAnswerId="23733" CreationDate="2020-09-24T15:58:52.940" Score="0" ViewCount="65" Body="&lt;p&gt;What are trap functions in genetic algorithms? Suppose you ran a GA with a trap function and examined the population midway through the run. Can someone explain what you would expect the population to look like?&lt;/p&gt;&#xA;" OwnerUserId="41181" LastEditorUserId="2444" LastEditDate="2020-12-08T16:08:44.303" LastActivityDate="2020-12-08T16:08:44.303" Title="What are trap functions in genetic algorithms?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;trap-functions&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-12-08T16:10:32.197" ContentLicense="CC BY-SA 4.0" />
  <row Id="23733" PostTypeId="2" ParentId="23732" CreationDate="2020-09-24T16:19:03.963" Score="2" Body="&lt;p&gt;Traps are functions that are designed to have a very obvious gradient that leads to basically the second-best solution, with the best one being very far removed from that, often the complement of the second-best.&lt;/p&gt;&#xA;&lt;p&gt;Take the 1-max problem. You have N bits, and the fitness of each individual is just the number of 1 bits in the string. If you run a GA on that, you'd expect to see an initial random distribution of 1s and 0s across the population fairly quickly start to converge to strings of all 1s. Now let's add one more clause to the fitness criteria. Instead of f(0)=0, let f(0)=N+1. This gives you a trap function. The &amp;quot;trap&amp;quot; is the solution of all 1s. Everything in the search space points to this being the best solution, but it's actually maximally far away from the best solution.&lt;/p&gt;&#xA;&lt;p&gt;In terms of dyanmics, now the optimal solution is the string of all 0s. But unless you happen to very quickly stumble across that string, you likely never will. That is, there's some slight chance that your initial population included 00001111 and 11110000 and you crossed them over in the middle and got the optimum. Or maybe you had 00001000 and you got lucky and mutated that one bit to a zero. But if you don't do that immediately, you're screwed, because the algorithm is going to pretty relentless drive all the zero bits out of the population. No matter what it does, flipping a 0 to a 1 is always better unless it would otherwise lead to the single string of all 0s, so there's constant pressure to find more and more 1 bits. A few generations in, you might have 11110110, but you're never realistically going to randomly mutate all six of those 1s to 0s. And you have to get it all in one shot. Any subset of fewer than six of those bits being flipped will have worse fitness than you started with, and be selected against.&lt;/p&gt;&#xA;" OwnerUserId="3365" LastActivityDate="2020-09-24T16:19:03.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23734" PostTypeId="1" CreationDate="2020-09-24T18:46:46.503" Score="1" ViewCount="48" Body="&lt;p&gt;What I want to do is from an Internet challenge to transform any given image into the Polish flag &lt;strong&gt;using the available filters and crop tool on the iPhone camera app&lt;/strong&gt;. &lt;a href=&quot;https://www.youtube.com/watch?v=cd1LHBJJiTg&amp;amp;t=7s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an example&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There aren't nearly enough of these videos to train a neural network using a labeled dataset, and (while I haven't ruled it out) I don't think automatically inserting a polish flag into an image then adding random filters to it to create my own dataset would work out.&lt;/p&gt;&#xA;&lt;p&gt;My thinking is that I would feed a neural network the image and it would output a value for each filter &amp;amp; cropping coordinates. Then, I could easily calculate the loss by comparing the resulting picture to a picture of the polish flag. The obvious problem here is that you don't know how each of the neurons in the last layer affects the loss so you can't perform back propagation.&lt;/p&gt;&#xA;&lt;p&gt;Is my best bet to mathematically calculate the loss (by this I mean as opposed to using high level libraries, which would be difficult but I'm sure it's possible) so I can find the partial derivative of each last layer neuron with respect to the loss function and then backpropagate? Would this even work? Are there any alternatives that you recommend?&lt;/p&gt;&#xA;" OwnerUserId="41183" LastEditorUserId="11539" LastEditDate="2020-09-29T16:55:06.383" LastActivityDate="2020-09-29T16:55:06.383" Title="Is this ML task possible?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23736" PostTypeId="2" ParentId="23734" CreationDate="2020-09-24T20:05:59.663" Score="1" Body="&lt;p&gt;I think the best thing to use here is a form of &amp;quot;structured prediction&amp;quot;. Our &amp;quot;target&amp;quot; is a &lt;em&gt;sequence&lt;/em&gt; of operations. The framework of structured prediction allows us to chain together as many filters as we want.&lt;/p&gt;&#xA;&lt;p&gt;With a neural network of fixed architecture, you would have to make sure you have enough space for all the filters you &lt;em&gt;might&lt;/em&gt; need.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-24T20:05:59.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23737" PostTypeId="1" CreationDate="2020-09-24T22:15:36.530" Score="1" ViewCount="55" Body="&lt;p&gt;I am trying to comprehend how the Gradient Descent works.&lt;/p&gt;&#xA;&lt;p&gt;I understand we have a cost function which is defined in terms of the following parameters,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$J(_{1},_{2},.... , w_{n}, b)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;the derivative would tell us which direction to adjust the parameters.&lt;/p&gt;&#xA;&lt;p&gt;i.e. &lt;span class=&quot;math-container&quot;&gt;$\dfrac{dJ(_{1},_{2},.... , w_{n}, b)}{d(_{1}}$&lt;/span&gt; is the rate of change of the cost w.r.t &lt;span class=&quot;math-container&quot;&gt;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The lecture kept on saying this very valuable as we are asking the question &lt;em&gt;how should I change &lt;span class=&quot;math-container&quot;&gt;$$&lt;/span&gt;&lt;/em&gt; to improve the cost?&lt;/p&gt;&#xA;&lt;p&gt;But then the Lecturer presented &lt;span class=&quot;math-container&quot;&gt;$w_{1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$w_{2}$&lt;/span&gt;, ... as scaler value, How can we differentiate a scalar value.&lt;/p&gt;&#xA;&lt;p&gt;I am fundamentally missing what is happening.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone please guide me to any blog post, a book that I should read to understand better?&lt;/p&gt;&#xA;" OwnerUserId="41187" LastEditorUserId="41187" LastEditDate="2020-09-24T23:03:28.523" LastActivityDate="2020-09-24T23:03:28.523" Title="How parameter adjustment works in Gradient Descent?" Tags="&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23738" PostTypeId="1" CreationDate="2020-09-24T22:39:03.863" Score="3" ViewCount="24" Body="&lt;p&gt;Is there a way to quantify the amount of information lost in the lossy part of an autoencoder where the original input is compressed to a representation with less degrees of freedom?&lt;/p&gt;&#xA;&lt;p&gt;I was thinking maybe to use somehow the mutual information either in the image or frequency domain.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathrm{I}(X ; Y)=\sum_{y \in \mathcal{Y}} \sum_{x \in \mathcal{X}} p_{(X, Y)}(x, y) \log \left(\frac{p_{(X, Y)}(x, y)}{p_{X}(x) p_{Y}(y)}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p_{(X,Y)}$&lt;/span&gt; is the joint probability density function which is deduced somehow empirically from a set of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; input and output to the network.&lt;/p&gt;&#xA;&lt;p&gt;Maybe it's not even an interesting question since the &lt;a href=&quot;https://medium.com/@jannik.zuern/but-what-is-an-autoencoder-26ec3386a2af&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss function&lt;/a&gt; evaluates exactly that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uKjYj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uKjYj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12975" LastActivityDate="2020-09-24T22:39:03.863" Title="How to quantify the amount of information lost by the decoder NN in an AE?" Tags="&lt;autoencoders&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23739" PostTypeId="2" ParentId="23737" CreationDate="2020-09-24T22:59:33.263" Score="3" Body="&lt;p&gt;Imagine we have the curve &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt;, and we want to find the minimum of this function. The derivate of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$2x$&lt;/span&gt;. Now, gradient descent works by updating our current estimate of the minimum, say &lt;span class=&quot;math-container&quot;&gt;$c_t$&lt;/span&gt;, by the following iterative process&#xA;&lt;span class=&quot;math-container&quot;&gt;$$c_{t+1} = c_t - \alpha \times \nabla_xf(x=c_t),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is some constant to control how much we shift towards the gradient.&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, this should make sense. Imagine our current estimate of the minimum is &lt;span class=&quot;math-container&quot;&gt;$c_t = -1$&lt;/span&gt;. The update would then give us &lt;span class=&quot;math-container&quot;&gt;$c_{t+1} = -1 - \alpha \times -2 = -1 + 2\alpha &amp;gt; -1$&lt;/span&gt;. As you can see, the update has shifted our estimate in the direction of the minimum. If our estimate were &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; then you can probably see that the update would again have shifted us in the direction of the minimum again.&lt;/p&gt;&#xA;&lt;p&gt;Now, what happens in machine learning is we have a loss function &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; that we typically want to find the minimum of in terms of the parameters of our model. By applying gradient descent on the loss function as we did above with &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; we iteratively apply the update rule which will eventually lead us to the minimum of our loss function with respect to the weights. The process is exactly the same as above except it is likely to happen in higher dimensions, where the derivative of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; becomes a vector of partial derivates. Note that whilst &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;'s are scalar values, we are not differentiating these values, rather we are differentiating the loss function with respect to these scalars.&lt;/p&gt;&#xA;&lt;p&gt;I would recommend you to try this out with a simple linear model of maybe 2 or 3 parameters. &lt;a href=&quot;https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931&quot; rel=&quot;nofollow noreferrer&quot;&gt;After a quick Google I found this article that may be useful&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-09-24T22:59:33.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23740" PostTypeId="1" AcceptedAnswerId="23748" CreationDate="2020-09-25T03:40:03.680" Score="3" ViewCount="72" Body="&lt;p&gt;In all examples I've ever seen, the learning rate of an optimisation method is always less than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. However, I've never found an explanation as to why this is. In addition to that, there are some cases where having a learning rate bigger than 1 is beneficial, such as in the case of &lt;a href=&quot;https://arxiv.org/abs/1708.07120&quot; rel=&quot;nofollow noreferrer&quot;&gt;super-convergence&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why is the learning rate generally less than 1? Specifically, when performing an update on a parameter, why is the gradient generally multiplied by a factor less than 1 (absolutely)?&lt;/p&gt;&#xA;" OwnerUserId="26726" LastEditorUserId="2444" LastEditDate="2020-11-29T12:25:37.813" LastActivityDate="2020-11-29T12:25:37.813" Title="Why is the learning rate generally beneath 1?" Tags="&lt;machine-learning&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;learning-rate&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23741" PostTypeId="1" AcceptedAnswerId="23742" CreationDate="2020-09-25T06:22:13.723" Score="0" ViewCount="36" Body="&lt;p&gt;I have read a lecture note of Prof. Andrew Ng. There was something about data normalization like how can we flatten an image of (64x64x3) into a (64x64x3)*x1 vector. After that there is pictorial representation of flatten&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l8S4Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l8S4Q.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As per the picture height, length and width of the picture is 64 , 64, 3. I think nx is a row vector which is then transpose to a column vector. If there is 3 pictures I think nx contains {64,64,3,64,64,3,64,64,3}. Am I right?&lt;/p&gt;&#xA;&lt;p&gt;To use a 64x64x3 image as an input to our neuron, we need to flatten the image into a (64x64x3)x1 vector. And to make Wᵀx + b output a single value z, &lt;strong&gt;we need W to be a (64x64x3)x1 vector&lt;/strong&gt;: &lt;strong&gt;(dimension of input)x(dimension of output), and b to be a single value.&lt;/strong&gt; With N number of images, we can make a matrix X of shape (64x64x3)xN. WᵀX + b outputs Z of shape 1xN containing z’s for every single sample, and by passing Z through a sigmoid function we get final ŷ of shape 1xN that contains predictions for every single sample. We do not have to explicitly create a b of 1xN with the same value copied N times, thanks to Python broadcasting.&lt;/p&gt;&#xA;&lt;p&gt;As per my understanding, Wᵀ = nx and x= nxᵀ.&lt;/p&gt;&#xA;&lt;p&gt;Is it Wᵀ= [64,64,3,64,64,3,64,64,3] and x = [64,64,3,64,64,3,64,64,3]ᵀ?&lt;/p&gt;&#xA;&lt;p&gt;In that case there product will be a symmetry matrix.&lt;/p&gt;&#xA;&lt;p&gt;Is there any significance of symmetry matrix?&lt;/p&gt;&#xA;&lt;p&gt;I just messed up all the things while flatten the image. If anyone has any idea please share with me.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="18384" LastEditorUserId="18384" LastEditDate="2020-09-25T07:02:20.200" LastActivityDate="2020-09-25T07:02:20.200" Title="Flatten image using Neural network and matrix transpose" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;thought-vectors&gt;&lt;normalisation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23742" PostTypeId="2" ParentId="23741" CreationDate="2020-09-25T06:37:44.403" Score="1" Body="&lt;p&gt;Yes, if you have 3 images (and by images I assume you mean samples) the flatten layer will be of the shape &lt;span class=&quot;math-container&quot;&gt;$12288*3$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$64*64*3=12288$&lt;/span&gt;). The size of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; however does not change, and nor does the size of &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; as these are parameters and are independent of the amount of samples passed through the network.&lt;/p&gt;&#xA;&lt;p&gt;ETA: I only answered the &amp;quot;Am I right?&amp;quot; part of your question because that's the only part of your questions that's actually a question. I don't know what you're trying to ask in the second half of your question&lt;/p&gt;&#xA;" OwnerUserId="26726" LastActivityDate="2020-09-25T06:37:44.403" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23744" PostTypeId="1" CreationDate="2020-09-25T08:31:00.370" Score="1" ViewCount="21" Body="&lt;p&gt;Could someone help me to understand in detail each step of this fuzzy diagram, because I am lost?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bUSdf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bUSdf.png&quot; alt=&quot;Fuzzy&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40555" LastEditorUserId="2444" LastEditDate="2020-09-25T10:05:38.167" LastActivityDate="2020-09-25T10:05:38.167" Title="Can someone explain and help to understand this fuzzy diagram?" Tags="&lt;machine-learning&gt;&lt;fuzzy-logic&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23745" PostTypeId="1" AcceptedAnswerId="23794" CreationDate="2020-09-25T10:12:08.510" Score="1" ViewCount="114" Body="&lt;p&gt;Consider &lt;a href=&quot;https://youtu.be/FgzM3zpZ55o?list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&amp;amp;t=2705&quot; rel=&quot;nofollow noreferrer&quot;&gt;this slide&lt;/a&gt; from a Stanford lecture on reinforcement learning. It states that a model is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the agent's representation of how the world changes in response to the agent's action.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I've been experimenting with Q-learning for simple problems such as OpenAI's &lt;a href=&quot;https://gym.openai.com/envs/FrozenLake-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FrozenLake&lt;/a&gt; and &lt;a href=&quot;https://gym.openai.com/envs/MountainCar-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mountain Car&lt;/a&gt;, which both are amenable to the Q-learning framework (the latter upon discretization). I consider the topologies of the lake and the mountain to be the &amp;quot;worlds&amp;quot; (aka. environments) in the two cases, respectively.&lt;/p&gt;&#xA;&lt;p&gt;Q-learning is said to be &lt;a href=&quot;https://spinningup.openai.com/en/latest/_images/rl_algorithms_9_15.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;model-free&amp;quot;&lt;/a&gt;. Given the two examples above, is it because neither the lake's topology nor that of the mountain are changed by the actions taken?&lt;/p&gt;&#xA;" OwnerUserId="30959" LastActivityDate="2020-09-27T13:35:53.027" Title="How does one know that a problem is &quot;model-free&quot; in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23746" PostTypeId="1" AcceptedAnswerId="23751" CreationDate="2020-09-25T12:28:41.060" Score="1" ViewCount="66" Body="&lt;p&gt;Say I trained a Neural Network (not RNN or CNN) to classify a particular data set.&lt;/p&gt;&#xA;&lt;p&gt;So I train using a specific data set &amp;amp; then I test using another and get an accuracy of 95% which is good enough.&lt;/p&gt;&#xA;&lt;p&gt;I then deploy this model in a production level environment where it will then be processing real world data.&lt;/p&gt;&#xA;&lt;p&gt;My question is, will this trained NN be constantly learning even in a production scenario? I can't figure out how it will because say it processes a dataset such as this:&#xA;&lt;code&gt;[ [1,2,3] ]&lt;/code&gt; and gets an output of &lt;code&gt;[ 0, 0.999, 0 ]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;In a training scenario it will compare the predicted output to the actual output and back propagate but in a real world scenario it will not know the actual value.&lt;/p&gt;&#xA;&lt;p&gt;So how does a trained model learn in a real world scenario?&lt;/p&gt;&#xA;&lt;p&gt;I am still very much a beginner in this field and I am not sure if the technology used is going to affect the answer to this question, but I am hoping to use Eclipse Deeplearning4J to create a NN. That being said the answer does not need to be restricted to this technology in particular as I am hoping more for the theory behind it and how it works.&lt;/p&gt;&#xA;" OwnerUserId="41196" LastEditorUserId="2444" LastEditDate="2020-09-25T13:56:19.480" LastActivityDate="2020-09-25T13:56:58.517" Title="How does a neural network that has been trained keep learning while in a real world scenario" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;online-learning&gt;&lt;incremental-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23748" PostTypeId="2" ParentId="23740" CreationDate="2020-09-25T13:22:54.027" Score="5" Body="&lt;p&gt;If the learning rate is greater than or equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; the Robbins-Monro condition&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sum _{{t=0}}^{{\infty }}a_{t}^{2}&amp;lt;\infty\label{1}\tag{1},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; is the learning rate at iteration &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, does not hold (given that a number bigger than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; squared becomes a bigger number), so stochastic gradient descent is not generally guaranteed to converge to a minimum [&lt;a href=&quot;https://arxiv.org/pdf/1401.0118.pdf&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;] (although the condition \ref{1} is a sum from &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t=\infty$&lt;/span&gt;, but, of course, we only iterate for a finite number of iterations). Moreover, note that, if the learning rate is bigger than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, you are essentially giving more weight to the gradient of the loss function than to the current value of the parameters (you give weight &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to the parameters).&lt;/p&gt;&#xA;&lt;p&gt;This is probably the main reason why the learning rate is usually in the range &lt;span class=&quot;math-container&quot;&gt;$(0, 1)$&lt;/span&gt; and there are methods to decay the learning rate, which can be beneficial (and there are several explanations of why this is the case [&lt;a href=&quot;https://arxiv.org/pdf/1908.01878.pdf&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;]).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-25T13:28:25.827" LastActivityDate="2020-09-25T13:28:25.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23749" PostTypeId="2" ParentId="23745" CreationDate="2020-09-25T13:29:21.033" Score="1" Body="&lt;p&gt;A reinforcement learning algorithm is considered model based if it uses estimates of the environments dynamics to help learn. For instance, in the Tabular Dyna-Q algorithm, every time you visit a state action tuple you store in a look-up table the reward received and the next state transitioned to, and after every execution of an action you loop &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times to further back up your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; table using these stored model values from the look-up table. I will attach a copy of the pseudo-code for an algorithm at the bottom of this post.&lt;/p&gt;&#xA;&lt;p&gt;Algorithms like vanilla &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning are model free because they don't require a model of the environment to learn.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VF35X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VF35X.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-09-25T13:29:21.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23750" PostTypeId="2" ParentId="23726" CreationDate="2020-09-25T13:32:58.943" Score="1" Body="&lt;p&gt;So if I understand correctly, you're proposing to use a neutral net with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; input units (let's say data is in &lt;span class=&quot;math-container&quot;&gt;$\mathbf{R}^N$&lt;/span&gt;), 1 hidden unit, and whatever the necessary output needs to be.&lt;/p&gt;&#xA;&lt;p&gt;Let's say we try to do this. Then each unit of the output layer is responsible for computing its output based on a single scalar input. So it's like as if you're saying the problem can be compressed to another problem that is 1-dimensional. Now let's say you're doing regression. By modeling your NN this way, you're saying there's a function that (approximately) maps a scalar (one dimensional object) to the target output. Suppose your output layer is a linear layer (which is usually the case in regression), this is saying that the dimension of the output space is at most 1. This is almost never the case, especially with complex data.&lt;/p&gt;&#xA;&lt;p&gt;Now suppose you had several hidden units (call this number &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;) and the weights are initialized properly. Then, it's likely that your hidden unit will have dimension larger than one, perhaps even &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, which allows for a richer output space as &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; increases.&lt;/p&gt;&#xA;&lt;p&gt;You can make similar arguments for the classification setting, but the output nonlinearity just makes it a little less clear.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastEditorUserId="37829" LastEditDate="2020-09-25T15:54:34.570" LastActivityDate="2020-09-25T15:54:34.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23751" PostTypeId="2" ParentId="23746" CreationDate="2020-09-25T13:51:24.480" Score="1" Body="&lt;p&gt;You are right. If you don't continuously train the neural network after you have deployed it, there is no way it can continuously learn or be updated with more information. You need to program the neural network to learn even after it has been deployed. There is &lt;strong&gt;no&lt;/strong&gt; such thing as a neural network that decides what it does without a human deciding first what it needs to do: this is a very common misconception (probably caused by the media and science fiction movies). It's also true that you need to label your data if you intend to train the neural network in a &lt;em&gt;supervised&lt;/em&gt; fashion, but there are other ways to train neural networks (e.g. by &lt;em&gt;reinforcement learning&lt;/em&gt;), depending also on the problem you want to solve.&lt;/p&gt;&#xA;&lt;p&gt;If you want to develop neural networks that can learn continually, you probably want to look into &lt;a href=&quot;https://arxiv.org/pdf/1802.07569.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;continual learning techniques for neural networks&lt;/em&gt;&lt;/a&gt;. Another term that you may be looking for is &lt;em&gt;online machine learning&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-25T13:56:58.517" LastActivityDate="2020-09-25T13:56:58.517" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23752" PostTypeId="5" CreationDate="2020-09-25T14:03:22.280" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-25T14:03:22.280" LastActivityDate="2020-09-25T14:03:22.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23753" PostTypeId="4" CreationDate="2020-09-25T14:03:22.280" Score="0" Body="For questions about the mask R-CNN model, proposed in the paper &quot;Mask R-CNN&quot; (2018) by Kaiming He et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-25T14:03:22.280" LastActivityDate="2020-09-25T14:03:22.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23754" PostTypeId="1" CreationDate="2020-09-25T14:17:21.430" Score="1" ViewCount="41" Body="&lt;p&gt;Let be a Bayesian multivariate normal distribution classifier with distinct covariance matrices for each class and isotropic, i.e. with equal values over the entire diagonal and zero otherwise, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\Sigma}_i=\sigma_i^2\mathbf{I},~\forall i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How can I compute the equation for estimating the parameter &lt;span class=&quot;math-container&quot;&gt;$\sigma_{i}$&lt;/span&gt; by the maximum likelihood method? Here &lt;span class=&quot;math-container&quot;&gt;$\sigma_{i,j}$&lt;/span&gt; is is the covariance between &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt;. So &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt; is just the variance of &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}_i = \{x^t_i\}^N_{t=1}$&lt;/span&gt; i.i.d, &lt;span class=&quot;math-container&quot;&gt;$x_i^t$&lt;/span&gt; is in the class &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_i^t \sim \mathcal{N}(\mu, \sigma^2)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Do I have to find the log-likelihood under &lt;span class=&quot;math-container&quot;&gt;$p(x) = \frac{1}{\sqrt{2 \pi} \sigma} \exp \left[ -\frac{(x-\mu)^2}{2 \sigma^2}\right]$&lt;/span&gt;, find the derivative and put it equal to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to find the maximum?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Suppose my data points are &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional, and I have &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; classes.&lt;/p&gt;&#xA;" OwnerUserId="41202" LastEditorUserId="41202" LastEditDate="2020-09-25T20:37:36.170" LastActivityDate="2020-09-25T20:37:36.170" Title="Estimating $\sigma_i$ according to maximum likelihood method" Tags="&lt;classification&gt;&lt;probability&gt;&lt;probability-distribution&gt;&lt;maximum-likelihood&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23755" PostTypeId="1" CreationDate="2020-09-25T14:27:45.220" Score="1" ViewCount="19" Body="&lt;p&gt;I want to use Neural network  algorithm over famous Iris dataset.&lt;/p&gt;&#xA;&lt;p&gt;Iris dataset attributes names&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;sepal length in cm&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;sepal width in cm&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;petal length in cm&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;petal width in cm&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Sample dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;5.1, 3.5, 1.4, 0.2&#xA;4.9, 3.0, 1.4, 0.2&#xA;4.7, 3.2, 1.3, 0.2&#xA;4.6, 3.1, 1.5, 0.2&#xA;5.0, 3.6, 1.4, 0.2&#xA;5.4, 3.9, 1.7, 0.4&#xA;4.6, 3.4, 1.4, 0.3&#xA;5.0, 3.4, 1.5, 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GyYDy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GyYDy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If this is the main concept of Neural Network then what will be the input for Wᵀ and b?&lt;/p&gt;&#xA;&lt;p&gt;I believe x would be a vector like&lt;/p&gt;&#xA;&lt;p&gt;X = [5.1   4.9 ....&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 3.5   3.0 ....&#xA; &#xA; 1.4   1.4 ...&#xA;&#xA; 0.2   0.2 .... ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please ignore my square bracket style as I did not find out vector styling big square bracket.&lt;/p&gt;&#xA;" OwnerUserId="18384" LastEditorUserId="18384" LastEditDate="2020-09-25T15:25:55.950" LastActivityDate="2020-09-25T15:25:55.950" Title="Neural network algorithm implementation for Iris dataset" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;logistic-regression&gt;&lt;numpy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23756" PostTypeId="1" AcceptedAnswerId="23813" CreationDate="2020-09-25T15:20:56.633" Score="6" ViewCount="145" Body="&lt;p&gt;For many problems in computer science, there is a formal, mathematical problem defition.&lt;br /&gt;&#xA;Something like: &lt;em&gt;Given ..., the problem is to ...&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can the Object Detection problem (i.e. detecting objects on an image) be formally defined?&lt;/p&gt;&#xA;&lt;p&gt;Given a set of pixels, the task is to decide&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;which pixels belong to an object at all,&lt;/li&gt;&#xA;&lt;li&gt;which pixels belong to the same object.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How can this be put into a formula?&lt;/p&gt;&#xA;" OwnerUserId="23496" LastActivityDate="2020-09-29T15:23:31.703" Title="Formal definition of the Object Detection problem" Tags="&lt;computer-vision&gt;&lt;math&gt;&lt;object-detection&gt;&lt;information-theory&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23758" PostTypeId="1" CreationDate="2020-09-25T18:59:57.903" Score="0" ViewCount="25" Body="&lt;p&gt;I am using tf.keras for a regression problem. I am building a simple feed-forward neural network with one hidden layer and tanh activation.&lt;/p&gt;&#xA;&lt;p&gt;(I started with the neural network toolbox as a part of predictive analytics offerings in SAS JMP. I was happy with the results -- my NN is 9 x 10 x 7. However, from domain knowledge, I know the outputs need to be non-negative. My SAS JMP license does not offer activation functions beyond tanh, linear, and Gaussian.)&lt;/p&gt;&#xA;&lt;p&gt;I had a hard time training a simple network in TensorFlow for my problem without normalization. (I haven't explored initializers yet.) Feature-wise normalization by subtracting the mean and dividing by the standard deviation offers far better results than just dividing by the max. (My idea was to use the latter with relu activation in a straightforward way.)&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to specify relu thresholds for each unit in the output layer so that I can ensure non-negative outputs after normalization? (In this case, it would be -mean/stddev instead of the default 0.)&lt;/p&gt;&#xA;&lt;p&gt;I also tried playing with exponential and elu with little success. Is there a best-practice approach while implementing a normalization scheme and ensuring non-negativity of the (original) outputs?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&#xA;Use of softplus instead of relu in the output layer yields better results with the normalization where I divide by max of values.&lt;/p&gt;&#xA;" OwnerUserId="41205" LastEditorUserId="41206" LastEditDate="2020-10-05T00:22:33.710" LastActivityDate="2020-10-05T00:22:33.710" Title="How to enforce non-negative outputs in a feed-forward neural network in tensorflow after appropriate scaling?" Tags="&lt;tensorflow&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23759" PostTypeId="5" CreationDate="2020-09-25T19:44:10.413" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-25T19:44:10.413" LastActivityDate="2020-09-25T19:44:10.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23760" PostTypeId="4" CreationDate="2020-09-25T19:44:10.413" Score="0" Body="For questions about binary classification problems, i.e. problems where we want to classify inputs into 1 of 2 classes." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-25T19:44:10.413" LastActivityDate="2020-09-25T19:44:10.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23761" PostTypeId="2" ParentId="5801" CreationDate="2020-09-25T19:48:52.140" Score="-1" Body="&lt;p&gt;Normally you only have two classes along with a threshold probability.&lt;/p&gt;&#xA;&lt;p&gt;It's how systems like YOLO work.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-09-25T19:48:52.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23762" PostTypeId="1" CreationDate="2020-09-25T19:57:11.993" Score="0" ViewCount="19" Body="&lt;p&gt;Pretty much the title.&#xA;I'm no expert but from what I know, if you add up enough sine functions with proper amplitudes and frequencies you can get any function you want as a result. With that knowledge, wouldn't it make sense to have neuron's activation function be a sine function?&lt;/p&gt;&#xA;" OwnerUserId="38668" LastActivityDate="2020-09-25T19:57:11.993" Title="Why is sine activation function not used frequently since we know from fourier transforms that sine functions can combine to fit any function?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23763" PostTypeId="1" CreationDate="2020-09-25T21:20:54.617" Score="1" ViewCount="11" Body="&lt;p&gt;I am designing and researching algorithms which I call of a wavefront nature. It is image analsyis agorithms when every pixel may change many times during the processing. I have heard this name before, but it seems it is not used widely. Are there other &amp;quot;wavefront algorithms&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="41207" LastActivityDate="2020-09-25T21:20:54.617" Title="What is a wavefront algorithm?" Tags="&lt;computer-vision&gt;&lt;image-processing&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23764" PostTypeId="1" CreationDate="2020-09-25T21:41:17.270" Score="0" ViewCount="113" Body="&lt;p&gt;A simple vacuum cleaner agent is having two sensors. One sensor senses the current location of the agent (floor A or floor B). The other sensor senses the status of the current location (Dirty or Clean). It is assumed that the agent can perform any one of the following three actions in a single time&#xA;unit:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Action R:&lt;/strong&gt; Agent moves from location A to location B in a single step i.e. single time unit.&lt;br /&gt;&#xA;&lt;strong&gt;Action L:&lt;/strong&gt; Agent moves from location B to location A in a single step i.e. single time unit.&lt;br /&gt;&#xA;&lt;strong&gt;Action S:&lt;/strong&gt; Agent sucks the dirt from current floor in a single step i.e. single time unit.&lt;/p&gt;&#xA;&lt;p&gt;Calculate the Performance Measure of this Vacuum Cleaner agent for each of the following four initial conditions by assuming that the agent operates for &lt;strong&gt;100 time units&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;i) [A, Clean], [B, Clean]&lt;br /&gt;&#xA;ii) [A, Dirt], [B, Clean]&lt;br /&gt;&#xA;iii) [A, Clean], [B, Dirt]&lt;br /&gt;&#xA;iv) [A, Dirt], [B, Dirt]&lt;/p&gt;&#xA;&lt;p&gt;Following criteria is assumed for measuring the performance:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In each time unit, only one action (either R or L or S) takes place by the agent.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;+2 mark&lt;/strong&gt; is rewarded to the agent for its every action (either R or L or S) till two floors are clean.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;-1 mark&lt;/strong&gt; is given to the agent for its every move (either R or L) from one location to other after both the floors are clean.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;&lt;strong&gt;My understanding of the problem:&lt;/strong&gt;&lt;/h2&gt;&#xA;&lt;p&gt;For part (i) [A,Clean],[B,Clean] can be followed by,&lt;/p&gt;&#xA;&lt;p&gt;[A,Clean],[B,Clean],[A,Clean] - Right&lt;br /&gt;&#xA;[A,Clean],[B,Clean],[A,Dirt]  - Suck&lt;br /&gt;&#xA;[A,Clean],[B,Clean],[B,Clean] - Left&lt;br /&gt;&#xA;[A,Clean],[B,Clean],[B,Dirt]  - Suck&lt;/p&gt;&#xA;&lt;p&gt;This gives us only 4 actions. So +2 for each?&lt;/p&gt;&#xA;&lt;p&gt;Since there are 100 time units given do we consider 100 actions since each action takes a single time unit? Does that mean we have to make a percept sequence with 100 actions?&lt;br /&gt;&#xA;If I assume the vacuum cleaner is a rational agent which would perform only the right actions according to the situation, then would it ever get -1 mark?&lt;/p&gt;&#xA;&lt;p&gt;I am unsure if I am going about this problem correctly. How do I calculate the performance? Do I need to make the state space tree? Please help.&lt;/p&gt;&#xA;" OwnerUserId="41209" LastActivityDate="2020-09-25T21:41:17.270" Title="How to calculate the performance measure of a vacuum cleaner agent?" Tags="&lt;intelligent-agent&gt;&lt;performance&gt;&lt;simple-reflex-agents&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23767" PostTypeId="1" CreationDate="2020-09-26T02:26:14.243" Score="1" ViewCount="12" Body="&lt;p&gt;In paper &amp;quot;&lt;a href=&quot;https://arxiv.org/abs/1906.02736&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMDP: Learning Continuous Latent Space Models for Representation Learning&lt;/a&gt;&amp;quot;, Gelada et al. state in the beginning of section 2.4&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The degree to which a value function of &lt;span class=&quot;math-container&quot;&gt;$\bar{\mathcal M}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\bar{V}^{\bar\pi}$&lt;/span&gt; approximates the value function &lt;span class=&quot;math-container&quot;&gt;$V^\bar\pi$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;${\mathcal M}$&lt;/span&gt; will depend on the Lipschitz norm of &lt;span class=&quot;math-container&quot;&gt;$\bar V^\bar\pi$&lt;/span&gt; .&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathcal M$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process&lt;/a&gt;(MDP) defined in the original state space &lt;span class=&quot;math-container&quot;&gt;$\mathcal S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\bar{\mathcal M}$&lt;/span&gt; is the MDP defined in the corresponding latent space &lt;span class=&quot;math-container&quot;&gt;$\bar{\mathcal S}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\bar\pi$&lt;/span&gt; is a policy defined on the latent space, which can be applied to &lt;span class=&quot;math-container&quot;&gt;$\mathcal M$&lt;/span&gt; by first mapping &lt;span class=&quot;math-container&quot;&gt;$s\in \mathcal S$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\bar{\mathcal S}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is how they draw the connection between &amp;quot;The degree to which a value function of &lt;span class=&quot;math-container&quot;&gt;$\bar{\mathcal M}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\bar{V}^{\bar\pi}$&lt;/span&gt; approximates the value function &lt;span class=&quot;math-container&quot;&gt;$V^\bar\pi$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;${\mathcal M}$&lt;/span&gt;&amp;quot; and &amp;quot; the Lipschitz norm of &lt;span class=&quot;math-container&quot;&gt;$\bar V^\bar\pi$&lt;/span&gt;&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="8689" LastActivityDate="2020-09-26T02:26:14.243" Title="Relation between a value function of an MDP and a value function of the corresponding latent MDP" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23769" PostTypeId="1" CreationDate="2020-09-26T08:41:21.637" Score="2" ViewCount="27" Body="&lt;p&gt;As far I know, this is how the latter two algorithms work...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Lloyd's algorithm&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose the number of clusters.&lt;/li&gt;&#xA;&lt;li&gt;Choose a distance metric (typically squared euclidean).&lt;/li&gt;&#xA;&lt;li&gt;Randomly assign each observation to a cluster and compute the cluster centroids.&lt;/li&gt;&#xA;&lt;li&gt;Iterate below until convergence (i.e. until cluster centroids stop changing):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assign each observation point to the cluster whose centroid is closest.&lt;/li&gt;&#xA;&lt;li&gt;Update cluster centroids only after a complete pass through all observations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Macqueen's Algorithm&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose the number of clusters.&lt;/li&gt;&#xA;&lt;li&gt;Choose a distance metric (typically squared euclidean).&lt;/li&gt;&#xA;&lt;li&gt;Randomly assign each observation to a cluster and compute the cluster centroids.&lt;/li&gt;&#xA;&lt;li&gt;Perform a complete pass of below (i.e. go through all observations):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assign an observation to a cluster whose centroid is closest.&lt;/li&gt;&#xA;&lt;li&gt;Immediately update the centroids for the two affected clusters (i.e. for the cluster that lost an observation and for the cluster that gained it).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Update centroids after a complete pass.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How does the Hartigan &amp;amp; Wong algorithm compare to these two above? I read this &lt;a href=&quot;http://www.tqmp.org/RegularArticles/vol09-1/p015/p015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; in an effort to understand but it's still not clear to me. The first three steps is the same as Lloyd's and Macqueen's algorithm (as described above), but then what does the algorithm do? Does it update the centroids as often as Macqueen's algorithm does, or as often as Lloyd's algorithm does? At what point does it take into consideration the within-cluster sum of squares and how does it fit into the algorithm?&lt;/p&gt;&#xA;&lt;p&gt;I'm generally confused when it comes to this algorithm and would very much appreciate a step-wise explanation as to what's going on.&lt;/p&gt;&#xA;" OwnerUserId="41219" LastActivityDate="2020-09-26T08:41:21.637" Title="How does Hartigan &amp; Wong algorithm compare to Lloyd's and Macqueen's algorithm in K-means clustering?" Tags="&lt;machine-learning&gt;&lt;algorithm&gt;&lt;clustering&gt;&lt;k-means&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23772" PostTypeId="1" AcceptedAnswerId="23916" CreationDate="2020-09-26T11:40:34.997" Score="3" ViewCount="77" Body="&lt;p&gt;I am still somewhat a novice in the ML world, but I had a strange idea about CNNs and wanted to ask if this would be a valid way to check the robustness of a general CNN that classifies certain images.&lt;/p&gt;&#xA;&lt;p&gt;Let's say that I make a CNN that takes in many different images of sports players performing a certain action (basketball shot, football kick, freestyle in swimming, flip in gymnastics, etc). Firstly, would it be possible for such a CNN to distinguish between such varied images and classify them accurately? And if so, can it be a good idea to compare this &amp;quot;larger&amp;quot; CNN to multiple &amp;quot;smaller&amp;quot; more specialized ones that take in images from one particular sport?&lt;/p&gt;&#xA;&lt;p&gt;In other words, I want to know that if I have a &amp;quot;larger&amp;quot; CNN that gives me an output like &amp;quot;football being kicked&amp;quot;, is there a way to then double-check that output with a smaller CNN that only focuses on football moves? In essence, could we create a  system where once you obtain an output from a general CNN, it automatically classifies the same image through a more specialized CNN, and then if the results are of similar accuracy, you know for sure that CNN works?&lt;/p&gt;&#xA;&lt;p&gt;Kind of like having a smaller CNN as a &amp;quot;ground-truth&amp;quot; for the bigger one? In my head it kind of goes like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;large_net_output = 'Football kick identified with 95.56% confidence' &#xA;&#xA;for sport in large_net:&#xA;    if sport == 'football':&#xA;        access = small_net_for_football&#xA;        return small_net_for_football_output&#xA;&#xA;    elif sport == 'swimming':&#xA;        access = small_net_for_swimming&#xA;        return small_net_for_swimming_output&#xA;&#xA;    elif sport == 'baseball':&#xA;        access = small_net_for_baseball&#xA;        return small_net_for_baseball_output&#xA;&#xA;# and so on....&#xA;&amp;gt;&amp;gt;&amp;gt; small_net_for_football_output = 'Football kick identified with 97.32% confidence'&#xA;&#xA;robustness_check = large_net_output - small_net_for_football_output&#xA;print(robustness_check)&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; 'Your system is accurate within a good range of 1.76%'&#xA;     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope this makes sense, and that this question does not cause any of your to cringe. Would appreciate any feedback on this!&lt;/p&gt;&#xA;" OwnerUserId="41222" LastEditorUserId="32410" LastEditDate="2021-04-26T12:23:11.870" LastActivityDate="2021-04-26T12:23:11.870" Title="Comparing a large/general CNN to a smaller more specialized one?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23773" PostTypeId="1" AcceptedAnswerId="23797" CreationDate="2020-09-26T15:45:48.927" Score="3" ViewCount="124" Body="&lt;p&gt;Consider a multi-armed bandit(MAB). There are &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; arms, with reward distributions &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$1 \leq i \leq k$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$\mu_i$&lt;/span&gt; denote the mean of the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; distribution.&lt;/p&gt;&#xA;&lt;p&gt;If we run the multi-armed bandit &lt;em&gt;experiment&lt;/em&gt;  for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; rounds, the &amp;quot;pseudo regret&amp;quot; is defined as &lt;span class=&quot;math-container&quot;&gt;$$\text{Regret}_T = \sum_{t=1}^T \mu^* - \mu_{it},$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mu^*$&lt;/span&gt; denotes the highest mean among all the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; distributions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why is regret defined like this?&lt;/strong&gt; From what I understand, at time-step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the actual reward received is &lt;span class=&quot;math-container&quot;&gt;$r_t \sim R_{it} $&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$\mu_{it}$&lt;/span&gt; - so shouldn't that be a part of the expression for regret instead?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2021-02-11T00:03:22.097" LastActivityDate="2021-02-11T00:03:22.097" Title="Why is regret so defined in MABs?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;rewards&gt;&lt;multi-armed-bandit&gt;&lt;regret&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23774" PostTypeId="1" AcceptedAnswerId="23784" CreationDate="2020-09-26T16:55:27.100" Score="2" ViewCount="98" Body="&lt;p&gt;Here is a linear regression model&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y = mx + b,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is known as &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-intercept, but also known as the &lt;strong&gt;bias&lt;/strong&gt; [&lt;a href=&quot;https://developers.google.com/machine-learning/crash-course/descending-into-ml/linear-regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the slope, and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the feature vector.&lt;/p&gt;&#xA;&lt;p&gt;As I understood, in machine learning, there is also the &lt;strong&gt;bias&lt;/strong&gt; that can cause the model to underfit.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;So, is there a connection between the bias term &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; in a linear regression model and the bias that can lead to under-fitting in machine learning?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="41226" LastEditorUserId="2444" LastEditDate="2020-09-27T10:56:16.007" LastActivityDate="2020-09-27T13:47:07.790" Title="Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?" Tags="&lt;machine-learning&gt;&lt;linear-regression&gt;&lt;bias-variance-tradeoff&gt;&lt;underfitting&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23775" PostTypeId="1" AcceptedAnswerId="23881" CreationDate="2020-09-26T16:56:12.660" Score="1" ViewCount="89" Body="&lt;p&gt;Is it legal to license and sell the output of a neural network that was trained on data that you don't own the license to?  For example, suppose you trained &lt;a href=&quot;https://en.wikipedia.org/wiki/WaveNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;WaveNet&lt;/a&gt; on a collection of popular music. Could you then sell the audio that the WaveNet produces? There are &lt;a href=&quot;https://en.wikipedia.org/wiki/Sampling_(music)#Legal_and_ethical_issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;copyright restrictions on using samples&lt;/a&gt; to produce music, but the output of a generative neural network might not include any exact replicas from the training data, so it's not clear to me whether those laws apply.&lt;/p&gt;&#xA;" OwnerUserId="41227" LastEditorUserId="2444" LastEditDate="2020-10-03T14:58:10.257" LastActivityDate="2020-10-03T14:58:10.257" Title="Is it legal to license and sell the output of a neural network that was trained on data that you don't own the license to?" Tags="&lt;neural-networks&gt;&lt;generative-model&gt;&lt;legal&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23777" PostTypeId="1" AcceptedAnswerId="23788" CreationDate="2020-09-26T17:57:08.653" Score="3" ViewCount="69" Body="&lt;p&gt;A model can be classified as parametric or non-parametric. How are models classified as parametric and non-parametric models? What is the difference between the two approaches?&lt;/p&gt;&#xA;" OwnerUserId="41228" LastEditorUserId="2444" LastEditDate="2020-09-26T21:36:05.317" LastActivityDate="2020-09-27T10:37:32.503" Title="What is the difference between parametric and non-parametric models?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;definitions&gt;&lt;models&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23778" PostTypeId="1" AcceptedAnswerId="23790" CreationDate="2020-09-26T18:58:33.010" Score="5" ViewCount="488" Body="&lt;p&gt;&lt;em&gt;Boosting&lt;/em&gt; refers to a family of algorithms which converts &lt;em&gt;weak learners&lt;/em&gt; to &lt;em&gt;strong learners&lt;/em&gt;. How does it happen?&lt;/p&gt;&#xA;" OwnerUserId="41228" LastEditorUserId="2444" LastEditDate="2020-09-26T21:28:30.847" LastActivityDate="2020-09-29T02:31:58.360" Title="How do weak learners become strong in boosting?" Tags="&lt;machine-learning&gt;&lt;gradient-boosting&gt;&lt;boosting&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23781" PostTypeId="2" ParentId="23778" CreationDate="2020-09-26T20:09:16.093" Score="1" Body="&lt;p&gt;You take a bunch of weak learners, each of them trained on a subset of the data.&lt;/p&gt;&#xA;&lt;p&gt;You then just get all of them to make a prediction, and you learn how much you can trust each one, resulting in a weighted vote or other type of combination of the individual predictions.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-09-26T20:09:16.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23782" PostTypeId="1" CreationDate="2020-09-26T20:18:14.873" Score="1" ViewCount="23" Body="&lt;p&gt;From statistical mechanics, the Boltzmann distribution over a system's energy states arises from the assumption that many replicas of the system are exchanging energy with each other.  The distribution of these replicas in each energy level is the maximum entropy distribution subject to the constraint that their total energy is fixed, and that any one assignment of energy levels to each replica, a &amp;quot;microstate&amp;quot;, satisfying this constraint, is equally probable.&lt;/p&gt;&#xA;&lt;p&gt;From machine learning, the so-called Energy-based model defines a Hamiltonian (energy function) to its various configurations, and uses Boltzmann's distribution to convert an &amp;quot;energy&amp;quot; to a probability over these configurations.  Thus, an EBM can model a probability distribution over some data domain.&lt;/p&gt;&#xA;&lt;p&gt;Is there some viewpoint by which one can interpret the EBM as a &amp;quot;system&amp;quot; exchanging energy with many other replicas of that system?  What semantic interpretation of EBMs connects them to the Boltzmann distribution's assumptions?&lt;/p&gt;&#xA;" OwnerUserId="41231" LastActivityDate="2020-09-26T20:18:14.873" Title="How are Energy Based models really connected to Statistical Mechanics?" Tags="&lt;boltzmann-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23784" PostTypeId="2" ParentId="23774" CreationDate="2020-09-26T22:22:09.300" Score="4" Body="&lt;p&gt;In machine learning, the term &lt;strong&gt;bias&lt;/strong&gt; can refer to at least 2 related concepts&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A (learnable) &lt;strong&gt;parameter&lt;/strong&gt; of a model, such as a linear regression model, which allows you to learn a shifted function. For example, in the case of a linear regression model &lt;span class=&quot;math-container&quot;&gt;$y = f(x) = mx + b$&lt;/span&gt;, the bias &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; allows you to shift the straight-line up an down: without the bias, you would only be able to control the slope &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; of the straight-line. Similarly, in a neural network, you can have a neuron that performs a linear combination of the inputs, then it uses a bias term to shift the straight-line, and you could also use the bias after having applied the activation function, but this will have a different effect.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Anything that &lt;strong&gt;guides&lt;/strong&gt; the learning algorithm (e.g. gradient descent with back-propagation) towards a &lt;em&gt;specific&lt;/em&gt; set of solutions. For example, if you use &lt;strong&gt;regularization&lt;/strong&gt;, you are biasing the learning algorithm to choose, typically, smoother or simpler functions. The bias term in the linear regression model is also a way of biasing the learning algorithm: you assume that the straight-line function does not necessarily go through zero, and this assumption affects the type of functions that you can learn (and this is why these two concepts of &lt;em&gt;bias&lt;/em&gt; are related!). So, there are many ways of biasing a learning algorithm.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The bias does not always lead you to the best solutions and can actually lead to the wrong solutions, but it is often useful in many ways, for example, it can speed up learning, as you restrict the number of functions that can be learned and searched. The bias (as described in point 2) is often discussed in the context of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff&quot; rel=&quot;nofollow noreferrer&quot;&gt;bias-variance trade-off&lt;/a&gt;, and both the &lt;em&gt;bias&lt;/em&gt; and the &lt;em&gt;variance&lt;/em&gt; are related to the concepts of &lt;em&gt;generalization, under-fitting&lt;/em&gt;, and &lt;em&gt;over-fitting&lt;/em&gt;. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff&quot; rel=&quot;nofollow noreferrer&quot;&gt;linked Wikipedia article&lt;/a&gt; explains these concepts quite well and provides an example of how the bias and variance are decomposed, so you should probably read that article for more details.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-27T13:47:07.790" LastActivityDate="2020-09-27T13:47:07.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23786" PostTypeId="1" CreationDate="2020-09-26T23:02:50.397" Score="3" ViewCount="29" Body="&lt;p&gt;This is a bit of a soft question, not sure if it's on topic, please let me know how I can improve it if it doesn't meet the criteria for the site.&lt;/p&gt;&#xA;&lt;p&gt;GPT models are unsupervised in nature and are (from my understanding) given a prompt and then they either answer the question or continue the sentence/paragraph. They also seem to be the most advanced models for producing natural language, capable of giving outputs with correct syntax and (to my eye at least) indistinguishable from something written by a human (sometimes at least!).&lt;/p&gt;&#xA;&lt;p&gt;However if I have a problem where I have an input (could be anything, but lets call it an image or video) and a description of the image or video as the output I could in theory train a model with convolutional filters to identify the object and describe the image (assuming any test data is within the bounds of the training data). However when I've seen models like this in the past the language is either quite simple or 'feels' like it's been produced by a machine.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way to either train a GPT model as a supervised learning model&lt;/strong&gt; with inputs (of some &lt;strong&gt;non language&lt;/strong&gt; type) and outputs (of sentences/paragraphs); or a similar type of &lt;strong&gt;machine learning model that can be used for this task&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;A few notes:&lt;/p&gt;&#xA;&lt;p&gt;I have seen the deep learning image captioning methods - these are what I mention above. I'm more looking for something that can take an input-output pair where the output is text and the input is any form.&lt;/p&gt;&#xA;" OwnerUserId="10505" LastEditorUserId="1847" LastEditDate="2020-09-27T07:20:08.743" LastActivityDate="2020-09-27T07:20:08.743" Title="Is there a complement to GPT/2/3 that can be trained using supervised learning methods?" Tags="&lt;natural-language-processing&gt;&lt;models&gt;&lt;natural-language-understanding&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23787" PostTypeId="2" ParentId="14359" CreationDate="2020-09-27T01:31:14.003" Score="1" Body="&lt;p&gt;The standard tool to work with XML files is XSLT. You may not need AI to solve this problem. But.. you have to learn how to program with XSLT ;)&#xA;On Windows you can use MSXML, if you work from C++ - msxsl.exe, C# has internal supoort for XSLT. That is what I know about. There are also non-MS tools.&lt;/p&gt;&#xA;" OwnerUserId="41207" LastEditorUserId="41207" LastEditDate="2020-09-27T02:42:16.500" LastActivityDate="2020-09-27T02:42:16.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23788" PostTypeId="2" ParentId="23777" CreationDate="2020-09-27T02:27:15.083" Score="5" Body="&lt;h1&gt;Parametric Methods&lt;/h1&gt;&#xA;&lt;p&gt;A parametric approach (Regression, Linear Support Vector Machines) has a fixed number of parameters and it makes a lot of assumptions about the data. This is because they are used for known data distributions. i.e, it makes a lot of presumptions about the data&lt;/p&gt;&#xA;&lt;h1&gt;Non-Parametric Methods&lt;/h1&gt;&#xA;&lt;p&gt;A non-parametric approach (k-Nearest Neighbours, Decision Trees) has a flexible number of parameters, there are no presumptions about the data distribution. The model tries to &amp;quot;explore&amp;quot; the distribution and thus has a flexible number of parameters.&lt;/p&gt;&#xA;&lt;h2&gt;Comparision&lt;/h2&gt;&#xA;&lt;p&gt;Comparatively speaking, parametric approaches are computationally faster and have more statistical power when compared to non-parametric methods&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Hope this cleared your doubts &lt;/p&gt;&#xA;" OwnerUserId="40434" LastEditorUserId="2444" LastEditDate="2020-09-27T10:37:32.503" LastActivityDate="2020-09-27T10:37:32.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23790" PostTypeId="2" ParentId="23778" CreationDate="2020-09-27T03:00:37.663" Score="6" Body="&lt;p&gt;As @desertnaut mentioned in the comment&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;No weak learner becomes strong; it is the ensemble of the weak learners that turns out to be strong&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Boosting is an ensemble method that integrates multiple models(called as weak learners) to produce a supermodel (Strong learner).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tdxd7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tdxd7.png&quot; alt=&quot;Boosting technique is illustrated&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Basically boosting is to train weak learners sequentially,  each trying to correct its predecessor. For boosting, we need to specify a weak model (e.g. regression, shallow decision trees, etc.), and then we try to improve each weak learner to learn something from the data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;AdaBoost&lt;/em&gt; is a boosting algorithm where a decision tree with a single split is used as a weak learner. Also, we have &lt;em&gt;gradient boosting&lt;/em&gt; and &lt;em&gt;XG boosting&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41226" LastEditorUserId="41226" LastEditDate="2020-09-29T02:31:58.360" LastActivityDate="2020-09-29T02:31:58.360" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23791" PostTypeId="2" ParentId="23778" CreationDate="2020-09-27T03:02:01.313" Score="1" Body="&lt;p&gt;In Boosting, we improve the overall metrics of the model by sequentially building weak models and then building upon the weak metrics of previous models.&lt;/p&gt;&#xA;&lt;p&gt;We start out by applying basic non-specific algorithms to the problem, which returns some weak prediction functions by taking arbitrary solutions (like sparse weights or assigning equal weights/attention). We improve upon this in the following predictions by adjusting weights to those having a higher error rate. After going through many iterations, we combine it to create a single Strong Prediction Function which has better metrics.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Some popular Boosting Algorithms :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AdaBoost&lt;/li&gt;&#xA;&lt;li&gt;Gradient Tree Boosting&lt;/li&gt;&#xA;&lt;li&gt;XGBoost&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-09-27T03:02:01.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23792" PostTypeId="1" CreationDate="2020-09-27T10:05:28.783" Score="1" ViewCount="49" Body="&lt;p&gt;For weeks I've been working with this toy game of &lt;code&gt;Rock-Paper-Scissor&lt;/code&gt;. I want to use a &lt;code&gt;PPO&lt;/code&gt; agent learn to beat a computer opponent whose logic is defined as the code bellow.&lt;/p&gt;&#xA;&lt;p&gt;For short, this computer opponent, named &lt;code&gt;abbey&lt;/code&gt;, uses a strategy that tracks all two consecutive plays of the agent, and gives the opposite play of the most likely guess of the agent's next play, according to the agent's last play.&lt;/p&gt;&#xA;&lt;p&gt;I design the agent(using gym env) to have an internal state, keeping track of all counts of its two consecutive plays, in a 3x3 matrix. And then I normalized each row of the matrix to be an observation of the agent, representing the probabilities of the second play given the previous one. So the agent will get the same knowledge as what &lt;code&gt;abbey&lt;/code&gt; knows.&lt;/p&gt;&#xA;&lt;p&gt;Then I copied an &lt;code&gt;PPO&lt;/code&gt; network algorithm from some RL book, which works well with &lt;code&gt;CartPole&lt;/code&gt;. Then I did some minor changes which are commented in the code bellow.&lt;/p&gt;&#xA;&lt;p&gt;But the algorithm does not converge even a little, and &lt;code&gt;abbey&lt;/code&gt; always wins the agent about 60% of the time from first run to the last.&lt;/p&gt;&#xA;&lt;p&gt;I doubt the state and observation space I designed is the reason why it does not converge. All I get is that the agent maybe should find something from the histories of its own successes and fails, and find its way out.&lt;/p&gt;&#xA;&lt;p&gt;Can you give me some advice for the designing of a state space?&#xA;Thank you very much.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;### define a Rock-Paper-Scissor opponent&#xA;&#xA;abbey_state = []&#xA;play_order=[{&#xA;              &amp;quot;RR&amp;quot;: 0,&#xA;              &amp;quot;RP&amp;quot;: 0,&#xA;              &amp;quot;RS&amp;quot;: 0,&#xA;              &amp;quot;PR&amp;quot;: 0,&#xA;              &amp;quot;PP&amp;quot;: 0,&#xA;              &amp;quot;PS&amp;quot;: 0,&#xA;              &amp;quot;SR&amp;quot;: 0,&#xA;              &amp;quot;SP&amp;quot;: 0,&#xA;              &amp;quot;SS&amp;quot;: 0,&#xA;          }]&#xA;def abbey(prev_opponent_play,&#xA;          re_init=False):&#xA;    if not prev_opponent_play:&#xA;        prev_opponent_play = 'R'&#xA;    global abbey_state, play_order&#xA;    if re_init:&#xA;        abbey_state = []&#xA;        play_order=[{&#xA;              &amp;quot;RR&amp;quot;: 0,&#xA;              &amp;quot;RP&amp;quot;: 0,&#xA;              &amp;quot;RS&amp;quot;: 0,&#xA;              &amp;quot;PR&amp;quot;: 0,&#xA;              &amp;quot;PP&amp;quot;: 0,&#xA;              &amp;quot;PS&amp;quot;: 0,&#xA;              &amp;quot;SR&amp;quot;: 0,&#xA;              &amp;quot;SP&amp;quot;: 0,&#xA;              &amp;quot;SS&amp;quot;: 0,&#xA;          }]&#xA;    abbey_state.append(prev_opponent_play)&#xA;    last_two = &amp;quot;&amp;quot;.join(abbey_state[-2:])&#xA;    if len(last_two) == 2:&#xA;        play_order[0][last_two] += 1&#xA;    potential_plays = [&#xA;        prev_opponent_play + &amp;quot;R&amp;quot;,&#xA;        prev_opponent_play + &amp;quot;P&amp;quot;,&#xA;        prev_opponent_play + &amp;quot;S&amp;quot;,&#xA;    ]&#xA;    sub_order = {&#xA;        k: play_order[0][k]&#xA;        for k in potential_plays if k in play_order[0]&#xA;    }&#xA;    prediction = max(sub_order, key=sub_order.get)[-1:]&#xA;    ideal_response = {'P': 'S', 'R': 'P', 'S': 'R'}&#xA;    return ideal_response[prediction]&#xA;&#xA;&#xA;### define the gym env&#xA;import gym&#xA;from gym import spaces&#xA;from collections import defaultdict&#xA;import numpy as np&#xA;&#xA;ACTIONS = [&amp;quot;R&amp;quot;, &amp;quot;P&amp;quot;, &amp;quot;S&amp;quot;]&#xA;games = 1000&#xA;&#xA;class RockPaperScissorsEnv(gym.Env):&#xA;  metadata = {'render.modes': ['human']}&#xA;&#xA;  def __init__(self):&#xA;    super(RockPaperScissorsEnv, self).__init__()&#xA;    self.action_space = spaces.Discrete(3)&#xA;    self.observation_space = spaces.Box(low=0.0, high=1.0,&#xA;                                        shape=(3,3), dtype=float)&#xA;    self.reset()&#xA;&#xA;  def step(self, actions):&#xA;    assert actions == 0 or actions == 1 or actions == 2&#xA;    opponent_play = self.opponent_play()&#xA;&#xA;    self.prev_plays[self.prev_actions * 3 + actions] += 1&#xA;    reward = self.calc_reward(actions, opponent_play)&#xA;    terminal = False&#xA;&#xA;    self.calc_state(self.timestep, opponent_play, actions)&#xA;    self.prev_actions = actions&#xA;    self.prev_opponent_play = opponent_play&#xA;    self.timestep += 1&#xA;    return self.get_ob(), reward, terminal, None&#xA;&#xA;  def reset(self):&#xA;    self.opponent = abbey&#xA;    self.timestep = 0&#xA;    self.prev_opponent_play = 0&#xA;    self.prev_actions = 0&#xA;    self.prev_plays = defaultdict(int)&#xA;    self.init_state = np.zeros((3,3), dtype=int)&#xA;    # the internal state&#xA;    self.state = np.copy(self.init_state)&#xA;    self.results = {&amp;quot;win&amp;quot;: 0, &amp;quot;lose&amp;quot;: 0, &amp;quot;tie&amp;quot;: 0}&#xA;    return self.get_ob()&#xA;&#xA;  def render(self, mode='human'):&#xA;    pass&#xA;&#xA;  def close (self):&#xA;    pass&#xA;&#xA;  def calc_reward(self, actions, play):&#xA;    if self.timestep % games == games - 1:&#xA;      pass&#xA;    if actions == play:&#xA;      self.results['tie'] += 1&#xA;      return 0&#xA;    elif actions == 0 and play == 1:&#xA;      self.results['lose'] += 1&#xA;      return -0.3&#xA;    elif actions == 1 and play == 2:&#xA;      self.results['lose'] += 1&#xA;      return -0.3&#xA;    elif actions == 2 and play == 0:&#xA;      self.results['lose'] += 1&#xA;      return -0.3&#xA;    elif (actions == 1 and play == 0) or (actions == 2 and play == 1) or (actions == 0 and play == 2):&#xA;      self.results['win'] += 1&#xA;      return 0.3&#xA;    else:&#xA;      raise NotImplementedError('calc_reward something get wrong')&#xA;&#xA;  def opponent_play(self):&#xA;    re_init = (self.timestep == 0)&#xA;    opp_play = self.opponent(ACTIONS[self.prev_actions], re_init=re_init)&#xA;    return ACTIONS.index(opp_play)&#xA;&#xA;  def calc_state(self, timestep, opponent_play, actions):&#xA;    self.state[self.prev_actions][actions] += 1&#xA;&#xA;  def get_ob(self):&#xA;    '''return observations'''&#xA;    state0 = self.state[0]&#xA;    sum0 = state0.sum()&#xA;    state1 = self.state[1]&#xA;    sum1 = state1.sum()&#xA;    state2 = self.state[2]&#xA;    sum2 = state2.sum()&#xA;    init = np.ones(3, dtype=float) / 3.0&#xA;    ob = np.array([&#xA;      state0 / sum0 if sum0 else init,&#xA;      state1 / sum1 if sum1 else init,&#xA;      state2 / sum2 if sum2 else init,&#xA;    ])&#xA;    # print(ob)&#xA;    return ob&#xA;&#xA;&#xA;&#xA;### Learning Algo copied from some book&#xA;&#xA;import  matplotlib&#xA;from    matplotlib import pyplot as plt&#xA;matplotlib.rcParams['font.size'] = 18&#xA;matplotlib.rcParams['figure.titlesize'] = 18&#xA;matplotlib.rcParams['figure.figsize'] = [9, 7]&#xA;matplotlib.rcParams['axes.unicode_minus']=False&#xA;&#xA;plt.figure()&#xA;&#xA;import  gym,os&#xA;import  numpy as np&#xA;import  tensorflow as tf&#xA;from    tensorflow import keras&#xA;from    tensorflow.keras import layers,optimizers,losses&#xA;from    collections import namedtuple&#xA;env = RockPaperScissorsEnv()&#xA;env.seed(2222)&#xA;tf.random.set_seed(2222)&#xA;np.random.seed(2222)&#xA;os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'&#xA;assert tf.__version__.startswith('2.')&#xA;&#xA;&#xA;&#xA;gamma = 0.98&#xA;epsilon = 0.2&#xA;batch_size = 32&#xA;&#xA;Transition = namedtuple('Transition', ['state', 'action', 'a_log_prob', 'reward', 'next_state'])&#xA;&#xA;class Actor(keras.Model):&#xA;    def __init__(self):&#xA;        super(Actor, self).__init__()&#xA;        self.fc1 = layers.Dense(18, kernel_initializer='he_normal') # I changed 100 to 18&#xA;        self.fc2 = layers.Dense(3, kernel_initializer='he_normal') # I changed 4 to 3&#xA;&#xA;    def call(self, inputs):&#xA;        x = tf.nn.relu(self.fc1(inputs))&#xA;        x = self.fc2(x)&#xA;        x = tf.nn.softmax(x, axis=1)&#xA;        return x&#xA;&#xA;class Critic(keras.Model):&#xA;    def __init__(self):&#xA;        super(Critic, self).__init__()&#xA;        self.fc1 = layers.Dense(18, kernel_initializer='he_normal') # I changed 100 to 18&#xA;        self.fc2 = layers.Dense(1, kernel_initializer='he_normal')&#xA;&#xA;    def call(self, inputs):&#xA;        x = tf.nn.relu(self.fc1(inputs))&#xA;        x = self.fc2(x)&#xA;        return x&#xA;&#xA;&#xA;&#xA;&#xA;class PPO():&#xA;    def __init__(self):&#xA;        super(PPO, self).__init__()&#xA;        self.actor = Actor()&#xA;        self.critic = Critic()&#xA;        self.buffer = []&#xA;        self.actor_optimizer = optimizers.Adam(1e-3)&#xA;        self.critic_optimizer = optimizers.Adam(3e-3)&#xA;&#xA;    def select_action(self, s):&#xA;        s = tf.constant(s, dtype=tf.float32)&#xA;        # s = tf.expand_dims(s, 0)   # I removed this line, otherwise we will get a (1,3,3) tensor and later we will get an error&#xA;        prob = self.actor(s)&#xA;        a = tf.random.categorical(tf.math.log(prob), 1)[0]&#xA;        a = int(a)&#xA;        return a, float(prob[0][a])&#xA;&#xA;    def get_value(self, s):&#xA;        s = tf.constant(s, dtype=tf.float32)&#xA;        s = tf.expand_dims(s, axis=0)&#xA;        v = self.critic(s)[0]&#xA;        return float(v)&#xA;&#xA;    def store_transition(self, transition):&#xA;        self.buffer.append(transition)&#xA;&#xA;    def optimize(self):&#xA;        state = tf.constant([t.state for t in self.buffer], dtype=tf.float32)&#xA;        action = tf.constant([t.action for t in self.buffer], dtype=tf.int32)&#xA;        action = tf.reshape(action,[-1,1])&#xA;        reward = [t.reward for t in self.buffer]&#xA;        old_action_log_prob = tf.constant([t.a_log_prob for t in self.buffer], dtype=tf.float32)&#xA;        old_action_log_prob = tf.reshape(old_action_log_prob, [-1,1])&#xA;&#xA;        R = 0&#xA;        Rs = []&#xA;        for r in reward[::-1]:&#xA;            R = r + gamma * R&#xA;            Rs.insert(0, R)&#xA;        Rs = tf.constant(Rs, dtype=tf.float32)&#xA;&#xA;        for _ in range(round(10*len(self.buffer)/batch_size)):&#xA;&#xA;            index = np.random.choice(np.arange(len(self.buffer)), batch_size, replace=False)&#xA;&#xA;            with tf.GradientTape() as tape1, tf.GradientTape() as tape2:&#xA;&#xA;                v_target = tf.expand_dims(tf.gather(Rs, index, axis=0), axis=1)&#xA;&#xA;                v = self.critic(tf.gather(state, index, axis=0))&#xA;                delta = v_target - v&#xA;                advantage = tf.stop_gradient(delta)&#xA;                a = tf.gather(action, index, axis=0)&#xA;                pi = self.actor(tf.gather(state, index, axis=0)) &#xA;                indices = tf.expand_dims(tf.range(a.shape[0]), axis=1)&#xA;                indices = tf.concat([indices, a], axis=1)&#xA;                pi_a = tf.gather_nd(pi, indices)&#xA;                pi_a = tf.expand_dims(pi_a, axis=1)&#xA;                # Importance Sampling&#xA;                ratio = (pi_a / tf.gather(old_action_log_prob, index, axis=0))&#xA;                surr1 = ratio * advantage&#xA;                surr2 = tf.clip_by_value(ratio, 1 - epsilon, 1 + epsilon) * advantage&#xA;                policy_loss = -tf.reduce_mean(tf.minimum(surr1, surr2))&#xA;                value_loss = losses.MSE(v_target, v)&#xA;            grads = tape1.gradient(policy_loss, self.actor.trainable_variables)&#xA;            self.actor_optimizer.apply_gradients(zip(grads, self.actor.trainable_variables))&#xA;            grads = tape2.gradient(value_loss, self.critic.trainable_variables)&#xA;            self.critic_optimizer.apply_gradients(zip(grads, self.critic.trainable_variables))&#xA;&#xA;        self.buffer = []&#xA;&#xA;&#xA;def main():&#xA;    agent = PPO()&#xA;    returns = []&#xA;    total = 0&#xA;    for i_epoch in range(500):&#xA;        state = env.reset()&#xA;        for t in range(games):&#xA;            action, action_prob = agent.select_action(state)&#xA;            if t == 999:&#xA;              print(action, action_prob)&#xA;            next_state, reward, done, _ = env.step(action)&#xA;            # print(next_state, reward, done, action)&#xA;            trans = Transition(state, action, action_prob, reward, next_state)&#xA;            agent.store_transition(trans)&#xA;            state = next_state&#xA;            total += reward&#xA;            if done:&#xA;                if len(agent.buffer) &amp;gt;= batch_size:&#xA;                    agent.optimize()&#xA;                break&#xA;        print(env.results)&#xA;&#xA;        if i_epoch % 20 == 0:&#xA;            returns.append(total/20)&#xA;            total = 0&#xA;            print(i_epoch, returns[-1])&#xA;&#xA;    print(np.array(returns))&#xA;    plt.figure()&#xA;    plt.plot(np.arange(len(returns))*20, np.array(returns))&#xA;    plt.plot(np.arange(len(returns))*20, np.array(returns), 's')&#xA;    plt.xlabel('epochs')&#xA;    plt.ylabel('total return')&#xA;    plt.savefig('ppo-tf.svg')&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;    print(&amp;quot;end&amp;quot;)&#xA;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41152" LastEditorUserId="41152" LastEditDate="2020-09-28T13:23:20.460" LastActivityDate="2020-09-28T13:23:20.460" Title="How to design an observation(state) space for a simple `Rock-Paper-Scissor` game?" Tags="&lt;tensorflow&gt;&lt;deep-rl&gt;&lt;actor-critic-methods&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23793" PostTypeId="1" CreationDate="2020-09-27T10:58:06.053" Score="3" ViewCount="35" Body="&lt;p&gt;In Sutton and Barto's Book in chapter 12, they state that if weights sum to 1, then an equation's updates have &amp;quot;guaranteed convergence properties&amp;quot;. Actually why it ensures convergence?&lt;/p&gt;&#xA;&lt;p&gt;There is a full citation from the mentioned fragment in Richard S. Sutton and Andrew G. Barto. Second Edition:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Now we note that a valid update can be done not just toward any n-step return, but toward any average of n-step returns for different ns. For example, an update can be done toward a target that is half of a two-step return and half of a four-step return: &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}G_{t:t+2} + \frac{1}{2}G_{t:t+4}$&lt;/span&gt;. Any set of n-step returns can be averaged in this way, even an infinite set, as long as the weights on the component returns are positive and sum to 1. The composite return possesses an error reduction property similar to that of individual n-step returns (7.3) and thus can be used to construct updates with guaranteed convergence properties.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="31324" LastActivityDate="2020-09-27T10:58:06.053" Title="Why weighting by lambda that sums to 1 ensures convergence in eligibility trace?" Tags="&lt;reinforcement-learning&gt;&lt;eligibility-traces&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23794" PostTypeId="2" ParentId="23745" CreationDate="2020-09-27T11:12:54.590" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Q-learning is said to be &amp;quot;model-free&amp;quot;. Given the two examples above, is it because neither the lake's topology nor that of the mountain are changed by the actions taken?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. That's not why Q-learning is model-free. Q-learning assumes that the underlying environment (FrozenLake or MountainCar, for example) can be modelled as a &lt;strong&gt;Markov decision process (MDP)&lt;/strong&gt;, which is a mathematical model that describes problems where decisions/actions can be taken and the outcomes of those decisions are at least partially stochastic (or random). More precisely, an MDP is composed of&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A set of actions &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (that the RL agent can take); for example, up and down, in some grid world&lt;/li&gt;&#xA;&lt;li&gt;A set of states &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (where the RL agent can be);&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;transition function&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$p(s_{t+1} = s' \mid s_{t} = s , a_t = a)$&lt;/span&gt; (aka the &lt;strong&gt;model&lt;/strong&gt;), which represents the probability of going to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; at time step &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;, given that at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; the RL agent is in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and takes action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;reward function&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt; (sometimes also denoted as &lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$r(s, s')$&lt;/span&gt;, although &lt;a href=&quot;https://ai.stackexchange.com/q/10442/2444&quot;&gt;these can have different semantics&lt;/a&gt;); the reward function gives the reward (or reinforcement) to the RL agent when it takes an action in a certain state and moves to another state; the reward function can also be included in the transition function, i.e., often you will also see &lt;span class=&quot;math-container&quot;&gt;$p(s_{t+1} = s', r_{t+1} = r \mid s_{t} = s , a_t = a)$&lt;/span&gt;, and this is the &lt;strong&gt;model&lt;/strong&gt;: this is what we mean by &lt;strong&gt;model&lt;/strong&gt; in reinforcement learning, it's this &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (which is a probability distribution)!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A model-free algorithm is any algorithm that does not use or estimate this &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. Q-learning, &lt;a href=&quot;http://www.incompleteideas.net/book/RLbook2020.pdf#page=153&quot; rel=&quot;nofollow noreferrer&quot;&gt;if you look at its pseudocode&lt;/a&gt;, does not make use of this model. Q-learning estimates the value function &lt;span class=&quot;math-container&quot;&gt;$q(s, a)$&lt;/span&gt; by interacting with the environment (taking actions and receiving rewards), but, meanwhile, it does not know or keep track of the dynamics (i.e. &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;) of the environment, and that's why it's model-free.&lt;/p&gt;&#xA;&lt;p&gt;And, no, the value function is not what we mean by &amp;quot;model&amp;quot; in reinforcement learning. The value function is, as the name suggests, a function.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How does one know that a problem is &amp;quot;model-free&amp;quot; in reinforcement learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A problem is not model-free or model-based. An &lt;strong&gt;algorithm&lt;/strong&gt; is model-free or model-based. Again, a model-free algorithm does not use or estimate &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, a model-based one uses (and/or estimates) it.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given the two examples above, is it because neither the lake's topology nor that of the mountain are changed by the actions taken?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. As stated in the &lt;a href=&quot;https://ai.stackexchange.com/a/23749/2444&quot;&gt;other answer&lt;/a&gt;, you could apply the model-based algorithm Dyna-Q to these environments.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-27T13:35:53.027" LastActivityDate="2020-09-27T13:35:53.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23797" PostTypeId="2" ParentId="23773" CreationDate="2020-09-27T15:41:56.427" Score="3" Body="&lt;p&gt;In short, you don't regret your bad luck that you could do nothing about, you regret your bad choices that you could have done something about if only you knew.&lt;/p&gt;&#xA;&lt;p&gt;The point of regret as a metric therefore is to compare your choices with the ideal choices. This makes sense in MABs, because although the primary goal is to gain the most reward, the &lt;em&gt;learning&lt;/em&gt; part of the goal is to calculate from experience what are the best choices - usually whilst sacrificing as little as possible in the process.&lt;/p&gt;&#xA;&lt;p&gt;The formula captures that concept, so does not concern itself with individual rewards in the past that could have been due to good or bad luck. Hence it uses expected (or mean) rewards.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-27T15:50:53.120" LastActivityDate="2020-09-27T15:50:53.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23798" PostTypeId="1" CreationDate="2020-09-27T17:56:39.083" Score="0" ViewCount="21" Body="&lt;p&gt;As far as I can tell, most NLP tasks today use word embeddings and recurrent networks or transformers.&lt;/p&gt;&#xA;&lt;p&gt;Are there any examples of state-of-the-art NLP applications that are still n-gram based and use Naive Bayes?&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="2444" LastEditDate="2020-09-27T20:50:15.617" LastActivityDate="2020-09-27T20:50:15.617" Title="Are there any examples of state-of-the-art NLP applications that are still n-gram based and use Naive Bayes?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;state-of-the-art&gt;&lt;naive-bayes&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23799" PostTypeId="1" CreationDate="2020-09-27T22:37:06.030" Score="0" ViewCount="44" Body="&lt;p&gt;I have an agent (drone) that has to allocate subchannels for different types of User Equipment.&lt;/p&gt;&#xA;&lt;p&gt;I have represented the subchannel allocation with a 2-dimentional binary matrix, that is initialized to all zeros as there is no requests at the beginning of the episode.&lt;/p&gt;&#xA;&lt;p&gt;When the agent chooses an action, it has to choose which subchannels to allocate to which UEs, hence populating the matrix with 1s.&lt;/p&gt;&#xA;&lt;p&gt;I have no idea how to do it.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastActivityDate="2021-02-26T19:07:45.903" Title="How to let the agent choose how to populate a state space matrix in RL (using python)" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23800" PostTypeId="1" CreationDate="2020-09-28T00:39:12.807" Score="0" ViewCount="19" Body="&lt;p&gt;I have an action space that is a matrix. I am struggling with finding how to choose how to take the action when the number of possible actions is huge.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastActivityDate="2020-09-28T00:39:12.807" Title="How to design the step() method in a custom-built environment when the action space is a 2-dimentional matrix?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;environment&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23801" PostTypeId="1" CreationDate="2020-09-28T00:57:57.983" Score="2" ViewCount="25" Body="&lt;p&gt;I see so many papers claim to have an algorithm that beats 'human-level performance' for semantic segmentation tasks, but I can't find any papers reporting on what the human-level performance actually is. An analysis of the similarity between segmentations drawn by multiple different human experts would be good. Could someone point me towards a paper that reports on something like that?&lt;/p&gt;&#xA;" OwnerUserId="9983" LastActivityDate="2020-09-28T00:57:57.983" Title="What is human-level performance for semantic segmentation?" Tags="&lt;image-segmentation&gt;&lt;performance&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23802" PostTypeId="2" ParentId="5861" CreationDate="2020-09-28T09:46:25.133" Score="1" Body="&lt;p&gt;Check out Figure 6 in this paper: &lt;a href=&quot;https://arxiv.org/abs/2006.15704&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch Distributed: Experiences on Accelerating Data Parallel Training&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It breaks down the latency of the forward pass, the backward pass, the communication step, and the optimization step for running both ResNet50 and BERT on a NVIDIA Tesla V100 GPUs.&lt;/p&gt;&#xA;&lt;p&gt;From measuring the pixels in the figure, I estimated the times for the forward, backward, and optimization steps as a percentage of their total time combined. (I ignored the communication step shown in the figure because that was only to show how long an unoptimized communication step would take when doing data-parallel training). Here are the estimates I got:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Forward: 23%&lt;/li&gt;&#xA;&lt;li&gt;Backward: 74%&lt;/li&gt;&#xA;&lt;li&gt;Optimization: 3%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So the backward pass takes about 3x as long as the forward pass, and the optimization step is relatively fast.&lt;/p&gt;&#xA;" OwnerUserId="41264" LastEditorUserId="41264" LastEditDate="2020-10-01T02:29:38.680" LastActivityDate="2020-10-01T02:29:38.680" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23803" PostTypeId="1" CreationDate="2020-09-28T10:05:19.477" Score="1" ViewCount="60" Body="&lt;p&gt;I have been looking into transformers lately and been reading tons of tutorials. All of them address the intuition behind attention, which I understand, but they treat training the weight matrices for query, key and value like it is the most trivial thing? So how are they trained? Is the error just backpropagated the weights are updated accordingly?&lt;/p&gt;&#xA;" OwnerUserId="41265" LastActivityDate="2020-09-28T10:05:19.477" Title="How are weight matrices in attention trained?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;backpropagation&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23804" PostTypeId="1" CreationDate="2020-09-28T11:34:54.760" Score="1" ViewCount="28" Body="&lt;p&gt;I am new to the field of AI but due to the high level of abstraction that comes with services such as Google VisionAI I got motivated to write an application that detects symbols in photos based on tensorflow.js and a custom model trained in Google Vision AI.&lt;/p&gt;&#xA;&lt;p&gt;My App is about identifying symbols in photos, very similar to traffic signs or logo detection. Now I wonder if&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I should train the model based on real, distorted and complex photos that contain those symbols and lots of background noise&lt;/li&gt;&#xA;&lt;li&gt;if it was enough to train the model based on cropped, clean symbols&lt;/li&gt;&#xA;&lt;li&gt;A hybrid of both&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I started with option a and it works fine, however it was a lot of work to create the training dataset. Does the model need the distorted background to work?&lt;/p&gt;&#xA;" OwnerUserId="41268" LastActivityDate="2020-09-28T12:24:20.907" Title="Should I prefer cropped images or realistic images for object detection?" Tags="&lt;tensorflow&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23805" PostTypeId="2" ParentId="23804" CreationDate="2020-09-28T12:24:20.907" Score="1" Body="&lt;p&gt;Google recommandation seems to answer this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The training data should be as close as possible to the data on which&#xA;predictions are to be made.&lt;/p&gt;&#xA;&lt;p&gt;For example, if your use case involves blurry and low-resolution&#xA;images (such as from a security camera), your training data should be&#xA;composed of blurry, low-resolution images. In general, you should also&#xA;consider providing multiple angles, resolutions, and backgrounds for&#xA;your training images.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://cloud.google.com/vision/automl/object-detection/docs/prepare&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/vision/automl/object-detection/docs/prepare&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Would you agree, also in the case of symbols?&lt;/p&gt;&#xA;" OwnerUserId="41268" LastActivityDate="2020-09-28T12:24:20.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23806" PostTypeId="1" CreationDate="2020-09-28T12:36:03.010" Score="1" ViewCount="121" Body="&lt;p&gt;I'm wanting to build a NN that can create a policy for each possible state. I want to combine this with MCTS to eliminate randomness so when expansion occurs, I can get the probability of the move to winning.&lt;/p&gt;&#xA;&lt;p&gt;I am confident (I believe) in how to code the neural network, but the input shape is the hardest part here. I am firstly wanting to try with 2 player chess and then expand to 3 player chess.&lt;/p&gt;&#xA;&lt;p&gt;What is the best vector/matrix to use for the input in a chess game? How should the input be fed into a neural network to output the most promising move from the position? In addition, what format should it look like (i.e [001111000], etc.)?&lt;/p&gt;&#xA;" OwnerUserId="41271" LastEditorUserId="2444" LastEditDate="2021-03-09T17:57:52.017" LastActivityDate="2021-03-09T17:57:52.017" Title="To solve chess with deep RL and MCTS, how should I represent the input (the state) to a neural network?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;monte-carlo-tree-search&gt;&lt;chess&gt;&lt;state-spaces&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23807" PostTypeId="1" CreationDate="2020-09-28T13:19:58.557" Score="2" ViewCount="52" Body="&lt;p&gt;I'm currently looking at NN to deal with noisy data. I like the Autoencoder approach &lt;a href=&quot;https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957&lt;/a&gt; because it seems to be adaptive and does not require to be trained on specific training data.&lt;/p&gt;&#xA;&lt;p&gt;However, as it is described in this article it seems to rely on having none-noise samples in the input data that are true to the ground truth, so I wonder if an autoencoder also could work in the case of white or blue noise instead of salt-and-pepper noise?&lt;/p&gt;&#xA;" OwnerUserId="32111" LastActivityDate="2021-02-26T22:08:15.717" Title="Are Autoencoders for noise-reduction only suited to deal with salt-and-pepper kind of noise?" Tags="&lt;autoencoders&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23809" PostTypeId="1" CreationDate="2020-09-28T18:48:17.733" Score="0" ViewCount="65" Body="&lt;p&gt;I currently learning on Transformers, so check my understanding I tried implementing a small transformer-based language model and compare it to RNN based language model. Here's the code for transformer. I'm using PyTorch inbuilt layer for Transformer Encoder&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class TransformerLM_1(nn.Module):&#xA;&#xA;    def __init__(self, head, vocab_size, embedding_size, dropout = 0.1, device = 'cpu', &#xA;                 pad_idx = 0, start_idx = 1, end_idx = 2, unk_idx = 3):&#xA;      &#xA;        super(TransformerLM_1, self).__init__()&#xA;      &#xA;        self.head = head&#xA;        self.embedding_size = embedding_size&#xA;        self.vocab_size = vocab_size&#xA;        self.device = device&#xA;        self.embed = WordEmbedding(self.vocab_size, self.embedding_size, pad_idx)&#xA;        self.postional_encoding = PostionalEncoding(embedding_size, device)&#xA;        self.decoder = nn.TransformerEncoderLayer(self.embedding_size, self.head)&#xA;        self.out_linear = nn.Linear(self.embedding_size, vocab_size)&#xA;        self.dropout = dropout&#xA;        self.pad_idx = pad_idx&#xA;        self.start_idx = start_idx&#xA;        self.end_idx = end_idx&#xA;        self.unk_idx = unk_idx&#xA;        self.device = device&#xA;&#xA;    &#xA;    def make_src_mask(self, src_sz):&#xA;        mask = (torch.triu(torch.ones(src_sz, src_sz)) == 1).transpose(0, 1)&#xA;        mask = mask.float().masked_fill(mask == 0, 10e-20).masked_fill(mask == 1, float(0.0))&#xA;        mask = mask.to(self.device)&#xA;        return mask&#xA;&#xA;    def forward(self, x):&#xA;        dec_in = x.clone()[:, :-1]&#xA;        src_mask = self.make_src_mask(dec_in.size()[1])&#xA;        src = self.embed(dec_in)&#xA;        src = self.postional_encoding(src) &#xA;        src = src.transpose(0,1)&#xA;        transformer_out = self.decoder(src, src_mask)&#xA;        out = self.out_linear(transformer_out)&#xA;        return out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm using teacher forcing to make it converge faster. From what I saw from the results, the text generated by the RNN model is better than transformer's.&lt;/p&gt;&#xA;&lt;p&gt;Here is sample generated text with the expected&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Expected: you had to have been blind not to see the scenario there for what it was and is and will continue to be for months and even years a part of south carolina that has sustained a blow that the red cross expects will cost that organization alone some &lt;span class=&quot;math-container&quot;&gt;$ n million &amp;lt;eos&amp;gt; &#xA;Predicted: some &amp;lt;unk&amp;gt; been the been &amp;lt;unk&amp;gt; not be $&lt;/span&gt; the total has was the may has &amp;lt;unk&amp;gt; the that that be to the &amp;lt;unk&amp;gt; the &#xA;&#xA;Expected: citicorp and chase are attempting to put together a new lower bid &amp;lt;eos&amp;gt; &#xA;Predicted: a are &amp;lt;unk&amp;gt; carries n't to the together with &amp;lt;unk&amp;gt; jersey than &#xA;&#xA;Expected: it ' s amazing the amount of money that goes up their nose out to the dog track or to the tables in las vegas mr . katz says &amp;lt;eos&amp;gt; &#xA;Predicted: &amp;lt;unk&amp;gt; ' s &amp;lt;unk&amp;gt; comeback money of the in mr to their &amp;lt;unk&amp;gt; and of &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; or or &amp;lt;unk&amp;gt; the money &#xA;&#xA;Expected: moreover while asian and middle eastern investors &amp;lt;unk&amp;gt; gold and help &amp;lt;unk&amp;gt; its price silver does n't have the same &amp;lt;unk&amp;gt; dealers say &amp;lt;eos&amp;gt; &#xA;Predicted: the production the routes &amp;lt;unk&amp;gt; of its &#xA;&#xA;Expected: a board of control spokesman said the board had not seen the claim and declined to comment &amp;lt;eos&amp;gt; &#xA;Predicted: the board said declined of said &#xA;&#xA;Expected: property capital trust said it dropped its plan to liquidate because it was n't able to realize the value it had expected &amp;lt;eos&amp;gt; &#xA;Predicted: the claims markets said its was n &amp;lt;unk&amp;gt; to sell insolvent of was n't disclosed to sell its plan &#xA;&#xA;Expected: similarly honda motor co . ' s sales are so brisk that workers &amp;lt;unk&amp;gt; they have n't had a saturday off in years despite the government ' s encouragement of more leisure activity &amp;lt;eos&amp;gt; &#xA;Predicted: the honda ' credit . s s &amp;lt;unk&amp;gt; &#xA;&#xA;Expected: we expect a big market in the future so in the long term it will be profitable &amp;lt;eos&amp;gt; &#xA;Predicted: it can it &amp;lt;unk&amp;gt; board &#xA;&#xA;Expected: u . k . composite or &amp;lt;unk&amp;gt; insurers which some equity analysts said might be heavily hit by the earthquake disaster helped support the london market by showing only narrow losses in early trading &amp;lt;eos&amp;gt; &#xA;Predicted: the . s . s trading sell said which &amp;lt;unk&amp;gt; traders market said the be able in the the earthquake &#xA;&#xA;Expected: this will require us to define and &amp;lt;unk&amp;gt; what is necessary or appropriate care &amp;lt;eos&amp;gt; &#xA;Predicted: &amp;lt;unk&amp;gt; is be the $ &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; is the to &amp;lt;unk&amp;gt; and or &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see Transformer fails to grasp grammar compared to RNN. Is there anything wrong with my understanding?&lt;/p&gt;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&lt;p&gt;This is one example that caught my eye&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Expected: also the big board met with angry stock specialists &amp;lt;eos&amp;gt; &#xA;Predicted: also met specialists board met the stock big with after &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Most of the words predicted have is from the expected but in a different order. I have read that transformers are permutation invariant which is the reason why we include positional encoding with the word embedding.&lt;/p&gt;&#xA;" OwnerUserId="41279" LastEditorUserId="41279" LastEditDate="2020-09-29T05:41:48.573" LastActivityDate="2020-09-29T05:41:48.573" Title="Transformer Language Model generating meaningless text" Tags="&lt;natural-language-processing&gt;&lt;pytorch&gt;&lt;transformer&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23810" PostTypeId="1" AcceptedAnswerId="23814" CreationDate="2020-09-28T20:10:17.030" Score="1" ViewCount="127" Body="&lt;p&gt;I am running a drone simulator for collision avoidance using a slight variant of D3QN. The training is usually costly (runs for at least a week) and I have observed that reward function gradually increases during training and then drastically drops. In the simulator, this corresponds to the drone exhibiting cool collision avoidance after a few thousand episodes. However, after training for more iterations it starts taking &lt;em&gt;counterintuitive&lt;/em&gt; actions such as simply crashing into a wall (I have checked to ensure that there is no exploration at play over here).&lt;/p&gt;&#xA;&lt;p&gt;Does this have to do with overfitting? I am unable to understand why my rewards are falling this way.&lt;/p&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2020-09-28T22:37:18.253" LastActivityDate="2020-09-29T08:47:42.563" Title="Why do my rewards reduce after extensive training using D3QN?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;rewards&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23811" PostTypeId="1" CreationDate="2020-09-28T23:31:49.077" Score="0" ViewCount="119" Body="&lt;p&gt;I have come across the Monte Carlo tree search (MCTS) algorithm, but I can't find what the tree should look like. For example, does it still represent a minimax process, i.e. player 1 from the root has its child nodes as probabilities of moves, then from those child nodes, the next move is player 2, etc.? Is this how the tree looks like, so when we backpropagate we update only the winning player nodes?&lt;/p&gt;&#xA;" OwnerUserId="41285" LastEditorUserId="2444" LastEditDate="2020-10-29T11:00:57.877" LastActivityDate="2021-03-28T14:03:35.363" Title="How does the MCTS tree look like?" Tags="&lt;search&gt;&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23812" PostTypeId="1" CreationDate="2020-09-29T02:03:31.397" Score="1" ViewCount="27" Body="&lt;p&gt;The title is one of the special things in &lt;a href=&quot;https://arxiv.org/pdf/1710.10196.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Progressive GAN&lt;/a&gt;, a paper of the NVIDIA team. By using this method, they introduced that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our approach ensures that the dynamic range, and thus the learning speed, is the same for all weights.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In details, they inited all learnable parameters by normal distribution &lt;span class=&quot;math-container&quot;&gt;$N(0,1)$&lt;/span&gt;. During training time, each forward time, they will scale the result with &lt;strong&gt;per-layer normalization constant from He's initializer&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I reproduced the code from &lt;a href=&quot;https://github.com/facebookresearch/pytorch_GAN_zoo&quot; rel=&quot;nofollow noreferrer&quot;&gt;pytorch GAN zoo Github's repo&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def forward(self, x, equalized):&#xA;    # generate He constant depend on the size of tensor W&#xA;    size = self.module.weight.size()&#xA;    fan_in = prod(size[1:])&#xA;    weight = math.sqrt(2.0 / fan_in)&#xA;    '''&#xA;    A module example:&#xA;&#xA;    import torch.nn as nn&#xA;    module = nn.Conv2d(nChannelsPrevious, nChannels, kernelSize, padding=padding, bias=bias) &#xA;    '''&#xA;    x = self.module(x)&#xA;&#xA;    if equalized:&#xA;        x *= self.weight&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At first, I thought the &lt;strong&gt;He constant&lt;/strong&gt; will be &lt;span class=&quot;math-container&quot;&gt;$c = \frac{\sqrt{2}}{\sqrt{n_l}}$&lt;/span&gt; as &lt;a href=&quot;https://arxiv.org/pdf/1502.01852.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;He's paper&lt;/a&gt;. Normally, &lt;span class=&quot;math-container&quot;&gt;$n_l &amp;gt; 2$&lt;/span&gt; so &lt;span class=&quot;math-container&quot;&gt;$w_l$&lt;/span&gt; can be scale up which lead to the gradient in backpropagation is increase as the formula in ProGan's paper &lt;span class=&quot;math-container&quot;&gt;$\hat{w}_i=\frac{w_i}{c}$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; prevent vanishing gradient.&lt;/p&gt;&#xA;&lt;p&gt;However, the code shows that &lt;span class=&quot;math-container&quot;&gt;$\hat{w}_i=w_i*c$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In summary, I can't understand why to scale down the parameter many times during training will help the learning speed be more stable.&lt;/p&gt;&#xA;&lt;p&gt;I asked this question on some communities e.g: StackOverflow, mathematics, Data Science, and still haven't had an answer.&lt;/p&gt;&#xA;&lt;p&gt;Please help me explain it, thank you!&lt;/p&gt;&#xA;" OwnerUserId="41287" LastEditorUserId="2444" LastEditDate="2020-09-29T16:23:23.420" LastActivityDate="2020-09-29T16:23:23.420" Title="Why scaling down the parameter many times during training will help the learning speed be the same for all weights in Progressive GAN?" Tags="&lt;training&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;papers&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23813" PostTypeId="2" ParentId="23756" CreationDate="2020-09-29T02:34:18.517" Score="5" Body="&lt;p&gt;This is just an idea&lt;/p&gt;&#xA;&lt;p&gt;Given a set of pixels, the task is to decide:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Which pixel is the center of an object?&lt;/li&gt;&#xA;&lt;li&gt;What is the size of the bounding boxes with the center is the pixel in part 1?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Formula, consider this is a 2D image, call &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; is the horizontal and vertical coordinate and &lt;span class=&quot;math-container&quot;&gt;$(w_i,h_i)$&lt;/span&gt; is the size of bouding box of object &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{For }m \in[x,x+w_i] \text{ and } n\in[y,y+h_i]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c_i(m,n) = \begin{cases}&#xA;    1, \text{if pixel at position (m,n) is belongs to object i,}\\&#xA;    0, \text{else}&#xA;\end{cases}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="41287" LastActivityDate="2020-09-29T02:34:18.517" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23814" PostTypeId="2" ParentId="23810" CreationDate="2020-09-29T07:42:50.993" Score="1" Body="&lt;p&gt;It is not 100% clear, but this seems like an instance of &lt;a href=&quot;https://en.wikipedia.org/wiki/Catastrophic_interference&quot; rel=&quot;nofollow noreferrer&quot;&gt;catastrophic forgetting&lt;/a&gt;. This is something that often impacts reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;I have answered &lt;a href=&quot;https://datascience.stackexchange.com/questions/56053/why-could-my-ddqn-get-significantly-worse-after-beating-the-game-repeatedly/56055#56055&quot;&gt;a very similar question on Data Science stack exchange&lt;/a&gt;, and reproduce the same answer here.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is called &amp;quot;catastrophic forgetting&amp;quot; and can be a serious problem in many RL scenarios.&lt;/p&gt;&#xA;&lt;p&gt;If you trained a neural network to recognise cats and dogs and did the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Train it for many epochs on a full dataset until you got a high accuracy.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Continue to train it, but remove all the cat pictures.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Then in a relatively short space of time, the NN would start to lose accuracy. It would forget what a cat looks like. It would learn that its task was to switch the dog prediction as high as possible, just because on average everything in the training population was a dog.&lt;/p&gt;&#xA;&lt;p&gt;Something very similar happens in your DQN experience replay memory. Once it gets good at a task, it may only experience success. Eventually, only successful examples are in its memory. The NN forgets what failure looks like (what the states are, and what it should predict for their values), and predicts high values for everything.&lt;/p&gt;&#xA;&lt;p&gt;Later on, when something bad happens and the NNs high predicted value is completely wrong, the error can be high. In addition the NN may have incorrectly &amp;quot;linked&amp;quot; features of the state representation so that it cannot distinguish which parts of the feature space are the cause of this. This creates odd effects in terms of what it learns about values of all states. Often the NN will behave incorrectly for a few episodes but then re-learn optimal behaviour. But it is also possible that it completely breaks and never recovers.&lt;/p&gt;&#xA;&lt;p&gt;There is lots of active research into catastrophic forgetting and I suggest you search that term to find out some of the many types of mitigation you could use.&lt;/p&gt;&#xA;&lt;p&gt;For Cartpole, I found a very simple hack made the learning very stable. Keep aside some percentage of replay memory stocked with the initial poor performing random exploration. Reserving say 10% to this long term memory is enough to make learning in Cartpole rock solid, as the NN always has a few examples of what not to do. The idea unfortunately does not scale well to more complex environments, but it is a nice demonstration. For a more sophisticated look at similar solutions you could see the paper &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Catastrophic_interference&quot; rel=&quot;nofollow noreferrer&quot;&gt;The importance of experience replay database composition in deep reinforcement learning&lt;/a&gt;&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-29T08:47:42.563" LastActivityDate="2020-09-29T08:47:42.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23815" PostTypeId="2" ParentId="23811" CreationDate="2020-09-29T09:26:14.307" Score="0" Body="&lt;p&gt;In a &lt;strong&gt;MCTS&lt;/strong&gt; (Monte Carlo tree search) the resulting tree is expanded by two policies:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;tree-policy&lt;/em&gt; (selects next and unknown child nodes by a exploration-exploitation tradeoff)&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;default policy&lt;/em&gt; (selects known child nodes till terminal state e.g. with random uniform moves).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The term terminal state is often considered to be the end a tree. If you consider designing an agent with MCTS that is able to play chess, then the terminal state would a win, lose or draw. One game like this is called a &lt;em&gt;playout&lt;/em&gt; (sometimes also &lt;em&gt;roll-out&lt;/em&gt;). The main concept of MCTS relies on repeating this playouts many times and updating the tree according to the scores. The tree is built up in an incremental and asymmetric manner.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DsSXt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DsSXt.png&quot; alt=&quot;Asymmetric tree growth&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Imagine designing a chess-playing agent. The tree, or rather how the tree is built up can be like this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Root Node&lt;/strong&gt;: Starting with an empty board&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Child Nodes&lt;/strong&gt;: Board configuration after each move (opponent and agent)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Selection of Child Nodes&lt;/strong&gt;: If the opponent is a human, then the selection of child nodes is performed by his/her own strategy. The agent is selecting moves according to a policy.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Terminal State&lt;/strong&gt;: Score of the game (win/lose/draw).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So after each move (opponent or agent) the current child node/root node is expanded by the resulting child nodes.&lt;/p&gt;&#xA;&lt;p&gt;If you want to dive deeper into MCTS, then I would recommend to take a look at &lt;a href=&quot;https://ieeexplore.ieee.org/document/6145622&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Monte Carlo Tree Search Methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26494" LastActivityDate="2020-09-29T09:26:14.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23817" PostTypeId="1" AcceptedAnswerId="23831" CreationDate="2020-09-29T13:10:29.617" Score="1" ViewCount="106" Body="&lt;p&gt;I have to build a model where I pre-process the data with a Gaussian kernel. The data are an &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; matrix (i.e one channel), but not an image, thus I can't refer to this matrix as an image and to its elements as pixels. The Gaussian kernel is built by the following function (more i.e. &lt;a href=&quot;https://www.cs.auckland.ac.nz/courses/compsci373s1c/PatricesLectures/Gaussian%20Filtering_1up.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{equation}&#xA;\begin{aligned}&#xA;g(x,y,\sigma) = &#xA;\dfrac{1}{2\pi\sigma^2} e^{\dfrac{-(x^2+y^2)}{2\sigma^2}}.&#xA;\end{aligned}&#xA;\end{equation}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This kernel is moving one by one element and doing convolution. In my case, most of the elements are zero, the matrix is sparse.&lt;/p&gt;&#xA;&lt;p&gt;How can I describe/understand the process of convolving the original data with a Gaussian kernel?&lt;/p&gt;&#xA;&lt;p&gt;I have been looking for some articles, but I am unable to find any mathematical explanations, only explanation in words or pseudo-code.&lt;/p&gt;&#xA;" OwnerUserId="40205" LastEditorUserId="2444" LastEditDate="2020-11-07T00:42:27.537" LastActivityDate="2020-11-07T00:42:27.537" Title="How to mathematically describe the convolution operation (with a Gaussian kernel)?" Tags="&lt;convolutional-neural-networks&gt;&lt;math&gt;&lt;image-processing&gt;&lt;data-preprocessing&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23819" PostTypeId="1" CreationDate="2020-09-29T13:53:55.657" Score="1" ViewCount="18" Body="&lt;p&gt;I created a VGG based U-Net in order to perform image segmentation task on yeast cells images obtained by a microscope.&lt;/p&gt;&#xA;&lt;p&gt;There are a couple of problems with the data:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There is inhomogeneity in the amount of yeast in the images. 1 image can have hundred of yeast cells while others can have less the one hundred.&lt;/li&gt;&#xA;&lt;li&gt;The GT segmentation map is also incomplete, and some of the cells are not labeled.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;All in all the model, given the above problem, is able to learn in some manner. My problem is that the segmentation maps seem incomplete.&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bgUso.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bgUso.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pMmMp.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pMmMp.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My loss function contains BCE, I was wondering if there is a way to force the model to create a 'fuller' segmentation maps. Something like using Random fields of some sort. Or maybe to enhance my loss function to overcome the above-mentioned problems.&lt;/p&gt;&#xA;&lt;p&gt;I wish to stay in the domain of simple architectures rather than using more sophisticated ones such as RCNN.&lt;/p&gt;&#xA;&lt;p&gt;Would appreciate any suggestions&lt;/p&gt;&#xA;" OwnerUserId="41293" LastEditorUserId="41293" LastEditDate="2020-10-01T05:27:49.607" LastActivityDate="2020-10-01T05:27:49.607" Title="Model output segmentation maps which are not full" Tags="&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;&lt;u-net&gt;&lt;semi-supervised-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23821" PostTypeId="1" CreationDate="2020-09-29T14:05:24.017" Score="2" ViewCount="23" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;You might think to apply some classifier combination techniques like ensembling, bagging and boosting but these methods would not help. Actually, “ensembling, boosting, bagging” won’t help since their purpose is to reduce variance. Naive Bayes has no variance to minimize.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The above paragraph is mentioned in this &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can they say the purpose of these methods is to reduce variance?&lt;/li&gt;&#xA;&lt;li&gt;Naive Bayes has no variance, is it true?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="41226" LastActivityDate="2020-09-29T14:05:24.017" Title="Why don't ensembling, bagging and boosting help to improve accuracy of Naive bayes classifier?" Tags="&lt;machine-learning&gt;&lt;naive-bayes&gt;&lt;bias-variance-tradeoff&gt;&lt;ensemble-learning&gt;&lt;boosting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23822" PostTypeId="1" CreationDate="2020-09-29T14:11:49.980" Score="3" ViewCount="52" Body="&lt;p&gt;According to DeepMinds's paper &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prioritized Experience Replay&lt;/a&gt; (2016), specifically Appendix B.2.1 &amp;quot;Proportional prioritization&amp;quot; (&lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf#page=13&quot; rel=&quot;nofollow noreferrer&quot;&gt;p. 13&lt;/a&gt;), one should equally divide the priority range &lt;span class=&quot;math-container&quot;&gt;$[0, p_\text{total}]$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; ranges, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the size of the batch, and sample a random variable within these sub-ranges. This random variable is then used to sample an experience from the sum-tree according to its priority (probability).&lt;/p&gt;&#xA;&lt;p&gt;Why do we need to do that? Why not simply sampling &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; random variables in &lt;span class=&quot;math-container&quot;&gt;$[0, p_\text{total}]$&lt;/span&gt; and getting &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; variables from the sum-tree without dividing the priority range into &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; different ranges? Isn't this the same?&lt;/p&gt;&#xA;" OwnerUserId="41294" LastEditorUserId="2444" LastEditDate="2020-11-01T19:04:52.163" LastActivityDate="2020-11-01T19:04:52.163" Title="Why is it necessary to divide the priority range according to the batch size in Prioritized Experience Replay?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23825" PostTypeId="1" CreationDate="2020-09-29T15:33:55.940" Score="7" ViewCount="865" Body="&lt;p&gt;What does &amp;quot;ground truth&amp;quot; mean in the context of AI especially in the context of machine learning?&lt;/p&gt;&#xA;&lt;p&gt;I am a little confused because I have read that the ground truth is the same as a label in supervised learning. And I think that's not quite right. I thought that ground truth refers to a model (or maybe the nature) of a problem. I always considered it as something philosophical (and that's what also the vocabulary 'ground truth' implies), because in ML we often don't build a describing model of the problem (like in classical mechanics) but rather some sort of a simulator that behaves like it is a describing model. That's what we/I call sometimes black box.&lt;/p&gt;&#xA;&lt;p&gt;What is the correct understanding?&lt;/p&gt;&#xA;" OwnerUserId="27777" LastEditorUserId="2444" LastEditDate="2020-09-29T16:04:49.170" LastActivityDate="2020-10-01T19:08:40.930" Title="What is meant by &quot;ground truth&quot; in the context AI?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23826" PostTypeId="2" ParentId="23825" CreationDate="2020-09-29T15:41:52.583" Score="7" Body="&lt;p&gt;In the context of ML, ground truth refers to information provided by direct observation (empirical evidence). If you're training an algorithm to classify your data, then the ground truth will be the actual, true labels which could for example be manually annotated by an domain expert. Please note that the models prediction or the inferred labels, are &lt;strong&gt;not&lt;/strong&gt; considered ground truth.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2020-09-30T12:36:14.273" LastActivityDate="2020-09-30T12:36:14.273" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23828" PostTypeId="2" ParentId="23825" CreationDate="2020-09-29T16:51:10.103" Score="5" Body="&lt;p&gt;It really depends on what words you put after &amp;quot;ground truth&amp;quot;. Sometimes people will talk about &amp;quot;ground truth labels&amp;quot;, for example in the context of classification or regression problems. The &amp;quot;ground truth labels&amp;quot; in such a case would refer to the true labels of instances; the labels that we use as target labels for instances from a training set, or the labels that we expect our models to output (and &amp;quot;punish&amp;quot; them for if they fail to do so) when evaluating/testing a trained model. This basically follows &lt;a href=&quot;https://ai.stackexchange.com/a/23826/1641&quot;&gt;razvanc92's answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Ground truth&amp;quot; can also refer to something more abstract though, something that we know exists in some form or another, but we may not even know how to express it. For example, there may be &amp;quot;ground truth laws of physics&amp;quot;, the laws of physics that our world &amp;quot;follows&amp;quot;. We may build or train a simulator trying to approximate those ground truth functions / laws, but we may not actually know how to explicitly express all of them.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-09-29T16:51:10.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23830" PostTypeId="2" ParentId="23799" CreationDate="2020-09-29T17:29:07.270" Score="0" Body="&lt;p&gt;I think this question is hinting at the problem of choosing an &lt;strong&gt;exploration strategy&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The simplest strategy is to use the so called &lt;em&gt;epsilon-greedy strategy&lt;/em&gt; (or &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy). This means that you select an action at random &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; percent of the times that an agent has to select an action. The other times, the agent takes the action that its current policy dictates. Usually, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; declines throughout the learning process.&lt;/p&gt;&#xA;&lt;p&gt;There are many different exploration strategies to choose from, with the aforementioned one being really the most simple one. For getting an impression of how diverse these different approaches to exploration strategies are, please be referred to Google Scholar (searching for &amp;quot;reinforcement learning exploration strategies&amp;quot;).&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-09-29T17:29:07.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23831" PostTypeId="2" ParentId="23817" CreationDate="2020-09-29T17:35:39.490" Score="2" Body="&lt;p&gt;Mathematically, the convolution is an operation that takes two functions, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, and produces a third function, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. Concisely, we can denote the convolution operation as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f \circledast g = h$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the context of computer vision and, in particular, image processing, the convolution is widely used to apply a so-called &lt;em&gt;kernel&lt;/em&gt; (aka filter) to an &lt;em&gt;input&lt;/em&gt; (typically, an image, but this does not have to be the case). The input (e.g. an image), the kernel, and the output of the convolution, in this context, is usually a matrix or a tensor. In image processing, the convolution is typically used to e.g. blur images or maybe to remove noise.&lt;/p&gt;&#xA;&lt;p&gt;However, in the beginning, I said that the convolution is an operation that takes two functions (and not matrices) and produces a third one, so these two explanations of the convolution do not seem to be consistent, right?&lt;/p&gt;&#xA;&lt;p&gt;The answer to this question is that the two explanations are consistent with each other. More precisely, if you have a function &lt;span class=&quot;math-container&quot;&gt;$f : X \rightarrow Y$&lt;/span&gt; (assuming that &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is discrete/countable), you can represent it in a vector form as follows &lt;span class=&quot;math-container&quot;&gt;$\mathbf{f} = [y_1, y_2, \dots, y_n]$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{f}$&lt;/span&gt; is a vector that contains all outputs of the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (for all possible inputs).&lt;/p&gt;&#xA;&lt;p&gt;In image processing, an image and a kernel can also be thought of as a function with a discrete domain (i.e. the pixels), so the matrices that represent the image or the kernel are just the vector forms of the corresponding functions. See &lt;a href=&quot;https://ai.stackexchange.com/a/21401/2444&quot;&gt;this answer&lt;/a&gt; for more details about representing an image as a function.&lt;/p&gt;&#xA;&lt;p&gt;Once you understand that the convolution in image processing is really the convolution operation as defined in mathematics, then you can simply look up the mathematical definition of the convolution operation.&lt;/p&gt;&#xA;&lt;p&gt;In the discrete case (i.e. you can think of the function as vectors, as explained above), the convolution is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle h[n] =  (f  \circledast g)[n]=\sum _{m=-M}^{M}f[n-m]g[m].} \tag{1}\label{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can read equation &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; as follows&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f  \circledast g$&lt;/span&gt; is the convolution of the input function (or matrix) &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and the kernel &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$(f  \circledast g)[n]$&lt;/span&gt; is the output of the convolution &lt;span class=&quot;math-container&quot;&gt;$f  \circledast g$&lt;/span&gt; at index (or input position) &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (so you need to apply equation \ref{1} for all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, if you want to have &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and not just &lt;span class=&quot;math-container&quot;&gt;$h[n]$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;So, the result of the convolution at &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h[n]$&lt;/span&gt;, is defined as &lt;span class=&quot;math-container&quot;&gt;$\sum _{m=-M}^{M}f[n-m]g[m]$&lt;/span&gt;, a sum that goes from &lt;span class=&quot;math-container&quot;&gt;$m = -M$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$m = M$&lt;/span&gt;. Here &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; may be half of the length of the kernel matrix. For example, if you use the following Gaussian kernel, then &lt;span class=&quot;math-container&quot;&gt;$M = 2$&lt;/span&gt; (and I assume that the center of the kernel is at coordinate &lt;span class=&quot;math-container&quot;&gt;$(0, 0)$&lt;/span&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{g} = &#xA;\frac{1}{273}&#xA;\begin{bmatrix}&#xA;1 &amp;amp; 4 &amp;amp; 7 &amp;amp; 4 &amp;amp; 1 \\&#xA;4 &amp;amp; 16 &amp;amp; 26 &amp;amp; 16 &amp;amp; 4 \\&#xA;7 &amp;amp; 26 &amp;amp; 41 &amp;amp; 26 &amp;amp; 7 \\&#xA;4 &amp;amp; 16 &amp;amp; 26 &amp;amp; 16 &amp;amp; 4 \\&#xA;1 &amp;amp; 4 &amp;amp; 7 &amp;amp; 4 &amp;amp; 1&#xA;\end{bmatrix}&#xA;\label{2}\tag{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here are some notes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The kernel \ref{2} is symmetric around the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; axes: this actually implies that the convolution is equal to the cross-correlation, so you don't even have to worry about their equivalence or not (in case you have ever worried about it, which would have happened only if you already came across the cross-correlation). See &lt;a href=&quot;https://ai.stackexchange.com/q/21999/2444&quot;&gt;this question&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The kernel \ref{2} is the vector form of the function form of the 2d Gaussian kernel (the one in your question): more precisely, an integer-valued approximation of the 2D Gaussian kernel when &lt;span class=&quot;math-container&quot;&gt;$\sigma = 1$&lt;/span&gt; (as stated in your slides).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The convolution can be implemented as matrix multiplication. This may not be useful now, but it's something useful to know if you want to implement it. See &lt;a href=&quot;https://ai.stackexchange.com/q/11172/2444&quot;&gt;this question&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Question for you: what is the result of the application of this Gaussian kernel to any input? What does this kernel intuitively do? Once you fully understand the convolution, you can answer this question.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-09-29T17:54:55.420" LastActivityDate="2020-09-29T17:54:55.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23832" PostTypeId="2" ParentId="23467" CreationDate="2020-09-29T19:43:40.543" Score="0" Body="&lt;p&gt;In &lt;strong&gt;rehearsal&lt;/strong&gt;, you do &lt;em&gt;not&lt;/em&gt; necessarily train with &lt;em&gt;all&lt;/em&gt; old training data, but you can just use &lt;em&gt;some&lt;/em&gt; of it [&lt;a href=&quot;https://www.tandfonline.com/doi/pdf/10.1080/09540099550039318?casa_token=HhD-UUb3VhkAAAAA:rveRjk5kTKF1dXwuOBsZoAkEfwD6NCQ8axBAtvQB0KONwXdaYbyH5tsfn-KkmbwIk4R2pECS0JM5&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], which you add to your current (or new) training data.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;strong&gt;batch learning&lt;/strong&gt;, at every epoch, you typically train with all training data, every step with a different batch (or subset) of the training data; so, if you have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; training examples and your batch size is &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, then you will have &lt;span class=&quot;math-container&quot;&gt;$\lfloor N/M \rfloor $&lt;/span&gt; gradient descent steps.&lt;/p&gt;&#xA;&lt;p&gt;There is also &lt;strong&gt;pseudo-rehearsal&lt;/strong&gt;, which is an alternative to rehearsal that is useful when you may not have access to the previously used training data [&lt;a href=&quot;https://www.tandfonline.com/doi/pdf/10.1080/09540099550039318?casa_token=HhD-UUb3VhkAAAAA:rveRjk5kTKF1dXwuOBsZoAkEfwD6NCQ8axBAtvQB0KONwXdaYbyH5tsfn-KkmbwIk4R2pECS0JM5&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] (for example, because it is expensive to store).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-09-29T19:43:40.543" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23833" PostTypeId="2" ParentId="23807" CreationDate="2020-09-29T21:23:24.280" Score="0" Body="&lt;p&gt;The autoencoder proposed in the link is similar to a Denoising autoencoder (DAE), in sense that both starting from a noisy image try to reconstruct the original image. The difference is that the noisy pixel are ignored during the backpropagation. The DAE takes in input an image, where before the forward phase noise is applied on the image. On output the DAE have to reconstruct the image without noise.&lt;/p&gt;&#xA;&lt;p&gt;So, given an image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\epsilon \sim \mathcal{N}(0,\,\sigma^{2})$&lt;/span&gt; the noise is applied to the image obtaining the image &lt;span class=&quot;math-container&quot;&gt;$\hat{I}$&lt;/span&gt;. Now, &lt;span class=&quot;math-container&quot;&gt;$\hat{I}$&lt;/span&gt; is the DAE input and the autoencoder is trained to minimize the function &lt;span class=&quot;math-container&quot;&gt;$L(I,g(f(\hat{I})))$&lt;/span&gt;. Where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the encoder and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the decoder.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ENy27.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ENy27.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/kiQvg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kiQvg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The image on the left is an image of the MNIST dataset where &lt;span class=&quot;math-container&quot;&gt;$\epsilon \sim \mathcal{N}(0,\,0.3)$&lt;/span&gt; and on the right there is the reconstruction of a deep DAE. You can change the type of noise in the image, the results should be the same. So the answer to your question is no, autoencoder are suited also for other type of noise.&lt;/p&gt;&#xA;&lt;p&gt;If interested you can read the paper &lt;em&gt;&amp;quot;What Regularized Auto-Encoders Learn from the Data-Generating Distribution&amp;quot; Guillaume Alain, Yoshua Bengio&lt;/em&gt; where is shown that the DAE is learning an approximation of the gradient of the data distribution.&lt;/p&gt;&#xA;" OwnerUserId="41306" LastEditorUserId="41306" LastEditDate="2020-09-29T21:31:13.480" LastActivityDate="2020-09-29T21:31:13.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23834" PostTypeId="1" CreationDate="2020-09-29T22:01:43.640" Score="0" ViewCount="73" Body="&lt;p&gt;I am trying to write custom dataset and dataloader for pascal-voc-2007. It is a multi-label classification problem. There is csv file to hold the name of the images and their corresponding labels. I want to resize images and create one-hot encoded vectors for labels. Also I don't want to load images to RAM, I want to load them when they appear in mini-batch.&lt;/p&gt;&#xA;&lt;p&gt;So far I did all of them but with gtx 1070 (I dont't think it is relevant when loading data), and i5 8600k, Samsung M.2 solid state, just loading 1 epoch of  data(without feeding to the network, I just iterate over dataloader and look) took 1 min 7s.&lt;/p&gt;&#xA;&lt;p&gt;Is it normal or can be considered as slow? Or is there some optimization problem in my code?&lt;/p&gt;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;d = {}&#xA;for e in df[&amp;quot;labels&amp;quot;]:&#xA;        l = e.split(&amp;quot; &amp;quot;)&#xA;        for k in l:&#xA;            if k not in d:&#xA;                d[k] = len(d)&#xA;&#xA;&#xA;def create_code(df, idx):&#xA;    l = [0] * len(d)&#xA;    &#xA;    labels = df.iloc[idx][&amp;quot;labels&amp;quot;].split(&amp;quot; &amp;quot;)&#xA;    labels_code = []&#xA;    &#xA;    for e in labels:&#xA;        l[d[e]] = 1&#xA;    &#xA;    return torch.tensor(l, dtype = torch.long).view(1,-1)&#xA;&#xA;&#xA;def transform_image(pth):&#xA;    img = Image.open(pth)&#xA;    return (ToTensor()(img.resize((256,256), resample = PIL.Image.ANTIALIAS))).unsqueeze(0).type(torch.float64)&#xA;&#xA;&#xA;class Dataset():&#xA;    def __init__(self, df, path): &#xA;        self.df = df&#xA;        self.path = path&#xA;        &#xA;    def __len__(self): &#xA;        return len(self.df)&#xA;    def __getitem__(self, idxs):&#xA;        if isinstance(idxs, int):&#xA;            imgs = transform_image(self.path/self.df.iloc[idxs][&amp;quot;fname&amp;quot;])&#xA;            labels = create_code(self.df, idxs)&#xA;        else:&#xA;            sub_df = self.df.iloc[idxs]&#xA;            imgs = transform_image(self.path/sub_df.iloc[0][&amp;quot;fname&amp;quot;])&#xA;            labels = create_code(sub_df, 0)&#xA;            for i in range(1, len(idxs)):&#xA;                e = sub_df.iloc[i][&amp;quot;fname&amp;quot;]&#xA;                imgs = torch.cat((imgs,transform_image(self.path/e)))&#xA;                labels = torch.cat((labels, create_code(sub_df, i)))&#xA;            &#xA;        return imgs, labels&#xA;&#xA;            &#xA;class DataLoader():&#xA;    def __init__(self, ds, bs): &#xA;        self.ds, self.bs = ds, bs&#xA;    def __iter__(self):&#xA;        n = len(self.ds)&#xA;        l = torch.randperm(n)&#xA;&#xA;        &#xA;        for i in range(0, n, self.bs): &#xA;            idxs_l = l[i:i+self.bs]&#xA;            yield self.ds[idxs_l]&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;train_ds = Dataset(df_train, path/&amp;quot;train&amp;quot;)&#xA;train_dl = DataLoader(train_ds, 64)&#xA;&#xA;%%time&#xA;for e in train_dl:&#xA;    pass&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The last part took 1 min 7s&lt;/p&gt;&#xA;" OwnerUserId="31427" LastActivityDate="2020-09-29T22:01:43.640" Title="Multi-label dataloading bottleneck Pytorch" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23836" PostTypeId="1" CreationDate="2020-09-29T23:30:26.210" Score="0" ViewCount="27" Body="&lt;p&gt;I am struggling to learn certain Evolutionary algorithm concepts and also relations between each of them. I am going through the Linkage Learning Genetic Algorithm (LLGA) right now and came across this question:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Which 6-bit string of LLGA would represent an optimal solution for trap-3?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone give me an answer or explain it?&lt;/p&gt;&#xA;" OwnerUserId="41276" LastEditorUserId="2444" LastEditDate="2020-10-04T20:11:00.043" LastActivityDate="2020-10-04T20:11:00.043" Title="Which 6-bit string would represent an optimal solution for trap-3 in the Linkage Learning Genetic Algorithm?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23837" PostTypeId="1" CreationDate="2020-09-30T08:37:08.503" Score="1" ViewCount="45" Body="&lt;p&gt;I am trying to implement a CNN (U-Net) for semantic segmentation of similar large grayscale ~4600x4600px medical images. The area I want to segment is the empty space (gap) between a round object in the middle of the picture and an outer object, which is ring-shaped, and contains the round object. This gap is &amp;quot;thin&amp;quot; and is only a small proportion of the whole image.&#xA;In my problem having a small a gap is good, since then the two objects have a good connection to each other.&lt;/p&gt;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to feed such large images on a CNN?&#xA;Downscaling the images seems a bad idea since the gap is thin and most of the relevant information will be lost. From what I've seen CNN are trained on much smaller images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since the problem is symmetric in some sense is it a good idea to split the image in 4 (or more) smaller images?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are CNNs able to detect such small regions in such a huge image? From what I've seen in the literature, mostly larger objects are segmented such as organs etc.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would appreciate some ideas and help. It is my first post on the site so hopefully I didn't make any mistakes.&lt;/p&gt;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="41311" LastActivityDate="2020-09-30T08:37:08.503" Title="Training a CNN for semantic segmentation of large 4600x4600px images" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;image-recognition&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23838" PostTypeId="1" AcceptedAnswerId="23839" CreationDate="2020-09-30T08:46:08.347" Score="2" ViewCount="256" Body="&lt;p&gt;While reading the book &lt;strong&gt;AI A modern approach, 4th ed&lt;/strong&gt;, I came across the section of &amp;quot;Agent program&amp;quot; with following text:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It is instructive to consider why the table-driven approach to agent&#xA;construction is doomed to failure. Let &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; be the set of possible&#xA;percepts and let &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; be the lifetime of the agent (the total number of&#xA;percepts it will receive).&lt;/p&gt;&#xA;&lt;p&gt;The lookup table will contain &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^T |P|^T$&lt;/span&gt; entries.&lt;/p&gt;&#xA;&lt;p&gt;Consider the automated taxi: the visual input from a single camera&#xA;(eight cameras is typical) comes in at the rate of roughly 70 mb per&#xA;sec. (30 frames per sec, 1080 X 720 pixels, with 24 bits of color&#xA;information).&lt;/p&gt;&#xA;&lt;p&gt;This gives a lookup table with over &lt;span class=&quot;math-container&quot;&gt;$10^{600,000,000,000}$&lt;/span&gt; for an&#xA;hour's driving.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Could someone please explain how the lookup table number is derived? (or what the author's point is which I am missing). If I were to multiply all of the numbers &lt;span class=&quot;math-container&quot;&gt;$30 × 1080 × 720 × 24 × 8 × 3600$&lt;/span&gt;, then I get &lt;span class=&quot;math-container&quot;&gt;$1.6124314e+13$&lt;/span&gt; which comes very close I think, but can't get what would be the reason to build a table (even though a theoretic one) in such a way - something which is &lt;em&gt;obviously intractable&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My core question is this:&lt;/p&gt;&#xA;&lt;p&gt;Assuming &lt;span class=&quot;math-container&quot;&gt;$10^{600,000,000,000}$&lt;/span&gt; is derived from &lt;span class=&quot;math-container&quot;&gt;$30 × 1080 × 720 × 24 × 8 × 3600$&lt;/span&gt;, what is the purpose of storing data in the look up table at pixel precision? Wouldn't storing higher level of details be enough to solve these kind of problems (ie, autonomous driving)? Coming more from standard software database systems, I am missing that point. Thanks&lt;/p&gt;&#xA;" OwnerUserId="41312" LastEditorUserId="41312" LastEditDate="2020-09-30T13:30:31.630" LastActivityDate="2020-09-30T13:30:31.630" Title="Why would the lookup table (of a table-driven artificial agent) need to store data at pixel precision?" Tags="&lt;math&gt;&lt;intelligent-agent&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23839" PostTypeId="2" ParentId="23838" CreationDate="2020-09-30T11:00:50.427" Score="3" Body="&lt;p&gt;A tabular system for agent decisions is a direct and simple map of percept to control choice. For each percept received, the agent looks up the percept and cross-references it to the action it should take. In order to construct this, you need to list all percepts in full detail, with the associated control choice.&lt;/p&gt;&#xA;&lt;p&gt;Clearly that is not going to be feasible for the automated taxi example. No-one would think to build such a table to handle natural image inputs. That is the author's point.&lt;/p&gt;&#xA;&lt;p&gt;However, a tabular structure is a reasonable theoretical construct for mapping an arbitrary discrete function, and also is practical for simple environments.&lt;/p&gt;&#xA;&lt;p&gt;To answer your extended question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Assuming &lt;span class=&quot;math-container&quot;&gt;$10^{600,000,000,000}$&lt;/span&gt; is derived from &lt;span class=&quot;math-container&quot;&gt;$30 × 1080 × 720 × 24 × 8 × 3600$&lt;/span&gt;, what is the purpose of storing data in the look up table at pixel precision?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is the only way to get a map from percept to control using a tabular system.&lt;/p&gt;&#xA;&lt;p&gt;By proposing any kind of summarisation or approximation of the input-to-output function to solve this, you have gone beyond the capability of a tabular system. That again is the author's point.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Wouldn't storing higher level of details be enough to solve these kind of problems (ie, autonomous driving)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If it is really obvious to you that this is the solution, then that's a good thing as you are thinking ahead. However, you should also consider what that means in terms of what you might be giving up, from a theoretical perspective. For instance, a tabular system can make radically different decisions based on very minor differences between percepts, whilst any form of processing of the inputs to make them easier to handle is necessarily going to remove information that &lt;em&gt;might&lt;/em&gt; be important.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-09-30T13:02:28.150" LastActivityDate="2020-09-30T13:02:28.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23840" PostTypeId="1" CreationDate="2020-09-30T11:23:01.807" Score="4" ViewCount="59" Body="&lt;p&gt;A stable/smooth learning validation curve often seems to keep improving over more epochs than an unstable learning curve. My intuition is that dropping the learning rate and increasing the patience of a model that produces a stable learning curve could lead to better validation fit.&lt;/p&gt;&#xA;&lt;p&gt;The counter argument is that jumps in the curve could mean that the model has just learned something significant, but they often jump back down or tail off after that.&lt;/p&gt;&#xA;&lt;p&gt;Is one better than the other? Is it possible to take aspects of both to improve learning?&lt;/p&gt;&#xA;" OwnerUserId="41315" LastEditorUserId="32410" LastEditDate="2021-04-23T01:43:34.977" LastActivityDate="2021-04-23T01:43:34.977" Title="Is stable learning preferable to jumps in accuracy/loss" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;learning-rate&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23841" PostTypeId="2" ParentId="23840" CreationDate="2020-09-30T13:04:53.857" Score="2" Body="&lt;p&gt;There is an approach to machine learning, called &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simulated Annealing&lt;/a&gt;, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger rate will cover a broader range, while the increasingly lower rate then produces a less 'erratic' climb towards a maximum.&lt;/p&gt;&#xA;&lt;p&gt;If you only use a low rate, you risk getting stuck in a local maximum, while too large a rate will not find the best solution but might end up close to one. Adjusting the rate gives you the best of both.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-09-30T13:04:53.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23842" PostTypeId="1" CreationDate="2020-09-30T14:09:41.073" Score="4" ViewCount="122" Body="&lt;p&gt;I was watching this series: &lt;a href=&quot;https://www.youtube.com/watch?v=aircAruvnKk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=aircAruvnKk&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The series demonstrates neural networks by building a simple number recognizing network.&lt;/p&gt;&#xA;&lt;p&gt;It got me thinking: Why neural networks try to recognize multiple labels instead of just one? In the above example, the network tries to recognize numbers from 0 to 9. What is the benefit of trying to recognize so many thins simultaneously? Wouldn't it make it easier to reason about if there would be 10 different neural networks which would specialize to recognize only one number at a time?&lt;/p&gt;&#xA;" OwnerUserId="41318" LastEditorUserId="41318" LastEditDate="2020-11-30T12:12:34.610" LastActivityDate="2021-02-01T10:05:35.737" Title="Why neural networks tend to be trained to recognize multiple things instead of just one?" Tags="&lt;neural-networks&gt;" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23843" PostTypeId="2" ParentId="23840" CreationDate="2020-09-30T15:41:21.117" Score="1" Body="&lt;p&gt;If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven dataset distribution, we can observe those jumps/irregularities in the training curve.&lt;/p&gt;&#xA;&lt;p&gt;And yes, those jumps do mean it might've found something significant in the landscape. Those jumps can arise while the model is exploring the multiple local minima of the landscape.&lt;/p&gt;&#xA;&lt;p&gt;During Machine Learning Optimization, we usually use algorithms like &lt;strong&gt;Stochastic Gradient Descent&lt;/strong&gt; and &lt;strong&gt;Adam&lt;/strong&gt; to find &lt;em&gt;Local Minima's&lt;/em&gt; whereas approaches like &lt;strong&gt;Simulated Annealing&lt;/strong&gt; find &lt;em&gt;global minima&lt;/em&gt;. There have been multiple discussions around why to use local minima's instead of global minima. Some argue that local minima are just as useful as global minima in case of machine learning problems.&lt;/p&gt;&#xA;&lt;p&gt;Thus, Stable Learning is preferable as it symbolizes that the model is converging to local minima.&lt;/p&gt;&#xA;&lt;h1&gt;References&lt;/h1&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You can read &lt;a href=&quot;https://arxiv.org/pdf/1906.06821.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Optimization Methods from&#xA;a Machine Learning Perspective&lt;/a&gt; by Shiliang Sun, Zehui Cao, Han Zhu, and Jing Zhaoto et al. and read about all the optimization functions commonly used in Machine Learning.&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-09-30T15:41:21.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23844" PostTypeId="2" ParentId="22976" CreationDate="2020-09-30T17:07:16.767" Score="1" Body="&lt;p&gt;One of the essential pre-processing we do on the corpus involves treating the variable-length sentences to a fixed length. There are various ways in which we can do this:&lt;/p&gt;&#xA;&lt;h1&gt;Truncate&lt;/h1&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This involves reducing the length of all the sentences to the length of the shortest sentence in the corpus. This is generally not done as it reduces the amount of information that we can learn from the corpus. This image shows pre sequence truncation, where we remove from the back to make the sentences of the same length.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PBgbz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PBgbz.png&quot; alt=&quot;Truncate Example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Padding&lt;/h1&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is the most preferred method when it comes to handling the problem of variable length sentences. In this approach, we increase the size of each vector to the longest sentence in the corpus. There are two ways to this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Post-Padding&lt;/strong&gt;: Adding zeroes in the ending&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Pre-Padding&lt;/strong&gt;: Adding zeroes in the beginning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1&gt;References&lt;/h1&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1903.07288.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Effect of Padding on LSTMs and CNNs&lt;/a&gt; by Dwarampudi Mahidhar Reddy and N V Subba Reddy, et al.&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-09-30T17:07:16.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23846" PostTypeId="2" ParentId="10549" CreationDate="2020-09-30T20:28:50.303" Score="0" Body="&lt;p&gt;This is more meant like a comment to the previous answer. I also originally thought that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \nabla_{\theta}\log \pi_{\theta}(f_{\theta}(\varepsilon, s)\mid s) = \nabla_{a}\log\pi_{\theta}(a\mid s)\vert_{a=f_{\theta}(\varepsilon,s)}\nabla_{\theta}f_{\theta}(\varepsilon, s), $$&lt;/span&gt; instead of&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \nabla_{\theta}\log \pi_{\theta}(f_{\theta}(\varepsilon, s)\mid s) = \nabla_{a}\log\pi_{\theta}(a\mid s)\vert_{a=f_{\theta}(\varepsilon,s)}\nabla_{\theta}f_{\theta}(\varepsilon, s) + \nabla_{\theta}\log\pi_{\theta}(a\mid s)\vert_{a=f_{\theta}(\varepsilon, s)}. $$&lt;/span&gt;&#xA;The following is certainly less elegant, but I hope that it gives some additional intuition why we need to take the gradient with respect to &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. For simplicity, I will assume that &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is one-dimensional, but the same argument would apply for higher dimensions. In the SAC paper, they assume that &lt;span class=&quot;math-container&quot;&gt;$\pi_{\theta}$&lt;/span&gt; is a Gaussian distribution &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(\mu_{\theta}(s), \sigma_{\theta}(s))$&lt;/span&gt;. Therefore:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \log\pi_{\theta}(a\mid s)=-\frac{1}{2}\log(2\pi) - \log\sigma_{\theta}(s)-\frac{(a-\mu_{\theta}(s))^2}{2\sigma_{\theta}(s)^2}. $$&lt;/span&gt;&#xA;Then the gradient becomes:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align} \nabla_{\theta}\log \pi_{\theta}(f_{\theta}(\varepsilon, s)\mid s)&amp;amp;=-\frac{\nabla_{\theta}\sigma_{\theta}(s)}{\sigma_{\theta}(s)}-\frac{(f_{\theta}(\varepsilon, s)-\mu_{\theta}(s))(\nabla_{\theta}f_{\theta}(\varepsilon,s)-\nabla_{\theta}\mu_{\theta}(s))}{\sigma_{\theta}(s)^2} \\&amp;amp;+\frac{(f_{\theta}(\varepsilon, s)-\mu_{\theta}(s))^2\nabla_{\theta}\sigma_{\theta}(s)}{\sigma_{\theta}(s)^3}.\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let us calculate now the terms on the right hand side:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \nabla_{a}\log\pi_{\theta}(a\mid s)\vert_{a=f_{\theta}(\varepsilon,s)}=-\frac{f_{\theta}(\varepsilon,s)-\mu_{\theta}(s)}{\sigma_{\theta}(s)^2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and &lt;span class=&quot;math-container&quot;&gt;\begin{align}\nabla_{\theta}\log\pi_{\theta}(a\mid s)\vert_{a=f_{\theta}(\varepsilon, s)}&amp;amp;=-\frac{\nabla_{\theta}\sigma_{\theta}(s)}{\sigma_{\theta}(s)}+\frac{(f_{\theta}(\varepsilon, s)-\mu_{\theta}(s))\nabla_{\theta}\mu_{\theta}(s)}{\sigma_{\theta}(s)^2}\\ &amp;amp;+\frac{(f_{\theta}(\varepsilon, s)-\mu_{\theta}(s))^2\nabla_{\theta}\sigma_{\theta}(s)}{\sigma_{\theta}(s)^3},\end{align}&lt;/span&gt;&#xA;which proves the equality. For the Q-function, we apply the chain rule as usual as &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; does not depend on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41323" LastActivityDate="2020-09-30T20:28:50.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23847" PostTypeId="1" AcceptedAnswerId="23862" CreationDate="2020-09-30T22:12:40.903" Score="4" ViewCount="323" Body="&lt;p&gt;I know with policy gradients used in an environment with a discrete action space are updated with &lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Delta \theta_{t}=\alpha \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) v_{t}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt; could be many things that represent how good the action was. And I know that this can be calculated by performing cross entropy loss with the target being what the network would have outputted if it were completely confident in its action (zeros with the index of the action chosen being one). But I don’t understand how to apply that to policy gradients that output the mean and variance of a Gaussian distribution for a continuous action space. &lt;strong&gt;What is the loss for these types of policy gradients?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried keeping the variance constant and updating the output with mean squared error loss and the target being the action it took. I thought this would end up pushing the mean towards actions with greater total rewards but it got nowhere in OpenAI’s Pendulum environment.&lt;/p&gt;&#xA;&lt;p&gt;It would also be very helpful if it was described in a way with a loss function and a target, like how policy gradients with discrete action spaces can be updated with cross entropy loss. That is how I understand it best but it is okay if that is not possible.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; for @Philipp. The way I understand it is that the loss function is the same with a continuous action space and the only thing that changes is the distribution that we get the log-probs from. In PyTorch we can use a Normal distribution for continuous action space and Categorical for discrete action space. The answer from David Ireland goes into the math but in PyTorch, that looks like &lt;code&gt;log_prob = distribution.log_prob(action_taken)&lt;/code&gt; for any type of distribution. It makes sense that for bad actions we would want to decrease the probability of taking the action. Below is working code for both types of action spaces to compare them. The continuous action space code should be correct but the agent will not learn because it is harder to learn the right actions with a continuous action space and our simple method isn't enough. Look into more advanced methods like &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ppo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO&lt;/a&gt; and &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/ddpg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;from torch.distributions.categorical import Categorical #discrete distribution&#xA;import numpy as np&#xA;import gym&#xA;import math&#xA;import matplotlib.pyplot as plt&#xA;&#xA;class Agent(nn.Module):&#xA;    def __init__(self,lr):&#xA;        super(Agent,self).__init__()&#xA;        self.fc1 = nn.Linear(4,64)&#xA;        self.fc2 = nn.Linear(64,32)&#xA;        self.fc3 = nn.Linear(32,2) #neural network with layers 4,64,32,2&#xA;&#xA;        self.optimizer = optim.Adam(self.parameters(),lr=lr)&#xA;&#xA;    def forward(self,x):&#xA;        x = torch.relu(self.fc1(x)) #relu and tanh for output&#xA;        x = torch.relu(self.fc2(x))&#xA;        x = torch.sigmoid(self.fc3(x))&#xA;        return x&#xA;&#xA;env = gym.make('CartPole-v0')&#xA;agent = Agent(0.001) #hyperparameters&#xA;DISCOUNT = 0.99&#xA;total = []&#xA;&#xA;for e in range(500): &#xA;    log_probs, rewards = [], []&#xA;    done = False&#xA;    state = env.reset()&#xA;    while not done:&#xA;        #mu = agent.forward(torch.from_numpy(state).float())&#xA;        #distribution = Normal(mu, SIGMA)&#xA;        distribution = Categorical(agent.forward(torch.from_numpy(state).float()))&#xA;        action = distribution.sample()&#xA;        log_probs.append(distribution.log_prob(action))&#xA;        state, reward, done, info = env.step(action.item())&#xA;        rewards.append(reward)&#xA;        &#xA;    total.append(sum(rewards))&#xA;&#xA;    cumulative = 0&#xA;    d_rewards = np.zeros(len(rewards))&#xA;    for t in reversed(range(len(rewards))): #get discounted rewards&#xA;        cumulative = cumulative * DISCOUNT + rewards[t]&#xA;        d_rewards[t] = cumulative&#xA;    d_rewards -= np.mean(d_rewards) #normalize&#xA;    d_rewards /= np.std(d_rewards)&#xA;&#xA;    loss = 0&#xA;    for t in range(len(rewards)):&#xA;        loss += -log_probs[t] * d_rewards[t] #loss is - log prob * total reward&#xA;&#xA;    agent.optimizer.zero_grad()&#xA;    loss.backward() #update&#xA;    agent.optimizer.step()&#xA;&#xA;    if e%10==0:&#xA;        print(e,sum(rewards)) &#xA;        plt.plot(total,color='blue') #plot&#xA;        plt.pause(0.0001)    &#xA;&#xA;&#xA;def run(i): #to visualize performance&#xA;    for _ in range(i):&#xA;        done = False&#xA;        state = env.reset()&#xA;        while not done:&#xA;            env.render()&#xA;            distribution = Categorical(agent.forward(torch.from_numpy(state).float()))&#xA;            action = distribution.sample()&#xA;            state,reward,done,info = env.step(action.item())&#xA;        env.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Above is the discrete action space code for CartPole and below is the continuous action space code for Pendulum. &lt;code&gt;Sigma&lt;/code&gt; (variance or standard deviation) is constant here but adding it is easy. Just make the final layer have two neurons and make sure &lt;code&gt;sigma&lt;/code&gt; is not negative. Again, the pendulum code won't work because most environments with continuous action spaces are too complicated for such a simple method. Making it work would probably require a lot of testing for hyper parameters.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;from torch.distributions.normal import Normal #continuous distribution&#xA;import numpy as np&#xA;import gym&#xA;import math&#xA;import matplotlib.pyplot as plt&#xA;import keyboard&#xA;&#xA;class Agent(nn.Module):&#xA;    def __init__(self,lr):&#xA;        super(Agent,self).__init__()&#xA;        self.fc1 = nn.Linear(3,64)&#xA;        self.fc2 = nn.Linear(64,32)&#xA;        self.fc3 = nn.Linear(32,1) #neural network with layers 3,64,32,1&#xA;&#xA;        self.optimizer = optim.Adam(self.parameters(),lr=lr)&#xA;&#xA;    def forward(self,x):&#xA;        x = torch.relu(self.fc1(x)) #relu and tanh for output&#xA;        x = torch.relu(self.fc2(x))&#xA;        x = torch.tanh(self.fc3(x)) * 2&#xA;        return x&#xA;&#xA;env = gym.make('Pendulum-v0')&#xA;agent = Agent(0.01) #hyperparameters&#xA;SIGMA = 0.2&#xA;DISCOUNT = 0.99&#xA;total = []&#xA;&#xA;for e in range(1000): &#xA;    log_probs, rewards = [], []&#xA;    done = False&#xA;    state = env.reset()&#xA;    while not done:&#xA;        mu = agent.forward(torch.from_numpy(state).float())&#xA;        distribution = Normal(mu, SIGMA)&#xA;        action = distribution.sample().clamp(-2.0,2.0)&#xA;        log_probs.append(distribution.log_prob(action))&#xA;        state, reward, done, info = env.step([action.item()])&#xA;        #reward = abs(state[1])&#xA;        rewards.append(reward)&#xA;        &#xA;    total.append(sum(rewards))&#xA;&#xA;    cumulative = 0&#xA;    d_rewards = np.zeros(len(rewards))&#xA;    for t in reversed(range(len(rewards))): #get discounted rewards&#xA;        cumulative = cumulative * DISCOUNT + rewards[t]&#xA;        d_rewards[t] = cumulative&#xA;    d_rewards -= np.mean(d_rewards) #normalize&#xA;    d_rewards /= np.std(d_rewards)&#xA;&#xA;    loss = 0&#xA;    for t in range(len(rewards)):&#xA;        loss += -log_probs[t] * d_rewards[t] #loss is - log prob * total reward&#xA;&#xA;    agent.optimizer.zero_grad()&#xA;    loss.backward() #update&#xA;    agent.optimizer.step()&#xA;&#xA;    if e%10==0:&#xA;        print(e,sum(rewards)) &#xA;        plt.plot(total,color='blue') #plot&#xA;        plt.pause(0.0001)&#xA;        if keyboard.is_pressed(&amp;quot;space&amp;quot;): #holding space exits training&#xA;            raise Exception(&amp;quot;Exited&amp;quot;)&#xA;&#xA;&#xA;def run(i): #to visualize performance&#xA;    for _ in range(i):&#xA;        done = False&#xA;        state = env.reset()&#xA;        while not done:&#xA;            env.render()&#xA;            distribution = Normal(agent.forward(torch.from_numpy(state).float()), SIGMA)&#xA;            action = distribution.sample()&#xA;            state,reward,done,info = env.step([action.item()])&#xA;        env.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;David Ireland also wrote this on a &lt;a href=&quot;https://ai.stackexchange.com/questions/23625/what-exactly-does-the-reinforce-update-equation-mean&quot;&gt;different question&lt;/a&gt; I had:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The algorithm doesn't change in this situation. Say your NN outputs the mean parameter of the Gaussian, then logπ(at|st) is just the log of the normal density evaluated at the action you took where the mean parameter in the density is the output of your NN. You are then able to backpropagate through this to update the weights of your network.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="41026" LastEditorUserId="36821" LastEditDate="2021-04-04T20:16:56.407" LastActivityDate="2021-04-04T20:16:56.407" Title="What is the loss for policy gradients with continuous actions?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;deterministic-policy&gt;" AnswerCount="1" CommentCount="12" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23848" PostTypeId="1" AcceptedAnswerId="23849" CreationDate="2020-10-01T03:09:03.643" Score="0" ViewCount="50" Body="&lt;p&gt;I am new to machine learning and recently I joined a course where I was given a logistic regression assignment in which I had to split 20% of the training dataset for the validation dataset and then use the validation dataset to capture the minimum possible loss and then use the test dataset to find the accuracy of the model.&#xA;Below is my code for implementing logistic regression&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class LogReg(LinReg):&#xA;    def __init__(self, n_dim, bias=True):&#xA;        if bias:&#xA;            n_dim = n_dim + 1&#xA;        super(LogReg, self).__init__(n_dim)&#xA;        self.bias = bias&#xA;    &#xA;  def __call__(self, x):&#xA;      return x.mm(self.theta).sigmoid()&#xA;&#xA;  def compute_loss(self, x, y, lambda_reg):&#xA;      # The function has a generic implementation, and can also work for the neural nets!&#xA;      predictions = self(x)&#xA;      loss = -(y * torch.log(predictions) + (1-y) * torch.log(1 - predictions)).mean()&#xA;      regularizer = self.theta.transpose(0, 1).mm(self.theta)&#xA;      return loss + regularizer.mul(lambda_reg)&#xA;  @staticmethod&#xA;  def add_bias(x):&#xA;      ones = torch.ones((x.size(0), 1), dtype=torch.float32)&#xA;      x_hat = torch.cat((ones, x), dim=-1)&#xA;      return x_hat&#xA;&#xA;  def fit(self, x, y, num_iter=10, mb_size=32, lr=1e-1, lambda_reg=1e-2, reset=True):&#xA;      N = x.size(0)&#xA;      losses = []&#xA;      x_hat = x&#xA;      # Adding a bias term if needed&#xA;      if self.bias:&#xA;          x_hat = self.add_bias(x)&#xA;      if reset:&#xA;          self.reset() # Very important if you want to call fit multiple times&#xA;      num_batches = x.size(0) // mb_size&#xA;      # The outer loop goes over `epochs`&#xA;      # The inner loop goes over the whole training data&#xA;      for it in range(num_iter):&#xA;          loss_per_epoch = 0&#xA;          for batch_it in range(num_batches):&#xA;              # has been implemented for the linear model&#xA;              self.zero_grad()&#xA;&#xA;              ind = torch.randint(0, N, (mb_size, 1)).squeeze()&#xA;              x_mb, y_mb = x_hat[ind, :], y[ind, :]&#xA;&#xA;              loss = self.compute_loss(x_mb, y_mb, lambda_reg)&#xA;&#xA;              loss.backward()&#xA;              self.theta.data = self.theta.data - lr*self.grad().data&#xA;              loss_per_epoch += loss.item()&#xA;          &#xA;          loss_per_epoch /= num_batches&#xA;          losses.append(loss_per_epoch)&#xA;      &#xA;      return losses&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How should I use the validation set at the level of epoch to find the best loss?&lt;/p&gt;&#xA;" OwnerUserId="41329" LastEditorUserId="41293" LastEditDate="2020-10-01T10:18:18.593" LastActivityDate="2020-10-01T10:18:18.593" Title="How to use validation dataset in my logistic regression model?" Tags="&lt;python&gt;&lt;pytorch&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23849" PostTypeId="2" ParentId="23848" CreationDate="2020-10-01T05:36:29.787" Score="1" Body="&lt;p&gt;So generally, when you seperate your training data to 80%-20% then you fit method should get &lt;strong&gt;2&lt;/strong&gt; &lt;code&gt;x&lt;/code&gt;,&lt;code&gt;y&lt;/code&gt;. better to call them &lt;code&gt;x_train&lt;/code&gt;,&lt;code&gt;y_train&lt;/code&gt;, &lt;code&gt;x_val&lt;/code&gt;, &lt;code&gt;y_val&lt;/code&gt; or something similar.&lt;/p&gt;&#xA;&lt;p&gt;Now its important you do the split before entering the fit, and not do it for each epoch or something alike.&lt;/p&gt;&#xA;&lt;p&gt;Once you do that and the fit method should be something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def fit(self, x_train, y_train, x_val, y_val, num_iter=10, mb_size=32, lr=1e-1, lambda_reg=1e-2, reset=True):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you should, at the end of each epoch, test the performance of the model on the validation set entirely and calculate the desired metric for evaluation. If you improved it's better to save the current model. This is done repeatedly for each epoch until the end of the training and you will guarantee to have the model who gave you the best results on the validation set rather than on the training set, which might be overfitting it.&lt;/p&gt;&#xA;&lt;p&gt;I will do it in a separate method with the following flow:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;iterate on each sample in the validation set&lt;/li&gt;&#xA;&lt;li&gt;for each one calculate the loss/metric&lt;/li&gt;&#xA;&lt;li&gt;append it to some list&lt;/li&gt;&#xA;&lt;li&gt;return the mean of that list&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;And if the average result is better from some previously saved one, save the new model&lt;/p&gt;&#xA;" OwnerUserId="41293" LastActivityDate="2020-10-01T05:36:29.787" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23850" PostTypeId="1" CreationDate="2020-10-01T08:32:32.753" Score="3" ViewCount="63" Body="&lt;p&gt;I'm new to reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;As it is common in RL, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy search for the behavior/exploration is used. So, at the beginning of the training, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is high, and therefore a lot of random actions are chosen. With time, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; decreases and we often choose the best action.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I was wondering, e.g. in Q-Learning, if &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is small, e.g. 0.1 or 0.01, do the Q-values really still change? Do they just change their direction, i.e. the best action remains the best action but the Q-values diverge further, or do the values really change again so that the best action always changes for a given state?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the Q-values really do still change strongly, is it because of the remaining random actions, which we still have at &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0$&lt;/span&gt; or would it still change at &lt;span class=&quot;math-container&quot;&gt;$\epsilon=0$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="41331" LastEditorUserId="2444" LastEditDate="2020-10-02T10:30:29.613" LastActivityDate="2020-10-02T10:30:29.613" Title="Can we stop training as soon as epsilon is small?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;value-functions&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23851" PostTypeId="1" CreationDate="2020-10-01T08:52:21.330" Score="0" ViewCount="24" Body="&lt;p&gt;I am training a stateful LSTM network, with a time series consisting of about 500000 data points spread over 5 years. This time series is split up to batches of 100 timesteps, and fed into the network. The predicted signal contains artefacts every 100 timesteps, even though I expect the network to maintain state across batches and not do this &amp;quot;reset looking&amp;quot; behaviour. Any ideas of where my problem lies?&lt;/p&gt;&#xA;&lt;p&gt;Example of artefacts. It looks like exponentially decaying spikes, making it look like some sort of state reset and then finding its way back again.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZoUUf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZoUUf.png&quot; alt=&quot;Example of artefacts. It looks like exponentially decaying spikes, making it look like some sort of state reset and then finding its way back again.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How the data is split:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    samples_to_remove_from_training_set = x_train.shape[0]%num_of_steps&#xA;x_train = x_train[:-samples_to_remove_from_training_set]&#xA;y_train = y_train[:-samples_to_remove_from_training_set]&#xA;normed_train_dataset = np.reshape(x_train.to_numpy(), newshape=(-1, num_of_steps, no_of_features))&#xA;train_labels = np.reshape(y_train.to_numpy(), newshape=(-1, num_of_steps, 1))&#xA;assert normed_train_dataset.shape[0] == train_labels.shape[0]&#xA;# test set&#xA;samples_to_remove_from_test_set = x_test.shape[0]%num_of_steps&#xA;print(x_test)&#xA;x_test = x_test[:-samples_to_remove_from_test_set]&#xA;print(x_test)&#xA;y_test = y_test[:-samples_to_remove_from_test_set]&#xA;normed_test_dataset = np.reshape(x_test.to_numpy(), newshape=(-1, num_of_steps, no_of_features))&#xA;print(normed_test_dataset.shape)&#xA;test_labels = np.reshape(y_test.to_numpy(), newshape=(-1, num_of_steps, 1))&#xA;assert normed_test_dataset.shape[0] == test_labels.shape[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The network:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  model.add(layers.LSTM(10, activation='tanh', batch_input_shape=(batch_size, num_of_steps, no_of_features), return_sequences=True, stateful=True))&#xA;  model.add(layers.Dropout(dropout_rate))&#xA;  model.add(layers.Dense(units=10, activation='linear'))&#xA;  model.add(layers.Dropout(dropout_rate))&#xA;  model.add(layers.Dense(units=1, activation='linear'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How predictions are made:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.reset_states()&#xA;train_predicitons = pd.Series(np.reshape(model.predict(normed_train_dataset, batch_size=batch_size),newshape=(-1)))&#xA;model.reset_states()&#xA;predictions = pd.Series(np.reshape(model.predict(normed_test_dataset, batch_size=batch_size),newshape=(-1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="40358" LastActivityDate="2020-10-01T08:52:21.330" Title="Stateful many-to-many RNN generating artefacts at regular intervals" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23852" PostTypeId="1" CreationDate="2020-10-01T10:17:36.063" Score="1" ViewCount="42" Body="&lt;p&gt;While the advantages of actor-only algorithms, the ones that search directly the policy without the use of the value function, are clear (possibility of having a continuous action space, a stochastic policy...), I can't figure out the disadvantages, if not general statements about less stability or bigger variance with respect to critic-only methods (with which I refer to methods that are based on the value function).&lt;/p&gt;&#xA;" OwnerUserId="37169" LastEditorUserId="2444" LastEditDate="2020-10-02T10:03:57.867" LastActivityDate="2020-10-02T10:03:57.867" Title="What are the disadvantages of actor-only methods with respect to value-based ones?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;policy-gradients&gt;&lt;policy-based-methods&gt;&lt;value-based-methods&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23854" PostTypeId="2" ParentId="23842" CreationDate="2020-10-01T14:05:39.720" Score="0" Body="&lt;p&gt;In practice you never want to classify just a single digit rather than series. In such case you have to pass a patch of image to multiple network, which would make it inconvenient. If you built different accurate models, training parameters will not significantly reduced. For example sloppy written 6, in a single model the probability of being 6 and 0 would be close, not same if you consider likelihood you may get closest answer. While with different models the probability may vary in a greater scale and you may not have good generalization as you have in single model. At the end everything boils down to generalization and in my experience neural networks trained with multiple things have good generalization property that a single.&lt;/p&gt;&#xA;" OwnerUserId="41286" LastActivityDate="2020-10-01T14:05:39.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23855" PostTypeId="2" ParentId="23723" CreationDate="2020-10-01T14:58:00.797" Score="0" Body="&lt;p&gt;Thank you very much for your help, all of you.&lt;/p&gt;&#xA;&lt;p&gt;I finally find on the Internet key words : &amp;quot;Dialog act classification&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I don't know yet how to implement it, but it's a good start !&lt;/p&gt;&#xA;" OwnerUserId="41176" LastActivityDate="2020-10-01T14:58:00.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23856" PostTypeId="2" ParentId="23154" CreationDate="2020-10-01T16:13:02.273" Score="1" Body="&lt;p&gt;It is hard to tell what exactly is better because these are hyperparameters. However, the sigmoid activation function is closer to biological neurons.&lt;/p&gt;&#xA;&lt;p&gt;In the paper below, Bengio demonstrates why ReLU activation functions are better for hidden layers. In summary, they increase the sparsity of calculations (matrix in each layer shod multiply to its relative weights), and because of that, it made it possible for data to classify faster and easier.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35757" LastActivityDate="2020-10-01T16:13:02.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23857" PostTypeId="2" ParentId="16939" CreationDate="2020-10-01T18:25:37.563" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iMYJP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iMYJP.png&quot; alt=&quot;Table&quot; /&gt;&lt;/a&gt;&#xA;? This means that there are not promising versions of this algorithm fro regression until 2012.&#xA;After your question, I have found one of the survey research paper which is done or &lt;a href=&quot;https://www.researchgate.net/publication/233843976_Ensemble_Approaches_for_Regression_A_Survey#:%7E:text=The%20goal%20of%20ensemble%20regression,phase%2C%20and%20the%20integration%20phase.&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensemple methods for regression&lt;/a&gt;. This table also extracted from this paper.&#xA;Read this paper, it will help you a lot more&lt;/p&gt;&#xA;&lt;p&gt;This one is &lt;a href=&quot;https://www.mdpi.com/2220-9964/9/6/370/pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;latest paper&lt;/a&gt; published on object detection with an ensemble approach&lt;/p&gt;&#xA;" OwnerUserId="41226" LastActivityDate="2020-10-01T18:25:37.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23862" PostTypeId="2" ParentId="23847" CreationDate="2020-10-01T23:44:31.823" Score="3" Body="&lt;p&gt;This update rule can still be applied in the continuous domain.&lt;/p&gt;&#xA;&lt;p&gt;As pointed out in the comments, suppose we are parameterising our policy using a Gaussian distribution, where our neural networks take as input the state we are in and output the parameters of a Gaussian distribution, the mean and the standard deviation which we will denote as &lt;span class=&quot;math-container&quot;&gt;$\mu(s, \theta)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma(s, \theta)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; shows the dependancy of the state and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the parameters of our network.&lt;/p&gt;&#xA;&lt;p&gt;I will assume a one-dimensional case for ease of notation but this can be extended to multi-variate cases. Our policy is now defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\pi(a_t | s_t) = \frac{1}{\sqrt{2\pi \sigma(s_t, \theta)^2}} \exp\left(-\frac{1}{2}\left(\frac{a_t - \mu(s_t, \theta)}{\sigma(s_t, \theta)}\right)^2\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, we can easily take the logarithm of this and find the derivative with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, and so nothing changes and the loss you use is the same. You simply evaluate the derivative of the log of your policy with respect to the network parameters, multiply by &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and take a gradient step in this direction.&lt;/p&gt;&#xA;&lt;p&gt;To implement this (as I'm assuming you don't want to calculate the NN derivatives by hand) then you could do something along the lines of the following in Pytorch.&lt;/p&gt;&#xA;&lt;p&gt;First you want to pass your state through your NN to get the mean and standard deviation of the Gaussian distribution. Then you want to simulate &lt;span class=&quot;math-container&quot;&gt;$z \sim N(0,1)$&lt;/span&gt; and calculate &lt;span class=&quot;math-container&quot;&gt;$a = \mu(s,\theta) + \sigma(s, \theta) \times z$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$a \sim N( \mu(s, \theta), \sigma(s, \theta))$&lt;/span&gt; -- this is the reparameterisation trick that makes backpropagation through the network easier as it takes the randomness from a source that doesn't depend on the parameters of the network. &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is your action that you will execute in your environment and use to calculate the gradient by simply writing the code &lt;code&gt;torch.log(normal_pdf(a, \mu(s, \theta), \sigma(s, \theta)).backward()&lt;/code&gt; -- here &lt;code&gt;normal_pdf()&lt;/code&gt; is any function in Python that calculates the pdf of a normal distribution for a given point and parameters.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-10-02T22:42:06.120" LastActivityDate="2020-10-02T22:42:06.120" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="23863" PostTypeId="2" ParentId="23850" CreationDate="2020-10-01T23:52:32.390" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How much the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values change does not depend on the value of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, rather the value of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; dictates how likely you are to take a random action and thus take an action that could give rise to a large TD error -- that is a large difference between the returns you expected from taking this action as to what you actually observed. How much the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-value changes depends on the magnitude of this TD error.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-learning is not guaranteed to converge if there is no exploration. Part of the convergence criteria assumes that each state-action pair will be visited infinitely often in an infinite number of episodes, and so if there is no exploration then this will not happen.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-10-01T23:52:32.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23864" PostTypeId="1" AcceptedAnswerId="23975" CreationDate="2020-10-02T06:54:50.333" Score="0" ViewCount="28" Body="&lt;p&gt;Just starting learning things about tensorflow and NN.&#xA;As an exercise I decided to create a dataset of images, watermarked and not, in order to binary classify these. First of all, the dataset ( you can see it &lt;a href=&quot;https://www.kaggle.com/felicepollano/watermarked-not-watermarked-images&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; ) was created artificially by me applying some random watermarks.&#xA;First doubt, in the dataset I don't have both images one watermarked and one not, would be better to have?&#xA;Second, frustrating: model stand on 0.5 accuracy, so it just produce random output :(&#xA;Model I tried is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential([&#xA;    tf.keras.layers.Conv2D(16,(1,1), activation='relu', input_shape=(150, 150, 3)),&#xA;    tf.keras.layers.MaxPool2D(2,2),&#xA;    tf.keras.layers.Conv2D(32,(3,3), activation='relu'),&#xA;    tf.keras.layers.MaxPool2D(2,2),&#xA;    tf.keras.layers.Conv2D(64,(3,3), activation='relu'),&#xA;    tf.keras.layers.MaxPool2D(2,2),&#xA;    tf.keras.layers.Flatten(),&#xA;    tf.keras.layers.Dense(128, activation='elu'),&#xA;    tf.keras.layers.Dense(64, activation='elu'),&#xA;    tf.keras.layers.Dense(32, activation='relu'),&#xA;    tf.keras.layers.Dense(1,activation=&amp;quot;sigmoid&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and then compiled as this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(optimizer='adam',&#xA;              loss='binary_crossentropy',&#xA;              metrics = ['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here below the fit:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;history = model.fit(train_data,&#xA;                              validation_data=valid_data,&#xA;                              steps_per_epoch=100,&#xA;                              epochs=15,&#xA;                              validation_steps=50,&#xA;                              verbose=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As for any other details, code is &lt;a href=&quot;https://www.kaggle.com/felicepollano/starter-watermarked-not-watermarked-e54da0df-f&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;I already checked for technical issues, I'm pretty sure image enter properly, train and validation dataset are 80/20, about 12K images for training. However accuracy bounches up and down around .5 while fitting. How can I improve?&lt;/p&gt;&#xA;" OwnerUserId="41346" LastActivityDate="2020-10-08T15:05:56.290" Title="Image Classification for watermarks with poor results" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;image-recognition&gt;&lt;binary-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23865" PostTypeId="1" CreationDate="2020-10-02T09:01:37.100" Score="0" ViewCount="33" Body="&lt;p&gt;I've been learning how the Monte Carlo Tree Search works and implemented it to play a game of Connect 4. I think I've got the algorithm correct as it plays the game quite well (it can usually beat me extremely quickly), but I was curious to see how it played against the version at &lt;a href=&quot;https://connect4.gamesolver.org/en&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://connect4.gamesolver.org/en&lt;/a&gt;. My AI puts up a reasonable fight but always loses eventually. In the blog for that site it explains how it's using Minimax + alpha beta pruning and a load of extra optimisations I don't understand (I'm still new to this).&lt;/p&gt;&#xA;&lt;p&gt;So my question is should an AI based on MCTS be able to play as well as an optimised Minimax algorithm and if not, what kinds of optimisations and improvements should I be looking at making to get it to that level?&lt;/p&gt;&#xA;" OwnerUserId="41351" LastActivityDate="2020-10-02T09:01:37.100" Title="How good should I be expecting my Monte Carlo Tree Search to be?" Tags="&lt;monte-carlo-tree-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23867" PostTypeId="5" CreationDate="2020-10-02T10:06:03.093" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T10:06:03.093" LastActivityDate="2020-10-02T10:06:03.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23868" PostTypeId="4" CreationDate="2020-10-02T10:06:03.093" Score="0" Body="For questions about value-based reinforcement learning (RL) methods (or algorithms), which first learn a value function and then derive the policy from it. An example of a value-based RL algorithm is Q-learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T10:06:03.093" LastActivityDate="2020-10-02T10:06:03.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23869" PostTypeId="5" CreationDate="2020-10-02T10:10:21.883" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T10:10:21.883" LastActivityDate="2020-10-02T10:10:21.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23870" PostTypeId="4" CreationDate="2020-10-02T10:10:21.883" Score="0" Body="For questions about policy-based (or policy search) reinforcement learning (RL) methods (or algorithms), which are RL algorithms that directly learn a policy (rather than first learning a value function). An example of a policy search algorithm is REINFORCE, which falls into the category of &quot;policy gradient&quot; algorithms (or &quot;policy gradients&quot;), which is a subset of policy-based algorithms that uses gradient information to guide the search." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T10:10:21.883" LastActivityDate="2020-10-02T10:10:21.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23871" PostTypeId="1" CreationDate="2020-10-02T10:31:39.780" Score="1" ViewCount="48" Body="&lt;p&gt;I have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (time) sequences of data with length &lt;span class=&quot;math-container&quot;&gt;$2048$&lt;/span&gt;. Each of these sequences correseponds to a different target output. However, I know that only a small part of the sequence is needed to actually predict this target output, say a sub-sequence of length &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I could split up each of the sequences into &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; partitions of &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt;, so that I end up with &lt;span class=&quot;math-container&quot;&gt;$16N$&lt;/span&gt; training smaples. However, I could drastically increase the number of training samples if I use a sliding window instead: there are &lt;span class=&quot;math-container&quot;&gt;$2048-128 = 1920$&lt;/span&gt; unique sub-sequences of length &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt; that preserve the time series. That means I could in fact generate &lt;span class=&quot;math-container&quot;&gt;$1920N$&lt;/span&gt; unique training samples, even though most of the input is overlapping.&lt;/p&gt;&#xA;&lt;p&gt;I could also use a larger increment between individual &amp;quot;windows&amp;quot;, which would reduce the number of sub-sequences but it could remove any autocorrelation between them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is it better to split my data into &lt;span class=&quot;math-container&quot;&gt;$16N$&lt;/span&gt; non-overlapping sub-sequences or &lt;span class=&quot;math-container&quot;&gt;$1920N$&lt;/span&gt; partially overlapping sub-sequences?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="5344" LastEditDate="2020-10-07T07:42:10.717" LastActivityDate="2020-10-07T07:42:10.717" Title="Is it better to split sequences into overlapping or non-overlapping training samples?" Tags="&lt;neural-networks&gt;&lt;sample-complexity&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23872" PostTypeId="5" CreationDate="2020-10-02T10:32:37.460" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-02T10:32:37.460" LastActivityDate="2020-10-02T10:32:37.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23873" PostTypeId="4" CreationDate="2020-10-02T10:32:37.460" Score="0" Body="For questions about the $\epsilon$-greedy policy, which is typically used as a behavioural policy (i.e. a policy used to interact with the environment) during the interaction of reinforcement learning agents with the environment." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-03T10:07:00.843" LastActivityDate="2020-10-03T10:07:00.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23874" PostTypeId="2" ParentId="7408" CreationDate="2020-10-02T14:43:38.580" Score="1" Body="&lt;p&gt;Firstly, note that the Gaussian policies you describe &lt;em&gt;are not&lt;/em&gt; equivalent to &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, mainly for this reason: for a fixed policy, the policy's variance in the Gaussian case depends on the state, while in the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy case it does not. Right off the bat, the Gaussian policy should achieve less regret than &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy.&lt;/p&gt;&#xA;&lt;p&gt;Other approaches to exploration in continuous action spaces include :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Parameterizing the policy differently. You're not limited to gaussians, rather any parameterizable distribution (particularly those that can be reparameterized according to the reparameterization trick) will do.&lt;/li&gt;&#xA;&lt;li&gt;Using an entropy bonus. You can subtract your policy's entropy in the expression for your loss function, which helps prevent your policy from becoming &amp;quot;too deterministic&amp;quot; before the agent learns the environment sufficiently.&lt;/li&gt;&#xA;&lt;li&gt;Surprise/curiosity based methods. From this I mean methods that do reward shaping based on some measure of uncertainty in the policy -- at each transition, this measure of uncertainty is added to the reward. See &amp;quot;Exploration by Random Network Distillation&amp;quot; for example.&lt;/li&gt;&#xA;&lt;li&gt;Maximum Entropy methods. These have slightly different objectives than standard RL that also emphasize policy entropy, so they should promote exploration. See SAC for example.&lt;/li&gt;&#xA;&lt;li&gt;Use deterministic policy gradients. Then you can literally apply &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy if you want, or simply add noise to the output of the policy. See TD3 for example.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I doubt this is an exhaustive list, but I hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-10-02T14:43:38.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23875" PostTypeId="1" AcceptedAnswerId="23887" CreationDate="2020-10-02T15:48:01.457" Score="3" ViewCount="106" Body="&lt;p&gt;I'm unable to find online, or understand from context - the difference between &lt;strong&gt;estimation&lt;/strong&gt; error and &lt;strong&gt;approximation&lt;/strong&gt; error in the context of machine learning (and, specifically, reinforcement learning).&lt;/p&gt;&#xA;&lt;p&gt;Could someone please explain with the help of examples and/or references?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2021-05-18T10:49:15.883" LastActivityDate="2021-05-18T10:49:15.883" Title="What's the difference between estimation and approximation error?" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;bias-variance-tradeoff&gt;&lt;approximation-error&gt;&lt;estimation-error&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23876" PostTypeId="1" CreationDate="2020-10-02T16:33:14.453" Score="0" ViewCount="73" Body="&lt;p&gt;I want to create a system so that when a human being says a word or command through a microphone, such as &amp;quot;shut down&amp;quot;, the system can execute that command &amp;quot;shut down&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I used the Deep Speech algorithm on the Persian language database, which takes audio data through a microphone and returns the text. The problem I have now is what I have to do from this point on. I need to explain that my system has to work offline and also the number of commands I have is limited.&lt;/p&gt;&#xA;" OwnerUserId="41355" LastActivityDate="2020-10-02T21:45:18.730" Title="speech comment detection by deep speech mozilla for data set" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;speech-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23877" PostTypeId="1" AcceptedAnswerId="24033" CreationDate="2020-10-02T16:43:57.083" Score="1" ViewCount="61" Body="&lt;p&gt;I try to create autonomous car using keyboard data so this is a multi class classification problem. I have keys W,A,S and D. So I have four categories. My model should decide what key should be pressed based on the screenshot (or some other data, I have some ideas). I have some API that I can use to capture keyboard data and screen (while gathering data) and also to simulate keyboard events (in autonomous mode when the car is driven by neural network).&lt;/p&gt;&#xA;&lt;p&gt;Should I create another category called for example &amp;quot;NOKEY&amp;quot;? I will use sigmoid function on each output neuron (instead of using softmax on the all neurons) to have probabilities from 0 to one for each category. But I could have very low probabilities for each neuron. And it can mean either that no key should be pressed or that the network doesn't know what to do. So maybe I should just create additional &amp;quot;artificial&amp;quot; category?&#xA;What is the standard way to deal with such situations?&lt;/p&gt;&#xA;" OwnerUserId="40943" LastEditorUserId="40943" LastEditDate="2020-10-05T21:32:16.953" LastActivityDate="2020-10-14T18:58:18.390" Title="Should I use additional empty category in some categorical problems?" Tags="&lt;categorical-data&gt;&lt;sigmoid&gt;&lt;multi-label-classification&gt;&lt;softmax&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23880" PostTypeId="2" ParentId="23876" CreationDate="2020-10-02T21:45:18.730" Score="1" Body="&lt;p&gt;The problem you state is a well known problem, and it is called &amp;quot;keyword spotting&amp;quot; os KWS. If you add a wake up word before it (like &amp;quot;hey google/siri&amp;quot;), you can also use &amp;quot;voice command&amp;quot; system to alleviate the problem.&lt;/p&gt;&#xA;&lt;p&gt;There are two kind of KWS systems: those which develop to detect a hard coded set of keywords, and those who are flexible to amend the keyword set. As you state that you trained an acoustic model using deep speech, it looks that you choose the latter approach.&lt;/p&gt;&#xA;&lt;p&gt;If you trained a deep speech model, you done the easy half of the solution. The other half which is more tricky is to develop a search algorithm based on your acoustic model. I think wfst based algorithms and prefix beam search algorithms are what you should be looking for.&lt;/p&gt;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="41365" LastActivityDate="2020-10-02T21:45:18.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23881" PostTypeId="2" ParentId="23775" CreationDate="2020-10-03T00:06:28.393" Score="1" Body="&lt;p&gt;Disclaimer: I am not an attorney and this does not constitute formal legal advice.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the output is novel the copyright resides with the creator&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In this case, almost certainly the human who utilizes the algorithm†.  There was a recent US patent case &amp;quot;Dabus&amp;quot; [U.S. Patent Application No.: 16/524,350] where the human programmers tried to claim an AI as an inventor, which was &lt;a href=&quot;https://www.uspto.gov/sites/default/files/documents/16524350_22apr2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;rejected by USPTO&lt;/a&gt;.  This is largely because &lt;a href=&quot;https://www.jonesday.com/en/insights/2020/05/reboot-required-artificial-intelligence-system-cannot-be-named-as-an-inventor-under-us-patent-law-uspto-says&quot; rel=&quot;nofollow noreferrer&quot;&gt;an inventor is defined as a &amp;quot;natural person&amp;quot;&lt;/a&gt;.  But it's an interesting challenge to the notion of inventorship and authorship.&lt;/p&gt;&#xA;&lt;p&gt;An argument regarding output specifically as a salable commodity would be that most human creative endeavor is result of previous work that forms the basis for the novel arrangement of elements in the form of an original work.&lt;/p&gt;&#xA;&lt;p&gt;Music is more specific because melodies are mathematical, and what are traditionally protected, although &lt;a href=&quot;https://www.jonesday.com/en/insights/2018/07/blurred-lines-between-inspiration-and-infringement&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blurred Lines case blurred this line&lt;/a&gt;, in that the ruling there was the production aspect, the &amp;quot;feel&amp;quot;, was what was plagiarized.&lt;/p&gt;&#xA;&lt;p&gt;And, in fact, algorithmic music generation, which is heavily utilized in pop music, depends on combinatorial functions producing novel sequences, not protected by prior copyright.&lt;/p&gt;&#xA;&lt;p&gt;Samples are clear cut—it is excerpting from copyrighted work.  A wave form is then copyrightable, in that a work of recorded music is simply an arrangement of waveforms in some combination and sequence.&lt;/p&gt;&#xA;&lt;p&gt;If the output is procedural, and a process that can be automated, that would enter the domain of patent law, utility patents specifically.&lt;/p&gt;&#xA;&lt;p&gt;Design output would also be in the realm of design patents, which are specific/generalized arrangements of elements in products that are not processes.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A note on liability:&lt;/p&gt;&#xA;&lt;p&gt;Law is a process, where precedent plays a major factor.  Because law uses natural language, there is ambiguity, and jurisprudence is the process of clarifying the meaning and application via challenges.  Especially when in uncharted territory &amp;quot;no one knows&amp;quot; until a case has been ruled on, and rulings can be challenged all the way up to the Supreme Court, which may or may not accept a suit.&lt;/p&gt;&#xA;&lt;p&gt;Damages for copyright are 3x, but damages can be very hard to prove.  Intellectual Property litigation is also enormously expensive, thus unlikely to be pursued unless there is a potential financial benefit.  (The first step in any potential copyright violation is typically a &amp;quot;cease and desist&amp;quot; letter, with no legal action if acceded to. In a case such as &amp;quot;Blurred Lines&amp;quot;, where there is significant financial return to the alleged infringer, the award or settlement presumably exceeds the cost of litigation.)&lt;/p&gt;&#xA;&lt;p&gt;However, deep-pocket players can use litigation, or threat of litigation, to disincentivize competitors, and, if the target has limited resources, can produce the desired outcome regardless of the strength of the claim.  (It's not uncommon in law in general to file frivolous suits as a strategy, although there is typically a financial penalty  such as reimbursement of the defendant's legal fees if the suit is found to be frivolous and dismissed.) &amp;quot;Patent trolling&amp;quot; became so much of a problem before 2013, the entire philosophy of patentability, what is patentable, had to be revisited.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;†Who has rights to an algorithmic process is also a legal question.&lt;/p&gt;&#xA;&lt;p&gt;Software is patentable, being simply a type of computer, regardless of medium, but most software is not patented, and protected instead as &amp;quot;trade secrets&amp;quot; and via non-disclosure agreements.  (This may be due to the glacial pace of the patent process compared to the software development process, but also per the from the necessity of making patents public.  i.e. if Google patented their search algorithm, it would be an instruction manual on how to exploit it.)&lt;/p&gt;&#xA;&lt;p&gt;Software is copyrightable as the specific lines of code.  Copyright naturally resides with the creator, but the process of copyright is be utilized to document the right against potential infringement.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-10-03T01:52:19.827" LastActivityDate="2020-10-03T01:52:19.827" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23883" PostTypeId="1" CreationDate="2020-10-03T08:49:40.430" Score="1" ViewCount="24" Body="&lt;p&gt;&lt;strong&gt;what Machine Learning tool can understand in which location and orientation a picture was taken from?&lt;/strong&gt;&#xA;That is from pictures of similar objects, say for example pictures of car interiors.&lt;/p&gt;&#xA;&lt;p&gt;So given a side vent picture it will show me all the pictures with a side vent with similar views (if a picture shows a vent from the cockpit it will not show me pictures of vents taken from outside the car with open door).&lt;/p&gt;&#xA;&lt;p&gt;If the problem is too complicated for just one tool, could you address me to which particular field of Artificial Intelligence should I research in?&lt;/p&gt;&#xA;&lt;p&gt;this goes close to what I am looking for:&#xA;&lt;a href=&quot;https://ai.googleblog.com/2020/03/real-time-3d-object-detection-on-mobile.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.googleblog.com/2020/03/real-time-3d-object-detection-on-mobile.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But I thought to ask if there are more specific appropriate examples.&lt;/p&gt;&#xA;" OwnerUserId="41370" LastActivityDate="2020-10-03T08:49:40.430" Title="Machine Learning Techniques for Objects Location/Orientation in Images" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23884" PostTypeId="1" CreationDate="2020-10-03T11:03:30.667" Score="2" ViewCount="312" Body="&lt;p&gt;During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on.&lt;/p&gt;&#xA;&lt;p&gt;However, on the NLP side, I see that the layers of the BERT model aren't ever frozen. What is the reason for this?&lt;/p&gt;&#xA;" OwnerUserId="41373" LastEditorUserId="2444" LastEditDate="2020-10-03T14:37:32.413" LastActivityDate="2020-10-03T14:37:32.413" Title="Why aren't the BERT layers frozen during fine-tuning tasks?" Tags="&lt;natural-language-processing&gt;&lt;computer-vision&gt;&lt;bert&gt;&lt;transfer-learning&gt;&lt;fine-tuning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23885" PostTypeId="1" CreationDate="2020-10-03T12:45:22.547" Score="0" ViewCount="44" Body="&lt;p&gt;I've written my own version of SAC(v2) for a problem with continuous action space. While training, the losses for the value network and both q functions steadily decrease down to 0.02-0.03. The loss for my actor/agent is negative and decreases to about -0.25 (I've read that it doesn't matter whether it is negative or not, but I'm not 100% sure). Despite that, the output variance from the Gaussian policy is way too high (make all the outcomes nearly uniformly likely) and is not decreasing during training.&#xA;Does anyone know what can be a cause of that?&lt;/p&gt;&#xA;&lt;p&gt;My implementation is mostly based on &lt;a href=&quot;https://github.com/keiohta/tf2rl/blob/master/tf2rl/algos/sac.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/keiohta/tf2rl/blob/master/tf2rl/algos/sac.py&lt;/a&gt;, but I resigned from computing td_errors.&lt;/p&gt;&#xA;&lt;p&gt;Here is the code (in case you need it).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;from tensorflow.keras.layers import *&#xA;from src.anfis.anfis_layers import *&#xA;from src.model.sac_layer import *&#xA;from src.anfis.anfis_model import AnfisGD&#xA;&#xA;hidden_activation = 'elu'&#xA;output_activation = 'linear'&#xA;&#xA;&#xA;class NetworkModel:&#xA;    def __init__(self, training):&#xA;&#xA;        self.parameters_count = 2&#xA;        self.results_count = 1&#xA;        self.parameters_sets_count = [3, 4]&#xA;        self.parameters_sets_total_count = sum(self.parameters_sets_count)&#xA;&#xA;        self.models = {}&#xA;        self._initialise_layers()  # initialises self.models[]&#xA;&#xA;        self.training = training&#xA;&#xA;        self.train()&#xA;&#xA;    def _initialise_layers(self):&#xA;        # ------------&#xA;        # LAYERS &amp;amp; DEBUG&#xA;        # ------------&#xA;&#xA;        f_states = Input(shape=(self.parameters_count,))&#xA;        f_actions = Input(shape=(self.results_count,))&#xA;&#xA;        # = tf.keras.layers.Dense(10)# AnfisGD(self.parameters_sets_count)&#xA;        #f_anfis = model_anfis(densanf)#model_anfis(f_states)&#xA;        f_policy_1 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_states)&#xA;        f_policy_2 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_policy_1)&#xA;        f_policy_musig = tf.keras.layers.Dense(2, activation=output_activation)(f_policy_2)&#xA;        f_policy = GaussianLayer()(f_policy_musig)&#xA;&#xA;        #self.models[&amp;quot;anfis&amp;quot;] = tf.keras.Model(inputs=f_states, outputs=f_anfis)&#xA;        #self.models[&amp;quot;forward&amp;quot;] = tf.keras.Model(inputs=f_states, outputs=model_anfis.anfis_forward(f_states))&#xA;&#xA;        self.models[&amp;quot;actor&amp;quot;] = tf.keras.Model(inputs=f_states, outputs=f_policy)&#xA;&#xA;        self.models[&amp;quot;critic-q-1&amp;quot;] = generate_q_network([f_states, f_actions])&#xA;        self.models[&amp;quot;critic-q-2&amp;quot;] = generate_q_network([f_states, f_actions])&#xA;&#xA;        self.models[&amp;quot;critic-v&amp;quot;] = generate_value_network(f_states)&#xA;        self.models[&amp;quot;critic-v-t&amp;quot;] = generate_value_network(f_states)&#xA;&#xA;        # self.models[&amp;quot;anfis&amp;quot;].compile(&#xA;        #     loss=tf.losses.mean_absolute_error,&#xA;        #     optimizer=tf.keras.optimizers.SGD(&#xA;        #         clipnorm=0.5,&#xA;        #         learning_rate=1e-3),&#xA;        #     metrics=[tf.keras.metrics.RootMeanSquaredError()]&#xA;        # )&#xA;        # self.models[&amp;quot;forward&amp;quot;].compile(&#xA;        #     loss=tf.losses.mean_absolute_error,&#xA;        #     optimizer=tf.keras.optimizers.SGD(&#xA;        #         clipnorm=0.5,&#xA;        #         learning_rate=1e-3),&#xA;        #     metrics=[tf.keras.metrics.RootMeanSquaredError()]&#xA;        # )&#xA;        self.models[&amp;quot;actor&amp;quot;].compile(&#xA;            loss=tf.losses.mean_squared_error,&#xA;            optimizer=tf.keras.optimizers.Adam(&#xA;                learning_rate=1e-3),&#xA;            metrics=[tf.keras.metrics.RootMeanSquaredError()]&#xA;        )&#xA;&#xA;    def act(self, din):&#xA;        data_input = tf.convert_to_tensor([din], dtype='float64')&#xA;        data_output = self.models[&amp;quot;actor&amp;quot;](data_input)[0]&#xA;        return data_output.numpy()[0]&#xA;&#xA;    def train(self):&#xA;        self.training.train(self, hybrid=False)&#xA;&#xA;&#xA;def mean(y_true, y_pred): #ignore y_pred&#xA;    return tf.reduce_mean(y_true)&#xA;&#xA;&#xA;def generate_value_network(inputs):&#xA;    # SAC Critic Value (Estimating rewards of being in state s)&#xA;    f_critic_v1 = tf.keras.layers.Dense(5, activation=hidden_activation)(inputs)&#xA;    f_critic_v2 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_critic_v1)&#xA;    f_critic_v = tf.keras.layers.Dense(1, activation=output_activation)(f_critic_v2)&#xA;    m_value = tf.keras.Model(inputs=inputs, outputs=f_critic_v)&#xA;    m_value.compile(&#xA;        loss=tf.losses.mean_squared_error,&#xA;        optimizer=tf.keras.optimizers.Adam(&#xA;            learning_rate=1e-3),&#xA;        metrics=[tf.keras.metrics.RootMeanSquaredError()]&#xA;    )&#xA;    return m_value&#xA;&#xA;&#xA;def generate_q_network(inputs):&#xA;    # SAC Critic Q (Estimating rewards of taking action a while in state s)&#xA;    f_critic_q_concatenate = tf.keras.layers.Concatenate()(inputs)&#xA;    f_critic_q1 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_critic_q_concatenate)&#xA;    f_critic_q2 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_critic_q1)&#xA;    f_critic_q = tf.keras.layers.Dense(1, activation=output_activation)(f_critic_q2)&#xA;&#xA;    m_q = tf.keras.Model(inputs=inputs, outputs=f_critic_q)&#xA;    m_q.compile(&#xA;        loss=tf.losses.mean_squared_error,&#xA;        optimizer=tf.keras.optimizers.Adam(&#xA;            learning_rate=1e-3),&#xA;        metrics=[tf.keras.metrics.RootMeanSquaredError()]&#xA;    )&#xA;    return m_q;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;from src.model.training import Training&#xA;import numpy as np&#xA;import tensorflow as tf&#xA;from src.constructs.experience_holder import ExperienceHolder&#xA;&#xA;&#xA;class SACTraining(Training):&#xA;&#xA;    def __init__(self, environment):&#xA;        super().__init__()&#xA;        self.environment = environment&#xA;        self.models = None&#xA;        self.parameters_sets_count = None&#xA;        self.parameters_sets_total_count = 0&#xA;        self.parameters_count = 0&#xA;&#xA;        self.gamma = 0.99&#xA;        self.alpha = 1.0&#xA;        self.beta = 0.003&#xA;        self.tau = 0.01&#xA;&#xA;        self.experience = ExperienceHolder(capacity=10000, cells=5)  # state, action, reward, state', done&#xA;&#xA;    def train(self, simulation_model, **kwargs):&#xA;        self.models = simulation_model.models&#xA;        self.parameters_count = simulation_model.parameters_count&#xA;        self.parameters_sets_count = simulation_model.parameters_sets_count&#xA;        self.parameters_sets_total_count = simulation_model.parameters_sets_total_count&#xA;&#xA;        self.train_sac(&#xA;            self.models,&#xA;            epochs=300, max_steps=200, experience_batch=128, simulation=self.environment)&#xA;&#xA;    def train_sac(self, models, epochs, max_steps, experience_batch, simulation):&#xA;&#xA;        # deterministic random&#xA;        np.random.seed(0)&#xA;&#xA;        history = []&#xA;        epoch_steps = 128&#xA;        simulation.reset()&#xA;        update_net(models['critic-v'], models['critic-v-t'], 1.0)&#xA;&#xA;        for i in range(epochs):&#xA;            print(&amp;quot;epoch: &amp;quot;, i)&#xA;            episode_reward = 0&#xA;            reset = False&#xA;            j = 0&#xA;            while not(j &amp;gt; epoch_steps and reset):&#xA;                j += 1&#xA;                reset = False&#xA;                # ---------------------------&#xA;                # Observe state s and select action according to current policy&#xA;                # ---------------------------&#xA;&#xA;                # Get simulation state&#xA;                state_raw = simulation.get_normalised()&#xA;                # state_unwound = [[i for t in state for i in t]]&#xA;&#xA;                state = [state_raw[0]]  # TODO&#xA;                state_tf = tf.convert_to_tensor(state)&#xA;&#xA;                # Get actions distribution from current model&#xA;                # and their approx value from critic&#xA;                actions_tf, _, _ = models['actor'](state_tf)&#xA;                actions = list(actions_tf.numpy()[0])&#xA;&#xA;                # ---------------------------&#xA;                # Execute action in the environment&#xA;                # ---------------------------&#xA;                reward, done = simulation.step_nominalised(actions)&#xA;                episode_reward += reward&#xA;&#xA;                # ---------------------------&#xA;                # Observe next state&#xA;                # ---------------------------&#xA;&#xA;                state_l_raw = simulation.get_normalised()&#xA;                state_l = [state_l_raw[0]]  # TODO&#xA;&#xA;                # ---------------------------&#xA;                # Store information in replay buffer&#xA;                # ---------------------------&#xA;&#xA;                self.experience.save((state, actions, reward, state_l, 1 if not done else 0))&#xA;&#xA;                if done or simulation.step_counter &amp;gt; max_steps:&#xA;                    simulation.reset()&#xA;                    reset = True&#xA;&#xA;            # ---------------------------&#xA;            # Updating network&#xA;            # ---------------------------&#xA;            if self.experience.size() &amp;gt; 500:  # update_counter_limit:&#xA;                exp = self.experience.replay(min(experience_batch, int(self.experience.size() * 0.8)))&#xA;                states_tf = tf.convert_to_tensor(exp[0], dtype='float64')&#xA;                actions_tf = tf.convert_to_tensor(exp[1], dtype='float64')&#xA;                rewards_tf = tf.convert_to_tensor(exp[2], dtype='float64')&#xA;                states_l_tf = tf.convert_to_tensor(exp[3], dtype='float64')&#xA;                not_dones_tf = tf.convert_to_tensor(exp[4], dtype='float64')&#xA;&#xA;                with tf.GradientTape(watch_accessed_variables=True, persistent=True) as tape:&#xA;&#xA;                    q_1_current = models['critic-q-1']([states_tf, actions_tf])&#xA;                    q_2_current = models['critic-q-2']([states_tf, actions_tf])&#xA;                    v_l_current = models['critic-v-t'](states_l_tf)&#xA;&#xA;                    q_target = tf.stop_gradient(rewards_tf + not_dones_tf * self.gamma * v_l_current)&#xA;                    q_1_loss = tf.reduce_mean((q_target - q_1_current) ** 2)&#xA;                    q_2_loss = tf.reduce_mean((q_target - q_2_current) ** 2)&#xA;&#xA;                    v_current = models['critic-v'](states_tf)&#xA;                    actions, policy_loss, sigma = models['actor'](states_tf)&#xA;                    q_1_policy = models['critic-q-1']([states_tf, actions_tf])&#xA;                    q_2_policy = models['critic-q-2']([states_tf, actions_tf])&#xA;                    q_min_policy = tf.minimum(q_1_policy, q_2_policy)&#xA;&#xA;                    v_target = tf.stop_gradient(q_min_policy - self.alpha * policy_loss)&#xA;                    v_loss = tf.reduce_mean((v_target - v_current)**2)&#xA;&#xA;                    a_loss = tf.reduce_mean(self.alpha * policy_loss - q_min_policy)&#xA;&#xA;                backward(tape, models['critic-q-1'], q_1_loss)&#xA;                backward(tape, models['critic-q-2'], q_2_loss)&#xA;                backward(tape, models['critic-v'], v_loss)&#xA;                update_net(models['critic-v'], models['critic-v-t'], self.tau)&#xA;&#xA;                backward(tape, models['actor'], a_loss)&#xA;&#xA;                del tape&#xA;           &#xA;                print('Loss:\n\tvalue: {}\n\tq1   : {}\n\tq2   : {}\n\tactor (ascent): {}'.format(&#xA;                     tf.reduce_mean(v_loss),&#xA;                     tf.reduce_mean(q_1_loss),&#xA;                     tf.reduce_mean(q_2_loss),&#xA;                     tf.reduce_mean(a_loss) #Gradient ascent&#xA;&#xA;                ))&#xA;                print('Episode Reward: {}'.format(episode_reward))&#xA;                print('Batch sigma: {}'.format(tf.reduce_mean(sigma)))&#xA;&#xA;&#xA;def update_net(model, target, tau):&#xA;    len_vars = len(model.trainable_variables)&#xA;    for i in range(len_vars):&#xA;        target.trainable_variables[i] = tau * model.trainable_variables[i] + (1.0 - tau) * target.trainable_variables[i]&#xA;&#xA;&#xA;def backward(tape, model, loss):&#xA;    grads = tape.gradient(loss, model.trainable_variables)&#xA;    model.optimizer.apply_gradients(&#xA;        zip(grads, model.trainable_variables))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;from tensorflow.keras import Model&#xA;import tensorflow as tf&#xA;import tensorflow_probability as tfp&#xA;&#xA;&#xA;class GaussianLayer(Model):&#xA;    def __init__(self, **kwargs):&#xA;        super(GaussianLayer, self).__init__(**kwargs)&#xA;&#xA;    def call(self, inputs, **kwargs):&#xA;        mu, log_sig = tf.split(inputs, num_or_size_splits=2, axis=1)&#xA;&#xA;        log_sig_clip = tf.clip_by_value(log_sig, -20, 2)&#xA;        sig = tf.exp(log_sig_clip)&#xA;&#xA;        distribution = tfp.distributions.Normal(mu, sig)&#xA;        output = distribution.sample()&#xA;        actions = tf.tanh(output)&#xA;&#xA;        return actions, \&#xA;            distribution.log_prob(output) - \&#xA;            tf.reduce_sum(tf.math.log(1 - actions ** 2 + 1e-12), axis=1, keepdims=True), \&#xA;            tf.stop_gradient(tf.keras.backend.abs(actions - tf.tanh(mu)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41374" LastEditorUserId="41374" LastEditDate="2020-10-04T11:41:18.650" LastActivityDate="2020-10-04T11:41:18.650" Title="Variance of the Gaussian policy is not decreasing while training the agent using Soft Actor-Critic method" Tags="&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;actor-critic-methods&gt;&lt;loss&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23887" PostTypeId="2" ParentId="23875" CreationDate="2020-10-03T16:50:58.080" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf#page=64&quot; rel=&quot;nofollow noreferrer&quot;&gt;Section &lt;strong&gt;5.2 Error Decomposition&lt;/strong&gt;&lt;/a&gt; of the book &lt;em&gt;Understanding Machine Learning: From Theory to Algorithms&lt;/em&gt; (2014) gives a description of the &lt;strong&gt;approximation error&lt;/strong&gt; and &lt;strong&gt;estimation error&lt;/strong&gt; in the context of &lt;strong&gt;empirical risk minimization (ERM)&lt;/strong&gt;, so in the context of learning theory. I will just summarise their definition. If you want to know more about these topics, I suggest that you read that section.  If you are looking for the definitions of these topics in other contexts (e.g. reinforcement learning), you should ask a new question and provide more context (i.e. a paper or book that uses those terms in RL), because I also don't intend to extend this answer, although it is possible that these definitions below apply to other contexts too.&lt;/p&gt;&#xA;&lt;h3&gt;Approximation error&lt;/h3&gt;&#xA;&lt;p&gt;The &lt;strong&gt;approximation error&lt;/strong&gt; (AE), aka inductive bias, denote as &lt;span class=&quot;math-container&quot;&gt;$\epsilon_{\mathrm{app}}$&lt;/span&gt;, is the error due to the specific &lt;em&gt;choice of hypothesis class&lt;/em&gt; (or set) &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here are some properties.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The larger &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; is, the smaller this error is (this should be intuitive!)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This error does &lt;strong&gt;not&lt;/strong&gt; depend on the training data&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Estimation error&lt;/h3&gt;&#xA;&lt;p&gt;The &lt;strong&gt;estimation error&lt;/strong&gt; (EE) is the &lt;em&gt;difference&lt;/em&gt; between the &lt;em&gt;approximation error&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\epsilon_{\mathrm{app}}$&lt;/span&gt; and the &lt;em&gt;training error&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$L_{\mathcal{D}}\left(h_{S}\right)$&lt;/span&gt;, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\epsilon_{\mathrm{est}}=L_{\mathcal{D}}\left(h_{S}\right)-\epsilon_{\mathrm{app}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here are some properties.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The EE arises because the &lt;strong&gt;empirical risk&lt;/strong&gt; is just a proxy for the &lt;strong&gt;expected risk&lt;/strong&gt; (i.e. the true risk, which is not computable because we don't know the underlying probability distribution that generated the labeled data).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The EE depends on the&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;training data (given that it is defined as a function of the training error, which depends on the training data), and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the choice of the hypothesis class (given that it is defined as a function of &lt;span class=&quot;math-container&quot;&gt;$\epsilon_{\mathrm{app}}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Bias-complexity tradeof&lt;/h3&gt;&#xA;&lt;p&gt;If we increase the size and complexity of the hypothesis class, the &lt;strong&gt;approximation error&lt;/strong&gt; &lt;em&gt;decreases&lt;/em&gt;, but the &lt;strong&gt;estimation error&lt;/strong&gt; may &lt;em&gt;increase&lt;/em&gt; (i.e. we may have &lt;em&gt;over-fitting&lt;/em&gt;). On the other hand, if we decrease the size and complexity of the hypothesis class, the &lt;strong&gt;estimation error&lt;/strong&gt; may decrease, but the &lt;em&gt;bias&lt;/em&gt; may increase (i.e. we may have &lt;em&gt;under-fitting&lt;/em&gt;). So, we have a bias-complexity trade-off (where the bias refers to the approximation error or inductive bias) and the complexity refers to the complexity of the hypothesis class.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-03T17:07:47.933" LastActivityDate="2020-10-03T17:07:47.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23888" PostTypeId="1" CreationDate="2020-10-03T18:33:10.370" Score="1" ViewCount="29" Body="&lt;p&gt;I have a dataset with four features:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the x coordinate&lt;/li&gt;&#xA;&lt;li&gt;the y coordinate&lt;/li&gt;&#xA;&lt;li&gt;the velocity magnitude&lt;/li&gt;&#xA;&lt;li&gt;angle&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now, I want to measure the distance between two points in the dataset, taking into account the facts that the angle dimension is toroidal, and taking into account the difference in nature of the dimensions (2 of them are distances, one of them is velocity magnitude, and the other an angle).&lt;/p&gt;&#xA;&lt;p&gt;What kind of distance function would suit this need?&lt;/p&gt;&#xA;&lt;p&gt;If I have to go for an &lt;span class=&quot;math-container&quot;&gt;$L^p$&lt;/span&gt; norm, can I determine which value of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; would be apt by some means?&lt;/p&gt;&#xA;&lt;p&gt;Also, if you are aware, please, let me know how such problems have been solved in various applications.&lt;/p&gt;&#xA;" OwnerUserId="35576" LastEditorUserId="2444" LastEditDate="2020-10-04T21:37:01.293" LastActivityDate="2020-10-04T21:37:01.293" Title="How to find distance between 2 points when dimensions are all of different nature?" Tags="&lt;datasets&gt;&lt;loss-functions&gt;&lt;linear-algebra&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23889" PostTypeId="1" AcceptedAnswerId="23923" CreationDate="2020-10-03T20:18:01.007" Score="5" ViewCount="391" Body="&lt;p&gt;I'm trying to implement transformer model using &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/transformer#create_the_transformer&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;.  In the decoder block of the Transformer model, a mask is passed to &amp;quot;&lt;strong&gt;pad and mask future tokens in the input received by the decoder&lt;/strong&gt;&amp;quot;. This mask is added to attention weights.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;&#xA;def create_look_ahead_mask(size):&#xA;    mask = 1 - tf.linalg.band_part(tf.ones((size, size)), -1, 0)&#xA;    return mask&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now my question is, how is doing this step (adding mask to the attention weights) equivalent to revealing the words to model one by one? I simply can't grasp the intuition of it's role. Most tutorials won't even mention this step like it's very obvious. Please help me understand. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="41379" LastEditorUserId="41379" LastEditDate="2020-10-06T10:06:11.027" LastActivityDate="2021-02-27T13:54:30.303" Title="What is the purpose of Decoder mask (triangular mask) in Transformer?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23890" PostTypeId="1" CreationDate="2020-10-03T22:13:13.923" Score="0" ViewCount="35" Body="&lt;p&gt;I am using TensorFlow + Keras to make a CNN/RNN. I'm quite new to AI, I've only made a few relatively basic networks for image regression/classification. The end goal of my project is to determine the level of valence, arousal, and dominance of an input sentence. I have converted all of the dataset input data to decimals between 0 and 1. I know how to make a network using a binary dataset, but I am struggling with this non-binary dataset. I have data from the &lt;a href=&quot;https://github.com/JULIELab/EmoBank/tree/master/corpus&quot; rel=&quot;nofollow noreferrer&quot;&gt;EmoBank dataset&lt;/a&gt;. My code thus far looks nearly identical to the official Tensorflow tutorials listed below, but it's not working with the non-binary dataset.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/tutorials/text/text_classification_rnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/text/text_classification_rnn&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.tensorflow.org/tutorials/text/word_embeddings&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/text/word_embeddings&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am able to make tensors for each sentence in the dataset with its corresponding decimal, but I am struggling with training it and/or finding the right model/optimizer.&lt;/p&gt;&#xA;&lt;p&gt;I am getting this error:&lt;br /&gt;&#xA;&lt;code&gt;AttributeError: 'TokenTextEncoder' object has no attribute 'subwords'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Dataset: Custom tf.data.TextLineDataset&lt;br /&gt;&#xA;Tokenizer: tfds.features.text.Tokenizer&lt;br /&gt;&#xA;Encoder: tfds.features.text.TokenTextEncoder&lt;br /&gt;&#xA;Optimizer: Adam&lt;/p&gt;&#xA;&lt;p&gt;Here is the main training code (it's rough, sorry):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;embedding_dim=16&#xA;&#xA;model = tf.keras.Sequential([&#xA;  tf.keras.layers.Embedding(data_size, embedding_dim),&#xA;  tf.keras.layers.GlobalAveragePooling1D(),&#xA;  tf.keras.layers.Dense(16, activation='relu'),&#xA;  tf.keras.layers.Dense(1)&#xA;])&#xA;&#xA;model.summary()&#xA;&#xA;&#xA;adam = tf.keras.optimizers.Adam(1e-4)&#xA;model.compile(optimizer=adam,&#xA;        loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),&#xA;        metrics=['accuracy'])&#xA;&#xA;history = model.fit(train_data, epochs=10, validation_data=test_data, validation_steps=20, steps_per_epoch=(data_size/20))&#xA;&#xA;import matplotlib.pyplot as plt&#xA;&#xA;history_dict = history.history&#xA;&#xA;acc = history_dict['accuracy']&#xA;val_acc = history_dict['val_accuracy']&#xA;loss=history_dict['loss']&#xA;val_loss=history_dict['val_loss']&#xA;&#xA;epochs = range(1, len(acc) + 1)&#xA;&#xA;plt.figure(figsize=(12,9))&#xA;plt.plot(epochs, loss, 'bo', label='Training loss')&#xA;plt.plot(epochs, val_loss, 'b', label='Validation loss')&#xA;plt.title('Training and validation loss')&#xA;plt.xlabel('Epochs')&#xA;plt.ylabel('Loss')&#xA;plt.legend()&#xA;plt.show()&#xA;&#xA;e = model.layers[0]&#xA;weights = e.get_weights()[0]&#xA;print(weights.shape) # shape: (vocab_size, embedding_dim)&#xA;&#xA;import io&#xA;&#xA;out_v = io.open('vecs.tsv', 'w', encoding='utf-8')&#xA;out_m = io.open('meta.tsv', 'w', encoding='utf-8')&#xA;&#xA;for num, word in enumerate(encoder.subwords):&#xA;  vec = weights[num+1] # skip 0, it's padding.&#xA;  out_m.write(word + &amp;quot;\n&amp;quot;)&#xA;  out_v.write('\t'.join([str(x) for x in vec]) + &amp;quot;\n&amp;quot;)&#xA;out_v.close()&#xA;out_m.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am also struggling with which type of loss calculation to use; BinaryCrossentropy always yields 0.0000e+00, but the loss reads fine though.&lt;/p&gt;&#xA;&lt;p&gt;If anyone can help with any part of this I'd really appreciate it!&lt;/p&gt;&#xA;" OwnerUserId="41383" LastEditorUserId="41383" LastEditDate="2020-10-03T23:49:03.830" LastActivityDate="2020-10-03T23:49:03.830" Title="How to make a CNN/RNN on a non-binary dataset?" Tags="&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23891" PostTypeId="1" AcceptedAnswerId="23895" CreationDate="2020-10-04T04:35:17.683" Score="1" ViewCount="46" Body="&lt;p&gt;We have n dimension input for SOM and the output 2-D clusters. How does it happen?&lt;/p&gt;&#xA;" OwnerUserId="41226" LastActivityDate="2020-10-04T15:33:03.547" Title="How does dimensionality reduction occur in Self organizing Map (SOM)?" Tags="&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;self-organizing-map&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23893" PostTypeId="1" CreationDate="2020-10-04T08:48:19.027" Score="0" ViewCount="34" Body="&lt;p&gt;I followed a tutorial for ddqn to beat pong, it beats it with a perfect score in keras, but trying to translate it to pytorch it doesn't learn at all. What am I missing? I pasted all the code for each model in case I miss something subtle.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KERAS AGENT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import random&#xA;&#xA;import gym&#xA;&#xA;from agent_base import Agent&#xA;from agent_augments.memory import ReplayMemory&#xA;&#xA;import numpy as np&#xA;from keras.models import Sequential&#xA;from keras.layers import Conv2D, Dense, Flatten&#xA;from keras.optimizers import Adam&#xA;&#xA;&#xA;class PongAgent(Agent, ReplayMemory):&#xA;&#xA;    def __init__(self,&#xA;                 state_space,&#xA;                 action_space,&#xA;                 channels=4,&#xA;                 batch_size=64,&#xA;                 epsilon=1,&#xA;                 epsilon_min=0.01,&#xA;                 epsilon_decay=1e-5,&#xA;                 gamma=0.99,&#xA;                 learning_rate=0.0002,&#xA;                 save_freq=500,&#xA;                 training=True):&#xA;        ReplayMemory.__init__(self, capacity=10000)&#xA;&#xA;        self.state_space = state_space&#xA;        self.action_space = action_space&#xA;        self.channels = channels&#xA;        self.batch_size = batch_size&#xA;        self.save_freq = save_freq&#xA;        self.iteration_count = 0&#xA;        self.training = training&#xA;        self.replace = 1000&#xA;&#xA;        self.epsilon = epsilon&#xA;        self.epsilon_min = epsilon_min&#xA;        self.epsilon_decay = epsilon_decay&#xA;        self.gamma = gamma&#xA;&#xA;        self.q_eval = Sequential([&#xA;            Conv2D(filters=32, kernel_size=3, strides=2, activation='relu',&#xA;                   input_shape=state_space, data_format='channels_first'),&#xA;            Conv2D(filters=64, kernel_size=5, strides=2, activation='relu',&#xA;                   input_shape=state_space, data_format='channels_first'),&#xA;            Conv2D(filters=64, kernel_size=3, strides=1, activation='relu',&#xA;                   input_shape=state_space, data_format='channels_first'),&#xA;            Flatten(),&#xA;            Dense(512, activation='relu'),&#xA;            Dense(action_space)]&#xA;        )&#xA;        self.q_eval.compile(optimizer=Adam(lr=learning_rate), loss='mean_squared_error')&#xA;&#xA;        self.q_next = Sequential([&#xA;            Conv2D(filters=32, kernel_size=3, strides=2, activation='relu',&#xA;                   input_shape=state_space, data_format='channels_first'),&#xA;            Conv2D(filters=64, kernel_size=5, strides=2, activation='relu',&#xA;                   input_shape=state_space, data_format='channels_first'),&#xA;            Conv2D(filters=64, kernel_size=3, strides=1, activation='relu',&#xA;                   input_shape=state_space, data_format='channels_first'),&#xA;            Flatten(),&#xA;            Dense(512, activation='relu'),&#xA;            Dense(action_space)]&#xA;        )&#xA;        self.q_next.compile(optimizer=Adam(lr=learning_rate), loss='mean_squared_error')&#xA;&#xA;    def replace_target_network(self):&#xA;        if self.iteration_count % self.replace == 0:&#xA;            self.q_next.set_weights(self.q_eval.get_weights())&#xA;&#xA;    def decay_epsilon(self):&#xA;        self.epsilon = self.epsilon - self.epsilon_decay \&#xA;            if self.epsilon &amp;gt; self.epsilon_min else self.epsilon_min&#xA;&#xA;    def act(self, state) -&amp;gt; int:&#xA;        if random.random() &amp;gt; self.epsilon:&#xA;            return np.argmax(self.q_eval.predict(np.expand_dims(state, 0))).item()&#xA;        else:&#xA;            return random.randrange(self.action_space)&#xA;&#xA;    def learn(self, *args, **kwargs):&#xA;        self.memorise(kwargs.get('state'),&#xA;                      kwargs.get('action'),&#xA;                      kwargs.get('reward'),&#xA;                      kwargs.get('next_state'),&#xA;                      kwargs.get('done'))&#xA;&#xA;        if not self.training:&#xA;            return&#xA;&#xA;        if self.get_mem_count() &amp;gt;= self.batch_size:&#xA;            sample = self.sample(self.batch_size)&#xA;            states = np.stack([i[0] for i in sample])&#xA;            actions = np.array([i[1] for i in sample])&#xA;            rewards = np.array([i[2] for i in sample])&#xA;            next_states = np.stack([i[3] for i in sample])&#xA;            dones = np.array([i[4] for i in sample])&#xA;&#xA;            self.replace_target_network()&#xA;&#xA;            q_eval = self.q_eval.predict(states)&#xA;&#xA;            q_next = self.q_next.predict(next_states)&#xA;&#xA;            q_target = q_eval[:]&#xA;            indices = np.arange(self.batch_size)&#xA;            q_target[indices, actions] = rewards + \&#xA;                                    self.gamma*np.max(q_next, axis=1)*(1 - dones)&#xA;            self.q_eval.train_on_batch(states, q_target)&#xA;&#xA;            self.decay_epsilon()&#xA;            self.iteration_count += 1&#xA;&#xA;    def memorise(self, state, action, reward, next_state, done):&#xA;        self.store(&#xA;            state,&#xA;            action,&#xA;            reward,&#xA;            next_state,&#xA;            done)&#xA;&#xA;    def get_epsilon(self):&#xA;        return self.epsilon&#xA;&#xA;class SkipEnv(gym.Wrapper):&#xA;    def __init__(self, env=None, skip=4):&#xA;        super(SkipEnv, self).__init__(env)&#xA;        self._skip = skip&#xA;&#xA;    def step(self, action):&#xA;        t_reward = 0.0&#xA;        done = False&#xA;        for _ in range(self._skip):&#xA;            obs, reward, done, info = self.env.step(action)&#xA;            t_reward += reward&#xA;            if done:&#xA;                break&#xA;        return obs, t_reward, done, info&#xA;&#xA;    def reset(self):&#xA;        self._obs_buffer = []&#xA;        obs = self.env.reset()&#xA;        self._obs_buffer.append(obs)&#xA;        return obs&#xA;&#xA;&#xA;class PreProcessFrame(gym.ObservationWrapper):&#xA;    def __init__(self, env=None):&#xA;        super(PreProcessFrame, self).__init__(env)&#xA;        self.observation_space = gym.spaces.Box(low=0, high=255,&#xA;                                                shape=(80, 80, 1), dtype=np.uint8)&#xA;&#xA;    def observation(self, obs):&#xA;        return PreProcessFrame.process(obs)&#xA;&#xA;    @staticmethod&#xA;    def process(frame):&#xA;        new_frame = np.reshape(frame, frame.shape).astype(np.float32)&#xA;&#xA;        new_frame = 0.299 * new_frame[:, :, 0] + 0.587 * new_frame[:, :, 1] + \&#xA;                    0.114 * new_frame[:, :, 2]&#xA;&#xA;        new_frame = new_frame[35:195:2, ::2].reshape(80, 80, 1)&#xA;&#xA;        return new_frame.astype(np.uint8)&#xA;&#xA;&#xA;class MoveImgChannel(gym.ObservationWrapper):&#xA;    def __init__(self, env):&#xA;        super(MoveImgChannel, self).__init__(env)&#xA;        self.observation_space = gym.spaces.Box(low=0.0, high=1.0,&#xA;                                                shape=(self.observation_space.shape[-1],&#xA;                                                       self.observation_space.shape[0],&#xA;                                                       self.observation_space.shape[1]),&#xA;                                                dtype=np.float32)&#xA;&#xA;    def observation(self, observation):&#xA;        return np.moveaxis(observation, 2, 0)&#xA;&#xA;&#xA;class ScaleFrame(gym.ObservationWrapper):&#xA;    def observation(self, obs):&#xA;        return np.array(obs).astype(np.float32) / 255.0&#xA;&#xA;&#xA;class BufferWrapper(gym.ObservationWrapper):&#xA;    def __init__(self, env, n_steps):&#xA;        super(BufferWrapper, self).__init__(env)&#xA;        self.observation_space = gym.spaces.Box(&#xA;            env.observation_space.low.repeat(n_steps, axis=0),&#xA;            env.observation_space.high.repeat(n_steps, axis=0),&#xA;            dtype=np.float32)&#xA;&#xA;    def reset(self):&#xA;        self.buffer = np.zeros_like(self.observation_space.low, dtype=np.float32)&#xA;        return self.observation(self.env.reset())&#xA;&#xA;    def observation(self, observation):&#xA;        self.buffer[:-1] = self.buffer[1:]&#xA;        self.buffer[-1] = observation&#xA;        return self.buffer&#xA;&#xA;&#xA;def make_env(env_name):&#xA;    env = gym.make(env_name)&#xA;    env = SkipEnv(env)&#xA;    env = PreProcessFrame(env)&#xA;    env = MoveImgChannel(env)&#xA;    env = BufferWrapper(env, 4)&#xA;    return ScaleFrame(env)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;PYTORCH AGENT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import random&#xA;&#xA;import gym&#xA;import torch&#xA;from torch.nn.modules import Module, Linear, Conv2d&#xA;import torch.nn.functional as fn&#xA;import numpy as np&#xA;from agent_base import Agent&#xA;from agent_augments.memory import ReplayMemory&#xA;&#xA;if torch.cuda.is_available():&#xA;    device = torch.device(&amp;quot;cuda:0&amp;quot;)&#xA;else:&#xA;    device = torch.device(&amp;quot;cpu&amp;quot;)&#xA;&#xA;print(&amp;quot;Using device: {}&amp;quot;.format(device))&#xA;&#xA;&#xA;class DQN(Module):&#xA;&#xA;    def __init__(self,&#xA;                 action_space,&#xA;                 transform,&#xA;                 channels=1,&#xA;                 learning_rate=0.001):&#xA;        super(DQN, self).__init__()&#xA;&#xA;        self.in_layer = Conv2d(channels, 32, 3, 2)&#xA;        self.hidden_conv_1 = Conv2d(32, 64, 5, 2)&#xA;        self.hidden_conv_2 = Conv2d(64, 64, 3, 1)&#xA;        self.hidden_fc1 = Linear(64 * 16 * 16, 512)&#xA;        self.output = Linear(512, action_space)&#xA;        self.loss = torch.nn.MSELoss()&#xA;&#xA;        self.transform = transform&#xA;&#xA;        self.optimizer = torch.optim.Adam(&#xA;            self.parameters(), lr=learning_rate)&#xA;&#xA;        self.to(device)&#xA;&#xA;        self.train()&#xA;&#xA;    def forward(self, state):&#xA;        in_out = fn.relu(self.in_layer(state))&#xA;        in_out = fn.relu(self.hidden_conv_1(in_out))&#xA;        in_out = fn.relu(self.hidden_conv_2(in_out))&#xA;        in_out = in_out.view(-1, 64 * 16 * 16)&#xA;        in_out = fn.relu(self.hidden_fc1(in_out))&#xA;        return self.output(in_out)&#xA;&#xA;class PongAgent(Agent, Module, ReplayMemory):&#xA;&#xA;    def __init__(self,&#xA;                 state_space,&#xA;                 action_space,&#xA;                 channels=4,&#xA;                 batch_size=64,&#xA;                 epsilon=1,&#xA;                 epsilon_min=0.01,&#xA;                 epsilon_decay=1e-5,&#xA;                 gamma=0.99,&#xA;                 learning_rate=0.0002,&#xA;                 save_freq=500,&#xA;                 training=True):&#xA;        Module.__init__(self)&#xA;        ReplayMemory.__init__(self, capacity=10000)&#xA;&#xA;        self.state_space = state_space&#xA;        self.action_space = action_space&#xA;        self.channels = channels&#xA;        self.batch_size = batch_size&#xA;        self.save_freq = save_freq&#xA;        self.iteration_count = 0&#xA;        self.training = training&#xA;&#xA;        self.epsilon = epsilon&#xA;        self.epsilon_min = epsilon_min&#xA;        self.epsilon_decay = epsilon_decay&#xA;        self.gamma = gamma&#xA;        self.replace_every = 1000&#xA;&#xA;        self.q_net = DQN(action_space, None, channels, learning_rate)&#xA;        self.target_net = DQN(action_space, None, channels, learning_rate)&#xA;&#xA;    def update_target_network(self):&#xA;        if self.iteration_count % self.replace_every == 0:&#xA;            self.target_net.load_state_dict(self.q_net.state_dict())&#xA;&#xA;    def decay_epsilon(self):&#xA;        self.epsilon = self.epsilon - self.epsilon_decay \&#xA;            if self.epsilon &amp;gt; self.epsilon_min else self.epsilon_min&#xA;&#xA;    def act(self, state) -&amp;gt; int:&#xA;        if random.random() &amp;gt; self.epsilon:&#xA;            state = torch.tensor(state, device=device)&#xA;            return self.q_net(state.unsqueeze(0)).argmax(1).item()&#xA;        else:&#xA;            return random.randrange(self.action_space)&#xA;&#xA;    def learn(self, *args, **kwargs):&#xA;        self.memorise(kwargs.get('state'),&#xA;                      kwargs.get('action'),&#xA;                      kwargs.get('reward'),&#xA;                      kwargs.get('next_state'),&#xA;                      kwargs.get('done'))&#xA;&#xA;        if not self.training:&#xA;            return&#xA;&#xA;        if self.get_mem_count() &amp;gt;= self.batch_size:&#xA;            sample = self.sample(self.batch_size)&#xA;            states = torch.stack([i[0] for i in sample])&#xA;            actions = torch.tensor([i[1] for i in sample], device=device)&#xA;            rewards = torch.tensor([i[2] for i in sample], dtype=torch.float32, device=device)&#xA;            next_states = torch.stack([i[3] for i in sample])&#xA;            dones = torch.tensor([i[4] for i in sample], dtype=torch.uint8, device=device)&#xA;&#xA;            self.update_target_network()&#xA;&#xA;            current_q_vals = self.q_net(states)&#xA;            next_q_vals = self.target_net(next_states)&#xA;            q_target = current_q_vals.clone().detach()&#xA;            q_target[torch.arange(states.size()[0]), actions] = rewards + (self.gamma * next_q_vals.max(dim=1)[0]) * (&#xA;                ~dones).float()&#xA;&#xA;            self.q_net.optimizer.zero_grad()&#xA;            loss = self.q_net.loss(current_q_vals, q_target)&#xA;            loss.backward()&#xA;            self.q_net.optimizer.step()&#xA;&#xA;            self.decay_epsilon()&#xA;            self.iteration_count += 1&#xA;&#xA;    def memorise(self, state, action, reward, next_state, done):&#xA;        # state = self.transform(state).to(device)&#xA;        # next_state = self.transform(next_state).to(device)&#xA;        state = torch.tensor(state, device=device)&#xA;        next_state = torch.tensor(next_state, device=device)&#xA;&#xA;        self.store(&#xA;            state,&#xA;            action,&#xA;            reward,&#xA;            next_state,&#xA;            done)&#xA;&#xA;    def flatten(self, x):&#xA;        flattened_count = 1&#xA;        for dim in x.shape[1:]:&#xA;            flattened_count *= dim&#xA;        return x.view(-1, flattened_count)&#xA;&#xA;    def get_epsilon(self):&#xA;        return self.epsilon&#xA;&#xA;    def save_model(self):&#xA;        torch.save(self.q_net.state_dict(), &amp;quot;D:/models/q_net-agent-q-episode-{}&amp;quot;.format(self.iteration_count))&#xA;        torch.save(self.target_net.state_dict(), &amp;quot;D:/models/target_net-agent_q-episode-{}&amp;quot;.format(self.iteration_count))&#xA;&#xA;&#xA;class SkipEnv(gym.Wrapper):&#xA;    def __init__(self, env=None, skip=4):&#xA;        super(SkipEnv, self).__init__(env)&#xA;        self._skip = skip&#xA;&#xA;    def step(self, action):&#xA;        t_reward = 0.0&#xA;        done = False&#xA;        for _ in range(self._skip):&#xA;            obs, reward, done, info = self.env.step(action)&#xA;            t_reward += reward&#xA;            if done:&#xA;                break&#xA;        return obs, t_reward, done, info&#xA;&#xA;    def reset(self):&#xA;        self._obs_buffer = []&#xA;        obs = self.env.reset()&#xA;        self._obs_buffer.append(obs)&#xA;        return obs&#xA;&#xA;class PreProcessFrame(gym.ObservationWrapper):&#xA;    def __init__(self, env=None):&#xA;        super(PreProcessFrame, self).__init__(env)&#xA;        self.observation_space = gym.spaces.Box(low=0, high=255,&#xA;                                                shape=(80,80,1), dtype=np.uint8)&#xA;    def observation(self, obs):&#xA;        return PreProcessFrame.process(obs)&#xA;&#xA;    @staticmethod&#xA;    def process(frame):&#xA;&#xA;        new_frame = np.reshape(frame, frame.shape).astype(np.float32)&#xA;&#xA;        new_frame = 0.299*new_frame[:,:,0] + 0.587*new_frame[:,:,1] + \&#xA;                    0.114*new_frame[:,:,2]&#xA;&#xA;        new_frame = new_frame[35:195:2, ::2].reshape(80,80,1)&#xA;&#xA;        return new_frame.astype(np.uint8)&#xA;&#xA;class MoveImgChannel(gym.ObservationWrapper):&#xA;    def __init__(self, env):&#xA;        super(MoveImgChannel, self).__init__(env)&#xA;        self.observation_space = gym.spaces.Box(low=0.0, high=1.0,&#xA;                            shape=(self.observation_space.shape[-1],&#xA;                                   self.observation_space.shape[0],&#xA;                                   self.observation_space.shape[1]),&#xA;                            dtype=np.float32)&#xA;&#xA;    def observation(self, observation):&#xA;        return np.moveaxis(observation, 2, 0)&#xA;&#xA;class ScaleFrame(gym.ObservationWrapper):&#xA;    def observation(self, obs):&#xA;        return np.array(obs).astype(np.float32) / 255.0&#xA;&#xA;class BufferWrapper(gym.ObservationWrapper):&#xA;    def __init__(self, env, n_steps):&#xA;        super(BufferWrapper, self).__init__(env)&#xA;        self.observation_space = gym.spaces.Box(&#xA;                             env.observation_space.low.repeat(n_steps, axis=0),&#xA;                             env.observation_space.high.repeat(n_steps, axis=0),&#xA;                             dtype=np.float32)&#xA;&#xA;    def reset(self):&#xA;        self.buffer = np.zeros_like(self.observation_space.low, dtype=np.float32)&#xA;        return self.observation(self.env.reset())&#xA;&#xA;    def observation(self, observation):&#xA;        self.buffer[:-1] = self.buffer[1:]&#xA;        self.buffer[-1] = observation&#xA;        return self.buffer&#xA;&#xA;def make_env(env_name):&#xA;    env = gym.make(env_name)&#xA;    env = SkipEnv(env)&#xA;    env = PreProcessFrame(env)&#xA;    env = MoveImgChannel(env)&#xA;    env = BufferWrapper(env, 4)&#xA;    return ScaleFrame(env)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41097" LastActivityDate="2020-10-04T08:48:19.027" Title="Pytorch and keras ddqn seem identical, only keras learns" Tags="&lt;convolutional-neural-networks&gt;&lt;q-learning&gt;&lt;keras&gt;&lt;dqn&gt;&lt;pytorch&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23894" PostTypeId="1" CreationDate="2020-10-04T11:15:38.287" Score="0" ViewCount="32" Body="&lt;p&gt;I am an amateur with codes but still interested by ML and NN applications.&#xA;How would I go to convert one file format to another by using neural network. e.g: mp3 to wav, fbx to obj, wav to png, unknown to a known format&lt;/p&gt;&#xA;&lt;p&gt;It is basically like having a neural network reverse engineer a format.&lt;/p&gt;&#xA;" OwnerUserId="41390" LastActivityDate="2020-10-04T11:15:38.287" Title="How to convert a certain file format into another using neural network?" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23895" PostTypeId="2" ParentId="23891" CreationDate="2020-10-04T15:20:45.910" Score="2" Body="&lt;p&gt;SOM (Self-Organinizing Map) is a type of artificial neural network (ANN), introduced by the Finnish professor Teuvo Kohonen in the 1980s, that is trained using unsupervised learning to produce a low-dimensional, discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction.&lt;/p&gt;&#xA;&lt;p&gt;SOM produces a mapping from a multidimensional input space onto a lattice of clusters, i.e. neurons, in a way that preserves their topology, so that&#xA;neighboring neurons respond to “similar” input patterns.&lt;/p&gt;&#xA;&lt;p&gt;It uses three basic processes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Competition&lt;/li&gt;&#xA;&lt;li&gt;Cooperation&lt;/li&gt;&#xA;&lt;li&gt;Adaptation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;In competition,&lt;/em&gt; each neuron is assigned a weight vector with the same&#xA;dimensionality d as the input space. Any given input pattern is compared to the weight vector of each neuron and the closest neuron is declared the winner.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;In cooperation&lt;/em&gt;, the activation of the winning neuron is spread to neurons in its immediate neighborhood, and as a result this allows topologically close neurons to become sensitive to similar patterns. The size of the neighborhood is initially large, but shrinks over time, where an initially large neighborhood promotes a topology-preserving mapping and smaller neighborhoods allows neurons to specialize in the latter stages of training.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;In adaptation&lt;/em&gt;, the winner neuron and its topological neighbors are adapted to make their weight vectors more similar to the input pattern that caused the activation.&lt;/p&gt;&#xA;&lt;p&gt;So, a Self-Organizing Map (SOM) is to encode a large set of input vectors &lt;span class=&quot;math-container&quot;&gt;$\textbf{x}$&lt;/span&gt; by finding a smaller set of “representatives” or&#xA;“prototypes” or “code-book vectors” &lt;span class=&quot;math-container&quot;&gt;$\textbf{w}$&lt;/span&gt; that provide a good approximation to the original input space. This is the basic idea of vector quantization theory, the motivation of which is dimensionality reduction or data compression. Performing a gradient descent style minimization on SOM's loss function (eg. the sum of the Euclidean distances between the input sample and each neuron) does lead to the SOM weight update algorithm, which confirms that it is generating the best possible discrete low dimensional approximation to the input space (at least assuming it does not get trapped in a local minimum of the error function).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;To answer your question, you should take into consideration that the Dimensionality reduction takes place in fields that deal with large numbers of observations and/or large numbers of variables. Thus, SOM helps finding good &amp;quot;prototypes&amp;quot;, in a way that each input pattern belongs to exactly one of them. As a result, the training instances are mapped to the training &amp;quot;prototypes&amp;quot; and the whole training set is mapped to a new one with less instances.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition, the &amp;quot;prototypes&amp;quot; neurons resulted by SOM can often be used as good centers in RBF networks or to classify patterns with the LVQ family algorithms.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-10-04T15:33:03.547" LastActivityDate="2020-10-04T15:33:03.547" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23896" PostTypeId="2" ParentId="7608" CreationDate="2020-10-04T16:39:51.847" Score="2" Body="&lt;p&gt;Use of Transposed Convolution can lead to checkerboard artifacts.  So we prefer to up-sample and then apply convolution. You can check this article for more information &lt;a href=&quot;https://distill.pub/2016/deconv-checkerboard/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://distill.pub/2016/deconv-checkerboard/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41392" LastActivityDate="2020-10-04T16:39:51.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23897" PostTypeId="1" CreationDate="2020-10-04T23:38:49.493" Score="2" ViewCount="31" Body="&lt;p&gt;I am trying to create a model that is using a &lt;em&gt;one-shot learning&lt;/em&gt; approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this approach (it is going to be a university project). The task would be to classify objects, probably from drone/satellite image (of course zoomed one).&lt;/p&gt;&#xA;&lt;p&gt;My question is, do you think it would be ok to use a model for face recognition, such as &lt;a href=&quot;https://www.cs.toronto.edu/%7Eranzato/publications/taigman_cvpr14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepFace&lt;/a&gt; or &lt;a href=&quot;http://elijah.cs.cmu.edu/DOCS/CMU-CS-16-118.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenFace&lt;/a&gt;, and, using transfer learning, retrain it on my classes?&lt;/p&gt;&#xA;" OwnerUserId="38252" LastEditorUserId="2444" LastEditDate="2020-10-05T21:11:18.847" LastActivityDate="2020-10-05T21:11:18.847" Title="Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?" Tags="&lt;object-recognition&gt;&lt;transfer-learning&gt;&lt;one-shot-learning&gt;&lt;deep-face&gt;&lt;open-face&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23898" PostTypeId="1" AcceptedAnswerId="23904" CreationDate="2020-10-04T23:48:19.497" Score="2" ViewCount="64" Body="&lt;p&gt;I am wondering what is believed to be the reason for superiority of transformer?&lt;/p&gt;&#xA;&lt;p&gt;I see that some people believe because of the attention mechanism used, it’s able to capture much longer dependencies. However, as far as I know, you can use attention also with RNN  architectures as in the famous paper attention is introduced(&lt;a href=&quot;https://arxiv.org/pdf/1409.0473&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)).&lt;/p&gt;&#xA;&lt;p&gt;I am wondering whether the only reason for the superiority of transformers is because they can be highly parallelized and trained on much more data?&lt;/p&gt;&#xA;&lt;p&gt;Is there any experiment comparing transformers and RNN+attention trained on the exact same amount of data comparing the two?&lt;/p&gt;&#xA;" OwnerUserId="40551" LastActivityDate="2020-10-05T09:44:26.683" Title="Any comparison between transformer and RNN+Attention on the same dataset?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23902" PostTypeId="1" CreationDate="2020-10-05T03:17:02.310" Score="1" ViewCount="226" Body="&lt;p&gt;In section 7.3 of the book &lt;a href=&quot;https://cs.calvin.edu/courses/cs/344/kvlinden/resources/AIMA-3rd-edition.pdf#page=261&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (3rd edition), it's written&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;An inference algorithm that derives only entailed sentences is called &lt;strong&gt;sound&lt;/strong&gt; or &lt;strong&gt;truth-preserving&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The property of &lt;strong&gt;completeness&lt;/strong&gt; is also desirable: an inference algorithm is complete if it can derive any sentence that is entailed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, this does not make much sense to me. I'd like someone to kindly elaborate on this.&lt;/p&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2021-01-24T13:29:48.427" LastActivityDate="2021-01-24T13:29:48.427" Title="What is the difference between derivation and entailment?" Tags="&lt;comparison&gt;&lt;logic&gt;&lt;norvig-russell&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23904" PostTypeId="2" ParentId="23898" CreationDate="2020-10-05T09:19:48.303" Score="2" Body="&lt;p&gt;If you go through the &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;main introductory paper of the transformer&lt;/a&gt; (&amp;quot;Attention is all you need&amp;quot;), you can find the comparison of the model with other state-of-the-art machine translation method:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o4SRo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o4SRo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;https://arxiv.org/pdf/1606.04199.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep-Att + PosUnk&lt;/a&gt; is a method that has utilized RNN and attention for the translation task. As you can see, the training cost for the transformer with self-attention is &lt;span class=&quot;math-container&quot;&gt;$2.3 \cdot 10^{19}$&lt;/span&gt; (FLOPs) and &lt;span class=&quot;math-container&quot;&gt;$1.0 \cdot 10^{20}$&lt;/span&gt; (FLOPs) for the &amp;quot;Deep-Att + PosUnk&amp;quot; method (the transformer is 4 times faster) on &amp;quot;WMT14 English-to-French&amp;quot; dataset.&lt;/p&gt;&#xA;&lt;p&gt;Please note that the BLEU is a crucial factor here (not merely training cost). Hence, you can see the BLEU‌ value of the transformer superior to the &lt;a href=&quot;https://arxiv.org/pdf/1610.10099.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ByteNet&lt;/a&gt; (Neural Machine Translation in Linear Time). Although the ByteNet has not adopted the RNN, you can find the comparison of the ByteNet with other &amp;quot;RNN + Attention&amp;quot; methods in its original paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/abdit.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/abdit.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hence, by transitivity property of the BLEU score, you can find that the transformer has already outperformed other &amp;quot;RNN‌ + Attention&amp;quot; methods in terms of the BLEU score (please check their performance on &amp;quot;WMT14&amp;quot; dataset).&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-10-05T09:44:26.683" LastActivityDate="2020-10-05T09:44:26.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23905" PostTypeId="1" CreationDate="2020-10-05T09:28:44.073" Score="0" ViewCount="81" Body="&lt;p&gt;I am quite new to neural networks, and would like to save myself some of the learning curve by having some rules of thumb about hidden layer sizes.&lt;/p&gt;&#xA;&lt;p&gt;I would also like to have a rule of thumb for the ratio between the size of the dataset and the size or depth of the network.&lt;/p&gt;&#xA;&lt;p&gt;Currently, for example, I need to create an autoencoder for a dataset which is about 3000 images of a small size, say 25x25.&#xA;I would assume the innermost layer would be of size ~10, but I don't really know why would it be either more or less, or what I should see in order to want to change it in either direction.&lt;br /&gt;&#xA;Same goes for any other layer.&lt;/p&gt;&#xA;&lt;p&gt;Are there any thumb rules for network sizes? Are there any thumb rules for optimization of sizes? I find it hard to believe trial and error is all there is, but I found now course that teaches how to do that.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2020-10-05T21:23:46.737" Title="Rules of thumb for hidden layer sizes" Tags="&lt;neural-networks&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2020-10-05T21:19:49.787" ContentLicense="CC BY-SA 4.0" />
  <row Id="23906" PostTypeId="1" CreationDate="2020-10-05T10:59:52.110" Score="0" ViewCount="51" Body="&lt;p&gt;I'm doing a student project where I construct a model predicting the number of languages that a given Wikipedia article is translated into (for example, the article &lt;a href=&quot;https://en.wikipedia.org/wiki/Toyota&quot; rel=&quot;nofollow noreferrer&quot;&gt;TOYOTA&lt;/a&gt; is translated into 93 languages). I've tried extracting basic info (article length, number of links, etc.) to create a simple regression model, but can't get the &lt;span class=&quot;math-container&quot;&gt;$R^2$&lt;/span&gt; value above &lt;span class=&quot;math-container&quot;&gt;$0.25$&lt;/span&gt; or so.&lt;/p&gt;&#xA;&lt;p&gt;What's the most appropriate NLP algorithm for &lt;strong&gt;regression&lt;/strong&gt; problems? Almost all examples I find online are &lt;strong&gt;classification&lt;/strong&gt; problems. FYI I'm aware of the basics of NLP preprocessing (tokenization, lemmatization, bag of words, etc).&lt;/p&gt;&#xA;" OwnerUserId="41405" LastEditorUserId="2444" LastEditDate="2020-10-07T10:28:56.340" LastActivityDate="2021-03-06T12:01:35.163" Title="What is the best algorithm to solve the regression problem of predicting the number of languages a Wikipedia article can be translated to?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23907" PostTypeId="1" CreationDate="2020-10-05T13:42:47.370" Score="0" ViewCount="38" Body="&lt;p&gt;Is it possible to detect if a face is real or computer generated?&lt;/p&gt;&#xA;&lt;p&gt;Today the computer generated face look pretty realistic to me. Here is an article about such generated images&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.theverge.com/platform/amp/tldr/2019/2/15/18226005/ai-generated-fake-people-portraits-thispersondoesnotexist-stylegan&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.theverge.com/platform/amp/tldr/2019/2/15/18226005/ai-generated-fake-people-portraits-thispersondoesnotexist-stylegan&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to make the difference?&lt;/p&gt;&#xA;" OwnerUserId="41409" LastEditorUserId="2444" LastEditDate="2020-10-05T20:59:13.423" LastActivityDate="2020-10-05T20:59:13.423" Title="Is it possible to detect if a face is real or computer generated?" Tags="&lt;deepfakes&gt;&lt;face-recognition&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2020-10-05T21:26:40.397" ContentLicense="CC BY-SA 4.0" />
  <row Id="23908" PostTypeId="2" ParentId="11375" CreationDate="2020-10-05T15:25:36.050" Score="5" Body="&lt;p&gt;As a supplement to nbro's nice answer, I think a major difference between RL and optimal control lies in the motivation behind the problem you're solving. As has been pointed out by comments and answers here (as well as the OP), the line between RL and optimal control can be quite blurry.&lt;/p&gt;&#xA;&lt;p&gt;Consider the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic%E2%80%93Gaussian_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear-Quadratic-Gaussian (LQG)&lt;/a&gt; algorithm, which is generally considered to be an optimal control method. Here a controller is computed given a stochastic model of the environment and a cost function.&lt;/p&gt;&#xA;&lt;p&gt;Now, consider AlphaZero, which is obviously thought of as an RL algorithm. AlphaZero learns a value function (and thus a policy/controller) in a perfect information setting with a known deterministic model.&lt;/p&gt;&#xA;&lt;p&gt;So, it's not the stochasticity that separates RL from optimal control, as some people believe. It's also not the presence of a known model. I argue that the difference between RL and optimal control comes from the &lt;strong&gt;generality&lt;/strong&gt; of the algorithms.&lt;/p&gt;&#xA;&lt;p&gt;For instance, generally, when applying LQG and other optimal control algorithms, you have a specific environment in mind and the big challenge is modeling the environment and the reward function to achieve the desired behavior. In RL, on the other hand, the environment is generally thought of as a sort of black box. While in the case of AlphaZero the model of the environment is known, the reward function itself was not designed specifically for the game of chess (for instance, it's +1 for a win and -1 for a loss, regardless of chess, go, etc.). Furthermore, the neat thing with AlphaZero is that we can use it to train agents in virtually any perfect information game without changing the algorithm at all. Another difference with RL here is that the agent iteratively improves itself, while optimal control algorithms learn controllers offline and then stay fixed.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastEditorUserId="2444" LastEditDate="2021-02-12T20:00:39.700" LastActivityDate="2021-02-12T20:00:39.700" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23909" PostTypeId="2" ParentId="20794" CreationDate="2020-10-05T16:08:24.333" Score="0" Body="&lt;p&gt;From my understanding of the paper, &lt;span class=&quot;math-container&quot;&gt;$Z^{pres}$&lt;/span&gt; keeps track of the objects in the scene. For every step of the sequential inference, &lt;span class=&quot;math-container&quot;&gt;$z^{pres,i}$&lt;/span&gt; takes either a 0 or a 1. A 1 represents that an object is present and it has to be explained by the remaining latent variables. 0 indicates that all the objects have been explained and inference is complete.&lt;/p&gt;&#xA;" OwnerUserId="41414" LastActivityDate="2020-10-05T16:08:24.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23910" PostTypeId="1" CreationDate="2020-10-05T19:08:55.510" Score="10" ViewCount="170" Body="&lt;p&gt;The problem of automated theorem proving (ATP) seems to be very similar to playing board games (e.g. chess, go, etc.): it can also be naturally stated as a problem of a decision tree traversal. However, there is a dramatic difference in progress on those 2 tasks: board games are successfully being solved by reinforcement learning techniques nowadays (see AlphaGo and AlphaZero), but ATP is still nowhere near to automatically proving even freshman-level theorems. What does make ATP so hard compared to board games playing?&lt;/p&gt;&#xA;" OwnerUserId="41418" LastActivityDate="2020-11-07T04:02:40.257" Title="Why is automated theorem proving so hard?" Tags="&lt;automated-theorem-proving&gt;&lt;automated-reasoning&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23911" PostTypeId="5" CreationDate="2020-10-05T21:13:31.770" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-05T21:13:31.770" LastActivityDate="2020-10-05T21:13:31.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23912" PostTypeId="4" CreationDate="2020-10-05T21:13:31.770" Score="0" Body="For questions about DeepFace, which was introduced in the paper &quot;DeepFace: Closing the Gap to Human-Level Performance in Face Verification&quot; (2014) by Yaniv Taigman et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-05T21:13:31.770" LastActivityDate="2020-10-05T21:13:31.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23913" PostTypeId="5" CreationDate="2020-10-05T21:16:37.950" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-05T21:16:37.950" LastActivityDate="2020-10-05T21:16:37.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23914" PostTypeId="4" CreationDate="2020-10-05T21:16:37.950" Score="0" Body="For questions about OpenFace, introduced in &quot;OpenFace: A general-purpose face recognition library with mobile applications&quot; (2016) by Brandon Amos et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-05T21:16:37.950" LastActivityDate="2020-10-05T21:16:37.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23915" PostTypeId="1" AcceptedAnswerId="23963" CreationDate="2020-10-06T00:57:46.623" Score="3" ViewCount="105" Body="&lt;p&gt;Pokemon is a game where 2 players each select 6 Pokemon (a team) at the beginning of the game without knowing the other player's team. Every Pokemon has one or two types. Every type is either weak, neutral or strong against every other type. This means that every 2 Pokemon matchup will either have a winner or be a tie. This also means that any team can be ranked against any other team based on the number of winning matchups they have.&lt;/p&gt;&#xA;&lt;p&gt;I want to write a program that can find the optimal Pokemon team out of a set of 70 provided Pokemon.&#xA;A team is considered optimal if it has the greatest number of winning matchups against any other team. Basically, I want to calculate which team will have the most amount of favorable matchups if you were to battle it against every other possible team.&lt;/p&gt;&#xA;&lt;p&gt;What algorithm would be best for doing this? It is not feasible to compute matchups for every possible team. Can I do some sort of A* search with enough pruning to make it computationally feasible?&lt;/p&gt;&#xA;" OwnerUserId="41421" LastActivityDate="2020-10-08T04:14:24.130" Title="How to find the optimal pokemon team" Tags="&lt;ai-design&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23916" PostTypeId="2" ParentId="23772" CreationDate="2020-10-06T04:14:47.790" Score="1" Body="&lt;p&gt;After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a &amp;quot;&lt;strong&gt;Big Task agnostic CNN&lt;/strong&gt;&amp;quot; to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read &lt;a href=&quot;https://arxiv.org/pdf/2006.10029.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Big Self-Supervised Models are Strong Semi-Supervised Learners&lt;/a&gt; by &lt;em&gt;Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi and Geoffrey Hinton&lt;/em&gt;. The code for the following can be found &lt;a href=&quot;https://github.com/google-research/simclr&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. But I feel that training such a model would take up a lot of computational resources.&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-10-06T04:14:47.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23920" PostTypeId="2" ParentId="23906" CreationDate="2020-10-06T07:44:15.817" Score="0" Body="&lt;p&gt;I think it's difficult to tell wich algorithm is &amp;quot;the best&amp;quot; or &amp;quot;the simplest&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I had the same issue of choosing the suited NLP algorithm for my dataset and I used :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I recommanded you to test many algorithms as you can to find the best for your needed.&lt;/p&gt;&#xA;" OwnerUserId="41176" LastActivityDate="2020-10-06T07:44:15.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23922" PostTypeId="1" CreationDate="2020-10-06T08:41:11.710" Score="1" ViewCount="53" Body="&lt;p&gt;What is the status of the capsule networks?&lt;/p&gt;&#xA;&lt;p&gt;I got an impression that capsule networks turned out not to be so useful in applications more complicated than the MNIST (at least according to &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/comments/a83fpx/r_what_is_the_status_about_the_capsule_networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this reddit discussion&lt;/a&gt;​).&lt;/p&gt;&#xA;&lt;p&gt;Is this really the case? Or can they be a promising research direction (and if so, is there any specific application for which they seem the most promising)?&lt;/p&gt;&#xA;" OwnerUserId="36769" LastActivityDate="2020-10-18T04:14:16.990" Title="What is the status of the capsule networks?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;capsule-neural-network&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23923" PostTypeId="2" ParentId="23889" CreationDate="2020-10-06T10:43:53.687" Score="4" Body="&lt;p&gt;The Transformer model presented in this tutorial is an auto-regressive Transformer. Which means that prediction of next token only depends on it's previous tokens.&lt;/p&gt;&#xA;&lt;p&gt;So in order to predict next token, you have to make sure that only previous token are attended. (If not, this would be a cheating because model already knows whats next).&lt;/p&gt;&#xA;&lt;p&gt;So attention mask would be like this&lt;br /&gt;&#xA;[0, 1, 1, 1, 1]&lt;br /&gt;&#xA;[0, 0, 1, 1, 1]&lt;br /&gt;&#xA;[0, 0, 0, 1, 1]&lt;br /&gt;&#xA;[0, 0, 0, 0, 1]&lt;br /&gt;&#xA;[0, 0, 0, 0, 0]&lt;br /&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example: If you are translating English to Spanish&lt;br /&gt;&#xA;&lt;strong&gt;Input: How are you ?&lt;br /&gt;&lt;/strong&gt;&#xA;&lt;strong&gt;Target: &amp;lt; start &amp;gt; Como estas ? &amp;lt; end &amp;gt;&lt;br /&gt;&lt;/strong&gt;&#xA;Then decoder will predict something like this&lt;br /&gt;&#xA;&amp;lt; start &amp;gt; (it will be given to decoder as initial token)&lt;br /&gt;&#xA;&amp;lt; start &amp;gt; Como&lt;br /&gt;&#xA;&amp;lt; start &amp;gt; Como estas&lt;br /&gt;&#xA;&amp;lt; start &amp;gt; Como estas ?&lt;br /&gt;&#xA;&amp;lt; start &amp;gt; Como estas ? &amp;lt; end &amp;gt;&lt;br /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now compare this step by step prediction sequences to attention mask given above, It would make sense now to you&lt;/p&gt;&#xA;" OwnerUserId="41110" LastActivityDate="2020-10-06T10:43:53.687" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23928" PostTypeId="1" AcceptedAnswerId="23932" CreationDate="2020-10-06T19:24:34.350" Score="1" ViewCount="32" Body="&lt;p&gt;The E step on the EM algorithm asks us to set the value of the variational lower bound to be equal to the posterior probability of the latent variable, given the data points and parameters. Clearly we are not taking any expectations here, then why is it called the Expectation step? Am I missing something here?&lt;/p&gt;&#xA;" OwnerUserId="35576" LastActivityDate="2020-10-07T08:12:05.223" Title="Why is the E step in expectation maximisation algorithm called so?" Tags="&lt;bayesian-probability&gt;&lt;expectation-maximization&gt;&lt;bayesian-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23930" PostTypeId="1" CreationDate="2020-10-05T16:18:33.207" Score="4" ViewCount="22" Body="&lt;p&gt;The problem of automated theorem proving (ATP) seems to be very similar to playing board games (e.g. chess, go, etc.): it can also be naturally stated as a problem of a decision tree traversal. However, there is a dramatic difference in progress on those 2 tasks: board games are successfully being solved by reinforcement learning techniques nowadays (see AlphaGo and AlphaZero), but ATP is still nowhere near to automatically proving even freshman-level theorems. What does make ATP so hard compared to board games playing?&lt;/p&gt;&#xA;" OwnerDisplayName="Ivan Ivanov" LastActivityDate="2020-10-06T22:12:28.823" Title="Why is automated theorem proving so hard?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2020-10-07T19:41:51.833" ContentLicense="CC BY-SA 4.0" />
  <row Id="23931" PostTypeId="2" ParentId="23910" CreationDate="2020-10-05T18:29:53.880" Score="2" Body="&lt;p&gt;There are two ways to look at the problem, one in terms of logic and the other in terms of psychology.&lt;/p&gt;&#xA;&lt;p&gt;To get any -start- on automation of mathematics, you need to formalize the part you want. It has only been since the early part of the 20th c that most day to day math had been formalized with logic and set theory. And even though &lt;a href=&quot;https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gödel's incompleteness theorems&lt;/a&gt; say (very loosely) that there is no algorithm to decide theorem-hood for mathematical statements (that include a theory of arithmetic), that still leaves a lot of math that -can- be decided. But that has taken the &lt;a href=&quot;https://en.wikipedia.org/wiki/Reverse_mathematics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reverse Mathematics program&lt;/a&gt; (still ongoing) to say specifically what subsets of math are decidable or to what degree (what logical assumptions are necessary) they are undecidable.&lt;/p&gt;&#xA;&lt;p&gt;So &lt;a href=&quot;https://en.wikipedia.org/wiki/Presburger_arithmetic&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorems in arithmetic of just '+'&lt;/a&gt; (that is, dropping '*') can be decided, &lt;a href=&quot;https://en.wikipedia.org/wiki/Euclidean_geometry#Axiomatic_formulations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euclidean geometry&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Tarski%E2%80%93Seidenberg_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be decided&lt;/a&gt;, single variable differential calculus can be decided but &lt;a href=&quot;https://en.wikipedia.org/wiki/Richardson%27s_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;not single variable integral calculus&lt;/a&gt;. These examples show that what we know to be decidable is pretty elementary. And most of the things we care about are very un-elementary (almost by definition).&lt;/p&gt;&#xA;&lt;p&gt;As to psychology, the theorems and proofs that you learn in mathematics classes are nowhere near &lt;em&gt;like&lt;/em&gt; their formalizations. Most mathematicians aren't pushing symbols around in their heads like a computer does. A mathematician is more like an artist, visualizing dreams and connecting metaphors just on their barely conscious images borne out of repetition. That is, machines and mathematicians just work on different representations (despite what non-mathematicians might imagine).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To address your specific question, yes, mathematical theorems and the systems to prove them are very similar in a technical sense. Games (often, not always) can be modeled as trees, And likewise proofs can often be modeled as trees. Without writing you a library of books about games and proofs, let's just say that the mathematical proofs that are like games that are won by Alpha Zero are not for particularly interesting theorems. Winning a game of go is more like proving that a very very large boolean formula. Most mathematical theorems require a lot of ingenuity in introducing steps in their proof trees. It may be mechanical after the fact to check that a proof is correct, but discovering the proof almost needs magic to come up with a step in the game. Sure, some things in math &lt;em&gt;are&lt;/em&gt; automatable (as mentioned before, derivatives), but some mathematical systems (such as integration) are provably impossible to find proofs of all true statements.&lt;/p&gt;&#xA;&lt;p&gt;Another difference between theorem proving and games is that proofs have to be air tight on all paths, whereas with games one side just has to eke out a single win over the other side.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A separate issue entirely that may contribute to the difficulty is that we just may not yet have the tooling available, ie editors, notation, proof assistants that make it easy to do what should be easy. Or it could just be that mathematicians don't have the fluency with theorem proving systems.&lt;/p&gt;&#xA;&lt;p&gt;Or it could be that if there were automated theorem provers good enough, mathematicians just wouldn't care too much for them because they'd take away the fun of finding the proofs themselves.&lt;/p&gt;&#xA;" OwnerUserId="41442" OwnerDisplayName="Mitch" LastEditorUserId="41442" LastEditDate="2020-10-08T03:13:13.410" LastActivityDate="2020-10-08T03:13:13.410" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23932" PostTypeId="2" ParentId="23928" CreationDate="2020-10-06T22:21:58.907" Score="1" Body="&lt;p&gt;In expectation step, firstly we calculate the posterior of latent variable &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; and then the &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(θ | θ^{(t)})$&lt;/span&gt; is defined as the expected value of the log likelihood of &lt;span class=&quot;math-container&quot;&gt;$θ$&lt;/span&gt;, with respect to the current conditional contribution of &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and the current estimates of &lt;span class=&quot;math-container&quot;&gt;$θ^{(t)}$&lt;/span&gt;&lt;/strong&gt;. In maximization step, we update &lt;span class=&quot;math-container&quot;&gt;$θ$&lt;/span&gt; using the argmax on &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, with respect to &lt;span class=&quot;math-container&quot;&gt;$θ$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(θ | θ^{(t)}) = E_{Z|X,θ^{(t)}}[logL(θ;Χ,Z)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;To be more intuitive, think of k-means as a special case of EM, where in expectation step the &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; variables are defined, that is the latent variables indicating the membership in a cluster, and calculated in a hard assignment way. In maximization step the &lt;span class=&quot;math-container&quot;&gt;$μ$&lt;/span&gt;s of the clusters are updated. If you want to see the corresponding relation for &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; in k-means, I suggest you read the chapter 9.3.2 in C.Bishop's book: Pattern Recognition and Machine Learning.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-10-07T08:12:05.223" LastActivityDate="2020-10-07T08:12:05.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23933" PostTypeId="1" AcceptedAnswerId="23934" CreationDate="2020-10-07T02:21:48.087" Score="4" ViewCount="86" Body="&lt;p&gt;This is the Short Corridor problem taken from the Sutton &amp;amp; Barto book. Here it's written:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The problem is difficult because all the states appear identical under the function approximation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But this doesn't make much sense as we can always choose states as 0,1,2 and corresponding feature vectors as&lt;/p&gt;&#xA;&lt;p&gt;x(S = 0,right) = [1  0  0  0  0  0]&lt;br /&gt;&#xA;x(S = 0 , left)  = [0  1  0  0  0  0]&lt;br /&gt;&#xA;x(S = 1,right) = [0  0  1  0  0  0]&lt;br /&gt;&#xA;x(S = 1 , left) = [0  0  0  1  0  0]&lt;br /&gt;&#xA;x(S = 2,right) = [0  0  0  0  1  0]&lt;br /&gt;&#xA;x(S = 2 , left) = [0  0  0  0  0  1]\&lt;/p&gt;&#xA;&lt;p&gt;So why is it written that all the states appear identical under the function approximation?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lU6Nm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lU6Nm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="2444" LastEditDate="2020-10-07T17:32:25.130" LastActivityDate="2020-10-07T17:32:25.130" Title="Why do all states appear identical under the function approximation in the Short Corridor task?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;function-approximation&gt;&lt;sutton-barto&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23934" PostTypeId="2" ParentId="23933" CreationDate="2020-10-07T02:51:50.713" Score="3" Body="&lt;p&gt;You can choose those states, but is the agent aware of the state it is in? From the text, it seems that the agent cannot distinguish between the three states. Its observation function is completely uninformative.&lt;/p&gt;&#xA;&lt;p&gt;This is why a &lt;em&gt;stochastic&lt;/em&gt; policy is what is needed. This is common for POMDPs, whereas for regular MDPs we can always find a &lt;em&gt;deterministic&lt;/em&gt; policy that is guaranteed to be optimal.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-10-07T02:51:50.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23935" PostTypeId="2" ParentId="23933" CreationDate="2020-10-07T07:50:52.210" Score="1" Body="&lt;p&gt;In toy problems like the Short Corridor task, you can choose the state representation to explore a key property, such as the ability of a particular method to solve it. Often this is done to extremes and heavily simplified.&lt;/p&gt;&#xA;&lt;p&gt;That is what is going on here. The state space that the agent is allowed to use is made highly degenerate with respect to the problem. This stands in for perhaps more complex partially observable systems, but in a way that is really clear to the reader. Also, it is still possible to derive analytically what the best policy should be, so methods can be examined as to how well they deal with the core issue (here, that state data is ambiguous).&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-10-07T07:50:52.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23936" PostTypeId="1" CreationDate="2020-10-07T08:16:07.970" Score="0" ViewCount="32" Body="&lt;p&gt;I am training a U-Net for semantic segmentation of large medical images (4096x4096px). The two classes are &amp;quot;too&amp;quot; unbalanced. The white pixels are just about 0.1% (or less) of the whole image. The Dice Coeff loss function seems to not be working since it predicts always black pixels.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any specialized loss function for such unbalanced data? I can not find anything that works.&lt;/li&gt;&#xA;&lt;li&gt;Is the U-Net arcitecture suitable for such segmentation tasks?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have tried to train with the following setup:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Epochs:          50&#xA;    Batch size:      4&#xA;    Learning rate:   1e-05&#xA;    Training size:   451&#xA;    Validation size: 23&#xA;    Checkpoints:     True&#xA;    Device:          cuda&#xA;    Images scaling:  0.25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and also with batch size of 1 and learning rate of 10^4.&#xA;I would appreciate some help.&lt;/p&gt;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="41311" LastActivityDate="2020-10-07T08:16:07.970" Title="Choice of loss function for semantic segmentation" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23937" PostTypeId="1" CreationDate="2020-10-07T10:49:21.020" Score="1" ViewCount="14" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/abs/1909.01377&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Equilibrium Model&lt;/a&gt; the neural network can be seen as &amp;quot;infinitely deep&amp;quot;. Training learns a nonlinear function as usual. But there is no forward propagation of input data through layers. Instead, a root finding problem is solved when data comes in.&lt;/p&gt;&#xA;&lt;p&gt;My question is, what is actually the function for which roots are searched, I'm struggling to see what would be unknown when data is available and parameters have been found in training?&lt;/p&gt;&#xA;" OwnerUserId="41448" LastActivityDate="2020-10-07T10:49:21.020" Title="Root finding in Deep Equilibrium Models" Tags="&lt;optimization&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23938" PostTypeId="1" CreationDate="2020-10-07T13:20:47.793" Score="1" ViewCount="41" Body="&lt;p&gt;I'm processing a semi-structured scientific document and trying to extract some specific concepts. I've actually made quite good progress without machine-learning so far, but I got to a block of true free text and I'm wondering whether a very narrow sense NLP/learning algorithm can help.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, there are concepts I know to be important that are discussed in this section, but I'll need some NLP to get the 'sentiment'. I thought this might 'entity sentiment' analysis, however, I'm not trying to capture the writer's emotion about a concept. It's literally whether the writer of the text thinks the entity is present, absent, or is uncertain about an entity.&lt;/p&gt;&#xA;&lt;p&gt;Simple example. &amp;quot;First, there are horns. And second, the sheer size of this enormous fossil record argues for an herbivore or omnivore. The jaws are large, but this is not a carnivore.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;And say my entities are horns (presence or absence), and type of dinosaur (herbivore, omnivore, carnivore). Desired output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Horns (present)&#xA;Carnivore (absent)&#xA;Herbivore (possible/present) -- fine if it thinks 'present'&#xA;Omnivore (possible/present) -- fine if it thinks 'present'&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What is the class of NLP analysis that takes an explicit input entity (or list of entities) and tries to assess based on context whether that entity is present or absent according to the writer of the input text?&lt;/strong&gt; It's actually fine if this isn't a learning algorithm (maybe better). Bonus if you have suggestions for python packages that could be used in this narrow sense. I've looked casually through NLTK and spacy packages but they're both vast, wasn't obvious which class of model or functions I'd need to solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="41451" LastEditorUserId="41451" LastEditDate="2020-10-09T11:43:15.277" LastActivityDate="2020-10-09T11:43:15.277" Title="Determining if an entity in free text is 'present' or 'absent'; what is this called in NLP?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;named-entity-recognition&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23939" PostTypeId="5" CreationDate="2020-10-07T16:51:04.537" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T16:51:04.537" LastActivityDate="2020-10-07T16:51:04.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23940" PostTypeId="4" CreationDate="2020-10-07T16:51:04.537" Score="0" Body="For questions about designing (or defining) reward functions e.g. for reinforcement learning problems." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T16:51:04.537" LastActivityDate="2020-10-07T16:51:04.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23941" PostTypeId="1" AcceptedAnswerId="23952" CreationDate="2020-10-07T17:03:55.540" Score="1" ViewCount="87" Body="&lt;p&gt;In RL, both the KL divergence (DKL) and Total variational divergence (DTV) are used to measure the distance between two policies. I'm most familiar with using DKL as an early stopping metric during policy updates to ensure the new policy doesn't deviate much from the old policy.&lt;/p&gt;&#xA;&lt;p&gt;I've seen DTV mostly being used in papers giving approaches to safe RL when placing safety constraints on action distributions. Such as in &lt;a href=&quot;https://arxiv.org/pdf/1705.10528&quot; rel=&quot;nofollow noreferrer&quot;&gt;Constrained Policy Optimization&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1805.07708&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lyapunov Approach to safe RL&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've also seen that they are related by this formula:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{TV} = \sqrt{0.5 D_{KL}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When you compute the &lt;span class=&quot;math-container&quot;&gt;$D_{KL}$&lt;/span&gt; between two polices, what does that tell you about them, and how is it different from what a &lt;span class=&quot;math-container&quot;&gt;$D_{TV}$&lt;/span&gt; between the same two policies tells you?&lt;/p&gt;&#xA;&lt;p&gt;Based on that, are there any specific instances to prefer one over the other?&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="2444" LastEditDate="2020-10-07T22:27:00.083" LastActivityDate="2020-10-08T14:59:14.223" Title="When should one prefer using Total Variational Divergence over KL divergence in RL" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;probability-distribution&gt;&lt;kl-divergence&gt;&lt;total-variational-distance&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23942" PostTypeId="5" CreationDate="2020-10-07T17:26:04.977" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T17:26:04.977" LastActivityDate="2020-10-07T17:26:04.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23943" PostTypeId="4" CreationDate="2020-10-07T17:26:04.977" Score="0" Body="For questions about the sparsity of the rewards (or reward function), which can slow down learning. Reward shaping can be used to solve this problem." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T17:26:04.977" LastActivityDate="2020-10-07T17:26:04.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23945" PostTypeId="1" CreationDate="2020-10-07T18:13:43.397" Score="0" ViewCount="14" Body="&lt;p&gt;I have a dataset of streets - and each street contains several houses.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;street_id | house_id | number of calls | number of people living&#xA;1         | 1        | 10              | 5&#xA;1         | 2        | 3               | 1&#xA;2         | 3        | 4               | 2&#xA;2         | 4        | 15              | 19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My goal is to find for &lt;strong&gt;each street, the most popular house&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I &lt;strong&gt;have GT&lt;/strong&gt; - streets with several houses, and for each street, which house was the most popular house.&lt;/li&gt;&#xA;&lt;li&gt;For the data example above - the results should be &lt;em&gt;(street 1 -&amp;gt; house 1, street 2 -&amp;gt; house 4)&lt;/em&gt; because houses &lt;em&gt;#1, #4&lt;/em&gt; are the most popular in their street.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;We can find the most popular street using features about each house -&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;number of calls made from house&lt;/li&gt;&#xA;&lt;li&gt;number of people living in house&lt;/li&gt;&#xA;&lt;li&gt;etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But we don't need to find all of the popular houses from all of the houses - &lt;strong&gt;we need for each street, to find the most popular house.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do you know how can I solve this task?&lt;br /&gt;&#xA;I've thought about regular classification, but that would results giving all of the popular houses, and not one for each street.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="41456" LastActivityDate="2020-10-07T18:13:43.397" Title="ML Algorithm for getting top pick in each sample" Tags="&lt;machine-learning&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23946" PostTypeId="1" AcceptedAnswerId="23960" CreationDate="2020-10-07T18:19:44.470" Score="2" ViewCount="51" Body="&lt;p&gt;In the average reward setting we have:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(\pi)\doteq \lim_{h\rightarrow\infty}\frac{1}{h}\sum_{t=1}^{h}\mathbb{E}[R_{t}|S_0,A_{0:t-1}\sim\pi]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r(\pi)\doteq \lim_{t\rightarrow\infty}\mathbb{E}[R_{t}|S_0,A_{0:t-1}\sim\pi]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;How is the second equation derived from the first?&lt;/p&gt;&#xA;" OwnerUserId="37611" LastEditorUserId="36821" LastEditDate="2020-10-07T23:52:29.573" LastActivityDate="2020-10-07T23:57:01.570" Title="How do we derive the expression for average reward setting in continuing tasks?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;function-approximation&gt;&lt;sutton-barto&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23947" PostTypeId="2" ParentId="23922" CreationDate="2020-10-07T20:01:05.083" Score="-1" Body="&lt;p&gt;ML is full with things that supposed to work better (in theory).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Sigmoid function seems better than ReLu.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;L1 seems way better than L2.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Spikes neural network seem to be better than standard neural network.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A shallow neural network with a lot of neurons has more parameters than a deep one with the same amount of neurons. So, in theory, has to be more powerful.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Is common to forget the big impact of the training capacity. And easy to get charmed by its theoretical capacity.&lt;br /&gt;&#xA;Training process are still in diapers. We lack a good understating of it.&lt;br /&gt;&#xA;Today, Capsules neural network are a very powerful architecture, hard to train.&lt;/p&gt;&#xA;" OwnerUserId="41126" LastActivityDate="2020-10-07T20:01:05.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23948" PostTypeId="1" AcceptedAnswerId="23959" CreationDate="2020-10-07T20:34:03.607" Score="2" ViewCount="43" Body="&lt;p&gt;I am reading &amp;quot;Reinforcement Learning: An Introduction (2nd edition)&amp;quot; authored by Sutton and Barto. In Section 9, On-policy prediction with approximation, it first gives the mean squared value error objective function in (9.1):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\bar{VE}(\boldsymbol{w}) = \sum_{s \in S} \mu(s)[v_{\pi}(s) - \hat{v}(s,\boldsymbol{w})]^2$&lt;/span&gt;. (9.1)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{w}$&lt;/span&gt; is a vector of the parameterized function &lt;span class=&quot;math-container&quot;&gt;$\hat{v}(s,\boldsymbol{w})$&lt;/span&gt; that approximates the value function &lt;span class=&quot;math-container&quot;&gt;$v_{\pi}(s)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; is the fraction of time spent in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, which measures the &amp;quot;importance&amp;quot; of state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\bar{VE}(\boldsymbol{w})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In (9.4), it states an update rule of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{w}$&lt;/span&gt; by gradient descent:&#xA;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{w}_{t+1} = \boldsymbol{w} -\frac{1}{2}\alpha \nabla[v_{\pi}(S_t) - \hat{v}(S_t,\boldsymbol{w})]^2$&lt;/span&gt;. (9.4)&lt;/p&gt;&#xA;&lt;p&gt;I have two questions regarding (9.4).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why &lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; is not in (9.4)?&lt;/li&gt;&#xA;&lt;li&gt;Why is it the &amp;quot;minus&amp;quot; instead of &amp;quot;+&amp;quot; in (9.4)? In other words, why is it &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{w} -\frac{1}{2}\alpha \nabla[v_{\pi}(S_t) - \hat{v}(S_t,\boldsymbol{w})]^2$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{w} +\frac{1}{2}\alpha \nabla[v_{\pi}(S_t) - \hat{v}(S_t,\boldsymbol{w})]^2$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40506" LastEditorUserId="2444" LastEditDate="2020-10-09T20:48:30.700" LastActivityDate="2020-10-09T20:48:30.700" Title="Why is the fraction of time spent in state $s$, $\mu(s)$, not in the update rule of the parameters?" Tags="&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;function-approximation&gt;&lt;sutton-barto&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23949" PostTypeId="2" ParentId="21604" CreationDate="2020-10-07T20:48:29.393" Score="0" Body="&lt;p&gt;You will need to convert that to something that a neural network can understand.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Movie Name&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Is useless. At least you want to judge a movie by its name.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Description&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You will need to perform a Tokenization. Grab the x's more common words and convert them on an array.&lt;br /&gt;&#xA;I recommend you to see these videos from TensorFlow.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=fNxaJsNG3-s&amp;amp;t=18s&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=fNxaJsNG3-s&amp;amp;t=18s&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There you can find the Google Colab Links for the job.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Director&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There are two options. A One Hot Array or an integer for every director.&lt;br /&gt;&#xA;One Hot Array may be better if you don't have an order of similarity of the directors. But will increase the size of the inputs.&lt;br /&gt;&#xA;If you have an order of similarity of the directors an integer for each director will work fine.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rating&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Nothing to do here. Is ready to go.&lt;/p&gt;&#xA;&lt;p&gt;You can perform this work directly on excel, but tensorflow has great tools for it.&#xA;It’s hard to run the model after you have trained it on excel.&lt;br /&gt;&#xA;If you are to comfortable on excel or you can install new software on your computer, I’ve made a backpropagation algorithm that runs on excel and give you a formula to paste in a module.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/TorrensJoaquin/Multivariant-Nonlinear-Regression-in-VBA-Small-Neural-Network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/TorrensJoaquin/Multivariant-Nonlinear-Regression-in-VBA-Small-Neural-Network&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41126" LastActivityDate="2020-10-07T20:48:29.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23950" PostTypeId="1" CreationDate="2020-10-07T20:52:31.653" Score="0" ViewCount="24" Body="&lt;p&gt;I want to train a convolutional neural network for object detection (say YOLO) to detect faces. Consider this image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dGw0L.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dGw0L.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this training image, I have many people, but only 2 of them are annotated. Is having this kind of images (where target classes are not all annotated) will train the network to ignore positives?&lt;/p&gt;&#xA;&lt;p&gt;If yes, are there any techniques to solve the issue apart from annotating the data (I don't have enough resources to do that).&lt;/p&gt;&#xA;" OwnerUserId="30497" LastActivityDate="2020-10-08T06:33:12.763" Title="Is training a CNN object detector on an image containing multiple targets that are not all annotated will teach it to miss targets?" Tags="&lt;computer-vision&gt;&lt;training&gt;&lt;object-detection&gt;&lt;supervised-learning&gt;&lt;semi-supervised-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23952" PostTypeId="2" ParentId="23941" CreationDate="2020-10-07T22:08:47.063" Score="1" Body="&lt;p&gt;I did not read those two specified linked/cited papers and I am not currently familiar with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Total_variation_distance_of_probability_measures&quot; rel=&quot;nofollow noreferrer&quot;&gt;total variation distance&lt;/a&gt;, but I think I can answer some of your questions, given that I am reasonably familiar with the KL divergence.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When you compute the &lt;span class=&quot;math-container&quot;&gt;$D_{KL}$&lt;/span&gt; between two polices, what does that tell you about them&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The KL divergence is a measure of &amp;quot;distance&amp;quot; (or divergence, as the name suggests) between two probability distributions (i.e. probability measures) or probability densities. In reinforcement learning, &lt;a href=&quot;https://ai.stackexchange.com/q/12274/2444&quot;&gt;(stochastic) policies&lt;/a&gt; are probability distributions. For example, in the case your Markov decision process (MDP) has a discrete set of actions, then your policy can be denoted as &lt;span class=&quot;math-container&quot;&gt;$$\pi(a \mid s),$$&lt;/span&gt;which is the conditional probability distribution over all possible actions, given a specific state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Hence, the KL divergence is a natural measure of how two policies are similar or different.&lt;/p&gt;&#xA;&lt;p&gt;There are 4 properties of the KL divergence that you always need to keep in mind&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It is asymmetric, i.e., in general, &lt;span class=&quot;math-container&quot;&gt;$D_{KL}(q, p) \neq D_{KL}(p, q)$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; are p.d.s); consequently, the KL divergence &lt;strong&gt;cannot&lt;/strong&gt; be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;metric&lt;/a&gt; (because metrics are symmetric!)&lt;/li&gt;&#xA;&lt;li&gt;It is always non-negative&lt;/li&gt;&#xA;&lt;li&gt;It is zero when &lt;span class=&quot;math-container&quot;&gt;$p = q$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;It is unbounded, i.e. it can be arbitrarily large; so, in other words, two probability distributions can be infinitely different, which may not be very intuitive: in fact, in the past, I used the KL divergence and, because of this property, it wasn't always clear how I should interpret the KL divergence (but this may also be due to my not extremely solid understanding of this measure).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;and how is it different from what a &lt;span class=&quot;math-container&quot;&gt;$D_{TV}$&lt;/span&gt; between the same two policies tells you?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$D_{TV}$&lt;/span&gt; is also a measure of the distance between two probability distributions, but it is &lt;strong&gt;bounded&lt;/strong&gt;, specifically, in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt; [&lt;a href=&quot;https://arxiv.org/pdf/math/0209021.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. This property may be useful in some circumstances (which ones?). In any case, the fact that it lies in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt; potentially makes its interpretation more &lt;em&gt;intuitive&lt;/em&gt;. More precisely, if you know the maximum and minimum values that a measure can give you, you can have a better idea of the relative difference between probability distributions. For instance, imagine that you have p.d.s &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p'$&lt;/span&gt;. If you compute &lt;span class=&quot;math-container&quot;&gt;$D_{TV}(q, p)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D_{TV}(q, p')$&lt;/span&gt;, you can have a sense (in terms of percentage) of how much &lt;span class=&quot;math-container&quot;&gt;$p'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; differ with respect to &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The choice between &lt;span class=&quot;math-container&quot;&gt;$D_{TV}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D_{KL}$&lt;/span&gt; is probably motivated by their specific properties (and it will probably depend on a case by case basis, and I expect the authors of the research papers to motivate the usage of a specific measure/&lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;metric&lt;/a&gt;). However, keep in mind that there is not always a closed-form solution not even to calculate the KL divergence, so you may need to approximate it (e.g. by sampling: note that the KL divergence is defined as an expectation/integral so you can approximate it with a sampling technique). So, this (computability and/or approximability) may also be a parameter to take into account when choosing one over the other.&lt;/p&gt;&#xA;&lt;p&gt;By the way, I think that your definition of the &lt;em&gt;total variational divergence&lt;/em&gt; is wrong, although the DTV is related to the DKL, specifically, as follows [&lt;a href=&quot;https://arxiv.org/pdf/math/0209021.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;D_{TV} \leq \sqrt{\frac{1}{2} D_{KL}}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the DTV is bounded by the KL divergence. Given that the KL divergence is unbounded (e.g. it can take very big values, such as 600k, this bound should be very loose).&lt;/p&gt;&#xA;&lt;p&gt;Take a look at the paper &lt;a href=&quot;https://arxiv.org/pdf/math/0209021.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On choosing and bounding probability metrics&#xA;&lt;/a&gt; (2002, by Alison L. Gibbs and  Francis Edward Su) or &lt;a href=&quot;http://www.yaroslavvb.com/papers/peres-markov.pdf#page=58&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt; for information about &lt;span class=&quot;math-container&quot;&gt;$D_{TV}$&lt;/span&gt; (and other measures/metrics).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-08T14:59:14.223" LastActivityDate="2020-10-08T14:59:14.223" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23953" PostTypeId="5" CreationDate="2020-10-07T22:28:09.090" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T22:28:09.090" LastActivityDate="2020-10-07T22:28:09.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23954" PostTypeId="4" CreationDate="2020-10-07T22:28:09.090" Score="0" Body="For questions about the total variational distance (which can be used as a measure of the distance between probability distributions). See e.g. the paper &quot;On choosing and bounding probability metrics&quot; (2002, by Alison L. Gibbs and Francis Edward Su) for more details about this and other measures." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T22:28:09.090" LastActivityDate="2020-10-07T22:28:09.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23955" PostTypeId="5" CreationDate="2020-10-07T22:40:43.560" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T22:40:43.560" LastActivityDate="2020-10-07T22:40:43.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23956" PostTypeId="4" CreationDate="2020-10-07T22:40:43.560" Score="0" Body="For questions that involve the connect four game, which is a two-player connection board game, in which the players choose a color and then take turns dropping colored discs into a seven-column, six-row vertically suspended grid. The questions need to be related to AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T22:40:43.560" LastActivityDate="2020-10-07T22:40:43.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23957" PostTypeId="5" CreationDate="2020-10-07T22:58:05.710" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T22:58:05.710" LastActivityDate="2020-10-07T22:58:05.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23958" PostTypeId="4" CreationDate="2020-10-07T22:58:05.710" Score="0" Body="For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-07T22:58:05.710" LastActivityDate="2020-10-07T22:58:05.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23959" PostTypeId="2" ParentId="23948" CreationDate="2020-10-07T23:34:59.943" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mu(s)$&lt;/span&gt; is not in equation (9.4) because we are &lt;em&gt;assuming&lt;/em&gt; that the examples by which we update our parameter &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;, i.e. the frequency of which we will observe the states during online training, is the same. That is, it is a constant with respect to &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and since we are differentiating it can be somewhat disregarded as a constant of proportionality -- it essentially can be 'absorbed' by &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The minus is there because we are performing gradient &lt;em&gt;&lt;strong&gt;&lt;em&gt;descent&lt;/em&gt;&lt;/strong&gt;&lt;/em&gt;. For more information on this, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;the wikipedia page&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-10-07T23:34:59.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23960" PostTypeId="2" ParentId="23946" CreationDate="2020-10-07T23:50:43.063" Score="2" Body="&lt;p&gt;We assume that our MDP is ergodic. Loosely speaking, this means that wherever the MDP starts (i.e. no matter which state we start in) or any actions the agent takes early on can only have a limited effect on the MDP and in the limit (as &lt;span class=&quot;math-container&quot;&gt;$t \rightarrow \infty$&lt;/span&gt;) the expectation of being in a given state depends only on the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; and the transition dynamics of the MDP.&lt;/p&gt;&#xA;&lt;p&gt;This means that, eventually, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[R_t] = \mathbb{E}[R_{t+1}]$&lt;/span&gt; for some large &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Therefore, as we take the average of our expected values of the rewards received for an infinitely long period of time, this will have converged due to what I just mentioned of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[R_t] = \mathbb{E}[R_{t+1}]$&lt;/span&gt;. To see why the two are equal, recall that the reward received is dependent on the current state and the action taken -- to better emphasise this I will briefly denote the reward at time step &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$R(S_t, A_t)$&lt;/span&gt;. If we are in the steady state distribution, that is, the state distribution is now fixed, and our actions are still taken according to our policy, then the expected value of &lt;span class=&quot;math-container&quot;&gt;$R(S_t, A_t)$&lt;/span&gt; will be the same for all future &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; since neither the policy nor the state distribution are changing (recall that the average rewards are a way of evaluating a policy in the average-reward setting so for sure this does not change).&lt;/p&gt;&#xA;&lt;p&gt;A way to think of this is that since we know that, eventually, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[R_t]$&lt;/span&gt; will equal &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[R_{t+1}]$&lt;/span&gt;, and so if we keep have an infinite number of these, the average of them will of course converge to the same value. Imagine if I gave you the sequence 1, 2, 3, 4, 4, 4, 4, ........, 4 and asked you to take the average - if we had an infinite amount of 4's then the average would of course be 4.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-10-07T23:57:01.570" LastActivityDate="2020-10-07T23:57:01.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23961" PostTypeId="2" ParentId="13848" CreationDate="2020-10-07T23:52:02.103" Score="0" Body="&lt;p&gt;Our recent work solves this problem by using the idea of a forward-looking actor.&lt;/p&gt;&#xA;&lt;p&gt;We use a neural network to forecast the next state given the current state and current action. Then plug it into the actor training with considering the value of future states. We use our idea on TD3 and make a new algorithm TD3-FORK, which solves this problem with as few as four hours. &lt;a href=&quot;https://github.com/honghaow/FORK/tree/master/BipedalWalkerHardcore&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/honghaow/FORK/tree/master/BipedalWalkerHardcore&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41432" LastActivityDate="2020-10-07T23:52:02.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23962" PostTypeId="1" CreationDate="2020-10-08T00:42:41.423" Score="1" ViewCount="23" Body="&lt;p&gt;Suppose I have two fitted ensemble models &lt;span class=&quot;math-container&quot;&gt;$F_1 := (f_1, f_2, f_3, \cdots f_n)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G_1 := (g_1, g_2, g_3, \cdots g_n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And they were using the same ensemble methods (boosting or bagging).&lt;/p&gt;&#xA;&lt;p&gt;And I am using some measurement for model performance &lt;span class=&quot;math-container&quot;&gt;$M: f_i \to \mathbb{R}^+$&lt;/span&gt;, higher the better.&lt;/p&gt;&#xA;&lt;p&gt;And I know beforehand &lt;span class=&quot;math-container&quot;&gt;$M(f_i) \gt M(g_i), \forall i \in [1,n]$&lt;/span&gt;, can I conclude &lt;span class=&quot;math-container&quot;&gt;$M(F) \gt M(G) $&lt;/span&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="41462" LastEditorUserId="41462" LastEditDate="2020-11-15T00:13:16.663" LastActivityDate="2020-11-15T00:13:16.663" Title="Would performance of atomic models matter in ensemble methods?" Tags="&lt;machine-learning&gt;&lt;performance&gt;&lt;ensemble-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23963" PostTypeId="2" ParentId="23915" CreationDate="2020-10-08T04:14:24.130" Score="1" Body="&lt;p&gt;After my initial comment (where I suggest that it might not be enough info) I believe I actually came up with an idea.&lt;/p&gt;&#xA;&lt;p&gt;Start with the full set of pokemon. For every possible type, identify the count of pokemon that are strong against that type. For this, you'll end up with a &lt;code&gt;List&amp;lt;(pokemonId, types, List&amp;lt;weakAgainst&amp;gt;)&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Minimize &lt;code&gt;List&amp;lt;weakAgainst&amp;gt;.Count()&lt;/code&gt; and from the possible set of pokemonIds, select one at random. Without knowing anything else besides type, this pokemon is as good as any other with the same weakness count (this is the point of my original comment).&lt;/p&gt;&#xA;&lt;p&gt;From the list of weaknesses that this selected pokemon has, select a pokemon from your list that is strong against the weakness, minimizing the amount of weaknesses again. Likely more than one will match this criteria, again, select one at random.&lt;/p&gt;&#xA;&lt;p&gt;Keep repeating this pattern until you obtain the 6 in your team. This is, statistically speaking, one of the best teams that you can gather.&lt;/p&gt;&#xA;&lt;p&gt;For all the combinations that you might find here, some teams will have less weaknesses, since we're &amp;quot;randomly&amp;quot; walking down a tree of possibilities. This very much sounds like a &lt;strong&gt;minimax-prunning algorithm&lt;/strong&gt;, where each pokemon selection (minimizing your weaknesses) can be met with potential opponents that will maximize your weak points.&lt;/p&gt;&#xA;&lt;p&gt;Simplified, put together:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input: allPokemon: list&amp;lt;pokemonId, weakAgainst, strongAgainst&amp;gt;&#xA;&#xA;var: teamWeakAgainst: []&#xA;var: teamStrongAgainst: []&#xA;var: selectedTeam: []&#xA;&#xA;while (size(selectedTeam) &amp;lt; 6)&#xA;  goodMatches &amp;lt;- allPokemon.max(p -&amp;gt; size(p.strongAgainst.intersect(teamWeakAgainst)))&#xA;  goodMatches &amp;lt;- goodMatches.min(p -&amp;gt; size(p.weakAgainst))&#xA;  goodMatches &amp;lt;- goodMatches.max(p -&amp;gt; size(p.strongAgainst))&#xA;&#xA;  selectedPokemon &amp;lt;- goodMatches.random()&#xA;&#xA;  teamWeakAgainst -= selectedPokemon.strongAgainst&#xA;  teamWeakAgainst += selectedPokemon.weakAgainst # not counting previously selected pokemon because the current one adds another &amp;quot;weakness&amp;quot;, even if it was already accounted for&#xA;&#xA;  selectedTeam += selectedPokemon&#xA;&#xA;output: selectedTeam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From this algorithm is it not obvious where the &amp;quot;max&amp;quot; portion is. We're minimizing our losses (weaknesses) but we're considering all possible opponent teams equally, so there is no real maximization of the opponent choices. For a set of ideas, check below.&lt;/p&gt;&#xA;&lt;p&gt;Note that this algorithm will give you a set of &amp;quot;teams&amp;quot; that are equally good in the sense that they'll have the same amount of minimized weaknesses and maximized strengths against other possible teams. But even if pokemon are different, the numbers will be the same, just different types.&lt;/p&gt;&#xA;&lt;p&gt;For a more complex approach, you might want to consider how prevalent some pokemon are (you might not need to optimize against a super rare mythical type, but rather the very common types available in the game), how likely is it that certain pokemon can have better / faster attacks, what is the probability of battle IVs, how frequent can a trainer switch pokemon in battle, etc. Again, I know this is not what you asked for, but for the sake of the example, this will become so complex that instead of a search algorithm, a simulation (Monte Carlo?) approach might be simpler to build teams out of statistical testing.&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2020-10-08T04:14:24.130" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23964" PostTypeId="1" CreationDate="2020-10-08T05:14:53.913" Score="2" ViewCount="375" Body="&lt;p&gt;There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often not available.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://paperswithcode.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papers with Code&lt;/a&gt; (and &lt;a href=&quot;https://github.com/paperswithcode&quot; rel=&quot;nofollow noreferrer&quot;&gt;the associated Github repo&lt;/a&gt;) already lists many research papers and often there is a link to the associated Github repo with the code, but sometimes the code is missing. So, are there alternatives to Papers with Code (for such cases)?&lt;/p&gt;&#xA;" OwnerUserId="15368" LastEditorUserId="2444" LastEditDate="2020-10-09T08:44:17.713" LastActivityDate="2021-05-10T00:20:09.167" Title="What are some alternatives to &quot;Papers with Code&quot;?" Tags="&lt;papers&gt;&lt;resource-request&gt;&lt;implementation&gt;" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23965" PostTypeId="2" ParentId="23950" CreationDate="2020-10-08T06:33:12.763" Score="1" Body="&lt;p&gt;The neural network will learn what we teach it, for example with that image only, after finish training, your model will hard to recognize humans with dark skin, glasses, big eyes, etc, the features that two annotated targets don't have.&lt;/p&gt;&#xA;&lt;p&gt;If your data is big enough, and contain all the feature of humans face, the result should be good.&lt;/p&gt;&#xA;&lt;p&gt;If not, I recommend &lt;a href=&quot;https://arxiv.org/abs/1911.04252&quot; rel=&quot;nofollow noreferrer&quot;&gt;a semi unsupervised learning method&lt;/a&gt; which called Noisy Student. Quick explanation, you take a part of data, add noise (augmentation, drop-out, stochastic depth), and train. Then use that model to label the rest of the dataset and train a new bigger teacher (such as YOLOv3 &amp;gt; YOLOv1) and repeat it. They told that this method was even better than you labeled all the dataset.&lt;/p&gt;&#xA;&lt;p&gt;You will need to choose the data to train the teacher really carefully, then pray for the result.&lt;/p&gt;&#xA;" OwnerUserId="41287" LastActivityDate="2020-10-08T06:33:12.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23966" PostTypeId="1" CreationDate="2020-10-08T07:02:18.940" Score="0" ViewCount="26" Body="&lt;p&gt;I am training a word-level language model using the transformer module available in Pytorch. I am getting a really good training loss and the model is able to reproduce the sentences in the training set properly. When I try to generate new sentences with only the &lt;code&gt;&amp;lt;sos&amp;gt;&lt;/code&gt; token as the input the output the usually a set of pad tokens&lt;/p&gt;&#xA;&lt;p&gt;This is the output when the input is only the &lt;code&gt;&amp;lt;sos&lt;/code&gt; token&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and this is the output of the training data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;sos&amp;gt; mr . gross said he had hoped that a cost savings of &lt;span class=&quot;math-container&quot;&gt;$ n million would result in even greater growth than the n n annual earnings increase at &amp;amp; t has told analysts it expects in the future &amp;lt;eos&amp;gt; &#xA;mr . gross said he had hoped that a cost savings of $&lt;/span&gt; n million would result of even higher growth than $ n n annual earnings increase in &amp;amp; t . told analysts it expects in the future &amp;lt;eos&amp;gt; &#xA;&#xA;&amp;lt;sos&amp;gt; finnish conglomerate &amp;lt;unk&amp;gt; oy ab said it reached an agreement to buy dutch cable company &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; for n million finnish &amp;lt;unk&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$ n million &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;finnish conglomerate corp &amp;lt;unk&amp;gt; ab said it reached an agreement to buy the cable company and &amp;lt;unk&amp;gt; &amp;lt;unk&amp;gt; for n million finnish &amp;lt;unk&amp;gt; $&lt;/span&gt; n million &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&#xA;&amp;lt;sos&amp;gt; the university said the two companies agreed to spend n million canadian dollars &lt;span class=&quot;math-container&quot;&gt;$ n million over n years on research at canadian universities if they are successful in acquiring the vaccine maker &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;the university said the two weeks n to spend n million canadian dollars $&lt;/span&gt; n million over n years on research at canadian universities if they are successful to acquiring &amp;lt;unk&amp;gt; the maker &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&#xA;&amp;lt;sos&amp;gt; per-share net rose to n yen from n yen &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;per-share net rose to n yen from n yen &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&#xA;&amp;lt;sos&amp;gt; in an era of sound &amp;lt;unk&amp;gt; ' and instant opinion polls it is dangerous to apply broad labels to a single term &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;in an era of sound i ' and instant opinion polls it is dangerous to apply broad labels to a single term &amp;lt;eos&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &amp;lt;pad&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first line is the input and the second line has the output. I'm using an attention mask so that the model doesn't look ahead and learn. I am using teacher forcing for training the model, hence you can see that the expected output sentence is the input sentence one shifted to the right. I am also masking the input to while training.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggest why is this happening or at least why would a language model produce such a output. Even in the perspective of RNN would be helpful&lt;/p&gt;&#xA;" OwnerUserId="41279" LastEditorUserId="41279" LastEditDate="2020-10-08T09:37:30.370" LastActivityDate="2020-10-08T09:37:30.370" Title="Transformer Language model produces only &lt;pad&gt; tokens when generating new sentences" Tags="&lt;natural-language-processing&gt;&lt;pytorch&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23967" PostTypeId="2" ParentId="10682" CreationDate="2020-10-08T07:56:28.673" Score="0" Body="&lt;p&gt;They're pretty much the same thing - in that the underlying logic of neural networks is fuzzy. A neural network will take a variety of valued inputs, give them different weight in relation to eachother, and arrive at a decision which normally also has a value. Nowhere in that process is there anything like the sequences of either-or decisions which characterize non-fuzzy mathematics, almost all of computer programming, and digital electronics. Back in the 1980s there was a debate about what AI would eventually look like - some researchers tried to program 'common sense' with huge bivalent decision trees, while others used neural networks which pretty soon found their way into a multitude of electronic devices. Obviously the underlying logic of the latter approach is radically different to the former, even if neural nets are built on top of bivalent electronics. However, the use of the term 'fuzzy logic' seems to have been downplayed since the 80s, perhaps because colloquially it sometimes implies uncertainty. This is shame because it offers a more accurate way to model complex situations.&lt;/p&gt;&#xA;" OwnerUserId="41469" LastEditorUserId="41469" LastEditDate="2020-10-08T08:08:06.970" LastActivityDate="2020-10-08T08:08:06.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23968" PostTypeId="1" AcceptedAnswerId="23988" CreationDate="2020-10-08T08:05:18.027" Score="3" ViewCount="105" Body="&lt;p&gt;From &lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/interpretability-importance.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page in Interpretable-ml book&lt;/a&gt; and &lt;a href=&quot;https://www.analyticsvidhya.com/blog/2019/08/decoding-black-box-step-by-step-guide-interpretable-machine-learning-models-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article on Analytics Vidhya&lt;/a&gt;, it means to know what has happened inside an ML model to arrive at the result/prediction/conclusion.&lt;/p&gt;&#xA;&lt;p&gt;In linear regression, new data will be multiplied with weights and bias will be added to make a prediction.&lt;/p&gt;&#xA;&lt;p&gt;And in boosted tree models, it is possible to &lt;a href=&quot;https://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.plot_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;plot all the decisions&lt;/a&gt; as trees that results in a prediction.&lt;/p&gt;&#xA;&lt;p&gt;And in feed-forward neural networks, we will have weights and biases just like linear regression and we just multiply weights and add bias at each layer, limiting values to some extent using some kind of activation function at every layer, arriving finally at prediction.&lt;/p&gt;&#xA;&lt;p&gt;In CNNs, it is possible to see what happens to the input after having passed through a CNN block and what features are extracted after pooling (ref: &lt;a href=&quot;https://www.kaggle.com/aakashnain/what-does-a-cnn-see?scriptVersionId=11003133&quot; rel=&quot;nofollow noreferrer&quot;&gt;what does a CNN see?&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Like I stated above, one can easily know what happens inside an ML model to make a prediction or conclusion. And I am unclear as to what makes them un-interpretable!. So, what exactly makes an algorithm or it's results un-interpretable or why are these called black box models? Or am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="38060" LastEditorUserId="38060" LastEditDate="2020-10-09T09:25:18.197" LastActivityDate="2020-10-09T10:03:03.957" Title="What exactly is an interpretable machine learning model?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;explainable-ai&gt;&lt;black-box&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23969" PostTypeId="1" CreationDate="2020-10-08T09:53:34.347" Score="0" ViewCount="14" Body="&lt;p&gt;I have a network architecture for feature extraction and I wanted to concatenate layers of the same dimensions but with different feature channels.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;layer1:768*512*16&#xA;layer2:768*512*112&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i would like to concatenate them but I wanted to know if for concatenation the feature channels should be same or if I concatenate there would be feature loss becuase of different number of feature channels?&#xA;Any suggestion on this would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="41471" LastActivityDate="2020-10-08T09:53:34.347" Title="Cocatenate feature extractor layers with different channels" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23970" PostTypeId="2" ParentId="3647" CreationDate="2020-10-08T12:18:30.550" Score="0" Body="&lt;p&gt;Another good (although a bit old) and freely available online book (apart from the one suggested in &lt;a href=&quot;https://ai.stackexchange.com/a/3652/2444&quot;&gt;this answer&lt;/a&gt;) is &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks - A Systematic Introduction&lt;/a&gt; (1996) by &lt;a href=&quot;https://en.wikipedia.org/wiki/Ra%C3%BAl_Rojas&quot; rel=&quot;nofollow noreferrer&quot;&gt;Raul Rojas&lt;/a&gt;. This book contains several exercises at the end of each chapter and covers topics that you will not find in many online courses.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-10-08T12:18:30.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23971" PostTypeId="1" CreationDate="2020-10-08T13:25:58.510" Score="0" ViewCount="38" Body="&lt;p&gt;I'm using Tensorflow to feed a DCGAN 3000 320x320 colored images of cars. The goal is to generate new cars. I've been training on Google Colab for the past 10 hours or so. I guess I can expect results soon but since this is one of my first GAN attempts, I was wondering if there's a way to optimize the layer hyperparameters on both models. Here are both of my model structures:&lt;/p&gt;&#xA;&lt;p&gt;GENERATOR:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential()&#xA;model.add(layers.Dense(20*20*128, use_bias=False, input_shape=(100,)))&#xA;model.add(layers.BatchNormalization())&#xA;model.add(layers.ReLU())&#xA;&#xA;model.add(layers.Reshape((20, 20, 128)))&#xA;assert model.output_shape == (None, 20, 20, 128)&#xA;&#xA;model.add(layers.Conv2DTranspose(128, (6, 6), strides=(1, 1), padding='same', use_bias=False))&#xA;assert model.output_shape == (None, 20, 20, 128)&#xA;model.add(layers.BatchNormalization())&#xA;model.add(layers.ReLU())&#xA;&#xA;model.add(layers.Conv2DTranspose(64, (6, 6), strides=(2, 2), padding='same', use_bias=False))&#xA;assert model.output_shape == (None, 40, 40, 64)&#xA;model.add(layers.BatchNormalization())&#xA;model.add(layers.ReLU())&#xA;&#xA;model.add(layers.Conv2DTranspose(32, (6, 6), strides=(2, 2), padding='same', use_bias=False))&#xA;assert model.output_shape == (None, 80, 80, 32)&#xA;model.add(layers.BatchNormalization())&#xA;model.add(layers.ReLU())&#xA;&#xA;model.add(layers.Conv2DTranspose(16, (6, 6), strides=(2, 2), padding='same', use_bias=False))&#xA;assert model.output_shape == (None, 160, 160, 16)&#xA;model.add(layers.BatchNormalization())&#xA;model.add(layers.ReLU())&#xA;&#xA;model.add(layers.Conv2DTranspose(3, (6, 6), strides=(2, 2), padding='same', use_bias=False, activation='tanh'))&#xA;assert model.output_shape == (None, 320, 320, 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;DISCRIMINATOR:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = tf.keras.Sequential()&#xA;model.add(layers.Conv2D(64, (6, 6), strides=(2, 2), padding='same', input_shape=[320, 320, 3]))&#xA;model.add(layers.LeakyReLU())&#xA;model.add(layers.Dropout(0.3))&#xA;&#xA;model.add(layers.Conv2D(128, (6, 6), strides=(2, 2), padding='same'))&#xA;model.add(layers.LeakyReLU())&#xA;model.add(layers.Dropout(0.3))&#xA;&#xA;model.add(layers.Conv2D(256, (6, 6), strides=(2, 2), padding='same'))&#xA;model.add(layers.LeakyReLU())&#xA;model.add(layers.Dropout(0.3))&#xA;&#xA;model.add(layers.Conv2D(512, (6, 6), strides=(2, 2), padding='same'))&#xA;model.add(layers.LeakyReLU())&#xA;model.add(layers.Dropout(0.3))&#xA;&#xA;model.add(layers.Flatten())&#xA;model.add(layers.Dense(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way to make them better by increasing the number of convolutions, convolution size... considering the fact that I have 3000 320x320 colored images of cars?&lt;/p&gt;&#xA;" OwnerUserId="32539" LastActivityDate="2020-10-08T13:25:58.510" Title="How to optimize my GAN generator and discriminator models' structures?" Tags="&lt;generative-adversarial-networks&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;dc-gan&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23972" PostTypeId="5" CreationDate="2020-10-08T14:16:17.020" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-08T14:16:17.020" LastActivityDate="2020-10-08T14:16:17.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23973" PostTypeId="4" CreationDate="2020-10-08T14:16:17.020" Score="0" Body="For questions about multi-objective reinforcement learning (MORL)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-08T14:16:17.020" LastActivityDate="2020-10-08T14:16:17.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23974" PostTypeId="2" ParentId="23941" CreationDate="2020-10-08T14:50:34.117" Score="1" Body="&lt;p&gt;To add to nbro's answer, I'd say also that much of the time the distance measure isn't simply a design decision, rather it comes up naturally from the model of the problem. For instance, minimizing the KL divergence between your policy and the softmax of the Q values at a given state is equivalent to policy optimization where the optimality at a given state is Bernoulli with respect to the exponential of the reward (see maximum entropy RL algorithms). As another example, the KL divergence in the VAE loss is a result of the model and not just a blind decision.&lt;/p&gt;&#xA;&lt;p&gt;I'm less familiar with total variation distance, but I know there's a nice relationship between the total variation distance of a state probability vector and a Markov chain stationary distribution relative to the timestep and the mixing time of the chain.&lt;/p&gt;&#xA;&lt;p&gt;Finally, another thing to consider is the properties of the gradients of these divergence measures. Note that the gradient of the total variation distance might blow up as the distance tends to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Additionally, one must consider if unbiased estimators of the gradients from samples can be feasible. While this is generally the case with KL divergence, I'm not sure about total variation distance (as in, I literally don't know), and this is generally &lt;em&gt;not&lt;/em&gt; the case with the Wasserstein metric (see Marc G. Bellemare et. al's paper &amp;quot;The Cramér distance as a solution to biased wasserstein gradients&amp;quot;). However, of course there's other scenarios where the tables are turned -- for instance, the distributional bellman operator is a contraction in the supremal Wasserstein metric but &lt;em&gt;not&lt;/em&gt; in KL or total variation distance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL; DR:&lt;/strong&gt; Many times mathematical/statistical constraints suggest particular metrics.&lt;/p&gt;&#xA;" OwnerUserId="37829" LastActivityDate="2020-10-08T14:50:34.117" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23975" PostTypeId="2" ParentId="23864" CreationDate="2020-10-08T15:05:56.290" Score="0" Body="&lt;p&gt;Well probably the response is that previous approach was a little naive.&#xA;I managed to fave some interesting result &lt;a href=&quot;https://www.kaggle.com/felicepollano/starter-watermarked-not-watermarked-e54da0df-f?scriptVersionId=44273914&quot; rel=&quot;nofollow noreferrer&quot;&gt;with this kernel&lt;/a&gt; that allow me to have an &lt;strong&gt;accuracy of 0.969&lt;/strong&gt;  and a &lt;strong&gt;validation accuracy of 0.931&lt;/strong&gt;.&#xA;Model I used is based on &lt;strong&gt;ResNet50&lt;/strong&gt; with the following additional layers ( and the last one just for binary classification ):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tf.keras.layers.Dense(64, activation='relu'),&#xA;tf.keras.layers.Dense(32, activation='relu'),&#xA;tf.keras.layers.Dense(1,activation=&amp;quot;sigmoid&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Even if the net is already trained, I trained each layer again otherwise I did not have any sensible accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Training history is like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iuyIS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iuyIS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Still far to be really good, but progress.&lt;/p&gt;&#xA;" OwnerUserId="41346" LastActivityDate="2020-10-08T15:05:56.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23976" PostTypeId="1" CreationDate="2020-10-08T17:55:06.940" Score="0" ViewCount="16" Body="&lt;p&gt;So I am trying to enforce better separability in my deep learning model and was wondering what I can use besides cross entropy loss to do that? Could maybe using logarithm with different basis in cross entropy (i.e. using lower basis of logarithm than &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; to gain steeper losses on small values, or bigger basis of logarithm to enforce plateaued losses). What would you suggest on doing?&lt;/p&gt;&#xA;" OwnerUserId="41484" LastActivityDate="2020-10-08T17:55:06.940" Title="Loss function for better class separability in multi class classification" Tags="&lt;classification&gt;&lt;loss-functions&gt;&lt;loss&gt;&lt;cross-entropy&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23977" PostTypeId="1" CreationDate="2020-10-08T18:35:29.930" Score="0" ViewCount="36" Body="&lt;p&gt;I'm not an expert in AI or NN, I gathered most of the information I have from the internet, and I'm looking for advice and guidance.&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to design a NN that is going to be used by all the agents of my simulation (each agent will have its own matrix of weights). This is what I plan to have:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The NN will have 1 input layer and 1 output layer (no hidden layers).&lt;/li&gt;&#xA;&lt;li&gt;The number of inputs will always be greater than the number of outputs.&lt;/li&gt;&#xA;&lt;li&gt;The outputs represent the probability of an action being taken by the agent (the output node with the highest value will identify the action that will be taken). Which means there are as many output nodes are there are actions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;When an agent takes an action it receives a reward: a number that represents how well the agent performed. This happens &amp;quot;online&amp;quot; that is, the agent is trained on the fly.&lt;/p&gt;&#xA;&lt;p&gt;What I would like to know if how to best train the NN: that is, how to update the weights of my matrix to maximise the rewards long term.&lt;/p&gt;&#xA;&lt;p&gt;From the research I made it seems this is close to the concept of Reinforcement Learning, but even if it was, it's not clear to me how to apply it to such a simple NN shape.&lt;/p&gt;&#xA;" OwnerUserId="41382" LastActivityDate="2020-10-09T06:22:28.313" Title="How to train the NN of simple agents given a reward system?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;intelligent-agent&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23978" PostTypeId="1" CreationDate="2020-10-08T19:29:24.147" Score="1" ViewCount="105" Body="&lt;p&gt;Is there any reason why skip connections would not provide the same benefits to fully connected layers as it does for convolutional?&lt;/p&gt;&#xA;&lt;p&gt;I've read the ResNet paper and it says that the applications should extend to &amp;quot;non-vision&amp;quot; problems, so I decided to give it a try for a tabular data project I'm working on.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Try 1:&lt;/strong&gt; My first try was to only skip connections when the input to a block matched the output size of the block (the block has &lt;code&gt;depth - 1&lt;/code&gt; number of layers with &lt;code&gt;in_dim&lt;/code&gt; nodes plus a layer with &lt;code&gt;out_dim&lt;/code&gt; nodes :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class ResBlock(nn.Module):&#xA;    def __init__(self, depth, in_dim, out_dim, act='relu', act_first=True):&#xA;        super().__init__()&#xA;        self.residual = nn.Identity()&#xA;        self.block = block(depth, in_dim, out_dim, act)&#xA;        self.ada_pool = nn.AdaptiveAvgPool1d(out_dim)&#xA;        self.activate = get_act(act)&#xA;        self.apply_shortcut = (in_dim == out_dim)&#xA;        &#xA;    def forward(self, x):&#xA;        if self.apply_shortcut:&#xA;            residual = self.residual(x)&#xA;     &#xA;            x = self.block(x)&#xA;            return self.activate(x + residual)&#xA;        return self.activate(self.block(x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The accompanying loss curve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ANdIC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ANdIC.png&quot; alt=&quot;![image|378x278](upload://5LXklb7m5pBA7h3gqiQ026DV7BH.png)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Try 2:&lt;/strong&gt; I thought to myself &lt;strong&gt;&amp;quot;Great, it's doing something!&amp;quot;&lt;/strong&gt;, so then I decided to reset and go for 30 epochs from scratch. I don't have the image saved, but this training only made it 5 epochs and then the training and validation loss curves exploded by several orders of magnitude:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JAleG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JAleG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Try 3:&lt;/strong&gt; Next, I decided to try to implement the paper's idea of reducing the input size to match the output when they don't match: &lt;code&gt;y = F(x, {Wi}) + Mx&lt;/code&gt;. I chose average pooling in place of matrix &lt;code&gt;M&lt;/code&gt; to accomplish this, and my loss curve became.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vZNMM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vZNMM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only difference in my code is that I added average pooling so I could use shortcut connections when input and output sizes are different:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class ResBlock(nn.Module):&#xA;    def __init__(self, depth, in_dim, out_dim, act='relu', act_first=True):&#xA;        super().__init__()&#xA;        self.residual = nn.Identity()&#xA;        self.block = block(depth, in_dim, out_dim, act)&#xA;        # squeeze/pad input  to output size&#xA;        self.ada_pool = nn.AdaptiveAvgPool1d(out_dim) &#xA;        self.activate = get_act(act)&#xA;        self.apply_pool = (in_dim != out_dim)&#xA;        &#xA;    def forward(self, x):&#xA;        # if in and out dims are different apply the padding/squeezing:&#xA;        if self.apply_pool:&#xA;            residual = self.ada_pool(self.residual(x).unsqueeze(0)).squeeze(0)&#xA;        else: residual = self.residual(x)&#xA;            &#xA;        x = self.block(x)&#xA;        return self.activate(x + residual)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a conceptual error in my application of residual learning? A bug in my code? Or is resididual learning just not applicable to this kind of data/network?&lt;/p&gt;&#xA;" OwnerUserId="37691" LastActivityDate="2020-10-08T19:29:24.147" Title="How to use residual learning applied to fully connected networks?" Tags="&lt;feedforward-neural-network&gt;&lt;residual-networks&gt;&lt;binary-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23979" PostTypeId="2" ParentId="23977" CreationDate="2020-10-08T19:30:32.010" Score="1" Body="&lt;p&gt;You might be able to glean what you want from Chapter 13 or Sutton &amp;amp; Barto's &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;, which deals with policy gradient algorithms, and includes pseudocode for a variety of agents based on linear approximation using softmax regression. From your description, you appear to be using - or should consider - softmax regression for the policy function.&lt;/p&gt;&#xA;&lt;p&gt;Probably a good bet using your setup will be &lt;a href=&quot;https://www.datahubbs.com/reinforce-with-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;REINFORCE with baseline&lt;/a&gt;, which does not add requirements to track anything more than your current weights, sum of rewards seen after each state &amp;amp; action pair, and mean rewards (or other baseline). It does require an episodic environment though, such as a game that ends.&lt;/p&gt;&#xA;&lt;p&gt;Essentially REINFORCE and its variants require you to figure out the return (sum of rewards received after each action to the end of the episode) and perform a gradient step update multiplied by that return (minus a baseline which can help with stability).&lt;/p&gt;&#xA;&lt;p&gt;With REINFORCE and a discrete distribution, the gradient due to action selection is the same as supervised learning for multi-classification evaluated as if whichever action the agent took was the correct one. When you multiply this gradient by the return as well, it will make steps in the direction of better action choices larger. Regardless of the fact that you may train towards selecting many different actions, the better actions will win the race in the end due to this weighting, and have the highest probability of being selected after a large number of training episodes.&lt;/p&gt;&#xA;&lt;p&gt;A multi-agent problem may bring you some problems, such as instability due to dependency of performance on what the other agents are doing. It is not possible to say how much of an issue this will be. However, I would advise not jumping straight into your main problem, and try some reinforcement learning methods with a single agent on toy problems first. If you build up to solving the problem you are currently facing, then you will be much more confident in how you have implemented the learning. There are plenty of stepping-stone projects and pre-built environments to solve if you search for them. A good starting point is &lt;a href=&quot;https://gym.openai.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI's Gym&lt;/a&gt;, which presents a standardised set of toy - and not so toy - problems that you can test any learning algorithm against.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-09T06:22:28.313" LastActivityDate="2020-10-09T06:22:28.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23980" PostTypeId="1" AcceptedAnswerId="23981" CreationDate="2020-10-08T20:12:37.457" Score="1" ViewCount="94" Body="&lt;p&gt;I know that my question probably seems like being asked many times, but Ill try&#xA;to be more speciffic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Limitations to my question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I am NOT asking about convolutional neural networks, so please, try not to mention this as an example or as an answer as long as it is possible. (maybe only in question number 3)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My question is NOT about classification using neural networks&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am asking about a &amp;quot;simple&amp;quot; neural network designed to solve the regression type of problem. Let's say it has 2 inputs and 1 output.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Preambula:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As far as I understood, from the &lt;em&gt;universal approximation theorem&lt;/em&gt;, in such a case, even if the model is nonlinear, only one hidden layer can perfectly fit a nonlinear model, as shown here&#xA;&lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://neuralnetworksanddeeplearning.com/chap4.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this specific case, is there any added value in using extra layers?&#xA;(maybe the model will be more precise, or faster training?)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Suppose in 1st question the answer was there is no added value. In such a case will the added value appear if I enlarge inputs from two inputs as described above, to some larger number?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 3&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Suppose in 2nd question the answer was there is no added value. I am still trying to pinpoint the situation where it STARTS making sense in adding more layers AND where it makes NO sense at all using one layer.&lt;/p&gt;&#xA;" OwnerUserId="36453" LastEditorUserId="2444" LastEditDate="2020-10-09T08:36:18.407" LastActivityDate="2020-10-10T20:35:42.657" Title="When are multiple hidden layers necessary?" Tags="&lt;neural-networks&gt;&lt;regression&gt;&lt;hidden-layers&gt;&lt;non-linear-regression&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23981" PostTypeId="2" ParentId="23980" CreationDate="2020-10-08T21:42:09.780" Score="0" Body="&lt;p&gt;A very wide but shallow neural network is going to be harder to train.&lt;br /&gt;&#xA;You can check that with the playground of tensorflow or with the MPG example in Google Colab.&lt;br /&gt;&#xA;The relationship between architecture and learning capabilities is not fully understood, but, empirically, thats what you see.&lt;br /&gt;&#xA;But making the network too deep creates more problems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Vanishing gradients.&lt;/li&gt;&#xA;&lt;li&gt;Less parameters for the same amount of neurons.&lt;/li&gt;&#xA;&lt;li&gt;Exploding gradients.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It is for that reason that humans and neural networks are the one deciding a good architecture.&lt;/p&gt;&#xA;" OwnerUserId="41126" LastEditorUserId="41026" LastEditDate="2020-10-10T20:35:42.657" LastActivityDate="2020-10-10T20:35:42.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23983" PostTypeId="1" CreationDate="2020-10-09T02:23:18.287" Score="3" ViewCount="115" Body="&lt;p&gt;We can visualize single, two, and three dimensions using websites or imagination.&lt;/p&gt;&#xA;&lt;p&gt;In the context of AI and, in particular, machine learning, AI researchers often have to deal with multi-dimensional random vectors.&lt;/p&gt;&#xA;&lt;p&gt;Suppose if we consider a dataset of human faces, each image is a vector in higher dimensional space and needs to understand measures on them.&lt;/p&gt;&#xA;&lt;p&gt;How do they imagine them?&lt;/p&gt;&#xA;&lt;p&gt;I can only imagine with 3D and then approximating with higher dimensions. Is there any way to visualize in higher dimensions for research?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="2444" LastEditDate="2020-10-13T22:40:47.703" LastActivityDate="2020-10-13T22:40:47.703" Title="How do AI researchers imagine higher dimensions?" Tags="&lt;research&gt;&lt;academia&gt;&lt;dimensionality-reduction&gt;&lt;data-visualization&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23984" PostTypeId="2" ParentId="23889" CreationDate="2020-10-09T02:46:33.420" Score="4" Body="&lt;p&gt;We give the target input into the transformer decoder while training the model. So it is easy for the model to &amp;quot;peek ahead&amp;quot; and learn what the next word would be. To ensure that this doesn't happen we apply an additive mask after the dot product between Query and Key. In the original paper &amp;quot;Attention is all you need&amp;quot;, the triangular matrix had 0's in the lower triangle and -10e9 (You can see negative infinity used in recent examples) in the upper triangle. So when the mask is added to the attention score the attention scores in the upper triangle would be really low. When this matrix is passed through the softmax function, these really low values become close to 0, which essentially means not to attend to the words after timestep t. To put in matrix format,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[8.1, 0.04, 5.2, 4.2]&#xA;[0.5, 9.2, 2.33, 0.7]&#xA;[0.2, 0.4, 6.11, 1.0]&#xA;[3.1, 2.1. 2.19, 8.1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Let the above matrix &lt;code&gt;A&lt;/code&gt; the result of the dot product between query and key. The &lt;code&gt;A[0][0]&lt;/code&gt; contains the attention score of the first-word query to the first word of the key, &lt;code&gt;A[0][1]&lt;/code&gt; contains the attention score of the first word of the query to the second of the key, and so on. So as you can see the after adding the mask and performing softmax on &lt;code&gt;A&lt;/code&gt;, the result would be,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[8.1, 0.0, 0.0, 0.0]&#xA;[0.5, 9.2, 0.0, 0.0]&#xA;[0.2, 0.4, 6.11, 0.0]&#xA;[3.1, 2.1. 2.19, 8.1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This forces the transformer only to attend to words that are before it. You can check out the Transformer lecture available in CS224n for full detail.&lt;/p&gt;&#xA;" OwnerUserId="41279" LastEditorUserId="41279" LastEditDate="2020-10-09T07:02:39.670" LastActivityDate="2020-10-09T07:02:39.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23986" PostTypeId="2" ParentId="23964" CreationDate="2020-10-09T03:35:23.597" Score="4" Body="&lt;p&gt;Recently arxiv.org added a Code Tab towards the end of paper descriptions. Which contains links to both the official and community code.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/08v3H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/08v3H.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't know if this is the case for all the papers or not till know. But I'm sure it'll be extended to all the papers in a short while.&lt;/p&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-10-09T03:35:23.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23988" PostTypeId="2" ParentId="23968" CreationDate="2020-10-09T10:03:03.957" Score="1" Body="&lt;p&gt;In a simple linear model of the form &lt;span class=&quot;math-container&quot;&gt;$y = \beta_0 + \beta_1 x $&lt;/span&gt; we can see that increasing &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by a unit will increase the prediction on &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\beta_1$&lt;/span&gt;. Here we can completely determine what the effect on the models prediction will be by increasing &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. With more complex models such as neural networks it is much more difficult to tell due to all the calculations that a single data point is involved in. For instance, in a CNN as you mentioned, if I changed the value of a pixel in an image we were passing through the CNN you wouldn't really be able to tell me exactly the effect this would have on the prediction like you can with the linear model.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-10-09T10:03:03.957" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23989" PostTypeId="2" ParentId="5960" CreationDate="2020-10-09T13:13:56.080" Score="0" Body="&lt;p&gt;I only have one good news... There is nothing wrong with your code. Neural networks tend to do that. Especially with a really complex function.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Increasing the amount of neurons will not decrease how the error is distributed.&lt;/li&gt;&#xA;&lt;li&gt;There are better loss functions for each case but is not a really effective solution.&lt;/li&gt;&#xA;&lt;li&gt;Neural networks are really good managing noise. So, they are good ignoring minorities. It's a common expression &amp;quot;ANN are racist&amp;quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I recommend you to deploy a histogram &lt;em&gt;DataSet&lt;/em&gt; vs &lt;em&gt;Output Value&lt;/em&gt;. To see if you have too much more data from the central region than in the frontier.&#xA;If you can generate more data at will. Generate more values in the specific zones with more errors.&lt;br /&gt;&#xA;This will increase the error and force the backpropagation algorithm to improve in that area.&lt;br /&gt;&#xA;More information on your optimization algorithm may be useful. But, like I said, everything seems perfectly normal.&lt;/p&gt;&#xA;" OwnerUserId="41126" LastEditorUserId="40434" LastEditDate="2020-10-10T16:11:22.370" LastActivityDate="2020-10-10T16:11:22.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23990" PostTypeId="2" ParentId="23983" CreationDate="2020-10-09T14:40:54.390" Score="5" Body="&lt;p&gt;The most I can visualize or perceive are 4 dimensions. Yes, 4, because I can also watch videos (which have 3 spatial dimensions and 1 temporal one). Remember Einstein's &lt;a href=&quot;https://en.wikipedia.org/wiki/Spacetime&quot; rel=&quot;noreferrer&quot;&gt;spacetime&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;When dealing with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional spaces, for &lt;span class=&quot;math-container&quot;&gt;$n &amp;gt; 4$&lt;/span&gt;, I simply do not care about visualizing them in my head, but, as someone suggests, we can think of them as &amp;quot;degrees of freedom&amp;quot;. Maybe something like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tesseract&quot; rel=&quot;noreferrer&quot;&gt;tesseract&lt;/a&gt; may be interesting to you, but that's not really useful to me, to be honest.&lt;/p&gt;&#xA;&lt;p&gt;When dealing with the math that involves &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional spaces or objects, you often do not have to visualize anything, but just have to apply the rules. For example, if you are multiplying multi-dimensional arrays, you just need to make sure that the external dimensions match, and stuff like that.&lt;/p&gt;&#xA;&lt;p&gt;There are cases, when dealing e.g. with TensorFlow's tensors, where you can imagine that there are matrices for each of the elements at that coordinate of the tensor, but that's not very common.&lt;/p&gt;&#xA;&lt;p&gt;In case you really want to visualize &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional objects, you could first project them to &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; dimensions with some dimensionality reduction technique (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding&quot; rel=&quot;noreferrer&quot;&gt;t-SNE&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-09T14:46:55.337" LastActivityDate="2020-10-09T14:46:55.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23993" PostTypeId="1" CreationDate="2020-10-09T21:14:47.943" Score="3" ViewCount="43" Body="&lt;p&gt;In many applications and domains, computer vision, natural language processing, image segmentation, and many other tasks, neural networks (with a certain architecture) are considered to be by far the most powerful machine learning models.&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless, algorithms, based on different approaches, such as ensemble models, like &lt;em&gt;random forests&lt;/em&gt; and &lt;em&gt;gradient boosting&lt;/em&gt;, are not completely abandoned, and actively developed and maintained by some people.&lt;/p&gt;&#xA;&lt;p&gt;Do I correctly understand that the neural networks, despite being very flexible and universal approximators, for a certain kind of tasks, regardless of the choice of the architecture, are not the optimal models?&lt;/p&gt;&#xA;&lt;p&gt;For the tasks in computer vision, the core feature, which makes CNNs superior, is the &lt;em&gt;translational invariance&lt;/em&gt; and the encoded ability to capture the proximity properties of an image or some sequential data. And the more recent &lt;em&gt;transformer&lt;/em&gt; models have the ability to choose which of the neighboring data properties is more important for its output.&lt;/p&gt;&#xA;&lt;p&gt;But let's say I have a dataset, without a certain structure and patterns, some number of numerical columns, a lot of categorical columns, and in the feature space (for classification task) the classes are separated by some nonlinear hypersurface, would the ensemble models be the optimal choice in terms of performance and computational time?&lt;/p&gt;&#xA;&lt;p&gt;In this case, I do not see a way to exploit CNNs or attention-based neural networks. The only thing that comes to my head, in this case, is the ordinary MLP. It seems that, on the one hand, it would take significantly more time to train the weights than the trees from the ensemble. On the other hand, both kinds of models work without putting prior knowledge to data and assumptions on its structure. So, given enough amount of time, it should give a comparable quality.&lt;/p&gt;&#xA;&lt;p&gt;Or can there be some reasoning that neural network is sometimes bound to give rather a poor quality?&lt;/p&gt;&#xA;" OwnerUserId="38846" LastEditorUserId="2444" LastEditDate="2021-05-20T09:44:05.730" LastActivityDate="2021-05-20T09:44:05.730" Title="When do the ensemble methods beat neural networks?" Tags="&lt;convolutional-neural-networks&gt;&lt;transformer&gt;&lt;ensemble-learning&gt;&lt;random-forest&gt;&lt;gradient-boosting&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23995" PostTypeId="1" CreationDate="2020-10-10T01:27:00.920" Score="2" ViewCount="30" Body="&lt;p&gt;I'm working on creating a model that locates the object in the scene (2D image or 3D scene) using a natural language query. I came across this paper on &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Hu_Natural_Language_Object_CVPR_2016_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural language object retrieval&lt;/a&gt;, which mentions that this task is different from text-based image retrieval, in the sense that natural language object retrieval requires an understanding of objects in the image, spatial configurations, etc. I am not able to see the difference between these two approaches. Could you please explain it with an example?&lt;/p&gt;&#xA;" OwnerUserId="41514" LastEditorUserId="2444" LastEditDate="2020-10-12T19:51:34.033" LastActivityDate="2020-10-12T19:51:34.033" Title="What is the difference between text-based image retrieval and natural language object retrieval?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;comparison&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23996" PostTypeId="1" AcceptedAnswerId="23999" CreationDate="2020-10-10T01:52:16.437" Score="4" ViewCount="210" Body="&lt;p&gt;The question is more or less in the title.&lt;/p&gt;&#xA;&lt;p&gt;A Markov decision process consists of a state space, a set of actions, the transition probabilities and the reward function. If I now take an agent's point of view, does this agent &amp;quot;know&amp;quot; the transition probabilities, or is the only thing that he knows the state he ended up in and the reward he received when he took an action?&lt;/p&gt;&#xA;" OwnerUserId="36978" LastEditorUserId="2444" LastEditDate="2020-10-10T09:06:44.667" LastActivityDate="2020-10-10T13:10:55.017" Title="Is the state transition matrix known to the agents in a Markov decision processes?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;model-based-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23998" PostTypeId="1" CreationDate="2020-10-10T06:56:42.177" Score="3" ViewCount="88" Body="&lt;p&gt;Let us assume we have a GRU network containing &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; layers to process a training dataset with &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; tuples, &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; features, and &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt; nodes in each layer.&lt;/p&gt;&#xA;&lt;p&gt;I have a pretty basic idea how the complexity of algorithms are calculated, however, with the presence of multiple factors that affect the performance of a GRU network including the number of layers, the amount of training data (which needs to be large), number of units in each layer, epochs and maybe regularization techniques, training with back-propagation through time,  I am messed up. I have found an intriguing answer for neural networks complexity out here &lt;a href=&quot;https://ai.stackexchange.com/questions/5728/what-is-the-time-complexity-for-training-a-neural-network-using-back-propagation&quot;&gt;What is the time complexity for training a neural network using back-propagation?&lt;/a&gt;, but that was not enough to clear my doubt.&lt;/p&gt;&#xA;&lt;p&gt;So, what is the time complexity of the algorithm, which uses back-propagation through time, to train GRU networks?&lt;/p&gt;&#xA;" OwnerUserId="41519" LastEditorUserId="2444" LastEditDate="2020-10-10T22:14:55.547" LastActivityDate="2020-10-10T22:14:55.547" Title="What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?" Tags="&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;time-complexity&gt;&lt;gated-recurrent-unit&gt;&lt;bptt&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23999" PostTypeId="2" ParentId="23996" CreationDate="2020-10-10T08:35:33.397" Score="4" Body="&lt;p&gt;In reinforcement learning (RL), there are some agents that need to know the state transition probabilities, and other agents that do not need to know. In addition, some agents may need to be able to sample the results of taking an action somehow, but do not strictly need to have access to the probability matrix. This might be the case if the agent is allowed to backtrack for instance, or to query some other systems that simulates the target environment.&lt;/p&gt;&#xA;&lt;p&gt;Any agent that needs to have access to the state transition matrix, or look-ahead samples of the environment is called model-based. The model in this case can either be a distribution model i.e. the state transition matrix, or it can be a sampling model that simulates the outcome from a given state/action combination.&lt;/p&gt;&#xA;&lt;p&gt;The state transition function &lt;span class=&quot;math-container&quot;&gt;$p(r, s'|s, a)$&lt;/span&gt; which returns the probability of observing reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; given the start state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, is another way to express the distribution model. It often maps simply to the state transition matrix, but can be a more complete description of the model.&lt;/p&gt;&#xA;&lt;p&gt;One example model-based approach is &lt;a href=&quot;https://towardsdatascience.com/the-value-iteration-algorithm-4714f113f7c5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Value Iteration&lt;/a&gt;, and that requires access to the full distribution model in order to process value update steps. Also, any reinforcement learning that involves planning must use some kind of model. MCTS, as used in AlphaGo, uses a sampling model for instance.&lt;/p&gt;&#xA;&lt;p&gt;Many RL approaches are model-free. They do not require access to a model. They work by sampling from the environment, and over time learn the impact on expected results due to behaviour of the unknown state transition function. Example methods that do this are Monte Carlo Control, SARSA, Q learning, REINFORCE.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to combine model-free and model-based methods by using observations to build an approximate model of the environment, and using it in some form of planning. &lt;a href=&quot;https://medium.com/@ranko.mosic/online-planning-agent-dyna-q-algorithm-and-dyna-maze-example-sutton-and-barto-2016-7ad84a6dc52b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dyna-Q&lt;/a&gt; is an approach which does this by simply remembering past transitions and re-using them in the background to refine its value estimates. Arguably, the experience replay table in DQN is a similar form of background planning (the algorithm is essentially the same). However, more sophisticated model-learning and reuse is not generally as successful, and is not seen commonly in practice. See &lt;a href=&quot;https://ai.stackexchange.com/questions/10620/how-can-we-estimate-the-transition-model-and-reward-function/23647#23647&quot;&gt;How can we estimate the transition model and reward function?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In general, model-based methods on the same environment can learn faster than model-free methods, since they start with more information that they do not need to learn. However, it is quite common need to learn without having an accurate model available, so there is lots of interest in model-free learning. Sometimes an accurate model is possible in theory, but it would be more work to calculate predictions from the model than to work statistically from the observations.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-10T13:10:55.017" LastActivityDate="2020-10-10T13:10:55.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24001" PostTypeId="1" CreationDate="2020-10-10T12:51:51.753" Score="4" ViewCount="62" Body="&lt;p&gt;I have been working on some RL project, where the policy is controlling the robot using its joint angles.Throughout the project I have noticed some phenomenon, which caught my attention. I have decided to create a very simplified script to investigate the problem. There it goes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The environment&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a robot, with two rotational joints, so 2 degrees of freedom. This means its continuous action space (joint rotation angle) has a dimensionality of 2. Let's denote this action vector by &lt;strong&gt;a&lt;/strong&gt;. I vary the maximum joint rotation angle per step from 11 to 1 degrees and make sure that the environment is allowed to do a reasonable amounts of steps before the episode is forced to terminate on time-out.&lt;/p&gt;&#xA;&lt;p&gt;Our goal is to move the robot by getting its current joint configuration &lt;strong&gt;c&lt;/strong&gt; closer to the goal joint angle configuration &lt;strong&gt;g&lt;/strong&gt; (also two dimensional input vector).&#xA;Hence, the reward I have chosen is e^(-L2_distance(&lt;strong&gt;c&lt;/strong&gt;, &lt;strong&gt;g&lt;/strong&gt;)).&lt;/p&gt;&#xA;&lt;p&gt;The smaller the L2_distance, the exponentially higher the reward, so I am sure that the robot is properly incentivised to reach the goal quickly.&lt;/p&gt;&#xA;&lt;p&gt;Reward function (y-axis: reward, x-axis: L2 distance):&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/R4kNS.png&quot; alt=&quot;Reward function (y-axis: reward, x-axis: L2 distance)&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the pseudocode for every step goes like:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;move the joints by predicted joint angle delta&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;collect the reward&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if time-out or joint deviates too much into some unrealistic configuration: terminate.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Very simple environment, not to have too many moving parts in our problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;RL algorithm&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I use Catalyst framework to train my agent in the actor-critic setting using TD3 algorithm. By using a tested framework, which I am quite familiar with, I am quite sure that there are no implementational bugs.&lt;/p&gt;&#xA;&lt;p&gt;The policy is goal-driven so the actor consumes the concatenated current and goal joint configuration &lt;strong&gt;a&lt;/strong&gt;= policy([&lt;strong&gt;c&lt;/strong&gt;,&lt;strong&gt;g&lt;/strong&gt;])&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The big question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;When the robot has only two degrees of freedom, the training quickly converges and the robots learns to solve the task with high accuracy (final L2 distance smaller than 0.01).&lt;/p&gt;&#xA;&lt;p&gt;Performance of the converged 2D agent. y-axis: joint angle value, x-axis: no of episodes. Crosses denote the desired goal state of the robot.:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2b7Xw.png&quot; alt=&quot;Performance of the converged 2D agent. y-axis: joint angle value, x-axis: no of episodes. Crosses denote the desired goal state of the robot.&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, if the problem gets more complicated - I increase the joint dimensions to 4D or 6D, the robot initially learns to approach the target, but it never &amp;quot;fine-tunes&amp;quot; its movement. Some joints tend to oscillate around the end-point, some of them tend to overshoot.&lt;/p&gt;&#xA;&lt;p&gt;I have been experimenting with different ideas: making the network wider and deeper, changing the action step. I have not tried optimizer scheduling yet. No matter how many samples the agent receives or how long it trains, it never learns to approach targets with required degree of accuracy (L2_distance smaller than 0.05).&lt;/p&gt;&#xA;&lt;p&gt;Performance of the converged 4D agent. y-axis: joint angle value, x-axis: no of episodes. Crosses denote the desired goal state of the robot.:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OnPoU.png&quot; alt=&quot;Performance of the converged 4D agent. y-axis: joint angle value, x-axis: no of episodes. Crosses denote the desired goal state of the robot.&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Training curve for 2D agent (red) and 4D agent (orange). 2D agent quickly minimises the L2 distance to something smaller than 0.05, while the 4D agent struggles to go below 0.1.:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/S9gZV.png&quot; alt=&quot;Training curve for 2D agent (red) and 4D agent (orange). 2D agent quickly minimises the L2 distance to something smaller than 0.05, while the 4D agent struggles to go below 0.1.&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Literature research&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have looked into papers which describe motion planning in joint space using TD3 algorithm.&lt;/p&gt;&#xA;&lt;p&gt;There are not many differences from my approach:&#xA;&lt;a href=&quot;https://arxiv.org/abs/1906.00214&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link 1&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;ved=2ahUKEwiy4KCU3Z_sAhXEnVwKHXzfBZcQFjABegQIBRAC&amp;amp;url=https%3A%2F%2Fwww.mdpi.com%2F2076-3417%2F10%2F2%2F575%2Fpdf&amp;amp;usg=AOvVaw2-D78jGxqQspBGrhbGR0Yk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link 2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Their problem is much more difficult because the policy needs to also learn the model of the obstacles in joint space, not only the notion of the goal. The only thing which is special about them is that they use quite wide and shallow networks. But this is the only peculiar thing.&#xA;I am really interested, what do you guys would advise me to do, so that the robot can reach high accuracy in higher joint configuration dimensions? What am I missing here?!&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any help in that matter!&lt;/p&gt;&#xA;" OwnerUserId="41525" LastEditorUserId="32410" LastEditDate="2020-11-14T02:16:49.887" LastActivityDate="2020-11-14T02:16:49.887" Title="Difficulty in agent's learning with increasing dimensions of continuous actions" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="24005" PostTypeId="1" CreationDate="2020-10-10T15:43:50.183" Score="2" ViewCount="59" Body="&lt;p&gt;In reinforcement learning, the &lt;em&gt;return&lt;/em&gt; is defined as some function of the rewards. For example, you can have the &lt;em&gt;discounted return&lt;/em&gt;, where you multiply the rewards received at later time steps by increasingly smaller numbers, so that the rewards closer to the current time step have a higher weight. You can also have &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-step returns or &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-returns.&lt;/p&gt;&#xA;&lt;p&gt;Recently, I have come across the concept of &lt;strong&gt;return-to-go&lt;/strong&gt; in a few research papers, such as &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prioritized Experience Replay&lt;/a&gt; (appendix &lt;strong&gt;A. Prioritization Variants&lt;/strong&gt;, p. 12) or &lt;a href=&quot;https://arxiv.org/pdf/1911.01546.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Being Optimistic to Be Conservative: Quickly Learning a CVaR Policy&lt;/a&gt; (section &lt;strong&gt;Theoretical Analysis&lt;/strong&gt;, p. 3).&lt;/p&gt;&#xA;&lt;p&gt;What exactly is the &lt;em&gt;return-to-go&lt;/em&gt;? How is it mathematically defined? In which situations do we need to care about it? The name suggests that this is the return starting from a certain time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, but wouldn't this be the same thing as the &lt;em&gt;return&lt;/em&gt; (which is defined starting from a certain time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and often denoted as &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; for that same reason)?&lt;/p&gt;&#xA;&lt;p&gt;There is also the concept of &lt;strong&gt;reward-to-go&lt;/strong&gt;. For example, the reward-to-go is analyzed in the paper &lt;a href=&quot;https://www.jmlr.org/papers/volume17/14-335/14-335.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning the Variance of the Reward-To-Go&lt;/a&gt;, which states that the &lt;em&gt;expected reward-to-go&lt;/em&gt; is the value function, which seems to be consistent with &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html#don-t-let-the-past-distract-you&quot; rel=&quot;nofollow noreferrer&quot;&gt;this explanation of the reward-to-go&lt;/a&gt;, where the reward-to-go is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat{R}_{t} \doteq \sum_{t^{\prime}=t}^{T} R\left(s_{t^{\prime}}, a_{t^{\prime}}, s_{t^{\prime}+1}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We also had a few questions that involve the reward-to-go: for example, &lt;a href=&quot;https://ai.stackexchange.com/q/21784/2444&quot;&gt;this&lt;/a&gt; or &lt;a href=&quot;https://ai.stackexchange.com/q/9614/2444&quot;&gt;this&lt;/a&gt;. How is the return-to-go related to the reward-to-go? Are they the same thing? For example, in &lt;a href=&quot;https://arxiv.org/pdf/2005.02305.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the return-to-go seems to be used as a synonym for reward-to-go (as used in &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html#don-t-let-the-past-distract-you&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;), i.e. they call &lt;span class=&quot;math-container&quot;&gt;$R(t)$&lt;/span&gt; the &amp;quot;return to-go&amp;quot; (e.g. on page 2), which should be the return starting from time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, which should actually be the reward-to-go.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T15:48:54.720" LastActivityDate="2020-10-10T15:48:54.720" Title="What is the return-to-go in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;definitions&gt;&lt;reward-to-go&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24007" PostTypeId="1" AcceptedAnswerId="24009" CreationDate="2020-10-10T16:19:33.347" Score="2" ViewCount="94" Body="&lt;p&gt;I am new to Neural Networks and my questions are still very basic.&#xA;I know that most of neural networks allow and even ask user to chose hyper-parameters like:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;amount of hidden layers&lt;/li&gt;&#xA;&lt;li&gt;amount of neurons in each layer&lt;/li&gt;&#xA;&lt;li&gt;amount of inputs and outputs&lt;/li&gt;&#xA;&lt;li&gt;batches and epochs steps and some stuff related to back-propagation and gradient descent&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But as I keep reading and youtubing, I understand that there are another important &amp;quot;mini-parameters&amp;quot; such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;activation functions type&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;activation functions fine-tuning (for example shift and slope of sigmoid)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tMAQw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tMAQw.png&quot; alt=&quot;types of sigmoid finetuning&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;whether there is an activation funciton in the output&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;range of weights (are they from zero to one or from -1 to 1  or -100 to +100 or any other range)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;are the weights normally distributed or they just random&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;etc...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Actually the question is:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Part a:&lt;/p&gt;&#xA;&lt;p&gt;Do I understand right that most of  neural networks do not allow to change those &amp;quot;mini-parameters&amp;quot;, as long as you are using &amp;quot;readymade&amp;quot; solutions?&#xA;In other words if I want to have an access to those &amp;quot;mini-parameters&amp;quot; I need to program the whole neural network by myself or there are  &amp;quot;semi-finished products&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Part b:&lt;strong&gt;(edited)&lt;/strong&gt;&#xA;For someone who uses neural network as an everyday routine tool to solve problems(Like data scientist), How common and how often do those people deal with fine tuning things which I refer to as &amp;quot;mini-parameters&amp;quot;? Or those parameters are usually adjusted by a neural network developers who create the frameworks like pytorch, tensorflow etc?&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much&lt;/p&gt;&#xA;" OwnerUserId="36453" LastEditorUserId="36453" LastEditDate="2020-10-11T07:57:06.253" LastActivityDate="2020-10-11T07:57:06.253" Title="Which hyperparameters in neural network are accesible to users adjustment" Tags="&lt;activation-function&gt;&lt;hyper-parameters&gt;&lt;weights&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24008" PostTypeId="1" CreationDate="2020-10-10T16:20:25.380" Score="3" ViewCount="43" Body="&lt;p&gt;I am trying to implement Novelty search; I understand why it can work better than the standard Genetic Algorithm based solution which just rewards according to the objective.&#xA;I am working on a problem which requires to generate a fixed number of points in a 2d box centered at the origin.&#xA;In this problem, how can I identify which is a novel configuration of points?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I have thought of one way of doing this: We call the mean of one configuration of points to be the mean of all points in that configuration (let's say this tuple is &lt;span class=&quot;math-container&quot;&gt;$(m_x, m_y)$&lt;/span&gt;, we store the mean of all configurations generated till now, now for a new configuration it's novelty can be defined as the distance of the mean of this new configuration with &lt;span class=&quot;math-container&quot;&gt;$(m_x, m_y)$&lt;/span&gt;.&lt;br&gt;&#xA;But I think it will not work greatly as some very different configuration of points can also have the same mean.&lt;/p&gt;&#xA;" OwnerUserId="41487" LastEditorUserId="41487" LastEditDate="2020-10-11T11:26:59.543" LastActivityDate="2020-10-14T15:17:41.897" Title="Measuring novel configuration of points" Tags="&lt;genetic-algorithms&gt;&lt;optimization&gt;&lt;novelty-search&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24009" PostTypeId="2" ParentId="24007" CreationDate="2020-10-10T17:07:27.553" Score="3" Body="&lt;p&gt;In general, many of the parameters you mentioned are called hyperparameters. All hyperparameters are user-adjusted (or user-programmed) in training phase. Some hyperparameters are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;learning rate,&lt;/li&gt;&#xA;&lt;li&gt;batch size,&lt;/li&gt;&#xA;&lt;li&gt;epochs,&lt;/li&gt;&#xA;&lt;li&gt;optimizer,&lt;/li&gt;&#xA;&lt;li&gt;layers,&lt;/li&gt;&#xA;&lt;li&gt;activation functions etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;To answer your (a) part of your question&lt;/strong&gt;, there are obsiously many frameworks and libraries, for example in python; TensorFlow, pytorch and so on. You might never create a net from the very beginning; maybe only in order to understand the forward and backpropagation algorithms. When we call from scatch networks, we mean that these networks are trained from scratch, with learnable weights and chosen hyperparameters; with no transfer learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;To answer your (b) part of your question&lt;/strong&gt;, I can understand from it that you mean when a net is good enough. Dependently of your data, of course, a neural network is good enough, when it is trained adequately on them. That is, you should be aware of &lt;a href=&quot;https://ai.stackexchange.com/questions/17287/is-running-more-epochs-really-a-direct-cause-of-overfitting/21091#21091&quot;&gt;overfitting&lt;/a&gt;, &lt;a href=&quot;https://medium.com/greyatom/what-is-underfitting-and-overfitting-in-machine-learning-and-how-to-deal-with-it-6803a989c76&quot; rel=&quot;nofollow noreferrer&quot;&gt;underfitting&lt;/a&gt;, and in general of the model you are trying to train with all its parameters and hyperparameters.&lt;/p&gt;&#xA;&lt;p&gt;Since, you are at the very beginning with Machine Learning, I propose you read some books, in order to get everything needed, in terms of Mathematical and Computer Science aspects.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2020-10-10T17:07:27.553" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24011" PostTypeId="5" CreationDate="2020-10-10T17:45:05.227" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T17:45:05.227" LastActivityDate="2020-10-10T17:45:05.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24012" PostTypeId="4" CreationDate="2020-10-10T17:45:05.227" Score="0" Body="For questions about the concept of &quot;reward-to-go&quot;, which comes up e.g. in the context of policy gradients. The &quot;expected reward-to-go for all states&quot; is sometimes used as a synonym for &quot;value function&quot;. See e.g. the paper &quot;Learning the Variance of the Reward-To-Go&quot; (2016) by Aviv Tamar et al. for more details." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T17:45:05.227" LastActivityDate="2020-10-10T17:45:05.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24013" PostTypeId="1" CreationDate="2020-10-10T19:36:17.270" Score="1" ViewCount="135" Body="&lt;p&gt;I am aware that back-propagation through time is used for training the recurrent neural network. But I am not able to understand how this happens for the bi-directional versions of the recurrent neural networks?&lt;/p&gt;&#xA;&lt;p&gt;So, I was hoping if anyone help me with:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Understanding with an example the training of bi-directional recurrent neural networks using back-propagation through time? (I tried following the original paper &lt;a href=&quot;https://ieeexplore.ieee.org/document/650093&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/document/650093&lt;/a&gt;, but it was kind of confusing for me when they perform the backward pass for training)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="41519" LastEditorUserId="36055" LastEditDate="2020-10-27T01:29:29.240" LastActivityDate="2020-10-27T01:29:29.240" Title="How does back-propagation through time work for optimizing the weights of a bidirectional RNN?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24014" PostTypeId="5" CreationDate="2020-10-10T21:56:53.137" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T21:56:53.137" LastActivityDate="2020-10-10T21:56:53.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24015" PostTypeId="4" CreationDate="2020-10-10T21:56:53.137" Score="0" Body="For questions about the &quot;forward pass&quot; algorithm of a neural network, i.e. the algorithm that transforms the input into the output of the neural network." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T21:56:53.137" LastActivityDate="2020-10-10T21:56:53.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24016" PostTypeId="5" CreationDate="2020-10-10T22:09:36.537" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T22:09:36.537" LastActivityDate="2020-10-10T22:09:36.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24017" PostTypeId="4" CreationDate="2020-10-10T22:09:36.537" Score="0" Body="For questions related to sequence-to-sequence (seq2seq) machine learning models/architectures, used e.g. in machine translation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T22:09:36.537" LastActivityDate="2020-10-10T22:09:36.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24018" PostTypeId="5" CreationDate="2020-10-10T22:17:40.967" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T22:17:40.967" LastActivityDate="2020-10-10T22:17:40.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24019" PostTypeId="4" CreationDate="2020-10-10T22:17:40.967" Score="0" Body="For questions about the backpropagation through time (BPTT) algorithm, which is often used to find the gradient of the objective function with respect to the parameters of a recurrent neural network (RNN) when training the RNN with gradient descent." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-10T22:17:40.967" LastActivityDate="2020-10-10T22:17:40.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24020" PostTypeId="1" CreationDate="2020-10-11T00:13:09.570" Score="4" ViewCount="434" Body="&lt;p&gt;I am trying to do the standard MNIST dataset image recognition test with a standard feed forward NN, but my network failed pretty badly. Now I have debugged it quite a lot and found &amp;amp; fixed some errors, but I had a few more ideas. For one, I am using the sigmoid activation function and MSE as an error function, but the internet suggests that I should rather use softmax for the output layer, and cross entropy loss as an error function. Now I get that softmax is a nice activation function for this task, because you can treat the output as a propability vector. But, while being a nice thing to have, that's more of a convinience thing, isn't it? Easier to visualize?&lt;/p&gt;&#xA;&lt;p&gt;But when I looked at what the derivative of softmax &amp;amp; CEL combined is (my plan was to compute that in one step and then treat the activation function of the last layer as linear, as not to apply the softmax derivative again), I found:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{δE}{δi}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; − &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(With &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; being the input of the last layer, &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; the one hot target vector and &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; the prediction vector).&lt;/p&gt;&#xA;&lt;p&gt;That is the same as the MSE derivative. So what benefits does softmax + CEL actually have when propagating, if the gradients produced by them are exactly the same?&lt;/p&gt;&#xA;" OwnerUserId="17769" LastActivityDate="2021-03-11T00:03:15.380" Title="What is the advantage of using cross entropy loss &amp; softmax?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;cross-entropy&gt;&lt;mean-squared-error&gt;&lt;softmax&gt;" AnswerCount="2" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24021" PostTypeId="1" CreationDate="2020-10-11T01:36:53.183" Score="0" ViewCount="38" Body="&lt;p&gt;I use the DCC algorithm to cluster some data.&#xA;The whole algorithm is available here, but shortly it is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;construct mkNN graph of the data points (the connected components of it are the clusters).&lt;/li&gt;&#xA;&lt;li&gt;pretrain an Autoencoder for data dimension-reduction.&lt;/li&gt;&#xA;&lt;li&gt;train the Autoencoder witn it's objective wih another objective of the clustering which tries to reduce the distance between the dimension-reducted data points.&lt;/li&gt;&#xA;&lt;li&gt;the final connected components of the becoming close points(if close enough (by a threshhold) they remain connected, if they moved far from each other - the edge is gone and some others appear and e.t.c) are the clusters.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/shahsohil/DCC&quot; rel=&quot;nofollow noreferrer&quot;&gt;full algorithm with link to the article&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wngz3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wngz3.png&quot; alt=&quot;objective&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dy872.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dy872.png&quot; alt=&quot;algorithm&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Could you possibly guess why am I getting the most of the data in the first 0-index cluster (consistently, with tuning all of the hyperparameters) ?&#xA;The data has good potential for a very good clustering as I can see from the tSNE visualization of the output. But the clustering isn't good. I thought maybe to enlarge the k parameter of the mKNN graph construction, but it didn't work for that, just reduces the whole amount of clusters, but the data samples continue to clash into one cluster.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion and theoretical discussion is welcomed.&lt;/p&gt;&#xA;" OwnerUserId="41535" LastEditorUserId="36055" LastEditDate="2020-10-11T23:25:16.337" LastActivityDate="2020-10-11T23:25:16.337" Title="Deep Continuous Clustering algorithm - just one output cluster" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;autoencoders&gt;&lt;clustering&gt;&lt;dimensionality-reduction&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24023" PostTypeId="2" ParentId="23335" CreationDate="2020-10-11T08:43:21.860" Score="1" Body="&lt;p&gt;You can calculate the memory requirement analytically, but it's still not going to beat physical test in practice as there are so many unknown variables in the system which can takes the GPU memory. Maybe tensorflow will decide to store the gradients, then you have to take into account the memory usage of it also.&lt;/p&gt;&#xA;&lt;p&gt;The way I do it is by setting the GPU memory limit to a high value e.g. 1GB, then test the model inference speed. Then I repeat the process with half the memory. I do it until the model refuses to run or the model speed drops. For example, I start with 1GB, then 512MB, then 256MB, eventually I got to 32 MB and the model speed drops. At 16MB, the model refuses to run. So I know that 64 MB is the minimum requirement I should use for my model. If I want to get a more precise number, I'd repeat the binary search process a couple more time between 64 MB and 32 MB.&lt;/p&gt;&#xA;&lt;p&gt;You can see how to limit the GPU memory here:  &lt;a href=&quot;https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gpus = tf.config.experimental.list_physical_devices('GPU')&#xA;if gpus:&#xA;  # Restrict TensorFlow to only allocate 1GB of memory on the first GPU&#xA;  try:&#xA;    tf.config.experimental.set_virtual_device_configuration(&#xA;        gpus[0],&#xA;        [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=1024)])&#xA;    logical_gpus = tf.config.experimental.list_logical_devices('GPU')&#xA;    print(len(gpus), &amp;quot;Physical GPUs,&amp;quot;, len(logical_gpus), &amp;quot;Logical GPUs&amp;quot;)&#xA;  except RuntimeError as e:&#xA;    # Virtual devices must be set before GPUs have been initialized&#xA;    print(e)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20819" LastEditorUserId="20819" LastEditDate="2020-10-12T03:48:21.020" LastActivityDate="2020-10-12T03:48:21.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24024" PostTypeId="1" CreationDate="2020-10-11T08:47:53.040" Score="1" ViewCount="131" Body="&lt;p&gt;I am pretty new to RL and I am trying to code a simple RL task with pytorch.&lt;/p&gt;&#xA;&lt;p&gt;The goal/task is the following:&#xA;The initial state is &lt;span class=&quot;math-container&quot;&gt;$t_o$&lt;/span&gt; and the agent takes an action &lt;span class=&quot;math-container&quot;&gt;$\Delta_t$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$t_o +\Delta_t = t_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt; equals 450 or 475 then it gets a reward, else he does not get a reward.&lt;/p&gt;&#xA;&lt;p&gt;I am training the agent with DQN algorithm on a NN ( with: 2 Linear layes: fist layer n_in=1 n_out 128 and second layer n_in=128 and n_out=5):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;observation space(&lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt;) is 700 --&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$t_i \in [0,700[$&lt;/span&gt;&lt;br /&gt;&#xA;action space (&lt;span class=&quot;math-container&quot;&gt;$\Delta_t$&lt;/span&gt;) is 5 --&amp;gt; (&lt;span class=&quot;math-container&quot;&gt;$\Delta_t \in [-50,-25,0,25,50]$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;epsilon_start=0.9#e-greedy threshold start value&lt;br /&gt;&#xA;epsilon_end=0.01#e-greedy threshold end value&lt;br /&gt;&#xA;epsilon_decay=200#e-greedy threshold decay learning_rate=0.001# NN&lt;br /&gt;&#xA;optimizer learning rate batch_size=64#Q-learning batch size&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Unfortunately it does not seem to converge to the values &lt;span class=&quot;math-container&quot;&gt;$t_i=$&lt;/span&gt; 450 or 475. I doesn't seem to care about getting a reward.&#xA;How can I improve my code so that the agent learns what I am trying to teach him?&#xA;I put my code below in case the explanations were not clear enough:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from gym import spaces&#xA;&#xA;class RL_env(gym.Env):&#xA;    metadata = {'render.modes': ['human']}&#xA;&#xA;    &#xA;    def __init__(self):&#xA;        super(RL_env, self).__init__()&#xA;        &#xA;        n_actions_delta = 1 #delta_t&#xA;        self.action_space = spaces.Discrete(5)&#xA;        &#xA;        n_observations = 1 #time&#xA;    &#xA;        self.observation_space = spaces.Discrete(700)&#xA;       &#xA;        #initial time&#xA;        self.time = 0&#xA;        &#xA;        self.done = 0&#xA;        self.reward = 0&#xA;&#xA;    def reset(self):&#xA;        self.reward = 0&#xA;        self.done = False&#xA;        return self.reward&#xA;       &#xA;    def step(self,delta_t):&#xA;        print('self time',self.time)&#xA;        d_t = np.arange(-50,70,25)&#xA;        &#xA;        self.time = (self.time + d_t[delta_t])%700&#xA;        print('delta time',d_t[delta_t],'--&amp;gt;','self time',self.time)&#xA;        &#xA;       &#xA;        &#xA;        if self.time == 475 or self.time == 450:&#xA;            self.reward = 1&#xA;            &#xA;            &#xA;        else:&#xA;            self.reward += 0&#xA;        &#xA;            &#xA;        info = {}&#xA;        print('total reward',self.reward)&#xA;        print('\n')&#xA;        return self.time,self.reward, self.done, info&#xA;    &#xA;&#xA;    &#xA;    &#xA;    def render(self, mode='human', close=False):&#xA;        print()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import torch&#xA;import torch.nn as nn&#xA;import torch.optim as optim&#xA;import torch.nn.functional as F&#xA;from torch.autograd import Variable&#xA;from torch.distributions import Categorical&#xA;dtype = torch.float&#xA;device = torch.device(&amp;quot;cpu&amp;quot;)&#xA;import random&#xA;import math&#xA;import sys&#xA;if not sys.warnoptions:#igrnore warnings&#xA;    import warnings&#xA;    warnings.simplefilter(&amp;quot;ignore&amp;quot;)&#xA;&#xA;#hyper parameters&#xA;epsilon_start=0.9&#xA;#e-greedy threshold start value&#xA;epsilon_end=0.01#e-greedy threshold end value&#xA;epsilon_decay=200#e-greedy threshold decay&#xA;learning_rate=0.001# NN optimizer learning rate&#xA;batch_size=64#Q-learning batch size &#xA;&#xA;env = RL_env()&#xA;&#xA;&#xA;#use replay memory (-&amp;gt; to stabilize and improve our algorithm)for training: store transitions observed by agent,&#xA;#then reuse this data later&#xA;#sample from it randomly (batch built by transitions are decorrelated)&#xA;class ReplayMemory:#allowes the agent to learn from earlier memories (speed up learning and break undesirable temporal correlations)&#xA;    def __init__(self, capacity):&#xA;        self.capacity = capacity&#xA;        self.memory = []&#xA;    def push(self, transition):#saves transition&#xA;        self.memory.append(transition)&#xA;        if len(self.memory)&amp;gt;self.capacity:#if length of memory arra is larger than capacity (fixed)&#xA;            del self.memory[0]#remove 0th element&#xA;&#xA;    def sample(self, batch_number):#samples randomly a transition to build batch&#xA;        return random.sample(self.memory, batch_number)&#xA;&#xA;    def __len__(self):&#xA;        return len(self.memory)&#xA;    &#xA;#Dqn NN (we want to maximize the discounted, cumulative reward)&#xA;#idea of Q-learning: we want to approximate with NN maximal Q-function (gives max return of action in given state)&#xA;#training update rule: use the fact that every Q-function for some policy obeys the Bellman equation&#xA;#difference between the two sides of the equality is known as the temporal difference error (we want to min -&amp;gt; Huber loss)&#xA;#calculate over batch of transitions sampled from the replay memory&#xA;class DqnNet(nn.Module):&#xA;    def __init__(self):&#xA;        super(DqnNet, self).__init__()&#xA;        &#xA;        state_space = 1&#xA;        action_space = env.action_space.n&#xA;        num_hid = 128&#xA;        self.fc1 = nn.Linear(state_space, num_hid)&#xA;        self.fc2 = nn.Linear(num_hid, action_space)&#xA;        self.gamma=0.5 #Q-learning discount factor (ensures that reward sum converges, &#xA;                        #makes actions from far future less important)&#xA;    def forward(self, x):&#xA;        x = F.relu(self.fc1(x))&#xA;        x = F.sigmoid(self.fc2(x))&#xA;        return x&#xA;&#xA;#select action accordingly to epsilon greedy policy&#xA;#sometimes we use model for choosing action, other times sample uniformly &#xA;#probability of choosing a random action will start at epsilon_start and will decay (epsilon_decay) exponentially&#xA;#towards epsilon_end&#xA;steps_done=0&#xA;def predict_action(state):&#xA;    global steps_done&#xA;    sample=random.random()#random number&#xA;    eps_threshold=epsilon_end+(epsilon_start-epsilon_end)*math.exp(-1.*steps_done/epsilon_decay)&#xA;    steps_done += 1&#xA;    if sample&amp;gt;eps_threshold:  &#xA;        x  = eps_threshold,model(Variable(state,).type(torch.FloatTensor)).data.max(0)[1].view(1, 1)&#xA;        return x#chose action from model&#xA;    &#xA;    else:&#xA;        x = eps_threshold,torch.tensor([[random.randrange(env.action_space.n)]])&#xA;        return x#choose random action uniformly&#xA;&#xA;#wtih the update_policy function we perform a single step of the optimization&#xA;#first sample a batch, concatenate all the tensors into a single one, compute Q-value and max Q-value, &#xA;#and combine them into loss&#xA;def update_policy():&#xA;    if len(memory)&amp;lt;batch_size:#we want to sample a batch of size 64&#xA;        return&#xA;    transitions = memory.sample(batch_size)#take random transition batch from experience replay memory&#xA;    batch_state, batch_action, batch_next_state, batch_reward = zip(*transitions)#convert batch-array of Transitions&#xA;                                                                              #to Transition of batch-arrays   &#xA;    #--&amp;gt;zip(*) takes iterables as arguments and return iterator&#xA;    &#xA;    batch_state = Variable(torch.cat(batch_state))#concatenate given sequence tensors in the given dimension&#xA;    batch_state = batch_state.resize(batch_size,1)&#xA;    batch_action = Variable(torch.cat(batch_action))&#xA;    batch_next_state = Variable(torch.cat(batch_next_state))&#xA;    batch_next_state = batch_next_state.reshape(batch_size,1)&#xA;    batch_reward = Variable(torch.cat(batch_reward))&#xA;    &#xA;    #print('model batch state',model(Variable(batch_state[0])))&#xA;    current_q_values = model(batch_state).gather(1, batch_action)#current Q-values estimated for all actions,&#xA;                                                                 #compute Q, then select the columns of actions taken,&#xA;                                                                 #these are the actions which would've been taken&#xA;                                                                 #for each batch state according to policy_net&#xA;    max_next_q_values = model(batch_next_state).detach().max(1)[0]#predicted Q-values for non-final-next-states&#xA;                                                                  #(-&amp;gt; gives max Q)&#xA;    expected_q_values = batch_reward + (model.gamma * max_next_q_values)&#xA;&#xA;    #loss is measured from error between current and newly expected Q values (Huber Loss)&#xA;    loss = F.smooth_l1_loss(current_q_values, expected_q_values)&#xA;&#xA;    # backpropagation of loss to NN --&amp;gt; optimize model&#xA;    optimizer.zero_grad()&#xA;    loss.backward()&#xA;    optimizer.step()&#xA;    return loss, np.sum(expected_q_values.numpy())&#xA;    &#xA;&#xA;def train(episodes):&#xA;    scores = []&#xA;    Losses = []&#xA;    Bellman = []&#xA;    Epsilon = []&#xA;    Times = []&#xA;    Deltas = []&#xA;    &#xA;    &#xA;    &#xA;    for episode in range(episodes):  &#xA;        state=env.reset()#reset environment&#xA;        print('\n')&#xA;        print('episode',episode)&#xA;           &#xA;        epsilon_action = predict_action(torch.FloatTensor([state]))&#xA;        &#xA;        action = epsilon_action[1] #after each time step predict action&#xA;&#xA;        next_state, reward, done,info = env.step(action.item())#step through environment using chosen action&#xA;        &#xA;        epsilon = epsilon_action[0]&#xA;        Epsilon.append(epsilon)&#xA;        print(reward,'reward')&#xA;              &#xA;        state=next_state&#xA;        Times.append(state)&#xA;        scores.append(reward)            &#xA;&#xA;    &#xA;        memory.push((torch.FloatTensor([state]),action,torch.FloatTensor([next_state]),&#xA;                         torch.FloatTensor([reward])))#action is already a tensor&#xA;        up = update_policy()#update_policy()#update policy&#xA;            &#xA;        if up != None:&#xA;            Losses.append(Variable(up[0]))&#xA;            print('loss',Variable(up[0]))&#xA;            Bellman.append(up[1])&#xA;&#xA;        #calculate score to determine when the environment has been solved&#xA;        mean_score=np.mean(scores[-50:])#mean of score of last 50 episodes&#xA;        #every 50th episode print score&#xA;        if episode%50 == 0:&#xA;            print('Episode {}\tScore: {}\tAverage score(last 50 episodes): {:.2f}'.format(episode,scores[-50:],mean_score))&#xA;&#xA;    &#xA;    #print('Losses',Losses)&#xA;    Losses = torch.stack(Losses).numpy()&#xA;    #print('Losses',Losses)&#xA;    plt.plot(np.arange(len(Losses)),Losses)&#xA;    plt.xlabel('Training iterations')&#xA;    plt.ylabel('Loss')&#xA;    plt.show()&#xA;    &#xA;    Bellman = np.array(Bellman)&#xA;    #print('Bellman',Bellman,'\n')&#xA;    plt.plot(np.arange(len(Bellman)),Bellman)&#xA;    plt.xlabel('Training iterations')&#xA;    plt.ylabel('Bellman target')&#xA;    plt.show()&#xA;    &#xA;    #print('scores',scores)&#xA;    plt.plot(np.arange(len(scores)),scores)&#xA;    plt.xlabel('Training iterations')&#xA;    plt.ylabel('Reward')&#xA;    plt.show()&#xA;    &#xA;    #print('epsilon',Epsilon)&#xA;    plt.plot(np.arange(len(Epsilon)),Epsilon)&#xA;    plt.xlabel('Training iterations')&#xA;    plt.ylabel('Epsilon')&#xA;    plt.show()&#xA;    &#xA;    print('Times',Times[-25:])&#xA;    print('Deltas',Deltas[-25:])&#xA;    &#xA;    Times = np.array(Times)&#xA;    print('Times',Times)&#xA;    #plt.figure(figsize=(31,20))&#xA;    plt.figure(figsize=(9,7))&#xA;    plt.plot(np.arange(len(Times)),(np.array(Times)))&#xA;    plt.xlabel('Training iterations')&#xA;    plt.ylabel('t')&#xA;    plt.show()&#xA;     &#xA;    Times_1 = np.array(Times[-300:])&#xA;    print('t',Times)&#xA;    plt.figure(figsize=(9,7))&#xA;    plt.plot(np.arange(len(Times_1)),(np.array(Times_1)))&#xA;    plt.xlabel('Last 300 Training iterations')&#xA;    plt.ylabel('t')&#xA;    plt.ylim(0,1000)&#xA;    plt.show()&#xA;    &#xA;model = DqnNet()#policy         &#xA;memory = ReplayMemory(20000)&#xA;optimizer = optim.Adam(model.parameters(), lr=learning_rate)&#xA;&#xA;train(10000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41538" LastEditorUserId="41538" LastEditDate="2020-10-11T15:27:43.930" LastActivityDate="2020-10-11T21:04:59.300" Title="Reinforcement learning simple problem: agent not learning, wrong action" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24026" PostTypeId="2" ParentId="24020" CreationDate="2020-10-11T09:46:36.553" Score="-1" Body="&lt;p&gt;If you look at the definition of the cross-entropy (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), you will see that it is defined for probability distributions (in fact, it comes from information theory). You can also show that the maximization of the (binomial/Bernoulli) log-likelihood is equivalent to the minimization of the cross-entropy, i.e. when you minimize the cross-entropy you actually maximize the log-likelihood of the parameters given your labelled data. Hence the use of the softmax is theoretically founded.&lt;/p&gt;&#xA;&lt;p&gt;Regarding the supposed derivative of the cross-entropy loss function preceded by the softmax, even if that derivative is correct (&lt;a href=&quot;https://www.ics.uci.edu/%7Epjsadows/notes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;I didn't think about it and I don't want to think about it now&lt;/a&gt;), note that then &lt;span class=&quot;math-container&quot;&gt;$t - o$&lt;/span&gt; is different depending on whether &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; is a probability vector or an unnormalized vector (which can take arbitrarily large numbers). If &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; is a probability vector and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; a one-hot encoded vector (i.e. also a probability vector), then all numbers of &lt;span class=&quot;math-container&quot;&gt;$t - o$&lt;/span&gt; will be in the range &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;. However, if &lt;span class=&quot;math-container&quot;&gt;$o_i$&lt;/span&gt; can be arbitrarily large, e.g. &lt;span class=&quot;math-container&quot;&gt;$o_i = 10$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$t_i - o_i \in [-10, -9]$&lt;/span&gt;. So, the propagated error would be different if &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; was not a probability vector.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-11T20:46:23.627" LastActivityDate="2020-10-11T20:46:23.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24028" PostTypeId="1" CreationDate="2020-10-11T10:38:10.587" Score="2" ViewCount="75" Body="&lt;p&gt;For example, if AlphaZero plays with an opponent who has a right to move chess figures any way she wants, or make more than 1 move in a turn? Will a neural network adapt to that, as it adapted to an absurd move made by Lee Sedol in 2015?&lt;/p&gt;&#xA;" OwnerUserId="19666" LastEditorUserId="2444" LastEditDate="2020-10-11T14:41:01.840" LastActivityDate="2020-10-12T08:35:29.567" Title="What happens when an opponent a neural network is playing with does not obey the rules of the game (i.e. cheats)?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;game-ai&gt;&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24029" PostTypeId="2" ParentId="24028" CreationDate="2020-10-11T12:15:25.110" Score="1" Body="&lt;p&gt;The behaviour when playing against &amp;quot;cheats&amp;quot; depends on how the agent has been trained, and how different the game becomes from the training scenarios. It will also depend on how much of the agent's behaviour is driven by training, and how much by just-in-time planning.&lt;/p&gt;&#xA;&lt;p&gt;In general, unless game playing bots are written specifically to detect or cope with opponents that are given unfair advantages, they will continue to play in the same style as if the cheating had not occurred, and assuming that the rules are still being followed strictly. If the cheating player only makes one or two rules-breaking moves, and the resulting game state is still something feasible within the game, then the agent should continue to play well. If the agent significantly outclasses the human opponent, it may still win.&lt;/p&gt;&#xA;&lt;p&gt;A completed, trained agent will not adapt its style to &amp;quot;now my opponent can cheat&amp;quot;. An agent still being trained could do so in theory, but it would take many games with cheating allowed for it to learn tactics that cope with an opponent that had an unfair advantage.&lt;/p&gt;&#xA;&lt;p&gt;Agents that plan by looking ahead during play can cope with more unusual/unseen game state - things that may not have been seen in training. However, they still look ahead on the assumption that game play is as desiged/trained for, they cannot adapt to new rules unless those rules are added to the planning by the bot designers. For instance if the allowed cheating was a limited number of extra moves, but only for the human player, the effects of that could be coded into the planning engine, and the bot would &amp;quot;adapt&amp;quot; with help from its designers.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[AlphaGo] adapted to an absurd move made by Lee Sedol in 2015?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Assuming you are referring to &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol#Game_4&quot; rel=&quot;nofollow noreferrer&quot;&gt;game 4&lt;/a&gt;, then as far as I know, AlphaGo did not &amp;quot;adapt&amp;quot; to this play, after Lee Sedol managed to put it in a losing position then it started playing badly as it could not find a winning strategy from the board positions it was in, and could not recover. I don't think any effort was put into refining AlphaGo during this game or afterwards to patch it for game 5.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-10-11T12:15:25.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24030" PostTypeId="2" ParentId="24013" CreationDate="2020-10-11T15:09:18.003" Score="1" Body="&lt;p&gt;I have not implement the backprop of a bi-directional RNN from scratch so I can't be sure my answer is correct but I hope it helps.&lt;/p&gt;&#xA;&lt;p&gt;You can see how bi-directional RNN works from &lt;a href=&quot;https://coursera.org/share/4de91466d7926b472263bf4157d73e94&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; video from Andrew NG. I got the image below from that video:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GrpCg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GrpCg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For more clarity:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fxCbG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fxCbG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So if you know how to backprop through a simple RNN, you should be able to do so for bi-directional RNN.&lt;/p&gt;&#xA;&lt;p&gt;If you need more detail, let me know.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-10-11T15:09:18.003" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24031" PostTypeId="2" ParentId="24024" CreationDate="2020-10-11T15:16:25.443" Score="1" Body="&lt;p&gt;I see some issues in your code of the environment. Firstly, and probably most importantly, you should not be incrementing the reward. In your code, every time the agent hits &lt;span class=&quot;math-container&quot;&gt;$t=475$&lt;/span&gt; for example, the reward given by the environment increases by 1. So if the agent oscillates between &lt;span class=&quot;math-container&quot;&gt;$t=450$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t=475$&lt;/span&gt;, at each timestep the environment gives a greater and greater reward. Your Q values have no reason to converge. Instead of incrementing the reward, you should just return a reward of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; at those two desired states and return &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; otherwise.&lt;/p&gt;&#xA;&lt;p&gt;Secondly, it appears that you don't reset the state during &lt;code&gt;reset()&lt;/code&gt;, I think you should have &lt;code&gt;self.time = 0&lt;/code&gt; in there (unless this was intentional).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; : I also question why you're using DQN for this problem. This problem should be very easy to solve with standard (tabular) Q Learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2&lt;/strong&gt; : Your training loop looks wrong. You reset the environment at each iteration in your loop, so your episodes are one timestep long each. What I think you want instead is to have another loop under &lt;code&gt;reset()&lt;/code&gt; something like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for episode in range(episodes):&#xA;  s = env.reset()&#xA;  episode_score = 0 &#xA;  for step in range(max_timesteps):&#xA;    a = agent.predict_action(s)&#xA;    ns, r, d, _ = env.step(a)&#xA;    episode_score += r&#xA;   ... &#xA;    agent.update_policy()&#xA;   ... &#xA;  scores.append(episode_score)&#xA; ... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37829" LastEditorUserId="37829" LastEditDate="2020-10-11T21:04:59.300" LastActivityDate="2020-10-11T21:04:59.300" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="24032" PostTypeId="2" ParentId="23877" CreationDate="2020-10-11T16:01:19.853" Score="1" Body="&lt;p&gt;I know this is not a straight answer to your question, but I couldn't comment on your post so decided to post it (so maybe I will delete it after you received a better answer).&lt;/p&gt;&#xA;&lt;p&gt;I think &lt;a href=&quot;https://youtu.be/ks4MPfMq8aQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; playlist by &lt;code&gt;sentdex&lt;/code&gt; can be handy as he goes through a lot of details to teach a neural network model that can drive cars in GTA-V by simply looking at each frame of the game. You can find the code of each step in &lt;a href=&quot;https://github.com/sentdex/pygta5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; link.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-10-11T16:01:19.853" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24033" PostTypeId="2" ParentId="23877" CreationDate="2020-10-11T17:49:17.540" Score="2" Body="&lt;p&gt;In short: yes, you must allow &amp;quot;do nothing&amp;quot; decision as a first level result.&lt;/p&gt;&#xA;&lt;p&gt;Your system must decide the action to be taken, including &amp;quot;do nothing&amp;quot; action. This is different to low network outputs, that can be translated as &amp;quot;don't know what to do&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the network can result in:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;quot;I don't know what to do now&amp;quot; when all results in the output have low probabilities. (Obviously, this is a bad network result, to be fixed as much as possible).&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;I know I must do nothing&amp;quot;, when &amp;quot;do nothing&amp;quot; action has high probability, greater than the others.&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;I know I must do W&amp;quot;, when &amp;quot;W&amp;quot; action has high probability, greater than the others.&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Kind regards.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-14T18:58:18.390" LastActivityDate="2020-10-14T18:58:18.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24034" PostTypeId="2" ParentId="24028" CreationDate="2020-10-11T18:10:25.660" Score="1" Body="&lt;p&gt;&amp;quot;Will a neural network adapt to that ?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&lt;p&gt;The big functional difference between human mind and neural networks : &lt;strong&gt;human mind learns by itself, a NN not&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If we call NN the net with its layers, weights, ... this is a static system, unable to learn anything new. The back-propagation algorithm that made intelligent the NN runs outside the NN itself, in a different stage, different hardware and software, software that is not a NN but classic programming.&lt;/p&gt;&#xA;&lt;p&gt;Thus, a NN never learns nothing while playing, driving, or any other action it is designed for.&lt;/p&gt;&#xA;&lt;p&gt;If, in the learning stage, some cheats are done, the learning algorithm will learn and adapt to these cheats, thus the resulting NN configuration will be able to react to these cheats in the best way. But this is equivalent, in fact, to learn a different game where these cheats are valid movements.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-12T08:35:29.567" LastActivityDate="2020-10-12T08:35:29.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24035" PostTypeId="2" ParentId="17306" CreationDate="2020-10-11T18:34:07.423" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is there an AI technology out there or being developed that can&#xA;predict human behaviour ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If it can predict (all) human behavior, it can act as an human, thus, it will be the first real (strong) AI. This has not happened yet.&lt;/p&gt;&#xA;&lt;p&gt;I must remark that the question contains a lot of weakly defined terms. Fix these terms can help to work in the question subject:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;quot;human behavior&amp;quot; : the behavior of an individual or the behavior of all humans as a set ? (bigger groups tends to be more predictable).&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;irrational decision-makers&amp;quot; : it assumes that it exists a &amp;quot;rational&amp;quot; way of take all decisions.&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;humans are perfectly rational, but obviously this isn't the case&amp;quot; : same as previous.&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;better models and therefore  better models of recessions&amp;quot; : recessions are not caused by &amp;quot;irrational decisions&amp;quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(In fact, each of these points could be an independent question at this site, more according to its manifest that all the plague of questions about neural nets).&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-11T18:51:13.750" LastActivityDate="2020-10-11T18:51:13.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24038" PostTypeId="1" CreationDate="2020-10-11T22:27:58.580" Score="-1" ViewCount="56" Body="&lt;p&gt;My friend is working at a pizza shop. He takes cigarette breaks in an area that is covered by the public webcam of our town.&lt;/p&gt;&#xA;&lt;p&gt;I now want to train a convolutional neural network to be able to detect when he is smoking.&lt;/p&gt;&#xA;&lt;p&gt;Can somebody direct me in the right direction, which tools/tutorials I should look at for this classification task? I already saved 18 hours worth of pictures, one a minute. He is in 28 of these images, I will probably save a few more, maybe 2-3 days. But I don't really know how to start this.&lt;/p&gt;&#xA;" OwnerUserId="41554" LastEditorUserId="11539" LastEditDate="2020-10-15T02:46:29.160" LastActivityDate="2020-10-15T02:46:29.160" Title="How can I train a CNN to detect when a person is smoking outside of shop given images from a video camera?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;object-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24039" PostTypeId="2" ParentId="24020" CreationDate="2020-10-11T22:44:05.630" Score="1" Body="&lt;p&gt;&lt;strong&gt;Short answer: larger gradients&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is not the derivative of the softmax function. &lt;span class=&quot;math-container&quot;&gt;$t - o$&lt;/span&gt; is the combined derivative of the softmax function and cross entropy loss. Cross entropy loss is used to simplify the derivative of the softmax function. In the end, you do end up with a different gradients. It would be like if you ignored the sigmoid derivative when using MSE loss and the outputs are different. Using softmax and cross entropy loss has different uses and benefits compared to using sigmoid and MSE. It will help prevent gradient vanishing because the derivative of the sigmoid function only has a large value in a very small space of it. It is similar to using a different cross entropy loss where the combined derivative of the loss and sigmoid is &lt;span class=&quot;math-container&quot;&gt;$t - o$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Information on derivatives of cross entropy &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;with sigmoid function&lt;/a&gt; and &lt;a href=&quot;https://peterroelants.github.io/posts/cross-entropy-softmax/&quot; rel=&quot;nofollow noreferrer&quot;&gt;with softmax function&lt;/a&gt;. I would also suggest some more research on cross entropy loss functions beyond my links.&lt;/p&gt;&#xA;" OwnerUserId="41026" LastEditorUserId="41026" LastEditDate="2020-10-11T22:57:12.587" LastActivityDate="2020-10-11T22:57:12.587" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24040" PostTypeId="1" CreationDate="2020-10-12T06:38:26.800" Score="1" ViewCount="70" Body="&lt;p&gt;I'm reading the paper &lt;a href=&quot;https://arxiv.org/pdf/1806.07366.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Ordinary Differential Equations&lt;/a&gt; and I have a simple question about adjoint method. When we train NODE, it uses a blackbox ODESolver to compute gradients through model parameters, hidden states, and time. It uses another quantity &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t) = \partial L / \partial \mathbf{z}(t)$&lt;/span&gt; called &lt;em&gt;adjoint&lt;/em&gt;, which also satisfies another ODE. As I understand, the authors build a single ODE that computes all the gradients &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_{0})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \theta$&lt;/span&gt; by solving that single ODE. However, I can't understand how do we know the value &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_1)$&lt;/span&gt; which corresponds to the initial condition for the ODE corresponds to the adjoint. I'm using &lt;a href=&quot;https://msurtsukov.github.io/Neural-ODE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial as a reference, and it defines custom forward and backward methods for solving ODE. However, for the backward computation (especially &lt;code&gt;ODEAdjoint&lt;/code&gt; class in the tutorial) we need to pass &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}$&lt;/span&gt; for backpropagation, and this enables us to compute &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_i)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_{i+1})$&lt;/span&gt;, but we still need to know the adjoint value &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_N)$&lt;/span&gt;. I do not understand well about how pytorch's &lt;code&gt;autograd&lt;/code&gt; package works, and this seems to be a barrier to understand this. Could anyone explain how it operates, and where &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_1)$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$\partial L / \partial \mathbf{z}(t_N)$&lt;/span&gt; if this is more comfortable) comes from? Thanks in advance.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here's my &lt;em&gt;guess&lt;/em&gt; for the initial adjoint from simple example. Let &lt;span class=&quot;math-container&quot;&gt;$d\mathbf{z}/dt = Az$&lt;/span&gt; be a 2-dim linear ODE with given &lt;span class=&quot;math-container&quot;&gt;$A \in \mathbb{R}^{2\times 2}$&lt;/span&gt;. If we use Euler's method as a ODE solver, then the estimate for &lt;span class=&quot;math-container&quot;&gt;$z(t_1)$&lt;/span&gt; is explicitly given as &lt;span class=&quot;math-container&quot;&gt;$$\hat{\mathbf{z}}(t_1) = \mathrm{ODESolve}(\mathbf{z}(t_0), f, t_0, t_1, \theta))= \left(I + \frac{t_1 - t_0}{N}A\right)^{N} \mathbf{z}(t_0) $$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of steps for Euler's method (so that &lt;span class=&quot;math-container&quot;&gt;$h = (t_1 - t_0) /N$&lt;/span&gt; is the step size). If we use MSE loss for training, then the loss will be&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(\mathbf{z}(t_1)) = \Bigl|\Bigl| \mathbf{z}_1 - \left(I + \frac{t_1 - t_0}{N}A\right)^N\mathbf{z}(t_0)\Bigr|\Bigr|_2^2&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}_1$&lt;/span&gt; is the true value at time &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;, which is &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}_1 = e^{A(t_1 - t_0)}\mathbf{z}(t_0)$&lt;/span&gt;. Since adjoint &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t) = \partial L / \partial \mathbf{z}(t)$&lt;/span&gt; satisfies &lt;span class=&quot;math-container&quot;&gt;$$\frac{d\mathbf{a}(t)}{dt} = -\mathbf{a}(t)^{T} \frac{\partial f(\mathbf{z}(t), t, \theta)}{\partial \mathbf{z}} = \mathbf{0},$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t)$&lt;/span&gt; is constant and we get &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t_0) = \mathbf{a}(t_1)$&lt;/span&gt;. So we do not need to use augmented ODE for computing &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t)$&lt;/span&gt;. However, I still don't know what &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t_1) = \partial L / \partial \mathbf{z}(t_1)$&lt;/span&gt; should be.  If my understanding is correct, since &lt;span class=&quot;math-container&quot;&gt;$L = ||\mathbf{z}_1 - \mathbf{z}(t_1)||^{2}_{2}$&lt;/span&gt;, it seems that the answer might be&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L}{\partial \mathbf{z}(t_1)} = 2(\mathbf{z}(t_1) - \mathbf{z}_1).&#xA;$$&lt;/span&gt;&#xA;However, this doesn't seem to be true: if it is, and if we have multiple datapoints at &lt;span class=&quot;math-container&quot;&gt;$t_1, t_2, \dots, t_N$&lt;/span&gt;, then the loss is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L = \frac{1}{N} \sum_{i=1}^{N}||\mathbf{z}_i  -\mathbf{z}(t_i)||_{2}^{2}&#xA;$$&lt;/span&gt;&#xA;and we may have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L}{\partial \mathbf{z}(t_i)} = \frac{2}{N} (\mathbf{z}(t_i) - \mathbf{z}_i),&#xA;$$&lt;/span&gt;&#xA;which means that we don't need to solve ODE associated to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}(t)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="30886" LastEditorUserId="30886" LastEditDate="2020-10-12T07:51:57.520" LastActivityDate="2021-05-19T11:01:00.107" Title="Computation of initial adjoint for NODE" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24042" PostTypeId="2" ParentId="24038" CreationDate="2020-10-12T19:12:57.823" Score="1" Body="&lt;p&gt;Since this a classification problem you will use a CNN preferably. Then you need to fix an architecture of the CNN like VGGNet or Resnet or Le-net. You can find details on architectures here- &lt;a href=&quot;https://towardsdatascience.com/neural-network-architectures-156e5bad51ba&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Network Architecures&lt;/a&gt;. As a beginner you can use VGG 16. You can read about the architecure here- &lt;a href=&quot;https://medium.com/towards-artificial-intelligence/the-architecture-and-implementation-of-vgg-16-b050e5a5920b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Medium.com blog on VGG 16&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;which tools/tutorials i should look at for this classification task?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Tools that you can use:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A python IDE like PyCharm or Jupyter Notebook&lt;/li&gt;&#xA;&lt;li&gt;Keras and Tensorflow packages&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Since, deep learning requires lot of training dataset and demands huge computation power, you can opt for cloud computing platforms like Google Colab or Azure to run your code on, unless you have enough GPU power on your local machine. The above tools are if you want to code yourself. If you want to use GUI(and not write code) Azure Machine Learning Studio is a starting point. Matlab Deep Learning toolbox also provides an excellent GUI with pre trained models on above architectures. However, if you write code on Matlab then you have to ensure your target GPU is NVIDIA as Matlab(Parallel Processing Toolbox, which supports doing computations on GPU) supports CUDA only. It won't work with AMD GPU.&lt;/p&gt;&#xA;&lt;p&gt;If you want to opt to write code, you can find a step by step implementation on dogs vs cats classification here- &lt;a href=&quot;https://towardsdatascience.com/step-by-step-vgg16-implementation-in-keras-for-beginners-a833c686ae6c&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dogs vs Cats classification&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41564" LastActivityDate="2020-10-12T19:12:57.823" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24043" PostTypeId="1" CreationDate="2020-10-12T20:21:13.270" Score="1" ViewCount="51" Body="&lt;p&gt;I'm aware that convergence proofs for Monte Carlo tree search exist in the case of deterministic zero sum games and Markov decision processes.&lt;/p&gt;&#xA;&lt;p&gt;I have come across research which applies MCTS to zero-sum stochastic games, however I was unable to find proof that such an approach is guaranteed to converge to the optimal solution.&lt;/p&gt;&#xA;&lt;p&gt;If anyone is able to provide references or an explanation explaining why or why not MCTS is guaranteed to converge to the optimal solution in this setting I would appreciate it a lot.&lt;/p&gt;&#xA;" OwnerUserId="41567" LastActivityDate="2020-10-12T20:21:13.270" Title="Is Monte Carlo tree search guaranteed to converge to the optimal solution in two player zero-sum stochastic games?" Tags="&lt;monte-carlo-tree-search&gt;&lt;convergence&gt;&lt;game-theory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24045" PostTypeId="1" AcceptedAnswerId="24046" CreationDate="2020-10-12T21:59:30.000" Score="2" ViewCount="347" Body="&lt;p&gt;Consider the following coding line related to CNNS&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Conv2D(64, (3,3), strides=(2, 2), padding='same')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is a convolution layer with filter size &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; and step size of &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am confused about the need for &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt; filters.&lt;/p&gt;&#xA;&lt;p&gt;Are they doing the same task? Obviously, it is no. (one is enough in this case)&lt;/p&gt;&#xA;&lt;p&gt;Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content?&lt;/p&gt;&#xA;&lt;p&gt;I am finding difficulty in visualizing it.&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="2444" LastEditDate="2020-10-13T21:25:24.927" LastActivityDate="2020-10-15T21:18:16.030" Title="What is the need for so many filters in a CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;keras&gt;&lt;filters&gt;&lt;convolutional-layers&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24046" PostTypeId="2" ParentId="24045" CreationDate="2020-10-12T22:40:17.353" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The filters (aka kernels) are the &lt;strong&gt;learnable parameters&lt;/strong&gt; of the CNN, in the same way that the weights of the connections between the neurons (or nodes) are the learnable parameters of a multi-layer perceptron (or feed-forward neural network).&lt;/p&gt;&#xA;&lt;p&gt;So, the value of these filters is not fixed or pre-determined, but will depend on how you train the CNN, i.e. the learning algorithm, the objective function and the data. If you use gradient descent as the learning algorithm, you will be minimizing a loss (aka cost or error) function (e.g. the cross-entropy, in the case of classification problems). To do that, you need to find the gradient of the loss function with respect to the filters. You then apply a step of gradient descent (i.e. you add a scaled version of the gradient of the loss function with respect to the parameters to the parameters), so that this loss decreases.&lt;/p&gt;&#xA;&lt;p&gt;To answer your question more directly, the only thing that usually changes is just the value of the filters. The &lt;a href=&quot;https://ai.stackexchange.com/q/21999/2444&quot;&gt;convolution (or cross-correlation)&lt;/a&gt; operation is the same for all filters.&lt;/p&gt;&#xA;&lt;p&gt;Why do you use more than one filter? The usual explanation is that each filter, when convolved with the input, will extract different features from it, and the specific features that they will extract will depend on the specific values of the filters, which, in turn, depend on the data, so we can say that CNNs are data-driven feature extractors. If you are familiar with image processing techniques, then you know that different filters, when convolved with the same image, can have different effects (e.g. blurring or de-noising).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-15T21:18:16.030" LastActivityDate="2020-10-15T21:18:16.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24047" PostTypeId="2" ParentId="24045" CreationDate="2020-10-12T22:42:44.817" Score="1" Body="&lt;p&gt;All filters move across the same area, but the filter values (also called filter kernels) are different for each filter. This makes it possible to &amp;quot;filter out&amp;quot; different features.&lt;/p&gt;&#xA;" OwnerUserId="34358" LastActivityDate="2020-10-12T22:42:44.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24048" PostTypeId="2" ParentId="22316" CreationDate="2020-10-12T23:01:05.337" Score="0" Body="&lt;p&gt;We don't need multiple environments.&#xA;On-policy algorithms require that new training samples are collected with the newest policy, so we can't use an experience buffer. However we can use the newest policy to collect multiple samples, even over multiple epochs, before updating the weights. This update can be a batch update.&lt;/p&gt;&#xA;" OwnerUserId="34358" LastActivityDate="2020-10-12T23:01:05.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24049" PostTypeId="1" CreationDate="2020-10-13T01:30:13.313" Score="2" ViewCount="34" Body="&lt;p&gt;I am currently implementing the Moth-Flame Optimization (MFO) Algorithm, based on the paper: &lt;a href=&quot;https://fardapaper.ir/mohavaha/uploads/2020/03/Fardapaper-Moth-flame-optimization-algorithm-A-novel-nature-inspired-heuristic-paradigm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moth-Flame Optimization Algorithm: A Novel Nature-inspired Heuristic Paradigm&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To calculate the values of the Moths, it uses two arrays of values, which contain upper and lower values for each variable. However, as far as I can see. it mentions nothing about what these values are. Quoting from the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As can be seen, there are two other arrays called &lt;span class=&quot;math-container&quot;&gt;$ub$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$lb$&lt;/span&gt;. These matrixes define the upper and lower bounds of the variables as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ub = [ub_1, ub_2, ub_3, \dots,ub_{n-1}, ub_n]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$ub_i$&lt;/span&gt; indicates the upper bound of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th variable.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$lb = [lb_1, lb_2, lb_3, \dots, ub_{n-1}, ub_n]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$lb_i$&lt;/span&gt; indicates the lower bound of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th variable&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;After that, it says nothing more about this matter&lt;/p&gt;&#xA;&lt;p&gt;So, if anyone has any idea of how these bound values are determined, please tell me!&lt;/p&gt;&#xA;" OwnerUserId="41573" LastEditorUserId="2444" LastEditDate="2020-10-13T21:02:13.727" LastActivityDate="2020-10-13T21:02:13.727" Title="How are the lower and upper bound values of the moths determined in the Moth-Flame Optimization algorithm?" Tags="&lt;optimization&gt;&lt;papers&gt;&lt;meta-heuristics&gt;&lt;moth-flame-optimization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24050" PostTypeId="1" CreationDate="2020-10-13T09:47:07.680" Score="1" ViewCount="31" Body="&lt;p&gt;So I'm stack to something that it's probably very easy but I can't get my head around it. I'm building a Neural Network that will consist of many layers with non-linear activation functions (probably ReLUs) and the last output layer will be linear because we are trying to catch a specific number and not a probability. I've done the forward propagation calculations but I'm stuck at the back propagation ones.&lt;/p&gt;&#xA;&lt;p&gt;Let's say that I'm gonna use the cross entropy loss function: (we will implement the MSE as well)   &lt;span class=&quot;math-container&quot;&gt;$-(y \log (a)+(1-y) \log (1-a))$&lt;/span&gt;. (I understand that this is not a good option for a regression problem)&lt;/p&gt;&#xA;&lt;p&gt;So we can easily find the dJ/dA &lt;span class=&quot;math-container&quot;&gt;$d A=\frac{\partial J}{\partial A}=-\left(\frac{y}{A}-\frac{1-y}{1-A}\right)$&lt;/span&gt; of the last layer and we can start going backwards finding the &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial J}{\partial Z^{[L]}}$&lt;/span&gt; which we can calculate from the equation: &lt;span class=&quot;math-container&quot;&gt;$d Z^{[L]}=d A*g^{\prime}\left(Z^{[L]}\right)$&lt;/span&gt; The problem lies at the second part of this equation where the derivative of g is.&lt;/p&gt;&#xA;&lt;p&gt;What will be the outcome since we have a linear activation function which derivative is equal with 1? (Activation function: f(x) = x, f'(x) = 1)&lt;/p&gt;&#xA;&lt;p&gt;Will it be an identity matrix with the shape of Z[L] or a matrix full of ones with the same shape again? I'm asking about the term &lt;span class=&quot;math-container&quot;&gt;$g^{\prime}\left(Z^{[L]}\right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="41488" LastEditorUserId="41488" LastEditDate="2020-10-13T15:14:17.433" LastActivityDate="2020-10-13T15:14:17.433" Title="Linear output layer back propagation" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;linear-regression&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24051" PostTypeId="1" CreationDate="2020-10-13T12:39:33.783" Score="1" ViewCount="27" Body="&lt;p&gt;I want to build HELM neural network that consists of autoencoder (AE) and one class classification (OC).&lt;/p&gt;&#xA;&lt;p&gt;HELM with AE and OC have following shape:&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PB7fL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PB7fL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is, hidden layer output of AE is input of OC.&lt;/p&gt;&#xA;&lt;p&gt;Training of HELM consists of training AE and OC separately. In order to train each neural network in HELM, first there are generated random weights and biases between input and hidden layers, and then based of them and activation function (for example sigmoid), only weights between hidden and output layers are trained. But then what's the point in training weights between hidden and output layers in AE, since only output of its hidden layer is provided as input into one-class classifier? What is point in use AE in HELM if weights between input and hidden layers of AE are basicaly random?&lt;/p&gt;&#xA;&lt;p&gt;Following paper (page 6):&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1810.05550.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1810.05550.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;confirms that output of hidden layer of AE is input for OC, but also on the contrary in Algorithms 2 and 3 (pages 6, 7), there is shown that input of OC is AE input vector multiplied by matrix of weights between hidden and output layer, what sounds weird for me.&lt;/p&gt;&#xA;" OwnerUserId="41511" LastEditorUserId="41511" LastEditDate="2020-10-13T12:48:53.263" LastActivityDate="2020-10-13T12:48:53.263" Title="Role of autoencoder in Hierarchical Extreme Learning Machine" Tags="&lt;classification&gt;&lt;training&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24052" PostTypeId="1" CreationDate="2020-10-13T14:04:37.287" Score="1" ViewCount="13" Body="&lt;p&gt;I am reading RBMs from &lt;a href=&quot;https://christian-igel.github.io/paper/TRBMAI.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. In Fig1 they show an example of generating hand-written digit using RBMs. This is the figure they are showing:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uPxrk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uPxrk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the learning step first we sample &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$h \sim P(h|v)$&lt;/span&gt; and then reconstructe &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$v \sim P(v|h)$&lt;/span&gt;. Now in the generating step they are sampling &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$v \sim P(v|h)$&lt;/span&gt;. My question is, in generating step if we do not sample &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$h \sim P(h|v)$&lt;/span&gt; how can we get &lt;span class=&quot;math-container&quot;&gt;$P(v|h)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastActivityDate="2020-10-13T14:04:37.287" Title="How Restricted Boltzman Machine (RBM) generates hand-written digit?" Tags="&lt;deep-learning&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24054" PostTypeId="1" CreationDate="2020-10-13T14:27:32.777" Score="0" ViewCount="61" Body="&lt;p&gt;I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't know how tall are other objects. Is it possible to write a code or do some geometry to know other objects distances from camera using only the height of one object? For example I have an image where 5 meters away there is a box which is 1 meter high, I wanna know the distance to human who is 12 meters away, or to know a distance to a dog who is 7 meters away. Maybe you guys know any datasets or models which deal with the same problem as I am facing. Any help will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="41591" LastActivityDate="2021-03-13T23:05:05.403" Title="Is it possible to know the distance objects are from camera based on only knowing one object's height?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;datasets&gt;&lt;math&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24055" PostTypeId="1" CreationDate="2020-10-13T14:44:10.947" Score="1" ViewCount="52" Body="&lt;p&gt;Let's say I'm looking for any item that has a certain shape (outline) in a photo. but I can further classify it only according to particular features, that most of them are expected to be shown only in a smaller area of the the object itself.&lt;/p&gt;&#xA;&lt;p&gt;How may I give more weight, in the model, to that particular area, in order to avoid wrong classification issues?&lt;/p&gt;&#xA;&lt;p&gt;What is the flow, and are there specific tools that should be used for that purpose?&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;p&gt;I want to detect all triangles in the image, and try to classify them like this:&#xA;If triangle has 3 lines in its corner, it's A type. if only two lines, it's B type.&lt;/p&gt;&#xA;&lt;p&gt;So the triangles outline composes 100% of the object, but we can see that the area where the red lines are present, is only about 10% of the object area. How may I give more weight and tell the model to carefully look for the details in that area, so it doesn't confuse A with B or vice versa, just because the other 90% of the shape is similar.&lt;/p&gt;&#xA;&lt;p&gt;And Of course, I want the certainty level to be as close as possible to 100%, for both A and B, and to be distinguished from the other option.&lt;/p&gt;&#xA;&lt;p&gt;So my goal is the get this output:&lt;/p&gt;&#xA;&lt;p&gt;Purple Triangle ==&amp;gt; Type A, certainty, 99%. Type B: certainty: 50%&lt;/p&gt;&#xA;&lt;p&gt;Green Triangle ==&amp;gt; Type B, certainty, 99%. Type A: certainty: 50%&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cSDot.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cSDot.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41590" LastActivityDate="2020-11-30T20:03:40.373" Title="Computer vision - Can you put more weight on a specific part of the object?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24058" PostTypeId="1" CreationDate="2020-10-13T17:01:51.900" Score="0" ViewCount="33" Body="&lt;p&gt;I'm training a model with the transformer encoder architecture on a given fixed set of data.&lt;/p&gt;&#xA;&lt;p&gt;The task I'm solving has a trivial approximation which consists in copying part of the input to the output, which achieves a loss of about &lt;code&gt;140&lt;/code&gt; (after un-normalizing) on my validation set.&lt;/p&gt;&#xA;&lt;p&gt;I've managed, with a given model size and hyperparameters (learning rate &lt;code&gt;1e-3&lt;/code&gt; and batch size &lt;code&gt;1000&lt;/code&gt;), to achieve a better loss of about &lt;code&gt;98&lt;/code&gt;, but it took over 12 hours.&lt;/p&gt;&#xA;&lt;p&gt;I then tried to accelerate learning by different techniques: increased learning rate would quickly lead to divergent models, reducing batch size as well, and a combination of reducing batch size and learning rate too. This was the only settings at which I was able to get these results. All of these combinations would get stuck at about &lt;code&gt;240&lt;/code&gt; in loss, for some reason.&lt;/p&gt;&#xA;&lt;p&gt;I then tried reducing the model size, but this actually made my learning &lt;strong&gt;better&lt;/strong&gt; with the loss getting stuck at various values depending on these parameters, between &lt;code&gt;120&lt;/code&gt; and &lt;code&gt;180&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I therefore have two questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why is my loss getting &amp;quot;stuck&amp;quot; at these values? For a given model size, I've tried every combination of increasing/decreasing batch size and learning rates, some learning rate scheduling, but there simply seems to be a barrier which cannot be surpassed.&lt;/li&gt;&#xA;&lt;li&gt;Why do bigger models have higher barriers? I have for example tried a full-sized transformer encoder (depth of &lt;code&gt;6&lt;/code&gt;, dimension of &lt;code&gt;1024&lt;/code&gt; and feed-forward dimension of &lt;code&gt;2048&lt;/code&gt;) but the loss gets stuck at &lt;code&gt;360&lt;/code&gt;, much more than the tiny models (depth of &lt;code&gt;2&lt;/code&gt;, dimension of &lt;code&gt;64&lt;/code&gt; and feed-forward dimension of &lt;code&gt;128&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have tried looking at gradient norm in the &amp;quot;full-sized&amp;quot; model from this last point, and it doesn't vanish to zero but stabilizes around &lt;code&gt;1&lt;/code&gt; to &lt;code&gt;5&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41596" LastActivityDate="2020-10-13T17:01:51.900" Title="Bigger models get higher losses" Tags="&lt;machine-learning&gt;&lt;gradient-descent&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24060" PostTypeId="5" CreationDate="2020-10-13T21:03:11.917" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-13T21:03:11.917" LastActivityDate="2020-10-13T21:03:11.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24061" PostTypeId="4" CreationDate="2020-10-13T21:03:11.917" Score="0" Body="For questions about the Moth-Flame Optimization algorithm introduced in the paper &quot;Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm&quot; (2015) by Seyedali Mirjalili et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-13T21:03:11.917" LastActivityDate="2020-10-13T21:03:11.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24062" PostTypeId="2" ParentId="2279" CreationDate="2020-10-14T00:29:27.243" Score="0" Body="&lt;p&gt;One of the suggestions in the accepted answer was SSD.&#xA;On their website, SSD mentioned a competitor, faster_rcnn.&#xA;faster_rcnn was deprecated in favor of Detectron.&#xA;Detectron was deprecated in favor of Detectron2.&#xA;Long live detectron2.&lt;/p&gt;&#xA;&lt;p&gt;It looks pretty cool and powerful:&#xA;&lt;a href=&quot;https://github.com/facebookresearch/detectron2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/facebookresearch/detectron2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32709" LastActivityDate="2020-10-14T00:29:27.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24063" PostTypeId="1" AcceptedAnswerId="24066" CreationDate="2020-10-14T00:33:23.920" Score="2" ViewCount="44" Body="&lt;p&gt;I am interested in models that exhibit behavior. My goal is a model that survives indefinitely on a two dimensional resource landscape. One dimension represents the location (0 to 1) and the second says if there is a resource available at that location (-1 = resource one, 0 = no resource, 1 = resource two).&lt;/p&gt;&#xA;&lt;p&gt;The landcape looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;location = [0, 0.2, 0.4, 0.6, 0.8, 1]&#xA;resource = [-1, 0,   0,   0,   0,  1] (I added spaces so the elements line up)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My model represents an organism deciding if it will move or rest on the landscape at each time step. The organism has reserves of each resource. The organism fills its reserve of a resource if it rests on the resource and loses 1 unit of both resources at each time step. I am considering neural networks to represent my organisms. The input would be 4 values; The location on the landscape, the resource value at that location, and the reserve levels of resource one and two. The output would be 3 values; move right, rest, move left. The highest value decides what happens. To survive indefinitely the model will have to bounce between the ends of the landscape, briefly resting on the resource. Model evaluation would go like this: start the model in the middle of the landscape with full resource reserves. Allow time to pass until one of the resource reserves is depleted (the organism dies).&lt;/p&gt;&#xA;&lt;p&gt;My question is this: Can my loss function be evaluating the model until it dies? 1/survival time could be the loss value to be minimized by gradient descent. Is this a reinforcement learning problem (I don't think so..?) Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="31703" LastEditorUserId="2444" LastEditDate="2020-10-14T10:18:23.463" LastActivityDate="2020-10-14T13:26:09.067" Title="Evaluate model multiple times in loss function? Is this reinforcement learning?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24065" PostTypeId="1" AcceptedAnswerId="24070" CreationDate="2020-10-14T07:28:30.523" Score="1" ViewCount="48" Body="&lt;p&gt;As a &lt;strong&gt;routine&lt;/strong&gt; (in typical everyday  tasks) of a data scientist, should they usually decide about weights and biases &lt;strong&gt;range&lt;/strong&gt; and &lt;strong&gt;initial values&lt;/strong&gt; as a function of which data they are planning to insert as an input, and which type of data they expect to get in the output? Or we usually do not deal with such fine-tuning, and let the algorithm to do it?&#xA;One could answer that normalizing inputs solves the problem and no need to fit weights and biases, but I guess they depend also on expected output.&lt;/p&gt;&#xA;&lt;p&gt;To summarize:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;is it common to deal with weights and biases in everyday tasks or in most of the cases existing algorithms do it well?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;what are the rules of thumb for how to decide about range and initial values of weights and biases?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36453" LastEditorUserId="11539" LastEditDate="2020-10-14T18:25:48.763" LastActivityDate="2020-10-14T18:25:48.763" Title="Should the range and initial values of weights and biases be adjusted to fit input and output data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;hyper-parameters&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24066" PostTypeId="2" ParentId="24063" CreationDate="2020-10-14T10:04:54.607" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Can my loss function be evaluating the model until it dies? 1/survival time could be the loss value to be minimized by gradient descent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In order to use backpropagation and gradient descent, you have to relate the loss function directly to the output of the neural network. Your proposed loss function is too indirect, it is not possible to turn it &lt;em&gt;directly&lt;/em&gt; into a gradient that could be used to alter the neural network weights.&lt;/p&gt;&#xA;&lt;p&gt;In addition, the specific function of time you have chosen will be difficult to optimise, as incrememental improvements from e.g. 10 to 11 time steps surviving will provide a much lower signal for adjusting behaviour than the improvement from  e.g. 2 to 3 lifetime. If the environment has enough randomness (and typically these kind of a-life scenarios do), then the signal here could be swamped by random events and very hard to optimise, requiring a larger number of samples in order to extract expected improvements to the loss function.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is this a reinforcement learning problem (I don't think so..?)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is very close to a definition of reinforcement learning (RL) problem. For a RL problem you need the following things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;An environment in which an agent exists, and which has a measurable state.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A set of actions = sequential decisions that need to be made, and that have consequences.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The consequences of any action are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A consistent (but allowing for some randomness) change to the state of the environment. Here you have change to agent's location and resource levels.&lt;/li&gt;&#xA;&lt;li&gt;A consistent scalar reward value that measures how well the agent is achieving a goal. This can also be partially random, and can also be sparse, only achieved in certain specific states.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In your problem definition you don't have a reward signal, but it would be easy to add one. A suitable one would be &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; per time step.&lt;/p&gt;&#xA;&lt;p&gt;Technically your problem would also be partially observable (sometimes called a POMDP), in that the agent does not get to see resources available in other locations. It only knows its current location, its internal state and resources available at its current location. This is not a major issue, although you should note that adding some kind of memory (either open-ended memory as in a recurrent neural network, or explicitly added to the state) would allow for more efficient agents. That's not a RL issue as such, any learning process without ability to form or use memories would be limited in this environment, and you might want to look into that as a later experiment.&lt;/p&gt;&#xA;&lt;p&gt;How RL helps is that it provides a framework to convert your problem definition into measurements and &lt;em&gt;gradients&lt;/em&gt; for the neural network to improve its performance.&lt;/p&gt;&#xA;&lt;p&gt;As you have set up your neural network to predict best action choice, this would naturally lend itself to policy gradient methods in RL, such as REINFORCE. As it is a simple problem, I would expect REINFORCE with baseline to perform well enough for it.&lt;/p&gt;&#xA;&lt;p&gt;I will not explain the algorithm here in full detail, but the basic approach is to have the agent act with the current network for a few episodes, collecting data on its choices and performance. You will get a dataset of (state, action, return = sum of rewards to end of episode). You then use that as a labelled dataset to train a minibatch as if the action choice was correct ground truth, but multiply each gradient by (return - baseline) where the baseline is typically the average return seen so far from that state. You may need a second neural network during training to estimate that expected return (aka state value). After using a minibatch once, you will need to discard it, as it represents results for the previous iteration of the network before the weight updates. There are ways around this, but not typically done in REINFORCE - instead the approach is to just keep generating data and train on the new mini-batch as fast as you can go.&lt;/p&gt;&#xA;&lt;p&gt;RL offers other methods which may work just as well to solve your problem, but I suspect REINFORCE will serve you well since it will allow the agent to randomise direction choice which is important when it cannot see where the resources are and has no memory of where it has searched.&lt;/p&gt;&#xA;&lt;p&gt;You don't have to use RL for this problem. An alternative that may work for you is using genetic algorithms to tune the network architecture and weights. It avoids using gradients, but I would still recommend a simple fitness function equal to number of time steps survived. There is a framework called NEAT which is ideal for this sort of a-life control problem.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-14T13:26:09.067" LastActivityDate="2020-10-14T13:26:09.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24067" PostTypeId="1" CreationDate="2020-10-14T10:32:06.000" Score="3" ViewCount="254" Body="&lt;p&gt;I am using Q-learning and SARSA to solve a problem. The agent learns to go from the start to the goal without falling in the holes.&lt;/p&gt;&#xA;&lt;p&gt;At each state, I can choose the action corresponding to the maximum Q value at the state (the greedy action that the agent would take). And all the actions connect some states together. I think that would show me a road from start to goal, which means the result converges.&lt;/p&gt;&#xA;&lt;p&gt;But some others think that as long as the agent learns how to reach the goal, the result converges. Sometimes the success rate is very high but we cannot get the road from Q table. I don't know which one means the agent is trained totally and what the converged result means.&lt;/p&gt;&#xA;" OwnerUserId="41613" LastEditorUserId="2444" LastEditDate="2020-10-14T15:25:54.123" LastActivityDate="2020-10-14T15:43:52.777" Title="How to determine if Q-learning has converged in practice？" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;convergence&gt;&lt;temporal-difference-methods&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24068" PostTypeId="1" AcceptedAnswerId="24115" CreationDate="2020-10-14T10:58:00.953" Score="2" ViewCount="208" Body="&lt;p&gt;I was reading the &lt;em&gt;&amp;quot;Deep Learning with Python&amp;quot;&lt;/em&gt; by François Chollet. He mentioned separable convolution as following&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This is equivalent to separating the learning of spatial features and&#xA;the learning of channel-wise features, which makes a lot of sense if&#xA;you assume that spatial locations in the input are highly correlated,&#xA;but different channels are fairly independent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But I could not understand what he meant by saying &amp;quot;correlated spatial locations&amp;quot;. Can some explain what he means or the purpose of separable convolutions? (except performance-related part).&lt;/p&gt;&#xA;&lt;p&gt;Edit: Separable convolution means that first depthwise convolution is applied then pointwise convolution is applied.&lt;/p&gt;&#xA;" OwnerUserId="41615" LastEditorUserId="41615" LastEditDate="2020-10-17T13:51:51.197" LastActivityDate="2020-10-18T15:03:32.537" Title="When should we use separable convolution?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24069" PostTypeId="1" CreationDate="2020-10-14T12:48:52.810" Score="2" ViewCount="35" Body="&lt;p&gt;We can use DDPG to train agents to stack objects. And stacking objects can be viewed as first grasping followed by pick and place. In this context, how does meta-reinforcement learning fit? Does it mean I can use grasp, pick and place as training tasks and generalize to assembling objects?&lt;/p&gt;&#xA;" OwnerUserId="41617" LastEditorUserId="2444" LastEditDate="2020-10-16T22:54:41.913" LastActivityDate="2020-10-16T22:54:41.913" Title="What exactly does meta-learning in reinforcement learning setting mean?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;meta-learning&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24070" PostTypeId="2" ParentId="24065" CreationDate="2020-10-14T14:04:36.780" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;is it common to deal with weights and biases in everyday tasks or in most of the cases existing algorithms do it well?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;No&lt;/strong&gt;; and it is no coincidence that you will not be able to find any reference to such a practice in any course or tutorial about neural networks. Such a practice would require a whole additional level of (business/SME) know-how in order to meaningfully apply neural networks to real-world problems, and fortunately this is not necessary.&lt;/p&gt;&#xA;&lt;p&gt;The desired situation is for both weights &amp;amp; biases to remain in a relatively small* range around zero, among other reasons because this avoids the exploding &amp;amp; vanishing gradient problems, which is catastrophic for learning; trying to adjust for the scale of our inputs &amp;amp; outputs by scaling accordingly the model weights &amp;amp; biases is the wrong approach, and never followed.&lt;/p&gt;&#xA;&lt;p&gt;[*How small? Well, see &lt;a href=&quot;https://stackoverflow.com/questions/48865055/multilayer-perceptron-with-sigmoid-activation-produces-straight-line-on-sin2x/48866370#48866370&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/43636321/tensorflow-training-accuracy-not-improving-in-mnist-data/43644546#43644546&quot;&gt;here&lt;/a&gt; for what a change in just the &lt;em&gt;standard deviation&lt;/em&gt; of a zero-mean weight initialization from &lt;code&gt;1.0&lt;/code&gt; and &lt;code&gt;0.1&lt;/code&gt; to &lt;code&gt;0.01&lt;/code&gt; can do to the model performance]&lt;/p&gt;&#xA;&lt;p&gt;It is a well-established fact by now that neural nets work with normalized inputs, and, depending on the problem, normalized &lt;em&gt;outputs&lt;/em&gt; as well; which answers to your objection:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;One could answer that normalizing inputs solves the problem and no need to fit weights and biases, but I guess they depend also on expected output.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It depends indeed, but the solution here is to scale the &lt;em&gt;output(s)&lt;/em&gt; as well, and not the weights.&lt;/p&gt;&#xA;&lt;p&gt;Now, it's true that &lt;em&gt;de-scaling&lt;/em&gt; the outputs back to their original range in order to be able to meaningfully compare them with the ground truth (and possibly compute more meaningful metrics, like the error on our true output range, and not the scaled one) is AFAIK seldom mentioned in introductory expositions; but this is indeed the correct thing to do (especially for regression problems, where metrics like MSE are scale-sensitive), instead of trying to manually intervene on the weights range. For details, see own answers in &lt;a href=&quot;https://stackoverflow.com/questions/48973140/how-to-interpret-mse-in-keras-regressor/49009442#49009442&quot;&gt;How to interpret MSE in Keras Regressor&lt;/a&gt; and &lt;a href=&quot;https://datascience.stackexchange.com/questions/82285/ann-regression-accuracy-and-loss-stuck/82296#82296&quot;&gt;ANN regression accuracy and loss stuck&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;what are the rules of thumb for how to decide about range and initial values of weights and biases?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Range aside, the general rule, as already implied, is to initialize the biases with zeros and the weights with small random values around zero. Nevertheless, the exact details are an area of active research. Currently used initialization schemes that are already integrated into the relevant frameworks (Tensorflow, Keras, Pytorch etc) are the &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Glorot (or Xavier)&lt;/a&gt; and &lt;a href=&quot;https://ieeexplore.ieee.org/document/7410480&quot; rel=&quot;nofollow noreferrer&quot;&gt;He&lt;/a&gt; initializations (for a nice overview, see &lt;a href=&quot;https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Initialization in Neural Networks: A Journey From the Basics to Kaiming&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Beyond these routinely-used approaches that have already reached the practitioner's workbench, and moving closer to the front of active theoretical research, the &lt;a href=&quot;https://openreview.net/forum?id=rJl-b3RcF7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lottery Ticket Hypothesis&lt;/a&gt; (finding &amp;quot;winning&amp;quot; weight initializations that require minimal training) is an ultra-hot topic lately.&lt;/p&gt;&#xA;" OwnerUserId="11539" LastEditorUserId="11539" LastEditDate="2020-10-14T17:22:19.657" LastActivityDate="2020-10-14T17:22:19.657" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24071" PostTypeId="2" ParentId="24067" CreationDate="2020-10-14T15:01:49.167" Score="4" Body="&lt;p&gt;A typical and practical way to measure the convergence to &lt;strong&gt;some&lt;/strong&gt; solution (so not necessarily the optimal one!) of any numerical iterative algorithm (such as RL algorithms) is to check if the current solution has not changed (much) with respect to the previous one. In your case, the solutions are value functions, so you could check if your algorithm has converged to some value function e.g. as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;c(q_t, q_{t-1}, \epsilon) =&#xA;\begin{cases}&#xA;1,  &amp;amp;\text{if } |q_t(s, a) - q_{t-1}(s, a)| &amp;lt; \epsilon, \forall s \in S, a \in A \\&#xA;0, &amp;amp; \text{otherwise}&#xA;\end{cases},&#xA;\tag{1}\label{1}&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the &amp;quot;convergence&amp;quot; function (aka termination condition) that returns &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (true) if your RL algorithm has converged to some small enough neighbourhood of value functions (where those value functions are &amp;quot;indistinguishable&amp;quot;), and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; otherwise&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$q_t$&lt;/span&gt; is the value function at iteration &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is a threshold (aka precision or tolerance) value, which is a hyper-parameter that you can set depending on your &amp;quot;tolerance&amp;quot; (hence the name); this value is typically something like &lt;span class=&quot;math-container&quot;&gt;$10^{-6}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Of course, this requires that you keep track of two value functions.&lt;/p&gt;&#xA;&lt;p&gt;You can also define your &amp;quot;convergence&amp;quot; function &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in \ref{1} differently. For example, rather than using the absolute value, you could use the relative error, i.e. &lt;span class=&quot;math-container&quot;&gt;$\left|\frac{q_t(s, a) - q_{t-1}(s, a)}{q_t(s, a)} \right|$&lt;/span&gt;. Moreover, given that RL algorithms are exploratory (i.e. stochastic) algorithms, the value function may not change (much) from one iteration to the other, but, in the next one, it could significantly change because of your exploratory/behavioural actions, so you may also want to take into account more iterations, i.e. after e.g. &lt;span class=&quot;math-container&quot;&gt;$N &amp;gt; 1$&lt;/span&gt; iterations, if the value function does not change much, then you could say (maybe probabilistically) that your RL algorithm has converged to some small neighbourhood of value functions in the space of value functions.&lt;/p&gt;&#xA;&lt;p&gt;Note that these approaches do not guarantee that your RL algorithm has converged to the &lt;strong&gt;global optimal&lt;/strong&gt; value function, but to &lt;strong&gt;some locally optimal&lt;/strong&gt; value function (or, more precisely, small neighborhood of value functions). &lt;a href=&quot;https://ai.stackexchange.com/q/11679/2444&quot;&gt;Q-learning is guaranteed to converge to the optimal value function in the tabular setting&lt;/a&gt; (your setting), but this is in the limit; in practice, it is more difficult to know if Q-learning has converged to an optimal or near-optimal value function.&lt;/p&gt;&#xA;&lt;p&gt;Maybe you can also have a look at episodic returns of the policy derived from your final value function, but without upper and lower bounds on the optimal returns, you don't know much about the global optimality of your policy/value function.&lt;/p&gt;&#xA;&lt;p&gt;Yes, you can check if the policy makes the agent reach the goal, but many policies could do that job, i.e. that does not say that the policy is the best (or optimal) one, i.e. that's a necessary (provided the goal is reachable and the reward function models your actual goal) but not sufficient condition (for optimality). The optimality here is usually a function of the return (given that is what you are usually trying to optimize).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-14T15:43:52.777" LastActivityDate="2020-10-14T15:43:52.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24072" PostTypeId="1" CreationDate="2020-10-14T15:03:45.813" Score="1" ViewCount="28" Body="&lt;p&gt;A recent paper &amp;quot;MagGAN High Resolution Face Attribute Editing with Mask Guided GAN&amp;quot; published this month (October 2020) describe how an approach has been developed to deal with specific face attribute editing.&lt;/p&gt;&#xA;&lt;p&gt;The thing is that in this paper and related work (StarGAN, CycleGAN, AttGAN, STGAN ...) seems to tackle the process of adding / editing a face attribute (e.g. adding a hat or mustache...), but not really modifying the face attribute (like eyes, nose, lips ...)&lt;/p&gt;&#xA;&lt;p&gt;Is there anyway we can make a model that can edit for example the nose type/size or any related works already published?&lt;/p&gt;&#xA;" OwnerUserId="41621" LastEditorUserId="11539" LastEditDate="2020-10-15T16:15:49.140" LastActivityDate="2020-10-15T16:15:49.140" Title="GAN for specific face attribute modification" Tags="&lt;generative-adversarial-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24073" PostTypeId="2" ParentId="24008" CreationDate="2020-10-14T15:17:41.897" Score="1" Body="&lt;p&gt;You can define different measures in this way:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Maximum distance of the new point with all points of the configuration (&lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Minimum distance of the new point with all points of the configuration (&lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{M}{\text{Maximum distance between two points of the configuration(}D)}$&lt;/span&gt;: normalize (1) measure&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{N}{D}$&lt;/span&gt;:normalize (2) measure&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can get more ideas from &lt;a href=&quot;https://online.stat.psu.edu/stat555/node/85/#:%7E:text=For%20most%20common%20hierarchical%20clustering,when%20they%20are%20perfectly%20correlated.&quot; rel=&quot;nofollow noreferrer&quot;&gt;distance measures in hierarchical clustering methods&lt;/a&gt;. To select a proper one, you need to elaborate on the context of these points.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2020-10-14T15:17:41.897" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24074" PostTypeId="2" ParentId="23669" CreationDate="2020-10-14T16:18:37.303" Score="1" Body="&lt;p&gt;In the paper that you cite, &lt;a href=&quot;https://papers.nips.cc/paper/7253-inverse-reward-design&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inverse Reward Design&lt;/a&gt; (2017), the authors actually define what they mean by &amp;quot;proxy reward function&amp;quot;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We formalize this in a probabilistic model that relates the &lt;strong&gt;proxy (designed) reward&lt;/strong&gt; to the true reward&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, the &lt;strong&gt;proxy&lt;/strong&gt; reward function is the reward function designed by the human, which may not necessarily be the reward function that he/she intended (i.e. it may be a misspecified reward function), given that the human may have forgotten to model/incorporate certain (unpredicted by the human) scenarios or situations that the agent may face. This usage of the word &amp;quot;proxy&amp;quot; is thus consistent with the general usage of the word in computer science, i.e. a &amp;quot;proxy reward function&amp;quot; is a reward function that is used &lt;strong&gt;instead of&lt;/strong&gt; the intended (optimal) reward function.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-10-14T16:18:37.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24075" PostTypeId="1" CreationDate="2020-10-14T17:26:52.680" Score="1" ViewCount="17" Body="&lt;p&gt;I am currently working on a novel application in NLP where I try to classify empathic and non-empathic texts. I would like to compare the performance of my model to some benchmark models. As I am working with models based on Word2Vec embeddings, the benchmark models should also be based on Word2Vec, however I am looking for some relatively easy, quick to implement models.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="37792" LastActivityDate="2020-10-14T17:26:52.680" Title="Bechmark models for Text Classification / Sentiment Classification" Tags="&lt;natural-language-processing&gt;&lt;word-embedding&gt;&lt;text-classification&gt;&lt;word2vec&gt;&lt;benchmarks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24077" PostTypeId="1" CreationDate="2020-10-14T20:02:24.240" Score="2" ViewCount="45" Body="&lt;p&gt;To my knowledge, there are at least 6 different variants of Actor Critic:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{array}{l l l l}&#xA;\text{actor gradient} &amp;amp; \text{critic gradient} &amp;amp; \text{actor gradient biased} &amp;amp; \text{name} \\&#xA;\hline&#xA;\sum_t \nabla_\theta \log \pi_\theta(a_t | s_t) \cdot (R_t - V_\theta) &amp;amp; \nabla_\theta \frac{1}{T} \sum_t (R_t - V_\theta(s_t))^2 &amp;amp; \text{no} &amp;amp; \text{Monte Carlo Advantage Actor Critic V (?)} \\&#xA;\sum_t \nabla_\theta \log \pi_\theta(a_t | s_t) \cdot (R_t - \sum_a \pi_\theta(a | s_t) Q_\theta(s_t, a)) &amp;amp; \nabla_\theta \frac{1}{T} \sum_t (R_t - Q_\theta(s_t, a_t))^2 &amp;amp; \text{no} &amp;amp; \text{Monte Carlo Advantage Actor Critic Q (?)} \\&#xA;\sum_t \nabla_\theta \log \pi_\theta(a_t | s_t) \cdot Q_\theta(s_t, a_t) &amp;amp; \text{same as above} &amp;amp; \text{yes} &amp;amp; \text{Actor Critic (?)} \\&#xA;\sum_t \nabla_\theta \log \pi_\theta(a_t | s_t) \cdot (Q_\theta(s_t, a_t) - \sum_a \pi_\theta(a | s_t) Q_\theta(s_t, a)) &amp;amp; \text{same as above} &amp;amp; \text{yes} &amp;amp; \text{Advantage Actor Critic (?)} \\&#xA;\sum_t \sum_a \nabla_\theta \pi_\theta(a | s_t) \cdot Q_\theta(s_t, a) &amp;amp; \text{same as above} &amp;amp; \text{yes} &amp;amp; \text{Mean Actor Critic} \\&#xA;\sum_t \sum_a \nabla_\theta \log \pi_\theta(a_t | s_t) \cdot (r_t + \gamma V(s_{t+1}) - V(s_t)) &amp;amp;  \text{same as MCAACV} &amp;amp; \text{yes} &amp;amp; \text{TD Advantage Actor Critic (?)}&#xA;\end{array}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem I'm having is that &lt;em&gt;all&lt;/em&gt; methods which use a biased actor gradient are performing abysmally (even worse than their random initializations) on the simple &lt;a href=&quot;https://gym.openai.com/envs/CartPole-v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CartPole&lt;/a&gt; task. See the graph below for an example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X17Gu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X17Gu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know that, ideally, we want &lt;span class=&quot;math-container&quot;&gt;$Q_\theta$&lt;/span&gt; to reflect the true Q values under the current policy &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt;, thus we probably want to &amp;quot;slow down&amp;quot; how quickly &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt; changes in order to give time for &lt;span class=&quot;math-container&quot;&gt;$Q_\theta$&lt;/span&gt; to learn the true Q values of &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt;. Methods like TRPO, PPO, etc. try to do this, but the &lt;a href=&quot;https://arxiv.org/pdf/1709.00503.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mean Actor Critic paper&lt;/a&gt; seems to get the biased-gradient methods (including their own) working without them (TRPO is separate from A2C and MAC in the Table 2). One thing I tried was scaling down the actor gradient in comparison to the critic gradient, but that didn't seem to help.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know what the issue might be, and how to fix it?&lt;/p&gt;&#xA;" OwnerUserId="3373" LastActivityDate="2020-10-14T20:02:24.240" Title="Problems with gradient-biased actor critic methods" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;advantage-actor-critic&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24078" PostTypeId="1" CreationDate="2020-10-14T21:55:20.303" Score="1" ViewCount="46" Body="&lt;p&gt;I'm researching the use of emotion recognition in Intelligent Tutoring Systems and trying to more effectively find and formally reference materials. My question is whether this is the most formal terminology (i.e. &amp;quot;emotion recognition&amp;quot;), because I've also seen &amp;quot;affect recognition&amp;quot; and &amp;quot;affective computing&amp;quot;. Maybe it's a matter of taste, but I know sometimes the market terminology is different from the engineering terminology and I'd like to be more in tune with the engineers.&lt;/p&gt;&#xA;&lt;p&gt;Maybe there is a leading classification system of related technologies (e.g. facial recognition, sentiment analysis, etc.)?&lt;/p&gt;&#xA;&lt;p&gt;I'm seeing the &amp;quot;affective-computing&amp;quot; tag now, but not sure if these tags reflect a formal classification system in the field of AI.&lt;/p&gt;&#xA;" OwnerUserId="41626" LastEditorUserId="2444" LastEditDate="2021-02-01T00:45:19.967" LastActivityDate="2021-02-01T00:45:19.967" Title="What is the formal terminology for emotion recognition AI?" Tags="&lt;terminology&gt;&lt;facial-recognition&gt;&lt;affective-computing&gt;&lt;emotion-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24079" PostTypeId="2" ParentId="24054" CreationDate="2020-10-14T22:46:35.717" Score="0" Body="&lt;p&gt;Monocular depth estimation basically does this and it implicitly brings in knowledge about object size. But you need to have prior information.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-10-14T22:46:35.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24081" PostTypeId="1" CreationDate="2020-10-15T00:41:11.373" Score="1" ViewCount="61" Body="&lt;p&gt;Back before deep learning, there were a lot of different attempts at computer vision. Some involved Conditional Random Fields and Markov Random Fields, which were both computationally difficult and hard to understand/implement.&lt;/p&gt;&#xA;&lt;p&gt;Are these areas still being developed in the computer vision domain? What was the end result of this line of study? I haven't seen any papers on this topic be cited in top-performing benchmarks, so I assume nobody cares about them anymore, but I wanted to ask.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-10-17T11:52:17.163" LastActivityDate="2021-03-17T09:07:35.890" Title="Are Markov Random Fields and Conditional Random Fields still used in computer vision?" Tags="&lt;computer-vision&gt;&lt;state-of-the-art&gt;&lt;conditional-random-field&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24082" PostTypeId="1" CreationDate="2020-10-15T02:59:32.383" Score="1" ViewCount="38" Body="&lt;p&gt;I have been trying to figure out whether if I train a model and then while predicting is it possible to train images too just like humans&#xA;Somehow converting valid images to the dataset by asking us when an object is shown&#xA;Like the Google Photos somehow they ask us if they predicted a face correctly and then reinforces on it&lt;/p&gt;&#xA;" OwnerUserId="30884" LastActivityDate="2020-10-15T04:11:59.903" Title="Training while predicting on dataset" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;ai-milestones&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24083" PostTypeId="2" ParentId="24082" CreationDate="2020-10-15T04:11:59.903" Score="1" Body="&lt;p&gt;Yes, this method of training a model is commonly known as &lt;strong&gt;online learning&lt;/strong&gt; and specific learning algorithms have been designed for this purpose, such as, Stochastic Gradient Descent(SGD). As opposed to Batch Gradient descent, which computes gradients over the entire training set at each step, the SGD algorithm computes gradients for individual samples and updates the model's parameters.&lt;/p&gt;&#xA;&lt;p&gt;The online learning strategy is not specific to Reinforcement Learning(RL) methods but is also suitable for the broader classification of Machine Learning algorithms. Suppose, we are given a simple model pretrained on a large Face Recognition dataset, and wish the model to continuously adapt to new faces, we may keep feeding in the new input images with their correct labels (obtained from feedback), as they appear, and apply SGD to incrementally update the model. To go about this using an RL based approach, we may penalize the model with negative rewards for every label misclassified and update it.&lt;/p&gt;&#xA;" OwnerUserId="38280" LastActivityDate="2020-10-15T04:11:59.903" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24086" PostTypeId="1" CreationDate="2020-10-15T15:07:24.000" Score="0" ViewCount="28" Body="&lt;p&gt;There are two sources that I'm using to to try and understand why LSTMs reduce the likelihood of the vanishing gradient problem associated with RNNs.&lt;/p&gt;&#xA;&lt;p&gt;Both of these sources mention the reason LSTMs are able to reduce the likelihood of the vanishing gradient problem is because&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The gradient contains the forget gate's vector of activions&lt;/li&gt;&#xA;&lt;li&gt;The addition of four gradient values help balance gradient values&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I understand (1), but I don't understand what (2) means.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Sources are SLIDE 119 on &lt;a href=&quot;http://cs231n.stanford.edu/slides/2020/lecture_10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cs231n.stanford.edu/slides/2020/lecture_10.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sentence beginning &amp;quot;Another important property to notice is that the cell state&amp;quot; on &lt;a href=&quot;https://medium.com/datadriveninvestor/how-do-lstm-networks-solve-the-problem-of-vanishing-gradients-a6784971a577&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/datadriveninvestor/how-do-lstm-networks-solve-the-problem-of-vanishing-gradients-a6784971a577&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any insight would greatly be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="26159" LastActivityDate="2020-10-15T15:07:24.000" Title="In LSTMs, how does the additive property enables better balancing of gradient values during backpropagation?" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;backpropagation&gt;&lt;vanishing-gradient-problem&gt;&lt;bptt&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24087" PostTypeId="1" AcceptedAnswerId="24089" CreationDate="2020-10-15T15:12:31.810" Score="3" ViewCount="118" Body="&lt;p&gt;For simplicity, let's assume we want to solve a regression problem, where we have one independent variable and one dependent variable, which we want to predict. Let's also assume that there is a nonlinear relationship between the independent and dependent variables.&lt;/p&gt;&#xA;&lt;p&gt;No matter the way we do it, we just need to build a proper curved line based on existing observations, such that the prediction is the best.&lt;/p&gt;&#xA;&lt;p&gt;I know we can solve this problem with neural networks, but I also know other ways to create such curves. For example:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;splines&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;kriging&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;lowess&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Something I think would also work (do not know if exists): fitting curve using a series of Fourier sine waves, and so on&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it true that neural networks are just one of the ways to fit a non-linear curve to the data?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the advantages and disadvantages of choosing a neural network over other approaches? (maybe it becomes better when I have many independent variables, and another little guess: maybe the neural network is better in omitting the effect of linear dependent input variables?)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36453" LastEditorUserId="2444" LastEditDate="2020-10-16T22:48:20.817" LastActivityDate="2021-02-15T22:25:56.160" Title="What is the difference between neural networks and other ways of curve fitting?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;comparison&gt;&lt;non-linear-regression&gt;&lt;curve-fitting&gt;" AnswerCount="3" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24089" PostTypeId="2" ParentId="24087" CreationDate="2020-10-15T21:10:15.903" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In some sense, you're right that a neural net is just another tool to fit data. However, it's &lt;em&gt;quite&lt;/em&gt; the tool! There's this &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal approximation theorem&lt;/a&gt; saying that, under decent conditions, a neural network can get as close as you want to a wide class of functions. This means that you can get the network to give you complicated shapes with squiggles all over if that's the right trend.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The universal approximation theorem is a big upside. You don't have to specify that you want to model with sine curves or a particular type of spline. You just let the computer figure that out for you. The result is the ability to model complex patterns and make accurate predictions. The drawback is that the modeling can pick up on coincidences in the data that look like a trend but are not. This causes overfitting. When your goal is to make accurate predictions, a model that has overfit does nothing for you. A second drawback is that neural networks are hard to interpret. A third drawback is that they can take a long time to train, while a linear regression is just a matrix inversion and a couple of matrix products (the &lt;span class=&quot;math-container&quot;&gt;$\hat{\beta}=(X^TX)^{-1}X^Ty$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25529" LastEditorUserId="25529" LastEditDate="2020-10-16T12:58:45.703" LastActivityDate="2020-10-16T12:58:45.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24092" PostTypeId="1" CreationDate="2020-10-16T00:39:16.427" Score="0" ViewCount="19" Body="&lt;p&gt;I am reading the &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT paper&lt;/a&gt;. In the paper, they say that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Although this allows us to obtain a bidirec- tional pre-trained model, a downside is that we are creating a mismatch between pre-training and fine-tuning, since the [MASK] token does not ap- pear during fine-tuning. To mitigate this, we do not always replace “masked” words with the ac- tual [MASK] token. The training data generator chooses 15% of the token positions at random for prediction. If the i-th token is chosen, we replace the i-th token with (1) the [MASK] token 80% of the time (2) a random token 10% of the time (3) the unchanged i-th token 10% of the time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am wondering, what will happen when they keep the token unchanged? Is it necessary for better training? I cannot see the mismatch clearly. If they do not do that, what will happen? Also, when the model is given the unchanged token, does it predict the exact token correctly in your experience?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="40551" LastActivityDate="2020-10-16T00:39:16.427" Title="What BERT predicts when the token supposed to be masked is not masked?" Tags="&lt;natural-language-processing&gt;&lt;papers&gt;&lt;transformer&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24093" PostTypeId="1" CreationDate="2020-10-16T07:04:05.373" Score="0" ViewCount="58" Body="&lt;p&gt;I have read up on some research papers that state that shallow pruning is possible when &lt;code&gt;sum of current node - upperbound of current player &amp;lt;= upperbound of previous player&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, what I noticed is that this only works when there is equal sum across all nodes.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible if every node has a different sum? For example, if 3 children have sums of 10, 15, 20?&lt;/p&gt;&#xA;&lt;p&gt;Here is the paper I am reading: &lt;a href=&quot;https://www.cc.gatech.edu/%7Ethad/6601-gradAI-fall2015/Korf_Multi-player-Alpha-beta-Pruning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiplayer alpha-beta pruning&lt;/a&gt; (by Richard E. Korf, 1991)&lt;/p&gt;&#xA;" OwnerUserId="41650" LastEditorUserId="2444" LastEditDate="2020-10-17T11:49:36.253" LastActivityDate="2020-10-17T11:49:36.253" Title="When is shallow pruning possible in alpha-beta pruning for multi-player games?" Tags="&lt;game-ai&gt;&lt;game-theory&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24094" PostTypeId="1" CreationDate="2020-10-16T07:45:02.123" Score="0" ViewCount="90" Body="&lt;p&gt;I am not fully understanding how to train a GAN's generator. I have a few questions below, but let me first describe what I am doing.&lt;/p&gt;&#xA;&lt;p&gt;I am using the MNIST dataset.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I generate a batch of random images (the faked ones) with the generator.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I train the discriminator with the set composed of faked images and real MNIST images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After the training phase, the discriminator modifies the weights in the direction of recognizing fake (probability 0) from real (probability 1) ones.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At this point, I have to consider the combined model of generator and discriminator (keep untrainable the discriminator) and put in the generator as input the faked images with the tag of 1s (as was real one).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;p&gt;Why do I have to set to real these fake images, and what fake images are these? The one generated in the first round from the generator itself? Or only the one classified as faked by the discriminator? (Then they could be both real images classified wrongly or fake images classified in the right way). Finally, what the generator does to these faked images?&lt;/p&gt;&#xA;" OwnerUserId="23717" LastEditorUserId="2444" LastEditDate="2020-10-17T20:30:54.847" LastActivityDate="2021-03-18T20:07:05.933" Title="What is the right way to train a generator in a GAN?" Tags="&lt;training&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;mnist&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24095" PostTypeId="2" ParentId="24094" CreationDate="2020-10-16T08:53:18.837" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Why I have to set to real these fake images and what fake images are these?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You set them to &amp;quot;real&amp;quot; label for the discriminator when training the generator, because that is the goal of the generator, to produce an output of 1 (probability of being a real image) when tested.&lt;/p&gt;&#xA;&lt;p&gt;Usually you will generate a new batch of generated images for this step in training. You just used the last generated mini-batch to train the discriminator, so you expect them to score worse. Sending the exact same images again will cause correlation between the two minibatches that you want to avoid. It would not be a disaster, but training GANs can be quite difficult and sensitive to details like this, so it is better to keep generating new images and not re-use the previous ones.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The one generated in the first round from the generator itself?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. New images generated just for training the generator.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Or only the one classified as faked by the discriminator? (then they could be both real images classified wrongly or fake images classified in the right way).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. New images generated just for training the generator.&lt;/p&gt;&#xA;&lt;p&gt;Out of interest though, if the discriminator classifies a fake image as 100% real (with a probability close to 1), then the generator will not learn anything from that. The gradients would all be zero.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Finally what the generator does to these faked images?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Nothing is done to the images themselves - unless perhaps you are keeping some copies to render and monitor training progress etc. The images occur within the combined generator/discriminator network, effectively as a hidden layer. The images are represented as artificial neuron output, so they are &lt;em&gt;involved&lt;/em&gt; in backpropagation calculations for that layer (with no difference to any other hidden layer in a CNN), but are not changed directly.&lt;/p&gt;&#xA;&lt;p&gt;The generator uses the gradients calculated from the combined discriminator/generator network to update its weights using gradient descent. Importantly in this phase of the updates, the discriminator weights are not changed.&lt;/p&gt;&#xA;&lt;p&gt;In terms of training the generator/discriminator combined network to update the generator:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The input to the combined network is some new random input vectors (typically a vector with independent truncated normal distribution for each element).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &amp;quot;ground truth&amp;quot; label is 1 for every item.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The discriminator parameters must be &amp;quot;frozen&amp;quot;, somehow excluded from being updated.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run the minibatch forward to get loss and backpropagate to get gradients for the whole network including the generator.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply a gradient step (usually via some optimiser, such as Adam).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-16T10:32:22.470" LastActivityDate="2020-10-16T10:32:22.470" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24096" PostTypeId="1" CreationDate="2020-10-16T10:26:39.013" Score="2" ViewCount="69" Body="&lt;p&gt;Meta-programming is programming that can modify itself, which is an appealing concept to me, given that I associate the idea of modifying one's self with general intelligences (like humans).&lt;/p&gt;&#xA;&lt;p&gt;I think that the combination of meta-programming with evolutionary algorithms could lead to the development of an AGI system, provided that the creation of an AGI system is possible.&lt;/p&gt;&#xA;&lt;p&gt;So, is there any research on developing AGI systems based on meta-programming?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="2444" LastEditDate="2020-10-16T23:22:13.567" LastActivityDate="2020-10-16T23:22:13.567" Title="Is there any research on developing AGI systems based on meta-programming?" Tags="&lt;reference-request&gt;&lt;agi&gt;&lt;computer-programming&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24097" PostTypeId="1" CreationDate="2020-10-16T12:51:56.380" Score="2" ViewCount="34" Body="&lt;p&gt;To train the discriminator network in GANs we set the label for the true samples as &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for fake ones. Then we use binary cross-entropy loss for training.&lt;/p&gt;&#xA;&lt;p&gt;Since we set the label &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for true samples that means &lt;span class=&quot;math-container&quot;&gt;$p_{data}(x) = 1$&lt;/span&gt; and now binary cross-entropy loss is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_1 = \sum_{i=1}^{N} P_{data}(x_i)log(D(x)) + (1-P_{data}(x_i))log(1-D(x))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_1 = \sum_{i=1}^{N} P_{data}(x_i)log(D(x))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_1 = E_{x \sim P_{data}(x)}[log(D(x))]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the second part, since we set the label &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for fake samples that means &lt;span class=&quot;math-container&quot;&gt;$p_{z}(z) = 0$&lt;/span&gt; and now binary cross-entropy loss is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_2 = \sum_{i=1}^{N} P_{z}(z_i)log(D_{G}(z)) + (1-P_{z}(z_i))log(1-D_{G}(z))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_2 = \sum_{i=1}^{N} 1-P_{z}(z_i)log(1-D_{G}(z))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_2 = E_{z \sim \bar{P_{z}(z)}}[log(1-D_{G}(z))]$$&lt;/span&gt;&#xA;Now we combine those two losses and get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_D = E_{x \sim P_{data}(x)}[log(D(x))] + E_{z \sim \bar{P_{z}(z)}}[log(1-D_{G}(z))]$$&lt;/span&gt;&#xA;When I was reading about &lt;a href=&quot;https://arxiv.org/pdf/1406.2661.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GANs&lt;/a&gt; I saw that the loss function for discriminator is defined as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_D = E_{x \sim P_{data}(x)}[log(D(x))] + E_{z \sim P_{z}(z)}[log(1-D_{G}(z))]$$&lt;/span&gt;&#xA;Should not it be &lt;span class=&quot;math-container&quot;&gt;$E_{z \sim \bar{P_{z}(z)}}$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$E_{z \sim P_{z}(z)}$&lt;/span&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-10-17T06:49:43.717" LastActivityDate="2020-10-17T06:49:43.717" Title="How to define loss function for Discriminator in GANs?" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24100" PostTypeId="1" AcceptedAnswerId="24108" CreationDate="2020-10-16T15:28:42.010" Score="1" ViewCount="74" Body="&lt;p&gt;Training an SVM with an RBF kernel model with &lt;code&gt;c = 5.5&lt;/code&gt; and &lt;code&gt;gamma = 1.06&lt;/code&gt;, for a 5-class classification problem on the NSL-KDD train data-set with 122 features using one vs rest strategy takes &lt;span class=&quot;math-container&quot;&gt;$2162$&lt;/span&gt; seconds. Also, considering binary classification (&lt;code&gt;c = 10&lt;/code&gt;, &lt;code&gt;gamma = 4&lt;/code&gt;), it takes &lt;span class=&quot;math-container&quot;&gt;$520.56$&lt;/span&gt; seconds.&lt;/p&gt;&#xA;&lt;p&gt;After dimensionality reduction, from 122 to 30, using a sparse auto-encoder, the training time falls dramatically, from &lt;span class=&quot;math-container&quot;&gt;$2162$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$240$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$520$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$170$&lt;/span&gt;, while using the same hyperparameters for the RBF-kernel.&lt;/p&gt;&#xA;&lt;p&gt;What is the reason for that? Is it not true that using kernel neutralized the effect of high dimensions?&lt;/p&gt;&#xA;" OwnerUserId="41662" LastEditorUserId="2444" LastEditDate="2020-10-18T09:48:20.000" LastActivityDate="2020-10-18T09:48:20.000" Title="Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;autoencoders&gt;&lt;support-vector-machine&gt;&lt;feature-selection&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24101" PostTypeId="1" CreationDate="2020-10-16T18:00:58.693" Score="0" ViewCount="22" Body="&lt;p&gt;this may sound naive, but I’m getting a really puzzling result. I was experimenting with MNIST on vanilla MLP &lt;code&gt;(784, 256, 128, 10)&lt;/code&gt; with &lt;code&gt;ReLU&lt;/code&gt; and &lt;code&gt;batch size 64&lt;/code&gt;. When I trained on multiple random initializations I got the following result. It seems  for some initializations of weights and biases, the network does not converge to the desired minima. However, if I perform the same experiment with &lt;code&gt;Tanh/Sigmoid&lt;/code&gt;, all initializations eventually converge to same desired minima.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gKWZ7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gKWZ7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I understand that convergence for &lt;code&gt;SGD&lt;/code&gt; depends on initial position in the landscape but why is it only happening with &lt;code&gt;ReLU&lt;/code&gt; and not &lt;code&gt;Tanh&lt;/code&gt; . I have tried multiple initializations (uniform, He, orthogonal) but all yield same behavior. Seems like &lt;code&gt;ReLU&lt;/code&gt; is quite sensitive to choice of initial W &amp;amp; B.&lt;/p&gt;&#xA;" OwnerUserId="28091" LastActivityDate="2020-10-16T18:00:58.693" Title="Random Initializations with ReLU gives puzzling results" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;multilayer-perceptron&gt;&lt;relu&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24103" PostTypeId="2" ParentId="24040" CreationDate="2020-10-16T20:57:05.587" Score="0" Body="&lt;p&gt;Maybe later I can work through your math though I can say this.&lt;/p&gt;&#xA;&lt;p&gt;First, a forward pass is done to obtain predictions of &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, at every &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Then the adjoint state is run backward in time for every &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Which gives the learning impulse. So an initial run is done to obtain values of the dynamical system at all time points and the last of these is the initial point for the backward pass.&lt;/p&gt;&#xA;&lt;p&gt;This is probably not enough to satisfy your question, hopefully, I can get more specific with respect to the math later.&lt;/p&gt;&#xA;" OwnerUserId="41565" LastEditorUserId="41565" LastEditDate="2021-04-19T09:45:18.823" LastActivityDate="2021-04-19T09:45:18.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24104" PostTypeId="5" CreationDate="2020-10-16T22:50:53.017" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-16T22:50:53.017" LastActivityDate="2020-10-16T22:50:53.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24105" PostTypeId="4" CreationDate="2020-10-16T22:50:53.017" Score="0" Body="For questions about the concept of curve fitting in the context of machine learning and other related areas." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-16T22:50:53.017" LastActivityDate="2020-10-16T22:50:53.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24106" PostTypeId="1" CreationDate="2020-10-17T03:33:54.803" Score="1" ViewCount="57" Body="&lt;p&gt;I know that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}[g(X) \mid A] = \sum\limits_{x} g(x) p_{X \mid A}(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for any random variable &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, consider the following expression.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{\pi} \left[ \sum \limits_{k=0}^{\infty} \gamma^{k}r_{t+k+1} \mid s_t = s, a_t = a \right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is used for the calculation of Q values.&lt;/p&gt;&#xA;&lt;p&gt;I can understand that event the following&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\{s_t = s, a_t = a\}$&lt;/span&gt; .i.e., agent has been performed action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; on state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(X)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\sum\limits_{k=0}^{\infty} \gamma^{k}r_{t+k+1}$&lt;/span&gt; i.e., return (long run reward).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I didn't understand what is &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; here. i.e., what is the random variable on which we are calculating long run rewards?&lt;/p&gt;&#xA;&lt;p&gt;My guess is policy function. It is averaging long run rewards over all possible policy functions. Is it true?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="2444" LastEditDate="2020-10-18T10:06:15.423" LastActivityDate="2020-10-18T11:28:05.077" Title="In the definition of the state-action value function, what is the random variable we take the expectation of?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;value-functions&gt;&lt;notation&gt;&lt;random-variable&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24107" PostTypeId="1" CreationDate="2020-10-17T08:36:52.527" Score="0" ViewCount="30" Body="&lt;p&gt;I have trained a WAN with gradient penalty and the loss values ​​seem to me much higher than the examples I have seen on the net. The generator receives 2 images as input and must generate a prediction of a possible middle image (such as a middle frame between two consecutive frames of a video) and the discriminator receives 3 images as input and must understand if the second image is a possible image that is between the first and third.&#xA;These are the loss functions during training:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YnHJN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YnHJN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am training the network for another 15 epochs now and the generator loss has reached values ​​like -300 in some epochs, while in others it returns to low values. However, the results seem to be improving. Are these values ​​normal or is there something wrong?&lt;/p&gt;&#xA;" OwnerUserId="40372" LastActivityDate="2020-10-17T08:36:52.527" Title="Wasserstein GAN with gradient penality - Loss values" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;&lt;wasserstein-metric&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24108" PostTypeId="2" ParentId="24100" CreationDate="2020-10-17T09:28:43.253" Score="1" Body="&lt;p&gt;SVM complexity is &lt;span class=&quot;math-container&quot;&gt;$O(\max(n,d)\min(n,d)^2)$&lt;/span&gt; according to Chapelle, Olivier. &amp;quot;Training a support vector machine in the primal.&amp;quot; Neural Computation 19.5 (2007): 1155-1178.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of instances and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the number of dimensions. I'm assuming that you have more instances than dimensions giving a complexity of &lt;span class=&quot;math-container&quot;&gt;$O(nd^2)$&lt;/span&gt;. Hopefully this explains fully why reducing the number of dimensions will reduce the training time.&lt;/p&gt;&#xA;" OwnerUserId="34473" LastActivityDate="2020-10-17T09:28:43.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24109" PostTypeId="2" ParentId="24106" CreationDate="2020-10-17T10:00:35.103" Score="3" Body="&lt;p&gt;I am using the convention of uppercase &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; for random variable and lowercase &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; for an individual observation. It is possible your source material did not do this, which might be causing your confusion. However, it is the convention used in Sutton &amp;amp; Barto's &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What I didn't understand what is  here. i.e., what is the random variable on which we are calculating long run rewards?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The random variable is &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;, the reward at each time step. The distribution of &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; in turn depends on the distribution of &lt;span class=&quot;math-container&quot;&gt;$S_{t-1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A_{t-1}$&lt;/span&gt; plus the policy and state progression rules. There is no need to include the &lt;em&gt;process&lt;/em&gt; that causes the distribution of each &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; in every equation. Although sometimes it is useful to do so, for example when deriving the Bellman equations for value functions.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My guess is policy function. It is averaging long run rewards over all possible policy functions. Is it true?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, this is not true. In fact it is the more usual assumption that the policy function &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; remains constant over the expectation, and this is what the subscript &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\pi}[...]$&lt;/span&gt; means.&lt;/p&gt;&#xA;&lt;p&gt;The expectation is over randomness due to the policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, plus randomness due to the environment, which can be described by the function &lt;span class=&quot;math-container&quot;&gt;$p(r, s'|s,a)$&lt;/span&gt; - the probability of observing reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; given starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking acion &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. These two functions combine to create the distribution of &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;. It is possible that both functions are deterministic in practice, thus &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; is also deterministic. However, RL theory works on the more general stochastic case, which is also used to model exploratory actions, even if the target policy and environment are deterministic.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-18T11:28:05.077" LastActivityDate="2020-10-18T11:28:05.077" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="24112" PostTypeId="1" CreationDate="2020-10-17T13:20:56.650" Score="2" ViewCount="57" Body="&lt;p&gt;I am trying to implement an AI bot for my Agar.io clone using deep neural network.&lt;br /&gt;&#xA;However, I am struggling with the state and action space of the AI bot.&lt;br /&gt;&#xA;Because the bot can take real number for position and velocity, can I say the state space is continuous?&lt;br /&gt;&#xA;For the action space, I am thinking something like (velocityX, velocityY, &amp;quot;split to half&amp;quot;, &amp;quot;eject mass&amp;quot;).&lt;br /&gt;&#xA;What should be the number of input nodes in the input layer for my Neural network? And what are those input(observations, rewards)?&lt;br /&gt;&#xA;As the number of players and AI bots are changing, how can I train a dynamic network with changing input node number?&lt;br /&gt;&#xA;For the outputs, how can I get a continuous action output like velocity?&lt;/p&gt;&#xA;&lt;p&gt;As a reference, you can learn about the game rules from this short youtube video:&lt;br /&gt;&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=xAry4_9yuis&quot; rel=&quot;nofollow noreferrer&quot;&gt;20 Rules and Game Mechanism of Agar (How to Play Agar.io)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41676" LastEditorUserId="41676" LastEditDate="2020-10-17T13:31:51.077" LastActivityDate="2020-10-18T05:46:58.583" Title="How to define Agar.io state and action space?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;game-ai&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24114" PostTypeId="1" CreationDate="2020-10-17T14:36:47.060" Score="1" ViewCount="86" Body="&lt;p&gt;I have a 2-dimentional matrix as an action space, the rows being a resource to be allocated, and the columns are the users that we will allocate the resources to. (I built my own RL environment)&lt;/p&gt;&#xA;&lt;p&gt;The possible actions are 'Zero' or 'One'. One if the resource was allocated to the user, Zero if not.&lt;/p&gt;&#xA;&lt;p&gt;I have a constraint related to the resource allocation, which states that each resource can be allocated to one user only, and the resource should only be allocated to users who have requested a resource to be allocated to them, and that would be the state space which is another matrix.&lt;/p&gt;&#xA;&lt;p&gt;A penalty would be applied if the agent violates the constraints and the episode would end and the reward would equal the penalty. Otherwise, the reward would equal the sum of all the users that were satisfied with the allocation.&lt;/p&gt;&#xA;&lt;p&gt;I am struggling with the implementation. The agent starts by exploring, then little by little it starts exploiting. When it gets to be more exploitative, I've noticed that the action matrix's values are all set to 'One', and the penalty always has the same value from episode to episode.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastActivityDate="2021-03-17T01:04:30.677" Title="How to create a Q-Learning agent when we have a matrix as an action space?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24115" PostTypeId="2" ParentId="24068" CreationDate="2020-10-17T14:46:04.717" Score="1" Body="&lt;p&gt;&lt;strong&gt;Context of the question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a &lt;a href=&quot;https://livebook.manning.com/book/deep-learning-with-python/chapter-7/v-6/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to the text cited in the question.&lt;/p&gt;&#xA;&lt;p&gt;It refers to the usage of SeparableConv2D (tf, keras name). A related question on StackOverflow is &amp;quot;&lt;a href=&quot;https://stackoverflow.com/questions/54705367/what-is-the-difference-between-separableconv2d-and-conv2d-layers&quot;&gt;What is the difference between SeparableConv2D and Conv2D layers&lt;/a&gt;&amp;quot;. This answer points to this excellent article by Chi-Feng Wang:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Basic Introduction to Separable Convolutions&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Answer to the question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In image processing, a separable convolution converts a NxM convolution to two convolutions with kernels Nx1 and 1xM. Using this idea, in NN a SeparableConv2D converts a WxHxD convolution (width x height x depth, where depth means number of incoming features ) to two convolutions with kernels WxHx1 and 1x1xD.&lt;/p&gt;&#xA;&lt;p&gt;Note the first kernel doesn't handles information across features, thus, it is &amp;quot;learning of spatial features&amp;quot;. The 1x1xD kernel doesn't handles different points, it is &amp;quot;learning of channel-wise features&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;About the phrase &amp;quot;spatial locations in the input are highly correlated&amp;quot;, my understanding of what the author means is: Assume we have a channel (feature) image that each pixel measures the &amp;quot;distance to the background&amp;quot;. When we pass from one pixel to a neighbors one, it is expected some continuity in the value (except for edge pixels): correlation. Instead, if we have a channel that measures &amp;quot;brightness&amp;quot; and another one that measures &amp;quot;distance to background&amp;quot; the two values for one specific pixel has little correlation.&lt;/p&gt;&#xA;&lt;p&gt;Finally, about title question &amp;quot;When should we use separable convolution?&amp;quot; : if the final output must depend of some features of one pixel and some other features of neighbors pixels in a very unpredictable way, a complete WxHxD convolution must be used. However if, as is more usual, you can handle first spatial dependencies (neighborhood) to extract pixel features and next handle pixel-by-pixel these features to get the output, better use a WxHx1 followed by 1x1xD, saving lots of network parameters, thus, saving training time.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-18T15:03:32.537" LastActivityDate="2020-10-18T15:03:32.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24117" PostTypeId="1" AcceptedAnswerId="24154" CreationDate="2020-10-17T19:33:04.053" Score="6" ViewCount="134" Body="&lt;p&gt;The main goal is: &lt;em&gt;Find the smallest possible neural network to approximate the &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt; function.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I want to find a qualitative reason why this network is the smallest possible network.&lt;/p&gt;&#xA;&lt;p&gt;I have created 8000 random &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; values with corresponding target values &lt;span class=&quot;math-container&quot;&gt;$sin(x)$&lt;/span&gt;. The network, which am currently considering, consists of 1 input neuron, 3 neurons in two hidden layers, and 1 output neuron:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Network architecture:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MBElA.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MBElA.png&quot; alt=&quot;Network architecture&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The neural network can be written as function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y = sig(w_3 \cdot sig(w_1 \cdot x) + w_4 \cdot sig(w_2 \cdot x)),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\text{sig}$&lt;/span&gt; is the sigmoid activation function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; activation function:&lt;/strong&gt;&lt;br /&gt;&#xA;When I use &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; as an activation function, the network is able to hit the 2 extrema of the &lt;span class=&quot;math-container&quot;&gt;$sin$&lt;/span&gt; function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tvpYv.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tvpYv.png&quot; alt=&quot;*&lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; activation function&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Sigmoid activation function:&lt;/strong&gt;&lt;br /&gt;&#xA;However, when I use the sigmoid activation function &lt;span class=&quot;math-container&quot;&gt;$\text{sig}$&lt;/span&gt;, only the first extremum is hit. The network output is not a periodic function but converges:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wVnzq.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wVnzq.png&quot; alt=&quot;Sigmoid activation function&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My questions are now:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why does one get a better approximation with the &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; activation function? What is a qualitative argument for that?&lt;/li&gt;&#xA;&lt;li&gt;Why does one need at least 3 hidden neurons? What is the reason that the approximation with &lt;span class=&quot;math-container&quot;&gt;$tanh$&lt;/span&gt; does not work anymore, if one uses only 2 hidden neurons?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I really appreciate all your ideas on this problem!&lt;/p&gt;&#xA;" OwnerUserId="23496" LastActivityDate="2020-10-20T06:34:46.397" Title="Smallest possible network to approximate the $sin$ function" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;function-approximation&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24118" PostTypeId="2" ParentId="24114" CreationDate="2020-10-18T00:55:31.130" Score="0" Body="&lt;p&gt;I was thinking this strategy may work.&lt;/p&gt;&#xA;&lt;p&gt;So, Q-learning takes vector input as state representation let's say your vector has n dimensions i.e. [&lt;span class=&quot;math-container&quot;&gt;$n_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n_2$&lt;/span&gt;,..., &lt;span class=&quot;math-container&quot;&gt;$n_{n-1}$&lt;/span&gt;]&lt;/p&gt;&#xA;&lt;p&gt;Now, from my interpretation you want to populate a matrix with 0 and 1's given the state vector but action-space has a high complexity e.g. an 8*8 matrix has 64 cells i.e. &lt;span class=&quot;math-container&quot;&gt;$2^{64}$&lt;/span&gt; possible actions if you want the action to be a matrix.&lt;/p&gt;&#xA;&lt;p&gt;I suggest this:&lt;/p&gt;&#xA;&lt;p&gt;Fill each cell, one at a time. i.e. Your agent has only two possible action 0 and 1.&#xA;To indicate to your agent that you are at a specific cell, concatenate the row and column number to the state vector before passing it as input to the Q-learning agent.&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;p&gt;If you original state-vector is [55, 22, 100, 4] and you have to fill cell at position (10, 30) of the matrix, the state-vector should be modified as follows:&#xA;[55, 22, 100, 4, 10, 30].&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure of the sample-efficiency of this approach though.&lt;/p&gt;&#xA;" OwnerUserId="30174" LastActivityDate="2020-10-18T00:55:31.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24119" PostTypeId="1" CreationDate="2020-10-18T00:58:56.423" Score="0" ViewCount="29" Body="&lt;p&gt;Old Photo Restoration via Deep Latent Space Translation (&lt;a href=&quot;https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;In the article, it says : &amp;quot;&lt;em&gt;We propose to restore old photos that suffer from severe degradation through a deep learning approach. Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize ...&lt;/em&gt; &amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I have a bit of difficulties to understand how they built their dataset of photos. At the beginning, I thought the idea was to create synthetic old photos from a set modern photos like &lt;a href=&quot;https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IMDB-WIKI – 500k+ face images with age and gender labels&lt;/a&gt;. Hence, I thought the idea was to use deep learning with the synthetic photos dataset (artificially degraded) as input and the modern photos as target. Once the model is train, we could've use it with real old photos instead of artificially build photos. It seems not the way to go. Can you explain to me how they generate their dataset and use it in their VAEs?&lt;/p&gt;&#xA;" OwnerUserId="41202" LastEditorUserId="41202" LastEditDate="2020-10-18T18:15:36.057" LastActivityDate="2020-10-18T18:15:36.057" Title="How did they use their dataset with VAEs?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24121" PostTypeId="1" CreationDate="2020-10-18T02:35:32.887" Score="1" ViewCount="127" Body="&lt;p&gt;I'm trying to have a simple autoencoder but with variable latent length (the network can produce variable latent lengths with respect to the complexity of the input), but I've not seen any related work to get idea from. Have you seen any related work? Do you have any idea to do so?&lt;/p&gt;&#xA;&lt;p&gt;Actually, I want to use this autoencoder for transmitting the data over a noisy channel, so having a variable-length may help.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastEditorUserId="2444" LastEditDate="2020-10-18T09:58:59.087" LastActivityDate="2020-10-18T20:30:46.737" Title="Is it possible to have a variable-length latent vector in an autoencoder?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;autoencoders&gt;&lt;latent-variable&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24123" PostTypeId="2" ParentId="23559" CreationDate="2020-10-18T05:05:19.943" Score="1" Body="&lt;h2&gt;Art of NPC creation&lt;/h2&gt;&#xA;&lt;p&gt;I'm assuming this is a standard game, not a game theory application. These AIs tend to be far simpler in theory than actual artificially intelligent agents which are used to solve real world problems. The challenge of building games is that there are few right answers. A game NPC opponent could know practically nothing of the player and be guided by random world exploration activating certain characteristics (attack, strategy) whenever a human is in range or on the other end could know basically everything and have some limiting factors built into the code which reduces their challenge level to the player. This is the art aspect of gaming design. NPCs with a high level of complexity could learn from their interactions and build a &amp;quot;pattern&amp;quot; with some internal reward scoring system for success (similar to reinforced learning). This would represent a very challenging game NPC, especially if the strategy changes upon failure. Think about who will be playing your games, the audience that you would tend to draw, and hone the NPC challengers to them. Hardcore gamers deserve a challenge, and hobby gamers deserve a chance to win. Where that balance between rewarding game experience and challenging game experience lies depends on the 'feel' which you are cultivating as the game designer. You mentioned resources and exploration as elements of the game. For exploration types of games, you may rely on environmental interactions to offset player interactions. A distracted NPC (gathering resources) is less likely to heavily attack the human player unless this player enters the perimeters of the NPC base or scouting area. This may work to keep the gameplay balanced. Keep in mind that some limitations for your NPCs must be absolute, or the game will experience unexpected behavior (bugs). A completely unrestricted NPC is a liability. If the NPC goes too far from the player or gets stuck, it completely ruins the feel of your game.&lt;/p&gt;&#xA;" OwnerUserId="16959" LastActivityDate="2020-10-18T05:05:19.943" CommentCount="0" CommunityOwnedDate="2020-10-18T05:05:19.943" ContentLicense="CC BY-SA 4.0" />
  <row Id="24124" PostTypeId="2" ParentId="24112" CreationDate="2020-10-18T05:46:58.583" Score="3" Body="&lt;p&gt;The state space is certainly continuous, assuming that you can somehow feed that AI exact coordinates. You may have to resort to CNNs if you do not have access to this information. For the action space, you should consider how the game actually plays. Since you use a mouse to simply show the direction, you could use (x,y) positions of the mouse as an action, or even just the angle &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; of the mouse cursor in a circle around the agent. If you are playing on the site, then your observations would have to be from a CNN, it should be possible to use the score as your reward, as well as the possibility for eating things or distance from opponents that are bigger or smaller as intermediate rewards. The number of nodes in your network is something that you must find experimentally, you may like to research what kind of architectures other people have used in this field. You shouldn't need to account for different numbers of players. Nothing special needs to be done to get a single node giving a continuous output to represent the angle &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, or alternatively two nodes representing x and y. You can then use tanh or sigmoid to limit the output node values for eject and split actions.&lt;/p&gt;&#xA;" OwnerUserId="34473" LastActivityDate="2020-10-18T05:46:58.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24125" PostTypeId="1" CreationDate="2020-10-18T05:57:43.680" Score="1" ViewCount="94" Body="&lt;p&gt;this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations may or may not be right sometimes. Please correct me if any of them are incorrect or do not make sense.&lt;/p&gt;&#xA;&lt;p&gt;So I have learned CBOW and Skip-Gram models. I think I have understood the CBOW model, and here is my interpretation: First, we fix a number of neighbors of the unknown center word which we would like to predict; let the number be &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. We then input the original characteristic vectors (vectors of zeros and ones only) of those &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; context words. By multiplying those vectors by a matrix, we obtain &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; new vectors. Next, we take the average of those &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; vectors and this is our hidden layer, namely &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;. We finally multiply &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; with another matrix, and that is the &amp;quot;empirical&amp;quot; result.&lt;/p&gt;&#xA;&lt;p&gt;I tried to follow the logic to Skip-Gram similarly, but I have been stuck. I understand that Skip-Gram is kind of a &amp;quot;reversal&amp;quot; of CBOW, but the specific steps have given me a hard time. So in Skip-Gram, we only have a center word, and based upon that we are trying to predict &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; context words. By similar steps, we obtain a hidden layer, which is again a vector. The final process also involves multiplication with a matrix, but I don't know how we can get &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; new vectors based upon one, unless we have &lt;span class=&quot;math-container&quot;&gt;$2m$&lt;/span&gt; different matrices?&lt;/p&gt;&#xA;&lt;p&gt;I appreciate any insights or answers for my question. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="41685" LastEditorUserId="36055" LastEditDate="2020-10-21T14:22:28.757" LastActivityDate="2020-10-21T14:22:28.757" Title="The mathematics in the CBOW and Skip-Gram models" Tags="&lt;natural-language-processing&gt;&lt;math&gt;&lt;word2vec&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24126" PostTypeId="2" ParentId="24121" CreationDate="2020-10-18T08:43:36.420" Score="-1" Body="&lt;p&gt;You might want to look at an encoder-decoder sequence to sequence model. This model allows you to input and output data with variable length.&lt;/p&gt;&#xA;" OwnerUserId="34358" LastActivityDate="2020-10-18T08:43:36.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24127" PostTypeId="2" ParentId="24081" CreationDate="2020-10-18T08:57:12.403" Score="0" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/abs/1611.07004&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Image-to-Image Translation with Conditional Adversarial Networks&lt;/strong&gt;&lt;/a&gt; paper (popularly known as &lt;em&gt;pix2pix&lt;/em&gt;), they used a Markovian Discriminator to effectively model the image as a Markovian Random Field.&lt;/p&gt;&#xA;&lt;p&gt;There were some papers in the last 5 years concerning Markov Fields. Here are some of them:-&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1601.04589&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1604.04382&quot; rel=&quot;nofollow noreferrer&quot;&gt;Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-10-18T08:57:12.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24128" PostTypeId="1" CreationDate="2020-10-18T10:08:27.873" Score="1" ViewCount="97" Body="&lt;p&gt;Could someone clear my doubt on the loss function used in &lt;a href=&quot;https://arxiv.org/pdf/1609.05473.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqGAN paper&lt;/a&gt; . The paper uses policy gradient method to train the generator which is a recurrent neural network here.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have I interpreted the terms correctly?&lt;/li&gt;&#xA;&lt;li&gt;What are we summing over? The entire vocabulary of words?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Loss function - my interpretation:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4ux3u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4ux3u.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41372" LastEditorUserId="41372" LastEditDate="2020-11-06T17:44:59.310" LastActivityDate="2020-11-06T17:44:59.310" Title="SeqGAN - Policy gradient objective function interpretation" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;generative-adversarial-networks&gt;&lt;reinforce&gt;&lt;seq2seq&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24129" PostTypeId="2" ParentId="24125" CreationDate="2020-10-18T10:51:41.473" Score="1" Body="&lt;p&gt;The following figure from &lt;a href=&quot;https://aegis4048.github.io/demystifying_neural_network_in_skip_gram_language_modeling&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; can be helpful:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AsvbM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AsvbM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This figure represents &amp;quot;Skip-Gram model structure. Current center word is 'passes'&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2020-10-18T10:51:41.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24130" PostTypeId="1" CreationDate="2020-10-18T11:36:46.363" Score="0" ViewCount="23" Body="&lt;p&gt;I am trying to implement a Q-learning algorithm in Python for a 3D gird world as part of an assignment, wherein the environment.py file defines the actions to be taken. So far I have tried several times, but not once do the values of the reward function in any of the iterations change. This is my latest version of the code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from collections import defaultdict&#xA;import numpy as np&#xA;import argparse&#xA;import random&#xA;&#xA;#Libraries for plotting&#xA;import matplotlib&#xA;import matplotlib.style&#xA;import plotting&#xA;&#xA;from environment import TreasureCube&#xA;&#xA;matplotlib.style.use('ggplot')&#xA;&#xA;class RandomAgent(object):&#xA;    def __init__(self):&#xA;        self.action_space = ['left','right','forward','backward','up','down'] # in TreasureCube&#xA;        self.Q = defaultdict(lambda: np.zeros(len(self.action_space)))  #Q is an array of n zeroes (here n=6).&#xA;&#xA;    def take_action(self, state):&#xA;        action = random.choice(self.action_space)&#xA;        return action&#xA;&#xA;    def train(self, state, action, next_state, reward):&#xA;        pass&#xA;&#xA;class Agent:&#xA;    def __init__ (self):&#xA;        self.action_space = ['left','right','forward','backward','up','down']&#xA;        self.n = len(self.action_space)&#xA;        self.Q = defaultdict(lambda: np.zeros(self.n))&#xA;        self.epsilon = 0.01&#xA;        self.alpha = 0.5&#xA;        self.gamma = 0.99&#xA;&#xA;    def epsilon_greedy_policy(self, Q, epsilon, n):&#xA;        def policy_function(state):&#xA;            #Selects a random action with epsilon probability out of all available actions.&#xA;            action_probabilities = np.ones(self.n, dtype=float) * self.epsilon / self.n&#xA;            &#xA;            #Selects the greedy action with (1 - epsilon) probability.&#xA;            best_action = np.argmax(self.Q[state]) &#xA;            action_probabilities[best_action] += (1.0 - self.epsilon)&#xA;            &#xA;            return action_probabilities&#xA;        return policy_function&#xA;&#xA;    def take_action(self, state, action_probabilities):&#xA;        if (np.random.random_sample() &amp;gt; self.epsilon):  #np.random.random_sample is an alias for np.random.random&#xA;            return np.argmax(self.Q[state])    #Selects the greedy action with (1 - epsilon) probability.&#xA;        else:&#xA;            return np.random.choice(np.arange(self.n))#, p=self.epsilon_greedy_policy(state))  #Selects a random action.&#xA;&#xA;    def train(self, state, action, next_state, reward):&#xA;        #Store the existing Q(s, a) values. (NOT NEEDED)&#xA;        q_old = self.Q[state][action]&#xA;        best_next_action = np.argmax(self.Q[next_state])&#xA;&#xA;        #Retrieve the expected value for next state (or 0 if no next state, i.e., end of episode)&#xA;        q_next = self.Q[next_state][best_next_action] if next_state is not None else 0&#xA;&#xA;        #Update the Q-table.&#xA;        self.Q[state][action] += self.alpha * ((reward + self.gamma * q_next) - q_old)&#xA;&#xA;        #print(self.Q[state][action])&#xA;&#xA;def test_cube(max_episode, max_step, gamma = 0.99, alpha = 0.5, epsilon = 0.01, n = 6):&#xA;    env = TreasureCube(max_step=max_step)&#xA;    agent = Agent()&#xA;    stats = plotting.EpisodeStats(episode_lengths = np.zeros(n), episode_rewards = np.zeros(n)) &#xA;    policy = agent.epsilon_greedy_policy(agent.Q, epsilon, n) #Declaring the policy before the loop.&#xA;&#xA;    for episode_num in range(0, max_episode):&#xA;        state = env.reset()&#xA;        terminate = False&#xA;        t = 0&#xA;        episode_reward = 0&#xA;        while not terminate:&#xA;            action_probabilities = policy(state)&#xA;            action = agent.take_action(state, action_probabilities)&#xA;            reward, terminate, next_state = env.step(action)&#xA;            episode_reward += reward&#xA;            stats.episode_rewards[episode_num] = episode_reward&#xA;            #you can comment the following two lines, if the output is too much&#xA;            #env.render() # comment&#xA;            #print(f'step: {t}, action: {action}, reward: {reward}') # comment&#xA;            agent.train(state, action, next_state, reward)&#xA;            state = next_state&#xA;            t += 1&#xA;            stats.episode_lengths[episode_num] = t&#xA;&#xA;        print(f'episode: {episode_num}, total_steps: {t} episode reward: {episode_reward}')&#xA;&#xA;    return stats&#xA;&#xA;if __name__ == '__main__':&#xA;    parser = argparse.ArgumentParser(description='Test')&#xA;    parser.add_argument('--max_episode', type=int, default=500)&#xA;    parser.add_argument('--max_step', type=int, default=500)&#xA;    args = parser.parse_args()&#xA;&#xA;    test_cube(args.max_episode, args.max_step)&#xA;    #plotting.plot_episode_stats(stats) &#xA;&#xA;    #Code to run: python test3.py --max_episode 50 --max_step 50&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The environment.py file is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import time&#xA;import random&#xA;&#xA;from abc import ABC, abstractmethod&#xA;&#xA;class AbstractEnvironment(ABC):&#xA;    def __init__(self):&#xA;        self.agent_sign = '+'&#xA;        self.goal_sign = 'G'&#xA;        self.corridor_sign = '-'&#xA;&#xA;    def render(self):&#xA;        raise NotImplemented&#xA;&#xA;    def reset(self):&#xA;        raise NotImplemented&#xA;&#xA;    def step(self, action):&#xA;        raise NotImplemented&#xA;&#xA;class TreasureCube(AbstractEnvironment):&#xA;    def __init__(self, max_step=20):&#xA;        super(TreasureCube, self).__init__()&#xA;        self.dim = 4&#xA;        self.max_step = max_step&#xA;        self.curr_pos = [0, 0, 0]  # (z, x, y)&#xA;        self.time_step = 0&#xA;        self.end_pos = [self.dim - 1, self.dim - 1, self.dim - 1]&#xA;        self.visual_state = []&#xA;        self.seed = None&#xA;        self.set_seed()&#xA;        self.all_actions = ['right', 'left', 'up', 'down', 'forward', 'backward']&#xA;        self.slip_actions = dict()&#xA;        self.set_action_rules()&#xA;&#xA;    def reset(self):&#xA;        self.curr_pos = [0, 0, 0]&#xA;        self.time_step = 0&#xA;        self.end_pos = [self.dim - 1, self.dim - 1, self.dim - 1]&#xA;        self._reset_visual_states(self.curr_pos, self.end_pos)&#xA;        return ''.join(str(pos) for pos in self.curr_pos)&#xA;&#xA;    def step(self, action, stochastic=True):&#xA;        in_action = action  # action from agent&#xA;        #assert action in self.all_actions&#xA;        reward = -0.1&#xA;        is_terminate = False&#xA;        #pre_pos = self.curr_pos&#xA;        r = random.random()&#xA;        if action == 'right':&#xA;            if r &amp;lt; 0.1:&#xA;                action = 'up'&#xA;            elif r &amp;lt; 0.2:&#xA;                action = 'down'&#xA;            elif r &amp;lt; 0.3:&#xA;                action = 'forward'&#xA;            elif r &amp;lt; 0.4:&#xA;                action = 'backward'&#xA;            else:&#xA;                action = 'right'&#xA;        elif action == 'left':&#xA;            if r &amp;lt; 0.1:&#xA;                action = 'up'&#xA;            elif r &amp;lt; 0.2:&#xA;                action = 'down'&#xA;            elif r &amp;lt; 0.3:&#xA;                action = 'forward'&#xA;            elif r &amp;lt; 0.4:&#xA;                action = 'backward'&#xA;            else:&#xA;                action = 'left'&#xA;        elif action == 'up':&#xA;            if r &amp;lt; 0.1:&#xA;                action = 'left'&#xA;            elif r &amp;lt; 0.2:&#xA;                action = 'right'&#xA;            elif r &amp;lt; 0.3:&#xA;                action = 'forward'&#xA;            elif r &amp;lt; 0.4:&#xA;                action = 'backward'&#xA;            else:&#xA;                action = 'up'&#xA;        elif action == 'down':&#xA;            if r &amp;lt; 0.1:&#xA;                action = 'left'&#xA;            elif r &amp;lt; 0.2:&#xA;                action = 'right'&#xA;            elif r &amp;lt; 0.3:&#xA;                action = 'forward'&#xA;            elif r &amp;lt; 0.4:&#xA;                action = 'backward'&#xA;            else:&#xA;                action = 'down'&#xA;        elif action == 'forward':&#xA;            if r &amp;lt; 0.1:&#xA;                action = 'left'&#xA;            elif r &amp;lt; 0.2:&#xA;                action = 'right'&#xA;            elif r &amp;lt; 0.3:&#xA;                action = 'up'&#xA;            elif r &amp;lt; 0.4:&#xA;                action = 'down'&#xA;            else:&#xA;                action = 'forward'&#xA;        else:&#xA;            if r &amp;lt; 0.1:&#xA;                action = 'left'&#xA;            elif r &amp;lt; 0.2:&#xA;                action = 'right'&#xA;            elif r &amp;lt; 0.3:&#xA;                action = 'up'&#xA;            elif r &amp;lt; 0.4:&#xA;                action = 'down'&#xA;            else:&#xA;                action = 'backward'&#xA;&#xA;        if not stochastic:&#xA;            action = in_action&#xA;&#xA;        assert action in self.all_actions&#xA;        if action == 'left':&#xA;            if self.curr_pos[1] == 0:  # wall&#xA;                pass&#xA;            else:&#xA;                self.curr_pos[1] -= 1&#xA;        elif action == 'right':&#xA;            if self.curr_pos[1] == self.dim - 1:  # wall&#xA;                pass&#xA;            elif self.curr_pos[1] == self.dim - 2 and self.curr_pos[0] == self.dim - 1 and self.curr_pos[2] == self.dim - 1:&#xA;                self.curr_pos[1] += 1&#xA;                is_terminate = True&#xA;                reward = 1&#xA;            else:&#xA;                self.curr_pos[1] += 1&#xA;&#xA;        elif action == 'forward':&#xA;            if self.curr_pos[0] == self.dim - 1:  # wall&#xA;                pass&#xA;            elif self.curr_pos[0] == self.dim - 2 and self.curr_pos[1] == self.dim - 1 and self.curr_pos[2] == self.dim - 1:&#xA;                self.curr_pos[0] += 1&#xA;                is_terminate = True&#xA;                reward = 1&#xA;            else:&#xA;                self.curr_pos[0] += 1&#xA;        elif action == 'backward':&#xA;            if self.curr_pos[0] == 0:  # wall&#xA;                pass&#xA;            else:&#xA;                self.curr_pos[0] -= 1&#xA;&#xA;        elif action == 'up':&#xA;            if self.curr_pos[2] == self.dim - 1:  # wall&#xA;                pass&#xA;            elif self.curr_pos[2] == self.dim - 2 and self.curr_pos[0] == self.dim - 1 and self.curr_pos[1] == self.dim - 1:&#xA;                self.curr_pos[2] += 1&#xA;                is_terminate = True&#xA;                reward = 1&#xA;            else:&#xA;                self.curr_pos[2] += 1&#xA;        elif action == 'down':&#xA;            if self.curr_pos[2] == 0:&#xA;                pass&#xA;            else:&#xA;                self.curr_pos[2] -= 1&#xA;&#xA;        assert action in self.all_actions&#xA;        self.time_step += 1&#xA;        if self.time_step == self.max_step - 1:&#xA;            is_terminate = True&#xA;&#xA;        self._reset_visual_states(self.curr_pos, self.end_pos)&#xA;        return reward, is_terminate, ''.join(str(pos) for pos in self.curr_pos)&#xA;&#xA;    def render(self):&#xA;        print(' '.join(['*'] * self.dim))&#xA;        for i in range(self.dim):&#xA;            for line in self.visual_state[i]:&#xA;                print(' '.join(line))&#xA;            print(' '.join(['#'] * self.dim))&#xA;        print(' '.join(['*'] * self.dim))&#xA;&#xA;    def set_seed(self, seed=10086):&#xA;        self.seed = seed&#xA;        random.seed(seed)&#xA;&#xA;    def _reset_visual_states(self, agent_pos, goal_pos):&#xA;        self.visual_state = [[[self.corridor_sign] * self.dim for _ in range(self.dim)] for _ in range(self.dim)]&#xA;        self.visual_state[agent_pos[0]][agent_pos[1]][agent_pos[2]] = self.agent_sign&#xA;        self.visual_state[goal_pos[0]][goal_pos[1]][goal_pos[2]] = self.goal_sign&#xA;&#xA;    def set_action_rules(self):&#xA;        self.slip_actions['right'] = ['up', 'down', 'forward', 'backward', 'right']&#xA;        self.slip_actions['left'] = ['up', 'down', 'forward', 'backward', 'left']&#xA;        self.slip_actions['up'] = ['left', 'right', 'forward', 'backward', 'up']&#xA;        self.slip_actions['down'] = ['left', 'right', 'forward', 'backward', 'down']&#xA;        self.slip_actions['forward'] = ['left', 'right', 'up', 'down', 'forward']&#xA;        self.slip_actions['backward'] = ['left', 'right', 'up', 'down', 'backward']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is as follows. With 50 iterations, -4.8999..., with 100 iterations, -9.8999..., and so on. This is where I do not know what to fix. I am sure I am going wrong somewhere, but where I do not know.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python test3.py --max_episode 50 --max_step 50&#xA;episode: 0, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 1, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 2, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 3, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 4, total_steps: 49 episode reward: -4.899999999999999&#xA;...&#xA;episode: 45, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 46, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 47, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 48, total_steps: 49 episode reward: -4.899999999999999&#xA;episode: 49, total_steps: 49 episode reward: -4.899999999999999&#xA;&lt;span class=&quot;math-container&quot;&gt;````&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41686" LastActivityDate="2020-10-18T11:36:46.363" Title="Q-learning in Python (3D grid navigation): Values do not change between episodes" Tags="&lt;python&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24132" PostTypeId="2" ParentId="23700" CreationDate="2020-10-18T16:22:11.060" Score="0" Body="&lt;p&gt;You could use Ray RLlib. It has support for parallel environments, even over multiple GPUs and compute nodes.&lt;/p&gt;&#xA;" OwnerUserId="34358" LastActivityDate="2020-10-18T16:22:11.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24133" PostTypeId="2" ParentId="23666" CreationDate="2020-10-18T16:30:36.467" Score="0" Body="&lt;p&gt;Since you are looking at a single iteration and expect a meaningful change my guess is that you aren't training for long enough. Q-learning can take very long, for many environments it takes millions of iterations.&lt;/p&gt;&#xA;" OwnerUserId="34358" LastActivityDate="2020-10-18T16:30:36.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24134" PostTypeId="2" ParentId="23640" CreationDate="2020-10-18T16:40:40.227" Score="0" Body="&lt;p&gt;You can modify the environments and create custom models using Ray RLlib: &lt;a href=&quot;https://docs.ray.io/en/latest/rllib.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.ray.io/en/latest/rllib.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34358" LastActivityDate="2020-10-18T16:40:40.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24135" PostTypeId="2" ParentId="24121" CreationDate="2020-10-18T17:37:37.050" Score="1" Body="&lt;p&gt;If you use RNNs, then I think the solution is to use padding (zero padding) with max sequence length (that is the max number of words in a text) in order to tell your model to skip the zeros when possible. In that way, your model will try to learn a good representation of your input with fixed size. If you do not know this dimension, a solution may be to grid search this hyperparameter.&lt;/p&gt;&#xA;&lt;p&gt;If you still want to exploit the dimensionality difference, maybe you can train different models with fixed size dimension of the representation dependently of the dimension of the input. That is, for example, use one for small, one for medium and one for large dimensions, but this should surely require to have a large and quite balanced initial dataset.&lt;/p&gt;&#xA;&lt;p&gt;Another idea could be to use the autoencoder with a fixed latent dimension. Then, do effective clustering on your samples using their latent representation, considering that similar representations should have similar dimensionality requirements (?). After that, you could train your initial dataset on k models, the same number as the clusters. That is, there should be k different latent spaces. The goal is to match each instance to the correct model. At first, you should train them all with each instance, but as the training progresses, you should maybe go with binary search for each instance in order it to find the correct model, assuming that there is total order in the measuring of the dimensionality requirements. Of course, this is just an idea, I don't know if it is going to be really helpful at all.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="36055" LastEditDate="2020-10-18T20:30:46.737" LastActivityDate="2020-10-18T20:30:46.737" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24136" PostTypeId="1" AcceptedAnswerId="24137" CreationDate="2020-10-18T18:22:29.917" Score="0" ViewCount="83" Body="&lt;p&gt;It seems like transfer learning is only applicable to neural networks. Is this a correct assumption?&lt;/p&gt;&#xA;&lt;p&gt;While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio data, or text data. I was not able to find an example of training a neural network on numerical data.&lt;/p&gt;&#xA;&lt;p&gt;I want to use transfer learning in this particular scenario: I have a lot of numerical data from an old environment, with binary classification labels, and utilizing this, want to train on a new environment to do the same binary classification.&lt;/p&gt;&#xA;&lt;p&gt;The dataset would look something like this &lt;a href=&quot;https://i.stack.imgur.com/1CKRY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1CKRY.png&quot; alt=&quot;Sample Table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is this possible? What would the model look like?&lt;/p&gt;&#xA;" OwnerUserId="41693" LastActivityDate="2020-10-18T19:09:59.280" Title="Transfer Learning of Numerical Data" Tags="&lt;transfer-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24137" PostTypeId="2" ParentId="24136" CreationDate="2020-10-18T19:03:02.283" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;It seems like transfer learning is only applicable to neural networks. Is this a correct assumption?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. &lt;a href=&quot;https://en.wikipedia.org/wiki/Transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wiki&lt;/a&gt; page give you pointers of several examples in other methodologies.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio data, or text data. I was not able to find an example of training a neural network on numerical data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All the cases you say are converted to numerical data. Image and audio usually via sampling, text via one-hot encoding.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I want to use transfer learning in this particular scenario: I have a lot of numerical data from an old environment, with binary classification labels, and utilizing this, want to train on a new environment to do the same binary classification.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That is not transfer learning. Transfer learning applies when there are a change in the domain (input features) or in the task (output labels).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The dataset would look something like this Sample Table&#xA;Is this possible? What would the model look like?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For a simple case as the one you present, probably a simple network with one hidden layer will be enough. Train it with original pairs of {features,label} or, if not available, use the current predictor to obtain the label from the features.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-18T19:09:59.280" LastActivityDate="2020-10-18T19:09:59.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24138" PostTypeId="2" ParentId="12870" CreationDate="2020-10-18T21:55:15.390" Score="22" Body="&lt;p&gt;&lt;em&gt;Explainable AI&lt;/em&gt; and &lt;em&gt;model interpretability&lt;/em&gt; are hyper-active and hyper-hot areas of current research (think of holy grail, or something), which have been brought forward lately not least due to the (often tremendous) success of deep learning models in  various tasks, plus the necessity of algorithmic fairness &amp;amp; accountability.&lt;/p&gt;&#xA;&lt;p&gt;Here are some state of the art algorithms and approaches, together with implementations and frameworks.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Model-agnostic approaches&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;LIME: Local Interpretable Model-agnostic Explanations (&lt;a href=&quot;https://arxiv.org/abs/1602.04938&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/marcotcr/lime&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, &lt;a href=&quot;https://cran.r-project.org/web/packages/lime/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R port&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SHAP: A Unified Approach to Interpreting Model Predictions (&lt;a href=&quot;https://arxiv.org/abs/1705.07874&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/slundberg/shap&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python package&lt;/a&gt;, &lt;a href=&quot;https://github.com/redichh/ShapleyR&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package&lt;/a&gt;). GPU implementation for tree models by NVIDIA using RAPIDS - GPUTreeShap (&lt;a href=&quot;https://arxiv.org/abs/2010.13972&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/rapidsai/gputreeshap&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://developer.nvidia.com/blog/explaining-and-accelerating-machine-learning-for-loan-delinquencies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Anchors: High-Precision Model-Agnostic Explanations (&lt;a href=&quot;https://homes.cs.washington.edu/%7Emarcotcr/aaai18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/marcotcr/anchor&quot; rel=&quot;nofollow noreferrer&quot;&gt;authors' Python code&lt;/a&gt;, &lt;a href=&quot;https://github.com/viadee/javaAnchorExplainer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java&lt;/a&gt; implementation)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Diverse Counterfactual Explanations (DiCE) by Microsoft (&lt;a href=&quot;https://arxiv.org/abs/1905.07697&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/microsoft/dice&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://www.microsoft.com/en-us/research/blog/open-source-library-provides-explanation-for-machine-learning-through-diverse-counterfactuals/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1602.07043&quot; rel=&quot;nofollow noreferrer&quot;&gt;Black Box Auditing&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1412.3756&quot; rel=&quot;nofollow noreferrer&quot;&gt;Certifying and Removing Disparate Impact&lt;/a&gt; (authors' &lt;a href=&quot;https://github.com/algofairness/BlackBoxAuditing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;FairML: Auditing Black-Box Predictive Models, by Cloudera Fast Forward Labs (&lt;a href=&quot;http://blog.fastforwardlabs.com/2017/03/09/fairml-auditing-black-box-predictive-models.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1611.04967&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/adebayoj/fairml&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;SHAP seems to enjoy &lt;a href=&quot;https://twitter.com/christophmolnar/status/1162460768631177216?s=11&quot; rel=&quot;nofollow noreferrer&quot;&gt;high popularity&lt;/a&gt; among practitioners; the method has firm theoretical foundations on co-operational game theory (Shapley values), and it has in a great degree integrated the LIME approach under a common framework. Although model-agnostic, specific &amp;amp; efficient implementations are available for neural networks (&lt;a href=&quot;https://shap.readthedocs.io/en/latest/generated/shap.DeepExplainer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepExplainer&lt;/a&gt;) and tree ensembles (&lt;a href=&quot;https://shap.readthedocs.io/en/latest/generated/shap.TreeExplainer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TreeExplainer&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/s42256-019-0138-9.epdf?shared_access_token=RCYPTVkiECUmc0CccSMgXtRgN0jAjWel9jnR3ZoTv0O81kV8DqPb2VXSseRmof0Pl8YSOZy4FHz5vMc3xsxcX6uT10EzEoWo7B-nZQAHJJvBYhQJTT1LnJmpsa48nlgUWrMkThFrEIvZstjQ7Xdc5g%3D%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Neural network approaches&lt;/strong&gt; (mostly, but not exclusively, for computer vision models)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The Layer-wise Relevance Propagation (LRP) toolbox for neural networks (&lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140&quot; rel=&quot;nofollow noreferrer&quot;&gt;2015 paper @ PLoS ONE&lt;/a&gt;, &lt;a href=&quot;http://www.jmlr.org/papers/v17/15-618.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2016 paper @ JMLR&lt;/a&gt;, &lt;a href=&quot;http://heatmapping.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://github.com/sebastian-lapuschkin/lrp_toolbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://github.com/VigneshSrinivasan10/interprettensor&quot; rel=&quot;nofollow noreferrer&quot;&gt;TF Slim wrapper&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization (&lt;a href=&quot;https://arxiv.org/abs/1610.02391&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, authors' &lt;a href=&quot;https://github.com/ramprs/grad-cam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Torch code&lt;/a&gt;, &lt;a href=&quot;https://github.com/Ankush96/grad-cam.tensorflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow code&lt;/a&gt;, &lt;a href=&quot;https://github.com/meliketoy/gradcam.pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch code&lt;/a&gt;, yet another &lt;a href=&quot;https://github.com/jacobgil/pytorch-grad-cam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pytorch implementation&lt;/a&gt;, Keras &lt;a href=&quot;http://nbviewer.jupyter.org/github/fchollet/deep-learning-with-python-notebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;example notebook&lt;/a&gt;, Coursera &lt;a href=&quot;https://www.coursera.org/projects/scene-classification-gradcam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Guided Project&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Axiom-based Grad-CAM (XGrad-CAM): Towards Accurate Visualization and Explanation of CNNs, a refinement of the existing Grad-CAM method (&lt;a href=&quot;https://arxiv.org/abs/2008.02312&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/Fu0511/XGrad-CAM&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability (&lt;a href=&quot;https://arxiv.org/abs/1706.05806&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/google/svcca&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://ai.googleblog.com/2017/11/interpreting-deep-neural-networks-with.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TCAV: Testing with Concept Activation Vectors (&lt;a href=&quot;https://arxiv.org/abs/1711.11279&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICML 2018 paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/tensorflow/tcav&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Integrated Gradients (&lt;a href=&quot;https://arxiv.org/abs/1703.01365&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/ankurtaly/Integrated-Gradients&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://www.tensorflow.org/tutorials/interpretability/integrated_gradients&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensorflow tutorial&lt;/a&gt;, &lt;a href=&quot;https://paperswithcode.com/paper/axiomatic-attribution-for-deep-networks#code&quot; rel=&quot;nofollow noreferrer&quot;&gt;independent implementations&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Network Dissection: Quantifying Interpretability of Deep Visual Representations, by MIT CSAIL (&lt;a href=&quot;http://netdissect.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://github.com/CSAILVision/NetDissect&quot; rel=&quot;nofollow noreferrer&quot;&gt;Caffe code&lt;/a&gt;, &lt;a href=&quot;https://github.com/CSAILVision/NetDissect-Lite&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch port&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GAN Dissection: Visualizing and Understanding Generative Adversarial Networks, by MIT CSAIL (&lt;a href=&quot;https://gandissect.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, with links to paper &amp;amp; code)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explain to Fix: A Framework to Interpret and Correct DNN Object Detector Predictions (&lt;a href=&quot;https://arxiv.org/abs/1811.08011&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/gudovskiy/e2x&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transparecy-by-Design (TbD) networks (&lt;a href=&quot;https://arxiv.org/abs/1803.05268&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/davidmascharka/tbd-nets&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://mybinder.org/v2/gh/davidmascharka/tbd-nets/binder?filepath=full-vqa-example.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;demo&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1711.09784&quot; rel=&quot;nofollow noreferrer&quot;&gt;Distilling a Neural Network Into a Soft Decision Tree&lt;/a&gt;, a 2017 paper by Geoff Hinton, with various independent &lt;a href=&quot;https://paperswithcode.com/paper/distilling-a-neural-network-into-a-soft&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch implementations&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Understanding Deep Networks via Extremal Perturbations and Smooth Masks (&lt;a href=&quot;https://arxiv.org/abs/1910.08485&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;), implemented in &lt;a href=&quot;https://github.com/facebookresearch/torchray&quot; rel=&quot;nofollow noreferrer&quot;&gt;TorchRay&lt;/a&gt; (see below)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Understanding the Role of Individual Units in a Deep Neural Network (&lt;a href=&quot;https://arxiv.org/abs/2009.05041&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;, &lt;a href=&quot;https://www.pnas.org/content/early/2020/08/31/1907375117&quot; rel=&quot;nofollow noreferrer&quot;&gt;2020 paper @ PNAS&lt;/a&gt;, &lt;a href=&quot;https://github.com/davidbau/dissect&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://dissect.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GNNExplainer: Generating Explanations for Graph Neural Networks (&lt;a href=&quot;https://proceedings.neurips.cc/paper/2019/file/d80b7040b773199015de6d3b4293c8ff-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/RexYing/gnn-model-explainer&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Benchmarking Deep Learning Interpretability in Time Series Predictions (&lt;a href=&quot;https://proceedings.neurips.cc//paper/2020/file/47a3893cc405396a5c30d91320572d6d-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; @ NeurIPS 2020, &lt;a href=&quot;https://github.com/ayaabdelsalam91/TS-Interpretability-Benchmark&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; utilizing &lt;a href=&quot;https://captum.ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;Captum&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Concept Whitening for Interpretable Image Recognition (&lt;a href=&quot;https://www.nature.com/articles/s42256-020-00265-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/2002.01650&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;, &lt;a href=&quot;https://github.com/zhiCHEN96/ConceptWhitening&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Libraries &amp;amp; frameworks&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As interpretability moves toward the mainstream, there are already frameworks and toolboxes that incorporate more than one of the algorithms and techniques mentioned and linked above; here is a partial list:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The ELI5 Python library (&lt;a href=&quot;https://github.com/TeamHG-Memex/eli5&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://eli5.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;DALEX - moDel Agnostic Language for Exploration and eXplanation (&lt;a href=&quot;https://modeloriented.github.io/DALEX/&quot; rel=&quot;nofollow noreferrer&quot;&gt;homepage&lt;/a&gt;, &lt;a href=&quot;https://github.com/ModelOriented/DALEX&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://www.jmlr.org/papers/volume19/18-416/18-416.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;JMLR paper&lt;/a&gt;), part of the &lt;a href=&quot;https://modeloriented.github.io/DrWhy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DrWhy.AI&lt;/a&gt; project&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The What-If tool by Google, a feature of the open-source TensorBoard web application, which let users analyze an ML model without writing code (&lt;a href=&quot;https://pair-code.github.io/what-if-tool/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://github.com/pair-code/what-if-tool&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://ai.googleblog.com/2018/09/the-what-if-tool-code-free-probing-of.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Language Interpretability Tool (LIT) by Google, a visual, interactive model-understanding tool for NLP models (&lt;a href=&quot;https://pair-code.github.io/lit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project page&lt;/a&gt;, &lt;a href=&quot;https://github.com/PAIR-code/lit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://ai.googleblog.com/2020/11/the-language-interpretability-tool-lit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lucid, a collection of infrastructure and tools for research in neural network interpretability by Google (&lt;a href=&quot;https://github.com/tensorflow/lucid&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;; papers: &lt;a href=&quot;https://distill.pub/2017/feature-visualization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feature Visualization&lt;/a&gt;, &lt;a href=&quot;https://distill.pub/2018/building-blocks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Building Blocks of Interpretability&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/facebookresearch/torchray&quot; rel=&quot;nofollow noreferrer&quot;&gt;TorchRay&lt;/a&gt; by Facebook, a PyTorch package implementing several visualization methods for deep CNNs&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;iNNvestigate Neural Networks (&lt;a href=&quot;https://github.com/albermax/innvestigate&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://jmlr.org/papers/v20/18-540.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;JMLR paper&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;tf-explain - interpretability methods as Tensorflow 2.0 callbacks (&lt;a href=&quot;https://github.com/sicara/tf-explain&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://tf-explain.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;, &lt;a href=&quot;https://blog.sicara.com/tf-explain-interpretability-tensorflow-2-9438b5846e35&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;InterpretML by Microsoft (&lt;a href=&quot;https://interpret.ml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;homepage&lt;/a&gt;, &lt;a href=&quot;https://github.com/Microsoft/interpret&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; still in alpha, &lt;a href=&quot;https://arxiv.org/abs/1909.09223&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Captum by Facebook AI - model interpetability for Pytorch (&lt;a href=&quot;https://captum.ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;homepage&lt;/a&gt;, &lt;a href=&quot;https://github.com/pytorch/captum&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://ai.facebook.com/blog/open-sourcing-captum-a-model-interpretability-library-for-pytorch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;intro blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Skater, by Oracle (&lt;a href=&quot;https://github.com/oracle/Skater&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://oracle.github.io/Skater/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Alibi, by SeldonIO (&lt;a href=&quot;https://github.com/SeldonIO/alibi&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://docs.seldon.io/projects/alibi/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AI Explainability 360, commenced by IBM and moved to the Linux Foundation (&lt;a href=&quot;https://ai-explainability-360.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;homepage&lt;/a&gt;, &lt;a href=&quot;https://github.com/Trusted-AI/AIX360&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://aix360.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;, &lt;a href=&quot;http://aix360.mybluemix.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM Bluemix&lt;/a&gt;, &lt;a href=&quot;https://www.ibm.com/blogs/research/2019/08/ai-explainability-360/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ecco: explaining transformer-based NLP models using interactive visualizations (&lt;a href=&quot;https://www.eccox.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;homepage&lt;/a&gt;, &lt;a href=&quot;https://github.com/jalammar/ecco&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, &lt;a href=&quot;https://jalammar.github.io/explaining-transformers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Recipes for Machine Learning Interpretability in H2O Driverless AI (&lt;a href=&quot;https://github.com/h2oai/driverlessai-recipes/tree/master/explainers&quot; rel=&quot;nofollow noreferrer&quot;&gt;repo&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reviews &amp;amp; general papers&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://dl.acm.org/doi/10.1145/3236009&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Methods for Explaining Black Box Models&lt;/a&gt; (2018, ACM Computing Surveys)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.pnas.org/content/116/44/22071&quot; rel=&quot;nofollow noreferrer&quot;&gt;Definitions, methods, and applications in interpretable machine learning&lt;/a&gt; (2019, PNAS)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s42256-019-0048-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead&lt;/a&gt; (2019, Nature Machine Intelligence, &lt;a href=&quot;https://arxiv.org/abs/1811.10154&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.mdpi.com/2079-9292/8/8/832&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning Interpretability: A Survey on Methods and Metrics&lt;/a&gt; (2019, Electronics)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/2009.11698&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principles and Practice of Explainable Machine Learning&lt;/a&gt; (2020, preprint)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/2010.09337&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges&lt;/a&gt; (keynote at &lt;a href=&quot;https://kdd.isti.cnr.it/xkdd2020/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2020 ECML XKDD workshop&lt;/a&gt; by Christoph Molnar, &lt;a href=&quot;https://slideslive.com/38933066/interpretable-machine-learning-state-of-the-art-and-challenges&quot; rel=&quot;nofollow noreferrer&quot;&gt;video &amp;amp; slides&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1566253519308103&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI&lt;/a&gt; (2020, Information Fusion)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/2010.10596&quot; rel=&quot;nofollow noreferrer&quot;&gt;Counterfactual Explanations for Machine Learning: A Review&lt;/a&gt; (2020, preprint, &lt;a href=&quot;https://twitter.com/yudapearl/status/1321038352355790848?s=11&quot; rel=&quot;nofollow noreferrer&quot;&gt;critique&lt;/a&gt; by Judea Pearl)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ff06-2020.fastforwardlabs.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretability 2020&lt;/a&gt;, an applied research report by Cloudera Fast Forward, updated regularly&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/Eric-Wallace/interpretability-tutorial-emnlp2020&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpreting Predictions of NLP Models&lt;/a&gt; (EMNLP 2020 tutorial)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explainable NLP Datasets (&lt;a href=&quot;https://exnlpdatasets.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/2102.12060&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;, &lt;a href=&quot;https://twitter.com/sarahwiegreffe/status/1365057786250489858&quot; rel=&quot;nofollow noreferrer&quot;&gt;highlights&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/2103.11251&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;eBooks&lt;/strong&gt; (available online)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://christophm.github.io/interpretable-ml-book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretable Machine Learning&lt;/a&gt;, by Christoph Molnar, with &lt;a href=&quot;https://github.com/christophM/iml&quot; rel=&quot;nofollow noreferrer&quot;&gt;R code&lt;/a&gt; available&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pbiecek.github.io/ema/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explanatory Model Analysis&lt;/a&gt;, by DALEX creators Przemyslaw Biecek and Tomasz Burzykowski, with both R &amp;amp; Python code snippets&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.h2o.ai/wp-content/uploads/2019/08/An-Introduction-to-Machine-Learning-Interpretability-Second-Edition.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Machine Learning Interpretability&lt;/a&gt; (2nd ed. 2019), by H2O&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Online courses &amp;amp; tutorials&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.kaggle.com/learn/machine-learning-explainability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning Explainability&lt;/a&gt;, Kaggle tutorial&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.coursera.org/projects/scene-classification-gradcam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explainable AI: Scene Classification and GradCam Visualization&lt;/a&gt;, Coursera guided project&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://Explainable%20Machine%20Learning%20with%20LIME%20and%20H2O%20in%20R&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explainable Machine Learning with LIME and H2O in R&lt;/a&gt;, Coursera guided project&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://interpretable-ml-class.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpretability and Explainability in Machine Learning&lt;/a&gt;, Harvard COMPSCI 282BR&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Other resources&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://explained.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;explained.ai&lt;/a&gt; blog&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &lt;a href=&quot;https://twitter.com/ledell/status/995930308947140608&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twitter thread&lt;/a&gt;, linking to several interpretation tools available for R&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A whole bunch of resources in the &lt;a href=&quot;https://github.com/jphall663/awesome-machine-learning-interpretability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Awesome Machine Learning Interpetability&lt;/a&gt; repo&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11539" LastEditorUserId="11539" LastEditDate="2021-04-27T10:51:20.617" LastActivityDate="2021-04-27T10:51:20.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24139" PostTypeId="2" ParentId="3981" CreationDate="2020-10-19T09:19:26.043" Score="1" Body="&lt;p&gt;There are several ways to add new classes to the trained model, which require just training for the new classes.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Incremental training (&lt;a href=&quot;https://github.com/khurramjaved96/incremental-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;continuously learn a stream of data (&lt;a href=&quot;https://github.com/creme-ml/creme&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;online machine learning (&lt;a href=&quot;https://github.com/GMvandeVen/continual-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Transfer Learning Twice&lt;/li&gt;&#xA;&lt;li&gt;Continual learning approaches (Regularization, Expansion, Rehearsal) (&lt;a href=&quot;https://github.com/facebookresearch/Adversarial-Continual-Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="41707" LastEditorUserId="2444" LastEditDate="2020-11-17T18:37:19.453" LastActivityDate="2020-11-17T18:37:19.453" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24140" PostTypeId="1" AcceptedAnswerId="24141" CreationDate="2020-10-19T10:07:15.257" Score="2" ViewCount="242" Body="&lt;p&gt;I attended an introductory class about neural network and I had a question regarding how to choose the number of hidden units per hidden layer.&lt;/p&gt;&#xA;&lt;p&gt;I remember that the Professor saying that there is no rule for choosing the number of hidden units and that having many of them along with many hidden layers can cause the network to overfit the data and under learn.&lt;/p&gt;&#xA;&lt;p&gt;However, I still have this question where assuming that we have a network with an input layer of &lt;code&gt;n&lt;/code&gt;  input nodes, a first hidden layer of &lt;code&gt;4&lt;/code&gt; hidden units, a second layer of &lt;code&gt;X&lt;/code&gt; hidden units and an output layer of &lt;code&gt;5&lt;/code&gt; units. Now if I follow the Professor's saying, it would mean that I am allowed to have &lt;code&gt;X = 3&lt;/code&gt; or &lt;code&gt;X = 4&lt;/code&gt; in layer 2.&lt;/p&gt;&#xA;&lt;p&gt;Is that actually allowed? Won't we have some sort of information gain passing from 4 (or 3) nodes to 5?  The example is illustrated below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TFOVJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TFOVJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40411" LastActivityDate="2020-10-19T10:49:18.983" Title="Can the hidden layer prior to the ouput layer have less hidden units than the output layer?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24141" PostTypeId="2" ParentId="24140" CreationDate="2020-10-19T10:25:18.987" Score="7" Body="&lt;p&gt;A layer with bigger number of nodes than previous one is something very common. Some examples are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;strategies encoder-decoder (autoencoders) where the encoder typically  has layers with a decreasing number of nodes (until the compressed/encoded data) and the decoder has layers increasing in number of nodes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;bidirectional recurrent networks where in the forward direction number nodes decreases and in the backward increases.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;generators, that from a random vector generates, by example, a full image.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As general rule: decrease number of nodes forces the net to filter/resume/abstract/summarize the internal signal information (discarding useless information or noise) while increase number of nodes means apply current information to generate an answer value for a specific question/target.&lt;/p&gt;&#xA;&lt;p&gt;Allow me a strongly simplified example: assume you want a system that, from a photo of an animal, answers the questions: number of legs? has beak ? flies ? . Net inputs are images of birds and dogs.&lt;/p&gt;&#xA;&lt;p&gt;The net architecture can have layers of decreasing size until a single node that will decide &amp;quot;is bird or dog ?&amp;quot;. From this single item of information (the only one need to answer all the questions) the output layer will have 3 nodes, each one answering one of the specific target questions: number or legs ? 4 if dog, 2 if bird, etc .&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-19T10:49:18.983" LastActivityDate="2020-10-19T10:49:18.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24143" PostTypeId="2" ParentId="8190" CreationDate="2020-10-19T12:02:33.053" Score="2" Body="&lt;p&gt;Hopfield networks, a special case of RNNs, were first proposed in 1982: &lt;a href=&quot;https://www.pnas.org/content/79/8/2554&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pnas.org/content/79/8/2554&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Otherwise (shameless plug, I am the author) a non-technical timeline for NLP can be found here: &lt;a href=&quot;https://blog.exxcellent.de/ki-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.exxcellent.de/ki-machine-learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aBMl0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aBMl0.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41709" LastActivityDate="2020-10-19T12:02:33.053" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24144" PostTypeId="1" CreationDate="2020-10-19T12:25:33.910" Score="2" ViewCount="41" Body="&lt;p&gt;I'm currently in the middle of a project (for my thesis) constructing a deep neural network. Since I'm still in the research part, I'm trying to find various ways and techniques to initialize weights. Obviously, every way will be evaluated and we will choose the one that fits best with our data set and our desired outcome.&lt;/p&gt;&#xA;&lt;p&gt;I'm all familiar with the Xavier initialization, the classic random one, the He initialization, and zeros. Searching through papers I came across the SCAWI one (&lt;a href=&quot;https://ieeexplore.ieee.org/document/143378&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistically Controlled Activation Weight Initialization&lt;/a&gt;). &lt;em&gt;If you have used this approach, how efficient is it?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;(Also, do you know any good sources to find more of these?)&lt;/p&gt;&#xA;" OwnerUserId="41488" LastEditorUserId="2444" LastEditDate="2020-10-22T09:46:59.430" LastActivityDate="2020-10-22T09:46:59.430" Title="How efficient is SCAWI weight initialization method?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24145" PostTypeId="1" CreationDate="2020-10-19T13:38:40.923" Score="1" ViewCount="182" Body="&lt;p&gt;I have been digging up of articles across the internet in context of computational complexity of GRU. Interestingly, I came across this article, &lt;a href=&quot;http://cse.iitkgp.ac.in/%7Epsraja/FNNs%20,RNNs%20,LSTM%20and%20BLSTM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cse.iitkgp.ac.in/~psraja/FNNs%20,RNNs%20,LSTM%20and%20BLSTM.pdf&lt;/a&gt;, where it takes the following notations:&lt;/p&gt;&#xA;&lt;p&gt;Let I be the number of inputs, K be the number of outputs and H be the number of cells in the hidden layer&lt;/p&gt;&#xA;&lt;p&gt;And then goes on to explain the computational complexity of FNNs, RNNs, BRNNs, LSTM and BLSTM computational complexity is O(W) i.e., the total number of edges in the network.&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For FNN: W = IH + HK ( I get this part as, for fully connected networks, we have connections from each input to each node in hidden and subsequently for hidden to output nodes)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For RNN: W = IH + &lt;strong&gt;H^2&lt;/strong&gt; + HK ( The formula is pretty same as is it for FNN &lt;strong&gt;but where does this H^2 come into picture?&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For LSTM : W = &lt;strong&gt;4&lt;/strong&gt;IH + &lt;strong&gt;4&lt;/strong&gt;H^2 + &lt;strong&gt;3&lt;/strong&gt;H + HK (&lt;strong&gt;It becomes more difficult as it comes down to LSTM as to where the 4's and 3's come into the equation?&lt;/strong&gt; )&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Continuing with these notations, can I get a similar notation for GRU as well? This can be very helpful for understanding.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="41519" LastActivityDate="2021-02-17T14:04:53.767" Title="What is the computational complexity in terms of Big-O notation of a Gated Recurrent Unit Neural network?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;computational-complexity&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24147" PostTypeId="2" ParentId="24094" CreationDate="2020-10-19T18:19:28.577" Score="0" Body="&lt;p&gt;I'd like to add some details to the Neil Slater's answer.&lt;/p&gt;&#xA;&lt;p&gt;In order to generate data, we want to find some unknown distribution. Since we do not know anything about a real distribution, we can approximate it using GAN. It was shown that optimizing the loss function of the original GAN is equivalent to&#xA;minimizing Jensen-Shannon divergence between the real distribution and the estimated (generator) distribution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ktbwcm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ktbwcm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;code&gt;q&lt;/code&gt; is our initial distribution and &lt;code&gt;p&lt;/code&gt; is the target distribution.&lt;/p&gt;&#xA;&lt;p&gt;While training the Discriminator as a normal classification model (which learns to distinguish between generated and real samples), we want to push the Generator towards the real data distribution. To make the generated samples look realistic, we pass true labels to calculate the gradient in that direction. Here is the intuition behind generator training:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Bfdcj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bfdcj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-10-19T18:24:43.660" LastActivityDate="2020-10-19T18:24:43.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24150" PostTypeId="2" ParentId="2922" CreationDate="2020-10-19T20:31:11.157" Score="3" Body="&lt;p&gt;Although asked over 3 years ago, the question is still interesting and while I agree with the original answer, a lot can be added to it.&lt;/p&gt;&#xA;&lt;p&gt;First, I'd like to point out that the term &amp;quot;knowledge base&amp;quot; is very ambiguous and it means different things to different people. For example, there is no sharp distinction between knowledge base and neural network. By now NN can be so large that it essentially encodes knowledge as GPT does. So the distinction becomes a question of interface. And NN are no longer as opaque since many new techniques are available to probe the knowledge inside NN. Even more fundamental dustinction between symbolic and neural reasoning becoming less important when hybrid AI combines both in a intertwined fashion. So the historical divisions were largely about technologies and not the essence of AI.&lt;/p&gt;&#xA;&lt;p&gt;Second, when it comes to NLP there is a fundamental distinction between language as surface form of information used for communication and knowledge as deep information which cannot be accessed directly even with traditional database technologies. That fundamental divide makes the historical differences even less relevant today. NLP is where that interplay between surface and deep forms was at the forefront of AI but the same is now happening with vision and planning. The question becomes - How do we architect the interface between deep knowledge (however it is represented) and surface communication? At the moment the natural language seems to be the only viable answer. So, for example there is effort to develop a natural language interface to replace plethora of query languages use by systems.&lt;/p&gt;&#xA;&lt;p&gt;My personal prediction is that natural language will slowly evolve to include variety of technical languages and multimodal interactions. But it is not clear at all how this will happen.&lt;/p&gt;&#xA;" OwnerUserId="5231" LastEditorUserId="5231" LastEditDate="2020-10-22T21:27:07.263" LastActivityDate="2020-10-22T21:27:07.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24151" PostTypeId="1" CreationDate="2020-10-19T22:12:18.020" Score="1" ViewCount="35" Body="&lt;p&gt;I am training a network through reinforcement learning. The policy network learns rotations, but depending on the actual input (state), the output of the network should be restricted to be in certain bounds otherwise it mostly fails to reach these bounds. I am using &lt;code&gt;tanh&lt;/code&gt; as last activation function. So, I wonder if there could be a way to modify this last activation function s.th. it can adaptively change bounds depending on input? Or would this have a negative impact in learning?&lt;/p&gt;&#xA;&lt;p&gt;I would also be open for papers or publications tackling these kind of problems. Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="41715" LastActivityDate="2020-10-19T22:12:18.020" Title="Dynamically adapting activation function" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;activation-function&gt;&lt;policies&gt;&lt;state-of-the-art&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24153" PostTypeId="1" CreationDate="2020-10-20T05:22:04.817" Score="2" ViewCount="59" Body="&lt;p&gt;I am trying to debug a convolutional neural network. I am seeing gradients close to zero.&lt;/p&gt;&#xA;&lt;p&gt;How can I decide whether these gradients are vanishing or not? Is there some threshold to decide on vanishing gradient by looking at the values?&lt;/p&gt;&#xA;&lt;p&gt;I am getting values close to &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; decimal places (e.g. &lt;span class=&quot;math-container&quot;&gt;$0.0001$&lt;/span&gt;) and, in some cases, close to &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; decimal places (e.g. &lt;span class=&quot;math-container&quot;&gt;$0.00001$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bwapq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bwapq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The CNN seems not to be learning since the histogram of weight is also quite similar in all epochs.&lt;/p&gt;&#xA;&lt;p&gt;I am using the ReLU activation function and Adam optimizer. What could be the reason for the vanishing gradient in the case of the ReLU activation function?&lt;/p&gt;&#xA;&lt;p&gt;If it is possible, please, point me to some resources that might be helpful.&lt;/p&gt;&#xA;" OwnerUserId="32394" LastEditorUserId="2444" LastEditDate="2020-12-13T12:52:50.653" LastActivityDate="2020-12-13T12:52:50.653" Title="How to decide if gradients are vanishing?" Tags="&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;relu&gt;&lt;vanishing-gradient-problem&gt;&lt;adam&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24154" PostTypeId="2" ParentId="24117" CreationDate="2020-10-20T06:34:46.397" Score="6" Body="&lt;p&gt;Before anything, the function you have wrote for the network lacks the bias variables (I'm sure you used bias to get those beautiful images, otherwise your &lt;code&gt;tanh&lt;/code&gt; network had to start from zero).&lt;/p&gt;&#xA;&lt;p&gt;Generally I would say it's impossible to have a good approximation of sinus with just 3 neurons, but if you want to consider one period of sinus, then you can do something. for clarity look at this picture:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nSAJK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nSAJK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've write the code for this task in colab and you can find it &lt;a href=&quot;https://colab.research.google.com/drive/1X4wdJ4v_uq2yD-ixjpLeErVPKC_RgYiI?usp=sharing&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, and you can play with it if you want.&lt;/p&gt;&#xA;&lt;p&gt;If you run the network several times you may get different results (because of different initializations) and you can see some of them at the &lt;strong&gt;Results&lt;/strong&gt; section of the link above. What you showed us in the images above are just two possibilities. But it's interesting that you can get better results with &lt;code&gt;tanh&lt;/code&gt; rather than &lt;code&gt;sigmoid&lt;/code&gt; and if you want to know why, I highly recommend you to look at &lt;a href=&quot;https://www.youtube.com/watch?v=wEoyxE0GP2M&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=6&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; lecture of CS231n. In summary it's because &lt;code&gt;tanh&lt;/code&gt; has the negative part and the network can learn better with it.&lt;/p&gt;&#xA;&lt;p&gt;But actually their power of approximation are almost similar because &lt;code&gt;2*sigmoid(1.5*x) - 1&lt;/code&gt; almost looks the same as &lt;code&gt;tanh(x)&lt;/code&gt; and you can find it by looking the picture below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/iJ5Ex.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iJ5Ex.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So why you can't get the same results as &lt;code&gt;tanh&lt;/code&gt;? that's because &lt;code&gt;tanh&lt;/code&gt; suits the problem better and if the network wants to get the same result as &lt;code&gt;tanh&lt;/code&gt; with &lt;code&gt;sigmoid&lt;/code&gt; it should learn their transformation parameters and learning these parameters makes the learning task harder. So It's not impossible to get the same result with &lt;code&gt;sigmoid&lt;/code&gt; but it's harder. And to show you that its possible, I have set the parameters of the network using &lt;code&gt;sigmoid&lt;/code&gt; manually and got the result below (you can get better results if you have more time):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6njbk.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6njbk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;At last if you want to know why you can't get the same result with 2 neurons instead of 3 neurons, it's better to understand what does the network do with 3 neurons.&lt;br /&gt;&#xA;If you look at the output of the first layer, you may see something like this (which are outputs of two neurons it has):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rcsNq.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rcsNq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the next layer gets the difference between the output of these two neurons (which is like sinus) and applies &lt;code&gt;sigmoid&lt;/code&gt; or &lt;code&gt;tanh&lt;/code&gt; to it, and that's how you get a good result. But when you have just one neuron in the first layer, you can't imagine some scenario like this and approximating one period of sinus is out of it's ability (underfitting).&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-10-20T06:34:46.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24155" PostTypeId="1" CreationDate="2020-10-20T06:56:42.523" Score="1" ViewCount="43" Body="&lt;p&gt;Suppose we are training an environment with 2 collaborative agents with Reinforcement Learning. We define the following example: There is a midfielder and a striker. The midfielder's reward depends on how many goals are scored, which however depends on the attacker's performance. And the striker's performance depends on how good the midfielder is at making his passes.&lt;/p&gt;&#xA;&lt;p&gt;For this type of problem, what do you recommend to study?&lt;/p&gt;&#xA;" OwnerUserId="41726" LastEditorUserId="2444" LastEditDate="2020-10-31T15:16:14.300" LastActivityDate="2020-10-31T15:16:14.300" Title="Which reinforcement learning approach to use when there are 2 collaborative agents?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24157" PostTypeId="1" CreationDate="2020-10-20T07:40:50.737" Score="2" ViewCount="158" Body="&lt;p&gt;I'm writing a DQN agent for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hunt_the_Wumpus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wumpus game&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is the reward function to train the Q-networks (target network and policy) the same as the score of the game, i.e. +1000 for picking up gold, -1000 for falling in pits and dying from the wumpus, -1 each move?&lt;/p&gt;&#xA;&lt;p&gt;This is naturally cumulative, in that the score changes after each action taken by the agent. Alternatively, is it just a +1 for win, -1 for a loss and 0 in all other situations?&lt;/p&gt;&#xA;" OwnerUserId="41728" LastEditorUserId="2444" LastEditDate="2020-10-21T16:15:20.470" LastActivityDate="2020-10-21T16:15:20.470" Title="How should I define the reward function to solve the Wumpus game with deep Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24158" PostTypeId="1" CreationDate="2020-10-20T07:57:01.210" Score="2" ViewCount="30" Body="&lt;p&gt;I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum percentage of ones in labels is 0.034%.&lt;/p&gt;&#xA;&lt;p&gt;Below is the distribution of percentage of one's in each labels&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9JuNE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9JuNE.png&quot; alt=&quot;here you can find the distribution of percentage of one's present in labels&quot; /&gt;&lt;/a&gt;&#xA;If I simply build a multilabel classification single model. The score it gives may be high but it got biased towards zeros very much so, it doesn't give probability of a label to be one very high. And if I want to build for each label different model, I can treat it as a bunch of imbalanced data and apply smote algorithm to each model, But I have a doubt whether can smote produce a good amount of data to balance because we know how imbalance my data is. Now, doubt is can I gave a try to autoencoders, which I heard good at fraud detection when the data is having a percentage of one's less than 1% or such. Will it perform better in my case? because if it can work well, then I will study autoencoders.&lt;/p&gt;&#xA;" OwnerUserId="38737" LastActivityDate="2020-10-20T07:57:01.210" Title="how to handle highly imbalanced multilabel classification?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;autoencoders&gt;&lt;multi-label-classification&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24159" PostTypeId="2" ParentId="24157" CreationDate="2020-10-20T08:25:46.830" Score="1" Body="&lt;p&gt;The reward function is up to you when you set the goals for the agent.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the goal is to score as highly as possible, before ending the game, then use the score. You may want to scale the score down if you are using neural networks, to prevent needing to handle very large error values in early phases of learning.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the goal is to win the game, and you do not care about the score, then use the win/loss end result. I am not familiar with the game, but if it is possible to win the game - e.g. reach an exit - whilst not collecting all the gold, then the agent may choose to do that if it reduces the chance of losing.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The second option is harder for the agent to assess. You may want the current score to be one of the state variables, as the score is likely to be correlated with win/loss.&lt;/p&gt;&#xA;&lt;p&gt;Most computer games are designed around giving a numerical score as feedback for human play, with high score tables, players considered &amp;quot;better&amp;quot; if they get more points etc. If you want your agent to compete in the same way, then using the score directly will help achieve that goal.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-10-20T08:25:46.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24160" PostTypeId="1" CreationDate="2020-10-20T08:29:43.110" Score="0" ViewCount="64" Body="&lt;p&gt;I am training a deep learning algorithm using an NVIDIA GEFORCE RTX GPU, with 16 GB RAM. I went through my image database and generated 50,000 training samples with the Labelimg software; the images were dimensions of 4000 x 3000 pixels and numbered 1037 in total. I am using TensorFlow 1.15 in a conda environment. The algorithm is the SSD V1 MOBILENET algorithm that was downloaded from Tensorflow's Github, and can be seen here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model {&#xA;  ssd {&#xA;    num_classes: 1&#xA;    box_coder {&#xA;      faster_rcnn_box_coder {&#xA;        y_scale: 10.0&#xA;        x_scale: 10.0&#xA;        height_scale: 5.0&#xA;        width_scale: 5.0&#xA;      }&#xA;    }&#xA;    matcher {&#xA;      argmax_matcher {&#xA;        matched_threshold: 0.5&#xA;        unmatched_threshold: 0.5&#xA;        ignore_thresholds: false&#xA;        negatives_lower_than_unmatched: true&#xA;        force_match_for_each_row: true&#xA;      }&#xA;    }&#xA;    similarity_calculator {&#xA;      iou_similarity {&#xA;      }&#xA;    }&#xA;    anchor_generator {&#xA;      ssd_anchor_generator {&#xA;        num_layers: 6&#xA;        min_scale: 0.2&#xA;        max_scale: 0.95&#xA;        aspect_ratios: 1.0&#xA;        aspect_ratios: 2.0&#xA;        aspect_ratios: 0.5&#xA;        aspect_ratios: 3.0&#xA;        aspect_ratios: 0.3333&#xA;      }&#xA;    }&#xA;    image_resizer {&#xA;      fixed_shape_resizer {&#xA;        height: 300&#xA;        width: 300&#xA;      }&#xA;    }&#xA;    box_predictor {&#xA;      convolutional_box_predictor {&#xA;        min_depth: 0&#xA;        max_depth: 0&#xA;        num_layers_before_predictor: 0&#xA;        use_dropout: false&#xA;        dropout_keep_probability: 0.8&#xA;        kernel_size: 1&#xA;        box_code_size: 4&#xA;        apply_sigmoid_to_scores: false&#xA;        conv_hyperparams {&#xA;          activation: RELU_6,&#xA;          regularizer {&#xA;            l2_regularizer {&#xA;              weight: 0.00004&#xA;            }&#xA;          }&#xA;          initializer {&#xA;            truncated_normal_initializer {&#xA;              stddev: 0.03&#xA;              mean: 0.0&#xA;            }&#xA;          }&#xA;          batch_norm {&#xA;            train: true,&#xA;            scale: true,&#xA;            center: true,&#xA;            decay: 0.9997,&#xA;            epsilon: 0.001,&#xA;          }&#xA;        }&#xA;      }&#xA;    }&#xA;    feature_extractor {&#xA;      type: 'ssd_mobilenet_v1'&#xA;      min_depth: 16&#xA;      depth_multiplier: 1.0&#xA;      conv_hyperparams {&#xA;        activation: RELU_6,&#xA;        regularizer {&#xA;          l2_regularizer {&#xA;            weight: 0.00004&#xA;          }&#xA;        }&#xA;        initializer {&#xA;          truncated_normal_initializer {&#xA;            stddev: 0.03&#xA;            mean: 0.0&#xA;          }&#xA;        }&#xA;        batch_norm {&#xA;          train: true,&#xA;          scale: true,&#xA;          center: true,&#xA;          decay: 0.9997,&#xA;          epsilon: 0.001,&#xA;        }&#xA;      }&#xA;    }&#xA;    loss {&#xA;      classification_loss {&#xA;        weighted_sigmoid {&#xA;        }&#xA;      }&#xA;      localization_loss {&#xA;        weighted_smooth_l1 {&#xA;        }&#xA;      }&#xA;      hard_example_miner {&#xA;        num_hard_examples: 3000&#xA;        iou_threshold: 0.99&#xA;        loss_type: CLASSIFICATION&#xA;        max_negatives_per_positive: 3&#xA;        min_negatives_per_image: 0&#xA;      }&#xA;      classification_weight: 1.0&#xA;      localization_weight: 1.0&#xA;    }&#xA;    normalize_loss_by_num_matches: true&#xA;    post_processing {&#xA;      batch_non_max_suppression {&#xA;        score_threshold: 1e-8&#xA;        iou_threshold: 0.6&#xA;        max_detections_per_class: 100&#xA;        max_total_detections: 100&#xA;      }&#xA;      score_converter: SIGMOID&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;train_config: {&#xA;  batch_size: 3&#xA;  optimizer {&#xA;    rms_prop_optimizer: {&#xA;      learning_rate: {&#xA;        exponential_decay_learning_rate {&#xA;          initial_learning_rate: 0.004&#xA;          decay_steps: 800720&#xA;          decay_factor: 0.95&#xA;        }&#xA;      }&#xA;      momentum_optimizer_value: 0.9&#xA;      decay: 0.9&#xA;      epsilon: 1.0&#xA;    }&#xA;  }&#xA;  fine_tune_checkpoint: &amp;quot;ssd_mobilenet_v1_coco_2018_01_28/model.ckpt&amp;quot;&#xA;  from_detection_checkpoint: true&#xA;  num_steps: 200000&#xA;  data_augmentation_options {&#xA;    random_horizontal_flip {&#xA;    }&#xA;  }&#xA;  data_augmentation_options {&#xA;    ssd_random_crop {&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;train_input_reader: {&#xA;  tf_record_input_reader {&#xA;    input_path: &amp;quot;data/train.record&amp;quot;&#xA;  }&#xA;  label_map_path: &amp;quot;data/object-detection.pbtxt&amp;quot;&#xA;}&#xA;&#xA;eval_config: {&#xA;  num_examples: 8000&#xA;  # Note: The below line limits the evaluation process to 10 evaluations.&#xA;  # Remove the below line to evaluate indefinitely.&#xA;  max_evals: 10&#xA;}&#xA;&#xA;eval_input_reader: {&#xA;  tf_record_input_reader {&#xA;    input_path: &amp;quot;data/test.record&amp;quot;&#xA;  }&#xA;  label_map_path: &amp;quot;training/object-detection.pbtxt&amp;quot;&#xA;  shuffle: false&#xA;  num_readers: 1&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have been training this algorithm for about 36 hours, and have only been able to get the loss down to between 6-8%. I need it to get down around 1%.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way to estimate how long it will take to train this algorithm?&lt;/strong&gt; Right now, the average step speed is around 1.5 seconds. I have overclocked my GPU for performance, so that helps a bit. Sample output is below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UN19Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UN19Z.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-10-20T08:29:43.110" Title="How long will it take to train SSD V1 Mobilenet image recognition algorithm?" Tags="&lt;deep-learning&gt;&lt;training&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24161" PostTypeId="1" CreationDate="2020-10-20T08:30:33.267" Score="3" ViewCount="89" Body="&lt;p&gt;Hello :) I'm pretty new to this community, so let me know if I posted anything incorrectly and I'll try to change it.&lt;/p&gt;&#xA;&lt;p&gt;I'm working on the project which aim is to create self-driving agent in CARLA. I built a neural network Xception (decaying  ε-greedy). The other parameters are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;EPISODES: 100&lt;br /&gt;&#xA;GAMMA: 0.3&lt;br /&gt;&#xA;EPSILON_DECAY: 0.9&lt;br /&gt;&#xA;MIN_EPSILON: 0.001&#xA;BATCH: 16&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Due to the limited computer resources I chose 100 or 300 epochs to train the model, but it generates much fluctuations:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/oY1YY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oY1YY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fTrcv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fTrcv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;EPISODES: 100&lt;br /&gt;&#xA;GAMMA: 0.7&#xA;EPSILON_DECAY: 0.9&lt;br /&gt;&#xA;MIN_EPSILON: 0.001&#xA;BATCH: 16&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/icpZs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/icpZs.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Can anyone suggest how can I improve my results? Or it is only the issue of small number of epochs?&lt;/p&gt;&#xA;" OwnerUserId="40980" LastEditorUserId="40980" LastEditDate="2020-10-20T15:03:29.007" LastActivityDate="2020-10-21T10:00:05.280" Title="Improving DQN with fluctuations" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24163" PostTypeId="1" CreationDate="2020-10-20T11:41:22.150" Score="1" ViewCount="23" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xvf5w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xvf5w.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the above diagram, the shape of some of the matrices can be seen in the yellow highlight. For instance:&lt;/p&gt;&#xA;&lt;p&gt;The hidden state at timestep t-1 (&lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt;) has shape &lt;span class=&quot;math-container&quot;&gt;$(na, m)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The input data at timestep t (&lt;span class=&quot;math-container&quot;&gt;$x_{t}$&lt;/span&gt;) has shape &lt;span class=&quot;math-container&quot;&gt;$(nx, m)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Z_{t}$&lt;/span&gt; has shape &lt;span class=&quot;math-container&quot;&gt;$(na+nx, m)$&lt;/span&gt; since the hidden state and input data are concatenated in LSTMs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_{c}$&lt;/span&gt; has shape &lt;span class=&quot;math-container&quot;&gt;$(na, na+nx)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_{c}$&lt;/span&gt; • &lt;span class=&quot;math-container&quot;&gt;$Z_{t}$&lt;/span&gt; has shape &lt;span class=&quot;math-container&quot;&gt;$(na, m)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$i_{t}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_{i}$&lt;/span&gt; • &lt;span class=&quot;math-container&quot;&gt;$Z_{t}$&lt;/span&gt; has shape &lt;span class=&quot;math-container&quot;&gt;$(na, m)$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$ĉ_{t}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When working through the network to the point &lt;span class=&quot;math-container&quot;&gt;$i_{t}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ĉ_{t}$&lt;/span&gt;, how can these two be dot producted when the multiplication is not of the form (m x n)(n x p) as per the matrix multiplication definition?:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ChwZK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ChwZK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="26159" LastEditDate="2020-10-20T12:04:05.897" LastActivityDate="2020-10-20T12:49:29.503" Title="How do LSTMs work if the following two matrices are not able to be multiplied?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24164" PostTypeId="2" ParentId="24157" CreationDate="2020-10-20T11:42:39.557" Score="1" Body="&lt;p&gt;The reward function belongs the the environment and it is the only way the agent can explore the world given a state.&lt;/p&gt;&#xA;&lt;p&gt;If we want agent to do something specific, we must provide rewards to it in such a way that it will achieve our goals. It is thus very important that the reward function accurately indicates the exact behaviour.&lt;/p&gt;&#xA;&lt;p&gt;Depending on your goal you can construct the function such that the agent will try to finish the game as fast as possible, or collect the maximum score.&lt;/p&gt;&#xA;&lt;p&gt;For example, certain reward functions can cause an agent to commit suicide in order to avoid more severe punishment in form of negative reward in the future (e.g. if the step reward very small). Or it will go the safest way without collecting gold, if falling in pits punishment is very big. In other words, you should experiment with your reward function to find a tradeoff.&lt;/p&gt;&#xA;&lt;p&gt;Check out this &lt;a href=&quot;https://www.youtube.com/watch?v=pVIFc72VYH8&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt; for more intuition behind it.&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-10-20T11:48:04.407" LastActivityDate="2020-10-20T11:48:04.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24165" PostTypeId="2" ParentId="24161" CreationDate="2020-10-20T12:43:01.897" Score="1" Body="&lt;p&gt;It is not clear form your question, how you use your replay buffer. Basically, you have to store all states/reward tuples and train your agent on the entire buffer.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, you should give the agent time to explore (all) states of the world. But if you want to speed up training, you can try to implement &lt;a href=&quot;https://medium.com/@jonathan_hui/rl-importance-sampling-ebfb28b4a8c6&quot; rel=&quot;nofollow noreferrer&quot;&gt;importance sampling&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-10-21T10:00:05.280" LastActivityDate="2020-10-21T10:00:05.280" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24166" PostTypeId="2" ParentId="24163" CreationDate="2020-10-20T12:49:29.503" Score="1" Body="&lt;p&gt;Turns out the reason is because, for places where a dot is shown in the image above, they're actually element-wise multiplications, not dot products. A lot of sources use an X or . to denote multiplication, but don't clearly indicate they mean element-wise multiplication.&lt;/p&gt;&#xA;" OwnerUserId="26159" LastActivityDate="2020-10-20T12:49:29.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24168" PostTypeId="1" CreationDate="2020-10-20T15:28:42.037" Score="1" ViewCount="40" Body="&lt;p&gt;I am using an AlexNet architecture as my Convolutional Neural Network.&#xA;A learning rate of 0.00007 and 128 batch_size.&#xA;I have 20000 data and 10% test, 40% validation, and 50% for training.&#xA;I used 100 epochs to train my network and here are my results for Loss and Accuracy.&#xA;I would like to ask how can I get closer validation and training loss in these plots?&#xA;At first, I guess the number of epochs was not enough, but I tried more epochs and my results didn't change.&#xA;Can I say my training process is complete with this distance between train and validation loss?&#xA;Is there any way to have closer loss plots?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bqEPo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bqEPo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ipr0N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ipr0N.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="33792" LastEditDate="2020-10-20T17:27:24.367" LastActivityDate="2020-10-21T10:45:30.070" Title="How to have closer validation loss and training loss in training a CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;convergence&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24169" PostTypeId="1" CreationDate="2020-10-20T15:46:20.000" Score="1" ViewCount="11" Body="&lt;p&gt;I've been reading &lt;a href=&quot;https://arxiv.org/pdf/1811.07489.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; coordinate systems defined by &lt;span class=&quot;math-container&quot;&gt;$\{A_j,b_j\}_{j=1}^F$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; denotes the rotation of the frame as an orientation matrix and &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt; represents the origin of the frame. Each datapoint &lt;span class=&quot;math-container&quot;&gt;$\xi_t$&lt;/span&gt; is observed from the viewpoint of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; different experts/frames, with &lt;span class=&quot;math-container&quot;&gt;$\xi_t^{(j)} = A_j^{-1}(\xi_t - b_j)$&lt;/span&gt; denoting the datapoint w.r.t. frame &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How is &lt;span class=&quot;math-container&quot;&gt;$\xi_t^{(j)} = A_j^{-1}(\xi_t - b_j)$&lt;/span&gt; derived? I understand that we must subtract &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt;, but I'm not sure if I should pre-multiply by &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$A_j^{-1}$&lt;/span&gt;, so it'd be great if someone could help me understand this better. Since &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; is an orientation matrix, I'd guess that it's orthogonal, and so &lt;span class=&quot;math-container&quot;&gt;$A_j^{-1} = A_j^T$&lt;/span&gt; - and it may just be a matter of convention (i.e. depending on how &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; is defined). The details aren't clear from the paper though, and I'd appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-10-20T15:46:20.000" Title="How do I find the data-point with respect to a given frame?" Tags="&lt;math&gt;&lt;robotics&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24170" PostTypeId="1" CreationDate="2020-10-20T15:55:27.323" Score="2" ViewCount="13" Body="&lt;p&gt;I've been reading &lt;a href=&quot;https://arxiv.org/pdf/1811.07489.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; that formulates invariant task-parametrized HSMMs. The task parameters are represented in &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; coordinate systems defined by &lt;span class=&quot;math-container&quot;&gt;$\{A_j,b_j\}_{j=1}^F$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; denotes the rotation of the frame as an orientation matrix and &lt;span class=&quot;math-container&quot;&gt;$b_j$&lt;/span&gt; represents the origin of the frame. Each datapoint &lt;span class=&quot;math-container&quot;&gt;$\xi_t$&lt;/span&gt; is observed from the viewpoint of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; different experts/frames, with &lt;span class=&quot;math-container&quot;&gt;$\xi_t^{(j)} = A_j^{-1}(\xi_t - b_j)$&lt;/span&gt; denoting the datapoint w.r.t. frame &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. I quote from the abstract:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Generalizing manipulation skills to new situations requires &lt;strong&gt;extracting invariant patterns from demonstrations&lt;/strong&gt;. For example, the robot needs to &lt;strong&gt;understand the demonstrations at a higher level&lt;/strong&gt; while being invariant to the appearance of the objects, geometric aspects of objects such as its position, size, orientation and viewpoint of the observer in the demonstrations.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The algorithm takes as input the demonstrations with respect to different coordinate systems describing virtual landmarks or objects of interest with a task-parameterized formulation, and adapt the segments according to the environmental changes in a systematic manner.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Though it makes some intuitive sense, I'm not fully convinced why working with multiple coordinate systems would help us capture invariant patterns in demonstrations, and leave aside the scene-specific details. That is the goal, right? On a very high level, I see that having access to more &amp;quot;viewpoints&amp;quot; may help the robot understand the environment better, and neglect viewpoint-specific biases to focus on invariant patterns across different frames. However, this is very handwavy - and I'd love to know specific details about why using multiple viewpoints is a good idea in this case.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="35585" LastActivityDate="2020-10-20T15:55:27.323" Title="How do multiple coordinate systems help in capturing invariant features?" Tags="&lt;robotics&gt;&lt;hidden-markov-model&gt;&lt;imitation-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24171" PostTypeId="1" CreationDate="2020-10-20T16:14:34.583" Score="1" ViewCount="69" Body="&lt;p&gt;I am working on implementing an RL agent and I want to demonstrate its effectiveness over a bounded problem space. The setting is essentially a &lt;a href=&quot;https://en.wikipedia.org/wiki/Queueing_theory#Queueing_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;queueing network&lt;/a&gt; and so it can be represented as a graph. I want to consider the agent's performance over all graphs up to order &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and with average degree from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (edgeless) to &lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt; (fully connected).&lt;/p&gt;&#xA;&lt;p&gt;I have looked into generating random graphs using the Erdős–Rényi model, for example. My thought is that I could show the average performance of my agent for different settings of number of nodes and edge probability (under this particular graph generation model).&lt;/p&gt;&#xA;&lt;p&gt;Are there any established techniques that are along the lines of this approach?&lt;/p&gt;&#xA;" OwnerUserId="31466" LastEditorUserId="31466" LastEditDate="2020-10-20T21:15:22.807" LastActivityDate="2020-10-20T21:15:22.807" Title="How can I evaluate a reinforcement learning algorithm over an entire problem space?" Tags="&lt;reinforcement-learning&gt;&lt;graph-theory&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24172" PostTypeId="1" CreationDate="2020-10-20T16:45:45.367" Score="0" ViewCount="32" Body="&lt;p&gt;Tried to extract only specific information from the images but Couldn't, We have to automate this process using this as the format of the Images keeps on changing.&lt;/p&gt;&#xA;&lt;p&gt;Link&lt;a href=&quot;https://i.stack.imgur.com/RarrU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sample data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I have Tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from cv2 import cv2&#xA;import pytesseract&#xA;from pytesseract import Output&#xA;import pandas as pd&#xA;&#xA;img = cv2.imread(r&amp;quot;C:/Users/Documents/pdf2txt/invoice.jpg&amp;quot;)&#xA;gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)&#xA;thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]&#xA;&#xA;custom_config = r'-l eng --oem 1 --psm 6 '&#xA;d = pytesseract.image_to_data(thresh, config=custom_config, output_type=Output.DICT)&#xA;df = pd.DataFrame(d)&#xA;&#xA;df1 = df[(df.conf != '-1') &amp;amp; (df.text != ' ') &amp;amp; (df.text != '')]&#xA;pd.set_option('display.max_rows', None)&#xA;pd.set_option('display.max_columns', None)&#xA;&#xA;sorted_blocks = df1.groupby('block_num').first().sort_values('top').index.tolist()&#xA;for block in sorted_blocks:&#xA;    curr = df1[df1['block_num'] == block]&#xA;    sel = curr[curr.text.str.len() &amp;gt; 3]&#xA;    # sel = curr&#xA;    char_w = (sel.width / sel.text.str.len()).mean()&#xA;    prev_par, prev_line, prev_left = 0, 0, 0&#xA;    text = ''&#xA;    for ix, ln in curr.iterrows():&#xA;        # add new line when necessary&#xA;        if prev_par != ln['par_num']:&#xA;            text += '\n'&#xA;            prev_par = ln['par_num']&#xA;            prev_line = ln['line_num']&#xA;            prev_left = 0&#xA;        elif prev_line != ln['line_num']:&#xA;            text += '\n'&#xA;            prev_line = ln['line_num']&#xA;            prev_left = 0&#xA;&#xA;        added = 0  # num of spaces that should be added&#xA;        if ln['left'] / char_w &amp;gt; prev_left + 1:&#xA;            added = int((ln['left']) / char_w) - prev_left&#xA;            text += ' ' * added&#xA;        text += ln['text'] + ' '&#xA;        prev_left += len(ln['text']) + added + 1&#xA;    text += '\n'&#xA;    print(text)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output We Need :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; a) INVOICE NO.&#xA; b) CUSTOMER NO.&#xA; c) SUBTOTAL&#xA; d) TOTAL&#xA; e) DATE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As New to Python Please Suggest !!&lt;/p&gt;&#xA;" OwnerUserId="41742" LastActivityDate="2020-10-20T16:45:45.367" Title="Extracting specific information from an Invoice images" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;python&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24176" PostTypeId="1" CreationDate="2020-10-20T21:28:54.003" Score="1" ViewCount="85" Body="&lt;p&gt;Machine Learning books generally explains that the error calculated for a given sample &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$e_i = y_i - \hat{y_i}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; is the target output and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the actual output given by the network. So, a loss function &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is calculated:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L = \frac{1}{2N}\sum^{N}_{i=1}(e_i)^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above scenario is explained for a binary classification/regression problem. Now, let's assume a MLP network with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; neurons in the output layer for a multiclass classification problem (generally one neuron per class).&lt;/p&gt;&#xA;&lt;p&gt;What changes in the equations above? Since we now have multiple outputs, both &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; should be a vector?&lt;/p&gt;&#xA;" OwnerUserId="41747" LastEditorUserId="4709" LastEditDate="2020-10-21T16:20:22.913" LastActivityDate="2020-10-22T09:21:56.770" Title="How is the error calculated with multiple output neurons in the neural network?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;feedforward-neural-network&gt;&lt;multilayer-perceptron&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24177" PostTypeId="1" CreationDate="2020-10-20T23:26:57.317" Score="0" ViewCount="38" Body="&lt;p&gt;Can anyone help me with the algorithm used to evaluate the board once a player has played his turn in the game of Go?&#xA;I have a vector representing the board, with a dimension for each postion . Player 1 places a value of 1 on the positions in the vcetor while player two places a 0.5. i would like to parse the vector to the algorithm and it evaluates which peices are to be removed if any.&lt;/p&gt;&#xA;&lt;p&gt;I would also be greatful if i could get an algorithm for evaluating the board right at the end of the game.As well as one for displaying the board from such vector as an image.&#xA;Thankyou&lt;/p&gt;&#xA;" OwnerUserId="28585" LastActivityDate="2020-10-20T23:26:57.317" Title="Algorithm to evaluate Go board after turn" Tags="&lt;go&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24178" PostTypeId="2" ParentId="24176" CreationDate="2020-10-21T04:45:35.200" Score="1" Body="&lt;p&gt;Assuming you're using softmax on the last layer for classification, it sounds like a simple application of cross entropy loss from here on out: &lt;a href=&quot;https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation&quot;&gt;https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;" OwnerUserId="34473" LastEditorUserId="34473" LastEditDate="2020-10-21T12:10:16.293" LastActivityDate="2020-10-21T12:10:16.293" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24179" PostTypeId="1" CreationDate="2020-10-21T04:45:43.850" Score="0" ViewCount="31" Body="&lt;p&gt;I am taking a course in Machine Learning and the Professor introduced us to the XOR problem.&lt;/p&gt;&#xA;&lt;p&gt;I understand the XOR problem is not linearly separable and we need to employ Neural Network for this problem.&lt;/p&gt;&#xA;&lt;p&gt;However, he mentioned XOR works better with Bipolar representation(-1, +1) which I have not really understand.&lt;/p&gt;&#xA;&lt;p&gt;I am wondering what Bipolar representation would be better than Binary Representation? Whats the rationale for saying so?&lt;/p&gt;&#xA;" OwnerUserId="41187" LastActivityDate="2020-10-21T04:45:43.850" Title="XOR problem with bipolar representation" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24180" PostTypeId="2" ParentId="9275" CreationDate="2020-10-21T08:04:34.093" Score="1" Body="&lt;p&gt;Your best bet would be to formulate the problem in &lt;a href=&quot;https://en.wikipedia.org/wiki/Planning_Domain_Definition_Language&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDDL&lt;/a&gt;, which should be fairly easy, and then use a standard planner to generate a plan from that description.&lt;/p&gt;&#xA;&lt;p&gt;In PDDL you describe the properties and the possible actions, the start state and the goal state, and the planner will then take this to produce a sequence of actions that leads from the start state to the goal state. There is a &lt;a href=&quot;http://editor.planning.domains/&quot; rel=&quot;nofollow noreferrer&quot;&gt;planner available on-line&lt;/a&gt; that you can use.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-10-21T08:04:34.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24181" PostTypeId="1" CreationDate="2020-10-21T08:43:29.850" Score="2" ViewCount="73" Body="&lt;p&gt;In AlphaGo Zero, MCTS is used along with policy networks. Some sources say MCTS (or planning in general) increases the sample efficiency.&lt;/p&gt;&#xA;&lt;p&gt;Assumed the transition model is known and the computational cost of interacting through planning is the same as interacting with the environment, I do not see the difference between playing many games versus playing a single game, but plan at each step.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, given a problem with a known transition model, how do we know combining learning and tree search will likely be better than pure learning?&lt;/p&gt;&#xA;" OwnerUserId="41751" LastEditorUserId="41751" LastEditDate="2020-11-13T00:10:49.800" LastActivityDate="2020-11-13T00:10:49.800" Title="Why is tree search/planning used in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;monte-carlo-tree-search&gt;&lt;alphago-zero&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24182" PostTypeId="2" ParentId="24176" CreationDate="2020-10-21T09:21:44.280" Score="1" Body="&lt;p&gt;As you say, the outputs are modeled as a vector, each output in one vector component.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;In regression problems:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The most common loss function, like in the scalar case, is the square error. Skipping constants, it is defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E=\sum_i ||\mathbf{y_i}-\mathbf{\hat{y_i}}||^2 = \sum_i (\mathbf{y_i}-\mathbf{\hat{y_i}})(\mathbf{y_i}-\mathbf{\hat{y_i}})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{y_i}$&lt;/span&gt; vector is expected value for sample &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; (note I do not use the same naming convention than the question).&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{\hat{y_i}}$&lt;/span&gt; vector is network output for same sample&lt;/li&gt;&#xA;&lt;li&gt;||.|| is vector norm&lt;/li&gt;&#xA;&lt;li&gt;product of vectors is scalar/inner product.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The derivative respect some NN parameter &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial}{\partial w}E=\frac{\partial}{\partial w} \sum_i (\mathbf{y_i}-\mathbf{\hat{y_i}})(\mathbf{y_i}-\mathbf{\hat{y_i}}) = -2 \sum_i (\mathbf{y_i}-\mathbf{\hat{y_i}})\frac{\partial \mathbf{\hat{y_i}}}{\partial w} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;being &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \mathbf{\hat{y_i}}}{\partial w}$&lt;/span&gt; the term that backtracking algorithm evaluates.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Multi-class classification problems:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Two options appears as most usual ones:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a) optimize the square error of probabilities. Target vector will be of the form (0,...0,1,0,...0) while network output will be something as (0.2,0.1,0.8,0.4,...). This case can be solved like regression ones.&lt;/li&gt;&#xA;&lt;li&gt;b) optimize entropy. In this case, an usual loss function is cross-entropy:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ E = - \sum_c p(c) log(\hat p(c)) \text{  [definition]} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ E = - \sum_i \sum_c p_i(c) log(\hat p_i(c)) = - \sum_i log(\hat p_i(c_i)) \text{  [average]} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is some class&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p(c)$&lt;/span&gt; is probability of class &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in train dataset&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat p(c)$&lt;/span&gt; is probability of class &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in network output&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is sample number&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; is expected (correct) class of sample &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p_i(c)$&lt;/span&gt; is ground truth, usually taken as 1 if &lt;span class=&quot;math-container&quot;&gt;$c=c_i$&lt;/span&gt;, 0 otherwise as in latest expression.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat p_i(c)$&lt;/span&gt; is net output for sample &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and class &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-22T09:21:56.770" LastActivityDate="2020-10-22T09:21:56.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24186" PostTypeId="1" CreationDate="2020-10-21T12:11:37.633" Score="1" ViewCount="42" Body="&lt;p&gt;I am really trying to understand deep learning models like RNN, LSTMs etc. I have gone through many tutorials of RNN and have learned that RNN cannot work for long Range dependencies, like:&lt;/p&gt;&#xA;&lt;p&gt;Consider trying to predict the last word in the text “I grew up in France… I speak fluent French.” Recent information suggests that the next word is probably the name of a language, but if we want to narrow down which language, we need the context of France, from further back. It’s entirely possible for the gap between the relevant information and the point where it is needed to become very large. Unfortunately, as that gap grows, RNNs become unable to learn to connect the information.&lt;/p&gt;&#xA;&lt;p&gt;it comes due to &lt;strong&gt;vanish gradient&lt;/strong&gt; problem. However, I could not understand that how to vanish gradient creates an issue for RNN to not work for long-range dependencies.&#xA;Since, as I know that vanish gradient usually comes when we have many hidden layers and the gradient for the first layer usually produced too low and that affects the training process. However, everyone connects this issue with vanish gradient, technically what is the relationship RNN (long-range dependencies) with vanish gradient?&lt;/p&gt;&#xA;&lt;p&gt;I am really sorry if it is a weird question&lt;/p&gt;&#xA;" OwnerUserId="41756" LastActivityDate="2020-10-21T13:26:58.703" Title="How does vanish gradient restrict RNN to not work for long range dependencies?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;gradient-descent&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24187" PostTypeId="1" AcceptedAnswerId="24198" CreationDate="2020-10-21T12:17:42.870" Score="2" ViewCount="235" Body="&lt;p&gt;Assume that I have a fully connected network that takes in a vector containing &lt;code&gt;1025&lt;/code&gt; elements. First &lt;code&gt;1024&lt;/code&gt; elements are related to the input image of size &lt;code&gt;32 x 32 x 1&lt;/code&gt;, and the last element in the vector (&lt;code&gt;1025-th element&lt;/code&gt;) is a control bit that I call it &lt;em&gt;special input&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When this bit is &lt;code&gt;zero&lt;/code&gt;, the network should predict if there is a &lt;code&gt;cat&lt;/code&gt; in the image or not, and when this bit is &lt;code&gt;one&lt;/code&gt;, it should predict if there is a &lt;code&gt;dog&lt;/code&gt; in the image or not.&lt;/p&gt;&#xA;&lt;p&gt;So how can I tell the network that your &lt;code&gt;1025-th element&lt;/code&gt; should be special to you and you should pay more attention to it?&lt;/p&gt;&#xA;&lt;p&gt;Note that it's just an example and the real problem is more complex than this. So please don't bypass the goal of this question by using tricks special to this example. Any idea is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-10-21T22:18:01.597" Title="How to tell a neural network that: &quot;your i-th input is special&quot;" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;feedforward-neural-network&gt;" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24188" PostTypeId="1" CreationDate="2020-10-21T13:09:16.310" Score="1" ViewCount="41" Body="&lt;p&gt;Are there any known models/techniques to determine whether a person in a store is a customer or a store representative?&lt;/p&gt;&#xA;&lt;p&gt;For example, customer representatives can wear uniforms and then one possible way to identify customer representatives is by the color of their uniform, texture, etc. On the other hand, a customer can also wear the same color clothes as that of a customer representative. Likewise, a customer representative could be wearing &amp;quot;normal clothes.&amp;quot; So the main problems that may occur could be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A customer becomes misclassified as a customer representative.&lt;/li&gt;&#xA;&lt;li&gt;A customer representative representative becomes misclassified as a customer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So using clothing as the only proxy to classify people as customers or customer representatives seems to be flaky. Any other known ideas?&lt;/p&gt;&#xA;" OwnerUserId="41759" LastEditorUserId="2444" LastEditDate="2020-10-21T16:27:19.837" LastActivityDate="2020-10-22T07:20:07.590" Title="Are there any known models/techniques to determine whether a person in a store is a customer or a store representative?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;reference-request&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24190" PostTypeId="2" ParentId="24186" CreationDate="2020-10-21T13:19:42.017" Score="2" Body="&lt;p&gt;Vanishing gradient is: as the gradient starts to flow from the end of the network (right side of the network) to the start of the network (left side of the network), it will be multiplied by numbers less than 1 and gradually it will become weaker and weaker and when it arrives to the first layers, it's so weak that makes almost no change in initial layers parameters.&lt;/p&gt;&#xA;&lt;p&gt;Now in case of RNNs you can unroll the network, and now you can see that it is like a deep network. For clarity look at the image below (taken from a course by Andrew NG and edited):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3KixL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3KixL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Red arrows show the way of gradient backpropagation and you can see that in each step they are multiplied by a number (actually a matrix is multiplied in another matrix). If this number is less than one, then it results in vanishing gradient. If this number is greater than one it will result in exposure (which can be controlled by simply clipping it to a max value). The higher the steps, the more of vanishing or exploding effect.&lt;/p&gt;&#xA;&lt;p&gt;But they have found a solution for this problem, LSTM or GRU. What these units do is that they will make a highway for gradient to backpropagate through (in each state it will be multiplied by 1). So gradient can travel for longer distance (from French into France). Even though, the problem still exists for very long relations.&lt;br /&gt;&#xA;If you know what is a ResNet and how it works, you can find the same concept behind LSTM or GRU and ResNets as both make a highway for gradient to flow back.&lt;/p&gt;&#xA;&lt;p&gt;You can have an intuition of how LSTM or GRU works by following the forward pass and assuming what they are doing as locking the concepts in memory cells. Like when the network sees the word &lt;code&gt;France&lt;/code&gt; it will understand that it's an important word and maybe it will become handy later. So it will put the &lt;code&gt;France&lt;/code&gt; word in one of it's memory cells and keeps it there for several steps and when it wants to guess the language it will use this memory cell to predict &lt;code&gt;French&lt;/code&gt; not &lt;code&gt;English&lt;/code&gt; or &lt;code&gt;Persian&lt;/code&gt;. Then it can release that memory cell and use this cell for something else.&lt;/p&gt;&#xA;&lt;p&gt;If you want to learn more and get better intuition, I highly recommend you to look at this link: &lt;a href=&quot;https://www.youtube.com/watch?v=6niqTuYFZLQ&amp;amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;amp;index=10&quot; rel=&quot;nofollow noreferrer&quot;&gt;CS231n of MIT - lecture 10&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41547" LastEditorUserId="41547" LastEditDate="2020-10-21T13:26:58.703" LastActivityDate="2020-10-21T13:26:58.703" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24192" PostTypeId="1" CreationDate="2020-10-21T14:16:13.733" Score="3" ViewCount="18" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The basic idea of MFA is to perform &lt;strong&gt;subspace clustering&lt;/strong&gt; by assuming the covariance structure for each component of the form, &lt;span class=&quot;math-container&quot;&gt;$\Sigma_i = \Lambda_i \Lambda_i^T + \Psi_i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\Lambda_i \in \mathbb{R}^{D\times d}$&lt;/span&gt;, is the &lt;em&gt;factor loadings matrix&lt;/em&gt; with &lt;span class=&quot;math-container&quot;&gt;$d &amp;lt; D$&lt;/span&gt; for parsimonious representation of the data, and &lt;span class=&quot;math-container&quot;&gt;$Ψ_i$&lt;/span&gt; is the diagonal noise matrix. Note that the mixture of probabilistic principal component analysis (MPPCA) model is a special case of MFA with&#xA;the distribution of the errors assumed to be isotropic with &lt;span class=&quot;math-container&quot;&gt;$Ψ_i = Iσ_i^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What is meant by subspace clustering here, and how does &lt;span class=&quot;math-container&quot;&gt;$\Sigma_i = \Lambda_i \Lambda_i^T + \Psi_i$&lt;/span&gt; accomplish the same? I understand that this is a dimensionality reduction technique since &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(\Lambda_i) \leq d &amp;lt; D$&lt;/span&gt;. It'd be great if someone could help me understand more, and/or suggest resources I could look into for learning about this as an &lt;em&gt;absolute beginner&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From what I understand, &lt;span class=&quot;math-container&quot;&gt;$x = \Lambda z + u$&lt;/span&gt; is &lt;em&gt;one factor-analyzer&lt;/em&gt; (right?), i.e. the generative model in maximum likelihood factor analysis. &lt;a href=&quot;https://www.cs.toronto.edu/%7Ehinton/absps/tr-96-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; goes on to define a mixture of factor-analyzers indexed by &lt;span class=&quot;math-container&quot;&gt;$\omega_j$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$j = 1,...,m$&lt;/span&gt;. The generative model now obeys the distribution &lt;span class=&quot;math-container&quot;&gt;$$P(x) = \sum_{i=1}^m \int P(x|z,\omega_j)P(z|\omega_j)P(\omega_j)dz$$&lt;/span&gt; where, &lt;span class=&quot;math-container&quot;&gt;$P(z|\omega_j) = P(z) = \mathcal{N}(0,I)$&lt;/span&gt;. &lt;strong&gt;How does this help/achieve the desired objective?&lt;/strong&gt; Why take the sum from &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;? Where is subspace clustering happening, and what's happening on a high-level when we are using this mixture of factor-analyzers?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="35585" LastEditDate="2020-10-21T14:48:48.247" LastActivityDate="2020-10-21T14:48:48.247" Title="What is meant by subspace clustering in MFA?" Tags="&lt;clustering&gt;&lt;dimensionality-reduction&gt;&lt;statistics&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24193" PostTypeId="1" CreationDate="2020-10-21T14:30:49.440" Score="0" ViewCount="19" Body="&lt;p&gt;I am a beginner in AI methods. I have a collection of x(t) data, where x are some signal amplitudes and t is a time. My testing data are divided into two classes, say those from good and bad experimental samples. I need to classify the signals from unknown samples as good or bad according to their similarity to these two classes. What kind of a neural network is the best in this case? Could you recommend me some example in the literature where such a problem is considered?&lt;/p&gt;&#xA;" OwnerUserId="41761" LastActivityDate="2020-10-21T14:30:49.440" Title="What is the best neural network model to classify an x(t) signal according two classes?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24194" PostTypeId="1" AcceptedAnswerId="24196" CreationDate="2020-10-21T15:30:22.660" Score="0" ViewCount="82" Body="&lt;p&gt;I have created my own RL environment where I have a 2-dimensional matrix as a state space, the rows represent the users that are asking for a service, and 3 columns representing 3 types of users; so if a user U0 is of type 1 is asking for a service, then the first row would be (0, 1, 0) (first column is type 0, second is type 1...).&lt;/p&gt;&#xA;&lt;p&gt;The state space values are randomly generated each episode.&lt;/p&gt;&#xA;&lt;p&gt;I also have an action space, representing which resources were allocated to which users.&#xA;The action space is a 2-dimensional matrix, the rows being the resources that the agent has, and the columns represent the users. So, suppose we have 5 users and 6 resources, if user 1 was allocated resource 2, then the 3rd line would be like this: ('Z': a value zero was chosen, 'O': a value one was chosen)&#xA;(Z, O, Z, Z, Z)&lt;/p&gt;&#xA;&lt;p&gt;The possible actions are a list of tuples, the length of the list is equal to the number of users + 1, and the length of each tuple is equal to the number of users.&#xA;Each tuple has one column set to 'O', and the rest to 'Z'. (Each resource can be allocated to one user only). So the number of the tuples that have one column = 'O', is equal to the number of users, and then there is one tuple that has all columns set to 'Z', which means that the resource was not allocated to any users.&lt;/p&gt;&#xA;&lt;p&gt;Now, when the agent chooses the action, for the first resource it picks an action from the full list of possible action, then for the second resource, the action previously chosen is removed from the possible actions, so it chooses from the actions left, and so on and so forth; and that's because each user can be allocated one resource only. The action tuple with all 'Z' can always be chosen.&lt;/p&gt;&#xA;&lt;p&gt;When the agent allocates a resource to a user that didn't request a service, a penalty is given (varies with the number of users that didn't ask for a service but were allocated a resource), otherwise, a reward is given (also varies depending on the number of users that were satisfied).&lt;/p&gt;&#xA;&lt;p&gt;The problem is, the agent always tends to pick the same actions, and those actions are the tuple with all 'Z' for all the users. I tried to play with the q_values initial values; q_values is a dictionary with 2 keys: 1st key: the state being a tuple representing each possible state from the state space, meaning (0, 0, 0) &amp;amp; (1, 0, 0) &amp;amp; (0, 1, 0) &amp;amp; (0, 0, 1), combined with each action from the possible actions list.&#xA;I also tried different learning_rate values, different penalties and rewards etc. But it always does the same thing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5sLZb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5sLZb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MbxCV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MbxCV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42372" LastActivityDate="2020-10-21T16:07:20.383" Title="Q-learning agent stuck at taking same actions" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24195" PostTypeId="2" ParentId="24187" CreationDate="2020-10-21T16:00:08.733" Score="2" Body="&lt;p&gt;Assume the image can contain objects of class &lt;span class=&quot;math-container&quot;&gt;$C_1 \dots C_c$&lt;/span&gt;. Assume a set of additional inputs that has a meaning of questions as &amp;quot;contains the image a C_i or C_j or ... ?&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The main problem for the system is classify the image in classes &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;. Second problem is answer the implicit question proposed by the remainder inputs.&lt;/p&gt;&#xA;&lt;p&gt;Thus, better combine two NNs:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;first one an object recognizer, with input the image data.&lt;/li&gt;&#xA;&lt;li&gt;second one a NN that will answer the question implicit in remainder bits, with inputs the output of previous NN and the &amp;quot;question&amp;quot; bits.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In concrete for the example that the question describe:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a NN to classify dog/cat from 32x32 image. Two outputs for probabilities of dog an cat. Or simply one binary output 0:&amp;quot;is a dog&amp;quot;, 1:&amp;quot;is a cat&amp;quot;.&lt;/li&gt;&#xA;&lt;li&gt;a second NN with the &amp;quot;1025&amp;quot; binary input ( 0: &amp;quot;look for dogs&amp;quot;, 1: &amp;quot;look for cats&amp;quot;) and output from previous one. In this case, if everything is ok, it will infer the logic &amp;quot;a==b&amp;quot;: &amp;quot;is a dog&amp;quot; and &amp;quot;look for dogs&amp;quot; or &amp;quot;is a cat&amp;quot; and &amp;quot;look for cats&amp;quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that if you try to solve the problem directly with a full-connected NN with all inputs (1025 in the example), you loss the possibility of use CNN and max-pooling layers, etc. Moreover, split decreases training times, join subjects increases them &amp;quot;exponentially&amp;quot;. Not a promising way.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-21T16:06:00.370" LastActivityDate="2020-10-21T16:06:00.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24196" PostTypeId="2" ParentId="24194" CreationDate="2020-10-21T16:07:20.383" Score="0" Body="&lt;p&gt;I am confused. For the initial &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values, you generate one for each possible row &lt;span class=&quot;math-container&quot;&gt;$(1, 0, 0), (0,0,0), \ldots$&lt;/span&gt; so you would have 4 states.&lt;/p&gt;&#xA;&lt;p&gt;However, from the first paragraph it seems that the states themselves are matrices (one row for each user), so the state space is a set of such matrices.&lt;/p&gt;&#xA;&lt;p&gt;That means that your &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-table should have a row for each possible matrix, and a column for each possible total assignment of items to users.&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-10-21T16:07:20.383" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24197" PostTypeId="1" AcceptedAnswerId="24208" CreationDate="2020-10-21T16:19:38.077" Score="2" ViewCount="40" Body="&lt;p&gt;I understand that Hidden Markov Models are used to learn about hidden variables &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; with the help of observable variables &lt;span class=&quot;math-container&quot;&gt;$\xi_i$&lt;/span&gt;. On Wikipedia, I read that while the &lt;span class=&quot;math-container&quot;&gt;$\xi_i$&lt;/span&gt;'s can be continuous (say Gaussian), the &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt;'s are &lt;strong&gt;discrete&lt;/strong&gt;. Is this necessary, and why? Are there ways in which I could extend this to continuous domains?&lt;/p&gt;&#xA;" OwnerUserId="35585" LastEditorUserId="2444" LastEditDate="2020-10-25T09:49:48.960" LastActivityDate="2020-10-25T09:49:48.960" Title="Is there an equivalent model to the Hidden Markov Model for continuous hidden variables?" Tags="&lt;machine-learning&gt;&lt;hidden-markov-model&gt;&lt;kalman-filter&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24198" PostTypeId="2" ParentId="24187" CreationDate="2020-10-21T16:34:50.340" Score="2" Body="&lt;p&gt;The main benefit of deep learning is that you don't have to manually design features.&lt;/p&gt;&#xA;&lt;p&gt;Classic Machine Learning algorithms always include the &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feature engineering&lt;/a&gt; step, whereas neural networks are able to crate features automatically during learning. The classic example is CNN. In the first layer it creates simple features that representing lines, the last layers represent abstract features. Of course, some tasks do require feature engineering (e.g. signal processing).&lt;/p&gt;&#xA;&lt;p&gt;In your case, if you want to take advantage of the CNN network, you can also add an additional input layer for the flag (e.g. as one-hot vector). Here is an illustration taken from this &lt;a href=&quot;https://stackoverflow.com/questions/53516662/two-inputs-to-one-model-in-keras&quot;&gt;answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cSC40m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cSC40m.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-10-21T22:18:01.597" LastActivityDate="2020-10-21T22:18:01.597" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24199" PostTypeId="1" AcceptedAnswerId="24210" CreationDate="2020-10-21T16:37:55.903" Score="3" ViewCount="60" Body="&lt;p&gt;Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously? Is there an agreed up protocol on how to extract this information?&lt;/p&gt;&#xA;&lt;p&gt;What combination of layers works: convolution + LSTM? What would be the alternatives?&lt;/p&gt;&#xA;" OwnerUserId="32434" LastEditorUserId="2444" LastEditDate="2020-10-21T23:15:45.460" LastActivityDate="2020-10-22T07:37:35.613" Title="Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24200" PostTypeId="1" CreationDate="2020-10-21T17:35:37.533" Score="1" ViewCount="65" Body="&lt;p&gt;All else being equal, including total neuron count, I give the following definitions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;wide is a parallel ensemble, where good chunks of the neurons have the same inputs because the inputs are shared and they have different outputs.&lt;/li&gt;&#xA;&lt;li&gt;deep is a series ensemble, where for the most part neurons have as input the output of other neurons and few inputs are shared.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For CART ensembles the parallel (wide) ensemble is a random forest while the series (deep) ensemble is a gradient boosted machine.  For several years the GBM was the &lt;a href=&quot;https://www.kdnuggets.com/2016/03/xgboost-implementing-winningest-kaggle-algorithm-spark-flink.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;winningest&amp;quot; on kaggle&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is there a parallel of that applied to Neural networks?  Is there some reasonable measure that indicates whether deep outperforms wide when it comes to neural networks?  If I had the same count of weights to throw at a tough problem, all else being equal should they be applied more strongly in parallel or in series?&lt;/p&gt;&#xA;" OwnerUserId="2263" LastActivityDate="2020-10-22T07:48:52.313" Title="Has &quot;deep vs. wide&quot; been resolved?" Tags="&lt;deep-neural-networks&gt;&lt;gradient-boosting&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24201" PostTypeId="1" CreationDate="2020-10-21T14:33:40.110" Score="2" ViewCount="265" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt;&lt;br /&gt;&#xA;Did AlphaGo and AlphaGo play 100 repetitions of the same sequence of boards, or were there 100 different games?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;br /&gt;&#xA;Alphago was the first superhuman go player, but it had human tuning and training.&lt;/p&gt;&#xA;&lt;p&gt;AlphaGo zero learned to be more superhuman than superhuman.  Its supremacy was shown by how it beat AlphaGo perfectly in 100 games.&lt;/p&gt;&#xA;&lt;p&gt;My understanding of AlphaGo and AlphaGo are that they are deterministic, not stochastic.&lt;/p&gt;&#xA;&lt;p&gt;If they are deterministic, then given a board position they will always make the same move.&lt;/p&gt;&#xA;&lt;p&gt;The way that mathematicians count the possible games in chess is to account for different board positions.  As I understand it, and I could be wrong, if they have the exact same sequence of board positions then it does not count as a different game.&lt;/p&gt;&#xA;&lt;p&gt;If they make the same sequence of moves 100 times, then they did not play 100 different games, but played one game for 100 repetitions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;br /&gt;&#xA;So, using the mathematical definition, did AlphaGo and AlphaGo Zero play only one game for 100 iterations or did they play 100 different games?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;References:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.scientificamerican.com/article/ai-versus-ai-self-taught-alphago-zero-vanquishes-its-predecessor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scientificamerican.com/article/ai-versus-ai-self-taught-alphago-zero-vanquishes-its-predecessor/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://deepmind.com/blog/article/alphago-zero-starting-scratch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://deepmind.com/blog/article/alphago-zero-starting-scratch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://mathworld.wolfram.com/Stochastic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mathworld.wolfram.com/Stochastic.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2263" OwnerDisplayName="EngrStudent" LastEditorUserId="2263" LastEditDate="2020-10-21T17:58:32.550" LastActivityDate="2020-10-21T21:36:57.400" Title="Did Alphago zero actually beat Alphago 100 games to 0?" Tags="&lt;alphago-zero&gt;&lt;alphago&gt;&lt;deterministic-policy&gt;&lt;stochastic-policy&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24203" PostTypeId="2" ParentId="24201" CreationDate="2020-10-21T19:17:21.453" Score="5" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Did AlphaGo and AlphaGo [Zero] play 100 repetitions of the same sequence of boards, or were there 100 different games?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There were 100 different games. &lt;a href=&quot;https://www.alphago-games.com/#agzero_vs_aglee&quot; rel=&quot;nofollow noreferrer&quot;&gt;You can view some example games between AlphaGo [Lee] and AlphaGo Zero here&lt;/a&gt;. They are clearly all different.&lt;/p&gt;&#xA;&lt;p&gt;This statement in the question shows a misunderstanding:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My understanding of AlphaGo and AlphaGo [Zero] are that they are deterministic, not stochastic.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search (MCTS)&lt;/a&gt; algorithm used for look-ahead planning in AlphaGo and Alpha Zero is inherently stochastic. It samples from the huge tree of possibilities in a game like Go by making weighted random choices at all branch points. That means play can progress stochastically with two such agents opposing each other, as many board states will resolve into selecting semi-randomly between &amp;quot;best&amp;quot; moves that would be very closely ranked by each agent in the limit of very long search times.&lt;/p&gt;&#xA;&lt;p&gt;Whilst this solves the main point of your question, it is worth noting that there can be a related effect in self-play algorithms, even if they are partially stochastic. That is, it is possible to have one agent that develops a specific weakness by chance, that another agent consistently takes advantage of, such that agent A consistently beats agent B, and wins in a very similar fashion each time (maybe deterministically, maybe across a range of different games all with a similar mistake). However it may be the case that also:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Neither agent is strong in general.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another agent C can beat B consistently, but will lose to A consistently. There would then be no clear way to rank agents A, B, and C without further measurements.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Agents trained through self play therefore do need to be trained and tested against a wide range of opponents to verify this is not happening and that the  skill level assessment is valid more generally. I believe this was done with all the AlphaGo variants built by DeepMind.&lt;/p&gt;&#xA;&lt;p&gt;The MCTS algorithm does help a little with this scenario as it can correct for weaknesses in how a trained neural network rates early board positions. The look-ahead planning of MCTS makes initial ratings less relevant to eventual action selection - effectively it refines those learned ratings using the samples from current position.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-21T21:36:57.400" LastActivityDate="2020-10-21T21:36:57.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24204" PostTypeId="1" AcceptedAnswerId="24205" CreationDate="2020-10-21T20:35:12.847" Score="2" ViewCount="116" Body="&lt;p&gt;My question is about &lt;strong&gt;neuroevolution&lt;/strong&gt; (genetic algorithm + neural network): I want to create &lt;strong&gt;artificial life&lt;/strong&gt; by evolving agents. But instead of relying on a fitness function, I would like to have the agents reproduce with some mutation applied to the genes of their offspring and have some agents die through natural selection. Achieve evolution in this manner is my goal.&lt;/p&gt;&#xA;&lt;p&gt;Is this feasible? And has there been some prior work on this? Also, is it somehow possible to incorporate &lt;strong&gt;NEAT&lt;/strong&gt; into this scheme?&lt;/p&gt;&#xA;&lt;p&gt;So far, I've implemented most of the basics in amethyst (a parallel game engine written in Rust), but I'm worried that the learning will happen very slowly. Should I approach this problem differently?&lt;/p&gt;&#xA;" OwnerUserId="41768" LastEditorUserId="2444" LastEditDate="2020-10-25T10:01:41.063" LastActivityDate="2020-10-25T14:01:18.667" Title="Is it possible to perform neuroevolution without a fitness function?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;neuroevolution&gt;&lt;fitness-functions&gt;&lt;artificial-life&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24205" PostTypeId="2" ParentId="24204" CreationDate="2020-10-21T21:29:30.410" Score="2" Body="&lt;p&gt;You do not always need an &lt;em&gt;explictly coded&lt;/em&gt; fitness function to perform genetic algorithm searches. The more general need is for a selection process that favours individuals that perform better at the core tasks in an environment (i.e. that are &amp;quot;more fit&amp;quot;). One way of assessing performance is to award a numerical score, but other approaches are possible, including:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tournament_selection&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tournament selection&lt;/a&gt; where two or more individuals compete in a game, and the winner is selected.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Opportunity-based selection, where agents in a shared environment - typically with limited resources and chances to compete - may reproduce as one of the available actions, provided they meet some criteria such as having collected enough of some resource. I was not able to find a canonical name for this form of selection, but it is commonly implemented in artificial life projects.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A key distinction between &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_life&quot; rel=&quot;nofollow noreferrer&quot;&gt;A-life&lt;/a&gt; projects and GA optimisation projects is that in A-life projects there is no goal behaviour or target performance. Typically A-life projects are simulations with an open ended result and the developer runs a genetic algorithm to &amp;quot;see what happens&amp;quot; as opposed to &amp;quot;make the best game-player&amp;quot;. If your project is like this then you are most likely looking for the second option here.&lt;/p&gt;&#xA;&lt;p&gt;To discover more details about this kind of approach, you could try searching &amp;quot;artifical life genetic algorithms&amp;quot; as there are quite a few projects of this type published online, some of which use NEAT.&lt;/p&gt;&#xA;&lt;p&gt;Technically, you could view either of the methods listed above as ways of &lt;em&gt;sampling&lt;/em&gt; comparisons between individuals against an unknown fitness function. Whether or not a true fitness function could apply is then partly a matter of philosophy. More importantly for you as the developer, is that you do not have to &lt;em&gt;write&lt;/em&gt; one. Instead you can approximately measure fitness using various methods of individual selection.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So far I've implemented most of the basics in amethyst (a parallel game engine written in rust), but I'm worried that the learning will happen very slowly. Should I approach this problem differently?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is difficult to say whether you should approach the problem differently. However, the biggest bottlenecks against successful GA approaches are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Time/CPU resources needed to assess agents.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Size of search space for genomes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both of these can become real blockers for ambitious a-life projects. It is common to heavily simplify agents and environments in attempts address these issues.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-25T14:01:18.667" LastActivityDate="2020-10-25T14:01:18.667" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="24206" PostTypeId="2" ParentId="23618" CreationDate="2020-10-21T22:06:31.717" Score="0" Body="&lt;p&gt;The economic value would be high indeed, as, combined with robotics, AGI would be able to replace all human workers.  So:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Whatever the economic value of the sum of human labor is, in an ideal sense&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Of course, there would also be the question of the &lt;strong&gt;cost of computation&lt;/strong&gt;, the cost of the hardware &amp;amp; software required for AGI, and whether that cost is higher or lower than the cost of human labor.  (My guess is biological machines such as humans would be cheaper, both in production and processing, until AGI leverages molecular computing via an inexpensive, ubiquitous substrate.  Also worth noting that biological systems such as humans and canines may be more fault-tolerant, and more resilient in that they can persist even where the technological base collapses.)&lt;/p&gt;&#xA;&lt;p&gt;Currently, cost of training even narrowly superintelligent Neural Networks which exceed humans at a single function is extremely high.&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2020-10-21T22:56:43.323" LastActivityDate="2020-10-21T22:56:43.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24208" PostTypeId="2" ParentId="24197" CreationDate="2020-10-22T05:31:17.630" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kalman filter&lt;/a&gt; is what you're looking for.&lt;/p&gt;&#xA;&lt;p&gt;According to Wikipedia:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The Kalman filter may be regarded as analogous to the hidden Markov model, with the key difference that the hidden state variables take values in a continuous space (as opposed to a discrete state space as in the hidden Markov model).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="41777" LastActivityDate="2020-10-22T05:31:17.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24209" PostTypeId="2" ParentId="24188" CreationDate="2020-10-22T07:20:07.590" Score="-1" Body="&lt;p&gt;Access to the back of the store, basically if you can track them entering and leaving an employees only area.&lt;/p&gt;&#xA;&lt;p&gt;But this requires a really good tracking algorithm and if you lose track you're going to have to fall back on some other properties.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-10-22T07:20:07.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24210" PostTypeId="2" ParentId="24199" CreationDate="2020-10-22T07:37:35.613" Score="3" Body="&lt;p&gt;Yes, there are different ways. What I think you are looking for is under the research field of Localization and Mapping. Which divides in the following subfields:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ROrlO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ROrlO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For getting current (the robot) position and trajectory go to models for Odometry Estimation&lt;/li&gt;&#xA;&lt;li&gt;For getting a representation of the world around the robot go to models for Mapping&lt;/li&gt;&#xA;&lt;li&gt;If you want &lt;strong&gt;both of them&lt;/strong&gt; (I am guessing you want). Go for SLAM (Simultaneous Localization and Mapping) models&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here it is the amazing survey that links you to tons of papers with different models for each category. If you want to know what are the most common architectural blocks (LSTM, ConvLSTM, RNN...) used for your problem, read the most promising papers under your target category.&lt;/p&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;p&gt;Survey: &lt;a href=&quot;https://arxiv.org/abs/2006.12567&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/2006.12567&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-10-22T07:37:35.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24211" PostTypeId="2" ParentId="24200" CreationDate="2020-10-22T07:48:52.313" Score="2" Body="&lt;p&gt;I am not sure what you are really looking for but I leave here &lt;a href=&quot;https://arxiv.org/pdf/1905.11946.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; here, where some intuition into that direction is provided. This paper compares the performance of a deep learning model scaling in 3 dimensions: resolution, width and depth. As depicted in their definition:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DqVpQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DqVpQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you go to section 3.2 you will see how scaling the different dimensions independently (resolution, width and depth) affects performance and how by performing a compound scaling the performance of the model is maximized (so there is a close relation). It is a very thorough ablation study. For me this was the paper where I understood how these Width, Depth and Resolution parameters come together.&lt;/p&gt;&#xA;&lt;p&gt;Rethinking Model Scaling for Convolutional Neural Network: &lt;a href=&quot;https://arxiv.org/abs/1905.11946&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1905.11946&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-10-22T07:48:52.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24213" PostTypeId="5" CreationDate="2020-10-22T09:48:31.453" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-22T09:48:31.453" LastActivityDate="2020-10-22T09:48:31.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24214" PostTypeId="4" CreationDate="2020-10-22T09:48:31.453" Score="0" Body="For questions about the different techniques of initializing weights (or parameters) of machine learning models." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-22T09:48:31.453" LastActivityDate="2020-10-22T09:48:31.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24217" PostTypeId="2" ParentId="15936" CreationDate="2020-10-22T20:29:26.310" Score="2" Body="&lt;p&gt;A few of us have spent quite a bit of time thinking about this. I summarised our work in a Medium article here: &lt;a href=&quot;https://towardsdatascience.com/deep-learning-vs-puzzle-games-e996feb76162&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/deep-learning-vs-puzzle-games-e996feb76162&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Would love to hear what you think.&lt;/p&gt;&#xA;&lt;p&gt;Spoiler: so far, good old SAT seems to beat fancy AI algorithms!&lt;/p&gt;&#xA;" OwnerUserId="41793" LastActivityDate="2020-10-22T20:29:26.310" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24219" PostTypeId="1" AcceptedAnswerId="24229" CreationDate="2020-10-23T00:24:12.980" Score="1" ViewCount="76" Body="&lt;p&gt;Given a video, I'm trying to classify whether it is a graphical (computer-generated) or realistic scene. For instance, if it contains computer-generated graphics, credit, moving bugs, blue screen, etc. it will be computer-generated graphics, and if it is a realistic scene captured by camera, it will be a realistic scene.&lt;/p&gt;&#xA;&lt;p&gt;How can we achieve that with AI? Do we have any working solutions available?&lt;/p&gt;&#xA;&lt;p&gt;Some examples of graphical scenes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xiantm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xiantm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4TxE5m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4TxE5m.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SxOUAm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SxOUAm.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="2444" LastEditDate="2020-10-25T09:57:30.143" LastActivityDate="2020-10-25T09:57:30.143" Title="How can I determine whether a video's frame is realistic (was recorded by a camera) or contains computer-generated graphics?" Tags="&lt;deep-learning&gt;&lt;classification&gt;&lt;ai-design&gt;&lt;object-detection&gt;&lt;scene-classification&gt;" AnswerCount="1" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24221" PostTypeId="1" CreationDate="2020-10-23T10:32:13.097" Score="2" ViewCount="120" Body="&lt;p&gt;In the literature, there are at least two action selection strategies associated with the UCB1's action selection strategy/policy. For example, in the paper &lt;a href=&quot;https://arxiv.org/pdf/1402.6028.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithms for the multi-armed bandit problem&lt;/a&gt; (2000/2014), at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, an action is selected using the following formula&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;a^*(t) \doteq \arg \max _{i=1 \ldots k}\left(\hat{\mu}_{i}+\sqrt{\frac{2 \ln t}{n_{i}}}\right) \tag{1}\label{1},&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat{\mu}_{i}$&lt;/span&gt; is an estimate of the expected return for arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; is the number of times the action &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is selected&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of arms/actions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;On the other hand, &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=57&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto&lt;/a&gt; (2nd edition of the book) provide a slightly different formula (equation 2.10)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;a^*(t) \doteq \arg \max _{i=1 \ldots k}\left(\hat{\mu}_{i}+c\sqrt{\frac{\ln t}{n_{i}}}\right) \tag{2}\label{2},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$c &amp;gt; 0$&lt;/span&gt; is a hyper-parameter that controls the amount of exploration (as explained in the book or &lt;a href=&quot;https://ai.stackexchange.com/a/20170/2444&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Why do we have these two formulas? I suppose that both are &amp;quot;upper confidence bounds&amp;quot; (and, in both cases, they are constants, though one is a hyper-parameter), but why (and when) would we use one over the other? They are not equivalent because &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; only needs to be greater than &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, i.e. it can be arbitrarily large (although, in the mentioned book, the authors use &lt;span class=&quot;math-container&quot;&gt;$c=2$&lt;/span&gt; in one experiment/figure). If &lt;span class=&quot;math-container&quot;&gt;$c = \sqrt{2}$&lt;/span&gt;, then they are the same.&lt;/p&gt;&#xA;&lt;p&gt;The answer to my question can probably be found in &lt;a href=&quot;https://homes.di.unimi.it/cesa-bianchi/Pubblicazioni/ml-02.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper that introduced UCB1&lt;/a&gt; (which actually defines the UCB1 as in \ref{1}),  or in a paper that derives the bound, in the sense that the bound probably depends on some probability of error, but I have not fully read it yet, so, if you know the answer, feel free to derive both bounds and relate the two formulas.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-23T16:35:38.347" LastActivityDate="2020-10-23T19:06:22.403" Title="Why do we have two similar action selection strategies for UCB1?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;sutton-barto&gt;&lt;multi-armed-bandit&gt;&lt;upper-confidence-bound&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24222" PostTypeId="1" AcceptedAnswerId="24225" CreationDate="2020-10-23T10:59:09.280" Score="2" ViewCount="56" Body="&lt;p&gt;I am reading this blog post: &lt;a href=&quot;https://ruder.io/optimizing-gradient-descent/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ruder.io/optimizing-gradient-descent/index.html&lt;/a&gt;. In the section about AdaGrad, it says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It adapts the learning rate to the parameters, performing smaller updates (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. high learning rates) for parameters associated with infrequent features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But I am not sure about the meaning of &lt;em&gt;infrequent features&lt;/em&gt;: is it that the value of a given feature changes rarely?&lt;/p&gt;&#xA;" OwnerUserId="41801" LastEditorUserId="2444" LastEditDate="2020-10-23T14:58:40.913" LastActivityDate="2020-10-23T15:28:50.167" Title="What do we mean by &quot;infrequent features&quot;?" Tags="&lt;deep-learning&gt;&lt;terminology&gt;&lt;optimizers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24223" PostTypeId="5" CreationDate="2020-10-23T11:48:44.920" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-23T11:48:44.920" LastActivityDate="2020-10-23T11:48:44.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24224" PostTypeId="4" CreationDate="2020-10-23T11:48:44.920" Score="0" Body="For questions about the upper confidence bound (UCB)-based algorithms or action selection strategies in the context e.g. of bandit or reinforcement learning problems." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-23T11:48:44.920" LastActivityDate="2020-10-23T11:48:44.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24225" PostTypeId="2" ParentId="24222" CreationDate="2020-10-23T11:53:51.970" Score="2" Body="&lt;p&gt;We will describe the input to the network as a vector, called features vector. Each component of this vector is usually related to some &amp;quot;real world&amp;quot; information, by example &amp;quot;age of the person&amp;quot;, &amp;quot;number of atoms&amp;quot;, &amp;quot;...&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In very usual situations, a specific component of the input vector will have near than always the same value. This is more usual in binary components or components that has a small set of possible values.&lt;/p&gt;&#xA;&lt;p&gt;However, usually &lt;strong&gt;in the cases where this component has a value different from the most usual one, this component is very important and informative&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;These values of this kind of components are called &lt;em&gt;infrequent features&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;(example: &amp;quot;rains?&amp;quot; is in my city 99.9% of time &amp;quot;false&amp;quot;. However, when it is true, it is a key factor to all questions about the behavior of the population).&lt;/p&gt;&#xA;&lt;p&gt;The problem with these features: as unusual values are infrequent, the net has few chances to learn from them, and some learning algorithms could fail to give them the weight that they must (taken into account that, as has been said, these components are very important when they take a value different to the most frequent one).&lt;/p&gt;&#xA;&lt;p&gt;Some adaptive learning rate algorithms as AdaGrad tries to solve this issue.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-23T15:28:50.167" LastActivityDate="2020-10-23T15:28:50.167" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24226" PostTypeId="1" CreationDate="2020-10-23T13:01:31.147" Score="1" ViewCount="28" Body="&lt;p&gt;I am working on a project where I encountered a component which takes 96 arguments (all integer values) and outputs 12 float values.&#xA;I would like to find a useful combination of these 96 values to receive the output that I want while avoiding random guessing, so the desired behavior would be that I provide the outcome and receive the 96 inputs to use them in my component.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately I am not that experienced in that field. If I think about how I can implement this, my first thought was a kind of classification task, since I could build a dataset but the problem here is that I need integer values.&lt;/p&gt;&#xA;&lt;p&gt;A second guess was a regression but would that be possible for y as an output vector?&#xA;Are there some other approaches that could fit to my use case?&lt;/p&gt;&#xA;" OwnerUserId="41804" LastActivityDate="2020-10-23T13:01:31.147" Title="How to find a parameter combination for a black box using AI?" Tags="&lt;classification&gt;&lt;ai-design&gt;&lt;ai-basics&gt;&lt;regression&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24227" PostTypeId="2" ParentId="24221" CreationDate="2020-10-23T19:06:22.403" Score="1" Body="&lt;p&gt;In the PDF of the original paper for UCB1 you linked, in page 242-243 the authors proves why non-optimal machines get played much less (in fact, logarithmically less) than the optimal ones. &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; decides whether they indeed would, and &lt;span class=&quot;math-container&quot;&gt;$c=\sqrt{2}$&lt;/span&gt; is the minimum choice of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We want to show that the number of runs for non-optimal machines (&lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt;, for non-optimal &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;s, in your notation) is asymptotically logarithmic. In other words, you may run them for a few times and well, it's fine, but not too often. We're devising some indicator value &lt;span class=&quot;math-container&quot;&gt;$a_i(t)=\hat \mu_i+(\epsilon...)$&lt;/span&gt; such that the &lt;em&gt;mistaken&lt;/em&gt; cases, where values of non-optimal ones surpass values of optimal ones (&lt;span class=&quot;math-container&quot;&gt;$a^*(t)&amp;lt;a_i(t)$&lt;/span&gt;), are minimized.&lt;/p&gt;&#xA;&lt;p&gt;Think about the last inequality. We know that &lt;span class=&quot;math-container&quot;&gt;$\mu^* &amp;gt; \mu_i$&lt;/span&gt; (again, optimal and non-optimal ones). Therefore, for that inequality to be true, it seems either the left-hand side should be &lt;em&gt;quite&lt;/em&gt; small or the right-hand side should be &lt;em&gt;quite&lt;/em&gt; large. But wait, &lt;span class=&quot;math-container&quot;&gt;$\hat \mu$&lt;/span&gt;s are actually some random trials for &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, so we cannot claim directly from &lt;span class=&quot;math-container&quot;&gt;$\mu^* &amp;gt; \mu_i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\hat{\mu^*} &amp;gt; \hat{\mu_i}$&lt;/span&gt;; it might be that we just need more trials.&lt;/p&gt;&#xA;&lt;p&gt;The equations (7), (8) and (9) of the paper is the three conditions mentioned in the paragraph above; left-hand side is small, right-hand side is large or the trials are lacking. Well, in fact, as we stated &lt;em&gt;the number of runs ... is &lt;strong&gt;asymptotically&lt;/strong&gt; logarithmic&lt;/em&gt; at first, the third case can be eliminated(!), assuming that we've run this machine enough.&lt;/p&gt;&#xA;&lt;p&gt;For the first and second case, since &lt;span class=&quot;math-container&quot;&gt;$\hat \mu_i$&lt;/span&gt; is the average of some random variable in &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;, we can use Chernoff–Hoeffding bound (or so called in the paper; stated as &lt;a href=&quot;https://en.wikipedia.org/wiki/Hoeffding%27s_inequality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hoeffding's inequality&lt;/a&gt; in Wikipedia). Now, a good choice of &lt;span class=&quot;math-container&quot;&gt;$(\epsilon ...)$&lt;/span&gt; will guarantee (from Hoeffding's inequality) that the first two cases will occur &lt;em&gt;sufficiently scarcely&lt;/em&gt;, or in other words, in the order of &lt;span class=&quot;math-container&quot;&gt;$t^{-4}$&lt;/span&gt;. To achieve this, we need &lt;span class=&quot;math-container&quot;&gt;$c \ge \sqrt 2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now back to the third case, the &lt;em&gt;enough&lt;/em&gt; number of runs is actually &lt;span class=&quot;math-container&quot;&gt;$l = \left \lceil 2 c^2 \ln n / (\mu^* - \mu_i)^2\right \rceil$&lt;/span&gt;. Thus, you may choose larger &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; but receive longer convergence speed in penalty.&lt;/p&gt;&#xA;&lt;p&gt;Funnily enough, after all the proofs the authors find &lt;span class=&quot;math-container&quot;&gt;$c=1/4$&lt;/span&gt; to converge well and actually perform substantially better(!!) than &lt;span class=&quot;math-container&quot;&gt;$c=\sqrt{2}$&lt;/span&gt;. It seems they could not prove the bound as we did above.&lt;/p&gt;&#xA;" OwnerUserId="41808" LastActivityDate="2020-10-23T19:06:22.403" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24228" PostTypeId="1" CreationDate="2020-10-23T19:43:10.450" Score="0" ViewCount="44" Body="&lt;p&gt;&lt;strong&gt;Algorithm:&lt;/strong&gt; Classification by k-nearest neighbors with Euclidean distance (neighbors.KNeighborsClassifier).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Determine the important hyperparameters (2 maximum) that can significantly influence learning outcomes. Also give the range of values that should be tested for each of these hyperparameters in order to make a good fit for the configuration of the classifiers on the Pendigits game.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The hyper-paramaters for the KNN might be the number of neighbors and weights. How can we give a range of values that make a good fit?&lt;/p&gt;&#xA;" OwnerUserId="41202" LastEditorUserId="2444" LastEditDate="2020-10-26T08:31:32.323" LastActivityDate="2020-10-26T13:49:16.867" Title="How can I determine the k-NN's hyper-parameters and their range that significantly influence the outcome of classification?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;hyper-parameters&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24229" PostTypeId="2" ParentId="24219" CreationDate="2020-10-23T21:36:18.933" Score="2" Body="&lt;p&gt;As per your requirements, I would suggest that you start with any simple CNN network.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;CNNs take advantage of the hierarchical pattern in data and assemble&#xA;more complex patterns using smaller and simpler patterns. Therefore,&#xA;on the scale of connectedness and complexity, CNNs are on the lower&#xA;extreme.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here is a Keras example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = models.Sequential()&#xA;model.add(layers.Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=image_shape))&#xA;model.add(layers.MaxPooling2D((2, 2)))&#xA;model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))&#xA;model.add(layers.MaxPooling2D((2, 2)))&#xA;model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))&#xA;model.add(layers.Flatten())&#xA;model.add(layers.Dense(64, activation='relu'))&#xA;# output layer&#xA;model.add(layers.Dense(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;image_shape&lt;/code&gt; is the resolution and number of channels of images (e.g. 128x128x3 for RGB images). I also suggest you downscaling the images to a lower resolution. You will also have to crop the images as they must all be the same &lt;code&gt;image_shape&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Also take a look at the &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D&quot; rel=&quot;nofollow noreferrer&quot;&gt;MaxPooling2D&lt;/a&gt;  and &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;BatchNormalization&lt;/a&gt; layers.&lt;/p&gt;&#xA;&lt;p&gt;Since you only have real and CGI images, this becomes a binary classification problem. Therefore you can have a single output (0 - CGI, 1 - real). Such problems can be solved with &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/losses/BinaryCrossentropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;BinaryCrossentropy&lt;/a&gt; loss.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.compile(loss=losses.BinaryCrossentropy(from_loits=True), optimizer='adam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, you can fit your model&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;history = model.fit(train_images, train_labels, epochs=1000, validation_data=(test_images, test_labels))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can find a complete example &lt;a href=&quot;https://www.tensorflow.org/tutorials/images/cnn&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Please note that depending on your data, the model can become biased if your dataset is unbalanced. That is, if all of your CGI images have text, and only a small fraction of the real images also have text, they might be misclassified. Therefore, I recommend that you &lt;a href=&quot;https://cs231n.github.io/understanding-cnn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;visualize&lt;/a&gt; your model to better understand what it has learned. Here is an &lt;a href=&quot;https://arxiv.org/pdf/1902.10178.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; of such a problem we faced at our university.&lt;/p&gt;&#xA;&lt;p&gt;There are also more advanced CNN architectures such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Residual_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNet&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1409.1556&quot; rel=&quot;nofollow noreferrer&quot;&gt;VGG&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/abs/1506.02640&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLO&lt;/a&gt;. You can also extend your model with time series (i.e. video) using &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSTM&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Gated_recurrent_unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRU&lt;/a&gt; architecture.&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-10-24T09:48:05.107" LastActivityDate="2020-10-24T09:48:05.107" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24231" PostTypeId="1" AcceptedAnswerId="24240" CreationDate="2020-10-24T00:55:13.220" Score="4" ViewCount="120" Body="&lt;p&gt;I started reading some reinforcement learning literature, and it seems to me that all approaches to solving reinforcement learning problems are about finding the value function (state-value function or action-state value function).&lt;/p&gt;&#xA;&lt;p&gt;Are there any algorithms or methods that do not try to calculate the value function but try to solve a reinforcement learning problem differently?&lt;/p&gt;&#xA;&lt;p&gt;My question arose because I was not convinced that there is no better approach than finding the value functions. I am aware that given the value function we can define an optimal policy, but are there not other ways to find such an optimal policy?&lt;/p&gt;&#xA;&lt;p&gt;Also, is the reason why I don't encounter any non value-based methods that these are just less successful?&lt;/p&gt;&#xA;" OwnerUserId="36978" LastEditorUserId="2444" LastEditDate="2020-11-01T19:10:15.523" LastActivityDate="2020-11-11T15:11:33.770" Title="Is reinforcement learning only about determining the value function?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;actor-critic-methods&gt;&lt;policy-based-methods&gt;&lt;value-based-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24232" PostTypeId="1" AcceptedAnswerId="24234" CreationDate="2020-10-24T01:08:55.440" Score="1" ViewCount="177" Body="&lt;p&gt;I have seen 2 forms of softmax cross-entropy loss and are confused by the two. Which one is the right one?&#xA;For example in this &lt;a href=&quot;https://www.quora.com/Is-the-softmax-loss-the-same-as-the-cross-entropy-loss&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quora answer&lt;/a&gt;, there are 2 answers:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L(\mathbf{w})=\frac{1}{N} \sum_{n=1}^{N} H\left(p_{n}, q_{n}\right)=-\frac{1}{N} \sum_{n=1}^{N}\left[y_{n} \log \hat{y}_{n}+\left(1-y_{n}\right) \log \left(1-\hat{y}_{n}\right)\right]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathrm{L}(y, \hat{y})=-\Sigma y(i) \log \hat{y}(i)$&lt;/span&gt;, which is only the first part of the version one.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23549" LastEditorUserId="2444" LastEditDate="2020-10-24T01:29:48.130" LastActivityDate="2020-10-24T09:16:16.777" Title="Why are there two versions of softmax cross entropy? Which one to use in what situation?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;cross-entropy&gt;&lt;softmax&gt;&lt;categorical-crossentropy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24234" PostTypeId="2" ParentId="24232" CreationDate="2020-10-24T09:07:50.660" Score="4" Body="&lt;p&gt;It's the same thing, first version is the special case of the more general one. In the first case you only have two classes, it's binary cross-entropy, and they also included iteration over batch of samples. In the second case you have multiple classes and in the current form it's only for a single sample.&lt;/p&gt;&#xA;&lt;p&gt;In the first case there is only one output, if you had two outputs it would have been&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;-\frac{1}{N} \sum_{n=1}^N \sum_{j=1}^2 y_{n,j} \log(\hat y_{n,j})&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; iterates over batch samples, and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; over two classes. The reason why it was written like that is that if you have two classes you can have only one output because you can immediately conclude about probability of the second class if you have the probability of the first class, it would simply be &lt;span class=&quot;math-container&quot;&gt;$p_1 = 1-p_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the second case, with batch samples included, it would be&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; -\frac{1}{N} \sum_{n=1}^N \sum_{j=1}^c y_{n,j} \log(\hat y_{n,j})&#xA;\end{equation}&lt;/span&gt;&#xA;wheren &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; iterates over batch samples, and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; output classes.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2020-10-24T09:07:50.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24236" PostTypeId="2" ParentId="23708" CreationDate="2020-10-24T09:46:53.173" Score="0" Body="&lt;p&gt;If the filter is separable, that is, the NxM kernel can be mathematically equal to the convolution of a Nx1 filter and a 1xM filter, there are a very important increase in performance.&lt;/p&gt;&#xA;&lt;p&gt;Using separable convolution, the network can made an optimal usage of the shared memory and of the parallelism in memory access. See &lt;a href=&quot;https://www.google.es/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;ved=2ahUKEwitnfqX98zsAhULkhQKHREPDtwQFjAAegQIBhAC&amp;amp;url=http%3A%2F%2Fdeveloper.download.nvidia.com%2Fassets%2Fcuda%2Ffiles%2FconvolutionSeparable.pdf&amp;amp;usg=AOvVaw1eIqyLuJT6nJDVK7jmyMmX&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; excellent article for details. These improvements are bigger for bigger kernels.&lt;/p&gt;&#xA;&lt;p&gt;Also the training is improved, starting by the simple fact that a NxM filter has a number of parameters proportional to N*M but the related separable one has N+M.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-24T11:45:03.217" LastActivityDate="2020-10-24T11:45:03.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24237" PostTypeId="1" CreationDate="2020-10-24T09:47:36.310" Score="1" ViewCount="31" Body="&lt;p&gt;I'm struggling a little with understanding &lt;a href=&quot;https://github.com/hill-a/stable-baselines/blob/v2.9.0/stable_baselines/a2c/a2c.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;the OpenAI implementation of A2C in the &lt;code&gt;baselines&lt;/code&gt; (version 2.9.0) package&lt;/a&gt;. From my understanding, one &lt;code&gt;step_model&lt;/code&gt; acts in different parallel environments and gathers experiences (calculates the gradients, I think), and sends them to the &lt;code&gt;train_model&lt;/code&gt; that trains with them. After this, the &lt;code&gt;step_model&lt;/code&gt; gets updated from the &lt;code&gt;train_model&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What I am unsure about is if both &lt;code&gt;step_model&lt;/code&gt; and &lt;code&gt;train_model&lt;/code&gt; are actor-critic models or if &lt;code&gt;step_model&lt;/code&gt; is actor and &lt;code&gt;train_model&lt;/code&gt; is a critic (or vice versa). Does the &lt;code&gt;step_model&lt;/code&gt; use the advantage function or is it just the &lt;code&gt;train_model&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="41823" LastEditorUserId="2444" LastEditDate="2020-11-05T11:15:37.900" LastActivityDate="2020-11-05T11:15:37.900" Title="What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;implementation&gt;&lt;advantage-actor-critic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24238" PostTypeId="1" AcceptedAnswerId="24258" CreationDate="2020-10-24T10:59:36.807" Score="0" ViewCount="74" Body="&lt;p&gt;Finite state automata and transducers are computational models that were widely used decades before in natural language processing for morphological parsing and other nlp tasks. I wonder if these computational models are still used in NLP nowadays for significant purposes. If these models are in use, can you give me some examples ?&lt;/p&gt;&#xA;" OwnerUserId="38292" LastEditorUserId="38292" LastEditDate="2020-10-24T11:22:08.077" LastActivityDate="2020-10-26T16:43:05.047" Title="Are FSA and FSTs used in NLP nowadays?" Tags="&lt;natural-language-processing&gt;&lt;computational-linguistics&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24240" PostTypeId="2" ParentId="24231" CreationDate="2020-10-24T13:16:35.960" Score="7" Body="&lt;p&gt;There are many algorithms that are not based on finding a value function. The most common ones are policy gradients. These methods attempt to map states to actions through a neural network. They learn the optimal policy directly, not through a value function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://spinningup.openai.com/en/latest/_images/rl_algorithms_9_15.svg&quot; alt=&quot;RL Algorithms&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;The important part of the image is when &lt;strong&gt;Model-Free RL&lt;/strong&gt; splits into  &lt;strong&gt;Policy Optimization&lt;/strong&gt; (which includes policy gradients) and &lt;strong&gt;Q-Learning&lt;/strong&gt;. Later you can see the two sections coming back together in algorithms that are a mix of both techniques. Even the bottom three methods in policy optimization involve some form of learning a value function. The best and most advanced algorithms use value function learning and policy optimization. The value function is only for training. Then when the agent is tested, it only uses the policy.&lt;/p&gt;&#xA;&lt;p&gt;The most likely reason you have only heard of value function methods is because policy gradients are more complicated. There are many algorithms more advanced than ones that only use value functions and policy gradients can learn to operate in continuous actions spaces (an action can be between -1 and 1, like when moving a robot arm) while value functions can only operate with discrete action spaces (move 1 right or 1 left).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary:&lt;/strong&gt; Yes, there are other methods that learn the optimal policy without a value function. The best algorithms use both types of reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://spinningup.openai.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpinningUp&lt;/a&gt; website has a lot of information about reinforcement learning algorithms and implementations. You can learn more about direct policy optimization there. That is also where I got the image from.&lt;/p&gt;&#xA;&lt;p&gt;This answer is specific to the most common types of Model-Free RL. There are other algorithms related to the RL problem that do not learn value functions, like &lt;a href=&quot;https://thegradient.pub/learning-from-humans-what-is-inverse-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverse reinforcement learning&lt;/a&gt; and &lt;a href=&quot;https://medium.com/@SmartLabAI/a-brief-overview-of-imitation-learning-8a8a75c44a9c&quot; rel=&quot;nofollow noreferrer&quot;&gt;imitation reinforcement learning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41026" LastEditorUserId="41026" LastEditDate="2020-11-11T15:11:33.770" LastActivityDate="2020-11-11T15:11:33.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24242" PostTypeId="1" CreationDate="2020-10-24T19:30:23.730" Score="0" ViewCount="49" Body="&lt;p&gt;Suppose we have a sequence of still images each of which has been contaminated by some particles(ex, dust/sand/smoke) making the images very poor in certain areas.&lt;/p&gt;&#xA;&lt;p&gt;What architecture would be best to teach image regeneration using multiple frames? The simplest technique is to simply find a way to detect what parts of the image are contaminated and uncontaminated and pull uncontaminated sections from each frame.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-10-24T19:30:23.730" Title="Deep Learning based image restoration using multiple frames" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;signal-processing&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24244" PostTypeId="2" ParentId="24204" CreationDate="2020-10-25T10:16:02.807" Score="1" Body="&lt;p&gt;How can you assess the quality of any solution without a measure of quality, which, in the context of genetic algorithms, is &lt;em&gt;known as&lt;/em&gt; fitness function? The term &lt;em&gt;fitness function&lt;/em&gt; is due to the well-known phrase &amp;quot;&lt;strong&gt;Survival of the Fittest&lt;/strong&gt;&amp;quot;, which is often used to describe the Darwinian theory of natural selection (which genetic algorithms are based on). However, note that the fitness function can take any form, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;How well this solution performs in a game?&lt;/em&gt; (in this case, solutions could, for example, be policies to play a game), or&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;How close this solution is to a minimum/maximum of some function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (more precisely, if you want to find the maximum of the function &lt;span class=&quot;math-container&quot;&gt;$f(x) = x^2$&lt;/span&gt;, then individuals are scalars in &lt;span class=&quot;math-container&quot;&gt;$\hat{x} \in \mathbb{R}$&lt;/span&gt;, and the fitness could be determined by &lt;span class=&quot;math-container&quot;&gt;$f'(\hat{x})$&lt;/span&gt; or by how big &lt;span class=&quot;math-container&quot;&gt;$f(\hat{x})$&lt;/span&gt; with respect to other individuals); check how I did it &lt;a href=&quot;https://github.com/nbro/function_max_with_ga/blob/master/main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)?&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The definition of the fitness function depends on what problem you want to solve and which solutions you want to find.&lt;/p&gt;&#xA;&lt;p&gt;So, you need some kind of fitness function in genetic algorithms to perform selection in a reasonable way, so that to maintain the &amp;quot;best solutions&amp;quot; in the population. More precisely, while selecting the new individuals for the new generation (i.e. iteration), if you don't use a fitness (which you can also call &lt;em&gt;performance&lt;/em&gt;, if you like) function to understand which individuals deserve to live or die, how do you know that the new solutions are better than the previous ones? You cannot know this without a fitness/performance function, so you cannot also logically decide which individuals to kill before the next generation. Mutations alone just change the solutions, i.e. they are used to explore the space of solutions.&lt;/p&gt;&#xA;&lt;p&gt;Genetic algorithms are always composed of&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a population of solutions/individuals/chromosomes (i.e. usually at least &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; solutions)&lt;/li&gt;&#xA;&lt;li&gt;operations to randomly (or stochastically) change existing solutions to create new ones (typically mutations and crossovers)&lt;/li&gt;&#xA;&lt;li&gt;a selection process that selects the new solutions/individuals for the next generation (or to be combined and mutated)&lt;/li&gt;&#xA;&lt;li&gt;a fitness function to help you decide which solutions need to be selected (or even combined and mutated)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For more info about genetic algorithms or, more generally, evolutionary algorithms, take a look at chapter 8 and 9 of the book &lt;a href=&quot;https://papers.harvie.cz/unsorted/computational-intelligence-an-introduction.pdf#page=160&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Intelligence: An Introduction&lt;/a&gt; by Andries P. Engelbrecht.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-25T11:08:36.417" LastActivityDate="2020-10-25T11:08:36.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24245" PostTypeId="1" CreationDate="2020-10-25T11:49:00.467" Score="0" ViewCount="32" Body="&lt;p&gt;Consider the scenario in which I am measuring certain &lt;span class=&quot;math-container&quot;&gt;$f(a,x)$&lt;/span&gt;, which i want to be the target value for some related input &lt;span class=&quot;math-container&quot;&gt;$g(a,x)$&lt;/span&gt;. In other words, I am trying to map&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$g(a,x)\Rightarrow f(a,x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(a,x) = a x^{-3/4}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;With &lt;span class=&quot;math-container&quot;&gt;$x \in [1,20]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a=[1,2,3]$&lt;/span&gt;. I now perform &amp;quot;measurements&amp;quot; for &lt;span class=&quot;math-container&quot;&gt;$x=[1,2,...,20]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a=[1,2,3]$&lt;/span&gt;.The histogram of my target values looks awful:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bzZlf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bzZlf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And that means that the neural network will favor smaller output values. I believe there are two solutions to make the distribution more &amp;quot;even&amp;quot; so that the neural network treats each target value equally:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1. Transform the output variable&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is troublesome, because for example the mapping &lt;span class=&quot;math-container&quot;&gt;$f'(a,x)=f(a,x)^{-3/4}$&lt;/span&gt; still leads to a skewed distribution because of the varying constant &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wsCtR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wsCtR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2. Perform measurements for certain &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; such that the output is more evenly distributed. Constraint: I must always measure for each &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I still want to sample the whole range, so I could do&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x=\mathrm{linspace}(1,20^{-3/4},n=19)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_i'= x_i^{-4/3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But this only gives &amp;quot;plateaus&amp;quot; of uniformity, one for each unique &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. You can see this by increasing the number of points to sample:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ez2E9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ez2E9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So my question: what is the best way to generate/transform the data for my &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$a=[1,2,3]$&lt;/span&gt; so that the target values are distributed in such a way that it does not favour certain values?&lt;/p&gt;&#xA;&lt;p&gt;PS: &lt;span class=&quot;math-container&quot;&gt;$f(a,x)$&lt;/span&gt; is something that I have to measure in the real world, I cannot do more than 20 measurements.&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2020-11-05T11:39:03.843" LastActivityDate="2020-11-05T11:39:03.843" Title="Transforming neural network target values before training" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;&lt;imbalanced-datasets&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24248" PostTypeId="1" CreationDate="2020-10-25T22:23:05.950" Score="0" ViewCount="53" Body="&lt;p&gt;I am new to RL and mujoco. I just set up mujoco and am testing the FetchPickAndPlace environment. I called the following methods:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;env.action_space.sample()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;returns &lt;code&gt;array([ 0.14159757, -0.6169547 ,  0.56277525,  0.21222822], dtype=float32)&lt;/code&gt;&#xA;and &lt;code&gt;env.reset()&lt;/code&gt; returns&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    {'observation': array([ 1.34782473e+00,  7.48951529e-01,  4.13637614e-01,  1.42999837e+00,&#xA;         6.54897670e-01,  4.24702091e-01,  8.21736315e-02, -9.40538593e-02,&#xA;         1.10644764e-02, -1.50186348e-06,  1.24443593e-03, -3.85214084e-07,&#xA;         5.92637053e-07,  1.13569317e-13,  8.95029727e-04, -4.10490913e-05,&#xA;         4.90967749e-05,  1.88857148e-07, -2.90549459e-07,  4.01961762e-18,&#xA;        -8.95036946e-04,  4.10443989e-05, -2.26900915e-06,  3.77638244e-07,&#xA;         4.38184449e-05]),&#xA; 'achieved_goal': array([1.42999837, 0.65489767, 0.42470209]),&#xA; 'desired_goal': array([1.48356396, 0.62833857, 0.42469975])}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could someone explain the meanings of the numbers in the arrays?&lt;/p&gt;&#xA;&lt;p&gt;Control: is it possible to manually write a program to manipulate the robot to achieve the goal (grab the object first, then move to the goal).&lt;/p&gt;&#xA;" OwnerUserId="40506" LastActivityDate="2020-10-25T22:23:05.950" Title="mujoco environment in openai gym: observation and action explanation and control" Tags="&lt;reinforcement-learning&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24258" PostTypeId="2" ParentId="24238" CreationDate="2020-10-26T16:43:05.047" Score="0" Body="&lt;p&gt;Both are used, for example,  in the &lt;a href=&quot;https://gate.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATE&lt;/a&gt; framework, which is still widely used. I suspect that this also applies to many other applications.&lt;/p&gt;&#xA;&lt;p&gt;I would think that many recent academic publications are now on other approaches, as FSAs and FSTs are fairly established and mature technologies, but I've been out of academia for a while now.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastActivityDate="2020-10-26T16:43:05.047" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24259" PostTypeId="1" CreationDate="2020-10-26T17:26:59.447" Score="2" ViewCount="38" Body="&lt;p&gt;From the part titled &lt;em&gt;Introducing Latent Variables&lt;/em&gt; under subsection 2.2 in this &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-33275-3_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Introducing Latent Variables.&lt;/strong&gt; Suppose we want to model an &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-dimensional&#xA;unknown probability distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (e.g., each component of a sample corresponds to one of m pixels of an image). Typically, not all variables &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X} = (X_v)_{v \in V}$&lt;/span&gt; in an MRF need to correspond to some observed component, and the number of nodes is larger than &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. We split &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; into visible (or observed) variables &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V} = (V_1,...,V_m)$&lt;/span&gt; corresponding to the components of the observations and latent (or hidden) variables &lt;span class=&quot;math-container&quot;&gt;$\mathbf{H} = (H_1,...,H_n)$&lt;/span&gt; given by the remaining &lt;span class=&quot;math-container&quot;&gt;$n = |\mathbf{V}| − m$&lt;/span&gt; variables. Using latent variables allows to describe complex distributions over the visible variables by means of simple (conditional) distributions. In this case the Gibbs distribution of an MRF describes the joint probability distribution of &lt;span class=&quot;math-container&quot;&gt;$(\mathbf{V},\mathbf{H})$&lt;/span&gt; and one is usually interested in the marginal distribution of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V}$&lt;/span&gt; which is given by:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{v}) = \sum_{\mathbf{h}} p(\mathbf{v},\mathbf{h}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h})}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Z = \sum_{\mathbf{v},\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h})}$&lt;/span&gt;. While the visible variables correspond to the components of an observation, the latent variables introduce dependencies between the visible variables (e.g., between pixels of an input image).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have a question about this part:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While the visible variables correspond to the components of an observation, the latent variables introduce dependencies between the visible variables (e.g., between pixels of an input image).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Given a set of nodes &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; in a Markov Random Field &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, the joint distribution of all the nodes is given by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{X}) = \frac{1}{Z} \prod_{c \in C} \phi(c)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is the partition function and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the set of cliques in &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. To ensure that the joint distribution is positive, the following factors can be used:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\phi(c) = e^{-E(c)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Such that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{X}) = \frac{1}{Z} e^{-\sum_{c \in C} E(c)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the energy function.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure why there is a need to introduce hidden variables and express &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v})$&lt;/span&gt; as a marginalization of &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v},\mathbf{h})$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$\mathbf{h}$&lt;/span&gt;. Why can't &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v})$&lt;/span&gt; be expressed as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{v}) = \frac{1}{Z} e^{-\sum_{v \in \mathbf{v}} E(v)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;directly? I think it may be because the factors only encode dependencies between variables in cliques, and so may not be able to encode dependencies between variables that are in two separate cliques. The purpose of the hidden variables are then to encode these &amp;quot;long-range&amp;quot; dependencies between visible variables not in cliques. However, I am not sure about this reasoning.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;By the way, I am aware of &lt;a href=&quot;https://ai.stackexchange.com/questions/4084/what-is-the-role-of-the-hidden-vectors-in-restricted-boltzmann-machines?rq=1&quot;&gt;this&lt;/a&gt; question, but I think the answer is not specific enough.&lt;/p&gt;&#xA;" OwnerUserId="41856" LastEditorUserId="41856" LastEditDate="2020-10-27T03:23:31.137" LastActivityDate="2020-10-27T03:23:31.137" Title="Purpose of the hidden variables in a Restricted Boltzmann Machine" Tags="&lt;restricted-boltzmann-machine&gt;&lt;graphical-model&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24260" PostTypeId="1" CreationDate="2020-10-27T00:28:57.980" Score="2" ViewCount="34" Body="&lt;p&gt;I'm new to the AI Stackexchange and wasn't certain if this should go here or to Maths instead but thought the context with ML may be useful to understand my problem. I hope posting this question here could help another student learning about Support Vector Machines some day.&lt;/p&gt;&#xA;&lt;p&gt;I'm currently learning about Support Vector Machines at university and came across a weird step I could not understand. We were talking about basic SVMs and formulated the optimisation problem &lt;span class=&quot;math-container&quot;&gt;$\max_{w,b} \{ \frac{1}{||w||} \min_n(y^{(n)}f(x^{(n)}))\}$&lt;/span&gt; which we then simplified down to &lt;span class=&quot;math-container&quot;&gt;$\max_{w,b} \{ \frac{1}{||w||}\}$&lt;/span&gt; by introducing &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; as a scaling factor for &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; according to the margin of the SVM. Now our lecturer converted it without explanation into a quadratic optimisation problem as &lt;span class=&quot;math-container&quot;&gt;$\min_{w,b}\{\frac{1}{2} ||w||^2\}$&lt;/span&gt; which I could not explain myself. I hope someone with context can help me how this is possible and what math or trick is behind this approach?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Notation information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; - weight matrix&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; - bias (sometimes denoted &lt;span class=&quot;math-container&quot;&gt;$w_0$&lt;/span&gt; I believe?)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x^{(n)}$&lt;/span&gt; - Independent variable (vector)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y^{(n)}$&lt;/span&gt; - Dependent variable (scalar classifying the input in a binary classifcation as &lt;span class=&quot;math-container&quot;&gt;$y=1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$y=-1$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="41859" LastEditorUserId="9947" LastEditDate="2020-10-28T10:49:37.063" LastActivityDate="2020-10-28T14:53:17.347" Title="Support Vector Machine Convert optimisation problem from argmax to argmin" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;support-vector-machine&gt;&lt;binary-classification&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24261" PostTypeId="1" AcceptedAnswerId="24272" CreationDate="2020-10-27T00:38:44.707" Score="8" ViewCount="166" Body="&lt;p&gt;I'm reading chapter one of the book called &lt;a href=&quot;https://dl.uswr.ac.ir/bitstream/Hannan/141305/2/9783319944623.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks and Deep Learning&lt;/a&gt; from Aggarwal.&lt;/p&gt;&#xA;&lt;p&gt;In section 1.2.1.1 of the book, I'm learning about the perceptron. One thing that book says is, if we use the sign function for the following loss function: &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=0}^{N}[y_i - \text{sign}(W * X_i)]^2$&lt;/span&gt;, that loss function will NOT be differentiable. Therefore, the book suggests us to use, instead of the sign function in the loss function, the perceptron criterion which will be defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ L_i = \max(-y_i(W * X_i), 0) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The question is: Why is the perceptron criterion function differentiable? Won't we face a discontinuity at zero? Is there anything that I'm missing here?&lt;/p&gt;&#xA;" OwnerUserId="41860" LastEditorUserId="2444" LastEditDate="2021-02-15T22:49:02.870" LastActivityDate="2021-02-15T22:49:02.870" Title="Why is the perceptron criterion function differentiable?" Tags="&lt;optimization&gt;&lt;gradient-descent&gt;&lt;perceptron&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24262" PostTypeId="1" CreationDate="2020-10-27T01:16:12.343" Score="1" ViewCount="57" Body="&lt;p&gt;Across the literature of artificial intelligence, especially machine learning, it is normal to treat the tuples of datasets as vectors.&lt;/p&gt;&#xA;&lt;p&gt;Although there is a convention to treat them as data points. Treating them as vectors is also considerable.&lt;/p&gt;&#xA;&lt;p&gt;It is easy to understand the tuples of datasets as points over space. But what is the purpose of treating them as vectors?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="18758" LastEditDate="2021-03-19T02:34:52.263" LastActivityDate="2021-03-19T02:34:52.263" Title="What is the reason for taking tuples as vectors rather than points?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;datasets&gt;&lt;vectors&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24265" PostTypeId="2" ParentId="24262" CreationDate="2020-10-27T07:00:31.463" Score="4" Body="&lt;p&gt;They are equivalent. When we consider a particular instance as a vector, we are not literally imagining it as an arrow with it's head at the point coordinates and tail at the origin. It's just when you are working with a tuple of numbers in a mathematical context, it is conventional to call it a vector. This language follows into machine learning which is usually based on associated linear algebra.&lt;/p&gt;&#xA;" OwnerUserId="34473" LastActivityDate="2020-10-27T07:00:31.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24266" PostTypeId="1" CreationDate="2020-10-27T11:59:12.577" Score="0" ViewCount="31" Body="&lt;p&gt;I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to train. Of the same 10 images, I am using 3 images for validation. For the evaluation, I am using the evaluation method used in COCO dataset which is also provided as .py scripts in the TorchVision's github repository.&lt;/p&gt;&#xA;&lt;p&gt;To have enough samples for training, I am oversampling the same 10 images by a factor of 100 i.e. I end up with 1000 images that I can use to train my model. Similarly, I end up having 300 images that I can use for validation.&lt;/p&gt;&#xA;&lt;p&gt;Now, the problem is that I am getting 0% mAP after train and 0% recall. I have two questions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q1. Why would it return 0% mAP?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If it has something to do with the fact that I am oversampling to a large extent, then my next question is&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q2.  Shouldn't the oversampling just cause the model to Overfit and instead provide a higher training as well as validation accuracy for my case (since I have picked the validation data from the training data itself?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Solved&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;It was just a coding error. Fixed it and now I am getting an mAP &amp;gt;90%.&lt;/p&gt;&#xA;" OwnerUserId="41869" LastEditorUserId="41869" LastEditDate="2020-10-30T18:48:45.667" LastActivityDate="2020-10-30T18:48:45.667" Title="Training Object Detection model on just 10 images" Tags="&lt;training&gt;&lt;object-detection&gt;&lt;pytorch&gt;&lt;overfitting&gt;&lt;mask-rcnn&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24267" PostTypeId="1" CreationDate="2020-10-27T12:38:07.053" Score="0" ViewCount="37" Body="&lt;p&gt;I wish to get MSE &amp;lt; 0.5 on test data (&lt;a href=&quot;https://easyupload.io/zr7xf3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://easyupload.io/zr7xf3&lt;/a&gt;) which is 20% of given data chosen randomly. But I am reaching 0.73 using both plain Ridge Regression as well as a neural network with about 6 layers with some elementary regularization, dropout and choice of other parameters. Overfitting also occurs.&lt;/p&gt;&#xA;&lt;p&gt;Suggest. I believe a Bayesian optimization or a genetic algorithm for parameters is required.&lt;/p&gt;&#xA;&lt;p&gt;I did no feature selection (as top 4 features showed no improvement) and non-linear methods exploration.&lt;/p&gt;&#xA;&lt;p&gt;My solutions -&#xA;Ridge - Alpha = 0.002 (Grid searched)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Neural Network efforts =&#xA;reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2,&#xA;                              patience=10, min_lr=0.001)&#xA;&#xA;&#xA;es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=10, restore_best_weights=True)&#xA;&#xA;&#xA;model_b = Sequential()&#xA;&#xA;&#xA;model_b.add(Dense(2048, kernel_initializer='he_uniform',input_dim = X.shape[1], activation='relu', kernel_regularizer=regularizers.l2(l2=1e-6)))&#xA;model_b.add(BatchNormalization(beta_regularizer = regularizers.l2(0.00001)))&#xA;&#xA;&#xA;# The Hidden Layers :&#xA;model_b.add(Dense(1024, kernel_initializer='lecun_normal',activation='selu',kernel_regularizer=regularizers.l2(l2=1e-6)))&#xA;&#xA;&#xA;model_b.add(BatchNormalization(beta_regularizer = regularizers.l2(0.00001)))&#xA;&#xA;&#xA;model_b.add(Dense(1024, kernel_initializer='lecun_normal',activation='selu',kernel_regularizer=regularizers.l2(l2=1e-6)))&#xA;&#xA;&#xA;model_b.add(BatchNormalization(beta_regularizer = regularizers.l2(0.00001)))&#xA;&#xA;&#xA;model_b.add(Dropout(0.5))&#xA;&#xA;&#xA;model_b.add(Dense(512, kernel_initializer='normal',activation='relu'))&#xA;&#xA;&#xA;model_b.add(Dense(512, kernel_initializer='normal',activation='relu'))&#xA;&#xA;&#xA;model_b.add(Dense(256, kernel_initializer='normal',activation='relu'))&#xA;&#xA;# The Output Layer :&#xA;&#xA;&#xA;model_b.add(Dense(1, kernel_initializer='normal',activation='linear'))&#xA;optimizer = SGD(lr = 0.0001)&#xA;&#xA;&#xA;model_b.compile(loss='mean_squared_error', optimizer= optimizer)&#xA;&#xA;&#xA;model_b.fit(X_train, y_train, batch_size=70,&#xA;              epochs=256,&#xA;              validation_data=(X_test, y_test),callbacks = [es])&#xA;&#xA;&#xA;predb = model_b.predict(X_test)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If anyone has free time, may answer.&lt;/p&gt;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="41870" LastActivityDate="2020-10-27T12:38:07.053" Title="Unable to meet desired mean squared error" Tags="&lt;neural-networks&gt;&lt;keras&gt;&lt;genetic-algorithms&gt;&lt;hyperparameter-optimization&gt;&lt;principal-component-analysis&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24268" PostTypeId="2" ParentId="17218" CreationDate="2020-10-27T14:15:35.747" Score="0" Body="&lt;p&gt;As described in &lt;a href=&quot;https://www.pisciottablog.com/2020/10/24/unbalanced-datasets-in-machine-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;, this problem is known as &amp;quot;unbalanced dataset&amp;quot; problem, which can have different solution approaches. If you use supervised learning, augmentation approaches&#xA;could  help. Otherwise, unsupervised approaches need some proper distance measure for outliers detection.&lt;/p&gt;&#xA;" OwnerUserId="32260" LastActivityDate="2020-10-27T14:15:35.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24269" PostTypeId="1" CreationDate="2020-10-27T17:43:31.887" Score="3" ViewCount="69" Body="&lt;p&gt;I am building an artificial town that has randomly generated people with different personalities. People in this artificial town have attributes that change their happiness, ego, rep, etc.&#xA;I already have most of the code done, but I ran into a problem. I don't know how I could generate conversations between two people.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Does anyone know a specific AI/NLG technique that I could use to accomplish that?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;My knowledge in NLG used to be pretty vast, but I haven't worked with NLG or NLP in over a year, so I'm a little rusty. The only thing I need to make this a success is for the people to generate their own dialogue based on where they are or who they are talking to.&lt;/p&gt;&#xA;&lt;p&gt;Some examples of dialogue that I would want to be generated are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Peter&lt;/strong&gt;: Hello, John. How is your wife Jane?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;John&lt;/strong&gt;: Hello, Peter. My wife Jane is doing great!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Peter&lt;/strong&gt;: What's your child's name?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;John&lt;/strong&gt;: My child's name is Johnny.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The agents will have access to the datasheets with all of the names, emotions, and reputations on it.  I'm hoping to build a datasheet for each person, so that they only know as much data as they learn from the dialogue they have with the other people who know that data.  The dialogue should not repeat as each dialogue should act as a sort of random get request.&lt;/p&gt;&#xA;" OwnerUserId="41877" LastEditorUserId="2444" LastEditDate="2020-11-01T12:10:00.690" LastActivityDate="2020-11-01T12:10:00.690" Title="How can I generate artificial conversations between two AIs?" Tags="&lt;reference-request&gt;&lt;chat-bots&gt;&lt;natural-language-generation&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24270" PostTypeId="2" ParentId="24261" CreationDate="2020-10-27T17:53:45.423" Score="3" Body="&lt;p&gt;Since we're dealing with real-values variables, it is almost certainly the case that the argument of the function will not be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you care strongly about that point, you can just use sub-gradients instead (and we do have sub-gradients for this function, so there is no problem).&lt;/p&gt;&#xA;" OwnerUserId="40573" LastActivityDate="2020-10-27T17:53:45.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24271" PostTypeId="1" AcceptedAnswerId="24795" CreationDate="2020-10-27T18:21:15.937" Score="1" ViewCount="147" Body="&lt;p&gt;How is information theory applied to machine learning, and in particular to deep learning, in practice? I'm more interested in concepts that yielded concrete innovations in ML, rather than theoretical constructions.&lt;/p&gt;&#xA;&lt;p&gt;Note that, I'm aware that basic concepts such as entropy is used for training decision trees, and so on. I'm looking for applications which use slightly more advanced concepts from information theory, whatever they are.&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="32621" LastEditDate="2020-10-28T02:08:58.543" LastActivityDate="2020-11-23T01:57:17.827" Title="Applications of Information Theory in Machine Learning" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;information-theory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24272" PostTypeId="2" ParentId="24261" CreationDate="2020-10-27T20:29:48.823" Score="7" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\max(-y_i(w x_i), 0)$&lt;/span&gt; is not partial derivable respect &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$w x_i=0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Loss functions are problematic when not derivable in some point, but even more when they are flat (constant) in some interval of the weights.&lt;/p&gt;&#xA;&lt;p&gt;Assume &lt;span class=&quot;math-container&quot;&gt;$y_i = 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w x_i &amp;lt; 0$&lt;/span&gt; (that is, an error of type &amp;quot;false negative&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;In this case, function &lt;span class=&quot;math-container&quot;&gt;$[y_i - \text{sign}(w x_i)]^2 = 4$&lt;/span&gt;. Derivative on all interval &lt;span class=&quot;math-container&quot;&gt;$w x_i &amp;lt; 0$&lt;/span&gt; is zero, thus, the learning algorithm has no any way to decide if it is better increase or decrease &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In same case, &lt;span class=&quot;math-container&quot;&gt;$\max(-y_i(w x_i), 0) = - w x_i$&lt;/span&gt;, partial derivative is &lt;span class=&quot;math-container&quot;&gt;$-x_i$&lt;/span&gt;. The learning algorithm knows that it must increase &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; value if &lt;span class=&quot;math-container&quot;&gt;$x_i&amp;gt;0$&lt;/span&gt;, decrease otherwise. This is the real reason this loss function is considered more practical than previous one.&lt;/p&gt;&#xA;&lt;p&gt;How to solve the problem at &lt;span class=&quot;math-container&quot;&gt;$w x_i = 0$&lt;/span&gt; ? simply, if you increase &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and the result is an exact &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, assign to it a very small value, &lt;span class=&quot;math-container&quot;&gt;$w=\epsilon$&lt;/span&gt;. Similar logic for remainder cases.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-10-27T21:04:52.813" LastActivityDate="2020-10-27T21:04:52.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24273" PostTypeId="1" CreationDate="2020-10-28T02:01:38.687" Score="1" ViewCount="50" Body="&lt;p&gt;The paper, &lt;a href=&quot;https://arxiv.org/pdf/1507.06527.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Recurrent Q-Learning for Partially Observable MDPs&lt;/a&gt;, talks about stacking multiple observations in the input of a convolutional neural network.&lt;/p&gt;&#xA;&lt;p&gt;How does this exactly work? Do the convolutional filters loop over each observation (image)?&lt;/p&gt;&#xA;&lt;p&gt;(I know this isn't the right group to request this, but I'll highly appreciate if someone could also suggest a framework that helps with this.)&lt;/p&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2020-10-30T19:27:45.107" LastActivityDate="2020-10-30T19:27:45.107" Title="How does one stack multiple observations in the input layer of a convolutional neural network?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24274" PostTypeId="2" ParentId="24260" CreationDate="2020-10-28T03:28:29.623" Score="1" Body="&lt;p&gt;So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion.&lt;/p&gt;&#xA;&lt;p&gt;Generally speaking maximising &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{||w||}$&lt;/span&gt; is identical to minimising &lt;span class=&quot;math-container&quot;&gt;$||w||$&lt;/span&gt;. As &lt;span class=&quot;math-container&quot;&gt;$||w||$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{||w||}$&lt;/span&gt; decreases, the overall value increases, i.e. we maximise it. The reason for choosing &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}||w||^2$&lt;/span&gt; turns out to be a less mathematic and more practical one as our optimisation algorithms used perform better on quadratic functions and the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt; seems to be a rather arbitrary scaling choice.&lt;/p&gt;&#xA;&lt;p&gt;If anyone has something to add, I'd love to hear any details though!&lt;/p&gt;&#xA;" OwnerUserId="41859" LastEditorUserId="41859" LastEditDate="2020-10-28T14:53:17.347" LastActivityDate="2020-10-28T14:53:17.347" CommentCount="0" CommunityOwnedDate="2020-10-28T03:28:29.623" ContentLicense="CC BY-SA 4.0" />
  <row Id="24277" PostTypeId="5" CreationDate="2020-10-28T09:14:46.817" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural-language_generation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Natural-language_generation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-28T09:14:46.817" LastActivityDate="2020-10-28T09:14:46.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24278" PostTypeId="4" CreationDate="2020-10-28T09:14:46.817" Score="0" Body="For questions about the generation of human-readable text from structured data. This is related to natural-language-processing (NLP), and statistical NLP models can be used in NLG too. However, NLG also applies to templating, report writing and scripted conversations.&#xD;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-10-28T09:14:46.817" LastActivityDate="2020-10-28T09:14:46.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24279" PostTypeId="1" CreationDate="2020-10-28T11:27:01.307" Score="4" ViewCount="73" Body="&lt;p&gt;Multi-label assignment is the task in machine learning to assign to each input value a set of categories from a fixed vocabulary where the categories need not be statistically independent, so precluding building a set of independent classifiers each classifying the inputs as belong to each of the categories or not.&lt;/p&gt;&#xA;&lt;p&gt;Machine learning also needs a measure by which the model may be evaluated. So this is the question how do we evaluate a multi-label classifier?&lt;/p&gt;&#xA;&lt;p&gt;We can’t use the normal recall,  accuracy and  F measures since they require a binary is it correct or not measure of each categorisation. Without such a measure we have no obvious means to evaluate models nor to measure concept drift.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastEditorUserId="32410" LastEditDate="2021-04-27T07:02:54.143" LastActivityDate="2021-05-27T07:03:15.270" Title="How do you measure multi-label classification accuracy?" Tags="&lt;reference-request&gt;&lt;performance&gt;&lt;metric&gt;&lt;multi-label-classification&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24280" PostTypeId="1" CreationDate="2020-10-28T14:40:12.727" Score="0" ViewCount="41" Body="&lt;p&gt;When I started to look into the deep High-Resolution Net paper, I came to know about the HigherHRNet and the Bottom Up HRNet as well. But I can't find any metrics or proof on which is better for a single person pose estimation task. I would like to know which of these three is the most accurate. Help me out here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1902.09212&quot; rel=&quot;nofollow noreferrer&quot;&gt;HRNet Paper Link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.semanticscholar.org/paper/HigherHRNet%3A-Scale-Aware-Representation-Learning-Cheng-Xiao/12037f79437e75a9c76be044e8da44a23b5a1d93&quot; rel=&quot;nofollow noreferrer&quot;&gt;HigherHRNet Paper Link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38087" LastEditorUserId="38087" LastEditDate="2020-10-31T15:37:35.643" LastActivityDate="2020-10-31T15:37:35.643" Title="Which is more accurate between HRnet-w48 and HigherHRnet-w48 for single person pose estimation?" Tags="&lt;object-detection&gt;&lt;feature-extraction&gt;&lt;action-recognition&gt;&lt;pose-estimation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24282" PostTypeId="1" CreationDate="2020-10-28T18:26:25.137" Score="1" ViewCount="160" Body="&lt;p&gt;I've been trying to understand where the formulas for &lt;strong&gt;Xavier&lt;/strong&gt; and &lt;strong&gt;Kaiming He&lt;/strong&gt; initialization come from. My understanding is that these initialization schemes come from a desire to keep the gradients stable during back-propagation (avoiding vanishing/exploding gradients).&lt;/p&gt;&#xA;&lt;p&gt;I think I can understand the justification for Xavier initialization, and I'll sketch it below. For He initialization, what the original paper actually shows is that that initialization scheme keeps the &lt;em&gt;pre-activation&lt;/em&gt; values (the weighted sums) stable throughout the network. &lt;strong&gt;Most sources I've found explaining Kaiming He initialization seem to just take it as &amp;quot;obvious&amp;quot; that stable pre-activation values will somehow lead to stable gradients&lt;/strong&gt;, and don't even mention the apparent mismatch between what the math shows and what we're actually trying to accomplish.&lt;/p&gt;&#xA;&lt;p&gt;The justification for Xavier initialization (introduced &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) is as follows, as I understand it:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;As an approximation, pretend the activation functions don't exist and we have a linear network. The actual paper says we're assuming the network starts out in the &amp;quot;linear regime&amp;quot;, which for the sigmoid activations they're interested in would mean we're assuming the pre-activations at every layer will be close to zero. I don't see how this could be justified, so I prefer to just say we're disregarding the activation functions entirely, but in any case that's not what I'm confused about here.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Zoom in on one edge in the network. It looks like &lt;span class=&quot;math-container&quot;&gt;$x\to_{w} y$&lt;/span&gt;, connecting the input or activation value &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to the activation value &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, with the weight &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;. When we do gradient descent we consider &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial C}{\partial w}$&lt;/span&gt;, and we have:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial w}=x\frac{\partial C}{\partial y}$$&lt;/span&gt;&#xA;So if we want to avoid unstable &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial C}{\partial w}$&lt;/span&gt;-s, a sufficient (not necessary, but that's fine) condition is to keep both those factors stable - the activations and the gradients with respect to activations. So we try to do that.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To measure the &amp;quot;size&amp;quot; of an activation, let's look at its mean and variance (where the randomness comes from the random weights). If we use zero-mean random weights all i.i.d. on each layer, then we can show that all of the activation values in our network are zero-mean, too. So controlling the size comes down to controlling the variance (big variance means it tends to have large absolute value and vice versa). Since the gradients with respect to activations are calculated by basically running the neural network backwards, we can show that they're all zero-mean too, so controlling their size comes down to controlling their variance as well.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We can show that all the activations on a given layer are identically distributed, and ditto for the gradients with respect to activations on a given layer. If &lt;span class=&quot;math-container&quot;&gt;$v_n$&lt;/span&gt; is the variance of the activations on layer &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and if &lt;span class=&quot;math-container&quot;&gt;$v'_n$&lt;/span&gt; is the variance of the gradients, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_{n+1}=v_n k_n \sigma^2$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v'_n=v_{n+1} k_{n+1} \sigma^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$k_i$&lt;/span&gt; is the number of neurons on the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th layer, and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; is the variance of the weights between the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-th and &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt;-th layers. So to keep either of the growth factors from being too crazy, we would want &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; to be equal to both &lt;span class=&quot;math-container&quot;&gt;$1/k_n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1/k_{n+1}$&lt;/span&gt;. We can compromise by setting it equal to the harmonic mean or the geometric mean or something like that.&lt;/p&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;This stops the activations from exploding out of control, &lt;em&gt;and&lt;/em&gt; stops the gradients with respect to activations from exploding out of control, which by step (2) stops the gradients with respect to the weights (which at the end of the day are the &lt;em&gt;only things we really care about&lt;/em&gt;) from growing out of control.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, when I look at the paper on &lt;a href=&quot;https://arxiv.org/pdf/1502.01852.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;He initialiation&lt;/a&gt;, it seems like almost every step in this logic breaks down. First of all, the math, if I understand correctly, shows that He initialization can control the &lt;em&gt;pre-activations&lt;/em&gt;, not the activations. Therefore, the logic from step (2) above that this tells us something about the gradients with respect to the weights fails. Second of all, the activation values in a ReLU network like the authors are considering are not zero-mean, as they point out themselves, but this means that even the reasoning as to why we should care about the variances, from step (3), fails. The variance is only relevant for Xavier initialization because in that setting the mean is always zero, so the variance is a reasonable proxy for &amp;quot;bigness&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;So while I can see how the authors show that He initialization controls the variances of the pre-activations in a ReLU network, for me the entire reason as to why we should care about doing this has fallen apart.&lt;/p&gt;&#xA;" OwnerUserId="1931" LastActivityDate="2020-10-28T18:26:25.137" Title="What is the justification for Kaiming He initialization?" Tags="&lt;deep-learning&gt;&lt;weights-initialization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24285" PostTypeId="2" ParentId="24279" CreationDate="2020-10-29T03:50:01.190" Score="0" Body="&lt;p&gt;Your intuition is correct. We do use other metrics for multi-label classification. The meaning of evaluation itself changes. Apart from grading the classifier on whether it classifies correctly or not, we also have to penalize it, if it chooses the wrong class appropriately. You could use the following metrics:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;micro/macro averaging Recall/Precision, etc.&lt;/li&gt;&#xA;&lt;li&gt;Hamming Loss&lt;/li&gt;&#xA;&lt;li&gt;Subset Accuracy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-10-29T03:50:01.190" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24286" PostTypeId="1" CreationDate="2020-10-29T07:50:52.713" Score="0" ViewCount="42" Body="&lt;p&gt;I'm working on SLAP (storage location assignment problem) using genetic algorithm implemented manually in the C++ programming language. The problem is fairly simple, we do have &lt;code&gt;N&lt;/code&gt; products, which we want to allocate to &lt;code&gt;M&lt;/code&gt; warehouse location slots (&lt;code&gt;N&lt;/code&gt; might and might not be equal to &lt;code&gt;M&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Let's begin with the encoding of the chromosomes. The chromosome length is equal to number of products (i.e. each product is one gene). Each product has one integer value (allele value), representing the location its allocated to.&lt;/p&gt;&#xA;&lt;p&gt;Let me show you on simple example.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Products         Average picking rate        Location slots       Location number&#xA;Prod1            0.4                         Location 1, slot 1   (1)   // 3rd best&#xA;Prod2            0.3                         Location 1, slot 2   (2)   // 4th best&#xA;Prod3            0.2                         Location 2, slot 1   (3)   // The best&#xA;Prod4            0.1                         Location 2, slot 2   (4)   // 2nd best&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We aim for optimal allocation of products (Prod1-4) to location slots (1-4). The better the allocation is, the faster we can process all the products in customer orders. Now let's say the Location 2 is closer to the warehouse entrance/exit, so its more attractive, and the lower the location slot number is, the faster we can pick product out of the location slot. So the optimal allocation should be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Product     Location number&#xA;Prod1       3&#xA;Prod2       4&#xA;Prod3       1&#xA;Prod4       2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And expressed as the chromosome:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+---+---+---+---+&#xA;| 3 | 4 | 1 | 2 |&#xA;+---+---+---+---+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This allocation will lead to the best warehouse performance. Now let me show you my crossover operator (based on TSP crossover &lt;a href=&quot;https://www.permutationcity.co.uk/projects/mutants/tsp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.permutationcity.co.uk/projects/mutants/tsp.html&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;void crossoverOrdered(std::vector&amp;lt;int32_t&amp;gt;&amp;amp; lhsInd, std::vector&amp;lt;int32_t&amp;gt;&amp;amp; rhsInd)&#xA;{&#xA;    int32_t a, b;&#xA;    int32_t pos =  0;&#xA;    int32_t placeholder = -1;&#xA;    int32_t placeholderCount = 0;&#xA;&#xA;    std::vector&amp;lt;int32_t&amp;gt; o1, o1_missing, o1_replacements;&#xA;    std::vector&amp;lt;int32_t&amp;gt; o2, o2_missing, o2_replacements;&#xA;&#xA;    while(true)&#xA;    {&#xA;        do&#xA;        {&#xA;            a = randomFromInterval(pos, constants::numberDimensions);&#xA;            b = randomFromInterval(pos, constants::numberDimensions);&#xA;        }&#xA;        while(a == b);&#xA;&#xA;        if(a &amp;gt; b) std::swap(a, b);&#xA;&#xA;        // Insert from first parent&#xA;        for(int32_t i = pos; i &amp;lt; a; ++i)&#xA;        {&#xA;            o1.push_back(lhsInd.at(i));&#xA;            o2.push_back(rhsInd.at(i));&#xA;        }&#xA;&#xA;        // Insert placeholders&#xA;        for(int32_t i = a; i &amp;lt; b; ++i)&#xA;        {&#xA;            ++placeholderCount;&#xA;            o1.push_back(placeholder);&#xA;            o2.push_back(placeholder);&#xA;        }&#xA;&#xA;        if(b &amp;gt;= constants::numberDimensions - 1)&#xA;        {&#xA;            for(int32_t i = b; i &amp;lt; constants::numberDimensions; ++i)&#xA;            {&#xA;                o1.push_back(lhsInd.at(i));&#xA;                o2.push_back(rhsInd.at(i));&#xA;            }&#xA;&#xA;            break;&#xA;        }&#xA;        else&#xA;        {&#xA;            pos = b;&#xA;        }&#xA;    }&#xA;&#xA;    // Find missing elements&#xA;    for(int32_t i = 0; i &amp;lt; constants::problemMax; ++i)&#xA;    {&#xA;        if(std::find(o1.begin(), o1.end(), i) == o1.end()) o1_missing.push_back(i);&#xA;        if(std::find(o2.begin(), o2.end(), i) == o2.end()) o2_missing.push_back(i);&#xA;    }&#xA;&#xA;    // Filter missing elements and leave only those which are in the second parent (keep the order)&#xA;    for(int32_t i = 0; i &amp;lt; static_cast&amp;lt;int32_t&amp;gt;(rhsInd.size()); i++)&#xA;    {&#xA;        if(std::find(o1_missing.begin(), o1_missing.end(), rhsInd.at(i)) != o1_missing.end()) o1_replacements.push_back(rhsInd.at(i));&#xA;    }&#xA;&#xA;    // Filter missing elements and leave only those which are in the second parent (keep the order)&#xA;    for(int32_t i = 0; i &amp;lt; static_cast&amp;lt;int32_t&amp;gt;(lhsInd.size()); i++)&#xA;    {&#xA;        if(std::find(o2_missing.begin(), o2_missing.end(), lhsInd.at(i)) != o2_missing.end()) o2_replacements.push_back(lhsInd.at(i));&#xA;    }&#xA;&#xA;    // Replace placeholders in offspring 1&#xA;    for(int32_t i = 0; i &amp;lt; placeholderCount; ++i)&#xA;    {&#xA;            auto it = std::find(o1.begin(), o1.end(), placeholder);&#xA;            *it     = o1_replacements.at(i);&#xA;    }&#xA;&#xA;    // Replace placeholders in offspring 2&#xA;    for(int32_t i = 0; i &amp;lt; placeholderCount; ++i)&#xA;    {&#xA;            auto it = std::find(o2.begin(), o2.end(), placeholder);&#xA;            *it     = o2_replacements.at(i);&#xA;    }&#xA;&#xA;    // Assign new offsprings&#xA;    lhsInd.assign(o1.begin(), o1.end());&#xA;    rhsInd.assign(o2.begin(), o2.end());&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My mutation operator(s):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;void mutateOrdered(std::vector&amp;lt;int32_t&amp;gt;&amp;amp; ind)&#xA;{&#xA;    int32_t a, b;&#xA;&#xA;    do&#xA;    {&#xA;        a = randomFromInterval(0, constants::numberDimensions);&#xA;        b = randomFromInterval(0, constants::numberDimensions);&#xA;    }&#xA;    while(a == b);&#xA;&#xA;    std::rotate(ind.begin() + a, ind.begin() + b, ind.begin() + b + 1);&#xA;}&#xA;&#xA;void mutateInverse(std::vector&amp;lt;int32_t&amp;gt;&amp;amp; ind)&#xA;{&#xA;    int32_t a, b;&#xA;&#xA;    do&#xA;    {&#xA;        a = randomFromInterval(0, constants::numberDimensions);&#xA;        b = randomFromInterval(0, constants::numberDimensions);&#xA;    }&#xA;    while(a == b);&#xA;&#xA;    if(a &amp;gt; b) std::swap(a, b);&#xA;&#xA;    std::reverse(ind.begin() + a, ind.begin() + b);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to use roulette, truncate, tournament and rank selection alorithms, but each with similar results.&lt;/p&gt;&#xA;&lt;p&gt;This is my configuration:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;populationSize = 20&#xA;selectionSize = 5&#xA;eliteSize = 1&#xA;probabilityCrossover = 0.6&#xA;probabilityMutateIndividual = 0.4&#xA;probabilityMutateGene = 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My fitness function is fairly simple, since it's real number returned by simulation program which simulates picking of orders on the current allocation we gave it. Unfortunately I cannot provide this program as its confidential. It's just a real number representing how good the current allocation is, the better the allocation is, the lower the number is (i.e. its minimization problem).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This genetic algorithm can find better solutions than just random allocation, the problem is, it gets &amp;quot;stuck&amp;quot; after lets say few thousand generations and it fails to improve furthermore, even though there are better solutions, and it will go even 20k generations with exact same ellite chromosom (don't improve at all). I tried to increase crossover/mutation probability and population size but none of it worked. Thanks for any help.&lt;/p&gt;&#xA;" OwnerUserId="18760" LastEditorUserId="156" LastEditDate="2021-05-22T19:47:20.090" LastActivityDate="2021-05-22T19:47:20.090" Title="Genetic algorithm stuck and cannot find an optimal solution" Tags="&lt;genetic-algorithms&gt;&lt;c++&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24287" PostTypeId="1" AcceptedAnswerId="24288" CreationDate="2020-10-29T08:49:57.753" Score="3" ViewCount="44" Body="&lt;p&gt;Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DHjHh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DHjHh.png&quot; alt=&quot;image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If so could you give some names or links to research papers?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="32410" LastEditDate="2021-04-26T16:32:32.270" LastActivityDate="2021-04-26T16:32:32.270" Title="Are there deep neural networks that have inputs connected with deeper hidden layers?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;architecture&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24288" PostTypeId="2" ParentId="24287" CreationDate="2020-10-29T09:03:06.773" Score="2" Body="&lt;p&gt;This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: &lt;a href=&quot;https://arxiv.org/pdf/1512.03385.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResNet&lt;/a&gt;, &lt;a href=&quot;https://openreview.net/pdf?id=HkwVAXyCW&quot; rel=&quot;nofollow noreferrer&quot;&gt;SkipRNN&lt;/a&gt;. In addition &lt;a href=&quot;https://www.aclweb.org/anthology/C16-1020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; you can find a paper that empirically explores the skip connections for sequential tagging, or this &lt;a href=&quot;https://link.springer.com/article/10.1007/s11265-020-01518-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt; for speech enhancement.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2020-10-29T09:03:06.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24290" PostTypeId="1" CreationDate="2020-10-29T10:10:49.847" Score="0" ViewCount="96" Body="&lt;p&gt;The system I'm trying to implement is a microcontroller with a connected microphone which have to recognise single words. the feature extraction is done using MFCC (and is working).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the system have to recognise [predefined, up to 20] single words each one up to 1 seconds length&lt;/li&gt;&#xA;&lt;li&gt;input audio is sampled with a frequency of 10KHz and 8 bits resolution&lt;/li&gt;&#xA;&lt;li&gt;the window is 256 sample wide (25.6 ms), hann windowed with a 15ms step (overlaying windows)&lt;/li&gt;&#xA;&lt;li&gt;the total MFCC features representing each window, is about 18 features&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I've done the above things, and tested the outputs for accuracy and computation speed so there is not much concern about the computations. now I have to implement a HMM for word recognition. I've read about the HMM and I think these parameters need to be addressed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the hidden states are the &amp;quot;actual&amp;quot; pieces of the word with 25.6ms length &lt;code&gt;represented in 18 MFCC features&lt;/code&gt;. and they count up to maximum of 64 sets in a single word (because the maximum length for input word is 1sec and each window is (25.6 - 10)millisecs)&lt;/li&gt;&#xA;&lt;li&gt;I should use Viterbi algorithm to find out the most probable word spoken untill the current state. so, if the user is saying &amp;quot;STOP&amp;quot;, the Viterbi can suggest it (with proper learning of course) when the user has spoken &amp;quot;STO..&amp;quot; . so it's some kind of prediction too.&lt;/li&gt;&#xA;&lt;li&gt;I have to determine the other HMM parameters like the emission and transition. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Viterbi_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt; for Viterbi which has written the algorithm, shows the input/output as:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WW7R6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WW7R6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;from the above:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;what is observation space?&lt;/strong&gt; the user may talk anything so it seems indefinite to me&lt;/li&gt;&#xA;&lt;li&gt;the state space is obviously the set containing all the possible &lt;code&gt;MFCC feature sets&lt;/code&gt; used in the learned word set. &lt;strong&gt;how I learn or hardcode that ?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;thanks for reading this long question patiently.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="18124" LastEditorUserId="18124" LastEditDate="2020-10-29T11:21:30.260" LastActivityDate="2020-10-29T11:21:30.260" Title="Determining observation and state spaces for viterbi algorithm in a simple word recognition system using HMM" Tags="&lt;speech-recognition&gt;&lt;hidden-markov-model&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24291" PostTypeId="1" CreationDate="2020-10-29T11:13:48.420" Score="0" ViewCount="61" Body="&lt;p&gt;There are multiple reinforcement learning (RL) benchmarks (i.e. a set of environments where we can test our RL algorithms), for example, the &lt;a href=&quot;https://arxiv.org/pdf/1801.00690.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepMind Control Suite&lt;/a&gt;. However, given that I am currently not particularly familiar with them or their details, I don't really know if any of them comes with the optimal policy (or policies) for each of the environments. So, are there any reinforcement learning benchmarks where the optimal policy is known for each environment? (So, please, do not list any benchmark that does not meet this requirement, given that I am not interested in those and that's not my question).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-10-29T11:13:48.420" Title="Are there any reinforcement learning benchmarks where the optimal policy is known for each environment?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;policies&gt;&lt;benchmarks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24292" PostTypeId="2" ParentId="10575" CreationDate="2020-10-29T11:28:03.113" Score="0" Body="&lt;p&gt;There is also the simpler &lt;strong&gt;action value function&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$$q_*(a) = \mathbb{E} \left[ R_t &#xA; \mid A_t = a\right],$$&lt;/span&gt; which we try to approximate when solving &lt;em&gt;context-free&lt;/em&gt; &lt;strong&gt;bandit problems&lt;/strong&gt;. You can also similarly define the action value function for &lt;em&gt;contextual&lt;/em&gt; bandit problems by also conditioning on the context (rather than just on the action).&lt;/p&gt;&#xA;&lt;p&gt;See chapter 2 of the book &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=48&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction&lt;/a&gt; (2nd edition) by Barto and Sutton for more details.&lt;/p&gt;&#xA;&lt;p&gt;There are also the &lt;strong&gt;afterstate value functions&lt;/strong&gt;, &lt;span class=&quot;math-container&quot;&gt;$v(s')$&lt;/span&gt;. Check &lt;a href=&quot;https://ai.stackexchange.com/q/24816/2444&quot;&gt;this question&lt;/a&gt; and &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=158&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 6.8&lt;/a&gt; of the just cited book.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, there's the &lt;strong&gt;state-action-goal value function&lt;/strong&gt;, &lt;span class=&quot;math-container&quot;&gt;$q(s, a, g)$&lt;/span&gt;, as described in the paper &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hindsight Experience Replay&lt;/a&gt; (2017), although I am not sure how we can mathematically define it (i.e. as a function of other value functions or as a Bellman equation).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T14:13:36.230" LastActivityDate="2020-11-23T14:13:36.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24295" PostTypeId="2" ParentId="14224" CreationDate="2020-10-29T14:40:39.247" Score="0" Body="&lt;p&gt;I have already given &lt;a href=&quot;https://ai.stackexchange.com/a/14226/2444&quot;&gt;an answer&lt;/a&gt; and there are &lt;a href=&quot;https://ai.stackexchange.com/a/14247/2444&quot;&gt;other good answers&lt;/a&gt;, but I would like to give another answer by quoting an excerpt from an old paper by &lt;a href=&quot;https://en.wikipedia.org/wiki/Norbert_Wiener&quot; rel=&quot;nofollow noreferrer&quot;&gt;Norbert Wiener&lt;/a&gt;, i.e. &lt;a href=&quot;https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some Moral and Technical Consequences of Automation&lt;/a&gt; (&lt;a href=&quot;https://science.sciencemag.org/content/131/3410/1355&quot; rel=&quot;nofollow noreferrer&quot;&gt;1960, Science&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As is now generally admitted, over a limited range of operation, machines act far more rapidly than human beings and are far more precise in performing the details of their operations. This being the case, even when machines do not in any way transcend man's intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant. To be effective in warding off disastrous consequences, our understanding of our man-made machines should in general develop &lt;a href=&quot;https://en.wikipedia.org/wiki/Pari_passu&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;pari passu&lt;/em&gt;&lt;/a&gt; with the performance of the machine. By the very slowness of our human actions, our effective control of our machines may be nullified. By the time we are able to react to information conveyed by our senses and stop the car we are driving, it may already have run head on into a wall.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-10-29T14:40:39.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24296" PostTypeId="1" AcceptedAnswerId="24382" CreationDate="2020-10-29T15:36:28.397" Score="2" ViewCount="65" Body="&lt;p&gt;My implementation of NEAT consistently fails to solve XOR completely. The species converge on different sub-optimal networks which map all input examples but one correctly (most commonly (1,1,0)). Do you have any ideas as to why that is?&lt;/p&gt;&#xA;&lt;p&gt;Some information which might be relevant:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I use a plain logistic activation function in each non-input node 1/(1 + exp(-x)).&lt;/li&gt;&#xA;&lt;li&gt;Some of the weights seem to grow quite large in magnitude after a large number of epochs.&lt;/li&gt;&#xA;&lt;li&gt;I use the sum squared error as the fitness function.&lt;/li&gt;&#xA;&lt;li&gt;Anything over 0.5 is considered a 1 (for comparing the output with the expected)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is one example of an evolved network. Node 0 is a bias node, the other red node is the output, the green are inputs and the blue &amp;quot;hidden&amp;quot;. Disregard the labels on the connections.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VaZcM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VaZcM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;EDIT: following the XOR suggestions on the &lt;a href=&quot;https://www.cs.ucf.edu/%7Ekstanley/neat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT users page&lt;/a&gt; of steepening the gain of the sigmoid function, a network that solved XOR was found for the first time after ca 50 epochs. But it still fails most of the time. Here is the network which successfully solved XOR:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8bEmj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8bEmj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41343" LastEditorUserId="41343" LastEditDate="2020-10-30T11:27:50.807" LastActivityDate="2020-11-02T15:00:25.173" Title="Evolved networks fail to solve XOR" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;neat&gt;&lt;neuroevolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24298" PostTypeId="1" CreationDate="2020-10-29T17:29:26.580" Score="1" ViewCount="22" Body="&lt;p&gt;I am currently learning about autoencoders and I follow &lt;a href=&quot;https://www.tensorflow.org/tutorials/generative/autoencoder&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/generative/autoencoder&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;When denoising images, authors of tutorial add an additional axis to the data and I cannot find any explanation why... I would appreciate any answer or suggestion :)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x_train = x_train[..., tf.newaxis]&#xA;x_test = x_test[..., tf.newaxis]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then the encoder is built from the following layers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; self.encoder = tf.keras.Sequential([&#xA;      layers.Input(shape=(28, 28, 1)), &#xA;      layers.Conv2D(16, (3,3), activation='relu', padding='same', strides=2),&#xA;      layers.Conv2D(8, (3,3), activation='relu', padding='same', strides=2)])&#xA;    &#xA; self.decoder = tf.keras.Sequential([&#xA;      layers.Conv2DTranspose(8, kernel_size=3, strides=2, activation='relu', padding='same'),&#xA;      layers.Conv2DTranspose(16, kernel_size=3, strides=2, activation='relu', padding='same'),&#xA;      layers.Conv2D(1, kernel_size=(3,3), activation='sigmoid', padding='same')])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38252" LastActivityDate="2020-10-29T17:29:26.580" Title="Why do we add additional axis in CNN autoencoder while denoising?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;autoencoders&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24300" PostTypeId="1" CreationDate="2020-10-29T19:18:52.527" Score="0" ViewCount="39" Body="&lt;p&gt;I'm studying with the book Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow, and I'm trying to implement the Deep Q-Network example that can be found on Github but that the input parameters for the model is a multi-dimensional array with my own environment inheriting from gym, and with a one-dimensional input works fine, similar to the CartPole-v1 example.&lt;/p&gt;&#xA;&lt;p&gt;Git Example:&#xA;&lt;a href=&quot;https://github.com/ageron/handson-ml2/blob/master/18_reinforcement_learning.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ageron/handson-ml2/blob/master/18_reinforcement_learning.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But in this case the input is not a current state, but several previous temperature states, and what I want is for the system to learn to predict when the temperature is going to be maintained, when it is going to rise or when it is going to fall.&lt;/p&gt;&#xA;&lt;p&gt;The way I see that the robot learns is that it manages to predict the trend based on the previous states, but I need to be able to enter more input parameters, such as amount of fuel, revolutions, voltage, speed and also temperature for the prediction.&lt;/p&gt;&#xA;&lt;p&gt;The problem in particular is that since I am studying until now, I lack understanding and experience, I get several errors that I have not been able to repair, (if I manage to implement it using an RNN correctly) and when trying to calculate the Q_values ​​in the training_step function I get several errors just when I get to step 50 and the function is launched in training.&lt;/p&gt;&#xA;&lt;p&gt;The key parts that I changed were the following&lt;/p&gt;&#xA;&lt;p&gt;Model&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_shape = [49, 5] # == env.observation_space.shape&#xA;n_outputs = 3 # == env.action_space.n&#xA;&#xA;model = keras.models.Sequential([&#xA;    keras.layers.Dense(32, activation=&amp;quot;elu&amp;quot;, input_shape=input_shape),&#xA;    keras.layers.Dense(32, activation=&amp;quot;elu&amp;quot;),&#xA;    keras.layers.Dense(n_outputs)&#xA;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Action&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def epsilon_greedy_policy(state, epsilon=0):&#xA;  if np.random.rand() &amp;lt; epsilon:&#xA;    return np.random.randint(3)&#xA;  else:&#xA;    Q_values = model.predict(state[np.newaxis])[:, -1][..., np.newaxis]&#xA;    return np.argmax(Q_values[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;training_step&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def training_step(batch_size):&#xA;  experiences = sample_experiences(batch_size)&#xA;  states, actions, rewards, next_states, dones = experiences&#xA;  next_Q_values = model.predict(next_states)[:, -1][..., np.newaxis]&#xA;  best_next_actions = np.argmax(next_Q_values, axis=1)&#xA;  next_mask = tf.one_hot(best_next_actions, n_outputs).numpy()&#xA;  next_best_Q_values = (target.predict(next_states)[:, -1][..., np.newaxis] * next_mask).sum(axis=1)&#xA;  target_Q_values = (rewards + (1- dones) * discount_rate * next_best_Q_values)&#xA;  target_Q_values = target_Q_values.reshape(-1, 1)&#xA;  mask = tf.one_hot(actions, n_outputs)&#xA;  with tf.GradientTape() as tape:&#xA;    all_Q_values = model(states)&#xA;    Q_values = tf.reduce_sum(all_Q_values * mask, axis=1, keepdims=True)&#xA;    loss = tf.reduce_mean(loss_fn(target_Q_values, Q_values))&#xA;  grads = tape.gradient(loss, model.trainable_variables)&#xA;  optimizer.apply_gradients(zip(grads, model.trainable_variables))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;ipython-input-84-d762453f8933&amp;gt; in training_step(batch_size)&#xA;      6   next_mask = tf.one_hot(best_next_actions, n_outputs).numpy()&#xA;      7   next_best_Q_values = (target.predict(next_states)[:, -1][..., np.newaxis] * next_mask).sum(axis=1)&#xA;----&amp;gt; 8   target_Q_values = (rewards + (1- dones) * discount_rate * next_best_Q_values)&#xA;      9   target_Q_values = target_Q_values.reshape(-1, 1)&#xA;     10   mask = tf.one_hot(actions, n_outputs)&#xA;&#xA;ValueError: operands could not be broadcast together with shapes (32,) (32,3) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I just tried to adjust the prediction based on what I learned in RNN (which is how I verify the prediction data) but it is evident that I am not understanding well how the Q values work, or how to adjust everything to use multiple inputs, I appreciate if you can help me.&lt;/p&gt;&#xA;" OwnerUserId="41914" LastActivityDate="2020-10-29T19:18:52.527" Title="How to use Deep Q-Network with two-dimensional input? Hands-on Machine Learning 2" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;q-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24301" PostTypeId="1" CreationDate="2020-10-29T19:46:02.437" Score="1" ViewCount="194" Body="&lt;p&gt;From my understanding, in a tissue where nuclei are present and need to be detected, we need to predict bounding boxes (either rectangular/circular or in the shape of the nucleus, i.e. as in instance segmentation). However, a lot of research papers start with semantic segmentation. Again, what I understood is semantic segmentation won't give the location, bounding box or count of nuclei. It will just tell that some stuff is probably nuclei and rest is probably background.&lt;/p&gt;&#xA;&lt;p&gt;So, what is the bridging that I am missing when trying to detect nuclei from semantic segmentation. I have personally done semantic segmentation but I can't seem to count/predict bounding boxes because I can't understand how to do that (for example if semantic segmentation gave a probable region for nuclei which is actually a mixture of 3 nuclei overlapping). Semantic segmentation (in the example) just stops right there.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Thresholding algorithm like Watershed might not work in some cases as demonstrated in [Nuclei Detection][1] at 23:30 onwards.&lt;/li&gt;&#xA;&lt;li&gt;Edge detection between segmented nuclei and background would not separate overlapping nuclei.&lt;/li&gt;&#xA;&lt;li&gt;Finding local maxima and putting a dot there might give rise to false positives.&lt;/li&gt;&#xA;&lt;li&gt;Finding IoU but what if the output of segmentation is not a region of classification (1s and 0s) but a continuous probability map from values between 0 to 1.&lt;/li&gt;&#xA;&lt;li&gt;Isn't finding contours and getting bounding boxes from masks using opencv a parametric method? What I mean is, it being an image processing technique, there are chances it will work for some images and won't work for some.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="41564" LastEditorUserId="41564" LastEditDate="2020-11-01T05:39:39.257" LastActivityDate="2020-11-01T05:39:39.257" Title="Getting bounding box/boundaries from segmentations in UNet Nuclei Segmentation" Tags="&lt;object-detection&gt;&lt;image-segmentation&gt;&lt;u-net&gt;&lt;mask-rcnn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24302" PostTypeId="1" CreationDate="2020-10-29T20:14:08.010" Score="1" ViewCount="32" Body="&lt;p&gt;In a Restricted Boltzmann Machine (RBM), the likelihood function is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the energy function and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is the partition function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Z = \sum_{\mathbf{v},\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The log-likelihood function is therefore:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ln(p(\mathbf{v};\mathbf{\theta})) = ln\left(\sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}\right) - ln\left(\sum_{\mathbf{v},\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since the log-likelihood function cannot be computed, its gradient is used instead with gradient descent to find the optimal parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial ln(p(\mathbf{v};\mathbf{\theta}))}{\partial \mathbf{\theta}} = -\frac{1}{\sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}} \sum_{\mathbf{h}} \left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \cdot e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}\right] + \frac{1}{\sum_{\mathbf{v},\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}} \sum_{\mathbf{v},\mathbf{h}} \left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \cdot e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\mathbf{h}|\mathbf{v}) = \frac{p(\mathbf{v},\mathbf{h})}{p(\mathbf{v})} = \frac{\frac{1}{Z} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}}{\frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}} = \frac{e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}}{\sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial ln(p(\mathbf{v};\mathbf{\theta}))}{\partial \mathbf{\theta}} = -\sum_{\mathbf{h}} \left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \cdot p(\mathbf{h}|\mathbf{v}) \right] + \frac{1}{\sum_{\mathbf{v},\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}} \sum_{\mathbf{v},\mathbf{h}} \left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \cdot e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, since:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}}{Z} = \frac{e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}}{\sum_{\mathbf{v},\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}} = p(\mathbf{v},\mathbf{h})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align} \frac{\partial ln(p(\mathbf{v};\mathbf{\theta}))}{\partial \mathbf{\theta}} &amp;amp;= -\sum_{\mathbf{h}} \left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \cdot p(\mathbf{h}|\mathbf{v}) \right] + \sum_{\mathbf{v},\mathbf{h}} \left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \cdot p(\mathbf{v},\mathbf{h})\right] \\ &amp;amp;= -\mathbb{E}_{p(\mathbf{h}|\mathbf{v})}\left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \right] + \mathbb{E}_{p(\mathbf{v},\mathbf{h})}\left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \right] \end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since both of these are expectations, they can be approximated using Monte Carlo integration:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial ln(p(\mathbf{v};\mathbf{\theta}))}{\partial \mathbf{\theta}} \approx -\frac{1}{N} \sum_{i = 1}^{N} \left[\frac{\partial E(\mathbf{v},\mathbf{h}_i;\mathbf{\theta})}{\partial \mathbf{\theta}} \right] + \frac{1}{M} \sum_{j=1}^{M} \left[\frac{\partial E(\mathbf{v}_j,\mathbf{h}_j;\mathbf{\theta})}{\partial \mathbf{\theta}} \right] $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first term can be computed beacuse it is easy to sample from &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{h}|\mathbf{v})$&lt;/span&gt;. However, it is difficult to sample from &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v},\mathbf{h})$&lt;/span&gt; directly, but since it is easy to sample from &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v}|\mathbf{h})$&lt;/span&gt;, then Gibbs sampling is used to sample from both &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{h}|\mathbf{v})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v}|\mathbf{h})$&lt;/span&gt; to approximate a sample from &lt;span class=&quot;math-container&quot;&gt;$p(\mathbf{v},\mathbf{h})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is my understanding and math correct so far?&lt;/li&gt;&#xA;&lt;li&gt;In the expression for the gradient of the log-likelihood, can expectations be interchanged with partial derivatives such that:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align} \frac{\partial ln(p(\mathbf{v};\mathbf{\theta}))}{\partial \mathbf{\theta}} &amp;amp;= -\mathbb{E}_{p(\mathbf{h}|\mathbf{v})}\left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \right] + \mathbb{E}_{p(\mathbf{v},\mathbf{h})}\left[\frac{\partial E(\mathbf{v},\mathbf{h};\mathbf{\theta})}{\partial \mathbf{\theta}} \right] \\ &amp;amp;= - \frac{\partial}{\partial \mathbf{\theta}} \mathbb{E}_{p(\mathbf{h}|\mathbf{v})}\left[E(\mathbf{v},\mathbf{h};\mathbf{\theta}) \right] + \frac{\partial}{\partial \mathbf{\theta}} \mathbb{E}_{p(\mathbf{v},\mathbf{h})}\left[E(\mathbf{v},\mathbf{h};\mathbf{\theta}) \right] \\ &amp;amp;= \frac{\partial}{\partial \mathbf{\theta}} \left(\mathbb{E}_{p(\mathbf{v},\mathbf{h})}\left[E(\mathbf{v},\mathbf{h};\mathbf{\theta}) \right] - \mathbb{E}_{p(\mathbf{h}|\mathbf{v})}\left[E(\mathbf{v},\mathbf{h};\mathbf{\theta}) \right] \right) \\ &amp;amp;\approx \frac{\partial}{\partial \mathbf{\theta}} \left(\frac{1}{M} \sum_{j=1}^{M} \left[E(\mathbf{v}_j,\mathbf{h}_j;\mathbf{\theta}) \right] - \frac{1}{N} \sum_{i = 1}^{N} \left[E(\mathbf{v},\mathbf{h}_i;\mathbf{\theta}) \right] \right) \end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;After approximating the gradient of the log-likelihood, the update rule for the parameter vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\theta}$&lt;/span&gt; is:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{\theta}_{t+1} = \mathbf{\theta}_{t} + \epsilon \frac{\partial ln(p(\mathbf{v};\mathbf{\theta}))}{\partial \mathbf{\theta}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is the learning rate. Is this update rule correct?&lt;/p&gt;&#xA;" OwnerUserId="41856" LastEditorUserId="2444" LastEditDate="2020-10-31T14:56:38.013" LastActivityDate="2020-10-31T14:56:38.013" Title="How do I derive the gradient of the log-likelihood of an RBM?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;math&gt;&lt;restricted-boltzmann-machine&gt;&lt;graphical-model&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24305" PostTypeId="1" CreationDate="2020-10-29T22:38:48.813" Score="1" ViewCount="187" Body="&lt;p&gt;I am training a CNN with a batch size of 128, but I have some fluctuations in the validation loss, which are greater than one. I want to increase my batch size to 150 or 200, but, in the code examples I have come across, the batch size is always something like 32, 64, 128, or 256. Is it a rule? Can I use other values for it?&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="2444" LastEditDate="2020-10-30T19:01:48.510" LastActivityDate="2020-10-30T19:01:48.510" Title="Are there any rules for choosing batch size?" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;batch-size&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ClosedDate="2020-10-30T19:02:39.627" ContentLicense="CC BY-SA 4.0" />
  <row Id="24306" PostTypeId="1" AcceptedAnswerId="24324" CreationDate="2020-10-30T00:25:25.873" Score="0" ViewCount="34" Body="&lt;p&gt;How to manually draw  a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-NN decision boundary with &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt; knowing the dataset&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n6TBy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n6TBy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;the labels are&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ONnUN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ONnUN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the euclidean distance between two points is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L0gm8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L0gm8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41202" LastEditorUserId="2444" LastEditDate="2020-10-30T18:54:18.213" LastActivityDate="2020-10-30T19:16:18.983" Title="How to manually draw a $k$-NN decision boundary with $k=1$ given the dataset and labels?" Tags="&lt;machine-learning&gt;&lt;k-nearest-neighbors&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24307" PostTypeId="1" CreationDate="2020-10-30T01:38:10.590" Score="3" ViewCount="76" Body="&lt;p&gt;What are the state-of-the-art results in OpenAI's gym environments? Is there a link to a paper/article that describes them and how these SOTA results were calculated?&lt;/p&gt;&#xA;" OwnerUserId="41824" LastEditorUserId="2444" LastEditDate="2020-10-30T19:14:09.057" LastActivityDate="2021-03-30T19:04:15.437" Title="What are the state-of-the-art results in OpenAI's gym environments?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;gym&gt;&lt;state-of-the-art&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24308" PostTypeId="1" CreationDate="2020-10-30T05:35:53.523" Score="0" ViewCount="41" Body="&lt;p&gt;I am working in a project named &amp;quot;Handwritten Math Evaluation&amp;quot;. So what basically happens in this is that there are 11 classes of (0 - 9) and (+, -) each containing 50 clean handwritten digits in them. Then I trained a CNN model for it with 80 % of data used in training and 20 % using in testing of model which results in an accuracy of 98.83 %. Here is the code for the architecture of CNN model:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd &#xA;import numpy as np &#xA;import pickle &#xA;np.random.seed(1212) &#xA;import keras &#xA;from keras.models import Model &#xA;from keras.layers import *&#xA;from keras import optimizers &#xA;from keras.layers import Input, Dense &#xA;from keras.models import Sequential &#xA;from keras.layers import Dense &#xA;from keras.layers import Dropout &#xA;from keras.layers import Flatten &#xA;from keras.layers.convolutional import Conv2D &#xA;from keras.layers.convolutional import MaxPooling2D &#xA;from keras.utils import np_utils &#xA;from keras import backend as K  &#xA;from keras.utils.np_utils import to_categorical &#xA;from keras.models import model_from_json&#xA;import matplotlib.pyplot as plt&#xA;model = Sequential() &#xA;model.add(Conv2D(30, (5, 5), input_shape =(28,28,1), activation ='relu')) &#xA;model.add(MaxPooling2D(pool_size =(2, 2))) &#xA;model.add(Conv2D(15, (3, 3), activation ='relu')) &#xA;model.add(MaxPooling2D(pool_size =(2, 2))) &#xA;model.add(Dropout(0.2)) &#xA;model.add(Flatten()) &#xA;model.add(Dense(128, activation ='relu')) &#xA;model.add(Dense(50, activation ='relu')) &#xA;model.add(Dense(12, activation ='softmax')) &#xA;# Compile model &#xA;model.compile(loss ='categorical_crossentropy', &#xA;            optimizer ='adam', metrics =['accuracy']) &#xA;model.fit(X_train, y_train, epochs=1000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now each image in dataset is preprocessed as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import cv2&#xA;im = cv2.imread(path)&#xA;im_gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)&#xA;ret, im_th = cv2.threshold(im_gray, 90, 255, cv2.THRESH_BINARY_INV)&#xA;ctrs, hier = cv2.findContours(im_th.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)&#xA;rects = [cv2.boundingRect(ctr) for ctr in ctrs]&#xA;rect = rects[0]&#xA;im_crop =im_th[rect[1]:rect[1]+rect[3],rect[0]:rect[0]+rect[2]]&#xA;im_resize = cv2.resize(im_crop,(28,28))&#xA;im_resize = np.array(im_resize)&#xA;im_resize=im_resize.reshape(28,28)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have made an evaluation function which solves simple expression like 7+8 :-&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def evaluate(im):&#xA;    s = ''&#xA;    data = []&#xA;    im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)&#xA;    ret, im_th = cv2.threshold(im_gray, 90, 255, cv2.THRESH_BINARY_INV)&#xA;    ctrs, hier = cv2.findContours(im_th.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)&#xA;    sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])&#xA;    boundingBoxes = [cv2.boundingRect(c) for c in ctrs]&#xA;    look_up = ['0','1','2','3','4','5','6','7','8','9','+','-'] &#xA;    i=0&#xA;    for c in ctrs:&#xA;        rect = boundingBoxes[i]&#xA;        im_crop = im_th[rect[1]:rect[1]+rect[3], rect[0]:rect[0]+rect[2]]&#xA;        im_resize = cv2.resize(im_crop,(28,28))&#xA;        im_resize = np.array(im_resize)&#xA;        im_resize = im_resize.reshape(28,28,1)&#xA;        data.append(im_resize)&#xA;        i+=1&#xA;    data = np.array(data)&#xA;    predictions = model.predict(data)&#xA;    i=0&#xA;    while i&amp;lt;len(boundingBoxes):&#xA;        rect = boundingBoxes[i]&#xA;        print(rect[2],rect[3])&#xA;        print(predictions[i])&#xA;        s += look_up[predictions[i].argmax()]&#xA;        i+=1&#xA;    return s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need help extending this, thought for compound fractions but the problem is that they are identical to subtraction sign when resized to (28, 28). So I need help in distinguishing between them.&lt;/p&gt;&#xA;&lt;p&gt;This is my first question, so please let me know if any details are left.&lt;/p&gt;&#xA;" OwnerUserId="41931" LastEditorUserId="32410" LastEditDate="2020-11-13T18:22:41.737" LastActivityDate="2020-11-13T18:22:41.737" Title="Distinguishing between handwritten compound fraction and subtraction" Tags="&lt;computer-vision&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;keras&gt;&lt;opencv&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24309" PostTypeId="1" CreationDate="2020-10-30T05:36:13.360" Score="0" ViewCount="42" Body="&lt;p&gt;I am reading the &lt;a href=&quot;https://arxiv.org/pdf/1312.4400.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Network in Network&lt;/a&gt; paper. This is the equation they introduce for their &lt;code&gt;mlpconv&lt;/code&gt; layer (equation 2 in the paper, page 3):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;f_{i, j, k_{1}}^{1} &amp;amp;=\max \left(w_{k_{1}}^{1} x_{i, j}+b_{k_{1}}, 0\right) \\&#xA;&amp;amp; \vdots \\&#xA;f_{i, j, k_{n}}^{n} &amp;amp;=\max \left(w_{k_{n}}^{n} f_{i, j}^{n-1}+b_{k_{n}}, 0\right) .&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This paper is largely recognized for introducing &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolutions&lt;/strong&gt;, but the &lt;code&gt;mlpconv&lt;/code&gt; layer does not seem to resemble it. It seems that the output of a &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolution is simply the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_{i, j, k_{1}}^{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This would be a &lt;em&gt;single feature map&lt;/em&gt;, and you would calculate them in &lt;strong&gt;parallel&lt;/strong&gt; using the patch corresponding to &lt;span class=&quot;math-container&quot;&gt;$x(i, j)$&lt;/span&gt;. You would &lt;strong&gt;not&lt;/strong&gt; use a &amp;quot;depth&amp;quot; component in a typical &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolution, but the &lt;code&gt;mlpconv&lt;/code&gt; layer does exactly that, stacking the previous outputs on top of one another.&lt;/p&gt;&#xA;&lt;p&gt;Why is the Network in Network paper usually referenced only for &lt;span class=&quot;math-container&quot;&gt;$1 \times 1$&lt;/span&gt; convolutions when it also has a depth component to it? Is my understanding wrong?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;I've been doing some thinking. These actually are 1x1 convolutions, since 1 input pixel is connected to multiple neurons in a dense layer, meaning that each neuron essentially acts as a channel. For example, if your input is 1 pixel and your output is 10 dense neurons (like in the &amp;quot;mlpconv layer&amp;quot;), then that is equivalent to a 1x1x10 convolution. In the paper, when they are chaining dense/fully-connected layers, they are basically stacking 1x1 convolutions.&lt;/p&gt;&#xA;" OwnerUserId="41932" LastEditorUserId="41932" LastEditDate="2020-11-01T05:18:44.523" LastActivityDate="2020-11-01T05:18:44.523" Title="Is the &quot;mlpconv&quot; layer in the &quot;Network in Network&quot; paper computing $1 \times 1$ convolutions or not?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;papers&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24311" PostTypeId="1" CreationDate="2020-10-30T08:58:48.547" Score="1" ViewCount="62" Body="&lt;p&gt;In object detection, we can resize images by keeping the ratio the same as the original image, which is often known as &amp;quot;letterbox&amp;quot; resize.&lt;/p&gt;&#xA;&lt;p&gt;My questions are&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why do we need to resize images? If we resize images to have all the same dimensions, given that some original images are too long vertically or horizontally, we will lose a lot of features in those images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the &amp;quot;letterbox&amp;quot; method is better than &amp;quot;normal resize&amp;quot; (i.e. without keeping the aspect ratio, e.g. the result of the application of &lt;a href=&quot;https://docs.opencv.org/master/da/d54/group__imgproc__transform.html#ga47a974309e9102f5f08231edc7e7529d&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV's &lt;code&gt;resize&lt;/code&gt; function with the parameter &lt;code&gt;interpolation&lt;/code&gt; set to &lt;code&gt;cv2.INTER_AREA&lt;/code&gt;&lt;/a&gt;), why don't people apply it in the classification task?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="41287" LastEditorUserId="2444" LastEditDate="2020-11-07T11:23:56.670" LastActivityDate="2020-11-07T11:23:56.670" Title="Why do we resize images before using them for object detection?" Tags="&lt;classification&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;&lt;data-preprocessing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24315" PostTypeId="2" ParentId="3920" CreationDate="2020-10-30T11:09:40.210" Score="1" Body="&lt;p&gt;ML, being a relatively young and fast-developing field, has numerous (near-)synonyms for many concepts.&lt;/p&gt;&#xA;&lt;p&gt;One paradigm difference is whether a model is learned from a static, pre-defined set of data, or whether it adapts as new data is presented to it over time.&lt;/p&gt;&#xA;&lt;p&gt;Some of the terms used to describe these two paradigms respectively (with subtle differences in meaning between authors/terms) are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Offline / batch / &lt;a href=&quot;https://www.cs.uic.edu/%7Eliub/lifelong-learning/introduction-lifelong-learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;isolated&lt;/a&gt; learning&lt;/li&gt;&#xA;&lt;li&gt;Online / continual / continuous / incremental / lifelong learning&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Further, some familiar branches of ML (like Transfer Learning and Multi-task learning) have a lot of intersection with Continual Learning.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;sup&gt;&#xA;&lt;p&gt;Related q's:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/questions/23226/what-is-the-difference-between-active-learning-and-online-learning&quot;&gt;What is the difference between active learning and online learning?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/897/online-vs-offline-learning&quot;&gt;https://stats.stackexchange.com/questions/897/online-vs-offline-learning&lt;/a&gt;&#xA;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23503" LastEditorUserId="23503" LastEditDate="2020-11-09T11:43:24.067" LastActivityDate="2020-11-09T11:43:24.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24316" PostTypeId="1" CreationDate="2020-10-30T11:34:19.857" Score="1" ViewCount="24" Body="&lt;p&gt;I am trying to train a Unet network with Synthetic data to do binary segmentation due to the fact that is is not easy to collect real data.&lt;/p&gt;&#xA;&lt;p&gt;And there is something in the training process that I do not understand.&lt;/p&gt;&#xA;&lt;p&gt;I have a gap in the IoU metrics between the training and the validation (despite having really similar data).&lt;/p&gt;&#xA;&lt;p&gt;My training Iou is around 95 % and my validation is around 70 %.&#xA;And the dice loss is around 0.007.&#xA;The IoU is calculated on the inverted mask used for the loss.&lt;/p&gt;&#xA;&lt;p&gt;So I do not understand why there is this gap whereas the images in validation has been created from the same background dataset and the same object dataset which has been randomly placed on background ( + rotation and rescaled randomly). The only difference is an aggressive data augmentation used for training dataset.&lt;/p&gt;&#xA;&lt;p&gt;In my opinion, it is not overfitting since the loss value and comportment is very similar for train and val. Moreover, it seems very unlikely to me that the model overfit with same backgrounds and objects or at least model should have very good IoU for train and val if it was overfitting.&lt;/p&gt;&#xA;&lt;p&gt;So could the data augmentation  lead to the model learning features which corresponds to data augmented data (even if loss is similar) and not to the real data explaining the gap in IoU between train and val ?&lt;/p&gt;&#xA;" OwnerUserId="27718" LastEditorUserId="27718" LastEditDate="2020-11-02T07:37:44.877" LastActivityDate="2020-11-02T07:37:44.877" Title="Could the data augmentation lead to the model learning features which corresponds to data augmented data and not to the real data?" Tags="&lt;training&gt;&lt;loss-functions&gt;&lt;image-segmentation&gt;&lt;metric&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24317" PostTypeId="1" CreationDate="2020-10-30T12:55:45.497" Score="0" ViewCount="18" Body="&lt;p&gt;I made a stack of bidirectional LSTM layers following by Dense layers (with swish activation functions) in order to predict a continuous value between 0 and 2. I compiled the model with mean_squared_error loss function and Adam optimizer. I have a batch size of 32.&lt;/p&gt;&#xA;&lt;p&gt;The issue I encounter is that the extremities are under predicted by the network.&#xA;&lt;em&gt;i.e&lt;/em&gt; The prediction doesn't go upper ~1.85 and lower ~ 0.15. I would like to affine the prediction in the extremities. See image below (the point at 2:2 is not part of the LSTM prediction):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Lnc6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Lnc6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried to min-max scaled the labels from 0 to 1 instead of 0-2 and I tried other activation functions (sigmoid, linenar) but it doesn't change much the prediction. May be there is a specific loss function or optimizer to use in this case?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for helping me!&lt;/p&gt;&#xA;" OwnerUserId="41942" LastEditorUserId="41942" LastEditDate="2020-10-30T13:02:06.463" LastActivityDate="2020-10-30T13:02:06.463" Title="How to afine the extremity values in regression prediction with Keras?" Tags="&lt;keras&gt;&lt;long-short-term-memory&gt;&lt;loss-functions&gt;&lt;optimization&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24318" PostTypeId="1" CreationDate="2020-10-30T13:59:40.747" Score="0" ViewCount="15" Body="&lt;p&gt;I have a database containing summary of the movie, the price of the movie, location, language. I would like to make a prediction on 'like' based on previous like of a user.&lt;/p&gt;&#xA;&lt;p&gt;I am wonder how the output of spacy be fed into the input of a Keras. I understand that spacy offer named recognition which extract the keyword from a text, and entity linking which map the keyword to a concept id. However, how would the resulting concept ids subsequently be fed into a machine learning model together with price, location, language, etc.&lt;/p&gt;&#xA;&lt;p&gt;Or I should just word embed the text, then concat the output of the embedded layer with other categorical/numerical features.&lt;/p&gt;&#xA;" OwnerUserId="41943" LastActivityDate="2020-10-30T13:59:40.747" Title="How would named recognition and entity linking interface with subsequent machine learning model?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;keras&gt;&lt;spacy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24319" PostTypeId="2" ParentId="24305" CreationDate="2020-10-30T15:32:18.983" Score="1" Body="&lt;p&gt;Data that has a size of a factor of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; (aka &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; for some integer &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;) allows for easier memory management because data can be organized contiguously (without gaps). This allows for faster memory reading and thus faster iteration time in general. From a computational point of view, this is important as it can be taken advantage of by the compiler and speed up iteration loops. This is why batch sizes are chosen as such in practice. However, this does not necessarily imply better training results.&lt;/p&gt;&#xA;&lt;p&gt;In regards to your question of &amp;quot;Can I use other values for batch size?&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;Yes, you can use different values, and, for the most part, you probably won't see the difference in computational performance because of the speed of modern training APIs. So, until training a large model with a lot of compute where this optimization has a larger impact, feel free to experiment :)&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="2444" LastEditDate="2020-10-30T18:59:30.187" LastActivityDate="2020-10-30T18:59:30.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24322" PostTypeId="1" CreationDate="2020-10-30T16:19:03.097" Score="1" ViewCount="41" Body="&lt;p&gt;I'm trying to get a detected car's orientation when object detection is applied. For instance, when we apply object detection on a car and get a bounding box, is there any ways or methods to calculate where the heading is or the orientation or direction of the car (just 2D plane is fine)?&lt;/p&gt;&#xA;&lt;p&gt;Any thoughts or ideas would be helpful.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/km1Ey.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/km1Ey.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-10-31T16:48:54.510" Title="Get object's orientation or angle after object detection" Tags="&lt;neural-networks&gt;&lt;ai-design&gt;&lt;object-detection&gt;&lt;object-recognition&gt;&lt;yolo&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24324" PostTypeId="2" ParentId="24306" CreationDate="2020-10-30T18:00:13.643" Score="1" Body="&lt;p&gt;This is a rather involved task. What to do from a high-level theoretical perspective might be easy to see, but it's difficult putting that into code from scratch.&lt;/p&gt;&#xA;&lt;p&gt;Doing this in Python using existing libraries in not too complicated, though.&#xA;See for example this &lt;a href=&quot;https://pythonspot.com/k-nearest-neighbors/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; or this &lt;a href=&quot;https://stackoverflow.com/a/52407052/11478452&quot;&gt;StackOverflow post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Theoretically, I would first plot (draw) the points from your dataset in a graph and then watch out for &amp;quot;decision points&amp;quot; half-way &lt;em&gt;in between any two near-by points (from the dataset) from distinct classes&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For the next step, keep in mind those decision points. Given close-by data points from classes (e.g.) &lt;em&gt;A&lt;/em&gt; and &lt;em&gt;B&lt;/em&gt;, &lt;em&gt;imagine a straight line connecting these two points from separate classes&lt;/em&gt;. Next, take the point half-way along that imaginary line (i.e. your decision point) and draw a &amp;quot;soft&amp;quot; line (maybe using pencil instead of pen) orthogonal/perpendicular to that imaginary line which intersects the imaginary line in the decision point.&#xA;Do that for all combinations of &amp;quot;reasonably&amp;quot; close points from different classes.&lt;/p&gt;&#xA;&lt;p&gt;Parts of the lines you have just drawn will define the final decision boundary. Next, think of each line as consisting of multiple elements, which are separated from one another by means of the points of intersection with other drawn lines. In other words, split lines into elements wherever they intersect other lines. Now, decide which of these elements to outline as the eventual decision boundary (finally using a &lt;strong&gt;pen&lt;/strong&gt; instead of pencil). This step simply involves human intelligence and is difficult to describe. Whenever a line's element accurately separates (logically speaking) two classes, indicate it using a pen. Otherwise, if it does not contribute to separating two classes, don't indicate it.&lt;/p&gt;&#xA;&lt;p&gt;After having indicated the final decision boundary using a pen, simply erase the pencil drawings. Now, you should be left with a decision boundary.&lt;/p&gt;&#xA;&lt;p&gt;I hope this is clear and accurate enough.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-10-30T19:16:18.983" LastActivityDate="2020-10-30T19:16:18.983" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24326" PostTypeId="1" CreationDate="2020-10-30T19:40:25.220" Score="1" ViewCount="29" Body="&lt;p&gt;My agent uses an &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy strategy to learn. The exploration rate (i.e. &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;) decays throughout the training. I've seen examples where people update &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; every time an action is taken, while others update it at the end of the episode. If updated at every action, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is more continuous. Does it matter? Is there a standard? Is one better than another?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-10-31T22:46:11.507" LastActivityDate="2020-10-31T22:46:11.507" Title="Should the exploration rate be updated at the end of the episode or at every step?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24327" PostTypeId="1" AcceptedAnswerId="24335" CreationDate="2020-10-30T20:26:22.190" Score="1" ViewCount="35" Body="&lt;p&gt;Let's assume I have a simple feedforward neural network whose input contains binary 0/1 features and output is also binary two classes.&lt;/p&gt;&#xA;&lt;p&gt;Is it better, worse, or maybe totally indifferent, for every such binary feature to be in just one column or maybe it would be better to split one feature into two columns in a way that the second column will have the opposite value, like that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;feature_x (one column scenario) &#xA;&#xA;[0]&#xA;&#xA;[1]&#xA;&#xA;[0]&#xA;&#xA;&#xA;feature_x (two columns scenario)&#xA;&#xA;[0, 1]&#xA;&#xA;[1, 0]&#xA;&#xA;[0, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know this might seem a bit weird and probably it is not necessary, but I have a feeling like there might be a difference for a network especially for its inner workings and how neurons in the next layers see such data. Has anyone ever researched that aspect?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="32410" LastEditDate="2021-04-26T16:32:14.807" LastActivityDate="2021-04-26T16:32:14.807" Title="Should binary feature be in one or two columns in deep neural networks?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;data-preprocessing&gt;&lt;geometric-deep-learning&gt;&lt;binary-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24328" PostTypeId="2" ParentId="24322" CreationDate="2020-10-30T20:54:51.833" Score="1" Body="&lt;p&gt;I think the problem can be phrased (more generally) as a &lt;em&gt;Pose Estimation Problem&lt;/em&gt;.  That term might help in obtaining better search results when searching for relevant papers.&lt;/p&gt;&#xA;&lt;p&gt;One paper that I found on the given topic was this &lt;a href=&quot;https://arxiv.org/abs/1912.04363&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt;. Even if it is maybe (for whatever reason) not what you are looking for precisely, it might still contain valuable references to different techniques that might be applicable to your problem at hand.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-10-30T20:54:51.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24329" PostTypeId="2" ParentId="21838" CreationDate="2020-10-31T01:34:30.230" Score="1" Body="&lt;p&gt;There are two &amp;quot;inputs&amp;quot; into Wavenet:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the previously generated samples of the waveform, which are usually encoded into multiple channels, like into 256 channels using 8-bit &lt;a href=&quot;https://en.wikipedia.org/wiki/%CE%9C-law_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;mu-law encoding&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;local conditioning, which can be things like linguistic features such as phoneme classes (used in the original wavenet paper), or frequencies like mel spectrogram values (used in the &lt;a href=&quot;https://arxiv.org/pdf/1712.05884.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tacotron 2 paper&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The local conditioning signal usually has a much lower resolution than the waveform itself. For example, the Tacotron 2 paper mentioned using &amp;quot;50 ms frame size, 12.5 ms frame hop&amp;quot; to derive its mel spectogram values. At 24 kHz, each waveform sample has a duration of just 0.0416 ms. So in order to using the spectogram information to condition the waveform generation, it would have to be upsampled to spread out along the time dimension. (If you were locally conditioning using letters, you might turn &amp;quot;dog&amp;quot; into &amp;quot;dddddoooooooooggg&amp;quot; in order to use the letter &amp;quot;d&amp;quot; to generate 5 samples in the output waveform, etc.).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the input to the WaveNet, isn't this a mel-spectrum input and not just 1 floating point value for raw audio?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the implementation you linked to, it looks like variables that start with &amp;quot;c&amp;quot; refer to &amp;quot;conditioning&amp;quot; signals. So &amp;quot;hparams.cin_channels&amp;quot; indicates how many channels the input conditioning signal has. If that signal is mel spectogram with 80 channels, then it would be set to 80.&lt;/p&gt;&#xA;&lt;p&gt;It looks like the &amp;quot;inference/input_convolution&amp;quot; layer is processing the actual waveform, not the spectogram conditioning signal.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Is there reason for upsampling stride values to be [11, 25], like are the specific numbers 11 and 25 special or relevant in affecting other shapes/dimensions?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I suspect (though I'm not too familiar with tensorflow conventions) that the second dimension is time, which is why you're seeing the model upsample the conditioning signal in that direction. Those values may be required to match the resolution of the waveform being generated.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Why is the input-channels in residual_block_causal_conv 128 and residual_block_cin_conv 80? What exactly is their inputs?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &amp;quot;c&amp;quot; in &amp;quot;cin_conv&amp;quot; appears to mean that it is for the conditioning signal, which is why is has a different dimension than the waveform's embedding size.&lt;/p&gt;&#xA;&lt;p&gt;I hope this helps. You could also try opening up an issue on that github repo to try getting help directly from the author.&lt;/p&gt;&#xA;" OwnerUserId="41227" LastActivityDate="2020-10-31T01:34:30.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24330" PostTypeId="2" ParentId="8007" CreationDate="2020-10-31T02:20:28.127" Score="1" Body="&lt;p&gt;The original wavenet and the implementation you linked to is globally conditioned on speaker embeddings, which means that the network was given a unique identifier for the person speaking each time it trained on an audio clip. This allows the network to learn to mimic the voice of each person in the training data, but it only learns &lt;em&gt;their&lt;/em&gt; voices, not arbitrary people's voices.&lt;/p&gt;&#xA;&lt;p&gt;You might be able to do what you're describing by globally conditioning on different features of the speakers, such as age etc, though this would require a lot a speakers to span those dimensions.&lt;/p&gt;&#xA;&lt;p&gt;I think it would definitely be possible to use style transfer for this, similar to the work done on tweaking people's age in images [1]. Some similar work has been done for audio in order to change the accent of a voices between accents [2], so I wouldn't be surprised to see more examples of this in the near future.&lt;/p&gt;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.youtube.com/watch?v=2qMw8sOsNg0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=2qMw8sOsNg0&lt;/a&gt;&#xA;[2] &lt;a href=&quot;https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41227" LastActivityDate="2020-10-31T02:20:28.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24331" PostTypeId="1" CreationDate="2020-10-31T04:41:17.643" Score="1" ViewCount="13" Body="&lt;p&gt;I'm trying to see how to detect the location of a soccer ball in the field using the live camera. What are some ways to achieve this?&lt;/p&gt;&#xA;&lt;p&gt;1- Assuming we have a fixed camera with a wide shot. How to find the ball location on the actuall field?&lt;/p&gt;&#xA;&lt;p&gt;2- A camera is zooming into the ball. But we know the location of the camera and maybe it's turning angle. Can we estimate the ball location on the field using this info? Or maybe we need additional info? Can we do it with two cameras as reference points?&lt;/p&gt;&#xA;&lt;p&gt;Any thoughts would be helpful.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hh2oSm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hh2oSm.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VraAtm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VraAtm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-10-31T04:41:17.643" Title="Find object's location in an area using computer vision" Tags="&lt;neural-networks&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24332" PostTypeId="1" AcceptedAnswerId="24384" CreationDate="2020-10-31T06:23:52.827" Score="1" ViewCount="141" Body="&lt;p&gt;I'm facing a situation where I've to fetch probabilities from BERT MLM for multiple words in a single sentence.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Original : &amp;quot;Mountain Dew is an energetic drink&amp;quot;&#xA;Masked : &amp;quot;[MASK] is an energetic drink&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But BERT MLM task doesn't consider two tokens at a time for the MASK. I strongly think that there should be some sort of work around that I'm unable to find other than fine-tuning.&lt;/p&gt;&#xA;" OwnerUserId="25676" LastActivityDate="2020-11-02T15:25:07.997" Title="Is there a way to provide multiple masks to BERT in MLM task?" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;&lt;language-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24333" PostTypeId="1" CreationDate="2020-10-31T08:21:43.130" Score="1" ViewCount="18" Body="&lt;p&gt;I have 999 signals, each with separate day timestamp, each T=10s long, sampled with fs=25kHz. This gives N=250,000 samples in total.&lt;/p&gt;&#xA;&lt;p&gt;My task was to obtain the averaged magnitude spectrum for each signal. For example, for k=100, the signal is divided into k-equal fragments, 0.1s and 2500 samples long. Then FFT is computed on all fragments and mean value is calculated for all spectral component (mean for each frequency from DC to Nyquist Frequency).&#xA;The averaged spectrum for each signal for k=100 contains 1251 values and 1251 frequency points (0-fs/2).&lt;/p&gt;&#xA;&lt;p&gt;My question is, how can prepare the train dataset for multiple Machine Learning models based on that data, so I can predict when is the threshold time, before the failure of the machine occurs?&#xA;Do i treat each spectral component (frequency) as separate feature? Or there is a different approach ?&lt;/p&gt;&#xA;" OwnerUserId="41953" LastActivityDate="2020-10-31T08:21:43.130" Title="How can I predict an anomaly based on FFT of multiple signals?" Tags="&lt;python&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24334" PostTypeId="2" ParentId="4190" CreationDate="2020-10-31T08:32:47.077" Score="2" Body="&lt;p&gt;This is an active research topic.&lt;/p&gt;&#xA;&lt;p&gt;Consider reading&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;J.Pitrat's &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1848211015&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings, the conscience of a conscious machine&lt;/em&gt;&lt;/a&gt; book&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This &lt;a href=&quot;https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Understanding machine learning&lt;/em&gt;&lt;/a&gt; book&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;old Lenat's papers on &lt;a href=&quot;https://en.wikipedia.org/wiki/Eurisko&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eurisko&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;and recent papers such as &lt;a href=&quot;https://infoscience.epfl.ch/record/192344/files/Bordes_AAAI_2011.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning Structured Embeddings of Knowledge Bases&lt;/em&gt;&lt;/a&gt; and several others in &lt;a href=&quot;https://www.journals.elsevier.com/artificial-intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Intelligence Journal&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;about the &lt;a href=&quot;https://en.wikipedia.org/wiki/MILEPOST_GCC&quot; rel=&quot;nofollow noreferrer&quot;&gt;MILEPOST GCC&lt;/a&gt; and &lt;a href=&quot;http://ctuning.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ctuning&lt;/a&gt; projects&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2352154618301943&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; on &lt;em&gt;Reconciling deep learning with symbolic artificial intelligence: representing objects and relations&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My hope is to get funded so that &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; could (in a few years) be able to deduce some rules and metarules.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="2444" LastEditDate="2020-10-31T18:14:17.640" LastActivityDate="2020-10-31T18:14:17.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24335" PostTypeId="2" ParentId="24327" CreationDate="2020-10-31T09:31:18.320" Score="0" Body="&lt;p&gt;You're simply adding a redundant feature by having it as two: &lt;span class=&quot;math-container&quot;&gt;$X_2 = 1 - X_1$&lt;/span&gt;. It would be equally useful to duplicate the first column. At best this will not improve your model, at worst it will decrease accuracy.&lt;/p&gt;&#xA;" OwnerUserId="34473" LastActivityDate="2020-10-31T09:31:18.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24336" PostTypeId="2" ParentId="24322" CreationDate="2020-10-31T16:48:54.510" Score="1" Body="&lt;p&gt;There is a paper &lt;a href=&quot;https://github.com/natanielruiz/deep-head-pose&quot; rel=&quot;nofollow noreferrer&quot;&gt;face pose estimation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It uses a very straight forward technique, and very obvious augmentaions to achieve nice results.&lt;/p&gt;&#xA;&lt;p&gt;You could use exactly the same if you have a tagged dataset for cars rather than for faces.&lt;/p&gt;&#xA;&lt;p&gt;I was able to reproduce the results myself a while back.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastActivityDate="2020-10-31T16:48:54.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24337" PostTypeId="5" CreationDate="2020-10-31T17:17:17.207" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T17:17:17.207" LastActivityDate="2020-10-31T17:17:17.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24338" PostTypeId="4" CreationDate="2020-10-31T17:17:17.207" Score="0" Body="For questions about the CartPole environment, which is often used as a benchmark to assess the quality of reinforcement learning algorithms (or agents)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T17:17:17.207" LastActivityDate="2020-10-31T17:17:17.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24339" PostTypeId="5" CreationDate="2020-10-31T17:26:46.290" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T17:26:46.290" LastActivityDate="2020-10-31T17:26:46.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24340" PostTypeId="4" CreationDate="2020-10-31T17:26:46.290" Score="0" Body="For questions about the tic-tac-toe (aka noughts and crosses) in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T17:26:46.290" LastActivityDate="2020-10-31T17:26:46.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24341" PostTypeId="1" CreationDate="2020-10-31T17:35:38.393" Score="1" ViewCount="56" Body="&lt;p&gt;I am looking for a way to re-identify/classify/recognize x real life objects (x &amp;lt; 50) with a camera. Each object should be presented to the AI only once for learning and there's always only one of these objects in the query image. New objects should be addable to the list of &amp;quot;known&amp;quot; objects. The objects are not necessarily part of ImageNet nor do I have a training dataset with various instances of these objects.&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the beginning I have no &amp;quot;known&amp;quot; objects. Now I present a&#xA;smartphone, a teddy bear and a pair of scissors to the system. It&#xA;should learn to re-identify these three objects if presented in the&#xA;future. The objects will be the exact same objects, i.e. not a different phone, but definitely in a different viewing angle, lighting etc.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My understanding is that I would have to place each object in an embedding space and do a simple nearest neighbor lookup in that space for the queries. Maybe just use a trained ResNet, cut off the classification and simply use the output vector for each object? Not sure what the best way would be.&lt;/p&gt;&#xA;&lt;p&gt;Any advice or hint to the right direction would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="41963" LastActivityDate="2021-03-31T13:03:10.353" Title="Single-Shot Learning for Object Re-Identification" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;object-recognition&gt;&lt;one-shot-learning&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24342" PostTypeId="1" CreationDate="2020-10-31T17:47:50.040" Score="3" ViewCount="83" Body="&lt;p&gt;I'm coding my own version of MuZero. However, I don't understand how it supposed to learn to play well for both players in a two-player game.&lt;/p&gt;&#xA;&lt;p&gt;Take Go for example. If I use a single MCTS to generate an entire game (to be used in the training stage), couldn't MuZero learn to play badly for black in order to become good at predicting a win for white? What is forcing it to play well at every turn?&lt;/p&gt;&#xA;" OwnerUserId="10202" LastEditorUserId="10202" LastEditDate="2020-11-01T13:43:14.560" LastActivityDate="2020-11-01T13:43:14.560" Title="How does MuZero learn to play well for both sides of a two-player game?" Tags="&lt;reinforcement-learning&gt;&lt;self-play&gt;&lt;muzero&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24344" PostTypeId="5" CreationDate="2020-10-31T18:06:49.830" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T18:06:49.830" LastActivityDate="2020-10-31T18:06:49.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24345" PostTypeId="4" CreationDate="2020-10-31T18:06:49.830" Score="0" Body="For questions about the Z3 Theorem Prover, a cross-platform satisfiability modulo theories (SMT) solver by Microsoft." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T18:06:49.830" LastActivityDate="2020-10-31T18:06:49.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24346" PostTypeId="2" ParentId="24307" CreationDate="2020-10-31T18:37:10.433" Score="0" Body="&lt;p&gt;There is the &lt;a href=&quot;https://github.com/openai/gym/wiki/Leaderboard&quot; rel=&quot;nofollow noreferrer&quot;&gt;leaderboard&lt;/a&gt; page at the gym GitHub repository that contains links to specific implementations that &amp;quot;solve&amp;quot; the different gym environments, where &amp;quot;to solve&amp;quot; means &amp;quot;to reach a certain level of performance&amp;quot;, which, given a fixed reward function, is typically measured as the average (episodic) return/reward. For example, in the case of the CartPole environment, you solve it when you get an average reward of &lt;span class=&quot;math-container&quot;&gt;$195.0$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; consecutive trials.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-10-31T18:37:10.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24348" PostTypeId="2" ParentId="24342" CreationDate="2020-10-31T20:49:09.310" Score="1" Body="&lt;p&gt;Both players are represented by the exact same network with the exact same weights(similar to AplhaGO, AlphaGoZero and AlphaZero). So, they will both behave identical. Because you only have a single network, MuZero can not learn two different policies, but only one.&lt;/p&gt;&#xA;&lt;p&gt;You can also think of this in the following way: MuZero actually learn to play only with white(or black, but just one of them) without knowing to play with the other color (at least multiple implementations of the previous algorithms like AlphaGo Zero and AlphaZero that are similar to MuZero are doing exactly this). So, in order to trick it to also be able to play with the other color, when the network need to play with black, you just flip the colors on the table so that black becomes white(and white becomes black) and the network knows what to do. After choosing the move, you flip the whole thing back and that is usually how it is done. So, from the perspective of your network, it will always play white, but because you do the flipping of the colors you can actually put them to play against each other without them knowing that.&lt;/p&gt;&#xA;&lt;p&gt;Even without using this trick of flipping the color of the table, by doing the MCTS simulations, you will have for each state the statistics for the actions, and usually as you do more simulations, this statistics show you which actions are the best in each state. And when you train, you try to imitate this. So, your network will learn in each state which actions are the best, and this is the reason why it learns to take the best possible action in each state.&lt;/p&gt;&#xA;" OwnerUserId="37919" LastActivityDate="2020-10-31T20:49:09.310" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24349" PostTypeId="1" CreationDate="2020-10-31T20:49:18.150" Score="2" ViewCount="88" Body="&lt;p&gt;Are there any meaningful books entirely written by an artificial intelligence? I mean something with meaning, unlike random words or empty books.&#xA;Something that can be charactersed as fiction literature.&lt;/p&gt;&#xA;&lt;p&gt;If yes, then I think it is also interesting to know if any of those books is available for sale. Is there a specific name for such books? Like &amp;quot;robot books&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="41409" LastEditorUserId="41409" LastEditDate="2020-11-01T11:45:30.257" LastActivityDate="2020-11-02T12:19:30.247" Title="Are there any meaningful books entirely written by an artificial intelligence?" Tags="&lt;natural-language-processing&gt;&lt;applications&gt;&lt;natural-language-understanding&gt;&lt;natural-language-generation&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24351" PostTypeId="5" CreationDate="2020-10-31T22:12:47.320" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T22:12:47.320" LastActivityDate="2020-10-31T22:12:47.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24352" PostTypeId="4" CreationDate="2020-10-31T22:12:47.320" Score="0" Body="For questions about the MuZero algorithm proposed in the paper &quot;Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model&quot; (2019)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-10-31T22:12:47.320" LastActivityDate="2020-10-31T22:12:47.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24353" PostTypeId="1" AcceptedAnswerId="24354" CreationDate="2020-11-01T00:28:06.003" Score="0" ViewCount="118" Body="&lt;p&gt;Similarly to &lt;a href=&quot;https://ai.stackexchange.com/q/15444/2444&quot;&gt;this question&lt;/a&gt; that I asked some time ago, what is the optimal value function of the shifted (by some constant &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;) version of some reward function? More precisely, let's assume that &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt; is our original reward function and &lt;span class=&quot;math-container&quot;&gt;$q_*(s, a)$&lt;/span&gt; the corresponding optimal state-action value function. What would be the optimal state-action value function of the new reward function &lt;span class=&quot;math-container&quot;&gt;$r'(s, a) \triangleq r(s, a) + c$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$c \in \mathbb{R}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T00:36:08.543" LastActivityDate="2020-11-02T13:47:45.470" Title="What is the optimal value function of the shifted version of the reward function?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;bellman-equations&gt;&lt;reward-shaping&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24354" PostTypeId="2" ParentId="24353" CreationDate="2020-11-01T00:28:06.003" Score="1" Body="&lt;h3&gt;Theorem&lt;/h3&gt;&#xA;&lt;p&gt;The optimal state-action value function of &lt;span class=&quot;math-container&quot;&gt;$r'(s, a) \triangleq r(s, a) + c$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$c \in \mathbb{R}$&lt;/span&gt;, would be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q_*(s, a) + c + c\gamma + c \gamma^2 + c \gamma^3 + \dots &#xA;&amp;amp;=q_*(s, a) + c \left( 1 + \gamma + \gamma^2 + \gamma^3 + \dots \right) \\&#xA;&amp;amp;= q_*(s, a) + c \left( \sum_{k=0}^{\infty} \gamma^{k} \right) \\&#xA;&amp;amp;=q_*(s, a) + c\left(\frac{1}{1 - \gamma}\right) \\&#xA;&amp;amp;=q_*(s, a) + \frac{c}{1 - \gamma},&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt; is the discount factor (and &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^{\infty} \gamma^{k}$&lt;/span&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Geometric_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;geometric series&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Therefore, the optimal policy does not change if we shift the reward function by some constant.&lt;/p&gt;&#xA;&lt;h3&gt;Proof&lt;/h3&gt;&#xA;&lt;p&gt;The proof follows the same reasoning of &lt;a href=&quot;https://ai.stackexchange.com/a/15445/2444&quot;&gt;the proof in this answer&lt;/a&gt;, i.e. we show that the Bellman optimality equation also holds in this case.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q_*(s,a) + \frac{c}{1 - \gamma} &#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)\left((r + c) + \gamma \max_{a' \in\mathcal{A}(s')} \left( q_*(s',a') + \frac{c}{1 - \gamma}  \right) \right) \tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$\frac{c}{1 - \gamma}$&lt;/span&gt; is a constant, it does not affect the max, given that we add this constant to all pairs &lt;span class=&quot;math-container&quot;&gt;$(s', a')$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$a' \in\mathcal{A}(s')$&lt;/span&gt;. This holds even if &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is negative. So, we can take that addition out of the max, but note that we sum (rather than multiply, like &lt;a href=&quot;https://ai.stackexchange.com/a/15445/2444&quot;&gt;the proof in the other answer&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q_*(s,a) + \frac{c}{1 - \gamma} &#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)\left((r + c) + \gamma \left (\frac{c}{1 - \gamma} +  \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right) \right) \\&#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)\left((r + c) +  \frac{c \gamma}{1 - \gamma} +  \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right) \\&#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)\left(r + \frac{c(1 - \gamma) + c \gamma}{1 - \gamma} +  \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right) \\&#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)\left(r + \frac{c - c\gamma + c \gamma}{1 - \gamma} +  \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right) \\&#xA;&amp;amp;= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} \left ( p(s',r \mid s,a)\frac{c}{1 - \gamma} \right) + \\ &amp;amp; \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}  \left( p(s',r \mid s,a) \left(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right) \right) \tag{2}\label{2} \\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$p(s',r \mid s,a)$&lt;/span&gt; is a probability distribution, then the affine combination of &lt;span class=&quot;math-container&quot;&gt;$\frac{c}{1 - \gamma}$&lt;/span&gt;, where the affine weights are &lt;span class=&quot;math-container&quot;&gt;$p(s',r \mid s,a)$&lt;/span&gt;, is &lt;span class=&quot;math-container&quot;&gt;$\frac{c}{1 - \gamma}$&lt;/span&gt;. To see that this is true, suppose that we have the probability vector &lt;span class=&quot;math-container&quot;&gt;$[0.2, 0.5, 0.3]$&lt;/span&gt; and let &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; be a constant, then &lt;span class=&quot;math-container&quot;&gt;$0.2c + 0.5c + 0.3c = c$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, equation \ref{2} becomes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;q_*(s,a) + \frac{c}{1 - \gamma} &#xA;&amp;amp;= \frac{c}{1 - \gamma}  + \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a) \left(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right)\\&#xA;q_*(s,a)&#xA;&amp;amp;=\sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a) \left(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a') \right)&#xA;\end{align}&lt;/span&gt;&#xA;which is the Bellman optimality equation (see &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=86&quot; rel=&quot;nofollow noreferrer&quot;&gt;equation 3.20 of Sutton &amp;amp; Barto book&lt;/a&gt;, p. 64).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-02T13:47:45.470" LastActivityDate="2020-11-02T13:47:45.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24355" PostTypeId="2" ParentId="3502" CreationDate="2020-11-01T01:20:24.500" Score="2" Body="&lt;p&gt;Reinforcement learning (and, in particular, bandit) algorithms have been and can be used to solve problems other than games, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://rob.schapire.net/papers/www10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recommender systems&lt;/a&gt; (actually used in practice by e.g. &lt;a href=&quot;https://netflixtechblog.com/artwork-personalization-c589f074ad76&quot; rel=&quot;nofollow noreferrer&quot;&gt;Netflix&lt;/a&gt; or &lt;a href=&quot;https://azure.microsoft.com/en-us/services/cognitive-services/personalizer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsoft&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1909.09571.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Portfolio optimization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.pnas.org/content/pnas/106/52/22387.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clinical trials&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1611.01578.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyper-parameter optimization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Self-driving cars (although I am not aware of any real self-driving car that uses just reinforcement learning; however, in principle, RL can be used in this context too)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In general, any problem that can be modelled as the maximization of some notion of &lt;em&gt;reward&lt;/em&gt;, where you need to interact with some environment (with some states) by taking some actions, can, &lt;em&gt;in principle&lt;/em&gt;, be solved by reinforcement learning. Take a look at &lt;a href=&quot;https://arxiv.org/pdf/1908.06973.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this pre-print paper&lt;/a&gt; (2019) for other applications.&lt;/p&gt;&#xA;&lt;p&gt;However, note that there are several obstacles that prevent RL algorithms from being widely adopted to solve real-world problems, starting from poor sample complexity (i.e. they require many samples to reach a good performance) or the partial inability to evaluate their performance online without affecting the users.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T14:50:45.430" LastActivityDate="2020-11-01T14:50:45.430" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24358" PostTypeId="1" AcceptedAnswerId="24359" CreationDate="2020-11-01T04:52:06.063" Score="0" ViewCount="42" Body="&lt;p&gt;(&lt;em&gt;I have a very primitive understanding of neural networks, so please forgive the lack of technicality here.&lt;/em&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I am used to seeing a &lt;em&gt;neuron&lt;/em&gt; in a neural network as something that-&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Takes the inputs and multiplies them by their weights,&lt;/li&gt;&#xA;&lt;li&gt;then sums them up,&lt;/li&gt;&#xA;&lt;li&gt;and after that it applies the activation function to the sum.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now, what if it was &amp;quot;smarter&amp;quot;? Say, a &lt;em&gt;single&lt;/em&gt; neuron could do the function of an &lt;em&gt;entire layer&lt;/em&gt; in a network, could that make the network more effective? This comes from an article I was reading at &lt;a href=&quot;https://www.quantamagazine.org/neural-dendrites-reveal-their-computational-power-20200114/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quanta&lt;/a&gt;, where the author says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Later, Mel and several colleagues looked more closely at how the cell might be managing multiple inputs within its individual dendrites. What they found surprised them: The dendrites generated local spikes, had their own nonlinear input-output curves and had their own activation thresholds, distinct from those of the neuron as a whole. The dendrites themselves could act as AND gates, or as a host of other computing devices.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;...realised that this meant that they could conceive of a single neuron as a two-layer network. The dendrites would serve as nonlinear computing subunits, collecting inputs and spitting out intermediate outputs. Those signals would then get combined in the cell body, which would determine how the neuron as a whole would respond.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My thoughts: I know that &lt;a href=&quot;https://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Backpropagation&lt;/a&gt; is used to &amp;quot;teach&amp;quot; the network in the normal case, and the fact that neurons are simply activation buttons is somehow related to that. So, if neurons were to be more complicated, it would reduce efficiency. However, I am not sure of this: &lt;em&gt;why&lt;/em&gt; would complex individual components make the network less effective?&lt;/p&gt;&#xA;" OwnerUserId="41971" LastEditorUserId="2444" LastEditDate="2020-11-01T10:47:37.423" LastActivityDate="2020-11-01T10:47:37.423" Title="If neurons performed the operation of an entire layer, would that make the neural network more effective?" Tags="&lt;neural-networks&gt;&lt;artificial-neuron&gt;&lt;hidden-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24359" PostTypeId="2" ParentId="24358" CreationDate="2020-11-01T10:16:06.030" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Say, a single neuron could do the function of an entire layer in a network, could that make the network more effective?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That depends what you mean by &amp;quot;more effective&amp;quot;. In terms of number of neurons to achieve the same result, then you should need fewer units. In terms of being able to calculate an end result for any specific problem, then no, because you can generally solve a problem using simpler units by adding more of them.&lt;/p&gt;&#xA;&lt;p&gt;If this could somehow be done using less resources, it might reduce overall costs. In a biological system, there are possibly overheads per cell in order to maintain it on top of the costs for calculation, so it may be better to do more than the simplest calculation in each cell. Further to that, there may be an optimal amount of processing that each cell could do (this is all conjecture on my part).&lt;/p&gt;&#xA;&lt;p&gt;In an artifical neural network, the calculations are the only thing being considered, there is no separate overhead per neuron.&lt;/p&gt;&#xA;&lt;p&gt;There are neural network architectures with complex &amp;quot;sub-units&amp;quot;. Probably the most well known are the &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrent neural network designs for LSTM&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Gated_recurrent_unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;gated recurrent units (GRU)&lt;/a&gt;, plus &amp;quot;skip connections&amp;quot; in &lt;a href=&quot;https://en.wikipedia.org/wiki/Residual_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;residual neural networks&lt;/a&gt;. For efficiency these are normally processed in groups per layer with matrix processing functions, but you can also view them as per-neuron complexities.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I am not sure of this- &lt;em&gt;why&lt;/em&gt; would complex individual components make the network less effective?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If the complexity was not used, or not really needed, in some of the units, then it would be wasted capacity. In a biological system, this might correspond to maintaining cells larger than they needed to be. In an artificial system, it would mean using memory and CPU to calculate interim values that were not needed for the task at hand.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-01T10:16:06.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24360" PostTypeId="1" CreationDate="2020-11-01T10:33:42.287" Score="1" ViewCount="233" Body="&lt;p&gt;Target network in DQN is known to make the network more stable, and the loss is like &amp;quot;how good I'm now compared to using the target&amp;quot;. What I don't understand is, if the target network is the stable one, why do we keep using/saving the first model as the predictor instead of the target?&lt;/p&gt;&#xA;&lt;p&gt;I see in the code everywhere:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Model&lt;/li&gt;&#xA;&lt;li&gt;Target model&lt;/li&gt;&#xA;&lt;li&gt;Train model&lt;/li&gt;&#xA;&lt;li&gt;Copy to target&lt;/li&gt;&#xA;&lt;li&gt;Get loss between them&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;At the end, the model is saved and used for prediction and not the target.&lt;/p&gt;&#xA;" OwnerUserId="41979" LastEditorUserId="2444" LastEditDate="2020-11-01T10:51:31.500" LastActivityDate="2020-11-01T12:00:03.710" Title="Why not use the target network in DQN as the predictor after training" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24361" PostTypeId="1" AcceptedAnswerId="24366" CreationDate="2020-11-01T11:21:40.090" Score="1" ViewCount="65" Body="&lt;p&gt;I would like to create an AI for the 1 player version of the card game called &amp;quot;The Game&amp;quot; by Steffen Benndorf (rules here: &lt;a href=&quot;https://nsv.de/wp-content/uploads/2018/05/the-game-english.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nsv.de/wp-content/uploads/2018/05/the-game-english.pdf&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The game works with four rows of cards. Two rows are in ascending order (numbers 1–99), and two rows are in descending order (numbers 100–2). The goal is to lay as many cards as possible, all 98 if possible, in four rows of cards. The player can have at maximum 8 cards in his hand and has to play at least 2 cards before drawing again. He can only play a greater value on an ascending row and a smaller value on a descending row with one single exception that lets him play in the reverse order: whenever the value of the number card is exactly 10 higher or lower.&lt;/p&gt;&#xA;&lt;p&gt;I already implemented a very simple hard coded AI that just pick the card with the smallest difference and prioritize a +10/-10 play when possible. With some optimizations i can get the AI to score 20 points (the number of cards left) on average wich is decent (less than 10 points in an excellent score) but i'm stuck there and i would like to go further.&lt;/p&gt;&#xA;&lt;p&gt;As there is randomness because of the draw pile, i was wondering if it was possible to implement a robust and not hard coded AI to play this game. Currently my AI is playing piecemeal with a very simple heuristic. I do not see how to improve this heuristic so I am wondering if it is possible to improve the performance by having a view over several turns for example. But I don't see how to simulate the next rounds since they will depend on the cards drawn.&lt;/p&gt;&#xA;" OwnerUserId="41980" LastEditorUserId="1847" LastEditDate="2020-11-01T12:56:36.223" LastActivityDate="2020-11-01T14:00:44.117" Title="AI to play a solo card game" Tags="&lt;game-ai&gt;&lt;ai-basics&gt;&lt;monte-carlo-tree-search&gt;&lt;monte-carlo-methods&gt;&lt;minimax&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24362" PostTypeId="2" ParentId="6579" CreationDate="2020-11-01T11:29:44.787" Score="0" Body="&lt;h3&gt;What is it?&lt;/h3&gt;&#xA;&lt;p&gt;An experience replay (ER) buffer is an array/list (or buffer) &lt;span class=&quot;math-container&quot;&gt;$D = [e_1, \dots, e_N ]$&lt;/span&gt; where you store the transitions that the agent collects while interacting with the environment. These transitions are usually represented as tuples of the form &lt;span class=&quot;math-container&quot;&gt;$e_t = (s_t, a_t, r_t, s_{t+1})$&lt;/span&gt;, where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; is the state of the agent at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; is the action taken by the agent when in state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; is the reward received by the environment after having taken action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s_{t+1}$&lt;/span&gt; is the next state the agent ended up in after that action&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The agent then samples (e.g. uniformly) transitions from this ER buffer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to perform an update of the value function &lt;span class=&quot;math-container&quot;&gt;$\hat{q}(s, a)$&lt;/span&gt;. The ER buffer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; can thus be thought of as a dataset.&lt;/p&gt;&#xA;&lt;h3&gt;Why do we need it?&lt;/h3&gt;&#xA;&lt;p&gt;The motivation for experience replay (in the &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN paper&lt;/a&gt; that popularized this technique) is that learning becomes more stable. In fact, the authors of the DQN paper write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To alleviate the problems of correlated data and non-stationary distributions, we use an experience replay mechanism [&lt;a href=&quot;http://isl.anthropomatik.kit.edu/pdf/Lin1993.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;13&lt;/a&gt;] which randomly samples previous transitions, and thereby smooths the training distribution over many past behaviors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;By using experience replay the behavior distribution is averaged&#xA;over many of its previous states, smoothing out learning and avoiding oscillations or divergence in the parameters.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T15:16:15.973" LastActivityDate="2020-11-01T15:16:15.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24363" PostTypeId="2" ParentId="24360" CreationDate="2020-11-01T11:42:36.947" Score="1" Body="&lt;p&gt;The learning model and target model are only different by N steps (typically a few thousand) out the entire taining process. If the process is near complete, they will also be quite similar.&lt;/p&gt;&#xA;&lt;p&gt;The target model is not inherently more stable in terms of producing &amp;quot;correct&amp;quot; or &amp;quot;better&amp;quot; Q values. Instead it is kept static for a period of time in order to stabilise the temporal difference (TD) target &lt;span class=&quot;math-container&quot;&gt;$R_{t+1} + \gamma \text{max}_{a'} \hat{q}(S_{t+1}, a', \theta^{-})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Due to the copying stage that you listed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you return the target model, this is identical to returning the learning model from N steps beforehand.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Which in turn means that there was no point in doing the last N steps of training. You may as well have returned the learning model at the same point as copying.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Or another way of thinking about it: Returning the learning model at the end of the training process is identical to copying it to the target model as normal, then returning the target model immediately.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-01T11:42:36.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24364" PostTypeId="2" ParentId="24360" CreationDate="2020-11-01T11:48:23.023" Score="1" Body="&lt;p&gt;Target network is not more stable. Both networks are the same in the regard that no one is more stable than the other. The reason for using a target network is that your current network after each step is updated. So, by not using a target network and using just the current network, after each update the rewards for many states will be modified slightly. So, for a particular state, after each update, the reward will be modified which will lead to an unstable reward. This happens because when you update Q value function for state S, you may also slightly change the reward it predicts for state S' in the next step. So, each update to Q value function changes slightly the rewards for many other states, which makes the predicted rewards in all those states to be slightly unstable, as they are changed very often and you dont have a clear reward in those states.&lt;/p&gt;&#xA;&lt;p&gt;However, by using a target network, your reward will be more stable(as you dont update your target network at each step, but you only update the current network).&lt;/p&gt;&#xA;&lt;p&gt;So, by using a target network, the update is more stable, not the individual networks. And because the target network is an older version of the current network, it is sensible to use the current network as it was trained more than the target network&lt;/p&gt;&#xA;" OwnerUserId="37919" LastEditorUserId="37919" LastEditDate="2020-11-01T12:00:03.710" LastActivityDate="2020-11-01T12:00:03.710" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24365" PostTypeId="2" ParentId="24341" CreationDate="2020-11-01T12:47:53.917" Score="0" Body="&lt;p&gt;I have put my initial idea to a test and used a small pretrained CNN (MobileNet) to compute features for reference images and stored the feature vectors in a &amp;quot;database&amp;quot;. Query images go through the exact same network and the resulting feature vector is used for nearest neighbor retrieval in the DB.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from glob import glob&#xA;&#xA;import torch&#xA;from PIL import Image&#xA;from numpy.linalg import norm&#xA;from torchvision import transforms&#xA;from torchvision.models import mobilenet_v2&#xA;&#xA;model = mobilenet_v2(pretrained=True)&#xA;model.eval()&#xA;&#xA;preprocess = transforms.Compose([&#xA;    transforms.Resize(256),&#xA;    transforms.CenterCrop(224),&#xA;    transforms.ToTensor(),&#xA;    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),&#xA;])&#xA;&#xA;# Generate DB&#xA;db = {}&#xA;dp_paths = glob('db/*.jpg')&#xA;for path in dp_paths:&#xA;    image = preprocess(Image.open(path)).unsqueeze(0)&#xA;    with torch.no_grad():&#xA;        output = model(image)&#xA;    db[output] = path&#xA;&#xA;# Query&#xA;image = preprocess(Image.open('queries/box.jpg')).unsqueeze(0)&#xA;with torch.no_grad():&#xA;    query = model(image)&#xA;&#xA;# Nearest Neighbor (poor man's version)&#xA;min_distance = float('inf')&#xA;candidate = None&#xA;for k, v in db.items():&#xA;    distance = norm(k.numpy() - query.numpy())&#xA;    if distance &amp;lt; min_distance:&#xA;        min_distance = distance&#xA;        candidate = v&#xA;&#xA;print(candidate, min_distance)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At least with my 5 test reference images and several query images it worked without a single failed &amp;quot;classification&amp;quot;. However, I am not sure if it will stand up to a larger test...&lt;/p&gt;&#xA;" OwnerUserId="41963" LastEditorUserId="41963" LastEditDate="2020-11-01T13:00:54.370" LastActivityDate="2020-11-01T13:00:54.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24366" PostTypeId="2" ParentId="24361" CreationDate="2020-11-01T13:37:38.883" Score="1" Body="&lt;p&gt;There are a few different ways to improve on your simple heuristic approach, but they mostly resolve to these three things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Find a better heuristic&lt;/strong&gt;. This could be done by calculating probabilities of results, or running loads of training simulations and somehow tuning the heuristic function.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Look-ahead search/planning&lt;/strong&gt;. There are many possible search algorithms. Most rely on you being able to simulate the impact of future decisions before taking them.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Take account of more player knowledge&lt;/strong&gt;. So far your simple heuristic does not take account of which cards have already been played (thus which values remain to be drawn).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Currently my AI is playing piecemeal with a very simple heuristic. I do not see how to improve this heuristic so I am wondering if it is possible to improve the performance by having a view over several turns for example. But I don't see how to simulate the next rounds since they will depend on the cards drawn.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think the main conceptual barrier you have to improvements is how to account for the complex behaviour of probabilities for drawing specific useful cards. There are a few ways to do this, but I think the simplest would be some kind of rollout (simulated look ahead), which might lead to more sophisticated algorithm such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search (MCTS)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here's how a really simple variant might work:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For each possible choice of play in the game that you are currently looking at:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Simulate the remaining deck (shuffle a copy of the known remaining cards)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Play a simulation (a &amp;quot;rollout&amp;quot;) to the end of game against the simulated deck using a simple heuristic (your current greedy choice version should be good as long as it is fast enough, but even random choices can work). Take note of the final score.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat 1.1 and 1.2 as many times as you can afford to (given allowed decision time). Average the result and save it as a score for the choice of play being considered.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of choosing the next play by your heuristic, choose the one that scores best out of all the simulations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This statistical mean of samples works in a lot of cases because it avoids the complexity and time-consuming calculations that would be required to make a perfect decision analytically from probability theory. The important things it does in your case is look-ahead planning plus taking account of additional knowledge that the player has about the state of the game.&lt;/p&gt;&#xA;&lt;p&gt;MCTS is like the above but nested so that the simulations are made from multiple starting points.&lt;/p&gt;&#xA;&lt;p&gt;In terms of robustness, provided you run enough rollouts per decision to be confident about the mean scores, then it should be OK.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-01T14:00:44.117" LastActivityDate="2020-11-01T14:00:44.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24367" PostTypeId="1" CreationDate="2020-11-01T13:54:57.283" Score="0" ViewCount="10" Body="&lt;p&gt;I wonder if there is some framework or method to help visualising inner workings of a feedforward deep neural network?&lt;/p&gt;&#xA;&lt;p&gt;What I mean by this is something similar to what is being done with CNNs where we can show what geometrical shape some neuron is responsible for finding in the picture, but for numerical tabular data. For example method that would show that for example given neuron is activated when three input features have certain values.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2020-11-01T13:54:57.283" Title="Is there a framework or method that would help visualise inner workings of a feedforward neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;feedforward-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24368" PostTypeId="1" CreationDate="2020-11-01T13:58:03.080" Score="2" ViewCount="42" Body="&lt;p&gt;I'm working on reimplementing the &lt;a href=&quot;https://arxiv.org/abs/1911.08265&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuZero paper&lt;/a&gt;. In the description of the MCTS (page 12), they indicate that a new node with associated state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is to be initialized with &lt;span class=&quot;math-container&quot;&gt;$Q(s,a) = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$N(s,a) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(s,a) = p_a$&lt;/span&gt;. From this, I understand that the root node with state &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; will have edges with zero visits each, zero value and policy evaluated on &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt; by the prediction network.&lt;/p&gt;&#xA;&lt;p&gt;So far so good. Then they explain how actions are selected, according to the equation (also on page 12):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Ltgz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Ltgz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But for the very first action (from the root node) this will give a vector of zeros as argument to the argmax: &lt;span class=&quot;math-container&quot;&gt;$Q(s_0,a) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sum_bN(s_0,b)=0$&lt;/span&gt;, so even though &lt;span class=&quot;math-container&quot;&gt;$P(s_0,a)$&lt;/span&gt; is not zero, it will be multiplied by a zero weight.&lt;/p&gt;&#xA;&lt;p&gt;Surely there is a mistake somewhere? Or is it that the very first action is uniformly random?&lt;/p&gt;&#xA;" OwnerUserId="10202" LastActivityDate="2020-11-28T14:22:29.663" Title="How to choose the first action in a Monte Carlo Tree Search?" Tags="&lt;monte-carlo-tree-search&gt;&lt;muzero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24369" PostTypeId="1" CreationDate="2020-11-01T15:42:36.117" Score="1" ViewCount="28" Body="&lt;p&gt;I am using the default implementations of REINFORCE, DQN and c51 available from the tf.agents repo (links). As you can see, DQN manages to improve performance while REINFORCE seems to suffer from catastrophic forgetting. OTOH, c51 is not able to learn much and performs like a random policy throughout.&lt;/p&gt;&#xA;&lt;p&gt;The environment looks like this -&#xA;action = [66, 1], states = [20, 1], max possible state value = 20, steps per episode = 20.&#xA;Hidden Layer dimension = (128, 128). learning rate = 0.001 (constant throughout).&#xA;Epsilon (exploration factor) = 0.2 with decay of 0.05 every 4000 episodes. Discount factor = 0.9 and relay memory size = 10,000&lt;/p&gt;&#xA;&lt;p&gt;Every episode runs for 20 steps and the rewards are collected for every step.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GcHpd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GcHpd.png&quot; alt=&quot;actual episode value this x-axis value multiplied by 50&quot; /&gt;&lt;/a&gt;&#xA;&lt;em&gt;actual episode value is the plot is the x-axis value multiplied by 50&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;What could be the possible reasons for such a performance of c51 and DQN? And based on the state space, are my hyperparameters correct or some of them need more tuning? I will increase the replay memory size but other than that to check for catastrophic forgetting, I am not sure how to diagnose other issues.&lt;/p&gt;&#xA;" OwnerUserId="41984" LastActivityDate="2020-11-01T15:42:36.117" Title="REINFORCE Agent suddenly drops. How to verify if it's due to catastrophic forgetting?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;dqn&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24370" PostTypeId="5" CreationDate="2020-11-01T16:50:11.237" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T16:50:11.237" LastActivityDate="2020-11-01T16:50:11.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24371" PostTypeId="4" CreationDate="2020-11-01T16:50:11.237" Score="0" Body="For questions about the FrozenLake environment, which is a very simple environment often used in the context of reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T16:50:11.237" LastActivityDate="2020-11-01T16:50:11.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24372" PostTypeId="5" CreationDate="2020-11-01T19:57:56.750" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T19:57:56.750" LastActivityDate="2020-11-01T19:57:56.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24373" PostTypeId="4" CreationDate="2020-11-01T19:57:56.750" Score="0" Body="For questions about the double DQN model introduced in the paper &quot;Deep Reinforcement Learning with Double Q-learning&quot; (2015) by Hado van Hasselt et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T19:57:56.750" LastActivityDate="2020-11-01T19:57:56.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24374" PostTypeId="1" CreationDate="2020-11-01T20:29:10.283" Score="0" ViewCount="23" Body="&lt;p&gt;Say I want to build a detection model that detects the existence of &lt;strong&gt;X&lt;/strong&gt; or &lt;strong&gt;NO X&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The only piece of information I have, though, is a high res. RGB image, say 100k (width) x ~1000 pixels (height).&lt;/p&gt;&#xA;&lt;p&gt;Let's also assume I cannot browse the internet to grab more data. I am stuck with this High resolution image. Can I somehow &amp;quot;slice&amp;quot; this image into multiple images and use said images as input data for my CNN?&lt;/p&gt;&#xA;&lt;p&gt;How would I do so?&lt;/p&gt;&#xA;" OwnerUserId="41990" LastEditorUserId="41990" LastEditDate="2020-11-02T06:53:06.420" LastActivityDate="2020-11-02T06:53:06.420" Title="Generating data from a High-Res. RGB image for a CNN" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24375" PostTypeId="1" CreationDate="2020-11-01T23:09:39.607" Score="6" ViewCount="201" Body="&lt;p&gt;If we shift the rewards by any constant (which is a type of reward shaping), the optimal state-action value function (and so optimal policy) does not change. The proof of this fact can be found &lt;a href=&quot;https://ai.stackexchange.com/q/24353/2444&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If that's the case, then why does a negative reward for every step encourage the agent to quickly reach the goal (which is a specific type of behavior/policy), given that such a reward function has the same optimal policy as the shifted reward function where all rewards are positive (or non-negative)?&lt;/p&gt;&#xA;&lt;p&gt;More precisely, let &lt;span class=&quot;math-container&quot;&gt;$s^*$&lt;/span&gt; be the goal state, then consider the following reward function&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;r_1(s, a)=&#xA;\begin{cases}&#xA;-1, &amp;amp; \text{ if }  s \neq s^*\\&#xA;0, &amp;amp; \text{ otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This reward function &lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt; is supposed to encourage the agent to reach &lt;span class=&quot;math-container&quot;&gt;$s^*$&lt;/span&gt; as quickly as possible, so as to avoid being penalized.&lt;/p&gt;&#xA;&lt;p&gt;Let us now define a second reward function as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;r_2(s, a) &#xA;&amp;amp;\triangleq r_1(s, a) + 1\\&#xA;&amp;amp;=&#xA;\begin{cases}&#xA;0, &amp;amp; \text{ if }  s \neq s^*\\&#xA;1, &amp;amp; \text{ otherwise}&#xA;\end{cases}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This reward function has the same optimal policy as &lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt;, but does not incentivize the agent to reach &lt;span class=&quot;math-container&quot;&gt;$s^*$&lt;/span&gt; as quickly as possible, given that the agent does not get penalized for every step. So, in theory, &lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r_2$&lt;/span&gt; lead to the same behavior. If that's the case, then why do people say that &lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt; encourage the agents to reach &lt;span class=&quot;math-container&quot;&gt;$s^*$&lt;/span&gt; as quickly as possible? Is there a proof that shows that &lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt; encourages a different type of behaviour than &lt;span class=&quot;math-container&quot;&gt;$r_2$&lt;/span&gt; (and how is that even possible given what I have just said)?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-01T23:21:28.157" LastActivityDate="2020-12-05T20:01:25.320" Title="Why does a negative reward for every step really encourage the agent to reach the goal as quickly as possible?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;reward-shaping&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24377" PostTypeId="1" CreationDate="2020-11-02T08:48:17.180" Score="0" ViewCount="23" Body="&lt;p&gt;For my BSc thesis I am trying to classify asset price direction (up/down/neutral) using numerical features and Swedish text. The text is short financial news (ca 50 words each) that have sentiment values (in most basic form negative/ positive/neutral). I have in total around 3500 daily observations.&lt;/p&gt;&#xA;&lt;p&gt;Before diving into the problem initially I thought it would be sufficient to just label the news data and use it as feature in a ensemble machine learning model such as random forest. In parallel I would need to train a text classifier (using bag of words and tfidf) such as SVM in order to classify out-of-sample inputs. However, I discovered that labeling introduces a non-negligible amount of class noise, as some text is more ambiguous and cannot easily be classified (e.g. does slightly positive sentiment fall into the neutral or positive category?). For reference, I don't have access to a sentiment analysis tool for financial news in Swedish that could handle the task for me automatically.&lt;/p&gt;&#xA;&lt;p&gt;I read about using deep learning models for automatic feature extraction from text and classification, but then comparing this setup performance to regular machine learning models such as RF (which is the goal of my thesis) becomes very problematic since the feature inputs are not the same.&lt;/p&gt;&#xA;&lt;p&gt;Using the features extracted automatically by tfidf does not seem feasible, as this yields a size 2-3000 vector and I have a small dataset.&lt;/p&gt;&#xA;&lt;p&gt;Therefore I need a solution for handling the text efficiently and uniformly in order for it to be used as feature for training my DL and ML models. So my question is what is the best way to approach this problem? Sentiment analysis with manual labeling as before? NLP approach? Other?&lt;/p&gt;&#xA;&lt;p&gt;Your ideas and suggestions are much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="41537" LastActivityDate="2020-11-02T10:32:13.840" Title="What is the best way to do classification using both text and numerical data?" Tags="&lt;text-classification&gt;&lt;feature-selection&gt;&lt;feature-extraction&gt;&lt;multi-label-classification&gt;&lt;feature-engineering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24379" PostTypeId="2" ParentId="24349" CreationDate="2020-11-02T10:48:57.997" Score="3" Body="&lt;p&gt;James Ryan has done a lot of 'archaeological' work on this; you can find references to his work on &lt;a href=&quot;https://www.jamesryan.world/projects#/storygen-history/&quot; rel=&quot;nofollow noreferrer&quot;&gt;his website&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Story generation has been a dream for a long time (in computing terms), and various genres have been explored, with not that much success. There have been episodes of a Western written by a computer (and actually filmed and acted out by human actors, &lt;a href=&quot;https://lizadaly.com/pages/saga-iii/&quot; rel=&quot;nofollow noreferrer&quot;&gt;see summary here&lt;/a&gt;), and various books, but the technology is nowhere near good enough to produce something worthwhile reading or watching without heavy editing.&lt;/p&gt;&#xA;&lt;p&gt;So far it's only good for curiosity value.&lt;/p&gt;&#xA;&lt;p&gt;There is &lt;a href=&quot;https://nanogenmo.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NaNoGenMo&lt;/a&gt;, where since 2013 people work on programs generating novels. But most of them — again — are more interesting for curiosity. They either take an existing work and modify it procedurally, or generate random templated text (eg travel reports through a ficticious, auto-generated landscape). I don't think anyone has met the target of creating a 50k novel by computer yet.&lt;/p&gt;&#xA;&lt;p&gt;Modern approaches with deep learning generators can produce reams of more or less well-formed text, but that doesn't make a compelling story. It's the meaning that is currently the problem, and as AI researchers keep finding out, it's hard.&lt;/p&gt;&#xA;" OwnerUserId="2193" LastEditorUserId="2193" LastEditDate="2020-11-02T12:19:30.247" LastActivityDate="2020-11-02T12:19:30.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24382" PostTypeId="2" ParentId="24296" CreationDate="2020-11-02T15:00:25.173" Score="1" Body="&lt;p&gt;The problem was due to the following issues in my implementation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The offspring generated in the crossover was not mutated (!)&lt;/li&gt;&#xA;&lt;li&gt;The mutations did not occur with the expected frequencies (too few links and weight mutations)&lt;/li&gt;&#xA;&lt;li&gt;The sigmoid activation had to be steepened&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Another thing that previously caused issues was the network.activate function. Make sure that you wait for the network to stabilize when doing classification tasks, so all signals have time to propagate through the network.&lt;/p&gt;&#xA;" OwnerUserId="41343" LastActivityDate="2020-11-02T15:00:25.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24383" PostTypeId="2" ParentId="11000" CreationDate="2020-11-02T15:12:40.833" Score="1" Body="&lt;p&gt;Specially on the problems related with PDEs you can find a relatively new article that is using a new approach to solve complex problems and improving the performance of classical approach.&lt;/p&gt;&#xA;&lt;p&gt;Examples arise in molecular dynamics, micro-mechanics,&#xA;and turbulent flows. You can find the paper called: Fourier Neural Operator for Parametric PDEs from a recent research &lt;a href=&quot;https://arxiv.org/pdf/2010.08895v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt; , &lt;a href=&quot;https://arxiv.org/abs/2010.08895v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A more detailed view on this new concept &lt;a href=&quot;https://www.technologyreview.com/2020/10/30/1011435/ai-fourier-neural-network-cracks-navier-stokes-and-partial-differential-equations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and find more on the paper's references.&lt;/p&gt;&#xA;" OwnerUserId="41965" LastEditorUserId="41965" LastEditDate="2020-11-02T15:19:20.483" LastActivityDate="2020-11-02T15:19:20.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24384" PostTypeId="2" ParentId="24332" CreationDate="2020-11-02T15:25:07.997" Score="0" Body="&lt;p&gt;I've found the answer in the original &lt;a href=&quot;https://github.com/google-research/bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT git repo&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;***** New May 31st, 2019: Whole Word Masking Models *****&lt;/p&gt;&#xA;&lt;p&gt;This is a release of several new models which were the result of an&#xA;improvement the pre-processing code.&lt;/p&gt;&#xA;&lt;p&gt;In the original pre-processing code, we randomly select WordPiece&#xA;tokens to mask. For example:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;Input Text: &#xA;the man jumped up , put his basket on phil ##am ##mon ' s&#xA;head Original Masked Input: [MASK] man [MASK] up , put his [MASK] on&#xA;phil [MASK] ##mon ' s head&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The new technique is called Whole Word Masking. In this case, we&#xA;always mask all of the the tokens corresponding to a word at once. The&#xA;overall masking rate remains the same.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;Whole Word Masked Input: &#xA;the man [MASK] up , put his basket on [MASK] [MASK] [MASK] ' s head&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The training is identical -- we still predict each masked WordPiece&#xA;token independently. The improvement comes from the fact that the&#xA;original prediction task was too 'easy' for words that had been split&#xA;into multiple WordPieces.&lt;/p&gt;&#xA;&lt;p&gt;This can be enabled during data generation by passing the flag&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;--do_whole_word_mask=True  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;to create_pretraining_data.py.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="25676" LastActivityDate="2020-11-02T15:25:07.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24385" PostTypeId="1" CreationDate="2020-11-02T17:33:38.733" Score="0" ViewCount="21" Body="&lt;p&gt;I am training some self play reinforcement learning agents to play 2 player board games like Connect 4, &lt;a href=&quot;https://en.wikipedia.org/wiki/Reversi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Othello&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_of_the_Amazons&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Game of the Amazons&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For each game, there is a single neural network with 2 outputs:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Value: the predicted the result of the game (-1 for loss, 0 for draw, and 1 for win)&lt;/li&gt;&#xA;&lt;li&gt;Policy: a probability distribution over all the possible moves based on what it thinks it should play&#xA;The training data is generated using self-play, with &lt;a href=&quot;https://en.wikipedia.org/wiki/Game_of_the_Amazons&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo Tree Search&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The loss for the value is the mean squared error, and the loss for the policy is the categorical cross entropy. These 2 losses are then combined with some relative weighting to determine the overall loss for the network. I want to choose the relative weighting for each of the 3 games such that both of the two losses to contribute roughly equally to the overall loss.&lt;/p&gt;&#xA;&lt;p&gt;The value loss tends to stay somewhere in the 0.2 to 0.6 range for each of the games. However, I am getting wildly different values for the policy loss. The typical policy loss ranges from around 1 for Connect 4, 0.2 for Othello, and 6 * 10^-5 for Amazons. I suspect that this is due to the drastically different number of possible moves (roughly 7 for Connect 4, 64 for Othello, and 57600 for Amazons).&lt;/p&gt;&#xA;&lt;p&gt;How can I systematically determine the correct relative weighting for each of the games?&lt;/p&gt;&#xA;" OwnerUserId="42009" LastActivityDate="2020-11-02T17:33:38.733" Title="Relative Weighting of Loss Weights for Self-Play Reinforcement Learning" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24387" PostTypeId="1" CreationDate="2020-11-02T20:53:13.753" Score="3" ViewCount="61" Body="&lt;p&gt;I've read that decision trees are able to solve XOR operation so I conclude that XGBoost algorithm can solve it as well.&lt;/p&gt;&#xA;&lt;p&gt;But my tests on the datasets (datasets that should be highly &amp;quot;xor-ish&amp;quot;) do not produce good results, so I wanted to ask whether XGBoost is able to solve this type of problem at all, or maybe I should use a different algorithm like ANN?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I found a similar question with a negative answer &lt;a href=&quot;https://discuss.xgboost.ai/t/is-that-possible-to-fit-a-xor-problem-with-xgboost/1794&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please confirm that XGBoost cannot perform XOR operation due to &amp;quot;greedy approach&amp;quot; and whether this can maybe be changed in parameters?&lt;/p&gt;&#xA;" OwnerUserId="22659" LastEditorUserId="32410" LastEditDate="2021-04-26T16:31:50.503" LastActivityDate="2021-04-26T16:31:50.503" Title="Can XGBoost solve XOR problem?" Tags="&lt;machine-learning&gt;&lt;problem-solving&gt;&lt;gradient-boosting&gt;&lt;xor-problem&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24390" PostTypeId="1" CreationDate="2020-11-03T02:18:11.160" Score="1" ViewCount="33" Body="&lt;p&gt;As I read online following areas in mathematics comes into play in ML research&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear Algebra&lt;/li&gt;&#xA;&lt;li&gt;Calculus&lt;/li&gt;&#xA;&lt;li&gt;Differential Equations&lt;/li&gt;&#xA;&lt;li&gt;Probability&lt;/li&gt;&#xA;&lt;li&gt;Statistics&lt;/li&gt;&#xA;&lt;li&gt;Discrete Mathematics&lt;/li&gt;&#xA;&lt;li&gt;Optimization&lt;/li&gt;&#xA;&lt;li&gt;Analytic Geometry&lt;/li&gt;&#xA;&lt;li&gt;Topology&lt;/li&gt;&#xA;&lt;li&gt;Numerical and Real Analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Can / Is any other areas of math used in ML research? If so what other areas? ex: Number theory&lt;/p&gt;&#xA;" OwnerUserId="42015" LastActivityDate="2020-11-03T02:18:11.160" Title="Can any area of math come into play in Machine Learning Research?" Tags="&lt;machine-learning&gt;&lt;math&gt;&lt;research&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24391" PostTypeId="1" AcceptedAnswerId="24401" CreationDate="2020-11-03T04:45:54.260" Score="1" ViewCount="153" Body="&lt;p&gt;When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the &lt;em&gt;training&lt;/em&gt; data into batches/sets of different samples from different classes. Should we also shuffle the &lt;em&gt;test&lt;/em&gt; dataset?&lt;/p&gt;&#xA;" OwnerUserId="32621" LastEditorUserId="2444" LastEditDate="2020-11-03T11:08:49.583" LastActivityDate="2020-11-03T13:57:53.987" Title="Should we also shuffle the test dataset when training with SGD?" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;datasets&gt;&lt;stochastic-gradient-descent&gt;&lt;testing&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24395" PostTypeId="1" CreationDate="2020-11-03T06:05:01.813" Score="0" ViewCount="41" Body="&lt;p&gt;I am trying to build an RL agent that interacts with an environment, a 2D grid of dimensions 20*10: each (i,j) square in the grid gives out some reward to the agent when it visits that square. Each square emits a fixed reward over episodes. Rewards don't change drastically in a given column - they decrease linearly with 1 unit decrement. Rewards from starting points of the columns also don't vary much.&lt;/p&gt;&#xA;&lt;p&gt;I have implemented Greedy in the limit of the Infinite Exploration Monte Carlo(GLIE Monte Carlo) to evaluate a state-action pair. Policy improvement is done with being 'epsilon-greedy over action-value function.' The agent is working well since the reward collection increases as the agent interacts with the environment for a higher number of episodes(I have tried up to a million episodes). I am using a discount factor of 0.9, and each episode terminates when the agent has taken 60 decisions. &lt;strong&gt;Does the latter assumption violate the definition of GLIE because the agent won't be able to explore some state-action pair infinitely many times?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am confused about which Bellman equation(Expectation Equation or Optimality Equation) I should use to evaluate states/squares in the grid when the episode number is low, say 100,000 or 500,000. &lt;strong&gt;Should I apply the Bellman Optimality Equation only if the reward collection reaches a plateau? How do I know if the policy has converged to an optimal one?&lt;/strong&gt; I would like to know if there is any simpler heuristic out there on guessing policy convergence.&lt;/p&gt;&#xA;&lt;p&gt;Thanks! Kindly let me know if I should ask some of the questions asked here(bold sentences) in a separate post.&lt;/p&gt;&#xA;" OwnerUserId="42020" LastEditorUserId="42020" LastEditDate="2020-11-03T11:20:48.707" LastActivityDate="2020-11-03T11:20:48.707" Title="Is using Bellman Optimality Equation to evaluate states a bad idea when episode number is low?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24397" PostTypeId="1" CreationDate="2020-11-03T09:01:06.350" Score="1" ViewCount="42" Body="&lt;p&gt;Consider the &lt;a href=&quot;https://doc.lagout.org/science/Artificial%20Intelligence/Machine%20learning/Machine%20Learning_%20An%20Algorithmic%20Perspective%20%282nd%20ed.%29%20%5BMarsland%202014-10-08%5D.pdf#page=129&quot; rel=&quot;nofollow noreferrer&quot;&gt;equation 4.57&lt;/a&gt; (p. 108) from section 4.6 of the Book &lt;a href=&quot;https://doc.lagout.org/science/Artificial%20Intelligence/Machine%20learning/Machine%20Learning_%20An%20Algorithmic%20Perspective%20%282nd%20ed.%29%20%5BMarsland%202014-10-08%5D.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning: An Algorithmic Perspective&lt;/a&gt;, where the derivative of the softmax function is explained&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\delta_o(\kappa) = (y_\kappa - t_\kappa)y_\kappa(\delta_{\kappa K} - y_K),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is derived from equation 4.55 (p. 107)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{\kappa}(1 - y_{\kappa}),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is to compute the diagonal of the Jacobian, and equation 4.56  (p. 107)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-y_{\kappa}y_K$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the book, it is not explained how they go from 4.55 and 4.56 to 4.57, it is just given, but I cannot follow how it is derived.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, in equation 4.57, the Kronecker's delta function is used, but how would one handle cases &lt;span class=&quot;math-container&quot;&gt;$i=j$&lt;/span&gt;, then we must have some for loop? Does having an &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; imply we need a nested for loop?&lt;/p&gt;&#xA;&lt;p&gt;Also, I have tried to just compute the derivative of softmax according to the &lt;span class=&quot;math-container&quot;&gt;$i=j$&lt;/span&gt; case only, and my model was faster (since we're not computing the jacobian) and accurate, but this assumes the error function is logarithmic, which I would like to code the general case.&lt;/p&gt;&#xA;" OwnerUserId="41915" LastEditorUserId="41915" LastEditDate="2020-11-04T07:16:08.917" LastActivityDate="2020-11-04T07:16:08.917" Title="How am I supposed to code equation 4.57 from the book &quot;Machine Learning: An Algorithmic Perspective&quot;?" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;backpropagation&gt;&lt;implementation&gt;&lt;softmax&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24400" PostTypeId="1" CreationDate="2020-11-03T11:19:16.587" Score="1" ViewCount="60" Body="&lt;p&gt;For my school project, I have to develop an agent to play my game.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GEzsq.png&quot; alt=&quot;The Board Game&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;The base I have is a 'GameManager' which call 2 AIs, each taking a random move to do.&lt;/p&gt;&#xA;&lt;p&gt;To make my AI perform, I decided to make a deep RL algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Here is how I've designed my solution.&lt;/p&gt;&#xA;&lt;p&gt;1st : the board is a 8x8 board. making &lt;strong&gt;112 possible lines to draw&lt;/strong&gt;.&#xA;2nd : on each decision, my Agent has to choose 1 line in the remaining one.&#xA;3rd : each &lt;strong&gt;decision the Agent take is one among 112&lt;/strong&gt; possible.&lt;/p&gt;&#xA;&lt;p&gt;I read some codes on the internet, the most relevant for me was a 'CartPole' example, which is a cart we have to slide to prevent a mass to fall.&lt;/p&gt;&#xA;&lt;p&gt;I made an architecture which is this one:&#xA;a game is simulated:&#xA;the board is clean, making all 112 possibilities available.&#xA;Our Agent is interroged by the gameManager to make a move passing him the actual state of the game&#xA;(&lt;strong&gt;the state shape is a 112*1 vector of Boolean values&lt;/strong&gt;, 1 means a line can be drawn, 0 means there is already a line on this position)&#xA;(&lt;strong&gt;the action shape is a vector of 112*1 Boolean values&lt;/strong&gt;, All values are set to 'False' except the line we want to draw)&#xA;So, our Agent return his move decision.&lt;/p&gt;&#xA;&lt;p&gt;Each time our agent perform a move, i store the initial state, the action we take, the reward we get performing the action, the state we reach and a boolean to know if the game is done or not.&lt;/p&gt;&#xA;&lt;p&gt;The rewards I choose are:&#xA;+1 if our action make us close a box,&#xA;-1 if our action make other close a box,&#xA;+10 if our action make us win the game,&#xA;-10 if our action make us loose the game&lt;/p&gt;&#xA;&lt;p&gt;The point is it's my 1st Deep learning project and I'm not sure about the mecanism i'm doing.&#xA;when i launch a simulation, the Neural Network is running, but the move he does seems not to be better and better.&lt;/p&gt;&#xA;&lt;p&gt;I give you the code I've wrote:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here is the gameManager code:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while True:&#xA;hasMadeABox = False&#xA;gameIsEnd = False&#xA;rewardFCB = 0&#xA;doneFCB = False&#xA;&#xA;&#xA;if GRAPHIC_MODE:&#xA;    for event in pygame.event.get():&#xA;        if event.type == pygame.QUIT:&#xA;            pygame.quit()&#xA;            sys.exit()&#xA;    disp_board()&#xA;&#xA;if playerTurns==&amp;quot;1&amp;quot;:&#xA;    stateFCB = possibilitiesToBoolList(possible_moves[0])&#xA;&#xA;boolArrayPossibleMoves = possibilitiesToBoolList(possible_moves[0])&#xA;&#xA;theAction = players[playerTurns].play(boxes, possible_moves[0], boolArrayPossibleMoves, False)&#xA;#print(possible_moves[0])&#xA;#print(theAction)&#xA;if playerTurns ==&amp;quot;1&amp;quot;:&#xA;    actionFCB = theAction&#xA;&#xA;if playerTurns==&amp;quot;1&amp;quot;:&#xA;    is_box = move(True, theAction)&#xA;&#xA;elif playerTurns==&amp;quot;2&amp;quot;:&#xA;    is_box = move(False, theAction)&#xA;&#xA;if is_box:&#xA;    if playerTurns ==&amp;quot;1&amp;quot;:&#xA;        #rewardFCB = 1&#xA;        rewardFCB = 1&#xA;        pass&#xA;    else:&#xA;        rewardFCB = -1&#xA;    hasMadeABox = True&#xA;&#xA;if check_complete():&#xA;    gameIsEnd = True&#xA;    rewardFCB += 10 if score[0]&amp;gt;score[1] else -10 #does loosing is a reward null or negativ ?&#xA;    queueOfLastGame.pop(0)&#xA;&#xA;    #Scotch pour affichage winrate&#xA;    isWin = 1 if score[0]&amp;gt;score[1] else -1&#xA;    queueOfLastGame.append(isWin)&#xA;    if queueOfLastGame.count(-1)+queueOfLastGame.count(1) &amp;gt; 0:&#xA;        print(queueOfLastGame.count(1)/(queueOfLastGame.count(-1)+queueOfLastGame.count(1)) * 100 , &amp;quot; % Winrate&amp;quot;)&#xA;&#xA;    doneFCB = True&#xA;&#xA;&#xA;if playerTurns==&amp;quot;1&amp;quot; and hasMadeABox:&#xA;    #si c'est notre IA vient de faire un carré&#xA;    #on connait directement l'état qui succede&#xA;    nextStateFCB = possibilitiesToBoolList(possible_moves[0])&#xA;&#xA;if playerTurns==&amp;quot;2&amp;quot;:&#xA;    nextStateFCB = possibilitiesToBoolList(possible_moves[0])&#xA;&#xA;&#xA;if nextStateFCB is not None:&#xA;    bufferSARS.append([stateFCB, actionFCB, rewardFCB, nextStateFCB, doneFCB])&#xA;    #ai_player_1.remember(stateFCB, actionFCB, rewardFCB, nextStateFCB, doneFCB)&#xA;    rewardFCB = 0&#xA;    nextStateFCB = None&#xA;&#xA;if gameIsEnd:&#xA;    flushBufferSARS()&#xA;    reset()&#xA;    continue&#xA;&#xA;#switch user to play if game is not end&#xA;if not hasMadeABox:&#xA;    playerTurns=&amp;quot;1&amp;quot; if playerTurns == &amp;quot;2&amp;quot; else &amp;quot;2&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;And here's my code about the Agent:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class Agent:&#xA;def __init__(self, name, possibleActions, stateSize, actionSize, isHuman=False, alpha=0.001, alphaDecay=0.01, batchSize=2048, learningRate=0.1, epsilon= 0.9, gamma = 0.996, hasToTrain=True):&#xA;&#xA;    self._memory = deque(maxlen=100000)&#xA;    self._actualEpisode=1&#xA;    self._episodes=7000&#xA;    self._name=name&#xA;    self._possibleAction=possibleActions&#xA;    self._isHuman=isHuman&#xA;    self._epsilon=epsilon&#xA;    self._epsilonDecay = 0.99&#xA;    self._epsilonMin = 0.05&#xA;    self._gamma=gamma&#xA;    self._stateSize=stateSize&#xA;    self._actionSize=actionSize&#xA;    self._alpha=alpha&#xA;    self._alphaDecay=alphaDecay&#xA;    self._hasToTrain=hasToTrain&#xA;    self._batchSize=batchSize&#xA;&#xA;    self._totalIllegalMove = 0&#xA;    self._totalLegalMove = 0&#xA;&#xA;    self._path = &amp;quot;./modelWeightSave/&amp;quot;&#xA;&#xA;    self._model = self._buildModel()&#xA;&#xA;&#xA;def save_model(self):&#xA;    self._model.save(self._path)&#xA;&#xA;def getName(self):&#xA;    return self._name&#xA;&#xA;def _buildModel(self):&#xA;    model = Sequential()&#xA;&#xA;    model.add(Dense(128, input_dim=self._stateSize, activation='relu'))&#xA;    model.add(Dense(256, kernel_initializer='normal', activation='relu'))&#xA;    model.add(Dense(256, kernel_initializer='normal', activation='relu'))&#xA;    model.add(Dense(256, kernel_initializer='normal', activation='relu'))&#xA;    model.add(Dense(128, kernel_initializer='normal', activation='relu'))&#xA;    model.add(Dense(self._actionSize, kernel_initializer='normal', activation='relu'))&#xA;&#xA;    model.compile(loss='categorical_crossentropy', optimizer=Adam(lr=self._alpha), metrics=['accuracy'])&#xA;    #if os.path.isfile(self._path):&#xA;        #model.load_weights(self._path)&#xA;    return model&#xA;&#xA;def act(self, UNUSED_state, stateAsBool):&#xA;    playableIndexes = []&#xA;    for i in range(len(stateAsBool[0])):&#xA;        if stateAsBool[0][i] == 1:&#xA;            playableIndexes.append(i)&#xA;    indexForRand = playableIndexes[random.randint(0, len(playableIndexes) - 1)]&#xA;&#xA;    if np.random.random() &amp;lt;= self._epsilon:&#xA;        action= [0]*self._actionSize&#xA;        action[indexForRand]=1&#xA;&#xA;    else:&#xA;        arrayState = np.array(stateAsBool)&#xA;&#xA;        action = self._model.predict(arrayState)&#xA;        #Set index of max esperence to 1, we play this line.&#xA;        tmp=[0]*self._actionSize&#xA;        tmp[np.argmax(action)] = 1&#xA;        action = tmp&#xA;&#xA;        isLegalMove = True&#xA;        if sum(action) != 1:&#xA;            isLegalMove = False&#xA;        for i in range(len(action)):&#xA;            if action[i] == 1:&#xA;                if stateAsBool[0][i] == 0:&#xA;                    isLegalMove = False&#xA;                    break&#xA;&#xA;        if isLegalMove:&#xA;            pass&#xA;            #print(&amp;quot;Legal move&amp;quot;)&#xA;        else:&#xA;            #print(&amp;quot;Illegal move&amp;quot;)&#xA;            #AI try to play on an already draw line, we choose a random line in remainings&#xA;            self._totalIllegalMove+=1&#xA;            action = [0] * self._actionSize&#xA;            action[indexForRand] = 1&#xA;&#xA;    #print(&amp;quot;My AI took action : &amp;quot;,action)&#xA;    return action&#xA;&#xA;def remember(self, state, action, reward, nextState, done):&#xA;    self._memory.append((state.copy(), action, reward, nextState, done))&#xA;&#xA;    self._actualEpisode+=1&#xA;    if self._actualEpisode &amp;gt; self._episodes:&#xA;        self._actualEpisode = 0&#xA;        self.replay(self._batchSize)&#xA;&#xA;def replay(self, batchSize):&#xA;    x_batch, y_batch = [], []&#xA;    minibatch = random.sample(self._memory, min(len(self._memory), self._batchSize))&#xA;    for state, action, reward, next_state, done in minibatch:&#xA;        actionIndex = np.argmax(action)&#xA;        y_target = self._model.predict(state)&#xA;        y_target[0][actionIndex] = reward if done else reward + self._gamma * np.max(self._model.predict(next_state)[0])&#xA;        x_batch.append(state[0])&#xA;        y_batch.append(y_target[0])&#xA;    self._model.fit(np.array(x_batch), np.array(y_batch),epochs=10, batch_size=len(x_batch), verbose=1)&#xA;&#xA;    if self._epsilon &amp;gt; self._epsilonMin:&#xA;        self._epsilon *= self._epsilonDecay&#xA;&#xA;    self.save_model()&#xA;&#xA;def play(self, board, state, statesAsBool, player):&#xA;    actionTaken= self.act(state, statesAsBool)&#xA;    return actionTaken&#xA;&#xA;def callBackOnPreviousMove(self, state, action, reward, nextState, done):&#xA;    self.remember(state, action, reward, nextState, done)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example of output i have during fit method:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Epoch 1/10 &#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9612 - accuracy: 0.8867&#xA; &#xA;Epoch 2/10 &#xA;&#xA;1/1 [==============================] - 0s 998us/step - loss: 109.9467 - accuracy: 0.8867 &#xA;&#xA;Epoch 3/10 &#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9456 - accuracy: 0.8867 &#xA;&#xA;Epoch 4/10 &#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9332 - accuracy: 0.8867 &#xA;&#xA;Epoch 5/10 &#xA;&#xA;1/1 [==============================] - 0s 998us/step - loss: 109.9339 - accuracy: 0.8867 &#xA;&#xA;Epoch 6/10 &#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9337 - accuracy: 0.8867 &#xA;&#xA;Epoch 7/10 &#xA;&#xA;1/1 [==============================] - 0s 997us/step - loss: 109.9305 - accuracy: 0.8867 &#xA;&#xA;Epoch 8/10 &#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9314 - accuracy: 0.8867 &#xA;&#xA;Epoch 9/10 &#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9306 - accuracy: 0.8867 &#xA;&#xA;Epoch 10/10&#xA;&#xA;1/1 [==============================] - 0s 0s/step - loss: 109.9301 - accuracy: 0.8867&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;My questions are:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is my architecture good&#xA;(inputs = [0,0,1,1,0,0,1,0.....,1,0] (112x1 shape) to represent the state, and&lt;br /&gt;&#xA;output = [0,0,0,0,0,0,0,0,0,1,0,0,0...0,0,0,0] (112x1 shape with only one '1') )&#xA;to represent an action ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to nicely choose the architecture of the Neural Network model (self._model) (I have only the basics of Neural Network, so I don't really know all activation fonction, how to design the hiden layers, choose a loss...)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To train my NN, is it good to call the 'fit' function with (state, action) as parameter to make it learn?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there something really important I forget in my design to make it work?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42025" LastEditorUserId="42025" LastEditDate="2020-11-04T18:12:59.820" LastActivityDate="2020-11-04T18:12:59.820" Title="How to design my Neural Network for Game AI" Tags="&lt;machine-learning&gt;&lt;ai-design&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24401" PostTypeId="2" ParentId="24391" CreationDate="2020-11-03T12:02:58.430" Score="2" Body="&lt;h2&gt;Short answer&lt;/h2&gt;&#xA;&lt;p&gt;Shuffling affects learning (i.e. the updates of the parameters of the model), but, during testing or validation, you are not learning. So, it should not make any difference whether you shuffle or not the test or validation data (unless you are computing some metric that depends on the order of the samples), given that you will not be computing any gradient, but just the loss or some metric/measure like the accuracy, which is not sensitive to the order or the samples you use to compute it. However, the specific samples that you use affects the computation of the loss and these quality metrics. So, how you split your original data into training, validation and test datasets affects the computation of the loss and metrics during validation and testing.&lt;/p&gt;&#xA;&lt;h2&gt;Long answer&lt;/h2&gt;&#xA;&lt;p&gt;Let me describe how gradient descent (GD) and &lt;em&gt;stochastic&lt;/em&gt; gradient descent (SGD) are used to train machine learning models and, in particular, neural networks.&lt;/p&gt;&#xA;&lt;h3&gt;Gradient descent (GD)&lt;/h3&gt;&#xA;&lt;p&gt;When training ML models with GD, you have a loss (aka cost) function &lt;span class=&quot;math-container&quot;&gt;$L(\theta; D)$&lt;/span&gt; (e.g. the cross-entropy or mean squared error) that you are trying to minimize, where &lt;span class=&quot;math-container&quot;&gt;$\theta \in \mathbb{R}^m$&lt;/span&gt; is a vector of parameters of your model and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is your labeled training dataset.&lt;/p&gt;&#xA;&lt;p&gt;To minimize this function using GD, you compute the &lt;a href=&quot;https://ai.stackexchange.com/a/20380/2444&quot;&gt;&lt;em&gt;gradient&lt;/em&gt; of your loss function &lt;span class=&quot;math-container&quot;&gt;$L(\theta; D)$&lt;/span&gt; with respect to the parameters of your model &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; given the training samples&lt;/a&gt;. Let's denote this gradient by &lt;span class=&quot;math-container&quot;&gt;$\nabla_\theta L(\theta; D) \in \mathbb{R}^m$&lt;/span&gt;. Then we perform a step of gradient descent&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\theta \leftarrow \theta - \alpha \nabla_\theta L(\theta; D) \label{1}\tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Stochastic gradient descent (SGD)&lt;/h3&gt;&#xA;&lt;p&gt;You can also minimize &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; using &lt;em&gt;stochastic&lt;/em&gt; gradient descent, i.e. you compute an approximate (or &lt;em&gt;stochastic&lt;/em&gt;) version of &lt;span class=&quot;math-container&quot;&gt;$ \nabla_\theta L(\theta; D)$&lt;/span&gt;, which we can denote as &lt;span class=&quot;math-container&quot;&gt;$\tilde{\nabla}_\theta L(\theta; B) \approx \nabla_\theta L(\theta; D)$&lt;/span&gt;, which is typically computed with a subset of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; of your training dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$B \subset D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|B| &amp;lt; |D|$&lt;/span&gt;. The step of SGD is exactly the same as the step of GD, but we use &lt;span class=&quot;math-container&quot;&gt;$\tilde{\nabla}_\theta L(\theta; B)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\theta \leftarrow \theta - \alpha \tilde{\nabla}_\theta L(\theta; B) \label{2}\tag{2}&#xA;$$&lt;/span&gt;&#xA;If we split &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; subsets (or batches) &lt;span class=&quot;math-container&quot;&gt;$B_i$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, k$&lt;/span&gt; (and these subsets usually have the same size, i.e. &lt;span class=&quot;math-container&quot;&gt;$|B_i| = |B_j|, \forall i$&lt;/span&gt;, apart from one of them, which may contain fewer samples), then the SGD step needs to be performed &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; times, in order to go through all training samples.&lt;/p&gt;&#xA;&lt;h3&gt;Sampling, shuffling, and convergence&lt;/h3&gt;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$\tilde{\nabla}_\theta L(\theta; B_i) \approx \nabla_\theta L(\theta; D), \forall i$&lt;/span&gt;, it should be clear that the way you split the samples into batches can affect learning (i.e. the updates of the parameters).&lt;/p&gt;&#xA;&lt;p&gt;For instance, you could consider your dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; as an ordered sequence/list, and just split it into &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; sub-sequences. Without shuffling this ordered sequence before splitting, you will always get the same batches, which means that, if there's some information associated with the specific ordering of this sequence, then it may bias the learning process. That's one of the reasons why you may want to shuffle the data.&lt;/p&gt;&#xA;&lt;p&gt;So, you could &lt;em&gt;uniformly&lt;/em&gt; choose samples from &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; to create your batches &lt;span class=&quot;math-container&quot;&gt;$B_i$&lt;/span&gt; (and this is a way of shuffling, in the sense that you will be uniformly building these batches at random), but you can also sample differently and you could also re-use the same samples in different batches (i.e. sampling with replacement). Of course, all these approaches can affect how learning proceeds.&lt;/p&gt;&#xA;&lt;p&gt;Typically, when analyzing the convergence properties of SGD, you require that your samples are &lt;a href=&quot;https://ai.stackexchange.com/a/10844/2444&quot;&gt;i.i.d.&lt;/a&gt; and that the learning rate &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; satisfies some conditions (&lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Robbins–Monro conditions&lt;/a&gt;). If that's not the case, then SGD may not converge to the correct answer. That's why sampling or shuffling can play an important role in SGD.&lt;/p&gt;&#xA;&lt;h3&gt;Testing and validation&lt;/h3&gt;&#xA;&lt;p&gt;During testing or validation, you are just computing the loss or some metric (like the accuracy) and not a stochastic gradient (i.e. you are not updating the parameters, by definition: you just do it during training). The way you compute the loss or accuracy should not be sensitive to the order of the samples, so shuffling should not affect the computation of the loss or accuracy. For instance, if you use the mean squared error, then you will need to compute&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;L(\theta; D_\text{test}) &#xA;&amp;amp;= \operatorname {MSE} \\&#xA;&amp;amp;= {\frac {1}{n}}\sum _{i=1}^{n}(f_\theta(x_i)-{\hat {y_{i}}})^{2}\\&#xA;&amp;amp;= {\frac {1}{n}}\sum _{i=1}^{n}(y_{i}-{\hat {y_{i}}})^{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f_\theta$&lt;/span&gt; is your ML model&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th input&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y_{i}$&lt;/span&gt; is the true label for input &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\hat {y_{i}}$&lt;/span&gt; is the output of the model&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of samples you use to compute the MSE&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is an average, so it doesn't really matter whether you shuffle or not. Of course, it matters which samples you use though!&lt;/p&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/q/245502/82135&quot;&gt;Here&lt;/a&gt; you can find some informal answers to the question &amp;quot;Why do we shuffle the training data while training a neural network?&amp;quot;. There are other papers that partially answer this and/or other related questions more formally, such as &lt;a href=&quot;http://proceedings.mlr.press/v97/haochen19a/haochen19a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; or &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0925231219300578&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-03T13:57:53.987" LastActivityDate="2020-11-03T13:57:53.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24402" PostTypeId="1" AcceptedAnswerId="24403" CreationDate="2020-11-03T14:18:22.270" Score="-1" ViewCount="44" Body="&lt;p&gt;I have read what the loss function is but I am not sure if I have understood it. For each neuron in the output layer the loss function is equal most usually to the square of the difference value of the neuron and the result we want. Is that correct? Most sites don't help me understanding so answer would be appreciated a lot.&lt;/p&gt;&#xA;" OwnerUserId="42011" LastActivityDate="2020-11-03T14:26:17.320" Title="Loss function definition" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24403" PostTypeId="2" ParentId="24402" CreationDate="2020-11-03T14:26:17.320" Score="0" Body="&lt;p&gt;A loss function is what helps you &amp;quot;train&amp;quot; your neural network to do what you want it to do. A better way to word it to begin with would be an &amp;quot;objective&amp;quot; function. This function describes what objective you'd like your neural network to fit to (or to be good at).&lt;/p&gt;&#xA;&lt;p&gt;The loss function that you've described is &amp;quot;squared error&amp;quot;, which, as the name suggests, is the squared difference between the expected output and the output from the neural network. This trains the network to match the expected output value.&lt;/p&gt;&#xA;&lt;p&gt;Other loss (or &amp;quot;objective&amp;quot;) functions could train your network to look for different things. For example, training on cross entropy loss helps your network learn certain probabilities. That's why it's usually used for classification, like when you want to determine which digit from 0-9 was fed into your MNIST classifier.&lt;/p&gt;&#xA;" OwnerUserId="5240" LastActivityDate="2020-11-03T14:26:17.320" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24406" PostTypeId="1" CreationDate="2020-11-03T15:54:28.487" Score="1" ViewCount="155" Body="&lt;p&gt;I am trying to create my own gym environment for the A3C algorithm (one implementation is &lt;a href=&quot;https://github.com/dgriff777/rl_a3c_pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). The custom environment is a simple login form for any site. I want to create an environment from an image. The idea is to take a screenshot of the web page and create an environment from this screenshot for the A3C algorithm. I know the doc and protocol for creating a custom environment. But I don't understand how to create an environment, exactly, based on a screenshot.&lt;/p&gt;&#xA;&lt;p&gt;If I do so&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.observation_space = gym.spaces.Box(low=0, high=255, shape=(128, 128, 3), dtype=np.uint8)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get a new pic.&lt;/p&gt;&#xA;&lt;p&gt;Here's the algorithm that I am trying to implement (page 39 of the master's thesis &lt;a href=&quot;https://aaltodoc.aalto.fi/bitstream/handle/123456789/37895/master_Eskonen_Juha_2019.pdf?sequence=1&amp;amp;isAllowed=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Reinforcement Learning in Automated User Interface Testing&lt;/a&gt; by Juha Eskonen).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tfrJu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tfrJu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42038" LastEditorUserId="2444" LastEditDate="2020-11-04T11:22:15.727" LastActivityDate="2020-11-23T15:24:25.013" Title="How do I create a custom gym environment based on an image?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;&lt;a3c&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24407" PostTypeId="1" CreationDate="2020-11-03T17:03:43.487" Score="1" ViewCount="64" Body="&lt;p&gt;I've been trying to find the optimal number of epochs that I should train my neural network (that I just implemented) for.&lt;/p&gt;&#xA;&lt;p&gt;The visualizations below show the neural network being run with a variable number of epochs. It is quite obvious that the accuracy increases with the number of epochs. However, at 75 epochs, we see a dip before the accuracy continues to rise. What is the cause of this?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZRIlj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZRIlj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42039" LastEditorUserId="2444" LastEditDate="2020-11-05T11:28:19.920" LastActivityDate="2020-11-05T11:28:19.920" Title="Why does the accuracy drop while the loss decrease, as the number of epochs increases?" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;&lt;loss&gt;&lt;accuracy&gt;&lt;epochs&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24408" PostTypeId="2" ParentId="24407" CreationDate="2020-11-03T17:55:48.233" Score="1" Body="&lt;p&gt;Decrease of loss does not essentially lead to increase of accuracy (most of the time it happens but sometime it may not happen). To know why, you can have a look at &lt;a href=&quot;https://stats.stackexchange.com/questions/282160/how-is-it-possible-that-validation-loss-is-increasing-while-validation-accuracy&quot;&gt;this&lt;/a&gt; question. The network cares about decreasing the loss and it does not care about the accuracy at all. So it's no surprise to see what you presented.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Additional note&lt;/em&gt;: If you use &lt;strong&gt;batch&lt;/strong&gt; approaches to teach your network, &lt;em&gt;or&lt;/em&gt; if you choose a &lt;strong&gt;big step size&lt;/strong&gt; you may also see that the loss increases sometimes but in the case of batching, the hope is that its trend is to be decreased.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-11-03T17:55:48.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24409" PostTypeId="1" CreationDate="2020-11-03T22:44:04.067" Score="0" ViewCount="24" Body="&lt;p&gt;I know that several tokenization methods that are used for tranformer models like WordPiece for Bert and BPE for Roberta and others. What I was wondering if there is also a transformer which uses a method for tokenization similarly to the embeddings that are used in the fasttext library, so based on the summations of embeddings for the n-grams the words are made of.&lt;/p&gt;&#xA;&lt;p&gt;To me it seems weird that this way of creating word(piece) embeddings that can function as the input of a transformer isn't used in these new transformer architectures. Is there a reason why this is not tried yet? Or is this question just an result of my inability to find the right papers/repo's.&lt;/p&gt;&#xA;" OwnerUserId="42045" LastActivityDate="2020-11-03T22:44:04.067" Title="Is there a pretrained (NLP) transformer that uses subword n-gram embeddings for tokenization like fasttext?" Tags="&lt;transformer&gt;&lt;word-embedding&gt;&lt;bert&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24410" PostTypeId="1" CreationDate="2020-11-04T00:01:44.493" Score="0" ViewCount="39" Body="&lt;p&gt;I'm trying to do some research about semantic segmentation for webpages, in particular e-commerce webpages. I found some articles which provide some solutions based on very old dataset and those solutions in my opinion can't be effective for modern websites, in particular e-commerce. I would like to semantically infer the images bounding box, text, price etc..&lt;/p&gt;&#xA;&lt;p&gt;Another problem is related with the size of webpage screenshot which are huge, I resized to 1024x512, but I think that I can't resize the image more otherwise I loose quality.&lt;/p&gt;&#xA;&lt;p&gt;I built a very complex neural network in order to semantically infer text, images and background, (not classification but just segmentation), and the results are not so bad, but they are far from my expectations which seems strange to me, as we have many DNN able to do semantic segmentation of road, building, car etc for example. One problem is for sure the lack of a dataset with detailed labels. I didn't find any dataset that can satisfy my requests.&lt;/p&gt;&#xA;&lt;p&gt;QUESTION: Any idea to help the network learn better the structure of a webpage just with a screenshot?&lt;/p&gt;&#xA;&lt;p&gt;My DNN essentially is built as an auto-encoder architecture based on Segnet, with some modifications, skip connections, unpooling etc, I think that it is a good network.&lt;/p&gt;&#xA;&lt;p&gt;references:&#xA;&lt;a href=&quot;https://clgiles.ist.psu.edu/pubs/CVPR2017-connets.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://clgiles.ist.psu.edu/pubs/CVPR2017-connets.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-981-13-0020-2_33&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/chapter/10.1007/978-981-13-0020-2_33&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="32694" LastEditDate="2020-11-04T12:50:12.887" LastActivityDate="2020-11-04T12:50:12.887" Title="Is Webpage Semantic Segmentation possible nowadays?" Tags="&lt;deep-learning&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24411" PostTypeId="2" ParentId="8190" CreationDate="2020-11-04T07:59:21.067" Score="1" Body="&lt;p&gt;Warren McCulloch and Walter Pitts talk about recurrent neural nets in their paper McCulloch, W.S., Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5, 115–133 (1943). &lt;a href=&quot;https://doi.org/10.1007/BF02478259&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1007/BF02478259&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;They finish their introduction with the paragraph:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The nervous system contains many circular paths, whose activity so regenerates the excitation of any participant neuron that reference to time past becomes indefinite, although it still implies that afferent activity has realized one of a certain class of configurations over time. Precise specification of these implications by means of recursive functions, and determination of those that can be embodied in the activity of nervous nets, completes the theory.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Their paper contains a section titled:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The Theory: Nets Without Circles.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;in which they introduce feed-forward (nets without cycles) and recurrent (nets with cycles) networks, and the next section, titled&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;The Theory: Nets with Circles.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;in which they prove a few theorems about recurrent neural networks.&lt;/p&gt;&#xA;&lt;p&gt;Marvin Minsky quotes them, and discusses recurrent neural networks extensively throughout his book, Computation: Finite and Infinite Machines (1967). Prentice Hall, ISBN: 0131655639,9780131655638&lt;/p&gt;&#xA;&lt;p&gt;I am not sure, are there earlier references.&lt;/p&gt;&#xA;" OwnerUserId="15524" LastActivityDate="2020-11-04T07:59:21.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24413" PostTypeId="1" AcceptedAnswerId="24415" CreationDate="2020-11-04T08:19:29.470" Score="0" ViewCount="68" Body="&lt;p&gt;The theory of evolution seems to be intelligent as it creates life&lt;/p&gt;&#xA;&lt;p&gt;The mechanism of evolutionary theory consists of mutation, recombination, and natural selection like a genetic algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Isn't this evolutionary mechanism itself the same as the essence of human intelligence?&lt;/p&gt;&#xA;" OwnerUserId="23500" LastEditorUserId="1847" LastEditDate="2020-11-04T09:16:53.920" LastActivityDate="2020-11-04T12:26:02.650" Title="Isn't evolutionary theory the essence of intelligence after all?" Tags="&lt;philosophy&gt;&lt;evolutionary-algorithms&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24415" PostTypeId="2" ParentId="24413" CreationDate="2020-11-04T10:50:52.287" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The theory of evolution seems to be intelligent as it creates life&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When you say &amp;quot;seems to be intelligent&amp;quot; that begs the question: How are you defining &amp;quot;intelligent&amp;quot;? Which of course is still one of the big issues in AI research.&lt;/p&gt;&#xA;&lt;p&gt;I think there are some flaws with the argument that &amp;quot;creates life&amp;quot; = &amp;quot;intelligent&amp;quot;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Evolution does &lt;em&gt;not&lt;/em&gt; create life. It operates on entities where there is a copy mechanism which is not 100% reliable, plus a selective environment that also impacts likelihood of further copies being made. Some form of proto-life (an &lt;a href=&quot;https://www.newscientist.com/article/2080986-ultimate-family-tree-tracing-the-oldest-ancestor-of-all-life/#:%7E:text=In%20the%20beginning%20was%20Ida,bacteria%20to%20the%20giant%20panda.&quot; rel=&quot;nofollow noreferrer&quot;&gt;initial Darwinian ancestor or Ida&lt;/a&gt;) needed to exist &lt;em&gt;before&lt;/em&gt; evolution started.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The process of creating the first proto-life capable of undergoing evolution is generally thought to be a large semi-random search through chemical combinations. Random search &lt;em&gt;is&lt;/em&gt; sometimes used in optimisation problems, and might be studied as part of AI search topics. However, it would normally be considered something of a baseline algorithm, and definitley not tick the boxes for all the general traits of intelligence.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Isn't this evolutionary mechanism itself the same as the essence of human intelligence?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_intelligence#Challenges&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on artificial intellegence lists challenges faced by researchers and developers in AI&lt;/a&gt;. The categories chosen there are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Reasoning, problem solving; Knowledge representation; Planning; Learning; Natural language processing; Perception; Motion and manipulation; Social intelligence; General intelligence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Together, these are mainly traits of mammalian, avian and a few other multicellular species, with a few traits such as language heavily focused on humans.&lt;/p&gt;&#xA;&lt;p&gt;I think it is important to separate out the mechanism whereby these traits arose naturally - which is generally agreed to be via an evolutionary process - from how those traits function. Artifical intelligence may use a little bit of reverse engineering from the natural traits in order to inspire design, but most AI systems do not use theory of evolution directly.&lt;/p&gt;&#xA;&lt;p&gt;When used directly, evolutionary algorithms can be used to solve search and optimisation problems. Also they can be used to solve simplified problems in perception and motion/manipulation. However, we are not able to scale up such algorithms to solve all aspects of general intelligence. Instead, systems like machine learning are designed to work from analysis of the problem, inspired in part by working natural systems. These work far more efficiently than evolutionary algorithms. There are no competitive evolutionary variants of AlphaZero, Watson, GPT-3 or neural-networks used in image processing.&lt;/p&gt;&#xA;&lt;p&gt;Evolutionary algorithms have their place in AI in practice and research. However, they do not define or encapsulate a form of general intelligence.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-04T12:26:02.650" LastActivityDate="2020-11-04T12:26:02.650" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24416" PostTypeId="1" CreationDate="2020-11-04T12:49:55.233" Score="1" ViewCount="30" Body="&lt;p&gt;I have to write the formalization of the loss function of my network, built following the WGAN-GP model. The discriminator takes 3 consecutive images as input (such as 3 consecutive frames of a video) and must evaluate if the intermediate image is a possible image between the first and the third.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jJ7EX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jJ7EX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I thought something like this, but is it correct to identify x1, x2 and x3 coming from Pr even if they are 3 consecutive images? Only the first is chosen randomly, the others are simply the next two.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XVihd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XVihd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I replaced Pr with p_r(x1, x3) and p_r(x1, x2, x3) to reinforce the fact that x2 and x3 are taken after x1, so they depend on the choice of x1. Is it more correct this way?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/W6jTV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W6jTV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40372" LastEditorUserId="2444" LastEditDate="2021-01-25T18:56:40.133" LastActivityDate="2021-01-25T18:56:40.133" Title="WGAN-GP Loss formalization" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;loss&gt;&lt;wasserstein-metric&gt;&lt;wasserstein-gan&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24417" PostTypeId="1" CreationDate="2020-11-04T17:16:17.530" Score="0" ViewCount="16" Body="&lt;p&gt;I have two sound datasets and each one has 80% normal and 20% anomalous data points. The first one is a rock song and the second one is a mellow indie song. I use half of the normal data as a baseline in each dataset. I identify anomalies using isolation forest in each dataset and found 25 anomalies in the first rock song dataset and 12 in the mellow threshold. Now my question is how can I classify an anomaly as a rock song specific one? Do you think building a simple linear regression classifier should work?&lt;/p&gt;&#xA;" OwnerUserId="42060" LastActivityDate="2020-11-04T17:16:17.530" Title="How to classify anomalies between two sound datasets?" Tags="&lt;classification&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24418" PostTypeId="1" CreationDate="2020-11-05T07:56:13.447" Score="2" ViewCount="237" Body="&lt;p&gt;Variational autoencoders have two components in their loss function. The first component is the reconstruction loss, which for image data, is the pixel-wise difference between the input image and output image. The second component is the Kullback–Leibler divergence which is introduced in order to make image encodings in the latent space more 'smooth'. Here is the loss function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text { loss }&#xA;&amp;amp;=&#xA;\|x-\hat{x}\|^{2}+\operatorname{KL}\left[N\left(\mu_{x}, \sigma_{x}\right), \mathrm{N}(0,1)\right] \\&#xA;&amp;amp;=&#xA;\|x-\mathrm{d}(z)\|^{2}+\operatorname{KL}\left[N\left(\mu_{x^{\prime}} \sigma_{x}\right), \mathrm{N}(0,1)\right]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am running some experiments on a dataset of famous artworks using Variational Autoencoders. My question concerns scaling the two components of the loss function in order to manipulate the training procedure to achieve better results.&lt;/p&gt;&#xA;&lt;p&gt;I present two scenarios. The first scenario does not scale the loss components.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zOs3u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zOs3u.png&quot; alt=&quot;VAE - no scaling&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here you can see the two components of the loss function. Observe that the order of magnitude of the Kullback–Leibler divergence is significantly smaller than that of the reconstruction loss. Also observe that 'my famous' paintings have become unrecognisable. The image shows the reconstructions of the input data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OkCvG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OkCvG.png&quot; alt=&quot;unscaled reconstruction&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the second scenario I have &lt;strong&gt;scaled the KL term with 0.1&lt;/strong&gt;. Now we can see that the reconstructions are looking much better.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xc9zP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xc9zP.png&quot; alt=&quot;VAE - with scaling&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/N9Fvi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N9Fvi.png&quot; alt=&quot;Scaled Reconstruction&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it mathematically sound to train the network by scaling the components of the loss function? Or am I effectively excluding the KL term in the optimisation?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to understand this in terms of gradient descent?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it fair to say that we are telling the model &amp;quot;we care more about the image reconstructions than 'smoothing' the latent space&amp;quot;?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am confident that my network design (convolutional layers, latent vector size) have the capacity to learn parameters to create proper reconstructions as a Convolutional Autoencoder with the same parameters is able to reconstruct perfectly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/5716/2444&quot;&gt;Here&lt;/a&gt; is a similar question.&lt;/p&gt;&#xA;&lt;p&gt;Image Reference:&#xA;&lt;a href=&quot;https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34180" LastEditorUserId="2444" LastEditDate="2020-11-06T01:42:19.493" LastActivityDate="2021-04-05T02:06:55.583" Title="What is the impact of scaling the KL divergence and reconstruction loss in the VAE objective function?" Tags="&lt;loss-functions&gt;&lt;gradient-descent&gt;&lt;variational-autoencoder&gt;&lt;kl-divergence&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24420" PostTypeId="2" ParentId="8168" CreationDate="2020-11-05T12:27:37.873" Score="0" Body="&lt;p&gt;People usually say that genetic algorithms are used to solve optimization problems, but when it comes to optimizing a &lt;em&gt;specific&lt;/em&gt; function given in an analytic form (i.e. when it comes to finding a maximum or minimum of such a function), it may not be clear how to proceed. I have created a complete but simple implementation and explanation of how to solve this problem &lt;a href=&quot;https://github.com/nbro/function_max_with_ga&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but let me also describe here the main idea behind the approach. Before that, let's briefly review genetic algorithms (GAs).&lt;/p&gt;&#xA;&lt;h3&gt;Genetic algorithms&lt;/h3&gt;&#xA;&lt;p&gt;Genetic algorithms are composed of&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a &lt;strong&gt;population&lt;/strong&gt; (i.e. a set) of individuals (also known as chromosomes or genotypes), which represent the solutions to some problem&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a &lt;strong&gt;fitness function&lt;/strong&gt; that evaluates each individual (i.e. how &amp;quot;good&amp;quot; it is, maybe compared to other individuals, where &amp;quot;good&amp;quot; depends on the problem)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;genetic operations&lt;/strong&gt; to stochastically change the individuals in the population: typically, these operations are the &lt;strong&gt;mutation&lt;/strong&gt; and &lt;strong&gt;cross-over&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a method to &lt;strong&gt;select individuals&lt;/strong&gt; for the cross-over (where you combine 2 or more individuals to produce other individuals); the selection is also a &lt;em&gt;genetic operation&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;How to solve your problem?&lt;/h3&gt;&#xA;&lt;p&gt;To solve &lt;em&gt;any&lt;/em&gt; problem with genetic algorithms, you first need to address all the four points above, i.e. define what your individuals (i.e. solutions) are, how to compute the fitness of a solution (i.e. how good it is), and define the specific evolutionary operations (specifically, mutation, cross-over and selection).&lt;/p&gt;&#xA;&lt;p&gt;In your specific problem, the solutions are &lt;span class=&quot;math-container&quot;&gt;$\hat{x} \in \mathbb{R}$&lt;/span&gt;, such that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x)=\frac{-x^{2}}{10}+3 x$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is a (local or global) maximum, i.e. &lt;span class=&quot;math-container&quot;&gt;$f(\hat{x}) \geq f(x)$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$x \in  \mathbb{R}$&lt;/span&gt; in a neighbourhood of &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;(Note that this is just the definition of the problem of &lt;em&gt;function maximization&lt;/em&gt;: if you are not familiar with it, you should probably get familiar with it before trying to understand this answer or even trying to solve this problem with GAs).&lt;/p&gt;&#xA;&lt;p&gt;Therefore, in this case, the &lt;strong&gt;individuals&lt;/strong&gt; are real numbers (which are the inputs to &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;fitness function&lt;/strong&gt; can be a function that computes &lt;span class=&quot;math-container&quot;&gt;$f(x_i)$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; in your population, then compares &lt;span class=&quot;math-container&quot;&gt;$f(x_i)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$f(x_j)$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;. The higher the &lt;span class=&quot;math-container&quot;&gt;$f(x_i)$&lt;/span&gt;, the closer it is to a maximum.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;genetic operations&lt;/strong&gt; can be implemented in different ways. You should think about it. If you are familiar with GAs and you know now that solutions are real numbers, at least one way of implementing these genetic operations should come to your mind at this point. Keep in mind that your solutions should be in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 32]$&lt;/span&gt;, i.e. this is a &lt;em&gt;constrained&lt;/em&gt; optimization problem. If you do not have any idea on how to implement them, take a look at &lt;a href=&quot;https://github.com/nbro/function_max_with_ga&quot; rel=&quot;nofollow noreferrer&quot;&gt;my implementation/explanation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-05T14:58:09.107" LastActivityDate="2020-11-05T14:58:09.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24421" PostTypeId="2" ParentId="5716" CreationDate="2020-11-05T13:43:10.203" Score="-1" Body="&lt;p&gt;Don't know if you have this doubt anymore, but this would be helpful for those who are facing similar problems-&lt;/p&gt;&#xA;&lt;p&gt;You will need to find the correct weights with which you add these two loses by hyperparameter search. That is, find the best &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; for the loss-&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L = Loss_1 + \lambda(Loss_2)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$Loss_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Loss_2$&lt;/span&gt; can be any losses. Here, we take them as SSIM and L1-regularization losses, respectively.&#xA;You can keep the gradient flow of the regularization loss below some percentage of that of &lt;span class=&quot;math-container&quot;&gt;$Loss_1$&lt;/span&gt; by choosing the correct value of gradient clipping for a hyperparameter. Note that by setting this hyperparameter to low, you may even impede its performance(the exact opposite of the case mentioned here). For this specific case, L1 regularization has a constant gradient, equal to the hyperparameter itself. So, by keeping it around 10% of the max gradient(or 10% of the max loss generally works as well), we should not face these kinds of problem.&lt;/p&gt;&#xA;" OwnerUserId="40843" LastActivityDate="2020-11-05T13:43:10.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24422" PostTypeId="1" CreationDate="2020-11-05T15:47:47.287" Score="1" ViewCount="24" Body="&lt;p&gt;I came across the concept of &lt;strong&gt;&amp;quot;deep learning primitives&amp;quot;&lt;/strong&gt; from the Nvidia talk &lt;a href=&quot;https://info.nvidia.com/rs/156-OFN-742/images/Jetson_AGX_Xavier_New_Era_Autonomous_Machines.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jetson AGX Xavier New Era Autonomous Machines&lt;/a&gt; (on slide 44).&lt;/p&gt;&#xA;&lt;p&gt;There doesn't seem to be a lot of articles in the community on this concept. I was able to find one definition from &lt;a href=&quot;https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/ch01.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where it defined deep learning primitives as the &amp;quot;fundamental building blocks of deep networks&amp;quot; like fully connected layers, convolutions layers, etc.&lt;/p&gt;&#xA;&lt;p&gt;I was curious to find out if a &lt;strong&gt;self-attention layer&lt;/strong&gt; is a primitive, I came across this &lt;a href=&quot;https://github.com/oneapi-src/oneDNN/issues/533&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenDNN issue&lt;/a&gt; and one person explained that self-attention layers can be built by other primitives like inner product, concat, etc.&lt;/p&gt;&#xA;&lt;p&gt;So my question is what exactly are &lt;strong&gt;primitives&lt;/strong&gt; in deep learning? What makes a convolution layer a primitive and a self-attention layer not a primitive?&lt;/p&gt;&#xA;" OwnerUserId="42082" LastActivityDate="2020-11-05T15:47:47.287" Title="What exactly are deep learning primitives?" Tags="&lt;deep-learning&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24423" PostTypeId="2" ParentId="24418" CreationDate="2020-11-05T17:28:16.917" Score="0" Body="&lt;p&gt;Ans 1.&lt;/p&gt;&#xA;&lt;p&gt;The motive of Variational Inference(on which VAE is based), is to decrease &lt;span class=&quot;math-container&quot;&gt;$KL(q(z|x)||p(z))$&lt;/span&gt;, where p(z) is our chosen distribution of the hidden variable z. After doing some math, we can write this expression as-&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;KL(q||x) = log(p(x)) - \Sigma_z q(z)log(\frac{p(x,z)}{q(z)})&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For a given x, the first term of RHS is constant. So we maximise the second term so that the KL divergence goes to zero.&lt;/p&gt;&#xA;&lt;p&gt;We can write the second term as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E_{q(z)}[log(p(x|z)] - KL(q(z|x)||p(z))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(try writing p(x,z) as &lt;span class=&quot;math-container&quot;&gt;$\frac{p(x|z)}{p(z)}$&lt;/span&gt; and then expand. Here, p(z) is the distribution of our choice, i.e. a Gaussian distribution). We argue that the process from z to x is deterministic and z is gaussian. So, the first term could be replaced by &lt;span class=&quot;math-container&quot;&gt;$exp(log(-||x-\hat{x}||^2))$&lt;/span&gt;(we replaced z by &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; because it's deterministic- this is now the exact proof). So we have-&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;maximize(-||x-\hat(x)||-KL(q(z|x)||p(z)))&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and we get our loss function.&lt;/p&gt;&#xA;&lt;p&gt;We also know that variational autoencoders &lt;strong&gt;almost never&lt;/strong&gt; find the optimal solution, so I am not sure how would playing around with the weights affect it(Nor do I know if it makes sense mathematically).&lt;/p&gt;&#xA;&lt;p&gt;Ans 2.&lt;/p&gt;&#xA;&lt;p&gt;We can say that the KL divergence has a regularising effect.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/intuitively-understanding-variational-autoencoders-1bfe67eb5daf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This page&lt;/a&gt; has some nice experiments which will help you understand what happens to the latent space when you decrease the KL divergence part.&lt;/p&gt;&#xA;&lt;p&gt;Ans 3.&lt;/p&gt;&#xA;&lt;p&gt;Yes, you can say that. You are fixing the dimensions, but are lenient on the distribution. In fact, you are approaching autoencoders by doing this.&lt;/p&gt;&#xA;&lt;p&gt;Separate-&lt;/p&gt;&#xA;&lt;p&gt;I want to point you towards &lt;a href=&quot;https://ermongroup.github.io/cs228-notes/inference/variational/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; article. It explains why we choose to minimise &lt;span class=&quot;math-container&quot;&gt;$KL(q(z|x)||p(z))$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$KL(p(z)||q(z|x))$&lt;/span&gt; (the latter is intractable) and what would happen if we choose less independent variables for our estimator q(z).&lt;/p&gt;&#xA;&lt;p&gt;Also, have you tried increasing the dimensions of the latent space? It can also have a 'de-regularizing' effect. It seems that the model is underfitting the data- the reconstruction loss is high with the normal loss, compared to when you decrease the regularizing term.&lt;/p&gt;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="40843" LastEditorUserId="40843" LastEditDate="2020-11-05T17:55:11.660" LastActivityDate="2020-11-05T17:55:11.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24425" PostTypeId="2" ParentId="24181" CreationDate="2020-11-05T17:56:05.317" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Some sources say MCTS (or planning in general) increases the sample efficiency.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If we're thinking purely about experiments run in simulations, then I'd estimate there may be cases where a combination of pure learning + MCTS (or some other form of planning / model-based aspect) may be more efficient, and there may be different cases where only a single one of those techniques on its own may perform better. So then I wouldn't say this is always necessarily true.&lt;/p&gt;&#xA;&lt;p&gt;Often though, when saying &amp;quot;sample efficiency&amp;quot;, we would only count the steps actually taken in a &amp;quot;real&amp;quot; environment, but maybe not count steps taken in a lookahead search or planning algorithm. In pure simulations this may seem like a weird distinction to make, but it's more sensible when you consider that we often use simulations just because they're convenient ways to evaluate and compare new algorithms, but often not the end goal. Often, the end goal would be to apply something in the &amp;quot;real world&amp;quot;, for example on a robot or something. In such a situation, collecting pure learning experiences can be very expensive (time-consuming, maybe also risky because the robot may fall over and break, etc.). But you may be able to also provide that robot with a learned model, or simulator, and have the robot use that to also run its own search or planning algorithms on an approximated version of the real world. This is a clear case where performing such is much faster and cheaper and less risky than collecting true experiences in the real world for pure RL.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Assumed the transition model is known and the computational cost of interacting through planning is the same as interacting with the environment, I do not see the difference between playing many games versus playing a single game, but plan at each step.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Jumping back to the case where we actually are working purely in simulation, and where there's no difference in computational cost between steps taken for pure learning vs. steps taken in search/planning, there absolutely are still some differences. If you run search, you use extra time and have temporary extra memory usage (which frees up again after completing your search algorithm) to make one really good decision in the &amp;quot;main&amp;quot; environment. You could view the steps taken in MCTS simulations as a form of learning as well, but they have a different purpose from the steps taken in a pure RL setting; these steps are taken with the sole goal of learning how to act well in the root state. All search time and memory usage is dedicated to that single decision. This can enable smarter decision-making than if you're going purely off of what you learned through pure RL, due to 1) focusing more effort on just a single decision, and 2) not being constrained by the capacity of your learning algorithm to actually learn a good decision (simple function approximators may simply not be capable of accurately representing strong policies, and more complex function approximators will take a huge amount of time to learn). This smarter decision-making thanks to search can in turn also improve the quality of the experience used by your pure RL component.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Finally, since you started out by mentioning AlphaGo Zero, I'd like to emphasise that AlphaGo Zero and similar approaches are typically used in multi-agent adversarial domains (like zero-sum games). Pure RL approaches for such multi-agent settings do exist, but there has been significantly less research towards them than pure RL approaches for single-agent settings. Most of these pure RL approaches are really only applicable to single-agent settings, and can easily have poor performance when applied to these kinds of multi-agent settings. Search algorithms like MCTS on the other hand are very well-established techniques in these adversarial domains like games, and the combination of them with learning approaches appears to allow even for learning approaches to be used which are not explicitly &amp;quot;aware&amp;quot; of the fact that they're operating in such a multi-agent domain.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-11-05T17:56:05.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24428" PostTypeId="2" ParentId="24375" CreationDate="2020-11-05T18:51:51.147" Score="1" Body="&lt;h1&gt;Your examples are equivalent. But it is possible to find a constant yielding  a different optimal policy.&lt;/h1&gt;&#xA;&lt;p&gt;Your examples are absolutely equivalent. The agent maximizes the reward, and only way to do so is by reaching &lt;span class=&quot;math-container&quot;&gt;$s^*$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Consider &lt;span class=&quot;math-container&quot;&gt;$r_3$&lt;/span&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;r_3(s, a)=&#xA;\begin{cases}&#xA;1, &amp;amp; \text{ if }  s \neq s^*\\&#xA;2, &amp;amp; \text{ otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;With a sufficiently large &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, moving infinitely without reaching &lt;span class=&quot;math-container&quot;&gt;$s^*$&lt;/span&gt; is now the optimal solution.&lt;/p&gt;&#xA;&lt;p&gt;For the generic case&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;r_4(s, a)=&#xA;\begin{cases}&#xA;\alpha, &amp;amp; \text{ if }  s \neq s^*\\&#xA;\beta, &amp;amp; \text{ otherwise}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;the threshold is found by comparing the results of the series &lt;span class=&quot;math-container&quot;&gt;$\alpha + \alpha^2 + \alpha^3 + ... + \alpha^{t_m}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t_m$&lt;/span&gt; is the maximum episode length, and &lt;span class=&quot;math-container&quot;&gt;$\alpha + \alpha^2 + \alpha^3 + ... + \alpha^{t^*}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt; is the length of the episode following the fastest policy.&lt;/p&gt;&#xA;&lt;p&gt;In the example of &lt;span class=&quot;math-container&quot;&gt;$r_3$&lt;/span&gt;, it is trivial to find examples where the fastest policy isn't optimal. Imagine a race, the agent starts on the left and gets either &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; points, depending on where it is. With &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.9$&lt;/span&gt; and no time-limit (infinite episodes) the optimal policy is  to move randomly, but in the second-to-last house, avoid the goal state. With &lt;span class=&quot;math-container&quot;&gt;$\gamma = 0.1$&lt;/span&gt;, the optimal policy is to move randomly (not really, probably there would be a slight advantage in moving right), but in the second-to-last house, enter the goal.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JBRyt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JBRyt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7496" LastEditorUserId="7496" LastEditDate="2020-11-05T19:37:35.970" LastActivityDate="2020-11-05T19:37:35.970" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24429" PostTypeId="2" ParentId="22627" CreationDate="2020-11-05T20:45:28.783" Score="3" Body="&lt;p&gt;I believe &lt;a href=&quot;http://web.archive.org/web/20201126231906/https://www.cs.mcgill.ca/%7Ewlh/grl_book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Graph Representation Learning&lt;/em&gt;&lt;/a&gt; book by William L. Hamilton is a great resource to start&lt;/p&gt;&#xA;" OwnerUserId="11303" LastEditorUserId="2444" LastEditDate="2021-01-15T11:24:24.827" LastActivityDate="2021-01-15T11:24:24.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24430" PostTypeId="2" ParentId="15397" CreationDate="2020-11-05T22:25:19.580" Score="-1" Body="&lt;p&gt;Yes, it's called hypothesis testing but normally you need a little bit more than pure MLE.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-11-05T22:25:19.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24431" PostTypeId="5" CreationDate="2020-11-05T23:25:02.557" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-05T23:25:02.557" LastActivityDate="2020-11-05T23:25:02.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24432" PostTypeId="4" CreationDate="2020-11-05T23:25:02.557" Score="0" Body="For questions about the &quot;Generative Adversarial Imitation Learning&quot; (GAIL) method for imitation learning introduced in a 2016 paper with the same name and published in NIPS." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-05T23:25:02.557" LastActivityDate="2020-11-05T23:25:02.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24433" PostTypeId="2" ParentId="22270" CreationDate="2020-11-05T23:40:15.013" Score="1" Body="&lt;p&gt;The authors of the paper &lt;a href=&quot;https://arxiv.org/pdf/1710.11248.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Robust Rewards with Adversarial Inverse Reinforcement Learning&lt;/a&gt; (2018, published in ICRL), which introduced the inverse RL technique &lt;strong&gt;AIRL&lt;/strong&gt;, argue that &lt;strong&gt;GAIL&lt;/strong&gt; fails to generalize to different environment's dynamics. Specifically, in section 7.2 (p. 7), they describe an experiment where they disable and shrink the two front legs of the &lt;a href=&quot;https://gym.openai.com/envs/Ant-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ant&lt;/a&gt;, then, based on the results, they conclude&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GAIL learns successfully in the training domain, but does not acquire a representation that is suitable for transfer to test domains.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;On the other hand, according to their experiments, AIRL is more robust to changes in the environment's dynamics.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-05T23:45:50.413" LastActivityDate="2020-11-05T23:45:50.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24434" PostTypeId="1" AcceptedAnswerId="24435" CreationDate="2020-11-06T03:02:40.363" Score="0" ViewCount="36" Body="&lt;p&gt;I am training a convolutional neural network to detect objects (weeds amongst crops, in my case) using TensorFlow. The original dimensions of the raw training photos are 4000 x 3000 pixels, which must be resized to become workable. The idea here is to label objects in the training images (using Label-Img), train the model, and use it to detect weeds in certain situations.&lt;/p&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow 2 Detection Model Zoo&lt;/a&gt;, there are algorithms designed for different speeds, which involves initially resizing the images to a specified dimension. Although this is not a coding question, here is an example of SSD ResNet-50, which initially resizes the input images to 1024 x 1024 pixels:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model {&#xA;  ssd {&#xA;    num_classes: 1&#xA;    image_resizer {&#xA;      fixed_shape_resizer {&#xA;        height: 1024&#xA;        width: 1024&#xA;      }&#xA;    }&#xA;    feature_extractor {&#xA;      type: &amp;quot;ssd_resnet50_v1_fpn_keras&amp;quot;&#xA;      depth_multiplier: 1.0&#xA;      min_depth: 16&#xA;      conv_hyperparams {&#xA;        regularizer {&#xA;          l2_regularizer {&#xA;            weight: 0.00039999998989515007&#xA;          }&#xA;        }&#xA;        initializer {&#xA;          truncated_normal_initializer {&#xA;            mean: 0.0&#xA;            stddev: 0.029999999329447746&#xA;          }&#xA;        }&#xA;        activation: RELU_6&#xA;        batch_norm {&#xA;          decay: 0.996999979019165&#xA;          scale: true&#xA;          epsilon: 0.0010000000474974513&#xA;        }&#xA;      }&#xA;      override_base_feature_extractor_hyperparams: true&#xA;      fpn {&#xA;        min_level: 3&#xA;        max_level: 7&#xA;      }&#xA;    }&#xA;    box_coder {&#xA;      faster_rcnn_box_coder {&#xA;        y_scale: 10.0&#xA;        x_scale: 10.0&#xA;        height_scale: 5.0&#xA;        width_scale: 5.0&#xA;      }&#xA;    }&#xA;    matcher {&#xA;      argmax_matcher {&#xA;        matched_threshold: 0.5&#xA;        unmatched_threshold: 0.5&#xA;        ignore_thresholds: false&#xA;        negatives_lower_than_unmatched: true&#xA;        force_match_for_each_row: true&#xA;        use_matmul_gather: true&#xA;      }&#xA;    }&#xA;    similarity_calculator {&#xA;      iou_similarity {&#xA;      }&#xA;    }&#xA;    box_predictor {&#xA;      weight_shared_convolutional_box_predictor {&#xA;        conv_hyperparams {&#xA;          regularizer {&#xA;            l2_regularizer {&#xA;              weight: 0.00039999998989515007&#xA;            }&#xA;          }&#xA;          initializer {&#xA;            random_normal_initializer {&#xA;              mean: 0.0&#xA;              stddev: 0.009999999776482582&#xA;            }&#xA;          }&#xA;          activation: RELU_6&#xA;          batch_norm {&#xA;            decay: 0.996999979019165&#xA;            scale: true&#xA;            epsilon: 0.0010000000474974513&#xA;          }&#xA;        }&#xA;        depth: 256&#xA;        num_layers_before_predictor: 4&#xA;        kernel_size: 3&#xA;        class_prediction_bias_init: -4.599999904632568&#xA;      }&#xA;    }&#xA;    anchor_generator {&#xA;      multiscale_anchor_generator {&#xA;        min_level: 3&#xA;        max_level: 7&#xA;        anchor_scale: 4.0&#xA;        aspect_ratios: 1.0&#xA;        aspect_ratios: 2.0&#xA;        aspect_ratios: 0.5&#xA;        scales_per_octave: 2&#xA;      }&#xA;    }&#xA;    post_processing {&#xA;      batch_non_max_suppression {&#xA;        score_threshold: 9.99999993922529e-09&#xA;        iou_threshold: 0.6000000238418579&#xA;        max_detections_per_class: 100&#xA;        max_total_detections: 100&#xA;        use_static_shapes: false&#xA;      }&#xA;      score_converter: SIGMOID&#xA;    }&#xA;    normalize_loss_by_num_matches: true&#xA;    loss {&#xA;      localization_loss {&#xA;        weighted_smooth_l1 {&#xA;        }&#xA;      }&#xA;      classification_loss {&#xA;        weighted_sigmoid_focal {&#xA;          gamma: 2.0&#xA;          alpha: 0.25&#xA;        }&#xA;      }&#xA;      classification_weight: 1.0&#xA;      localization_weight: 1.0&#xA;    }&#xA;    encode_background_as_zeros: true&#xA;    normalize_loc_loss_by_codesize: true&#xA;    inplace_batchnorm_update: true&#xA;    freeze_batchnorm: false&#xA;  }&#xA;}&#xA;train_config {&#xA;  batch_size: 64&#xA;  data_augmentation_options {&#xA;    random_horizontal_flip {&#xA;    }&#xA;  }&#xA;  data_augmentation_options {&#xA;    random_crop_image {&#xA;      min_object_covered: 0.0&#xA;      min_aspect_ratio: 0.75&#xA;      max_aspect_ratio: 3.0&#xA;      min_area: 0.75&#xA;      max_area: 1.0&#xA;      overlap_thresh: 0.0&#xA;    }&#xA;  }&#xA;  sync_replicas: true&#xA;  optimizer {&#xA;    momentum_optimizer {&#xA;      learning_rate {&#xA;        cosine_decay_learning_rate {&#xA;          learning_rate_base: 0.03999999910593033&#xA;          total_steps: 100000&#xA;          warmup_learning_rate: 0.013333000242710114&#xA;          warmup_steps: 2000&#xA;        }&#xA;      }&#xA;      momentum_optimizer_value: 0.8999999761581421&#xA;    }&#xA;    use_moving_average: false&#xA;  }&#xA;  fine_tune_checkpoint: &amp;quot;PATH_TO_BE_CONFIGURED&amp;quot;&#xA;  num_steps: 100000&#xA;  startup_delay_steps: 0.0&#xA;  replicas_to_aggregate: 8&#xA;  max_number_of_boxes: 100&#xA;  unpad_groundtruth_tensors: false&#xA;  fine_tune_checkpoint_type: &amp;quot;classification&amp;quot;&#xA;  use_bfloat16: true&#xA;  fine_tune_checkpoint_version: V2&#xA;}&#xA;train_input_reader {&#xA;  label_map_path: &amp;quot;PATH_TO_BE_CONFIGURED&amp;quot;&#xA;  tf_record_input_reader {&#xA;    input_path: &amp;quot;PATH_TO_BE_CONFIGURED&amp;quot;&#xA;  }&#xA;}&#xA;eval_config {&#xA;  metrics_set: &amp;quot;coco_detection_metrics&amp;quot;&#xA;  use_moving_averages: false&#xA;}&#xA;eval_input_reader {&#xA;  label_map_path: &amp;quot;PATH_TO_BE_CONFIGURED&amp;quot;&#xA;  shuffle: false&#xA;  num_epochs: 1&#xA;  tf_record_input_reader {&#xA;    input_path: &amp;quot;PATH_TO_BE_CONFIGURED&amp;quot;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Because I will be labeling many pictures in the future, I need to decide on a dimension to resize my original ones to (literature review says 1100 x 1100 has been used in previous projects).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;If I were to change the image resizer in the code above to 1100 x 1100, for example, would that have any effect on model accuracy/training loss?&lt;/strong&gt; Would it even run? I'm fairly new to this, so any insights on this would be greatly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Note: I am using a NVIDIA GPU, so that helps speed the process quite a bit. Google Colab also can be used.&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-11-06T05:41:03.077" Title="Will changing the dimension reduction size of a neural network (i.e. SSD ResNet-50) change the overall outcome and accuracy of the model?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;object-detection&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24435" PostTypeId="2" ParentId="24434" CreationDate="2020-11-06T05:41:03.077" Score="1" Body="&lt;p&gt;That depends! You can try it. &lt;strong&gt;But if you change the sizes, you have to ensure that you do not mismatch the shapes.&lt;/strong&gt; As far as size is concerned it won't affect the accuracy much unless you are significantly changing it.&lt;/p&gt;&#xA;&lt;p&gt;Since the default is 1024x1024, and you are making 1100x1100, there wont be any issues.&lt;/p&gt;&#xA;&lt;p&gt;Remember, there is a tradeoff in terms of speed and the amount of information you can derive from the image. The larger the image size, the higher the computation time and the image information you have.&lt;/p&gt;&#xA;" OwnerUserId="35791" LastActivityDate="2020-11-06T05:41:03.077" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24437" PostTypeId="1" AcceptedAnswerId="24589" CreationDate="2020-11-06T06:19:43.150" Score="1" ViewCount="98" Body="&lt;p&gt;I am working on a problem that involves two tasks - detection and classification. There is no single dataset for both tasks. I am training two models, separate on detection dataset and another on classification dataset. I use the images from the detection dataset as input and get classification predictions on top of detected bounding boxes.&lt;/p&gt;&#xA;&lt;p&gt;Dataset description :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Classification - Image of the single object (E.g. Car) in the center with a classification label.&lt;/li&gt;&#xA;&lt;li&gt;Detection - Image with multiple objects (E.g. 4 Cars) with bounding box annotations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Task - Detect objects(e.g. cars) from detection datasets and classify them into various categories.&lt;/p&gt;&#xA;&lt;p&gt;How do I verify whether the classification model trained on the classification dataset is working on images from detection dataset? (In terms of classification accuracy)&lt;/p&gt;&#xA;&lt;p&gt;I cannot manually label the images from the detection dataset for individual class labels. (Need expert domain knowledge)&lt;/p&gt;&#xA;&lt;p&gt;How do I verify my classification model?&lt;/p&gt;&#xA;&lt;p&gt;Is there any technique to do this ? Like domain transfer or any weakly-supervised method ?&lt;/p&gt;&#xA;" OwnerUserId="35791" LastActivityDate="2020-11-16T22:19:54.667" Title="How to verify classification model trained on classification dataset on a detection dataset for classification purpose?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;object-detection&gt;&lt;image-processing&gt;" AnswerCount="2" CommentCount="5" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24438" PostTypeId="1" CreationDate="2020-11-06T07:25:59.893" Score="1" ViewCount="28" Body="&lt;p&gt;I have seen that a &lt;em&gt;center loss&lt;/em&gt; is beneficial in computer vision, especially in face recognition. I have tried to understand this concept from the following material&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ydwen.github.io/papers/WenECCV16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Discriminative Feature Learning Approach for Deep Face Recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.slideshare.net/JisungDavidKim/center-loss-for-face-recognition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.slideshare.net/JisungDavidKim/center-loss-for-face-recognition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, I could not understand the concept clearly. If someone can explain with the help of an example, that would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="41756" LastEditorUserId="2444" LastEditDate="2020-11-06T11:58:20.007" LastActivityDate="2020-11-06T11:58:20.007" Title="What is a &quot;center loss&quot;?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;loss-functions&gt;&lt;face-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24440" PostTypeId="1" AcceptedAnswerId="26001" CreationDate="2020-11-06T11:10:53.640" Score="4" ViewCount="109" Body="&lt;p&gt;A few years ago when I was in university, I had implemented (for my final year project) an &lt;em&gt;Itinerary Planning System&lt;/em&gt;, which incorporates an AI technique called &amp;quot;case-based reasoning&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Is case-based reasoning a machine learning technique or &lt;a href=&quot;https://www.manning.com/books/grokking-artificial-intelligence-algorithms?utm_source=affiliate&amp;amp;utm_medium=affiliate&amp;amp;a_aid=pw&amp;amp;a_bid=6a1b836a&quot; rel=&quot;nofollow noreferrer&quot;&gt;an AI technique&lt;/a&gt; (that is not machine learning)?&lt;/p&gt;&#xA;" OwnerUserId="41843" LastEditorUserId="2444" LastEditDate="2021-01-26T22:43:18.187" LastActivityDate="2021-02-01T01:53:19.143" Title="Is case-based reasoning a machine learning technique?" Tags="&lt;machine-learning&gt;&lt;definitions&gt;&lt;ai-field&gt;&lt;case-based-reasoning&gt;&lt;instance-based-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24441" PostTypeId="1" CreationDate="2020-11-06T11:38:06.467" Score="0" ViewCount="39" Body="&lt;p&gt;This question is connected to &lt;a href=&quot;https://ai.stackexchange.com/questions/24342/how-does-muzero-learn-to-play-well-for-both-sides-of-a-two-player-game&quot;&gt;a question&lt;/a&gt; that I asked some time ago.&lt;/p&gt;&#xA;&lt;p&gt;This is how I understood the training procedure takes place (please correct any conceptual mistakes here):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Many complete games are generated entirely with Monte-Carlo Tree Search (MCTS) and a fixed policy/value/state network. Then we iterate many times the following steps:&lt;/li&gt;&#xA;&lt;li&gt;A random game is sampled and a random position at step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the game is picked&lt;/li&gt;&#xA;&lt;li&gt;The network plays &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; steps, at each step producing a policy &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}_{t+k}$&lt;/span&gt; and a value &lt;span class=&quot;math-container&quot;&gt;$v_{t+k}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;At the same time the MCTS produces a policy &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\pi}_{t+k}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The way MuZero computes the loss function is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_t = \sum_{k=1}^K \underbrace{(u - v_{t+k})^2}_\mathrm{value\ loss} + \underbrace{\boldsymbol{\pi}_{t+k} \log \mathbf{p}_{t+k}}_\mathrm{policy\ loss} + \underbrace{||\theta||^2}_\mathrm{L2\ regularization}&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is the observed final reward of the episode, which is known because the episode in memory goes until the end.&lt;/p&gt;&#xA;&lt;p&gt;I have two major doubts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;why should one use &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, if the moves that we end up picking over those &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; steps can deviate from the moves in the original game that we sampled?&lt;/li&gt;&#xA;&lt;li&gt;if one player wins and one player loses, shouldn't we add the cross entropy of the policy for the winner and &lt;em&gt;subtract&lt;/em&gt; the cross entropy of the policy of the loser? Is the answer that in any case the MCTS policy &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\pi}_{t+k}$&lt;/span&gt; is &amp;quot;stronger&amp;quot; than the bare network policy &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}_{t+k}$&lt;/span&gt; and therefore we can still learn from it regardless if that player won or lost?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10202" LastActivityDate="2020-11-06T11:38:06.467" Title="Understanding the MuZero loss function for a two-player game" Tags="&lt;loss-functions&gt;&lt;cross-entropy&gt;&lt;muzero&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24443" PostTypeId="1" AcceptedAnswerId="24444" CreationDate="2020-11-06T11:58:46.600" Score="1" ViewCount="51" Body="&lt;p&gt;I am learning PyTorch on Udacity. In lesson 8, section 11: &lt;a href=&quot;https://classroom.udacity.com/courses/ud188/lessons/2f4910ee-6d67-47df-97e2-f35db67cbc19/concepts/062bfbc6-34c8-4e5c-b072-6479eca5a385&quot; rel=&quot;nofollow noreferrer&quot;&gt;Training the Model&lt;/a&gt;, the instructor writes:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Then I have my embedding and hidden dimension. The embedding dimension is just a smaller representation of my vocabulary of 70k words and I think any value between like 200 and 500 or so would work, here. I've chosen 400. Similarly, for our hidden dimension, I think 256 hidden features should be enough to distinguish between positive and negative reviews.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are more than 70000 different words. How could those more than 70000 unique words be represented by just 400 embeddings? How does an embedding look like? Is it a number?&lt;/p&gt;&#xA;&lt;p&gt;Moreover, why would 256 hidden features be enough?&lt;/p&gt;&#xA;" OwnerUserId="27988" LastEditorUserId="2444" LastEditDate="2020-11-07T13:40:33.757" LastActivityDate="2020-11-07T13:40:33.757" Title="Why is an embedding of dimension 400 enough to represent 70000 words?" Tags="&lt;recurrent-neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;word-embedding&gt;&lt;hyper-parameters&gt;&lt;sentiment-analysis&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24444" PostTypeId="2" ParentId="24443" CreationDate="2020-11-06T12:50:09.397" Score="1" Body="&lt;p&gt;The specific term you are looking for is &amp;quot;word embedding&amp;quot; and not just &amp;quot;embedding&amp;quot;.&lt;/p&gt;&#xA;&lt;h3&gt;How to numerically represent textual data?&lt;/h3&gt;&#xA;&lt;p&gt;Neural networks (typically) require as inputs (and produce as outputs) numerical data (i.e. numbers, vectors, matrices, or higher-dimensional arrays). So, when processing textual data, we first need to encode (or convert) the text into a numerical representation. There are different ways to do it, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;one-hot encoding&lt;/strong&gt; (in that case, if you have 70000 words, you would have &lt;em&gt;sparse&lt;/em&gt; vectors with 70000 entries where only one of those entries is equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and all other entries are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;: see &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/word_embeddings&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article &lt;/a&gt; for more info)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;map each word to a number&lt;/strong&gt; (in this case, you would have 70000 numbers, one for each word)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;word embeddings&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Each of these representations has different benefits and drawbacks. For instance, if you map each word to a number, then you just need to keep track of &lt;span class=&quot;math-container&quot;&gt;$70000$&lt;/span&gt; numbers. In the case of one-hot encoding or word embeddings, you will need more memory. However, nowadays, word embeddings are widely used in natural language processing/understanding/generation tasks (and given that your question is about word embeddings), so let me briefly describe them.&lt;/p&gt;&#xA;&lt;h3&gt;Word embeddings&lt;/h3&gt;&#xA;&lt;p&gt;There are different word embedding techniques (such as &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/word2vec&quot; rel=&quot;nofollow noreferrer&quot;&gt;word2vec&lt;/a&gt;). However, they are all based on the same ideas&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Words that are similar (or related) in meaning should be mapped to vectors (i.e. the &amp;quot;word embeddings&amp;quot;) that are also similar in some sense (for instance, their cosine similarity should be high). For instance, the words &amp;quot;man&amp;quot; and &amp;quot;boy&amp;quot; should be mapped to vectors that are similar.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;These word embeddings are learned (rather than hard-coded or manually specified) given the data&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The size of the word embeddings is a hyper-parameter (this should answer your question!)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h3&gt;Hyper-parameters&lt;/h3&gt;&#xA;&lt;p&gt;To answer your question(s) more directly, the choice of the dimension of the embeddings or the number of &amp;quot;hidden features&amp;quot; (which are both &lt;em&gt;hyper-parameters&lt;/em&gt;) was probably more or less arbitrary or based on the instructor's experience. In general, it is difficult to determine the optimal choice of any hyper-parameter. Sometimes you can just use numbers that other people have used in the past and have noticed that work &amp;quot;well enough&amp;quot;. If you really want to find more appropriate values of the hyper-parameters, you could use some hyper-parameter optimization technique, such as &lt;em&gt;Bayesian optimization&lt;/em&gt; or a simple &lt;em&gt;grid search&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;You can find many resources online that explain the concept of &amp;quot;word embeddings&amp;quot; more in detail. For instance&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://machinelearningmastery.com/what-are-word-embeddings/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Are Word Embeddings for Text?&lt;/a&gt; (by Jason Brownlee)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tensorflow.org/tutorials/text/word_embeddings&quot; rel=&quot;nofollow noreferrer&quot;&gt;Word embeddings&lt;/a&gt; (A TensorFlow article/tutorial on word embeddings).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-06T13:01:06.127" LastActivityDate="2020-11-06T13:01:06.127" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24445" PostTypeId="2" ParentId="24311" CreationDate="2020-11-06T13:51:12.850" Score="2" Body="&lt;p&gt;There are different questions and even different lines of thought here. Let's go through them&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;On resizing&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why do we need to resize?&lt;/strong&gt; To fit the network input which is fixed when nets are no Fully Convolutional Networks (FCN)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;What if my net is FCN?&lt;/strong&gt; Still makes sense to resize to bound the dimension of the input features you want to detect (a person on a small image VS big image). Take into account that the kernel sizes do not vary although the image size does.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;On keeping aspect ratio&lt;/strong&gt; (or &lt;em&gt;letterbox&lt;/em&gt; as some people like to say)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Why to keep aspect ratio?&lt;/strong&gt; This is more of a philosophical question. It is believed that keeping the aspect ratio helps the nets to learn the natural variability in object sizes (say a person bounding box can not be super hight and super thin, because that would be a street light).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Why not to keep aspect ratio?&lt;/strong&gt; If you resize without keeping the aspect ratio and the aspect ratio distortion is not &lt;em&gt;mega super very huge&lt;/em&gt;, the networks will still learn. In other words, if your input images don't have crazy aspect ratios, then there is no difference between adding or not a bit of distortion. In fact, sometimes it will even act as a regularization or augmentation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conlusion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As long as your application is not too specific and your input images aspect ratios are bounded (this is, if  you train with images from any regular camera), you should not worry too much about this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;When to worry about this?&lt;/strong&gt;&lt;/em&gt; When you train with huge vertical or horizontal images or if you train with images taken from some very specific devices like geophisical, radio, or optical sensors. In these cases you should pay special attention on how to resize or split an image. For example, with a recording of a radio sensor, if you resize with aspect ratio deformation, a wave from a specific frequency would transform to another because of the sine wave warp)&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-11-06T13:51:12.850" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24446" PostTypeId="1" CreationDate="2020-11-06T14:05:06.020" Score="0" ViewCount="28" Body="&lt;p&gt;I was reading this &lt;a href=&quot;https://arxiv.org/abs/1911.13181&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; which applies a modified version of the transformers for traffic forecasting. I am somewhat familiar with the transformer architecture and how it functions, but, in the paper, the authors are using &lt;em&gt;scheduled sampling&lt;/em&gt;. For those who are not familiar with the concept, it basically means that sometimes you're feeding the decoder the ground truth and other times its own prediction.&lt;/p&gt;&#xA;&lt;p&gt;How can such a mechanism work? As far as I know,  &lt;strong&gt;during training&lt;/strong&gt;, the decoder should be fed the ground truth shifted to the right by one (where a symbolic &amp;quot;start&amp;quot; token is introduced). How could a scheduled sampling fit in here?&lt;/p&gt;&#xA;&lt;p&gt;The only way I could see is to modify the training procedure to resemble the inference/testing where we are predicting one future observation at a time, but this will increase the training time significantly.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="2444" LastEditDate="2020-11-09T10:55:51.610" LastActivityDate="2020-11-09T10:55:51.610" Title="How does scheduled sampling for transformers work?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;papers&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24447" PostTypeId="1" CreationDate="2020-11-06T14:27:46.453" Score="0" ViewCount="21" Body="&lt;p&gt;Here's my problem: I work with medical image classification, and currently I have 3 classes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;class A: images with lesion 1 only; and images with lesion 1 and N other lesions&lt;/li&gt;&#xA;&lt;li&gt;class B: images with 2 other lesions (no lesion 1)&lt;/li&gt;&#xA;&lt;li&gt;class C: images with no lesion&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The goal is to classify into &amp;quot;lesion 1&amp;quot;, &amp;quot;other lesion&amp;quot;, &amp;quot;no lesion&amp;quot;. I'd like to know some approach/method/paper/clue for this classification. I think the presence of other lesions on both class A and B is confusing the model (the validation accuracy and f1-score are very low).&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="42100" LastActivityDate="2020-11-06T14:27:46.453" Title="CNN to detect presence/absense of label on images with mixed labels" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;multi-label-classification&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24448" PostTypeId="1" CreationDate="2020-11-06T14:32:21.280" Score="1" ViewCount="31" Body="&lt;p&gt;I'm trying to understand if a 3D convolution of the sort performed in a convolutional layer of a CNN is &lt;em&gt;associative&lt;/em&gt;. Specifically, is the following true:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;X \otimes(W \cdot Q)=(X \otimes W) \cdot Q,&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\otimes$&lt;/span&gt; is a convolution,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a 3D input to a convolution layer,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is a 4D weights matrix reshaped into 2 dimensions,&lt;/li&gt;&#xA;&lt;li&gt;and &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is a PCA transformation matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To elaborate: say I take my 512 convolutional filters of shape (&lt;span class=&quot;math-container&quot;&gt;$3 \times 3 \times 512$&lt;/span&gt;), flatten across these three dimensions to give a (&lt;span class=&quot;math-container&quot;&gt;$4096 \times 512$&lt;/span&gt;) matrix &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, and perform PCA on that matrix, reducing it to say dimensions of (&lt;span class=&quot;math-container&quot;&gt;$4096 \times 400$&lt;/span&gt;), before reshaping back into (&lt;span class=&quot;math-container&quot;&gt;$400$&lt;/span&gt;) 3d filters and performing convolution.&lt;/p&gt;&#xA;&lt;p&gt;Is this the same as when I convolve &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, and then perform PCA on that output &lt;em&gt;using the same transformation matrix as before&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I know that matrix multiplication is associative i.e. &lt;span class=&quot;math-container&quot;&gt;$A(BC)=(AB)C$&lt;/span&gt;, and I have found that convolution operations can be rewritten as matrix multiplication.&lt;/p&gt;&#xA;&lt;p&gt;So my question is, &lt;em&gt;if I rewrite the convolution as matrix multiplication, is it associative with respect to the PCA transformation (another matrix multiplication)&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;For example, does &lt;span class=&quot;math-container&quot;&gt;$X' \cdot (W' \cdot Q) = (X' \cdot W') \cdot Q$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$X'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W'$&lt;/span&gt; represent the matrices necessary to compute the convolution in matrix multiplication form?&lt;/p&gt;&#xA;&lt;p&gt;To try and figure it out, I looked to see how convolutions could be represented as matrix multiplications, since I know matrix multiplications are associative. I've seen a few posts/sites explaining how 2D convolutions can be rewritten as matrix multiplication using Toeplitz matrices (e.g. &lt;a href=&quot;https://github.com/alisaaalehi/convolution_as_multiplication/blob/master/Convolution_as_multiplication.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this Github repository&lt;/a&gt; or &lt;a href=&quot;https://ai.stackexchange.com/q/11172/2444&quot;&gt;this AI SE post&lt;/a&gt;), however, I'm having trouble expanding on it for my question.&lt;/p&gt;&#xA;&lt;p&gt;I've also coded out simple convolutions with a &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; matrix of &lt;span class=&quot;math-container&quot;&gt;$4 \times 3$&lt;/span&gt;, an &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; matrix of &lt;span class=&quot;math-container&quot;&gt;$4 \times 2$&lt;/span&gt;, and using &lt;code&gt;sklearn&lt;/code&gt;'s PCA to reduce &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$4 \times 2$&lt;/span&gt;. If I do this both ways, the output is not the same, leading me to think this kind of associativity does not exist. But how can I explain this with linear algebra?&lt;/p&gt;&#xA;&lt;p&gt;Can anyone explain whether this is or is not the case, with a linear algebra explanation?&lt;/p&gt;&#xA;" OwnerUserId="42102" LastEditorUserId="2444" LastEditDate="2020-11-07T01:23:26.787" LastActivityDate="2020-11-07T01:23:26.787" Title="Is the 3d convolution associative given that it can be represented as matrix multiplication?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;linear-algebra&gt;&lt;principal-component-analysis&gt;&lt;convolutional-layers&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24450" PostTypeId="1" AcceptedAnswerId="24452" CreationDate="2020-11-06T16:34:13.040" Score="2" ViewCount="46" Body="&lt;p&gt;The update equation for SARSA is &lt;span class=&quot;math-container&quot;&gt;$Q(S,A) = R + \gamma Q(S',A')$&lt;/span&gt;. Consider this: I take an action &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that leads to the terminal state. Now my &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; would be one of the terminal states. So...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Intuitively, how does it make sense to take an action &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt; when the environment already ended? Or is this something you just do anyway?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once a terminal state-action pair is reached, you update the previous state-action pair and then start the game loop all over again. But this means that the terminal state-action pair (&lt;span class=&quot;math-container&quot;&gt;$Q(S',A')$&lt;/span&gt; in my example) is never updated. So, if your initial estimate of &lt;span class=&quot;math-container&quot;&gt;$Q(S',A')$&lt;/span&gt; was wrong, you would never be able to fix it which would be very problematic. (And you can't set all the terminal values to zero because you are using function approximators)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, how do I resolve these issues?&lt;/p&gt;&#xA;" OwnerUserId="42103" LastEditorUserId="2444" LastEditDate="2020-11-06T17:50:35.340" LastActivityDate="2020-11-06T17:50:35.340" Title="Intuitively, how does it make sense to take an action $A'$ when the environment already ended?" Tags="&lt;reinforcement-learning&gt;&lt;sarsa&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2020-11-06T17:59:15.897" ContentLicense="CC BY-SA 4.0" />
  <row Id="24451" PostTypeId="1" CreationDate="2020-11-06T16:51:59.050" Score="1" ViewCount="28" Body="&lt;p&gt;I am training a deep learning model for object detection. The consensus is that the more images that you have, the better the results will be. All the tutorials that I have seen say that more images are key.&lt;/p&gt;&#xA;&lt;p&gt;I am labeling objects in my images with Label-Img, which provides the algorithm with specific training samples on the images. For my images, I am using photos with dimensions of 1100 x 1100 pixels. In my case, I could generate anywhere between 50-100 high-quality training samples per image. For example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DXCOm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DXCOm.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In cases such as this where large numbers of training samples can be generated from a single image, do you really need several hundred images? Or can you lessen the number of images because of the number of training samples?&lt;/p&gt;&#xA;" OwnerUserId="32750" LastActivityDate="2020-11-07T04:45:13.477" Title="When training deep learning models for object detection in images, do you need a large number of images, or a large number of training samples?" Tags="&lt;deep-learning&gt;&lt;training&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24452" PostTypeId="2" ParentId="24450" CreationDate="2020-11-06T17:13:31.810" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Intuitively, how does it make sense to take an action A' when the environment already ended?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It doesn't make sense, in that nothing can happen once the agent reaches a terminal state. However, it is often modelled as an &amp;quot;absorbing state&amp;quot; where the action is unimportant (either &lt;code&gt;null&lt;/code&gt; or value ignored) with value by definition of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And you can't set all the terminal values to zero because you are using function approximators&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The value is zero by definition. There is no need to approximate it. So don't use function approximators for action values in terminal states.  When &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; is terminal, the update becomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(S,A) \leftarrow Q(S,A) + \alpha(R - Q(S,A))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Look at any implementation of Q learning and you will see a conditional calculation for the update value, that uses some variant of the above logic when &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; is terminal. For OpenAI Gym environments for instance, it will use the &lt;code&gt;done&lt;/code&gt; flag.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-06T17:13:31.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24453" PostTypeId="1" AcceptedAnswerId="24457" CreationDate="2020-11-06T17:30:14.363" Score="2" ViewCount="61" Body="&lt;p&gt;When we find the eigenvectors of a graph (say in the context of spectral clustering), what exactly is the vector space involved here? Of what vector space (or eigenspace) are we finding the eigenvalues of?&lt;/p&gt;&#xA;" OwnerUserId="35576" LastEditorUserId="2444" LastEditDate="2020-11-07T00:49:25.273" LastActivityDate="2020-11-08T19:01:14.440" Title="What exactly is the eigenspace of a graph (in spectral clustering)?" Tags="&lt;graphs&gt;&lt;clustering&gt;&lt;linear-algebra&gt;&lt;graph-theory&gt;&lt;spectral-analysis&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24454" PostTypeId="1" CreationDate="2020-11-06T18:04:00.040" Score="0" ViewCount="55" Body="&lt;p&gt;I am studying RL. I was thinking whether a new state value or the observation is provided by the environment before the agent actually implements the action.&lt;/p&gt;&#xA;&lt;p&gt;Take the maze problem as an example. Each state consists of all the available cells information, provided by the environment. But what if the environment is unknown? For example, there is a maze with an unknown destination cell. The agent needs to find the destination cell. The state is 1 or 0, meaning the destination reached or not. But the environment, which is the maze, can only provide the state at cell &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; which is 0 or 1 only when the agent reaches cell &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can this still be solved by RL? I am confused about the environment setup.&lt;/p&gt;&#xA;" OwnerUserId="42107" LastEditorUserId="2444" LastEditDate="2020-11-06T22:06:46.607" LastActivityDate="2021-04-06T11:02:10.223" Title="How can reinforcement learning be applied when the goal location or environment is unknown?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24456" PostTypeId="1" CreationDate="2020-11-06T19:42:52.810" Score="1" ViewCount="46" Body="&lt;p&gt;I was told that AlphaGo (or some related program) was not explicitly taught even the rules of Go -- if it was &amp;quot;just given the rulebook&amp;quot;, what does this mean? Literally, a book written in English to read?&lt;/p&gt;&#xA;" OwnerUserId="42111" LastEditorUserId="2444" LastEditDate="2020-11-07T00:44:14.087" LastActivityDate="2020-11-07T00:44:14.087" Title="Is which sense was AlphaGo &quot;just given a rule book&quot;?" Tags="&lt;reinforcement-learning&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;&lt;alphago&gt;&lt;muzero&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24457" PostTypeId="2" ParentId="24453" CreationDate="2020-11-06T19:43:29.923" Score="1" Body="&lt;p&gt;In spectral clustering we not find the eigenvectors of a graph (a graph is not a matrix) but the eigenvalues/eigenvectors of the Laplacian matrix related to the adjacency matrix of the graph:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;graph =&amp;gt; adjacency matrix =&amp;gt; Laplacian matrix =&amp;gt; eigenvalues (spectrum)&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The adjacency matrix describes the &amp;quot;similarity&amp;quot; between two graph vertexs. In the most simple case (undirected unweighted simple graph), a value &amp;quot;1&amp;quot; in the matrix means two vertex joined by an edge, a value &amp;quot;0&amp;quot; means no edge between these vertex.&lt;/p&gt;&#xA;&lt;p&gt;So, the space under the adjacency matrix is the space of connectivity, being row &amp;quot;i&amp;quot; of a column vector a measure of the connectivity with vertex &amp;quot;i&amp;quot;. In other words, the adjacency and Laplacian matrix map from vertexs to vertex connectivity.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Assume a simple graph with 3 vertex {1,2,3} and edges (1,2) and (2,3). The respective Laplacian matrix is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A=\begin{pmatrix}&#xA;1 &amp;amp; -1 &amp;amp; 0\\&#xA;-1 &amp;amp; 2 &amp;amp; -1\\&#xA;0 &amp;amp; -1 &amp;amp; 1&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;a) vertex 1, than in vertex space is (1,0,0) maps to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A\begin{pmatrix}&#xA;1\\&#xA;0\\&#xA;0&#xA;\end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;1\\&#xA;-1\\&#xA;0&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;if we analyze the product result, component by component, it means:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;vertex 1 is connected to 1 node.&lt;/li&gt;&#xA;&lt;li&gt;vertex 2 is connected to vertex 1&lt;/li&gt;&#xA;&lt;li&gt;vertex 3 is not connected to vertex 1.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;b) the set of vertexs 1 and 2, that is represented in vertex space as (1,1,0), maps to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A\begin{pmatrix}&#xA;1\\&#xA;1\\&#xA;0&#xA;\end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;0\\&#xA;1\\&#xA;-1&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;meaning that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;vertex 1 is internal or external to the set {1,2}, not frontier (in this concrete case, it is internal: belongs to set and has no edge with any node out of the set).&lt;/li&gt;&#xA;&lt;li&gt;vertex 2 is a vertex in the set and connected to one vertex out of the set (internal frontier).&lt;/li&gt;&#xA;&lt;li&gt;vertex 3 is a vertex not in the set but connected to it (external frontier).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Finally, see what happens if multiply (inner/scalar product) previous result by the vertex vector again:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;1 &amp;amp; 1 &amp;amp; 0&#xA;\end{pmatrix}&#xA;A\begin{pmatrix}&#xA;1\\&#xA;1\\&#xA;0&#xA;\end{pmatrix}&#xA;= 1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;it gives the number of edges that connects the set of nodes {1,2} with the remainder graph.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-11-08T19:01:14.440" LastActivityDate="2020-11-08T19:01:14.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24459" PostTypeId="2" ParentId="24456" CreationDate="2020-11-06T23:32:48.013" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;it was &amp;quot;just given the rulebook&amp;quot;, what does this mean? Literally a book written in English to read?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The program was &lt;strong&gt;not&lt;/strong&gt; given a natural language version of the rules to interpret. That might be an interesting AI challenge in its own way, but none of the current cutting-edge game playing reinforcement learning systems do much in the way of natural language processing.&lt;/p&gt;&#xA;&lt;p&gt;Instead, &amp;quot;just given the rulebook&amp;quot; is a rough metaphor for what actually happened: The rules of Go were implemented as functions that the game-playing agent could query. The functions can answer things such as &amp;quot;when the board looks like this, what are my valid actions?&amp;quot; and &amp;quot;if the board looks like this, have I won yet?&amp;quot;. The board state might be represented by a matrix with stone positions encoded using numbers. Outputs might be a similar matrix of numbers for valid action choices (where the player is allowed to put stones in Go) or perhaps a single number, &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for not won yet, &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for a move that wins the game.&lt;/p&gt;&#xA;&lt;p&gt;There may even be further helper functions that help assess moves (e.g. a value for how many enemy stones would be captured if a piece was played in a specific location), but the bare minimum needed is &amp;quot;what moves are valid?&amp;quot; and &amp;quot;has anyone won?&amp;quot;. A very common third function, useful for look-ahead planning is &amp;quot;if the board starts like this, and I take that action, what will the board look like next&amp;quot; - with this function, an agent can look ahead to future positions to help search for winning moves.&lt;/p&gt;&#xA;&lt;p&gt;This approach is common in game playing agents. A learning agent can in theory also learn the rules of the game through trial and error, as long as it receives some feedback when it has broken the rules. However, more often the goal of training the agent is only to play as well as possible - learning the rules from scratch would be extra work for the agent, and maybe not an interesting problem to solve. So the agent is given helper functions by the developers that allow it to explore only valid moves according to the rules.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-06T23:39:12.707" LastActivityDate="2020-11-06T23:39:12.707" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24460" PostTypeId="5" CreationDate="2020-11-07T01:05:41.157" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T01:05:41.157" LastActivityDate="2020-11-07T01:05:41.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24461" PostTypeId="4" CreationDate="2020-11-07T01:05:41.157" Score="0" Body="For questions about the conditional variational auto-encoder (CVAE), which was introduced in the paper &quot;Learning Structured Output Representation using Deep Conditional Generative Model&quot; (2015, NIPS) by Kihyuk Sohn et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T01:05:41.157" LastActivityDate="2020-11-07T01:05:41.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24462" PostTypeId="5" CreationDate="2020-11-07T01:07:05.767" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T01:07:05.767" LastActivityDate="2020-11-07T01:07:05.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24463" PostTypeId="4" CreationDate="2020-11-07T01:07:05.767" Score="0" Body="For questions about the Evidence Lower BOund (ELBO) objective function, which is typically optimized in the context of variational auto-encoders or variational Bayesian neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T01:07:05.767" LastActivityDate="2020-11-07T01:07:05.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24464" PostTypeId="2" ParentId="24454" CreationDate="2020-11-07T01:39:22.710" Score="0" Body="&lt;p&gt;When reading about RL and RL agents/algorithms, you always need to keep in mind that, &lt;em&gt;typically&lt;/em&gt;, the RL agent/algorithm is trying to &lt;strong&gt;maximize the reward&lt;/strong&gt; (or something equivalent, such as minimizing the regret) in the long run (i.e also the reward that you may receive in the future): that's its (mathematical) goal. Whether that  &lt;em&gt;also&lt;/em&gt; corresponds to the &lt;em&gt;high-level&lt;/em&gt; goal (e.g. reaching some physical location in some world) that the (human) designer of the so-called &lt;em&gt;reward function&lt;/em&gt; (i.e. the function that gives the reward to RL agent) had in mind is a different story.&lt;/p&gt;&#xA;&lt;p&gt;To maximize the reward, the RL agent interacts with the environment by taking actions, receiving rewards, and moving to other states. Initially, the RL agent does not know which actions lead to more rewards, so it may take random actions (this is known as &lt;em&gt;exploration&lt;/em&gt;). Once it starts to understand the dynamics of the environment, it may start to take only the actions that lead to high reward (this is known as &lt;em&gt;exploitation&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;To answer your question directly, the RL agent can indeed take actions without knowing the dynamics of the environment. However, initially, it may need to take some random actions (which may lead to low rewards), so that to get more rewards in the long run.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="40671" LastEditDate="2020-11-07T10:41:15.387" LastActivityDate="2020-11-07T10:41:15.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24465" PostTypeId="1" AcceptedAnswerId="24481" CreationDate="2020-11-07T01:46:12.047" Score="2" ViewCount="49" Body="&lt;p&gt;One can easily retrieve the optimal policy from the action value function but how about obtaining it from the state value function?&lt;/p&gt;&#xA;" OwnerUserId="32517" LastActivityDate="2020-11-07T13:09:38.117" Title="Is it possible to retrieve the optimal policy from the state value function?" Tags="&lt;reinforcement-learning&gt;&lt;dynamic-programming&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24468" PostTypeId="1" AcceptedAnswerId="24477" CreationDate="2020-11-07T05:48:26.397" Score="0" ViewCount="33" Body="&lt;p&gt;How significant is adding a ReLU to fully connected (FC) layers? Is it necessary, or how is the performance of a model affected by adding ReLU to FC layers?&lt;/p&gt;&#xA;" OwnerUserId="41585" LastEditorUserId="2444" LastEditDate="2020-11-07T10:55:28.647" LastActivityDate="2020-11-07T11:22:26.763" Title="How is the performance of a model affected by adding a ReLU to fully connected layers?" Tags="&lt;neural-networks&gt;&lt;deep-neural-networks&gt;&lt;activation-function&gt;&lt;performance&gt;&lt;relu&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24469" PostTypeId="2" ParentId="19891" CreationDate="2020-11-07T05:53:43.690" Score="0" Body="&lt;p&gt;Hi for me this worked perfectly. I encoded with conv2d and dense and then I flatten I and reshape in the decoder after the dense layer so the encoder and decoder are symmetrical. The only difference is that in my case I use images (224,224,1)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        # create encoder&#xA;        # 28,28 -&amp;gt; 1st conv2d (filter 3x3,relu activation, padding, strides == 'jumps')&#xA;        self.encoder = tf.keras.Sequential([layers.Input(shape=(224,224,1)),&#xA;                                            layers.Conv2D(16,kernel_size=3,activation='relu',padding='same',strides=2),                &#xA;                                            layers.Conv2D(8,kernel_size=3,activation='relu',padding='same',strides=2),&#xA;                                            layers.Conv2D(4,kernel_size=3,activation='relu',padding='same',strides=2),&#xA;                                            layers.Flatten(),&#xA;                                            layers.Dense(units=3136,activation='sigmoid')]) # (28,28,4)&#xA;        &#xA;        # deconvolution -&amp;gt; decoding &#xA;        self.decoder = tf.keras.Sequential([layers.Input(shape=(3136)),&#xA;                                            layers.Dense(units=3136,activation='sigmoid'),&#xA;                                            layers.Reshape((28,28,4)),&#xA;                                            layers.Conv2DTranspose(4,kernel_size=3,activation='relu',padding='same',strides=2),&#xA;                                            layers.Conv2DTranspose(8,kernel_size=3,strides=2,activation='relu',padding='same'),&#xA;                                            layers.Conv2DTranspose(16,kernel_size=3,strides=2,activation='relu',padding='same'),&#xA;                                            layers.Conv2D(1,kernel_size=(3,3),activation='sigmoid',padding='same')])&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38252" LastActivityDate="2020-11-07T05:53:43.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24470" PostTypeId="1" CreationDate="2020-11-07T06:48:59.583" Score="0" ViewCount="151" Body="&lt;p&gt;How many fully connected layers should be added to a convolutional neural network? Does it depend on input size to the fully connected layer? If so, how do we decide? What if the input size of the fully connected layer is very high? For example, 401408 (in my case)?&lt;/p&gt;&#xA;" OwnerUserId="41585" LastEditorUserId="2444" LastEditDate="2020-11-07T18:51:59.340" LastActivityDate="2020-11-07T18:51:59.340" Title="How to determine the number of fully connected layers for a convolutional neural network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;fully-connected-layer&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24471" PostTypeId="1" AcceptedAnswerId="24993" CreationDate="2020-11-07T07:09:22.200" Score="0" ViewCount="53" Body="&lt;p&gt;In a Convolutional Neural Network, unlike the fully connected layers, the same filter is used multiple times on the input while convolving - so during backpropagation, we get multiple derivatives for the filter parameters w.r.t the loss function. My question is, why do we sum all the derivatives to get the final gradient? Because, we don't sum the output of the convolution during forward pass. So, isn't it more sensible to average them? What is the intuition behind this?&lt;/p&gt;&#xA;&lt;p&gt;PS: although I said CNN, what I'm actually doing is correlation for simplicity of learning.&lt;/p&gt;&#xA;" OwnerUserId="42117" LastActivityDate="2021-01-03T12:00:15.663" Title="In CNNs, why do we sum the filter derivatives w.r.t the loss function to get the final gradient?" Tags="&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;convolution&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24472" PostTypeId="2" ParentId="18783" CreationDate="2020-11-07T07:42:15.190" Score="0" Body="&lt;p&gt;I solved my problem with data augmentation and decreasing learning rate value and more epochs.&lt;/p&gt;&#xA;" OwnerUserId="33792" LastActivityDate="2020-11-07T07:42:15.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24474" PostTypeId="2" ParentId="12786" CreationDate="2020-11-07T07:54:08.357" Score="0" Body="&lt;p&gt;This question has been asked a year ago when I faced this problem I searched and it hasn't any answers on it, I tried different ways and finally, data augmentation helped me.&#xA;I used data augmentation and a very small learning rate. If the fluctuations are big, the batch size should be increased, and the learning rate should be decreased. After all, using more epochs helps you to have an almost smooth plot.&lt;/p&gt;&#xA;" OwnerUserId="33792" LastActivityDate="2020-11-07T07:54:08.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24477" PostTypeId="2" ParentId="24468" CreationDate="2020-11-07T11:22:26.763" Score="2" Body="&lt;p&gt;ReLU is &lt;a href=&quot;https://en.wikipedia.org/wiki/Piecewise_linear_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;piecewise linear function&lt;/a&gt; that outputs the received input directly if it's positive, or outputs a zero. i.e., &lt;span class=&quot;math-container&quot;&gt;$max(0, x)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://miro.medium.com/max/1000/1*3JUMOqugWKB2SDra6x6v0A.png&quot; alt=&quot;img alt&quot; /&gt;&lt;/p&gt;&#xA;&lt;h3&gt;How significant is adding relu to full connected layers?&lt;/h3&gt;&#xA;&lt;p&gt;ReLU, being an &lt;a href=&quot;https://en.wikipedia.org/wiki/Activation_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;activation function&lt;/a&gt;, will determine what the output of the nodes in your FCs are. Since it's a non-linear function, one significance is it will allow the nodes in your model to learn  complex mappings between the inputs and the outputs. Compared to using a linear function, it will allow back-propagation since it has a derivative, allowing the neural network to have the advantage of stacking of multiple FC layers.&lt;/p&gt;&#xA;&lt;h3&gt;Is it necessary?&lt;/h3&gt;&#xA;&lt;p&gt;ReLU (and non-linear activation functions in general) will introduce non-linear properties in a neural network that enables it to learn more complex arbitrary structures in the inputs. Without activation functions &lt;strong&gt;between the layers&lt;/strong&gt;, your neural network will simply be a linear function, regardless of the number of layers it has. Why? A linear function + linear function gives a linear function. Additionally, see this &lt;a href=&quot;https://stackoverflow.com/a/9783865&quot;&gt;answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;How is the performance affected by adding ReLU?&lt;/h3&gt;&#xA;&lt;p&gt;Compared to no activation function at all, it will be slower to train but &lt;a href=&quot;https://stats.stackexchange.com/a/253394/296297&quot;&gt;will only behave like a linear regression model&lt;/a&gt;. So ReLU will increase the power of the model by making it non-linear.&#xA;However, compared to other non-linear activation functions like tanH, ReLU will speed up training as (1) its computation step is cheaper i.e., &lt;span class=&quot;math-container&quot;&gt;$0.0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; without additional operations (2) its gradient just depends on the sign of the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. See &lt;a href=&quot;https://stats.stackexchange.com/questions/226923/why-do-we-use-relu-in-neural-networks-and-how-do-we-use-it&quot;&gt;this answer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40671" LastActivityDate="2020-11-07T11:22:26.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24478" PostTypeId="1" AcceptedAnswerId="24504" CreationDate="2020-11-07T11:55:47.240" Score="1" ViewCount="239" Body="&lt;p&gt;When using iterative deepening, is the space complexity, &lt;span class=&quot;math-container&quot;&gt;$O(d)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the branching factor and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; the length of the optimal path (assuming that there is indeed one)?&lt;/p&gt;&#xA;" OwnerDisplayName="user42125" LastEditorUserId="2444" LastEditDate="2020-11-09T19:10:39.200" LastActivityDate="2020-11-09T19:27:11.320" Title="What is the space complexity of iterative deepening search?" Tags="&lt;search&gt;&lt;branching-factors&gt;&lt;homework&gt;&lt;space-complexity&gt;&lt;iddfs&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24480" PostTypeId="2" ParentId="24443" CreationDate="2020-11-07T13:03:36.163" Score="1" Body="&lt;p&gt;I finally grasped the concept of word embedding. Thanks to @nbro, after reading the 2 articles s/he recommended&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://machinelearningmastery.com/what-are-word-embeddings/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Are Word Embeddings for Text?&lt;/a&gt; and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.tensorflow.org/tutorials/text/word_embeddings&quot; rel=&quot;nofollow noreferrer&quot;&gt;Word embeddings&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;the 1st article gives me a good idea about the big picture of the Word Embeddings; whereas the 2nd article is actually the one which clears my mind.&lt;/p&gt;&#xA;&lt;p&gt;I am an visual person, I understand things better if I could see how the things, in this case the Word Embeddings, look like(This answered my 2nd questions).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XENZd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XENZd.png&quot; alt=&quot;Word Embeddings&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After seeing this image, my 1st question is answered and I realized that Word Embeddings is a 2 dimensional array where the number of rows of the array is decided by the number of unique words in your vocabulary and the columns/width is decided by yourself. Normally between 8 up to 1024 according to the 2nd article.&lt;/p&gt;&#xA;&lt;p&gt;The columns/width within the course I am learning from is called embedding_dim, which I found hard to comprehend. Since each word embedding is a vector (this answered my 3rd question), for example the cat is [1.2, -0.1, 4.3, 3.2], and the vector is a meta concept for me which is easy to understand, I would like to call the embedding_dim : &lt;em&gt;&lt;strong&gt;embedding_vector_width&lt;/strong&gt;&lt;/em&gt; or &lt;em&gt;&lt;strong&gt;embedding_vector_length&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For the 256 hidden features, how many of them would be enough, I think it's the same concept of how to figure out how many embedding_vector_width should be.&lt;/p&gt;&#xA;" OwnerUserId="27988" LastActivityDate="2020-11-07T13:03:36.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24481" PostTypeId="2" ParentId="24465" CreationDate="2020-11-07T13:09:38.117" Score="2" Body="&lt;p&gt;You can obtain the optimal policy from the optimal state value function if you also have the state transition and reward model for the environment &lt;span class=&quot;math-container&quot;&gt;$p(s',r|s,a)$&lt;/span&gt; - the probability of receiving reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and arriving in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; when starting in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This looks like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi^*(s) = \text{argmax}_a [\sum_{s',r} p(s',r|s,a)(r + \gamma v^*(s'))]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are variations of this function, depending on how you represent knowledge of the environment. For instance, you don't actually need the full distribution model for reward, an expected reward function and separate distribution model for state transition rules would also work.&lt;/p&gt;&#xA;&lt;p&gt;Without at least an approximate model of the environment, you cannot derive a policy from state values. If all you have is state values, then to pick an optimal action, you absolutely need the ability to look ahead a time step at what the next state might be for each action choice.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-07T13:09:38.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24482" PostTypeId="1" CreationDate="2020-11-07T14:25:35.570" Score="0" ViewCount="21" Body="&lt;p&gt;What algorithm to use when to find the &lt;strong&gt;optimal&lt;/strong&gt; path from a starting point to the end point in a multiagent game with &lt;strong&gt;collision avoidance&lt;/strong&gt;? I am in control of the agent while having zero ideas of where other agents (can be more than 1 opponent agents) would go. What algorithm should I use? I looked up something like RRT* but it seems slow to run. Are there any faster algorithms which satisfy the aforementioned constraints? Please list as many as possible, with the problem formulation. Thanks in advance!&lt;/p&gt;&#xA;&lt;p&gt;You may assume the task for other agents is the same as my agent. And there can be more than 1 targets. But for any agent, go to one of them is enough.&lt;/p&gt;&#xA;" OwnerUserId="38775" LastEditorUserId="38775" LastEditDate="2020-11-08T13:16:41.417" LastActivityDate="2020-11-08T13:16:41.417" Title="Optimal path finding in multiagent game with collision avoidance" Tags="&lt;game-ai&gt;&lt;search&gt;&lt;multi-agent-systems&gt;&lt;optimal-policy&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24483" PostTypeId="1" AcceptedAnswerId="24492" CreationDate="2020-11-07T15:29:52.103" Score="1" ViewCount="667" Body="&lt;p&gt;When using the breadth-first search algorithm, is the space complexity &lt;span class=&quot;math-container&quot;&gt;$O(b^d)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the branching factor and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; the length of the optimal path (assuming that there is indeed one)?&lt;/p&gt;&#xA;" OwnerDisplayName="user42125" LastEditorUserId="2444" LastEditDate="2020-11-09T19:06:28.947" LastActivityDate="2020-11-09T19:09:04.203" Title="What is the space complexity of breadth-first search?" Tags="&lt;search&gt;&lt;breadth-first-search&gt;&lt;branching-factors&gt;&lt;space-complexity&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24484" PostTypeId="1" AcceptedAnswerId="24491" CreationDate="2020-11-07T16:51:05.090" Score="1" ViewCount="62" Body="&lt;p&gt;I am currently studying the textbook &lt;em&gt;Neural Networks and Deep Learning&lt;/em&gt; by Charu C. Aggarwal. In chapter &lt;strong&gt;1.2.1 Single Computational Layer: The Perceptron&lt;/strong&gt;, the author says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Different choices of activation functions can be used to simulate different types of models used in machine learning, like &lt;em&gt;least-squares regression with numeric targets&lt;/em&gt;, the &lt;em&gt;support vector machine&lt;/em&gt;, or a &lt;em&gt;logistic regression classifier&lt;/em&gt;. Most of the basic machine learning models can be easily represented as simple neural network architectures.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I remember reading something about it being mathematically proven that neural networks can approximate any function, and therefore any machine learning method, or something along these lines. Am I remembering this correctly? Would someone please clarify my thoughts?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="16521" LastEditDate="2020-11-07T18:10:26.087" LastActivityDate="2020-11-07T19:52:38.910" Title="Can most of the basic machine learning models be easily represented as simple neural network architectures?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;activation-function&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24485" PostTypeId="5" CreationDate="2020-11-07T17:10:03.077" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T17:10:03.077" LastActivityDate="2020-11-07T17:10:03.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24486" PostTypeId="4" CreationDate="2020-11-07T17:10:03.077" Score="0" Body="For questions about the branching factor of a tree's (or graph's) node, i.e. the number of children of that node (or its outdegree). The time and space complexity of many traditional search algorithms, such as A* or depth-first search, is often given as a function of the branching factor." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-07T17:10:03.077" LastActivityDate="2020-11-07T17:10:03.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24487" PostTypeId="2" ParentId="50" CreationDate="2020-11-07T17:29:59.017" Score="-1" Body="&lt;p&gt;It's basically not possible to test besides some empirical experiments. All the generalization bounds only apply if your process actually follows the model assumptions which you don't actually know to be true.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="9947" LastEditDate="2020-11-08T13:07:02.920" LastActivityDate="2020-11-08T13:07:02.920" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24488" PostTypeId="1" CreationDate="2020-11-07T17:59:59.250" Score="0" ViewCount="43" Body="&lt;p&gt;Let say we are in an environment where a random agent can easily explore all the states of an environment (for example: tic-tac-toe).&lt;/p&gt;&#xA;&lt;p&gt;In those environments, using off-policy algorithm, is it a good practice to train using exclusively random actions, instead or epsilon-greedy, Boltzmann or whatever ?&lt;/p&gt;&#xA;&lt;p&gt;For my mind, it seems logical, but I have never heard about it before.&lt;/p&gt;&#xA;" OwnerUserId="23818" LastEditorUserId="23818" LastEditDate="2020-11-08T00:03:44.747" LastActivityDate="2020-11-08T00:03:44.747" Title="Off-policy full-random training in easy-to-explore environment" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;policies&gt;&lt;off-policy-methods&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="0" CommentCount="20" ContentLicense="CC BY-SA 4.0" />
  <row Id="24489" PostTypeId="1" AcceptedAnswerId="24503" CreationDate="2020-11-07T18:00:50.107" Score="1" ViewCount="181" Body="&lt;p&gt;Is the space complexity of the bidirectional search, where the breadth-first search is used for both the forward and backward search, &lt;span class=&quot;math-container&quot;&gt;$O(b^{d/2})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the branching factor and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; the length of the optimal path (assuming that there is indeed one)?&lt;/p&gt;&#xA;" OwnerDisplayName="user42125" LastEditorUserId="2444" LastEditDate="2020-11-09T19:03:29.960" LastActivityDate="2020-11-09T19:25:36.310" Title="What is the space complexity of bidirectional search?" Tags="&lt;search&gt;&lt;branching-factors&gt;&lt;breadth-first-search&gt;&lt;space-complexity&gt;&lt;bidirectional-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24490" PostTypeId="1" CreationDate="2020-11-07T18:52:52.957" Score="1" ViewCount="46" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://fardapaper.ir/mohavaha/uploads/2020/03/Fardapaper-Moth-flame-optimization-algorithm-A-novel-nature-inspired-heuristic-paradigm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm&lt;/a&gt; (2015, published in &lt;a href=&quot;https://www.journals.elsevier.com/knowledge-based-systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge-Based Systems&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The test functions are divided to three groups: unimodal, multi-modal, and composite. The unimodal functions (&lt;span class=&quot;math-container&quot;&gt;$F1 - F7$&lt;/span&gt;) are suitable for benchmarking the &lt;strong&gt;exploitation&lt;/strong&gt; of algorithms since they have one global optimum and no local optima. In contrary, multi-modal functions (&lt;span class=&quot;math-container&quot;&gt;$F8 - F13$&lt;/span&gt;) have a massive number of local optima and are helpful to examine &lt;strong&gt;exploration&lt;/strong&gt; and local optima avoidance of algorithms&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I imagine that exploration means it goes searching for something in unknown regions from a starting point. But exploitation would search more around the starting (or current point).&lt;/p&gt;&#xA;&lt;p&gt;It is more or less that? What else differentiates both concepts?&lt;/p&gt;&#xA;" OwnerUserId="41573" LastEditorUserId="2444" LastEditDate="2020-11-08T11:30:33.610" LastActivityDate="2020-11-08T11:30:33.610" Title="What is the difference between exploitation and exploration in the context of optimization?" Tags="&lt;comparison&gt;&lt;optimization&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;meta-heuristics&gt;&lt;moth-flame-optimization&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24491" PostTypeId="2" ParentId="24484" CreationDate="2020-11-07T19:52:38.910" Score="1" Body="&lt;p&gt;I think the author refers to both different choices of activation function and loss. It is explained in more detail in chapter 2. In particular 2.3 is ilustrative of this point.&lt;/p&gt;&#xA;&lt;p&gt;I don't think there is a relation between this argument and universal approximation theorems, which state that certain classes of neural networks can approximate any &lt;em&gt;function&lt;/em&gt; in certain domains, rather than any &lt;em&gt;learning algorithm&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36115" LastActivityDate="2020-11-07T19:52:38.910" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24492" PostTypeId="2" ParentId="24483" CreationDate="2020-11-07T21:29:20.453" Score="0" Body="&lt;p&gt;The space complexity of the breadth-first search algorithm is &lt;span class=&quot;math-container&quot;&gt;$O(b^d$&lt;/span&gt;) &lt;em&gt;in the worst case&lt;/em&gt;, and it corresponds to the largest possible number of nodes that may be stored in the &lt;a href=&quot;https://ai.stackexchange.com/a/5951/2444&quot;&gt;frontier&lt;/a&gt; at once, where the frontier is the set of nodes (or states) that you are currently considering for expansion.&lt;/p&gt;&#xA;&lt;p&gt;You can take a look at &lt;a href=&quot;https://www.cin.ufpe.br/%7Etfl2/artificial-intelligence-modern-approach.9780131038059.25368.pdf#page=96&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 3.5 (page 74)&lt;/a&gt; of the book &lt;a href=&quot;https://www.cin.ufpe.br/%7Etfl2/artificial-intelligence-modern-approach.9780131038059.25368.pdf#page=96&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (3rd edition, by Norvig and Russell) for more info about the time and space complexity of BFS.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T19:09:04.203" LastActivityDate="2020-11-09T19:09:04.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24493" PostTypeId="1" AcceptedAnswerId="26887" CreationDate="2020-11-08T05:30:05.943" Score="2" ViewCount="46" Body="&lt;p&gt;While reading the Mutual Information Neural Estimation (MINE) paper [&lt;a href=&quot;https://arxiv.org/abs/1801.04062&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] I came across section &lt;strong&gt;3.2 Correcting the bias from the stochastic gradients&lt;/strong&gt;. The proposed method requires the computation of the gradient&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat{G}_B = \mathbb{E}_B[\nabla_{\theta}T_{\theta}] - \frac{\mathbb{E_B}[\nabla_{\theta}T_{\theta}e^{T_{\theta}}]}{\mathbb{E}_B[e^{T_{\theta}}]},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_B$&lt;/span&gt; denotes the expectation operation w.r.t. a minibatch &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$T_{\theta}$&lt;/span&gt; is a neural network parameterized by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. The authors claim that this gradient estimation is biased and that can be reduced by simply performing an exponential moving average filtering.&lt;/p&gt;&#xA;&lt;p&gt;Can someone give me a hint to understand these two points:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why is &lt;span class=&quot;math-container&quot;&gt;$\hat{G}_B$&lt;/span&gt; biased, and&lt;/li&gt;&#xA;&lt;li&gt;How does the exponential moving average reduce the bias?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42139" LastEditorUserId="2444" LastEditDate="2020-11-08T12:09:38.993" LastActivityDate="2021-03-18T07:20:36.647" Title="In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;generative-adversarial-networks&gt;&lt;stochastic-gradient-descent&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24494" PostTypeId="1" AcceptedAnswerId="24499" CreationDate="2020-11-08T10:49:07.233" Score="2" ViewCount="707" Body="&lt;p&gt;If uniform cost search is used for both the forward and backward search in bidirectional search, is it guaranteed the solution is optimal?&lt;/p&gt;&#xA;" OwnerDisplayName="user42125" LastEditorDisplayName="user42125" LastEditDate="2020-11-20T15:18:09.137" LastActivityDate="2020-11-20T15:18:09.137" Title="If uniform cost search is used for bidirectional search, is it guaranteed the solution is optimal?" Tags="&lt;search&gt;&lt;proofs&gt;&lt;uniform-cost-search&gt;&lt;bidirectional-search&gt;&lt;optimality&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24495" PostTypeId="5" CreationDate="2020-11-08T11:53:28.180" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-08T11:53:28.180" LastActivityDate="2020-11-08T11:53:28.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24496" PostTypeId="4" CreationDate="2020-11-08T11:53:28.180" Score="0" Body="For questions about the bidirectional search, which is a search approach where two searches are performed simultaneously, one starting from the start node and the other starting from the goal node, until both meet." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-08T11:53:28.180" LastActivityDate="2020-11-08T11:53:28.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24497" PostTypeId="5" CreationDate="2020-11-08T11:55:02.967" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-08T11:55:02.967" LastActivityDate="2020-11-08T11:55:02.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24498" PostTypeId="4" CreationDate="2020-11-08T11:55:02.967" Score="0" Body="For questions about the optimality of artificial intelligence algorithms, including search or reinforcement learning algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-08T11:55:02.967" LastActivityDate="2020-11-08T11:55:02.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24499" PostTypeId="2" ParentId="24494" CreationDate="2020-11-08T12:35:58.727" Score="1" Body="&lt;h2&gt;UCS is optimal (but not necessarily complete)&lt;/h2&gt;&#xA;&lt;p&gt;Let's first recall that the uniform-cost search (UCS) is optimal (i.e. if it finds a solution, &lt;a href=&quot;https://ai.stackexchange.com/a/24522/2444&quot;&gt;which is not guaranteed unless the costs on the edges are big enough&lt;/a&gt;, that solution is optimal) and it expands nodes with the smallest value of the evaluation function &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the length/cost of the path from the goal/start node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Is bidirectional search with UCS optimal?&lt;/h2&gt;&#xA;&lt;p&gt;The problem of bidirectional search with UCS for the forward and backward searches is that UCS does not proceed layer-by-layer (&lt;a href=&quot;https://ai.stackexchange.com/a/24503/2444&quot;&gt;as breadth-first search does, which ensures that when the forward and backward searches meet, the optimal path has been found, assuming they &lt;em&gt;both&lt;/em&gt; expand one level at each iteration&lt;/a&gt;), so the forward search may explore one part of the search space while the backward search may explore a different part, and it could happen (although I don't have the proof: I need to think about it a little bit more!), that these searches do not meet. So, I will consider both cases:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;when the forward and backward searches do not &amp;quot;meet&amp;quot; (the worst case, in terms of time and space complexity)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;when they meet (the non-degenerate case)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Degenerate case&lt;/h3&gt;&#xA;&lt;p&gt;Let's consider the case when the forward search does &lt;strong&gt;not&lt;/strong&gt; meet the backward search (the worst/degenarate case).&lt;/p&gt;&#xA;&lt;p&gt;If we assume that &lt;a href=&quot;https://ai.stackexchange.com/a/24522/2444&quot;&gt;the costs on the edges are big enough&lt;/a&gt; and the start node &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is reachable from &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; (or vice-versa), then bidirectional search eventually degenerates to two independent uniform-cost searches, which are optimal, which makes BS optimal too.&lt;/p&gt;&#xA;&lt;h3&gt;Non-generate case&lt;/h3&gt;&#xA;&lt;p&gt;Let's consider the case when the forward search &lt;strong&gt;meets&lt;/strong&gt; the backward search.&lt;/p&gt;&#xA;&lt;p&gt;To ensure optimality, we cannot just stop searching when we take off both the frontiers the same &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. To see why, consider this example. We take off the first frontier node &lt;span class=&quot;math-container&quot;&gt;$n_1$&lt;/span&gt; with cost &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, then we take off the same frontier node &lt;span class=&quot;math-container&quot;&gt;$n_2$&lt;/span&gt; with cost &lt;span class=&quot;math-container&quot;&gt;$N+10$&lt;/span&gt;. Meanwhile, we take off the &lt;em&gt;other&lt;/em&gt; frontier node &lt;span class=&quot;math-container&quot;&gt;$n_2$&lt;/span&gt; with cost &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and the node &lt;span class=&quot;math-container&quot;&gt;$n_1$&lt;/span&gt; with cost &lt;span class=&quot;math-container&quot;&gt;$K + 1$&lt;/span&gt;. So, we have two paths: one with cost &lt;span class=&quot;math-container&quot;&gt;$N+(K + 1)$&lt;/span&gt; and one with cost &lt;span class=&quot;math-container&quot;&gt;$(N+10)+K$&lt;/span&gt;, which is bigger than &lt;span class=&quot;math-container&quot;&gt;$N+(K + 1)$&lt;/span&gt;, but we took off both frontiers &lt;span class=&quot;math-container&quot;&gt;$n_2$&lt;/span&gt; first.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://ai.stackexchange.com/a/24552/2444&quot;&gt;the other answer&lt;/a&gt; for more details and resources that could be helpful to understand the appropriate stopping condition for the BS.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-11T21:34:28.723" LastActivityDate="2020-11-11T21:34:28.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24500" PostTypeId="1" CreationDate="2020-11-08T14:20:56.567" Score="1" ViewCount="29" Body="&lt;p&gt;Agrawal and Goyal (&lt;a href=&quot;http://proceedings.mlr.press/v23/agrawal12/agrawal12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proceedings.mlr.press/v23/agrawal12/agrawal12.pdf&lt;/a&gt; page 3) discussed how we can extend Thompson sampling for bernoulli bandits to Thompson sampling for stochastic bandits in general by simply Bernoulli sampling with the received reward &lt;span class=&quot;math-container&quot;&gt;$r_t \in [0,1]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is whether such extension from Bernoulli bandits to general stochastic bandits hold in general and not only for Thompson sampling. E.g. can I prove properties such as lower bounds on regret for Bernoulli bandits and always transfer these results to general stochastic bandits?&lt;/p&gt;&#xA;" OwnerUserId="36978" LastActivityDate="2020-11-08T14:20:56.567" Title="Multi-armed bandits: reducing stochastic multi-armed bandits to bernoulli bandits" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;thompson-sampling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24501" PostTypeId="1" CreationDate="2020-11-08T14:21:23.417" Score="1" ViewCount="40" Body="&lt;p&gt;My goal is to create an ML model to be able to classify different game stages, e.g., dialog with a non-player character, exploration, combat with enemy, in-game menu etc.&lt;/p&gt;&#xA;&lt;p&gt;In order to do that, I am looking for an agent pre-trained on such a game. I am intending to develop a model using this pre-trained agent to produce a data set (frames-labels) and finally I will use that data set to train a model to classify those different stages.&lt;/p&gt;&#xA;&lt;p&gt;I could only find a pre-trained model for the Doom, however, it is not much appropriate for my case because it does not have different game stages (it is merely based on running &amp;amp; shooting).&#xA;Training my own Reinforcement Learning Agent is a whole another workload in terms of both time and GPU such a game needs.&lt;/p&gt;&#xA;&lt;p&gt;Any single idea could help me a lot. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="41691" LastEditorUserId="32410" LastEditDate="2020-11-10T13:10:57.927" LastActivityDate="2020-11-10T13:10:57.927" Title="Where can I find pre-trained agents able to play games with multiple stages like exploration, dialog, combat?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24503" PostTypeId="2" ParentId="24489" CreationDate="2020-11-08T22:49:04.780" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.cin.ufpe.br/%7Etfl2/artificial-intelligence-modern-approach.9780131038059.25368.pdf#page=102&quot; rel=&quot;nofollow noreferrer&quot;&gt;Norvig &amp;amp; Russell's book (section 3.5)&lt;/a&gt; states that the space complexity of the bidirectional search (which &lt;a href=&quot;https://ai.stackexchange.com/a/24492/2444&quot;&gt;corresponds to the largest possible number of nodes that you save in the frontier&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$O(2b^{d/2}) = O(b^{d/2}).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The intuition behind this result is that (as opposed to e.g. uniform-cost search or breadth-first search, which have space (and time) complexity of &lt;span class=&quot;math-container&quot;&gt;$O(b^{d})$&lt;/span&gt;) is that the forward and backward searches only have to go half way, so you will not eventually need to expand all &lt;span class=&quot;math-container&quot;&gt;$b^{d}$&lt;/span&gt; leaves, but only half of them.&lt;/p&gt;&#xA;&lt;p&gt;However, this space complexity is correct if you use a breadth-first search for the forward and backward searches (which is your scenario!), given that breadth-first search, assuming a finite branching factor, expands &lt;em&gt;one level at a time&lt;/em&gt;, so it's guaranteed that both the forward and backward searches meet in the middle. This can be seen in figure 3.17 of the same book, where you can see that both searches have the same &amp;quot;radius&amp;quot;. Moreover, the &lt;em&gt;only&lt;/em&gt; nodes that you need to store in the frontier are the ones on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Circumference&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;circumference&lt;/em&gt;&lt;/a&gt; (not all nodes that you see in the image)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dMtlO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dMtlO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, if you used another search algorithm to perform the forward and backward searches, the space complexity may be different. This is true if e.g. the searches do not meet and then they end up exploring all the state space.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T19:25:36.310" LastActivityDate="2020-11-09T19:25:36.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24504" PostTypeId="2" ParentId="24478" CreationDate="2020-11-08T23:21:44.630" Score="0" Body="&lt;p&gt;As stated in my other answers &lt;a href=&quot;https://ai.stackexchange.com/a/24503/2444&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/24492/2444&quot;&gt;here&lt;/a&gt;, the space complexity of these search algorithms is calculated by looking at the largest possible number of nodes that you may need to save in the &lt;em&gt;frontier&lt;/em&gt; during the search.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Iterative deepening search (IDS)&lt;/em&gt; is a search algorithm that iteratively uses depth-first search (DFS), which has a space complexity of &lt;span class=&quot;math-container&quot;&gt;$O(bm)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the maximum depth, with progressively bigger depths, and it turns out that its space complexity is &lt;span class=&quot;math-container&quot;&gt;$O(bd)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;(Note that there is also the informed version of IDS, which uses the informed search algorithm A*, rather than DFS, known as &lt;a href=&quot;https://ai.stackexchange.com/a/8909/2444&quot;&gt;&lt;em&gt;iterative deepening A*&lt;/em&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T19:27:11.320" LastActivityDate="2020-11-09T19:27:11.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24505" PostTypeId="5" CreationDate="2020-11-08T23:28:15.997" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-08T23:28:15.997" LastActivityDate="2020-11-08T23:28:15.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24506" PostTypeId="4" CreationDate="2020-11-08T23:28:15.997" Score="0" Body="For questions about the iterative deepening (ID) search algorithm, which is an algorithm that runs a depth-first search (DFS) with progressively bigger depths. There is also iterative deepening A*, which is iterative deepening with A* (rather than DFS). So, iterative deepening should be more precisely called iterative deepening depth-first search (IDDFS)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-08T23:28:15.997" LastActivityDate="2020-11-08T23:28:15.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24507" PostTypeId="1" CreationDate="2020-11-09T02:13:51.107" Score="0" ViewCount="19" Body="&lt;p&gt;This is a general question.&#xA;I am training a neural machine translation model with different batch sizes: 16, 32, and 64 for the same amount of epochs (10).&lt;/p&gt;&#xA;&lt;p&gt;I am getting very low validation scores for batch sizes 16 and 32, and getting higher scores for batch size 64.&lt;/p&gt;&#xA;&lt;p&gt;Does this make any sense? Why would the validation scores be lower for batch sizes 16/32.&lt;/p&gt;&#xA;" OwnerUserId="42152" LastEditorUserId="40671" LastEditDate="2020-11-09T09:53:06.093" LastActivityDate="2020-11-09T09:53:06.093" Title="Effect of Batch Size on a Translation Model's Validation Scores" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;machine-translation&gt;&lt;batch-size&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24508" PostTypeId="1" CreationDate="2020-11-09T07:33:35.830" Score="4" ViewCount="110" Body="&lt;p&gt;I had a question today that I feel it must have an answer already, so I'm shopping around.&lt;/p&gt;&#xA;&lt;p&gt;If we ask a model to learn the binary OR function, we get perfect accuracy with every model (as far as I know).&lt;/p&gt;&#xA;&lt;p&gt;If we ask a model to learn the XOR function we get perfect accuracy with some models and an approximation with others (e.g. perceptrons).&lt;/p&gt;&#xA;&lt;p&gt;This is due to the way perceptrons are designed -- it's a surface the algorithm can't learn. But again, with a multi-layered neural network, we can get 100% accuracy.&lt;/p&gt;&#xA;&lt;p&gt;So can we perfectly learn a solved game as well?&lt;/p&gt;&#xA;&lt;p&gt;Tic-tac-toe is a solved game; an optimal move exists for both players in every state of the game. So in theory our model could learn tic-tac-toe as well as it could a logic function, right?&lt;/p&gt;&#xA;" OwnerUserId="17612" LastEditorUserId="36578" LastEditDate="2020-11-09T23:27:57.647" LastActivityDate="2021-04-12T09:04:35.503" Title="Can models get 100% accuracy on solved games?" Tags="&lt;machine-learning&gt;&lt;function-approximation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24509" PostTypeId="5" CreationDate="2020-11-09T11:24:47.170" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-11-09T11:24:47.170" LastActivityDate="2020-11-09T11:24:47.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24510" PostTypeId="4" CreationDate="2020-11-09T11:24:47.170" Score="0" Body="Planning Domain Definition Language is used to define planning domain models used by AI Planning engines. Use this tag for questions about modeling in PDDL syntax, parsing PDDL models, interpreting constructs the PDDL syntax allows and about widely used tools such as VAL/Validate, or well known planner implementations." OwnerUserId="42161" LastEditorUserId="42161" LastEditDate="2020-11-10T13:10:15.353" LastActivityDate="2020-11-10T13:10:15.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24511" PostTypeId="2" ParentId="10620" CreationDate="2020-11-09T12:08:27.323" Score="1" Body="&lt;p&gt;The original question about &lt;em&gt;both&lt;/em&gt; the estimation of the &lt;em&gt;transition model&lt;/em&gt;, often denoted as &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, and the &lt;em&gt;reward function&lt;/em&gt;, sometimes denoted as &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;, arose because I was thinking about the probability distribution often denoted as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\color{red}{p}\left(s^{\prime}, r \mid s, a\right) \doteq \operatorname{Pr}\left\{S_{t}=s^{\prime}, R_{t}=r \mid S_{t-1}=s, A_{t-1}=a\right\},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is often called the &lt;em&gt;model&lt;/em&gt; (and &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=70&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto&lt;/a&gt; also call it the &lt;em&gt;dynamics function&lt;/em&gt;, given that it defines the &lt;em&gt;dynamics&lt;/em&gt; of the environment), which incorporates both the transition model and reward function. In fact, both the transition probability distribution (of the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;) and &lt;a href=&quot;https://ai.stackexchange.com/q/10442/2444&quot;&gt;different types of reward functions&lt;/a&gt; can be written as a function of this &lt;em&gt;dynamics function&lt;/em&gt;. More precisely, we have the following results&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\color{orange}{p}\left(s^{\prime} \mid s, a\right) \doteq \operatorname{Pr}\left\{S_{t}=s^{\prime} \mid S_{t-1}=s, A_{t-1}=a\right\}=\sum_{r \in \mathcal{R}} \color{red}{p}\left(s^{\prime}, r \mid s, a\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\color{blue}{r}(s, a) \doteq \mathbb{E}\left[R_{t} \mid S_{t-1}=s, A_{t-1}=a\right]=\sum_{r \in \mathcal{R}} r \sum_{s^{\prime} \in \mathcal{S}} \color{red}{p}\left(s^{\prime}, r \mid s, a\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\color{cyan}{r}\left(s, a, s^{\prime}\right) \doteq \mathbb{E}\left[R_{t} \mid S_{t-1}=s, A_{t-1}=a, S_{t}=s^{\prime}\right]=\sum_{r \in \mathcal{R}} r \frac{\color{red}{p}\left(s^{\prime}, r \mid s, a\right)}{\color{orange}{p}\left(s^{\prime} \mid s, a\right)}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Given that &lt;a href=&quot;https://ai.stackexchange.com/a/23647/2444&quot;&gt;this answer&lt;/a&gt; already (partially) answers the question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How can we estimate the transition model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;but not the question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How can we estimate the reward function?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I will provide here an answer to this second question.&lt;/p&gt;&#xA;&lt;p&gt;We can use &lt;strong&gt;inverse reinforcement learning&lt;/strong&gt; techniques, i.e. techniques that estimate the reward function given trajectories of the form &lt;span class=&quot;math-container&quot;&gt;$$\tau = (s_1, a_1, r_1, s_{2}, \dots, s_{T-1}, a_{T-1}, r_{T-1}, s_{T}),$$&lt;/span&gt; which are often assumed to have been generated by some optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;, to estimate the reward function. An example of such a technique is &lt;a href=&quot;https://arxiv.org/pdf/1710.11248.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIRL&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T12:22:19.570" LastActivityDate="2020-11-09T12:22:19.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24513" PostTypeId="5" CreationDate="2020-11-09T17:22:21.023" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T17:22:21.023" LastActivityDate="2020-11-09T17:22:21.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24514" PostTypeId="4" CreationDate="2020-11-09T17:22:21.023" Score="0" Body="For questions about potential-based reward shaping, which was introduced in the paper &quot;Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping&quot; by Andrew Y. Ng et al, in 1999." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T17:22:21.023" LastActivityDate="2020-11-09T17:22:21.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24516" PostTypeId="1" AcceptedAnswerId="24522" CreationDate="2020-11-09T18:22:05.793" Score="2" ViewCount="138" Body="&lt;p&gt;I was reading &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0136042597&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach 3rd Edition&lt;/a&gt;, and I have reached to the UCS algorithm.&lt;/p&gt;&#xA;&lt;p&gt;I was reading the proof that UCS is complete.&lt;/p&gt;&#xA;&lt;p&gt;The book state that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Completeness is guaranteed provided the cost of every step exceeds some small positive constant &lt;span class=&quot;math-container&quot;&gt;$\epsilon .$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And that's because UCS will be stuck if there is a path with an infinite&#xA;sequence of zero-cost actions.&lt;/p&gt;&#xA;&lt;p&gt;Why the step cost must exceed &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;? Isn't enough for it to be greater than zero?&lt;/p&gt;&#xA;" OwnerUserId="36578" LastEditorUserId="2444" LastEditDate="2020-11-09T19:01:14.113" LastActivityDate="2020-11-09T23:26:59.480" Title="Why is the completeness of UCS guaranteed only if the cost of every step exceeds some small positive constant?" Tags="&lt;search&gt;&lt;uniform-cost-search&gt;&lt;completeness&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24517" PostTypeId="5" CreationDate="2020-11-09T19:04:18.897" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T19:04:18.897" LastActivityDate="2020-11-09T19:04:18.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24518" PostTypeId="4" CreationDate="2020-11-09T19:04:18.897" Score="0" Body="For questions about the completeness of search algorithms. A search algorithm is complete if it is guaranteed to find one solution (if there is one)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-09T19:04:18.897" LastActivityDate="2020-11-09T19:04:18.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24519" PostTypeId="2" ParentId="23019" CreationDate="2020-11-09T22:23:10.237" Score="1" Body="&lt;p&gt;The answer to your question can be found in the original paper that introduced the &lt;strong&gt;max-margin&lt;/strong&gt; and &lt;strong&gt;projection&lt;/strong&gt; imitation learning (IL) algorithms: &lt;a href=&quot;https://ai.stanford.edu/%7Eang/papers/icml04-apprentice.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apprenticeship Learning via Inverse Reinforcement Learning&lt;/a&gt; (by Abbel and Ng, 2004, ICML). Specifically, &lt;a href=&quot;https://ai.stanford.edu/%7Eang/papers/icml04-apprentice.pdf#page=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;theorem 1 (section 4, page 4)&lt;/a&gt; states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Let an &lt;span class=&quot;math-container&quot;&gt;$\text{MDP} \setminus R$&lt;/span&gt;, features &lt;span class=&quot;math-container&quot;&gt;$ \phi : S \rightarrow [0, 1]^k$&lt;/span&gt;, and any &lt;span class=&quot;math-container&quot;&gt;$\epsilon &amp;gt; 0$&lt;/span&gt; be given. Then the apprenticeship learning algorithm (both max-margin and projection versions) will terminate with &lt;span class=&quot;math-container&quot;&gt;$t^{(i)} \leq \epsilon$&lt;/span&gt; after at most&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$n=O\left(\frac{k}{(1-\gamma)^{2} \epsilon^{2}} \log \frac{k}{(1-\gamma) \epsilon}\right)$$&lt;/span&gt;&#xA;iterations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the dimension of the feature vectors, so it's clear that the number of iterations needed for these algorithms to terminate scales with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. The proof of this theorem can be found in appendix A of the same paper (and all other terms are defined in the paper, which you should read to understand all the details). Of course, this result holds (only) for these specific IL algorithms (which are the algorithms the author of your slides, Abbel, is referring to). See also theorem 2 and the experiments section (in particular, figure 4, which shows the performance as a function of the number of trajectories) of the same paper.&#xA;&lt;a href=&quot;https://github.com/rhklite/apprenticeship_inverse_RL/blob/master/Apprenticeship%20Learning%20via%20Inverse%20Reinforcement%20Learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;These slides&lt;/a&gt; provide a nice overview of the contents of this paper, so I suggest that you read them too.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-10T13:44:05.670" LastActivityDate="2020-11-10T13:44:05.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24522" PostTypeId="2" ParentId="24516" CreationDate="2020-11-09T22:58:44.693" Score="4" Body="&lt;p&gt;Let's consider a problem where all edge costs are greater than zero, but not above some &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Image a problem where we have an infinite path where the first edge is cost &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2}$&lt;/span&gt;, the next is &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{4}$&lt;/span&gt;, the following is &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{8}$&lt;/span&gt;, and so on forever. Every edge is greater than zero, meeting the condition being proposed in the question. However, this path overall has finite cost (1) even through there are an infinite number of states on that path. So, on this problem UCS will never reach paths with cost greater than 1. Thus, if the solution cost is 2, UCS will not find any solution to this problem, and thus it would not be a complete algorithm. So, all edges being greater than zero is not sufficient.&lt;/p&gt;&#xA;&lt;p&gt;For most search algorithms to be complete, there must be a finite number of states with any given cost. (To be slightly more precise, there must exist some fixed &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; such that in each range size &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; there are a finite number of states.)&lt;/p&gt;&#xA;" OwnerUserId="17493" LastEditorUserId="17493" LastEditDate="2020-11-09T23:26:59.480" LastActivityDate="2020-11-09T23:26:59.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24523" PostTypeId="2" ParentId="24508" CreationDate="2020-11-10T02:42:46.690" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;So can we perfectly learn a solved game as well?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The short answer is yes. If your model has enough complexity it can theoretically learn any behavior you want.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So in theory our model could learn tic-tac-toe&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Tic Tac Toe has already been solved. Another popular game that has been solved is Checkers, by the algorithm Chinook.&lt;/p&gt;&#xA;&lt;p&gt;To be more specific, in Reinforcement Learning we make the assumption that any decision making process can be modeled as an MDP (Markov Decision Process). Once there, there are a host of different methods like Q-Learning and TD that theoretically converge towards the optimal policy - the one the plays perfectly.&lt;/p&gt;&#xA;&lt;p&gt;Now, just because it is theoretically possible doesn't mean it will always empirically work. In games that are very complex and have a large state space it is extremely difficult to perfectly solve. This is because the only feasible way to go about them is to approximate and getting perfect play even in small edge cases becomes much more difficult as a result.&lt;/p&gt;&#xA;&lt;p&gt;If you want to learn more about this topic I would highly recommend this series &lt;a href=&quot;https://www.youtube.com/watch?v=2pWv7GOvuf0&amp;amp;list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&amp;amp;ab_channel=DeepMind&quot; rel=&quot;nofollow noreferrer&quot;&gt;RL Course by David Silver&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42103" LastEditorUserId="1847" LastEditDate="2020-11-13T07:41:33.947" LastActivityDate="2020-11-13T07:41:33.947" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24524" PostTypeId="1" CreationDate="2020-11-10T05:38:30.587" Score="1" ViewCount="292" Body="&lt;p&gt;According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?&lt;/p&gt;&#xA;" OwnerUserId="42182" LastEditorUserId="2444" LastEditDate="2020-11-10T09:55:08.213" LastActivityDate="2020-11-10T11:38:28.067" Title="How is Google Translate able to convert texts of different lengths?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;machine-translation&gt;&lt;google-translate&gt;&lt;seq2seq&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24525" PostTypeId="2" ParentId="24524" CreationDate="2020-11-10T09:39:18.860" Score="3" Body="&lt;p&gt;Usually, in natural language processing (NLP), they are using Sequence to Sequence Learning (Seq2Seq) with Neural Networks, such as Recurrent Neural Networks or more recently the Transformer (you can find two very good papers &lt;a href=&quot;https://arxiv.org/abs/1409.3215&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;During &lt;em&gt;training&lt;/em&gt;, to ensure the same size of the input and output they can just search for the longest sentence they have in the dataset or select a number that is high enough and pad all the other sentences with 0. In addition, they add a stop token where the sentence ends, so that the model is aware of this. When &lt;em&gt;decoding (inference)&lt;/em&gt;, the decoder is going to predict one word at a time until it predicts the stop token, which signals that the translation is done.&lt;/p&gt;&#xA;&lt;p&gt;If you're interested to see an actual implementation, I would recommend looking at &lt;a href=&quot;https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial which does a good job at explaining the code and how it works.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2020-11-10T11:38:28.067" LastActivityDate="2020-11-10T11:38:28.067" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24526" PostTypeId="1" CreationDate="2020-11-10T09:59:20.273" Score="2" ViewCount="64" Body="&lt;p&gt;I'm reading the book Grokking Deep Learning. Regarding weight updates during training, it has the following code and explanation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;direction_and_amount = (pred - goal_pred) * input&#xA;weight = weight - direction_and_amount&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It explains the motivation behind multiplying the prediction difference with &lt;strong&gt;input&lt;/strong&gt; using three cases: scaling, negative reversal and stopping.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are scaling, negative reversal, and stopping? These three attributes have the combined effect of translating the pure error into the absolute amount you want to change weight.  They do so by addressing three major edge cases where the pure error isn’t sufficient to make a good modification to weight.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These three cases are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Negative input,&lt;/li&gt;&#xA;&lt;li&gt;zero input and&lt;/li&gt;&#xA;&lt;li&gt;the value of input (scaling).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Negative and zero cases are very obvious. However, I didn't understand scaling. Regarding scaling, there's the following explanation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Scaling is the third effect on the pure error caused by multiplying it by input. Logically, if the input is big, your weight update should also be big. This is more of a side effect, because it often goes out of control. Later, you’ll use alpha to address when that happens.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But I didn't understand it. Considering the linear regression problem, why weight update should be big if the input is big?&lt;/p&gt;&#xA;" OwnerUserId="42191" LastEditorUserId="2444" LastEditDate="2020-11-10T14:42:09.713" LastActivityDate="2020-11-10T14:42:09.713" Title="Why should the weight updates be proportional to input?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24527" PostTypeId="2" ParentId="3981" CreationDate="2020-11-10T10:16:01.320" Score="0" Body="&lt;p&gt;You &lt;em&gt;could&lt;/em&gt; use &lt;em&gt;transfer learning&lt;/em&gt; (i.e. use a pre-trained model, then change its last layer to accommodate the new classes, and re-train this slightly modified model, maybe with a lower learning rate) to achieve that, but transfer learning does &lt;strong&gt;not&lt;/strong&gt; necessarily attempt to retain any of the previously acquired information (especially if you don't use very small learning rates, you keep on training and you do not freeze the weights of the convolutional layers), but only to speed up training or when your new dataset is not big enough, by starting from a model that has already learned general features that are supposedly similar to the features needed for your specific task. There is also the related &lt;em&gt;domain adaptation&lt;/em&gt; problem.&lt;/p&gt;&#xA;&lt;p&gt;There are more suitable approaches to perform &lt;strong&gt;incremental class learning&lt;/strong&gt; (which is what you are asking for!), which directly address the &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;&lt;strong&gt;catastrophic forgetting&lt;/strong&gt; problem&lt;/a&gt;. For instance, you can take a look at this paper &lt;a href=&quot;https://arxiv.org/pdf/1903.07864.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Class-incremental Learning via Deep Model Consolidation&lt;/a&gt;, which proposes the &lt;strong&gt;Deep Model Consolidation (DMC)&lt;/strong&gt; approach. There are other &lt;strong&gt;continual/incremental learning&lt;/strong&gt; approaches, many of them are described &lt;a href=&quot;https://ai.stackexchange.com/a/24529/2444&quot;&gt;here&lt;/a&gt; or in more detail &lt;a href=&quot;https://reader.elsevier.com/reader/sd/pii/S0893608019300231&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-10T12:25:16.283" LastActivityDate="2020-11-10T12:25:16.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24529" PostTypeId="2" ParentId="23567" CreationDate="2020-11-10T11:11:51.543" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your intuition that it is possible to perform &lt;em&gt;incremental&lt;/em&gt; (aka &lt;em&gt;continual&lt;/em&gt;, &lt;em&gt;continuous&lt;/em&gt; or &lt;em&gt;lifelong&lt;/em&gt;) &lt;em&gt;learning&lt;/em&gt; by changing the NN's topology is correct. However, &lt;strong&gt;dynamically adapting the NN's topology&lt;/strong&gt; is &lt;em&gt;just&lt;/em&gt; one approach to continual learning (a specific example of this approach is &lt;a href=&quot;https://arxiv.org/pdf/1708.01547.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DEN&lt;/a&gt;). So, there are other approaches, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;regularization approaches&lt;/strong&gt; (examples are &lt;a href=&quot;https://arxiv.org/pdf/1606.09282.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LwF&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1612.00796.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;EWC&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1703.04200.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SI&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;ensemble approaches&lt;/strong&gt; (examples are &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.16.1297&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learn++&lt;/a&gt;, &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.409.2942&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEL&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1701.08734.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PathNet&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/23832/2444&quot;&gt;rehearsal&lt;/a&gt; approaches&lt;/strong&gt; (an example is &lt;a href=&quot;https://arxiv.org/pdf/1611.07725.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;iCaRL&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For more details about these and other approaches (and problems related to continual learning and catastrophic forgetting in neural networks), take a look at &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608019300231&quot; rel=&quot;nofollow noreferrer&quot;&gt;this very nice review of continual learning approaches in neural networks&lt;/a&gt;. You should also check &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are there ways to implement continuous learning in a deep neural network for image recognition?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. Many of the approaches focus on image recognition and classification, and often the experiments are performed on MNIST or similar datasets (e.g. see &lt;a href=&quot;https://arxiv.org/pdf/1902.06494.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does such an implementation make sense if the labels have to be specially prepared in advance?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, you can prepare your dataset in advance, and then later train incrementally (in fact, in the experiments I have seen in some of these papers, they usually do this to simulate the continual learning scenario), but I am not sure about the optimality of this approach. Maybe with batch learning (i.e. the usual offline learning where you train on all data), you would achieve higher performance.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-10T12:48:38.913" LastActivityDate="2020-11-10T12:48:38.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24530" PostTypeId="2" ParentId="14047" CreationDate="2020-11-10T11:45:40.920" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/14306/2444&quot;&gt;This answer&lt;/a&gt; already mentions some of the approaches. More concretely, currently, the most common approaches to continual learning (i.e. learning with progressively more data while attempting to address the &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;&lt;em&gt;catastrophic forgetting&lt;/em&gt; problem&lt;/a&gt;) are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;dynamic/changing topologies&lt;/strong&gt; approaches&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;regularization&lt;/strong&gt; approaches&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/23832/2444&quot;&gt;&lt;strong&gt;rehearsal&lt;/strong&gt;&lt;/a&gt; (or &lt;strong&gt;pseudo-rehearsal&lt;/strong&gt;) approaches&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;ensemble&lt;/strong&gt; approaches&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;hybrid&lt;/strong&gt; approaches&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You can also take a look at &lt;a href=&quot;https://ai.stackexchange.com/a/24529/2444&quot;&gt;this answer&lt;/a&gt;. If you are interested in an exhaustive overview of the &lt;em&gt;state-of-the-art&lt;/em&gt; (at least, until 2019), you should read the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608019300231&quot; rel=&quot;nofollow noreferrer&quot;&gt;Continual lifelong learning with neural networks: A review&lt;/a&gt; (2019, by Parisi et al.).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-10T11:53:07.663" LastActivityDate="2020-11-10T11:53:07.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24532" PostTypeId="5" CreationDate="2020-11-10T13:19:11.013" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-10T13:19:11.013" LastActivityDate="2020-11-10T13:19:11.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24533" PostTypeId="4" CreationDate="2020-11-10T13:19:11.013" Score="0" Body="For questions about the &quot;Learning without Forgetting&quot; (LwF) continual learning approach introduced in the paper &quot;Learning without Forgetting&quot; (2016) by Zhizhong Li and Derek Hoiem." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-10T13:19:11.013" LastActivityDate="2020-11-10T13:19:11.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24535" PostTypeId="1" CreationDate="2020-11-10T14:44:04.217" Score="1" ViewCount="48" Body="&lt;p&gt;I am currently working with a categorical-binary RBM, where there are 50 categorical visible units and 25 binary hidden units. The categorical visible units are expressed in one-hot encoding format, such that if there is 5 categories, then the visible units are expressed as a &lt;span class=&quot;math-container&quot;&gt;$50 \times 5$&lt;/span&gt; array, where each row is the one-hot encoding of a category from 1 to 5.&lt;/p&gt;&#xA;&lt;p&gt;Ideally, the RBM should be able to reconstruct the visible units. However, since the visible units are in one-hot encoding, then the visible units array contains a lot of zeros. This means the RBM quickly learns to guess all zeros for the entire array to minimize the reconstruction loss. How can I force the RBM to not do this and to instead guess 1's where the category occurs and 0's otherwise?&lt;/p&gt;&#xA;&lt;p&gt;Note that I would still have this problem with a regular autoencoder.&lt;/p&gt;&#xA;" OwnerUserId="41856" LastEditorUserId="41856" LastEditDate="2020-11-10T16:25:16.360" LastActivityDate="2020-11-10T16:25:16.360" Title="How can I reconstruct sparse one-hot encodings using an RBM?" Tags="&lt;overfitting&gt;&lt;variational-autoencoder&gt;&lt;restricted-boltzmann-machine&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24536" PostTypeId="1" CreationDate="2020-11-10T17:13:34.173" Score="0" ViewCount="9" Body="&lt;p&gt;I have a problem. i have designed a network that takes in as input a segment of the output vector. The problem is, depending on the eigenvectors of the weight matrix, the series converges to a constant output. I need to generate unique outputs at each stage.&lt;/p&gt;&#xA;&lt;p&gt;In the network i form a binary vector with the top 10 % values of the output vector as 1's and the rest 0's. So i tried multiplying each output of the hidden layer by the elemntwise product with the input vector, so that i remove the outputs of 90%of the neurons by dropping them out. Then i have a unique topology to process each input to produce an output.&lt;/p&gt;&#xA;&lt;p&gt;when i do this the 4 decimal place onwards becomes unique. So i tried using only that section as input after multiplying it by 10. but the problem is that the sign of the number stays the same.&lt;/p&gt;&#xA;&lt;p&gt;How should i be doing this?&lt;/p&gt;&#xA;" OwnerUserId="41824" LastActivityDate="2020-11-10T17:13:34.173" Title="Cyclic network's output converges to a constant" Tags="&lt;neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24538" PostTypeId="1" CreationDate="2020-11-10T20:04:00.163" Score="1" ViewCount="24" Body="&lt;p&gt;I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy loss on 21 classes 0 - 20. You can find the code here: &lt;a href=&quot;https://github.com/parthv21/VOC-Semantic-Segmentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/parthv21/VOC-Semantic-Segmentation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My Architecture:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   self.encoder = nn.Sequential(&#xA;        nn.Conv2d(3, 64, 3, stride=2, padding=1),&#xA;        nn.LeakyReLU(),&#xA;        nn.Conv2d(64, 128, 3, stride=2, padding=1),&#xA;        nn.LeakyReLU(),&#xA;        nn.Conv2d(128, 256, 3, stride=2, padding=1),&#xA;        nn.LeakyReLU(),&#xA;        nn.Conv2d(256, 512, 3, stride=2, padding=1),&#xA;    )&#xA;&#xA;    self.decorder = nn.Sequential(&#xA;        nn.ConvTranspose2d(512, 256, 3, stride=2, padding=1, output_padding=1),&#xA;        nn.LeakyReLU(),&#xA;        nn.ConvTranspose2d(256, 128, 3, stride=2, padding=1, output_padding=1),&#xA;        nn.LeakyReLU(),&#xA;        nn.ConvTranspose2d(128, 64, 3, stride=2, padding=1, output_padding=1),&#xA;        nn.LeakyReLU(),&#xA;        nn.ConvTranspose2d(64, 21, 3, stride=2, padding=1, output_padding=1),&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After 200 iterations I am getting the following output&lt;/p&gt;&#xA;&lt;h3&gt;Training Data&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0OH2i.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0OH2i.jpg&quot; alt=&quot;Segmentation output on training data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Validation Data&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fzs9V.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fzs9V.jpg&quot; alt=&quot;Segmentation output on validation data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is a more complex architecture the only way I can fix this problem? Or can I fix this with a different loss function like dice or more regularization? The same issue happened after training for 100 iterations. So the model is not generalizing for some reason.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also tried adding weights to &lt;code&gt;CrossEntropy&lt;/code&gt; such that w_label = 1 - frequency(label). The idea was that 0 label for the background which was more common would contribute less to the loss, and other labels which were rare, would contribute more to the loss. But that did not help:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RLm0j.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RLm0j.jpg&quot; alt=&quot;weighted cross-entropy loss&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another thing I tried was ignoring label 0 for background in the loss. But that created horrible results even for training data:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O7grn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O7grn.jpg&quot; alt=&quot;ignoring background label in cross entropy loss&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42204" LastEditorUserId="2444" LastEditDate="2020-11-12T10:51:21.893" LastActivityDate="2020-11-12T10:51:21.893" Title="How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;autoencoders&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24539" PostTypeId="1" CreationDate="2020-11-11T03:03:33.030" Score="2" ViewCount="46" Body="&lt;p&gt;I recently came across the &lt;code&gt;featuretools&lt;/code&gt; package, which facilitates automated feature engineering.  Here's an explanation of the package:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://towardsdatascience.com/automated-feature-engineering-in-python-99baf11cc219&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/automated-feature-engineering-in-python-99baf11cc219&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Automated feature engineering aims to help the data scientist by&#xA;automatically creating many candidate features out of a dataset from&#xA;which the best can be selected and used for training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I only have limited experience with ML/AI techniques, but general AI is something that I'd been thinking about for a while before exploring existing ML techniques.  One idea that kept popping up was the idea of analyzing not just raw data for patterns but derivatives of data, not unlike what &lt;code&gt;featuretools&lt;/code&gt; can do.  Here's an example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3ondk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3ondk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's not especially difficult to see the figure above as two squares, one that is entirely green and one with a blue/green horizontal gradient.  This is true despite the fact that the gradient square is not any one color and its edge is the same color as the green square (i.e., there is no hard boundary).&lt;/p&gt;&#xA;&lt;p&gt;However, let's say that we calculate the difference between each pixel and the pixel to its immediate left.  Ignoring for a moment that RGB is 3 separate values, let's call the difference between each pixel column in the gradient square &lt;code&gt;X&lt;/code&gt;.  The original figure is then transformed into this, essentially two homogenous blocks of values.  We could take it one step further to identify a hard boundary (applying a similar left-to-right transformation again) between the two squares.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JNqGb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JNqGb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Once a transformation is performed, there should be some way to assess the significance of the transformation output.  This is a simple and clean example where there are two blocks of homogenous values (i.e., the output is clearly not random).  If it's true that our minds use any kind of similar transformation process, the number of transformations that we perform would likely be practically countless, even in brief instances of perception.&lt;/p&gt;&#xA;&lt;p&gt;Ultimately, this transformation process could facilitate finding the existence of order in data.  Within this framework, perhaps &amp;quot;intelligence&amp;quot; could be defined simply as the ability to detect order, which could require applying many transformations in a row, a wide variety of types of transformations, an ability to apply transformations with a high probability of finding something significant, an ability to assess significance, etc.&lt;/p&gt;&#xA;&lt;p&gt;Just curious if anyone has thoughts on this, if there are similar ideas out there beyond simple automated feature engineering, etc.&lt;/p&gt;&#xA;" OwnerUserId="30154" LastActivityDate="2020-11-11T10:49:04.567" Title="Is automated feature engineering a path to general AI?" Tags="&lt;superintelligence&gt;&lt;feature-selection&gt;&lt;feature-engineering&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24541" PostTypeId="1" CreationDate="2020-11-11T07:33:04.577" Score="0" ViewCount="36" Body="&lt;p&gt;I try to replicate the results of &lt;a href=&quot;https://link.springer.com/article/10.1007/s11263-015-0872-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-pooling) as feature vectors.&lt;/p&gt;&#xA;&lt;p&gt;To configure the model accordingly i do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from tensorflow.keras.models import Model&#xA;from tensorflow.keras.applications import VGG16&#xA;    &#xA;base_model = VGG16(include_top=False)  # Cut off the fully-connected-layers&#xA;model = Model(inputs=base_model.input, outputs=base_model.get_layer('block5_conv3').output)  # Discard the last max-pooling-layer&#xA;model.layers[-1].activation = None  # Change activation-function of last layer from Relu to Linear&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and i get a feature-tensor of shape &lt;code&gt;(1, 14, 14, 512)&lt;/code&gt; for the default input-shape of &lt;code&gt;(224, 224, 3)&lt;/code&gt;. So far, this looks the way i would want it to be. However, the authors state:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The VGG-M &lt;em&gt;[equivalent to VGG16 in the context of the paper]&lt;/em&gt; convolutional features are extracted as the output of the last  convolutional layer, directly from the linear filters excluding ReLU  and max pooling, which &lt;strong&gt;yields a field of 512-dimensional descriptor  vectors&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now, i have stated above, that the number 512 is part of my output-shape. However i thought, that it means that i get back 512 individual image-patches of size 14x14! The only way i could think of, that would get me 512-dimensional descriptor vectors would be something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;features = model.predict(img)&#xA;    &#xA;feature_vectors = []&#xA;    for i in range(features.shape[1]):&#xA;        for j in range(features.shape[2]):&#xA;            feature_vectors.append(features[0, i, j, :])&#xA;feature_vectors = np.array(feature_vectors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But then i would slice through all of the existing image-patches!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Did the authors mean to do just that? Is this a common practice anyone her has used before? All the tutorials or blogposts i found online just &lt;code&gt;flatten()&lt;/code&gt; the output-tensor and add it to the database of existing feature-vectors.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The authors also state, that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;...local descriptors are extracted at multiple scales, obtained by rescaling the image by factors &lt;span class=&quot;math-container&quot;&gt;$2^s, s=−3,−2.5,\dots,1.5$&lt;/span&gt; (but, for efficiency, discarding scales that would make the image larger than &lt;span class=&quot;math-container&quot;&gt;$1024^2$&lt;/span&gt; pixels).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I can totally extract images at different scales, by specifying the &lt;code&gt;input_shape&lt;/code&gt;, when instantiating the VGG16-model. My method of getting 512-dimensional feature-vectors stated above would also work in that case, even though the output-tensor could be much larger (i.e. a shape of &lt;code&gt;(1, 64, 64, 512)&lt;/code&gt;) in case of a bigger image.&lt;/p&gt;&#xA;&lt;p&gt;Is this the way to do feature-extraction at multiple scales?&lt;/p&gt;&#xA;" OwnerUserId="31589" LastActivityDate="2021-04-11T09:07:15.690" Title="Extract Features at Multiple Image-Scales" Tags="&lt;keras&gt;&lt;image-processing&gt;&lt;feature-extraction&gt;&lt;pretrained-models&gt;&lt;vgg&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24543" PostTypeId="1" CreationDate="2020-11-11T08:29:58.627" Score="3" ViewCount="118" Body="&lt;p&gt;While reading the original paper of &lt;a href=&quot;https://arxiv.org/pdf/1801.01290.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Soft Actor Critic&lt;/a&gt;, I came across on page number 5, under equation (5) and (6)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_{V}(\psi)=\mathbb{E}_{\mathbf{s}_{t} \sim \mathcal{D}}\left[\frac{1}{2}\left(V_{\psi}\left(\mathbf{s}_{t}\right)-\mathbb{E}_{\mathbf{a}_{t} \sim \pi_{\phi}}\left[Q_{\theta}\left(\mathbf{s}_{t}, \mathbf{a}_{t}\right)-\log \pi_{\phi}\left(\mathbf{a}_{t} \mid \mathbf{s}_{t}\right)\right]\right)^{2}\right]&#xA;\tag{5}\label{5}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\hat{\nabla}_{\psi} J_{V}(\psi)=\nabla_{\psi} V_{\psi}\left(\mathbf{s}_{t}\right)\left(V_{\psi}\left(\mathbf{s}_{t}\right)-Q_{\theta}\left(\mathbf{s}_{t}, \mathbf{a}_{t}\right)+\log \pi_{\phi}\left(\mathbf{a}_{t} \mid \mathbf{s}_{t}\right)\right)&#xA;\tag{6}\label{6}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The following quote:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;where the actions are sampled according to the current policy, instead of the replay buffer&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the context of deriving the formulation of the (estimated) gradient for the &lt;strong&gt;value function square residual error&lt;/strong&gt; (Equation 5 in the paper)&lt;/p&gt;&#xA;&lt;p&gt;I'm having a hard time understanding &lt;em&gt;why&lt;/em&gt; they use the action sampled from the current policy instead of the replay buffer. My intuition tells me that this is because SAC is an &lt;em&gt;off policy&lt;/em&gt; Reinforcement Learning algorithm, and Q-learning uses &lt;span class=&quot;math-container&quot;&gt;$\max Q$&lt;/span&gt; in one-step Q-value function update (to keep it off-policy), but why would sampling &lt;em&gt;one&lt;/em&gt; action from the &lt;em&gt;current policy&lt;/em&gt; still make it &lt;em&gt;off-policy&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I first asked a friend of mine (researcher in RL) and the answer I got was&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;If the action is sampled with the current policy given any state the update is on-policy.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I've checked &lt;a href=&quot;https://spinningup.openai.com/en/latest/algorithms/sac.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpinningUpRL by OpenAI's explanation of SAC&lt;/a&gt; but they only make it more clear &lt;em&gt;which&lt;/em&gt; action is sampled from the current policy, and which one is from the replay buffer, but does not specify &lt;em&gt;why&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Does this have anything to do with the stochastic policy? Or the entropy term in the update equation?&lt;/p&gt;&#xA;&lt;p&gt;So I'm still quite confused. Link/references to explanation are also appreciated!&lt;/p&gt;&#xA;" OwnerUserId="27559" LastEditorUserId="2444" LastEditDate="2020-11-23T01:43:03.433" LastActivityDate="2020-11-23T01:43:03.433" Title="In Soft Actor Critic, why is the action sampled from current policy instead of replay buffer on value function update?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;value-functions&gt;&lt;off-policy-methods&gt;&lt;soft-actor-critic&gt;" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="24544" PostTypeId="2" ParentId="24539" CreationDate="2020-11-11T10:49:04.567" Score="1" Body="&lt;p&gt;Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are &lt;em&gt;meaningful&lt;/em&gt;, as opposed to those that simply exist with no utility, needs some guidance.&lt;/p&gt;&#xA;&lt;p&gt;In machine learning, feature engineering is typically a search for features that improve performance at a specific task, such as classification or regression. The &amp;quot;intelligence&amp;quot; is partially in the manner of searching, and in setting the goals in the first place. Automated feature engineering typically uses fairly crude search algorithms to look for good features, such as random combination, and follows up with raw processing power in order to cover large numbers of options. Automated feature engineering also does not set the goals for any task, or feed back active behaviour from the output of ML to direct the search. An intelligent agent might actively search for data in its environment in order to test an idea (e.g. repeat an action in order to discover whether the same thing happens, or move in order to observe an interesting event better). Feature engineering is a separate issue to this.&lt;/p&gt;&#xA;&lt;p&gt;There are some theories of general intelligence that are focused on smart pattern matching as a key component. For instance, &lt;a href=&quot;https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jürgen Schmidhuber&lt;/a&gt; has long been a proponent of the reward system for intelligence being compression of observations and predictions. In such a system, better pattern matching discovered by an agent is an intrinsic reward signal as it allows for better compression of world models used by the agent.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hutter1.net/ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Marcus Hutter&lt;/a&gt; is another well-known AI researcher, who has proposed a framework called &lt;a href=&quot;https://en.wikipedia.org/wiki/AIXI&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIXI&lt;/a&gt; which incorporates similar ideas. An agent operating using AIXI will benefit from discovering features in its observations that improve its own predictions of what will happen next. Some form of automated feature engineering could well be core to such an agent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-11T10:49:04.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24545" PostTypeId="1" CreationDate="2020-11-11T11:00:31.907" Score="2" ViewCount="25" Body="&lt;p&gt;... and how do I reword my question in the title?&lt;/p&gt;&#xA;&lt;p&gt;I have a dataset where each &amp;quot;instance&amp;quot; has a &amp;quot;series&amp;quot; of multiple photos taken from different angles. I need to classify each instance as a 0 or a 1.&lt;/p&gt;&#xA;&lt;p&gt;A little over half of the images in each series probably do not contain the information required for a classification. Only some of the images are taken from an angle where the relevant clue is visible.&lt;/p&gt;&#xA;&lt;p&gt;For training I have many such series and they are labelled at a series level, but not at an image level.&lt;/p&gt;&#xA;&lt;p&gt;My current approach is to use a standard architecture like ResNet. I pass each image through the CNN then I combine the features by averaging, then put that through a sigmoid activated layer. I'm concerned that the network won't be able to learn because the &amp;quot;clue&amp;quot; is so buried among everything else.&lt;/p&gt;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a better/standard way to do this? Would going RNN help? What if the images are not really in a  meaningful sequence?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If my way is good, is arithmetic averaging the right way to combine the features?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Would it be worth spending the time to label each image as &amp;quot;has positive clue&amp;quot;/&amp;quot;does not have positive clue&amp;quot;? Should I add a &amp;quot;not possible to tell&amp;quot;? What if it is possible to tell but it's just humans that can't tell?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2020-11-11T11:12:28.653" LastActivityDate="2020-11-11T11:12:28.653" Title="What are ways to learn a classifier for labelling a series of images rather than individual images?" Tags="&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24548" PostTypeId="1" CreationDate="2020-11-11T16:48:15.200" Score="2" ViewCount="64" Body="&lt;p&gt;I have a question regarding the loss function of target networks and current (online) networks. I understand the action value function. What I am unsure about is why we seek to minimise the loss between the &lt;code&gt;qVal&lt;/code&gt; for the next state in our target network and the &lt;strong&gt;current&lt;/strong&gt; state in the local network. The Nature paper by Mnih et al. is well explained, however, I am not getting from it the purpose of the above. Here is my training portion from a script I am running:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for state, action, reward, next_state, done in minibatch:&#xA;    target_qVal = self.model.predict(state)&#xA;&#xA;    # print(target_qVal)&#xA;&#xA;    if done:&#xA;        target_qVal[0][action] = reward #done&#xA;    else:&#xA;        # predicted q value for next state from target model&#xA;        pred = self.target_model.predict(next_state)[0]&#xA;        target_qVal[0][action] = reward + self.gamma * np.amax(pred)&#xA;&#xA;    # indentation position?&#xA;    self.model.fit(np.array(state), &#xA;                   np.array(target_qVal), &#xA;                   batch_size=batch_size,&#xA;                   verbose=0, &#xA;                   shuffle=False, &#xA;                   epochs=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I understand that the expected return is the immediate reward plus the cumulative sum of discounted rewards looking into the future &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; (correct me if I'm wrong in my understanding) when following a given policy.&lt;/p&gt;&#xA;&lt;p&gt;My fundamental misunderstanding is the loss equation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L = [r + \gamma \max Q(s',a'; \theta') - Q(s,a; \theta)],$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\theta'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the weights of the target and online neural networks, respectively.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Why do we aim to minimize the Q value of the next state in the target model and the Q value of the current state in the online model?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;A bonus question would be, in order to collect &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; values for dimensionality reduction (as in Mnih et al t-sne plot), would I simply collect the &lt;code&gt;target_qVal[0]&lt;/code&gt; values during training and feed them into a list after each step to accumulate the Q values over time?&lt;/p&gt;&#xA;" OwnerUserId="34530" LastEditorUserId="2444" LastEditDate="2020-11-12T10:46:57.150" LastActivityDate="2021-04-11T11:07:40.707" Title="Why do we minimise the loss between the target Q values and 'local' Q values?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24550" PostTypeId="1" CreationDate="2020-11-11T17:27:09.117" Score="1" ViewCount="66" Body="&lt;p&gt;In GAN (generative adversarial networks), let us take &amp;quot;binary cross-entropy&amp;quot; as the loss function for discriminator &lt;span class=&quot;math-container&quot;&gt;$$(overall \; loss = -\sum log(D(x_i)) -\sum log(1-D(G(z_i))) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ where \; x_i = real \; image \; pixel \; matrix$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ and \; z_i = a \; vector \; from \; latent \; space$$&lt;/span&gt;.&#xA;Let us define discriminator real loss and fake loss:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ d_{fake \; loss} = -\sum log(1-D(G(Z)))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ d_{real \; loss} = -\sum log(D(x))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ d_{fake \; loss} \; implies \; discriminator \; loss \; against \; fake \; images$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ d_{real \; loss} \; implies \; discriminator \; loss \; against \; real \; images$$&lt;/span&gt;&#xA;Generator Loss :&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ g_{loss} = -\sum log(D(G(z_i)))$$&lt;/span&gt;&#xA;Since the functions are similar, we should be expecting some similarity in graphical patterns (i.e since none of the functions are inherently oscillatory, I expect that if one comes out to be oscillatory, the other one should be the same as well). But, If you refer to chapter 10 of the book &amp;quot;Generative Adversarial Networks with python by Jason Brownlee&amp;quot;, we find some difference. The following are the graphs published in the book&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KG6O1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KG6O1.png&quot; alt=&quot;Line plots for loss and accuracy for a stable GAN (1st GRAPH plots the losses)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5yPOF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5yPOF.png&quot; alt=&quot;Line plots for loss and accuracy for a stable GAN with mode collapse&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone explain the difference in the plots between discriminator fake loss and generator loss (mathematically)?&lt;/p&gt;&#xA;" OwnerUserId="42181" LastEditorUserId="42181" LastEditDate="2020-11-12T14:25:28.433" LastActivityDate="2021-04-12T11:03:30.273" Title="Explain the difference in graphical patterns between discriminator fake loss and generator loss in GAN" Tags="&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24551" PostTypeId="1" AcceptedAnswerId="24560" CreationDate="2020-11-11T19:18:00.473" Score="0" ViewCount="27" Body="&lt;p&gt;Suppose we have the fuzzy membership grade for a person &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; with a set &lt;span class=&quot;math-container&quot;&gt;$S = \text{set of tall people}$&lt;/span&gt; be &lt;span class=&quot;math-container&quot;&gt;$0.9$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mu_S(x)=0.9$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Does this mean that the probability of person &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; being tall is &lt;span class=&quot;math-container&quot;&gt;$0.9$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="42230" LastEditorUserId="2444" LastEditDate="2020-11-12T10:35:14.123" LastActivityDate="2020-11-12T10:35:14.123" Title="How to calculate probability from fuzzy membership grade?" Tags="&lt;probability&gt;&lt;fuzzy-logic&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24552" PostTypeId="2" ParentId="24494" CreationDate="2020-11-11T19:41:04.907" Score="1" Body="&lt;p&gt;It depends on the stopping condition.  If the stopping condition is &amp;quot;stop as soon as any vertex is encountered by both the forward and backward scan&amp;quot;, then bidirectional uniform-cost search is not a correct algorithm -- it is not guaranteed to output the optimal path.  But it is possible to adjust the stopping condition to make bidirectional uniform-cost search guaranteed to output an optimal solution.&lt;/p&gt;&#xA;&lt;p&gt;See the following resources for details, and the correct stopping condition:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.85.911&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing Point-to-Point Shortest Paths from External Memory&lt;/a&gt;.  Andrew V. Goldberg, Renato F. Werneck.  ALENEX/ANALCO 2005.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/goldberg-sofsem07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Point-to-point shortest path algorithms with preprocessing&lt;/a&gt;.  Andrew V. Goldberg.  International Conference on Current Trends in Theory and Practice of Computer Science, 2007.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.princeton.edu/courses/archive/spr06/cos423/Handouts/EPP%20shortest%20path%20algorithms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Efficient Point-to-Point Shortest Path Algorithms&lt;/a&gt;.  Andrew V. Goldberg, Chris Harrelson, Haim Kaplan, Renato F. Wemeck.&lt;/p&gt;&#xA;&lt;p&gt;I found these resources by looking at the Wikipedia article on &lt;a href=&quot;https://en.wikipedia.org/wiki/Bidirectional_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;bidirectional search&lt;/a&gt;; it mentions that the termination condition has been articulated by Andrew Goldberg et al and cites the third reference above.  Then a quick search on Google Scholar immediately turned up the other papers as well.&lt;/p&gt;&#xA;&lt;p&gt;Lesson for the future: It can be useful to spend a little time checking standard resources (such as Wikipedia and textbooks), and checking the literature (e.g., with Google Scholar).  Many natural questions have already been answered in the literature.&lt;/p&gt;&#xA;" OwnerUserId="1794" LastEditorUserId="1794" LastEditDate="2020-11-11T19:53:33.783" LastActivityDate="2020-11-11T19:53:33.783" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24553" PostTypeId="1" CreationDate="2020-11-11T20:50:49.077" Score="0" ViewCount="15" Body="&lt;p&gt;I'm approaching at Siamese Networks in order to use them for Image Similarity. I found that many people use famous models like VGG or ResNet to build the vectors that will go on the distance layer in order to be compared with the other vector image and then they use contrastive loss. Then I found that other people added some other layers after the distance one in order to get in output with a sigmoid a similarity label (0,1 or discretized).&lt;/p&gt;&#xA;&lt;p&gt;Is this final processing useful? These networks are born in order to work with the distance vector right?&lt;/p&gt;&#xA;&lt;p&gt;So, I'm trying to understand the main differences between the two approaches, if any.&lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2020-11-12T10:33:58.937" LastActivityDate="2020-11-12T10:33:58.937" Title="Does a Siamese Network need other trainable layers after the distance layer?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;siamese-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24554" PostTypeId="1" CreationDate="2020-11-12T00:32:33.493" Score="0" ViewCount="32" Body="&lt;p&gt;Here is the formula of a radial basis kernel&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k\left(x_{i}, x_{j}\right)=\exp \left(-\frac{d\left(x_{i}, x_{j}\right)^{2}}{2 l^{2}}\right),&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; are feature vectors.&lt;/p&gt;&#xA;&lt;p&gt;I have two very related questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Given two matrices (that represent images), how does one extract features from it? Would each value in the matrix (every pixel) represent a single feature?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once I figure out how to extract the features, how do I use the formula above? Should I sum the &lt;span class=&quot;math-container&quot;&gt;$k(x_i, x_j)$&lt;/span&gt; values for each feature?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2020-11-12T10:30:24.297" LastActivityDate="2020-11-12T10:30:24.297" Title="How does one measure image similarity using Radial Basis Kernel?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24555" PostTypeId="1" CreationDate="2020-11-12T01:24:51.900" Score="0" ViewCount="22" Body="&lt;p&gt;I've been trying to learn about CNN's and reinforcement learning and I found this project to play with: &lt;a href=&quot;https://github.com/adityajn105/flappy-bird-deep-q-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/adityajn105/flappy-bird-deep-q-learning&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to change the code to work with RGB input instead of grayscale. Pre-precosessing part is fine, but I'm having a problem with &lt;em&gt;state&lt;/em&gt; and &lt;em&gt;next_state&lt;/em&gt; I guess because they're deque and when deque is appended shape is (4,H,W) because it's appended 4 times (4 frames). Problem I'm having is when I append frames to deque which are RGB it becomes something like (4,H,W,3). I tried some stuff that came to mind and that I googled and read about online, but I still had problems with dimensions. What should be done so that it works with RGB instead of grayscale?&lt;/p&gt;&#xA;" OwnerUserId="42233" LastActivityDate="2020-11-12T04:04:23.093" Title="DQN rgb input channels problem using pytorch" Tags="&lt;convolutional-neural-networks&gt;&lt;dqn&gt;&lt;double-dqn&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24557" PostTypeId="2" ParentId="24548" CreationDate="2020-11-12T03:45:12.613" Score="1" Body="&lt;p&gt;The loss function is designed in a way to approximate the bellman optimality for &lt;span class=&quot;math-container&quot;&gt;$Q^*(s,a)$&lt;/span&gt;. Given an optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Q^*(s,a)$&lt;/span&gt; satisfies the equation &lt;span class=&quot;math-container&quot;&gt;$$Q^*(s,a) = r(s) + \gamma max_{a'}\sum_{s'}P(s'|s,a)Q^*(s',a')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;At convergence, the highest &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value that I can get taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is equal to the reward I get for taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and the max &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values at the next state.&lt;/p&gt;&#xA;&lt;p&gt;You can see in the loss function that the DQN tries to attain a &lt;span class=&quot;math-container&quot;&gt;$Q^*(s,a)$&lt;/span&gt; value that closely approximates the equation above.&lt;/p&gt;&#xA;&lt;p&gt;On a side note, in model based RL setting, the bellman optimality for &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt; is expressed as &lt;span class=&quot;math-container&quot;&gt;$$Q^*(s,a) = r(s) + \gamma \sum_{s'}P(s'|s,a)V^*(s)$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$V^*(s')$&lt;/span&gt; is used in the model based case because &lt;span class=&quot;math-container&quot;&gt;$V^*(s)$&lt;/span&gt; by definition represents the highest possible value attainable at state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; following &lt;span class=&quot;math-container&quot;&gt;$\pi^*$&lt;/span&gt;. In model free setting, &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; is replaced by &lt;span class=&quot;math-container&quot;&gt;$Q^*$&lt;/span&gt; because computing &lt;span class=&quot;math-container&quot;&gt;$V^*$&lt;/span&gt; is not useful in achieving model free control without a transition model &lt;span class=&quot;math-container&quot;&gt;$P(s'|s,a)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-11-12T03:45:12.613" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24558" PostTypeId="2" ParentId="24555" CreationDate="2020-11-12T04:04:23.093" Score="1" Body="&lt;p&gt;You can reshape it to (12, H, W) using NumPy reshape function. By the way, this will only increase the complexity of this problem. If you want to practice RL then just get the idea from their code and try implementing on some other problem/game.&lt;/p&gt;&#xA;" OwnerUserId="42234" LastActivityDate="2020-11-12T04:04:23.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24559" PostTypeId="2" ParentId="24541" CreationDate="2020-11-12T08:13:26.050" Score="0" Body="&lt;p&gt;Not sure I have understood well your second question but I am gonna try to see if I can help you.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yes, the authors meant just that. I see were the confusion might come. So:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The authors say: &lt;em&gt;&amp;quot;[...] yields a field of 512-dimensional descriptor vectors&amp;quot;&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;You say: &lt;em&gt;&amp;quot;[...] i get back 512 individual image-patches of size 14x14&amp;quot;&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;You get: &lt;em&gt;&amp;quot;i get a feature-tensor of shape &lt;code&gt;(1, 14, 14, 512)&lt;/code&gt;&amp;quot;&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So when the authors say a &lt;em&gt;field of K-dimensional vector&lt;/em&gt;, with the word &lt;em&gt;field&lt;/em&gt; they implicitly say that is a 2D vector (so &lt;span class=&quot;math-container&quot;&gt;$[H,W]$&lt;/span&gt;) of K dimensions, so &lt;span class=&quot;math-container&quot;&gt;$[H,W,K]$&lt;/span&gt; which is exactly what your are getting (only with the batch dimension first, that Keras ignores in its input, that's why I prefer pytorch, it is more explicit).&lt;/p&gt;&#xA;&lt;p&gt;So being explicit, from an input image &lt;span class=&quot;math-container&quot;&gt;$I:[B,H,W,C]=[1, 224, 224, 3]$&lt;/span&gt; you get a field (or patch, or feature) vector, a 2D vector, of &lt;span class=&quot;math-container&quot;&gt;$K=512$&lt;/span&gt; dimensions (or features, or descriptors) so &lt;span class=&quot;math-container&quot;&gt;$F:[B,H_f, W_f, K]=[1,14, 14, 512]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What you are doing in your snippet is taking 1D vectors of &lt;span class=&quot;math-container&quot;&gt;$K=512$&lt;/span&gt; dimensions, so you are extracting K-dimensional vectors from a K-dimensional feature map (or patch or field).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;flatten()&lt;/code&gt; goes in the second question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are a lot of ways of doing feature-extraction at multiple scales. Say for example you extract features at 3 levels so you get:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_1:[H_1, W_1, C_1]=[56, 56, 100]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_2:[H_2, W_2, C_2]=[26, 26, 200]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_3:[H_3, W_3, C_3]=[13, 13, 300]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This vectors are not concatenable. You indeed could &lt;code&gt;flatten()&lt;/code&gt; them over the &lt;span class=&quot;math-container&quot;&gt;$[H,W]$&lt;/span&gt; dimensions and concat over the channel dimension. But you would loose the 2D spatial relationship of images. What is normally done (from FPN, Retinanet EfficientNet, EfficientDet...) is the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You extract the previous feature maps at different scales&lt;/li&gt;&#xA;&lt;li&gt;You process them over a FCN head of your network (so last dimension is bounding box coordinates or class probabilities)&lt;/li&gt;&#xA;&lt;li&gt;You concatenate over last dimension&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;After extracting &lt;span class=&quot;math-container&quot;&gt;$P_1, P_2, P_3$&lt;/span&gt; and passing by a FCN classification head on your network you would get (given you have 10 classes):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_1':[H_1, W_1, C_1]=[56, 56, 10]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_2':[H_2, W_2, C_2]=[26, 26, 10]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_3':[H_3, W_3, C_3]=[13, 13, 10]$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now you have a classification vector per each &amp;quot;pixel&amp;quot; on each feature dimension. So now it makes more sense to flatten them to get 1D vectors of 10 class probs:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$P_{concat}=[N, 10]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the flattened number of dimensions from &lt;span class=&quot;math-container&quot;&gt;$H,W$&lt;/span&gt; of each scale feature map&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you want the implementation details go to RetinaNet in papers with code. I would link one very nice implementation but it is in pytorch, and I see you use Keras.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-11-12T08:13:26.050" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24560" PostTypeId="2" ParentId="24551" CreationDate="2020-11-12T09:07:25.423" Score="3" Body="&lt;p&gt;No, you can't extract any probability from a fuzzy membership grade. The uncertainty expressed by fuzzy logic is about partial truth, not about probability. &lt;span class=&quot;math-container&quot;&gt;$ \mu_S(x) = 0.9 $&lt;/span&gt; doesn't mean that &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$ x $&lt;/span&gt; is tall&amp;quot; is true with a probability of 0.9, but that &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$ x $&lt;/span&gt; is tall&amp;quot; is 90% true (notice the difference in semantics). You have to think about fuzzy logic as an extension of logic (as its name implies), rather than an extension of probability.&lt;/p&gt;&#xA;&lt;p&gt;It's true, however, that fuzzy logic is flexible and lets you define how the membership grades are combined in logic formulae, to the extend you can replicate probability theory within the fuzzy logic framework. Wikipedia has a good overview on this: &lt;a href=&quot;https://en.wikipedia.org/wiki/Fuzzy_logic#Comparison_to_probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Fuzzy_logic#Comparison_to_probability&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, please understand that, in general, fuzzy membership &lt;span class=&quot;math-container&quot;&gt;$ \neq $&lt;/span&gt; probability. How we come up with the fuzzy membership grade is subjective and application-dependent. Conversely, probabilities have a well-defined and unambiguous interpretation. The point of being fuzzy is to replicate our reasoning process which, even if is not necessarily formal and rigorous, is often very accurate. To do so it needs a set of (admittedly arbitrary) rules on how to calculate the &amp;quot;truthfulness&amp;quot; of logical formulae. This may turn to be very useful in applications where manipulating probabilities or coming up with them in the first place is not tractable.&lt;/p&gt;&#xA;" OwnerUserId="37359" LastActivityDate="2020-11-12T09:07:25.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24561" PostTypeId="1" CreationDate="2020-11-12T11:14:47.310" Score="0" ViewCount="38" Body="&lt;p&gt;I am following a course on machine learning and am confused about the bias-variance trade-off relationship&#xA;to learning curves in classification.&lt;/p&gt;&#xA;&lt;p&gt;I am seeing some conflicting information online on this.&lt;/p&gt;&#xA;&lt;p&gt;The scikit-learn learning curve looks like the top 2 curves here: &lt;img src=&quot;https://scikit-learn.org/stable/_images/sphx_glr_plot_learning_curve_0011.png&quot; alt=&quot;Learning curve&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand is: how do we read bias from this? If we look at this image&#xA;where each blue dot is a model. I think the bias would be the green curve being high. But high bias&#xA;indicates underfitting, right? So shouldn't the red curve be high then too?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Ftse4.mm.bing.net%2Fth%3Fid%3DOIP.-pwwSpcPJcxzDaRH40w95QAAAA%26pid%3DApi&amp;amp;f=1&quot; alt=&quot;Bias variance tradeoff&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;High variance would be the gap between green and red, is this correct?&lt;/p&gt;&#xA;&lt;p&gt;My question is how do the red and green curves relate to underfitting and overfitting,&#xA;and how do learning curves fit with the figure with the concentric circles? Is bias purely related to the red curve, or is a model with a low validation score and high train score also a high bias model?&lt;/p&gt;&#xA;" OwnerUserId="42236" LastEditorUserId="42236" LastEditDate="2020-11-12T11:50:34.193" LastActivityDate="2020-11-12T11:50:34.193" Title="Bias-variance tradeoff and learning curves for non-deep learning models" Tags="&lt;bias-variance-tradeoff&gt;&lt;learning-curve&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24562" PostTypeId="2" ParentId="14047" CreationDate="2020-11-12T12:07:02.447" Score="2" Body="&lt;p&gt;There are lots of different approaches that try to avoid catastrophic forgetting in neural networks. It is impossible to summarize all contributions here.&lt;/p&gt;&#xA;&lt;p&gt;However, in addition to the already mentioned techniques, there are &lt;strong&gt;sparsity&lt;/strong&gt; approaches that try to disentangle internal representations of the network on different tasks or learning steps. Sparsity usually helps, but the network has to learn to use it, imposing a structural sparsity by construction is not enough. Also, you can leverage &lt;strong&gt;bayesian&lt;/strong&gt; approaches, through which you can associate a confidence measure to each of your weights and use this measure to mitigate forgetting. Also, &lt;strong&gt;meta-learning&lt;/strong&gt; can be employed to meta-learn a model which is robust to forgetting on different sequences of tasks.&lt;/p&gt;&#xA;&lt;p&gt;What I can suggest you in addition, is to take a look at &lt;a href=&quot;https://wiki.continualai.org/research.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ContinualAI wiki&lt;/a&gt;, which maintains a list of updated publications classified by the type of Continual Learning strategy and tagged with additional information. (&lt;strong&gt;Disclaimer&lt;/strong&gt;: I am a member of &lt;a href=&quot;https://www.continualai.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ContinualAI association&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="42237" LastEditorUserId="2444" LastEditDate="2020-12-12T12:02:12.357" LastActivityDate="2020-12-12T12:02:12.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24564" PostTypeId="1" AcceptedAnswerId="24566" CreationDate="2020-11-12T15:37:59.890" Score="1" ViewCount="52" Body="&lt;p&gt;For a lot of VAE implementations I've seen in code, it's not really obvious to me how it equates to ELBO.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(X)=H(Q)-H(Q:P(X,Z))=\sum_ZQ(Z)logP(Z,X)-\sum_ZQ(Z)log(Q(Z))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above is the definition of ELBO where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is some input, &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; is a latent variable, &lt;span class=&quot;math-container&quot;&gt;$H()$&lt;/span&gt; is the equation for entropy. &lt;span class=&quot;math-container&quot;&gt;$Q()$&lt;/span&gt; is a distribution being used to approximate distribution &lt;span class=&quot;math-container&quot;&gt;$P()$&lt;/span&gt;, which in the above case both &lt;span class=&quot;math-container&quot;&gt;$P()$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q()$&lt;/span&gt; are discrete distributions, because of the sum.&lt;/p&gt;&#xA;&lt;p&gt;A lot of the times when VAEs are built for reconstructing discrete data types, let's say for example an image, where each pixel can be black or white or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. The objective function written in code I always see is written as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Encoder(Y) \rightarrow Z_u, Z_{\sigma}$&lt;/span&gt;&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ReparamterizationTrick (Z_\mu, Z_\sigma) \rightarrow Z$&lt;/span&gt;&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$Decoder(Z) \rightarrow \hat{Y}$&lt;/span&gt;&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$L(Y)= CrossEntropy(\hat{Y}, Y) - 0.5*(1+Z_{\sigma}-Z_{\mu}^2-exp(Z_\sigma))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; represents the latent embedding of the AutoEncoder and &lt;span class=&quot;math-container&quot;&gt;$Z_\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Z_\sigma$&lt;/span&gt; represent the mean and standard deviation for sampling for &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; from a gaussian distribution. &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; represents the binary image trying to be reconstructed and &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}$&lt;/span&gt; represents it's reconstruction from the VAE. As we can see from ELBO it's the entropy of the latent distribution being learned &lt;span class=&quot;math-container&quot;&gt;$Q()$&lt;/span&gt;, which is gaussian as with VAE they usually are and the cross entropy of the latent distribution being learned &lt;span class=&quot;math-container&quot;&gt;$Q()$&lt;/span&gt; and the actual distribution &lt;span class=&quot;math-container&quot;&gt;$P()$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; intersected with &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The main points that confuse me is how the cross entropy(CE) of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}$&lt;/span&gt; equates to the CE of the distribution for generating latents and it's gaussian approximation and how &lt;span class=&quot;math-container&quot;&gt;$(0.5*(1+Z_{\sigma}-Z_{\mu}^2-exp(Z_\sigma)))$&lt;/span&gt; equates to the entropy? Is it just assumed the CE of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}$&lt;/span&gt; also leads to the CE of the latent distribution with it's approximation cause they're part of &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}$&lt;/span&gt;'s generation? Still seems a bit off cause you're getting the cross entropy of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; with it's reconstruction, not the gaussian distribution for learning latents &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note: &lt;span class=&quot;math-container&quot;&gt;$Z_\sigma$&lt;/span&gt; is usually not softplused to be strictly positive as required by a Gaussian dist so I think that's what &lt;span class=&quot;math-container&quot;&gt;$exp(Z_\sigma)$&lt;/span&gt; is for.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="30885" LastEditDate="2020-11-12T16:09:40.703" LastActivityDate="2020-11-13T13:36:44.960" Title="How does implementation of VAE's objective function equate to ELBO?" Tags="&lt;variational-autoencoder&gt;&lt;cross-entropy&gt;&lt;categorical-crossentropy&gt;&lt;evidence-lower-bound&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24565" PostTypeId="1" AcceptedAnswerId="24692" CreationDate="2020-11-12T16:34:40.303" Score="1" ViewCount="80" Body="&lt;p&gt;I have built my own RL environment, where a state is composed of two elements: the agent's position and a matrix of 0s and 1s (1 if a user has requested a service from the agent, 0 otherwise); an action is composed of 3 elements: the movement the agent chooses (up, down, left or right), a matrix of 0s and 1s (1 if a resource has been allocated to a user, 0 otherwise), and a vector representing the allocation of another type of resource (the vector contains the values allocated to the users).&lt;/p&gt;&#xA;&lt;p&gt;I am currently trying to build a Deep Q Learning agent, I am a bit confused however as to what model (example Sequential), what type of layers (example Dense layers), how many layers, what activation mode I should use, and what the state and action sizes are. (Taking this code as a reference &lt;a href=&quot;https://github.com/the-deep-learners/TensorFlow-LiveLessons/blob/master/notebooks/cartpole_dqn.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;cartpole dqn agent&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I also do not know what my inputs and outputs should be.&lt;/p&gt;&#xA;&lt;p&gt;The examples I have come across are rather simple and I don't know how to approach setting it all up for my agent.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastEditorUserId="42372" LastEditDate="2020-11-12T17:55:51.250" LastActivityDate="2020-11-18T11:56:19.257" Title="DQN layers when state space and action space are multi dimensional" Tags="&lt;deep-learning&gt;&lt;python&gt;&lt;keras&gt;&lt;deep-rl&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24566" PostTypeId="2" ParentId="24564" CreationDate="2020-11-12T22:15:09.657" Score="1" Body="&lt;p&gt;I don't want to think about the correctness of your supposed ELBO equation now. Nevertheless, it's true that the ELBO can be rewritten in different ways (e.g. if you expand the KL divergence below, by applying its definition, you will end up with a different but equivalent version of the ELBO). I will use the most common (and definitely most intuitive, at least to me) one, which you can find in &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper that originally introduced the VAE&lt;/a&gt;, which you should use as a reference, when you are confused (although that paper may require at least 2 readings before you fully understand it).&lt;/p&gt;&#xA;&lt;p&gt;Here's the most common form of the ELBO (for the VAE), which immediately explains the common VAE implementations (including yours):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}\left(\boldsymbol{\theta}, \boldsymbol{\phi} ; \mathbf{x}^{(i)}\right)=&#xA;\underbrace{&#xA;\color{red}{&#xA;-D_{K L}\left(q_{\boldsymbol{\phi}}\left(\mathbf{z} \mid \mathbf{x}^{(i)}\right) \| p_{\boldsymbol{\theta}}(\mathbf{z})\right)&#xA;}&#xA;}_{\text{KL divergence}}&#xA;+&#xA;\underbrace{&#xA;\color{green}{&#xA;\mathbb{E}_{q_{\boldsymbol{\phi}}\left(\mathbf{z} \mid \mathbf{x}^{(i)}\right)}\left[\color{blue}{ \log p_{\boldsymbol{\theta}}\left(\mathbf{x}^{(i)} \mid \mathbf{z}\right)}\right]&#xA;}&#xA;}_{\text{Expected log-likelihood}},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{\boldsymbol{\theta}}(\mathbf{z})$&lt;/span&gt; is the prior over &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; (i.e. the latent variable); &lt;strong&gt;in practice&lt;/strong&gt;, the prior is not actually parametrized by &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; (i.e. it's just a Gaussian with mean zero and variance one, or whatever, depending on your assumptions about &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt;!), but in the paper they assume that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; (see figure 1).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{\boldsymbol{\phi}}\left(\mathbf{z} \mid \mathbf{x}^{(i)}\right)$&lt;/span&gt; is the encoder parametrized by &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\phi}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{\boldsymbol{\theta}}$&lt;/span&gt; is the decoder, parametrized by &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you assume that &lt;span class=&quot;math-container&quot;&gt;$p_{\boldsymbol{\theta}}(\mathbf{z})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q_{\boldsymbol{\phi}}\left(\mathbf{z} \mid \mathbf{x}^{(i)}\right) $&lt;/span&gt; are Gaussian distributions, then it turns out that the KL divergence has an analytical form, which is also derived in the VAE paper (appendix B)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\color{red}{&#xA;-D_{K L}\left(q_{\boldsymbol{\phi}}\left(\mathbf{z} \mid \mathbf{x}^{(i)}\right) \| p_{\boldsymbol{\theta}}(\mathbf{z})\right)}&#xA;= &#xA;\color{red}{&#xA;\frac{1}{2} \sum_{j=1}^{J}\left(1+\log \left(\left(\sigma_{j}\right)^{2}\right)-\left(\mu_{j}\right)^{2}-\left(\sigma_{j}\right)^{2}\right)&#xA;}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hence &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/vae/main.py#L80&quot; rel=&quot;nofollow noreferrer&quot;&gt;this implementation&lt;/a&gt;, which should be equivalent to your term &lt;span class=&quot;math-container&quot;&gt;$0.5*(1+Z_{\sigma}-Z_{\mu}^2-exp(Z_\sigma))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The expected log-likelihood is actually an expectation, so you cannot compute it exactly, in general. Hence, you can approximate it with Monte Carlo sampling (aka sampling averages: remember the law of large numbers?). More concretely, if you assume that you have a Bernoulli likelihood, i.e. &lt;span class=&quot;math-container&quot;&gt;$p_{\boldsymbol{\theta}}\left(\mathbf{x}^{(i)} \mid \mathbf{z}\right)$&lt;/span&gt; is a Bernoulli, then its definitions is (again from the VAE paper, Appendix C.1)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\color{blue}{ \log p(\mathbf{x} \mid \mathbf{z})}= \color{blue}{ \sum_{i=1}^{D} x_{i} \log y_{i}+\left(1-x_{i}\right) \cdot \log \left(1-y_{i}\right) }\tag{1}\label{1},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; is the output of the decoder (i.e. the reconstruction/generation of the original input).&lt;/p&gt;&#xA;&lt;p&gt;This formula should be very familiar to you if you are familiar with the cross-entropy. In fact, minimizing the cross-entropy is equivalent to maximizing the log-likelihood (this may still be confusing because of the flipped signs in the ELBO above, but just remember that maximizing a function is equivalent to minimizing its negative!). Hence &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/vae/main.py#L73&quot; rel=&quot;nofollow noreferrer&quot;&gt;this loss&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To answer/address some of your questions/doubts directly&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;and how &lt;span class=&quot;math-container&quot;&gt;$(0.5*(1+Z_{\sigma}-Z_{\mu}^2-exp(Z_\sigma)))$&lt;/span&gt; equates to the entropy?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I answered this above. That's just the analytical expression for the KL divergence (by the way, the KL divergence is also known as &lt;em&gt;relative entropy&lt;/em&gt;: lol?): yes, but resolve me the logs and exponentials, though!&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Still seems a bit off cause you're getting the cross-entropy of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; with its reconstruction, not the Gaussian distribution for learning the latent &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As you can see from the definition of the ELBO above (from the VAE paper), the expected log-likelihood is, as the name suggests, an expectation, with respect to the encoder (i.e. the Gaussian, in case you choose a Gaussian). However, the equivalence is between the log-likelihood (which is the term &lt;strong&gt;inside&lt;/strong&gt; the expectation) and the cross-entropy, i.e. once you have sampled from the encoder, you just need to compute the term inside the expectation (i.e. the cross-entropy). Your term &lt;span class=&quot;math-container&quot;&gt;$CrossEntropy(\hat{Y}, Y)$&lt;/span&gt; represents the CE but &lt;strong&gt;after you have sampled a latent variable from the encoder (or Gaussian)&lt;/strong&gt;, otherwise, you could not have obtained the reconstruction &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}$&lt;/span&gt; (i.e. the reconstruction depends on this latent variable, see figure 1).&lt;/p&gt;&#xA;&lt;p&gt;In equation \ref{1}, note that there is no expectation. In fact, in the implementations, you may just sample &lt;strong&gt;once&lt;/strong&gt; from the encoder, and then immediately compute the ELBO. I have seen this also in the implementations of Bayesian neural networks (basically, normal neural networks with the same principles of the VAE). However, in principle, you could sample multiple times from the Gaussian encoder, compute \ref{1} multiple times, then average it, to compute a better approximation of the &lt;strong&gt;expected&lt;/strong&gt; log-likelihood.&lt;/p&gt;&#xA;&lt;p&gt;Hopefully, some of the information in this answer is clear. Honestly, I don't have much time to write a better answer now (maybe I will come back later). In any case, I think you can find all answers to your questions/doubts in the VAE paper (although, as I said, you may need to read it at least twice to understand it).&lt;/p&gt;&#xA;&lt;p&gt;By the way, the simplest/cleanest implementation of the VAE that I have found so far is &lt;a href=&quot;https://github.com/pytorch/examples/blob/master/vae/main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-13T13:36:44.960" LastActivityDate="2020-11-13T13:36:44.960" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24567" PostTypeId="1" CreationDate="2020-11-12T23:02:19.977" Score="0" ViewCount="102" Body="&lt;p&gt;Using &lt;a href=&quot;https://github.com/tzutalin/labelImg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Label-img&lt;/a&gt;, I have successfully labeled my images (dimensions 1100 x 1100 pixels), and am currently training the SSD ResNet50 model (from the &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow 2 Detection Model Zoo&lt;/a&gt;). I downloaded the model script (i.e. config file), modified it to include the correct paths to my folders, and am currently training it.&lt;/p&gt;&#xA;&lt;p&gt;Is this technically considered transfer learning? I have read a lot about it, and I am not quite sure exactly what this is. In the future, I will be learning how to fine-tune these models, but for now, my question is this.&lt;/p&gt;&#xA;&lt;p&gt;If this is not transfer learning, could anyone provide an example to how it is different from what I am doing?&lt;/p&gt;&#xA;" OwnerUserId="32750" LastEditorUserId="11539" LastEditDate="2020-11-13T11:48:40.993" LastActivityDate="2020-11-13T11:48:40.993" Title="Am I using transfer learning when I use SSD ResNet 50 model architecture?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24568" PostTypeId="2" ParentId="24550" CreationDate="2020-11-13T09:18:32.447" Score="0" Body="&lt;p&gt;Your statement that  we should be expecting some similarity in graphical patterns is not correct. The GAN loss takes the following form:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_G \max_D V(D,G) = \mathbb{E}_{x \sim p_{data}(x)} [\log D(x)] + \\&#xA;+ \mathbb{E}_{z \sim p_z(z)} [\log (1 - D(G(z)))]\label{minimax}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We want to maximize this loss w.r.t. D in order to distinguish between real and fake samples (&lt;span class=&quot;math-container&quot;&gt;$D(x)\rightarrow 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D(G(z))\rightarrow 0$&lt;/span&gt;), whereas the task of G is exactly the &lt;strong&gt;opposite&lt;/strong&gt;. We want to minimize the function w.r.t. G so that the difference between real and generated data will be minimal. Thus, the problem becomes a minimax non-cooperative game. Here is a &lt;a href=&quot;https://danieltakeshi.github.io/2017/03/05/understanding-generative-adversarial-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;good explanation&lt;/a&gt; of the loss.&lt;/p&gt;&#xA;&lt;p&gt;Training GANs requires finding a Nash equilibrium of a non-convex game with continuous, high dimensional parameters. GANs are typically trained using gradient descent techniques that are designed to find a low value of a cost function, rather than to find the Nash equilibrium of a game. Two models are trained simultaneously and each model updates its cost independently with no respect to another player in the game.&lt;/p&gt;&#xA;&lt;p&gt;In other words, as soon as D becomes better, G becomes also better. Updating the gradient of both models concurrently cannot guarantee a convergence. For more information, see &lt;a href=&quot;https://arxiv.org/abs/1606.03498&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improved Techniques for Training GANs&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1701.07875&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wasserstein GAN&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, since D and G play a non-cooperative game, it can be shown that there are some cases when it is impossible to find a Nash equilibrium at all:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jm8Y7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jm8Y7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-11-13T10:27:03.070" LastActivityDate="2020-11-13T10:27:03.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24569" PostTypeId="1" CreationDate="2020-11-13T11:49:59.977" Score="1" ViewCount="35" Body="&lt;p&gt;I tried different values of genetic algorithm operators:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;many crossover rates from 20% to 80%&lt;/li&gt;&#xA;&lt;li&gt;many crossover rates from 1% to 20%&lt;/li&gt;&#xA;&lt;li&gt;varying the population size&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The study of different parameter values is called quantitative parameter tuning or sensitivity analysis. What is the difference between the two terms?&lt;/p&gt;&#xA;" OwnerUserId="41713" LastEditorUserId="2444" LastEditDate="2020-11-13T12:22:01.997" LastActivityDate="2020-11-13T14:01:02.240" Title="What is the difference between sensitivity analysis and parameter tuning?" Tags="&lt;comparison&gt;&lt;terminology&gt;&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24570" PostTypeId="2" ParentId="24569" CreationDate="2020-11-13T14:01:02.240" Score="1" Body="&lt;p&gt;I would generally assume that parameter tuning is the process of finding the combination of hyperparameters (e.g., population size, crossover and mutation operators and rates, etc.) that yield the best performance on your problem. When you're thinking about the way that performance varies with parameter choice, this is the &amp;quot;what&amp;quot;. What is the best choice of parameters.&lt;/p&gt;&#xA;&lt;p&gt;Sensitivity analysis is the &amp;quot;how&amp;quot; or &amp;quot;how much&amp;quot;. If I change my crossover rate from 0.9 to 1.0, how significant is the change in performance of my algorithm. Is my performance more or less stable across a wide range of choices (good) or is it highly dependent on finding this one little peak in the parameter space and every other choice is much worse (bad).&lt;/p&gt;&#xA;" OwnerUserId="3365" LastActivityDate="2020-11-13T14:01:02.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24571" PostTypeId="1" CreationDate="2020-11-13T19:31:26.543" Score="1" ViewCount="20" Body="&lt;p&gt;What is the consensus regarding NN &amp;quot;capacity&amp;quot; or expressive power? I remember reading somewhere that expressive power grows exponentially with depth, but I cannot seem to find that exact paper.&lt;/p&gt;&#xA;&lt;p&gt;If I have some dataset and some neural network, is there some heuristic, theorem, or result that may help me to decide whether this particular neural network I've chosen has enough expressive power (or capacity) to learn that data?&lt;/p&gt;&#xA;&lt;p&gt;Something similar to VC-dimension and VC-inequality, but regarding more complex models such as NNs.&lt;/p&gt;&#xA;&lt;p&gt;I suspect that there is no simple answer, but, generally, what would be the answer to this question?&lt;/p&gt;&#xA;&lt;p&gt;Overfitting on some subset might be a start, but that doesn't really tell me how the model behaves when there's more data, it only tells me that it's not fundamentally broken and can learn &lt;em&gt;something&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I know it's a complex matter, but I'll be grateful for any help, be it some practical stuff, as well as some references, papers, etc. Of course, I googled some papers, but if you have something particularly interesting, please share.&lt;/p&gt;&#xA;" OwnerUserId="40977" LastEditorUserId="2444" LastEditDate="2020-11-13T20:30:19.280" LastActivityDate="2020-11-13T20:30:19.280" Title="Given a dataset and a neural network, is there some heuristic or theorem to determine whether this neural network has enough capacity?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computational-learning-theory&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-11-14T01:32:20.643" ContentLicense="CC BY-SA 4.0" />
  <row Id="24572" PostTypeId="1" CreationDate="2020-11-13T20:39:38.953" Score="1" ViewCount="32" Body="&lt;p&gt;I have started studying ML just a short while ago, so that my questions will be very elementary. That being so, if they are not welcome, just tell me and I'll stop asking them.&lt;/p&gt;&#xA;&lt;p&gt;I gave myself a homework project which is to make an ML algorithm be able to learn that, if the last digit of a number &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0, 2, 4, 5, 6$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt;, then it cannot be a prime, provided &lt;span class=&quot;math-container&quot;&gt;$n &amp;gt; 5$&lt;/span&gt;. Note that, if a number &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; ends with &lt;span class=&quot;math-container&quot;&gt;$0, 2, 4, 6, 8$&lt;/span&gt;, then it is even, so it is divisible by &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, hence not prime. Similarly, numbers ending in &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; are divisible by &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;, so they cannot be prime.&lt;/p&gt;&#xA;&lt;p&gt;Which ML approach should I choose to solve this problem? I know that I don't need ML to solve this problem, but I am just trying to understand which ML approach I could use to solve this problem.&lt;/p&gt;&#xA;&lt;p&gt;So far, I have only learned about two ML approaches, namely linear regression (LR) and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-nearest neighborhoods, but they both seem inappropriate in this case since LR seems to be a good choice in finding numerical relations between input and output data and KNN seems to be good at finding clusters, and &amp;quot;primality&amp;quot; has neither of these characteristics.&lt;/p&gt;&#xA;" OwnerUserId="42266" LastEditorUserId="2444" LastEditDate="2020-11-14T18:42:35.450" LastActivityDate="2020-11-14T18:42:35.450" Title="Which ML approach could determine that a number greater than 5 is not prime, knowing that a number is not prime if it ends with an even digit or 5?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;homework&gt;&lt;primality-test&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24574" PostTypeId="1" CreationDate="2020-11-14T04:35:50.050" Score="3" ViewCount="73" Body="&lt;p&gt;From &lt;a href=&quot;https://arxiv.org/pdf/1911.08265.pdf#page=13&quot; rel=&quot;nofollow noreferrer&quot;&gt;the MuZero paper&lt;/a&gt; (Appendix E, page 13):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In chess, 8 planes are used to encode the action. The first one-hot plane encodes which position the piece was moved from. The next two planes encode which position the piece was moved to: a one-hot plane to encode the target position, if on the board, and a second binary plane to indicate whether the target was valid (on the board) or not. This is necessary because for simplicity our policy action space enumerates a superset of all possible actions, not all of which are legal, and we use the same action space for policy prediction and to encode the dynamics function input. The remaining five binary planes are used to indicate the type of promotion, if any (queen, knight, bishop, rook, none).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is the second binary plane all zeros or all ones?  Or, something else?&#xA;How is it known if the move is off the board?  For my game, I know if it is a legal move on the board, but do not know if the move is off the board.&lt;/p&gt;&#xA;" OwnerUserId="42271" LastEditorUserId="2444" LastEditDate="2020-11-14T15:23:57.383" LastActivityDate="2020-12-16T03:04:50.450" Title="How is MuZero's second binary plane for chess defined?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;papers&gt;&lt;muzero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24575" PostTypeId="1" CreationDate="2020-11-14T05:06:33.717" Score="0" ViewCount="27" Body="&lt;p&gt;For example, if we set the random seed to be 0, will we run into any problems? For example, maybe for seed 0, we can only reach a certain training error, but other seeds will converge to a much lower error&lt;/p&gt;&#xA;&lt;p&gt;I'm specifically concerned about supervised learning on point cloud data, but curious about whether it matters in general whenever you use a neural network.&lt;/p&gt;&#xA;" OwnerUserId="21158" LastEditorUserId="2444" LastEditDate="2020-11-14T14:05:17.393" LastActivityDate="2020-11-14T17:21:42.250" Title="Are there any downsides of using a fixed seed for a neural network's weight initialization?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;weights-initialization&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24576" PostTypeId="1" AcceptedAnswerId="24630" CreationDate="2020-11-14T05:57:33.670" Score="1" ViewCount="32" Body="&lt;p&gt;It seems to me that average pooling can be replaced by a strided convolution with a constant kernel. For instance, a 3x3 pooling would be equivalent to a strided convolution (of stride &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;) with a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; matrix of constants, with each entry being &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{9}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, I haven't found any mention of this fact online (perhaps it's too trivial of an observation)? Why then are explicit pooling layers needed if they can be realized by convolutions?&lt;/p&gt;&#xA;" OwnerUserId="23416" LastEditorUserId="2444" LastEditDate="2020-11-16T00:02:22.397" LastActivityDate="2020-11-16T00:16:17.420" Title="Is average pooling equivalent to a strided convolution with a specific constant kernel?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;implementation&gt;&lt;convolution&gt;&lt;pooling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24578" PostTypeId="1" CreationDate="2020-11-14T07:02:42.347" Score="0" ViewCount="16" Body="&lt;p&gt;I want to train a classification neural network then predict some inputs. It's classic classification example as you know. I'm implementing this with python.  I searched a lot about it and I always faced similar solution. I saw &amp;quot;bag of words&amp;quot; model. Then I implement it without library and created my word vectors. I'm not scraping news from web. I define them in some arrays like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;healthArticles = [&amp;quot;&amp;quot;&amp;quot;bla bla bla&amp;quot;&amp;quot;&amp;quot;,&amp;quot;&amp;quot;&amp;quot;other healt articles&amp;quot;&amp;quot;&amp;quot;]&#xA;sportArticles = [&amp;quot;&amp;quot;&amp;quot;bla bla bla&amp;quot;&amp;quot;&amp;quot;,&amp;quot;&amp;quot;&amp;quot;other sport articles&amp;quot;&amp;quot;&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and etc.&#xA;I refined my news, I cleared symbols, numbers. I only manipulate with word stem. Then I got word vectors of each articles like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1, 2, 2, 1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 6, 2, 1, 1, 1, 1, 2, 1, 1, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 6, 1, 1, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,    1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]&#xA;[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I shortened my vectors when writing to here. They are much longer of this. The last index is representing the category of the news.&#xA;So far, everything was fine. I searched and I implemented the model. But now I'm looking for classification. I saw Keras examples. I tried to implement various implemention but I can't. Cause they used library for convert words to vector. Then they create some keras layers, denses to fit this vector models. I can't code my own neural network because I don't have time. So I'm trying to use library for my train, test and prediction operations. I'm writing an example then I'm deleting it. So I can't put code part to here cause . I did a lot of research and viewed many samples.And additionaly, I don't want to copy use other peoples code. I'm trying to understand the logic of this classification method.&lt;/p&gt;&#xA;&lt;p&gt;Can someone enlighten me on this? How can I continue my project?&lt;/p&gt;&#xA;" OwnerUserId="42274" LastActivityDate="2020-11-14T07:02:42.347" Title="Python Create Keras Neural Network with Bag Of Words Model for News Classification" Tags="&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24579" PostTypeId="1" AcceptedAnswerId="24592" CreationDate="2020-11-14T10:52:17.187" Score="1" ViewCount="50" Body="&lt;p&gt;In &lt;a href=&quot;https://www.nature.com/articles/nature16961&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaGo&lt;/a&gt;, the authors initialised a policy gradient network with weights trained from imitation learning. I believe this gives it a very good starting policy for the policy gradient network. the imitation network was trained on labelled data of (state, expert action) to output a softmax policy denoting the probability of actions for each state.&lt;/p&gt;&#xA;&lt;p&gt;In my case, I would like to use weights learnt from the imitation network as initial starting weights for a DQN. Initial rewards from running the policy are high but start to decrease (for a while) as the DQN trains and later increases again.&lt;/p&gt;&#xA;&lt;p&gt;This could suggest that the effect of initialising weights from the imitation network has little effect, since it kind of undo's the effect of initialising weights from the imitation network.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-11-14T14:46:35.277" LastActivityDate="2020-11-14T17:53:40.687" Title="Initialising DQN with weights from imitation learning rather than policy gradient network" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;alphago&gt;&lt;imitation-learning&gt;" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="24580" PostTypeId="1" CreationDate="2020-11-14T11:27:26.390" Score="0" ViewCount="28" Body="&lt;p&gt;Consider an environment where there are 2 outcomes (e.g. dead and alive) and a discrete set of actions. For example, a game where the agent has 2 guns &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; to shoot a monster (the monster dies only if the correct gun is used).&lt;/p&gt;&#xA;&lt;p&gt;Let's say we store the experience &lt;span class=&quot;math-container&quot;&gt;$e_1 = (s,a_1, r_1, s'_1)$&lt;/span&gt; in the replay buffer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;: we have the monster to kill&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;: choose and use gun &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r_1$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; (the wrong gun)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s'_1$&lt;/span&gt;: monster is alive&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But we also save the alternative situation &lt;span class=&quot;math-container&quot;&gt;$e_2 = (s, a_2, r_2, s'_2)$&lt;/span&gt; in the replay buffer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;: the same state (we have the monster)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;: choose and use gun &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r_2$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$(-1) * r = 1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$s'_2$&lt;/span&gt;: the monster is dead&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I can't find a topic about this technique, or don't know what to look for.&lt;/p&gt;&#xA;" OwnerUserId="41979" LastEditorUserId="2444" LastEditDate="2020-11-14T14:38:38.327" LastActivityDate="2020-11-14T14:38:38.327" Title="What are the implications of storing the alternative situation (that could have been experienced) in the replay buffer?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24582" PostTypeId="2" ParentId="24437" CreationDate="2020-11-14T12:03:13.750" Score="2" Body="&lt;p&gt;To verify the accuracy of the classification stage, you will need labeled images with a single car.&lt;/p&gt;&#xA;&lt;p&gt;To train and verify accuracy of the detection stage and full system, you can:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;in the datasets with images with multiple cars, manually, mark the image rectangles that contains one car.&lt;/li&gt;&#xA;&lt;li&gt;from previous, split the image in one or more ones, each one containing a single car.&lt;/li&gt;&#xA;&lt;li&gt;pass each one of the previous image with a single car to the classification stage (that means assume classification has 100% accuracy). Record its outputs (labeled cars).&lt;/li&gt;&#xA;&lt;li&gt;now, from output of steps 1) and 3), you can produce labeled images with multiple cars. Use it to train the detector and verify full system accuracy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12630" LastActivityDate="2020-11-14T12:03:13.750" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24583" PostTypeId="2" ParentId="18648" CreationDate="2020-11-14T12:37:10.777" Score="0" Body="&lt;p&gt;I interpret this as the following, I will use the uppercase matrix notation &lt;span class=&quot;math-container&quot;&gt;$\mathbf{K_{*}}$&lt;/span&gt;, etc...&lt;/p&gt;&#xA;&lt;p&gt;The covariance matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{K_{xx}}$&lt;/span&gt; summarizes everything we know about the input feature space. I think of it as a unique signature that describes the data we have in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{d_x}$&lt;/span&gt;. Along with the example training data, we have the labels &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;, which give us a concrete definition of what the prediction values for &lt;span class=&quot;math-container&quot;&gt;$\mathbf{K_{xx}}$&lt;/span&gt; should be.&lt;/p&gt;&#xA;&lt;p&gt;We can then ask the question &amp;quot;What can we multiply the unique signature &lt;span class=&quot;math-container&quot;&gt;$\mathbf{K_{xx}}$&lt;/span&gt; by in order to get the training outputs?&amp;quot; This takes the familiar form of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}\mathbf{x} = \mathbf{b}$&lt;/span&gt;...&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\mathbf{K}_{xx}\mathbf{z} &amp;amp;= \mathbf{y} \\&#xA;\mathbf{z} &amp;amp;= \mathbf{K}_{xx}^{-1}\mathbf{y}&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; then represents the vector that the covariance matrix transforms into the outputs. Then all we have to do is multiply the new covariance of the train/test points by that same vector to get the predictions...&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{\hat{y}} = \mathbf{K_{*x}}\mathbf{K_{xx}^{-1}}\mathbf{y}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I should add that I am relatively certain about this, but I am still learning this stuff myself so I am not 100&lt;/p&gt;&#xA;" OwnerUserId="42278" LastActivityDate="2020-11-14T12:37:10.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24584" PostTypeId="2" ParentId="3817" CreationDate="2020-11-14T12:44:35.547" Score="0" Body="&lt;p&gt;Yes, you are trying to predict a real number output, so this is a regression problem. To know what kind of algorithm would be best I think you have to ask how much data you have and what you know already about the relationships of the numbers. If you try simple linear regression, what kind of error will you get?&lt;/p&gt;&#xA;&lt;p&gt;If you were to try linear regression and you get an error that is acceptable, then it may be a very simple problem. Beyond linear regression you can look to more advanced things such as Gaussian processes and neural networks which will all make the kinds of predictions you are seeking.&lt;/p&gt;&#xA;" OwnerUserId="42278" LastActivityDate="2020-11-14T12:44:35.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24585" PostTypeId="1" CreationDate="2020-11-14T12:59:49.627" Score="2" ViewCount="28" Body="&lt;p&gt;I answered another question &lt;a href=&quot;https://ai.stackexchange.com/questions/18648/interpretation-of-inverse-matrix-in-mean-calculation-in-gaussian-process&quot;&gt;here&lt;/a&gt; about the mean prediction of a GP, but I have a hard time coming up with an intuitive explanation of the variance prediction of a GP.&lt;/p&gt;&#xA;&lt;p&gt;Thew specific equation that I am speaking of is equation 2.26 in &lt;a href=&quot;http://www.gaussianprocess.org/gpml/chapters/RW2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Gaussian process book&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{V}[f_*] = k(x_*, x_*) - k_{x*}^TK_{xx}^{-1}k_{x*}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a number of questions about this...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;if &lt;span class=&quot;math-container&quot;&gt;$k(x_*, x_*)$&lt;/span&gt; is the result of the kernel function with a single point &lt;span class=&quot;math-container&quot;&gt;$x_*$&lt;/span&gt;, then won't this value always be 1 (assuming an RBF kernel) since the kernel will give 1 for a covariance with itself (&lt;span class=&quot;math-container&quot;&gt;$k(x, x) =\exp\{-\frac{1}{2}|| x - x ||^2\}$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the kernel value &lt;span class=&quot;math-container&quot;&gt;$k(x_*, x_*)$&lt;/span&gt; is indeed one for any single arbitrary point, then how can I interpret the last multiplication on the RHS? &lt;span class=&quot;math-container&quot;&gt;$K_{xx}^{-1}k_{x*}$&lt;/span&gt; is the solution to &lt;span class=&quot;math-container&quot;&gt;$Ax = b$&lt;/span&gt;, which is the vector which &lt;span class=&quot;math-container&quot;&gt;$K_{xx}$&lt;/span&gt; projects into &lt;span class=&quot;math-container&quot;&gt;$k_{x*}$&lt;/span&gt;, but then my intuition breaks down and I cannot explain anymore.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the kernel value &lt;span class=&quot;math-container&quot;&gt;$k(x_*, x_*)$&lt;/span&gt; is indeed one for any single arbitrary point, then can we view the whole term as the prior variance being reduced by the some sort of similarity between the test point and the training points?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it every possible for this variance to be greater than 1? Or is the prior variance of 1 seen as the maximum, which can only be reduced by observing more data?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42278" LastActivityDate="2020-11-14T12:59:49.627" Title="How to interpret the variance calculation in a Guassian process" Tags="&lt;linear-algebra&gt;&lt;gaussian-process&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24586" PostTypeId="5" CreationDate="2020-11-14T13:36:46.003" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-14T13:36:46.003" LastActivityDate="2020-11-14T13:36:46.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24587" PostTypeId="4" CreationDate="2020-11-14T13:36:46.003" Score="0" Body="For questions about primality tests in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-14T13:36:46.003" LastActivityDate="2020-11-14T13:36:46.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24589" PostTypeId="2" ParentId="24437" CreationDate="2020-11-14T16:39:15.703" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can see from the question that existing information on detection and&#xA;classification in the small automotive vehicle domain has been&#xA;located (in the form of two independent sets of vectors usable for&#xA;machine training), and there is no already existing mapping or&#xA;other correspondence between the elements of one set and the&#xA;elements of the other.  They were obtained independently, remain&#xA;independent, and are linked only by the conventions of the domain&#xA;(today's aesthetically acceptable and thermodynamically workable&#xA;forms of small vehicles).&lt;/p&gt;&#xA;&lt;p&gt;The goal stated in the question is to create a computer vision system&#xA;that both detects cars and classifies them leveraging the&#xA;information contained in the two distinct sets.&lt;/p&gt;&#xA;&lt;p&gt;In the vision systems of mammals, there are also two distinct equivalences&#xA;of sets; one arising from a genetic algorithm, the DNA that is&#xA;expressed during the formation of the neural net geometry and&#xA;bio-electro-chemistry of the visual system in early development;&#xA;and the cognitive and coordinative pathways in the cerebrum and&#xA;cerebellum.&lt;/p&gt;&#xA;&lt;p&gt;If a robot, wheelchair, or other vehicle is to avoid traffic, we must&#xA;produce a system that in some way matches or exceeds the&#xA;collision avoidance performance of mammals.&#xA;In crime prevention, toll collection, sales lot inventory,&#xA;county traffic analysis, and other like applications,&#xA;performance will again be expected to match or exceed the&#xA;performance of biological systems.&#xA;If a person can record the make, model, year, color, and&#xA;license plate strings, so should the machine we employ in&#xA;these capacities.&lt;/p&gt;&#xA;&lt;p&gt;Consequently, this question is pertinent beyond academic curiosity, as&#xA;it is applicable in current research and development of products.&lt;/p&gt;&#xA;&lt;p&gt;That this question author notices the lack of a unified data&#xA;set that can be used to train it to detect and characterize&#xA;in a single network objects of interest is apropos and&#xA;key to the challenge of finding a solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Approach&lt;/em&gt;&#xA;The simplest approach would be to compose the system of two functions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\mathcal{D}: \mathbb{I}^4 \to {(\mathbb{I}^2, \mathbb{I}^2)}_1, \; {(\mathbb{I}^2, \mathbb{I}^2)}_2, \; ... $&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\mathcal{C}: {(\mathbb{I}^2, \mathbb{I}^2)}_i \to {(\mathbb{I})}_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The four dimensions of input for &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;, the detector, are horizontal position, vertical position,&#xA;rgb index, and brightness to decribe the pixelized image; and the output are bounding boxes as two &amp;quot;corner&amp;quot; coordinates corresponding to each identified vehicle, the second coordinate being either relative to the first or to a specific corner of the entire frame.&#xA;The categorizer, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt;, receives as input bounding boxes and produces as output the index&#xA;or code that maps to the categories corresponding to the labels of the training set available for&#xA;categorization.&#xA;The system can then be described as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\mathcal{S}: \mathcal{C} \circ \mathcal{D}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the system is not color, subtract one from the above dimensionality of the input.  If the system processes video, add one to the dimensionality of the input and consider using LSTM or GRU cell types.&lt;/p&gt;&#xA;&lt;p&gt;The above substitution represented by &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$\circ$&lt;/span&gt;&amp;quot; appears to be what is meant by, &amp;quot;I use the images from the detection dataset as input&#xA;and get classification predictions on top of detected bounding boxes.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The interrogative, &amp;quot;How do I verify whether the classification model trained on the&#xA;classification dataset is working on images from detection dataset?&#xA;(In terms of classification accuracy),&amp;quot;&#xA;appears to refer to the fact that labels do not exist for the second set that correspond to&#xA;input elements of the first set, so an accuracy metric cannot be directly obtained.&#xA;Since there is no obvious automatic way of generating labels for the vehicles in the pre-detected images&#xA;containing potentially multiple vehicles,&#xA;there is no way to check actual results against expected results.&#xA;Composing multiple vehicle images from the categorization set to use as test input to&#xA;the entire system &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; will only be useful in evaluating an aspect&#xA;of the performance of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;, not &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Solution&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only way to evaluate the accuracy and reliability of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; is with portions&#xA;of the set used to train it that were excluded from the training and trust&#xA;that the vehicles depicted in those images were sufficiently representative&#xA;of the concept &amp;quot;car&amp;quot; to provide consistency of accuracy and reliability across&#xA;the range of those detected by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; in the application of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt;.&#xA;This means that the leveraging of the information, even if optimized to the degree&#xA;possible by any arbitrary algorithm or parallelism in the set of all possible&#xA;algorithms or parallelisms, is limited by the categorization training set.&#xA;The number of set elements and the comprehensiveness and distribution of categories&#xA;within that set must be sufficient to achieve an approximate equality between&#xA;these two accuracy metrics.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Categorizing a test sample from the labeled set for &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; excluded from the training&lt;/li&gt;&#xA;&lt;li&gt;Categorizing the vehicles isolated by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; from its training input&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;With Additional Resources&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Of course this discussion is in a particular environment, that of the system&#xA;defined as the two artificial networks, one involving convolution based&#xA;recognition and the other involving feature extraction, and the two training&#xA;sets.&#xA;What is needed is a wider environment where known vehicles are in view so that&#xA;performance data of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; is evaluated and a tap on the transfer&#xA;of information between &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; can be used&#xA;to differentiate between mistakes made on either side of the tap point.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Unsupervised Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another course of action could be to not use the training set for categorization&#xA;on the training of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; at all, but rather use feature extraction and&#xA;auto-correlation in an &amp;quot;unsupervised&amp;quot; approach, and then evaluate the results of&#xA;on the basis of the final convergence metrics at the point when stability in&#xA;categorization is detected.  In this case, the images in the bounding boxes&#xA;output by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt; would be used as training data.&lt;/p&gt;&#xA;&lt;p&gt;The auto-trained network realizing &lt;span class=&quot;math-container&quot;&gt;$\mathcal{C}$&lt;/span&gt; can then be further evaluated&#xA;using the entire categorization training set.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Further Research&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hybrids of these two approaches are possible. Also, the independent training only in the rarest of cases leads to optimal performance. Understanding feedback as originally treated with rigor by MacColl in chapter 8 of his &lt;em&gt;Fundamental Theory of Servomechanisms&lt;/em&gt;, later applied to the problem of linearity and stability of analog circuitry, and then to training, first in the case of GANs, may lead to effective methods to bi-train the two networks.&lt;/p&gt;&#xA;&lt;p&gt;That evolved biological networks are trained &lt;em&gt;in situ&lt;/em&gt; is an indicator that the most optimal performance can be gained by finding training architectures and information flow strategies that create optimality in both components simultaneously.  No biological niche has ever been filled by a neural component that is first optimized and then inserted or copied in some way to a larger brain system.  That is no proof that such component-ware can be optimal, but there is also no proof that the DNA driven systems that have emerged are not nearly optimized for the majority of terrestrial conditions.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2020-11-16T22:19:54.667" LastActivityDate="2020-11-16T22:19:54.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24590" PostTypeId="2" ParentId="17971" CreationDate="2020-11-14T16:50:50.197" Score="1" Body="&lt;p&gt;I was struggling with the same question. This is what I came up with after thinking it through.&lt;/p&gt;&#xA;&lt;p&gt;With depth-first-search, you backtrack to a node that is a non-expanded child of your parent (or the parent of the parent when your parent has no more non-expanded children (and so on going up the tree)). So the space complexity is limited by your ancestors and the children of these ancestors. Which translates in m*b where m is the max path length (so max number of ancestors) and b is the branching factor (number of children per ancestor).&lt;/p&gt;&#xA;&lt;p&gt;With greedy search when you backtrack you can jump to any evaluated but unexpanded node, you passed going down on paths earlier. So the algorithm, when backtracking, can make pretty random jumps throughout the tree leaving lots of sibling nodes unexpanded. You will have to remember the value of the evaluation function for all these unexpanded nodes though because possibly they are next up when backtracking occurs. So in a theoretical very worst-case scenario that could mean that almost the whole tree needs to be remembered. Hence O(b^m).&lt;/p&gt;&#xA;&lt;p&gt;I know there are still gaps in the reasoning but intuitively this makes me understand it best.&lt;/p&gt;&#xA;" OwnerUserId="42281" LastActivityDate="2020-11-14T16:50:50.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24591" PostTypeId="2" ParentId="24575" CreationDate="2020-11-14T17:21:42.250" Score="1" Body="&lt;p&gt;When you use a particular seed, it actually ceases to become a random initialization and is instead fixed. I believe the only reason to actually do this would be for reliable reproduction in research and not as a method of training production models.&lt;/p&gt;&#xA;" OwnerUserId="42278" LastActivityDate="2020-11-14T17:21:42.250" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24592" PostTypeId="2" ParentId="24579" CreationDate="2020-11-14T17:53:40.687" Score="1" Body="&lt;p&gt;My understanding is that you are first training a policy network using imitation learning. Then you are adjusting that trained network in some way to be a value network for DQN. The most obvious change would be to remove &lt;em&gt;softmax&lt;/em&gt; activation whilst keeping the network layer sizes identical. This would then present Q estimates for all actions from any given state.&lt;/p&gt;&#xA;&lt;p&gt;The initial estimates would not be trained Q values though, they would be the &amp;quot;preferences&amp;quot; or the logits for probabilities to support a near optimal action choice. The main thing that will be likely in the new network is that for the one near optimal action choice, the network would predict the highest action value. As you derive the target policy by taking the maximising action, initially this looks good. However, the problem is that the Q values that this network predicts can have little to no relation to the real expected returns experienced by the agent under the target policy.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Initial rewards from running the policy are high but start to decrease (for a while) as the DQN trains and later increases again.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think what is happening is that initially the greedy policy derived from your Q network is very similar to the policy learned during imitation learning. However, the value estimates are very wrong. This leads to large error values, large corrections needed, and radical changes to network weights throughout in order to change the network from an approximate policy function to an approximate action value function. The loss of performance occurs because there is not a smooth transition between the two very different functions that also maintains correct maximising actions.&lt;/p&gt;&#xA;&lt;p&gt;I don't think this can be completely fixed. However you might get some insight into potential work-arounds by considering that you are not just doing imitation learning here. Instead you are performing both imitation learning (to copy a near optimal policy) and transfer learning (to re-use network weights on a related task).&lt;/p&gt;&#xA;&lt;p&gt;Approaches that help with transfer learning may also help here. For instance, you could freeze the layers closer to input features, or reduce the learning rate for those layers. You do either of these things on the assumption that the low-level derived features (in the hidden layers) that the first network has learned are still useful for the new task.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-14T17:53:40.687" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24593" PostTypeId="1" AcceptedAnswerId="24607" CreationDate="2020-11-14T18:23:46.803" Score="0" ViewCount="174" Body="&lt;p&gt;I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s.&#xA;The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different from the input matrix), and a vector of values.&lt;/p&gt;&#xA;&lt;p&gt;The first input is fed to an MLP network, the second input (matrix) is fed to a convolutional layer, then their outputs are fed to a FC network, or at least that's the idea.&lt;/p&gt;&#xA;&lt;p&gt;This is my attempt so far, having this &lt;a href=&quot;https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; as a reference.&lt;/p&gt;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&lt;p&gt;First, create the MLP network&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def create_mlp(self, arr, regress=False): # for the position input&#xA;        # define MLP network&#xA;        print(&amp;quot;Array&amp;quot;, arr)&#xA;        model = Sequential()&#xA;        model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation=&amp;quot;relu&amp;quot;))&#xA;        model.add(Dense((env.rows * env.cols)//2, activation=&amp;quot;relu&amp;quot;))&#xA;        &#xA;        # check to see if the regression node should be added&#xA;        if regress:&#xA;            model.add(Dense(1, activation=&amp;quot;linear&amp;quot;))&#xA;            &#xA;        # return our model&#xA;        return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, the CNN&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def create_cnn(self, width, height, depth=1, regress=False): # for the matrix&#xA;        # initialize the input shape and channel dimension&#xA;        inputShape = (height, width, depth)&#xA;        output_nodes = 6e2&#xA;        &#xA;        # define the model input&#xA;        inputs = Input(shape=inputShape)&#xA;&#xA;        # if this is the first CONV layer then set the input&#xA;        # appropriately&#xA;        x = inputs&#xA;        &#xA;        input_layer = Input(shape=(width, height, depth))&#xA;        conv1 = Conv2D(100, 3, padding=&amp;quot;same&amp;quot;, activation=&amp;quot;relu&amp;quot;, input_shape=inputShape) (input_layer)&#xA;        pool1 = MaxPooling2D(pool_size=(2,2), padding=&amp;quot;same&amp;quot;)(conv1)&#xA;        flat = Flatten()(pool1)&#xA;        hidden1 = Dense(200, activation='softmax')(flat) #relu&#xA;&#xA;        batchnorm1 = BatchNormalization()(hidden1) &#xA;        output_layer = Dense(output_nodes, activation=&amp;quot;softmax&amp;quot;)(batchnorm1) &#xA;        output_layer2 = Dense(output_nodes, activation=&amp;quot;relu&amp;quot;)(output_layer) &#xA;        output_reshape = Reshape((int(output_nodes), 1))(output_layer2)&#xA;        model = Model(inputs=input_layer, outputs=output_reshape)&#xA;&#xA;        # return the CNN&#xA;        return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, concatenate the two&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def _build_model(self):&#xA;        # create the MLP and CNN models&#xA;        mlp = self.create_mlp(env.stateSpacePos)&#xA;        cnn = self.create_cnn(3, len(env.UEs))&#xA;        &#xA;        # create the input to our final set of layers as the *output* of both&#xA;        # the MLP and CNN&#xA;        combinedInput = concatenate([mlp.output, cnn.output])&#xA;        &#xA;        # our final FC layer head will have two dense layers, the final one&#xA;        # being our regression head&#xA;        x = Dense(len(env.stateSpacePos), activation=&amp;quot;relu&amp;quot;)(combinedInput)&#xA;        x = Dense(1, activation=&amp;quot;linear&amp;quot;)(x)&#xA;        &#xA;        # our final model will accept categorical/numerical data on the MLP&#xA;        # input and images on the CNN input, outputting a single value&#xA;        model = Model(inputs=[mlp.input, cnn.input], outputs=x)&#xA;        &#xA;        opt = Adam(lr=self.learning_rate, decay=self.epsilon_decay)&#xA;        model.compile(loss=&amp;quot;mean_absolute_percentage_error&amp;quot;, optimizer=opt)&#xA;        &#xA;        print(model.summary())&#xA;        &#xA;        return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 50), (None, 600, 1)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The line of code that gives the error is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;combinedInput = concatenate([mlp.output, cnn.output])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the MLP summary&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JWVh3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JWVh3.png&quot; alt=&quot;MLP Summary&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And this is the CNN summary&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sU6ag.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sU6ag.png&quot; alt=&quot;CNN Summary&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm a beginner at this, and I'm not where my mistakes are, the code does not work obviously but I do not know how to correct it.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastEditorUserId="42372" LastEditDate="2020-11-14T21:45:07.383" LastActivityDate="2020-11-15T13:57:34.967" Title="Keras DQN Model with Multiple Inputs and Multiple Outputs" Tags="&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;tensorflow&gt;&lt;dqn&gt;&lt;multilayer-perceptron&gt;" AnswerCount="1" CommentCount="4" ClosedDate="2020-11-15T18:03:02.550" ContentLicense="CC BY-SA 4.0" />
  <row Id="24595" PostTypeId="1" AcceptedAnswerId="24596" CreationDate="2020-11-14T19:48:38.607" Score="0" ViewCount="358" Body="&lt;p&gt;I am trying understand machine learning inferece, and i would like to know what exactly is the difference between Google Coral USB and Movidius Intel Neural Compute Stick 2. From what i could gather the google coral USB speeds up the frame rate, but that doesn't look clear to me. My questions are:&#xA;What exactly is the benefit from both of them in units? Like, is it frame rate? prediction speed? Are both visual processing units? And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?&lt;/p&gt;&#xA;" OwnerUserId="42284" LastActivityDate="2020-11-15T00:04:57.227" Title="Difference between Neural Compute Stick 2 and Google Coral USB for edge computing" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;inference&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-11-15T00:58:14.467" ContentLicense="CC BY-SA 4.0" />
  <row Id="24596" PostTypeId="2" ParentId="24595" CreationDate="2020-11-14T23:47:32.630" Score="0" Body="&lt;p&gt;I do not know much about the Neural Compute Stick but I can tell you a little bit about the Coral Edge TPU, since I used it myself (which I think also applies to the Neural Compute Stick).&lt;/p&gt;&#xA;&lt;p&gt;The Edgte TPU is a specialized ASIC which is very efficient at the main calculations (convolutions, relu etc.) for neural network inferencing. That means it cannot be used for training* a neural network but for deploying a neural network in production after it has been trained and optimized/quantized (precision reduced from float32 to int8). But I think you already new that, as it seems from your question.&lt;/p&gt;&#xA;&lt;p&gt;Now to your actual question in terms of speed: You cannot really compare speed of such chip in terms of framerate alone, nor can you call it a visual processing unit or not. The google coral is a general ASIC that is very fast at doing convolutions/relus etc. For what you gonna use your neural network (like Image recognition, or maybe stock predicitons) is complitly up to you, and especially up to your neural network for which task it was trained for. The only limitation you have here is regarding the supported layer operations. E.g. it is not possible to do operations like 3D convolutions or some fancy new non-linear activation functions (There was an overview of supported operations in the docs that I cannot find right now).&lt;/p&gt;&#xA;&lt;p&gt;Furthermore the framerate also depends on your NN architecture, image resolution etc. so a comparison here is completly misleading. If you want to have a general indication of speed look at how many int8 operations it can handle per second (TOPS) also under what energy consuption (Watts), if you care.&lt;/p&gt;&#xA;&lt;p&gt;The main advantage of this unit compared to a GPU that is usually used for training+inferencing is much lower energy consumption and unit cost. With roughly 4 Watts on the Edge TPU (as far as I can remember) compared to e.g. 250 Watts GPU. Energy consumption also dictates the necessary cooling solution, which can just be a passive for the Edge Tpu in many cases. Regarding the unit costs, I guess you can easily sees that yourself, if you keep in mind that you can get &amp;quot;nearly&amp;quot; similar inferencing speeds than with a full workstation class GPU. Furthermore, such unit has a much smaller formfactor and weight, which makes it perfect for applications at the edge. (I should also add that the efficiency gain is also a lot due to the precision reduction to int8 (quantization). Quantization is also possible on GPUs.)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not quite sure what you mean with this. The general workflow would look like this: you use a GPU for training your neural network, after training you optimize/quantize your network, and lastly you deploy your neural network to the &amp;quot;computer board&amp;quot; of your choice that just needs to have a CPU and the Edge TPU and do inferencing operations/predictions for your task.&lt;/p&gt;&#xA;&lt;p&gt;*Transfer learning is possible of e.g. the last layer. But no backpropagation for full NN training.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastEditorUserId="13104" LastEditDate="2020-11-15T00:04:57.227" LastActivityDate="2020-11-15T00:04:57.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24597" PostTypeId="5" CreationDate="2020-11-15T01:12:05.230" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T01:12:05.230" LastActivityDate="2020-11-15T01:12:05.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24598" PostTypeId="4" CreationDate="2020-11-15T01:12:05.230" Score="0" Body="For questions about the concept of weight (or parameter) of a machine learning model, such as a neural network or a linear regression model. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T01:12:05.230" LastActivityDate="2020-11-15T01:12:05.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24599" PostTypeId="5" CreationDate="2020-11-15T01:17:15.343" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T01:17:15.343" LastActivityDate="2020-11-15T01:17:15.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24600" PostTypeId="4" CreationDate="2020-11-15T01:17:15.343" Score="0" Body="For questions about the super-resolution of images (and videos)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T01:17:15.343" LastActivityDate="2020-11-15T01:17:15.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24602" PostTypeId="1" CreationDate="2020-11-15T07:11:03.997" Score="1" ViewCount="65" Body="&lt;p&gt;Some RL algorithms can only be used for environments with continuous action spaces (e.g TD3, SAC), while others only for discrete action spaces (DQN), and some for both&lt;/p&gt;&#xA;&lt;p&gt;REINFORCE and other policy gradient variants have the choice of using a categorical policy for discrete actions and a Gaussian policy for continuous action spaces, which explains how they can support both. Is that interpretation completely correct?&lt;/p&gt;&#xA;&lt;p&gt;For algorithms that learn a Q function or a Q function and a policy, what places this restriction for their use on either discrete or continuous action spaces environments?&lt;/p&gt;&#xA;&lt;p&gt;In the same regard, if an algorithm suited for discrete spaces is to be tuned to handle continuous action spaces, or vice versa. What does such a configuration involve?&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="2444" LastEditDate="2020-11-15T23:37:40.673" LastActivityDate="2020-11-15T23:37:40.673" Title="What adapts an algorithm to continuous or to discrete action spaces?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;reinforce&gt;&lt;continuous-action-spaces&gt;&lt;discrete-action-spaces&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24603" PostTypeId="1" CreationDate="2020-11-15T08:02:00.280" Score="1" ViewCount="47" Body="&lt;p&gt;I started researching the subject of self-replication in neural networks, and unexpectedly I saw that there is not much research on this subject. I should mention I am new in the field of NNs.&lt;/p&gt;&#xA;&lt;p&gt;This idea seems to be very appealing, but now I am having problems coming up with an actual use-case. &lt;a href=&quot;https://arxiv.org/abs/1803.05859&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; from 2018 seems to be one of the main ones addressing this topic.&lt;/p&gt;&#xA;&lt;p&gt;So, what are the use-cases of self-replicating neural networks? Why isn't this subject more thoroughly researched?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: good point made in the &lt;a href=&quot;https://old.reddit.com/r/MachineLearning/comments/js4bkz/d_why_arent_self_replicating_neural_networks_more/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reddit post&lt;/a&gt; I also made:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;it's really just take machine learning model and have it predict its&#xA;own weights&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="42217" LastEditorUserId="2444" LastEditDate="2020-11-15T12:24:08.657" LastActivityDate="2020-11-15T12:24:08.657" Title="What are the use-cases of self-replicating neural networks?" Tags="&lt;neural-networks&gt;&lt;applications&gt;&lt;self-replication&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24605" PostTypeId="1" AcceptedAnswerId="24608" CreationDate="2020-11-15T12:44:34.773" Score="1" ViewCount="40" Body="&lt;p&gt;In a general DQN framework, if I have an idea of some actions being better than some other actions, is it possible to make the agent select the better actions more often ?&lt;/p&gt;&#xA;" OwnerUserId="41984" LastActivityDate="2020-11-15T15:18:25.223" Title="In reinforcement learning, is it possible to make some actions more likely?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24606" PostTypeId="1" CreationDate="2020-11-15T13:42:00.723" Score="1" ViewCount="26" Body="&lt;p&gt;I am not sure if I really understand how anchor boxes are defined. As far as I understand, in YOLO algorithm you define a set of &amp;quot;good&amp;quot; shapes (anchor boxes) that may contain the object you are trying to detect for each cell.&lt;/p&gt;&#xA;&lt;p&gt;However, I don't really understand how do you predefine the shape of these anchor boxes. As far as I have seen, there are examples in which the algorithm outputs b&lt;sub&gt;x&lt;/sub&gt;, b&lt;sub&gt;y&lt;/sub&gt;, b&lt;sub&gt;h&lt;/sub&gt; and b&lt;sub&gt;w&lt;/sub&gt; values for each anchor box. Are you actually giving &amp;quot;freedom&amp;quot; to the algorithm to define these four values, or somehow is it being defined a fixed ratio between b&lt;sub&gt;h&lt;/sub&gt; and b&lt;sub&gt;w&lt;/sub&gt; for each of the anchor boxes? And how is this ratio defined in the output?&lt;/p&gt;&#xA;" OwnerUserId="42296" LastEditorUserId="42296" LastEditDate="2020-11-15T16:49:16.650" LastActivityDate="2020-11-15T16:49:16.650" Title="How is the shape of the anchor boxes predefined in YOLO algorithm?" Tags="&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24607" PostTypeId="2" ParentId="24593" CreationDate="2020-11-15T13:57:34.967" Score="1" Body="&lt;p&gt;Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work.&#xA;Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate.&lt;/p&gt;&#xA;&lt;p&gt;I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. Because you need to flatten feature to use dense layer later.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def create_mlp(self, arr, regress=False): &#xA;        # define MLP network&#xA;        print(&amp;quot;Array&amp;quot;, arr)&#xA;        model = Sequential()&#xA;        model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation=&amp;quot;relu&amp;quot;))&#xA;        model.add(Dense((env.rows * env.cols)//2, activation=&amp;quot;relu&amp;quot;))&#xA;        **model.add.flatten() ### shape = (None, 1600)**&#xA;        # check to see if the regression node should be added&#xA;        if regress:&#xA;            model.add(Dense(1, activation=&amp;quot;linear&amp;quot;))&#xA;        # return our model&#xA;        return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And just remove the reshape layer in &lt;strong&gt;create_cnn&lt;/strong&gt; function.&#xA;(output shape should be = (None, 600)).&lt;/p&gt;&#xA;&lt;p&gt;then concatenate two model&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;combinedInput = concatenate([mlp.output, cnn.output])  ## output shape =(None, 2200)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Later you can just use Dense layer as your code. I don't know how can you used dense (next to concatenate layer) without flatten the feature in &lt;strong&gt;create_mlp&lt;/strong&gt; function.&lt;/p&gt;&#xA;&lt;p&gt;Your code should work this way.&#xA;You can read &lt;a href=&quot;https://stackoverflow.com/questions/57202016/concatenate-two-models-with-tensorflow-keras&quot;&gt;this&lt;/a&gt; simple one for better understanding.&lt;/p&gt;&#xA;" OwnerUserId="42283" LastActivityDate="2020-11-15T13:57:34.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24608" PostTypeId="2" ParentId="24605" CreationDate="2020-11-15T15:18:25.223" Score="0" Body="&lt;p&gt;For single-step Q learning, the behaviour policy can be any stochastic policy without any further adjustment to the update rules.&lt;/p&gt;&#xA;&lt;p&gt;You don't &lt;em&gt;have&lt;/em&gt; to use &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy based on current Q function approximation, although that is a common choice because it works well in general cases. However, you should always allow some chance of taking all actions if you want the algorithm to converge - if you fixed things so that bad actions were never taken, the agent would never learn that they had low value.&lt;/p&gt;&#xA;&lt;p&gt;Probably the simplest way to use your initial idea of best actions is to write a function that returns your assessment of which action to take, and use that with some probability in preference to a completely random choice. At some point you will also want to stop referencing the helper function (unles it is guaranteed perfect) and use some form of standard &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy based on current Q values.&lt;/p&gt;&#xA;&lt;p&gt;I have done similar with a DQN learning to play Connect 4, where the agent would use a look-ahead search function that could see e.g. 7 steps ahead. If that was inconclusive it would use argmax of current Q values. Both these fixed action choices could be replaced, with probability &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, with a random action choice to ensure exploration. It worked very well. You could replace the look-ahead search in my example with any function that returned &amp;quot;best&amp;quot; actions for any reason.&lt;/p&gt;&#xA;&lt;p&gt;There are some other ways you can skew action selection towards better looking action choices. You could look into &lt;a href=&quot;https://arxiv.org/pdf/1705.10257.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boltzmann exploration&lt;/a&gt; or &lt;a href=&quot;https://www.geeksforgeeks.org/upper-confidence-bound-algorithm-in-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;upper confidence bounds (UCB)&lt;/a&gt; as other ways to create behaviour policies for DQN.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-15T15:18:25.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24609" PostTypeId="1" CreationDate="2020-11-15T15:25:12.000" Score="2" ViewCount="35" Body="&lt;p&gt;I noticed that SARSA has been rarely used in the deep RL setting. Usually, the training for DQN is done off-policy. I think one of the major reasons for this is due to greater sample efficiency in training due to the reuse of experiences in training off-policy. For SARSA, I would think that at every time step of an update, a stochastic gradient update of that sample would have to be performed and then the sample would have to be thrown away.&lt;/p&gt;&#xA;&lt;p&gt;The approach while might take longer to train, might still allow the agent to do relatively well. Would deep SARSA implementation perform as well as DQN in terms of final performance (since SARSA would definitely take longer to train)?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-11-15T23:29:24.127" LastActivityDate="2020-11-15T23:29:24.127" Title="Is Deep SARSA learning a feasible approach?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;off-policy-methods&gt;&lt;sarsa&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24611" PostTypeId="2" ParentId="21926" CreationDate="2020-11-15T17:26:59.773" Score="0" Body="&lt;p&gt;I don't know what is your dataset exactly look like. But based on assumption, I would like to refer something --&lt;/p&gt;&#xA;&lt;p&gt;You can think your MDP environment this way&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;action = {stay, go}&lt;/p&gt;&#xA;&lt;p&gt;reward = {something like based on visitor's satisfaction maybe &lt;strong&gt;rating&lt;/strong&gt;}&lt;/p&gt;&#xA;&lt;p&gt;state = {current money in hand, city, other some variable those key feature to make next iteration &lt;strong&gt;action&lt;/strong&gt;}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am working on a project-based stock market trading (sorry, I can not share detail).&#xA;In stock market trading problem, we need to decide action sequentially ( like per hour or maybe day). If your problem (and data as I assume) is a sequential action selection problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;More detail: the first day you visited NY and enjoyed your moment and cost &lt;span class=&quot;math-container&quot;&gt;$800. Now you wanted to continue the tour in NY or want to go Washington D.C/ Miami in next day. You need to take action based on several subjects (transportation, time in transport vehicle etc). So, action is either stay or go. what is state possibly? state = {NY, $&lt;/span&gt;1200, tired or not maybe etc...}&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;your &lt;strong&gt;reward function&lt;/strong&gt; design may be trickier to design.&lt;/p&gt;&#xA;&lt;p&gt;I will suggest studying some RL-based real-life problem which will improve understanding more. You can study &lt;a href=&quot;https://towardsdatascience.com/deep-reinforcement-learning-for-automated-stock-trading-f1dad0126a02&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; and try to relate with your problem.&lt;/p&gt;&#xA;" OwnerUserId="42283" LastActivityDate="2020-11-15T17:26:59.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24612" PostTypeId="1" AcceptedAnswerId="24614" CreationDate="2020-11-15T18:36:36.927" Score="2" ViewCount="40" Body="&lt;p&gt;For policy evaluation purposes, can we use the Q-learning algorithm even though, technically, it is meant for control?&lt;/p&gt;&#xA;&lt;p&gt;Maybe like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have the policy to be evaluated as the behaviour policy.&lt;/li&gt;&#xA;&lt;li&gt;Update the Q value conventionally (i.e. updating &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; using the action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; giving highest &lt;span class=&quot;math-container&quot;&gt;$Q(s',a')$&lt;/span&gt; value)&lt;/li&gt;&#xA;&lt;li&gt;The final &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; values will reflect the values for the policy being evaluated.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Am I missing something here, given that I have not seen Q-learning being used anywhere for evaluation purposes?&lt;/p&gt;&#xA;" OwnerUserId="38385" LastEditorUserId="2444" LastEditDate="2020-11-15T23:13:50.333" LastActivityDate="2020-11-15T23:31:42.603" Title="Can we use Q-learning update for policy evaluation (not control)?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;policy-evaluation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24613" PostTypeId="1" AcceptedAnswerId="24631" CreationDate="2020-11-15T19:10:40.963" Score="2" ViewCount="84" Body="&lt;p&gt;Why aren't exploration techniques, such as UCB or Thompson sampling, typically used in bandit problems, used in full RL problems?&lt;/p&gt;&#xA;&lt;p&gt;Monte Carlo Tree Search may use the above-mentioned methods in its selection step, but why do value-based and policy gradient methods not use these techniques?&lt;/p&gt;&#xA;" OwnerUserId="32517" LastEditorUserId="2444" LastEditDate="2020-11-16T23:18:01.557" LastActivityDate="2021-02-11T19:24:18.933" Title="Why aren't exploration techniques, such as UCB or Thompson sampling, used in full RL problems?" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;upper-confidence-bound&gt;&lt;thompson-sampling&gt;&lt;exploration-strategies&gt;" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24614" PostTypeId="2" ParentId="24612" CreationDate="2020-11-15T19:43:24.120" Score="1" Body="&lt;p&gt;For off-policy learning you must have two policies - a behaviour policy and a target policy. If the two policies are the same, then you end up with SARSA, not Q learning.&lt;/p&gt;&#xA;&lt;p&gt;You cannot use Q learning directly for evaluating a fixed target policy, because it directly learns optimal value function as the target policy, regardless of the behaviour policy. Instead you must use another variant of off-policy learning that can evaluate an arbitrary target policy.&lt;/p&gt;&#xA;&lt;p&gt;Your suggested algorithm is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have the policy to be evaluated as the behaviour policy.&lt;/li&gt;&#xA;&lt;li&gt;Update the Q value conventionally (i.e. updating &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; using the action &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; giving highest &lt;span class=&quot;math-container&quot;&gt;$Q(s',a')$&lt;/span&gt; value)&lt;/li&gt;&#xA;&lt;li&gt;The final &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; values will reflect the values for the policy being evaluated.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This will not work for evaluating the behaviour policy. If the behaviour policy was stochastic and covered all possible state/action choices, then it will still be Q learning and converge on the optimal value function - maybe very slowly if the behaviour policy did not get to important states very often.&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;trick&amp;quot; to off-policy is that the environment interaction part uses the behaviour policy to collect data, and the update step uses the target policy to calculate estimated returns. In general for off-policy updates, there can be corrections required to re-weight the estimated returns. However, one nice thing about single-step TD methods is that there are no such additional corrections needed.&lt;/p&gt;&#xA;&lt;p&gt;So this gives a way to do off-policy TD learning, using an approach called &lt;a href=&quot;https://www.geeksforgeeks.org/expected-sarsa-in-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expected SARSA&lt;/a&gt;. To use Expected SARSA, you will need to know the distribution of action choices i.e. know &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; for the target policy.&lt;/p&gt;&#xA;&lt;p&gt;This is the variant of your description that will work to evaluate your target policy &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have any stochastic policy that &amp;quot;covers&amp;quot; the target policy as the behaviour policy.&lt;/li&gt;&#xA;&lt;li&gt;Update the Q value using Expected SARSA &lt;span class=&quot;math-container&quot;&gt;$Q(s,a) = Q(s,a) + \alpha(r + \gamma [\sum_{a'} \pi(a'|s')Q(s',a')]  - Q(s,a))$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;The final &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; values will reflect the values for the policy being evaluated.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Worth noting that Expected SARSA with a target policy of &lt;span class=&quot;math-container&quot;&gt;$\pi(s) = \text{argmax}_a Q(s,a)$&lt;/span&gt; is exactly Q learning. Expected SARSA is a strict generalisation of Q learning that allows for learning the value function of &lt;em&gt;any&lt;/em&gt; target policy. You may not see it used as much as Q learning, because the goal of learning an optimal value function is more common in practice.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-15T23:31:42.603" LastActivityDate="2020-11-15T23:31:42.603" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24615" PostTypeId="1" CreationDate="2020-11-15T21:49:14.033" Score="2" ViewCount="27" Body="&lt;p&gt;I'm training an RL agent using the DQN algorithm to do a specific task. The environment is represented by a list of &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; integer numbers from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$20$&lt;/span&gt;. An example would be &lt;span class=&quot;math-container&quot;&gt;$[5, 15, 8, 8, 0, \dots]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is it okay to pass the list as floats to the dense layer, or would that impede the learning process? What is the right way to go about passing integer numbers to the neural network?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-11-15T22:57:14.060" LastActivityDate="2020-11-15T22:57:14.060" Title="What is the appropriate way of passing a list of integers that represents the environment to a neural network's dense layer?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;environment&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24616" PostTypeId="1" CreationDate="2020-11-15T22:22:17.627" Score="3" ViewCount="28" Body="&lt;p&gt;When studying bounding box-based detectors, it's not clear to me if data augmentation includes adding random rotations.&lt;/p&gt;&#xA;&lt;p&gt;If random rotations are added, how is the new bounding box calculated?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2020-11-15T22:43:06.533" LastActivityDate="2020-11-15T22:43:06.533" Title="If random rotations are included in the data augmentation process, how are the new bounding boxes calculated?" Tags="&lt;deep-learning&gt;&lt;object-detection&gt;&lt;object-recognition&gt;&lt;yolo&gt;&lt;data-augmentation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24617" PostTypeId="1" CreationDate="2020-11-15T22:33:34.770" Score="2" ViewCount="22" Body="&lt;p&gt;My agent receives &lt;span class=&quot;math-container&quot;&gt;$1, 0, -1$&lt;/span&gt; rewards for winning, drawing, and losing the game, respectively. What would be the consequences of setting reward to &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; for draws? Would that encourage the agent to win more or will it have no effect at all? Is it appropriate to do so?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-11-15T22:50:30.093" LastActivityDate="2020-11-15T22:50:30.093" Title="How can I discourage the RL agent from drawing in a zero-sum game?" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;zero-sum-games&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24618" PostTypeId="5" CreationDate="2020-11-15T22:51:02.033" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T22:51:02.033" LastActivityDate="2020-11-15T22:51:02.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24619" PostTypeId="4" CreationDate="2020-11-15T22:51:02.033" Score="0" Body="For questions about zero-sum games in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T22:51:02.033" LastActivityDate="2020-11-15T22:51:02.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24620" PostTypeId="5" CreationDate="2020-11-15T23:06:40.530" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:06:40.530" LastActivityDate="2020-11-15T23:06:40.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24621" PostTypeId="4" CreationDate="2020-11-15T23:06:40.530" Score="0" Body="For questions about the Trust Region Policy Optimization (TRPO) algorithm, which was introduced in the paper &quot;Trust Region Policy Optimization&quot; (2015) by J. Schulman et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:06:40.530" LastActivityDate="2020-11-15T23:06:40.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24622" PostTypeId="5" CreationDate="2020-11-15T23:09:37.107" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:09:37.107" LastActivityDate="2020-11-15T23:09:37.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24623" PostTypeId="4" CreationDate="2020-11-15T23:09:37.107" Score="0" Body="For questions about exploration strategies (or techniques) used in reinforcement learning or bandit problems. Examples of exploration strategies are random strategy, $\epsilon$-greedy, greedy (no exploration), Upper Confidence Bound (UCB), or Thompson sampling." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:09:37.107" LastActivityDate="2020-11-15T23:09:37.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24624" PostTypeId="5" CreationDate="2020-11-15T23:11:35.130" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:11:35.130" LastActivityDate="2020-11-15T23:11:35.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24625" PostTypeId="4" CreationDate="2020-11-15T23:11:35.130" Score="0" Body="For questions about Thompson sampling, which is a technique for choosing actions (that addresses the exploration-exploitation dilemma) in the multi-armed bandit and reinforcement learning problems." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:11:35.130" LastActivityDate="2020-11-15T23:11:35.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24626" PostTypeId="5" CreationDate="2020-11-15T23:38:21.400" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:38:21.400" LastActivityDate="2020-11-15T23:38:21.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24627" PostTypeId="4" CreationDate="2020-11-15T23:38:21.400" Score="0" Body="For questions about continuous action spaces in the context of reinforcement learning (or other artificial intelligence sub-fields). There is also the tag for discrete action spaces." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:38:21.400" LastActivityDate="2020-11-15T23:38:21.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24628" PostTypeId="5" CreationDate="2020-11-15T23:39:51.850" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:39:51.850" LastActivityDate="2020-11-15T23:39:51.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24629" PostTypeId="4" CreationDate="2020-11-15T23:39:51.850" Score="0" Body="For questions about discrete action spaces in the context of reinforcement learning (or other artificial intelligence sub-fields). There is also the tag for continuous action spaces." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-15T23:39:51.850" LastActivityDate="2020-11-15T23:39:51.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24630" PostTypeId="2" ParentId="24576" CreationDate="2020-11-16T00:07:38.323" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is average pooling equivalent to a strided convolution with a specific constant kernel?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why then are explicit pooling layers needed if they can be realized by convolutions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is probably because the convolution is more expensive than the usual/natural implementation (i.e. just summing and then dividing).&lt;/p&gt;&#xA;&lt;p&gt;To see why, let's consider your example. If you implement pooling with a convolution (or cross-correlation), we would need to perform &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; multiplications, then &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; summations, for a total of &lt;span class=&quot;math-container&quot;&gt;$17$&lt;/span&gt; operations. If we implement pooling as usual, we would need to perform &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; summations and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; division (or multiplication), for a total of &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; operations.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, convolution may also be more prone to numerical instability (multiplications of numbers in the range &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt; are not nice), but I am not completely sure about this, given that we multiply always by the same numbers.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T00:16:17.420" LastActivityDate="2020-11-16T00:16:17.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24631" PostTypeId="2" ParentId="24613" CreationDate="2020-11-16T00:36:57.387" Score="3" Body="&lt;p&gt;You can indeed use UCB in the RL setting. See e.g. section &lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf#page=530&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;38.5 Upper Confidence Bounds for Reinforcement Learning&lt;/strong&gt; (page 521)&lt;/a&gt; of the book &lt;a href=&quot;https://tor-lattimore.com/downloads/book/book.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bandit Algorithms&lt;/a&gt; by Csaba Szepesvari and Tor Lattimore for the details.&lt;/p&gt;&#xA;&lt;p&gt;However, compared to &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy (widely used in RL), UCB1 is more computationally expensive, given that, for each action, you need to recompute this upper confidence bound &lt;strong&gt;for every time step&lt;/strong&gt; (or, equivalently, action taken during learning).&lt;/p&gt;&#xA;&lt;p&gt;To see why, let's take a look at &lt;a href=&quot;https://homes.di.unimi.it/cesa-bianchi/Pubblicazioni/ml-02.pdf#page=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;the UCB1 formula&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\underbrace{\bar{x}_{j}}_{\text{value estimate}}+\underbrace{\sqrt{\frac{2 \ln n}{n_{j}}}}_{\text{UCB}},&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\bar{x}_{j}$&lt;/span&gt; is the value estimate for action &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n_{j}$&lt;/span&gt; is the number of times action &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; has been taken&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the total number of actions taken so far&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, at each time step (or new action taken), we need to recompute that square root &lt;em&gt;for each action&lt;/em&gt;, which depends on other factors that evolve during learning.&lt;/p&gt;&#xA;&lt;p&gt;So, the higher time complexity than &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy is probably the first reason why UCB1 is not so much used in RL, where interaction with the environment can be the bottleneck. You could argue that this recomputation (for each action) also needs to be done in bandits. Yes, it's true, but, in the RL problem, you have multiple states, so you need to compute value estimates for each action in all states (i.e. the full RL problem is more complex than bandits or contextual bandits).&lt;/p&gt;&#xA;&lt;p&gt;Moreover, &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy is so conceptually simple that everyone can easily implement it in less than &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; minutes (though this is not really a problem, given that both are simple to implement).&lt;/p&gt;&#xA;&lt;p&gt;I am currently not familiar with Thompson sampling, but I guess (from some implementations I have seen) it's also not as cheap as &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy, where you just need to perform an argmax (can be done in constant time if you keep track of the highest value) or sample a random integer (it's also relatively cheap). There's a tutorial on Thompson sampling &lt;a href=&quot;https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, which also includes a section dedicated to RL, so you may want to read it.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-11T18:02:08.677" LastActivityDate="2021-02-11T18:02:08.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24632" PostTypeId="2" ParentId="10839" CreationDate="2020-11-16T00:39:33.710" Score="0" Body="&lt;p&gt;The reason is very simple. It is due to the cost function. For instance, for making the binary cross-entropy cost function, we discard the joint probability by considering the i.i.d condition. So we will have products of marginal probabilities. The final result will be the sum that we have in the binary cross-entropy cost function after applying the logarithm.&lt;/p&gt;&#xA;&lt;p&gt;Update due to the request:&lt;/p&gt;&#xA;&lt;p&gt;In a dataset, suppose we have n examples. Our goal is to maximize the probability of examples to happen. Consequently, we demand to maximize the following probability:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x_1, x_2, ..., x_n)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As it can be observed, the joint probability is hard to be interpreted. The point is that it is hard to be calculated. Suppose each example is binary. Then the probability distribution will need &lt;span class=&quot;math-container&quot;&gt;$2^n$&lt;/span&gt; number of values to be defined properly.&lt;/p&gt;&#xA;&lt;p&gt;If we sample the examples i.i.d the above distribution becomes the product of marginal probabilities:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(x_1, ..., x_n)=\prod_{i}P(x_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above probability is far easier to compute.&#xA;We tend to maximize this probability. Due to the fact that logarithm is a strictly increasing function, we can take the logarithm of the above formula for numerical stability, because while multiplying many numbers, the outcome can become very small or big soon.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, we try to minimize the negative of the maximization problem. It is more common among optimization practitioners.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$log P(x_1, ..., x_n)=\sum_i{log(P(x_i))}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Cost=-\sum_i{log(P(x_i))}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is our goal to maximize the likelihood.&lt;/p&gt;&#xA;&lt;hr&gt;&#xA;&lt;p&gt;The probability distribution is a Bernoulli distribution for binary classification problems and in neural networks, we demand to find &lt;span class=&quot;math-container&quot;&gt;$P(y=1|x)$&lt;/span&gt; for each example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(y=1|x)=\hat{p}^y(1-\hat{p})^{(1-y)}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat{p}$&lt;/span&gt; is the output of the network. Our goal is to maximize the probability of all the dataset. Like above, we minimize the cost function which will be like the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$cost=-\sum_i{y^{(i)}log(\hat{y}^{(i)}) + (1-y^{(i)})log(1-\hat{y}^{(i)})}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="11599" LastEditorUserId="11599" LastEditDate="2020-11-16T10:55:25.050" LastActivityDate="2020-11-16T10:55:25.050" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24633" PostTypeId="2" ParentId="24574" CreationDate="2020-11-16T02:30:34.983" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is the second binary plane all zeros or all ones? Or, something else? How is it known if the move is off the board? For my game, I know if it is a legal move on the board, but do not know if the move is off the board.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The second binary plane is one-hot by definition, there is a single one and everything else is zero. If this definition is not met, it's no longer &amp;quot;one-hot&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The paper doesn't state how exactly to implement &amp;quot;off the board&amp;quot; implementation. Research paper wouldn't go into coding level. However, detecting &amp;quot;off the board&amp;quot; is not a challenging task.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://webcache.googleusercontent.com/search?q=cache:djj-G4T_PwgJ:https://craftychess.com/hyatt/boardrep.html+&amp;amp;cd=2&amp;amp;hl=en&amp;amp;ct=clnk&amp;amp;gl=au&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://webcache.googleusercontent.com/search?q=cache:djj-G4T_PwgJ:https://craftychess.com/hyatt/boardrep.html+&amp;amp;cd=2&amp;amp;hl=en&amp;amp;ct=clnk&amp;amp;gl=au&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The next step in board representation evolution is to enclose the board inside a larger array, so that illegal squares are &amp;quot;off&amp;quot; the edge and are easily detectable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A possibility is add borders to your board. Crafty did that. Extend the board to 10x10. Not 9x9 because you need to deal with knight jumping.&lt;/p&gt;&#xA;&lt;p&gt;Exactly how you should be doing it is implementation defined. We don't know what Google did because AlphaZero is not open source. I'm here just giving you an example.&lt;/p&gt;&#xA;" OwnerUserId="6014" LastActivityDate="2020-11-16T02:30:34.983" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24634" PostTypeId="1" CreationDate="2020-11-16T07:56:04.370" Score="0" ViewCount="37" Body="&lt;p&gt;I am solving a problem for which I have to select the best possible servers (level 1) to hit for a given data. These servers (level 1) in turn hit some other servers (level 2) to complete the request. The level 1 servers have the same set of level 2 servers integrated with them. For a particular request, I am getting success or failure as a response.&lt;/p&gt;&#xA;&lt;p&gt;For this, I am using Thompson Sampling with Bernoulli prior. On success, I am considering reward as 1 and, for failure, it is 0. But in case of failure, I am receiving errors as well. In some error, it is evident that the error is due to some issue at the server (level 1) end, and hence reward 0 makes sense, but some error results from request data errors or issue at level 2 servers. For these kinds of errors, we can't penalize the level 1 servers with reward 0 nor can we reward them with value 1.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I am using 0.5 as a reward for such cases.&lt;/p&gt;&#xA;&lt;p&gt;Exploring over the Internet, I couldn't find any method/algorithm to calculate the reward for such cases in a proper (informed) way.&lt;/p&gt;&#xA;&lt;p&gt;What could be the possible way to calculate reward in such cases?&lt;/p&gt;&#xA;" OwnerUserId="42313" LastEditorUserId="2444" LastEditDate="2020-11-17T10:48:37.620" LastActivityDate="2020-11-17T10:48:37.620" Title="Thompson sampling with Bernoulli prior and non-binary reward update" Tags="&lt;reinforcement-learning&gt;&lt;reward-design&gt;&lt;reward-functions&gt;&lt;bayesian-statistics&gt;&lt;thompson-sampling&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24635" PostTypeId="1" CreationDate="2020-11-16T09:11:04.630" Score="0" ViewCount="27" Body="&lt;p&gt;I have database of sequential events for multiple animals.&#xA;The events are represented by integers so it looks something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Animal A: [1,6,4,2,5,7,8] &#xA;Animal B: [1,6,5,4,1,6,7]&#xA;Animal C: [5,4,2,1,6,4,3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can manually see that for each event 6 event 1 first happens.&#xA;And event 4 happens quickly after a 1,6 combination.&#xA;But these are easy to spot in such a small dataset, the real lists are 10000+ events per animal.&#xA;Is there a way to use an algorithm or machine learning to search for these kinds of patterns?&lt;/p&gt;&#xA;" OwnerUserId="42315" LastActivityDate="2020-11-16T09:11:04.630" Title="Find repeating patterns in sequence data" Tags="&lt;machine-learning&gt;&lt;pattern-recognition&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24636" PostTypeId="1" CreationDate="2020-11-16T09:49:00.387" Score="0" ViewCount="33" Body="&lt;p&gt;In my project, I am detecting only one class, which is &amp;quot;airplane&amp;quot;, using yolov5. However, at some frames, the neural network labels some of the buildings as airplanes, which obviously are not. This noise happens like 1 frame among 60 frames. How should I treat this issue? Which algorithms can be applied to filter out?&lt;/p&gt;&#xA;" OwnerUserId="42311" LastEditorUserId="2444" LastEditDate="2020-11-16T12:16:35.347" LastActivityDate="2020-11-16T12:16:35.347" Title="Object detection noise filtering" Tags="&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="24637" PostTypeId="1" CreationDate="2020-11-16T11:24:55.860" Score="1" ViewCount="41" Body="&lt;p&gt;I have come up with some examples of CNNs (segmentation CNNs) that use ELU (exponential linear unit) as an activation function.&lt;/p&gt;&#xA;&lt;p&gt;What are the benefits of this activation function over others, such as RELU or leaky RELU?&lt;/p&gt;&#xA;" OwnerUserId="42296" LastEditorUserId="2444" LastEditDate="2020-11-16T11:55:03.730" LastActivityDate="2020-11-16T11:55:03.730" Title="What are the benefits of using ELU over other activation functions in CNNs?" Tags="&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;relu&gt;&lt;leaky-relu&gt;&lt;elu&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24638" PostTypeId="5" CreationDate="2020-11-16T11:50:16.703" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T11:50:16.703" LastActivityDate="2020-11-16T11:50:16.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24639" PostTypeId="4" CreationDate="2020-11-16T11:50:16.703" Score="0" Body="For questions about the Exponential Linear Unit (ELU) activation function that can be used in neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T11:50:16.703" LastActivityDate="2020-11-16T11:50:16.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24640" PostTypeId="5" CreationDate="2020-11-16T11:52:16.610" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T11:52:16.610" LastActivityDate="2020-11-16T11:52:16.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24641" PostTypeId="4" CreationDate="2020-11-16T11:52:16.610" Score="0" Body="For questions about the Leaky ReLU activation function, which can be used in neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T11:52:16.610" LastActivityDate="2020-11-16T11:52:16.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24642" PostTypeId="1" AcceptedAnswerId="24730" CreationDate="2020-11-16T11:59:03.140" Score="1" ViewCount="77" Body="&lt;p&gt;I am currently studying game theory based on Peter Norvig's 3rd edition introduction to artificial intelligence book. In chapter 17.5, the two player zero sum game can be solved by using the &lt;span class=&quot;math-container&quot;&gt;$\textbf{minimax}$&lt;/span&gt; theorem&#xA;&lt;span class=&quot;math-container&quot;&gt;$$max_x \, min_y \, x^TAy = min_x \, max_y \, x^TAy = v$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is  the probability distribution of actions by the max player (in the left equation) and the min player (in the right equation).&lt;/p&gt;&#xA;&lt;p&gt;Regarding the minimax theorem, I have 2 questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do both the min and the max players have the same probability distribution of actions ? In the book by Peter Norvig, the book demonstrated that in the game of &lt;span class=&quot;math-container&quot;&gt;$\textbf{Morra}$&lt;/span&gt;, both the min and max player had &lt;span class=&quot;math-container&quot;&gt;$[\frac{7}{12}:one, \frac{5}{12}:two]$&lt;/span&gt; for the probability distributions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, regarding minimax game tree, is the difference between minimax game tree and the zero sum game the fact that for minimax game tree, the opponent can react to the first player's move whereas for zero sum game defined in 17.5, both players are unaware of each other's move ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-12-19T16:02:00.150" Title="Optimal mixed strategy in two player zero sum games" Tags="&lt;minimax&gt;&lt;game-theory&gt;&lt;zero-sum-games&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24643" PostTypeId="1" CreationDate="2020-11-16T12:07:39.813" Score="4" ViewCount="80" Body="&lt;p&gt;What is the reason behind the name &amp;quot;Transformers&amp;quot;, for Multi Head Self-Attention-based neural networks from &lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is All You Need&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I have been googling this question for a long time, and nowhere I can find any explanation.&lt;/p&gt;&#xA;" OwnerUserId="26580" LastEditorUserId="26580" LastEditDate="2020-11-16T12:46:19.963" LastActivityDate="2021-03-16T12:58:58.363" Title="Why are &quot;Transformers&quot; called this way?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24645" PostTypeId="5" CreationDate="2020-11-16T12:41:42.340" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T12:41:42.340" LastActivityDate="2020-11-16T12:41:42.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24646" PostTypeId="4" CreationDate="2020-11-16T12:41:42.340" Score="0" Body="For questions about the concept of independent and identically distributed (often shortened as i.i.d., iid or IID) random variables (or samples) and assumption, which is often made in statistics and machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T12:41:42.340" LastActivityDate="2020-11-16T12:41:42.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24647" PostTypeId="1" CreationDate="2020-11-16T15:01:33.420" Score="2" ViewCount="74" Body="&lt;p&gt;When learning off-policy with multi-step returns, we want to update the value of &lt;span class=&quot;math-container&quot;&gt;$Q(s_1, a_1)$&lt;/span&gt; using rewards from the trajectory &lt;span class=&quot;math-container&quot;&gt;$\tau = (s_1, a_1, r_1, s_2, a_2, r_2, ..., s_n, a_n, r_n, s_n+1)$&lt;/span&gt;. We want to learn the target policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; while behaving according to policy &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;. Therefore, for each transition &lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, r_t, s_{t+1})$&lt;/span&gt;, we apply the importance ratio &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi(a_t | s_t)}{\mu(a_t | s_t)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is: if we are training at every step, the behavior policy may change at each step and therefore the transitions of the trajectory &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; are not obtained from the current behavior policy, but from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; behavior policies. Why do we use the current behavior policy in the importance sampling? Should each transition use the probability of the behavior policy of the timestep at which that transition was collected? For example by storing the likelihood &lt;span class=&quot;math-container&quot;&gt;$\mu_t(a_t | s_t)$&lt;/span&gt; along with the transition?&lt;/p&gt;&#xA;" OwnerUserId="32583" LastEditorUserId="2444" LastEditDate="2020-11-17T10:56:15.383" LastActivityDate="2020-11-18T06:29:39.553" Title="When learning off-policy with multi-step returns, why do we use the current behaviour policy in importance sampling?" Tags="&lt;reinforcement-learning&gt;&lt;off-policy-methods&gt;&lt;value-functions&gt;&lt;importance-sampling&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24648" PostTypeId="1" AcceptedAnswerId="24684" CreationDate="2020-11-16T15:03:16.577" Score="1" ViewCount="53" Body="&lt;p&gt;I was trying Google Cloud's Vision API, and how the dominant colors part shows. I uploaded a sample image, and here is the results for the dominant colors. I realized it doesn't simply count pixel colors and cluster them. The background has many gray pixels which are not included.&lt;/p&gt;&#xA;&lt;p&gt;How does it perform dominant colors? How can I do something similar?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wjXTJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wjXTJ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastActivityDate="2020-11-17T23:53:47.050" Title="How to compute dominant colors in an image?" Tags="&lt;machine-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;image-processing&gt;&lt;google-cloud&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24649" PostTypeId="1" AcceptedAnswerId="24665" CreationDate="2020-11-16T15:41:37.780" Score="1" ViewCount="33" Body="&lt;p&gt;I have a dataset of 3000 8x8 images, and I would like to train a GAN for an image generation purpose.&lt;/p&gt;&#xA;&lt;p&gt;I am planning to start with a simple GAN model and see if it overfits. Before training, I try to do a comparison of the discriminator model prediction using real image input against the whole GAN model prediction using random seed input. My thought process is that since this model is not trained yet, the output for real images and fake images by the discriminator should not be predictable.&lt;/p&gt;&#xA;&lt;p&gt;However, the discriminator model prediction using real image input always returns a value very close to &lt;code&gt;1.0&lt;/code&gt;, and the whole GAN model prediction using random seed input always returns a value near &lt;code&gt;0.5&lt;/code&gt; with a small deviation. I suspect that during training, the model would simply pull the &lt;code&gt;0.5&lt;/code&gt; value near &lt;code&gt;0.0&lt;/code&gt; and would never actually learn from the dataset.&lt;/p&gt;&#xA;&lt;p&gt;I try to increase the training parameters and different initializers, but the output is still the same.&lt;/p&gt;&#xA;&lt;p&gt;By ruling out the possibility a bad dataset, what could be the reason for this situation?&lt;/p&gt;&#xA;&lt;p&gt;This is some sneak peek of the generator and discriminator model building: &lt;a href=&quot;https://pastebin.com/ehMDP7k6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/ehMDP7k6&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32511" LastEditorUserId="32511" LastEditDate="2020-11-17T11:20:37.703" LastActivityDate="2020-11-17T15:42:02.333" Title="GAN model predictions before training is predictable" Tags="&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24650" PostTypeId="1" CreationDate="2020-11-16T15:57:14.587" Score="0" ViewCount="50" Body="&lt;p&gt;I believe that Bengio is on record as describing Deep Learning as &amp;quot;A collection of methods for delivering intelligence&amp;quot; (or something very similar).&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have an actual reference for this?&lt;/p&gt;&#xA;" OwnerUserId="42" LastActivityDate="2020-11-16T15:57:14.587" Title="Reference for Yoshua Bengio quote" Tags="&lt;deep-learning&gt;&lt;agi&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24651" PostTypeId="1" CreationDate="2020-11-16T15:58:10.880" Score="2" ViewCount="19" Body="&lt;p&gt;I am trying to train a neural network using reinforcement learning / policy gradient methods. The states, i.e. the inputs, as well as the actions I am trying to sample are vectors with each element being a real number like in this question: &lt;a href=&quot;https://math.stackexchange.com/questions/3179912/policy-gradient-reinforcement-learning-for-continuous-state-and-action-space&quot;&gt;https://math.stackexchange.com/questions/3179912/policy-gradient-reinforcement-learning-for-continuous-state-and-action-space&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The answer that was given there already helped me a lot. Also, I have been trying to understand Chapter 13: Policy Gradient Methods from &amp;quot;Reinforcement Learning: An Introduction&amp;quot; by Sutton et al. and in particular Section 13.7: Policy Parametrization for Continous Actions.&lt;/p&gt;&#xA;&lt;p&gt;My current level of understanding that I can use the weights in the network to calculate the mean/means and the standard deviation/covariance matrix. I can then use them to define a multivariate Gaussian distribution and randomly sample an action from there.&lt;/p&gt;&#xA;&lt;p&gt;For now, I have one main question: In the book it says, that I have to split the weights, i.e. the policy's parameter vector, into two parts: &lt;span class=&quot;math-container&quot;&gt;$\theta = [\theta_{\mu}, \theta_{\sigma}]$&lt;/span&gt;. I can then use each part together with a feature vector to calculate the means and the covariance matrix. However, I was wondering, how this is usually done? Do I train two separate networks? I am not sure how an architecture of this will look like. Also, I am not sure what the output nodes will be in this case. Do they have any meaning like for supervised learning?&lt;/p&gt;&#xA;&lt;p&gt;So far, I have only found papers that talk about this issue rather theoretically like it is presented in the book. I would be very happy to understand how this is actually implemented. Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="42323" LastActivityDate="2020-11-16T15:58:10.880" Title="Understanding neural network achitectures in policy gradient reinforcement learning for continuous state and action space" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24652" PostTypeId="2" ParentId="12870" CreationDate="2020-11-16T16:56:56.983" Score="0" Body="&lt;p&gt;Jacques Pitrat's last book  &lt;a href=&quot;https://www.kobo.com/us/en/ebook/artificial-beings&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings&#xA;The Conscience of a Conscious Machine&lt;/em&gt;&lt;/a&gt;  has some interesting chapters (§2, §3, §5, §6, §7) related to the questions of explanations and meta-explanations in AI systems.&lt;/p&gt;&#xA;&lt;p&gt;It describes a reflexive, &amp;quot;expert system&amp;quot; like, meta-knowledge based approach to explainable AI&lt;/p&gt;&#xA;&lt;p&gt;You could also read his papers&#xA;&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/0167739X96000118&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Implementation of a reflective system&lt;/em&gt;&lt;/a&gt; (1996)&#xA;and &lt;a href=&quot;http://jacques.pitrat.pagesperso-orange.fr/A%20Step%20toward%20an%20Artificial%20AI%20Scientist.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A Step toward an Artificial AI Scientist&lt;/em&gt;&lt;/a&gt; online (there could be a typo in it: &amp;quot;pile&amp;quot; is the French word for stack, including the call stack).&lt;/p&gt;&#xA;&lt;p&gt;You might also look into J.Pitrat's &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; and into the ongoing &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; project (it is in November 2020).&lt;/p&gt;&#xA;&lt;p&gt;PS. J.Pitrat (born in 1934) passed away in October 2019. French readers could see &lt;a href=&quot;https://afia.asso.fr/journee-hommage-j-pitrat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. His blog might disappear in a few months.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-11-19T12:26:22.510" LastActivityDate="2020-11-19T12:26:22.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24653" PostTypeId="5" CreationDate="2020-11-16T18:25:06.880" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T18:25:06.880" LastActivityDate="2020-11-16T18:25:06.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24654" PostTypeId="4" CreationDate="2020-11-16T18:25:06.880" Score="0" Body="For questions about state spaces, in the context of reinforcement learning or other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T18:25:06.880" LastActivityDate="2020-11-16T18:25:06.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24655" PostTypeId="5" CreationDate="2020-11-16T18:25:26.540" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T18:25:26.540" LastActivityDate="2020-11-16T18:25:26.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24656" PostTypeId="4" CreationDate="2020-11-16T18:25:26.540" Score="0" Body="For questions about continuous state spaces, in the context of reinforcement learning or other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T18:25:26.540" LastActivityDate="2020-11-16T18:25:26.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24657" PostTypeId="5" CreationDate="2020-11-16T18:25:40.070" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T18:25:40.070" LastActivityDate="2020-11-16T18:25:40.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24658" PostTypeId="4" CreationDate="2020-11-16T18:25:40.070" Score="0" Body="For questions about discrete state spaces, in the context of reinforcement learning or other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-16T18:25:40.070" LastActivityDate="2020-11-16T18:25:40.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24659" PostTypeId="1" CreationDate="2020-11-17T05:40:58.907" Score="1" ViewCount="28" Body="&lt;p&gt;&lt;a href=&quot;https://www.pyimagesearch.com/2020/11/16/gans-with-keras-and-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is the tutorial that I used to learn about GANs. In this tutorial, it taught us to intentionally provide false labels to &amp;quot;fool&amp;quot; the discriminator, but does it make the discriminator actually inaccurate? I don't quite understand his explanation, can anyone help me?&lt;/p&gt;&#xA;" OwnerUserId="42182" LastEditorUserId="2444" LastEditDate="2020-11-17T11:00:34.007" LastActivityDate="2020-11-17T11:00:34.007" Title="Why do we need to provide false labels to the discriminator on purpose to train GANs?" Tags="&lt;computer-vision&gt;&lt;training&gt;&lt;keras&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24660" PostTypeId="1" CreationDate="2020-11-17T05:58:44.927" Score="0" ViewCount="11" Body="&lt;p&gt;I'm trying to make news classification. Here is the neural network:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y = dataset.iloc[:, vocabularySize]&#xA;matrix = CountVectorizer(max_features=vocabularySize)&#xA;X = matrix.fit_transform(data).toarray()&#xA;X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)  &#xA;&#xA;classifier = GaussianNB()&#xA;&#xA;classifier.fit(X_train, y_train)&#xA;y_pred = classifier.predict(X_test)&#xA;&#xA;accuracy = accuracy_score(y_test, y_pred)&#xA;print(f'accuracy: {accuracy}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;y is the column of news categories. And x is the vectors of my news contents. Code is working fine and it makes prediction on the test group. But I want to predict the category of the specific news with this trained neural network.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;There is my test group: 301321110301321033123232123223313131232033031000303020 &#xA;&#xA;There is predictions of them: [2 0 1 2 2 1 1 1 1 3 0 0 1 2 0 1 3 1 1 2 3 2 3 2 1 2 3 2 2 0 1 1 3 1 3 1 2 1 2 0 3 2 1 3 0 0 0 0 1 0 3 0 3 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I want to give a random news as input and want it's predicted category. But it works on big data which can separate to train and test groups. How can I predict a single news category? It takes my all news data set and separate them train and test but I want to do fit this neural network with train group but give a any news as test.&lt;/p&gt;&#xA;" OwnerUserId="42274" LastEditorUserId="41060" LastEditDate="2020-11-17T09:52:14.243" LastActivityDate="2020-11-17T09:52:14.243" Title="How to make specific test data prediction with fitted GaussianNB Classifier in Python" Tags="&lt;natural-language-processing&gt;&lt;classification&gt;&lt;python&gt;&lt;text-classification&gt;&lt;naive-bayes&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24661" PostTypeId="2" ParentId="24659" CreationDate="2020-11-17T08:00:59.133" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;in this tutorial, it taught us to intentionally provide false labels to &amp;quot;fool&amp;quot; the discriminator, does it make discriminator actually inaccurate?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When training GANs, the training steps for the generator and discriminator are separate:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a training stage for the discriminator, where it is presented with a mix of generated and real data, all correctly labelled. It is important at this stage to not update the generator (otherwise it will get worse by helping to make the fake data look more fake).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a training stage for the generator, where the &lt;em&gt;discriminator&lt;/em&gt; is presented with only generated data, all labelled incorrectly as if it were real. It is important at this stage to not update the discriminator.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Typically you will alternate between the two stages frequently, making small updates to discriminator and generator separately. Some GANs use metrics to decide how much of each to do, because you don't want either the generator or discriminator to win outright and stop progress - at least at the start.&lt;/p&gt;&#xA;&lt;p&gt;So yes there is a stage where you deliberately &amp;quot;fool&amp;quot; the discriminator, because being able to do so is the goal of the generator. However, one key detail of this stage is that the discriminator weights are not updated from that faked data. Instead the gradients from that stage are used only to update and improve the generator.&lt;/p&gt;&#xA;&lt;p&gt;It may help if you don't think of the false labels as being &amp;quot;fool the discriminator&amp;quot;, but instead they are &amp;quot;measure how well the generator is fooling the discriminator&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-17T09:00:49.187" LastActivityDate="2020-11-17T09:00:49.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24663" PostTypeId="1" CreationDate="2020-11-17T11:09:16.283" Score="0" ViewCount="10" Body="&lt;p&gt;I am working on a project where I have to first classify the Subjects of the given question and then the respective Chapter and then the sub-topic. In a nutshell, I have to predict the Subject, Grade and Chapter Name of a Given text. I have trained my &lt;strong&gt;Subject&lt;/strong&gt; Classification model with &lt;strong&gt;BERT last 40% trainable + Focal Loss + Class Weights + 4 output classes&lt;/strong&gt; and got around &lt;code&gt;0.91 f-1 macro&lt;/code&gt; as there is imbalance in my dataset. The next problem is to classify the Chapter name of the question.&lt;/p&gt;&#xA;&lt;p&gt;So my problems have tokens like &lt;code&gt;fe(iron), kj (kilo joule), ligand, pascal, ppm&lt;/code&gt; which are subjject specific and I can not afford to discard any info. Please tell me if&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Can I use FastText, GloVe or some other embeddings because I don't think they'll able to capture these things? Or should I use my own Embeddings?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Which one of the above models in your opinions do you think will give me good results? I don't think if BERT is capable of capturing the subjject wise info or should I train my Attention/Self-Attention/Transformer model from scratch?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am using a dataset of 120000 text questions. Please suggest some strategy.&lt;/p&gt;&#xA;" OwnerUserId="36062" LastActivityDate="2020-11-17T11:09:16.283" Title="Best strategy for Classification of Science Subjects. Phy, Chem , Maths and Bio? BERT, Transformers, Attention+SLTM, Self-Attention+LSTM?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24665" PostTypeId="2" ParentId="24649" CreationDate="2020-11-17T12:37:24.407" Score="2" Body="&lt;p&gt;I took a look at your &lt;a href=&quot;https://pastebin.com/ehMDP7k6&quot; rel=&quot;nofollow noreferrer&quot;&gt;model&lt;/a&gt;. It seems you have incorrect architecture. The &lt;code&gt;Conv2D&lt;/code&gt; layers in your D should have following params: &lt;code&gt;(n_filters, kernel=3, padding='same')&lt;/code&gt;, where &lt;code&gt;n_filters&lt;/code&gt; is the number of filters and it usually should be doubled as per &lt;a href=&quot;https://arxiv.org/abs/1511.06434&quot; rel=&quot;nofollow noreferrer&quot;&gt;DCGAN&lt;/a&gt; architecture. You can also use &lt;code&gt;strides&lt;/code&gt; but since your images are small it won't make any sense.&lt;/p&gt;&#xA;&lt;p&gt;The D network also should not include &lt;code&gt;UpSampling2D&lt;/code&gt; layer. This layer can be used in G network instead of &lt;code&gt;Conv2DTranspose&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Your images should be normalized to the range of &lt;code&gt;[-1, 1]&lt;/code&gt; and the activation function of G should be &lt;code&gt;tanh&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You also included &lt;code&gt;m.add(keras.layers.Reshape((8, 8, 3)))&lt;/code&gt; to your G network which is the final size of your data. Since you have the final size in the first layers you don't have to include upsampling &lt;code&gt;Conv2DTranspose&lt;/code&gt; layers.&lt;/p&gt;&#xA;&lt;p&gt;Your final model should look like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def make_discriminator_model():&#xA;    model = tf.keras.Sequential()&#xA;    model.add(layers.Conv2D(8, kernel=3, strides=1, padding='same',&#xA;                                     input_shape=[8, 8, 1]))&#xA;    model.add(layers.LeakyReLU())&#xA;    model.add(layers.Dropout(0.3))&#xA;&#xA;    model.add(layers.Conv2D(16, kernel=3, strides=1, padding='same'))&#xA;    model.add(layers.LeakyReLU())&#xA;    model.add(layers.Dropout(0.3))&#xA;&#xA;    model.add(layers.Flatten())&#xA;    model.add(layers.Dense(1))&#xA;&#xA;    return model&#xA;&#xA;def make_generator_model():&#xA;    model = tf.keras.Sequential()&#xA;    model.add(layers.Dense(8*8*16, use_bias=False, input_shape=(100,)))&#xA;    model.add(layers.BatchNormalization())&#xA;    model.add(layers.LeakyReLU())&#xA;&#xA;    model.add(layers.Reshape((8, 8, 16)))&#xA;&#xA;    model.add(layers.Conv2D(8, kernel=3, strides=1, padding='same'))&#xA;    model.add(layers.BatchNormalization())&#xA;    model.add(layers.LeakyReLU())&#xA;&#xA;    model.add(layers.Conv2D(1, kernel=3, strides=1, padding='same', activation='tanh'))&#xA;&#xA;    return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that I did not test the model, so you should adapt it to the shape of your data.&lt;/p&gt;&#xA;&lt;p&gt;I encourage you to follow &lt;a href=&quot;https://www.tensorflow.org/tutorials/generative/dcgan&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; regarding overfitting&lt;/p&gt;&#xA;&lt;p&gt;GANs do not have overfitting problem in the classical sense. Instead, vanilla GAN have other problems like vanishing gradient. It happens when the D becomes overconfident regarding fake samples. In that case, D stops providing useful information to the G, since its error becomes 0. To find out if training fell into this problem, you should plot D and G loss. It will look as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6sof5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6sof5.png&quot; alt=&quot;DCGAN loss&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another problem is the mode collapse. In that case, the G tricks the D by producing only one type of samples which looks realistic but G won't represent the real data distribution. Therefore, the generated samples will be homogeneous.&lt;/p&gt;&#xA;&lt;p&gt;For more information, see &lt;a href=&quot;https://arxiv.org/abs/1606.03498&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improved Techniques for Training GANs&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1701.07875&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wasserstein GAN&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-11-17T15:42:02.333" LastActivityDate="2020-11-17T15:42:02.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24666" PostTypeId="1" CreationDate="2020-11-17T12:51:02.533" Score="0" ViewCount="16" Body="&lt;p&gt;Let us imagine a face database with several subjects, each subject having multiple face images. How do we determine which is the best face suitable for face recognition purposes?&lt;/p&gt;&#xA;" OwnerUserId="42346" LastActivityDate="2021-02-03T15:08:35.860" Title="In a face database containing multiple images per subject, how do we determine the face image which is most suited for face recognition?" Tags="&lt;deep-learning&gt;&lt;face-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24667" PostTypeId="2" ParentId="14320" CreationDate="2020-11-17T12:55:04.903" Score="0" Body="&lt;p&gt;The claim that Neural Network with a single hidden layer can model any functions is proven in Cybenko's Approximation by superpositions of a sigmoidal function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/BF02551274&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/article/10.1007/BF02551274&lt;/a&gt;&#xA;check also: &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_approximation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Universal_approximation_theorem&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The thing is that the neural network using sigmoidal functions, which are non-linear functions can.&lt;/p&gt;&#xA;" OwnerUserId="35957" LastActivityDate="2020-11-17T12:55:04.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24668" PostTypeId="1" AcceptedAnswerId="24669" CreationDate="2020-11-17T12:56:31.920" Score="2" ViewCount="396" Body="&lt;p&gt;Every computer science student (including myself, when I was doing my bachelor's in CS) probably encountered the famous single-source shortest path &lt;strong&gt;Dijkstra's algorithm (DA)&lt;/strong&gt;. If you also took an introductory course on artificial intelligence (&lt;a href=&quot;https://ai.stackexchange.com/q/2514/2444&quot;&gt;as I did a few years ago, during my bachelor's&lt;/a&gt;), you should have also encountered some search algorithms, in particular, the &lt;strong&gt;uniform-cost search (UCS)&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A few articles on the web (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia article on DA&lt;/a&gt;) say that DA (or a variant of it) is equivalent to the UCS. The famous Norvig and Russell's book &lt;em&gt;Artificial Intelligence: A Modern Approach&lt;/em&gt; (3rd edition) even states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The two-point shortest-path algorithm of &lt;a href=&quot;http://www-m3.ma.tum.de/foswiki/pub/MN0506/WebHome/dijkstra.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dijkstra (1959)&lt;/a&gt; is the origin of uniform-cost search. These works also introduced the idea of explored and frontier sets (closed and open lists).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How exactly is DA equivalent to UCS?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-17T13:01:43.307" LastActivityDate="2020-11-17T13:01:43.307" Title="What is the difference between the uniform-cost search and Dijkstra's algorithm?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;uniform-cost-search&gt;&lt;dijkstras-algorithm&gt;&lt;shortest-path-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24669" PostTypeId="2" ParentId="24668" CreationDate="2020-11-17T12:56:31.920" Score="1" Body="&lt;p&gt;The answer to my question can be found in the paper &lt;a href=&quot;https://www.aaai.org/ocs/index.php/SOCS/SOCS11/paper/viewFile/4017/4357&quot; rel=&quot;nofollow noreferrer&quot;&gt;Position Paper: Dijkstra's Algorithm versus Uniform Cost Search or a Case Against Dijkstra's Algorithm&lt;/a&gt; (2011), in particular section &lt;strong&gt;Similarities of DA and UCS&lt;/strong&gt;, so you should read this paper for all the details.&lt;/p&gt;&#xA;&lt;p&gt;DA and UCS are logically equivalent (i.e. they process the same vertices in the same order), but they do it differently. In particular, the main practical difference between the &lt;em&gt;single-source&lt;/em&gt; DA and UCS is that, in DA, &lt;strong&gt;all&lt;/strong&gt; nodes are initially inserted in a priority queue, while in UCS nodes are inserted lazily.&lt;/p&gt;&#xA;&lt;p&gt;Here is the pseudocode (taken from the cited paper) of DA&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OXoFf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OXoFf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the pseudocode of the best-first search (BFS), of which UCS is just a particular case. Actually, this is the pseudocode of UCS where &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the cost of the path from the source node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (although the title indicates that this is the pseudocode of BFS).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nFxKH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nFxKH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-11-17T12:56:31.920" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24670" PostTypeId="1" CreationDate="2020-11-17T13:16:09.003" Score="0" ViewCount="26" Body="&lt;p&gt;I have been using AIgym to train my RL agents. I am now trying to take advantage of the different difficulty settings that the ALE offers.&lt;/p&gt;&#xA;&lt;p&gt;However I can't find a resource that explains which difficulty is 'easy' or 'hard'. The AIgym library accepts integer values [0,1,2...] to set the game difficulty, but based on my experience with Pong, the difficulty of [0] seems to be herder than the difficulty of [1], so the difficulty settings are not sorted in ascending order.&lt;/p&gt;&#xA;" OwnerUserId="42348" LastActivityDate="2020-11-17T13:16:09.003" Title="Is there a resource that explains which settings mean 'High' or 'Low' difficulty in the ALE environment?" Tags="&lt;reinforcement-learning&gt;&lt;open-ai&gt;&lt;gym&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24671" PostTypeId="1" CreationDate="2020-11-17T13:37:26.590" Score="1" ViewCount="29" Body="&lt;p&gt;Quick questions to see whether I understand GCNs correctly.&lt;/p&gt;&#xA;&lt;p&gt;Is it correct that, if I have trained a GCN, it can take arbitrary graphs as input, assuming the feature size is the same?&lt;/p&gt;&#xA;&lt;p&gt;I can't seem to find explicit literature on this.&lt;/p&gt;&#xA;" OwnerUserId="42349" LastEditorUserId="2444" LastEditDate="2020-11-17T14:08:38.943" LastActivityDate="2020-11-20T17:43:51.937" Title="How does a GCN handle new input graphs?" Tags="&lt;geometric-deep-learning&gt;&lt;graphs&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24672" PostTypeId="1" CreationDate="2020-11-17T16:43:15.963" Score="-1" ViewCount="83" Body="&lt;p&gt;My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output.&lt;/p&gt;&#xA;&lt;p&gt;Take the scenario:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;One prints a PDF file to a text file (the text file does not have images, but has the main text, headings, page numbers, some other footer text, and so on, and keeps the same number of columns  - two for instance - of text);&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This text file is submitted to a tool that strips everything that is not the main content of the text in one single text column (one text stream), keeping the section titles, paragraphs, and the text in a readable form (does not mix columns);&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The tool generates a new version of the original text file containing only the main text portion, ready to be used for other purposes where the striped parts would be considered noise.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How to model this problem in a way a neural network can handle it?&lt;/p&gt;&#xA;&lt;h2&gt;Update 1&lt;/h2&gt;&#xA;Here are some clarifications on the problem.&#xA;&lt;h3&gt;PDF file&lt;/h3&gt;&#xA;&lt;p&gt;The picture below shows two pages of a pdf version of a scientific paper. This is just to set the context, the PDF file is not the input for this problem, it is just to understand where the actual input data comes from.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SwBQl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SwBQl.jpg&quot; alt=&quot;PDF version of a scientific papers with parts of interest in colored blocks&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The color boxes show some parts of interest for this discussion. Red boxes are headers and footers. &lt;strong&gt;We are not interested in them&lt;/strong&gt;. Blue and green boxes are content text blocks. Diferent colors were used to enphasize the text is organized in columns and that is part of the problem. &lt;strong&gt;Those blue and green boxes are what we actually want&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Text file&lt;/h3&gt;&#xA;If a use the &quot;save as text file&quot; feature of my free PDF reader, I get a text file similar to the image below.&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P8XwL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P8XwL.jpg&quot; alt=&quot;Text version of the PDF file&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The text file is continuous, but I put the equivalent of the first two pages of the PDF file side-by-side just make things easier to compare. We can see the very same colored boxes. In terms of words, those boxes contain the same text as in the PDF version.&lt;/p&gt;&#xA;&lt;h3&gt;Understanding the problem&lt;/h3&gt;&#xA;When we read a paper, we are usually not very interested in footers or headers. The main content is what we actually read and that will provide us with the knowledge we are looking for. In this case, the text inside blue and green boxes.&#xA;&lt;p&gt;So, what we want here is to generate a new version of the input (text) file organized in one single text stream (one column if you will), with the text layed-out in a form someone can read it, which means, alternating the blue and the green boxes.&lt;/p&gt;&#xA;&lt;p&gt;However, if the original PDF has no footers, it should work in the same way, providing the main text content. If the text comes in three of four columns, the final product must be a text in good conditions to be read without loosing any information.&lt;/p&gt;&#xA;&lt;p&gt;Any pictures will be simply striped off the text version of the paper and we are fine with that.&lt;/p&gt;&#xA;" OwnerUserId="42356" LastEditorUserId="42356" LastEditDate="2020-11-24T17:10:49.497" LastActivityDate="2020-11-24T17:10:49.497" Title="How to extract the main text from a formated text file?" Tags="&lt;neural-networks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24673" PostTypeId="1" CreationDate="2020-11-17T17:44:41.483" Score="0" ViewCount="29" Body="&lt;p&gt;I have two measuring devices. Both measure the same thing. One is accurate, the other is not, but does correlate with a non-fixed offset, some outliers, and some noise.&lt;/p&gt;&#xA;&lt;p&gt;I won't always be using the accurate device. The nonfixed offset makes things difficult, but I'm certain there is sufficient similarity to make a link using a machine learning (or AI) technique and to convert one set of numbers to a good approximation of the other.&lt;/p&gt;&#xA;&lt;p&gt;One is a footbed power meter and gives power in Watts every second. The other is a crank-based power meter, also outputting Watts at 1Hz. The footbed power is much less than the crank (which I know to be accurate), but it does track the increases and decreases in power, just with more noise and, as I say, a non-fixed offset (and by non-fixed I mean, at low power the offset is different to that at high power, I don't mean it isn't consistent, it is consistent). Both measure cadence which may be a useful metric to help find a pattern.&lt;/p&gt;&#xA;&lt;p&gt;I will be collecting sets of data from both and hoped to plug the footbed data in as a column of values with the crank data as another column representing the truth, so after training, the model would be able to transform the footbed data to an approximation of the crank data.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I'm completely lost as to how to begin. I've tried searching, but, clearly, I'm using the wrong keywords. Does anyone have any pointers, please?&lt;/p&gt;&#xA;" OwnerUserId="42358" LastEditorUserId="2444" LastEditDate="2020-11-19T11:17:55.887" LastActivityDate="2021-04-18T12:49:53.280" Title="Which machine learning technique can I use to match one set of data points to another?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;linear-regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24674" PostTypeId="1" CreationDate="2020-11-17T18:00:15.700" Score="2" ViewCount="28" Body="&lt;p&gt;I am working on a segmentation of MRI images of the thigh. I am trying to segment the fascia, there is a slight imbalance between the background and the mask. I have about 1400 images from 30 patients for training and 200 for validation. I am working with keras. The loss function is combination of weighted cross entropy and dice loss (smooth factor of dice loss = 2)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def combo_loss(y_true, y_pred,alpha=0.6, beta=0.6): # beta before 0.4&#xA;    return alpha*tf.nn.weighted_cross_entropy_with_logits(y_true, y_pred, pos_weight=beta)+((1-alpha)*dice_coefficient_loss(y_true, y_pred))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I use a value alpha greater thatn 0.5 (the weighted cross entropy) the loss rapidly decreases during the first epoch. Afterwards if slowly decreases in a linear manner.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Zsy2y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zsy2y.png&quot; alt=&quot;Alpha = 0.6&quot; /&gt;&lt;/a&gt;&#xA;Why is this happening? What would be a good approach reasonable approach to choose the values of alpha and beta?&lt;/p&gt;&#xA;" OwnerUserId="42357" LastActivityDate="2020-11-17T18:00:15.700" Title="Loss function decays linearly in segmentation MRI fascia" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;loss-functions&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24675" PostTypeId="5" CreationDate="2020-11-17T18:47:17.580" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-17T18:47:17.580" LastActivityDate="2020-11-17T18:47:17.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24676" PostTypeId="4" CreationDate="2020-11-17T18:47:17.580" Score="0" Body="For questions about action spaces in the context of reinforcement learning and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-17T18:47:17.580" LastActivityDate="2020-11-17T18:47:17.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24677" PostTypeId="2" ParentId="24647" CreationDate="2020-11-17T20:03:05.540" Score="1" Body="&lt;p&gt;According to my understanding, you don't use just the current behavior policy for sampling. The importance sampling ratio is calculated as the product of the probability ratios for both the target and behaviour policy throughout the trajectory.&lt;/p&gt;&#xA;&lt;p&gt;See the calculation below, where the product is happening for all the probabilities throughout the trajectories. (screenshot from Chapter 5, section 5.5 (page 85) of Sutton &amp;amp; Barto)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bNTl3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bNTl3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42262" LastEditorUserId="42262" LastEditDate="2020-11-18T06:29:39.553" LastActivityDate="2020-11-18T06:29:39.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24678" PostTypeId="2" ParentId="23711" CreationDate="2020-11-17T20:37:16.560" Score="0" Body="&lt;p&gt;I would say any normalization such as min-max or standard deviation is fine as far as the scaling factor is provided as a feature, since time-series of different scale might behave differently.&lt;/p&gt;&#xA;" OwnerUserId="42330" LastActivityDate="2020-11-17T20:37:16.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24679" PostTypeId="2" ParentId="23089" CreationDate="2020-11-17T20:41:02.800" Score="0" Body="&lt;p&gt;I would say offsets from previous N-event is necessary in this case. You can also encode their max and avg as other features in order to compress the irregular sampling times into a vector.&lt;/p&gt;&#xA;" OwnerUserId="42330" LastActivityDate="2020-11-17T20:41:02.800" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24680" PostTypeId="2" ParentId="22853" CreationDate="2020-11-17T20:55:12.830" Score="0" Body="&lt;p&gt;Of course depends on your type of data, but Holt-Winter models can have different degree of complexity and use moving average, trend, and seasonality. This is most useful if the data is not hierarchical, meaning that the time-series are independent from each other.&#xA;If time-series are relatives of each other then you can also try aggregating them, predict at aggregate level and then disaggregate. The following can be a good resource:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://otexts.com/fpp2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://otexts.com/fpp2/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42330" LastActivityDate="2020-11-17T20:55:12.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24681" PostTypeId="1" CreationDate="2020-11-17T21:25:55.627" Score="2" ViewCount="49" Body="&lt;p&gt;I might just be overthinking a very simple question but nonetheless the following has been bugging me a lot.&lt;/p&gt;&#xA;&lt;p&gt;Given an MDP with non-trivial state and action sets, we can implement the SARSA algorithm to find the optimal policy or the optimal state-action-value function &lt;span class=&quot;math-container&quot;&gt;$Q^*(s,a)$&lt;/span&gt; by the following iteration:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_t,a_t)\leftarrow Q(s_t,a_t) + \alpha(r_t + \gamma Q(s_{t+1}, a_{t+1}) - Q(s_t,a_t)).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Assuming each state-action pair is visited infinitely often, fix one such pair &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; and denote the time sequence of visiting the said pair as &lt;span class=&quot;math-container&quot;&gt;$t_1 &amp;lt; t_2 &amp;lt; t_3 &amp;lt; \dots t_n\dots.$&lt;/span&gt; Also, let &lt;span class=&quot;math-container&quot;&gt;$Q_{t_n}(s,a) = X_n$&lt;/span&gt; for ease of notation and consider the sequence of random variables:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X_0, X_1, \dots X_n,\dots $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can &lt;span class=&quot;math-container&quot;&gt;$\{X_n\}_{n\geq 0}$&lt;/span&gt; be though of a discrete-time Markov Chain on &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; ? My intuition says no, because the recurrence equation will look like:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X_{n+1} = (1-\alpha)X_n + \alpha(r_{t_n} +\gamma Q_{t_n}(s',a'))$$&lt;/span&gt;&#xA;and that last term &lt;span class=&quot;math-container&quot;&gt;$Q_{t_n}(s',a')$&lt;/span&gt; will be dependent on the path even if we condition on &lt;span class=&quot;math-container&quot;&gt;$X_n = x.$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I am not quite able to rigorously write an answer in the either direction. I will greatly appreciate if someone can resolve this issue that I am having in either direction.&lt;/p&gt;&#xA;" OwnerUserId="38234" LastActivityDate="2020-11-17T21:25:55.627" Title="Can $Q$-learning or SARSA be thought of a Markov Chain?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;sarsa&gt;&lt;markov-chain&gt;&lt;markov-property&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24682" PostTypeId="1" AcceptedAnswerId="24734" CreationDate="2020-11-17T22:23:45.607" Score="2" ViewCount="88" Body="&lt;p&gt;I was reading about IDA* and I found this &lt;a href=&quot;https://algorithmsinsight.wordpress.com/graph-theory-2/ida-star-algorithm-in-general/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; explaining IDA* and providing &lt;a href=&quot;https://algorithmsinsight.files.wordpress.com/2016/03/ida-star.gif?w=1326&quot; rel=&quot;nofollow noreferrer&quot;&gt;an animation&lt;/a&gt; for it.&lt;/p&gt;&#xA;&lt;p&gt;Here is a picture of the solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/paLIe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/paLIe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know what is the cutoff condition (it depends on F), and the search is like DFS if the value of (f) of the node is less or equal to the cutoff, and like IDF it is iterative&lt;/p&gt;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&lt;p&gt;In the animation, when the threshold is 7 and after expanding the parent of the goal (14), they stated that a solution has been found, so, if we found the goal after expanding a node which is &amp;lt;= the cutoff value, can we consider it the solution without applying any condition-test on it? (it's F &amp;lt;= threshold): for example, if there was another level where there is a goal and it can be found with value 13 (less than 14), like the following pic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vvLpV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vvLpV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;when the threshold is 7, 11 will not be expanded so we will never get (13)&lt;/p&gt;&#xA;&lt;p&gt;So, what is the correct solution?&lt;/p&gt;&#xA;" OwnerUserId="42300" LastEditorUserId="2444" LastEditDate="2020-11-19T01:06:46.317" LastActivityDate="2020-11-21T11:49:06.617" Title="When does IDA* consider the goal has been found?" Tags="&lt;search&gt;&lt;ida-star&gt;&lt;stopping-conditions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24683" PostTypeId="1" CreationDate="2020-11-17T23:45:25.807" Score="1" ViewCount="48" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://www.danieldjohnson.com/2015/08/03/composing-music-with-recurrent-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Composing Music With Recurrent Neural Networks&lt;/a&gt; by Daniel D. Johnson. But I am really confused about the input passed to this network. If we pass notes of music along the time axis, then what is passed along the note axis?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W0waA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W0waA.png&quot; alt=&quot;bi-axial netowork architecture&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The author says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If we make a stack of identical recurrent neural networks, one for each output note, and give each one a local neighborhood (for example, one octave above and below) around the note as its input, then we have a system that is invariant in both time and notes: the network can work with relative inputs in both directions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This might mean that the inputs passed to the network along the note axis are fixed representations of notes in the vocabulary. But I am not sure.&lt;/p&gt;&#xA;&lt;p&gt;I am also having a hard time understanding the input passed to this network as the author explains a few paragraphs below. (Position, Pitchclass, Previous Vicinity, Previous Context, Beat).&lt;/p&gt;&#xA;&lt;p&gt;Also, at some point, the author talks about RNN along the note axis. But in the architecture, there only seems to be RNN along the time axis. I would really appreciate is anyone could give me some more information to understand how this Biaxial Network is setup. &lt;a href=&quot;https://deepdarklearning.wordpress.com/2016/12/15/composing-classical-music-with-recurrent-neural-network/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; article by Deep Dark Learning was a little helpful but I am still not fully sure what is going on here.&lt;/p&gt;&#xA;" OwnerUserId="42204" LastEditorUserId="42204" LastEditDate="2020-11-18T19:39:19.143" LastActivityDate="2020-11-18T19:39:19.143" Title="How is input defined for a biaxial lstm network for generating music?" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;language-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24684" PostTypeId="2" ParentId="24648" CreationDate="2020-11-17T23:53:47.050" Score="1" Body="&lt;p&gt;You can find an explanation &lt;a href=&quot;https://github.com/googleapis/google-cloud-node/issues/1324#issuecomment-250733976&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (github of the googleapi):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My current understanding of a color's score is a combination of two things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the focus of the image?&lt;/li&gt;&#xA;&lt;li&gt;What is the color of that focus?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For example, given the following image:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V7faF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V7faF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The focus is clearly the cat, and therefore the color annotation for this image with the highest score (0.15) will be RGB = (232, 183, 135) which is the beige color:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e3fv7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e3fv7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The green of the grass (despite having more pixels in the image dedicated to it) has a much lower score by virtue of the algorithm's detection that it's the background and not the focus of the image.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In other words, higher &amp;quot;scores&amp;quot; means higher confidence that the color in question is prominent in the central focus of the image.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is analogous to your case. Therefore, using a background removal this can help to find the focus of the image, and then make the histogram of the remaining objects' color. An example of the background removal using deep learning can be found in &lt;a href=&quot;https://towardsdatascience.com/background-removal-with-deep-learning-c4f2104b3157&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2020-11-17T23:53:47.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24685" PostTypeId="1" AcceptedAnswerId="24700" CreationDate="2020-11-17T23:54:26.677" Score="1" ViewCount="232" Body="&lt;p&gt;I am using AlexNet CNN to classify my dataset which contains 10 classes and 1000 data for each class, with 60-30-10, splits for train, validation, and test. I used different batch sizes, learning rates, activation functions, and initializers. I'm using the sparse categorical cross-entropy loss function.&lt;/p&gt;&#xA;&lt;p&gt;However, while training, my loss value is greater than one (almost equal to 1.2) in the first epoch, but until epoch 5 it comes near 0.8.&#xA;Is it normal? If not, how can I solve this?&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="2444" LastEditDate="2020-11-19T12:07:37.423" LastActivityDate="2020-11-19T12:07:37.423" Title="Can the (sparse) categorical cross-entropy be greater than one?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;object-recognition&gt;&lt;categorical-crossentropy&gt;&lt;alexnet&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24686" PostTypeId="1" AcceptedAnswerId="24694" CreationDate="2020-11-18T07:04:41.930" Score="5" ViewCount="311" Body="&lt;p&gt;If there are two different optimal policies &lt;span class=&quot;math-container&quot;&gt;$\pi_1, \pi_2$&lt;/span&gt; in a reinforcement learning task, will the linear combination (or &lt;a href=&quot;https://en.wikipedia.org/wiki/Affine_combination&quot; rel=&quot;nofollow noreferrer&quot;&gt;affine combination&lt;/a&gt;) of the two policies &lt;span class=&quot;math-container&quot;&gt;$\alpha \pi_1 + \beta \pi_2, \alpha + \beta = 1$&lt;/span&gt; also be an optimal policy?&lt;/p&gt;&#xA;&lt;p&gt;Here I give a simple demo:&lt;/p&gt;&#xA;&lt;p&gt;In a task, there are three states &lt;span class=&quot;math-container&quot;&gt;$s_0, s_1, s_2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s_1, s_2$&lt;/span&gt; are both terminal states. The action space contains two actions &lt;span class=&quot;math-container&quot;&gt;$a_1, a_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;An agent will start from &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, it can choose &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;, then it will arrive &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt;，and receive a reward of &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;. In &lt;span class=&quot;math-container&quot;&gt;$s_0$&lt;/span&gt;, it can also choose&#xA;&lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;, then it will arrive &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt;, and receive a reward of &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In this simple demo task, we can first derive two different optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi_1(a_1|s_0) = 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_2(a_2 | s_0) = 1$&lt;/span&gt;. The combination of &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and&lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$\pi: \pi(a_1|s_0) = \alpha, \pi(a_2|s_0) = \beta$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is an optimal policy, too. Because any policy in this task is an optimal policy.&lt;/p&gt;&#xA;" OwnerUserId="40633" LastEditorUserId="40633" LastEditDate="2020-11-18T13:47:16.470" LastActivityDate="2020-11-20T14:13:48.603" Title="Given two optimal policies, is an affine combination of them also optimal?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;policies&gt;&lt;optimal-policy&gt;&lt;optimality&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24688" PostTypeId="2" ParentId="11504" CreationDate="2020-11-18T07:44:45.153" Score="0" Body="&lt;p&gt;I could give you my $0.02 on fraud detection.&lt;/p&gt;&#xA;&lt;ol start=&quot;0&quot;&gt;&#xA;&lt;li&gt;Read everything you can on the Equifax breach and seek to secure your data&lt;/li&gt;&#xA;&lt;li&gt;Benfords Law would be a good place to start&lt;/li&gt;&#xA;&lt;li&gt;If you can isolate log activity that is inhumanly consistant, if you &amp;quot;ip_address&amp;quot;, &amp;quot;id&amp;quot; and/or &amp;quot;timestamp&amp;quot; all show a constant 3 second gap between activity or its always a random choice between 3 and 6 seconds between them.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If you plan on investing the time and resources that ML or Ai require, you will need to isolate &amp;quot;good data&amp;quot; as training data and train your model on that. Perhaps you could get the IP address of your known top 10 customers and include that.&lt;/p&gt;&#xA;&lt;p&gt;Then begin training with that as your sample data, with keeping your test data seperate.&lt;/p&gt;&#xA;&lt;p&gt;I'm sure there is alot more but I would need to know context of the info provided in terms of what kind of fraud you are looking for, what you have tried or why they think there is fraud to be found.&lt;/p&gt;&#xA;" OwnerUserId="34095" LastActivityDate="2020-11-18T07:44:45.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24689" PostTypeId="2" ParentId="24686" CreationDate="2020-11-18T09:06:01.383" Score="2" Body="&lt;p&gt;Yes, in general any linear combination of probability distributions between optimal policies is also an optimal policy. In fact &lt;em&gt;any&lt;/em&gt; combination with each state treated separately will also be an optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;This can be seen using the equation for optimal deterministic policy in terms of optimal value function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi^*(s) = \text{argmax}_a [\sum_{r,s'}p(r,s'|s,a)(r + \gamma v^*(s'))] = \text{argmax}_a [q^*(s,a)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only way to have multiple equivalent optimal policies is when there are two or more actions tied for &lt;span class=&quot;math-container&quot;&gt;$\text{max}_a [q^*(s,a)]$&lt;/span&gt;. If that is the case, then it does not matter - in terms of expected future reward - which of those actions is taken in the affected state. It is possible to take one or the other action.&lt;/p&gt;&#xA;&lt;p&gt;You can use any rule you wish to decide which of the tied-for-max actions to take, including a random choice. Creating a new policy that is a linear combination of optimal policies is one way to make that random choice. You could equally say that you would make one action choice on a Monday and another on Tuesday - in terms of being optimal choices it does not matter at all how you break ties.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-18T09:06:01.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24690" PostTypeId="1" AcceptedAnswerId="24693" CreationDate="2020-11-18T11:04:22.447" Score="1" ViewCount="34" Body="&lt;p&gt;I'm a beginner with a classic &amp;quot;racing car&amp;quot; sandbox and a homemade simple neural network.&lt;/p&gt;&#xA;&lt;p&gt;My pattern:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Copy the &amp;quot;top car&amp;quot; (without mutation) to the next generation&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If there are some cars still running (because simulation reached the 30s win condition), then copy a mutated version of them for the next generation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fill the rest of the pool with mutation of the &amp;quot;top car&amp;quot;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;But this is just some dumb intuitive pattern I made on the fly while playing with my code. Perhaps I should copy the cars that are still running as-is instead of mutating them. Or, perhaps, some selection method I don't know about.&lt;/p&gt;&#xA;&lt;p&gt;A new random track is generated at each new generation. a &amp;quot;top car&amp;quot; may be good on a track and crash immediately on the next track. I just feel that basing everything on the top car is wrong because of the track randomness.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there some known pattern for selecting a batch of candidates?&lt;/strong&gt; (paper, google-fu keyword, interesting blog, etc.)&lt;/p&gt;&#xA;&lt;p&gt;I don't know what to search for. I don't even know the name of my network or any vocabulary related to AI.&lt;/p&gt;&#xA;" OwnerUserId="42373" LastEditorUserId="2444" LastEditDate="2020-11-18T12:40:00.460" LastActivityDate="2020-11-18T12:40:00.460" Title="Is there some known pattern for selecting a batch of candidates for the next generation?" Tags="&lt;neural-networks&gt;&lt;reference-request&gt;&lt;genetic-algorithms&gt;&lt;genetic-operators&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24692" PostTypeId="2" ParentId="24565" CreationDate="2020-11-18T11:56:19.257" Score="1" Body="&lt;p&gt;This is not necessarily the only way to do this but it would be the approach I'd take.&lt;/p&gt;&#xA;&lt;p&gt;Assuming your agents position is a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^d$&lt;/span&gt;, then I would have the network take as input this position vector and pass it through a fully connected layer. I would also take as input the matrix and pass it through a convolutional layer(s) and flatten the output so it is now also a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{d'}$&lt;/span&gt;. I would then concatenate these together so you have a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{d + d'}$&lt;/span&gt; and pass this through some fully connected layers as usual.&lt;/p&gt;&#xA;&lt;p&gt;As for &lt;em&gt;how many&lt;/em&gt; layers and which activation to use this is something you'll have to do by trial and error as this really is problem specific.&lt;/p&gt;&#xA;&lt;p&gt;Your output would typically be a score for each of the action combinations, though as one of your outputs is a matrix this could potentially make things expensive to compute as you would need &lt;span class=&quot;math-container&quot;&gt;$2^{n \times m}$&lt;/span&gt; binary representations just for the matrix (&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of rows and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the number of columns).&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-11-18T11:56:19.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24693" PostTypeId="2" ParentId="24690" CreationDate="2020-11-18T12:10:11.903" Score="1" Body="&lt;p&gt;The most general descriptive frameworks covering what you are trying to do are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sequential_decision_making&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequential decision making&lt;/a&gt; (article is a stub, but the term a good launching point to discover different wys of modelling and solving these kind of problems)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Optimal_control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal control&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These put some context around your problem, and might give you some pointers. For instance, reinforcement learning is an alternative approach to the evolutionary system you are trying to build.&lt;/p&gt;&#xA;&lt;p&gt;The specific AI system you appear to be building is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithm&lt;/a&gt;, and more specific still you are attempting to find a neural network that is optimal at a task by searching for the best network using a system of population generation, selection and mutation which repeats.&lt;/p&gt;&#xA;&lt;p&gt;There are lots of ways to set up a system like this, so your approach is not necessarily &lt;em&gt;wrong&lt;/em&gt;. However, I think there are two key things that would improve what you have built so far:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Fitness_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;fitness function&lt;/a&gt; for selection. Score each car, perhaps by how far it got before crashing when the episode ends. To reduce luck factor on random courses, you could make this score the mean result from e.g. 3 different courses (it is not necessary, but may address your concern that selection is too random in your case). Select some fraction of top scoring cars, or look into other selection approaches - e.g. weighted selection based on fitness score or ranking.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Add &amp;quot;sex&amp;quot;, more properly known as genome &lt;a href=&quot;https://en.wikipedia.org/wiki/Crossover_(genetic_algorithm)&quot; rel=&quot;nofollow noreferrer&quot;&gt;crossover&lt;/a&gt; between selected population members. Mutating individuals is limiting because it silos improvements to a single line of ancestry - if there are two good mutations found at random you rely on that single line finding both of them. Whilst crossover allows sharing of good mutations between lines, making it much more likely that two good mutations will end up in the same individual.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There is a framework called &lt;a href=&quot;https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEAT&lt;/a&gt; which covers the issues above plus has other features useful for evolving neural networks. It often does well at control scenarios like the one you are considering. You may want to look into it, if your focus is mainly on solving the control problem. However, it is relatively advanced from where you are, so if your current focus to learn by building from scratch you may get more initially from implementing fitness functions and crossover yourself.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-18T12:10:11.903" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24694" PostTypeId="2" ParentId="24686" CreationDate="2020-11-18T12:13:49.753" Score="2" Body="&lt;h3&gt;Short answer&lt;/h3&gt;&#xA;&lt;p&gt;Two policies are different if they take different actions in a specific state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (or they give different probabilities of taking those actions in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;). There can be more than one optimal policy for a given value function: this only happens when two actions have the same value in a given state. Nevertheless, both policies lead to the same &lt;strong&gt;expected return&lt;/strong&gt;. So, although they take different actions, those actions lead to the same &lt;strong&gt;expected return&lt;/strong&gt;, so it doesn't matter which one you take: both actions are optimal.&lt;/p&gt;&#xA;&lt;h3&gt;Long answer&lt;/h3&gt;&#xA;&lt;p&gt;There are a few important points that need to be understood before understanding that an affine combination of optimal policies is also optimal.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is optimal if and only if &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s) \geq v_{\pi'}(s)$&lt;/span&gt;, for all states &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi' \neq \pi \in \Pi$&lt;/span&gt; [&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=84&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;];&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In that case, we denote &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\pi_*$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_* = \pi \geq \pi'$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$\pi' \neq \pi \in \Pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In simple words, a policy is optimal if it leads to more or equal &lt;strong&gt;expected return&lt;/strong&gt;, in all states, with respect to all other policies&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Optimal policies share the same state and state-action value functions [&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=85&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;], i.e. &lt;span class=&quot;math-container&quot;&gt;$v_*$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q_*$&lt;/span&gt;, respectively&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In other words, if &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; are optimal policies, then &lt;span class=&quot;math-container&quot;&gt;$v_{\pi_1}(s) = v_{\pi_2}(s) = v_{\pi_*}(s)$&lt;/span&gt; and  &lt;span class=&quot;math-container&quot;&gt;$q_{\pi_1}(s, a) = q_{\pi_2}(s, a) = q_{\pi_*}(s, a)$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$s \in S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a \in A$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Consequently, two optimal policies &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; can differ in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; takes action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; takes action &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_1 \neq a_2$&lt;/span&gt;) if and only if there exist actions &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;v_{*}(s) &#xA;&amp;amp;= q_{\pi_1}(s, a_1) \\ &#xA;&amp;amp;= q_{\pi_1}(s, a_2) \\ &#xA;&amp;amp;= q_{\pi_2}(s, a_2) \\&#xA;&amp;amp;= q_{\pi_2}(s, a_1) \\&#xA;&amp;amp;= \max _{a \in \mathcal{A}(s)} q_{\pi_{*}}(s, a) \\&#xA;&amp;amp;= \max _{a \in \mathcal{A}(s)} q_{\pi_1}(s, a) \\&#xA;&amp;amp;= \max _{a \in \mathcal{A}(s)} q_{\pi_2}(s, a) \tag{1} \label{1}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This holds for &lt;a href=&quot;https://ai.stackexchange.com/a/12275/2444&quot;&gt;deterministic&lt;/a&gt; (i.e. policies that always take the same action in a given state, i.e. they give probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to one action) and &lt;a href=&quot;https://ai.stackexchange.com/a/12275/2444&quot;&gt;stochastic&lt;/a&gt; (give &lt;strong&gt;non-zero&lt;/strong&gt; probability only to &lt;strong&gt;optimal&lt;/strong&gt; actions) optimal policies&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, two different optimal policies &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; lead to the same expected return, for all states. Given that optimality is defined in terms of expected return, then, if &lt;span class=&quot;math-container&quot;&gt;$a_1 = \pi_1(s) \neq \pi_2(s) = a_2$&lt;/span&gt;, for some state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, then, it doesn't matter whether you take &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;, because both lead to the same expected return. So, as written in &lt;a href=&quot;https://ai.stackexchange.com/a/24689/2444&quot;&gt;this answer&lt;/a&gt;, you can either take action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;: both are optimal in terms of expected returns and this follows from equation \ref{1} above.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this simple demo task, we can first derive two different optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi_1(a_1|s_0) = 1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\pi_2(a_2 | s_0) = 1$&lt;/span&gt;. The combination of &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and&lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$\pi: \pi(a_1|s_0) = \alpha, \pi(a_2|s_0) = \beta$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is an optimal policy, too. Because any policy in this task is an optimal policy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, correct. The reason is simple. In your case, &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt; give probability &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to one action, &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; respectively, so they must give probability &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to any other actions. &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; will give a probability &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; to action &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; and probability &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; to action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt;, but, given that &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; lead to the same expected return (i.e. they are both optimal), it doesn't matter whether you take &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;, even if &lt;span class=&quot;math-container&quot;&gt;$\alpha \ll \beta$&lt;/span&gt; (or vice-versa).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T14:13:48.603" LastActivityDate="2020-11-20T14:13:48.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24696" PostTypeId="1" CreationDate="2020-11-18T14:46:08.570" Score="0" ViewCount="12" Body="&lt;p&gt;With the release of &lt;a href=&quot;https://www.nvidia.com/content/dam/en-zz/Solutions/data-center/gated-resources/nvidia-dgx-superpod-a100.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;NVIDIA's DGX SuperPOD of A100 GPUs&lt;/strong&gt;&lt;/a&gt;, supercomputers will/are becoming more and more common-place.&lt;/p&gt;&#xA;&lt;p&gt;What potential deep learning tasks/applications might become more accessible with this large amount of computing? These applications/tasks will require significant computing for either data-extraction, training, etc.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;An example of applications requiring large compute would be:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;GPT (or other large NLU/NLP model) Training&lt;/li&gt;&#xA;&lt;li&gt;Conducting Neural Architecture Search (NAS)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="36013" LastEditorUserId="36013" LastEditDate="2020-11-20T15:17:44.570" LastActivityDate="2020-11-20T15:17:44.570" Title="What Deep Learning Applications Might Require Super-Computers or &quot;SuperPODs&quot;" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;gpu&gt;&lt;gpt&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24697" PostTypeId="1" CreationDate="2020-11-18T15:02:07.033" Score="0" ViewCount="13" Body="&lt;p&gt;To create a classifier for a fixed corpus of texts is straightforward. Take all the documents, form the tfidf matrix and from that matrix take a subset that is tagged accordingly. The classifier built against that subset can then be used to tag the remainder of the corpus represented in the tfidf matrix.&lt;/p&gt;&#xA;&lt;p&gt;However, if the corpus is constantly increasing in size and perhaps vocabulary, eg streaming news articles, this approach does not seem valid. There is no longer a fixed tfidf matrix, since it will change as new articles arrive. Even if the change was small enough when a single new document arrived the cost of recompiling the matrix would render the method impractical and over time inaccurate as the changes piled up.&lt;/p&gt;&#xA;&lt;p&gt;So what is the best way of building a text classifier that can adapt to newly arriving articles? A semi supervised approach would seem possible but I haven’t found suitable references.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastEditorUserId="26382" LastEditDate="2020-11-18T20:04:04.783" LastActivityDate="2020-11-18T20:04:04.783" Title="How can I classify text documents arriving as a stream" Tags="&lt;text-classification&gt;&lt;semi-supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24698" PostTypeId="2" ParentId="5715" CreationDate="2020-11-18T17:14:59.853" Score="1" Body="&lt;p&gt;RLS is a second order optimizer, so, unlike LMS which takes into account an approximation of the derivative of the gradient, RLS also considers the second order derivative. You can study more about second order methods in sub-section &amp;quot;8.6 Approximate Second-Order Methods&amp;quot; of the following book available online:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.deeplearningbook.org/contents/optimization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.deeplearningbook.org/contents/optimization.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Deep Learning&#xA;An MIT Press book&#xA;Ian Goodfellow and Yoshua Bengio and Aaron Courville&lt;/p&gt;&#xA;" OwnerUserId="42330" LastActivityDate="2020-11-18T17:14:59.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24699" PostTypeId="1" CreationDate="2020-11-18T17:36:59.150" Score="1" ViewCount="71" Body="&lt;p&gt;I'm experimenting with an RL agent that interacts with the following environment. The learning algorithm is double DQN. The neural network represents the function from state to action. It's build with Keras sequential model and has two dense layers. The observation in the environment consists of the following features&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the agent's position in an N-dimensional grid,&lt;/li&gt;&#xA;&lt;li&gt;metrics that represent the hazards of adjacent cells (temperatures, toxicity, radiation, etc.) of adjacent cells, and&lt;/li&gt;&#xA;&lt;li&gt;some parameters that represent the agent's current characteristics (health, mood, etc.).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;There are patterns to the distribution of hazards and the agent's goal is to learn to navigate safely through space.&lt;/p&gt;&#xA;&lt;p&gt;I am concatenating these features, in the aforementioned order, into a tensor, which is fed into the double DQN.&lt;/p&gt;&#xA;&lt;p&gt;Does the order in which the features are concatenated to create the state (or observation) matter? Is it possible to group the features in some way to increase the learning speed? If I mix up the features randomly, would that have any effect or it doesn't matter to the agent?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="38076" LastEditDate="2020-11-20T16:58:44.013" LastActivityDate="2020-11-20T16:58:44.013" Title="Does the order in which the features are concatenated to create the state (or observation) matter?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;feature-selection&gt;&lt;state-spaces&gt;&lt;observation-spaces&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24700" PostTypeId="2" ParentId="24685" CreationDate="2020-11-18T18:34:07.850" Score="1" Body="&lt;p&gt;Both the &lt;strong&gt;sparse categorical cross-entropy (SCE)&lt;/strong&gt; and the &lt;strong&gt;categorical cross-entropy (CCE)&lt;/strong&gt; can be greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. By the way, they are the same exact loss function: the only difference is really the implementation, where the SCE assumes that the labels (or classes) are given as integers, while the CCE assumes that the labels are given as one-hot vectors.&lt;/p&gt;&#xA;&lt;p&gt;Here is the explanation with examples.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$(x, y) \in D$&lt;/span&gt; be an input-output pair from the labelled dataset &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the input and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the ground-truth class/label for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, which is an integer between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C-1$&lt;/span&gt;. Let's suppose that your neural network &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; produces a probability vector &lt;span class=&quot;math-container&quot;&gt;$f(x) = \hat{y} \in [0, 1]^C$&lt;/span&gt; (e.g. with a softmax), where &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i \in [0, 1]$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th element of &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The formula for SCE is (which is consistent with &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/losses/sparse_categorical_crossentropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;the TensorFlow implementation of this loss function&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{SCE}(y, \hat{y}) = - \ln (\hat{y}_{y}) \label{1}\tag{1},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_{y}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;th element of the output probability vector &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; that corresponds to the probability that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; belongs to class &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, according to &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Actually, the equation \ref{1} is also the definition of the CCE with one-hot vectors as targets (which behave as &lt;a href=&quot;https://www.statlect.com/fundamentals-of-probability/indicator-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;indicator functions&lt;/a&gt;). The only difference between CCE and SSE is really just the representation of the targets, which can slightly change the implementation under the hood. Moreover, note that this is the definition of the CE for only &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; training pair. If you have multiple pairs, you have to compute the CE for all pairs, then average these CEs (for a reference, see &lt;a href=&quot;http://users.isr.ist.utl.pt/%7Ewurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf#page=229&quot; rel=&quot;nofollow noreferrer&quot;&gt;equation 4.108, section 4.3.4 &lt;em&gt;Multiclass logistic regression&lt;/em&gt; of Bishop's book &lt;em&gt;PRML&lt;/em&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Let's have a look at a concrete example with concrete numbers. Let &lt;span class=&quot;math-container&quot;&gt;$C=5$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y = 3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = [0.2, 0.2, 0.1, 0.4, 0.1]$&lt;/span&gt;, then the SCE is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text{SCE}(y, \hat{y}) &#xA;&amp;amp;= &#xA;- \ln (0.4) \approx 0.92,&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = [0.2, 0.2, 0.2, 0.1, 0.3]$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_{y} = 0.1$&lt;/span&gt;, and we still have &lt;span class=&quot;math-container&quot;&gt;$y = 3$&lt;/span&gt;, then the CCE is &lt;span class=&quot;math-container&quot;&gt;$2.3 &amp;gt; 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can execute this Python code to check yourself.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import tensorflow as tf # Install TensorFlow 2.3!&#xA;&#xA;y = 1&#xA;y_true = [3] # sparse label (integer)&#xA;y_true2 = [0, 0, 0, 1, 0] # one-hot vector&#xA;&#xA;for y_y in [0.4, 0.1]:&#xA;    sce_np = -(y * np.log(y_y))&#xA;    print(&amp;quot;SCE (NumPy) =&amp;quot;, sce_np)&#xA;&#xA;y_preds = [[0.2, 0.2, 0.1, 0.4, 0.1],&#xA;           [0.2, 0.2, 0.2, 0.1, 0.3]]&#xA;for y_pred in y_preds:&#xA;    sce_tf = tf.keras.losses.sparse_categorical_crossentropy(y_true, y_pred)&#xA;    cce_tf = tf.keras.losses.categorical_crossentropy(y_true2, y_pred)&#xA;&#xA;    print(&amp;quot;SCE (TensorFlow) =&amp;quot;, sce_tf)&#xA;    print(&amp;quot;CCE (TensorFlow) =&amp;quot;, cce_tf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To answer the following question more directly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;However, while training, my loss value is greater than one (almost equal to 1.2) in the first epoch, but until epoch 5 it comes near 0.8. Is it normal? If not, how can I solve this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. It can happen, as explained above. (However, this does not mean that you do not have mistakes in your code.)&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T12:00:53.390" LastActivityDate="2020-11-19T12:00:53.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24702" PostTypeId="1" AcceptedAnswerId="24703" CreationDate="2020-11-18T19:52:55.390" Score="1" ViewCount="96" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&lt;p&gt;From my understanding (and following along with &lt;a href=&quot;http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or classification layer) becomes more separate. As such, we can then apply a simple classifier(s) to the representation to chop up the regions where the different classes exist (as shown by &lt;a href=&quot;https://medium.com/@vivek.yadav/how-neural-networks-learn-nonlinear-functions-and-classify-linearly-non-separable-data-22328e7e5be1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;If this is true and say we have some noisy data where the classes are not easily separable, would it make sense to push the input to a higher dimension, so we can more easily separate it later in the network?&lt;/p&gt;&#xA;&lt;p&gt;For example, I have some tabular data that is a bit noisy, say it has 50 dimensions (input size of 50). To me, it seems logical to project the data to a higher dimension, such that it makes it easier for the classifier to separate. In essence, I would project the data to say 60 dimensions (layer out dim = 60), so the network can represent the data with more dimensions, allowing us to linearly separate it. (I find this similar to how SVMs can classify the data by pushing it to a higher dimension).&lt;/p&gt;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&lt;p&gt;Why, if the above is correct, do we not see many neural network architectures projecting the data into higher dimensions first then reducing the size of each layer thereafter?&lt;/p&gt;&#xA;&lt;p&gt;I learned that if we have more hidden nodes than input nodes, the network will memorize rather than generalize.&lt;/p&gt;&#xA;" OwnerUserId="42384" LastEditorUserId="2444" LastEditDate="2020-11-19T00:49:32.357" LastActivityDate="2020-11-24T15:41:34.017" Title="Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;&lt;multilayer-perceptron&gt;&lt;generalization&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24703" PostTypeId="2" ParentId="24702" CreationDate="2020-11-18T21:36:31.613" Score="0" Body="&lt;p&gt;To better understand this you should think in terms of &lt;strong&gt;capacity&lt;/strong&gt;. Capacity is a theoretical notion that shows how much information your network can model.&lt;/p&gt;&#xA;&lt;p&gt;The capacity of a network (given sufficient training) ties in directly with the bias/variance tradeoff:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;too little capacity and your network isn't able to learn the complex relationships in the data.&lt;/li&gt;&#xA;&lt;li&gt;too much capacity and your network has the capability of learning the noise in the dataset, besides the useful relationships.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;At some point a network reaches a point where it has a high enough capacity to &lt;strong&gt;memorize&lt;/strong&gt; the whole training set!&lt;/p&gt;&#xA;&lt;p&gt;Now, by increasing the number of hidden neurons you essentially increase the capacity of your network. If the network already has enough capacity to &lt;em&gt;learn&lt;/em&gt; the problem, then by increasing the neurons you are giving the network the capability of overfitting more easily.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: all the above assume that the network is trained sufficiently (i.e. no early stopping, etc).&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-11-18T21:36:31.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24704" PostTypeId="1" CreationDate="2020-11-18T21:56:35.170" Score="2" ViewCount="43" Body="&lt;p&gt;In &lt;em&gt;Artiﬁcial Intelligence A Modern Approach&lt;/em&gt;, search algorithms are divided into &lt;strong&gt;tree-search&lt;/strong&gt; version and &lt;strong&gt;graph-search&lt;/strong&gt; version, where the &lt;strong&gt;graph-search&lt;/strong&gt; version keeps an extra &lt;strong&gt;explored set&lt;/strong&gt; to avoid expanding nodes repeatedly.&lt;/p&gt;&#xA;&lt;p&gt;However, in &lt;strong&gt;breadth-first search&lt;/strong&gt; or &lt;strong&gt;A* search&lt;/strong&gt;, I think we still need to keep the expanded nodes in the memory so that we can &lt;strong&gt;track the path&lt;/strong&gt; from the root to the goal. (The node structure contains its parent node which can be used to extract the solution path, but only if we have those nodes kept in the memory)&lt;/p&gt;&#xA;&lt;p&gt;So, if I'm right, why do we need the &lt;strong&gt;tree-search&lt;/strong&gt; version in BFS and A*, given that the expanded nodes still need to be stored? Why not just use the &lt;strong&gt;graph-search&lt;/strong&gt; version then?&lt;/p&gt;&#xA;&lt;p&gt;If I'm wrong, so how do we track the solution path given that the expanded nodes have been discarded?&lt;/p&gt;&#xA;" OwnerUserId="42387" LastEditorUserId="2444" LastEditDate="2020-11-18T22:09:08.487" LastActivityDate="2020-11-18T22:09:08.487" Title="Why do we use the tree-search version of breadth-first search or A*?" Tags="&lt;search&gt;&lt;a-star&gt;&lt;breadth-first-search&gt;&lt;graph-search&gt;&lt;tree-search&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24705" PostTypeId="5" CreationDate="2020-11-19T01:23:54.603" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T01:23:54.603" LastActivityDate="2020-11-19T01:23:54.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24706" PostTypeId="4" CreationDate="2020-11-19T01:23:54.603" Score="0" Body="For questions about observation spaces in the context of reinforcement learning and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T01:23:54.603" LastActivityDate="2020-11-19T01:23:54.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24708" PostTypeId="1" CreationDate="2020-11-19T02:28:30.717" Score="0" ViewCount="26" Body="&lt;p&gt;I consider pre-training a &lt;a href=&quot;https://github.com/ultralytics/yolov5&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOLOv5&lt;/a&gt; with &lt;a href=&quot;https://storage.googleapis.com/openimages/web/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Open Images Object Detection dataset&lt;/a&gt;. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-tune the model on MS COCO dataset.&lt;/p&gt;&#xA;&lt;p&gt;I would like to do it, if I can improve AP by ~7%. Do you think that it is possible, and I have a logical expectation? Unfortunately, I could not find anywhere anyone has tried an Open Images pre-trained object detector with MSCOCO training.&lt;/p&gt;&#xA;" OwnerUserId="10664" LastEditorUserId="2444" LastEditDate="2020-11-19T12:36:04.937" LastActivityDate="2020-11-19T12:36:04.937" Title="Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?" Tags="&lt;object-detection&gt;&lt;yolo&gt;&lt;pretrained-models&gt;&lt;fine-tuning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24710" PostTypeId="1" CreationDate="2020-11-19T06:35:17.873" Score="0" ViewCount="21" Body="&lt;p&gt;Suppose I'm using the Spambase dataset, with 57 real continuous variables and one binary label, and I'd like to implement the &lt;a href=&quot;https://www.geeksforgeeks.org/ml-find-s-algorithm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Find-S&lt;/a&gt; or LGG or similar algorithms.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I represent the dataset for usage in the algorithm? I've tried binning by standard deviation, binning by heuristics.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there an adaption for concept learning for continuous variables?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="41728" LastEditorUserId="2444" LastEditDate="2020-11-19T11:43:02.980" LastActivityDate="2020-11-19T11:43:02.980" Title="Utilizing continuous variables in concept learning algorithms" Tags="&lt;machine-learning&gt;&lt;computational-learning-theory&gt;&lt;concept-learning&gt;&lt;find-s&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24711" PostTypeId="2" ParentId="24672" CreationDate="2020-11-19T07:38:15.723" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;How to extract the main content text from a formated text file?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;I am not sure that just a neural network is the best approach to your problem.&lt;/h3&gt;&#xA;&lt;p&gt;Traditional &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_language_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural language processing&lt;/a&gt; software are using something else, and generally using a complex mix of several techniques. I am supposing you are processing written text available as some file (in a file format you are &lt;em&gt;very&lt;/em&gt; familiar with, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Office_Open_XML&quot; rel=&quot;nofollow noreferrer&quot;&gt;OOXML&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/PDF&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/HTML5&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTML5&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Read the &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural-language_understanding&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipage on natural-language understanding&lt;/a&gt; and the one on &lt;a href=&quot;https://en.wikipedia.org/wiki/Parse_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;parse trees&lt;/a&gt; (or concrete syntax trees).&lt;/p&gt;&#xA;&lt;p&gt;BTW, you might use &lt;a href=&quot;https://www.latex-project.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LaTeX&lt;/a&gt; or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lout_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lout&lt;/a&gt; formatter to produce some PDF file. Both are open-source software (easily available on most Linux distributions, including &lt;a href=&quot;https://debian.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Debian&lt;/a&gt; or &lt;a href=&quot;https://ubuntu.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ubuntu&lt;/a&gt;). I recommend you to try generating some PDF file using them, and experiment on the generated PDF file. And a lot of AI papers are available (as preprints) in PDF form.&lt;/p&gt;&#xA;&lt;p&gt;You could also use, as a PDF input to experiment your software, &lt;a href=&quot;http://starynkevitch.net/Basile/bismon-chariot-doc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; or &lt;a href=&quot;http://starynkevitch.net/Basile/refpersys-design.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;that&lt;/a&gt; draft reports (you might enjoy reading them too...). &lt;strong&gt;If in 2021 your software is capable of &amp;quot;understanding&amp;quot; and &amp;quot;abstracting/summarizing&amp;quot; these PDF files, please send me an email&lt;/strong&gt; to &lt;code&gt;basile@starynkevitch.net&lt;/code&gt; explaining (in written English) how you did build your neural network and what is the output of your software.&lt;/p&gt;&#xA;&lt;p&gt;There are several issues:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;extracting the non-textual things (e.g. HTML tags from HTML input, or strings from a PDF file, or some &lt;a href=&quot;https://en.wikipedia.org/wiki/LaTeX&quot; rel=&quot;nofollow noreferrer&quot;&gt;LaTeX&lt;/a&gt; one).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;detecting the human language used in your text (e.g. French or English or Russian or Chinese). &lt;a href=&quot;https://en.wikipedia.org/wiki/N-gram&quot; rel=&quot;nofollow noreferrer&quot;&gt;N-gram&lt;/a&gt; based techniques come to mind.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;having a data structure or database representing a dictionnary of a thousand (at least) of significant words (in English or Russian or whatever human language you are interested in) related to the domain you want to handle (that dictionary would be different if you want to parse weather forecasts or documentation related to the automotive industry, since the word &lt;code&gt;pressure&lt;/code&gt; or &lt;code&gt;speed&lt;/code&gt; relates to different concepts. Notice also that &amp;quot;weather&amp;quot; and &amp;quot;time&amp;quot; are expressed in French by the &lt;em&gt;same&lt;/em&gt; word: &amp;quot;temps&amp;quot; - as in &amp;quot;le temps qu'il fait&amp;quot; for ongoing weather and &amp;quot;le temps qui passe&amp;quot; for the flow of time). A &amp;quot;Queen&amp;quot; is not the same for a chess player and an historian. A program translating -or just analyzing- chess comments from English won't use the same word for translating / understanding &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bishop_(chess)&quot; rel=&quot;nofollow noreferrer&quot;&gt;bishop&lt;/a&gt;&amp;quot; (in chess, &amp;quot;fou&amp;quot; in French, literally the crazy guy, unrelated to religion; in Russian chess books it would be &amp;quot;слон&amp;quot;,  literally an elephant) than another program translating / analyzing historical comments from English (e.g. about &lt;a href=&quot;https://en.wikipedia.org/wiki/Mary,_Queen_of_Scots&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mary Stuart&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;modeling inside your software some domain-specific knowledge related to your analyzed text, since you would handle differently weather forecast text to textual comments of chess competitions, or textual exercises in any computer science or programming book (like &lt;a href=&quot;https://en.wikipedia.org/wiki/Introduction_to_Algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLRS&lt;/a&gt;). You could use some &lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)&quot; rel=&quot;nofollow noreferrer&quot;&gt;frame&lt;/a&gt;-based representations, like in &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; or in &lt;a href=&quot;https://en.wikipedia.org/wiki/Cyc&quot; rel=&quot;nofollow noreferrer&quot;&gt;CyC&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;building a &lt;a href=&quot;https://en.wikipedia.org/wiki/Semantic_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;semantic network&lt;/a&gt; representing the input text. I believe you might need some prior one representing domain-specific knowledge in the area of the analyzed text (e.g. a program analyzing comments on chess games needs to know the rule of chess; another program analyzing &lt;a href=&quot;https://stackoverflow.com/&quot;&gt;StackOverflow&lt;/a&gt; answers probably needs to know something about operating systems in general). In think that in English &amp;quot;overflow&amp;quot; or &amp;quot;overheating&amp;quot; means &lt;em&gt;very different concepts&lt;/em&gt; to software developers and to weather forecasters or climate experts.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Look also for inspiration into &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; of the late Jacques Pitrat. He did wrote an interesting book on your topic.&lt;/p&gt;&#xA;&lt;p&gt;You might look inside the &lt;a href=&quot;https://www.decoder-project.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DECODER&lt;/a&gt; European project, and read more about &lt;a href=&quot;https://en.wikipedia.org/wiki/Expert_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;expert systems&lt;/a&gt; and their &lt;a href=&quot;https://en.wikipedia.org/wiki/Inference_engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;inference engine&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Knowledge_base&quot; rel=&quot;nofollow noreferrer&quot;&gt;knowledge bases&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Your project could give you some PhD.&lt;/h2&gt;&#xA;&lt;p&gt;You certainly need several years of work to achieve your goals. I suggest contacting some academic in your area to be your advisor.&lt;/p&gt;&#xA;&lt;p&gt;Notice that on Linux the &lt;a href=&quot;https://linux.die.net/man/1/pdftotext&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pdf2text&lt;/code&gt; software&lt;/a&gt; is extracting text from PDF files. It is open-source, but I won't say it is an AI software. However, you could use it thru &lt;a href=&quot;https://man7.org/linux/man-pages/man3/popen.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;popen(3)&lt;/a&gt;. See also &lt;a href=&quot;https://man7.org/linux/man-pages/man7/regex.7.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;regex(7)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;BTW, the &lt;a href=&quot;https://en.wikipedia.org/wiki/PDF&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt; specification is public as &lt;a href=&quot;https://www.iso.org/standard/63534.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ISO 32000-2:2017&lt;/a&gt; (and is related to &lt;a href=&quot;https://en.wikipedia.org/wiki/PostScript&quot; rel=&quot;nofollow noreferrer&quot;&gt;PostScript&lt;/a&gt;). Get it and read it, and see also this &lt;a href=&quot;https://youtu.be/KmP7pbcAl-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;youtube video&lt;/a&gt; or &lt;a href=&quot;https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/pdf_reference_archives/PDFReference.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this 978 pages document&lt;/a&gt;. On Linux, most PDF files can usually be inspected with &lt;a href=&quot;https://man7.org/linux/man-pages/man1/od.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;od(1)&lt;/a&gt; or &lt;a href=&quot;https://man7.org/linux/man-pages/man1/less.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;less(1)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My HP Office Pro 8610 printer (connected to a Linux desktop) is capable of printing some PDF and of scanning into a PDF file. But if I print on paper some PDF file and scan it, the PDF file did change a lot, even if visually it looks the same.&lt;/p&gt;&#xA;&lt;p&gt;Notice that some drawings -or photos- could be embedded in a PDF file, and appear to a non-blind human reader as letters.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-11-19T18:49:24.133" LastActivityDate="2020-11-19T18:49:24.133" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="24712" PostTypeId="1" CreationDate="2020-11-19T09:13:14.700" Score="0" ViewCount="18" Body="&lt;p&gt;Let's say I have a dataset for binary classification. And I am going to conduct 5-fold cross-validation and get AUC scores for each fold (mean AUC score too). However, if I set the training epoch to &lt;span class=&quot;math-container&quot;&gt;$30$&lt;/span&gt;, and, with that classifier, test all my test folds iteratively, I will be getting AUC scores of test folds at specific weights of a classifier (at &lt;span class=&quot;math-container&quot;&gt;$30$&lt;/span&gt;s epochs).&lt;/p&gt;&#xA;&lt;p&gt;Is it a fair evaluation if I watch some metrics (like validation loss) until stagnation and reduce the learning rate accordingly for each fold separately and get their respective AUC score, then report the mean AUC score?&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="2444" LastEditDate="2020-11-19T14:52:12.163" LastActivityDate="2020-11-19T14:52:12.163" Title="Is it a fair evaluation if each model of k-fold cross validation is trained with different epochs and the mean AUC score gathered out of k folds?" Tags="&lt;cross-validation&gt;&lt;binary-classification&gt;&lt;epochs&gt;&lt;roc-auc&gt;&lt;validation-loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24714" PostTypeId="2" ParentId="24673" CreationDate="2020-11-19T10:22:02.620" Score="1" Body="&lt;p&gt;OK, so I found the answer - it is to use multiple linear regression. I think this can be marked a solved, but I don't have enough rep to do that.&lt;/p&gt;&#xA;" OwnerUserId="42358" LastActivityDate="2020-11-19T10:22:02.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24715" PostTypeId="5" CreationDate="2020-11-19T11:46:02.813" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T11:46:02.813" LastActivityDate="2020-11-19T11:46:02.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24716" PostTypeId="4" CreationDate="2020-11-19T11:46:02.813" Score="0" Body="For questions about the Find-S concept learning algorithm presented in section 2.4 (page 26) of Tom Mitchell's &quot;Machine Learning&quot; book, where the name stands for &quot;Finding a maximally Specific hypothesis&quot;." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T11:46:02.813" LastActivityDate="2020-11-19T11:46:02.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24717" PostTypeId="5" CreationDate="2020-11-19T11:48:57.803" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T11:48:57.803" LastActivityDate="2020-11-19T11:48:57.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24718" PostTypeId="4" CreationDate="2020-11-19T11:48:57.803" Score="0" Body="For questions about concept learning in the context of machine learning and learning theory. See chapter 2 of Tom Mitchell's book &quot;Machine Learning&quot; for more details." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T11:48:57.803" LastActivityDate="2020-11-19T11:48:57.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24719" PostTypeId="1" CreationDate="2020-11-19T12:51:02.663" Score="2" ViewCount="242" Body="&lt;p&gt;I've been training a VAE to reconstruct human names and when I train it on a batch size of 100+ after about 5 hours of training it tends to just output the same thing regardless of the input and I'm using teacher forcing as well. When I use a lower batch size for example 1 it super overfitted and a batch size of 16 tended to give a much better generalization. Is there something about VAEs that would make this happen? Or is it just my specific problem?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastActivityDate="2020-11-19T12:51:02.663" Title="Why would a VAE train much better with batch sizes closer to 1 over batch size of 100+?" Tags="&lt;variational-autoencoder&gt;&lt;batch-size&gt;&lt;batch-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24720" PostTypeId="5" CreationDate="2020-11-19T12:52:11.330" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T12:52:11.330" LastActivityDate="2020-11-19T12:52:11.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24721" PostTypeId="4" CreationDate="2020-11-19T12:52:11.330" Score="0" Body="For questions about Prim's algorithm that finds the minimum spanning tree. The questions need to be asked in the context of AI." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T12:52:11.330" LastActivityDate="2020-11-19T12:52:11.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24722" PostTypeId="5" CreationDate="2020-11-19T12:55:28.943" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T12:55:28.943" LastActivityDate="2020-11-19T12:55:28.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24723" PostTypeId="4" CreationDate="2020-11-19T12:55:28.943" Score="0" Body="For questions about the shortest path Dijkstra's algorithm, which was conceived by Edsger W. Dijkstra in 1956 and published three years later." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T12:55:28.943" LastActivityDate="2020-11-19T12:55:28.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24724" PostTypeId="1" AcceptedAnswerId="24738" CreationDate="2020-11-19T14:02:00.620" Score="1" ViewCount="111" Body="&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=0g4j2k_Ggc4&amp;amp;list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&amp;amp;index=5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is David Silver's lecture on that&lt;/a&gt;. Look at 9:30 to 10:30.&lt;/p&gt;&#xA;&lt;p&gt;He says that, since it is model-free learning, the environment's dynamics are unknown, so the action-value function &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is used.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;But then state-values are already calculated (via first-visit or every-visit). So, why aren't these values used?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Secondly, even if we were to use &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s,a) = R(s) + \gamma \sum_{s'}P(s'|s,a)V^{\pi}(s')$&lt;/span&gt;, so we still need to know the transition model, which is unknown.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;" OwnerUserId="42398" LastEditorUserId="2444" LastEditDate="2020-11-20T02:13:40.400" LastActivityDate="2020-11-20T14:09:51.520" Title="Why does Monte Carlo policy evaluation relies on action-value function rather than state-value function?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;value-functions&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24725" PostTypeId="5" CreationDate="2020-11-19T14:55:26.027" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T14:55:26.027" LastActivityDate="2020-11-19T14:55:26.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24726" PostTypeId="4" CreationDate="2020-11-19T14:55:26.027" Score="0" Body="For questions about the validation loss, i.e. the loss of a model (e.g. a neural network) computed on the validation data (or dataset), which is different from the testing/test data (i.e. the data that you use after having trained your model(s) and selected the best model according to the validation loss/performance)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T14:55:26.027" LastActivityDate="2020-11-19T14:55:26.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24727" PostTypeId="5" CreationDate="2020-11-19T15:21:18.030" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T15:21:18.030" LastActivityDate="2020-11-19T15:21:18.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24728" PostTypeId="4" CreationDate="2020-11-19T15:21:18.030" Score="0" Body="For questions about the C programming language in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-19T15:21:18.030" LastActivityDate="2020-11-19T15:21:18.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24730" PostTypeId="2" ParentId="24642" CreationDate="2020-11-19T15:41:16.230" Score="1" Body="&lt;p&gt;I will answer the first question question based on information I have gathered so far. The probability of each action for the &lt;span class=&quot;math-container&quot;&gt;$\textbf{two player zero sum game}$&lt;/span&gt; need not be the same for both players. It turns out that in the game of Morra, the probability vectors just turn out to be the same value.&lt;/p&gt;&#xA;&lt;p&gt;In general to determine &lt;span class=&quot;math-container&quot;&gt;$\textbf{optimal mixed strategies}$&lt;/span&gt; for two player two action game: Suppose we have 2 actions &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;. In a mixed strategy, we let the probability of the row player taking action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; be &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. The probability of the row player taking &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; is then &lt;span class=&quot;math-container&quot;&gt;$1-p$&lt;/span&gt;. Now, we will try to find the probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; such that the column player is indifferent to either action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;. (I.e the expected payoff for column player if row player plays &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the same for either actions).&lt;/p&gt;&#xA;&lt;p&gt;Solving for &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, we know that the column player can play any mixed strategy since any strategy played by the column player will yield the same payoff (since expected payoff is the same for either &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;). Since column player can play any mixed strategy, we want to find a probability distribution over &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt; for the column player such that the row player is indifferent to either action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;. We let the column player take action &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; with probability &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. It turns out we can repeat the same process for the column player and find the values of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. The game value for the row player can be computed from &lt;span class=&quot;math-container&quot;&gt;$x^TAy$&lt;/span&gt; and this turns out to be the same value for the column player as well. (Only in zero sum games)&lt;/p&gt;&#xA;&lt;p&gt;Doing this process yields a &lt;span class=&quot;math-container&quot;&gt;$\textbf{mixed strategy Nash Equilibrium}$&lt;/span&gt;, using such an approach does not always work however in the scenario when one action choice always dominates another action choice. (I.e in the case of the prisoner's dilemma of Alice and Bob, no matter what mixed strategy Bob tries to use, there will never be a mixed strategy such that Alice is indifferent to refusing or testifying). She will always testify.&lt;/p&gt;&#xA;&lt;p&gt;I am not too sure about how to solve zero sum games involving more than 2 actions. I think linear programming would have to be used but i am not to familiar with how it can be applied.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastActivityDate="2020-11-19T15:41:16.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24731" PostTypeId="2" ParentId="205" CreationDate="2020-11-19T16:51:05.343" Score="1" Body="&lt;p&gt;Software Reverse Engineering is one of my hobby.&lt;/p&gt;&#xA;&lt;p&gt;First things first: forget about headers. All information about headers and separate C file is gone.&lt;/p&gt;&#xA;&lt;p&gt;You're missing some crucial step, IMHO.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Compilation creates one or multiple object files (.o), then the linker creates an executable.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You should work from disassembled code. The disassembler works pretty well with some exceptions (self-modifying code, self-extracting executable, various obfuscation techniques) and will take care of a lot of work for you: identifying various sections, finding functions, guessing (fairly accurately) the calling convention.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then the compiler optimization will mess up your code in a very clever way and some part of your original code will never ever be seen again (hey, look, your 200 lines of bugged code always return 0 anyway so I'll just replace them with &amp;quot;xor eax, eax&amp;quot;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sometimes, it's fine, and sometimes it produces unreadable C code (vectorizations that have no C equivalent and will be decompiled into hundreds of lines of intrinsic instead of a fine readable &amp;quot;for&amp;quot; loop).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm not done yet. You also have exceptions and interrupts, structures, union, function pointers, function inlining, threading, system call and signals, loop unrolling, etc.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Going down (from human-readable to binary) is relatively easy compared to going up (decompilation) because so much information is lost during the compilation process.&lt;/p&gt;&#xA;&lt;p&gt;My best bet would be to have a bunch of disassembled function produced by a disassembler and produce an LLVM intermediate representation using your AI, then compare it with the LLVM IR produced by Clang (&lt;code&gt;clang -S -emit-llvm foo.c&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;An infinite quantity of C code can produce the exact same code. Therefore, I think it's meaningless to make an AI read C code for the purpose of decompilation: the information is lost forever.&lt;/p&gt;&#xA;&lt;p&gt;Commercial and Open/Free decompilers do not produce C code either, they produce some kind of pseudo-C full of errors, missing code, or code even less readable than the ASM.&lt;/p&gt;&#xA;&lt;p&gt;The following code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;int main() {&#xA;    int toto = 0x0000BEEF;&#xA;    int titi = 0xDEAD0000;&#xA;    toto = toto | titi;&#xA;    return toto;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;produces this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;int __cdecl main(int argc, const char **argv, const char **envp)&#xA;{&#xA;     return -559038737;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is the disassembled version:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mov     eax, 0DEADBEEFh&#xA;retn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Plus a few thousands lines of assembly that are unrelated to your code but are needed to make the program work.&lt;/p&gt;&#xA;&lt;p&gt;You can't go back and you have no way of knowing this is the exact same code unless&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;you can do static analysis (very easy in this case, but absurdly difficult in the real world)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;or compare the IR or ASM produced by both code with the same compiler with the same options on the same architecture and operating system.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42373" LastEditorUserId="2444" LastEditDate="2020-11-19T21:34:48.317" LastActivityDate="2020-11-19T21:34:48.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24732" PostTypeId="1" AcceptedAnswerId="24776" CreationDate="2020-11-19T18:28:29.267" Score="1" ViewCount="188" Body="&lt;p&gt;I know this is a general question, but I'm just looking for intuition. What are the characteristics of problems (in terms of state-space, action-space, environment, or anything else you can think of) that are well solvable with the family of DQN algorithms? What kind of problems are not well fit for DQNs?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-11-21T21:32:33.370" LastActivityDate="2020-11-22T20:03:13.030" Title="What kind of problems is DQN algorithm good and bad for?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;applications&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24734" PostTypeId="2" ParentId="24682" CreationDate="2020-11-19T23:07:13.160" Score="1" Body="&lt;p&gt;According to &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0134610997&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach 4th edition&lt;/a&gt; in IDA* the cutoff is the &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;-cost(&lt;span class=&quot;math-container&quot;&gt;$g+h$&lt;/span&gt;); at each iteration, the cutoff value is the smallest &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;-cost of any node that exceeded the cutoff on the previous iteration.&lt;/p&gt;&#xA;&lt;p&gt;In other words, each iteration exhaustively searches an &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;-contour, finds a node just beyond that contour, and uses that node's &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;-cost as the next contour.&lt;/p&gt;&#xA;&lt;p&gt;And we must test if the node is a goal node when it was selected for expansion, otherwise, the algorithm is not optimal anymore (the proof is similar to that one in A*).&lt;/p&gt;&#xA;&lt;p&gt;I think the animation that the site provided is misleading because in the code which is written in the last of the same site we have that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;function Search(node, g, threshold)              //recursive function&#xA;{&#xA;&#xA;  f = g + heuristic(node);&#xA;  if(f &amp;gt; threshold)             //greater f encountered&#xA;         return f;&#xA;  if(node == Goal)               //Goal node found&#xA;         return FOUND;&#xA;  integer min = MAX_INT;     //min = Minimum integer&#xA;  foreach(tempnode in nextnodes(node))&#xA;  {&#xA;&#xA;     //recursive call with next node as current node for depth search&#xA;     integer temp=search(tempnode, g + cost(node, tempnode), threshold);  &#xA;     if(temp == FOUND)            //if goal found&#xA;       return FOUND;&#xA;     if(temp &amp;lt; min)     //find the minimum of all 'f' greater than threshold encountered                                &#xA;       min = temp;&#xA;&#xA;     }&#xA;     return min;  //return the minimum 'f' encountered greater than threshold&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And in the previous code, we test if the node is the goal node only when it was selected for expansion.&lt;/p&gt;&#xA;" OwnerUserId="36578" LastEditorUserId="2444" LastEditDate="2020-11-21T11:49:06.617" LastActivityDate="2020-11-21T11:49:06.617" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24735" PostTypeId="1" CreationDate="2020-11-19T23:48:55.430" Score="1" ViewCount="35" Body="&lt;p&gt;My question may be a bit hard to explain...&#xA;My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2.&lt;/p&gt;&#xA;&lt;p&gt;From this action_mean, I create a normal distribution where the network has to learn to adjust the standard deviation. The output of the network is a sample of this normal distribution.&lt;/p&gt;&#xA;&lt;p&gt;Since the value of action_mean is directly taken from the input, somehow the gradient can't be computed or gives &lt;em&gt;Nones&lt;/em&gt;, respectively, because the output of the net is not completely connected with the input.&lt;/p&gt;&#xA;&lt;p&gt;Would there be a way to link my action_mean with the input value, without changing the input values itself? To describe my problem, I attached a simplified computational graph how tensorboard shows it.&lt;/p&gt;&#xA;&lt;p&gt;I would be very thankful for any help!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q1uAk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q1uAk.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41715" LastActivityDate="2020-11-19T23:48:55.430" Title="How to use unmodified input in neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;gradient-descent&gt;&lt;graphs&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24736" PostTypeId="1" CreationDate="2020-11-20T00:15:00.213" Score="2" ViewCount="69" Body="&lt;p&gt;I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an instance segmentation task.&lt;/p&gt;&#xA;&lt;p&gt;I am thinking about using Faster R-CNN, Mask R-CNN, or ResNet for the instance segmentation task, which is pre-trained in an SSL way by solving a pretext task, with the aim that this will lead to higher accuracy and also teach the CNNs with fewer examples during the downstream task.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to use SSL to pre-train e.g. a faster R-CNN on a pretext task (for example, rotation), then use this pre-trained model for instance segmentation with the aim to get better accuracy?&lt;/p&gt;&#xA;" OwnerUserId="42034" LastEditorUserId="2444" LastEditDate="2020-11-20T16:50:38.983" LastActivityDate="2020-11-20T16:56:45.937" Title="Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?" Tags="&lt;transfer-learning&gt;&lt;r-cnn&gt;&lt;self-supervised-learning&gt;&lt;instance-segmentation&gt;&lt;pretext-tasks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24738" PostTypeId="2" ParentId="24724" CreationDate="2020-11-20T01:55:48.037" Score="3" Body="&lt;p&gt;In Model Based Reinforcement learning, state and state-action values for all states can be calculated based on the bellman equations. The equations are taken from &lt;a href=&quot;https://ai.stanford.edu/%7Eang/papers/icml00-irl.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrew Ng's Algorithms for Inverse Reinforcement Learning&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$$V^{\pi}(s) = R(s) + \gamma \sum_{s'}P(s'|s,a)V^{\pi}(s') \\ Q^{\pi}(s,a) = R(s) + \gamma \sum_{s'}P(s'|s,a)V^{\pi}(s')$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this setting, &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}$&lt;/span&gt; can be obtained from &lt;span class=&quot;math-container&quot;&gt;$V^{\pi}$&lt;/span&gt; because we have access to the transition model &lt;span class=&quot;math-container&quot;&gt;$P(s'|s,a)$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}$&lt;/span&gt; values allow us to carry out a step in &lt;span class=&quot;math-container&quot;&gt;$\textbf{policy improvement}$&lt;/span&gt; as in policy iteration.&lt;/p&gt;&#xA;&lt;p&gt;To answer the first bullet point, the first visit or every state visit policy evaluation in the model free setting for &lt;span class=&quot;math-container&quot;&gt;$\textbf{state values}$&lt;/span&gt; is not helpful in determining how to carry out model free control because we cannot compute &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s,a)$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$V^{\pi}$&lt;/span&gt; in the model free case.&lt;/p&gt;&#xA;&lt;p&gt;The update for SARSA in model free control is &lt;span class=&quot;math-container&quot;&gt;$$Q(s,a) \rightarrow Q(s,a) + \alpha (r(s) + \gamma Q(s',a') - Q(s,a))$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Even though we do not know the transition model, we are essentially &lt;span class=&quot;math-container&quot;&gt;$\textbf{sampling}$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$P(s'|s,a)$&lt;/span&gt; by allowing the environment to provide us the possible next states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; that we may end up in. The following update for SARSA is equivalent to computing &lt;span class=&quot;math-container&quot;&gt;$$Q^{\pi}(s,a) = R(s) + \gamma E_{s' \sim P(s'|s,a)}[Q^{\pi}(s',a')]$$&lt;/span&gt;&#xA;Essentially this should give the same &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}(s,a)$&lt;/span&gt; values when we have the ground truth &lt;span class=&quot;math-container&quot;&gt;$P(s'|s,a)$&lt;/span&gt; values for the model free case.&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="32780" LastEditDate="2020-11-20T14:09:51.520" LastActivityDate="2020-11-20T14:09:51.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24740" PostTypeId="1" CreationDate="2020-11-20T08:24:38.930" Score="1" ViewCount="117" Body="&lt;p&gt;I am building my first ANN from scratch. I know that I need a transfer function and I want to use the sigmoid function as my teacher recommended that. That function can be between 0 and 1, but my input values for the network are between -5 and 20. Someone told me that I need to scale the function so that it is in the range of -5 and 20 instead of 0 and 1. Is this true? Why?&lt;/p&gt;&#xA;" OwnerUserId="42414" LastEditorUserId="2444" LastEditDate="2020-11-20T13:01:26.413" LastActivityDate="2020-11-20T21:04:19.920" Title="How to use sigmoid as transfer function when input is not (0,1) range in ANN?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;&lt;sigmoid&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24741" PostTypeId="1" AcceptedAnswerId="24903" CreationDate="2020-11-20T09:35:38.227" Score="3" ViewCount="159" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1610.02391.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GradCAM paper&lt;/a&gt; section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification problem). That is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A^k$&lt;/span&gt; the final feature maps&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha_k^c$&lt;/span&gt; the average pooled partial derivatives of the output class scores &lt;span class=&quot;math-container&quot;&gt;$y^c$&lt;/span&gt; with respect to the the final feature maps &lt;span class=&quot;math-container&quot;&gt;$A_k$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The second point is clear to me. The stronger the derivative, the more important the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th channel of the final feature maps is.&lt;/p&gt;&#xA;&lt;p&gt;The first point is not, because the implicit assumption is that non-zero activations have more significance than activations close to zero. I know it's tempting to take that as a given, but for me it's not so obvious. After all, neurons have biases, and a bias can arbitrarily shift the reference point, and hence what 0 means. We can easily transform two neurons [0, 1] to [1, 0] with a linear transformation.&lt;/p&gt;&#xA;&lt;p&gt;So why should it matter which regions of the final feature maps are strongly activated?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To address a comment further down, this table explains why I'm thinking about &lt;strong&gt;magnitude&lt;/strong&gt; rather than &lt;strong&gt;sign&lt;/strong&gt; of the activations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZjQB7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZjQB7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It comes from thinking about the possible variations of&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L_{Grad-CAM}^c = ReLU\bigl( \sum_k \alpha_k^c A^k \bigr)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2020-11-29T17:32:12.337" LastActivityDate="2020-11-29T17:32:12.337" Title="In GradCAM, why is activation strength considered an indicator of relevant regions?" Tags="&lt;neural-networks&gt;&lt;deep-neural-networks&gt;&lt;explainable-ai&gt;&lt;grad-cam&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24742" PostTypeId="1" CreationDate="2020-11-20T11:21:51.413" Score="2" ViewCount="38" Body="&lt;p&gt;I am trying to implement the basic RL algorithms to learn on &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/reinforcejs/gridworld_td.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; 10x10 GridWorld (from REINFORCEJS by Kaparthy).&lt;/p&gt;&#xA;&lt;p&gt;Currently I am stuck at TD(0). No matter how many episodes I run, when I am updating the policy after all episodes are done according to the value function I dont get the optimal value function which I obtain when &lt;em&gt;toggle td learning&lt;/em&gt; on the grid from the link I provided above.&lt;/p&gt;&#xA;&lt;p&gt;The only way I am getting the optimal policy is when I am updating the policy in each iteration and then following the updated policy when calculating the TD target. But according to the algorithm from Sutton and Barto a given policy (which is fixed over all episodes - see line 1 below) should be evaluated.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xMqpp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xMqpp.png&quot; alt=&quot;TD0FromSuttonAndBarto&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using &lt;code&gt;alpha=0.1&lt;/code&gt; and &lt;code&gt;gamma=0.9&lt;/code&gt; after 1000 episodes my Td(0) algo finds the following value function&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[-0.04 -0.03 -0.03 -0.05 -0.08 -0.11 -0.09 -0.06 -0.05 -0.07]&#xA; [-0.08 -0.04 -0.04 -0.06 -0.1  -0.23 -0.11 -0.06 -0.07 -0.11]&#xA; [-0.13  -inf  -inf  -inf  -inf -0.58  -inf  -inf  -inf -0.25]&#xA; [-0.24 -0.52 -1.23 -2.6   -inf -1.4  -1.28 -1.12 -0.95 -0.62]&#xA; [-0.28 -0.49 -0.87 -1.28  -inf -2.14 -2.63 -1.65 -1.38 -1.04]&#xA; [-0.27 -0.42 -0.64 -0.94  -inf  0.97 -1.67 -2.01 -2.79 -1.62]&#xA; [-0.26 -0.36 -0.69 -0.93  -inf -1.17 -1.72 -1.92 -2.75 -1.82]&#xA; [-0.25 -0.38 -0.67 -2.27  -inf -2.62 -2.74 -1.55 -1.31 -1.14]&#xA; [-0.23 -0.31 -0.66 -1.2  -0.98 -1.24 -1.48 -1.02 -0.7  -0.7 ]&#xA; [-0.2  -0.29 -0.43 -0.62 -0.64 -0.77 -0.87 -0.67 -0.54 -0.48]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;-inf&lt;/code&gt;are walls in the grid. If I update the policy according to that value function I am getting.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[['e   ' 'e   ' 'w   ' 'w   ' 'w   ' 'w   ' 'e   ' 'e   ' 's   ' 'w   ']&#xA; ['e   ' 'n   ' 'n   ' 'w   ' 'w   ' 'e   ' 'e   ' 'e   ' 'n   ' 'w   ']&#xA; ['n   ' 'XXXX' 'XXXX' 'XXXX' 'XXXX' 'n   ' 'XXXX' 'XXXX' 'XXXX' 'n   ']&#xA; ['n   ' 'w   ' 'w   ' 's   ' 'XXXX' 'n   ' 'e   ' 'e   ' 'e   ' 'n   ']&#xA; ['n   ' 'w   ' 'w   ' 'w   ' 'XXXX' 's   ' 'n   ' 'n   ' 'n   ' 'n   ']&#xA; ['s   ' 'w   ' 'w   ' 'w   ' 'XXXX' 's   ' 'w   ' 'n   ' 'n   ' 'n   ']&#xA; ['s   ' 'w   ' 'w   ' 'w   ' 'XXXX' 'n   ' 'w   ' 's   ' 's   ' 's   ']&#xA; ['s   ' 'w   ' 'w   ' 'w   ' 'XXXX' 'n   ' 'e   ' 's   ' 's   ' 's   ']&#xA; ['s   ' 'w   ' 'w   ' 's   ' 's   ' 's   ' 's   ' 'e   ' 's   ' 's   ']&#xA; ['n   ' 'w   ' 'w   ' 'w   ' 'w   ' 'w   ' 'e   ' 'e   ' 'e   ' 'w   ']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;(n, w, s, e)&lt;/code&gt; = &lt;code&gt;(north, west, south, east)&lt;/code&gt;. According to the result from Andrey Kaparthys simulation (from &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/reinforcejs/gridworld_td.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) the final policy should look like this&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mOFLZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mOFLZ.png&quot; alt=&quot;ReinforceJSResult&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I did not use any exploration&lt;/li&gt;&#xA;&lt;li&gt;when the agent ends up in the final state &lt;code&gt;[5, 5]&lt;/code&gt; I used the value of the starting state &lt;code&gt;[0, 0]&lt;/code&gt; as the value of its successor state &lt;code&gt;V(S_{t+1})&lt;/code&gt;. The episode is then finished and the agent starts again in the starting state.&lt;/li&gt;&#xA;&lt;li&gt;In &lt;strong&gt;every&lt;/strong&gt; state the agent takes a random action taken from north, west, south or east. If he ends in a wall the value of the next state is just the value where the agent currently is in. And it stays in its state and takes a random action again.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am scratching my head on this for a while now but I dont understand what I am missing.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The value function has to converge. Meaning my policy should be the same as on the website (picture 2)?&lt;/li&gt;&#xA;&lt;li&gt;Only the value of my final state is positive while on the website simulation the whole optimal trajectory has positive values. I know that this is because on the website they update the policy in every step. But shouldn't it also work without updating it iteratively like I did it?&lt;/li&gt;&#xA;&lt;li&gt;Since I am taking a random action (from &lt;code&gt;n,w,s,e&lt;/code&gt;) in every step in every episode for example state &lt;code&gt;[6, 5]&lt;/code&gt; or &lt;code&gt;[6, 6]&lt;/code&gt; (the one below the terminal state) can not really take advantage of the positivity of the terminal state since they are surrounded by more negative-reward-states than this positive-reward-state. This is why after so many iterations the values are getting negative.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I appreciate any help. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="29667" LastEditorUserId="29667" LastEditDate="2020-11-20T11:33:16.710" LastActivityDate="2020-11-20T11:33:16.710" Title="Why is TD(0) not converging to the optimal policy?" Tags="&lt;reinforcement-learning&gt;&lt;reinforce&gt;&lt;td-lambda&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24743" PostTypeId="2" ParentId="24740" CreationDate="2020-11-20T12:26:13.997" Score="1" Body="&lt;p&gt;There are several functions that can be denoted as &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;sigmoid functions&lt;/em&gt;&lt;/a&gt;, such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;logistic function&lt;/strong&gt;&lt;/a&gt; and the &lt;strong&gt;hyperbolic tangent&lt;/strong&gt;, given that they have an &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;-shaped curve. You can find more info about them in &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;the related Wikipedia article&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, when people use the term &lt;em&gt;sigmoid function&lt;/em&gt;, they typically refer to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;logistic function&lt;/em&gt;&lt;/a&gt;, which is a function of the form &lt;span class=&quot;math-container&quot;&gt;$$\sigma: \mathbb{R} \rightarrow [0, 1],$$&lt;/span&gt; So, it accepts any real number and produces a number between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, so you can pass any real number (as input) to the sigmoid.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;em&gt;standard&lt;/em&gt; logistic function &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is mathematically defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma(x)=\frac{1}{1+e^{-x}}\in [0, 1]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the graph of the logistic function. The &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-axis contains the output values, i.e. &lt;span class=&quot;math-container&quot;&gt;$\sigma(x)$&lt;/span&gt; for any &lt;span class=&quot;math-container&quot;&gt;$x \in \mathbb{R}$&lt;/span&gt;, while the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis contains the input values, i.e. &lt;span class=&quot;math-container&quot;&gt;$x \in \mathbb{R}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ApvJ3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ApvJ3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;That function can be between 0 and 1 but my input values for the network are between -5 and 20. Someone told me that I need to scale the function so that it is in the range of -5 and 20 instead of 0 and 1. Is this true? Why?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, in principle, you do &lt;strong&gt;not&lt;/strong&gt; have to scale your inputs, for the reasons I have just explained. However, this does not mean that, in practice, it may not be a good idea to &lt;a href=&quot;https://stats.stackexchange.com/q/10289/82135&quot;&gt;standardize or normalize&lt;/a&gt; your inputs before feeding them into the neural network. See the &lt;strong&gt;vanishing gradient problem (VGP)&lt;/strong&gt; &lt;a href=&quot;https://machinelearningmastery.com/how-to-fix-vanishing-gradients-using-the-rectified-linear-activation-function/&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this article&lt;/a&gt; (by J. Brownlee), &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;or this one&lt;/a&gt; (by M. Nielsen), or in &lt;a href=&quot;https://www.youtube.com/watch?v=qhXZsFVxGKo&amp;amp;ab_channel=Deeplearning.ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; (by A. Ng). Moreover, you should probably actually use another activation function, such as the ReLU, which does not suffer from the VGP (which, nevertheless, should not occur, if your neural network does not contain more than a couple of hidden layers).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T13:17:13.600" LastActivityDate="2020-11-20T13:17:13.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24744" PostTypeId="1" CreationDate="2020-11-20T14:01:46.860" Score="0" ViewCount="36" Body="&lt;p&gt;I have to do a project that detects fabric surface errors and I will use machine learning methods to deal with it. I have a dataset that includes around six thousand fabric surface images with the size 256x256. This dataset is labeled, one thousand of it was labeled as NOK that means fabric surface with error, and the rest was labeled as OK which means fabric surface without an error. &lt;br/&gt;&#xA;I read a lot of papers about fabric surface error detection with machine learning methods, and I saw that &amp;quot;autoencoders&amp;quot; are used to do it. But as I saw that the autoencoders are used in unsupervised learning models without labels. I need to do it with supervised learning models. Is there any model that can I use for fabric surface error detection with images in the supervised learning? Can be autoencoders used for it or is there any better model to do it?&lt;/p&gt;&#xA;" OwnerUserId="42421" LastActivityDate="2020-11-20T14:01:46.860" Title="Which models can I use for supervised learning with images?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;unsupervised-learning&gt;&lt;autoencoders&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24745" PostTypeId="5" CreationDate="2020-11-20T16:12:56.167" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T16:12:56.167" LastActivityDate="2020-11-20T16:12:56.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24746" PostTypeId="4" CreationDate="2020-11-20T16:12:56.167" Score="0" Body="For questions about instance segmentation, which is the task of detecting and delineating each distinct object of interest appearing in an image (even the ones belong to the same class). Instance segmentation is different from semantic segmentation, where objects that belong to the same class are not differentiated. Collectively, instance and semantic segmentation fall into the category of image segmentation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T16:18:55.023" LastActivityDate="2020-11-20T16:18:55.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24747" PostTypeId="5" CreationDate="2020-11-20T16:14:52.897" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T16:14:52.897" LastActivityDate="2020-11-20T16:14:52.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24748" PostTypeId="4" CreationDate="2020-11-20T16:14:52.897" Score="0" Body="For questions about pre-text tasks (also known as auxiliary tasks) in the context of self-supervised learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T16:14:52.897" LastActivityDate="2020-11-20T16:14:52.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24749" PostTypeId="1" AcceptedAnswerId="27622" CreationDate="2020-11-20T16:16:31.027" Score="2" ViewCount="47" Body="&lt;p&gt;I have a scenario where, in an ideal situation, the greedy approach is the best, but when non-idealities are introduced which can be learned, DQN starts doing better. So, after checking what DQN achieved, I tried C51 using the standard implementation from &lt;code&gt;tf.agents&lt;/code&gt; (&lt;a href=&quot;https://www.tensorflow.org/agents/tutorials/9_c51_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;). A very nice description is given &lt;a href=&quot;https://flyyufelix.github.io/2017/10/24/distributional-bellman.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. But, as shown in the image, C51 does extremely bad.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1NyWQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1NyWQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, C51 stays at the same level throughout. When learning, the loss right from the first iteration is around 10e-3 and goes on to 10e-5, which definitely impacts the change in the weights. But I am not sure how this can be solved.&lt;/p&gt;&#xA;&lt;p&gt;The scenario is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;1 episode consists of 10 steps and the episode only ends after the 10th step, the episode never ends earlier.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;states at each step are integer values and can take values between 0 and 1. In the image, states are of shape 20*1.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;actions have the shape 20*1&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;learning rate = 10e-3&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;exploration factor &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; starts out at 0.2 and decays up to 0.01&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;C51 has 3 additional parameters, which help it to learn the distribution of q-values&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;num_atoms = 51 # u/param {type:&amp;quot;integer&amp;quot;} &#xA;min_q_value = -20 # u/param {type:&amp;quot;integer&amp;quot;} &#xA;max_q_value = 20 # u/param {type:&amp;quot;integer&amp;quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;num_atoms&lt;/code&gt; is the number of support that the learned distribution will have, and &lt;code&gt;min_q_value&lt;/code&gt; and &lt;code&gt;max_q_value&lt;/code&gt; are the endpoints of the q-value distribution. I set them as 51 (the first paper and other implementations keep it as 51 and hence the name 51), and the min and max are set as the min and max possible rewards.&lt;/p&gt;&#xA;&lt;p&gt;So, if anyone could help me with fine-tuning the parameters for C51 to work, I would be very grateful.&lt;/p&gt;&#xA;" OwnerUserId="41984" LastEditorUserId="2444" LastEditDate="2020-11-21T22:10:29.870" LastActivityDate="2021-05-03T06:21:55.343" Title="How should I change the hyper-parameters of the C51 algorithm, in order to obtain higher reward?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;hyperparameter-optimization&gt;&lt;performance&gt;&lt;c51&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24750" PostTypeId="2" ParentId="24736" CreationDate="2020-11-20T16:32:01.917" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is it possible to use SSL to pre-train e.g. a faster R-CNN on a pretext task (for example, rotation), then use this pre-trained model for instance segmentation with the aim to get better accuracy?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, it's possible and this has already been done. I don't know the details (because I have not yet read those papers), but I will provide you with some links to some potentially useful papers (based on their titles and abstracts) and associated code.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/%7Epathak/papers/cvprw18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning Instance Segmentation by Interaction&lt;/a&gt; (&lt;a href=&quot;https://github.com/pathak22/seg-by-interaction&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, 2018)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/2005.09484.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction&lt;/a&gt; (&lt;a href=&quot;https://github.com/aeitel/self_deepmask&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, 2019)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F978-3-030-59728-3_27&quot; rel=&quot;nofollow noreferrer&quot;&gt;White Matter Tract Segmentation with Self-supervised Learning&lt;/a&gt; (&lt;a href=&quot;https://github.com/MIC-DKFZ/TractSeg&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, 2020)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F978-3-030-59710-8_58&quot; rel=&quot;nofollow noreferrer&quot;&gt;Self-supervised Depth Estimation to Regularise Semantic Segmentation in Knee Arthroscopy&lt;/a&gt; (&lt;a href=&quot;https://github.com/ThomasLiu1021/geo-sem&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, 2020)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F978-3-030-59716-0_12&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simulation of Brain Resection for Cavity Segmentation Using Self-supervised and Semi-supervised Learning&lt;/a&gt; (&lt;a href=&quot;https://github.com/fepegar/resector&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;, 2020)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You can probably find more relevant papers &lt;a href=&quot;https://github.com/JunMa11/MICCAI-OpenSourcePapers&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where I also found some of the just cited papers.&lt;/p&gt;&#xA;&lt;p&gt;The pre-text tasks designed in these papers could be useful in your case, but it may also turn out that you need to develop other pre-text tasks or combine multiple of them.&lt;/p&gt;&#xA;&lt;p&gt;Maybe you can start from some pre-trained faster R-CNN or some appropriate model for instance segmentation (that you can find on the web, for example, &lt;a href=&quot;https://github.com/matterport/Mask_RCNN&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), which has been pre-trained on some imagery data similar to yours (either with SSL or by other means), then try to fine-tune this model with your labeled dataset for instance segmentation, and see if you get better results than just training a faster R-CNN from scratch. Eventually, if this pre-trained model does not lead to higher performance, you could pre-train it yourself with some SSL technique that you can come up with or one that is described in the literature. Of course, you should probably use a pre-trained model that has been pre-trained with data that is relevant for your &lt;em&gt;downstream task&lt;/em&gt; (i.e. the instance segmentation task). You didn't describe the details of your unlabelled and labeled data, so I cannot be more specific (and I wouldn't currently be able to, in any case, because I didn't fully read those papers, and my experience with SSL techniques is mostly theoretical).&lt;/p&gt;&#xA;&lt;p&gt;For more info about SSL, take a look at &lt;a href=&quot;https://ai.stackexchange.com/a/10624/2444&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/a/22860/2444&quot;&gt;this&lt;/a&gt; answers.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T16:56:45.937" LastActivityDate="2020-11-20T16:56:45.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24751" PostTypeId="2" ParentId="22184" CreationDate="2020-11-20T17:15:04.077" Score="0" Body="&lt;p&gt;Most (if not all) self-supervised learning techniques for (visual or textual) representation learning use pre-text tasks, and many pre-text tasks have been proposed in recent years.&lt;/p&gt;&#xA;&lt;p&gt;However, as I say in &lt;a href=&quot;https://ai.stackexchange.com/a/10624/2444&quot;&gt;my other answer&lt;/a&gt; (which you cite), the term SSL has &lt;strong&gt;also&lt;/strong&gt; been used (at least, in robotics: for example, see &lt;a href=&quot;https://arxiv.org/pdf/1809.07207.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which I am very familiar with) to refer to techniques that automatically (although approximately) label the unlabelled dataset for your downstream task (i.e. image recognition), i.e. they automatically create a labeled dataset of pairs &lt;span class=&quot;math-container&quot;&gt;$(x_i, \hat{y}_i)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is an image that contains an object and &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt; is the automatically (and possibly approximately) generated label (such as &amp;quot;dog&amp;quot;). This latter use of the term SSL is closer to some &lt;a href=&quot;https://watermark.silverchair.com/nwx106.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;weakly supervised learning (WSL)&lt;/a&gt; techniques. Actually, it can be considered a WSL technique.&lt;/p&gt;&#xA;&lt;p&gt;Now, in &lt;a href=&quot;https://arxiv.org/pdf/1809.07207.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this specific paper&lt;/a&gt;, they actually solve some kind of pre-text task, i.e. they exploit the relations between two different sensors to produce the labels.&lt;/p&gt;&#xA;&lt;p&gt;To answer your question more directly: in all SSL papers that I have come across, some kind of pre-text task is always solved, i.e., in some way, you need to automatically generate the supervisory signal, and that task that we solve with the automatically generated learning signal (with the purpose of learning representations or generating a labeled dataset) can be considered the pre-text task (which may coincide with the downstream task, for example, in the case you're training an auto-encoder with an unlabelled dataset for the purposes of image denoising).&lt;/p&gt;&#xA;&lt;p&gt;In any case, I wouldn't bother too much about it. Just keep your context in mind when reading your paper. If you're really worried about it, then you should probably read almost all SSL-related papers, but, in that case, by the end of that, you will be an expert on the topic and you will not need our help (or my help).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T20:07:58.807" LastActivityDate="2020-11-21T20:07:58.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24752" PostTypeId="2" ParentId="24671" CreationDate="2020-11-20T17:43:51.937" Score="1" Body="&lt;p&gt;Graph neural networks, of which GCNs are a specific type, are able to handle arbitrary graphs as input. GNNs operate first over &amp;quot;neighborhoods&amp;quot; of nodes to compute individual node representations and then optionally apply a pooling function to reduce these to a single graph-level representation that can be used in classification. This means that GNNs work &amp;quot;locally&amp;quot; and do not contain implicit assumptions about the graph topology.&lt;/p&gt;&#xA;&lt;p&gt;For example, the update equation for a given node's representation in a GCN can be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h_v^{t+1} = \sigma\left({\bf W}^{t+1} \sum_{u \in \mathcal{N}_v} {\bf L}_{uv}~ h_u^t \right) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h_v^l$&lt;/span&gt; is the representation of node &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; at update &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is an activation function, &lt;span class=&quot;math-container&quot;&gt;${\bf W^{t}}$&lt;/span&gt; is a weight matrix, &lt;span class=&quot;math-container&quot;&gt;${\bf L_{uv}}$&lt;/span&gt; is the value of the graph Laplacian (which is a matrix) at nodes &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, and finally &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}_v$&lt;/span&gt; is the neighborhood of &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;. Looking at this expression it becomes clear that the value of the summation is always of the same dimension as &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, no matter how you define the neighborhood &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}_v$&lt;/span&gt;. So you are correct that as long as the node representation size is static, the network can take arbitrary graphs as input.&lt;/p&gt;&#xA;&lt;p&gt;I highly recommend &lt;a href=&quot;https://arxiv.org/abs/1912.12693&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial paper on GNNs&lt;/a&gt;, which I used when first learning about them. Section 2.3 specifically answers your question and discusses how things like cycles and non-positional graphs are handled.&lt;/p&gt;&#xA;" OwnerUserId="37972" LastActivityDate="2020-11-20T17:43:51.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24753" PostTypeId="1" AcceptedAnswerId="24756" CreationDate="2020-11-20T17:47:43.700" Score="2" ViewCount="36" Body="&lt;p&gt;Suppose, I have several sequences that include a series of text (the length of sequence can be varied). Also, I have some related reward value. however, the value is not continuous like the text. It has many missing values. Here is an example of the dataset.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sequence 1        Sequence 2 .............. Sequence n&#xA;------------      ----------                -------------&#xA;Action  Reward    Action  Reward            Action  Reward&#xA;  A                 C                          D&#xA;  B       5         A                          B      6&#xA;  C                 A       7                  A       &#xA;  C       6         B       10                 D           &#xA;  A                 C                          A           &#xA;  B       2         A                          B           &#xA;  ..                ...                        ...&#xA; ...               .....                      .....&#xA;  D       5         C      4                   D          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I want to predict the next action based on the reward value. The idea is I want to predict the actions that leads to more rewards. Previously, I used only action data to predict the next action using LSTM and GRU. However, how could I use this reward value in this prediction? I was if thinking Reinforcement learning (MDP) could solve the problem. However, as the rewards are discrete, I am not sure if RL could do that. Also, is it possible to solve this problem with Inverse RL? I have some knowledge of deep learning. However, I am new to reinforcement learning. If anyone gives me some suggestion or provide me useful papers link regarding this problem, it could help me a lot.&lt;/p&gt;&#xA;" OwnerUserId="18795" LastActivityDate="2020-11-20T19:17:42.250" Title="Predict next event based on previous events and discrete reward values" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;prediction&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24754" PostTypeId="5" CreationDate="2020-11-20T18:37:46.743" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T18:37:46.743" LastActivityDate="2020-11-20T18:37:46.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24755" PostTypeId="4" CreationDate="2020-11-20T18:37:46.743" Score="0" Body="For questions about the Hindsight Experience Replay (HER), proposed in the paper &quot;Hindsight Experience Replay&quot; (2017) by Marcin Andrychowicz et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T18:37:46.743" LastActivityDate="2020-11-20T18:37:46.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24756" PostTypeId="2" ParentId="24753" CreationDate="2020-11-20T19:17:42.250" Score="1" Body="&lt;p&gt;Your problem does look like it could be a good match to reinforcement learning, or at least the related idea of contextual bandits. Whether or not it would be a good match to the full reinforcement learning algorithm depends on whether any of the data you are processing could be considered part of an environment &lt;em&gt;state&lt;/em&gt;, and whether or not that &lt;em&gt;state&lt;/em&gt; evolves based on rules that an agent could learn to take advantage of.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Previously, I used only action data to predict the next action using LSTM and GRU. However, how could I use this reward value in this prediction?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are a few different ways to do this using reinforcement learning theory. The simplest is to build a regression predictor that approximates a sum of future rewards (also known as the &lt;em&gt;return&lt;/em&gt; or &lt;em&gt;utility&lt;/em&gt;) depending on a proposed action from the current state. Then you could use the &lt;em&gt;value function approximator&lt;/em&gt; (formal name for the predictor you just built) to predict results from each possible action and pick the maximising one. It is possible to learn such a value function from a historical dataset using methods such as Q learning.&lt;/p&gt;&#xA;&lt;p&gt;The subject is too complex to teach from scratch in a single answer here. A good learning resource is &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction by Sutton &amp;amp; Barto&lt;/a&gt;, which the authors have made available for free.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;However, as the rewards are discrete, I am not sure if RL could do that.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes it can. Reinforcement learning just requires that rewards in each situation follow a consistent &lt;em&gt;distribution&lt;/em&gt; of values. Always returning discrete values is not a problem, neither is always returning the same value in the same situation. Randomness in the reward value - such as sometimes returning a discrete value and other times not in the same situation - is also OK. You can treat missing values as zero, since you are concerned only with the sum of received rewards, using zero when no value is available has no effect on what will be considered the optimal solution.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Also, is it possible to solve this problem with Inverse RL?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Probably not. Inverse RL is concerned with figuring out the parameters that an existing agent is working with by observing it. For instance you could use it to observe a creature's behaviour and figure out which rewards were more valuable to it. In your case you have the reward values, so you don't need to figure them out.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Caveat&lt;/strong&gt;: You need to figure out what constitutes &lt;em&gt;state&lt;/em&gt; in your environment. If there is some state that can be used for predictions, but the agent's behaviour never changes the state, then you may want to spend some time modelling your problem as a &lt;a href=&quot;https://www.cs.ubc.ca/labs/lci/mlrg/slides/2019_summer_5_contextual_bandits.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;contextual bandit&lt;/a&gt; instead. Bandit algorithms are introduced in the same book, Reinforcement Learning: An Introduction, but only as much is needed to teach about the full RL problem - bandit solvers can get far more sophisticated than the book considers.&lt;/p&gt;&#xA;&lt;p&gt;Note that if the history of agent actions impacts the reward (e.g. it is a matter of timing when to take the right action), then that history is part of the state, and you likely do have a full reinforcement learning problem to solve.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-20T19:17:42.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24757" PostTypeId="1" CreationDate="2020-11-20T19:41:21.797" Score="0" ViewCount="34" Body="&lt;p&gt;I'm building an agent to solve the Taxi environment. I've seen this problem solved with Q-Learning algorithms but my DQN consistently fails to learn anything. The environment has a discrete observation space, I one-hot encode the state before feeding it to the DQN. I also went ahead to implement Hindsight Experience Replay to help the learning process but the DQN still doesn't learn anything. What can I do to fix this?&lt;/p&gt;&#xA;&lt;p&gt;I've heard that DQN doesn't excel at environments that require planning to succeed, if that's the case, which algorithms would work well for this environment?&lt;/p&gt;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&lt;p&gt;When I posted this question, my DQN was learning from only 2 step transitions, since this environment can go on for several timesteps without any positive reward, I updated the agent to use transitions of 200 steps. Since I'm using Hindsight Experience Replay, my agent is sure to receive rewards within 200 timesteps even if it didn't meet the goal. I tried this and my agent still hasn't improved, it continually performs worse than the random agent baseline. I checked the contents of the buffer, I observed transitions that do lead to several rewards because their goals have been modified during HER and yet the DQN agent doesn't learn anything.&lt;/p&gt;&#xA;&lt;p&gt;Also, I'm using TensorFlow's tf_agents for my implementation. Here's a &lt;a href=&quot;https://colab.research.google.com/drive/1pdWrDe7ZwzZ2bOWTR6KlEClxMq5X-Bhl?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to the code&lt;/a&gt;. I repurposed &lt;a href=&quot;https://www.tensorflow.org/agents/tutorials/1_dqn_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I hope this helps&lt;/p&gt;&#xA;" OwnerUserId="42431" LastEditorUserId="42431" LastEditDate="2020-11-22T19:59:20.153" LastActivityDate="2020-11-22T19:59:20.153" Title="DQN fails to learn useful policy for the Taxi environment (Dietterich 200)" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24758" PostTypeId="1" CreationDate="2020-11-20T19:55:43.223" Score="1" ViewCount="44" Body="&lt;p&gt;One of the reviewers of &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the HER paper&lt;/a&gt; (which was accepted as a NIPS conference paper) &lt;a href=&quot;https://proceedings.neurips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Reviews.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;wrote&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Overall, I'd say that it's not a huge/deep idea, but a very nice addition to the learning toolbox. When it is applicable, it seems like an idea that should be used. I worry that it is rarely applicable, though, without a tremendous amount of task insight. Consider the &amp;quot;kick the ball into the goal&amp;quot; example. If we want to speed up the learning of the task, we would likely need to use the idea to help guide the robot to the ball, to guide the robot's foot to make contact with the ball, and to guide the robot's aim to send the ball into the goal. At this level, it doesn't feel particularly different from &lt;strong&gt;(potential-based) reward shaping&lt;/strong&gt;, which has been used in precisely this way.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Potential-based reward shaping (PBRS) consists in augmenting the original/natural reward function &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$F(s, s') = \gamma \phi(s') - \phi(s)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the current state, &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; the next state after &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; the discount factor and &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is the so-called &lt;em&gt;potential function&lt;/em&gt; (which can be anything: say what!?), i.e. the new shaped reward function is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;r'(s, a, s') = r(s, a, s') + F(s, s')&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;How exactly is HER (which essentially turns every successful or unsuccessful trajectory into a successful trajectory, by making one of the states in that trajectory a fake goal state, then replays these unsuccessful trajectories) related to PBRS? Sure, both attempt to deal with sparse rewards (and I see how), and, intuitively, HER is a reward shaping technique. However, now, I am trying to understand if there is a &lt;strong&gt;formal&lt;/strong&gt; attempt to &lt;strong&gt;mathematically&lt;/strong&gt; connect these two approaches. More precisely, how would we formalize HER in terms of PBRS? Can we do that? Can we come up with a potential function that corresponds to what HER is doing? Is there any research on this?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-20T20:34:36.147" LastActivityDate="2020-11-20T20:34:36.147" Title="How exactly is hindsight experience replay related to potential-based reward shaping?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;reference-request&gt;&lt;hindsight-experience-replay&gt;&lt;potential-reward-shaping&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24759" PostTypeId="2" ParentId="24740" CreationDate="2020-11-20T21:04:19.920" Score="1" Body="&lt;p&gt;I presume when you say input you may be referring to the target values (the things you are trying to predict). If not, then some parts of your question might not make sense, like your proposal to apply a scaling.&lt;/p&gt;&#xA;&lt;p&gt;In any case I would consider what the target distribution is before using a sigmoid and applying a scaling. The thing about a sigmoid is that the model has to be very confident in order to predict a value close to the boundaries. So in your case, the model would have to be really confident about something for it to output a -5, and it would have to be really confident of the opposite to output a +20. I would be surprised if this is what you are really looking for in this case.&lt;/p&gt;&#xA;&lt;p&gt;Let's say your target distribution is uniform. Then your best bet is to use a linear activation, and maybe use max/min to clip it. Then it would behave a lot like a ReLU.&lt;/p&gt;&#xA;&lt;p&gt;If I haven't quite nailed it with this answer, can you give more info on what you are predicting?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-11-20T21:04:19.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24760" PostTypeId="1" AcceptedAnswerId="24761" CreationDate="2020-11-21T05:04:04.093" Score="0" ViewCount="77" Body="&lt;p&gt;I am trying to understand what is meant by following equations in the &lt;a href=&quot;https://arxiv.org/abs/1803.04189&quot; rel=&quot;nofollow noreferrer&quot;&gt;Noise2Noise paper&lt;/a&gt; by Nvidia.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gSKGh.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gSKGh.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What is meant by the equation in this image? What is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_y\{y\}$&lt;/span&gt;? And how should I try to visualize these equations?&lt;/p&gt;&#xA;" OwnerUserId="27601" LastEditorUserId="2444" LastEditDate="2020-11-23T10:56:47.900" LastActivityDate="2020-11-23T10:56:47.900" Title="What is the meaning of these equations in Noise2Noise paper?" Tags="&lt;papers&gt;&lt;supervised-learning&gt;&lt;notation&gt;&lt;expectation&gt;&lt;mean-squared-error&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24761" PostTypeId="2" ParentId="24760" CreationDate="2020-11-21T11:38:55.193" Score="1" Body="&lt;p&gt;The equation you are referring to is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Mean Squared Error&lt;/em&gt;&lt;/a&gt; (or &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; loss) and it is used for &lt;a href=&quot;https://en.wikipedia.org/wiki/Regression_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;regression&lt;/em&gt;&lt;/a&gt; tasks, where the goal is to predict a real value given some input.&lt;/p&gt;&#xA;&lt;p&gt;In your case, the inputs are measurements of temperature &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, either at a certain point in time or point in space or both or none, this is not clear from the image. Now, the goal would be to predict the temperature at a new point in space, time, or both, where we don't have access to a measurement. That is we would like to find a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; (e.g. a simple linear function) which we can use for prediction. But how can we measure which function is &amp;quot;best&amp;quot;? We introduce a &lt;em&gt;loss function&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$L(f,y)$&lt;/span&gt;, another function which tells us how good our proposed function is.&lt;/p&gt;&#xA;&lt;p&gt;Visually it looks like this (&lt;a href=&quot;https://medium.com/@jorgesleonel/linear-regression-307937441a8b&quot; rel=&quot;nofollow noreferrer&quot;&gt;image source&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/im1oe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/im1oe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Red crossed are measurements, the black line is our function we use for prediction and the green dotted lines are the errors (the distance from our prediction to the real measurement). In this example salary depends on experience.&lt;/p&gt;&#xA;&lt;p&gt;Now, the paper introduces the constant mean of all measurements as &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$z = \mathbb{E}_y\{y\} = \frac{1}{N}\sum_i^N y_i$&lt;/span&gt;, as our function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, which is known to be the minimizer for the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; loss in the case where there is no dependence on other variables (e.g. time or space).&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="2444" LastEditDate="2020-11-21T14:18:27.513" LastActivityDate="2020-11-21T14:18:27.513" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24762" PostTypeId="5" CreationDate="2020-11-21T12:49:39.913" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T12:49:39.913" LastActivityDate="2020-11-21T12:49:39.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24763" PostTypeId="4" CreationDate="2020-11-21T12:49:39.913" Score="0" Body="For questions about the Random Network Distillation (RND) as used in the paper &quot;Exploration by Random Network Distillation&quot; (2018) by Yuri Burda et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T12:49:39.913" LastActivityDate="2020-11-21T12:49:39.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24764" PostTypeId="5" CreationDate="2020-11-21T12:54:44.040" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T12:54:44.040" LastActivityDate="2020-11-21T12:54:44.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24765" PostTypeId="4" CreationDate="2020-11-21T12:54:44.040" Score="0" Body="For theoretical questions that involve the PyBullet library/API, which is a Python module for physics simulation, robotics, and deep reinforcement learning based on the Bullet Physics SDK. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T12:54:44.040" LastActivityDate="2020-11-21T12:54:44.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24766" PostTypeId="1" AcceptedAnswerId="24773" CreationDate="2020-11-21T13:08:17.997" Score="2" ViewCount="88" Body="&lt;p&gt;I am currently studying the textbook &lt;em&gt;Neural Networks and Deep Learning&lt;/em&gt; by Charu C. Aggarwal. Chapter &lt;strong&gt;1.2.1.2 Relationship with Support Vector Machines&lt;/strong&gt; says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The perceptron criterion is a shifted version of the hinge-loss used in support vector machines (see Chapter 2). The hinge loss looks even more similar to the zero-one loss criterion of Equation 1.7, and is defined as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_i^{svm} = \max\{ 1 - y_i(\overline{W} \cdot \overline{X}_i), 0 \} \tag{1.9}$$&lt;/span&gt;&#xA;Note that the perceptron does not keep the constant term of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; on the right-hand side of Equation 1.7, whereas the hinge loss keeps this constant within the maximization function. This change does not affect the algebraic expression for the gradient, but it does change which points are lossless and should not cause an update. The relationship between the&#xA;perceptron criterion and the hinge loss is shown in Figure 1.6. This similarity becomes&#xA;particularly evident when the perceptron updates of Equation 1.6 are rewritten as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\overline{W} \Leftarrow \overline{W} + \alpha \sum_{(\overline{X}, y) \in S^+} y \overline{X} \tag{1.10}$$&lt;/span&gt;&#xA;Here, &lt;span class=&quot;math-container&quot;&gt;$S^+$&lt;/span&gt; is defined as the set of all misclassified training points &lt;span class=&quot;math-container&quot;&gt;$\overline{X} \in S$&lt;/span&gt; that satisfy the condition &lt;span class=&quot;math-container&quot;&gt;$y(\overline{W} \cdot \overline{X}) &amp;lt; 0$&lt;/span&gt;. This update seems to look somewhat different from the perceptron, because the perceptron uses the error &lt;span class=&quot;math-container&quot;&gt;$E(\overline{X})$&lt;/span&gt; for the update, which is replaced with &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the update above. A key point is that the (integer) error value &lt;span class=&quot;math-container&quot;&gt;$E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$&lt;/span&gt; can never be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for misclassified points in &lt;span class=&quot;math-container&quot;&gt;$S^+$&lt;/span&gt;. Therefore, we have &lt;span class=&quot;math-container&quot;&gt;$E(\overline{X}) = 2y$&lt;/span&gt; &lt;em&gt;for misclassified points&lt;/em&gt;, and &lt;span class=&quot;math-container&quot;&gt;$E(X)$&lt;/span&gt; can be replaced with &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the updates after absorbing the factor of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; within the learning rate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Equation 1.7 is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L_i^{(0/1)} = \dfrac{1}{2} (y_i - \text{sign}\{ \overline{W} \cdot \overline{X_i} \})^2 = 1 - y_i \cdot \text{sign} \{ \overline{W} \cdot \overline{X_i} \} \tag{1.7}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And figure 1.6 is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nmMyU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nmMyU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is said that we are dealing with the case of binary classification, where &lt;span class=&quot;math-container&quot;&gt;$y \in \{ -1, +1 \}$&lt;/span&gt;. But the author claims that &lt;span class=&quot;math-container&quot;&gt;$E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$&lt;/span&gt;, which doesn't include the case of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. So shouldn't the &lt;span class=&quot;math-container&quot;&gt;$\{ -2, +2 \}$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$&lt;/span&gt; be &lt;span class=&quot;math-container&quot;&gt;$\{ -2, 0, +2 \}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="16521" LastEditDate="2020-11-21T15:19:05.547" LastActivityDate="2021-02-07T16:04:54.170" Title="Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?" Tags="&lt;loss-functions&gt;&lt;support-vector-machine&gt;&lt;perceptron&gt;&lt;binary-classification&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24767" PostTypeId="1" CreationDate="2020-11-21T13:19:13.200" Score="5" ViewCount="297" Body="&lt;p&gt;I am currently studying the textbook &lt;em&gt;Neural Networks and Deep Learning&lt;/em&gt; by Charu C. Aggarwal. Chapter &lt;strong&gt;1.2.1.2 Relationship with Support Vector Machines&lt;/strong&gt; says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The perceptron criterion is a shifted version of the hinge-loss used in support vector machines (see Chapter 2). The hinge loss looks even more similar to the zero-one loss criterion of Equation 1.7, and is defined as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L_i^{svm} = \max\{ 1 - y_i(\overline{W} \cdot \overline{X}_i), 0 \} \tag{1.9}$$&lt;/span&gt;&#xA;Note that the perceptron does not keep the constant term of &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; on the right-hand side of Equation 1.7, whereas the hinge loss keeps this constant within the maximization function. This change does not affect the algebraic expression for the gradient, but it does change which points are lossless and should not cause an update. The relationship between the&#xA;perceptron criterion and the hinge loss is shown in Figure 1.6. This similarity becomes particularly evident when the perceptron updates of Equation 1.6 are rewritten as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\overline{W} \Leftarrow \overline{W} + \alpha \sum_{(\overline{X}, y) \in S^+} y \overline{X} \tag{1.10}$$&lt;/span&gt;&#xA;Here, &lt;span class=&quot;math-container&quot;&gt;$S^+$&lt;/span&gt; is defined as the set of all misclassified training points &lt;span class=&quot;math-container&quot;&gt;$\overline{X} \in S$&lt;/span&gt; that satisfy the condition &lt;span class=&quot;math-container&quot;&gt;$y(\overline{W} \cdot \overline{X}) &amp;lt; 0$&lt;/span&gt;. This update seems to look somewhat different from the perceptron, because the perceptron uses the error &lt;span class=&quot;math-container&quot;&gt;$E(\overline{X})$&lt;/span&gt; for the update, which is replaced with &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the update above. A key point is that the (integer) error value &lt;span class=&quot;math-container&quot;&gt;$E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$&lt;/span&gt; can never be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for misclassified points in &lt;span class=&quot;math-container&quot;&gt;$S^+$&lt;/span&gt;. Therefore, we have &lt;span class=&quot;math-container&quot;&gt;$E(\overline{X}) = 2y$&lt;/span&gt; &lt;em&gt;for misclassified points&lt;/em&gt;, and &lt;span class=&quot;math-container&quot;&gt;$E(X)$&lt;/span&gt; can be replaced with &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the updates after absorbing the factor of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; within the learning rate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Equation 1.6 is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\overline{W} \Leftarrow \overline{W} + \alpha \sum_{\overline{X} \in S} E(\overline{X})\overline{X}, \tag{1.6}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a randomly chosen subset of training points, &lt;span class=&quot;math-container&quot;&gt;$\overline{X} = [x_1, \dots, x_d]$&lt;/span&gt; is a data instance (vector of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; feature variables), &lt;span class=&quot;math-container&quot;&gt;$\overline{W} = [w_1, \dots, w_d]$&lt;/span&gt; are the weights, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate, and &lt;span class=&quot;math-container&quot;&gt;$E(\overline{X}) = (y - \hat{y})$&lt;/span&gt; is an error value, where &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = \text{sign}\{ \overline{W} \cdot \overline{X} \}$&lt;/span&gt; is the prediction and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the observed value of the binary class variable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Equation 1.7 is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L_i^{(0/1)} = \dfrac{1}{2} (y_i - \text{sign}\{ \overline{W} \cdot \overline{X_i} \})^2 = 1 - y_i \cdot \text{sign} \{ \overline{W} \cdot \overline{X_i} \} \tag{1.7}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And figure 1.6 is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nmMyU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nmMyU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Figure 1.6 looks unclear to me. What is figure 1.6 showing, and how is it relevant to the point that the author is trying to make?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="16521" LastEditDate="2021-04-23T07:53:21.597" LastActivityDate="2021-05-09T23:49:47.697" Title="How should we interpret this figure that relates the perceptron criterion and the hinge loss?" Tags="&lt;loss-functions&gt;&lt;support-vector-machine&gt;&lt;perceptron&gt;&lt;binary-classification&gt;&lt;hinge-loss&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24768" PostTypeId="1" CreationDate="2020-11-21T17:06:02.007" Score="1" ViewCount="30" Body="&lt;p&gt;I'm trying to read &lt;a href=&quot;https://arxiv.org/pdf/1609.08144.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B9wi6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B9wi6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a picture of the architecture. I just want to understand the encoder for now (the left block). The pink and green blocks are LSTM cells, I understand.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are these LSTM cells the same kind that are described &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_short-term_memory#/media/File:LSTM_cell.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;on Wikipedia&lt;/a&gt;? Because those have two outputs (the &amp;quot;mask&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$c_t$&lt;/span&gt; and the state &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt;) and three inputs. Which arrow is which in the Google diagram?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Where are the recurrent connections going? Does the hidden state of each LSTM cell go into the cell on the right at the next timestep? The cell above? Into itself? All three? What about the mask vector?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What goes where? That is, where do the inputs enter? The text &lt;span class=&quot;math-container&quot;&gt;$x_3, x_2, ...$&lt;/span&gt; at the bottom of the encoder block is not clear to me. So the different tokens of the input go into different cells? What if there are more tokens in the sentence than cells?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1931" LastEditorUserId="2444" LastEditDate="2020-11-23T13:14:40.820" LastActivityDate="2020-11-23T13:14:40.820" Title="How does Google's 2016 GNMT architecture work?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;papers&gt;&lt;google&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24769" PostTypeId="1" CreationDate="2020-11-21T17:35:34.320" Score="1" ViewCount="54" Body="&lt;p&gt;Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average length of a message without losing information as it is transmitted. (&lt;a href=&quot;https://towardsdatascience.com/entropy-the-pillar-of-both-thermodynamics-and-information-theory-138d6e4872fa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/entropy-the-pillar-of-both-thermodynamics-and-information-theory-138d6e4872fa&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I don't know whether Neural Network actually deals with information flow or not. This information flow is taken from the idea of entropy. Since I haven't found any paper or ideas based on the law of energy for neural networks. The law of energy states that energy can neither be created nor destroyed. If it is creating information (energy) (e.g. in the case of a generative model), then some information may be lost while updating weights. How is Neural Network ensuring this energy conservation?&lt;/p&gt;&#xA;" OwnerUserId="19780" LastEditorUserId="2444" LastEditDate="2020-11-22T19:44:42.470" LastActivityDate="2020-11-22T19:44:42.470" Title="How does NN follows law of energy conservation?" Tags="&lt;neural-networks&gt;&lt;generative-model&gt;&lt;information-theory&gt;&lt;minimum-description-length&gt;&lt;entropy&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24770" PostTypeId="5" CreationDate="2020-11-21T17:51:40.377" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T17:51:40.377" LastActivityDate="2020-11-21T17:51:40.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24771" PostTypeId="4" CreationDate="2020-11-21T17:51:40.377" Score="0" Body="For questions about the hinge loss, which is typically used for &quot;maximum-margin&quot; classification, most notably for support vector machines (SVMs)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T17:51:40.377" LastActivityDate="2020-11-21T17:51:40.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24773" PostTypeId="2" ParentId="24766" CreationDate="2020-11-21T19:39:37.663" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in &lt;span class=&quot;math-container&quot;&gt;$ S^+$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;And &lt;span class=&quot;math-container&quot;&gt;$S+$&lt;/span&gt; is defined as the set of all misclassified training points &lt;span class=&quot;math-container&quot;&gt;$X \in S$&lt;/span&gt; that satisfy the condition &lt;span class=&quot;math-container&quot;&gt;$y(\overline{W} \cdot \overline{X})&amp;lt;0 $&lt;/span&gt; which means that &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ (\overline{W} \cdot \overline{X}) $&lt;/span&gt; will have different signs i.e if &lt;span class=&quot;math-container&quot;&gt;$y &amp;lt; 0$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$(\overline{W} \cdot \overline{X}) &amp;gt; 0$&lt;/span&gt; and vice versa.&lt;/li&gt;&#xA;&lt;li&gt;Finally the condition on &lt;span class=&quot;math-container&quot;&gt;$S^+$&lt;/span&gt; constraints the the &lt;span class=&quot;math-container&quot;&gt;$ E(X)$&lt;/span&gt; to these values when &lt;span class=&quot;math-container&quot;&gt;$ y &amp;lt; 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(\overline{W} \cdot \overline{X}) &amp;gt; 0$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ E(X) = (-1 - \text{sign}(+ve)) = -2 $$&lt;/span&gt;&#xA;or when &lt;span class=&quot;math-container&quot;&gt;$ y &amp;gt; 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(\overline{W} \cdot \overline{X}) &amp;lt; 0$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ E(X) = (1 - \text{sign}(-ve)) = 2 $$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22540" LastEditorUserId="16521" LastEditDate="2021-02-07T16:04:54.170" LastActivityDate="2021-02-07T16:04:54.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24774" PostTypeId="5" CreationDate="2020-11-21T20:59:28.860" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T20:59:28.860" LastActivityDate="2020-11-21T20:59:28.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24775" PostTypeId="4" CreationDate="2020-11-21T20:59:28.860" Score="0" Body="For questions about the C51 algorithm. See the paper &quot;A Distributional Perspective on Reinforcement Learning&quot; (2017) by Marc G. Bellemare et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T20:59:28.860" LastActivityDate="2020-11-21T20:59:28.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24776" PostTypeId="2" ParentId="24732" CreationDate="2020-11-21T21:20:28.467" Score="2" Body="&lt;p&gt;I don't currently have much practical experience with DQN, but I can partially answer this question also based on my theoretical knowledge and other info that I found.&lt;/p&gt;&#xA;&lt;p&gt;DQN is &lt;em&gt;typically&lt;/em&gt; used for&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;discrete action spaces&lt;/strong&gt; (although there have been attempts to apply it to continuous action spaces, such as &lt;a href=&quot;http://proceedings.mlr.press/v48/gu16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;discrete&lt;/strong&gt; and &lt;strong&gt;continuous state spaces&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;problems where the &lt;strong&gt;optimal policy is &lt;a href=&quot;https://ai.stackexchange.com/q/12274/2444&quot;&gt;deterministic&lt;/a&gt;&lt;/strong&gt; (an example where the optimal policy is &lt;strong&gt;not&lt;/strong&gt; deterministic is &lt;a href=&quot;https://en.wikipedia.org/wiki/Rock_paper_scissors&quot; rel=&quot;nofollow noreferrer&quot;&gt;rock-paper-scissors&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;off-policy&lt;/strong&gt; learning (Q-learning is an off-policy algorithm, but the point is that, if you have a problem/application where data can only be or has been gathered by a policy that is unrelated to the policy you want to estimate, then DQN is suitable, though there are other off-policy algorithms, such as DDPG)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This guide&lt;/a&gt; also states that DQN is slower to train, but more &lt;em&gt;sample efficient&lt;/em&gt; than other approaches, due to the use of the experience replay buffer.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, if you have a small state and action space, it is probably a good idea to just use tabular Q-learning (i.e. no function approximation), given that &lt;a href=&quot;https://ai.stackexchange.com/q/11679/2444&quot;&gt;it is guaranteed to converge to the optimal value function&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://ai.stackexchange.com/q/12255/2444&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://ai.stackexchange.com/q/6196/2444&quot;&gt;this&lt;/a&gt; questions and &lt;a href=&quot;https://flyyufelix.github.io/2017/10/12/dqn-vs-pg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; (which compares DQN with policy gradients).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T20:03:13.030" LastActivityDate="2020-11-22T20:03:13.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24777" PostTypeId="1" CreationDate="2020-11-21T22:56:03.180" Score="0" ViewCount="38" Body="&lt;p&gt;I can find the number of parameters AIs such as GPT3 or OpenAI Five have, but I'm having a difficult time googling the number of parameters a self driving car has because Google searches keeps using the other definition of parameters when google searching. So how many parameters does a self-driving car typically have? I'm curious as to the computer requirements that will be needed to operate a fully functioning self driving car.&lt;/p&gt;&#xA;" OwnerUserId="40112" LastActivityDate="2020-11-21T22:56:03.180" Title="How many parameters does the AI of a typical self driving car have?" Tags="&lt;autonomous-vehicles&gt;&lt;weights&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24778" PostTypeId="5" CreationDate="2020-11-21T23:07:41.743" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T23:07:41.743" LastActivityDate="2020-11-21T23:07:41.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24779" PostTypeId="4" CreationDate="2020-11-21T23:07:41.743" Score="0" Body="For questions about embeddings (not necessarily just word embeddings, for which there is a specific tag) in the context of machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-21T23:07:41.743" LastActivityDate="2020-11-21T23:07:41.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24780" PostTypeId="5" CreationDate="2020-11-22T00:36:06.187" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T00:36:06.187" LastActivityDate="2020-11-22T00:36:06.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24781" PostTypeId="4" CreationDate="2020-11-22T00:36:06.187" Score="0" Body="For questions about the normalization of the reward in the context of reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T00:36:06.187" LastActivityDate="2020-11-22T00:36:06.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24782" PostTypeId="5" CreationDate="2020-11-22T00:36:34.853" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T00:36:34.853" LastActivityDate="2020-11-22T00:36:34.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24783" PostTypeId="4" CreationDate="2020-11-22T00:36:34.853" Score="0" Body="For questions about variance reduction techniques in the context of artificial intelligence, in particular, sampling or Monte Carlo methods. An example of a variance reduction technique is Flipout, which was proposed in the paper &quot;Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches&quot; (2018) by Yeming Wen et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T00:36:34.853" LastActivityDate="2020-11-22T00:36:34.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24784" PostTypeId="1" CreationDate="2020-11-22T09:28:45.667" Score="1" ViewCount="43" Body="&lt;p&gt;In deep learning, using more layers in a neural network adds the capacity to capture more features. In most RL papers, their experiments use a 2 layer neural network. &lt;a href=&quot;https://arxiv.org/pdf/1711.06782&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning to Reset&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1705.10528&quot; rel=&quot;nofollow noreferrer&quot;&gt;Constrained Policy Optimization&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1705.08551&quot; rel=&quot;nofollow noreferrer&quot;&gt;Model-based RL with stability guarantees&lt;/a&gt; just to name a few - these are papers I personally remember but there are definitely many others.&lt;/p&gt;&#xA;&lt;p&gt;I came across &lt;a href=&quot;https://ai.stackexchange.com/questions/11539/is-reinforcement-learning-using-shallow-neural-networks-still-deep-reinforcement&quot;&gt;this question&lt;/a&gt; whose answers generally agree that yes, RL using a shallow network is considered deep RL, but the reason for preference of shallow networks was not part of the question.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1810.05017.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaMimic&lt;/a&gt; (2018), the authors trained the largest neural net RL algorithm at the time (a residual net with 20 convolution layers) for one-shot imitation learning. The paper demonstrates that larger networks as policy approximators generalize better and represent many behaviours.&lt;/p&gt;&#xA;&lt;p&gt;So, why are shallow 2 layer networks so widely used?&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="2444" LastEditDate="2020-11-22T11:25:45.707" LastActivityDate="2020-11-23T08:28:48.297" Title="Why are shallow networks so prevalent in RL?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24786" PostTypeId="1" CreationDate="2020-11-22T14:06:53.637" Score="0" ViewCount="7" Body="&lt;p&gt;I want to recognize pieces of chessboard diagram (not real 3d pieces but just diagram).&lt;/p&gt;&#xA;&lt;p&gt;I split this task in some operation like rotation/cutting/segmentation.&lt;/p&gt;&#xA;&lt;p&gt;First of all I want to detect chessboard rotation (x/y/z) before segmentation. I will have unlimited number of rotated chessboards with labels from photos done with camera. Photos can be done with tolerance of 5 degree for x, y (about) and 20 for z.&lt;/p&gt;&#xA;&lt;p&gt;z is rotation of axis of lens to point of view and x, y is tilts of lens (like in normal camera).&lt;/p&gt;&#xA;&lt;p&gt;As result it is need to get x, y, z in -180, 180 range (practically can be only -20, 20 since nobody do wrong photos).&lt;/p&gt;&#xA;&lt;p&gt;For example like this (input is image and output is x, y, z (value with tolerance of 0.5 degree can be enough - so it can be class of rotation):&lt;/p&gt;&#xA;&lt;p&gt;No rotation&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wb3MG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wb3MG.png&quot; alt=&quot;No rotation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some 2d rotation (z axis only)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xo4y2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xo4y2.png&quot; alt=&quot;Some 2d rotation (z axis only)&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8rePe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8rePe.png&quot; alt=&quot;Some 2d rotation (z axis only)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some 3d rotation (x, y, z - camera photo)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t6FIi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t6FIi.png&quot; alt=&quot;Some 3d rotation&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j54Fo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j54Fo.png&quot; alt=&quot;Some 3d rotation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I assume that paper with chessboard diagram is not too much curved but can be.&lt;/p&gt;&#xA;" OwnerUserId="42460" LastActivityDate="2020-11-22T14:06:53.637" Title="How to find rotation x/y/z of chessboard diagram with what network architecture?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24787" PostTypeId="2" ParentId="22329" CreationDate="2020-11-22T14:52:08.670" Score="2" Body="&lt;p&gt;The purpose of the input network is to embed the input tuple into a state/task representation, that can then be fed into the RNN hidden state at each time step.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(o^a_t,m^a′_{t−1},u^a_{t−1},a)$&lt;/span&gt; (input) &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; input network (embedding) &lt;span class=&quot;math-container&quot;&gt;$\rightarrow$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$z_t$&lt;/span&gt; (task representation)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;According to to section 6.1 of the paper, the input is a tuple represented as &lt;span class=&quot;math-container&quot;&gt;$(o^a_t,m^a′_{t−1},u^a_{t−1},a)$&lt;/span&gt;. Each of these  terms is described in sec 3 as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$o_t$&lt;/span&gt; - The observation. The authors assume a &lt;a href=&quot;https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;POMDP&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Two types of actions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$u_t$&lt;/span&gt; - An environment action selects by all the agents at each time step in exchange for a team reward&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$m_t$&lt;/span&gt; - A communication action, observed by other agent but has no direct impact on the reward or env&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; - The agent (This being a multi-agent DQN algorithm)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;They form the input to the Deep Recurrent Q-Network architecture.&lt;/p&gt;&#xA;&lt;p&gt;The purpose of the embedding network is to receive a tuple of these inputs and produce a state&#xA;embedding &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;. This state embedding is then fed  into a hidden state &lt;span class=&quot;math-container&quot;&gt;$h^a_{t-1}$&lt;/span&gt; of the RNN.&lt;/p&gt;&#xA;&lt;p&gt;Though the authors refer to it as an input network that produces embedding, in practice, they use different embedding functions for each of these inputs. The final task/state embedding &lt;span class=&quot;math-container&quot;&gt;$z_t^a$&lt;/span&gt; is expressed as a sum:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;z_t^a = \text{MLP}(o_t^a) + \text{MLP}(m_{t-1}) + \text{LookupTable}(u_{t-1}) + \text{LookupTable}(a)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The summed embeddings are all of the same size.&lt;/p&gt;&#xA;&lt;p&gt;This answer makes an assumption that by &amp;quot;embedding layer&amp;quot; you meant the input embedding network. &lt;em&gt;Since the paper makes no reference to a single embedding layer in the model architecture.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="40671" LastEditDate="2020-11-22T14:58:57.960" LastActivityDate="2020-11-22T14:58:57.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24788" PostTypeId="1" CreationDate="2020-11-22T14:52:41.033" Score="0" ViewCount="57" Body="&lt;p&gt;I am diving in data-to-text generation for long articles (&amp;gt; 1000 words). After creating a template and fill it with data I am currently going down on paragraph level and adding different paragraphs, which are randomly selected and put together. I also added on a word level different outputs for date, time and number formats.&lt;/p&gt;&#xA;&lt;p&gt;The challenge I see is, that when creating large amounts of such generated texts they become boring to read as the uniqueness for the reader goes down.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, I also think it's easy to detect that such texts have been autogenerated. However, I still have to validate this hypotheses.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if there is an even better method to bring in variability in such a text?&lt;/p&gt;&#xA;&lt;p&gt;Can you suggest any methods, papers, resources or share your experience within this field.&lt;/p&gt;&#xA;&lt;p&gt;I highly appreciate your replies!&lt;/p&gt;&#xA;" OwnerUserId="32329" LastEditorUserId="32329" LastEditDate="2020-11-22T19:34:15.350" LastActivityDate="2020-11-30T03:53:11.140" Title="Making generated texts from &quot;data-to-text&quot; more variable" Tags="&lt;natural-language-processing&gt;&lt;text-generation&gt;&lt;natural-language-generation&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24789" PostTypeId="2" ParentId="23670" CreationDate="2020-11-22T17:30:50.123" Score="1" Body="&lt;p&gt;There are &lt;strong&gt;many variations&lt;/strong&gt; of the original VAE (proposed in the 2013 paper &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Auto-Encoding Variational Bayes&lt;/a&gt;), with &lt;strong&gt;different purposes&lt;/strong&gt; (such as the generation of discrete data or graphs). Of course, I cannot enumerate all of them, so here I will list only the ones I am currently aware of.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/2015/file/8d55a249e6baa5c06772297520da2051-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conditional variational auto-encoder (CVAE)&lt;/a&gt; (2015, NeurIPS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://research.google/pubs/pub44904/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial autoencoders (AAE)&lt;/a&gt; (2015, ICLR)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://proceedings.neurips.cc/paper/2016/file/6ae07dcb33ec3b7c814df797cbda0f87-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ladder variational autoencoder (LVAE)&lt;/a&gt; (2016,  NeurIPS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v48/larsen16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VAE/GAN&lt;/a&gt; (2016, PMLR)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://proceedings.neurips.cc/paper/2017/file/7a98af17e63a0ac09ce2e96d03992fbc-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vector quantised variational autoencoder (VQ-VAE)&lt;/a&gt; (2017, NeurIPS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1511.06406.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Denoising variational autoencoder (DVAE)&lt;/a&gt; (2017, AAAI)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.01925.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grammar variational autoencoder (GVAE)&lt;/a&gt; (2017, ICML)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.quantum-machine.org/workshops/nips2017/assets/pdf/sdvae_workshop_camera_ready.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Syntax-directed variational autoencoder (SD-VAE)&lt;/a&gt; (2017, ICLR)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1611.05013.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PixelVAE&lt;/a&gt; (2017)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-030-01418-6_41&quot; rel=&quot;nofollow noreferrer&quot;&gt;GraphVAE&lt;/a&gt; (2018, ICANN)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/2018/file/b8a03c5c15fcfa8dae0b03351eb1742f-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Constrained graph variational autoencoder (CGVAE)&lt;/a&gt; (2018, NeurIPS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pubs.acs.org/doi/pdf/10.1021/acscentsci.7b00572&quot; rel=&quot;nofollow noreferrer&quot;&gt;Character VAE (CVAE)&lt;/a&gt; (2018, ACS central science)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/2018/file/1458e7509aa5f47ecfb92536e7dd1dc7-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Regularized VAE (RVAE)&lt;/a&gt; (2018, NeurIPS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v80/jin18a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Junction tree variational autoencoder (JT-VAE)&lt;/a&gt; (2018, PMLR)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ojs.aaai.org//index.php/AAAI/article/view/4538&quot; rel=&quot;nofollow noreferrer&quot;&gt;InfoVAE/MMD-VAE&lt;/a&gt; (2019, AAAI)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://proceedings.mlr.press/v97/kajino19a/kajino19a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular hypergraph grammar variational autoencoder (MHG-VAE)&lt;/a&gt; (2019, ICML)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/2019/file/e205ee2a5de471a70c1fd1b46033a75f-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAG variational autoencoder (D-VAE)&lt;/a&gt; (2019, NeurIPS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8438540&quot; rel=&quot;nofollow noreferrer&quot;&gt;Denoising adversarial autoencoders (iDAAE and DAAE)&lt;/a&gt; (2019, IEEE Transactions on Neural Networks and Learning Systems)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://papers.nips.cc/paper/2019/file/a2186aa7c086b46ad4e8bf81e2a3a19b-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MACRo-mIcro Disentangled Variational Auto-Encoder (MacridVAE)&lt;/a&gt; (2019, NeurIPS)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;VAEs have been used for &lt;strong&gt;drug design&lt;/strong&gt; (see e.g. &lt;a href=&quot;http://proceedings.mlr.press/v80/jin18a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;JT-VAE&lt;/a&gt;) and in &lt;strong&gt;reinforcement learning&lt;/strong&gt; (see e.g. &lt;a href=&quot;https://papers.nips.cc/paper/2018/hash/2de5d16682c3c35007e4e92982f1a2ba-Abstract.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;world models&lt;/a&gt;). I don't know if they have been used for self-driving cars, but it's possible.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T17:35:57.027" LastActivityDate="2020-11-22T17:35:57.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24790" PostTypeId="1" CreationDate="2020-11-22T18:09:03.677" Score="1" ViewCount="55" Body="&lt;p&gt;I am training an agent with an Actor-Critic network and update it with TRPO so far. Now, I tried out PPO and the results are drastically different and bad. I only changed from TRPO to PPO, the rest of the environment and rewards are the same. PPO is just a more efficient method compared to TRPO and has proven to be a state-of-the-art method in RL. So, why shouldn't it work? I just thought to ask if someone knows roughly how to transform configuration parameters from TRPO to PPO.&lt;/p&gt;&#xA;&lt;p&gt;Here some more details about my configurations.&lt;/p&gt;&#xA;&lt;h3&gt;TRPO&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Actor loss:  &lt;span class=&quot;math-container&quot;&gt;$-\log(\pi) * A$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; are advantages&lt;/li&gt;&#xA;&lt;li&gt;Critic Loss: MSE(predicted_values, discounted return)&lt;/li&gt;&#xA;&lt;li&gt;Desired KL Divergence for Actor and Critic: 0.005&lt;/li&gt;&#xA;&lt;li&gt;Conjugate gradient iterations: 20&lt;/li&gt;&#xA;&lt;li&gt;Residual tolerance in conjugate gradient: 1e-10&lt;/li&gt;&#xA;&lt;li&gt;Damping coefficient for Fisher Product: 1e-3&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3gz1It.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3gz1It.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;  &lt;a href=&quot;https://i.stack.imgur.com/6MieUt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6MieUt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;  &lt;a href=&quot;https://i.stack.imgur.com/iDQAyt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iDQAyt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;PPO&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Actor and Critic optimizer and learning rate: Adam with 0.0001&lt;/li&gt;&#xA;&lt;li&gt;Actor loss: negative minimum of either:&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\pi}{\pi_{old}} * A$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$clamp(\frac{\pi}{\pi_{old}}, 1-0.1, 1+0.1) * A$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Critic loss: MSE(predicted_values, discounted_rewards)&lt;/li&gt;&#xA;&lt;li&gt;Optimization iterations: 10&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D45vkt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D45vkt.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/zREYut.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zREYut.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/iq80Bt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iq80Bt.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The rest of my problem set-up is absolutely the same. But somehow I get completely different results while training, as you can see on the plots above. I also changed learning rates and optimization iterations, gradient clipping, optimizing with mini-batches and &lt;span class=&quot;math-container&quot;&gt;$-log(\pi) * A$&lt;/span&gt; as Loss for PPO, but neither helped. Taking importance sampling &lt;span class=&quot;math-container&quot;&gt;$\frac{\pi}{\pi_{old}} * A$&lt;/span&gt;  as loss for TRPO gives the same results there.&lt;/p&gt;&#xA;&lt;p&gt;Can someone please help me to understand where could be the problem? Or which parameters I would need to change in PPO?&lt;/p&gt;&#xA;" OwnerUserId="41715" LastEditorUserId="2444" LastEditDate="2020-11-23T20:17:09.930" LastActivityDate="2020-11-23T20:17:09.930" Title="Why does PPO lead to a worse performance than TRPO in the same task?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;hyperparameter-optimization&gt;&lt;proximal-policy-optimization&gt;&lt;trust-region-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24791" PostTypeId="5" CreationDate="2020-11-22T18:34:01.403" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T18:34:01.403" LastActivityDate="2020-11-22T18:34:01.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24792" PostTypeId="4" CreationDate="2020-11-22T18:34:01.403" Score="0" Body="For questions about the concept of (information) entropy in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-22T18:34:01.403" LastActivityDate="2020-11-22T18:34:01.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24793" PostTypeId="1" CreationDate="2020-11-22T19:17:19.423" Score="2" ViewCount="102" Body="&lt;p&gt;I'm trying to understand how DeepFakes are generated and so far I understood that they're mostly generated through the usage of GANs and autoencoders.&lt;/p&gt;&#xA;&lt;p&gt;The autoencoders part is understandable, but what I cannot understand is how to generate faces with GANs that match destination face.&lt;/p&gt;&#xA;&lt;p&gt;GANs consist of a generator and a discriminator. The generator is getting &lt;strong&gt;noise input&lt;/strong&gt; which is randomly selected from a normal distribution and feedback from the discriminator. The discriminator is taught how the real data looks like and just classifies if the data fed to him is real or fake. Depending on the answer - one of them (generator/discriminator) updates its model. If the discriminator guesses right, the generator is getting updated if not, then the discriminator is the one that is updating its model.&lt;/p&gt;&#xA;&lt;p&gt;So after the training part is over, we can feed the generator more noise to achieve more fake data. In DeepFake videos, we normally try to swap the destination face with the input face. My problem with that is that the destination face has specific features for example it has closed eyes, smiles, rotates its head. If we feed the generator noise, how can we control the process to achieve similar facial features that are in the destination face?&lt;/p&gt;&#xA;&lt;p&gt;I've found papers about GANs that can control some of the features of generated faces (StyleGANs). Although I'm not sure how would it be possible to extract &amp;quot;special features&amp;quot; of destination face and generate them with StyleGANs.&lt;/p&gt;&#xA;&lt;p&gt;I will be extremely grateful for any help in understanding the concept of DeepFake with GANs.&#xA;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="42461" LastEditorUserId="42461" LastEditDate="2020-11-22T19:54:09.883" LastActivityDate="2020-11-24T19:28:22.710" Title="Generating fake faces containing specific features with GANs" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24794" PostTypeId="1" CreationDate="2020-11-22T20:58:32.113" Score="1" ViewCount="18" Body="&lt;p&gt;I have a regression task that I tray to solve with AI.&#xA;I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance)&lt;/p&gt;&#xA;&lt;p&gt;I understand basic principle: Look if model overfit or underfit - according change the parameters. In theory.&#xA;I would ask for help with two graphs:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If I understand correctly what is going on&lt;/li&gt;&#xA;&lt;li&gt;How would you attack the situation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;1. Graph&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p1P5H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p1P5H.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I use LightGBM&lt;/li&gt;&#xA;&lt;li&gt;learning_rate = 1&lt;/li&gt;&#xA;&lt;li&gt;max_depth = 3&lt;/li&gt;&#xA;&lt;li&gt;num_leaves = 2**15,&lt;/li&gt;&#xA;&lt;li&gt;number of iterations = 4000&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If I understand this model is Underfitting. The validation and training is falling, but not very much...&#xA;BUT: The number of iteration is now too large and place higher number is not ok. Learning rate is 1 (as hight as it gets). Only the max_depth is low, but if it is higher (I try 30) the graph is same, just the values are worse.&lt;/p&gt;&#xA;&lt;p&gt;So, what to do, so that model would not underfit.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Graph&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jfT68.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jfT68.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I use Neural Nets&lt;/li&gt;&#xA;&lt;li&gt;epochs=200,&lt;/li&gt;&#xA;&lt;li&gt;batch_size=64&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The model&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;i = Input(shape=(100,), name='input')&#xA;x = Dense(128)(i)&#xA;x = Dense(64)(i)&#xA;o = Dense(1, activation='relu', name='output')(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here I am not sure. This doesn't really looks like underfit, but more that doesn't converged.&#xA;Is this right?&lt;/p&gt;&#xA;&lt;p&gt;So, should I create more complex model (more neurones or more layers?)?&lt;/p&gt;&#xA;&lt;p&gt;And how much epochs do I need to see this behaviour? Because, in the beginning I use only 10 epochs, for faster development, and I thought that model is overfitting. Only when I use more epochs I see that I was wrong.&lt;/p&gt;&#xA;&lt;p&gt;How would you start to &amp;quot;debug&amp;quot; this neural net? What would be the plan of attack?&lt;/p&gt;&#xA;" OwnerUserId="26993" LastActivityDate="2020-11-22T20:58:32.113" Title="React on train-validation curve after trening" Tags="&lt;neural-networks&gt;&lt;overfitting&gt;&lt;convergence&gt;&lt;gradient-boosting&gt;&lt;validation-loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24795" PostTypeId="2" ParentId="24271" CreationDate="2020-11-22T21:30:43.850" Score="1" Body="&lt;p&gt;Apart from the &lt;strong&gt;entropy&lt;/strong&gt; and the &lt;strong&gt;cross-entropy&lt;/strong&gt;, which are widely used in deep learning and you seem to be aware of, there is also the &lt;strong&gt;Kullback-Leibler divergence&lt;/strong&gt; (also known as &lt;em&gt;relative entropy&lt;/em&gt;), which is widely used in the context of &lt;a href=&quot;http://proceedings.mlr.press/v37/blundell15.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;variational Bayesian neural networks&lt;/em&gt;&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1312.6114.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;variational auto-encoders&lt;/em&gt;&lt;/a&gt;, given that it's often part of the loss function that is minimized, i.e. the &lt;strong&gt;Evidence Lower BOund&lt;/strong&gt;, which is a proxy objective function for the KL divergence between the prior and posterior distributions (which actually corresponds to the &lt;strong&gt;minimum description length&lt;/strong&gt; needed to encode the data: huh?). See &lt;a href=&quot;https://ai.stackexchange.com/a/20574/2444&quot;&gt;this answer&lt;/a&gt; for more details. There is also the &lt;strong&gt;mutual information&lt;/strong&gt;, which has also been used as a measure of &lt;em&gt;uncertainty&lt;/em&gt; in the context of Bayesian neural networks.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T01:57:17.827" LastActivityDate="2020-11-23T01:57:17.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24796" PostTypeId="5" CreationDate="2020-11-23T01:43:44.413" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T01:43:44.413" LastActivityDate="2020-11-23T01:43:44.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24797" PostTypeId="4" CreationDate="2020-11-23T01:43:44.413" Score="0" Body="For questions about Soft Actor-Critic (SAC), which was proposed in &quot;Soft Actor-Critic:&#xD;&#xA;Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor&quot; by Tuomas Haarnoja et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T01:43:44.413" LastActivityDate="2020-11-23T01:43:44.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24798" PostTypeId="1" CreationDate="2020-11-23T04:10:04.577" Score="2" ViewCount="86" Body="&lt;p&gt;I'm trying to implement the logic for a Sudoku XV puzzle, that it's essentially a standard sudoku with the addition of X and V markers between some pairs of squares. X markers in adjacent pairs requires that the sum of the two values is 10. Similarly, the V marks requires that the sum of the values is equal to 5.&lt;/p&gt;&#xA;&lt;p&gt;(Assume that &lt;span class=&quot;math-container&quot;&gt;$$ S_{xyz} $$&lt;/span&gt; stands for [digit][row][column])&lt;/p&gt;&#xA;&lt;p&gt;I've written the following CNF formulae that handle the logic of a standard Sudoku puzzle:&lt;/p&gt;&#xA;&lt;p&gt;There is at least one number in each entry:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \bigwedge_{x=1}9\bigwedge_{y=1}9\bigwedge_{z=1}9S_{xyz} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each number appears at most once in each row:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \bigwedge_{y=1}9\bigwedge_{z=1}9\bigwedge_{x=1}{8\bigwedge_{i=x+1}9}{(\lnot S}_{xyz\ }\vee\lnot S_{iyz\ }) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each number appears at most once in each column:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \bigwedge_{x=1}9\bigwedge_{z=1}9\bigwedge_{y=1}{8\bigwedge_{i=x+1}9}{(\lnot S}_{xyz\ }\vee\lnot S_{xiz\ }) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each number appears at most once in each 3x3 sub-grid:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\bigwedge_{z=1}9\bigwedge_{i=0}2\bigwedge_{j=0}{2\bigwedge_{x=1}2\bigwedge_{y+1}3\bigwedge_{k=x+1}3\bigwedge_{l=1,\ \ y \neq l}3}{(\lnot S}_{(3i+x)(3j+y)z\ }\vee\lnot S_{(3i+k)(3j+l)z\ })&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, I'm stuck, and I don't really know how I can express the logic for X and V markers, and most importantly how to invalidate squares that contain neither an X nor a V marker that have digits summing to 5 or 10.&lt;/p&gt;&#xA;" OwnerUserId="42467" LastEditorUserId="2444" LastEditDate="2020-11-23T11:18:30.917" LastActivityDate="2020-11-23T11:18:30.917" Title="How to translate sudoku XV boards in CNF format?" Tags="&lt;ai-design&gt;&lt;logic&gt;&lt;sudoku&gt;&lt;conjunctive-normal-form&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24799" PostTypeId="1" AcceptedAnswerId="24803" CreationDate="2020-11-23T04:12:38.527" Score="3" ViewCount="107" Body="&lt;p&gt;Both MC and TD are model-free and they both follow a sample trajectory (in the case of TD, the trajectory is cut-short) to estimate the return (we basically are sampling Q values). Other than that, the underlying structure of both algorithms is exactly the same. However, from the blogs and texts I read, the equations are expressed in terms of V and NOT Q. Why is that?&lt;/p&gt;&#xA;" OwnerUserId="42398" LastEditorUserId="2444" LastEditDate="2020-11-23T13:25:31.700" LastActivityDate="2020-11-23T13:37:23.667" Title="Is the expected value we sample in TD-learning action-value Q or state-value V?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;temporal-difference-methods&gt;&lt;value-functions&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24800" PostTypeId="2" ParentId="10371" CreationDate="2020-11-23T06:48:37.463" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Are there companies that still use expert systems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are still some expert system inference engines available in open source form, in particular &lt;a href=&quot;http://www.clipsrules.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLIPS rules&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A specialization of your question could be: what companies are using CLIPS in 2020 ?&lt;/p&gt;&#xA;&lt;p&gt;I don't have any ideas, even if I did try in &lt;a href=&quot;https://github.com/bstarynk/clips-rules-gcc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bstarynk/clips-rules-gcc&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt; project is right now (in November 2020) discussing the idea of incorporating such rules in it.&lt;/p&gt;&#xA;&lt;p&gt;Read of course quickly Jacques Pitrat's blog on  &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bootstrappingartificialintelligence.fr/WordPress3/&lt;/a&gt; and his last book (describing the design of an ambitious symbolic artificial intelligence system -CAIA- with expert system ideas)  &lt;em&gt;Artificial Beings&#xA;The Conscience of a Conscious Machine&lt;/em&gt; (&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1848211015&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;ISBN 9 781848211018&lt;/a&gt;) - his CAIA system is on &lt;a href=&quot;https://github.com/bstarynk/caia-pitrat&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bstarynk/caia-pitrat&lt;/a&gt; but there is absolutely no documentation, since Jacques Pitrat passed away in October 2019. His CAIA system was capable of generating all the 500KLOC of its C code from some kind of expert system rules (whose design is described in Jacques Pitrat's books and papers).&lt;/p&gt;&#xA;&lt;p&gt;I am not a native English speaker (since I am French) but &lt;strong&gt;I heard that expert systems are called&lt;/strong&gt; (in 2020) &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Business_rule_management_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;business rule management systems&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I heard that major banks in France (maybe &lt;a href=&quot;https://en.wikipedia.org/wiki/BNP_Paribas&quot; rel=&quot;nofollow noreferrer&quot;&gt;BNP&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Soci%C3%A9t%C3%A9_G%C3%A9n%C3%A9rale&quot; rel=&quot;nofollow noreferrer&quot;&gt;Société Générale&lt;/a&gt;) are using such systems to decide to give some loan or some credit to persons and companies (in particular for people buying their flat - or a brand new automobile - with a bank credit and debt during dozen of years).&lt;/p&gt;&#xA;&lt;p&gt;The French banking system is very opaque: you won't be able to understand their internal software, and banks are not publishing any document about the design of their software. At most they would publish the &lt;em&gt;name&lt;/em&gt; of their AI systems, but nothing public about the software design.&lt;/p&gt;&#xA;&lt;p&gt;According to rumors, &lt;a href=&quot;https://www.lexifi.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lexifi&lt;/a&gt; or &lt;a href=&quot;https://www.yseop.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yseop&lt;/a&gt; might use some kind of very proprietary expert system technology and sell services with them. But their software tools are closed source and very proprietary.&lt;/p&gt;&#xA;&lt;p&gt;Regarding expert system for games, see also recent papers by &lt;a href=&quot;https://www.lamsade.dauphine.fr/%7Ecazenave/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Professor Tristan Cazenave&lt;/a&gt;. He did use some kind of expert system technology for games.&lt;/p&gt;&#xA;&lt;p&gt;My guess is that large Internet companies like Google or Amazon are using expert system technology inside their internal software (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Search_engine&quot; rel=&quot;nofollow noreferrer&quot;&gt;search engines&lt;/a&gt;). &lt;a href=&quot;https://en.wikipedia.org/wiki/Watson_(computer)&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM Watson&lt;/a&gt; is rumored to use them also.&lt;/p&gt;&#xA;&lt;p&gt;BTW, &lt;a href=&quot;https://www.gnu.org/software/make/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU make&lt;/a&gt; might be considered as some very crude expert system engine driving building of software artifact from source code.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-11-23T07:48:38.683" LastActivityDate="2020-11-23T07:48:38.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24801" PostTypeId="1" CreationDate="2020-11-23T07:34:36.010" Score="4" ViewCount="74" Body="&lt;p&gt;I find the logistic map absolutely fascinating. Both in itself (because I love fractal) and because it is observed in nature (see: &lt;a href=&quot;https://www.youtube.com/watch?v=ovJcsL7vyrk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=ovJcsL7vyrk&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if anyone tried it as an activation function in some way or another with any kind of success.&lt;/p&gt;&#xA;&lt;p&gt;I like it because it has some kind of &amp;quot;I'm not sure what to do&amp;quot; above ~3.0 and the less confidence the more chaotic the response is. It gives the possibility to explore some other solution to escape a local optimum (not sure I use this word correctly).  And below 3 it's still a nice and smooth activation function like, eg, a tanh.&lt;/p&gt;&#xA;&lt;p&gt;Eg : the reward i got isn't the reward i expect, and the higher the difference the more i'll explore other solution. But it's still gradual, from 1 choice, to 2 choice, 4, 8, 16, ... until it become chaotic. (giving the possibility to experiment some pseudo-random choice). And below this threshold it still act as a usable &amp;quot;good old&amp;quot; activation function.&lt;/p&gt;&#xA;&lt;p&gt;Another good side is that it's gpu-friendly and don't need many iteration for this application since a little bit of uncertainty (even below the threshold) isn't undesirable. see : &lt;a href=&quot;https://upload.wikimedia.org/wikipedia/commons/6/63/Logistic_Map_Animation.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://upload.wikimedia.org/wikipedia/commons/6/63/Logistic_Map_Animation.gif&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edit : so, ok, i tested it on my extremely naive racetrack. (feedforward, no feedback, no error, no fitness, only genetic selection for the car that didn't crash). It does work, for sure. I don't see any advantage in practive but with such a naive NN, there isn't much i can tell.&lt;/p&gt;&#xA;&lt;p&gt;My implementation :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def logi(r):&#xA;    x = .6  # the initial population doesn't matter so i took .6&#xA;    for _ in range(random.randrange(10,40)):&#xA;        x = r * x * (1 - x)&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The activation take 8% of my laptop cpu (while is was invisible on my radar with leaky leru)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XX7zz.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XX7zz.gif&quot; alt=&quot;Logistic map bifurcation diagram&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42373" LastEditorUserId="2444" LastEditDate="2020-11-24T10:23:15.993" LastActivityDate="2020-11-24T10:23:15.993" Title="Has the logistic map ever been used as an activation function?" Tags="&lt;neural-networks&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="19" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24802" PostTypeId="2" ParentId="24784" CreationDate="2020-11-23T08:23:32.927" Score="1" Body="&lt;p&gt;Simple policy parametizations, including linear functions in some cases, can solve continous control tasks in RL. It's therefore not necessary to have a complex approximator for the function to be expressive enough in capturing the desired agent's behaviour in popular RL benchmarks&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.02660.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards Generalization and Simplicity in Continous Control&lt;/a&gt; tries to answer the question &amp;quot;What are the simplest set of ingredients needed to succeed in some of the popular benchmarks?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Among them is the representation capacity required of the function approximators.&lt;/p&gt;&#xA;&lt;p&gt;It shows that in popular OpenAI gym locomotion benchmarks (Swimmer 3D,  3D Humanoid, Ant, Walker), using &lt;a href=&quot;http://www.cs.cmu.edu/afs/cs.cmu.edu/project/learn-43/lib/photoz/.g/web/glossary/linear.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Radial_basis_function_kernel&quot; rel=&quot;nofollow noreferrer&quot;&gt;RBF&lt;/a&gt; policy architectures is just as competitive in the reward attained as using neural nets. Here's a summary:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The more Fourier Featurers RBF has, the better the performance&lt;/li&gt;&#xA;&lt;li&gt;A 2 layer neural net policy's performance is slightly short of a 500-feature RBF&lt;/li&gt;&#xA;&lt;li&gt;A linear representation is unable to learn some of the locomotion behaviour, e.g walking&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I assume RBF, and its use of Conjugate Gradients, has some shortcomings which make shallow Neural networks more preferrable. (I'm not familiar with RBF, and stand to be corrected).&lt;/p&gt;&#xA;&lt;p&gt;Overall, the the authors use these result to suggest that complex policy architectures should not be a default choice in RL unless side-by-side comparisons with simpler alternatives suggest otherwise.&lt;/p&gt;&#xA;&lt;p&gt;This might explain the prevalence of shallow networks. It's also possible that since most RL research (presently) doesn't try to show how we can learn better representations using complex policy architectures, but rather how to better make use of representation capacities by addressing issues like sample efficiency, shallow networks are expressive enough in the adapted benchmarks. As expressed in Brail's comment, this is also useful for setting up fair comparisons.&lt;/p&gt;&#xA;&lt;p&gt;However, the effect of network architecture design in RL has not been widely pursued.&lt;/p&gt;&#xA;&lt;p&gt;It's also worth mentioning the findings of this work &lt;a href=&quot;https://arxiv.org/abs/2010.09163&quot; rel=&quot;nofollow noreferrer&quot;&gt;D2RL, (2020)&lt;/a&gt; which shows one can gain significant performance improvement by using input concatenations in the policy and value function, and that more layers in a &lt;em&gt;vanilla&lt;/em&gt; neural net decreases the agent performance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5DtMh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5DtMh.png&quot; alt=&quot;Effect of no. of layers on SAC performance&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;The effect of increasing the number of FCs in SAC. Performance drops after 2 layers (D2RL)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Further Reads&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.reddit.com/r/reinforcementlearning/comments/drph2m/why_is_everyone_using_shallow_neural_networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why is everyone using shallow networks&lt;/a&gt; (Reddit thread)&lt;/p&gt;&#xA;" OwnerUserId="40671" LastEditorUserId="40671" LastEditDate="2020-11-23T08:28:48.297" LastActivityDate="2020-11-23T08:28:48.297" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24803" PostTypeId="2" ParentId="24799" CreationDate="2020-11-23T08:39:21.460" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;However, from the blogs and texts I read, the equations are expressed in terms of V and NOT Q. Why is that?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;MC and TD are methods for associating value estimates to time step based on experienced gained in later time steps. It does not matter what kind of value estimate is being associated across time, because all value functions are expressing the same thing in general, which is the &lt;em&gt;expected return&lt;/em&gt; conditioned on a &amp;quot;current position&amp;quot; within the MDP. In MC the association is directly with a sampled return, in TD with a sampled combination of immediate reward and a later value estimate - most commonly in TD the same kind of value estimate (e.g. matching later state value estimates to state values).&lt;/p&gt;&#xA;&lt;p&gt;Both approaches can be analysed and used from the perspective of both state value (V) and action value (Q) functions. They also apply to other value functions - e.g. afterstate values.&lt;/p&gt;&#xA;&lt;p&gt;It is quite common for textbooks and tutorials to use the slightly simpler state value function to explain how MC or TD learning work in general, outside of being used for any purpose. You can also use the state value function for model-free &lt;em&gt;policy evaluation&lt;/em&gt; in MC and TD.&lt;/p&gt;&#xA;&lt;p&gt;However, without a model, you cannot use state value function for control (i.e. to learn an optimal policy). To pick the best action using state values, you need to do something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a [ \sum_{r,s'} p(r,s'|s,a)(r + \gamma v(s'))]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem here is that &lt;span class=&quot;math-container&quot;&gt;$p(r,s'|s,a)$&lt;/span&gt; is a model of the environment. So, if it is needed, the control method would not be model-free.&lt;/p&gt;&#xA;&lt;p&gt;Hence when you learn about MC or TD in a &lt;em&gt;control&lt;/em&gt; scenario, model-free methods to learn optimal policies, then you generally &lt;em&gt;need&lt;/em&gt; to use an action value (sometimes you can use an afterstate value, if the action involves choosing the next state directly).&lt;/p&gt;&#xA;&lt;p&gt;With an action value function, the greedy policy becomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(s) = \text{argmax}_a q(s, a)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This does not refer to any model of the environment. So it can be used when you have none.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-11-23T13:37:23.667" LastActivityDate="2020-11-23T13:37:23.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24804" PostTypeId="1" AcceptedAnswerId="24843" CreationDate="2020-11-23T11:00:13.750" Score="2" ViewCount="28" Body="&lt;p&gt;I'm trying to read &lt;a href=&quot;https://arxiv.org/pdf/1609.08144.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper describing Google's LSTM architecture for machine translation. It features this diagram on page 4:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/B9wi6.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm interested in the encoder block, on the left. Apparently, the pink and green cells are LSTMs. However, I can't tell if the x-axis is space or time. That is, are the LSTM cells on a given row all &lt;em&gt;the same&lt;/em&gt; cell, with time flowing forward from left to right? The diagram on the next page in the paper seems to suggest that.&lt;/p&gt;&#xA;" OwnerUserId="1931" LastEditorUserId="2444" LastEditDate="2020-11-23T13:12:43.797" LastActivityDate="2020-11-24T15:39:14.460" Title="Does this diagram represent several LSTMs, or one through several timesteps?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;long-short-term-memory&gt;&lt;papers&gt;&lt;google&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24805" PostTypeId="5" CreationDate="2020-11-23T11:20:35.810" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T11:20:35.810" LastActivityDate="2020-11-23T11:20:35.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24806" PostTypeId="4" CreationDate="2020-11-23T11:20:35.810" Score="0" Body="For questions about boolean formulas in a conjunctive normal form (CNF), i.e. a disjunction (ANDs) of literals, in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T11:20:35.810" LastActivityDate="2020-11-23T11:20:35.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24807" PostTypeId="5" CreationDate="2020-11-23T11:21:11.643" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T11:21:11.643" LastActivityDate="2020-11-23T11:21:11.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24808" PostTypeId="4" CreationDate="2020-11-23T11:21:11.643" Score="0" Body="For questions that involve the famous Sudoku game in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T11:21:11.643" LastActivityDate="2020-11-23T11:21:11.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24810" PostTypeId="2" ParentId="20127" CreationDate="2020-11-23T12:24:31.967" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/20135/2444&quot;&gt;The accepted answer&lt;/a&gt; does not provide a good definition of &lt;em&gt;over-fitting&lt;/em&gt;, which actually exists and is a defined concept in reinforcement learning too. For example, the paper &lt;a href=&quot;http://proceedings.mlr.press/v97/cobbe19a/cobbe19a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantifying Generalization in Reinforcement Learning&lt;/a&gt; completely focuses on this issue. Let me give you more details.&lt;/p&gt;&#xA;&lt;h3&gt;Over-fitting in supervised learning&lt;/h3&gt;&#xA;&lt;p&gt;In &lt;strong&gt;supervised learning (SL)&lt;/strong&gt;, &lt;strong&gt;over-fitting&lt;/strong&gt; is defined as the difference (or gap) in the performance of the ML model (such as a neural network) on the training and test datasets. If the model performs significantly better on the training dataset than on the test dataset, then the ML model has over-fitted the training data. Consequently, it has not generalized (well enough) to other data other than the training data (i.e. the test data). The relationship between over-fitting and &lt;a href=&quot;https://ai.stackexchange.com/a/16737/2444&quot;&gt;generalization&lt;/a&gt; should now be clearer.&lt;/p&gt;&#xA;&lt;h3&gt;Over-fitting in reinforcement learning&lt;/h3&gt;&#xA;&lt;p&gt;In &lt;strong&gt;reinforcement learning (RL)&lt;/strong&gt; (you can find a brief recap of what RL is &lt;a href=&quot;https://ai.stackexchange.com/a/20957/2444&quot;&gt;here&lt;/a&gt;), you want to find an &lt;strong&gt;optimal policy&lt;/strong&gt; or value function (from which the policy can be derived), which can be represented by a neural network (or another model). A policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is optimal in environment &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; if it leads to the highest cumulative reward in the long run in that environment &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, which is often mathematically modelled as a (partially or fully observable) Markov decision process.&lt;/p&gt;&#xA;&lt;p&gt;In some cases, you are also interested in knowing whether your policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; can also be used in a different environment than the environment it has been trained in, i.e. you're interested in knowing if the knowledge acquired in that training environment &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; can be &lt;strong&gt;transferred&lt;/strong&gt; to a different (but typically related) environment (or task) &lt;span class=&quot;math-container&quot;&gt;$E'$&lt;/span&gt;. For example, you may only be able to train your policy in a simulated environment (because of resource/safety constraints), then you want to transfer this learned policy to the real world. In those cases, you can define the concept of &lt;strong&gt;over-fitting&lt;/strong&gt; in a similar way to the way we define over-fitting in SL. The only difference may be that you may say that the learned policy &lt;em&gt;has over-fitted the training environment&lt;/em&gt; (rather than saying that the ML model has over-fitted the training dataset), but, given that the environment provides the &lt;em&gt;data&lt;/em&gt;, then you could even say in RL that your policy has over-fitted the training data.&lt;/p&gt;&#xA;&lt;h3&gt;Catastrophic forgetting&lt;/h3&gt;&#xA;&lt;p&gt;There is also the issue of &lt;a href=&quot;https://ai.stackexchange.com/a/13293/2444&quot;&gt;&lt;strong&gt;catastrophic forgetting (CF)&lt;/strong&gt;&lt;/a&gt; in RL, i.e., while learning, your RL agent may forget what it's previously learned, and this can even happen in the same environment. Why am I talking about CF? Because what it's happening to you is probably CF, i.e., &lt;em&gt;while learning&lt;/em&gt;, the agent performs well for a while, then its performance drops (although I have read a paper that strangely defines CF differently in RL). You could also say that over-fitting is happening in your case, but, if you are continuously training and the performance changes, then CF is probably what you need to investigate. So, you should reserve the word &lt;em&gt;over-fitting&lt;/em&gt; in RL when you're interested in &lt;strong&gt;transfer learning&lt;/strong&gt; (i.e. the training and test environments do not coincide).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T15:38:49.910" LastActivityDate="2020-11-23T15:38:49.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24811" PostTypeId="1" CreationDate="2020-11-23T12:40:36.673" Score="2" ViewCount="23" Body="&lt;p&gt;I am a bit stuck trying to understand how a lane change is performed from an operational point of view.&lt;/p&gt;&#xA;&lt;p&gt;Let's assume a self-driving car uses an occupancy grid map for local planning, this map may even have the detected lane boundaries. It's following a slow car and decides to overtake, but how does it know where the centre of the adjacent lane is? Does it use a separate map, or is there a separate data structure that is used to keep the lane information which informs the car where the centre of the adjacent lane is?&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, does the car just decide to start drifting off to the side until it picks up the lane boundaries and then centres itself?&lt;/p&gt;&#xA;" OwnerUserId="42480" LastActivityDate="2020-11-23T12:40:36.673" Title="How do self-driving cars perform lane changes?" Tags="&lt;autonomous-vehicles&gt;&lt;control-theory&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24812" PostTypeId="5" CreationDate="2020-11-23T13:10:57.313" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:10:57.313" LastActivityDate="2020-11-23T13:10:57.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24813" PostTypeId="4" CreationDate="2020-11-23T13:10:57.313" Score="0" Body="For questions about cryptocurrencies in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:10:57.313" LastActivityDate="2020-11-23T13:10:57.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24814" PostTypeId="5" CreationDate="2020-11-23T13:34:29.083" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:34:29.083" LastActivityDate="2020-11-23T13:34:29.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24815" PostTypeId="4" CreationDate="2020-11-23T13:34:29.083" Score="0" Body="For questions about the Gradient-weighted Class Activation Mapping (GradCAM) technique for explainable AI, which was proposed in the paper &quot;Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization&quot; (2019) by Ramprasaath R. Selvaraju et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:34:29.083" LastActivityDate="2020-11-23T13:34:29.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24816" PostTypeId="1" AcceptedAnswerId="24840" CreationDate="2020-11-23T13:48:58.537" Score="4" ViewCount="119" Body="&lt;p&gt;In &lt;a href=&quot;https://ai.stackexchange.com/a/24803/2444&quot;&gt;this answer&lt;/a&gt;, &lt;strong&gt;afterstate value functions&lt;/strong&gt; are mentioned, and that temporal-difference (TD) and Monte Carlo (MC) methods can also use these value functions. Mathematically, how are these value functions defined? Yes, they are a function of the next state, but what's the Bellman equation here? Is it simply defined as &lt;span class=&quot;math-container&quot;&gt;$v(s') = \mathbb{E}\left[ R_t \mid S_t = s, A_t = a, S_{t+1} = s' \right]$&lt;/span&gt;? If yes, how can we define it in terms of the state, &lt;span class=&quot;math-container&quot;&gt;$v(s)$&lt;/span&gt;, and state-action, &lt;span class=&quot;math-container&quot;&gt;$q(s, a)$&lt;/span&gt;, value functions, or as a Bellman (recursive) equation?&lt;/p&gt;&#xA;&lt;p&gt;Sutton &amp;amp; Barto's book (2nd edition) informally describe afterstate value functions in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=158&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 6.8&lt;/a&gt;, but they don't provide a formal definition (i.e. Bellman equation in terms of reward or other value functions), so that's why I am asking this question.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T14:14:20.290" LastActivityDate="2021-02-17T14:37:36.783" Title="How are afterstate value functions mathematically defined?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;&lt;afterstate-value-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24817" PostTypeId="5" CreationDate="2020-11-23T13:50:16.717" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T13:50:16.717" LastActivityDate="2020-11-23T13:50:16.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24818" PostTypeId="4" CreationDate="2020-11-23T13:50:16.717" Score="0" Body="For questions about the concept of a &quot;afterstate value function&quot;, as informally defined in section 6.8 of Sutton and Barto's book (2nd edition). Afterstates are also known as &quot;post-decision states&quot; in the literature." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-24T11:22:05.113" LastActivityDate="2020-11-24T11:22:05.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24820" PostTypeId="2" ParentId="24406" CreationDate="2020-11-23T15:24:25.013" Score="1" Body="&lt;p&gt;The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way to create an environment based on image. If you want to use implemented algorithm for open ai gym and want to change environment for your own, could do something like this: &lt;a href=&quot;https://towardsdatascience.com/creating-a-custom-openai-gym-environment-for-stock-trading-be532be3910e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/creating-a-custom-openai-gym-environment-for-stock-trading-be532be3910e&lt;/a&gt;. If need to change env complitely than it is necessary to rewrite your algorithm for your tasks.&lt;/p&gt;&#xA;" OwnerUserId="42038" LastActivityDate="2020-11-23T15:24:25.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24821" PostTypeId="1" CreationDate="2020-11-23T16:43:57.657" Score="0" ViewCount="24" Body="&lt;p&gt;I have a project where i'm supposed to find the distance and the height of a specific object in an image taken by a drone using &lt;strong&gt;one camera&lt;/strong&gt;. I have looked into perspective transformation/correction but i couldn't find any solutions regarding the camera tilt. I have also tried using &lt;strong&gt;OpenCV&lt;/strong&gt;'s &lt;strong&gt;cv2.getPerspectiveTransform&lt;/strong&gt; and &lt;strong&gt;cv2.warpPerspective&lt;/strong&gt; but i was not able to transform the whole image as i need ( to be able to calculate the distance from the pixel count between objects).&#xA;The image gets distorted by the perspective so i would like to be able to transform my image to eye bird view like image retrieving the lost pixels from the tilt and translating the pixel count to a real world&#xA;measurement. How can i achieve this?&lt;/p&gt;&#xA;" OwnerUserId="42486" LastActivityDate="2020-11-23T16:43:57.657" Title="Find the distance between two objects from a 45 degree tilted camera images taken from a drone with a specific elevation from the ground" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;opencv&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24822" PostTypeId="1" CreationDate="2020-11-23T16:55:53.470" Score="0" ViewCount="24" Body="&lt;p&gt;I want to try a hierarchical reinforcement learning (HRL) approach to hard logical problems with combinatorial complexity, i.e. games like chess or Rubik's cube. The majority of HRL papers I have found so far focus either on training a control policy or they tackle quite simple games.&lt;/p&gt;&#xA;&lt;p&gt;By HRL I mean all methods that (among others):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;split hard and complex problem into a series of simpler ones&lt;/li&gt;&#xA;&lt;li&gt;create desired intermediate goals (or spaces of such goals)&lt;/li&gt;&#xA;&lt;li&gt;somehow think in terms of 'what to achieve' rather than 'how to achieve'&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Do you know any examples of solving logically hard problems with HRL or maybe just any promising approaches to such problems?&lt;/p&gt;&#xA;" OwnerUserId="42483" LastActivityDate="2020-11-23T16:55:53.470" Title="Hierarchical reinforcement learning for combinatorial complexity" Tags="&lt;reinforcement-learning&gt;&lt;combinatorial-games&gt;&lt;hierarchical-rl&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24823" PostTypeId="1" AcceptedAnswerId="24828" CreationDate="2020-11-23T17:32:28.553" Score="5" ViewCount="388" Body="&lt;p&gt;I am reading about local search: hill climbing, and its types, and simulated annealing&lt;/p&gt;&#xA;&lt;p&gt;One of the hill climbing versions is &amp;quot;stochastic hill climbing&amp;quot;, which has the following definition:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Stochastic hill climbing does not examine for all its neighbor before moving. Rather, this search algorithm selects one neighbor node at random and decides whether to choose it as a current state or examine another state&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Some sources mentioned that it can be used to avoid local optima.&lt;/p&gt;&#xA;&lt;p&gt;Then I was reading about simulated annealing and its definition:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;At every iteration, a random move is chosen. If it improves the situation then the move is accepted, otherwise it is accepted with some probability less than 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, what is the main difference between the two approaches? Does the stochastic choose only random (uphill) successor? If it chooses only (uphill-successors), then how does it avoid local optima?&lt;/p&gt;&#xA;" OwnerUserId="42300" LastEditorUserId="2444" LastEditDate="2020-11-23T20:27:16.453" LastActivityDate="2020-11-23T20:56:06.190" Title="What is the difference between Stochastic Hill Climbing and Simulated Annealing?" Tags="&lt;comparison&gt;&lt;simulated-annealing&gt;&lt;meta-heuristics&gt;&lt;local-search&gt;&lt;stochastic-hill-climbing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24824" PostTypeId="1" CreationDate="2020-11-23T18:48:32.113" Score="0" ViewCount="34" Body="&lt;p&gt;I performed semantic segmentation with U-net. My dataset consists of grayscale images of defects. After training the dataset for I got an metric accuracy of only 0.3 - 0.4 IOU. Eventhough it is merely low it performs well enough to identify instances that are huge means the prediction performs well enough in places where there is a standard intensity change(color change) and they are bigger instances. There are many other instances where there is no color change and it occoupies only few pixels in image(smaller instances) and the prediction rate is almost 0 on these instances.&lt;/p&gt;&#xA;&lt;p&gt;I also tried Resdiual connection in the downsampling part of U-net likewise in ResNet. But still its the same and for smaller instances I used dilated convolution blocks in between the skip connections for encoder and decoder of U-net based on some papers. But still I cannot have a higher accuracy in my network and prediction rate for smaller instances are really poor. Although I use only 350 images for training with Data Augmentations. My image size is also 256,256.&lt;/p&gt;&#xA;&lt;p&gt;Is there any other method I can try to increase the accuracy and prediction rate for smaller instances?&#xA;Any suggestion would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="41471" LastActivityDate="2021-04-23T20:08:25.843" Title="Semantic segmentation failing in small instance detection" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;&lt;u-net&gt;&lt;binary-classification&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24825" PostTypeId="1" CreationDate="2020-11-23T19:10:08.207" Score="1" ViewCount="27" Body="&lt;p&gt;In a 2-armed-bandit problem, an agent has an opportunity to see n reward for each action. Now the agent should choose actions m times and maximize the expected reward in these m decisions. but it cant see the reward of them. what is the best approach for this?&lt;/p&gt;&#xA;" OwnerUserId="42487" LastActivityDate="2020-11-23T19:10:08.207" Title="Multi-armed bandit problem without getting rewards" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24826" PostTypeId="5" CreationDate="2020-11-23T20:28:25.413" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T20:28:25.413" LastActivityDate="2020-11-23T20:28:25.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24827" PostTypeId="4" CreationDate="2020-11-23T20:28:25.413" Score="0" Body="For questions about the stochastic hill climbing algorithm, which a variant of the hill climbing algorithm that chooses, at random, one of the uphill moves (i.e. moves that improve the objective)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T21:17:56.070" LastActivityDate="2020-11-23T21:17:56.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24828" PostTypeId="2" ParentId="24823" CreationDate="2020-11-23T20:56:06.190" Score="4" Body="&lt;p&gt;Russell and Norvig's book (3rd edition) describe these two algorithms (section 4.1.1., p. 122) and this book is the reference that you should generally use when studying search algorithms in artificial intelligence. I am familiar with simulated annealing (SA), given that I implemented it in the past to solve a combinatorial problem, but I am not very familiar with stochastic hill climbing (SHC), so let me quote the parts of the book that describe SHC.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Stochastic hill climbing&lt;/strong&gt; chooses at random from among the uphill moves; the probability of selection can vary with the steepness of the uphill move. This usually converges more slowly than steepest ascent, but in some state landscapes, it finds better solutions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, SHC chooses at random one &amp;quot;uphill move&amp;quot;, i.e. a move that improves the objective function (for example, if you're trying to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;travelling salesman problem&lt;/em&gt;&lt;/a&gt;, a &amp;quot;uphill move&amp;quot; could be any change to the current Hamiltonian cycle, a solution, so that the new Halmitonian cycle has a shorter cost) among the uphill moves (so among some set of moves that improve the objective).&lt;/p&gt;&#xA;&lt;p&gt;In &lt;strong&gt;simulated annealing&lt;/strong&gt;, you perform some move. If that move leads to a better solution, you always keep the better solution. If it leads to a worse solution, you accept that worse solution with a certain probability. There are other details, such as how you accept the worse solution (which you can find in Russell and Norvig's book), but this should already clarify that SA is different from SHC: &lt;strong&gt;SA can accept worse solutions in order to escape from local minima, while SHC accepts only uphill moves.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-11-23T20:56:06.190" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24829" PostTypeId="5" CreationDate="2020-11-23T21:33:12.067" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T21:33:12.067" LastActivityDate="2020-11-23T21:33:12.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24830" PostTypeId="4" CreationDate="2020-11-23T21:33:12.067" Score="0" Body="For questions that involve the famous travelling salesman problem (TSP) in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-23T21:33:12.067" LastActivityDate="2020-11-23T21:33:12.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24831" PostTypeId="1" CreationDate="2020-11-23T22:03:11.567" Score="1" ViewCount="18" Body="&lt;p&gt;I know the original Transformer and the GPT (1-3) use two slightly different &lt;strong&gt;positional encoding&lt;/strong&gt; techniques.&lt;/p&gt;&#xA;&lt;p&gt;More specifically, in GPT they say positional encoding is &lt;em&gt;learned&lt;/em&gt;. What does that mean? OpenAI's papers don't go into detail very much.&lt;/p&gt;&#xA;&lt;p&gt;How do they really differ, mathematically speaking?&lt;/p&gt;&#xA;" OwnerUserId="26580" LastActivityDate="2020-11-23T22:03:11.567" Title="How Positional Encoding differs between Transformer and GPT?" Tags="&lt;transformer&gt;&lt;gpt&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24832" PostTypeId="1" CreationDate="2020-11-23T22:40:29.340" Score="0" ViewCount="28" Body="&lt;p&gt;I'm running motionEye as my NVR for my home surveillance system.  This software works pretty great for motion detection but at night it gets a lot of false positives due to bugs/spiderwebs/etc in my video captures.  While hard drive space is cheap, it does add up and makes going through these captures fairly tedious and time consuming.&lt;/p&gt;&#xA;&lt;p&gt;I know that solutions do exist (Doods/Deep Stacks) that are able to identify cars/people/animals in photos.  The problem with this is that parked cars also trigger a positive detection.  I'm wondering if anyone can tell me if there is something like Doods/Deep Stacks that is able to detect moving cars/people/animals in a capture?  If so, I could script something that would process the recordings to see if it had something I was interested in and keep it or delete it otherwise.  This would greatly reduce the amount of space needed for storing these clips and would allow for me to keep more of the clips that contain the objects I'm looking for.&lt;/p&gt;&#xA;&lt;p&gt;Any help in this is most appreciated.  All the best to you!&lt;/p&gt;&#xA;" OwnerUserId="42488" LastEditorUserId="42488" LastEditDate="2020-11-23T23:36:11.127" LastActivityDate="2020-11-23T23:36:11.127" Title="AI for filtering out false positives in surveillance video captures" Tags="&lt;video-classification&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24833" PostTypeId="2" ParentId="24824" CreationDate="2020-11-23T23:26:06.830" Score="0" Body="&lt;p&gt;You may find it useful to categorize these smaller defects as a separate class, then introduce a class weights matrix to penalize incorrect classification of the smaller defects more heavily. If these small defects represent a very small portion of the total number of pixels in your training data, then the model may be stuck at a local minimum, where it just predicts them as zero because the loss is not heavily penalized for this. So you need to add extra penalty for the incorrect classification.&lt;/p&gt;&#xA;&lt;p&gt;Assuming you create a new class for the small defects, the class weights matrix is implemented as a keras callback function.&lt;/p&gt;&#xA;&lt;p&gt;If you do not want to create a multiclass problem, there is also this paper : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5977596/#!po=32.4561&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5977596/#!po=32.4561&lt;/a&gt;&#xA;The multilevel unet architecture will not solve class imbalance if that is truly the underlying issue, but if may make the network more aware of smaller scale changes in the images you're working with.&lt;/p&gt;&#xA;" OwnerUserId="42489" LastEditorUserId="42489" LastEditDate="2020-11-24T18:48:25.220" LastActivityDate="2020-11-24T18:48:25.220" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24834" PostTypeId="2" ParentId="24767" CreationDate="2020-11-24T00:41:07.560" Score="-2" Body="&lt;p&gt;I can't fully explain the part because I forgot what it talks about.&lt;/p&gt;&#xA;&lt;p&gt;However, regarding the hinge loss, it is basically allowing your SVM to tolerate misclassifications without increasing the cost function.&lt;/p&gt;&#xA;&lt;p&gt;For example, you give someone 1 dollar or 1 euro. You can forgive them, you tolerate it. Your hinge loss is 0 for lending someone 1d dollar. However, if you give them 10 dollars or 100, you will ask them to refund you ASAP because you can't tolerate that much loss!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nj05a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nj05a.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42490" LastActivityDate="2020-11-24T00:41:07.560" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24836" PostTypeId="5" CreationDate="2020-11-24T02:37:52.350" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-24T02:37:52.350" LastActivityDate="2020-11-24T02:37:52.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24837" PostTypeId="4" CreationDate="2020-11-24T02:37:52.350" Score="0" Body="For questions that are part of homework where you need some help. Please, describe what you have tried so far and don't just ask for the solution." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-24T02:37:52.350" LastActivityDate="2020-11-24T02:37:52.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24840" PostTypeId="2" ParentId="24816" CreationDate="2020-11-24T12:59:15.527" Score="3" Body="&lt;p&gt;Based on &lt;a href=&quot;http://ai.berkeley.edu/exams/sp14_midterm1_solutions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://adp.princeton.edu/Papers/Powell%20-%20Perspectives%20of%20approximate%20dynamic%20programming.pdf#page=20&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; resources, let me give an answer to my own question, but, essentially, I will just rewrite the contents of the first resource here, for reproducibility, with some minor changes to the notation (to be consistent with Sutton &amp;amp; Barto's book, 2nd edition). Note that I am not fully sure if this formulation is universal (i.e. maybe there are other ways of formulating it), but the contents of the first resource seem to be consistent with the contents in &lt;a href=&quot;http://adp.princeton.edu/Papers/Powell%20-%20Perspectives%20of%20approximate%20dynamic%20programming.pdf#page=20&quot; rel=&quot;nofollow noreferrer&quot;&gt;the second resource&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Setup&lt;/h3&gt;&#xA;&lt;p&gt;Let's assume that we have an infinite-horizon MDP&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{M} = (\mathcal{S}, \mathcal{Y}, \mathcal{A}, \mathcal{T}, \mathcal{R}, \gamma),$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}$&lt;/span&gt; is the set of states&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y} \subseteq \mathcal{S}$&lt;/span&gt; is the set of &lt;em&gt;afterstates&lt;/em&gt; (aka &lt;em&gt;post-decision states&lt;/em&gt; or &amp;quot;end of period&amp;quot; states [&lt;a href=&quot;http://adp.princeton.edu/Papers/Powell%20-%20Perspectives%20of%20approximate%20dynamic%20programming.pdf#page=20&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], which can also be written as &lt;em&gt;after-states&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is the set of actions&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{T}$&lt;/span&gt; is the transition function&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; is the reward function&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is a discount factor&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y \in \mathcal{Y}$&lt;/span&gt; be an afterstate&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f: \mathcal{S} \times \mathcal{A} \rightarrow \mathcal{Y}$&lt;/span&gt; be a &lt;em&gt;deterministic&lt;/em&gt; function (from state-action pairs to afterstates), so we have &lt;span class=&quot;math-container&quot;&gt;$f(s, a) = y$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The transition function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{T}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}$&lt;/span&gt; is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathcal{T}(s, a, s^{\prime}) &#xA;&amp;amp;\doteq P ( s^{\prime} \mid f(s, a)) \\&#xA;&amp;amp;= P ( s^{\prime} \mid y)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;A transition is composed of 2 steps&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a deterministic step, where we apply the deterministic function &lt;span class=&quot;math-container&quot;&gt;$f(s, a) = y$&lt;/span&gt;, which depends on an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; taken in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, followed by&lt;/li&gt;&#xA;&lt;li&gt;a stochastic step, where we apply the probability distribution &lt;span class=&quot;math-container&quot;&gt;$P (s^{\prime} \mid y)$&lt;/span&gt;, which does not depend on the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; anymore, but only on &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, I have denoted afterstates with a different letter, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, because afterstates are reached with a deterministic function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, while other states, &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, are reached with &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;After having taken the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, we get a reward (i.e. we get a reward in step 1), but we do not get a reward after the stochastic step (given that no action is taken).&lt;/p&gt;&#xA;&lt;p&gt;So, we can define the reward function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; for this MDP as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal{R} (s, a, s^{\prime} ) \doteq \mathcal{R}(s, a)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The situation is illustrated by the following diagram&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fi2on.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fi2on.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, here, &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is the stochastic transition function (i.e. a probability distribution) as used above. Note that, here, &lt;span class=&quot;math-container&quot;&gt;$r_t$&lt;/span&gt; is a specific realization of &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt; (the random variable) in the formulas below.&lt;/p&gt;&#xA;&lt;h3&gt;State value function&lt;/h3&gt;&#xA;&lt;p&gt;Let's recall the definition of the &lt;strong&gt;state value function&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s)$&lt;/span&gt; for a given policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; (as defined in Sutton &amp;amp; Barto, section 3.5)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;v_{\pi}(s) &#xA;&amp;amp;\doteq &#xA;\mathbb{E}_{\pi}\left[G_{t} \mid S_{t}=s\right] \\&#xA;&amp;amp;=&#xA;\mathbb{E}_{\pi}\left[\sum_{k=0}^{\infty} \gamma^{k} R_{t+k+1} \mid S_{t}=s\right],&#xA;\end{align}&lt;/span&gt;&#xA;for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt; and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;G_{t} &#xA;&amp;amp;\doteq &#xA;\sum_{k=0}^{\infty} \gamma^{k} R_{t+k+1} \\&#xA;&amp;amp;=&#xA;R_{t+1} + \gamma R_{t+2} + \gamma^{2} R_{t+3}+ \cdots \\&#xA;&amp;amp;=&#xA;\mathcal{R}(s_t, a_t) + \gamma \mathcal{R}(s_{t+1}, a_{t+1})+\gamma^{2} \mathcal{R}(s_{t+2}, a_{t+2}) +\cdots,&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\pi(s_t) = a_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(s_t, a_t) = R_{t+1}$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$t=0, 1, 2, \dots$&lt;/span&gt;. (So, note that &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R} \neq R_t$&lt;/span&gt;: the first is the reward function, while the second is a random variable that represents the reward received after having taken action &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; in step &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;optimal state value function&lt;/strong&gt; is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{*}(s) \doteq \max _{\pi} v_{\pi}(s)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Afterstate value function&lt;/h3&gt;&#xA;&lt;p&gt;Similarly, we will define the afterstate value function, but we will use the letter &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; just to differentiate it from &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;w_{\pi}\left(y\right)&#xA;&amp;amp;\doteq &#xA;\mathbb{E}_{\pi}\left[G_{t+1} \mid Y_{t}=y\right] &#xA;\\&#xA;&amp;amp;=&#xA;\mathbb{E}_{\pi}\left[\sum_{k=0}^{\infty} \gamma^{k} R_{t+k+2} \mid Y_{t}=y\right] &#xA;\\&#xA;&amp;amp;=&#xA;\mathbb{E}_{\pi}\left[&#xA;R_{t+2} + \gamma R_{t+3}+\gamma^{2} R_{t+4} + \cdots&#xA; \mid Y_{t}=y\right] &#xA;\\&#xA;&amp;amp;=&#xA;\mathbb{E}_{\pi}\left[&#xA;\mathcal{R}(s_{t+1}, a_{t+1})+\gamma \mathcal{R}(s_{t+2}, a_{t+2}) + \gamma^{2} \mathcal{R}(s_{t+3}, a_{t+3}) + \cdots&#xA; \mid Y_{t}=y\right] ,&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(s_{t+1}, a_{t+1}) = R_{t+2}$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the value of an afterstate &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, i.e. given &lt;span class=&quot;math-container&quot;&gt;$Y_t = y$&lt;/span&gt;) is defined as the expectation of the return starting from the state that you ended up in &lt;em&gt;after&lt;/em&gt; the afterstate &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This seems reasonable to me and is similar to my proposal for the definition of the afterstate value function in the question, although I was not considering any deterministic functions in a potential formulation, and I was also not thinking of afterstates as &lt;em&gt;intermediate&lt;/em&gt; states, reached by a deterministic step, between the usual states.&lt;/p&gt;&#xA;&lt;p&gt;Similarly to the optimal state value function, we also define the &lt;strong&gt;optimal afterstate value function&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{*}(y) \doteq \max _{\pi} w_{\pi}(y)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Afterstate value function defined in terms of the state value function&lt;/h3&gt;&#xA;&lt;p&gt;We can define the afterstate value function in terms&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{*}(y) &#xA;= &#xA;\sum_{s^{\prime}} P (s^{\prime} \mid y ) v_{*} ( s^{\prime} )&#xA;$$&lt;/span&gt;&#xA;In other words, &lt;span class=&quot;math-container&quot;&gt;$w_{*}(y)$&lt;/span&gt; is defined as an expectation over the value of next possible states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; from the afterstate &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This seems to be correct and consistent with the above definitions.&lt;/p&gt;&#xA;&lt;h3&gt;More equations&lt;/h3&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;http://ai.berkeley.edu/exams/sp14_midterm1_solutions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://adp.princeton.edu/Papers/Powell%20-%20Perspectives%20of%20approximate%20dynamic%20programming.pdf#page=20&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; resources, the &lt;strong&gt;state value function is also defined in terms of afterstate value function&lt;/strong&gt; as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{*}(s)=\max_{a}\left(\mathcal{R}(s, a)+\gamma w_{*}(f(s, a))\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;Bellman equation for afterstate value function&lt;/strong&gt; (from which an update rule can be derived) is given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{*}(y)&#xA;=&#xA;\sum_{s^{\prime}} &#xA;P(s^{\prime} \mid y ) &#xA;\max_{a} ( &#xA;\mathcal{R} (s^{\prime}, a)&#xA;+&#xA;\gamma &#xA;w_{*}(f ( s^{\prime}, a ))),&#xA;$$&lt;/span&gt;&#xA;which is really similar to the Bellman equation for the state value function.&lt;/p&gt;&#xA;&lt;p&gt;Finally, we can also &lt;strong&gt;express the state-action value function in terms of the afterstate value function&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;q_\pi(s_t, a_t)&#xA;=&#xA;\mathcal{R}\left(s_{t}, a_{t}\right)+\gamma w_{\pi}\left(f\left(s_{t}, a_{t}\right)\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that this answer is already quite long, see the resource for more details (including an algorithm based on the afterstate Bellman equation).&lt;/p&gt;&#xA;&lt;h3&gt;Implementation&lt;/h3&gt;&#xA;&lt;p&gt;If you are the type of person that understands the concepts by looking at the code, then &lt;a href=&quot;https://github.com/cpita/TicTacToeAI&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Github project&lt;/a&gt;, which implements a Monte Carlo method that uses afterstates to play tic-tac-toe, may be useful. Afterstates are useful in tic-tac-toe because it is a 2-player game, where two agents take actions in turn, so we can estimate the action that you should take deterministically (as if it was the &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; above) before the other agent takes an action (probabilistically), at least, this is my current interpretation of the usefulness of afterstates in this game (and similar games/problems).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-17T14:37:36.783" LastActivityDate="2021-02-17T14:37:36.783" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24841" PostTypeId="1" CreationDate="2020-11-24T13:56:16.793" Score="0" ViewCount="29" Body="&lt;p&gt;I have a system that takes 32 inputs (all of which are 1 or 0) and generates 32 outputs (all of which are complex numbers that lie roughly in the range of (0,2)). The response of this system to its inputs are linear (meaning that its response is equivalent to multiplying with a 32*32 complex matrix).&lt;/p&gt;&#xA;&lt;p&gt;This system is well-defined by 256 &amp;quot;settings&amp;quot;, all of which real numbers in the range (0,1). But its properties are non-linear with the settings, meaning that the response matrix mentioned would not change linearly according to these settings.&lt;/p&gt;&#xA;&lt;p&gt;My goal is to use some kind of neural network to predict this response matrix using the settings as the only input.&lt;/p&gt;&#xA;&lt;p&gt;The dataset consists of 50000 (can be extended if necessary) input-output-settings tuples, which of course doesn't allow me to compute the target response matrix exactly. Moreover, input patterns come in a sparse form, meaning that, for most of the time, input vectors are mostly zeroes with a small number of ones.&lt;/p&gt;&#xA;&lt;p&gt;My approach was to use ComplexCNN with sigmoid to generate the response matrix, then multiplying it with the input vectors to get a predicted output, comparing it with those from the dataset, and computing a loss (using basic &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; loss). I implemented and trained the model using PyTorch, but results came out very badly, as the input, output, and matrix are sparse, causing gradient problems, which often lead to the neural network to give out &lt;code&gt;nan&lt;/code&gt; as output.&lt;/p&gt;&#xA;&lt;p&gt;So, might there be any way to change the network architecture to adapt to this type of data? How can I solve the problem of non-converging gradients?&lt;/p&gt;&#xA;" OwnerUserId="42507" LastEditorUserId="2444" LastEditDate="2020-11-28T22:24:45.610" LastActivityDate="2020-11-28T22:24:45.610" Title="How to generate a matrix out of sparse data?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;data-preprocessing&gt;&lt;structured-data&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24842" PostTypeId="2" ParentId="24702" CreationDate="2020-11-24T14:53:15.797" Score="0" Body="&lt;p&gt;The accepted answer does not answer the question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why, if the above is correct, do we not see many neural network architectures projecting the data into higher dimensions first then reducing the size of each layer thereafter?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, it's true that if you increase the number of hidden neurons, you generally increase the capacity (in fact, &lt;a href=&quot;https://ai.stackexchange.com/a/17881/2444&quot;&gt;the VC dimension of neural networks is typically expressed as a function of the number of parameters&lt;/a&gt;), but you're also suggesting to &lt;em&gt;decrease&lt;/em&gt; the size afterwards.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;https://www.tensorflow.org/swift/tutorials/model_training_walkthrough?hl=zh-cn&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this tutorial&lt;/a&gt;, they use 10 hidden neurons for the first hidden layer of an MLP, while the dataset contains only 4 features, which means that there are &lt;span class=&quot;math-container&quot;&gt;$4*10 = 40$&lt;/span&gt; weights (aka parameters). In &lt;a href=&quot;https://www.tensorflow.org/tutorials/structured_data/feature_columns&quot; rel=&quot;nofollow noreferrer&quot;&gt;this other tutorial&lt;/a&gt;, there are less than 20 features (and only a few of them are used) and the first hidden layer has 128 neurons, which means that each feature is connected to &lt;span class=&quot;math-container&quot;&gt;$128$&lt;/span&gt; hidden neurons, so there should be &lt;span class=&quot;math-container&quot;&gt;$4*128 = 512$&lt;/span&gt; weights. So, MLPs can easily have more hidden neurons in the first layer and weights that connect the input(s) to the hidden neurons than the number of features.&lt;/p&gt;&#xA;&lt;p&gt;In the case of CNNs, you may have fewer scalar parameters because of the properties of CNNs, i.e. parameter sharing. For instance, if you have &lt;span class=&quot;math-container&quot;&gt;$64$&lt;/span&gt; filters of shape &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; and you want to process grayscale images of shape &lt;span class=&quot;math-container&quot;&gt;$32 \times 32$&lt;/span&gt;, then you have &lt;span class=&quot;math-container&quot;&gt;$3 * 3 * 64 = 576  &amp;lt; 1024 = 32*32$&lt;/span&gt;. Note that the number of parameters of the first layer does not change as a function of the input in the case of CNNs, so if that same CNN processes an image of size &lt;span class=&quot;math-container&quot;&gt;$5 \times 5$&lt;/span&gt;, then the first layer contains more scalar parameters than pixels.&lt;/p&gt;&#xA;&lt;p&gt;So, in general, NNs can project the data to higher-dimensional space. The typical NNs that project to a lower-dimensional space are auto-encoders or data compressors, in general, that's why they are called in this way.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-24T15:41:34.017" LastActivityDate="2020-11-24T15:41:34.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24843" PostTypeId="2" ParentId="24804" CreationDate="2020-11-24T15:39:14.460" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;are the LSTM cells on a given row all the same cell, with time flowing forward from left to right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes this is correct&lt;/p&gt;&#xA;&lt;p&gt;The x-axis on this figure is basically the &lt;strong&gt;time&lt;/strong&gt; axis. Essentially all pink boxes in the same row are &lt;strong&gt;the same&lt;/strong&gt; LSTM cell, with different inputs from the same sequence. At each timestep, the cell takes an input and produces an output which is fed to the next layer. At the 8-th layer, the outputs over all timesteps are inputted at the same time to the attention layer.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2020-11-24T15:39:14.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24844" PostTypeId="1" AcceptedAnswerId="24845" CreationDate="2020-11-24T15:46:17.280" Score="1" ViewCount="51" Body="&lt;p&gt;I would like some guidance on how to design an Environment for a Reinforcement Learning agent where the stopping conditions and rewards for the environment change based on an initial set of input parameters.&lt;/p&gt;&#xA;&lt;p&gt;For example, let's say that a system generated alert triggers the instantiation of the RL environment, whereby the RL agent is launched to make decisions in the environment, based on the alert. The alert has two priorities &amp;quot;HIGH&amp;quot; and &amp;quot;LOW&amp;quot;, when the priority is &amp;quot;HIGH&amp;quot; the stopping condition is a reward of &amp;quot;100&amp;quot; and when the priority is &amp;quot;LOW&amp;quot;, the stopping condition is a reward of &amp;quot;1000&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In this scenario, is it preferable to create two separate environments based on the priority (input parameter) of the alert? Or is this a common requirement that should be designed into the environment/agent? If so, how? Note that I have simplified the scenario, so there could be multiple conditions (e.g., alert, system type, etc), but I am just trying to find a basic solution for the general case.&lt;/p&gt;&#xA;" OwnerUserId="42508" LastEditorUserId="2444" LastEditDate="2020-11-25T09:10:32.263" LastActivityDate="2020-11-25T09:10:32.263" Title="If the reward function of an environment depends on some initial conditions, should I create a separate environment for each condition?" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24845" PostTypeId="2" ParentId="24844" CreationDate="2020-11-24T17:32:23.470" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In this scenario, is it preferable to create two separate environments based on the priority (input parameter) of the alert?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is difficult to make a hard rule here.&lt;/p&gt;&#xA;&lt;p&gt;If the resulting environments can be cleanly sorted into a few different categories, and the ideal behaviour and/or the states visited are radically different in each scenario, then maybe a few different agents optimised for each scenario could work well.&lt;/p&gt;&#xA;&lt;p&gt;A more general approach however, is to include the episode start data as part of the state that the agent observes on each time step. A single agent can then in theory learn the different behaviours required depending on the initial values, plus still generalise from anything shared between the multiple scenarios.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The alert has two priorities &amp;quot;HIGH&amp;quot; and &amp;quot;LOW&amp;quot;, when the priority is &amp;quot;HIGH&amp;quot; the stopping condition is a reward of &amp;quot;100&amp;quot; and when the priority is &amp;quot;LOW&amp;quot;, the stopping condition is a reward of &amp;quot;1000&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This may work against you. RL agents do not respond to the absolute values of rewards, other than how they compare to other rewards also available within the same episode (or continuing environment).&lt;/p&gt;&#xA;&lt;p&gt;If there is only ever one issue to solve at a time, and no conflict between solving either of the &amp;quot;HIGH&amp;quot; or &amp;quot;LOW&amp;quot; priority problems (such as splittig resources or effort between them), the different reward system seems redundant. Solved is solved. You might rate the &lt;em&gt;usefulness&lt;/em&gt; of an agent that solves the &amp;quot;LOW&amp;quot; priority issue well higher, but it seems to me that this describes what you should work on first, not the goals of the agent. To influence the goals of the agent, &lt;em&gt;both&lt;/em&gt; rewards would need to be available within the same episode or continuing environment, requiring the agent to make a choice between them.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-11-24T17:32:23.470" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24846" PostTypeId="1" CreationDate="2020-11-25T00:21:31.720" Score="0" ViewCount="21" Body="&lt;p&gt;I'm training a neural network to solve a maze.&lt;/p&gt;&#xA;&lt;p&gt;My process is the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Randomly generate a small maze&lt;/li&gt;&#xA;&lt;li&gt;Spawn hundreds of cars at the start, that will go through the same one maze&lt;/li&gt;&#xA;&lt;li&gt;Assign the same base neural network to each car, this would be the best from previous generation&lt;/li&gt;&#xA;&lt;li&gt;Do the mutations: for each neural network of those cars, slightly and randomly modify all the weights&lt;/li&gt;&#xA;&lt;li&gt;Run the simulation and wait 60 seconds&lt;/li&gt;&#xA;&lt;li&gt;Run the algorithm to determine the best generation&lt;/li&gt;&#xA;&lt;li&gt;Repeat&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Right now, to determine the best generation, I'm using the car that has the shortest path to the end.&lt;/p&gt;&#xA;&lt;p&gt;But my neural network isn't learning, that is, no car ever reaches the end for all randomly generated maze.&lt;/p&gt;&#xA;&lt;p&gt;I believe that I would need to include some measurement of the dead-end and/or backtracking, but don't know exactly what to do.&lt;/p&gt;&#xA;&lt;p&gt;What would be the best / correct criteria for determining the best generation?&lt;/p&gt;&#xA;" OwnerUserId="42513" LastEditorUserId="2444" LastEditDate="2020-11-25T09:17:04.413" LastActivityDate="2020-11-25T09:17:04.413" Title="What would be the best criteria for determining the best generation on a maze training by mutation?" Tags="&lt;neural-networks&gt;&lt;genetic-algorithms&gt;&lt;learning-algorithms&gt;&lt;mutation-operators&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24847" PostTypeId="1" CreationDate="2020-11-25T03:58:45.273" Score="1" ViewCount="25" Body="&lt;p&gt;I'm trying to implement the minimax algorithm with alpha beta pruning on a game that works like this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Player 1 plays (x1, y1).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Player 2 can only see the x-value (x1) that Player 1 played (and not the y-value y1). Player 2 plays (x2, y2).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An action event happens, which may change the heuristic of the current game state.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Player 2 plays (x3, y3).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Player 1 can only see the x-value (x3) that Player 2 played (and not the y-value y3). Player 1 plays (x4, y4).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Action event. The game continues with alternating starting players for a maximum depth of 10.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To do so, I have been treating each turn as you regularly would with the minimax algorithm, with each player making moves given the set of moves already played, including the possibly hidden move from the turn before. However, I've noticed that my algorithm will return moves for Player 2 that assume that Player 1 plays a certain way when, in a &amp;quot;real&amp;quot; game, it may be the case that Player 1 plays something else (and vice versa). &lt;strong&gt;For example, if Player 2 could guarantee a win on a given turn under all circumstances (when Player 1 plays first for that series), it might not play optimally when it assumes Player 1 will not play its maximum-strength move.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I believe it is doing this precisely because it assumes that all moves are visible (a fully visible game state). And indeed, if that were the case, the sequence of moves it returns would be optimal.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I remedy this?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I do not believe that a probability-based algorithm (e.g. Expectiminimax) is necessary, since the game is entirely deterministic. The partial visibility part is making things difficult, though.&lt;/p&gt;&#xA;&lt;p&gt;Something tells me that changing the turn order in my algorithm might be a solution to this problem, since the action event is the only time the game heuristic is changed.&lt;/p&gt;&#xA;" OwnerUserId="42518" LastActivityDate="2020-11-25T03:58:45.273" Title="Minimax algorithm with only partial visibility" Tags="&lt;algorithm&gt;&lt;minimax&gt;&lt;game-theory&gt;&lt;alpha-beta-pruning&gt;&lt;optimality&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24848" PostTypeId="1" AcceptedAnswerId="24871" CreationDate="2020-11-25T05:40:40.663" Score="3" ViewCount="69" Body="&lt;p&gt;I am currently studying the paper &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.170&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning and Evaluating Classifiers under Sample Selection Bias&lt;/em&gt; by Bianca Zadrozny&lt;/a&gt;. In the introduction, the author says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;One of the most common assumptions in the design of learning algorithms is that the training data consist of examples drawn independently from the same underlying distribution as the examples about which the model is expected to make predictions. In many real-world applications, however, this assumption is violated because we do not have complete control over the data gathering process.&lt;/p&gt;&#xA;&lt;p&gt;For example, suppose we are using a learning method to induce a model that predicts the side-effects of a treatment for a given patient. Because the treatment is not given randomly to individuals in the general population, the available examples are not a random sample from the population. Similarly, suppose we are learning a model to predict the presence/absence of an animal species given the characteristics of a geographical location. Since data gathering is easier in certain regions than others, we would expect to have more data about certain regions than others.&lt;/p&gt;&#xA;&lt;p&gt;In both cases, even though the available examples are not a random sample from the true underlying distribution of examples, we would like to learn a predictor from the examples that is as accurate as possible for this distribution. Furthermore, we would like to be able to estimate its accuracy for the whole population using the available data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's this part that I am confused about:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In both cases, even though the available examples are not a random sample from the true underlying distribution of examples, we would like to learn a predictor from the examples that is as accurate as possible for this distribution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What exactly is &amp;quot;this distribution&amp;quot;? Is it referring to the true underlying distribution, or the distribution of our sample (which, as was said, is not necessarily a &amp;quot;good&amp;quot; reflection of the underlying distribution, since it is not a random sample)?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastActivityDate="2020-11-26T20:40:11.503" Title="Is this referring to the true underlying distribution, or the distribution of our sample?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;probability-distribution&gt;&lt;selection-bias&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24849" PostTypeId="1" CreationDate="2020-11-25T08:37:00.163" Score="1" ViewCount="51" Body="&lt;p&gt;What is the impact of using a:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;low crossover rate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;high crossover rate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;low mutation rate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;high mutation rate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="41713" LastEditorUserId="2444" LastEditDate="2020-11-28T03:10:11.490" LastActivityDate="2020-11-28T03:10:11.490" Title="What is the impact of changing the crossover and mutation rates?" Tags="&lt;genetic-algorithms&gt;&lt;hyperparameter-optimization&gt;&lt;crossover-operators&gt;&lt;mutation-operators&gt;&lt;genetic-operators&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24850" PostTypeId="1" CreationDate="2020-11-25T10:12:35.870" Score="0" ViewCount="37" Body="&lt;p&gt;I am trying to implement AlexNet in Tensorflow. Matlab has a built-in AlexNet implementation that I use with the cats and dogs dataset (mledu-datasets/cats_and_dogs_filtered.zip). In Matlab, everything works and both training and evaluation loss goes down as epochs go up. However in my TensorFlow implementation validation loss never goes down and gets stuck around 0.6.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vSCDB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vSCDB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Implementation in Matlab is as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YruuB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YruuB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And here is my implementation in TF.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = keras.models.Sequential([&#xA;  keras.layers.experimental.preprocessing.Rescaling(1./255, input_shape=(img_height, img_width, 3)),&#xA;  # Conv 1&#xA;  keras.layers.Conv2D(filters=96, kernel_size=(11,11), strides=(4,4), activation='relu',&#xA;                      #input_shape=(img_height, img_width, 3)&#xA;                      ),&#xA;  #keras.layers.BatchNormalization(),&#xA;  keras.layers.MaxPool2D(pool_size=(3,3), strides=(2,2)),&#xA;  &#xA;  # Conv 2&#xA;  keras.layers.Conv2D(filters=128, groups=2, kernel_size=(5,5), strides=(1,1), activation='relu', padding=&amp;quot;same&amp;quot;),&#xA;  keras.layers.BatchNormalization(),&#xA;  keras.layers.MaxPool2D(pool_size=(3,3), strides=(2,2)),&#xA;  # Conv 3&#xA;  keras.layers.Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), activation='relu', padding=&amp;quot;same&amp;quot;),&#xA;  keras.layers.BatchNormalization(),&#xA;  # Conv 4&#xA;  keras.layers.Conv2D(filters=192, groups=2, kernel_size=(1,1), strides=(1,1), activation='relu', padding=&amp;quot;same&amp;quot;),&#xA;  keras.layers.BatchNormalization(),&#xA;  # Conv 5&#xA;  keras.layers.Conv2D(filters=128, groups=2, kernel_size=(1,1), strides=(1,1), activation='relu', padding=&amp;quot;same&amp;quot;),&#xA;  keras.layers.BatchNormalization(),&#xA;  keras.layers.MaxPool2D(pool_size=(3,3), strides=(2,2)),&#xA;  &#xA;  # FC&#xA;  keras.layers.Flatten(),&#xA;  keras.layers.Dense(4096, activation='relu'),&#xA;  keras.layers.Dropout(0.5),&#xA;  keras.layers.Dense(4096, activation='relu'),&#xA;  keras.layers.Dropout(0.5),&#xA;  keras.layers.Dense(len(class_names), activation='softmax')&#xA;])&#xA;&#xA;model.compile(optimizer=tf.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True),&#xA;              loss='categorical_crossentropy',&#xA;              metrics=['accuracy'])&#xA;&#xA;history = model.fit(train_ds,&#xA;                    validation_data=val_ds,&#xA;                    epochs=epochs,&#xA;                    batch_size=batch_size,&#xA;                    shuffle = True,&#xA;                    verbose=0, callbacks=[plot_losses])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using the same optimizer and learning rate for both. My question is, is the problem has to do with the group convolutions or something else? Everything else seems to the be same. No matter what I try I can not make val_loss go down.&lt;/p&gt;&#xA;" OwnerUserId="19186" LastEditorUserId="2444" LastEditDate="2020-11-25T11:03:36.157" LastActivityDate="2020-11-25T11:03:36.157" Title="Why does the validation loss not go down when training my TensorFlow implementation of AlexNet?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;matlab&gt;&lt;alexnet&gt;&lt;validation-loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24851" PostTypeId="1" AcceptedAnswerId="24865" CreationDate="2020-11-25T12:14:54.177" Score="0" ViewCount="54" Body="&lt;p&gt;I would be grateful for some guidance on a RL problem I am trying to solve where multiple RL agents use a common/global policy at the initial state of an episode in the RL Environment, and then update this common/shared policy once the episode is completed.&lt;/p&gt;&#xA;&lt;p&gt;Below is an example of the problem scenario:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An alert triggers a RL agent to execute a &amp;quot;episode&amp;quot; in the Environment&lt;/li&gt;&#xA;&lt;li&gt;Multiple alerts (e.g., episodes) can occur at the same time, or, one alert may still be being processed (e.g., the episode has not finished) before another alert is triggered (e.g., another episode begins).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Below are the conditions of the Environment and desired behaviour of the RL Agent:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multiple episodes can run at once (e.g., another episode starts before another finishes).&lt;/li&gt;&#xA;&lt;li&gt;For each episode a &amp;quot;instance&amp;quot; of the RL agent uses the latest version of a common policy.&lt;/li&gt;&#xA;&lt;li&gt;After each episode the RL agent updates the common policy.&lt;/li&gt;&#xA;&lt;li&gt;Common policy updates are &amp;quot;queued&amp;quot; using versioning in code to prevent race conditions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Q: How can multiple RL agents in this case use a common policy at the beginning of an episode and then update a common policy after completing it? - All I have found are discussions related to Q-Learning, where agents can update a shared Q-table, or later update a &amp;quot;global&amp;quot; Q-table without any examples of how this can be achieved and whether there are also methods for other approaches such as TD rather than only Q-Learning, for &lt;a href=&quot;https://www.researchgate.net/post/How_can_I_solve_the_problem_of_multi-agent_learning_while_agents_interact_just_through_the_environment_by_using_a_single-agent_Q-Learning_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Q: Does this sound like a traditional multi-agent scenario, at least conceptually? If so, how might one go about implementing this, any examples would be really helpful.&lt;/p&gt;&#xA;&lt;p&gt;Any help on this is greatly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;Since doing more investigation I have found this reference on Mathworks:&#xA;&lt;a href=&quot;https://www.mathworks.com/help/reinforcement-learning/ug/train-dqn-agent-for-lane-keeping-assist-using-parallel-computing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;, which is similar to the above problem, but not exact.&lt;/p&gt;&#xA;" OwnerUserId="42508" LastEditorUserId="42508" LastEditDate="2020-11-25T18:26:14.990" LastActivityDate="2020-11-26T07:23:43.607" Title="How to use and update a shared/global policy between Reinforcement Learning Agents" Tags="&lt;reinforcement-learning&gt;&lt;environment&gt;&lt;policies&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24852" PostTypeId="1" CreationDate="2020-11-25T13:07:46.040" Score="1" ViewCount="21" Body="&lt;p&gt;I am trying to set up a neural network architecture that is able to learn the points of one function (blue curves) from the points of an other one (red curves). I think that it could be somehow similar to the problem of learning a functional like it was described in this question &lt;a href=&quot;https://stats.stackexchange.com/questions/158348/can-a-neural-network-learn-a-functional-and-its-functional-derivative&quot;&gt;here&lt;/a&gt;. I don't know at all what this (let's call it) functional looks like, I just see the 'blue' response of it to the 'red' input.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SvfNK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SvfNK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The inputs of the network would be the (e.g. 100) points of a red curve and the outputs would probably be the (e.g. 50) points of the blue curve. This is where my problem begins. It tried to implement a simple dense network with two hidden layers and around 200-300 neurons each. Obviously it didn't learn much.&lt;/p&gt;&#xA;&lt;p&gt;I have the feeling that I somehow need to tell the network that the points next to each other (e.g. input points &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;) are correlated and the function they belong to is differentiable. For the inputs this could be archieved by using convolutional layers, I suppose. But I don't really now how to specify, that the output nodes are correlated with each other as well.&lt;/p&gt;&#xA;&lt;p&gt;At the beginning I had high hopes in the approach using Generalized Regression Neural Networks as presented &lt;a href=&quot;http://article.nadiapub.com/IJSIP/vol7_no2/8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where a lowpass filter is implemented with NNs. However, as I understood, only the filter coefficients are predicted. As I don't know anything about the general structure of my functional, this will not help me here ...&lt;/p&gt;&#xA;&lt;p&gt;Do you have any other suggestions for NN architectures that could be helpful for this problem? Any hint is appreciated, thank you!&lt;/p&gt;&#xA;" OwnerUserId="42323" LastActivityDate="2020-11-25T13:07:46.040" Title="Neural network architecture with inputs and outputs being an unkown function each" Tags="&lt;neural-networks&gt;&lt;regression&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24854" PostTypeId="1" CreationDate="2020-11-25T19:51:14.640" Score="0" ViewCount="126" Body="&lt;p&gt;I have a set of users that can be one of 3 types.&lt;/p&gt;&#xA;&lt;p&gt;They will randomly request a service from the UAV which is a drone used as a Base Station.&lt;/p&gt;&#xA;&lt;p&gt;The UAV (the agent) is tasked with allocating subchannels (a resource type) and power values (another resource type) to the different users, all while choosing a new position at every step to maximize the power values (I use a mathematical equation in my code to calculate that).&lt;/p&gt;&#xA;&lt;p&gt;A penalty is returned in case a user didn't request a service but was allocated a resource, and a reward is returned otherwise.&lt;/p&gt;&#xA;&lt;p&gt;The penalty's value varies depending on the number of the users that were allocated resources they didn't ask for.&lt;/p&gt;&#xA;&lt;p&gt;The state is defined as a combination of a matrix of 0s and 1s, and the agent's (UAV) position (The agent is placed in a 2D grid).&lt;/p&gt;&#xA;&lt;p&gt;The rows of the matrix represent the users, and the columns represent the type of users (3 types).&lt;/p&gt;&#xA;&lt;p&gt;If a &lt;code&gt;row == (0, 0, 0)&lt;/code&gt;, it means that the user at that row is not requesting any services.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;code&gt;row == (1, 0, 0)&lt;/code&gt; or &lt;code&gt;row == (0, 1, 0)&lt;/code&gt; or &lt;code&gt;row == (0, 0, 1)&lt;/code&gt;, it means that the user is requesting a service from the UAV. (There can be at most one column with the value of 1 in each row, we cannot have, for example, &lt;code&gt;row == (1, 1, 0)&lt;/code&gt; or &lt;code&gt;(1, 1, 1)&lt;/code&gt; etc)&lt;/p&gt;&#xA;&lt;p&gt;The action space is defined as a combination of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Another matrix of 0s (or rather the letter 'Z') and 1s (or rather the letter 'O'), with the rows representing the first type of resources (subchannels) that the UAV has, and the columns represent the users.&lt;/li&gt;&#xA;&lt;li&gt;The agent's chosen movement (Up, Down, Left or Right).&lt;/li&gt;&#xA;&lt;li&gt;And a vector with the different power values that were assigned to the users, hence the vector's length = number of users.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As for the matrix, suppose we have 5 users and 10 subchannels, if subchannel 0 was allocated to User0, then the first row (&lt;code&gt;row[0]&lt;/code&gt;) of the matrix would look like this: &lt;code&gt;(O, Z, Z, Z, Z)&lt;/code&gt;&#xA;(Each subchannel can be allocated to one user only, and each user can be allocated one subchannel only as well at every time step / So we have one 'O' only in each row and each column)&lt;/p&gt;&#xA;&lt;p&gt;I have created a functioning Q-Learning Agent, taking a very small number of users, subchannels, power values and grid dimensions.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Grid rows = 5&#xA;Grid columns = 5&#xA;number of subchannels = 4&#xA;number of users = 3 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, regarding the Q-table, we have &lt;code&gt;Q[state, action]&lt;/code&gt;, with state representing each row of the state space, and the action is each possible action that can be taken when we have that state.&lt;/p&gt;&#xA;&lt;p&gt;The state being every possible combination of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All the possible positions the UAV can take.&lt;/li&gt;&#xA;&lt;li&gt;And all the possible state matrices that we can have.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The action being every possible combination of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All the possible movement action chosen by the UAV.&lt;/li&gt;&#xA;&lt;li&gt;All the possible subchannel allocation matrices we can have.&lt;/li&gt;&#xA;&lt;li&gt;And all the possible power vectors with the values the UAV chose to give to the users.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So the Q-table is very huge already even with a very small sample.&lt;/p&gt;&#xA;&lt;p&gt;For example, suppose we have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;state = [ 5, (&#xA;  (0, 1, 0),&#xA;  (1, 0, 0),&#xA;  (0, 0, 0)&#xA; )&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;5 being the agent's position, and the 2D tuple is the matrix of requests. User0 and User1 are asking for a service, but User2 isn't.&lt;/p&gt;&#xA;&lt;p&gt;Then one entry of the Q-table would be:&#xA;&lt;code&gt;Q[state, (U, ((O, Z, Z), (Z, Z, Z), (Z, O, Z), (Z, Z, Z)), (12, 12, 0))] = value&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The second tuple is the matrix that represents the allocation of the subchannels.&#xA;We have 3 users, which means we have 3 columns.&lt;/p&gt;&#xA;&lt;p&gt;We have 4 subchannels, which means we have 4 rows.&lt;/p&gt;&#xA;&lt;p&gt;Z = means allocate zero to the user at that column, which means that the subchannel at that row was not allocated to the user at that column.&lt;/p&gt;&#xA;&lt;p&gt;O = means that the subchannel at that row was allocated to the user at that column. (so the value of that action is 1)&lt;/p&gt;&#xA;&lt;p&gt;U = the agent chose to go up so the value is (-1)&lt;/p&gt;&#xA;&lt;p&gt;The third tuple is the power vector, it's the power values allocated to the users.&lt;/p&gt;&#xA;&lt;p&gt;In this example, no penalty would be returned, because User0 and User1 who were allocated subchannels and non-zero power values, did actually ask for a service as we see in the requests matrix.&lt;/p&gt;&#xA;&lt;p&gt;Obviously, I cannot scale up my sample because I get MemoryError, so I have to use a DQN agent.&#xA;However, I'm having a hard time figuring out how to build my model architecture (Neural Networks and the like).&lt;/p&gt;&#xA;&lt;p&gt;I hope someone can guide me.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastActivityDate="2020-11-25T19:51:14.640" Title="How should I build this DQN agent?" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24856" PostTypeId="1" CreationDate="2020-11-25T22:18:09.333" Score="2" ViewCount="37" Body="&lt;p&gt;When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN.&lt;/p&gt;&#xA;&lt;p&gt;Lets say instead of using dense embeddings, I used a one-hot representation for each word, and fed that sequence into the RNN. My question is which of these two outcomes is correct:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Due to the way in which an RNN combines inputs, since these vectors are all orthogonal, absolutely nothing can be combined, and the entire setup does not make sense.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The setup does make sense and it will still work, but not be as effective as using a dense embedding.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I know I could run an experiment and see what happens, but this is fundamentally a theoretical question, and I would appreciate if someone could clarify so that I have a better understanding of how RNNs combine inputs. I suspect that the answer to this question would be the same regardless of whether we are discussing a vanilla RNN or an LSTM or  GRU, but if that is not the case, please explain why.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="12201" LastEditDate="2020-11-26T02:01:38.597" LastActivityDate="2020-11-26T02:01:38.597" Title="Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?" Tags="&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;word-embedding&gt;&lt;gated-recurrent-unit&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24857" PostTypeId="1" CreationDate="2020-11-26T00:29:10.030" Score="2" ViewCount="77" Body="&lt;p&gt;I am trying to understand Reinforcement Learning and already explored different Youtube videos, blog posts, and Wikipedia articles.&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand is the impact of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;. What value should it take? &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0.6$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$0.7$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;What does it mean when &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 1$&lt;/span&gt;? If &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 1$&lt;/span&gt;, does it mean that the agent explores randomly? If this intuition is right, then it will not learn anything - right? On the other hand, if I set &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0$&lt;/span&gt;, does this imply that the agent doesn't explore?&lt;/p&gt;&#xA;&lt;p&gt;For a typical problem, what is the recommended value for this parameter?&lt;/p&gt;&#xA;" OwnerUserId="41187" LastEditorUserId="2444" LastEditDate="2020-11-26T01:03:45.920" LastActivityDate="2020-11-26T19:02:48.503" Title="What should the value of epsilon be in the Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;hyperparameter-optimization&gt;&lt;hyper-parameters&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24859" PostTypeId="2" ParentId="12144" CreationDate="2020-11-26T02:06:51.080" Score="0" Body="&lt;p&gt;It appears that the rank based method would be slightly better in terms of time complexity because you sort only once for k number of sampling operations. This article (not mine) explains in detail clearly. &lt;a href=&quot;https://towardsdatascience.com/how-to-implement-prioritized-experience-replay-for-a-deep-q-network-a710beecd77b&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://towardsdatascience.com/how-to-implement-prioritized-experience-replay-for-a-deep-q-network-a710beecd77b&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21352" LastActivityDate="2020-11-26T02:06:51.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24860" PostTypeId="2" ParentId="24857" CreationDate="2020-11-26T02:09:05.350" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What does it mean when ϵ=0 and ϵ=1? If ϵ=1, does it mean that the agent explores randomly? If this intuition is right, then it will not learn anything - right? On the other hand, if I set ϵ=0, does this imply that the agent doesn't explore?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are correct, when  ϵ=1 the agent acts randomly. When  ϵ=0, the agent always takes the current greedy actions. Both of these scenarios are not ideal. Always acting greedily will prevent the agent from exploring possibly better parts of the  state space, and instead the agent may get stuck in a local optimum. And always exploring randomly is obviously not ideal as well. Thus, we need to balance between these two. This is often called the balance between exploration and exploitation.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For a typical problem, what is the recommended value for this parameter?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;ϵ is a hyper parameter. It is impossible to know in advance what the ideal value is, and it is highly dependent on the problem at hand. There is no general answer to this question.&lt;/p&gt;&#xA;&lt;p&gt;That being said, the most common values that I have seen typically range between 0.01 and 0.1. But I want to stress, there is no ideal value that works for all problems. A typical strategy is to try several values and see which one works best. For more information, you might want to look up hyper parameter tuning.&lt;/p&gt;&#xA;&lt;p&gt;Another common practice is slowly decaying epsilon over time (often this is called &amp;quot;annealing&amp;quot; or &amp;quot;simulated annealing&amp;quot;). Depending on the algorithm, decaying epsilon to zero may be a requirement for convergence. In some contexts, an algorithm that decays epsilon over time is called a GLIE algorithm. For example, see &lt;a href=&quot;https://ai.stackexchange.com/questions/6486/why-is-glie-monte-carlo-control-an-on-policy-control&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12201" LastEditorUserId="12201" LastEditDate="2020-11-26T19:02:48.503" LastActivityDate="2020-11-26T19:02:48.503" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24861" PostTypeId="1" CreationDate="2020-11-26T03:14:50.563" Score="3" ViewCount="93" Body="&lt;p&gt;I am self-studying about Reinforcement Learning using different online resources. I now have a basic understanding of how RL works.&lt;/p&gt;&#xA;&lt;p&gt;I saw this in &lt;a href=&quot;https://artint.info/2e/html/ArtInt2e.Ch12.S7.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a book&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Q-learning is an off-policy learner. An off-policy learner learns the value of an optimal policy independently of the agent’s actions, as long as it explores enough.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;An on-policy learner learns the value of the policy being carried out by the agent, including the exploration steps.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, I am not quite understanding the difference. Secondly, I came across that off-policy learner works better than the on-policy agent. I don't understand why that would be i.e. why off-policy would be better than the on-policy.&lt;/p&gt;&#xA;" OwnerUserId="41187" LastEditorUserId="2444" LastEditDate="2020-11-26T10:39:30.693" LastActivityDate="2020-11-28T02:01:13.920" Title="Why does off-policy learning outperform on-policy learning?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;off-policy-methods&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24862" PostTypeId="1" CreationDate="2020-11-26T04:51:28.870" Score="1" ViewCount="32" Body="&lt;p&gt;I have 2 photos, and my goal is to detect the face in one and place it on the face of the person in the other photo- basically face detection and replacement. It's not deep fakes. It's more of a computer vision-based approach for smoothing the edges and stitching.&lt;/p&gt;&#xA;&lt;p&gt;How can we achieve that? What are the approaches and techniques to do that? Some tutorials or code or github repos would be very helpful.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8x1nZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8x1nZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9053" LastEditorUserId="9053" LastEditDate="2020-11-26T05:04:06.603" LastActivityDate="2020-11-26T20:05:53.203" Title="Face detection and replacement in photos" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;object-detection&gt;&lt;image-processing&gt;&lt;face-recognition&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24863" PostTypeId="2" ParentId="24861" CreationDate="2020-11-26T05:23:44.453" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/q/184657/82135&quot;&gt;This post&lt;/a&gt; contains many answers that describe the difference between on-policy vs. off-policy.&lt;/p&gt;&#xA;&lt;p&gt;Your book may be referring to how the current (DQN-based) state-of-the-art (SOTA) algorithms, such as &lt;a href=&quot;https://openreview.net/pdf?id=H1Dy---0Z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ape-X&lt;/a&gt;, &lt;a href=&quot;https://openreview.net/pdf?id=r1lyTjAqYX&quot; rel=&quot;nofollow noreferrer&quot;&gt;R2D2&lt;/a&gt;, &lt;a href=&quot;https://deepmind.com/blog/article/Agent57-Outperforming-the-human-Atari-benchmark&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agent57&lt;/a&gt; are technically &amp;quot;off-policy&amp;quot;, since they use a (very large!) replay buffer, often filled in a distributed manner. This has a number of benefits, such as reusing experience and not forgetting important experiences.&lt;/p&gt;&#xA;&lt;p&gt;Another benefit is that you can collect a lot of experience distributedly. Since RL is typically not bottlenecked by the computation for training but rather from collecting experiences, the distributed replay buffer in Ape-X can enable much faster training, in terms of seconds but not sample complexity.&lt;/p&gt;&#xA;&lt;p&gt;However, it's important to emphasize that these replay-buffer approaches are &lt;strong&gt;almost on-policy&lt;/strong&gt;, in the sense that the replay buffer is constantly updated with new experiences. So, the policy in the replay buffer is &amp;quot;not too different&amp;quot; from your current policy (just a few gradient steps away). Most importantly, this allows the policy to learn from its own mistakes if it makes any...&lt;/p&gt;&#xA;&lt;p&gt;Off-policy learning, in general, can also refer to &lt;em&gt;batch RL&lt;/em&gt; (a.k.a. &lt;em&gt;offline RL&lt;/em&gt;), where you're provided a dataset of experiences from another behavior policy, and your goal is to improve over it. Notably, you don't get to rollout your current policy in any way! In this case, algorithms that worked well with a replay-buffer (like DQN, SAC) fail miserably, since they over-estimate the value of actions when they extrapolate outside the &amp;quot;support&amp;quot; of the dataset. See &lt;a href=&quot;https://arxiv.org/pdf/1812.02900.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the BCQ paper&lt;/a&gt; which illustrates how a lot of &amp;quot;off-policy&amp;quot; algorithms like DQN fail when the &amp;quot;distance between the two policies is large&amp;quot;. For this task, SOTA is a form of &lt;em&gt;weighted behavioral cloning&lt;/em&gt; called &lt;em&gt;Critic Regularized Regression (CRR)&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's also worth noting that &lt;a href=&quot;https://en.wikipedia.org/wiki/Importance_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;importance sampling&lt;/a&gt; can correct off-policy gradients to be on-policy; but the farther away your target policy is, the larger the variance. This is especially deadly for long horizon tasks (often called &lt;em&gt;curse of horizon&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;To summarize, using replay-buffer (which makes the algorithm off-policy), especially a distributed one, can offer a lot of benefits over pure on-policy algorithms. However, this is a very special class of off-policy algorithms, where the behavioral policy is close to your policy.&lt;/p&gt;&#xA;&lt;p&gt;But in general, off-policy is a lot harder than on-policy; you'll suffer from extrapolation bias if you use DQN-based approaches, and exponential variance blow-up if you use importance sampling to correct for it.&lt;/p&gt;&#xA;" OwnerUserId="36922" LastEditorUserId="2444" LastEditDate="2020-11-28T02:01:13.920" LastActivityDate="2020-11-28T02:01:13.920" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="24864" PostTypeId="2" ParentId="21398" CreationDate="2020-11-26T05:33:05.963" Score="2" Body="&lt;p&gt;Let's fix some notation: we're collecting data from behavior policy &lt;span class=&quot;math-container&quot;&gt;$\pi_0$&lt;/span&gt; and we want to evaluate a policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;.&#xA;Of course, if we had plenty of data from policy &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; that would be the best way to evaluate &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; as we just take the empirical average (without any importance sampling) and CLT gives us confidence intervals that shrink at &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{\sqrt n}$&lt;/span&gt; rates.&lt;/p&gt;&#xA;&lt;p&gt;However, collecting data from &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is often time-consuming and costly: you may need to productionize it at a company, and if &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; were dangerous, some damage could be done during rollouts. So how can we make the best use of our data from any policies, not necessarily &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, to evaluate &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;? This is the question of off-policy evaluation, and you're right that IS is one approach.&lt;/p&gt;&#xA;&lt;p&gt;This picture from a &lt;a href=&quot;https://www.youtube.com/watch?v=lzA5K4im2no&amp;amp;ab_channel=CriteoLabs&quot; rel=&quot;nofollow noreferrer&quot;&gt;great talk by Thorsten&lt;/a&gt; provides nice intuition on why the weighting is unbiased &lt;a href=&quot;https://i.stack.imgur.com/h770z.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h770z.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36922" LastActivityDate="2020-11-26T05:33:05.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24865" PostTypeId="2" ParentId="24851" CreationDate="2020-11-26T07:23:43.607" Score="0" Body="&lt;p&gt;This sounds like distributed RL, and most of the work goes in building the distributed system; the actual RL part is just a DQN (+some tricks from Rainbow DQN).&#xA;NB: multi-agent RL arises when the agents interact with each other in the same environment (like Hanabi the card game), while in this case we have multiple agents that collect experiences in parallel.&#xA;Here's a possible design from the &lt;a href=&quot;https://openreview.net/pdf?id=H1Dy---0Z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ape-X paper&lt;/a&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/a6zb7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a6zb7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36922" LastActivityDate="2020-11-26T07:23:43.607" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24866" PostTypeId="1" CreationDate="2020-11-26T14:39:25.667" Score="1" ViewCount="18" Body="&lt;p&gt;Is it possible to train an Object Detector (e.g. SSD), to detect when something is not in the image. Imagine an assembly line that transports some objects. Each object needs to have 5 screws. If the Object Detector detects 4 screws, we know that one is missing, hence there is an anomaly.&lt;/p&gt;&#xA;&lt;p&gt;Actually this is an Anomaly Detection task where there is something else than a screw (e.g. a hole), but unsupervised anomaly detectors are hard to train and not as stable as object detectors.&lt;/p&gt;&#xA;&lt;p&gt;Is my assumption correct, that even though it is not really an object detection task, one can use such methods?&lt;/p&gt;&#xA;" OwnerUserId="23063" LastActivityDate="2020-11-26T14:39:25.667" Title="Object Detection as a means of Anomaly Detection" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24867" PostTypeId="1" AcceptedAnswerId="24915" CreationDate="2020-11-26T16:26:22.347" Score="2" ViewCount="60" Body="&lt;p&gt;Given an alphabet &lt;span class=&quot;math-container&quot;&gt;$I=\left\{i_1,i_2,\dots,i_n\right\}$&lt;/span&gt; and a sequence &lt;span class=&quot;math-container&quot;&gt;$S=[e_1,e_2,\dots,e_m]$&lt;/span&gt;, where items &lt;span class=&quot;math-container&quot;&gt;$e_j \in I$&lt;/span&gt;, I am interested in finding every single pattern (subsequence of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;) that appears in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; more than &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; times, and that has a length &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; between two limits: &lt;span class=&quot;math-container&quot;&gt;$m&amp;lt;L&amp;lt;M$&lt;/span&gt;. In case of overlapping between patterns, only the longest pattern should be considered.&lt;/p&gt;&#xA;&lt;p&gt;For example, given &lt;span class=&quot;math-container&quot;&gt;$N=3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m=3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M=6$&lt;/span&gt;, an alphabet &lt;span class=&quot;math-container&quot;&gt;$I=\left\{A,B,C,D,E,F\right\}$&lt;/span&gt;, and the following sequence &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (the asterisks simply mark the positions of the patterns in this example):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;S=[A,A,*F,F,A,A*,B,D,E,F,E,D,*F,F,A,A*,F,C,*C,A,B,D,D*,C,C,*C,A,B,D,D*,C,A,C,B,E,A,B,C,*F,F,A,A*,E,A,B,C,A,D,E,*F,F,A,A*,B,C,D,A,E,A,B,*C,A,B,D,D*,]&#xA;$$&lt;/span&gt;&#xA;The sought algorithm should be able to return the following patterns:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;[C,A,B,D,D], [F,F,A,A]&#xA;$$&lt;/span&gt;&#xA;together with their respective positions within &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;An available Python implementation would be very desirable (and a parallel implementation, even more so).&lt;/p&gt;&#xA;&lt;p&gt;I was reading about &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.6.2704&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BIDE algorithms&lt;/a&gt;, but I think this is not the correct approach here. Any ideas? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="41986" LastEditorUserId="41986" LastEditDate="2020-11-26T16:43:29.840" LastActivityDate="2020-11-30T12:37:34.780" Title="Mining repeated subsequences in a given sequence" Tags="&lt;python&gt;&lt;pattern-recognition&gt;&lt;sequence-modelling&gt;&lt;data-mining&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24868" PostTypeId="1" AcceptedAnswerId="24869" CreationDate="2020-11-26T18:30:43.763" Score="1" ViewCount="43" Body="&lt;p&gt;I'm trying to wrap my mind around the concepts of &lt;strong&gt;regularisation&lt;/strong&gt; and &lt;strong&gt;optimisation&lt;/strong&gt; in neural nets, especially around their differences.&lt;/p&gt;&#xA;&lt;p&gt;In my current understanding, regularisation is intended to tackle overfitting whereas optimisation is about convergence.&lt;/p&gt;&#xA;&lt;p&gt;However, even though regularisation adds terms to the loss function, both approaches seem to do most of their things during the update phase, i.e. they work directly on how weights are updated.&lt;/p&gt;&#xA;&lt;p&gt;If both concepts are focused on updating weights,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;what are the conceptual differences, or why aren't both L2 and Adam, for example, called either optimisers or regularisers?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Can/should I use them together?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37735" LastEditorUserId="2444" LastEditDate="2020-11-26T18:34:59.077" LastActivityDate="2020-11-26T19:05:30.043" Title="What are the conceptual differences between regularisation and optimisation in deep neural nets?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;deep-neural-networks&gt;&lt;optimization&gt;&lt;regularization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24869" PostTypeId="2" ParentId="24868" CreationDate="2020-11-26T18:51:37.560" Score="2" Body="&lt;p&gt;You are correct.&lt;/p&gt;&#xA;&lt;p&gt;The main conceptual difference is that optimization is about finding the set of parameters/weights that maximizes/minimizes some objective function (which can also include a regularization term), while regularization is about limiting the values that your parameters can take during the optimization/learning/training, so optimization with regularisation (especially, with &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; regularization) can be thought of as &lt;em&gt;constrained optimization&lt;/em&gt;, but, in some cases, such as dropout, it can also be thought of as a way of &lt;em&gt;introducing noise&lt;/em&gt; in the training process.&lt;/p&gt;&#xA;&lt;p&gt;You should use regularisation when your neural network is &lt;em&gt;big&lt;/em&gt; (where big, of course, is not well-defined) and you have &lt;em&gt;little&lt;/em&gt; data (where little is also not well-defined). Why do you want to use regularisation in this case? Because big neural networks have more &lt;a href=&quot;https://ai.stackexchange.com/a/17881/2444&quot;&gt;&lt;em&gt;capacity&lt;/em&gt;&lt;/a&gt;, so they can memorize the training data more easily (i.e. they can over-fit the training data). If the training data is not representative of the whole data distribution (that you are trying to capture with your neural network), then your neural network may fail on other data from that data distribution, i.e. it may not &lt;em&gt;generalize&lt;/em&gt;. Regularization techniques, such as &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; penalties and dropout, limit the complexity of the functions that can be represented by your neural network (remember that, for a specific set of weights &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, a neural network represents a specific function &lt;span class=&quot;math-container&quot;&gt;$f_\theta$&lt;/span&gt;), so they prevent your neural network from learning a complicated function that would just over-fit the training data.&lt;/p&gt;&#xA;&lt;p&gt;Of course, you can also use regularisation when you have a very large dataset or your neural network is not that big. However, in principle, the case mentioned above is the case where you will likely need some kind of regularisation. So, as a rule of thumb, the bigger your NN is and the smaller your training dataset is, the more likely you will need some kind of regularisation.&lt;/p&gt;&#xA;&lt;p&gt;The typical way to visualize that your neural network is over-fitting is to look at the evolution of the loss function (a type of objective function, which you want to minimize) for the training dataset and the validation dataset (a dataset that you do not use for training, i.e. updating the weights of the neural network). If the training loss is very small, while the validation loss is bigger (stays constant or even increases), as you train more the neural network, that's a good sign of over-fitting, and it suggests that you may need some kind of regularisation. However, note that, even with regularisation, it is not guaranteed that you will find a good set of weights that is able to generalize to all data (not seen during training), but it's more likely.&lt;/p&gt;&#xA;&lt;p&gt;There are other forms of regularization, such as the KL divergence in the context of Bayesian neural networks or variational auto-encoders, but these are more advanced topics that you don't need to know now. In any case, the KL divergence has the same role as the other regularisation techniques that I mentioned above, i.e., in some way, it restricts/limits the possible functions that you can learn.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-26T19:05:30.043" LastActivityDate="2020-11-26T19:05:30.043" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24871" PostTypeId="2" ParentId="24848" CreationDate="2020-11-26T20:40:11.503" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In both cases, even though the available examples are not a random sample from &lt;strong&gt;the true underlying distribution&lt;/strong&gt; of examples, we would like to learn a predictor from the examples that is as accurate as possible for &lt;strong&gt;this distribution&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The true underlying distribution&amp;quot; is the closest &amp;quot;distribution&amp;quot; that is explicitly mentioned as such in the part of the text preceding the phrase &amp;quot;this distribution&amp;quot;, so that's what it's referring to. For clarity, I've put the two things that are &amp;quot;the same&amp;quot; in bold in the above quote.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-11-26T20:40:11.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24872" PostTypeId="1" AcceptedAnswerId="24878" CreationDate="2020-11-26T21:53:53.183" Score="1" ViewCount="68" Body="&lt;p&gt;I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either &amp;quot;good&amp;quot; or &amp;quot;bad&amp;quot;). I am using the &lt;code&gt;tensorflow.keras.applications.VGG16&lt;/code&gt; class as base model with a custom classifier on top, like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_layer = layers.Input(shape=(self.image_size, self.image_size, 3), name=&amp;quot;model_input&amp;quot;)&#xA;base_model = VGG16(weights=&amp;quot;imagenet&amp;quot;, include_top=False, input_tensor=input_layer)&#xA;model_head = base_model.output&#xA;model_head = layers.AveragePooling2D(pool_size=(4, 4))(model_head)&#xA;model_head = layers.Flatten()(model_head)&#xA;model_head = layers.Dense(256, activation=&amp;quot;relu&amp;quot;)(model_head)&#xA;model_head = layers.Dropout(0.5)(model_head)&#xA;model_head = layers.Dense(len(self.image_classes), activation=&amp;quot;softmax&amp;quot;)(model_head)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see in the last (output) layer I am using a &lt;code&gt;softmax&lt;/code&gt; activation function. Then I compile the whole model with the &lt;code&gt;categorical_crossentropy&lt;/code&gt; loss function and train with one-hot-encoded image data (labels).&lt;/p&gt;&#xA;&lt;p&gt;All in all the model performs quite well, I am happy with the results, I achieve over 99% test and validation accuracy with our data set. There is one thing I don't understand though:&lt;/p&gt;&#xA;&lt;p&gt;When I call &lt;code&gt;predict()&lt;/code&gt; on the Keras model and look at the prediction results, then these are always either 0 or 1 (or at least very, very close to that, like 0.000001 and 0.999999). So my classifier seems to be quite sure whether an image belongs to either class &amp;quot;good&amp;quot; or &amp;quot;bad&amp;quot; (for example, if I am using only two classes). I was under the assumption, however, that usually these predictions are not that clear, more in terms of &lt;em&gt;&amp;quot;the model thinks with a probability of 80% that this image belongs to class A&amp;quot;&lt;/em&gt; - but as said in my case it's always 100% sure.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas why this might be the case?&lt;/p&gt;&#xA;" OwnerUserId="14504" LastEditorUserId="2444" LastEditDate="2020-11-28T02:23:29.580" LastActivityDate="2020-11-29T19:07:57.833" Title="Why is my Keras prediction always close to 100% for one image class?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;keras&gt;&lt;bayesian-deep-learning&gt;&lt;uncertainty-quantification&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24873" PostTypeId="2" ParentId="24872" CreationDate="2020-11-26T23:13:31.547" Score="1" Body="&lt;p&gt;Without more details about the nature of the dataset, it is impossible to know for sure. However, here are a few likely causes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You were calling predict on training data, not testing data. The network will be a lot more sure about images that it trained on than on images it has never seen before.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your model overfit the data. This can happen when you use an overly complex model on a small dataset. You may want to experiment with regularization.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You were looking at too small a sample of images. Did you run predict on every image, or just a few? If the latter, it is possible you just picked a sample that the network is very confident about.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12201" LastActivityDate="2020-11-26T23:13:31.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24874" PostTypeId="1" CreationDate="2020-11-26T23:43:59.423" Score="2" ViewCount="58" Body="&lt;p&gt;I'm working on a classifier for the famous MNIST handwritten data set.&lt;/p&gt;&#xA;&lt;p&gt;I want to create a few features on my own, and I want to be able to estimate which feature might perform better before actually training the classifier. Lets say that I create the feature which calculates the ratio of ink used between two halves of a digit. Note that with ink I mean how much white there is used (which ranges from 0-255 per pixel).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vnQg4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vnQg4.png&quot; alt=&quot;Example digit&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l4P3S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l4P3S.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example I would calculate the ratio between the total amount of white in the left and right halves (seperated by red line). I could also do the same with the op and bottom half, or seperate the digits diagonally. With this I can calculate the mean and standard deviation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N2s4s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N2s4s.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I imagine that the left / right ratio might give some differences for other numbers. But the ratios might all be closer to the average.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there some method for estimating which feature might perform better compared to others? I.e. is there a method which gives a numerical value on how &amp;quot;seperable&amp;quot; a data set is?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="42556" LastActivityDate="2020-11-26T23:43:59.423" Title="Statistical method for selecting features for classification" Tags="&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;feature-extraction&gt;&lt;mnist&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24875" PostTypeId="1" CreationDate="2020-11-27T15:47:34.673" Score="1" ViewCount="29" Body="&lt;p&gt;I am implementing the A3C algorithm and I want to add off-policy training using &lt;a href=&quot;https://arxiv.org/abs/1606.02647&quot; rel=&quot;nofollow noreferrer&quot;&gt;Retrace&lt;/a&gt; but I am having some trouble understanding how to compute the retrace target. Retrace is used in combination with A3C for example in the &lt;a href=&quot;https://arxiv.org/abs/1704.04651&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reactor&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I often see the retrace update written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\Delta Q(s, a) = \sum_{t' = t}^{T} \gamma^{t'-t}\left(\prod_{j=t+1}^{t'}c_j\right) \delta_{t'}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$\delta_{t'} = r(s_{t'}, a_{t'}) + \gamma \mathbb{E}[Q(s_{t'+1}, a_{t'+1})] - Q(s_{t'}, a_{t'})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_j$&lt;/span&gt; being the Retrace factors &lt;span class=&quot;math-container&quot;&gt;$c_j = \lambda \min(c, \frac{\pi(a_j|s_j)}{b(a_j|s_j)})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, when employing neural networks to approximate &lt;span class=&quot;math-container&quot;&gt;$Q_{\theta}(s, a)$&lt;/span&gt; it is often easier to define a loss&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mathcal{L}_{\theta} = \left(G_t - Q(s, a)\right)^2&#xA;\end{equation}&lt;/span&gt;&#xA;and let the backward function and the optimizer do the update. How can I write the Retrace target &lt;span class=&quot;math-container&quot;&gt;$G_t$&lt;/span&gt; to use in such a setup?&lt;/p&gt;&#xA;&lt;p&gt;Is it correct to write it as follows?&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;G_t = \sum_{t'=t}^T \gamma^{t'-t} \left(\prod_{j=t+1}^{t'}c_j\right) (r_{t'} + \gamma Q(s_{t'+1}, a_{t'+1}) - Q(s_{t'}, a_{t'}))&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and then compute &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; as above, take the gradient &lt;span class=&quot;math-container&quot;&gt;$\nabla\mathcal{L}_{\theta}$&lt;/span&gt; and perform the update step &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a_t) = Q(s_t, a_t) + \alpha \nabla\mathcal{L}_{\theta}$&lt;/span&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="32583" LastEditorUserId="32583" LastEditDate="2020-11-28T17:05:07.543" LastActivityDate="2020-11-28T17:05:07.543" Title="How to compute the Retrace target for multi-step off-policy Reinforcement Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;deep-rl&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24876" PostTypeId="2" ParentId="24849" CreationDate="2020-11-27T16:54:01.993" Score="2" Body="&lt;p&gt;The crossover rate, &lt;span class=&quot;math-container&quot;&gt;$p_c \in [0, 1]$&lt;/span&gt;, is a hyper-parameter that controls the rate at which solutions are subjected to crossover. So, the higher &lt;span class=&quot;math-container&quot;&gt;$p_c$&lt;/span&gt;, the more crossovers you perform, so the more &lt;strong&gt;diversity&lt;/strong&gt; (in terms of solutions/chromosomes) you may introduce in the population. Typical values of &lt;span class=&quot;math-container&quot;&gt;$p_c$&lt;/span&gt; are in the range &lt;span class=&quot;math-container&quot;&gt;$[0.5, 1.0]$&lt;/span&gt;.  For example, in &lt;a href=&quot;https://github.com/nbro/function_max_with_ga&quot; rel=&quot;nofollow noreferrer&quot;&gt;this simple implementation&lt;/a&gt;, I used &lt;span class=&quot;math-container&quot;&gt;$p_c = 0.8$&lt;/span&gt;. In this case, I manually searched for this value (and it didn't take me much time to make the GA find the solution I was looking for), but in other cases/problems you may need to perform some hyper-parameter optimization (e.g. with grid search).&lt;/p&gt;&#xA;&lt;p&gt;The mutation rate, &lt;span class=&quot;math-container&quot;&gt;$p_m \in [0, 1]$&lt;/span&gt; (also a hyper-parameter) controls the rate at which you mutate single chromosomes or the genes of single chromosomes. For example, if you have a binary chromosome of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; bits, you could flip the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th bit with probability &lt;span class=&quot;math-container&quot;&gt;$p_m$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; independently. Mutation can &lt;strong&gt;prevent premature convergence&lt;/strong&gt;, but, if &lt;span class=&quot;math-container&quot;&gt;$p_m$&lt;/span&gt; is very high, the genetic algorithm becomes a random search, so the value of &lt;span class=&quot;math-container&quot;&gt;$p_m$&lt;/span&gt; is typically not high. &lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=286385&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; (which you probably should read because it contains more information about the mutation and crossover operations, although it is a bit old) states that typical values are in the range &lt;span class=&quot;math-container&quot;&gt;$[0.005, 0.05]$&lt;/span&gt;, but, in &lt;a href=&quot;https://github.com/nbro/function_max_with_ga&quot; rel=&quot;nofollow noreferrer&quot;&gt;this implementation&lt;/a&gt;, I used &lt;span class=&quot;math-container&quot;&gt;$p_m = 0.5$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, in general, higher values of these rates promote exploration at the expense of exploitation. In this sense, this is similar to the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy behaviour policy in Q-learning (a famous reinforcement learning algorithm).&lt;/p&gt;&#xA;&lt;p&gt;However, note that each of these operations may undo the other operation. For instance, let's suppose that we have the binary chromosomes &lt;span class=&quot;math-container&quot;&gt;$p_1 = [0, 1, 0]$&lt;/span&gt;  and &lt;span class=&quot;math-container&quot;&gt;$p_2 = [0, 0, 1]$&lt;/span&gt;, which we can combine by replacing (with probability &lt;span class=&quot;math-container&quot;&gt;$p_c$&lt;/span&gt;) the bit at position &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; with the bit of the chromosome &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; at the same position &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. So, after the crossover, you could get the children &lt;span class=&quot;math-container&quot;&gt;$c_1 = [0, 0, 0]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_2 = [0, 1, 1]$&lt;/span&gt;. If you perform a mutation by flipping the bits, then you could end up with the same chromosomes as the original ones, that's also one of the reasons why you may not want to have a very high mutation rate. So, it's not completely true that, the higher the mutation rate, the more you explore, because you can end up with the same solutions as before, so you have not explored anything.&lt;/p&gt;&#xA;&lt;p&gt;You could also have an adaptive value for these parameters. See e.g. &lt;a href=&quot;https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=286385&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for an example of such an adaptive schedule.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-28T02:55:49.340" LastActivityDate="2020-11-28T02:55:49.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24878" PostTypeId="2" ParentId="24872" CreationDate="2020-11-28T02:20:45.130" Score="2" Body="&lt;p&gt;Traditional neural networks can be &lt;strong&gt;over-confident&lt;/strong&gt; (i.e. give a probability close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) even when they are wrong, so you should &lt;strong&gt;not&lt;/strong&gt; interpret the probability that it produces as a measure of &lt;em&gt;uncertainty&lt;/em&gt; (i.e. as a measure of how much it is confident that the associated predicted class is the correct one), as that it is essentially wrong. See &lt;a href=&quot;https://ai.stackexchange.com/a/17671/2444&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://stats.stackexchange.com/a/437769/82135&quot;&gt;this&lt;/a&gt; answers for more details about this.&lt;/p&gt;&#xA;&lt;p&gt;Given that this overconfidence is not desirable in many scenarios (such as healthcare, where doctors also want to know how &lt;em&gt;confident&lt;/em&gt; the model is about its predictions, in order to decide whether to give a certain medication to the patient or not), the ML community has been trying to incorporate &lt;em&gt;uncertainty quantification/estimation&lt;/em&gt; in neural networks. If you are interested in this topic, you could read the paper &lt;a href=&quot;http://proceedings.mlr.press/v37/blundell15.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Uncertainty in Neural Network&lt;/a&gt; (2015) by Blundell et al., which proposes a specific type of Bayesian neural network, i.e. a neural network that models the uncertainty over the actual values of the weights, from which we may also quantify/estimate the uncertainty about the inputs. This paper should not be too difficult to read if you are already familiar with the details of variational-autoencoders.&lt;/p&gt;&#xA;&lt;p&gt;So, the answer to your question is: yes, it's possible that the output probability is close to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; because neural networks can be over-confident. (I am assuming that the values returned by &lt;code&gt;tf.keras&lt;/code&gt;'s &lt;code&gt;predict&lt;/code&gt; method are probabilities: I don't remember anymore, so I assumed that you did not make any mistake).&lt;/p&gt;&#xA;&lt;p&gt;A similar question was already asked in the past &lt;a href=&quot;https://ai.stackexchange.com/q/17721/2444&quot;&gt;here&lt;/a&gt;. &lt;a href=&quot;https://ai.stackexchange.com/a/17722/2444&quot;&gt;The accepted answer&lt;/a&gt; should provide more details about different types of uncertainty and solutions.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-29T19:07:57.833" LastActivityDate="2020-11-29T19:07:57.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24879" PostTypeId="5" CreationDate="2020-11-28T02:40:24.147" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-28T02:40:24.147" LastActivityDate="2020-11-28T02:40:24.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24880" PostTypeId="4" CreationDate="2020-11-28T02:40:24.147" Score="0" Body="For questions about uncertainty quantification (aka uncertainty estimation) in the context of artificial intelligence, in particular, in the context of Bayesian machine learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-28T02:40:24.147" LastActivityDate="2020-11-28T02:40:24.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24881" PostTypeId="2" ParentId="20735" CreationDate="2020-11-28T02:46:55.200" Score="0" Body="&lt;p&gt;Okay so your CNN model is taking an Image and outputting the bounding boxes for them. That means the last layer of CNN model must be having four outputs which are generating real numbers. This is a regression problem.&lt;/p&gt;&#xA;&lt;p&gt;In that case, you can take L1 loss (mean absolute error) or L2 loss (mean square error) as your loss function. I have created a similar project, I have used L1 loss.&lt;/p&gt;&#xA;&lt;p&gt;Suppose your input image is &lt;strong&gt;x&lt;/strong&gt; and &lt;strong&gt;predicted_bb&lt;/strong&gt; is the model output and &lt;strong&gt;real_bb&lt;/strong&gt; is your original bounding box for image x. Then you should proceed as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;predicted_bb = model(x)&#xA;&#xA;# CALCULATE LOSS BETWEEN THE ACTUAL &amp;amp; PREDICTED BB COORDINATES&#xA;&#xA;loss_bb = torch.nn.functional.l1_loss(predicted_bb, real_bb, reduction=&amp;quot;none&amp;quot;).sum(1)&#xA;&#xA;# SET GRADIENTS TO ZERO&#xA;&#xA;optimizer.zero_grad()&#xA;&#xA;# BACKPROPOGATE THE LOSS&#xA;&#xA;loss.backward()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For tensorflow keras&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;predicted_bb = Dense(4, activation='relu')(x)&#xA;&#xA;model = Model(inputs = image_input, outputs = [predicted_bb])&#xA;model.compile(loss=['mae'], optimizer='adam', metrics =['accuracy'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="42583" LastActivityDate="2020-11-28T02:46:55.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24882" PostTypeId="2" ParentId="20118" CreationDate="2020-11-28T04:10:38.413" Score="1" Body="&lt;p&gt;I will give one perspective on this from the domain of robotics. You are right that most RL agents are trained in simulation particularly for research papers, because it allows researchers to in theory benchmark their approaches in a common environment. Many of the environments exist strictly as a test bed for new algorithms and are not even physically realizable, e.g. &lt;a href=&quot;https://gym.openai.com/envs/HalfCheetah-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HalfCheetah&lt;/a&gt;. You could in theory have a separate simulator say running in another process that you use as your planning model, and the &amp;quot;real&amp;quot; simulator is then your environment. But really that's just a mocked setup for what you really want in the end, which is having a real-world agent in a real-world environment.&lt;/p&gt;&#xA;&lt;p&gt;What you describe could be very useful, with one important caveat: the simulator needs to in fact be a good model of the real environment. For robotics and many other interesting domains, this is a tall order. Getting a physics simulator that faithfully replicates the real-world environment can be tricky, as one may need accurate friction coefficients, mass and center of mass, restitution coefficients, material properties, contact models, and so on. Oftentimes the simulator is too crude an approximation of the real-world environment to be useful as a planner.&lt;/p&gt;&#xA;&lt;p&gt;That doesn't mean we're completely hosed though. &lt;a href=&quot;https://arxiv.org/pdf/1810.05687.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; uses highly parallelized simulators to search for simulation parameters that approximate the real-world well. What's interesting is it's not even necessarily finding the correct real-world values for e.g. friction coefficients and such, but it finds values for parameters that, taken together, produces simulations that match the real-world experience. The better the simulation gets at approximating what's going on in the real world, the more viable it is to use the simulator for task planning. I think with the advent of &lt;a href=&quot;https://developer.nvidia.com/isaac-gym&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPU-optimized physics simulators&lt;/a&gt; we will see simulators be a more useful tool even for real-world agents, as you can try many different things in parallel to get a sense of what is the likely outcome of a planned action sequence.&lt;/p&gt;&#xA;" OwnerUserId="20955" LastActivityDate="2020-11-28T04:10:38.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24883" PostTypeId="1" CreationDate="2020-11-28T04:37:57.533" Score="2" ViewCount="70" Body="&lt;p&gt;I know this might be specific to different problems, but does anyone know if there is any rule of thumb or references on what constitutes a large state space?&lt;/p&gt;&#xA;&lt;p&gt;I know that, according to multiple papers, tabular Q-learning is not suitable for these problems with large state spaces, but I can't find any indication of what would constitute a large state space&lt;/p&gt;&#xA;&lt;p&gt;I've been working on a problem where my state space is about 30x30 and updating my Q-learning with the tabular method runs and works well. Would 100x100 start to become too big or 400x400?&lt;/p&gt;&#xA;" OwnerUserId="42586" LastEditorUserId="2444" LastEditDate="2020-11-28T11:56:34.777" LastActivityDate="2020-11-28T12:51:00.177" Title="What constitutes a large space state (in Q-learning)?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;state-spaces&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24884" PostTypeId="1" AcceptedAnswerId="24894" CreationDate="2020-11-28T08:27:21.330" Score="1" ViewCount="55" Body="&lt;p&gt;A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models.&lt;/p&gt;&#xA;&lt;p&gt;So, why does the word &lt;em&gt;dataset&lt;/em&gt; contain the word &lt;em&gt;set&lt;/em&gt;? Does it have any relation with the mathematical set, where order and repetition do not matter?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="2444" LastEditDate="2020-11-29T12:20:18.267" LastActivityDate="2020-11-29T12:20:18.267" Title="Is it okay to think of any dataset in artificial intelligence as a mathematical set?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;&lt;math&gt;&lt;definitions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24886" PostTypeId="2" ParentId="24883" CreationDate="2020-11-28T10:55:17.107" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I know this might be specific to different problems but does anyone know if there is any rule of thumb or references on what constitutes a large state space?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Not really, it is all relative. There are two main ways in which the scale of a value table might be too much:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Memory required to represent the table. This is relatively simple to calculate for any size.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Time required to populate the table with accurate estimates. This depends on how you are collecting that data, and how much variance there is given the same state, action choices.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you are using action values you need to allow for the fact that the table size is not just &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}|$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S} \times \mathcal{A}|$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you are running fast, local simulations of the environment, then table sizes of a million or even a hundred million are not unreasonable. Software language or library choice for the simulation and agent can have a significant impact at these larger scales.&lt;/p&gt;&#xA;&lt;p&gt;If any of the state description is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Continuous_or_discrete_variable&quot; rel=&quot;nofollow noreferrer&quot;&gt;continuous variable&lt;/a&gt;, then the table size would become infinite in theory, so no finite table that could actually be realised on a computer could fully capture it. You must use some form of approximation to get practical results. However, even then you can still use tabular approaches when approximation involves discretising the state variable directly - there are a few different methods used in machine learning to do that - e.g. &lt;a href=&quot;http://web.cs.ucla.edu/%7Esherstov/pdf/sara05-tiling.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tile coding&lt;/a&gt;. You can also use tile coding in large discrete spaces.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I've been working on a problem where my state space is about 30x30 and updating my Q-learning with the tabular method runs and works well. Would 100x100 start to become too big or 400x400?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It depends on the action space too, but 400x400 gives state space of 160,000. Assuming &amp;lt; 10 actions, then this is still well within the realm of tabular methods. You probably have less than 1,000,000 parameters to handle for a tabular method, which compares well to a moderately-sized neural network, with the advantage of better stability and possibility to create a fully optimal agent. If simulating the environemnt is fast, something at that scale may only take a few minutes to a few hours to fully optimise in tabular form.&lt;/p&gt;&#xA;&lt;p&gt;As with most machine learning though, if you care about some optimisation, such as memory size of the agent, speed at which it solves the problem, or some other metric, then you will need to try experiments with different approaches. Then you will have some knowledge about best approach for your problem, given your definition of &amp;quot;best agent&amp;quot;. The experience you gain with that &lt;em&gt;might&lt;/em&gt; extend and apply to similar problems in future.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-11-28T12:51:00.177" LastActivityDate="2020-11-28T12:51:00.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24887" PostTypeId="1" AcceptedAnswerId="25016" CreationDate="2020-11-28T12:34:30.540" Score="2" ViewCount="67" Body="&lt;p&gt;In Q-learning, is it mandatory to know all possible states that can the agent may end up in?&lt;/p&gt;&#xA;&lt;p&gt;I have a network with 4 source nodes, 3 sink nodes, and 4 main links. The initial state is the status network where the sink nodes have its resources at its maximum. In a random manner, I generate service requests from the source nodes to the sink nodes. These service requests are generated at random timesteps, which means that, from state to state, the network status may stay the same.&lt;/p&gt;&#xA;&lt;p&gt;When a service request is launched, the resources from the sink nodes change, and the network status changes.&lt;/p&gt;&#xA;&lt;p&gt;The aim of the agent is to balance the network by associating each service request to a sink node along with the path.&lt;/p&gt;&#xA;&lt;p&gt;I know that in MDP you are supposed to have a finite set of states, my question is: if that finite set of states is supposed to be all possible states that can happen, or is just a number that you consider enough to optimize the Q-table?&lt;/p&gt;&#xA;" OwnerUserId="42591" LastEditorUserId="2444" LastEditDate="2020-11-30T01:33:16.947" LastActivityDate="2020-12-06T10:17:06.227" Title="Do I need to know in advance all possible number of states in Q-Learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;state-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24889" PostTypeId="1" CreationDate="2020-11-28T13:38:00.043" Score="2" ViewCount="27" Body="&lt;p&gt;I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern.&lt;/p&gt;&#xA;&lt;p&gt;The data I have is some (64k in total) input-parameter-output match.&lt;/p&gt;&#xA;&lt;p&gt;My goal is to compress these 256 parameters to a smaller scale (like 32) using an encoder of some kind while being able to preserve the response pattern.&lt;/p&gt;&#xA;&lt;p&gt;But I can't seem to find a proper network for this particular problem, because I'm not trying to fit these parameters (they all have a mean of one and variance of 1/4), but rather its influence on the output, so traditional data-specific operations will not work in this case.&lt;/p&gt;&#xA;" OwnerUserId="42507" LastEditorUserId="11539" LastEditDate="2020-11-29T19:16:42.170" LastActivityDate="2020-11-29T19:16:42.170" Title="Compressing Parameters of an Response System" Tags="&lt;autoencoders&gt;&lt;variational-autoencoder&gt;&lt;dimensionality-reduction&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24890" PostTypeId="2" ParentId="24368" CreationDate="2020-11-28T14:22:29.663" Score="1" Body="&lt;p&gt;Yeah, it seems that you're right and based on the description of the paper it would indeed behave uniformly random at the very first iteration (or maybe just always deterministically pick whichever action happens to be the first one in the list). I can't find anything that would suggest otherwise in the paper, and also &lt;a href=&quot;https://arxiv.org/src/1911.08265v2/anc/pseudocode.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;the pseudocode they put on arXiv suggests the same behaviour&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I can't really think of any reason why this would be better than playing according to &lt;span class=&quot;math-container&quot;&gt;$P(s, a)$&lt;/span&gt; in the very first iteration though. Maybe it creates a tiny little bit more variety in the tree search results you get? I suppose all the stochasticity that is normally inherent to a vanilla MCTS is no longer present in MuZero (or AlphaZero), because they always run for exactly the same number of iterations, and don't have any sort of random rollouts anymore; this would at least introduce a tiny bit of variation.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, I also can't imagine this choice really making a meaningful difference in practice, except maybe if you go down to extremely low iteration counts (like if you run fewer iterations than there are legal actions in the root node).&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-11-28T14:22:29.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24891" PostTypeId="1" CreationDate="2020-11-28T17:07:49.003" Score="3" ViewCount="99" Body="&lt;p&gt;Convolution Neural Network (CNNs) operate over strict grid-like structures (&lt;span class=&quot;math-container&quot;&gt;$M \times N \times C$&lt;/span&gt; images), whereas Graph Neural Networks (GNNs) can operate over all-flexible graphs, with an undefined number of neighbors and edges.&lt;/p&gt;&#xA;&lt;p&gt;On the face of it, GNNs appear to be neural architectures that can subsume CNNs. Are GNNs really generalized architectures that can operate arbitrary functions over arbitrary graph structures?&lt;/p&gt;&#xA;&lt;p&gt;An obvious follow-up - &lt;strong&gt;How can we derive a CNN out of a GNN&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Since non-spectral GNNs are based on message-passing that employ &lt;em&gt;permutation-invariant&lt;/em&gt; functions, is it possible to derive a CNN from a base-architecture of GNN?&lt;/p&gt;&#xA;" OwnerUserId="42600" LastEditorUserId="2444" LastEditDate="2020-11-28T18:43:59.560" LastActivityDate="2021-05-28T11:00:10.053" Title="How can we derive a Convolution Neural Network from a more generic Graph Neural Network?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;geometric-deep-learning&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="24892" PostTypeId="1" CreationDate="2020-11-28T17:31:47.570" Score="0" ViewCount="43" Body="&lt;p&gt;Consider the regularized SVM approach:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;g (\alpha) = \sum_{i=1}^{n} \alpha_i - \frac{1}{2}\sum_{i=1}^{n} \sum_{j=1}^{n} \alpha_i \alpha_j y_i y_j \langle \phi(x_i), \phi(x_j)\rangle_H&#xA;\end{equation}&lt;/span&gt;&#xA;under the constraints &lt;span class=&quot;math-container&quot;&gt;$0\leq \alpha_i \leq C$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i=1, \dots, n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^{n} \alpha_i y_i = 0$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is some feature-map into a Hilbert space &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What is the easiest method to solve this? Is something like the SOM-Algorithm necessarily needed?&lt;/p&gt;&#xA;&lt;p&gt;If one tries to solve this analytically (setting the gradient to zero and solving the system of n linear equations), it doesn't work because the parameter &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; does not appear...&lt;/p&gt;&#xA;" OwnerUserId="42601" LastEditorUserId="2444" LastEditDate="2020-11-28T19:00:22.470" LastActivityDate="2020-11-28T19:00:22.470" Title="What is the easiest method to solve a regularized SVM with Lagrangian?" Tags="&lt;regularization&gt;&lt;support-vector-machine&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24893" PostTypeId="1" CreationDate="2020-11-28T18:02:41.060" Score="0" ViewCount="36" Body="&lt;p&gt;I have this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gym&#xA;import numpy as np&#xA;import copy&#xA;import math&#xA;import random&#xA;&#xA;class Connection:&#xA;    def __init__(self, connectedNeuron):&#xA;        self.connectedNeuron = connectedNeuron&#xA;        self.weight = np.random.normal()&#xA;        self.dWeight = 0.0&#xA;&#xA;&#xA;class Neuron:&#xA;    eta = 1&#xA;    alpha = 0.01&#xA;&#xA;    def __init__(self, layer):&#xA;        self.dendrons = []&#xA;        self.error = 0.0&#xA;        self.gradient = 0.0&#xA;        self.output = 0.0&#xA;        if layer is None:&#xA;            pass&#xA;        else:&#xA;            for neuron in layer:&#xA;                con = Connection(neuron)&#xA;                self.dendrons.append(con)&#xA;&#xA;    def addError(self, err):&#xA;        self.error = self.error + err&#xA;&#xA;    def sigmoid(self, x):&#xA;        return 1 / (1 + math.exp(-x * 1.0))&#xA;&#xA;    def dSigmoid(self, x):&#xA;        return x * (1.0 - x)&#xA;&#xA;    def setError(self, err):&#xA;        self.error = err&#xA;&#xA;    def setOutput(self, output):&#xA;        self.output = output&#xA;&#xA;    def getOutput(self):&#xA;        return self.output&#xA;&#xA;    def feedForword(self):&#xA;        sumOutput = 0&#xA;        if len(self.dendrons) == 0:&#xA;            return&#xA;        for dendron in self.dendrons:&#xA;            sumOutput = sumOutput + dendron.connectedNeuron.getOutput() * dendron.weight&#xA;        self.output = self.sigmoid(sumOutput)&#xA;&#xA;    def backPropagate(self):&#xA;        self.gradient = self.error * self.dSigmoid(self.output);&#xA;        &#xA;        for dendron in self.dendrons:&#xA;            dendron.dWeight = Neuron.eta * (&#xA;            self.gradient * dendron.connectedNeuron.output) + self.alpha * dendron.dWeight;&#xA;            dendron.weight = dendron.weight + dendron.dWeight;&#xA;            dendron.connectedNeuron.addError(dendron.weight * self.gradient);&#xA;        self.error = 0;&#xA;&#xA;&#xA;class Network:&#xA;    def __init__(self, topology):&#xA;        self.layers = []&#xA;        for numNeuron in topology:&#xA;            layer = []&#xA;            for i in range(numNeuron):&#xA;                if (len(self.layers) == 0):&#xA;                    layer.append(Neuron(None))&#xA;                else:&#xA;                    layer.append(Neuron(self.layers[-1]))&#xA;            layer.append(Neuron(None))&#xA;            layer[-1].setOutput(1)&#xA;            self.layers.append(layer)&#xA;&#xA;    def setInput(self, inputs):&#xA;        for i in range(len(inputs)):&#xA;            self.layers[0][i].setOutput(inputs[i])&#xA;&#xA;    def feedForword(self):&#xA;        for layer in self.layers[1:]:&#xA;            for neuron in layer:&#xA;                neuron.feedForword();&#xA;&#xA;    def backPropagate(self, loss):&#xA;        for i in range(len(loss)):&#xA;            #self.layers[-1][i].setError(self.layers[-1][i].getOutput()- target[i]) was originally this&#xA;            self.layers[-1][i].setError(loss[i])&#xA;        for layer in self.layers[::-1]:&#xA;            for neuron in layer:&#xA;                neuron.backPropagate()&#xA;&#xA;    def getError(self, target):&#xA;        err = 0&#xA;        for i in range(len(target)):&#xA;            e = (target[i] - self.layers[-1][i].getOutput())&#xA;            err = err + e ** 2&#xA;        err = err / len(target)&#xA;        err = math.sqrt(err)&#xA;        return err&#xA;&#xA;    def getResults(self):&#xA;        output = []&#xA;        for neuron in self.layers[-1]:&#xA;            output.append(neuron.getOutput())&#xA;        output.pop()&#xA;        return output&#xA;&#xA;    def getThResults(self):&#xA;        output = []&#xA;        for neuron in self.layers[-1]:&#xA;            o = neuron.getOutput()&#xA;            print(o)&#xA;            if (o &amp;gt; 0.5):&#xA;                o = 1&#xA;            else:&#xA;                o = 0&#xA;            output.append(o)&#xA;        output.pop()&#xA;        return output&#xA;&#xA;&#xA;policynetwork = Network([4,10,10,2])&#xA;targetnetwork = copy.deepcopy(policynetwork)&#xA;environment = gym.make(&amp;quot;CartPole-v0&amp;quot;)&#xA;environment.reset()&#xA;explorationorexploitation = 0&#xA;exploitationtreshold = 0&#xA;exploitationrise = 0.00005&#xA;oldobservation = [0,0,0,0]&#xA;observation = [0,0,0,0]&#xA;experiences = []&#xA;memorysize = 20000&#xA;gamma = 0.99&#xA;whentoupdatethenetwork = 10&#xA;howmanytimesrange = 20000&#xA;samplespertrainingtime = 20&#xA;&#xA;for howmanytimes in range(0, howmanytimesrange):&#xA;    exploitationtreshold += exploitationrise&#xA;    print(howmanytimes)&#xA;    &#xA;    exploitationtreshold = exploitationtreshold&#xA;    while True:&#xA;        policynetwork.setInput(observation)&#xA;        policynetwork.feedForword()&#xA;        explorationorexploitation = np.random.uniform()&#xA;        if explorationorexploitation &amp;gt; exploitationtreshold:&#xA;            action = random.randint(0,1)&#xA;        else:&#xA;            &#xA;            action = np.argmax(policynetwork.getResults())&#xA;        &#xA;        &#xA;        &#xA;&#xA;        observation, reward, done, info = environment.step(action)&#xA;        &#xA;        experiences.append([oldobservation, action, reward, observation])&#xA;        #print(experiences[0])&#xA;        if len(experiences) &amp;gt; memorysize-1:&#xA;            totalloss = 0&#xA;&#xA;            for i in range(samplespertrainingtime):&#xA;                sampletobotrained = random.choice(experiences)&#xA;                if len(experiences) == 0:&#xA;                    print(&amp;quot;huutista&amp;quot;)&#xA;                &#xA;&#xA;&#xA;&#xA;&#xA;                policynetwork.setInput(sampletobotrained[0])&#xA;                policynetwork.feedForword()&#xA;                qsa = policynetwork.getResults()&#xA;&#xA;&#xA;&#xA;                targetnetwork.setInput(sampletobotrained[3])&#xA;                targetnetwork.feedForword()&#xA;                qprimesa = targetnetwork.getResults()&#xA;                loss = []&#xA;                for paska in range(0, len(qprimesa)):&#xA;                    loss.append(((float(sampletobotrained[2])+(float(gamma)*float(max(qprimesa))))-(qsa[sampletobotrained[1]])))&#xA;                    print(((float(sampletobotrained[2])+(float(gamma)*float(max(qprimesa))))-(qsa[sampletobotrained[1]])))&#xA;                    totalloss += ((float(sampletobotrained[2])+(float(gamma)*float(max(qprimesa))))-(qsa[sampletobotrained[1]])) #originally was this whole thing squared on both of these&#xA;                #print(loss)&#xA;&#xA;            &#xA;&#xA;                policynetwork.backPropagate(loss)&#xA;                #print(loss)&#xA;&#xA;&#xA;            print(&amp;quot;Average total loss:&amp;quot; +str(totalloss/samplespertrainingtime))&#xA;&#xA;            experiences.pop(0)&#xA;&#xA;            &#xA;            &#xA;&#xA;        &#xA;        oldobservation = observation&#xA;        #environment.render()&#xA;        if done:&#xA;            environment.reset()&#xA;            break&#xA;&#xA;    if howmanytimes % whentoupdatethenetwork == 0:&#xA;        targetnetwork = copy.deepcopy(policynetwork)&#xA;    &#xA;&#xA;    howmanytimes = howmanytimes + 1&#xA;&#xA;print(&amp;quot;Training complete.&amp;quot;)&#xA;while True:&#xA;    targetnetwork.setInput(observation)&#xA;    targetnetwork.feedForword()&#xA;    action = np.argmax([policynetwork.layers[-1][0].output, policynetwork.layers[-1][1].output])&#xA;&#xA;    observation,_,done,_ =environment.step(action)&#xA;    environment.render()&#xA;    if done:&#xA;        environment.reset()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the code is supposed to do the classical q learning example of the balancing pole thingy, but this code fails to decrease the loss. What i am doing wrong? I tried nudging all of the values, but nothing seems to help. I am trying to follow this tutorial series: &lt;a href=&quot;https://www.youtube.com/watch?v=nyjbcRQ-uQ8&amp;amp;list=PLZbbT5o_s2xoWNVdDudn51XM8lOuZ_Njv&amp;amp;index=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, the episodes which are conserned with actually training the deep q networks. My guess is that i have understood something wrong in calculating the loss. The neural network class is third party, but i had to modify some bits, for example you can see my comment on the backpropagate() function of the neural network.&lt;/p&gt;&#xA;&lt;p&gt;And also this somehow kinda works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gym&#xA;import numpy as np&#xA;import copy&#xA;import math&#xA;import random&#xA;&#xA;class Connection:&#xA;    def __init__(self, connectedNeuron):&#xA;        self.connectedNeuron = connectedNeuron&#xA;        self.weight = np.random.normal()&#xA;        self.dWeight = 0.0&#xA;&#xA;&#xA;class Neuron:&#xA;    eta = 0.01&#xA;    alpha = 0.01&#xA;&#xA;    def __init__(self, layer):&#xA;        self.dendrons = []&#xA;        self.error = 0.0&#xA;        self.gradient = 0.0&#xA;        self.output = 0.0&#xA;        if layer is None:&#xA;            pass&#xA;        else:&#xA;            for neuron in layer:&#xA;                con = Connection(neuron)&#xA;                self.dendrons.append(con)&#xA;&#xA;    def addError(self, err):&#xA;        self.error = self.error + err&#xA;&#xA;    def sigmoid(self, x):&#xA;        return 1 / (1 + math.exp(-x * 1.0))&#xA;&#xA;    def dSigmoid(self, x):&#xA;        return x * (1.0 - x)&#xA;&#xA;    def setError(self, err):&#xA;        self.error = err&#xA;&#xA;    def setOutput(self, output):&#xA;        self.output = output&#xA;&#xA;    def getOutput(self):&#xA;        return self.output&#xA;&#xA;    def feedForword(self):&#xA;        sumOutput = 0&#xA;        if len(self.dendrons) == 0:&#xA;            return&#xA;        for dendron in self.dendrons:&#xA;            sumOutput = sumOutput + dendron.connectedNeuron.getOutput() * dendron.weight&#xA;        self.output = self.sigmoid(sumOutput)&#xA;&#xA;    def backPropagate(self):&#xA;        self.gradient = self.error * self.dSigmoid(self.output);&#xA;        &#xA;        for dendron in self.dendrons:&#xA;            dendron.dWeight = Neuron.eta * (&#xA;            self.gradient * dendron.connectedNeuron.output) + self.alpha * dendron.dWeight;&#xA;            dendron.weight = dendron.weight + dendron.dWeight;&#xA;            dendron.connectedNeuron.addError(dendron.weight * self.gradient);&#xA;        self.error = 0;&#xA;&#xA;&#xA;class Network:&#xA;    def __init__(self, topology):&#xA;        self.layers = []&#xA;        for numNeuron in topology:&#xA;            layer = []&#xA;            for i in range(numNeuron):&#xA;                if (len(self.layers) == 0):&#xA;                    layer.append(Neuron(None))&#xA;                else:&#xA;                    layer.append(Neuron(self.layers[-1]))&#xA;            layer.append(Neuron(None))&#xA;            layer[-1].setOutput(1)&#xA;            self.layers.append(layer)&#xA;&#xA;    def setInput(self, inputs):&#xA;        for i in range(len(inputs)):&#xA;            self.layers[0][i].setOutput(inputs[i])&#xA;&#xA;    def feedForword(self):&#xA;        for layer in self.layers[1:]:&#xA;            for neuron in layer:&#xA;                neuron.feedForword();&#xA;&#xA;    def backPropagate(self, loss):&#xA;        for i in range(len(loss)):&#xA;            #self.layers[-1][i].setError(self.layers[-1][i].getOutput()- target[i]) was originally this&#xA;            self.layers[-1][i].setError(loss[i])&#xA;        for layer in self.layers[::-1]:&#xA;            for neuron in layer:&#xA;                neuron.backPropagate()&#xA;&#xA;    def getError(self, target):&#xA;        err = 0&#xA;        for i in range(len(target)):&#xA;            e = (target[i] - self.layers[-1][i].getOutput())&#xA;            err = err + e ** 2&#xA;        err = err / len(target)&#xA;        err = math.sqrt(err)&#xA;        return err&#xA;&#xA;    def getResults(self):&#xA;        output = []&#xA;        for neuron in self.layers[-1]:&#xA;            output.append(neuron.getOutput())&#xA;        output.pop()&#xA;        return output&#xA;&#xA;    def getThResults(self):&#xA;        output = []&#xA;        for neuron in self.layers[-1]:&#xA;            o = neuron.getOutput()&#xA;            print(o)&#xA;            if (o &amp;gt; 0.5):&#xA;                o = 1&#xA;            else:&#xA;                o = 0&#xA;            output.append(o)&#xA;        output.pop()&#xA;        return output&#xA;&#xA;&#xA;policynetwork = Network([4,10,2])&#xA;targetnetwork = copy.deepcopy(policynetwork)&#xA;environment = gym.make(&amp;quot;CartPole-v0&amp;quot;)&#xA;environment.reset()&#xA;explorationorexploitation = 0&#xA;exploitationtreshold = 0&#xA;exploitationrise = 0.0001&#xA;oldobservation = [0,0,0,0]&#xA;observation = [0,0,0,0]&#xA;experiences = []&#xA;memorysize = 2000&#xA;gamma = 0.99&#xA;whentoupdatethenetwork = 10&#xA;howmanytimesrange = 1000&#xA;samplespertrainingtime = 20&#xA;&#xA;for howmanytimes in range(0, howmanytimesrange):&#xA;    exploitationtreshold += exploitationrise&#xA;    print(howmanytimes)&#xA;    &#xA;    exploitationtreshold = exploitationtreshold&#xA;    while True:&#xA;        targetnetwork.setInput(observation)&#xA;        targetnetwork.feedForword()&#xA;        explorationorexploitation = np.random.uniform()&#xA;        if explorationorexploitation &amp;gt; exploitationtreshold:&#xA;            action = random.randint(0,1)&#xA;        else:&#xA;            &#xA;            action = np.argmax([targetnetwork.layers[-1][0].output, targetnetwork.layers[-1][1].output])&#xA;        &#xA;        &#xA;        &#xA;&#xA;        observation, reward, done, info = environment.step(action)&#xA;        &#xA;        experiences.append([oldobservation, action, reward, observation])&#xA;        #print(experiences[0])&#xA;        if len(experiences) &amp;gt; memorysize-1:&#xA;&#xA;            for i in range(samplespertrainingtime):&#xA;                sampletobotrained = random.choice(experiences)&#xA;&#xA;&#xA;&#xA;&#xA;                policynetwork.setInput(sampletobotrained[0])&#xA;                policynetwork.feedForword()&#xA;                qsa = policynetwork.getResults()&#xA;&#xA;&#xA;&#xA;                targetnetwork.setInput(sampletobotrained[3])&#xA;                targetnetwork.feedForword()&#xA;                qprimesa = targetnetwork.getResults()&#xA;                loss = []&#xA;                for paska in range(0, len(qprimesa)):&#xA;                    loss.append(((float(sampletobotrained[2])+(float(gamma)*float(max(qprimesa))))-(qsa[paska]))**2)&#xA;&#xA;            &#xA;&#xA;                policynetwork.backPropagate(loss)&#xA;                #print(loss)&#xA;&#xA;&#xA;&#xA;&#xA;            experiences.pop(random.randrange(len(experiences)))&#xA;&#xA;            &#xA;            &#xA;            &#xA;&#xA;        &#xA;        oldobservation = observation&#xA;        #environment.render()&#xA;        if done:&#xA;            environment.reset()&#xA;            break&#xA;&#xA;    if howmanytimes % whentoupdatethenetwork == 0:&#xA;        targetnetwork = copy.deepcopy(policynetwork)&#xA;    &#xA;&#xA;    howmanytimes = howmanytimes + 1&#xA;&#xA;print(&amp;quot;Training complete.&amp;quot;)&#xA;while True:&#xA;    targetnetwork.setInput(observation)&#xA;    targetnetwork.feedForword()&#xA;    action = np.argmax([targetnetwork.layers[-1][0].output, targetnetwork.layers[-1][1].output])&#xA;&#xA;    observation,_,done,_ =environment.step(action)&#xA;    environment.render()&#xA;    if done:&#xA;        environment.reset()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am lost, so i will cling onto any kind of help that i can get my hands on.&lt;/p&gt;&#xA;" OwnerUserId="42599" LastEditorUserId="11539" LastEditDate="2020-11-29T19:16:34.750" LastActivityDate="2020-11-29T19:16:34.750" Title="Deep Q-learning network not working" Tags="&lt;neural-networks&gt;&lt;q-learning&gt;&lt;dqn&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24894" PostTypeId="2" ParentId="24884" CreationDate="2020-11-28T18:12:44.097" Score="3" Body="&lt;p&gt;It's true that your original dataset can contain duplicates, so it should not be called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Set_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;set&lt;/em&gt;&lt;/a&gt;, in order to be consistent with the mathematical definition of a &lt;em&gt;set&lt;/em&gt;. There are mathematical objects known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiset&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;multi-sets&lt;/em&gt;&lt;/a&gt; that can contain duplicates, but the order of the elements is still not relevant. There are also &lt;a href=&quot;https://en.wikipedia.org/wiki/Tuple&quot; rel=&quot;nofollow noreferrer&quot;&gt;tuples&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequences&lt;/a&gt;, where the order of the elements matters.&lt;/p&gt;&#xA;&lt;p&gt;If you want to get rid of the duplicate elements in your dataset, you could perform a &lt;a href=&quot;https://en.wikipedia.org/wiki/Data_pre-processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;pre-processing step&lt;/a&gt; where you remove them. Even if you do that, it is often the case that, if you are learning with mini-batches (i.e. using &lt;a href=&quot;https://ai.stackexchange.com/a/11675/2444&quot;&gt;mini-batch stochastic gradient descent&lt;/a&gt;), these mini-batches could contain the same elements, because you may sample the same element in different batches or even in the same batch (this is known as &lt;a href=&quot;https://stattrek.com/statistics/dictionary.aspx?definition=sampling_without_replacement&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;sampling with replacement&lt;/em&gt;&lt;/a&gt;). Of course, this depends on how you sample your training dataset to build the batches (or mini-batches). So, if you do not want duplicates even in the mini-batches, you need to perform &lt;a href=&quot;https://stattrek.com/statistics/dictionary.aspx?definition=sampling_without_replacement&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;sampling without replacement&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, there are datasets that contain elements whose order in the dataset can be relevant for the predictions, such as datasets of &lt;a href=&quot;https://en.wikipedia.org/wiki/Time_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;time-series data&lt;/a&gt;, while, in mathematical sets and multi-sets, the order of the elements does not matter.&lt;/p&gt;&#xA;&lt;p&gt;So, yes, it is often called a &lt;em&gt;dataset&lt;/em&gt; (or &lt;em&gt;data set&lt;/em&gt;), but it is not necessarily a set in a mathematical sense. In general, it should just be interpreted as a &lt;em&gt;collection of data&lt;/em&gt;. In scenarios where the order of the elements or the existence of duplicates in the dataset (or any other information or property of your collection of data) is relevant, you should probably emphasize/note it.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-28T20:10:29.840" LastActivityDate="2020-11-28T20:10:29.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24895" PostTypeId="1" CreationDate="2020-11-28T19:04:56.557" Score="2" ViewCount="62" Body="&lt;p&gt;Slightly generalizing the definition in &lt;a href=&quot;https://www.researchgate.net/profile/Herbert_Jaeger3/publication/215385037_The_echo_state_approach_to_analysing_and_training_recurrent_neural_networks-with_an_erratum_note%27/links/566a003508ae62b05f027be3/The-echo-state-approach-to-analysing-and-training-recurrent-neural-networks-with-an-erratum-note.pdf?origin=publication_detail&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jaeger 2001&lt;/a&gt;, let's define a &lt;strong&gt;reservoir system&lt;/strong&gt; to be any system of the form&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h_{t}=f(h_{t-1}, x_t)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_t=g(Wh_t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; are fixed and &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is a learnable weight matrix. The idea is we feed a sequence of inputs &lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; into the system, which has some fixed initial state &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt;, and thereby generate the sequence of outputs &lt;span class=&quot;math-container&quot;&gt;$y_t$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is fixed (for example, a randomly generated RNN) we can then attempt to learn &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; in some way in order to get the system to have the behavior that we want.&lt;/p&gt;&#xA;&lt;p&gt;Now we add the &lt;strong&gt;echo state condition&lt;/strong&gt;: the system has the echo state condition iff for any left-infinite sequence &lt;span class=&quot;math-container&quot;&gt;$...x_{-3}, x_{-2}, x_{-1}, x_0$&lt;/span&gt;, there is only one sequence of states &lt;span class=&quot;math-container&quot;&gt;$h_t$&lt;/span&gt; consistent with this input sequence.&lt;/p&gt;&#xA;&lt;p&gt;Seen from this perspective, any training procedure that could be applied to an echo state system could be applied to a generic reservoir system. So &lt;strong&gt;what do we get out of the echo state condition&lt;/strong&gt;? Is there some reason to think echo state systems will generalize better, or be more quickly trainable? Jaeger does not seem to attempt to argue in this direction, he just describes how to train an ESN, but as I've said, nothing about these training methods seems to require the echo state property.&lt;/p&gt;&#xA;" OwnerUserId="1931" LastActivityDate="2021-04-30T18:01:05.093" Title="What do echo state networks give us over a generic RNN resevoir?" Tags="&lt;neural-networks&gt;&lt;echo-state-network&gt;&lt;reservoir-computing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24897" PostTypeId="1" CreationDate="2020-11-29T01:41:06.810" Score="0" ViewCount="15" Body="&lt;p&gt;I am trying to find the best way to provide a neural network with both an image and some annotations about the image. Specifically, I'm creating a network to calculate an approximate 'cost' to go from point a to point b across an image (really a section of SMB1 level, which is basically just a more abstract image).&lt;/p&gt;&#xA;&lt;p&gt;I can generate all the proper training data, but I worry that including an array of pixel data (as is usually done with image recognition) in addition to two (x,y) points in space will make it difficult for the network to associate the arbitrary position values with their respective locations on the image. However, I also worry that adding positional data (aka an additional value for every pixel denoting it as start, end, or just space through which to path) will make the network equally 'confused' (impede its learning). Is there an accepted way to pass small amounts of positional image data to a network so that learning goes smoothly and quickly?&lt;/p&gt;&#xA;&lt;p&gt;Additionally, would this likely lead to any additional network layout constraints? I know using CNNs is the way to go for image recognition/processing of structures that can be in different locations (which is definitely the case here); should something specific in terms of connections be added for whatever extra input neurons will contain the annotation/position data?&lt;/p&gt;&#xA;" OwnerUserId="42604" LastActivityDate="2020-11-29T01:41:06.810" Title="Add Additional Positional Information to Image Classification Neural Network" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;&lt;convolutional-layers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24898" PostTypeId="1" CreationDate="2020-11-29T11:34:10.347" Score="1" ViewCount="20" Body="&lt;p&gt;Capsule Networks use an encoder-decoder structure, where the encoder part consists of the capsule layers (PrimiaryCaps and DigitCaps) and is also the part of the capsule network which performs the actual classification. On the other hand, the decoder attempts to reconstruct the original image from the output of the correct DigitCap. The Decoder in the Capsule Network is used as the regularizer of the network as it helps the network learn better features.&lt;/p&gt;&#xA;&lt;p&gt;I can see how the decoder is helpful for datasets such as MNIST where all image classes have clear differences and the input size if the image is quite small. However, if the input has large dimensions and differences between image classes are quite small, I see the decoder network as overkill, as it will find it hard to reconstruct images for different classes.&lt;/p&gt;&#xA;&lt;p&gt;In my case, my dataset consists of 3D MRI images of patients which have Alzheimer's Disease and those who do not. I am down-sampling the images and producing 8 3D patches which will be used as input to the network. The patches still have high dimensions considering that these are 3D, and there are not many clear differences between patches of the two image classes.&lt;/p&gt;&#xA;&lt;p&gt;My questions here are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How significant is the decoder part of the capsule network? CNNs that perform image classification, usually do not have a decoder part. Why does the capsule network rely on the decoder to learn better features?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any alternatives to the decoder within the capsule network, acting as a regularizer? Can the decoder be ignored completely?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42613" LastEditorUserId="2444" LastEditDate="2020-12-04T13:51:51.477" LastActivityDate="2020-12-04T13:51:51.477" Title="How significant is the decoder part of the capsule network?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;regularization&gt;&lt;capsule-neural-network&gt;&lt;decoders&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24899" PostTypeId="5" CreationDate="2020-11-29T13:48:27.647" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-29T13:48:27.647" LastActivityDate="2020-11-29T13:48:27.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24900" PostTypeId="4" CreationDate="2020-11-29T13:48:27.647" Score="0" Body="For questions about greedy policies in the context of reinforcement learning and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-29T13:48:27.647" LastActivityDate="2020-11-29T13:48:27.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24901" PostTypeId="1" CreationDate="2020-11-29T14:06:48.103" Score="0" ViewCount="30" Body="&lt;p&gt;I am implementing an AI for a mobile checkers game, and have used alpha-beta pruning with Minimax.&lt;/p&gt;&#xA;&lt;p&gt;Now I have the problem of horizontal effect, and need to do Quiesence search to avoid that.&#xA;Any advice on what makes a position volatile for a checkers game?&lt;/p&gt;&#xA;&lt;p&gt;I want to consider the cases when player can take a piece, and also when any piece can be taken by opponent a volatile position, and continue searching for another depth.&#xA;Anything else?&lt;/p&gt;&#xA;" OwnerUserId="42610" LastActivityDate="2020-11-29T15:02:21.493" Title="What is a good way of identifying volatile positions for a checkers game?" Tags="&lt;game-ai&gt;&lt;minimax&gt;&lt;game-theory&gt;&lt;alpha-beta-pruning&gt;&lt;checkers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24902" PostTypeId="2" ParentId="24901" CreationDate="2020-11-29T15:02:21.493" Score="1" Body="&lt;p&gt;When combatting the horizon affect, you want to consider any short term actions that will greatly affect your position evaluation. Thus, in addition to captures, you will also want to include:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When the opponent can make a king next move&lt;/li&gt;&#xA;&lt;li&gt;When the current player can make a king next move&lt;/li&gt;&#xA;&lt;li&gt;When the only legal moves left will lead to capture the turn after for the opponent&lt;/li&gt;&#xA;&lt;li&gt;When the only legal moves left will lead to capture the turn after for the current player&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;3 and 4 are commonly called &lt;a href=&quot;https://en.wikipedia.org/wiki/Zugzwang&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zugzwang&lt;/a&gt; and play a very prominent role in high level checkers, but may be a bit more difficult to implement. However, they will contribute perhaps most of all when combatting the horizon affect.&lt;/p&gt;&#xA;" OwnerUserId="12201" LastActivityDate="2020-11-29T15:02:21.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24903" PostTypeId="2" ParentId="24741" CreationDate="2020-11-29T15:25:04.033" Score="1" Body="&lt;p&gt;I think you are misreading the relevant passage here.&lt;/p&gt;&#xA;&lt;p&gt;Since you do not specify exact excerpt(s), I take that by &amp;quot;&lt;em&gt;implicit assumption&lt;/em&gt;&amp;quot; you refer to the equation (2) (application of a ReLU) and the corresponding text explanation (bold emphasis mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NZXHl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NZXHl.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We apply a ReLU to the linear combination of maps because we are only&#xA;interested in the features that have a &lt;em&gt;positive&lt;/em&gt; influence on the class&#xA;of interest, i.e. pixels whose intensity should be &lt;em&gt;increased&lt;/em&gt; in order&#xA;to increase &lt;span class=&quot;math-container&quot;&gt;$y^c$&lt;/span&gt;. &lt;strong&gt;Negative pixels are likely to belong to other&#xA;categories in the image&lt;/strong&gt;. As expected, without this ReLU, localization maps sometimes highlight more than just the desired class and perform worse at localization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The first thing to notice here is that this choice is not at all about activations close to zero, as you seem to believe,  but about &lt;strong&gt;negative&lt;/strong&gt; ones; and since &lt;em&gt;negative&lt;/em&gt; activations are indeed likely to belong to other categories/classes than the one being &amp;quot;explained&amp;quot; at a given trial, it is very natural to exclude them using a ReLU.&lt;/p&gt;&#xA;&lt;p&gt;Grad-CAM maps are essentially &lt;em&gt;localization&lt;/em&gt; ones; this is apparent already from the paper abstract (emphasis mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Our approach – Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’  in  a classification  network  or  a  sequence  of  words in captioning network) flowing into the final convolutional layer to produce a coarse &lt;strong&gt;localization&lt;/strong&gt; map highlighting the important regions in the image for predicting the concept.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and they are even occasionally referred to as &amp;quot;Grad-CAM &lt;em&gt;localizations&lt;/em&gt;&amp;quot; (e.g. in the caption of Fig. 14); taking a standard example figure from the paper, e.g. this part of Fig. 1:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ldZeR.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ldZeR.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;it is hard to see how including the negative values of the map (i.e. removing the &lt;em&gt;thresholding&lt;/em&gt; imposed by the ReLU) would not lead to maps that include irrelevant parts of the image, hence resulting in a worse localization.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A general remark: while your claim that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;After all, neurons have biases, and a bias can arbitrarily shift the reference point, and hence what 0 means&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;is correct as long as we treat the network as an arbitrary mathematical model, we can no longer treat a &lt;em&gt;trained&lt;/em&gt; network as such. For a trained network (which Grad-CAM is all about), the exact values of both biases &amp;amp; weights matter, and we cannot transform them arbitrarily.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; (after comments):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are you pointing out that it's called &amp;quot;localization&amp;quot; and therefore must be so?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is called &amp;quot;localization&amp;quot; because it &lt;em&gt;is&lt;/em&gt; localization, literally (&amp;quot;look, &lt;em&gt;here&lt;/em&gt; is the &amp;quot;dog&amp;quot; in the picture, &lt;em&gt;not&lt;/em&gt; there&amp;quot;).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I could make a similar challenge &amp;quot;why does positive mean X and why does negative mean Y, and why will this always be true in any trained network?&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is not at all like that; positive means X and negative means not-X &lt;em&gt;in the presence of class X&lt;/em&gt; (i.e. a specific X is present), in the &lt;em&gt;specific&lt;/em&gt; network, and all this in a localization context; notice that Grad-CAM for &amp;quot;dog&amp;quot; is different from the one for &amp;quot;cat&amp;quot; in the picture above.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why is it that a trained network tends to make 0 mean &amp;quot;insignficant&amp;quot; in the deeper layers? [...] why is it that a network should invariably make the positive activations be the ones that support the prediction rather than the negative ones? Couldn't a network just learn it the other way around but use a negative sign on the weights of the final dense layer (so negative flips to positive and thus supports the highest scoring class)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Again, beware of such symmetry/invariability arguments when such symmetries/invariabilities are &lt;em&gt;broken&lt;/em&gt;; and they are indeed broken here for a very simple reason (albeit hidden in the context), i.e. the specific &lt;strong&gt;one-hot encoding&lt;/strong&gt; of the labels: we have encoded &amp;quot;cat&amp;quot; and &amp;quot;dog&amp;quot; as (say) &lt;code&gt;[0, 1]&lt;/code&gt; and &lt;code&gt;[1, 0]&lt;/code&gt; respectively, so, since we are interested in these 1s (which indicate class presence), it makes sense to look for the &lt;em&gt;positive&lt;/em&gt; activations of the (late) convolutional layers. This breaks the positive/negative symmetry. Should we had chosen to encode them as &lt;code&gt;[0, -1]&lt;/code&gt; and &lt;code&gt;[-1, 0]&lt;/code&gt; respectively (&amp;quot;minus-one-hot encoding&amp;quot;), then yes, your argument would hold, and we would be interested in the &lt;em&gt;negative&lt;/em&gt; activations. But since we take the one-hot encoding as given, the problem is no longer symmetric/invariant around zero - by using the specific label encoding, we have actually chosen a side (and thus broken the symmetry)...&lt;/p&gt;&#xA;" OwnerUserId="11539" LastEditorUserId="11539" LastEditDate="2020-11-29T17:08:28.297" LastActivityDate="2020-11-29T17:08:28.297" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24905" PostTypeId="2" ParentId="9033" CreationDate="2020-11-29T16:13:45.230" Score="0" Body="&lt;p&gt;Q-learning is an off-policy learning algorithm. We are following the behaviour policy, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, which is &lt;span class=&quot;math-container&quot;&gt;$\epsilon-$&lt;/span&gt;greedy. This behaviour policy need not be an optimal policy rather it is a more explorable policy. But we are learning the target policy, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, which is argmax of state action value &lt;span class=&quot;math-container&quot;&gt;$(Q(s,a))$&lt;/span&gt;. This target policy is by definition optimal policy.&lt;/p&gt;&#xA;&lt;p&gt;From the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy improvement theorem we can show that for any &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy (I think you are referring to this as a non-optimal policy) we are still making progress towards the optimal policy and when &lt;span class=&quot;math-container&quot;&gt;$\pi^{'}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; that is our optimal policy (Rich Sutton's book Chapter-5). Here &lt;span class=&quot;math-container&quot;&gt;$\pi^{'}$&lt;/span&gt; is the new policy and &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is the previous policy.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fzhlj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fzhlj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Think of this diagram, where we are selecting action based on &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy but still making progress towards the optimal policy &lt;span class=&quot;math-container&quot;&gt;$\pi_*$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="2444" LastEditDate="2020-11-29T17:51:17.687" LastActivityDate="2020-11-29T17:51:17.687" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24910" PostTypeId="1" AcceptedAnswerId="24917" CreationDate="2020-11-29T22:13:42.997" Score="1" ViewCount="54" Body="&lt;p&gt;I'm using tf.Keras to build a deep-fully connected autoencoder. My input dataset is a dataframe with shape (19947,), and the purpose of the autoencoder is to predict normalized gene expression values. They are continuous values that range from [0,~620000].&lt;/p&gt;&#xA;&lt;p&gt;I tried different architectures and I'm using relu activation for all layers. To optimize I'm using adam with mae loss.&lt;/p&gt;&#xA;&lt;p&gt;The problem I have is the network trains successfully (although the train loss is still terrible) but when I'm predicting I notice that although the predictions do make sense for some nodes, there are always a certain number of nodes that only output 0. I've tried changing the number of nodes of my bottleneck layer (output) and it always happens even when I decrease the output number.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas on what I'm doing wrong?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;tf.Keras code:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_layer = keras.Input(shape=(19947,))&#xA;simple_encoder = keras.models.Sequential([&#xA;    input_layer,&#xA;    keras.layers.Dense(512, activation='relu'),&#xA;    keras.layers.Dense(128, activation='relu'),&#xA;    keras.layers.Dense(16, activation='relu')&#xA;])&#xA;simple_decoder = keras.models.Sequential([&#xA;    keras.layers.Dense(128, activation='relu'),&#xA;    keras.layers.Dense(512, activation='relu'),&#xA;    keras.layers.Dense(19947, activation='relu')&#xA;])&#xA;simple_ae = keras.models.Sequential([simple_encoder, simple_decoder])&#xA;simple_ae.compile(optimizer='adam', loss='mae')&#xA;simple_ae.fit(X_train, X_train,&#xA;              epochs=1000,&#xA;              validation_data=(X_valid, X_valid),&#xA;              callbacks=[early_stopping])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Output of encoder.predict with 16 nodes on the bottleneck layer. 7 nodes predict only 0's and 8 nodes predict &amp;quot;correctly&amp;quot;&lt;/em&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rBpEX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rBpEX.png&quot; alt=&quot;Output from encoder.predict with 16 nodes on bottleneck layer&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42631" LastActivityDate="2020-11-30T07:25:07.790" Title="Autoencoder: predictions missing for nodes in the bottleneck layer" Tags="&lt;python&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;autoencoders&gt;&lt;bottlenecks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24911" PostTypeId="1" AcceptedAnswerId="24918" CreationDate="2020-11-29T23:31:02.220" Score="0" ViewCount="51" Body="&lt;p&gt;Q-learning uses the maximizing value at each step, which implies that there is a probability distribution and it happens to choose the one with the highest probability. There is no direct mapping between a particular state to ONLY a particular action but a bunch of actions with varying probabilities. I don't understand.&lt;/p&gt;&#xA;" OwnerUserId="42398" LastEditorUserId="2444" LastEditDate="2020-11-30T20:40:30.997" LastActivityDate="2020-11-30T20:41:44.133" Title="Why is the policy implied by Q-learning deterministic, when it always chooses the action with highest probability?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24912" PostTypeId="5" CreationDate="2020-11-30T00:00:30.080" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-30T00:00:30.080" LastActivityDate="2020-11-30T00:00:30.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24913" PostTypeId="4" CreationDate="2020-11-30T00:00:30.080" Score="0" Body="For questions about mini-batch (or batch) gradient descent, which is gradient descent with typically more than one sample of input-label pairs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-11-30T00:00:30.080" LastActivityDate="2020-11-30T00:00:30.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24914" PostTypeId="1" CreationDate="2020-11-30T00:11:50.097" Score="3" ViewCount="28" Body="&lt;p&gt;We have an image classifier that was built using CNN with faster R-CNN and Yolov5.&lt;/p&gt;&#xA;&lt;p&gt;It is designated to run on 3D objects.&#xA;All of those objects have similar &amp;quot;features&amp;quot; structure, but the actual features of each object class are somewhat different one from another. Therefore, we strive to detect the classes based on those differences in features.&lt;/p&gt;&#xA;&lt;p&gt;In theory there are thousands of different classes, but for now we have trained the model to detect 4 types of classes, by training it on data sets that includes many images from different angles for each of those 4 classes (1,000 images each).&lt;/p&gt;&#xA;&lt;p&gt;The main problem we face is that whenever the model runs on an &amp;quot;unknown&amp;quot; object, it may still classify it as one of our 4 classes, and sometimes it will do it with a high probability score (0.95), which hinders the whole credibility of our model results.&lt;/p&gt;&#xA;&lt;p&gt;We think it might be since we are using SoftMax, which seems to force the model to assign an unknown object to one of the 4 classes.&lt;/p&gt;&#xA;&lt;p&gt;We want to know what will the best way to overcome this issue.&lt;/p&gt;&#xA;&lt;p&gt;We tried adding a new, fifth &amp;quot;trash&amp;quot; class, with 1,000 images of &amp;quot;other&amp;quot; objects that do not belong to our four classes, but it significantly reduced the confidence level for our test images, so we are not sure if this is at all a progress.&lt;/p&gt;&#xA;" OwnerUserId="41590" LastActivityDate="2020-11-30T00:11:50.097" Title="Image classification - Need method to classify &quot;unknown&quot; objects as &quot;trash&quot; (3D objects)" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;classification&gt;&lt;object-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24915" PostTypeId="2" ParentId="24867" CreationDate="2020-11-30T00:21:41.930" Score="1" Body="&lt;p&gt;You can do this similar to the BIDE approach. It can be done like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class TreeNode:&#xA;    def __init__(self, element, depth, count=0, parent=None):&#xA;        self.count= count&#xA;        self.element= element&#xA;        self.depth= depth&#xA;        self.subnodes= dict()&#xA;        self.parent= parent&#xA;        &#xA;    def __repr__(self):&#xA;        return f'{self.__class__.__name__}({self.element}, {self.depth}, {self.count})'&#xA;        &#xA;    def get_subnode(self, element, create=True):&#xA;        if create is True:&#xA;            return self.subnodes.setdefault(element, TreeNode(&#xA;                element, &#xA;                self.depth+1, &#xA;                count=0, &#xA;                parent=self&#xA;            )&#xA;        )&#xA;        else:&#xA;            return self.subnodes.get(element)&#xA;&#xA;    def get_subnode_increasing_count(self, element):&#xA;        subnode= self.get_subnode(element)&#xA;        subnode.count+= 1&#xA;        return subnode&#xA;    &#xA;    def harvest(self, N, m, M, prefix=[], append_result_to=None):&#xA;        # get all sequences which occurs more than N times&#xA;        # and have a length (depth) of m &amp;lt; depth &amp;lt; M&#xA;        prefix= prefix.copy()&#xA;        prefix_len= len(prefix)&#xA;        if append_result_to is None:&#xA;            result= list()&#xA;        else:&#xA;            result= append_result_to&#xA;        if self.element is not None:&#xA;            prefix.append(self.element)&#xA;            prefix_len+= 1&#xA;        if N &amp;gt;= self.count or prefix_len &amp;gt;= M:&#xA;            return result&#xA;        if N &amp;lt; self.count and m &amp;lt; prefix_len and prefix_len &amp;lt; M:&#xA;            # the subsequence matches the constraints&#xA;            result.append(prefix)&#xA;        for subnode in self.subnodes.values():&#xA;            subnode.harvest(&#xA;                N, &#xA;                m, &#xA;                M, &#xA;                prefix=prefix, &#xA;                append_result_to=result&#xA;            )&#xA;        return result&#xA;        &#xA;    def get_prefix(self):&#xA;        if self.parent is None:&#xA;            prefix= []&#xA;        else:&#xA;            prefix= self.parent.get_prefix()&#xA;        if self.element is not None:&#xA;            prefix.append(self.element)&#xA;        return prefix&#xA;        &#xA;    def print_tree_counts(self, leaves_only=False, min_count=0):&#xA;        if leaves_only is False or not self.subnodes:&#xA;            if self.count &amp;gt;= min_count:&#xA;                print(self.get_prefix(), self.count)&#xA;        for subnode in self.subnodes.values():&#xA;            subnode.print_tree_counts(leaves_only=leaves_only, min_count=min_count)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;sequential approach&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;def find_patterns(S, N, m, M):&#xA;    root= TreeNode(None, 0, 0)&#xA;    active_nodes= []&#xA;    for el in S:&#xA;        root.count+=1&#xA;        # append the root node&#xA;        active_nodes.append(root)&#xA;        # now replace all nodes in active nodes by the node&#xA;        # that is reached one level deeper following el&#xA;        active_nodes= [node.get_subnode_increasing_count(el) for node in active_nodes if node.depth &amp;lt; M]&#xA;&#xA;    # now harvest the tree after applying the restrictions&#xA;    return root, root.harvest(N, m, M)&#xA;&#xA;S='AAFFAABDEFEDFFAAFCCABDDCCCABDDCACBEABCFFAAEABCADEFFAABCDAEABCABDD'&#xA;root, patterns= find_patterns(S, 3, 3, 6)&#xA;patterns&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[['F', 'F', 'A', 'A']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Your second pattern occurs exactly 3 times and so doesn't fullfill the requirement of &amp;gt; 3 occurances ([C,A,B,D,D]).&lt;/p&gt;&#xA;&lt;h3&gt;modifications for parallel processing&lt;/h3&gt;&#xA;&lt;p&gt;To make it processable in parallel you can do a slide modification. Just create another method in TreeNode, that allows to merge nodes. Like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def merge_nodes(self, other_nodes):&#xA;    # merge other_nodes into this node&#xA;    # including all subnodes&#xA;    if len(other_nodes) &amp;gt; 0:&#xA;        elements= set()&#xA;        for other_node in other_nodes:&#xA;            self.count+= other_node.count&#xA;            elements.update(other_node.subnodes.keys())&#xA;        # elements now contains the set of the next elements &#xA;        # with which the sequence continues across the &#xA;        # other nodes&#xA;        for element in elements:&#xA;            # get the node of the resulting tree that represents&#xA;            # the sequnce continuing with element, if there is&#xA;            # no such subnode, create one, since there is at least&#xA;            # one other node that counted sequence seq + element&#xA;            my_subnode= self.get_subnode(element, create=True)&#xA;            other_subnodes= list()&#xA;            for other_node in other_nodes:&#xA;                # now get the subnode for each other node, that&#xA;                # represents the same sequence (seq + element)&#xA;                other_subnode= other_node.get_subnode(element, create=False)&#xA;                if other_subnode is not None:&#xA;                    other_subnodes.append(other_subnode)&#xA;            # merge the subnode the same way&#xA;            my_subnode.merge_nodes(other_subnodes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you can call find_patterns on subtrees, but you need to take into account, that you may not split the input sequence normally. You need to define some overlapping sequence part, so that patterns, that begin at the end of one sequence-fragment can be completed but that this overlapping sequences need to be counted differently (otherwise you get double-counts, which lead to wrong inputs). So you have to make sure, that only patterns, which began in the sequence fragment are continued with the overlap part, but that no new patterns are started in the overlap part, because you count them elsewere:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def find_patterns(S, N, m, M, overlap=[]):&#xA;    root= TreeNode(None, 0, count=0, parent=None)&#xA;    active_nodes= []&#xA;    def process_element(active_nodes, element, M):&#xA;        return [&#xA;            node.get_subnode_increasing_count(element) &#xA;                for node in active_nodes &#xA;                    if node.depth &amp;lt; M&#xA;        ]&#xA;    for el in S:&#xA;        root.count+=1&#xA;        # append the root node&#xA;        active_nodes.append(root)&#xA;        # now replace all nodes in active nodes by the node&#xA;        # that is reached one level deeper following el&#xA;        active_nodes= process_element(active_nodes, el, M)&#xA;    # complete the already started sequences with the&#xA;    # overlapping sequence (the sequence that may be&#xA;    # processed in another process)&#xA;    for el in overlap:&#xA;        active_nodes= process_element(active_nodes, el, M)&#xA;    # now harvest the tree after applying the restrictions&#xA;    return root, root.harvest(N, m, M)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;split_point= len(S) // 2&#xA;S1= S[split_point:]&#xA;S2= S[:split_point]&#xA;S1_overlap= S2[:10] &#xA;# in fact you could have just used :5 above, &#xA;# but that doesn't affect the result&#xA;r1, _= find_patterns(S1, 3, 3, 6, overlap=S1_overlap)&#xA;# Note: you could safely start the comand in the next &#xA;# line in parallel to the previous call of find_patterns&#xA;# and of course, you could split your sequence in as&#xA;# many fragments, as you like, as long as you maintain&#xA;# the overlap correctly and call merge_nodes afterwards&#xA;r2, _= find_patterns(S2, 3, 3, 6) &#xA;r1.merge_nodes([r2])&#xA;patterns= r1.harvest(3, 3, 6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This yields the same result as above. To make it a bit clearer, what I mean with the overlap part:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S1 is 'CBEABCFFAAEABCADEFFAABCDAEABCABDD'&#xA;S2 is 'AAFFAABDEFEDFFAAFCCABDDCCCABDDCA'&#xA;S1_overlap is 'AAFFAABDEF'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With &lt;code&gt;find_patterns(S1, 3, 3, 6)&lt;/code&gt; you would only search for repeated patterns in S1, so it would not consider patterns which begin with the part BDD (starting at the end of S1) and are continued within S2. With &lt;code&gt;find_patterns(S1, 3, 3, 6, overlap=S1_overlap)&lt;/code&gt; I consider these patterns as well.&lt;/p&gt;&#xA;" OwnerUserId="26402" LastEditorUserId="26402" LastEditDate="2020-11-30T12:37:34.780" LastActivityDate="2020-11-30T12:37:34.780" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24916" PostTypeId="2" ParentId="24788" CreationDate="2020-11-30T03:53:11.140" Score="1" Body="&lt;p&gt;The state of the art in text generation is the GPT model. &lt;a href=&quot;https://en.wikipedia.org/wiki/GPT-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT-3&lt;/a&gt;, which was just released in summer of 2020, has been used to generate many very impressive articles, and is widely considered the best text generation model. &lt;a href=&quot;https://www.technologyreview.com/2020/07/20/1005454/openai-machine-learning-language-generator-gpt-3-nlp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; article and &lt;a href=&quot;https://www.technologyreview.com/2020/10/08/1009845/a-gpt-3-bot-posted-comments-on-reddit-for-a-week-and-no-one-noticed/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one should give you an example of how powerful it is at text generation.&lt;/p&gt;&#xA;&lt;p&gt;GPT is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer&lt;/a&gt; based architecture, somewhat like &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt;. The main difference is it only takes into account left context, which is why it is so well suited for text generation.&lt;/p&gt;&#xA;&lt;p&gt;GPT-3 is still very new and is not available for free. However, GPT-2, the previous release of the model, is available for free. While obviously not as advanced as GPT-3, it is still quite impressive in its own right, and for someone trying to generate text, it is the clear choice.&lt;/p&gt;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://minimaxir.com/2019/09/howto-gpt2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to a tutorial explaining the basics to get you started with GPT-2.&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in diving into the relevant research papers:&lt;/p&gt;&#xA;&lt;p&gt;Here are the open AI papers on GPT 1-3&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improving Language Understanding by Generative Pre-Training&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Language Models are Unsupervised Multitask Learners&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/2005.14165&quot; rel=&quot;nofollow noreferrer&quot;&gt;Language Models are Few-Shot Learners&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Additionally, if you have never seen transformers before, take a look at:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1706.03762&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12201" LastActivityDate="2020-11-30T03:53:11.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24917" PostTypeId="2" ParentId="24910" CreationDate="2020-11-30T07:25:07.790" Score="0" Body="&lt;p&gt;My closest guess is because you are using the activation function ReLU, which pushes data to be greater than zero. However, because of your data's nature, the autoencoder is highly dependent on negative calculations to reconstruct your data, but the best it can achieve is zero.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the context of artificial neural networks, the rectifier is an activation function defined as the positive part of its argument:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = x^+ = max(0, x)$$&lt;/span&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Rectifier_(neural_networks)&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35757" LastActivityDate="2020-11-30T07:25:07.790" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="24918" PostTypeId="2" ParentId="24911" CreationDate="2020-11-30T07:59:37.230" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Q-learning uses the maximizing value at each step,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Mostly true. The &lt;em&gt;target&lt;/em&gt; policy that Q-learning learns the action values of is the one with the maximum value. While training, Q-learning will take one action randomly (typically with a high probability of taking the action with maximum value), whilst it makes updates to value estimates assuming it will always take the maximising action next.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;which implies that there is a probability distribution and it happens to choose the one with the highest probability.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Not true. This is not implied at all. The action values that Q-learning estimates are not probabilities, but expected sums of future reward. The probability distribution for &lt;em&gt;exploring&lt;/em&gt; actions is added outside of the Q table or Q function estimate. There is no implied probability distribution of actions in the target policy. When implementing the target policy, you may decide to break ties randomly, but that is not an important detail.&lt;/p&gt;&#xA;&lt;p&gt;It is worth noting that even &lt;em&gt;if&lt;/em&gt; you were working with a table of action probabilities, then an agent that always chose the one with the maximum probability would be deterministic. A list of numbers from a table or non-stochastic function (which is how a Q table or Q function is implemented, and also how policy functions are implemented for methods that do process probabilities), even if it represents probabilities, cannot be random in itself. Instead, it must be interpreted by a process that decides how to use them. A process that includes a random number generator can use the numbers to generate a probability distribution that it samples from.&lt;/p&gt;&#xA;&lt;p&gt;In Q-learning, the process that updates the Q table estimates does not include a random number generator, so as a consequence it is deterministic. However, the choice of which actions to explore is often random, and it is sometimes the case that the environment includes randomness in state transitions or reward values. So Q-learning taken as a whole is a random process.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-11-30T20:41:44.133" LastActivityDate="2020-11-30T20:41:44.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24919" PostTypeId="1" CreationDate="2020-11-30T08:43:50.160" Score="2" ViewCount="423" Body="&lt;p&gt;I'm having a problem when loading a model in keras:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = load_model('model.h5', custom_objects={'mean_iou': metrics.mean_iou})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As an error I get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   ~/anaconda3/envs/idcard/lib/python3.6/site-packages/keras/engine/saving.py in load_model(filepath, custom_objects, compile)&#xA;    417     f = h5dict(filepath, 'r')&#xA;    418     try:&#xA;--&amp;gt; 419         model = _deserialize_model(f, custom_objects, compile)&#xA;    420     finally:&#xA;    421         if opened_new_file:&#xA;&#xA;~/anaconda3/envs/idcard/lib/python3.6/site-packages/keras/engine/saving.py in _deserialize_model(f, custom_objects, compile)&#xA;    249         layer_weights = model_weights_group[name]&#xA;    250         weight_names = layer_weights['weight_names']&#xA;--&amp;gt; 251         if weight_names:&#xA;    252             filtered_layer_names.append(name)&#xA;    253 &#xA;&#xA;ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone have a clue why I'm getting this error?&lt;/p&gt;&#xA;" OwnerUserId="38093" LastActivityDate="2020-11-30T09:02:41.177" Title="The truth value of an array with more than one element is ambiguous - Loading a model saved in h5 format keras 2.2.4" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;models&gt;&lt;u-net&gt;" AnswerCount="1" CommentCount="1" ClosedDate="2020-11-30T10:33:06.570" ContentLicense="CC BY-SA 4.0" />
  <row Id="24920" PostTypeId="2" ParentId="24919" CreationDate="2020-11-30T08:45:59.460" Score="2" Body="&lt;p&gt;It turns out that the solution to this problem is in the version of &lt;strong&gt;h5py&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you have h5py == 3.1.* then you need to downgraded it to &lt;em&gt;&lt;strong&gt;h5py == 2.10.0&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Version of Keras needs to be 2.2.4&lt;/p&gt;&#xA;&lt;p&gt;Or you can upgrade the version of Keras to &lt;strong&gt;2.2.5&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="38093" LastEditorUserId="38093" LastEditDate="2020-11-30T09:02:41.177" LastActivityDate="2020-11-30T09:02:41.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24921" PostTypeId="1" AcceptedAnswerId="24969" CreationDate="2020-11-30T12:35:42.677" Score="3" ViewCount="140" Body="&lt;p&gt;The &lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; book by Goodfellow et al. states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Convolutional networks stand out as an example of neuroscientiﬁc principles inﬂuencing deep learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Are convolutional neural networks (CNNs) really inspired by the human brain?&lt;/p&gt;&#xA;&lt;p&gt;If so, how? In particular, what structures within the brain do CNN-like neuron groupings occur?&lt;/p&gt;&#xA;" OwnerUserId="37533" LastEditorUserId="2444" LastEditDate="2020-12-03T13:10:37.943" LastActivityDate="2020-12-04T13:17:52.957" Title="Are convolutional neural networks inspired by the human brain?" Tags="&lt;convolutional-neural-networks&gt;&lt;brain&gt;&lt;neuroscience&gt;&lt;neocognitron&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24923" PostTypeId="1" CreationDate="2020-11-30T14:29:28.393" Score="1" ViewCount="59" Body="&lt;p&gt;I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like &lt;span class=&quot;math-container&quot;&gt;$(a-b)^2 = a^2 -2ab +b^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OgQGk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OgQGk.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27601" LastEditorUserId="27601" LastEditDate="2020-11-30T14:50:14.410" LastActivityDate="2021-04-29T17:12:41.417" Title="Explanation of this L2 minimization equation" Tags="&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;math&gt;&lt;loss-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24924" PostTypeId="2" ParentId="24923" CreationDate="2020-11-30T15:05:50.437" Score="0" Body="&lt;p&gt;As in the second line, the first two terms are &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\hat{y}}$&lt;/span&gt;, it means the variable of the expectation is &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_i$&lt;/span&gt; and you can take out &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;s from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}_{\hat{y}}$&lt;/span&gt;s. Now we can use &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}\{\hat{y}_i\} = y_i$&lt;/span&gt; and rewrite the terms of the second line likes the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbb{E}_{\hat{y}}(\sum_i y_i)^2 = (\sum_i y_i)^2 \\&#xA;\mathbb{E}_{\hat{y}}\left[(\sum_i y_i)(\sum_i \hat{y}_i)\right] = (\sum_i y_i) \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i) = (\sum_i y_i)^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The second line is written based on linearity of the expectation and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}\{\hat{y}_i\} = y_i$&lt;/span&gt;. Hence, we can rewrite the second line like the following as &lt;span class=&quot;math-container&quot;&gt;$\sum_i y_i = \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the final step is using this formula &lt;span class=&quot;math-container&quot;&gt;$Var(X) = E(X^2) - (E(X))^2$&lt;/span&gt;  and take &lt;span class=&quot;math-container&quot;&gt;$X = \sum_i \hat{y}_i$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right] = \frac{1}{N^2} Var(\sum_i \hat{y}_i)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;From variance to covariance, you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Variance&quot; rel=&quot;nofollow noreferrer&quot;&gt;this formula&lt;/a&gt;: &lt;span class=&quot;math-container&quot;&gt;$$Var(\sum_{i=1}^nX_i) = \sum_{i=1}^n\sum_{j=1}^n cov(X_i, X_j)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-11-30T15:12:27.343" LastActivityDate="2020-11-30T15:12:27.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24925" PostTypeId="1" AcceptedAnswerId="24960" CreationDate="2020-11-30T16:30:14.643" Score="1" ViewCount="59" Body="&lt;p&gt;I was going through &lt;a href=&quot;https://arxiv.org/pdf/1502.05477.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the TRPO paper&lt;/a&gt;, and there was a line under Appendix D &amp;quot;Approximating Factored Policies with Neural Networks&amp;quot; in the last paragraph which I am unable to understand&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The action consists of a tuple &lt;span class=&quot;math-container&quot;&gt;$(a_1, a_2..... , a_K)$&lt;/span&gt; of integers &lt;span class=&quot;math-container&quot;&gt;$a_k\in\{1, 2,......,N_k\} $&lt;/span&gt; and each of these components is assumed to have a categorical distribution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I can't seem to get how each component has a categorical distribution. I think it should be the tuple that has a categorical distribution.&#xA;I think I am getting something wrong.&lt;/p&gt;&#xA;" OwnerUserId="42644" LastEditorUserId="2444" LastEditDate="2020-12-01T22:24:49.530" LastActivityDate="2020-12-02T22:24:28.377" Title="Why does each component of the tuple that represents an action have a categorical distribution in the TRPO paper?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;trust-region-policy-optimization&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24927" PostTypeId="1" AcceptedAnswerId="24928" CreationDate="2020-11-30T23:13:07.090" Score="2" ViewCount="38" Body="&lt;p&gt;Why is it useful to define the return as the sum of the rewards from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; onward rather than up to &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;The return for an MDP is usually defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t=R_{t+1}+R_{t+2}+ \dots +R_T$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why is this defined as the return? Is there anything useful about this?&lt;/p&gt;&#xA;&lt;p&gt;It seems like it's more useful to define the return as &lt;span class=&quot;math-container&quot;&gt;$$G_t=R_0+ \dots+R_t,$$&lt;/span&gt; because your &amp;quot;return&amp;quot;, so to speak, is the &amp;quot;profit from investment&amp;quot; so it seems like your return will be your accumulated reward from taking actions up to that point.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-12-02T10:14:39.360" LastActivityDate="2020-12-02T10:14:39.360" Title="Why is it useful to define the return as the sum of the rewards from time $t$ onward rather than up to $t$?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;markov-decision-process&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24928" PostTypeId="2" ParentId="24927" CreationDate="2020-11-30T23:35:40.440" Score="3" Body="&lt;p&gt;It wouldn't make sense to define the return as you propose, from time 0 to &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Once we are in a state at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; we don't care what the returns &lt;em&gt;have been&lt;/em&gt;, rather what they will be in the future, thus returns are defined as the total sum of discounted rewards from the current time step onwards. This allows the agent to make decisions about which actions to take based on how valuable taking said action is in the current state at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; -- clearly the rewards previous to this have no effect upon that.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-12-01T00:50:05.607" LastActivityDate="2020-12-01T00:50:05.607" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24929" PostTypeId="1" AcceptedAnswerId="24932" CreationDate="2020-11-30T23:36:00.763" Score="1" ViewCount="39" Body="&lt;p&gt;For episodic tasks with an absorbing state, why can't &lt;span class=&quot;math-container&quot;&gt;$\gamma=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T= \infty$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;In Sutton and Barto's book, they say that, for episodic tasks with absorbing states that becomes an infinite sequence, then the return  is defined by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t=\sum_{k=t+1}^{T}\gamma^{k-t-1}R_k$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This allows the return to be the same whether the sum is over the first &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; rewards, where &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is the time of termination or over the full infinite sequence, with &lt;span class=&quot;math-container&quot;&gt;$T=\infty$&lt;/span&gt; xor &lt;span class=&quot;math-container&quot;&gt;$\gamma=1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why can't we have both? I don't see how they can both be set to those parameters. It seems like, if you have an absorbing state, the rewards from terminal onward will just be 0 and not be affected by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here's the full section of the book on page 57 in the 2nd edition&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cI2ih.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cI2ih.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think the reasoning behind this also leads to why for policy evaluation where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s)=\sum_a\pi(a|s)\sum_{s',r}p(s',r|s,a)[r+\gamma v_\pi(s')]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Has an existence and uniqueness guarantee only if &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt; or termination is guaranteed under &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;&amp;quot;(page 74). This part I'm also a bit confused by, but seems related.&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-12-01T01:21:16.580" LastActivityDate="2020-12-01T01:42:14.213" Title="For episodic tasks with an absorbing state, why can't we both have $\gamma=1$ and $T= \infty$ in the definition of the return?" Tags="&lt;reinforcement-learning&gt;&lt;hyper-parameters&gt;&lt;return&gt;&lt;discount-factor&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24931" PostTypeId="1" CreationDate="2020-12-01T01:16:52.887" Score="0" ViewCount="15" Body="&lt;p&gt;I trained a dataset with a 5-fold cross validation to search for hyper-parameters by an AUROC metric. For splitting I used&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;five_splits = list(StratifiedKFold(n_splits=5, shuffle=True).split(X, y))&lt;/code&gt; and looped through each fold.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T0043.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T0043.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I trained 80% of the dataset with optimal (for now) hyper-parameters from 5-fold CV, and plotted the AUROC on 20% of the dataset. For splitting I used&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L2Iif.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L2Iif.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't think that shuffling of the samples or randomness may cause such a great gap between the scores.&#xA;Also, the pre-processing and any other operations are identical in each training. I repeated this training for several times to no avail.&lt;/p&gt;&#xA;&lt;p&gt;I can't seem to find the reason for what the cause of this &lt;strong&gt;~0.06&lt;/strong&gt; difference might be.&lt;/p&gt;&#xA;" OwnerUserId="31870" LastActivityDate="2020-12-01T01:16:52.887" Title="Why does 0.8:0.2 divided dataset have a much greater AUROC than its 5-fold cross validated counterpart's mean AUROC?" Tags="&lt;neural-networks&gt;&lt;multilayer-perceptron&gt;&lt;cross-validation&gt;&lt;roc-auc&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24932" PostTypeId="2" ParentId="24929" CreationDate="2020-12-01T01:42:14.213" Score="3" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$T = \infty$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; cannot be both true at the same time because the return defined in equation 3.11 is supposed to be a &lt;em&gt;unified&lt;/em&gt; definition of the return for &lt;em&gt;both&lt;/em&gt; continuing and episodic tasks. In the case of continuing tasks, &lt;span class=&quot;math-container&quot;&gt;$T = \infty$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma = 1$&lt;/span&gt; cannot be true at the same time, because the return may not be finite in that case (as I think you already understood).&lt;/p&gt;&#xA;&lt;p&gt;Moreover, note that, in that &lt;em&gt;specific&lt;/em&gt; example of the book, they assume that the agent ends up in an absorbing state, so this specific sum is finite, no matter whether &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is finite or &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;, given that, once you enter the absorbing state, you will always get a reward of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Of course, if you discount those specific rewards, the sum will still be finite. However, in general, if you had a different MDP where the absorbing state is not reachable (i.e. the episode never ends), then the return could not be finite.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-12-01T01:42:14.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24933" PostTypeId="1" CreationDate="2020-12-01T02:13:53.703" Score="1" ViewCount="38" Body="&lt;p&gt;In a lot of explanations online for Xavier Initialization, I see the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a high value or vanishing to zero. In other words, we need to initialize the weights in such a way that the variance remains the same for x and y. This initialization process is known as Xavier initialization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Source &lt;a href=&quot;https://prateekvjoshi.com/2016/03/29/understanding-xavier-initialization-in-deep-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://prateekvjoshi.com/2016/03/29/understanding-xavier-initialization-in-deep-neural-networks/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, the intuition behind why var(output) should equal var(inputs) is never explained. Does anyone know why intuitively var(output) should equal var(inputs)?&lt;/p&gt;&#xA;" OwnerUserId="26159" LastEditorUserId="2444" LastEditDate="2020-12-01T13:10:31.307" LastActivityDate="2020-12-01T13:10:31.307" Title="Why should variance(output) equal variance(input) in Xavier Initialisation?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;feedforward-neural-network&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24935" PostTypeId="1" CreationDate="2020-12-01T09:53:51.590" Score="1" ViewCount="16" Body="&lt;p&gt;I'm building a tool that should assist a director to broadcast a racing game. I want this tool to suggest the human director which car to focus on and with which camera (among the available ones). I can access quite a lot of data about the current race so I can extrapolate some parameters(like car positions, how many cars near to each other there are, how close they are, last time the camera was switched etc) to be used in the decision making process. I would like the AI to learn from the human director in order to suggest him according to his &amp;quot;direction style&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My idea is to split the problem in 2 sub problems: the first is the choice of the car to focus on and the second is the choice of the camera to use (or cameras, since is fairly common to switch cameras while following the same car). My plan was to use some sort of Q-learning, rewarding the AI whenever one of the generated suggestions is chosen by the director but I guess it would be really difficult to define a set of states and moreover it would probably take ages before it would start to give some useful suggestions.&lt;/p&gt;&#xA;&lt;p&gt;Are there some other good approaches I could consider? I'm also thinking about using a neural network so maybe the learning process would be faster.&lt;/p&gt;&#xA;" OwnerUserId="42648" LastActivityDate="2020-12-01T09:53:51.590" Title="Looking for a good approach for building an automated director for a racing game spectator mode" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24937" PostTypeId="2" ParentId="22734" CreationDate="2020-12-01T11:54:07.470" Score="0" Body="&lt;p&gt;After doing many runs with GPT-2 355M I come to the conclusion that anything below 20k tokens yield worse and worse results the less tokens you have regardless of how many steps you train for. For my application the sweet spot is around 30-35k tokens.&lt;/p&gt;&#xA;" OwnerUserId="38955" LastActivityDate="2020-12-01T11:54:07.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24938" PostTypeId="1" CreationDate="2020-12-01T11:55:42.913" Score="1" ViewCount="80" Body="&lt;p&gt;I am trying to implement an Actor Critic method that controls an RC car. For this I have implemented a simulated environment and actor critic tensorflowjs models.&lt;/p&gt;&#xA;&lt;p&gt;My intention is to train a model to navigate an environment without colliding with various obstacles.&lt;/p&gt;&#xA;&lt;p&gt;For this I have the following:&lt;/p&gt;&#xA;&lt;p&gt;State(continuous):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the sensors distance(left, middle, right): [0..1,0..1,0..1]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Action(discrete):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;4 possible actions(move forward, move back, turn left, turn right)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Reward(cumulative):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;moving forward is encouraged&lt;/li&gt;&#xA;&lt;li&gt;being close to an obstacle is penalized&lt;/li&gt;&#xA;&lt;li&gt;colliding with an obstacle is penalized&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The structure of the models:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;buildActor() {&#xA;      const model = tf.sequential();&#xA;      model.add(tf.layers.inputLayer({inputShape: [this.stateSize],}));&#xA;&#xA;      model.add(tf.layers.dense({&#xA;        units: parseInt(this.config.hiddenUnits),&#xA;        activation: 'relu',&#xA;        kernelInitializer: 'glorotUniform',&#xA;      }));&#xA;&#xA;      model.add(tf.layers.dense({&#xA;        units: parseInt(this.config.hiddenUnits/2),&#xA;        activation: 'relu',&#xA;        kernelInitializer: 'glorotUniform',&#xA;      }));&#xA;&#xA;      model.add(tf.layers.dense({&#xA;        units: this.actionSize,&#xA;        activation: 'softmax',&#xA;        kernelInitializer: 'glorotUniform',&#xA;      }));&#xA;&#xA;      this.compile(model, this.actorLearningRate);&#xA;&#xA;      return model;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;buildCritic() {&#xA;      const model = tf.sequential();&#xA;&#xA;      model.add(tf.layers.inputLayer({inputShape: [this.stateSize],}));&#xA;&#xA;      model.add(tf.layers.dense({&#xA;        units: parseInt(this.config.hiddenUnits),&#xA;        activation: 'relu',&#xA;        kernelInitializer: 'glorotUniform',&#xA;      }));&#xA;&#xA;      model.add(tf.layers.dense({&#xA;        units: parseInt(this.config.hiddenUnits/2),&#xA;        activation: 'relu',&#xA;        kernelInitializer: 'glorotUniform',&#xA;      }));&#xA;&#xA;      model.add(tf.layers.dense({&#xA;        units: this.valueSize,&#xA;        activation: 'linear',&#xA;        kernelInitializer: 'glorotUniform',&#xA;      }));&#xA;&#xA;      this.compile(model, this.criticLearningRate);&#xA;&#xA;      return model;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The models are compiled with an adam optimized and huber loss:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;compile(model, learningRate) {&#xA;      model.compile({&#xA;        optimizer: tf.train.adam(learningRate),&#xA;        loss: tf.losses.huberLoss,&#xA;      });&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Training:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trainModel(state, action, reward, nextState) {&#xA;      let advantages = new Array(this.actionSize).fill(0);&#xA;&#xA;      let normalizedState = normalizer.normalizeFeatures(state);&#xA;      let tfState = tf.tensor2d(normalizedState, [1, state.length]);&#xA;      let normalizedNextState = normalizer.normalizeFeatures(nextState);&#xA;      let tfNextState = tf.tensor2d(normalizedNextState, [1, nextState.length]);&#xA;&#xA;      let predictedCurrentStateValue = this.critic.predict(tfState).dataSync();&#xA;      let predictedNextStateValue = this.critic.predict(tfNextState).dataSync();&#xA;&#xA;      let target = reward + this.discountFactor * predictedNextStateValue;&#xA;      let advantage = target - predictedCurrentStateValue;&#xA;      advantages[action] = advantage;&#xA;      // console.log(normalizedState, normalizedNextState, action, target, advantages);&#xA;&#xA;      this.actor.fit(tfState, tf.tensor([advantages]), {&#xA;        epochs: 1,&#xA;      }).then(info =&amp;gt; {&#xA;          this.latestActorLoss = info.history.loss[0];&#xA;          this.actorLosses.push(this.latestActorLoss);&#xA;        }&#xA;      );&#xA;&#xA;      this.critic.fit(tfState, tf.tensor([target]), {&#xA;        epochs: 1,&#xA;      }).then(info =&amp;gt; {&#xA;          this.latestCriticLoss = info.history.loss[0];&#xA;          this.criticLosses.push(this.latestCriticLoss);&#xA;        }&#xA;      );&#xA;&#xA;      this.advantages.push(advantage);&#xA;      pushToEvolutionChart(this.epoch, this.latestActorLoss, this.latestCriticLoss, advantage);&#xA;      this.epoch++;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You ca give the simulation a spin on &lt;a href=&quot;https://sergiuionescu.github.io/esp32-auto-car/sym/sym.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sergiuionescu.github.io/esp32-auto-car/sym/sym.html&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;I found that some behaviors are being picked up - the model learns to prioritize moving forward after a few episodes, but then hits the wall and it reprioritizes spinning - but seems to completely 'forget' that moving forward was ever prioritized.&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to follow &lt;a href=&quot;https://keras.io/examples/rl/actor_critic_cartpole/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://keras.io/examples/rl/actor_critic_cartpole/&lt;/a&gt; to a certain extent, but have not found an equivalent of the way back-propagation is handled there - GradientTape.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to perform training similar to the Keras example in Tensorflowjs?&lt;/p&gt;&#xA;&lt;p&gt;The theory i've went through on Actor Critic mentions that the Critic should estimate the reward yet to be obtain until the rest of the episode, but i am training the critic with:&#xA;&lt;code&gt;reward + this.discountFactor * predictedNextStateValue&lt;/code&gt; where &lt;strong&gt;reward&lt;/strong&gt; is the cumulative reward until the current step.&#xA;Should i keep track of a maximum total reward in previous episodes and subtract my reward from that instead?&lt;/p&gt;&#xA;&lt;p&gt;When i am training the actor i am generating a zero filled advantages tensor:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;let advantages = new Array(this.actionSize).fill(0);&#xA;let target = reward + this.discountFactor * predictedNextStateValue;&#xA;let advantage = target - predictedCurrentStateValue;&#xA;advantages[action] = advantage;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All other actions than the taken one will receive a 0 advantage. Could this discourage any previous actions the were proven beneficial?&#xA;Should i average out the advantages per state and action?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for having the patience to go trough all of this.&lt;/p&gt;&#xA;" OwnerUserId="42665" LastEditorUserId="42665" LastEditDate="2020-12-05T19:15:05.993" LastActivityDate="2021-05-04T21:09:30.927" Title="Advantage Actor Critic model implementation with Tensorflowjs" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24939" PostTypeId="2" ParentId="23703" CreationDate="2020-12-01T14:59:08.630" Score="1" Body="&lt;p&gt;If you're using a library such as &lt;a href=&quot;https://github.com/google/trax&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trax&lt;/a&gt; which contains great submodules for various Transformers (Skipping, BERT, Vanilla and Reformer) you can use the inbuilt &lt;code&gt;trax.data.inputs.add_loss_weights()&lt;/code&gt; function and provide a value for the &lt;code&gt;id_to_mask&lt;/code&gt; parameter.&lt;/p&gt;&#xA;&lt;p&gt;Example Usage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;train_generator = trax.data.inputs.add_loss_weights(&#xA;data_generator(batch_size, x_train, y_train,vocab['&amp;lt;PAD&amp;gt;'], True),&#xA;id_to_mask=vocab['&amp;lt;PAD&amp;gt;'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are some resources for building Transformers in Trax:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/SauravMaheshkar/Trax-Examples/blob/main/NLP/NER%20using%20Reformer.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Named Entity Recognition with Reformer&lt;/a&gt; (Masking Implemented) and the associated &lt;a href=&quot;https://sauravmaheshkar.github.io/fastpages/transformers/jupyter/nlp/2020/11/29/_11_19_NER_Using_Reformer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; (contains an in-depth explanation of NER and Transformers)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/SauravMaheshkar/Trax-Examples/blob/main/NLP/Deep%20N-Gram.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep N-Gram Models&lt;/a&gt; (without masking but I recommend you go through these first to get a hang of the library)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="40434" LastActivityDate="2020-12-01T14:59:08.630" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24940" PostTypeId="1" CreationDate="2020-12-01T15:09:19.533" Score="0" ViewCount="28" Body="&lt;p&gt;I have a bit of difficulties working on the article :&lt;a href=&quot;https://arxiv.org/pdf/2009.07047v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/2009.07047v1.pdf&lt;/a&gt;. I want to implement the &lt;code&gt;VAE_1&lt;/code&gt;, but it used VAE-GAN. I am familiar with VAE and GAN separetely, but a mixture of VAE-GAN looks quite challenging to implement. To better understand a VAE-GAN, I observe the link &lt;a href=&quot;https://pravn.wordpress.com/category/vae-gan-vaegan/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pravn.wordpress.com/category/vae-gan-vaegan/&lt;/a&gt;, but they used an auxiliary network called &lt;code&gt;Aux()&lt;/code&gt;. I am not sure to understand the link with Aux() and what is in the article &lt;a href=&quot;https://arxiv.org/pdf/2009.07047v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/2009.07047v1.pdf&lt;/a&gt; at the point &lt;strong&gt;Domain alignment in the VAE latent space&lt;/strong&gt;. Do I have to use such auxiliary network to implement &lt;code&gt;VAE_1&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="42628" LastEditorUserId="42628" LastEditDate="2020-12-01T17:56:14.923" LastActivityDate="2020-12-01T17:56:14.923" Title="Why using Aux() to implement VAE_1?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24941" PostTypeId="1" CreationDate="2020-12-01T15:34:43.987" Score="1" ViewCount="17" Body="&lt;p&gt;Can anyone help me understand these functions described in the paper &lt;a href=&quot;https://arxiv.org/abs/1803.04189&quot; rel=&quot;nofollow noreferrer&quot;&gt;Noise2Noise: Learning Image Restoration without Clean Data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have read the portion A.4 in the appendix but need a more detailed and easier to understand explanation. Especially where signum or sign function (sgn) is coming from along with other explanations.&lt;/p&gt;&#xA;&lt;p&gt;(10),(11),(12)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rf9hc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rf9hc.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27601" LastEditorUserId="2444" LastEditDate="2020-12-01T16:32:21.030" LastActivityDate="2020-12-01T16:32:21.030" Title="What is the intuition behind equations 10, 11 and 12 of the paper &quot;Noise2Noise: Learning Image Restoration without Clean Data&quot;?" Tags="&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;papers&gt;&lt;activation-function&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24942" PostTypeId="1" CreationDate="2020-12-01T15:35:26.377" Score="0" ViewCount="115" Body="&lt;p&gt;I have been searching this but did not find the answer, so sorry if this is a duplicated question.&lt;/p&gt;&#xA;&lt;p&gt;I was working with cross-validation, where some doubts came to my mind, and I am not sure about which is the correct answer.&lt;/p&gt;&#xA;&lt;p&gt;Lets say I have a mixed dataset, with numerical and categorical features. I want to perform a K-Fold Cross-Validation with it, with a &lt;strong&gt;K=10&lt;/strong&gt;. Some of this numerical features are missing, so I decided that I will replace those NaNs with the average of that feature.&lt;/p&gt;&#xA;&lt;p&gt;My steps are the following ones:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Read entire dataset&lt;/li&gt;&#xA;&lt;li&gt;Perform One Hot Encoding to categorical features.&lt;/li&gt;&#xA;&lt;li&gt;Divide my data into different folds. Lets say that I will use 90% for training, 10% for validating.&lt;/li&gt;&#xA;&lt;li&gt;For every different combination of folds, I replace the missing values from the traininig and validating sets &lt;strong&gt;separately&lt;/strong&gt;. This means, in one hand I get the average of the missing values of the &lt;strong&gt;training&lt;/strong&gt; part, and on the other hand the average of the missing values of the &lt;strong&gt;validating&lt;/strong&gt; part.&lt;/li&gt;&#xA;&lt;li&gt;Normalize the data of the training and validating sets between [0, 1] &lt;strong&gt;separately&lt;/strong&gt;, like I did before.&lt;/li&gt;&#xA;&lt;li&gt;Train the correspondant model.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So lets put a simple example of a dataset of 20 rows with N columns. Once I do steps 1 and 2, in the first iteration I will select the 18 first rows as a training set, and the last two rows as validating set.&#xA;I fill the missing values of the 9 first rows with the average of those 18 rows. Then the same for the 2 last rows.&#xA;Then, again, normalize in the same way, separately. And do this for every combination of folds.&lt;/p&gt;&#xA;&lt;p&gt;I am doing it like this, because otherwise, from my understanding, is that you are training your model with a biased data. You should not have access to the validation data, thus you should not be able to do the average with those numbers. Hence I am using only the numbers of the training part. If I do the average with the entire dataset, this will make my model to overfitting.&lt;/p&gt;&#xA;&lt;p&gt;I am not so sure about the normalization step, as I do not really think this will have the same impact. But here I do not really know...&lt;/p&gt;&#xA;&lt;p&gt;Is this aproach correct? Or should I do the average and normalization with the entire dataset? Why?&lt;/p&gt;&#xA;&lt;p&gt;Kind regards.&lt;/p&gt;&#xA;" OwnerUserId="42672" LastEditorUserId="42672" LastEditDate="2020-12-02T09:37:51.627" LastActivityDate="2021-05-01T15:09:20.863" Title="How to fill NaNs in Cross-Validation?" Tags="&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;cross-validation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24944" PostTypeId="2" ParentId="24895" CreationDate="2020-12-01T16:50:06.833" Score="0" Body="&lt;p&gt;The echo state condition states that differences in the input sequence results in separate trajectories of reservoir states.&lt;/p&gt;&#xA;&lt;p&gt;Which means that when you make the reservoir large enough any difference in the input sequence result in a linearly separable difference in the state space.&lt;/p&gt;&#xA;&lt;p&gt;This is quite similar to the Kerneltrick in e.g. Support Vector machines where the data becomes linearly separable in the feature space. In reservoir computing the reservoir is in a way a random feature space.&lt;/p&gt;&#xA;&lt;p&gt;To sum up, the echo state condition ensures that the signal becomes linearly separable.&lt;/p&gt;&#xA;" OwnerUserId="42601" LastActivityDate="2020-12-01T16:50:06.833" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24945" PostTypeId="1" CreationDate="2020-12-01T17:08:40.010" Score="0" ViewCount="49" Body="&lt;p&gt;There are three domains in this article: &lt;a href=&quot;https://arxiv.org/pdf/2009.07047v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Old Photo Restoration via Deep Latent Space Translation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The real old pictures noted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;the artificial old pictures noted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;the ground truth pictures noted by &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So far, I have a dataset for &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal{Y}$&lt;/span&gt;, but not for &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt;. We have not seen the article referencing the &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; dataset.&lt;/p&gt;&#xA;&lt;p&gt;What could be a good &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}$&lt;/span&gt; dataset for this article?&lt;/p&gt;&#xA;&lt;p&gt;The domain definitions are explained in section &lt;em&gt;3.1 Restoration via latent space translation&lt;/em&gt; and the different datasets used are defined in the point &lt;em&gt;4.1 Implementation&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42628" LastEditorUserId="2444" LastEditDate="2020-12-02T22:02:57.687" LastActivityDate="2020-12-02T22:02:57.687" Title="What could be a good $\mathcal{R}$ dataset in the article &quot;Old Photo Restoration via Deep Latent Space Translation&quot;?" Tags="&lt;deep-learning&gt;&lt;datasets&gt;&lt;papers&gt;&lt;image-restoration&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24948" PostTypeId="5" CreationDate="2020-12-02T00:02:56.983" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-02T00:02:56.983" LastActivityDate="2020-12-02T00:02:56.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24949" PostTypeId="4" CreationDate="2020-12-02T00:02:56.983" Score="0" Body="For questions about image restoration, i.e. the task of restoring a corrupted image to its &quot;original&quot; version." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-02T00:02:56.983" LastActivityDate="2020-12-02T00:02:56.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24950" PostTypeId="1" CreationDate="2020-12-02T01:40:08.323" Score="0" ViewCount="25" Body="&lt;p&gt;This is a question from page 94 of Sutton and Barto's RL book 2020.&lt;/p&gt;&#xA;&lt;p&gt;I read in someone's compiled GitHub answers to this book's exercises their answer was: &amp;quot;No because each state in an episode of blackjack is unique.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I think my answer is more &lt;em&gt;yes&lt;/em&gt;, but I'm thinking in terms of casino blackjack, where they have multiple decks shuffled together and add in the dropped cards back into the deck every X games in order to prevent card counting and 1 game can be seen as an episode. I think in this case that first-visit MC and every-visit MC would have drastically different results, given that, at the start of the new episode, the state of the deck, which is only partially observed, will change the value of taking an action given a state (because I believe the cards left in the deck affect the value of an action, but the deck is not totally observable).&lt;/p&gt;&#xA;&lt;p&gt;If this is blackjack, where the discarded cards are added back in and shuffled every episode, I'll agree that it shouldn't make a difference.&lt;/p&gt;&#xA;&lt;p&gt;Are there any flaws in my conjecture?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-12-04T13:42:16.337" LastActivityDate="2020-12-04T13:42:16.337" Title="Suppose every-visit MC was used instead of first-visit MC on blackjack. Would you expect the results to be different?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;sutton-barto&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24951" PostTypeId="2" ParentId="16610" CreationDate="2020-12-02T06:16:41.247" Score="1" Body="&lt;p&gt;I believe that there is no clear answer to your question. It essentially boils down to whether &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_universe_hypothesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;you are a reductionist&lt;/a&gt; – whether you believe that quantitative measurements can truly give justice to the complexity of the real world, and that a framework such as expectation maximization can losslessly capture what we care about as humans in the performing of tasks.&lt;/p&gt;&#xA;&lt;p&gt;From a non-reductionist perspective, one would be aware that almost any mathematical representation of complex real-world goals will necessarily be a proxy rather than the true goal (as many goals are not mathematically formalizable, such as what we perceive as &amp;quot;good music&amp;quot; or &amp;quot;meaning&amp;quot;), and thus the reward hypothesis is at best an approximation. Based on this, a non-reductionist's reward hypothesis could be rephrased as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;that all of what we mean by goals and purposes can be &lt;s&gt;well thought of&lt;/s&gt; &lt;b&gt;approximately operationalized (albeit at a certain domain-dependent loss)&lt;/b&gt; as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Clearly the original (stricter) version of the reward hypothesis does apply to some cases, such as purely-quantitative domains (e.g. maximizing $ earned on the stock market, or maximizing score in a video game), but as soon as the problem involves enough &amp;quot;complexity&amp;quot; (e.g. humans, or wherever you think the boundary should be), a non-reductionist would say that mathematics is clearly not fit to the task to truly capture the intended goal.&lt;/p&gt;&#xA;&lt;p&gt;More info on the reward hypothesis (as presented by &lt;a href=&quot;https://www.littmania.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michael Littman&lt;/a&gt; himself) is &lt;a href=&quot;https://www.coursera.org/lecture/fundamentals-of-reinforcement-learning/michael-littman-the-reward-hypothesis-q6x0e&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I would have added it as a comment to the question but do not have enough reputation.&lt;/p&gt;&#xA;" OwnerUserId="42207" LastEditorUserId="42207" LastEditDate="2020-12-06T21:51:08.987" LastActivityDate="2020-12-06T21:51:08.987" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24952" PostTypeId="1" AcceptedAnswerId="24953" CreationDate="2020-12-02T13:12:33.663" Score="2" ViewCount="267" Body="&lt;p&gt;I have a network with nodes and links, each of them with a certain amount of resources (that can take discrete values) at the initial state. At random time steps, a service is generated, and, based on the agent's action, the network status changes, reducing some of those nodes and links resources.&lt;/p&gt;&#xA;&lt;p&gt;The number of all possible states that the network can have is too large to calculate, especially since there is the random factor when generating the services.&lt;/p&gt;&#xA;&lt;p&gt;Let's say that I set the state space large enough (for example, 5000), and I use Q-Learning for 1000 episodes. Afterwards, when I test the agent (&lt;span class=&quot;math-container&quot;&gt;$\max Q(s,a)$&lt;/span&gt;), what could happen if the agent faces a state that did not encounter during the training phase?&lt;/p&gt;&#xA;" OwnerUserId="42591" LastEditorUserId="2444" LastEditDate="2020-12-04T10:11:33.677" LastActivityDate="2020-12-04T10:11:33.677" Title="What happens when the agent faces a state that never before encountered?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;state-spaces&gt;&lt;observation-spaces&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24953" PostTypeId="2" ParentId="24952" CreationDate="2020-12-02T13:38:53.837" Score="2" Body="&lt;p&gt;Having too many states to actually visit is a common problem in RL. This is exactly why we often use function approximation. If you replace your q table with a good function approximator such as a neural network, it should be able to generelize well to states it has not yet encountered.&lt;/p&gt;&#xA;&lt;p&gt;If you do not use a function approximator but stick with a table, the agent will have no idea what to do when it encounters a new state. For more information,  see Reinforcement Learning  by Sutton and Barto, chapter 9.&lt;/p&gt;&#xA;" OwnerUserId="12201" LastActivityDate="2020-12-02T13:38:53.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24954" PostTypeId="2" ParentId="24952" CreationDate="2020-12-02T14:00:53.997" Score="1" Body="&lt;p&gt;I will try to explain this problem with the very tangible example of chess. In chess, the number of possible states is any configuration that you can make with the pieces on the board. So, the starting position is a state, and after you did one move you are in a different state. The total number of chess states is more than &lt;span class=&quot;math-container&quot;&gt;$10^{100}$&lt;/span&gt;. It is therefore very unlikely that a chess bot has seen all the states in training when playing a match.&lt;/p&gt;&#xA;&lt;p&gt;So, how does the algorithm solve this? For the answer, we have to look at how an RL algorithm chooses which move is the best. This obviously depends on the implementation of the algorithm, but, generally, the calculation of 'how good' a move is, is done with the use of an approximation taking into account the 'potential future reward'. If you capture the queen, that would probably be good (not a chess expert here), even though you have not seen this exact state before. If you go further than this, a network might be able to approximate what happens many moves in the future. The specifics come down to implementation, etc., but this is the gist of it.&lt;/p&gt;&#xA;" OwnerUserId="34383" LastEditorUserId="2444" LastEditDate="2020-12-03T00:42:36.173" LastActivityDate="2020-12-03T00:42:36.173" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24955" PostTypeId="2" ParentId="24942" CreationDate="2020-12-02T14:09:03.407" Score="0" Body="&lt;p&gt;I would do the exact same thing as you are describing! One of the main reasons that you would want to do cross-validation is to prevent that your model is unable to generalize later. Therefore, you take out a random small subset which will be your new small validation set and do all the 'operations' on that which you are also doing on your training set. This way, you can check that these 'operations' are thus also generalizable (if they work for all different cross-validations).&lt;/p&gt;&#xA;" OwnerUserId="34383" LastActivityDate="2020-12-02T14:09:03.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24956" PostTypeId="1" CreationDate="2020-12-02T16:14:46.363" Score="0" ViewCount="17" Body="&lt;p&gt;I'd like to find a method in which the depth or perspective of an image is estimated. I imagine this could perhaps be done based on how quickly classified objects, such as cars or humans, grow or shrink in the image as they move closer or away from the camera. See my examples below.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if there is a common technique or approach in doing this? I'm guessing I'd need to compare the width and height of classified objects relative to their y-position in the image? I'm not so sure though, and I love some tips to know where to start looking.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LGhW7m.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LGhW7m.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; &lt;br/&gt;&lt;sup&gt;&lt;em&gt;Image with low image depth variance. Cars appear roughly the same size no matter their position in the image.&lt;/em&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/svCqAm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/svCqAm.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; &lt;br/&gt;&lt;sup&gt;&lt;em&gt;Image with high image depth variance. Cars farther away appear much smaller than those closer to the camera.&lt;/em&gt; &lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="42694" LastActivityDate="2020-12-02T16:14:46.363" Title="Estimating depth/perspective of image" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24957" PostTypeId="1" CreationDate="2020-12-02T19:15:08.303" Score="0" ViewCount="35" Body="&lt;p&gt;Newbie to CV here so sorry of this is basic. Here's the deal, I have a program that I run many times. and each run I produce a screenshot. I need to compare screenshots from N-1 and N runs and make sure they aren't different in any dramatic way. Of course there are some minor changes like logos and pictures getting updated, etc.&lt;/p&gt;&#xA;&lt;p&gt;SO far I've used something as simple as absdiff from opencv to highlight the difference regions and then use some sort of threshold to determine whether something passes or not. But I want to make it slightly intelligent but I'm not 100% sure how to proceed. Google hasn't yielded ghe best answers.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, I want to train the model on many different pairs of images and have the output be binary, yes or no depending on whether it should pass or not. In theory, I should be able to plug in 2 images and based on previous training, it should be able to tell me whether there is significant difference or not. What are some ways I might approach this, particularly with regards to what kinds of models to use.&lt;/p&gt;&#xA;&lt;p&gt;The requirements here might seem amorphous but that's kinda the nature of the problem. the differences could be, in theory, anything. I am hoping that there will be patterns between different images and that a model would pick up on that. Things like the name of a document is 045 instead of 056 or a logo is slightly updated.&lt;/p&gt;&#xA;" OwnerUserId="42641" LastActivityDate="2021-05-03T03:12:36.187" Title="Training a model to identify certain differences between images?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;image-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24958" PostTypeId="5" CreationDate="2020-12-02T21:31:34.753" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-02T21:31:34.753" LastActivityDate="2020-12-02T21:31:34.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24959" PostTypeId="4" CreationDate="2020-12-02T21:31:34.753" Score="0" Body="For questions that involve the Iris dataset, which is composed of 150 instances, 4 attributes/features, and 3 classes. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-02T21:31:34.753" LastActivityDate="2020-12-02T21:31:34.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24960" PostTypeId="2" ParentId="24925" CreationDate="2020-12-02T22:24:28.377" Score="0" Body="&lt;p&gt;I'm not sure specifically which Atari games present this type of action space, but you can imagine a game in which you can perform multiple different types of actions at the same timestep (i.e. the different &amp;quot;factors&amp;quot; they mention in the paper).&lt;/p&gt;&#xA;&lt;p&gt;As an example, imagine a game in which you can both move and jump at the same time. In that case, you might have a 4-dimensional discrete action space for moving (NSWE), and a 2-dimensional discrete action space for jumping (yes/no jump), both of which will require a Categorical distribution which has the size of that factor (&lt;span class=&quot;math-container&quot;&gt;$N_k$&lt;/span&gt; in the paper).&lt;/p&gt;&#xA;&lt;p&gt;So in this case, you would need to have a categorical distribution for each factor, unless you were to turn these two factors into 1 joint 4*2-dimensional factor and learn a single categorical distribution on that (which would likely be less efficient).&lt;/p&gt;&#xA;" OwnerUserId="42207" LastActivityDate="2020-12-02T22:24:28.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24961" PostTypeId="1" CreationDate="2020-12-03T00:11:17.637" Score="0" ViewCount="18" Body="&lt;p&gt;I would like to add some more samples to my dataset which consists of two parts: 1. image and 2. numerical data. For each image in the dataset there is its corresponding numerical data as well.&#xA;If it were to image only classification, I would have used cropping augmentation method (others wouldn't not suit the dataset). However, it is now a multiple input to one output (binary classification) task, I'm struggling to come up with a better solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Could someone name some of the techniques I can apply to such a type of classification?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;One example I can think of is this. Since I can't apply other augmentation methods except cropping, I may increase one sample of data by cropping the center, left and right parts (by preserving ~90% of original image).&lt;/p&gt;&#xA;&lt;p&gt;(cat image and its numerical data were made up for explanation)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CrMq5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CrMq5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If this type of image augmentation is still sensible, I would just broadcast the same number of numerical samples (namely, 3 times) as one image's numerical data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gPa9K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gPa9K.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What do you think of this way of data augmentation?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="31870" LastEditorUserId="31870" LastEditDate="2020-12-14T03:58:40.737" LastActivityDate="2020-12-14T03:58:40.737" Title="How to perform data augmentation on multiple input classification task?" Tags="&lt;deep-learning&gt;&lt;multilayer-perceptron&gt;&lt;binary-classification&gt;&lt;data-augmentation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24962" PostTypeId="1" AcceptedAnswerId="25007" CreationDate="2020-12-03T01:23:00.800" Score="1" ViewCount="75" Body="&lt;p&gt;My environment is set up so that my self-driving agent can take maximum of 400 steps (which is the end goal) before it resets with a completion reward. Despite attaining the end goal during the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy stage, it still kills/crashes itself in subsequent episodes.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if this common in RL (D3QN) scenarios.&lt;/p&gt;&#xA;&lt;p&gt;A graph showing episodes vs steps has been placed below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ILqrC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ILqrC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As one can see, the agent reaches 400 steps in episode 1000. But, in the subsequent episode, it falls down below 50 steps.&lt;/p&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="31755" LastEditDate="2020-12-05T04:31:24.337" LastActivityDate="2020-12-05T16:47:21.813" Title="Should my agent be taking varying number of steps?" Tags="&lt;reinforcement-learning&gt;&lt;epsilon-greedy-policy&gt;" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="24963" PostTypeId="2" ParentId="18407" CreationDate="2020-12-03T01:31:54.423" Score="0" Body="&lt;p&gt;An Autoencoder helps you in learning an embedding space that can be used in a PCA or T-SNE to classify different categories of images in an unsupervised fashion. Since you are trying to reconstruct an input image, the model tries to learn an underlying distribution of patterns in your images which are useful for reconstructing and those patterns translate to an embedding space&lt;/p&gt;&#xA;&lt;p&gt;For example, if you want an autoencoder to learn patterns in cat vs dog images and define the embedding space to be a 16 dimensional vector, the model will learn 16 different patterns that can help you in reconstructing the image.&lt;/p&gt;&#xA;&lt;p&gt;In this case, it is better to balance your dataset with the classes you want to learn(have equal number of cat vs dog images in your training set) so that you don't induce bias(so that the model doesn't favour learning more about cats than dogs)&lt;/p&gt;&#xA;&lt;p&gt;I would argue that overfitting is not a good thing for autoencoder because your embedding space will then get restricted only to your training domain(for ex, the whiskers of a cat in your training sample might be smaller when compared to the cats in your test sample, you don't want the embedding space to learn size of whiskers to that precision, but rather learn whiskers are important). It's important to have any model to generalize across different circumstances.&#xA;I agree to the fact that you need to train the system for longer epochs to learn good representations.&lt;/p&gt;&#xA;&lt;p&gt;I hope this gives you a better intuition on the parameters to consider to learn a good embedding space.&lt;/p&gt;&#xA;" OwnerUserId="30939" LastActivityDate="2020-12-03T01:31:54.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24964" PostTypeId="2" ParentId="23842" CreationDate="2020-12-03T01:52:28.477" Score="0" Body="&lt;p&gt;Imagine a small kid who has no idea about the world around it. You teach the kid how to write the number &amp;quot;6&amp;quot; and that is the only thing that it knows.&lt;/p&gt;&#xA;&lt;p&gt;Now, No matter what other number you show the kid , it's gonna always respond with &amp;quot;6&amp;quot; because that is the only thing it knows or it has learned.&lt;/p&gt;&#xA;&lt;p&gt;You teach the kid how to write the number &amp;quot;9&amp;quot;, so now it knows how to differentiate a &amp;quot;6&amp;quot; from a &amp;quot;9&amp;quot; and no matter what other number you show the kid, there is a 50 % chance of it responding with a &amp;quot;6&amp;quot; or a &amp;quot;9&amp;quot; because it knows only that much.&lt;/p&gt;&#xA;&lt;p&gt;The purpose of a neural network is to understand underlying distribution in data that can help it in classifying different numbers. It's important to have a classifier that understands general characteristics of numbers and help us with our task.&#xA;If you have 10 neural networks trained on 10 different digits, and you show each of these networks the number &amp;quot;10&amp;quot;, each network will output the number on which it was trained because that is all it knows(similar to the naive kid above).&lt;/p&gt;&#xA;&lt;p&gt;I hope this answers your question!&lt;/p&gt;&#xA;" OwnerUserId="30939" LastActivityDate="2020-12-03T01:52:28.477" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24966" PostTypeId="1" AcceptedAnswerId="24999" CreationDate="2020-12-03T03:14:49.547" Score="3" ViewCount="75" Body="&lt;h2&gt;AlphaGo Zero&lt;/h2&gt;&#xA;&lt;p&gt;AlphaGo Zero uses a Monte-Carlo Tree Search where the selection phase is governed by &lt;span class=&quot;math-container&quot;&gt;$\operatorname*{argmax}\limits_a\left( Q(s_t, a) + U(s_t, a) \right)$&lt;/span&gt;, where:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the exploitation parameter is &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a) = \displaystyle \frac{\displaystyle \sum_{v_i \in (s_t, a)} v_i}{N(s_t, a)}$&lt;/span&gt; (i.e. the mean of the values &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; of all simulations that passes through edge &lt;span class=&quot;math-container&quot;&gt;$(s_t, a)$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;the exploration parameter is &lt;span class=&quot;math-container&quot;&gt;$U(s_t, a) = c_{puct} P(s_t,a) \frac{\sqrt{\sum_b N(s_t, b)}}{1 + N(s_t, a)}$&lt;/span&gt; (i.e. the prior probability &lt;span class=&quot;math-container&quot;&gt;$P(s_t, a)$&lt;/span&gt;, weighted by the constant &lt;span class=&quot;math-container&quot;&gt;$c_{puct}$&lt;/span&gt;, the number of simulations that passes through &lt;span class=&quot;math-container&quot;&gt;$(s_t, a)$&lt;/span&gt;, as well as the number of simulations that passes through  &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The prior probability &lt;span class=&quot;math-container&quot;&gt;$P(s_t, a)$&lt;/span&gt; and simulation value &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; are both outputted by the deep neural network &lt;span class=&quot;math-container&quot;&gt;$f_{\theta}(s_t)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This neural network takes as an input the raw board representation s of the position and its history, and outputs both move probabilities and a value, (p, v) = fθ(s). The vector of move probabilities p represents the probability of selecting each move a (including pass), pa = Pr(a| s). The value v is a scalar evaluation, estimating the probability of the current player winning from position s.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;My confusion&lt;/h2&gt;&#xA;&lt;p&gt;My confusion is that &lt;span class=&quot;math-container&quot;&gt;$P(s_t, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; are probabilities normalized to different distributions, resulting in &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; being about 80x larger than &lt;span class=&quot;math-container&quot;&gt;$P(s_t,a)$&lt;/span&gt; on average.&lt;/p&gt;&#xA;&lt;p&gt;The neural network outputs &lt;span class=&quot;math-container&quot;&gt;$(p, v)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a probability vector given &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, normalized over all possible actions in that turn. &lt;span class=&quot;math-container&quot;&gt;$p_a = P(s_t, a)$&lt;/span&gt; is the probability of choosing action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;. A game of Go has about 250 moves per turn, so on average each move has probability &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{250}$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}\left[ P(s_t, a) \right] = \frac{1}{250}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is the probability of winning given state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, normalized over all possible end-game conditions (win/tie/lose). For simplicity sake, let us assume &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E} \left[ v_i \right] \ge \frac{1}{3}$&lt;/span&gt;, where the game is played randomly and each outcome is equally likely.&lt;/p&gt;&#xA;&lt;p&gt;This means that the expected value of &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; is at least 80x larger than the expected value of &lt;span class=&quot;math-container&quot;&gt;$P(s_t, a)$&lt;/span&gt;. &lt;strong&gt;The consequence of this is that &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a)$&lt;/span&gt; is at least 80x larger than &lt;span class=&quot;math-container&quot;&gt;$U(s_t, a)$&lt;/span&gt; on average.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the above is true, then the selection stage will be dominated by the &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a)$&lt;/span&gt; term, so AlphaGo Zero should tend to avoid edges with no simulations in them (edges where  &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a) = 0$&lt;/span&gt;) unless all existing &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a)$&lt;/span&gt; terms are extremely small (&lt;span class=&quot;math-container&quot;&gt;$&amp;lt; \frac{1}{250}$&lt;/span&gt;), or the MCTS has so much simulations in them that the &lt;span class=&quot;math-container&quot;&gt;$\frac{\sqrt{\sum_b N(s_t, b)}}{1 + N(s_t, a)}$&lt;/span&gt; term in &lt;span class=&quot;math-container&quot;&gt;$U(s_t, a)$&lt;/span&gt; evens out the magnitudes of the two terms. The latter is not likely to happen since I believe AlphaGo Zero only uses &lt;span class=&quot;math-container&quot;&gt;$1,600$&lt;/span&gt; simluations per move, so &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\sum_b N(s_t, b)}$&lt;/span&gt; caps out at &lt;span class=&quot;math-container&quot;&gt;$40$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Selecting only viable moves&lt;/h2&gt;&#xA;&lt;p&gt;Ideally, MCTS shouldn't select every possible move to explore. It should only select viable moves given state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, and ignore all the bad moves. Let &lt;span class=&quot;math-container&quot;&gt;$m_t$&lt;/span&gt; is the number of viable moves for state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;, and let &lt;span class=&quot;math-container&quot;&gt;$P(s_t, a)$&lt;/span&gt; = 0 for all moves &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that are not viable. Also, let's assume the MCTS never selects a move that is not viable.&lt;/p&gt;&#xA;&lt;p&gt;Then the previous section is partly alleviated, because now &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E} \left[ P(s_t, a) \right] = \frac{1}{m_t}$&lt;/span&gt;. &lt;strong&gt;As a result, &lt;span class=&quot;math-container&quot;&gt;$Q(s_T, a)$&lt;/span&gt; should only be &lt;span class=&quot;math-container&quot;&gt;$\frac{m_t}{3}$&lt;/span&gt; times larger than &lt;span class=&quot;math-container&quot;&gt;$U(s_t, a)$&lt;/span&gt; on average&lt;/strong&gt;. Assuming &lt;span class=&quot;math-container&quot;&gt;$m_t \le 6$&lt;/span&gt;, then there shouldn't be too much of an issue&lt;/p&gt;&#xA;&lt;p&gt;However, this means that AlphaGo Zero works ideally only when the number of viable moves is small. In a game state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; where there are many viable moves (&lt;span class=&quot;math-container&quot;&gt;$&amp;gt;30$&lt;/span&gt;) (e.g. a difficult turn with many possible choices), the selection phase of the MCTS will deteriorate as described in the previous section.&lt;/p&gt;&#xA;&lt;h2&gt;Questions&lt;/h2&gt;&#xA;&lt;p&gt;I guess my questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is my understanding correct, or have I made mistake(s) somewhere?&lt;/li&gt;&#xA;&lt;li&gt;Does &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a)$&lt;/span&gt; usually dominate &lt;span class=&quot;math-container&quot;&gt;$U(s_t, a)$&lt;/span&gt; by this much in practice when the game state has many viable moves? Is the selection phase usually dominated by &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a)$&lt;/span&gt; during these game states?&lt;/li&gt;&#xA;&lt;li&gt;Does the fact that &lt;span class=&quot;math-container&quot;&gt;$Q(s_t, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$U(s_t, a)$&lt;/span&gt; being in such different orders of magnitude (when the game state has many viable moves) affect the quality of the MCTS algorithm, or is MCTS robust to this effect and still produces high quality policies?&lt;/li&gt;&#xA;&lt;li&gt;How common is it for a game state to have many viable moves (&amp;gt;30) in Go?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-03T06:42:49.683" LastActivityDate="2020-12-04T20:08:44.673" Title="AlphaGo Zero: does $Q(s_t, a)$ dominate $U(s_t, a)$ in difficult game states?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;&lt;alphago-zero&gt;&lt;alphago&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24968" PostTypeId="2" ParentId="24921" CreationDate="2020-12-03T10:28:15.280" Score="1" Body="&lt;p&gt;In the eye, the retinal ganglion cells have a &lt;a href=&quot;https://en.wikipedia.org/wiki/Receptive_field&quot; rel=&quot;nofollow noreferrer&quot;&gt;receptive field&lt;/a&gt; that is equivalent to some types of convolution filters, most of them edge detectors.&lt;/p&gt;&#xA;&lt;p&gt;The brain is a big unknown, nobody knows how it does to organize, memorize, create concepts, learns the language, ... . Thus, it is not possible to establish a parallelism.&lt;/p&gt;&#xA;&lt;p&gt;In particular, brain has a capacity of handle independence in scale &lt;strong&gt;and rotation&lt;/strong&gt; that CNN's are not able to reproduce.&lt;/p&gt;&#xA;&lt;p&gt;As general remark about NN and brain: even when it is always said that &amp;quot;neural network cells&amp;quot; are &amp;quot;inspired&amp;quot; in biological neurons, there are critical differences that made this similitude only a &amp;quot;inspiration&amp;quot;. Thus, comparison of a CNN or any other kind of NN with brain is always a fuzzy comparison. The biggest difference is probably the learning capacity: the human brain learns by itself, while the neural network needs an external system (the back-propagation algorithm) that feeds the NN with the learned parameters.&lt;/p&gt;&#xA;" OwnerUserId="12630" LastEditorUserId="12630" LastEditDate="2020-12-03T10:33:54.680" LastActivityDate="2020-12-03T10:33:54.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24969" PostTypeId="2" ParentId="24921" CreationDate="2020-12-03T10:40:53.537" Score="1" Body="&lt;p&gt;Yes, CNNs are inspired by the human brain [&lt;a href=&quot;https://arxiv.org/pdf/1404.7828.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://www.cs.princeton.edu/courses/archive/spr08/cos598B/Readings/Fukushima1980.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;]. More specifically, their operations, the convolution and pooling, are inspired by the human brain. However, note that, nowadays, CNNs are mainly trained with gradient descent (GD) and back-propagation (BP), which seems &lt;strong&gt;not&lt;/strong&gt; to be a biologically plausible way of learning, but, given the success of GD and BP, there have been attempts to connect GB and BP with the way humans learn [&lt;a href=&quot;https://brainscan.uwo.ca/research/cores/computational_core/uploads/11May2020-Lillicrap_NatNeuroRev_2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;neocognitron&lt;/strong&gt;, the first convolutional neural network [&lt;a href=&quot;https://arxiv.org/pdf/1404.7828.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], proposed by &lt;strong&gt;Kunihiko Fukushima&lt;/strong&gt; in 1979-1980, and described in the paper &lt;a href=&quot;https://www.cs.princeton.edu/courses/archive/spr08/cos598B/Readings/Fukushima1980.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position&lt;/a&gt;, already uses convolutional and pooling (specifically, averaging pooling) layers [&lt;a href=&quot;https://arxiv.org/pdf/1404.7828.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. The neocognitron was inspired by the work of Hubel and Wiesel described in the 1959 paper &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1363130/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Receptive fields of single neurones in the cat's striate cortex&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is an excerpt from the 1980 Fukushima's paper.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The mechanism of pattern recognition in the brain is little known, and it seems to be almost impossible to reveal it only by conventional physiological experiments. So, we take a slightly different approach to this problem. If we could make a neural network model which has the same capability for pattern recognition as a human being, it would give us a powerful clue to the understanding of the neural mechanism in the brain. In this paper, we discuss how to synthesize a neural network model in order to endow it an ability of pattern recognition like a human being.&lt;/p&gt;&#xA;&lt;p&gt;Several models were proposed with this intention (Rosenblatt, 1962; Kabrisky, 1966; Giebel, 1971; Fukushima, 1975). The response of most of these models, however, was severely affected by the shift in position and/or by the distortion in shape of the input patterns. Hence, their ability for pattern recognition&#xA;was not so high.&lt;/p&gt;&#xA;&lt;p&gt;In this paper, we propose an improved neural network model. The structure of this network has been suggested by that of the visual nervous system of the vertebrate. This network is self-organized by &amp;quot;learning without a teacher&amp;quot;, and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their position nor by small distortion of their shapes. This network is given a nickname &amp;quot;neocognitron&amp;quot;, because it is a further extention of the &amp;quot;cognitron&amp;quot;, which also is a self-organizing multilayered neural network model proposed by the author before (Fukushima, 1975)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, Fukushima did not train the neocognitron with gradient descent (and back-propagation) but with local learning rules (which are more biologically plausible), and that's probably why he doesn't get more credit, as I think he should.&lt;/p&gt;&#xA;&lt;p&gt;You should read at least Fukushima's paper for more details, which I will not replicate here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Section 9.4 of the Deep Learning book&lt;/a&gt; also contains details about how CNNs are inspired by neuroscience findings.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T13:17:52.957" LastActivityDate="2020-12-04T13:17:52.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24970" PostTypeId="1" CreationDate="2020-12-03T12:07:56.300" Score="0" ViewCount="28" Body="&lt;p&gt;I am doing multi class segmentation using UNet. My input to the model is &lt;code&gt;HxWxC&lt;/code&gt; and my output is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;outputs = layers.Conv2D(n_classes, (1, 1), activation='sigmoid')(decoder0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using &lt;code&gt;SparseCategoricalCrossentropy&lt;/code&gt; I can train the network fine. Now I would like to also try dice coefficient as the loss function. Implemented as follows,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def dice_loss(y_true, y_pred, smooth=1e-6):&#xA;    y_true = tf.cast(y_true, tf.float32)&#xA;    y_pred = tf.math.sigmoid(y_pred)&#xA;&#xA;    numerator = 2 * tf.reduce_sum(y_true * y_pred) + smooth&#xA;    denominator = tf.reduce_sum(y_true + y_pred) + smooth&#xA;&#xA;    return 1 - numerator / denominator&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However I am actually getting an increasing loss instead of decreasing loss. I have checked multiple sources but all the material I find use diceloss for binary classification and not multiclass. So my question is is is there a problem with the implementation.&lt;/p&gt;&#xA;" OwnerUserId="19186" LastActivityDate="2020-12-03T12:07:56.300" Title="Implementing Multiclass Dice Loss Function" Tags="&lt;tensorflow&gt;&lt;loss-functions&gt;&lt;image-segmentation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24971" PostTypeId="1" AcceptedAnswerId="25009" CreationDate="2020-12-03T12:14:45.153" Score="0" ViewCount="44" Body="&lt;p&gt;What is the impact of the number of features on the prediction power of an ANN model (in general)? Does an increase in the number of features mean a more powerful prediction model (for approximation purpose)?&lt;/p&gt;&#xA;&lt;p&gt;I'm asking these questions because I am wondering if there is any benefit in using two variables (rather than one) to predict one output.&lt;/p&gt;&#xA;&lt;p&gt;If there is a scientific paper that answers my question, I would thank you.&lt;/p&gt;&#xA;" OwnerUserId="41210" LastEditorUserId="2444" LastEditDate="2020-12-04T14:13:17.750" LastActivityDate="2020-12-05T13:04:16.347" Title="What is the impact of the number of features on the prediction power of a neural network?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reference-request&gt;&lt;feature-selection&gt;&lt;features&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24972" PostTypeId="1" CreationDate="2020-12-03T12:28:22.767" Score="1" ViewCount="49" Body="&lt;p&gt;I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on &lt;em&gt;&lt;strong&gt;downstream&lt;/strong&gt;&lt;/em&gt; tasks when the network is trained on &lt;em&gt;&lt;strong&gt;pretext&lt;/strong&gt;&lt;/em&gt; tasks.&lt;/p&gt;&#xA;&lt;p&gt;Suppose that I want to do image classification on my own set of images. I have limited data on these images and maybe I can use self-supervised learning to achieve better accuracies on these limited data.&lt;/p&gt;&#xA;&lt;p&gt;Let's say that I try to train a neural network on a pretext task of &lt;a href=&quot;https://arxiv.org/abs/1505.05192&quot; rel=&quot;nofollow noreferrer&quot;&gt;predicting the patch position relative to the center patch&lt;/a&gt; on different images that are readily available in quantity, such as cats, dogs, etc.&lt;/p&gt;&#xA;&lt;p&gt;If I try to initialise the weights of my neural network, then do image classification on my own images, which are vastly different from that of the images used in the pretext task, would self-supervised learning work because the images for the pretext and downstream tasks are different?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TLDR&lt;/strong&gt;: Must the images used in the pretext task and the downstream tasks be the same?&lt;/p&gt;&#xA;" OwnerUserId="32780" LastEditorUserId="2444" LastEditDate="2020-12-04T12:15:30.520" LastActivityDate="2021-05-08T11:19:46.977" Title="Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?" Tags="&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;transfer-learning&gt;&lt;self-supervised-learning&gt;&lt;pretext-tasks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24974" PostTypeId="5" CreationDate="2020-12-03T13:09:56.480" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-03T13:09:56.480" LastActivityDate="2020-12-03T13:09:56.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24975" PostTypeId="4" CreationDate="2020-12-03T13:09:56.480" Score="0" Body="For questions about the neocognitron, proposed in 1979 by Kunihiko Fukushima, which is considered (e.g. according to &quot;Deep Learning in Neural Networks: An Overview&quot; by J. Schmidhuber) the first convolutional neural network. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-03T13:19:31.407" LastActivityDate="2020-12-03T13:19:31.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24977" PostTypeId="5" CreationDate="2020-12-03T14:40:14.720" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-03T14:40:14.720" LastActivityDate="2020-12-03T14:40:14.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24978" PostTypeId="4" CreationDate="2020-12-03T14:40:14.720" Score="0" Body="For questions about the concept of inductive (learning) bias in machine learning and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-03T14:40:14.720" LastActivityDate="2020-12-03T14:40:14.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24979" PostTypeId="2" ParentId="5322" CreationDate="2020-12-03T15:37:27.127" Score="0" Body="&lt;p&gt;A simple example of this is token embeddings. If &amp;quot;prior knowledge&amp;quot; just means anything known prior to creation of the graph, then using pretrained vector embeddings meets this criteria. This is simply a way to provide a fixed method for projecting tokens into higher-dimensional space instead of training it at the same time as the rest of the model. Given that vector embeddings are somewhat interpretable and that the same embedding can be reused across tasks and models, I'd consider pretrained embeddings to be prior knowledge being incorporated.&lt;/p&gt;&#xA;&lt;p&gt;The embeddings could also technically be handcrafted, but I'm not aware of any work like that and am skeptical of its usefulness in deep models.&lt;/p&gt;&#xA;" OwnerUserId="29873" LastActivityDate="2020-12-03T15:37:27.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24983" PostTypeId="2" ParentId="24957" CreationDate="2020-12-04T01:24:59.983" Score="0" Body="&lt;p&gt;If your image just slightly changes, all you need is the simple algorithm, just find the keyword &amp;quot;k-Nearest Neighbor&amp;quot; or take a look at &lt;a href=&quot;https://cs231n.github.io/classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To locate where is the difference, you can find the difference by subtracting two images by this script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def extract_diff(imageA, imageB):&#xA;    '''&#xA;    Find the different between two image:&#xA;        + Input: two RGB image&#xA;        + Output: binary image show different between two image&#xA;    Assume the different between two image in each channel will be bigger or equal 30&#xA;    '''&#xA;    subtract = imageB.astype(np.float32) - imageA.astype(np.float32)&#xA;    mask_motion = cv2.inRange(np.abs(subtract),(30,30,30),(255,255,255))&#xA;    # mask_motion[mask_motion==255] = 1 # scale to 1 to reduce computation&#xA;    return mask_motion&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And locate where the position with value 255 in the image. Next time you should this kind of question in the stack overflow community, there is a more active community, they can give you a better solution than mine.&lt;/p&gt;&#xA;" OwnerUserId="41287" LastActivityDate="2020-12-04T01:24:59.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24984" PostTypeId="1" AcceptedAnswerId="25060" CreationDate="2020-12-04T01:27:28.127" Score="4" ViewCount="129" Body="&lt;p&gt;I'm learning about Actor-Critic reinforcement learning algorithms. One source I encountered mentioned that Actor and Critic can either share one network (but use different output layers) or they can use two completely separate networks. In &lt;a href=&quot;https://youtu.be/2vJtbAha3To?t=398&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; he mentions that using two separate networks works for simpler problems, such as Mountain Car. However, more complex problems like Lunar Lander works better with a shared network. Why is that? Could you explain what difference that choosing one design over another would that make?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="38076" LastEditDate="2020-12-04T18:58:05.487" LastActivityDate="2020-12-08T04:34:03.007" Title="What difference does it make whether Actor and Critic share the same network or not?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24986" PostTypeId="2" ParentId="16509" CreationDate="2020-12-04T03:59:49.720" Score="0" Body="&lt;p&gt;I would recommend taking a look at Bilingual Evaluation Understudy(BELU) score which is commonly used in evaluating machine translation results by sequence to sequence model. Here is the reference &lt;a href=&quot;https://en.wikipedia.org/wiki/BLEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/BLEU&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30939" LastActivityDate="2020-12-04T03:59:49.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24987" PostTypeId="2" ParentId="16172" CreationDate="2020-12-04T05:04:52.403" Score="0" Body="&lt;p&gt;The newer models generally outperform older ones on the ImageNet challenge in their accuracy scores*. This does not necessarily mean that this difference in performance will be reflected in your particular classification problem.&lt;/p&gt;&#xA;&lt;p&gt;The closer your problem is to the ImageNet one, the more likely that the relative model performances will be similar. However when you perform transfer learning you will often have to fine-tune the model to achieve a stronger performance, the better you tune the model will effect performance, and there will often be a difference in which model is performing best on a given task. You can see papers in various classification tasks where VGG may be performing best, or Inception, or even AlexNet. I believe the simplest models (AlexNet has only 8 layers) may be the easiest to fine tune, and also may require the smallest amount of data for good performance.&lt;/p&gt;&#xA;&lt;p&gt;*There are exceptions, MobileNet is more recent but the innovation is that it is a smaller model rather than the strongest model i.e. it is designed to be useable on mobile devices rather than running on the latest GPU.&lt;/p&gt;&#xA;" OwnerUserId="40833" LastActivityDate="2020-12-04T05:04:52.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24989" PostTypeId="1" AcceptedAnswerId="24990" CreationDate="2020-12-04T08:26:40.853" Score="1" ViewCount="66" Body="&lt;p&gt;&lt;em&gt;Q-Learning is guaranteed to converge if &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; decreases over time.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=183&quot; rel=&quot;nofollow noreferrer&quot;&gt;On page 161&lt;/a&gt; of the RL book by Sutton and Barto, 2nd edition, section 8.1, they write that Dyna-Q is guaranteed to converge if each action-state pair is selected an infinite number of times and if &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; decreases appropriately over time.&lt;/p&gt;&#xA;&lt;p&gt;It seems that it would be better if &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; increased over time, as it is the learning rate of the gradient of Q-function values (&lt;span class=&quot;math-container&quot;&gt;$R+\gamma\max_aQ(S',a)-Q(S,A)$&lt;/span&gt;), and, initially, they start off incredibly inaccurate, because they are initialized arbitrarily and over time converge to the true values, hence you'd want to weight them more as time increases rather than decrease it?&lt;/p&gt;&#xA;&lt;p&gt;Why is this a convergence criterion?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2020-12-21T22:10:25.990" LastActivityDate="2020-12-21T22:10:25.990" Title="If $\alpha$ decreases over time, why is Q-learning guaranteed to converge?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;convergence&gt;&lt;learning-rate&gt;&lt;dyna&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24990" PostTypeId="2" ParentId="24989" CreationDate="2020-12-04T09:13:44.577" Score="5" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Why is this a convergence criterion?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is because &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; are stochastic. A large learning rate applied when these values have variance would not converge to mean, but would wander around typically within some value proportional to &lt;span class=&quot;math-container&quot;&gt;$\alpha\sigma$&lt;/span&gt; of the true value, where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the standard deviation of the term &lt;span class=&quot;math-container&quot;&gt;$R + \gamma\text{max}_aQ(S',a)$&lt;/span&gt;. If you reduce &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; towards zero, then this expected error will also reduce to zero.&lt;/p&gt;&#xA;&lt;p&gt;For deterministic environments, it should be possible to prove convergence with large &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In the special case of static policy, tabular learning and &lt;span class=&quot;math-container&quot;&gt;$\alpha = \frac{1}{N(s,a)}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N(s,a)$&lt;/span&gt; is number of visits to state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, then the expected error for each Q value is the MSE from basic stats i.e. &lt;span class=&quot;math-container&quot;&gt;$\frac{\sigma_{TD}}{\sqrt{N(s,a)}}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="2444" LastEditDate="2020-12-04T10:16:26.900" LastActivityDate="2020-12-04T10:16:26.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24991" PostTypeId="5" CreationDate="2020-12-04T10:21:50.857" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T10:21:50.857" LastActivityDate="2020-12-04T10:21:50.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24992" PostTypeId="4" CreationDate="2020-12-04T10:21:50.857" Score="0" Body="For questions about the concept of an n-gram in the context of natural language processing and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T10:21:50.857" LastActivityDate="2020-12-04T10:21:50.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24993" PostTypeId="2" ParentId="24471" CreationDate="2020-12-04T11:28:15.473" Score="0" Body="&lt;p&gt;I really liked the question. Yes, we sum over derivatives. First of all think what backpropagation is trying to do: &lt;em&gt;finding the affect of each parameter on the loss&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So as you said:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the same filter is used multiple times on the input while convolving&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;meaning that each kernel affects the final loss in several ways, so those affects should be summed together, not averaged.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-12-04T11:28:15.473" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24994" PostTypeId="5" CreationDate="2020-12-04T17:36:52.307" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T17:36:52.307" LastActivityDate="2020-12-04T17:36:52.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24995" PostTypeId="4" CreationDate="2020-12-04T17:36:52.307" Score="0" Body="For questions about the bag-of-words model, which is often used in natural language processing and information retrieval." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T17:36:52.307" LastActivityDate="2020-12-04T17:36:52.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24996" PostTypeId="5" CreationDate="2020-12-04T17:43:02.607" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T17:43:02.607" LastActivityDate="2020-12-04T17:43:02.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24997" PostTypeId="4" CreationDate="2020-12-04T17:43:02.607" Score="0" Body="For questions about the softmax policy in the context of reinforcement learning and other AI sub-fields." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T17:43:02.607" LastActivityDate="2020-12-04T17:43:02.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24998" PostTypeId="1" CreationDate="2020-12-04T18:04:38.577" Score="1" ViewCount="72" Body="&lt;p&gt;I have a Reinforcement Learning environment where the state is a 2D matrix with 0s and 1s (only one column with the value of 1 in each row).&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(&#xA; (0, 1, 0),&#xA; (0, 0, 1),&#xA; (1, 0, 0),&#xA; (0, 0, 0),&#xA; (0, 1, 0)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The action the agent must take is for each row in the input, choose one resource out of 12 resources the agent has if there is a column with the value of 1 in that row, else choose no resource if the row has 0s only (example: &lt;code&gt;row[3]&lt;/code&gt; wouldn't have any resources chosen for it by the agent). The rows correspond to the users the agent must allocate resources to.&lt;/p&gt;&#xA;&lt;p&gt;In the step() method in the RL environment, the agent would receive a reward or a penalty depending on the action. If the reward is positive, the agent updates the state matrix, putting a 0 instead of 1 in the rows corresponding to the users that were allocated resources, which should be the next state. If the reward is negative, the episode ends, the environment resets and a new state is received by the agent&lt;/p&gt;&#xA;&lt;p&gt;It came to my understanding that, in a deep learning approach, the DQN agent would receive a 2D matrix of 0s and 1s as input to its neural network (the state matrix), and output a vector with the chosen resources for each row of the input.&lt;/p&gt;&#xA;&lt;p&gt;The network must choose a resource out of 12 resources for each row if that row has a 1 in it, and no resource is chosen if there is no column with the value of 1 in that row of the input. In other words, the network must choose an element out of 12 and output a vector with the chosen elements, depending on the input matrix.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to do this using Deep Q-Learning and neural networks ?&lt;/p&gt;&#xA;" OwnerUserId="42372" LastEditorUserId="42372" LastEditDate="2020-12-04T19:45:38.963" LastActivityDate="2020-12-04T19:45:38.963" Title="DQN Agent with a 2D matrix as input in Keras" Tags="&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;deep-rl&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24999" PostTypeId="2" ParentId="24966" CreationDate="2020-12-04T20:08:44.673" Score="2" Body="&lt;p&gt;I don't think you've necessarily made any real mistakes in your calculations or anything like that, that all seems accurate. I can't really confidently answer your questions about &amp;quot;Does X usually happen?&amp;quot; or &amp;quot;How common is X?&amp;quot;, would have to experiment to make sure of that. I think we can also confidently immediately answer the question about whether MCTS is robust and can still produce high quality policies with &amp;quot;yes&amp;quot;, since we've seen state-of-the-art, superhuman results in a bunch of games using these techniques.&lt;/p&gt;&#xA;&lt;p&gt;But I do think there's a few important details that may change your perception:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;MCTS does not compare &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; values to &lt;span class=&quot;math-container&quot;&gt;$U(s, a)$&lt;/span&gt; values in its selection phase. It compares &lt;span class=&quot;math-container&quot;&gt;$Q(s, a) + U(s, a)$&lt;/span&gt; expressions of actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, to &lt;span class=&quot;math-container&quot;&gt;$Q(s, b) + U(s, b)$&lt;/span&gt; expressions for different actions &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. So, the difference in magnitudes &lt;span class=&quot;math-container&quot;&gt;$Q(s, a) - U(s, a)$&lt;/span&gt; is not nearly as important as the difference in magnitude &lt;span class=&quot;math-container&quot;&gt;$Q(s, a) - Q(s, b) + U(s, a) - U(s, b)$&lt;/span&gt;!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For any single given state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, it is certainly not the case that we expect the different &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;-values to be have a nice average like &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt; or anything like that. There will likely be plenty of states &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; where we're already in such a strong position that we can afford to make a mistake or two and still expect to win; all the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values here will be close to &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt;. There will also be many states where we're in such a terrible position that we expect to lose no matter what; all the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values here will be close to &lt;span class=&quot;math-container&quot;&gt;$0.0$&lt;/span&gt;. And then there will of course be states that a network is not sure about, which will have &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; values somewhere in between. I suspect that &amp;quot;in between&amp;quot; won't often be a nice mix of all sorts of different values though. If it's something like &lt;span class=&quot;math-container&quot;&gt;$0.7$&lt;/span&gt;, and there's higher values that attract more attention, during training the MCTS + network will likely become very interested in learning more about that state, and very quickly learn whether that should really just be a &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt; or whether it should be lowered. For this reason, I imagine that in unsure states, values will have a tendency to hover around &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MCTS will only let the &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; term &lt;strong&gt;dominate the selection phase for as long as it believes that this is actually likely to lead to a win&lt;/strong&gt;. If this is correct and indeed leads to a win, well, that's great, no need to explore anything else! During the tree search, if further investigation of this action leads the MCTS to believe that it actually is a loss, the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; value will drop (ideally towards &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;), and then it will automatically stop being a dominant term. If the tree search fails to adjust for this in time, and we end up wandering down this losing path anyway, we'll get a value signal of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; at the end and update our value network and in the future we'll know better than to repeat this mistake.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-12-04T20:08:44.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25000" PostTypeId="5" CreationDate="2020-12-04T22:36:09.650" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T22:36:09.650" LastActivityDate="2020-12-04T22:36:09.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25001" PostTypeId="4" CreationDate="2020-12-04T22:36:09.650" Score="0" Body="For questions about visual place recognition, the task of recognizing revisited places accurately by&#xD;&#xA;using visual information." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-04T22:36:09.650" LastActivityDate="2020-12-04T22:36:09.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25002" PostTypeId="2" ParentId="1635" CreationDate="2020-12-05T00:39:37.543" Score="0" Body="&lt;p&gt;The paper &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.45.5646&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The First Law of Robotics (a call to arms)&lt;/a&gt; (AAAI-94), by Weld and Etzioni, discusses the first Asimov's law, some technical issues it gives rise to (some of them are already mentioned in the other answers), and how they could be addressed (they propose a simplistic way to formalize the first law, but they don't claim it is the right way to do it). You should read it for more details.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-05T00:50:27.480" LastActivityDate="2020-12-05T00:50:27.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25003" PostTypeId="1" CreationDate="2020-12-05T00:58:59.020" Score="4" ViewCount="213" Body="&lt;p&gt;In the news, DeepMind's AlphaFold is said to have solved the protein folding problem using neural networks, but isn't this a problem only optimised quantum computers can solve?&lt;/p&gt;&#xA;&lt;p&gt;To my limited understating, the issue is that there are too many variables (atomic forces) to consider when simulating how an amino acid chain would fold, in which case only a quantum computer can be used to simulate it.&lt;/p&gt;&#xA;&lt;p&gt;Is the neural network just making a very good estimate, or is it simulating the actual protein structure?&lt;/p&gt;&#xA;" OwnerUserId="5708" LastEditorUserId="2444" LastEditDate="2020-12-05T01:10:36.603" LastActivityDate="2020-12-07T19:21:51.183" Title="Is AlphaFold just making a good estimate of the protein structure?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deepmind&gt;&lt;quantum-computing&gt;&lt;alpha-fold&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25004" PostTypeId="5" CreationDate="2020-12-05T01:12:39.383" Score="0" Body="&lt;p&gt;Link to original paper: &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1923-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41586-019-1923-7&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="19524" LastEditDate="2020-12-09T23:01:32.473" LastActivityDate="2020-12-09T23:01:32.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25005" PostTypeId="4" CreationDate="2020-12-05T01:12:39.383" Score="0" Body="For questions about DeepMind's AlphaFold 1 and AlphaFold 2. AlphaFold 1 is described in the paper &quot;Improved protein structure prediction using potentials from deep learning&quot; (Nature volume 577, pages 706–710(2020))." OwnerUserId="2444" LastEditorUserId="19524" LastEditDate="2020-12-09T23:01:23.083" LastActivityDate="2020-12-09T23:01:23.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25006" PostTypeId="2" ParentId="25003" CreationDate="2020-12-05T02:04:29.963" Score="8" Body="&lt;p&gt;AlphaFold (version &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1923-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; and &lt;a href=&quot;https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;) &lt;strong&gt;predicts&lt;/strong&gt; (so &lt;strong&gt;estimates&lt;/strong&gt;) the 3D shape of the protein from the sequence of amino acids. AlphaFold's performance is measured with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Global_distance_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;global distance test (GDT)&lt;/a&gt;, which is a measure of similarity between two protein structures (the prediction and the ground-truth) that ranges from 0 to 100.&lt;/p&gt;&#xA;&lt;p&gt;There is a &lt;a href=&quot;https://www.youtube.com/watch?v=KpedmJdrTpY&quot; rel=&quot;nofollow noreferrer&quot;&gt;short video&lt;/a&gt; and a &lt;a href=&quot;https://www.youtube.com/watch?v=gg7WjuFs8F4&quot; rel=&quot;nofollow noreferrer&quot;&gt;longer one&lt;/a&gt; (both by DeepMind) that summarise the issue of &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_folding&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein folding&lt;/a&gt;, how it is important, how well AlphaFold approximately solves it (in the competition &lt;a href=&quot;https://en.wikipedia.org/wiki/CASP&quot; rel=&quot;nofollow noreferrer&quot;&gt;Critical Assessment of protein Structure Prediction (CASP)&lt;/a&gt;), i.e. AlphaFold 2 achieves a median GDT score of 92.4 (and 87 on the hardest proteins), which is a lot higher than AlphaFold 1's GDT score of 58 (which was the highest achieved score at the time), where, according to &lt;a href=&quot;http://moult.ibbr.umd.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;John Moult&lt;/a&gt; (president of CASP), a score around 90 is considered a satisfactory solution to the protein folding problem. You can find more details about AlphaFold 2 in &lt;a href=&quot;https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology&quot; rel=&quot;nofollow noreferrer&quot;&gt;this DeepMind blog post&lt;/a&gt; and about AlphaFold 1 in &lt;a href=&quot;https://deepmind.com/blog/article/AlphaFold-Using-AI-for-scientific-discovery&quot; rel=&quot;nofollow noreferrer&quot;&gt;this other blog post&lt;/a&gt; or the &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1923-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;associated paper published in Nature this year&lt;/a&gt;. You can find the code for AlphaFold 1 &lt;a href=&quot;https://github.com/deepmind/deepmind-research/tree/master/alphafold_casp13&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but there are other community/open-source implementations.&lt;/p&gt;&#xA;&lt;p&gt;Despite the importance of the problem and achievement, there is clearly a lot of hype about this breakthrough (given also that it was achieved by DeepMind). This is also discussed in &lt;a href=&quot;https://www.youtube.com/watch?v=W7wJDJ56c88&amp;amp;ab_channel=LexFridman&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; by Lex Fridman.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T19:21:51.183" LastActivityDate="2020-12-07T19:21:51.183" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25007" PostTypeId="2" ParentId="24962" CreationDate="2020-12-05T10:35:46.223" Score="1" Body="&lt;p&gt;Your graph looks to me like a typical learning curve plotted for training process in reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;Looking at it in detail I can say:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is clearly some learning occurring.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a strong random element throughout. As you say the epsilon is reduced to near zero by episode 2000, and I would assume a driving simulation is mostly deterministic but with a lot of state hidden from the agent, this mainly implies that episodes are started in different states.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There may be an effect from continued learning that causes the agent performance to vary so much. However, it is more likely towards the end that the agent is encountering new never-seen-before states and making poor decisions in them.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The resulting graph matches to my experience of training DQN agents where some of the hyperparameters are off. Probably you need to do some kind of exploration or search through those parameters.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;From your comments:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Perhaps the model is subsequently overfitting? Or maybe there is some catastrophic forgetting?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These are possibilities that I cannot rule out by looking at the graph.&lt;/p&gt;&#xA;&lt;p&gt;However, I think it is more likely that you have some agent design or hyperparameters that are not a good fit to the problem. Those hyperparameters could be almost anything - not enough episodes to cover variation, neural network too simple, neural network too complicated, epsilon decay too fast, poor choice of regularisation, experience replay memory too small, poor choice of optimiser, etc etc.&lt;/p&gt;&#xA;&lt;p&gt;It is also possible you have limited the state model to the point that learning is hard or even impossible. A common beginner's mistake here is to use static sensor information such as a single current screenshot on each timestep, so the agent has no way to assess how fast it is going, or which way it is already turning etc (if your input includes direct knowledge of current speed, turning etc then this is probably not a problem for you, I am just raising one of many possibilities).&lt;/p&gt;&#xA;&lt;p&gt;One thing that may help you understand the agent's learning performance better is instead of plotting the &lt;em&gt;learning&lt;/em&gt; episode graph, plot a &lt;em&gt;test&lt;/em&gt; graph, perhaps once every ~100 episodes, where you run some number of episodes without learning and with &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0$&lt;/span&gt;, and take average of the metrics you are interested in. This will remove some of the randomness from the values you are plotting and give you a much better sense of training progress than your current plot.&lt;/p&gt;&#xA;&lt;p&gt;With Q learning, you may also get significant benefit by setting a non-zero minimum &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon = \text{max}(\text{eps_decay}(n), 0.01)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\text{eps_decay}(n)$&lt;/span&gt; is your current epsilon decay function for step &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. That is because the off-policy updates with exploration are still helpful at all stages of learning.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-12-05T16:47:21.813" LastActivityDate="2020-12-05T16:47:21.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25009" PostTypeId="2" ParentId="24971" CreationDate="2020-12-05T13:04:16.347" Score="0" Body="&lt;p&gt;Based on the clarifications given in the comments on the original question, i will try my best to give an answer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;If the number of features increases, does the approximation capability of an ANN improve theoretically speaking?&lt;/strong&gt;&#xA;It depends on whether this additional feature adds 'usefulness' to the data that was already supported. If you add a feature that is already in the dataset, it obviously will not increase the approximation capabilities. If you add a feature that is not yet in the current set of features and does influence the thing you are trying to approximate, then yes! There are some exceptions in this case of course, such as what if features are multicorrelated etc. It is not super important or measurable in ANNs, but statistics has a bunch of theory on this stuff. Simple statistical multiple regression has 4 assumptions and also checks for multicollinearity to see whether it is possible get 'useful' results from the data etc. As ANNs is super general, this is not applicable, but you could argue that similar practices could be looked into when seeking the best possible 'theoretical' validity of the features that you are using.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Does adding more features make your ANNs produce more accurate results (in practice)?&lt;/strong&gt;&#xA;Again, it depends. If you indeed added a useful feature, then the possibility exists that your ANN will in practice be able to produce more accurate results. But if your ANN is not able to converge to a solution, then your results will be gibberish. It is more likely that a network is unable to converge if you just throw data at it. More data requires bigger networks etc. So it is highly dependent on your training method. AKA, if your added feature is useful and the method you use is sufficient, then I'd argue that, yes, adding a feature will most likely make your ANNs produce more accurate results.&lt;/p&gt;&#xA;" OwnerUserId="34383" LastActivityDate="2020-12-05T13:04:16.347" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25010" PostTypeId="2" ParentId="24938" CreationDate="2020-12-05T19:30:09.893" Score="0" Body="&lt;p&gt;After tinkering a bit more with my experiment, i got it to consistently manifest the intended behavior after around 200 episodes.&lt;/p&gt;&#xA;&lt;p&gt;Changes to the model itself were minimal: i replaced the loss function on the actor to &lt;code&gt;tf.losses.softmaxCrossEntropy&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Some changes to the training environment seemed to have a significant impact and improved training:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ending the episode after the reward reaches a minimum threshold - this prevents the models being polluted when the car was stock in a corner or flat against a wall.&lt;/li&gt;&#xA;&lt;li&gt;Making sure that the reward used in training was inline with the action that produced that reward - the model training in my case is asynchronous to the physics of the environment - i am sampling the simulation and providing inputs, but the simulation is not interrupted by model related processing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42665" LastActivityDate="2020-12-05T19:30:09.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25011" PostTypeId="1" CreationDate="2020-12-06T00:51:05.047" Score="2" ViewCount="52" Body="&lt;p&gt;I noticed that the TensorFlow library includes a &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;use_bias&lt;/code&gt; parameter for the &lt;code&gt;Dense&lt;/code&gt; layer&lt;/a&gt;, which is set to &lt;code&gt;True&lt;/code&gt; by default, but allows you to disable it. At first glance, it seems unfavorable to turn off the biases, as this may negatively affect data fitting and prediction.&lt;/p&gt;&#xA;&lt;p&gt;What is the purpose of layers without biases?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-12-07T14:19:59.190" LastActivityDate="2020-12-07T14:19:59.190" Title="What's the purpose of layers without biases?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;hidden-layers&gt;&lt;algorithmic-bias&gt;&lt;fully-connected-layer&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25012" PostTypeId="1" CreationDate="2020-12-06T03:00:21.877" Score="5" ViewCount="129" Body="&lt;p&gt;There are certain proteins that contain metal components, known as &lt;a href=&quot;http://en.wikipedia.org/wiki/metalloprotein&quot; rel=&quot;nofollow noreferrer&quot;&gt;metalloproteins&lt;/a&gt;. Commonly, the metal is at the active site which needs the most prediction precision. Typically, there is only one (or a few) metals in a protein, which contains far more other atoms. So, the structural data that we could be used to train AlphaFold will contain far less information about the metal elements. Not to mention most proteins don't have metals at all (it is estimated that only 1/2-1/4 of all proteins contain metals [&lt;a href=&quot;http://en.wikipedia.org/wiki/metalloprotein&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]).&lt;/p&gt;&#xA;&lt;p&gt;Given that maybe there is not enough structural data about protein local structure around metal atoms (e.g. Fe, Zn, Mg, etc.), then AlphaFold cannot predict local structure around metals well. Is that right?&lt;/p&gt;&#xA;&lt;p&gt;I also think that the more complex electron shell of metal also makes the data less useful, since its bounding pattern is more flexible than carbon, etc.&lt;/p&gt;&#xA;" OwnerUserId="25322" LastEditorUserId="19524" LastEditDate="2020-12-06T23:01:32.923" LastActivityDate="2021-04-27T15:00:04.420" Title="Can AlphaFold predict proteins with metals well?" Tags="&lt;machine-learning&gt;&lt;alpha-fold&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25013" PostTypeId="1" AcceptedAnswerId="25026" CreationDate="2020-12-06T08:25:45.947" Score="2" ViewCount="37" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt;, this section confuses me:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In our model, we share the same weight matrix between the two embedding layers [in the encoding section] and the pre-softmax linear transformation [output of the decoding section]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Shouldn't the weights be different, and not the same? Here is my understanding:&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, let us use the English-to-French translation task where we have &lt;span class=&quot;math-container&quot;&gt;$n^e$&lt;/span&gt; number of English words in our dictionary and &lt;span class=&quot;math-container&quot;&gt;$n^f$&lt;/span&gt; number of French words.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In the encoding layer, the input tokens are &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$n^e$&lt;/span&gt; one-hot vectors, and are embedded with a &lt;span class=&quot;math-container&quot;&gt;$n^e$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; learned embedding matrix.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the output of the decoding layer, the final step is a linear transformation with weight matrix &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$n^f$&lt;/span&gt;, and then applying softmax to get the probability of each french word, and choosing the french word with the highest probability.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How is it that the &lt;span class=&quot;math-container&quot;&gt;$n^e$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$n^{model}$&lt;/span&gt; input embedding matrix share the same weights as the &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$n^f$&lt;/span&gt; decoding output linear matrix? To me, it seems more natural for both these matrices to be learned independently from each other via the training data, right? Or am I misinterpreting the paper?&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="2444" LastEditDate="2020-12-06T11:21:28.973" LastActivityDate="2020-12-06T21:03:43.913" Title="Transformers: how does the decoder final layer output the desired token?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;bert&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25015" PostTypeId="1" AcceptedAnswerId="25030" CreationDate="2020-12-06T09:23:08.503" Score="2" ViewCount="54" Body="&lt;p&gt;I was reading the paper &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It seems like the last step of the encoder is a LayerNorm(relu(WX + B) + X), i.e. an add + normalization. This should result in a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; matrix, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the length of the input to the encoder.&lt;/p&gt;&#xA;&lt;p&gt;How do we convert this &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; matrix into the keys &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and values &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; that are fed into the decoder's encoder-decoder attention step?&lt;/p&gt;&#xA;&lt;p&gt;Note that, if &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the number of attention heads in the model, the dimensions of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; should both be &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$\frac{d^{model}}{h}$&lt;/span&gt;. For &lt;span class=&quot;math-container&quot;&gt;$h=8$&lt;/span&gt;, this means we need a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$\frac{d^{model}}{4}$&lt;/span&gt; matrix.&lt;/p&gt;&#xA;&lt;p&gt;Do we simply add an extra linear layer that learns a &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$\frac{d^{model}}{4}$&lt;/span&gt; weight matrix?&lt;/p&gt;&#xA;&lt;p&gt;Or do we use the output of the final Add &amp;amp; Norm layer, and simply use the first &lt;span class=&quot;math-container&quot;&gt;$\frac{d^{model}}{4}$&lt;/span&gt; columns of the matrix and discard the rest?&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-06T21:19:34.330" LastActivityDate="2020-12-06T22:29:03.577" Title="Transformers: how to get the output (keys and values) of the encoder?" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;bert&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25016" PostTypeId="2" ParentId="24887" CreationDate="2020-12-06T10:17:06.227" Score="1" Body="&lt;p&gt;When you start off learning about Q-learning, you start with a simple example that has a few states. For each of the states, you try to &lt;em&gt;estimate&lt;/em&gt; what the 'value' is of that state. Because there are so few states, it is possible to store these values in a table (it is also useful for the intuitiveness of the explanation).&lt;/p&gt;&#xA;&lt;p&gt;However, if you start trying to solve more 'real-life' problems, the number of states can be insanely huge. The essence however will stay the same. You are trying to estimate what you want to do next, based on an estimation of how good each state is that you can end up in. However, now the values are 'most of the times' not stored in a table anymore, as the approach will often come down to using an ANN to estimate the value function.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Answer to your question&lt;/strong&gt;: You are going to run into a lot of problems when training some model with Q-learning if your table is not able to store the values of possible states that it can come across. In practice, most implementations do not use a Q-table and just use an ANN, which alleviates the problem of having to 'define' how many states your problem consists of.&lt;/p&gt;&#xA;" OwnerUserId="34383" LastActivityDate="2020-12-06T10:17:06.227" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25017" PostTypeId="1" CreationDate="2020-12-06T11:38:21.737" Score="0" ViewCount="37" Body="&lt;p&gt;I just took a course on deep learning where one part of the syllabus was image classification and object recognition using CNNs, but I wonder how deep learning can be applied to apply certain filters or complex edition on a given photography. For example, a common action on basic photography is to remove light reflexion from human eyes when doing a portrait photo.&lt;/p&gt;&#xA;&lt;p&gt;How could this be done with a deep learning algorithm? The simplest idea seems to train a network using two versions for each photography in the training set: one without light correction and another one with it. But I guess that there might be more suitable procedures to perform such kind of actions with better results.&lt;/p&gt;&#xA;&lt;p&gt;I would love to hear what the community has to say about this. Any recommendation?&lt;/p&gt;&#xA;" OwnerUserId="42760" LastEditorUserId="2444" LastEditDate="2020-12-07T11:12:25.733" LastActivityDate="2020-12-07T11:12:25.733" Title="How to edit a photo using deep learning?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25018" PostTypeId="1" CreationDate="2020-12-06T13:21:26.717" Score="1" ViewCount="118" Body="&lt;p&gt;I have a Reinforcement-Learning environment where the state is an array of 0s and 1s with length equals to the number of users the agent must satisfy (11 users).&lt;/p&gt;&#xA;&lt;p&gt;The agent must choose one of 12 resources for the 11 users according to the state array. If &lt;code&gt;state[0] == 1&lt;/code&gt;, that means that user0 needs a resource, so the agent must choose a resource out of the 12 resources it has. So, the action array's first element would be, for example: &lt;code&gt;action[0] = 10&lt;/code&gt;, which means that resource 10 was allocated to user0.&lt;/p&gt;&#xA;&lt;p&gt;If the next user (user1) is asking for a resource as well, then the number of resources to choose from is &lt;code&gt;12 - 1&lt;/code&gt;, in other words, because resource10 was already allocated to user0, it cannot be allocated to another user.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;code&gt;state[X] == 0&lt;/code&gt;, it means that userX is not asking for a resource, therefore it must not be allocated any resource.&lt;/p&gt;&#xA;&lt;p&gt;An example of a state array:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;An example of an action array according to the state array example: (resource count starts at 0 | -1 indicates no resource was allocated)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[10, 2, -1, -1, -1, 3, 11, 5, -1, -1, -1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm new to Reinforcement Learning and Deep Learning, and I have no idea how to translate that into a neural network.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastActivityDate="2020-12-07T01:43:50.547" Title="How to build a DQN agent with state and action being arrays?" Tags="&lt;deep-learning&gt;&lt;keras&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;deep-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25019" PostTypeId="2" ParentId="25011" CreationDate="2020-12-06T13:53:47.410" Score="1" Body="&lt;p&gt;Bias is one of the hyperparameters in neural networks, which let you shift activation function. Disabling bias means setting bias to be zero.&lt;/p&gt;&#xA;&lt;p&gt;Even though, in many cases, bias is a big help for successful learning, in some cases, you may want to add an extra constraint to your neural network in finding the objective function. For example, in the paper below, a zero-bias layer as the last layer helps increase the output's interpretability.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/9173537&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/abstract/document/9173537&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35757" LastActivityDate="2020-12-06T13:53:47.410" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25020" PostTypeId="5" CreationDate="2020-12-06T15:28:00.133" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T15:28:00.133" LastActivityDate="2020-12-06T15:28:00.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25021" PostTypeId="4" CreationDate="2020-12-06T15:28:00.133" Score="0" Body="For questions about the different Inception models. The first inception model was proposed in &quot;Going deeper with convolutions&quot; (2014) by Christian Szegedy et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T15:28:00.133" LastActivityDate="2020-12-06T15:28:00.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25022" PostTypeId="5" CreationDate="2020-12-06T15:32:56.100" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T15:32:56.100" LastActivityDate="2020-12-06T15:32:56.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25023" PostTypeId="4" CreationDate="2020-12-06T15:32:56.100" Score="0" Body="For questions related to ImageNet, a large visual database designed for use in visual object recognition software research." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T15:32:56.100" LastActivityDate="2020-12-06T15:32:56.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25024" PostTypeId="5" CreationDate="2020-12-06T20:10:50.167" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T20:10:50.167" LastActivityDate="2020-12-06T20:10:50.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25025" PostTypeId="4" CreationDate="2020-12-06T20:10:50.167" Score="0" Body="For questions about selection operators (or methods) for evolutionary algorithms. Examples of selection methods are the fitness proportionate selection (aka roulette wheel selection), tournament selection, and rank-based selection." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-06T20:10:50.167" LastActivityDate="2020-12-06T20:10:50.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25026" PostTypeId="2" ParentId="25013" CreationDate="2020-12-06T21:03:43.913" Score="1" Body="&lt;p&gt;I found the answer by reading the paper referenced by that section, &lt;a href=&quot;https://arxiv.org/pdf/1608.05859.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using the output embedding to improve language models&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Based on this observation, we propose threeway weight tying (TWWT), where the input embedding of the decoder, the output embedding of&#xA;the decoder and the input embedding of the encoder are all tied. The single source/target vocabulary of this model is the union of both the source&#xA;and target vocabularies. In this model, both in the&#xA;encoder and decoder, all subwords are embedded&#xA;in the same duo-lingual space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It seems like they learned a single embedding matrix (&lt;span class=&quot;math-container&quot;&gt;$n^e + n^f$&lt;/span&gt;) x &lt;span class=&quot;math-container&quot;&gt;$d^{model}$&lt;/span&gt; in dimension.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-06T21:03:43.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25028" PostTypeId="2" ParentId="9105" CreationDate="2020-12-06T21:38:59.283" Score="1" Body="&lt;h2&gt;No magical formula&lt;/h2&gt;&#xA;&lt;p&gt;As already stated &lt;a href=&quot;https://ai.stackexchange.com/a/15901/2444&quot;&gt;in this answer&lt;/a&gt;, the definition of the fitness function depends on the problem, given that it essentially determines the solutions that you are looking for, and it raises similar issues to the ones you would encounter while defining a &lt;a href=&quot;https://ai.stackexchange.com/a/22854/2444&quot;&gt;&lt;em&gt;reward function&lt;/em&gt;&lt;/a&gt; in reinforcement learning, such as &lt;em&gt;fitness misspecification&lt;/em&gt; (in fact, the concepts of a reward function and fitness functions are similar, although, in some cases, a fitness function is more similar to a cost function in supervised learning problems: as an example, take a look at &lt;a href=&quot;https://github.com/DEAP/deap/blob/master/examples/gp/symbreg.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;the fitness functions used to solve symbolic regression&lt;/a&gt;). The design of a fitness function is an engineering problem, in the sense that you need to think about the solutions you are looking for, and what represents a good and bad solution. So, I cannot give you the magical formula to define the fitness function for all problems, but I can give you some info that can be useful to guide the design of a fitness function.&lt;/p&gt;&#xA;&lt;h3&gt;What is a fitness function?&lt;/h3&gt;&#xA;&lt;p&gt;Let's start with the definition. A fitness function is a function that maps the chromosome representation into a scalar value (a real number) that quantifies the quality of the chromosome (i.e. solution), so it's a function of the form&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f : \Gamma \rightarrow \mathbb{R},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\Gamma$&lt;/span&gt; is the space of chromosomes (where a chromosome is a numerical encoding of a solution for your problem that is suitable for evaluation and modification).&lt;/p&gt;&#xA;&lt;h3&gt;Different types of optimization problems&lt;/h3&gt;&#xA;&lt;p&gt;There are different optimization problems that affect the definition of the fitness function [&lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Unconstrained&lt;/strong&gt;: this is the simplest case where the fitness function corresponds to your objective function&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Constrained&lt;/strong&gt;: in this case, your fitness function could be composed of two terms: the original objective and a penalty term (that penalizes solutions that do not satisfy the constraints)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Multi-objective&lt;/strong&gt;: where you have multiple fitness functions (one for each objective) and the final fitness function is a combination of these multiple fitness functions; in this context, you will often encounter terms like &lt;strong&gt;Pareto-optimality&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Dynamic&lt;/strong&gt; (or noisy), where the fitness of solutions can change over time or depend on some noise component (e.g. Gaussian noise)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Encodings and fitness functions&lt;/h3&gt;&#xA;&lt;p&gt;In genetic algorithms, a form of evolutionary algorithms, the chromosomes are often assumed to be binary (i.e. &lt;span class=&quot;math-container&quot;&gt;$\Gamma$&lt;/span&gt; is a space of binary arrays), so this can limit the way you can evaluate them.&lt;/p&gt;&#xA;&lt;p&gt;In other evolutionary approaches, the solutions may be encoded differently and represent something different than just a collection of numbers. In particular, in genetic programming, the solutions are programs, so the fitness should correspond to something that the program you are looking for is supposed to accomplish. For example, if you are looking for an analytical expression (e.g. a polynomial) that minimises the squared error with another expression, then the fitness would be e.g. the squared error.&lt;/p&gt;&#xA;&lt;p&gt;See section 10.3 (p. 180) of [&lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] for more details.&lt;/p&gt;&#xA;&lt;h3&gt;Cheap fitness evaluation&lt;/h3&gt;&#xA;&lt;p&gt;One of the most desirable properties that you should look for while designing a fitness function is how cheap it is to evaluate the fitness of an individual. Ideally, the fitness evaluation should be quite &lt;strong&gt;cheap&lt;/strong&gt; in order for the evolutionary algorithm to be feasible and practically useful. If it takes 1 year to perform an evaluation of an individual, then you don't get anything done. Alternatively, you can approximate the fitness of an individual, if the computation of the exact fitness is too expensive.&lt;/p&gt;&#xA;&lt;h3&gt;Absolute and relative measures&lt;/h3&gt;&#xA;&lt;p&gt;The fitness function can provide an &lt;strong&gt;absolute&lt;/strong&gt; or &lt;strong&gt;relative&lt;/strong&gt; (to other chromosomes in the population or other competing populations) measure of the quality of a chromosome (or solution). Relative fitness functions are used in co-evolutionary algorithms and are suited for situations where an absolute measure of the quality of a solution is not possible [&lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;h3&gt;Fitness sharing&lt;/h3&gt;&#xA;&lt;p&gt;There is also the concept of &lt;strong&gt;fitness sharing&lt;/strong&gt; (see section 9.6.1 of [&lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], p. 165) where the fitness of an individual can be adjusted based on the fitness of other individuals&lt;/p&gt;&#xA;&lt;h2&gt;Further reading&lt;/h2&gt;&#xA;&lt;p&gt;Apart from the [&lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], which has several chapters on evolutionary computation, you could take a look at &lt;a href=&quot;https://bi.snu.ac.kr/SEMINAR/EvolComp/EvolComp_Vol2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Computation 2: Advanced Algorithms and Operators&lt;/a&gt;. The paper &lt;a href=&quot;http://www.nelsonrobotics.org/paper_archive_nelson/nelson-jras-2009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fitness functions in evolutionary robotics: A survey and analysis&lt;/a&gt; (2009) also seems useful from the title and abstract (and other parts that I quickly read).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T14:04:11.443" LastActivityDate="2021-01-20T14:04:11.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25030" PostTypeId="2" ParentId="25015" CreationDate="2020-12-06T22:29:03.577" Score="2" Body="&lt;p&gt;I have read the OpenNMT source code (&lt;a href=&quot;https://github.com/OpenNMT/OpenNMT-py/blob/cd29c1dbfb35f4a2701ff52a1bf4e5bdcf02802e/onmt/modules/multi_headed_attn.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/OpenNMT/OpenNMT-py/blob/cd29c1dbfb35f4a2701ff52a1bf4e5bdcf02802e/onmt/modules/multi_headed_attn.py&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;It seems like an extra linear layer learns the weights &lt;span class=&quot;math-container&quot;&gt;$W^{key}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W^{value}$&lt;/span&gt; (plus biases), so to get the output (keys and values), you multiply the output of the encoder's final add + norm layer by &lt;span class=&quot;math-container&quot;&gt;$W^{key}$&lt;/span&gt; to get the keys, and by &lt;span class=&quot;math-container&quot;&gt;$W^{value}$&lt;/span&gt; to get the values.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, these weights and biases seem to be independent across each of the decoding layer. So you feed the same encoder output (add + norm layer output), but multiply by different &lt;span class=&quot;math-container&quot;&gt;$W^{key}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W^{value}$&lt;/span&gt; matrices and add by different biases for each of the decoding layer, resulting in different keys and values for each layer&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-06T22:29:03.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25031" PostTypeId="2" ParentId="25012" CreationDate="2020-12-06T22:50:48.607" Score="4" Body="&lt;p&gt;Let me address first some of the things you wrote in your question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;There are certain proteins that contain metal components, known as&#xA;metalloproteins.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Natural proteins do not ever contain metal components as far as we know. Natural proteins are composed of natural amino acids which only contain H,C,O,N,S. Selenocysteine contains Se (also a non-metal!) but it's a proteinogenic amino acid, which means it's a precursor to proteins but doesn't typically show up in the protein itself. From the Wikipedia page that you gave us in your question: &amp;quot;Metalloprotein is a generic term for a protein that contains a metal ion cofactor&amp;quot; and &amp;quot;A cofactor is a non-protein chemical compound or metallic ion&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;But your question still deserves an answer, because even though it would incorrect to call a co-factor &amp;quot;part&amp;quot; of the protein, they can still affect the folding and the overall shape. But let's first address one last part of your question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given that maybe there is not enough structural data about protein&#xA;local structure around metal atoms (e.g. Fe/Zn/Mg), then AlphaFold&#xA;cannot predict local structure around metals well. Is that right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The first sentence of the Wikipedia article that you linked in your question, says &amp;quot;For instance, at least 1000 human proteins (out of ~20,000) contain zinc-binding protein domains [3] although there may be up to 3000 human zinc metalloproteins [4].&amp;quot; Therefore, while metalloproteins might not be the majority of all proteins, there's a decent enough number of them that are of relevance to the human body, and therefore constructing training databases that contain enough metalloproteins (or even 100% metalloproteins, if desired) is not difficult.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mattermodeling.stackexchange.com/q/3849/5&quot;&gt;I mentioned a bit elsewhere&lt;/a&gt; that AlphaFold was used to predict protein structures in the CASP competition, for which you can &lt;a href=&quot;https://predictioncenter.org/casp14/targetlist.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;see for yourself&lt;/a&gt; that many/most of the proteins for which contestants (such as DeepMind) need to predict the structure, come from studies of proteins of relevance to humans because the CASP structures typically come from X-ray crystallography studies, which are typically done on proteins of relevance to humans.&lt;/p&gt;&#xA;&lt;p&gt;You can also &lt;a href=&quot;https://predictioncenter.org/decoysets2019/results.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;see for yourself&lt;/a&gt; not only the &amp;quot;target list&amp;quot; that I showed above, but also the results of the competition which will show how well AlphaFold performed in CASP13 (2018) and CASP14 (2020) for metalloproteins.&lt;/p&gt;&#xA;&lt;p&gt;Finally:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I also think that the more complex electron shell of metal also makes&#xA;the data less useful, since its bounding pattern is more flexible than&#xA;carbon, etc.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is true that metals are typically harder to model than C,H,O,N,S, and even Se, if doing &lt;em&gt;ab initio&lt;/em&gt; calculations on the metals or &lt;a href=&quot;https://mattermodeling.stackexchange.com/questions/tagged/metal-complexes&quot;&gt;metail-containing complexes&lt;/a&gt;. However, the purpose of machine learning in protein folding studies is to &lt;em&gt;&lt;strong&gt;skip&lt;/strong&gt;&lt;/em&gt; &lt;em&gt;ab initio&lt;/em&gt;, statistical-dynamical and/or molecular-dynamical calculations of the relevant structures and simply use training data to predict the protein structures. That being said, there needs to be enough training data available (as you correctly pointed out) to learn what happens near the metal co-factors: &lt;strong&gt;The answer to this&lt;/strong&gt; is that there are indeed enough metalloproteins to sufficiently populate a training set, but &lt;strong&gt;they won't contain enough of the specific metals involved in every metalloprotein&lt;/strong&gt;. For example, lots of data will be available for proteins containing Fe since Fe is in hemoglobin (for example) which is essential to the functioning of red blood cells to absorb oxygen, but the protein &lt;a href=&quot;https://en.wikipedia.org/wiki/Vanabins&quot; rel=&quot;nofollow noreferrer&quot;&gt;vanabins&lt;/a&gt; contains vanadium which is much rarer and therefore training data involving it will be much less available. You're also correct that metal elements can form more bonds than typical elements found in organic compounds.&lt;/p&gt;&#xA;&lt;p&gt;So it depends on the metal in the relevant co-factor. Fe-based co-factors will have quite a lot of training data available, as well as Mg-based ones, Zn-based ones, and a lot of other ones which contain the &amp;quot;more common&amp;quot; metals. For proteins like vanabins which contains vanadium, you are quite correct that training data will be limited, but also keep in mind that vanabins is a very rare protein found in &lt;a href=&quot;https://en.wikipedia.org/wiki/Ascidiacea&quot; rel=&quot;nofollow noreferrer&quot;&gt;sea squirts&lt;/a&gt; and we already know more about its structure (through X-ray crystallography, which means we don't need machine learning for it) than we even know about what it even does. The chances of other vanadium-containing co-factors in metalloproteins being very significant is too low to justify working on protein folding algorithms specifically for them.&lt;/p&gt;&#xA;" OwnerUserId="19524" LastEditorUserId="36737" LastEditDate="2021-04-27T15:00:04.420" LastActivityDate="2021-04-27T15:00:04.420" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="25032" PostTypeId="2" ParentId="25018" CreationDate="2020-12-07T01:09:12.627" Score="2" Body="&lt;p&gt;Let's assume you would like to work with a classic DQN. You need to train the Q-network where inputs are the states and actions. The DQN is a function of Q(states, actions). The network is supposed to predict Q-value. The agent must pick up the action that produces the highest Q by giving the all possible actions to the network, in your case. Let's assume the current state is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You must pick up an item for the user which state is 1? you need to generate the all possible actions, for example,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1, 2, -1, -1, -1, 3, 4, 5, -1, -1, -1]&#xA;[12, 1, -1, -1, -1, 2, 3, 4, -1, -1, -1]&#xA;[11, 12, -1, -1, -1, 1, 2, 3, -1, -1, -1]&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will need to generate all of these possible actions into the Q-network. Your worst case would be around 479 million possible actions to calculate in DQN.&lt;/p&gt;&#xA;&lt;p&gt;If you would like to implement DQNs for this problem, I do recommend you to check out REINFORCE -&amp;gt; DQN -&amp;gt; Advantage Actor-Critic (A2C), which is the combination of policy network and DQN. With A2C, you can produce continuous actions with a policy network, action in range a[i]∈(0,1)*12, then probably round the generated actions. Then, the next problem is the policy network may produce an impossible action. So, you might skip to the nearest possible item.&lt;/p&gt;&#xA;&lt;p&gt;For further reading on continuous action domains, I recommend to check out &lt;a href=&quot;https://arxiv.org/abs/1509.02971&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDPG&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42774" LastEditorUserId="42774" LastEditDate="2020-12-07T01:43:50.547" LastActivityDate="2020-12-07T01:43:50.547" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25034" PostTypeId="1" CreationDate="2020-12-07T07:48:26.407" Score="0" ViewCount="22" Body="&lt;p&gt;I have academic pdf data. I am using OCR for converting it into text format. The pdf has a few mathematical equations and terms which are acting as noise for my task. Any way through which the task of removing the equations and extracting only the theoretical text data could be automated?&lt;/p&gt;&#xA;" OwnerUserId="17052" LastEditorUserId="2444" LastEditDate="2020-12-13T12:07:52.837" LastActivityDate="2020-12-13T12:07:52.837" Title="Is it possible to automatically remove ignore (or remove) the equations (and other noisy elements) while performing OCR?" Tags="&lt;natural-language-processing&gt;&lt;data-preprocessing&gt;&lt;optical-character-recognition&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25039" PostTypeId="5" CreationDate="2020-12-07T13:44:42.277" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T13:44:42.277" LastActivityDate="2020-12-07T13:44:42.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25040" PostTypeId="4" CreationDate="2020-12-07T13:44:42.277" Score="0" Body="For questions about inductive programming, which are techniques that attempt to learn programs from examples or constraints. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T13:44:42.277" LastActivityDate="2020-12-07T13:44:42.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25041" PostTypeId="1" AcceptedAnswerId="25054" CreationDate="2020-12-07T13:46:00.990" Score="1" ViewCount="31" Body="&lt;p&gt;The Decoder mask, also called &amp;quot;look-ahead mask&amp;quot;, is applied in the Decoder side to prevent it from attending future tokens. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[0, 1, 1, 1, 1]&#xA;[0, 0, 1, 1, 1]&#xA;[0, 0, 0, 1, 1]&#xA;[0, 0, 0, 0, 1]&#xA;[0, 0, 0, 0, 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But is this mask applied only in the first Decoder block? Or to all its blocks?&lt;/p&gt;&#xA;" OwnerUserId="26580" LastActivityDate="2020-12-08T00:10:26.617" Title="Is the Decoder mask (triangular mask) applied only in the first decoder block, or to all blocks in Decoder?" Tags="&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25042" PostTypeId="1" CreationDate="2020-12-07T14:13:05.710" Score="0" ViewCount="16" Body="&lt;p&gt;I'm currently starting a research project focused on NLP.&lt;/p&gt;&#xA;&lt;p&gt;One of the steps involved in this project will be the development of a text simplification system, probably using a neural encoder-decoder architecture.&lt;/p&gt;&#xA;&lt;p&gt;For most Text Simplification research available, the most commonly used dataset is one derived from pairing Wikipedia entries in both English and Simplified English. My problem arises from the fact that the focus of my research is not on the English Language, but rather in Portuguese, specifically Portugal Portuguese.&lt;/p&gt;&#xA;&lt;p&gt;There exists no Simple Portuguese Wikipedia page and it seems that there exists no publicly available text simplification dataset in Portugal Portuguese at all. Due to this fact I'm curious if there would be any way of tackling this problem. Maybe having a dataset simply of complex Portuguese and simple portuguese, but with no pairings, although I'm not quite sure how that could be formulated to train a NN with.&lt;/p&gt;&#xA;&lt;p&gt;So my question is if there are any text simplification datasets in Portugal (or maybe Bazil, as a last resource) Portuguese and, if not, what would be the optimal way to build that dataset.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="31147" LastEditorUserId="31147" LastEditDate="2020-12-07T15:45:31.863" LastActivityDate="2020-12-07T15:45:31.863" Title="Finding or creating a dataset for Neural Text Simplification" Tags="&lt;natural-language-processing&gt;&lt;machine-translation&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25043" PostTypeId="5" CreationDate="2020-12-07T14:56:12.533" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T14:56:12.533" LastActivityDate="2020-12-07T14:56:12.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25044" PostTypeId="4" CreationDate="2020-12-07T14:56:12.533" Score="0" Body="For questions about the question answering discipline, which is concerned with building systems that automatically answer questions posed by humans in a natural language." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T14:56:12.533" LastActivityDate="2020-12-07T14:56:12.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25045" PostTypeId="1" CreationDate="2020-12-07T15:44:12.203" Score="0" ViewCount="38" Body="&lt;p&gt;I have a custom environment, where the state is a 2D matrix of 11 rows (equals to number of users to satisfy) and 3 columns. Each column can take the value of either 0 or 1, and in each row, there can be only one column with the value 1.&lt;/p&gt;&#xA;&lt;p&gt;Example of a state matrix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(&#xA; (1, 0, 0),&#xA; (0, 0, 0),&#xA; (0, 1, 0),&#xA; (1, 0, 0),&#xA; (0, 0, 1),&#xA; (0, 0, 1),&#xA; (0, 0, 0),&#xA; (0, 0, 0),&#xA; (1, 0, 0),&#xA; (0, 1, 0),&#xA; (0, 0, 0)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The action is another 2D matrix, with 12 rows (equals total number of resources available to the agent) and 11 columns (equals number of users the agent must satisfy). The agent must choose a resource for each user. Each user can only have one resource, and each resource can be given to only one user at each time step.&lt;/p&gt;&#xA;&lt;p&gt;Example of action matrix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(&#xA; ('O', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'O', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'O', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'O', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'O', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'O', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'O', 'Z'),&#xA; ('Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z')&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;'O' stands for One, which means the resource was allocated to the user, and 'Z' stands for Zero which means that the resource wasn't allocated to the user.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;row[0] = ('O', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z')&lt;/code&gt;&#xA;which means resource0 is given to user0 (because &lt;code&gt;row[0][0] == 'O'&lt;/code&gt;), because in the state matrix, &lt;code&gt;row[0] = (1, 0, 0)&lt;/code&gt; which means that user0 is asking for a resource to be given to him.&lt;/p&gt;&#xA;&lt;p&gt;A reward is returned by the step() method in the Reinforcement Learning environment if the agent hasn't given a resource to a user that didn't ask for a resource to be given. Otherwise, a penalty is returned.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the state matrix has &lt;code&gt;row[1] = (0, 0, 0)&lt;/code&gt;; if, say, in the action matrix &lt;code&gt;row[1] = ('Z', 'O', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z', 'Z')&lt;/code&gt;, then resource1 is allocated to user1 (because &lt;code&gt;row[1][1] == 'O'&lt;/code&gt;), but user1 didn't ask for a resource, hence a penalty is returned.&lt;/p&gt;&#xA;&lt;p&gt;The reward varies depending on which column is equal to 1 in each row in the state matrix.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to create a CNN where the input is the state matrix and the output is the action matrix. I have tried it using a convolution part followed by a de-convolution part. However, I'm failing to see how I should indicate to the neural network that for each row of the state matrix, it must choose a resource to allocate to the user if he asked for it (constraint 1), and if a resource is chosen, it is locked for that user and cannot be given to another user at that time step (constraint 2).&lt;/p&gt;&#xA;&lt;p&gt;Instead of having the output of the CNN as a 2D matrix, I thought I'd make the output an array with the resources numbered from 0 to 11 (12 resources), chosen by the network for each user, then manually transform it into a matrix because the environment needs it as a matrix for the reward calculation. However, I'm not sure if that's a feasible idea.&lt;/p&gt;&#xA;" OwnerUserId="42372" LastEditorUserId="42372" LastEditDate="2020-12-07T17:14:46.547" LastActivityDate="2020-12-07T17:14:46.547" Title="CNN for a DQN agent with a 2D matrix state and action as a 2D matrix" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;keras&gt;&lt;deep-rl&gt;&lt;deep-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25046" PostTypeId="1" CreationDate="2020-12-07T16:33:20.157" Score="2" ViewCount="40" Body="&lt;p&gt;In page 4 of the paper &lt;a href=&quot;https://arxiv.org/pdf/2009.07047v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/2009.07047v1.pdf&lt;/a&gt;, it says the encoder &lt;span class=&quot;math-container&quot;&gt;$E_{R,X}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$VAE_1$&lt;/span&gt; tries to fool the discriminator with a contradictory loss to ensure that &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; are mapped to the same space. What do they mean by &amp;quot;contradictory loss&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="42707" LastActivityDate="2021-01-06T19:01:05.077" Title="What do they mean by &quot;contradictory loss&quot;?" Tags="&lt;generative-adversarial-networks&gt;&lt;variational-autoencoder&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25047" PostTypeId="2" ParentId="3723" CreationDate="2020-12-07T17:59:20.933" Score="3" Body="&lt;p&gt;In principle, yes, you can also evolve the genetic algorithm (or, in general, evolutionary algorithm), i.e. you can evolve its operations (such as the mutation and cross-over) and hyper-parameters (such as the size of the population or mutation rate). For example, you could use &lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic programming&lt;/a&gt; to evolve the cross-over operation of a genetic algorithm. However, these genetic operators and hyper-parameters are usually designed and determined by a human and do not change during the evolution process. Nevertheless, there is a framework for evolutionary computation in which reproduction and mutation can also evolve, known as &lt;a href=&quot;http://faculty.hampshire.edu/lspector/pubs/ace.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;auto-constructive evolution&lt;/a&gt;, and there are other examples in the literature of meta-optimization methods applied to evolutionary algorithms, such as&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://people.idsia.ch/%7Ejuergen/diploma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-... hook&lt;/a&gt; (1987, Schmidhuber's thesis)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007%2F3-540-58484-6_285&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parallel optimization of evolutionary algorithms&lt;/a&gt; (1994, by T. Bäck)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.7230&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adaptive and Self-adaptive Evolutionary Computations&lt;/a&gt; (1995, by Peter Angeline)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://journals.tubitak.gov.tr/elektrik/issues/elk-01-9-1/elk-9-1-2-0008-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Meta-Genetic Programming: Co-evolving the Operators of Variation&lt;/a&gt; (1998, by B. Edmonds)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;More generally, the optimization of an optimization algorithm is known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Meta-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;meta-optimization&lt;/a&gt; and/or &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperparameter_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;hyper-parameter optimization/tuning&lt;/a&gt;, so the idea of meta-optimization is not just restricted to evolutionary algorithms, but can be applied also to e.g. deep learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-28T18:41:58.503" LastActivityDate="2020-12-28T18:41:58.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25048" PostTypeId="2" ParentId="25046" CreationDate="2020-12-07T18:07:55.643" Score="1" Body="&lt;p&gt;The contradictory loss is the same loss function that the discriminator would normally use, except with deliberately incorrect labels. That is, when you train the generator, the output of the generator is fed to the discriminator, but instead of the correct label (typically &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for a false image), the opposite label is applied (e.g. &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for a real image).&lt;/p&gt;&#xA;&lt;p&gt;This label is condractictory in that it is incorrect and the opposite of the ground truth. However, this is useful, because reducing the loss of this signal represents the goal of the generator. It is important to note that during this phase of the training, you should &lt;em&gt;not&lt;/em&gt; update any parameters of the discriminator. The process is followed in order to find gradients for improving the generator only.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2020-12-07T18:07:55.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25049" PostTypeId="5" CreationDate="2020-12-07T19:16:25.727" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T19:16:25.727" LastActivityDate="2020-12-07T19:16:25.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25050" PostTypeId="4" CreationDate="2020-12-07T19:16:25.727" Score="0" Body="For questions about the concept of a chromosome (or genotype) in the context of genetic algorithms (a specific type of evolutionary algorithms). The chromosome can be represented as a binary or real-valued array, but there are other representations of solutions, especially in genetic programming (another sub-field of evolutionary algorithms), where solutions are often represented as trees." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T19:16:25.727" LastActivityDate="2020-12-07T19:16:25.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25051" PostTypeId="5" CreationDate="2020-12-07T21:30:36.460" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T21:30:36.460" LastActivityDate="2020-12-07T21:30:36.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25052" PostTypeId="4" CreationDate="2020-12-07T21:30:36.460" Score="0" Body="For questions about the concept of a hypothesis in the context of learning theory." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-07T21:30:36.460" LastActivityDate="2020-12-07T21:30:36.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25053" PostTypeId="1" AcceptedAnswerId="25094" CreationDate="2020-12-07T23:18:06.043" Score="3" ViewCount="72" Body="&lt;p&gt;The paper &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt; describes the transformer architecture that has an encoder and a decoder.&lt;/p&gt;&#xA;&lt;p&gt;However, I wasn't clear on what the cost function to minimize is for such an architecture.&lt;/p&gt;&#xA;&lt;p&gt;Consider a translation task, for example, where give an English sentence &lt;span class=&quot;math-container&quot;&gt;$x_{english} = [x_0, x_1, x_2, \dots, x_m]$&lt;/span&gt;, the transformer decodes the sentence into a French sentence &lt;span class=&quot;math-container&quot;&gt;$x_{french}' = [x_0', x_1', \dots, x_n']$&lt;/span&gt;. Let's say the true label is &lt;span class=&quot;math-container&quot;&gt;$y_{french} = [y_0, y_1, \dots, y_p]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What is the object function of the transformer? Is it the MSE between &lt;span class=&quot;math-container&quot;&gt;$x_{french}'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_{french}$&lt;/span&gt;? And does it have any weight regularization terms?&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-07T23:27:49.507" LastActivityDate="2020-12-10T07:16:59.040" Title="What is the cost function of a transformer?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;loss-functions&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25054" PostTypeId="2" ParentId="25041" CreationDate="2020-12-08T00:10:26.617" Score="1" Body="&lt;p&gt;The masking should be applied to all Decoder blocks, otherwise in some blocks, past words can attend to future words, which would be cheating during training.&lt;/p&gt;&#xA;&lt;p&gt;This is reflected in &lt;a href=&quot;https://nlp.seas.harvard.edu/2018/04/03/attention.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Annotated Transformer&lt;/a&gt; as well. Notice that in the Decoder class, the forward function applies the same mask to each layer of the decoder:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class Decoder(nn.Module):&#xA;    &amp;quot;Generic N layer decoder with masking.&amp;quot;&#xA;    def __init__(self, layer, N):&#xA;        super(Decoder, self).__init__()&#xA;        self.layers = clones(layer, N)&#xA;        self.norm = LayerNorm(layer.size)&#xA;        &#xA;    def forward(self, x, memory, src_mask, tgt_mask):&#xA;        for layer in self.layers:&#xA;            x = layer(x, memory, src_mask, tgt_mask)&#xA;        return self.norm(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-08T00:10:26.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25055" PostTypeId="1" CreationDate="2020-12-08T00:49:59.640" Score="1" ViewCount="124" Body="&lt;p&gt;The paper &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention Is All You Need&lt;/a&gt; describes the Transformer architecture, which describes attention as a function of the queries &lt;span class=&quot;math-container&quot;&gt;$Q = x W^Q$&lt;/span&gt;, keys &lt;span class=&quot;math-container&quot;&gt;$K = x W^K$&lt;/span&gt;, and values &lt;span class=&quot;math-container&quot;&gt;$V = x W^V$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{Attention(Q, K, V)} = \text{softmax}\left( \frac{QK^T}{\sqrt{d_k}} \right) V \\&#xA;= \text{softmax}\left( \frac{x W^Q (W^K)^T x}{\sqrt{d_k}} \right) x W^V$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the Transformer, there are 3 different flavors of attention:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Self-attention in the Encoder&lt;/strong&gt;, where the queries, keys, and values all come from the input to the Encoder.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Encoder-Decoder attention in the Decoder&lt;/strong&gt;, where the queries come from the input to the Decoder, and the keys and values come from the output of the Encoder&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Masked self-attention in the Decoder&lt;/strong&gt;, where the queries, keys and values all come from the input to the Decoder, and, for each token, the &lt;span class=&quot;math-container&quot;&gt;$\text{softmax}\left( \frac{QK^T}{\sqrt{d_k}} \right)$&lt;/span&gt; operation is masked out (zero'd out) for all tokens to the right of that token (to prevent look-ahead, which is cheating during training).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What is the gradient (i.e. the partial derivatives of the loss function w.r.t. &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W^Q$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W^K$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W^V$&lt;/span&gt;, and any bias term(s)) of each of these attention units? I am having a difficult time wrapping my head around derivating a gradient equation because I'm not sure how the softmax function interacts with the partial derivatives, and also, for the Encoder-Decoder attention in the Decoder, I'm not clear how to incorporate the encoder output into the equation.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-08T01:23:24.213" LastActivityDate="2020-12-08T01:23:24.213" Title="What is the gradient of an attention unit?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;gradient-descent&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25056" PostTypeId="2" ParentId="4286" CreationDate="2020-12-08T01:10:44.010" Score="3" Body="&lt;p&gt;A neural network can be reduced to a linear regression model only if we use linear activation functions (i.e. &lt;span class=&quot;math-container&quot;&gt;$\sigma(x) = x$&lt;/span&gt;), and only if we do not use any  neural network specific techniques such as convolution, residuals, etc., as shown below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{neural network}(x) = \sigma_n(W_{n} \sigma_{n-1}(W_{n-1}\dots\sigma_1(W_1 x + b_1) + \dots + b_{n-1}) + b_n) \\&#xA;= W_n (W_{n-1} \dots (W_1 x + b_1) + \dots + b_{n-1}) + b_n \\&#xA;= \left( W_n W_{n-1} \dots W_1 \right) x + \left( W_n W_{n-1} \dots W_2 \right) b_1 + \left( W_n W_{n-1} \dots W_3 \right) b_2 + \dots + W_n b_{n-1} + b_n \\&#xA;= W_z x + b_z$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$W_z = \displaystyle \prod_i W_i$&lt;/span&gt; is a weight matrix and &lt;span class=&quot;math-container&quot;&gt;$b_z$&lt;/span&gt; is some vector constant.&lt;/p&gt;&#xA;&lt;p&gt;This follows the linear regression model form &lt;span class=&quot;math-container&quot;&gt;$y = Wx + b$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is the weight matrix and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the vector constant. As a result, you can analytically solve for &lt;span class=&quot;math-container&quot;&gt;$W_z$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_z$&lt;/span&gt; using linear regression techniques, and no longer need gradient descent.&lt;/p&gt;&#xA;&lt;p&gt;Note that for this to work well as a linear regression, you need to check for the OLS data assumptions, such as making sure the regressors have no collinearity, the residuals have no heteroskedasticity, there's no auto-regression, and that the errors are roughly normally distributed (&lt;a href=&quot;https://www.statisticssolutions.com/assumptions-of-linear-regression/&quot; rel=&quot;nofollow noreferrer&quot;&gt;more info&lt;/a&gt;). Deep neural networks with non-linear activation functions do not require these assumptions since they are universal approximators, although checking for some conditions may help make the task easier to predict (&lt;a href=&quot;https://stats.stackexchange.com/a/472634/303597&quot;&gt;more info*&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;*&lt;em&gt;Note - this link talks specifically about time series data with neural network, but the same concept applies to any task in general.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The neural network specific techniques, such as ReLu, convolutions, residuals, etc., is what allows the network to learn non-linear relationships, and therefore make neural networks something more than just repeated applications of linear regression.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-08T20:19:11.627" LastActivityDate="2020-12-08T20:19:11.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25057" PostTypeId="2" ParentId="21237" CreationDate="2020-12-08T02:07:02.287" Score="3" Body="&lt;p&gt;In statistics, if &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; are independent and randomly distributed variables:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[X + Y] = \mathbb{E}[X] + \mathbb{E}[Y] \\&#xA;Var(X + Y) = Var(X) + Var(Y) \\&#xA;\mathbb{E}[XY] = \mathbb{E}[X]\mathbb{E}[Y] \\&#xA;Var(XY) = (Var(X) + \mathbb{E}[X]^2)(Var(Y) + \mathbb{E}[Y]^2) - \mathbb{E}[X]^2\mathbb{E}[Y]^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; be random &lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt; matrices, where each entry is a some random distribution with &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; mean and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; variance. Every entry is independent from each other.&lt;/p&gt;&#xA;&lt;p&gt;Since each entry of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; have identical distribution, we can focus only on the top-left-most element of &lt;span class=&quot;math-container&quot;&gt;$QK$&lt;/span&gt; without loss of generality. The same applies to every other element.&lt;/p&gt;&#xA;&lt;p&gt;The top-left-most element of &lt;span class=&quot;math-container&quot;&gt;$QK$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\displaystyle \sum_{i=0}^{d_k} Q_{1,i} K_{i, 1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; are independent:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[Q_{1, i} K_{i, 1}] = \mathbb{E}[Q_{1, i}] \mathbb{E}[K_{i, 1}] = 0 \\&#xA;Var(Q_{1, i} K_{i, 1}) = (Var(Q_{1, i}) + \mathbb{E}[Q_{1, i}]^2)(Var(K_{i, 1}) + \mathbb{E}[K_{i, 1}]^2) - \mathbb{E}[Q_{1, i}]^2\mathbb{E}[K_{i, 1}]^2 = 1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And so summing up &lt;span class=&quot;math-container&quot;&gt;$d_k$&lt;/span&gt; of them:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E} \left[\displaystyle \sum_{i=0}^{d_k} Q_{1,i} K_{i, 1} \right] = \displaystyle \sum_{i=0}^{d_k} \mathbb{E} \left[ Q_{1,i} K_{i, 1}  \right] = 0 \\&#xA;Var\left(\displaystyle \sum_{i=0}^{d_k} Q_{1,i} K_{i, 1} \right) = \displaystyle \sum_{i=0}^{d_k} Var\left( Q_{1,i} K_{i, 1}  \right) = d_k$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For your code block, you are computing the dot product of matrices &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, when you should be doing a matrix multiplication (the attention function multiplies &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; after all, which is the vectorized form of dot-product -- it doesn't actually do dot product). It should work out to unit variance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;edit: the last paragraph is incorrect, as dot is the same as matrix multiply in the above case&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-08T20:51:10.567" LastActivityDate="2020-12-08T20:51:10.567" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25058" PostTypeId="1" CreationDate="2020-12-08T03:25:11.300" Score="0" ViewCount="23" Body="&lt;p&gt;I have 6600 images and I am supposed to know the rotation of the object in each image. So, given an image, I want to regress to a single value.&lt;/p&gt;&#xA;&lt;p&gt;My attempt: I use Resnet-18 to extract a feature vector of length 1000 from an image. This is then passed to three fully-connected layers: fc(1000, 512) -&amp;gt; fc(512, 64) -&amp;gt; fc(64, 1)&lt;/p&gt;&#xA;&lt;p&gt;The problem I am facing right now is that my training loss and validation loss immediately go down after the first 5 epochs and then they barely change. But my training and validation accuracy fluctuates wildly throughout.&lt;/p&gt;&#xA;&lt;p&gt;I understand that I am experiencing over-fitting and I have done the following to deal with it:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;data augmentation (Gaussian noise and color jittering)&lt;/li&gt;&#xA;&lt;li&gt;L1 regularization&lt;/li&gt;&#xA;&lt;li&gt;dropout&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So far, nothing seems to be changing the results much. The next thing I haven't tried is reducing the size of my neural net. Will that help? If so, how should I reduce the size?&lt;/p&gt;&#xA;" OwnerUserId="42805" LastActivityDate="2020-12-08T03:25:11.300" Title="How should I use deep learning to find the rotation of an object from its 2D image?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;regression&gt;&lt;feature-extraction&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25059" PostTypeId="1" CreationDate="2020-12-08T03:31:42.723" Score="0" ViewCount="62" Body="&lt;p&gt;I am working on LSTM and CNN to solve the time series prediction problem.&lt;/p&gt;&#xA;&lt;p&gt;I have seen some tutorial examples of time series prediction using CNN-LSTM. But I don't know if it is better than what I predicted using LSTM.&lt;/p&gt;&#xA;&lt;p&gt;Could using LSTM and CNN together be better than predicting using LSTM alone?&lt;/p&gt;&#xA;" OwnerUserId="41045" LastEditorUserId="2444" LastEditDate="2020-12-08T11:24:22.693" LastActivityDate="2020-12-08T11:24:22.693" Title="Time series prediction using LSTM and CNN-LSTM: which is better?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25060" PostTypeId="2" ParentId="24984" CreationDate="2020-12-08T04:34:03.007" Score="5" Body="&lt;p&gt;One can expect the optimal high-level features required to choose the next action and to evaluate a state to be quite similar. Because of that, it is a reasonable idea to share the same network for both policy and value function – you are essentially parameter sharing the feature-extraction part of your neural network, and fine tuning the different heads of your network on the two different tasks: action choice and value prediction.&lt;/p&gt;&#xA;&lt;p&gt;Using two vs one networks is mostly a question of sample efficiency: theoretically, in both case your AC algorithm should work. In practice however, it will usually be useful to have parameter sharing as the representations encouraged by one of the tasks might be highly useful for the other and vice-versa, enabling one task to cause the other task to get unstuck from local optima. Another reason why this might work better is simply because you do not have to learn the same (or at least similar) representations from scratch twice – leading to a more sample efficient training.&lt;/p&gt;&#xA;" OwnerUserId="42207" LastActivityDate="2020-12-08T04:34:03.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25061" PostTypeId="1" CreationDate="2020-12-08T08:01:51.377" Score="1" ViewCount="77" Body="&lt;p&gt;In autonomous driving, we know that the behaviour prediction module is concerned with understanding how the agents in the environment will behave.&#xA;Similarly, in the perception module, the tracking algorithms are responsible for getting an estimate of the object's state over time.&lt;/p&gt;&#xA;" OwnerUserId="42806" LastEditorUserId="2444" LastEditDate="2020-12-08T17:19:59.903" LastActivityDate="2021-05-07T19:03:05.273" Title="What is the difference between object tracking and trajectory prediction?" Tags="&lt;comparison&gt;&lt;autonomous-vehicles&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25062" PostTypeId="1" CreationDate="2020-12-08T09:44:03.710" Score="4" ViewCount="41" Body="&lt;p&gt;I'm trying to create a simple blogpost on RNNs, that should give a better insight into how they work in Keras. Let's say:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = keras.models.Sequential()&#xA;model.add(keras.layers.SimpleRNN(5, return_sequences=True, input_shape=[None, 1]))&#xA;model.add(keras.layers.SimpleRNN(5, return_sequences=True))&#xA;model.add(keras.layers.Dense(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I came up with the following visualization (this is only a sketch), which I'm quite unsure about:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3sIX7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3sIX7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The RNN architecture is comprised of 3 layers represented in the picture.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; is this correct? Is the input &amp;quot;flowing&amp;quot; thought each layer neuron to neuron or only though the layers, like in the picture below. Is there anything else that is not correct - any other visualizations to look into?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QaM1A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QaM1A.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; my assumptions are based on my understanding from what I saw in Geron's book. The recurrent neurons are connected, see: &lt;a href=&quot;https://pasteboard.co/JDXTFVw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pasteboard.co/JDXTFVw.png&lt;/a&gt; ... he then proceeds to talk about connections between different layers, see: &lt;a href=&quot;https://pasteboard.co/JDXTXcz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pasteboard.co/JDXTXcz.png&lt;/a&gt; - did I misunderstand him or is it just a peculiarity in keras framework?&lt;/p&gt;&#xA;" OwnerUserId="42808" LastEditorUserId="42808" LastEditDate="2020-12-08T11:39:36.087" LastActivityDate="2021-05-07T13:03:40.273" Title="How to graphically represent a RNN architecture implemented in Keras?" Tags="&lt;keras&gt;&lt;recurrent-neural-networks&gt;&lt;data-visualization&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25063" PostTypeId="2" ParentId="25062" CreationDate="2020-12-08T10:41:08.433" Score="0" Body="&lt;p&gt;The first image is correct. The information will flow from left to right in each layer and from top to bottom in between layers.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastEditorUserId="20430" LastEditDate="2020-12-08T11:45:04.387" LastActivityDate="2020-12-08T11:45:04.387" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25064" PostTypeId="1" AcceptedAnswerId="25080" CreationDate="2020-12-08T10:46:55.690" Score="2" ViewCount="66" Body="&lt;p&gt;I am a medical doctor working on methodological aspects of health-oriented ML. Reproducibility, replicability, generalisability are critical in this area. Among many questions, some are raised by adversarial attacks (AA).&lt;/p&gt;&#xA;&lt;p&gt;My question is to be considered from a literature review point of view: suppose I want to check an algorithm from an AA point of view:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is there a systematic methodology approach to be used, relating format of the data, type of models, and AA? Conceptually, is there a taxonomy of AA? If so, practically, are some AA considered as gold standards?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42809" LastEditorUserId="2444" LastEditDate="2020-12-09T10:21:56.397" LastActivityDate="2020-12-09T10:47:27.253" Title="Is there a taxonomy of adversarial attacks?" Tags="&lt;machine-learning&gt;&lt;reference-request&gt;&lt;adversarial-ml&gt;&lt;healthcare&gt;&lt;taxonomy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25065" PostTypeId="1" CreationDate="2020-12-08T11:08:19.397" Score="2" ViewCount="22" Body="&lt;h2&gt;Context&lt;/h2&gt;&#xA;&lt;p&gt;I was making a Transformer Model to convert English Sentences to German Sentences. But the loss stops reducing after some time.&lt;/p&gt;&#xA;&lt;h2&gt;Code&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;import string&#xA;import re&#xA;from tensorflow.keras.preprocessing.text import Tokenizer&#xA;from tensorflow.keras.preprocessing.sequence import pad_sequences&#xA;from tensorflow.keras.models import Model&#xA;from tensorflow.keras.layers import Embedding, LSTM, RepeatVector, Dense, Dropout, BatchNormalization, TimeDistributed, AdditiveAttention, Input, Concatenate, Flatten&#xA;from tensorflow.keras.layers import Activation, LayerNormalization, GRU, GlobalAveragePooling1D, Attention&#xA;from tensorflow.keras.optimizers import Adam&#xA;from tensorflow.nn import tanh, softmax&#xA;import time&#xA;from tensorflow.keras.losses import SparseCategoricalCrossentropy, CategoricalCrossentropy&#xA;from numpy import array&#xA;from tensorflow.keras.utils import plot_model&#xA;from sklearn.utils import shuffle&#xA;import time&#xA;import tensorflow as tf&#xA;from numpy import array&#xA;import numpy as np&#xA;from tensorflow.keras.models import load_model&#xA;from tensorflow.keras.datasets.imdb import load_data&#xA;&#xA;def load_data(filename):&#xA;    file = open(filename, 'r')&#xA;    text = file.read()&#xA;    file.close()&#xA;    return text&#xA;&#xA;def to_lines(text):&#xA;    return text.split('\n')&#xA;&#xA;def clean_data(pair):&#xA;    pair = 'start_seq_ ' + pair + ' end_seq_'&#xA;&#xA;    re_print = re.compile('[^%s]' % re.escape(string.printable))&#xA;    table = str.maketrans('', '', string.punctuation)&#xA;    tokens = [token.translate(table) for token in pair.split()]&#xA;    tokens = [token.lower() for token in tokens]&#xA;    tokens = [re_print.sub('', token) for token in tokens]&#xA;    tokens = [token for token in tokens if token.isalpha()]&#xA;    return tokens&#xA;&#xA;lines = to_lines(load_data('/content/drive/My Drive/spa.txt'))&#xA;&#xA;english_pair = []&#xA;german_pair = []&#xA;language = []&#xA;for line in lines:&#xA;    if line != '':&#xA;        pairs = line.split('\t')&#xA;        english_pair.append(clean_data(pairs[0]))&#xA;        german_pair.append(clean_data(pairs[1]))&#xA;&#xA;        language.append(clean_data(pairs[0]))&#xA;        language.append(clean_data(pairs[1]))&#xA;&#xA;english_pair = array(english_pair)&#xA;german_pair = array(german_pair)&#xA;language = array(language)&#xA;&#xA;def create_tokenizer(data):&#xA;    tokenizer = Tokenizer()&#xA;    tokenizer.fit_on_texts(data)&#xA;    return tokenizer&#xA;&#xA;def max_len(lines):&#xA;    length = []&#xA;    for line in lines:&#xA;        length.append(len(line))&#xA;    return max(length)&#xA;&#xA;tokenizer = create_tokenizer(language)&#xA;&#xA;vocab_size = len(tokenizer.word_index) + 1&#xA;&#xA;max_len = max_len(language)&#xA;&#xA;def create_sequences(sequences, max_len):&#xA;    sequences = tokenizer.texts_to_sequences(sequences)&#xA;    sequences = pad_sequences(sequences, maxlen=max_len, padding='post')&#xA;    return sequences&#xA;&#xA;X1 = create_sequences(english_pair, max_len)&#xA;X2 = create_sequences(german_pair, max_len)&#xA;Y = create_sequences(german_pair, max_len)&#xA;&#xA;&#xA;X1, X2, Y = shuffle(X1, X2, Y)&#xA;&#xA;training_samples = int(X1.shape[0] * 1.0)&#xA;&#xA;train_x1, train_x2, train_y = X1[:training_samples], X2[:training_samples], Y[:training_samples]&#xA;test_x1, test_x2, test_y = X1[training_samples:], X2[training_samples:], Y[training_samples:]&#xA;&#xA;train_x2 = train_x2[:, :-1]&#xA;test_x2 = test_x2[:, :-1]&#xA;train_y = train_y[:, 1:].reshape(-1, max_len-1)&#xA;test_y = test_y[:, 1:].reshape(-1, max_len-1)&#xA;&#xA;train_x2 = pad_sequences(train_x2, maxlen=max_len, padding='post')&#xA;test_x2 = pad_sequences(test_x2, maxlen=max_len, padding='post')&#xA;&#xA;train_y = pad_sequences(train_y, maxlen=max_len, padding='post')&#xA;test_y = pad_sequences(test_y, maxlen=max_len, padding='post')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All code above just prepares the Data, so if you want you can skip that part.&#xA;Code After this starts implementing the Transformer Model.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class EncoderBlock(tf.keras.layers.Layer):&#xA;    def __init__(self, mid_ffn_dim, embed_dim, num_heads, max_len, batch_size):&#xA;        super(EncoderBlock, self).__init__()&#xA;        # Variables&#xA;        self.batch_size = batch_size&#xA;        self.max_len = max_len&#xA;        self.mid_ffn_dim = mid_ffn_dim&#xA;        self.embed_dim = embed_dim&#xA;        self.num_heads = num_heads&#xA;        self.attention_vector_len = self.embed_dim // self.num_heads&#xA;        if self.embed_dim % self.num_heads != 0:&#xA;            raise ValueError('I am Batman!')&#xA;&#xA;        # Trainable Layers&#xA;        self.mid_ffn = Dense(self.mid_ffn_dim, activation='relu')&#xA;        self.final_ffn = Dense(self.embed_dim)&#xA;&#xA;        self.layer_norm1 = LayerNormalization(epsilon=1e-6)&#xA;        self.layer_norm2 = LayerNormalization(epsilon=1e-6)&#xA;&#xA;        self.combine_heads = Dense(self.embed_dim)&#xA;&#xA;        self.query_dense = Dense(self.embed_dim)&#xA;        self.key_dense = Dense(self.embed_dim)&#xA;        self.value_dense = Dense(self.embed_dim)&#xA;&#xA;    def separate_heads(self, x):&#xA;        x = tf.reshape(x, (-1, self.max_len, self.num_heads, self.attention_vector_len))&#xA;        return tf.transpose(x, perm=[0, 2, 1, 3])&#xA;&#xA;    def compute_self_attention(self, query, key, value):&#xA;        score = tf.matmul(query, key, transpose_b=True)&#xA;        dim_key = tf.cast(tf.shape(key)[-1], tf.float32)&#xA;        scaled_score = score / tf.math.sqrt(dim_key)&#xA;        weights = tf.nn.softmax(scaled_score, axis=-1)&#xA;        output = tf.matmul(weights, value)&#xA;        return output&#xA;&#xA;    def self_attention_layer(self, x):&#xA;        query = self.query_dense(x)&#xA;        key = self.key_dense(x)&#xA;        value = self.value_dense(x)&#xA;&#xA;        query_heads = self.separate_heads(query)    &#xA;        key_heads = self.separate_heads(key)&#xA;        value_heads = self.separate_heads(value)&#xA;&#xA;        attention = self.compute_self_attention(query_heads, key_heads, value_heads)&#xA;&#xA;        attention = tf.transpose(attention, perm=[0, 2, 1, 3]) &#xA;        attention = tf.reshape(attention, (-1, self.max_len, self.embed_dim))&#xA;&#xA;        output = self.combine_heads(attention)&#xA;        return output&#xA;        &#xA;    def get_output(self, x):&#xA;        attn_output = self.self_attention_layer(x)&#xA;        out1 = self.layer_norm1(x + attn_output)&#xA;&#xA;        ffn_output = self.final_ffn(self.mid_ffn(out1))&#xA;&#xA;        encoder_output = self.layer_norm2(out1 + ffn_output)&#xA;        return encoder_output&#xA;&#xA;class DecoderBlock(tf.keras.layers.Layer):&#xA;    def __init__(self, mid_ffn_dim, embed_dim, num_heads, max_len, batch_size):&#xA;        super(DecoderBlock, self).__init__()&#xA;        # Variables&#xA;        self.batch_size = batch_size&#xA;        self.max_len = max_len&#xA;        self.mid_ffn_dim = mid_ffn_dim&#xA;        self.embed_dim = embed_dim&#xA;        self.num_heads = num_heads&#xA;        self.attention_vector_len = self.embed_dim // self.num_heads&#xA;        if self.embed_dim % self.num_heads != 0:&#xA;            raise ValueError('I am Batman!')&#xA;&#xA;        # Trainable Layers&#xA;&#xA;        self.query_dense1 = Dense(self.embed_dim, name='query_dense1')&#xA;        self.key_dense1 = Dense(self.embed_dim, name='key_dense1')&#xA;        self.value_dense1 = Dense(self.embed_dim, name='value_dense1')&#xA;&#xA;        self.mid_ffn = Dense(self.mid_ffn_dim, activation='relu', name='dec_mid_ffn')&#xA;        self.final_ffn = Dense(self.embed_dim, name='dec_final_ffn')&#xA;&#xA;        self.layer_norm1 = LayerNormalization(epsilon=1e-6)&#xA;        self.layer_norm2 = LayerNormalization(epsilon=1e-6)&#xA;        self.layer_norm3 = LayerNormalization(epsilon=1e-6)&#xA;&#xA;        self.combine_heads = Dense(self.embed_dim, name='dec_combine_heads')&#xA;&#xA;        self.query_dense2 = Dense(self.embed_dim, name='query_dense2')&#xA;        self.key_dense2 = Dense(self.embed_dim, name='key_dense2')&#xA;        self.value_dense2 = Dense(self.embed_dim, name='value_dense2')&#xA;&#xA;    def separate_heads(self, x):&#xA;        x = tf.reshape(x, (-1, self.max_len, self.num_heads, self.attention_vector_len))&#xA;        return tf.transpose(x, perm=[0, 2, 1, 3])&#xA;&#xA;    def compute_self_attention(self, query, key, value):&#xA;        score = tf.matmul(query, key, transpose_b=True)&#xA;        dim_key = tf.cast(tf.shape(key)[-1], tf.float32)&#xA;        scaled_score = score / tf.math.sqrt(dim_key)&#xA;        weights = tf.nn.softmax(scaled_score, axis=-1)&#xA;        output = tf.matmul(weights, value)&#xA;        return output&#xA;&#xA;    def masking(self, x):&#xA;        b = []&#xA;        for batch in range(x.shape[0]):&#xA;            bat = []&#xA;            for head in range(x.shape[1]):&#xA;                headd = []&#xA;                for word in range(x.shape[2]):&#xA;                    current_word = []&#xA;                    for represented_in in range(x.shape[3]):&#xA;                        if represented_in &amp;gt; word:&#xA;                          current_word.append(np.NINF)&#xA;                        else:&#xA;                          current_word.append(0)&#xA;                    headd.append(current_word)&#xA;                bat.append(headd)&#xA;            b.append(bat)&#xA;        return b&#xA;&#xA;    def compute_masked_self_attention(self, query, key, value):&#xA;        score = tf.matmul(query, key, transpose_b=True)&#xA;        score = score + self.masking(score)&#xA;        score = tf.convert_to_tensor(score)&#xA;                &#xA;        dim_key = tf.cast(tf.shape(key)[-1], tf.float32)&#xA;        scaled_score = score / tf.math.sqrt(dim_key)&#xA;        weights = tf.nn.softmax(scaled_score, axis=-1)&#xA;        output = tf.matmul(weights, value)&#xA;        return output&#xA;&#xA;    def masked_self_attention_layer(self, x):&#xA;        query = self.query_dense1(x)&#xA;        key = self.key_dense1(x)&#xA;        value = self.value_dense1(x)&#xA;&#xA;        query_heads = self.separate_heads(query)    &#xA;        key_heads = self.separate_heads(key)&#xA;        value_heads = self.separate_heads(value)&#xA;&#xA;        attention = self.compute_masked_self_attention(query_heads, key_heads, value_heads)&#xA;&#xA;        attention = tf.transpose(attention, perm=[0, 2, 1, 3]) &#xA;        attention = tf.reshape(attention, (-1, self.max_len, self.embed_dim))&#xA;&#xA;        output = self.combine_heads(attention)&#xA;        return output&#xA;&#xA;    def second_attention_layer(self, x, encoder_output):&#xA;        query = self.query_dense2(x)&#xA;        key = self.key_dense2(encoder_output)&#xA;        value = self.value_dense2(encoder_output)&#xA;&#xA;        query_heads = self.separate_heads(query)    &#xA;        key_heads = self.separate_heads(key)&#xA;        value_heads = self.separate_heads(value)&#xA;&#xA;        attention = self.compute_self_attention(query_heads, key_heads, value_heads)&#xA;&#xA;        attention = tf.transpose(attention, perm=[0, 2, 1, 3]) &#xA;        attention = tf.reshape(attention, (-1, self.max_len, self.embed_dim))&#xA;&#xA;        output = self.combine_heads(attention)&#xA;        return output&#xA;      &#xA;    def get_output(self, x, encoder_output):&#xA;        masked_attn_output = self.masked_self_attention_layer(x)&#xA;        out1 = self.layer_norm1(x + masked_attn_output)&#xA;&#xA;        mutli_head_attn_output = self.second_attention_layer(out1, encoder_output)&#xA;        out2 = self.layer_norm2(out1 + mutli_head_attn_output)&#xA;&#xA;        ffn_output = self.final_ffn(self.mid_ffn(out2))&#xA;        decoder_output = self.layer_norm3(out2 + ffn_output)&#xA;        return decoder_output&#xA;&#xA;embed_dim = 512&#xA;mid_ffn_dim = 1024&#xA;&#xA;num_heads = 8&#xA;max_len = max_len&#xA;batch_size = 32&#xA;&#xA;encoder_block1 = EncoderBlock(mid_ffn_dim, embed_dim, num_heads, max_len, batch_size)&#xA;encoder_block2 = EncoderBlock(mid_ffn_dim, embed_dim, num_heads, max_len, batch_size)&#xA;encoder_block3 = EncoderBlock(mid_ffn_dim, embed_dim, num_heads, max_len, batch_size)&#xA;&#xA;decoder_block1 = DecoderBlock(mid_ffn_dim, embed_dim, num_heads, max_len, batch_size)&#xA;decoder_block2 = DecoderBlock(mid_ffn_dim, embed_dim, num_heads, max_len, batch_size)&#xA;decoder_block3 = DecoderBlock(mid_ffn_dim, embed_dim, num_heads, max_len, batch_size)&#xA;&#xA;# Define Loss and Optimizer&#xA;loss_object = SparseCategoricalCrossentropy()&#xA;optimizer = Adam()&#xA;&#xA;embedding = Embedding(vocab_size, embed_dim, name='embedding')&#xA;position_embedding = Embedding(vocab_size, embed_dim)&#xA;&#xA;final_transformer_layer = Dense(vocab_size, activation='softmax')&#xA;&#xA;def positional_embedding(x):&#xA;    positions = tf.range(start=0, limit=max_len, delta=1)&#xA;    positions = position_embedding(positions)&#xA;    return x + positions&#xA;&#xA;def train_step(english_sent, german_sent, german_trgt):&#xA;    with tf.GradientTape() as tape:&#xA;        english_embedded = embedding(english_sent)&#xA;        german_embedded = embedding(german_sent)&#xA;&#xA;        english_positioned = positional_embedding(english_embedded)&#xA;        german_positioned = positional_embedding(german_embedded)&#xA;&#xA;        # Encoders&#xA;        encoder_output = encoder_block1.get_output(english_positioned)&#xA;        encoder_output = encoder_block2.get_output(encoder_output)&#xA;        encoder_output = encoder_block3.get_output(encoder_output)&#xA;&#xA;        # Decoders&#xA;        decoder_output = decoder_block1.get_output(german_positioned, encoder_output)&#xA;        decoder_output = decoder_block2.get_output(decoder_output, encoder_output)&#xA;        decoder_output = decoder_block3.get_output(decoder_output, encoder_output)&#xA;&#xA;        # Final Output&#xA;        transformer_output = final_transformer_layer(decoder_output)&#xA;&#xA;        # Compute Loss&#xA;        loss = loss_object(german_trgt, transformer_output)&#xA;&#xA;    variables = embedding.trainable_variables + position_embedding.trainable_variables + encoder_block1.trainable_variables + encoder_block2.trainable_variables&#xA;    variables += encoder_block3.trainable_variables + decoder_block1.trainable_variables + decoder_block2.trainable_variables + decoder_block3.trainable_variables&#xA;    variables += final_transformer_layer.trainable_variables&#xA;&#xA;    gradients = tape.gradient(loss, variables)&#xA;    optimizer.apply_gradients(zip(gradients, variables))&#xA;&#xA;    return float(loss)&#xA;&#xA;def train(epochs=10):&#xA;    batch_per_epoch = int(train_x1.shape[0] / batch_size)&#xA;    for epoch in range(epochs):&#xA;        for i in range(batch_per_epoch):&#xA;            english_sent_x = train_x1[i*batch_size : (i*batch_size)+batch_size].reshape(batch_size, max_len)&#xA;            german_sent_x = train_x2[i*batch_size : (i*batch_size)+batch_size].reshape(batch_size, max_len)&#xA;            german_sent_y = train_y[i*batch_size : (i*batch_size)+batch_size].reshape(batch_size, max_len, 1)&#xA;&#xA;            loss = train_step(english_sent_x, german_sent_x, german_sent_y)&#xA;&#xA;            print('Epoch ', epoch, 'Batch ', i, '/', batch_per_epoch, 'Loss ', loss)&#xA;&#xA;train()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the Code is done! But the loss stops reducing at around value of 1.2 after some time. Why is this happening?&lt;/p&gt;&#xA;&lt;h2&gt;Maybe Important&lt;/h2&gt;&#xA;&lt;p&gt;I tried debugging the model, by passing random input integers, and the model was still performing the same way it did when I gave real Sentences as input.&lt;/p&gt;&#xA;&lt;p&gt;When I tried training the model with just 1 training sample, the loss stops reducing at around 0.2. When I train it with 2 training samples, the result was the approximately the same as when I trained it with 1 training sample.&lt;/p&gt;&#xA;&lt;p&gt;When I stopped shuffling the dataset the loss gone till around 0.7 and again stopped learning.&lt;/p&gt;&#xA;&lt;p&gt;I tried simplifying the model by removing some encoder and decoder blocks but the results were approximately the same. I even tried making the model more complex but the results were again approximately the same.&lt;/p&gt;&#xA;" OwnerUserId="42813" LastActivityDate="2020-12-08T11:08:19.397" Title="Why does the loss stops reducing after a point in this Transformer Model?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;keras&gt;&lt;transformer&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25067" PostTypeId="5" CreationDate="2020-12-08T15:43:08.290" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-08T15:43:08.290" LastActivityDate="2020-12-08T15:43:08.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25068" PostTypeId="4" CreationDate="2020-12-08T15:43:08.290" Score="0" Body="For questions that involve constrained optimization problems (in the context of artificial intelligence)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-08T15:43:08.290" LastActivityDate="2020-12-08T15:43:08.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25069" PostTypeId="5" CreationDate="2020-12-08T16:09:45.493" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-08T16:09:45.493" LastActivityDate="2020-12-08T16:09:45.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25070" PostTypeId="4" CreationDate="2020-12-08T16:09:45.493" Score="0" Body="For questions about trap functions (also known as deceptive functions) in the context of evolutionary algorithms, which are functions that are created to make the EA fail (i.e. find a local optimum, rather than the global optimum)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-08T16:09:45.493" LastActivityDate="2020-12-08T16:09:45.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25072" PostTypeId="2" ParentId="2245" CreationDate="2020-12-08T16:36:03.690" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/2246/2444&quot;&gt;This answer&lt;/a&gt; already gives the idea of what a &lt;em&gt;trap function&lt;/em&gt; (sometimes known as &lt;strong&gt;deceptive function&lt;/strong&gt;) is. However, given that the work on &lt;em&gt;trap functions&lt;/em&gt; is not abundant in the literature (at least this topic is not covered extensively in one of my reference books, i.e. &lt;a href=&quot;http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf#page=244&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;, on page 211, exercises 11.7, only mentions a specific deceptive function, but does not define what a trap function is), let me also provide you with a few references, in case you are looking for more details and formulations.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/BF01531277&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sufficient conditions for deceptive and easy binary functions&lt;/a&gt; (1994, by Deb and Goldberg) provides a definition and formulation of a trap function in section 4.1, p. 397, but also many other specific definitions of deceptive functions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-662-44320-0_13&quot; rel=&quot;nofollow noreferrer&quot;&gt;Global Optimization of Multimodal Deceptive Functions&lt;/a&gt; (EvoCOP 2014, by Iclănzan), which defines &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-trap functions in section 4, p. 150&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The book &lt;a href=&quot;https://digi.lib.ttu.ee/services/copycat-pdf.php?ID=248&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evolutionary Computation 1: Basic Algorithms and Operators&lt;/a&gt; also mentions deceptive functions and deceptive problems several times. Moreover, the Python package &lt;a href=&quot;https://github.com/schlatterbeck/GA_kit&quot; rel=&quot;nofollow noreferrer&quot;&gt;GA_kit&lt;/a&gt; includes &lt;a href=&quot;https://github.com/schlatterbeck/GA_kit/blob/master/deceptive.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;a module to evaluate GAs with deceptive functions&lt;/a&gt;, in case you learn more by looking or playing with the code.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-08T17:52:40.220" LastActivityDate="2020-12-08T17:52:40.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25073" PostTypeId="2" ParentId="25061" CreationDate="2020-12-08T16:44:38.163" Score="1" Body="&lt;p&gt;Although both processes might be doing estimations (because data sources aren't perfect and/or have noise), there is a key difference:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Object tracking cares solely abouth where objects are now. That means that there is actual sensor data that can support the current position. For example: from a camera and a lidar, the computer predicts where a vehicle stands.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Trajectory prediction is done with the main purpose of predicting where objects will be in the future, meaning that there is no sensor data yet. For example: from past data, the computer predicts where a vehicle will be after 1 second from now.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, both processes might need each other. In order to predict a trajectory, it could be easier to work with curated positions given by an object tracker than raw sensor data. On the other hand, past predictions might be given as input to an object tracker, in order to mitigate noise in the sensor data and acting so as a belief-based filter.&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2020-12-08T16:44:38.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25074" PostTypeId="1" AcceptedAnswerId="25087" CreationDate="2020-12-08T21:04:22.383" Score="2" ViewCount="488" Body="&lt;p&gt;I want to create a Deep Learning model that measures the distance between the camera and certain objects in an image. Is it possible? Please, let me know some resources related to this task.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-12-09T22:33:56.040" LastActivityDate="2020-12-10T09:44:41.673" Title="How to calculate the distance between the camera and an object using Computer Vision?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;reference-request&gt;&lt;depth-estimation&gt;&lt;stereo-vision&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25075" PostTypeId="1" CreationDate="2020-12-09T06:18:41.517" Score="2" ViewCount="27" Body="&lt;p&gt;Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Deploy directly a supervised learning algorithm that maps X to y&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Deploy a (variational) auto-encoder for learning useful features, and then using these for training the supervised learning algorithm&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I would like to be pointed to some papers/blogs that explain which technique is better and when or where they conduct empirical benchmarking experiments.&lt;/p&gt;&#xA;" OwnerUserId="42832" LastEditorUserId="2444" LastEditDate="2020-12-09T10:15:38.093" LastActivityDate="2020-12-09T10:15:38.093" Title="Literature on the advantages of using an auto-encoder for classification" Tags="&lt;classification&gt;&lt;reference-request&gt;&lt;autoencoders&gt;&lt;supervised-learning&gt;&lt;transfer-learning&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25077" PostTypeId="1" CreationDate="2020-12-09T10:14:56.017" Score="2" ViewCount="36" Body="&lt;p&gt;I want to code up one time step in a LSTM. My focus is on understanding the functioning of the &lt;strong&gt;forget gate layer&lt;/strong&gt;, &lt;strong&gt;input gate layer&lt;/strong&gt;, &lt;strong&gt;candidate values&lt;/strong&gt;, &lt;strong&gt;present&lt;/strong&gt; and &lt;strong&gt;future&lt;/strong&gt; &lt;strong&gt;cell states&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Lets assume that my hidden state at t-1 and xt are the following. For simplicity, lets assume that the weight matrices are identity matrices, and  all biases are zero.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;htminus1 = np.array( [0, 0.5, 0.1, 0.2, 0.6] )&#xA;xt = np.array( [-0.1, 0.3, 0.1, -0.25, 0.1] )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I understand that forget state is sigmoid of &lt;code&gt;htminus1&lt;/code&gt; and &lt;code&gt;xt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, is it?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ft = 1 / ( 1 + np.exp( -( htminus1 + xt ) ) )&#xA;&#xA;&amp;gt;&amp;gt; ft = array([0.47502081, 0.68997448, 0.549834  , 0.4875026 , 0.66818777])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am referring to &lt;a href=&quot;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; to implement of one iteration of one block LSTM. The link says that &lt;code&gt;ft&lt;/code&gt; should be 0 or 1. Am I missing something here?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How do I get the forget gate layer as per schema given in the below mentioned picture? An example will be illustrative for me.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZN8k0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZN8k0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Along the same lines, &lt;strong&gt;how do I get the input gate layer, &lt;code&gt;it&lt;/code&gt; and vector of new candidate values, &lt;code&gt;\tilde{C}_t&lt;/code&gt; as per the following picture?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NoG1h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NoG1h.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, &lt;strong&gt;how do I get the new hidden state &lt;code&gt;ht&lt;/code&gt; as per the scheme given in the following picture?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A simple, example will be helpful for me in understanding. Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/meCCp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/meCCp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42837" LastEditorUserId="42837" LastEditDate="2020-12-09T10:51:32.593" LastActivityDate="2020-12-10T11:30:02.633" Title="Understanding LSTM through example" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25078" PostTypeId="5" CreationDate="2020-12-09T10:27:06.877" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-09T10:27:06.877" LastActivityDate="2020-12-09T10:27:06.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25079" PostTypeId="4" CreationDate="2020-12-09T10:27:06.877" Score="0" Body="For questions that ask for or involve a taxonomy of AI/ML topics." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-09T10:27:06.877" LastActivityDate="2020-12-09T10:27:06.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25080" PostTypeId="2" ParentId="25064" CreationDate="2020-12-09T10:47:27.253" Score="2" Body="&lt;p&gt;There are already a couple of papers in the literature that attempt to provide a taxonomy and survey of &lt;strong&gt;adversarial attacks&lt;/strong&gt;. I will just list the two that I think are reliable enough that you can probably use as a reference.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1574013718303289&quot; rel=&quot;nofollow noreferrer&quot;&gt;A taxonomy and survey of attacks against machine learning&lt;/a&gt; (2019) by Nikolaos Pitropakis et al., published in &lt;a href=&quot;https://www.sciencedirect.com/journal/computer-science-review/about/aims-and-scope&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Science Review&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1712.07107.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adversarial Examples: Attacks and Defenses for Deep Learning&lt;/a&gt; (2019) by Xiaoyong Yuan et al., published in &lt;a href=&quot;https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=5962385&quot; rel=&quot;nofollow noreferrer&quot;&gt;IEEE Transactions on Neural Networks and Learning Systems&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Needless to say, there are different adversarial attacks, such as the &lt;a href=&quot;https://www.tensorflow.org/tutorials/generative/adversarial_fgsm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast Gradient Signed Method (FGSM)&lt;/a&gt;, and they can be classified into different categories, such as &lt;strong&gt;evasion attacks&lt;/strong&gt; or &lt;strong&gt;poising attacks&lt;/strong&gt;. You can find a lot more info in these cited papers.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-12-09T10:47:27.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25081" PostTypeId="2" ParentId="25074" CreationDate="2020-12-09T11:06:28.923" Score="2" Body="&lt;p&gt;You can use libraries OpenCV and Python to &lt;a href=&quot;https://www.pyimagesearch.com/2015/01/19/find-distance-camera-objectmarker-using-python-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;find the distance.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can refer this : &lt;a href=&quot;https://towardsdatascience.com/vehicle-detection-and-distance-estimation-7acde48256e1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vehicle detection and distance estimation.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since you didn't mention about dataset,you may consider datasets and methods in this &lt;a href=&quot;https://arxiv.org/pdf/1806.10890.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If your camera is fixed/has many objects infront of it,you might use &lt;a href=&quot;https://github.com/saivarshittha/latitude_longitude_bounds&quot; rel=&quot;nofollow noreferrer&quot;&gt;nearest object around camera approach.&lt;/a&gt;.This approach is extremely useful if you want to deal with latitudes and longitudes.&lt;/p&gt;&#xA;" OwnerUserId="41585" LastActivityDate="2020-12-09T11:06:28.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25082" PostTypeId="2" ParentId="25077" CreationDate="2020-12-09T11:52:08.657" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZNnrA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZNnrA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is an image to better understand lstm... At &lt;span class=&quot;math-container&quot;&gt;$f_t$&lt;/span&gt;, we are taking the sigmoid of a weight matrix * the input at the current timestep + another weight matrix * &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Code Sample for &lt;span class=&quot;math-container&quot;&gt;$f_t$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import math&#xA;&#xA;def sigmoid(values):&#xA;    sigmoid_applied = []&#xA;    for value in values:&#xA;        result = 1 / (1 + math.pow(math.e, -value))&#xA;        sigmoid_applied.append(result)&#xA;    return np.array(sigmoid_applied)&#xA;&#xA;w1 = np.random.uniform(0, 1, size=[hidden_vector_len, input_len])&#xA;w2 = np.random.uniform(0, 1, size=[hidden_vector_len, hidden_vector_len])&#xA;&#xA;f_t = sigmoid(np.dot(w1, input) + np.dot(w2, prev_hidden_state)) # Its matrix multiplication and not just simple multiplication&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note - There is also a bias term which I haven't included here for simplicity&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you understood &lt;span class=&quot;math-container&quot;&gt;$f_t$&lt;/span&gt;, you can do the same for other states also.&lt;/p&gt;&#xA;&lt;p&gt;If you feel I am wrong anywhere in this post, then please do consider adding a comment&lt;/p&gt;&#xA;" OwnerUserId="42813" LastEditorUserId="42813" LastEditDate="2020-12-10T11:30:02.633" LastActivityDate="2020-12-10T11:30:02.633" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25084" PostTypeId="1" CreationDate="2020-12-09T14:37:04.667" Score="1" ViewCount="122" Body="&lt;p&gt;During my research for Google DeepMind's Go-playing program &lt;a href=&quot;https://www.nature.com/articles/nature16961&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha Go&lt;/a&gt; and its successor &lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha Go Zero&lt;/a&gt;, I discovered that the system uses a clever pipeline and an interplay of blocks of both policy and value networks to play the game of Go in such a way, that it is able to outperform even the best players in the world. This is in particular remarkable, because the game of Go was considered to be unsolvable a few years ago. This success gained international attention and it was labeled as a breakthrough in the community of AI. It is also not a secret that the research team behind AlphaGo and AlphaGo Zero used lots of computation power to create such a sophisticated system.&lt;/p&gt;&#xA;&lt;p&gt;But, since each board configuration is considered as a distinct state, where algorithms can be applied really well, and just consider AlphaGo Zero, which uses no prior knowledge and can figure out how the play the game of go from scratch, my question is the following:&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to state (theoretically) how the performance of AlphaGo would be in continuous action spaces (e.g. self-driving cars)?&lt;/p&gt;&#xA;" OwnerUserId="26494" LastEditorUserId="2444" LastEditDate="2020-12-10T10:29:25.130" LastActivityDate="2020-12-10T10:29:25.130" Title="What would be the AlphaGo's performance in continuous action space?" Tags="&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;&lt;alphago&gt;&lt;deepmind&gt;&lt;continuous-action-spaces&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25085" PostTypeId="1" CreationDate="2020-12-09T14:54:20.567" Score="0" ViewCount="19" Body="&lt;p&gt;I am trying to set up an experiment where an agent is exploring an n x n gridworld environment, of which the agent can see some fraction at any given time step. I'd like the agent to build up some internal model of this gridworld.&lt;/p&gt;&#xA;&lt;p&gt;Now the environment is time-varying, so I figured it would useful to try using an LSTM so the agent can learn potentially useful information about how the environment changes. However, since the agent can only see some of the environment, each observation that could be used to train this model would be incomplete (i.e. the problem is partially-observable from this perspective). Thus I imagine that training such a network would be difficult since there would be large gaps in the data - for example, it may make an observation at position [0, 0] at t = 0, and then not make another observation there until say t = 100.&lt;/p&gt;&#xA;&lt;p&gt;My question is twofold&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a canonical way of working around partial observability in LSTMs? Either direct advice or pointing to useful papers would both be appreciated.&lt;/li&gt;&#xA;&lt;li&gt;Can an LSTM account for gaps in time between observations?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="42846" LastActivityDate="2020-12-09T14:54:20.567" Title="Using an LSTM for model-based RL in a POMDP" Tags="&lt;reinforcement-learning&gt;&lt;long-short-term-memory&gt;&lt;model-based-methods&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25086" PostTypeId="1" AcceptedAnswerId="25090" CreationDate="2020-12-09T18:28:52.423" Score="6" ViewCount="140" Body="&lt;p&gt;I'm doing a project on Reinforcement Learning. I programmed an agent that uses DDQN. There are a lot of tutorials on that, so the code implementation was not that hard.&lt;/p&gt;&#xA;&lt;p&gt;However, I have problems understanding how one should come up with this kind of algorithms by starting from the Bellman equation, and I don't find a good understandable explanation addressing this derivation/path of reasoning.&lt;/p&gt;&#xA;&lt;p&gt;So, my questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How is the loss to train the DQN derived from (or theoretically motivated by) the Bellman equation?&lt;/li&gt;&#xA;&lt;li&gt;How is it related to the usual Q-learning update?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;According to my current notes, the Bellman equation looks like this&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{\pi} (s,a) = \sum_{s'} P_{ss'}^a (r_{s,a} + \gamma \sum_{a'} \pi(a'|s') Q_{\pi} (s',a')) \label{1}\tag{1} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which, to my understanding, is a recursive expression that says:&#xA;The state-action pair gives a reward that is equal to the sum over all possible states &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; with the probability of getting to this state after taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (denoted as &lt;span class=&quot;math-container&quot;&gt;$P_{ss'}^a$&lt;/span&gt;, which means the environment acts on the agent) times the reward the agent got from taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; + discounted sum of the probability of the different possible actions &lt;span class=&quot;math-container&quot;&gt;$a'$&lt;/span&gt; times the reward of the state, action pair &lt;span class=&quot;math-container&quot;&gt;$s',a'$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The Q-Learning iteration (intermediate step) is often denoted as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q^{new}(s,a) \leftarrow  Q(s,a) + \alpha (r + \gamma \max_a Q(s',a') - Q(s,a)) \label{2}\tag{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which means that the new state, action reward is the old Q value + learning rate, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, times the temporal difference, &lt;span class=&quot;math-container&quot;&gt;$(r + \gamma \max_a Q(s',a') - Q(s,a))$&lt;/span&gt;, which consists of the actual reward the agent received + a discount factor times the Q function of this new state-action pair minus the old Q function.&lt;/p&gt;&#xA;&lt;p&gt;The Bellman equation can be converted into an update rule because an algorithm that uses that update rule converges, as &lt;a href=&quot;https://ai.stackexchange.com/a/20163/2444&quot;&gt;this answer&lt;/a&gt; states.&lt;/p&gt;&#xA;&lt;p&gt;In the case of (D)DQN, &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; is estimated by our NN that leads to an action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and we receive &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then we feed in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; as well as &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; into our NN (with Double DQN we feed them into different NNs). The &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(s',a')$&lt;/span&gt; is performed on the output of our target network. This q-value is then multiplied with &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is added to the product. Then this sum replaces the q-value from the other NN. Since this basic NN outputted &lt;span class=&quot;math-container&quot;&gt;$Q(s,a)$&lt;/span&gt; but should have outputted &lt;span class=&quot;math-container&quot;&gt;$r + \gamma \max_a Q(s',a')$&lt;/span&gt; we train the basic NN to change the weights, so that it would output closer to this temporal target difference.&lt;/p&gt;&#xA;" OwnerUserId="42849" LastEditorUserId="2444" LastEditDate="2020-12-10T17:39:06.333" LastActivityDate="2020-12-10T17:39:06.333" Title="How is the DQN loss derived from (or theoretically motivated by) the Bellman equation, and how is it related to the Q-learning update?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;loss-functions&gt;&lt;bellman-equations&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25087" PostTypeId="2" ParentId="25074" CreationDate="2020-12-09T19:43:51.583" Score="5" Body="&lt;p&gt;&lt;strong&gt;In general, calculation of distance between camera and object is impossible if you don't have further scene dependent information.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To my knowledge you have 3 options:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Stereo Vision&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you have 2 cameras looking at the same scene from a different point of view you can calculate the distance with classical Computer Vision algorithms. This is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_stereo_vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;stereo vision&lt;/a&gt;, or also &lt;a href=&quot;https://en.wikipedia.org/wiki/Epipolar_geometry&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiview geometry&lt;/a&gt;.&#xA;Stereo Vision is the reason why humans can infer the distance to objects around them (because we have 2 eyes).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Structure from Motion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;You move your camera and therefore change your viewpoint and can essentially do stereo mapping over time. &lt;a href=&quot;https://en.wikipedia.org/wiki/Structure_from_motion&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structure from Motion&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scene Understanding&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why is it then still possible for a one-eyed person to infer depth to some extent? Because humans have lots of scene dependent understanding. If you see a rubber duck that takes half of your field of view, you know it's pretty close because you know a rubber duck is not big. If you don't know the size of rubber ducks it is impossible to know whether you see a big rubber duck that is far away or a small rubber duck that is really close.&lt;/p&gt;&#xA;&lt;p&gt;This is where Deep Learning based models come into play. A recent overview over monocular&#xA;depth estimation can be found in &lt;a href=&quot;https://arxiv.org/pdf/2003.06620.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zhao2020&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42852" LastEditorUserId="42852" LastEditDate="2020-12-10T09:44:41.673" LastActivityDate="2020-12-10T09:44:41.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25088" PostTypeId="1" CreationDate="2020-12-09T23:33:02.773" Score="1" ViewCount="17" Body="&lt;p&gt;Previously, I have build a donkey car where the steering of the two front wheels was done using a motor servo. This project was a success and the car was able to drive autonomously after training was done.&lt;/p&gt;&#xA;&lt;p&gt;source &lt;a href=&quot;https://www.donkeycar.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Donkey Car 2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now:&#xA;I have this  &lt;a href=&quot;https://images.app.goo.gl/ooXYvWmwYiNLqpwj6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rc Car kit&lt;/a&gt; that has two motors on the back, powering two wheels and a trolley wheel in front.&#xA;The steering is supposed to be done by playing around with the two back motors.&lt;/p&gt;&#xA;&lt;p&gt;My question is:&#xA;Is there any method modify the donkey car code, so I can train the model?&lt;/p&gt;&#xA;&lt;p&gt;Considering that Donkey Car uses the angle of the servo to train the model, and now I just have the information of the two back wheels and no servo steering the vehicle&lt;/p&gt;&#xA;&lt;p&gt;Not sure if there approach that is specific to this concept.&lt;/p&gt;&#xA;" OwnerUserId="38666" LastActivityDate="2020-12-09T23:33:02.773" Title="Algorithms for training a two motor powered rc car without steering servo" Tags="&lt;machine-learning&gt;&lt;python&gt;&lt;autonomous-vehicles&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25089" PostTypeId="1" CreationDate="2020-12-09T23:37:03.133" Score="1" ViewCount="48" Body="&lt;p&gt;(Disclaimer: I don't know much about ML/AI, besides some basic ideas behind it all.)&lt;/p&gt;&#xA;&lt;p&gt;It seems like ML/AI models can often be boiled down to statistics, where certain levers (weights) get fine-tuned based on the specific input of a large set of training data.&lt;/p&gt;&#xA;&lt;p&gt;Clearly, ML/AI models don't only distinguish themselves in their training data alone, otherwise there would not be so many improvements happening in the field all the time. My question therefore is: What &lt;em&gt;does&lt;/em&gt; distinguish different models of the same category?&lt;/p&gt;&#xA;&lt;p&gt;If I have an AI that completes real-life pictures that have some missing parts, and an AI that completes a painting with missing parts, what key concepts separates the two?&lt;/p&gt;&#xA;&lt;p&gt;If I have an AI detecting text in an image, and an AI detecting... trees in an image, what key concepts separates the two?&lt;/p&gt;&#xA;&lt;p&gt;In other words, what is stopping me from &amp;quot;taking&amp;quot; an existing implementation of a certain AI category, and just feeding it my specific training set + rewards (i.e. judgement criteria for good vs bad output), in order to solve a specific task?&lt;/p&gt;&#xA;&lt;p&gt;In yet again other words, if I wanted to use ML/AI to build a new model for a specific task, what concepts and topics would I need to pay extra attention to? (I guess you could say I'm trying to reverse engineer the learning process of the field here. I don't have the time to properly teach myself and become an &amp;quot;expert&amp;quot;, but find it all very interesting and would still like to &lt;em&gt;use&lt;/em&gt; some of the wonderful things people have done.)&lt;/p&gt;&#xA;" OwnerUserId="42857" LastEditorUserId="2444" LastEditDate="2020-12-10T01:35:57.767" LastActivityDate="2021-05-11T12:06:35.563" Title="Aside from specific training sets, what distinguishes the capabilities of different AI implementations?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;models&gt;&lt;unsupervised-learning&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25090" PostTypeId="2" ParentId="25086" CreationDate="2020-12-10T00:28:09.697" Score="2" Body="&lt;p&gt;The Bellman equation in RL is usually defined &lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \sum_a \pi(a|s) \sum_{s', r} p(s', r|s, a)\left[r + v_\pi(s')\right] = \mathbb{E}_{s' \sim p, a \sim \pi}\left[r(s, a) + v_\pi(s')\right] \; .$$&lt;/span&gt;&#xA;The way you have written it is correct, but I just thought I would point this out. Regardless, your intuition is correct in that it expresses a recursive relationship such that the value of your current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is equal to the sum of the expected reward from this state plus the expected value of the state you transition into.&lt;/p&gt;&#xA;&lt;p&gt;You do, in fact, implement the Q-learning update in Deep Q-Learning. The loss function that you minimise in DQN is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ L(\theta) = \mathbb{E}_{(s,a,r,s')\sim U(D)}\left[\left( r + \gamma \max_{a'}Q(s', a'; \theta^-) - Q(s, a; \theta)\right)^2 \right]\;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$U(D)$&lt;/span&gt; denotes uniformly at random from replay buffer &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are your network parameters (the network parameterises the Q-function), and &lt;span class=&quot;math-container&quot;&gt;$\theta^-$&lt;/span&gt; are a previous iteration of the parameters that are updated every &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; episodes to help with convergence of the network.&lt;/p&gt;&#xA;&lt;p&gt;As you can see, the loss function is minimising the 'Bellman error' error from your equation 2. Lets think about why this is.&lt;/p&gt;&#xA;&lt;p&gt;The TD update you provide is gradually shifting the Q value for &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; towards &lt;span class=&quot;math-container&quot;&gt;$r + \max_a Q(s', a)$&lt;/span&gt; - this is what we want after all since it eventually converges to the optimal Q-function.&lt;/p&gt;&#xA;&lt;p&gt;Now lets think about the Deep Q-learning case. We want our network to approximate &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; and so if we train the network, using the MSE loss, with &lt;span class=&quot;math-container&quot;&gt;$r + \max_a Q(s', a)$&lt;/span&gt; as our target then our network will gradually be shifted towards predicting &lt;span class=&quot;math-container&quot;&gt;$r + \max_aQ(s', a)$&lt;/span&gt; (which again would give us optimal Q-values for state-action pairs), just like with the TD update.&lt;/p&gt;&#xA;&lt;p&gt;This is assuming that you know how training of neural networks works so if you don't then I would recommend asking/searching for a relevant question that explains this.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-12-10T10:48:43.857" LastActivityDate="2020-12-10T10:48:43.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25091" PostTypeId="1" CreationDate="2020-12-10T00:43:49.637" Score="1" ViewCount="18" Body="&lt;p&gt;In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. In several examples that I've encountered, the noise vector had 100 numbers (&lt;a href=&quot;https://www.tensorflow.org/tutorials/generative/dcgan&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementation  1&lt;/a&gt;, &lt;a href=&quot;https://github.com/eriklindernoren/Keras-GAN/blob/master/dcgan/dcgan.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementation 2&lt;/a&gt;). Is there a reason this number is used? How does noise size affect the generation image?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="38076" LastEditDate="2020-12-10T01:48:29.090" LastActivityDate="2020-12-10T01:48:29.090" Title="How does noise input size affect fake image generation with GANs?" Tags="&lt;neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;hyper-parameters&gt;&lt;deepfakes&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25094" PostTypeId="2" ParentId="25053" CreationDate="2020-12-10T07:16:59.040" Score="3" Body="&lt;p&gt;I took a look at the &lt;a href=&quot;https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer_test.py#L142&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensor2Tensor's source code implementation&lt;/a&gt; as per @nbro's suggestion, and it seems like the loss function is the cross entropy between the predicted (&lt;span class=&quot;math-container&quot;&gt;$\|sentence length\|$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$\|vocab\|$&lt;/span&gt;) probability matrix (right before taking the argmax to find the token to output), and the &lt;span class=&quot;math-container&quot;&gt;$\|sentence length\|$&lt;/span&gt;-length vector of token IDs as the true label.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-10T07:16:59.040" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25095" PostTypeId="1" AcceptedAnswerId="25104" CreationDate="2020-12-10T08:59:38.080" Score="0" ViewCount="80" Body="&lt;p&gt;I am not familiar with Deep learning and Pytorch. And I want to know how to deal, in general with such a situation. So, I was wondering if I used a pretrained model (EfficientNet for example) if I want to change the _fc attribute and use conv2d in it, how can I recover a 2D structure? Because the pretrained model flattens it just before _fc.&lt;/p&gt;&#xA;&lt;p&gt;for example, the pretrained model outputs a flattened feature vector of 1280 elements what I did is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.efficient_net._fc = nn.Sequential(&#xA;                nn.Linear(1280, 1225),&#xA;                nn.Unflatten(dim=1, unflattened_size=(1, 35, 35)),&#xA;                nn.Conv2d(1, 35, kernel_size=1),&#xA;                ...,&#xA;                )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I didn't have a specific height and width to recover in the 2D structure, so I assumed that h = w = &lt;em&gt;some size&lt;/em&gt; and I use a linear layer whose output is equal to the square of the &amp;quot;&lt;em&gt;some size&lt;/em&gt;&amp;quot;. In the example above 35² = 1225. I am not sure if the unflatten is the correct way to do this. Then I added the conv2d. My code works but it doesn't give good results which probably means that the 2D structure I recovered does not capture any meaningful information.&#xA;Can anyone enlighten me with general knowledge about how things are done in my situation, or give me some comments? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="40411" LastEditorUserId="40411" LastEditDate="2020-12-10T10:15:23.907" LastActivityDate="2020-12-10T22:09:40.913" Title="How to use a conv2d layer after a flatten?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;pytorch&gt;&lt;pretrained-models&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25096" PostTypeId="1" CreationDate="2020-12-10T10:14:32.983" Score="1" ViewCount="4" Body="&lt;p&gt;When we translate a text from one language to another, how does the frequency of various POS tags change? So let's say we have a text in English with 10% nouns, 20% adjectives, 15% adverbs, 25% verbs, etc, which we now translate to German or French or Hindi. Can we say that in these other languages the POS tag frequency will remain same as earlier?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&#xA;Kushal.&lt;/p&gt;&#xA;" OwnerUserId="42863" LastActivityDate="2020-12-10T10:14:32.983" Title="POS Tag Frequency and Language Translation" Tags="&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;language-model&gt;&lt;pos-tagging&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25097" PostTypeId="1" AcceptedAnswerId="25116" CreationDate="2020-12-10T12:45:18.867" Score="1" ViewCount="287" Body="&lt;p&gt;Since transformers are good at processing sequential data, can we also use them for audio classification problems (same as RNNs)?&lt;/p&gt;&#xA;" OwnerUserId="42865" LastEditorUserId="2444" LastEditDate="2020-12-11T11:06:49.350" LastActivityDate="2020-12-11T11:06:49.350" Title="Can we use transformers for audio classification tasks?" Tags="&lt;reference-request&gt;&lt;applications&gt;&lt;transformer&gt;&lt;sequence-modelling&gt;&lt;audio-processing&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25098" PostTypeId="1" CreationDate="2020-12-10T13:08:09.247" Score="1" ViewCount="19" Body="&lt;p&gt;I am looking for some known approach, or some previous work, on the following problem:&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; be an alphabet of symbols and &lt;span class=&quot;math-container&quot;&gt;$\Sigma^*$&lt;/span&gt; be the set of all the strings that you can compose from this alphabet. Furthermore, let &lt;span class=&quot;math-container&quot;&gt;$f:\Sigma^*\rightarrow2^{\Sigma^*}$&lt;/span&gt; be a function that assigns a certain set of &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt;-strings to each &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt;-string. Suppose you have a dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}\subseteq\Sigma^*\times2^{\Sigma^*}$&lt;/span&gt; of input-output pairs.&lt;/p&gt;&#xA;&lt;p&gt;With this data, the goal is to learn a function &lt;span class=&quot;math-container&quot;&gt;$f^\prime:\Sigma^*\rightarrow2^{\Sigma^*}$&lt;/span&gt; that, given a string &lt;span class=&quot;math-container&quot;&gt;$\sigma\in\Sigma^*$&lt;/span&gt;, gives any superset of &lt;span class=&quot;math-container&quot;&gt;$f(\sigma)$&lt;/span&gt;, e.g. &lt;span class=&quot;math-container&quot;&gt;$f^\prime(\sigma)\supseteq f(\sigma)$&lt;/span&gt;. Of course, returning the set of all strings is not a good solution, so &lt;span class=&quot;math-container&quot;&gt;$f^\prime(\sigma)$&lt;/span&gt; should not be much larger than &lt;span class=&quot;math-container&quot;&gt;$f(\sigma)$&lt;/span&gt; (to give a rough idea, if &lt;span class=&quot;math-container&quot;&gt;$|f(\sigma)|=10$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$|f^\prime(\sigma)|=100$&lt;/span&gt; would still be ok, but &lt;span class=&quot;math-container&quot;&gt;$|f^\prime(\sigma)|=10000$&lt;/span&gt; wouldn't). To give an intuitive reason behind this, I have already an algorithm which, given a &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; and a set &lt;span class=&quot;math-container&quot;&gt;$S\supseteq f(\sigma)$&lt;/span&gt;, returns &lt;span class=&quot;math-container&quot;&gt;$f(\sigma)$&lt;/span&gt;. However, this algorithm has a extremely high time-complexity (growing with &lt;span class=&quot;math-container&quot;&gt;$|S|$&lt;/span&gt;), and I want to use this machine learning approach to narrow down the search.&lt;/p&gt;&#xA;&lt;p&gt;I would like to use any Machine Learning approach (from Evolutionary Computing to Deep Learning) to solve this problem.&lt;/p&gt;&#xA;&lt;p&gt;So far my only idea would be to use an encoder-decoder architecture. I construct character embeddings for all symbols in &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt;, and then through some neural architecture (I was thinking about an LSTM) I aggregate them to obtein a string representation. Given this, the decoder generates in sequence all elements of the corresponding set (by a similar, but inverse, fashion).&lt;/p&gt;&#xA;&lt;p&gt;This is clearly not optimal, because sets lack any meaningful order, and this approach is order-dependent (by nature of LSTMs and decoders in general). Of course I could always sort all sets, but this still imposes a structure to my problem that is not there, and I feel like this could make it harder to solve.&lt;/p&gt;&#xA;&lt;p&gt;So, in sum, my question is: &lt;strong&gt;Is there any known approach to the problem of generating sets of objects from a given input in the literature? If not, how could I improve my approach?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="23527" LastEditorUserId="23527" LastEditDate="2020-12-10T14:06:32.357" LastActivityDate="2020-12-10T14:06:32.357" Title="Is there any known approach to generate sets of objects?" Tags="&lt;reference-request&gt;&lt;supervised-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25099" PostTypeId="1" CreationDate="2020-12-10T13:14:30.273" Score="0" ViewCount="185" Body="&lt;p&gt;Attention models/gates are used to focus/pay attention to the important regions. According to &lt;a href=&quot;https://arxiv.org/pdf/1804.03999.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the authors describe that a model with Attention Gate (AG) can be trained from scratch. Then the AGs automatically learn to focus on the target.&lt;/p&gt;&#xA;&lt;p&gt;What I am having trouble understanding is that, in the context of computer vision, doesn't a filter from the convolutional layers learn the region of interest?&lt;/p&gt;&#xA;&lt;p&gt;The authors say that adding Attention Gate reduces complexity when compared with multi-stage CNNs. But the job a trained AG would do is the same as that of a filter in a convolutional layer that would lead to the correct output, right?&lt;/p&gt;&#xA;" OwnerUserId="41564" LastEditorUserId="2444" LastEditDate="2020-12-11T11:11:16.750" LastActivityDate="2020-12-12T18:48:51.947" Title="What is the difference between Attention Gate and CNN filters?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;attention&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25100" PostTypeId="1" CreationDate="2020-12-10T14:18:43.867" Score="0" ViewCount="11" Body="&lt;p&gt;I have around 3M BW images and I would like to organize them in as few clusters as possible in way which is meaningful for the dataset &lt;strong&gt;without any prior knowledge for this data&lt;/strong&gt;, as they come from domains which are not represented in classic models trained in classic datasets like ImageNET etc.&lt;/p&gt;&#xA;&lt;p&gt;My target is to produce meaningful labels for images,&#xA;By meaningful I mean in a &lt;strong&gt;machinic&lt;/strong&gt; way. I don't want exactly to have human readable labels, but I would like for the labels not to be that arbitrary (thus to relate with higher-semantically or &lt;strong&gt;deep&lt;/strong&gt; correspondences from the vast amount of pictures, which I supply to the model).&lt;/p&gt;&#xA;&lt;p&gt;Are you aware of any method able to do something like this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="42868" LastEditorUserId="42868" LastEditDate="2020-12-10T15:55:05.520" LastActivityDate="2020-12-10T15:55:05.520" Title="Deep Unsupervised clustering on big data with no prior knowledge" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;clustering&gt;&lt;data-mining&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25101" PostTypeId="1" CreationDate="2020-12-10T16:25:30.843" Score="1" ViewCount="46" Body="&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_tree_search#Exploration_and_exploitation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;, in the Monte-Carlo Tree Search algorithm, you should choose the node that maximizes the value:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\displaystyle {\frac {w_{i}}{n_{i}}}+c{\sqrt {\frac {\ln N_{i}}{n_{i}}}}},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;${w_{i}}$&lt;/span&gt; stands for the number of wins for the node considered after the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th move,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;${n_{i}}$&lt;/span&gt; stands for the number of simulations for the node considered after the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th move,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$N_{i}$&lt;/span&gt; stands for the total number of simulations after the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th move run by the parent node of the one considered&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the exploration parameter—theoretically equal to&lt;span class=&quot;math-container&quot;&gt;$\sqrt{2}$&lt;/span&gt;; in practice usually chosen empirically.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here (and I've seen in other places as well) it claims that the theoretical ideal value for &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\sqrt{2}$&lt;/span&gt;. Where does this value come from?&lt;/p&gt;&#xA;&lt;p&gt;(Note: I did post this same &lt;a href=&quot;https://stats.stackexchange.com/q/500245/82135&quot;&gt;question&lt;/a&gt; on cross-validated before I knew about this (more relevant) site)&lt;/p&gt;&#xA;" OwnerUserId="42870" LastEditorUserId="2444" LastEditDate="2020-12-10T17:34:06.453" LastActivityDate="2020-12-10T17:34:06.453" Title="Why is the ideal exploration parameter in the UCT algorithm $\sqrt{2}$?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-tree-search&gt;&lt;exploration-exploitation-tradeoff&gt;&lt;upper-confidence-bound&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25102" PostTypeId="1" AcceptedAnswerId="25115" CreationDate="2020-12-10T16:27:22.130" Score="2" ViewCount="113" Body="&lt;p&gt;Raul Rojas's book on Neural Networks dedicates &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 8.4.3&lt;/a&gt; to explaining how to do second-order backpropagation, that is, computing the Hessian of the error function with respect to two weights at a time.&lt;/p&gt;&#xA;&lt;p&gt;What problems are easier to solve using this approach rather than first-order backpropagation?&lt;/p&gt;&#xA;" OwnerUserId="14892" LastEditorUserId="2444" LastEditDate="2020-12-12T11:30:41.337" LastActivityDate="2020-12-12T11:30:41.337" Title="Why is second-order backpropagation useful?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;optimization&gt;&lt;numerical-algorithms&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25103" PostTypeId="1" CreationDate="2020-12-10T16:59:48.363" Score="0" ViewCount="14" Body="&lt;p&gt;I have a multivariate function. It takes 4, real valued inputs: a, b, c, d and returns 1 complex number, z.&#xA;I wanted to use Neural Networks to predict the value z' for a generic input a', b', c', d'.&#xA;I created an MLP network which takes the 4 inputs and outputs 2 numbers x and y, which I interpret as z=x+iy. My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What NN architecture do you recommend for multivariate function interpolation?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I normalise the data when, in principle, the input can be any real number?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In order to normalise the data, I decided to apply the logistic function on both the inputs and outputs and then apply the inverse of the logistic function on the results. Is this beneficial?&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;I'm using KERAS and my NN has the following structure:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Input of dimension 4&lt;/p&gt;&#xA;&lt;p&gt;Dense layer of size s1, activation function relu,&lt;/p&gt;&#xA;&lt;p&gt;Dense layer of size s2, activation function relu,&lt;/p&gt;&#xA;&lt;p&gt;Dense layer of size 2, activation function linear.&lt;/p&gt;&#xA;&lt;p&gt;Loss function is mean squared error, optimizer Adam. I can create hundreds of thousands of training data.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any comments regarding the above structure?&lt;/p&gt;&#xA;" OwnerUserId="42871" LastActivityDate="2020-12-10T16:59:48.363" Title="NN for multivariate function interpolation" Tags="&lt;multilayer-perceptron&gt;&lt;interpolation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25104" PostTypeId="2" ParentId="25095" CreationDate="2020-12-10T22:09:40.913" Score="1" Body="&lt;p&gt;To answer the question in the title, your enclosed method is a valid way to use 2d convs after a flattened feature vector. However, the bad results you experience could come from the structure of your model or from the way you train it. Regarding you last question, it is very hard to give you an advice without knowing your intentions in detail. Regardless, here are my two cents.&lt;/p&gt;&#xA;&lt;p&gt;First of all, in the case you actually want to use this approach, to have a pretrained model and add your layers after one of its layers, you might want to keep the parameters of the original network intact at least until your newly initialized layers get trained properly. To achieve that, you need to use the gradients for updating only the parts you added as described &lt;a href=&quot;https://discuss.pytorch.org/t/detach-no-grad-and-requires-grad/16915/14&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this comment&lt;/a&gt;. (You definitely should read the other comments there as well to get a better picture.)&lt;/p&gt;&#xA;&lt;p&gt;Secondly, it might be worth reconsidering if you really want to add your layers after the very last layers of the pretrained network. Depending on your goals, using the output of some prior layers as the input to your layers might be more beneficial to you. (Just do not forget to keep the parameters of the pretrained model intact as I advised in the prior point.)&lt;/p&gt;&#xA;&lt;p&gt;Lastly, the structure of your layers should also be reconsidered. 2d convs with a kernel size of 1x1 in this situation seems strange to my limited experience, but without knowing what you want to do, its hard to give any solid advice in this regard.&lt;/p&gt;&#xA;&lt;p&gt;Therefore you might be better off splitting your question into smaller parts and work your way through them one by one.&lt;/p&gt;&#xA;&lt;p&gt;(My reputation is not high enough otherwise I would have left a comment.)&lt;/p&gt;&#xA;" OwnerUserId="40549" LastActivityDate="2020-12-10T22:09:40.913" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25105" PostTypeId="1" CreationDate="2020-12-10T22:14:46.170" Score="1" ViewCount="34" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/abs/1908.03265&quot; rel=&quot;nofollow noreferrer&quot;&gt;On the Variance of the Adaptive Learning Rate and Beyond&lt;/a&gt;, in section 2, the authors write&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To further analyze this phenomenon, we visualize the histogram of the absolute value of gradients on a log scale in Figure 2. We observe that, without applying warmup, the gradient distribution is distorted to have a mass center in relatively small values within 10 updates. Such gradient distortion means that the vanilla Adam is trapped in bad/suspicious local optima after the first few updates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here is figure 2 from the paper.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JbhuE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JbhuE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain this part?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Such gradient distortion means that the vanilla Adam is trapped in bad/suspicious local optima after the first few updates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why is this true?&lt;/p&gt;&#xA;" OwnerUserId="42535" LastEditorUserId="2444" LastEditDate="2020-12-11T00:21:59.097" LastActivityDate="2021-05-10T07:06:30.610" Title="Why is Adam trapped in bad/suspicious local optima after the first few updates?" Tags="&lt;deep-learning&gt;&lt;papers&gt;&lt;optimization&gt;&lt;learning-rate&gt;&lt;adam&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25106" PostTypeId="1" CreationDate="2020-12-10T22:29:42.990" Score="0" ViewCount="25" Body="&lt;p&gt;After reading this topic on GitHub &lt;a href=&quot;https://github.com/pjreddie/darknet/issues/742&quot; rel=&quot;nofollow noreferrer&quot;&gt;how long time it takes to train YOLOV3 on coco dataset&lt;/a&gt; I was wondering how researchers deal with long training times while inventing new architectures.&lt;/p&gt;&#xA;&lt;p&gt;I imagine that to evaluate the model you need to train it first. How do they make tweaks in their architectures, e.g. tweaking layers, adding pooling, dropout, etc., if training can take a few days? Is it pure art and it is designed roughly or is it a more deliberate process?&lt;/p&gt;&#xA;&lt;p&gt;What are the steps of engineering new architecture using deep neural networks?&lt;/p&gt;&#xA;" OwnerUserId="42878" LastActivityDate="2020-12-11T07:06:48.347" Title="What is the process of inventing deep neural network models? How researchers deal with long training times?" Tags="&lt;deep-learning&gt;&lt;research&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25107" PostTypeId="1" CreationDate="2020-12-10T23:57:56.997" Score="0" ViewCount="19" Body="&lt;p&gt;I want to know what is the best way to find the most suitable k in knn algorithm for Item-based collaborative filtering .&#xA;I want to make a recommendation system for an application to recommend the most similar items (products) for the product which user liked the most based on rating values .&#xA;The application is an ecommerce application and the products are published by the users (as social apps)&#xA;the data which used as input to the algorithm is not huge. It depends on how many products the users post in the application. &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;note: if it makes sense, I used cosine similarity to find the similarity between items.&lt;/p&gt;&#xA;" OwnerUserId="42879" LastEditorUserId="42879" LastEditDate="2020-12-11T00:11:40.243" LastActivityDate="2020-12-11T00:11:40.243" Title="How can I find the optimal K for K-nearest neighbor Algorithm" Tags="&lt;recommender-system&gt;&lt;k-nearest-neighbors&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25108" PostTypeId="1" CreationDate="2020-12-11T02:43:13.370" Score="1" ViewCount="17" Body="&lt;p&gt;I'm working on a project to equip model locomotives with sound boards. I'm in the process of designing the board at the moment, and the idea is to allow users to load their own sound files onto an SD card plugged into the board.&lt;/p&gt;&#xA;&lt;p&gt;Conventionally, model locomotive sounds are collected from high-fidelity microphones placed on and around the real engine in question. The engine is started up then put through idle and all of the different notches, as well as dynamic braking, horn and bell sounds, etc. This practice is very expensive because you have to find a willing (usually small) railroad or museum, pay for travel expenses, and diesel fuel ain't exactly cheap at the volumes these engines go through. Secondly, newer engines are hard to record because railroads aren't exactly in the business of letting hobbyists tape microphones all over their money making machines. As such, the main cost for a sound board comes not from the circuit's BOM cost, but from the effort required to get sounds from locomotives.&lt;/p&gt;&#xA;&lt;p&gt;What there's plenty of are YouTube videos of amateur rail enthusiasts taking videos of locomotive sightings at close(ish) proximity, including startups and shutdowns. My question is - is there a way to take a bunch of different audio recordings of the same engine, remove noise and the doppler effect, and from that create a profile that can be used to simulate what the engine might sound like at different throttle notches? Is machine learning the right tool for this?&lt;/p&gt;&#xA;" OwnerUserId="42883" LastActivityDate="2020-12-11T02:43:13.370" Title="Can Machine Learning be used to synthesize engine sounds?" Tags="&lt;machine-learning&gt;&lt;audio-processing&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25109" PostTypeId="1" CreationDate="2020-12-11T03:54:32.363" Score="4" ViewCount="75" Body="&lt;p&gt;I trained a simple model to recognize handwritten numbers from the mnist dataset. Here it is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model = Sequential([&#xA;    Conv2D(filters=1, kernel_size=(3,1), padding='valid', strides=1, input_shape=(28, 28, 1)),&#xA;    Flatten(),&#xA;    Dense(10, activation='softmax')])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I experimented with varying the number of filters for the convolutional layer, while keeping other parameters constant(learning rate=0.0001, number of episodes=2000, training batch size=512). I used 1, 2, 4, 8, and 16 filters, and the model accuracy was 92-93% for each of them.&lt;/p&gt;&#xA;&lt;p&gt;From my understanding, during the training the filters may learn to recognize various types of edges in the image (e.g, vertical, horizontal, round). This experiment made me wonder whether any of the filters end up being duplicate -- having the same or similar weights. Is there anything that prevents them from that?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="38076" LastEditDate="2020-12-30T16:14:15.877" LastActivityDate="2020-12-30T16:14:15.877" Title="Is there anything that ensures that convolutional filters don't end up the same?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;filters&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25111" PostTypeId="2" ParentId="25109" CreationDate="2020-12-11T06:16:43.117" Score="4" Body="&lt;p&gt;No, nothing really prevents the weights from being different. In practice though they end up almost always different because it makes the model more expressive (i.e. more powerful), so gradient descent learns to do that. If a model has &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; features, but 2 of them are the same, then the model effectively has &lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt; features, which is a less expressive model than that of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; features, and therefore usually has a larger loss function.&lt;/p&gt;&#xA;&lt;p&gt;But even if the weights are different, some of them can be very similar. If you visualize the first layer of your convolution filters, and you have a large number of them (e.g. 100), you will see some of them are learning to detect roughly the same edges (same orientation and placement). These features are so similar, they are effectively redundant in the model and do not add to its predictive power.&lt;/p&gt;&#xA;&lt;p&gt;There's actually an entire field of research on identifying redundant features and pruning them. Le Cun shows in &lt;a href=&quot;http://yann.lecun.com/exdb/publis/pdf/lecun-90b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal Brain Damage&lt;/a&gt; that pruning out redundant features not only make the model smaller and faster for inference, but can also even improve the model's accuracy.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://blog.dataiku.com/making-neural-networks-smaller-for-better-deployment-solving-the-size-problem-of-cnns-using-network-pruning-with-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a blog post for a high level overview of one of the pruning methods for more info.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-11T06:16:43.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25112" PostTypeId="2" ParentId="21118" CreationDate="2020-12-11T06:32:56.127" Score="3" Body="&lt;p&gt;I think the colloquial understanding of Gödel's incompleteness theorems allows them to be too broadly applied. Gödel's second incompleteness regards the &lt;strong&gt;consistency&lt;/strong&gt; of a formal system, which is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Consistency&quot; rel=&quot;nofollow noreferrer&quot;&gt;technical concept&lt;/a&gt; of formal systems that means the system cannot prove every formula. It is commonly framed as a system not being able to prove both a formula and its negation (e.g. &lt;span class=&quot;math-container&quot;&gt;$2+2=4$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$2+2 \neq 4$&lt;/span&gt;), since many logical systems allow &lt;a href=&quot;https://en.wikipedia.org/wiki/Principle_of_explosion&quot; rel=&quot;nofollow noreferrer&quot;&gt;anything to be proven from a contradiction&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The second incompleteness theorem states that if a consistent formal system is expressive enough to encode basic arithmetic (&lt;a href=&quot;https://en.wikipedia.org/wiki/Peano_axioms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Peano arithmetic&lt;/a&gt;), then that system cannot prove its own consistency. This implies that we must use a stronger system B to prove the consistency of A. The system needs to be able to represent arithmetic because that is what is used to define the representability conditions of Gödel's proof that allowed him to formally construct the self-referential formulae central to the incompleteness theorems.&lt;/p&gt;&#xA;&lt;p&gt;Here I diverge with my own opinion on this, I feel the concept of consistency in formal systems has no obvious bearing on the limits of artificial intelligence. An intelligent agent need not know anything about formal consistency to reach its level of intelligence -- the vast majority of humans have never encountered this concept, and yet they are still intelligent. Even many mathematicians don't give it a second thought unless they are in the trenches of mathematical logic. One would have to take an overly narrow view of artificial intelligence to allow Gödel's second incompleteness to serve as a limitation to it.&lt;/p&gt;&#xA;&lt;p&gt;I caution against the popular informal restatements of Gödel's incompleteness theorems. These theorems were undoubtedly earth-shattering in the study of foundational mathematics and still have grand implications today, but projecting those results too far away from their rigorous origins is going to lead to many stray conclusions.&lt;/p&gt;&#xA;" OwnerUserId="20955" LastEditorUserId="20955" LastEditDate="2020-12-11T18:12:18.183" LastActivityDate="2020-12-11T18:12:18.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25113" PostTypeId="2" ParentId="25105" CreationDate="2020-12-11T06:56:16.963" Score="0" Body="&lt;p&gt;The authors describe their belief in Section 3:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Due to the lack of samples in the early stage, the adaptive learning rate has an undesirably large variance, which leads to suspicious/bad local optima.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Diving further, in section 3.2:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Adam uses the exponential moving average to calculate the adaptive learning rate.  For gradients &lt;span class=&quot;math-container&quot;&gt;$\{g_1,\dots,g_t\}$&lt;/span&gt;, their exponential moving average has a larger variance than their simple average.  Also, in the early stage (&lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is small), the difference of the exponential weights of &lt;span class=&quot;math-container&quot;&gt;$\{g_1,\dots,g_t\}$&lt;/span&gt; is relatively small (up to &lt;span class=&quot;math-container&quot;&gt;$1−β^{t−1}_2$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It seems like the root issue is that exponential moving average, while great with many data samples, have too large of a variance with few data samples. It is this variance that sometimes allow the gradient descend in a bad optima.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-11T06:56:16.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25114" PostTypeId="2" ParentId="25106" CreationDate="2020-12-11T07:06:48.347" Score="2" Body="&lt;p&gt;Part of the answer is that new architectures are often variants of existing architectures. There are some rough heuristics people follow (e.g. using powers of 2 in layer sizes, changing layer sizes according to a schedule, adding normalization layers, etc.). However, even when building off an existing architecture, there can still be a lot of toying around with different configurations and hyperparameters to require copious training time.&lt;/p&gt;&#xA;&lt;p&gt;Researchers working for a company will likely have the luxury of some serious compute, e.g. GPU clusters or cloud services like AWS. This can really make a huge difference over training a model on your PC GPU. Further, many models can be trained at once. Many academic institutions also employ cluster computing on dedicated GPU clusters - for example, here at the University of Utah our department has a cluster with 8 really excellent GPUs.&lt;/p&gt;&#xA;&lt;p&gt;One last thing I will note, is that it's not always necessary to train the model to completion on the full training set to get a sense of which network configuration and hyperparameter settings are working better than others. If I have a massive dataset that will take days to train on, I may take a smaller subset of the training data and do some quick comparisons between different models to get a feel for what changes have the greatest impact on test performance. It's not completely reliable, but it can guide intuition to narrow down the number of models you want to train on the full dataset to convergence. For example, using this technique I was able to find that predictions from my model were fairly invariant to the exact layer sizes I was using and was far more sensitive to a hyperparameter in my optimization solver.&lt;/p&gt;&#xA;" OwnerUserId="20955" LastActivityDate="2020-12-11T07:06:48.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25115" PostTypeId="2" ParentId="25102" CreationDate="2020-12-11T07:25:21.743" Score="3" Body="&lt;p&gt;Second-order optimization algorithms like Hessian optimization have more information on the curvature of the loss function, so converge much, much faster than first-order optimization algorithms like gradient descent. I remember reading somewhere that if you have &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; weights in the neural network, one iteration of a second-order optimization algorithm will reduce the loss function at approximately the same rate as &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; iterations of a standard first-order optimization algorithm. However, with recent advancements to gradient descent (momentum, adaptive rates, etc), the difference isn't as large anymore -- @EmmanuelMess pointed out a &lt;a href=&quot;https://core.ac.uk/download/pdf/187245955.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; that states:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The performance of the proposed first order and second order methods with adaptive gain (BP-AG, CGFR-AG, BFGS-AG) with standard second order methods without gain (BP, CGFR, BFGS) in terms of speed of convergence evaluated in the number of epochs and CPU time. Based on some simulation results, it’s showed that the proposed algorithm had shown improvements in the convergence rate with 40% faster than other standard algorithms without losing their accuracy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/a/145446/303597&quot;&gt;Here&lt;/a&gt; is a great post explaining the background behind the math of why this is the case.&lt;/p&gt;&#xA;&lt;p&gt;Also, second-order gradients can help the optimizer identify states like saddle points, and help the solver get out of those states. Saddle points give standard gradient descent a lot of issues, as the gradient descent has difficulty and is slow to move out of the saddle point. Fixing the saddle point issue is one of the motivations for improving gradient descent over the last two decades (SDG with momentum, adaptive learning rates, ADAM, etc). &lt;a href=&quot;https://stats.stackexchange.com/a/279094/303597&quot;&gt;More info&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gjDzm.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gjDzm.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The issue though is that to compute the second-order derivative requires a matrix &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; in size, as opposed to gradient descent which requires a matrix &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in size. The memory and computation become intractable for large networks, especially if you have millions of weights.&lt;/p&gt;&#xA;&lt;p&gt;Some approaches exist which efficiently approximates the second-order optimizations, solving the tractability problem. A popular one is &lt;a href=&quot;http://users.iems.northwestern.edu/%7Enocedal/PDFfiles/limited.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;L-BFGS&lt;/a&gt;. I haven't played around with it much, but I believe L-BFGS is not as popular as gradient descent algorithms (such as SGD-M, ADAM) because it is still very memory demanding (requires storing about 20-100 previous gradient evaluations), and cannot work in a stochastic context (you cannot sample mini-batches to train on; you must train the entire dataset in one pass per iteration). If those two are not an issue for you, then L-BFGS works pretty well I believe.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="2444" LastEditDate="2020-12-12T11:26:57.727" LastActivityDate="2020-12-12T11:26:57.727" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25116" PostTypeId="2" ParentId="25097" CreationDate="2020-12-11T07:34:58.260" Score="1" Body="&lt;p&gt;Yes, Transformers can be used to work with audio data, such as audio processing (audio classification, speaker identification, etc) (&lt;a href=&quot;https://arxiv.org/pdf/2005.08575.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Audio ALBERT&lt;/a&gt;), speech-to-text (&lt;a href=&quot;https://arxiv.org/pdf/2001.02674.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Streaming Automatic Speech Recognition with the Transformer Model&lt;/a&gt;), and text-to-speech (&lt;a href=&quot;https://arxiv.org/pdf/1809.08895.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Speech Synthesis with Transformer Network&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-11T07:34:58.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25117" PostTypeId="5" CreationDate="2020-12-11T11:13:20.640" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-11T11:13:20.640" LastActivityDate="2020-12-11T11:13:20.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25118" PostTypeId="4" CreationDate="2020-12-11T11:13:20.640" Score="0" Body="For questions about audio processing tasks in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-11T11:13:20.640" LastActivityDate="2020-12-11T11:13:20.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25119" PostTypeId="5" CreationDate="2020-12-11T14:35:42.470" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-11T14:35:42.470" LastActivityDate="2020-12-11T14:35:42.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25120" PostTypeId="4" CreationDate="2020-12-11T14:35:42.470" Score="0" Body="For questions about rule-based systems in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-11T14:35:42.470" LastActivityDate="2020-12-11T14:35:42.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25121" PostTypeId="2" ParentId="13010" CreationDate="2020-12-11T15:14:23.527" Score="0" Body="&lt;p&gt;If you know the latitude-longitude of the trucks and the center, you can do the following,&lt;/p&gt;&#xA;&lt;p&gt;Given the latitude-longitude location of the center and the radius in which you want to search the presence of a truck(say R) around center, you can find the latitude-longitude bounds of the space around center within R radius by : &lt;a href=&quot;http://janmatuschek.de/LatitudeLongitudeBoundingCoordinates&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can find python implementation here:&lt;a href=&quot;https://github.com/saivarshittha/latitude_longitude_bounds/blob/master/latitude_longitude_bounds.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Once you know the bounds, you can simply check if the truck's location falls within the latitude-longitude bounds.&lt;/p&gt;&#xA;" OwnerUserId="41585" LastActivityDate="2020-12-11T15:14:23.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25124" PostTypeId="1" CreationDate="2020-12-11T23:42:38.183" Score="1" ViewCount="13" Body="&lt;p&gt;I have been reading recently about value and policy iteration. I tried to code the algorithms to understand them better and in the process I discovered something and I am not sure why is the case (or if my code is doing the right thing)&lt;/p&gt;&#xA;&lt;p&gt;If I compute the expected value until convergence I get the following grid:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q5f4G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q5f4G.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I compute the &lt;strong&gt;optimal&lt;/strong&gt; value with value iteration I will get the following.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yslvp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yslvp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, the values are different but their relative magnitude is similar, i.e. the 3rd column in the last row has the greatest value in both computations.&lt;/p&gt;&#xA;&lt;p&gt;I believe this makes sense, as the expected value will tend to accumulate values in &amp;quot;promising&amp;quot; cells. But I assume that the first computation won't help much because it does not tell us what is the optimal policy, whereas the second, does.&lt;/p&gt;&#xA;&lt;p&gt;Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="42853" LastEditorUserId="2444" LastEditDate="2020-12-12T12:09:45.750" LastActivityDate="2020-12-12T12:09:45.750" Title="Are the relative magnitudes of the learned and optimal state value function the same?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;value-iteration&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25125" PostTypeId="5" CreationDate="2020-12-12T01:04:13.953" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T01:04:13.953" LastActivityDate="2020-12-12T01:04:13.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25126" PostTypeId="4" CreationDate="2020-12-12T01:04:13.953" Score="0" Body="For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T01:04:13.953" LastActivityDate="2020-12-12T01:04:13.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25127" PostTypeId="1" AcceptedAnswerId="25130" CreationDate="2020-12-12T01:42:32.290" Score="2" ViewCount="81" Body="&lt;p&gt;I am trying to understand and reproduce the &lt;strong&gt;&lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proximal Policy Optimization (PPO)&lt;/a&gt;&lt;/strong&gt; algorithm in detail.  One thing that I find missing in the &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; introducing the algorithm is how exactly actions &lt;span class=&quot;math-container&quot;&gt;$a_t$&lt;/span&gt; are generated given the policy network &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta(a_t|s_t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From the &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/ppo1/pposgd_simple.py#L32&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt;, I saw that &lt;em&gt;discrete&lt;/em&gt; actions get sampled from some probability distribution (which I assume to be discrete in this case) parameterized by the output probabilities generated by &lt;span class=&quot;math-container&quot;&gt;$\pi_\theta$&lt;/span&gt; given the state &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, what I don't understand is how &lt;em&gt;continuous&lt;/em&gt; actions are sampled/generated from the policy network. Are they also sampled from a (probably continuous) distribution? In that case, which type of distribution is used and which parameters are predicted by the policy network to parameterize said distribution?&lt;/p&gt;&#xA;&lt;p&gt;Also, is there any official literature that I could cite which introduces the method by which PPO generates its action outputs?&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2020-12-16T20:56:21.757" LastActivityDate="2020-12-16T20:56:21.757" Title="How are continuous actions sampled (or generated) from the policy network in PPO?" Tags="&lt;reinforcement-learning&gt;&lt;implementation&gt;&lt;proximal-policy-optimization&gt;&lt;continuous-action-spaces&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25128" PostTypeId="1" CreationDate="2020-12-12T06:05:46.757" Score="1" ViewCount="16" Body="&lt;p&gt;Context: &lt;a href=&quot;https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Double Q-learning&lt;/a&gt; was introduced to prevent the maximization bias from q-learning. Instead of learning a single Q-network, we can learn two (or in general &lt;span class=&quot;math-container&quot;&gt;$K &amp;gt; 1$&lt;/span&gt;) and our Q-estimate would be the min across all these Q-networks.&lt;/p&gt;&#xA;&lt;p&gt;Question:&#xA;Does it make sense to share the layers of these Q-networks (except the last layer)?&lt;/p&gt;&#xA;&lt;p&gt;So, instead of having 2 networks of size [64, 64, 2] (with ~8.5K parameters in total) we can have one network of size [64, 64, 4] (with ~4.3K params).&lt;/p&gt;&#xA;&lt;p&gt;I couldn't see much of a downside to this, but all the implementations I've seen keep two completely different networks.&lt;/p&gt;&#xA;" OwnerUserId="36922" LastEditorUserId="2444" LastEditDate="2020-12-12T11:02:44.660" LastActivityDate="2020-12-12T11:02:44.660" Title="Would it make sense to share the layers (except the last one) of the neural networks in Double DQN?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;implementation&gt;&lt;double-dqn&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25129" PostTypeId="2" ParentId="20158" CreationDate="2020-12-12T06:13:16.343" Score="0" Body="&lt;p&gt;To me, Bellman update is simply supervised learning: right hand side (bootstrap) is a sample of the left hand side (conditional expectation).. The Bellman equation simply explains that the right hand side is such a sample.&lt;/p&gt;&#xA;" OwnerUserId="36922" LastActivityDate="2020-12-12T06:13:16.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25130" PostTypeId="2" ParentId="25127" CreationDate="2020-12-12T06:22:09.017" Score="2" Body="&lt;p&gt;As long as your policy (propensity) is differentiable, everything's is good. Discrete, continuous, other, doesn't matter! :)&lt;/p&gt;&#xA;&lt;p&gt;A common example for continuous spaces is the &lt;a href=&quot;https://spinningup.readthedocs.io/zh_CN/latest/algorithms/sac.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;reparameterization trick&lt;/a&gt;, where your policy outputs &lt;span class=&quot;math-container&quot;&gt;$\mu, \sigma = \pi(s)$&lt;/span&gt; and the action is &lt;span class=&quot;math-container&quot;&gt;$a \sim \mathcal{N}(\mu, \sigma)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36922" LastEditorUserId="2444" LastEditDate="2020-12-12T12:05:16.137" LastActivityDate="2020-12-12T12:05:16.137" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25131" PostTypeId="1" AcceptedAnswerId="25151" CreationDate="2020-12-12T06:30:18.620" Score="2" ViewCount="67" Body="&lt;p&gt;Before deep learning, I worked with machine learning problems where the data had a large class imbalance (30:1 or worse ratios). At that time, all the classifiers struggled, even after under-sampling the represented classes and creating synthetic examples of the underrepresented classes -- except Random Forest, which was a bit more robust than the others, but still not great.&lt;/p&gt;&#xA;&lt;p&gt;What are guidelines for class distribution when it comes to deep learning (CNNs, ResNets, transformers, etc)? Must the representation of each class be 1:1? Or maybe it's &amp;quot;good enough&amp;quot; as long as it is under some ratio like 2:1? Or is deep learning completely immune to class imbalance as long as we have enough training data?&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, as a general guideline, should each class have a certain minimum number of training examples (maybe some multiple of the number of weights of the network)?&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-13T01:33:25.337" LastActivityDate="2020-12-17T00:40:58.557" Title="How robust are deep networks to class imbalance?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;datasets&gt;&lt;data-preprocessing&gt;&lt;imbalanced-datasets&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25133" PostTypeId="2" ParentId="25089" CreationDate="2020-12-12T10:26:35.283" Score="0" Body="&lt;p&gt;If I understand what you mean correctly then the answer is &lt;em&gt;basically nothing&lt;/em&gt;. Fundamentlaly all ML algorithms are ding the same thing, which is to optimize some weights for a certain output (this is true even for non-parametric methods in an implict way, but lets not dive too deep here). The only differences are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Dataset the model is trained on&lt;/li&gt;&#xA;&lt;li&gt;Specific dimensionality of inputs and outputs to make them compatible with the input data and the output labels.&lt;/li&gt;&#xA;&lt;li&gt;Complexity that can be expressed by the model (the number of weights and their structure in layers in the case of Neural Networks).&lt;/li&gt;&#xA;&lt;li&gt;Changes to the optimization process (gradient clipping, regularization, ecc.)&lt;/li&gt;&#xA;&lt;li&gt;Structural changes to the architecture that embed assumptions about the specific problem setting (e.g. 2D-Convulutions embed assumptions for images, softmax activations embeds the assumption for classification with probabilities, hidden states embed the assumptions about memory and &amp;quot;fogetting&amp;quot;, ecc...)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Therefore, if you have a &amp;quot;similar&amp;quot; problem setting where you can assume that 3, 4, 5 can be the same without problems, then you can just make the appropriate changes in 2 and change the dataset (1) to get a model that works on something different.&lt;/p&gt;&#xA;&lt;p&gt;Of course, being able to tell how &amp;quot;similar&amp;quot; a problem is to another and what possible things could be different is quite tricky and requires a lot of knowledge about the domain of Machine Learning and how every algorithm works and why.&lt;/p&gt;&#xA;&lt;p&gt;In essence, I'm saying that in principle you could take a model and train it for a new setting with very minor and simple changes that don't require a lot of knowledge. However, without the wider knowledge on the field of ML/AI you won't be able to tell if what you are doing is ok and how well can it work in general.&lt;/p&gt;&#xA;" OwnerUserId="42903" LastActivityDate="2020-12-12T10:26:35.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25134" PostTypeId="1" CreationDate="2020-12-12T12:07:03.977" Score="0" ViewCount="39" Body="&lt;p&gt;I’ve set up a neural network model to experiment with predicting foreign exchange rates based on various economic data. The model learned fine and the test data is OK (&lt;span class=&quot;math-container&quot;&gt;$R^2 = 0.88$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;But I can't figure out how to input data for scenarios where new data is outside the range of the datasets used to train the model. For example, if US debt is increased (again), then it will be outside the data range used for the training datasets, so, when normalised using the same parameters as the training dataset (0-1 scale), it will be greater than 1, so the model rejects it.&lt;/p&gt;&#xA;&lt;p&gt;Everything I've read says to normalise the new data using the training data parameters (understandably), but I can't find anything that explains how to use the model to make predictions where new data is outside the range of the training datasets.&lt;/p&gt;&#xA;&lt;p&gt;In parallel, I've used a regression model, but I'm fairly new to neural networks and would like to find a way of using these for this kind of prediction model. Any help gratefully received.&lt;/p&gt;&#xA;" OwnerUserId="42905" LastEditorUserId="2444" LastEditDate="2020-12-12T17:36:00.593" LastActivityDate="2020-12-12T17:36:00.593" Title="How to deal with predictions for data outside the range of the training dataset in neural networks?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;prediction&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25135" PostTypeId="5" CreationDate="2020-12-12T12:24:48.670" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T12:24:48.670" LastActivityDate="2020-12-12T12:24:48.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25136" PostTypeId="4" CreationDate="2020-12-12T12:24:48.670" Score="0" Body="For questions that involve imbalanced (or unbalanced) datasets." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T12:24:48.670" LastActivityDate="2020-12-12T12:24:48.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25137" PostTypeId="1" CreationDate="2020-12-12T14:58:53.140" Score="1" ViewCount="96" Body="&lt;p&gt;When &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Proximal Policy Optimization (PPO)&lt;/strong&gt;&lt;/a&gt; was released, it was accompanied by a &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; describing it.&lt;/p&gt;&#xA;&lt;p&gt;Later, the authors at OpenAI introduced a second version of PPO, called &lt;a href=&quot;https://github.com/openai/baselines/blob/master/baselines/ppo2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO2&lt;/a&gt; (whereas the original version is now commonly referred to as &lt;a href=&quot;https://github.com/openai/baselines/tree/master/baselines/ppo1&quot; rel=&quot;nofollow noreferrer&quot;&gt;PPO1&lt;/a&gt;). Unfortunately, the several changes made between PPO1 and PPO2 are pretty much undocumented (as stated over &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/modules/ppo2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Someone associated with OpnenAI's &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;baselines&lt;/a&gt; Deep Reinforcement Learning repository &lt;a href=&quot;https://github.com/openai/baselines/issues/485#issuecomment-413722708&quot; rel=&quot;nofollow noreferrer&quot;&gt;commented&lt;/a&gt; that the main advancement of PPO2 (compared to PPO1) was the use of a more advanced parallelism strategy, leading to improved performance. Unfortunately, the person omitted naming further changes made.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Now, I was wondering if anyone is aware of a (reliable) source of information or (preferably) even some published literature that lists &lt;strong&gt;all the numerous differences&lt;/strong&gt; between PPO1 and PPO2.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="2444" LastEditDate="2020-12-12T16:37:22.357" LastActivityDate="2020-12-12T16:37:22.357" Title="What are the differences between Proximal Policy Optimization versions PPO1 and PPO2?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;reference-request&gt;&lt;proximal-policy-optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25138" PostTypeId="1" CreationDate="2020-12-12T15:11:45.723" Score="0" ViewCount="44" Body="&lt;p&gt;I've read in &lt;a href=&quot;https://stackoverflow.com/q/5215856/3924118&quot;&gt;this&lt;/a&gt; discussion that &amp;quot;reinforcement learning is a way of finding the value function of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process&lt;/a&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I want to implement an RL model, whose state space and action space dimensions would increase, as the MDP progresses. But I don't know how to define it it terms of e.g. Q-learning or some similar method.&lt;/p&gt;&#xA;&lt;p&gt;Precisely, I want to create a model, that would generate &lt;a href=&quot;https://en.wikipedia.org/wiki/Boolean_circuit&quot; rel=&quot;nofollow noreferrer&quot;&gt;boolean circuits&lt;/a&gt;. At each step, it could perform four different actions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;apply &lt;span class=&quot;math-container&quot;&gt;$AND$&lt;/span&gt; gate on two wires,&lt;/li&gt;&#xA;&lt;li&gt;apply &lt;span class=&quot;math-container&quot;&gt;$OR$&lt;/span&gt; gate on two wires,&lt;/li&gt;&#xA;&lt;li&gt;apply &lt;span class=&quot;math-container&quot;&gt;$NOT$&lt;/span&gt; gate on one wire,&lt;/li&gt;&#xA;&lt;li&gt;add new wire.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Each of the first three actions could be performed on any currently available wires (targets). Also, the number of wires will change over time. It might increase if we perform fourth action, or decrese after e.g. application of an &lt;span class=&quot;math-container&quot;&gt;$AND$&lt;/span&gt; gate (taking as input two wires and outputting just one).&lt;/p&gt;&#xA;" OwnerUserId="17411" LastEditorUserId="17411" LastEditDate="2020-12-13T12:04:09.833" LastActivityDate="2020-12-13T12:04:09.833" Title="How to implement RL model with increasing dimensions of state space and action space?" Tags="&lt;reinforcement-learning&gt;&lt;action-spaces&gt;&lt;discrete-action-spaces&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="25139" PostTypeId="1" CreationDate="2020-12-12T15:11:51.977" Score="1" ViewCount="12" Body="&lt;p&gt;if I make an application for movies and each user in the system can rate the movies. And I want to make a recommendation system to recommend movies to active user based on his rating for other movies. using item based collaborative filtering using KNN. &lt;br&gt;&#xA;when we find the similarities between the movies and pick the top k items, which approach is correct? &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;1- calculate the similarities between all movies and then take the top k for every movie the user rated it highly. (the dataset is a matrix represent the rating values for each item from each user)&lt;/p&gt;&#xA;&lt;p&gt;2- The KNN is applied to all the movies that the user likes, one after the other, and we find the similarity between each movie and the films that the user did not rate, so that for each film we take the top K of similar films (the user not rated yet) for each movie the user rated highly, then show it to the user . (the dataset for each time we apply knn is a matrix contain rating for each item the user rated and all other items that the user not rated yet).&lt;/p&gt;&#xA;" OwnerUserId="42879" LastEditorUserId="42879" LastEditDate="2020-12-12T15:21:40.473" LastActivityDate="2020-12-12T15:21:40.473" Title="what is the correct approach for KNN in item based recommendation system?" Tags="&lt;machine-learning&gt;&lt;recommender-system&gt;&lt;k-nearest-neighbors&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25140" PostTypeId="1" CreationDate="2020-12-12T15:28:56.507" Score="0" ViewCount="14" Body="&lt;p&gt;Let's say, that we have a simple neural network as the one depicted below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r5eyx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r5eyx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The goal of this NN is to classify inputs into &lt;strong&gt;two&lt;/strong&gt; separate categories. Usualy, each neuron would correspond to one category, and we would pick the correct category given by the neuron, which outputs the largest value.&lt;/p&gt;&#xA;&lt;p&gt;But let's say, that now we want to use the same architecture, but to classify inputs into &lt;strong&gt;four&lt;/strong&gt; categories. We could encode these categories as &lt;span class=&quot;math-container&quot;&gt;$00$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$01$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$11$&lt;/span&gt;. Now, to obtain a &amp;quot;code&amp;quot; for each category, we could round output of each neuron to 0, if it's smaller than 0.5, and to 1, if it's greater or equal to 0.5.&lt;/p&gt;&#xA;&lt;p&gt;If we denote outputs of neurons &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in the last layer as &lt;span class=&quot;math-container&quot;&gt;$O_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$O_1$&lt;/span&gt;, we could get the following example predictions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ O_0 = 0.2, O_1 = 0.9 \rightarrow 01 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ O_0 = 0.3, O_1 = 0.4 \rightarrow 00 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ O_0 = 0.6, O_1 = 0.75 \rightarrow 11 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does this scheme has any potential of working correctly?&lt;/p&gt;&#xA;" OwnerUserId="17411" LastActivityDate="2020-12-12T15:28:56.507" Title="Can we encode categories binarily in a classification task using neurons outputs as consecutive bits?" Tags="&lt;neural-networks&gt;&lt;classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25141" PostTypeId="1" CreationDate="2020-12-12T18:31:13.077" Score="0" ViewCount="14" Body="&lt;p&gt;Currently I want to train a deep learning model on finger detection (drawing bounding boxes around fingers), so I haven't found an annotated dataset, so I decided to make it myself. but the number of samples are going to be huge, so I need a self annotating mechanism, that will help me to annotate images. what can you suggest?&lt;/p&gt;&#xA;" OwnerUserId="36107" LastActivityDate="2020-12-12T18:31:13.077" Title="Are there any good self annotating methods in machine learning" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="25142" PostTypeId="2" ParentId="25099" CreationDate="2020-12-12T18:48:51.947" Score="1" Body="&lt;p&gt;CNNs work by applying filters over the entire image. The same filter is applied at every pixel in the image. That is, the same weights are used at every pixel.&lt;/p&gt;&#xA;&lt;p&gt;Note, when I say &amp;quot;at every pixel&amp;quot; this means across the spatial dimension HxW of the image. You can also have attention in the channel dimension. See for example Squeeze and Excitation: &lt;a href=&quot;https://arxiv.org/pdf/1709.01507.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1709.01507.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;While this is one of the strengths of CNNs, since it drastically reduces the number of parameters of a network, you can imagine that it may not make sense to treat every part of the image the same regardless of the content. This is what the attention gate is for.&lt;/p&gt;&#xA;&lt;p&gt;By performing an element-wise multiplication of the output of a CNN layer with a gate tensor (typically clamped to the 0-1 range with a sigmoid) we can effectively down weigh or ignore features in certain areas of the image.&lt;/p&gt;&#xA;&lt;p&gt;CNNs without AG can typically learn the same things but may need more channels and layers. Attention gates allow to treat filters differently depending on the content and hence make it easier to learn with less filters.&lt;/p&gt;&#xA;&lt;p&gt;What you use as the attention tensor can vary. In the paper you referenced, the tensor was one of the input tensors of the convolutional layer. In this case, it is called self-attention. However, with attention in general this gate tensor may also come from other information such as another network.&lt;/p&gt;&#xA;" OwnerUserId="42911" LastActivityDate="2020-12-12T18:48:51.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25143" PostTypeId="2" ParentId="6231" CreationDate="2020-12-12T20:41:19.403" Score="0" Body="&lt;p&gt;Okay, so instead of telling you to just not have recurrent connections, i'm actually going to tell you how to identify them.&lt;/p&gt;&#xA;&lt;p&gt;First thing you need to know is that recurrent connections are calculated &lt;strong&gt;after&lt;/strong&gt; all other connections &lt;em&gt;and&lt;/em&gt; neurons. So which connection is recurrent and which is not depends on the &lt;strong&gt;order of calculation&lt;/strong&gt; of your NN.&#xA;Also, the first time when you put data into the system, we'll just assume that every connection is zero, otherwise some or all neurons can't be calculated.&lt;/p&gt;&#xA;&lt;p&gt;Lets say we have this neural network:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TKiZF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Network&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We devide this network into 3 layers (even though conceptually it has 4 layers):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Input Layer  [1, 2]&#xA;Hidden Layer [5, 6, 7]&#xA;Output Layer [3, 4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First rule: &lt;strong&gt;All outputs from the output layer are recurrent connections&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Second rule: &lt;strong&gt;All outputs from the input layer may be calculated first&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We create two arrays. One containing the order of calculation of all neurons &lt;em&gt;and&lt;/em&gt; connections and one containing all the (potentially) recurrent connections.&#xA;Right now these arrays look somewhat like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Order of &#xA;calculation: [1-&amp;gt;5, 2-&amp;gt;7 ]&#xA;&#xA;Recurrent:   [ ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now we begin by looking at the output layer. Can we calculate Neuron 3? No? Because 6 is missing. Can we calculate 6? No? Because 5 is missing. And so on. It looks somewhat like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3, 6, 5, 7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem is that we are now stuck in a loop. So we introduce a temporary array storing all the neuron id's that we already visited:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[3, 6, 5, 7]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now we ask: Can we calculate 7? No, because 6 is missing. But we already visited 6...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[3, 6, 5, 7,] &amp;lt;- 6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Third rule is: &lt;strong&gt;When you visit a neuron that has already been visited before, set the connection that you followed to this neuron as a recurrent connection&lt;/strong&gt;.&#xA;Now your arrays look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Order of &#xA;calculation: [1-&amp;gt;5, 2-&amp;gt;7 ]&#xA;&#xA;Recurrent:   [6-&amp;gt;7 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you finish the process and in the end join the order of calculation array with your recurrent array so, that the recurrent array follows after the other array.&#xA;It looks somethat like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1-&amp;gt;5, 2-&amp;gt;7, 7, 7-&amp;gt;4, 7-&amp;gt;5, 5, 5-&amp;gt;6, 6, 6-&amp;gt;3, 3, 4, 6-&amp;gt;7]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Let's assume we have [x-&amp;gt;y, y]&lt;/p&gt;&#xA;&lt;p&gt;Where x-&amp;gt;y is the calculation of x*weight(x-&amp;gt;y)&lt;/p&gt;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&lt;p&gt;Where y is the calculation of Sum(of inputs to y). So in this case Sum(x-&amp;gt;y) or just x-&amp;gt;y.&lt;/p&gt;&#xA;&lt;p&gt;There are still some problems to solve here. For example: What if the only input of a neuron is a recurrent connection? But i guess you'll be able to solve this problem on your own...&lt;/p&gt;&#xA;" OwnerUserId="42913" LastEditorUserId="42913" LastEditDate="2020-12-13T13:50:58.540" LastActivityDate="2020-12-13T13:50:58.540" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25144" PostTypeId="1" CreationDate="2020-12-12T20:57:02.893" Score="0" ViewCount="105" Body="&lt;p&gt;I ran into a 2019-Entrance Exam question as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Z8Dd3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z8Dd3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The answer mentioned is (4), but some search on google showed me maybe (1) and (2) is equal to (4). Why would k-means be the algorithm with the highest bias? (Can you please also provide references to valid material to study more?)&lt;/p&gt;&#xA;" OwnerUserId="42854" LastEditorUserId="2444" LastEditDate="2020-12-12T21:33:26.407" LastActivityDate="2021-05-11T23:04:05.363" Title="Why does k-means have more bias than spectral clustering and GMM?" Tags="&lt;machine-learning&gt;&lt;k-means&gt;&lt;bias-variance-tradeoff&gt;&lt;spectral-clustering&gt;&lt;gaussian-mixture-models&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25145" PostTypeId="2" ParentId="25144" CreationDate="2020-12-12T21:28:56.050" Score="0" Body="&lt;p&gt;I'm not an expert on clustering, but here's my take below. Note that this is only based on theoretical arguments, I haven't had enough clustering experience to say if this is generally true in practice.&lt;/p&gt;&#xA;&lt;h2&gt;K-means vs GMM&lt;/h2&gt;&#xA;&lt;p&gt;K-means has a higher bias than GMM because it is a special case of GMM. K-means specifically assumes the clustering is spherical (meaning each dimension is weighted equally important) and that the clustering problem is a hard clustering problem (each data point can only belong to one label). So, theoretically, K-means should perform equal to GMM (under very specific conditions) or worse. &lt;a href=&quot;https://stats.stackexchange.com/a/269619/303597&quot;&gt;More info&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;K-means vs GMM (identity covariance matrix)&lt;/h2&gt;&#xA;&lt;p&gt;K-means has a higher bias than GMM (identity covariance matrix) because it is also a special case. K-mean specifically assumes the hard clustering problem, but GMM does not. Because of this, GMM has stronger estimates for the mean of the centroids. More specifically,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[GMM] estimates the cluster means as weighted means, not assigning observations in a crisp manner to one of the clusters. In this way it avoids the problem explained above and it will be consistent as ML estimator (in general this is problematic because of issues of degeneration of the covariance matrix, however not if you assume them spherical and equal).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In practice, if you generate observations from a number of Gaussians with same spherical covariance matrix and different means, K-means will therefore overestimate the distances between the means, whereas the ML-estimator for the mixture model will not.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, theoretically, K-means should perform equal to GMM (identity covariance matrix) or worse. &lt;a href=&quot;https://stats.stackexchange.com/a/489615/303597&quot;&gt;More info&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;K-means vs Spectral clustering&lt;/h2&gt;&#xA;&lt;p&gt;K-means has a higher bias then spectral clustering because spectral clustering effectively uses K-means after processing more information from the matrices.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Spectral clustering usually is spectral embedding, followed by k-means in the spectral domain.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So yes, it also uses k-means. But not on the original coordinates, but on an embedding that roughly captures connectivity. Instead of minimizing squared errors in the input domain, it minimizes squared errors on the ability to reconstruct neighbors. That is often better.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/a/36387/108095&quot;&gt;More info&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-12T21:28:56.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25146" PostTypeId="5" CreationDate="2020-12-12T21:37:53.503" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T21:37:53.503" LastActivityDate="2020-12-12T21:37:53.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25147" PostTypeId="4" CreationDate="2020-12-12T21:37:53.503" Score="0" Body="For questions about spectral clustering techniques, which make use of the spectrum (eigenvalues) of the similarity matrix of the data to perform dimensionality reduction before clustering in fewer dimensions." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-12T21:37:53.503" LastActivityDate="2020-12-12T21:37:53.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25148" PostTypeId="1" AcceptedAnswerId="25149" CreationDate="2020-12-12T22:22:23.537" Score="3" ViewCount="124" Body="&lt;p&gt;I have a difficult time understanding the &amp;quot;multi-head&amp;quot; notion in the original &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer paper&lt;/a&gt;. What makes the learning in each head unique? Why doesn't the neural network learn the same set of parameters for each attention head? Is it because we break &lt;em&gt;query, key&lt;/em&gt; and &lt;em&gt;value&lt;/em&gt; vectors into smaller dimensions and feed each portion to a different head?&lt;/p&gt;&#xA;" OwnerUserId="15498" LastEditorUserId="2444" LastEditDate="2020-12-12T22:54:52.387" LastActivityDate="2021-03-17T12:19:30.717" Title="What is different in each head of a multi-head attention mechanism?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;papers&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25149" PostTypeId="2" ParentId="25148" CreationDate="2020-12-12T22:39:41.727" Score="2" Body="&lt;p&gt;The reason each head is different is because they each learn a different set of weight matrices &lt;span class=&quot;math-container&quot;&gt;$\{ W_i^Q, W_i^K, W_i^V \}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the index of the head. To clarify, the input to each attention head is the same. For attention head &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q_i(x) = x W_i^Q \\&#xA;K_i(x) = x W_i^K \\&#xA;V_i(x) = x W_i^V \\&#xA;attention_i(x) = softmax \left(\frac{Q_i(x) K_i(x)^T}{\sqrt{d_k}} \right) V_i(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Notice that the input to each head is &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (either the semantic + positional embedding of the decoder input for the first decoder layer, or the output of the previous decoder layer). &lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;More info&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The question as to why gradient descent learns each set of weight matrices &lt;span class=&quot;math-container&quot;&gt;$\{ W_i^Q, W_i^K, W_i^V \}$&lt;/span&gt; to be different across each attention head is very similar to &lt;a href=&quot;https://ai.stackexchange.com/questions/25109/is-there-anything-that-ensures-that-convolutional-filters-end-up-the-same/25111#25111&quot;&gt;&amp;quot;Is there anything that ensures that convolutional filters end up the same?&amp;quot;&lt;/a&gt;, so maybe you might find the answer there helpful for you:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;No, nothing really prevents the weights from being different. In practice though they end up almost always different because it makes the model more expressive (i.e. more powerful), so gradient descent learns to do that. If a model has n features, but 2 of them are the same, then the model effectively has n−1 features, which is a less expressive model than that of n features, and therefore usually has a larger loss function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-13T00:51:17.630" LastActivityDate="2020-12-13T00:51:17.630" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25150" PostTypeId="1" AcceptedAnswerId="25190" CreationDate="2020-12-13T01:31:44.467" Score="0" ViewCount="65" Body="&lt;p&gt;I know convolutional neural networks are commonly used for image recognition, but I was wondering if they would be able to distinguish between predominantly text-based documents vs something like objects. For example, if you trained using images of the first page of invoices matched to a vendor name, could you get a CNN to predict the vendor based on an image? If not, is there a different AI technique better suited that is purely image-based, or would it require OCR and leveraging the text in the invoice?&lt;/p&gt;&#xA;&lt;p&gt;Update: based on a comment, my ask my not be clear. I'm not trying to see if the CNN can differentiate between a document (mostly text based image) and a photo image. I want to know if based on a gif/jpeg/png of a document (no OCR performed) a CNN would be able to classify the documents, which basically could be used as a means of identifying the vendor.&lt;/p&gt;&#xA;" OwnerUserId="42916" LastEditorUserId="42916" LastEditDate="2020-12-14T14:19:09.513" LastActivityDate="2020-12-14T20:42:41.010" Title="Can a convolutional neural network classify text document images?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25151" PostTypeId="2" ParentId="25131" CreationDate="2020-12-13T02:40:43.747" Score="2" Body="&lt;p&gt;@nbro pointed out the paper &lt;a href=&quot;https://arxiv.org/pdf/1710.05381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A systematic study of the class imbalance problem in convolutional neural networks&lt;/a&gt;, which tested class imbalance LeNet for MNIST, on a custom CNN for CIFAR-10, and on ResNet for ImageNet. The paper found that by artificially creating class imbalance on those data sets, the neural networks are significantly deteriorated. The ROC AUC drops by 5-10%, and accuracy decreases by 20-30%. These effects are worsened on more complex tasks.&lt;/p&gt;&#xA;&lt;p&gt;There are 3 noteworthy class imbalance approaches to partially alleviate this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Undersampling&lt;/strong&gt;: sampling the over-represented class less often&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Oversampling&lt;/strong&gt;: sampling the under-represented class more often&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Thresholding&lt;/strong&gt;: after the neural network learns the weights, during inference, multiply the output class probability by a weight (the class prior, which is different for each class). The weight is the inverse of the class representation of the dataset (i.e. the inverse of &lt;span class=&quot;math-container&quot;&gt;$\frac{numInstances(c)}{\displaystyle \sum_i numInstances(i)}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is the current class and &lt;span class=&quot;math-container&quot;&gt;$numInstances(i)$&lt;/span&gt; is the number of unique instances of class &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in the training set.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The paper concludes with the following best practices:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Regarding the choice of a method to handle CNN training on imbalanced dataset we conclude the following.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The method that in most of the cases outperforms all others with respect to multi-class ROC AUC was oversampling.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For extreme ratio of imbalance and large portion of classes being minority, undersampling performs on a par with oversampling. If training time is an issue, undersampling is a better choice in such a scenario since it dramatically reduces the size of the training set&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To achieve the best accuracy, one should apply thresholding to compensate for prior class probabilities. &lt;strong&gt;A combination of thresholding with baseline and oversampling is the most preferable&lt;/strong&gt;, whereas it should not be combined with undersampling.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance. The higher a fraction of minority classes in the imbalanced training set, the more imbalance ratio should be reduced.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Oversampling does not cause overfitting of convolutional neural networks, as opposed to some classical machine learning models.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The last point is very interesting, because oversampling is known to cause overfitting in classical machine learning models and many have advised against doing it.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="2444" LastEditDate="2020-12-17T00:40:58.557" LastActivityDate="2020-12-17T00:40:58.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25152" PostTypeId="1" AcceptedAnswerId="25475" CreationDate="2020-12-13T05:56:36.010" Score="1" ViewCount="94" Body="&lt;p&gt;I know this question may be so silly, but I can not prove it.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;http://cs231n.stanford.edu/slides/2020/lecture_8.pdf#page=17&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford slide&lt;/a&gt; (page 17), they define the formula of SGD with momentum like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{t}=\rho v_{t-1}+\nabla f(x_{t-1}) &#xA;\\ &#xA;x_{t}=x_{t-1}-\alpha v_{t},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$v_{t+1}$&lt;/span&gt; is the momentum value&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is a friction, let say it's equal 0.9&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla f(x_{t-1})$&lt;/span&gt; is the gradient of the objective function at iteration &lt;span class=&quot;math-container&quot;&gt;$t-1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_t$&lt;/span&gt; are the parameters&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the learning rate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, in &lt;a href=&quot;https://arxiv.org/pdf/1609.04747.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and many other documents, they define the equation like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{t}=\rho v_{t-1}+\alpha \nabla f(x_{t-1}) &#xA;\\ &#xA;x_{t}=x_{t-1}- v_{t},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; still have the same value as in the previous formula.&lt;/p&gt;&#xA;&lt;p&gt;I think it should be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{t}=\alpha \rho v_{t-1}+\alpha \nabla f(x_{t-1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;if we want to multiply the learning rate inside the equation.&lt;/p&gt;&#xA;&lt;p&gt;In some other document (&lt;a href=&quot;https://towardsdatascience.com/stochastic-gradient-descent-with-momentum-a84097641a5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;) or normal form of momentum, they define like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{t}= \rho v_{t-1}+ (1- \rho) \nabla f(x_{t-1}) &#xA;\\ &#xA;x_{t}=x_{t-1}-\alpha v_{t}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can not understand how can they prove those equations are similar. Can someone help me?&lt;/p&gt;&#xA;" OwnerUserId="41287" LastEditorUserId="41287" LastEditDate="2020-12-31T01:42:08.863" LastActivityDate="2020-12-31T06:33:00.633" Title="How are these equations of SGD with momentum equivalent?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;optimization&gt;&lt;stochastic-gradient-descent&gt;&lt;momentum&gt;" AnswerCount="1" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25153" PostTypeId="1" AcceptedAnswerId="25184" CreationDate="2020-12-13T06:05:34.123" Score="1" ViewCount="59" Body="&lt;p&gt;I made a simple feedforward neural network (FFNN) to predict &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\sin(x)$&lt;/span&gt;. It failed. Does it mean the model has overfitted? Why doesn't it work?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XKBbB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XKBbB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(1234567890)&#xA;Var3 &amp;lt;- runif(500, 0, 20)&#xA;mydata3 &amp;lt;- data.frame(Sin=sin(Var3),Var=Var3)&#xA;set.seed(1234567890)&#xA;winit &amp;lt;- runif(5500, -1, 1)&#xA;#hidUnit &amp;lt;- c(9,1)&#xA;set.seed(1234567890)&#xA;nn3 &amp;lt;-neuralnet(formula = Var~Sin,data = mydata3,&#xA;                hidden =c(4,2,1),startweights =winit,&#xA;              learningrate = 0.01,act.fct = &amp;quot;tanh&amp;quot;)&#xA;&#xA;plot(mydata3, cex=2,main='Predicting x from Sin(x)',&#xA;     pch = 21,bg=&amp;quot;darkgrey&amp;quot;,&#xA;     ylab=&amp;quot;X&amp;quot;,xlab=&amp;quot;Sin(X)&amp;quot;)&#xA;points(mydata3[,1],predict(nn3,mydata3), col=&amp;quot;darkred&amp;quot;, &#xA;       cex=1,pch=21,bg=&amp;quot;red&amp;quot;)&#xA;&#xA;legend(&amp;quot;bottomleft&amp;quot;, legend=c(&amp;quot;true&amp;quot;,&amp;quot;predicted&amp;quot;), pch=c(21,21),&#xA;       col = c(&amp;quot;darkgrey&amp;quot;,&amp;quot;red&amp;quot;),cex = 0.65,bty = &amp;quot;n&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="42920" LastEditorUserId="42920" LastEditDate="2020-12-13T23:43:42.433" LastActivityDate="2020-12-14T10:10:23.320" Title="Why does my neural network to predict $x$ given $\sin(x)$ not generalize?" Tags="&lt;r&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25154" PostTypeId="2" ParentId="5625" CreationDate="2020-12-13T06:27:15.570" Score="0" Body="&lt;p&gt;No, not all fully observable environments are episodic. Let's take a look again at the definitions from the book:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Fully Observable Environment&lt;/strong&gt; (&lt;a href=&quot;https://cs.calvin.edu/courses/cs/344/kvlinden/resources/AIMA-3rd-edition.pdf#page=61&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 2.3.2&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If an agent’s sensors give it access to the complete state of the environment at each point in time, then we say that the task environment is fully observable. A task environment is effectively fully observable if the sensors detect all aspects that are relevant to the choice of action&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Episodic Environment&lt;/strong&gt; (&lt;a href=&quot;https://cs.calvin.edu/courses/cs/344/kvlinden/resources/AIMA-3rd-edition.pdf#page=61&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 2.3.2&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In an episodic task environment, the agent’s experience is divided into atomic episodes. In each episode the agent receives a percept and then performs a single action. Crucially, the next episode does not depend on the actions taken in previous episodes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Take note of the &amp;quot;crucial&amp;quot; part at the end of the definition of episodic environment. A fully observable environment that is not episodic (and therefore &lt;em&gt;sequential&lt;/em&gt; in the book's taxonomy) is chess. Chess is fully observable because the player can view the positions of all active pieces on the chess board, and that is all the information that needs to be known in order to take the optimal action. But chess is &lt;strong&gt;not&lt;/strong&gt; episodic, because the player's current move depends on all previous moves, and the current move will have downstream effects in later turns.&lt;/p&gt;&#xA;&lt;p&gt;In fact, if you look at Figure 2.6 in the book on pg. 45, they provide three examples of fully observable sequential (i.e. not episodic) environments: crossword puzzles, chess, and backgammon. There are of course many more. Most games are sequential as that is the main appeal of them - how to best sequence my moves now in order to ensure victory over my opponent at a future time?&lt;/p&gt;&#xA;" OwnerUserId="20955" LastEditorUserId="2444" LastEditDate="2021-01-24T00:37:21.487" LastActivityDate="2021-01-24T00:37:21.487" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25155" PostTypeId="1" CreationDate="2020-12-13T07:43:21.643" Score="1" ViewCount="25" Body="&lt;p&gt;It may already be obvious that I am just a practitioner and just a beginner to Deep Learning. I am still figuring out lots of &amp;quot;WHY&amp;quot;s and &amp;quot;HOW&amp;quot;s of DL.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, if I train a feed-forward neural network, or an image classifier with CNNs, or just an OCR problem with GRUs, using something like Keras, and it performs very poorly or takes more time to train than it should be, it may be because of the gradients getting vanished or exploding, or some other problem.&lt;/p&gt;&#xA;&lt;p&gt;But, if it is due to the gradients getting very small or very big during the training, how do I figure that out? Doing what will I able to infer that something has happened due to the gradient values?&lt;/p&gt;&#xA;&lt;p&gt;And what are the precautions I should take to avoid it from the beginning (since training DL models with accelerated computing costs money) and if it has happened, how do I fix it?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This question may sound like a duplicate of &lt;a href=&quot;https://ai.stackexchange.com/q/24153/38060&quot;&gt;How to decide if gradients are vanishing?&lt;/a&gt;, but actually not, since that question focuses on CNNs, while I am asking about problem with gradients in all kinds of deep learning algorithms.&lt;/p&gt;&#xA;" OwnerUserId="38060" LastEditorUserId="38060" LastEditDate="2020-12-13T13:02:25.503" LastActivityDate="2020-12-13T13:02:25.503" Title="How do I infer exploding or vanishing gradients in Keras?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;backpropagation&gt;&lt;vanishing-gradient-problem&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25156" PostTypeId="1" CreationDate="2020-12-13T09:11:06.263" Score="0" ViewCount="9" Body="&lt;p&gt;These systems are discrete and their state changes are rule based. Example:&lt;/p&gt;&#xA;&lt;p&gt;Given a chess position, generate a series of moves that will lead to it (there may be many, one, or none, but I only need one valid one)&lt;/p&gt;&#xA;&lt;p&gt;If there are many examples, (end state, state change sequence) pairs, is there a neural network or other machine learning algorithm that can generate the latter given the former in a way that extrapolates to previously unseen board positions?&lt;/p&gt;&#xA;&lt;p&gt;In my use cases, it will be possible to find or generate large datasets for both.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;chess end positions and moves, billions of pairs (e.g. from &lt;a href=&quot;https://database.lichess.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;mathematical proofs, 35k+ pairs (see the last line &lt;a href=&quot;https://gitlab.com/void4/mmannotate/-/raw/master/prop.mm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, the second part is a sequence of tokens/references that proves the first statement)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42922" LastActivityDate="2020-12-13T09:11:06.263" Title="Given a system state, generate a sequence of state changes that lead to it" Tags="&lt;deep-learning&gt;&lt;algorithm&gt;&lt;generative-model&gt;&lt;chess&gt;&lt;discrete-state-spaces&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25157" PostTypeId="1" CreationDate="2020-12-13T11:15:05.853" Score="3" ViewCount="36" Body="&lt;p&gt;I have been reading &lt;a href=&quot;https://www.tensorflow.org/guide/keras/transfer_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;this TensorFlow tutorial on transfer learning&lt;/a&gt;, where they unfroze the whole model and then they say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When you unfreeze a model that contains &lt;code&gt;BatchNormalization&lt;/code&gt; layers in order to do fine-tuning, you should keep the &lt;code&gt;BatchNormalization&lt;/code&gt; layers in inference mode by passing &lt;code&gt;training=False&lt;/code&gt; when calling the base model. Otherwise the updates applied to the non-trainable weights will suddenly destroy what the model has learned.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My question is: why? The model's weights are adapting to the new data, so why do we keep the old mean and variance, which was calculated on ImageNet? This is very confusing.&lt;/p&gt;&#xA;" OwnerUserId="36107" LastEditorUserId="2444" LastEditDate="2020-12-13T13:06:42.253" LastActivityDate="2020-12-13T13:06:42.253" Title="Why shouldn't batch normalisation layers be learnable during fine-tuning?" Tags="&lt;convolutional-neural-networks&gt;&lt;tensorflow&gt;&lt;transfer-learning&gt;&lt;batch-normalization&gt;&lt;fine-tuning&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25158" PostTypeId="1" AcceptedAnswerId="25164" CreationDate="2020-12-13T12:33:43.243" Score="1" ViewCount="284" Body="&lt;p&gt;I am reading college notes on state search space. The notes (which are not publicly available) say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;To do state-search space, the strategy involves two parts: defining a heuristic function, and identifying an evaluation function.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The heuristic is a smart search of the available space. The evaluation function may be well-defined (e.g. the solution solves a problem and receives a score) or may itself be the heuristic (e.g. if chess says pick A or B as the next move and picks A, the evaluation function is the heuristic).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Understand the difference between the heuristic search algorithm and the heuristic evaluation function.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm trying step 3 (to understand). Can I check, using the A* search as an example, that the:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Heuristic function:&lt;/strong&gt; estimated cost from the current node to the goal, i.e. it's a heuristic that's calculating the simplest way to get to the goal (in A*; &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt;), so the heuristic function is calculating &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; for a series of options and picking the best one.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Evaluation function:&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="2444" LastEditDate="2021-02-06T18:33:58.283" LastActivityDate="2021-02-06T18:33:58.283" Title="What is the difference between the heuristic function and the evaluation function in A*?" Tags="&lt;comparison&gt;&lt;search&gt;&lt;a-star&gt;&lt;heuristic-functions&gt;&lt;evaluation-functions&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25159" PostTypeId="5" CreationDate="2020-12-13T12:39:42.297" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T12:39:42.297" LastActivityDate="2020-12-13T12:39:42.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25160" PostTypeId="4" CreationDate="2020-12-13T12:39:42.297" Score="0" Body="For questions about the concept of momentum in the context of gradient descent." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T12:39:42.297" LastActivityDate="2020-12-13T12:39:42.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25161" PostTypeId="5" CreationDate="2020-12-13T12:42:53.383" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T12:42:53.383" LastActivityDate="2020-12-13T12:42:53.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25162" PostTypeId="4" CreationDate="2020-12-13T12:42:53.383" Score="0" Body="For questions about optimization methods/algorithms (also know as optimizers) in the context of machine learning and other AI subfields. Examples of optimizers are plain (stochastic) gradient descent, Adam, SGD with momentum, Adagrad, and RMSprop." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T12:42:53.383" LastActivityDate="2020-12-13T12:42:53.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25164" PostTypeId="2" ParentId="25158" CreationDate="2020-12-13T14:03:56.937" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is the difference between the heuristic function and the evaluation function in A*?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;strong&gt;evaluation function&lt;/strong&gt;, often denoted as &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, is the function that you use to choose which node to expand during one iteration of A* (i.e. decide which node to take from the frontier, determine the next possible actions and which next nodes those actions lead to, and add those nodes to the frontier). Typically, you expand the node &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$f(n)$&lt;/span&gt; is the smallest, i.e. &lt;span class=&quot;math-container&quot;&gt;$n^* = \operatorname{argmin}f(n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the case of &lt;em&gt;informed&lt;/em&gt; search algorithms (such as A*), the &lt;strong&gt;heuristic function&lt;/strong&gt; is a component of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, which can be written as &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n) + h(n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h(n)$&lt;/span&gt; is the heuristic function. The heuristic function estimates the cost of the cheapest path from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to the goal. Just for completeness, &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; is the actual cost from the start node to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (which can be computed exactly during the search). In the case of uninformed search algorithms, you can actually view the evaluation function as just &lt;span class=&quot;math-container&quot;&gt;$f(n) = g(n)$&lt;/span&gt;, i.e. the heuristic function is always zero.&lt;/p&gt;&#xA;&lt;p&gt;So, you're right.&lt;/p&gt;&#xA;&lt;p&gt;For more details, you can take a look at section 3.5 (p. 92) of the book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; (3rd edition), by Norvig and Russell (you can find the pdf online).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T14:09:05.990" LastActivityDate="2020-12-13T14:09:05.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25165" PostTypeId="1" AcceptedAnswerId="25191" CreationDate="2020-12-13T14:45:48.650" Score="1" ViewCount="163" Body="&lt;p&gt;In &lt;a href=&quot;https://www.cs.cmu.edu/%7Eaarti/Class/10701/exams/midterm2010f_sol.pdf#page=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;these notes&lt;/a&gt;, we have the following statement&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The depth of a learned decision tree can be larger than the number of training examples used to create the tree&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This statement is false, according to the same &lt;a href=&quot;https://www.cs.cmu.edu/%7Eaarti/Class/10701/exams/midterm2010f_sol.pdf#page=3&quot; rel=&quot;nofollow noreferrer&quot;&gt;notes&lt;/a&gt;, where it is written&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;False&lt;/strong&gt;: Each split of the tree must correspond to at least one training example, therefore, if there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; training examples, a path in the tree can have length at most &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: There is a pathological situation in which the depth of a learned decision tree can be larger than number of training examples &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; - if the number of features is larger than &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and there exist training examples which have same feature values but different labels.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I had written on my notes that the depth of a decision tree only depends on the number of features of the training set and not on the number of training samples. So, what does the depth of the decision tree depend on?&lt;/p&gt;&#xA;" OwnerUserId="42928" LastEditorUserId="2444" LastEditDate="2020-12-13T20:39:07.183" LastActivityDate="2021-01-13T22:04:20.367" Title="What does the depth of a decision tree depend on?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;decision-tree&gt;&lt;features&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25166" PostTypeId="1" AcceptedAnswerId="25167" CreationDate="2020-12-13T16:20:36.710" Score="3" ViewCount="113" Body="&lt;p&gt;I want to determine some distance between two policies &lt;span class=&quot;math-container&quot;&gt;$\pi_1 (a \mid s)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2 (a \mid s)$&lt;/span&gt;, i.e. something like &lt;span class=&quot;math-container&quot;&gt;$\vert \vert \pi_1 (a \mid s) - \pi_2(a \mid s) \vert \vert$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi_i (a\mid s)$&lt;/span&gt; is the vector &lt;span class=&quot;math-container&quot;&gt;$(\pi_i (a_1 \mid s), \dots, \pi_i(a_n \mid s))$&lt;/span&gt;. I am looking for a sensible notion for such a distance.&lt;/p&gt;&#xA;&lt;p&gt;Are there some standard norms/metrics used in the literature for determining a distance between policies?&lt;/p&gt;&#xA;" OwnerUserId="36978" LastEditorUserId="2444" LastEditDate="2020-12-13T21:03:56.740" LastActivityDate="2020-12-13T21:03:56.740" Title="Are there some notions of distance between two policies?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;policies&gt;&lt;kl-divergence&gt;&lt;wasserstein-metric&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25167" PostTypeId="2" ParentId="25166" CreationDate="2020-12-13T17:01:51.387" Score="5" Body="&lt;p&gt;Given that policies are probability distributions, in principle, you can use any metric or measure of distance that can be used to compare two probability distributions. (Note that notions of distance are not necessarily &lt;a href=&quot;https://en.wikipedia.org/wiki/Metric_(mathematics)&quot; rel=&quot;noreferrer&quot;&gt;metrics&lt;/a&gt; in a mathematical sense).&lt;/p&gt;&#xA;&lt;p&gt;A common measure is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Relative_entropy&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Kullback–Leibler divergence&lt;/strong&gt;&lt;/a&gt; (which is not a metric, in a mathematical sense, given that it does not satisfy certain required conditions for being a metric). For example, in section 4 of &lt;a href=&quot;https://arxiv.org/pdf/1707.06347.pdf&quot; rel=&quot;noreferrer&quot;&gt;the PPO paper&lt;/a&gt;, the KL divergence is used as a regulariser (which is actually quite common, for instance, in the context of variational Bayesian neural networks). The &lt;a href=&quot;http://proceedings.mlr.press/v37/schulman15.pdf&quot; rel=&quot;noreferrer&quot;&gt;TRPO&lt;/a&gt; also uses the KL divergence.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Wasserstein_metric&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Wasserstein metric&lt;/strong&gt;&lt;/a&gt; has also been used in RL, for instance, in &lt;a href=&quot;https://arxiv.org/pdf/1707.06887.pdf&quot; rel=&quot;noreferrer&quot;&gt;distributional RL&lt;/a&gt; (but, in this case, not to compare policies but distributions over value functions).&lt;/p&gt;&#xA;&lt;p&gt;You can find more info about statistical distances &lt;a href=&quot;https://en.wikipedia.org/wiki/Statistical_distance&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. The specific distance that you use may depend on the problem that you want to solve and the properties that you want your distance to have. For example, the KL divergence is unbounded above, so, if that's not desirable, you could choose another one. The paper &lt;a href=&quot;https://arxiv.org/pdf/math/0209021.pdf&quot; rel=&quot;noreferrer&quot;&gt;On choosing and bounding probability metrics&lt;/a&gt; (2002, by Gibbs and Su) may also be useful. &lt;a href=&quot;https://ai.stackexchange.com/a/23952/2444&quot;&gt;Here&lt;/a&gt; I also talk about the KL divergence and total variation.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-12-13T17:01:51.387" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25168" PostTypeId="5" CreationDate="2020-12-13T18:03:19.560" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T18:03:19.560" LastActivityDate="2020-12-13T18:03:19.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25169" PostTypeId="4" CreationDate="2020-12-13T18:03:19.560" Score="0" Body="For questions about heuristic functions in the context of search algorithms (and other AI subfields)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T18:03:19.560" LastActivityDate="2020-12-13T18:03:19.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25170" PostTypeId="5" CreationDate="2020-12-13T18:06:13.777" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T18:06:13.777" LastActivityDate="2020-12-13T18:06:13.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25171" PostTypeId="4" CreationDate="2020-12-13T18:06:13.777" Score="0" Body="For questions about evaluation functions in the context of search algorithms (and other AI subfields)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T18:06:13.777" LastActivityDate="2020-12-13T18:06:13.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25173" PostTypeId="1" CreationDate="2020-12-13T20:25:08.270" Score="2" ViewCount="76" Body="&lt;p&gt;I have a time series sequence with 10 million steps. In step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, I have a 400 dimensional feature vector &lt;span class=&quot;math-container&quot;&gt;$X_t$&lt;/span&gt; and a scalar value &lt;span class=&quot;math-container&quot;&gt;$y_t$&lt;/span&gt; which I want to predict during inference time and I know during the train time. I want to use a &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformer model&lt;/a&gt;. I have 2 questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If I want to embed the 400 dimensional input feature vector into another space before feeding into the transformer, what are the pros and cons of using let's say 1024 and 64 for the embedding space dimension? Should I use a dimension more than 400 or less?&lt;/li&gt;&#xA;&lt;li&gt;When doing &lt;em&gt;position embedding&lt;/em&gt;, I cannot use a maximum position length of 10 million as that blows up the memory. What is the best strategy here if I want to use maximum position length of 512? Should I chunk the 10 million steps into blocks of size 512 and feed each block separately into the transformer? If so, how can I connect the subsequent blocks to take full advantage of parallelization while keeping the original chronological structure of the sequence data?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15498" LastActivityDate="2020-12-13T20:25:08.270" Title="How to handle long sequences with transformers?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;time-series&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25174" PostTypeId="5" CreationDate="2020-12-13T21:06:04.960" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T21:06:04.960" LastActivityDate="2020-12-13T21:06:04.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25175" PostTypeId="4" CreationDate="2020-12-13T21:06:04.960" Score="0" Body="For questions about the Wasserstein metric/distance (or Kantorovich–Rubinstein metric), which is a metric (distance function) defined between probability distributions." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-13T21:06:04.960" LastActivityDate="2020-12-13T21:06:04.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25176" PostTypeId="2" ParentId="25165" CreationDate="2020-12-13T22:02:24.597" Score="0" Body="&lt;p&gt;While this sounds obvious, the depth of the tree depends on how your algorithm builds the tree. For a fixed dataset &lt;span class=&quot;math-container&quot;&gt;$\mathcal{D}$&lt;/span&gt;, there're many algorithms, such as ID3, C4.5, CART, etc. (and their variants) to build your tree. For the most part, these algorithms recursively partition the dataset, so it's never possible to get a tree larger than &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{D}|$&lt;/span&gt;. Large/deep trees are also prone to overfitting and are computationally expensive, so these algorithms typically prune the tree so it's much smaller than &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{D}|$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In fact,&#xA;&lt;a href=&quot;https://www.cis.upenn.edu/%7Emkearns/papers/topdown.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kearns and Mansour showed&lt;/a&gt; that under the weak learning assumption (i.e. at each node, there's a split that classifies the data at this node better than a random classifier, by &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;) to achieve &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; training error, it suffices to make &lt;span class=&quot;math-container&quot;&gt;$(1/\epsilon)^{\mathcal{O}(\log(1/\epsilon)/\gamma^2)}$&lt;/span&gt; splits (and thus depth is upper bounded by this).&lt;/p&gt;&#xA;&lt;p&gt;But of course, you can always cook up trees of arbitrary depth...&lt;/p&gt;&#xA;" OwnerUserId="36922" LastActivityDate="2020-12-13T22:02:24.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25177" PostTypeId="1" CreationDate="2020-12-13T22:15:01.470" Score="4" ViewCount="93" Body="&lt;p&gt;I am currently studying the paper &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.170&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning and Evaluating Classifiers under Sample Selection Bias&lt;/em&gt; by Bianca Zadrozny&lt;/a&gt;. In section &lt;strong&gt;3. Learning under sample selection bias&lt;/strong&gt;, the author says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We can separate classifier learners into two categories:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;local: the output of the learner depends asymptotically only on &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;global: the output of the learner depends asymptotically both on &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; and on &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The term &amp;quot;asymptotically&amp;quot; refers to the behavior of the learner as the number of training examples grows. The names &amp;quot;local&amp;quot; and &amp;quot;global&amp;quot; were chosen because &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; is a global distribution over the entire input space, while &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; refers to many local distributions, one for each value of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Local learners are not affected by sample selection bias because, by definition &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x, s = 1) = P(y \mid x)$&lt;/span&gt; while global learners are affected because the bias changes &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then, in section &lt;strong&gt;3.1.1. Naive Bayes&lt;/strong&gt;, the author says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In practical Bayesian learning, we often make the assumption that the features are independent given the label &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, that is, we assume that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(x_1, x_2, \dots, x_n \mid y) = P(x_1 \mid y) P(x_2 \mid y) \dots P(x_n \mid y).$$&lt;/span&gt;&#xA;This is the so-called naive Bayes assumption.&#xA;With naive Bayes, unfortunately, the estimates of &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; obtained from the biased sample are incorrect. The posterior probability &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; is estimated as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\dfrac{P(x_1 \mid y, s = 1) \dots P(x_n \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)} ,$$&lt;/span&gt;&#xA;which is different (even asymptotically) from the estimate of &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; obtained with naive Bayes without sample selection bias. We cannot simplify this further because there are no independence relationships between each &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Therefore, naive Bayes learners are global learners.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Since it is said that, for global learners, the output of the learner depends asymptotically both on &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; and on &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt;, what is it about &lt;span class=&quot;math-container&quot;&gt;$\dfrac{P(x_1 \mid y, s = 1) \dots P(x_n \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)}$&lt;/span&gt; that indicates that naive Bayes learners are global learners?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;EDIT: To be clear, if we take the example given for the local learner case (section &lt;strong&gt;3.1. Bayesian classifiers&lt;/strong&gt;), then it is evident:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bayesian classifiers compute posterior probabilities &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; using Bayes' rule:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(y \mid x) = \dfrac{P(x \mid y)P(y)}{P(x)}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$P(x \mid y)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(y)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; are estimated from the training data. An example &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is classified by choosing the label &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; with the highest posterior &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We can easily show that bayesian classifiers are not affected by sample selection bias. By using the biased sample as training data, we are effectively estimating &lt;span class=&quot;math-container&quot;&gt;$P(x \mid y, s = 1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(x \mid s = 1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(y \mid s = 1)$&lt;/span&gt; instead of estimating &lt;span class=&quot;math-container&quot;&gt;$P(x \mid y)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(y)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt;. However, when we substitute these estimates into the equation above and apply Bayes' rule again, we see that we still obtain the desired posterior probability &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\dfrac{P(x \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)} = P(y \mid x, s = 1) = P(y \mid x)$$&lt;/span&gt;&#xA;since we are assuming that &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; are independent given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Note that even though the estimates of &lt;span class=&quot;math-container&quot;&gt;$P(x \mid y, s = 1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(x \mid s = 1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(y \mid s = 1)$&lt;/span&gt; are different from the estimates of &lt;span class=&quot;math-container&quot;&gt;$P(x \mid y)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(y)$&lt;/span&gt;, the differences cancel out. Therefore, bayesian learners are local learners.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note that we get &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt;. However, in the global case, it is not clear how we get &lt;span class=&quot;math-container&quot;&gt;$P(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; (as is required for global leaners) from &lt;span class=&quot;math-container&quot;&gt;$\dfrac{P(x_1 \mid y, s = 1) \dots P(x_n \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="16521" LastEditDate="2020-12-16T07:32:31.467" LastActivityDate="2020-12-16T07:32:31.467" Title="$\frac{P(x_1 \mid y, s = 1) \dots P(x_n \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)}$ indicates that naive Bayes learners are global learners?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;naive-bayes&gt;&lt;selection-bias&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25178" PostTypeId="1" CreationDate="2020-12-14T02:02:23.210" Score="3" ViewCount="134" Body="&lt;p&gt;I have a question regarding the time delay in reinforcement learning (RL).&lt;/p&gt;&#xA;&lt;p&gt;In the RL, one has state, reward and action. It is usually assumed that (as far as I understand it) when the action is executed on the system, the state changes immediately and that the new state can then be analysed (influencing the reward) to determine the next action. However, what if there is a time delay in this process. For example, when some action is executed at time &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;, we can only get its effect on the system at &lt;span class=&quot;math-container&quot;&gt;$t_2$&lt;/span&gt; (You can imagine a flow: the actuator is in the upstream region and the sensor is in the downstream region, so that there will be a time delay between the action and the state). How do we deal with this time delay in RL?&lt;/p&gt;&#xA;" OwnerUserId="42941" LastEditorUserId="40671" LastEditDate="2021-05-14T14:12:49.780" LastActivityDate="2021-05-14T14:12:49.780" Title="How to deal with the time delay in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;time-series&gt;&lt;delayed-rewards&gt;&lt;delayed-rl&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25180" PostTypeId="1" CreationDate="2020-12-14T04:41:18.583" Score="0" ViewCount="14" Body="&lt;p&gt;I am currently implementing a closed-domain FAQ chatbot using &lt;a href=&quot;https://www.sbert.net/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sbert.net/index.html&lt;/a&gt; as my main model for answering questions from the users. I wish to extend the functionality of the chatbot by allowing it to identify context from a user's query, and make use of the context in the subsequent conversation.&lt;/p&gt;&#xA;&lt;p&gt;To achieve the above functionality, I am hoping to achieve the following:&lt;/p&gt;&#xA;&lt;p&gt;Identify questions that need to be augmented with the context Examples of such questions are &amp;quot;How to apply&amp;quot; and &amp;quot;What is the fee&amp;quot;, which essentially lacks a subject.&#xA;Augment the question with the existing context&amp;quot;&#xA;For example, Inputs: (1) question = &amp;quot;How to apply&amp;quot; and (2) context = &amp;quot;Machine Learning&amp;quot; will give an output: augmented_question = &amp;quot;How to apply for Machine Learning&amp;quot; The resulting augmented_question should be syntactically and semantically accurate.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of any existing AI/Deep Learning/Models that can be used to achieve the above two tasks?&lt;/p&gt;&#xA;" OwnerUserId="42944" LastActivityDate="2020-12-14T04:41:18.583" Title="Chatbot with Context Management and Awareness" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;chat-bots&gt;&lt;natural-language-understanding&gt;&lt;dialogue-systems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25181" PostTypeId="2" ParentId="5509" CreationDate="2020-12-14T06:15:05.313" Score="0" Body="&lt;p&gt;Seems this question was asked and answered in &lt;a href=&quot;https://github.com/openai/baselines/issues/316&quot; rel=&quot;nofollow noreferrer&quot;&gt;openai/baselines github issue&lt;/a&gt;. The issue has been closed for a while.&lt;/p&gt;&#xA;&lt;p&gt;Below is an answer provided by @matthiasplappert which has the most &amp;quot;thumbs up&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To clarify: PPO is an on-policy algorithm so you are correct that going over the same data multiple times is technically incorrect.&lt;/p&gt;&#xA;&lt;p&gt;However, we found that PPO is actually quite okay with doing this and we still get stable convergence. This is likely due to the proximal trust region constrained that we enforce, which means that the policy cannot change that much anyway when going over the current set of transitions multiple times, making it still approximately on-policy. You can of course get rid of this but then you'll need more samples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="42919" LastActivityDate="2020-12-14T06:15:05.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25183" PostTypeId="1" CreationDate="2020-12-14T09:30:43.083" Score="1" ViewCount="63" Body="&lt;p&gt;I was training a CNN model on TensorFlow. After a while I came back and saw this loss curve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nTIrC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nTIrC.png&quot; alt=&quot;training loss curve(green) and validation loss curve(gray)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The green curve is training loss and the gray one is validation loss. I know that before epoch 394 the model in heavily overfitted, but I have no idea what happened after that.&lt;/p&gt;&#xA;&lt;p&gt;Also, this is accuracy curves if it helps:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u7R6t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u7R6t.png&quot; alt=&quot;accuracy curves&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm using categorical cross-entropy and this is the model I am using:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tcPrn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tcPrn.png&quot; alt=&quot;model architecture&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and here is link to PhysioNet's challenge which I am working on: &lt;a href=&quot;https://physionet.org/content/challenge-2017/1.0.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://physionet.org/content/challenge-2017/1.0.0/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42948" LastEditorUserId="42948" LastEditDate="2020-12-14T10:58:46.450" LastActivityDate="2021-05-13T12:04:50.720" Title="Can someone explain me what does this loss curve says?" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;overfitting&gt;&lt;loss&gt;" AnswerCount="1" CommentCount="8" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25184" PostTypeId="2" ParentId="25153" CreationDate="2020-12-14T10:10:23.320" Score="1" Body="&lt;p&gt;The labels are not unique for the input domain &lt;code&gt;[0,20]&lt;/code&gt;. Think about &lt;code&gt;sin(x)=0&lt;/code&gt;, &lt;code&gt;x&lt;/code&gt; could be &lt;code&gt;0&lt;/code&gt;, &lt;code&gt;pi&lt;/code&gt;, &lt;code&gt;2*pi&lt;/code&gt;, &lt;code&gt;3*pi&lt;/code&gt;, ..., &lt;code&gt;n*pi&lt;/code&gt;, all are correct from a mathematical point of view, but this is not reflected in your MSE loss. At this point your NN has to guess the correct label from your input data. Predicting the mean of your input data is the safest bet for the network.&lt;/p&gt;&#xA;&lt;p&gt;In essence you're trying to build a arcsin function with a NN. If only consider x values in &lt;code&gt;[-0.5*pi,0.5*pi]&lt;/code&gt;, the labels are unique and your network should work.&lt;/p&gt;&#xA;" OwnerUserId="37120" LastActivityDate="2020-12-14T10:10:23.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25185" PostTypeId="2" ParentId="25183" CreationDate="2020-12-14T10:17:35.587" Score="0" Body="&lt;p&gt;Follow &lt;a href=&quot;https://stackoverflow.com/questions/37044600/sudden-drop-in-accuracy-while-training-a-deep-neural-net&quot;&gt;this answer&lt;/a&gt; from StackOverflow, I think your problem related to the second cases where your loss gets Nan value.&lt;/p&gt;&#xA;&lt;p&gt;Maybe you should try to use the larger datatype (for example float 16 -&amp;gt; float 32)&lt;/p&gt;&#xA;" OwnerUserId="41287" LastActivityDate="2020-12-14T10:17:35.587" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="25187" PostTypeId="1" CreationDate="2020-12-14T17:29:19.893" Score="0" ViewCount="67" Body="&lt;p&gt;How should this problem be framed in the domain of RL for preventing users from exceeding their bank account balance and being overdrawn?&lt;/p&gt;&#xA;&lt;p&gt;For example, a user has 1000 in an account, and proceeds to withdraw 300, 400, 500, making the user overdrawn by 200 :((300+400+500) - 1000).&lt;/p&gt;&#xA;&lt;p&gt;Treating this as a supervised learning problem, I could use logistic regression. The input feature is the transaction amounts. The input features for a training instance, 300,400,500 and the output feature occurs if the account is overdrawn or not overdrawn with corresponding values of 1 and 0 respectively. For simplicity, we will assume the number of transactions is consistent and is always 3.&lt;/p&gt;&#xA;&lt;p&gt;For RL, a state could be represented as a series of transactions, but how should the reward be assigned?&lt;/p&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;p&gt;Here my RL implementation of the problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;from collections import defaultdict&#xA;gamma = .1&#xA;alpha = 0.1&#xA;epsilon = 0.1&#xA;n_episode = 2000&#xA;overdraft_limit = 1000&#xA;&#xA;length_episode = [0] * n_episode&#xA;total_reward_episode = [0] * n_episode&#xA;&#xA;episode_states = [[700,100,200,290,500] , [400,100,200,300,500] , [212, 500,100,100,200,500]]&#xA;&#xA;def gen_epsilon_greedy_policy(n_action, epsilon):&#xA;    def policy_function(state, Q):&#xA;        probs = torch.ones(n_action) * epsilon / n_action&#xA;        best_action = torch.argmax(Q[state]).item()&#xA;        probs[best_action] += 1.0 - epsilon&#xA;        action = torch.multinomial(probs, 1).item()&#xA;        return action&#xA;    return policy_function&#xA;&#xA;def is_overdrawn(currentTotal):&#xA;    return currentTotal &amp;gt;= overdraft_limit&#xA;&#xA;# Actions are overdrawn or not, 0 - means it is not overdrawn, 1 - means that it will be overdrawn&#xA;def get_reward(action, currentTotal):&#xA;    if action == 0 and is_overdrawn(currentTotal):&#xA;        return -1&#xA;    elif action == 0 and not is_overdrawn(currentTotal):&#xA;        return 1&#xA;    if action == 1 and is_overdrawn(currentTotal):&#xA;        return 1&#xA;    elif action == 1 and not is_overdrawn(currentTotal):&#xA;        return -1&#xA;    else :&#xA;        raise Exception(&amp;quot;Action not found&amp;quot;) &#xA;&#xA;def q_learning(gamma, n_episode, alpha,n_action):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Obtain the optimal policy with off-policy Q-learning method&#xA;    @param gamma: discount factor&#xA;    @param n_episode: number of episodes&#xA;    @return: the optimal Q-function, and the optimal policy&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Q = defaultdict(lambda: torch.zeros(n_action))&#xA;    for ee in episode_states : &#xA;        for episode in range(n_episode):&#xA;            state = ee[0]&#xA;            index = 0&#xA;            currentTotal = 0&#xA;            while index &amp;lt; len(ee)-1 :&#xA;                currentTotal = currentTotal + state&#xA;                next_state = ee[index+1] &#xA;                action = epsilon_greedy_policy(state, Q)&#xA;#                 print(action)&#xA;                reward = get_reward(action, currentTotal)&#xA;                td_delta = reward + gamma * torch.max(Q[next_state]) - Q[state][action]&#xA;                Q[state][action] += alpha * td_delta&#xA;&#xA;                state = next_state&#xA;                index = index + 1&#xA;&#xA;                length_episode[episode] += 1&#xA;                total_reward_episode[episode] += reward&#xA;                &#xA;    policy = {}&#xA;    for state, actions in Q.items():&#xA;        policy[state] = torch.argmax(actions).item()&#xA;    return Q, policy&#xA;&#xA;epsilon_greedy_policy = gen_epsilon_greedy_policy(2, epsilon)&#xA;&#xA;optimal_Q, optimal_policy = q_learning(gamma, n_episode, alpha, 2)&#xA;&#xA;print('The optimal policy:\n', optimal_policy)&#xA;print('The optimal Q:\n', optimal_Q)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This code prints:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;The optimal policy:&#xA; {700: 0, 100: 0, 200: 1, 290: 1, 500: 0, 400: 0, 300: 1, 212: 0}&#xA;The optimal Q:&#xA; defaultdict(&amp;lt;function q_learning.&amp;lt;locals&amp;gt;.&amp;lt;lambda&amp;gt; at 0x7f9371b0a3b0&amp;gt;, {700: tensor([ 1.1110, -0.8890]), 100: tensor([ 1.1111, -0.8889]), 200: tensor([-0.8889,  1.1111]), 290: tensor([-0.9998,  1.0000]), 500: tensor([ 1.1111, -0.8889]), 400: tensor([ 1.1110, -0.8890]), 300: tensor([-1.0000,  1.0000]), 212: tensor([ 1.1111, -0.8888])})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The optimal policy is to inform us if 700 is added to the balance, then the customer will not overdraw (0). If 200 is added to the balance, then the customer will overdraw(1). What avenues can I explore to improve upon this method as this is quite basic, but I'm unsure as to what approach I should take in order to improve the solution.&lt;/p&gt;&#xA;&lt;p&gt;For example, this solution just looks at the most recent additions to the balance to determine if the customer is overdrawn. Is this a case of adding new features to the training data?&lt;/p&gt;&#xA;&lt;p&gt;I'm just requesting a critique on this solution so I can improve it. How can I improve the representation of the state?&lt;/p&gt;&#xA;" OwnerUserId="12964" LastEditorUserId="43231" LastEditDate="2020-12-29T14:36:56.263" LastActivityDate="2020-12-29T14:36:56.263" Title="How to frame this problem using RL?" Tags="&lt;reinforcement-learning&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="25189" PostTypeId="2" ParentId="18648" CreationDate="2020-12-14T20:21:10.940" Score="1" Body="&lt;h2&gt;Mathematical Interpretation&lt;/h2&gt;&#xA;&lt;p&gt;Note that equation (2.23) is simply calculating the conditional distribution of equation (2.21) and then finding the mean. Your question reduces to:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Given normal variables &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, why is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[Y|X] = \mu_y + Cov(X, Y)Cov(Y, Y)^{-1}(x - \mu_x)$&lt;/span&gt;? (note: in the book, &lt;span class=&quot;math-container&quot;&gt;$\mu_x = \mu_y = 0$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Deriving the conditional probability mean is complicated (see &lt;a href=&quot;http://home.iitk.ac.in/%7Ezeeshan/pdf/The%20Bivariate%20Normal%20Distribution.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Bivariate Normal Distribution&lt;/a&gt;, page 3). A more intuitive look can be seen in the first graph in &lt;a href=&quot;https://online.stat.psu.edu/stat414/lesson/21/21.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here, the mean of &lt;span class=&quot;math-container&quot;&gt;$Y|X$&lt;/span&gt; is linear in what value &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; takes. The line starts at the intercept &lt;span class=&quot;math-container&quot;&gt;$\mu_y$&lt;/span&gt;, and increases with slope &lt;span class=&quot;math-container&quot;&gt;$\rho (\frac{\sigma_y}{\sigma_x}) = \frac{Cov(X,Y)}{\sigma_x \sigma_y}\left(\frac{\sigma_y}{\sigma_x}\right)=\frac{Cov(X,Y)}{\sigma_x^2} = \frac{Cov(X,Y)}{Var(x)} = \frac{Cov(X,Y)}{Cov(X,X)}$&lt;/span&gt;. So the mathematical interpretation of &lt;span class=&quot;math-container&quot;&gt;$Cov(X,Y)Cov(X,X)^{-1}$&lt;/span&gt; is that &lt;strong&gt;it is the slope of the relationship between the mean of &lt;span class=&quot;math-container&quot;&gt;$Y|X$&lt;/span&gt; and the value of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; that you are given&lt;/strong&gt;. As you are given a higher value &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, say &lt;span class=&quot;math-container&quot;&gt;$x + \delta$&lt;/span&gt;, then the mean of &lt;span class=&quot;math-container&quot;&gt;$Y|X$&lt;/span&gt; raises by &lt;span class=&quot;math-container&quot;&gt;$Cov(X,Y)Cov(X,X)^{-1}\delta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why is there even a &lt;span class=&quot;math-container&quot;&gt;$Cov(X,Y)Cov(X,X)^{-1}$&lt;/span&gt; term there? For some reason, multiplying &lt;span class=&quot;math-container&quot;&gt;$Y|X$&lt;/span&gt; by a &lt;span class=&quot;math-container&quot;&gt;$Cov(X,Y)Cov(X,X)^{-1}$&lt;/span&gt; term makes &lt;span class=&quot;math-container&quot;&gt;$Y|X$&lt;/span&gt; completely independent of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (which makes sense as the definition of &amp;quot;conditional probability&amp;quot;, because you are already given a value of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;). This is just a mathematical property, I don't know if there's an intuitive explanation as to why.&lt;/p&gt;&#xA;&lt;h2&gt;Human Interpretation&lt;/h2&gt;&#xA;&lt;p&gt;In case your post just want an intuition as to why there is a &lt;span class=&quot;math-container&quot;&gt;$Cov(X,X)^{-1}$&lt;/span&gt; in the prediction of a Gaussian process (and ignoring the conditional probability fluff), I don't think there's a real basis for this, it would only be coincidental as the authors simply used the conditional probability mean formula, but I would guess &lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$Cov(X,X)^{-1}$&lt;/span&gt; somehow normalizes the values of covariance matrix &lt;span class=&quot;math-container&quot;&gt;$Cov(X_*, X)$&lt;/span&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, if the training set &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; has a lot of outliers and therefore extremely high variance (e.g. all non-diagonal entries in millions), then it is very likely that &lt;span class=&quot;math-container&quot;&gt;$Cov(X_*, X)$&lt;/span&gt; would also be extremely high as &lt;span class=&quot;math-container&quot;&gt;$X_*$&lt;/span&gt; follows the same distribution as &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (unless each data in &lt;span class=&quot;math-container&quot;&gt;$X_*$&lt;/span&gt; matches the exact same variance in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;). It doesn't make sense to multiply &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; by millions though, as &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is already a somewhat decent estimator/prior.&lt;/p&gt;&#xA;&lt;p&gt;It makes more sense to normalize &lt;span class=&quot;math-container&quot;&gt;$Cov(X_*, X)$&lt;/span&gt; by dividing it with the training data variance &lt;span class=&quot;math-container&quot;&gt;$Cov(X,X) = Var(X)$&lt;/span&gt; so that the ratio &lt;span class=&quot;math-container&quot;&gt;$Cov(X_*, X)Cov(X, X)^{-1}$&lt;/span&gt; tends to be more closer to 1 when &lt;span class=&quot;math-container&quot;&gt;$X_*$&lt;/span&gt; follows the same distribution as &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (which should be the case). If the ratio is exactly 1, then &lt;span class=&quot;math-container&quot;&gt;$X_*$&lt;/span&gt; has the exact same distribution as &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, so you just return the prior estimate &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. If the ratio is far away from &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, then the test set distribution is wildly different than the training set distribution, so you return a number far away from &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-14T20:21:10.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25190" PostTypeId="2" ParentId="25150" CreationDate="2020-12-14T20:42:41.010" Score="0" Body="&lt;p&gt;Yes, it is possible, CNNs can also be used for OCR (see the &lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST task&lt;/a&gt; and &lt;a href=&quot;https://nanonets.com/blog/deep-learning-ocr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt;), although it's not the common way for OCR because it is considered a bit overkill and inefficient. Furthermore, OCR is considered a solved problem already and don't need deep learning to do well, unless perhaps under unfavorable conditions like dark lighting, complex background of the document, weird fonts, occlusion of text, etc.&lt;/p&gt;&#xA;&lt;p&gt;Breaking down your questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a CNN can likely classify an image as &amp;quot;this image has a word document in it&amp;quot; vs &amp;quot;this image does not have a word document in it&amp;quot;. The word document itself, and the outline of the words, should form distinct enough features for the CNN to discriminate against&lt;/li&gt;&#xA;&lt;li&gt;a CNN can likely identify the vendor based on the logo of the vendor if you have enough training images of it at various distortions (angles, occlusions, different lighting, etc)&lt;/li&gt;&#xA;&lt;li&gt;a CNN can likely do OCR well enough to extract the text, and then an entity extraction model can identify the vendor from the words. However, this approach would be uncommon as OCR is already considered a solved problem. But a neural network can potentially help preprocessing difficult OCR images (rotating image, identifying text boundaries, fix lighting, perform super-resolution, etc) to increase OCR accuracy&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-14T20:42:41.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25191" PostTypeId="2" ParentId="25165" CreationDate="2020-12-14T21:55:32.183" Score="0" Body="&lt;p&gt;As stated in &lt;a href=&quot;https://ai.stackexchange.com/a/25176/2444&quot;&gt;the other answer&lt;/a&gt;, in general, the depth of the decision tree depends on the decision tree algorithm, i.e. the algorithm that builds the decision tree (for regression or classification).&lt;/p&gt;&#xA;&lt;p&gt;To address your notes more directly and why that statement may not be always true, let's take a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/ID3_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;the ID3 algorithm&lt;/a&gt;, for instance. Here's the initial part of its pseudocode.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID3 (Examples, Target_Attribute, Attributes)&#xA;    Create a root node for the tree&#xA;    If all examples are positive, Return the single-node tree Root, with label = +.&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, in general, the depth of the tree may not depend on the number of features, but it may just depend on the labels or training examples (which is a degenerate case), although, in most cases, it will also depend on the number of features, because each node represents a split of the training examples based on some condition that needs to be true for some feature (e.g. the height of the people must be less than 150cm).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-12-14T21:55:32.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25192" PostTypeId="2" ParentId="25178" CreationDate="2020-12-14T22:38:32.517" Score="1" Body="&lt;p&gt;Most RL algorithms assume a discretization of time (although RL can also be applied to continuous-time problems [&lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]), i.e., in theory, it doesn't really matter what the actual time between consecutive time steps is, but, in practice, you may have delays in the rewards or observations, so you cannot perform e.g. the TD updates immediately. One natural solution to your problem would be to keep track (e.g. in a buffer) of the reward obtained and the next state that the agent ended up in after having taken a certain action in a certain state, or use some kind of synchronization mechanism (note that I've just come up with these solutions, so I don't know if this has been done or not to solve problems). In practice, this may not work (in all cases), for example, during real-time inference, where you need to decide quickly what you need to do even without full information about the current state or reward.&lt;/p&gt;&#xA;&lt;p&gt;Note that, in RL, rewards are often said to be &lt;a href=&quot;https://ai.stackexchange.com/a/12190/2444&quot;&gt;&lt;em&gt;delayed&lt;/em&gt;&lt;/a&gt;, in the sense that&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;you may know the consequences of an action only many time-steps after you have taken it (determining the consequences of an action is known as the &lt;a href=&quot;https://ai.stackexchange.com/q/12908/2444&quot;&gt;credit assignment problem&lt;/a&gt;), or&lt;/li&gt;&#xA;&lt;li&gt;you may get a non-zero reward only when the agent gets to a goal/final state (in this last case, these rewards are also known as &lt;em&gt;sparse&lt;/em&gt;).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;These two problems are common in RL. However, if I understand correctly your concerns, this is a bit different than your problem, because your problem also involves the potential delay of the state or even reward that was supposed to arrive at a previous time step, which can be due e.g. to an erratic or broken sensor/actuator. For instance, if you are using &lt;a href=&quot;https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DQN&lt;/a&gt;, which typically builds an approximation of the current state by concatenating the last frames captured by your camera, if you have delays in the frames that cause the natural order of the frames to be changed, this could lead to a bad approximation of the current state, which could actually lead to a catastrophic event. So, yes, this is an important problem that needs to be tackled.&lt;/p&gt;&#xA;&lt;p&gt;Given that I am not really familiar with the actual existing solutions, I'll refer you to the paper &lt;a href=&quot;https://arxiv.org/pdf/1904.12901.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Challenges of Real-World Reinforcement Learning&lt;/a&gt; that I read a few weeks ago, which mentions this issue and points you to other research work that attempted to address it. Take a look at &lt;a href=&quot;https://ai.stackexchange.com/a/7344/2444&quot;&gt;this answer&lt;/a&gt; too, if you're more interested in delayed/sparse rewards.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-14T23:24:42.207" LastActivityDate="2020-12-14T23:24:42.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25193" PostTypeId="1" AcceptedAnswerId="25202" CreationDate="2020-12-14T23:29:51.263" Score="1" ViewCount="35" Body="&lt;p&gt;Are the state-action values and the state value function equivalent for a given policy? I would assume so as the value function is defined as &lt;span class=&quot;math-container&quot;&gt;$V(s)=\sum_a \pi(a|s)Q_{\pi}(s,a)$&lt;/span&gt;. If we are operating a greedy policy and hence acting optimally, doesn't this mean that in fact the policy is deterministic and then &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for the optimal action and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for all others? Would this then lead to an equivalence between the two?&lt;/p&gt;&#xA;&lt;p&gt;Here is my work to formulate some form of proof where I start with the idea that a policy is defined to be better than a current policy if for all states then &lt;span class=&quot;math-container&quot;&gt;$Q_{\pi}(S,\pi^∗(s))\geq Vπ_{\pi}(s)$&lt;/span&gt; :&lt;/p&gt;&#xA;&lt;p&gt;I iteratively apply the optimal policy to each time step until I eventually get to a fully optimal time step of rewards&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Vπ_{\pi}(s)≤Q_{\pi}(S,\pi^∗(s))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=Eπ[R_{t+1}+\gamma V_{\pi}(St+1)|St=s]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\leq E[Rt+1+\gamma Q_{\pi}(S_{t+1},\pi^∗(S_{t+1})|S_t=s]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\leq E[Rt+1+\gamma Rt+2+\gamma 2Q \pi^*(S_{t+2},\pi^∗(S_{t+2})|S_t=s]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\leq E[R_{t+1}+\gamma R_{t+2}+....|S_t=s]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=V\pi^∗(s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would say that our final two lines are in fact inequalities, and for me this makes intuitive sense in that if we are always taking a deterministic greedy action our value function and Q function are the same. As detailed here, for a given policy and state we have that &lt;span class=&quot;math-container&quot;&gt;$V(s)=\sum_a \pi(a|s)Q_{\pi}(s,a)$&lt;/span&gt; and if the policy is optimal and hence greedy then &lt;span class=&quot;math-container&quot;&gt;$\pi(a|s)$&lt;/span&gt; is deterministic.&lt;/p&gt;&#xA;" OwnerUserId="42966" LastEditorUserId="2444" LastEditDate="2020-12-14T23:46:01.433" LastActivityDate="2020-12-15T12:17:56.733" Title="Equivalence of the $Q(s,a)$ and $V(s)$ under optimality conditions?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;&lt;optimal-policy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25194" PostTypeId="1" CreationDate="2020-12-14T23:32:09.713" Score="2" ViewCount="44" Body="&lt;p&gt;Suppose we want to estimate a continuous function &lt;span class=&quot;math-container&quot;&gt;$f:\mathbb R^2 \rightarrow \mathbb R$&lt;/span&gt; based on a sample using a NN (around 1000 examples). This function is not bounded. Which architecture would you choose ? How many layers/neurons ? Which activation functions ? Which loss function ?&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, I would go with one hidden layer, 2 neurons, &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; loss, and maybe the Bent identity for the output and a sigmoid in the hidden layer ?&lt;/p&gt;&#xA;&lt;p&gt;What are the advantages of doing something &amp;quot;fancier&amp;quot; than that ?&lt;/p&gt;&#xA;&lt;p&gt;Would you also have chosen to use a NN for this job or would you have considered a regression SVM for example or something else (knowing that precision is the goal)?&lt;/p&gt;&#xA;" OwnerUserId="42969" LastActivityDate="2020-12-15T21:50:21.317" Title="Which NN would you choose to estimate a continuous function $f:\mathbb R^2 \rightarrow \mathbb R$?" Tags="&lt;neural-networks&gt;&lt;loss-functions&gt;&lt;activation-function&gt;&lt;regression&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25199" PostTypeId="1" CreationDate="2020-12-15T06:35:52.867" Score="10" ViewCount="195" Body="&lt;p&gt;In many implementations/tutorials of GANs that I've seen so far (e.g. &lt;a href=&quot;https://github.com/eriklindernoren/Keras-GAN/blob/master/dcgan/dcgan.py&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;), the generator and discriminator start with no prior knowledge. They continuously improve their performance with training. This makes me wonder — is it possible to use a pre-trained discriminator? I have two motivations for doing so:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Eliminating the overhead of training the discriminator&lt;/li&gt;&#xA;&lt;li&gt;Being able to use already existing cool models&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Would the generator be able to learn just the same, or is it dependent on the fact that they start from scratch?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastActivityDate="2020-12-24T16:47:38.163" Title="Can I start with perfect discriminator in GAN?" Tags="&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25200" PostTypeId="1" CreationDate="2020-12-15T10:41:51.980" Score="0" ViewCount="13" Body="&lt;p&gt;I want to compare multiple computer vision &lt;strong&gt;Multi-Object Tracking (MOT)&lt;/strong&gt; methods on my own dataset, so first I want to choose the best metrics for this task. I have carried out some research in scientific literature and I come to the conclusion that there are three main metrics sets:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Metrics from &lt;a href=&quot;https://www.researchgate.net/publication/4246190_Tracking_of_Multiple_Partially_Occluded_Humans_based_on_Static_Body_Part_Detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Tracking of Multiple, Partially Occluded Humans based on Static Body Part&#xA;Detection&amp;quot;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/26523191_Evaluating_multiple_object_tracking_performance_The_CLEAR_MOT_metrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLEAR MOT metrics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1609.01775&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID scores&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Therefore, I wonder to which of the above metrics should I attach the greatest importance?&lt;/p&gt;&#xA;&lt;p&gt;And I would like to ask if anyone has encountered a similar issue and has any thoughts on this topic that could justify and help me to choose the best metrics for above task.&lt;/p&gt;&#xA;" OwnerUserId="30992" LastActivityDate="2020-12-15T10:41:51.980" Title="What are the best metrics for Multi-Object Tracking (MOT) evaluation and why?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25201" PostTypeId="1" AcceptedAnswerId="25203" CreationDate="2020-12-15T11:33:24.890" Score="0" ViewCount="45" Body="&lt;p&gt;I am confused about the workings of the first- and every-visit MC.&lt;/p&gt;&#xA;&lt;p&gt;My first question is, when we have multiple traces, do we average over traces or the total number of times we have visited that state?&lt;/p&gt;&#xA;&lt;p&gt;So, in the example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tau_1 = \text{House} +3, \text{House} +2, \text{School} -4, \text{House} +4, \text{School} -3, \text{Holidays}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\tau_2 = \text{House} -2, \text{House} +3, \text{School} -3, \text{Holidays},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where we have states of either House, Holidays, or School, with the numerical values being the immediate rewards.&lt;/p&gt;&#xA;&lt;p&gt;For every-visit MC to find the state value of HOUSE, with &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;=1, my intuition would be to create a return list, R, that looks like the following&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R_1=[3+2−4+4−3, 2−4+4−3, 4−3]= [2, −1, 1]$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$R_2=[−2+3−3, 3−3]=[−2, 0]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$R_1+R_2=[2,−1, 1,−2, 0]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which, when averaged over 5 visits, is 0 and the correct answer, but I would like if you could confirm if the methodology is correct?&lt;/p&gt;&#xA;&lt;p&gt;However, another approach would be to compute the average returns for each trace. Which is correct?&lt;/p&gt;&#xA;" OwnerUserId="42966" LastEditorUserId="2444" LastEditDate="2020-12-16T10:07:48.510" LastActivityDate="2020-12-16T10:07:48.510" Title="When we have multiple traces, do we average over traces or the total number of times we have visited that state?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;monte-carlo-methods&gt;&lt;model-based-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25202" PostTypeId="2" ParentId="25193" CreationDate="2020-12-15T12:17:56.733" Score="1" Body="&lt;p&gt;In general they are &lt;em&gt;not&lt;/em&gt; the same and that should be clear as to why -- mathematically you are conditioning on an extra random variable being known in the state-action value function. You have the correct relationship between them, but I think your understanding of the two may be slightly off. The state-action value function is a function of both &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; whereas the value function is a function of just &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. As you have noted, the value function is equal to the expected value of the state-action value function, with the expectation taken over the action distribution induced by the policy.&lt;/p&gt;&#xA;&lt;p&gt;However, if you have a deterministic policy, then two are equivalent &lt;em&gt;only if you evaluate the state-action value function at the action that the deterministic policy gives&lt;/em&gt;. That is, &lt;span class=&quot;math-container&quot;&gt;$v_\pi(s) = Q(s, \pi(s))$&lt;/span&gt;. This is because, as you say, for a given state the policy will assign probability one to a certain action and 0 to all other actions in the action space and so the equation I just wrote is what the expectation in your question reduces to.&lt;/p&gt;&#xA;&lt;p&gt;Now, you might think 'well the policy is deterministic so we are always going to choose this action, so my theory was correct'. This is not true. The definition of the state-action value function is the value of the expected future (possibly discounted) returns given that we are in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and given that we have taken action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. This means that we have already chosen our action and we are &lt;em&gt;not&lt;/em&gt; choosing it according to the policy -- the policy will only choose all &lt;em&gt;future&lt;/em&gt; actions.&lt;/p&gt;&#xA;&lt;p&gt;You also say that &amp;quot;If we are operating a greedy policy and hence acting optimally&amp;quot; which is not true. Just because we act greedily it does &lt;em&gt;not&lt;/em&gt; imply we are acting optimally. Value functions are functions of a policy. If the policy is completely random then you would get certain value and state-action value functions which you could act greedily to but this would not be optimal.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-12-15T12:17:56.733" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25203" PostTypeId="2" ParentId="25201" CreationDate="2020-12-15T12:30:27.413" Score="2" Body="&lt;p&gt;For every visit MC you create a list for each state. Every time you enter a state you calculate the returns for the episode and append these returns to a list. Once you have done this for all the episode you want to average over you simply calculate the value of a state to be the average of this list of returns for the state.&lt;/p&gt;&#xA;&lt;p&gt;First visit MC is almost the same except that you only append the returns to the state returns list for the first time you visit the state in an episode.&lt;/p&gt;&#xA;&lt;p&gt;Your workings are correct, you average over the number of times you have visited the state (in every visit MC). So, in your example, you would get the value of HOUSE to be 0, as you stated. If you were doing first visit MC then the returns for episode 1 would be 2 and the returns for episode 2 would be -2, which you would then average over the two first visits to again give you 0 (this would not always be the case that they are equal after 2 episodes but in the limit both methods do converge to the true state value function).&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-12-15T12:30:27.413" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25204" PostTypeId="1" CreationDate="2020-12-15T13:57:00.977" Score="0" ViewCount="65" Body="&lt;p&gt;Reinforcement Learning: An Introduction second edition, Richard S. Sutton and Andrew G. Barto:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We made two unlikely assumptions above in order to easily obtain this guarantee of&#xA;convergence for the Monte Carlo method. ... For now we focus on the assumption that policy evaluation operates on an infinite&#xA;number of episodes. This assumption is relatively easy to remove. In fact, the same issue&#xA;arises even in classical DP methods such as iterative policy evaluation, which also converge&#xA;only asymptotically to the true value function.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;There is a second approach to avoiding the infinite number of episodes nominally&#xA;required for policy evaluation, in which we give up trying to complete policy evaluation&#xA;before returning to policy improvement. On each evaluation step we move the value&#xA;function toward q⇡k, but we do not expect to actually get close except over many steps.&#xA;We used this idea when we first introduced the idea of GPI in Section 4.6. One extreme&#xA;form of the idea is value iteration, in which only one iteration of iterative policy evaluation&#xA;is performed between each step of policy improvement. The in-place version of value&#xA;iteration is even more extreme; there we alternate between improvement and evaluation&#xA;steps for single states.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The original pseudocode:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Monte Carlo ES (Exploring Starts), for estimating&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi \approx \pi_{*}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Initialize:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi(s) \in \mathcal{A}(s)$&lt;/span&gt; (arbitrarily), for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$Q(s, a) \in \mathbb{R}$&lt;/span&gt; (arbitrarily), for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}, a \in \mathcal{A}(s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$Returns(s, a) \leftarrow$&lt;/span&gt; empty list, for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}, a \in \mathcal{A}(s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Loop forever (for each episode):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; Choose &lt;span class=&quot;math-container&quot;&gt;$S_{0} \in \mathcal{S}, A_{0} \in \mathcal{A}\left(S_{0}\right)$&lt;/span&gt; randomly such that all pairs have probability &lt;span class=&quot;math-container&quot;&gt;$\geq 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; Generate an episode from &lt;span class=&quot;math-container&quot;&gt;$S_{0}, A_{0},$&lt;/span&gt; following &lt;span class=&quot;math-container&quot;&gt;$\pi: S_{0}, A_{0}, R_{1}, \ldots, S_{T-1}, A_{T-1}, R_{T}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$G \leftarrow 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; Loop for each step of episode, &lt;span class=&quot;math-container&quot;&gt;$t=T-1, T-2, \ldots, 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$G \leftarrow \gamma G+R_{t+1}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad$&lt;/span&gt; Unless the pair &lt;span class=&quot;math-container&quot;&gt;$S_{t}, A_{t}$&lt;/span&gt; appears in &lt;span class=&quot;math-container&quot;&gt;$S_{0}, A_{0}, S_{1}, A_{1} \ldots, S_{t-1}, A_{t-1}:$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\quad$&lt;/span&gt; Append &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$Returns\left(S_{t}, A_{t}\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$Q\left(S_{t}, A_{t}\right) \leftarrow \text{average}\left(Returns\left(S_{t}, A_{t}\right)\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi\left(S_{t}\right) \leftarrow \arg \max _{a} Q\left(S_{t}, a\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I want to make the same algorithm but with a model. The book states:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;With a model, state values alone are sufficient to determine a policy; one simply looks ahead one step and chooses whichever action leads to the best combination of reward and next state, as we did in the chapter on DP.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So based on the 1st quote I must use &amp;quot;stars exploration&amp;quot; and &amp;quot;one evaluation — one improvement&amp;quot; ideas (as well as in model-free version) to make the algorithm converge.&lt;/p&gt;&#xA;&lt;p&gt;My version of the pseudocode:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Monte Carlo ES (Exploring Starts), for estimating &lt;span class=&quot;math-container&quot;&gt;$\pi \approx \pi_{*}$&lt;/span&gt; (with model)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Initialize:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi(s) \in \mathcal{A}(s)$&lt;/span&gt; (arbitrarily), for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$V(s) \in \mathbb{R}$&lt;/span&gt; (arbitrarily), for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$Returns(s) \leftarrow$&lt;/span&gt; empty list, for all &lt;span class=&quot;math-container&quot;&gt;$s \in \mathcal{S}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Loop forever (for each episode):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; Choose &lt;span class=&quot;math-container&quot;&gt;$S_{0} \in \mathcal{S}, A_{0} \in \mathcal{A}\left(S_{0}\right)$&lt;/span&gt; randomly such that all pairs have probability &lt;span class=&quot;math-container&quot;&gt;$\geq 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; Generate an episode from &lt;span class=&quot;math-container&quot;&gt;$S_{0}, A_{0},$&lt;/span&gt; following &lt;span class=&quot;math-container&quot;&gt;$\pi: S_{0}, A_{0}, R_{1}, \ldots, S_{T-1}, A_{T-1}, R_{T}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$G \leftarrow 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad$&lt;/span&gt; Loop for each step of episode, &lt;span class=&quot;math-container&quot;&gt;$t=T-1, T-2, \ldots, 1$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$G \leftarrow \gamma G+R_{t+1}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad$&lt;/span&gt; Unless &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt; appears in &lt;span class=&quot;math-container&quot;&gt;$S_{0}, S_{1}, \ldots, S_{t-1}:$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\quad$&lt;/span&gt; Append &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$Returns \left(S_{t}\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$V\left(S_{t}\right)\leftarrow\text{average}\left(Returns\left(S_{t}\right)\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\quad\quad\quad$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\pi\left(S_{t-1}\right) \leftarrow \operatorname{argmax}_{a} \sum_{s^{\prime}, r} p\left(s^{\prime}, r \mid S_{t-1}, a\right)\left[\gamma V\left(s^{\prime}\right)+r\right]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;— Here I update the policy in &lt;span class=&quot;math-container&quot;&gt;$S_{t-1}$&lt;/span&gt; because the step before we update &lt;span class=&quot;math-container&quot;&gt;$V(S_{t})$&lt;/span&gt; and changes to &lt;span class=&quot;math-container&quot;&gt;$V(S_{t})$&lt;/span&gt; don't affect &lt;span class=&quot;math-container&quot;&gt;$\pi (S_{t})$&lt;/span&gt;, but affect &lt;span class=&quot;math-container&quot;&gt;$ \pi (S_{t-1})$&lt;/span&gt;, as &lt;span class=&quot;math-container&quot;&gt;$S_{t}$&lt;/span&gt; is in &lt;span class=&quot;math-container&quot;&gt;$S'$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$S_{t-1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Pseudocode as images:&lt;/p&gt;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;&lt;p&gt; &lt;a href=&quot;https://i.stack.imgur.com/jo686.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jo686.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&#xA; &lt;a href=&quot;https://i.stack.imgur.com/qlcLW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qlcLW.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="42983" LastEditorUserId="42983" LastEditDate="2020-12-17T09:56:14.650" LastActivityDate="2020-12-17T09:56:14.650" Title="Is my pseudocode titled &quot;Monte Carlo Exploring Starts (with model)&quot; correct?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25205" PostTypeId="1" CreationDate="2020-12-15T14:20:14.653" Score="6" ViewCount="207" Body="&lt;p&gt;The KL Divergence is quite easy to compute in closed form for simple distributions -such as Gaussians- but has some not-very-nice properties. For example, it is not symmetrical (thus it is not a metric) and it does not respect the triangular inequality.&lt;/p&gt;&#xA;&lt;p&gt;What is the reason it is used so often in ML? Aren't there other statistical distances that can be used instead?&lt;/p&gt;&#xA;" OwnerUserId="32583" LastActivityDate="2020-12-22T15:40:27.503" Title="Why is KL divergence used so often in Machine Learning?" Tags="&lt;probability-distribution&gt;&lt;kl-divergence&gt;&lt;wasserstein-metric&gt;&lt;total-variational-distance&gt;" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25206" PostTypeId="2" ParentId="25194" CreationDate="2020-12-15T15:51:56.610" Score="1" Body="&lt;p&gt;It depends on the complexity of your problem. &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^2 \rightarrow \mathbb{R}^1$&lt;/span&gt; looks simple, but I can give you some nonsense complicated examples that need a deep network. So, the complexity of the problem sets the number of layers and neurons. The kind of problem will determine the architecture of your network (if it needs memory or not). In most cases, the &lt;em&gt;mean squared error&lt;/em&gt; is ok. However, for the activation function, I would go with the ReLU.&lt;/p&gt;&#xA;&lt;p&gt;If the SVM is good enough for your problem, then go with it. Your question is general and an exact answer needs more information about the problem.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastEditorUserId="2444" LastEditDate="2020-12-15T21:50:21.317" LastActivityDate="2020-12-15T21:50:21.317" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25207" PostTypeId="2" ParentId="12240" CreationDate="2020-12-15T16:29:00.107" Score="0" Body="&lt;p&gt;First for clarity I want to differ &amp;quot;convolutional kernel&amp;quot; and &amp;quot;filter&amp;quot; here, let's say a filter has several convolutional kernels. Second in NLP 1D-convolution is mostly used, so different from CV, which typically use a &lt;span class=&quot;math-container&quot;&gt;$k \times k$&lt;/span&gt; convolutional kernel, in NLP generally the kernel size is only &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Having listed above hope things could become a little clearer: If we adopt the same concepts as we seen in typical CV scenario, per my understanding, the size of convolutional kernel is only &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, but the count of kernels, or the dimension of new &amp;quot;channel&amp;quot;, is set to &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt;. One kernel maps &lt;span class=&quot;math-container&quot;&gt;$k \times d$&lt;/span&gt; elements into one element, &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt; kernels together convert the input to a vector of which the dimention is &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As the calculation process is mapping a matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{k \times d}$&lt;/span&gt; to a vector &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{2d \times 1}$&lt;/span&gt;, if expressed in a form of matrix multiplication, or in a full-connected view, we can firstly expand the input matrix, concatenate all rows together to get a new big vector of which the dimension is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{kd \times 1}$&lt;/span&gt;, then multiply a weight matrix &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{W} \in \mathbb{R}^{2d \times kd}$&lt;/span&gt; could get the final vector&lt;/p&gt;&#xA;&lt;p&gt;The key problem that roots the confusion is beause, in my opinion, the &amp;quot;kernel&amp;quot; imported here is a little different from the same concept used in a typical CNN. Again, if we describe in a traditional CNN terminology, the kernel size should be &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, and there are &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt; kernels applied.&lt;/p&gt;&#xA;&lt;p&gt;Hope my understanding is correct.&lt;/p&gt;&#xA;" OwnerUserId="42984" LastActivityDate="2020-12-15T16:29:00.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25208" PostTypeId="1" CreationDate="2020-12-15T18:16:07.753" Score="0" ViewCount="37" Body="&lt;p&gt;I am training an AlexNet Convolutional Neural Network to classify images in a dataset. I want to know if there is any general rule for using data augmentation in training a neural network. How can I make sure about the amount of data, and how can I know if I need more data?&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="2444" LastEditDate="2020-12-16T09:11:38.470" LastActivityDate="2020-12-16T09:11:38.470" Title="Is there any rule of thumb to determine the amount of data needed to train a CNN" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;datasets&gt;&lt;data-augmentation&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25209" PostTypeId="1" CreationDate="2020-12-15T20:47:19.627" Score="0" ViewCount="82" Body="&lt;p&gt;I'm trying to understand the structure of the custom recurrent policy introduced in the documentation of the &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/guide/custom_policy.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stable Baselines&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;How exactly is the Lstm NN constructed? (check code below)&lt;/p&gt;&#xA;&lt;p&gt;From what I understood from the documentation: in this case &lt;code&gt;net_arch=[8, 'lstm']&lt;/code&gt; means that before the LsTm there is an NN with hidden layers of size 8. &lt;br /&gt;&#xA;A crude illustration would be:&lt;/p&gt;&#xA;&lt;p&gt;observation (input) -&amp;gt; 8 hidden nodes -&amp;gt; Lstm -&amp;gt; action (output)&lt;/p&gt;&#xA;&lt;p&gt;Let's say, I want to construct the following Network:&lt;/p&gt;&#xA;&lt;p&gt;observation -&amp;gt; hidden layer of 8 nodes -&amp;gt; hidden layer of 16 nodes -&amp;gt; Lstm -&amp;gt; hidden layer of 16 nodes -&amp;gt; output layer (outputs: from policy and value network)&lt;/p&gt;&#xA;&lt;p&gt;Would I have to write:&#xA;&lt;code&gt;net_arch=[8,16, 'lstm',16]&lt;/code&gt;? Is this correct?&#xA;Also, what exactly does it mean &lt;code&gt;feature_extractor='mlp']&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    class CustomLSTMPolicy(LstmPolicy):&#xA;    def __init__(self, sess, ob_space, ac_space, n_env, n_steps, n_batch, n_lstm=64, reuse=False, **_kwargs):&#xA;        super().__init__(sess, ob_space, ac_space, n_env, n_steps, n_batch, n_lstm, reuse,&#xA;                         net_arch=[8, 'lstm', dict(vf=[5, 10], pi=[10])],&#xA;                         layer_norm=True, feature_extraction=&amp;quot;mlp&amp;quot;, **_kwargs)```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="41538" LastEditorUserId="43231" LastEditDate="2020-12-29T14:35:22.890" LastActivityDate="2020-12-29T14:35:22.890" Title="Understanding Stable Baselines Custom Policies" Tags="&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;architecture&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25210" PostTypeId="1" CreationDate="2020-12-16T04:18:44.000" Score="4" ViewCount="82" Body="&lt;p&gt;(The math problem here just serves as an example, my question is on this type of problems in general).&lt;/p&gt;&#xA;&lt;p&gt;Given two &lt;a href=&quot;https://en.wikipedia.org/wiki/Schur_polynomial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schur polynomials&lt;/a&gt;, &lt;span class=&quot;math-container&quot;&gt;$s_\mu$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$s_\nu$&lt;/span&gt;, we know that we can decompose their product into a linear combination of other Schur polynomials.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$s_\mu s_\nu = \sum_\lambda c_{\mu,\nu}^\lambda s_\lambda$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and we call &lt;span class=&quot;math-container&quot;&gt;$c_{\mu,\nu}^\lambda$&lt;/span&gt; the LR coefficient (always an non-negative integer).&lt;/p&gt;&#xA;&lt;p&gt;Hence, a natural supervised learning problem is to predict whether the LR coefficient is of a certain value or not given the tuple &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;\mu, \nu, \lambda&amp;gt;$&lt;/span&gt;. This is not difficult.&lt;/p&gt;&#xA;&lt;p&gt;My question is: can we either use ML/RL to do anything else other than predicting (in this situation) or extract anything from the prediction result? In other words, a statement like &amp;quot;oh, I am 98% confident that this LR coefficient is 0&amp;quot; does not imply anything mathematically interesting?&lt;/p&gt;&#xA;" OwnerUserId="41751" LastEditorUserId="2444" LastEditDate="2021-01-17T16:53:58.530" LastActivityDate="2021-02-16T19:08:00.767" Title="Can we use ML to do anything else other than predicting (in the case of mathematical problems)?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;datasets&gt;&lt;math&gt;&lt;supervised-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25213" PostTypeId="1" CreationDate="2020-12-16T09:30:54.583" Score="0" ViewCount="42" Body="&lt;p&gt;I recently watched this &lt;a href=&quot;https://www.youtube.com/watch?v=oNT4iwU6Pew&amp;amp;ab_channel=MichaelPenn&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;, in which he solves the equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x)+f\left(\frac{1}{1-x}\right) = x$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The answer is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = \frac{x^3-x+1}{2x(x-1)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried to solve this functional equation using the following models (optimization is done through Adam in each case):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A multilayer perception with one hidden layer of 100 nodes and relu as the activation function.&lt;/li&gt;&#xA;&lt;li&gt;Two hidden layers with 100 nodes, everything else being same as 1.&lt;/li&gt;&#xA;&lt;li&gt;Defined a custom layer&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;L(x) = A + Bx + Cx^2 + Dx^3&#xA;$$&lt;/span&gt;&#xA;and the model is then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M(x) = \frac{L_1(x)}{L_2(x)}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; are two different instantiations of L.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;But, to my surprise, even model 3 didn't work. So, my question is how can I solve this functional equation using neural networks?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://colab.research.google.com/drive/1LdTdKr8RwPYTTwWbn5QMZmaWvrT62GTk?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is my colab notebook link.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user43000" LastActivityDate="2020-12-16T09:30:54.583" Title="Can I solve the below functional equation using neural networks?" Tags="&lt;neural-networks&gt;&lt;math&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25214" PostTypeId="1" CreationDate="2020-12-16T13:21:41.830" Score="0" ViewCount="22" Body="&lt;p&gt;I am currently studying Recurrent neural network for solving optimization problems, which is popular back in the 1990s.&lt;/p&gt;&#xA;&lt;p&gt;I am very familiar with the traditional feedforward neural network, however, when I read the paper about the neural network in the 1990s, the following math expression troubles me so much,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{du}{dt} = W*v(t)+\theta$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v(t) = F(u(t))$$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta $&lt;/span&gt; are the weight matrix,  &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; are the neuron states before and after activation function &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is what exactly is this  &lt;span class=&quot;math-container&quot;&gt;$\frac{du}{dt}$&lt;/span&gt; mean and how it gets involved in the iterative process. Or in other words, how the neural network system evolves by this &lt;span class=&quot;math-container&quot;&gt;$\frac{du}{dt}$&lt;/span&gt;, in other literature also referred to &lt;span class=&quot;math-container&quot;&gt;$\frac{dx}{dt}(t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For more information, the paper I am reading is Wang, Jun. &lt;a href=&quot;https://pdf.sciencedirectassets.com/271125/1-s2.0-S0893608000X00767/1-s2.0-0893608094900418/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEG4aCXVzLWVhc3QtMSJHMEUCIQCNuDHBr%2FxBJCYFGeYwFCRrXv5fooVriiCYf0Dfk%2FnCBAIgJ3nxrXjTef%2FpzdW1K7FYKpv%2FDAvAXB9c80kRZyJBBTYqtAMIFxADGgwwNTkwMDM1NDY4NjUiDNuM3E50Jkk%2B9Gs2BCqRA%2FhhvdOlAWUIHNzM4a8aMcbB6RxPbQ%2FTW7grPtH2aagrbnExSrOqm1KRZXVJIeMC6HZIJVC24GQHPeH%2BHiF0RKB8EuF655eDYi8vbXYzRGgIBwDuAQulHOaGYVzriizh6J%2BnOSwtGOFnqoLI5tNP%2Brp4e%2FoyXnnOpvYyIvt4qNHezPIQNq57QXebQC49mG1kII9qsZJpJlEpdyXqkDI82NRbRJnXj%2FE9eMxShbAwHaEN3n%2FKRTV0OTtwZAhaJv4weCXzUVEQaDMSTKoYWKXNOD06RWfDIwS%2FUtu%2Bwpl1zzQw%2F8vqT%2BPRfwH1B7%2BKLNpzZOgunkhZxwLgKg%2BFUrZIARe%2F3Mj5F63NN7Q6qHS1e65%2Fc7%2BQtgeI2Lipw79Ox8xVZUn%2BGr6od%2BOoektH1Fp32LeZT16JjVuENAFUgS6%2B4TOfIWOnqDEISiSOLofDZG3B9lrLbM5wf4HDutrhQmVN1p0jKOqRaM%2BG6myuz%2BxIdrTfwVEtEvXV5f65vbLRL1QyV67laLhzwPmPU1sIsdfL%2F7ZvMPCh6P4FOusBtuH%2BH1ob6a04aIgKLpgRtHTirLlVluODFl6MzkLyvvGnuWAKwBl4hXN8oJR1%2Bod%2FORGsspOybgnXRoVtBhbycxMJkqjb3KXKzbrc4cIPZrIhRxuKa60mLtggyTuemSejaL8lZ%2BRdyQ0SPMDyJ1%2BklYeLNr75pf54V3G%2BtauZ3pQyurmc4OfCLUIoEc9%2Fpf6Di8OabSdAPUstfPAS%2FRru93XRNTt07Ohu46sLv9Sj3PwEJnHHTxc2f8M5CegOUflqEwZgUKLw0NA9u5pM0L0PxcwEsCDakoijlHkV6%2FbG08VLEskA68X9WHvbkg%3D%3D&amp;amp;X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;amp;X-Amz-Date=20201216T150014Z&amp;amp;X-Amz-SignedHeaders=host&amp;amp;X-Amz-Expires=300&amp;amp;X-Amz-Credential=ASIAQ3PHCVTYXRFWWLM4%2F20201216%2Fus-east-1%2Fs3%2Faws4_request&amp;amp;X-Amz-Signature=cc06ebbaa1077bc189259e745c70efb33cca4e74518ec350e67402abd3c13ef5&amp;amp;hash=0dbb9762c6cbd08c1c8d4c1cf8da00b894fb9d7f8b9c891b539af6fb19b5a64e&amp;amp;host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&amp;amp;pii=0893608094900418&amp;amp;tid=spdf-00e358ca-b1e1-4118-861d-6dd6fde51ff8&amp;amp;sid=553447379d6790444e490b36c0df667dd411gxrqb&amp;amp;type=client&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;A Deterministic Annealing Neural Network for Convex Programming.&amp;quot; Neural Networks vol. 7 no. 4 (1994): 629-641.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43011" LastEditorUserId="43011" LastEditDate="2020-12-16T15:00:39.677" LastActivityDate="2020-12-16T15:00:39.677" Title="What is $\frac{du}{dt}$ represent in recurrent neural network" Tags="&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25215" PostTypeId="1" CreationDate="2020-12-16T15:39:16.527" Score="2" ViewCount="56" Body="&lt;p&gt;I am working on image segmentation of MRI thigh images with deep learning (Unet). I noticed that I get a higher average dice accuracy over my predicted masks if I have less samples in the test data set.&#xA;I am calculating it in tensorflow as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def dice_coefficient(y_true, y_pred, smooth=0.00001):&#xA;y_true_f = K.flatten(y_true)&#xA;y_pred_f = K.flatten(y_pred)&#xA;intersection = K.sum(y_true_f * y_pred_f)&#xA;return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the difference is 0.003 if I have 4x more samples.&lt;/p&gt;&#xA;&lt;p&gt;I am calculating the dice coefficient over each MRI 2D slice&lt;/p&gt;&#xA;&lt;p&gt;Why could this be?&lt;/p&gt;&#xA;&lt;p&gt;This figure shows how the accuracy decreases with the fraction of samples. I start with 0.1 of the data until the whole data set. The splitting of the data was random &lt;a href=&quot;https://i.stack.imgur.com/xRppE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xRppE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42357" LastEditorUserId="42357" LastEditDate="2020-12-22T20:21:50.970" LastActivityDate="2020-12-22T20:21:50.970" Title="Why do I get higher average dice accuracy for less data" Tags="&lt;tensorflow&gt;&lt;data-science&gt;&lt;u-net&gt;&lt;accuracy&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25216" PostTypeId="1" AcceptedAnswerId="25219" CreationDate="2020-12-16T15:44:52.873" Score="2" ViewCount="48" Body="&lt;p&gt;I have a question about the training data used during the update/back-propagation step of the neural network in AlphaZero.&lt;/p&gt;&#xA;&lt;p&gt;From the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The data for each time-step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is stored as (&lt;span class=&quot;math-container&quot;&gt;$s_t, \pi_t, z_t$&lt;/span&gt;) where &lt;span class=&quot;math-container&quot;&gt;$z_t = \pm r_T$&lt;/span&gt; is the game winner from the perspective of the current player at step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. In parallel (Figure 1b), new network parameters &lt;span class=&quot;math-container&quot;&gt;$\Theta_i$&lt;/span&gt; are trained from data (&lt;span class=&quot;math-container&quot;&gt;$s,\pi, z$&lt;/span&gt;) sampled uniformly among all time-steps of the last iteration(s) of self-play&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Regarding the policy at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;), I understood this as the probability distribution of taking some action that is proportional to the visit count to each child node, i.e. during MCTS, given some parent node (state) at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, if some child node (subsequent state) &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is visited &lt;span class=&quot;math-container&quot;&gt;$N_a$&lt;/span&gt; times and all children nodes are visited &lt;span class=&quot;math-container&quot;&gt;$\sum_b N_b$&lt;/span&gt; times, then the probability of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (and its corresponding move) being sampled is &lt;span class=&quot;math-container&quot;&gt;$\frac{N_a}{\sum_b N_b}$&lt;/span&gt;, and this parametrizes the distribution &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;. Is this correct? If this is the case, then for some terminal state &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, we can't parametrize a distribution because we have no children nodes (states) to visit. Does that mean we don't add (&lt;span class=&quot;math-container&quot;&gt;$s_T, \pi_T, z_T$&lt;/span&gt;) to the training data?&lt;/p&gt;&#xA;&lt;p&gt;Also, a followup question regarding the loss function:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$l = (z-v)^2 - \pi^T log\textbf{p} + c||\Theta||^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm confused about this &lt;span class=&quot;math-container&quot;&gt;$\pi^T$&lt;/span&gt; notation. My best guess is that this is a vector of actions sampled from all policies in the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; X &lt;span class=&quot;math-container&quot;&gt;$(s_t, \pi_t, z_t)$&lt;/span&gt; minibatch, but I'm not sure. (PS the &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; used in &lt;span class=&quot;math-container&quot;&gt;$\pi^T$&lt;/span&gt; is different from the &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; used to denote a terminal state if you look at the paper. Sorry for the confusion, I don't know how to write two different looking T's)&lt;/p&gt;&#xA;" OwnerUserId="43016" LastEditorUserId="2444" LastEditDate="2020-12-17T01:22:16.437" LastActivityDate="2020-12-17T01:22:16.437" Title="In AlphaZero, do we need to store the data of terminal states?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;alphazero&gt;&lt;chess&gt;&lt;notation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25217" PostTypeId="1" AcceptedAnswerId="25225" CreationDate="2020-12-16T15:58:47.593" Score="0" ViewCount="87" Body="&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1706.03762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is all you need&lt;/a&gt; paper, on the 4th page, we have equation 1, which describes the self-attention mechanism of the transformer architecture&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text { Attention }(Q, K, V)=\operatorname{softmax}\left(\frac{Q K^{T}}{\sqrt{d_{k}}}\right) V&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Everything is fine up to here.&lt;/p&gt;&#xA;&lt;p&gt;Then they introduce the multi-head attention, which is described by the following equation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\text { MultiHead }(Q, K, V) &amp;amp;=\text { Concat}\left(\text {head}_{1}, \ldots, \text {head}_{\mathrm{h}}\right) W^{O} \\&#xA;\text { where head}_{\mathrm{i}} &amp;amp;=\text {Attention}\left(Q W_{i}^{Q}, K W_{i}^{K}, V W_{i}^{V}\right)&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Once the multi-head attention is motivated at the end of page 4, they state that for a single head (the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th head), the query &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and key &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; inputs are first linearly projected by &lt;span class=&quot;math-container&quot;&gt;$W_i^Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_i^K$&lt;/span&gt;, then dot product is calculated, let's say &lt;span class=&quot;math-container&quot;&gt;$Q_i^p =  Q W_i^Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K_i^p = K W_i^K$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, the dot product of the projected query and key becomes the following from simple linear algebra.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_i^p {K_i^p}^\intercal = Q W_i^Q {W_i^K}^T K^T =  Q W_i K^T,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W_i = W_i^Q {W_i^K}^T$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is the outer product of query projection by the key projection matrix. However, it is a matrix with shape &lt;span class=&quot;math-container&quot;&gt;$d_{model} \times d_{model}$&lt;/span&gt;. Why did the authors not define only a &lt;span class=&quot;math-container&quot;&gt;$W_i$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$W_i^Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_i^K$&lt;/span&gt; pair which have &lt;span class=&quot;math-container&quot;&gt;$2 \times d_{model} \times d_{k}$&lt;/span&gt; elements? In deep learning applications, I think it would be very inefficient.&lt;/p&gt;&#xA;&lt;p&gt;Is there something that I am missing, like these 2 matrices &lt;span class=&quot;math-container&quot;&gt;$W_i^Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_i^K$&lt;/span&gt; should be separate because of this and that?&lt;/p&gt;&#xA;" OwnerUserId="43019" LastEditorUserId="43019" LastEditDate="2020-12-17T05:32:12.643" LastActivityDate="2020-12-17T05:32:12.643" Title="In the multi-head attention mechanism of the transformer, why do we need both $W_i^Q$ and ${W_i^K}^T$?" Tags="&lt;deep-learning&gt;&lt;transformer&gt;&lt;attention&gt;&lt;weights&gt;&lt;efficiency&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25218" PostTypeId="1" CreationDate="2020-12-16T16:03:02.620" Score="5" ViewCount="299" Body="&lt;p&gt;I'm dealing with a (stochastic) Multi Armed Bandit (MAB) with a large number of arms.&lt;/p&gt;&#xA;&lt;p&gt;Consider a pizza machine that produces a pizza depending on an input &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; (equivalent to an arm). The (finite) set of arms &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is given by &lt;span class=&quot;math-container&quot;&gt;$K=X_1\times X_2 \times X_3\times X_4$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$X_j$&lt;/span&gt; denote the set of possible amounts of ingredient &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;e.g.&#xA;&lt;span class=&quot;math-container&quot;&gt;$X_1=\{$&lt;/span&gt; small, medium, large &lt;span class=&quot;math-container&quot;&gt;$\}$&lt;/span&gt; (amount of cheese) or&#xA;&lt;span class=&quot;math-container&quot;&gt;$X_2=\{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10\}$&lt;/span&gt; (slices of salami)&lt;/p&gt;&#xA;&lt;p&gt;Thus, running the pizza machine with input &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is equivalent to pulling arm &lt;span class=&quot;math-container&quot;&gt;$i\in K$&lt;/span&gt;. Due to the different permutations, the number of arms &lt;span class=&quot;math-container&quot;&gt;$|K|$&lt;/span&gt; is very large (between 100,000 and 1,000,000). Depending on the pulled arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, the machine generates a pizza (associated with a reward that indicates how delicious the pizza is).&#xA;However, the machine's rewards are non-static. Pulling an arm &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; generates a reward according to an unknown (arm specific) distribution &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt;, with all rewards drawn from &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt; beeing i.i.d.. In addition, it is possible to normalize all rewards to the interval [0,1].&lt;/p&gt;&#xA;&lt;p&gt;The above problem corresponds to the standard problem of a stochastic MAB, but is characterized by the large number of arms. In the case of the pizza machine, several days of computation time are available to determine the best pizza, so the number of itarations is allowed to be large as well.&lt;/p&gt;&#xA;&lt;p&gt;In my investigation of MAB algorithms addressing a large number of arms, I came across studies that could call up to a few thousand arms.&lt;/p&gt;&#xA;&lt;p&gt;Are there algorithms that exist in the MAB domain, which specifically deal with large problem instances (e.g.with &lt;span class=&quot;math-container&quot;&gt;$|K|&amp;gt;100,000$&lt;/span&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="21287" LastEditorUserId="43231" LastEditDate="2020-12-27T14:44:43.227" LastActivityDate="2021-05-26T16:04:40.447" Title="Multi Armed Bandits with large number of arms" Tags="&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;epsilon-greedy-policy&gt;&lt;upper-confidence-bound&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25219" PostTypeId="2" ParentId="25216" CreationDate="2020-12-16T16:21:14.783" Score="0" Body="&lt;p&gt;I'm not 100% sure whether or not they added any data for terminal game states, but it's very reasonable to indeed make the choice not to include data for terminal game states. As you rightly pointed out, we don't have any meaningful targets to update the policy head towards in those cases, and this is not really a problem because we would also never actually make use of the policy output in a terminal game state. For the value head we &lt;em&gt;could&lt;/em&gt; provide meaningful targets to update towards, but again we would never actually have to make use of such outputs; if we encounter a terminal game state in a tree search, we just back up the true value of that terminal game state instead of making a call to the network to obtain a value function approximation.&lt;/p&gt;&#xA;&lt;p&gt;In theory, I could imagine some cases where training the value head on terminal game states might be slightly beneficial despite not being strictly necessary; it could enable generalisation to similar game states that are not terminal (but close to being terminal), and speed up learning for those. For example, if you have a game where the goal is to complete a line of &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; pieces, training the value head on terminal states where you actually have a line of &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; pieces and have entirely won the game might generalise and speed up learning for similar game states where you may not yet have &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt; pieces in a line, but are very close to that goal. That said, intuitively I really don't feel like this would provide a big benefit (if any), and we could probably also come up with cases where it would be harmful.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In the &lt;span class=&quot;math-container&quot;&gt;$\pi^{\text{T}}$&lt;/span&gt; notation, &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; is a vector (for any arbitrary time step, the time step is not specified here) containing a discrete probability distribution over actions (visit counts of MCTS, normalised into a probability distribution), and the &lt;span class=&quot;math-container&quot;&gt;$\text{T}$&lt;/span&gt; simply denotes that we take the transpose of that vector. Personally I don't like the notation though, I prefer something like &lt;span class=&quot;math-container&quot;&gt;$\pi^{\top}$&lt;/span&gt; which is more clearly distinct from a letter &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\text{T}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, once you understand that to denote the transpose, you'll see that &lt;span class=&quot;math-container&quot;&gt;$\pi^{\top}\log(\mathbf{p})$&lt;/span&gt; is a dot product between two vectors, which then ends up being a single scalar.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-12-16T16:21:14.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25220" PostTypeId="1" CreationDate="2020-12-16T16:26:29.830" Score="1" ViewCount="72" Body="&lt;p&gt;I have a question regarding the action space of the policy network used in AlphaZero.&lt;/p&gt;&#xA;&lt;p&gt;From the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We represent the policy π(a|s) by a 8 × 8 × 73 stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8 × 8 positions identifies the square from which to “pick up” a piece. The first 56 planes encode possible ‘queen moves’ for any piece: a number of squares [1..7] in which the piece will be moved, along one of eight relative compass directions {N,NE,E,SE,S,SW,W,NW}......The policy in Go is represented identically to AlphaGo Zero (29), using a flat distribution over 19 × 19 + 1 moves representing possible stone placements and the pass move. We also tried using a flat distribution over moves for chess and shogi; the final result was almost identical although training was slightly slower.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't understand why a stack of planes is used for the action space here. I'm also not entirely sure I understand how this representation is used. My guess is that for Chess, the 8x8 plane represents the board, and each square has a probability assigned to it of picking up a piece on that square (let's assume that all illegal moves haven't been masked yet, so all squares have probability mass on them). From there, we choose from possible 'Queen' type moves or 'Knight' type moves, which total to 73 different types of moves. Is this interpretation correct? How would one go from this representation to sampling a legal move (i.e. how is this used to parametrize a distribution I can actually sample moves from?)&lt;/p&gt;&#xA;&lt;p&gt;During MCTS when expanding a leaf node, we get &lt;span class=&quot;math-container&quot;&gt;$p_a$&lt;/span&gt;, the probability of taking some action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; from the policy head, so I would also need to be able to go from this 'planes' representation to the probability of taking a specific action.&lt;/p&gt;&#xA;&lt;p&gt;The paper also mentions trying out 'flat distributions', which I'm not entirely sure what this means either.&lt;/p&gt;&#xA;" OwnerUserId="43016" LastEditorUserId="43016" LastEditDate="2020-12-16T20:14:58.507" LastActivityDate="2020-12-16T20:14:58.507" Title="Stack of Planes as the Action Space Representation for AlphaZero (Chess)" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;deep-rl&gt;&lt;chess&gt;&lt;alphazero&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25221" PostTypeId="1" AcceptedAnswerId="25224" CreationDate="2020-12-16T17:20:13.787" Score="1" ViewCount="56" Body="&lt;p&gt;Does the image is logistic regression or SVM, and why?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3AYfN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3AYfN.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33475" LastEditorUserId="33475" LastEditDate="2020-12-16T23:49:10.360" LastActivityDate="2020-12-16T23:49:10.360" Title="Does the image is logistic regression or SVM, and why?" Tags="&lt;machine-learning&gt;&lt;support-vector-machine&gt;&lt;logistic-regression&gt;" AnswerCount="1" CommentCount="9" ClosedDate="2020-12-17T20:47:29.690" ContentLicense="CC BY-SA 4.0" />
  <row Id="25222" PostTypeId="1" AcceptedAnswerId="25230" CreationDate="2020-12-16T19:56:54.663" Score="1" ViewCount="33" Body="&lt;p&gt;In articles that describe neural architectures with multiple attention layers of the same form, are the weight matrices usually the same across the layers?  Consider as an example, &amp;quot;Attention is all you need&amp;quot;. The authors stack several layers of multi-head self-attention in which each layer has the same number of heads.  Each head &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; involves a trainable weight matrix &lt;span class=&quot;math-container&quot;&gt;$W_{i}^{Q}$&lt;/span&gt;.  There is no subscript, superscript, or any other indication that this matrix is different for each layer.  My questions is this: are there separate &lt;span class=&quot;math-container&quot;&gt;$W_{i}^{Q}$&lt;/span&gt; for layers &lt;span class=&quot;math-container&quot;&gt;$1,2,3,...$&lt;/span&gt; or is this a single matrix shared throughout layers?&lt;/p&gt;&#xA;&lt;p&gt;My intuition is that the authors of the paper wanted to cut down on notation, and that the matrices are different in different layers.  But I want to be sure I understand this, since I see the same kind of thing in many other papers as well.&lt;/p&gt;&#xA;" OwnerUserId="43025" LastActivityDate="2020-12-17T01:45:18.143" Title="In attention models with multiple layers, are weight matrices shared across layers?" Tags="&lt;transformer&gt;&lt;attention&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25224" PostTypeId="2" ParentId="25221" CreationDate="2020-12-16T20:49:27.390" Score="2" Body="&lt;p&gt;The straight dashed-line shows the typical decision line in logistic regression or any linear classifier. The dashed-circle shows the decision line from SVM.&#xA;Obviously, since the data is not linearly separable in the original 2D feature space, if someone makes a higher dimension space by taking into account non-linear interaction of the original 2 features then they can discriminate between x and o data using a linear discriminator applied in higher dimensions. This shows the beauty of kernel methods that can make a linear yet high-dimensional (infinite dimensions indeed) problem from a non-linear low-dimensional problem (finite dimensions actually).&lt;/p&gt;&#xA;" OwnerUserId="42330" LastActivityDate="2020-12-16T20:49:27.390" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25225" PostTypeId="2" ParentId="25217" CreationDate="2020-12-16T20:53:41.610" Score="0" Body="&lt;p&gt;I'll use notation from the paper you cited, and any other readers should refer to the paper (widely available) for definitions of notation.  The utility of using &lt;span class=&quot;math-container&quot;&gt;$W^Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W^K$&lt;/span&gt;, rather than &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, lies in the fact that they allow us to add fewer parameters to our architecture.  &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; has dimension &lt;span class=&quot;math-container&quot;&gt;$d_{model} \times d_{model}$&lt;/span&gt;, which means that we are adding &lt;span class=&quot;math-container&quot;&gt;$d_{model}^2$&lt;/span&gt; parameters to our architecture.  &lt;span class=&quot;math-container&quot;&gt;$W^Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W^K$&lt;/span&gt; each have dimension &lt;span class=&quot;math-container&quot;&gt;$d_{model} \times d_k$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$d_k=\frac{d_{model}}{h}$&lt;/span&gt;. If we use these two matrices, we only add &lt;span class=&quot;math-container&quot;&gt;$2\frac{d_{model}^2}{h}$&lt;/span&gt; parameters to our architecture, even though their multiplication (with the transpose) allows us to have the correct dimensions for matrix multiplication with &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We do use &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; attention heads, which then brings our number of parameters back up, but the multiple heads let the model attend to different pieces of information in our data.&lt;/p&gt;&#xA;" OwnerUserId="43025" LastEditorUserId="43025" LastEditDate="2020-12-16T21:01:58.257" LastActivityDate="2020-12-16T21:01:58.257" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25226" PostTypeId="1" CreationDate="2020-12-16T21:00:21.683" Score="0" ViewCount="58" Body="&lt;p&gt;From &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the AlphaZero paper&lt;/a&gt;, the caption of Table S1 (p. 13)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Table S1: Input features used by AlphaZero in Go, Chess and Shogi respectively. The first set of features are repeated for each position in a &lt;span class=&quot;math-container&quot;&gt;$T = 8$&lt;/span&gt;-step history. Counts are represented by a single real-valued input; other input features are represented by a one-hot encoding using the specified number of binary input planes. The current player is denoted by P1 and the opponent by P2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I wanted to get some clarification on which features are represented by single real-valued inputs, and which &amp;quot;other input features&amp;quot; are represented by one-hot encodings.&lt;/p&gt;&#xA;&lt;p&gt;I'm also not sure what the line&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;other input features are represented by a one-hot encoding using the specified number of binary input planes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;means here.&lt;/p&gt;&#xA;&lt;p&gt;For Chess, specifically, we have the following features): P1 Piece, P2 Piece, Repetitions, Colour, Total move count, P1 Castling, P2 Castling, No-progress count.&lt;/p&gt;&#xA;&lt;p&gt;I think I understand P1 Piece and P2 Piece, these are just 6 binary planes, one for each piece type, that denotes whether a piece exists for each of the 64 squares on the board. Correct me if I'm wrong.&lt;/p&gt;&#xA;&lt;p&gt;Repetitions, Colour, total move count, P1 Castling, P2 Castling, and No-progress count - are these all represented by &amp;quot;single real-valued input&amp;quot;? For example, if it's White's turn to move, and white is represented as 1, does this mean the plane for color is an 8x8 matrix of 1s? Another example: if the move count is 114, then the input plane for this feature is an 8x8 matrix of 114s...is this correct?&lt;/p&gt;&#xA;&lt;p&gt;Follow up question: why are there 2 planes for repetitions?&lt;/p&gt;&#xA;" OwnerUserId="43016" LastEditorUserId="2444" LastEditDate="2020-12-19T18:19:17.033" LastActivityDate="2020-12-19T18:19:17.033" Title="In AlphaZero, which features are one-hot encoded and which are single real-valued?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;alphazero&gt;&lt;features&gt;&lt;feature-engineering&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25228" PostTypeId="1" AcceptedAnswerId="25234" CreationDate="2020-12-16T23:04:52.357" Score="1" ViewCount="526" Body="&lt;p&gt;Can someone explain to me with a proof or example why you can't linearly separate XOR (and therefore need a neural network, the context I'm looking at it in)?&lt;/p&gt;&#xA;&lt;p&gt;I understand why it's not linearly separable if you draw it graphically (e.g. &lt;a href=&quot;https://medium.com/@lucaspereira0612/solving-xor-with-a-single-perceptron-34539f395182&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), but I can't seem to find a formal proof somewhere, I wanted to try and understand it with either an equation or example written down. I'm wondering if one exists (I guess it has to do with contradictions?), but I can't seem to find it? I have seen &lt;a href=&quot;https://ai.stackexchange.com/q/16291/2444&quot;&gt;this&lt;/a&gt;, but it's more a reason than a proof.&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="2444" LastEditDate="2021-01-19T02:19:37.883" LastActivityDate="2021-01-19T02:19:37.883" Title="Is there a proof to explain why XOR cannot be linearly separable?" Tags="&lt;proofs&gt;&lt;perceptron&gt;&lt;xor-problem&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25229" PostTypeId="2" ParentId="25217" CreationDate="2020-12-17T00:57:52.470" Score="0" Body="&lt;p&gt;In practice, matrices &lt;span class=&quot;math-container&quot;&gt;$W^Q, W^K, W^V$&lt;/span&gt; (each of size &lt;span class=&quot;math-container&quot;&gt;$d_{model}$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$d_{model}$&lt;/span&gt;) are completely removed instead, and Transformer implementations just learn a single set of matrices &lt;span class=&quot;math-container&quot;&gt;$\{ W_i^{Q*}, W_i^{K*}, W_i^{V*} \}$&lt;/span&gt; (each of size &lt;span class=&quot;math-container&quot;&gt;$d_{model}$&lt;/span&gt; x &lt;span class=&quot;math-container&quot;&gt;$\frac{d_{model}}{h}$&lt;/span&gt;) for each head, where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_i^{Q*} = W^Q W_i^Q \\&#xA;W_i^{K*} = W^K W_i^K \\&#xA;W_i^{V*} = W^V W_i^V $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;so that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Q_i(x) = x W_i^{Q*} = x W^Q W_i^{Q*} = Q W_i^Q\\&#xA;K_i(x) = x W_i^{K*} = x W^K W_i^{K*} = K W_i^K\\&#xA;V_i(x) = x W_i^{V*} = x W^V W_i^{V*} = V W_i^V\\&#xA;head_i(x) = softmax \left(\frac{Q_i(x) K_i(x)^T}{\sqrt{d_k}} \right) V_i(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I can confirm this with the original Transformer implementation in Tensor2Tensor, and also the BERT code that uses the encoding part of the Transformer.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-17T00:57:52.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25230" PostTypeId="2" ParentId="25222" CreationDate="2020-12-17T01:45:18.143" Score="1" Body="&lt;p&gt;Weights are not normally shared across Transformer layers in vanilla Transformers. However, there has been research done in testing out sharing weights, and sometimes they improve the scores. Here are some examples:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1909.11942.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ALBERT&lt;/a&gt; is an improvement on BERT (so only uses the encoding side, no decoder), and shows that sharing the attention weights only &lt;span class=&quot;math-container&quot;&gt;$\left\{ W_i^Q, W_i^K, W_i^V \right\}$&lt;/span&gt; across all Transformer layers for large networks either result in the same accuracy or slightly improved accuracy, while significantly reducing model size. Sharing the position-wise FFN layer though hindered performance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1910.10683.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Text-to-Text Transfer Transformer&lt;/a&gt; shows that sharing the weights between encoder and decoder layer of the transformer (so e.g. layer 1 encoding weights = layer 1 decoding weights) barely affected accuracy (it dropped by 0.5%), but the model size is halved.&lt;/p&gt;&#xA;&lt;p&gt;I'm sure there are more papers I have forgotten. Sharing weights is still an active area of research, but for vanilla transformers it is assumed they do not share weights.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-17T01:45:18.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25231" PostTypeId="5" CreationDate="2020-12-17T01:48:23.880" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T01:48:23.880" LastActivityDate="2020-12-17T01:48:23.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25232" PostTypeId="4" CreationDate="2020-12-17T01:48:23.880" Score="0" Body="For questions about intelligent personal assistants (IPAs), aka intelligent virtual assistants (IVA) or simply chatbots, which are software agents that can perform tasks or services for an individual based on commands or questions. A famous IPA is, for example, Siri." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T01:48:23.880" LastActivityDate="2020-12-17T01:48:23.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25234" PostTypeId="2" ParentId="25228" CreationDate="2020-12-17T02:49:24.000" Score="1" Body="&lt;p&gt;Before proving that XOR cannot be linearly separable, we first need to prove a  lemma:&lt;/p&gt;&#xA;&lt;h2&gt;Lemma 1&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Lemma:&lt;/strong&gt; If 3 points are collinear and the middle point has a different label than the other two, then these 3 points cannot be linearly separable.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Uzig.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Uzig.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Proof:&lt;/strong&gt; Let us label the points as point &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; have the same label, and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; has a different label. They are all collinear with line &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Assume the contradiction, so a line can linearly separate &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. This means a line must cross between segment &lt;span class=&quot;math-container&quot;&gt;$AB$&lt;/span&gt;, and segment &lt;span class=&quot;math-container&quot;&gt;$BC$&lt;/span&gt; to linearly separate these three points (by definition of linear separability). Let us label the point where the line crosses segment &lt;span class=&quot;math-container&quot;&gt;$AB$&lt;/span&gt; and point &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, and the point where the line corsses segment &lt;span class=&quot;math-container&quot;&gt;$BC$&lt;/span&gt; and point &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, since segments &lt;span class=&quot;math-container&quot;&gt;$AB$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$BC$&lt;/span&gt; are collinear to line &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;, points &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; also falls on line &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;. Since only one unique line can cross 2 points, it must be that the only line that passes segments &lt;span class=&quot;math-container&quot;&gt;$AB$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$BC$&lt;/span&gt; and (therefore separates points &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;) is line &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, line &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; cannot linearly separate &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, since line &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; also crosses them. Therefore, no line exists can separate &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Main proof&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G7g23.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G7g23.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Consider these 4 points that represent a XOR table. Let us label them clock-wise, so the top-left point as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, top-right point as &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, bottom-right point as &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, and bottom-left point as &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. So &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; have the same label, and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; have the same label. We want to show that points &lt;span class=&quot;math-container&quot;&gt;$A, B, C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; cannot be linearly separable.&lt;/p&gt;&#xA;&lt;p&gt;Assume the contradiction, and that there is a line that can separate these 4 points.&lt;/p&gt;&#xA;&lt;p&gt;Imagine a fifth point that lies in the center, and let us label this as point &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; lies in the center, the three points &lt;span class=&quot;math-container&quot;&gt;$A, E$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; are collinear. Similarly, since &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; lies in the center, the three points &lt;span class=&quot;math-container&quot;&gt;$B, E$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; are collinear.&lt;/p&gt;&#xA;&lt;p&gt;Because we assume a line can linearly separate &lt;span class=&quot;math-container&quot;&gt;$A, B, C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, then this line must label point &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; as some label. If &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; shares the same label as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, then the points &lt;span class=&quot;math-container&quot;&gt;$B, E$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; will become &amp;quot;collinear points where the middle point has a different label&amp;quot;, which by Lemma 1 cannot be linearly separable. Likewise, if &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; shares the same label as &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, then the points &lt;span class=&quot;math-container&quot;&gt;$A, E$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; will become &amp;quot;collinear points where the middle point has a different label&amp;quot;, which by Lemma 1 cannot be linearly separable.&lt;/p&gt;&#xA;&lt;p&gt;Therefore it is impossible to give a label to &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; while satisfying linear separability. As a result, our assumption must be false, and the four points &lt;span class=&quot;math-container&quot;&gt;$A, B, C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; cannot be linearly separable.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-17T09:57:04.130" LastActivityDate="2020-12-17T09:57:04.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25235" PostTypeId="1" CreationDate="2020-12-17T03:33:48.140" Score="0" ViewCount="21" Body="&lt;p&gt;I'm trying to model a &lt;strong&gt;generative model for classification problem&lt;/strong&gt;, especially aiming to solve an imbalanced data problem.&lt;/p&gt;&#xA;&lt;p&gt;However, I couldn't get intuitive understanding for &lt;strong&gt;generative classifier in terms of deep learning&lt;/strong&gt; (or machine learning). Classical generative classifiers such as Bayes classifier are clear with such a closed form of equations.&lt;/p&gt;&#xA;&lt;p&gt;In a viewpoint of '&lt;strong&gt;neural network&lt;/strong&gt;', it is necessary to model a network structure with input, output, and loss function to update parameters. Assuming class distribution &lt;span class=&quot;math-container&quot;&gt;$p(y)$&lt;/span&gt; is known, network should be like below, I think:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fernX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fernX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Q1) I wonder that the network structure makes sense for classification even though the diagram is very simplified version. Additionally, &lt;strong&gt;how can I return the log likelihood function &lt;span class=&quot;math-container&quot;&gt;$\log p_{\theta} (x | y)$&lt;/span&gt; as an output layer in keras?&lt;/strong&gt; (How can I constitute the network structure in keras?)&lt;/p&gt;&#xA;&lt;p&gt;Q2) How is &lt;strong&gt;loss function&lt;/strong&gt; defined to update &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="25846" LastEditorUserId="25846" LastEditDate="2020-12-20T22:35:52.670" LastActivityDate="2020-12-20T22:35:52.670" Title="Network structure of generative model for classification" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;keras&gt;&lt;generative-model&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25236" PostTypeId="2" ParentId="16187" CreationDate="2020-12-17T05:31:30.677" Score="0" Body="&lt;p&gt;Discriminative models give the probability of an element in the feature space &lt;span class=&quot;math-container&quot;&gt;$x \in X$&lt;/span&gt; belonging to a class &lt;span class=&quot;math-container&quot;&gt;$y \in Y$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$p(Y|X)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is the set of classes in a classification problem.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;em&gt;discriminative feature representation&lt;/em&gt; in this context means the feature map/s  which are the output/s of the convolutional layers of the backbone (the convolutional layers of the neural network) which (presumably) differ on the basis of their class, and can be used to discriminate which class the original image belongs to, in the case of image classification.&lt;/p&gt;&#xA;" OwnerUserId="30962" LastEditorUserId="30962" LastEditDate="2020-12-17T05:58:01.333" LastActivityDate="2020-12-17T05:58:01.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25237" PostTypeId="1" CreationDate="2020-12-17T10:04:28.317" Score="0" ViewCount="17" Body="&lt;p&gt;The ROC-AUC curve is invariant under a flip of the labels. I don't know if it's a famous result, so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not been able to prove or disprove it yet.&lt;/p&gt;&#xA;&lt;p&gt;The reason this is important is that &lt;a href=&quot;https://www.biostat.wisc.edu/%7Epage/rocpr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;when data sets are highly imbalanced ROC-AUC curves perform well mechanically and it is better to look at PR-AUC curves&lt;/a&gt;. This obviously raises the question of a better curve for PR-AUC for one class being better for the other class as well or not.&lt;/p&gt;&#xA;&lt;h3&gt;Proof that ROC-AUC is invariant under label flip&lt;/h3&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{array}{|c|c|c|}&#xA;\hline&#xA;&amp;amp; \text{Predicted positive} &amp;amp; \text{Predicted negative} \\&#xA;\hline&#xA;\hline&#xA;\text{Positives} &amp;amp; TP(\xi) &amp;amp; FN(\xi) \\&#xA;\hline&#xA;\text{Negatives} &amp;amp; FP(\xi) &amp;amp; TN(\xi) \\&#xA;\hline&#xA;\end{array}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\xi \in [0,1]$&lt;/span&gt; is the output of a classifier.&lt;/p&gt;&#xA;&lt;p&gt;For a fixed dataset we have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;TP(\xi) + FN(\xi) = P \\&#xA;FP(\xi)+TN(\xi) = N&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; are actual number of negatives and positives. The ROC-AUC defined wrt to this definition of positives and negatives is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = \frac{1}{PN} \int_0^1 TP(\xi)~d FP(\xi)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now lets look at what happens under a label flip. We get&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{eqnarray}&#xA;\tilde A &amp;amp;=&amp;amp; \frac{1}{PN} \int_1^0 TN(\xi)~d FN(\xi) \\&#xA;&amp;amp;=&amp;amp; - \frac{1}{PN} \int_1^0 (N-FP(\xi)~d TP(\xi) \\&#xA;&amp;amp;=&amp;amp;  \frac{1}{PN} \int_0^1 (N-FP(\xi)~d TP(\xi) \\&#xA;&amp;amp;=&amp;amp; \frac{1}{P} TP(\xi) |_0^1 - \frac{1}{PN} (TP(\xi))(FP(\xi))|_0^1 + A \\&#xA;&amp;amp;=&amp;amp; A&#xA;\end{eqnarray}&#xA;$$&lt;/span&gt;&#xA;where the last equality follows from &lt;span class=&quot;math-container&quot;&gt;$TP(0)=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$TP(1)=P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$TN(1)=N$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21511" LastEditorUserId="2444" LastEditDate="2020-12-17T11:21:27.037" LastActivityDate="2020-12-17T11:21:27.037" Title="Is the PR AUC invariant under label flip?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;proofs&gt;&lt;imbalanced-datasets&gt;&lt;roc-auc&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25238" PostTypeId="1" CreationDate="2020-12-17T10:39:57.233" Score="0" ViewCount="46" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HdmQP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HdmQP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to understand RMSprop for a long time, but there's something that keeps eluding me.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$dW$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$db$&lt;/span&gt; are matrices (that's what I understand from the element-wise comment), so that must mean that &lt;span class=&quot;math-container&quot;&gt;$S_{dW}$&lt;/span&gt; is a matrix (or tensor) as well.&lt;/p&gt;&#xA;&lt;p&gt;So, in the update rule, they divide a matrix by another matrix? From what I saw in google, no such action exists.&lt;/p&gt;&#xA;" OwnerUserId="43043" LastEditorUserId="2444" LastEditDate="2020-12-18T00:47:20.937" LastActivityDate="2021-05-17T03:04:30.470" Title="RMSprop equation - dividing by a matrix?" Tags="&lt;deep-learning&gt;&lt;optimization&gt;&lt;gradient-descent&gt;&lt;optimizers&gt;&lt;rms-prop&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25239" PostTypeId="1" CreationDate="2020-12-17T10:49:21.657" Score="1" ViewCount="56" Body="&lt;p&gt;Let's say we are training a new neural network from scratch. I calculate the mean and standard deviation of my dataset (assume I am training a fully convolutional neural net and my dataset is images) and I standardise each channel of all images based on that mean and standard deviation. My output will be another image.&lt;/p&gt;&#xA;&lt;p&gt;I want to use for example VGG for perceptual loss (VGG's weights will be frozen). Perceptual loss is when you input your prediction to a pretrained network to extract features from it. Then you do the same for the ground truth and the L2 distance between the features from ground truth and features from prediction is called perceptual loss.&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, I am supposed to standardise my data based on the mean and standard deviation VGG was trained with (since I am using VGG for inference essentially), which is different than the mean and standard deviation of my dataset. What is the correct way to do this? &lt;strong&gt;Should I undo the standardization of my dataset by multiplying standard deviation and adding the original mean to the output of my network, and then restandardise using VGG's statistics to calculate the loss?&lt;/strong&gt; Or should I continue without restandardising?&lt;/p&gt;&#xA;" OwnerUserId="19549" LastEditorUserId="19549" LastEditDate="2020-12-22T14:02:53.010" LastActivityDate="2021-05-21T16:02:10.183" Title="How to normalize for perceptual loss when training neural net from scratch?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25240" PostTypeId="2" ParentId="25210" CreationDate="2020-12-17T11:01:04.970" Score="2" Body="&lt;p&gt;There are quite a few examples of papers where they try and 'teach' neural networks to 'learn' how to solve math problems. Most of the time, sadly, it comes down to training on a large dataset after which the network can 'solve' the sort of basic problems, but is unable to generalize this to larger problems. That is, if you train a neural network to solve addition, it will be inherently constrained by the dataset. It might be able to semi-sufficiently solve addition with 3 or even 4 digits, depending on how big your dataset is, but throw in an addition question containing 2 10 digit numbers, and it will almost always fail.&lt;/p&gt;&#xA;&lt;p&gt;The latest example that I can remember where they tried this is in the General Language Model GPT-3, which was not made to solve equations per se, but does 'a decent job' on the stuff that was in the dataset. Facebook AI made an 'advanced math solver' with a specific architecture that i have not looked into which might disproof my point, but you can look into that.&lt;/p&gt;&#xA;&lt;p&gt;In the end, this comes down to 'what is learning' and 'what do you want to accomplish'. Most agree that these network are not able to generalize beyond their datasets. Some might say that not being able to generalize does not mean that it is not learning. It might just be learning slower. I believe that these models are inherently limited to what is presented in the dataset. Given a good dataset, it might be able to generalize to cases 'near and in-between', but I have yet to see a case where this sort of stuff generalizes to cases 'far outside' the dataset.&lt;/p&gt;&#xA;" OwnerUserId="34383" LastActivityDate="2020-12-17T11:01:04.970" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25241" PostTypeId="5" CreationDate="2020-12-17T11:54:10.343" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T11:54:10.343" LastActivityDate="2020-12-17T11:54:10.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25242" PostTypeId="4" CreationDate="2020-12-17T11:54:10.343" Score="0" Body="For questions related to discriminative models, such as logistic regression or decision trees." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T11:54:10.343" LastActivityDate="2020-12-17T11:54:10.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25243" PostTypeId="5" CreationDate="2020-12-17T11:55:18.210" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T11:55:18.210" LastActivityDate="2020-12-17T11:55:18.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25244" PostTypeId="4" CreationDate="2020-12-17T11:55:18.210" Score="0" Body="For questions related to the discriminator component of GANs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T11:55:18.210" LastActivityDate="2020-12-17T11:55:18.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25245" PostTypeId="5" CreationDate="2020-12-17T11:55:44.580" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T11:55:44.580" LastActivityDate="2020-12-17T11:55:44.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25246" PostTypeId="4" CreationDate="2020-12-17T11:55:44.580" Score="0" Body="For questions related to the generator component of GANs." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-17T11:55:44.580" LastActivityDate="2020-12-17T11:55:44.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25247" PostTypeId="2" ParentId="20214" CreationDate="2020-12-17T11:56:48.277" Score="0" Body="&lt;p&gt;This is also a question I stumble upon, thanks for the explaination from ted, it is very helpfull, I will try to elaborate a little bit. Let's still use DeepMind's Simon Osindero's &lt;a href=&quot;https://docs.google.com/presentation/d/e/2PACX-1vQwrivdqqBR8teLQ7prKtiDyMLSqgGBzTxfQ6BKXPVvpFpLRUQOmqTm57LEMIy3IIK14RTLcBcT-PCO/pub?start=false&amp;amp;loop=false&amp;amp;delayms=60000&amp;amp;slide=id.g1a727d4a2c_0_814&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide&lt;/a&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lV7Ty.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lV7Ty.jpg&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&#xA;The grey block on the left we are looking at is only a cross entropy operation, the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (a vector) could be the softmax output from previous layer (not the input for the neutral network), and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (a scalar) is the cross entropy result of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. To propagate the gradient back, we need to calculate the gradient of &lt;span class=&quot;math-container&quot;&gt;$dy/dx_i$&lt;/span&gt;, which is &lt;span class=&quot;math-container&quot;&gt;$-p_i/x_i$&lt;/span&gt; for each element in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&#xA;As we know the softmax function scale the logits into the range [0,1], so if in one training step, the neutral network becomes super confident and predict one of the probabilties &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; to be 0 then we have a numerical problem in calculting &lt;span class=&quot;math-container&quot;&gt;$dy/dx_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;While in the other case where we take the logits and calculate the softmax and crossentropy at one shot (XentLogits function), we don't have this problem. Because the derivative of XentLogits is &lt;span class=&quot;math-container&quot;&gt;$dy/dx_i = y - p_i$&lt;/span&gt;, a more elaborated derivation can be found &lt;a href=&quot;http://machinelearningmechanic.com/deep_learning/2019/09/04/cross-entropy-loss-derivative.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="43045" LastEditorUserId="43045" LastEditDate="2020-12-17T12:09:45.357" LastActivityDate="2020-12-17T12:09:45.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25248" PostTypeId="1" CreationDate="2020-12-17T13:02:40.863" Score="1" ViewCount="72" Body="&lt;p&gt;I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the exact same DDPG to solve Pendulum-v0 and it works, but my environment is a more complex with a continuous space/action space.&lt;/p&gt;&#xA;&lt;p&gt;How do you deal with local optima problem in reinforcement learning? is it just an exploitation issue?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;More details:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My state space is not pixels, it is numerical, in fact it's a metro line simulator and the state space is the velocity, the position of each train on the line and the number of passengers at each station. I need to control the different trains so I am not trying to control only one train but all the operational trains and each one can have different actions such as speed or not, stay longer on the next station or not etc.&lt;/p&gt;&#xA;&lt;p&gt;1/ I am using the same ANN architecture for the actor and critic: 3 FC layers with (512, 256, 256) hidden units.&lt;/p&gt;&#xA;&lt;p&gt;2/ Adam optimizer for both the actor and critic with a small lr=1e-7 and clipnorm=1.&lt;/p&gt;&#xA;&lt;p&gt;3/ nb_steps_warmup_critic=1000, nb_steps_warmup_actor=1000&lt;/p&gt;&#xA;&lt;p&gt;4/ SequentialMemory(limit=1000000, window_length=1)&lt;/p&gt;&#xA;&lt;p&gt;5/ The environement is a simulator of a metro line with a continuous state and action space&lt;/p&gt;&#xA;" OwnerUserId="43047" LastEditorUserId="43047" LastEditDate="2020-12-20T21:17:44.393" LastActivityDate="2020-12-20T21:17:44.393" Title="How to deal with KerasRL DDPG algorithm getting stuck in a local optima?" Tags="&lt;reinforcement-learning&gt;&lt;keras&gt;&lt;open-ai&gt;&lt;gym&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="25251" PostTypeId="1" CreationDate="2020-12-17T20:19:34.777" Score="0" ViewCount="28" Body="&lt;p&gt;Currently I'm working on a project that the input to the network is SWT of a signal which is 2D array. Currently I'm using &lt;strong&gt;batch normalization&lt;/strong&gt; to normalize inputs(Stationary Wavelet Transform of signals). Is it better to normalize the signal itself and then calculate SWT or is there a better normalization technique for this kind of input data?&lt;/p&gt;&#xA;" OwnerUserId="42948" LastEditorUserId="42948" LastEditDate="2020-12-21T06:55:16.660" LastActivityDate="2020-12-21T06:55:16.660" Title="What's the best way to normalize 2D input data?" Tags="&lt;batch-normalization&gt;&lt;normalisation&gt;&lt;weight-normalization&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25253" PostTypeId="1" CreationDate="2020-12-17T22:00:33.320" Score="0" ViewCount="38" Body="&lt;p&gt;I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise.&lt;/p&gt;&#xA;&lt;p&gt;However, I was wondering if attention, applied either to seq2seq RNN/GRU/LSTM or via Transformers, can contribute to improving the overall performance (as well as giving some sort of interpretability through the attention weights?) in the case of relatively short sequences (let's say around 20-30 elements each).&lt;/p&gt;&#xA;" OwnerUserId="43057" LastEditorUserId="2444" LastEditDate="2020-12-19T18:28:53.247" LastActivityDate="2021-05-18T20:04:20.367" Title="Can the attention mechanism improve the performance in the case of short sequences?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;transformer&gt;&lt;attention&gt;&lt;performance&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25254" PostTypeId="1" CreationDate="2020-12-17T23:15:37.383" Score="3" ViewCount="57" Body="&lt;p&gt;Two of the most popular initialization schemes for neural network weights today are &lt;a href=&quot;http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xavier&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/pdf/1502.01852v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;He&lt;/a&gt;. Both methods propose random weight initialization with a variance dependent on the number of input and output units. Xavier proposes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W \sim \mathcal{U}\Bigg[-\frac{\sqrt{6}}{\sqrt{n_{in}+n_{out}}},\frac{\sqrt{6}}{\sqrt{n_{in}+n_{out}}}\Bigg]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for networks with &lt;span class=&quot;math-container&quot;&gt;$\text{tanh}$&lt;/span&gt; activation function and He proposes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W \sim \mathcal{N}(0,\sqrt{s/n_{in}})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$\text{ReLU}$&lt;/span&gt; activation. Both initialization schemes are implemented in the most commonly used deep learning libraries for python, &lt;a href=&quot;https://pytorch.org/docs/stable/nn.init.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch&lt;/a&gt; and &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/initializers&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;.&lt;br /&gt;&#xA;However, &lt;strong&gt;for both versions we have a normal and uniform version&lt;/strong&gt;. Now the main argument of both papers is about the variance of the information at initialization time (which is dependent on the non-linearity) and that it should stay constant across all layers when back-propagating. I see how one can simply adjust the bounds &lt;span class=&quot;math-container&quot;&gt;$[-a,a]$&lt;/span&gt; of a uniform variable in such a way that the random variable has the desired standard deviation and vice versa (&lt;span class=&quot;math-container&quot;&gt;$\sigma = a/\sqrt{3}$&lt;/span&gt;), but &lt;strong&gt;I'm not sure why we need a normal and a uniform version for both schemes?&lt;/strong&gt; Wouldn't it be just enough to have only normal or only uniform? Or uniform Xavier and normal He as proposed in their papers?&lt;/p&gt;&#xA;&lt;p&gt;I can imagine uniform distributions are easier to sample from a computational point of view, but since we do the initialization operation only once at the beginning, the computational cost is negligible compared to that from training. Further uniform variables are bounded, so there are no long tail observations as one would expect in a normal. I suppose that's why both libraries have truncated normal initializations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Are there any theoretical, computational or empirical justifications for when to use a normal over a uniform, or a uniform over a normal weight initialization regardless of the final weight variance?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="37120" LastEditorUserId="37120" LastEditDate="2020-12-25T13:06:54.737" LastActivityDate="2020-12-25T13:06:54.737" Title="Why is there a Uniform and Normal version of He / Xavier initialization in DL libraries?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25255" PostTypeId="1" CreationDate="2020-12-18T00:13:21.850" Score="4" ViewCount="72" Body="&lt;p&gt;In the Deep Learning book by Goodfellow et al., &lt;a href=&quot;http://imlab.postech.ac.kr/dkim/class/csed514_2019s/DeepLearningBook.pdf#page=453&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 11.4.5 (p. 438)&lt;/a&gt;, the following claims can be found:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Currently, we cannot unambiguously recommend Bayesian hyperparameter&#xA;optimization as an established tool for achieving better deep learning results or for obtaining those results with less effort. Bayesian hyperparameter optimization sometimes performs comparably to human experts, sometimes better, but fails catastrophically on other problems. It may be worth trying to see if it works on a particular problem but is not yet sufficiently mature or reliable&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Personally, I never used Bayesian hyperparameter optimization. I prefer the simplicity of grid search and random search.&lt;/p&gt;&#xA;&lt;p&gt;As a first approximation, I'm considering easy AI tasks, such as multi-class classification problems with DNNs and CNNs.&lt;/p&gt;&#xA;&lt;p&gt;In which cases should I take it into consideration, is it worth it?&lt;/p&gt;&#xA;" OwnerUserId="36907" LastEditorUserId="2444" LastEditDate="2020-12-19T18:46:23.783" LastActivityDate="2020-12-19T18:46:23.783" Title="Bayesian hyperparameter optimization, is it worth it?" Tags="&lt;deep-learning&gt;&lt;hyperparameter-optimization&gt;&lt;bayesian-optimization&gt;&lt;grid-search&gt;&lt;random-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25256" PostTypeId="2" ParentId="25238" CreationDate="2020-12-18T01:05:54.000" Score="0" Body="&lt;p&gt;Yes, you are correct, &lt;span class=&quot;math-container&quot;&gt;$S_{dW}$&lt;/span&gt;, in this case, is the matrix/tensor and it will have the same shape as gradient matrix/tensor. The update rule divides each element in gradient to each gradient in &lt;span class=&quot;math-container&quot;&gt;$S_{dW}$&lt;/span&gt; or you can say &amp;quot;element-wise division&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;To avoid the zero variable in &lt;span class=&quot;math-container&quot;&gt;$S_{dW}$&lt;/span&gt;, the equation usually like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W=W-\alpha \frac{dW}{\sqrt{S_{dW}+\epsilon}} \text{  where } \epsilon=10^{-8}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you want to implement it manually with Python, you can take a look at &lt;a href=&quot;https://gist.github.com/acl21/64213c8f9f65bc362d595c7bec359548#file-rmsprop-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41287" LastActivityDate="2020-12-18T01:05:54.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25257" PostTypeId="1" AcceptedAnswerId="25384" CreationDate="2020-12-18T03:55:59.130" Score="1" ViewCount="70" Body="&lt;p&gt;As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression.&lt;/p&gt;&#xA;&lt;p&gt;I was interested in trying a harder problem and have been working on a network that, given a private key, could perform point multiplication along ECC curve to obtain a public key.&lt;/p&gt;&#xA;&lt;p&gt;I have tried training on a dataset of generated keypairs, but am seeing the test loss values bounce around like crazy with train loss values eventually decreasing after many epochs due to what I assume is overfitting.&lt;/p&gt;&#xA;&lt;p&gt;Is this public key generation problem even solvable with a deep learning architecture? If so, am I doing something wrong with my current approach?&lt;/p&gt;&#xA;" OwnerUserId="43059" LastActivityDate="2020-12-25T22:37:11.967" Title="Regression For Elliptical Curve Public Key Generation Possible?" Tags="&lt;deep-learning&gt;&lt;deep-neural-networks&gt;&lt;residual-networks&gt;" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="25258" PostTypeId="2" ParentId="25255" CreationDate="2020-12-18T04:56:16.940" Score="2" Body="&lt;p&gt;Efficiently integrating HPO frameworks into an existing project is non-trivial. Most common datasets/tasks already have established architectures/hyperparameters/etc. and require only a few additional tuning parameters. In this case, the benefits (assuming they exist) brought by Bayesian HPO techniques lack behind development time (simplicity), and this is one dominant reason that users prefer grid search or random search over some more sophisticated Bayesian optimization techniques.&lt;/p&gt;&#xA;&lt;p&gt;For a very large scale problem and an entirely new task (or dataset) for which the user has no intuition on &amp;quot;good hyperparameters&amp;quot;, Bayesian HPO may be a good option to consider over random or grid search. This is because there may be a very large number of hyperparameters (due to insufficient domain knowledge), and integrating HPO into the project may result in far less time than grid search over all possible hyperparameters.&lt;/p&gt;&#xA;" OwnerUserId="33444" LastActivityDate="2020-12-18T04:56:16.940" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25259" PostTypeId="2" ParentId="25218" CreationDate="2020-12-18T05:14:27.740" Score="0" Body="&lt;p&gt;Without any knowledge on the references you came across, I am assuming that the authors were considering common applications of MAB (planning, online learning, etc.) for which the time horizon is usually small. In such applications, we usually cannot afford a large average regret which is inevitable for standard MAB algorithms due to the &lt;span class=&quot;math-container&quot;&gt;$\sqrt{|K|}$&lt;/span&gt; factor.&lt;/p&gt;&#xA;&lt;p&gt;Depending on the application or additional constraints you can impose on your problem, there are several works that consider structured stochastic MABs in which there are much better guarantees than the pessimistic &lt;span class=&quot;math-container&quot;&gt;$\sqrt{K}$&lt;/span&gt; bounds. One variant of structured MABs is learning on graphs [1], where one can obtain regret bounds growing with &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\beta(G)}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\beta(G)$&lt;/span&gt; is the independence number of a graph &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;[1] F. Liu, Z. Zheng, N. Shroff, &amp;quot;Analysis of Thompson Sampling for Graphical Bandits Without the Graphs&amp;quot;, ArXiv. 2018.&lt;/p&gt;&#xA;" OwnerUserId="33444" LastActivityDate="2020-12-18T05:14:27.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25260" PostTypeId="1" CreationDate="2020-12-18T06:00:13.610" Score="2" ViewCount="30" Body="&lt;p&gt;Sorry if I sound confused. I read that data to be fed to a machine are divided into training, validation and test data. Both training and validation data are used for developing the model. Test data is used only for testing the model and no tuning of the model is done using test data.&lt;/p&gt;&#xA;&lt;p&gt;Why is there a need to separate out training and validation data since both sets of data are for developing/tuning the model? Why not keep things simple and combine both data sets into a single one?&lt;/p&gt;&#xA;" OwnerUserId="16839" LastActivityDate="2020-12-19T07:50:03.603" Title="Why can't we combine both training and validation data, given that both types of data are used for developing the model?" Tags="&lt;machine-learning&gt;&lt;datasets&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25261" PostTypeId="2" ParentId="12340" CreationDate="2020-12-18T07:03:44.067" Score="0" Body="&lt;p&gt;One possibility: If you are using a dropout regularization layer in your network, it is reasonable that the validation error is smaller than the training error. Because usually dropout is activated when training but deactivated when evaluating on the validation set. You get a more smooth (usually means better) function in the latter case.&lt;/p&gt;&#xA;" OwnerUserId="43069" LastEditorUserId="2444" LastEditDate="2020-12-19T23:40:05.173" LastActivityDate="2020-12-19T23:40:05.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25262" PostTypeId="1" AcceptedAnswerId="25356" CreationDate="2020-12-18T11:52:44.913" Score="1" ViewCount="37" Body="&lt;p&gt;In section 2 the paper &lt;a href=&quot;https://arxiv.org/pdf/1805.00909.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review&lt;/a&gt; the author is discussing formulating the RL problem as a probabilistic graphical model. They introduce a binary optimality variable &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}_t$&lt;/span&gt; which denotes whether time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; was optimal (1 if so, 0 otherwise). They then define the probability that this random variable equals 1 to be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{P}(\mathcal{O}_t = 1 | s_t, a_t) = \exp(r(s_t, a_t)) \; .$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is why do they do this? In the paper they make no assumptions about the value of the rewards (e.g. bounding it to be non-positive) so in theory the rewards can take any value and thus the RHS can be larger than 1. This is obviously invalid for a probability. It would make sense if there was some normalising constant, or if the author said that the probability is proportional to this, but they don't.&lt;/p&gt;&#xA;&lt;p&gt;I have searched online and nobody seems to have asked this question which makes me feel like I am missing something quite obvious so I would appreciate if somebody could clear this up for me please.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-12-23T18:30:05.000" Title="In RL as probabilistic inference, why do we take a probability to be $\exp(r(s_t, a_t))$?" Tags="&lt;reinforcement-learning&gt;&lt;probability&gt;&lt;inference&gt;&lt;graphical-model&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25263" PostTypeId="2" ParentId="25260" CreationDate="2020-12-18T12:30:48.323" Score="1" Body="&lt;p&gt;Usually we are splinting the training dataset because we want to fine tune, find the best hyper-parameters, for our model. If we combine the validation and training dataset given a network complex enough we could achieve perfect performance for the given task. But having very good performance on the training dataset does not mean our model is useful. It might be the case that our model is very good on the training dataset, but it fails to generalize, so when given samples outside of the training dataset the performance will significantly drop (this is called over-fitting). By splitting the dataset, we are testing how well our model is performing on new/unseen data.&lt;/p&gt;&#xA;" OwnerUserId="20430" LastActivityDate="2020-12-18T12:30:48.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25264" PostTypeId="1" CreationDate="2020-12-18T13:57:11.400" Score="3" ViewCount="86" Body="&lt;p&gt;In the paper &amp;quot;ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery&amp;quot;, the authors talk about using:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Feature Pyramid Networks (as the architecture)&lt;/li&gt;&#xA;&lt;li&gt;EfficientNet-B2 (as the backbone)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7aAaX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7aAaX.png&quot; alt=&quot;Extract from Paper &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What's the difference between architectures and backbones? I can't find much online. Specifically, what are their respective purposes? From a high-level perspective, what would integrating the two look like?&lt;/p&gt;&#xA;" OwnerUserId="43083" LastEditorUserId="43231" LastEditDate="2020-12-29T14:37:07.023" LastActivityDate="2021-05-28T18:01:25.267" Title="What's the difference between architectures and backbones?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;terminology&gt;&lt;architecture&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25265" PostTypeId="1" CreationDate="2020-12-18T14:33:15.450" Score="0" ViewCount="56" Body="&lt;p&gt;I am new to machine learning, so I am not sure which algorithms to look at for my business problem. Most of what I am seeing in tools like KNIME are geared toward making a prediction/classification, and focusing on the accuracy of that single prediction/classification.&lt;/p&gt;&#xA;&lt;p&gt;Instead, in general terms, I want to optimize toward maximum profit of a business process/strategy, rather than simply trying to choose the &amp;quot;best&amp;quot; transaction from within a set of possible transactions, which is quite different. The latter approach will simply give the best &amp;quot;transaction success percentage&amp;quot;, without regard for overall profit of the strategy in the aggregate.&lt;/p&gt;&#xA;&lt;p&gt;This is how the business problem is structured: Each Opportunity is a type of business strategy &amp;quot;game&amp;quot; between Entities. Each Entity is unaware, unaffected, and uninterested in conditions or events outside of the Opportunity, such as Observers. Each Opportunity is an independent event with no affect on other concurrent or future Opportunities, and with no effect on decisions unrelated to the Opportunity itself. Each Opportunity will have one and only one Awarded Entity, which is the Entity that &amp;quot;wins&amp;quot; the business process.&lt;/p&gt;&#xA;&lt;p&gt;Observers, however, may create a Market for each Opportunity. Within such an independent, ephemeral Market, the Observers may bid among themselves as to which Entity will be the Awarded Entity for the Opportunity. Each Bid is a fixed-size transaction. A Bid is associated with only one Entity within the Opportunity. Thus, a Bid is a type of vote on the outcome of the Opportunity. There is no limit to the number of Bids that an Observer may place into the Market, but each Observer may only place Bids on a single Entity within the Market. Thus, the total amount of Bids on an individual Entity within the Opportunity represent the confidence level, within that Market, of the prediction.&lt;/p&gt;&#xA;&lt;p&gt;At the resolution of the Opportunity, one Entity will be the Awarded Entity for the Opportunity. This determination is made based on factors outside of the control of the Observers. The Observers have no influence over the Opportunity nor the Entities within it. When the Awarded Entity is determined, the total value of Bids placed on that Entity are refunded to the Observers that placed them. Additionally, the value of all Bids placed on other Entities are shared among the Observers who bid correctly on the Awarded Entity. Each Observer that placed a correct Bid on the Awarded Entity is entitled to a fraction of the remaining Market value, in equal proportion to the number of fixed Bids placed. In other words, the Market is a zero sum scenario. Bids may be placed at any time during the duration of the Opportunity, from when its Market is created, up until a deadline just shortly before its resolution. The total number of outstanding Bids on each Entity is another data point that is available in real time, and which fluctuates during the duration of the Opportunity, based on total Bids placed and the ratios between Bids on each Entity participating in the Opportunity.&lt;/p&gt;&#xA;&lt;p&gt;To support the Observers' evaluation and prediction of Awarded Entity within the scope of Opportunities, there are thousands of data points available, as well as extensive history and analytics regarding each Entity involved. Each Observer will employ their own unique strategy to predict Opportunity outcomes. The objective of this algorithm is to optimize a prediction strategy that does not optimize for &amp;quot;percentage of correct predictions&amp;quot;, but rather &amp;quot;maximum gain&amp;quot;. Rather than be &amp;quot;most correct most often&amp;quot;, the model should strive to use the data to create advantages for maximum gain in the aggregate, rather than strive to be the most correct. An Observer is rewarded not for being correct most often, but for recognizing inefficiencies in the Bids within the Market.&lt;/p&gt;&#xA;&lt;p&gt;I am considering hand-coding a genetic algorithm for this, so that I can write a custom fitness function that computes overall profitability of the strategy, and run the generations to optimize profit instead of individual selection accuracy. However, I'd rather use an out-of-the-box algorithm supported by a tool like KNIME if possible.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14854" LastEditorUserId="14854" LastEditDate="2020-12-24T03:17:33.770" LastActivityDate="2020-12-24T03:17:33.770" Title="What is the best algorithm for optimizing profit, rather than making predictions?" Tags="&lt;algorithm&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25266" PostTypeId="1" CreationDate="2020-12-18T15:22:58.540" Score="1" ViewCount="48" Body="&lt;p&gt;I came across Grenander's work &amp;quot;Probabilities on Algebraic Structures&amp;quot; recently and found that much of Grenander's work focused on what he called &amp;quot;Pattern Theory.&amp;quot; He's written many texts on the matter and from what I've seen, they seem like an attempt to unify some mathematical underpinnings of pattern representation. However, I'm not sure what this really means in practice, nor how it relates to results we already have in learning theory. The mathematical aspect of the work is really quite intriguing, but I am skeptical as to its practicality.&lt;/p&gt;&#xA;&lt;p&gt;Are there any applications of Grenander's pattern theory? Either for getting a better theoretical understanding of certain methods of pattern recognition or for directly implementing algorithms?&lt;/p&gt;&#xA;&lt;p&gt;Some links to what I'm referring to: &lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia entry&lt;/a&gt;, &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/B005NKFBME&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;, &lt;a href=&quot;https://www.springer.com/gp/book/9780387901749&quot; rel=&quot;nofollow noreferrer&quot;&gt;first book of 3 set volume&lt;/a&gt;, and &lt;a href=&quot;https://www.brown.edu/research/projects/pattern-theory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;research group, including fields medalist David Mumford&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="43084" LastEditorUserId="43084" LastEditDate="2020-12-20T19:00:11.817" LastActivityDate="2021-01-19T19:03:55.463" Title="What is &quot;Pattern Theory&quot;?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;theory&gt;&lt;soft-question&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25267" PostTypeId="2" ParentId="25266" CreationDate="2020-12-18T15:54:00.263" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_matching&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pattern-matching&lt;/a&gt; is a classical sub-problem of AI systems. Read about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rete_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rete algorithm&lt;/a&gt;, or pattern matching in &lt;a href=&quot;https://en.wikipedia.org/wiki/Prolog&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prolog&lt;/a&gt;. Or about &lt;a href=&quot;https://en.wikipedia.org/wiki/Unification_(computer_science)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unification&lt;/a&gt;. Or about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Warren_Abstract_Machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;Warren Abstract Machine&lt;/a&gt;. Or about &lt;a href=&quot;https://en.wikipedia.org/wiki/Topological_data_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;topological data analysis&lt;/a&gt; like in &lt;a href=&quot;http://gudhi.gforge.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GUDHI&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A typical example is the &lt;a href=&quot;http://www.clipsrules.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLIPSrules&lt;/a&gt; expert system shell. Or the &lt;code&gt;match&lt;/code&gt; expression construct of &lt;a href=&quot;https://www.ocaml.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ocaml&lt;/a&gt; or in &lt;a href=&quot;https://en.wikipedia.org/wiki/Haskell_(programming_language)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haskell&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In some ways, pattern matching is related to compilers: the &lt;a href=&quot;https://gcc.gnu.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GCC&lt;/a&gt; compiler (or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stalin_(Scheme_implementation)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stalin&lt;/a&gt; one, or &lt;a href=&quot;http://www-sop.inria.fr/members/Manuel.Serrano/bigloo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bigloo&lt;/a&gt;, or &lt;a href=&quot;https://sbcl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBCL&lt;/a&gt;) is somehow doing pattern matching on &lt;a href=&quot;https://en.wikipedia.org/wiki/Abstract_syntax_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;AST&lt;/a&gt;s. So &lt;a href=&quot;https://en.wikipedia.org/wiki/Compiler-compiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;parser generator&lt;/a&gt;s (e.g. &lt;a href=&quot;https://www.gnu.org/software/bison/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU bison&lt;/a&gt;, or &lt;a href=&quot;https://github.com/drh/iburg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iburg&lt;/a&gt;) are related to pattern matching.&lt;/p&gt;&#xA;&lt;p&gt;Patterns also have a different meaning in &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_vision&quot; rel=&quot;nofollow noreferrer&quot;&gt;computer vision&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A classical paper is of course &lt;a href=&quot;http://homepages.inf.ed.ac.uk/wadler/papers/pattern/pattern.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Derivation of a Pattern-Matching Compiler&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A more interesting approach is described in Jacques Pitrat's last book:&#xA;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1848211015&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings: the conscience of a conscious machine&lt;/em&gt;&lt;/a&gt; ISBN-13: 978-1848211018&lt;/p&gt;&#xA;&lt;p&gt;You could also look into Pitrat's &lt;a href=&quot;http://bootstrappingartificialintelligence.fr/WordPress3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;PS. Jacques Pitrat passed away in oct. 2019.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-12-18T16:19:59.860" LastActivityDate="2020-12-18T16:19:59.860" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25268" PostTypeId="1" AcceptedAnswerId="25306" CreationDate="2020-12-18T16:22:51.167" Score="4" ViewCount="101" Body="&lt;p&gt;I'm trying to learn about reinforcement learning techniques. I have little background in machine learning from university, but never more than using a CNN on the MNIST database.&lt;/p&gt;&#xA;&lt;p&gt;My first project was to use reinforcement learning on tic-tac-toe and that went well.&lt;/p&gt;&#xA;&lt;p&gt;In the process, I thought about creating an AI that can play a card game like &lt;a href=&quot;https://en.wikipedia.org/wiki/Magic:_The_Gathering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Magic: The Gathering&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Yu-Gi-Oh!&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yu-gi-oh&lt;/a&gt;, etc. However, I need to think of a way to define an action space. Not only are there thousands of combinations of cards possible in a single deck, but we also have to worry about the various types of decks the machine is playing and playing against.&lt;/p&gt;&#xA;&lt;p&gt;Although I know this is probably way too advanced for a beginner, I find attempting a project like this, challenging and stimulating. So, I looked into several different approaches for defining an action space. But I don't think this example falls into a continuous action space, or one in which I could remove actions when they are not relevant.&lt;/p&gt;&#xA;&lt;p&gt;I found this post on this &lt;a href=&quot;https://ai.stackexchange.com/q/2219/2444&quot;&gt;stack exchange&lt;/a&gt; that seems to be asking the same question. However, the answer I found didn't seem to solve any of my problems.&lt;/p&gt;&#xA;&lt;p&gt;Wouldn't defining the action space as another level of game states just mask the exact same problem?&lt;/p&gt;&#xA;&lt;p&gt;My main question boils down to:&lt;/p&gt;&#xA;&lt;p&gt;Is there an easy/preferred way to make an action space for a game as complex as Magic? Or, is there another technique (other than RL) that I have yet to see that is better used here?&lt;/p&gt;&#xA;" OwnerUserId="43065" LastEditorUserId="2444" LastEditDate="2021-01-17T21:46:48.947" LastActivityDate="2021-01-17T21:46:48.947" Title="How should I define the action space for a card game like Magic: The Gathering?" Tags="&lt;reinforcement-learning&gt;&lt;ai-design&gt;&lt;action-spaces&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25269" PostTypeId="1" AcceptedAnswerId="25370" CreationDate="2020-12-18T16:24:19.343" Score="1" ViewCount="31" Body="&lt;p&gt;In the back-propogation algorithm, the error term is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E=\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2,&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_k$&lt;/span&gt; is a vector of outputs from the network, &lt;span class=&quot;math-container&quot;&gt;$y_k$&lt;/span&gt; is the vector of correct labels (and we work out the error by calculating predicted-observed, squaring the answer, and then summing the answers for each &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; (and dividing by 2).&lt;/p&gt;&#xA;&lt;p&gt;How do you prove that if this answer is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (i.e., if &lt;span class=&quot;math-container&quot;&gt;$E=0$&lt;/span&gt;), then &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_k=y_k$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="2444" LastEditDate="2020-12-24T12:43:03.033" LastActivityDate="2020-12-24T12:59:00.723" Title="How do I prove that the MSE is zero when all predictions are equal to the corresponding labels?" Tags="&lt;deep-learning&gt;&lt;proofs&gt;&lt;mean-squared-error&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25270" PostTypeId="1" AcceptedAnswerId="25271" CreationDate="2020-12-18T17:08:31.987" Score="0" ViewCount="90" Body="&lt;p&gt;I have a trained LSTM language model and want to use it to generate text. The standard approach for this seems to be:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Apply softmax function&lt;/li&gt;&#xA;&lt;li&gt;Take a weighted random choice to determine the next word&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This is working reasonably well for me, but it would be nice to play around with other options. Are there any good alternatives to this?&lt;/p&gt;&#xA;" OwnerUserId="43089" LastEditorUserId="2444" LastEditDate="2020-12-18T21:28:43.763" LastActivityDate="2020-12-18T21:28:43.763" Title="Are there any good alternatives to an LSTM language model for text generation?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;language-model&gt;&lt;text-generation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25271" PostTypeId="2" ParentId="25270" CreationDate="2020-12-18T20:50:22.137" Score="3" Body="&lt;p&gt;The current state of the art in natural language generation are all auto-regressive transformer models. Transformers no longer use recurrent neural networks such as LSTM, because the recurrences makes long dependencies messy to calculate. Instead, Transformers only keep the attention layers, and apply attention on all the existing text so far, which can be done in parallel so therefore very fast, while being able to attend to long dependencies (e.g. understanding that &amp;quot;it&amp;quot; refers to &amp;quot;John&amp;quot; from 3 sentences ago). They are also faster to train than LSTMs (on powerful GPUs at least). The downside is more memory requirement, and you need large models and large datasets (LSTMs work better for small models and small datasets). &lt;a href=&quot;http://jalammar.github.io/illustrated-transformer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is some background info on how they work.&lt;/p&gt;&#xA;&lt;p&gt;Auto-regressive transformer models only use the decoder for text generation, and removes the encoder. Given an input, they predict the next word.&lt;/p&gt;&#xA;&lt;p&gt;The most well-known one is GPT (&lt;a href=&quot;https://github.com/openai/gpt-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT-3&lt;/a&gt; has 175B parameters; &lt;a href=&quot;https://github.com/openai/gpt-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPT-2&lt;/a&gt; has 1.5B parameters, and GPT-1 has 175M parameters) GPT is developed by OpenAI and is a commercial, paid-software if you want to use their official model, but I'm sure with a little digging you can find &lt;a href=&quot;https://huggingface.co/transformers/pretrained_models.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;community-trained models&lt;/a&gt; that will perform slightly worse but is at least free to use. GPT is basically a vanilla transformer, but trained on a huge, huge dataset with a huge, huge model to achieve state-of-the-art performance.&lt;/p&gt;&#xA;&lt;p&gt;Other auto-regressive transformer models include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://CTRL&quot; rel=&quot;nofollow noreferrer&quot;&gt;CTRL&lt;/a&gt; by Salesforce, which uses the novel idea of control codes to guide the style of generation (e.g. to generate Wikipedia article style text or book review style text).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/zihangdai/xlnet&quot; rel=&quot;nofollow noreferrer&quot;&gt;XLNet&lt;/a&gt; by Google AI Brain team, which handles longer sequences more accuractely than the others because it re-introduces reccurrence back in the transformer model, allowing it to remember past sequences. Otherwise, vanilla transformers cannot handle dependencies that crosses sequences (note: a sequence is limited by the max length you can feed into the model, bottle-necked by your memory requirement, and can contain many sentences or paragraphs).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/lucidrains/reformer-pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reformer&lt;/a&gt; by Google Research, which is a more efficient transformer that significantly reduces the memory requirement while also being faster to compute on long sequences.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If your goal is just to generate English or another commonly researched language, you can use an existing pre-trained language model and avoid doing any training yourself. This saves a lot of time, and there should at least be free community-trained models readily available. Otherwise, for obscure tasks, you'd have to train one yourself, and these state of the art models will take immense resources.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-18T21:25:12.747" LastActivityDate="2020-12-18T21:25:12.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25272" PostTypeId="2" ParentId="25253" CreationDate="2020-12-18T21:32:53.003" Score="0" Body="&lt;p&gt;They shouldn't have any issues with short sequences, as short dependencies are easier to learn. The only difficult cases are long dependencies which is where most of the research is geared at. However, this is assuming that by &amp;quot;short sequence&amp;quot; you mean a sequence of text that is fully contained within itself, i.e. there is no cross-sequence dependencies.&lt;/p&gt;&#xA;&lt;p&gt;For example, if you have a really long paragraph that doesn't fit in a transformer model, you would have to break that paragraph into many &amp;quot;short sequences&amp;quot;, but each of these sequence may have a dependency that depends on another sequence, i.e. cross-sequence dependencies. For these cross-sequence dependencies, any model with recurrence should do better than ones without (e.g. RNN, LSTM, Transformer-XL).&lt;/p&gt;&#xA;&lt;p&gt;If each short sequence is self-contained, then all of the models should perform pretty well.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-18T21:32:53.003" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25273" PostTypeId="2" ParentId="25264" CreationDate="2020-12-18T22:17:16.230" Score="0" Body="&lt;p&gt;The vocabulary is definitely non-standard and a bit confusing, but Feature Pyramid Networks is used as a feature extractor, and its output is then fed into EfficientNet-B2 to be used to classify the image. One neural network model is concatenated at the end of the other.&lt;/p&gt;&#xA;&lt;p&gt;So it seems like &amp;quot;architecture&amp;quot; is the front half of the neural network model which takes as input the satellite image and extracts image featuers, and then is directly connected to the back half of the model (hence &amp;quot;backbone&amp;quot;), which takes the features extracted from the &amp;quot;architecture&amp;quot; and makes a classification.&lt;/p&gt;&#xA;&lt;p&gt;This terminology is definitely non-standard here, at least in the AI community, and if you ask anyone here I think it will be uncommon for them to naturally think about the words &amp;quot;architecture&amp;quot; vs &amp;quot;backbone&amp;quot; in this way unless they specialize in a similar field to the authors.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-18T22:17:16.230" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25276" PostTypeId="1" CreationDate="2020-12-19T02:10:27.840" Score="0" ViewCount="40" Body="&lt;p&gt;I am doing self study of Reinforcement Learning with Q-learning using online resources like blog posts, youtube videos and books and at this point, I have learned the underpinning concepts of Reinforcement learning and how to update the q values using look up table.&lt;/p&gt;&#xA;&lt;p&gt;I came across that convergence is no longer guaranteed when the Q-function is approximated.&#xA;&lt;a href=&quot;https://wiki.ubc.ca/Course:CPSC522/Reinforcement_Learning_with_Function_approximation#:%7E:text=Function%20Approximation,-Optimal%20and%20Approximate&amp;amp;text=One%20thing%20that%20needs%20to,the%20weights%20of%20the%20network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wiki.ubc.ca/Course:CPSC522/Reinforcement_Learning_with_Function_approximation#:~:text=Function%20Approximation,-Optimal%20and%20Approximate&amp;amp;text=One%20thing%20that%20needs%20to,the%20weights%20of%20the%20network&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to understand convergence considering the Bellman equation. Also trying to understand why when using approximation, convergence is no longer guaranteed.&lt;/p&gt;&#xA;&lt;p&gt;Why is that?&lt;/p&gt;&#xA;" OwnerUserId="41187" LastActivityDate="2020-12-19T13:44:48.893" Title="Why convergence is not guaranteed when using approximation?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;q-learning&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-12-19T12:42:20.857" ContentLicense="CC BY-SA 4.0" />
  <row Id="25277" PostTypeId="1" CreationDate="2020-12-19T02:31:13.490" Score="1" ViewCount="60" Body="&lt;p&gt;I am learning reinforcement learning with Q-learning using online resources, like blog posts, youtube videos, and books. At this point, I have learned the underpinning concepts of reinforcement learning and how to update the q values using a lookup table.&lt;/p&gt;&#xA;&lt;p&gt;Now, I want to create a neural network to replace the lookup table and approximate the Q-function, but I am not sure how to design the neural network. What would be the architecture for my neural network? What are the inputs and outputs?&lt;/p&gt;&#xA;&lt;p&gt;Here are the two options I can think of.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The input of the neural network is &lt;span class=&quot;math-container&quot;&gt;$(s_i, a_i)$&lt;/span&gt; and the output is &lt;span class=&quot;math-container&quot;&gt;$Q(s_i,a_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The input is &lt;span class=&quot;math-container&quot;&gt;$(s_i)$&lt;/span&gt; and the output is a vector &lt;span class=&quot;math-container&quot;&gt;$[Q(s_i,a_1), Q(s_i,a_2), \dots, Q(s_i,a_N)]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is there any other alternative architecture?&lt;/p&gt;&#xA;&lt;p&gt;Also, how to reason about which model would be logically better?&lt;/p&gt;&#xA;" OwnerUserId="41187" LastEditorUserId="2444" LastEditDate="2020-12-19T13:24:58.393" LastActivityDate="2020-12-19T13:24:58.393" Title="How to build a Neural Network to approximate the Q-function?" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25278" PostTypeId="2" ParentId="25260" CreationDate="2020-12-19T07:50:03.603" Score="1" Body="&lt;p&gt;There are two possible forms of overfitting. First related to the training only (fitting weights) and second related to  architecture (fitting hyperparameters) and these two must be checked in two different stages. When you check performance of the given model you have two do this on unseen data, so you fit weights on training data and check it on validation (unseen to the weights fitting process) set and next you fit hyperparameters on validation set and check it on test (unseen to the hyperparameters fitting process) set.&lt;/p&gt;&#xA;" OwnerUserId="22659" LastActivityDate="2020-12-19T07:50:03.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25279" PostTypeId="2" ParentId="25277" CreationDate="2020-12-19T09:29:18.640" Score="2" Body="&lt;p&gt;I had the same question when I first learned RL. The architectural design may depend on the task you're considering. Since you're moving from a tabular Q-learning to function approximation, I'm suspecting that you are considering a relatively small action space; in this case, you should use option 2 where the input is the state and the number of output nodes matches the number of available actions.&#xA;One main reason for this choice: when exploiting the learnt function (i.e. policy is &lt;span class=&quot;math-container&quot;&gt;$\arg \max_a Q(s,a)$&lt;/span&gt;), for option (1) you would have to repeatedly run a forward pass for all available actions, whereas option (2) requires a single forward pass and run arg max either through numpy or the framework you're using.&lt;/p&gt;&#xA;" OwnerUserId="33444" LastActivityDate="2020-12-19T09:29:18.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25281" PostTypeId="2" ParentId="25205" CreationDate="2020-12-19T09:52:35.003" Score="2" Body="&lt;p&gt;This question is very general in the sense that the reason may differ depending on the area of ML you are considering. Below are two different areas of ML where the KL-divergence is a natural consequence:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Classification: maximizing the log-likelihood (or minimizing the negative log-likelihood) is equivalent to minimizing KL divergence as typical used in DL-based classification &lt;strong&gt;where one-hot targets are commonly used as reference&lt;/strong&gt; (see &lt;a href=&quot;https://stats.stackexchange.com/a/357974&quot;&gt;https://stats.stackexchange.com/a/357974&lt;/a&gt;). Furthermore, if you have a one-hot vector &lt;span class=&quot;math-container&quot;&gt;$e_y$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; at index &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, minimizing cross-entropy &lt;span class=&quot;math-container&quot;&gt;$\min_{\hat{p}}H(e_y, \hat{p}) = - \sum_y e_y \log \hat{p}_y = - \log \hat{p}$&lt;/span&gt; boils down to maximizing the log-likelihood. In summary, maximizing the log-likelihood is arguably a natural objective, and KL-divergence (with 0 log 0 defined as 0) comes up because of its equivalence to log-likelihood under typical settings, rather than explicitly being motivated as the objective.&lt;/li&gt;&#xA;&lt;li&gt;Multi-armed bandits (a sub-area of reinforcement learning): Upper confidence bound (UCB) is an algorithm derived from standard concentration inequalities. If we consider MABs with Bernoulli rewards, we can apply Chernoff's bound and optimize over the free parameter to obtain an upper bound expressed in terms of KL divergence as stated below (see &lt;a href=&quot;https://page.mi.fu-berlin.de/mulzer/notes/misc/chernoff.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://page.mi.fu-berlin.de/mulzer/notes/misc/chernoff.pdf&lt;/a&gt; for some different proofs).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$X_1, \dots, X_n$&lt;/span&gt; be i.i.d. Bernoulli RVs with parameter &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(\sum_i X_i \geq (p+t)n) \leq \inf_\lambda M_X (\lambda) e^{-\lambda t} = \exp(-n D_{KL}(p+t||p)).$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="33444" LastEditorUserId="33444" LastEditDate="2020-12-20T12:15:21.267" LastActivityDate="2020-12-20T12:15:21.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25283" PostTypeId="1" CreationDate="2020-12-19T11:18:24.430" Score="2" ViewCount="37" Body="&lt;p&gt;I get the fundamental idea of how tilings work, but, in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=241&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barton and Sutton's book, Reinforcement Learning: An Introduction (2nd edition)&lt;/a&gt;, a diagram, on page 219 (figure 9.11), showing the variations of uniform offset tiling has confused me.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QRYwa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QRYwa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't understand why all 8 of these figures are instances of uniformly offset tilings. I thought uniformly offset meant ALL tilings have to be offset an equal amount from each other which is only the case for the bottom left figure. Is my understanding wrong?&lt;/p&gt;&#xA;" OwnerUserId="42514" LastEditorUserId="2444" LastEditDate="2021-01-29T20:40:31.343" LastActivityDate="2021-01-29T20:40:31.343" Title="How does uniform offset tiling work with function approximation?" Tags="&lt;reinforcement-learning&gt;&lt;function-approximation&gt;&lt;feature-selection&gt;&lt;tile-coding&gt;&lt;coarse-coding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25284" PostTypeId="1" CreationDate="2020-12-19T12:46:33.333" Score="1" ViewCount="20" Body="&lt;p&gt;I'm developing a method to document and query representation as concept vectors (bag-of-concepts). I want to train a machine learning model on ranking (learning to rank a task). So I have document vector V1 and query vector V2. How should I use these two numerical vectors in learning the way to rank a task? What are the possible scenarios?&lt;/p&gt;&#xA;&lt;p&gt;Do I calculate relevance (similarity) by cosine and then enter the result as a single feature into a neural network? Is it correct to apply Hadamard to produce a single vector representing the features of a document and query pair, and then train a neural network with it? Can two vectors (document and query vector) be entered into the Siamese network in order to evaluate the relevance? One told me this is not possible because the network only take raw text as input and extracts features. Hence, it is useless to enter a vector that was generated by my vectorization method.&lt;/p&gt;&#xA;" OwnerUserId="43101" LastEditorUserId="43231" LastEditDate="2021-01-09T15:55:29.177" LastActivityDate="2021-01-09T15:55:29.177" Title="On learning to rank tasks. Could it be that the input of the Siamese network is a vector, or should it be exclusively raw text?" Tags="&lt;machine-learning&gt;&lt;similarity&gt;&lt;bag-of-features&gt;&lt;siamese-neural-network&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25285" PostTypeId="5" CreationDate="2020-12-19T13:41:56.813" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T13:41:56.813" LastActivityDate="2020-12-19T13:41:56.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25286" PostTypeId="4" CreationDate="2020-12-19T13:41:56.813" Score="0" Body="For questions related to the problem of program synthesis, in the context of artificial intelligence or that use AI techniques (such as genetic programming). " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T13:41:56.813" LastActivityDate="2020-12-19T13:41:56.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25287" PostTypeId="1" CreationDate="2020-12-19T13:46:36.233" Score="3" ViewCount="78" Body="&lt;p&gt;I am currently studying the paper &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.170&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning and Evaluating Classifiers under Sample Selection Bias&lt;/em&gt; by Bianca Zadrozny&lt;/a&gt;. In section &lt;strong&gt;3.2. Logistic Regression&lt;/strong&gt;, the author says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;3.2. Logistic regression&lt;/strong&gt;&#xA;In logistic regression, we use maximum likelihood to find the parameter vector &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; of the following model:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(y = 1 \mid x) = \dfrac{1}{1 + \exp(\beta_0 + \beta_1 x_1 + \dots + \beta_n x_n)}$$&lt;/span&gt;&#xA;With sample selection bias we will instead fit:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(y = 1 \mid x, s = 1) = \dfrac{1}{1 + \exp(\beta_0 + \beta_1 x_1 + \dots + \beta_n x_n)}$$&lt;/span&gt;&#xA;However, because we are assuming that &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is independent of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; we have that &lt;span class=&quot;math-container&quot;&gt;$P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$&lt;/span&gt;. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced. Asymptotically, as long as &lt;span class=&quot;math-container&quot;&gt;$P(s = 1 \mid x)$&lt;/span&gt; is greater than zero for all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, the results on a selected sample approach the results on a random sample. In fact, this is true for any learner that models &lt;span class=&quot;math-container&quot;&gt;$P(y \mid x)$&lt;/span&gt; directly. These are all local learners.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This part is unclear to me:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;However, because we are assuming that &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is independent of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; we have that &lt;span class=&quot;math-container&quot;&gt;$P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$&lt;/span&gt;. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What is meant by &amp;quot;the number of examples is reduced&amp;quot;, and why is this the case?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="2444" LastEditDate="2020-12-19T19:13:55.703" LastActivityDate="2021-02-05T18:07:15.683" Title="What is meant by &quot;the number of examples is reduced&quot;, and why is this the case?" Tags="&lt;classification&gt;&lt;papers&gt;&lt;logistic-regression&gt;&lt;selection-bias&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25288" PostTypeId="2" ParentId="25205" CreationDate="2020-12-19T14:11:02.443" Score="1" Body="&lt;p&gt;In ML we always deal with unknown probability distributions from which the data comes. The most common way to calculate the distance between real and model distribution is &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence.&lt;/p&gt;&#xA;&lt;h3&gt;Why Kullback–Leibler divergence?&lt;/h3&gt;&#xA;&lt;p&gt;Although there are other loss functions (e.g. MSE, MAE), &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence is natural when we are dealing with probability distributions. It is a fundamental equation in information theory that quantifies, in bits, how close two probability distributions are. It is also called relative entropy and, as the name suggests, it is closely related to entropy, which in turn is a central concept in information theory. Let's recall the definition of entropy for a discrete case:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H = -\sum_{i=1}^{N} p(x_i) \cdot \text{log }p(x_i)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you observed, entropy on its own is just a measure of a single probability distribution. If we slightly modify this formula by adding a second distribution, we get &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{KL}(p||q) = \sum_{i=1}^{N} p(x_i)\cdot (\text{log }p(x_i) - \text{log }q(x_i))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a data distribution and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; is model distribution.&lt;/p&gt;&#xA;&lt;p&gt;As we can see, &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence is the most natural way to compare 2 distributions. Moreover, it's pretty easy to calculate. This &lt;a href=&quot;https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; provides more intuition on this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Essentially, what we're looking at with the KL divergence is the&#xA;expectation of the log difference between the probability of data in&#xA;the original distribution with the approximating distribution. Again,&#xA;if we think in terms of &lt;span class=&quot;math-container&quot;&gt;$log_2$&lt;/span&gt; we can interpret this as &amp;quot;how many&#xA;bits of information we expect to lose&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h3&gt;Cross entropy&lt;/h3&gt;&#xA;&lt;p&gt;Cross-entropy is commonly used in machine learning as a loss function where we have softmax (or sigmoid) output layer, since it represents a predictive distribution over classes. The one-hot output represents a model distribution &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, while true labels represent a target distribution &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. Our goal is to push &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; as close as possible. We could take a mean squared error over all values, or we could sum the absolute differences, but the one measure which is motivated by information theory is cross-entropy. It gives the average number of bits needed to encode samples distributed as &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, using &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; as the encoding distribution.&lt;/p&gt;&#xA;&lt;p&gt;Cross-entropy based on entropy and generally calculates the difference between two probability distributions and closely related to &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence. The difference is that it calculates the the total entropy between the distributions, while &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence represents relative entropy. Corss-entropy can be defined as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H(p, q) = H(p) + D_{KL}(p \parallel q)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first term in this equation is the entropy of the true probability distribution &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; that is omitted during optimization, since the entropy of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is constant. Hence, minimizing cross-entropy is the same as optimizing &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence.&lt;/p&gt;&#xA;&lt;h3&gt;Log likelihood&lt;/h3&gt;&#xA;&lt;p&gt;It can be also &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross_entropy#Relation_to_log-likelihood&quot; rel=&quot;nofollow noreferrer&quot;&gt;shown&lt;/a&gt; that maximizing the (log) likelihood is equivalent to minimizing the cross entropy.&lt;/p&gt;&#xA;&lt;h3&gt;Limitations&lt;/h3&gt;&#xA;&lt;p&gt;As you mentioned, &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence is not symmetrical. But in most cases this is not critical, since we want to estimate the model distribution by pushing it towards real one, but not vice versa. There is also a symmetrized version called &lt;a href=&quot;https://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jensen–Shannon divergence&lt;/a&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_{JS}(p||q)=\frac{1}{2}D_{KL}(p||m)+\frac{1}{2}D_{KL}(q||m)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$m=\frac{1}{2}(p+q)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The main disadvantage of &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; is that both the unknown distribution and the model distribution must have support. Otherwise the &lt;span class=&quot;math-container&quot;&gt;$D_{KL}(p||q)$&lt;/span&gt; becomes &lt;span class=&quot;math-container&quot;&gt;$+\infty$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D_{JS}(p||q)$&lt;/span&gt; becomes &lt;span class=&quot;math-container&quot;&gt;$log2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Second, it should be noted that &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; is not a metric, since it violates triangle inequality. That is, in some cases it won't tell us if we are going the right direction when estimating our model distribution. Here is an example taken from &lt;a href=&quot;https://stats.stackexchange.com/a/351153/137038&quot;&gt;this answer&lt;/a&gt;. Given two discrete distributions &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, we calculate &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence and Wasserstein metric:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aN5Ld.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aN5Ld.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; divergence remained the same, while the Wasserstein metric decreased.&lt;/p&gt;&#xA;&lt;p&gt;But as mentioned in comments, Wasserstein metric is highly intractable in a continuous space. We still can use it by applying the Kantorovich-Rubinstein duality used in &lt;a href=&quot;https://arxiv.org/pdf/1701.07875.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wasserstein GAN&lt;/a&gt;. You can also find more on this topic in this &lt;a href=&quot;https://jonathan-hui.medium.com/gan-wasserstein-gan-wgan-gp-6a1a2aa1b490&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The 2 drawbacks of &lt;span class=&quot;math-container&quot;&gt;$KL$&lt;/span&gt; can be mitigated by adding noise. More on it in this &lt;a href=&quot;https://openaccess.thecvf.com/content_CVPR_2019/papers/Jenni_On_Stabilizing_Generative_Adversarial_Training_With_Noise_CVPR_2019_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12841" LastEditorUserId="12841" LastEditDate="2020-12-22T15:40:27.503" LastActivityDate="2020-12-22T15:40:27.503" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25289" PostTypeId="5" CreationDate="2020-12-19T18:46:59.847" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T18:46:59.847" LastActivityDate="2020-12-19T18:46:59.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25290" PostTypeId="4" CreationDate="2020-12-19T18:46:59.847" Score="0" Body="For questions related to the grid search method, which is typically used for hyper-parameter optimization." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T18:46:59.847" LastActivityDate="2020-12-19T18:46:59.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25291" PostTypeId="5" CreationDate="2020-12-19T18:48:13.977" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T18:48:13.977" LastActivityDate="2020-12-19T18:48:13.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25292" PostTypeId="4" CreationDate="2020-12-19T18:48:13.977" Score="0" Body="For questions related to the random search method used for hyper-parameter optimization. See, for example, the paper &quot;Random Search for Hyper-Parameter Optimization&quot; (2012) by James Bergstra and Yoshua Bengio." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T18:48:13.977" LastActivityDate="2020-12-19T18:48:13.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25293" PostTypeId="1" CreationDate="2020-12-19T21:11:04.013" Score="0" ViewCount="28" Body="&lt;p&gt;The goal of this program is to predict a game outcome given a &lt;code&gt;game-reference-id&lt;/code&gt;, which is a serial number like so:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;id,totalGreen,totalBlue,totalRed,totalYellow,sumNumberOnGreen,sumNumberOnBlue,sumNumberOnRed,sumNumberOnYellow,gameReferenceId,createdAt,updatedAt&#xA;1,1,3,2,0,33,27,41,0,1963886,2020-08-07 20:27:49,2020-08-07 20:27:49&#xA;2,1,4,1,0,36,110,31,0,1963887,2020-08-07 20:28:37,2020-08-07 20:28:37&#xA;3,1,3,2,0,6,33,83,0,1963888,2020-08-07 20:29:27,2020-08-07 20:29:27&#xA;4,2,2,2,0,45,58,44,0,1963889,2020-08-07 20:30:17,2020-08-07 20:30:17&#xA;5,0,2,4,0,0,55,82,0,1963890,2020-08-07 20:31:07,2020-08-07 20:31:07&#xA;6,2,4,0,0,36,116,0,0,1963891,2020-08-07 20:31:57,2020-08-07 20:31:57&#xA;7,3,2,1,0,93,16,40,0,1963892,2020-08-07 20:32:47,2020-08-07 20:32:47&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ustedung-my.sharepoint.com/:x:/g/personal/ugbanawaji_ekenekiso_ust_edu_ng/Eb2CqFbiH-JOnA1KTg5GYi4BmNZDD11oto8mtgRkdKFHgg?e=UiLcFN&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;'s the link for a full training dataset.&lt;/p&gt;&#xA;&lt;p&gt;After training the model, it becomes difficult to use the model to predict the game output, since the &lt;code&gt;game-reference-id&lt;/code&gt; is the only independent column, while others are random.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to flip the features with the labels during prediction?&lt;/p&gt;&#xA;" OwnerUserId="43108" LastEditorUserId="2444" LastEditDate="2020-12-20T12:51:42.433" LastActivityDate="2020-12-20T12:51:42.433" Title="Is it possible to flip the features and labels after training a model?" Tags="&lt;machine-learning&gt;&lt;features&gt;&lt;labels&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25294" PostTypeId="1" AcceptedAnswerId="26610" CreationDate="2020-12-19T22:03:43.873" Score="6" ViewCount="108" Body="&lt;p&gt;Some algorithms in the literature allow recovering the input data used to train a neural network. This is done using the gradients (updates) of weights, such as in &lt;a href=&quot;https://arxiv.org/abs/1906.08935&quot; rel=&quot;noreferrer&quot;&gt;Deep Leakage from Gradients&lt;/a&gt; (2019) by Ligeng Zhu et al.&lt;/p&gt;&#xA;&lt;p&gt;In case the neural network is trained using encrypted (homomorphic) input data, what could be the output of the above algorithm? Will the algorithm recover the data in clear or encrypted (as it was fed encrypted)?&lt;/p&gt;&#xA;" OwnerUserId="43113" LastEditorUserId="2444" LastEditDate="2020-12-19T23:47:51.203" LastActivityDate="2021-04-22T15:21:26.220" Title="During neural network training, can gradients leak sensitive information in case training data fed is encrypted (homomorphic)?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;gradient-descent&gt;&lt;security&gt;&lt;training-datasets&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25295" PostTypeId="1" CreationDate="2020-12-19T22:43:46.820" Score="1" ViewCount="32" Body="&lt;p&gt;I'm working with data that is ranked. So the inputs are 1,2,3 etc. This means the smaller numbers (ranks) are preferred to the larger ones. Hence the order is important. I want to estimate a number using regression; however, with the constraint that the order of the numbers must be monotonic non-linear.&lt;/p&gt;&#xA;&lt;p&gt;Imagine the following input table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1, 2&#xA;2, 3&#xA;3, 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For instance, if the output is 1000 for each input, then the estimation could be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 * (800) + 2 * (100) = 1000&#xA;2 * (300) + 3 * (60) = 780&#xA;3 * (150) + 1 * (400) = 850&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Evidently, the estimated Xs are monotonic non-linear decreasing.&#xA;For the first column: 1 -&amp;gt; 800, 2 -&amp;gt; 600 (2x300), 3 -&amp;gt; 450 (3x150)&#xA;for the second columns: 1 -&amp;gt; 400, 2 -&amp;gt; 200 (2x100), 3-&amp;gt; 180 (2x60)&lt;/p&gt;&#xA;&lt;p&gt;So here's the question. Can I ensure my model (neural network) enforces the given constraint? I am using Keras.&lt;/p&gt;&#xA;" OwnerUserId="43115" LastEditorUserId="16909" LastEditDate="2020-12-29T14:34:34.253" LastActivityDate="2020-12-29T14:34:34.253" Title="Can I constrain my neurons in a neural network in according to the orders of the input?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;keras&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25296" PostTypeId="1" CreationDate="2020-12-19T22:53:42.280" Score="3" ViewCount="101" Body="&lt;p&gt;In &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 8&lt;/a&gt;, section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation.&lt;/p&gt;&#xA;&lt;p&gt;But the chapter doesn't explain what asymmetric relaxation would be or how it is done.&lt;/p&gt;&#xA;&lt;p&gt;So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes?&lt;/p&gt;&#xA;" OwnerUserId="14892" LastEditorUserId="14892" LastEditDate="2020-12-30T19:07:44.170" LastActivityDate="2021-05-29T20:03:20.263" Title="What is asymmetric relaxation backpropagation?" Tags="&lt;neural-networks&gt;&lt;terminology&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25297" PostTypeId="2" ParentId="25293" CreationDate="2020-12-19T23:11:18.173" Score="1" Body="&lt;p&gt;The body of your post seems to be asking a completely separate question than the title of your post, so I will answer both:&lt;/p&gt;&#xA;&lt;h2&gt;&amp;quot;Body: How do I complete the goal of this program?&amp;quot;&lt;/h2&gt;&#xA;&lt;p&gt;Your dataset does not have the dependent variable, which is the outcome of the game (win/loss/draw). What I am assuming is that you have a way of looking up the outcome of the game from either the &amp;quot;id&amp;quot; field or the &amp;quot;gameReferenceId&amp;quot; field.&lt;/p&gt;&#xA;&lt;p&gt;So you would have to augment the dataset with a new column, &amp;quot;gameOutcome&amp;quot;, which has values (win/loss/draw), by looking up the outcome of each game (each row) and adding that to your dataset.&lt;/p&gt;&#xA;&lt;p&gt;Once you have this, you have the 12 independent variables (the 12 columns already there), and the 1 dependent variable (the &amp;quot;gameOutcome&amp;quot;), and the prediction task should be straightforward from there.&lt;/p&gt;&#xA;&lt;h2&gt;&amp;quot;Title: Given a label, how do I predict features?&amp;quot;&lt;/h2&gt;&#xA;&lt;p&gt;(note: this section will not help your program)&lt;/p&gt;&#xA;&lt;p&gt;What you are looking for are &lt;strong&gt;generative models&lt;/strong&gt;. Generative models can generate instances given the label and/or some random seed. This is a completely different model than &lt;strong&gt;discriminative models&lt;/strong&gt;, which given the instance predicts the label (this is the one you have).&lt;/p&gt;&#xA;&lt;p&gt;The simplest generative model is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Naive_Bayes_classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;Naive Bayes&lt;/a&gt; model. While Naive Bayes is normally used as a discriminative model (to classify a label), it has enough information to generate instances given a label as well. &lt;a href=&quot;https://www.quora.com/Why-is-naive-Bayes-considered-a-generative-model&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; are some tips on how to turn Naive Bayes into a generative model.&lt;/p&gt;&#xA;&lt;p&gt;If you are looking for generative deep neural network models, &lt;a href=&quot;https://openai.com/blog/generative-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; by OpenAI has a nice overview explaining the three modern approaches (generative adversarial networks, variational autoencoders, and autoregressive models). The blog explains at a high level what the strengths and weaknesses are of each approach, and links to some open source projects for you to try it out as well.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-19T23:11:18.173" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25298" PostTypeId="5" CreationDate="2020-12-19T23:49:30.667" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-19T23:49:30.667" LastActivityDate="2020-12-19T23:49:30.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25299" PostTypeId="4" CreationDate="2020-12-19T23:49:30.667" Score="0" Body="For questions related to the dataset used to train machine learning models, such as neural networks. The training dataset is different from the validation and test datasets, which are used for early stopping (and/or hyperparameter optimization) and to test the final model's performance, respectively." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T00:05:53.260" LastActivityDate="2020-12-20T00:05:53.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25300" PostTypeId="5" CreationDate="2020-12-20T12:02:46.613" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T12:02:46.613" LastActivityDate="2020-12-20T12:02:46.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25301" PostTypeId="4" CreationDate="2020-12-20T12:02:46.613" Score="0" Body="For questions related to the human brain in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T12:02:46.613" LastActivityDate="2020-12-20T12:02:46.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25302" PostTypeId="5" CreationDate="2020-12-20T12:38:38.690" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T12:38:38.690" LastActivityDate="2020-12-20T12:38:38.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25303" PostTypeId="4" CreationDate="2020-12-20T12:38:38.690" Score="0" Body="For questions related to the problem of labelling data and data labelling techniques in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T16:27:38.303" LastActivityDate="2020-12-20T16:27:38.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25304" PostTypeId="5" CreationDate="2020-12-20T12:45:55.940" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T12:45:55.940" LastActivityDate="2020-12-20T12:45:55.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25305" PostTypeId="4" CreationDate="2020-12-20T12:45:55.940" Score="0" Body="For questions related to the labels associated with data points." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T12:45:55.940" LastActivityDate="2020-12-20T12:45:55.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25306" PostTypeId="2" ParentId="25268" CreationDate="2020-12-20T13:20:12.703" Score="1" Body="&lt;p&gt;There are several different ways you can model the state and action spaces in such &lt;a href=&quot;https://en.wikipedia.org/wiki/Extensive-form_game#:%7E:text=An%20extensive%2Dform%20game%20is,player%20has%20about%20the%20other&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequential (extensive-form) environments/games&lt;/a&gt;. For environments with small action spaces or those typically introduced to beginning-RL students, the state space and action space remains constant along an agent's trajectory (termed &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal-form_game#:%7E:text=The%20normal%2Dform%20representation%20of,strategy%20spaces%20and%20payoff%20functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;normal form games&lt;/a&gt; when there are multiple agents). In sequential games which can be illustrated as trees, a &amp;quot;state&amp;quot; is analogous to &amp;quot;information set&amp;quot; which is defined as the sequence (tuple) of actions and observations since the beginning of the game's episode. Terminal states (leaf nodes) exist, and the action space &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}[x]$&lt;/span&gt; at an information set &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; can be defined as a union of action sequences that can be taken to each terminal state, not counting terminal states that cannot be reached from the current information set.&lt;/p&gt;&#xA;&lt;p&gt;In the above examples, I discussed games such as the examples you stated where more than one agent can interact with the environment, but this is a generalization over RL and can be applied to when only one agent is maximizing its reward.&lt;/p&gt;&#xA;" OwnerUserId="33444" LastActivityDate="2020-12-20T13:20:12.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25307" PostTypeId="1" AcceptedAnswerId="25322" CreationDate="2020-12-20T14:14:12.253" Score="3" ViewCount="74" Body="&lt;p&gt;We were given a list of labeled data (around 100) of known positive cases, i.e. people that have a certain disease, i.e. all these people are labeled with the same class (disease). We also have a much larger amount of data that we can label as negative cases (all patients that are not on the known positive patient's list).&lt;/p&gt;&#xA;&lt;p&gt;I know who the positives are, but how do I select negative cases to create a labeled dataset of both positives and negatives, on which to first train a neural network, and then test it?&lt;/p&gt;&#xA;&lt;p&gt;This is a common problem in the medical field, where doctors have lists of patients that are positive, but, in our case, we were not given a specific list of negative cases.&lt;/p&gt;&#xA;&lt;p&gt;I argued for picking a number that represents the true prevalence of the disease (around 1-2%). However, I was told that this isn't necessary and to do a 50:50 split of positives to negatives. It seems that doing it this way will not generalize outside our test and train datasets.&lt;/p&gt;&#xA;&lt;p&gt;What would you do in this case?&lt;/p&gt;&#xA;" OwnerUserId="41283" LastEditorUserId="2444" LastEditDate="2020-12-21T23:38:29.270" LastActivityDate="2020-12-21T23:38:29.270" Title="How do I select the (number of) negative cases, if I'm given a set of positive cases?" Tags="&lt;neural-networks&gt;&lt;imbalanced-datasets&gt;&lt;selection-bias&gt;&lt;training-datasets&gt;&lt;test-datasets&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25308" PostTypeId="1" CreationDate="2020-12-20T16:39:17.500" Score="1" ViewCount="60" Body="&lt;p&gt;Suppose that a neural network is trained with encrypted (for example, with &lt;a href=&quot;https://www.ijcai.org/Proceedings/2018/0547.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;homomorphic encryption&lt;/a&gt; and, more precisely, with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Paillier_cryptosystem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paillier partial scheme&lt;/a&gt;) data. Moreover, suppose that it is also trained with encrypted weights.&lt;/p&gt;&#xA;&lt;p&gt;If the neural network's weights are decrypted, is the performance of the neural network theoretically preserved or affected?&lt;/p&gt;&#xA;" OwnerUserId="43113" LastEditorUserId="2444" LastEditDate="2020-12-21T18:05:06.057" LastActivityDate="2020-12-21T18:05:06.057" Title="Is the performance of a neural network, which was trained with encrypted data and weights, affected if the weights are decrypted?" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;performance&gt;&lt;weights&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25309" PostTypeId="5" CreationDate="2020-12-20T16:41:46.590" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T16:41:46.590" LastActivityDate="2020-12-20T16:41:46.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25310" PostTypeId="4" CreationDate="2020-12-20T16:41:46.590" Score="0" Body="For questions related to the reward hypothesis (RH), i.e. the hypothesis that goals (in reinforcement learning) can be defined as the maximization of the expected value of the cumulative sum of the received reward, which is a scalar signal. According to Barto and Sutton's book, the RH was suggested by Michael Littman." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-20T16:41:46.590" LastActivityDate="2020-12-20T16:41:46.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25312" PostTypeId="1" CreationDate="2020-12-20T19:35:39.070" Score="1" ViewCount="74" Body="&lt;p&gt;I want to help people with cancer who are under chemotherapy, and generally people who have lost their hair to Virtually Try-On Toupees/Wigs on their head.&lt;/p&gt;&#xA;&lt;p&gt;VTO must support both the frontal and side positions of the head.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;At first,&lt;/strong&gt; I thought I could use traditional deep-learning to find landmarks, then place the hairstyle on the head.&lt;/p&gt;&#xA;&lt;p&gt;But the results were unrealistic and inaccurate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2D Placing of Hair:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some Hairstyles and Portraits fit well together:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ymWL3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ymWL3.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But some don't:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/87H1I.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/87H1I.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;They need manual modification:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6x2p2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6x2p2.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sometimes the original hair needs to be erased:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZEIV6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZEIV6.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;GANs promise more natural results with less manual engineering&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;changing hairstyle using GAN:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gdxmA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gdxmA.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;changing hairstyle using GAN:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NX2UM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NX2UM.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So, I decided to use GANs for this Task.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is it a good choice or is there an alternate solution?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="43128" LastEditorUserId="43231" LastEditDate="2020-12-29T14:27:35.910" LastActivityDate="2020-12-29T14:27:35.910" Title="Hairstyle Virtual Try On" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;variational-autoencoder&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25314" PostTypeId="1" CreationDate="2020-12-20T21:15:46.313" Score="1" ViewCount="13" Body="&lt;p&gt;The most used pre-trained networks for computer vision (e.g. ResNet50) are trained on 3 channels (RGB). At the same time, many cameras used in robotics return RGB-D outputs, that is including depth information.&lt;/p&gt;&#xA;&lt;p&gt;Fine tuning 3-channels pre-trained network on RGB-D outputs force us to give up one of the channels. Instead, is there a publicly available vision pre-trained network trained on RGB-D (4) channels?&lt;/p&gt;&#xA;" OwnerUserId="16363" LastActivityDate="2020-12-20T21:15:46.313" Title="Is there a pre-trained network trained on RGB-D (4) channels?" Tags="&lt;computer-vision&gt;&lt;datasets&gt;&lt;reference-request&gt;&lt;pretrained-models&gt;" AnswerCount="0" CommentCount="1" ClosedDate="2020-12-21T17:35:39.727" ContentLicense="CC BY-SA 4.0" />
  <row Id="25315" PostTypeId="1" CreationDate="2020-12-20T21:44:30.563" Score="1" ViewCount="143" Body="&lt;p&gt;I came across the term &lt;strong&gt;MNLI-(m/mm)&lt;/strong&gt; in Table 1 of the paper &lt;a href=&quot;https://arxiv.org/pdf/1810.04805.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding&lt;/a&gt;. I know what MNLI stands for, i.e. Multi-Genre Natural Language Inference, but I'm just unsure about the &lt;strong&gt;-(m/mm)&lt;/strong&gt; part.&lt;/p&gt;&#xA;&lt;p&gt;I tried to find some information about this in the paper &lt;a href=&quot;https://arxiv.org/pdf/1804.07461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding&lt;/a&gt;, but this explained only the basic Multi-Genre Language Inference concept. I assume that the &lt;strong&gt;m/mm&lt;/strong&gt; part was introduced later, but this doesn't make any sense because the BERT paper appeared earlier.&lt;/p&gt;&#xA;&lt;p&gt;It would be nice if someone knows this or has a paper that explains this.&lt;/p&gt;&#xA;" OwnerUserId="43132" LastEditorUserId="2444" LastEditDate="2020-12-21T00:28:32.397" LastActivityDate="2020-12-21T00:28:32.397" Title="What is MNLI-(m/mm)?" Tags="&lt;terminology&gt;&lt;transformer&gt;&lt;bert&gt;&lt;natural-language-understanding&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25316" PostTypeId="1" CreationDate="2020-12-20T21:49:13.787" Score="5" ViewCount="255" Body="&lt;p&gt;What does the Bellman equation actually say? And are there many flavours of that?&lt;/p&gt;&#xA;&lt;p&gt;I get a little confused when I look for the Bellman equation, because I feel like people are telling slightly different things about what it is. And I think the Bellman Equation is just basic philosophy and you can do whatever you want with that.&lt;/p&gt;&#xA;&lt;p&gt;The interpretations that I have seen so far:&lt;/p&gt;&#xA;&lt;p&gt;Let's consider this grid world.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+&#xA;| S6 | S7 | S8 |&#xA;+----+----+----+&#xA;| S3 | S4 | S5 |&#xA;+----+----+----+&#xA;| S0 | S1 | S2 |&#xA;+----+----+----+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rewards: S1:10; S3:10&lt;/li&gt;&#xA;&lt;li&gt;Starting Point: S0&lt;/li&gt;&#xA;&lt;li&gt;Horizon: 2&lt;/li&gt;&#xA;&lt;li&gt;Actions: Up, Down, Left, Right (If an action is not valid because there is no space, you remain in your position)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;The V-Function/Value:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It tells you how good is it to be in a certain state.&lt;/p&gt;&#xA;&lt;p&gt;With a horizon of 2, one can reach:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S0==&amp;gt;S3 (Up)   (R 5)&#xA;S0==&amp;gt;S0 (Down) (R 0)&#xA;S0==&amp;gt;S1 (Right)(R10)&#xA;S0==&amp;gt;S0 (Left) (R 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From that onwards&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S0==&amp;gt;S3 (Up)   (R 5)&#xA;S0==&amp;gt;S0 (Down) (R 0)&#xA;S0==&amp;gt;S1 (Right)(R10)&#xA;S0==&amp;gt;S0 (Left) (R 0)&#xA;&#xA;S1==&amp;gt;S4 (Up)   (R 0)&#xA;S1==&amp;gt;S1 (Down) (R10)&#xA;S1==&amp;gt;S2 (Right)(R 0)&#xA;S1==&amp;gt;S0 (Left) (R 0)&#xA;&#xA;S3==&amp;gt;S6 (Up)   (R 0)&#xA;S3==&amp;gt;S0 (Down) (R 0)&#xA;S3==&amp;gt;S3 (Right)(R 5)&#xA;S3==&amp;gt;S2 (Left) (R10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Considering no discount, this would mean that it is R=45 good to be in S0, because these are the options. Of course, you can't grab every reward, because you have to decide. Do I need to consider the best next state yet, because this would obviously reduce my expected total reward, but as I can only make two steps it would tell me what is really possible. Not what the overall Reward R(s) in that range is.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Q-Function/Value&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This function takes a state and an action, but I am not sure. If that means that I have a reward function that just considers my actions as well to give me a reward. Because in the previous example I just have to land on a state (It doesn't really matter how I get there). But this time I get a reward, when I choose a certain action. R(s,a)&#xA;But otherwise I do not rate the best action and select that next state to calculate the next state. I choose every next step and from that I choose the 2nd next.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Optimization V-function or Q-function&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This works the same as V-Function or Q-Function, but it just considers the next best award. Some sort of greedy approach:&lt;/p&gt;&#xA;&lt;p&gt;First step:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S0==&amp;gt;S3 (Up)   (R 5) [x]&#xA;S0==&amp;gt;S0 (Down) (R 0) [x]&#xA;S0==&amp;gt;S1 (Right)(R10)&#xA;S0==&amp;gt;S0 (Left) (R 0) [x]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Second Step:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S1==&amp;gt;S4 (Up)   (R 0) [x]&#xA;S1==&amp;gt;S1 (Down) (R10) &#xA;S1==&amp;gt;S2 (Right)(R 0) [x]&#xA;S1==&amp;gt;S0 (Left) (R 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, this would say that is the best I can do in two steps. I know that there is a problem, because when I just follow a greedy approach I risk that I won't get the best result, if I would have had a reward of 1000 on S2 later.&lt;/p&gt;&#xA;&lt;p&gt;But still, I just want to know, if I have a correct understanding. I know there might be many flavours and interpretations but at least I want to know that is the correct name of these approaches.&lt;/p&gt;&#xA;" OwnerUserId="43131" LastEditorUserId="42731" LastEditDate="2020-12-21T19:29:23.447" LastActivityDate="2020-12-21T19:29:23.447" Title="What is the Bellman Equation actually telling?" Tags="&lt;reinforcement-learning&gt;&lt;definitions&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25317" PostTypeId="1" CreationDate="2020-12-20T23:06:19.053" Score="1" ViewCount="22" Body="&lt;p&gt;There are few challenges I am facing when building a resume recommendation for a particular job positing.&lt;/p&gt;&#xA;&lt;p&gt;Let's say we convert the resume into a vector on n-dimensions and job description also as an n-dimension vector and in order to see how similar, we can use any similarity metrics like cosine.&lt;/p&gt;&#xA;&lt;p&gt;Now, for me the biggest problem with such approach is not able to provide more importance to the job title required. Some times, for a cloud engineer position I am getting java developer resume recommended in top 10 just because some of the skills/keywords are overlapping between two so their embeddings becomes similar.&lt;/p&gt;&#xA;&lt;p&gt;I want to provide more weightage to the job title as well. What are some possible things I can do to make my recommendations consider or put bit emphasis on job title.&lt;/p&gt;&#xA;&lt;p&gt;Note:- A common job title lookup in resume will fails because people write job titles in multiple ways.&#xA;(java engineer (or) java developer) | (cloud engineer or aws engineer) etc.,&lt;/p&gt;&#xA;&lt;p&gt;How can I overcome this issue?&lt;/p&gt;&#xA;" OwnerUserId="32867" LastEditorUserId="32867" LastEditDate="2020-12-21T08:21:31.203" LastActivityDate="2020-12-21T08:21:31.203" Title="Building a resume recommendation for a job post?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25318" PostTypeId="2" ParentId="25199" CreationDate="2020-12-20T23:09:27.573" Score="6" Body="&lt;p&gt;If you start with perpect discriminator, loss function will be saturated, and gradient of loss will be very small, so feedback for the generator also will be small, and learning will be slow down as a result. Actually, it is allways desired for discriminator and generator to learn balancedly. Additionally, it is claimed that Wasserstein Loss take care of this problem.&lt;/p&gt;&#xA;&lt;p&gt;You can find more information in &lt;a href=&quot;https://lilianweng.github.io/lil-log/2017/08/20/from-GAN-to-WGAN.html#vanishing-gradient&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. (I strongly suggest to read)&lt;/p&gt;&#xA;&lt;p&gt;Also from the paper &amp;quot; &lt;a href=&quot;https://arxiv.org/abs/1701.04862&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards principled methods for training generative adversarial networks&lt;/a&gt;&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In theory, one would expect therefore that we would first train the discriminator as close as we can to optimality (so the&#xA;cost function on &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; better approximates the JSD), and then do gradient steps on  &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, alternating these&#xA;two things. However, this doesn’t work. In practice, as the discriminator gets better, the updates&#xA;to the generator get consistently worse. The original GAN paper argued that this issue arose from&#xA;saturation, and switched to another similar cost function that doesn’t have this problem. However,&#xA;even with this new cost function, updates tend to get worse and optimization gets massively unstable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note 1: &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is parameters of generator&lt;/p&gt;&#xA;&lt;p&gt;Note 2: JSD: Jensen–Shannon divergence. For optimal discrimantor, loss is equal to &lt;span class=&quot;math-container&quot;&gt;$L(D^{ \ast }, g_{\theta})=  2 JSD(P_{r}, P_{g}) - 2log(2)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also from the paper of &lt;a href=&quot;https://arxiv.org/abs/1701.07875&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wasserstein Loss&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The fact that the EM distance is continuous and differentiable a.e. means that&#xA;we can (and should) train the critic till optimality. The argument is simple, the more we train the critic, the more reliable gradient of the Wasserstein we get, which is actually useful by the fact that Wasserstein is differentiable almost everywhere. For the JS, as the discriminator gets better the gradients get more reliable but the true gradient is 0 since the JS is locally saturated and we get vanishing gradients ... In Figure 2 we show a proof of concept of this, where we train a GAN discriminator and a WGAN critic till optimality. The discriminator learns very quickly to distinguish between fake and real, and as expected provides no reliable gradient information. The critic, however, can’t saturate, and converges to a linear function that gives remarkably clean gradients everywhere.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sZPDK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sZPDK.jpg&quot; alt=&quot;Figure 2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="41615" LastEditorUserId="41615" LastEditDate="2020-12-24T16:47:38.163" LastActivityDate="2020-12-24T16:47:38.163" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25320" PostTypeId="1" CreationDate="2020-12-20T23:37:16.233" Score="2" ViewCount="72" Body="&lt;p&gt;To derive the policy gradient, we start by writing the equation for the probability of a certain trajectory (e.g. see &lt;a href=&quot;https://spinningup.openai.com/en/latest/spinningup/rl_intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;spinningup tutorial&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;P_\theta(\tau) &amp;amp;= P_\theta(s_0, a_0, s_1, a_1, \dots, s_T, a_T) \\&#xA;&amp;amp; = p(s_0) \prod_{i=0}^T \pi_\theta(a_i | s_i) p(s_{i+1} | s_i, a_i)&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The expression is based on the chain rule for probability. My understanding is that the application of the chain rule should give up this expression:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p(s_0)\prod_{i=0}^T \pi_\theta(a_i|s_i, a_{i-1}, s_{i-1}, a_{i-2}, \dots, s_0, a_0) p(s_{i+1} | s_i, a_i, s_{i-1}, a_{i-1}, \dots, a_0, s_0)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the Markov property should be applicable, producing the desired equality. This should only depend on the latest state-action pair.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here are my questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this true?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I watched this lecture about policy gradients, and &lt;a href=&quot;https://youtu.be/VSPYKXm_hMA?list=PL_iWQOsE6TfURIIhCrlt-wj9ByIVpbfGc&amp;amp;t=376&quot; rel=&quot;nofollow noreferrer&quot;&gt;at this time&lt;/a&gt; during the lecture, Sergey says that: &amp;quot;at no point did we use the Markov property when we derived the policy gradient&amp;quot;, which left me confused. I assumed that the initial step of calculating the trajectory probability was using the Markov property.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="24080" LastEditorUserId="2444" LastEditDate="2020-12-29T19:24:33.427" LastActivityDate="2020-12-29T19:24:33.427" Title="Policy gradient: Does it use the Markov property?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;policy-gradients&gt;&lt;markov-property&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25321" PostTypeId="2" ParentId="25316" CreationDate="2020-12-20T23:41:09.880" Score="2" Body="&lt;p&gt;For a Markov Decision Process &lt;span class=&quot;math-container&quot;&gt;$(\mathcal{S}, \mathcal{A}, P, R)$&lt;/span&gt; (here &lt;span class=&quot;math-container&quot;&gt;$P(s, s') = \mathbb{P}(S_{t+1} = s' | S_t = s, A_t = a))$&lt;/span&gt;;, let us define the value of being in a certain state. That is,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[\sum_{i=0}^\infty \gamma^{i+t}r(s_{t+i}, a_{i+t}) | S_t =s\right].$$&lt;/span&gt;&#xA;That is, the value of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is equal to the expected value of the discounted sum of future rewards, where the expectation is taken with respect to &lt;span class=&quot;math-container&quot;&gt;$\pi(\cdot | s)$&lt;/span&gt; the action policy and the environment dynamics &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We will now define&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{i=0}^\infty \gamma^{i+t}r(s_{t+i}, a_{i+t}) = G_t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, we can rewrite this as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[G_t | S_t =s\right] =  \mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[ r(s_t, a_t) + \gamma G_{t+1} | S_t =s\right].$$&lt;/span&gt;&#xA;The RHS is now in a nicer format and we can express it as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[G_t | S_t =s\right] = \mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[ r(s_t, a_t) + \gamma v_\pi(s') | S_t =s\right],$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is the state that we transition into at time &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is a random variable and we are taking the expectation over this according to the action policy and the environment dynamics -- this is because the state we transition into depends first on the action we select and then the environment dynamics &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can see that we now have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$v_\pi(s) = \mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[ r(s_t, a_t) + \gamma v_\pi(s') | S_t =s\right].$$&lt;/span&gt;&#xA;This is the Bellman equation (at least a form of it) and it expresses a recursive relationship between the values of states. That is, the value of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is equal to the expected immediate reward from being in this state plus the value of being in the state that we transition into. This relationship is useful in Reinforcement Learning as many algorithms use this equation to form update rules to approximate the value/state-action value function of the MDP, such as the  SARSA algorithm, so it is more than just a philosophy, it is the driving force behind many of the RL algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Now, when I say at least a form of it, that is because in RL it is also common to see a Bellman equation for the state action value function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$q_\pi(s, a) = \mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[ G_t | S_t = s, A_t = a\right] = \mathbb{E}_{a_i \sim \pi, s_i \sim P}\left[ r(s_t, a_t) + v_\pi(s') | S_t = s, A_t = a\right];$$&lt;/span&gt;&#xA;noting that the value function &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is the expectation of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; over the action space, i.e. we marginalise out the action we condition on -- thus there is the recursive relationship.&lt;/p&gt;&#xA;&lt;p&gt;As pointed out in the comments for this question, it is also worth noting that Bellman equations originated in Dynamic Programming, which exist to solve planning problems such as the knapsack problem.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2020-12-21T10:35:44.857" LastActivityDate="2020-12-21T10:35:44.857" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25322" PostTypeId="2" ParentId="25307" CreationDate="2020-12-21T00:05:20.040" Score="4" Body="&lt;h2&gt;Short answer&lt;/h2&gt;&#xA;&lt;p&gt;To select the proper dataset to construct, you should first figure out a metric to use to compare, and then select the dataset construction that gives the better metric. There is no single best metric, it depends on the task and your interpretation on what type of error is more important.&lt;/p&gt;&#xA;&lt;p&gt;If you believe it is important that errors should not be normalized across class, then use the overall accuracy, and keep your dataset distribution same as the natural distribution (so 1-2% positive cases).&lt;/p&gt;&#xA;&lt;p&gt;If you believe it is important that errors should be normalized across class, then use PR-AUC or ROC-AUC, and re-balance your dataset so that the samples a little more closer to 1:1. The exact ratio will only be determined after testing and comparing the PR-AUC or ROC-AUC metrics.&lt;/p&gt;&#xA;&lt;h2&gt;How to select the best metric?&lt;/h2&gt;&#xA;&lt;p&gt;Two popular metrics are ROC-AUC and PR-AUC. ROC curves (&lt;a href=&quot;https://en.wikipedia.org/wiki/Receiver_operating_characteristic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Receiver Operating Characteristics&lt;/a&gt;) plot the true positive rate vs false positive rate, while PR curves (&lt;a href=&quot;https://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow noreferrer&quot;&gt;Precision and Recall&lt;/a&gt;) plot the precision vs recall. AUC stands for &amp;quot;area under curve&amp;quot;, because you can achieve any single point in the curve by specifying the classifier threshold, so the sum of all points (i.e. the entire area under the curve) is the most general way of comparing if one model is doing better than another.&lt;/p&gt;&#xA;&lt;p&gt;Although both ROC curves and PR curves equalize class imbalance at some capacity, PR curves are more sensitive to class imbalance. The paper &lt;a href=&quot;http://pages.cs.wisc.edu/%7Ejdavis/davisgoadrichcamera2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Relationship Between Precision-Recall and ROC Curves&lt;/a&gt; concludes that if the PR-AUC is good then the ROC-AUC will always also be good, but not the other way around. The difference is due to the fact that if the dataset has huge class imbalance, a false positive hurts PR curves significantly more than ROC curves.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, total accuracy does not normalize class imbalance at all, so therefore favors the majority class.&lt;/p&gt;&#xA;&lt;p&gt;As a result:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if you do not care about normalizing the measures of class imbalance, choose total accuracy, which will optimize for the most # of correct cases (regardless of class)&lt;/li&gt;&#xA;&lt;li&gt;if you want to normalize your metric across class imbalance, and normalizing false positive errors across classes is at all important to you, choose PR-AUC&lt;/li&gt;&#xA;&lt;li&gt;if you want to normalize your metric across class imbalance, and don't care about normalizing false positive errors, PR-AUC or ROC-AUC may both be good for you&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If it helps, for most imbalance problems, people usually go for PR curves.&lt;/p&gt;&#xA;&lt;p&gt;By the way, (&lt;a href=&quot;https://arxiv.org/pdf/1710.05381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;) studies class imbalance in neural networks by optimizing the ROC curves, and show that you should definitely have equal numbers of positive and negative examples. So if you want the best performance in terms of ROC-AUC, you should do the 50:50 split. I haven't read any similar study that optimizes for PR-AUC, but my intuition tells me that it will have the same conclusion (you should do 50:50 split to optimize for PR-AUC as well).&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-21T03:15:07.847" LastActivityDate="2020-12-21T03:15:07.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25323" PostTypeId="5" CreationDate="2020-12-21T00:39:47.183" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-21T00:39:47.183" LastActivityDate="2020-12-21T00:39:47.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25324" PostTypeId="4" CreationDate="2020-12-21T00:39:47.183" Score="0" Body="For questions related to test (or testing) datasets in the context of machine learning. A test dataset is any dataset that is not used for training the model but just to evaluate it, in particular, its ability to generalize to unseen data." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-21T00:39:47.183" LastActivityDate="2020-12-21T00:39:47.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25325" PostTypeId="1" CreationDate="2020-12-21T04:55:31.177" Score="1" ViewCount="25" Body="&lt;p&gt;I work for a video tutoring organisation that uses &lt;a href=&quot;https://www.twilio.com/video&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twilio Video API&lt;/a&gt; to carry out the video class. I want to run the object detection and other computer vision models on the input stream. I am stuck at how to get this input stream, i think i will need the stream link of that stream which i can read by open cv2.VideoCapture)(....) method and then run my inference using some pretrained model or using inbuilt face detection from open-cv. is there a way i can do it using tensorflow js.. Can anyone help me with this. I want to understand what should be the parameter under brackets of cv2.VideoCapture(). is there any other way?&#xA;Here are some links that i have gone through&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/twilio/twilio-video.js/issues/333&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/twilio/twilio-video.js/issues/333&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.twilio.com/blog/build-video-chat-application-python-javascript-twilio-programmable-video&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.twilio.com/blog/build-video-chat-application-python-javascript-twilio-programmable-video&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.infoq.com/articles/video-stream-analytics-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.infoq.com/articles/video-stream-analytics-opencv/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;if the question is not clear, let me know what else is needed .. i can give more details&lt;/p&gt;&#xA;" OwnerUserId="40978" LastEditorUserId="40978" LastEditDate="2020-12-21T11:20:11.790" LastActivityDate="2020-12-21T11:20:11.790" Title="Reading Data from a Twilio Stream" Tags="&lt;computer-vision&gt;&lt;python&gt;&lt;javascript&gt;&lt;video-classification&gt;&lt;opencv&gt;" AnswerCount="0" CommentCount="3" ClosedDate="2020-12-21T17:32:42.253" ContentLicense="CC BY-SA 4.0" />
  <row Id="25326" PostTypeId="1" CreationDate="2020-12-21T10:28:11.300" Score="0" ViewCount="46" Body="&lt;p&gt;In the last years a lot of different Explainable AI solutions were released. I have been looking for a solution, which is able to compare different models in real use cases. For example, you have a model, which can predict current house prices. Therefore, you train it every two weeks, because you want to learn the new trends.&lt;/p&gt;&#xA;&lt;p&gt;The current Explainable AI solutions are only for a single model. Could you compare the outputs, e.g. SHAP to see changes in feature behavior? Is there a way to extract the change between these models to find new trends or problems in the data?&lt;/p&gt;&#xA;&lt;p&gt;I couldn’t find research papers or solutions which talk about my issues. Therefore, it is very likely that my idea doesn’t make sense, but what have I misunderstood.&lt;/p&gt;&#xA;" OwnerUserId="43152" LastActivityDate="2020-12-21T10:28:11.300" Title="Explainable AI for Model Comparison" Tags="&lt;explainable-ai&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25327" PostTypeId="2" ParentId="25316" CreationDate="2020-12-21T11:38:29.070" Score="1" Body="&lt;p&gt;Just to add to the previous answer some more background and intuition. The background of Bellman equation comes from optimal control theory of dynamic systems of form (in discrete time case)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;s_{k+1} = f_d(s_k, a_k) \tag{1}&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$s_k$&lt;/span&gt; represents state at time &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_k$&lt;/span&gt; action at time &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. The goal is to optimize multistage objective function of form&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;V_N(s_0) = \sum_{k=0}^N J(s_k, a_k)&#xA;\end{equation}&lt;/span&gt;&#xA;while satisfying dynamic constraints &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$J(\cdot)$&lt;/span&gt; is a stage cost at time &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. The product of this optimization are optimal control policies &lt;span class=&quot;math-container&quot;&gt;$a_k = \pi_k(s_k)$&lt;/span&gt; which provide optimal value for the multistage objective function. Bellman's principle of optimality states that, for the multistage optimization problems, the objective function value at timestep &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; should satisfy&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;V^*_{k}(s_k) &amp;amp;= \min_{a_k}[J(s_k, a_k) + V^*_{k-1}(s_{k+1})]\\&#xA;&amp;amp;= \min_{a_k}[J(s_k, a_k) + V^*_{k-1}(f_d(s_k, a_k))]&#xA;\end{align}&lt;/span&gt;&#xA;This can be of course proven, you can find the proof in any optimal control/dynamic programming book.&lt;/p&gt;&#xA;&lt;p&gt;This also makes intuitive sense. Consider that you are at timestep &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (end of trajectory you want to optimize). You only need to consider 1 action &lt;span class=&quot;math-container&quot;&gt;$a_N$&lt;/span&gt;. You would now go through all possible states &lt;span class=&quot;math-container&quot;&gt;$s_N$&lt;/span&gt; and pick action &lt;span class=&quot;math-container&quot;&gt;$a_N$&lt;/span&gt; which minimizes your stage cost &lt;span class=&quot;math-container&quot;&gt;$J(s_N, a_N)$&lt;/span&gt; for all those states separately. After you did that, you would go one step backwards in time and find optimal action &lt;span class=&quot;math-container&quot;&gt;$a_{N-1}$&lt;/span&gt; for all states &lt;span class=&quot;math-container&quot;&gt;$s_{N-1}$&lt;/span&gt;. According to the Bellman optimality principle, you only need to consider action &lt;span class=&quot;math-container&quot;&gt;$a_{N-1}$&lt;/span&gt; in your optimization, because you already know action &lt;span class=&quot;math-container&quot;&gt;$a_N$&lt;/span&gt; (calculated previously) which would minimize &lt;span class=&quot;math-container&quot;&gt;$J(s_N, a_N)$&lt;/span&gt; for any possible future state &lt;span class=&quot;math-container&quot;&gt;$s_N$&lt;/span&gt;. Then you would keep going backwards in time until &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt;. This is very useful because you don't need to consider all &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; timesteps at once, you only consider one timestep at a time which prevents combinatorial explosion for large &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;. Bellman optimality principle has been adapted to many different applications, including RL and stochastic systems.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2020-12-21T11:48:20.073" LastActivityDate="2020-12-21T11:48:20.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25328" PostTypeId="1" CreationDate="2020-12-21T18:20:07.583" Score="1" ViewCount="23" Body="&lt;p&gt;I’m reading the paper &lt;a href=&quot;https://arxiv.org/abs/1312.6203&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spectral Networks and Deep Locally Connected Networks on Graphs&lt;/a&gt; and I’m having a hard time understanding the notation shown in the picture below (the scribbles are mine):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4KzJF.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4KzJF.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Specifically, I don’t understand the notation for the matrix F. Why does it include an i and a j?&lt;/p&gt;&#xA;" OwnerUserId="27433" LastEditorUserId="27433" LastEditDate="2021-01-18T16:24:06.060" LastActivityDate="2021-01-18T16:24:06.060" Title="Spectral Networks and Deep Locally Connected Networks on Graphs" Tags="&lt;graph-neural-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25329" PostTypeId="1" AcceptedAnswerId="25339" CreationDate="2020-12-21T19:00:46.320" Score="2" ViewCount="98" Body="&lt;p&gt;I am searching for an academic (i.e. with maths formulae) textbook which covers (at least) the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GAN&lt;/li&gt;&#xA;&lt;li&gt;LSTM and transformers (e.g. seq2seq)&lt;/li&gt;&#xA;&lt;li&gt;Attention mechanism&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The closest match I got is &lt;em&gt;Deep Learning&lt;/em&gt; (2016, MIT Press) but it only deals with part of the above subjects.&lt;/p&gt;&#xA;" OwnerUserId="34314" LastEditorUserId="2444" LastEditDate="2020-12-21T22:47:03.517" LastActivityDate="2020-12-22T19:09:44.980" Title="Recent deep learning textbook (i.e. covering at least GANs, LSTM and transformers and attention)" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;generative-adversarial-networks&gt;&lt;transformer&gt;&lt;attention&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25330" PostTypeId="1" CreationDate="2020-12-21T20:06:02.227" Score="0" ViewCount="55" Body="&lt;p&gt;I have trained multiple one-class vanilla variational autoencoders that each learn the distribution of one class and have the same architecture. The classes are mostly discrete, but there are several cases where two or more classes are &amp;quot;pretty similar&amp;quot;, which I'd like to quantify.&lt;/p&gt;&#xA;&lt;p&gt;What is the best way to compare the different VAEs?&lt;/p&gt;&#xA;&lt;p&gt;Would it be better to create ONE Conditional VAE, conditioning on each class? I haven't done this yet because it very expensive to train on all examples at once.&lt;/p&gt;&#xA;&lt;p&gt;My initial idea was to create 1 model per class and then run examples from one class through a different class's VAE in order to compare the ELBO scores. From there I would be able to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a hierarchical clustering of all of the input examples using the ELBO against all model as the features.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a fully connected bipartite-network with input examples on one side, VAE models on the other, and ELBO score edge weights, where I could use a mixed-membership weighted stochastic block model (or other GNN) to find class relationships.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I've only seen one paper use ELBO scores as similarity metric (&lt;a href=&quot;https://doi.org/10.1101/235655&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1101/235655&lt;/a&gt;). I have also seen a symmetric KL divergence to compare two VAEs (&lt;a href=&quot;https://doi.org/10.1016/j.ins.2020.06.065&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1016/j.ins.2020.06.065&lt;/a&gt;). Another intriguing idea is to use one-class attention-VAEs (&lt;a href=&quot;https://arxiv.org/abs/1911.07389v7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1911.07389v7&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;What is that standard approach for this kind of problem?&lt;/p&gt;&#xA;&lt;p&gt;If you have any suggestions or other ways to interpret this problem, I would really appreciate it!&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&#xA;Eli&lt;/p&gt;&#xA;" OwnerUserId="43160" LastActivityDate="2020-12-21T20:06:02.227" Title="How to compare multiple one-class variational autoencoders?" Tags="&lt;autoencoders&gt;&lt;generative-model&gt;&lt;variational-autoencoder&gt;&lt;explainable-ai&gt;&lt;anomaly-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25331" PostTypeId="2" ParentId="25329" CreationDate="2020-12-21T20:23:50.447" Score="2" Body="&lt;p&gt;I recommend &lt;a href=&quot;https://www.amazon.fr/Introduction-Deep-Learning-Eugene-Charniak/dp/0262039516&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Introduction to Deep Learning&lt;/em&gt;&lt;/a&gt; by Eugene Charniak ISBN 978-0-262-03951-2 (MIT 2018). It mentions &lt;code&gt;GAN&lt;/code&gt; &amp;amp; &lt;code&gt;LSTM&lt;/code&gt; &amp;amp; &lt;code&gt;Attention&lt;/code&gt; (all three occurs in the index).&lt;/p&gt;&#xA;&lt;p&gt;But read also Pitrat's last book: &lt;a href=&quot;https://www.amazon.fr/Artificial-Beings-Conscience-Conscious-Machine-ebook/dp/B00BPAGQWK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Artificial Beings: The Conscience of a Conscious Machine&lt;/em&gt;&lt;/a&gt; - it does cover machine learning (but not in the &amp;quot;deep learning&amp;quot; sense) but was published before 2016.&lt;/p&gt;&#xA;&lt;p&gt;And see also &lt;a href=&quot;http://refpersys.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefPerSys&lt;/a&gt;. If you speak French, also see &lt;a href=&quot;https://afia.asso.fr/journee-hommage-j-pitrat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and more generally the &lt;a href=&quot;https://afia.asso.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AFIA&lt;/a&gt; organization.&lt;/p&gt;&#xA;" OwnerUserId="3335" LastEditorUserId="3335" LastEditDate="2020-12-21T21:08:41.563" LastActivityDate="2020-12-21T21:08:41.563" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="25332" PostTypeId="1" CreationDate="2020-12-21T20:41:43.350" Score="4" ViewCount="43" Body="&lt;p&gt;Regularization of weights (e.g. L1 or L2) keeps them small and standardized, which can help reduce data overfitting. From &lt;a href=&quot;https://machinelearningmastery.com/weight-regularization-to-reduce-overfitting-of-deep-learning-models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, regularization sounds favorable in many cases, but is it always encouraged? Are there scenarios in which it should be avoided?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2020-12-21T20:57:00.790" LastActivityDate="2020-12-21T20:57:00.790" Title="When is using weight regularization bad?" Tags="&lt;neural-networks&gt;&lt;regularization&gt;&lt;weights&gt;&lt;l2-regularization&gt;&lt;l1-regularization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25333" PostTypeId="2" ParentId="13221" CreationDate="2020-12-22T05:03:28.993" Score="3" Body="&lt;p&gt;My guess:&lt;/p&gt;&#xA;&lt;p&gt;If you add a few CNN layers before the input of the given model and train only those layers while keeping the given model's parameters frozen, you might get better result.&lt;/p&gt;&#xA;&lt;p&gt;Essentially these few extra layers would &amp;quot;transform&amp;quot; your input image into the appropriate shape, but with more accuracy since its trained and not hard coded.&lt;/p&gt;&#xA;" OwnerUserId="35679" LastActivityDate="2020-12-22T05:03:28.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25334" PostTypeId="1" CreationDate="2020-12-22T06:50:00.570" Score="2" ViewCount="21" Body="&lt;p&gt;I have seen at quite a few places the use of two-tower architecture. &lt;a href=&quot;https://arxiv.org/pdf/2002.05515.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt;(Fig 6) is one of the examples. Each tower computes embedding of a concept which is orthogonal to the concepts in the rest of the towers. Also, I have seen that the next step on getting an n-dimensional embedding from the towers an inner product is taken to measure Cosine similarity.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know how to interpret this Inner Product. The two n-dimensional embeddings represents different concepts. What is the meaning of similarity here ?&lt;/p&gt;&#xA;" OwnerUserId="43164" LastActivityDate="2020-12-22T06:50:00.570" Title="Interpretation of Inner Product in a two-tower model" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25335" PostTypeId="1" AcceptedAnswerId="25405" CreationDate="2020-12-22T08:30:41.363" Score="3" ViewCount="41" Body="&lt;p&gt;I need to solve a video classification problem. While looking for solutions, I only found solutions that transform this problem into a series of simpler image classification tasks. However, this method has a downside: we ignore the temporal relationship between the frames.&lt;/p&gt;&#xA;&lt;p&gt;So, how can I perform video classification, with a CNN, by taking into account the temporal relationships between frames?&lt;/p&gt;&#xA;" OwnerUserId="43170" LastEditorUserId="2444" LastEditDate="2020-12-29T11:11:01.050" LastActivityDate="2020-12-29T11:11:01.050" Title="How can I do video classification while taking into account the temporal dependencies of the frames?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;reference-request&gt;&lt;video-classification&gt;&lt;action-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25337" PostTypeId="2" ParentId="25239" CreationDate="2020-12-22T11:44:02.563" Score="0" Body="&lt;p&gt;I am a bit confused with the part of &lt;em&gt;&amp;quot;training a new neural network from scratch&amp;quot;&lt;/em&gt; and then the part of &lt;em&gt;&amp;quot;VGG's weights will be frozen&amp;quot;&lt;/em&gt; because the answer changes since they are different cases. It can also happen, from what I read, that you are using VGG as backbone of another network for transfer learning but it is not specified either. Anyway, I will try my best.&lt;/p&gt;&#xA;&lt;p&gt;The cases I see:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Training a network from scratch&lt;/strong&gt;: no, you don't need to undo your normalization. In fact, when training from scratch you can normalize however you want.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Training a network from scratch starting from pretrained weights&lt;/strong&gt;: no, you don't need to change your normalization. Although it would help to faster convergence because the weights are fitted to other kind of normalization. But as the network trains it will fit to your new normalization without any problem.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Training a network that uses VGG as backbone with frozen weights&lt;/strong&gt;: no, you don't need to change your normalization although it would help. In spite of having your backbone fitted for a specific kind of normalization your network head is still an universal approximator that would fit to any other kind of normalization. However if you use the same normalization you are easing the network's job (that's why we normalize in the first place)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;When you really need to force same normalization is in &lt;em&gt;inference&lt;/em&gt;. When you want to infer you really need to do the same normalization in preprocessing or you won't have any meaningful results.&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2020-12-22T11:44:02.563" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25338" PostTypeId="1" CreationDate="2020-12-22T18:49:35.900" Score="0" ViewCount="28" Body="&lt;p&gt;Given a set of constraints on the input data, I am looking for papers that discuss using symbolic methods (decision trees, rule based, etc.) as a separate source of certainty in a classification task. For example if I want to find a stop sign, I let the deep network make a guess. I then independently check its plausibility before making a decision by using handcrafted features such as a circle. I want the two methods to be completely agnostic towards each other. Is there a name for this method? Does someone know papers on that? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="43183" LastEditorUserId="43231" LastEditDate="2020-12-29T14:34:52.080" LastActivityDate="2020-12-29T14:34:52.080" Title="Papers on using symbolic methods as constraint on neural network?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;research&gt;&lt;symbolic-ai&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25339" PostTypeId="2" ParentId="25329" CreationDate="2020-12-22T19:02:34.890" Score="2" Body="&lt;p&gt;There are a few more books that were published after 2016 that cover some of the topics you are interested in. I've not read any of them, so I don't really know whether they are good or not, but I try to summarise if they cover some of the topics you may be interested in.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://faculty.neu.edu.cn/yury/AAI/Textbook/Deep%20Learning%20with%20Python.pdfhttp://faculty.neu.edu.cn/yury/AAI/Textbook/Deep%20Learning%20with%20Python.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning with Python&lt;/a&gt; (2017), by Francois Chollet (author of the initial Keras library), which covers GANs in &lt;a href=&quot;http://faculty.neu.edu.cn/yury/AAI/Textbook/Deep%20Learning%20with%20Python.pdf#page=328&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 8.5&lt;/a&gt; (p. 305), but it does not seem to cover transformers and attention mechanisms, although it covers other intermediate/advanced topics (not sure to which extent), such as text generation with LSTMs, DeepDream, Neural Style Transfer and VAEs&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.hdip-data-analytics.com/_media/resources/pdf/s4/grokking_deep_learning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grokking deep learning&lt;/a&gt; (2019), by Andrew Trask, which seems to cover some intermediate/advanced topics (such as LSTMs and related tasks), but no transformers or GANs (unless I missed them); you can find the accompanying code &lt;a href=&quot;https://github.com/iamtrask/Grokking-Deep-Learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://books.google.ch/books?hl=en&amp;amp;lr=&amp;amp;id=RKegDwAAQBAJ&amp;amp;oi=fnd&amp;amp;pg=PP1&amp;amp;dq=Generative+Deep+Learning+david+foster&amp;amp;ots=bqE76IUkF6&amp;amp;sig=vXSbEW8S7ohXariAAhpbuckkKZs&amp;amp;redir_esc=y#v=onepage&amp;amp;q=Generative%20Deep%20Learning%20david%20foster&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play&lt;/a&gt;, by David Foster, which covers many variants of GANs, VAEs and other stuff&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;first transformer was published in 2017&lt;/a&gt;, so I guess there may not yet be a book that extensively covers it and other related models, such as the GPT models (if you're interested in CV, check &lt;a href=&quot;https://machinelearningmastery.com/computer-vision-books/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;, although it seems to list books that cover mostly traditional CV techniques). The attention mechanisms are older and can probably be found in textbooks that cover machine translation topics (such as seq2seq models with LSTMs), such as &lt;a href=&quot;http://mt-class.org/jhu/assets/nmt-book.pdf#page=57&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-22T19:09:44.980" LastActivityDate="2020-12-22T19:09:44.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25340" PostTypeId="1" AcceptedAnswerId="25342" CreationDate="2020-12-22T20:58:45.180" Score="1" ViewCount="39" Body="&lt;p&gt;In AlphaZero, we collect (&lt;span class=&quot;math-container&quot;&gt;$s_t, \pi_t, z_t$&lt;/span&gt;) tuples from self-play, where &lt;span class=&quot;math-container&quot;&gt;$s_t$&lt;/span&gt; is the board state, &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt; is the policy, and &lt;span class=&quot;math-container&quot;&gt;$z_t$&lt;/span&gt; is the reward from winning/losing the game. In other DeepRL off-policy algorithms (I'm assuming here that AlphaZero is off-policy (?)) like DQN, we maintain a memory buffer (say, 1 million samples) and overwrite the buffer with newer samples if it's at capacity. Do we do the same for AlphaZero? Or do we continually add new samples without overwriting older ones? The latter option sounds very memory heavy, but I haven't read anywhere that older samples are overwritten.&lt;/p&gt;&#xA;" OwnerUserId="43016" LastActivityDate="2020-12-23T00:08:21.243" Title="Is there a training data capacity limit for AlphaZero (Chess)?" Tags="&lt;machine-learning&gt;&lt;deep-rl&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25342" PostTypeId="2" ParentId="25340" CreationDate="2020-12-22T23:56:20.123" Score="1" Body="&lt;p&gt;AlphaZero is on-policy*, which partially answers your question.&lt;/p&gt;&#xA;&lt;p&gt;An on-policy algorithm is not the same as an online policy though, it is not required that updates are made on every step. It is simply required that all data used in the update is taken from the same &amp;quot;current&amp;quot; policy.&lt;/p&gt;&#xA;&lt;p&gt;In practice, AlphaZero buffers results from games played with the current policy to create a dataset used to update its neural networks. That buffer is then emptied after the data has been used.&lt;/p&gt;&#xA;&lt;p&gt;From the &lt;a href=&quot;https://arxiv.org/pdf/1712.01815.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaZero paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;At the end of the game, the terminal position &lt;span class=&quot;math-container&quot;&gt;$s_T$&lt;/span&gt; is&#xA;scored according to the rules of the game to compute the game outcome &lt;span class=&quot;math-container&quot;&gt;$z: −1$&lt;/span&gt; for a loss, &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for a draw, and &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; for a win. The neural network parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are updated so as to minimise the error between the predicted outcome &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt; and the game outcome &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, and to maximise the similarity of the policy vector &lt;span class=&quot;math-container&quot;&gt;$p_t$&lt;/span&gt;&#xA;to the search probabilities &lt;span class=&quot;math-container&quot;&gt;$\pi_t$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This implies only a single game is buffered in this way before running each update and then discarding the dataset generated in that game. Theoretically the same approach could be used with any number of games for each update step (provided the training system has capacity to store more moves).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;* AlphaZero is on-policy because the core algorithm requires using a specific policy and then updating it to match an improved version of the same policy discovered using MCTS for planning during play.&lt;/p&gt;&#xA;&lt;p&gt;It could be possible to construct an off-policy update mechanism using similar MCTS routine. I am not sure why this is not considered, but suspect it would be due to complexity/efficiency of the algorithm compared to the ease of generating new game data.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-12-23T00:08:21.243" LastActivityDate="2020-12-23T00:08:21.243" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25343" PostTypeId="2" ParentId="7522" CreationDate="2020-12-23T00:10:20.207" Score="0" Body="&lt;p&gt;Nowadays, there are many resources that cover the back-propagation algorithm and some of them provide step-by-step examples.&lt;/p&gt;&#xA;&lt;p&gt;However, in addition to &lt;a href=&quot;https://ai.stackexchange.com/a/7545/2444&quot;&gt;the other answer&lt;/a&gt;, I would like to mention the online book &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neural Networks and Deep Learning&lt;/a&gt; by Nielsen that covers &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the back-propagation algorithm&lt;/a&gt; (and other topics) in detail and, at the same, intuitively, although some could disagree. You can find the associated source code &lt;a href=&quot;https://github.com/mnielsen/neural-networks-and-deep-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (which I had consulted a few years ago when I was learning about the topic).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-12-23T00:10:20.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25344" PostTypeId="1" CreationDate="2020-12-23T06:14:49.180" Score="1" ViewCount="65" Body="&lt;p&gt;In reinforcement learning, there are model-based versus model-free methods. Within model-based ones, there are policy-based and value-based methods.&lt;/p&gt;&#xA;&lt;p&gt;AlphaGo Deepmind RL model has beaten the best Go human player. What kind of reinforcement model does it use? Why is this particular model appropriate for Go game?&lt;/p&gt;&#xA;" OwnerUserId="16839" LastActivityDate="2020-12-24T01:25:43.663" Title="What kind of reinforcement learning method does AlphaGo Deepmind use to beat the best human Go player?" Tags="&lt;reinforcement-learning&gt;&lt;policies&gt;&lt;model-based-methods&gt;&lt;model-free-methods&gt;&lt;value-based-methods&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25345" PostTypeId="1" CreationDate="2020-12-23T07:28:18.840" Score="0" ViewCount="30" Body="&lt;p&gt;While training a CNN model, I used an &lt;a href=&quot;https://keras.io/api/layers/regularizers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;l1_l2&lt;/code&gt; regularization&lt;/a&gt; (i.e. I applied both &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; regularization) on the final layers. While training, I saw the training and validation losses are dropping very nicely, but the accuracies aren't changing at all! Is that due to the high regularization rate?&lt;/p&gt;&#xA;" OwnerUserId="42948" LastEditorUserId="2444" LastEditDate="2020-12-23T19:23:24.870" LastActivityDate="2020-12-23T19:23:24.870" Title="What is the effect of too harsh regularization?" Tags="&lt;convolutional-neural-networks&gt;&lt;loss-functions&gt;&lt;accuracy&gt;&lt;l2-regularization&gt;&lt;l1-regularization&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25346" PostTypeId="1" CreationDate="2020-12-23T07:56:31.160" Score="0" ViewCount="57" Body="&lt;p&gt;I recently tried to reproduce the results of &lt;a href=&quot;https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;double Q-learning&lt;/a&gt;. However, the results are not satisfying. I have also tried to compare double Q learning with Q-learning in Taxi-v3, FrozenLake without slippery, Roulette-v0, etc. But Q-learning outperforms double Q-learning in all of these environments.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure whether if there is something wrong with my implementation as many materials about double Q actually focus on double DQN. While at the same time of checking, I wonder is there any toy example that can exemplify the performance of double Q-learning?&lt;/p&gt;&#xA;" OwnerUserId="43200" LastEditorUserId="36821" LastEditDate="2020-12-27T10:18:49.683" LastActivityDate="2020-12-27T10:18:49.683" Title="Is there any toy example that can exemplify the performance of double Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;double-q-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25347" PostTypeId="1" CreationDate="2020-12-23T09:20:26.087" Score="0" ViewCount="10" Body="&lt;p&gt;I hope the question is understandable.&lt;/p&gt;&#xA;&lt;p&gt;I just wanted to ask if the hidden state, which is passed through the timesteps/cells of an RNN/LSTM/GRU to deliver information from &lt;span class=&quot;math-container&quot;&gt;$\text{cell}_{i-1}$&lt;/span&gt; to the &lt;span class=&quot;math-container&quot;&gt;$\text{cell}_{i}$&lt;/span&gt; contains information (after the last timestep) which can contribute to further classification of the input-sequence.&lt;/p&gt;&#xA;&lt;p&gt;So, what would happen when we would pass the hidden state through dense layers and finally to a classification layer? Is the information of the hidden state useful out of the actual RNN?&lt;/p&gt;&#xA;" OwnerUserId="43202" LastEditorUserId="2444" LastEditDate="2020-12-23T09:25:55.207" LastActivityDate="2020-12-23T09:25:55.207" Title="Is the information in the hiddenstate of a RNN worth processing further after the input passes the RNN?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25348" PostTypeId="1" CreationDate="2020-12-23T11:12:04.720" Score="1" ViewCount="40" Body="&lt;p&gt;This is my first question on this forum and I would like to welcome everyone.&#xA;I am trying to implement DDQN Agent playing Othello (Reversi) game. I have tried multiple things but the agent which seems to be properly initialized does not learn against random opponent. Actually the score is about 50-60% won games out of nearly 500. Generally if it gets some score after first 20-50 episodes it stays on the same level. I have doubts on the process of learning and how to decide when the agent is trained. Current flow is as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize game state.&lt;/li&gt;&#xA;&lt;li&gt;With epsilon greedy policy choose the action to make based on currently available actions depending on game state&lt;/li&gt;&#xA;&lt;li&gt;Get opponent to make his action&lt;/li&gt;&#xA;&lt;li&gt;Get the reward as number of flipped places that remain after opponent move.&lt;/li&gt;&#xA;&lt;li&gt;Save the observation to replay buffer&lt;/li&gt;&#xA;&lt;li&gt;If number of elements in replay buffer equals or higher than batch size do the training.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I do not know is when do I know when to stop the training. Previously this agent trained against MinMax algorithm learned how to win 100% games because MinMax played exactly the same every time. I would like the agent to generalize the game. Right now I save the network weights after the game is won but I think it does not matter. I can't see that this agent find some policy and improve over time. Whole code for the environment, agent and training loop can be found here: &lt;a href=&quot;https://github.com/MikolajMichalski/RL_othello_mgr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/MikolajMichalski/RL_othello_mgr&lt;/a&gt; I would appreciate any help. I would like to understand how the RL works :)&lt;/p&gt;&#xA;" OwnerUserId="43161" LastActivityDate="2020-12-23T11:12:04.720" Title="DDQN Agent in Othello (Reversi) game struggle to learn" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;double-dqn&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="25349" PostTypeId="1" CreationDate="2020-12-23T13:11:29.020" Score="2" ViewCount="30" Body="&lt;p&gt;I have a dataset consisting of a set of samples. Each sample consists of two distinct desctized signals S1(t), S2(t). Both signals are synchronous; however, they show different aspects of a phenomena.&lt;/p&gt;&#xA;&lt;p&gt;I want to train a Convolutional Neural Network, but I don't know which architecture is appropriate for this kind of data. &lt;br/&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can consider two channels for input, each corresponding to one of the signals. But, I don't think convolving two signals can produce appropriate features.&lt;/p&gt;&#xA;&lt;p&gt;I believe the best way is to process each signal separately in the first layers, then join them in the classification layers in the final step. How can I achieve this? What architecture should I use?&lt;/p&gt;&#xA;" OwnerUserId="43208" LastEditorUserId="43231" LastEditDate="2020-12-25T02:47:44.740" LastActivityDate="2020-12-25T02:47:44.740" Title="Appropriate convolutional neural network architecture when the input consists of two distinct signals" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25350" PostTypeId="2" ParentId="25349" CreationDate="2020-12-23T14:07:30.560" Score="1" Body="&lt;p&gt;I don't know what you mean by &lt;em&gt;desctized signals&lt;/em&gt; but if I understand your question correctly, separating two signal and passing them through same architecture of CNN (even with different parameters) is not a good idea. Because when they are together (as different channels) they will be treated differently by the CNN (each channel has its own parameters) and even this way the network is able to combine these two signals and get better results by information extracted from this combination.&lt;/p&gt;&#xA;" OwnerUserId="41547" LastActivityDate="2020-12-23T14:07:30.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25351" PostTypeId="1" CreationDate="2020-12-23T15:00:43.667" Score="0" ViewCount="19" Body="&lt;p&gt;I'm making a platform that will collect data about the relationship between different images. For example, if I have three images: a Christmas tree, a gift and Santa... then these will be connected by the user with the word Christmas.&#xA;&lt;br&gt;&#xA;I want to know where this kind of data can be helpful. For example to train some ML model somewhere in the medical field or some other place that can make good use of it for the betterment of society.&lt;/p&gt;&#xA;" OwnerUserId="43211" LastActivityDate="2020-12-23T15:00:43.667" Title="Where can a dataset of relationship between images be used?" Tags="&lt;datasets&gt;&lt;structured-data&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25352" PostTypeId="1" AcceptedAnswerId="25363" CreationDate="2020-12-23T16:09:01.743" Score="1" ViewCount="70" Body="&lt;p&gt;I'm trying to understand the solution to question 4 of &lt;a href=&quot;https://alex.smola.org/teaching/10-701-15/exam/midterm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; midterm paper.&lt;/p&gt;&#xA;&lt;p&gt;The question and solution is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pdTDw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pdTDw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I thought that the process for updating weights was:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;error = target - guess&#xA;new_weight = current_weight + (error)(input)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do not understand for example, for number 2 below, how that sum is determined. For example, I want to understand whether to update the weight or not. The calculation is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x1(w1) + x2(w2)&#xA;(10)(1) + (10)(1) = 20&#xA;20 &amp;gt; 0, therefore update.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the equation to obtain the same answer in the solution is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1(10 + 10) 20&#xA;20 &amp;gt; 0, therefore update.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I understand that these two equations are essentially the same, but written differently. But for example, in step 5, what do the elements in g5 mean. What do the -8, -16 and -2 represent?&lt;/p&gt;&#xA;&lt;p&gt;p.s. I know in a previous (now deleted) post of mine, I asked a question related to the use of LaTeX instead for maths equations. If someone can show me a simple way to convert these equations online, I'm more than happy to use it. However, I'm unfamiliar with this software, so I need some sort of converter.&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="43231" LastEditDate="2020-12-25T11:10:07.217" LastActivityDate="2020-12-25T11:10:07.217" Title="What is the equation to update the weights in the perceptron algorithm?" Tags="&lt;machine-learning&gt;&lt;perceptron&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25353" PostTypeId="2" ParentId="25349" CreationDate="2020-12-23T16:09:49.110" Score="0" Body="&lt;p&gt;You can safely give both signals as input in different channels. Actually, it's the best way. This way, the network is able to find low-degree patterns that involve both signals early in training. This will therefore enable the early discovery of more complex patterns, too.&lt;/p&gt;&#xA;&lt;p&gt;Differently from what one might understand from your question, the two signals will not be convolved against eachother, as it's typically done in signal processing. The convolution taking place is that of the first layer kernels with a two-component signal (the one you give as input). It can happen that there's a first order pattern that can only be recognised by looking at both signals at the same time. If that's not the case, the kernels will ignore one signal or the other (having the corresponding weights a value of zero).&lt;/p&gt;&#xA;" OwnerUserId="27444" LastActivityDate="2020-12-23T16:09:49.110" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25356" PostTypeId="2" ParentId="25262" CreationDate="2020-12-23T18:30:05.000" Score="0" Body="&lt;p&gt;After doing some further reading, it turns out that negative rewards &lt;em&gt;are&lt;/em&gt; an assumption for this distribution to hold. However, the author notes that as long as you don't receive a reward of infinity for any action then it is possible to re-scale your rewards by subtracting the maximum value of your potential rewards so that they are always negative.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2020-12-23T18:30:05.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25358" PostTypeId="1" AcceptedAnswerId="25382" CreationDate="2020-12-23T20:39:26.650" Score="0" ViewCount="64" Body="&lt;p&gt;I am totally new to artificial intelligence and neural networks and have a broad question that I hope is appropriate to ask here.&lt;/p&gt;&#xA;&lt;p&gt;I am an ecologist working in animal movement and I want to use AI to apply to my field. This will be one of the few times this has been attempted so there is not much literature to help me here.&lt;/p&gt;&#xA;&lt;p&gt;My dataset is binary. In short I have presence(1) and absence(0) of animal locations that are associated with a series of covariates (~20 environmental conditions such as temperature etc.). I have ~1 million rows of data to train the model on with a ratio of 1:100 (presence:absence).&lt;/p&gt;&#xA;&lt;p&gt;Once trained, I would like a model that can predict if an animal will be in a location (or give a probability) based on new covariates (environmental conditions).&lt;/p&gt;&#xA;&lt;p&gt;Is this sort of thing possible using AI?&lt;/p&gt;&#xA;&lt;p&gt;If so where should I be looking for resources. I write in 'R', should I learn 'Python'?&lt;/p&gt;&#xA;&lt;p&gt;Any input is much appreciated here!&lt;/p&gt;&#xA;" OwnerUserId="43221" LastEditorUserId="43221" LastEditDate="2020-12-25T18:04:12.020" LastActivityDate="2020-12-25T18:55:01.383" Title="Can you use machine learning for binary data?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;applications&gt;&lt;probability&gt;&lt;model-based-methods&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25361" PostTypeId="2" ParentId="25287" CreationDate="2020-12-24T03:24:23.987" Score="-1" Body="&lt;p&gt;I think what he means is that, while distribution &lt;span class=&quot;math-container&quot;&gt;$P(y  \mid x,s)$&lt;/span&gt; has three variables, &lt;span class=&quot;math-container&quot;&gt;$P(y  \mid x)$&lt;/span&gt; has two variables. The number of parameters required to describe a distribution (or samples to approximate it) grows exponentially with the number of variables (for more info see for instance Ian goodfellow, .. deep learning&amp;quot;).&lt;/p&gt;&#xA;" OwnerUserId="43208" LastEditorUserId="43208" LastEditDate="2021-01-06T09:50:30.573" LastActivityDate="2021-01-06T09:50:30.573" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25363" PostTypeId="2" ParentId="25352" CreationDate="2020-12-24T06:09:56.067" Score="1" Body="&lt;p&gt;I will tell you my knowledge, correct me if I am wrong.&lt;/p&gt;&#xA;&lt;p&gt;Perceptron Learning Algorithm (PLA) is a simple method to solve the &lt;strong&gt;binary classification&lt;/strong&gt; problem.&lt;/p&gt;&#xA;&lt;p&gt;Define a function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_w(x) = w^Tx + b&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x \in \mathbb{R}^n$&lt;/span&gt; is an input vector that contains data points and &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is a vector with the same dimension as &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; which present for the parameters of our model.&lt;/p&gt;&#xA;&lt;p&gt;Call &lt;span class=&quot;math-container&quot;&gt;$y=label(x)=\{1,-1\}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; are the label of each &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; vector.&lt;/p&gt;&#xA;&lt;p&gt;The PLA will predict a class like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;y=label(x)=sgn(f_w(x))=sgn(w^Tx+b)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(The definition of sgn function can be found in &lt;a href=&quot;https://en.wikipedia.org/wiki/Sign_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;this wiki&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;We can understand that PLA tries to define a line (in 2D, or a plane in 3D, and hyperplane in more than 3 dimensions coordinate, I will assume it in 2D from now on) which separate our data into two areas. So how can we find that line? Just like every other machine learning problems, define a cost function, then optimize the parameters to have the smallest cost value.&lt;/p&gt;&#xA;&lt;p&gt;Now, let define the cost function first, you can see that if a data point lies in the correct area, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; have the same sign, which means &lt;span class=&quot;math-container&quot;&gt;$y(w^Tx+b) &amp;gt; 0$&lt;/span&gt; and otherwise. Similar to your example, I will define:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;g(x)=y(w^Tx+b)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We ignore all the points in the safe zone (&lt;span class=&quot;math-container&quot;&gt;$g(x)&amp;gt;0$&lt;/span&gt;), only update to rotate or move the line to adapt with the misclassified points (&lt;span class=&quot;math-container&quot;&gt;$g(x)\le 0$&lt;/span&gt;), here, you can understand why we only update if &lt;span class=&quot;math-container&quot;&gt;$g(x)\le0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We need to define a cost function to minimize it, so our cost function will become:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(w)=\displaystyle\sum_{x_i\in U}(-y_i(w^Tx_i+b))&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;is the set of the misclassified points&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is the label of data point &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th data vector&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is parameters of our model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For each data point, we have the derivative is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial L}{\partial w} = -y_ix_i \\&#xA;\frac{\partial L}{\partial b} = -y_i&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, update them by Stochastic gradient descent (SGD), we get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w = w - \frac{\partial L}{\partial w} = w + y_ix_i \\&#xA;b = b - \frac{\partial L}{\partial b} = b + y_i&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For your last question, notice that the weight and bias changed from &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;-th updated, so we have:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;y_5 = 1, x_5 = (4,8), w = (-2,-2), b = -2 \\&#xA;\Rightarrow g_5 = +1 \times (4\times(-2) + 8\times(-2)+ (-2)) = -8 -16 -2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="41287" LastActivityDate="2020-12-24T06:09:56.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25364" PostTypeId="1" AcceptedAnswerId="25372" CreationDate="2020-12-24T08:21:40.353" Score="0" ViewCount="41" Body="&lt;p&gt;What ensemble methods are used in the state-of-the-art models?&lt;/p&gt;&#xA;&lt;p&gt;When I surveyed the state-of-the-art methods of classification and detection, e.g. ImageNet, COCO, etc., I noticed that are few or even no references to the use of ensemble methods like bagging or boosting.&lt;/p&gt;&#xA;&lt;p&gt;Is it a bad idea to use them?&lt;/p&gt;&#xA;&lt;p&gt;However, I observed that many use ensemble in Kaggle competitions.&lt;/p&gt;&#xA;&lt;p&gt;What makes it so different between the two groups of researchers?&lt;/p&gt;&#xA;" OwnerUserId="18276" LastEditorUserId="2444" LastEditDate="2020-12-24T12:05:17.477" LastActivityDate="2020-12-25T09:52:17.237" Title="What ensemble methods are used in the state-of-the-art models?" Tags="&lt;machine-learning&gt;&lt;object-detection&gt;&lt;ensemble-learning&gt;&lt;boosting&gt;&lt;bagging&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25365" PostTypeId="1" CreationDate="2020-12-24T08:53:18.597" Score="0" ViewCount="11" Body="&lt;p&gt;This is a &lt;a href=&quot;https://www.youtube.com/watch?v=4PPolJT1gfI&quot; rel=&quot;nofollow noreferrer&quot;&gt;quiz&lt;/a&gt; on randomized hill climbing from Udacity's &lt;a href=&quot;https://classroom.udacity.com/courses/ud262&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning class&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Hbnv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Hbnv.png&quot; alt=&quot;img&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.youtube.com/watch?v=Ts9vLVxyqdY&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; in green throws a value of &lt;strong&gt;5.39&lt;/strong&gt; out of nowhere.  How do we get &lt;strong&gt;5.39 + v&lt;/strong&gt; and &lt;strong&gt;10 + v&lt;/strong&gt;?  The explanation is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...at which point we have to start the whole process over again...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Not sure what they are trying to say there.  Finally, we end up with a value &lt;strong&gt;v&lt;/strong&gt; of &lt;strong&gt;29.78&lt;/strong&gt; that is greater than &lt;strong&gt;10&lt;/strong&gt;, the greatest number of steps to find a local maximum.&lt;/p&gt;&#xA;&lt;p&gt;The explanation suggests four basins of attraction, one of which is good.  I can see 5 basins, and the two around the global maximum seems good.  Am I right?  If not, what are the four right basins, and the good one?&lt;/p&gt;&#xA;&lt;p&gt;It feels like a lot has been skipped from the video.  I am really confused.  Could someone explain this to me in plain English?  Really appreciate it!&lt;/p&gt;&#xA;" OwnerUserId="43229" LastEditorUserId="43229" LastEditDate="2020-12-24T09:21:14.927" LastActivityDate="2020-12-24T09:21:14.927" Title="Need help understanding the solution to this randomized hill climbing quiz" Tags="&lt;hill-climbing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25366" PostTypeId="1" CreationDate="2020-12-24T09:01:08.223" Score="1" ViewCount="23" Body="&lt;p&gt;I was going through &lt;a href=&quot;https://papers.nips.cc/paper/1997/file/c930eecd01935feef55942cc445f708f-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper by Hansen&lt;/a&gt;. This paper proposes policy improvement by first converting set of &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; vectors into finite state controller and then comparing them to obtain improved policy. The whole algorithm is summmarised as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LKJ0m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LKJ0m.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Section 4 of this paper explains the algorithm with example. I am unable to get the example, specifically how it forms those states, what are the numbers inside each state and how they are exactly calculated:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hmSE8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hmSE8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41169" LastEditorUserId="36821" LastEditDate="2020-12-25T10:16:27.830" LastActivityDate="2020-12-25T10:16:27.830" Title="Understanding example for Improved Policy Iteration for POMDPs" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;pomdp&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25367" PostTypeId="1" CreationDate="2020-12-24T09:52:22.783" Score="3" ViewCount="94" Body="&lt;p&gt;In lecture 4 of &lt;a href=&quot;https://gnn.seas.upenn.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this course&lt;/a&gt;, the instructor argues that GNNs are generalizations of CNNs, and that one can recover CNNs from GNNs.&lt;/p&gt;&#xA;&lt;p&gt;He presents the following diagram (on the right) and mentions that it represents both a CNN and a GNN. In particular, he mentions that if we particularize the graph shift operator (i.e the matrix S, which in the case of a GNN could represent the adjacency matrix or the Laplacian) to represent a directed line graph, then we obtain a time convolutional filter (which I hadn't heard of before watching this, but now I know that all it does is shift the graph signal in the direction of the arrows at each time step).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sGOlQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sGOlQ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That part I understand. What I don’t understand is how we can obtain 2D CNNs (the ones that we would for example apply to images) from GNNs. I was wondering if someone could explain.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: I found part of the answer &lt;a href=&quot;https://gnn.seas.upenn.edu/time-images-and-graphs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. However, it seems that the image convolution as defined is a bit different from what I’m used to. It seems like the convolution considers pixels only to the left and above of the “current” pixel whereas I’m used to convolutions considering both left, right, above, and below&lt;/p&gt;&#xA;" OwnerUserId="27433" LastEditorUserId="27433" LastEditDate="2020-12-27T12:58:04.680" LastActivityDate="2021-03-09T23:00:56.127" Title="Are Graph Neural Networks generalizations of Convolutional Neural Networks?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;comparison&gt;&lt;graph-neural-networks&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25368" PostTypeId="1" AcceptedAnswerId="25374" CreationDate="2020-12-24T11:29:36.297" Score="1" ViewCount="71" Body="&lt;p&gt;Reading the Retrace paper (&lt;a href=&quot;https://arxiv.org/pdf/1606.02647.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Safe and efficient off-policy reinforcement learning&lt;/a&gt;) I saw they often use a matrix form of the Bellman operators, for example as in the picture below. How do we derive those forms? Could you point me to some reference in which the matter is explained?&lt;/p&gt;&#xA;&lt;p&gt;I am familiar with the tabular RL framework, but I'm having trouble understanding the steps from operators to this matrix form. For example, why does &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi} = (I -\gamma P^{\pi})^{-1}r$&lt;/span&gt;? I know that for the value &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; we can write&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;V =  R + \gamma P^{\pi} V \\&#xA;V - \gamma P^{\pi} V =  R \\&#xA;V (I -\gamma P^{\pi}) =  R \\ &#xA;V = R(I - \gamma P^{\pi})^{-1}&#xA;\end{align}&lt;/span&gt;&#xA;but this seems slightly different.&lt;/p&gt;&#xA;&lt;p&gt;Picture from &lt;em&gt;Safe and efficient off-policy reinforcement learning&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hXzw6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hXzw6.png&quot; alt=&quot;Picture from Retrace paper&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="32583" LastEditorUserId="2444" LastEditDate="2020-12-24T12:17:03.730" LastActivityDate="2020-12-24T18:07:51.773" Title="How to derive matrix form of the Bellman operators?" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;value-functions&gt;&lt;bellman-equations&gt;&lt;bellman-operators&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25369" PostTypeId="1" AcceptedAnswerId="25396" CreationDate="2020-12-24T11:38:50.910" Score="2" ViewCount="212" Body="&lt;p&gt;My company has full access to beta testing for GPT-3. We wanted to try it for some games or game mechanics within Unity3D. Is it possible to use it for dialogues or with unity scripts?&lt;/p&gt;&#xA;&lt;p&gt;The Documents of OpenAI does not say anything about this possibility, so I'm not sure.&lt;/p&gt;&#xA;" OwnerUserId="43190" LastEditorUserId="43231" LastEditDate="2020-12-24T12:27:32.710" LastActivityDate="2020-12-28T21:12:51.810" Title="Is it possible to integrate the GPT-3 by OpenAPI inside Unity3D or any game-engine?" Tags="&lt;game-ai&gt;&lt;gpt&gt;&lt;dialogue-systems&gt;&lt;integration&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25370" PostTypeId="2" ParentId="25269" CreationDate="2020-12-24T12:42:12.133" Score="2" Body="&lt;p&gt;This is very easy to prove.&lt;/p&gt;&#xA;&lt;p&gt;Let's first prove that, if &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_k = y_k$&lt;/span&gt;, then the &lt;span class=&quot;math-container&quot;&gt;$E = 0$&lt;/span&gt;. I will leave all steps, so that it's super clear.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;E&#xA;&amp;amp;=\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2 \\&#xA;&amp;amp;=\frac{1}{2}\sum_k(y_k - y_k)^2\\&#xA;&amp;amp;=\frac{1}{2}\sum_k(0)^2\\&#xA;&amp;amp;=\frac{1}{2}\sum_k 0\\&#xA;&amp;amp;=\frac{1}{2} 0\\&#xA;&amp;amp;=0\\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;To prove the other way around, i.e. if &lt;span class=&quot;math-container&quot;&gt;$E = 0$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_k = y_k$&lt;/span&gt;, you can do as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2 &#xA;&amp;amp;=E\\&#xA;&amp;amp;=0&#xA;\end{align}&lt;/span&gt;&#xA;Recall now that any number squared is non-negative (i.e. positive or zero). Given that &lt;span class=&quot;math-container&quot;&gt;$(\hat{y}_k - y_k)^2 $&lt;/span&gt; is non-negative, then &lt;span class=&quot;math-container&quot;&gt;$\sum_k(\hat{y}_k - y_k)^2$&lt;/span&gt; is a sum of non-negative numbers. The only way that a sum of non-negative numbers is equal to zero is if all numbers are zero, so we must have &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_k = y_k$&lt;/span&gt; (because any non-zero number squared is non-zero).&lt;/p&gt;&#xA;&lt;p&gt;(Note that &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the &lt;strong&gt;mean squared error&lt;/strong&gt;, i.e. a loss function, and it's &lt;strong&gt;not&lt;/strong&gt; the back-propagation algorithm, which is just the algorithm that you use to compute partial derivatives of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; with respect to the parameters of the model, which are not even visible in the way you wrote &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-24T12:59:00.723" LastActivityDate="2020-12-24T12:59:00.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25371" PostTypeId="1" CreationDate="2020-12-24T13:57:35.350" Score="3" ViewCount="47" Body="&lt;p&gt;I'm trying to understand Artificial Life (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Artificial_life&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a simple background) in Computational Evolution.&lt;/p&gt;&#xA;&lt;p&gt;I understand that in this set of methods, you set up a dynamic environment (e.g. the ecology of the environment) and then you set a series of rules; e.g.:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You need energy to reproduce.&lt;/li&gt;&#xA;&lt;li&gt;You intake energy from food sources.&lt;/li&gt;&#xA;&lt;li&gt;For nourishment, you can&#xA;eat plants, animals, or steal food.&lt;/li&gt;&#xA;&lt;li&gt;You must stay alive until you reproduce.&lt;/li&gt;&#xA;&lt;li&gt;Every action consumes energy.&lt;/li&gt;&#xA;&lt;li&gt;When you have no energy left, you die.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I think I need a set of rules that govern the survival of an artificial life. You run the environment and see what persists (there's a set of rules instead of a fitness score), and the individuals that survive are said to be successful.&lt;/p&gt;&#xA;&lt;p&gt;I can imagine a scenario where a successful organism in this environment consumes a lot of food, reproduces, but possibly runs out of energy and dies. I'm wondering if there's ever a situation where an organism does very little (or nothing), and still be successful? I'm not sure if this question makes sense, please let me know if clarification is needed. Given the specified environment, I want to know if the most active organism will always be the most successful. The most active organism would be the one that obtains the most food/energy/reproduces the most. Or is it possible to not be the most active organism and still be successful?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="43231" LastEditDate="2020-12-25T11:09:49.997" LastActivityDate="2021-01-31T22:01:50.143" Title="Is it possible that the fittest individuals in an Artificial Life population may be successful by not actively pursuing the rules of the environment?" Tags="&lt;genetic-algorithms&gt;&lt;artificial-life&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25372" PostTypeId="2" ParentId="25364" CreationDate="2020-12-24T14:18:47.780" Score="2" Body="&lt;p&gt;In my opinion, it is not because ensemble methods are not good, just the state-of-the-art and Kaggle competitions are two different fields.&lt;/p&gt;&#xA;&lt;p&gt;Kaggle competitions can be understood as an industry project where the target (accuracy, distance value, etc) is the most important, and they can select some &lt;strong&gt;computationally expensive&lt;/strong&gt; way such as ensemble methods to reach it.&lt;/p&gt;&#xA;&lt;p&gt;The state-of-the-art models in other ways belong to the research area, where the most important is the &lt;strong&gt;contribution&lt;/strong&gt; for science, you can not just combine a lot of models then call it is the research (and so unfair with some small researcher groups). If you want to contribute something depend on &lt;strong&gt;ensemble idea&lt;/strong&gt;, it should be like &lt;a href=&quot;https://arxiv.org/abs/1503.02531&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="41287" LastEditorUserId="41287" LastEditDate="2020-12-25T09:52:17.237" LastActivityDate="2020-12-25T09:52:17.237" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25373" PostTypeId="1" CreationDate="2020-12-24T15:20:20.073" Score="0" ViewCount="73" Body="&lt;p&gt;I would like to build a real-time binary classifier that can predict an event of interest that is occurring as soon as it starts. These are electromyographic signals, and the event classification should be able to classify the event as early as possible. This is because the next stage of the algorithm has to make a decision before the end of the event.&lt;/p&gt;&#xA;&lt;p&gt;I don't know what kind of algorithm/approach I should use here. I suppose RNN with LSTM cells should do the job, but the dataset is quite small as physiological signals are not easy to gather.&lt;/p&gt;&#xA;&lt;p&gt;I have seen many algorithms that windowed the signals (from the training set) and labeled each window as an event of interest if at least part of the event is contained in the window. Each window is then fed to a machine learning algorithm. Then, the prediction uses a sliding window in real-time. But this approach doesn't take into account the temporal aspect of the event as there is no link between each window seen by the ML algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any tips or resources I could use to solve the problem?&lt;/p&gt;&#xA;" OwnerUserId="43239" LastEditorUserId="43239" LastEditDate="2020-12-26T18:13:15.653" LastActivityDate="2021-05-25T20:00:15.047" Title="How to do early classification of time series event with small dataset?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;ai-design&gt;&lt;prediction&gt;&lt;time-series&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25374" PostTypeId="2" ParentId="25368" CreationDate="2020-12-24T17:54:35.217" Score="2" Body="&lt;p&gt;There's not much to derive here it's simply a definition of Bellman operator, it comes from Bellman equation. If you're wondering why&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;Q^{\pi} = (I - \gamma P^{\pi})^{-1}r \tag{1}&#xA;\end{equation}&lt;/span&gt;&#xA;they state that &lt;span class=&quot;math-container&quot;&gt;$Q^{\pi}$&lt;/span&gt; is a fixed point which means if you apply Bellman operator to it you get the same value&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;T^{\pi}(Q^{\pi}) = Q^{\pi}&#xA;\end{equation}&lt;/span&gt;&#xA;You can easily check that since from &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;r = (I-\gamma P^{\pi})Q^{\pi}&#xA;\end{equation}&lt;/span&gt;&#xA;if you plug it in definition of Bellman operator you get&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;T^{\pi}(Q^{\pi}) &amp;amp;= r + \gamma P^{\pi} Q^{\pi}\\&#xA;&amp;amp;= (I - \gamma P^{\pi})Q^{\pi}+  \gamma P^{\pi} Q^{\pi}\\&#xA;&amp;amp;= Q^{\pi}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2020-12-24T18:07:51.773" LastActivityDate="2020-12-24T18:07:51.773" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25375" PostTypeId="1" CreationDate="2020-12-24T20:53:17.233" Score="1" ViewCount="14" Body="&lt;p&gt;I saw this post: &lt;a href=&quot;https://venturebeat.com/2018/12/04/google-ai-generates-images-of-3d-models-with-realistic-lighting-and-reflections/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google AI generates images of 3D models with realistic lighting and reflections&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZU78G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZU78G.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there any project to test this capability for combining two 3d cad files of a similar model and get a new combined model as shown (Hoogle AI ...)? I would like to use software such as Blender, SolidWorks or some project via Google Colab (testable without the need to install any software).&lt;/p&gt;&#xA;" OwnerUserId="32824" LastEditorUserId="2444" LastEditDate="2020-12-25T12:17:58.783" LastActivityDate="2020-12-25T12:17:58.783" Title="Searching for 3D cad AI synthesis project (New CAD file form two similar CAD model)" Tags="&lt;image-generation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25376" PostTypeId="1" CreationDate="2020-12-25T08:41:28.567" Score="1" ViewCount="15" Body="&lt;p&gt;I was reading &lt;a href=&quot;https://arxiv.org/ftp/arxiv/papers/1301/1301.7380.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper by Hansen.&lt;/p&gt;&#xA;&lt;p&gt;It says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A correspondence between vectors and one-step policy choices plays an important role in this interpretation of a policy. Each vector in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}'$&lt;/span&gt; corresponds to the choice of an action, and for each possible observation, choice of a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt;. Among all possible one-step policy choices, the vectors in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}'$&lt;/span&gt; correspond to those that optimize the value of some belief state. To describe this correspondence between vectors and one-step policy choices, we introduce the following notation. For each vector &lt;span class=&quot;math-container&quot;&gt;$\mathcal{v}_i$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}'$&lt;/span&gt;, let &lt;span class=&quot;math-container&quot;&gt;$a(i)$&lt;/span&gt; denote the choice of action and, for each possible observation &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, let &lt;span class=&quot;math-container&quot;&gt;$l(i,z)$&lt;/span&gt; denote the index of the successor vector in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{V}$&lt;/span&gt;. Given this correspondence between vectors and one-step policy choices, Kaelbling et al. (1996) point out that an optimal policy for a finite-horizon POMDP can be represented by an acyclic finite-state controller in which each machine state corresponds to a vector in a nonstationary value function.&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yAKWl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yAKWl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am unable to guess how the left-side finite-state controller is formed from the right side belief space diagram. Does the above text provide enough explanation for the conversion? If yes, I am not really able to fully get it. Can someone please explain?&lt;/p&gt;&#xA;" OwnerUserId="41169" LastEditorUserId="2444" LastEditDate="2020-12-25T12:14:59.307" LastActivityDate="2020-12-25T12:14:59.307" Title="How to obtain the policy in the form of a finite-state controller from the value function vectors over the belief space of the POMDP?" Tags="&lt;papers&gt;&lt;value-functions&gt;&lt;policies&gt;&lt;pomdp&gt;&lt;policy-iteration&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25379" PostTypeId="1" CreationDate="2020-12-25T12:05:37.687" Score="2" ViewCount="46" Body="&lt;p&gt;I am currently studying the paper &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.170&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Learning and Evaluating Classifiers under Sample Selection Bias&lt;/em&gt; by Bianca Zadrozny&lt;/a&gt;. In section &lt;strong&gt;3.4. Support vector machines&lt;/strong&gt;, the author says the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;3.4. Support vector machines&lt;/strong&gt;&lt;br /&gt;&#xA;In its basic form, the support vector machine (SVM) algorithm (Joachims, 2000a) learns the parameters &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; describing a linear decision rule&#xA;&lt;span class=&quot;math-container&quot;&gt;$$h(x) = \text{sign}(a \cdot x + b),$$&lt;/span&gt;&#xA;whose sign determines the label of an example, so that the smallest distance between each training example and the decision boundary, i.e. the margin, is maximized. Given a sample of examples &lt;span class=&quot;math-container&quot;&gt;$(x_i, y_i)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$y_i \in \{ -1, 1 \}$&lt;/span&gt;, it accomplishes margin maximization by solving the following optimization problem:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{minimize:} \ V(a, b) = \dfrac{1}{2} a \cdot a \\ \text{subject to:} \ \forall i : \ y_i[a \cdot x_i + b] \ge 1$$&lt;/span&gt;&#xA;The constraint requires that all examples in the training set are classified correctly. Thus, sample selection bias will not systematically affect the output of this optimization, assuming that the selection probability &lt;span class=&quot;math-container&quot;&gt;$P(s = 1 \mid x)$&lt;/span&gt; is greater than zero for all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How does the constraint that all examples in the training set are classified correctly imply that sample selection bias will not systematically affect the output of the optimisation? Furthermore, why is it necessary to assume that the selection probability is greater than zero for all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;? These are not clear to me.&lt;/p&gt;&#xA;" OwnerUserId="16521" LastActivityDate="2020-12-25T12:05:37.687" Title="How does the support vector machine constraint imply that sample selection bias will not systematically affect the output of the optimisation?" Tags="&lt;classification&gt;&lt;papers&gt;&lt;optimization&gt;&lt;support-vector-machine&gt;&lt;selection-bias&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25382" PostTypeId="2" ParentId="25358" CreationDate="2020-12-25T18:44:44.500" Score="2" Body="&lt;p&gt;Of course you can use AI (specially Deep Learning) in your application. your covariates will be the input to your AI model and the model should predict probability of presence. The model has no problem with binary data and binary data is common in this field.&lt;/p&gt;&#xA;&lt;p&gt;Also note that 1:100 ratio is not good and the network will probably learn to output &lt;em&gt;absence&lt;/em&gt; for any input (this way it gets 99% accuracy but really it's not doing anything). So you should probably balance them (using almost same data, or telling network to pay more attention to &lt;em&gt;presence&lt;/em&gt; data (by weighting the related loss)).&lt;/p&gt;&#xA;&lt;p&gt;I think nowdays you can find Deep Learning in any popular coding language. But most of DL community use python and its really easy to learn.&lt;/p&gt;&#xA;&lt;p&gt;If you want to learn Deep Learning there are a lot of sources on internet. But I suggest you the Deep Learning courses of &lt;em&gt;Deeplearning.ai&lt;/em&gt; in &lt;em&gt;coursera&lt;/em&gt; (If you have a lot of time) and &lt;em&gt;CS231n&lt;/em&gt; of Stanford university on youtube (If you have time)&lt;/p&gt;&#xA;" OwnerUserId="41547" LastEditorUserId="41547" LastEditDate="2020-12-25T18:55:01.383" LastActivityDate="2020-12-25T18:55:01.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25383" PostTypeId="2" ParentId="25373" CreationDate="2020-12-25T22:15:39.207" Score="0" Body="&lt;p&gt;One kind of system you could look into are &lt;a href=&quot;http://www.scholarpedia.org/article/Echo_state_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Echo State Networks (ESNs)&lt;/a&gt;.&#xA;They are relatively cheap to train and can learn to predict output signals to an  arbitrary degree of precision.&lt;/p&gt;&#xA;&lt;p&gt;All you need to train the system is some labeled training data.&#xA;Thus, if you have a sequence of measurements/feature values and the corresponding sequence of class labels, you can train and fine-tune these kinds of systems to output some required class label very quickly after the onset of an event encoded in the measured feature values.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2020-12-25T22:15:39.207" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25384" PostTypeId="2" ParentId="25257" CreationDate="2020-12-25T22:37:11.967" Score="0" Body="&lt;p&gt;I don't know of any neural network that can do cryptography well, so you would have to do a little experimenting yourself. The main thing that sticks out to me is that doing operations in the elliptical curve requires the modulus operator since it works in finite field, and I think neural networks have a hard time learning the modulus operator in general. So I would focus on that first. Some things to help the network learn the modulus operator:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would try to increase the hidden layers to a number 4x-10x bigger than the input dimension, which maps it to a higher dimension to hopefully learn more complex behavior.&lt;/li&gt;&#xA;&lt;li&gt;I would use less layers, maybe only 2-4 hidden layers, to speed up the development time.&lt;/li&gt;&#xA;&lt;li&gt;Most importantly, I would train with a LOT of data points (25% of the total possible finite field possibilities). I don't think the neural network can learn unless the number of data is this high.&lt;/li&gt;&#xA;&lt;li&gt;For reference, &lt;a href=&quot;http://stackoverflow.com/a/62095681/3667125&quot;&gt;someone&lt;/a&gt; got a network to learn the modulus operator using these points.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For rapid iterating, I would test with a much smaller finite field. E.g. use a 8 bit security and see if the neural network can do well with that first before moving on to the full 256-bit security key (or whatever your end goal is).&lt;/p&gt;&#xA;&lt;p&gt;Taking this a step further, I would first test to see if the neural network can even perform a point addition in the elliptical curve well, because if it doesn't then it definitely can't do point multiplication which is needed to compute the public key.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-25T22:37:11.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25385" PostTypeId="1" CreationDate="2020-12-26T03:41:22.313" Score="0" ViewCount="15" Body="&lt;p&gt;I need to detect text ellipses in scanned text documents. These ellipses are defined by lines next to text.&lt;/p&gt;&#xA;&lt;p&gt;Here is a sample:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0UHYA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0UHYA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(This text is a training for pupils. They have to fill in the appropriate word endings)&lt;/p&gt;&#xA;&lt;p&gt;I have already tried Hough Lines. I wasn't too happy with it because it required too many assumptions. If I set them &amp;quot;wrong&amp;quot;, it wouldn't work.&lt;/p&gt;&#xA;&lt;p&gt;I have also tried my own approach where I walk through each pixel and search for connected horizontal lines. This was also error prone because it also required some assumptions.&lt;/p&gt;&#xA;&lt;p&gt;I would now like to use machine learning, but I don't know which one I should use.&lt;/p&gt;&#xA;&lt;p&gt;I guess that I have to label the data. I think I should make a list for each text document and write down the start point and end point for each ellipsis. Would I then have to use Linear Regression, or which approach is most suitable for my case?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="43260" LastActivityDate="2020-12-26T03:41:22.313" Title="How do I find text ellipses using machine learning?" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25386" PostTypeId="2" ParentId="18497" CreationDate="2020-12-26T04:59:32.953" Score="1" Body="&lt;p&gt;Today one of the challenges is learning representations/concepts that are causally invariant. Once we have good representations then we can work on the reasoning aspect. There are 2 camps of people today. One believes that symbolic manipulation cannot be achieved properly by deep networks. Hence, they separate the task of extracting a lower-dimensional representation of objects from the visual scenes from the task of reasoning with knowledge-graphs. The other camp feels that we can do end-to-end training of a neural network and it can learn how to jointly learn a good lower-dimensional representation for each symbol along with learning how to reason with them. I am no expert at this but here are a few papers that I find are worthy for you to read -&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Neuro Symbolic Concent Learner &lt;a href=&quot;https://openreview.net/pdf?id=rJgMlhRctm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/vacancy/NSCL-PyTorch-Release&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Learning Reasoning Strategies in End-to-End Differentiable Proving &lt;a href=&quot;http://proceedings.mlr.press/v119/minervini20a/minervini20a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/uclnlp/ctp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Neuro-Symbolic Visual Reasoning: Disentangling “Visual” from “Reasoning” &lt;a href=&quot;https://arxiv.org/pdf/2006.11524v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Knowledge Infused Learning (K-IL):&#xA;Towards Deep Incorporation of Knowledge in Deep Learning &lt;a href=&quot;https://arxiv.org/pdf/1912.00512.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Visual Concept-Metaconcept Learning &lt;a href=&quot;https://arxiv.org/pdf/2002.01464.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt; &lt;a href=&quot;http://vcml.csail.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Project Page&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;CVPR 2020 workshop on Neuro-Symbolic Visual Reasoning and Program Synthesis &lt;a href=&quot;https://www.youtube.com/watch?v=Opunfo422uQ&amp;amp;list=PLX0h2D8LfCPHtT0-u6xjYaGFPT6-3cLJ1&amp;amp;index=1&amp;amp;ab_channel=NSCV%2720&quot; rel=&quot;nofollow noreferrer&quot;&gt;youtube videos&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If you are looking for community maintained lists, here is one  &lt;a href=&quot;https://github.com/ccclyu/awesome-deeplogic&quot; rel=&quot;nofollow noreferrer&quot;&gt;list of papers&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7330" LastEditorUserId="7330" LastEditDate="2020-12-26T05:57:31.500" LastActivityDate="2020-12-26T05:57:31.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25388" PostTypeId="1" CreationDate="2020-12-26T11:00:41.290" Score="1" ViewCount="25" Body="&lt;p&gt;I am training a neural network with a dataset that has 51 classes and 6766 data in it. I used 80% for the training set, 10% for validation, and 10% for the test. After training I got confusion matrix and I find out the last class is missed in the test set. So, I used data augmentation and used 27064 data and 80-10-10 splits again, but the last class name is missed again. I changed the size of the test split but the problem was not solved, and in every trial that I made, only the last class name is missed. How can I solve this?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; my dataset is images and in the original dataset, I have 104 data from the last class, after augmentation the entire dataset the last class has 416 data.&lt;/p&gt;&#xA;" OwnerUserId="33792" LastEditorUserId="33792" LastEditDate="2020-12-29T08:24:55.463" LastActivityDate="2020-12-29T08:24:55.463" Title="The last target name is missed in the test set" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;training&gt;" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="25390" PostTypeId="1" AcceptedAnswerId="25394" CreationDate="2020-12-26T15:42:06.640" Score="0" ViewCount="38" Body="&lt;p&gt;I am trying to make a neural network framework from scratch in C++ just for fun, and to test my backpropagation, I thought it would be an easy way to test the functionality if I give it one input - &lt;strong&gt;a randomized size 10 vector&lt;/strong&gt;, and one output: &lt;strong&gt;a size 5 vector containing all 1s&lt;/strong&gt;, and train it a bunch of times to see if the loss will decrease. Essentially trying to make it overfit&lt;/p&gt;&#xA;&lt;p&gt;The problem is that for each run that I do, the loss either shoots up and goes to nan, or reduces a lot, going to 0.000452084 or other similar small values. However even in the low end of things, my output (which should be close to all 1s, as the &amp;quot;ground truth&amp;quot;) is something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0.000263654&#xA;1e-07&#xA;8.55893e-05&#xA;1e-07&#xA;0.999651&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The only close value close to 1 being the last element.&lt;/p&gt;&#xA;&lt;p&gt;My network consists of the input layer 10 neurons, one 10 neuron dense layer with RELU activation, and another 5 neuron dense layer for output, with SoftMax activation. I am using categorical cross entropy as my loss function, and I am normalizing my gradient by dividing it by the norm of my gradient if it is over 1.0. I initialize my weights to be random values between -0.1 and 0.1&lt;/p&gt;&#xA;&lt;p&gt;To calculate the gradient of the loss function, I use &lt;code&gt;-groundTruth/predictedOutput&lt;/code&gt;. To calculate&#xA;the other derivatives, I dot the derivative of that layer with the gradient of the previous layer with respects to its activation function.&lt;/p&gt;&#xA;&lt;p&gt;Before this problem I was having exploding gradients, which the gradient scaling fixed, however it was very weird that that would even happen on a very small network like this, which could be related to the problem I am currently having. Is the implementation not correct or am I missing something very obvious?&lt;/p&gt;&#xA;&lt;p&gt;Any ideas about this weird behavior, and where I should look first? I am not sure how to show a minimal reproduceable example as that would require me to paste the whole codebase, but I am happy to show pieces of code with explanation. Any advice welcomed!!&lt;/p&gt;&#xA;" OwnerUserId="43270" LastActivityDate="2020-12-26T20:54:23.760" Title="Loss randomly changing, incorrect output (even for low loss) when trying to overfit on a single set of input and output" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25392" PostTypeId="1" CreationDate="2020-12-26T19:33:31.997" Score="1" ViewCount="94" Body="&lt;p&gt;I am attempting a project involving training an agent to play a game using deep reinforcement learning.&#xA;This project has a few features that complicate the design of the neural network:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The action space per state is very large (can be over 1000 per state)&lt;/li&gt;&#xA;&lt;li&gt;The set of actions available in each state very wildly between states, both in size and the actions available.&lt;/li&gt;&#xA;&lt;li&gt;The total action-space (the union of each state's action-space) is way too large to enumerate.&lt;/li&gt;&#xA;&lt;li&gt;The action space is discrete, not continuous.&lt;/li&gt;&#xA;&lt;li&gt;The game is adversarial, multi-agent.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Most RL neural networks I've seen involve the input of a state, and an output of a constance action size, where each element of the output is either an action's q-score or probability. But since my game's action space non-constant per state, I believe this design will not work for this game.&lt;/p&gt;&#xA;&lt;p&gt;I have seen an alpha-go style network, which outputs a probability for all actios ever possible, and then zeros out actions not possible in the given state and re-normalized the probabilities. However, since the total action-space in my game is way to large to enumerate, I don't believe this solution will work either.&lt;/p&gt;&#xA;&lt;p&gt;I have seen several network designs for large, discrete action spaces, including:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;design a network to input a state-action pair and output a single score value, and train it via a value-based algorithm (such as q-learning). To select an action given a state, pass in every possible state-action pair to get each action's score, then select the action with the highest score.&lt;/li&gt;&#xA;&lt;li&gt;(Wolpertinger architecture) have a network output a continous embedding, which is then mapped to a discrete action, and train it via deterministic policy gradient.&lt;/li&gt;&#xA;&lt;li&gt;divide actions into a sequence of simpler actions, and train an RNN to output a sequence of these simpler actions. Train this network via a value-based algorithm (such as q-learning).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, all of these solutions are designed for either value-based or deterministic policy gradient algorithms; none of them output probabilities over the action space. This seems to be an issue since at least a very large portion of the multi-agent deep-RL algorithms I've seen involve a network that outputs a probability over the action-space. Therfore, I don't want to limit myself to value-based and deterministic-policy algorithms.&lt;/p&gt;&#xA;&lt;p&gt;How can I design a neural network that outputs a probability over the action space for my game?&#xA;If not, what would be some good solutions to this problem?&lt;/p&gt;&#xA;" OwnerUserId="38448" LastActivityDate="2020-12-26T19:33:31.997" Title="Designing Policy-Network for Deep-RL with Large, Variable Action Space" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25394" PostTypeId="2" ParentId="25390" CreationDate="2020-12-26T20:54:23.760" Score="0" Body="&lt;p&gt;Softmax activation always adds up to 1, because it's designed to deal with probabilities (in problems of classification, those probabilities represent how likely the network thinks an object belongs to a specific class). You can verify that by summing up the numbers of your output layer. So currently your network is trying to do the impossible, to produce output that sums up to 5, instead of 1. Therefore, loss will never become stable. Since you want your output layer to produce all ones, you need to use some other activation, for example, linear. Linear activation does not have the same constraint that Softmax does.&lt;/p&gt;&#xA;" OwnerUserId="38076" LastActivityDate="2020-12-26T20:54:23.760" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25395" PostTypeId="1" CreationDate="2020-12-26T21:15:20.447" Score="2" ViewCount="24" Body="&lt;p&gt;After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How will the filter (or kernel) size affect the transpose convolution operation (e.g. when using &lt;a href=&quot;https://pytorch.org/docs/stable/generated/torch.nn.ConvTranspose2d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ConvTranspose2d&lt;/code&gt;&lt;/a&gt;)? Will a larger kernel help upsample with better detail or a small-sized kernel? And how would padding fit in this scenario?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At what rate should the depth decrease while upsampling i.e from (Dx24x24) to (D/2 or D/4, 48, 48)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="43254" LastEditorUserId="2444" LastEditDate="2021-01-05T22:48:05.407" LastActivityDate="2021-01-05T22:48:05.407" Title="How will the filter size affect the transpose convolution operation?" Tags="&lt;convolutional-neural-networks&gt;&lt;pytorch&gt;&lt;hyper-parameters&gt;&lt;filters&gt;&lt;transpose-convolution&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25396" PostTypeId="2" ParentId="25369" CreationDate="2020-12-26T21:41:17.657" Score="2" Body="&lt;p&gt;Yes, OpenAI will release an API for GPT-3, so any developer can integrate it into their application. I don't believe the document for their API is public yet, so we don't know what the final interface will look like, but it's likely to be a simple REST API. In the future, I imagine your developers can take advantage of their API, or alternatively there will be community-made scripts for you to use/copy.&lt;/p&gt;&#xA;&lt;p&gt;The pricing for using their API is explained &lt;a href=&quot;https://thenextweb.com/neural/2020/09/03/openai-reveals-the-pricing-plans-for-its-api-and-it-aint-cheap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Note that they charge per token, which might be important in case your game plans to make live calls to GPT-3 during gameplay (as opposed to mining a huge corpus of answers to build an offline database).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://beta.openai.com/?app=customer-service&quot; rel=&quot;nofollow noreferrer&quot;&gt;use cases&lt;/a&gt; of GPT-3 suggests that you can legally use them for commercial products, although I couldn't find a definitive license or user agreement document.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-26T21:41:17.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25399" PostTypeId="2" ParentId="22667" CreationDate="2020-12-27T11:17:16.150" Score="2" Body="&lt;p&gt;I don't have a definite answer, but only a suspicion/idea:&lt;/p&gt;&#xA;&lt;p&gt;Looking at Figure 1 from the &lt;a href=&quot;https://arxiv.org/pdf/1701.07875.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGAN paper&lt;/a&gt;, we clear see that the JS divergence on the right is not continuous at &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, hence not differentiable at &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. However, the EM plot on the left is continuous also at &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. You could now argue that we have a kink there, so it should not be differentiable there either, but they might have a different notion of differentiability, I am honestly not sure about it right now.  &lt;a href=&quot;https://i.stack.imgur.com/0gXDg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0gXDg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43286" LastActivityDate="2020-12-27T11:17:16.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25400" PostTypeId="1" AcceptedAnswerId="25404" CreationDate="2020-12-27T11:39:32.590" Score="1" ViewCount="30" Body="&lt;p&gt;I have a card game where on a player's turn, the player sequentially draws two cards. Each card may be drawn from another player's discard stack (face up), or from the deck (face down).&lt;/p&gt;&#xA;&lt;p&gt;Thinking how to encode this into an action space, I could naively assume the two draws are independent. The action space would simply be a binary vector of 2 * (1 + (number_of_players - 1)), which I could post-filter to limit for empty draw piles (and can't draw from own pile).&lt;/p&gt;&#xA;&lt;p&gt;However, when playing the game myself, I noticed that it's sometimes advantageous to draw the initial card from the deck, then select the draw pile for the second card based on the value of the first one drawn. But how would this be encoded into an action space? Would it be better to think of these are two separate actions, even thought they are part of the same &amp;quot;turn&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="5154" LastActivityDate="2020-12-27T17:11:53.560" Title="RL: Encoding action conditioned on previous action" Tags="&lt;reinforcement-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25402" PostTypeId="5" CreationDate="2020-12-27T14:24:35.923" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-27T14:24:35.923" LastActivityDate="2020-12-27T14:24:35.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25403" PostTypeId="4" CreationDate="2020-12-27T14:24:35.923" Score="0" Body="For questions related to grammatical evolution, which is a form of (or approach to) genetic programming." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-27T14:24:35.923" LastActivityDate="2020-12-27T14:24:35.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25404" PostTypeId="2" ParentId="25400" CreationDate="2020-12-27T15:31:05.373" Score="2" Body="&lt;p&gt;It is hard to say for certain without knowing full details and results of experiments.&lt;/p&gt;&#xA;&lt;p&gt;However, if the game allows for splitting decisions up, it will likely be better for the agent to take advantage of extra knowledge of the value of any previously hidden card just taken from the draw pile.&lt;/p&gt;&#xA;&lt;p&gt;In general, if each player decision is taken sequentially, resulting in changes to state, then it is a separate action on a separate time step according to the MDP theoretical model used in reinforcement learning (RL). You might want to describe/notate the time steps differently so that they match how the game play proceeds. However, for the purposes of RL, each decision point should be on a new time step, and should result in a new state, new value estimates etc.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, whether or not the current choice is the player's first card or second card to be drawn needs to be part of the state. This detail of the state &lt;em&gt;might&lt;/em&gt; already be covered by the number of cards in the player's hand, if logically the number of cards is always the same at each stage. However, if hand size can vary for other reasons, it is worth adding an explicit flag for &amp;quot;first draw choice&amp;quot; or similar so that the agent can use the information.&lt;/p&gt;&#xA;&lt;p&gt;You have some freedom for encoding the action space. If drawing cards is the only possible action in this game at all stages, then a binary output vector of 1 + (number_of_players - 1) dimensions would be suitable. Other encodings may work well too, it depends if there is any logical structure to the choices or some derived data that encodes useful game information.&lt;/p&gt;&#xA;&lt;p&gt;It may be useful to arrange the action choices so that the index for drawing from each player's discard pile is considered &lt;em&gt;relatively&lt;/em&gt; to the current player's turn. That is, instead of actions being arranged &lt;span class=&quot;math-container&quot;&gt;$[draw, discard P1, discard P3, discard P4, discard P5]$&lt;/span&gt; for P2, they would be arranged &lt;span class=&quot;math-container&quot;&gt;$[draw, discard P3, discard P4, discard P5, discard P1]$&lt;/span&gt; and for P3 would be different: &lt;span class=&quot;math-container&quot;&gt;$[draw, discard P4, discard P5, discard P1, discard P2]$&lt;/span&gt; . . . that would inherently allow for the cyclical nature of turns. State representation would need to similarly rotate knowledge about each player to match this. You might not need to do this, but I would recommend it for games where there is a lot of common logic regarding action choices relative to turn position that you could take advantage of. The opposite would apply (and you would use absolute player positions) if there were important differences throughout the game between being P1, P2, P3 etc.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2020-12-27T17:11:53.560" LastActivityDate="2020-12-27T17:11:53.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25405" PostTypeId="2" ParentId="25335" CreationDate="2020-12-27T16:56:26.350" Score="1" Body="&lt;p&gt;Look at spatio-temporal CNNs which extend the image-based CNN in 2D to 3D to handle time.  These are commonly used to detect or classify action in a video.  People have used them to identify specific actions in various sports such as kicking a soccer ball, throwing a baseball or dribbling a basketball.  They have been used to identify fire, smoke, deep fakes, and violence in videos.  Below are some articles to help get started.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/introduction-to-video-classification-6c6acbc57356&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Video Classification, by Connor Shorten&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/are-you-ready-for-a-video-classification-challenge-d044e3b202b6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Are you ready for a Video Classification Challenge? Preparation Guide for Video Classification, by Neha Goel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://towardsdatascience.com/mict-net-for-human-action-recognition-in-videos-3a18e4f97342&quot; rel=&quot;nofollow noreferrer&quot;&gt;MiCT-Net for Human Action Recognition in Videos, by Florent Mahoudeau&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Source code:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/HHTseng/video-classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video Classification&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/kenshohara/video-classification-3d-cnn-pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;Video Classification Using 3D ResNet&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2020-12-27T17:04:14.813" LastActivityDate="2020-12-27T17:04:14.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25406" PostTypeId="1" CreationDate="2020-12-27T17:46:09.347" Score="3" ViewCount="32" Body="&lt;p&gt;I implemented the unet in tensorflow for the segmentation of MRI images of the thigh. I noticed I always get a higher validation accuracy  by a small gap, independently of the initial split. One example&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dskC9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dskC9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I researched when this could be possible:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When we have an &amp;quot;easy&amp;quot; validation set. I trained it for different initial splitting, all of them showed a higher validation accuracy.&lt;/li&gt;&#xA;&lt;li&gt;Regularization and augmentation may reduce the training accuracy. I removed the augmentation and  dropout regularization and still observed the same gap, the only difference was that it took more epochs to reach convergence.&lt;/li&gt;&#xA;&lt;li&gt;The last thing I found was that in keras the training accuracy and loss are averaged over each iteration of the corresponding epoch, while the validation accuracy and loss is calculated from the model at the end of the epoch, which might make the the training loss higher and accuracy lower.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So I thought that if I train and validate on the same set, then I should get the same curve but shifted by one epoch. So I trained only on 2 batches and validated on the same 2 batches (withou dropout or augmentation). I still think there is something else happening because they dont look quite the same and at least at the end when the weights are not changing anymore, the training and validation accuracy should be the same (but still the validation accuracy is higher by a small gap)&#xA;this is the plot&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9sJlK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9sJlK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xBUt9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xBUt9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there anything else that can be increasing the loss values, this is the model I am using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def unet_no_dropout(pretrained_weights=None, input_size=(512, 512, 1)):&#xA;inputs = tf.keras.layers.Input(input_size)&#xA;conv1 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same', kernel_initializer='he_normal')(inputs)&#xA;conv1 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv1)&#xA;pool1 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv1)&#xA;conv2 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool1)&#xA;conv2 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv2)&#xA;pool2 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv2)&#xA;conv3 = tf.keras.layers.Conv2D(256, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool2)&#xA;conv3 = tf.keras.layers.Conv2D(256, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv3)&#xA;pool3 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv3)&#xA;conv4 = tf.keras.layers.Conv2D(512, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool3)&#xA;conv4 = tf.keras.layers.Conv2D(512, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv4)&#xA;#drop4 = tf.keras.layers.Dropout(0.5)(conv4)&#xA;pool4 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv4)&#xA;&#xA;conv5 = tf.keras.layers.Conv2D(1024, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool4)&#xA;conv5 = tf.keras.layers.Conv2D(1024, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv5)&#xA;#drop5 = tf.keras.layers.Dropout(0.5)(conv5)&#xA;&#xA;up6 = tf.keras.layers.Conv2D(512, 2, activation='relu', padding='same', kernel_initializer='he_normal')(&#xA;    tf.keras.layers.UpSampling2D(size=(2, 2))(conv5))&#xA;merge6 = tf.keras.layers.concatenate([conv4, up6], axis=3)&#xA;#merge6 = tf.keras.layers.concatenate([conv4, up6], axis=3)&#xA;conv6 = tf.keras.layers.Conv2D(512, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge6)&#xA;conv6 = tf.keras.layers.Conv2D(512, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv6)&#xA;&#xA;up7 = tf.keras.layers.Conv2D(256, 2, activation='relu', padding='same', kernel_initializer='he_normal')(&#xA;    tf.keras.layers.UpSampling2D(size=(2, 2))(conv6))&#xA;merge7 = tf.keras.layers.concatenate([conv3, up7], axis=3)&#xA;conv7 = tf.keras.layers.Conv2D(256, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge7)&#xA;conv7 = tf.keras.layers.Conv2D(256, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv7)&#xA;&#xA;up8 = tf.keras.layers.Conv2D(128, 2, activation='relu', padding='same', kernel_initializer='he_normal')(&#xA;    tf.keras.layers.UpSampling2D(size=(2, 2))(conv7))&#xA;merge8 = tf.keras.layers.concatenate([conv2, up8], axis=3)&#xA;conv8 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge8)&#xA;conv8 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv8)&#xA;&#xA;up9 = tf.keras.layers.Conv2D(64, 2, activation='relu', padding='same', kernel_initializer='he_normal')(&#xA;    tf.keras.layers.UpSampling2D(size=(2, 2))(conv8))&#xA;merge9 = tf.keras.layers.concatenate([conv1, up9], axis=3)&#xA;conv9 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge9)&#xA;conv9 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv9)&#xA;conv9 = tf.keras.layers.Conv2D(2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv9)&#xA;conv10 = tf.keras.layers.Conv2D(1, 1, activation='sigmoid')(conv9)&#xA;&#xA;model = tf.keras.Model(inputs=inputs, outputs=conv10)&#xA;&#xA;model.compile(optimizer = Adam(lr = 2e-4), loss = 'binary_crossentropy', metrics = [tf.keras.metrics.Accuracy()])&#xA;#model.compile(optimizer=tf.keras.optimizers.Adam(2e-4), loss=combo_loss(alpha=0.2, beta=0.4), metrics=[dice_accuracy])&#xA;#model.compile(optimizer=RMSprop(lr=0.00001), loss=combo_loss, metrics=[dice_accuracy])&#xA;&#xA;if (pretrained_weights):&#xA;    model.load_weights(pretrained_weights)&#xA;&#xA;return model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and this is how I save the model&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model_checkpoint = tf.keras.callbacks.ModelCheckpoint('unet_ThighOuterSurfaceval.hdf5',monitor='val_loss', verbose=1, save_best_only=True)&#xA;model_checkpoint2 = tf.keras.callbacks.ModelCheckpoint('unet_ThighOuterSurface.hdf5', monitor='loss', verbose=1, save_best_only=True)&#xA;&#xA;model = unet_no_dropout()&#xA;history = model.fit(genaug, validation_data=genval, validation_steps=len(genval), steps_per_epoch=len(genaug), epochs=80, callbacks=[model_checkpoint, model_checkpoint2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="42357" LastActivityDate="2020-12-27T17:46:09.347" Title="Validation accuracy higher than training accurarcy" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;image-segmentation&gt;&lt;u-net&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25408" PostTypeId="1" CreationDate="2020-12-27T18:02:12.733" Score="0" ViewCount="42" Body="&lt;p&gt;In the paper &lt;a href=&quot;https://www.researchgate.net/publication/266739905_Using_Cyclic_Scheduling_to_Generate_Believable_Behavior_in_Games&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using Cyclic Scheduling to Generate Believable Behavior in Games&lt;/a&gt;, Richard Zhao and Duane Szafron describe a tool that is able to generate schedules for NPCs.&lt;/p&gt;&#xA;&lt;h2&gt;The tool&lt;/h2&gt;&#xA;&lt;p&gt;The tool offers a visual interface were the designer can set several constraints.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OiNr8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;The designer can set an objective for an specific hour in the timeline;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OzEKv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;The designer can set the number of hours required by an specific objective. Also it's possible to add a transitional hour;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EAeF7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;The designer can separate the objective hours into groups;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hvlgR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;The designer can set several roles for each objective;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HBMvm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;The designer can set the probability for each role.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A5Reb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Based on the constraints set, the tool generates the schedule&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;The algorithm&lt;/h2&gt;&#xA;&lt;p&gt;To generate the schedule, it is used a greedy depth-first search algorithm, describe in pseudocode as the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;schedule = []&#xA;itemList = sort(itemList) // longest objective block first&#xA;CyclicSchedule(itemList)&#xA;&#xA;bool CyclicSchedule(itemList)&#xA;    if itemList is empty, return true&#xA;    item = pop(itemList)&#xA;    for each location L in schedule&#xA;        if satisfiable(L, item)&#xA;            schedule[L] = item&#xA;            returnValue = CyclicSchedule(itemList)&#xA;            if (returnValue == true)&#xA;                return true&#xA;            else&#xA;                schedule[L] = &amp;quot;&amp;quot;&#xA;    return false&#xA;&#xA;bool satisfiable(location L, obj block Item)&#xA;    if (length of obj block &amp;gt; available obj blocks at L)&#xA;        return false&#xA;    else if (non-consecutive obj block requirement is violated)&#xA;        return false&#xA;    else if (currently assigned obj hours &amp;gt; specified max obj hours)&#xA;        return false&#xA;    else&#xA;        return true&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;My question&lt;/h2&gt;&#xA;&lt;p&gt;For studying purpouse, I'm trying to reproduce this algorithm in JavaScript, but I'm completely stucked... here's what I managed until now:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;var schedule = [];&#xA;var itemList = [];&#xA;&#xA;cyclicSchedule(itemList);&#xA;&#xA;function cyclicSchedule(itemList) {&#xA;    if (itemList.length == 0) {&#xA;        return true;&#xA;    }        &#xA;&#xA;    var item = itemList.pop();&#xA;&#xA;    for (const l of schedule) {&#xA;        if (satisfiable(l, item)) {&#xA;            schedule[l] = item;&#xA;            returnValue = cyclicSchedule(itemList);&#xA;            if (returnValue) {&#xA;                return true;&#xA;            }   &#xA;            else {&#xA;                schedule[l] = '';&#xA;            }&#xA;        }&#xA;    }&#xA;&#xA;    return false&#xA;}&#xA;&#xA;function satisfiable(l, item) {&#xA;    return true;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know how can implement it using JavaScript?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="43300" LastEditorUserId="43300" LastEditDate="2020-12-28T13:13:07.307" LastActivityDate="2020-12-28T13:13:07.307" Title="How can I translate this deep-first search pseudocode into JavaScript?" Tags="&lt;game-ai&gt;&lt;constraint-satisfaction-problems&gt;&lt;javascript&gt;&lt;depth-first-search&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25410" PostTypeId="1" AcceptedAnswerId="25412" CreationDate="2020-12-27T20:13:05.920" Score="4" ViewCount="66" Body="&lt;p&gt;In many articles (for example, in &lt;a href=&quot;https://arxiv.org/pdf/1506.02640.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the YOLO paper&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1907.03465.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; or &lt;a href=&quot;https://www.aclweb.org/anthology/K18-1005/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;), I see the term &amp;quot;unified&amp;quot; being used. I was wondering what the meaning of &amp;quot;unified&amp;quot; in this case is.&lt;/p&gt;&#xA;" OwnerUserId="43305" LastEditorUserId="2444" LastEditDate="2020-12-27T22:47:53.840" LastActivityDate="2021-01-08T17:11:07.807" Title="What is a unified neural network model?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;terminology&gt;&lt;architecture&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25411" PostTypeId="1" CreationDate="2020-12-27T21:34:04.087" Score="5" ViewCount="86" Body="&lt;p&gt;The &lt;a href=&quot;https://arxiv.org/pdf/1701.07875.pdf&quot; rel=&quot;noreferrer&quot;&gt;WGAN paper&lt;/a&gt; concretely proposes Algorithm 1 (cf. page 8). Now, they also state what their loss for the critic and the generator is.&lt;/p&gt;&#xA;&lt;p&gt;When implementing the critic loss (so lines 5 and 6 of Algorithm 1), they maximize the parameters &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; (instead of minimizing them as one would normally do) by writing &lt;span class=&quot;math-container&quot;&gt;$w \leftarrow w + \alpha \cdot \text{RMSProp}\left(w, g_w \right)$&lt;/span&gt;. Their loss seems to be &lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{m}\sum_{i = 1}^{m}f_{w}\left(x^{\left(i\right)} \right) - \frac{1}{m}\sum_{i = 1}^{m} f_{w}\left( g_{\theta}\left( z^{\left( i\right)}\right)\right).\quad \quad (1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the critic, i.e. a neural network, and the way this loss is implemented in PyTorch in &lt;a href=&quot;https://www.youtube.com/watch?v=pG0QZ7OddX4&quot; rel=&quot;noreferrer&quot;&gt;this youtbe video&lt;/a&gt; (cf. minutes 11:00 to 12:26) is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;critic_real = critic(real_images)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;critic_fake = critic(generator(noise))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;loss_critic = -(torch.mean(critic_real) - torch.mean(critic_fake))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; In my own experiments with the &lt;a href=&quot;http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html&quot; rel=&quot;noreferrer&quot;&gt;CelebA dataset&lt;/a&gt;, I found that the critic loss is negative, and that the quality of the images is better if the negative critic loss is &lt;em&gt;higher&lt;/em&gt; instead of lower, so &lt;span class=&quot;math-container&quot;&gt;$-0.75$&lt;/span&gt; for the critic loss resulted in better generated iamges than a critic loss of &lt;span class=&quot;math-container&quot;&gt;$-1.26$&lt;/span&gt; e.g.&lt;/p&gt;&#xA;&lt;p&gt;Is there an error in the implementation in the youtube video of Eq. (1) and Algorithm 1 of the WGAN paper maybe? In my opinion, the implementation in the video is correct, but I am still confused then on why I get better images when the loss is higher ...&lt;/p&gt;&#xA;&lt;p&gt;Cheers!&lt;/p&gt;&#xA;" OwnerUserId="43286" LastEditorUserId="2444" LastEditDate="2021-01-25T19:05:11.747" LastActivityDate="2021-01-25T19:05:11.747" Title="Wasserstein GAN: Implemention of Critic Loss Correct?" Tags="&lt;papers&gt;&lt;generative-adversarial-networks&gt;&lt;implementation&gt;&lt;wasserstein-gan&gt;" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25412" PostTypeId="2" ParentId="25410" CreationDate="2020-12-27T21:38:03.263" Score="2" Body="&lt;p&gt;A unified neural network model consists of one neural network as opposed to other models that rely on two or more neural networks.&lt;/p&gt;&#xA;&lt;p&gt;For example, from page two of &lt;a href=&quot;https://arxiv.org/pdf/1506.02640.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the YOLO paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;2. Unified Detection&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We unify the separate components of object detection into a single neural network. Our network uses features from the entire image to predict each bounding box. It also predicts all bounding boxes across all classes for an image simultaneously. This means our network reasons globally about the full image and all the objects in the image. The YOLO design enables end-to-end training and realtime speeds while maintaining high average precision.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://arxiv.org/pdf/1907.03465.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper by Xu and Wang&lt;/a&gt;, they add a branch to handle tracking to the Faster R-CNN architecture, which was designed for object detection.  This is a 'unification' of two models.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://www.aclweb.org/anthology/K18-1005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper by Ebrahimi et al.&lt;/a&gt;, on pages 42-43, you can see they are fusing multiple neural networks into one unified model:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We propose a unified user geolocation method that relies on a fusion of neural networks, incorporating different types of available information: tweet message, users' social relationships, and metadata fields embedded in tweets and profiles.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="2444" LastEditDate="2020-12-27T22:49:19.227" LastActivityDate="2020-12-27T22:49:19.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25414" PostTypeId="1" AcceptedAnswerId="25415" CreationDate="2020-12-28T01:08:51.600" Score="0" ViewCount="61" Body="&lt;p&gt;The term &lt;em&gt;codon&lt;/em&gt; is used in the context of grammatical evolution (GE), sometimes, without being explicitly defined. For example, it is used in &lt;a href=&quot;https://arxiv.org/pdf/1703.08535.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which introduces and describes &lt;a href=&quot;https://github.com/PonyGE/PonyGE2&quot; rel=&quot;nofollow noreferrer&quot;&gt;PonyGE 2&lt;/a&gt;, a Python library for GE, but it's not clearly defined.  So, what is a codon?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2020-12-29T00:03:06.360" Title="What is a &quot;codon&quot; in grammatical evolution?" Tags="&lt;terminology&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;&lt;grammatical-evolution&gt;&lt;codon&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25415" PostTypeId="2" ParentId="25414" CreationDate="2020-12-28T01:08:51.600" Score="1" Body="&lt;h3&gt;Grammatical evolution&lt;/h3&gt;&#xA;&lt;p&gt;To understand what a codon is, we need to understand what GE is, so let me first provide a brief description of this approach.&lt;/p&gt;&#xA;&lt;p&gt;Grammatical evolution (GE) is an approach to genetic programming where the &lt;strong&gt;genotypes&lt;/strong&gt; are binary (or integer) arrays, which are mapped to the &lt;strong&gt;phenotypes&lt;/strong&gt; (i.e. the actual solutions, which can be represented as trees, which, in turn, represent programs or functions), using a grammar (for example, expressed in &lt;a href=&quot;https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form&quot; rel=&quot;nofollow noreferrer&quot;&gt;Backus-Naur form&lt;/a&gt;). So, the genotypes (i.e. what is mutated, combined, or searched) and the phenotypes (the actual solutions, which are programs) are different in GE, and the genotype needs to be mapped to the phenotype to get the actual solution (or program), but this is not the case in all GP approaches (for example, in tree-based GP, the genotype and the phenotype are the same, i.e. trees, which represent functions).&lt;/p&gt;&#xA;&lt;h3&gt;Codons&lt;/h3&gt;&#xA;&lt;p&gt;In GE, a &lt;strong&gt;codon&lt;/strong&gt; is a subsequence of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; bits of the genotype (assuming that genotypes are binary arrays). For example, let's say that we have only two symbols in our grammar, i.e. &lt;code&gt;a&lt;/code&gt; (a number) and &lt;code&gt;b&lt;/code&gt; (another number). In this case, we only need 2 bits to differentiate the two. If we had 3 symbols,  &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt; and &lt;code&gt;+&lt;/code&gt; (the addition operator), we would need at least a sequence of 2 bits to define each symbol. So, in this case, we could have the following mapping&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;a&lt;/code&gt; is represented by &lt;code&gt;00&lt;/code&gt; (or the integer &lt;code&gt;0&lt;/code&gt;),&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;b&lt;/code&gt; is represented by &lt;code&gt;01&lt;/code&gt; (or &lt;code&gt;1&lt;/code&gt;), and&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;+&lt;/code&gt; is represented by &lt;code&gt;10&lt;/code&gt; (or &lt;code&gt;2&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The operation &lt;code&gt;a+b&lt;/code&gt; could then be represented by the binary sequence &lt;code&gt;001001&lt;/code&gt; (or the integer sequence &lt;code&gt;021&lt;/code&gt;). The 2-bit subsequences &lt;code&gt;00&lt;/code&gt;, &lt;code&gt;01&lt;/code&gt; and &lt;code&gt;10&lt;/code&gt; (or their integer counterparts) are the &lt;strong&gt;codons&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;What do we need codons for?&lt;/h3&gt;&#xA;&lt;p&gt;In GE, codons are used to index the specific choice of a production rule. To understand this, let's define a simple grammar, which is composed of&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a set of non-terminals (e.g. functions) &lt;span class=&quot;math-container&quot;&gt;$N = \{ \langle \text{expr} \rangle, \langle \text{op} \rangle, \langle \text{operand} \rangle, \langle \text{var} \rangle \}$&lt;/span&gt;&#xA;,&lt;/li&gt;&#xA;&lt;li&gt;a set of terminals (e.g. specific numbers or letters) &lt;span class=&quot;math-container&quot;&gt;$\mathrm{T}=\{1,2,3,4,+,-, /, *, \mathrm{x}, \mathrm{y}\}$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;li&gt;a set of production rules &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;an initial production rule &lt;span class=&quot;math-container&quot;&gt;$S = \langle \text{expr} \rangle $&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In this case, the set of production rules &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\langle \text{expr} \rangle &amp;amp; ::= &#xA;\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle \; | \;  \langle \text{operand} \rangle \\&#xA;\langle \text{op} \rangle &amp;amp; ::= + \;  | \;  - \;  | \; * \; | \; / \\&#xA;\langle \text{operand} \rangle &amp;amp; ::= 1 \;  | \; 2 \; | \;  3 \; | \;  4 \;  | \;  \langle \text{var} \rangle \\&#xA;\langle \text{var} \rangle &amp;amp; ::= \mathrm{x} \; | \;  \mathrm{y}&#xA;\end{align}&lt;/span&gt;&#xA;So, there are four productions rules. To be clear, &lt;span class=&quot;math-container&quot;&gt;$\langle \text{var} \rangle  ::= \mathrm{x} \; | \;  \mathrm{y}$&lt;/span&gt; is a production rule. The symbol &lt;span class=&quot;math-container&quot;&gt;$|$&lt;/span&gt; means &amp;quot;or&amp;quot;, so the left-hand side of each production is a non-terminal (and note that all non-terminals are denoted with angle brackets &lt;span class=&quot;math-container&quot;&gt;$\langle \cdot \rangle$&lt;/span&gt;), which is defined as (or can be replaced with) one of the right-hand choices, which can be a non-terminal or terminal. The first choice of each production rule is at index &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. The second choice at index &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, and so on. So, for example, in the case of the production &lt;span class=&quot;math-container&quot;&gt;$\langle \text{var} \rangle  ::= \mathrm{x} \; | \;  \mathrm{y}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathrm{x}$&lt;/span&gt; is the choice at index &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathrm{y}$&lt;/span&gt; is the choice at index &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The codons are the indices that we use to select the production rule's choice while transforming (or mapping) the genotype into a phenotype (an actual program).&lt;/p&gt;&#xA;&lt;p&gt;So, we start with the first production, in this case, &lt;span class=&quot;math-container&quot;&gt;$S = \langle \text{expr} \rangle$&lt;/span&gt;. If it's a non-terminal, then we replace it with one of its right-hand side choices. In this case, there are two choices&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$&lt;/span&gt; (choice at index &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle \text{operand} \rangle$&lt;/span&gt; (choice at index &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If our genotype (integer representation) is, for example, &lt;span class=&quot;math-container&quot;&gt;$01$&lt;/span&gt; (note that this is a sequence of integers), we would replace &lt;span class=&quot;math-container&quot;&gt;$\langle \text{expr} \rangle$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$&lt;/span&gt;, then we would replace the first &lt;span class=&quot;math-container&quot;&gt;$\langle \text{expr} \rangle$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\langle \text{operand} \rangle$&lt;/span&gt;, so we would get &lt;span class=&quot;math-container&quot;&gt;$\langle \text{operand} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$&lt;/span&gt;, and so on and so forth, until we get an expression that only contains terminals, or the genotype is terminated. There can be other ways of mapping the genotype to the phenotype, but this is the purpose of codons.&lt;/p&gt;&#xA;&lt;h3&gt;Codons in biology&lt;/h3&gt;&#xA;&lt;p&gt;The term &lt;em&gt;codon&lt;/em&gt; has its origins in &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_code&quot; rel=&quot;nofollow noreferrer&quot;&gt;biology&lt;/a&gt;: a subsequence of 3 &lt;a href=&quot;https://en.wikipedia.org/wiki/Nucleotide&quot; rel=&quot;nofollow noreferrer&quot;&gt;nucleotides&lt;/a&gt; is known as a &lt;em&gt;codon&lt;/em&gt;, which is mapped to an amino acid in order to produce the proteins. The set of all mappings from codons to amino acids is known as &lt;em&gt;genetic code&lt;/em&gt;. Take a look at &lt;a href=&quot;https://www.khanacademy.org/science/ap-biology/gene-expression-and-regulation/translation/a/the-genetic-code-discovery-and-properties&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; for a gentle introduction to the subject. So, in GE, codons also have a similar role to the role of codons in biology, i.e. they are used to build the actual phenotypes (in biology, the phenotypes would be the proteins or, ultimately, the organism).&lt;/p&gt;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&lt;p&gt;Codons do not have to be 2-bit subsequences, but they can be &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-bit subsequences, for some arbitrary &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. The term &lt;strong&gt;codon&lt;/strong&gt; is similar to the term &lt;strong&gt;gene&lt;/strong&gt;, which is also often used to refer to specific subsequences of the genotype (for example, in genetic algorithms), although they may not be synonymous (at least in biology, &lt;a href=&quot;https://en.wikipedia.org/wiki/Gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;genes&lt;/a&gt; are made up of sequences of codons, so they are not synonymous). Moreover, the binary &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-bit codons can first be mapped to integers, so codons can also just be integers, as e.g. used &lt;a href=&quot;https://www.researchgate.net/publication/261165893_Termination_in_Grammatical_Evolution_grammar_design_wrapping_and_tails&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or illustrated in figure 2.2 of &lt;a href=&quot;http://www.cs.uu.nl/docs/vakken/ias/stuff/Foundations%20in%20Grammatical%20Evolution%20-%20Dempsey%20et%20al.%20-%20Ch.%202%20-%202009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this chapter&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;You can find more info about codons in the book &lt;a href=&quot;https://www.researchgate.net/publication/221900758_Genetic_Programming_An_Introduction_on_the_Automatic_Evolution_of_computer_programs_and_its_Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Programming: An Introduction&lt;/a&gt; by Wolfgang Banzhaf et al., specifically sections 9.2.2. (p. 255), 9.2.3. (an example) and 2.3 (p. 39), or in &lt;a href=&quot;http://www.cs.uu.nl/docs/vakken/ias/stuff/Foundations%20in%20Grammatical%20Evolution%20-%20Dempsey%20et%20al.%20-%20Ch.%202%20-%202009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chapter 2&lt;/a&gt; of the book &lt;a href=&quot;https://link.springer.com/book/10.1007%2F978-3-642-00314-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foundations in Grammatical Evolution&lt;/a&gt; by Dempsey et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-29T00:03:06.360" LastActivityDate="2020-12-29T00:03:06.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25416" PostTypeId="5" CreationDate="2020-12-28T01:09:57.663" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-28T01:09:57.663" LastActivityDate="2020-12-28T01:09:57.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25417" PostTypeId="4" CreationDate="2020-12-28T01:09:57.663" Score="0" Body="For questions related to the concept of a codon in grammatical evolution and genetic programming." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-28T01:09:57.663" LastActivityDate="2020-12-28T01:09:57.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25418" PostTypeId="2" ParentId="20118" CreationDate="2020-12-28T05:33:31.280" Score="0" Body="&lt;p&gt;The question is generalizability. I completely agree though but, ideally the policy found will generalize to more complex environments the model hasn't seen. You could also run a planner on a new scenario but the issue is that it would be too computationally demanding for real time.&lt;/p&gt;&#xA;" OwnerUserId="32390" LastActivityDate="2020-12-28T05:33:31.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25421" PostTypeId="1" CreationDate="2020-12-28T08:18:56.187" Score="1" ViewCount="83" Body="&lt;p&gt;I have time-series data. When I take an action, it impacts the next state, because my action directly determines the next state, but it is not known what the impact is.&lt;/p&gt;&#xA;&lt;p&gt;To be concrete: I have &lt;span class=&quot;math-container&quot;&gt;$X(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a(t-1)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$X(t)$&lt;/span&gt; is n-dimensional time-series data and &lt;span class=&quot;math-container&quot;&gt;$a(t)$&lt;/span&gt; is 1-dimensional time-series data. At time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, they together represent the observation/state space. Also, at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent makes a decision about &lt;span class=&quot;math-container&quot;&gt;$a(t)$&lt;/span&gt;. This decision (action) &lt;span class=&quot;math-container&quot;&gt;$a(t)$&lt;/span&gt; directly defines the next state space &lt;span class=&quot;math-container&quot;&gt;$X(t+1)$&lt;/span&gt; and rewards, by some function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; &amp;amp; &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f(a(t), X(t)) = X(t+1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(a(t), X(t)) = R(t+1)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have to estimate this impact, i.e. where I will end up (what will be the next state). I decided to use a model-based RL algorithm, because, from my knowledge, model-based RL does exactly this.&lt;/p&gt;&#xA;&lt;p&gt;Can you advise me on a good paper and Github code, to implement this project?&lt;/p&gt;&#xA;&lt;p&gt;As I noticed, there do not exist many works on Model-based RL.&lt;/p&gt;&#xA;" OwnerUserId="43317" LastEditorUserId="2444" LastEditDate="2020-12-28T22:36:23.187" LastActivityDate="2021-05-29T07:02:00.220" Title="Model-based RL for time series data" Tags="&lt;reinforcement-learning&gt;&lt;reference-request&gt;&lt;time-series&gt;&lt;model-based-methods&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25422" PostTypeId="2" ParentId="25421" CreationDate="2020-12-28T09:58:36.727" Score="0" Body="&lt;p&gt;To my knowledge, there does not exist anything along the lines of model-based reinforcement learning with time-sensitive data. I think the best chance you have is to try to abstract the data that you have into a model which is not time-sensitive.&lt;/p&gt;&#xA;&lt;p&gt;What would happen when you get past the timestamps of your original data? I am guessing that when testing this model, you will have states with time stamps past your original latest timestamp. Or are you only using your model in the time stamps of your training data? Is it going to be a useful model then?&lt;/p&gt;&#xA;&lt;p&gt;Firstly, please note that the &lt;a href=&quot;https://ai.stackexchange.com/help/on-topic&quot;&gt;topics&lt;/a&gt; of this forum are mostly theoretical. This question leans on the boundary. If you frame your question a bit differently, ending your post with a clear question and also putting that question in the title, you will get more responses.&lt;/p&gt;&#xA;&lt;p&gt;I feel like the question is 'What are the possibilities for model-based reinforcement learning for time-sensitive data?' But this question is inherently vague as it implies that you have a model according to the definitions of models in RL, which are not time sensitive. If you would like a better answer then please provide more information or ask your question on a more practical forum if it is more practical than what I guessed.&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: would have just put some comments under the post as this is not a real sufficient answer, but my reputation is not high enough for that :P&lt;/p&gt;&#xA;" OwnerUserId="34383" LastActivityDate="2020-12-28T09:58:36.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25423" PostTypeId="1" CreationDate="2020-12-28T14:49:26.917" Score="2" ViewCount="27" Body="&lt;p&gt;Firstly as an example here is the architecture of YOLOv2&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aKOAR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aKOAR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to understand the depth of an output of a convolutional layer. For example, the first convolutional layer has the shape 3x3x32. So there are 32 filters with shape 3x3, but each filter has 3 layers and these 3 layers convolve over 3 layers of the input. At the end, values of the 3 layers are summed up and to generate 1 layer. For 32 filters, we get an output with 32 layers.&lt;/p&gt;&#xA;&lt;p&gt;If we look at the next layer, 64 filters with size 3x3 and each filter should have 32 layers. Because input has 32 layers. Is this inference true? If it is not, how does it work?&lt;/p&gt;&#xA;" OwnerUserId="43305" LastEditorUserId="2444" LastEditDate="2020-12-28T15:40:58.447" LastActivityDate="2020-12-28T15:40:58.447" Title="Do filters have as many layers as the depth of the input in CNNs?" Tags="&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;yolo&gt;&lt;filters&gt;&lt;convolutional-layers&gt;" AnswerCount="0" CommentCount="2" ClosedDate="2020-12-28T15:54:22.963" ContentLicense="CC BY-SA 4.0" />
  <row Id="25424" PostTypeId="5" CreationDate="2020-12-28T15:26:27.723" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-28T15:26:27.723" LastActivityDate="2020-12-28T15:26:27.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25425" PostTypeId="4" CreationDate="2020-12-28T15:26:27.723" Score="0" Body="For questions related to the faster R-CNN model, which was proposed in &quot;Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks&quot; (2015) by Shaoqing Ren et al. and published in NeurIPS. Faster R-CNN is an improved version of fast R-CNN, which, in turn, is an improved version of R-CNN." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-28T15:26:27.723" LastActivityDate="2020-12-28T15:26:27.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25426" PostTypeId="1" AcceptedAnswerId="25429" CreationDate="2020-12-28T16:19:44.350" Score="1" ViewCount="55" Body="&lt;p&gt;I have two questions regarding the Selection and Expansion steps in the Monte Carlo Tree Search Algorithm. In order to state the questions, I recall the algorithm that I believe is the one most commonly associated with the MCTS. It is described as a repeated iteration of the following four steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;em&gt;Selection&lt;/em&gt;: Start from root R. Choose a leaf node L by iteration of some choice algorithm that determines which child node to choose at each branching. UTC being a prominent choice.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Expansion&lt;/em&gt;: Create one or more offspring nodes, unless L is terminal. Choose one of them, say C.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Simulation&lt;/em&gt;: Play the game from C, randomly or according to some heuristic.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Backpropagation&lt;/em&gt;: Update rewards and number of simulations for each node on the branch R--&amp;gt;C.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;When implementing this algorithm by myself I was unclear about the following interpretation of step 1 and 2:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q1. When expanding the choices at the leaf node L, do I expand all, a few or just one child?&lt;/strong&gt; If I expand all, then the tree grows exponentially large on each MCTS step, I suspect. When I expand one or a few, then either the selection step itself becomes problematic or the term leaf does. The first problem arises, because after the expansion step the node L is no longer a leaf and can never be chosen again during the selection step and in turn all the children that were not expanded will never be probed. If, however, the node L keeps being a leaf node, contrary to graph-theoretic nomenclature, then during the selection step one would need to check at each node, whether there are non-expanded child-nodes. According to which algorithm should one then choose whether to continue down the tree or expand at this non-leaf &amp;quot;leaf&amp;quot; some more yet unexpanded children?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q2&lt;/strong&gt;. Related to the first question, but slightly more in the direction of the exploitation-exploration part of the selection, I am puzzled about the UTC selection step, which again raises issues for each of the above-mentioned expansion methods: In case that a few or all child-nodes are chosen during expansion at the leaf, one is faced with the problem that &lt;strong&gt;some of those nodes will not be simulated in that MCTS step and subsequently will have a diverging UTC value&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$w_i/n_i + c \sqrt{\frac{\ln{N_i}}{n_i}}\to \infty$&lt;/span&gt;, since &lt;span class=&quot;math-container&quot;&gt;$n_i\to 0$&lt;/span&gt;. On the other hand, in case that only one child is chosen, we are facing the issue that no UTC value can be assigned to the &amp;quot;unborn&amp;quot; children on the way. In other words, one cannot use UTC to decide whether to choose a child node according to UTC at each branching or to expand the tree at that node (since all nodes within the tree may have some unexpanded child nodes).&lt;/p&gt;&#xA;" OwnerUserId="43332" LastActivityDate="2020-12-28T16:53:17.883" Title="Unclear definition of a &quot;leaf&quot; and diverging UTC values in the Monte Carlo Tree Search" Tags="&lt;monte-carlo-tree-search&gt;&lt;monte-carlo-methods&gt;&lt;tree-search&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25427" PostTypeId="1" CreationDate="2020-12-28T16:23:05.973" Score="3" ViewCount="116" Body="&lt;p&gt;I want to build a player for the following game:&#xA;You have a board where position &lt;code&gt;1&lt;/code&gt; is your player, position &lt;code&gt;2&lt;/code&gt; is the rival player, &lt;code&gt;-1&lt;/code&gt; is a blocked cell and some positive value is a bonus. You can move up, down, left, or right. Also, each bonus has a timer until it disappears (number of steps). Furthermore, each move has a timeout limit.  At the end game, when at least one of the players is stuck, we check the scores and announce the winner.&lt;/p&gt;&#xA;&lt;p&gt;Board example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; -1 -1  0  0  0 -1 -1  -1&#xA; -1  0 -1 -1 -1  0  0  340&#xA; -1 -1  0  0  0 -1  0   0&#xA; -1  0  0 -1  1 -1  0  -1&#xA; -1  0  0 -1 -1  0  0   0&#xA;  0  0 -1 -1 -1  0  2  -1&#xA;  0 -1  0  0 -1  0  0  600&#xA; -1 -1  0  0 -1 -1 -1  -1&#xA;  0 -1  0  0  0  0 -1  -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm using the MiniMax algorithm with a time limit to play the game. If we got to children, we return &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt; for a player win, &lt;span class=&quot;math-container&quot;&gt;$-\infty$&lt;/span&gt; for the rival win, and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for a tie. If we got to a specific depth, we calculate the heuristic value. If we got timeout in some place in MiniMax, then we return the last calculated direction.  I'm trying to figure out a good strategy to win this game or get to a tie if no solution is possible.&lt;/p&gt;&#xA;&lt;p&gt;What heuristic function would you define?&lt;/p&gt;&#xA;&lt;p&gt;What I thought - four factors:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f_A$&lt;/span&gt; - The number of steps possible from each direction from the current position.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f_B$&lt;/span&gt; - The analytical distance from the center.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f_C=\max_{b\in Bonus}\frac{X * I}{Y}$&lt;/span&gt; - where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is value of the bonus,  &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; if we can get to the bonus, before it disappears (otherwise &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is the distance between the bonus and the player.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$F_D$&lt;/span&gt; - The distance between the players.&#xA;The final formula:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(s)=0.5\cdot(9-f_A(s))+0.2\cdot f_C(s)-0.2\cdot f_D(s)-0.1\cdot f_B(s)&#xA;$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm not sure if it will be a good strategy for that game or not. How would you define the heuristic function? It should also be quick to calculate it because the game has a timeout for each move.&lt;/p&gt;&#xA;&lt;p&gt;In order words, what will give us the best indication that our player is going to win/lose/tie?&lt;/p&gt;&#xA;" OwnerUserId="43333" LastEditorUserId="2444" LastEditDate="2021-01-05T19:22:58.307" LastActivityDate="2021-01-06T11:07:23.120" Title="Strategy for playing a board game with Minimax algorithm" Tags="&lt;game-ai&gt;&lt;minimax&gt;&lt;exploration-strategies&gt;&lt;heuristic-functions&gt;&lt;board-games&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25428" PostTypeId="1" CreationDate="2020-12-28T16:26:41.633" Score="1" ViewCount="232" Body="&lt;p&gt;Suppose that a simple feedforward neural network (FFNN) contains &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; hidden layers, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; training examples, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; features, and &lt;span class=&quot;math-container&quot;&gt;$n_l$&lt;/span&gt; nodes in each layer. What is the space complexity to train this FFNN using back-propagation?&lt;/p&gt;&#xA;&lt;p&gt;I know how to find the space complexity of algorithms. I found an answer &lt;a href=&quot;https://neuron.eng.wayne.edu/tarek/MITbook/chap2/2_4.html#:%7E:text=In%20a%20neural%20net%20simulation,a%20measure%20of%20Kolmogorov%20complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but here it is said that the space complexity depends on the number of units, but I think it must also depend on the input size.&lt;/p&gt;&#xA;&lt;p&gt;Can someone help me in finding its worst-case space complexity?&lt;/p&gt;&#xA;" OwnerUserId="43329" LastEditorUserId="2444" LastEditDate="2020-12-28T17:24:20.227" LastActivityDate="2021-05-28T19:03:44.160" Title="What is the space complexity for training a neural network using back-propagation?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;computational-complexity&gt;&lt;space-complexity&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25429" PostTypeId="2" ParentId="25426" CreationDate="2020-12-28T16:53:17.883" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q1. When expanding the choices at the leaf node L, do I expand all, a few or just one child?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Expanding all nodes or expanding just one node are both possible. There are different advantages and disadvantages. The obvious disadvantage of immediately expanding them all is that your memory usage will grow more quickly. I suppose that the primary advantage is that you no longer need to keep track separately of &amp;quot;actions that are legal but for which I did not yet create child nodes&amp;quot; and &amp;quot;already-created nodes&amp;quot;, which I guess might sometimes lead to better computational efficiency (especially when memory isn't a concern for you, for instance if you do relatively few iterations anyway or if you have a huge amount of memory available).&lt;/p&gt;&#xA;&lt;p&gt;In the more &amp;quot;modern&amp;quot; variants of MCTS that also use trained policy networks (like all the work inspired by AlphaGo / AlphaGo Zero / AlphaZero), it typically makes most sense to expand all children at once, because the trained network will immediately give you policy values for all children anyway, so then you can prime them all at once with those policy head outputs and also immediately start using all of those values.&lt;/p&gt;&#xA;&lt;p&gt;In the case where you choose to expand only one child at a time, indeed the terminology of &amp;quot;leaf&amp;quot; nodes becomes confusing / incorrect. I never really like the term &amp;quot;leaf&amp;quot; node in the context of MCTS anyway; they're really not leaf nodes (unless they happen to represent terminal game states), they're just nodes for which did not yet choose to instantiate all the child nodes. It's a bit more verbose, but I prefer referring to them as &amp;quot;nodes that have not yet been fully expanded&amp;quot;. That would change the description of the Selection phase to something more like:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;Selection&lt;/em&gt;: Start from root R. Choose a node L that has not yet been fully expanded (or a terminal node) by iteration of some choice algorithm that determines which child node to choose at each branching.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In case that a few or all child-nodes are chosen during expansion at the leaf, one is faced with the problem that &lt;strong&gt;some of those nodes will not be simulated in that MCTS step and subsequently will have a diverging UTC value&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$w_i/n_i + c \sqrt{\frac{\ln{N_i}}{n_i}}\to \infty$&lt;/span&gt;, since &lt;span class=&quot;math-container&quot;&gt;$n_i\to 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That's true, but I don't see that as a problem. Suppose we're at a non-fully-expanded node; a node where maybe some legal actions already have corresponding child nodes, or maybe none do, but at least some legal actions still don't have corresponding child nodes. We can view these as having a visit count of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, which we can view as leading to a UCB1 value of &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;. Since UCT picks nodes (or actions) according to an &lt;span class=&quot;math-container&quot;&gt;$\arg\max$&lt;/span&gt; over the UCB1 values, we can simply think of this as UCT always preferring to pick actions that we did not yet expand over actions that we've already expanded and created a child node for. This leads to an implementation where we'll only re-visit a node that we've already visited before if we have already visited each of its possible siblings at least once too.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2020-12-28T16:53:17.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25430" PostTypeId="2" ParentId="25428" CreationDate="2020-12-28T17:41:55.957" Score="0" Body="&lt;p&gt;I will not tell you what the exact space complexity of training an FFNN with GD and BP is (because that actually depends on the specific implementation of GD and BP and I don't want to dive into the details of some specific implementation now, maybe later!), but I will guide you towards the specific answer, which you should be able to figure out alone (although it may take some time because you need to understand all the details of the BP algorithm), if you understand this answer.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Space_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;space complexity&lt;/a&gt; of an algorithm is just the amount of memory that you need to use during the execution of the algorithm. The space complexity, like the time complexity, is typically expressed as a function of the size of the input (or the number of inputs that you have) and, usually, &lt;a href=&quot;https://en.wikipedia.org/wiki/Big_O_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;in big-O notation, i.e. in the limiting case&lt;/a&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is not the same thing as &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(n)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Omega(n)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\Theta(n)$&lt;/span&gt;. Moreover, you can also express the space/time complexity both in the worst, best, or average case, and this is orthogonal to upper (expressed with &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}$&lt;/span&gt;), lower (&lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;), or tight (&lt;span class=&quot;math-container&quot;&gt;$\Theta$&lt;/span&gt;) bounds (check &lt;a href=&quot;https://cs.stackexchange.com/a/23082/20691&quot;&gt;this&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;If you use gradient descent (GD) and back-propagation (BP) to train an FFNN, at each training iteration (i.e. a GD update), you need to store all the matrices that represent the parameters (or weights) of the FFNN, as well as the gradients and the learning rate (or other hyper-parameters). Let's denote the vector that contains all parameters of the FFNN as &lt;span class=&quot;math-container&quot;&gt;$\theta \in \mathbb{R}^m$&lt;/span&gt;, so it has &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; components. The gradient vector has the same dimensionality as &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, so we need &lt;strong&gt;at least&lt;/strong&gt; to store &lt;span class=&quot;math-container&quot;&gt;$2m + 1$&lt;/span&gt; parameters.&lt;/p&gt;&#xA;&lt;p&gt;Depending on how you implement BP, you may need more memory. For example, if you need to store all the intermediate terms of the partial derivatives, that will require more memory. To compute exactly the amount of required memory, you will have to expand the gradient vector into all their components (which may not be a pleasant experience). As I just said, this only contributes to the space complexity if you need to store these intermediate components, so, ultimately, the space complexity of an algorithm depends on the &lt;strong&gt;specific&lt;/strong&gt; implementation of the algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, to be precise, we cannot just say that the space complexity is &lt;span class=&quot;math-container&quot;&gt;$2m + 1$&lt;/span&gt; or whatever the amount of memory that you require is (although many careless or ignorant people will just say that), because we are not expressing this complexity as a function of the size of the input in the &lt;strong&gt;limiting case&lt;/strong&gt; (which is usually done when expressing the space complexity of an algorithm), the number of layers or the number of units per layer (and you probably want to express the space complexity as a function of these 3 possible variable hyper-parameters).&lt;/p&gt;&#xA;&lt;p&gt;If you take a look at &lt;a href=&quot;https://ai.stackexchange.com/q/13612/2444&quot;&gt;this answer&lt;/a&gt;, where I describe how to compute the time complexity of the forward pass of an MLP (or FFNN) &lt;em&gt;as a function of the number of inputs and outputs, the number of layers, and units per layers&lt;/em&gt;, then you can express the space complexity for training an FFNN in the same way. Given that you are already familiar with how space and time complexities of an algorithm are calculated (and given that this answer is already quite long), I will not repeat the description here.&lt;/p&gt;&#xA;&lt;p&gt;In any case, to answer one of your questions more directly, yes, the space complexity will depend on the number of inputs that you have, because the number of inputs will determine the number of weights in the first layer, which you need to store in memory. This is true in the case of FFNNs (or MLPs) but note that this would not be true in the case of CNNs (i.e. the number of parameters in the convolutional layers does not depend on the size of the input), and that's why CNNs are often said to be more memory efficient.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-29T14:58:40.547" LastActivityDate="2020-12-29T14:58:40.547" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="25431" PostTypeId="2" ParentId="24891" CreationDate="2020-12-28T20:58:19.020" Score="0" Body="&lt;h2&gt;Spectral Graph Convolution&lt;/h2&gt;&#xA;&lt;p&gt;We use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolution Theorem&lt;/a&gt; to define convolution for graphs. The Convolution Theorem states that the Fourier transform of the convolution of two functions is the pointwise product of their Fourier transforms:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{F}(w*h) = \mathcal{F}(w) \odot \mathcal{F}(h) \tag{1}\label{1} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ w * h = \mathcal{F}^{-1}(\mathcal{F}(w)\odot\mathcal{F}(h))  \tag{2}\label{2}$$&lt;/span&gt;&#xA;Here &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the filter in spatial domain(time domain) and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the signal in spatial domain(time domain). For images this signal &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$2D$&lt;/span&gt; matrix and for other cases this &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; can be a &lt;span class=&quot;math-container&quot;&gt;$1D$&lt;/span&gt; signal.&lt;/p&gt;&#xA;&lt;p&gt;Assume we have &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; number of node in a graph. In graph fourier transform the eigenvalues carry the notion of frequency. &lt;span class=&quot;math-container&quot;&gt;$\Lambda$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$ n \times n$&lt;/span&gt; egenvalue matrix and it is a diagonal matrix. We can write equation 2 as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$w * h = \phi(\phi^{T}w \odot \phi^{T}h) = \phi\hat{w}(\Lambda)\phi^{T}h \tag{3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is the eigenvector matrix of graph Laplacian &lt;span class=&quot;math-container&quot;&gt;$\in R^{n \times n}$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$\hat{w}(\Lambda)$&lt;/span&gt; is the filter in spectral domain(frequency domain) &lt;span class=&quot;math-container&quot;&gt;$\in R^{n \times n}$&lt;/span&gt; a diagonal matrix, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$1D$&lt;/span&gt; graph signal &lt;span class=&quot;math-container&quot;&gt;$\in R^{n}$&lt;/span&gt; in spatial domain and w is the filter in spatial domain &lt;span class=&quot;math-container&quot;&gt;$\in R^{n}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Vanilla Spectral GCN&lt;/h3&gt;&#xA;&lt;p&gt;We define the spatial convolutional layer such that given layer &lt;span class=&quot;math-container&quot;&gt;$h^{l}$&lt;/span&gt; , the activation of the next layer is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h^{l+1}=\sigma(w^l*h^l)  \tag{4}\label{4},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; represents a nonlinear activation and &lt;span class=&quot;math-container&quot;&gt;$w^l$&lt;/span&gt; is a spatial filter and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the graph signal.&lt;/p&gt;&#xA;&lt;p&gt;We can perform the above equation in terms of spectral graph convolution operation as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h^{l+1}=\sigma(\hat{w}^l*\hat{h}^l)  \tag{5}\label{5},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hat{w}$&lt;/span&gt; is the same filter but in the spectral domain(frequency domain). In case of vanilla GCN this equation yeild to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ h^{l+1} = \sigma(\phi\hat{w}^{l}(\Lambda)\phi^{T}h)  \tag{6}\label{6}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, we will learn the &lt;span class=&quot;math-container&quot;&gt;$\hat{w}$&lt;/span&gt; using backpropagation.&lt;/p&gt;&#xA;&lt;p&gt;This vanilla GCN has several limitations, like larger time complexity and this does not guarantee localization in the spatial domain that we get from CNN's filter.&lt;/p&gt;&#xA;&lt;p&gt;In next works, such as SplineGCNs, &lt;a href=&quot;https://arxiv.org/pdf/1606.09375.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChebNet&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/pdf/1609.02907.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Klipf and Welling's GCN&lt;/a&gt;, and many other works address those issues, and try to solve them.&lt;/p&gt;&#xA;&lt;p&gt;Note that we can think of ChebNet and Klipf and Welling's GCN as a message-passing system, but, in the background, they are computing spectral convolution and also they use some standard assumption that's why we do not need any eigenvector and we implement them in the spatial domain, but still they are spectral convolution.&lt;/p&gt;&#xA;&lt;p&gt;There is also another branch in graph convolution called spatial graph convolution. I only talked about the spectral graph convolution.&lt;/p&gt;&#xA;" OwnerUserId="28048" LastEditorUserId="28048" LastEditDate="2020-12-29T08:22:24.777" LastActivityDate="2020-12-29T08:22:24.777" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25432" PostTypeId="2" ParentId="25296" CreationDate="2020-12-28T21:09:50.283" Score="0" Body="&lt;p&gt;I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications.&lt;/p&gt;&#xA;&lt;p&gt;There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mathworld.wolfram.com/SymmetricSuccessiveOverrelaxationMethod.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symmetric relaxation&lt;/a&gt; can be a challenging subject, so I'll include a few links &lt;a href=&quot;https://www.researchgate.net/publication/245356183_Calculation_of_Laminar_Separated_Flow_in_Symmetric_Two-Dimensional_Diffusers&quot; rel=&quot;nofollow noreferrer&quot;&gt;academic research&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1902.05157.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Academic research&lt;/a&gt; arxiv.org is another site I use for research. Hope this helps.&lt;/p&gt;&#xA;&lt;p&gt;I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. &lt;a href=&quot;https://medium.com/bcggamma/lagrangian-relaxation-can-solve-your-optimization-problem-much-much-faster-daa9edc47cc9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Medium Lagrangian Relaxation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34095" LastEditorUserId="2444" LastEditDate="2020-12-28T22:02:16.310" LastActivityDate="2020-12-28T22:02:16.310" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25433" PostTypeId="2" ParentId="25369" CreationDate="2020-12-28T21:12:51.810" Score="1" Body="&lt;p&gt;I found these links so hopefully they help.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;https://openai.com/blog/openai-api/&#xA;https://nordicapis.com/on-gpt-3-openai-and-apis/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="34095" LastActivityDate="2020-12-28T21:12:51.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25434" PostTypeId="1" CreationDate="2020-12-28T21:53:02.707" Score="0" ViewCount="99" Body="&lt;p&gt;Say I use an RNN, whatever is the cell's type, to perform time series classification. It can thus be seen as sequence classification. The time series is split into random, equal size, overlapping window. Once trained the goal is to make real time classification. So, here is tow scenario:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each window is a sequence of &lt;code&gt;n&lt;/code&gt; samples used to train the RNN on &lt;code&gt;n&lt;/code&gt; time steps. Then, the real-time classification is done with a sliding widow.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In that case: (1) is there any other methods than using a sliding window in real time ? (2) Is there any links that is made between a window and the previous one(s) ?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;From each window is extracted a summary vector.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In that case: do i have to construct sequences of &lt;code&gt;n&lt;/code&gt; summary vector from &lt;code&gt;n&lt;/code&gt; windows, or can I use a one time step RNN which takes the last previous internal state into the first current internal state ?&lt;/p&gt;&#xA;&lt;p&gt;This post comes from the fact that, from what i saw, there is no temporal links that are made between sequences. RNN's are just trained with randomly picked up sequences. But if tow sequences follows, the second one may be related to the first one. Or the last one may even relate to a distant one.&lt;/p&gt;&#xA;" OwnerUserId="43239" LastEditorUserId="43239" LastEditDate="2021-02-02T14:33:26.733" LastActivityDate="2021-02-02T14:33:26.733" Title="How are temporal links made between following sequences in RNN?" Tags="&lt;classification&gt;&lt;recurrent-neural-networks&gt;&lt;long-short-term-memory&gt;&lt;time-series&gt;&lt;sequence-modelling&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25435" PostTypeId="1" CreationDate="2020-12-29T01:32:39.440" Score="0" ViewCount="14" Body="&lt;p&gt;I have been trying to use a linear regression with Turicreate to predict the a certain number based on a variety of input numbers. My process is pretty simple: I have four columns in my training dataset (csv file). The columns are numbers 1, 2, 3 and the output. There isn't necessarily a mathematical relationship between the numbers in the sense that you can't just say n1 + n2*n3 = the output.&lt;/p&gt;&#xA;&lt;p&gt;I have realized that the more I train the model, the less accurate it becomes. I think this is because the data doesn't really fit a regression: as I said, you can't just perform a mathematical equation on the numbers to get an output. Roughly speaking, the relationship between the numbers is that if n1 is less than 5, and if n2 is less than n3, then the output should be fairly high. The higher n1 is, the lower the output should be.&lt;/p&gt;&#xA;&lt;p&gt;I believe that a &lt;a href=&quot;https://ai.stackexchange.com/questions/13343/which-models-accept-numerical-parameters-and-produce-a-numerical-output&quot;&gt;multivariable nonlinear regression&lt;/a&gt; is the best way to do this, but I am not completely sure. Is that the best way to do this, or is there a different type of model that would be better?&lt;/p&gt;&#xA;" OwnerUserId="33078" LastActivityDate="2020-12-29T01:32:39.440" Title="Generating numerical output based on multiple inputs" Tags="&lt;neural-networks&gt;&lt;non-linear-regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25436" PostTypeId="1" CreationDate="2020-12-29T05:39:37.220" Score="0" ViewCount="23" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: &lt;strong&gt;HOW DO I APPLY &lt;code&gt;LSH&lt;/code&gt; WITH A DEEP LEARNING MODEL TO BUILD A IMAGE-IMAGE SEARCH ENGINE ON &amp;gt;20M IMAGES&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I want to build a system where I am helping my colleagues to improve the search of incoming images. I'll be putting the pictures of how my data looks like at the end.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;: In the very broad terms, goal is to return the &lt;code&gt;tok-k&lt;/code&gt; similar images from more than 20M images. My idea before was to get some &lt;code&gt;similarity measure&lt;/code&gt; (l-1,l-2,cosine, SSIM etc etc) to get the similarity of two images &lt;strong&gt;OR&lt;/strong&gt; to use a &lt;code&gt;Siamese Network&lt;/code&gt; &lt;strong&gt;BUT&lt;/strong&gt; , obviously, saving &lt;code&gt;n*m&lt;/code&gt; dimensional vector of each image in memory is impossible and on top of that, comparison will take WHOLE LOT OF TIME when we are talking about live search obviously &lt;code&gt;O(N)&lt;/code&gt; time.&lt;/p&gt;&#xA;&lt;p&gt;So I stumbled across &lt;a href=&quot;https://www.pyimagesearch.com/2014/12/01/complete-guide-building-image-search-engine-python-opencv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This OpenCV based search engine&lt;/a&gt;. And I get the idea that you decrease the dimensions to get a latent space, save it in memory, and then compare. But still, there 2 problems with that, I want to work with images that has &lt;code&gt;text&lt;/code&gt; in them and obviously, comparing to let's say 20M is still a problem.&lt;/p&gt;&#xA;&lt;p&gt;So I think I need to train a &lt;code&gt;Neural Network&lt;/code&gt; for my images from scratch say &lt;code&gt;ResNet&lt;/code&gt; or  use existing weights.&lt;/p&gt;&#xA;&lt;p&gt;So I got to know about &lt;code&gt;LSH&lt;/code&gt; and &lt;code&gt;Near-Duplicate&lt;/code&gt; algorithm where comparison is &amp;lt; &lt;code&gt;O(N)&lt;/code&gt; for sure in worst case.&lt;/p&gt;&#xA;&lt;p&gt;Now the problem is that I do not know how to use &lt;code&gt;LSH&lt;/code&gt; with the &lt;code&gt;CNN&lt;/code&gt; Neural Network. How do I train my model so that it can learn?&lt;/p&gt;&#xA;&lt;p&gt;My idea for dimensionality reduction is that:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Train a &lt;code&gt;siamese&lt;/code&gt; network with &lt;code&gt;Triplet Loss&lt;/code&gt; which gives me 2 outputs and train it on similar and non similar images.&lt;/li&gt;&#xA;&lt;li&gt;when the model is fully trained, get the low dimensional representations of all of the Images in memory (excluding the last layer)&lt;/li&gt;&#xA;&lt;li&gt;Now when a new image comes, pass to the &lt;code&gt;siamese&lt;/code&gt; network, get it's representation and compare it with the given representations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;HOW DO I APPLY &lt;code&gt;LSH&lt;/code&gt; WITH THIS MODEL&lt;/strong&gt;? How do I predict the &lt;code&gt;top-k&lt;/code&gt; similar images because 20M images in memory is still not feasible. Please help. Any code, reference, video, tutorial will be helpful. (Not research paper)&lt;/p&gt;&#xA;" OwnerUserId="36062" LastActivityDate="2020-12-29T05:39:37.220" Title="How to build a commercial Image-Image search engine using LSH / Near Duplicate or some other algo on more than 20M images" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25437" PostTypeId="1" CreationDate="2020-12-29T06:40:01.373" Score="0" ViewCount="56" Body="&lt;p&gt;I was having looking at this &lt;a href=&quot;https://www.youtube.com/watch?v=CIfsB_EYsVI&amp;amp;t=2039s&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture&lt;/a&gt; by Ian Goodfellow and my doubt is around 18:00 timestamp where he explains generation of adversarial examples using FGSM.&lt;/p&gt;&#xA;&lt;p&gt;He mentions that the there is a linear relationship between the input to the model and the output as the activation functions are piece-wise linear with a small number of pieces. I'm not very clear what he means by input and output. Is he referring to inputs and outputs of a single layer or the input image and final output?&lt;/p&gt;&#xA;&lt;p&gt;He states that the relation between the parameters (weights) of a model and the output are non-linear which is what makes it difficult to train a neural network, thus it is much &lt;strong&gt;easier to find an adversarial example&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Could someone explain what is linear in what? and how linearity helps in adversarial example construction?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: As per my understanding FGSM method relies on the linearity of the loss function with respect to the input image. It constructs an adversarial example by perturbing the input in the direction of the gradient of the loss function w.r.t image. I am not able to understand why this works?&lt;/p&gt;&#xA;" OwnerUserId="43272" LastEditorUserId="43272" LastEditDate="2021-01-01T13:19:56.460" LastActivityDate="2021-01-31T15:09:05.950" Title="Why is it easier to construct adversarial examples relative to training neural networks?" Tags="&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25438" PostTypeId="1" CreationDate="2020-12-29T08:44:52.780" Score="0" ViewCount="14" Body="&lt;p&gt;Computer programs have been produced for games such as Chess, Go, Poker, StarCraft 2, Dota. The best ones, Deep Blue and AlphaGo , AlphaZero, Pluribus,... are now considered better than the best human players. More to the point, the computers' game results have been influencing human play.&lt;/p&gt;&#xA;&lt;p&gt;Apparently, computers are not yet better than human players in Bridge. There can be computer simulations of various hands and hypothetical opposing hands. But what progress have computers made in playing human players in tournaments? Have any new theories of bidding or play evolved as a result of computer-human interaction in Bridge?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is question was &lt;a href=&quot;https://boardgames.stackexchange.com/questions/50507/what-progress-has-been-made-in-computerized-bridge-play&quot;&gt;asked at Board &amp;amp; Card Games Q&amp;amp;A&lt;/a&gt; however I think here it might get better answer&lt;/p&gt;&#xA;" OwnerUserId="43351" LastEditorUserId="43351" LastEditDate="2020-12-29T11:16:23.917" LastActivityDate="2020-12-29T11:16:23.917" Title="What progress has been made in computerized bridge play?" Tags="&lt;games-of-chance&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25439" PostTypeId="1" CreationDate="2020-12-29T09:27:06.210" Score="3" ViewCount="81" Body="&lt;p&gt;AI reached a super-human level in many complex games such as &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0004370201001291&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chess&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/nature24270&quot; rel=&quot;nofollow noreferrer&quot;&gt;Go&lt;/a&gt; ,&lt;a href=&quot;https://science.sciencemag.org/content/365/6456/885.abstract?casa_token=nyS4YED061UAAAAA:-YZnlUJVkOPOC7UOVxgUyaT_4XK_QqXMlJMovMSWyCxoAyCCzSB1xTSndhlu2IGPU1fXj1HDdvLunIc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Texas hold’em Poker&lt;/a&gt;, &lt;a href=&quot;https://openai.com/projects/five/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dota2&lt;/a&gt; and &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1724-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;StarCarft2&lt;/a&gt;. However it still did not reach this level in trick-taking card games.&lt;/p&gt;&#xA;&lt;p&gt;Why there is no &lt;strong&gt;super-human&lt;/strong&gt; AI playing imperfect-information, multi-player, trick-taking card games such as Spades, Whist, Hearts, Euchre and Bridge?&lt;/p&gt;&#xA;&lt;p&gt;In particular, what &lt;strong&gt;are the obstacles&lt;/strong&gt; for making a super-human AI in those games?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I think those are the reasons that makes Spades hard for AI to master:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Imperfect information&lt;/strong&gt; games pose two distinct problems: move selection and inference.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The size of the game tree isn't small, however larger games have been mastered.&lt;/p&gt;&#xA;&lt;p&gt;I. &lt;strong&gt;History size&lt;/strong&gt;: &lt;span class=&quot;math-container&quot;&gt;$14!^4 = 5.7\cdot10^{43}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;II. There are &lt;span class=&quot;math-container&quot;&gt;$\frac{52!}{13!^4}= 5.4\cdot10^{28}$&lt;/span&gt; possible initial states.&lt;/p&gt;&#xA;&lt;p&gt;III. Each initial information set can be completed into a full state in &lt;span class=&quot;math-container&quot;&gt;$\frac{39!}{13!^3}=8.45\cdot10^{16} $&lt;/span&gt; ways&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Evaluation only at terminal states&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Multiplayer&lt;/strong&gt; games:&lt;/p&gt;&#xA;&lt;p&gt;I. harder to prune - search algorithms are less effective&lt;/p&gt;&#xA;&lt;p&gt;II. opponent modeling is hard&lt;/p&gt;&#xA;&lt;p&gt;III. Goal choosing - several goals are available, need to change goals during rounds according to the reveled information.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Agent need to coordinate with a partner: &lt;strong&gt;conventions&lt;/strong&gt;, &lt;strong&gt;signals&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="43351" LastEditorUserId="43351" LastEditDate="2021-01-21T06:40:30.983" LastActivityDate="2021-01-21T06:40:30.983" Title="Why multiplayer, imperfect information, trick-taking card games are hard for AI?" Tags="&lt;gaming&gt;&lt;state-of-the-art&gt;&lt;computational-complexity&gt;&lt;imperfect-information&gt;&lt;games-of-chance&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25440" PostTypeId="2" ParentId="23087" CreationDate="2020-12-29T10:31:50.990" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;First, is it even possible to use DDPG for multi-dimensional&#xA;continuous action spaces?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, DDPG was primarily developed to deal with continuous action space you can find out more &lt;a href=&quot;https://towardsdatascience.com/deep-deterministic-policy-gradient-ddpg-theory-and-implementation-747a3010e82f&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://towardsdatascience.com/deep-deterministic-policy-gradients-explained-2d94655a9b7b&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://openreview.net/pdf?id=HyxnH64KwS&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have not found any code examples to learn from and many of the&#xA;papers I have read are near the limit of my understanding in this&#xA;area.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can check &lt;a href=&quot;https://keras-rl.readthedocs.io/en/latest/agents/ddpg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kerasRL&lt;/a&gt; ddpg implementation it is quite easy to understand and can solve your problem of a model with multiple outputs..&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;why might my actor network be outputting values clustered near its&#xA;max/min values, and why would the values in either cluster all be the&#xA;same?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think it is just a convergence problem try out small learning rates and/or small target_model_update param.&lt;/p&gt;&#xA;&lt;p&gt;If this doesn't work for you check TRPO and/or PPO..&lt;/p&gt;&#xA;" OwnerUserId="43047" LastActivityDate="2020-12-29T10:31:50.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25441" PostTypeId="1" CreationDate="2020-12-29T11:27:04.450" Score="0" ViewCount="25" Body="&lt;p&gt;This is for a personal project, and I am no very skilled programmer, language is Python, needs to work offline. That said: I have a list of 200 key actions, eg. 'Building sandcastles', 'Paintball' or 'Swinging at the playground'. And I have about a page of natural text loosely revolving around some of these actions. How do I detect where the natural text mentions one of these actions, and which one?&lt;/p&gt;&#xA;&lt;p&gt;Ideally this would be something smart, some easy-to-use machine learning Python library, which I can just feed my keywords and natural text. But it would need to be something straightforward, as I'm not skilled enough to set this up myself. What libraries should I be looking at?&lt;/p&gt;&#xA;&lt;p&gt;As a non-smart fallback, I could imagine processing my keywords and search for these strings in my natural text, but that feels very cumbersome ... So, needless to say, any help is appreciated a lot, thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="31985" LastActivityDate="2020-12-29T11:27:04.450" Title="Python: Recognize actions / keywords in natural text?" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;text-classification&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25442" PostTypeId="1" CreationDate="2020-12-29T13:48:54.393" Score="1" ViewCount="65" Body="&lt;p&gt;I have been looking at the NER example with Trax in this notebook: &lt;a href=&quot;https://github.com/google/trax/blob/master/trax/examples/NER_using_Reformer.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/google/trax/blob/master/trax/examples/NER_using_Reformer.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However the notebook only gives an example as far as training the model. I can't find any examples of how to use this model to extract entities from a new string of text.&lt;/p&gt;&#xA;&lt;p&gt;I've tried the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Instantiate the model in 'predict' mode. When trying this I get the same error reported in &lt;a href=&quot;https://github.com/google/trax/issues/556&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/google/trax/issues/556&lt;/a&gt; &lt;code&gt;AssertionError: In call to configurable 'SelfAttention' (&amp;lt;class 'trax.layers.research.efficient_attention.SelfAttention'&amp;gt;)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Instantiate the model in 'eval' mode and then running &lt;code&gt;model(sentence)&lt;/code&gt; as I would with other models. In this case the instantiation works but I get the following error when running the model: &lt;code&gt;TypeError: Serial.forward input must be a tuple or list; instead got &amp;lt;class 'numpy.ndarray'&amp;gt;&lt;/code&gt;. Presumably this is because in 'eval' mode the model needs 2 entries passed in rather than one sentence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How can I use this Reformer to extract entities from a new sentence?&lt;/p&gt;&#xA;" OwnerUserId="43256" LastActivityDate="2020-12-29T13:48:54.393" Title="Using Reformer model to do NER in Trax" Tags="&lt;transformer&gt;&lt;named-entity-recognition&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25444" PostTypeId="5" CreationDate="2020-12-29T21:34:17.747" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-29T21:34:17.747" LastActivityDate="2020-12-29T21:34:17.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25445" PostTypeId="4" CreationDate="2020-12-29T21:34:17.747" Score="0" Body="For questions related to AI techniques that generate stories or plots." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2020-12-29T21:34:17.747" LastActivityDate="2020-12-29T21:34:17.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25446" PostTypeId="1" CreationDate="2020-12-29T21:54:09.610" Score="0" ViewCount="109" Body="&lt;p&gt;During the last couple of days, I am experimenting with the different schedulers of learning rate decay offered by Keras (link &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/schedules&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). Specifically, I have been using &lt;em&gt;InverseTimeDecay: A LearningRateSchedule that uses an inverse time decay schedule.&lt;/em&gt; This type of scheduler uses the following two parameters:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;decay_steps&lt;/li&gt;&#xA;&lt;li&gt;decay_rate&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would like to understand how those two parameters are initialized and taken into consideration by the InverseTimeDecay class. So far, what I know is that the following formula is used (based on the documentation):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def decayed_learning_rate(step):&#xA;  return initial_learning_rate / (1 + decay_rate * step / decay_step)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The parameter initialization I use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def optimizer_adam_v2(hparams):&#xA;    # Inverse Time Decay&#xA;    initial_learning_rate = 0.001&#xA;    decay_steps = int(np.ceil(len(x_train)//32))*10 #len(x_train) = 31334, and 10 represents the epoch. So after 10 epochs or similarly 9791 steps the decay rate will be applied.&#xA;    decay_rate = 0.5&#xA;    learning_rate_fn = keras.optimizers.schedules.InverseTimeDecay(initial_learning_rate, decay_steps, decay_rate)&#xA;&#xA;    return keras.optimizers.Adam(learning_rate=learning_rate_fn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Based on the values above and the formula, the &lt;code&gt;decay_learning_rate&lt;/code&gt; should be equal to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;decay_learning_rate = 0.001 / (1 + 0.5 * step / 9791) = 0.0009999489353010264&#xA;# 9791 = int(np.ceil(31334//32))*10 #decay_steps parameter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When the training process begins the learning rate starts to drop by the decay_learning_rate at every step of the training. However, based on the &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/schedules/InverseTimeDecay&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; (  &lt;code&gt;decay_steps&lt;/code&gt; is &amp;quot;How often to apply decay&amp;quot;). the learning rate should start dropping after the training pass the value of the decay_steps. So after 9791 steps (approximately 10 epochs). Instead, the learning rate starts to drop from epoch 0.&lt;/p&gt;&#xA;&lt;p&gt;Why is this happening? Why the decay starts from epoch 0 and not from epoch 10 (9791 steps have passed).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;[UPDATE] - 02.01.2021&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is also observed with the Piecewise Constant Decay (link &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/schedules/PiecewiseConstantDecay&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Piecewise constant decay is a similar scheduler to InverseTimeDecay with a much simpler application. The concept is that for every N number of steps the learning rate is constant and changes based on array of values.&lt;/p&gt;&#xA;&lt;p&gt;For example I have implemented a piecewise decay scheduler like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def optimizer_adam_v2():&#xA;    # PiecewiseConstantDecay&#xA;    step = tf.Variable(0, trainable=False)&#xA;    boundaries = [1000, 5000]&#xA;    values = [0.001, 0.0005, 0.00025]&#xA;&#xA;    learning_rate_fn = keras.optimizers.schedules.PiecewiseConstantDecay(boundaries, values)&#xA;&#xA;    # Later, whenever we perform an optimization step, we pass in the step.&#xA;    learning_rate_adam = learning_rate_fn(step)&#xA;&#xA;    return keras.optimizers.Adam(learning_rate=learning_rate_adam)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In short, &lt;em&gt;based always on the relevant documentation written by Tensorflow experts&lt;/em&gt;, what the code above does is to use a learning rate of 0.001 for the first 1000 steps, 0.0005 for the next 5000 steps (until 6000 steps completed), and the for the rest of the training use 0.00025 learning rate.&#xA;Since I monitored the learning rate of my scheduler I got the following result&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QJn3Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QJn3Y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;With the indication &lt;code&gt;lr&lt;/code&gt; I monitor the learning rate of the Adam optimizer. Until epoch 15 the learning rate didn't change whatsoever. Based on step calculation methodology, 15 epochs is approximately equal to &lt;code&gt;len(x_train)//batch_size * epochs = (31335//32)*15 = 979*15 = 14,687 steps.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;After all, my question is still on the table. Why the learning rate of Adam does not obey the change rules of each scheduler.&lt;/p&gt;&#xA;&lt;p&gt;You may find the implementation of the &lt;code&gt;Piecewise constant decay&lt;/code&gt; scheduler on my colab notebook &lt;a href=&quot;https://colab.research.google.com/drive/1BsdrgZDtoDEASTikKa4LEVAz7IBzp3jw?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="38530" LastEditorUserId="38530" LastEditDate="2021-01-02T14:11:46.530" LastActivityDate="2021-01-02T14:11:46.530" Title="How the parameters of decay_rate &amp; decay_steps are taken into consideration while computing InverseTimeDecay()" Tags="&lt;tensorflow&gt;&lt;keras&gt;&lt;learning-rate&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25447" PostTypeId="1" CreationDate="2020-12-29T22:33:28.183" Score="1" ViewCount="32" Body="&lt;p&gt;From the AlphaZero paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The input to the neural network is an N × N × (M T + L) image stack that represents state using a concatenation of T sets of M planes of size N × N . Each set of planes represents the board position at a time-step t − T + 1, ..., t, and is set to zero for time-steps less than 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;From the original AlphaGo Zero paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Expand and evaluate (Figure 2b). The leaf node &lt;span class=&quot;math-container&quot;&gt;$s_L$&lt;/span&gt; is added to a queue for neural network evaluation, &lt;span class=&quot;math-container&quot;&gt;$(d_i(p), v) = f_\Theta(d_i(s_L))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; is a dihedral reﬂection or rotation selected uniformly at random from i∈[1..8]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Ignoring the dihedral reflection, the formula in the original paper &lt;span class=&quot;math-container&quot;&gt;$f_\Theta(s_L)$&lt;/span&gt; implies that only the board corresponding to &lt;span class=&quot;math-container&quot;&gt;$s_L$&lt;/span&gt; is passed to the neural network for evaluation when expanding a node in MCTS, not including the 7 boards from the 7 previous time steps. Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="43016" LastActivityDate="2020-12-29T22:33:28.183" Title="Are inputs into AlphaZero the same during the evaluate step in MCTS and during test time?" Tags="&lt;deep-rl&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25448" PostTypeId="1" CreationDate="2020-12-29T23:48:51.480" Score="0" ViewCount="67" Body="&lt;p&gt;I was reading the paper called &lt;a href=&quot;https://arxiv.org/pdf/1606.03498.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Improved Techniques for Training GANs&lt;/em&gt;&lt;/a&gt;. And, in the one-sided label smoothing part, they said that optimum discriminator with label smoothing is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ D^*(x)=\frac{\alpha \cdot p_{data}(x) + \beta \cdot p_{model}(x)}{p_{data}(x) + p_{model}(x)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I could not understand where this is come from. How did we get this result?&lt;/p&gt;&#xA;&lt;p&gt;Note: By the way, I knew how to find optimal discriminator in vanilla GAN i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ D^*(x) = \frac{p_{r}(x)}{p_{r}(x) + p_g(x)} $$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="41615" LastActivityDate="2020-12-30T01:54:17.783" Title="Optimum Discriminator for label smoothed GAN" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25450" PostTypeId="2" ParentId="25448" CreationDate="2020-12-30T01:54:17.783" Score="1" Body="&lt;p&gt;The equation most likely comes from one of the following references:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;David Warde-Farley and Ian Goodfellow. Adversarial perturbations of deep neural networks. In Tamir Hazan, George Papandreou, and Daniel Tarlow, editors, Perturbations, Optimization, and Statistics, chapter 11. 2016. Book in preparation for MIT Press.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. &lt;a href=&quot;https://arxiv.org/pdf/1512.00567.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rethinking the Inception Architecture for Computer Vision&lt;/a&gt;. ArXiv e-prints, December 2015.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I was not able to retrieve the book but Szegedy et al. discuss on page 6 (see &amp;quot;Model Regularization via Label Smoothing&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;Ian Goodfellow discusses this topic on page 31 of his tutorial:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I. Goodfellow. &lt;a href=&quot;https://arxiv.org/pdf/1701.00160.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIPS 2016 Tutorial: Generative Adversarial Networks&lt;/a&gt;. ArXiv e-prints April 2017.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A similar question was asked and answered in the Data Science community: &lt;a href=&quot;https://datascience.stackexchange.com/questions/28764/one-sided-label-smoothing-in-gans&quot;&gt;https://datascience.stackexchange.com/questions/28764/one-sided-label-smoothing-in-gans&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2020-12-30T01:54:17.783" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25451" PostTypeId="1" AcceptedAnswerId="25472" CreationDate="2020-12-30T02:03:50.093" Score="3" ViewCount="76" Body="&lt;p&gt;From the AlphaGo Zero paper, during MCTS, statistics for each new node are initialized as such:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;${N(s_L, a) = 0, W (s_L, a) = 0, Q(s_L, a) = 0, P (s_L, a) = p_a}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The PUCT algorithm for selecting the best child node is &lt;span class=&quot;math-container&quot;&gt;$a_t = argmax(Q(s,a) + U(s,a))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$U(s,a) = c_{puct} P(s,a) \frac{\sqrt{\sum_b N(s,b)}}{1 + N(s, a)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If we start from scratch with a tree that only contains the root node and no children have been visited yet, then this should evaluate to 0 for all actions &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that we can take from the root node. Do we then simply uniformly sample an action to take?&lt;/p&gt;&#xA;&lt;p&gt;Also, during the expand() step when we add an unvisited node &lt;span class=&quot;math-container&quot;&gt;$s_L$&lt;/span&gt; to the tree, this node's children will also have not been visited, and we run into the same problem where PUCT will return 0 for all actions. Do we do the same uniform sampling here as well?&lt;/p&gt;&#xA;" OwnerUserId="43016" LastActivityDate="2020-12-31T00:19:17.020" Title="How does AlphaZero's MCTS work when starting from the root node?" Tags="&lt;deep-rl&gt;&lt;monte-carlo-tree-search&gt;&lt;alphazero&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25452" PostTypeId="2" ParentId="17433" CreationDate="2020-12-30T02:34:30.313" Score="0" Body="&lt;p&gt;I'm curious to know the results of your efforts, here are a few studies that appear to be in the realm of your need.&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/337922291_Analysis_and_Recognition_of_Hand-Drawn_Images_with_Effective_Data_Handling&quot; rel=&quot;nofollow noreferrer&quot;&gt;first academic link&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/306081553_Classification_of_Photo_and_Sketch_Images_Using_Convolutional_Neural_Networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;second academic link&lt;/a&gt;&#xA;&lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0183838&quot; rel=&quot;nofollow noreferrer&quot;&gt;Journal link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34095" LastActivityDate="2020-12-30T02:34:30.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25453" PostTypeId="1" CreationDate="2020-12-30T03:27:55.367" Score="0" ViewCount="20" Body="&lt;p&gt;I've been trying to solve an optimization problem with ITERATED LOCAL SEARCH (ILS) method. I've generated the initial solution, then followed the steps of ILS. However, even after running the algorithm for a long time, it just keeps returning worse results. Could the perturbation function be the problem?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastActivityDate="2020-12-30T03:27:55.367" Title="What to do when Iterated Local Search (ILS) method keeps making the solution worse?" Tags="&lt;optimization&gt;&lt;meta-heuristics&gt;&lt;iterated-local-search&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25454" PostTypeId="1" CreationDate="2020-12-30T03:33:27.130" Score="0" ViewCount="15" Body="&lt;p&gt;I've been having troubles in defining the perturb function for the problem I am working on. I know that this function should help you in &amp;quot;jumping&amp;quot; to another &amp;quot;hill&amp;quot;, but not so far as to be similar to as if we used a randomized way. What are some tips that would help me to define a better perturb function? What should I consider and is there a way of thinking you would suggest me to try to apply?&lt;/p&gt;&#xA;" OwnerUserId="21832" LastActivityDate="2020-12-30T03:33:27.130" Title="What do you keep in mind while defining a perturbation function for ILS algorithm?" Tags="&lt;optimization&gt;&lt;iterated-local-search&gt;&lt;perturb&gt;&lt;perturbation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25457" PostTypeId="1" CreationDate="2020-12-30T07:32:11.313" Score="3" ViewCount="499" Body="&lt;p&gt;A &lt;em&gt;model&lt;/em&gt; can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;em&gt;function&lt;/em&gt; can be defined as a set of ordered pairs with one-to-many property from domain to co-domain/range.&lt;/p&gt;&#xA;&lt;p&gt;What is the fundamental difference between them in formal terms?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="18758" LastEditDate="2020-12-31T07:54:22.723" LastActivityDate="2021-03-02T10:18:53.080" Title="What is the fundamental difference between an ML model and a function?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;math&gt;&lt;definitions&gt;&lt;models&gt;" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25458" PostTypeId="1" CreationDate="2020-12-30T08:41:08.567" Score="3" ViewCount="308" Body="&lt;p&gt;I'm trying to understand the &lt;a href=&quot;https://paperswithcode.com/method/r1-regularization&quot; rel=&quot;nofollow noreferrer&quot;&gt;R1 regularization function&lt;/a&gt;, both the abstract concept and every symbol in the formula.&#xA;According to the article, the definition of R1 is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It penalizes the discriminator from deviating from the Nash Equilibrium via penalizing the gradient on real data alone: when the generator distribution produces the true data distribution and the discriminator is equal to 0 on the data manifold, the gradient penalty ensures that the discriminator cannot create a non-zero gradient orthogonal to the data manifold without suffering a loss in the GAN game.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$R_1(\psi  ) = \frac{\gamma}{2}E_{pD(x)}\left [ \left \| \bigtriangledown D_{\psi}(x) \right \|^2 \right ]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have basic understanding of how GAN's and back-propagation works. I understand the idea of punishing the discriminator when he deviates from the Nash equilibrium. The rest of it gets murky, even if it might be basic math. For example, I'm not sure why it matters if the gradient is orthogonal to the data.&lt;/p&gt;&#xA;&lt;p&gt;On the equation part, it's even more unclear. The discriminator input is always an image, so I assume &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an image. Then what is &lt;span class=&quot;math-container&quot;&gt;$\psi$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;(I understand this is somewhat of a basic question, but seems there are no blogs about it for us simple non-researchers, math challenged people who fail to understand the original article )&lt;/p&gt;&#xA;" OwnerUserId="43381" LastEditorUserId="2444" LastEditDate="2020-12-30T11:44:38.513" LastActivityDate="2021-05-03T18:06:30.563" Title="Can someone explain R1 regularization function in simple terms?" Tags="&lt;machine-learning&gt;&lt;generative-adversarial-networks&gt;&lt;regularization&gt;&lt;r1-regularization&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25459" PostTypeId="1" CreationDate="2020-12-30T10:04:54.760" Score="2" ViewCount="100" Body="&lt;p&gt;I'm following this &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; which enumerates the various types of attention.&lt;/p&gt;&#xA;&lt;p&gt;It mentions content-based attention where the alignment scoring function for the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th encoder hidden state with respect to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th context vector is the cosine distance:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;e_{ij} = \frac{\mathbf{h}^{enc}_{j}\cdot\mathbf{h}^{dec}_{i}}{||\mathbf{h}^{enc}_{j}||\cdot||\mathbf{h}^{dec}_{i}||}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;It also mentions dot-product attention:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;e_{ij} = \mathbf{h}^{enc}_{j}\cdot\mathbf{h}^{dec}_{i}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;To me, it seems like these are only different by a factor. If we fix &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; such that we are focusing on only one time step in the decoder, then that factor is only dependent on &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. Specifically, it's &lt;span class=&quot;math-container&quot;&gt;$1/\mathbf{h}^{enc}_{j}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So we could state: &amp;quot;the only adjustment content-based attention makes to dot-product attention, is that it scales each alignment score inversely with the norm of the corresponding encoder hidden state before softmax is applied.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;What's the motivation behind making such a minor adjustment? What are the consequences?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Follow up question:&lt;/p&gt;&#xA;&lt;p&gt;What's more, is that in &lt;a href=&quot;https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Attention is All you Need&lt;/a&gt; they introduce the scaled dot product where they divide by a constant factor (square root of size of encoder hidden vector) to avoid vanishing gradients in the softmax. Any reason they don't just use cosine distance?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2020-12-30T10:19:43.603" LastActivityDate="2021-04-19T08:55:49.723" Title="What's the difference between content-based attention and dot-product attention?" Tags="&lt;neural-networks&gt;&lt;attention&gt;&lt;seq2seq&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25460" PostTypeId="2" ParentId="25457" CreationDate="2020-12-30T10:29:32.850" Score="5" Body="&lt;h3&gt;A model as a set of functions&lt;/h3&gt;&#xA;&lt;p&gt;In some cases in machine learning, a model can be thought of as a &lt;strong&gt;set of functions&lt;/strong&gt;, so here's the first difference.&lt;/p&gt;&#xA;&lt;p&gt;For example, a neural network with an &lt;strong&gt;arbitrary&lt;/strong&gt; vector of parameters &lt;span class=&quot;math-container&quot;&gt;$\theta \in \mathbb{R}^m$&lt;/span&gt; is often denoted as a model, then a specific combination of these parameters represents a specific function. More specifically, suppose that we have a neural network with 2 inputs, 1 hidden neuron (with a ReLU activation function, denoted as &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, that follows a linear combination of the inputs), and 1 output neuron (with a sigmoid activation function, &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;). The inputs are connected to the only hidden unit and these connections have a real-valued weight. If we ignore biases, then there are 3 parameters, which can be grouped in the parameter vector &lt;span class=&quot;math-container&quot;&gt;$\theta = [\theta_1, \theta_2, \theta_3] \in \mathbb{R}^3 $&lt;/span&gt;. The set of functions that this neural network represents is defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x_1, x_2) = \sigma (\theta_3 \phi(x_1 \theta_1 + x_2 \theta_2)) \tag{1}\label{1},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this case, the equation \ref{1} represents the model, given the parameter space &lt;span class=&quot;math-container&quot;&gt;$\Theta = \mathbb{R}^3$&lt;/span&gt;. For any specific values that &lt;span class=&quot;math-container&quot;&gt;$\theta_1, \theta_2,$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_3$&lt;/span&gt; can take, we have a specific (deterministic) function &lt;span class=&quot;math-container&quot;&gt;$f: \mathbb{R} \rightarrow [0, 1]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For instance, &lt;span class=&quot;math-container&quot;&gt;$\theta = [0.2, 10, 0.4]$&lt;/span&gt; represents some specific function, namely&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x_1, x_2) = \sigma (0.4 \phi(x_1 0.2 + x_2 10.0)) \tag{2}\label{2}&#xA;$$&lt;/span&gt;&#xA;You can plot this function (with Matplotlib) for some values of the inputs to see how it looks. Note that &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; can be arbitrary (because those are just the inputs, which I assumed to be real numbers).&lt;/p&gt;&#xA;&lt;p&gt;This interpretation of a model is roughly equivalent to the definition of a &lt;a href=&quot;https://ai.stackexchange.com/a/23005/2444&quot;&gt;hypothesis class (or space)&lt;/a&gt; in computational learning theory, which is essentially &lt;a href=&quot;https://ai.stackexchange.com/a/16794/2444&quot;&gt;a set of functions&lt;/a&gt;. So, this definition of a model is useful to understand &lt;a href=&quot;https://ai.stackexchange.com/q/13317/2444&quot;&gt;the universal approximation theorems for neural networks&lt;/a&gt;, which state that you can find a specific set of parameters such that you can approximately compute some given function arbitrarily well, given that some conditions are met.&lt;/p&gt;&#xA;&lt;p&gt;This interpretation can also be applied to decision trees, HMM, RNNs, and all these ML models.&lt;/p&gt;&#xA;&lt;h3&gt;A model in reinforcement learning&lt;/h3&gt;&#xA;&lt;p&gt;The term model is also sometimes used to refer to a probability distribution, for example, in the context of reinforcement learning, where &lt;span class=&quot;math-container&quot;&gt;$p(s', r \mid s, a)$&lt;/span&gt; is a probability distribution over the next state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; and reward &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; given the current state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; taken in that state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Check &lt;a href=&quot;https://ai.stackexchange.com/q/4456/2444&quot;&gt;this question&lt;/a&gt; for more details. A probability distribution could also be thought of as a (possibly infinitely large) set of functions, but it is not just a set of functions, because you can also sample from a probability distribution (i.e. there's some stochasticity associated with a probability distribution). So, a probability distribution can be considered a statistical model or can be used to represent it. Check &lt;a href=&quot;https://ai.stackexchange.com/a/12354/2444&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;A function as a model&lt;/h3&gt;&#xA;&lt;p&gt;A specific function (e.g. the function in \ref{2}) can also be a model, in the sense that it models (or approximates) another function. In other words, a person may use the term &lt;em&gt;model&lt;/em&gt; to refer to a function that attempts to approximate another function that you want to model/approximate/compute.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-03-02T10:18:53.080" LastActivityDate="2021-03-02T10:18:53.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25462" PostTypeId="1" CreationDate="2020-12-30T14:24:37.323" Score="0" ViewCount="15" Body="&lt;p&gt;I am working on an application of vehicle detection, the purpose is to check if road is congested or not. The solution that came to my mind is to detect the vehicles moving in each direction separately and then apply limit on the number of vehicles in each direction. I have drawn horizontal lines that detect if the car is moving up or moving down. The problem I am facing is that I am unable to draw vertical lines.&#xA;If I have height and width of a frame, how can we get parallel vertical lines from that information. One solution for horizontal that I got from somewhere is;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Upper/Lower Lines&#xA;line_up = int(2*(h/6))&#xA;line_down = int(3*(h/5))&#xA;&#xA;up_limit = int(1*(h/6))&#xA;down_limit = int(4*(h/5))&#xA;&#xA;print(&amp;quot;Blue line y:&amp;quot;, str(line_down))&#xA;print(&amp;quot;Red line y:&amp;quot;, str(line_up))&#xA;line_down_color = (255,0,0)&#xA;line_up_color = (0,0,255)&#xA;pt1 = [0, line_down]&#xA;pt2 = [w, line_down]&#xA;pts_L1 = np.array([pt1,pt2], np.int32)&#xA;pts_L1 = pts_L1.reshape((-1,1,2))&#xA;pt3 = [0, line_up]&#xA;pt4 = [w, line_up]&#xA;pts_L2 = np.array([pt3,pt4], np.int32)&#xA;pts_L2 = pts_L2.reshape((-1,1,2))&#xA;&#xA;pt5 = [0, up_limit]&#xA;pt6 = [w, up_limit]&#xA;pts_L3 = np.array([pt5,pt6], np.int32)&#xA;pts_L3 = pts_L3.reshape((-1,1,2))&#xA;pt7 = [0, down_limit]&#xA;pt8 = [w, down_limit]&#xA;pts_L4 = np.array([pt7,pt8], np.int32)&#xA;pts_L4 = pts_L4.reshape((-1,1,2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to change, variables for vertical lines, like h to w. But I am not getting the desired functionality. Any help in this regard would really be appreciated.&#xA;I also want to know how ;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line_up = int(2*(h/6))&#xA;line_down = int(3*(h/5))&#xA;&#xA;up_limit = int(1*(h/6))&#xA;down_limit = int(4*(h/5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this upper code is helping to get expected horizontal lines.&lt;/p&gt;&#xA;" OwnerUserId="42499" LastActivityDate="2020-12-30T14:24:37.323" Title="Drawing verticle lines on video frames in vehicle detection" Tags="&lt;neural-networks&gt;&lt;python&gt;&lt;autonomous-vehicles&gt;&lt;opencv&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25463" PostTypeId="1" CreationDate="2020-12-30T14:51:36.040" Score="0" ViewCount="31" Body="&lt;p&gt;Suppose I have the following toy data set:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yIuLn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yIuLn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each instance has multiple labels at a time. Now you can see I have 2 instances for &lt;strong&gt;Label2&lt;/strong&gt;, however, only one, one instance for other labels. It means, we have class imbalanced issues. I read about the term &amp;quot;&lt;strong&gt;Adding Class Weights for an imbalanced dataset&lt;/strong&gt;&amp;quot;, however, I could not understand, how actually it works and beneficial. Can anyone explain this method generally as well as according to my given toy data set. Other than that, how we handle this Miss Lable (nan)?&lt;/p&gt;&#xA;" OwnerUserId="41756" LastActivityDate="2020-12-31T18:58:29.630" Title="Adding Class Weights for imbalanced dataset?" Tags="&lt;weights&gt;&lt;multi-label-classification&gt;&lt;imbalanced-datasets&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25464" PostTypeId="1" CreationDate="2020-12-30T16:13:11.343" Score="2" ViewCount="80" Body="&lt;p&gt;I found this &lt;a href=&quot;https://ai.stackexchange.com/questions/25109/is-there-anything-that-ensures-that-convolutional-filters-dont-end-up-the-same&quot;&gt;question&lt;/a&gt; very interesting, and this is a follow up on it.&lt;/p&gt;&#xA;&lt;p&gt;Presumably, we'd want all the filters to converge towards some complementary set, where each filter fills as large a niche as possible (in terms of extracting useful information from the previous layer), without overlapping with another filter.&lt;/p&gt;&#xA;&lt;p&gt;A quick thought experiment tells me (please correct me if I'm wrong) that if two filters are identical down to maximum precision, then without adding in any other form of stochastic differentiation between them, their weights will be updated in the same way at each step of gradient descent during training. Thus, it would be a very bad idea to initialise all filters in the same way prior to training, as they would all be updated in exactly the same way (see footnote 1).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, a quick thought experiment isn't enough to tell me what would happen to two filters that are &lt;em&gt;almost&lt;/em&gt; identical, as we continue to train the network. &lt;strong&gt;Is there some mechanism causing them to then diverge away from one another, thereby filling their own &amp;quot;complementary niches&amp;quot; in the layer?&lt;/strong&gt; My intuition tells me that there must be, otherwise using many filters just wouldn't work. But during back-propagation, each filter is downstream, and so they don't have any way of communicating with one another. At the risk of anthropomorphising the network, I might ask &amp;quot;How do the two filters collude with one another to benefit the network as a whole?&amp;quot;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Footnotes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why do I think this? Because the expession for the partial derivative of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;th filter weights with respect to the cost &lt;span class=&quot;math-container&quot;&gt;$\partial W^k/\partial C$&lt;/span&gt; will be identical for all &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. From the perspective of back-propagation, all paths through the filters look exactly the same.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2020-12-30T16:37:32.950" LastActivityDate="2021-01-01T07:30:58.970" Title="Is there anything that ensures that convolutional filters end up different from one another?" Tags="&lt;convolutional-neural-networks&gt;&lt;gradient-descent&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25465" PostTypeId="1" CreationDate="2020-12-30T16:56:57.597" Score="1" ViewCount="62" Body="&lt;p&gt;In my problem, the agent does not follow the successive order of states, but selects with &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy the best pair (state, action) from a priority queue. More specifically, when my agent goes to a state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and opens its available actions &lt;span class=&quot;math-container&quot;&gt;$\{ a_i \}$&lt;/span&gt;, then it estimates each &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; pair (regression with DQN) and stores it into the queue. In order for my agent to change to state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, it picks the best pair from the queue instead of following one of the available actions &lt;span class=&quot;math-container&quot;&gt;$\{ a_i \}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. I note that a state has a partially-different action set from the others.&lt;/p&gt;&#xA;&lt;p&gt;However, in this way, how can I model my MDP if my agent does not follow the successive order of states?&lt;/p&gt;&#xA;&lt;p&gt;More specifically, I have a focused crawler that has an input of a few seeds URLs. I want to output as many as possible relevant URLs with the seeds. I model the RL framework as follows.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;State: the webpage,&lt;/li&gt;&#xA;&lt;li&gt;Actions: the outlink URLs of the state webpage,&lt;/li&gt;&#xA;&lt;li&gt;Reward: from external source I know if the webpage content is relevant.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The problem is that, while crawling, if the agent keeps going forward by following the successive state transition, it can fall into crawling traps or local optima. That is the reason why a priority queue is used importantly in crawling. The crawling agent does not follow anymore the successive order of state transitions. Each state-action pair is added to the priority queue with its estimated action value. For each time, it selects the most promising state-action pair among all pairs in the queue. I note that each URL action can be estimated taking into account the state-webpage where it was extracted.&lt;/p&gt;&#xA;" OwnerUserId="36055" LastEditorUserId="2444" LastEditDate="2020-12-31T15:50:38.383" LastActivityDate="2021-01-03T08:38:21.760" Title="How can I model a problem as an MDP if the agent does not follow the successive order of states?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;markov-decision-process&gt;" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="25466" PostTypeId="1" CreationDate="2020-12-30T17:01:40.087" Score="0" ViewCount="15" Body="&lt;p&gt;While a user is navigating through a website, we need to detect whether the user is having trouble finding something in realtime. The output is used to trigger an event that should pop up a FAQ page or a chat assistant automatically which will provide further help to the user.&lt;/p&gt;&#xA;&lt;p&gt;Is there any existing solutions/studies where we can feed in the user clicks, mouse movement, scroll patterns, etc., and the solution detects if the user is having trouble finding something?&lt;/p&gt;&#xA;" OwnerUserId="43389" LastEditorUserId="2444" LastEditDate="2020-12-31T14:03:07.227" LastActivityDate="2020-12-31T14:03:07.227" Title="Is there any solution to the problem of detecting whether a user is having trouble finding something while surfing a webpage?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;ai-design&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25467" PostTypeId="1" AcceptedAnswerId="25470" CreationDate="2020-12-30T17:23:22.403" Score="2" ViewCount="75" Body="&lt;p&gt;In the machine learning literature, I often see it said that something is &amp;quot;embedded&amp;quot; in some space. For instance, that something is &amp;quot;embedded&amp;quot; in feature space, or that our data are &amp;quot;embedded&amp;quot; in dot product space, etc. However, I've never actually seen an explanation of what this is supposed to mean. So what does it actually mean to say that something is &amp;quot;embedded&amp;quot; in some space?&lt;/p&gt;&#xA;" OwnerUserId="16521" LastEditorUserId="2444" LastEditDate="2020-12-30T20:04:56.460" LastActivityDate="2020-12-30T20:44:36.620" Title="In the machine learning literature, what does it mean to say that something is &quot;embedded&quot; in some space?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;embeddings&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25468" PostTypeId="1" CreationDate="2020-12-30T17:26:08.900" Score="0" ViewCount="50" Body="&lt;p&gt;I was experimenting seq2seq model which was the bi-LSTM encoder/decoder with attention. When I compared the training times on GPU vs CPU while varying the batch size, I got&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ktwDM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ktwDM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;CPU on the small batch size (32) is fastest. It's even ~1.5 time&#xA;faster than GPU using the same batch size.&lt;/li&gt;&#xA;&lt;li&gt;When I increase the batch size (upto 2000), GPU becomes faster than CPU due to the&#xA;parallelization.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;(2) looks reasonable to me. However, I am a bit perplexed by the observation (1). The average sequence length is around 15~20. Even with the batch size 32, I expected that GPU should be faster but it turned out not. I used PyTorch LSTM.&lt;/p&gt;&#xA;&lt;p&gt;Does it look normal? In RNN-style seq2seq, could CPU be faster than GPU?&lt;/p&gt;&#xA;" OwnerUserId="43391" LastActivityDate="2020-12-30T17:26:08.900" Title="Training speed in GPU vs CPU for LSTM" Tags="&lt;deep-learning&gt;&lt;recurrent-neural-networks&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25469" PostTypeId="2" ParentId="25457" CreationDate="2020-12-30T20:31:17.857" Score="0" Body="&lt;p&gt;In simple terms, a &lt;strong&gt;neural network model&lt;/strong&gt; is a &lt;em&gt;function approximator&lt;/em&gt; which tries to fit the curve of the hypothesis function. A function itself has an equation which will generate a fixed curve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7acni.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7acni.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If we have the equation (i.e., the function), we do not need neural network for its input data. However, when we only have some notion of its curve (or the input and output data) we seek a function approximator, so that for new, unseen input data, we can generate the output.&lt;/p&gt;&#xA;&lt;p&gt;Training this neural network is all about getting as close an approximation to the original (unknown function) as possible.&lt;/p&gt;&#xA;" OwnerUserId="33781" LastEditorUserId="33781" LastEditDate="2020-12-31T06:48:53.823" LastActivityDate="2020-12-31T06:48:53.823" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25470" PostTypeId="2" ParentId="25467" CreationDate="2020-12-30T20:38:46.457" Score="1" Body="&lt;p&gt;Embedding is the process of representing data (from a source domain) in a new (or target) domain. Usually, the source domain is discrete, and the target domain is continuous. For example, embedding words into the continuous vector space can be done by the word2vec method.&lt;/p&gt;&#xA;&lt;p&gt;The main reason behind using the embedding is doing meaningful mathematical computations in the target domain, which is not possible or straightforward in the source domain. For example, summing two words &lt;code&gt;&amp;quot;brother&amp;quot; - &amp;quot;man&amp;quot; + &amp;quot;woman&amp;quot;&lt;/code&gt; not meaningful in the word and character levels. However, when using word2vec, &lt;code&gt;embedding(&amp;quot;brother&amp;quot;) - embedding(&amp;quot;man&amp;quot;) + embedding(&amp;quot;woman&amp;quot;)&lt;/code&gt; can be meaningful and comparable with other embedded vectors; It should be near the embedded vector of &amp;quot;sister&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="4446" LastEditorUserId="4446" LastEditDate="2020-12-30T20:44:36.620" LastActivityDate="2020-12-30T20:44:36.620" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25471" PostTypeId="2" ParentId="25457" CreationDate="2020-12-30T23:56:43.427" Score="1" Body="&lt;p&gt;Any model can be considered to be a function. The term &amp;quot;model&amp;quot; simply denotes a function being used in a particular way, namely to approximate some other function of interest.&lt;/p&gt;&#xA;" OwnerUserId="43400" LastActivityDate="2020-12-30T23:56:43.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25472" PostTypeId="2" ParentId="25451" CreationDate="2020-12-30T23:58:29.103" Score="2" Body="&lt;p&gt;I looked at the Python pseudo-code attached to the Data S1 of the &lt;a href=&quot;https://science.sciencemag.org/content/suppl/2018/12/05/362.6419.1140.DC1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supplementary Materials&lt;/a&gt; of the AlphaZero paper. Here is my findings:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Contrary to the paper, AlphaZero does not store &lt;span class=&quot;math-container&quot;&gt;$\{N(s, a), W(S, a), Q(s, a), P(s, a)\}$&lt;/span&gt; statistics for each edge &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt;. Instead, AlphaZero stores &lt;span class=&quot;math-container&quot;&gt;$\{N(s), W(S), Q(s), P(s)\}$&lt;/span&gt; statistics for each node &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;When a leaf node &lt;span class=&quot;math-container&quot;&gt;$S_L$&lt;/span&gt; is expanded, it's visit count, value scores, and action policies are immediately updated in &lt;span class=&quot;math-container&quot;&gt;$\{N(s), W(S), Q(s), P(s)\}$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$N(s)$&lt;/span&gt; is at least &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. This is why in the paper, the backprop step updates for all time steps &lt;span class=&quot;math-container&quot;&gt;$t \le L$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$t &amp;lt; L$&lt;/span&gt;. It makes sense to update &lt;span class=&quot;math-container&quot;&gt;$s_L$&lt;/span&gt; even though there is no corresponding &lt;span class=&quot;math-container&quot;&gt;$a_L$&lt;/span&gt; to pair it with.&lt;/li&gt;&#xA;&lt;li&gt;Therefore, when a new leaf node is expanded, the value &lt;span class=&quot;math-container&quot;&gt;$U(s, a)$&lt;/span&gt; of a child of that leaf node will be nonzero, since &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\sum_b N(s,b)}$&lt;/span&gt; is actually computed as &lt;span class=&quot;math-container&quot;&gt;$N(s_{parent})$&lt;/span&gt; in the code, which is at least 1.&lt;/li&gt;&#xA;&lt;li&gt;Oddly enough, I think there might be a bug in the pseudocode, because at the beginning on the first iteration (starting at the root node), &lt;span class=&quot;math-container&quot;&gt;$U(s,a) = 0$&lt;/span&gt; for all child nodes of the root node. This is because at the first iteration, &lt;span class=&quot;math-container&quot;&gt;$N(s_{root}) = 0$&lt;/span&gt;. The value of all child nodes will be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, and since the authors chose to break ties according to Python's &lt;code&gt;max&lt;/code&gt; function, the algorithm simply chooses the first element it finds in case of a tie.&lt;/li&gt;&#xA;&lt;li&gt;After the first iteration, &lt;span class=&quot;math-container&quot;&gt;$N(s_{root}) &amp;gt; 0$&lt;/span&gt; and so &lt;span class=&quot;math-container&quot;&gt;$U(s,a) \neq 0$&lt;/span&gt; and things proceed as normal since the backprop step will have updated the visit count of the root node. So this possible bug/unintuitive behavior only affects the first iteration. It is extremely minor and insignificant, and does not affect the outcome of the MCTS, which is probably why it went unnoticed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-31T00:19:17.020" LastActivityDate="2020-12-31T00:19:17.020" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="25475" PostTypeId="2" ParentId="25152" CreationDate="2020-12-31T06:33:00.633" Score="3" Body="&lt;p&gt;The first two equations are equivalent. The last equation can be equivalent if you scale &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; appropriately.&lt;/p&gt;&#xA;&lt;h2&gt;Equation 1&lt;/h2&gt;&#xA;&lt;p&gt;Consider the equation from the Stanford slide:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{t}=\rho v_{t-1}+\nabla f(x_{t-1}) &#xA;\\ &#xA;x_{t}=x_{t-1}-\alpha v_{t},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's evaluate the first few &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt; so that we can arrive at a closed form solution:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_0 = 0 \\&#xA;v_1 =  \rho v_0 + \nabla f(x_0) = \nabla f(x_0)\\&#xA;v_2 = \rho v_1 + \nabla f(x_1) = \rho \nabla f(x_0) + \nabla f(x_1)\\&#xA;v_3 = \rho v_2 + \nabla f(x_2) = \rho^2 \nabla f(x_0) + \rho \nabla f(x_1) + \nabla f(x_2)\\&#xA;\dots \\&#xA;v_t = \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \nabla f(x_i)&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the closed form update is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_t = x_{t-1} - \alpha \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \nabla f(x_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Equation 2&lt;/h2&gt;&#xA;&lt;p&gt;Now consider the equation from the paper:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{t}=\rho v_{t-1}+\alpha \nabla f(x_{t-1}) &#xA;\\ &#xA;x_{t}=x_{t-1}- v_{t},&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We again evaluate the first few &lt;span class=&quot;math-container&quot;&gt;$v_t$&lt;/span&gt; to arrive at a closed form solution:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_0 = 0 \\&#xA;v_1 =  \rho v_0 + \alpha \nabla f(x_0) = \alpha \nabla f(x_0)\\&#xA;v_2 = \rho v_1 + \alpha \nabla f(x_1) = \rho \alpha \nabla f(x_0) + \alpha \nabla f(x_1)\\&#xA;v_3 = \rho v_2 + \alpha \nabla f(x_2) = \rho^2 \alpha \nabla f(x_0) + \rho \alpha \nabla f(x_1) + \alpha \nabla f(x_2)\\&#xA;\dots \\&#xA;v_t = \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \alpha \nabla f(x_i)&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the closed from update is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_t = x_{t-1} - \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \alpha \nabla f(x_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, this is equivalent to the previous closed form update. The only difference is if &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is inside or outside the summation, but since it is a constant, it doesn't really matter anyways.&lt;/p&gt;&#xA;&lt;h2&gt;Equation 3&lt;/h2&gt;&#xA;&lt;p&gt;As for the last equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v_{t}= \rho v_{t-1}+ (1- \rho) \nabla f(x_{t-1}) &#xA;\\ &#xA;x_{t}=x_{t-1}-\alpha v_{t}&#xA;$$&lt;/span&gt;&#xA;Let's do the same thing:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v_0 = 0 \\&#xA;v_1 =  \rho v_0 + (1-\rho) \nabla f(x_0) = (1-\rho) \nabla f(x_0)\\&#xA;v_2 = \rho v_1 + (1-\rho) \nabla f(x_1) = \rho (1-\rho) \nabla f(x_0) + (1-\rho) \nabla f(x_1)\\&#xA;v_3 = \rho v_2 + (1-\rho) \nabla f(x_2) = \rho^2 (1-\rho) \nabla f(x_0) + \rho (1-\rho) \nabla f(x_1) + (1-\rho) \nabla f(x_2)\\&#xA;\dots \\&#xA;v_t = \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} (1-\rho) \nabla f(x_i)&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And so the closed form update is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_t = x_{t-1} - \alpha \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} (1-\rho) \nabla f(x_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This equation is equivalent to the other two as long as you scale &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; by a factor of &lt;span class=&quot;math-container&quot;&gt;$\displaystyle \frac{1}{1-\rho}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-31T06:33:00.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25476" PostTypeId="2" ParentId="25464" CreationDate="2020-12-31T07:56:30.857" Score="1" Body="&lt;p&gt;Yes, your thought experiment is correct, and the concept is known as &lt;strong&gt;breaking the symmetry&lt;/strong&gt;. This is why biases can be initialized to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (bias initialization doesn't matter), but weights should be randomly initialized to different numbers -- to break the symmetry. Otherwise, if not, the network will function as if it has &lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt; filters (or however many filters that are unique) instead of the full &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; filters.&lt;/p&gt;&#xA;&lt;p&gt;As for your main question, if two filters are initialized to very similar values, they may branch out as long as that is what minimizes the training loss. There is no collusion or coordination going on; each filter updates completely independently. You can even freeze all the other filters and only perform gradient descent on one filter at a time. Each filter just follows the direction of their gradient to minimize the training loss.&lt;/p&gt;&#xA;&lt;p&gt;Consider the backprop equations as defined by &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; online book:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D5xG0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D5xG0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The gradient of the current layer's weights depends on&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The future layers' weights, errors, and activation function's derivatives&lt;/li&gt;&#xA;&lt;li&gt;The current layer's activation function's derivative, and&lt;/li&gt;&#xA;&lt;li&gt;The previous layer's outputs.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Each weight in the layer (i.e. each filter in the layer) looks at different parts of these three components (indexed by &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; in equation &lt;span class=&quot;math-container&quot;&gt;$BP4$&lt;/span&gt;). It is this different perspective that allows them to update their gradients in different directions, even if their initial weights are very similar to each other. Note that it is possible that they end up with the same gradient, but it is very unlikely.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2020-12-31T08:40:11.153" LastActivityDate="2020-12-31T08:40:11.153" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25477" PostTypeId="1" AcceptedAnswerId="25482" CreationDate="2020-12-31T13:21:34.020" Score="1" ViewCount="49" Body="&lt;p&gt;As I've been dabbling into the sliding window concept, I stumbled on a question that asked me to find the number of windows needed on a 1D image of &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; size, knowing the window size &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and the stride &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As much as I tried, I couldn't find a formula by myself (the closest I got was this one : &lt;span class=&quot;math-container&quot;&gt;$N=\frac{W + x(K-S)}{K}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; was the number of overlapping rectangle zones, which seemed to be &lt;span class=&quot;math-container&quot;&gt;$x=N-1$&lt;/span&gt; but the reccurence wasn't what I was looking for and it could be wrong as I was reasoning through induction).&lt;/p&gt;&#xA;&lt;p&gt;I find the right formula on Internet at last (this one : &lt;span class=&quot;math-container&quot;&gt;$N=\frac{W-K+2P}{S}+1$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; the padding but my problem didn't needed one) but I can't find the proof of it.&lt;/p&gt;&#xA;&lt;p&gt;Is there any place where I could find the proof ?&lt;/p&gt;&#xA;" OwnerUserId="43411" LastActivityDate="2020-12-31T16:13:03.287" Title="Output volume proof for convolutional neural network" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25478" PostTypeId="1" CreationDate="2020-12-31T13:39:40.497" Score="2" ViewCount="33" Body="&lt;p&gt;The traditional setting of multiagent reinforcement learning (MARL) is the mode in which there is set of agents and external environment. And the reward is given to each agent - individually or collectively - by the external environment.&lt;/p&gt;&#xA;&lt;p&gt;My question is - is there MARL model in which the reward is given by one agent to the other agent, meaning that one agent is incurring costs and other agent - revenue (or maybe even a profit?&lt;/p&gt;&#xA;&lt;p&gt;Effectively that means distributed supervision: only some agents face the environment with real reward/supervision and then this supervision is more or less effectively propgated to other agents that learn/do their own specialized tasks that are part of collective task ececuted/solved distributively in MARL.&lt;/p&gt;&#xA;" OwnerUserId="8332" LastEditorUserId="8332" LastEditDate="2020-12-31T14:25:07.527" LastActivityDate="2021-01-02T20:09:18.850" Title="Is there multi-agent reinforcement learning model in which (some of the) reward is given by other agent and not by the external environment?" Tags="&lt;reinforcement-learning&gt;&lt;rewards&gt;&lt;multi-agent-systems&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25479" PostTypeId="1" CreationDate="2020-12-31T14:09:50.160" Score="0" ViewCount="12" Body="&lt;p&gt;I'm trying to solve question 30 of &lt;a href=&quot;http://130.243.105.49/%7Etdt/ann/tentor/exam2003B.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&lt;p&gt;The short version of the question is if someone could show me how to do this, I would appreciate it (the answer should be A; -0.0660).&lt;/p&gt;&#xA;&lt;p&gt;The long version of the question is, is my thinking right (to find the error using back-propogation at H2):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The error (from question 29) for I1, I2 and I3 are 0.1479, -0.0929 and 0.1054, respectively.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The network architecture is:&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ia58H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ia58H.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;The weights are:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SxJvw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SxJvw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I thought what I had to do:&lt;/p&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Find the total amount of weights that led to each output error (I took the absolute values to get the total sum of errors, is this right?):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g6djD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g6djD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which gave me:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OxbmX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OxbmX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;So then I worked out the proportion of each weight that came from my node of interest (y2):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Bgxne.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bgxne.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;And then I worked out the proportion of error that came from that proportion of weight:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tStgX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tStgX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If someone could show me where I'm going wrong (because as mentioned above, I know what the right answer should be), I'd appreciate it. Also, as mentioned above, I've added a link to the original question 30 on the original exam, if it helps (it's from 17 years ago, not an exam I'm doing, just trying to understand it).&lt;/p&gt;&#xA;" OwnerUserId="42926" LastActivityDate="2020-12-31T14:09:50.160" Title="How to update the error on hidden nodes using back-propogation, given the error on the output nodes and weights" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;backpropagation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25480" PostTypeId="1" CreationDate="2020-12-31T14:32:16.623" Score="0" ViewCount="22" Body="&lt;p&gt;The following images are&lt;/p&gt;&#xA;&lt;p&gt;a) The weights of a logistic regression model trained on MNIST.&lt;/p&gt;&#xA;&lt;p&gt;b) The sign of the weights of a logistic regression&lt;/p&gt;&#xA;&lt;p&gt;How do these images  represent the weights?&lt;/p&gt;&#xA;&lt;p&gt;Would be grateful for any help.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1412.6572&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source of the research paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kf3tH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kf3tH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35616" LastActivityDate="2020-12-31T14:32:16.623" Title="How do we interpret the images of weights in logistic regression" Tags="&lt;weights&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25481" PostTypeId="1" CreationDate="2020-12-31T14:57:23.990" Score="1" ViewCount="43" Body="&lt;p&gt;I'm well aware of the inner workings of CNN models for object detection, and although I've not worked on a semantic segmentation problem I can imagine how it works.&lt;/p&gt;&#xA;&lt;p&gt;With these types of models, we need to say &amp;quot;segment out the humans&amp;quot;, or &amp;quot;segment out the X&amp;quot;. But what about when I say something like &amp;quot;segment out the subject of this photo, whatever it happens to be&amp;quot;. For example, see this service: &lt;a href=&quot;https://removal.ai/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://removal.ai/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Without too much imagination I might guess that they apply a multiclass segmentation model and just show any foreground pixels, no matter what class they belong to. So we'd hope that the subject is in one of the classes that the model was trained for, and that there are no other class instances in the image that shouldn't be captured. But is there a more general way?&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2020-12-31T15:16:47.960" LastActivityDate="2021-01-04T00:39:25.130" Title="How does general image background removal AI work?" Tags="&lt;computer-vision&gt;&lt;image-segmentation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25482" PostTypeId="2" ParentId="25477" CreationDate="2020-12-31T16:13:03.287" Score="2" Body="&lt;p&gt;You can think about the problem in the following way (without padding, as the padding case is a simple extension of base case with &lt;span class=&quot;math-container&quot;&gt;$\tilde{W}:=W + 2P$&lt;/span&gt;).&lt;br /&gt;&#xA;You want to know how many windows are necessary to cover an image of size &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;, given a window of size &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and stride &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. So your image is a vector with indices &lt;span class=&quot;math-container&quot;&gt;$1, 2\dots, W$&lt;/span&gt;; as you put the first window on the image, the window will cover the indices from &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;. As we apply stride (meaning that we translate the window), we will  get a sequence considering the last covered index. The first element of this sequence is &lt;span class=&quot;math-container&quot;&gt;$i_1=K$&lt;/span&gt;, where the &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; indexing &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is not the number of times we apply stride but the number of covering windows we have. So in this first case applying no stride and we have 1 covering window. Applying stride once to our window, the window will cover the indices form &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$K+S$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$i_2=K+(2 -1 )S$&lt;/span&gt;. In general you get &lt;span class=&quot;math-container&quot;&gt;$i_n=K+(n-1)S$&lt;/span&gt;.&lt;br /&gt;&#xA;Now, if we can exactly cover &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; then there is a number &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$i_N=W$&lt;/span&gt;. This means &lt;span class=&quot;math-container&quot;&gt;$$K+(N-1)S=W,$$&lt;/span&gt; which rearranging gives &lt;span class=&quot;math-container&quot;&gt;$$N=\frac{W-K}{S} + 1.$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="42424" LastActivityDate="2020-12-31T16:13:03.287" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25483" PostTypeId="1" AcceptedAnswerId="25517" CreationDate="2020-12-31T16:22:20.577" Score="0" ViewCount="228" Body="&lt;p&gt;I try to apply Transformers to an unusual use case - predict the next user session based on the previous one. A user session is described by a list of events per second, e.g. whether the user watches a particular video, clicks a specific button, etc. Typical sessions are around 20-30 seconds, I pad them to 45 seconds. Here's a visual example of 2 subsequent sessions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ndcOw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ndcOw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;x&lt;/code&gt; axis is time in seconds, &lt;code&gt;y&lt;/code&gt; axis is the list of events (black line divides the 2 sessions). I extend the vocabulary with 2 additional tokens - start and end of a session (&lt;code&gt;&amp;lt;sos&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;eos&amp;gt;&lt;/code&gt;), where &lt;code&gt;&amp;lt;sos&amp;gt;&lt;/code&gt; is a one-hot vector at the very beginning and &lt;code&gt;&amp;lt;eos&amp;gt;&lt;/code&gt; - similar vector at the end of the session (which makes this long red line).&lt;/p&gt;&#xA;&lt;p&gt;Now I use these extended vectors of events as &lt;em&gt;embeddings&lt;/em&gt; and want to train a Transformer model to predict the next events in the current session based on previous events in this (target) session and all events in the previous (source) session. So pretty much like seq2seq autoregressive models, but in a bit unusual settings.&lt;/p&gt;&#xA;&lt;p&gt;Here's the problem. When I train a Transformer using the built-in PyTorch components and square subsequent mask for the target, my generated (&lt;em&gt;during training&lt;/em&gt;) output is too good to be true:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hdCym.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hdCym.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Although there's some noise, many event vectors in the output are modeled exactly as in the target. After checking train-val-test split is correct, my best guess is that the model cheats by attending to the same day in the target, which the mask should have prevented. The mask is (5x5 version for brevity):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[0., -inf, -inf, -inf, -inf],&#xA; [0., -inf, -inf, -inf, -inf],&#xA; [0., 0., -inf, -inf, -inf],&#xA; [0., 0., 0., -inf, -inf],&#xA; [0., 0., 0., 0., -inf]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that since I use &lt;code&gt;&amp;lt;sos&amp;gt;&lt;/code&gt; in both - source and target - &lt;code&gt;mask[i, i]&lt;/code&gt; is set to &lt;code&gt;-inf&lt;/code&gt; (except for &lt;code&gt;mask[0, 0]&lt;/code&gt; for numerical reasons), so the output timestamp &lt;code&gt;i&lt;/code&gt; should not attend to the target timestamp &lt;code&gt;i&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The code for the model's &lt;code&gt;forward&lt;/code&gt; method:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def forward(self, src, tgt):&#xA;    memory = self.encoder(src)&#xA;    out = self.decoder(tgt, memory, self.tgt_mask.type_as(tgt))&#xA;    out = torch.sigmoid(out)&#xA;    return out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried to avoid the target mask altogether and set it to all &lt;code&gt;-inf&lt;/code&gt; (again, except for the first column for numerical stability), but the result is always the same.&lt;/p&gt;&#xA;&lt;p&gt;Am I using the mask the wrong way? If the mask looks fine, what other reasons could lead to such a &amp;quot;perfect&amp;quot; result?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;After shifting the target to the right as suggested in the accepted answer I get the following result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iXT00.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iXT00.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which is much more realistic. One suspicious thing is that &lt;code&gt;out[t]&lt;/code&gt; now resembles &lt;code&gt;tgt[t - 1]&lt;/code&gt;, but it can be explained by the fact that the user state tends to be &amp;quot;sticky&amp;quot;, e.g. if a user watches a video at &lt;code&gt;t - 1&lt;/code&gt;, most likely he will watch it at &lt;code&gt;t&lt;/code&gt; as well.&lt;/p&gt;&#xA;" OwnerUserId="43413" LastEditorUserId="43413" LastEditDate="2021-01-03T22:43:59.540" LastActivityDate="2021-01-04T02:03:31.007" Title="Transformers: How to use the target mask properly?" Tags="&lt;pytorch&gt;&lt;transformer&gt;" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="25484" PostTypeId="1" CreationDate="2020-12-31T17:19:28.050" Score="1" ViewCount="14" Body="&lt;p&gt;I've come across two types of image classification tasks&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;cat/dog classification&lt;/strong&gt; the whole picture is either a cat or a dog. Simple.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;this image contains a cat classification&lt;/strong&gt;. There's a whole chaotic scene, and the image may contain a cat nestled in there somewhere.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Type 2 seems to be way more prevalent in real life application. Here are just some examples:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Determining the sex of an insect. Maybe the male and female look pretty much the same, but the male has a small bump in some location that takes up a tiny part of the image.&lt;/li&gt;&#xA;&lt;li&gt;Determining the presence of an animal call in an audio spectrogram.&lt;/li&gt;&#xA;&lt;li&gt;Finding defects in road surfaces.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My question is: for task type 2, what are some key modifications we'd make to the normal approach of post-training on an ImageNet trained architecture like Resnet? Shouldn't the architecture be modified somehow to be more fitted to task type 2?&lt;/p&gt;&#xA;&lt;p&gt;Before someone mentions using object detection algorithms I'd like to add the rule that we only have global image labels, not bounding box annotation or co-ordinates of any sort.&lt;/p&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2020-12-31T17:19:28.050" Title="Considerations when doing image classification where the object is not the subject" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25485" PostTypeId="1" CreationDate="2020-12-31T17:30:07.900" Score="1" ViewCount="77" Body="&lt;p&gt;I have implemented an AI agent to play checkers based on the design written in the first chapter of &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0070428077&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&#xA;Machine Learning, Tom Mitchell, McGraw Hill, 1997.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We train the agent by letting it plays against its self.&lt;/p&gt;&#xA;&lt;p&gt;I wrote the prediction to get how good a board is for white, so when the white plays he must choose the next board with the maximum value, and when black plays it must choose the next board with the minimum value.&lt;/p&gt;&#xA;&lt;p&gt;Also, I let the agent explore other states by making it choose a random board among the valid next boards, I let that probability to be equal to &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The final boards will have training values:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;100 if this final board is &lt;code&gt;win&lt;/code&gt; for white.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-100 if this final board is &lt;code&gt;lose&lt;/code&gt; for white.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0 if this final board is &lt;code&gt;draw&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The intermediate boards will have a training value equal to the prediction of the next board where it is white turn.&lt;/p&gt;&#xA;&lt;p&gt;The model is based on a linear combination of some features (see the book for full description).&lt;/p&gt;&#xA;&lt;p&gt;I start by initializing the parameters of the model to random values.&lt;/p&gt;&#xA;&lt;p&gt;When I train the agent, he lost against himself always or draw in a stupid way, but the error converges to zero.&lt;/p&gt;&#xA;&lt;p&gt;I was thinking that maybe we should let the learning rate smaller (like 1e-5), and when I do that the agent learns in a better way.&lt;/p&gt;&#xA;&lt;p&gt;I think that this happened because of the credit assignment problem, because a good move may appear in a loose game, therefore, considered a loose move, so white will never choose it when he plays, but when we let the learning rate to be very small that existence of a good move in a losing game will change its value by a very little amount, and that good move should appear more in win games so its value converges to the right value.&lt;/p&gt;&#xA;&lt;p&gt;Is my reasoning correct? and if not so what is happened?&lt;/p&gt;&#xA;" OwnerUserId="36578" LastEditorUserId="36578" LastEditDate="2021-01-02T09:48:31.693" LastActivityDate="2021-01-02T09:48:31.693" Title="learning rate and credit assignment problem in checkers" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;&lt;checkers&gt;&lt;credit-assignment-problem&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25486" PostTypeId="1" CreationDate="2020-12-31T17:43:53.163" Score="0" ViewCount="64" Body="&lt;p&gt;I want to do alpha-beta pruning on this tree:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider nodes J and K. K is the max. Therefore, node D has an alpha value of 20, node B has a beta value of 20.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Move to Node E. Pass the beta value of 20 to node E. Node L has an alpha value of 30, therefore, at this point 30 (alpha) &amp;gt; 20 (beta) and we can prune the E to M branch.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now is my question. My original beta value at node B was 20, and the alpha value passed up to node A was 20. Then, in step 2, I changed the alpha value to be 30. Do i then change the beta value at node B to be 30, and the alpha value at node A to be 30? (and therefore pass 30 as the alpha value to node C)? Or do I keep the original value of 20 at node B, A and C?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XpHg5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XpHg5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42926" LastActivityDate="2021-02-05T15:05:30.397" Title="Does the alpha/beta value of parent nodes change if the alpha beta value of the child node changes?" Tags="&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25487" PostTypeId="2" ParentId="25464" CreationDate="2020-12-31T17:45:21.243" Score="0" Body="&lt;p&gt;Here I am just trying to simplify what @user3667125 already said uses math arguments&lt;/p&gt;&#xA;&lt;p&gt;Say we have a cost function &lt;span class=&quot;math-container&quot;&gt;$J(x, y; F(\cdot; \Theta))$&lt;/span&gt; which regards training a NN &lt;span class=&quot;math-container&quot;&gt;$F(\cdot, \Theta)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; input and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; expected output&lt;/p&gt;&#xA;&lt;p&gt;Gradient descent tells us how to upgrade each &lt;span class=&quot;math-container&quot;&gt;$\theta_{i} \in \Theta$&lt;/span&gt; and it is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \theta_{i}(t+1) = \theta_{i}(t) - \alpha \frac{\partial J(x,y; \Theta)}{\partial \theta_{i}} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; training time&lt;/p&gt;&#xA;&lt;p&gt;So let's focus on a specific component of the NN &lt;span class=&quot;math-container&quot;&gt;$f(x; \theta)$&lt;/span&gt; and from its perspective, we can say the computation is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ h(f(g(x); \theta), y) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h(x,y)$&lt;/span&gt; representing all the subsequent components + loss function&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; representing all the previous computation&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; input&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; expected output&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;so its update is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Delta \theta(t) = \frac{\partial h(f(g(x(t), \theta), y(t)))}{\partial \theta} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x(t)$&lt;/span&gt; the concrete input at &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; time&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y(t)$&lt;/span&gt; expected output at &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; time&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta(t)$&lt;/span&gt; the concrete value of the parameter at &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Applying the chain rule we have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Delta \theta(t) = h'(f(g(x(t), \theta(t))), y(t)) f'(g(x(t)), \theta(t)) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;so the gradient observed by a certain parameter &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; depends on&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta(t)$&lt;/span&gt; the current value of the parameter&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x(t)$&lt;/span&gt; the current input and more specifically by &lt;span class=&quot;math-container&quot;&gt;$g(x(t))$&lt;/span&gt; the processing of this input by the previous part of the NN&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y(t)$&lt;/span&gt; the expected output&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$h'(\cdot)$&lt;/span&gt; the gradient of the subsequent part of the network&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So even if we have 2 weights with the same value &lt;span class=&quot;math-container&quot;&gt;$\theta_{i}(t) = \theta_{j}(t) \quad i \neq j$&lt;/span&gt; at a certain point in training time, they can still see different gradients since&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the upstream processing &lt;span class=&quot;math-container&quot;&gt;$g(x(t))$&lt;/span&gt; can be different&lt;/li&gt;&#xA;&lt;li&gt;the gradient backpropagating from the downstream processing &lt;span class=&quot;math-container&quot;&gt;$h'(\cdot, y(t))$&lt;/span&gt; can be different&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="1963" LastEditDate="2021-01-01T07:30:58.970" LastActivityDate="2021-01-01T07:30:58.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25488" PostTypeId="2" ParentId="25463" CreationDate="2020-12-31T18:58:29.630" Score="1" Body="&lt;p&gt;The paper &lt;a href=&quot;https://arxiv.org/pdf/1710.05381.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A systematic study of the class imbalance problem&#xA;in convolutional neural networks&lt;/a&gt; is a great overview on class imbalance approaches. Section 2 summarizes various methods commonly used. They categorize &amp;quot;Adding Class Weights for an imbalanced dataset&amp;quot; under the technique &amp;quot;Cost sensitive learning&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cost sensitive learning.&lt;/strong&gt; This method assigns different cost to misclassification of examples from different classes [44]. With respect to neural networks it can be implemented in various ways. One approach is threshold moving [22] or post scaling [23] that is applied in the inference phase after the classifier is already trained. Similar strategy is to adapt the output of the network and also use it in the backward pass of backpropagation algorithm [45]. Another adaptation of neural network to be cost sensitive is to modify the learning rate such that higher cost examples contribute more to the update of weights. And finally we can train the network by minimizing the misclassification cost instead of standard loss function&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Without further context, &amp;quot;Adding Class Weights for an imbalanced dataset&amp;quot; can mean many things as enumerated by the above. But if I had to guess, the most common meaning is that they weigh the misclassification cost differently per label by multiplying it with a different weight variable.&lt;/p&gt;&#xA;&lt;p&gt;For example, maybe label 1, 2, and 3, when misclassified, gets a 1x multiplier in the training loss (the standard weight), but label 4 gets a 3x multiplier because it is roughly 3x more important, and label 5 gets a 0.5x multiplier because it is 0.5x less important.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://datascience.stackexchange.com/questions/13490/how-to-set-class-weights-for-imbalanced-classes-in-keras&quot;&gt;Here&lt;/a&gt; is an example of how to do this with Keras.&lt;/p&gt;&#xA;&lt;p&gt;(FYI the paper above recommends you over-sample the minority class rather than using the cost-sensitive learning approach to help with class imbalance)&lt;/p&gt;&#xA;&lt;p&gt;As for missing labels, the quickest solution is just to skip those training instances when training for that particular missing label. If you want to still use the training instances somehow, a common approach is to fix the missing label with &lt;a href=&quot;http://www.cs.cmu.edu/%7E10701/slides/17_SSL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;semi-supervised learning approaches&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One way to do semi-supervised learning is &lt;strong&gt;self-training&lt;/strong&gt;, where your model is trained on only the labeled instances, and then makes predictions on the missing labels. High-confidence predictions of the missing labels are then added to the training data, and the model is trained on the new training data. This process repeats until convergence of the training data.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastActivityDate="2020-12-31T18:58:29.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25489" PostTypeId="2" ParentId="25457" CreationDate="2020-12-31T19:05:46.010" Score="0" Body="&lt;p&gt;Every model is a function. Not every function is a model.&lt;/p&gt;&#xA;&lt;p&gt;A function uniquely maps elements of some set to elements of another set, possibly the same set.&lt;/p&gt;&#xA;&lt;p&gt;Every AI model is a function because they are implemented as computer programs and every computer program is a function uniquely mapping the combination of the sequence of bits in memory and storage at program start up, plus inputs, to the sequence of bits in memory and storage, plus output, at program termination.&lt;/p&gt;&#xA;&lt;p&gt;However, a 'model' is very specifically a representation of something. Take the logistic curve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x) = \frac{L}{1 + e^{k(x-x_{0})} }&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given arbitrary real values for &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$x_{0}$&lt;/span&gt;, that's a function. However, given much more specific values learned from data, it can be a model of population growth.&lt;/p&gt;&#xA;&lt;p&gt;Similarly, a neural network with weights initialized to all zeros is a function, but a very uninteresting function with the rather limited codomain &lt;span class=&quot;math-container&quot;&gt;$\{0\}$&lt;/span&gt;. However, if you then train the network by feeding it a bunch of data until the weights converge to give predictions or actions roughly corresponding to some real world generating process, now you have a model of that generating process.&lt;/p&gt;&#xA;" OwnerUserId="43397" LastActivityDate="2020-12-31T19:05:46.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25490" PostTypeId="2" ParentId="2452" CreationDate="2020-12-31T19:24:16.097" Score="0" Body="&lt;p&gt;What you're describing is a CRUD app &lt;em&gt;plus&lt;/em&gt; a &lt;a href=&quot;https://en.wikipedia.org/wiki/Recommender_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommender system&lt;/a&gt;. A CRUD app on its own doesn't perform any similarity ranking or recommendation functions. Stack Exchange is doing at least keyword matching and possibly semantic parsing as a feature layer on top of the basic CRUD functions.&lt;/p&gt;&#xA;" OwnerUserId="43397" LastActivityDate="2020-12-31T19:24:16.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25491" PostTypeId="1" AcceptedAnswerId="25497" CreationDate="2020-12-31T21:22:05.033" Score="3" ViewCount="141" Body="&lt;p&gt;I was pondering on loss function of GAN and following thing turned out&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA; L(D, G) &#xA;&amp;amp; = \mathbb{E}_{x \sim p_{r}(x)} [\log D(x)] + \mathbb{E}_{x \sim p_g(x)} [\log(1 - D(x)] \\&#xA;&amp;amp; = \int_x \bigg( p_{r}(x) \log(D(x)) + p_g (x) \log(1 - D(x)) \bigg) dx \\ &#xA;&amp;amp; =-\left[CE(p_r(x), D(x))+CE(p_g(x), 1-D(x)) \right] \\&#xA;\end{aligned}&lt;/span&gt;&#xA;Where CE stands for cross-entopy. Then, by using law of large numbers:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{aligned}&#xA;L(D, G) &#xA;&amp;amp; = \mathbb{E}_{x \sim p_{r}(x)} [\log D(x)] + \mathbb{E}_{x \sim p_g(x)} [\log(1 - D(x)] \\&#xA;&amp;amp; =\lim_{m\to \infty}\frac{1}{m}\sum_{i=1}^{m}\left[1\cdot \log(D(x^{(i)}))+1\cdot \log(1-D(x^{(i)}))\right]\\&#xA;&amp;amp; =- \lim_{m \to \infty} \frac{1}{m}\sum_{i=1}^{m} \left[CE(1, D(x))+CE(0, D(x))\right]&#xA;\end{aligned}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, I got very strange result.This should be wrong intuitively because in last equation first part is for real samples, and second is for generated samples. However, I am curious about where is the mistakes?&#xA;(Please explain with math)&#xA;(Also, I am aware of &lt;a href=&quot;https://ai.stackexchange.com/questions/11793/understanding-notation-of-goodfellows-gan-objective-function&quot;&gt;this&lt;/a&gt; post and it explained nothing )&lt;/p&gt;&#xA;" OwnerUserId="41615" LastEditorUserId="41615" LastEditDate="2021-01-01T19:19:06.820" LastActivityDate="2021-01-03T23:23:42.923" Title="Mathematical Analysis of the Loss function of GAN" Tags="&lt;deep-learning&gt;&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;cross-entropy&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25492" PostTypeId="2" ParentId="14112" CreationDate="2020-12-31T22:44:50.197" Score="1" Body="&lt;p&gt;The difference is much simpler than you might have anticipated: In the quantum computing community, machine learning algorithms designed to be used on quantum computers as opposed to classical computers, would fall under &amp;quot;quantum machine learning&amp;quot;. There's really nothing more to it!&lt;/p&gt;&#xA;&lt;p&gt;There is a short paper published in Nature called &amp;quot;Quantum Machine Learning&amp;quot; which was &lt;a href=&quot;https://quantumcomputing.stackexchange.com/a/2152/2293&quot;&gt;mentioned before&lt;/a&gt;, and it might give you all the answers you need about what &amp;quot;Quantum Machine Learning&amp;quot; is.&lt;/p&gt;&#xA;" OwnerUserId="19524" LastActivityDate="2020-12-31T22:44:50.197" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25493" PostTypeId="2" ParentId="25491" CreationDate="2020-12-31T23:35:21.190" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\textbf{Remark.}$&lt;/span&gt; I'd leave this as a comment if I could.&lt;/p&gt;&#xA;&lt;p&gt;Regarding notation (which I believe may be the cause of your issue here), the loss function is better written as&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;\operatorname{Loss} &amp;amp;= \frac{1}{m}\sum_{i=1}^m \left(\log D\big(x^{(i)}\big) + \log\Big(1-D\big(G\big(z^{(i)}\big)\big)\right)\\&#xA;&amp;amp;\approx \mathbb{E}_x[\log D(x)] + \mathbb{E}_z[\log(1-D(G(z)))],&#xA;\end{align*}&lt;/span&gt;&#xA;where the noise vectors, &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, come from a suitable distribution, and &lt;span class=&quot;math-container&quot;&gt;$G(z)$&lt;/span&gt; denotes the output of the generator; the &lt;span class=&quot;math-container&quot;&gt;$\approx$&lt;/span&gt; symbol here implicitly assumes that the appropriate form of the Law of Large Numbers (LLN) applies.&lt;/p&gt;&#xA;&lt;p&gt;Most importantly, the dependence on G is not trivial (for instance, what if &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; never learns and always produces the same output?).&lt;/p&gt;&#xA;&lt;p&gt;Also, the expectations should depend on their respective distributions, even when using LLN. For example, think of how you calculate the expectation of a discrete random variable.&lt;/p&gt;&#xA;" OwnerUserId="43417" LastActivityDate="2020-12-31T23:35:21.190" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25495" PostTypeId="2" ParentId="20656" CreationDate="2021-01-01T05:57:36.803" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Generally, if one googles &amp;quot;quantum machine learning&amp;quot; or anything similar the general gist of the results is that quantum computing will greatly speed up the learning process of our &amp;quot;classical&amp;quot; machine learning algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is correct. A lot of machine learning methods involve linear algebra, and it often takes far fewer quantum operations to do things in linear algebra than the number of classical operations that would be needed. To be more specific, for a matrix of size &lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt;, if a classical computer needs &lt;span class=&quot;math-container&quot;&gt;$f(N)$&lt;/span&gt; operations to do some linear algebra operation, such as diagonalization which can take &lt;span class=&quot;math-container&quot;&gt;$f(N)=\mathcal{O}(N^3)$&lt;/span&gt; operations on a classical computer, a quantum computer would often need only &lt;span class=&quot;math-container&quot;&gt;$\log_2 f(N)$&lt;/span&gt; operations, which inn (in and only in) the language of computational complexity theory, means &lt;em&gt;&lt;strong&gt;exponential speed-up&lt;/strong&gt;&lt;/em&gt;. The &amp;quot;only in&amp;quot; part is there because we have made here an assumption that &amp;quot;fewer quantum operations&amp;quot; means &amp;quot;speed-up&amp;quot;, which for now is something we only know to be true in the world of &amp;quot;computational complexity theory&amp;quot;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;However, &amp;quot;speed-up&amp;quot; itself does not seem very appealing to me as the current leaps made in AI/ML are generally due to novel architectures or methods, not faster training.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I disagree. Take vanilla deep learning for example (without GANs or any of the other things that came up in the last decade). Hinton and Bengio had been working on deep learning for decades, so why did interest in deep learning &lt;em&gt;suddenly&lt;/em&gt; start growing so much from 2011-2014 after a roughly monotonic curve from 1988-2010? Not that this rise started before newer advances such as GANs and &lt;a href=&quot;https://arxiv.org/abs/1608.06993&quot; rel=&quot;nofollow noreferrer&quot;&gt;DenseNet&lt;/a&gt; were developed:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kgC7s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kgC7s.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;br&gt;&lt;br&gt;&#xA;Notice also the similarity between the above graph and these ones:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xf4k0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xf4k0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;These days pretty much everyone doing deep learning uses GPUs if they have access to GPUs, and what is possible to accomplish is &lt;em&gt;&lt;strong&gt;extremely&lt;/strong&gt;&lt;/em&gt; tied to what computing power a group has. I don't want to undermine the importance of new &lt;em&gt;methods&lt;/em&gt; and new &lt;em&gt;algorithms,&lt;/em&gt; but GPUs did play a big role for at least some areas of machine learning, such as deep learning.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are there any quantum machine learning methods in development that are fundamentally different from &amp;quot;classical&amp;quot; methods?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think you mean: &amp;quot;Most quantum machine learning algorithms are simply based on classical machine learning algorithms, but with some sub-routine sped-up by the QPU instead of a GPU -- are there any quantum algorithms that are not based on classical machine learning algorithms, and are entirely different&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The answer is yes, and more experts might be able to tell you more &lt;a href=&quot;https://quantumcomputing.stackexchange.com/questions/ask&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One thing you might consider looking at is &lt;a href=&quot;https://arxiv.org/abs/1601.02036&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Boltzmann Machines&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another things I'll mention is that a child prodigy named &lt;a href=&quot;https://en.wikipedia.org/wiki/Ewin_Tang&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ewin Tang&lt;/a&gt; who began university at age 14, discovered at around the age of 17 some &lt;em&gt;&lt;strong&gt;classical algorithms&lt;/strong&gt;&lt;/em&gt; that were inspired by quantum algorithms rather than the other way around, and&#xA;the comments on the Stack Exchange question &lt;a href=&quot;https://quantumcomputing.stackexchange.com/q/5525/2293&quot;&gt;Quantum machine learning after Ewin Tang&lt;/a&gt; might give you more insight on that. This is related to something called &lt;em&gt;&lt;strong&gt;dequantization&lt;/strong&gt;&lt;/em&gt; of quantum algorithms.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;By this I mean that these methods are (almost*) impossible to perform on &amp;quot;classical&amp;quot; computers. *except for simulation of the quantum computer of course&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Unfortunately quantum computers can't do anything that's impossible for classical computers to do, apart from the fact that they might be able to do some things faster. &lt;em&gt;&lt;strong&gt;Classical computation is Turing complete, meaning anything that can be computed can be computed on a big enough classical computer.&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="19524" LastActivityDate="2021-01-01T05:57:36.803" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25496" PostTypeId="2" ParentId="25437" CreationDate="2021-01-01T08:52:28.617" Score="0" Body="&lt;p&gt;input to output is linear refers to the input X i.e image and output is the output logits/softmax from the network.&lt;/p&gt;&#xA;&lt;p&gt;So how does linearity help in constructing adversarial examples? Imagine a simple logistic regressor and a simple 2D space. There is a definite boundary beyond which the label that the model(i.e logisitc regressor in this case) changes. So if we move perpendicular to the boundary (i.e the line represented by the model in this case) we can get to another class's space. So if we perturb the input in this direction, the model outputs wrong class. { Refer the slide with the title Adversarial Examples from Excessive Linearity for diagram }&lt;/p&gt;&#xA;&lt;p&gt;Now imagine the neural network trained on imagenet, there are so many boundaries and a small change can just shift change the class the model would predict. Now it is important to note that these subspaces of the image remain nearly the same if we train VGG or ResNet etc. So this explains how an adversarial example on one network effects another.&lt;/p&gt;&#xA;&lt;p&gt;You may ask how such small change can effect. this is because the vectors we deal it is not 2d or 3d, it is very large and small changes add up.&lt;/p&gt;&#xA;" OwnerUserId="35616" LastActivityDate="2021-01-01T08:52:28.617" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25497" PostTypeId="2" ParentId="25491" CreationDate="2021-01-01T09:44:58.490" Score="4" Body="&lt;p&gt;I guess the issue is you lost track of where the samples came from and since you requested a math explanation I'll try to go step by step using my notation and without checking other material to avoid being biased by how other authors present it&lt;/p&gt;&#xA;&lt;p&gt;So we start from&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ L(D,G) = E_{x \sim p_{r}(x)} \log(D(x)) + E_{x \sim p_{g}(x)}\log(1 - D(x)) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;then you apply the definition of &lt;span class=&quot;math-container&quot;&gt;$E_{\cdot}(\cdot)$&lt;/span&gt; operator in the continuous case&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ L(D,G) = \int_{x} \log(D(x)) p_{r}(x)dx + \int_{x}\log(1 - D(x))p_{g}(x)dx $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;then you Monte Carlo sample it to approximate it&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ L(D,G) = \frac{1}{n} \sum_{i=1}^{n} \log(D(x_{i}^{(r)}))  + \frac{1}{m} \sum_{j=1}^{m}\log(1 - D(x_{j}^{(g)})) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see here I have kept the samples from the 2 distributions separated and used a notation that allows to track their origin so now you can use the right label in the Cross Entropy&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ L(D,G) = \frac{1}{n} \sum_{i=1}^{n} L_{ce}(1, D(x_{i}^{(r)}))  + \frac{1}{m} \sum_{j=1}^{m} L_{ce}(0, D(x_{j}^{(g)})) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But you could also have decided to merge the 2 integrals before to have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ L(D,G) = \int_{x} \left( \log(D(x)) p_{r}(x) + \log(1 - D(x))p_{g}(x) \right) dx $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is mathematically legit operation, however the issue is when you try to&#xA;discretize this with Monte Carlo sampling.&lt;/p&gt;&#xA;&lt;p&gt;You can't just replace the integral with one sum since you are Monte Carlo sampling and here, contrary to what we have done above, you do not have 1 distribution per integral to sample but in the same integral you have 2 distributions and for each sample you have to say what distribution it comes from which is where the issue is in your notation since you lost track of this information and it seems all the samples come from one distribution&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="41615" LastEditDate="2021-01-03T23:23:42.923" LastActivityDate="2021-01-03T23:23:42.923" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25498" PostTypeId="1" AcceptedAnswerId="25515" CreationDate="2021-01-01T10:38:11.200" Score="1" ViewCount="53" Body="&lt;p&gt;I am new to NLP and AI in general. I am just expecting springboard information so that I can skip all the introduction to NLP websites.  I have just started studying NLP and want to know how to go about solving this problem. I am creating a chatbot that will take voice input from customers ordering food at restaurants. The customer input I am expecting as;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I want to order Chicken Biryani&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can I have a Veg Pizza, please&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Coca-cola  etc&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I want to write an algorithm that can separate the name of the food item from the user input and compare it with the list of food items in my menu card and come up with the right item.&lt;/p&gt;&#xA;&lt;p&gt;I am new to NLP, I am studying it online for this particular project, I can do the required coding, I just need help with the overall algo or sort of flow chart. It will save my time tremendously. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="43434" LastEditorUserId="32410" LastEditDate="2021-04-24T03:34:05.033" LastActivityDate="2021-04-24T03:34:05.033" Title="How to design a NLP algorithm to find a food item in menu card list?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;algorithm&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25499" PostTypeId="1" AcceptedAnswerId="25504" CreationDate="2021-01-01T13:13:22.833" Score="2" ViewCount="56" Body="&lt;p&gt;I was reading the book &lt;a href=&quot;https://raw.githubusercontent.com/janishar/mit-deep-learning-book-pdf/master/complete-book-pdf/Ian%20Goodfellow%2C%20Yoshua%20Bengio%2C%20Aaron%20Courville%20-%20Deep%20Learning%20(2017%2C%20MIT).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; by Ian Goodfellow. I had a doubt in the Maximum likelihood estimation section (Pg 131). I understand till the Eq 5.58 which describes what is being maximized in the problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA; \theta_{\text{ML}} = \text{argmax}_{\theta} \sum_1^m \log(p_{\text{model}}(x^{(i)};\theta)) &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However the next equation 5.59 restates this equation as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\theta_{\text{ML}} = \text{argmax}_{\theta}  E_{x \sim \hat{p}_{\text{data}}}(\log(p_{\text{model}}(x;\theta)) &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$$\hat{p}_{\text{data}}$$&lt;/span&gt; is described as the empirical distribution defined by the training data.&#xA;Could someone explain what is meant by this empirical distribution? It seems to be different from the distribution parametrized by theta as that is described by &lt;span class=&quot;math-container&quot;&gt;$$ p_{\text{model}} $$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="43272" LastEditorUserId="16521" LastEditDate="2021-01-01T19:12:13.837" LastActivityDate="2021-01-02T23:17:07.963" Title="What is emperical distribution in MLE?" Tags="&lt;probability-distribution&gt;&lt;maximum-likelihood&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25500" PostTypeId="1" AcceptedAnswerId="25501" CreationDate="2021-01-01T15:00:14.450" Score="0" ViewCount="37" Body="&lt;p&gt;This is the back-propogation rule for the output layer of a multi-layer network:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W_{jk} := W_{jk} - C \dfrac{\delta E}{\delta W_{jk}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What does this rule do in the more ambiguous cases such as:&lt;/p&gt;&#xA;&lt;p&gt;(1) The output of a hidden node is near the middle of a sigmoid curve?&lt;/p&gt;&#xA;&lt;p&gt;(2) The graph of error with respect to weight is near a maximum or minimum?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="16521" LastEditDate="2021-01-01T15:52:15.980" LastActivityDate="2021-01-01T16:08:16.507" Title="Backpropogation rule for the output layer of a multi-layer network - What does the rule do in ambiguous cases?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25501" PostTypeId="2" ParentId="25500" CreationDate="2021-01-01T16:02:19.593" Score="1" Body="&lt;p&gt;I assume you are considering a network where the activation function of the last layer is a sigmoid, so the output of your network is &lt;span class=&quot;math-container&quot;&gt;$$\tilde{y}=\sigma(W^{L}\cdot f(X, W^1, \dots, W^{L-1})),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is the input vector, and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is obtained by feeding the input to the network up to the layer &lt;span class=&quot;math-container&quot;&gt;$L-1$&lt;/span&gt;. Let's also call &lt;span class=&quot;math-container&quot;&gt;$Z:= W^{L}\cdot f(X, W^1, \dots, W^{L-1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The error term is computed as &lt;span class=&quot;math-container&quot;&gt;$$E(y, \tilde{y})=E(y, \sigma(Z)),$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the actual output. Let's get the derivative of the error with respect to the output of the last node (the input of the sigmoid)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial E}{z_i}=\frac{\partial E}{\partial \tilde{y}}\frac{\partial\tilde{y}}{\partial z_i}=\frac{\partial E}{\partial \tilde{y}}\frac{\partial\sigma}{\partial z_i}.$$&lt;/span&gt;&#xA;The update rule is &lt;span class=&quot;math-container&quot;&gt;$$z_i =z_i - C\frac{\partial E}{z_i}= z_i - C\frac{\partial E}{\partial \tilde{y}}\frac{\partial\sigma}{\partial z_i}.$$&lt;/span&gt;&#xA;Now we can analyse your questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To be close to the middle of the sigmoid means that &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; is close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;; moreover the derivative &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial\sigma}{\partial z_i}$&lt;/span&gt; reaches its maximum value when it is evaluated in &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. This means that the term &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial\sigma}{\partial z_i}$&lt;/span&gt; gets &amp;quot;large&amp;quot; as &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; approaches the center of the sigmoid, contributing more to the update of the weight. Of course it is difficult to say what happens in general, as the term &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial \tilde{y}}$&lt;/span&gt; is also in the expression and it is possible that this term gets really small (or big) for &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. You just know that the term &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial\sigma}{\partial z_i}$&lt;/span&gt; is trying to push &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; further away from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, so the idea should be that the closer to the center the larger the update.&lt;/li&gt;&#xA;&lt;li&gt;To be close to an extremum point of the loss means that the derivative of the loss with respect to &lt;span class=&quot;math-container&quot;&gt;$\tilde{y}$&lt;/span&gt; is close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Since &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial \tilde{y}}$&lt;/span&gt; is backpropagated in a multiplicative fashion, the rule of thumb is that the closer you are to an extremum the smaller the updates get. Though, as above, is kind of difficult to say what will happen when updating a general node, as some of the term in the multiplication can be very large, making the update large even if &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial \tilde{y}}$&lt;/span&gt; is small.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For instance, what happens if you are close to the center of the sigmoid but also close to an extremum of the loss? You will have a multiplication of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; terms, one trying to make the update small and the other trying to make the update large, and what matter are the orders of magnitude involved.&lt;br /&gt;&#xA;In conclusion, the rules of thumb are as in the points 1. and 2., but they are no guarantee that you won't find any special cases.&lt;/p&gt;&#xA;" OwnerUserId="42424" LastEditorUserId="42424" LastEditDate="2021-01-01T16:08:16.507" LastActivityDate="2021-01-01T16:08:16.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25502" PostTypeId="2" ParentId="22514" CreationDate="2021-01-01T16:47:56.357" Score="0" Body="&lt;p&gt;There's nothing stopping you from training a model with whatever tags you want.&lt;/p&gt;&#xA;&lt;p&gt;Using what you describe as &amp;quot;usual&amp;quot; format means you would have approx half as many tags as using the IOB format. In theory this means your model will develop higher accuracy faster and with less training data. On the downside, you will need to do more work when interpreting the results in order to be confident where one named entity ends and another one begins.&lt;/p&gt;&#xA;&lt;p&gt;I made a &lt;a href=&quot;https://gist.github.com/alanbuxton/804082dbd5f3a5e85ab1e68e5a4317cd&quot; rel=&quot;nofollow noreferrer&quot;&gt;notebook to do some empirical tests on this&lt;/a&gt; which uses 17 tags in the IOB format and 9 in the &amp;quot;usual&amp;quot; format.&lt;/p&gt;&#xA;&lt;p&gt;TL;DR using the &amp;quot;usual&amp;quot; format did not produce a noticeable improvement in model quality.&lt;/p&gt;&#xA;" OwnerUserId="43256" LastActivityDate="2021-01-01T16:47:56.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25504" PostTypeId="2" ParentId="25499" CreationDate="2021-01-01T17:43:43.787" Score="2" Body="&lt;p&gt;The idea behind this kind of reasoning is that there is a &amp;quot;true&amp;quot; distribution (unknown to us, mere mortals) and that the data is generated following this distribution. But what we don't really know the shape of the distribution, all we know is the distribution of the data that we have. This is called the empirical distribution.   Let's see a simple example to illustrate the point.&lt;br /&gt;&#xA;Let's consider a die. Each number is equally likely to show up if we throw the die, so the true underlying distribution is uniform over the set &lt;span class=&quot;math-container&quot;&gt;$\{1, 2,\dots6\}$&lt;/span&gt;. Now let's say you ask your friend to throw the die 60 times, what you will see is likely something close to uniform over the set &lt;span class=&quot;math-container&quot;&gt;$\{1, 2,\dots6\}$&lt;/span&gt; (not really uniform though, as that would be highly unlikely). This distribution is the empirical one, and as you collect more and more samples it will converge to the actual underlying distribution.&lt;br /&gt;&#xA;In your case what happened is the following:&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$x_1,\dots, x_m$&lt;/span&gt; is your sample (in the example above, the &lt;span class=&quot;math-container&quot;&gt;$60$&lt;/span&gt; numbers that you see as your friend throws the die). This sample defines a distribution, &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_{data}$&lt;/span&gt;. In the example above, &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_{data}$&lt;/span&gt; would likely be close to the uniform distribution over &lt;span class=&quot;math-container&quot;&gt;$\{1, \dots, 6\}$&lt;/span&gt;. Now you can think about the sum over &lt;span class=&quot;math-container&quot;&gt;$x_1\dots, x_m$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$\log(p_{model}(x^{(i)};\theta))$&lt;/span&gt; as the average of &lt;span class=&quot;math-container&quot;&gt;$\log(p_{model}(x;\theta))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is drawn according to &lt;span class=&quot;math-container&quot;&gt;$\hat{p}_{data}$&lt;/span&gt;.&lt;br /&gt;&#xA;Let me make another example with some actual numbers. Let's say you toss a fair coin &lt;span class=&quot;math-container&quot;&gt;$7$&lt;/span&gt; times and you see &lt;span class=&quot;math-container&quot;&gt;$$\{H, H, H, T, H, T, H\}.$$&lt;/span&gt;&#xA;The empirical distribution is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(H)=5/7$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(T)=2/7$&lt;/span&gt;. So if you compute the expectation &lt;span class=&quot;math-container&quot;&gt;$$\mathbb{E}_{x \sim \hat{p}_{data}}[\log(p_{model}(x;\theta)]$$&lt;/span&gt; you get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\log(p_{model}(H;\theta)\cdot\mathbb{P}(H) +\log(p_{model}(T;\theta)\cdot\mathbb{P}(T) = \\ \frac{5}{7}\log(p_{model}(H;\theta) +\frac{2}{7}\log(p_{model}(T;\theta)),$$&lt;/span&gt;&#xA;which is what you will get if you compute the first sum you wrote.&lt;/p&gt;&#xA;" OwnerUserId="42424" LastEditorUserId="36821" LastEditDate="2021-01-02T23:17:07.963" LastActivityDate="2021-01-02T23:17:07.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25505" PostTypeId="1" AcceptedAnswerId="25510" CreationDate="2021-01-01T18:59:13.607" Score="2" ViewCount="38" Body="&lt;p&gt;When implementing a genetic algorithm, I understand the basic idea is to have an initial population of a certain size. Then, we pick two individuals from a population, construct two new individuals (using mutation and crossover), repeat this process X number of times and the replace the old population with the new population, based on selecting the fittest.&lt;/p&gt;&#xA;&lt;p&gt;In this method, the population size remains fixed. In reality in evolution, populations undergo fluctuations in population sizes (e.g. population bottlenecks, and new speciations).&lt;/p&gt;&#xA;&lt;p&gt;I understand the disadvantages of variable populations sizes from a biological view are, for example, a bottleneck will reduce the population to minimal levels, so not much evolution will occur. Are there disadvantages to using variable population sizes in genetic algorithms, from a programming perspective? I was thinking the numbers per population could follow a distribution of some sort so they don't just randomly fluctuate erratically, but maybe this does not make sense to do.&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="42926" LastEditDate="2021-01-01T19:10:26.703" LastActivityDate="2021-01-06T10:44:04.847" Title="Are there any disadvantages to using a variable population size in genetic algorithms?" Tags="&lt;algorithm&gt;&lt;genetic-algorithms&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25506" PostTypeId="1" AcceptedAnswerId="25508" CreationDate="2021-01-01T19:19:55.430" Score="0" ViewCount="35" Body="&lt;p&gt;I understand that in each generation of a genetic algorithm, that generation must re-prove it's fitness (and then the fittest of that population is taken for the next population).&lt;/p&gt;&#xA;&lt;p&gt;In this case, I guess it's a presumption that if you take the fittest of each generation, and use them to form the basis of the next generation, that your population as a whole is getting fitter with time.&lt;/p&gt;&#xA;&lt;p&gt;But algorithmically, how can I detect this? If there's no end goal known, then I can't measure the error/distance from goal? So how can you tell how much each generation is becoming fitter by?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastActivityDate="2021-01-06T10:28:56.653" Title="How to detect that the fitness landscape of a genetic algorithm is changing over time?" Tags="&lt;algorithm&gt;&lt;genetic-algorithms&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25507" PostTypeId="1" CreationDate="2021-01-01T19:46:22.897" Score="0" ViewCount="31" Body="&lt;p&gt;I'm just starting to explore topics within computer vision and curious if there are any concepts in that area that could be applied to segmenting multivariate time series with the goal of grouping individual data points similar to how a human might do the same.  I know that there are a number of time series segmentation methods, but in-depth explanations of multivariate methods are more scarce and it seems like somewhat of an underdeveloped topic overall. Since segmentation is such a fundamental part of CV and is inherently multidimensional, I'm wondering if concepts there can be modified to apply to time series.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, I'd like to be able segment a time series and reformulate a prediction problem as something closer to a language processing problem.  The process would look something like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Segment a multivariate time series into near-homogenous segments of variable length.  Some degree of preprocessing might be required but I can worry about that separately.&lt;/li&gt;&#xA;&lt;li&gt;Encode the properties of each segment based on summary statistics (e.g., mean, variance, derivative values, etc.) such that the segments fall into discrete buckets.&lt;/li&gt;&#xA;&lt;li&gt;Each bucket will represent a &amp;quot;word&amp;quot; and the goal of the model will be to predict the next word given a series of words, i.e., the next segment given a series of segments.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In a few days of reading about CV, it seems like there's a ton to learn.  If there are traditional time series segmentation techniques that are more suitable, that would be of interest, but I'd still be curious about a CV approach since that approach likely better aligns with how a person might look at a graph to identify segments.&lt;/p&gt;&#xA;" OwnerUserId="30154" LastEditorUserId="2444" LastEditDate="2021-01-31T17:09:38.603" LastActivityDate="2021-01-31T17:09:38.603" Title="Time series analysis using computer vision principles" Tags="&lt;natural-language-processing&gt;&lt;computer-vision&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25508" PostTypeId="2" ParentId="25506" CreationDate="2021-01-01T19:53:49.537" Score="1" Body="&lt;p&gt;There is no exact way to assess that a genetic algorithm has located a global optima. Indeed there may be multiple global optima. You must fall back to heuristic methods. The fitness of a population is the maximum fitness of any individual. Unless specific measures are taken to maintain diversity the population will converge to an optima, local or global. At that point all individuals will, except for mutation, be identical. You could take the fittest individual of such a population as your solution, but you will not know if the solution is a global or local optima.&lt;/p&gt;&#xA;&lt;p&gt;Two reasonable heuristics are these. First, run the algorithm till it converges and maintains its fitness for a number of further generations. Or second run the algorithm multiple times, and take the fittest of all the located solutions. Neither is exact.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastActivityDate="2021-01-01T19:53:49.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25510" PostTypeId="2" ParentId="25505" CreationDate="2021-01-01T21:05:27.057" Score="1" Body="&lt;p&gt;Population size is a tricky issue even in pure biological models. Biological population sizes obviously vary. The two great protagonists of the argument were Ronald Fisher and Sewell Wright, with argument being between Fisher favouring few large populations against Wright’s many small interconnected populations. There is evidence that evolution occurs more rapidly in Wright’s model but the evidence is inconclusive. The theory concentrates on the probability that a mutation will occur and then become dominant in a population. In a small population a beneficial mutation is more likely to be selected for reproduction, but premature convergence is a serious danger.  While in a larger population a mutant is less likely to be removed from the population during reproduction. I would strongly recommend a read of &lt;a href=&quot;https://www.amazon.co.uk/Games-Life-Explorations-Evolution-Behavior/dp/0486812898/ref=sr_1_1?adgrpid=50071920181&amp;amp;dchild=1&amp;amp;gclid=CjwKCAiArbv_BRA8EiwAYGs23Le0IAnwjMmZOb3JgCcTwuAFFZmPdRHcHNC1PEkxRDsNAA3YoTvDERoC0r8QAvD_BwE&amp;amp;hvadid=259146746543&amp;amp;hvdev=c&amp;amp;hvlocphy=1007199&amp;amp;hvnetw=g&amp;amp;hvqmt=e&amp;amp;hvrand=4888512412962858368&amp;amp;hvtargid=kwd-314887220106&amp;amp;hydadcr=11468_1841728&amp;amp;keywords=games%20of%20life%20karl%20sigmund&amp;amp;qid=1609534890&amp;amp;sr=8-1&amp;amp;tag=googhydr-21&quot; rel=&quot;nofollow noreferrer&quot;&gt;Games of life&lt;/a&gt; by Karl Sigmund.&lt;/p&gt;&#xA;" OwnerUserId="26382" LastActivityDate="2021-01-01T21:05:27.057" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25511" PostTypeId="2" ParentId="25371" CreationDate="2021-01-01T21:24:02.653" Score="0" Body="&lt;p&gt;It is entirely possible!&lt;/p&gt;&#xA;&lt;p&gt;You see, the agents will perform whatever actions are available to them, and if the evolutionary algorithm is setup correctly, whatever set of actions provides them with a bigger survival rate will be the one that gets explored and reproduced the most.&lt;/p&gt;&#xA;&lt;p&gt;Here is a very interesting list of &amp;quot;Specification Gaming&amp;quot; in AI, where the agents happened to &amp;quot;game&amp;quot; the rules to reach their goals (metric optimization) without actually doing what the creators intended: &lt;a href=&quot;https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml&lt;/a&gt; (&lt;a href=&quot;https://medium.com/@deepmindsafetyresearch/specification-gaming-the-flip-side-of-ai-ingenuity-c85bdb0deeb4&quot; rel=&quot;nofollow noreferrer&quot;&gt;second link&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="190" LastActivityDate="2021-01-01T21:24:02.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25512" PostTypeId="1" AcceptedAnswerId="25514" CreationDate="2021-01-01T22:59:48.977" Score="1" ViewCount="56" Body="&lt;p&gt;Referring to &lt;a href=&quot;https://ai.stackexchange.com/q/15431/2444&quot;&gt;this post&lt;/a&gt;, in the following formula to update the state-action value&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Q(s,a) = Q(s,a) + \alpha (G − Q(s,a)),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is the value of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; (the return) the same for every state-action &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; pair?&lt;/p&gt;&#xA;&lt;p&gt;I am a little confused about this point, so I will thank any clarification.&lt;/p&gt;&#xA;" OwnerUserId="33566" LastEditorUserId="2444" LastEditDate="2021-01-02T12:33:47.847" LastActivityDate="2021-01-02T12:34:33.243" Title="When updating the state-action value in the Monte Carlo method, is the return the same for each state-action pair?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;return&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25513" PostTypeId="1" AcceptedAnswerId="25523" CreationDate="2021-01-02T00:15:13.793" Score="0" ViewCount="55" Body="&lt;p&gt;I'm working on the code trying to generate new images using DCGAN model. The structure of my code is from the PyTorch tutorial &lt;a href=&quot;https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;I'm a bit confused trying to find and understand how the latent vector is transforming to the feature map from the Generator part (this line of code is what I'm interested in) :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nn.ConvTranspose2d(  nz, ngf * 8, 4, 1, 0, bias=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It means the latent vector (nz) of shape 100x1 transforming to the 512 matrices of 4x4 size (ngf=64). How  does it happen? Hence, I can't even clear to myself how the length of the latent vector influence to the generated image.&#xA;P.S. The left part of the Genarator structure is clear.&lt;/p&gt;&#xA;&lt;p&gt;The only idea that I got is :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;E.g. there is a latent vector of size 100 as input (100 random values).&lt;/li&gt;&#xA;&lt;li&gt;We interact each value of the input latent vector with a 4x4 kernel.&lt;/li&gt;&#xA;&lt;li&gt;In this way we get 100 different 4x4 matrices (each matrix is for one value from latent vector).&lt;/li&gt;&#xA;&lt;li&gt;Then we summarize all these 100 matrices and get one final matrix - one feature map.&lt;/li&gt;&#xA;&lt;li&gt;We get necessary number of feature maps taking different kernels.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is this right? Or does it happen in other way?&lt;/p&gt;&#xA;" OwnerUserId="41783" LastEditorUserId="41783" LastEditDate="2021-01-02T16:03:36.053" LastActivityDate="2021-01-02T17:22:48.383" Title="How is the latent vector transforming to a feature map in DCGAN (Generator structure)?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;generative-adversarial-networks&gt;&lt;dc-gan&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25514" PostTypeId="2" ParentId="25512" CreationDate="2021-01-02T01:43:39.183" Score="2" Body="&lt;p&gt;The discussion uses poor notation, there should be a time index. You obtain a list of tuples &lt;span class=&quot;math-container&quot;&gt;$(s_t, a_t, r_t, s_{t+1})$&lt;/span&gt; and then, for every visit MC, you update&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q(s_t, a_t) = Q(s_t, a_t) + \alpha (G_t - Q(s_t, a_t))\;;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$G_t = \sum_{k=0}^\infty \gamma^k r_{t+k}$&lt;/span&gt;, for each &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; in the episode. You can see that the returns for each time step are calculated for time timestep onwards, and so are not necessarily the same across time steps.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="2444" LastEditDate="2021-01-02T12:34:33.243" LastActivityDate="2021-01-02T12:34:33.243" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25515" PostTypeId="2" ParentId="25498" CreationDate="2021-01-02T01:45:09.053" Score="2" Body="&lt;p&gt;Since you want a shortcut use the &lt;a href=&quot;https://spoonacular.com/food-api/docs/detect-foods-in-text&quot; rel=&quot;nofollow noreferrer&quot;&gt;spoonacular API&lt;/a&gt;.  Below is a test with your words.  You can see it had trouble with 'Coca' and 'veg'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZzoTg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZzoTg.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What you need is 'named-entity recognition' for food.  This is not a new thing but clearly not a solved problem.  The &lt;a href=&quot;https://github.com/vabraham/foodie_favorites&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foodie Favorites&lt;/a&gt; repository attempts to solve the problem from scratch.&lt;/p&gt;&#xA;&lt;p&gt;If want to do some research and dig deeper see &lt;a href=&quot;https://academic.oup.com/database/article/doi/10.1093/database/baz121/5611291&quot; rel=&quot;nofollow noreferrer&quot;&gt;FoodBase corpus: a new resource of annotated food entities&lt;/a&gt;.  From the abstract:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It consists of 12,844 food entity annotations describing 2105 unique food entities. Additionally, we provided a weakly annotated corpus on an additional 21,790 recipes. It consists of 274,053 food entity annotations, 13,079 of which are unique.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2021-01-02T01:45:09.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25516" PostTypeId="1" CreationDate="2021-01-02T02:31:44.563" Score="0" ViewCount="71" Body="&lt;p&gt;I'm not very knowledgeable in this field but I'm wondering if any research or information already exists for the following situation:&lt;/p&gt;&#xA;&lt;p&gt;I have some data that may or may not look similar to each other. Each represents a node that represents a vector of size 128.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9XT1o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9XT1o.png&quot; alt=&quot;similar vectors&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And they are inserted into a tree graph according to similarity.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Vsy4C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vsy4C.png&quot; alt=&quot;better graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and a new node is created with an edge connecting the most similar vertex node found in the entire tree graph.&lt;/p&gt;&#xA;&lt;p&gt;Except I'm wasting a lot of time searching through the entire graph to insert each new node when I could narrow down my search according to previous information. Imagine a trunk node saying &amp;quot;Oh, I saw a node like you before, it went down this branch. And don't bother going down that other branch.&amp;quot; I could reduce the cost of searching the entire tree structure if there was a clever way to remember if a similar node went down a certain path.&lt;/p&gt;&#xA;&lt;p&gt;I've thought about some ways to use caching or creating a look-up table but these are very memory intensive methods and will become slower the longer the program runs on. I have some other ideas I am playing around with but I was hoping someone could point me in the right direction before I started trying out weird ideas.&lt;/p&gt;&#xA;&lt;p&gt;Edit: added a better (more realistic) graph picture&lt;/p&gt;&#xA;" OwnerUserId="43419" LastEditorUserId="43419" LastEditDate="2021-01-06T05:43:57.683" LastActivityDate="2021-02-05T06:00:40.073" Title="A way to leverage machine learning to reduce DFS/BFS search time on a tree graph?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;breadth-first-search&gt;&lt;depth-first-search&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25517" PostTypeId="2" ParentId="25483" CreationDate="2021-01-02T02:44:14.230" Score="1" Body="&lt;p&gt;The main issue during training is that you haven't right-shifted the input of the decoder, which is probably why you set the diagonals of &lt;code&gt;mask&lt;/code&gt; to -inf (when it should be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Also, just an FYI, although you haven't focused on evaluation/prediction yet, I will explain the evaluation/prediction here as well for completeness, since it works so differently than training, and also since you will need it when generating the graphs for debugging.&lt;/p&gt;&#xA;&lt;h2&gt;Training&lt;/h2&gt;&#xA;&lt;p&gt;Both &lt;code&gt;tgt&lt;/code&gt; and &lt;code&gt;tgt_mask&lt;/code&gt; should be changed to simulate auto-regressive properties.&lt;/p&gt;&#xA;&lt;p&gt;You are feeding in &lt;code&gt;tgt&lt;/code&gt; as the input to the decoder, where &lt;code&gt;tgt&lt;/code&gt; is the ground truth target sequence to predict. &lt;code&gt;tgt&lt;/code&gt; should have dimension &lt;code&gt;length(sequence)&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\|dict\|$&lt;/span&gt;. Additionally, you are feeding in mask where the diagonals are &lt;code&gt;-inf&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Instead, you should do the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Add 2 new special tokens to your vocabulary that signals the start and end of decoding, i.e. &lt;code&gt;&amp;lt;START&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;END&amp;gt;&lt;/code&gt;. So your vocabulary will need to be extended to 16 events.&lt;/li&gt;&#xA;&lt;li&gt;Every input to the decoder (training, evaluation, prediciton) should have &lt;code&gt;&amp;lt;START&amp;gt;&lt;/code&gt; as the first token. So during training, when you want to feed in &lt;code&gt;tgt&lt;/code&gt; to the decoder, you should right-shift &lt;code&gt;tgt&lt;/code&gt; by adding in the &lt;code&gt;&amp;lt;START&amp;gt;&lt;/code&gt; token at the beginning.&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;tgt_shifted&lt;/code&gt; will now be of dimension &lt;code&gt;length(sequence)+1&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\| dict \|$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;tgt_shifted_mask&lt;/code&gt; will now be of dimension &lt;code&gt;length(sequence)+1&lt;/code&gt; x &lt;code&gt;length(sequence)+1&lt;/code&gt;. Diagonal should be all &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;the inputs to the decoder should be &lt;code&gt;tgt_shifted&lt;/code&gt;, &lt;code&gt;tgt_shifted_mask&lt;/code&gt;, and &lt;code&gt;memory&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;the output of the decoder will have dimension &lt;code&gt;length(sequence)+1&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\| dict\|$&lt;/span&gt;, and will not be right-shifted because it will not have &lt;code&gt;&amp;lt;START&amp;gt;&lt;/code&gt; as the first word. But it should have &lt;code&gt;&amp;lt;END&amp;gt;&lt;/code&gt; as the last word.&lt;/li&gt;&#xA;&lt;li&gt;Append &lt;code&gt;&amp;lt;END&amp;gt;&lt;/code&gt; to the ground truth &lt;code&gt;tgt&lt;/code&gt;, and turn it into one-hot encoding, so &lt;code&gt;tgt&lt;/code&gt; should have dimension &lt;code&gt;length(sequence)+1&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\|dict\|$&lt;/span&gt;. Your loss operator should be some sort of element-wise comparison between &lt;code&gt;tgt&lt;/code&gt; and the output of the decoder.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that you only run the decoder once per training batch during training, i.e. the decoder simultaneously predicts the logits of all &lt;code&gt;length(sequence)&lt;/code&gt; tokens at the same time. On the other hand, during evaluation/prediction, you must run the decoder &lt;code&gt;length(sequence)&lt;/code&gt; times, since you can only use the decoder to predict one token at a time.&lt;/p&gt;&#xA;&lt;h2&gt;Evaluation/Prediction&lt;/h2&gt;&#xA;&lt;p&gt;During evaluation/prediction, the model should generate its own sentence entirely from scratch, one word at a time:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;out&lt;/code&gt; should be initialized as a single token, &lt;code&gt;&amp;lt;START&amp;gt;&lt;/code&gt;, and have dimensions &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;  x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\|dict\|$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Do the following in a loop until termination or max output length is reached:&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate a &lt;code&gt;out_mask&lt;/code&gt; for the current version of &lt;code&gt;out&lt;/code&gt;. &lt;code&gt;out_mask&lt;/code&gt; should be a matrix with dimension &lt;code&gt;length(out)&lt;/code&gt; x &lt;code&gt;length(out)&lt;/code&gt;. Diagonal should be all &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Pass &lt;code&gt;out&lt;/code&gt;, &lt;code&gt;out_mask&lt;/code&gt;, and &lt;code&gt;memory&lt;/code&gt; into the decoder layer. The output of the decoder will be a &lt;code&gt;length(out)&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$d_{model}$&lt;/span&gt; tensor (note that in your case, &lt;span class=&quot;math-container&quot;&gt;$d_{model} = \|dict\|$&lt;/span&gt;), which indicates &lt;code&gt;tokenIdx&lt;/code&gt; x &lt;code&gt;batchIdx&lt;/code&gt; x &lt;code&gt;vocabularyIdx&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Convert the last time step (e.g. &lt;code&gt;decoderOutput[-1,:,:]&lt;/code&gt; if all sequences in your batch have the same length) into probabilities, and take all the events that surpasses some probability threshold and set them to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Set all other events to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. This is the new time step that the decoder layer predicts. So add this time step to &lt;code&gt;out&lt;/code&gt; to append it's length by &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Repeat the previous 3 steps in a loop until you reach the &lt;code&gt;&amp;lt;END&amp;gt;&lt;/code&gt; token, which signals the end of decoding, or until the max output length is reached.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;During evaluation, you can calculate the validation loss by using the logits of only the final iteration of the decoder (which should have dimension &lt;code&gt;length(sequence)+1&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\|dict\|$&lt;/span&gt;), and compare those logits with the ground truth with the &lt;code&gt;&amp;lt;END&amp;gt;&lt;/code&gt; token added at the end (resulting in dimensions &lt;code&gt;length(sequence)+1&lt;/code&gt; x &lt;code&gt;batchSize&lt;/code&gt; x &lt;span class=&quot;math-container&quot;&gt;$\|dict\|$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="42699" LastEditDate="2021-01-04T02:03:31.007" LastActivityDate="2021-01-04T02:03:31.007" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="25519" PostTypeId="2" ParentId="23842" CreationDate="2021-01-02T09:23:13.183" Score="0" Body="&lt;p&gt;Your question seems to be talking about two slightly different topics:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pros and cons of 'one vs rest' approach in multi-class classification&lt;/li&gt;&#xA;&lt;li&gt;Use of Neural Networks in single-output vs multi-class classification problems&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;One vs Rest in Multi-Class Classification&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Recognising digits is an example of multi-class classification. The approach you outline is the kind of approach summarised in the &amp;quot;One vs Rest&amp;quot; section of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiclass_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page on multi-class classification&lt;/a&gt;. The page notes the following issues with this approach:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Firstly, the scale of the confidence values may differ between the binary classifiers. Second, even if the class distribution is balanced in the training set, the binary classification learners see unbalanced distributions because typically the set of negatives they see is much larger than the set of positives.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You might also like to look into another approach called One vs One (&lt;a href=&quot;https://machinelearningmastery.com/one-vs-rest-and-one-vs-one-for-multi-class-classification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;'One vs Rest' vs 'One vs One'&lt;/a&gt;) which sets up the classification problem as a set of binary alternatives. In the digit recognition case you'd end up with a classifier for &amp;quot;1 or 2?&amp;quot;, &amp;quot;1 or 3?&amp;quot;, &amp;quot;1 or 4?&amp;quot; etc. This might help with the &amp;quot;4 vs 9&amp;quot; problem but it does mean an enormous amount of classifiers, that might be better represented in some kind of network. Perhaps even a network inspired by brain neurons.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Use of Neural Networks in single output vs multi-class classification&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is nothing magical about a neural network that means it has to be used for multi-class classification. Nor is there anything magical about it that makes it the only option for multi-class classification.&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://builtin.com/data-science/how-build-neural-network-keras&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using a Neural Network for sentiment analysis&lt;/a&gt; outputs one single answer about how positive/negative a piece of text is.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/pramodini18/Digit-recognition-using-SVM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digit recognition using SVM&lt;/a&gt; uses something that isn't a neural network for multi-class classification&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A 10-class neural network is used to identify digits because this has turned out to be an efficient way of doing so when compared with one vs rest and one vs all approaches.&lt;/p&gt;&#xA;&lt;p&gt;A bit off-topic, perhaps, but if you think about this in the context of &lt;a href=&quot;https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;T5&lt;/a&gt;, there does seem to be a trend of moving towards larger more multi-purpose models rather than lots of small specialised models.&lt;/p&gt;&#xA;" OwnerUserId="43256" LastActivityDate="2021-01-02T09:23:13.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25520" PostTypeId="1" CreationDate="2021-01-02T09:26:15.193" Score="0" ViewCount="17" Body="&lt;p&gt;I'm trying to understand how the logic gates (e.g. AND, OR, NOT, NAND) can be built into single-layer perceptrons.&lt;/p&gt;&#xA;&lt;p&gt;I understand specific examples of weights and thresholds for the gates, but I'm stuck on understanding how to translate these to general inequalities for these gates.&lt;/p&gt;&#xA;&lt;p&gt;Is my reasoning correct in the table below, or are there cases where these general inequalities do not make sense for this problem? Am I missing other logic gates that can be done in a similar fashion? (e.g., I know XOR cannot)?&lt;/p&gt;&#xA;&lt;p&gt;In the table below, a perceptron has two input nodes, and one output node. W1 and W2 are the weights (real values) on those input nodes. T is the threshold, above which, the perceptron will fire. I have come up with example values that would work for each logic gate (e.g., for the AND gate, a perceptron with two input weights, W1 = 1 and W2 = 1, and a threshold = 2, will fire, and I'm trying to understand more generally, what is the equation needed for each gate).&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Gate&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Example W1, W2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Threshold&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;General inequalities&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AND&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1,1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;W1 + W2 &amp;gt;= t, where W1, W2 &amp;gt; 0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;OR&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1,1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;W1 &amp;gt; t or W2 &amp;gt; t&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;NOT&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;-1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;-0.49&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;W1 &amp;gt; 2(t)&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;NAND&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;-2,-2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;W1 + W2 &amp;lt;= t&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="42926" LastEditorUserId="42926" LastEditDate="2021-01-03T10:40:13.660" LastActivityDate="2021-01-03T10:40:13.660" Title="What are the general inequalities needed for the logic gate perceptrons?" Tags="&lt;machine-learning&gt;&lt;perceptron&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25521" PostTypeId="1" CreationDate="2021-01-02T09:48:55.003" Score="0" ViewCount="31" Body="&lt;p&gt;I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and threshold that will only fire for points inside the rectangle.'&lt;/p&gt;&#xA;&lt;p&gt;I understand that this is an example of a rectangle drawn as a NN (i.e. this NN will fire, if the point is in the rectangle, where the rectangle is defined by the lines X = 4; X = 1, Y = 2, Y = 5).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/k7ldN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k7ldN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this diagram, since it's a rectangle, the equations of the line in this example are x = 4, x =1, y=2, y=5, so I left the other weights out (as they equal to 0).&lt;/p&gt;&#xA;&lt;p&gt;I'm now wondering how this could be translated to a 3D structure. For example, if a 3D shape was defined by the points:&lt;/p&gt;&#xA;&lt;p&gt;(0,0,0), (0,1,0), (0,0,1), (0,1,1), (1,0,0), (1,1,0), (1,0,1), (1,1,1)&lt;/p&gt;&#xA;&lt;p&gt;I wanted to draw a hyperplane that separates the corner point (1,1,1) from the other points in this cube. Can this 3D shape be drawn similarly to below (maybe it would be easier to understand, if there were other numbers except 1 and 0 in the co-ordinates)?&lt;/p&gt;&#xA;&lt;p&gt;Would I draw this with 3 nodes in the input layer, still one node in the output layer, I just don't understand what the hidden layer should look like? Would it have 24 nodes? One for each surface of the cube, with relevant X and Y values?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="42926" LastEditDate="2021-01-03T12:45:20.170" LastActivityDate="2021-01-03T12:45:20.170" Title="How to draw a 3-dimensonal shape's neural network" Tags="&lt;neural-networks&gt;&lt;multilayer-perceptron&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25522" PostTypeId="1" AcceptedAnswerId="25575" CreationDate="2021-01-02T11:58:45.030" Score="0" ViewCount="43" Body="&lt;p&gt;If I have a neural network, and say the 6th output node of the neural network is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x_6 = w_{16}y_1 + w_{26}y_2 + w_{36}y_3$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What does that make the derivative of:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial x_6}{\partial w_{26}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I guess that it's how is &lt;span class=&quot;math-container&quot;&gt;$x_6$&lt;/span&gt; changing with respect to &lt;span class=&quot;math-container&quot;&gt;$w_{26}$&lt;/span&gt;, so, therefore, is it equal to &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt; (since the output, &lt;span class=&quot;math-container&quot;&gt;$y_2$&lt;/span&gt;, will change depending on the weight added to the input)?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="2444" LastEditDate="2021-01-02T23:21:26.130" LastActivityDate="2021-01-04T22:42:20.240" Title="What is the derivative of a specific output with respect to a specific weight?" Tags="&lt;neural-networks&gt;&lt;gradient-descent&gt;&lt;calculus&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25523" PostTypeId="2" ParentId="25513" CreationDate="2021-01-02T17:11:28.437" Score="0" Body="&lt;p&gt;Noise vector (batch_size, 100, 1, 1) is deconvoloved with filter_1 (100, 4, 4). Result is feature_map_1 (1, 4, 4). And since there are 512 filters, so there will be 512 feature maps. Output shape will be (batch_size, 512, 4, 4).&lt;/p&gt;&#xA;&lt;p&gt;I think you need better undersanting for convolutional calculations in general. In this &lt;a href=&quot;https://stackoverflow.com/questions/43306323/keras-conv2d-and-input-channels&quot;&gt;stack&lt;/a&gt; it was explained very well.&lt;/p&gt;&#xA;" OwnerUserId="41615" LastEditorUserId="41615" LastEditDate="2021-01-02T17:22:48.383" LastActivityDate="2021-01-02T17:22:48.383" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25524" PostTypeId="2" ParentId="22859" CreationDate="2021-01-02T17:15:52.420" Score="0" Body="&lt;p&gt;What you should do as part of your exploration is to learn various models of increasing complexity. Start from a simple linear model, ending in multi-layer neural networks (with non-linear activations of course). If the nonlinear models are better then that implies that your data do not follow a linear hyperplane.&lt;/p&gt;&#xA;&lt;p&gt;Also check this out for recent trends: &lt;a href=&quot;https://machinelearningmastery.com/auto-sklearn-for-automated-machine-learning-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/auto-sklearn-for-automated-machine-learning-in-python/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43358" LastActivityDate="2021-01-02T17:15:52.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25526" PostTypeId="1" AcceptedAnswerId="25528" CreationDate="2021-01-02T17:22:06.950" Score="0" ViewCount="74" Body="&lt;p&gt;If the derivative is supposed to give the rate of change of a function at that point, then why is the derivative of the softmax layer (a vector) the Jacobian matrix, which has a different shape than the output/softmax vector? Why is the shape of the softmax vector's derivative (the Jacobian) different than the shape of the derivative of the other activation functions, such as the ReLU and sigmoid?&lt;/p&gt;&#xA;" OwnerUserId="43270" LastEditorUserId="2444" LastEditDate="2021-01-02T21:33:03.030" LastActivityDate="2021-01-02T21:33:03.030" Title="Why is the derivative of the softmax layer shaped differently than the derivative of other neurons?" Tags="&lt;backpropagation&gt;&lt;loss-functions&gt;&lt;gradient-descent&gt;&lt;linear-algebra&gt;&lt;softmax&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25527" PostTypeId="1" CreationDate="2021-01-02T17:49:19.987" Score="9" ViewCount="126" Body="&lt;p&gt;I learned about the &lt;em&gt;universal approximation theorem&lt;/em&gt; from &lt;a href=&quot;http://neuralnetworksanddeeplearning.com/chap4.html&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number of neurons. Or mathematically, &lt;span class=&quot;math-container&quot;&gt;${|g(x)−f(x)|&amp;lt; \epsilon}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;${g(x)}$&lt;/span&gt; is the approximation, &lt;span class=&quot;math-container&quot;&gt;${f(x)}$&lt;/span&gt; is the target function and  is &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is an arbitrary bound.&lt;/p&gt;&#xA;&lt;p&gt;A polynomial of degree &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; has at maximum &lt;span class=&quot;math-container&quot;&gt;$n-1$&lt;/span&gt; turning points (where the derivative of the polynomial changes sign). With each new turning point, the approximation seems to become more complex.&lt;/p&gt;&#xA;&lt;p&gt;I'm not necessarily looking for a formula, but I'd like to get a general idea on how to figure out the sufficient number of neurons is for a reasonable approximation of a polynomial with a single layer of the neural network (you may consider &amp;quot;reasonable&amp;quot; to be &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 0.0001$&lt;/span&gt;). To ask in other words, how would adding one more neuron affect the model's ability to express a polynomial?&lt;/p&gt;&#xA;" OwnerUserId="38076" LastEditorUserId="2444" LastEditDate="2021-01-02T23:06:30.820" LastActivityDate="2021-01-02T23:06:30.820" Title="What is the number of neurons required to approximate a polynomial of degree n?" Tags="&lt;neural-networks&gt;&lt;computational-learning-theory&gt;&lt;multilayer-perceptron&gt;&lt;function-approximation&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25528" PostTypeId="2" ParentId="25526" CreationDate="2021-01-02T19:08:31.960" Score="3" Body="&lt;p&gt;When you use the softmax activation function is usually as a last layer of your network and to get an output that is a vector. Now your confusion is about shapes, so let's review a bit of calculus.&lt;br /&gt;&#xA;If you have a function &lt;span class=&quot;math-container&quot;&gt;$$f:\mathbb{R}\rightarrow\mathbb{R}$$&lt;/span&gt;&#xA;the derivative is a function on its own and you have &lt;span class=&quot;math-container&quot;&gt;$$f':\mathbb{R}\rightarrow\mathbb{R}.$$&lt;/span&gt;&#xA;If you increase the dimension of the input space have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f:\mathbb{R}^n\rightarrow\mathbb{R}.$$&lt;/span&gt;&#xA;The &amp;quot;derivative&amp;quot; in this case is called gradient and it is a vector collecting the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; partial derivatives of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The input space of the gradient function is &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; (the same as for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;), but the output is the collection of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; derivatives, so the output space is also &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;. In other words&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla f:\mathbb{R}^n\rightarrow\mathbb{R}^n,$$&lt;/span&gt;&#xA;which makes sense as for each point &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; of the input space you get a vector (&lt;span class=&quot;math-container&quot;&gt;$\nabla f(x)$&lt;/span&gt;) as output.&lt;br /&gt;&#xA;So far so good, but what happens if you consider a function that takes a vector as input and spits out a vector as output, i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f:\mathbb{R}^n\rightarrow\mathbb{R}^m?$$&lt;/span&gt;&#xA;How to compute the equivalent of the derivative? (This is the softmax case, where you have a vector as input and a vector as output.)&lt;br /&gt;&#xA;You can reduce this case to the previous case considering &lt;span class=&quot;math-container&quot;&gt;$f=(f_1, \dots, f_m)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f_i:\mathbb{R}^n\rightarrow\mathbb{R}.$&lt;/span&gt; Now for each &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt; you can compute the gradient &lt;span class=&quot;math-container&quot;&gt;$\nabla f_i$&lt;/span&gt; and end up with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; gradients. When you evaluate them at a point &lt;span class=&quot;math-container&quot;&gt;$x\in\mathbb{R}^n$&lt;/span&gt; you get &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$n-$&lt;/span&gt;dimensional vectors. These vector can be collected in a matrix, which is the Jacobian; formally&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Jf:\mathbb{R}^n\rightarrow\mathbb{R}^{m\times n}.$$&lt;/span&gt;&#xA;Finally, to answer your question, you get a Jacobian &amp;quot;instead&amp;quot;  of a gradient (they all represent the same concept) because the output of the softmax is not a single number but a vector.&lt;br /&gt;&#xA;By the way the sigmoid and relu are functions with one dimensional input and output, so they don't really have a gradient but a derivative. The trick is that people write &lt;span class=&quot;math-container&quot;&gt;$\sigma(W)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; is vector or a matrix, but they mean that &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is applied component-wise, as a function from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt; (I know, it's confusing).&lt;br /&gt;&#xA;(I know, I kind of skipped your other question, but this answer was already long and I think you can convince yourself that the dimensions match correctly (with the Jacobian) for the update rule to work. If not I'll edit)&lt;/p&gt;&#xA;" OwnerUserId="42424" LastActivityDate="2021-01-02T19:08:31.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25529" PostTypeId="1" CreationDate="2021-01-02T20:07:47.443" Score="1" ViewCount="32" Body="&lt;p&gt;I'm reading &lt;a href=&quot;https://page.mi.fu-berlin.de/rojas/neural/chapter/K3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; book chapter, and I'm looking at the questions on the last page. Can someone explain question 2 on the last page to me, or show me an example of a solution so I can understand it?&lt;/p&gt;&#xA;&lt;p&gt;The question is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Consider a simple perceptron with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; bipolar inputs and threshold &lt;span class=&quot;math-container&quot;&gt;$\theta = 0$&lt;/span&gt;. Restrict each of the weights to have the value &lt;span class=&quot;math-container&quot;&gt;$−1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Give the smallest upper bound you can find for the number of functions from &lt;span class=&quot;math-container&quot;&gt;$\{−1, 1 \}^n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\{−1, 1\}$&lt;/span&gt; which are computable by this perceptron. Prove that the upper bound is sharp, i.e., that all functions are different.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What I understand:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A perceptron is a very simple network with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; input nodes, a weight assigned to each input nodes, which are then summed to be above/not above a threshold (&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In this example, there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; input nodes, and the value of each input node is either &lt;span class=&quot;math-container&quot;&gt;$−1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. And we want to map them to outputs of either &lt;span class=&quot;math-container&quot;&gt;$−1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I'm confused about:&#xA;Is it asking how many different ways can you map input values of &lt;span class=&quot;math-container&quot;&gt;$\{−1, 1\}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\{−1, 1\}$&lt;/span&gt; output?&lt;/p&gt;&#xA;&lt;p&gt;For example, is the answer, where each tuple in this list is input1, input2 and label, as described above:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[(1,1,1), (1,1,-1), (-1,1,-1), (-1,1,1), (1,-1,1), (1,-1,-1), (-1,-1,-1)]$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="2444" LastEditDate="2021-01-03T12:01:57.907" LastActivityDate="2021-01-03T12:01:57.907" Title="What is the smallest upper bound for a number of functions in a range that are computable by a perceptron?" Tags="&lt;machine-learning&gt;&lt;function-approximation&gt;&lt;perceptron&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25530" PostTypeId="2" ParentId="25478" CreationDate="2021-01-02T20:09:18.850" Score="1" Body="&lt;p&gt;This is mostly an implementation architecture problem, and the thing is that basically you can implement anything in the traditional setting. To do so instead of having &lt;code&gt;Env&amp;lt;-&amp;gt;Agent1&amp;lt;-&amp;gt;Agent2&lt;/code&gt;, you should have &lt;code&gt;Agent1&amp;lt;-&amp;gt;SuperEnv&amp;lt;-&amp;gt;Agent2&lt;/code&gt; where &lt;code&gt;SuperEnv&lt;/code&gt; contains &lt;code&gt;Env&lt;/code&gt;, and simply uses the reward given to &lt;code&gt;SuperEnv&lt;/code&gt; by &lt;code&gt;Agent1&lt;/code&gt; and passes it to &lt;code&gt;Agent2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I know this might seem a little counter-intuitive when comparing the implementation to the real-world problem setting, but the consistency of the RL structure (i.e. Environment that interacts with all the agent) is very important for your solutions to be easily understandable by others.&lt;/p&gt;&#xA;" OwnerUserId="42903" LastActivityDate="2021-01-02T20:09:18.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25531" PostTypeId="1" CreationDate="2021-01-03T00:58:30.543" Score="1" ViewCount="21" Body="&lt;p&gt;I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see &lt;a href=&quot;https://arxiv.org/abs/1511.08308&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chiu and Nichols (2016)&lt;/a&gt; for the motivation). I haven't found any empirical evidence of whether or not marking the word boundaries makes a difference. As an example, a 1-D CNN with kernel size 2 would take &lt;code&gt;&amp;quot;the&amp;quot;&lt;/code&gt; as input and use &lt;code&gt;{&amp;quot;th&amp;quot;, &amp;quot;he&amp;quot;}&lt;/code&gt; in its filters. But if I explicitly marked the boundaries it would give me &lt;code&gt;{&amp;quot;t&amp;quot;, &amp;quot;th&amp;quot;, &amp;quot;he&amp;quot;, &amp;quot;h&amp;quot;}&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is there a go-to paper or project that definitively answers this question?&lt;/p&gt;&#xA;" OwnerUserId="19703" LastEditorUserId="19703" LastEditDate="2021-01-03T18:30:02.690" LastActivityDate="2021-01-03T18:30:02.690" Title="Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;embeddings&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25532" PostTypeId="1" AcceptedAnswerId="25533" CreationDate="2021-01-03T01:59:09.930" Score="1" ViewCount="36" Body="&lt;p&gt;Does it make sense to incorporate constant states in the Markov Decision Process and employ a reinforcement learning algorithm to solve it?&lt;/p&gt;&#xA;&lt;p&gt;For instance, for applications of personalization, I would like to include users' attributes, like age and gender, as states, along with other changing states. Does it make sense to do so? I have a sequential problem, so I assume the contextual bandit does not fit here.&lt;/p&gt;&#xA;" OwnerUserId="23707" LastEditorUserId="2444" LastEditDate="2021-01-03T19:33:01.883" LastActivityDate="2021-01-03T19:33:01.883" Title="Does it make sense to include constant states into reinforcement learning formulation?" Tags="&lt;reinforcement-learning&gt;&lt;state-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25533" PostTypeId="2" ParentId="25532" CreationDate="2021-01-03T07:27:34.393" Score="0" Body="&lt;p&gt;It is, I suppose, a philosophical question whether data that &lt;em&gt;describes&lt;/em&gt; a whole episode and does not respond to events within it is part of the state, or is part of some other structure.&lt;/p&gt;&#xA;&lt;p&gt;However, the practical response is to view such descriptive data as defining an &lt;em&gt;instance&lt;/em&gt; of a class of related environments, and to include it in the state features. This may be done for two main reasons:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The static data is a relevant parameter of the environment, affecting state transitions and rewards.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is possible to generalise over the population of all values that the parameters can take.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In simple environments, generalisation might only be that the same agent can learn about all variations in a single combined training session. You could use a tabular RL method, starting randomly with one of the possible variations until all were sufficiently covered.&lt;/p&gt;&#xA;&lt;p&gt;In more complex environments, generalisation may also occur through functon approximation, in a similar manner to contextual bandits. In your personalisation example, you are not expecting to train the agent for all possible user descriptions, but hope that people with similar age, gender etc descriptions will respond similarly to an agent that personalises content.&lt;/p&gt;&#xA;&lt;p&gt;Philosophically, the contextual data is either part of a larger state space (with a restriction that transitions between different contexts do not happen within an episode), or it is metadata that impacts the &amp;quot;real&amp;quot; state transitions and rewards. Pragmatically, to allow the data to influence value functions and policies, it is necessary to use it in the arguments of those functions. Whether you then view it as part of the state feature vector or as something that is concatenated to state features is a personal choice. Most of the literature I have seen assumes without comment that it is part of the state.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-01-03T10:35:53.997" LastActivityDate="2021-01-03T10:35:53.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25534" PostTypeId="1" AcceptedAnswerId="25577" CreationDate="2021-01-03T08:01:46.773" Score="1" ViewCount="47" Body="&lt;p&gt;I have an interesting problem related to training the model on two different datasets for the target feature on images taken on different conditions, which might affect the model's ability to generalize.&lt;/p&gt;&#xA;&lt;p&gt;To explain I will give examples of images from the two different datasets.&lt;/p&gt;&#xA;&lt;p&gt;Dataset 1 sample image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lkduq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lkduq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Dataset 2 sample image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wy9LN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wy9LN.png&quot; alt=&quot;[Dataset 2 sample image:2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you see the images are captured in two completely different conditions. I am afraid that the model will infer from the background information that it shouldn't use to predict the plant diseases, what makes the problem worse is that some plant diseases only exist in one dataset and not the other, if all the diseases are contained in both datasets then I wouldn't think there would be a problem.&lt;/p&gt;&#xA;&lt;p&gt;I am assuming I need a way to unify the background by somehow detecting the leaf pixels in the images and unifying the background in a way that makes the model focuses on the important features.&lt;/p&gt;&#xA;&lt;p&gt;I've tried some segmentation methods but the methods I tried don't always give desirable results for all the images.&lt;/p&gt;&#xA;&lt;p&gt;What is the recommended approach here? All help is appreciated&lt;/p&gt;&#xA;&lt;p&gt;Further explanation of the problem.&lt;/p&gt;&#xA;&lt;p&gt;Ok so I will explain one more thing, my model on the two datasets works fine when training and validating, It got a 94% accuracy.&lt;/p&gt;&#xA;&lt;p&gt;The problem is, even though the model performs well on the datasets I have, I am afraid that when I use the model on real-life conditions (say someone capturing an image with their phone) the model will be heavily biased towards predicting labels in the second dataset (the one with actual background) since the background is similar and it somehow associated the background with the inference process.&lt;/p&gt;&#xA;&lt;p&gt;I have tried downloading a leaf image of a label that is contained on the first dataset ( the one with the white background), where the image had a real-life background, the model as expected failed to predict the correct label and predicted a label contained in the second dataset, I am assuming it was due to the background. I have tried this experiment multiple times, and the model consistently failed in similar scenarios&lt;/p&gt;&#xA;&lt;p&gt;I used some Interpretability techniques as well to visualize the important pixels and it seems like the model is using the background for inference, but I am not an expert in interpreting these graphs so I am not 100% sure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ueZvC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ueZvC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1To4R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1To4R.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IMIQw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IMIQw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vzNcl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vzNcl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43480" LastEditorUserId="2444" LastEditDate="2021-01-03T14:06:33.590" LastActivityDate="2021-01-04T20:09:13.440" Title="Training a classifier on different datasets with different image conditions for different labels causes the model to infer using the background" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-recognition&gt;&lt;image-segmentation&gt;&lt;domain-adaptation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25535" PostTypeId="2" ParentId="25465" CreationDate="2021-01-03T08:38:21.760" Score="1" Body="&lt;p&gt;Your problem fundamentally is that you are confusing what the state and actions are in this setting. Webpages are not your states; your state is the &lt;strong&gt;entire&lt;/strong&gt; priority queue of &lt;code&gt;(website-outlink)&lt;/code&gt; pairs + the &lt;code&gt;(new_website-outlink)&lt;/code&gt; pairs. Your action is which pair you select.&lt;/p&gt;&#xA;&lt;p&gt;Now this is a variable sized state-space and variable sized action-space problem setting at the same time. To deal with this lets start by noting that &lt;code&gt;state==observation&lt;/code&gt; need &lt;strong&gt;not&lt;/strong&gt; be (in general). So what is your observation? Your observation is a variable-sized batch of either:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;(website-outlink)&lt;/code&gt; pairs &lt;strong&gt;or&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;next_website&lt;/code&gt; (where each &lt;code&gt;next_website&lt;/code&gt; is determined by its corresponding pair)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Both of these observations could work just fine, choosing between one or the other is just a matter of whether you want your agent to learn &amp;quot;which links to open before opening them&amp;quot; or &amp;quot;which links are meaningful (after opening them)&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;What your priority queue is essentially doing is just adding a neat trick that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Saves the computational complexity of keeping the state ordered (remember that your state is not a &lt;code&gt;website&lt;/code&gt;, but the list/batch of &lt;code&gt;website-outlink&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Avoids needlessly recomputing the Q-values for each of your actions (remember that an action is not selecting an outlink from &lt;code&gt;new_website&lt;/code&gt;, but selecting an outlink from all available choices in the queue)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note however that to actually have the second saving it is crucial to store the Q-values for each pair!!!&lt;/p&gt;&#xA;&lt;p&gt;Last important thing to note is that in a scenario where you use a Replay Buffer (which I guess is likely given that you chose a DQN), you can't use the priority queue whilst learning from the RB. To see why (and to see in detail how your learning process actually looks like), start by remembering that your Q-value updates are given by the formula &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning#Algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;; your state &lt;code&gt;s_t&lt;/code&gt; is a (quasi-ordered&lt;sup&gt;1&lt;/sup&gt;) batch of pairs. &lt;code&gt;Q(s_t, a_t)&lt;/code&gt; is just the output of running your DQN regression on &lt;strong&gt;just&lt;/strong&gt; the best website/pair in this batch (you &lt;strong&gt;have&lt;/strong&gt; to add an index to denote the best choice when adding transitions to the RB, in order to be consistent about which action was taken from this state). To compute the estimate of the optimal future value however you will &lt;strong&gt;have to&lt;/strong&gt; recompute the Q-value of every single website/pair in the next state. You CANNOT use the priority queue when training from the RB.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;sub&gt; You have the priority queue ordered for all websites you had in it whilst looking at the last website, but all the &lt;code&gt;new_website-outlink&lt;/code&gt; pairs you are now adding are not ordered yet. You still have to run the agent on them and then you can order them with the rest of the priority queue to generate the next state (which still will not be ordered because you will have &lt;code&gt;new_new_website-outink&lt;/code&gt; pairs). &lt;/sub&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="42903" LastActivityDate="2021-01-03T08:38:21.760" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25537" PostTypeId="1" CreationDate="2021-01-03T09:59:18.043" Score="1" ViewCount="39" Body="&lt;p&gt;I was facing a problem I mentioned in a &lt;a href=&quot;https://ai.stackexchange.com/q/25485/36578&quot;&gt;previous question&lt;/a&gt; but after a while, I realize that maybe the problem in the dataset not in the learning rate.&lt;/p&gt;&#xA;&lt;p&gt;I build the dataset from white positions only i.e the boards when it's white's turn.&lt;/p&gt;&#xA;&lt;p&gt;Each data set consists of one game.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;First&lt;/strong&gt;, I tried to let the agent plays a game then learn from it immediately, but that did not work,&#xA;and the agent converges to one state of playing (he only lose or win against itself, or draw in a stupid way).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second&lt;/strong&gt;, I tried to let the agent plays about 1000 games against its self then train on each game separately, but that also does not work.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Note:&lt;/strong&gt;&lt;/em&gt; the first and second approach describes one iteration of the learning process, I let the agent repeat them so in total it trained on about 50000 games.&lt;/p&gt;&#xA;&lt;p&gt;Is my approach wrong? or my dataset must be built in another way? maybe train the agent on several games at once?&lt;/p&gt;&#xA;&lt;p&gt;My project is here if someone needs to take a closer look at it: &lt;a href=&quot;https://github.com/KaisHasan/CheckerBot&quot; rel=&quot;nofollow noreferrer&quot;&gt;CheckerBot&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36578" LastEditorUserId="36578" LastEditDate="2021-01-03T12:25:37.293" LastActivityDate="2021-01-03T12:25:37.293" Title="Is training on single game each time appropriate for an agent to learn to play checkers" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;training&gt;&lt;datasets&gt;&lt;checkers&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25539" PostTypeId="1" CreationDate="2021-01-03T10:33:30.580" Score="1" ViewCount="78" Body="&lt;p&gt;I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural networks, similar to the one below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/6qq6CQL.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, nodes &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt; are inputs, and &lt;code&gt;4&lt;/code&gt; is the output node, the rest of the nodes are hidden nodes. Each of these nodes can have some activation function defined for them (not necessary that all the hidden nodes have the same activation function)&lt;/p&gt;&#xA;&lt;p&gt;Now, I want to perform the forward pass of this neural network with some data, and, based on how well it performed in that task, I assign it with a fitness value, which is used as part of the NEAT evolutionary algorithm to move towards better architectures and weights.&lt;/p&gt;&#xA;&lt;p&gt;So, as part of the evolution process, I can have connections that can cause internal loops in the hidden layers and there is the possibility that a skip connection is made. Because of this, I feel the regular matrix-based forward pass (of fully connected MLPs) will not work in order to perform the forward pass of these evolved neural networks, and hence I want to know if an algorithm exists that can solve this problem.&lt;/p&gt;&#xA;&lt;p&gt;In short, I want this neural network to just take the inputs and provide me outputs - no training involved at all, so I'm not interested in the back-propagation part now.&lt;/p&gt;&#xA;&lt;p&gt;The only way to solve this that I see is to use something on the lines of a job queue (the queue will consist of the nodes that needs processing in order). I feel this is extremely inefficient and I cannot allocate this simulation method a proper stop condition. Or, even when to take output from the neural network graph and consider it.&lt;/p&gt;&#xA;&lt;p&gt;Can anybody at least point me in the right direction?&lt;/p&gt;&#xA;" OwnerUserId="43484" LastEditorUserId="2444" LastEditDate="2021-01-03T14:23:55.767" LastActivityDate="2021-01-28T08:00:20.390" Title="How can I perform the forward pass in a neural network evolved with NEAT, given that some connections may not exist or there may be loopy connections?" Tags="&lt;neural-networks&gt;&lt;neat&gt;&lt;neuroevolution&gt;&lt;fitness-functions&gt;&lt;forward-pass&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25540" PostTypeId="2" ParentId="23964" CreationDate="2021-01-03T11:09:42.703" Score="3" Body="&lt;p&gt;Another good resource is the free CatalyzeX browser extension — it adds in-line links to any relevant code wherever you come across papers on various websites: AI/ML Papers with Code Everywhere - CatalyzeX&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://chrome.google.com/webstore/detail/find-code-for-research-pa/aikkeehnlfpamidigaffhfmgbkdeheil&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chrome extension&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://addons.mozilla.org/en-US/firefox/addon/code-finder-catalyzex/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Firefox extension&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The corresponding website is &lt;a href=&quot;https://www.catalyzex.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;catalyzeX.com&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Full disclosure: I'm one of the creators. It's actively maintained and all feedback and requests are welcome!&lt;/p&gt;&#xA;" OwnerUserId="43485" LastEditorUserId="2444" LastEditDate="2021-05-10T00:20:09.167" LastActivityDate="2021-05-10T00:20:09.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25541" PostTypeId="1" CreationDate="2021-01-03T11:17:38.440" Score="0" ViewCount="26" Body="&lt;p&gt;I'm trying to understand question 4 of &lt;a href=&quot;https://www.csee.umbc.edu/courses/undergraduate/471/spring19/01/exams/mt18a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&lt;p&gt;This is a tree pruned with alpha-beta pruning:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zumxG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zumxG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I understand technically how this pruning worked (i.e. calculating mix/max/alpha/beta values). My question is, how do you interpret the output of this tree? Which way should the max player go?&lt;/p&gt;&#xA;&lt;p&gt;Watching &lt;a href=&quot;https://www.youtube.com/watch?v=l-hh51ncgDI&amp;amp;amp%3Bab_channel=SebastianLague&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; YouTube video, the best path for max is suggested to be: take the max of the first layer (below the starting below), then the minimum of the next layer and then the max.&lt;/p&gt;&#xA;&lt;p&gt;In that case:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The games starts with a point of 7&lt;/li&gt;&#xA;&lt;li&gt;Max chooses from 6, ≤5 or 7, so will choose the max of 7.&lt;/li&gt;&#xA;&lt;li&gt;Then max wants min to choose the min of 7, ≥8, ≥9, so 7.&lt;/li&gt;&#xA;&lt;li&gt;Then max will choose the max leaf node (which is 7, already implied from step above).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So I think this might be the best path, start at node 7, then choose 7 from the next layer, then choose 7 from the next layer. Can someone confirm this please?&lt;/p&gt;&#xA;&lt;p&gt;Conversely, what is the worst first move for the max player? I think it is to first play the &lt;span class=&quot;math-container&quot;&gt;$\leq 5$&lt;/span&gt; node (node B) and then I guess they only have one choice after that.&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="42926" LastEditDate="2021-01-03T19:10:43.380" LastActivityDate="2021-01-03T19:10:43.380" Title="How to interpret the output of alpha-beta pruning in terms of which position to play next in a game" Tags="&lt;search&gt;&lt;minimax&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25542" PostTypeId="1" CreationDate="2021-01-03T12:56:49.457" Score="1" ViewCount="27" Body="&lt;p&gt;Given the original paper (&lt;a href=&quot;https://arxiv.org/pdf/1809.02864.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1809.02864.pdf&lt;/a&gt;), I would like to implement the Accelegrad algorithm for which I report the pseudocode of the paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hfqd9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hfqd9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the pseudocode, the authors refer to a compact convex set &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; of diameter &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. The question is whether I can know such elements. I think that they are theoretical conditions to satisfy some theorems. The problem is that the diameter &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is used in the learning rate and also the convex set &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is used to perform the projection of the gradient descent. How can I proceed?&lt;/p&gt;&#xA;" OwnerUserId="32694" LastEditorUserId="2444" LastEditDate="2021-01-03T14:22:44.853" LastActivityDate="2021-01-03T14:22:44.853" Title="How to derive compact convex set K and its diameter D to program Accelegrad algorithm in practice?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;papers&gt;&lt;optimization&gt;&lt;gradient-descent&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25543" PostTypeId="1" CreationDate="2021-01-03T15:08:55.727" Score="0" ViewCount="134" Body="&lt;p&gt;I have been trying to solve the OpenAI lunar lander game with a DQN taken from this paper&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2006.04938v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/2006.04938v2.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The issue is that it takes 12 hours to train 50 episodes so something must be wrong.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;import random&#xA;import gym&#xA;import numpy as np&#xA;from collections import deque&#xA;import tensorflow as tf&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras.layers import Dense&#xA;from tensorflow.keras.optimizers import Adam&#xA;from tensorflow.keras import Model&#xA;&#xA;ENV_NAME = &amp;quot;LunarLander-v2&amp;quot;&#xA;&#xA;DISCOUNT_FACTOR = 0.9&#xA;LEARNING_RATE = 0.001&#xA;&#xA;MEMORY_SIZE = 2000&#xA;TRAIN_START = 1000&#xA;BATCH_SIZE = 24&#xA;&#xA;EXPLORATION_MAX = 1.0&#xA;EXPLORATION_MIN = 0.01&#xA;EXPLORATION_DECAY = 0.99&#xA;&#xA;class MyModel(Model):&#xA;    def __init__(self, input_size, output_size):&#xA;        super(MyModel, self).__init__()&#xA;        self.d1 = Dense(128, input_shape=(input_size,), activation=&amp;quot;relu&amp;quot;)&#xA;        self.d2 = Dense(128, activation=&amp;quot;relu&amp;quot;)&#xA;        self.d3 = Dense(output_size, activation=&amp;quot;linear&amp;quot;)&#xA;&#xA;    def call(self, x):&#xA;        x = self.d1(x)&#xA;        x = self.d2(x)&#xA;        return self.d3(x)&#xA;&#xA;class DQNSolver():&#xA;&#xA;    def __init__(self, observation_space, action_space):&#xA;        self.exploration_rate = EXPLORATION_MAX&#xA;&#xA;        self.action_space = action_space&#xA;        self.memory = deque(maxlen=MEMORY_SIZE)&#xA;&#xA;        self.model = MyModel(observation_space,action_space)&#xA;        self.model.compile(loss=&amp;quot;mse&amp;quot;, optimizer=Adam(lr=LEARNING_RATE))&#xA;&#xA;    def remember(self, state, action, reward, next_state, done):&#xA;        self.memory.append((state, action, reward, next_state, done))&#xA;&#xA;    def act(self, state):&#xA;        if np.random.rand() &amp;lt; self.exploration_rate:&#xA;            return random.randrange(self.action_space)&#xA;        q_values = self.model.predict(state)&#xA;        return np.argmax(q_values[0])&#xA;&#xA;    def experience_replay(self):&#xA;        if len(self.memory) &amp;lt; BATCH_SIZE:&#xA;            return&#xA;        batch = random.sample(self.memory, BATCH_SIZE)&#xA;        state_batch, q_values_batch = [], []&#xA;        for state, action, reward, state_next, terminal in batch:&#xA;            # q-value prediction for a given state&#xA;            q_values_cs = self.model.predict(state)&#xA;            # target q-value&#xA;            max_q_value_ns = np.amax(self.model.predict(state_next)[0])&#xA;            # correction on the Q value for the action used&#xA;            if terminal:&#xA;                q_values_cs[0][action] = reward&#xA;            else:&#xA;                q_values_cs[0][action] = reward + DISCOUNT_FACTOR * max_q_value_ns&#xA;            state_batch.append(state[0])&#xA;            q_values_batch.append(q_values_cs[0])&#xA;        # train the Q network&#xA;        self.model.fit(np.array(state_batch),&#xA;                        np.array(q_values_batch),&#xA;                        batch_size = BATCH_SIZE,&#xA;                        epochs = 1, verbose = 0)&#xA;        self.exploration_rate *= EXPLORATION_DECAY&#xA;        self.exploration_rate = max(EXPLORATION_MIN, self.exploration_rate)&#xA;&#xA;def lunar_lander():&#xA;    env = gym.make(ENV_NAME)&#xA;    observation_space = env.observation_space.shape[0]&#xA;    action_space = env.action_space.n&#xA;    dqn_solver = DQNSolver(observation_space, action_space)&#xA;    episode = 0&#xA;    print(&amp;quot;Running&amp;quot;)&#xA;    while True:&#xA;        episode += 1&#xA;        state = env.reset()&#xA;        state = np.reshape(state, [1, observation_space])&#xA;        scores = []&#xA;        score = 0&#xA;        while True:&#xA;            action = dqn_solver.act(state)&#xA;            state_next, reward, terminal, _ = env.step(action)&#xA;            state_next = np.reshape(state_next, [1, observation_space])&#xA;            dqn_solver.remember(state, action, reward, state_next, terminal)&#xA;            dqn_solver.experience_replay()&#xA;            state = state_next&#xA;            score += reward&#xA;            if terminal:&#xA;                print(&amp;quot;Episode: &amp;quot; + str(episode) + &amp;quot;, exploration: &amp;quot; + str(dqn_solver.exploration_rate) + &amp;quot;, score: &amp;quot; + str(score))&#xA;                scores.append(score)&#xA;                break&#xA;        if np.mean(scores[-min(100, len(scores)):]) &amp;gt;= 195:&#xA;            print(&amp;quot;Problem is solved in {} episodes.&amp;quot;.format(episode))&#xA;            break&#xA;    env.close&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    lunar_lander()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are the logs&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;root@b11438e3d3e8:~# /usr/bin/python3 /root/test.py&#xA;2021-01-03 13:42:38.055593: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1&#xA;2021-01-03 13:42:39.338231: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1&#xA;2021-01-03 13:42:39.368192: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.368693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: &#xA;pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1&#xA;coreClock: 1.8095GHz coreCount: 20 deviceMemorySize: 7.92GiB deviceMemoryBandwidth: 298.32GiB/s&#xA;2021-01-03 13:42:39.368729: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1&#xA;2021-01-03 13:42:39.370269: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10&#xA;2021-01-03 13:42:39.371430: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10&#xA;2021-01-03 13:42:39.371704: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10&#xA;2021-01-03 13:42:39.373318: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10&#xA;2021-01-03 13:42:39.374243: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10&#xA;2021-01-03 13:42:39.377939: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7&#xA;2021-01-03 13:42:39.378118: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.378702: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.379127: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0&#xA;2021-01-03 13:42:39.386525: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 3411185000 Hz&#xA;2021-01-03 13:42:39.386867: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4fb44c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:&#xA;2021-01-03 13:42:39.386891: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version&#xA;2021-01-03 13:42:39.498097: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.498786: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4fdf030 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:&#xA;2021-01-03 13:42:39.498814: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1&#xA;2021-01-03 13:42:39.498987: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.499416: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: &#xA;pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1&#xA;coreClock: 1.8095GHz coreCount: 20 deviceMemorySize: 7.92GiB deviceMemoryBandwidth: 298.32GiB/s&#xA;2021-01-03 13:42:39.499448: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1&#xA;2021-01-03 13:42:39.499483: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10&#xA;2021-01-03 13:42:39.499504: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10&#xA;2021-01-03 13:42:39.499523: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10&#xA;2021-01-03 13:42:39.499543: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10&#xA;2021-01-03 13:42:39.499562: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10&#xA;2021-01-03 13:42:39.499581: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7&#xA;2021-01-03 13:42:39.499643: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.500113: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.500730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0&#xA;2021-01-03 13:42:39.500772: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1&#xA;2021-01-03 13:42:39.915228: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:&#xA;2021-01-03 13:42:39.915298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 &#xA;2021-01-03 13:42:39.915322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N &#xA;2021-01-03 13:42:39.915568: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.916104: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero&#xA;2021-01-03 13:42:39.916555: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6668 MB memory) -&amp;gt; physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)&#xA;Running&#xA;2021-01-03 13:42:40.267699: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the GPU stats&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+-----------------------------------------------------------------------------+&#xA;| NVIDIA-SMI 450.66       Driver Version: 450.66       CUDA Version: 11.0     |&#xA;|-------------------------------+----------------------+----------------------+&#xA;| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |&#xA;| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |&#xA;|                               |                      |               MIG M. |&#xA;|===============================+======================+======================|&#xA;|   0  GeForce GTX 1080    Off  | 00000000:01:00.0  On |                  N/A |&#xA;|  0%   53C    P2    46W / 198W |   7718MiB /  8111MiB |      0%      Default |&#xA;|                               |                      |                  N/A |&#xA;+-------------------------------+----------------------+----------------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, TensorFlow does not compute on the GPU but reserves the memory so I'm assuming it's because the inputs of the neural networks are too small and it uses the CPU instead.&lt;/p&gt;&#xA;&lt;p&gt;To make sure the GPU was installed properly, I ran a sample from their documentation and it uses the GPU.&lt;/p&gt;&#xA;&lt;p&gt;Is it an issue with the algorithm or the code? Is there a way to utilize the GPU in this case?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="30875" LastActivityDate="2021-01-03T16:14:38.497" Title="Simple DQN too slow to train" Tags="&lt;reinforcement-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;q-learning&gt;&lt;open-ai&gt;" AnswerCount="2" CommentCount="5" ClosedDate="2021-01-03T17:17:22.380" ContentLicense="CC BY-SA 4.0" />
  <row Id="25544" PostTypeId="2" ParentId="25543" CreationDate="2021-01-03T16:02:28.490" Score="1" Body="&lt;p&gt;When it comes to GPU usage,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nvidia-smi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;shows the usage at the time it was executed. You should try running&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;watch -n0.01 nvidia-smi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to see the usage of GPU every 0.01 second.&#xA;It should output some small usage for current model, like 5%. You could try to increase you model, to e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.d1 = Dense(1024, input_shape=(input_size,), activation=&amp;quot;relu&amp;quot;)&#xA;self.d2 = Dense(1024, activation=&amp;quot;relu&amp;quot;)&#xA;self.d3 = Dense(output_size, activation=&amp;quot;linear&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to see if the usage of GPU increased.&lt;/p&gt;&#xA;" OwnerUserId="43110" LastActivityDate="2021-01-03T16:02:28.490" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25545" PostTypeId="2" ParentId="25543" CreationDate="2021-01-03T16:14:38.497" Score="1" Body="&lt;p&gt;Is it training at all? Or is agent performance not improving over time? Q learning can be pretty unstable. I would recommend logging the sum of rewards received by the agent at the end of each episode and the model loss to help in the debugging process. The sum of rewards will show you if the agent is improving over time and the model loss will give you a rough idea about how stable the convergence is. I would recommend using tensorboard to log these metrics (&lt;a href=&quot;https://www.tensorflow.org/tensorboard/get_started#using_tensorboard_with_other_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tensorboard/get_started#using_tensorboard_with_other_methods&lt;/a&gt;). You will be able to monitor these metrics throughout the training process. You could also just print these metrics at the end of every epoch and monitor them in your console. You really just need someway to see what's going on during training.&lt;/p&gt;&#xA;&lt;p&gt;In the paper you linked, it also mentioned double q learning, which in your code does not seem to be implemented. Vanilla q learning can have a reputation of being overoptimistic in the values that it assigns to states. This results in compounding approximation errors, which tend to destabilize learning. Using double q learning may help speed up convergence. If you need help with double q learning check out this paper: &lt;a href=&quot;https://arxiv.org/pdf/1509.06461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1509.06461.pdf&lt;/a&gt;, and this github page: &lt;a href=&quot;https://github.com/jihoonerd/Deep-Reinforcement-Learning-with-Double-Q-learning/blob/master/ddqn/agent/ddqn_agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jihoonerd/Deep-Reinforcement-Learning-with-Double-Q-learning/blob/master/ddqn/agent/ddqn_agent.py&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you use double q learning, you may have to write your own custom training loop. This can be achieved by using the gradient tape object. Make sure to wrap this new function in a tf.function decorator. This will tell the TensorFlow back-end to compile that bit of code, making it run faster (&lt;a href=&quot;https://www.tensorflow.org/guide/function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/guide/function&lt;/a&gt;). There are also some handy speed up tips in this post (&lt;a href=&quot;https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic&lt;/a&gt;). They even wrap the environment step functions in tf.functions.The article uses actor-critic, which is a combination of policy gradient and q learning techniques, but you can swap out their neural network update code with the q learning functionality that you need.&lt;/p&gt;&#xA;&lt;p&gt;If you need help with double q learning check out this paper: &lt;a href=&quot;https://arxiv.org/pdf/1509.06461.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1509.06461.pdf&lt;/a&gt;, and this github page: &lt;a href=&quot;https://github.com/jihoonerd/Deep-Reinforcement-Learning-with-Double-Q-learning/blob/master/ddqn/agent/ddqn_agent.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jihoonerd/Deep-Reinforcement-Learning-with-Double-Q-learning/blob/master/ddqn/agent/ddqn_agent.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40428" LastActivityDate="2021-01-03T16:14:38.497" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25547" PostTypeId="2" ParentId="12671" CreationDate="2021-01-03T20:36:56.073" Score="0" Body="&lt;p&gt;As opposed to what is written &lt;a href=&quot;https://ai.stackexchange.com/a/12672/2444&quot;&gt;in this answer&lt;/a&gt;, you can have the analytical expression of the function that the neural network computes, even if that neural network computes a non-linear function. Take a look at &lt;a href=&quot;https://ai.stackexchange.com/a/25460/2444&quot;&gt;this example&lt;/a&gt;: i.e. it is just the expression that you use to perform the forward pass of the neural network. The problem is how to interpret this function, how it generally behaves, and how it is different from the usually unknown function that the neural network is supposed to approximately compute: that's why neural networks are called &lt;em&gt;black-box&lt;/em&gt; models, because they are not easily interpretable.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2021-01-03T20:36:56.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25548" PostTypeId="1" CreationDate="2021-01-03T21:49:53.823" Score="0" ViewCount="24" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://web.stanford.edu/%7Ehastie/Papers/tangent.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper, that is discussing the use of distance metrics for character recognition predicton.&lt;/p&gt;&#xA;&lt;p&gt;I can see the advantages of using a distance metrics in predictions like character recognition: You can model a set of known character's pixel vectors, and then measure a new unseen vector against this model, get the distance, and if the distance is low, predict that this unseen vector belongs to a particular class.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if there's any disadvantages to using distance metrics as the cost function in character recognition? For example, I was thinking maybe the distance calculation is slow for large images (would have to calculate the distance between each item in two long image vectors)?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastActivityDate="2021-02-03T02:05:25.543" Title="What are the disadvantages to using a distance metric in character recognition prediction" Tags="&lt;optical-character-recognition&gt;&lt;metric&gt;&lt;handwritten-characters&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25549" PostTypeId="5" CreationDate="2021-01-03T23:08:53.383" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-03T23:08:53.383" LastActivityDate="2021-01-03T23:08:53.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25550" PostTypeId="4" CreationDate="2021-01-03T23:08:53.383" Score="0" Body="For questions related to probability theory in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-03T23:08:53.383" LastActivityDate="2021-01-03T23:08:53.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25551" PostTypeId="2" ParentId="25548" CreationDate="2021-01-04T00:27:53.587" Score="0" Body="&lt;p&gt;As I see it, the question boils down to the comparison between distance (function/metric) based &lt;em&gt;Optical Character Recognition&lt;/em&gt; (OCR) and (for example) OCR done by means of Convolutional Neural Networks (CNNs). Particularly, it focuses on the cons of the former option.&lt;/p&gt;&#xA;&lt;p&gt;There are a few potential problems associated with using distance based OCR systems.&#xA;First of all, this approach requires an appropriate distance metric to deliver good results. Different types of distance metrics/functions are sensitive to different features in the input images. For example, some functions might penalize absolute differences, while others penalize squared differences, where the latter punishes differences of magnitude &lt;span class=&quot;math-container&quot;&gt;$&amp;gt; 1$&lt;/span&gt; stronger than the former, while the former penalizes differences of (absolute) magnitude &lt;span class=&quot;math-container&quot;&gt;$&amp;lt; 1$&lt;/span&gt; stronger than the latter. Which type of distance metric works best for a given problem has commonly to be determined empirically.&lt;/p&gt;&#xA;&lt;p&gt;Also, distance based OCR systems may possibly be more sensitive to, for example, different levels in illumination than CNNs. While both sorts of classifiers could profit from data augmentation (i.e. adding variations of the existing training data to the dataset), OCR based on CNNs has the benefit that CNN training procedures produce classifiers that commonly generalize well to slight variations in the incoming data, while some slight novel tilt (or again variation in illumination) may break the distance based classification procedure, but may have not too detrimental effects on CNN based classifiers due to their rather strong generalization capabilities.&lt;/p&gt;&#xA;&lt;p&gt;Of course, one can also try to increase the robustness of distance based OCR systems, but this is commonly associated with developing exhaustive preprocessing pipelines to standardize the appearance of incoming character images.&#xA;Thus, in terms of system design, it is often easier to set up a CNN-based neural network architecture and train it (possibly drawing upon regularization to boost generalization of the trained system even further) than trying to design complicated preprocessing strategies to shift, rotate, and normalize character images exhaustively to boost performance of a distance based OCR system.&lt;/p&gt;&#xA;&lt;p&gt;To sum up, after all, there are a lot of potential issues related to using plain distance (metric/function) based OCR systems, most of which, however, can strongly be alleviated when drawing upon clever (pre-)processing pipelines to standardize input images before performing distance based classification.&lt;/p&gt;&#xA;&lt;p&gt;From all those issues mentioned above, the strongest disadvantage of this technique, however, (which cannot be alleviated so easily) is that it doesn't generalize too easily to novel input data, where CNNs might perform better with appropriate training data (+ augmentation and regularization in general).&lt;/p&gt;&#xA;&lt;p&gt;You also mentioned the aspect of computational cost associated with distance based classification. First of all, I am not an expert in computational complexity/cost related questions. However, most image related computations can efficiently be executed on graphics cards. So, subtracting images pixel-wise, for example,is not very costly given it can be executed on a graphics card. But of course, when using distance based OCR, the computational cost associated with computing distances and comparing distance values scales with the reference dataset size.&#xA;Using CNNs, the time needed to perform a classification is always the same irrespective of the training dataset size.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2021-01-04T00:27:53.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25552" PostTypeId="2" ParentId="25481" CreationDate="2021-01-04T00:39:25.130" Score="1" Body="&lt;p&gt;In image segmentation the target is actually an image, with the same dimensions as the input, where each pixel has a label depending on which class it represents. It is not uncommon for such a dataset to have a &amp;quot;background&amp;quot; class that essentially consists of the pixels not belonging to any other class. If not you can always group together classes typically associated with background (e.g. &amp;quot;sky&amp;quot;, &amp;quot;cloud&amp;quot;, &amp;quot;grass&amp;quot;, &amp;quot;mountain&amp;quot;, etc.) to form the class &amp;quot;background&amp;quot;. Likewise you could group all other possible classes of interest (e.g. &amp;quot;person&amp;quot;, &amp;quot;car&amp;quot;, &amp;quot;horse&amp;quot;, etc.) into the class &amp;quot;foreground&amp;quot;. With this dataset you could train an image segmentation model that predicts if a pixel belongs to the background or the foreground, without actually classifying it into a &amp;quot;person&amp;quot; or a &amp;quot;car&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;So suppose you want to make your own removal.ai, you could:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;find one or more &lt;strong&gt;diverse&lt;/strong&gt; image segmentation datasets (it needs to be diverse so that it will work on any generic photo uploaded to the site)&lt;/li&gt;&#xA;&lt;li&gt;check all the unique classes in the labels&lt;/li&gt;&#xA;&lt;li&gt;group all classes associated with backgrounds into class 0 (i.e. &amp;quot;background&amp;quot; class)&lt;/li&gt;&#xA;&lt;li&gt;group all classes associated with foregrounds into class 1 (i.e. &amp;quot;foreground&amp;quot; class)&lt;/li&gt;&#xA;&lt;li&gt;train an image segmentation model with these two classes&lt;/li&gt;&#xA;&lt;li&gt;profit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2021-01-04T00:39:25.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25553" PostTypeId="1" CreationDate="2021-01-04T01:43:00.253" Score="5" ViewCount="136" Body="&lt;p&gt;I was studying the off-policy policy improvement method. Then I encountered &lt;em&gt;importance sampling&lt;/em&gt;. I completely understood the mathematics behind the calculation, but I am wondering what is the practical example of importance sampling.&lt;/p&gt;&#xA;&lt;p&gt;For instance, in a &lt;a href=&quot;https://www.youtube.com/watch?v=V8f8ueBc9sY&amp;amp;t=509s&amp;amp;ab_channel=BenLambert&quot; rel=&quot;nofollow noreferrer&quot;&gt;video&lt;/a&gt;, it is said that we need to calculate the expected value of a biased dice, here &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt;, in terms of the expected value of fair dice, &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. Here is a screenshot of the video.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TuBMq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TuBMq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why do we need that, when we have the probability distribution of the biased dice?&lt;/p&gt;&#xA;" OwnerUserId="43507" LastEditorUserId="2444" LastEditDate="2021-01-04T22:21:48.883" LastActivityDate="2021-04-07T09:40:46.220" Title="Why do we need importance sampling?" Tags="&lt;reinforcement-learning&gt;&lt;monte-carlo-methods&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25555" PostTypeId="1" CreationDate="2021-01-04T06:26:43.303" Score="2" ViewCount="44" Body="&lt;p&gt;The basic approach to non-maximum-suppression makes sense, but I am kind of confused about how you handle nested bounding boxes.&lt;/p&gt;&#xA;&lt;p&gt;Suppose you have two predicted boxes, with one completely enclosing another. What happens under this circumstance (in particular, in the case of the Single Shot MultiBox Detector)? Which bounding box do you select?&lt;/p&gt;&#xA;" OwnerUserId="32390" LastEditorUserId="2444" LastEditDate="2021-01-07T17:55:54.980" LastActivityDate="2021-01-07T17:55:54.980" Title="How are nested bounding boxes handled in object detection (and in particular in the case of the SSD)?" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;&lt;non-max-suppression&gt;&lt;single-shot-multibox-detector&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25557" PostTypeId="1" CreationDate="2021-01-04T06:47:20.230" Score="1" ViewCount="131" Body="&lt;p&gt;Let us assume that I am working on a dataset of black and white dog images.&lt;/p&gt;&#xA;&lt;p&gt;Each image is of size &lt;span class=&quot;math-container&quot;&gt;$28 \times 28$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, I can say that I have a sample space &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; of all possible images. And &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; is the probability distribution for dog images. It is easy to understand that all other images get a probability value of zero. And it is obvious that &lt;span class=&quot;math-container&quot;&gt;$n(S)= 2^{28 \times 28}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, I am going to design a generative model that sample from &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; using &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; rather than random sampling.&lt;/p&gt;&#xA;&lt;p&gt;My generative model is a neural network that takes random noise (say, of length 100) and generates an image of the size &lt;span class=&quot;math-container&quot;&gt;$28 \times 28$&lt;/span&gt;. My function is learning a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, which is totally different from the function &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;. It is because of the reason that  &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is from &lt;span class=&quot;math-container&quot;&gt;$R^{100}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; is from &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the literature, I often read the phrases that &lt;strong&gt;our generative model learned &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;&lt;/strong&gt; or &lt;strong&gt;our goal is to get &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;&lt;/strong&gt;, etc.,  but in fact, they are trying to learn &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, which just obeys &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt; while giving its output.&lt;/p&gt;&#xA;&lt;p&gt;Am I going wrong anywhere or the usage in literature is somewhat random?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastActivityDate="2021-01-04T08:16:22.167" Title="Confusion between function learned and the underlying distribution" Tags="&lt;terminology&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25558" PostTypeId="2" ParentId="25557" CreationDate="2021-01-04T08:16:22.167" Score="3" Body="&lt;p&gt;You're right! The generative model &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is not the same as the probability density  (p.d.f.) function &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;. The kind of phrases you've referred to are to be interpreted informally. You learn &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with the hope that sampling a latent vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;&#xA;from some known distribution (from which it is easy to sample), results in &lt;span class=&quot;math-container&quot;&gt;$f(z)$&lt;/span&gt; that has the probability density function &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;. However, merely learning &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; does not give you the power to estimate what &lt;span class=&quot;math-container&quot;&gt;$p_{data}(x)$&lt;/span&gt; is for some image &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Learning &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; only gives you the power to sample according to &lt;span class=&quot;math-container&quot;&gt;$p_{data}(\cdot)$&lt;/span&gt; (if you've learned an accurate such &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="36974" LastActivityDate="2021-01-04T08:16:22.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25559" PostTypeId="2" ParentId="25553" CreationDate="2021-01-04T09:20:04.427" Score="7" Body="&lt;p&gt;Importance sampling is typically used when the distribution of interest is difficult to sample from - e.g. it could be computationally expensive to draw samples from the distribution - or when the distribution is only known up to a multiplicative constant, such as in Bayesian statistics where it is intractable to calculate the marginal likelihood; that is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)} \propto p(x|\theta)p(\theta)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p(x)$&lt;/span&gt; is our marginal likelihood that may be intractable and so we can't calculate the full posterior and so other methods must be used to generate samples from this distribution. When I say intractable, note that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(x) = \int_{\Theta} p(x|\theta)p(\theta) d\theta$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and so intractable here means that either a) the integral has no analytical solution or b) a numerical method for computing this integral may be too expensive to run.&lt;/p&gt;&#xA;&lt;p&gt;In the instance of your die example, you are correct that you could calculate the theoretical expectation of the bias dice analytically and this would probably be a relatively simple calculation. However, to motivate why importance sampling may be be useful in this scenario, consider calculating the expectation using Monte Carlo methods. It would be much simpler to uniformly sample a random integer from 1-6 and calculate the importance sampling ratio &lt;span class=&quot;math-container&quot;&gt;$x \frac{g(x)}{f(x)}$&lt;/span&gt; than it would be to draw samples from the bias dice, not least because most programming languages have built in methods to randomly sample integers.&lt;/p&gt;&#xA;&lt;p&gt;As your question is tagged as reinforcement learning I will add why it is useful in the RL domain. One reason is that it may be our policy of interest is expensive to sample from, so instead we can just generate actions from some other simple policy whilst still learning about the policy of interest. Second, we could be interested in a policy that is deterministic (greedy) but still be able to explore, so we can have an off-policy distribution that explores much more frequently.&lt;/p&gt;&#xA;&lt;p&gt;NB: it may not be clear how you can use importance sampling if the distribution is only known up to a constant so see &lt;a href=&quot;https://ai.stackexchange.com/a/21360/36821&quot;&gt;this answer&lt;/a&gt; for an explanation.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-04-07T09:40:46.220" LastActivityDate="2021-04-07T09:40:46.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25560" PostTypeId="1" CreationDate="2021-01-04T09:50:57.077" Score="1" ViewCount="58" Body="&lt;p&gt;For action recognition or similar tasks, one can either use 3D CNN or combine 2D CNN with optical flow. See &lt;a href=&quot;https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Tran_Learning_Spatiotemporal_Features_ICCV_2015_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for details.&lt;/p&gt;&#xA;&lt;p&gt;Can someone tell the pros/cons of each, in terms of accuracy, cost such as computation and memory requirement, etc.? In other words, is the computation overhead of 3D CNN justified by its accuracy improvement? Under what scenarios would one prefer one over another?&lt;/p&gt;&#xA;&lt;p&gt;3D CNNs are also used for volumetric data, such as MRI images. Can 2D CNN + optical flow be used here?&lt;/p&gt;&#xA;&lt;p&gt;I understand 2D CNNs and 3D CNNs, but I do not know about optical flow (my background is not computer-vision).&lt;/p&gt;&#xA;" OwnerUserId="20491" LastEditorUserId="2444" LastEditDate="2021-01-05T12:03:30.327" LastActivityDate="2021-01-05T12:03:30.327" Title="What are the pros and cons of 3D CNN and 2D CNN combined with optical flow for action recognition?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;comparison&gt;&lt;action-recognition&gt;&lt;optical-flow&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25561" PostTypeId="1" CreationDate="2021-01-04T10:38:43.160" Score="4" ViewCount="108" Body="&lt;p&gt;I've been working on research into reproducing social behavior using multi-agent reinforcement learning. My focus has been on &lt;a href=&quot;https://github.com/cool-RR/grid_royale&quot; rel=&quot;nofollow noreferrer&quot;&gt;a GridWorld-style game&lt;/a&gt;, but I was thinking that maybe a simpler Prisoner's Dilemma game could be a better approach. I tried to find existing research papers in this direction, but couldn't find any, so I'd like to describe what I'm looking for in case anyone here knows of such research.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I'm looking for research into scenarios where multiple RL agents are playing Iterated Prisoner's Dilemma with each other, and social behaviors emerge.&lt;/strong&gt; Let me specify what I mean by &amp;quot;social behaviors.&amp;quot; Most research I've seen into RL/IPD (&lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0188046&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;) focuses on how to achieve the ideal strategy, and how to get there the fastest, and what common archetypes of strategies emerge. That is all nice and well, but not what I'm interested in.&lt;/p&gt;&#xA;&lt;p&gt;An agent executing a Tit-for-Tat strategy is giving positive reinforcement to the other player for &amp;quot;good&amp;quot; behavior, and negative reinforcement for &amp;quot;bad&amp;quot; behavior. That is why it wins. My key point here is that this carrot-and-stick method is done individually rather than in groups. &lt;strong&gt;I want to see it evolve within a group.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to see an entire group of agents evolve to punish and reward other players according to how they behaved with the group. I believe that fascinating group dynamics could be observed in that scenario.&lt;/p&gt;&#xA;&lt;p&gt;I programmed such a scenario a decade ago, but by writing an algorithm manually, not using deep RL. I want to do it using deep RL, but first I want to know whether there are existing attempts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Does anyone know whether such research exists?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="25904" LastEditorUserId="2444" LastEditDate="2021-01-12T00:01:55.590" LastActivityDate="2021-02-11T16:02:25.217" Title="Research into social behavior in Prisoner's Dilemma" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;reference-request&gt;&lt;research&gt;&lt;prisoner-dilemma&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25562" PostTypeId="1" CreationDate="2021-01-04T11:05:12.093" Score="0" ViewCount="46" Body="&lt;p&gt;I was wondering which AI techniques and architectures are used in environments that need predictions to continually improve by the feedback of the user.&#xA;So let's take some kind of recommendation system, but not for a number of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; products, but for some problem of higher space. It's initially trained, but should keep improving by the feedback and corrections applied by the user. The system should continue to improve its outcomes on-the-fly in production, with each interaction.&lt;/p&gt;&#xA;&lt;p&gt;Obviously, (deep) RL seems to fit this problem, but can you really deploy this learning process to production? Is it really capable of improving results on-the-fly?&lt;/p&gt;&#xA;&lt;p&gt;Are there any other techniques or architectures that can be used for that?&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for different approaches in general, in order to be able to compare them and find the right one for problems of that kind. Of course, there always is the option to retrain the whole network, but I was wondering whether there are some online, on-the-fly techniques that can be used to adjust the network?&lt;/p&gt;&#xA;" OwnerUserId="43542" LastEditorUserId="2444" LastEditDate="2021-01-04T15:19:31.127" LastActivityDate="2021-01-04T15:19:31.127" Title="What are good techniques for continuous learning in production?" Tags="&lt;reinforcement-learning&gt;&lt;incremental-learning&gt;&lt;active-learning&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25563" PostTypeId="1" CreationDate="2021-01-04T12:27:34.150" Score="1" ViewCount="17" Body="&lt;p&gt;When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;https://www.cs.toronto.edu/%7Eurtasun/courses/CSC411/10_nn1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (slide 20), given &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; (the output, defined in slide 18), &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; (the activated output) and a target &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, defined in slide 17, then:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\delta E}{\delta Z} = o(1-o)(o-t)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When I look for the same equation else, e.g. &lt;a href=&quot;https://studylib.net/doc/7329292/311--neural-networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, slide 14, it says, given &lt;span class=&quot;math-container&quot;&gt;$o$&lt;/span&gt; the output and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; the label, then (using slightly different notation &lt;span class=&quot;math-container&quot;&gt;$\beta_k$&lt;/span&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\beta_k = o_k(1-o_k)(y_k-o_k)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can see here that these two equations are almost the same, but not quite. One subtracts the output from the target, and one subtracts the target from the output.&lt;/p&gt;&#xA;&lt;p&gt;The reason why I'm asking this is I'm trying to do question 29 and 30 of &lt;a href=&quot;http://130.243.105.49/%7Etdt/ann/tentor/exam2002.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper, and they are using the second equation (&lt;span class=&quot;math-container&quot;&gt;$\beta_k$&lt;/span&gt;) but my college notes (that I can't copy and paste due to copyright) define the equation according to the first equation &lt;span class=&quot;math-container&quot;&gt;$\frac{\delta E}{\delta Z}$&lt;/span&gt;. I'm wondering which way is correct, do you subtract the target from the obtained output, or vice versa?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastEditorUserId="2444" LastEditDate="2021-01-05T00:31:48.117" LastActivityDate="2021-01-05T00:31:48.117" Title="For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?" Tags="&lt;neural-networks&gt;&lt;backpropagation&gt;&lt;calculus&gt;&lt;chain-rule&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25564" PostTypeId="1" CreationDate="2021-01-04T12:40:12.073" Score="0" ViewCount="25" Body="&lt;p&gt;I have a time series with both continuous and categorical features, and I want to do a prediction task.&lt;/p&gt;&#xA;&lt;p&gt;I will elaborate:&lt;/p&gt;&#xA;&lt;p&gt;The data is composed of 100Hz sampling of some voltages, kind of like an ecg signal, and of some categorical features such as &amp;quot;green&amp;quot;, &amp;quot;na&amp;quot; and so on.&lt;/p&gt;&#xA;&lt;p&gt;In total, the number of features can reach 300, of which most are continuous.&lt;/p&gt;&#xA;&lt;p&gt;The prediction should take in a chunk of frames and predict a categorical variable for this chunk of frames.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I want to create a deep learning model that can handle both categorical and continuous features.&lt;/p&gt;&#xA;&lt;p&gt;Best I can think of is two separate losses, like MSE and cross entropy, and a hyperparameter to tune between them, kind of like regularization.&lt;/p&gt;&#xA;&lt;p&gt;Best I could find on this subject was &lt;a href=&quot;https://stats.stackexchange.com/questions/26764/predicting-with-both-continuous-and-categorical-features&quot;&gt;this&lt;/a&gt;, with an answer from 2015.&lt;/p&gt;&#xA;&lt;p&gt;I wonder if something better was invented, since then, or maybe just someone here knows something better.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="21645" LastEditDate="2021-01-05T10:47:03.757" LastActivityDate="2021-01-05T10:47:03.757" Title="Correct way to work with both categorical and continuous features together" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;loss-functions&gt;&lt;categorical-data&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25567" PostTypeId="2" ParentId="24666" CreationDate="2021-01-04T14:33:02.267" Score="0" Body="&lt;p&gt;This should not matter that much as long as you do enough preprocessing?&lt;/p&gt;&#xA;&lt;p&gt;But one could use e.g. the Facenet architecture to extract the embedding vector for each face of the same subject.&lt;/p&gt;&#xA;&lt;p&gt;Then you could compute the covariance matrix and perform PCA on it. This would give you the most significant features with which you can decide which face to take.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you could do something in the direction of the eigenfaces: &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenface&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Eigenface&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42601" LastActivityDate="2021-01-04T14:33:02.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25569" PostTypeId="5" CreationDate="2021-01-04T15:17:25.163" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-04T15:17:25.163" LastActivityDate="2021-01-04T15:17:25.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25570" PostTypeId="4" CreationDate="2021-01-04T15:17:25.163" Score="0" Body="For questions related to the &quot;Single Shot MultiBox Detector&quot; (SSD), proposed in &quot;SSD: Single Shot MultiBox Detector&quot; (2016) by Wei Liu et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-04T15:17:25.163" LastActivityDate="2021-01-04T15:17:25.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25571" PostTypeId="5" CreationDate="2021-01-04T15:22:43.037" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-04T15:22:43.037" LastActivityDate="2021-01-04T15:22:43.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25572" PostTypeId="4" CreationDate="2021-01-04T15:22:43.037" Score="0" Body="For questions related to latent Dirichlet allocation (LDA), a generative probabilistic model for collections of&#xD;&#xA;discrete data such as text corpora. Take a look at &quot;Latent Dirichlet Allocation&quot; (2003, JMLR) by David M. Blei et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-04T15:22:43.037" LastActivityDate="2021-01-04T15:22:43.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25573" PostTypeId="1" CreationDate="2021-01-04T16:10:45.943" Score="2" ViewCount="81" Body="&lt;p&gt;Say I have an Machine/Deep learning algorithm I developed on a desktop pc to achieve a real-time classification of time series events from a sensor. Once the algorithm is trained and performs good, I want to implement it on an low power embbeded system, with the same sensor, to classify events in real-time:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I know if the low power embedded system is fast enough to allow real-time classification regarding the algorithm (knowing it in advance would avoid to implement and try multiple architectures) ?&lt;/li&gt;&#xA;&lt;li&gt;Machine/Deep learning algorithm are usually developed in python. Is there easy ways to transfer the code from python to a more embeddable langage ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="43239" LastEditorUserId="43239" LastEditDate="2021-01-11T08:49:35.373" LastActivityDate="2021-01-11T08:49:35.373" Title="How to know if a real-time classifier is achivable in a low-power emdedded system?" Tags="&lt;ai-design&gt;&lt;computational-complexity&gt;&lt;embedded-design&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25575" PostTypeId="2" ParentId="25522" CreationDate="2021-01-04T19:36:42.503" Score="2" Body="&lt;p&gt;Formally speaking &lt;span class=&quot;math-container&quot;&gt;$x_6$&lt;/span&gt; is a function of &lt;span class=&quot;math-container&quot;&gt;$w_{16},\ w_{26}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{36}$&lt;/span&gt;, that is &lt;span class=&quot;math-container&quot;&gt;$$x_6 =f(w_{16}, w_{26}, w_{36})=w_{16}y_1 + w_{26}y_2 + w_{36}y_3.$$&lt;/span&gt;&#xA;The derivative w.r.t. &lt;span class=&quot;math-container&quot;&gt;$w_{26}$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial x_6}{\partial w_{26}}= \frac{\partial w_{16}y_1}{\partial w_{26}} +\frac{\partial w_{26}y_2}{\partial w_{26}}  +\frac{\partial w_{36}y_3}{\partial w_{26}} = 0 +y_2 \frac{\partial w_{26}}{\partial w_{26}} + 0= y_2.$$&lt;/span&gt;&#xA;The first equality is obtained using the fact that the partial derivative is linear (so the derivative of the sum is the sum of the derivatives); the second equality comes from again from the linearity and from the fact that &lt;span class=&quot;math-container&quot;&gt;$w_{16}y_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{36}y_3$&lt;/span&gt; are constants with respect to &lt;span class=&quot;math-container&quot;&gt;$w_{26}$&lt;/span&gt;, so their partial derivative w.r.t. this variable is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Bonus&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Not really asked in the original question, but since I'm here let me have a bit of fun ;).&lt;/p&gt;&#xA;&lt;p&gt;Let's say &lt;span class=&quot;math-container&quot;&gt;$x_6$&lt;/span&gt; is the output of the sixth node after you apply an activation function, that is &lt;span class=&quot;math-container&quot;&gt;$$x_6 =\sigma(f(w_{16}, w_{26}, w_{36}))=\sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3).$$&lt;/span&gt;&#xA;You can compute the partial derivative applying the properties illustrated above, with the additional help of the chain rule&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial x_6}{\partial w_{26}}=\frac{\partial \sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3)}{\partial w_{26}}=\sigma'\frac{\partial w_{16}y_1}{\partial w_{26}} +\sigma'\frac{\partial w_{26}y_2}{\partial w_{26}}  +\sigma'\frac{\partial w_{36}y_3}{\partial w_{26}}=y_2\sigma'$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\sigma'$&lt;/span&gt; denotes the derivative of sigma with respect to its argument.&lt;/p&gt;&#xA;" OwnerUserId="42424" LastEditorUserId="2444" LastEditDate="2021-01-04T22:42:20.240" LastActivityDate="2021-01-04T22:42:20.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25576" PostTypeId="1" CreationDate="2021-01-04T19:49:58.197" Score="1" ViewCount="23" Body="&lt;p&gt;I am re-implementing vpg and using Spinning Up as reference implementation. I noticed that the default epoch size is &lt;a href=&quot;https://github.com/openai/spinningup/blob/038665d62d569055401d91856abb287263096178/spinup/algos/pytorch/vpg/vpg.py#L89&quot; rel=&quot;nofollow noreferrer&quot;&gt;4000&lt;/a&gt;. I also see cues in papers that big batch size is quite standard.&lt;/p&gt;&#xA;&lt;p&gt;My implementation doesn't batch XP together just applies the update after every episode. It turns out my implementation is more sample efficient than the reference implementation on simple problems (like CartPole or LunarLander) even though I haven't added the critic yet! Of course this could be due to number of reasons, for example I've only done parameter search on my implementation.&lt;/p&gt;&#xA;&lt;p&gt;But it would make sense anyway: bigger batch size is generally considered better only because the GPU is faster processing many samples parallel. Is this the reason here? It would make sense but it is surprising for me as I thought sample efficiency is considered more important than computing efficiency in RL.&lt;/p&gt;&#xA;" OwnerUserId="43565" LastActivityDate="2021-01-04T19:49:58.197" Title="Why do we use big batch/epoch size in policy gradient methods (vpg specifically)?" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;actor-critic-methods&gt;&lt;reinforce&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25577" PostTypeId="2" ParentId="25534" CreationDate="2021-01-04T20:09:13.440" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I am afraid that the model will infer from the background information that it shouldn't use to predict the plant diseases, what makes the problem worse is that some plant diseases only exist in one dataset and not the other&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I am afraid that when I use the model on real-life conditions (say someone capturing an image with their phone) the model will be heavily biased towards predicting labels in the second dataset (the one with actual background) since the background is similar and it somehow associated the background with the inference process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are right to be concerned about this. However, you are not alone in having a neural network model that relies heavily on the background or other confounding factors for your classification task.&lt;/p&gt;&#xA;&lt;p&gt;A small paper that addresses a similar issue is &lt;a href=&quot;https://arxiv.org/pdf/1712.01653.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Context Augmentation for Convolutional Neural Networks&lt;/a&gt;. Table 1 from this paper shows how the model trained and tested on only the background performs &lt;em&gt;better&lt;/em&gt; than model trained and tested on only the foreground. They address this by making a training set where the foreground objects are given many backgrounds. Given your imbalanced datasets, with some diseases only present in one dataset and not the other, I think context augmentation might work for you. The main barrier here is that you do not have segmentation maps for these. Are your datasets too large to segment yourself? You might consider automatic segmentation networks - especially for the images with white backgrounds.&lt;/p&gt;&#xA;&lt;p&gt;As for your interpretability methods, you should &lt;a href=&quot;https://arxiv.org/abs/1810.03292&quot; rel=&quot;nofollow noreferrer&quot;&gt;not expect too much from them&lt;/a&gt; though &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8760405?casa_token=Xfnqz3RKMN4AAAAA:MQzbNdTrx_rytJ4o9DTp8MZWEZ-i85EDLGWQrXVhGTZR8PSdXFvHHoJvBMWgh7ntO69Xlz3eZ2A&quot; rel=&quot;nofollow noreferrer&quot;&gt;some groups are able to get good results with them&lt;/a&gt; using GradCAM.&lt;/p&gt;&#xA;&lt;p&gt;Here is an alternative idea that may also solve your problem: If you are trying to make a system that can be used in the field, is it possible to have the technician or photographer put a white board under the diseased leaf?&lt;/p&gt;&#xA;" OwnerUserId="21471" LastActivityDate="2021-01-04T20:09:13.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25578" PostTypeId="2" ParentId="25516" CreationDate="2021-01-04T20:19:51.457" Score="0" Body="&lt;p&gt;It sounds like you may be looking for the &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;A* search algorithm&lt;/a&gt;. It is a search algorithm, like DFS and BFS, but it will explore only the most promising branches based on a heuristic function you supply. The difficult part of implementing this is deciding on a low-cost, admissible heuristic.&lt;/p&gt;&#xA;&lt;p&gt;Excited to reflect nbro's suggestion from comments.&lt;/p&gt;&#xA;" OwnerUserId="21471" LastEditorUserId="21471" LastEditDate="2021-01-05T16:16:10.200" LastActivityDate="2021-01-05T16:16:10.200" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25579" PostTypeId="1" CreationDate="2021-01-04T21:55:45.073" Score="0" ViewCount="14" Body="&lt;p&gt;I understand that this is the method for conducting back-propogation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;With a three layer network (input layer, one hidden layer, one output layer); start with input &lt;span class=&quot;math-container&quot;&gt;$I_i$&lt;/span&gt; as an exemplar input.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the output for this input (by multiplying by weights, checking against thresholds, and propogating through the network to the output); output = &lt;span class=&quot;math-container&quot;&gt;$O_k$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$y_k$&lt;/span&gt; is the target output for &lt;span class=&quot;math-container&quot;&gt;$O_k$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To update the weights leading to this output in the final layer (i.e. &lt;span class=&quot;math-container&quot;&gt;$w_{jk}$&lt;/span&gt;; between the one hidden layer and the output layer):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculate &lt;span class=&quot;math-container&quot;&gt;$\frac{\delta E}{\delta X_k} = y_k(1-y_k)(y_k-O_k)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Calculate &lt;span class=&quot;math-container&quot;&gt;$\frac{\delta E}{\delta w_{jk}} = \frac{\delta E}{\delta X_k}y_j$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;For all j,k; &lt;span class=&quot;math-container&quot;&gt;$w_{jk} := w_{jk} - C\frac{\delta E}{\delta w_{jk}}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Then for the &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt; weights (i.e. the weights between the input layer and the hidden layer):&lt;/p&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Calculate all the &lt;span class=&quot;math-container&quot;&gt;$\frac{\delta E}{\delta X_j} = y_j(1-y_j) \sum_k \frac{\delta E}{\delta X_k} w_{jk} $&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Calculate all the &lt;span class=&quot;math-container&quot;&gt;$\frac{\delta E}{\delta w_{ij}} = \frac{\delta E}{\delta x_j}I_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;For all i,j: &lt;span class=&quot;math-container&quot;&gt;$ w_{ij} = w_{ij} - C\frac{\delta E}{\delta w_{jk}}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I understand:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I understand what step 3 and 6 is doing. You have the weight, a learning rate (&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;) and the error you've calculated to be associated with that weight, and you're updating the weight to reflect the error.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For step 1,2,4 and 5, I understand the left hand side of each equation: 1 is measuring the change in total error for that input X, 2 is measuring total error for the weight &lt;span class=&quot;math-container&quot;&gt;$w_{jk}$&lt;/span&gt;, 5 is measuring the total error for the weight &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt; and 4 is measuring the error for that X value.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I don't understand is, in plain basic english, what the right hand side of equations 1,2,4 and 5 are doing. Can someone else with this?&lt;/p&gt;&#xA;" OwnerUserId="42926" LastActivityDate="2021-01-04T21:55:45.073" Title="Explanation of the partial derivatives in back-propogation algorithm" Tags="&lt;backpropagation&gt;&lt;algorithm&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25581" PostTypeId="1" CreationDate="2021-01-04T23:32:45.780" Score="0" ViewCount="10" Body="&lt;p&gt;My questions centers around what method is best to use parametrize a response function for an experiment. We are currently using &lt;em&gt;ab initio&lt;/em&gt; simulation to model our experiment's response. Unfortunately, this is very compute intensive and wasteful, e.g. 500x compute time for every second we run the experiment.&lt;/p&gt;&#xA;&lt;p&gt;Our input would be a set of parameters random chosen from a known and samplable PDF. From this vector we want to skip the &lt;em&gt;ab initio&lt;/em&gt; simulation. Rather we would like to put this into an AI model to give us a statistically likely result, either by allowing us to do importance sampling (answer &amp;quot;Will this actually show up in our experiment?&amp;quot;) and/or give us a statistically likely output of the &lt;em&gt;ab initio&lt;/em&gt; simulation (answer &amp;quot;If it shows up, what does it show up as?&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;We have been able to parametrize a subset of the simulation outputs (the second question above) using B-splines. Adding the missing pieces to the splines has proven prohibitive at this stage. We want to try an AI method to do this parametrization.&lt;/p&gt;&#xA;&lt;p&gt;To throw a wrench into the whole thing is that we care about the (long) tails of the distribution, i.e. the rare things are interesting. VAEs, GANs, and other generative models tend to struggle in the tails of the distribution. We could be using a MCMC/importance sampling method, but I don't think that will be any less computationally expensive than the &amp;quot;ab initio&amp;quot; simulation.&lt;/p&gt;&#xA;&lt;p&gt;The question boils down to is there a good AI-based generative model, functional approximiator, or dimensionality reduction method for this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="43555" LastActivityDate="2021-01-04T23:32:45.780" Title="Parametrizing non-analytical functions using generative models" Tags="&lt;generative-adversarial-networks&gt;&lt;generative-model&gt;&lt;variational-autoencoder&gt;&lt;maximum-likelihood&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25582" PostTypeId="1" CreationDate="2021-01-05T01:27:46.630" Score="0" ViewCount="18" Body="&lt;p&gt;I'm solving a sequential prediction task that has multiple features attached to each timestamp, and my goal is to predict one of them. However, the feature's label is highly imbalanced: only about 4% of them are positive and the remaining are negative. I've tried multiple strategies (weighted loss, data augmentation, ...), and, surprisingly, the best approach was using multi-objective learning. More precisely, I trained the model with one more classifier attached which predicts another feature, and train the model with the (weighted) sum of BCE loss for each feature.&lt;/p&gt;&#xA;&lt;p&gt;Also, it substantially reduced the training time. However, two tasks aren't perfectly correlated - as time goes, the model seems to fit into the latter task (not the original task) and the validation AUC for the original task decreases soon (although the performance is better than single objective training).&lt;/p&gt;&#xA;&lt;p&gt;I wonder if there's a good approach to improve the performance in this case. Also, I want to ask if transfer learning (pre-training on the task 2 and fine-tuning on the task 1) would help. I think this would be better than baseline since multi-objective learning showed a non-trivial correlation between two tasks, but I'm not sure how much it would help. Especially, I wonder if it gives a better results than multi-objective training. (It would be good if there's a reference for comparing multi-objective training and transfer learning in detail.)&lt;/p&gt;&#xA;" OwnerUserId="30886" LastEditorUserId="2444" LastEditDate="2021-01-08T01:33:08.040" LastActivityDate="2021-01-08T01:33:08.040" Title="Multi-objective training vs Transfer learning - pros and cons" Tags="&lt;time-series&gt;&lt;transfer-learning&gt;&lt;multi-task-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25584" PostTypeId="1" CreationDate="2021-01-05T05:40:40.733" Score="1" ViewCount="27" Body="&lt;p&gt;I am curious about the working of a Siamese network. So, let us suppose I am using a &lt;em&gt;triplet loss&lt;/em&gt; for my network and I have instantiated &lt;strong&gt;single CNN 3 times&lt;/strong&gt; and there are 3 inputs to the network. So, during a forward pass, each of the networks will give me an embedding for each image, and I can get the distance and calculate the loss and compare it with the output, so that my model is ready to propagate the gradients to update the weights.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Question&lt;/strong&gt;: How do these weights get updated during the back propagation? Just because we are using 3 inputs and 3 branches of the same network and we are passing the inputs one by one (I suppose), how do the gradients are updated? Are these series? Like the one branch will update, then the second and then the third. But won't it be a problem because each branch would try to update based on its output? If in parallel, then which branch is responsible for the gradients update? I mean to say that I am unable to get the idea how weights are updated in Siamese network. Can someone please explain in simpler terms?&lt;/p&gt;&#xA;" OwnerUserId="36062" LastEditorUserId="2444" LastEditDate="2021-01-06T12:13:06.840" LastActivityDate="2021-01-06T12:13:06.840" Title="How do gradients are flown back into the Siamese network when branching is done?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;backpropagation&gt;&lt;facial-recognition&gt;&lt;siamese-neural-network&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25585" PostTypeId="2" ParentId="20005" CreationDate="2021-01-05T09:24:27.290" Score="2" Body="&lt;p&gt;In NEAT, the innovation of a node does not affect the evolution directly. Only the connection genes and their innovation will matter. So you can simply have whole numbers as IDs under each Genome / Network.&lt;/p&gt;&#xA;&lt;p&gt;--EDIT-- (Complete reasoning)&lt;/p&gt;&#xA;&lt;p&gt;In the original paper, it is clearly stated that the nodes from the better genome is taken during crossover and then only the connections are cross-over'ed (in some method) and hence the innovation numbers for the connections. NEAT is connection-centric and does not care much about evolving nodes.&lt;/p&gt;&#xA;&lt;p&gt;Adding to that from basic neural networks theory, the nodes will never matter in a neural network because all the calculation and the learning happens in the connections. Think about a regular feed-forward network. You only care about the weight matrix which is a property of the connections instead of the nodes though both are present. Similarly in a NEAT generated network, the nodes will not matter, as all the learning takes place in the way these nodes are connected and the weights of the network.&lt;/p&gt;&#xA;&lt;p&gt;Further, the nodes list can easily be derived from the connection list and hence marking the connections, are enough.&lt;/p&gt;&#xA;" OwnerUserId="43484" LastEditorUserId="43484" LastEditDate="2021-01-08T17:03:22.113" LastActivityDate="2021-01-08T17:03:22.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25586" PostTypeId="1" AcceptedAnswerId="25590" CreationDate="2021-01-05T13:11:06.403" Score="6" ViewCount="344" Body="&lt;p&gt;I'm currently reading the paper &lt;a href=&quot;https://arxiv.org/pdf/2002.06440.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Federated Learning with Matched Averaging&lt;/a&gt; (2020), where the authors claim:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A basic fully connected (FC) NN can be formulated as: &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = \sigma(xW_1)W_2$&lt;/span&gt; [...]&lt;/p&gt;&#xA;&lt;p&gt;Expanding the preceding expression&#xA;&lt;span class=&quot;math-container&quot;&gt;$\hat{y} = \sum_{i=1}^{L} W_{2, i \cdot } \sigma(\langle x, W_{1,\cdot i} \rangle))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$ i\cdot$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\cdot i$&lt;/span&gt; denote the ith row and column correspondingly and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the number of hidden units.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm having a hard time wrapping my head around how it can be boiled down to this. Is this rigorous? Specifically, what is meant by the ith row and column? Is this formula for only one layer or does it work with multiple layers?&lt;/p&gt;&#xA;&lt;p&gt;Any clarification would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="43582" LastEditorUserId="43582" LastEditDate="2021-01-06T08:17:38.010" LastActivityDate="2021-01-08T17:56:18.943" Title="How to express a fully connected neural network succintly using linear algebra?" Tags="&lt;neural-networks&gt;&lt;papers&gt;&lt;feedforward-neural-network&gt;&lt;linear-algebra&gt;&lt;federated-learning&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25587" PostTypeId="1" AcceptedAnswerId="25594" CreationDate="2021-01-05T15:15:09.180" Score="0" ViewCount="29" Body="&lt;p&gt;Further to my &lt;a href=&quot;https://ai.stackexchange.com/q/25498/43434&quot;&gt;last question&lt;/a&gt;, I am training a custom entity of FOODITEM to be recognized by Spacy's  Name Entity Recognition engine. I am following tutorials online, following is the advise given in most of the tutorials;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Load the model or create an empty model&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We can create an empty model and train it with our annotated dataset or we can use the existing spacy model and re-train with our annotated data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But none of the tutorials tell how/why to choose between the two options. Also, I don't understand how will the choice affect my final output or the training of the model.&lt;/p&gt;&#xA;&lt;p&gt;How do I make the choice between a pre-trained model or a blank model? What are the factors to consider?&lt;/p&gt;&#xA;" OwnerUserId="43434" LastActivityDate="2021-01-06T05:02:44.003" Title="Should we use a pre-trained model or a blank model for custom entity training of NER in spacy?" Tags="&lt;natural-language-processing&gt;&lt;spacy&gt;&lt;named-entity-recognition&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25588" PostTypeId="1" CreationDate="2021-01-05T18:22:28.930" Score="3" ViewCount="64" Body="&lt;p&gt;I was going through the paper on &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;U-Net&lt;/a&gt;. U-net consists of a contracting path followed by an expanding path. Both the paths use a regular convolutional layer. I understand the use of convolutional layers in the contracting path, but I can't figure out the use of convolutional layers in the expansive path. Note that I'm not asking about the transpose convolutions, but the regular convolutions in the expansive path.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pBgmr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pBgmr.jpg&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31749" LastEditorUserId="2444" LastEditDate="2021-01-06T17:17:34.457" LastActivityDate="2021-01-06T17:17:34.457" Title="What is the use of the regular convolutional layer in expansion path of U-Net?" Tags="&lt;convolutional-neural-networks&gt;&lt;image-segmentation&gt;&lt;convolution&gt;&lt;u-net&gt;&lt;convolutional-layers&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25589" PostTypeId="1" AcceptedAnswerId="25592" CreationDate="2021-01-05T18:26:57.313" Score="2" ViewCount="63" Body="&lt;p&gt;Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories. Can existing open source frameworks also be used for this purpose, or are their implementations too limited?&lt;/p&gt;&#xA;&lt;p&gt;Can, for example, TensorFlow be used to craft an &lt;em&gt;original&lt;/em&gt; network architecture that shows improvements on existing benchmarks? Can original mathematical work be coded into a high-level framework like TensorFlow such that original research on network architectures/approaches be demonstrated in a paper?&lt;/p&gt;&#xA;&lt;p&gt;A quick search reveals many papers using C++ in their implementation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1904.04174.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1904.04174.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1701.03980.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1701.03980.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/2005.07039.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/2005.07039.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/2007.14236.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/2007.14236.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1908.05858.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1908.05858.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="43587" LastEditorUserId="43587" LastEditDate="2021-01-06T02:31:03.790" LastActivityDate="2021-01-06T02:31:03.790" Title="Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;python&gt;&lt;research&gt;&lt;pytorch&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25590" PostTypeId="2" ParentId="25586" CreationDate="2021-01-05T18:47:00.680" Score="8" Body="&lt;p&gt;The equation &lt;span class=&quot;math-container&quot;&gt;$$\hat{y} = \sigma(xW_\color{green}{1})W_\color{blue}{2} \tag{1}\label{1}$$&lt;/span&gt; is the equation of the forward pass of a single-hidden layer fully connected and feedforward neural network, i.e. a neural network with 3 layers, 1 input layer, 1 hidden layer, and 1 output layer, where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the input layer is connected to the hidden layer (all scalar inputs are connected to every neuron in the hidden layer)&lt;/li&gt;&#xA;&lt;li&gt;the hidden layer is connected to the output layer (all neurons in the hidden layer are connected to all neurons in the output layer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As an example, suppose that we have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; real-valued features, there are &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; hidden units (or neurons) and &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; output units, then the elements (feature vector and parameters) of equation \ref{1} would have the following shape&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x \in \mathbb{R}^{1 \times N}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W_\color{green}{1} \in \mathbb{R}^{N \times L}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2} \in \mathbb{R}^{L \times M}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is an activation function that is applicable to all elements of the matrix separately (i.e. component-wise). So, &lt;span class=&quot;math-container&quot;&gt;$\hat{y} \in \mathbb{R}^{1 \times M}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\hat{y} = \sum_{\color{red}{i} = 1}^{L} W_{\color{blue}{2}, \color{red}{i} \cdot} \sigma(\langle x, W_{\color{green}{1}, \cdot \color{red}{i}} \rangle )\label{2}\tag{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is another way of writing equation \ref{1}.&lt;/p&gt;&#xA;&lt;p&gt;Before going to the explanation, let's try to understand equation \ref{2} and its components.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_{\color{green}{1}, \cdot \color{red}{i}} \in \mathbb{R}^N$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$\color{red}{i}$&lt;/span&gt;th column of the matrix that connects the inputs to the hidden neurons, so it is a vector of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; elements (note that we sum over the number of hidden neurons, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Similarly, &lt;span class=&quot;math-container&quot;&gt;$ W_{\color{blue}{2}, \color{red}{i} \cdot} \in \mathbb{R}^M$&lt;/span&gt; is also a vector, but, in this case, it is a row of the matrix &lt;span class=&quot;math-container&quot;&gt;$ W_{\color{blue}{2}}$&lt;/span&gt; (rather than a column: why? because we use &lt;span class=&quot;math-container&quot;&gt;$\color{red}{i} \cdot$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\cdot \color{red}{i}$&lt;/span&gt;, which refers to the column).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;So, &lt;span class=&quot;math-container&quot;&gt;$\langle x, W_{\color{green}{1}, \cdot \color{red}{i}} \rangle$&lt;/span&gt; is the dot (or scalar) product between the feature vector &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$\color{red}{i}$&lt;/span&gt;th column of the matrix that connects the inputs to the hidden neurons, so it's a number (or scalar). &lt;a href=&quot;https://en.wikipedia.org/wiki/Dot_product&quot; rel=&quot;nofollow noreferrer&quot;&gt;Note that both &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_{\color{green}{1}, \cdot \color{red}{i}}$&lt;/span&gt; have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; elements, so the dot product is well-defined in this case&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ W_{\color{blue}{2}, \color{red}{i}} \sigma(\langle x, W_{\color{green}{1}, \color{red}{i}} \rangle))$&lt;/span&gt; is the product between a vector of shape &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; and a number &lt;span class=&quot;math-container&quot;&gt;$\sigma(\langle x, W_{\color{green}{1}, \cdot \color{red}{i}} \rangle )$&lt;/span&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Scalar_multiplication&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is also well-defined. You can multiply a real-number with a vector, it's like multiplying the real-number with each element of the vector&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{\color{red}{i} = 1}^{L} W_{\color{blue}{2}, \color{red}{i} \cdot} \sigma(\langle x, W_{\color{green}{1}, \cdot \color{red}{i}} \rangle )$&lt;/span&gt; is thus the sum of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; vectors of size &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, which makes &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; also have size &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, as in equation \ref{1}.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now, the question is: is equation \ref{2} really equivalent to equation \ref{1}? This is still not easy to see because &lt;span class=&quot;math-container&quot;&gt;$xW_\color{green}{1}$&lt;/span&gt; is a vector of shape &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, but, in equation \ref{2}, we do not have any vector of shape &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, but we have vectors of shape &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; (and the vectors of shape &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; are summed &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; times). First, note that &lt;span class=&quot;math-container&quot;&gt;$\sigma(xW_\color{green}{1}) = h\in \mathbb{R}^L$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$hW_\color{blue}{2}$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; dot products collected in a vector (i.e. &lt;span class=&quot;math-container&quot;&gt;$\hat{y} \in \mathbb{R}^M$&lt;/span&gt;), where the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th element of &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; was computed as a summation of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; elements (a dot product of two vectors is the element-wise multiplication of the elements of the vectors followed by the summation of these multiplications). Ha, still not clear!&lt;/p&gt;&#xA;&lt;p&gt;The easiest way (for me) to see that they are equivalent is to think that &lt;span class=&quot;math-container&quot;&gt;$\sigma(xW_\color{green}{1})$&lt;/span&gt; is a vector of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; elements &lt;span class=&quot;math-container&quot;&gt;$\sigma(xW_\color{green}{1}) = \ell = [l_1, l_2, \dots, l_L]$&lt;/span&gt;. Then you know that to multiply this vector with &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; (&lt;a href=&quot;https://math.stackexchange.com/a/1615805/168764&quot;&gt;from the left&lt;/a&gt;), you actually perform a dot product between &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; and each column &lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2}$&lt;/span&gt;. A dot product is essentially a sum, and that's why we sum in equation \ref{2}. So, essentially, in equation \ref{2}, we first multiple &lt;span class=&quot;math-container&quot;&gt;$l_1$&lt;/span&gt; with the first row of &lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2}$&lt;/span&gt; (i.e. by all elements of the first row). Then we multiply &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt; by the second row of &lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2}$&lt;/span&gt;. We do this for all &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; rows, then we sum the rows (to conclude the dot product). So, you can think of equation 2 as first perform all multiplications, then summing, rather than dot product-by-dot product.&lt;/p&gt;&#xA;&lt;p&gt;So, in my head, I have the following picture. To simplify the notation, let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; denote &lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2}$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$A_{ij}$&lt;/span&gt; is the element at the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th row and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th column of matrix &lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2}$&lt;/span&gt;. So, we have the following initial matrix&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = &#xA;\begin{bmatrix}&#xA;A_{11} &amp;amp; A_{12} &amp;amp; A_{13} &amp;amp; \dots &amp;amp; A_{1M} \\&#xA;A_{21} &amp;amp; A_{22} &amp;amp; A_{23} &amp;amp; \dots &amp;amp; A_{2M} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \dots &amp;amp; \vdots \\&#xA;A_{L1} &amp;amp; A_{L2} &amp;amp; A_{L3} &amp;amp; \dots &amp;amp; A_{LM} \\&#xA;\end{bmatrix}&#xA;= &#xA;\begin{bmatrix}&#xA;W_{\color{blue}{2}, \color{red}{1} \cdot}  \\&#xA;W_{\color{blue}{2}, \color{red}{2}  \cdot} \\&#xA;\vdots  \\&#xA;W_\color{blue}{2, \color{red}{L}  \cdot} \\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, in the first iteration of equation \ref{2}, we do the following&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;l_1 A_{11} &amp;amp; l_1 A_{12} &amp;amp; l_1 A_{13} &amp;amp; \dots &amp;amp; l_1 A_{1M} \\&#xA;A_{21} &amp;amp; A_{22} &amp;amp; A_{23} &amp;amp; \dots &amp;amp; A_{2M} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \dots &amp;amp; \vdots \\&#xA;A_{L1} &amp;amp; A_{L2} &amp;amp; A_{L3} &amp;amp; \dots &amp;amp; A_{LM} \\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the second, we do the following&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;l_1 A_{11} &amp;amp; l_1 A_{12} &amp;amp; l_1 A_{13} &amp;amp; \dots &amp;amp; l_1 A_{1M} \\&#xA;l_2 A_{21} &amp;amp; l_2 A_{22} &amp;amp; l_2 A_{23} &amp;amp; \dots &amp;amp; l_2 A_{2M} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \dots &amp;amp; \vdots \\&#xA;A_{L1} &amp;amp; A_{L2} &amp;amp; A_{L3} &amp;amp; \dots &amp;amp; A_{LM} \\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;Until we have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;l_1 A_{11} &amp;amp; l_1 A_{12} &amp;amp; l_1 A_{13} &amp;amp; \dots &amp;amp; l_1 A_{1M} \\&#xA;l_2 A_{21} &amp;amp; l_2 A_{22} &amp;amp; l_2 A_{23} &amp;amp; \dots &amp;amp; l_2 A_{2M} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \dots &amp;amp; \vdots \\&#xA;l_L A_{L1} &amp;amp; l_L A_{L2} &amp;amp; l_L A_{L3} &amp;amp; \dots &amp;amp; l_L A_{LM} \\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;Then we do a &lt;a href=&quot;https://www.tensorflow.org/api_docs/python/tf/math/reduce_sum&quot; rel=&quot;nofollow noreferrer&quot;&gt;reduce sum&lt;/a&gt; across the rows to end the dot product (i.e. for each column we sum the elements in the rows). This is exactly equivalent to first performing the dot product between &lt;span class=&quot;math-container&quot;&gt;$\ell$&lt;/span&gt; and the first column of &lt;span class=&quot;math-container&quot;&gt;$W_\color{blue}{2}$&lt;/span&gt;, then the second column, and so on.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-08T17:56:18.943" LastActivityDate="2021-01-08T17:56:18.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25591" PostTypeId="1" AcceptedAnswerId="25593" CreationDate="2021-01-05T18:51:20.720" Score="1" ViewCount="89" Body="&lt;p&gt;I started reading up on SVM and very little is defined of what are support values. I reckon it's they are denoted as &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; in most formulations.&lt;/p&gt;&#xA;" OwnerUserId="43588" LastEditorUserId="2444" LastEditDate="2021-01-06T12:09:37.143" LastActivityDate="2021-01-06T12:09:37.143" Title="What are support values in a support vector machine?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;definitions&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25592" PostTypeId="2" ParentId="25589" CreationDate="2021-01-05T23:57:09.717" Score="3" Body="&lt;p&gt;Your statement that researchers build their network from the ground-up using C++ or some other low level library &lt;strong&gt;couldn't be further from the truth&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You could take a look at &lt;a href=&quot;http://horace.io/pytorch-vs-tensorflow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this analysis&lt;/a&gt; showing the popularity of these two frameworks in the top ML conferences. The following Figure is taken from there.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/UZOeL.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;In CVPR-2020, for example, TensorFlow and pytorch combined for over 500 papers! Furthermore, because the two most active research entities (Google and Facebook) are backing these two frameworks, they are used in some of the most impactful research studies.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I want to give some reasons that support the popularity of these frameworks, but first I'm going to rephrase your question a bit:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why use TensorFlow/Pytorch in python rather than build your model on your own using C++?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: The reason I rephrased the question is because &lt;a href=&quot;https://www.tensorflow.org/api_docs/cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt; and &lt;a href=&quot;https://pytorch.org/cppdocs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch&lt;/a&gt; both have a C++ APIs.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Why are these frameworks so popular in contrast to lower-level programming languages?&lt;/h3&gt;&#xA;&lt;p&gt;Some reasons are the following&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Rapid prototyping&lt;/strong&gt;. Languages link C++, have bloated syntaxes, require low-level operations (e.g. memory management) and cannot be run interactively. This means it takes someone much less time to create and test a model in python than it does in C++.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;No need to re-invent the wheel&lt;/strong&gt;. Some operations are common in most networks (e.g. backpropagation), why re-implement them? Other functionalities are hard to implement on your own (e.g. parallel processing, GPU computation). Do data scientists need to have such a strong technical background to research neural networks?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Open-source&lt;/strong&gt;. They benefit from being opensource and can offer a great deal of tools at your disposal for building neural networks. You want to add batchnorm to your network? No worries, just import it and add it in a single line! Also, they offer the perfect opportunity for sharing pretrained models.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;They are optimized&lt;/strong&gt;. These frameworks are optimized to run as fast as possible on GPUs (if available) or CPUs. It would be virtually impossible for someone to write code that runs as fast on his own.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2021-01-05T23:57:09.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25593" PostTypeId="2" ParentId="25591" CreationDate="2021-01-06T04:07:31.017" Score="1" Body="&lt;p&gt;In the least-squares SVM (LS-SVM) the non-zero Lagrange multipliers (&lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;) are the support values.  The corresponding data points are the support vectors.  Johan Suykens explains this in &lt;a href=&quot;https://www.esat.kuleuven.be/sista/natoasi/suykens.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Least Squares Support Vector Machines&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="2444" LastEditDate="2021-01-06T12:09:08.997" LastActivityDate="2021-01-06T12:09:08.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25594" PostTypeId="2" ParentId="25587" CreationDate="2021-01-06T05:02:44.003" Score="1" Body="&lt;p&gt;The reason you would load a pre-existing model is that it offers something of value to your task (e.g. named entity recognition for food) and the cost of training it from scratch is not worth it.  For example, to train GPT-3 from scratch would cost several million dollars.  Typically someone will use a model like BERT and fine tune it.  This is called transfer learning.  With spaCy you will typically use en_core_web_sm which was trained on the OntoNotes corpus and includes named entities.  Making a custom food NER using en_core_web_sm should be more accurate than making one from scratch. You should be able to build a good model with and without transfer learning fairly quickly if you have a GPU.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2021-01-06T05:02:44.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25595" PostTypeId="1" CreationDate="2021-01-06T05:38:27.287" Score="4" ViewCount="115" Body="&lt;p&gt;The sklearn's documentation of the method  &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;roc_auc_score&lt;/code&gt;&lt;/a&gt; states that the parameter &lt;code&gt;multi_class&lt;/code&gt; can take the value &lt;code&gt;'OvR'&lt;/code&gt; (which stands for &lt;strong&gt;One-vs-Rest&lt;/strong&gt;) or &lt;code&gt;'OvO'&lt;/code&gt; (which stands for &lt;strong&gt;One-vs-One&lt;/strong&gt;). These values are only applicable for multi-class classification problems.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know in what particular cases we would use OvR as opposed to OvO? In the general academic literature, is there a preference given to one?&lt;/p&gt;&#xA;" OwnerUserId="33734" LastEditorUserId="2444" LastEditDate="2021-01-06T22:00:46.957" LastActivityDate="2021-01-06T22:00:46.957" Title="When computing the ROC-AUC score for multi-class classification problems, when should we use One-vs-Rest and One-vs-One?" Tags="&lt;machine-learning&gt;&lt;metric&gt;&lt;scikit-learn&gt;&lt;roc-auc&gt;&lt;multiclass-classification&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25596" PostTypeId="1" CreationDate="2021-01-06T05:46:33.883" Score="0" ViewCount="26" Body="&lt;p&gt;I'm using MATLAB 2019, Linux, and UNet (a CNN specifically designed for semantic segmentation). I'm training the network to classify all pixels in an image as either cell or background to get segmentations of cells in microscopic images. My problem is the network is classifying every single pixel as background, and seems to just be outputting all zeroes. The validation accuracy improves a little at the very start of the training but than plateaus at around 60% for the majority of the training time. The network doesn't seem to be training very well and I have no idea why.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone give me some hints about what I should look into more closely? I just don't even know where to start with debugging this.&lt;/p&gt;&#xA;&lt;p&gt;Here's my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    % Set datapath&#xA;    datapath = '/scratch/qbi/uqhrile1/ethans_lab_data';&#xA;    &#xA;    % Get training and testing datasets&#xA;    images_dataset = imageDatastore(strcat(datapath,'/bounding_box_cropped_resized_rgb'));&#xA;    load(strcat(datapath,'/gTruth.mat'));&#xA;    labels = pixelLabelDatastore(gTruth);&#xA;    [imdsTrain, imdsVal, imdsTest, pxdsTrain, pxdsVal, pxdsTest] = partitionCamVidData(images_dataset,labels);&#xA;    &#xA;    % Weight segmentation class importance by the number of pixels in each class&#xA;    pixel_count = countEachLabel(labels); % count number of each type of pixel&#xA;    frequency = pixel_count.PixelCount ./ pixel_count.ImagePixelCount; % calculate pixel type frequencies&#xA;    class_weights = mean(frequency) ./ frequency; % create class weights that balance the loss function so that more common pixel types won't be preferred&#xA;    &#xA;    % Specify the input image size.&#xA;    imageSize = [512 512 3];&#xA;    &#xA;    % Specify the number of classes.&#xA;    numClasses = 2;&#xA;    &#xA;    % Create network&#xA;    lgraph = unetLayers(imageSize,numClasses);&#xA;    &#xA;    % Replace the network's classification layer with a pixel classification&#xA;    % layer that uses class weights to balance the loss function&#xA;    pxLayer = pixelClassificationLayer('Name','labels','Classes',pixel_count.Name,'ClassWeights',class_weights);&#xA;    lgraph = replaceLayer(lgraph,&amp;quot;Segmentation-Layer&amp;quot;,pxLayer);&#xA;    &#xA;    %% TRAIN THE NEURAL NETWORK&#xA;    &#xA;    % Define validation dataset-with-labels&#xA;    validation_dataset_with_labels = pixelLabelImageDatastore(imdsVal,pxdsVal);&#xA;    &#xA;    % Training hyper-parameters: edit these settings to fine-tune the network&#xA;    options = trainingOptions('adam', 'LearnRateSchedule','piecewise', 'LearnRateDropPeriod',10, 'LearnRateDropFactor',0.3, 'InitialLearnRate',1e-3, 'L2Regularization',0.005, 'ValidationData',validation_dataset_with_labels, 'ValidationFrequency',10, 'MaxEpochs',3, 'MiniBatchSize',1, 'Shuffle','every-epoch');&#xA;    &#xA;    % Set up data augmentation to enhance training dataset&#xA;    aug_imgs = {};&#xA;    numberOfImages = length(imdsTrain.Files);&#xA;    for k = 1 : numberOfImages&#xA;        % Apply cutout augmentation&#xA;        img = readimage(imdsTrain,k);&#xA;        cutout_img = random_cutout(img);imwrite(cutout_img,strcat('/scratch/qbi/uqhrile1/ethans_lab_data/augmented_dataset/img_',int2str(k),'.tiff'));&#xA;    end&#xA;    aug_imdsTrain = imageDatastore('/scratch/qbi/uqhrile1/ethans_lab_data/augmented_dataset');&#xA;    % Add other augmentations&#xA;    augmenter = imageDataAugmenter('RandXReflection',true, 'RandXTranslation',[-10 10],'RandYTranslation',[-10 10]);&#xA;    % Combine augmented data with training data&#xA;    augmented_training_dataset = pixelLabelImageDatastore(aug_imdsTrain, pxdsTrain, 'DataAugmentation',augmenter);&#xA;    &#xA;    % Train the network&#xA;    [cell_segmentation_nn, info] = trainNetwork(augmented_training_dataset,lgraph,options);&#xA;    &#xA;    save cell_segmentation_nn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9983" LastActivityDate="2021-02-09T01:03:00.367" Title="Semantic segmentation CNN outputs all zeroes" Tags="&lt;convolutional-neural-networks&gt;&lt;training&gt;&lt;image-segmentation&gt;&lt;matlab&gt;&lt;u-net&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25597" PostTypeId="1" AcceptedAnswerId="25598" CreationDate="2021-01-06T07:26:37.690" Score="4" ViewCount="121" Body="&lt;p&gt;I am currently working on my master's thesis and going to apply &lt;strong&gt;Deep-SARSA&lt;/strong&gt; as my DRL algorithm. The problem is that there is &lt;strong&gt;no datasets available&lt;/strong&gt; and I guess that I should generate them somehow. &lt;strong&gt;Datasets generation&lt;/strong&gt; seems a common feature in this specific subject as stated in [&lt;a href=&quot;https://ieeexplore.ieee.org/document/9039672&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When a dataset is not available, learning is performed through experience.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am wondering &lt;strong&gt;how to generate datasets when the environment is not as simple as a tic-tac-toe&lt;/strong&gt; or a maze problem and &lt;strong&gt;what the experience means&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;PS: The environment consists of 15 mobile users and 3 edge servers, each of which covers a number of mobile users. Each mobile user might generate a computationally heavy-load task and at the beginning of each timestep and can process the task itself or requests its associated edge server to do the processing. If the associated edge server is not capable of processing, due to some reasons, it requests a nearby edge server to lend it a hand. The optimization problem (reward) is to reduce time and energy consumption (multi-objective optimization). Each server has a DRL agent that makes offloading decisions.&lt;/p&gt;&#xA;&lt;p&gt;I'd really appreciate your suggestions and help.&lt;/p&gt;&#xA;" OwnerUserId="43578" LastEditorUserId="2444" LastEditDate="2021-01-08T00:32:00.647" LastActivityDate="2021-01-08T07:39:35.650" Title="How should I generate datasets for a SARSA agent when the environment is not simple?" Tags="&lt;reinforcement-learning&gt;&lt;datasets&gt;&lt;environment&gt;&lt;sarsa&gt;&lt;on-policy-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25598" PostTypeId="2" ParentId="25597" CreationDate="2021-01-06T08:05:40.053" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I am wondering &lt;strong&gt;how to generate datasets when the environment is not as simple as a tic-tac-toe&lt;/strong&gt; or a maze problem&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is no difference in concept, which is why tic-tac-toe and maze problems are used to teach.&lt;/p&gt;&#xA;&lt;p&gt;As you have noted, the main difference between reinforcement learning (RL) and supervised learning is that RL does not use labeled datasets. If you are using SARSA then you would not expect to use any record of previous experience either because SARSA is designed to work on-policy and online - which means that data needs to be generated &lt;em&gt;during&lt;/em&gt; training. Training data for SARSA is typically stored only temporarily before being used, or is used immediately (you might keep a log of it for analysis or to help document your thesis, but that log will not be used for further training by the agent). This is different to Q-learning and DQN, which could in theory make use of longer-term stored experience.&lt;/p&gt;&#xA;&lt;p&gt;You have two main choices for acquiring data:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a real environment. In your case, set up 15 mobile users and 3 edge servers. Instrument the environment to collect state and reward data for the agent. Implement the agent as the real decision maker in this environment.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Simulate the environment. Write a simulation that models user behaviour and server loading. Instrument that to provide state and reward data, and integrate your learning agent with it. Typically the agent will call the environment's &lt;code&gt;step&lt;/code&gt; function, passing the action choice as an argument and receiving reward and state data back.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you can simulate the environment, this is likely to be preferable to you since you will likely use less compute resources (than 3 servers and 15 mobile phones) and can run the training faster than real time. Deep reinforcement learning can use a large amount of experience to converge on near-optimal policies, and fast simulations can help because they generate experience faster than reality.&lt;/p&gt;&#xA;&lt;p&gt;You can also do both approaches. Train an initial agent in simulation, then implement the real version once it reaches a good level of performance in simulation. You can even have the agent continue to learn and refine behaviour in production. Given that you are working with SARSA, this may be an important part of the intent of your project, that the agent continues to adapt to changes in user behaviour and server load over time. In fact this is a key advantage of SARSA over Q-learning, that it should be more reliable and safe to use in such a continuous learning scenario deployed to production.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;and &lt;strong&gt;what the experience means&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The experience in reinforcement learning is the record of states, actions and rewards that the agent encounters during training.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-01-08T07:39:35.650" LastActivityDate="2021-01-08T07:39:35.650" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25599" PostTypeId="1" CreationDate="2021-01-06T09:05:14.787" Score="1" ViewCount="20" Body="&lt;p&gt;I'm trying to replicate a paper from Google on view synthesis/lightfields from 2019: DeepView: &lt;a href=&quot;https://augmentedperception.github.io/deepview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;View Synthesis with Learned Gradient Descent&lt;/a&gt; and &lt;a href=&quot;https://openaccess.thecvf.com/content_CVPR_2019/papers/Flynn_DeepView_View_Synthesis_With_Learned_Gradient_Descent_CVPR_2019_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this is the PDF.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Basically the input to the neural network comes from a set of cameras which number is variable, and the output is a stack of images which number is also variable. For that they use both a Fully Convolutional Network and Learned Gradient Descent.&lt;/p&gt;&#xA;&lt;p&gt;I don't know if I am understanding this correctly: (in each LGD iteration) &lt;strong&gt;They use the same network for all depth slices AND all views. Is this correct?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the LGD network, not much important to the question but it helps you understand the setup. You can see at least 3 LGD iterations. Part b) is just the calculation they do in the &amp;quot;green gradient boxes&amp;quot; on part a).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b40wP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b40wP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the inside of the CNNs. On each LGD iteration they use basically the same architecture, but the weights are different per iteration.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o2t5n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o2t5n.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For me the confusing part is that they represent each view as a different network, but they don't represent each depth slice as a different network. As you can see in the next image they do say that they use the same parameters for all depth slices, and that the order of the views doesn't matter so it must be that they're also reusing the parameters for all views, right? So if I understand correctly, this is a matter of reusing the same model for all depths and all views. BTW note that the maxpool kind of operation is over each view.&lt;/p&gt;&#xA;&lt;p&gt;Also I have a question on the practicalities of the implementation. I'll be implementing this with normal 2D convolution layers, so if I want them to run independent of the views and depth slices, I guess I could concatenate views and depth slices in the &amp;quot;batch&amp;quot; dimension? I mean, before the maximum k operation, and then reuse the output.&lt;/p&gt;&#xA;&lt;p&gt;This is what they say:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jKSzN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jKSzN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="43595" LastEditorUserId="43595" LastEditDate="2021-01-06T15:11:54.553" LastActivityDate="2021-01-06T15:11:54.553" Title="In the DeepView paper, do they use the same FCN for all depth slices AND all views?" Tags="&lt;convolutional-neural-networks&gt;&lt;computer-vision&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25600" PostTypeId="1" AcceptedAnswerId="25621" CreationDate="2021-01-06T09:09:56.143" Score="2" ViewCount="85" Body="&lt;p&gt;Consider an image that contains one can (or bottle, or any similar oval object), which has texts all over it. In the image below, I have many bottles, but you can assume that each image only contains one such object.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ETf0Y.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ETf0Y.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As we can see, in each can, the text can flow from left to right, and any OCR system may miss the text on the left and right sides of the can, as they are not aligned with the camera angle.&lt;/p&gt;&#xA;&lt;p&gt;So, is there any solution/s for this, like preprocessing in a certain way, so that we can read the text or make this round object into a straight one? (If there is any Python program that can solve this problem, could you please share it with me?)&lt;/p&gt;&#xA;" OwnerUserId="37980" LastEditorUserId="2444" LastEditDate="2021-01-08T00:40:57.667" LastActivityDate="2021-01-08T00:40:57.667" Title="In OCR, how should I deal with the warped text on the sides of oval objects?" Tags="&lt;python&gt;&lt;image-processing&gt;&lt;data-preprocessing&gt;&lt;optical-character-recognition&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25601" PostTypeId="1" CreationDate="2021-01-06T09:16:09.493" Score="7" ViewCount="474" Body="&lt;p&gt;Starting from my own understanding, and scoped to the purpose of image generation, I'm well aware of the major architectural differences:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A GAN's generator samples from a relatively low dimensional random variable and produces an image. Then the discriminator takes that image and predicts whether the image belongs to a target distribution or not. Once trained, I can generate a variety of images just by sampling the initial random variable and forwarding through the generator.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A VAE's encoder takes an image from a target distribution and compresses it into a low dimensional latent space. Then the decoder's job is to take that latent space representation and reproduce the original image. Once the network is trained, I can generate latent space representations of various images, and interpolate between these before forwarding through the decoder which produces new images.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I'm more interested is the consequences of said architectural differences.&lt;/strong&gt; Why would I choose one approach over the other? And why? (for example, if GANs typically produce better quality images, any ideas why that is so? is it true in all cases or just some?)&lt;/p&gt;&#xA;" OwnerUserId="16871" LastEditorUserId="16871" LastEditDate="2021-01-06T12:30:36.587" LastActivityDate="2021-01-06T12:30:36.587" Title="What are the fundamental differences between VAE and GAN for image generation?" Tags="&lt;comparison&gt;&lt;applications&gt;&lt;generative-adversarial-networks&gt;&lt;variational-autoencoder&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25602" PostTypeId="2" ParentId="25588" CreationDate="2021-01-06T10:01:31.300" Score="3" Body="&lt;p&gt;The point is that in the expansive path you have two forms of information:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the information from the contracting path, which includes all &lt;strong&gt;high-level features extracted&lt;/strong&gt; from the original image.&lt;/li&gt;&#xA;&lt;li&gt;the information from the skip-connections, which copy a cropped version of the feature maps in the contracting path. Because, as we go forward through the expansive path these originate from earlier stages in the contracting path, these are progressively richer in &lt;strong&gt;detail&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Intuitively you can think of it as this: high-level features help the network tell which areas to group together, while details help it tell where each group starts and ends at the pixel level.&lt;/p&gt;&#xA;&lt;p&gt;The idea is to combine these two forms of information, i.e. the high-level features and the details, optimally.  To do this you need trainable layers that &lt;strong&gt;learn&lt;/strong&gt; this optimal combination. Here is where the convolution layers come to play.&lt;/p&gt;&#xA;" OwnerUserId="26652" LastActivityDate="2021-01-06T10:01:31.300" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25603" PostTypeId="2" ParentId="25506" CreationDate="2021-01-06T10:28:56.653" Score="0" Body="&lt;p&gt;There is no general method to detect a change in the fitness landscape, since changes can be very local and can occur in just a small area of the fitness landscape. For this reason nature inspired optimization algorithms usually maintain a diversified population to cope with environmental changes. a common mechanism is using several sub-populations and ensuring that these sub-populations do not overlap. Also, there are some heuristics proposed that can help the algorithms in detecting changes. for instance if you are using multiple sub-populations, you can double test one element of each sub-population at some generation to find out if an environmental change has occurred or not. albeit, there are some comprehensive heuristics have been proposed for change detection, like the one proposed in [R. Mukherjee, S. Debchoudhury, S. Das, Modified differential evolution with locality induced genetic operators for dynamic optimization]. In my opinion, Use simple methods. for instance, at the keep a small by diverse set of population members at some k generations, and reevaluate them after k generations to detect change.&lt;/p&gt;&#xA;" OwnerUserId="43208" LastActivityDate="2021-01-06T10:28:56.653" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25604" PostTypeId="2" ParentId="25505" CreationDate="2021-01-06T10:44:04.847" Score="0" Body="&lt;p&gt;using different population sizes at different stages of the optimization process can be beneficial. With large population sizes you can effectively explore the landscape to find proper areas. Large population sizes helps in finding global optima or high fitness local optima. However, using large populations require more fitness evaluations and waste of computational resources. With a small population you can effectively exploit a previously find appropriate area and get high accuracy solutions. For this reason, some works suggest to use a large population at the start of the algorithm and gradually, decrease its size, like [Improving the search performance of SHADE using linear population size reduction]. Also, some dynamic optimization methods, use dynamic population sizes. For instance, they create sub-populations when it is necessary to discover more optima or to cover the landscape, they decrease their sub-populations when they detect a change or new appropriate uncovered area in the landscape.&lt;/p&gt;&#xA;" OwnerUserId="43208" LastActivityDate="2021-01-06T10:44:04.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25605" PostTypeId="2" ParentId="25427" CreationDate="2021-01-06T11:07:23.120" Score="1" Body="&lt;p&gt;I'm not familiar with your game so I can't tell you what a good heuristic woul be in your specific case, but I can give you some advice on how to look for a good heuristic function.&lt;/p&gt;&#xA;&lt;p&gt;As a rule of thumb, the heuristic function for a MiniMax algorithm is best kept simple and efficient, so you can get deeper into the tree. But it depends on how costly it is to compute the heuristic function compared to simulating moves in the game.&lt;/p&gt;&#xA;&lt;p&gt;If the heuristic takes longer than simulating a game move, it might be worth simplifying it so it runs faster and you can look ahead further. This often leads to more emergent and advanced strategies that are hard to express mathematically. An extreme example of a simple heuristic would be the current player score minus the opponent's score. Since scores only change when someone lands on a bonus tile, many paths you take down the tree would have equal value, so you need to be able to look ahead many moves to find a non-zero heuristic and be able to prune parts of the tree. But because the heuristic is so fast to compute, you can do this and discover more non-obvious strategies, simply by brute force simulation. This leads to more emergent behavior, and would tell you more about different ways to play the game (if that is your goal).&lt;/p&gt;&#xA;&lt;p&gt;If simulating a game move takes longer than your current heuristic, it's probably not worth making the heuristic faster since the game simulation is the dominating factor in how deep you can go down the tree. This case is much more difficult because it means you have to find optimal strategies (and how to express them as a heuristic function) yourself. This is more of an art than a science - ask any chess grandmaster. I'd look up existing literature on the game and see if any existing strategies can be translated into a heuristic function. If there is no literature (e.g. because the game is new or unpopular), you could spend some time playing it yourself to discover what works. Alternatively (and perhaps more interesting), you could use a simple heuristic function with more emergent behavior, increase the time the MiniMax algorithm has to make moves, and play against your NPC opponent a few times to see what strategies it discovers. Or even have two NPCs with different heuristic functions play against each other. Then try to incorporate those into your final heuristic function. This is also a way to determine which heuristic function is better, if you have multiple candidates.&lt;/p&gt;&#xA;&lt;p&gt;There are ways to optimize the heuristic function automatically using machine learning (specifically reinforcement learning), but it's probably not worth opening that can of worms in your case.&lt;/p&gt;&#xA;" OwnerUserId="40060" LastActivityDate="2021-01-06T11:07:23.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25606" PostTypeId="5" CreationDate="2021-01-06T11:51:08.217" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-06T11:51:08.217" LastActivityDate="2021-01-06T11:51:08.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25607" PostTypeId="4" CreationDate="2021-01-06T11:51:08.217" Score="0" Body="For questions related to the task of image generation, which can be done, for example, with variational auto-encoders (VAEs) or generative adversarial networks (GANs)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-06T11:51:08.217" LastActivityDate="2021-01-06T11:51:08.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25608" PostTypeId="1" CreationDate="2021-01-06T11:55:52.453" Score="0" ViewCount="31" Body="&lt;p&gt;I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vwRGx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vwRGx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I denoted the portions of the formula for what I thought they represented. &lt;code&gt;.*&lt;/code&gt; represents an element wise multiplication in MATLAB. I tried to expand this for other cases, where I used Softmax as the last layer's activation, and categorical cross entropy for the loss. Since the implementation above requires the two parts to be the same shape, and the backwards pass of Softmax gives the 2D Jacobian for a 1D Vector, how would I element-wise multiply this with the loss function's derivative, another Vector? Would taking the dot product give me the result I need?&lt;/p&gt;&#xA;" OwnerUserId="43270" LastActivityDate="2021-01-06T11:55:52.453" Title="Backpropagation implementation not applicable for other cases" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;gradient-descent&gt;&lt;linear-algebra&gt;&lt;softmax&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25609" PostTypeId="2" ParentId="25601" CreationDate="2021-01-06T12:25:36.963" Score="1" Body="&lt;p&gt;GANs generally produce better photo-realistic images but can be difficult to work with.  Conversely, VAEs are easier to train but don’t usually give the best results.&lt;/p&gt;&#xA;&lt;p&gt;I recommend picking VAEs if you don’t have a lot of time to experiment with GANs and photorealism isn’t paramount.&lt;/p&gt;&#xA;&lt;p&gt;There are exceptions such as Google’s VQ-VAE 2 which can compete with GANs for image quality and realism.  There is also VAE-GAN and VQ-VAE-GAN.&lt;/p&gt;&#xA;&lt;p&gt;As a note, GANs and VAEs are not specifically for images and can be used for other data types/structures.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2021-01-06T12:25:36.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25610" PostTypeId="2" ParentId="25486" CreationDate="2021-01-06T14:47:11.290" Score="0" Body="&lt;p&gt;I couldn't understand your question clearly, however, it think you are making a slim mistake. let's look at the flowing code from &amp;quot;Russell &amp;quot; and do pruning step by step:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Fwvhe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fwvhe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TcWDQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TcWDQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;br/&gt;Assume your are In D and you have traversed its both children, Alpha at D becomes 20. We then return back to B, Beta becomes 20 (note that Alpha is -Inf in D). We go to E then L and then back to E. 30 is greater than Beta, so the rest is pruned. Not that Alpha remains -Inf in E, though its value here isn't important. we go back to B the value of Beta remains on changed 20. If we go to F and return back to B Beta remains unchanged and is still 20 (Note that Alpha is still -Inf). We return to A, Alpha becomes 20, And Beta remains +Inf. &lt;br/&gt;&#xA;So, Considering the code, the max node does not get the Alpha value from its children, rather it decides on its own by setting Alpha = max(Alpha, and returned value of its child) after visiting each child. the same is true for Beta.&lt;/p&gt;&#xA;" OwnerUserId="43208" LastEditorUserId="43208" LastEditDate="2021-01-06T14:57:25.950" LastActivityDate="2021-01-06T14:57:25.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25611" PostTypeId="1" CreationDate="2021-01-06T17:15:16.700" Score="0" ViewCount="16" Body="&lt;p&gt;I have a very different kind of problem at hand.&lt;/p&gt;&#xA;&lt;p&gt;I have a function plotted on the grid lines as shown below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1NSum.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1NSum.png&quot; alt=&quot;Image of the function&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, I need to get the x and y coordinates for this particular function at regular intervals.&lt;/p&gt;&#xA;&lt;p&gt;The only information I have is this image and the fact that each of the small squares of the grid represents 0.2 units.&lt;/p&gt;&#xA;&lt;p&gt;I found this to be interesting problem, but I don't know how to approach it.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can this problem be solved using any of the image processing&#xA;techniques?&lt;/li&gt;&#xA;&lt;li&gt;Can this we use any Deep Learning techniques here?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any though process on how to approach this problem is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="32813" LastActivityDate="2021-01-06T17:15:16.700" Title="Interpreting coordinates of the function from the image using Grid Lines" Tags="&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;image-processing&gt;&lt;opencv&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25612" PostTypeId="1" CreationDate="2021-01-07T04:43:25.520" Score="0" ViewCount="13" Body="&lt;p&gt;I am writing a code to train custom entities in Spacy's NER engine. I am stuck in understanding small part of code from an online tutorial. Here's a &lt;a href=&quot;https://gist.github.com/DataTurks/f6035b1e58497d52bf88517ff7bf64cf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to the tutorial&lt;/a&gt;. The following code is line number 12-15. I am new to python so stuck understanding this part of the code;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# add labels&#xA;    for _, annotations in TRAIN_DATA:&#xA;        for ent in annotations.get('entities'):&#xA;            ner.add_label(ent[2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Apparently, this for loop is adding custom labels to the NER. M questions are;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is an 'annotation'? (I googled for 'spacy annotation' but couldn't find the answer)&lt;/li&gt;&#xA;&lt;li&gt;Why are there two variables to the left of 'in', ('_' and 'annotation')?&lt;/li&gt;&#xA;&lt;li&gt;What does ent[2] return? What's at pos 2?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="43434" LastActivityDate="2021-01-07T04:43:25.520" Title="What is this for loop doing in custom NER code in Spacy and what is an 'annotation' in Spacy?" Tags="&lt;natural-language-processing&gt;&lt;python&gt;&lt;spacy&gt;&lt;named-entity-recognition&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25613" PostTypeId="1" CreationDate="2021-01-07T06:14:44.740" Score="0" ViewCount="25" Body="&lt;p&gt;The char-CNN-RNN encoder is relatively a popular encoder. It was proposed in &lt;a href=&quot;https://arxiv.org/pdf/1605.05395.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&lt;p&gt;Is it an encoder part of an autoencoder? Or an independent neural network?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="2444" LastEditDate="2021-01-08T01:38:18.207" LastActivityDate="2021-01-08T01:38:18.207" Title="Is the char-CNN-RNN text encoder is an encoder part of an auto-encoder?" Tags="&lt;autoencoders&gt;&lt;text-encoder&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25614" PostTypeId="1" CreationDate="2021-01-07T07:59:59.753" Score="4" ViewCount="71" Body="&lt;p&gt;How can I update the observation probability for a POMDP (or HMM), in order to have a more accurate prediction model?&lt;/p&gt;&#xA;&lt;p&gt;The POMDP relies on observation probabilities that match an observation to a state. This poses an issue as the probabilities are not exactly known. However, the idea is to make them more accurate over time. The simplest idea would be to count the appeared observation as well as the states and use Naive Bayes estimators.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;span class=&quot;math-container&quot;&gt;$P(s' \mid a,s)$&lt;/span&gt; is the probability that a subsequent state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt; is reached, given that the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and the previous state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; are known: In that simple case I can just count and then apply e.g. Naive Bayes estimators.&lt;/p&gt;&#xA;&lt;p&gt;But, if I have an observation probability &lt;span class=&quot;math-container&quot;&gt;$P(z \mid s')$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is the observation) depending on a state, it's not as trivial to just count up the observation and the states, as I can not say that a state really was reached (Maybe I made an observation, but I was in a different state than wanted). I can just make an observation and hope I was in a certain state. But I can not say if e.g. I was in &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; or maybe &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt;. I think the update of the observation probability is only possible in the late aftermath.&lt;/p&gt;&#xA;&lt;p&gt;So, what are good approaches to estimate my state?&lt;/p&gt;&#xA;" OwnerUserId="21157" LastEditorUserId="2444" LastEditDate="2021-01-08T00:09:43.673" LastActivityDate="2021-01-08T00:09:43.673" Title="How to update the observation probabilities in a POMDP?" Tags="&lt;reinforcement-learning&gt;&lt;pomdp&gt;&lt;state-spaces&gt;&lt;conditional-probability&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25615" PostTypeId="1" CreationDate="2021-01-07T08:20:05.743" Score="2" ViewCount="45" Body="&lt;p&gt;To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category).&lt;/p&gt;&#xA;&lt;p&gt;Few-shot learning seems to do the same, with maybe 5-20 images per category. Is that the only difference?&lt;/p&gt;&#xA;&lt;p&gt;In both cases, we initially train the neural network with a large dataset, then fine-tune it with our custom datasets.&lt;/p&gt;&#xA;&lt;p&gt;So, how is few-shot learning different from transfer learning?&lt;/p&gt;&#xA;" OwnerUserId="43623" LastEditorUserId="2444" LastEditDate="2021-01-07T17:22:05.267" LastActivityDate="2021-05-16T09:09:55.910" Title="How is few-shot learning different from transfer learning?" Tags="&lt;deep-learning&gt;&lt;comparison&gt;&lt;image-recognition&gt;&lt;transfer-learning&gt;&lt;few-shot-learning&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25616" PostTypeId="1" CreationDate="2021-01-07T10:11:26.967" Score="2" ViewCount="71" Body="&lt;p&gt;I have sentences with some grammatical errors , with no punctuations and digits written in words... something like below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TAQkT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TAQkT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can observe, a proper noun , &lt;strong&gt;winston&lt;/strong&gt; isnt highlighted with capital in Sample column. 'People' is spelled wrong and there are no punctuations in Sample column. The date in the first row isnt in right format. I have millions of rows like this and want to train a model to learn punctuations and corrections. Can a single BERT or T5 handle this task? or only option is to try 1 model for each task?&#xA;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="43626" LastEditorUserId="43626" LastEditDate="2021-01-13T09:28:58.470" LastActivityDate="2021-04-28T17:25:44.897" Title="T5 or BERT for sentence correction/generation task?" Tags="&lt;natural-language-processing&gt;&lt;bert&gt;&lt;text-generation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25617" PostTypeId="1" CreationDate="2021-01-07T12:34:06.263" Score="2" ViewCount="54" Body="&lt;p&gt;I have 2 small images. They are basically the same, but differ in rotation and size. I should estimate the parameters for affine transform to get them similar. What network structure can be suitable for this task? For example, those based on convolutional networks did badly, because the pictures are too small.&lt;a href=&quot;https://i.stack.imgur.com/tKQLP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tKQLP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43631" LastActivityDate="2021-02-07T21:01:25.123" Title="Can you give me a piece of advise of the network sructure that would be suitable for my task?" Tags="&lt;image-generation&gt;" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25618" PostTypeId="1" AcceptedAnswerId="25626" CreationDate="2021-01-07T14:39:25.790" Score="1" ViewCount="55" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1505.04597.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original U-Net paper&lt;/a&gt;, it is written&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The energy function is computed by a pixel-wise soft-max over the final&#xA;feature map combined with the cross entropy loss function.&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E=\sum_{\mathbf{x} \in \Omega} w(\mathbf{x}) \log \left(p_{\ell(\mathbf{x})}(\mathbf{x})\right) \tag{1}\label{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$w(\mathbf{x})$&lt;/span&gt; is a weight map (I'm not interested in that part right now), and &lt;span class=&quot;math-container&quot;&gt;$p_{k}(\mathbf{x})$&lt;/span&gt; is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p_{k}(\mathbf{x})=\exp \left(a_{k}(\mathbf{x})\right) /\left(\sum_{k^{\prime}=1}^{K} \exp \left(a_{k^{\prime}}(\mathbf{x})\right)\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The pixel-wise softmax with &lt;span class=&quot;math-container&quot;&gt;$a_{k}(\mathbf{x})$&lt;/span&gt; being the activation in feature channel &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; at pixel position &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; the number of classes. Then &lt;span class=&quot;math-container&quot;&gt;$\ell(\mathbf{x})$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$p_{\ell(\mathbf{x})}$&lt;/span&gt; is the true label of each pixel, i.e. if the pixel at position &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; is part of class &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$p_{\ell(\mathbf{x})}$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$p_1(\mathbf{x})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As far as is understand &lt;span class=&quot;math-container&quot;&gt;$-E$&lt;/span&gt; should be the cross-entropy function. Right? I've already done the math for the binary case (ignoring &lt;span class=&quot;math-container&quot;&gt;$w(\mathbf{x})$&lt;/span&gt;) and it seemed to be equal.&lt;/p&gt;&#xA;" OwnerUserId="43632" LastEditorUserId="2444" LastEditDate="2021-01-08T17:19:25.220" LastActivityDate="2021-01-08T17:19:25.220" Title="Have I understood the loss function from the original U-Net paper correctly?" Tags="&lt;loss-functions&gt;&lt;papers&gt;&lt;image-segmentation&gt;&lt;u-net&gt;&lt;cross-entropy&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25619" PostTypeId="2" ParentId="25617" CreationDate="2021-01-07T15:50:05.107" Score="1" Body="&lt;p&gt;Instead of NNs, you can use RANSAC algorithm to calculate homography matrix, but first you need to find feature points. However, if your images are blob-like, you may not get such a successful results. Here some presentations for better understanding: &lt;a href=&quot;https://www.cs.umd.edu/class/fall2019/cmsc426-0201/files/17_Homography-estimation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;cs.umd notes&lt;/a&gt; and &lt;a href=&quot;http://6.869.csail.mit.edu/fa12/lectures/lecture13ransac/lecture13ransac.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;csail.mit notes&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(Also, there might be better image processing tools.)&lt;/p&gt;&#xA;" OwnerUserId="41615" LastActivityDate="2021-01-07T15:50:05.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25620" PostTypeId="1" CreationDate="2021-01-07T18:15:26.487" Score="0" ViewCount="17" Body="&lt;p&gt;I currently implemented Deep Neuroevolution and used it on a couple of Atari games. For my implementation I used a similar Genetic Algorithm, network and setup as the Uber AI Deep Neuroevolution &lt;a href=&quot;https://arxiv.org/pdf/1712.06567.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; from 2018. Unfortunately my policies are not &amp;quot;diverse&amp;quot; and always take the same actions. So for instance I have one policy that always takes action 1 and another that always takes action 2. After many mutations, the policies change but still always take the same action i.e. I now have policies that only take move 3. Even after thousands of games and mutations they don't seem to improve. Particularly this effect occurs for the game of Pong. I believe this might be due to the fact that for Pong only few pixels change during the game (the player and the ball), which might lead to Neuroevolution evolving networks that don't pick up these small changes. Any idea how one can fix this?&lt;/p&gt;&#xA;" OwnerUserId="38975" LastActivityDate="2021-01-07T18:15:26.487" Title="Neuroevolution + RL: How to make sure my policies are more diverse?" Tags="&lt;reinforcement-learning&gt;&lt;evolutionary-algorithms&gt;&lt;neuroevolution&gt;&lt;atari-games&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25621" PostTypeId="2" ParentId="25600" CreationDate="2021-01-07T18:25:41.963" Score="2" Body="&lt;p&gt;There are many papers on this but the following is a good start:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://hackernoon.com/how-to-unwrap-wine-labels-programmatically-31c8c62b30ce&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to unwrap wine labels programmatically&lt;/a&gt;.  The author includes &lt;a href=&quot;https://github.com/Nepherhotep/unwrap_labels&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; in Python.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B46JW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B46JW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You mentioned you do not want to do a panoramic view but that has more than one meaning.  If I assume you mean you do not want to rotate the can while taking multiple photos, or you don't want to take multiple photos from different angles, you could try a &lt;a href=&quot;https://www.opto-e.com/basics/360-optics&quot; rel=&quot;nofollow noreferrer&quot;&gt;pericentric lens&lt;/a&gt;.  This would require some image processing to do the unwrapping.  More resolution is needed as the wrapping is much more severe.  The advantage though is that you will have a single image of the full cylindrical surface and won't miss any features or text.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N9ZA2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N9ZA2.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2021-01-07T21:19:39.077" LastActivityDate="2021-01-07T21:19:39.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25622" PostTypeId="1" CreationDate="2021-01-08T00:04:21.480" Score="2" ViewCount="21" Body="&lt;p&gt;I am using DDPG to solve a RL problem. The action space is given by the Cartesian product &lt;span class=&quot;math-container&quot;&gt;$[0,20]^4\times[0,6]^4$&lt;/span&gt;. The &lt;code&gt;actor&lt;/code&gt; is implemented as a deep neural network with an output dimension equals to &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; with &lt;code&gt;tanh&lt;/code&gt; activation.&lt;/p&gt;&#xA;&lt;p&gt;So, given a state &lt;code&gt;s&lt;/code&gt;, an action is given by &lt;code&gt;a = actor(s)&lt;/code&gt; where &lt;code&gt;a&lt;/code&gt; contains real numbers in &lt;code&gt;[-1,1]&lt;/code&gt;. Next, I map this action &lt;code&gt;a&lt;/code&gt; into a valid action &lt;code&gt;valid_a&lt;/code&gt; that belongs to the action space &lt;span class=&quot;math-container&quot;&gt;$[0,20]^4\times[0,6]^4$&lt;/span&gt;. Than, I use &lt;code&gt;valid_a&lt;/code&gt; to calculate the reward.&lt;/p&gt;&#xA;&lt;p&gt;My question is: how does the DDPG algorithm know about this mapping that I am doing? In what part of the DDPG algorithm should I specify this mapping? Should I provide a bijective mapping to guarantee that the DDPG algorithm learns bad from good actions?&lt;/p&gt;&#xA;" OwnerUserId="37642" LastActivityDate="2021-01-08T10:50:44.210" Title="How does DDPG algorithm know about my action mapping function?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;ddpg&gt;&lt;mapping-space&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25623" PostTypeId="1" CreationDate="2021-01-08T03:32:10.387" Score="0" ViewCount="23" Body="&lt;p&gt;I've just started learning LSTM, and some points in the process of calculating the gradients are getting me confused.&lt;/p&gt;&#xA;&lt;p&gt;Say, for example, we want to compute  &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W_i}L$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the loss for one input sequence &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$W_i$&lt;/span&gt; is the matrix of parameters to learn in the input gate &lt;span class=&quot;math-container&quot;&gt;$i_t$&lt;/span&gt;, and  &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt; the previous output from last time step as in the equation :&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$i_t =  \sigma(W_i h_{t-1} + U_i x(t) + b_i)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;At some point, during backpropagation, we need to do the &amp;quot;straightforward task&amp;quot;, computing &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W_i}i_t$&lt;/span&gt; which is equal to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial}{\partial W_i}i_t = \sigma(z_{i,t}) (1 - \sigma(z_{i,t}))  h_{t-1},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$$z_{i,t} = W_i h_{t-1} + U_i x(t) + b_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Well, my problem is here, I do think that &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt; also depends on &lt;span class=&quot;math-container&quot;&gt;$W_i$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$h_{t-1}$&lt;/span&gt; was computed with &lt;span class=&quot;math-container&quot;&gt;$W_i$&lt;/span&gt; (and other parameters) during the last time step. So, maybe, some more term should be added to the equation of &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial W_i}i_t$&lt;/span&gt; above.&lt;/p&gt;&#xA;&lt;p&gt;I hope you can understand my issue.&lt;/p&gt;&#xA;" OwnerUserId="43641" LastEditorUserId="2444" LastEditDate="2021-01-08T11:45:28.850" LastActivityDate="2021-01-08T11:45:28.850" Title="During Backpropagation in LSTM, why is the previous output $h_{t-1}$ considered constant w.r.t any $W$ while computing derivative?" Tags="&lt;long-short-term-memory&gt;&lt;backpropagation&gt;&lt;calculus&gt;&lt;bptt&gt;&lt;gradient&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25625" PostTypeId="1" CreationDate="2021-01-08T06:17:09.573" Score="2" ViewCount="68" Body="&lt;p&gt;I am new to reinforcement learning. May I ask a simple (and maybe a bit silly) question here? I am trying to use the &amp;quot;one-step actor-critic&amp;quot; method to train a robot on a gridworld. Let's focus on the actor as there is nothing puzzling me for the critic.&lt;/p&gt;&#xA;&lt;p&gt;I used a feedforward ANN with one hidden layer to parameterize the action preference function (i.e. the &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; function). The ANN has one bias node in the input layer to connect to all the hidden nodes. Therefore, there are three sets of weights associated with the &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; function -- the weights connecting the inputs to the hidden nodes (let's call it &lt;span class=&quot;math-container&quot;&gt;$W1$&lt;/span&gt; matrix), the weights connecting the hidden nodes to the outputs (let's call it &lt;span class=&quot;math-container&quot;&gt;$W2$&lt;/span&gt; matrix), and the weights connecting the bias node to the hidden nodes (let's call it &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; vector).&lt;/p&gt;&#xA;&lt;p&gt;I used the exponential soft-max as the policy function (i.e. the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; function). That is,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\pi(a|s,W1,W2,c) = \displaystyle\frac{e^{h(a,s,W1,W2,c)}}{\sum_be^{h(b,s,W1,W2,c)}}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The inputs to the ANN are the state/feature vector, and the outputs of the ANN are the action preference values (i.e. the &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; values). With these action preference values, the &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; function can compute the probabilities for each action to be chosen.&lt;/p&gt;&#xA;&lt;p&gt;It is easy to derive that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla_{W1/W2/c} \log \pi(a|s,W1,W2,c) = \nabla_{W1/W2/c} h(a,s,W1,W2,c)-\sum_b \big[\nabla_{W1/W2/c}h(b,s,W1,W2,c)\pi(b|s,W1,W2,c)\big]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the above, &lt;span class=&quot;math-container&quot;&gt;$/$&lt;/span&gt; means &amp;quot;or&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My puzzle is, I found that &lt;span class=&quot;math-container&quot;&gt;$\nabla_{W2} h(\cdot,s,W1,W2,c) \equiv \sigma$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the vector of the sigmoid activation values. That implies, &lt;span class=&quot;math-container&quot;&gt;$\nabla_{W2}$&lt;/span&gt; is independent of actions!? Consequently, &lt;span class=&quot;math-container&quot;&gt;$\nabla_{W2} \log \pi(\cdot|s,W1,W2,c) \equiv 0$&lt;/span&gt;, which implies that &lt;span class=&quot;math-container&quot;&gt;$W2$&lt;/span&gt; will not be updated at all...&lt;/p&gt;&#xA;&lt;p&gt;Where did I get wrong?&lt;/p&gt;&#xA;&lt;p&gt;(Actually, the above puzzle of mine extends to any policy gradient method as long as &lt;span class=&quot;math-container&quot;&gt;$\nabla \log \pi$&lt;/span&gt; is involved and a feedforward ANN is used to approximate the action preferences.)&lt;/p&gt;&#xA;" OwnerUserId="43644" LastEditorUserId="2444" LastEditDate="2021-01-09T01:13:55.970" LastActivityDate="2021-01-12T03:19:20.323" Title="$\nabla \log \pi$ with respect to some parameters constantly being zero" Tags="&lt;neural-networks&gt;&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25626" PostTypeId="2" ParentId="25618" CreationDate="2021-01-08T07:02:44.380" Score="1" Body="&lt;p&gt;Yes, &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the cross-entropy function and a direct generalization of the binary case.&lt;/p&gt;&#xA;&lt;p&gt;For the binary case, probability to belong to the class &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; is given by a sigmoid function &lt;span class=&quot;math-container&quot;&gt;$\sigma(x)$&lt;/span&gt; of the output &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, and the probability to belong to the class &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1 - \sigma(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Therefore the binary crossentropy will give:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;-\sum_i (l_i \log \sigma(x) + (1 - l_i)  \log (1 - \sigma(x))&#xA;$$&lt;/span&gt;&#xA;Where the sum is over all samples in the dataset. Because &lt;span class=&quot;math-container&quot;&gt;$l_i$&lt;/span&gt; is a binary variable one of these two terms would be zero. The gradient descent forces the model to predict true label with more confidence.&lt;/p&gt;&#xA;&lt;p&gt;For the multiclass case, now the output is a &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;-vector and the softmax function forces its elements to sum to one:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{i = 1}^{K} p_i = 0&#xA;$$&lt;/span&gt;&#xA;The true label is one-hot encoded vector, with &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; on the position of the true label, and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; elsewhere. The generalization of the binary case is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;-\sum_i \sum_{j = 1}^{K}(l_{ij} \log \sigma_j(x) + (1 - l_{ij})  \log (1 - \sigma_j(x))&#xA;$$&lt;/span&gt;&#xA;In this case, there will be &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; non-vanishing contributions.The minization of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; forces the classifier to predict the true label more confidently, and all other (to make &lt;span class=&quot;math-container&quot;&gt;$\log(1 - \sigma_j(x))$&lt;/span&gt;) as small  as possible.&lt;/p&gt;&#xA;" OwnerUserId="38846" LastActivityDate="2021-01-08T07:02:44.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25627" PostTypeId="1" AcceptedAnswerId="25636" CreationDate="2021-01-08T09:23:29.330" Score="0" ViewCount="563" Body="&lt;p&gt;From the tid-bits, I understand of neural networks (NN), the Loss function is the difference between predicted output and expected output of the NN. I am following &lt;a href=&quot;https://github.com/explosion/spaCy/blob/master/examples/training/train_ner.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;, the losses are included at line #81 in the &lt;code&gt;nlp.update()&lt;/code&gt; function.&lt;/p&gt;&#xA;&lt;p&gt;I am getting losses in the range 300-100. How to interpret them? &lt;strong&gt;What should be the ideal output of this losses variable&lt;/strong&gt;? I went through Spacy's documentation, but nothing much is written there about losses. Also, please let me know the links to relevant theories to understand this in general.&lt;/p&gt;&#xA;" OwnerUserId="43434" LastEditorUserId="5763" LastEditDate="2021-01-09T07:19:55.113" LastActivityDate="2021-01-09T07:19:55.113" Title="How to understand 'losses' in Spacy's custom NER training engine?" Tags="&lt;machine-learning&gt;&lt;natural-language-processing&gt;&lt;loss-functions&gt;&lt;spacy&gt;&lt;named-entity-recognition&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25628" PostTypeId="2" ParentId="25573" CreationDate="2021-01-08T09:49:41.777" Score="1" Body="&lt;p&gt;Regarding your first point, it depends on what neural network you would like to use, the sensor temporal resolution, and the capabilities of the embedded system. You can figure out the number of operations required for a forward pass of your network, then when combined with the internal clock of the embedded system, you can calculate the approximate time it would take for one classification event in real time.&lt;/p&gt;&#xA;&lt;p&gt;A good explanation is given here&#xA;&lt;a href=&quot;https://ai.stackexchange.com/questions/22913/what-is-the-computational-complexity-of-the-forward-pass-of-a-convolutional-neur&quot;&gt;What is the computational complexity of the forward pass of a convolutional neural network?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you have the computational complexity of your network, and the number of CPU cycles per seconds, you can roughly approximate the time.&lt;/p&gt;&#xA;" OwnerUserId="43651" LastEditorUserId="43651" LastEditDate="2021-01-08T10:00:01.097" LastActivityDate="2021-01-08T10:00:01.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25629" PostTypeId="1" CreationDate="2021-01-08T10:00:45.277" Score="0" ViewCount="29" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kalman Filter&lt;/a&gt; approaches and I've just implemented the &lt;a href=&quot;https://en.wikipedia.org/wiki/Extended_Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;extended Kalman filter (EKF)&lt;/a&gt; with my object 2D trajectory. However, I have a mess of alternative approaches that may fit better like Unscented Kalman Filter (UFK), &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;particle filters&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Adaptive_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;adaptive filtering&lt;/a&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;How can I choose the most suitable algorithm for my case? In addition, are there algorithms that can predict more than one step ahead?&lt;/p&gt;&#xA;" OwnerUserId="43650" LastEditorUserId="43650" LastEditDate="2021-01-08T11:48:14.697" LastActivityDate="2021-01-08T11:48:14.697" Title="Which is the best algorithm to predict the trajectory of a vehicle using lat/lon data?" Tags="&lt;prediction&gt;&lt;autonomous-vehicles&gt;&lt;forecasting&gt;&lt;kalman-filter&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25630" PostTypeId="1" CreationDate="2021-01-08T10:35:52.870" Score="4" ViewCount="171" Body="&lt;p&gt;Consider our parametric model &lt;span class=&quot;math-container&quot;&gt;$p_\theta$&lt;/span&gt; for an underlying probabilistic distribution &lt;span class=&quot;math-container&quot;&gt;$p_{data}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, the likelihood of an observation &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is generally defined as &lt;span class=&quot;math-container&quot;&gt;$L(\theta|x) = p_{\theta}(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The purpose of the likelihood is to quantify how good the parameters are. How can the probability density at a given observation, &lt;span class=&quot;math-container&quot;&gt;$p_{\theta}(x)$&lt;/span&gt;, measure how much good the parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are?&lt;/p&gt;&#xA;&lt;p&gt;Is there any relation between the &lt;em&gt;goodness of parameters&lt;/em&gt; and &lt;em&gt;the probability density value of an observation&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="18758" LastEditorUserId="2444" LastEditDate="2021-01-08T11:32:44.470" LastActivityDate="2021-01-08T12:53:49.683" Title="How can a probability density value be used for the likelihood calculation?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;probability-distribution&gt;&lt;maximum-likelihood&gt;&lt;probability-density&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25631" PostTypeId="2" ParentId="25622" CreationDate="2021-01-08T10:50:44.210" Score="1" Body="&lt;p&gt;I would recommend doing is allowing your network to output any real number and then clipping the output. For instance, I was working with an agent that had to learn an angle between &lt;span class=&quot;math-container&quot;&gt;$[0, 2\pi]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;. If the network outputted e.g. 10 in the first dimension then this would just be clipped to &lt;span class=&quot;math-container&quot;&gt;$2\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This way the agent only learns about actions within the action space and the weights of the network would eventually be adjusted to only output actions within this action space, provided that the boundaries aren't the optimal actions.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2021-01-08T10:50:44.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25632" PostTypeId="1" CreationDate="2021-01-08T11:07:07.600" Score="1" ViewCount="120" Body="&lt;p&gt;I'm studying machine learning and I came into a challenging question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Tprev.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Tprev.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The answer is 2. But based on my ML notes, all of them are true. Where are the wrong points?&lt;/p&gt;&#xA;" OwnerUserId="43653" LastEditorUserId="43653" LastEditDate="2021-01-08T13:25:56.967" LastActivityDate="2021-02-12T11:05:25.467" Title="If the training data are linearly separable, which of the following $L(w)$ has less optimum answer for $w$, when $y = w^Tx$?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;loss-functions&gt;&lt;support-vector-machine&gt;&lt;homework&gt;" AnswerCount="2" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="25633" PostTypeId="2" ParentId="25630" CreationDate="2021-01-08T11:11:04.687" Score="6" Body="&lt;p&gt;The probability density is used to 'measure how good' the parameters are because it is a natural way of quantifying if these parameters are good for the observed data.&lt;/p&gt;&#xA;&lt;p&gt;Also, as the notation often causes some confusion, &lt;span class=&quot;math-container&quot;&gt;$L(\theta | x)$&lt;/span&gt; denotes the probability of &lt;em&gt;all&lt;/em&gt; of your observed data, not just one value. Also the &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$|$&lt;/span&gt;&amp;quot; may cause confusion as it looks like we are conditioning on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; but this is not the case - it may be better practice to use &lt;span class=&quot;math-container&quot;&gt;$L(\theta; x)$&lt;/span&gt; which is the notation used when I was learning likelihood. Further, as you have written &lt;span class=&quot;math-container&quot;&gt;$L(\theta | x) = p_\theta(x)$&lt;/span&gt; I would like to clarify that if we are being precise in our definitions then this is only correct if you have one observed data point, assuming that you meant &lt;span class=&quot;math-container&quot;&gt;$p_\theta(x)$&lt;/span&gt; is the density of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In my example below I use &lt;span class=&quot;math-container&quot;&gt;$p_\mu(x)$&lt;/span&gt; to denote the density of the normal distribution with mean parameter &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, but the density of the likelihood is the &lt;em&gt;product&lt;/em&gt; of all the densities (because we assumed iid data). It is crucial that you understand that in general the likelihood is the probability of your observed data and not just the density or the product of densities as you may not always have iid and so it will not always boil down to taking the product of some densities.&lt;/p&gt;&#xA;&lt;p&gt;The idea of Maximum Likelihood is to maximising the (log-)likelihood for a given set of data. This means that we need to choose a probability distribution that is parameterised by some parameters &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\theta}$&lt;/span&gt; and then optimise the parameters such that the likelihood is maximum.&lt;/p&gt;&#xA;&lt;p&gt;Assuming we don't remove any constants then this makes intuitive sense as maximising the likelihood would be maximising the probability that the data came from this distribution -- i.e. the data we observed is most likely to have come from the given distribution with the given parameters. This is important as it means we then have the most likely model of our data which allows us to use this distribution to make inference about our data.&lt;/p&gt;&#xA;&lt;p&gt;As an example, imagine if I had some iid data &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, ..., x_n \sim \mathcal{N}(0, 1)$&lt;/span&gt;. Now I could try to fit a &lt;span class=&quot;math-container&quot;&gt;$\mathcal{N}(\mu, 1)$&lt;/span&gt; to this data and optimise for &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;. If I chose e.g. &lt;span class=&quot;math-container&quot;&gt;$\mu = -1000000$&lt;/span&gt; then the likelihood (again assuming we don't remove any constants) would be &lt;span class=&quot;math-container&quot;&gt;$\approx 0$&lt;/span&gt;. If I chose a value of &lt;span class=&quot;math-container&quot;&gt;$\mu = 0.1$&lt;/span&gt; then the likelihood would be much higher because this parameter is closer to the true parameter value.&lt;/p&gt;&#xA;&lt;p&gt;To see why it is higher, recall that the likelihood for iid data is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\prod_{i=1}^n p_\mu(x_i)$$&lt;/span&gt;&#xA;and if we evaluated our likelihood at &lt;span class=&quot;math-container&quot;&gt;$\mu=-10000000$&lt;/span&gt; then you're going to be taking the product of lots of numbers that are &lt;span class=&quot;math-container&quot;&gt;$\approx 0$&lt;/span&gt; - if you think about the bell curve shape of a Normal distribution that is centred at &lt;span class=&quot;math-container&quot;&gt;$-10000000$&lt;/span&gt; with variance 1 then the density at the true &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; values (recalling they are simulated from a unit Normal) would be approx &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; - whereas if we evaluated the &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; at the density of a Normal distribution centred at &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; then the density will be non-zero and so your likelihood will have a higher value.&lt;/p&gt;&#xA;&lt;p&gt;To summarise, the density value can be used to measure how good parameters are for a set of data as maximising wrt the parameters is analogous to maximising the probability that your data arose from said distribution.&lt;/p&gt;&#xA;&lt;p&gt;As an aside, note that the definition of likelihood is the probability of observing your data under some assumed distribution. For discrete random variables this is fine, but for continuous distributions we have to be a more subtle. For any continuous random variable &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; we have &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}(X=x) = 0$&lt;/span&gt;. However, for a very small &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; we can say that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbb{P}(x - \frac{\delta}{2} &amp;lt; X \leq x + \frac{\delta}{2}) = \int_{x - \frac{\delta}{2}}^{x + \frac{\delta}{2}} f_X(x) dx \approx \delta p_X(x) \; ;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;you can think of this approximation by visualising an integral and recalling that an integral represents the area under the curve, and so for small &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; this integral can be approximated by taking the area of a rectangle which is width &lt;span class=&quot;math-container&quot;&gt;$\times$&lt;/span&gt; height, where the width is &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; and the height is &lt;span class=&quot;math-container&quot;&gt;$f_X(x)$&lt;/span&gt;. This justifies our use of the density function for continuous random variables. Note that typically in maximum likelihood we omit any multiplicative constants as they don't depend on the parameters which is what happens with the &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; from this justification.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-01-08T12:53:49.683" LastActivityDate="2021-01-08T12:53:49.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25634" PostTypeId="1" CreationDate="2021-01-08T11:49:11.270" Score="2" ViewCount="47" Body="&lt;p&gt;In Barton and Sutton's book, &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=311&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reinforcement Learning: An Introduction (2nd edition), an expression, on page 289 (equation 12.2)&lt;/a&gt;, introduced the form of the &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G_t^{\lambda} = (1-\lambda)\sum_{n=1}^{\infty} \lambda^{n-1}G_{t:t+n} \label{12.2}\tag{12.2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with the truncated return defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ G_{t:t+n} \doteq R_{t+1} +\gamma R_{t+2} + \ldots + \gamma^{n-1} R_{t+n} + \gamma^{n}\hat{v}(S_{t+n}, \mathbf{w}_{t+n-1}) \label{12.1}\tag{12.1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, slightly later in the text, &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=312&quot; rel=&quot;nofollow noreferrer&quot;&gt;page 290 (equation 12.4)&lt;/a&gt;, the update algorithm for the offline &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return algorithm is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{w}_{t+1} \doteq \mathbf{w}_{t}+\alpha\left[G_{t}^{\lambda}-\hat{v}\left(S_{t}, \mathbf{w}_{t}\right)\right] \nabla \hat{v}\left(S_{t}, \mathbf{w}_{t}\right), \quad t=0, \ldots, T-1 \label{12.4}\tag{12.4}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is: &lt;em&gt;how do we bootstrap the truncated returns in the update algorithm?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The way the truncated return is currently defined can not plausibly be used, since we would not have access to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}_{t+n-1}$&lt;/span&gt;, as we are in the process of finding &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}_{t+1}$&lt;/span&gt;. I suspect &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}_{t}$&lt;/span&gt; is used for bootstrapping in all returns, but that would alter the definition of the truncated return which I just wanted to clarify.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;And as a follow-up question: &lt;em&gt;What weights are used for bootstrapping in the &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=298&quot; rel=&quot;nofollow noreferrer&quot;&gt;online &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return algorithm described on page 298&lt;/a&gt;?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I assume it's either &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}_{t-1}^{h}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w}_{h-1}^{h-1}$&lt;/span&gt;, it's briefly mentioned that the online &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;-return algorithm performs slightly better than the offline one at the end of the episode which leads me to believe the latter is used otherwise the two algorithms would be identical.&lt;/p&gt;&#xA;&lt;p&gt;Any insight into either question would be great.&lt;/p&gt;&#xA;" OwnerUserId="42514" LastEditorUserId="42514" LastEditDate="2021-01-09T12:43:38.037" LastActivityDate="2021-01-09T12:43:38.037" Title="How does bootstrapping work with the offline $\lambda$-return algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;td-lambda&gt;&lt;bootstrapping&gt;&lt;lambda-return&gt;&lt;lambda-return-algorithm&gt;" AnswerCount="0" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25635" PostTypeId="2" ParentId="21911" CreationDate="2021-01-08T13:41:04.433" Score="2" Body="&lt;p&gt;Your bias formula is a bit incorrect. You should subtract a true value, not an estimator.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathrm{b}(\hat{\theta}) \stackrel{\text { def }}{=} \mathbb{E}[\hat{\theta}]-\theta$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathrm{b}\left(\max _{a} Q\right)=\mathbb{E}\left[\max _{a} Q\right]-\max _{a} q=\mathbb{E}\left[\max _{a} Q\right]-\max _{a} \mathbb{E}[Q]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}\left[\max _{a} Q\right] \geq \max _{a} \mathbb{E}[Q] \Rightarrow \mathrm{b}\left(\max _{a} Q\right) \geq 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="42983" LastEditorUserId="2444" LastEditDate="2021-01-19T11:22:18.193" LastActivityDate="2021-01-19T11:22:18.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25636" PostTypeId="2" ParentId="25627" CreationDate="2021-01-08T14:50:21.233" Score="1" Body="&lt;p&gt;A critical goal of training a neural network is to minimize the loss.  Loss is not explained for spaCy because it is a general concept for machine learning and deep learning.  Loss is not specific to spaCy and although there are some finer details I don't believe that is your inquiry.&lt;/p&gt;&#xA;&lt;p&gt;In general, to understand loss functions, I recommend the following resources:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://machinelearningmastery.com/loss-and-loss-functions-for-training-deep-learning-neural-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loss and Loss Functions for Training Deep Learning Neural Networks&lt;/a&gt; by Jason Brownlee.  Jason writes some great articles and tutorials and I highly suggest reading his other articles and books.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://algorithmia.com/blog/introduction-to-loss-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Loss Functions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/ai%C2%B3-theory-practice-business/loss-functions-in-machine-learning-for-beginners-fastai-lesson-9-homework-2-2121954f1f77&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loss Functions in Machine Learning for Beginners&lt;/a&gt; by John Kaller&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you like videos watch:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=IVVVjBSk9N0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loss Functions Explained&lt;/a&gt; by Siraj Raval.  Siraj has many great videos on machine learning.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2021-01-08T15:01:59.910" LastActivityDate="2021-01-08T15:01:59.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25639" PostTypeId="1" CreationDate="2021-01-08T18:30:48.980" Score="0" ViewCount="16" Body="&lt;p&gt;Could you recommend me the most suitable algorithms for vehicle trajectory prediction? Although I'm studying kalman filter approaches and they are the most used, are there other common (NN, SVM, RF, etc.) techniques?&lt;/p&gt;&#xA;" OwnerUserId="43650" LastActivityDate="2021-01-08T18:30:48.980" Title="Recommended vehicle trajectory prediction algorithms (based only on lat/lon data)?" Tags="&lt;machine-learning&gt;&lt;prediction&gt;&lt;autonomous-vehicles&gt;&lt;forecasting&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25641" PostTypeId="2" ParentId="17098" CreationDate="2021-01-08T19:27:23.157" Score="-1" Body="&lt;p&gt;Maybe Just use simple Convnets (Pre-trained perhaps) and train it on the images of the teacher on the blackboard. You &lt;em&gt;could&lt;/em&gt; use a GAN to remove the teacher and complete the rest of the image (&lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwj9lKKSh43uAhWbfH0KHeaDC1UQFjAAegQIAxAC&amp;amp;url=http%3A%2F%2Fstanford.edu%2Fclass%2Fee367%2FWinter2018%2Ffu_guan_yang_ee367_win18_report.pdf&amp;amp;usg=AOvVaw2tG3bStRlys_NPLX9-XPep&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwj9lKKSh43uAhWbfH0KHeaDC1UQFjAAegQIAxAC&amp;amp;url=http%3A%2F%2Fstanford.edu%2Fclass%2Fee367%2FWinter2018%2Ffu_guan_yang_ee367_win18_report.pdf&amp;amp;usg=AOvVaw2tG3bStRlys_NPLX9-XPep&lt;/a&gt;) But that would be too troublesome.&lt;/p&gt;&#xA;&lt;p&gt;The best way would be to take real-time video chunks, use a Convolutional Network to detect the shapes you want, and return bounding boxes for the appropriate shapes (for the location purposes) and their 'classification' - whether a straight line, curved line, or some other user defined shape. You could also choose to use some other YOLO (You Only Look Once) technique. You can check out with: &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjAJegQIARAC&amp;amp;url=https%3A%2F%2Ftowardsdatascience.com%2Fobject-detection-using-deep-learning-approaches-an-end-to-end-theoretical-perspective-4ca27eee8a9a&amp;amp;usg=AOvVaw2TkW094_O5Q7-mcVMJ5SEN&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjAJegQIARAC&amp;amp;url=https%3A%2F%2Ftowardsdatascience.com%2Fobject-detection-using-deep-learning-approaches-an-end-to-end-theoretical-perspective-4ca27eee8a9a&amp;amp;usg=AOvVaw2TkW094_O5Q7-mcVMJ5SEN&lt;/a&gt; ;&lt;/p&gt;&#xA;&lt;p&gt;You also won't have to deal with the pesky teacher with the above method (assuming he doesn't stand in one place and constantly block a part of the BB). These methods are near SOTA and would be &lt;strong&gt;extremely&lt;/strong&gt; effective than conventional algos. Not to mention that using &lt;code&gt;Keras&lt;/code&gt; is a piece of cake with a giant community and endless resources to help you in case you get stuck on some problem. Since it is easy to use, you can setup a prototype in almost no time.&lt;/p&gt;&#xA;&lt;p&gt;Beginner's guide and Introduction-&amp;gt; &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjALegQIGhAC&amp;amp;url=https%3A%2F%2Fmachinelearningmastery.com%2Fobject-recognition-with-deep-learning%2F&amp;amp;usg=AOvVaw3M-b1gYnbTdzzwPahTTWWR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjALegQIGhAC&amp;amp;url=https%3A%2F%2Fmachinelearningmastery.com%2Fobject-recognition-with-deep-learning%2F&amp;amp;usg=AOvVaw3M-b1gYnbTdzzwPahTTWWR&lt;/a&gt; ;&lt;/p&gt;&#xA;&lt;p&gt;A research paper from ArXiv--&amp;gt; &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjAPegQIHhAC&amp;amp;url=https%3A%2F%2Farxiv.org%2Fpdf%2F1807.05511&amp;amp;usg=AOvVaw33CWoOX4LlrA3T_f75zVZu&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjAPegQIHhAC&amp;amp;url=https%3A%2F%2Farxiv.org%2Fpdf%2F1807.05511&amp;amp;usg=AOvVaw33CWoOX4LlrA3T_f75zVZu&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Training with YOLO: &lt;a href=&quot;https://machinelearningmastery.com/how-to-perform-object-detection-with-yolov3-in-keras/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://machinelearningmastery.com/how-to-perform-object-detection-with-yolov3-in-keras/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="36322" LastEditorUserId="36322" LastEditDate="2021-01-08T19:38:37.693" LastActivityDate="2021-01-08T19:38:37.693" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25642" PostTypeId="2" ParentId="25617" CreationDate="2021-01-08T19:42:43.200" Score="1" Body="&lt;p&gt;Just use &lt;code&gt;IMGAug&lt;/code&gt; library for Applying the 'zoom' augmentation on the images and a convnet (or even MLP) would have no problem in this task. Zooming on the image would be more than enough to use as, as long as your zoom is of sufficient power.&lt;/p&gt;&#xA;&lt;p&gt;So apply &amp;quot;zoom&amp;quot; of the same factor on all images, make them bigger, and Convolutional Neural Networks would do fine. Also, If color is not an important feature, then you would be better off rescaling images b/w &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt; and making them GrayScale&lt;/p&gt;&#xA;" OwnerUserId="36322" LastActivityDate="2021-01-08T19:42:43.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25643" PostTypeId="1" CreationDate="2021-01-08T19:48:29.787" Score="0" ViewCount="23" Body="&lt;p&gt;I would like some references of works that try to understand the functioning of any kind of RNN in natural language processing tasks. They can be any work that tries to explain the functioning of the model by studying the structure of the model itself. I have the feeling that it is very common for researchers to use models, but there is still little theory about how they work in solving natural language processing tasks.&lt;/p&gt;&#xA;" OwnerUserId="36175" LastEditorUserId="2444" LastEditDate="2021-01-09T01:37:20.793" LastActivityDate="2021-01-09T01:37:20.793" Title="Is there a reference that describes Recurrent Neural Networks for NLP tasks?" Tags="&lt;deep-learning&gt;&lt;natural-language-processing&gt;&lt;recurrent-neural-networks&gt;&lt;reference-request&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25645" PostTypeId="1" CreationDate="2021-01-08T20:21:12.830" Score="3" ViewCount="78" Body="&lt;p&gt;I am trying to understand the solution to part 4 of problem 3 from the midterm exam &lt;a href=&quot;https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-867-machine-learning-fall-2006/exams/midterm_f03soln.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;6.867 Machine learning: Mid-term exam (October 15, 2003)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For reproducibility, here is problem 3.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We consider here linear and non-linear support vector machines (SVM) of the form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{equation}&#xA;\begin{aligned}&#xA;\min w_{1}^{2} / 2 &amp;amp; \text { subject to } y_{i}\left(w_{1} x_{i}+w_{0}\right)-1 \geq 0, \quad i=1, \ldots, n, \text { or } \\&#xA;\min \mathbf{w}^{T} \mathbf{w} / 2 &amp;amp; \text { subject to } y_{i}\left(\mathbf{w}^{T} \Phi_{i}+w_{0}\right)-1 \geq 0, \quad i=1, \ldots, n&#xA;\end{aligned}&#xA;\end{equation}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\Phi_{i}$&lt;/span&gt; is a feature vector constructed from the corresponding real-valued input &lt;span class=&quot;math-container&quot;&gt;$x_{i}$&lt;/span&gt;. We wish to compare the simple linear SVM classifier &lt;span class=&quot;math-container&quot;&gt;$\left(w_{1} x+w_{0}\right)$&lt;/span&gt; and the non-linear classifier&#xA;&lt;span class=&quot;math-container&quot;&gt;$\left(\mathbf{w}^{T} \Phi+w_{0}\right)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\Phi=\left[x, x^{2}\right]^{T}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here is part 4 of the same problem.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In general, is the margin we would attain using scaled feature vectors&#xA;&lt;span class=&quot;math-container&quot;&gt;$\Phi=\left[2 x, 2 x^{2}\right]^{T}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;greater&lt;/li&gt;&#xA;&lt;li&gt;equal&lt;/li&gt;&#xA;&lt;li&gt;smaller&lt;/li&gt;&#xA;&lt;li&gt;any of the above&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The correct answer is the first (&lt;strong&gt;greater&lt;/strong&gt;). Why is that the case?&lt;/p&gt;&#xA;" OwnerUserId="43665" LastEditorUserId="43665" LastEditDate="2021-01-09T19:30:38.520" LastActivityDate="2021-01-09T19:30:38.520" Title="Why is the margin attained with $\Phi=\left[2 x, 2 x^{2}\right]^{T}$ greater than the margin attained with $\Phi=\left[x, x^{2}\right]^{T}$?" Tags="&lt;machine-learning&gt;&lt;classification&gt;&lt;support-vector-machine&gt;&lt;homework&gt;&lt;kernel-trick&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25646" PostTypeId="2" ParentId="25632" CreationDate="2021-01-08T20:58:27.287" Score="0" Body="&lt;p&gt;First things first. What is an optimal &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;?. In this case it is supposed to be not only the one that minimizes the &lt;strong&gt;emprical /sample loss&lt;/strong&gt;, but also &lt;strong&gt;non-trivial&lt;/strong&gt; as we shall sonn see. Now inspect the loss functions, we see a term &lt;span class=&quot;math-container&quot;&gt;$-y_iw^Tx_i$&lt;/span&gt; coming up. What exactly is this term? It can be anything. The correct term would have been &lt;span class=&quot;math-container&quot;&gt;$-y_i(w^Tx_i + b)$&lt;/span&gt;, or atleast I will assume this term, but the reasoning will exactly be the same without this term.&lt;/p&gt;&#xA;&lt;p&gt;Now this term is well defined and since the Eucledian distance of a point, along with sign, &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; from the hyperplane &lt;span class=&quot;math-container&quot;&gt;$w^Tx + b=0$&lt;/span&gt; (the proof for this can be found easily), is given by &lt;span class=&quot;math-container&quot;&gt;$w^Tx_i + b$&lt;/span&gt;. So if the datapoint is supposed to have &lt;span class=&quot;math-container&quot;&gt;$y_i=-1$&lt;/span&gt; but according to the classifier it gives &lt;span class=&quot;math-container&quot;&gt;$w^Tx_i + b = l_i &amp;gt; 0$&lt;/span&gt;, then one cane see &lt;span class=&quot;math-container&quot;&gt;$-y_il_i &amp;gt; 0$&lt;/span&gt;, hence &lt;span class=&quot;math-container&quot;&gt;$\max (0, -y_i l_i) = -y_i l_i$&lt;/span&gt; a positive loss, and the same will apply to case when &lt;span class=&quot;math-container&quot;&gt;$y_i = 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w^Tx_i + b = l_i &amp;lt;0 $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We see another assumption, the datapoints are linearly seperable. This means, there exists a classifier (or a hyperplane) which can completely seperate the 2 classes. Now if one inspects the loss function &lt;span class=&quot;math-container&quot;&gt;$1,3,4$&lt;/span&gt; all of them have one goal to make &lt;span class=&quot;math-container&quot;&gt;$\max (0, -y_i l_i) = 0, \forall i$&lt;/span&gt;. A point to note is that &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; does this by directly reducing &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^n \max (0, -y_i l_i)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; does this using the &lt;span class=&quot;math-container&quot;&gt;$0-1$&lt;/span&gt; loss i.e if the instance is wrongly classified (or &lt;span class=&quot;math-container&quot;&gt;$-y_il_i &amp;gt; 0$&lt;/span&gt;), then incur &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{n}$&lt;/span&gt; loss else &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; incurs a loss of &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{n}(w^Tx_i + b)^2$&lt;/span&gt; for each wrong classification. It is very important to note that the solution of all these 3 problems are all the same &lt;span class=&quot;math-container&quot;&gt;$w^*,b^*$&lt;/span&gt; which classifies the dataset correctly, which makes the problem equivalent. (This may not hold under some very similar looking loss functions, so it is a point to take care about, a problem is equivalent only if it results in the same solution).&lt;/p&gt;&#xA;&lt;p&gt;In &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, first &lt;span class=&quot;math-container&quot;&gt;$C \rightarrow \infty$&lt;/span&gt; is an incorrect statement, since then not only does it become mathematically unsound (atleast for me), it also means &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; is the same as &lt;span class=&quot;math-container&quot;&gt;$1,3,4$&lt;/span&gt;. Why? &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a weighting factor, it decides how much priority the algorithm minimizing loss will give to the term &lt;span class=&quot;math-container&quot;&gt;$C \sum_{i=1}^n \max(0,-y_il_i)$&lt;/span&gt;. If say &lt;span class=&quot;math-container&quot;&gt;$C=0$&lt;/span&gt; then you are basically optimizing &lt;span class=&quot;math-container&quot;&gt;$||w||_2^2$&lt;/span&gt; whose optimal value is at &lt;span class=&quot;math-container&quot;&gt;$w=0$&lt;/span&gt;. hence, we get nothing useful as optimizing &lt;span class=&quot;math-container&quot;&gt;$||w||_2^2$&lt;/span&gt; has no connection with our objective of reducing misclassifications. But if,  &lt;span class=&quot;math-container&quot;&gt;$0.5||w||_2^2 + C \sum_{i=1}^n \max(0,-y_il_i)$&lt;/span&gt; is used the algorithm is now giving some weightage to minimizing both the &lt;span class=&quot;math-container&quot;&gt;$||w||_2^2$&lt;/span&gt; as well as the misclassifications. If &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is very small the algorithm will prefer to minimize &lt;span class=&quot;math-container&quot;&gt;$||w||_2^2$&lt;/span&gt; rather than &lt;span class=&quot;math-container&quot;&gt;$C \sum_{i=1}^n \max(0,-y_il_i)$&lt;/span&gt;. Thus giving a solution which may not be optimal, as per the dataset. But, if &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is very large the algorithm will mostly try to minimize  &lt;span class=&quot;math-container&quot;&gt;$C \sum_{i=1}^n \max(0,-y_il_i)$&lt;/span&gt;, but still it will give some weightage in minimizng &lt;span class=&quot;math-container&quot;&gt;$||w||_2^2$&lt;/span&gt;, hence may not find an optimal hyperplane. If &lt;span class=&quot;math-container&quot;&gt;$C \rightarrow \infty$&lt;/span&gt; it is matheatically ill defined (as per my knowledge) so I won't go into it.&lt;/p&gt;&#xA;&lt;p&gt;But, all these aforementioned explanation (i.e the tradeoff between optimizaing two different objectives) will be useful only when there are some additional constraints, like instead of &lt;span class=&quot;math-container&quot;&gt;$\max(0,-y_il_i)$&lt;/span&gt; it is &lt;span class=&quot;math-container&quot;&gt;$\max(0,1-y_il_i)$&lt;/span&gt; where making &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; bigger actually makes sense as it will also increase &lt;span class=&quot;math-container&quot;&gt;$l_i$&lt;/span&gt; (i.e the same hyperplane denoted by a larger &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; say &lt;span class=&quot;math-container&quot;&gt;$kw$&lt;/span&gt; will make l_i large hence &lt;span class=&quot;math-container&quot;&gt;$-l_iy_i$&lt;/span&gt; small if correctly classified, but this doesn't make sense if we are talking about the same hyperplane), hence the &lt;span class=&quot;math-container&quot;&gt;$||w||_2^2$&lt;/span&gt; term opposes such an increase. This is a very famous formulation of loss used in SVM.&lt;/p&gt;&#xA;&lt;p&gt;But, the challenging part in this question is the assumption '&lt;strong&gt;linearly seperable&lt;/strong&gt;' and also the additional constraint is missing (which is present in SVMs). So if &lt;span class=&quot;math-container&quot;&gt;$w^*,b^*$&lt;/span&gt; incurs &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; classification error so will &lt;span class=&quot;math-container&quot;&gt;$kw^*, b^*$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is any scaling factor. And since &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is very large the second term in &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; must be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Thus, any &lt;span class=&quot;math-container&quot;&gt;$kw^*, b^*$&lt;/span&gt; will work to make the second term 0. But, the first term now becomes &lt;span class=&quot;math-container&quot;&gt;$0.5k^{2}||w^*||_2^2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$w^*$&lt;/span&gt; denotes the optimal hyperplane, and hence fixed. Thus the first term will choose  &lt;span class=&quot;math-container&quot;&gt;$|k| \rightarrow 0$&lt;/span&gt; or basically &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt; to minmize the 1st term and the second term now simply becomes &lt;span class=&quot;math-container&quot;&gt;$C\sum_{i=1}^n \max(0,-y_ib)$&lt;/span&gt; which can be made &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; by choosing &lt;span class=&quot;math-container&quot;&gt;$b=0$&lt;/span&gt;. Thus, a &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; loss when &lt;span class=&quot;math-container&quot;&gt;$kw^*=0,b^*=0$&lt;/span&gt; or a &lt;strong&gt;trivial solution&lt;/strong&gt; and definitely not optimal as this will be true for all linearly seperable datasets.&lt;/p&gt;&#xA;&lt;p&gt;I assumed and extra &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; term to leverage linear seperability, without it the reasoning becomes easier and you can follow the exact same line of reasoning, albeit the problem might not be linearly seperable anymore. The solution may seem not very elegant but this is a standard line of reasoning in optimization problems where the optimization variable is changed from &lt;span class=&quot;math-container&quot;&gt;$w \rightarrow k$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$w \in \mathbb{R^n}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k \in \mathbb{R}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In short for &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; the solution produced will be &lt;span class=&quot;math-container&quot;&gt;$w^*=0,b^*=0$&lt;/span&gt; which will be the best solution but easily seen as trivial and non optimal at all.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2021-01-13T10:53:38.763" LastActivityDate="2021-01-13T10:53:38.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25647" PostTypeId="2" ParentId="25632" CreationDate="2021-01-08T21:51:07.697" Score="0" Body="&lt;p&gt;Since the data is linearly separable linear model &lt;span class=&quot;math-container&quot;&gt;$y = w^Tx$&lt;/span&gt; will be able to perfectly classify all the examples. That means that loss functions &lt;span class=&quot;math-container&quot;&gt;$L_1(w), L_3(w)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_4(w)$&lt;/span&gt; will have a value of 0 (since all examples are correctly classified). For the loss &lt;span class=&quot;math-container&quot;&gt;$L_2(w)$&lt;/span&gt; second term will be 0 if all examples are correctly classified. The first term of &lt;span class=&quot;math-container&quot;&gt;$L_2(w)$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\frac{1}{2}||w||^2_2&#xA;\end{equation}&lt;/span&gt;&#xA;will be not be minimized to 0 (unless optimal &lt;span class=&quot;math-container&quot;&gt;$w = \mathbf{0}$&lt;/span&gt;) so &lt;span class=&quot;math-container&quot;&gt;$L_2(w) &amp;gt; 0$&lt;/span&gt;. Optimizer will try to minimize &lt;span class=&quot;math-container&quot;&gt;$L_2(w)$&lt;/span&gt; to be 0, so because of the penalty to &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; it will try to find the best tradeoff between minimizing &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and loss due to misclassification so the resulting &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; may not be the best to achieve optimal classification.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastEditorUserId="20339" LastEditDate="2021-01-08T21:57:23.113" LastActivityDate="2021-01-08T21:57:23.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25648" PostTypeId="5" CreationDate="2021-01-09T00:01:42.737" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T00:01:42.737" LastActivityDate="2021-01-09T00:01:42.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25649" PostTypeId="4" CreationDate="2021-01-09T00:01:42.737" Score="0" Body="For questions related to the $\lambda$-return, as defined in section 12.1 of the book &quot;Reinforcement Learning: An Introduction&quot; (2nd edition)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T00:01:42.737" LastActivityDate="2021-01-09T00:01:42.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25650" PostTypeId="5" CreationDate="2021-01-09T00:03:02.750" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T00:03:02.750" LastActivityDate="2021-01-09T00:03:02.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25651" PostTypeId="4" CreationDate="2021-01-09T00:03:02.750" Score="0" Body="For questions related to the online and offline $\lambda$-return algorithm as defined in section 12.4 of the book &quot;Reinforcement Learning: An Introduction&quot; (2nd edition)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T00:03:02.750" LastActivityDate="2021-01-09T00:03:02.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25653" PostTypeId="1" CreationDate="2021-01-09T08:48:02.503" Score="1" ViewCount="44" Body="&lt;p&gt;In a continuous action space (for instance, in PPO, TRPO, REINFORCE, etc.), during training, an action is sampled from the random distribution with &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;. This results in an inherent exploration. However, during testing, when we no longer need to explore but exploit, the action should be deterministic, i.e. just &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, right?&lt;/p&gt;&#xA;" OwnerUserId="32517" LastEditorUserId="32517" LastEditDate="2021-01-09T22:14:38.103" LastActivityDate="2021-01-09T22:14:38.103" Title="Are actions deterministic during testing in continuous action space PPO?" Tags="&lt;reinforcement-learning&gt;&lt;proximal-policy-optimization&gt;&lt;reinforce&gt;&lt;continuous-action-spaces&gt;&lt;exploration-strategies&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25656" PostTypeId="2" ParentId="25596" CreationDate="2021-01-09T15:35:23.307" Score="0" Body="&lt;p&gt;I have never used MATLAB for ml before, so it is difficult for me to understand all your code. My first association to your problem is class imabalance. Since you seem to have got a handle on that, the problem could be dying ReLU or bloated activations. To check if the ReLU is dying, you could look at the activations of the early layers of your network. If many values are zero, it should be dying ReLU.&lt;/p&gt;&#xA;" OwnerUserId="43632" LastActivityDate="2021-01-09T15:35:23.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25657" PostTypeId="1" AcceptedAnswerId="25667" CreationDate="2021-01-09T16:22:46.960" Score="3" ViewCount="94" Body="&lt;p&gt;I would like to employ DQN to solve a constrained MDP problem. The problem has constraints on action space. At different time steps till the end, the available actions are different. It has different possibilities as below.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0, 1, 2, 3, 4&lt;/li&gt;&#xA;&lt;li&gt;0, 2, 3, 4&lt;/li&gt;&#xA;&lt;li&gt;0, 3, 4&lt;/li&gt;&#xA;&lt;li&gt;0, 4&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Does this mean I need to learn 4 different Q networks for these possibilities? Also, correct me if I am wrong, it looks like if I specify the action size is 3, then it automatically assumes the actions are &lt;code&gt;0, 1, 2&lt;/code&gt;, but, in my case, it should be &lt;code&gt;0, 3, 4&lt;/code&gt;. How shall I implement this?&lt;/p&gt;&#xA;" OwnerUserId="23707" LastEditorUserId="2444" LastEditDate="2021-01-09T20:08:03.997" LastActivityDate="2021-01-09T20:33:40.707" Title="How to use DQN when the action space can be different at different time steps?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;action-spaces&gt;&lt;constrained-optimization&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25659" PostTypeId="5" CreationDate="2021-01-09T17:38:19.903" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T17:38:19.903" LastActivityDate="2021-01-09T17:38:19.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25660" PostTypeId="4" CreationDate="2021-01-09T17:38:19.903" Score="0" Body="For questions related to support vectors of a support vector machine (SVM)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T17:38:19.903" LastActivityDate="2021-01-09T17:38:19.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25662" PostTypeId="1" CreationDate="2021-01-09T18:50:46.223" Score="0" ViewCount="17" Body="&lt;p&gt;I'm trying to come up with a loss function for the case, in DDPG, where we have as many outputs from the critic as there are from the actor. So, there will be one Q value for each dimension in the actor's output.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I would have a separate loss function for each of the output dimensions, but this doesn't scale. Also, to shed more light, I have as output from the actor two output layers. One contains the actions, but rather than be continuous I have a SoftMax function on it. While the other contains a conditioning vector that I will use to condition the observations of the next time step. The conditioning vector also has a SoftMax, and I convert the top 2 % values to ones while keeping the rest as zeros before conditioning.&#xA;The reason I use DDPG rather than DQN for this, even though it's a discrete action space, is so that I can have the binary vector besides the actions as output.&lt;/p&gt;&#xA;&lt;p&gt;So, one loss function for this ensemble would go a long way to helping me out. So, this time there will be as many outputs from the critic as there are dimensions in the binary vector, +1 for the action segment.&lt;/p&gt;&#xA;" OwnerUserId="41824" LastEditorUserId="2444" LastEditDate="2021-01-12T00:43:00.713" LastActivityDate="2021-01-12T00:43:00.713" Title="Which loss function should I use to train DDGP with multiple q values, one for each of the output dimensions?" Tags="&lt;reinforcement-learning&gt;&lt;deep-rl&gt;&lt;loss-functions&gt;&lt;actor-critic-methods&gt;&lt;ddpg&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25663" PostTypeId="5" CreationDate="2021-01-09T19:23:41.787" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T19:23:41.787" LastActivityDate="2021-01-09T19:23:41.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25664" PostTypeId="4" CreationDate="2021-01-09T19:23:41.787" Score="0" Body="For questions related to the kernel trick, which is used, for example, in the context of support vector machines (SVMs)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-09T19:23:41.787" LastActivityDate="2021-01-09T19:23:41.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25665" PostTypeId="1" CreationDate="2021-01-09T19:27:38.900" Score="2" ViewCount="24" Body="&lt;p&gt;In a Markov Decision Process, is it possible that there exists no &amp;quot;dominated action&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;I define a dominated action the following way:&#xA;we say that &lt;span class=&quot;math-container&quot;&gt;$(s,a)$&lt;/span&gt; is a dominated action, if &lt;span class=&quot;math-container&quot;&gt;$\forall \pi, a \notin \text{argmax}\ q^{\pi}(s,.)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; are policies.&lt;/p&gt;&#xA;&lt;p&gt;For now, I am only considering the cases where all q-values are distinct and therefore the max is always unique.&#xA;I also only consider the case of deterministic policies (mappings from state space to action space).&lt;/p&gt;&#xA;&lt;p&gt;We can consider MDP in which each state has at least 2 actions available to get rid of the corner cases where there is only one possible policy.&lt;/p&gt;&#xA;&lt;p&gt;I am struggling to find a counter-example or a proof.&lt;/p&gt;&#xA;" OwnerUserId="43682" LastEditorUserId="43682" LastEditDate="2021-01-09T20:23:51.740" LastActivityDate="2021-01-09T20:23:51.740" Title="Does there necessarily exist &quot;dominated actions&quot; in a MDP?" Tags="&lt;reinforcement-learning&gt;&lt;markov-decision-process&gt;&lt;proofs&gt;&lt;value-functions&gt;&lt;policies&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25667" PostTypeId="2" ParentId="25657" CreationDate="2021-01-09T20:25:25.910" Score="2" Body="&lt;p&gt;There are two relevant neural network designs for DQN:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Model q function directly &lt;span class=&quot;math-container&quot;&gt;$Q(s,a): \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$&lt;/span&gt;, so neural network has concatenated input of state and action, and outputs a single real value. This is arguably the more natural fit to Q learning, but can be inefficient.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Model all q values for given state &lt;span class=&quot;math-container&quot;&gt;$Q(s,\cdot): \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}|}$&lt;/span&gt;, so neural network takes input of current state and outputs all action values related to that state as a vector.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For the first architecture, you can decide which actions to evaluate by how you construct the minibatch. You pre-filter to the allowed actions for each state.&lt;/p&gt;&#xA;&lt;p&gt;For the second architecture, you must post-filter the action values to those allowed by the state.&lt;/p&gt;&#xA;&lt;p&gt;There are other possibilities for constructing variable-length inputs and outputs to neural networks - e.g. using RNNs. However, these are normally not worth the extra effort. A pre- or post- filter on the actions for a NN that can process the whole action space (including impossible actions) is all you usually need. Don't worry that the neural network may calculate some non-needed or nonsense values.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-01-09T20:33:40.707" LastActivityDate="2021-01-09T20:33:40.707" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25669" PostTypeId="2" ParentId="25596" CreationDate="2021-01-10T00:13:58.870" Score="1" Body="&lt;p&gt;Similar to other answers, I don't know Matlab that well but you could try the following steps to debug your problem.&lt;/p&gt;&#xA;&lt;h1&gt;Make sure you can overfit to a single instance&lt;/h1&gt;&#xA;&lt;p&gt;from your dataset, pull out a single image with a good amount of true positives in it. Duplicate that images B times (where B = Batch Size) and then try to train your network with only that small dataset. If you can't overfit to a single instance, then something is really wrong and you should validate all functional aspects of your network. If you can overfit, then it's probably more of an algorithmic or data imbalance issue.&lt;/p&gt;&#xA;&lt;h1&gt;Validate Functional Aspects of the Network&lt;/h1&gt;&#xA;&lt;h3&gt;Make sure your input data and labels are correct.&lt;/h3&gt;&#xA;&lt;p&gt;Validate that your images are being correclty inputted into the network. You can do this by printing out any images right before they go into the training function. For the labels, make sure to manually inspect a few labels to be sure that the labels correctly match up the images.&lt;/p&gt;&#xA;&lt;h3&gt;Make sure that your loss function is correct.&lt;/h3&gt;&#xA;&lt;p&gt;Add a unit test or two to your loss function to validate that it is doing what it should be doing. Create a simiple example that you can easily validate.&lt;/p&gt;&#xA;&lt;h3&gt;Validate the rest of the non-algorithmic functionality&lt;/h3&gt;&#xA;&lt;p&gt;Anything that isn't a design choice should be validated. Make sure that the weights are the correct sizes, each layer has the correct number of weights, the intermediate features have the correct shape, etc.&lt;/p&gt;&#xA;&lt;h1&gt;Data Imbalance&lt;/h1&gt;&#xA;&lt;p&gt;If you were able to overfit to a single image, and validated the functional aspects of your network then you may be facing a data imbalance issue. Look at your dataset and see what % of your instances are true positives vs. true negatives. If you have an exteme imbalance (like 10% 90%) or something like that, then build a dataset that is more balanced and see if you can fit your data. If you fit the data with that more balanced dataset, then there's plenty of ways to fix your data imbalance issue. Google around for data imbalance and you should get a few good ideas. Some include focal loss, upsampling, etc...&lt;/p&gt;&#xA;&lt;h1&gt;Check your receptive field&lt;/h1&gt;&#xA;&lt;p&gt;The receptive field on a network is basically the area that can be looked at by the network for a specific region. This is controlled by the size of the convolution filters, stride, etc. If the data that you are segmenting doesn't fit within the receptive field of the model, then you might be saturating the loss function. tldr try playing around with kernel sizes.&lt;/p&gt;&#xA;" OwnerUserId="17408" LastActivityDate="2021-01-10T00:13:58.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25670" PostTypeId="1" CreationDate="2021-01-10T00:54:10.083" Score="1" ViewCount="54" Body="&lt;p&gt;I am currently trying to write a CNN from scratch, but I don't understand how to feed the information from a max-pooling layer to the next convolutional layer. Specifically, I don't know what to do with the 6 filtered and pooled images from the first convolutional and max-pooling layers. How do I feed those images into the next convolutional layer?&lt;/p&gt;&#xA;" OwnerUserId="43686" LastEditorUserId="2444" LastEditDate="2021-01-11T00:44:24.123" LastActivityDate="2021-01-11T00:44:24.123" Title="How do you pass the image from one convolutional layer to another in a CNN?" Tags="&lt;convolutional-neural-networks&gt;&lt;implementation&gt;&lt;convolutional-layers&gt;&lt;pooling-layers&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25671" PostTypeId="1" AcceptedAnswerId="25680" CreationDate="2021-01-10T02:27:02.730" Score="3" ViewCount="68" Body="&lt;p&gt;I am wondering what the parameter &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the function &lt;span class=&quot;math-container&quot;&gt;$g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$&lt;/span&gt; stands for in Section 6 (page 14) of the &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/BF00992696.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; introducing the REINFORCE family of algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Drawing an analogy to Equation 4 of the same paper, I would guess that it refers to the outcome (i.e. &lt;em&gt;sample&lt;/em&gt;) of sampling from a probability distribution parameterized by the parameters &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;. However, I am not sure whether that is correct or not.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="2444" LastEditDate="2021-01-10T15:42:55.207" LastActivityDate="2021-01-10T15:42:55.207" Title="What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;reinforce&gt;&lt;notation&gt;" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25672" PostTypeId="2" ParentId="25671" CreationDate="2021-01-10T11:06:45.823" Score="0" Body="&lt;p&gt;It states that &amp;quot;To simplify notation, we focus on one single unit and omit the usual unit index subscript throughout&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;So they are simply removing the i-th index from the equation for simplicity.  So g is a function of a given instance &amp;quot;y&amp;quot; and the parameters μ and σ.&lt;/p&gt;&#xA;" OwnerUserId="43651" LastActivityDate="2021-01-10T11:06:45.823" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25673" PostTypeId="1" CreationDate="2021-01-10T11:16:36.307" Score="0" ViewCount="7" Body="&lt;p&gt;I'm exploring this idea:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I record IMU data for &amp;quot;primitive&amp;quot; gestures (up-down, left-right, diagonal, circle...)&lt;/li&gt;&#xA;&lt;li&gt;I train a classifier to classify them (on my dataset, RandomForest achieves &amp;gt;90% accuracy, that's fine for me for now)&lt;/li&gt;&#xA;&lt;li&gt;I record IMU data for &amp;quot;composite&amp;quot; gestures (letters A, W, S...)&lt;/li&gt;&#xA;&lt;li&gt;I use the trained classifier as a feature extractor on this new dataset (concatenating 5-6 predictions to produce a new feature vector)&lt;/li&gt;&#xA;&lt;li&gt;I train a second classifier to predict composite gestures based on the extracted feature vector (+ some more aggregate features from the 5-6 primitive gestures)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm sure something of this kind already exists, but I cannot find the right Google query to search for it. Does this thing has a name? It doesn't have to be specific for gestures, I'm interested on the core idea.&lt;/p&gt;&#xA;&lt;p&gt;Already tried &amp;quot;Cascading classifiers&amp;quot; and &amp;quot;Classifiers chain&amp;quot;, but they're not relevant to my problem.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Please avoid NN related answers, I'm interested in &amp;quot;classic&amp;quot; ML classifiers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="43690" LastActivityDate="2021-01-10T11:16:36.307" Title="How to classify &quot;composite&quot; gestures from &quot;primitive&quot; gestures (with IMU data)" Tags="&lt;machine-learning&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25674" PostTypeId="1" CreationDate="2021-01-10T11:44:19.597" Score="2" ViewCount="70" Body="&lt;p&gt;I have an image and a mask. I want the image to be the same, but rotated, scaled and positioned like mask. What can I use?&lt;a href=&quot;https://i.stack.imgur.com/z3kSl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z3kSl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43631" LastActivityDate="2021-01-10T17:09:27.140" Title="What algorithm would you advise me to use for my task?" Tags="&lt;image-processing&gt;&lt;image-generation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25675" PostTypeId="1" CreationDate="2021-01-10T12:33:11.753" Score="2" ViewCount="20" Body="&lt;p&gt;I found a similar &lt;a href=&quot;https://ai.stackexchange.com/questions/11716/can-dqn-perform-better-than-double-dqn&quot;&gt;post&lt;/a&gt; about this issue, but unfortunately I did not find a proper answer. Are there any references where DQN is better than DoubleDQN, that is DoubleDQN does not improve DQN ?&lt;/p&gt;&#xA;" OwnerUserId="36055" LastActivityDate="2021-01-10T12:33:11.753" Title="Can DQN outperform DoubleDQN?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;reference-request&gt;&lt;double-dqn&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25676" PostTypeId="1" AcceptedAnswerId="25682" CreationDate="2021-01-10T12:39:09.197" Score="2" ViewCount="51" Body="&lt;p&gt;I came up with an NLP-related problem where I have a list of words and a string. My goal is to find any word in the list of words that is related to the given string.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example.&lt;/p&gt;&#xA;&lt;p&gt;Suppose a word from the list is &lt;strong&gt;healthy&lt;/strong&gt;. If the string has any of the following words: healthy, healthier, healthiest, not healthy, more healthy, zero healthy, etc., it will be extracted from the string.&lt;/p&gt;&#xA;&lt;p&gt;Also, I want to judge whether the extracted word/s is/are bearing positive/negative sentiment.&lt;/p&gt;&#xA;&lt;p&gt;Let me further explain what I mean by using the previous example.&lt;/p&gt;&#xA;&lt;p&gt;Our word was &lt;strong&gt;healthy&lt;/strong&gt;. So, for instance, if the word found in the string was &lt;strong&gt;healthier&lt;/strong&gt;, then we can say it is bearing &lt;strong&gt;positive&lt;/strong&gt; sentiment with respect to the word &lt;strong&gt;healthy&lt;/strong&gt;. If we find the word &lt;strong&gt;not healthy&lt;/strong&gt;, it is &lt;strong&gt;negative&lt;/strong&gt; with respect to the word &lt;strong&gt;healthy&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37980" LastEditorUserId="2444" LastEditDate="2021-01-11T00:31:33.670" LastActivityDate="2021-01-11T00:31:33.670" Title="How can I find words in a string that are related to a given word, then associate a sentiment to that found word?" Tags="&lt;natural-language-processing&gt;&lt;sentiment-analysis&gt;&lt;stemming&gt;&lt;lemmatization&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25677" PostTypeId="2" ParentId="9953" CreationDate="2021-01-10T12:45:09.823" Score="2" Body="&lt;p&gt;The answer is it depends.&lt;/p&gt;&#xA;&lt;p&gt;Are you referring to GPU vs CPU when training, or running inference?&#xA;When you say CPU is it an x86 CPU or ARM CPU?&#xA;What algorithms are running?&lt;/p&gt;&#xA;&lt;p&gt;Let's look at image classification. Suppose you select a &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convolutional Neural Network&lt;/a&gt;(CNN) as your algorithm. CNN's were created in the 1980s, but only really saw use when GPU's were used. In 2004 &lt;a href=&quot;https://doi.org/10.1016%2Fj.patcog.2004.01.013&quot; rel=&quot;nofollow noreferrer&quot;&gt;K. S. Oh and K. Jung&lt;/a&gt; showed that a GPU was 20x faster than a CPU. In &lt;a href=&quot;https://www.computer.org/csdl/proceedings-article/icdar/2005/24201115/12OmNylKAVX&quot; rel=&quot;nofollow noreferrer&quot;&gt;2005 Steinkrau et al&lt;/a&gt; showed a 3x speed increase.&lt;/p&gt;&#xA;&lt;p&gt;Dan Ciresan et al showed in 2010 and 2011 that a GPU can be up to 60 times faster than a CPU. They used the &lt;a href=&quot;https://en.wikipedia.org/wiki/MNIST_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt; handwritten digits benchmark. A simple google search of performance CPU vs GPU will result in many papers that show how much faster a GPU is compared to a CPU.&lt;/p&gt;&#xA;&lt;p&gt;In this &lt;a href=&quot;https://www.diva-portal.org/smash/get/diva2:1354858/FULLTEXT01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; Eric Lind and Avelin Parnigoso compare the performance differences between a CPU and a GPU using &lt;a href=&quot;https://www.tensorflow.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TensorFlow&lt;/a&gt;. In their study they used a Core i7-6500 2.5 GHz CPU and an Nvidia GTX 1070 GPU. In their study they used three CNN &lt;a href=&quot;https://en.wikipedia.org/wiki/AlexNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlexNet&lt;/a&gt;, Text Classifiction, and the Mnist Digit Classification.&lt;/p&gt;&#xA;&lt;p&gt;Eric and Avelin showed that GPU outperforms CPU when training, and that during inference the GPU can be slower. In their study AlexNet for example ran for 4 000 000ms on a CPU and under 100 000ms on a GPU. That's a 40x difference in training performance. Converting this to cost. Eric and Avelin did not mention how much RAM.&lt;br /&gt;&#xA;I am going to make some assumptions and select the following instance types from &lt;a href=&quot;https://aws.amazon.com/sagemaker/pricing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS&lt;/a&gt; for training:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ml.g4dn.xlarge [GPU Accelerated] \$0.736/hour [4 vCPU, 16 GiB RAM]&lt;/li&gt;&#xA;&lt;li&gt;ml.c5n.xlarge \$0.302/hour [4 vCPU, 10.5 GiB RAM]&lt;/li&gt;&#xA;&lt;li&gt;ml.c5n.2xlarge \$0.605/hour [8 vCPU, 21GiB RAM]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://aws.amazon.com/about-aws/whats-new/2018/11/introducing-amazon-ec2-c5n-instances/&quot; rel=&quot;nofollow noreferrer&quot;&gt;c5n's&lt;/a&gt; are powered by Intel 3.0 Ghz Xeon CPUs and the &lt;a href=&quot;https://aws.amazon.com/ec2/instance-types/g4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;g4dn&lt;/a&gt; feature Nvidia T4 GPUs.&lt;/p&gt;&#xA;&lt;p&gt;Based on this GPU Accelerated will run at less than 100 000ms or 100s while non GPU accelerated will run at 4 000 000ms or 4 000s. The result is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ml.g4dn.xlarge - estimated cost \$1.226&lt;/li&gt;&#xA;&lt;li&gt;ml.c5n.xlarge - estimated cost \$20.133&lt;/li&gt;&#xA;&lt;li&gt;ml.c5n.2xlarge - estimated cost \$40.333&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Based on the experiments showed by Eric and Avelin the CPU instance must be 40x less in cost to be on par with a GPU instance. As you can see the standard CPU is 20 and 40 times more expensive than the GPU. The thought experiment above is based on the paper times for training, not inference, and naturally to get a better answer should be benchmarked.&lt;/p&gt;&#xA;&lt;p&gt;However, AWS has released their own ARM based CPU's designed specifically for training, and inference.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://aws.amazon.com/ec2/graviton/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS Graviton Processor&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://aws.amazon.com/machine-learning/inferentia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS Inferentia&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://aws.amazon.com/machine-learning/trainium/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS Trainium&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both the Inferentia and Trainium chips offer 30% higher throughput and 45% lower cost per inference than Amazon EC2 G4.&lt;/p&gt;&#xA;" OwnerUserId="20508" LastEditorUserId="20508" LastEditDate="2021-01-19T09:11:57.200" LastActivityDate="2021-01-19T09:11:57.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25678" PostTypeId="1" CreationDate="2021-01-10T12:53:44.773" Score="3" ViewCount="232" Body="&lt;p&gt;The motivation for the introduction of &lt;em&gt;double DQN&lt;/em&gt; (and double Q-learning) is that the regular Q-learning (or DQN) can overestimate the Q value, but is there a brief explanation as to why it is overestimated?&lt;/p&gt;&#xA;" OwnerUserId="43246" LastEditorUserId="2444" LastEditDate="2021-01-10T17:55:31.273" LastActivityDate="2021-01-10T21:13:42.720" Title="Why does regular Q-learning (and DQN) overestimate the Q values?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;double-dqn&gt;&lt;double-q-learning&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25679" PostTypeId="2" ParentId="42" CreationDate="2021-01-10T13:29:49.417" Score="2" Body="&lt;p&gt;Actually, the hierarchical learning explanation given by &lt;a href=&quot;https://ai.stackexchange.com/a/51/2444&quot;&gt;mindcrime&lt;/a&gt; is not that acceptable anymore (&lt;a href=&quot;https://youtu.be/Z6rxFNMGdn0?t=260&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; was also indicated by Ian Goodfellow). Since there are neural networks with 150 layers or more, and this explanation does not make sense for such neural networks.  However, we can think of it as solving the knots of high dimensional manifolds, i.e. we transform the input into high dimensional space, and this helps us to find a better representation of the data.&lt;/p&gt;&#xA;&lt;p&gt;A geometric interpretation was explained as such in the book &lt;em&gt;Deep Learning with Python&lt;/em&gt; by François Chollet:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;...you can interpret a neural network as a very complex geometric transformation in a high-dimensional space, implemented via a long series of simple steps...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Imagine two sheets of colored paper: one red and one blue. Put one on top of the other. Now crumple them together into a small ball. That crumpled paper ball is your input data, and each sheet of paper is a class of data in a classification problem. What a neural network (or any other machine-learning model) is meant to do is figure out a transformation of the paper ball that would uncrumple it, so as to make the two classes cleanly separable again. With deep learning, this would be implemented as a series of simple transformations of the 3D space, such as those you could apply on the paper ball with your fingers, one movement at a time. Uncrumpling paper balls is what machine learning is about: finding neat representations for complex, highly folded data manifolds. At this point, you should have a pretty good intuition as to why deep learning excels at this: it takes the approach of incrementally decomposing a complicated geometric transformation into a long chain of elementary ones, which is pretty much the strategy a human would follow to uncrumple a paper ball. Each layer in a deep network applies a transformation that disentangles the data a little—and a deep stack of layers makes tractable an extremely complicated disentanglement process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I suggest you to read &lt;a href=&quot;https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this brilliant blog post&lt;/a&gt; to learn about the topological interpretation of deep learning.&lt;/p&gt;&#xA;&lt;p&gt;Also, &lt;a href=&quot;https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; toy interactive code may help you.&lt;/p&gt;&#xA;&lt;p&gt;In the context of machine learning, the concept of a &lt;em&gt;manifold&lt;/em&gt; can be illustrated as in the following figure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bpS1y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bpS1y.png&quot; alt=&quot;swiss&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the first part, data are 3-dimensional. However, we can find a transformation to get the second image, which shows that data is actually artificially high dimensional, i.e. it is a 2-dimensional manifold in 3-D space. This example may be thought of as a classification problem, and colors may represent classes, and we can find a trivial representation of the data for classification.&lt;/p&gt;&#xA;&lt;p&gt;Another example could be following figures from the blog I mentioned. In here, this classification problem cannot be solved without having a layer that has 3 or more hidden units, regardless of depth. So the notion of high dimensional transformation is important.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NfkSD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NfkSD.jpg&quot; alt=&quot;1st image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can map this data to 3-D, and find a plane to separate them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tyRcb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tyRcb.jpg&quot; alt=&quot;2nd iamge&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41615" LastEditorUserId="2444" LastEditDate="2021-01-10T15:08:04.167" LastActivityDate="2021-01-10T15:08:04.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25680" PostTypeId="2" ParentId="25671" CreationDate="2021-01-10T13:47:08.890" Score="2" Body="&lt;p&gt;If you take a look at the Wikipedia page related to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;normal distribution&lt;/a&gt;, you will see the definition of the Gaussian density&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{\displaystyle f(x)={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {1}{2}}\left({\frac {x-\mu }{\sigma }}\right)^{2}}} \label{1}\tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and you will see that the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in your formula corresponds to the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in equation \ref{1}.&lt;/p&gt;&#xA;&lt;p&gt;I've seen this notation in the context of computer vision and image processing, where the Gaussian kernel is used to blur images.&lt;/p&gt;&#xA;&lt;p&gt;So, as pointed out by someone in a comment, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; should indeed be the point where you evaluate the density.&lt;/p&gt;&#xA;&lt;p&gt;Maybe the confusing part is that all parameters are treated equally in terms of their purpose, while &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; are clearly the parameters that define the specific density, so they are not the inputs to the specific density.&lt;/p&gt;&#xA;&lt;p&gt;After having read the relevant section of the paper, I now understand why you're confused. The author refers to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; as the &lt;em&gt;output&lt;/em&gt; (not yet sure why: maybe it's the output of another unit that feeds this Gaussian unit?), but I think that this explanation still applies. The output of the Gaussian density &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is not &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, but the density that corresponds to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. In fact, in appendix &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; of the paper, the author says that &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is the &lt;a href=&quot;https://planetmath.org/supportoffunction&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;support&lt;/strong&gt;&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is an element of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-10T14:59:34.373" LastActivityDate="2021-01-10T14:59:34.373" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="25681" PostTypeId="2" ParentId="25670" CreationDate="2021-01-10T15:24:58.410" Score="1" Body="&lt;p&gt;The application of 1 kernel (aka filter) to an input (with a 2d convolution) is a matrix (a 2d array), which is often known as a &lt;strong&gt;feature map&lt;/strong&gt; (aka activation map). The application of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; kernels to the same input is a 3d array (sometimes called &lt;em&gt;tensor&lt;/em&gt;, though this may not be exactly correct, or 3d &lt;em&gt;volume&lt;/em&gt;) with depth &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, i.e. you have &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; concatenated feature maps. This 3d array is the input to the next convolutional layer, which needs to have 3d kernels of depth &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;: this is the main &lt;em&gt;requirement&lt;/em&gt; in the case of 2d convolutions (in the case of 3d convolutions, the kernels can have a different depth than the input 3d volume, but you can ignore this for now!).&lt;/p&gt;&#xA;&lt;p&gt;Another thing that you need to take care of is the &lt;strong&gt;padding&lt;/strong&gt; that you can add around this 3d volume. This, along with the &lt;strong&gt;stride&lt;/strong&gt; (i.e. the step with which you move the kernel), determines the height and width of the output of the convolutional layer.&lt;/p&gt;&#xA;&lt;p&gt;So, to recapitulate, what you have to do is set the depth of the kernels in the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$l+1$&lt;/span&gt; to be equal to the number of kernels that you use in the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;. So, let's say that the number of kernels is a hyper-parameter (note that this is usually the case!), and that the user sets the number of kernels &lt;span class=&quot;math-container&quot;&gt;$k=8$&lt;/span&gt; in the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, and, for simplicity, assume that all kernels are &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt;, then, in layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, you will perform &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; convolutions, one for each kernel. These &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; convolutions produce a 3d array of dimensions &lt;span class=&quot;math-container&quot;&gt;$w \times h \times 8$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; depend on the dimensions of the input to layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, the stride, and padding). Consequently, in the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$l+1$&lt;/span&gt;, the kernels need to be &lt;span class=&quot;math-container&quot;&gt;$3 \times 3 \times 8$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If it's the pooling layer that produces the 3d array to be passed to the next convolutional layer, you should still set the depth of the kernels in the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$l+1$&lt;/span&gt; to the number of kernels in the convolutional layer &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; (provided that, in the pooling layer, you only redimension the width and height of the 3d array, which is usually the case).&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-10T17:51:42.047" LastActivityDate="2021-01-10T17:51:42.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25682" PostTypeId="2" ParentId="25676" CreationDate="2021-01-10T16:39:52.593" Score="1" Body="&lt;p&gt;There are many ways to solve this problem.  One way is to apply stemming or lemmatization to reduce your words. Using NLTK's Porter stemmer for example on healthy, healthier, healthiest, not healthy, more healthy, and zero healthy gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;healthi , healthier , healthiest , not healthi , more healthi , zero healthi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This can help make word comparisons easier.&lt;/p&gt;&#xA;&lt;p&gt;Sentiment analysis on the phrases will provide positive, neutral, and negative scores.  There are a lot of algorithms for doing this but a common one is Valence Aware Dictionary and sEntiment Reasoner (VADER).  Here is a recent article with code using NLTK and the VADER lexicon:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://predictivehacks.com/how-to-run-sentiment-analysis-in-python-using-vader/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How To Run Sentiment Analysis In Python Using VADER&lt;/a&gt; by George Pipis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The following article also does sentiment analysis using NLTK and includes stemming and lemmatization.  Instead of VADER they use a Naive Bayes classifier on a labeled data set of tweets: &lt;a href=&quot;https://www.digitalocean.com/community/tutorials/how-to-perform-sentiment-analysis-in-python-3-using-the-natural-language-toolkit-nltk&quot; rel=&quot;nofollow noreferrer&quot;&gt;How To Perform Sentiment Analysis in Python 3 Using the Natural Language Toolkit (NLTK)&lt;/a&gt; by Saumik Daityari.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastEditorUserId="5763" LastEditDate="2021-01-10T16:55:45.037" LastActivityDate="2021-01-10T16:55:45.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25683" PostTypeId="1" CreationDate="2021-01-10T16:43:17.383" Score="0" ViewCount="32" Body="&lt;p&gt;I am attempting to use time-series classification algorithms for fraud detection applications. I have came across several works in the literature that propose novel techniques for multivariate time-series classification, however, most of these approaches treat each feature as an individual signal.&lt;/p&gt;&#xA;&lt;p&gt;Now, my processing of my data transforms a transactions dataset into a tensor; 1 dimension where each observation is an account, 1 dimension where each element is a transaction and 1 dimension for the transaction attributes. The transactions dataset has a large number of features, many of which are one-hot encoded categorical variables. Therefore, I am not really sure that multivariate time-series classification algorithms such as CNN or LSTM will work in this case, since it will treat every one-hot encoded feature as a signal on its own.&lt;/p&gt;&#xA;&lt;p&gt;What would be an alternative approach in this case? Would applying PCA on the data to capture the most significant features help instead of the ordinary features?&lt;/p&gt;&#xA;" OwnerUserId="43698" LastEditorUserId="43698" LastEditDate="2021-01-10T17:02:00.853" LastActivityDate="2021-01-10T17:02:00.853" Title="Multivariate time-series classification with many variables" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;tensorflow&gt;&lt;time-series&gt;" AnswerCount="0" CommentCount="3" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25684" PostTypeId="2" ParentId="25674" CreationDate="2021-01-10T17:09:27.140" Score="1" Body="&lt;p&gt;Assuming that the image is blank everywhere but where the face is drawn...&lt;/p&gt;&#xA;&lt;p&gt;The first step is to scale the image to the mask.  That doesn't require a detailed explanation here as it is too trivial a problem.&lt;/p&gt;&#xA;&lt;p&gt;Second, rotate the image by 90 degrees three times and save each one.&lt;/p&gt;&#xA;&lt;p&gt;Third, for the four versions of the image (the original and three rotations), do the image addition.  The only one with information will be the one that is rotationally aligned with the mask.&lt;/p&gt;&#xA;" OwnerUserId="5763" LastActivityDate="2021-01-10T17:09:27.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25685" PostTypeId="2" ParentId="25678" CreationDate="2021-01-10T17:44:31.027" Score="3" Body="&lt;p&gt;The overestimation comes from the random initialisation of your Q-value estimates. Obviously these will not be perfect (if they were then we wouldn't need to learn the true Q-values!). In many value based reinforcement learning methods such as SARSA or Q-learning the algorithms involve a &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; operator in the construction of the target policy. The most obvious case is, as you mentioned, Q-learning. The learning update is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a) = Q(s, a) + \alpha \left[r(s, a) + \gamma \max_a Q(s', a) - Q(s, a) \right] \;.$$&lt;/span&gt;&#xA;The Q-function for the state-action tuple we are considering is shifted towards the &lt;em&gt;max&lt;/em&gt; Q-function at the next state where the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; is taken with respect to the actions.&lt;/p&gt;&#xA;&lt;p&gt;Now, as mentioned our initial estimates of the Q-values are initialised randomly. This naturally leads to incorrect values. The consequence of this is that when we calculate &lt;span class=&quot;math-container&quot;&gt;$\max_aQ(s', a)$&lt;/span&gt; we could be choosing values that are grossly &lt;em&gt;overestimated&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As Q-learning (in the tabular case) is guaranteed to converge (under some mild assumptions) so the main consequence of the overestimation bias is that is severely slows down convergence. This of course can be overcome with Double Q-learning.&lt;/p&gt;&#xA;&lt;p&gt;The answer above is for the tabular Q-Learning case. The idea is the same for the the Deep Q-Learning, except note that Deep Q-learning has no convergence guarantees (when using a NN as the function approximator) and so the overestimation bias is more of a problem as it can mean the parameters of the network get stuck in sub-optimal values.&lt;/p&gt;&#xA;&lt;p&gt;As someone asked in the comments about always initialising the values to be very low numbers, this would not really work.&lt;/p&gt;&#xA;&lt;p&gt;Consider the following MDP taken from Sutton and Barto:&#xA;We start in state A, from which we can either go right with reward 0 leading to a terminal state or go left with reward 0 to state B. From state B we can take, say, 100 different actions, all of which lead to a terminal state and have reward drawn from a Normal distribution with mean -0.1 and variance 1.&lt;/p&gt;&#xA;&lt;p&gt;Now, clearly the optimal action from state A is to go right. However, when we go left and take an action in state B there is an (almost) 0.5 probability of getting a reward bigger than 0. Now, recall that the Q-value is shifted towards &lt;span class=&quot;math-container&quot;&gt;$r(s, a) + \max_a Q(s', a)$&lt;/span&gt;; because of the stochastic rewards when transitioning out of state B and the fact that we will likely see a positive reward the &lt;span class=&quot;math-container&quot;&gt;$\max_a Q(s', a)$&lt;/span&gt; will be positive.&lt;/p&gt;&#xA;&lt;p&gt;This means that when we take the left action the Q-value (Q(A, left)) is shifted towards a positive value, meaning that when we are in state A the value of moving left will be higher than moving right (which will gradually be being shifted towards the true value of 0) and so when following the &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;-greedy policy the greedy action will be to go left when in fact this is sub-optimal.&lt;/p&gt;&#xA;&lt;p&gt;Now, of course, we know that the true Q-values will eventually converge but if we do have, say, 100 actions then you can probably see that the time it will take for the Q-values to converge to the true value will potentially be a long time as we would have to keep choosing all the overestimated values until we had convergence.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-01-10T21:13:42.720" LastActivityDate="2021-01-10T21:13:42.720" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="25686" PostTypeId="5" CreationDate="2021-01-10T18:01:28.763" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-10T18:01:28.763" LastActivityDate="2021-01-10T18:01:28.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25687" PostTypeId="4" CreationDate="2021-01-10T18:01:28.763" Score="0" Body="For questions related to the (tabular) version of the double Q-learning algorithm, which was introduced in &quot;Double Q-learning&quot; (2010, NeurIPS) by Hado Hasselt." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-10T18:01:28.763" LastActivityDate="2021-01-10T18:01:28.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25688" PostTypeId="1" CreationDate="2021-01-10T22:10:20.987" Score="0" ViewCount="54" Body="&lt;p&gt;I am working on an RL problem where the time when the agent obtains the reward for taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is stochastic. In fact, there is no immediate reward for taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, and, for example, the agent may obtain the reward in time step &lt;span class=&quot;math-container&quot;&gt;$t+k$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$k&amp;gt;1$&lt;/span&gt;). I was wondering if this kind of reward function is categorized as a non-Markovian reward function, and which RL method works better (to approximate/find the optimum policy) in this environment?&lt;/p&gt;&#xA;&lt;p&gt;PS: It differs from the sparse reward problems. In my problem, there is a non-zero reward associated with every action taken. However, the agent does not receive any reward immediately. In fact, once the agent takes an action, like &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, at a time step like &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, the agent does not have any control over when he/she will receive the reward associated with that action. The time when the reward is received is stochastic.&lt;/p&gt;&#xA;" OwnerUserId="43709" LastEditorUserId="2444" LastEditDate="2021-01-12T10:04:09.717" LastActivityDate="2021-01-12T10:04:09.717" Title="Is my reward function non-Markovian?" Tags="&lt;reinforcement-learning&gt;&lt;terminology&gt;&lt;reference-request&gt;&lt;reward-functions&gt;&lt;markov-property&gt;" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25689" PostTypeId="5" CreationDate="2021-01-10T22:18:46.023" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-10T22:18:46.023" LastActivityDate="2021-01-10T22:18:46.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25690" PostTypeId="4" CreationDate="2021-01-10T22:18:46.023" Score="0" Body="For questions related to evolutionary computation approaches, which includes swarm intelligence and evolutionary algorithms (such as genetic algorithms), among other approaches." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-10T22:18:46.023" LastActivityDate="2021-01-10T22:18:46.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25691" PostTypeId="2" ParentId="42" CreationDate="2021-01-10T22:56:08.420" Score="0" Body="&lt;p&gt;One aspect that I'd like to add to the previous answers is the so-called &lt;em&gt;Curse of dimensionality&lt;/em&gt;. This concept refers to the problem that many algorithms have a time complexity that grows exponentially with the dimension of the data.&lt;/p&gt;&#xA;&lt;p&gt;As a simple example, let us consider a set &lt;span class=&quot;math-container&quot;&gt;$\{0,1\}^{D}$&lt;/span&gt; that has only two values per dimension. For example, &lt;span class=&quot;math-container&quot;&gt;$\{0,1\}^{2} = \{(0,0),(0,1),(1,0),(1,1)\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(0,1,0) \in \{0,1\}^{3}$&lt;/span&gt;. Now imagine that you are given a function &lt;span class=&quot;math-container&quot;&gt;$f: \{0,1\}^{\times D} \rightarrow \{TRUE, FALSE\}$&lt;/span&gt; that outputs TRUE exactly for one particular input. The goal is to determine that input.&lt;/p&gt;&#xA;&lt;p&gt;In the example, if nothing else is known about f, the best thing one can do is to try the inputs one after another. However, &lt;span class=&quot;math-container&quot;&gt;$\{0,1\}^{D}$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$2^D$&lt;/span&gt; elements. So the number of inputs one has to try out will in general be roughly &lt;span class=&quot;math-container&quot;&gt;$2^D$&lt;/span&gt; as well.&lt;/p&gt;&#xA;&lt;p&gt;However, there exist examples suffering from a curse of dimensionality that can be solved with deep learning, i.e. using neural networks with &lt;strong&gt;many&lt;/strong&gt; hidden layers.&#xA;One example of great practical importance is given by high-dimensional partial differential equations, see e.g this report:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sam.math.ethz.ch/sam_reports/reports_final/reports2017/2017-44_fp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sam.math.ethz.ch/sam_reports/reports_final/reports2017/2017-44_fp.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;or this example for heat equations:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1901.10854&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1901.10854&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also found this review on using deep learning to overcome the curse of dimensionality:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://cbmm.mit.edu/sites/default/files/publications/02_761-774_00966_Bpast.No_.66-6_28.12.18_K1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cbmm.mit.edu/sites/default/files/publications/02_761-774_00966_Bpast.No_.66-6_28.12.18_K1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43710" LastActivityDate="2021-01-10T22:56:08.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25692" PostTypeId="2" ParentId="77" CreationDate="2021-01-10T23:03:14.497" Score="0" Body="&lt;p&gt;Clojure, a dialect of Lisp (implemented for the Java Virtual Machine), was used to implement &lt;a href=&quot;https://github.com/lspector/Clojush&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clojush&lt;/a&gt;, a &lt;em&gt;PushGP&lt;/em&gt; system, i.e. a genetic programming (which is a sub-class of evolutionary algorithms) system based on the use of the &lt;a href=&quot;https://faculty.hampshire.edu/lspector/push.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Push&lt;/em&gt; programming language&lt;/a&gt;, which is a stack-based programming language. The lead developer of Clojush, Lee Spector, and other people still do research on these topics. So, yes, Lisp is still being used in artificial intelligence!&lt;/p&gt;&#xA;&lt;p&gt;However, it's also true that Python and C/C++ (to implement the low-level stuff) are probably the two most used programming languages in AI nowadays, especially for deep learning.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2021-01-10T23:03:14.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25693" PostTypeId="5" CreationDate="2021-01-11T00:33:21.603" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T00:33:21.603" LastActivityDate="2021-01-11T00:33:21.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25694" PostTypeId="4" CreationDate="2021-01-11T00:33:21.603" Score="0" Body="For questions about stemming, i.e. the process of reducing inflected words to their word stem, base, or root form. For example, after stemming, &quot;cats&quot; becomes &quot;cat&quot; (the base form of related words)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T00:33:21.603" LastActivityDate="2021-01-11T00:33:21.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25695" PostTypeId="5" CreationDate="2021-01-11T00:35:52.813" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T00:35:52.813" LastActivityDate="2021-01-11T00:35:52.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25696" PostTypeId="4" CreationDate="2021-01-11T00:35:52.813" Score="0" Body="For questions related to lemmatization, i.e. the process of grouping together the inflected forms of a word so they can be analysed as a single item. For example, after lemmatization, the words &quot;walk&quot;, &quot;walked&quot;, &quot;walks&quot; or &quot;walking&quot; could all be grouped under the stem (or base word) &quot;walk&quot;." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T00:35:52.813" LastActivityDate="2021-01-11T00:35:52.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25697" PostTypeId="5" CreationDate="2021-01-11T00:46:00.810" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T00:46:00.810" LastActivityDate="2021-01-11T00:46:00.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25698" PostTypeId="4" CreationDate="2021-01-11T00:46:00.810" Score="0" Body="For questions related to pooling layers in convolutional neural networks (or other related neural networks that make use of a pooling operation in some layer)." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T00:46:00.810" LastActivityDate="2021-01-11T00:46:00.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25699" PostTypeId="5" CreationDate="2021-01-11T01:02:32.993" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T01:02:32.993" LastActivityDate="2021-01-11T01:02:32.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25700" PostTypeId="4" CreationDate="2021-01-11T01:02:32.993" Score="0" Body="For questions related to the max-pooling operation, where the maximum of a set of numbers is chosen as the &quot;representative&quot; of that set of numbers, which is commonly used in convolutional neural networks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T01:02:32.993" LastActivityDate="2021-01-11T01:02:32.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25702" PostTypeId="1" CreationDate="2021-01-11T03:25:34.043" Score="2" ViewCount="27" Body="&lt;p&gt;I'm trying to use the &lt;a href=&quot;https://repositorio.unesp.br/bitstream/handle/11449/177329/2-s2.0-84926656505.pdf?sequence=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Binary Flower Pollination Algorithm (BFPA)&lt;/a&gt; for feature selection. In the BFPA, the sigmoid function is used to compute a binary vector that represents whether a feature is selected or not. Here are the relevant equations from the paper (page 4).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;S\left(x_{i}^{j}(t)\right)=\frac{1}{1+e^{-x_{i}^{j}(t)}} \tag{4}\label{4}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;x_{i}^{j}(t)=\left\{\begin{array}{ll}&#xA;1 &amp;amp; \text { if } S\left(x_{i}^{j}(t)\right)&amp;gt;\sigma \\&#xA;0 &amp;amp; \text { otherwise }&#xA;\end{array}\right.&#xA;\tag{5}\label{5}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In my case, I noticed that my algorithm sometimes returns a zero vector (i.e. all elements are zeros, such as &lt;span class=&quot;math-container&quot;&gt;$[0,0,0,0,0,0,0,0,0]$&lt;/span&gt;), which means that no feature is selected (a feature would be selected when it is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;), which makes the fitness function returns an error.&lt;/p&gt;&#xA;&lt;p&gt;Is it correct that sigmoid returns result like that?&lt;/p&gt;&#xA;" OwnerUserId="43279" LastEditorUserId="2444" LastEditDate="2021-01-12T00:32:34.623" LastActivityDate="2021-01-12T00:32:34.623" Title="In the Binary Flower Pollination Algorithm (using the sigmoid function), is it possible that no feature is selected?" Tags="&lt;machine-learning&gt;&lt;feature-selection&gt;&lt;swarm-intelligence&gt;&lt;sigmoid&gt;&lt;binary-flower-pollination-algorithm&gt;" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="25703" PostTypeId="1" CreationDate="2021-01-11T04:18:25.340" Score="0" ViewCount="19" Body="&lt;p&gt;Title says it all really.&lt;/p&gt;&#xA;&lt;p&gt;I want to train a network to take images of diagrams and produce a standard textual definition of them.&lt;/p&gt;&#xA;&lt;p&gt;What ML architecture is best for this?&lt;/p&gt;&#xA;" OwnerUserId="43714" LastActivityDate="2021-01-11T04:18:25.340" Title="What's the best machine learning algorithm / neural network architecture to use for a task that maps between images and textual descriptions of them?" Tags="&lt;neural-networks&gt;&lt;image-recognition&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25704" PostTypeId="5" CreationDate="2021-01-11T05:59:58.017" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2021-01-11T05:59:58.017" LastActivityDate="2021-01-11T05:59:58.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25705" PostTypeId="4" CreationDate="2021-01-11T05:59:58.017" Score="0" Body="For questions related to the gradient, a way of packing together all the partial derivative information of a function" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2021-01-18T01:00:00.987" LastActivityDate="2021-01-18T01:00:00.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25706" PostTypeId="5" CreationDate="2021-01-11T06:03:15.563" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://paperswithcode.com/method/r1-regularization&quot; rel=&quot;nofollow noreferrer&quot;&gt;R&lt;span class=&quot;math-container&quot;&gt;$_{1}$&lt;/span&gt; Regularization&lt;/a&gt; is a regularization technique and gradient penalty for training generative adversarial networks. It penalizes the discriminator from deviating from the Nash Equilibrium via penalizing the gradient on real data alone: when the generator distribution produces the true data distribution and the discriminator is equal to 0 on the data manifold, the gradient penalty ensures that the discriminator cannot create a non-zero gradient orthogonal to the data manifold without suffering a loss in the GAN game.&lt;/p&gt;&#xA;&lt;p&gt;This leads to the following regularization term:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$R_{1}\left(\psi\right) = \frac{\gamma}{2}E_{p_{D}\left(x\right)}\left[||\nabla{D_{\psi}\left(x\right)}||^{2}\right]$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2021-01-31T15:33:35.217" LastActivityDate="2021-01-31T15:33:35.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25707" PostTypeId="4" CreationDate="2021-01-11T06:03:15.563" Score="0" Body="For questions related to R$_{1}$ Regularization,  a regularization technique and gradient penalty for training generative adversarial networks" OwnerUserId="30725" LastEditorUserId="30725" LastEditDate="2021-01-19T16:18:51.430" LastActivityDate="2021-01-19T16:18:51.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25708" PostTypeId="1" CreationDate="2021-01-11T06:11:07.213" Score="0" ViewCount="16" Body="&lt;p&gt;I read the &lt;a href=&quot;https://arxiv.org/abs/1909.11942&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlBERT&lt;/a&gt; and saw that its architecture used &amp;quot;Parameter Sharing&amp;quot; among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters and also because it was observed that most BERT layers learned almost the same thing. My question is that if we consider that the first layer in AlBERT approximates a function (say &lt;em&gt;F(X)&lt;/em&gt;), then can it be said that overall AlBERT just applies that function repeatedly on the input to generate the output?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt; ŷ = F&amp;lt;sub&amp;gt;1&amp;lt;/sub&amp;gt;(F&amp;lt;sub&amp;gt;2&amp;lt;/sub&amp;gt;(F&amp;lt;sub&amp;gt;2&amp;lt;/sub&amp;gt;(...F&amp;lt;sub&amp;gt;*n*&amp;lt;/sub&amp;gt;(x))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;where &lt;em&gt;x&lt;/em&gt; is the tokenized input, &lt;em&gt;n&lt;/em&gt; is the number of layers in the encoders of AlBERT and ŷ is the output from the final layer?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here I assume that both the multi-head attention and feed-forward network together approximate F(x).&lt;/p&gt;&#xA;" OwnerUserId="43715" LastEditorUserId="43715" LastEditDate="2021-01-18T18:25:45.807" LastActivityDate="2021-01-18T18:25:45.807" Title="Is parameter sharing in AlBERT akin to repeated application of same function on input?" Tags="&lt;deep-learning&gt;&lt;math&gt;&lt;algorithm&gt;&lt;bert&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25711" PostTypeId="1" CreationDate="2021-01-11T13:18:36.690" Score="0" ViewCount="24" Body="&lt;p&gt;I am relatively new to the field of AI. I have a problem that I would like to solve with AI, but I don't know which buzzwords I should use to search for solutions.&lt;/p&gt;&#xA;&lt;p&gt;I have a &lt;em&gt;plantar pressure&lt;/em&gt; scan, like the one below (which was &lt;a href=&quot;https://en.wikipedia.org/wiki/Pedobarography#/media/File:Example_foot_pressure.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;taken from Wikipedia&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ddpfl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ddpfl.png&quot; alt=&quot;https://en.wikipedia.org/wiki/Pedobarography#/media/File:Example_foot_pressure.png&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to define and mark the relevant points in this image. For example, I want to put a point in the middle of each toe with the corresponding label, which toe it is, or also the middle of the heel. So, theoretically, I want to train an AI that can do that at some point.&lt;/p&gt;&#xA;&lt;p&gt;The question is whether such a thing is currently possible with AI? I can find many examples of object recognition on the web (e.g. recognition of a cat, etc.). A few examples I find for the recognition of keypoints by humans (e.g. arms, shoulders, etc.).&lt;/p&gt;&#xA;&lt;p&gt;In which direction do I have to search? I know that the presented example is not trivial, but I am mainly interested in whether it is theoretically possible. And if so, how something like this can be implemented.&lt;/p&gt;&#xA;" OwnerUserId="43726" LastEditorUserId="2444" LastEditDate="2021-01-12T00:36:48.730" LastActivityDate="2021-01-12T00:36:48.730" Title="Which AI technique should I use for (key)point detection (in an image of a plantar pressure)?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;reference-request&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25712" PostTypeId="1" CreationDate="2021-01-11T13:31:54.257" Score="4" ViewCount="157" Body="&lt;p&gt;There are proofs for the universal approximation theorem with just 1 hidden layer.&lt;/p&gt;&#xA;&lt;p&gt;The proof goes like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a &amp;quot;bump&amp;quot; function using 2 neurons.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create (infinitely) many of these step functions with different angles in order to create a tower-like shape.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Decrease the step/radius to a very small value in order to approximate a cylinder. &lt;strong&gt;This is what I'm not convinced of&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using these cylinders one can approximate any shape. (At this point it's basically just a packing problem like &lt;a href=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/e/ee/15_circles_in_a_square.svg/1024px-15_circles_in_a_square.svg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;http://www.youtube.com/watch?v=lkha188L4Gs&amp;amp;t=42m0s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video, minute 42&lt;/a&gt;, the lecturer says&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the limit that's going to be a perfect cylinder. If the cylinder is small enough. It's gonna be a perfect cylinder. Right ? I have control over the radius.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here are the slides.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HPvXo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HPvXo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathematik.uni-wuerzburg.de/fileadmin/10040900/2019/Seminar__Artificial_Neural_Network__24_9__.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a pdf version&lt;/a&gt; from another university, so you do not have to watch the video.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why am I not convinced?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I created a program to plot this, and even if I decrease the radius by orders of magnitude it still has the same shape.&lt;/p&gt;&#xA;&lt;p&gt;Let's start with a simple tower of radius 0.1:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mT75p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mT75p.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now let's decrease the radius to 0.01:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rq73M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rq73M.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, you might think that it gets close to a cylinder, but it just looks like it is approximating a perfect cylinder, because of the zoomed out effect.&lt;/p&gt;&#xA;&lt;p&gt;Let's zoom in:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Oxjtq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Oxjtq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's decrease the radius to 0.0000001.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T7YcQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T7YcQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Still not a perfect cylinder. In fact, the &amp;quot;quality&amp;quot; of the cylinder is the same.&lt;/p&gt;&#xA;&lt;p&gt;Python code to reproduce (requires NumPy and matplotlib): &lt;a href=&quot;https://pastebin.com/CMXFXvNj&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/CMXFXvNj&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q1 Is it true that we can get a perfect cylinder solely by decreasing the radius of the tower to 0 ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q2 If this true, why is there no difference when I plot it with different radii(0.1, vs 1e-7) ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Both towers have the same shape&lt;/p&gt;&#xA;&lt;p&gt;Clarification: What do I mean with: same shape ? Let's say we calculate the volume of an actual cylinder(Vc) with the same raius and height as our tower and divide it by the volume of the tower(Vt) .&lt;/p&gt;&#xA;&lt;p&gt;Vc = Volume Cylinder&lt;/p&gt;&#xA;&lt;p&gt;Vt = Volume Tower&lt;/p&gt;&#xA;&lt;p&gt;ratio(r) = Vc/Vt&lt;/p&gt;&#xA;&lt;p&gt;What this documents/lectures claim that is the ratio of these 2 volumes depends on the radius but in my view it's just constant.&lt;/p&gt;&#xA;&lt;p&gt;So what they are saying is that: lim r -&amp;gt; 0 for ratio(r) = 1&#xA;But my experiments show that: lim r -&amp;gt; 0 for ratio(r) = const and don't depend on the radius at all.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q3 Preface&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;An objection i got multiple times once from &lt;a href=&quot;https://ai.stackexchange.com/users/9947/duttaa&quot;&gt;Dutta&lt;/a&gt; and once from &lt;a href=&quot;https://ai.stackexchange.com/users/1794/d-w&quot;&gt;D.W&lt;/a&gt; is that just decreasing the radious and plotting it isn't mathematical rigorous.&lt;/p&gt;&#xA;&lt;p&gt;So let's assume in the limit of r=0 it's really a perfect cylinder.&lt;/p&gt;&#xA;&lt;p&gt;One possible explanation for this would be that the limit is a special case and one can't approximate towards it&lt;/p&gt;&#xA;&lt;p&gt;But if that is true this would imply that there is no use for it since it's impossible to have a radius of exactly zero. It would only be useful if we could get gradually closer to a perfect cylinder by decreasing the radius.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q3 So why should we even care about this then ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Further Clarifications&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The original universal approximation theorem proof for single hidden layer neural networks was done by G. Cybenko. Then I think people tried to make some visual explations for it. &lt;strong&gt;I am NOT questioning the paper !&lt;/strong&gt; But i am questioning the visual explanation given in the linked lecutre/pdf (made by other people)&lt;/p&gt;&#xA;" OwnerUserId="43729" LastEditorUserId="43729" LastEditDate="2021-01-21T12:42:40.667" LastActivityDate="2021-01-21T12:42:40.667" Title="Is it really possible to create the &quot;Perfect Cylinder&quot; used in Universal Approximation Theorem for 1-hidden layer Neural Network?" Tags="&lt;neural-networks&gt;&lt;feedforward-neural-network&gt;&lt;universal-approximation-theorems&gt;" AnswerCount="1" CommentCount="13" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25713" PostTypeId="1" CreationDate="2021-01-11T15:10:11.880" Score="2" ViewCount="34" Body="&lt;p&gt;I am just getting into medical image segmentation and have been able to understand the state-of-the-art architectures, like Double UNet, UNet++, and Multiresunet.&lt;/p&gt;&#xA;&lt;p&gt;What I haven't understood yet: Why are these approaches better for medical segmentation than, for example, HRNet-OCR, which currently tops the rankings of the Cityscapes dataset, and vice versa?&lt;/p&gt;&#xA;" OwnerUserId="43632" LastEditorUserId="2444" LastEditDate="2021-01-12T00:18:29.020" LastActivityDate="2021-01-12T00:18:29.020" Title="How and why do state-of-the-art models in medical segmentation differ from general segmentation models?" Tags="&lt;comparison&gt;&lt;models&gt;&lt;image-segmentation&gt;&lt;u-net&gt;&lt;state-of-the-art&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25714" PostTypeId="1" CreationDate="2021-01-11T16:03:28.437" Score="1" ViewCount="49" Body="&lt;p&gt;I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space dimension.&lt;/p&gt;&#xA;&lt;p&gt;My VAE structure is as follows : Input : 2048 -&amp;gt; 1024 -&amp;gt; 512 -&amp;gt; Latent space dimension -&amp;gt; 512 -&amp;gt; 1024 -&amp;gt; Output : 2048&lt;/p&gt;&#xA;&lt;p&gt;Here is what I get in terms of relative error when the latent space dimension goes from 2 to 100 :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hnG3X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hnG3X.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Everything works as expected at the beginning, but the error starts rising up at around 50 and I have no idea why. With a large latent space dimension, the output is orders of magnitude smaller than the input, which explains the relative error of value 1.&lt;/p&gt;&#xA;&lt;p&gt;Here is the same figure when I run the exact same experiment but with a normal autoencoder this time.&lt;a href=&quot;https://i.stack.imgur.com/390qh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/390qh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This time the results are consistent.&lt;/p&gt;&#xA;&lt;p&gt;What's wrong with my VAE ?&lt;/p&gt;&#xA;" OwnerUserId="43733" LastActivityDate="2021-01-11T16:03:28.437" Title="VAE giving near zero output when latent space dimension is large" Tags="&lt;autoencoders&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25715" PostTypeId="1" CreationDate="2021-01-11T16:30:58.600" Score="2" ViewCount="44" Body="&lt;p&gt;I am new to few-shot learning, and I wanted to get a hands-on understanding of it, using &lt;a href=&quot;https://keras.io/examples/vision/reptile/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reptile&lt;/a&gt; algorithm, applied to my custom dataset.&lt;/p&gt;&#xA;&lt;p&gt;My custom dataset has 30 categories, with 5 images per category, so this would be a 30 way 5 shot.&lt;/p&gt;&#xA;&lt;p&gt;Given a new image, I wish to be able to classify it into one of 30 categories. I changed &lt;code&gt;train_shots = 5&lt;/code&gt;, &lt;code&gt;classes = 30&lt;/code&gt; in the &lt;a href=&quot;https://keras.io/examples/vision/reptile/&quot; rel=&quot;nofollow noreferrer&quot;&gt;linked example&lt;/a&gt;, and got the training output as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;batch 0: train=0.050000 test=0.050000&#xA;batch 1: train=0.050000 test=0.050000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should the custom dataset be used as a validation set, with mini-ImageNet as a training dataset, so that the knowledge is transferred? Or can I use only a custom dataset with only &lt;span class=&quot;math-container&quot;&gt;$30*5=150$&lt;/span&gt; images for training?&lt;/p&gt;&#xA;" OwnerUserId="43623" LastEditorUserId="2444" LastEditDate="2021-01-12T00:16:10.930" LastActivityDate="2021-01-12T00:16:10.930" Title="In few-shot classification, should I use my custom dataset as the validation dataset and mini-ImageNet as the training dataset?" Tags="&lt;classification&gt;&lt;datasets&gt;&lt;meta-learning&gt;&lt;few-shot-learning&gt;&lt;reptile-algorithm&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25716" PostTypeId="1" CreationDate="2021-01-11T18:24:59.617" Score="0" ViewCount="18" Body="&lt;p&gt;So I'm trying to make my first neural network and have just finished my back propagation functions. I got the algebra from brilliant and thought I'd understood it, but my bug proves otherwise. The bug itself is that after 500 iterations, both of my output neurons give a value of 1 despite my target values being 0 or 1. None of my training data gives both neurons outputting 1 at the same time, yet it happens. I'm sorry for posting a lot of code here, but I can't think of a better way of showing off the potential problem areas.&lt;/p&gt;&#xA;&lt;p&gt;This calculates the error term for each neuron:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Calculates error for each neuron in this layer, which we know isn't the output layer&#xA;// Error Term = derivation of sigmoid * sum of (weight to next layer * error term in next layer)&#xA;prevLayer.neurons.at(neuron).neuronData.error = prevLayer.neurons.at(neuron).CalculateSumWeightsError(this-&amp;gt;neurons) *&#xA;    prevLayer.neurons.at(neuron).GetDerivedValue(theta);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the contents of the loop that calculates the delta weights for each connection:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;            // Calculates the connections new weight&#xA;            // delta weight = &#xA;            // -learning rate * &#xA;            // Derivative of activation * &#xA;            // input * &#xA;            // sum of (weights * error term of next layer)&#xA;&#xA;        float newWeight = alpha * prevLayer.neurons.at(neuron).GetDerivedValue(theta);&#xA;        newWeight *= prevLayer.neurons.at(neuron).neuronData.value;&#xA;        newWeight *= this-&amp;gt;neurons.at(errorNeuron).neuronData.sumErrorWeights;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This finds the total error and individual error terms in the output layer. It's part of a loop that iterates through all of them.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Calculate individual error terms&#xA;// Error Terms = target - output&#xA;neuralLayers.back().neurons.at(output).neuronData.error = targets[output] - neuralLayers.back().neurons.at(output).GetActivatedValue(theta);&#xA;// Calculate Total Error for the output layer&#xA;// Total Error = sum of 1/2*(target - output)^2&#xA;errorTotal += 0.5f * FMath::Pow((targets[output] - neuralLayers.back().neurons.at(output).GetActivatedValue(theta)), 2.f);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Like I said, this is my first attempt at a neural network and thought I'd got it right this time round. Any help or resources would be very much appreciated as I genuinely don't know what's gone wrong.&lt;/p&gt;&#xA;" OwnerUserId="43735" LastActivityDate="2021-01-11T18:24:59.617" Title="Neural Network trains towards 1 despite target" Tags="&lt;neural-networks&gt;&lt;training&gt;&lt;backpropagation&gt;&lt;feedforward-neural-network&gt;&lt;c++&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25717" PostTypeId="1" CreationDate="2021-01-11T18:32:32.917" Score="2" ViewCount="51" Body="&lt;p&gt;In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by performing multiple forward passes with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, then, for example, average these predictions.&lt;/p&gt;&#xA;&lt;p&gt;Let's suppose I want to fine-tune a pre-trained model and get MCD uncertainty estimations, how should I add dropout layers?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;on the fully-connected layers;&lt;/li&gt;&#xA;&lt;li&gt;after every convolutional layer.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I've read some papers and implementations where one applies dropout at fully-connected layers only, using a pre-trained model. However, when using a custom model, usually one adds dropout after every convolutional layer. This &lt;a href=&quot;https://arxiv.org/abs/1908.00792&quot; rel=&quot;nofollow noreferrer&quot;&gt;work&lt;/a&gt; builds two configurations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;dropout on the fully-connected layers;&lt;/li&gt;&#xA;&lt;li&gt;dropout after resnet blocks.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first configuration performs better, but I'm unsure if this is an actual uncertainty estimation from resnet. The results show that there is a correlation between high uncertainty predictions and incorrect predictions. So, would this be a good way of estimating uncertainty? My shot is &amp;quot;yes&amp;quot;, because even though there are no nodes being sampled from the backbone, the sampling in the fully-connected layer forces a smooth variation in the backbone, generating a low-variance ensemble. But, I'm quite a beginner on MCD so, any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="42100" LastEditorUserId="42100" LastEditDate="2021-01-15T19:52:08.130" LastActivityDate="2021-01-15T19:52:08.130" Title="How can I use Monte Carlo Dropout in a pre-trained CNN model?" Tags="&lt;deep-learning&gt;&lt;monte-carlo-methods&gt;&lt;dropout&gt;&lt;uncertainty-quantification&gt;&lt;mc-dropout&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25718" PostTypeId="5" CreationDate="2021-01-11T20:15:12.233" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T20:15:12.233" LastActivityDate="2021-01-11T20:15:12.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25719" PostTypeId="4" CreationDate="2021-01-11T20:15:12.233" Score="0" Body="For questions related to drug design using or that involve artificial intelligence techniques." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-11T20:15:12.233" LastActivityDate="2021-01-11T20:15:12.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25720" PostTypeId="5" CreationDate="2021-01-12T00:11:03.047" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-12T00:11:03.047" LastActivityDate="2021-01-12T00:11:03.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25721" PostTypeId="4" CreationDate="2021-01-12T00:11:03.047" Score="0" Body="For questions related to the Reptile algorithm, which was proposed in &quot;On First-Order Meta-Learning Algorithms&quot; (2018) by Alex Nichol et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-12T00:11:03.047" LastActivityDate="2021-01-12T00:11:03.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25722" PostTypeId="5" CreationDate="2021-01-12T00:28:41.723" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-12T00:28:41.723" LastActivityDate="2021-01-12T00:28:41.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25723" PostTypeId="4" CreationDate="2021-01-12T00:28:41.723" Score="0" Body="For questions related to the Binary Flower Pollination Algorithm (BFPA), which was proposed in &quot;Binary flower pollination algorithm and its application to feature selection&quot; (2015) by Douglas Rodrigues et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-12T00:28:41.723" LastActivityDate="2021-01-12T00:28:41.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25725" PostTypeId="1" CreationDate="2021-01-12T05:51:57.647" Score="3" ViewCount="43" Body="&lt;p&gt;I am writing about the role of machine learning scientists in developing a solution. Is there a term for the humans who do learning?&#xA;Can we call a &amp;quot;team of machine learning scientists with their computers working on some ML problem&amp;quot;   an intelligent agent? Is &amp;quot;cognizer&amp;quot; the right term? I know that &amp;quot;learner&amp;quot; is reserved for an ML algorithm. I just want a shorter term for their role in cognition, learning.&lt;/p&gt;&#xA;" OwnerUserId="43741" LastEditorUserId="2444" LastEditDate="2021-01-12T09:46:32.153" LastActivityDate="2021-01-12T11:09:10.777" Title="Is a team of ML scientists an &quot;intelligent agent&quot;?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;intelligent-agent&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25726" PostTypeId="1" AcceptedAnswerId="25848" CreationDate="2021-01-12T08:15:08.450" Score="1" ViewCount="46" Body="&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/pdf/1412.5068.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this research paper&lt;/a&gt;, we have the following claim&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the smoothness assumption that underlies many kernel methods such as Support Vector Machines (SVMs) does not hold for deep neural networks trained through backpropagation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Does &lt;em&gt;smoothness&lt;/em&gt; here refer to no sharp rise/fall in gradients?&lt;/p&gt;&#xA;" OwnerUserId="35616" LastEditorUserId="2444" LastEditDate="2021-01-12T18:35:22.417" LastActivityDate="2021-01-19T16:12:40.593" Title="What is the smoothness assumption in SVMs?" Tags="&lt;machine-learning&gt;&lt;terminology&gt;&lt;papers&gt;&lt;support-vector-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25727" PostTypeId="2" ParentId="25725" CreationDate="2021-01-12T08:50:21.100" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is there a term for the humans who do [machine] learning?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Typically you will see &amp;quot;AI researchers&amp;quot; for people studying machine intelligence in general, or &amp;quot;data scientists&amp;quot; for people working with statistics or studying specific solutions in machine learning. Both those terms are used quite flexibly, and generally understood to be scientists/engineers that work on machine intelligence problems. There are also other specialisms that might apply, such as &amp;quot;computer vision researcher&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;If you are writing only about researchers who are focused mostly on learning systems - such as new types of learning model, or the best way to train a model for some group of tasks - then &amp;quot;machine learning researchers&amp;quot; or &amp;quot;ML researchers&amp;quot; would be fine. If they are specific people, you may want to ask them directly what title they prefer, in case you accidentally misrepresent their role and work.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can we call a &amp;quot;team of machine learning scientists with their computers working on some ML problem&amp;quot; an intelligent agent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. That might be confused with something that the team is working on or creating. An &amp;quot;intelligent agent&amp;quot; can be the product of specific AI research. For instance, it is reasonable to consider AlphaZero to be an &amp;quot;intelligent agent&amp;quot; (not everyone would call it that, it depends on context, but not relevant to your question).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is &amp;quot;cognizer&amp;quot; a right term?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. That is an obscure term meaning something like &amp;quot;a being that perceives or knows&amp;quot;. No-one will naturally associate that word with a team of researchers and their computers.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I just want a shorter term for their role in cognition, learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think the most usual way to do this is to describe the team, and name them. For example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;The MIT computer science researchers are using the X1000 system to investigate  learning of abstract spaces. &lt;em&gt;The MIT team&lt;/em&gt; said yesterday . . .&amp;quot;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;Professor Hilary Smith is working with a team of graduates on the Learny McLearnface framework. &lt;em&gt;Smith's team&lt;/em&gt; have discovered that . . .&amp;quot;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is the normal way that you see a research team introduced and referred in science journalism in e.g. New Scientist or Scientific American.&lt;/p&gt;&#xA;&lt;p&gt;If you need to describe some abstract approach about a team working in general, then you can use a similar approach, naming your teams according to how you need to differentiate between them.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-01-12T11:09:10.777" LastActivityDate="2021-01-12T11:09:10.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25728" PostTypeId="5" CreationDate="2021-01-12T10:49:42.530" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-12T10:49:42.530" LastActivityDate="2021-01-12T10:49:42.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25729" PostTypeId="4" CreationDate="2021-01-12T10:49:42.530" Score="0" Body="For questions related to the concept of time-step, for instance, in Markov decision processes or reinforcement learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-12T10:49:42.530" LastActivityDate="2021-01-12T10:49:42.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25730" PostTypeId="1" CreationDate="2021-01-12T12:22:18.520" Score="1" ViewCount="30" Body="&lt;p&gt;I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks.&lt;/p&gt;&#xA;&lt;p&gt;What if for the weight initialization for pre-training, module A is initialized from some checkpoints, while B's is from random? Can I still claim the process as pre-training? Or the modules in the model must all be initialized from random, and so we can call it pre-training?&lt;/p&gt;&#xA;&lt;p&gt;If parts of the module's weights are 'contaminated' by checkpoints, it is only can be called fine-tune?&lt;/p&gt;&#xA;" OwnerUserId="43349" LastEditorUserId="2444" LastEditDate="2021-01-12T19:57:03.593" LastActivityDate="2021-01-12T19:57:03.593" Title="What is the definition of pre-training?" Tags="&lt;deep-learning&gt;&lt;definitions&gt;&lt;transfer-learning&gt;&lt;fine-tuning&gt;&lt;pre-training&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25732" PostTypeId="1" CreationDate="2021-01-12T14:54:59.177" Score="2" ViewCount="34" Body="&lt;p&gt;I am currently working on implementing a Multi-Agent System for Smart Grids.&lt;/p&gt;&#xA;&lt;p&gt;There's a lot of literature for that and some things confuse me. I have read that there is FIPA, which aimed to create a unified Agent Communication Language. So multiple Agents are talking to each other and FIPA specifies how the messages should be sent and processed. However, it is pretty old.&lt;/p&gt;&#xA;&lt;p&gt;In newer papers, where Multi-Agent Reinforcement Learning Algorithms are proposed, FIPA or generally any ACL isn't mentioned. I believe that is because in MARL, communication is done by observing the states of the other agents, rather than communicating explicitly. Also in MARL, the decision making is not based on negotiation like in FIPA but with the learned policy.&lt;/p&gt;&#xA;&lt;p&gt;I am now super confused if I got it right.&#xA;Is FIPA still a thing I should worry about when I design my Multi-Agent System?&#xA;Is there any other thing to handle communication in MARL other than sharing states?&lt;/p&gt;&#xA;&lt;p&gt;Any help would be really appreciated, thank you very much :)&lt;/p&gt;&#xA;" OwnerUserId="43752" LastActivityDate="2021-01-12T14:54:59.177" Title="What place do Agent Communications Language have in Multi-Agent Systems nowadays?" Tags="&lt;reinforcement-learning&gt;&lt;intelligent-agent&gt;&lt;multi-agent-systems&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25733" PostTypeId="1" CreationDate="2021-01-12T15:17:21.553" Score="1" ViewCount="28" Body="&lt;p&gt;My variational autoencoder seems to work for MNIST, but fails on slightly &amp;quot;harder&amp;quot; data.&lt;br /&gt;&#xA;By &amp;quot;fails&amp;quot; I mean there are at least two apparent problems:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Very poor reconstruction, for example sample reconstructions from the last epoch on validation set&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QaKQK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QaKQK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sLeIf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sLeIf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/EfD51.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EfD51.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;without any regularization at all.&lt;br /&gt;&#xA;The last reported losses from console are &lt;code&gt;val_loss=9.57e-5, train_loss=9.83e-5&lt;/code&gt; which I thought would imply exact reconstructions.&lt;/li&gt;&#xA;&lt;li&gt;validation loss is low (which does not seem to reflect the reconstruction), and &lt;strong&gt;always lower than training loss&lt;/strong&gt; which is very suspicious.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sLQs2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sLQs2.png&quot; alt=&quot;losses&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YIM1n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YIM1n.png&quot; alt=&quot;losses2&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For MNIST everything looks fine (with less layers!).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9dYGv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9dYGv.png&quot; alt=&quot;mnist recon&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I will give as much nformation as I can, since I am not sure what I should provide to help anyone help me.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Firstly, here is the &lt;strong&gt;full code&lt;/strong&gt;&lt;br /&gt;&#xA;You will notice loss calculation and logging is very simple and straight forward and I can't seem to find what's wrong.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;from torch import nn&#xA;import torch.nn.functional as F&#xA;from typing import List, Optional, Any&#xA;from pytorch_lightning.core.lightning import LightningModule&#xA;from Testing.Research.config.ConfigProvider import ConfigProvider&#xA;from pytorch_lightning import Trainer, seed_everything&#xA;from torch import optim&#xA;import os&#xA;from pytorch_lightning.loggers import TensorBoardLogger&#xA;# import tfmpl&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib&#xA;from Testing.Research.data_modules.MyDataModule import MyDataModule&#xA;from Testing.Research.data_modules.MNISTDataModule import MNISTDataModule&#xA;from Testing.Research.data_modules.CaseDataModule import CaseDataModule&#xA;import torchvision&#xA;from Testing.Research.config.paths import tb_logs_folder&#xA;from Testing.Research.config.paths import vae_checkpoints_path&#xA;from pytorch_lightning.callbacks.model_checkpoint import ModelCheckpoint&#xA;&#xA;&#xA;class VAEFC(LightningModule):&#xA;    # see https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73&#xA;    # for possible upgrades, see https://arxiv.org/pdf/1602.02282.pdf&#xA;    # https://stats.stackexchange.com/questions/332179/how-to-weight-kld-loss-vs-reconstruction-loss-in-variational&#xA;    # -auto-encoder&#xA;    def __init__(self, encoder_layer_sizes: List, decoder_layer_sizes: List, config):&#xA;        super(VAEFC, self).__init__()&#xA;        self._config = config&#xA;        self.logger: Optional[TensorBoardLogger] = None&#xA;        self.save_hyperparameters()&#xA;&#xA;        assert len(encoder_layer_sizes) &amp;gt;= 3, &amp;quot;must have at least 3 layers (2 hidden)&amp;quot;&#xA;        # encoder layers&#xA;        self._encoder_layers = nn.ModuleList()&#xA;        for i in range(1, len(encoder_layer_sizes) - 1):&#xA;            enc_layer = nn.Linear(encoder_layer_sizes[i - 1], encoder_layer_sizes[i])&#xA;            self._encoder_layers.append(enc_layer)&#xA;&#xA;        # predict mean and covariance vectors&#xA;        self._mean_layer = nn.Linear(encoder_layer_sizes[&#xA;                                         len(encoder_layer_sizes) - 2],&#xA;                                     encoder_layer_sizes[len(encoder_layer_sizes) - 1])&#xA;        self._logvar_layer = nn.Linear(encoder_layer_sizes[&#xA;                                           len(encoder_layer_sizes) - 2],&#xA;                                       encoder_layer_sizes[len(encoder_layer_sizes) - 1])&#xA;&#xA;        # decoder layers&#xA;        self._decoder_layers = nn.ModuleList()&#xA;        for i in range(1, len(decoder_layer_sizes)):&#xA;            dec_layer = nn.Linear(decoder_layer_sizes[i - 1], decoder_layer_sizes[i])&#xA;            self._decoder_layers.append(dec_layer)&#xA;&#xA;        self._recon_function = nn.MSELoss(reduction='mean')&#xA;        self._last_val_batch = {}&#xA;&#xA;    def _encode(self, x):&#xA;        for i in range(len(self._encoder_layers)):&#xA;            layer = self._encoder_layers[i]&#xA;            x = F.relu(layer(x))&#xA;&#xA;        mean_output = self._mean_layer(x)&#xA;        logvar_output = self._logvar_layer(x)&#xA;        return mean_output, logvar_output&#xA;&#xA;    def _reparametrize(self, mu, logvar):&#xA;        if not self.training:&#xA;            return mu&#xA;        std = logvar.mul(0.5).exp_()&#xA;        if std.is_cuda:&#xA;            eps = torch.FloatTensor(std.size()).cuda().normal_()&#xA;        else:&#xA;            eps = torch.FloatTensor(std.size()).normal_()&#xA;        reparameterized = eps.mul(std).add_(mu)&#xA;        return reparameterized&#xA;&#xA;    def _decode(self, z):&#xA;        for i in range(len(self._decoder_layers) - 1):&#xA;            layer = self._decoder_layers[i]&#xA;            z = F.relu((layer(z)))&#xA;&#xA;        decoded = self._decoder_layers[len(self._decoder_layers) - 1](z)&#xA;        # decoded = F.sigmoid(self._decoder_layers[len(self._decoder_layers)-1](z))&#xA;        return decoded&#xA;&#xA;    def _loss_function(self, recon_x, x, mu, logvar, reconstruction_function):&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        recon_x: generating images&#xA;        x: origin images&#xA;        mu: latent mean&#xA;        logvar: latent log variance&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        binary_cross_entropy = reconstruction_function(recon_x, x)  # mse loss TODO see if mse or cross entropy&#xA;        # loss = 0.5 * sum(1 + log(sigma^2) - mu^2 - sigma^2)&#xA;        kld_element = mu.pow(2).add_(logvar.exp()).mul_(-1).add_(1).add_(logvar)&#xA;        kld = torch.sum(kld_element).mul_(-0.5)&#xA;        # KL divergence Kullback–Leibler divergence, regularization term for VAE&#xA;        # It is a measure of how different two probability distributions are different from each other.&#xA;        # We are trying to force the distributions closer while keeping the reconstruction loss low.&#xA;        # see https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73&#xA;&#xA;        # read on weighting the regularization term here:&#xA;        # https://stats.stackexchange.com/questions/332179/how-to-weight-kld-loss-vs-reconstruction-loss-in-variational&#xA;        # -auto-encoder&#xA;        return binary_cross_entropy + kld * self._config.regularization_factor&#xA;&#xA;    def _parse_batch_by_dataset(self, batch, batch_index):&#xA;        if self._config.dataset == &amp;quot;toy&amp;quot;:&#xA;            (orig_batch, noisy_batch), label_batch = batch&#xA;            # TODO put in the noise here and not in the dataset?&#xA;        elif self._config.dataset == &amp;quot;mnist&amp;quot;:&#xA;            orig_batch, label_batch = batch&#xA;            orig_batch = orig_batch.reshape(-1, 28 * 28)&#xA;            noisy_batch = orig_batch&#xA;        elif self._config.dataset == &amp;quot;case&amp;quot;:&#xA;            orig_batch, label_batch = batch&#xA;&#xA;            orig_batch = orig_batch.float().reshape(&#xA;                    -1,&#xA;                    len(self._config.case.feature_list) * self._config.case.frames_per_pd_sample&#xA;            )&#xA;            noisy_batch = orig_batch&#xA;        else:&#xA;            raise ValueError(&amp;quot;invalid dataset&amp;quot;)&#xA;        noisy_batch = noisy_batch.view(noisy_batch.size(0), -1)&#xA;&#xA;        return orig_batch, noisy_batch, label_batch&#xA;&#xA;    def training_step(self, batch, batch_idx):&#xA;        orig_batch, noisy_batch, label_batch = self._parse_batch_by_dataset(batch, batch_idx)&#xA;&#xA;        recon_batch, mu, logvar = self.forward(noisy_batch)&#xA;&#xA;        loss = self._loss_function(&#xA;                recon_batch,&#xA;                orig_batch, mu, logvar,&#xA;                reconstruction_function=self._recon_function&#xA;        )&#xA;        # self.logger.experiment.add_scalars(&amp;quot;losses&amp;quot;, {&amp;quot;train_loss&amp;quot;: loss})&#xA;        tb = self.logger.experiment&#xA;        tb.add_scalars(&amp;quot;losses&amp;quot;, {&amp;quot;train_loss&amp;quot;: loss}, global_step=self.current_epoch)&#xA;        # self.logger.experiment.add_scalar(&amp;quot;train_loss&amp;quot;, loss, self.current_epoch)&#xA;        if batch_idx == len(self.train_dataloader()) - 2:&#xA;            # https://pytorch.org/docs/stable/_modules/torch/utils/tensorboard/writer.html#SummaryWriter.add_embedding&#xA;            # noisy_batch = noisy_batch.detach()&#xA;            # recon_batch = recon_batch.detach()&#xA;            # last_batch_plt = matplotlib.figure.Figure()  # read https://github.com/wookayin/tensorflow-plot&#xA;            # ax = last_batch_plt.add_subplot(1, 1, 1)&#xA;            # ax.scatter(orig_batch[:, 0], orig_batch[:, 1], label=&amp;quot;original&amp;quot;)&#xA;            # ax.scatter(noisy_batch[:, 0], noisy_batch[:, 1], label=&amp;quot;noisy&amp;quot;)&#xA;            # ax.scatter(recon_batch[:, 0], recon_batch[:, 1], label=&amp;quot;reconstructed&amp;quot;)&#xA;            # ax.legend(loc=&amp;quot;upper left&amp;quot;)&#xA;            # self.logger.experiment.add_figure(f&amp;quot;original last batch, epoch {self.current_epoch}&amp;quot;, last_batch_plt)&#xA;            # tb.add_embedding(orig_batch, global_step=self.current_epoch, metadata=label_batch)&#xA;            pass&#xA;        self.logger.experiment.flush()&#xA;        self.log(&amp;quot;train_loss&amp;quot;, loss, prog_bar=True, on_step=False, on_epoch=True)&#xA;        return loss&#xA;&#xA;    def _plot_batches(self, orig_batch, noisy_batch, label_batch, batch_idx, recon_batch, mu, logvar):&#xA;        # orig_batch_view = orig_batch.reshape(-1, self._config.case.frames_per_pd_sample,&#xA;        # len(self._config.case.feature_list))&#xA;        #&#xA;        # plt.figure()&#xA;        # plt.plot(orig_batch_view[11, :, 0].detach().cpu().numpy(), label=&amp;quot;feature 0&amp;quot;)&#xA;        # plt.legend(loc=&amp;quot;upper left&amp;quot;)&#xA;        # plt.show()&#xA;&#xA;        tb = self.logger.experiment&#xA;        if self._config.dataset == &amp;quot;mnist&amp;quot;:&#xA;            orig_batch -= orig_batch.min()&#xA;            orig_batch /= orig_batch.max()&#xA;            recon_batch -= recon_batch.min()&#xA;            recon_batch /= recon_batch.max()&#xA;&#xA;            orig_grid = torchvision.utils.make_grid(orig_batch.view(-1, 1, 28, 28))&#xA;            val_recon_grid = torchvision.utils.make_grid(recon_batch.view(-1, 1, 28, 28))&#xA;&#xA;            tb.add_image(&amp;quot;original_val&amp;quot;, orig_grid, global_step=self.current_epoch)&#xA;            tb.add_image(&amp;quot;reconstruction_val&amp;quot;, val_recon_grid, global_step=self.current_epoch)&#xA;&#xA;            label_img = orig_batch.view(-1, 1, 28, 28)&#xA;            pass&#xA;        elif self._config.dataset == &amp;quot;case&amp;quot;:&#xA;            orig_batch_view = orig_batch.reshape(-1, self._config.case.frames_per_pd_sample,&#xA;                                                 len(self._config.case.feature_list)).transpose(1, 2)&#xA;            recon_batch_view = recon_batch.reshape(-1, self._config.case.frames_per_pd_sample,&#xA;                                                   len(self._config.case.feature_list)).transpose(1, 2)&#xA;&#xA;            # plt.figure()&#xA;            # plt.plot(orig_batch_view[11, 0, :].detach().cpu().numpy())&#xA;            # plt.show()&#xA;            # pass&#xA;&#xA;            n_samples = orig_batch_view.shape[0]&#xA;            n_plots = min(n_samples, 4)&#xA;            first_sample_idx = 0&#xA;&#xA;            # TODO either plotting or data problem&#xA;            fig, axs = plt.subplots(n_plots, 1)&#xA;            for sample_idx in range(n_plots):&#xA;                for feature_idx, (orig_feature, recon_feature) in enumerate(&#xA;                        zip(orig_batch_view[sample_idx + first_sample_idx, :, :],&#xA;                            recon_batch_view[sample_idx + first_sample_idx, :, :])):&#xA;                    i = feature_idx&#xA;                    if i &amp;gt; 0: continue  # or scale issues don't allow informative plotting&#xA;&#xA;                    # plt.figure()&#xA;                    # plt.plot(orig_feature.detach().cpu().numpy(), label=f'orig{i}, sample{sample_idx}')&#xA;                    # plt.legend(loc='upper left')&#xA;                    # pass&#xA;&#xA;                    axs[sample_idx].plot(orig_feature.detach().cpu().numpy(), label=f'orig{i}, sample{sample_idx}')&#xA;                    axs[sample_idx].plot(recon_feature.detach().cpu().numpy(), label=f'recon{i}, sample{sample_idx}')&#xA;                    # sample{sample_idx}')&#xA;                    axs[sample_idx].legend(loc='upper left')&#xA;                pass&#xA;            # plt.show()&#xA;&#xA;            tb.add_figure(&amp;quot;recon_vs_orig&amp;quot;, fig, global_step=self.current_epoch, close=True)&#xA;&#xA;    def validation_step(self, batch, batch_idx):&#xA;        orig_batch, noisy_batch, label_batch = self._parse_batch_by_dataset(batch, batch_idx)&#xA;&#xA;        recon_batch, mu, logvar = self.forward(noisy_batch)&#xA;&#xA;        loss = self._loss_function(&#xA;                recon_batch,&#xA;                orig_batch, mu, logvar,&#xA;                reconstruction_function=self._recon_function&#xA;        )&#xA;&#xA;        tb = self.logger.experiment&#xA;        # can probably speed up training by waiting for epoch end for data copy from gpu&#xA;        # see https://sagivtech.com/2017/09/19/optimizing-pytorch-training-code/&#xA;        tb.add_scalars(&amp;quot;losses&amp;quot;, {&amp;quot;val_loss&amp;quot;: loss}, global_step=self.current_epoch)&#xA;&#xA;        label_img = None&#xA;        if len(orig_batch) &amp;gt; 2:&#xA;            self._last_val_batch = {&#xA;                &amp;quot;orig_batch&amp;quot;: orig_batch,&#xA;                &amp;quot;noisy_batch&amp;quot;: noisy_batch,&#xA;                &amp;quot;label_batch&amp;quot;: label_batch,&#xA;                &amp;quot;batch_idx&amp;quot;: batch_idx,&#xA;                &amp;quot;recon_batch&amp;quot;: recon_batch,&#xA;                &amp;quot;mu&amp;quot;: mu,&#xA;                &amp;quot;logvar&amp;quot;: logvar&#xA;            }&#xA;        # self._plot_batches(orig_batch, noisy_batch, label_batch, batch_idx, recon_batch, mu, logvar)&#xA;&#xA;        outputs = {&amp;quot;val_loss&amp;quot;:  loss, &amp;quot;recon_batch&amp;quot;: recon_batch, &amp;quot;label_batch&amp;quot;: label_batch,&#xA;                   &amp;quot;label_img&amp;quot;: label_img}&#xA;        self.log(&amp;quot;val_loss&amp;quot;, loss, prog_bar=True, on_step=False, on_epoch=True)&#xA;        return outputs&#xA;&#xA;    def validation_epoch_end(self, outputs: List[Any]) -&amp;gt; None:&#xA;        first_batch_dict = outputs[-1]&#xA;&#xA;        self._plot_batches(&#xA;                self._last_val_batch[&amp;quot;orig_batch&amp;quot;],&#xA;                self._last_val_batch[&amp;quot;noisy_batch&amp;quot;],&#xA;                self._last_val_batch[&amp;quot;label_batch&amp;quot;],&#xA;                self._last_val_batch[&amp;quot;batch_idx&amp;quot;],&#xA;                self._last_val_batch[&amp;quot;recon_batch&amp;quot;],&#xA;                self._last_val_batch[&amp;quot;mu&amp;quot;],&#xA;                self._last_val_batch[&amp;quot;logvar&amp;quot;]&#xA;        )&#xA;        self.log(name=&amp;quot;VAEFC_val_loss_epoch_end&amp;quot;, value={&amp;quot;val_loss&amp;quot;: first_batch_dict[&amp;quot;val_loss&amp;quot;]})&#xA;&#xA;    def test_step(self, batch, batch_idx):&#xA;        orig_batch, noisy_batch, label_batch = self._parse_batch_by_dataset(batch, batch_idx)&#xA;&#xA;        recon_batch, mu, logvar = self.forward(noisy_batch)&#xA;&#xA;        loss = self._loss_function(&#xA;                recon_batch,&#xA;                orig_batch, mu, logvar,&#xA;                reconstruction_function=self._recon_function&#xA;        )&#xA;&#xA;        tb = self.logger.experiment&#xA;        tb.add_scalars(&amp;quot;losses&amp;quot;, {&amp;quot;test_loss&amp;quot;: loss}, global_step=self.global_step)&#xA;&#xA;        return {&amp;quot;test_loss&amp;quot;: loss, &amp;quot;mus&amp;quot;: mu, &amp;quot;labels&amp;quot;: label_batch, &amp;quot;images&amp;quot;: orig_batch}&#xA;&#xA;    def test_epoch_end(self, outputs: List):&#xA;        tb = self.logger.experiment&#xA;&#xA;        avg_loss = torch.stack([x['test_loss'] for x in outputs]).mean()&#xA;        self.log(name=&amp;quot;test_epoch_end&amp;quot;, value={&amp;quot;test_loss_avg&amp;quot;: avg_loss})&#xA;&#xA;        if self._config.dataset == &amp;quot;mnist&amp;quot;:&#xA;            tb.add_embedding(&#xA;                    mat=torch.cat([o[&amp;quot;mus&amp;quot;] for o in outputs]),&#xA;                    metadata=torch.cat([o[&amp;quot;labels&amp;quot;] for o in outputs]).detach().cpu().numpy(),&#xA;                    label_img=torch.cat([o[&amp;quot;images&amp;quot;] for o in outputs]).view(-1, 1, 28, 28),&#xA;                    global_step=self.global_step,&#xA;            )&#xA;&#xA;    def configure_optimizers(self):&#xA;        optimizer = optim.Adam(self.parameters(), lr=self._config.learning_rate)&#xA;        return optimizer&#xA;&#xA;    def forward(self, x):&#xA;        mu, logvar = self._encode(x)&#xA;        z = self._reparametrize(mu, logvar)&#xA;        decoded = self._decode(z)&#xA;        return decoded, mu, logvar&#xA;&#xA;&#xA;def train_vae(config, datamodule, latent_dim, dec_layer_sizes, enc_layer_sizes):&#xA;    model = VAEFC(config=config, encoder_layer_sizes=enc_layer_sizes, decoder_layer_sizes=dec_layer_sizes)&#xA;&#xA;    logger = TensorBoardLogger(save_dir=tb_logs_folder, name='VAEFC', default_hp_metric=False)&#xA;    logger.hparams = config&#xA;&#xA;    checkpoint_callback = ModelCheckpoint(dirpath=vae_checkpoints_path)&#xA;    trainer = Trainer(deterministic=config.is_deterministic,&#xA;                      # auto_lr_find=config.auto_lr_find,&#xA;                      # log_gpu_memory='all',&#xA;                      # min_epochs=99999,&#xA;                      max_epochs=config.num_epochs,&#xA;                      default_root_dir=vae_checkpoints_path,&#xA;                      logger=logger,&#xA;                      callbacks=[checkpoint_callback],&#xA;                      gpus=1&#xA;                      )&#xA;    # trainer.tune(model)&#xA;    trainer.fit(model, datamodule=datamodule)&#xA;    best_model_path = checkpoint_callback.best_model_path&#xA;    print(&amp;quot;done training vae with lightning&amp;quot;)&#xA;    print(f&amp;quot;best model path = {best_model_path}&amp;quot;)&#xA;    return trainer&#xA;&#xA;&#xA;def run_trained_vae(trainer):&#xA;    # https://pytorch-lightning.readthedocs.io/en/latest/test_set.html&#xA;    # (1) load the best checkpoint automatically (lightning tracks this for you)&#xA;    trainer.test()&#xA;&#xA;    # (2) don't load a checkpoint, instead use the model with the latest weights&#xA;    # trainer.test(ckpt_path=None)&#xA;&#xA;    # (3) test using a specific checkpoint&#xA;    # trainer.test(ckpt_path='/path/to/my_checkpoint.ckpt')&#xA;&#xA;    # (4) test with an explicit model (will use this model and not load a checkpoint)&#xA;    # trainer.test(model)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Parameters&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am getting very similar results for any combination of parameters I am (manually) using. Maybe I didn't try something.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;num_epochs: 40&#xA;batch_size: 32&#xA;learning_rate: 0.0001&#xA;auto_lr_find: False&#xA;&#xA;noise_factor: 0.1&#xA;regularization_factor: 0.0&#xA;&#xA;train_size: 0.8&#xA;val_size: 0.1&#xA;num_workers: 1&#xA;&#xA;dataset: &amp;quot;case&amp;quot; # toy, mnnist, case&#xA;mnist:&#xA;  enc_layer_sizes: [784, 512,]&#xA;  dec_layer_sizes: [512, 784]&#xA;  latent_dim: 25&#xA;  n_classes: 10&#xA;  classifier_layers: [20, 10]&#xA;toy:&#xA;  enc_layer_sizes: [2, 200, 200, 200]&#xA;  dec_layer_sizes: [200, 200, 200, 2]&#xA;  latent_dim: 8&#xA;  centers_radius: 4.0&#xA;  n_clusters: 10&#xA;  cluster_size: 5000&#xA;case:&#xA;  #enc_layer_sizes: [ 1800, 600, 300, 100 ]&#xA;  #dec_layer_sizes: [ 100, 300, 600, 1800 ]&#xA;  #frames_per_pd_sample: 600&#xA;&#xA;  enc_layer_sizes: [ 10, 600, 300, 300 ]&#xA;  dec_layer_sizes: [ 600, 300, 300, 10 ]&#xA;  frames_per_pd_sample: 10&#xA;&#xA;  latent_dim: 300&#xA;  n_classes: 10&#xA;  classifier_layers: [ 20, 10 ] # unused right now.&#xA;&#xA;  feature_list:&#xA;    #- V_0_0 # 0, X&#xA;    #- V_0_1 # 0, Y&#xA;    #- V_0_2 # 0, Z&#xA;    - pads_0&#xA;  enc_kernel_sizes: [] # for conv&#xA;  end_strides: []&#xA;  dec_kernel_sizes: []&#xA;  dec_strides: []&#xA;&#xA;is_deterministic: False&#xA;&#xA;real_data_pd_dir: &amp;quot;D:/pressure_pd&amp;quot;&#xA;case_dir: &amp;quot;real_case_20_min&amp;quot;&#xA;case_file: &amp;quot;pressure_data_0.pkl&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Data&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For Mnist everything works fine.&lt;/p&gt;&#xA;&lt;p&gt;When changing to my specific data, results are as above.&lt;/p&gt;&#xA;&lt;p&gt;The data is a &lt;strong&gt;time series&lt;/strong&gt;, of several features. To dumb this down even more, I am feeding just a single feature, sliced to equal-length chunks, and fed into the input layer as a vector.&lt;br /&gt;&#xA;The fact that the data is a time series could maybe help modeling in the future, but for now I want to just refer to it as chunks of data, which I believe I am doing.&lt;/p&gt;&#xA;&lt;p&gt;code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from torch.utils.data import Dataset&#xA;import matplotlib.pyplot as plt&#xA;import torch&#xA;from Testing.Research.config.ConfigProvider import ConfigProvider&#xA;import os&#xA;import pickle&#xA;import pandas as pd&#xA;from typing import Tuple&#xA;import numpy as np&#xA;&#xA;&#xA;class CaseDataset(Dataset):&#xA;    def __init__(self, path):&#xA;        super(CaseDataset, self).__init__()&#xA;        self._path = path&#xA;&#xA;        self._config = ConfigProvider.get_config()&#xA;        self.frames_per_pd_sample = self._config.case.frames_per_pd_sample&#xA;        self._load_case_from_pkl()&#xA;        self.__len = len(self._full) // self.frames_per_pd_sample  # discard last non full batch&#xA;&#xA;    def _load_case_from_pkl(self):&#xA;        assert os.path.isfile(self._path)&#xA;        with open(self._path, &amp;quot;rb&amp;quot;) as f:&#xA;            p = pickle.load(f)&#xA;&#xA;        self._full: pd.DataFrame = p[&amp;quot;full&amp;quot;]&#xA;        self._subsampled: pd.DataFrame = p[&amp;quot;subsampled&amp;quot;]&#xA;        self._misc: pd.DataFrame = p[&amp;quot;misc&amp;quot;]&#xA;&#xA;        feature_list = self._config.case.feature_list&#xA;        self._features_df = self._full[feature_list].copy()&#xA;&#xA;        # normalize from -1 to 1&#xA;        features_to_normalize = self._features_df.columns&#xA;        self._features_df[features_to_normalize] = \&#xA;            self._features_df[features_to_normalize].apply(lambda x: (((x - x.min()) / (x.max() - x.min())) * 2) - 1)&#xA;&#xA;        pass&#xA;&#xA;    def __len__(self):&#xA;        # number of samples in the dataset&#xA;        return self.__len&#xA;&#xA;    def __getitem__(self, index: int) -&amp;gt; Tuple[np.array, np.array]:&#xA;        data_item = self._features_df.iloc[index * self.frames_per_pd_sample: (index + 1) * self.frames_per_pd_sample, :].values&#xA;        label = 0.0&#xA;        # plt.figure()&#xA;        # plt.plot(data_item[:, 0], label=&amp;quot;feature 0&amp;quot;)&#xA;        # plt.legend(loc=&amp;quot;upper left&amp;quot;)&#xA;        # plt.show()&#xA;        return data_item, label&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The amount of time-steps per batch does not seem to affect convergence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Train test val split&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;is done like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;from pytorch_lightning import LightningDataModule&#xA;import torchvision.datasets as datasets&#xA;from torchvision.transforms import transforms&#xA;import torch&#xA;from torch.utils.data import DataLoader&#xA;from torch.utils.data import Subset&#xA;from Testing.Research.config.paths import mnist_data_download_folder&#xA;from Testing.Research.datasets.real_cases.CaseDataset import CaseDataset&#xA;from typing import Optional&#xA;&#xA;&#xA;class CaseDataModule(LightningDataModule):&#xA;    def __init__(self, config, path):&#xA;        super().__init__()&#xA;        self._config = config&#xA;        self._path = path&#xA;&#xA;        self._train_dataset: Optional[Subset] = None&#xA;        self._val_dataset: Optional[Subset] = None&#xA;        self._test_dataset: Optional[Subset] = None&#xA;&#xA;    def prepare_data(self):&#xA;        pass&#xA;&#xA;    def setup(self, stage):&#xA;        # transform&#xA;        transform = transforms.Compose([transforms.ToTensor()])&#xA;        full_dataset = CaseDataset(self._path)&#xA;&#xA;        train_size = int(self._config.train_size * len(full_dataset))&#xA;        val_size = int(self._config.val_size * len(full_dataset))&#xA;        test_size = len(full_dataset) - train_size - val_size&#xA;        train, val, test = torch.utils.data.random_split(full_dataset, [train_size, val_size, test_size])&#xA;&#xA;        # assign to use in dataloaders&#xA;        self._full_dataset = full_dataset&#xA;        self._train_dataset = train&#xA;        self._val_dataset = val&#xA;        self._test_dataset = test&#xA;&#xA;    def train_dataloader(self):&#xA;        return DataLoader(self._train_dataset, batch_size=self._config.batch_size, num_workers=self._config.num_workers)&#xA;&#xA;    def val_dataloader(self):&#xA;        return DataLoader(self._val_dataset, batch_size=self._config.batch_size, num_workers=self._config.num_workers)&#xA;&#xA;    def test_dataloader(self):&#xA;        return DataLoader(self._test_dataset, batch_size=self._config.batch_size, num_workers=self._config.num_workers)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I believe having the validation loss consistently lower than the train loss shows something is very wrong here, but I can't put my finger on what, or come up with how to verify this.&lt;/li&gt;&#xA;&lt;li&gt;How can I just make the model auto-encode the data correctly? Basicaly, I would want it to learn the identity function, and for the loss to reflect that.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;The loss does not seem to reflect the reconstruction. I think this is probably the most fundamental issue&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;My thoughts&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Try a convolutional net instead of FC? maybe it would be able to better learn features?&lt;/li&gt;&#xA;&lt;li&gt;Out of ideas :(&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Will provide any lacking information.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="21645" LastEditDate="2021-01-12T15:32:44.963" LastActivityDate="2021-01-12T15:32:44.963" Title="variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;autoencoders&gt;&lt;pytorch&gt;&lt;variational-autoencoder&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25734" PostTypeId="2" ParentId="25561" CreationDate="2021-01-12T15:32:55.730" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://groups.google.com/g/rl-list/c/k9jNdOk6NZk/m/KQWKfyHsBgAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;I've asked this on rl-list&lt;/a&gt; and got lots of interesting references to research papers.&lt;/p&gt;&#xA;&lt;p&gt;So far the most promising one I've seen is this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mircomusolesi.org/papers/aaai20.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nicolas Anastassacos, Stephen Hailes and Mirco Musolesi. Partner Selection for the Emergence of Cooperation in Multi-Agent Systems using Reinforcement Learning. In AAAI 2020. New York City, NY, USA. February 2020.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25904" LastActivityDate="2021-01-12T15:32:55.730" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25735" PostTypeId="1" CreationDate="2021-01-12T18:41:06.487" Score="1" ViewCount="16" Body="&lt;p&gt;I am trying to reproduce the paper &lt;a href=&quot;https://arxiv.org/pdf/2005.13092.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Synthetic Petri Dish: A novel surrogate model for Rapid Architecture Search&lt;/a&gt;. In the paper, the authors try to reduce the architecture of an MLP model trained on MNIST (2 layers - 100 neurons) by initializing a &lt;strong&gt;motif network&lt;/strong&gt; from it, that is, 2 layers, 1 neuron each, and extracting the sigmoid function. I have been searching a lot, but I have not found the answer of how can someone extract an 'architectural motif' from a &lt;strong&gt;trained&lt;/strong&gt; neural network.&lt;/p&gt;&#xA;" OwnerUserId="43764" LastEditorUserId="2444" LastEditDate="2021-01-13T17:03:03.943" LastActivityDate="2021-01-13T17:03:03.943" Title="How can an &quot;architectural motif&quot; be extracted from a trained MLP?" Tags="&lt;neural-networks&gt;&lt;papers&gt;&lt;neural-architecture-search&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25736" PostTypeId="1" CreationDate="2021-01-12T18:55:35.697" Score="2" ViewCount="26" Body="&lt;p&gt;This might be moreof a question about nested function classes:&lt;/p&gt;&#xA;&lt;p&gt;For k class node classification in a graph with n nodes, and d feature vector.&#xA;I want to compare&lt;/p&gt;&#xA;&lt;p&gt;Architecture I: the GCN model of Kipf/ Welling with two graph convolutional layers:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{Y}=\operatorname{softmax}\left(\mathbf{A} \xi\left(\mathbf{A X W}_{1}\right) \mathbf{W}_{2}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times d,$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Y}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times k, \mathbf{A}$&lt;/span&gt; is a fixed &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; graph diffusion matrix, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}_{1}, \mathbf{W}_{2}$&lt;/span&gt; are learnable weight matrices of size &lt;span class=&quot;math-container&quot;&gt;$d \times d^{\prime}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d^{\prime} \times 2,$&lt;/span&gt; respectively, shared across all nodes, and &lt;span class=&quot;math-container&quot;&gt;$\xi$&lt;/span&gt; is a nonlinearity.&lt;/p&gt;&#xA;&lt;p&gt;Architecture II: a single-layer graph neural network of the form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{Y}=\operatorname{softmax}\left(\mathbf{A}^{2} \mathbf{X W}\right)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; is a learnable weight matrix of size &lt;span class=&quot;math-container&quot;&gt;$d \times 2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Now I'm wondering&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\operatorname{Can} \xi, d^{\prime}$&lt;/span&gt; be chosen in a way that both architectures have the same expressive power? (i.e. can represent the same class of functions)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\operatorname{Can} \xi, d^{\prime}$&lt;/span&gt; be chosen in a way that Architecture II is more expressive?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What would be the advantage in training complexity of Architecture II when applied to large-scale graphs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="43765" LastActivityDate="2021-01-12T18:55:35.697" Title="Graph Neural Networks: Quesitons about different GCN Architectures" Tags="&lt;convolutional-neural-networks&gt;&lt;activation-function&gt;&lt;feature-extraction&gt;&lt;graph-neural-networks&gt;&lt;feature-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25737" PostTypeId="1" CreationDate="2021-01-12T19:35:49.147" Score="0" ViewCount="13" Body="&lt;p&gt;I have been trying a lot define an objective function using linear programming for time scheduling problem.&#xA;Our problem is allocation of classes to time and all the courses that student.&lt;/p&gt;&#xA;&lt;p&gt;I have a set of students where each student will have a list of course they need to enroll.&lt;/p&gt;&#xA;&lt;p&gt;A set of courses defined with a configuration such that it may have subpart. There can be more than one subpart. classes for the courses are defined in subpart. Subpart may be lecture or lab, or lecture only and limit of students that class can have.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eOELP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eOELP.jpg&quot; alt=&quot;The image shows how a course structure is.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We have a set of available time for each class such that each time may have some penalty value assigned along with it. We need to allocate classes to time such that the penalty value is minimum.&lt;/p&gt;&#xA;&lt;p&gt;We also have  a set of constraints. Violation of each soft constraints gives some penalty value. So basically our aim is to allocate students to classes and classes to suitable time.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XFLfw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XFLfw.png&quot; alt=&quot;set of hard and soft constraints&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus objective is to minimize the penalty value ( violation of soft constraints and timing). and need to mathematically formulate them using linear programming. I am not sure if I am missing something that does not lead for formulation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;note&lt;/strong&gt; ; This problem is a problem statement inspired from international timetabling competition. &lt;a href=&quot;http://www.patatconference.org/patat2018/files/proceedings/paper27.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to the competition problem&lt;/a&gt; (I don't use room allocation in my problem).&lt;/p&gt;&#xA;" OwnerUserId="43767" LastActivityDate="2021-01-12T19:35:49.147" Title="how to define objective function using linear programming for university timetabling problem?" Tags="&lt;linear-programming&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25738" PostTypeId="1" CreationDate="2021-01-12T21:34:09.680" Score="0" ViewCount="28" Body="&lt;p&gt;Broad question to help in finding an appropriate method. So I have a given feature set of (DNA/genetic) predictors and a group of individuals which are either  cases or controls for diseaseX.  While I can build a classifier from the predictors (DNA), not everyone with diseaseX is genetic case of diseaseX, some are, but others are environmental subtypes. So basically the actual cases are split into cases that can and cannot be predicted by genetics, but there is no way of knowing which is which is in the cases. I am curious if there is a way to find the algorithm that predicts the cases that are also best predicted by the features I have.  I.e. I want to create a predictor that works for predicting the genetic subtype, when I don't know which cases are due to genetics (but know some are). Any ideas on the best algorithms to begin tackling this problem? Or areas that have had similar problems that I can learn from?&lt;/p&gt;&#xA;" OwnerUserId="43768" LastActivityDate="2021-01-12T21:34:09.680" Title="ML method for detecting which individuals are best predicted by the features" Tags="&lt;machine-learning&gt;&lt;applications&gt;&lt;research&gt;&lt;prediction&gt;&lt;clustering&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25739" PostTypeId="1" CreationDate="2021-01-12T22:29:27.787" Score="5" ViewCount="127" Body="&lt;p&gt;In general, what are the advantages of RL with actor-critic methods over actor-only (or policy-based) methods?&lt;/p&gt;&#xA;&lt;p&gt;This is not a comparison with the Q-learning series, but probably a method of learning the game with only the actor.&lt;/p&gt;&#xA;&lt;p&gt;I think it's effective to use only actors, especially for sparse rewards. Is that correct?&lt;/p&gt;&#xA;&lt;p&gt;Please, let me know if you have any specific use cases that use only actors.&lt;/p&gt;&#xA;" OwnerUserId="43246" LastEditorUserId="2444" LastEditDate="2021-01-13T10:29:21.270" LastActivityDate="2021-01-13T16:14:52.507" Title="What are the advantages of RL with actor-critic methods over actor-only methods?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;actor-critic-methods&gt;&lt;policy-based-methods&gt;&lt;continuous-tasks&gt;" AnswerCount="1" CommentCount="1" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25740" PostTypeId="2" ParentId="25739" CreationDate="2021-01-12T22:59:43.283" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In general, what are the advantages of RL with actor-critic methods over actor-only (or policy-based) methods?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;One practical benefit is that critics can use TD learning to bootstrap, allowing them to learn online on each step taken, plus learn in continuing problems. Pure actor algorithms like REINFORCE, cross-entropy method, and non-RL policy-only learners, such as genetic algorithms, require episodic problems. The smallest unit those can learn from is an entire episode. That is because without a critic providing value estimates, the only way to estimate return is to sample an actual return from the end of an episode.&lt;/p&gt;&#xA;&lt;p&gt;A TD-based critic may also have lower variance, which can aid in fast learning and stability, although this is not always a benefit. TD-based critics also have bias, which can cause instability. See &lt;a href=&quot;https://stats.stackexchange.com/questions/355820/why-do-temporal-difference-td-methods-have-lower-variance-than-monte-carlo-met&quot;&gt;Why do temporal difference (TD) methods have lower variance than Monte Carlo methods?&lt;/a&gt; on Cross Validated for more details on this.&lt;/p&gt;&#xA;&lt;p&gt;In practice, RL algorithm choice is a hyperparameter. As well as impacting difficulty of implementation, CPU and other resource costs, it can impact how well learning occurs depending on the problem being attempted. Usually, the only way you can tell a method is better for your problem is to try all the valid ones and measure their performance.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I think it's effective to use only actors, especially for sparse rewards. Is that correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The sparsity of reward is not a major factor here. The hard credit assignment problem that this brings into play means that the agent has to either assign a value, or pick an action, in states which have no direct feedback. All else being equal, both the value function and optimal action choice are hard to resolve when they depend on a large number of future policy decisions and state transitions which may vary.&lt;/p&gt;&#xA;&lt;p&gt;Which approach is best will depend on whether it is easier for a statistics-based learner to approximate the mapping from state to expected future reward, or to the action. The two functions can have different degrees of complexity.&lt;/p&gt;&#xA;&lt;p&gt;For an example where a policy function (for an actor) is simpler than a value function (for a critic or value-based method), you could consider a simple chase environment where a wolf tries to catch a rabbit on a simple continuous plane surface. The agent is the wolf, and gets a reward of +1 for catching the rabbit (I won't bother with other details, there are plenty of variations you could make).&lt;/p&gt;&#xA;&lt;p&gt;In the example environment, a simple strategy for the wolf is to turn to face the rabbit and move forwards. This is easy to map from current locations and facings of the wolf and the rabbit - for ultimate ease you can express the state as the rabbit's position and velocity in polar coordinates from the wolf's perspective. Compare that to the value function - it has to predict the time it will take to reach the rabbit given a current action choice. This is a far harder function to express based on the state, and as a result may also be harder to learn.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="1847" LastEditDate="2021-01-13T16:14:52.507" LastActivityDate="2021-01-13T16:14:52.507" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25741" PostTypeId="1" CreationDate="2021-01-13T01:18:57.427" Score="2" ViewCount="27" Body="&lt;p&gt;Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; and variance &lt;span class=&quot;math-container&quot;&gt;$\gamma^2$&lt;/span&gt; to prevent things like activations from exploding or shifting during the learning process.&lt;/p&gt;&#xA;&lt;p&gt;More specifically:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hat{x} = \displaystyle \frac{x - \mu_t}{\sqrt{\sigma_t^2 + \epsilon}}\label{1}\tag{1}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ BatchNorm_{\mu_t, \sigma_t}(x) = \gamma \hat{x} + \beta \label{2}\tag{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the layer input of the layer&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\mu_t, \sigma_t$&lt;/span&gt; is the sample mean and standard deviation at time step &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is a small constant, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; are learnable parameters so that the output is not necessarily standardized to mean &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and variance &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, but possibly to another mean and variance that may be better for the neural network.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My question is, why does BatchNorm first standardize the input &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; before applying the learnable parameters &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;? Isn't this redundant? The parameters &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; could learn to standardize the input themselves right?&lt;/p&gt;&#xA;&lt;p&gt;In fact, as training progresses, &lt;span class=&quot;math-container&quot;&gt;$\mu_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma_t$&lt;/span&gt; becomes updated to new values &lt;span class=&quot;math-container&quot;&gt;$\mu_{t+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma_{t+1}$&lt;/span&gt;, so the learned parameters at that time step, &lt;span class=&quot;math-container&quot;&gt;$\gamma_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta_t$&lt;/span&gt;, no longer apply for time step &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt; since that involves a different standardization process with a different mean and variance. So by adding this standardization step, it may even hurt the convergence of the layer during learning, since it is adding the gradient of &lt;span class=&quot;math-container&quot;&gt;$BatchNorm_{\mu_{t+1}, \sigma_{t+1}}(x)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$BatchNorm_{\mu_t, \sigma_t}(x)$&lt;/span&gt;, which are two different functions right?&lt;/p&gt;&#xA;&lt;p&gt;Why not just simply make it like this?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$BatchNorm(x) = \gamma x + \beta \label{3}\tag{3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This would simplify the calculation of the gradients, which would make learning faster to compute.&lt;/p&gt;&#xA;&lt;p&gt;BatchNorm is one of the most successful developments of deep learning, so I know my intuition on these things is wrong -- I'm just curious as to what I am missing.&lt;/p&gt;&#xA;" OwnerUserId="42699" LastEditorUserId="2444" LastEditDate="2021-01-13T10:56:26.763" LastActivityDate="2021-01-13T10:56:26.763" Title="Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;batch-normalization&gt;&lt;normalisation&gt;&lt;standardisation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25742" PostTypeId="2" ParentId="10419" CreationDate="2021-01-13T03:51:45.757" Score="0" Body="&lt;p&gt;Not sure about which function to call in GPFlow, but the principle is simple. If your &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; data was scaled:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y_{scaled} = {(Y - mean(Y)) \over {sd(Y)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So now if you have any prediction (from the scaled model) &lt;span class=&quot;math-container&quot;&gt;$\hat{Y}_{scaled}$&lt;/span&gt;, with stand error &lt;span class=&quot;math-container&quot;&gt;$\sigma_{\hat {Y}_{scaled}}$&lt;/span&gt;, then it is a normal distribution &lt;span class=&quot;math-container&quot;&gt;${\cal N}(\hat {Y}_{scaled}, \sigma^2_{\hat {Y}_{scaled}})$&lt;/span&gt;. Then you can very easily rescale it back to the original scale &lt;span class=&quot;math-container&quot;&gt;${\cal N}(\hat {Y}, \sigma^2_{\hat {Y}})$&lt;/span&gt;, where:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat {Y} = \hat {Y}_{scaled}\cdot sd(Y) + mean(Y)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma_{\hat {Y}} = \sigma_{\hat {Y}_{scaled}} \cdot sd(Y)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="30285" LastActivityDate="2021-01-13T03:51:45.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25743" PostTypeId="2" ParentId="22900" CreationDate="2021-01-13T04:01:51.413" Score="6" Body="&lt;p&gt;Rather than the survey by Liu et al. recommended above, I'd suggest you read the following survey paper for an overview of MORL (disclaimer - I was a co-author on this, but I genuinely think it is a much more useful introduction to this area)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.jair.org/index.php/jair/article/view/10836&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roijers, D. M., Vamplew, P., Whiteson, S., &amp;amp; Dazeley, R. (2013). A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48, 67-113.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Liu et al's survey, in my opinion, doesn't do much more than list and briefly describe the MORL algorithms which existed at that point. There's no deeper analysis of the field. The original version of their paper was also retracted due to blatant plagiarism of several other authors, including myself as can be confirmed here: &lt;a href=&quot;https://ieeexplore.ieee.org/document/6509978&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/document/6509978&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Our survey provides arguments for the need for multiobjective methods by describing 3 scenarios where agents using single-objective RL may be unable to provide a satisfactory solution that matches the needs of the user. Briefly, these are&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;strong&gt;unknown weights&lt;/strong&gt; scenario where the required trade-off between the objectives isn't known in advance, and so to be effective the agent must learn multiple policies corresponding to different trade-offs and then at run-time select the one which matches the current preferences (e.g. this can arise when the objectives correspond to different costs which vary in relative price over time);&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;strong&gt;decision support&lt;/strong&gt; scenario where scalarization of a reward vector is not viable (for example, in the case of subjective preferences, which defy explicit quantification), so the agent needs to learn a set of policies, and then present these to a user who will select their preferred option, and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the &lt;strong&gt;known weights&lt;/strong&gt; scenario where the desired trade-off between objectives is known, but its nature is such that the returns are non-additive (i.e. if the user's utility function is non-linear), and therefore standard single-objective methods based on the Bellman equation can't be directly applied.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;We propose a &lt;strong&gt;taxonomy&lt;/strong&gt; of MORL problems in terms of the &lt;em&gt;number of policies&lt;/em&gt; they require (single or multi-policy), the form of utility/scalarization function supported (linear or non-linear), and whether &lt;em&gt;deterministic&lt;/em&gt; or &lt;em&gt;stochastic&lt;/em&gt; policies are allowed, and relate this to the nature of the set of solutions which the MO algorithm needs to output. This taxonomy is then used to categorise existing MO planning and MORL methods.&lt;/p&gt;&#xA;&lt;p&gt;One final important contribution is identifying the distinction between maximising &lt;em&gt;Expected Scalarised Return (ESR)&lt;/em&gt; or &lt;em&gt;Scalarised Expected Return (SER)&lt;/em&gt;. The former is appropriate in cases where we are concerned about the results within each individual episode (for example, when treating a patient - that patient will only care about their own individual experience), while SER is appropriate if we care about the average return over multiple episodes. This has turned out to be a much more important issue than I anticipated at the time of the survey, and Diederik Roijers and his colleagues have examined it more closely since then (e.g. &lt;a href=&quot;http://roijers.info/pub/esr_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://roijers.info/pub/esr_paper.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="36575" LastEditorUserId="2444" LastEditDate="2021-01-15T19:16:31.817" LastActivityDate="2021-01-15T19:16:31.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25747" PostTypeId="1" CreationDate="2021-01-13T11:17:45.877" Score="2" ViewCount="58" Body="&lt;h3&gt;Context:&lt;/h3&gt;&#xA;&lt;p&gt;My team and I are working on a RL problem for a specific application. We have data collected from user interactions (states, actions, rewards, etc.).&lt;/p&gt;&#xA;&lt;p&gt;It is too costly for us to emulate agents. We decided therefore to concentrate on Offline RL techniques. For this, we are currently using the RL-Coach library by Intel, which offers support for Batch/Offline RL. More specifically, to evaluate policies in offline settings, we train a DDQN-BCQ model and evaluate the learned policies using Offline Policy Estimators (OPEs).&lt;/p&gt;&#xA;&lt;h3&gt;Problem:&lt;/h3&gt;&#xA;&lt;p&gt;In an Online RL setting, the decision of when to stop the training of an agent generally depends on the goal one wants to achieve (as described in this post: &lt;a href=&quot;https://stats.stackexchange.com/questions/322933/q-learning-when-to-stop-training&quot;&gt;https://stats.stackexchange.com/questions/322933/q-learning-when-to-stop-training&lt;/a&gt;). If the goal is to train until convergence (of rewards) but no longer, then you could for example stop when the standard deviation of your rewards over the last n steps drops under some threshold. If the goal is to compare the performance of two algorithms, then you should simply compare the two using the same number of training steps.&lt;/p&gt;&#xA;&lt;p&gt;However, in the Offline RL setting, I believe the conditions to stop training are not so clear. As stated above, no environement is directly available to evaluate our agents and the evaluation of the quality of the learned policy almost solely relies on OPEs, which are not always accurate.&lt;/p&gt;&#xA;&lt;p&gt;For me, I believe that there are two different options that would make sense. I am unsure if both those options are actually equivalent though.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The first option would be to stop training when the Q-values have converged/reached a plateau (i.e. when the Q-value network loss has converged) -- if they ever do, as we don't really have any guarantee of this happening with artificial neural networks. If the Q-values do reach a plateau, this would mean that our agent has reached some local optimum (or in the best case, the global optimum).&lt;/li&gt;&#xA;&lt;li&gt;The second option would be to only look at the OPEs reward estimation, and stop when they reach a plateau. However, different OPEs do not necessarily reach a plateau at the same time, as it can be seen in the figure below. In the Batch-RL tutorial of RL-Coach, it seems that they would simply select the agent at the epoch where the different OPEs give the highest policy value estimation, without checking that the loss of the network had converged or not (but this is only a tutorial, so I suppose we can't rely too much on it). &lt;a href=&quot;https://i.stack.imgur.com/HmYpU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HmYpU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h3&gt;Questions:&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What would be the best criteria for choosing when to stop the training of an agent in an Offline-RL setting?&lt;/li&gt;&#xA;&lt;li&gt;Also, the performance of an agent often heavily depends on the seed used for training. To evaluate the general performance, I believe you have to run multiple training with different seeds? However, in the end, you still want only a single agent to deploy. Should you simply select the one having the highest OPEs values among all the runs?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;P.S. I am not sure if this question should be splitted into two different posts, so please let me know if this is the case and I will edit the post!&lt;/p&gt;&#xA;" OwnerUserId="43780" LastEditorUserId="43780" LastEditDate="2021-01-18T14:40:27.173" LastActivityDate="2021-03-02T00:24:00.747" Title="Offline/Batch Reinforcement Learning: when to stop training and what agent to select" Tags="&lt;training&gt;&lt;q-learning&gt;&lt;off-policy-methods&gt;&lt;batch-learning&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25748" PostTypeId="2" ParentId="22416" CreationDate="2021-01-13T12:00:39.590" Score="2" Body="&lt;p&gt;I agree with Tomasz that the approach you are describing falls within the field of MORL. For a solid introduction MORL I would recommend the survey by Roijers, D. M., Vamplew, P., Whiteson, S., &amp;amp; Dazeley, R. (2013). A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48, 67-113.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.jair.org/index.php/jair/article/view/10836&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.jair.org/index.php/jair/article/view/10836&lt;/a&gt; (disclaimer: I'm an author in this, but I genuinely believe it will be useful to you).&lt;/p&gt;&#xA;&lt;p&gt;Our survey provides arguments for the need for multiobjective methods by describing three scenarios where agents using single-objective RL may be unable to provide a satisfactory solution which matches the needs of the user. Briefly these are (a) the unknown weights scenario where the required trade-off between the objectives isn't known in advance, and so to be effective the agent must learn multiple policies corresponding to different trade-offs and then at run-time select the one which matches the current preferences (eg this can arise when the objectives correspond to different costs which vary in relative price over time; (b) the decision support scenario where scalarization of a reward vector is not viable (for example, in the case of subjective preferences which defy explicit quantification), so the agent needs to learn a set of policies, and then present these to a user who will select their preferred option, and (c) the known weights scenario where the desired trade-off between objectives is known but its nature is such that the returns are non-additive (ie if the user's utility function is non-linear) and therefore standard single-objective methods based on the Bellman equation can't be directly applied.&lt;/p&gt;&#xA;&lt;p&gt;We propose a taxonomy of MORL problems in terms of the number of policies they require (single or multi-policy), the form of utility/scalarization function supported (linear or non-linear), and whether deterministic or stochastic policies are allowed, and relate this to the nature of the set of solutions which the MO algorithm needs to output. This taxonomy is then used to categorise existing MO planning and MORL methods.&lt;/p&gt;&#xA;&lt;p&gt;One final important contribution is identifying the distinction between maximising Expected Scalarised Return (ESR) or Scalarised Expected Return (SER). The former is appropriate in cases where we are concerned about the results within each individual episode (for example, when treating a patient - that patient will only care about their own individual experience), while SER is appropriate if we care about the average return over multiple episodes. This has turned out to be a much more important issue than I anticipated at the time of the survey, and Diederik Roijers and his colleagues have examined it more closely since then (eg &lt;a href=&quot;http://roijers.info/pub/esr_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://roijers.info/pub/esr_paper.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="36575" LastEditorUserId="2444" LastEditDate="2021-01-13T13:19:58.590" LastActivityDate="2021-01-13T13:19:58.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25750" PostTypeId="1" CreationDate="2021-01-13T13:34:53.003" Score="1" ViewCount="82" Body="&lt;p&gt;lets say I have three texts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&amp;quot;make a heading that says hello word&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;make a heading of hello world&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;create heading consist of hello world&amp;quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How can I fetch those groups of words using AI which is referring to heading i.e hello world in this case. Which AI frameworks or libraries can do that?&lt;/p&gt;&#xA;&lt;p&gt;in all examples heading is pointing to hello world (which i am referring as group of words). so basically i want those words which will be a part of heading or in other word there is a relationship between them. another example i can give is &amp;quot;I am watching Breaking bad&amp;quot; so there is a relationship between watching and breaking bad and i want to extract what are you watching.&lt;/p&gt;&#xA;&lt;p&gt;What's the best approach? Do I have to train a model for that or there are some other techniques that can get it done?&lt;/p&gt;&#xA;" OwnerUserId="43784" LastEditorUserId="43784" LastEditDate="2021-01-13T16:43:04.963" LastActivityDate="2021-03-22T20:01:15.120" Title="How to extract parameters from a text using AI/NLP" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;convolutional-neural-networks&gt;&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;" AnswerCount="2" CommentCount="6" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25752" PostTypeId="1" AcceptedAnswerId="25753" CreationDate="2021-01-13T16:36:59.663" Score="2" ViewCount="62" Body="&lt;p&gt;I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML.  I came across this blog: &lt;a href=&quot;https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, could someone help to clarify this for me: is this term in the red box literally interpreted as the gradient at &lt;span class=&quot;math-container&quot;&gt;$\theta_{k-1}$&lt;/span&gt; multiplied by the &lt;span class=&quot;math-container&quot;&gt;$\theta_k$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j6BjW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j6BjW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43791" LastEditorUserId="2444" LastEditDate="2021-01-14T16:36:36.033" LastActivityDate="2021-01-14T16:36:36.033" Title="What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?" Tags="&lt;math&gt;&lt;notation&gt;&lt;meta-learning&gt;&lt;gradient&gt;&lt;model-agnostic-meta-learning&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25753" PostTypeId="2" ParentId="25752" CreationDate="2021-01-13T20:36:18.397" Score="0" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta_{k-1}} \theta_k$&lt;/span&gt; is gradient of &lt;span class=&quot;math-container&quot;&gt;$\theta_k$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\theta_{k-1}$&lt;/span&gt;, it follows chain rule as noted in the side comment in the image. &lt;span class=&quot;math-container&quot;&gt;$\nabla_{\theta} \mathcal L(\theta_k)$&lt;/span&gt; is also not a Hessian but a gradient vector.&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2021-01-13T20:36:18.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25754" PostTypeId="1" CreationDate="2021-01-13T21:10:04.327" Score="1" ViewCount="32" Body="&lt;p&gt;A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant.&lt;/p&gt;&#xA;&lt;p&gt;Logically this seems like an easier function to learn when compared to something that can, when plotted, go up or down.&lt;/p&gt;&#xA;&lt;p&gt;Wikipedia has some example diagrams on &lt;a href=&quot;https://en.wikipedia.org/wiki/Monotonic_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;monotonic functions&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If I were to say that it is easier for a neural network to learn a monotonic function compared to a non-monotonic function would the statement be correct? If so, is there any reason to it other than 'it only goes one way'?&lt;/p&gt;&#xA;" OwnerUserId="32265" LastActivityDate="2021-01-13T21:10:04.327" Title="Are monotonically increasing functions easier to learn?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;math&gt;&lt;function-approximation&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25756" PostTypeId="1" CreationDate="2021-01-14T00:23:31.400" Score="0" ViewCount="35" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f9bsZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f9bsZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If all children Nodes of the selected node are terminal in the selection phase - you obviously run into a problem. So how do I prevent a Node to be selected that only has terminal children?&lt;/p&gt;&#xA;" OwnerUserId="43808" LastActivityDate="2021-01-14T00:23:31.400" Title="MCTS: What if all children of a node are terminal?" Tags="&lt;monte-carlo-tree-search&gt;&lt;minimax&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25757" PostTypeId="5" CreationDate="2021-01-14T00:30:19.130" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-14T00:30:19.130" LastActivityDate="2021-01-14T00:30:19.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25758" PostTypeId="4" CreationDate="2021-01-14T00:30:19.130" Score="0" Body="For questions related to Model-Agnostic Meta-Learning (MAML), proposed in &quot;Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks&quot; by Chelsea Finn et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-14T00:30:19.130" LastActivityDate="2021-01-14T00:30:19.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25759" PostTypeId="1" AcceptedAnswerId="25766" CreationDate="2021-01-14T04:13:11.043" Score="3" ViewCount="45" Body="&lt;p&gt;During the course of training a DQN agent, all visited states are stored in a replay buffer. Therefore would it be practically possible for a CNN, given a reasonable amount of data, to predict the next RL state (in the form of an image) for a given action?&lt;/p&gt;&#xA;&lt;p&gt;For example, take the game of Atari as shown below -&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lciZa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lciZa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Here the agent can take 2 major actions - go left and go right. Would a CNN be generate the image of the bar going right/left for the respective actions?&#xA;My practical knowledge of CNNs is quite limited and therefore I'm trying to gauge the abilities of CNNs before I take up a project.&lt;/p&gt;&#xA;" OwnerUserId="31755" LastActivityDate="2021-01-14T11:33:43.697" Title="Can a convolutional network predict states for a RL Agent" Tags="&lt;reinforcement-learning&gt;&lt;convolutional-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25760" PostTypeId="2" ParentId="16348" CreationDate="2021-01-14T05:21:31.050" Score="2" Body="&lt;p&gt;This is actually a highly technical term, which has been kind of misused and overgeneralized in many places.&lt;/p&gt;&#xA;&lt;p&gt;What does 'convergence' mean in a literal sense? It simply means that a sequence of terms indexed by &lt;span class=&quot;math-container&quot;&gt;$\mathbb{N}$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$X_1, X_2, X_3,..$&lt;/span&gt;) tends to a certain fixed value say &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\mathbb{N} \rightarrow \infty$&lt;/span&gt;, but may not achieve the fixed value. (there are a few technical details associated with this definition but I won't go into it as it requires some analysis)&lt;/p&gt;&#xA;&lt;p&gt;When it comes to ML we are looking at probabilistic or stochastic models. When we talk about convergence in ML, we generally mean 4 types of convergence:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.probabilitycourse.com/chapter7/7_2_5_convergence_in_probability.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convergence in Probability&lt;/a&gt;&lt;/strong&gt;: This means that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{N} \rightarrow \infty$&lt;/span&gt; your likelihood of &lt;span class=&quot;math-container&quot;&gt;$X_N$&lt;/span&gt; (a sequence of random variables) being very close to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; also increases i.e &lt;span class=&quot;math-container&quot;&gt;$P(\omega:[|X_N(\omega)-X(\omega)| &amp;gt;\epsilon]) \rightarrow 0$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$N\rightarrow \infty$&lt;/span&gt;. This type of convergence is mostly used in Statistical Learning Theory.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.probabilitycourse.com/chapter7/7_2_7_almost_sure_convergence.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Almost Sure Convergence&lt;/a&gt;&lt;/strong&gt;: This means that &lt;span class=&quot;math-container&quot;&gt;$\mathbb{N} \rightarrow \infty$&lt;/span&gt; your probability of &lt;span class=&quot;math-container&quot;&gt;$X_N$&lt;/span&gt; (a sequence of random variables) being very close to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; (&lt;strong&gt;NOTE&lt;/strong&gt;: Here there is no likelihood of being close to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, we straight up say it must be close to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;) i.e &lt;span class=&quot;math-container&quot;&gt;$P(\omega:[|X_N(\omega)-X(\omega)| &amp;gt;\epsilon]) = 0 $&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$N\rightarrow \infty$&lt;/span&gt;. This is a stronger verision of the previous convergence, and this is the type of convergence I have seen being used in RL.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.probabilitycourse.com/chapter7/7_2_4_convergence_in_distribution.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Convergence in Distribution&lt;/strong&gt;&lt;/a&gt;: This means that the distribution of a sequence of random variables tend to a certain distribution i.e&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{N\to \infty} F_{X_n} = F_X$$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Convergence in &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;'th moment&lt;/strong&gt;: This means that a sequence of random variables will converge to a certain mean as the sequence goes to infinity or simply put:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{N\to \infty} \mathbb E[|X_N - \mu|^r] \ 0$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the value to which the random variables in converge in &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;th moment.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A simple useful &lt;a href=&quot;http://web.mit.edu/14.381/www/ho6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt; for all the aforementioned modes of convergence.&lt;/p&gt;&#xA;&lt;p&gt;As a side note, this is meant as an informal reference, there is a lot of mathematical analysis involved in getting conditions for when these hold for a sequence of random variables.&lt;/p&gt;&#xA;&lt;p&gt;In the context of ML, one can think of &lt;span class=&quot;math-container&quot;&gt;$L(w_N,y_N,x_N)$&lt;/span&gt; in place of &lt;span class=&quot;math-container&quot;&gt;$X_N$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$w_N, x_N, y_N$&lt;/span&gt; will be the one deciding on the next step, and one can check if it satisfies any of the aforementioned convergence using some sufficient conditions. Note that when we do convex optimization we are talking about almost sure convergence (if the method used works), while for SGD due to stochasticity one might formulate it in the convergence in probability setting.&lt;/p&gt;&#xA;&lt;p&gt;As a concrete example, the PAC learning paradigm uses the convergence in probability framework (without going into details the idea of PAC learning is that with increasing size of dataset your confidence about your classifier increases which can be interpreted as some sort of convergence in probability with the actual loss as the random variable, check the PAC learning framework &lt;a href=&quot;http://mi.eng.cam.ac.uk/%7Ecz277/doc/Slides-PAC.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), while the Q-learning convergence (&lt;a href=&quot;http://users.isr.ist.utl.pt/%7Emtjspaan/readingGroup/ProofQlearning.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;proof&lt;/a&gt; as suggested in the comments) is an almost sure convergence under some assumptions (probably proved by Bertsekas and Tsitsiklis), CLT is an example of convergence in distribution.&lt;/p&gt;&#xA;" OwnerUserId="9947" LastEditorUserId="9947" LastEditDate="2021-02-13T10:14:32.963" LastActivityDate="2021-02-13T10:14:32.963" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25761" PostTypeId="1" CreationDate="2021-01-14T05:36:51.963" Score="2" ViewCount="46" Body="&lt;p&gt;I have a combined network consisting of two parts: one is for images and the other is for numerical data. Each sample is matched with a numerical case by an ID. For this combined network, a &lt;code&gt;lr&lt;/code&gt; of 0.01 was found to be best working via hyperparameter tunings&lt;/p&gt;&#xA;&lt;p&gt;However, when I trained them as a separate task (binary classification), a &lt;code&gt;lr&lt;/code&gt; of 0.001 for images and 0.01 for numerical data were best.&#xA;As for an AUC metric, the combined network (0.818) is performing on average of image and numerical networks (0.799 and 0.821, respectively).&#xA;Here I thought maybe the combined network's &lt;code&gt;lr&lt;/code&gt; is too high for image part and should apply lower &lt;code&gt;lr&lt;/code&gt; for that part. But, I don't know if it is possible.&lt;/p&gt;&#xA;&lt;p&gt;If anyone has any idea of what is what, let me know&lt;/p&gt;&#xA;" OwnerUserId="31870" LastActivityDate="2021-01-14T11:55:14.460" Title="Is it possible to train one part of the network with a particular learning rate and the other part with a different one?" Tags="&lt;neural-networks&gt;&lt;convolutional-neural-networks&gt;&lt;pytorch&gt;&lt;hyperparameter-optimization&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25762" PostTypeId="2" ParentId="15624" CreationDate="2021-01-14T06:01:08.507" Score="0" Body="&lt;p&gt;The following is my intuition about the behavior of the algorithm considering your question, based on my knowledge and experience with single BMU SOMs. I didn't verify it experimentally.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;At early stages of training:&lt;/strong&gt;&#xA;It should disturb the SOM topology preserving properties, as you're assigning the same pattern to different (and probably distant) locations in the lattice, making the distribution multimodal. This is not terrible on its own, as different positions could encode different relationships to neighbors, overcoming the limitations of a single 2D/3D neighborhood. But if your SOM is not large enough, the multiple modes may collapse, resulting in unimodal distributions covering large portions of the map, wasting space. It all depends on your training schedule.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;At later stages of training:&lt;/strong&gt;&#xA;If you started with a single BMU at early stages, it wouldn't provide any meaningful impact, as a SOM, by definition, clusters similar patterns at close positions of its lattice. In other words, the neurons closest to your input are already inside the BMU update radius. If you trained with multiple BMUs since the beginning, you would keep finding / activating the multiple maxima of your multimodal distributions for each pattern. The dimensionality reduction would become even more nonlinear than it already is with single BMUs. This may or may not be a problem, depending on your application.&lt;/p&gt;&#xA;" OwnerUserId="144" LastActivityDate="2021-01-14T06:01:08.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25764" PostTypeId="5" CreationDate="2021-01-14T10:35:24.760" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-14T10:35:24.760" LastActivityDate="2021-01-14T10:35:24.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25765" PostTypeId="4" CreationDate="2021-01-14T10:35:24.760" Score="0" Body="For questions related to Monte Carlo Dropout (aka MC Dropout or MCD), which is, for example, described in &quot;Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning&quot; (2015) Yarin Gal et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-14T10:35:24.760" LastActivityDate="2021-01-14T10:35:24.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25766" PostTypeId="2" ParentId="25759" CreationDate="2021-01-14T11:33:43.697" Score="2" Body="&lt;p&gt;It sounds like what you're suggesting is similar to what is done in methods that use a planner. These methods looks to learn the dynamics of the MDP to use to plan during training; that is they want to be able to learn the transition probabilities &lt;span class=&quot;math-container&quot;&gt;$p(s'| s, a)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://arxiv.org/abs/1911.08453&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper that I read recently they note that learning to predict environment dynamics when the state/action space is high dimensional, as is the case with images, is difficult; so whilst it may be possible in theory it would be difficult to do and if you were predicting many steps into the future then the error would compound.&lt;/p&gt;&#xA;&lt;p&gt;A way around this, as is done in the referenced paper, is to use predict environment dynamics in a latent space. This means that they use a latent variable to predict the next state using e.g. Variational Autoencoders.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2021-01-14T11:33:43.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25767" PostTypeId="2" ParentId="25761" CreationDate="2021-01-14T11:55:14.460" Score="1" Body="&lt;p&gt;Create two different optimizers and split the subnets' parameters into either with different lrs. You will have to call &lt;code&gt;optimizer1.step(), optimizer2.step()&lt;/code&gt; with a single &lt;code&gt;backward()&lt;/code&gt; call&lt;/p&gt;&#xA;" OwnerUserId="21738" LastActivityDate="2021-01-14T11:55:14.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25768" PostTypeId="1" AcceptedAnswerId="25774" CreationDate="2021-01-14T13:51:51.100" Score="0" ViewCount="29" Body="&lt;p&gt;I have an idea to adapt YOLO algorithm to my application, the original YOLO algorithm is for image classifications, which have 24 convolutional layers with output class of 1000, is it possible to replace the basic network of YOLO with Alexnet or Resnet or the custom network structure designed by myself? Noted that my application have input shape of 500 * 10000 * 1 and only 4 classes for classification.&lt;/p&gt;&#xA;" OwnerUserId="43822" LastActivityDate="2021-01-14T19:52:06.393" Title="Is it possible to modify or replace the basic network of YOLO?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25769" PostTypeId="1" CreationDate="2021-01-14T14:39:47.013" Score="0" ViewCount="64" Body="&lt;p&gt;Suppose I have a set of messages A,B,C,D and I want to produce the best message for a website user at a given time.&lt;/p&gt;&#xA;&lt;p&gt;For training I plan to show random users a random single message [A/B/C/D] and fill these columns (i'm simplifying the data for illustration)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;converted before&lt;/li&gt;&#xA;&lt;li&gt;funnel state (e.g awareness, search, decision)&lt;/li&gt;&#xA;&lt;li&gt;number of page views&lt;/li&gt;&#xA;&lt;li&gt;message shown [A-D]&lt;/li&gt;&#xA;&lt;li&gt;Time to convert (this will be updated later if there is a conversion)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I want to predict what is the best message to show to a specific user in order to maximise the chance of conversion (=min time to convert).&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure how to represent this for training and inference. Its not a simple prediction like predicting one of the given data points.&lt;/p&gt;&#xA;&lt;p&gt;One option is to run prediction of time to buy for each of the messages but&#xA;1- its not efficient&#xA;2- It will prefer messages that are shown closer to purchase time regardless if they fit the current user time.&lt;/p&gt;&#xA;" OwnerUserId="43826" LastActivityDate="2021-02-18T03:01:05.110" Title="How to predict the best from a set of messages - best practice" Tags="&lt;machine-learning&gt;&lt;training&gt;&lt;data-preprocessing&gt;&lt;feature-selection&gt;&lt;features&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25770" PostTypeId="1" CreationDate="2021-01-14T14:54:23.613" Score="1" ViewCount="70" Body="&lt;p&gt;Consider a POMDP with a finite number of environment states, &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{S}| = N$&lt;/span&gt;, but the number of belief states is &lt;a href=&quot;https://mathworld.wolfram.com/UncountablyInfinite.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;uncountably infinite&lt;/a&gt;. The belief state space is the convex hull of an &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; simplex. Each turn this space is sampled with a flat probability distribution. As you are sampling from an uncountably infinite set of belief states, the probability of a belief state recurring in a finite number of samples is zero.&lt;/p&gt;&#xA;&lt;p&gt;Now, let's suppose that there are a finite number of episodes, and each episode ends after 1-time step. At the only time step of the episode, the agent receives a belief state &lt;span class=&quot;math-container&quot;&gt;$b(s)$&lt;/span&gt; over some fixed set of contexts &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, selects a single action, and receives a single reward, before the episode ends.&lt;/p&gt;&#xA;&lt;p&gt;I understand that the belief state value function, &lt;span class=&quot;math-container&quot;&gt;$V(b)$&lt;/span&gt;, is piecewise linear and convex, with a single hyperplane for each action (see e.g. [&lt;a href=&quot;http://cs.brown.edu/research/ai/pomdp/tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]).&lt;/p&gt;&#xA;&lt;p&gt;My question is, given that I only observe the belief states and the sampled rewards, how do I identify the value function, given that a belief state &lt;span class=&quot;math-container&quot;&gt;$b(s)$&lt;/span&gt; has an infinitesimally small probability of occuring again?&lt;/p&gt;&#xA;&lt;p&gt;The expected reward for a given belief state &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is just a linear function &lt;span class=&quot;math-container&quot;&gt;$\alpha \cdot b$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the vector of the rewards for each state of the environment. But I cannot simply learn a linear model here because &lt;span class=&quot;math-container&quot;&gt;$\alpha \cdot b$&lt;/span&gt; gives me the expected reward for a given belief state, but I may never start with this belief state again and so cannot simply calculate the sample mean expected reward.&lt;/p&gt;&#xA;" OwnerUserId="43828" LastEditorUserId="2444" LastEditDate="2021-01-24T14:09:33.657" LastActivityDate="2021-01-24T14:09:33.657" Title="How do I learn the value function for a POMDP with a single-step horizon (bandit)?" Tags="&lt;reinforcement-learning&gt;&lt;value-functions&gt;&lt;multi-armed-bandit&gt;&lt;pomdp&gt;&lt;contextual-bandit&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25771" PostTypeId="1" CreationDate="2021-01-14T18:22:49.970" Score="3" ViewCount="50" Body="&lt;p&gt;I'm new to deep learning. I wanted to know: do we use pre-processing in deep learning? Or it is only used in machine learning. I searched for it and its methods on the internet, but I didn't find a suitable answer.&lt;/p&gt;&#xA;" OwnerUserId="30164" LastEditorUserId="2444" LastEditDate="2021-01-14T18:32:47.087" LastActivityDate="2021-01-14T19:46:34.503" Title="Is pre-processing used in deep learning?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;data-preprocessing&gt;" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25772" PostTypeId="2" ParentId="25771" CreationDate="2021-01-14T18:41:29.950" Score="1" Body="&lt;p&gt;Yes, sure, &lt;a href=&quot;https://ch.mathworks.com/help/deeplearning/deep-learning-data-management-and-preprocessing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;data pre-processing&lt;/a&gt; is also done in deep learning. For example, we often normalize (or scale) the inputs to neural networks. If the inputs are images, we often resize them so that they all have the same dimensions. Of course, the pre-processing step that you apply depends on your data, neural network, and task.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorflow/models/blob/master/official/nlp/transformer/data_pipeline.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; or &lt;a href=&quot;https://github.com/tensorflow/models/blob/master/official/vision/image_classification/mnist_main.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; are two examples of implementations that perform a pre-processing step (normalization in the second case). You can find more explanations and examples &lt;a href=&quot;https://ch.mathworks.com/help/deeplearning/deep-learning-data-management-and-preprocessing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and probably &lt;a href=&quot;https://github.com/tensorflow/models/tree/master/official&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; too.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-14T18:56:56.227" LastActivityDate="2021-01-14T18:56:56.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25773" PostTypeId="2" ParentId="25771" CreationDate="2021-01-14T19:46:34.503" Score="0" Body="&lt;p&gt;Adding to nbro's solution, the less you have to normalize/balance/preprocess/augment, the better, e.g. because then you know for sure that the accuracy is the achievement of the model rather than data combing. For example, if you can achieve the same accuracy using two approaches (e.g. with the image dataset):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;for each image, subtract global mean, divide by global standard deviation&lt;/li&gt;&#xA;&lt;li&gt;1)+random flips, random crops, color jitter, etc,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;then 1), if you can achieve a comparable accuracy, is a better solution, as the model is more general. The same refers to the balancing of the data - if you can train a good model without it, it's an additional strength.&lt;/p&gt;&#xA;" OwnerUserId="21738" LastActivityDate="2021-01-14T19:46:34.503" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25774" PostTypeId="2" ParentId="25768" CreationDate="2021-01-14T19:52:06.393" Score="0" Body="&lt;p&gt;No, the original (or any) YOLO is for object detection. You can easily replace the feature extractor (DarkNet53, if I'm not mistaken) with any other, as long as you maintain the correct number of weights in the detection layer.&lt;/p&gt;&#xA;" OwnerUserId="21738" LastActivityDate="2021-01-14T19:52:06.393" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25775" PostTypeId="1" CreationDate="2021-01-14T21:23:25.950" Score="1" ViewCount="243" Body="&lt;p&gt;Say I have a machine learning model trained on a laptop and I then want to embed/deploy the model on a microcontroller. How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;I know that TensorflowLite Micro generates a C header to be added in the project and then be embedded, but every example I read shows how it is done with neural networks, which seems legit as TensorFlow is primarily used for deep learning.&lt;/p&gt;&#xA;&lt;p&gt;But how can I do the same with any type of model, like the ones there is in scikit-learn? So, I'm not interested in necessarily doing this with TensorflowLite Micro, but I'm interested in the general approach to solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="43239" LastEditorUserId="43239" LastEditDate="2021-01-18T12:37:00.760" LastActivityDate="2021-05-23T13:47:33.030" Title="How to embed/deploy an arbitrary machine learning model on microcontrollers?" Tags="&lt;machine-learning&gt;&lt;tensorflow&gt;&lt;models&gt;" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25776" PostTypeId="2" ParentId="9662" CreationDate="2021-01-14T21:52:19.103" Score="0" Body="&lt;p&gt;One very popular RL algorithm that is capable of predicting multiple action outputs concurrently is &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proximal Policy Optimization&lt;/a&gt;. In that algorithm, one or more, say &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, tuples of outputs, &lt;span class=&quot;math-container&quot;&gt;$(\mu, \sigma)$&lt;/span&gt;, can be predicted at once (having &lt;span class=&quot;math-container&quot;&gt;$2*n$&lt;/span&gt; output nodes), where each tuple is used to parameterize a Gaussian distribution from which a respective action value is sampled. The thus sampled action values are then applied in the simulation/game. By slightly modifying this procedure, this can of course also be applied to discrete action spaces equally well.&lt;/p&gt;&#xA;&lt;p&gt;To get you started, multiple high-quality implementations of PPO are available for rapid prototyping, e.g. &lt;a href=&quot;https://github.com/openai/baselines&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAI baselines&lt;/a&gt; or &lt;a href=&quot;https://stable-baselines.readthedocs.io/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;stable-baselines&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastActivityDate="2021-01-14T21:52:19.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25777" PostTypeId="1" CreationDate="2021-01-14T22:59:05.210" Score="1" ViewCount="9" Body="&lt;p&gt;I have a data set of 3D images with some bounding box annotations. The images are too large to train something like YOLO 3D (would run out of memory), so I instead created slices of the 3D images with corresponding 2D bounding boxes and trained a 2D object detector. During inference, I assemble the 2D detections into 3D detections. I constructed some simple heuristics to do that, which works fine, but I am wondering if there aren't any established methods of doing that.&lt;/p&gt;&#xA;&lt;p&gt;I appreciate if you could point me in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="43712" LastActivityDate="2021-01-14T22:59:05.210" Title="Aggregating 2D object detections into 3D object detections" Tags="&lt;computer-vision&gt;&lt;object-detection&gt;&lt;yolo&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25780" PostTypeId="1" CreationDate="2021-01-15T11:40:03.313" Score="0" ViewCount="11" Body="&lt;p&gt;I have a Fuzzy controller with a desirable response, and I'm wanting to use the input and output data to derive an ANFIS based controller.&lt;/p&gt;&#xA;&lt;p&gt;ANFIS requires data for 2 inputs and 1 output, I have the output data and the input data, however, when I train the model and use it in my Fuzzy controller, the response is zero, I suspect there's a problem with the input data I'm using perhaps with normalising.&lt;/p&gt;&#xA;&lt;p&gt;Here's the model, I'm collecting the input and output data from the scopes EScope = Error, CEScope = Change in Error and Scope for the system output.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wmlE2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wmlE2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43852" LastActivityDate="2021-01-15T11:40:03.313" Title="Deriving an ANFIS based controller" Tags="&lt;neural-networks&gt;&lt;fuzzy-logic&gt;&lt;control-problem&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25781" PostTypeId="1" CreationDate="2021-01-15T13:22:35.357" Score="1" ViewCount="69" Body="&lt;p&gt;I just started working on a DRL project from scratch. The state of each episode can be expressed as a state set &lt;span class=&quot;math-container&quot;&gt;$S=(S^A, S^B, S^C, S^D)$&lt;/span&gt;. Each subset is a feature set of a constituent component of the environment, say, &lt;span class=&quot;math-container&quot;&gt;$S^A=(a_1, a_2, a_3)$&lt;/span&gt;. To model components, I decided to create four pythonic classes with attributes as features. For example, &lt;code&gt;class A&lt;/code&gt; is like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class A:&#xA;    def __init__(self, a1, a2, a3): &#xA;        self.a1 = a1&#xA;        self.a2 = a2&#xA;        self.a3 = a3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each class has some methods that help in the interaction with other components (classes) and is used in the environment's &lt;code&gt;step function&lt;/code&gt; to generate actions.&lt;/p&gt;&#xA;&lt;p&gt;I am going to create one instance of &lt;code&gt;class A&lt;/code&gt;, 10 instances of &lt;code&gt;class B&lt;/code&gt;, 20 instances of &lt;code&gt;class C&lt;/code&gt;, and a random number of between 1-10 instances of &lt;code&gt;class D&lt;/code&gt; at the beginning of each episode. So, my observation includes 33-42 states of entities.&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, the observation space is usually encoded in n-dimensional arrays as it is in OpenAI Gym. &lt;strong&gt;Is it possible, feasible, or considered good practice to store instances as a sub-state of the whole observation space?&lt;/strong&gt; In my case, it would be like storing 33-42 instances in an array (list) of 33-42 elements.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your time and suggestions!&lt;/p&gt;&#xA;" OwnerUserId="43578" LastEditorUserId="43578" LastEditDate="2021-01-18T16:55:08.287" LastActivityDate="2021-01-18T16:55:08.287" Title="Is object-based representation of the observation space feasible?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;environment&gt;&lt;state-spaces&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25782" PostTypeId="1" CreationDate="2021-01-15T15:10:25.733" Score="0" ViewCount="62" Body="&lt;p&gt;I am a (soon-to-become, to be honest) theoretical physicist. I want to learn a bit about AI. So as you know in physics we develop theories based on as few and as simple basic equations as possible which shall explain as much of the experimental results and observations as possible. I feel that this is kind of not how AI solves problems.&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that AI can be understood as a very generalized and abstract statistics software package handling input data in a general way to find the &amp;quot;best fit&amp;quot; to some form of problem. Is that correct? I know it isn't. But is it vaguely correct?&lt;/p&gt;&#xA;&lt;p&gt;I give you an example. In weather prediction there is a technique called MOS (model output statistics). It collects output from numerical weather prediction models (simulation software) as well as observational data and finds statistical relations between them to correct the model output for errors. For example, it might be that the intensity of precipitation in London is on average underestimated by the model by 10 %, so MOS will correct for that. Over time, it improves itself, because it collects more and more data. Is this already a form of AI?&lt;/p&gt;&#xA;" OwnerUserId="43857" LastEditorUserId="2444" LastEditDate="2021-01-16T17:02:02.903" LastActivityDate="2021-01-16T17:02:02.903" Title="Can AI be understood as a generalized statistics tool?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;definitions&gt;&lt;statistical-ai&gt;&lt;statistics&gt;" AnswerCount="1" CommentCount="0" ClosedDate="2021-01-16T16:51:12.070" ContentLicense="CC BY-SA 4.0" />
  <row Id="25783" PostTypeId="1" CreationDate="2021-01-15T15:55:19.413" Score="0" ViewCount="15" Body="&lt;p&gt;Attached is a plot showing that y regresses on individual features: &lt;a href=&quot;https://i.stack.imgur.com/pypYx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scatter Plot&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And as shown, each feature has minuscule predictive power on y. I have 130 features like this, all of them are more or less Normally distributed, some have long tails.&lt;/p&gt;&#xA;&lt;p&gt;Is a Neural Network a good solution for this kind of data? And do I still need to preprocess the data before feeding it to my model, I know I probably need to de-noise or use PCA before feeding it into my model, but is normalization necessary considering that the magnitude of the features are on a very similar scale?&lt;/p&gt;&#xA;" OwnerUserId="43838" LastActivityDate="2021-01-15T15:55:19.413" Title="Low Signal-To-Noise Ratio Data Processing and Model Choices" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;data-preprocessing&gt;&lt;data-science&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25784" PostTypeId="1" CreationDate="2021-01-15T18:08:54.560" Score="0" ViewCount="20" Body="&lt;p&gt;In the following plot, I have features that all have a very small predictive power on &lt;code&gt;y&lt;/code&gt;, there is a low signal-to-noise ratio.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XqCxZ.jpg&quot; alt=&quot;plot&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;In order to de-noise them, I tried PCA and k-mean clustering based on the principal components that are returned by PCA, but the entropy of &lt;code&gt;y&lt;/code&gt; on each cluster is still large. Are there better ways to de-noise this kind of data?&lt;/p&gt;&#xA;" OwnerUserId="43838" LastEditorUserId="3171" LastEditDate="2021-03-28T17:53:51.193" LastActivityDate="2021-03-28T17:53:51.193" Title="How to improve de-noise algorithm on low signal-to-noise ratio features?" Tags="&lt;neural-networks&gt;&lt;data-science&gt;&lt;k-means&gt;&lt;principal-component-analysis&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25785" PostTypeId="2" ParentId="25782" CreationDate="2021-01-15T19:36:21.450" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;My understanding is that AI can be understood as a very generalized and abstract statistics software package handling input data in a general way to find the &amp;quot;best fit&amp;quot; to some form of problem. Is that correct? I know it isn't. But is it vaguely correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. It's not correct, in my opinion, not even vaguely and in many ways.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AI is not (necessarily) abstract (although there are examples of theoretical frameworks of AI, such as &lt;a href=&quot;https://ai.stackexchange.com/a/10377/2444&quot;&gt;AIXI&lt;/a&gt;, which may be of interest to you, given that you are about to become a theoretical physicist)&lt;/li&gt;&#xA;&lt;li&gt;AI does not necessarily generalize statistics or statistical concepts&lt;/li&gt;&#xA;&lt;li&gt;AI is not just machine learning or, more precisely, supervised learning. You're very likely referring to &lt;em&gt;supervised learning&lt;/em&gt; when you say &amp;quot;best fit&amp;quot;, but there are other forms of learning (such as &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reinforcement learning&lt;/a&gt;) and there are other aspects of AI apart from learning, such as perception, control, etc. Some people sometimes claim that machine learning is glorified statistics, but, although the two are similar and related, I don't think that's exactly correct.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To know what AI is, you should read the book &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt;. If you want to know what machine learning is, then there are many books that you can read, such as &lt;a href=&quot;http://www.cs.cmu.edu/%7Etom/mlbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning&lt;/a&gt; (1997) by Tom M. Mitchell. If you are interested in the relationship between machine learning and statistics, you may be interested in &lt;a href=&quot;https://stats.stackexchange.com/q/6/82135&quot;&gt;this post&lt;/a&gt;. You should also read &lt;a href=&quot;https://ai.stackexchange.com/a/11387/2444&quot;&gt;this answer&lt;/a&gt;, which describes what AI is or may refer to. &lt;a href=&quot;https://ai.stackexchange.com/a/53/2444&quot;&gt;This answer&lt;/a&gt; gives a nice brief description of the difference between AI and ML.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-15T19:57:11.433" LastActivityDate="2021-01-15T19:57:11.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25786" PostTypeId="2" ParentId="3419" CreationDate="2021-01-16T03:36:35.793" Score="0" Body="&lt;p&gt;For the definition and calculation of perplexity, please refer to &lt;a href=&quot;https://stats.stackexchange.com/a/280794/103153&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Google proposed a human evaluation metric called Sensibleness and Specificity Average (SSA) which combines two fundamental aspects of a humanlike chatbot: making sense and being specific. And they conducted some experiments and found that perplexity aligns very well with the SSA.&lt;/p&gt;&#xA;&lt;p&gt;Here is the explanation in the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Perplexity measures how well the model predicts the test set data; in other words, how accurately it anticipates what people will say next.&lt;br /&gt;&#xA;Our results indicate most of the variance in the human metrics can be explained by the test perplexity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Their experiments showed a very strong correlation between SSA and perplexity(the lower the perplexity the higher the SSA).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VUmLB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VUmLB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/2001.09977.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards a Human-like Open-Domain Chatbot&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2021-01-16T03:36:35.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25787" PostTypeId="1" CreationDate="2021-01-16T05:54:52.280" Score="0" ViewCount="15" Body="&lt;p&gt;I am performing classification using AlexNet as transfer learning(simply say performing classification using CNN) for five types of class on 18000 images. These 18000 images are divided into Train, Test and validation set. I have the test accuracy 76% and for individual classes I got the accuracy as&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;83%&lt;/li&gt;&#xA;&lt;li&gt;74%&lt;/li&gt;&#xA;&lt;li&gt;86%&lt;/li&gt;&#xA;&lt;li&gt;89%&lt;/li&gt;&#xA;&lt;li&gt;63%&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now supposed I am using this model  on production and I fed some random image not from the 18000 images, but belongs to five class to get/predict the class of that image. My question is it possible to estimate   probability  of the predicted class? How likely it belong to the one out of five class?&lt;/p&gt;&#xA;" OwnerUserId="37233" LastActivityDate="2021-01-16T05:54:52.280" Title="How to get the prediction probability of random sample image from multiclass classification model?" Tags="&lt;convolutional-neural-networks&gt;&lt;probability&gt;&lt;multi-label-classification&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25788" PostTypeId="2" ParentId="2922" CreationDate="2021-01-16T07:13:25.770" Score="0" Body="&lt;p&gt;It seems that Automated Knowledge Base Construction would be unfavorable.&lt;/p&gt;&#xA;&lt;p&gt;As &lt;a href=&quot;https://matt-gardner.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matt Gardner&lt;/a&gt; noted in &lt;a href=&quot;https://nlphighlights.allennlp.org/083_knowledge_base_construction_with_sebastian_riedel&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLP Highlights&lt;/a&gt; in 2019 that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Um, but I know that Google, for instance, canceled their knowledge base construction project because there wasn’t high enough precision to actually be useful in their product.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The canceled project &lt;a href=&quot;https://research.google/pubs/pub45634/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge Vault&lt;/a&gt; is an Automated Knowledge Base Construction(AKBC) project launched in August 2014.&lt;/p&gt;&#xA;&lt;p&gt;There are three methods to integrate knowledge into the neural networks: 1) pre-trained models like &lt;a href=&quot;https://github.com/google-research/bert&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt;, &lt;a href=&quot;https://github.com/google-research/electra&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELECTRA&lt;/a&gt;; 2) &lt;a href=&quot;https://arxiv.org/abs/2005.11401&quot; rel=&quot;nofollow noreferrer&quot;&gt;retrieval-augmented generative model&lt;/a&gt;; 3) flesh out the triples into natural text as that in &lt;a href=&quot;https://ai.googleblog.com/2021/05/kelm-integrating-knowledge-graphs-with.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;KELM&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In a 2020 paper &lt;a href=&quot;https://ai.googleblog.com/2020/08/realm-integrating-retrieval-into.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;REALM: Integrating Retrieval into Language Representation Models&lt;/a&gt;, they utilized a retrieval rather than a knowledge base to enrich neural networks. And the best systems in the &lt;a href=&quot;https://efficientqa.github.io/assets/report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;NeurIPS 2020 EfficientQA competition&lt;/a&gt; all relied on retrieval.&lt;/p&gt;&#xA;&lt;p&gt;Knowledge bases that are actively being maintained receive a lot of annotation and curation, as stated in &lt;a href=&quot;https://nlphighlights.allennlp.org/083_knowledge_base_construction_with_sebastian_riedel&quot; rel=&quot;nofollow noreferrer&quot;&gt;that podcast&lt;/a&gt;. If curation and annotation are not sufficient, the knowledge base maybe cannot apply in AI.&lt;/p&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="5351" LastEditDate="2021-05-22T07:11:20.870" LastActivityDate="2021-05-22T07:11:20.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25789" PostTypeId="2" ParentId="9358" CreationDate="2021-01-16T07:49:58.453" Score="1" Body="&lt;p&gt;Human evaluation is the gold standard as stated in &lt;a href=&quot;https://nlphighlights.allennlp.org/120_evaluation_of_text_generation_with_asli_celikyilmaz&quot; rel=&quot;nofollow noreferrer&quot;&gt;this podcast&lt;/a&gt; by Asli Celikyilmaz, even if you only test a very small part of the generated text.&lt;/p&gt;&#xA;&lt;p&gt;You needed an automated method and this one: &lt;a href=&quot;https://ai.googleblog.com/2020/05/evaluating-natural-language-generation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLEURT by Google&lt;/a&gt; would be helpful. It's a flexible, semantic-level metric/model trained in a multi-stage way: 1) masked language model like BERT; 2) pre-training on synthetic sentence pairs; 3) fine-turning on public human ratings(WMT metrics shared task); 4) fine-tuning on application-specific human ratings. And it applies very well even in the presence of domain drift.&lt;/p&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2021-01-16T07:49:58.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25790" PostTypeId="1" CreationDate="2021-01-16T08:02:21.720" Score="1" ViewCount="29" Body="&lt;p&gt;I wrote two programs that simulated 10000 episodes in gym environment CartPole-v0.&lt;/p&gt;&#xA;&lt;p&gt;The first program takes random moves in every steps in each episode. The average reward over 10000 episodes is 22.1582.&lt;/p&gt;&#xA;&lt;p&gt;The second program uses a random policy in each episode. For each episode, initialize a 4 by 2 matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; with random numbers from a uniform distribution on &lt;span class=&quot;math-container&quot;&gt;$[0,1)$&lt;/span&gt; that maps state observations to action values. Then choose the action with higher value in each step. The average reward over 10000 episodes is 46.8291.&lt;/p&gt;&#xA;&lt;p&gt;The linear mapping given by &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; covers only a portion of the search space so it seems the way actions are selected in the first program is more &amp;quot;random&amp;quot; than the second program. How can we go about explaining the huge discrepancy in the average rewards obtained by the two methods?&lt;/p&gt;&#xA;" OwnerUserId="43579" LastActivityDate="2021-01-16T08:02:21.720" Title="Difference in average rewards between taking random actions and following random policies" Tags="&lt;reinforcement-learning&gt;&lt;gym&gt;" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25793" PostTypeId="1" CreationDate="2021-01-16T08:52:57.457" Score="2" ViewCount="42" Body="&lt;p&gt;I noticed that there are many studies in recent years on how to train/update neural networks faster/quicker with equal or better performance. I find the following methods(except the chips arms race):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;using few-shot learning, for instance, pre-taining and etc.&lt;/li&gt;&#xA;&lt;li&gt;using the &lt;a href=&quot;https://medium.com/appanion/the-minimum-viable-data-set-5deb45524726&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimum viable dataset&lt;/a&gt;, for instance using &lt;a href=&quot;https://bmsce.ac.in/conferences/CS/PID5919853.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(guided) progressive sampling&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;model compression, for instance, &lt;a href=&quot;https://arxiv.org/pdf/2009.06732.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;efficent transformers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ai.googleblog.com/2020/05/speeding-up-neural-network-training.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data echoing&lt;/a&gt;, or simply put let the data pass multiple times in the graph(or GPU)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is there a systematic structure on this topic and how can we update or train a model faster without loss of its capacity?&lt;/p&gt;&#xA;" OwnerUserId="5351" LastEditorUserId="5351" LastEditDate="2021-01-23T09:59:25.830" LastActivityDate="2021-01-23T09:59:25.830" Title="How to train/update neural networks faster without a decrease in performance?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;training&gt;&lt;efficiency&gt;" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25794" PostTypeId="1" CreationDate="2021-01-16T09:46:00.877" Score="0" ViewCount="15" Body="&lt;p&gt;I have a huge dataset where I have a tensor with &lt;code&gt;535&lt;/code&gt; channels but varying spatial dimension (but always a square) it can vary from &lt;code&gt;100X100&lt;/code&gt; to &lt;code&gt;700X700&lt;/code&gt;. What I wish to predict is a sort of a binary map with the same spatial resolution, so some sort of segmentation task but not classically.&lt;/p&gt;&#xA;&lt;p&gt;I feed the network with a batch size of 1 so that I would be able to train the model but I think that the way I pad the tensors to keep the dimension constant might be problematic because some times the &amp;quot;1&amp;quot; pixel are at the edges and when I pad the tensor I lost information (because I first decreased the size and then just pad randomly)&lt;/p&gt;&#xA;&lt;p&gt;My network implementation is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch.nn as nn&#xA;import torch.nn.functional as F&#xA;&#xA;&#xA;def pad_tensor(source, target):&#xA;    diff_y = target.size()[2] - source.size()[2]&#xA;    diff_x = target.size()[3] - source.size()[3]&#xA;&#xA;    source = F.pad(input=source,&#xA;                   pad=[diff_x // 2, diff_x - diff_x // 2, diff_y // 2, diff_y - diff_y // 2],&#xA;                   mode='circular')&#xA;    return source&#xA;&#xA;&#xA;class ResidualBlock(nn.Module):&#xA;    def __init__(self, in_c, mid_c, out_c, dropout_prob, dilation_val):&#xA;        super().__init__()&#xA;        self.act = nn.ELU(alpha=1.)&#xA;&#xA;        block = [&#xA;            nn.Conv2d(in_channels=in_c, out_channels=mid_c,&#xA;                      kernel_size=3, padding_mode='circular', dilation=dilation_val),&#xA;            nn.InstanceNorm2d(num_features=mid_c),&#xA;            nn.ELU(alpha=1.),&#xA;            nn.Dropout(p=dropout_prob),&#xA;            nn.Conv2d(in_channels=mid_c, out_channels=out_c,&#xA;                      kernel_size=3, padding_mode='circular', dilation=dilation_val),&#xA;            nn.InstanceNorm2d(num_features=out_c)&#xA;        ]&#xA;&#xA;        self.block = nn.Sequential(*block)&#xA;        return&#xA;&#xA;    def forward(self, x):&#xA;        res = self.block(x)&#xA;        res = pad_tensor(res, x)&#xA;        return self.act(res + x)&#xA;&#xA;class Net(nn.Module):&#xA;    def __init__(self, in_c, out_c, num_blocks, dropout_p):&#xA;        super().__init__()&#xA;        num_f = 64&#xA;&#xA;        head = [&#xA;            nn.Conv2d(in_channels=in_c, out_channels=num_f,&#xA;                      kernel_size=1, padding_mode='circular'),&#xA;            nn.InstanceNorm2d(num_features=num_f),&#xA;            nn.ELU(alpha=1.)&#xA;        ]&#xA;        self.head = nn.Sequential(*head)&#xA;&#xA;        main_block = []&#xA;        dilation = 1&#xA;        for _ in range(num_blocks):&#xA;            main_block.append(ResidualBlock(in_c=num_f, mid_c=num_f, out_c=num_f,&#xA;                                            dropout_prob=dropout_p, dilation_val=dilation))&#xA;            dilation *= 2&#xA;            if dilation &amp;gt; 16:&#xA;                dilation = 1&#xA;        self.main_block = nn.Sequential(*main_block)&#xA;&#xA;        final = [&#xA;            nn.Conv2d(in_channels=num_f, out_channels=out_c,&#xA;                      kernel_size=1, padding_mode='circular'),&#xA;            nn.Sigmoid()&#xA;        ]&#xA;        self.final = nn.Sequential(*final)&#xA;        return&#xA;&#xA;    def forward(self, x):&#xA;        res = self.head(x)&#xA;        res = self.main_block(res)&#xA;        return self.final(res)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a network that performs convolution and doesn't affect the spatial dimensions at all?&lt;/p&gt;&#xA;&lt;p&gt;I was also thinking about an encoder-decoder network but I wasn't sure I be able to keep the spatial dimensions properly.&lt;/p&gt;&#xA;" OwnerUserId="41293" LastActivityDate="2021-01-16T09:46:00.877" Title="feeding a NN with tensors with varying spatial dimensions" Tags="&lt;pytorch&gt;&lt;image-segmentation&gt;&lt;fully-convolutional-networks&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25795" PostTypeId="1" CreationDate="2021-01-16T10:48:14.353" Score="1" ViewCount="68" Body="&lt;p&gt;I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-class classifications.&lt;/p&gt;&#xA;&lt;p&gt;What confuses me mainly are the matrix-vector shapes and their multiplications, and their implementations in code.&lt;/p&gt;&#xA;" OwnerUserId="43270" LastActivityDate="2021-01-22T00:06:00.443" Title="How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?" Tags="&lt;deep-learning&gt;&lt;backpropagation&gt;&lt;math&gt;&lt;cross-entropy&gt;&lt;softmax&gt;" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25796" PostTypeId="1" CreationDate="2021-01-16T11:46:12.297" Score="1" ViewCount="54" Body="&lt;p&gt;What if I don't apply an activation function on some layers in a neural network. How will it affect the model?&lt;/p&gt;&#xA;&lt;p&gt;Take for instance the following code snippet:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def model(x):&#xA;    a = Conv2D(64, (3, 3))(x)                         &#xA;    x = Conv2D(64, (3, 3), activation = 'relu')(x)&#xA;    b = Conv2D(128, (3, 3))(x)&#xA;    x = Conv2D(128, (3, 3), activation = 'relu')(b)&#xA;    return x, a, b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="43870" LastEditorUserId="2444" LastEditDate="2021-01-16T22:03:01.230" LastActivityDate="2021-01-16T22:12:07.047" Title="What happens if there is no activation function in some layers of a neural network?" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;keras&gt;&lt;activation-function&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25797" PostTypeId="1" CreationDate="2021-01-16T15:33:32.847" Score="0" ViewCount="59" Body="&lt;p&gt;I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy the current input element at each time step. Moreover, I have made the length of the label the same as the length of the input sequence and no adjacent elements in the label sequence the same so that both CTC loss and categorical cross-entropy loss can be used. I found that when using categorical cross-entropy loss the model very quickly converges, whereas when using CTC loss it gets nowhere.&lt;/p&gt;&#xA;&lt;p&gt;I have &lt;a href=&quot;https://colab.research.google.com/drive/1gMqWlq80i5MA9BhOCfbpVLAg5BQSg0JO?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;uploaded by minimal example to colab&lt;/a&gt;. Does anyone know why CTC loss is not working in this case?&lt;/p&gt;&#xA;" OwnerUserId="38209" LastEditorUserId="2444" LastEditDate="2021-01-16T21:33:49.297" LastActivityDate="2021-02-16T16:03:00.470" Title="Why won't my model train with CTC loss?" Tags="&lt;neural-networks&gt;&lt;deep-learning&gt;&lt;tensorflow&gt;&lt;long-short-term-memory&gt;&lt;ctc-loss&gt;" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25799" PostTypeId="2" ParentId="5990" CreationDate="2021-01-16T19:10:30.483" Score="1" Body="&lt;p&gt;Artificial intelligence is a broad field: that's why &lt;a href=&quot;http://aima.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Artificial Intelligence: A Modern Approach&lt;/a&gt; may look a bit dense to newcomers, given that it covers many different aspects of AI, such as search, machine learning, and natural language processing.&lt;/p&gt;&#xA;&lt;p&gt;The first book in &lt;a href=&quot;https://ai.stackexchange.com/a/7658/2444&quot;&gt;this answer&lt;/a&gt; is a good book, but it focuses on evolutionary computation approaches, which are often considered part of AI too. The other books that you mention in your post also focus on subfields of AI, such as machine learning or image processing, so they do not cover all aspects of AI.&lt;/p&gt;&#xA;&lt;p&gt;If you want a good book similar to AIMA, you should definitely take a look at &lt;a href=&quot;https://courses.csail.mit.edu/6.034f/ai3/rest.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Artificial intelligence&lt;/strong&gt;&lt;/a&gt; (3rd edition, 1992) by Patrick Winston, &lt;a href=&quot;https://news.mit.edu/2019/patrick-winston-professor-obituary-0719&quot; rel=&quot;nofollow noreferrer&quot;&gt;who was a professor at MIT and also director of the AI Lab at MIT&lt;/a&gt;. You can also find his free course on Artificial Intelligence (which I highly recommend) &lt;a href=&quot;https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another similar book that attempts to give an extensive overview of the AI field is &lt;a href=&quot;https://www.sciencedirect.com/book/9781558604674/artificial-intelligence-a-new-synthesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Artificial Intelligence: A New Synthesis&lt;/strong&gt;&lt;/a&gt; (1998) by &lt;a href=&quot;https://ai.stanford.edu/%7Enilsson/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nils J. Nilsson&lt;/a&gt; (&lt;a href=&quot;https://books.google.ch/books?id=Gt7gKuzL_8AC&amp;amp;lpg=PP2&amp;amp;ots=Q-5jWM1AdJ&amp;amp;dq=Artificial%20Intelligence%3A%20A%20New%20Synthesis%20&amp;amp;lr&amp;amp;pg=PP2#v=onepage&amp;amp;q=Artificial%20Intelligence:%20A%20New%20Synthesis&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is the link to Google Books, which shows you a preview of many pages of the book), who also wrote other important books related to AI and the philosophy of AI, such as &lt;a href=&quot;https://ai.stanford.edu/%7Enilsson/QAI/qai.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;The Quest for Artificial Intelligence: A History of Ideas and Achievements&lt;/strong&gt;&lt;/a&gt; (2009), among other important contributions to the AI field, such as &lt;a href=&quot;https://ai.stanford.edu/%7Enilsson/Shakey.mp4&quot; rel=&quot;nofollow noreferrer&quot;&gt;the robot &lt;em&gt;Shakey&lt;/em&gt;&lt;/a&gt; and &lt;a href=&quot;https://apps.dtic.mil/sti/pdfs/ADA459687.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;STRIPS&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There's also &lt;a href=&quot;http://www.academia.dk/BiologiskAntropologi/Epidemiologi/DataMining/Artificial_Intelligence-A_Guide_to_Intelligent_Systems.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Artificial Intelligence A Guide to Intelligent Systems&lt;/strong&gt;&lt;/a&gt; (2nd edition, 2005) by &lt;a href=&quot;https://scholar.google.com/citations?user=XXsqjkgAAAAJ&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michael Negnevitsky&lt;/a&gt;, which covers expert systems, fuzzy systems, artificial neural networks, evolutionary computation, and hybrid systems.&lt;/p&gt;&#xA;&lt;p&gt;Another similar book (that I consulted in the past), which covers many different sub-fields or topics of AI, such as machine learning, search, planning, reasoning, and knowledge-based systems, is &lt;a href=&quot;https://artint.info/2e/html/ArtInt2e.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Artificial Intelligence: Foundations of Computational Agents&lt;/strong&gt;&lt;/a&gt; (2nd Edition) by David L. Poole and  Alan K. Mackworth.&lt;/p&gt;&#xA;&lt;p&gt;Just to give you more evidence that these are important books, apart from the last one, these are all available at my university library.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-28T00:17:59.187" LastActivityDate="2021-01-28T00:17:59.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25800" PostTypeId="5" CreationDate="2021-01-16T19:20:22.070" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-16T19:20:22.070" LastActivityDate="2021-01-16T19:20:22.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25801" PostTypeId="4" CreationDate="2021-01-16T19:20:22.070" Score="0" Body="For questions related to books in the context of artificial intelligence. For example, if you're looking for a reference AI book, you may use this tag. If you want someone to clarify something in an AI book, you can use this tag." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-16T19:20:22.070" LastActivityDate="2021-01-16T19:20:22.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25802" PostTypeId="2" ParentId="25796" CreationDate="2021-01-16T22:12:07.047" Score="1" Body="&lt;p&gt;If you do not specify an activation for a layer you are effectively creating a linear transformation through that layer.  From the documentation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;activation:&lt;/strong&gt; Activation function to use. If you don't specify anything, no activation is applied (see keras.activations).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30426" LastActivityDate="2021-01-16T22:12:07.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25803" PostTypeId="1" CreationDate="2021-01-16T23:14:50.633" Score="0" ViewCount="19" Body="&lt;p&gt;Suppose I want to design a neural network to choose one of several mutually exclusive options. This may normally be done via logistic regression, where the input of the network would be a &lt;em&gt;[batch x feature]&lt;/em&gt; tensor, and the output would be a &lt;em&gt;[batch x options]&lt;/em&gt; tensor passed through a &lt;em&gt;[softmax]&lt;/em&gt; activation across the second axis.&lt;/p&gt;&#xA;&lt;p&gt;However, for my problem, the number of options to choose from is arbitrary in size. Each option would have a vector of features associated with it. I have a neural network architecture I think may work this problem, but I am not sure.&lt;/p&gt;&#xA;&lt;p&gt;The input would be an &lt;em&gt;[options x features]&lt;/em&gt; tensor; the number of options available would correspond to the number of choices available, and the feature dimension would correspond to the features of a particular option. The output would be an &lt;em&gt;[options x 1]&lt;/em&gt; tensor, passed through a &lt;em&gt;[softmax]&lt;/em&gt; layer across the first axis; this wold enforce that the sum of the probabilities of each option would be 1. Since neural network frameworks like tensorflow and pytorch allow the batch axis to be arbitrary in size, this network could input and output an arbitrary number of options.&lt;/p&gt;&#xA;&lt;p&gt;The network would still be trained with the categorical cross entropy loss function, except it would be applied over the first axis rather than the second.&lt;/p&gt;&#xA;&lt;p&gt;Would this work? Is there another architecture that would achieve something similar?&lt;/p&gt;&#xA;" OwnerUserId="38448" LastActivityDate="2021-01-16T23:14:50.633" Title="Architecture for Logistic Regression with Arbitrary Number of Options" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;logistic-regression&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25804" PostTypeId="1" AcceptedAnswerId="25806" CreationDate="2021-01-16T23:52:51.263" Score="2" ViewCount="66" Body="&lt;p&gt;Why is neural networks being a deterministic mapping not always considered a good thing?&lt;/p&gt;&#xA;&lt;p&gt;So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my conclusion is that often times neural networks are used to model things in reality, which often time do have some stochasticity and since neural networks are deterministic if they are not trained on enough examples of the possible variance inputs in relation to outputs can have they cannot generalize well. Are there other reasons this is not a good thing?&lt;/p&gt;&#xA;" OwnerUserId="30885" LastEditorUserId="2444" LastEditDate="2021-01-17T00:23:59.397" LastActivityDate="2021-01-17T11:51:08.703" Title="Why is neural networks being a deterministic mapping not always considered a good thing?" Tags="&lt;neural-networks&gt;&lt;bayesian-deep-learning&gt;&lt;bayesian-neural-networks&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25805" PostTypeId="2" ParentId="25775" CreationDate="2021-01-17T00:30:58.750" Score="4" Body="&lt;p&gt;There are a few possible approaches to deploying a ML model to a microcontroller.&lt;/p&gt;&#xA;&lt;p&gt;The main limiting factor to deployment on microcontollers is that ML models are usually a representation of a set of &lt;em&gt;parameters&lt;/em&gt; that are intended to be used as input to a &lt;em&gt;prediction algorithm&lt;/em&gt; alongside a new datapoint. Most such models assume the presence of an accompanying &lt;em&gt;library&lt;/em&gt; that implements the algorithm in question. However, a microcrontoller may use an exotic chip architecture, or have very several or unusual resource constraints that prevent these standard libraries from being deployed easily.&lt;/p&gt;&#xA;&lt;p&gt;Presumably you will already have some way to get input into your microcontroller and to program it in order to call some function that you can write. If not, you will need to first figure out how to do that, and the right methods depend on your microcontroller. A common approach is to write assembly code or code in a very limited subset of C or another language. An alternative is to find a distribution of an interpreter for another language (e.g. Java, Python) that has been compiled to work on your chip. Either way, you will need some way to program the chip.&lt;/p&gt;&#xA;&lt;p&gt;Presuming you can program the chip, you have two fundamental challenges in deploying the model:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Most models are trained with very wide floating point numbers for their parameters. For example, 128-bit numbers may be used. On a standard computing environment, the CPU or GPU will be equipped to perform operations on wide datatypes efficiently. On a microcontroller, you may be limited to 8-bit or 16-bit integers. To work with your model parameters in an environment like this, you will need to either make the parameters smaller (usually by rounding them to fit in a much smaller numeric format, a process called &amp;quot;quantization&amp;quot;), or by finding or writing software that can simulate the operations you want (probably addition and multiplication) on a large datatype that is represented as a collection of smaller datatypes. The first approach may make the model perform poorly. The second may make model prediction very slow.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You need an implementation of the algorithm. Some algorithms like linear regression, linear discriminant analysis, or even decision trees are extremely easy to implement prediction for, and may require only addition, multiplication, and/or comparison. You might be able to write these yourself in a simple subset of C, or even in assembly (for example, prediction with linear regression should be just a simple loop). Other algorithms, like deep neural networks, may contain more complex operations, and may contain many such operations performed in complicated sequences. For these, you generally will need to find an distribution of a library that implements the algorithms, or compile one yourself. Compiling one yourself will require setting up a build toolchain for your specific microcontroller, and can be quite involved.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2021-01-17T00:30:58.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25806" PostTypeId="2" ParentId="25804" CreationDate="2021-01-17T01:15:02.113" Score="1" Body="&lt;p&gt;Your intuition is right. The main reason why a deterministic function can be undesirable (or even dangerous, as I will explain below with an example) is that we may not have enough data to learn the correct function, so we may end up learning the incorrect one. Right now, no other reason, from a theoretical point of view, comes to my mind, but below I will mention a few applications/cases where a deterministic function may not be desirable.&lt;/p&gt;&#xA;&lt;p&gt;If we had all data pairs &lt;span class=&quot;math-container&quot;&gt;$\{(x_i, y_i)\}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x_i \in \mathcal{X}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i = \mathcal{Y}$&lt;/span&gt; are, respectively, an input and output from the unknown function that you want to learn &lt;span class=&quot;math-container&quot;&gt;$f: \mathcal{X} \rightarrow \mathcal{Y}$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$f(x_i) = y_i$&lt;/span&gt;, then you could reconstruct &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;: whenever &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is given, you just need to return &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Of course, in reality, we almost never have a large enough (training) dataset to approximate out desired (but usually unknown) function. If we learn only one (deterministic) function, then, in principle, you can catastrophically fail, i.e. your approximation of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, denoted as &lt;span class=&quot;math-container&quot;&gt;$f_\theta$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are the parameters of the neural network or any other model), can produce outputs that are completely wrong.&lt;/p&gt;&#xA;&lt;p&gt;Let me try to give you a simple example. Let's say that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is defined as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f: \mathbb{N} \rightarrow \{0, 1\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You are given a training labelled dataset &lt;span class=&quot;math-container&quot;&gt;$$D = \{(4, 1), (11, 0), (8, 1), (31, 0), (16, 1), (7, 0) \}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Apparently&lt;/strong&gt;, our unknown function is defined as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;h_1(x)=&#xA;\begin{cases}&#xA;1, &amp;amp;x \text{ mod } 2 \equiv 0\\&#xA;0, &amp;amp;\text{otherwise}&#xA;\end{cases}\tag{1}\label{1}&#xA;\end{align}&lt;/span&gt;&#xA;Given that &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is small, your neural network, &lt;span class=&quot;math-container&quot;&gt;$f_\theta$&lt;/span&gt;, can easily &lt;strong&gt;overfit&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, i.e. learn to output &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is even and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; otherwise.&lt;/p&gt;&#xA;&lt;p&gt;However, what if &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is not that function in equation \ref{1} and we collected just a dataset that doesn't represent &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; well enough? If you look at &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; more carefully, you will see that another possible hypothesis for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the following&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;h_2(x)=&#xA;\begin{cases}&#xA;1, &amp;amp;x \text{ mod } 4 \equiv 0\\&#xA;0, &amp;amp;\text{otherwise}&#xA;\end{cases}\tag{2}\label{2}&#xA;\end{align}&lt;/span&gt;&#xA;However, given that your neural network can only compute one of these functions at a time, it could compute the wrong one. Let's say that &lt;span class=&quot;math-container&quot;&gt;$f_\theta \approx h_1$&lt;/span&gt;, then it should produce &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$x = 6$&lt;/span&gt; (an even number). If the correct unknown function was &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$f = h_2$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$f_\theta(6) = 1$&lt;/span&gt; would be wrong (because &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt; is not a multiple of &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Of course, this is just a toy example. However, there are many other cases where this can happen, which may not be desirable, such as &lt;strong&gt;healthcare&lt;/strong&gt;, &lt;strong&gt;medicine&lt;/strong&gt; or &lt;strong&gt;self-driving cars&lt;/strong&gt;, where the wrong prediction can lead to catastrophic consequences, such as the death of a person.&lt;/p&gt;&#xA;&lt;p&gt;If we maintain a probability distribution over the possible functions that are consistent with the observed data so far, we can (partially) avoid this issue. So, continuing with the example above, this probability distribution over functions should be highly uncertain about &lt;span class=&quot;math-container&quot;&gt;$x = 6$&lt;/span&gt;, whether it produces &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, because it has never seen the label for &lt;span class=&quot;math-container&quot;&gt;$x=6$&lt;/span&gt;, so a medical doctor or the human driver could intervene in the case of (high) uncertainty.&lt;/p&gt;&#xA;&lt;p&gt;For this reason, in the last decade, people have started to incorporate uncertainty estimation in neural networks. Neural networks that model uncertainty (to some degree) are often called &lt;strong&gt;Bayesian neural networks&lt;/strong&gt; (BNNs), and there are different approaches (such as &lt;a href=&quot;http://proceedings.mlr.press/v37/blundell15.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;variational BNNs&lt;/a&gt;, &lt;a href=&quot;https://arxiv.org/abs/1506.02142&quot; rel=&quot;nofollow noreferrer&quot;&gt;MC dropout&lt;/a&gt; or &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.446.9306&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo-based approaches&lt;/a&gt;). If you are interested in this topic, the paper &lt;a href=&quot;http://proceedings.mlr.press/v37/blundell15.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weight Uncertainty in Neural Network&lt;/a&gt; (2015) is a good start, especially if you are already familiar with VAEs. Given that this is a very new research area, the current solutions are still not very satisfactory. For example, you can find examples in the literature that report that MC dropout can produce very bad estimates of uncertainty (even in my master's thesis I have observed and thus concluded that this is the case), i.e. they can be highly certain when they should be highly uncertain.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T11:51:08.703" LastActivityDate="2021-01-17T11:51:08.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25808" PostTypeId="1" AcceptedAnswerId="25809" CreationDate="2021-01-17T05:05:26.677" Score="2" ViewCount="95" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bIZrp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bIZrp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't understand how the formula in the red circle is derived. The screenshot is taken from &lt;a href=&quot;http://www.marcgbellemare.info/static/publications/harutyunyan16qlambda.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43880" LastEditorUserId="2444" LastEditDate="2021-01-19T21:12:56.047" LastActivityDate="2021-01-19T21:12:56.047" Title="How to prove the formula of eligibility traces operator in reinforcement learning?" Tags="&lt;reinforcement-learning&gt;&lt;proofs&gt;&lt;bellman-equations&gt;&lt;eligibility-traces&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25809" PostTypeId="2" ParentId="25808" CreationDate="2021-01-17T11:48:51.233" Score="1" Body="&lt;p&gt;I will refer to &lt;span class=&quot;math-container&quot;&gt;$\mathcal T^{\pi} $&lt;/span&gt;as &lt;span class=&quot;math-container&quot;&gt;$\mathcal T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P^{\pi}$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; for notational simplicity&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;(\mathcal{T})^{n+1} Q &amp;amp;= \mathcal{T}(\mathcal{T}(...(\mathcal{T}(Q))))\\&#xA;&amp;amp;= r + \gamma P(r + \gamma P(...(r + \gamma P Q)))\\&#xA;&amp;amp;= r + r\sum_{i=1}^{n} \gamma^i P^i + \gamma^{n+1} P^{n+1} Q&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathcal{T}_{\lambda}Q &amp;amp;= (1-\lambda) \sum_{n=0}^{\infty} \lambda^n (\mathcal{T})^{n+1} Q\\&#xA;&amp;amp;=(1-\lambda)\{\lambda^0 (\mathcal T)^1Q + \lambda^1 (\mathcal T)^2Q + \lambda^2 (\mathcal T)^3Q + \ldots   \}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;when you plug in expression for &lt;span class=&quot;math-container&quot;&gt;$(\mathcal T)^i Q$&lt;/span&gt; inside this sum and rearrange you get 3 sums&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mathcal{T}_{\lambda}Q = (1-\lambda) \sum_{n=0}^{\infty} \lambda^n r + (1-\lambda)\sum_{n=1}^{\infty} \lambda^n \gamma^n P^n r + (1-\lambda)\sum_{n=0}^{\infty} \lambda^n \gamma^{n+1} P^{n+1} Q&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;sum:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;(1-\lambda) \sum_{n=0}^{\infty} \lambda^n r = r&#xA;\end{equation}&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;sum:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;(1-\lambda)\sum_{n=1}^{k} \lambda^n \gamma^n P^n r = (1-\lambda\gamma P)^{-1}(1 - \lambda^k \gamma^k P^k)\lambda\gamma P r&#xA;\end{equation}&lt;/span&gt;&#xA;As &lt;span class=&quot;math-container&quot;&gt;$k \rightarrow \infty$&lt;/span&gt; and since &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt; this is in the limit equal to&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;(1-\lambda)\sum_{n=1}^{\infty} \lambda^n \gamma^n P^n r = (1-\lambda\gamma P)^{-1}\lambda\gamma P r&#xA;\end{equation}&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;sum:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;(1-\lambda)\sum_{n=0}^{\infty} \lambda^n \gamma^{n+1} P^{n+1} Q = (1 - \gamma\lambda P)^{-1}(1-\lambda)\gamma P Q &#xA;\end{equation}&lt;/span&gt;&#xA;If you combine all 3 you get&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathcal{T}_{\lambda}Q &amp;amp;= r + (1-\lambda\gamma P)^{-1}\lambda\gamma P r + (1 - \gamma\lambda P)^{-1}(1-\lambda)\gamma P Q\\&#xA;&amp;amp;= r+ (1-\lambda\gamma P)^{-1}(\lambda \gamma P r + \gamma PQ - \lambda\gamma PQ)\\&#xA;&amp;amp;= r+ (1-\lambda\gamma P)^{-1}(\lambda \gamma P r + (\mathcal T)Q - r - \lambda\gamma PQ)\\&#xA;&amp;amp;= (1-\lambda\gamma P)^{-1}(r - \lambda \gamma P r + \lambda \gamma P r + (\mathcal T)Q - r - \lambda\gamma PQ)\\&#xA;&amp;amp;= (1-\lambda\gamma P)^{-1}( (\mathcal T)Q  - \lambda\gamma PQ + Q - Q)\\&#xA;&amp;amp;= Q + (1-\lambda\gamma P)^{-1}( (\mathcal T)Q  - Q)&#xA;\end{align}&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2021-01-17T11:48:51.233" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25810" PostTypeId="1" CreationDate="2021-01-17T12:36:25.197" Score="0" ViewCount="15" Body="&lt;p&gt;I am currently working on a music-generation project, inspired by an already existing project called Deepbach. My dataset are the Bach chorales, which are all composed of 4 independent (but related) voices (soprano, alto, tenor, bass). A voice is encoded as a sequence of notes and rests, in the form of a string. Each choral is therefore a list of 4 strings. My goal is to train a model that generates a choral, ie a list of 4 strings.&lt;/p&gt;&#xA;&lt;p&gt;For instance a voice is encoded as &amp;quot;A4,&lt;em&gt;,&lt;/em&gt;,&lt;em&gt;,B-4,&lt;/em&gt;,&lt;em&gt;,&lt;/em&gt;,A4,&lt;em&gt;,&lt;/em&gt;,&lt;em&gt;,G4,&lt;/em&gt;,&lt;em&gt;,&lt;/em&gt;,D5,&lt;em&gt;,&lt;/em&gt;,&lt;em&gt;,D5,&lt;/em&gt;,C5,&lt;em&gt;,B-4,&lt;/em&gt;,&lt;em&gt;,&lt;/em&gt;,A4,&lt;em&gt;,&lt;/em&gt;,&lt;em&gt;,C5,&lt;/em&gt;,&lt;em&gt;,&lt;/em&gt;,B-4,&lt;em&gt;,&lt;/em&gt;,&lt;em&gt;,A4,&lt;/em&gt;,&lt;em&gt;,&lt;/em&gt;,G4,&lt;em&gt;,&lt;/em&gt;,_,&amp;quot;... and a choral is a list of 4 strings like that.&lt;/p&gt;&#xA;&lt;p&gt;For now, I've managed to make a model with LSTMs that trains on all the soprano voices and generates a new soprano voice. Now I would like to &amp;quot;combine&amp;quot; 4 of those models such that it generates the 4 voices. However my issue is the following :&#xA;my implementation outputs the probability of a note given the previous and further notes (ie the context) in the same voice. What I want is to get the probability of a note in a voice given the context of the 4 other voices, but I have no idea how to to it... any ideas?&lt;/p&gt;&#xA;" OwnerUserId="43883" LastActivityDate="2021-01-17T12:36:25.197" Title="Text generation with LSTM with multiple correlated inputs" Tags="&lt;natural-language-processing&gt;&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25811" PostTypeId="2" ParentId="25775" CreationDate="2021-01-17T12:53:23.943" Score="2" Body="&lt;p&gt;If the library running the model can be compiled for your microcontroller,&#xA;then you can run your model on that microcontroller.&lt;/p&gt;&#xA;&lt;p&gt;If you train using one library and deploy using another library, you possible can convert your model to that library: &lt;a href=&quot;https://en.wikipedia.org/wiki/Open_Neural_Network_Exchange&quot; rel=&quot;nofollow noreferrer&quot;&gt;ONNX&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Some library links on &lt;a href=&quot;https://en.wikipedia.org/wiki/Edge_computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edge Computing&lt;/a&gt; in &lt;a href=&quot;https://en.wikipedia.org/wiki/Machine_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;ML&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tensorflow.org/lite/microcontrollers?hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microcontroller support for Tensorflow Lite&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Microsoft/EdgeML&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edge ML&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/pytorch/pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTorch&lt;/a&gt; supports android&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/scikit-learn/scikit-learn&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-learn&lt;/a&gt; uses only python libraries. &lt;a href=&quot;https://github.com/v923z/micropython-ulab&quot; rel=&quot;nofollow noreferrer&quot;&gt;ulab&lt;/a&gt; partly replaces numpy for  &lt;a href=&quot;https://github.com/micropython/micropython&quot; rel=&quot;nofollow noreferrer&quot;&gt;micropython&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To speed things up specific hardware is used: GPU's of course, but also &lt;a href=&quot;https://en.wikipedia.org/wiki/Tensor_Processing_Unit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensor Processing Units&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One can manually transfer an NN trained model to C code&#xA;(&lt;a href=&quot;https://arxiv.org/pdf/2007.01348.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;),&#xA;but there are also compilers:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.tensorflow.org/xla?hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;XLA&lt;/a&gt; of tensorflow&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/pytorch/glow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Glow&lt;/a&gt; of pytorch&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/iml130/nncg&quot; rel=&quot;nofollow noreferrer&quot;&gt;NNCG&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;XLA and Glow support x86-64 and ARM64.&#xA;The NNCG approach generates C code and thus should be more readily portable to general MCU's (&lt;a href=&quot;https://arxiv.org/pdf/2001.05572.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Further links: &lt;a href=&quot;https://github.com/crespum/edge-ai&quot; rel=&quot;nofollow noreferrer&quot;&gt;adge-ai&lt;/a&gt;,&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/36720498/convert-keras-model-to-c&quot;&gt;stackoverlow question&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43882" LastEditorUserId="43882" LastEditDate="2021-01-18T16:28:29.570" LastActivityDate="2021-01-18T16:28:29.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25812" PostTypeId="2" ParentId="20405" CreationDate="2021-01-17T13:34:52.693" Score="3" Body="&lt;p&gt;The question in &lt;a href=&quot;https://www.youtube.com/watch?v=kaahx4hMxmw&amp;amp;ab_channel=HelixNebula&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are you real?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does this question really mean? Is the guy asking whether the &lt;em&gt;apparent&lt;/em&gt; female (I don't know if she is a cyborg or not because I did not yet watch the TV series) is a human? So, is &amp;quot;real&amp;quot; a synonym for &amp;quot;human&amp;quot;? If that's the case, then the first implication (in the form of a question) of the statement&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If you can't tell, does it matter?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;in relation to AI is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can we create an AGI that is sufficiently similar to a human that we can't tell whether it's a human or not (by just normally interacting with it)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Of course, it's not clear what we mean by &amp;quot;normally interacting&amp;quot;. As far as I remember, this issue is also raised in the film &lt;a href=&quot;https://en.wikipedia.org/wiki/A.I._Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;A.I. Artificial Intelligence&lt;/a&gt;, where the AI (the kid interpreted by Haley Joel Osment) looks sufficiently real to the other kid, so he behaves as if he was a human kid, but then the human kid understands that he's a machine, and starts to behave differently (I hope I'm remembering the film correctly).&lt;/p&gt;&#xA;&lt;p&gt;So, the second question that we could ask is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Once we understand that it's not a human (for example, because it's made of other substances), would we humans start to behave differently and start treating the AGI differently?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As opposed to the first question, which is still an open problem, this second question can probably be answered by looking at our relationships with other humans (or entities, such as other animals). Often, we have an idea of a person. Once we discover something new about that person, which maybe we dislike, we may start to treat that person differently. I think this would very likely also happen in our eventual relationship with a sufficiently advanced AGI too, as depicted in &lt;a href=&quot;https://en.wikipedia.org/wiki/A.I._Artificial_Intelligence&quot; rel=&quot;nofollow noreferrer&quot;&gt;the mentioned film&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, let me try to address the other question&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are the implications of this statement in relation to experience, in relation to the self?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think that you're asking whether a sufficiently advanced AGI could be considered conscious or not. Of course, this is a very hard question to answer, because &lt;a href=&quot;https://plato.stanford.edu/entries/consciousness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;we still don't have a clear definition of consciousness or we don't yet agree on a standard definition&lt;/a&gt;, so I don't really have a definitive answer to this question. However, if consciousness is just a byproduct of perception and the ability to understand the world and its (physical) rules, then an AGI could be conscious (in a similar way that humans are also conscious). However, consciousness may not actually be necessary to correctly act in the world. In any case, the AI probably needs to know that it has a body and that it needs to protect it for its survival, if that's its main goal.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T13:44:46.330" LastActivityDate="2021-01-17T13:44:46.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25814" PostTypeId="1" CreationDate="2021-01-17T15:15:52.967" Score="2" ViewCount="65" Body="&lt;p&gt;Is there any current research on Gödel machines? It seems that the last article by Jürgen Schmidhuber on this topic was published in 2012: &lt;a href=&quot;http://people.idsia.ch/%7Ejuergen/goedelmachine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://people.idsia.ch/~juergen/goedelmachine.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="41482" LastEditorUserId="2444" LastEditDate="2021-01-17T15:33:44.393" LastActivityDate="2021-02-16T18:08:00.720" Title="Current research on Gödel machines" Tags="&lt;reference-request&gt;&lt;agi&gt;&lt;problem-solving&gt;&lt;godel-machine&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25815" PostTypeId="5" CreationDate="2021-01-17T15:28:43.753" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T15:28:43.753" LastActivityDate="2021-01-17T15:28:43.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25816" PostTypeId="4" CreationDate="2021-01-17T15:28:43.753" Score="0" Body="For questions related to the Gödel machine (GM), proposed in &quot;Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements&quot; (2003) by Jürgen Schmidhuber. As described in this paper, the GM is a general, self-referential, self-improving problem solver." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T15:28:43.753" LastActivityDate="2021-01-17T15:28:43.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25817" PostTypeId="2" ParentId="25797" CreationDate="2021-01-17T15:40:05.830" Score="0" Body="&lt;p&gt;The problem was the dimensions of the &lt;code&gt;logit_length&lt;/code&gt; argument to &lt;code&gt;tf.nn.ctc_loss&lt;/code&gt; was incorrect.&lt;/p&gt;&#xA;&lt;p&gt;It was this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tf.repeat(tf.shape(y_pred)[-1], tf.shape(y_pred)[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it should have been&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tf.repeat(tf.shape(y_pred)[-2], tf.shape(y_pred)[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="38209" LastActivityDate="2021-01-17T15:40:05.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25818" PostTypeId="2" ParentId="25814" CreationDate="2021-01-17T15:48:41.667" Score="0" Body="&lt;p&gt;This paper &lt;a href=&quot;https://arxiv.org/pdf/1806.02091.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Can Machines Design? An Artificial General Intelligence Approach&lt;/a&gt; (2018, presented at &lt;a href=&quot;http://agi-conference.org/2018/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGI-18&lt;/a&gt; and published in the related proceedings &lt;a href=&quot;https://www.springer.com/gp/book/9783319976754&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), which proposes the &lt;strong&gt;design Gödel machine&lt;/strong&gt;, may be useful to you.&lt;/p&gt;&#xA;&lt;p&gt;After a quick search, I have not found other relevant papers, so I suppose that the research on GMs is currently not very active.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-17T16:11:11.423" LastActivityDate="2021-01-17T16:11:11.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25819" PostTypeId="1" AcceptedAnswerId="26371" CreationDate="2021-01-17T17:31:29.747" Score="1" ViewCount="32" Body="&lt;p&gt;The validation accuracy of my 1D CNN is stuck on 0.5 and that's because I'm always getting the same prediction out of a balanced data set. At the same time my training accuracy keeps increasing and the loss decreasing as intended.&lt;/p&gt;&#xA;&lt;p&gt;Strangely, if I do &lt;code&gt;model.evaluate()&lt;/code&gt; on my training set (that has close to 1 accuracy in the last epoch), the accuracy will also be 0.5. How can the accuracy here differ so much from the training accuracy of the last epoch? I've also tried with a batch size of 1 for both training and evaluating and the problem persists.&lt;/p&gt;&#xA;&lt;p&gt;Well, I've been searching for different solutions for quite some time but still no luck. Possible problems I've already looked into:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My data set is properly balanced and shuffled;&lt;/li&gt;&#xA;&lt;li&gt;My labels are correct;&lt;/li&gt;&#xA;&lt;li&gt;Tried adding fully connected layers;&lt;/li&gt;&#xA;&lt;li&gt;Tried adding/removing dropout from the fully connected layers;&lt;/li&gt;&#xA;&lt;li&gt;Tried the same architecture, but with the last layer with 1 neuron and sigmoid activation;&lt;/li&gt;&#xA;&lt;li&gt;Tried changing the learning rates (went down to 0.0001 but still the same problem).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here's my code:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pathlib&#xA;import numpy as np&#xA;import ipynb.fs.defs.preprocessDataset as preprocessDataset&#xA;import pickle&#xA;import tensorflow as tf&#xA;from tensorflow.keras.models import Sequential&#xA;from tensorflow.keras import Input&#xA;from tensorflow.keras.layers import Conv1D, BatchNormalization, Activation, MaxPooling1D, Flatten, Dropout, Dense&#xA;from tensorflow.keras.optimizers import SGD&#xA;&#xA;main_folder = pathlib.Path.cwd().parent&#xA;datasetsFolder=f'{main_folder}\\datasets'&#xA;trainDataset = preprocessDataset.loadDataset('DatasetTime_Sg12p5_Ov75_Train',datasetsFolder)&#xA;testDataset = preprocessDataset.loadDataset('DatasetTime_Sg12p5_Ov75_Test',datasetsFolder)&#xA;&#xA;X_train,Y_train,Names_train=trainDataset[0],trainDataset[1],trainDataset[2]&#xA;X_test,Y_test,Names_test=testDataset[0],testDataset[1],testDataset[2]&#xA;&#xA;model = Sequential()&#xA;&#xA;model.add(Input(shape=X_train.shape[1:]))&#xA;&#xA;model.add(Conv1D(16, 61, strides=1, padding=&amp;quot;same&amp;quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Activation('relu'))&#xA;model.add(MaxPooling1D(2, strides=2, padding=&amp;quot;valid&amp;quot;))&#xA;&#xA;model.add(Conv1D(32, 3, strides=1, padding=&amp;quot;same&amp;quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Activation('relu'))&#xA;model.add(MaxPooling1D(2, strides=2, padding=&amp;quot;valid&amp;quot;))&#xA;&#xA;model.add(Conv1D(64, 3, strides=1, padding=&amp;quot;same&amp;quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Activation('relu'))&#xA;model.add(MaxPooling1D(2, strides=2, padding=&amp;quot;valid&amp;quot;))&#xA;&#xA;model.add(Conv1D(64, 3, strides=1, padding=&amp;quot;same&amp;quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Activation('relu'))&#xA;model.add(MaxPooling1D(2, strides=2, padding=&amp;quot;valid&amp;quot;))&#xA;&#xA;model.add(Conv1D(64, 3, strides=1, padding=&amp;quot;same&amp;quot;))&#xA;model.add(BatchNormalization())&#xA;model.add(Activation('relu'))&#xA;model.add(Flatten())&#xA;model.add(Dropout(0.5))&#xA;&#xA;model.add(Dense(200))&#xA;model.add(Activation('relu'))&#xA;&#xA;model.add(Dense(2))&#xA;model.add(Activation('softmax'))&#xA;&#xA;opt = SGD(learning_rate=0.01)&#xA;&#xA;model.compile(loss='binary_crossentropy',optimizer=opt,metrics=['accuracy'])&#xA;&#xA;model.summary()&#xA;&#xA;model.fit(X_train,Y_train,epochs=10,shuffle=False,validation_data=(X_test, Y_test))&#xA;&#xA;model.evaluate(X_train,Y_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here's model.fit():&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.fit(X_train,Y_train,epochs=10,shuffle=False,validation_data=(X_test, Y_test))&#xA;&#xA;Epoch 1/10&#xA;914/914 [==============================] - 277s 300ms/step - loss: 0.6405 - accuracy: 0.6543 - val_loss: 7.9835 - val_accuracy: 0.5000&#xA;Epoch 2/10&#xA;914/914 [==============================] - 270s 295ms/step - loss: 0.3997 - accuracy: 0.8204 - val_loss: 19.8981 - val_accuracy: 0.5000&#xA;Epoch 3/10&#xA;914/914 [==============================] - 273s 298ms/step - loss: 0.2976 - accuracy: 0.8730 - val_loss: 1.9558 - val_accuracy: 0.5002&#xA;Epoch 4/10&#xA;914/914 [==============================] - 278s 304ms/step - loss: 0.2897 - accuracy: 0.8776 - val_loss: 20.2678 - val_accuracy: 0.5000&#xA;Epoch 5/10&#xA;914/914 [==============================] - 277s 303ms/step - loss: 0.2459 - accuracy: 0.8991 - val_loss: 5.4945 - val_accuracy: 0.5000&#xA;Epoch 6/10&#xA;914/914 [==============================] - 268s 294ms/step - loss: 0.2008 - accuracy: 0.9181 - val_loss: 32.4579 - val_accuracy: 0.5000&#xA;Epoch 7/10&#xA;914/914 [==============================] - 271s 297ms/step - loss: 0.1695 - accuracy: 0.9317 - val_loss: 14.9538 - val_accuracy: 0.5000&#xA;Epoch 8/10&#xA;914/914 [==============================] - 276s 302ms/step - loss: 0.1423 - accuracy: 0.9452 - val_loss: 1.4420 - val_accuracy: 0.4988&#xA;Epoch 9/10&#xA;914/914 [==============================] - 266s 291ms/step - loss: 0.1261 - accuracy: 0.9497 - val_loss: 4.3830 - val_accuracy: 0.5005&#xA;Epoch 10/10&#xA;914/914 [==============================] - 272s 297ms/step - loss: 0.1142 - accuracy: 0.9548 - val_loss: 1.6054 - val_accuracy: 0.5009&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here's model.evaluate():&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;model.evaluate(X_train,Y_train)&#xA;&#xA;914/914 [==============================] - 35s 37ms/step - loss: 1.7588 - accuracy: 0.5009&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here's model.summary():&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Model: &amp;quot;sequential&amp;quot;&#xA;_________________________________________________________________&#xA;Layer (type)                 Output Shape              Param #   &#xA;=================================================================&#xA;conv1d (Conv1D)              (None, 4096, 16)          992       &#xA;_________________________________________________________________&#xA;batch_normalization (BatchNo (None, 4096, 16)          64        &#xA;_________________________________________________________________&#xA;activation (Activation)      (None, 4096, 16)          0         &#xA;_________________________________________________________________&#xA;max_pooling1d (MaxPooling1D) (None, 2048, 16)          0         &#xA;_________________________________________________________________&#xA;conv1d_1 (Conv1D)            (None, 2048, 32)          1568      &#xA;_________________________________________________________________&#xA;batch_normalization_1 (Batch (None, 2048, 32)          128       &#xA;_________________________________________________________________&#xA;activation_1 (Activation)    (None, 2048, 32)          0         &#xA;_________________________________________________________________&#xA;max_pooling1d_1 (MaxPooling1 (None, 1024, 32)          0         &#xA;_________________________________________________________________&#xA;conv1d_2 (Conv1D)            (None, 1024, 64)          6208      &#xA;_________________________________________________________________&#xA;batch_normalization_2 (Batch (None, 1024, 64)          256       &#xA;_________________________________________________________________&#xA;activation_2 (Activation)    (None, 1024, 64)          0         &#xA;_________________________________________________________________&#xA;max_pooling1d_2 (MaxPooling1 (None, 512, 64)           0         &#xA;_________________________________________________________________&#xA;conv1d_3 (Conv1D)            (None, 512, 64)           12352     &#xA;_________________________________________________________________&#xA;batch_normalization_3 (Batch (None, 512, 64)           256       &#xA;_________________________________________________________________&#xA;activation_3 (Activation)    (None, 512, 64)           0         &#xA;_________________________________________________________________&#xA;max_pooling1d_3 (MaxPooling1 (None, 256, 64)           0         &#xA;_________________________________________________________________&#xA;conv1d_4 (Conv1D)            (None, 256, 64)           12352     &#xA;_________________________________________________________________&#xA;batch_normalization_4 (Batch (None, 256, 64)           256       &#xA;_________________________________________________________________&#xA;activation_4 (Activation)    (None, 256, 64)           0         &#xA;_________________________________________________________________&#xA;flatten (Flatten)            (None, 16384)             0         &#xA;_________________________________________________________________&#xA;dropout (Dropout)            (None, 16384)             0         &#xA;_________________________________________________________________&#xA;dense (Dense)                (None, 200)               3277000   &#xA;_________________________________________________________________&#xA;activation_5 (Activation)    (None, 200)               0         &#xA;_________________________________________________________________&#xA;dense_1 (Dense)              (None, 2)                 402       &#xA;_________________________________________________________________&#xA;activation_6 (Activation)    (None, 2)                 0         &#xA;=================================================================&#xA;Total params: 3,311,834&#xA;Trainable params: 3,311,354&#xA;Non-trainable params: 480&#xA;_________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="43887" LastEditorUserId="43887" LastEditDate="2021-01-17T18:46:18.640" LastActivityDate="2021-02-13T17:11:55.607" Title="Keras 1D CNN always predicts the same result even if accuracy is high on training set" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;python&gt;&lt;keras&gt;&lt;1d-convolution&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25820" PostTypeId="1" AcceptedAnswerId="25825" CreationDate="2021-01-17T18:05:27.580" Score="0" ViewCount="74" Body="&lt;p&gt;I just finished Andrew Ngs's deep learning specialization, but RL was not covered, so I don't know the basics of RL. So, I have been having trouble understanding the cost function in deep Q-learning. Like other cost functions in machine learning, you usually have &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; (the network prediction) and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (the target, or what the network is being optimized for.)&lt;/p&gt;&#xA;&lt;p&gt;I've read through a few online articles on deep Q-learning. So far, there has been no mention of setting up a target state (&lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;) for the agent to produce. There has been mention of calculating a temporal-difference, however, which is where I am confused.&lt;/p&gt;&#xA;&lt;p&gt;When calculating the cost function, are you taking the input state (&lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt;) and a target state (&lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;) into consideration to determine the temporal-difference?&lt;/p&gt;&#xA;&lt;p&gt;Otherwise, I'm not sure how the cost function could determine a reward based on the input alone (state of the environment the agent is in.).&lt;/p&gt;&#xA;" OwnerUserId="20271" LastEditorUserId="2444" LastEditDate="2021-01-18T12:18:42.703" LastActivityDate="2021-01-18T15:47:14.193" Title="When calculating the cost in deep Q-learning, do we use both the input and target states?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;loss-functions&gt;&lt;temporal-difference-methods&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25821" PostTypeId="1" CreationDate="2021-01-17T21:17:13.303" Score="0" ViewCount="50" Body="&lt;p&gt;Popular deep learning frameworks have different default data formats (PyTorch with NCHW, TensorFlow with NHWC), which lead to published pretrained models with weights that may have an incompatible data format. Many searches and a lot of disappointing finds after, I believe there is currently no simple way to convert trained weights from one data format to the other (&lt;a href=&quot;https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html#converting-existing-models&quot; rel=&quot;nofollow noreferrer&quot;&gt;only PyTorch might have something in that regard&lt;/a&gt;), but I can't understand why this is the case.&lt;/p&gt;&#xA;&lt;p&gt;Let us consider the following example (I'll use pseudo-python syntax for convenience):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;filters = np.random.random([5,5,1,10], dtype=np.float32) # convolution filters for a 2D 5x5 conv on an input with 1 channel, output has 10 channels.&#xA;&#xA;image_nhwc = np.random.random([1,28,28,1], dtype=np.float32) # one image in NHWC data format&#xA;image_nchw = np.transpose(image_nhwc.copy(), (0,3,1,2)) # same image in NCHW format&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ignoring for the moment padding issues, strides, dilations, etc, and assuming there is a &lt;code&gt;conv2d&lt;/code&gt; method that implements the simple convolution of &lt;code&gt;filters&lt;/code&gt; on &lt;code&gt;image&lt;/code&gt; (and takes the data format as a parameter), I believe there ought to be some permutation &lt;code&gt;perm&lt;/code&gt; of &lt;code&gt;filters&lt;/code&gt; such that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out_nchw = conv2d(image_nchw, filters, 'NCHW')&#xA;out_nhwc = conv2d(image_nhwc, perm(filters), 'NHWC')&#xA;&#xA;assert np.allclose(out_nhwc == np.transpose(out_nchw, (0,2,3,1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i.e, the result of the two convolutions is the same after adapting the data format of the outputs to be the same (think of it as &amp;quot;I have trained weights for the NCHW model, applying &lt;code&gt;perm&lt;/code&gt; I get the trained weights for the NHWC equivalent model&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;For how hard I think about this, however, I can't come up with what &lt;code&gt;perm&lt;/code&gt; would be.&lt;/p&gt;&#xA;&lt;p&gt;So, the question finally: Am I right in thinking that &lt;code&gt;perm&lt;/code&gt; ought to exist? If so, how does one calculate it?&lt;/p&gt;&#xA;" OwnerUserId="22086" LastEditorUserId="32410" LastEditDate="2021-01-18T06:17:30.723" LastActivityDate="2021-01-18T06:17:30.723" Title="Conversion of convolution filter weights between NCHW and NHWC formats" Tags="&lt;tensorflow&gt;&lt;pytorch&gt;&lt;convolution&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25822" PostTypeId="1" AcceptedAnswerId="26021" CreationDate="2021-01-18T08:51:19.583" Score="1" ViewCount="51" Body="&lt;p&gt;I am working on a &lt;a href=&quot;https://arxiv.org/abs/1611.07004&quot; rel=&quot;nofollow noreferrer&quot;&gt;pix2pix GAN model&lt;/a&gt; that was inspired by the code in this &lt;a href=&quot;https://github.com/eriklindernoren/Keras-GAN/blob/master/pix2pix/pix2pix.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repository&lt;/a&gt;. The original code is working and I have already customized most of the code for my needs. However, there is one part I am unable to understand.&lt;/p&gt;&#xA;&lt;p&gt;The pix2pix GAN is a conditional GAN network that takes an image as a condition and outputs a modified image - such as blurry to clear, facades to buildings, filling up cut out part of an image, etc. The combined model thus takes as input a conditional image, the discriminator compares it with the dummy matrix named valid or fake, containing 0s or 1s according to validity (0 for generated samples, 1 for real samples). The generator loss is according to similarity with real sample + discriminator. The following code corresponds to what I told:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;self.combined = Model(inputs=[img_A, img_B], outputs=[valid, fake_A])&#xA;self.combined.compile(loss=['mse', 'mae'],&#xA;                      loss_weights=[1, 100],&#xA;                      optimizer=optimizer)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The losses are thus set as MSE for discriminator output and MAE for generator. That seems to be OK, but I can not understand why the implementation uses 1 and 100 for the weights of the discriminator and generator losses, respectively, which seems to imply that the discriminator loss is 100 times lower than the loss of the generator. I couldn't find the reason in the original article. Are my understandings of the GAN incorrect?&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I have posted this question on &lt;a href=&quot;https://stats.stackexchange.com/q/504673/82135&quot;&gt;Stats SE&lt;/a&gt;, but have no luck with answers. Maybe it is more suitable for AI.&lt;/p&gt;&#xA;" OwnerUserId="40205" LastEditorUserId="2444" LastEditDate="2021-01-19T11:38:14.990" LastActivityDate="2021-01-27T19:00:31.213" Title="In this implementation of pix2pix, why are the weights for the discriminator and generator losses set to 1 and 100 respectively?" Tags="&lt;loss-functions&gt;&lt;generative-adversarial-networks&gt;&lt;pix2pix&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25825" PostTypeId="2" ParentId="25820" CreationDate="2021-01-18T12:08:05.420" Score="3" Body="&lt;p&gt;I will first explain &lt;em&gt;briefly&lt;/em&gt; to you the difference between supervised learning and reinforcement learning to make sure that you don't have any misunderstandings. In supervised learning you are provided with some data &lt;span class=&quot;math-container&quot;&gt;$\{(\textbf{x}_i, y_i)\}_{i=1}^n$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\textbf{x}_i$&lt;/span&gt; are the features for data point &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; is its true label. Now, the aim of supervised learning is to learn a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that can accurately predict the label of a data point given its features. In deep learning this function is a neural network, &lt;span class=&quot;math-container&quot;&gt;$f_\theta(\cdot)$&lt;/span&gt;. To optimise the parameters we obtain the models prediction for the label of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, denoted typically by &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = f_\theta(x)$&lt;/span&gt; and we look to optimise the parameters of the function &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; by minimising the loss &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(\hat{y}, y)$&lt;/span&gt; (note that here the loss is a function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}: \mathcal{Y} \times \mathcal{Y} \rightarrow \mathbb{R}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In reinforcement learning things are quite different. We are not provided with any data. Instead we have a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_decision_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov Decision Process (MDP)&lt;/a&gt; and an environment that we can interact with. The state space is defined by the MDP and we can get samples, mainly tuples of the form &lt;span class=&quot;math-container&quot;&gt;$(s, a, r, s')$&lt;/span&gt;, that we can use to teach our agent how to find an optimal policy - typically an optimal policy is one that takes an action given the current state that will maximise the sum of the future rewards of the episode. So in reinforcement you can probably see that things are quite different to supervised learning, mainly in the way that we 'obtain' our data.&lt;/p&gt;&#xA;&lt;p&gt;Of course these explanations are gross oversimplifications of the learning process in both paradigms and should only be used as a brief example to try to emphasise the different between the two learning paradigms.&lt;/p&gt;&#xA;&lt;p&gt;Now, there are two ways you can parameterise your Q-function using a neural network:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The network takes as input the current state and outputs Q-values for each potential action; i.e. the output is a vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{|\mathcal{A}|}$&lt;/span&gt;;&lt;/li&gt;&#xA;&lt;li&gt;The network takes as input the state and action and outputs a real number which is the Q-value for the state, action tuple you pass as input to the network.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The second way is usually reserved for instances where you have a huge action space but only ever would consider a few feasible actions at each state - this saves computational complexity.&lt;/p&gt;&#xA;&lt;p&gt;Now, to answer your question:&lt;/p&gt;&#xA;&lt;p&gt;In case 1) you are assumed to have access to a transition tuple &lt;span class=&quot;math-container&quot;&gt;$(s, a, r, s')$&lt;/span&gt;. The temporal difference, which we will use as the target, is &lt;span class=&quot;math-container&quot;&gt;$\hat{y} = r + \max_a Q(s', a)$&lt;/span&gt;. Now, as our output of &lt;span class=&quot;math-container&quot;&gt;$Q(s', \cdot)$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^{|\mathcal{A}|}$&lt;/span&gt; what we do is make a forward pass of the network for the current &lt;span class=&quot;math-container&quot;&gt;$(s, a)$&lt;/span&gt; tuple, i.e. we get &lt;span class=&quot;math-container&quot;&gt;$x = Q(s, a)$&lt;/span&gt;, and then change the element that corresponds to the action which satisfies &lt;span class=&quot;math-container&quot;&gt;$\arg\max_a Q(s', a)$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt;, so we have now got our augmented input &lt;span class=&quot;math-container&quot;&gt;$\tilde{x}$&lt;/span&gt; which serves as our target.&lt;/p&gt;&#xA;&lt;p&gt;To make that step a bit clearer, suppose we have a 2-dimensional action space and the &lt;span class=&quot;math-container&quot;&gt;$\arg\max_a Q(s', a)$&lt;/span&gt; is the action in the first dimension, then we would change the first dimension of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to be &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We then train the network using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_squared_error#Loss_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mean Squared Error Loss&lt;/a&gt; between &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tilde{x}$&lt;/span&gt; - note that no gradient information is retained when we do the forward pass to get &lt;span class=&quot;math-container&quot;&gt;$\tilde{x}$&lt;/span&gt;; in fact we usually don't use a current version of the &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; network, we use an 'old' version of the network called the target network (I imagine there's probably a question about this network on the site already so I won't explain it in detail).&lt;/p&gt;&#xA;&lt;p&gt;In case 2) the idea is much more simple as the outputs of the network are scalars so you can just train your network using the MSE between the scalar values of &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{y}$&lt;/span&gt; as defined above, the caveat here is that to calculate &lt;span class=&quot;math-container&quot;&gt;$\arg\max_a Q(s', a)$&lt;/span&gt; you have to make &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}|$&lt;/span&gt; forward passes of the network for all possible &lt;span class=&quot;math-container&quot;&gt;$(s', a)$&lt;/span&gt;, for fixed &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;, tuples which is why method 1) is usually preferred.&lt;/p&gt;&#xA;&lt;p&gt;Now, whilst we may use techniques from supervised learning to optimise the Q-function, you can hopefully see the differences between supervised and reinforcement learning.&lt;/p&gt;&#xA;&lt;p&gt;In short, the temporal difference is the target that you train your network towards being able to predict, and the state is (typically) the input into your neural network. For a more in depth discussion of RL being framed as supervised learning please see &lt;a href=&quot;https://ai.stackexchange.com/questions/14167/can-supervised-learning-be-recast-as-reinforcement-learning-problem&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-01-18T15:47:14.193" LastActivityDate="2021-01-18T15:47:14.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25826" PostTypeId="2" ParentId="25775" CreationDate="2021-01-18T12:40:28.337" Score="3" Body="&lt;p&gt;I found:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For scikit-learn like models: &lt;a href=&quot;https://github.com/eloquentarduino/micromlgen&quot; rel=&quot;nofollow noreferrer&quot;&gt;MicroML&lt;/a&gt;, &lt;a href=&quot;https://github.com/Edge-Learning-Machine/Micro-LM&quot; rel=&quot;nofollow noreferrer&quot;&gt;Micro-LM&lt;/a&gt;, &lt;a href=&quot;https://github.com/adarsh1001/micro-learn&quot; rel=&quot;nofollow noreferrer&quot;&gt;Micro Learn&lt;/a&gt;, &lt;a href=&quot;https://github.com/nok/sklearn-porter&quot; rel=&quot;nofollow noreferrer&quot;&gt;sklearn-porter&lt;/a&gt;, &lt;a href=&quot;https://github.com/emlearn/emlearn&quot; rel=&quot;nofollow noreferrer&quot;&gt;emlearn&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;For deep learning models: &lt;a href=&quot;https://www.tensorflow.org/lite/microcontrollers&quot; rel=&quot;nofollow noreferrer&quot;&gt;tensorflow Lite Micro&lt;/a&gt;, &lt;a href=&quot;https://www.st.com/en/embedded-software/x-cube-ai.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;X-CUBE-AI&lt;/a&gt;, &lt;a href=&quot;https://medium.com/pytorch/glow-compiler-optimizes-neural-networks-for-low-power-nxp-mcus-e095abe14942&quot; rel=&quot;nofollow noreferrer&quot;&gt;Glow&lt;/a&gt;, &lt;a href=&quot;https://github.com/majianjia/nnom&quot; rel=&quot;nofollow noreferrer&quot;&gt;NNoM&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Both: &lt;a href=&quot;https://github.com/Microsoft/EdgeML&quot; rel=&quot;nofollow noreferrer&quot;&gt;EdgeML&lt;/a&gt;, &lt;a href=&quot;https://github.com/Microsoft/EdgeML&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELL&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These seems to partly fit my needs. But i am surprised that i cannot find something more general that either convert Python to C or to object file with ML support (to be used in C projects). Indeed, once trained, ML algo are &amp;quot;just&amp;quot; a bunch of additions/multiplications/comparisons.&lt;/p&gt;&#xA;&lt;p&gt;It would help to be able to convert scikit-learn pipeline for instance, as ML projects are rarely composed of only a single ML model taking raw data. But it seems that EdgeAI/TinyML is mainly focused on compiling deep learning models when it comes to deploy ML models on &amp;quot;bare-metal&amp;quot; microcontrollers.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your answers btw, it helps.&lt;/p&gt;&#xA;" OwnerUserId="43239" LastEditorUserId="43239" LastEditDate="2021-01-27T20:39:00.100" LastActivityDate="2021-01-27T20:39:00.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25827" PostTypeId="1" CreationDate="2021-01-18T13:32:49.313" Score="2" ViewCount="28" Body="&lt;p&gt;I have been training some kind of agent to reach a target using a Q-learning based approach, and I have tried two different types of rewards:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Long-term reward: &lt;span class=&quot;math-container&quot;&gt;$\mathrm{reward} = - \mathrm{distance}(\mathrm{agent,target})(t+1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Short-term reward: &lt;span class=&quot;math-container&quot;&gt;$\mathrm{reward} = \mathrm{distance}(\mathrm{agent,target})(t) - \mathrm{distance}(\mathrm{agent,target})(t+1)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In the first case, I am rewarding the current progress. In the second case, I am rewarding direct progression, but this may lead to less progression in the future. My question is, what kind of reward does Q-learning need?&lt;/p&gt;&#xA;&lt;p&gt;I understand that the &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; factor should incorporate long term rewards, so it makes more sense to reward direct progression. However, using long-term rewards gave better results for my scenario...&lt;/p&gt;&#xA;" OwnerUserId="5344" LastEditorUserId="2444" LastEditDate="2021-01-18T14:07:02.350" LastActivityDate="2021-01-18T14:07:02.350" Title="Is better to reward short- or long-term progress in Q-learning?" Tags="&lt;reinforcement-learning&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;reward-design&gt;&lt;reward-functions&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25828" PostTypeId="1" CreationDate="2021-01-18T15:52:05.737" Score="1" ViewCount="12" Body="&lt;p&gt;I have a question about &lt;a href=&quot;http://proceedings.mlr.press/v37/xuc15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Show, Attend and Tell: Neural Image CaptionGeneration with Visual Attention&lt;/a&gt; paper by Xu. The basic mechanism of stochastic hard attention is that each pixel of the input image has a corresponding parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt;, which describes the probability that this pixel will be chosen for further processing.&lt;/p&gt;&#xA;&lt;p&gt;But I don't see an explanation of how to train or define this parameter in the paper. Can someone explain how to train this &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; for each pixel?&lt;/p&gt;&#xA;" OwnerUserId="43912" LastEditorUserId="2444" LastEditDate="2021-01-19T12:06:12.703" LastActivityDate="2021-01-19T12:06:12.703" Title="How are the parameters $\alpha_i$ of hard attention trained?" Tags="&lt;computer-vision&gt;&lt;training&gt;&lt;papers&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25829" PostTypeId="1" CreationDate="2021-01-18T15:58:53.487" Score="1" ViewCount="65" Body="&lt;p&gt;I want to train some IA algorithm to be able to evaluate the maturity of a fruit (say, measured in numbers of days before rotten) based on an image of the fruit.&#xA;My first instinct is to go with convolutional neural network (CNN), since those have proven very efficient for recognizing images. However, I am not sure what the output layer should look like in this case.&lt;/p&gt;&#xA;&lt;p&gt;I could separate the data into a bunch of classes (1 day left, 2 days left, 3 days left, etc.) and use one output node for each of these classes, as in an usual classification task, but in doing so I completely lose the continuous nature of the output, which makes me think it might not be the optimal way to proceed.&lt;/p&gt;&#xA;&lt;p&gt;Another option would be to just have a unique output node, whose activation would correspond to the continuous value to predict, here the number of days left (normalized appropriately to lie between 0 and 1). This would have the advantage of taking the continuity into account, but I have been told that neural networks aren't made to predict values in that way, they really are best suited for classification into discrete classes.&lt;/p&gt;&#xA;&lt;p&gt;What do you think would be the best way to proceed? Is there another way to nudge a neural network so that its output is continuous? Or maybe CNN just aren't suited for this task? If you have any suggestions of other algorithms that would be efficient for this kind of task, I would be happy to know them.&lt;/p&gt;&#xA;" OwnerUserId="37262" LastActivityDate="2021-01-21T11:28:29.083" Title="Predicting continous value with CNN (prediction of fruit maturity)" Tags="&lt;convolutional-neural-networks&gt;&lt;image-recognition&gt;&lt;continuous-tasks&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25830" PostTypeId="5" CreationDate="2021-01-18T17:14:55.307" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T17:14:55.307" LastActivityDate="2021-01-18T17:14:55.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25831" PostTypeId="4" CreationDate="2021-01-18T17:14:55.307" Score="0" Body="For questions related to the Computational Theory of Mind (CTM), i.e. the theory that the mind is a computational system." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T17:35:18.727" LastActivityDate="2021-01-18T17:35:18.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25832" PostTypeId="1" CreationDate="2021-01-18T18:46:04.713" Score="1" ViewCount="47" Body="&lt;p&gt;Looking at some old notes I took on CNN's and I wrote down that the weights in a CNN are acting like filters in a CNN but to be honest I don't really know what the weights are acting as in a CNN and was wondering if someone could explain that clearly to me.&lt;/p&gt;&#xA;" OwnerUserId="20336" LastEditorUserId="2444" LastEditDate="2021-01-19T15:30:59.473" LastActivityDate="2021-02-18T17:33:53.780" Title="What are acting as weights in a convolution neural network?" Tags="&lt;convolutional-neural-networks&gt;&lt;weights&gt;" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25833" PostTypeId="5" CreationDate="2021-01-18T21:08:18.997" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T21:08:18.997" LastActivityDate="2021-01-18T21:08:18.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25834" PostTypeId="4" CreationDate="2021-01-18T21:08:18.997" Score="0" Body="For questions related to recurrent layers, such as layers composed of LSTM units." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T21:08:18.997" LastActivityDate="2021-01-18T21:08:18.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25835" PostTypeId="1" CreationDate="2021-01-18T21:22:22.283" Score="4" ViewCount="127" Body="&lt;p&gt;AI reached super-human level in many complex games, including imperfect information games such as &lt;a href=&quot;https://science.sciencemag.org/content/365/6456/885.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;six-player no-limit Texas hold’em poker&lt;/a&gt;.  However, it still did not reached that level in Trick-taking card games such as Spades, Bridge, Skat and Whist.  In a related question, I am asking &lt;a href=&quot;https://ai.stackexchange.com/questions/25439/why-multiplayer-imperfect-information-trick-taking-card-games-are-hard-for-ai&quot;&gt;Why Trick-Taking games are a challenge for AI&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;An important factor that makes those games a challenge for AI is their size, to be precise lets talk about the &lt;em&gt;State-space complexity&lt;/em&gt; which is define as the number of legal game positions reachable from the initial position of the game  &lt;a href=&quot;https://project.dke.maastrichtuniversity.nl/games/files/phd/SearchingForSolutions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[Victor Allis (section 6.2.4)]&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;What is the size of Spades?&lt;/h2&gt;&#xA;" OwnerUserId="43351" LastEditorUserId="43351" LastEditDate="2021-02-14T07:25:22.773" LastActivityDate="2021-02-14T07:25:22.773" Title="What is the state-space complexity of Spades?" Tags="&lt;game-ai&gt;&lt;game-theory&gt;&lt;games-of-chance&gt;&lt;state-space-complexity&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25837" PostTypeId="1" AcceptedAnswerId="25842" CreationDate="2021-01-18T22:58:24.903" Score="2" ViewCount="68" Body="&lt;p&gt;I have seen many papers using autoencoders to replace images (states) with latent representations. Some of those methods have shown higher rewards using such techniques. However, I do not understand how this helps the RL agent learn better. Perhaps viewing latent representations allows the agent to generalize to novel states more quickly?&lt;/p&gt;&#xA;&lt;p&gt;Here are 2 papers I have read -&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1509.06113&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Spatial Autoencoders for Visuomotor Learning&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1806.00630&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAQN: Deep Auto-encoder and Q-Network&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="31755" LastEditorUserId="2444" LastEditDate="2021-01-20T11:30:39.880" LastActivityDate="2021-01-20T11:30:39.880" Title="How does replacing states with latent representations help RL agents?" Tags="&lt;reinforcement-learning&gt;&lt;papers&gt;&lt;autoencoders&gt;&lt;generalization&gt;&lt;state-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25838" PostTypeId="2" ParentId="18181" CreationDate="2021-01-18T23:32:31.263" Score="0" Body="&lt;p&gt;I do not feel comfortable proclaiming that I would be a machine learning expert.&#xA;But I want to point out that there is indeed interest in applying tensor networks in machine learning settings.&#xA;Let me highlight three particular settings in the following.&lt;/p&gt;&#xA;&lt;p&gt;They can be used to find the governing equations behind dynamical systems analogous to the &lt;a href=&quot;https://www.pnas.org/content/113/15/3932&quot; rel=&quot;nofollow noreferrer&quot;&gt;SINDy algorithm&lt;/a&gt;.&#xA;The reasoning is that the dynamics of your system may not be sparse but may indeed have a low-rank structure. (See &lt;a href=&quot;https://arxiv.org/abs/1607.04444&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for a comparison of sparse and low-rank structures and &lt;a href=&quot;https://epubs.siam.org/doi/10.1137/18M1183480&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for a discussion about why low-rank matrices so often appear in big data.)&#xA;The resulting modification of SINDy is the appropriately named &lt;a href=&quot;https://arxiv.org/abs/1809.02448&quot; rel=&quot;nofollow noreferrer&quot;&gt;MANDy algorithm&lt;/a&gt;.&#xA;But if your dynamical law exhibits additional structure (despite being low-rank) then this structure is not used. (Requiring you to collect unnecessary samples.)&#xA;&lt;a href=&quot;https://arxiv.org/abs/2002.12388&quot; rel=&quot;nofollow noreferrer&quot;&gt;This work&lt;/a&gt; tackles this issue by allowing you to specify the structure that you expect your law to exhibit.&lt;/p&gt;&#xA;&lt;p&gt;Another area of interest is the application of tensor networks as a means of compression.&#xA;Tensor networks can be used to approximate multivariate functions.&#xA;Either directly (see &lt;a href=&quot;https://arxiv.org/abs/2007.00118&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; or &lt;a href=&quot;https://www.mis.mpg.de/fileadmin/uschmajew/uschmajew_approximationPreprint.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; for further details) or by representing their high-dimensional coefficient tensor (see e.g. &lt;a href=&quot;https://link.springer.com/article/10.1007/s00211-016-0850-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&#xA;The last paper also highlights where such functions may occur: when solving stochastic or parametric PDEs.&#xA;&amp;quot;What does this have to do with machine learning?&amp;quot; you may rightfully ask.&#xA;These PDEs are hard to solve and it is often easier to minimize a residual than to apply a Galerkin method. The results can be shown to be equivalent with high probability. (See &lt;a href=&quot;https://arxiv.org/abs/1810.01348&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for a proof of this statement and an application to some stochastic PDEs or &lt;a href=&quot;https://arxiv.org/abs/1911.00279&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for an application to an optimal control problem.)&lt;/p&gt;&#xA;&lt;p&gt;Moreover, there is the problem of tensor completion - a generalization of matrix completion to higher dimensions. (&lt;a href=&quot;https://arxiv.org/abs/1701.08045&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; and &lt;a href=&quot;https://arxiv.org/abs/1405.1773&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; are references for algorithms.)&#xA;These methods find application in data science where they can be used to decompose or denoise data. (Unfortunately, I don't have a reference with examples but I remember a presentation where these methods where used on EEG signals.)&lt;/p&gt;&#xA;&lt;p&gt;Additionally, I recently found this interesting symposium covering even more potential applications of tensor networks in machine learning:&#xA;&lt;a href=&quot;https://itsatcuny.org/calendar/quantum-inspired-machine-learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://itsatcuny.org/calendar/quantum-inspired-machine-learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42033" LastEditorUserId="42033" LastEditDate="2021-01-18T23:58:29.090" LastActivityDate="2021-01-18T23:58:29.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25839" PostTypeId="5" CreationDate="2021-01-18T23:36:39.847" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T23:36:39.847" LastActivityDate="2021-01-18T23:36:39.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25840" PostTypeId="4" CreationDate="2021-01-18T23:36:39.847" Score="0" Body="For questions related to the XOR problem, i.e. the problem of computing/approximating the XOR logical function, i.e. a function $f: \{0, 1\}^2  \rightarrow \{0, 1\} $ that returns 1 only when the two bits are different and 0 otherwise, i.e. $f(a, b) = 1 \iff a \neq b$ and $a, b \in \{0, 1\}$. " OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-18T23:36:39.847" LastActivityDate="2021-01-18T23:36:39.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25841" PostTypeId="2" ParentId="25769" CreationDate="2021-01-19T01:38:10.173" Score="0" Body="&lt;p&gt;One way you can definitely approach the problem is by using &lt;strong&gt;(Deep) Reinforcement Learning&lt;/strong&gt; &lt;strong&gt;(DRL)&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;YouTube is actually using DRL as well to suggest videos to users in order to maximize users' engagement with their website. For more information (and further references to papers explaining how other major companies implement their recommendation systems), see this &lt;a href=&quot;https://arxiv.org/abs/1905.12767&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. Just as a bit of motivation.&#xA;Other research into that direction could be found &lt;a href=&quot;https://arxiv.org/abs/1902.05570&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Actually, there are different DRL algorithms available and each might be suitable for approaching the problem you described above from a slightly different angle.&#xA;The way how I would roughly approach your problem is as follows:&lt;/p&gt;&#xA;&lt;p&gt;As I understand it, the goal is the following.&#xA;You want to show either of four messages to a user (only one at a time) and always display the message, which is expected to reduce some metric (=conversion time), to the user.&lt;/p&gt;&#xA;&lt;p&gt;In that case, an DRL algorithm, or &lt;em&gt;agent&lt;/em&gt;, could directly be trained on minimizing the conversion time without having to explicitly produce conversion time estimates per message &amp;amp; user. That might make phrasing the problem at hand much easier (in terms of modelling the learning task).&lt;/p&gt;&#xA;&lt;p&gt;The outcome of training an DRL agent consists of a so-called &lt;em&gt;optimal policy&lt;/em&gt; network, which dictates the agent which message to display to a given user, upon observing the provided user data, such that conversion time gets minimized for that user. The aforementioned network is a simple artificial Neural Network (NN) or Recurrent NN (RNN).&lt;/p&gt;&#xA;&lt;p&gt;Training the DRL agent will consist of two steps being repeated many times in alternating order.&lt;/p&gt;&#xA;&lt;p&gt;One step will include the sampling/generation of training data.&#xA;For this step, the algorithm (initially being untrained) will be applied to the problem at hand and predict messages for given users. Then, (in addition to the provided user data that was fed as input to the agent) each user's response (i.e. the conversion time) is recorded. This can later be used as the reward that the agent receives for having selected the selected message given the provided user data. Here, a low conversion time translates to a high reward.&lt;/p&gt;&#xA;&lt;p&gt;In the second step, the agent's policy network will be trained in order to make the agent's predictions more accurate in the next round of generating training data. Here, the data recorded during the previous step will be used.&#xA;How the policy network is optimized largely depends on which DRL algorithm you choose. The range of different methods varies vastly. Some algorithms try to estimate &lt;em&gt;utilities&lt;/em&gt; (i.e. measures of goodness) per &lt;em&gt;action&lt;/em&gt; given a certain state (here, &lt;em&gt;state = provided user data used to generate a prediction&lt;/em&gt;). In this case, an &lt;em&gt;action&lt;/em&gt; would be &lt;em&gt;choosing message A&lt;/em&gt; or &lt;em&gt;choosing message B&lt;/em&gt; etc. A popular algorithm implementing this procedure is &lt;a href=&quot;https://arxiv.org/abs/1312.5602&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Q-Learning&lt;/a&gt;.&#xA;However, if you rather want to predict probabilities per action, then &lt;a href=&quot;https://arxiv.org/abs/1707.06347&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proximal Policy Optimization&lt;/a&gt; might rather be what you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;Upon convergence, i.e. stabilization of the goal metric (=convergence time per user), the algorithm has arrived at its optimal policy (ideally speaking).&lt;/p&gt;&#xA;&lt;p&gt;The only drawback with this approach is that DRL usually requires quite a lot of training. But maybe you are lucky and pre-trained recommendation system models exist.&lt;/p&gt;&#xA;&lt;p&gt;If there is still anything unclear, feel free to ask in the comments.&lt;/p&gt;&#xA;" OwnerUserId="37982" LastEditorUserId="37982" LastEditDate="2021-01-19T02:29:20.283" LastActivityDate="2021-01-19T02:29:20.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25842" PostTypeId="2" ParentId="25837" CreationDate="2021-01-19T11:24:25.330" Score="3" Body="&lt;p&gt;In short, it is much easier for the agent to learn from a smaller dimensional state space. This is because the agent must also do representation learning; i.e. it must also infer &lt;em&gt;what&lt;/em&gt; the state is telling it as part of the learning process. If you think of the architecture used in DQN to solve Atari, they had a CNN that outputted a vector which was then passed through some dense layers. Here the representation learning was done by the CNN and was trained using an end-to-end approach i.e. all updates to the network weights were done through the reinforcement learning objective; that is there is no supervised or unsupervised learning that takes place.&lt;/p&gt;&#xA;&lt;p&gt;This can be particularly difficult when you combine images with sparse rewards as there is not a lot of feedback so the representation learning can take a long time. &lt;a href=&quot;https://arxiv.org/abs/2009.08319&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; gives a good description of the problem of decoupling representation learning from reinforcement learning with a nice solution.&lt;/p&gt;&#xA;&lt;p&gt;The other main 'problem setting' I have seen images replaced with a latent state is when the authors are looking at planning. The problem with doing any kind of planning is that a model of the transition dynamics, &lt;span class=&quot;math-container&quot;&gt;$p(s' | s, a)$&lt;/span&gt;, is needed. For high dimensional state spaces such as images, this can be very difficult to predict and even relatively small errors will quickly compound so if you use the model to predict multiple time steps into the future the planner is useless because of these compounding errors. I think there is a discussion on this in &lt;a href=&quot;https://arxiv.org/abs/1911.08453&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (certainly there will be references therein that point you in the right direction).&lt;/p&gt;&#xA;" OwnerUserId="36821" LastActivityDate="2021-01-19T11:24:25.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25844" PostTypeId="1" CreationDate="2021-01-19T14:25:11.570" Score="2" ViewCount="47" Body="&lt;p&gt;Considering weights initialization in my personal projects, I always used some standard techniques such as:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Glorot (also known as Xavier) initialization (2010).&lt;/li&gt;&#xA;&lt;li&gt;Mertens initialization (2010).&lt;/li&gt;&#xA;&lt;li&gt;He initialization (2015).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As it is a very active research field, are there some innovations in recent years that have increased the performance of DNNs?&lt;/p&gt;&#xA;&lt;p&gt;I am thinking specifically of architectures such as DNNs and CNNs with activation functions, such as ReLU,  ELU, PReLU, Leaky ReLU, SELU, Swish, and Mish.&lt;/p&gt;&#xA;" OwnerUserId="36907" LastEditorUserId="2444" LastEditDate="2021-01-20T21:03:29.633" LastActivityDate="2021-01-20T21:03:29.633" Title="Are there any new weight initialization techniques for DNN published after 2015?" Tags="&lt;deep-learning&gt;&lt;reference-request&gt;&lt;deep-neural-networks&gt;&lt;weights&gt;&lt;weights-initialization&gt;" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25845" PostTypeId="5" CreationDate="2021-01-19T14:26:50.280" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T14:26:50.280" LastActivityDate="2021-01-19T14:26:50.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25846" PostTypeId="4" CreationDate="2021-01-19T14:26:50.280" Score="0" Body="For questions related to the intractability of AI algorithms." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T14:26:50.280" LastActivityDate="2021-01-19T14:26:50.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25847" PostTypeId="2" ParentId="25832" CreationDate="2021-01-19T15:25:13.520" Score="1" Body="&lt;p&gt;There are many resources that answer your question, but, given that you're apparently new to machine learning (ML), deep learning (DL), and neural networks (NN), let me provide a simple answer that should clarify your doubts.&lt;/p&gt;&#xA;&lt;p&gt;The term &lt;strong&gt;weight&lt;/strong&gt; in the context of ML, DL, and NN is a synonym for &lt;strong&gt;parameter&lt;/strong&gt; (sometimes, in some contexts, such as linear regression, it is also known as &lt;strong&gt;coefficient&lt;/strong&gt;), which can be constant (i.e. do not change during, for example, learning/training) or learnable (i.e. change during the training/learning process). In a feedforward neural network (FFNN), with and without recurrent connections, the weights are the numbers on the connections between neurons in different or same layers. They are called &lt;strong&gt;weights&lt;/strong&gt; to emphasize that their role is to weigh the effect of one neuron on the other.&lt;/p&gt;&#xA;&lt;p&gt;In a CNN, the weights are the kernels/filters of the CNN, i.e. the matrices that you use to perform the convolution (or cross-correlation) operation in a convolutional layer. So, given that CNNs perform an operation that seems to be different than the linear combination followed by the non-linear activation function in FFNNs, you could think that the weights in a CNN are not very similar to the weights in FFNNs. However, this is not fully true, as CNNs can be viewed as FFNNs (with some specific structure): maybe &lt;a href=&quot;https://ai.stackexchange.com/a/18796/2444&quot;&gt;this answer&lt;/a&gt; will provide you more info about this topic. So, your notes are right!&lt;/p&gt;&#xA;&lt;p&gt;In my opinion, the best way to understand the role of the kernels (i.e. weights) of a CNN and all the details behind the convolution (or cross-correlation) operation is to first study some traditional computer vision and image processing techniques, so maybe you should pick a book that covers them.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://ai.stackexchange.com/a/21967/2444&quot;&gt;this answer&lt;/a&gt;, I briefly try to describe what a CNN is and what it can be used for. There are other answers on this site, such as &lt;a href=&quot;https://ai.stackexchange.com/q/21394/24445&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://ai.stackexchange.com/q/12510/2444&quot;&gt;this&lt;/a&gt;, and &lt;a href=&quot;https://ai.stackexchange.com/q/21999/2444&quot;&gt;this&lt;/a&gt;, which you may want to read, if not now later, to understand even more all the details of CNNs. A  decent reference book that covers CNNs (and other DL topics) is &lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deep Learning&lt;/a&gt; by Ian Goodfellow et al.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-18T17:33:53.780" LastActivityDate="2021-02-18T17:33:53.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25848" PostTypeId="2" ParentId="25726" CreationDate="2021-01-19T16:12:40.593" Score="2" Body="&lt;p&gt;Smoothness here is the mathematical &lt;a href=&quot;https://en.wikipedia.org/wiki/Smoothness&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition&lt;/a&gt;, so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's gradient, ad infinitum, depending on who defines smoothness).&lt;/p&gt;&#xA;&lt;p&gt;By any definition a lot of activation functions are not smooth, for example RELU. This means neural networks in general are not smooth.&lt;/p&gt;&#xA;" OwnerUserId="43947" LastActivityDate="2021-01-19T16:12:40.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25849" PostTypeId="1" AcceptedAnswerId="25885" CreationDate="2021-01-19T16:16:06.553" Score="1" ViewCount="40" Body="&lt;p&gt;I've solved many problems with neural networks, but rarely work with images.  I have about 18 hours into creating a bounding box regression network and it continues to utterly fail.  With some loss functions it will claim 80% accuracy during training and validation (with a truly massive loss on both) but testing the predictions reveals a bounding box that only moves one or two pixels in any given direction and seems to totally ignore the data.  I've now implemented a form of IoU loss, but find that IoU is pinned at zero... which is obviously true based on the outputs after training. :). I'd like someone to look this over and give me some advice on how to proceed next.&lt;/p&gt;&#xA;&lt;h2&gt;What I Have&lt;/h2&gt;&#xA;&lt;p&gt;I am generating 40000 examples of 200x100x3 images with a single letter randomly placed in each.  Simultaneously I am generating the ground truth bounding boxes for each training sample.  I have thoroughly validated that this all works and the data is correct.&lt;/p&gt;&#xA;&lt;h2&gt;What I Do To It&lt;/h2&gt;&#xA;&lt;p&gt;I am then transforming the 200x100x3 images down to greyscale to produce a 200x100x1 image.  The images are then normalized and the bounding boxes are scaled to fall between 0 and 1.  In simplified form, this happens:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x_train_normalized = (x_data - 127.5) / 127.5&#xA;y_train_scaled = boxes[:TRAIN]/[WIDTH,HEIGHT,WIDTH,HEIGHT]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've been through this data carefully, even reconstituting images and bounding boxes from it.  This is definitely working.&lt;/p&gt;&#xA;&lt;h2&gt;Training&lt;/h2&gt;&#xA;&lt;p&gt;To train, after trying &lt;code&gt;mse&lt;/code&gt; and many others, all of which fail equally badly, I have implemented a simple custom IOU loss function.  It actually returns &lt;code&gt;-ln(IoU)&lt;/code&gt;.  I made this change based on a paper since the loss was (oddly?) pinned at zero over multiple epochs.&lt;/p&gt;&#xA;&lt;p&gt;(Loss function:)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow.keras.backend as kb&#xA;def iou_loss(y_actual,y_pred):&#xA;    b1 = y_actual&#xA;    b2 = y_pred&#xA;#    tf.print(b1)&#xA;#    tf.print(b2)&#xA;    zero = tf.convert_to_tensor(0.0, b1.dtype)&#xA;    b1_ymin, b1_xmin, b1_ymax, b1_xmax = tf.unstack(b1, 4, axis=-1)&#xA;    b2_ymin, b2_xmin, b2_ymax, b2_xmax = tf.unstack(b2, 4, axis=-1)&#xA;    b1_width = tf.maximum(zero, b1_xmax - b1_xmin)&#xA;    b1_height = tf.maximum(zero, b1_ymax - b1_ymin)&#xA;    b2_width = tf.maximum(zero, b2_xmax - b2_xmin)&#xA;    b2_height = tf.maximum(zero, b2_ymax - b2_ymin)&#xA;    b1_area = b1_width * b1_height&#xA;    b2_area = b2_width * b2_height&#xA;&#xA;    intersect_ymin = tf.maximum(b1_ymin, b2_ymin)&#xA;    intersect_xmin = tf.maximum(b1_xmin, b2_xmin)&#xA;    intersect_ymax = tf.minimum(b1_ymax, b2_ymax)&#xA;    intersect_xmax = tf.minimum(b1_xmax, b2_xmax)&#xA;    intersect_width = tf.maximum(zero, intersect_xmax - intersect_xmin)&#xA;    intersect_height = tf.maximum(zero, intersect_ymax - intersect_ymin)&#xA;    intersect_area = intersect_width * intersect_height&#xA;&#xA;    union_area = b1_area + b2_area - intersect_area&#xA;    iou = -1 * tf.math.log(tf.math.divide_no_nan(intersect_area, union_area))&#xA;    return iou&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;The Network&lt;/h3&gt;&#xA;&lt;p&gt;This has been through many, many iterations.  As I said, I've solved many other problems with NNs... This is the first one to get me completely stuck.  At this point, the network is dramatically stripped down but continues to fail to train at all:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import tensorflow as tf&#xA;from tensorflow import keras&#xA;from tensorflow.keras import layers, optimizers&#xA;&#xA;tf.keras.backend.set_floatx('float32') # Use Float32s for everything&#xA;&#xA;input_shape = x_train_normalized.shape[-3:]&#xA;model = keras.Sequential()&#xA;model.add(layers.Conv2D(4, 16, activation = tf.keras.layers.LeakyReLU(alpha=0.2), input_shape=input_shape))&#xA;model.add(layers.MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))&#xA;model.add(layers.Dropout(0.2))&#xA;model.add(layers.Flatten())&#xA;model.add(layers.Dense(200, activation = tf.keras.layers.LeakyReLU(alpha=0.2)))&#xA;model.add(layers.Dense(64, activation=tf.keras.layers.LeakyReLU(alpha=0.2)))&#xA;model.add(layers.Dense(4, activation=&amp;quot;sigmoid&amp;quot;))&#xA;&#xA;model.compile(loss = iou_loss, optimizer = &amp;quot;adadelta&amp;quot;, metrics=['accuracy'])&#xA;history = model.fit(x_train_normalized, y_train_scaled, epochs=8, batch_size=100, validation_split=0.4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All pointers are welcome!  In the meantime I'm implementing a center point loss function to see if that helps at all.&lt;/p&gt;&#xA;" OwnerUserId="30426" LastEditorUserId="2444" LastEditDate="2021-01-21T01:57:25.257" LastActivityDate="2021-01-21T01:57:25.257" Title="Bounding Box Regression - An Adventure in Failure" Tags="&lt;tensorflow&gt;&lt;object-detection&gt;&lt;weights-initialization&gt;&lt;bounding-box&gt;" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25850" PostTypeId="2" ParentId="18785" CreationDate="2021-01-19T16:17:27.547" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://ai.stackexchange.com/a/18796/2444&quot;&gt;other answer&lt;/a&gt; gives a good overview of the differences between MLPs and CNNs, and it includes 2 diagrams that attempt to illustrate the main differences between MLPs and CNNs, i.e. sparse connectivity and weight sharing. However, these diagrams do not clarify what a &lt;strong&gt;neuron&lt;/strong&gt; in a CNN could be. A better diagram, which illustrates what a neuron is in a CNN, from a CNN &lt;em&gt;and&lt;/em&gt; MLP perspective, is the following (taken from &lt;a href=&quot;https://cs231n.github.io/convolutional-networks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the famous article on CNNs&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o2CwW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o2CwW.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, there are 2 main blocks (aka volumes): the orange block on the left (the input) and the blue/cyan volume on the right (the &lt;strong&gt;feature maps&lt;/strong&gt;, i.e. the outputs of the convolutional layer, i.e. after the application of the convolutions with different kernels).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;em&gt;circles&lt;/em&gt; in the visible stack of the cyan block represent the neurons (or, more precisely, their &lt;strong&gt;activations&lt;/strong&gt; or outputs). We only see &lt;span class=&quot;math-container&quot;&gt;$k=5$&lt;/span&gt; neurons stacked: this corresponds to the application of &lt;span class=&quot;math-container&quot;&gt;$k=5$&lt;/span&gt; different kernels (i.e. weights) to that specific subset of the input (aka &lt;strong&gt;receptive field&lt;/strong&gt;), hence the &lt;strong&gt;sparse connectivity&lt;/strong&gt; of CNNs. So, these neurons, in the same stack, are looking at the same small subset of the input, but with different weights (i.e. kernels). The neurons, which are not shown in this diagram, that are on the same (vertical) 2d plane (known as &lt;strong&gt;feature map&lt;/strong&gt;) of the same neuron (e.g. the first that we see from left to right) in the cyan volume are the neurons that share the same weights, i.e. we use the same kernel to produce their outputs.&lt;/p&gt;&#xA;&lt;p&gt;So, in this biological/neuroscientific view of the CNN, when you apply &lt;a href=&quot;https://ai.stackexchange.com/q/21999/2444&quot;&gt;the convolution (or cross-correlation) operation&lt;/a&gt; with 1 specific filter (or kernel), you are computing the activation (not to be confused with the &lt;em&gt;activation function&lt;/em&gt;, which is used to compute the activation!) i.e. the output of multiple neurons, all of them share the same weights. You stack all these activations on the same 2d plane (known as &lt;strong&gt;feature map&lt;/strong&gt;) of the output volume: note that this operation is just the convolution operation! When you compute the convolution with another kernel, you are again computing the activation of other multiple neurons, which share &lt;em&gt;another different&lt;/em&gt; weight matrix, and so on and so forth.&lt;/p&gt;&#xA;&lt;p&gt;Some authors prefer to use the term &lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;convolutional networks&lt;/strong&gt;&lt;/a&gt;, i.e. without the term &lt;strong&gt;neural&lt;/strong&gt;, probably because of this issue, i.e. it's not clear, especially to newcomers, what a neuron would be in a CNN, so the neuroscientific/biological view of CNNs is not always clear, although it's important to emphasize that &lt;a href=&quot;https://ai.stackexchange.com/a/24969/2444&quot;&gt;CNNs were inspired by the visual cortext&lt;/a&gt;, so this biological interpretation could (and should) be more widely known or less confusing/misunderstood.&lt;/p&gt;&#xA;&lt;p&gt;Now, let's address your question more directly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Aren't the filters the same, in the way that they convert an &amp;quot;image&amp;quot; to a new &amp;quot;image&amp;quot; based on the weights that are in that filter? And that the next layer uses this new &amp;quot;image&amp;quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The filters in a CNN correspond to the weights of an MLP.&lt;/p&gt;&#xA;&lt;p&gt;A neuron in a CNN can be viewed as performing exactly the same operation as a neuron in an MLP. The big differences between a CNN and an MLP (as explained also in &lt;a href=&quot;https://ai.stackexchange.com/a/18796/2444&quot;&gt;the other answer&lt;/a&gt;) are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Weight sharing&lt;/strong&gt;: Some neurons (not all!) in the same convolutional layer share the same weights. The convolution (or cross-correlation) is the operation that implements this partial forward pass with the same weights for different neurons.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Neurons in a CNN only look at a &lt;strong&gt;subset&lt;/strong&gt; of the input and not all inputs (i.e. receptive field), which leads to some notion of &lt;strong&gt;sparse connectivity&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A convolutional layer, in a CNN, is composed of neurons in a &lt;strong&gt;3d dimensional&lt;/strong&gt; volume (or, more precisely, their activations are organized in a 3d volume), rather than a 1-dimensional one, as in an MLP.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CNNs may use subsampling (aka pooling)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T16:58:19.803" LastActivityDate="2021-01-19T16:58:19.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25851" PostTypeId="5" CreationDate="2021-01-19T17:26:38.283" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T17:26:38.283" LastActivityDate="2021-01-19T17:26:38.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25852" PostTypeId="4" CreationDate="2021-01-19T17:26:38.283" Score="0" Body="For questions related to the selective search algorithm, for example, described in &quot;Selective Search for Object Recognition&quot; (2013) by J. R. R. Uijlings et al." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T17:26:38.283" LastActivityDate="2021-01-19T17:26:38.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25853" PostTypeId="1" CreationDate="2021-01-19T17:28:48.143" Score="4" ViewCount="67" Body="&lt;p&gt;In &lt;a href=&quot;http://page.mi.fu-berlin.de/rojas/neural/chapter/K9.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 9&lt;/a&gt;, section 9.1.6, Raul Rojas describes how committees of networks can reduce the prediction error by training N identical neural networks and averaging the results.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt; are the functions approximated by the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; neural nets, then:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q=\left|\frac{1}{N}(1,1, \ldots, 1) \mathbf{E}\right|^{2}=\frac{1}{N^{2}}(1,1, \ldots, 1) \mathbf{E} \mathbf{E}^{\mathrm{T}}(1,1, \ldots, 1)^{\mathrm{T}}\tag{9.4}\label{9.4}&#xA;$$&lt;/span&gt;&#xA;is the quadratic error of the average of the networks, where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{E}=\left(\begin{array}{cccc}&#xA;e_{1}^{1} &amp;amp; e_{2}^{1} &amp;amp; \cdots &amp;amp; e_{m}^{1} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;e_{1}^{N} &amp;amp; e_{2}^{N} &amp;amp; \cdots &amp;amp; e_{m}^{N}&#xA;\end{array}\right),&#xA;$$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E}$&lt;/span&gt;'s rows are the errors of the approximations of the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; functions, i.e. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{e}^{i} = f_i(\mathbf{x}^{i}) - t_i$&lt;/span&gt;, for each of the input-output pairs &lt;span class=&quot;math-container&quot;&gt;$\left(\mathbf{x}^{1}, t_{1}\right), \ldots,\left(\mathbf{x}^{m}, t_{m}\right)$&lt;/span&gt; used in training.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to assure that the errors for a neural network are uncorrelated to the errors of the others?&lt;/p&gt;&#xA;&lt;p&gt;Raul Rojas says that the uncorrelation of residual errors is true for a not too large &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$N &amp;lt; 4$&lt;/span&gt;). Why is that?&lt;/p&gt;&#xA;" OwnerUserId="14892" LastEditorUserId="14892" LastEditDate="2021-01-20T17:02:27.273" LastActivityDate="2021-01-20T17:02:27.273" Title="When do two identical neural networks have uncorrelated errors?" Tags="&lt;neural-networks&gt;&lt;proofs&gt;&lt;statistics&gt;&lt;ensemble-learning&gt;&lt;correlation&gt;" AnswerCount="0" CommentCount="10" FavoriteCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25855" PostTypeId="5" CreationDate="2021-01-19T17:38:05.733" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T17:38:05.733" LastActivityDate="2021-01-19T17:38:05.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25856" PostTypeId="4" CreationDate="2021-01-19T17:38:05.733" Score="0" Body="For questions related to grammar-guided genetic programming approaches, such as grammatical evolution." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T17:38:05.733" LastActivityDate="2021-01-19T17:38:05.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25857" PostTypeId="2" ParentId="14204" CreationDate="2021-01-19T18:33:16.953" Score="0" Body="&lt;p&gt;In tree-based genetic programming (TGP), you have a tree that represents a program or a function. The nodes in this tree are functions, while the edges represent the interactions between these functions. The leaves of this tree are the inputs (or random numbers) that you pass to this function. The incoming edges into a node represent the inputs, while the outgoing edges represent the output of the associated function.&lt;/p&gt;&#xA;&lt;p&gt;So, for example, consider the following function&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(x) &#xA;&amp;amp;= \sin(x^2) \\&#xA;&amp;amp;= \sin(y) &#xA;\tag{1}\label{1},&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$y = g(x)$&lt;/span&gt; (note that this is just a change of variable in order to illustrate the corresponding tree more clearly below!).&lt;/p&gt;&#xA;&lt;p&gt;The function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; will be represented by the following tree&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sin(y)&#xA;  |&#xA; g(x)&#xA;  |&#xA;  x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, how do we read this diagram? Essentially, we read it from the bottom-up. So, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is first passed to &lt;span class=&quot;math-container&quot;&gt;$y = g(x) = x^2$&lt;/span&gt;, which then passed to &lt;span class=&quot;math-container&quot;&gt;$\sin(y)$&lt;/span&gt;. In this case, given that we are dealing with mathematical operations, you expect &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to be a number, because, otherwise, what would &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\sin(x^2)$&lt;/span&gt; mean?&lt;/p&gt;&#xA;&lt;p&gt;Let's now consider a function of 2 inputs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(x, y) &#xA;&amp;amp;= x + y&#xA;\tag{2}\label{2},&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The corresponding tree would be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  +&#xA; / \&#xA;x   y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this case, you naturally expect that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are also numbers. However, this may not be actually the case. Let's say that you're evolving Python functions, then &lt;code&gt;x + y&lt;/code&gt; is well defined even if &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; are strings, i.e. that would be a concatenation operation.&lt;/p&gt;&#xA;&lt;p&gt;So, in this sense, we naturally expected the functions (the nodes in the tree) to get parameters/arguments with the right types, but, in some cases, more types are possible for the apparently same function.&lt;/p&gt;&#xA;&lt;p&gt;So, in general, types don't necessarily need to be enforced! It depends on the implementation of TGP.&lt;/p&gt;&#xA;&lt;p&gt;There's a specific approach to TGP where the idea is really to ensure type-safety, i.e. &lt;strong&gt;strongly-typed&lt;/strong&gt; (tree-based) &lt;strong&gt;genetic programming&lt;/strong&gt;, where functions can have a different number and type for their parameters and return values, but, in that case, only functions that are consistent with the signature of the function can be &amp;quot;connected&amp;quot; with that function in the tree.&lt;/p&gt;&#xA;&lt;p&gt;In weakly-typed GP, the types are not checked, so you could end up evolving functions of the form &lt;span class=&quot;math-container&quot;&gt;$f(x) = \sin(x)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a string. Of course, when you execute these programs/functions, the program may crash, but this is a different story, which you need to take care of (i.e. you need to ensure that your individuals can be evaluated), if you need this flexibility. This weakly-typed approach can also be viewed as a strongly-typed approach where all functions' parameters and return values have the same type.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/DEAP/deap&quot; rel=&quot;nofollow noreferrer&quot;&gt;DEAP&lt;/a&gt;, a well-known Python library for GAs and GP, provides both weakly and strongly-typed GP, so, if you are familiar with Python, you may want to start from it.&lt;/p&gt;&#xA;&lt;p&gt;To conclude and answer your question more directly, it's not true that in TGP you need all functions to have the same type for all parameters and return values.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T18:38:50.697" LastActivityDate="2021-01-19T18:38:50.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25858" PostTypeId="5" CreationDate="2021-01-19T18:50:13.900" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T18:50:13.900" LastActivityDate="2021-01-19T18:50:13.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25859" PostTypeId="4" CreationDate="2021-01-19T18:50:13.900" Score="0" Body="For questions related to tree-based genetic programming (TGP), which is one of the first approaches to GP." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T18:50:13.900" LastActivityDate="2021-01-19T18:50:13.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25860" PostTypeId="1" CreationDate="2021-01-19T20:23:59.823" Score="2" ViewCount="63" Body="&lt;p&gt;I'm a bit confused about the visualization of the upper bound (following the notation of (c.f. &lt;a href=&quot;https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sutton &amp;amp; Barto (2018)&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_t(a)+C\sqrt{\frac{\mathrm{ln}(t)}{N_t(a)}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In many blog posts about the UCB(1)-algorithm, such as visualized in the following image (c.f. &lt;a href=&quot;https://www.geeksforgeeks.org/upper-confidence-bound-algorithm-in-reinforcement-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt; ):&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://media.geeksforgeeks.org/wp-content/uploads/20200126022459/Screenshot-2020-01-26-at-2.24.37-AM.png&quot; alt=&quot;Text&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Isn't the upper (confidence) bound simply the upper bound of a &lt;strong&gt;one-sided confidence interval&lt;/strong&gt; instead of a &lt;strong&gt;two-sided confidence interval&lt;/strong&gt; as shown in the image above?&#xA;A lower bound of the interval is completely useless in this case, or am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="21287" LastEditorUserId="2444" LastEditDate="2021-01-19T22:09:48.250" LastActivityDate="2021-01-31T18:52:12.247" Title="In UCB, is the actual upper bound an upper bound of an one-sided or two-sided confidence interval?" Tags="&lt;machine-learning&gt;&lt;reinforcement-learning&gt;&lt;multi-armed-bandit&gt;&lt;upper-confidence-bound&gt;" AnswerCount="1" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25861" PostTypeId="1" CreationDate="2021-01-19T23:05:07.473" Score="1" ViewCount="40" Body="&lt;p&gt;I am trying to understand how batch normalization (BN) works in CNNs. Suppose I have a feature map tensor &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; of shape &lt;span class=&quot;math-container&quot;&gt;$(N, C, H, W)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the mini-batch size,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the number of channels, and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$H,W$&lt;/span&gt; is the spatial dimension of the tensor.&lt;/p&gt;&#xA;&lt;p&gt;Then it seems there could a few ways of going about this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Method 1:&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$T_{n,c,x,y} :=  \gamma*\frac {T_{c,x,y} - \mu_{x,y}} {\sqrt{\sigma^2_{x,y} + \epsilon}} + \beta$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mu_{x,y} = \frac{1}{NC}\sum_{n, c} T_{n,c,x,y}$&lt;/span&gt; is the mean for all channels &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; for each batch element &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; at spatial location &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; over the minibatch, and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma^2_{x,y} = \frac{1}{NC} \sum_{n, c} (T_{n, c,x,y}-\mu_{c})^2$&lt;/span&gt; is the variance of the minibatch for all channels &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; at spatial location &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Method 2:&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$T_{n,c,x,y} :=  \gamma*\frac {T_{c,x,y} - \mu_{c,x,y}} {\sqrt{\sigma^2_{c,x,y} + \epsilon}} + \beta$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mu_{c,x,y} = \frac{1}{N}\sum_{n} T_{n,c,x,y}$&lt;/span&gt; is the mean for a specific channels &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; for each batch element &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; at spatial location &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; over the minibatch, and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma^2_{c,x,y} = \frac{1}{N} \sum_{n} (T_{n, c,x,y}-\mu_{c})^2$&lt;/span&gt; is the variance of the minibatch for a channel &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; at spatial location &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Method 3:&lt;/strong&gt; For each channel &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; we compute the mean/variance over the entire spatial values for &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt; and apply the formula as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$T_{n, c,x,y} := \gamma*\frac {T_{n, c,x,y} - \mu_{c}} {\sqrt{\sigma^2_{c} + \epsilon}} + \beta$&lt;/span&gt;, where now &lt;span class=&quot;math-container&quot;&gt;$\mu_c = \frac{1}{NHW} \sum_{n,x,y} T_{n,c,x,y}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2{_c} = \frac{1}{NHW} \sum_{n,x,y} (T_{n,c,x,y}-\mu_c)^2 $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In practice which of these methods is used (if any) are correct for?&lt;/p&gt;&#xA;&lt;p&gt;The original paper on batch normalization , &lt;a href=&quot;https://arxiv.org/pdf/1502.03167.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1502.03167.pdf&lt;/a&gt; , states on page 5 section 3.2, last paragraph, left side of the page:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For convolutional layers, we additionally want the normalization to&#xA;obey the convolutional property – so that different elements of the&#xA;same feature map, at different locations, are normalized in the same&#xA;way. To achieve this, we jointly normalize all the activations in a&#xA;minibatch, over all locations. In Alg. 1, we let &lt;span class=&quot;math-container&quot;&gt;$\mathcal{B}$&lt;/span&gt; be the set of all&#xA;values in a feature map across both the elements of a mini-batch and&#xA;spatial locations – so for a mini-batch of size &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and feature maps of&#xA;size &lt;span class=&quot;math-container&quot;&gt;$p \times q$&lt;/span&gt;, we use the effective mini-batch of size &lt;span class=&quot;math-container&quot;&gt;$m^\prime = \vert \mathcal{B} \vert = m \cdot pq$&lt;/span&gt;. We learn a pair of parameters &lt;span class=&quot;math-container&quot;&gt;$\gamma^{(k)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta^{(k)}$&lt;/span&gt; per feature map,&#xA;rather than per activation. Alg. 2 is modified similarly, so that&#xA;during inference the BN transform applies the same linear&#xA;transformation to each activation in a given feature map.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm not sure what the authors mean by &amp;quot;per feature map&amp;quot;, does this mean per channel?&lt;/p&gt;&#xA;" OwnerUserId="30962" LastEditorUserId="30962" LastEditDate="2021-01-20T04:23:05.340" LastActivityDate="2021-01-20T04:23:05.340" Title="Understanding Batch Normalization for CNNs" Tags="&lt;convolutional-neural-networks&gt;&lt;batch-normalization&gt;" AnswerCount="0" CommentCount="2" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25862" PostTypeId="5" CreationDate="2021-01-19T23:55:06.023" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T23:55:06.023" LastActivityDate="2021-01-19T23:55:06.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25863" PostTypeId="4" CreationDate="2021-01-19T23:55:06.023" Score="0" Body="For questions related to layers of neural networks, so you can use this tag if your question involves the input, output, hidden, feedforward, recurrent, convolutional, pooling (aka subsampling), upsampling, etc., layers." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-19T23:55:06.023" LastActivityDate="2021-01-19T23:55:06.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25864" PostTypeId="1" CreationDate="2021-01-20T07:43:17.863" Score="3" ViewCount="57" Body="&lt;p&gt;The pseudocode below is taken from Barto and Sutton's &amp;quot;Reinforcement Learning: an introduction&amp;quot;. It shows an actor-critic implementation with eligibility traces. My question is: if I set &lt;span class=&quot;math-container&quot;&gt;$\lambda^{\theta}=1$&lt;/span&gt; and replace &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; with the immediate reward &lt;span class=&quot;math-container&quot;&gt;$R_t$&lt;/span&gt;, do I get a backwards implementation of REINFORCE?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dSgKV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dSgKV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43952" LastEditorUserId="2444" LastEditDate="2021-01-20T12:32:35.883" LastActivityDate="2021-01-20T12:32:35.883" Title="How to implement REINFORCE with eligibility traces?" Tags="&lt;reinforcement-learning&gt;&lt;actor-critic-methods&gt;&lt;reinforce&gt;&lt;eligibility-traces&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25866" PostTypeId="5" CreationDate="2021-01-20T10:53:42.390" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T10:53:42.390" LastActivityDate="2021-01-20T10:53:42.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25867" PostTypeId="4" CreationDate="2021-01-20T10:53:42.390" Score="0" Body="For questions related to Cortana, a virtual assistant developed by Microsoft, in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T10:53:42.390" LastActivityDate="2021-01-20T10:53:42.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25868" PostTypeId="5" CreationDate="2021-01-20T10:59:14.757" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T10:59:14.757" LastActivityDate="2021-01-20T10:59:14.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25869" PostTypeId="4" CreationDate="2021-01-20T10:59:14.757" Score="0" Body="For questions related to Siri, a virtual assistant that is part of Apple Inc.'s iOS, iPadOS, watchOS, macOS, and tvOS operating systems, in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T10:59:14.757" LastActivityDate="2021-01-20T10:59:14.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25870" PostTypeId="1" CreationDate="2021-01-20T11:57:51.703" Score="0" ViewCount="15" Body="&lt;p&gt;I've been doing a search for tracking systems based on bounding box for several days (like Sort, DeepSort, Motpy, ...). But neither is nonrealtime.&lt;/p&gt;&#xA;&lt;p&gt;My problem is based on the tracking of multiple objects in a video that is not in real time, my idea is to first process all the video making the detections and then apply the tracking (assuming that it is able to see the past, present and future) .&lt;/p&gt;&#xA;&lt;p&gt;Also, I dont discard use features like DeepSort, to do the matchings.&lt;/p&gt;&#xA;&lt;p&gt;But I can't find any algorithm that has nonrealtime tracking (that is, they only use past and present information).&lt;/p&gt;&#xA;&lt;p&gt;My interest is to improve the tracking as much as possible.&lt;/p&gt;&#xA;" OwnerUserId="43965" LastActivityDate="2021-01-20T11:57:51.703" Title="Nonrealtime tracking for object detection (with or without deep features)" Tags="&lt;machine-learning&gt;&lt;deep-learning&gt;&lt;computer-vision&gt;&lt;object-detection&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25871" PostTypeId="2" ParentId="18084" CreationDate="2021-01-20T12:59:54.857" Score="0" Body="&lt;p&gt;&lt;strong&gt;This is how I see it:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The state that a purely reactive agent is reacting to is, in fact, a subset of the set of all possible runs that end with a state. So in theory, &lt;em&gt;E&lt;/em&gt; (some state) is a subset of &lt;em&gt;R&lt;/em&gt; (finite set of all possible runs with a state as the last element). Standard and purely reactive agents are similar in the sense that the agents' purpose is to commit an action given some condition. It is entirely possible that a standard agent might be dealing with a run which encompasses only one element. Therefore, this would imply that for a discrete event in time, the standard agent's behaviour would be equivalent to a purely reactive agent, right? or am I completely off the rails?&lt;/p&gt;&#xA;&lt;p&gt;Sorry for not using the required symbols, I'm new here and am getting used to the styling :D&lt;/p&gt;&#xA;" OwnerUserId="43966" LastActivityDate="2021-01-20T12:59:54.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25872" PostTypeId="1" AcceptedAnswerId="25873" CreationDate="2021-01-20T14:50:17.113" Score="1" ViewCount="167" Body="&lt;p&gt;In reinforcement learning (RL), the reward function (RF), which &lt;a href=&quot;https://ai.stackexchange.com/q/10442/2444&quot;&gt;can be denoted as &lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r(s, s')$&lt;/span&gt; depending on its specific definition&lt;/a&gt;, provides the learning signal, which guides the RL algorithm/agent towards the desirable behaviour. As opposed to a loss (or cost) function in supervised learning, an RF determines the &lt;em&gt;goodness&lt;/em&gt; (or &lt;em&gt;quality&lt;/em&gt;) of an action in a certain state (in the case of &lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt;) with respect to an optimal behavior/policy. The reward function is typically hand-designed, but it can also be learned with &lt;a href=&quot;https://ai.stanford.edu/%7Eang/papers/icml00-irl.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverse RL techniques&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In evolutionary algorithms, a &lt;a href=&quot;https://ai.stackexchange.com/a/25028/2444&quot;&gt;fitness function (FF)&lt;/a&gt; is the function we use to evaluate an individual (or chromosome). The fitness of an individual, a number, represents the quality of that individual.&lt;/p&gt;&#xA;&lt;p&gt;Both the fitness function and the reward function are very similar, so what's the difference between them? Can a fitness function be thought of as a reward function? Can it be thought of as a cost function in SL problems?&lt;/p&gt;&#xA;" OwnerUserId="2444" LastActivityDate="2021-02-17T15:44:37.737" Title="What is the difference between a fitness function and a reward function?" Tags="&lt;reinforcement-learning&gt;&lt;loss-functions&gt;&lt;evolutionary-algorithms&gt;&lt;reward-functions&gt;&lt;fitness-functions&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25873" PostTypeId="2" ParentId="25872" CreationDate="2021-01-20T14:50:17.113" Score="1" Body="&lt;p&gt;In many cases, a fitness function (FF) is indeed similar to a reward function (RF), but, in other cases, it's more similar to a cost function (CF) as used in supervised learning (SL), and I explain below why. The FF, RF, and CF are used to evaluate the individuals, actions, and predictions, respectively, hence they can all be thought of as &lt;em&gt;evaluation&lt;/em&gt; functions (although this latter term has a more specific meaning in the context of search algorithms, like A*). Their primary difference is their &lt;em&gt;purpose&lt;/em&gt; and how they are used (i.e. for which problems they are used or what they evaluate), and not much what they return, which is typically a number (in all cases).&lt;/p&gt;&#xA;&lt;h3&gt;Reward function&lt;/h3&gt;&#xA;&lt;p&gt;As stated in the question, an RF can be viewed as evaluating different situations, depending on its definition or what we are interested in.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt; evaluates the goodness of being state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s, a)$&lt;/span&gt; evaluates the goodness of taking &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in the state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s, s')$&lt;/span&gt; evaluates the goodness of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, at the current state time step, then, independently of the action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; taken in &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, ending up in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s')$&lt;/span&gt; similar to &lt;span class=&quot;math-container&quot;&gt;$r(s)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$r(s, a, s')$&lt;/span&gt; the quality of being in state &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, taking action &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and ending up in state &lt;span class=&quot;math-container&quot;&gt;$s'$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Although I will not dwell on the details in this answer (and you can find more details &lt;a href=&quot;https://ai.stackexchange.com/q/10442/2444&quot;&gt;here&lt;/a&gt;), depending on the problem or the RL algorithms that you want to use to solve the problem, one definition may be more interesting or useful than another.&lt;/p&gt;&#xA;&lt;h3&gt;Fitness function&lt;/h3&gt;&#xA;&lt;p&gt;In evolutionary algorithms (EA), such as genetic algorithms (GA) or genetic programming (GP), the solutions are generally known as &lt;strong&gt;individuals&lt;/strong&gt;, which can be represented in different ways depending on the specific approach.&lt;/p&gt;&#xA;&lt;p&gt;However, in all cases, we often distinguish between &lt;strong&gt;genotypes&lt;/strong&gt; (aka &lt;em&gt;genomes&lt;/em&gt; or &lt;em&gt;chromosomes&lt;/em&gt;), which are representations of the actual solutions to a problem, which are more amenable to changes (i.e. to the mutation and crossover operations), and &lt;strong&gt;phenotypes&lt;/strong&gt;, which are the actual solutions to the problem. The reason why we use genotypes is that, often, it's easier to find solutions in a different but related space rather than in the original space of solutions, although some EAs can also just work with phenotypes.&lt;/p&gt;&#xA;&lt;p&gt;To be more concrete, in traditional genetic algorithms, the genotypes are binary vectors &lt;span class=&quot;math-container&quot;&gt;$\{0, 1 \}^N$&lt;/span&gt;, which restricts the space of possible solutions, which, in certain cases, it's a good thing, because the original space may be too huge (so it may be harder to converge to a good solution). These binary vectors can then be converted to the actual solutions using some encoding-decoding mechanism that we need to specify depending on the problem. More precisely, maybe each element of a binary vector corresponds to some binary choice in some game where you need to take a sequence of choices, and each time you can either say &amp;quot;yes&amp;quot; or &amp;quot;no&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;So, in EAs, an FF evaluates the quality (or fitness) of an individual and, more precisely, either a genotype or a phenotype (depending on the representation of the solutions).&lt;/p&gt;&#xA;&lt;p&gt;Often, the fitness function is really a cost function (CF), like in supervised learning. For example, if you want to solve a &lt;em&gt;symbolic regression&lt;/em&gt; problem using genetic programming (i.e. find an analytical or closed-form expression of a function), then the fitness function is really just a cost function, i.e. an individual would be like a prediction and the label would be generated by the actual true function (provided that we know it). More concretely, let's say that we want to find the analytical expression&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) = x^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this case, we can generate a labeled dataset of input-output pairs by just evaluating &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; at different inputs, so we may get the following labeled dataset&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D = \{(1, 1), (2, 4), (3, 9) \}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can then evaluate an individual, which in the case of GP is a function (or program) &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt;, using e.g. the mean squared error (MSE). For example, if we have the individual &lt;span class=&quot;math-container&quot;&gt;$g(x) = x^3$&lt;/span&gt;, we can compute the mean squared error as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{MSE}(D, g(x)) = \frac{1}{3} \left( (1 - 1)^2 + (4 - 8)^2 + (9  - 27)^2 \right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, in this case, the FF is the MSE. Of course, we could have bigger datasets than &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. In any case, the point is that you can use GP to solve SL problems.&lt;/p&gt;&#xA;&lt;p&gt;You could also use EAs to solve other types of problems, i.e. not just SL problems, for instance, RL or unsupervised learning problems. In the case of RL problems, the FF is more similar to an RF in RL, given that you will probably need to define the FF as a function that evaluates the qualities of actions, states, or combinations of these.&lt;/p&gt;&#xA;&lt;p&gt;So, to conclude, the definition of the FF in EAs depends on the problem that you want to solve, and it can be more similar to an RF or CF, depending on the problem you want to solve.&lt;/p&gt;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&lt;p&gt;If you are interested in how to design reward functions or fitness functions for your problem, you can take a look at &lt;a href=&quot;https://ai.stackexchange.com/q/22851/2444&quot;&gt;this&lt;/a&gt; (and &lt;a href=&quot;https://ai.stackexchange.com/q/12264/2444&quot;&gt;this&lt;/a&gt;) and &lt;a href=&quot;https://ai.stackexchange.com/q/9105/2444&quot;&gt;this&lt;/a&gt; questions, respectively. &lt;a href=&quot;https://ai.stackexchange.com/a/21585/2444&quot;&gt;This answer&lt;/a&gt; provides a brief explanation of the difference between RL and EAs.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-02-17T15:44:37.737" LastActivityDate="2021-02-17T15:44:37.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25874" PostTypeId="1" CreationDate="2021-01-20T17:03:02.650" Score="1" ViewCount="38" Body="&lt;p&gt;I was watching a lecture on Policy Gradient and had difficulty following it when importance sampling was introduced.&lt;/p&gt;&#xA;&lt;p&gt;It was shown that the gradient of the objective can be written as &lt;span class=&quot;math-container&quot;&gt;$$\nabla_\theta U(\theta) = E_{\tau \sim \pi_\theta}[\nabla_\theta P(\tau; \theta) R(\tau)],$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$P(\tau)$&lt;/span&gt; denotes the probability of getting the trajectory &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; and reward &lt;span class=&quot;math-container&quot;&gt;$R(\tau) = \sum_i R(\tau^{(i)})$&lt;/span&gt; is the sum of the rewards for trajectory.&lt;/p&gt;&#xA;&lt;p&gt;Then comes the part of my confusion. It was demonstrated that the same result can be derived with importance sampling and have no idea what this bit means (&lt;a href=&quot;https://youtu.be/S_gwYj1Q-44?t=1404&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/S_gwYj1Q-44?t=1404&lt;/a&gt; and onwards until the next slide, takes around a minute). It's said:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If we're curious about other policies, even if we have trajectories from an old policy, we can understand how good this other policy is going to be, let's look locally if we deviate from &lt;span class=&quot;math-container&quot;&gt;$\theta_{old}$&lt;/span&gt;, and it gives us the same gradient&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But I don't see how:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The gradient of new policy evaluated at &lt;span class=&quot;math-container&quot;&gt;$\theta_{old}$&lt;/span&gt; is the same as the gradient (I think he means in the same form as the derivation without importance sampling) at the location &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We even can evaluate the gradient of a policy at a different location other than &lt;span class=&quot;math-container&quot;&gt;$\theta_{old}$&lt;/span&gt;, which we currently have. We can find the derivatives at out current location, but we cannot do that if we don't have analytical expression of the function. He mentions finite perturbation, which I think achieves this, but I don't see how. To this, he also says:&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Also, you can do more, if we use the original derivation of the gradient, we can change the gradient and then use this equation to estimate how good the update was rather than just blindly trusting your gradient put you in the good spot.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In general, I'd like to get a better picture of the benefits of formulating the problem in this way&lt;/p&gt;&#xA;" OwnerUserId="43973" LastEditorUserId="2444" LastEditDate="2021-01-23T15:07:15.870" LastActivityDate="2021-01-23T15:07:15.870" Title="Understanding policy gradient derivation with importance sampling" Tags="&lt;reinforcement-learning&gt;&lt;policy-gradients&gt;&lt;off-policy-methods&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25876" PostTypeId="1" CreationDate="2021-01-20T17:28:23.907" Score="1" ViewCount="67" Body="&lt;p&gt;The code below is adapted from &lt;a href=&quot;https://github.com/kamenbliznashki/sutton_barto/blob/master/tiles3.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this implementation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from math import floor&#xA;&#xA;basehash = hash&#xA;&#xA;class IHT:&#xA;    &amp;quot;Structure to handle collisions&amp;quot;&#xA;    def __init__(self, sizeval):&#xA;        self.size = sizeval&#xA;        self.overfullCount = 0&#xA;        self.dictionary = {}&#xA;&#xA;    def count (self):&#xA;        return len(self.dictionary)&#xA;&#xA;    def getindex (self, obj, readonly=False):&#xA;        d = self.dictionary&#xA;        if obj in d: return d[obj]&#xA;        elif readonly: return None&#xA;        size = self.size&#xA;        count = self.count()&#xA;        if count &amp;gt;= size:&#xA;            if self.overfullCount==0: print('IHT full, starting to allow collisions')&#xA;            self.overfullCount += 1&#xA;            return basehash(obj) % self.size&#xA;        else:&#xA;            d[obj] = count&#xA;            return count&#xA;&#xA;def hashcoords(coordinates, m, readonly=False):&#xA;    if type(m)==IHT: return m.getindex(tuple(coordinates), readonly)&#xA;    if type(m)==int: return basehash(tuple(coordinates)) % m&#xA;    if m==None: return coordinates&#xA;&#xA;&#xA;def tiles(ihtORsize, numtilings, floats, ints=[], readonly=False):&#xA;    &amp;quot;&amp;quot;&amp;quot;returns num-tilings tile indices corresponding to the floats and ints&amp;quot;&amp;quot;&amp;quot;&#xA;    qfloats = [floor(f*numtilings) for f in floats]&#xA;    Tiles = []&#xA;    for tiling in range(numtilings):&#xA;        tilingX2 = tiling*2&#xA;        coords = [tiling]&#xA;        b = tiling&#xA;        for q in qfloats:&#xA;            coords.append( (q + b) // numtilings )&#xA;            b += tilingX2&#xA;        coords.extend(ints)&#xA;        Tiles.append(hashcoords(coords, ihtORsize, readonly))&#xA;    return Tiles&#xA;&#xA;if __name__ == '__main__':&#xA;    tc=IHT(4096)&#xA;    tiles = tiles(tc, 8, [0, 0.5], ints=[], readonly=False)&#xA;    print(tiles)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm trying to figure out how the function &lt;code&gt;tiles()&lt;/code&gt; works. It implements tile coding, which is explained in &lt;a href=&quot;http://incompleteideas.net/book/RLbook2020.pdf#page=239&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Reinforcement Learning: An Introduction&amp;quot; (2020) Sutton and Barto on page 217&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So far I've figured out:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;qfloats&lt;/code&gt; rescales the floating numbers to be the largest integer less than or equal to the original floating number, these are then re-scaled by the number of tilings.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then, for each tiling, a list is created, the first element of the list is the tiling number, followed by the coordinates of the floats for that tiling, i.e. each of the re-scaled numbers is offset by the tiling number, &lt;code&gt;b&lt;/code&gt; and integer divided by &lt;code&gt;numtilings&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, &lt;code&gt;hashcoords&lt;/code&gt; first checks the dictionary &lt;code&gt;d&lt;/code&gt; to see if this list has appeared before, if it has it returns the number relating to that list. It not it either creates a new entry with that list at the key and the count as the value or if the count is more than or equal to the size it adds one to &lt;code&gt;overfullCount&lt;/code&gt; and returns &lt;code&gt;basehash(obj) % self.size&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm struggling to understand two parts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the &lt;code&gt;tilingX2&lt;/code&gt; part doing?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is &lt;code&gt;tilingX2&lt;/code&gt; added to &lt;code&gt;b&lt;/code&gt; after the first coordinate has been calculated? It seems to me that each coordinate should be treated separately&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And why by a factor of 2?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the expression &lt;code&gt;basehash(obj) % self.size&lt;/code&gt; doing? I'm quite new to the concept of hashing. I know that, generally, they create a unique number for a given input (up to a limit), but I'm really struggling to understand what is going on in the line above.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="43975" LastEditorUserId="2444" LastEditDate="2021-01-21T02:41:47.563" LastActivityDate="2021-01-21T02:41:47.563" Title="Can someone explain to me this implementation of Tile Coding using Hash Tables?" Tags="&lt;reinforcement-learning&gt;&lt;python&gt;&lt;implementation&gt;&lt;function-approximation&gt;&lt;tile-coding&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25877" PostTypeId="1" CreationDate="2021-01-20T19:00:09.200" Score="1" ViewCount="37" Body="&lt;p&gt;I cannot make SAC learn a task in a certain environment. The point is that it actually sometimes finds a very good policy, but it never learns the policy in the end. I am using the SAC implementation from stable-baselines3, which is correct as far as I have seen.&#xA;I have an environment driven by complex dynamics. I have to control a variable to be in a certain range. Every time the variable goes out of minimum or maximum range the environment is done. The action is continuous (between 0 and 30). My goal is to keep the variable (1D) in the range for as long as possible (millions of steps per episode would be ideal). There are certain characteristics of the environment that may make it particular:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The action can only drive the variable to lower values. The variable can go up as a result of the environment dynamics (not controlled) and as a consequence of certain events (not controlled) that occur at random intervals.&lt;/li&gt;&#xA;&lt;li&gt;The observation is a noisy sample of the variable. The observation is just a real number.&lt;/li&gt;&#xA;&lt;li&gt;The effect of actions in the variable is usually delayed. That is, applying an action does not immediately lower the value of the variable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have tried SAC with many different hyperparameters. It sometimes find very good policies, policies that last for thousands and even millions of steps in evaluation or rollout. But it never learns such policies. Even saving the policy in those cases, they are not able to produce a lone episode later. In the attached image, it can be seen that during the training (in some evaluations) the policy is able to run for thousand of steps. But then it never learns that. I only show 500K here steps but I have run test for 1.5 million training timesteps.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XFvtc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XFvtc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, my question is (I have several ones actually):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is SAC not suitable for this problem? I have also run TD3 and PPO but without better results and SAC is the only one actually able to find those  policies that make very long episodes. Any other algorithm?&lt;/li&gt;&#xA;&lt;li&gt;I have tried several reward functions, and, in the end, a simple one that gives 1 for every step and 0 when done is the one that seems to give better results. In the image, the reward is one for every step and -100 when done.&lt;/li&gt;&#xA;&lt;li&gt;Since the values of the variable are time correlated due to the dynamics, I have also tried with RNN actors (with TF Agents), but results do not improve.&lt;/li&gt;&#xA;&lt;li&gt;I cannot see any relationship between the actor loss and critic loss and the results (maybe that is my problem). The loss seem to be larger when the episodes are longer (which is what I want).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any advice is highly appreciated. Thanks&lt;/p&gt;&#xA;" OwnerUserId="43977" LastActivityDate="2021-01-20T19:00:09.200" Title="How to make SAC (Soft-Actor-Critic) learn a policy?" Tags="&lt;loss-functions&gt;&lt;continuous-action-spaces&gt;&lt;soft-actor-critic&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25878" PostTypeId="1" CreationDate="2021-01-20T19:32:34.643" Score="0" ViewCount="39" Body="&lt;p&gt;I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given output.&lt;/p&gt;&#xA;" OwnerUserId="43980" LastEditorUserId="2444" LastEditDate="2021-01-20T22:13:19.743" LastActivityDate="2021-02-19T23:03:35.557" Title="How to make input variable as trainable parameter in a neural network?" Tags="&lt;neural-networks&gt;&lt;optimization&gt;&lt;inverse-rl&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25879" PostTypeId="2" ParentId="25878" CreationDate="2021-01-20T20:41:41.687" Score="0" Body="&lt;p&gt;It's just a typical optimization problem. You want to optimize function&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;f(\theta, x)&#xA;\end{equation}&lt;/span&gt;&#xA;with fixed parameters &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; (network weights) and optimization variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Typically the approach is consisted of choosing a search direction and direction step. For deciding search direction you can use the gradient of objective with respect to decision variable and have an update of form&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;x = x - \alpha \nabla_x f(\theta, x)&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\nabla_x f(\theta, x)$&lt;/span&gt; is the gradient. You can also use Newton or Quasi-Newton methods of form&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;x = x - \alpha B^{-1}\nabla_x f(\theta, x)&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is a Hessian or approximate Hessian of &lt;span class=&quot;math-container&quot;&gt;$f(\theta, x)$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Step length parameter is usually decided with line search or you can also use trust region approaches which decide step direction with constraints to it's length. For more details about optimization you can consult &lt;em&gt;Numerical Optimization&lt;/em&gt; by Nocedal and Wright&lt;/p&gt;&#xA;" OwnerUserId="20339" LastActivityDate="2021-01-20T20:41:41.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25881" PostTypeId="5" CreationDate="2021-01-20T22:37:33.827" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T22:37:33.827" LastActivityDate="2021-01-20T22:37:33.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25882" PostTypeId="4" CreationDate="2021-01-20T22:37:33.827" Score="0" Body="For questions related to Markov Reward Processes (MRPs) in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-20T22:37:33.827" LastActivityDate="2021-01-20T22:37:33.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25883" PostTypeId="5" CreationDate="2021-01-21T00:08:45.700" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T00:08:45.700" LastActivityDate="2021-01-21T00:08:45.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25884" PostTypeId="4" CreationDate="2021-01-21T00:08:45.700" Score="0" Body="For questions related to validation datasets, which are used for hyper-parameter optimization, early stopping or cross-validation." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T00:08:45.700" LastActivityDate="2021-01-21T00:08:45.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25885" PostTypeId="2" ParentId="25849" CreationDate="2021-01-21T00:35:30.483" Score="0" Body="&lt;p&gt;In the end, this problem turned out to be largely a matter of the gradient descent falling into local minima.&lt;/p&gt;&#xA;&lt;p&gt;For those reading for posterity, one of the issues in ML that is difficult to work around is that we cannot intuitively choose reasonable initial values for the weights, biases, and kernels (in the CNN).  As a result, we typically allow them to initialize randomly.  This can present some challenges.&lt;/p&gt;&#xA;&lt;p&gt;One of the biggest challenges is that when you start from a random starting point, it's difficult to tell someone how to completely replicate your experiments.  This isn't terribly important in the end since you can provide them with the saved parameters from your trained model.  However, this can also lead to networks that appear to be &amp;quot;bad&amp;quot; that are in fact perfectly fine.&lt;/p&gt;&#xA;&lt;p&gt;In this case, I had spent much of the time initializing the CNN with a uniform initializer (not present in the code above).  I will sometimes use a random seed or some other function to generate initial values so that I can better improve networks through genetic search tools.&lt;/p&gt;&#xA;&lt;p&gt;It seems that the uniform initializers combined with the various network iterations and this particular data lead to absolutely abysmal training performance and non-convergence.&lt;/p&gt;&#xA;&lt;p&gt;When I ran the network as above with random initializations and one or two tweaks, it converged well.  Some training iterations will pin one of the sides of the bounding box at the edge, some will never converge, but I've managed to successfully train several that are in the 96-98% accuracy range for the bounding boxes in my test set of 20000, so all is well!&lt;/p&gt;&#xA;" OwnerUserId="30426" LastActivityDate="2021-01-21T00:35:30.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25886" PostTypeId="5" CreationDate="2021-01-21T01:58:31.337" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T01:58:31.337" LastActivityDate="2021-01-21T01:58:31.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25887" PostTypeId="4" CreationDate="2021-01-21T01:58:31.337" Score="0" Body="For questions related to the concept of a bounding box in object detection or other computer vision tasks." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T01:58:31.337" LastActivityDate="2021-01-21T01:58:31.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25888" PostTypeId="5" CreationDate="2021-01-21T02:03:55.603" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T02:03:55.603" LastActivityDate="2021-01-21T02:03:55.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25889" PostTypeId="4" CreationDate="2021-01-21T02:03:55.603" Score="0" Body="For questions related to data compression in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T02:03:55.603" LastActivityDate="2021-01-21T02:03:55.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25890" PostTypeId="5" CreationDate="2021-01-21T02:23:20.000" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T02:23:20.000" LastActivityDate="2021-01-21T02:23:20.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25891" PostTypeId="4" CreationDate="2021-01-21T02:23:20.000" Score="0" Body="For questions related to tile coding, a form of coarse coding that is particularly well suited for use on sequential digital computers and for efficient online learning." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-21T02:23:20.000" LastActivityDate="2021-01-21T02:23:20.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25892" PostTypeId="1" CreationDate="2021-01-21T02:34:05.683" Score="1" ViewCount="82" Body="&lt;p&gt;Dall-E, it can generate many imaginative images from the description, even some peculiar images, how did they actually create this kind of dataset to train this AI , because there is not much of that kind of data which include weird images and descriptive text, how did they create this massive dataset. Does anyone have any idea?&lt;/p&gt;&#xA;&lt;p&gt;If you have no idea what I am talking about, please refer to this link: &lt;a href=&quot;https://openai.com/blog/dall-e/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openai.com/blog/dall-e/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="42182" LastEditorUserId="42182" LastEditDate="2021-01-23T03:27:25.563" LastActivityDate="2021-01-23T03:27:25.563" Title="What dataset might Elon Musk's Dall-E have used?" Tags="&lt;natural-language-processing&gt;&lt;datasets&gt;&lt;generative-adversarial-networks&gt;&lt;image-generation&gt;&lt;natural-language-understanding&gt;" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25893" PostTypeId="1" AcceptedAnswerId="25910" CreationDate="2021-01-21T02:42:02.513" Score="0" ViewCount="76" Body="&lt;p&gt;I have a problem similar to the vehicle routing problem (VRP) that I want to solve with reinforcement learning. In this problem, the agent starts from the point &lt;span class=&quot;math-container&quot;&gt;$(x_0, y_0)$&lt;/span&gt;, then it needs to travel through &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; other points, &lt;span class=&quot;math-container&quot;&gt;$(x_1, y_1), \dots, (x_n, y_n)$&lt;/span&gt;. The goal is to minimize the distance traveled.&lt;/p&gt;&#xA;&lt;p&gt;Right now, I am modeling a state as a point &lt;span class=&quot;math-container&quot;&gt;$(x, y)$&lt;/span&gt;. There are 8 possible actions: go east, go north-east, go north, go north-west, go-west, go south-west, go south, go south-east. Each action goes by a pace of 100 metres.&lt;/p&gt;&#xA;&lt;p&gt;After reaching near a destination point, that destination point is removed from the list of destination points.&lt;/p&gt;&#xA;&lt;p&gt;The reward is the reciprocal of total distance until all destination points reached (there's a short optimisation to arrange the remaining points for a better reward).&lt;/p&gt;&#xA;&lt;p&gt;I'm using a DNN model to keep the policy of a reinforcement learning agent, so this DNN maps a certain state to suitable action.&lt;/p&gt;&#xA;&lt;p&gt;However, after every action of the agent with a good reward, the training data are added with 1 more sample, it's kinda incremental learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Should the policy model be trained again and again with every new sample added in&lt;/strong&gt;? This does take too much time.&lt;/p&gt;&#xA;&lt;p&gt;Any better RL approach to the problem above?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2021-01-21T12:16:56.483" LastActivityDate="2021-01-28T02:48:14.553" Title="How to train a policy model incrementally to solve a problem similar to the vehicle routing problem?" Tags="&lt;reinforcement-learning&gt;&lt;training&gt;&lt;deep-rl&gt;&lt;incremental-learning&gt;&lt;travelling-salesman-problem&gt;" AnswerCount="1" CommentCount="9" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25894" PostTypeId="2" ParentId="25892" CreationDate="2021-01-21T03:27:58.750" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;DALL·E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;should be the same data they used to train the GPT-3&lt;/p&gt;&#xA;" OwnerUserId="43989" LastActivityDate="2021-01-21T03:27:58.750" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25895" PostTypeId="2" ParentId="25795" CreationDate="2021-01-21T05:06:37.260" Score="1" Body="&lt;p&gt;Alright. Consider an ordinary neural network, so, in the last layer, we have, &lt;span class=&quot;math-container&quot;&gt;$z^{[L]} = W^{[L]} a^{[L-1]} + b^{[L]}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a^{[L]} = \sigma(z^{[L]})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is the softmax activation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma(\mathbf z)_{i} = \frac{e^{z_i}}{\sum_k e^{z_k}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think, one of the most effective ways of not to get confused about all these matrices with different dimensions, is to simply get rid of the matrices and do all calculations componentwise. So, calculating the &lt;span class=&quot;math-container&quot;&gt;$ij$&lt;/span&gt; component of the jacobian matrix, we have:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_{ij} = \frac{\partial\sigma_i}{\partial z_j} = \frac{e^{z_i}}{\sum_k e^{z_k}}\delta_{ij} - \frac{e^{z_i}e^{z_j}}{(\sum_k e^{z_k})^2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Consider a cost &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, and we wish to calculate the derivative with respect to the weights of the last layer, that is, we are running the first step of backpropagation. Applying chain rule to the &lt;span class=&quot;math-container&quot;&gt;$ij$&lt;/span&gt;-component of the weight:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial C}{\partial W^{[L]}_{ij}} &#xA;= \sum_k\frac{\partial C}{\partial a_{k}}\frac{\partial a^{[L]}_{k}}{\partial W^{[L]}_{ij}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And, with one more chain rule:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial a^{[L]}_{k}}{\partial W^{[L]}_{ij}} = &#xA;\sum_s\frac{\partial a^{[L]}_{k}}{\partial z^{[L]}_s}\frac{\partial z^{[L]}_s}{\partial W^{[L]}_{ij}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now we can identify exactly where the jacobian matrix is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_{ij} = \frac{\partial\sigma_i}{\partial z_j} = \frac{\partial a^{[L]}_{i}}{\partial z^{[L]}_j}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus, the back-propagation step is just:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial C}{\partial W^{[L]}_{ij}} &#xA;= \sum_k\sum_s\frac{\partial C}{\partial a_{k}}\frac{\partial a^{[L]}_{k}}{\partial z^{[L]}_s}\frac{\partial z^{[L]}_s}{\partial W^{[L]}_{ij}}&#xA;= \sum_k\sum_s\frac{\partial C}{\partial a_{k}}J_{ks}\frac{\partial z^{[L]}_s}{\partial W^{[L]}_{ij}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, in component language, it is much simpler to identify where are all the matrix multiplications: The transpose of the gradient with respect to the cost function, multiplied by the jacobian matrix of the softmax activation (or whatever activation), and finally, the last derivative, which will evaluate to something depending on the activation of the previous layer.&lt;/p&gt;&#xA;&lt;p&gt;=].&lt;/p&gt;&#xA;" OwnerUserId="43988" LastEditorUserId="43988" LastEditDate="2021-01-22T00:06:00.443" LastActivityDate="2021-01-22T00:06:00.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25897" PostTypeId="2" ParentId="25750" CreationDate="2021-01-21T09:56:58.563" Score="0" Body="&lt;p&gt;I think U are looking for PLSA&#xA;for PLSA either U find out those topics(catogeries) with EM or NNMF&#xA;Personally I recommend NNMF&#xA;or u can use LDA which is Bayesian version of PLSA&lt;/p&gt;&#xA;&lt;p&gt;here is code for PLSA:&#xA;&lt;a href=&quot;https://github.com/Man-ash/Probabilistic-Latent-semantic-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Man-ash/Probabilistic-Latent-semantic-analysis&lt;/a&gt;&#xA;which use NNMF&lt;/p&gt;&#xA;&lt;p&gt;for EM method I code it by myself but i am not sure if it is right&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/65783880/problem-about-m-step-in-probabilistic-latent-semantic-analysis-with-code&quot;&gt;https://stackoverflow.com/questions/65783880/problem-about-m-step-in-probabilistic-latent-semantic-analysis-with-code&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43999" LastActivityDate="2021-01-21T09:56:58.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25899" PostTypeId="2" ParentId="25829" CreationDate="2021-01-21T11:28:29.083" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;but I have been told that neural networks aren't made to predict values in that way, they really are best suited for classification into discrete classes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't agree with this statement. I already trained many CNNs for regressions tasks where a continous output is trained and they generally perform very well.&lt;/p&gt;&#xA;&lt;p&gt;I think the general &amp;quot;advantage&amp;quot; for a classification approach over a regression approach is that there is usually some margin where the output of the NN can still be clipped to a specific class, which might not be the case in regression tasks. Furthermore a lot of the &amp;quot;better classification performance&amp;quot; is due to the fact that the cross-entropy loss of classification tasks always tries to increase probability of one class while reducing all others. But this advantage might not be really important in your case where a specific range of days (e.g 3-5 days) can be probable instead of just one single day (e.g. exactly day 4).&lt;/p&gt;&#xA;&lt;p&gt;Thats why I think you should defenitly try the regression approach. For this just normalize/clip your output layer to the min/max range of days. You should also defenitley decrease the learning rate at the end of training, I found this much further improves regression predictions to be more accurate.&lt;/p&gt;&#xA;&lt;p&gt;And lastly if you want to achieve higher performance and more robustness, in your case I highly recommend a bayesian approach where you average over an ensemble of NNs (this ensemble can be your single NN with different parts truned off via Dropout layers during inferencing). With this average you can get an estimate of the uncertainty (the variance over different predictions) which gives you a much better idea of the probability distribution over the different days for a specifiy input.&lt;/p&gt;&#xA;" OwnerUserId="13104" LastActivityDate="2021-01-21T11:28:29.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25901" PostTypeId="2" ParentId="25712" CreationDate="2021-01-21T12:20:27.273" Score="1" Body="&lt;p&gt;The more I think about it the more convinced I am that the visual explanation from the linked lecture is wrong. But the good news is there are still some ways to get close to the cylinder but not before the activation of the last neuron but instead afterwards. I haven't done it with simgoid. But I tried with ReLu instead for now.&lt;/p&gt;&#xA;&lt;p&gt;We can cut the tower at the very top (thanks to ReLu and a bias). The closer to the top we cut it the more it will be like a cylinder.&lt;/p&gt;&#xA;&lt;p&gt;We can control the height of the tower with the weights.&lt;/p&gt;&#xA;&lt;p&gt;First in 2d:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VRGzb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VRGzb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately the closer we put this towers together the more they will start two influence each other.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FyRwL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FyRwL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But we can counter that with a negative tower between them.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/isxkZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/isxkZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now in 3d:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KKUs3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KKUs3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This Answer is work in progress I will update it when i find out something new.&lt;/p&gt;&#xA;" OwnerUserId="43729" LastEditorUserId="43729" LastEditDate="2021-01-21T12:25:40.363" LastActivityDate="2021-01-21T12:25:40.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25903" PostTypeId="1" CreationDate="2021-01-21T13:40:27.147" Score="3" ViewCount="72" Body="&lt;p&gt;I have seeing a variation in importance sampling (IS) in Prioritized Experience Replay (PER) in some implementations regarding the original &lt;a href=&quot;https://arxiv.org/pdf/1511.05952.pdf#page=5&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; approach stated as (in section 3.4):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{i}=\left(\frac{1}{N} \cdot \frac{1}{P(i)}\right)^{\beta}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;w_{i}=\left(\frac{\min (P(i))}{P(i)}\right)^{\beta}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know where it comes from? A reference that explains the reason for that new formula and improvements obtained?&lt;/p&gt;&#xA;&lt;p&gt;My intuition guides me to some conclusions, not necessarily correct, using this new formula:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the beginning, supposing that the PER stills have empty positions, &lt;span class=&quot;math-container&quot;&gt;$\min(P(i)) \sim 0$&lt;/span&gt;, not giving too much weight for samples. But it grows substantially once the capacity is achieved as well as when the error becomes low (plus the incrementing Beta)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A code on github that applies this: &lt;a href=&quot;https://github.com/ljpzzz/machinelearning/blob/master/reinforcement-learning/ddqn_prioritised_replay.py#L120-L130&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="42682" LastEditorUserId="42682" LastEditDate="2021-01-29T16:24:27.367" LastActivityDate="2021-01-29T16:24:27.367" Title="Where does this variation of the importance sampling weight come from?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;experience-replay&gt;&lt;importance-sampling&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25904" PostTypeId="1" AcceptedAnswerId="26055" CreationDate="2021-01-21T14:40:00.603" Score="3" ViewCount="63" Body="&lt;p&gt;In my understanding, DQN is useful because it utilises a neural network as a q-value function approximator, which, after the training, can generalise to unseen states.&lt;/p&gt;&#xA;&lt;p&gt;I understand how that would work when the input is a vector of continuous values, however, I don't understand why DQN would be used with discrete state-spaces. If the input to the neural network is just an integer with no clear structure, how is this supposed to generalise?&lt;/p&gt;&#xA;&lt;p&gt;If, instead of feeding to the network just an integer, we fed a vector of integers, in which each element represents a characteristic of the state (separating things like speed, position, etc.) instead of collapsing everything in a single integer, would that generalise better?&lt;/p&gt;&#xA;" OwnerUserId="44004" LastEditorUserId="2444" LastEditDate="2021-01-22T21:21:28.110" LastActivityDate="2021-01-29T00:33:17.317" Title="Does DQN generalise to unseen states in the case of discrete state-spaces?" Tags="&lt;reinforcement-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;&lt;generalization&gt;&lt;discrete-state-spaces&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25905" PostTypeId="1" CreationDate="2021-01-21T18:34:05.157" Score="0" ViewCount="13" Body="&lt;p&gt;I am new to LSTMs and I was wondering if it is possible to have LSTM layer then dense then LSTM again and does it make sense?&lt;/p&gt;&#xA;" OwnerUserId="44010" LastActivityDate="2021-01-21T18:34:05.157" Title="Is it possible and if so does it make sense to have dense layers in between LSTM layers?" Tags="&lt;long-short-term-memory&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25906" PostTypeId="2" ParentId="2729" CreationDate="2021-01-21T20:19:14.593" Score="3" Body="&lt;p&gt;One way of verifying whether two boolean expressions are equivalent is to assign all possibilities to all variables, and comparing all results.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;A&lt;/th&gt;&#xA;&lt;th&gt;B&lt;/th&gt;&#xA;&lt;th&gt;f1&lt;/th&gt;&#xA;&lt;th&gt;f2&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;F&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;We can see &lt;code&gt;(F, F, F, T)&lt;/code&gt; does not equal &lt;code&gt;(F, T, T, T)&lt;/code&gt;, for example for the assignment &lt;code&gt;(A, B) = (T, F)&lt;/code&gt; we get result &lt;code&gt;(f1, f2) = (F, T)&lt;/code&gt; , meaning &lt;code&gt;f1 != f2&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21645" LastEditorUserId="2444" LastEditDate="2021-01-22T12:22:56.677" LastActivityDate="2021-01-22T12:22:56.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25907" PostTypeId="1" CreationDate="2021-01-21T21:17:31.170" Score="1" ViewCount="22" Body="&lt;p&gt;I'm working on a project to do sentiment analysis but my data is not long and properly formatted text. It's more likely to be very short sentences, e.g. tweets (in full tweet lingo), quick reviews of maybe 2-5 short sentences, etc.&lt;/p&gt;&#xA;&lt;p&gt;If my text is of that nature, what approach would you recommend? E.g. CNNs (spaCy has a ready-made text classifier), LSTM (e.g. something like Keras), etc.&lt;/p&gt;&#xA;&lt;p&gt;What are the pros/cons of your suggested approach (i.e. why is it better suited for classifying short paragraphs/sentences)?&lt;/p&gt;&#xA;&lt;p&gt;I'm starting out in the area so any links/papers/etc. will be most welcome!&lt;/p&gt;&#xA;" OwnerUserId="44015" LastEditorUserId="2444" LastEditDate="2021-01-22T12:10:49.650" LastActivityDate="2021-01-22T12:10:49.650" Title="What is the best approach for sentiment analysis when the text is very brief?" Tags="&lt;natural-language-processing&gt;&lt;reference-request&gt;&lt;sentiment-analysis&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25908" PostTypeId="1" CreationDate="2021-01-22T02:06:39.997" Score="0" ViewCount="15" Body="&lt;p&gt;I realize most NLP algorithms have multiple steps. (e.g. OCR/speech rec &amp;gt; syntax &amp;gt; semantics &amp;gt; response logic &amp;gt; semantic output &amp;gt; natural language output)&lt;/p&gt;&#xA;&lt;p&gt;Is there any NN model that can perform multiple steps in NLP at once? For example, a single network which accepts audio input and returns a semantic analysis of the given speech, or a single network which accepts text input and returns natural language output?&lt;/p&gt;&#xA;" OwnerUserId="44017" LastActivityDate="2021-01-22T02:06:39.997" Title="Is there any neural network model that can perform multiple NLP steps at once?" Tags="&lt;neural-networks&gt;&lt;natural-language-processing&gt;&lt;natural-language-understanding&gt;&lt;expert-system&gt;&lt;natural-language-generation&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25909" PostTypeId="1" CreationDate="2021-01-22T03:03:57.327" Score="0" ViewCount="23" Body="&lt;p&gt;Looking at &lt;a href=&quot;https://www.tensorflow.org/tutorials/text/transformer&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;matmul_qk = tf.matmul(q, k, transpose_b=True)  # (..., seq_len_q, seq_len_k)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;is the output of two tensors multiplied of shape:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;q: query shape == (Batch, heads, seq_len_q, depth)&#xA;k: key shape == (Batch, heads, seq_len_k, depth)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am failing to understand how the shapes result in this. The second tensor is transposed, so it should be &lt;code&gt;(depth, seq_len_k, heads, Batch)&lt;/code&gt;. I don't see how &lt;code&gt;(Batch, heads, seq_len_q, depth) * (depth, seq_len_k, heads, Batch) -&amp;gt; (Batch, heads, seq_len_q, seq_len_k)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="36486" LastEditorUserId="2444" LastEditDate="2021-01-22T20:21:05.663" LastActivityDate="2021-01-22T20:21:05.663" Title="Is the dimensionality of the query and key tensors in this implementation of the transformer's self-attention correct?" Tags="&lt;neural-networks&gt;&lt;tensorflow&gt;&lt;transformer&gt;&lt;implementation&gt;&lt;attention&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25910" PostTypeId="2" ParentId="25893" CreationDate="2021-01-22T05:11:25.753" Score="0" Body="&lt;p&gt;I found out a concept called 'Experience Replay', which trains a single step every time a new data sample is added instead of training to max epochs.&lt;/p&gt;&#xA;&lt;p&gt;That is, instead of this training loop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(max_paces):&#xA;    find action for max reward;&#xA;    add to trajectory to make inp;&#xA;    train(batch_size=len(inp), epochs=max_epochs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do training this way (for a single-episode ML problem, no incremental):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(max_epochs):&#xA;    reset environment;&#xA;&#xA;    for j in range(max_paces):&#xA;        find action for max reward;&#xA;        add to trajectory to make inp;&#xA;        train(batch_size=len(inp), epochs=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do training this way (for a multi-episode ML problem, incremental data):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(max_epochs):&#xA;    reset environment;&#xA;    inc_trajectory = get_random_past_data();&#xA;&#xA;    for j in range(max_paces):&#xA;        find action for max reward;&#xA;        add to inc_trajectory to make inp;&#xA;        train(batch_size=len(inp), epochs=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For multiple-episode problems especially those problems with unlimited episodes, the training loop needs to &lt;strong&gt;forget&lt;/strong&gt; (ie. exclude from training) old episodes which are very distant in the past, or &lt;strong&gt;select a random number&lt;/strong&gt; of old episodes (consider them &lt;strong&gt;a batch&lt;/strong&gt;, random batch) to be in every round of experience replay. Anyway, without eliminating some old data, the amount of training data are too much since it's unlimited number of episodes.&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2844" LastEditDate="2021-01-28T02:48:14.553" LastActivityDate="2021-01-28T02:48:14.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25911" PostTypeId="1" CreationDate="2021-01-22T08:36:57.070" Score="1" ViewCount="20" Body="&lt;p&gt;I implemented Dice loss class in &lt;code&gt;pytorch&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import torch&#xA;import torch.nn as nn&#xA;&#xA;&#xA;class DiceLoss(nn.Module):&#xA;    def __init__(self, weight=None, size_average=True):&#xA;        super(DiceLoss, self).__init__()&#xA;&#xA;    def forward(self, inputs, targets, smooth=1):&#xA;        smooth = 1.&#xA;&#xA;        input_flat = inputs.contiguous().view(-1)&#xA;        target_flat = targets.contiguous().view(-1)&#xA;&#xA;        intersection = (input_flat * target_flat).sum()&#xA;        A_sum = torch.sum(input_flat * input_flat)&#xA;        B_sum = torch.sum(target_flat * target_flat)&#xA;&#xA;        dsc = (2. * intersection + smooth) / (A_sum + B_sum + smooth)&#xA;        return 1 - dsc &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I tested it in 2 scenarios:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;where &lt;code&gt;inputs&lt;/code&gt; is the prediction from the network without applying activation (in my case sigmoid), only convolution with a kernel of size 1.&lt;/li&gt;&#xA;&lt;li&gt;where &lt;code&gt;inputs&lt;/code&gt; are the result of the network including activation of the sigmoid.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now I get comparable results between the 2 ways, but I was wondering what is the &amp;quot;Right&amp;quot; way out of the 2.&lt;/p&gt;&#xA;" OwnerUserId="41293" LastEditorUserId="2444" LastEditDate="2021-01-23T23:17:23.853" LastActivityDate="2021-01-23T23:17:23.853" Title="What does Dice Loss should receive in case of binary segmentation" Tags="&lt;deep-learning&gt;&lt;loss-functions&gt;&lt;activation-function&gt;&lt;image-segmentation&gt;&lt;dice-loss&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25912" PostTypeId="1" CreationDate="2021-01-22T09:21:52.427" Score="0" ViewCount="36" Body="&lt;p&gt;I've done implementing alpha-beta, and transpositional table on my search tree algorithm so I decided to implement move-ordering next. But once I implemented it, it's way more longer to respond than before?&lt;/p&gt;&#xA;&lt;p&gt;Here's my code so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;function sortMoves(chess)&#xA;{&#xA;    const listA = [], listB = chess.moves({ verbose: true });&#xA;    const scores = [];&#xA;&#xA;    // calc best moves&#xA;    const moves = [...listB];&#xA;    for (const move of moves)&#xA;    {&#xA;        const state = chess.move(move, {&#xA;            promotion: 'q'&#xA;        });&#xA;        scores.push(evaluate(chess.board()));&#xA;        chess.undo();&#xA;    }&#xA;    &#xA;    // sort move&#xA;    for (var i = 0; i &amp;lt; Math.min(5, moves.length); i++)&#xA;    {&#xA;        let maxEval = -Infinity;&#xA;        let maxIndex = 0;&#xA;        &#xA;        for (var j = 0; j &amp;lt; scores.length; j++)&#xA;        {&#xA;            if (scores[j] &amp;gt; maxEval)&#xA;            {&#xA;                maxEval = scores[j];&#xA;                maxIndex = j;&#xA;            }&#xA;        }&#xA;        &#xA;        scores[maxIndex] = -Infinity;&#xA;        listA.push(moves[maxIndex]);&#xA;        listB.splice(maxIndex, 1);&#xA;    }&#xA;    &#xA;    const newList = listA.concat(listB);&#xA;    return newList;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am expecting for this to respond quicker than before but it turns out it's not. So my question is am I actually sorting the moves correctly? or should I write some code for the alpha-beta pruning related to the sorted moves?&lt;/p&gt;&#xA;&lt;p&gt;Here's my &lt;code&gt;negamax&lt;/code&gt; function:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;function negamax(chess, depth, depthOrig, alpha, beta, color)&#xA;{&#xA;    // transposition table look up&#xA;    const alphaOrig = alpha;&#xA;    const hashKey = zobrist.hash(chess);&#xA;    const lookup = transposition.get(hashKey);&#xA;    if (lookup)&#xA;    {&#xA;        if (lookup.depth &amp;gt;= depth)&#xA;        {&#xA;            if (lookup.flag === EXACT)&#xA;                return lookup.score;&#xA;            else if (lookup.flag === LOWERBOUND)&#xA;                alpha = Math.max(alpha, lookup.score);&#xA;            else if (lookup.flag === UPPERBOUND)&#xA;                beta = Math.min(beta, lookup.score);&#xA;            &#xA;            if (alpha &amp;gt;= beta)&#xA;                return lookup.score;&#xA;        }&#xA;    }&#xA;&#xA;&#xA;    if (depth === 0 || chess.game_over())&#xA;    {&#xA;        // if current turn is checkmated,&#xA;        // remove the king on the board&#xA;        // so the AI knows if the move&#xA;        // will lead to checkmate or not, if&#xA;        // it's remove on the board, &#xA;        // the checkmated team will&#xA;        // reduce the king's value leading&#xA;        // the AI to move in checkmate&#xA;        const kingPos = getPiecePos(chess, 'k', chess.turn());&#xA;        if (chess.in_checkmate())&#xA;            chess.remove(kingPos);&#xA;        &#xA;        const evaluation = evaluate(chess.board());&#xA;        chess.put({ type: chess.KING, color: chess.turn() }, kingPos);&#xA;&#xA;        return color * evaluation;&#xA;    }&#xA;&#xA;    &#xA;    /* let moves = chess.moves();&#xA;    if (lookup)&#xA;    {&#xA;        console.log(moves, depth)&#xA;        const bestMove = lookup.move;&#xA;        const moveIndex = moves.indexOf(bestMove);&#xA;        const arr = moves.splice(moveIndex, 1);&#xA;        moves = arr.concat(moves);&#xA;        console.log(moves, depth)&#xA;    } */&#xA;    const moves = sortMoves(chess);&#xA;    /* const moves = chess.moves({ verbose: true }); */&#xA;&#xA;&#xA;    let count = 0;&#xA;    let score = -Infinity;&#xA;    let bestMove = null;&#xA;    if (lookup)&#xA;        bestMove = lookup.move;&#xA;&#xA;    &#xA;    for (const move of moves)&#xA;    {&#xA;        const state = chess.move(move, {&#xA;            promotion: 'q'&#xA;        });&#xA;        searchedMoves++;&#xA;        if (count === 0)&#xA;            score = -negamax(chess, depth-1, depthOrig, -beta, -alpha, -color);&#xA;        else&#xA;        {&#xA;            score = -negamax(chess, depth-1, depthOrig, -alpha-1, -alpha, -color);&#xA;            if (alpha &amp;lt; score &amp;lt; beta)&#xA;                score = -negamax(chess, depth-1, depthOrig, -beta, -score, -color);&#xA;        }&#xA;        chess.undo();&#xA;&#xA;        if (score &amp;gt; alpha)&#xA;        {&#xA;            alpha = score;&#xA;            bestMove = move;&#xA;        }&#xA;&#xA;        // do I add something on this part?&#xA;        count++;&#xA;        if (alpha &amp;gt;= beta)&#xA;            break;&#xA;    }&#xA;&#xA;    &#xA;    // transposition table store&#xA;    const key = zobrist.hash(chess);&#xA;    keyArr.push(key);&#xA;    const entry = new Transposition();&#xA;    entry.score = score;&#xA;    entry.depth = depth;&#xA;    entry.move = bestMove;&#xA;    if (score &amp;lt;= alphaOrig)&#xA;        entry.flag = UPPERBOUND;&#xA;    else if (score &amp;gt;= beta)&#xA;        entry.flag = LOWERBOUND;&#xA;    else&#xA;        entry.flag = EXACT;&#xA;    transposition.set(key, entry);&#xA;&#xA;&#xA;    return alpha;&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="44021" LastActivityDate="2021-01-22T09:21:52.427" Title="How to implement very simple move-ordering for alpha-beta pruning" Tags="&lt;chess&gt;&lt;alpha-beta-pruning&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25913" PostTypeId="1" AcceptedAnswerId="25916" CreationDate="2021-01-22T09:41:41.827" Score="9" ViewCount="1229" Body="&lt;p&gt;&lt;strong&gt;Q-learning&lt;/strong&gt; uses a table to store all state-action pairs. Q-learning is a model-free RL algorithm, so how could there be the one called &lt;strong&gt;Deep Q-learning&lt;/strong&gt;, as &lt;em&gt;deep&lt;/em&gt; means using DNN; or maybe the state-action table (Q-table) is still there but the DNN is only for input reception (e.g. turning images into vectors)?&lt;/p&gt;&#xA;&lt;p&gt;Deep Q-network seems to be only the DNN part of the Deep Q-learning program, and Q-network seems the short for Deep Q-network.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q-learning&lt;/strong&gt;, &lt;strong&gt;Deep Q-learning&lt;/strong&gt;, and &lt;strong&gt;Deep Q-network&lt;/strong&gt;, what are the differences? May be there a comparison table between these 3 terms?&lt;/p&gt;&#xA;" OwnerUserId="2844" LastEditorUserId="2444" LastEditDate="2021-01-22T21:31:51.797" LastActivityDate="2021-01-22T21:50:28.390" Title="What is the difference between Q-learning, Deep Q-learning and Deep Q-network?" Tags="&lt;reinforcement-learning&gt;&lt;comparison&gt;&lt;q-learning&gt;&lt;dqn&gt;&lt;deep-rl&gt;" AnswerCount="2" CommentCount="0" FavoriteCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25914" PostTypeId="1" CreationDate="2021-01-22T10:04:09.467" Score="3" ViewCount="38" Body="&lt;p&gt;I am taking a course about using matrix factorization for machine learning.&lt;/p&gt;&#xA;&lt;p&gt;The first thing that came into my mind is by using the matrix factorization we are always limited to linear relationships between the data, which is very limiting to predict complex patterns.&lt;/p&gt;&#xA;&lt;p&gt;In comparison with neural networks, where we can use a non-linear activation function. It seems to me that all the tasks that matrix factorization can achieve will score better using a simple multilayer neural network.&lt;/p&gt;&#xA;&lt;p&gt;So, can I conclude that &lt;a href=&quot;https://en.wikipedia.org/wiki/Non-negative_matrix_factorization&quot; rel=&quot;nofollow noreferrer&quot;&gt;NMF&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;matrix factorization&lt;/a&gt; for machine learning, in general, are not that practical, or there are cases where it's better to use NMF?&lt;/p&gt;&#xA;" OwnerUserId="44022" LastEditorUserId="2444" LastEditDate="2021-01-22T21:07:32.627" LastActivityDate="2021-01-22T21:07:32.627" Title="Is non-negative matrix factorization for machine learning obsolete?" Tags="&lt;neural-networks&gt;&lt;machine-learning&gt;&lt;reference-request&gt;&lt;applications&gt;" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="25916" PostTypeId="2" ParentId="25913" CreationDate="2021-01-22T10:25:58.830" Score="5" Body="&lt;p&gt;In Q-learning (and in general value based reinforcement learning) we are typically interested in learning a Q-function, &lt;span class=&quot;math-container&quot;&gt;$Q(s, a)$&lt;/span&gt;. This is defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q(s, a) = \mathbb{E}_\pi\left[ G_t | S_t = s, A_t = a \right]\;.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For tabular Q-learning, where you have a finite state and action space - note that in practice even the spaces being finite might not be enough to &lt;em&gt;not&lt;/em&gt; use DQN, if e.g. your state space contains a large number, say &lt;span class=&quot;math-container&quot;&gt;$10^{10000}$&lt;/span&gt;, of states, then it might not be manageable to maintain a separate Q-function for each state-action pair - you can maintain a table lookup that maintains your current estimate of the Q-value.&lt;/p&gt;&#xA;&lt;p&gt;When you have an infinite state space (and/or action space) then it becomes impossible to use a table, and so you need to use function approximation to generalise across states. This is typically done using a deep neural network due to their expressive power. As a technical aside, the Q-networks don't usually take state &lt;em&gt;and&lt;/em&gt; action as input, but take in a representation of the state (e.g. a &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;-dimensional vector, or an image) and output a real valued vector of size &lt;span class=&quot;math-container&quot;&gt;$|\mathcal{A}|$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{A}$&lt;/span&gt; is the action space.&lt;/p&gt;&#xA;&lt;p&gt;Now, it seems in your question that you're confused as to why you use a model (the neural network) when Q-learning is, as you rightly say, model-free. The answer here is that when we talk about Reinforcement Learnings being model-free we are not talking about how their value-functions or policy are parameterised, we are actually talking about whether the algorithms use a model of the transition dynamics to help with their learning. That is, a model free algorithm doesn't use any knowledge about &lt;span class=&quot;math-container&quot;&gt;$p(s' | s, a)$&lt;/span&gt; whereas model-based methods look to use this transition function - either because it is known exactly such as in Atari environments, or it must need to be approximated - to perform planning with the dynamics.&lt;/p&gt;&#xA;" OwnerUserId="36821" LastEditorUserId="36821" LastEditDate="2021-01-22T12:50:09.673" LastActivityDate="2021-01-22T12:50:09.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25917" PostTypeId="2" ParentId="13317" CreationDate="2021-01-22T12:25:26.100" Score="4" Body="&lt;h2&gt;&amp;quot;Modern&amp;quot; Guarantees for &lt;a href=&quot;https://en.wikipedia.org/wiki/Feedforward_neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Feed-Forward Neural Networks&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;My answer will complement nbro's above, which gave a very nice overview of universal approximation theorems for different types of commonly used architectures, by focusing on recent developments specifically for feed-forward networks.  I'll try an emphasis &lt;a href=&quot;https://ai.stackexchange.com/questions/1742/what-is-the-difference-between-machine-learning-and-deep-learning&quot;&gt;depth&lt;/a&gt; over breadth (sometimes called width) as much as possible.  &lt;em&gt;Enjoy!&lt;/em&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Part 1: Universal Approximation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Here I've listed a few recent universal approximation results that come to mind.&lt;/em&gt;  Remember, &lt;em&gt;universal approximation&lt;/em&gt; asks if feed-forward networks (or some other architecture type) can &lt;em&gt;approximate&lt;/em&gt; any (in this case continuous) function to arbitrary accuracy (I'll focus on the : &lt;em&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/560978/what-is-topology-of-compact-convergence&quot;&gt;uniformly on compacts&lt;/a&gt;&lt;/em&gt; sense).&lt;/p&gt;&#xA;&lt;p&gt;Let me mention, that there are two types of guarantees: &lt;em&gt;quantitative ones&lt;/em&gt; and &lt;em&gt;qualitative ones&lt;/em&gt;.  The latter are akin to &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0893608089900208&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hornik's results (Neural Networks - 1989)&lt;/a&gt; which simply state that some neural networks can approximate a given (continuous) function to arbitrary precision.  The former of these types of guarantees quantifies the number of parameters required for a neural network to actually perform the approximation and are akin to &lt;a href=&quot;https://ieeexplore.ieee.org/document/256500&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barron's (now) classical paper (IEEE - 1993)&lt;/a&gt;'s breakthrough results.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Shallow Case:&lt;/strong&gt;  If you want quantitative results only for &lt;em&gt;shallow&lt;/em&gt; networks:&#xA;Then &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0893608020301891#b9&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. Siegel and J. Xu (Neural Networks - 2020)&lt;/a&gt; will do the trick but (note: The authors deal with the Sobolev case but you get the continuous case immediately via the &lt;a href=&quot;https://mathoverflow.net/questions/127259/sharpness-of-the-sobolev-embedding-theorem&quot;&gt;Soblev-Morrey embedding theorem&lt;/a&gt;.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Deep (not narrow) &lt;a href=&quot;https://ai.stackexchange.com/questions/6468/why-do-we-prefer-relu-over-linear-activation-functions&quot;&gt;ReLU&lt;/a&gt; Case:&lt;/strong&gt;  If you want a quantitative proof for deep networks (but not too narrow) with ReLU activation function then &lt;a href=&quot;http://proceedings.mlr.press/v75/yarotsky18a/yarotsky18a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dimity Yarotsky's result (COLT - 2018)&lt;/a&gt; will do the trick!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Deep and Narrow:&lt;/strong&gt; To the best of my knowledge, the first quantitative proof for deep and narrow neural networks with general input and output spaces has recently appeared here:&#xA;&lt;a href=&quot;https://arxiv.org/abs/2101.05390&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/2101.05390 (preprint - 2021)&lt;/a&gt;.&lt;br /&gt;&#xA;The article is a constructive version of &lt;a href=&quot;http://proceedings.mlr.press/v125/kidger20a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;P. Kidger and T. Lyon's recent deep and narrow universal approximation theorem (COLT - 2020)&lt;/a&gt; (qualitative) for functions from &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^p$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^m$&lt;/span&gt; and &lt;a href=&quot;https://proceedings.neurips.cc/paper/2020/file/786ab8c4d7ee758f80d57e65582e609d-Paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Kratsios and E. Bilokpytov's recent Non-Euclidean Universal Approximation Theorem (NeurIPS - 2020).&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Part 2: Memory Capacity&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;A related concept is that of&lt;/em&gt; &lt;strong&gt;&amp;quot;memory capacity of a deep neural network&amp;quot;&lt;/strong&gt;.&lt;br /&gt;&#xA;These results seek to quantify the number of parameters needed for a deep network to learn (exactly) the assignment of some input data &lt;span class=&quot;math-container&quot;&gt;$\{x_n\}_{n=1}^N$&lt;/span&gt; to some output data &lt;span class=&quot;math-container&quot;&gt;$\{y_n\}_{n=1}^N$&lt;/span&gt;.  For example; you may want to take a look here:&lt;/p&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Memory Capacity of Deep ReLU networks:&lt;/strong&gt; R. Vershynin's very recent publication &lt;a href=&quot;https://epubs.siam.org/doi/pdf/10.1137/20M1314884&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations - (SIAM's SIMODS 2020)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="31649" LastEditorUserId="31649" LastEditDate="2021-01-22T15:17:02.490" LastActivityDate="2021-01-22T15:17:02.490" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25918" PostTypeId="2" ParentId="25913" CreationDate="2021-01-22T12:45:53.817" Score="9" Body="&lt;p&gt;Here is a table that attempts to &lt;em&gt;systematically&lt;/em&gt; show the differences between tabular Q-learning (TQL), deep Q-learning (DQL), and deep Q-network (DQN).&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;&lt;/th&gt;&#xA;&lt;th&gt;Tabular Q-learning (TQL)&lt;/th&gt;&#xA;&lt;th&gt;Deep Q-learning (DQL)&lt;/th&gt;&#xA;&lt;th&gt;Deep Q-network (DQN)&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Is it an RL algorithm?&lt;/td&gt;&#xA;&lt;td&gt;Yes&lt;/td&gt;&#xA;&lt;td&gt;Yes&lt;/td&gt;&#xA;&lt;td&gt;No (unless you use DQN to refer to DQL, which is done often!)&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Does it use neural networks?&lt;/td&gt;&#xA;&lt;td&gt;No. It uses a table.&lt;/td&gt;&#xA;&lt;td&gt;Yes&lt;/td&gt;&#xA;&lt;td&gt;No. DQN is the neural network.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Is it a model?&lt;/td&gt;&#xA;&lt;td&gt;No&lt;/td&gt;&#xA;&lt;td&gt;No&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/25460/2444&quot;&gt;Yes&lt;/a&gt; (&lt;a href=&quot;https://ai.stackexchange.com/q/4456/2444&quot;&gt;but usually not in the RL sense&lt;/a&gt;)&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Can it deal with continuous state spaces?&lt;/td&gt;&#xA;&lt;td&gt;No (unless you discretize them)&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/12257/2444&quot;&gt;Yes&lt;/a&gt;&lt;/td&gt;&#xA;&lt;td&gt;Yes (in the sense that it can get real-valued inputs for the states)&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Can it deal with continuous action spaces?&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/12257/2444&quot;&gt;Yes&lt;/a&gt; (but maybe not a good idea)&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/a/12257/2444&quot;&gt;Yes&lt;/a&gt; (but maybe not a good idea)&lt;/td&gt;&#xA;&lt;td&gt;Yes (but only the sense that it can produce real-valued outputs for actions).&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Does it converge?&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/11679/2444&quot;&gt;Yes&lt;/a&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/11679/2444&quot;&gt;Not necessarily&lt;/a&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/11679/2444&quot;&gt;Not necessarily&lt;/a&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Is it an online learning algorithm?&lt;/td&gt;&#xA;&lt;td&gt;Yes&lt;/td&gt;&#xA;&lt;td&gt;No, if you use &lt;a href=&quot;https://ai.stackexchange.com/q/6579/2444&quot;&gt;experience replay&lt;/a&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://ai.stackexchange.com/q/10839/2444&quot;&gt;No, but it can be used in an online learning setting&lt;/a&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-22T21:50:28.390" LastActivityDate="2021-01-22T21:50:28.390" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25919" PostTypeId="1" CreationDate="2021-01-22T12:58:47.717" Score="1" ViewCount="18" Body="&lt;p&gt;I am trying to design a good heuristic to solve a constraint satisfaction problem (CSP). I think that a possible heuristic to use is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$h_1(\text{state}) = \text{number of conflicts in state}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, of the possible solutions to the CSP, some have a lower cost (are better). So I can't just use &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; as my heuristic.&lt;/p&gt;&#xA;&lt;p&gt;Since the state space is pretty huge, I want to use the local search with a heuristic &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, that guides my variable assignments towards a low-cost solution while reducing the conflicts. The way I am thinking about going about this is: of the variable assignments which do not cause conflicts (are valid), apply &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; to them, pick the variable assignment which has the lowest/best value &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; value. So &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; would not handle conflicts, I would make sure any assignments considered by &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; are guaranteed to be valid.&lt;/p&gt;&#xA;&lt;p&gt;Ideally, though, I want &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; to both drive down the conflicts to 0 and simultaneously guide the assignments to the lowest cost solution. Is this generally possible?&lt;/p&gt;&#xA;" OwnerUserId="44028" LastEditorUserId="2444" LastEditDate="2021-01-22T16:37:38.433" LastActivityDate="2021-01-22T16:37:38.433" Title="CSP heuristic to simultaneously reduce conflicts and find near optimal assignment" Tags="&lt;heuristics&gt;&lt;constraint-satisfaction-problems&gt;&lt;local-search&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25922" PostTypeId="1" CreationDate="2021-01-22T14:38:39.147" Score="2" ViewCount="23" Body="&lt;p&gt;I'm reading the following &lt;a href=&quot;https://arxiv.org/abs/1702.00071&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; in which the author seems to do 2 things interesting:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The hidden-to-hidden weight matrix of the RNN is SVD decomposed and train separately.&lt;/li&gt;&#xA;&lt;li&gt;Each orthogonal part of the decomposition is optimized multiplicatively according to Cayley Transformation to maintain its orthogonal properties.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now, I'm not so strong with the math behind the technique, but I could be hand-waving and say that albeit being multiplicative, it is just another method of gradient descent, and each orthogonal part is still minimizing the Loss function. So far so good.&lt;/p&gt;&#xA;&lt;p&gt;But what they are doing is actually split the original optimization problem into multiple sub-optimization (2 for orthogonal matrices and n for the number of singular values), and then multiplied the result together. How can we be sure about the convergence and the optimality of such method? Or is this the case where we can say nothing and let the experiment speak for themselves?&lt;/p&gt;&#xA;" OwnerUserId="41781" LastActivityDate="2021-01-22T14:38:39.147" Title="Is it possible to ensure the convergence when training a RNN weight on its SVD decomposition?" Tags="&lt;neural-networks&gt;&lt;recurrent-neural-networks&gt;&lt;papers&gt;&lt;gradient-descent&gt;&lt;optimization&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25923" PostTypeId="1" CreationDate="2021-01-22T15:17:10.343" Score="2" ViewCount="20" Body="&lt;p&gt;I've come across a few binary classification problems lately where the labelling was challenging even for an expert. I'm wondering what I should do with this. Here are some of my suggestions to get the ball rolling:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make a third category called &amp;quot;unsure&amp;quot; then make it a three-class classification problem instead.&lt;/li&gt;&#xA;&lt;li&gt;Make a third category called &amp;quot;unsure&amp;quot; and just remove these from your training set.&lt;/li&gt;&#xA;&lt;li&gt;Make a third category called &amp;quot;unsure&amp;quot; and during training model this as a 0.5 such that the binary cross entropy loss looks like &lt;span class=&quot;math-container&quot;&gt;$-0.5\log(\hat{y})-0.5\log(1-\hat{y})$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Allow the labeller to pick a percentage on a sliding scale (or maybe multiple choice: (0%, 25%, 50%, 75%, 100%), and take that into account when calculating cross entropy (as in my point above).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I recently saw a &lt;a href=&quot;https://arxiv.org/abs/1804.02555&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; which goes for option 2, although that's not enough to convince me. Here's the relevant quote:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In case of a high-level risk, collision is imminent and&#xA;the driver must react in less than 0.5 s (TTC &amp;lt; 0.5s). For&#xA;low-level risk, the TTC is more than 2.0 s (TTC &amp;gt; 2.0s).&#xA;Videos that show intermediate-level risk (0.5 s ≤ TTC&#xA;≤ 2.0 s), which is a mixture of high- and low-level risks,&#xA;were not included in the NIDB because when training&#xA;a convnet, it must be possible to make a clear visual&#xA;distinction of risk.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16871" LastActivityDate="2021-01-22T15:17:10.343" Title="For binary classification learning problems, how should I label instances where I'm only 60% sure?" Tags="&lt;neural-networks&gt;&lt;classification&gt;&lt;data-labelling&gt;" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25924" PostTypeId="5" CreationDate="2021-01-22T15:52:41.797" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-22T15:52:41.797" LastActivityDate="2021-01-22T15:52:41.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25925" PostTypeId="4" CreationDate="2021-01-22T15:52:41.797" Score="0" Body="For questions related to the often not well-defined or vaguely defined notion of capacity of a neural network (or any other machine learning model). The capacity of a neural network can be precisely defined as its VC dimension." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-22T15:52:41.797" LastActivityDate="2021-01-22T15:52:41.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25926" PostTypeId="1" AcceptedAnswerId="25927" CreationDate="2021-01-22T17:24:44.783" Score="3" ViewCount="54" Body="&lt;p&gt;We all know that Genetic Algorithms can give an optimal or near-optimal solution. So, in some problems like NP-hard ones, with a trade-off between time and optimal solution the near-optimal solution is good enough.&lt;/p&gt;&#xA;&lt;p&gt;Since there is no guarantee to find the optimal solution, is GA considered to be a good choice for solving the Knuth problem?&lt;/p&gt;&#xA;&lt;p&gt;According to Artificial intelligence: A modern approach (third edition), section 3.2 (p. 73):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Knuth conjectured that, starting with the number 4, a sequence of&#xA;factorial, square root, and floor operations will reach any desired&#xA;positive integer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For example, 5 can be reached from 4:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;floor(sqrt(sqrt(sqrt(sqrt(sqrt((4!)!))))))&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, if we have a number (5) and we want to know the sequence of the operations of the 3 mentioned ones to reach the given number, each gene of the chromosome will be a number that represents a certain operation with an additional number for (no operation) and the fitness function will be the absolute difference between the given number and the number we get from applying the operations in a certain order for each the chromosome (to min). Let's consider that the number of the iterations (generations) is done with no optimal solution and the nearest number we have is 4 ( with fitness 1), the problem is that we can get 4 from applying no operation on 4 while for 5 we need many operations, so the near-optimal solution is not even near to the solution.&lt;/p&gt;&#xA;&lt;p&gt;So, is GA is not suitable for this kind of problems? Or the suggested chromosome representation and fitness function are not good enough?&lt;/p&gt;&#xA;" OwnerUserId="42300" LastEditorUserId="2444" LastEditDate="2021-01-23T00:35:14.323" LastActivityDate="2021-01-23T04:53:36.103" Title="Are Genetic Algorithms suitable for problems like the Knuth problem?" Tags="&lt;genetic-algorithms&gt;&lt;evolutionary-algorithms&gt;&lt;genetic-programming&gt;&lt;norvig-russell&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25927" PostTypeId="2" ParentId="25926" CreationDate="2021-01-22T17:48:40.627" Score="1" Body="&lt;p&gt;Before trying to answer your question more directly, let me clarify something.&lt;/p&gt;&#xA;&lt;p&gt;People often use the term &lt;em&gt;genetic algorithms&lt;/em&gt; (GAs), but, in many cases, what they really mean is &lt;em&gt;evolutionary algorithms&lt;/em&gt; (EAs), which is a collection of population-based (i.e. multiple solutions are maintained at the same time) optimization algorithms and approaches that are inspired by &lt;a href=&quot;https://en.wikipedia.org/wiki/Darwinism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Darwinism&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Survival_of_the_fittest&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;survival of the fittest&lt;/em&gt;&lt;/a&gt;. GAs is one of these approaches, where the chromosomes are binary and you have both the mutation and cross-over operation. There are other approaches, such as &lt;em&gt;evolution strategies&lt;/em&gt; or &lt;em&gt;genetic programming&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As you also noticed, EAs are meta-heuristics, and, although there is some research on their convergence properties [&lt;a href=&quot;https://www.researchgate.net/publication/3642792_Convergence_of_evolutionary_algorithms_in_general_search_spaces&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;], in practice, they may not converge. However, when any other potential approach has failed, EAs can be definitely useful.&lt;/p&gt;&#xA;&lt;p&gt;In your case, the problem is really to find a &lt;strong&gt;closed-form&lt;/strong&gt; (or &lt;strong&gt;analytical&lt;/strong&gt;) &lt;strong&gt;expression&lt;/strong&gt; of a function, which is composed of other smaller functions. This really is what &lt;a href=&quot;http://www0.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/poli08_fieldguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;genetic programming&lt;/strong&gt;&lt;/a&gt; (in particular, tree-based GP) was created for. In fact, the Knuth problem is a particular instance of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Symbolic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;symbolic regression problem&lt;/strong&gt;&lt;/a&gt;, which is a typical problem that GP is applied to. So, GP is probably the first approach you should try.&lt;/p&gt;&#xA;&lt;p&gt;Meanwhile, I have implemented a simple program in DEAP that tries to solve the Knuth problem. Check it &lt;a href=&quot;https://gist.github.com/nbro/dfdf39b6fe98e96c61d164dd3f6bd177&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The fitness of the best solution that it has found so far (with some seed) is 4 and the solution is &lt;code&gt;floor(sqrt(float(sqrt(4))))&lt;/code&gt; (here &lt;code&gt;float&lt;/code&gt; just converts the input to a floating-point number, to ensure type safety).  I used the &lt;em&gt;difference&lt;/em&gt; as the fitness function and ran the GP algorithm for 100 generations with 100 individuals for each generation (which is &lt;em&gt;not&lt;/em&gt; a lot!). I didn't tweak much the hyper-parameters, so, maybe, with the right seed and hyper-parameters, you can find the right solution.&lt;/p&gt;&#xA;&lt;p&gt;To address your concerns, in principle, you could use that encoding, but, as you note, the GA could indeed return &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; as the best solution (which isn't actually that far away from &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;), which you could avoid my killing, at every generation, any individuals that have just that value.&lt;/p&gt;&#xA;&lt;p&gt;I didn't spend too much time on &lt;a href=&quot;https://gist.github.com/nbro/dfdf39b6fe98e96c61d164dd3f6bd177&quot; rel=&quot;nofollow noreferrer&quot;&gt;my implementation&lt;/a&gt; and thinking about this problem, but, as I said above, even with genetic programming and using only Knuth's operations, it could get stuck in local optima. You could try to augment my (or your) implementation with other operations, such as the multiplication and addition, and see if something improves.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-23T04:53:36.103" LastActivityDate="2021-01-23T04:53:36.103" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="25930" PostTypeId="5" CreationDate="2021-01-23T00:57:11.833" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-23T00:57:11.833" LastActivityDate="2021-01-23T00:57:11.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25931" PostTypeId="4" CreationDate="2021-01-23T00:57:11.833" Score="0" Body="For questions related to the shortest path problem in the context of artificial intelligence." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-23T00:57:11.833" LastActivityDate="2021-01-23T00:57:11.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25932" PostTypeId="5" CreationDate="2021-01-23T01:57:14.087" Score="0" Body="" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-23T01:57:14.087" LastActivityDate="2021-01-23T01:57:14.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25933" PostTypeId="4" CreationDate="2021-01-23T01:57:14.087" Score="0" Body="For questions related to the general problem solver, a computer program created in 1959 by Herbert A. Simon, J. C. Shaw, and Allen Newell." OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-23T01:57:14.087" LastActivityDate="2021-01-23T01:57:14.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25934" PostTypeId="1" CreationDate="2021-01-23T08:45:22.500" Score="1" ViewCount="33" Body="&lt;p&gt;Most of the tutorials only teach us to split the whole dataset into three parts: training set, develop set, and test set. But in the industry, we are kind of doing test-driven development, and what comes most important is the building of our test set. We are not given a large corpus first, but we are discussing how to build the test set first.&lt;/p&gt;&#xA;&lt;p&gt;The most resource-saving method is just sampling(simple random sampling) cases from the log and then having them labeled, which represents the population. Perhaps we are concerning that some groups should be more important than others, then we do stratified sampling.&lt;/p&gt;&#xA;&lt;p&gt;Are there any better sampling methods?&lt;/p&gt;&#xA;&lt;p&gt;What to do when we are releasing a new feature and we cannot find any cases of that feature from the user log?&lt;/p&gt;&#xA;" OwnerUserId="5351" LastActivityDate="2021-01-23T22:18:43.447" Title="How to build a test set for a model in industry?" Tags="&lt;testing&gt;&lt;test-datasets&gt;" AnswerCount="1" CommentCount="0" FavoriteCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25935" PostTypeId="1" CreationDate="2021-01-23T09:55:49.637" Score="0" ViewCount="12" Body="&lt;p&gt;Similar to This person does not exist or This artwork does not exist, how might I go about creating a This street map does not exist, including choosing an appropriate AI model and scoping features?&lt;/p&gt;&#xA;&lt;p&gt;Challenges might include:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether it would be better to work from GIS data or from rendered map tiles. To work from tiles at the zoom level I would prefer, specifically street maps, I would need to set up my own local tile server per &lt;a href=&quot;https://switch2osm.org/serving-tiles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Open Street Map's website page on serving tiles&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Most of the world is sparsely populated but maps of imaginary cities would be more visually interesting.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;a) This could be mitigated by only choosing cities and towns as training data. This might be harder to manage if I am training from GIS data rather than easy-to-curate map tiles. Or perhaps I would need to create a pre-AI project to have it identify cities generated from my tile server and only pass along city tiles, discarding tiles that don't have cities.&lt;/p&gt;&#xA;&lt;p&gt;b) I can choose city segments and download data from &lt;a href=&quot;https://protomaps.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protomaps&lt;/a&gt; but it takes a few minutes to navigate to a city, choose a selection, and download the data. If I need to have 10 samples as training data, this is reasonable, but if I need 100 or 1000 then it would be too tedious. How would I determine how big a sample size would be needed?&lt;/p&gt;&#xA;&lt;p&gt;At this point I'm not sure I'm asking the right questions. New to AI here.&lt;/p&gt;&#xA;" OwnerUserId="44020" LastActivityDate="2021-01-23T09:55:49.637" Title="AI generator for imaginary street maps?" Tags="&lt;machine-learning&gt;&lt;comparison&gt;&lt;image-processing&gt;" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25936" PostTypeId="2" ParentId="1285" CreationDate="2021-01-23T10:34:26.957" Score="0" Body="&lt;p&gt;Have a look at this paper where the blockchain is used as a &amp;quot;federation&amp;quot; server for improving the parameters of local neural networks:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-03080421/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.archives-ouvertes.fr/hal-03080421/document&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="43582" LastActivityDate="2021-01-23T10:34:26.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25937" PostTypeId="1" AcceptedAnswerId="25938" CreationDate="2021-01-23T11:24:02.603" Score="2" ViewCount="66" Body="&lt;p&gt;I understand what cross-correlation does given a kernel and an input image, but the formula confuses me a little. Given here in &lt;a href=&quot;https://www.deeplearningbook.org/contents/convnets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goodfellow's Deep Learning&lt;/a&gt; (page 329), I can't quite understand what &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; are. Are they the dimensions of the kernel along the height and width dimensions?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S(i,j) =(K*I)(i,j) = \sum_m \sum_n I(i+m, j+n)K(m,n)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, for the input image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and kernel &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;, we take the sum product of &lt;span class=&quot;math-container&quot;&gt;$I*K$&lt;/span&gt;, but what do the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; represent? How is the input image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; indexed?&lt;/p&gt;&#xA;" OwnerUserId="42966" LastEditorUserId="2444" LastEditDate="2021-01-23T13:47:22.003" LastActivityDate="2021-01-23T13:47:22.003" Title="What do the variables in the cross-correlation formula mean?" Tags="&lt;deep-learning&gt;&lt;convolutional-neural-networks&gt;&lt;convolution&gt;&lt;notation&gt;&lt;cross-correlation&gt;" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25938" PostTypeId="2" ParentId="25937" CreationDate="2021-01-23T13:22:28.920" Score="2" Body="&lt;p&gt;It takes a little bit of time to fully understand the 2D &lt;a href=&quot;https://ai.stackexchange.com/q/21999/2444&quot;&gt;convolution/cross-correlation&lt;/a&gt; and to relate it to the usual diagrams of the convolution operation, so, before addressing your questions, let me first try to break the definition of the 2D cross-correlation down, from the left to right.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S(i,j) =(K*I)(i,j) = \sum_m \sum_n I(i+m, j+n)K(m,n) \label{1}\tag{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the function that is the cross-correlation of the functions &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$S(i, j)$&lt;/span&gt; is the cross-correlation of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; at the pixels &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The symbol &lt;span class=&quot;math-container&quot;&gt;$*$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$K*I$&lt;/span&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_mathematical_symbols_by_subject&quot; rel=&quot;nofollow noreferrer&quot;&gt;cross-correlation/convolution symbol&lt;/a&gt;, but sometimes the cross-correlation/convolution is also denoted as &lt;span class=&quot;math-container&quot;&gt;$\circledast$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$K*I$&lt;/span&gt; is the function that results from the cross-correlation of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$(K*I)(i,j)$&lt;/span&gt; is the value of the function &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (the cross-correlation of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;) at the inputs (or pixels) &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. In other words, &lt;span class=&quot;math-container&quot;&gt;$(K*I)(i,j)$&lt;/span&gt; is just another way of writing &lt;span class=&quot;math-container&quot;&gt;$S(i,j)$&lt;/span&gt; that emphasizes that we took the cross-correlation of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;, but they are exactly the same thing.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The double summation &lt;span class=&quot;math-container&quot;&gt;$\sum_m \sum_n$&lt;/span&gt; is because we are computing the &lt;strong&gt;2D&lt;/strong&gt; cross-correlation, i.e. over the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; dimensions of the image and kernel. This is just the definition of the 2D cross-correlation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; are the indices of the summations, one across the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis and the other across the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-axis. Let &lt;span class=&quot;math-container&quot;&gt;$m = x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n = y$&lt;/span&gt;, so we can rewrite equation \ref{1} as follows.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$S(i, j) =(K*I)(i, j) = \sum_x \sum_y I(i + x, j + y)K(x, y) \label{2}\tag{2}$$&lt;/span&gt;&#xA;Now, it should be clearer that we are indexing across the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; dimensions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, let me further restrict the range of the summations. Let's say from &lt;span class=&quot;math-container&quot;&gt;$x=-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt; and from &lt;span class=&quot;math-container&quot;&gt;$y=-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$y=1$&lt;/span&gt;, then we can rewrite equation \ref{2} as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$S(i, j) =(K*I)(i, j) = \sum_{x=-1}^1 \sum_{y=-1}^1 I(i + x, j + y)K(x, y) \label{3}\tag{3}$$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why do I want to do this? Let me explain why. Consider now the following kernel &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; (which happens to be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_(image_processing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian kernel&lt;/a&gt;)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;K = \begin{bmatrix}\ \ \color{blue}{\frac {1}{16}} &amp;amp;\ \ \frac {1}{8} &amp;amp;\ \ \frac {1}{16} \\\ \ \frac {1}{8} &amp;amp;\ \ \frac {1}{4} &amp;amp;\ \ \color{red}{ \frac {1}{8}} \\\ \ \frac {1}{16} &amp;amp;\ \ \frac {1}{8} &amp;amp;\ \ \frac {1}{16}\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;Note that this is the output of the function &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; or, more precisely, its &lt;a href=&quot;https://encyclopediaofmath.org/wiki/Support_of_a_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;support&lt;/a&gt;. Let's assume that &lt;span class=&quot;math-container&quot;&gt;$\frac {1}{4}$&lt;/span&gt; is at the index/pixel &lt;span class=&quot;math-container&quot;&gt;$(0, 0)$&lt;/span&gt;. Then, for example, the top-left &lt;span class=&quot;math-container&quot;&gt;$ \color{blue}{\frac {1}{16}} $&lt;/span&gt; is at index/pixel &lt;span class=&quot;math-container&quot;&gt;$(-1, -1)$&lt;/span&gt; and the middle-right &lt;span class=&quot;math-container&quot;&gt;$\color{red}{ \frac {1}{8}}$&lt;/span&gt; at pixel &lt;span class=&quot;math-container&quot;&gt;$(1, 0)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, consider any image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; represented as a 2D matrix (i.e. its support), with, for example, dimensions &lt;span class=&quot;math-container&quot;&gt;$U \times V$&lt;/span&gt;. For concreteness, let &lt;span class=&quot;math-container&quot;&gt;$U = 5$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V=5$&lt;/span&gt;. Moreover, the middle pixel of the image is at index &lt;span class=&quot;math-container&quot;&gt;$(0, 0)$&lt;/span&gt;, as for the kernel. Let's say the image is the following&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;I = &#xA;\begin{bmatrix}&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 \\&#xA;0 &amp;amp; 0 &amp;amp; \color{green}{0} &amp;amp; 1 &amp;amp; 1 \\&#xA;1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 \\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;So, &lt;span class=&quot;math-container&quot;&gt;$\color{green}{0}$&lt;/span&gt; is at index/pixel &lt;span class=&quot;math-container&quot;&gt;$(0,0)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, let's say that &lt;span class=&quot;math-container&quot;&gt;$i = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j=0$&lt;/span&gt; in equation \ref{3}. This means that we compute the cross-correlation between &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; at the index &lt;span class=&quot;math-container&quot;&gt;$(0, 0)$&lt;/span&gt;, where, in the case of the image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;, the value is &lt;span class=&quot;math-container&quot;&gt;$\color{green}{0}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With these settings, it should be clearer now that equation \ref{3} means that the cross-correlation between &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; at index/pixel &lt;span class=&quot;math-container&quot;&gt;$i=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j=0$&lt;/span&gt; is a 2D dot (or scalar) product. If it is not clear, then let be take the &lt;span class=&quot;math-container&quot;&gt;$3\times 3$&lt;/span&gt; submatrix of &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; centered at &lt;span class=&quot;math-container&quot;&gt;$(i, j) = (0, 0)$&lt;/span&gt; (below, &lt;span class=&quot;math-container&quot;&gt;$(0, 0)^{3 \times 3}$&lt;/span&gt; is just the notation that I came up with to indicate that).&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;I_{(0, 0)^{3 \times 3}} = &#xA;\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 1 \\&#xA;0 &amp;amp; \color{green}{0} &amp;amp; 1  \\&#xA;1 &amp;amp; 0 &amp;amp; 0 &#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;Then the cross-correlation in equation \ref{3} is just&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;S(i, j) = S(0, 0) = (K * I)(0, 0) = &#xA;\sum&#xA;\begin{bmatrix}&#xA;1 \frac {1}{16} &amp;amp; 0 \frac {1}{8} &amp;amp; 1 \frac {1}{16} \\&#xA;0 \frac {1}{8} &amp;amp; \color{green}{0} \frac {1}{4}  &amp;amp; 1 \frac {1}{8}  \\&#xA;1 \frac {1}{16} &amp;amp; 0 \frac {1}{8} &amp;amp; 0 \frac {1}{16} &#xA;\end{bmatrix},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\sum$&lt;/span&gt; is a sum across all elements, i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;S(0, 0) &#xA;&amp;amp;= 1 \frac {1}{16} + 0 \frac {1}{8} + 1 \frac {1}{16} + &#xA; 0 \frac {1}{8} + \color{green}{0} \frac {1}{4} + 1 \frac {1}{8} + &#xA;1 \frac {1}{16} + 0 \frac {1}{8} + 0 \frac {1}{16}  \\&#xA;&amp;amp; = \frac {1}{16} + \frac {1}{16} + \frac {1}{8} + \frac {1}{16} \\&#xA;&amp;amp; = \frac {3}{16} + \frac {1}{8} \\&#xA;&amp;amp;= \frac {5}{16} &#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now, to answer your questions/concerns more directly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are they the dimensions of the kernel along the height and width dimensions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. They are the indices that determine the neighbourhood around &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; where you want to compute the cross-correlation.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So, for the input image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and kernel &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;, we take the sum product of &lt;span class=&quot;math-container&quot;&gt;$I*K$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In this case, the symbol &lt;span class=&quot;math-container&quot;&gt;$*$&lt;/span&gt; does &lt;strong&gt;not&lt;/strong&gt; denote the product, but the cross-correlation. Maybe by &amp;quot;sum product&amp;quot; you meant the cross-correlation (or 2D dot product), but I'm not sure.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How is the input image &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; indexed?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The input image is indexed with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, but you start at &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, that's why you use the actual indices &lt;span class=&quot;math-container&quot;&gt;$i+m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j+n$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2444" LastEditorUserId="2444" LastEditDate="2021-01-23T13:36:24.733" LastActivityDate="2021-01-23T13:36:24.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25939" PostTypeId="1" CreationDate="2021-01-23T13:57:32.520" Score="0" ViewCount="49" Body="&lt;p&gt;I have a problem with applying alpha zero self-play to a game (Connect 6) with a huge branching factor (30,000 on average).&lt;/p&gt;&#xA;&lt;p&gt;I have implemented the MCTS as described but I found that during the MCTS simulations for the first move, because &lt;strong&gt;P(s, a)&lt;/strong&gt; is so much smaller than &lt;strong&gt;Q(s, a)&lt;/strong&gt;, the MCTS search tree is extremely narrow (in fact, it only had 1 branch per level, and later I have added dirichlet noise and it changed the tree to have 2 child branches instead of 1, per level).&lt;/p&gt;&#xA;&lt;p&gt;With some debugging I figured that after the first simulation, the last visited child would have a &lt;strong&gt;Q&lt;/strong&gt; value which is on average 0.5, and all the rest of the children would have 0 for their &lt;strong&gt;Q&lt;/strong&gt; values (because they haven't been explored) while their &lt;strong&gt;U&lt;/strong&gt; are way smaller, about 1/60000 on average. So comparing all the &lt;strong&gt;Q + U&lt;/strong&gt; on the 2nd and subsequent simulations would result in selecting the same node over and over again, building this narrow tree.&lt;/p&gt;&#xA;&lt;p&gt;To make matter worse, because the first simulation built a very narrow tree, and we don't clear the statistics on the subsequent simulations for the next move, 